2023-07-06T03:22:53.516 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-06T03:22:53.517 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895 --name rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi --verbose -- /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895/orig.config.yaml 2023-07-06T03:22:53.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-06T03:22:58.690 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895 branch: wip-rishabh-2023Jul5 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} email: null first_in_suite: false job_id: '7327895' kernel: flavor: default kdb: true sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-rishabh-2023Jul5 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - responding to mclientcaps\(revoke\) - not advance its oldest_client_tid - failing to advance its oldest client/flush tid - Too many inodes in cache - failing to respond to cache pressure - slow requests are blocked - failing to respond to capability release - MDS cache is too large - \(MDS_CLIENT_OLDEST_TID\) - \(MDS_CACHE_OVERSIZED\) sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul5 sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbLB4vrKtiUGEdyy5vFw5lZIMGJWLgfKu1FVCS9K8igDuk1xKRkTBDq+z0YIrYGDhUWIFXkqNli9BT6Jf9pD7c= smithi146.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDL+5Bv2zzEExDF42A9wR5B2XOKQwNmFgKCpXESaCkKVHsS7/uHuMqg/F279BCxmbt3UfjWgbqyr8R85CKHiAWLMGdbGdEdN1LUxOuDr/gOjjWR3uDEr2B0nmU/wOV3XY8LVRSAaxFdRWeNXRG03U7BMS/1QnhbWodZqDIA39ALk9Oe9tTWjG5uGOqOceku4+Y+0VMJliy9E0qxzLFhbww9vm3UHlKnkjgORQ653YzeJHSsgKOwL09ivX42nvXn3bjMtKMpH0j4Fe9bB/laS9ame+JDRy1M5uGNcYBHtuaiUGx7rnRClnq9WZ6yKdcqnmjyjhkrRb41BpkZKbO4hXCqT83ulMnB3f96TiSHkRoA55yB+TUQeVgf7F804u3gHnAsT97iZYt+YZRe8/E8cSZBL37bXzwC9+vmHlEmd/Z5od4xpsjOIoJwDzq22a9akFO1cwywWpKUKdD4B1BZYSs7dwj7pZDAt74Im3rKGGoqyrG5B+QBodBQbIeYtYIxvhc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_limits teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: b53f236c1077f1d295baafb9e0ca9a4b447019d7 timestamp: 2023-07-05_22:21:20 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-06T03:22:58.691 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa; will attempt to use it 2023-07-06T03:22:58.692 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks 2023-07-06T03:22:58.692 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-06T03:22:58.694 INFO:teuthology.task.internal:Checking packages... 2023-07-06T03:22:58.724 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '79365bb3b803780276d69efb2cc4afe5b160de9e' 2023-07-06T03:22:58.724 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-06T03:22:58.725 INFO:teuthology.packaging:ref: None 2023-07-06T03:22:58.725 INFO:teuthology.packaging:tag: None 2023-07-06T03:22:58.725 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul5 2023-07-06T03:22:58.725 INFO:teuthology.packaging:sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T03:22:58.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul5 2023-07-06T03:23:03.953 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4742.g79365bb3 2023-07-06T03:23:03.953 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-06T03:23:03.961 INFO:teuthology.task.internal:no buildpackages task found 2023-07-06T03:23:03.962 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-06T03:23:03.976 INFO:teuthology.task.internal:Saving configuration 2023-07-06T03:23:04.009 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-06T03:23:04.016 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T03:23:08.765 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 03:15:42.543866', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbLB4vrKtiUGEdyy5vFw5lZIMGJWLgfKu1FVCS9K8igDuk1xKRkTBDq+z0YIrYGDhUWIFXkqNli9BT6Jf9pD7c='} 2023-07-06T03:23:13.773 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 03:15:42.542009', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPh1AkseC1LrzyVF3CK9cEjoEfz+FMxSINBeBXuEK+aXqnuZgLgZx2EnLxv7C4emSq3iNU/RMXlLAIZsUa8J5X4='} 2023-07-06T03:23:13.773 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-06T03:23:13.783 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-07-06T03:23:13.784 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-07-06T03:23:13.784 INFO:teuthology.run_tasks:Running task console_log... 2023-07-06T03:23:13.875 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe9d8d64280>, signals=[15]) 2023-07-06T03:23:13.879 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-06T03:23:13.892 INFO:teuthology.task.internal:Opening connections... 2023-07-06T03:23:13.894 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:23:13.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T03:23:13.967 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:23:13.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T03:23:18.814 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-06T03:23:18.821 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2023-07-06T03:23:18.844 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2023-07-06T03:23:18.845 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2023-07-06T03:23:18.905 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2023-07-06T03:23:18.905 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8" 2023-07-06T03:23:18.906 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2023-07-06T03:23:18.906 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2023-07-06T03:23:18.906 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8" 2023-07-06T03:23:18.906 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8" 2023-07-06T03:23:18.906 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-06T03:23:18.906 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2023-07-06T03:23:18.906 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-06T03:23:18.906 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2023-07-06T03:23:18.907 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T03:23:18.907 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T03:23:18.907 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-06T03:23:18.908 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2023-07-06T03:23:21.771 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2023-07-06T03:23:21.810 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2023-07-06T03:23:21.811 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2023-07-06T03:23:21.872 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2023-07-06T03:23:21.873 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8" 2023-07-06T03:23:21.873 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2023-07-06T03:23:21.874 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2023-07-06T03:23:21.874 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8" 2023-07-06T03:23:21.874 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8" 2023-07-06T03:23:21.875 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-06T03:23:21.875 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2023-07-06T03:23:21.875 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-06T03:23:21.876 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2023-07-06T03:23:21.876 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T03:23:21.876 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T03:23:21.877 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-06T03:23:21.877 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2023-07-06T03:23:21.896 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-06T03:23:21.911 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-06T03:23:21.918 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-06T03:23:21.919 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2023-07-06T03:23:21.921 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2023-07-06T03:23:21.941 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-06T03:23:21.949 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-06T03:23:21.950 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2023-07-06T03:23:21.982 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2023-07-06T03:23:22.015 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-06T03:23:22.079 INFO:teuthology.run_tasks:Running task kernel... 2023-07-06T03:23:22.096 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'} 2023-07-06T03:23:22.097 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}}, timeout 300 2023-07-06T03:23:22.097 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T03:23:22.098 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T03:23:22.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:23:22.119 DEBUG:teuthology.task.kernel:sha1 for mon.a is 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T03:23:22.119 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "84139c18d56f518615369e8310ae4ecf07a36dcd"... 2023-07-06T03:23:22.120 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2023-07-06T03:23:22.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:23:22.121 DEBUG:teuthology.task.kernel:sha1 for mon.b is 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T03:23:22.121 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "84139c18d56f518615369e8310ae4ecf07a36dcd"... 2023-07-06T03:23:22.121 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2023-07-06T03:23:22.181 INFO:teuthology.orchestra.run.smithi097.stdout:4.18.0-489.el8.x86_64 2023-07-06T03:23:22.181 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T03:23:22.182 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-06T03:23:22.182 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T03:23:22.186 INFO:teuthology.orchestra.run.smithi146.stdout:4.18.0-489.el8.x86_64 2023-07-06T03:23:22.186 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T03:23:22.187 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-06T03:23:22.187 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T03:23:22.331 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum upgrade -y linux-firmware 2023-07-06T03:23:22.408 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum upgrade -y linux-firmware 2023-07-06T03:23:26.238 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 8 - AppStream 24 MB/s | 31 MB 00:01 2023-07-06T03:23:26.300 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 8 - AppStream 22 MB/s | 31 MB 00:01 2023-07-06T03:23:37.834 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 8 - BaseOS 6.8 MB/s | 41 MB 00:06 2023-07-06T03:23:38.004 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 8 - BaseOS 6.6 MB/s | 41 MB 00:06 2023-07-06T03:23:43.933 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 8 - Extras 92 kB/s | 18 kB 00:00 2023-07-06T03:23:44.105 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 8 - Extras 120 kB/s | 18 kB 00:00 2023-07-06T03:23:44.483 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.6 kB 00:00 2023-07-06T03:23:44.646 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.6 kB 00:00 2023-07-06T03:23:47.252 INFO:teuthology.orchestra.run.smithi097.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T03:23:47.372 INFO:teuthology.orchestra.run.smithi146.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T03:23:59.204 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 16 MB 00:11 2023-07-06T03:24:02.968 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 310 kB/s | 24 kB 00:00 2023-07-06T03:24:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-07-06T03:24:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-07-06T03:24:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2023-07-06T03:24:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-07-06T03:24:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2023-07-06T03:24:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: linux-firmware noarch 20230515-115.gitd1962891.el8 baseos 266 M 2023-07-06T03:24:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:24:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-07-06T03:24:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-07-06T03:24:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 1 Package 2023-07-06T03:24:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:24:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 266 M 2023-07-06T03:24:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2023-07-06T03:24:14.225 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 598 kB/s | 16 MB 00:26 2023-07-06T03:24:17.727 INFO:teuthology.orchestra.run.smithi097.stdout:linux-firmware-20230515-115.gitd1962891.el8.noa 24 MB/s | 266 MB 00:11 2023-07-06T03:24:17.727 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2023-07-06T03:24:17.728 INFO:teuthology.orchestra.run.smithi097.stdout:Total 24 MB/s | 266 MB 00:11 2023-07-06T03:24:17.913 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 430 kB/s | 24 kB 00:00 2023-07-06T03:24:19.306 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2023-07-06T03:24:19.392 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2023-07-06T03:24:19.392 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2023-07-06T03:24:21.057 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2023-07-06T03:24:21.096 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2023-07-06T03:24:21.371 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2023-07-06T03:24:21.371 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2023-07-06T03:24:21.372 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repo Size 2023-07-06T03:24:21.372 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2023-07-06T03:24:21.372 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2023-07-06T03:24:21.372 INFO:teuthology.orchestra.run.smithi146.stdout: linux-firmware noarch 20230515-115.gitd1962891.el8 baseos 266 M 2023-07-06T03:24:21.372 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:24:21.372 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2023-07-06T03:24:21.373 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2023-07-06T03:24:21.373 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 1 Package 2023-07-06T03:24:21.374 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:24:21.375 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 266 M 2023-07-06T03:24:21.375 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2023-07-06T03:24:31.247 INFO:teuthology.orchestra.run.smithi146.stdout:linux-firmware-20230515-115.gitd1962891.el8.noa 27 MB/s | 266 MB 00:09 2023-07-06T03:24:31.247 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2023-07-06T03:24:31.248 INFO:teuthology.orchestra.run.smithi146.stdout:Total 27 MB/s | 266 MB 00:09 2023-07-06T03:24:32.826 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2023-07-06T03:24:32.893 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2023-07-06T03:24:32.893 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2023-07-06T03:24:34.555 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2023-07-06T03:24:34.573 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2023-07-06T03:24:38.677 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2023-07-06T03:24:39.099 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-06T03:24:39.360 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T03:24:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T03:24:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-06T03:24:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T03:24:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:24:40.267 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2023-07-06T03:24:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: linux-firmware-20230515-115.gitd1962891.el8.noarch 2023-07-06T03:24:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:24:40.267 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-07-06T03:24:40.444 INFO:teuthology.task.kernel:Downloading kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on mon.a... 2023-07-06T03:24:40.444 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T03:24:40.590 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ 2023-07-06T03:24:40.590 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.4.0_g84139c18d56f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-06T03:24:40.593 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi097.front.sepia.ceph.com to complete... 2023-07-06T03:24:52.221 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2023-07-06T03:24:52.601 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-06T03:24:52.871 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T03:24:53.095 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T03:24:53.096 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : linux-firmware-20230515-115.gitd1962891.el8.noarch 1/2 2023-07-06T03:24:53.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06 03:24:53 URL:https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/kernel-6.4.0_g84139c18d56f-1.x86_64.rpm [379858600/379858600] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-06T03:24:53.425 INFO:teuthology.orchestra.run.smithi097.stderr:FINISHED --2023-07-06 03:24:53-- 2023-07-06T03:24:53.425 INFO:teuthology.orchestra.run.smithi097.stderr:Total wall clock time: 13s 2023-07-06T03:24:53.425 INFO:teuthology.orchestra.run.smithi097.stderr:Downloaded: 1 files, 362M in 13s (28.9 MB/s) 2023-07-06T03:24:53.427 INFO:teuthology.task.kernel:Installing kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on mon.a... 2023-07-06T03:24:53.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-06T03:24:53.790 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-07-06T03:24:53.791 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:24:53.791 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2023-07-06T03:24:53.791 INFO:teuthology.orchestra.run.smithi146.stdout: linux-firmware-20230515-115.gitd1962891.el8.noarch 2023-07-06T03:24:53.791 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:24:53.791 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2023-07-06T03:24:54.032 INFO:teuthology.task.kernel:Downloading kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on mon.b... 2023-07-06T03:24:54.032 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T03:24:54.252 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ 2023-07-06T03:24:54.253 DEBUG:teuthology.orchestra.run.smithi146:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.4.0_g84139c18d56f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-06T03:24:54.256 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi146.front.sepia.ceph.com to complete... 2023-07-06T03:24:55.103 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying... ######################################## 2023-07-06T03:24:55.146 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing... ######################################## 2023-07-06T03:24:55.256 INFO:teuthology.orchestra.run.smithi097.stdout:Updating / installing... 2023-07-06T03:25:07.082 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06 03:25:07 URL:https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/kernel-6.4.0_g84139c18d56f-1.x86_64.rpm [379858600/379858600] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-06T03:25:07.083 INFO:teuthology.orchestra.run.smithi146.stderr:FINISHED --2023-07-06 03:25:07-- 2023-07-06T03:25:07.083 INFO:teuthology.orchestra.run.smithi146.stderr:Total wall clock time: 13s 2023-07-06T03:25:07.083 INFO:teuthology.orchestra.run.smithi146.stderr:Downloaded: 1 files, 362M in 13s (28.9 MB/s) 2023-07-06T03:25:07.084 INFO:teuthology.task.kernel:Installing kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on mon.b... 2023-07-06T03:25:07.084 DEBUG:teuthology.orchestra.run.smithi146:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-06T03:25:08.773 INFO:teuthology.orchestra.run.smithi146.stdout:Verifying... ######################################## 2023-07-06T03:25:08.817 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing... ######################################## 2023-07-06T03:25:08.926 INFO:teuthology.orchestra.run.smithi146.stdout:Updating / installing... 2023-07-06T03:25:33.065 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-6.4.0_g84139c18d56f-1 ######################################## 2023-07-06T03:25:46.739 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-6.4.0_g84139c18d56f-1 ######################################## 2023-07-06T03:25:53.810 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-06T03:25:53.811 DEBUG:teuthology.orchestra.run.smithi097:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-06T03:25:53.901 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/System.map-6.4.0-g84139c18d56f 2023-07-06T03:25:53.901 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/config-6.4.0-g84139c18d56f 2023-07-06T03:25:53.901 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/vmlinuz-6.4.0-g84139c18d56f 2023-07-06T03:25:53.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f 2023-07-06T03:25:53.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel 2023-07-06T03:25:53.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch 2023-07-06T03:25:53.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86 2023-07-06T03:25:53.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto 2023-07-06T03:25:53.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-06T03:25:53.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-06T03:25:53.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-06T03:25:53.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-06T03:25:53.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-06T03:25:53.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-06T03:25:53.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-06T03:25:53.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-06T03:25:53.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-06T03:25:53.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-06T03:25:53.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-06T03:25:53.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-06T03:25:53.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-06T03:25:53.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-06T03:25:53.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-06T03:25:53.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-06T03:25:53.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-06T03:25:53.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-06T03:25:53.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-06T03:25:53.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-06T03:25:53.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel 2023-07-06T03:25:53.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu 2023-07-06T03:25:53.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce 2023-07-06T03:25:53.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-06T03:25:53.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm 2023-07-06T03:25:53.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-06T03:25:53.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-06T03:25:53.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm.ko 2023-07-06T03:25:53.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto 2023-07-06T03:25:53.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ansi_cprng.ko 2023-07-06T03:25:53.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/anubis.ko 2023-07-06T03:25:53.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/arc4.ko 2023-07-06T03:25:53.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx 2023-07-06T03:25:53.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_memcpy.ko 2023-07-06T03:25:53.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_pq.ko 2023-07-06T03:25:53.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-06T03:25:53.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_tx.ko 2023-07-06T03:25:53.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_xor.ko 2023-07-06T03:25:53.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/raid6test.ko 2023-07-06T03:25:53.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authenc.ko 2023-07-06T03:25:53.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authencesn.ko 2023-07-06T03:25:53.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blake2b_generic.ko 2023-07-06T03:25:53.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_common.ko 2023-07-06T03:25:53.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_generic.ko 2023-07-06T03:25:53.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/camellia_generic.ko 2023-07-06T03:25:53.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast5_generic.ko 2023-07-06T03:25:53.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast6_generic.ko 2023-07-06T03:25:53.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast_common.ko 2023-07-06T03:25:53.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ccm.ko 2023-07-06T03:25:53.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cmac.ko 2023-07-06T03:25:53.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crc32_generic.ko 2023-07-06T03:25:53.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_engine.ko 2023-07-06T03:25:53.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_user.ko 2023-07-06T03:25:53.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cts.ko 2023-07-06T03:25:53.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/des_generic.ko 2023-07-06T03:25:53.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/dh_generic.ko 2023-07-06T03:25:53.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/echainiv.ko 2023-07-06T03:25:53.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/essiv.ko 2023-07-06T03:25:53.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/fcrypt.ko 2023-07-06T03:25:53.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/khazad.ko 2023-07-06T03:25:53.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4.ko 2023-07-06T03:25:53.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4hc.ko 2023-07-06T03:25:53.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/md4.ko 2023-07-06T03:25:53.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/michael_mic.ko 2023-07-06T03:25:53.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcbc.ko 2023-07-06T03:25:53.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcrypt.ko 2023-07-06T03:25:53.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/rmd160.ko 2023-07-06T03:25:53.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/seed.ko 2023-07-06T03:25:53.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/serpent_generic.ko 2023-07-06T03:25:53.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tcrypt.ko 2023-07-06T03:25:53.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tea.ko 2023-07-06T03:25:53.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_common.ko 2023-07-06T03:25:53.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_generic.ko 2023-07-06T03:25:53.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/vmac.ko 2023-07-06T03:25:53.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/wp512.ko 2023-07-06T03:25:53.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xcbc.ko 2023-07-06T03:25:53.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xor.ko 2023-07-06T03:25:53.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xxhash_generic.ko 2023-07-06T03:25:53.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers 2023-07-06T03:25:53.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi 2023-07-06T03:25:53.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-06T03:25:53.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_pad.ko 2023-07-06T03:25:53.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/custom_method.ko 2023-07-06T03:25:53.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/ec_sys.ko 2023-07-06T03:25:53.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/platform_profile.ko 2023-07-06T03:25:53.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbs.ko 2023-07-06T03:25:53.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbshc.ko 2023-07-06T03:25:53.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/video.ko 2023-07-06T03:25:53.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata 2023-07-06T03:25:53.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/acard-ahci.ko 2023-07-06T03:25:53.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ahci_platform.ko 2023-07-06T03:25:53.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ata_generic.ko 2023-07-06T03:25:53.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/libahci_platform.ko 2023-07-06T03:25:53.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_acpi.ko 2023-07-06T03:25:53.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ali.ko 2023-07-06T03:25:53.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_amd.ko 2023-07-06T03:25:53.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_artop.ko 2023-07-06T03:25:53.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atiixp.ko 2023-07-06T03:25:53.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atp867x.ko 2023-07-06T03:25:53.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd640.ko 2023-07-06T03:25:53.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd64x.ko 2023-07-06T03:25:53.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cypress.ko 2023-07-06T03:25:53.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_efar.ko 2023-07-06T03:25:53.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt366.ko 2023-07-06T03:25:53.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt37x.ko 2023-07-06T03:25:53.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-06T03:25:53.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-06T03:25:53.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it8213.ko 2023-07-06T03:25:53.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it821x.ko 2023-07-06T03:25:53.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_jmicron.ko 2023-07-06T03:25:53.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_marvell.ko 2023-07-06T03:25:53.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_mpiix.ko 2023-07-06T03:25:53.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_netcell.ko 2023-07-06T03:25:53.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ninja32.ko 2023-07-06T03:25:53.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87410.ko 2023-07-06T03:25:53.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87415.ko 2023-07-06T03:25:53.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_oldpiix.ko 2023-07-06T03:25:53.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_opti.ko 2023-07-06T03:25:53.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_optidma.ko 2023-07-06T03:25:53.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pcmcia.ko 2023-07-06T03:25:53.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-06T03:25:53.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-06T03:25:53.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_piccolo.ko 2023-07-06T03:25:53.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_rdc.ko 2023-07-06T03:25:53.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sch.ko 2023-07-06T03:25:53.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_serverworks.ko 2023-07-06T03:25:53.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sil680.ko 2023-07-06T03:25:53.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sis.ko 2023-07-06T03:25:53.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sl82c105.ko 2023-07-06T03:25:53.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_triflex.ko 2023-07-06T03:25:53.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_via.ko 2023-07-06T03:25:53.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pdc_adma.ko 2023-07-06T03:25:53.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_inic162x.ko 2023-07-06T03:25:53.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_mv.ko 2023-07-06T03:25:53.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_nv.ko 2023-07-06T03:25:53.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_promise.ko 2023-07-06T03:25:53.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_qstor.ko 2023-07-06T03:25:53.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil.ko 2023-07-06T03:25:53.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil24.ko 2023-07-06T03:25:53.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sis.ko 2023-07-06T03:25:53.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_svw.ko 2023-07-06T03:25:53.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sx4.ko 2023-07-06T03:25:53.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_uli.ko 2023-07-06T03:25:53.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_via.ko 2023-07-06T03:25:53.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_vsc.ko 2023-07-06T03:25:53.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay 2023-07-06T03:25:53.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay/ks0108.ko 2023-07-06T03:25:53.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base 2023-07-06T03:25:53.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap 2023-07-06T03:25:53.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-06T03:25:53.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma 2023-07-06T03:25:53.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma/bcma.ko 2023-07-06T03:25:53.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block 2023-07-06T03:25:53.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe 2023-07-06T03:25:53.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe/aoe.ko 2023-07-06T03:25:53.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/brd.ko 2023-07-06T03:25:53.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd 2023-07-06T03:25:53.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd/drbd.ko 2023-07-06T03:25:53.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/floppy.ko 2023-07-06T03:25:53.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/loop.ko 2023-07-06T03:25:53.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx 2023-07-06T03:25:53.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-06T03:25:53.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/nbd.ko 2023-07-06T03:25:53.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk 2023-07-06T03:25:53.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk/null_blk.ko 2023-07-06T03:25:53.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/pktcdvd.ko 2023-07-06T03:25:53.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/rbd.ko 2023-07-06T03:25:53.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/virtio_blk.ko 2023-07-06T03:25:53.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback 2023-07-06T03:25:53.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-06T03:25:53.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkfront.ko 2023-07-06T03:25:53.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram 2023-07-06T03:25:53.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram/zram.ko 2023-07-06T03:25:53.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char 2023-07-06T03:25:53.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hangcheck-timer.ko 2023-07-06T03:25:53.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random 2023-07-06T03:25:53.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-06T03:25:53.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-06T03:25:53.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-06T03:25:53.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/via-rng.ko 2023-07-06T03:25:53.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-06T03:25:53.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi 2023-07-06T03:25:53.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-06T03:25:53.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-06T03:25:53.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-06T03:25:53.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-06T03:25:53.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-06T03:25:53.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-06T03:25:53.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/lp.ko 2023-07-06T03:25:53.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave 2023-07-06T03:25:53.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave/mwave.ko 2023-07-06T03:25:53.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ppdev.ko 2023-07-06T03:25:53.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tlclk.ko 2023-07-06T03:25:53.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm 2023-07-06T03:25:53.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm.ko 2023-07-06T03:25:53.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-06T03:25:53.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-06T03:25:53.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-06T03:25:53.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-06T03:25:53.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-06T03:25:53.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/uv_mmtimer.ko 2023-07-06T03:25:53.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/virtio_console.ko 2023-07-06T03:25:53.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq 2023-07-06T03:25:53.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-06T03:25:53.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-06T03:25:53.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-06T03:25:53.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-06T03:25:53.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-06T03:25:53.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-06T03:25:53.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto 2023-07-06T03:25:53.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp 2023-07-06T03:25:53.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-06T03:25:53.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp.ko 2023-07-06T03:25:53.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel 2023-07-06T03:25:53.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat 2023-07-06T03:25:53.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common 2023-07-06T03:25:53.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-06T03:25:53.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-06T03:25:53.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-06T03:25:53.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-aes.ko 2023-07-06T03:25:53.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-sha.ko 2023-07-06T03:25:53.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio 2023-07-06T03:25:53.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-06T03:25:53.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca 2023-07-06T03:25:53.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca/dca.ko 2023-07-06T03:25:53.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq 2023-07-06T03:25:53.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-06T03:25:53.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma 2023-07-06T03:25:53.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw 2023-07-06T03:25:53.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-06T03:25:53.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat 2023-07-06T03:25:53.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-06T03:25:53.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac 2023-07-06T03:25:53.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/amd64_edac.ko 2023-07-06T03:25:53.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/e752x_edac.ko 2023-07-06T03:25:53.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/edac_mce_amd.ko 2023-07-06T03:25:53.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3000_edac.ko 2023-07-06T03:25:53.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3200_edac.ko 2023-07-06T03:25:53.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5100_edac.ko 2023-07-06T03:25:53.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5400_edac.ko 2023-07-06T03:25:53.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7300_edac.ko 2023-07-06T03:25:53.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7core_edac.ko 2023-07-06T03:25:53.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i82975x_edac.ko 2023-07-06T03:25:53.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/ie31200_edac.ko 2023-07-06T03:25:53.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/sb_edac.ko 2023-07-06T03:25:53.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/x38_edac.ko 2023-07-06T03:25:53.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware 2023-07-06T03:25:53.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/edd.ko 2023-07-06T03:25:53.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-06T03:25:53.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio 2023-07-06T03:25:53.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-ich.ko 2023-07-06T03:25:53.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-06T03:25:53.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid 2023-07-06T03:25:53.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-appleir.ko 2023-07-06T03:25:53.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-aureal.ko 2023-07-06T03:25:53.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-axff.ko 2023-07-06T03:25:53.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-dr.ko 2023-07-06T03:25:53.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elecom.ko 2023-07-06T03:25:53.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elo.ko 2023-07-06T03:25:53.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-emsff.ko 2023-07-06T03:25:53.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gaff.ko 2023-07-06T03:25:53.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gt683r.ko 2023-07-06T03:25:53.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gyration.ko 2023-07-06T03:25:53.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-06T03:25:53.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-06T03:25:53.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtekff.ko 2023-07-06T03:25:53.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-hyperv.ko 2023-07-06T03:25:53.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-icade.ko 2023-07-06T03:25:53.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-keytouch.ko 2023-07-06T03:25:53.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-kye.ko 2023-07-06T03:25:53.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lcpower.ko 2023-07-06T03:25:53.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-led.ko 2023-07-06T03:25:53.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lenovo.ko 2023-07-06T03:25:53.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-06T03:25:53.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-06T03:25:53.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-multitouch.ko 2023-07-06T03:25:53.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-ortek.ko 2023-07-06T03:25:53.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-penmount.ko 2023-07-06T03:25:53.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-petalynx.ko 2023-07-06T03:25:53.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-picolcd.ko 2023-07-06T03:25:53.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-pl.ko 2023-07-06T03:25:53.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-plantronics.ko 2023-07-06T03:25:53.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-primax.ko 2023-07-06T03:25:53.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-rmi.ko 2023-07-06T03:25:53.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-06T03:25:53.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-common.ko 2023-07-06T03:25:53.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-06T03:25:53.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-06T03:25:53.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-06T03:25:53.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-06T03:25:53.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-06T03:25:53.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-06T03:25:53.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-06T03:25:53.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-06T03:25:53.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-06T03:25:53.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat.ko 2023-07-06T03:25:53.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-saitek.ko 2023-07-06T03:25:53.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-samsung.ko 2023-07-06T03:25:53.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-06T03:25:53.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sjoy.ko 2023-07-06T03:25:53.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sony.ko 2023-07-06T03:25:53.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-speedlink.ko 2023-07-06T03:25:53.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-steelseries.ko 2023-07-06T03:25:53.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sunplus.ko 2023-07-06T03:25:53.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-06T03:25:53.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tivo.ko 2023-07-06T03:25:53.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tmff.ko 2023-07-06T03:25:53.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-topseed.ko 2023-07-06T03:25:53.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-twinhan.ko 2023-07-06T03:25:53.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-uclogic.ko 2023-07-06T03:25:53.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-waltop.ko 2023-07-06T03:25:53.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-wiimote.ko 2023-07-06T03:25:53.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-xinmo.ko 2023-07-06T03:25:53.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zpff.ko 2023-07-06T03:25:53.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zydacron.ko 2023-07-06T03:25:53.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/uhid.ko 2023-07-06T03:25:53.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/wacom.ko 2023-07-06T03:25:53.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv 2023-07-06T03:25:53.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_balloon.ko 2023-07-06T03:25:53.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_utils.ko 2023-07-06T03:25:53.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_vmbus.ko 2023-07-06T03:25:53.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon 2023-07-06T03:25:53.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru.ko 2023-07-06T03:25:53.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru3.ko 2023-07-06T03:25:53.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-06T03:25:53.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7414.ko 2023-07-06T03:25:53.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7418.ko 2023-07-06T03:25:53.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adc128d818.ko 2023-07-06T03:25:53.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1025.ko 2023-07-06T03:25:53.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1026.ko 2023-07-06T03:25:53.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1029.ko 2023-07-06T03:25:53.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1031.ko 2023-07-06T03:25:53.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm9240.ko 2023-07-06T03:25:53.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ads7828.ko 2023-07-06T03:25:53.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7410.ko 2023-07-06T03:25:53.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7411.ko 2023-07-06T03:25:53.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7462.ko 2023-07-06T03:25:53.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7470.ko 2023-07-06T03:25:53.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7475.ko 2023-07-06T03:25:53.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7x10.ko 2023-07-06T03:25:53.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/amc6821.ko 2023-07-06T03:25:53.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/applesmc.ko 2023-07-06T03:25:53.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asb100.ko 2023-07-06T03:25:53.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asc7621.ko 2023-07-06T03:25:53.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-06T03:25:53.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/atxp1.ko 2023-07-06T03:25:53.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/coretemp.ko 2023-07-06T03:25:53.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-06T03:25:53.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dme1737.ko 2023-07-06T03:25:53.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds1621.ko 2023-07-06T03:25:53.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds620.ko 2023-07-06T03:25:53.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc1403.ko 2023-07-06T03:25:53.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc6w201.ko 2023-07-06T03:25:53.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71805f.ko 2023-07-06T03:25:53.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71882fg.ko 2023-07-06T03:25:53.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f75375s.ko 2023-07-06T03:25:53.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fam15h_power.ko 2023-07-06T03:25:53.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fschmd.ko 2023-07-06T03:25:53.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g760a.ko 2023-07-06T03:25:53.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g762.ko 2023-07-06T03:25:53.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl518sm.ko 2023-07-06T03:25:53.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl520sm.ko 2023-07-06T03:25:53.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-06T03:25:53.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5500_temp.ko 2023-07-06T03:25:53.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5k_amb.ko 2023-07-06T03:25:53.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmaem.ko 2023-07-06T03:25:53.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmpex.ko 2023-07-06T03:25:53.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina209.ko 2023-07-06T03:25:53.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina2xx.ko 2023-07-06T03:25:53.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/it87.ko 2023-07-06T03:25:53.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k10temp.ko 2023-07-06T03:25:53.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k8temp.ko 2023-07-06T03:25:53.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lineage-pem.ko 2023-07-06T03:25:53.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm63.ko 2023-07-06T03:25:53.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm73.ko 2023-07-06T03:25:53.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm75.ko 2023-07-06T03:25:53.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm77.ko 2023-07-06T03:25:53.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm78.ko 2023-07-06T03:25:53.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm80.ko 2023-07-06T03:25:53.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm83.ko 2023-07-06T03:25:53.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm85.ko 2023-07-06T03:25:53.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm87.ko 2023-07-06T03:25:53.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm90.ko 2023-07-06T03:25:53.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm92.ko 2023-07-06T03:25:53.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm93.ko 2023-07-06T03:25:53.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95234.ko 2023-07-06T03:25:53.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95241.ko 2023-07-06T03:25:53.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95245.ko 2023-07-06T03:25:53.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc2945.ko 2023-07-06T03:25:53.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4151.ko 2023-07-06T03:25:53.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4215.ko 2023-07-06T03:25:53.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4222.ko 2023-07-06T03:25:53.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4245.ko 2023-07-06T03:25:53.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4260.ko 2023-07-06T03:25:53.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4261.ko 2023-07-06T03:25:53.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max16065.ko 2023-07-06T03:25:53.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1619.ko 2023-07-06T03:25:53.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1668.ko 2023-07-06T03:25:53.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max197.ko 2023-07-06T03:25:53.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6639.ko 2023-07-06T03:25:53.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6650.ko 2023-07-06T03:25:53.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6697.ko 2023-07-06T03:25:53.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/mcp3021.ko 2023-07-06T03:25:53.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6683.ko 2023-07-06T03:25:53.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775-core.ko 2023-07-06T03:25:53.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775.ko 2023-07-06T03:25:53.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct7802.ko 2023-07-06T03:25:53.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87360.ko 2023-07-06T03:25:53.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87427.ko 2023-07-06T03:25:53.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pcf8591.ko 2023-07-06T03:25:53.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus 2023-07-06T03:25:53.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-06T03:25:53.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-06T03:25:53.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-06T03:25:53.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-06T03:25:53.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-06T03:25:53.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-06T03:25:53.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-06T03:25:53.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-06T03:25:53.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-06T03:25:53.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-06T03:25:53.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-06T03:25:53.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-06T03:25:53.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/powr1220.ko 2023-07-06T03:25:53.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5627.ko 2023-07-06T03:25:53.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5636.ko 2023-07-06T03:25:53.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-06T03:25:53.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht15.ko 2023-07-06T03:25:53.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht21.ko 2023-07-06T03:25:53.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/shtc1.ko 2023-07-06T03:25:53.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sis5595.ko 2023-07-06T03:25:53.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47b397.ko 2023-07-06T03:25:53.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m1.ko 2023-07-06T03:25:53.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m192.ko 2023-07-06T03:25:53.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/thmc50.ko 2023-07-06T03:25:53.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp102.ko 2023-07-06T03:25:53.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp103.ko 2023-07-06T03:25:53.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp401.ko 2023-07-06T03:25:53.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp421.ko 2023-07-06T03:25:53.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via-cputemp.ko 2023-07-06T03:25:53.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via686a.ko 2023-07-06T03:25:53.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt1211.ko 2023-07-06T03:25:53.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt8231.ko 2023-07-06T03:25:53.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627ehf.ko 2023-07-06T03:25:53.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627hf.ko 2023-07-06T03:25:53.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83781d.ko 2023-07-06T03:25:53.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83791d.ko 2023-07-06T03:25:53.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83792d.ko 2023-07-06T03:25:53.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83793.ko 2023-07-06T03:25:53.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83795.ko 2023-07-06T03:25:53.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l785ts.ko 2023-07-06T03:25:53.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l786ng.ko 2023-07-06T03:25:53.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c 2023-07-06T03:25:53.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos 2023-07-06T03:25:53.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-06T03:25:53.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-06T03:25:53.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses 2023-07-06T03:25:53.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-06T03:25:53.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-06T03:25:53.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-06T03:25:53.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-06T03:25:53.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-06T03:25:53.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-06T03:25:53.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-06T03:25:53.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-06T03:25:53.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-06T03:25:53.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-06T03:25:53.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-dev.ko 2023-07-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-mux.ko 2023-07-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-smbus.ko 2023-07-06T03:25:53.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-stub.ko 2023-07-06T03:25:53.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband 2023-07-06T03:25:53.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core 2023-07-06T03:25:53.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-06T03:25:53.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_core.ko 2023-07-06T03:25:53.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-06T03:25:53.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw 2023-07-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4 2023-07-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4 2023-07-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-06T03:25:53.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5 2023-07-06T03:25:53.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-06T03:25:53.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca 2023-07-06T03:25:53.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-06T03:25:53.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp 2023-07-06T03:25:53.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib 2023-07-06T03:25:53.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-06T03:25:53.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser 2023-07-06T03:25:53.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-06T03:25:53.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert 2023-07-06T03:25:53.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-06T03:25:53.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp 2023-07-06T03:25:53.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-06T03:25:53.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt 2023-07-06T03:25:53.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-06T03:25:53.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input 2023-07-06T03:25:53.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport 2023-07-06T03:25:53.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-06T03:25:53.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-06T03:25:53.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/gameport.ko 2023-07-06T03:25:53.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/lightning.ko 2023-07-06T03:25:53.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/ns558.ko 2023-07-06T03:25:53.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/joydev.ko 2023-07-06T03:25:53.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard 2023-07-06T03:25:53.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-06T03:25:53.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse 2023-07-06T03:25:53.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/appletouch.ko 2023-07-06T03:25:53.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/bcm5974.ko 2023-07-06T03:25:53.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/cyapatp.ko 2023-07-06T03:25:53.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-06T03:25:53.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/sermouse.ko 2023-07-06T03:25:53.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-06T03:25:53.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-06T03:25:53.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-06T03:25:53.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4 2023-07-06T03:25:53.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-06T03:25:53.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio 2023-07-06T03:25:53.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/altera_ps2.ko 2023-07-06T03:25:53.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/arc_ps2.ko 2023-07-06T03:25:53.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-06T03:25:53.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/serio_raw.ko 2023-07-06T03:25:53.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/sparse-keymap.ko 2023-07-06T03:25:53.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu 2023-07-06T03:25:53.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd 2023-07-06T03:25:53.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-06T03:25:53.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds 2023-07-06T03:25:53.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-blinkm.ko 2023-07-06T03:25:53.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lm3530.ko 2023-07-06T03:25:53.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lp3944.ko 2023-07-06T03:25:53.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-ss4200.ko 2023-07-06T03:25:53.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger 2023-07-06T03:25:53.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-06T03:25:53.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-06T03:25:53.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-06T03:25:53.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-06T03:25:53.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-06T03:25:53.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-06T03:25:53.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-06T03:25:53.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-06T03:25:53.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md 2023-07-06T03:25:53.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache 2023-07-06T03:25:53.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache/bcache.ko 2023-07-06T03:25:53.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-bio-prison.ko 2023-07-06T03:25:53.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache-smq.ko 2023-07-06T03:25:53.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache.ko 2023-07-06T03:25:53.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-crypt.ko 2023-07-06T03:25:53.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-delay.ko 2023-07-06T03:25:53.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-flakey.ko 2023-07-06T03:25:53.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-log-userspace.ko 2023-07-06T03:25:53.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-multipath.ko 2023-07-06T03:25:53.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-queue-length.ko 2023-07-06T03:25:53.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-raid.ko 2023-07-06T03:25:53.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-round-robin.ko 2023-07-06T03:25:53.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-service-time.ko 2023-07-06T03:25:53.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-switch.ko 2023-07-06T03:25:53.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-thin-pool.ko 2023-07-06T03:25:53.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-verity.ko 2023-07-06T03:25:53.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/faulty.ko 2023-07-06T03:25:53.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/linear.ko 2023-07-06T03:25:53.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/multipath.ko 2023-07-06T03:25:53.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data 2023-07-06T03:25:53.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-06T03:25:53.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid0.ko 2023-07-06T03:25:53.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid1.ko 2023-07-06T03:25:53.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid10.ko 2023-07-06T03:25:53.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid456.ko 2023-07-06T03:25:53.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message 2023-07-06T03:25:53.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion 2023-07-06T03:25:53.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptbase.ko 2023-07-06T03:25:53.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptctl.ko 2023-07-06T03:25:53.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptfc.ko 2023-07-06T03:25:53.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptlan.ko 2023-07-06T03:25:53.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptsas.ko 2023-07-06T03:25:53.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptscsih.ko 2023-07-06T03:25:53.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptspi.ko 2023-07-06T03:25:53.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd 2023-07-06T03:25:53.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_ich.ko 2023-07-06T03:25:53.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_sch.ko 2023-07-06T03:25:53.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/mfd-core.ko 2023-07-06T03:25:53.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/sm501.ko 2023-07-06T03:25:53.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/viperboard.ko 2023-07-06T03:25:53.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/vx855.ko 2023-07-06T03:25:53.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/wl1273-core.ko 2023-07-06T03:25:53.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc 2023-07-06T03:25:53.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl 2023-07-06T03:25:53.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-06T03:25:53.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds9802als.ko 2023-07-06T03:25:53.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds990x.ko 2023-07-06T03:25:53.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/bh1770glc.ko 2023-07-06T03:25:53.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710 2023-07-06T03:25:53.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710/cb710.ko 2023-07-06T03:25:53.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom 2023-07-06T03:25:53.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/at24.ko 2023-07-06T03:25:53.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-06T03:25:53.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-06T03:25:53.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/max6875.ko 2023-07-06T03:25:53.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/enclosure.ko 2023-07-06T03:25:53.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/hpilo.ko 2023-07-06T03:25:53.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29003.ko 2023-07-06T03:25:53.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29020.ko 2023-07-06T03:25:53.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d 2023-07-06T03:25:53.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-06T03:25:53.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-06T03:25:53.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei 2023-07-06T03:25:53.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-me.ko 2023-07-06T03:25:53.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-txe.ko 2023-07-06T03:25:53.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei.ko 2023-07-06T03:25:53.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru 2023-07-06T03:25:53.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-06T03:25:53.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp 2023-07-06T03:25:53.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-06T03:25:53.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-06T03:25:53.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-06T03:25:53.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_7xx1.ko 2023-07-06T03:25:53.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_core.ko 2023-07-06T03:25:53.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tsl2550.ko 2023-07-06T03:25:53.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_balloon.ko 2023-07-06T03:25:53.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci 2023-07-06T03:25:53.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-06T03:25:53.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net 2023-07-06T03:25:53.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding 2023-07-06T03:25:53.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding/bonding.ko 2023-07-06T03:25:53.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa 2023-07-06T03:25:53.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53 2023-07-06T03:25:53.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-06T03:25:53.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-06T03:25:53.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-06T03:25:53.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx 2023-07-06T03:25:53.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-06T03:25:53.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dummy.ko 2023-07-06T03:25:53.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/eql.ko 2023-07-06T03:25:53.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet 2023-07-06T03:25:53.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com 2023-07-06T03:25:53.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-06T03:25:53.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-06T03:25:53.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-06T03:25:53.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-06T03:25:53.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390 2023-07-06T03:25:53.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-06T03:25:53.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-06T03:25:53.989 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-06T03:25:53.989 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-06T03:25:53.989 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec 2023-07-06T03:25:53.989 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-06T03:25:53.989 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere 2023-07-06T03:25:53.989 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-06T03:25:53.989 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon 2023-07-06T03:25:53.989 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-06T03:25:53.990 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera 2023-07-06T03:25:53.990 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-06T03:25:53.990 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd 2023-07-06T03:25:53.990 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-06T03:25:53.990 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-06T03:25:53.990 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-06T03:25:53.990 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros 2023-07-06T03:25:53.990 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx 2023-07-06T03:25:53.991 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-06T03:25:53.991 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-06T03:25:53.991 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-06T03:25:53.991 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-06T03:25:53.991 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-06T03:25:53.991 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx 2023-07-06T03:25:53.991 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-06T03:25:53.992 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-06T03:25:53.992 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom 2023-07-06T03:25:53.992 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-06T03:25:53.992 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-06T03:25:53.992 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-06T03:25:53.992 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-06T03:25:53.992 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-06T03:25:53.992 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet 2023-07-06T03:25:53.993 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-06T03:25:53.993 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-06T03:25:53.993 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade 2023-07-06T03:25:53.993 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna 2023-07-06T03:25:53.993 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-06T03:25:53.993 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio 2023-07-06T03:25:53.993 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-06T03:25:53.993 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-06T03:25:53.994 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-06T03:25:53.994 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-06T03:25:53.994 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-06T03:25:53.994 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-06T03:25:53.994 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-06T03:25:53.994 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-06T03:25:53.994 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-06T03:25:53.995 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-06T03:25:53.995 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco 2023-07-06T03:25:53.995 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic 2023-07-06T03:25:53.995 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-06T03:25:53.995 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec 2023-07-06T03:25:53.995 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip 2023-07-06T03:25:53.995 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-06T03:25:53.995 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-06T03:25:53.996 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-06T03:25:53.996 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-06T03:25:53.996 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-06T03:25:53.996 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-06T03:25:53.996 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink 2023-07-06T03:25:53.996 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-06T03:25:53.996 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-06T03:25:53.996 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dnet.ko 2023-07-06T03:25:53.997 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex 2023-07-06T03:25:53.997 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet 2023-07-06T03:25:53.997 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-06T03:25:53.997 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ethoc.ko 2023-07-06T03:25:53.997 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/fealnx.ko 2023-07-06T03:25:53.997 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel 2023-07-06T03:25:53.997 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-06T03:25:53.998 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000 2023-07-06T03:25:53.998 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-06T03:25:53.998 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e 2023-07-06T03:25:53.998 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-06T03:25:53.998 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k 2023-07-06T03:25:53.998 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-06T03:25:53.998 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e 2023-07-06T03:25:53.998 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-06T03:25:53.999 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf 2023-07-06T03:25:53.999 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-06T03:25:53.999 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb 2023-07-06T03:25:53.999 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-06T03:25:53.999 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf 2023-07-06T03:25:53.999 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-06T03:25:53.999 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-06T03:25:53.999 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-06T03:25:54.000 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-06T03:25:54.000 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-06T03:25:54.000 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/jme.ko 2023-07-06T03:25:54.000 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell 2023-07-06T03:25:54.000 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-06T03:25:54.000 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-06T03:25:54.000 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-06T03:25:54.000 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox 2023-07-06T03:25:54.001 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-06T03:25:54.001 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-06T03:25:54.001 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-06T03:25:54.001 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-06T03:25:54.001 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-06T03:25:54.001 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-06T03:25:54.001 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel 2023-07-06T03:25:54.001 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-06T03:25:54.002 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom 2023-07-06T03:25:54.002 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-06T03:25:54.002 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-06T03:25:54.002 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi 2023-07-06T03:25:54.002 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-06T03:25:54.002 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-06T03:25:54.002 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion 2023-07-06T03:25:54.002 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-06T03:25:54.003 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia 2023-07-06T03:25:54.003 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-06T03:25:54.003 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines 2023-07-06T03:25:54.003 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-06T03:25:54.003 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-06T03:25:54.003 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic 2023-07-06T03:25:54.003 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-06T03:25:54.004 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-06T03:25:54.004 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-06T03:25:54.004 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-06T03:25:54.004 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-06T03:25:54.004 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc 2023-07-06T03:25:54.004 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-06T03:25:54.004 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek 2023-07-06T03:25:54.004 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-06T03:25:54.005 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-06T03:25:54.005 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-06T03:25:54.005 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-06T03:25:54.005 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker 2023-07-06T03:25:54.005 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-06T03:25:54.005 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc 2023-07-06T03:25:54.005 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-06T03:25:54.006 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan 2023-07-06T03:25:54.006 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-06T03:25:54.006 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis 2023-07-06T03:25:54.006 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-06T03:25:54.006 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-06T03:25:54.006 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc 2023-07-06T03:25:54.006 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-06T03:25:54.006 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-06T03:25:54.007 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-06T03:25:54.007 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-06T03:25:54.007 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro 2023-07-06T03:25:54.007 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-06T03:25:54.007 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-06T03:25:54.007 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-06T03:25:54.007 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun 2023-07-06T03:25:54.007 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-06T03:25:54.008 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-06T03:25:54.008 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-06T03:25:54.008 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-06T03:25:54.008 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti 2023-07-06T03:25:54.008 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-06T03:25:54.008 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti 2023-07-06T03:25:54.008 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-06T03:25:54.009 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via 2023-07-06T03:25:54.009 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-06T03:25:54.009 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-06T03:25:54.009 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet 2023-07-06T03:25:54.009 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-06T03:25:54.009 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-06T03:25:54.009 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom 2023-07-06T03:25:54.009 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-06T03:25:54.010 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/geneve.ko 2023-07-06T03:25:54.010 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv 2023-07-06T03:25:54.010 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-06T03:25:54.010 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154 2023-07-06T03:25:54.010 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-06T03:25:54.010 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ifb.ko 2023-07-06T03:25:54.010 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan 2023-07-06T03:25:54.010 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-06T03:25:54.011 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvlan.ko 2023-07-06T03:25:54.011 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvtap.ko 2023-07-06T03:25:54.011 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio 2023-07-06T03:25:54.011 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio.ko 2023-07-06T03:25:54.011 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-06T03:25:54.011 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-06T03:25:54.011 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mii.ko 2023-07-06T03:25:54.012 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/net_failover.ko 2023-07-06T03:25:54.012 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/netconsole.ko 2023-07-06T03:25:54.012 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/nlmon.ko 2023-07-06T03:25:54.012 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs 2023-07-06T03:25:54.012 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-07-06T03:25:54.012 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-06T03:25:54.012 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy 2023-07-06T03:25:54.013 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/amd.ko 2023-07-06T03:25:54.013 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-06T03:25:54.013 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-06T03:25:54.013 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-06T03:25:54.013 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm87xx.ko 2023-07-06T03:25:54.013 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/broadcom.ko 2023-07-06T03:25:54.013 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/cicada.ko 2023-07-06T03:25:54.013 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/davicom.ko 2023-07-06T03:25:54.014 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/dp83640.ko 2023-07-06T03:25:54.014 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/et1011c.ko 2023-07-06T03:25:54.014 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/icplus.ko 2023-07-06T03:25:54.014 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/lxt.ko 2023-07-06T03:25:54.014 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/marvell.ko 2023-07-06T03:25:54.014 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/micrel.ko 2023-07-06T03:25:54.014 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/national.ko 2023-07-06T03:25:54.014 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/phylink.ko 2023-07-06T03:25:54.015 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/qsemi.ko 2023-07-06T03:25:54.015 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/realtek.ko 2023-07-06T03:25:54.015 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/smsc.ko 2023-07-06T03:25:54.015 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/ste10Xp.ko 2023-07-06T03:25:54.015 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/vitesse.ko 2023-07-06T03:25:54.015 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp 2023-07-06T03:25:54.015 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-06T03:25:54.015 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_async.ko 2023-07-06T03:25:54.016 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-06T03:25:54.016 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-06T03:25:54.016 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-06T03:25:54.016 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-06T03:25:54.016 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppoe.ko 2023-07-06T03:25:54.016 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppox.ko 2023-07-06T03:25:54.016 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pptp.ko 2023-07-06T03:25:54.016 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip 2023-07-06T03:25:54.017 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slhc.ko 2023-07-06T03:25:54.017 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slip.ko 2023-07-06T03:25:54.017 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/sungem_phy.ko 2023-07-06T03:25:54.017 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tap.ko 2023-07-06T03:25:54.017 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team 2023-07-06T03:25:54.017 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team.ko 2023-07-06T03:25:54.017 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-06T03:25:54.018 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-06T03:25:54.018 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-06T03:25:54.018 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_random.ko 2023-07-06T03:25:54.018 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-06T03:25:54.018 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tun.ko 2023-07-06T03:25:54.018 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/veth.ko 2023-07-06T03:25:54.018 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/virtio_net.ko 2023-07-06T03:25:54.018 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3 2023-07-06T03:25:54.019 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-06T03:25:54.019 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan 2023-07-06T03:25:54.019 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan/vxlan.ko 2023-07-06T03:25:54.019 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback 2023-07-06T03:25:54.019 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-06T03:25:54.019 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netfront.ko 2023-07-06T03:25:54.019 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb 2023-07-06T03:25:54.019 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb/ntb.ko 2023-07-06T03:25:54.020 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme 2023-07-06T03:25:54.020 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host 2023-07-06T03:25:54.020 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme-core.ko 2023-07-06T03:25:54.020 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme.ko 2023-07-06T03:25:54.020 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport 2023-07-06T03:25:54.020 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport.ko 2023-07-06T03:25:54.020 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_cs.ko 2023-07-06T03:25:54.021 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_pc.ko 2023-07-06T03:25:54.021 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_serial.ko 2023-07-06T03:25:54.021 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci 2023-07-06T03:25:54.021 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug 2023-07-06T03:25:54.021 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-06T03:25:54.021 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie 2023-07-06T03:25:54.021 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-06T03:25:54.022 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/xen-pcifront.ko 2023-07-06T03:25:54.022 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia 2023-07-06T03:25:54.022 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/i82092.ko 2023-07-06T03:25:54.022 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/pd6729.ko 2023-07-06T03:25:54.022 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-06T03:25:54.022 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl 2023-07-06T03:25:54.022 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel 2023-07-06T03:25:54.022 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-06T03:25:54.023 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform 2023-07-06T03:25:54.023 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86 2023-07-06T03:25:54.023 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-06T03:25:54.023 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acerhdf.ko 2023-07-06T03:25:54.023 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-06T03:25:54.023 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-06T03:25:54.023 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-06T03:25:54.024 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-06T03:25:54.024 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-06T03:25:54.024 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-06T03:25:54.024 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-06T03:25:54.024 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-06T03:25:54.024 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-06T03:25:54.024 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/hdaps.ko 2023-07-06T03:25:54.024 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel 2023-07-06T03:25:54.025 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-06T03:25:54.025 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel_ips.ko 2023-07-06T03:25:54.025 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-06T03:25:54.025 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-06T03:25:54.025 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-06T03:25:54.025 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-06T03:25:54.025 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-06T03:25:54.025 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-06T03:25:54.026 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-06T03:25:54.026 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-06T03:25:54.026 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-06T03:25:54.026 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi.ko 2023-07-06T03:25:54.026 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap 2023-07-06T03:25:54.026 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-06T03:25:54.026 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-06T03:25:54.026 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps 2023-07-06T03:25:54.027 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients 2023-07-06T03:25:54.027 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-06T03:25:54.027 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-06T03:25:54.027 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps_parport.ko 2023-07-06T03:25:54.027 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/pps_core.ko 2023-07-06T03:25:54.027 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp 2023-07-06T03:25:54.027 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp.ko 2023-07-06T03:25:54.027 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp_kvm.ko 2023-07-06T03:25:54.028 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc 2023-07-06T03:25:54.028 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-06T03:25:54.028 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-06T03:25:54.028 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-06T03:25:54.028 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-06T03:25:54.028 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-06T03:25:54.028 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-06T03:25:54.029 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-06T03:25:54.029 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-06T03:25:54.029 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-06T03:25:54.029 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-06T03:25:54.029 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-06T03:25:54.029 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-em3027.ko 2023-07-06T03:25:54.029 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-06T03:25:54.029 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-06T03:25:54.030 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-06T03:25:54.030 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-06T03:25:54.030 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-06T03:25:54.030 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-06T03:25:54.030 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-max6900.ko 2023-07-06T03:25:54.030 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-06T03:25:54.030 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-06T03:25:54.030 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-06T03:25:54.031 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-06T03:25:54.031 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-06T03:25:54.031 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-06T03:25:54.031 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-06T03:25:54.031 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-06T03:25:54.031 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-06T03:25:54.031 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-06T03:25:54.031 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-06T03:25:54.032 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-06T03:25:54.032 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-x1205.ko 2023-07-06T03:25:54.032 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi 2023-07-06T03:25:54.032 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-9xxx.ko 2023-07-06T03:25:54.032 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-sas.ko 2023-07-06T03:25:54.032 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-xxxx.ko 2023-07-06T03:25:54.032 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/BusLogic.ko 2023-07-06T03:25:54.032 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/a100u2w.ko 2023-07-06T03:25:54.033 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid 2023-07-06T03:25:54.033 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-06T03:25:54.033 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/advansys.ko 2023-07-06T03:25:54.033 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx 2023-07-06T03:25:54.033 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-06T03:25:54.033 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-06T03:25:54.033 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/am53c974.ko 2023-07-06T03:25:54.033 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr 2023-07-06T03:25:54.034 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-06T03:25:54.034 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/atp870u.ko 2023-07-06T03:25:54.034 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi 2023-07-06T03:25:54.034 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-06T03:25:54.034 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa 2023-07-06T03:25:54.034 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa/bfa.ko 2023-07-06T03:25:54.034 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc 2023-07-06T03:25:54.034 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-06T03:25:54.035 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i 2023-07-06T03:25:54.035 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-06T03:25:54.035 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ch.ko 2023-07-06T03:25:54.035 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor 2023-07-06T03:25:54.035 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-06T03:25:54.035 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi 2023-07-06T03:25:54.035 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-06T03:25:54.035 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-06T03:25:54.036 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-06T03:25:54.036 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-06T03:25:54.036 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-06T03:25:54.036 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dc395x.ko 2023-07-06T03:25:54.036 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler 2023-07-06T03:25:54.036 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-06T03:25:54.036 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-06T03:25:54.037 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-06T03:25:54.037 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-06T03:25:54.037 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dmx3191d.ko 2023-07-06T03:25:54.037 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r 2023-07-06T03:25:54.037 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-06T03:25:54.037 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esp_scsi.ko 2023-07-06T03:25:54.037 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe 2023-07-06T03:25:54.037 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-06T03:25:54.038 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-06T03:25:54.038 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic 2023-07-06T03:25:54.038 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic/fnic.ko 2023-07-06T03:25:54.038 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hpsa.ko 2023-07-06T03:25:54.038 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hptiop.ko 2023-07-06T03:25:54.038 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hv_storvsc.ko 2023-07-06T03:25:54.038 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/initio.ko 2023-07-06T03:25:54.038 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ipr.ko 2023-07-06T03:25:54.039 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ips.ko 2023-07-06T03:25:54.039 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci 2023-07-06T03:25:54.039 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci/isci.ko 2023-07-06T03:25:54.039 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-06T03:25:54.039 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-06T03:25:54.039 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc 2023-07-06T03:25:54.039 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc/libfc.ko 2023-07-06T03:25:54.039 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi.ko 2023-07-06T03:25:54.040 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-06T03:25:54.040 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas 2023-07-06T03:25:54.040 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas/libsas.ko 2023-07-06T03:25:54.040 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc 2023-07-06T03:25:54.040 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-06T03:25:54.040 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid 2023-07-06T03:25:54.040 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid.ko 2023-07-06T03:25:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-06T03:25:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-06T03:25:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-06T03:25:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas 2023-07-06T03:25:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-06T03:25:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas 2023-07-06T03:25:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-06T03:25:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvumi.ko 2023-07-06T03:25:54.042 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001 2023-07-06T03:25:54.042 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-06T03:25:54.042 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pmcraid.ko 2023-07-06T03:25:54.042 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla1280.ko 2023-07-06T03:25:54.042 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx 2023-07-06T03:25:54.042 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-06T03:25:54.042 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-06T03:25:54.042 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx 2023-07-06T03:25:54.043 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-06T03:25:54.043 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/raid_class.ko 2023-07-06T03:25:54.043 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_debug.ko 2023-07-06T03:25:54.043 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-06T03:25:54.043 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-06T03:25:54.043 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-06T03:25:54.043 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-06T03:25:54.043 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-06T03:25:54.044 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ses.ko 2023-07-06T03:25:54.044 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/st.ko 2023-07-06T03:25:54.044 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/stex.ko 2023-07-06T03:25:54.044 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2 2023-07-06T03:25:54.044 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-06T03:25:54.044 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/virtio_scsi.ko 2023-07-06T03:25:54.044 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-06T03:25:54.044 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/wd719x.ko 2023-07-06T03:25:54.045 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb 2023-07-06T03:25:54.045 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb/ssb.ko 2023-07-06T03:25:54.045 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target 2023-07-06T03:25:54.045 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi 2023-07-06T03:25:54.045 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-06T03:25:54.045 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback 2023-07-06T03:25:54.045 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-06T03:25:54.045 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_file.ko 2023-07-06T03:25:54.046 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_iblock.ko 2023-07-06T03:25:54.046 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_mod.ko 2023-07-06T03:25:54.046 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_pscsi.ko 2023-07-06T03:25:54.046 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_user.ko 2023-07-06T03:25:54.046 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc 2023-07-06T03:25:54.046 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-06T03:25:54.046 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal 2023-07-06T03:25:54.046 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel 2023-07-06T03:25:54.047 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal 2023-07-06T03:25:54.047 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-06T03:25:54.047 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-06T03:25:54.047 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-06T03:25:54.047 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-06T03:25:54.047 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-06T03:25:54.047 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-06T03:25:54.047 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-06T03:25:54.048 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-06T03:25:54.048 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-06T03:25:54.048 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-06T03:25:54.048 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-06T03:25:54.048 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-06T03:25:54.048 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-06T03:25:54.048 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-06T03:25:54.049 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-06T03:25:54.049 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty 2023-07-06T03:25:54.049 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless 2023-07-06T03:25:54.049 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-06T03:25:54.049 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_gsm.ko 2023-07-06T03:25:54.049 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_hdlc.ko 2023-07-06T03:25:54.049 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/nozomi.ko 2023-07-06T03:25:54.049 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial 2023-07-06T03:25:54.050 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250 2023-07-06T03:25:54.050 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-06T03:25:54.050 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/arc_uart.ko 2023-07-06T03:25:54.050 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm 2023-07-06T03:25:54.050 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-06T03:25:54.050 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/synclink_gt.ko 2023-07-06T03:25:54.050 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs 2023-07-06T03:25:54.050 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core 2023-07-06T03:25:54.051 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-06T03:25:54.051 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host 2023-07-06T03:25:54.051 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-06T03:25:54.051 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio 2023-07-06T03:25:54.051 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio.ko 2023-07-06T03:25:54.051 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_aec.ko 2023-07-06T03:25:54.051 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_cif.ko 2023-07-06T03:25:54.051 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_pci_generic.ko 2023-07-06T03:25:54.052 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_sercos3.ko 2023-07-06T03:25:54.052 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb 2023-07-06T03:25:54.052 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class 2023-07-06T03:25:54.052 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-acm.ko 2023-07-06T03:25:54.052 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-06T03:25:54.052 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usblp.ko 2023-07-06T03:25:54.052 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usbtmc.ko 2023-07-06T03:25:54.052 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host 2023-07-06T03:25:54.053 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-06T03:25:54.053 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image 2023-07-06T03:25:54.053 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/mdc800.ko 2023-07-06T03:25:54.053 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/microtek.ko 2023-07-06T03:25:54.053 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc 2023-07-06T03:25:54.053 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/adutux.ko 2023-07-06T03:25:54.053 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/appledisplay.ko 2023-07-06T03:25:54.053 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi26.ko 2023-07-06T03:25:54.054 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi62.ko 2023-07-06T03:25:54.054 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ezusb.ko 2023-07-06T03:25:54.054 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/idmouse.ko 2023-07-06T03:25:54.054 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/iowarrior.ko 2023-07-06T03:25:54.054 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-06T03:25:54.054 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ldusb.ko 2023-07-06T03:25:54.054 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/legousbtower.ko 2023-07-06T03:25:54.054 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga 2023-07-06T03:25:54.055 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-06T03:25:54.055 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-06T03:25:54.055 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usb3503.ko 2023-07-06T03:25:54.055 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usblcd.ko 2023-07-06T03:25:54.055 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-06T03:25:54.055 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/uss720.ko 2023-07-06T03:25:54.055 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/yurex.ko 2023-07-06T03:25:54.055 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy 2023-07-06T03:25:54.056 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy/phy-generic.ko 2023-07-06T03:25:54.056 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial 2023-07-06T03:25:54.056 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/aircable.ko 2023-07-06T03:25:54.056 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ark3116.ko 2023-07-06T03:25:54.056 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-06T03:25:54.056 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ch341.ko 2023-07-06T03:25:54.056 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cp210x.ko 2023-07-06T03:25:54.056 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cyberjack.ko 2023-07-06T03:25:54.057 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-06T03:25:54.057 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-06T03:25:54.057 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/empeg.ko 2023-07-06T03:25:54.057 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-06T03:25:54.057 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-06T03:25:54.057 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-06T03:25:54.057 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_ti.ko 2023-07-06T03:25:54.058 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipaq.ko 2023-07-06T03:25:54.058 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipw.ko 2023-07-06T03:25:54.058 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ir-usb.ko 2023-07-06T03:25:54.058 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-06T03:25:54.058 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan.ko 2023-07-06T03:25:54.058 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-06T03:25:54.058 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-06T03:25:54.058 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-06T03:25:54.059 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mct_u232.ko 2023-07-06T03:25:54.059 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7720.ko 2023-07-06T03:25:54.059 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7840.ko 2023-07-06T03:25:54.059 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/navman.ko 2023-07-06T03:25:54.059 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/omninet.ko 2023-07-06T03:25:54.059 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/opticon.ko 2023-07-06T03:25:54.059 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/option.ko 2023-07-06T03:25:54.059 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/oti6858.ko 2023-07-06T03:25:54.060 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/pl2303.ko 2023-07-06T03:25:54.060 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcaux.ko 2023-07-06T03:25:54.060 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcserial.ko 2023-07-06T03:25:54.060 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/quatech2.ko 2023-07-06T03:25:54.060 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/safe_serial.ko 2023-07-06T03:25:54.060 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/sierra.ko 2023-07-06T03:25:54.060 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-06T03:25:54.060 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ssu100.ko 2023-07-06T03:25:54.061 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/symbolserial.ko 2023-07-06T03:25:54.061 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-06T03:25:54.061 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-06T03:25:54.061 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_debug.ko 2023-07-06T03:25:54.061 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-06T03:25:54.061 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/visor.ko 2023-07-06T03:25:54.061 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/whiteheat.ko 2023-07-06T03:25:54.061 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-06T03:25:54.062 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage 2023-07-06T03:25:54.062 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/uas.ko 2023-07-06T03:25:54.062 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-06T03:25:54.062 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-06T03:25:54.062 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-06T03:25:54.062 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-06T03:25:54.063 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-06T03:25:54.063 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-06T03:25:54.063 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-06T03:25:54.063 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-karma.ko 2023-07-06T03:25:54.063 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-06T03:25:54.063 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-06T03:25:54.063 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-06T03:25:54.063 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-06T03:25:54.064 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-06T03:25:54.064 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/usb-storage.ko 2023-07-06T03:25:54.064 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip 2023-07-06T03:25:54.064 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-06T03:25:54.064 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-06T03:25:54.064 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-06T03:25:54.064 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio 2023-07-06T03:25:54.065 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci 2023-07-06T03:25:54.065 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-06T03:25:54.065 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-06T03:25:54.065 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio.ko 2023-07-06T03:25:54.065 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-06T03:25:54.065 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost 2023-07-06T03:25:54.065 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost.ko 2023-07-06T03:25:54.065 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-06T03:25:54.066 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_net.ko 2023-07-06T03:25:54.066 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_scsi.ko 2023-07-06T03:25:54.066 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video 2023-07-06T03:25:54.066 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight 2023-07-06T03:25:54.066 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/apple_bl.ko 2023-07-06T03:25:54.066 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/lcd.ko 2023-07-06T03:25:54.066 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-06T03:25:54.066 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio 2023-07-06T03:25:54.067 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio.ko 2023-07-06T03:25:54.067 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_balloon.ko 2023-07-06T03:25:54.067 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_mmio.ko 2023-07-06T03:25:54.067 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci.ko 2023-07-06T03:25:54.067 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-06T03:25:54.067 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-06T03:25:54.067 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_ring.ko 2023-07-06T03:25:54.068 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1 2023-07-06T03:25:54.068 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters 2023-07-06T03:25:54.068 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2482.ko 2023-07-06T03:25:54.068 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2490.ko 2023-07-06T03:25:54.068 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves 2023-07-06T03:25:54.068 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-06T03:25:54.068 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-06T03:25:54.068 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-06T03:25:54.069 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-06T03:25:54.069 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-06T03:25:54.069 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-06T03:25:54.069 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-06T03:25:54.069 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-06T03:25:54.069 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-06T03:25:54.069 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-06T03:25:54.070 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-06T03:25:54.070 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/wire.ko 2023-07-06T03:25:54.070 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog 2023-07-06T03:25:54.070 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-06T03:25:54.070 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-06T03:25:54.070 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-06T03:25:54.070 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/hpwdt.ko 2023-07-06T03:25:54.070 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/i6300esb.ko 2023-07-06T03:25:54.071 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-06T03:25:54.071 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-06T03:25:54.071 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ib700wdt.ko 2023-07-06T03:25:54.071 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ibmasr.ko 2023-07-06T03:25:54.071 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-06T03:25:54.071 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-06T03:25:54.071 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it87_wdt.ko 2023-07-06T03:25:54.071 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/machzwd.ko 2023-07-06T03:25:54.072 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/nv_tco.ko 2023-07-06T03:25:54.072 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-06T03:25:54.072 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-06T03:25:54.072 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-06T03:25:54.072 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-06T03:25:54.072 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/softdog.ko 2023-07-06T03:25:54.072 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-06T03:25:54.072 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/via_wdt.ko 2023-07-06T03:25:54.073 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-06T03:25:54.073 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-06T03:25:54.073 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-06T03:25:54.073 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/wdt_pci.ko 2023-07-06T03:25:54.073 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/xen_wdt.ko 2023-07-06T03:25:54.073 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen 2023-07-06T03:25:54.073 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-06T03:25:54.073 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-evtchn.ko 2023-07-06T03:25:54.074 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntalloc.ko 2023-07-06T03:25:54.074 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntdev.ko 2023-07-06T03:25:54.074 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback 2023-07-06T03:25:54.074 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-06T03:25:54.074 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-privcmd.ko 2023-07-06T03:25:54.074 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs 2023-07-06T03:25:54.074 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-06T03:25:54.074 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs 2023-07-06T03:25:54.075 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p 2023-07-06T03:25:54.075 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p/9p.ko 2023-07-06T03:25:54.075 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs 2023-07-06T03:25:54.075 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs/affs.ko 2023-07-06T03:25:54.075 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs 2023-07-06T03:25:54.075 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs/befs.ko 2023-07-06T03:25:54.075 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/binfmt_misc.ko 2023-07-06T03:25:54.075 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs 2023-07-06T03:25:54.076 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs/btrfs.ko 2023-07-06T03:25:54.076 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles 2023-07-06T03:25:54.076 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles/cachefiles.ko 2023-07-06T03:25:54.076 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph 2023-07-06T03:25:54.076 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph/ceph.ko 2023-07-06T03:25:54.076 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda 2023-07-06T03:25:54.076 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda/coda.ko 2023-07-06T03:25:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs 2023-07-06T03:25:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs/cramfs.ko 2023-07-06T03:25:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm 2023-07-06T03:25:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm/dlm.ko 2023-07-06T03:25:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs 2023-07-06T03:25:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-06T03:25:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs 2023-07-06T03:25:54.077 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs/f2fs.ko 2023-07-06T03:25:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat 2023-07-06T03:25:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/fat.ko 2023-07-06T03:25:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/msdos.ko 2023-07-06T03:25:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/vfat.ko 2023-07-06T03:25:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache 2023-07-06T03:25:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache/fscache.ko 2023-07-06T03:25:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse 2023-07-06T03:25:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/cuse.ko 2023-07-06T03:25:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/fuse.ko 2023-07-06T03:25:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2 2023-07-06T03:25:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2/gfs2.ko 2023-07-06T03:25:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs 2023-07-06T03:25:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs/hfs.ko 2023-07-06T03:25:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus 2023-07-06T03:25:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus/hfsplus.ko 2023-07-06T03:25:54.079 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs 2023-07-06T03:25:54.080 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs/isofs.ko 2023-07-06T03:25:54.080 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs 2023-07-06T03:25:54.080 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs/jfs.ko 2023-07-06T03:25:54.080 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd 2023-07-06T03:25:54.080 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd/lockd.ko 2023-07-06T03:25:54.080 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix 2023-07-06T03:25:54.080 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix/minix.ko 2023-07-06T03:25:54.080 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs 2023-07-06T03:25:54.081 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs/netfs.ko 2023-07-06T03:25:54.081 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs 2023-07-06T03:25:54.081 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout 2023-07-06T03:25:54.081 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-06T03:25:54.081 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout 2023-07-06T03:25:54.081 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-06T03:25:54.081 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout 2023-07-06T03:25:54.081 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-06T03:25:54.082 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfs.ko 2023-07-06T03:25:54.082 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv3.ko 2023-07-06T03:25:54.082 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv4.ko 2023-07-06T03:25:54.082 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common 2023-07-06T03:25:54.082 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/grace.ko 2023-07-06T03:25:54.082 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/nfs_acl.ko 2023-07-06T03:25:54.082 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd 2023-07-06T03:25:54.083 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd/nfsd.ko 2023-07-06T03:25:54.083 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2 2023-07-06T03:25:54.083 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2/nilfs2.ko 2023-07-06T03:25:54.083 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls 2023-07-06T03:25:54.083 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-celtic.ko 2023-07-06T03:25:54.083 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-centeuro.ko 2023-07-06T03:25:54.083 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-croatian.ko 2023-07-06T03:25:54.083 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-cyrillic.ko 2023-07-06T03:25:54.084 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-gaelic.ko 2023-07-06T03:25:54.084 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-greek.ko 2023-07-06T03:25:54.084 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-iceland.ko 2023-07-06T03:25:54.084 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-inuit.ko 2023-07-06T03:25:54.084 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-roman.ko 2023-07-06T03:25:54.084 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-romanian.ko 2023-07-06T03:25:54.084 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-turkish.ko 2023-07-06T03:25:54.085 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1250.ko 2023-07-06T03:25:54.085 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1251.ko 2023-07-06T03:25:54.085 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1255.ko 2023-07-06T03:25:54.085 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp737.ko 2023-07-06T03:25:54.085 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp775.ko 2023-07-06T03:25:54.085 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp850.ko 2023-07-06T03:25:54.085 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp852.ko 2023-07-06T03:25:54.085 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp855.ko 2023-07-06T03:25:54.086 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp857.ko 2023-07-06T03:25:54.086 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp860.ko 2023-07-06T03:25:54.086 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp861.ko 2023-07-06T03:25:54.086 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp862.ko 2023-07-06T03:25:54.086 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp863.ko 2023-07-06T03:25:54.086 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp864.ko 2023-07-06T03:25:54.086 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp865.ko 2023-07-06T03:25:54.086 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp866.ko 2023-07-06T03:25:54.087 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp869.ko 2023-07-06T03:25:54.087 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp874.ko 2023-07-06T03:25:54.087 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp932.ko 2023-07-06T03:25:54.087 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp936.ko 2023-07-06T03:25:54.087 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp949.ko 2023-07-06T03:25:54.087 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp950.ko 2023-07-06T03:25:54.087 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_euc-jp.ko 2023-07-06T03:25:54.087 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-1.ko 2023-07-06T03:25:54.088 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-13.ko 2023-07-06T03:25:54.088 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-14.ko 2023-07-06T03:25:54.088 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-15.ko 2023-07-06T03:25:54.088 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-2.ko 2023-07-06T03:25:54.088 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-3.ko 2023-07-06T03:25:54.088 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-4.ko 2023-07-06T03:25:54.088 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-5.ko 2023-07-06T03:25:54.088 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-6.ko 2023-07-06T03:25:54.089 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-7.ko 2023-07-06T03:25:54.089 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-9.ko 2023-07-06T03:25:54.089 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-r.ko 2023-07-06T03:25:54.089 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-ru.ko 2023-07-06T03:25:54.089 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-u.ko 2023-07-06T03:25:54.089 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_utf8.ko 2023-07-06T03:25:54.089 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2 2023-07-06T03:25:54.090 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster 2023-07-06T03:25:54.090 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-06T03:25:54.090 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm 2023-07-06T03:25:54.090 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-06T03:25:54.090 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs 2023-07-06T03:25:54.090 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-06T03:25:54.090 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2.ko 2023-07-06T03:25:54.090 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-06T03:25:54.091 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-06T03:25:54.091 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-06T03:25:54.091 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs 2023-07-06T03:25:54.091 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs/overlay.ko 2023-07-06T03:25:54.091 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore 2023-07-06T03:25:54.091 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore/ramoops.ko 2023-07-06T03:25:54.091 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs 2023-07-06T03:25:54.092 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs/reiserfs.ko 2023-07-06T03:25:54.092 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs 2023-07-06T03:25:54.092 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs/romfs.ko 2023-07-06T03:25:54.092 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb 2023-07-06T03:25:54.092 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client 2023-07-06T03:25:54.092 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client/cifs.ko 2023-07-06T03:25:54.092 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common 2023-07-06T03:25:54.092 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_arc4.ko 2023-07-06T03:25:54.093 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_md4.ko 2023-07-06T03:25:54.093 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs 2023-07-06T03:25:54.093 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs/squashfs.ko 2023-07-06T03:25:54.093 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv 2023-07-06T03:25:54.093 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv/sysv.ko 2023-07-06T03:25:54.093 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf 2023-07-06T03:25:54.093 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf/udf.ko 2023-07-06T03:25:54.094 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs 2023-07-06T03:25:54.094 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs/ufs.ko 2023-07-06T03:25:54.094 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs 2023-07-06T03:25:54.094 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs/xfs.ko 2023-07-06T03:25:54.094 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel 2023-07-06T03:25:54.094 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu 2023-07-06T03:25:54.094 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu/rcutorture.ko 2023-07-06T03:25:54.094 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/torture.ko 2023-07-06T03:25:54.095 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace 2023-07-06T03:25:54.095 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-06T03:25:54.095 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib 2023-07-06T03:25:54.095 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/asn1_encoder.ko 2023-07-06T03:25:54.095 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc-itu-t.ko 2023-07-06T03:25:54.095 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc8.ko 2023-07-06T03:25:54.095 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto 2023-07-06T03:25:54.095 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libarc4.ko 2023-07-06T03:25:54.096 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libdes.ko 2023-07-06T03:25:54.096 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/libcrc32c.ko 2023-07-06T03:25:54.096 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lru_cache.ko 2023-07-06T03:25:54.096 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4 2023-07-06T03:25:54.096 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4_compress.ko 2023-07-06T03:25:54.096 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4hc_compress.ko 2023-07-06T03:25:54.096 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math 2023-07-06T03:25:54.097 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math/cordic.ko 2023-07-06T03:25:54.097 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6 2023-07-06T03:25:54.097 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6/raid6_pq.ko 2023-07-06T03:25:54.097 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon 2023-07-06T03:25:54.097 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-06T03:25:54.097 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_bm.ko 2023-07-06T03:25:54.097 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_fsm.ko 2023-07-06T03:25:54.097 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_kmp.ko 2023-07-06T03:25:54.098 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd 2023-07-06T03:25:54.098 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd/zstd_compress.ko 2023-07-06T03:25:54.098 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm 2023-07-06T03:25:54.098 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm/hwpoison-inject.ko 2023-07-06T03:25:54.098 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net 2023-07-06T03:25:54.098 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan 2023-07-06T03:25:54.098 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/6lowpan.ko 2023-07-06T03:25:54.099 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_dest.ko 2023-07-06T03:25:54.099 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_fragment.ko 2023-07-06T03:25:54.099 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_hop.ko 2023-07-06T03:25:54.099 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-06T03:25:54.099 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_mobility.ko 2023-07-06T03:25:54.099 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_routing.ko 2023-07-06T03:25:54.099 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_udp.ko 2023-07-06T03:25:54.099 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802 2023-07-06T03:25:54.100 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/garp.ko 2023-07-06T03:25:54.100 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/mrp.ko 2023-07-06T03:25:54.100 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/p8022.ko 2023-07-06T03:25:54.100 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/psnap.ko 2023-07-06T03:25:54.100 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/stp.ko 2023-07-06T03:25:54.100 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q 2023-07-06T03:25:54.100 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q/8021q.ko 2023-07-06T03:25:54.101 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p 2023-07-06T03:25:54.101 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet.ko 2023-07-06T03:25:54.101 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_fd.ko 2023-07-06T03:25:54.101 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_rdma.ko 2023-07-06T03:25:54.101 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_virtio.ko 2023-07-06T03:25:54.101 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv 2023-07-06T03:25:54.101 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv/batman-adv.ko 2023-07-06T03:25:54.102 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge 2023-07-06T03:25:54.102 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/br_netfilter.ko 2023-07-06T03:25:54.102 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/bridge.ko 2023-07-06T03:25:54.102 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter 2023-07-06T03:25:54.102 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-06T03:25:54.102 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-06T03:25:54.102 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-06T03:25:54.102 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-06T03:25:54.103 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-06T03:25:54.103 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-06T03:25:54.103 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-06T03:25:54.103 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-06T03:25:54.103 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-06T03:25:54.103 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-06T03:25:54.103 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-06T03:25:54.103 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-06T03:25:54.104 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-06T03:25:54.104 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-06T03:25:54.104 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-06T03:25:54.104 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-06T03:25:54.104 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-06T03:25:54.104 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-06T03:25:54.104 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-06T03:25:54.105 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-06T03:25:54.105 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtables.ko 2023-07-06T03:25:54.105 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-06T03:25:54.105 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-06T03:25:54.105 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-06T03:25:54.105 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph 2023-07-06T03:25:54.105 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph/libceph.ko 2023-07-06T03:25:54.105 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core 2023-07-06T03:25:54.106 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/failover.ko 2023-07-06T03:25:54.106 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/pktgen.ko 2023-07-06T03:25:54.106 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp 2023-07-06T03:25:54.106 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp.ko 2023-07-06T03:25:54.106 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_diag.ko 2023-07-06T03:25:54.106 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv4.ko 2023-07-06T03:25:54.106 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv6.ko 2023-07-06T03:25:54.106 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver 2023-07-06T03:25:54.107 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver/dns_resolver.ko 2023-07-06T03:25:54.107 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa 2023-07-06T03:25:54.107 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/dsa_core.ko 2023-07-06T03:25:54.107 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_brcm.ko 2023-07-06T03:25:54.107 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_dsa.ko 2023-07-06T03:25:54.107 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_none.ko 2023-07-06T03:25:54.107 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_trailer.ko 2023-07-06T03:25:54.107 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154 2023-07-06T03:25:54.108 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan 2023-07-06T03:25:54.108 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-06T03:25:54.108 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154.ko 2023-07-06T03:25:54.108 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-06T03:25:54.108 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4 2023-07-06T03:25:54.108 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ah4.ko 2023-07-06T03:25:54.108 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/esp4.ko 2023-07-06T03:25:54.109 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/fou.ko 2023-07-06T03:25:54.109 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/gre.ko 2023-07-06T03:25:54.109 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/inet_diag.ko 2023-07-06T03:25:54.109 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_gre.ko 2023-07-06T03:25:54.109 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_tunnel.ko 2023-07-06T03:25:54.109 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_vti.ko 2023-07-06T03:25:54.109 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipcomp.ko 2023-07-06T03:25:54.109 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipip.ko 2023-07-06T03:25:54.110 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter 2023-07-06T03:25:54.110 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-06T03:25:54.110 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-06T03:25:54.110 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-06T03:25:54.110 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-06T03:25:54.110 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-06T03:25:54.110 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-06T03:25:54.111 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-06T03:25:54.111 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-06T03:25:54.111 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-06T03:25:54.111 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-06T03:25:54.111 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-06T03:25:54.111 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-06T03:25:54.111 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-06T03:25:54.111 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-06T03:25:54.112 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-06T03:25:54.112 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-06T03:25:54.112 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-06T03:25:54.112 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-06T03:25:54.112 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-06T03:25:54.112 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-06T03:25:54.113 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-06T03:25:54.113 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_bic.ko 2023-07-06T03:25:54.113 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_dctcp.ko 2023-07-06T03:25:54.113 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_diag.ko 2023-07-06T03:25:54.113 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_highspeed.ko 2023-07-06T03:25:54.113 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_htcp.ko 2023-07-06T03:25:54.113 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_hybla.ko 2023-07-06T03:25:54.113 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_illinois.ko 2023-07-06T03:25:54.114 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_lp.ko 2023-07-06T03:25:54.114 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_scalable.ko 2023-07-06T03:25:54.114 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_vegas.ko 2023-07-06T03:25:54.114 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_veno.ko 2023-07-06T03:25:54.114 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_westwood.ko 2023-07-06T03:25:54.114 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_yeah.ko 2023-07-06T03:25:54.114 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tunnel4.ko 2023-07-06T03:25:54.114 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_diag.ko 2023-07-06T03:25:54.115 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_tunnel.ko 2023-07-06T03:25:54.115 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-06T03:25:54.115 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6 2023-07-06T03:25:54.115 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ah6.ko 2023-07-06T03:25:54.115 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/esp6.ko 2023-07-06T03:25:54.115 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/fou6.ko 2023-07-06T03:25:54.115 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_tunnel.ko 2023-07-06T03:25:54.116 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-06T03:25:54.116 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_vti.ko 2023-07-06T03:25:54.116 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ipcomp6.ko 2023-07-06T03:25:54.116 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter 2023-07-06T03:25:54.116 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-06T03:25:54.116 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-06T03:25:54.116 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-06T03:25:54.116 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-06T03:25:54.117 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-06T03:25:54.117 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-06T03:25:54.117 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-06T03:25:54.117 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-06T03:25:54.117 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-06T03:25:54.117 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-06T03:25:54.117 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-06T03:25:54.117 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-06T03:25:54.118 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-06T03:25:54.118 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-06T03:25:54.118 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-06T03:25:54.118 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-06T03:25:54.118 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-06T03:25:54.118 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-06T03:25:54.118 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-06T03:25:54.118 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-06T03:25:54.119 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-06T03:25:54.119 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-06T03:25:54.119 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-06T03:25:54.119 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-06T03:25:54.119 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/sit.ko 2023-07-06T03:25:54.119 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/tunnel6.ko 2023-07-06T03:25:54.119 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-06T03:25:54.119 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key 2023-07-06T03:25:54.120 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key/af_key.ko 2023-07-06T03:25:54.120 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp 2023-07-06T03:25:54.120 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_core.ko 2023-07-06T03:25:54.120 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-06T03:25:54.120 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_eth.ko 2023-07-06T03:25:54.120 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip.ko 2023-07-06T03:25:54.120 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip6.ko 2023-07-06T03:25:54.121 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_netlink.ko 2023-07-06T03:25:54.121 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ppp.ko 2023-07-06T03:25:54.121 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc 2023-07-06T03:25:54.121 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc/llc.ko 2023-07-06T03:25:54.121 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154 2023-07-06T03:25:54.121 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154/mac802154.ko 2023-07-06T03:25:54.121 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls 2023-07-06T03:25:54.121 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls/mpls_gso.ko 2023-07-06T03:25:54.122 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter 2023-07-06T03:25:54.122 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset 2023-07-06T03:25:54.122 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set.ko 2023-07-06T03:25:54.122 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-06T03:25:54.122 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-06T03:25:54.122 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-06T03:25:54.122 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-06T03:25:54.123 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-06T03:25:54.124 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-06T03:25:54.124 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-06T03:25:54.124 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-06T03:25:54.124 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-06T03:25:54.124 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-06T03:25:54.125 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-06T03:25:54.125 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-06T03:25:54.125 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-06T03:25:54.125 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-06T03:25:54.125 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-06T03:25:54.126 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs 2023-07-06T03:25:54.126 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-06T03:25:54.126 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-06T03:25:54.126 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-06T03:25:54.126 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-06T03:25:54.126 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-06T03:25:54.126 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-06T03:25:54.127 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-06T03:25:54.127 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-06T03:25:54.127 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-06T03:25:54.127 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-06T03:25:54.127 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-06T03:25:54.127 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-06T03:25:54.127 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-06T03:25:54.127 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-06T03:25:54.128 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conncount.ko 2023-07-06T03:25:54.128 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack.ko 2023-07-06T03:25:54.128 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-06T03:25:54.128 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-06T03:25:54.128 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-06T03:25:54.129 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-06T03:25:54.129 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-06T03:25:54.129 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-06T03:25:54.129 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-06T03:25:54.129 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-06T03:25:54.129 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-06T03:25:54.129 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-06T03:25:54.129 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-06T03:25:54.130 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-06T03:25:54.130 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-06T03:25:54.130 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table.ko 2023-07-06T03:25:54.130 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-06T03:25:54.130 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_log_syslog.ko 2023-07-06T03:25:54.130 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat.ko 2023-07-06T03:25:54.130 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-06T03:25:54.131 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-06T03:25:54.131 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_irc.ko 2023-07-06T03:25:54.131 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_sip.ko 2023-07-06T03:25:54.131 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-06T03:25:54.131 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-06T03:25:54.131 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_tables.ko 2023-07-06T03:25:54.132 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink.ko 2023-07-06T03:25:54.132 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-06T03:25:54.132 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_log.ko 2023-07-06T03:25:54.132 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-06T03:25:54.132 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-06T03:25:54.132 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_chain_nat.ko 2023-07-06T03:25:54.132 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_compat.ko 2023-07-06T03:25:54.133 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_ct.ko 2023-07-06T03:25:54.133 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-06T03:25:54.133 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fib.ko 2023-07-06T03:25:54.133 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-06T03:25:54.133 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_hash.ko 2023-07-06T03:25:54.133 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_limit.ko 2023-07-06T03:25:54.133 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_log.ko 2023-07-06T03:25:54.133 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_masq.ko 2023-07-06T03:25:54.134 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_nat.ko 2023-07-06T03:25:54.134 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_numgen.ko 2023-07-06T03:25:54.134 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_queue.ko 2023-07-06T03:25:54.134 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_quota.ko 2023-07-06T03:25:54.134 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_redir.ko 2023-07-06T03:25:54.134 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject.ko 2023-07-06T03:25:54.134 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject_inet.ko 2023-07-06T03:25:54.134 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_socket.ko 2023-07-06T03:25:54.135 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_tunnel.ko 2023-07-06T03:25:54.135 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_AUDIT.ko 2023-07-06T03:25:54.135 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-06T03:25:54.135 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-06T03:25:54.135 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-06T03:25:54.135 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CT.ko 2023-07-06T03:25:54.135 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_DSCP.ko 2023-07-06T03:25:54.136 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HL.ko 2023-07-06T03:25:54.136 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HMARK.ko 2023-07-06T03:25:54.136 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-06T03:25:54.136 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LED.ko 2023-07-06T03:25:54.136 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LOG.ko 2023-07-06T03:25:54.136 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-06T03:25:54.136 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NETMAP.ko 2023-07-06T03:25:54.136 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFLOG.ko 2023-07-06T03:25:54.137 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-06T03:25:54.137 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_RATEEST.ko 2023-07-06T03:25:54.137 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-06T03:25:54.137 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_SECMARK.ko 2023-07-06T03:25:54.137 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-06T03:25:54.138 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-06T03:25:54.138 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TEE.ko 2023-07-06T03:25:54.138 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TPROXY.ko 2023-07-06T03:25:54.138 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TRACE.ko 2023-07-06T03:25:54.138 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_addrtype.ko 2023-07-06T03:25:54.139 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_bpf.ko 2023-07-06T03:25:54.139 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cgroup.ko 2023-07-06T03:25:54.139 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cluster.ko 2023-07-06T03:25:54.139 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_comment.ko 2023-07-06T03:25:54.139 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connbytes.ko 2023-07-06T03:25:54.140 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlabel.ko 2023-07-06T03:25:54.140 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlimit.ko 2023-07-06T03:25:54.140 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connmark.ko 2023-07-06T03:25:54.140 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_conntrack.ko 2023-07-06T03:25:54.141 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cpu.ko 2023-07-06T03:25:54.141 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dccp.ko 2023-07-06T03:25:54.141 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_devgroup.ko 2023-07-06T03:25:54.141 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dscp.ko 2023-07-06T03:25:54.142 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ecn.ko 2023-07-06T03:25:54.142 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_esp.ko 2023-07-06T03:25:54.142 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hashlimit.ko 2023-07-06T03:25:54.142 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_helper.ko 2023-07-06T03:25:54.142 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hl.ko 2023-07-06T03:25:54.143 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipcomp.ko 2023-07-06T03:25:54.143 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_iprange.ko 2023-07-06T03:25:54.143 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipvs.ko 2023-07-06T03:25:54.143 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_l2tp.ko 2023-07-06T03:25:54.144 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_length.ko 2023-07-06T03:25:54.144 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_limit.ko 2023-07-06T03:25:54.144 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mac.ko 2023-07-06T03:25:54.144 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mark.ko 2023-07-06T03:25:54.144 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_multiport.ko 2023-07-06T03:25:54.145 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nat.ko 2023-07-06T03:25:54.146 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nfacct.ko 2023-07-06T03:25:54.146 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_osf.ko 2023-07-06T03:25:54.146 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_owner.ko 2023-07-06T03:25:54.146 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_physdev.ko 2023-07-06T03:25:54.146 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_pkttype.ko 2023-07-06T03:25:54.146 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_policy.ko 2023-07-06T03:25:54.147 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_quota.ko 2023-07-06T03:25:54.147 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_rateest.ko 2023-07-06T03:25:54.147 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_realm.ko 2023-07-06T03:25:54.147 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_recent.ko 2023-07-06T03:25:54.147 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_sctp.ko 2023-07-06T03:25:54.147 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_set.ko 2023-07-06T03:25:54.147 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_socket.ko 2023-07-06T03:25:54.147 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_state.ko 2023-07-06T03:25:54.148 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_statistic.ko 2023-07-06T03:25:54.148 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_string.ko 2023-07-06T03:25:54.148 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_tcpmss.ko 2023-07-06T03:25:54.148 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_time.ko 2023-07-06T03:25:54.148 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_u32.ko 2023-07-06T03:25:54.148 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink 2023-07-06T03:25:54.149 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink/netlink_diag.ko 2023-07-06T03:25:54.149 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh 2023-07-06T03:25:54.149 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh/nsh.ko 2023-07-06T03:25:54.149 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch 2023-07-06T03:25:54.149 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/openvswitch.ko 2023-07-06T03:25:54.149 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-geneve.ko 2023-07-06T03:25:54.149 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-gre.ko 2023-07-06T03:25:54.149 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-vxlan.ko 2023-07-06T03:25:54.150 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet 2023-07-06T03:25:54.150 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet/af_packet_diag.ko 2023-07-06T03:25:54.150 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds 2023-07-06T03:25:54.150 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds.ko 2023-07-06T03:25:54.150 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_rdma.ko 2023-07-06T03:25:54.150 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_tcp.ko 2023-07-06T03:25:54.150 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched 2023-07-06T03:25:54.151 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_csum.ko 2023-07-06T03:25:54.151 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_gact.ko 2023-07-06T03:25:54.151 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_ipt.ko 2023-07-06T03:25:54.151 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_mirred.ko 2023-07-06T03:25:54.151 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_nat.ko 2023-07-06T03:25:54.151 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_pedit.ko 2023-07-06T03:25:54.151 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_police.ko 2023-07-06T03:25:54.151 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_simple.ko 2023-07-06T03:25:54.152 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_skbedit.ko 2023-07-06T03:25:54.152 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_vlan.ko 2023-07-06T03:25:54.152 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_basic.ko 2023-07-06T03:25:54.152 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_bpf.ko 2023-07-06T03:25:54.152 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_flow.ko 2023-07-06T03:25:54.152 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_fw.ko 2023-07-06T03:25:54.152 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_route.ko 2023-07-06T03:25:54.153 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_u32.ko 2023-07-06T03:25:54.153 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_cmp.ko 2023-07-06T03:25:54.153 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_ipset.ko 2023-07-06T03:25:54.153 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_meta.ko 2023-07-06T03:25:54.153 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_nbyte.ko 2023-07-06T03:25:54.153 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_text.ko 2023-07-06T03:25:54.153 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_u32.ko 2023-07-06T03:25:54.153 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_choke.ko 2023-07-06T03:25:54.154 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_codel.ko 2023-07-06T03:25:54.154 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_drr.ko 2023-07-06T03:25:54.154 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_fq.ko 2023-07-06T03:25:54.154 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_gred.ko 2023-07-06T03:25:54.154 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hfsc.ko 2023-07-06T03:25:54.154 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hhf.ko 2023-07-06T03:25:54.154 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_htb.ko 2023-07-06T03:25:54.154 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_ingress.ko 2023-07-06T03:25:54.155 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio.ko 2023-07-06T03:25:54.155 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio_lib.ko 2023-07-06T03:25:54.155 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_multiq.ko 2023-07-06T03:25:54.155 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_netem.ko 2023-07-06T03:25:54.155 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_pie.ko 2023-07-06T03:25:54.155 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_plug.ko 2023-07-06T03:25:54.155 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_prio.ko 2023-07-06T03:25:54.156 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_qfq.ko 2023-07-06T03:25:54.156 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_red.ko 2023-07-06T03:25:54.156 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfb.ko 2023-07-06T03:25:54.156 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfq.ko 2023-07-06T03:25:54.156 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_tbf.ko 2023-07-06T03:25:54.156 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_teql.ko 2023-07-06T03:25:54.156 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp 2023-07-06T03:25:54.156 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp.ko 2023-07-06T03:25:54.157 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp_diag.ko 2023-07-06T03:25:54.157 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc 2023-07-06T03:25:54.157 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss 2023-07-06T03:25:54.157 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-06T03:25:54.157 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-06T03:25:54.157 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/sunrpc.ko 2023-07-06T03:25:54.157 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma 2023-07-06T03:25:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-06T03:25:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc 2023-07-06T03:25:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/diag.ko 2023-07-06T03:25:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/tipc.ko 2023-07-06T03:25:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix 2023-07-06T03:25:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix/unix_diag.ko 2023-07-06T03:25:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock 2023-07-06T03:25:54.159 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-06T03:25:54.159 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-06T03:25:54.159 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock.ko 2023-07-06T03:25:54.159 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-06T03:25:54.159 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-06T03:25:54.159 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm 2023-07-06T03:25:54.159 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-06T03:25:54.160 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security 2023-07-06T03:25:54.160 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys 2023-07-06T03:25:54.160 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys 2023-07-06T03:25:54.160 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-06T03:25:54.160 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys 2023-07-06T03:25:54.160 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys/trusted.ko 2023-07-06T03:25:54.160 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt 2023-07-06T03:25:54.160 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib 2023-07-06T03:25:54.161 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib/irqbypass.ko 2023-07-06T03:25:54.161 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias 2023-07-06T03:25:54.161 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias.bin 2023-07-06T03:25:54.161 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin 2023-07-06T03:25:54.161 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.bin 2023-07-06T03:25:54.161 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.modinfo 2023-07-06T03:25:54.161 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep 2023-07-06T03:25:54.161 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep.bin 2023-07-06T03:25:54.162 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.devname 2023-07-06T03:25:54.162 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.order 2023-07-06T03:25:54.162 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.softdep 2023-07-06T03:25:54.162 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols 2023-07-06T03:25:54.162 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols.bin 2023-07-06T03:25:54.163 DEBUG:teuthology.task.kernel:get_image_version: 6.4.0-g84139c18d56f 2023-07-06T03:25:54.163 DEBUG:teuthology.orchestra.run.smithi097:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-06T03:25:54.190 INFO:teuthology.orchestra.run.smithi097.stdout:postinstall scriptlet (using /bin/sh): 2023-07-06T03:25:54.191 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.4.0-g84139c18d56f -a -r /boot/System.map-6.4.0-g84139c18d56f ]; then 2023-07-06T03:25:54.191 INFO:teuthology.orchestra.run.smithi097.stdout:cp /boot/vmlinuz-6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm 2023-07-06T03:25:54.191 INFO:teuthology.orchestra.run.smithi097.stdout:cp /boot/System.map-6.4.0-g84139c18d56f /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T03:25:54.191 INFO:teuthology.orchestra.run.smithi097.stdout:rm -f /boot/vmlinuz-6.4.0-g84139c18d56f /boot/System.map-6.4.0-g84139c18d56f 2023-07-06T03:25:54.191 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/installkernel 6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T03:25:54.191 INFO:teuthology.orchestra.run.smithi097.stdout:rm -f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T03:25:54.192 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2023-07-06T03:25:54.192 INFO:teuthology.orchestra.run.smithi097.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-06T03:25:54.192 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-06T03:25:54.193 INFO:teuthology.orchestra.run.smithi097.stdout:new-kernel-pkg --remove 6.4.0-g84139c18d56f --rminitrd --initrdfile=/boot/initramfs-6.4.0-g84139c18d56f.img 2023-07-06T03:25:54.193 INFO:teuthology.orchestra.run.smithi097.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-06T03:25:54.193 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-install remove 6.4.0-g84139c18d56f 2023-07-06T03:25:54.193 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2023-07-06T03:25:54.194 INFO:teuthology.orchestra.run.smithi097.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-06T03:25:54.194 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-06T03:25:54.194 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/update-bootloader --remove 6.4.0-g84139c18d56f 2023-07-06T03:25:54.194 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2023-07-06T03:25:54.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -qi grub2-tools 2023-07-06T03:25:54.274 INFO:teuthology.orchestra.run.smithi097.stdout:Name : grub2-tools 2023-07-06T03:25:54.274 INFO:teuthology.orchestra.run.smithi097.stdout:Epoch : 1 2023-07-06T03:25:54.274 INFO:teuthology.orchestra.run.smithi097.stdout:Version : 2.02 2023-07-06T03:25:54.274 INFO:teuthology.orchestra.run.smithi097.stdout:Release : 129.el8 2023-07-06T03:25:54.274 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: x86_64 2023-07-06T03:25:54.275 INFO:teuthology.orchestra.run.smithi097.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-07-06T03:25:54.275 INFO:teuthology.orchestra.run.smithi097.stdout:Group : System Environment/Base 2023-07-06T03:25:54.275 INFO:teuthology.orchestra.run.smithi097.stdout:Size : 9265586 2023-07-06T03:25:54.275 INFO:teuthology.orchestra.run.smithi097.stdout:License : GPLv3+ 2023-07-06T03:25:54.275 INFO:teuthology.orchestra.run.smithi097.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-07-06T03:25:54.275 INFO:teuthology.orchestra.run.smithi097.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-07-06T03:25:54.275 INFO:teuthology.orchestra.run.smithi097.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-07-06T03:25:54.276 INFO:teuthology.orchestra.run.smithi097.stdout:Build Host : kbuilder.bsys.centos.org 2023-07-06T03:25:54.276 INFO:teuthology.orchestra.run.smithi097.stdout:Relocations : (not relocatable) 2023-07-06T03:25:54.276 INFO:teuthology.orchestra.run.smithi097.stdout:Packager : CentOS BuildSystem 2023-07-06T03:25:54.276 INFO:teuthology.orchestra.run.smithi097.stdout:Vendor : CentOS 2023-07-06T03:25:54.276 INFO:teuthology.orchestra.run.smithi097.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-06T03:25:54.276 INFO:teuthology.orchestra.run.smithi097.stdout:Summary : Support tools for GRUB. 2023-07-06T03:25:54.276 INFO:teuthology.orchestra.run.smithi097.stdout:Description : 2023-07-06T03:25:54.276 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:25:54.277 INFO:teuthology.orchestra.run.smithi097.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-06T03:25:54.277 INFO:teuthology.orchestra.run.smithi097.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-06T03:25:54.277 INFO:teuthology.orchestra.run.smithi097.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-06T03:25:54.277 INFO:teuthology.orchestra.run.smithi097.stdout:hardware devices. 2023-07-06T03:25:54.277 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:25:54.277 INFO:teuthology.orchestra.run.smithi097.stdout:This subpackage provides tools for support of all platforms. 2023-07-06T03:25:54.278 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-06T03:25:54.279 INFO:teuthology.task.kernel:Updating grub on smithi097 to boot 6.4.0-g84139c18d56f 2023-07-06T03:25:54.279 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-06T03:26:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2023-07-06T03:26:02.720 INFO:teuthology.orchestra.run.smithi097.stderr:done 2023-07-06T03:26:02.722 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2023-07-06T03:26:02.756 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.poqlkizyrK 2023-07-06T03:26:02.757 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.poqlkizyrK 2023-07-06T03:26:02.830 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.poqlkizyrK 2023-07-06T03:26:02.986 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.poqlkizyrK is 6KB 2023-07-06T03:26:03.034 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.poqlkizyrK 2023-07-06T03:26:03.052 DEBUG:teuthology.orchestra.run.smithi097:> sudo /bin/ls /boot/loader/entries || true 2023-07-06T03:26:03.132 INFO:teuthology.orchestra.run.smithi097.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-07-06T03:26:03.132 INFO:teuthology.orchestra.run.smithi097.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-07-06T03:26:03.132 INFO:teuthology.orchestra.run.smithi097.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-07-06T03:26:03.133 INFO:teuthology.orchestra.run.smithi097.stdout:a1b29d6c289747bdb22930598d39874a-6.4.0-g84139c18d56f.conf 2023-07-06T03:26:03.133 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.4.0-g84139c18d56f 2023-07-06T03:26:03.281 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2023-07-06T03:26:07.429 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi146.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-06T03:26:07.430 DEBUG:teuthology.orchestra.run.smithi146:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-06T03:26:07.496 INFO:teuthology.orchestra.run.smithi146.stdout:/boot/System.map-6.4.0-g84139c18d56f 2023-07-06T03:26:07.496 INFO:teuthology.orchestra.run.smithi146.stdout:/boot/config-6.4.0-g84139c18d56f 2023-07-06T03:26:07.496 INFO:teuthology.orchestra.run.smithi146.stdout:/boot/vmlinuz-6.4.0-g84139c18d56f 2023-07-06T03:26:07.496 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f 2023-07-06T03:26:07.497 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel 2023-07-06T03:26:07.497 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch 2023-07-06T03:26:07.497 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86 2023-07-06T03:26:07.497 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto 2023-07-06T03:26:07.497 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-06T03:26:07.497 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-06T03:26:07.497 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-06T03:26:07.498 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-06T03:26:07.498 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-06T03:26:07.498 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-06T03:26:07.498 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-06T03:26:07.498 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-06T03:26:07.498 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-06T03:26:07.499 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-06T03:26:07.499 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-06T03:26:07.499 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-06T03:26:07.499 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-06T03:26:07.499 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-06T03:26:07.499 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-06T03:26:07.499 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-06T03:26:07.500 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-06T03:26:07.500 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-06T03:26:07.500 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-06T03:26:07.500 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-06T03:26:07.500 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel 2023-07-06T03:26:07.500 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu 2023-07-06T03:26:07.500 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce 2023-07-06T03:26:07.501 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-06T03:26:07.501 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm 2023-07-06T03:26:07.501 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-06T03:26:07.501 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-06T03:26:07.501 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm.ko 2023-07-06T03:26:07.501 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto 2023-07-06T03:26:07.501 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ansi_cprng.ko 2023-07-06T03:26:07.502 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/anubis.ko 2023-07-06T03:26:07.502 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/arc4.ko 2023-07-06T03:26:07.502 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx 2023-07-06T03:26:07.502 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_memcpy.ko 2023-07-06T03:26:07.502 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_pq.ko 2023-07-06T03:26:07.502 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-06T03:26:07.502 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_tx.ko 2023-07-06T03:26:07.503 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_xor.ko 2023-07-06T03:26:07.503 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/raid6test.ko 2023-07-06T03:26:07.503 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authenc.ko 2023-07-06T03:26:07.503 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authencesn.ko 2023-07-06T03:26:07.503 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blake2b_generic.ko 2023-07-06T03:26:07.503 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_common.ko 2023-07-06T03:26:07.503 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_generic.ko 2023-07-06T03:26:07.504 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/camellia_generic.ko 2023-07-06T03:26:07.504 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast5_generic.ko 2023-07-06T03:26:07.504 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast6_generic.ko 2023-07-06T03:26:07.504 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast_common.ko 2023-07-06T03:26:07.504 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ccm.ko 2023-07-06T03:26:07.504 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cmac.ko 2023-07-06T03:26:07.504 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crc32_generic.ko 2023-07-06T03:26:07.505 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_engine.ko 2023-07-06T03:26:07.505 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_user.ko 2023-07-06T03:26:07.505 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cts.ko 2023-07-06T03:26:07.505 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/des_generic.ko 2023-07-06T03:26:07.505 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/dh_generic.ko 2023-07-06T03:26:07.505 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/echainiv.ko 2023-07-06T03:26:07.505 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/essiv.ko 2023-07-06T03:26:07.506 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/fcrypt.ko 2023-07-06T03:26:07.506 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/khazad.ko 2023-07-06T03:26:07.506 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4.ko 2023-07-06T03:26:07.506 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4hc.ko 2023-07-06T03:26:07.506 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/md4.ko 2023-07-06T03:26:07.506 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/michael_mic.ko 2023-07-06T03:26:07.507 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcbc.ko 2023-07-06T03:26:07.507 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcrypt.ko 2023-07-06T03:26:07.507 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/rmd160.ko 2023-07-06T03:26:07.507 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/seed.ko 2023-07-06T03:26:07.507 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/serpent_generic.ko 2023-07-06T03:26:07.507 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tcrypt.ko 2023-07-06T03:26:07.507 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tea.ko 2023-07-06T03:26:07.507 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_common.ko 2023-07-06T03:26:07.508 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_generic.ko 2023-07-06T03:26:07.508 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/vmac.ko 2023-07-06T03:26:07.508 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/wp512.ko 2023-07-06T03:26:07.508 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xcbc.ko 2023-07-06T03:26:07.508 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xor.ko 2023-07-06T03:26:07.508 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xxhash_generic.ko 2023-07-06T03:26:07.509 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers 2023-07-06T03:26:07.509 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi 2023-07-06T03:26:07.509 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-06T03:26:07.509 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_pad.ko 2023-07-06T03:26:07.509 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/custom_method.ko 2023-07-06T03:26:07.509 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/ec_sys.ko 2023-07-06T03:26:07.509 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/platform_profile.ko 2023-07-06T03:26:07.510 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbs.ko 2023-07-06T03:26:07.510 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbshc.ko 2023-07-06T03:26:07.510 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/video.ko 2023-07-06T03:26:07.510 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata 2023-07-06T03:26:07.510 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/acard-ahci.ko 2023-07-06T03:26:07.510 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ahci_platform.ko 2023-07-06T03:26:07.510 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ata_generic.ko 2023-07-06T03:26:07.510 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/libahci_platform.ko 2023-07-06T03:26:07.511 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_acpi.ko 2023-07-06T03:26:07.511 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ali.ko 2023-07-06T03:26:07.511 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_amd.ko 2023-07-06T03:26:07.511 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_artop.ko 2023-07-06T03:26:07.511 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atiixp.ko 2023-07-06T03:26:07.511 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atp867x.ko 2023-07-06T03:26:07.511 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd640.ko 2023-07-06T03:26:07.512 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd64x.ko 2023-07-06T03:26:07.512 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cypress.ko 2023-07-06T03:26:07.512 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_efar.ko 2023-07-06T03:26:07.512 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt366.ko 2023-07-06T03:26:07.512 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt37x.ko 2023-07-06T03:26:07.512 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-06T03:26:07.512 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-06T03:26:07.513 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it8213.ko 2023-07-06T03:26:07.513 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it821x.ko 2023-07-06T03:26:07.513 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_jmicron.ko 2023-07-06T03:26:07.513 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_marvell.ko 2023-07-06T03:26:07.513 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_mpiix.ko 2023-07-06T03:26:07.513 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_netcell.ko 2023-07-06T03:26:07.513 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ninja32.ko 2023-07-06T03:26:07.514 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87410.ko 2023-07-06T03:26:07.514 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87415.ko 2023-07-06T03:26:07.514 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_oldpiix.ko 2023-07-06T03:26:07.514 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_opti.ko 2023-07-06T03:26:07.514 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_optidma.ko 2023-07-06T03:26:07.514 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pcmcia.ko 2023-07-06T03:26:07.514 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-06T03:26:07.515 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-06T03:26:07.515 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_piccolo.ko 2023-07-06T03:26:07.515 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_rdc.ko 2023-07-06T03:26:07.515 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sch.ko 2023-07-06T03:26:07.515 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_serverworks.ko 2023-07-06T03:26:07.515 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sil680.ko 2023-07-06T03:26:07.516 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sis.ko 2023-07-06T03:26:07.516 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sl82c105.ko 2023-07-06T03:26:07.516 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_triflex.ko 2023-07-06T03:26:07.516 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_via.ko 2023-07-06T03:26:07.516 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pdc_adma.ko 2023-07-06T03:26:07.516 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_inic162x.ko 2023-07-06T03:26:07.517 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_mv.ko 2023-07-06T03:26:07.517 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_nv.ko 2023-07-06T03:26:07.517 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_promise.ko 2023-07-06T03:26:07.517 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_qstor.ko 2023-07-06T03:26:07.517 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil.ko 2023-07-06T03:26:07.517 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil24.ko 2023-07-06T03:26:07.517 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sis.ko 2023-07-06T03:26:07.517 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_svw.ko 2023-07-06T03:26:07.518 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sx4.ko 2023-07-06T03:26:07.518 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_uli.ko 2023-07-06T03:26:07.518 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_via.ko 2023-07-06T03:26:07.518 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_vsc.ko 2023-07-06T03:26:07.518 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay 2023-07-06T03:26:07.518 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay/ks0108.ko 2023-07-06T03:26:07.518 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base 2023-07-06T03:26:07.519 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap 2023-07-06T03:26:07.519 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-06T03:26:07.519 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma 2023-07-06T03:26:07.519 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma/bcma.ko 2023-07-06T03:26:07.519 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block 2023-07-06T03:26:07.519 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe 2023-07-06T03:26:07.519 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe/aoe.ko 2023-07-06T03:26:07.520 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/brd.ko 2023-07-06T03:26:07.520 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd 2023-07-06T03:26:07.520 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd/drbd.ko 2023-07-06T03:26:07.520 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/floppy.ko 2023-07-06T03:26:07.520 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/loop.ko 2023-07-06T03:26:07.520 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx 2023-07-06T03:26:07.520 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-06T03:26:07.521 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/nbd.ko 2023-07-06T03:26:07.521 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk 2023-07-06T03:26:07.521 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk/null_blk.ko 2023-07-06T03:26:07.521 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/pktcdvd.ko 2023-07-06T03:26:07.521 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/rbd.ko 2023-07-06T03:26:07.521 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/virtio_blk.ko 2023-07-06T03:26:07.521 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback 2023-07-06T03:26:07.521 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-06T03:26:07.522 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkfront.ko 2023-07-06T03:26:07.522 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram 2023-07-06T03:26:07.522 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram/zram.ko 2023-07-06T03:26:07.522 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char 2023-07-06T03:26:07.522 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hangcheck-timer.ko 2023-07-06T03:26:07.522 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random 2023-07-06T03:26:07.523 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-06T03:26:07.523 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-06T03:26:07.523 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-06T03:26:07.523 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/via-rng.ko 2023-07-06T03:26:07.524 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-06T03:26:07.524 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi 2023-07-06T03:26:07.524 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-06T03:26:07.524 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-06T03:26:07.524 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-06T03:26:07.525 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-06T03:26:07.525 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-06T03:26:07.525 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-06T03:26:07.525 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/lp.ko 2023-07-06T03:26:07.525 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave 2023-07-06T03:26:07.525 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave/mwave.ko 2023-07-06T03:26:07.525 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ppdev.ko 2023-07-06T03:26:07.526 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tlclk.ko 2023-07-06T03:26:07.526 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm 2023-07-06T03:26:07.526 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm.ko 2023-07-06T03:26:07.526 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-06T03:26:07.526 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-06T03:26:07.526 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-06T03:26:07.526 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-06T03:26:07.527 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-06T03:26:07.527 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/uv_mmtimer.ko 2023-07-06T03:26:07.527 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/virtio_console.ko 2023-07-06T03:26:07.527 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq 2023-07-06T03:26:07.527 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-06T03:26:07.527 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-06T03:26:07.527 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-06T03:26:07.528 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-06T03:26:07.528 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-06T03:26:07.528 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-06T03:26:07.528 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto 2023-07-06T03:26:07.528 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp 2023-07-06T03:26:07.528 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-06T03:26:07.528 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp.ko 2023-07-06T03:26:07.529 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel 2023-07-06T03:26:07.529 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat 2023-07-06T03:26:07.529 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common 2023-07-06T03:26:07.529 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-06T03:26:07.529 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-06T03:26:07.529 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-06T03:26:07.529 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-aes.ko 2023-07-06T03:26:07.530 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-sha.ko 2023-07-06T03:26:07.530 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio 2023-07-06T03:26:07.530 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-06T03:26:07.530 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca 2023-07-06T03:26:07.530 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca/dca.ko 2023-07-06T03:26:07.530 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq 2023-07-06T03:26:07.530 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-06T03:26:07.531 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma 2023-07-06T03:26:07.531 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw 2023-07-06T03:26:07.531 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-06T03:26:07.531 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat 2023-07-06T03:26:07.531 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-06T03:26:07.531 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac 2023-07-06T03:26:07.531 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/amd64_edac.ko 2023-07-06T03:26:07.531 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/e752x_edac.ko 2023-07-06T03:26:07.532 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/edac_mce_amd.ko 2023-07-06T03:26:07.532 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3000_edac.ko 2023-07-06T03:26:07.532 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3200_edac.ko 2023-07-06T03:26:07.532 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5100_edac.ko 2023-07-06T03:26:07.532 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5400_edac.ko 2023-07-06T03:26:07.532 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7300_edac.ko 2023-07-06T03:26:07.532 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7core_edac.ko 2023-07-06T03:26:07.533 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i82975x_edac.ko 2023-07-06T03:26:07.533 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/ie31200_edac.ko 2023-07-06T03:26:07.533 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/sb_edac.ko 2023-07-06T03:26:07.533 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/x38_edac.ko 2023-07-06T03:26:07.533 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware 2023-07-06T03:26:07.533 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/edd.ko 2023-07-06T03:26:07.533 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-06T03:26:07.534 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio 2023-07-06T03:26:07.534 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-ich.ko 2023-07-06T03:26:07.534 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-06T03:26:07.534 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid 2023-07-06T03:26:07.534 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-appleir.ko 2023-07-06T03:26:07.534 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-aureal.ko 2023-07-06T03:26:07.534 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-axff.ko 2023-07-06T03:26:07.535 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-dr.ko 2023-07-06T03:26:07.535 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elecom.ko 2023-07-06T03:26:07.535 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elo.ko 2023-07-06T03:26:07.535 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-emsff.ko 2023-07-06T03:26:07.535 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gaff.ko 2023-07-06T03:26:07.535 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gt683r.ko 2023-07-06T03:26:07.535 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gyration.ko 2023-07-06T03:26:07.535 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-06T03:26:07.536 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-06T03:26:07.536 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtekff.ko 2023-07-06T03:26:07.536 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-hyperv.ko 2023-07-06T03:26:07.536 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-icade.ko 2023-07-06T03:26:07.536 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-keytouch.ko 2023-07-06T03:26:07.536 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-kye.ko 2023-07-06T03:26:07.536 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lcpower.ko 2023-07-06T03:26:07.537 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-led.ko 2023-07-06T03:26:07.537 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lenovo.ko 2023-07-06T03:26:07.537 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-06T03:26:07.537 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-06T03:26:07.537 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-multitouch.ko 2023-07-06T03:26:07.537 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-ortek.ko 2023-07-06T03:26:07.537 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-penmount.ko 2023-07-06T03:26:07.538 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-petalynx.ko 2023-07-06T03:26:07.538 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-picolcd.ko 2023-07-06T03:26:07.538 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-pl.ko 2023-07-06T03:26:07.538 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-plantronics.ko 2023-07-06T03:26:07.538 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-primax.ko 2023-07-06T03:26:07.538 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-rmi.ko 2023-07-06T03:26:07.538 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-06T03:26:07.538 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-common.ko 2023-07-06T03:26:07.539 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-06T03:26:07.539 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-06T03:26:07.539 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-06T03:26:07.539 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-06T03:26:07.539 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-06T03:26:07.539 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-06T03:26:07.539 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-06T03:26:07.539 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-06T03:26:07.540 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-06T03:26:07.540 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat.ko 2023-07-06T03:26:07.540 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-saitek.ko 2023-07-06T03:26:07.540 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-samsung.ko 2023-07-06T03:26:07.540 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-06T03:26:07.540 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sjoy.ko 2023-07-06T03:26:07.540 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sony.ko 2023-07-06T03:26:07.541 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-speedlink.ko 2023-07-06T03:26:07.541 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-steelseries.ko 2023-07-06T03:26:07.541 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sunplus.ko 2023-07-06T03:26:07.541 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-06T03:26:07.541 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tivo.ko 2023-07-06T03:26:07.541 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tmff.ko 2023-07-06T03:26:07.541 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-topseed.ko 2023-07-06T03:26:07.541 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-twinhan.ko 2023-07-06T03:26:07.542 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-uclogic.ko 2023-07-06T03:26:07.542 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-waltop.ko 2023-07-06T03:26:07.542 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-wiimote.ko 2023-07-06T03:26:07.542 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-xinmo.ko 2023-07-06T03:26:07.542 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zpff.ko 2023-07-06T03:26:07.542 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zydacron.ko 2023-07-06T03:26:07.542 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/uhid.ko 2023-07-06T03:26:07.543 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/wacom.ko 2023-07-06T03:26:07.543 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv 2023-07-06T03:26:07.543 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_balloon.ko 2023-07-06T03:26:07.543 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_utils.ko 2023-07-06T03:26:07.543 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_vmbus.ko 2023-07-06T03:26:07.543 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon 2023-07-06T03:26:07.543 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru.ko 2023-07-06T03:26:07.543 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru3.ko 2023-07-06T03:26:07.544 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-06T03:26:07.544 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7414.ko 2023-07-06T03:26:07.544 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7418.ko 2023-07-06T03:26:07.544 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adc128d818.ko 2023-07-06T03:26:07.544 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1025.ko 2023-07-06T03:26:07.544 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1026.ko 2023-07-06T03:26:07.544 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1029.ko 2023-07-06T03:26:07.544 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1031.ko 2023-07-06T03:26:07.545 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm9240.ko 2023-07-06T03:26:07.545 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ads7828.ko 2023-07-06T03:26:07.545 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7410.ko 2023-07-06T03:26:07.545 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7411.ko 2023-07-06T03:26:07.545 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7462.ko 2023-07-06T03:26:07.546 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7470.ko 2023-07-06T03:26:07.546 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7475.ko 2023-07-06T03:26:07.546 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7x10.ko 2023-07-06T03:26:07.546 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/amc6821.ko 2023-07-06T03:26:07.547 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/applesmc.ko 2023-07-06T03:26:07.547 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asb100.ko 2023-07-06T03:26:07.547 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asc7621.ko 2023-07-06T03:26:07.547 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-06T03:26:07.547 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/atxp1.ko 2023-07-06T03:26:07.547 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/coretemp.ko 2023-07-06T03:26:07.548 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-06T03:26:07.548 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dme1737.ko 2023-07-06T03:26:07.548 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds1621.ko 2023-07-06T03:26:07.548 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds620.ko 2023-07-06T03:26:07.548 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc1403.ko 2023-07-06T03:26:07.548 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc6w201.ko 2023-07-06T03:26:07.548 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71805f.ko 2023-07-06T03:26:07.548 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71882fg.ko 2023-07-06T03:26:07.549 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f75375s.ko 2023-07-06T03:26:07.549 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fam15h_power.ko 2023-07-06T03:26:07.549 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fschmd.ko 2023-07-06T03:26:07.549 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g760a.ko 2023-07-06T03:26:07.549 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g762.ko 2023-07-06T03:26:07.549 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl518sm.ko 2023-07-06T03:26:07.549 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl520sm.ko 2023-07-06T03:26:07.550 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-06T03:26:07.550 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5500_temp.ko 2023-07-06T03:26:07.550 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5k_amb.ko 2023-07-06T03:26:07.550 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmaem.ko 2023-07-06T03:26:07.550 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmpex.ko 2023-07-06T03:26:07.550 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina209.ko 2023-07-06T03:26:07.550 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina2xx.ko 2023-07-06T03:26:07.550 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/it87.ko 2023-07-06T03:26:07.551 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k10temp.ko 2023-07-06T03:26:07.551 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k8temp.ko 2023-07-06T03:26:07.551 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lineage-pem.ko 2023-07-06T03:26:07.551 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm63.ko 2023-07-06T03:26:07.551 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm73.ko 2023-07-06T03:26:07.551 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm75.ko 2023-07-06T03:26:07.551 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm77.ko 2023-07-06T03:26:07.552 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm78.ko 2023-07-06T03:26:07.552 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm80.ko 2023-07-06T03:26:07.552 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm83.ko 2023-07-06T03:26:07.552 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm85.ko 2023-07-06T03:26:07.552 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm87.ko 2023-07-06T03:26:07.552 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm90.ko 2023-07-06T03:26:07.552 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm92.ko 2023-07-06T03:26:07.553 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm93.ko 2023-07-06T03:26:07.553 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95234.ko 2023-07-06T03:26:07.553 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95241.ko 2023-07-06T03:26:07.553 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95245.ko 2023-07-06T03:26:07.553 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc2945.ko 2023-07-06T03:26:07.554 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4151.ko 2023-07-06T03:26:07.554 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4215.ko 2023-07-06T03:26:07.554 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4222.ko 2023-07-06T03:26:07.554 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4245.ko 2023-07-06T03:26:07.554 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4260.ko 2023-07-06T03:26:07.554 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4261.ko 2023-07-06T03:26:07.555 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max16065.ko 2023-07-06T03:26:07.555 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1619.ko 2023-07-06T03:26:07.555 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1668.ko 2023-07-06T03:26:07.555 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max197.ko 2023-07-06T03:26:07.556 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6639.ko 2023-07-06T03:26:07.556 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6650.ko 2023-07-06T03:26:07.556 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6697.ko 2023-07-06T03:26:07.556 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/mcp3021.ko 2023-07-06T03:26:07.557 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6683.ko 2023-07-06T03:26:07.557 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775-core.ko 2023-07-06T03:26:07.557 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775.ko 2023-07-06T03:26:07.557 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct7802.ko 2023-07-06T03:26:07.557 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87360.ko 2023-07-06T03:26:07.558 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87427.ko 2023-07-06T03:26:07.558 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pcf8591.ko 2023-07-06T03:26:07.558 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus 2023-07-06T03:26:07.558 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-06T03:26:07.558 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-06T03:26:07.558 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-06T03:26:07.559 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-06T03:26:07.559 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-06T03:26:07.559 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-06T03:26:07.559 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-06T03:26:07.560 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-06T03:26:07.560 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-06T03:26:07.560 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-06T03:26:07.560 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-06T03:26:07.560 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-06T03:26:07.560 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/powr1220.ko 2023-07-06T03:26:07.560 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5627.ko 2023-07-06T03:26:07.561 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5636.ko 2023-07-06T03:26:07.561 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-06T03:26:07.561 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht15.ko 2023-07-06T03:26:07.561 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht21.ko 2023-07-06T03:26:07.561 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/shtc1.ko 2023-07-06T03:26:07.561 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sis5595.ko 2023-07-06T03:26:07.561 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47b397.ko 2023-07-06T03:26:07.561 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m1.ko 2023-07-06T03:26:07.562 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m192.ko 2023-07-06T03:26:07.562 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/thmc50.ko 2023-07-06T03:26:07.562 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp102.ko 2023-07-06T03:26:07.562 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp103.ko 2023-07-06T03:26:07.562 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp401.ko 2023-07-06T03:26:07.562 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp421.ko 2023-07-06T03:26:07.562 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via-cputemp.ko 2023-07-06T03:26:07.562 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via686a.ko 2023-07-06T03:26:07.563 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt1211.ko 2023-07-06T03:26:07.563 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt8231.ko 2023-07-06T03:26:07.563 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627ehf.ko 2023-07-06T03:26:07.563 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627hf.ko 2023-07-06T03:26:07.563 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83781d.ko 2023-07-06T03:26:07.563 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83791d.ko 2023-07-06T03:26:07.563 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83792d.ko 2023-07-06T03:26:07.564 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83793.ko 2023-07-06T03:26:07.564 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83795.ko 2023-07-06T03:26:07.564 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l785ts.ko 2023-07-06T03:26:07.564 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l786ng.ko 2023-07-06T03:26:07.564 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c 2023-07-06T03:26:07.564 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos 2023-07-06T03:26:07.564 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-06T03:26:07.564 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-06T03:26:07.565 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses 2023-07-06T03:26:07.565 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-06T03:26:07.565 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-06T03:26:07.565 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-06T03:26:07.565 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-06T03:26:07.565 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-06T03:26:07.565 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-06T03:26:07.566 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-06T03:26:07.566 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-06T03:26:07.566 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-06T03:26:07.566 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-06T03:26:07.566 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-06T03:26:07.566 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-06T03:26:07.567 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-06T03:26:07.567 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-06T03:26:07.567 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-06T03:26:07.567 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-06T03:26:07.567 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-06T03:26:07.568 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-06T03:26:07.568 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-06T03:26:07.568 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-06T03:26:07.568 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-06T03:26:07.568 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-06T03:26:07.568 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-06T03:26:07.568 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-dev.ko 2023-07-06T03:26:07.569 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-mux.ko 2023-07-06T03:26:07.569 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-06T03:26:07.569 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-smbus.ko 2023-07-06T03:26:07.569 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-stub.ko 2023-07-06T03:26:07.569 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband 2023-07-06T03:26:07.569 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core 2023-07-06T03:26:07.569 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-06T03:26:07.569 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_core.ko 2023-07-06T03:26:07.570 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-06T03:26:07.570 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-06T03:26:07.570 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-06T03:26:07.570 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-06T03:26:07.570 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-06T03:26:07.570 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw 2023-07-06T03:26:07.571 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4 2023-07-06T03:26:07.571 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-06T03:26:07.571 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4 2023-07-06T03:26:07.571 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-06T03:26:07.571 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5 2023-07-06T03:26:07.571 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-06T03:26:07.571 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca 2023-07-06T03:26:07.571 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-06T03:26:07.572 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp 2023-07-06T03:26:07.572 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib 2023-07-06T03:26:07.572 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-06T03:26:07.572 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser 2023-07-06T03:26:07.572 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-06T03:26:07.572 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert 2023-07-06T03:26:07.572 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-06T03:26:07.573 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp 2023-07-06T03:26:07.573 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-06T03:26:07.573 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt 2023-07-06T03:26:07.573 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-06T03:26:07.573 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input 2023-07-06T03:26:07.573 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport 2023-07-06T03:26:07.573 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-06T03:26:07.573 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-06T03:26:07.574 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/gameport.ko 2023-07-06T03:26:07.574 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/lightning.ko 2023-07-06T03:26:07.574 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/ns558.ko 2023-07-06T03:26:07.574 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/joydev.ko 2023-07-06T03:26:07.574 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard 2023-07-06T03:26:07.575 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-06T03:26:07.575 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse 2023-07-06T03:26:07.575 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/appletouch.ko 2023-07-06T03:26:07.575 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/bcm5974.ko 2023-07-06T03:26:07.575 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/cyapatp.ko 2023-07-06T03:26:07.575 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-06T03:26:07.575 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/sermouse.ko 2023-07-06T03:26:07.575 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-06T03:26:07.576 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-06T03:26:07.576 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-06T03:26:07.576 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4 2023-07-06T03:26:07.576 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-06T03:26:07.576 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio 2023-07-06T03:26:07.576 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/altera_ps2.ko 2023-07-06T03:26:07.576 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/arc_ps2.ko 2023-07-06T03:26:07.577 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-06T03:26:07.577 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/serio_raw.ko 2023-07-06T03:26:07.577 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/sparse-keymap.ko 2023-07-06T03:26:07.577 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu 2023-07-06T03:26:07.577 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd 2023-07-06T03:26:07.577 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-06T03:26:07.577 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds 2023-07-06T03:26:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-blinkm.ko 2023-07-06T03:26:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lm3530.ko 2023-07-06T03:26:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lp3944.ko 2023-07-06T03:26:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-ss4200.ko 2023-07-06T03:26:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger 2023-07-06T03:26:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-06T03:26:07.578 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-06T03:26:07.579 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-06T03:26:07.579 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-06T03:26:07.579 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-06T03:26:07.579 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-06T03:26:07.579 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-06T03:26:07.579 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-06T03:26:07.579 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md 2023-07-06T03:26:07.579 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache 2023-07-06T03:26:07.580 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache/bcache.ko 2023-07-06T03:26:07.580 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-bio-prison.ko 2023-07-06T03:26:07.580 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache-smq.ko 2023-07-06T03:26:07.580 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache.ko 2023-07-06T03:26:07.580 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-crypt.ko 2023-07-06T03:26:07.580 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-delay.ko 2023-07-06T03:26:07.580 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-flakey.ko 2023-07-06T03:26:07.580 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-log-userspace.ko 2023-07-06T03:26:07.581 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-multipath.ko 2023-07-06T03:26:07.581 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-queue-length.ko 2023-07-06T03:26:07.581 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-raid.ko 2023-07-06T03:26:07.581 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-round-robin.ko 2023-07-06T03:26:07.581 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-service-time.ko 2023-07-06T03:26:07.581 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-switch.ko 2023-07-06T03:26:07.582 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-thin-pool.ko 2023-07-06T03:26:07.582 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-verity.ko 2023-07-06T03:26:07.582 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/faulty.ko 2023-07-06T03:26:07.582 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/linear.ko 2023-07-06T03:26:07.582 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/multipath.ko 2023-07-06T03:26:07.582 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data 2023-07-06T03:26:07.582 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-06T03:26:07.582 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid0.ko 2023-07-06T03:26:07.583 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid1.ko 2023-07-06T03:26:07.583 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid10.ko 2023-07-06T03:26:07.583 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid456.ko 2023-07-06T03:26:07.583 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message 2023-07-06T03:26:07.583 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion 2023-07-06T03:26:07.583 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptbase.ko 2023-07-06T03:26:07.584 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptctl.ko 2023-07-06T03:26:07.584 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptfc.ko 2023-07-06T03:26:07.584 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptlan.ko 2023-07-06T03:26:07.584 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptsas.ko 2023-07-06T03:26:07.584 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptscsih.ko 2023-07-06T03:26:07.584 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptspi.ko 2023-07-06T03:26:07.584 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd 2023-07-06T03:26:07.584 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_ich.ko 2023-07-06T03:26:07.585 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_sch.ko 2023-07-06T03:26:07.585 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/mfd-core.ko 2023-07-06T03:26:07.585 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/sm501.ko 2023-07-06T03:26:07.585 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/viperboard.ko 2023-07-06T03:26:07.585 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/vx855.ko 2023-07-06T03:26:07.585 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/wl1273-core.ko 2023-07-06T03:26:07.585 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc 2023-07-06T03:26:07.585 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl 2023-07-06T03:26:07.586 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-06T03:26:07.586 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds9802als.ko 2023-07-06T03:26:07.586 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds990x.ko 2023-07-06T03:26:07.586 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/bh1770glc.ko 2023-07-06T03:26:07.586 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710 2023-07-06T03:26:07.586 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710/cb710.ko 2023-07-06T03:26:07.586 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom 2023-07-06T03:26:07.587 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/at24.ko 2023-07-06T03:26:07.587 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-06T03:26:07.587 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-06T03:26:07.587 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/max6875.ko 2023-07-06T03:26:07.587 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/enclosure.ko 2023-07-06T03:26:07.587 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/hpilo.ko 2023-07-06T03:26:07.587 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29003.ko 2023-07-06T03:26:07.588 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29020.ko 2023-07-06T03:26:07.588 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d 2023-07-06T03:26:07.588 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-06T03:26:07.588 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-06T03:26:07.588 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei 2023-07-06T03:26:07.588 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-me.ko 2023-07-06T03:26:07.588 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-txe.ko 2023-07-06T03:26:07.589 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei.ko 2023-07-06T03:26:07.589 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru 2023-07-06T03:26:07.589 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-06T03:26:07.589 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp 2023-07-06T03:26:07.589 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-06T03:26:07.589 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-06T03:26:07.589 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-06T03:26:07.589 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_7xx1.ko 2023-07-06T03:26:07.590 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_core.ko 2023-07-06T03:26:07.590 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tsl2550.ko 2023-07-06T03:26:07.590 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_balloon.ko 2023-07-06T03:26:07.590 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci 2023-07-06T03:26:07.590 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-06T03:26:07.590 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net 2023-07-06T03:26:07.590 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding 2023-07-06T03:26:07.590 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding/bonding.ko 2023-07-06T03:26:07.591 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa 2023-07-06T03:26:07.591 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53 2023-07-06T03:26:07.591 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-06T03:26:07.591 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-06T03:26:07.591 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-06T03:26:07.591 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx 2023-07-06T03:26:07.591 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-06T03:26:07.591 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dummy.ko 2023-07-06T03:26:07.592 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/eql.ko 2023-07-06T03:26:07.592 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet 2023-07-06T03:26:07.592 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com 2023-07-06T03:26:07.592 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-06T03:26:07.592 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-06T03:26:07.592 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-06T03:26:07.592 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-06T03:26:07.593 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390 2023-07-06T03:26:07.593 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-06T03:26:07.593 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-06T03:26:07.593 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-06T03:26:07.594 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-06T03:26:07.594 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec 2023-07-06T03:26:07.594 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-06T03:26:07.594 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere 2023-07-06T03:26:07.594 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-06T03:26:07.594 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon 2023-07-06T03:26:07.595 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-06T03:26:07.595 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera 2023-07-06T03:26:07.595 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-06T03:26:07.595 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd 2023-07-06T03:26:07.595 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-06T03:26:07.595 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-06T03:26:07.595 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-06T03:26:07.596 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros 2023-07-06T03:26:07.596 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx 2023-07-06T03:26:07.596 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-06T03:26:07.596 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-06T03:26:07.596 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-06T03:26:07.596 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-06T03:26:07.596 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-06T03:26:07.597 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx 2023-07-06T03:26:07.597 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-06T03:26:07.597 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-06T03:26:07.597 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom 2023-07-06T03:26:07.597 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-06T03:26:07.597 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-06T03:26:07.597 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-06T03:26:07.597 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-06T03:26:07.598 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-06T03:26:07.598 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet 2023-07-06T03:26:07.598 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-06T03:26:07.598 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-06T03:26:07.598 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade 2023-07-06T03:26:07.598 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna 2023-07-06T03:26:07.598 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-06T03:26:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio 2023-07-06T03:26:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-06T03:26:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-06T03:26:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-06T03:26:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-06T03:26:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-06T03:26:07.599 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-06T03:26:07.600 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-06T03:26:07.600 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-06T03:26:07.600 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-06T03:26:07.600 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-06T03:26:07.600 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco 2023-07-06T03:26:07.600 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic 2023-07-06T03:26:07.600 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-06T03:26:07.600 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec 2023-07-06T03:26:07.601 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip 2023-07-06T03:26:07.601 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-06T03:26:07.601 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-06T03:26:07.601 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-06T03:26:07.601 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-06T03:26:07.601 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-06T03:26:07.601 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-06T03:26:07.602 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink 2023-07-06T03:26:07.602 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-06T03:26:07.602 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-06T03:26:07.602 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dnet.ko 2023-07-06T03:26:07.602 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex 2023-07-06T03:26:07.602 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet 2023-07-06T03:26:07.602 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-06T03:26:07.602 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ethoc.ko 2023-07-06T03:26:07.603 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/fealnx.ko 2023-07-06T03:26:07.603 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel 2023-07-06T03:26:07.603 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-06T03:26:07.603 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000 2023-07-06T03:26:07.603 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-06T03:26:07.603 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e 2023-07-06T03:26:07.603 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-06T03:26:07.604 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k 2023-07-06T03:26:07.604 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-06T03:26:07.604 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e 2023-07-06T03:26:07.604 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-06T03:26:07.604 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf 2023-07-06T03:26:07.604 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-06T03:26:07.604 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb 2023-07-06T03:26:07.604 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-06T03:26:07.605 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf 2023-07-06T03:26:07.605 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-06T03:26:07.605 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-06T03:26:07.605 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-06T03:26:07.605 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-06T03:26:07.605 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-06T03:26:07.605 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/jme.ko 2023-07-06T03:26:07.605 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell 2023-07-06T03:26:07.606 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-06T03:26:07.606 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-06T03:26:07.606 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-06T03:26:07.606 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox 2023-07-06T03:26:07.606 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-06T03:26:07.607 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-06T03:26:07.607 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-06T03:26:07.607 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-06T03:26:07.607 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-06T03:26:07.607 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-06T03:26:07.607 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel 2023-07-06T03:26:07.607 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-06T03:26:07.607 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom 2023-07-06T03:26:07.608 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-06T03:26:07.608 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-06T03:26:07.608 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi 2023-07-06T03:26:07.608 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-06T03:26:07.608 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-06T03:26:07.608 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion 2023-07-06T03:26:07.609 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-06T03:26:07.609 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia 2023-07-06T03:26:07.609 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-06T03:26:07.609 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines 2023-07-06T03:26:07.609 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-06T03:26:07.609 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-06T03:26:07.610 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic 2023-07-06T03:26:07.610 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-06T03:26:07.610 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-06T03:26:07.610 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-06T03:26:07.610 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-06T03:26:07.610 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-06T03:26:07.611 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc 2023-07-06T03:26:07.611 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-06T03:26:07.611 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek 2023-07-06T03:26:07.611 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-06T03:26:07.611 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-06T03:26:07.611 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-06T03:26:07.612 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-06T03:26:07.612 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker 2023-07-06T03:26:07.612 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-06T03:26:07.612 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc 2023-07-06T03:26:07.612 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-06T03:26:07.613 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan 2023-07-06T03:26:07.613 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-06T03:26:07.613 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis 2023-07-06T03:26:07.613 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-06T03:26:07.613 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-06T03:26:07.614 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc 2023-07-06T03:26:07.614 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-06T03:26:07.614 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-06T03:26:07.614 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-06T03:26:07.615 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-06T03:26:07.615 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro 2023-07-06T03:26:07.615 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-06T03:26:07.615 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-06T03:26:07.615 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-06T03:26:07.616 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun 2023-07-06T03:26:07.616 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-06T03:26:07.616 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-06T03:26:07.616 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-06T03:26:07.616 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-06T03:26:07.617 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti 2023-07-06T03:26:07.617 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-06T03:26:07.617 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti 2023-07-06T03:26:07.617 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-06T03:26:07.617 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via 2023-07-06T03:26:07.617 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-06T03:26:07.618 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-06T03:26:07.618 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet 2023-07-06T03:26:07.618 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-06T03:26:07.618 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-06T03:26:07.618 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom 2023-07-06T03:26:07.619 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-06T03:26:07.619 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/geneve.ko 2023-07-06T03:26:07.619 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv 2023-07-06T03:26:07.619 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-06T03:26:07.619 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154 2023-07-06T03:26:07.619 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-06T03:26:07.620 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ifb.ko 2023-07-06T03:26:07.620 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan 2023-07-06T03:26:07.620 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-06T03:26:07.620 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvlan.ko 2023-07-06T03:26:07.620 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvtap.ko 2023-07-06T03:26:07.620 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio 2023-07-06T03:26:07.620 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio.ko 2023-07-06T03:26:07.621 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-06T03:26:07.621 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-06T03:26:07.621 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mii.ko 2023-07-06T03:26:07.621 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/net_failover.ko 2023-07-06T03:26:07.621 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/netconsole.ko 2023-07-06T03:26:07.621 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/nlmon.ko 2023-07-06T03:26:07.621 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs 2023-07-06T03:26:07.622 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-07-06T03:26:07.622 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-06T03:26:07.622 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy 2023-07-06T03:26:07.622 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/amd.ko 2023-07-06T03:26:07.622 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-06T03:26:07.622 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-06T03:26:07.622 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-06T03:26:07.622 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm87xx.ko 2023-07-06T03:26:07.623 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/broadcom.ko 2023-07-06T03:26:07.623 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/cicada.ko 2023-07-06T03:26:07.623 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/davicom.ko 2023-07-06T03:26:07.623 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/dp83640.ko 2023-07-06T03:26:07.623 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/et1011c.ko 2023-07-06T03:26:07.623 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/icplus.ko 2023-07-06T03:26:07.623 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/lxt.ko 2023-07-06T03:26:07.624 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/marvell.ko 2023-07-06T03:26:07.624 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/micrel.ko 2023-07-06T03:26:07.624 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/national.ko 2023-07-06T03:26:07.624 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/phylink.ko 2023-07-06T03:26:07.624 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/qsemi.ko 2023-07-06T03:26:07.624 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/realtek.ko 2023-07-06T03:26:07.624 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/smsc.ko 2023-07-06T03:26:07.624 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/ste10Xp.ko 2023-07-06T03:26:07.625 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/vitesse.ko 2023-07-06T03:26:07.625 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp 2023-07-06T03:26:07.625 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-06T03:26:07.625 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_async.ko 2023-07-06T03:26:07.625 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-06T03:26:07.625 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-06T03:26:07.625 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-06T03:26:07.626 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-06T03:26:07.626 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppoe.ko 2023-07-06T03:26:07.626 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppox.ko 2023-07-06T03:26:07.626 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pptp.ko 2023-07-06T03:26:07.626 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip 2023-07-06T03:26:07.626 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slhc.ko 2023-07-06T03:26:07.626 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slip.ko 2023-07-06T03:26:07.626 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/sungem_phy.ko 2023-07-06T03:26:07.627 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tap.ko 2023-07-06T03:26:07.627 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team 2023-07-06T03:26:07.627 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team.ko 2023-07-06T03:26:07.627 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-06T03:26:07.627 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-06T03:26:07.627 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-06T03:26:07.627 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_random.ko 2023-07-06T03:26:07.627 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-06T03:26:07.628 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tun.ko 2023-07-06T03:26:07.628 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/veth.ko 2023-07-06T03:26:07.628 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/virtio_net.ko 2023-07-06T03:26:07.628 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3 2023-07-06T03:26:07.628 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-06T03:26:07.628 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan 2023-07-06T03:26:07.628 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan/vxlan.ko 2023-07-06T03:26:07.628 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback 2023-07-06T03:26:07.629 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-06T03:26:07.629 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netfront.ko 2023-07-06T03:26:07.629 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb 2023-07-06T03:26:07.629 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb/ntb.ko 2023-07-06T03:26:07.629 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme 2023-07-06T03:26:07.629 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host 2023-07-06T03:26:07.629 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme-core.ko 2023-07-06T03:26:07.630 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme.ko 2023-07-06T03:26:07.630 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport 2023-07-06T03:26:07.630 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport.ko 2023-07-06T03:26:07.630 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_cs.ko 2023-07-06T03:26:07.630 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_pc.ko 2023-07-06T03:26:07.630 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_serial.ko 2023-07-06T03:26:07.630 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci 2023-07-06T03:26:07.631 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug 2023-07-06T03:26:07.631 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-06T03:26:07.631 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie 2023-07-06T03:26:07.631 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-06T03:26:07.631 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/xen-pcifront.ko 2023-07-06T03:26:07.631 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia 2023-07-06T03:26:07.631 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/i82092.ko 2023-07-06T03:26:07.631 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/pd6729.ko 2023-07-06T03:26:07.632 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-06T03:26:07.632 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl 2023-07-06T03:26:07.632 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel 2023-07-06T03:26:07.632 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-06T03:26:07.632 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform 2023-07-06T03:26:07.633 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86 2023-07-06T03:26:07.633 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-06T03:26:07.633 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acerhdf.ko 2023-07-06T03:26:07.633 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-06T03:26:07.633 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-06T03:26:07.633 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-06T03:26:07.633 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-06T03:26:07.634 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-06T03:26:07.634 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-06T03:26:07.634 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-06T03:26:07.634 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-06T03:26:07.634 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-06T03:26:07.634 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/hdaps.ko 2023-07-06T03:26:07.634 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel 2023-07-06T03:26:07.634 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-06T03:26:07.635 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel_ips.ko 2023-07-06T03:26:07.635 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-06T03:26:07.635 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-06T03:26:07.635 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-06T03:26:07.635 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-06T03:26:07.635 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-06T03:26:07.635 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-06T03:26:07.636 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-06T03:26:07.636 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-06T03:26:07.636 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-06T03:26:07.636 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi.ko 2023-07-06T03:26:07.636 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap 2023-07-06T03:26:07.636 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-06T03:26:07.637 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-06T03:26:07.637 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps 2023-07-06T03:26:07.637 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients 2023-07-06T03:26:07.637 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-06T03:26:07.637 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-06T03:26:07.637 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps_parport.ko 2023-07-06T03:26:07.637 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/pps_core.ko 2023-07-06T03:26:07.638 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp 2023-07-06T03:26:07.638 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp.ko 2023-07-06T03:26:07.638 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp_kvm.ko 2023-07-06T03:26:07.638 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc 2023-07-06T03:26:07.638 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-06T03:26:07.638 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-06T03:26:07.639 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-06T03:26:07.639 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-06T03:26:07.639 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-06T03:26:07.639 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-06T03:26:07.639 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-06T03:26:07.639 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-06T03:26:07.639 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-06T03:26:07.640 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-06T03:26:07.640 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-06T03:26:07.640 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-em3027.ko 2023-07-06T03:26:07.640 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-06T03:26:07.640 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-06T03:26:07.640 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-06T03:26:07.640 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-06T03:26:07.640 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-06T03:26:07.641 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-06T03:26:07.641 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-max6900.ko 2023-07-06T03:26:07.641 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-06T03:26:07.641 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-06T03:26:07.641 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-06T03:26:07.641 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-06T03:26:07.642 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-06T03:26:07.642 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-06T03:26:07.642 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-06T03:26:07.642 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-06T03:26:07.642 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-06T03:26:07.642 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-06T03:26:07.642 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-06T03:26:07.643 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-06T03:26:07.643 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-x1205.ko 2023-07-06T03:26:07.643 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi 2023-07-06T03:26:07.643 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-9xxx.ko 2023-07-06T03:26:07.643 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-sas.ko 2023-07-06T03:26:07.643 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-xxxx.ko 2023-07-06T03:26:07.643 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/BusLogic.ko 2023-07-06T03:26:07.644 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/a100u2w.ko 2023-07-06T03:26:07.644 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid 2023-07-06T03:26:07.644 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-06T03:26:07.644 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/advansys.ko 2023-07-06T03:26:07.645 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx 2023-07-06T03:26:07.645 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-06T03:26:07.645 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-06T03:26:07.645 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/am53c974.ko 2023-07-06T03:26:07.646 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr 2023-07-06T03:26:07.646 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-06T03:26:07.646 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/atp870u.ko 2023-07-06T03:26:07.646 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi 2023-07-06T03:26:07.647 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-06T03:26:07.647 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa 2023-07-06T03:26:07.648 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa/bfa.ko 2023-07-06T03:26:07.648 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc 2023-07-06T03:26:07.649 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-06T03:26:07.649 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i 2023-07-06T03:26:07.649 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-06T03:26:07.650 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ch.ko 2023-07-06T03:26:07.650 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor 2023-07-06T03:26:07.650 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-06T03:26:07.650 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi 2023-07-06T03:26:07.650 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-06T03:26:07.651 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-06T03:26:07.651 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-06T03:26:07.651 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-06T03:26:07.651 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-06T03:26:07.651 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dc395x.ko 2023-07-06T03:26:07.651 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler 2023-07-06T03:26:07.652 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-06T03:26:07.652 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-06T03:26:07.652 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-06T03:26:07.652 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-06T03:26:07.652 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dmx3191d.ko 2023-07-06T03:26:07.652 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r 2023-07-06T03:26:07.652 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-06T03:26:07.652 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esp_scsi.ko 2023-07-06T03:26:07.653 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe 2023-07-06T03:26:07.653 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-06T03:26:07.653 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-06T03:26:07.653 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic 2023-07-06T03:26:07.653 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic/fnic.ko 2023-07-06T03:26:07.653 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hpsa.ko 2023-07-06T03:26:07.653 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hptiop.ko 2023-07-06T03:26:07.654 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hv_storvsc.ko 2023-07-06T03:26:07.654 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/initio.ko 2023-07-06T03:26:07.654 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ipr.ko 2023-07-06T03:26:07.654 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ips.ko 2023-07-06T03:26:07.654 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci 2023-07-06T03:26:07.654 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci/isci.ko 2023-07-06T03:26:07.654 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-06T03:26:07.655 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-06T03:26:07.655 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc 2023-07-06T03:26:07.655 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc/libfc.ko 2023-07-06T03:26:07.655 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi.ko 2023-07-06T03:26:07.655 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-06T03:26:07.655 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas 2023-07-06T03:26:07.655 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas/libsas.ko 2023-07-06T03:26:07.655 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc 2023-07-06T03:26:07.656 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-06T03:26:07.656 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid 2023-07-06T03:26:07.656 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid.ko 2023-07-06T03:26:07.656 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-06T03:26:07.656 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-06T03:26:07.656 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-06T03:26:07.656 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas 2023-07-06T03:26:07.656 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-06T03:26:07.657 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas 2023-07-06T03:26:07.657 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-06T03:26:07.657 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvumi.ko 2023-07-06T03:26:07.657 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001 2023-07-06T03:26:07.658 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-06T03:26:07.658 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pmcraid.ko 2023-07-06T03:26:07.658 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla1280.ko 2023-07-06T03:26:07.658 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx 2023-07-06T03:26:07.658 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-06T03:26:07.658 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-06T03:26:07.658 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx 2023-07-06T03:26:07.659 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-06T03:26:07.659 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/raid_class.ko 2023-07-06T03:26:07.659 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_debug.ko 2023-07-06T03:26:07.659 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-06T03:26:07.659 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-06T03:26:07.659 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-06T03:26:07.659 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-06T03:26:07.659 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-06T03:26:07.660 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ses.ko 2023-07-06T03:26:07.660 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/st.ko 2023-07-06T03:26:07.660 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/stex.ko 2023-07-06T03:26:07.660 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2 2023-07-06T03:26:07.660 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-06T03:26:07.661 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/virtio_scsi.ko 2023-07-06T03:26:07.661 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-06T03:26:07.661 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/wd719x.ko 2023-07-06T03:26:07.661 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb 2023-07-06T03:26:07.661 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb/ssb.ko 2023-07-06T03:26:07.661 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target 2023-07-06T03:26:07.661 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi 2023-07-06T03:26:07.661 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-06T03:26:07.662 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback 2023-07-06T03:26:07.662 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-06T03:26:07.662 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_file.ko 2023-07-06T03:26:07.662 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_iblock.ko 2023-07-06T03:26:07.662 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_mod.ko 2023-07-06T03:26:07.662 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_pscsi.ko 2023-07-06T03:26:07.662 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_user.ko 2023-07-06T03:26:07.663 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc 2023-07-06T03:26:07.663 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-06T03:26:07.663 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal 2023-07-06T03:26:07.663 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel 2023-07-06T03:26:07.663 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal 2023-07-06T03:26:07.663 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-06T03:26:07.663 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-06T03:26:07.664 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-06T03:26:07.664 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-06T03:26:07.664 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-06T03:26:07.664 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-06T03:26:07.664 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-06T03:26:07.664 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-06T03:26:07.665 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-06T03:26:07.665 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-06T03:26:07.665 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-06T03:26:07.665 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-06T03:26:07.665 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-06T03:26:07.665 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-06T03:26:07.665 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-06T03:26:07.666 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty 2023-07-06T03:26:07.666 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless 2023-07-06T03:26:07.666 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-06T03:26:07.666 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_gsm.ko 2023-07-06T03:26:07.666 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_hdlc.ko 2023-07-06T03:26:07.666 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/nozomi.ko 2023-07-06T03:26:07.666 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial 2023-07-06T03:26:07.667 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250 2023-07-06T03:26:07.667 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-06T03:26:07.667 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/arc_uart.ko 2023-07-06T03:26:07.667 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm 2023-07-06T03:26:07.667 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-06T03:26:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/synclink_gt.ko 2023-07-06T03:26:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs 2023-07-06T03:26:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core 2023-07-06T03:26:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-06T03:26:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host 2023-07-06T03:26:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-06T03:26:07.668 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio 2023-07-06T03:26:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio.ko 2023-07-06T03:26:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_aec.ko 2023-07-06T03:26:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_cif.ko 2023-07-06T03:26:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_pci_generic.ko 2023-07-06T03:26:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_sercos3.ko 2023-07-06T03:26:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb 2023-07-06T03:26:07.670 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class 2023-07-06T03:26:07.670 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-acm.ko 2023-07-06T03:26:07.670 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-06T03:26:07.670 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usblp.ko 2023-07-06T03:26:07.670 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usbtmc.ko 2023-07-06T03:26:07.670 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host 2023-07-06T03:26:07.670 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-06T03:26:07.671 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image 2023-07-06T03:26:07.671 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/mdc800.ko 2023-07-06T03:26:07.671 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/microtek.ko 2023-07-06T03:26:07.671 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc 2023-07-06T03:26:07.672 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/adutux.ko 2023-07-06T03:26:07.672 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/appledisplay.ko 2023-07-06T03:26:07.672 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi26.ko 2023-07-06T03:26:07.672 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi62.ko 2023-07-06T03:26:07.672 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ezusb.ko 2023-07-06T03:26:07.672 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/idmouse.ko 2023-07-06T03:26:07.672 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/iowarrior.ko 2023-07-06T03:26:07.673 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-06T03:26:07.673 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ldusb.ko 2023-07-06T03:26:07.673 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/legousbtower.ko 2023-07-06T03:26:07.673 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga 2023-07-06T03:26:07.674 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-06T03:26:07.674 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-06T03:26:07.674 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usb3503.ko 2023-07-06T03:26:07.674 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usblcd.ko 2023-07-06T03:26:07.674 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-06T03:26:07.675 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/uss720.ko 2023-07-06T03:26:07.675 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/yurex.ko 2023-07-06T03:26:07.675 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy 2023-07-06T03:26:07.675 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy/phy-generic.ko 2023-07-06T03:26:07.675 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial 2023-07-06T03:26:07.675 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/aircable.ko 2023-07-06T03:26:07.675 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ark3116.ko 2023-07-06T03:26:07.676 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-06T03:26:07.676 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ch341.ko 2023-07-06T03:26:07.676 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cp210x.ko 2023-07-06T03:26:07.676 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cyberjack.ko 2023-07-06T03:26:07.676 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-06T03:26:07.676 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-06T03:26:07.676 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/empeg.ko 2023-07-06T03:26:07.677 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-06T03:26:07.677 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-06T03:26:07.677 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-06T03:26:07.677 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_ti.ko 2023-07-06T03:26:07.677 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipaq.ko 2023-07-06T03:26:07.677 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipw.ko 2023-07-06T03:26:07.677 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ir-usb.ko 2023-07-06T03:26:07.678 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-06T03:26:07.678 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan.ko 2023-07-06T03:26:07.678 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-06T03:26:07.678 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-06T03:26:07.678 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-06T03:26:07.678 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mct_u232.ko 2023-07-06T03:26:07.678 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7720.ko 2023-07-06T03:26:07.678 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7840.ko 2023-07-06T03:26:07.679 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/navman.ko 2023-07-06T03:26:07.679 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/omninet.ko 2023-07-06T03:26:07.679 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/opticon.ko 2023-07-06T03:26:07.679 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/option.ko 2023-07-06T03:26:07.679 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/oti6858.ko 2023-07-06T03:26:07.679 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/pl2303.ko 2023-07-06T03:26:07.679 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcaux.ko 2023-07-06T03:26:07.680 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcserial.ko 2023-07-06T03:26:07.680 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/quatech2.ko 2023-07-06T03:26:07.680 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/safe_serial.ko 2023-07-06T03:26:07.680 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/sierra.ko 2023-07-06T03:26:07.680 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-06T03:26:07.680 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ssu100.ko 2023-07-06T03:26:07.680 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/symbolserial.ko 2023-07-06T03:26:07.681 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-06T03:26:07.681 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-06T03:26:07.681 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_debug.ko 2023-07-06T03:26:07.681 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-06T03:26:07.681 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/visor.ko 2023-07-06T03:26:07.681 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/whiteheat.ko 2023-07-06T03:26:07.681 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-06T03:26:07.681 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage 2023-07-06T03:26:07.682 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/uas.ko 2023-07-06T03:26:07.682 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-06T03:26:07.682 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-06T03:26:07.682 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-06T03:26:07.682 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-06T03:26:07.682 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-06T03:26:07.682 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-06T03:26:07.683 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-06T03:26:07.683 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-karma.ko 2023-07-06T03:26:07.683 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-06T03:26:07.683 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-06T03:26:07.683 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-06T03:26:07.683 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-06T03:26:07.683 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-06T03:26:07.683 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/usb-storage.ko 2023-07-06T03:26:07.684 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip 2023-07-06T03:26:07.684 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-06T03:26:07.684 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-06T03:26:07.684 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-06T03:26:07.684 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio 2023-07-06T03:26:07.684 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci 2023-07-06T03:26:07.684 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-06T03:26:07.685 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-06T03:26:07.685 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio.ko 2023-07-06T03:26:07.685 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-06T03:26:07.685 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost 2023-07-06T03:26:07.685 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost.ko 2023-07-06T03:26:07.685 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-06T03:26:07.685 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_net.ko 2023-07-06T03:26:07.685 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_scsi.ko 2023-07-06T03:26:07.686 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video 2023-07-06T03:26:07.686 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight 2023-07-06T03:26:07.686 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/apple_bl.ko 2023-07-06T03:26:07.686 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/lcd.ko 2023-07-06T03:26:07.686 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-06T03:26:07.686 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio 2023-07-06T03:26:07.686 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio.ko 2023-07-06T03:26:07.686 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_balloon.ko 2023-07-06T03:26:07.687 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_mmio.ko 2023-07-06T03:26:07.687 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci.ko 2023-07-06T03:26:07.687 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-06T03:26:07.687 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-06T03:26:07.687 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_ring.ko 2023-07-06T03:26:07.687 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1 2023-07-06T03:26:07.687 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters 2023-07-06T03:26:07.687 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2482.ko 2023-07-06T03:26:07.688 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2490.ko 2023-07-06T03:26:07.688 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves 2023-07-06T03:26:07.688 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-06T03:26:07.688 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-06T03:26:07.688 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-06T03:26:07.688 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-06T03:26:07.688 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-06T03:26:07.689 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-06T03:26:07.689 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-06T03:26:07.689 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-06T03:26:07.689 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-06T03:26:07.689 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-06T03:26:07.689 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-06T03:26:07.689 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/wire.ko 2023-07-06T03:26:07.689 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog 2023-07-06T03:26:07.690 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-06T03:26:07.690 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-06T03:26:07.690 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-06T03:26:07.690 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/hpwdt.ko 2023-07-06T03:26:07.690 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/i6300esb.ko 2023-07-06T03:26:07.690 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-06T03:26:07.690 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-06T03:26:07.691 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ib700wdt.ko 2023-07-06T03:26:07.691 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ibmasr.ko 2023-07-06T03:26:07.691 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-06T03:26:07.691 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-06T03:26:07.691 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it87_wdt.ko 2023-07-06T03:26:07.691 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/machzwd.ko 2023-07-06T03:26:07.691 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/nv_tco.ko 2023-07-06T03:26:07.691 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-06T03:26:07.692 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-06T03:26:07.692 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-06T03:26:07.692 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-06T03:26:07.692 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/softdog.ko 2023-07-06T03:26:07.692 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-06T03:26:07.692 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/via_wdt.ko 2023-07-06T03:26:07.692 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-06T03:26:07.693 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-06T03:26:07.693 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-06T03:26:07.693 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/wdt_pci.ko 2023-07-06T03:26:07.693 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/xen_wdt.ko 2023-07-06T03:26:07.693 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen 2023-07-06T03:26:07.693 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-06T03:26:07.693 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-evtchn.ko 2023-07-06T03:26:07.693 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntalloc.ko 2023-07-06T03:26:07.694 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntdev.ko 2023-07-06T03:26:07.694 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback 2023-07-06T03:26:07.694 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-06T03:26:07.694 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-privcmd.ko 2023-07-06T03:26:07.694 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs 2023-07-06T03:26:07.694 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-06T03:26:07.694 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs 2023-07-06T03:26:07.694 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p 2023-07-06T03:26:07.695 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p/9p.ko 2023-07-06T03:26:07.695 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs 2023-07-06T03:26:07.695 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs/affs.ko 2023-07-06T03:26:07.695 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs 2023-07-06T03:26:07.695 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs/befs.ko 2023-07-06T03:26:07.695 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/binfmt_misc.ko 2023-07-06T03:26:07.695 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs 2023-07-06T03:26:07.695 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs/btrfs.ko 2023-07-06T03:26:07.696 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles 2023-07-06T03:26:07.696 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles/cachefiles.ko 2023-07-06T03:26:07.696 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph 2023-07-06T03:26:07.696 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph/ceph.ko 2023-07-06T03:26:07.696 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda 2023-07-06T03:26:07.696 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda/coda.ko 2023-07-06T03:26:07.697 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs 2023-07-06T03:26:07.697 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs/cramfs.ko 2023-07-06T03:26:07.697 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm 2023-07-06T03:26:07.697 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm/dlm.ko 2023-07-06T03:26:07.697 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs 2023-07-06T03:26:07.697 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-06T03:26:07.697 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs 2023-07-06T03:26:07.697 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs/f2fs.ko 2023-07-06T03:26:07.698 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat 2023-07-06T03:26:07.698 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/fat.ko 2023-07-06T03:26:07.698 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/msdos.ko 2023-07-06T03:26:07.698 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/vfat.ko 2023-07-06T03:26:07.698 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache 2023-07-06T03:26:07.698 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache/fscache.ko 2023-07-06T03:26:07.698 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse 2023-07-06T03:26:07.698 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/cuse.ko 2023-07-06T03:26:07.699 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/fuse.ko 2023-07-06T03:26:07.699 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2 2023-07-06T03:26:07.699 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2/gfs2.ko 2023-07-06T03:26:07.699 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs 2023-07-06T03:26:07.699 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs/hfs.ko 2023-07-06T03:26:07.699 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus 2023-07-06T03:26:07.699 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus/hfsplus.ko 2023-07-06T03:26:07.699 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs 2023-07-06T03:26:07.700 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs/isofs.ko 2023-07-06T03:26:07.700 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs 2023-07-06T03:26:07.700 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs/jfs.ko 2023-07-06T03:26:07.700 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd 2023-07-06T03:26:07.700 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd/lockd.ko 2023-07-06T03:26:07.700 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix 2023-07-06T03:26:07.700 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix/minix.ko 2023-07-06T03:26:07.700 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs 2023-07-06T03:26:07.701 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs/netfs.ko 2023-07-06T03:26:07.701 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs 2023-07-06T03:26:07.701 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout 2023-07-06T03:26:07.701 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-06T03:26:07.701 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout 2023-07-06T03:26:07.701 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-06T03:26:07.701 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout 2023-07-06T03:26:07.702 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-06T03:26:07.702 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfs.ko 2023-07-06T03:26:07.702 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv3.ko 2023-07-06T03:26:07.702 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv4.ko 2023-07-06T03:26:07.702 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common 2023-07-06T03:26:07.702 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/grace.ko 2023-07-06T03:26:07.702 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/nfs_acl.ko 2023-07-06T03:26:07.702 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd 2023-07-06T03:26:07.703 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd/nfsd.ko 2023-07-06T03:26:07.703 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2 2023-07-06T03:26:07.703 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2/nilfs2.ko 2023-07-06T03:26:07.703 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls 2023-07-06T03:26:07.703 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-celtic.ko 2023-07-06T03:26:07.703 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-centeuro.ko 2023-07-06T03:26:07.703 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-croatian.ko 2023-07-06T03:26:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-cyrillic.ko 2023-07-06T03:26:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-gaelic.ko 2023-07-06T03:26:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-greek.ko 2023-07-06T03:26:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-iceland.ko 2023-07-06T03:26:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-inuit.ko 2023-07-06T03:26:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-roman.ko 2023-07-06T03:26:07.704 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-romanian.ko 2023-07-06T03:26:07.705 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-turkish.ko 2023-07-06T03:26:07.705 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1250.ko 2023-07-06T03:26:07.705 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1251.ko 2023-07-06T03:26:07.705 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1255.ko 2023-07-06T03:26:07.705 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp737.ko 2023-07-06T03:26:07.705 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp775.ko 2023-07-06T03:26:07.705 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp850.ko 2023-07-06T03:26:07.705 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp852.ko 2023-07-06T03:26:07.706 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp855.ko 2023-07-06T03:26:07.706 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp857.ko 2023-07-06T03:26:07.706 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp860.ko 2023-07-06T03:26:07.706 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp861.ko 2023-07-06T03:26:07.706 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp862.ko 2023-07-06T03:26:07.706 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp863.ko 2023-07-06T03:26:07.706 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp864.ko 2023-07-06T03:26:07.707 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp865.ko 2023-07-06T03:26:07.707 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp866.ko 2023-07-06T03:26:07.707 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp869.ko 2023-07-06T03:26:07.707 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp874.ko 2023-07-06T03:26:07.707 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp932.ko 2023-07-06T03:26:07.707 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp936.ko 2023-07-06T03:26:07.707 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp949.ko 2023-07-06T03:26:07.707 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp950.ko 2023-07-06T03:26:07.708 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_euc-jp.ko 2023-07-06T03:26:07.708 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-1.ko 2023-07-06T03:26:07.708 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-13.ko 2023-07-06T03:26:07.708 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-14.ko 2023-07-06T03:26:07.708 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-15.ko 2023-07-06T03:26:07.708 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-2.ko 2023-07-06T03:26:07.709 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-3.ko 2023-07-06T03:26:07.709 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-4.ko 2023-07-06T03:26:07.709 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-5.ko 2023-07-06T03:26:07.709 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-6.ko 2023-07-06T03:26:07.709 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-7.ko 2023-07-06T03:26:07.709 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-9.ko 2023-07-06T03:26:07.709 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-r.ko 2023-07-06T03:26:07.709 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-ru.ko 2023-07-06T03:26:07.710 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-u.ko 2023-07-06T03:26:07.710 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_utf8.ko 2023-07-06T03:26:07.710 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2 2023-07-06T03:26:07.710 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster 2023-07-06T03:26:07.710 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-06T03:26:07.710 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm 2023-07-06T03:26:07.710 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-06T03:26:07.711 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs 2023-07-06T03:26:07.711 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-06T03:26:07.711 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2.ko 2023-07-06T03:26:07.711 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-06T03:26:07.711 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-06T03:26:07.711 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-06T03:26:07.711 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs 2023-07-06T03:26:07.712 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs/overlay.ko 2023-07-06T03:26:07.712 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore 2023-07-06T03:26:07.712 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore/ramoops.ko 2023-07-06T03:26:07.712 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs 2023-07-06T03:26:07.712 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs/reiserfs.ko 2023-07-06T03:26:07.712 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs 2023-07-06T03:26:07.712 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs/romfs.ko 2023-07-06T03:26:07.712 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb 2023-07-06T03:26:07.713 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client 2023-07-06T03:26:07.713 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client/cifs.ko 2023-07-06T03:26:07.713 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common 2023-07-06T03:26:07.713 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_arc4.ko 2023-07-06T03:26:07.713 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_md4.ko 2023-07-06T03:26:07.713 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs 2023-07-06T03:26:07.713 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs/squashfs.ko 2023-07-06T03:26:07.713 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv 2023-07-06T03:26:07.714 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv/sysv.ko 2023-07-06T03:26:07.714 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf 2023-07-06T03:26:07.714 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf/udf.ko 2023-07-06T03:26:07.714 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs 2023-07-06T03:26:07.714 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs/ufs.ko 2023-07-06T03:26:07.714 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs 2023-07-06T03:26:07.714 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs/xfs.ko 2023-07-06T03:26:07.715 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel 2023-07-06T03:26:07.715 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu 2023-07-06T03:26:07.715 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu/rcutorture.ko 2023-07-06T03:26:07.715 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/torture.ko 2023-07-06T03:26:07.715 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace 2023-07-06T03:26:07.715 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-06T03:26:07.715 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib 2023-07-06T03:26:07.716 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/asn1_encoder.ko 2023-07-06T03:26:07.716 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc-itu-t.ko 2023-07-06T03:26:07.716 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc8.ko 2023-07-06T03:26:07.716 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto 2023-07-06T03:26:07.716 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libarc4.ko 2023-07-06T03:26:07.716 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libdes.ko 2023-07-06T03:26:07.716 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/libcrc32c.ko 2023-07-06T03:26:07.716 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lru_cache.ko 2023-07-06T03:26:07.717 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4 2023-07-06T03:26:07.717 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4_compress.ko 2023-07-06T03:26:07.717 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4hc_compress.ko 2023-07-06T03:26:07.717 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math 2023-07-06T03:26:07.717 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math/cordic.ko 2023-07-06T03:26:07.717 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6 2023-07-06T03:26:07.717 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6/raid6_pq.ko 2023-07-06T03:26:07.718 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon 2023-07-06T03:26:07.718 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-06T03:26:07.718 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_bm.ko 2023-07-06T03:26:07.718 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_fsm.ko 2023-07-06T03:26:07.718 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_kmp.ko 2023-07-06T03:26:07.718 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd 2023-07-06T03:26:07.718 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd/zstd_compress.ko 2023-07-06T03:26:07.719 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm 2023-07-06T03:26:07.719 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm/hwpoison-inject.ko 2023-07-06T03:26:07.719 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net 2023-07-06T03:26:07.719 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan 2023-07-06T03:26:07.719 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/6lowpan.ko 2023-07-06T03:26:07.719 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_dest.ko 2023-07-06T03:26:07.720 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_fragment.ko 2023-07-06T03:26:07.720 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_hop.ko 2023-07-06T03:26:07.720 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-06T03:26:07.720 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_mobility.ko 2023-07-06T03:26:07.720 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_routing.ko 2023-07-06T03:26:07.720 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_udp.ko 2023-07-06T03:26:07.721 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802 2023-07-06T03:26:07.721 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/garp.ko 2023-07-06T03:26:07.721 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/mrp.ko 2023-07-06T03:26:07.721 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/p8022.ko 2023-07-06T03:26:07.721 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/psnap.ko 2023-07-06T03:26:07.721 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/stp.ko 2023-07-06T03:26:07.721 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q 2023-07-06T03:26:07.721 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q/8021q.ko 2023-07-06T03:26:07.722 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p 2023-07-06T03:26:07.722 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet.ko 2023-07-06T03:26:07.722 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_fd.ko 2023-07-06T03:26:07.722 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_rdma.ko 2023-07-06T03:26:07.722 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_virtio.ko 2023-07-06T03:26:07.722 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv 2023-07-06T03:26:07.722 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv/batman-adv.ko 2023-07-06T03:26:07.722 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge 2023-07-06T03:26:07.723 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/br_netfilter.ko 2023-07-06T03:26:07.723 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/bridge.ko 2023-07-06T03:26:07.723 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter 2023-07-06T03:26:07.723 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-06T03:26:07.723 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-06T03:26:07.723 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-06T03:26:07.723 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-06T03:26:07.724 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-06T03:26:07.724 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-06T03:26:07.724 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-06T03:26:07.724 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-06T03:26:07.724 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-06T03:26:07.724 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-06T03:26:07.724 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-06T03:26:07.724 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-06T03:26:07.725 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-06T03:26:07.725 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-06T03:26:07.725 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-06T03:26:07.725 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-06T03:26:07.725 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-06T03:26:07.725 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-06T03:26:07.725 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-06T03:26:07.726 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-06T03:26:07.726 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtables.ko 2023-07-06T03:26:07.726 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-06T03:26:07.726 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-06T03:26:07.726 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-06T03:26:07.726 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph 2023-07-06T03:26:07.726 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph/libceph.ko 2023-07-06T03:26:07.726 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core 2023-07-06T03:26:07.727 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/failover.ko 2023-07-06T03:26:07.727 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/pktgen.ko 2023-07-06T03:26:07.727 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp 2023-07-06T03:26:07.727 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp.ko 2023-07-06T03:26:07.727 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_diag.ko 2023-07-06T03:26:07.727 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv4.ko 2023-07-06T03:26:07.727 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv6.ko 2023-07-06T03:26:07.727 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver 2023-07-06T03:26:07.728 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver/dns_resolver.ko 2023-07-06T03:26:07.728 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa 2023-07-06T03:26:07.728 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/dsa_core.ko 2023-07-06T03:26:07.728 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_brcm.ko 2023-07-06T03:26:07.728 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_dsa.ko 2023-07-06T03:26:07.728 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_none.ko 2023-07-06T03:26:07.729 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_trailer.ko 2023-07-06T03:26:07.729 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154 2023-07-06T03:26:07.729 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan 2023-07-06T03:26:07.729 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-06T03:26:07.729 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154.ko 2023-07-06T03:26:07.729 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-06T03:26:07.729 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4 2023-07-06T03:26:07.729 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ah4.ko 2023-07-06T03:26:07.730 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/esp4.ko 2023-07-06T03:26:07.731 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/fou.ko 2023-07-06T03:26:07.731 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/gre.ko 2023-07-06T03:26:07.732 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/inet_diag.ko 2023-07-06T03:26:07.732 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_gre.ko 2023-07-06T03:26:07.732 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_tunnel.ko 2023-07-06T03:26:07.732 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_vti.ko 2023-07-06T03:26:07.732 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipcomp.ko 2023-07-06T03:26:07.732 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipip.ko 2023-07-06T03:26:07.732 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter 2023-07-06T03:26:07.732 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-06T03:26:07.733 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-06T03:26:07.733 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-06T03:26:07.733 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-06T03:26:07.733 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-06T03:26:07.733 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-06T03:26:07.733 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-06T03:26:07.733 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-06T03:26:07.733 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-06T03:26:07.734 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-06T03:26:07.734 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-06T03:26:07.734 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-06T03:26:07.734 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-06T03:26:07.734 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-06T03:26:07.734 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-06T03:26:07.734 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-06T03:26:07.735 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-06T03:26:07.735 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-06T03:26:07.735 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-06T03:26:07.735 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-06T03:26:07.735 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-06T03:26:07.735 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_bic.ko 2023-07-06T03:26:07.735 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_dctcp.ko 2023-07-06T03:26:07.735 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_diag.ko 2023-07-06T03:26:07.736 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_highspeed.ko 2023-07-06T03:26:07.736 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_htcp.ko 2023-07-06T03:26:07.736 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_hybla.ko 2023-07-06T03:26:07.736 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_illinois.ko 2023-07-06T03:26:07.736 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_lp.ko 2023-07-06T03:26:07.736 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_scalable.ko 2023-07-06T03:26:07.736 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_vegas.ko 2023-07-06T03:26:07.736 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_veno.ko 2023-07-06T03:26:07.737 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_westwood.ko 2023-07-06T03:26:07.737 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_yeah.ko 2023-07-06T03:26:07.737 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tunnel4.ko 2023-07-06T03:26:07.737 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_diag.ko 2023-07-06T03:26:07.737 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_tunnel.ko 2023-07-06T03:26:07.737 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-06T03:26:07.737 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6 2023-07-06T03:26:07.737 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ah6.ko 2023-07-06T03:26:07.738 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/esp6.ko 2023-07-06T03:26:07.738 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/fou6.ko 2023-07-06T03:26:07.738 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_tunnel.ko 2023-07-06T03:26:07.738 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-06T03:26:07.738 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_vti.ko 2023-07-06T03:26:07.738 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ipcomp6.ko 2023-07-06T03:26:07.738 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter 2023-07-06T03:26:07.738 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-06T03:26:07.739 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-06T03:26:07.739 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-06T03:26:07.739 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-06T03:26:07.739 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-06T03:26:07.739 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-06T03:26:07.739 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-06T03:26:07.739 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-06T03:26:07.739 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-06T03:26:07.740 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-06T03:26:07.740 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-06T03:26:07.740 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-06T03:26:07.740 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-06T03:26:07.740 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-06T03:26:07.740 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-06T03:26:07.740 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-06T03:26:07.741 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-06T03:26:07.741 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-06T03:26:07.741 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-06T03:26:07.741 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-06T03:26:07.741 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-06T03:26:07.741 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-06T03:26:07.741 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-06T03:26:07.741 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-06T03:26:07.742 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/sit.ko 2023-07-06T03:26:07.742 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/tunnel6.ko 2023-07-06T03:26:07.742 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-06T03:26:07.742 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key 2023-07-06T03:26:07.742 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key/af_key.ko 2023-07-06T03:26:07.742 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp 2023-07-06T03:26:07.742 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_core.ko 2023-07-06T03:26:07.742 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-06T03:26:07.743 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_eth.ko 2023-07-06T03:26:07.743 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip.ko 2023-07-06T03:26:07.743 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip6.ko 2023-07-06T03:26:07.743 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_netlink.ko 2023-07-06T03:26:07.743 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ppp.ko 2023-07-06T03:26:07.743 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc 2023-07-06T03:26:07.743 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc/llc.ko 2023-07-06T03:26:07.743 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154 2023-07-06T03:26:07.744 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154/mac802154.ko 2023-07-06T03:26:07.744 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls 2023-07-06T03:26:07.744 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls/mpls_gso.ko 2023-07-06T03:26:07.744 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter 2023-07-06T03:26:07.744 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset 2023-07-06T03:26:07.744 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set.ko 2023-07-06T03:26:07.744 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-06T03:26:07.744 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-06T03:26:07.745 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-06T03:26:07.745 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-06T03:26:07.745 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-06T03:26:07.745 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-06T03:26:07.745 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-06T03:26:07.745 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-06T03:26:07.745 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-06T03:26:07.745 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-06T03:26:07.746 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-06T03:26:07.746 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-06T03:26:07.746 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-06T03:26:07.746 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-06T03:26:07.746 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-06T03:26:07.746 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs 2023-07-06T03:26:07.746 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-06T03:26:07.747 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-06T03:26:07.747 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-06T03:26:07.747 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-06T03:26:07.747 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-06T03:26:07.747 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-06T03:26:07.747 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-06T03:26:07.747 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-06T03:26:07.747 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-06T03:26:07.748 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-06T03:26:07.748 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-06T03:26:07.748 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-06T03:26:07.748 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-06T03:26:07.748 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-06T03:26:07.748 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conncount.ko 2023-07-06T03:26:07.748 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack.ko 2023-07-06T03:26:07.748 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-06T03:26:07.749 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-06T03:26:07.749 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-06T03:26:07.749 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-06T03:26:07.749 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-06T03:26:07.749 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-06T03:26:07.749 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-06T03:26:07.749 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-06T03:26:07.749 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-06T03:26:07.750 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-06T03:26:07.750 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-06T03:26:07.750 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-06T03:26:07.750 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-06T03:26:07.750 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table.ko 2023-07-06T03:26:07.750 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-06T03:26:07.750 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_log_syslog.ko 2023-07-06T03:26:07.750 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat.ko 2023-07-06T03:26:07.751 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-06T03:26:07.751 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-06T03:26:07.751 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_irc.ko 2023-07-06T03:26:07.751 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_sip.ko 2023-07-06T03:26:07.751 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-06T03:26:07.751 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-06T03:26:07.751 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_tables.ko 2023-07-06T03:26:07.752 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink.ko 2023-07-06T03:26:07.752 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-06T03:26:07.752 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_log.ko 2023-07-06T03:26:07.752 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-06T03:26:07.752 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-06T03:26:07.752 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_chain_nat.ko 2023-07-06T03:26:07.752 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_compat.ko 2023-07-06T03:26:07.752 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_ct.ko 2023-07-06T03:26:07.753 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-06T03:26:07.753 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fib.ko 2023-07-06T03:26:07.753 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-06T03:26:07.753 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_hash.ko 2023-07-06T03:26:07.753 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_limit.ko 2023-07-06T03:26:07.753 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_log.ko 2023-07-06T03:26:07.753 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_masq.ko 2023-07-06T03:26:07.753 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_nat.ko 2023-07-06T03:26:07.754 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_numgen.ko 2023-07-06T03:26:07.754 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_queue.ko 2023-07-06T03:26:07.754 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_quota.ko 2023-07-06T03:26:07.754 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_redir.ko 2023-07-06T03:26:07.754 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject.ko 2023-07-06T03:26:07.754 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject_inet.ko 2023-07-06T03:26:07.754 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_socket.ko 2023-07-06T03:26:07.754 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_tunnel.ko 2023-07-06T03:26:07.755 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_AUDIT.ko 2023-07-06T03:26:07.755 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-06T03:26:07.755 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-06T03:26:07.755 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-06T03:26:07.755 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CT.ko 2023-07-06T03:26:07.755 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_DSCP.ko 2023-07-06T03:26:07.755 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HL.ko 2023-07-06T03:26:07.756 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HMARK.ko 2023-07-06T03:26:07.756 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-06T03:26:07.756 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LED.ko 2023-07-06T03:26:07.756 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LOG.ko 2023-07-06T03:26:07.756 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-06T03:26:07.756 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NETMAP.ko 2023-07-06T03:26:07.756 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFLOG.ko 2023-07-06T03:26:07.756 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-06T03:26:07.757 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_RATEEST.ko 2023-07-06T03:26:07.757 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-06T03:26:07.757 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_SECMARK.ko 2023-07-06T03:26:07.757 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-06T03:26:07.757 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-06T03:26:07.757 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TEE.ko 2023-07-06T03:26:07.757 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TPROXY.ko 2023-07-06T03:26:07.758 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TRACE.ko 2023-07-06T03:26:07.758 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_addrtype.ko 2023-07-06T03:26:07.758 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_bpf.ko 2023-07-06T03:26:07.758 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cgroup.ko 2023-07-06T03:26:07.758 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cluster.ko 2023-07-06T03:26:07.758 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_comment.ko 2023-07-06T03:26:07.758 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connbytes.ko 2023-07-06T03:26:07.758 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlabel.ko 2023-07-06T03:26:07.759 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlimit.ko 2023-07-06T03:26:07.759 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connmark.ko 2023-07-06T03:26:07.759 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_conntrack.ko 2023-07-06T03:26:07.759 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cpu.ko 2023-07-06T03:26:07.759 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dccp.ko 2023-07-06T03:26:07.759 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_devgroup.ko 2023-07-06T03:26:07.759 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dscp.ko 2023-07-06T03:26:07.760 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ecn.ko 2023-07-06T03:26:07.760 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_esp.ko 2023-07-06T03:26:07.760 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hashlimit.ko 2023-07-06T03:26:07.760 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_helper.ko 2023-07-06T03:26:07.760 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hl.ko 2023-07-06T03:26:07.760 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipcomp.ko 2023-07-06T03:26:07.760 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_iprange.ko 2023-07-06T03:26:07.760 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipvs.ko 2023-07-06T03:26:07.761 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_l2tp.ko 2023-07-06T03:26:07.761 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_length.ko 2023-07-06T03:26:07.761 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_limit.ko 2023-07-06T03:26:07.761 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mac.ko 2023-07-06T03:26:07.761 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mark.ko 2023-07-06T03:26:07.761 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_multiport.ko 2023-07-06T03:26:07.761 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nat.ko 2023-07-06T03:26:07.761 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nfacct.ko 2023-07-06T03:26:07.762 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_osf.ko 2023-07-06T03:26:07.762 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_owner.ko 2023-07-06T03:26:07.762 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_physdev.ko 2023-07-06T03:26:07.762 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_pkttype.ko 2023-07-06T03:26:07.762 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_policy.ko 2023-07-06T03:26:07.762 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_quota.ko 2023-07-06T03:26:07.762 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_rateest.ko 2023-07-06T03:26:07.762 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_realm.ko 2023-07-06T03:26:07.763 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_recent.ko 2023-07-06T03:26:07.763 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_sctp.ko 2023-07-06T03:26:07.763 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_set.ko 2023-07-06T03:26:07.763 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_socket.ko 2023-07-06T03:26:07.763 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_state.ko 2023-07-06T03:26:07.763 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_statistic.ko 2023-07-06T03:26:07.763 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_string.ko 2023-07-06T03:26:07.763 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_tcpmss.ko 2023-07-06T03:26:07.764 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_time.ko 2023-07-06T03:26:07.764 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_u32.ko 2023-07-06T03:26:07.764 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink 2023-07-06T03:26:07.764 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink/netlink_diag.ko 2023-07-06T03:26:07.764 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh 2023-07-06T03:26:07.764 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh/nsh.ko 2023-07-06T03:26:07.764 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch 2023-07-06T03:26:07.765 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/openvswitch.ko 2023-07-06T03:26:07.765 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-geneve.ko 2023-07-06T03:26:07.765 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-gre.ko 2023-07-06T03:26:07.765 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-vxlan.ko 2023-07-06T03:26:07.765 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet 2023-07-06T03:26:07.765 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet/af_packet_diag.ko 2023-07-06T03:26:07.765 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds 2023-07-06T03:26:07.765 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds.ko 2023-07-06T03:26:07.766 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_rdma.ko 2023-07-06T03:26:07.766 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_tcp.ko 2023-07-06T03:26:07.766 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched 2023-07-06T03:26:07.766 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_csum.ko 2023-07-06T03:26:07.766 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_gact.ko 2023-07-06T03:26:07.766 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_ipt.ko 2023-07-06T03:26:07.766 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_mirred.ko 2023-07-06T03:26:07.766 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_nat.ko 2023-07-06T03:26:07.767 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_pedit.ko 2023-07-06T03:26:07.767 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_police.ko 2023-07-06T03:26:07.767 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_simple.ko 2023-07-06T03:26:07.767 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_skbedit.ko 2023-07-06T03:26:07.767 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_vlan.ko 2023-07-06T03:26:07.767 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_basic.ko 2023-07-06T03:26:07.767 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_bpf.ko 2023-07-06T03:26:07.767 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_flow.ko 2023-07-06T03:26:07.768 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_fw.ko 2023-07-06T03:26:07.768 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_route.ko 2023-07-06T03:26:07.768 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_u32.ko 2023-07-06T03:26:07.768 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_cmp.ko 2023-07-06T03:26:07.768 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_ipset.ko 2023-07-06T03:26:07.768 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_meta.ko 2023-07-06T03:26:07.768 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_nbyte.ko 2023-07-06T03:26:07.768 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_text.ko 2023-07-06T03:26:07.769 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_u32.ko 2023-07-06T03:26:07.769 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_choke.ko 2023-07-06T03:26:07.769 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_codel.ko 2023-07-06T03:26:07.769 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_drr.ko 2023-07-06T03:26:07.769 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_fq.ko 2023-07-06T03:26:07.769 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_gred.ko 2023-07-06T03:26:07.769 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hfsc.ko 2023-07-06T03:26:07.769 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hhf.ko 2023-07-06T03:26:07.770 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_htb.ko 2023-07-06T03:26:07.770 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_ingress.ko 2023-07-06T03:26:07.770 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio.ko 2023-07-06T03:26:07.770 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio_lib.ko 2023-07-06T03:26:07.770 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_multiq.ko 2023-07-06T03:26:07.770 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_netem.ko 2023-07-06T03:26:07.770 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_pie.ko 2023-07-06T03:26:07.770 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_plug.ko 2023-07-06T03:26:07.771 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_prio.ko 2023-07-06T03:26:07.771 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_qfq.ko 2023-07-06T03:26:07.771 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_red.ko 2023-07-06T03:26:07.771 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfb.ko 2023-07-06T03:26:07.771 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfq.ko 2023-07-06T03:26:07.771 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_tbf.ko 2023-07-06T03:26:07.771 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_teql.ko 2023-07-06T03:26:07.771 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp 2023-07-06T03:26:07.772 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp.ko 2023-07-06T03:26:07.772 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp_diag.ko 2023-07-06T03:26:07.772 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc 2023-07-06T03:26:07.772 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss 2023-07-06T03:26:07.772 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-06T03:26:07.772 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-06T03:26:07.772 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/sunrpc.ko 2023-07-06T03:26:07.773 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma 2023-07-06T03:26:07.773 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-06T03:26:07.773 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc 2023-07-06T03:26:07.773 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/diag.ko 2023-07-06T03:26:07.773 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/tipc.ko 2023-07-06T03:26:07.773 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix 2023-07-06T03:26:07.773 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix/unix_diag.ko 2023-07-06T03:26:07.773 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock 2023-07-06T03:26:07.774 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-06T03:26:07.774 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-06T03:26:07.774 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock.ko 2023-07-06T03:26:07.774 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-06T03:26:07.774 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-06T03:26:07.774 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm 2023-07-06T03:26:07.774 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-06T03:26:07.774 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security 2023-07-06T03:26:07.775 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys 2023-07-06T03:26:07.775 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys 2023-07-06T03:26:07.775 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-06T03:26:07.775 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys 2023-07-06T03:26:07.775 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys/trusted.ko 2023-07-06T03:26:07.775 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt 2023-07-06T03:26:07.775 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib 2023-07-06T03:26:07.775 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib/irqbypass.ko 2023-07-06T03:26:07.776 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias 2023-07-06T03:26:07.776 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias.bin 2023-07-06T03:26:07.776 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin 2023-07-06T03:26:07.776 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.bin 2023-07-06T03:26:07.776 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.modinfo 2023-07-06T03:26:07.776 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep 2023-07-06T03:26:07.776 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep.bin 2023-07-06T03:26:07.776 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.devname 2023-07-06T03:26:07.777 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.order 2023-07-06T03:26:07.777 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.softdep 2023-07-06T03:26:07.777 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols 2023-07-06T03:26:07.777 INFO:teuthology.orchestra.run.smithi146.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols.bin 2023-07-06T03:26:07.778 DEBUG:teuthology.task.kernel:get_image_version: 6.4.0-g84139c18d56f 2023-07-06T03:26:07.778 DEBUG:teuthology.orchestra.run.smithi146:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-06T03:26:07.806 INFO:teuthology.orchestra.run.smithi146.stdout:postinstall scriptlet (using /bin/sh): 2023-07-06T03:26:07.807 INFO:teuthology.orchestra.run.smithi146.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.4.0-g84139c18d56f -a -r /boot/System.map-6.4.0-g84139c18d56f ]; then 2023-07-06T03:26:07.807 INFO:teuthology.orchestra.run.smithi146.stdout:cp /boot/vmlinuz-6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm 2023-07-06T03:26:07.807 INFO:teuthology.orchestra.run.smithi146.stdout:cp /boot/System.map-6.4.0-g84139c18d56f /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T03:26:07.807 INFO:teuthology.orchestra.run.smithi146.stdout:rm -f /boot/vmlinuz-6.4.0-g84139c18d56f /boot/System.map-6.4.0-g84139c18d56f 2023-07-06T03:26:07.807 INFO:teuthology.orchestra.run.smithi146.stdout:/sbin/installkernel 6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T03:26:07.807 INFO:teuthology.orchestra.run.smithi146.stdout:rm -f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T03:26:07.808 INFO:teuthology.orchestra.run.smithi146.stdout:fi 2023-07-06T03:26:07.808 INFO:teuthology.orchestra.run.smithi146.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-06T03:26:07.808 INFO:teuthology.orchestra.run.smithi146.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-06T03:26:07.808 INFO:teuthology.orchestra.run.smithi146.stdout:new-kernel-pkg --remove 6.4.0-g84139c18d56f --rminitrd --initrdfile=/boot/initramfs-6.4.0-g84139c18d56f.img 2023-07-06T03:26:07.808 INFO:teuthology.orchestra.run.smithi146.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-06T03:26:07.808 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-install remove 6.4.0-g84139c18d56f 2023-07-06T03:26:07.808 INFO:teuthology.orchestra.run.smithi146.stdout:fi 2023-07-06T03:26:07.809 INFO:teuthology.orchestra.run.smithi146.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-06T03:26:07.809 INFO:teuthology.orchestra.run.smithi146.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-06T03:26:07.809 INFO:teuthology.orchestra.run.smithi146.stdout:/sbin/update-bootloader --remove 6.4.0-g84139c18d56f 2023-07-06T03:26:07.809 INFO:teuthology.orchestra.run.smithi146.stdout:fi 2023-07-06T03:26:07.810 DEBUG:teuthology.orchestra.run.smithi146:> sudo rpm -qi grub2-tools 2023-07-06T03:26:07.887 INFO:teuthology.orchestra.run.smithi146.stdout:Name : grub2-tools 2023-07-06T03:26:07.888 INFO:teuthology.orchestra.run.smithi146.stdout:Epoch : 1 2023-07-06T03:26:07.888 INFO:teuthology.orchestra.run.smithi146.stdout:Version : 2.02 2023-07-06T03:26:07.888 INFO:teuthology.orchestra.run.smithi146.stdout:Release : 129.el8 2023-07-06T03:26:07.888 INFO:teuthology.orchestra.run.smithi146.stdout:Architecture: x86_64 2023-07-06T03:26:07.888 INFO:teuthology.orchestra.run.smithi146.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-07-06T03:26:07.889 INFO:teuthology.orchestra.run.smithi146.stdout:Group : System Environment/Base 2023-07-06T03:26:07.889 INFO:teuthology.orchestra.run.smithi146.stdout:Size : 9265586 2023-07-06T03:26:07.889 INFO:teuthology.orchestra.run.smithi146.stdout:License : GPLv3+ 2023-07-06T03:26:07.889 INFO:teuthology.orchestra.run.smithi146.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-07-06T03:26:07.889 INFO:teuthology.orchestra.run.smithi146.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-07-06T03:26:07.889 INFO:teuthology.orchestra.run.smithi146.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-07-06T03:26:07.889 INFO:teuthology.orchestra.run.smithi146.stdout:Build Host : kbuilder.bsys.centos.org 2023-07-06T03:26:07.889 INFO:teuthology.orchestra.run.smithi146.stdout:Relocations : (not relocatable) 2023-07-06T03:26:07.890 INFO:teuthology.orchestra.run.smithi146.stdout:Packager : CentOS BuildSystem 2023-07-06T03:26:07.890 INFO:teuthology.orchestra.run.smithi146.stdout:Vendor : CentOS 2023-07-06T03:26:07.890 INFO:teuthology.orchestra.run.smithi146.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-06T03:26:07.890 INFO:teuthology.orchestra.run.smithi146.stdout:Summary : Support tools for GRUB. 2023-07-06T03:26:07.890 INFO:teuthology.orchestra.run.smithi146.stdout:Description : 2023-07-06T03:26:07.890 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:26:07.890 INFO:teuthology.orchestra.run.smithi146.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-06T03:26:07.890 INFO:teuthology.orchestra.run.smithi146.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-06T03:26:07.891 INFO:teuthology.orchestra.run.smithi146.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-06T03:26:07.891 INFO:teuthology.orchestra.run.smithi146.stdout:hardware devices. 2023-07-06T03:26:07.891 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:26:07.891 INFO:teuthology.orchestra.run.smithi146.stdout:This subpackage provides tools for support of all platforms. 2023-07-06T03:26:07.892 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-06T03:26:07.892 INFO:teuthology.task.kernel:Updating grub on smithi146 to boot 6.4.0-g84139c18d56f 2023-07-06T03:26:07.892 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-06T03:26:13.933 INFO:teuthology.orchestra.run.smithi146.stderr:Generating grub configuration file ... 2023-07-06T03:26:16.104 INFO:teuthology.orchestra.run.smithi146.stderr:done 2023-07-06T03:26:16.107 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2023-07-06T03:26:16.143 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.AdcUiQqUVH 2023-07-06T03:26:16.143 DEBUG:teuthology.orchestra.run.smithi146:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AdcUiQqUVH 2023-07-06T03:26:16.216 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.AdcUiQqUVH 2023-07-06T03:26:16.369 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.AdcUiQqUVH is 6KB 2023-07-06T03:26:16.418 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.AdcUiQqUVH 2023-07-06T03:26:16.436 DEBUG:teuthology.orchestra.run.smithi146:> sudo /bin/ls /boot/loader/entries || true 2023-07-06T03:26:16.505 INFO:teuthology.orchestra.run.smithi146.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-07-06T03:26:16.506 INFO:teuthology.orchestra.run.smithi146.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-07-06T03:26:16.506 INFO:teuthology.orchestra.run.smithi146.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-07-06T03:26:16.506 INFO:teuthology.orchestra.run.smithi146.stdout:a1b29d6c289747bdb22930598d39874a-6.4.0-g84139c18d56f.conf 2023-07-06T03:26:16.507 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.4.0-g84139c18d56f 2023-07-06T03:26:16.666 DEBUG:teuthology.orchestra.run.smithi146:> sudo shutdown -r now 2023-07-06T03:26:33.300 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-07-06T03:26:33.301 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-07-06T03:26:33.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T03:26:46.676 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-07-06T03:26:46.676 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-07-06T03:26:46.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T03:26:56.624 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2023-07-06T03:27:05.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-07-06T03:27:05.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T03:27:10.192 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-07-06T03:27:19.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-07-06T03:27:19.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T03:27:27.280 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-07-06T03:27:39.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-07-06T03:27:39.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T03:28:05.661 DEBUG:teuthology.orchestra.remote:timed out 2023-07-06T03:28:11.189 DEBUG:teuthology.orchestra.run.smithi146:> true 2023-07-06T03:28:11.557 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-07-06T03:28:11.557 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.4.0-g84139c18d56f"... 2023-07-06T03:28:11.558 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2023-07-06T03:28:11.602 INFO:teuthology.orchestra.run.smithi146.stdout:6.4.0-g84139c18d56f 2023-07-06T03:28:11.603 DEBUG:teuthology.task.kernel:current kernel version is 6.4.0-g84139c18d56f vs 6.4.0-g84139c18d56f 2023-07-06T03:28:11.603 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-06T03:28:11.603 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-07-06T03:28:12.604 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-07-06T03:28:12.605 DEBUG:teuthology.orchestra.run.smithi146:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-06T03:28:12.732 INFO:teuthology.orchestra.run.smithi146.stdout:ttyS1 2023-07-06T03:28:12.755 DEBUG:teuthology.parallel:result is None 2023-07-06T03:28:17.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-07-06T03:28:17.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T03:28:17.988 DEBUG:teuthology.orchestra.run.smithi097:> true 2023-07-06T03:28:18.396 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-07-06T03:28:18.396 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.4.0-g84139c18d56f"... 2023-07-06T03:28:18.396 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2023-07-06T03:28:18.441 INFO:teuthology.orchestra.run.smithi097.stdout:6.4.0-g84139c18d56f 2023-07-06T03:28:18.441 DEBUG:teuthology.task.kernel:current kernel version is 6.4.0-g84139c18d56f vs 6.4.0-g84139c18d56f 2023-07-06T03:28:18.441 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-06T03:28:18.441 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-07-06T03:28:19.442 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-07-06T03:28:19.443 DEBUG:teuthology.orchestra.run.smithi097:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-06T03:28:19.829 INFO:teuthology.orchestra.run.smithi097.stdout:ttyS1 2023-07-06T03:28:19.851 DEBUG:teuthology.parallel:result is None 2023-07-06T03:28:19.852 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-06T03:28:19.859 INFO:teuthology.task.internal:Creating test directory... 2023-07-06T03:28:19.859 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T03:28:19.864 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T03:28:19.914 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-06T03:28:19.921 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-06T03:28:19.930 INFO:teuthology.task.internal:Creating archive directory... 2023-07-06T03:28:19.931 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T03:28:19.935 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T03:28:20.004 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-06T03:28:20.011 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-06T03:28:20.012 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T03:28:20.016 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T03:28:20.088 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T03:28:20.103 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T03:28:20.113 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T03:28:20.129 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T03:28:20.133 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-06T03:28:20.143 INFO:teuthology.task.internal:Configuring sudo... 2023-07-06T03:28:20.144 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T03:28:20.148 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T03:28:20.217 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-06T03:28:20.226 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-06T03:28:20.226 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T03:28:20.231 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T03:28:20.275 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T03:28:22.273 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T03:28:22.392 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T03:28:22.469 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T03:28:22.538 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:28:22.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T03:28:22.620 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T03:28:22.699 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T03:28:22.791 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T03:28:22.868 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T03:28:22.937 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:28:22.938 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T03:28:23.013 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2023-07-06T03:28:23.019 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2023-07-06T03:28:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T03:28:23.172 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T03:28:23.448 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-06T03:28:23.460 INFO:teuthology.task.internal:Starting timer... 2023-07-06T03:28:23.460 INFO:teuthology.run_tasks:Running task pcp... 2023-07-06T03:28:23.469 INFO:teuthology.run_tasks:Running task selinux... 2023-07-06T03:28:23.480 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-06T03:28:23.480 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2023-07-06T03:28:23.627 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2023-07-06T03:28:23.630 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2023-07-06T03:28:23.770 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2023-07-06T03:28:23.773 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-06T03:28:23.774 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2023-07-06T03:28:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2023-07-06T03:28:23.830 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2023-07-06T03:28:23.878 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2023-07-06T03:28:23.879 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi146.front.sepia.ceph.com': 'permissive'} 2023-07-06T03:28:23.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T03:28:23.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:28:23.949 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T03:28:24.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:28:24.019 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-06T03:28:24.019 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2023-07-06T03:28:24.087 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2023-07-06T03:28:24.155 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-06T03:28:24.165 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-06T03:28:24.171 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-06T03:28:24.322 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-06T03:28:24.348 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-06T03:28:24.350 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi146.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-06T03:38:46.361 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi146.front.sepia.ceph.com')] 2023-07-06T03:38:46.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-07-06T03:38:46.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T03:38:51.484 DEBUG:teuthology.orchestra.run.smithi097:> true 2023-07-06T03:38:51.617 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-07-06T03:38:51.618 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-07-06T03:38:51.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T03:38:51.706 DEBUG:teuthology.orchestra.run.smithi146:> true 2023-07-06T03:38:51.827 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-07-06T03:38:51.828 INFO:teuthology.run_tasks:Running task clock... 2023-07-06T03:38:51.838 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-06T03:38:51.839 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T03:38:51.840 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T03:38:51.844 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T03:38:51.844 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T03:38:51.919 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T03:38:51.921 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T03:38:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T03:38:51.953 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T03:38:52.041 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2023-07-06T03:38:52.044 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2023-07-06T03:38:52.068 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2023-07-06T03:38:52.070 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2023-07-06T03:38:52.100 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T03:38:52.102 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T03:38:52.132 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T03:38:52.134 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T03:38:52.219 INFO:teuthology.orchestra.run.smithi097.stderr:bash: ntpq: command not found 2023-07-06T03:38:52.220 INFO:teuthology.orchestra.run.smithi146.stderr:bash: ntpq: command not found 2023-07-06T03:38:52.226 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T03:38:52.226 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2023-07-06T03:38:52.227 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T03:38:52.227 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T03:38:52.227 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T03:38:52.227 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T03:38:52.228 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T03:38:52.229 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2023-07-06T03:38:52.229 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T03:38:52.229 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T03:38:52.229 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T03:38:52.229 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T03:38:52.229 INFO:teuthology.run_tasks:Running task install... 2023-07-06T03:38:52.238 DEBUG:teuthology.task.install:project ceph 2023-07-06T03:38:52.238 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}} 2023-07-06T03:38:52.239 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'} 2023-07-06T03:38:52.239 INFO:teuthology.task.install:Using flavor: default 2023-07-06T03:38:52.245 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-06T03:38:52.245 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-06T03:38:52.246 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'tag': None, 'wait_for_package': False} 2023-07-06T03:38:52.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T03:38:52.249 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'tag': None, 'wait_for_package': False} 2023-07-06T03:38:52.249 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T03:39:07.458 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/ 2023-07-06T03:39:07.458 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4742.g79365bb3 2023-07-06T03:39:12.472 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/ 2023-07-06T03:39:12.473 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4742.g79365bb3 2023-07-06T03:39:17.622 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T03:39:17.623 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:39:17.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T03:39:17.655 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T03:39:17.656 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:39:17.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T03:39:17.700 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T03:39:17.700 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/79365bb3b803780276d69efb2cc4afe5b160de9e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T03:39:17.734 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T03:39:17.735 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/79365bb3b803780276d69efb2cc4afe5b160de9e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T03:39:17.811 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T03:39:17.846 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T03:39:17.906 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T03:39:17.941 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T03:39:17.974 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2023-07-06T03:39:17.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2023-07-06T03:39:18.016 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2023-07-06T03:39:18.019 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2023-07-06T03:39:18.320 INFO:teuthology.orchestra.run.smithi097.stdout:73 files removed 2023-07-06T03:39:18.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T03:39:18.368 INFO:teuthology.orchestra.run.smithi146.stdout:73 files removed 2023-07-06T03:39:18.403 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T03:39:19.267 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 215 kB/s | 81 kB 00:00 2023-07-06T03:39:19.381 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 187 kB/s | 81 kB 00:00 2023-07-06T03:39:19.657 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 40 kB/s | 9.4 kB 00:00 2023-07-06T03:39:19.660 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-07-06T03:39:19.950 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 5.6 kB/s | 1.5 kB 00:00 2023-07-06T03:39:19.954 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 5.6 kB/s | 1.5 kB 00:00 2023-07-06T03:39:21.428 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 8 - AppStream 21 MB/s | 31 MB 00:01 2023-07-06T03:39:21.466 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 8 - AppStream 21 MB/s | 31 MB 00:01 2023-07-06T03:39:33.661 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 8 - BaseOS 6.7 MB/s | 41 MB 00:06 2023-07-06T03:39:34.065 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 8 - BaseOS 6.3 MB/s | 41 MB 00:06 2023-07-06T03:39:40.136 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 8 - Extras 105 kB/s | 18 kB 00:00 2023-07-06T03:39:40.595 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 8 - Extras common packages 23 kB/s | 6.6 kB 00:00 2023-07-06T03:39:40.699 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 8 - Extras 59 kB/s | 18 kB 00:00 2023-07-06T03:39:41.091 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 8 - Extras common packages 30 kB/s | 6.6 kB 00:00 2023-07-06T03:39:41.655 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 8 - PowerTools 6.9 MB/s | 6.3 MB 00:00 2023-07-06T03:39:42.067 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 8 - PowerTools 7.6 MB/s | 6.3 MB 00:00 2023-07-06T03:39:45.481 INFO:teuthology.orchestra.run.smithi097.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-07-06T03:39:45.961 INFO:teuthology.orchestra.run.smithi146.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T03:39:48.260 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 16 MB 00:02 2023-07-06T03:39:52.719 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 185 kB/s | 24 kB 00:00 2023-07-06T03:39:56.660 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T03:39:56.661 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T03:39:56.674 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-07-06T03:39:56.674 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-07-06T03:39:56.675 INFO:teuthology.orchestra.run.smithi146.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-07-06T03:39:56.676 INFO:teuthology.orchestra.run.smithi146.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-07-06T03:39:56.676 INFO:teuthology.orchestra.run.smithi146.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T03:39:56.678 INFO:teuthology.orchestra.run.smithi146.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T03:39:56.681 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T03:39:56.682 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T03:39:56.875 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2023-07-06T03:39:56.889 INFO:teuthology.orchestra.run.smithi146.stdout:==================================================================================================== 2023-07-06T03:39:56.889 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2023-07-06T03:39:56.889 INFO:teuthology.orchestra.run.smithi146.stdout:==================================================================================================== 2023-07-06T03:39:56.889 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2023-07-06T03:39:56.890 INFO:teuthology.orchestra.run.smithi146.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-07-06T03:39:56.890 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.4 k 2023-07-06T03:39:56.890 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 5.4 M 2023-07-06T03:39:56.890 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 847 k 2023-07-06T03:39:56.890 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 165 k 2023-07-06T03:39:56.890 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 1.5 M 2023-07-06T03:39:56.890 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 129 k 2023-07-06T03:39:56.890 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 1.7 M 2023-07-06T03:39:56.891 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 7.4 M 2023-07-06T03:39:56.891 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 47 k 2023-07-06T03:39:56.891 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 13 M 2023-07-06T03:39:56.891 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 48 M 2023-07-06T03:39:56.891 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 258 k 2023-07-06T03:39:56.891 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 166 k 2023-07-06T03:39:56.891 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 258 k 2023-07-06T03:39:56.891 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 18 k 2023-07-06T03:39:56.892 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel x86_64 0.189-2.el8 baseos 61 k 2023-07-06T03:39:56.892 INFO:teuthology.orchestra.run.smithi146.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-07-06T03:39:56.892 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-07-06T03:39:56.892 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 31 k 2023-07-06T03:39:56.892 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 770 k 2023-07-06T03:39:56.892 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 139 k 2023-07-06T03:39:56.892 INFO:teuthology.orchestra.run.smithi146.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-07-06T03:39:56.893 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel x86_64 2.32.1-42.el8 baseos 99 k 2023-07-06T03:39:56.893 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-07-06T03:39:56.893 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c appstream 1.6 M 2023-07-06T03:39:56.893 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c appstream 903 k 2023-07-06T03:39:56.893 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c appstream 5.1 M 2023-07-06T03:39:56.893 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 197 k 2023-07-06T03:39:56.893 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 370 k 2023-07-06T03:39:56.894 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 364 k 2023-07-06T03:39:56.894 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 107 k 2023-07-06T03:39:56.894 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 94 k 2023-07-06T03:39:56.894 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.5 M 2023-07-06T03:39:56.894 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 179 k 2023-07-06T03:39:56.894 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-07-06T03:39:56.894 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2023-07-06T03:39:56.894 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-8.el8 baseos 143 k 2023-07-06T03:39:56.895 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-debuginfod-client x86_64 0.189-2.el8 baseos 76 k 2023-07-06T03:39:56.895 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf x86_64 0.189-2.el8 baseos 232 k 2023-07-06T03:39:56.895 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libs x86_64 0.189-2.el8 baseos 302 k 2023-07-06T03:39:56.895 INFO:teuthology.orchestra.run.smithi146.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-07-06T03:39:56.895 INFO:teuthology.orchestra.run.smithi146.stdout: libblkid x86_64 2.32.1-42.el8 baseos 220 k 2023-07-06T03:39:56.895 INFO:teuthology.orchestra.run.smithi146.stdout: libfdisk x86_64 2.32.1-42.el8 baseos 253 k 2023-07-06T03:39:56.895 INFO:teuthology.orchestra.run.smithi146.stdout: libmount x86_64 2.32.1-42.el8 baseos 236 k 2023-07-06T03:39:56.896 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.7 M 2023-07-06T03:39:56.896 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.6 M 2023-07-06T03:39:56.896 INFO:teuthology.orchestra.run.smithi146.stdout: libsmartcols x86_64 2.32.1-42.el8 baseos 179 k 2023-07-06T03:39:56.896 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid x86_64 2.32.1-42.el8 baseos 98 k 2023-07-06T03:39:56.897 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux x86_64 2.32.1-42.el8 baseos 2.6 M 2023-07-06T03:39:56.897 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-user x86_64 2.32.1-42.el8 baseos 102 k 2023-07-06T03:39:56.897 INFO:teuthology.orchestra.run.smithi146.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-07-06T03:39:56.897 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-07-06T03:39:56.897 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2023-07-06T03:39:56.897 INFO:teuthology.orchestra.run.smithi146.stdout: automake noarch 1.16.1-7.el8 appstream 713 k 2023-07-06T03:39:56.898 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 22 M 2023-07-06T03:39:56.898 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 23 k 2023-07-06T03:39:56.898 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 2.3 M 2023-07-06T03:39:56.898 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 240 k 2023-07-06T03:39:56.898 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 4.6 M 2023-07-06T03:39:56.898 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 18 M 2023-07-06T03:39:56.899 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 15 k 2023-07-06T03:39:56.899 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 24 k 2023-07-06T03:39:56.899 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-07-06T03:39:56.899 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T03:39:56.899 INFO:teuthology.orchestra.run.smithi146.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-07-06T03:39:56.899 INFO:teuthology.orchestra.run.smithi146.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-07-06T03:39:56.899 INFO:teuthology.orchestra.run.smithi146.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-07-06T03:39:56.900 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 137 k 2023-07-06T03:39:56.900 INFO:teuthology.orchestra.run.smithi146.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-07-06T03:39:56.900 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-07-06T03:39:56.900 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T03:39:56.900 INFO:teuthology.orchestra.run.smithi146.stdout: libpq x86_64 13.5-1.el8 appstream 198 k 2023-07-06T03:39:56.900 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-07-06T03:39:56.900 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-07-06T03:39:56.901 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 475 k 2023-07-06T03:39:56.901 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-07-06T03:39:56.901 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.7 M 2023-07-06T03:39:56.901 INFO:teuthology.orchestra.run.smithi146.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-07-06T03:39:56.901 INFO:teuthology.orchestra.run.smithi146.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-07-06T03:39:56.901 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T03:39:56.901 INFO:teuthology.orchestra.run.smithi146.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-07-06T03:39:56.901 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-07-06T03:39:56.902 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-07-06T03:39:56.902 INFO:teuthology.orchestra.run.smithi146.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-07-06T03:39:56.902 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-07-06T03:39:56.902 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-07-06T03:39:56.902 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-07-06T03:39:56.902 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-07-06T03:39:56.902 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-07-06T03:39:56.903 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T03:39:56.903 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-07-06T03:39:56.903 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T03:39:56.903 INFO:teuthology.orchestra.run.smithi146.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T03:39:56.903 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T03:39:56.903 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 45 k 2023-07-06T03:39:56.904 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 119 k 2023-07-06T03:39:56.904 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T03:39:56.904 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-07-06T03:39:56.904 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T03:39:56.904 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T03:39:56.904 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-07-06T03:39:56.905 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T03:39:56.905 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T03:39:56.905 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T03:39:56.905 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-07-06T03:39:56.905 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-07-06T03:39:56.905 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-07-06T03:39:56.905 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-07-06T03:39:56.906 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T03:39:56.906 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T03:39:56.906 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-07-06T03:39:56.906 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-07-06T03:39:56.906 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T03:39:56.906 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T03:39:56.907 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-07-06T03:39:56.907 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-07-06T03:39:56.907 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-07-06T03:39:56.907 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T03:39:56.907 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T03:39:56.907 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-07-06T03:39:56.908 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-07-06T03:39:56.908 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-07-06T03:39:56.908 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-07-06T03:39:56.908 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-07-06T03:39:56.908 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2017.2-9.el8 appstream 54 k 2023-07-06T03:39:56.909 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T03:39:56.909 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-07-06T03:39:56.909 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T03:39:56.909 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T03:39:56.909 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-07-06T03:39:56.909 INFO:teuthology.orchestra.run.smithi146.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T03:39:56.909 INFO:teuthology.orchestra.run.smithi146.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T03:39:56.909 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T03:39:56.910 INFO:teuthology.orchestra.run.smithi146.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T03:39:56.910 INFO:teuthology.orchestra.run.smithi146.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T03:39:56.910 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T03:39:56.910 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T03:39:56.910 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T03:39:56.910 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-07-06T03:39:56.910 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T03:39:56.911 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-07-06T03:39:56.911 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T03:39:56.911 INFO:teuthology.orchestra.run.smithi146.stdout: uuid x86_64 1.6.2-43.el8 appstream 64 k 2023-07-06T03:39:56.911 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-07-06T03:39:56.911 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-07-06T03:39:56.911 INFO:teuthology.orchestra.run.smithi146.stdout:Enabling module streams: 2023-07-06T03:39:56.911 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql 10 2023-07-06T03:39:56.912 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:39:56.912 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2023-07-06T03:39:56.912 INFO:teuthology.orchestra.run.smithi146.stdout:==================================================================================================== 2023-07-06T03:39:56.912 INFO:teuthology.orchestra.run.smithi146.stdout:Install 128 Packages 2023-07-06T03:39:56.912 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 16 Packages 2023-07-06T03:39:56.912 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:39:56.913 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 212 M 2023-07-06T03:39:56.913 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2023-07-06T03:39:57.587 INFO:teuthology.orchestra.run.smithi146.stdout:(1/144): ceph-18.0.0-4742.g79365bb3.el8.x86_64. 50 kB/s | 6.4 kB 00:00 2023-07-06T03:39:57.820 INFO:teuthology.orchestra.run.smithi146.stdout:(2/144): ceph-fuse-18.0.0-4742.g79365bb3.el8.x8 3.5 MB/s | 847 kB 00:00 2023-07-06T03:39:58.121 INFO:teuthology.orchestra.run.smithi146.stdout:(3/144): ceph-base-18.0.0-4742.g79365bb3.el8.x8 8.1 MB/s | 5.4 MB 00:00 2023-07-06T03:39:58.146 INFO:teuthology.orchestra.run.smithi146.stdout:(4/144): ceph-immutable-object-cache-18.0.0-474 508 kB/s | 165 kB 00:00 2023-07-06T03:39:58.405 INFO:teuthology.orchestra.run.smithi146.stdout:(5/144): ceph-mds-18.0.0-4742.g79365bb3.el8.x86 8.1 MB/s | 2.3 MB 00:00 2023-07-06T03:39:58.447 INFO:teuthology.orchestra.run.smithi146.stdout:(6/144): ceph-mgr-18.0.0-4742.g79365bb3.el8.x86 5.2 MB/s | 1.5 MB 00:00 2023-07-06T03:39:58.988 INFO:teuthology.orchestra.run.smithi146.stdout:(7/144): ceph-mon-18.0.0-4742.g79365bb3.el8.x86 7.9 MB/s | 4.6 MB 00:00 2023-07-06T03:39:59.439 INFO:teuthology.orchestra.run.smithi146.stdout:(8/144): ceph-common-18.0.0-4742.g79365bb3.el8. 11 MB/s | 22 MB 00:01 2023-07-06T03:39:59.522 INFO:teuthology.orchestra.run.smithi146.stdout:(9/144): ceph-selinux-18.0.0-4742.g79365bb3.el8 294 kB/s | 24 kB 00:00 2023-07-06T03:40:00.597 INFO:teuthology.orchestra.run.smithi146.stdout:(10/144): ceph-osd-18.0.0-4742.g79365bb3.el8.x8 8.2 MB/s | 18 MB 00:02 2023-07-06T03:40:00.748 INFO:teuthology.orchestra.run.smithi146.stdout:(11/144): ceph-radosgw-18.0.0-4742.g79365bb3.el 7.7 MB/s | 13 MB 00:01 2023-07-06T03:40:00.773 INFO:teuthology.orchestra.run.smithi146.stdout:(12/144): cephfs-mirror-18.0.0-4742.g79365bb3.e 1.4 MB/s | 258 kB 00:00 2023-07-06T03:40:00.798 INFO:teuthology.orchestra.run.smithi146.stdout:(13/144): libcephfs-devel-18.0.0-4742.g79365bb3 621 kB/s | 31 kB 00:00 2023-07-06T03:40:00.849 INFO:teuthology.orchestra.run.smithi146.stdout:(14/144): libcephfs2-18.0.0-4742.g79365bb3.el8. 10 MB/s | 770 kB 00:00 2023-07-06T03:40:00.874 INFO:teuthology.orchestra.run.smithi146.stdout:(15/144): libcephsqlite-18.0.0-4742.g79365bb3.e 1.8 MB/s | 137 kB 00:00 2023-07-06T03:40:00.916 INFO:teuthology.orchestra.run.smithi146.stdout:(16/144): librados-devel-18.0.0-4742.g79365bb3. 2.0 MB/s | 139 kB 00:00 2023-07-06T03:40:00.974 INFO:teuthology.orchestra.run.smithi146.stdout:(17/144): libradosstriper1-18.0.0-4742.g79365bb 4.6 MB/s | 475 kB 00:00 2023-07-06T03:40:01.041 INFO:teuthology.orchestra.run.smithi146.stdout:(18/144): python3-ceph-argparse-18.0.0-4742.g79 670 kB/s | 45 kB 00:00 2023-07-06T03:40:01.108 INFO:teuthology.orchestra.run.smithi146.stdout:(19/144): python3-ceph-common-18.0.0-4742.g7936 1.7 MB/s | 119 kB 00:00 2023-07-06T03:40:01.184 INFO:teuthology.orchestra.run.smithi146.stdout:(20/144): python3-cephfs-18.0.0-4742.g79365bb3. 2.6 MB/s | 197 kB 00:00 2023-07-06T03:40:01.267 INFO:teuthology.orchestra.run.smithi146.stdout:(21/144): python3-rados-18.0.0-4742.g79365bb3.e 4.3 MB/s | 370 kB 00:00 2023-07-06T03:40:01.352 INFO:teuthology.orchestra.run.smithi146.stdout:(22/144): python3-rbd-18.0.0-4742.g79365bb3.el8 4.3 MB/s | 364 kB 00:00 2023-07-06T03:40:01.426 INFO:teuthology.orchestra.run.smithi146.stdout:(23/144): python3-rgw-18.0.0-4742.g79365bb3.el8 1.4 MB/s | 107 kB 00:00 2023-07-06T03:40:01.501 INFO:teuthology.orchestra.run.smithi146.stdout:(24/144): rbd-fuse-18.0.0-4742.g79365bb3.el8.x8 1.2 MB/s | 94 kB 00:00 2023-07-06T03:40:01.719 INFO:teuthology.orchestra.run.smithi146.stdout:(25/144): librgw2-18.0.0-4742.g79365bb3.el8.x86 8.4 MB/s | 6.7 MB 00:00 2023-07-06T03:40:01.802 INFO:teuthology.orchestra.run.smithi146.stdout:(26/144): rbd-nbd-18.0.0-4742.g79365bb3.el8.x86 2.1 MB/s | 179 kB 00:00 2023-07-06T03:40:01.869 INFO:teuthology.orchestra.run.smithi146.stdout:(27/144): ceph-grafana-dashboards-18.0.0-4742.g 347 kB/s | 23 kB 00:00 2023-07-06T03:40:01.969 INFO:teuthology.orchestra.run.smithi146.stdout:(28/144): rbd-mirror-18.0.0-4742.g79365bb3.el8. 7.4 MB/s | 3.5 MB 00:00 2023-07-06T03:40:01.995 INFO:teuthology.orchestra.run.smithi146.stdout:(29/144): ceph-mgr-cephadm-18.0.0-4742.g79365bb 1.0 MB/s | 129 kB 00:00 2023-07-06T03:40:02.245 INFO:teuthology.orchestra.run.smithi146.stdout:(30/144): ceph-mgr-dashboard-18.0.0-4742.g79365 6.2 MB/s | 1.7 MB 00:00 2023-07-06T03:40:02.345 INFO:teuthology.orchestra.run.smithi146.stdout:(31/144): ceph-mgr-modules-core-18.0.0-4742.g79 2.4 MB/s | 240 kB 00:00 2023-07-06T03:40:02.429 INFO:teuthology.orchestra.run.smithi146.stdout:(32/144): ceph-mgr-rook-18.0.0-4742.g79365bb3.e 569 kB/s | 47 kB 00:00 2023-07-06T03:40:02.512 INFO:teuthology.orchestra.run.smithi146.stdout:(33/144): ceph-prometheus-alerts-18.0.0-4742.g7 175 kB/s | 15 kB 00:00 2023-07-06T03:40:02.613 INFO:teuthology.orchestra.run.smithi146.stdout:(34/144): ceph-volume-18.0.0-4742.g79365bb3.el8 2.5 MB/s | 258 kB 00:00 2023-07-06T03:40:02.705 INFO:teuthology.orchestra.run.smithi146.stdout:(35/144): cephadm-18.0.0-4742.g79365bb3.el8.noa 1.8 MB/s | 166 kB 00:00 2023-07-06T03:40:02.780 INFO:teuthology.orchestra.run.smithi146.stdout:(36/144): cephfs-top-18.0.0-4742.g79365bb3.el8. 240 kB/s | 18 kB 00:00 2023-07-06T03:40:03.272 INFO:teuthology.orchestra.run.smithi146.stdout:(37/144): ceph-test-18.0.0-4742.g79365bb3.el8.x 13 MB/s | 48 MB 00:03 2023-07-06T03:40:03.372 INFO:teuthology.orchestra.run.smithi146.stdout:(38/144): ceph-mgr-diskprediction-local-18.0.0- 5.4 MB/s | 7.4 MB 00:01 2023-07-06T03:40:03.473 INFO:teuthology.orchestra.run.smithi146.stdout:(39/144): automake-1.16.1-7.el8.noarch.rpm 1.0 MB/s | 713 kB 00:00 2023-07-06T03:40:03.498 INFO:teuthology.orchestra.run.smithi146.stdout:(40/144): flex-2.6.1-9.el8.x86_64.rpm 2.5 MB/s | 320 kB 00:00 2023-07-06T03:40:03.523 INFO:teuthology.orchestra.run.smithi146.stdout:(41/144): jq-1.6-7.el8.x86_64.rpm 3.9 MB/s | 202 kB 00:00 2023-07-06T03:40:03.557 INFO:teuthology.orchestra.run.smithi146.stdout:(42/144): bison-3.0.4-10.el8.x86_64.rpm 2.4 MB/s | 688 kB 00:00 2023-07-06T03:40:03.582 INFO:teuthology.orchestra.run.smithi146.stdout:(43/144): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2023-07-06T03:40:03.607 INFO:teuthology.orchestra.run.smithi146.stdout:(44/144): librdkafka-0.11.4-3.el8.x86_64.rpm 4.1 MB/s | 354 kB 00:00 2023-07-06T03:40:03.641 INFO:teuthology.orchestra.run.smithi146.stdout:(45/144): libtool-2.4.6-25.el8.x86_64.rpm 8.3 MB/s | 709 kB 00:00 2023-07-06T03:40:03.666 INFO:teuthology.orchestra.run.smithi146.stdout:(46/144): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2023-07-06T03:40:03.691 INFO:teuthology.orchestra.run.smithi146.stdout:(47/144): oniguruma-6.8.2-2.el8.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2023-07-06T03:40:03.850 INFO:teuthology.orchestra.run.smithi146.stdout:(48/144): postgresql-10.23-1.module_el8+300+2c5 9.9 MB/s | 1.6 MB 00:00 2023-07-06T03:40:03.976 INFO:teuthology.orchestra.run.smithi146.stdout:(49/144): openblas-0.3.15-4.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-07-06T03:40:04.009 INFO:teuthology.orchestra.run.smithi146.stdout:(50/144): postgresql-contrib-10.23-1.module_el8 5.6 MB/s | 903 kB 00:00 2023-07-06T03:40:04.084 INFO:teuthology.orchestra.run.smithi146.stdout:(51/144): openblas-threads-0.3.15-4.el8.x86_64. 12 MB/s | 4.9 MB 00:00 2023-07-06T03:40:04.184 INFO:teuthology.orchestra.run.smithi146.stdout:(52/144): python3-jinja2-2.10.1-3.el8.noarch.rp 5.3 MB/s | 538 kB 00:00 2023-07-06T03:40:04.218 INFO:teuthology.orchestra.run.smithi146.stdout:(53/144): python3-jsonpatch-1.21-2.el8.noarch.r 794 kB/s | 27 kB 00:00 2023-07-06T03:40:04.252 INFO:teuthology.orchestra.run.smithi146.stdout:(54/144): python3-jsonpointer-1.10-11.el8.noarc 605 kB/s | 20 kB 00:00 2023-07-06T03:40:04.310 INFO:teuthology.orchestra.run.smithi146.stdout:(55/144): python3-mako-1.0.6-14.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-06T03:40:04.344 INFO:teuthology.orchestra.run.smithi146.stdout:(56/144): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-07-06T03:40:04.503 INFO:teuthology.orchestra.run.smithi146.stdout:(57/144): postgresql-server-10.23-1.module_el8+ 9.7 MB/s | 5.1 MB 00:00 2023-07-06T03:40:04.586 INFO:teuthology.orchestra.run.smithi146.stdout:(58/144): python3-babel-2.5.1-7.el8.noarch.rpm 8.3 MB/s | 4.8 MB 00:00 2023-07-06T03:40:04.612 INFO:teuthology.orchestra.run.smithi146.stdout:(59/144): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.0 MB/s | 225 kB 00:00 2023-07-06T03:40:04.637 INFO:teuthology.orchestra.run.smithi146.stdout:(60/144): python3-prettytable-0.7.2-14.el8.noar 879 kB/s | 44 kB 00:00 2023-07-06T03:40:04.663 INFO:teuthology.orchestra.run.smithi146.stdout:(61/144): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-07-06T03:40:04.688 INFO:teuthology.orchestra.run.smithi146.stdout:(62/144): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-07-06T03:40:04.705 INFO:teuthology.orchestra.run.smithi146.stdout:(63/144): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-06T03:40:04.730 INFO:teuthology.orchestra.run.smithi146.stdout:(64/144): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-06T03:40:04.805 INFO:teuthology.orchestra.run.smithi146.stdout:(65/144): python3-werkzeug-0.12.2-4.el8.noarch. 6.0 MB/s | 457 kB 00:00 2023-07-06T03:40:04.872 INFO:teuthology.orchestra.run.smithi146.stdout:(66/144): python3-numpy-1.14.3-10.el8.x86_64.rp 7.1 MB/s | 3.7 MB 00:00 2023-07-06T03:40:04.897 INFO:teuthology.orchestra.run.smithi146.stdout:(67/144): socat-1.7.4.1-1.el8.x86_64.rpm 3.4 MB/s | 323 kB 00:00 2023-07-06T03:40:04.922 INFO:teuthology.orchestra.run.smithi146.stdout:(68/144): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-07-06T03:40:04.948 INFO:teuthology.orchestra.run.smithi146.stdout:(69/144): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2023-07-06T03:40:05.223 INFO:teuthology.orchestra.run.smithi146.stdout:(70/144): elfutils-libelf-devel-0.189-2.el8.x86 205 kB/s | 61 kB 00:00 2023-07-06T03:40:05.248 INFO:teuthology.orchestra.run.smithi146.stdout:(71/144): fuse-2.9.7-17.el8.x86_64.rpm 276 kB/s | 83 kB 00:00 2023-07-06T03:40:05.323 INFO:teuthology.orchestra.run.smithi146.stdout:(72/144): keyutils-libs-devel-1.5.10-9.el8.x86_ 480 kB/s | 48 kB 00:00 2023-07-06T03:40:05.424 INFO:teuthology.orchestra.run.smithi146.stdout:(73/144): libaio-devel-0.3.112-1.el8.x86_64.rpm 187 kB/s | 19 kB 00:00 2023-07-06T03:40:05.499 INFO:teuthology.orchestra.run.smithi146.stdout:(74/144): krb5-devel-1.18.2-22.el8.x86_64.rpm 2.2 MB/s | 561 kB 00:00 2023-07-06T03:40:05.524 INFO:teuthology.orchestra.run.smithi146.stdout:(75/144): libcom_err-devel-1.45.6-5.el8.x86_64. 387 kB/s | 39 kB 00:00 2023-07-06T03:40:05.675 INFO:teuthology.orchestra.run.smithi146.stdout:(76/144): python3-scipy-1.0.0-21.module_el8.5.0 14 MB/s | 14 MB 00:00 2023-07-06T03:40:05.708 INFO:teuthology.orchestra.run.smithi146.stdout:(77/144): libgfortran-8.5.0-20.el8.x86_64.rpm 3.0 MB/s | 645 kB 00:00 2023-07-06T03:40:05.733 INFO:teuthology.orchestra.run.smithi146.stdout:(78/144): libquadmath-8.5.0-20.el8.x86_64.rpm 825 kB/s | 172 kB 00:00 2023-07-06T03:40:05.817 INFO:teuthology.orchestra.run.smithi146.stdout:(79/144): libselinux-devel-2.9-8.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2023-07-06T03:40:05.843 INFO:teuthology.orchestra.run.smithi146.stdout:(80/144): libsepol-devel-2.9-3.el8.x86_64.rpm 801 kB/s | 87 kB 00:00 2023-07-06T03:40:05.918 INFO:teuthology.orchestra.run.smithi146.stdout:(81/144): libuuid-devel-2.32.1-42.el8.x86_64.rp 992 kB/s | 99 kB 00:00 2023-07-06T03:40:05.943 INFO:teuthology.orchestra.run.smithi146.stdout:(82/144): libverto-devel-0.3.2-2.el8.x86_64.rpm 180 kB/s | 18 kB 00:00 2023-07-06T03:40:05.985 INFO:teuthology.orchestra.run.smithi146.stdout:(83/144): librabbitmq-0.9.0-4.el8.x86_64.rpm 151 kB/s | 47 kB 00:00 2023-07-06T03:40:06.019 INFO:teuthology.orchestra.run.smithi146.stdout:(84/144): libzstd-devel-1.4.4-1.el8.x86_64.rpm 436 kB/s | 44 kB 00:00 2023-07-06T03:40:06.144 INFO:teuthology.orchestra.run.smithi146.stdout:(85/144): pcre2-utf16-10.32-3.el8.x86_64.rpm 1.8 MB/s | 229 kB 00:00 2023-07-06T03:40:06.261 INFO:teuthology.orchestra.run.smithi146.stdout:(86/144): pcre2-utf32-10.32-3.el8.x86_64.rpm 1.9 MB/s | 220 kB 00:00 2023-07-06T03:40:06.320 INFO:teuthology.orchestra.run.smithi146.stdout:(87/144): pcre2-devel-10.32-3.el8.x86_64.rpm 1.8 MB/s | 605 kB 00:00 2023-07-06T03:40:06.370 INFO:teuthology.orchestra.run.smithi146.stdout:(88/144): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.2 MB/s | 249 kB 00:00 2023-07-06T03:40:06.470 INFO:teuthology.orchestra.run.smithi146.stdout:(89/144): python3-jwt-1.6.1-2.el8.noarch.rpm 426 kB/s | 43 kB 00:00 2023-07-06T03:40:06.496 INFO:teuthology.orchestra.run.smithi146.stdout:(90/144): python3-cryptography-3.2.1-6.el8.x86_ 3.6 MB/s | 641 kB 00:00 2023-07-06T03:40:06.546 INFO:teuthology.orchestra.run.smithi146.stdout:(91/144): openssl-devel-1.1.1k-9.el8.x86_64.rpm 5.3 MB/s | 3.2 MB 00:00 2023-07-06T03:40:06.579 INFO:teuthology.orchestra.run.smithi146.stdout:(92/144): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2023-07-06T03:40:06.605 INFO:teuthology.orchestra.run.smithi146.stdout:(93/144): python3-pycparser-2.14-14.el8.noarch. 1.0 MB/s | 109 kB 00:00 2023-07-06T03:40:06.646 INFO:teuthology.orchestra.run.smithi146.stdout:(94/144): python3-requests-oauthlib-1.0.0-1.el8 424 kB/s | 43 kB 00:00 2023-07-06T03:40:06.680 INFO:teuthology.orchestra.run.smithi146.stdout:(95/144): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 742 kB/s | 74 kB 00:00 2023-07-06T03:40:06.705 INFO:teuthology.orchestra.run.smithi146.stdout:(96/144): zlib-devel-1.2.11-21.el8.x86_64.rpm 578 kB/s | 58 kB 00:00 2023-07-06T03:40:15.174 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 541 kB/s | 16 MB 00:29 2023-07-06T03:40:19.496 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 392 kB/s | 24 kB 00:00 2023-07-06T03:40:23.462 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T03:40:23.463 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T03:40:23.476 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-07-06T03:40:23.476 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-07-06T03:40:23.477 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-07-06T03:40:23.478 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-07-06T03:40:23.478 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T03:40:23.480 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T03:40:23.483 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T03:40:23.484 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T03:40:23.672 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-07-06T03:40:23.686 INFO:teuthology.orchestra.run.smithi097.stdout:==================================================================================================== 2023-07-06T03:40:23.686 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2023-07-06T03:40:23.687 INFO:teuthology.orchestra.run.smithi097.stdout:==================================================================================================== 2023-07-06T03:40:23.687 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2023-07-06T03:40:23.687 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-07-06T03:40:23.687 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.4 k 2023-07-06T03:40:23.688 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 5.4 M 2023-07-06T03:40:23.688 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 847 k 2023-07-06T03:40:23.688 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 165 k 2023-07-06T03:40:23.688 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 1.5 M 2023-07-06T03:40:23.689 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 129 k 2023-07-06T03:40:23.689 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 1.7 M 2023-07-06T03:40:23.689 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 7.4 M 2023-07-06T03:40:23.690 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 47 k 2023-07-06T03:40:23.690 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 13 M 2023-07-06T03:40:23.690 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 48 M 2023-07-06T03:40:23.690 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 258 k 2023-07-06T03:40:23.690 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 166 k 2023-07-06T03:40:23.691 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 258 k 2023-07-06T03:40:23.691 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 18 k 2023-07-06T03:40:23.691 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.189-2.el8 baseos 61 k 2023-07-06T03:40:23.692 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-07-06T03:40:23.692 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-07-06T03:40:23.692 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 31 k 2023-07-06T03:40:23.692 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 770 k 2023-07-06T03:40:23.693 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 139 k 2023-07-06T03:40:23.693 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-07-06T03:40:23.693 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.32.1-42.el8 baseos 99 k 2023-07-06T03:40:23.694 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-07-06T03:40:23.694 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c appstream 1.6 M 2023-07-06T03:40:23.694 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c appstream 903 k 2023-07-06T03:40:23.695 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c appstream 5.1 M 2023-07-06T03:40:23.695 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 197 k 2023-07-06T03:40:23.695 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 370 k 2023-07-06T03:40:23.695 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 364 k 2023-07-06T03:40:23.696 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 107 k 2023-07-06T03:40:23.696 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 94 k 2023-07-06T03:40:23.696 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.5 M 2023-07-06T03:40:23.696 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 179 k 2023-07-06T03:40:23.697 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-07-06T03:40:23.697 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2023-07-06T03:40:23.697 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-8.el8 baseos 143 k 2023-07-06T03:40:23.697 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client x86_64 0.189-2.el8 baseos 76 k 2023-07-06T03:40:23.697 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf x86_64 0.189-2.el8 baseos 232 k 2023-07-06T03:40:23.698 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs x86_64 0.189-2.el8 baseos 302 k 2023-07-06T03:40:23.698 INFO:teuthology.orchestra.run.smithi097.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-07-06T03:40:23.698 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid x86_64 2.32.1-42.el8 baseos 220 k 2023-07-06T03:40:23.699 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk x86_64 2.32.1-42.el8 baseos 253 k 2023-07-06T03:40:23.699 INFO:teuthology.orchestra.run.smithi097.stdout: libmount x86_64 2.32.1-42.el8 baseos 236 k 2023-07-06T03:40:23.699 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.7 M 2023-07-06T03:40:23.699 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.6 M 2023-07-06T03:40:23.699 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols x86_64 2.32.1-42.el8 baseos 179 k 2023-07-06T03:40:23.699 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid x86_64 2.32.1-42.el8 baseos 98 k 2023-07-06T03:40:23.700 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux x86_64 2.32.1-42.el8 baseos 2.6 M 2023-07-06T03:40:23.700 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user x86_64 2.32.1-42.el8 baseos 102 k 2023-07-06T03:40:23.700 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-07-06T03:40:23.700 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-07-06T03:40:23.700 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2023-07-06T03:40:23.700 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.1-7.el8 appstream 713 k 2023-07-06T03:40:23.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 22 M 2023-07-06T03:40:23.701 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 23 k 2023-07-06T03:40:23.701 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 2.3 M 2023-07-06T03:40:23.701 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 240 k 2023-07-06T03:40:23.701 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 4.6 M 2023-07-06T03:40:23.701 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 18 M 2023-07-06T03:40:23.701 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 15 k 2023-07-06T03:40:23.702 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 24 k 2023-07-06T03:40:23.702 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-07-06T03:40:23.702 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T03:40:23.702 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-07-06T03:40:23.702 INFO:teuthology.orchestra.run.smithi097.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-07-06T03:40:23.702 INFO:teuthology.orchestra.run.smithi097.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-07-06T03:40:23.703 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 137 k 2023-07-06T03:40:23.703 INFO:teuthology.orchestra.run.smithi097.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-07-06T03:40:23.703 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-07-06T03:40:23.703 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T03:40:23.703 INFO:teuthology.orchestra.run.smithi097.stdout: libpq x86_64 13.5-1.el8 appstream 198 k 2023-07-06T03:40:23.703 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-07-06T03:40:23.703 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-07-06T03:40:23.703 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 475 k 2023-07-06T03:40:23.704 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-07-06T03:40:23.704 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.7 M 2023-07-06T03:40:23.704 INFO:teuthology.orchestra.run.smithi097.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-07-06T03:40:23.704 INFO:teuthology.orchestra.run.smithi097.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-07-06T03:40:23.704 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T03:40:23.704 INFO:teuthology.orchestra.run.smithi097.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-07-06T03:40:23.704 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-07-06T03:40:23.705 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-07-06T03:40:23.705 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-07-06T03:40:23.705 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-07-06T03:40:23.705 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-07-06T03:40:23.705 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-07-06T03:40:23.705 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-07-06T03:40:23.705 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-07-06T03:40:23.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T03:40:23.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-07-06T03:40:23.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T03:40:23.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T03:40:23.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T03:40:23.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 45 k 2023-07-06T03:40:23.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 119 k 2023-07-06T03:40:23.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T03:40:23.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-07-06T03:40:23.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T03:40:23.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T03:40:23.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-07-06T03:40:23.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T03:40:23.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T03:40:23.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T03:40:23.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-07-06T03:40:23.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-07-06T03:40:23.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-07-06T03:40:23.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-07-06T03:40:23.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T03:40:23.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T03:40:23.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-07-06T03:40:23.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-07-06T03:40:23.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T03:40:23.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T03:40:23.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-07-06T03:40:23.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-07-06T03:40:23.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-07-06T03:40:23.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T03:40:23.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T03:40:23.709 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-07-06T03:40:23.710 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-07-06T03:40:23.710 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-07-06T03:40:23.710 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-07-06T03:40:23.710 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-07-06T03:40:23.710 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2017.2-9.el8 appstream 54 k 2023-07-06T03:40:23.710 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T03:40:23.710 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-07-06T03:40:23.711 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T03:40:23.711 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T03:40:23.711 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-07-06T03:40:23.711 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T03:40:23.711 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T03:40:23.711 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T03:40:23.711 INFO:teuthology.orchestra.run.smithi097.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T03:40:23.712 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T03:40:23.712 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T03:40:23.712 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T03:40:23.712 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T03:40:23.712 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-07-06T03:40:23.712 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T03:40:23.713 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-07-06T03:40:23.713 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T03:40:23.713 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-43.el8 appstream 64 k 2023-07-06T03:40:23.713 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-07-06T03:40:23.713 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-07-06T03:40:23.713 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling module streams: 2023-07-06T03:40:23.713 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql 10 2023-07-06T03:40:23.714 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:40:23.714 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-07-06T03:40:23.714 INFO:teuthology.orchestra.run.smithi097.stdout:==================================================================================================== 2023-07-06T03:40:23.714 INFO:teuthology.orchestra.run.smithi097.stdout:Install 128 Packages 2023-07-06T03:40:23.714 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 16 Packages 2023-07-06T03:40:23.714 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:40:23.715 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 212 M 2023-07-06T03:40:23.715 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2023-07-06T03:40:24.173 INFO:teuthology.orchestra.run.smithi097.stdout:(1/144): ceph-18.0.0-4742.g79365bb3.el8.x86_64. 33 kB/s | 6.4 kB 00:00 2023-07-06T03:40:24.600 INFO:teuthology.orchestra.run.smithi097.stdout:(2/144): ceph-base-18.0.0-4742.g79365bb3.el8.x8 8.7 MB/s | 5.4 MB 00:00 2023-07-06T03:40:24.665 INFO:teuthology.orchestra.run.smithi097.stdout:(3/144): ceph-immutable-object-cache-18.0.0-474 2.4 MB/s | 165 kB 00:00 2023-07-06T03:40:24.748 INFO:teuthology.orchestra.run.smithi097.stdout:(4/144): ceph-fuse-18.0.0-4742.g79365bb3.el8.x8 1.4 MB/s | 847 kB 00:00 2023-07-06T03:40:24.907 INFO:teuthology.orchestra.run.smithi097.stdout:(5/144): ceph-mds-18.0.0-4742.g79365bb3.el8.x86 9.4 MB/s | 2.3 MB 00:00 2023-07-06T03:40:25.390 INFO:teuthology.orchestra.run.smithi097.stdout:(6/144): ceph-mon-18.0.0-4742.g79365bb3.el8.x86 9.5 MB/s | 4.6 MB 00:00 2023-07-06T03:40:25.707 INFO:teuthology.orchestra.run.smithi097.stdout:(7/144): ceph-common-18.0.0-4742.g79365bb3.el8. 13 MB/s | 22 MB 00:01 2023-07-06T03:40:25.749 INFO:teuthology.orchestra.run.smithi097.stdout:(8/144): ceph-mgr-18.0.0-4742.g79365bb3.el8.x86 1.5 MB/s | 1.5 MB 00:01 2023-07-06T03:40:25.833 INFO:teuthology.orchestra.run.smithi097.stdout:(9/144): ceph-selinux-18.0.0-4742.g79365bb3.el8 293 kB/s | 24 kB 00:00 2023-07-06T03:40:26.791 INFO:teuthology.orchestra.run.smithi097.stdout:(10/144): ceph-radosgw-18.0.0-4742.g79365bb3.el 12 MB/s | 13 MB 00:01 2023-07-06T03:40:26.975 INFO:teuthology.orchestra.run.smithi097.stdout:(11/144): ceph-osd-18.0.0-4742.g79365bb3.el8.x8 11 MB/s | 18 MB 00:01 2023-07-06T03:40:27.000 INFO:teuthology.orchestra.run.smithi097.stdout:(12/144): cephfs-mirror-18.0.0-4742.g79365bb3.e 1.2 MB/s | 258 kB 00:00 2023-07-06T03:40:27.026 INFO:teuthology.orchestra.run.smithi097.stdout:(13/144): libcephfs-devel-18.0.0-4742.g79365bb3 619 kB/s | 31 kB 00:00 2023-07-06T03:40:27.076 INFO:teuthology.orchestra.run.smithi097.stdout:(14/144): libcephfs2-18.0.0-4742.g79365bb3.el8. 10 MB/s | 770 kB 00:00 2023-07-06T03:40:27.120 INFO:teuthology.orchestra.run.smithi097.stdout:(15/144): libcephsqlite-18.0.0-4742.g79365bb3.e 1.4 MB/s | 137 kB 00:00 2023-07-06T03:40:27.152 INFO:teuthology.orchestra.run.smithi097.stdout:(16/144): librados-devel-18.0.0-4742.g79365bb3. 1.8 MB/s | 139 kB 00:00 2023-07-06T03:40:27.211 INFO:teuthology.orchestra.run.smithi097.stdout:(17/144): libradosstriper1-18.0.0-4742.g79365bb 5.1 MB/s | 475 kB 00:00 2023-07-06T03:40:27.277 INFO:teuthology.orchestra.run.smithi097.stdout:(18/144): python3-ceph-argparse-18.0.0-4742.g79 674 kB/s | 45 kB 00:00 2023-07-06T03:40:27.353 INFO:teuthology.orchestra.run.smithi097.stdout:(19/144): python3-ceph-common-18.0.0-4742.g7936 1.6 MB/s | 119 kB 00:00 2023-07-06T03:40:27.420 INFO:teuthology.orchestra.run.smithi097.stdout:(20/144): python3-cephfs-18.0.0-4742.g79365bb3. 2.9 MB/s | 197 kB 00:00 2023-07-06T03:40:27.537 INFO:teuthology.orchestra.run.smithi097.stdout:(21/144): librgw2-18.0.0-4742.g79365bb3.el8.x86 18 MB/s | 6.7 MB 00:00 2023-07-06T03:40:27.570 INFO:teuthology.orchestra.run.smithi097.stdout:(22/144): python3-rados-18.0.0-4742.g79365bb3.e 2.4 MB/s | 370 kB 00:00 2023-07-06T03:40:27.596 INFO:teuthology.orchestra.run.smithi097.stdout:(23/144): python3-rbd-18.0.0-4742.g79365bb3.el8 6.1 MB/s | 364 kB 00:00 2023-07-06T03:40:27.621 INFO:teuthology.orchestra.run.smithi097.stdout:(24/144): python3-rgw-18.0.0-4742.g79365bb3.el8 2.1 MB/s | 107 kB 00:00 2023-07-06T03:40:27.647 INFO:teuthology.orchestra.run.smithi097.stdout:(25/144): rbd-fuse-18.0.0-4742.g79365bb3.el8.x8 1.8 MB/s | 94 kB 00:00 2023-07-06T03:40:27.713 INFO:teuthology.orchestra.run.smithi097.stdout:(26/144): rbd-nbd-18.0.0-4742.g79365bb3.el8.x86 2.6 MB/s | 179 kB 00:00 2023-07-06T03:40:27.771 INFO:teuthology.orchestra.run.smithi097.stdout:(27/144): ceph-grafana-dashboards-18.0.0-4742.g 397 kB/s | 23 kB 00:00 2023-07-06T03:40:27.830 INFO:teuthology.orchestra.run.smithi097.stdout:(28/144): rbd-mirror-18.0.0-4742.g79365bb3.el8. 17 MB/s | 3.5 MB 00:00 2023-07-06T03:40:27.855 INFO:teuthology.orchestra.run.smithi097.stdout:(29/144): ceph-mgr-cephadm-18.0.0-4742.g79365bb 1.5 MB/s | 129 kB 00:00 2023-07-06T03:40:27.972 INFO:teuthology.orchestra.run.smithi097.stdout:(30/144): ceph-mgr-dashboard-18.0.0-4742.g79365 12 MB/s | 1.7 MB 00:00 2023-07-06T03:40:28.056 INFO:teuthology.orchestra.run.smithi097.stdout:(31/144): ceph-mgr-modules-core-18.0.0-4742.g79 2.8 MB/s | 240 kB 00:00 2023-07-06T03:40:28.131 INFO:teuthology.orchestra.run.smithi097.stdout:(32/144): ceph-mgr-rook-18.0.0-4742.g79365bb3.e 632 kB/s | 47 kB 00:00 2023-07-06T03:40:28.190 INFO:teuthology.orchestra.run.smithi097.stdout:(33/144): ceph-prometheus-alerts-18.0.0-4742.g7 249 kB/s | 15 kB 00:00 2023-07-06T03:40:28.299 INFO:teuthology.orchestra.run.smithi097.stdout:(34/144): ceph-mgr-diskprediction-local-18.0.0- 17 MB/s | 7.4 MB 00:00 2023-07-06T03:40:28.324 INFO:teuthology.orchestra.run.smithi097.stdout:(35/144): ceph-volume-18.0.0-4742.g79365bb3.el8 1.9 MB/s | 258 kB 00:00 2023-07-06T03:40:28.349 INFO:teuthology.orchestra.run.smithi097.stdout:(36/144): cephadm-18.0.0-4742.g79365bb3.el8.noa 3.3 MB/s | 166 kB 00:00 2023-07-06T03:40:28.374 INFO:teuthology.orchestra.run.smithi097.stdout:(37/144): cephfs-top-18.0.0-4742.g79365bb3.el8. 358 kB/s | 18 kB 00:00 2023-07-06T03:40:28.492 INFO:teuthology.orchestra.run.smithi097.stdout:(38/144): automake-1.16.1-7.el8.noarch.rpm 4.9 MB/s | 713 kB 00:00 2023-07-06T03:40:28.525 INFO:teuthology.orchestra.run.smithi097.stdout:(39/144): bison-3.0.4-10.el8.x86_64.rpm 4.5 MB/s | 688 kB 00:00 2023-07-06T03:40:28.567 INFO:teuthology.orchestra.run.smithi097.stdout:(40/144): jq-1.6-7.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-06T03:40:28.601 INFO:teuthology.orchestra.run.smithi097.stdout:(41/144): flex-2.6.1-9.el8.x86_64.rpm 4.2 MB/s | 320 kB 00:00 2023-07-06T03:40:28.634 INFO:teuthology.orchestra.run.smithi097.stdout:(42/144): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-07-06T03:40:28.660 INFO:teuthology.orchestra.run.smithi097.stdout:(43/144): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-07-06T03:40:28.710 INFO:teuthology.orchestra.run.smithi097.stdout:(44/144): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-07-06T03:40:28.735 INFO:teuthology.orchestra.run.smithi097.stdout:(45/144): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2023-07-06T03:40:28.761 INFO:teuthology.orchestra.run.smithi097.stdout:(46/144): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-07-06T03:40:28.936 INFO:teuthology.orchestra.run.smithi097.stdout:(47/144): openblas-0.3.15-4.el8.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2023-07-06T03:40:29.011 INFO:teuthology.orchestra.run.smithi097.stdout:(48/144): openblas-threads-0.3.15-4.el8.x86_64. 19 MB/s | 4.9 MB 00:00 2023-07-06T03:40:29.062 INFO:teuthology.orchestra.run.smithi097.stdout:(49/144): postgresql-10.23-1.module_el8+300+2c5 13 MB/s | 1.6 MB 00:00 2023-07-06T03:40:29.095 INFO:teuthology.orchestra.run.smithi097.stdout:(50/144): postgresql-contrib-10.23-1.module_el8 11 MB/s | 903 kB 00:00 2023-07-06T03:40:29.304 INFO:teuthology.orchestra.run.smithi097.stdout:(51/144): postgresql-server-10.23-1.module_el8+ 21 MB/s | 5.1 MB 00:00 2023-07-06T03:40:29.379 INFO:teuthology.orchestra.run.smithi097.stdout:(52/144): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-07-06T03:40:29.413 INFO:teuthology.orchestra.run.smithi097.stdout:(53/144): python3-jinja2-2.10.1-3.el8.noarch.rp 4.9 MB/s | 538 kB 00:00 2023-07-06T03:40:29.438 INFO:teuthology.orchestra.run.smithi097.stdout:(54/144): python3-jsonpatch-1.21-2.el8.noarch.r 454 kB/s | 27 kB 00:00 2023-07-06T03:40:29.463 INFO:teuthology.orchestra.run.smithi097.stdout:(55/144): python3-jsonpointer-1.10-11.el8.noarc 397 kB/s | 20 kB 00:00 2023-07-06T03:40:29.489 INFO:teuthology.orchestra.run.smithi097.stdout:(56/144): python3-mako-1.0.6-14.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-07-06T03:40:29.514 INFO:teuthology.orchestra.run.smithi097.stdout:(57/144): python3-markupsafe-0.23-19.el8.x86_64 770 kB/s | 39 kB 00:00 2023-07-06T03:40:29.556 INFO:teuthology.orchestra.run.smithi097.stdout:(58/144): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-07-06T03:40:29.589 INFO:teuthology.orchestra.run.smithi097.stdout:(59/144): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-07-06T03:40:29.631 INFO:teuthology.orchestra.run.smithi097.stdout:(60/144): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-07-06T03:40:29.698 INFO:teuthology.orchestra.run.smithi097.stdout:(61/144): python3-numpy-1.14.3-10.el8.x86_64.rp 18 MB/s | 3.7 MB 00:00 2023-07-06T03:40:29.724 INFO:teuthology.orchestra.run.smithi097.stdout:(62/144): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.3 MB/s | 126 kB 00:00 2023-07-06T03:40:29.749 INFO:teuthology.orchestra.run.smithi097.stdout:(63/144): python3-pyasn1-modules-0.3.7-6.el8.no 2.1 MB/s | 110 kB 00:00 2023-07-06T03:40:29.774 INFO:teuthology.orchestra.run.smithi097.stdout:(64/144): python3-pytz-2017.2-9.el8.noarch.rpm 1.0 MB/s | 54 kB 00:00 2023-07-06T03:40:29.825 INFO:teuthology.orchestra.run.smithi097.stdout:(65/144): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-07-06T03:40:29.879 INFO:teuthology.orchestra.run.smithi097.stdout:(66/144): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-07-06T03:40:29.909 INFO:teuthology.orchestra.run.smithi097.stdout:(67/144): uuid-1.6.2-43.el8.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2023-07-06T03:40:29.943 INFO:teuthology.orchestra.run.smithi097.stdout:(68/144): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.1 MB/s | 71 kB 00:00 2023-07-06T03:40:30.226 INFO:teuthology.orchestra.run.smithi097.stdout:(69/144): python3-scipy-1.0.0-21.module_el8.5.0 29 MB/s | 14 MB 00:00 2023-07-06T03:40:30.277 INFO:teuthology.orchestra.run.smithi097.stdout:(70/144): elfutils-libelf-devel-0.189-2.el8.x86 184 kB/s | 61 kB 00:00 2023-07-06T03:40:30.369 INFO:teuthology.orchestra.run.smithi097.stdout:(71/144): keyutils-libs-devel-1.5.10-9.el8.x86_ 524 kB/s | 48 kB 00:00 2023-07-06T03:40:30.535 INFO:teuthology.orchestra.run.smithi097.stdout:(72/144): fuse-2.9.7-17.el8.x86_64.rpm 269 kB/s | 83 kB 00:00 2023-07-06T03:40:30.644 INFO:teuthology.orchestra.run.smithi097.stdout:(73/144): krb5-devel-1.18.2-22.el8.x86_64.rpm 2.0 MB/s | 561 kB 00:00 2023-07-06T03:40:30.661 INFO:teuthology.orchestra.run.smithi097.stdout:(74/144): libaio-devel-0.3.112-1.el8.x86_64.rpm 149 kB/s | 19 kB 00:00 2023-07-06T03:40:30.737 INFO:teuthology.orchestra.run.smithi097.stdout:(75/144): libcom_err-devel-1.45.6-5.el8.x86_64. 421 kB/s | 39 kB 00:00 2023-07-06T03:40:30.853 INFO:teuthology.orchestra.run.smithi097.stdout:(76/144): libquadmath-8.5.0-20.el8.x86_64.rpm 1.4 MB/s | 172 kB 00:00 2023-07-06T03:40:30.954 INFO:teuthology.orchestra.run.smithi097.stdout:(77/144): librabbitmq-0.9.0-4.el8.x86_64.rpm 470 kB/s | 47 kB 00:00 2023-07-06T03:40:30.979 INFO:teuthology.orchestra.run.smithi097.stdout:(78/144): libgfortran-8.5.0-20.el8.x86_64.rpm 2.0 MB/s | 645 kB 00:00 2023-07-06T03:40:31.063 INFO:teuthology.orchestra.run.smithi097.stdout:(79/144): libselinux-devel-2.9-8.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2023-07-06T03:40:31.088 INFO:teuthology.orchestra.run.smithi097.stdout:(80/144): libsepol-devel-2.9-3.el8.x86_64.rpm 800 kB/s | 87 kB 00:00 2023-07-06T03:40:31.163 INFO:teuthology.orchestra.run.smithi097.stdout:(81/144): libuuid-devel-2.32.1-42.el8.x86_64.rp 991 kB/s | 99 kB 00:00 2023-07-06T03:40:31.180 INFO:teuthology.orchestra.run.smithi097.stdout:(82/144): libverto-devel-0.3.2-2.el8.x86_64.rpm 196 kB/s | 18 kB 00:00 2023-07-06T03:40:31.255 INFO:teuthology.orchestra.run.smithi097.stdout:(83/144): libzstd-devel-1.4.4-1.el8.x86_64.rpm 478 kB/s | 44 kB 00:00 2023-07-06T03:40:31.439 INFO:teuthology.orchestra.run.smithi097.stdout:(84/144): pcre2-devel-10.32-3.el8.x86_64.rpm 3.2 MB/s | 605 kB 00:00 2023-07-06T03:40:31.548 INFO:teuthology.orchestra.run.smithi097.stdout:(85/144): pcre2-utf16-10.32-3.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2023-07-06T03:40:31.665 INFO:teuthology.orchestra.run.smithi097.stdout:(86/144): pcre2-utf32-10.32-3.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-07-06T03:40:31.715 INFO:teuthology.orchestra.run.smithi097.stdout:(87/144): openssl-devel-1.1.1k-9.el8.x86_64.rpm 5.9 MB/s | 3.2 MB 00:00 2023-07-06T03:40:31.774 INFO:teuthology.orchestra.run.smithi097.stdout:(88/144): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.2 MB/s | 249 kB 00:00 2023-07-06T03:40:31.866 INFO:teuthology.orchestra.run.smithi097.stdout:(89/144): python3-jwt-1.6.1-2.el8.noarch.rpm 466 kB/s | 43 kB 00:00 2023-07-06T03:40:31.899 INFO:teuthology.orchestra.run.smithi097.stdout:(90/144): python3-cryptography-3.2.1-6.el8.x86_ 3.4 MB/s | 641 kB 00:00 2023-07-06T03:40:31.974 INFO:teuthology.orchestra.run.smithi097.stdout:(91/144): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2023-07-06T03:40:32.000 INFO:teuthology.orchestra.run.smithi097.stdout:(92/144): python3-pycparser-2.14-14.el8.noarch. 1.1 MB/s | 109 kB 00:00 2023-07-06T03:40:32.067 INFO:teuthology.orchestra.run.smithi097.stdout:(93/144): python3-requests-oauthlib-1.0.0-1.el8 465 kB/s | 43 kB 00:00 2023-07-06T03:40:32.092 INFO:teuthology.orchestra.run.smithi097.stdout:(94/144): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 813 kB/s | 74 kB 00:00 2023-07-06T03:40:32.168 INFO:teuthology.orchestra.run.smithi097.stdout:(95/144): zlib-devel-1.2.11-21.el8.x86_64.rpm 577 kB/s | 58 kB 00:00 2023-07-06T03:40:32.294 INFO:teuthology.orchestra.run.smithi097.stdout:(96/144): gperftools-libs-2.7-9.el8.x86_64.rpm 1.5 MB/s | 306 kB 00:00 2023-07-06T03:40:32.319 INFO:teuthology.orchestra.run.smithi097.stdout:(97/144): liboath-2.6.2-3.el8.x86_64.rpm 391 kB/s | 59 kB 00:00 2023-07-06T03:40:32.344 INFO:teuthology.orchestra.run.smithi097.stdout:(98/144): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2023-07-06T03:40:32.394 INFO:teuthology.orchestra.run.smithi097.stdout:(99/144): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 874 kB/s | 44 kB 00:00 2023-07-06T03:40:32.453 INFO:teuthology.orchestra.run.smithi097.stdout:(100/144): python3-asyncssh-2.7.0-2.el8.noarch. 3.5 MB/s | 476 kB 00:00 2023-07-06T03:40:32.478 INFO:teuthology.orchestra.run.smithi097.stdout:(101/144): python3-beautifulsoup4-4.6.3-2.el8.1 2.2 MB/s | 185 kB 00:00 2023-07-06T03:40:32.528 INFO:teuthology.orchestra.run.smithi097.stdout:(102/144): python3-certifi-2018.10.15-7.el8.noa 328 kB/s | 16 kB 00:00 2023-07-06T03:40:32.554 INFO:teuthology.orchestra.run.smithi097.stdout:(103/144): python3-cachetools-3.1.1-4.el8.noarc 439 kB/s | 33 kB 00:00 2023-07-06T03:40:32.587 INFO:teuthology.orchestra.run.smithi097.stdout:(104/144): python3-cheroot-8.5.2-1.el8.noarch.r 2.9 MB/s | 173 kB 00:00 2023-07-06T03:40:32.621 INFO:teuthology.orchestra.run.smithi097.stdout:(105/144): python3-cherrypy-18.4.0-1.el8.noarch 5.6 MB/s | 384 kB 00:00 2023-07-06T03:40:32.646 INFO:teuthology.orchestra.run.smithi097.stdout:(106/144): python3-google-auth-1.1.1-10.el8.noa 1.4 MB/s | 81 kB 00:00 2023-07-06T03:40:32.671 INFO:teuthology.orchestra.run.smithi097.stdout:(107/144): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-07-06T03:40:32.689 INFO:teuthology.orchestra.run.smithi097.stdout:(108/144): python3-jaraco-functools-2.0-4.el8.n 421 kB/s | 18 kB 00:00 2023-07-06T03:40:32.738 INFO:teuthology.orchestra.run.smithi097.stdout:(109/144): python3-logutils-0.3.5-11.el8.noarch 979 kB/s | 49 kB 00:00 2023-07-06T03:40:32.772 INFO:teuthology.orchestra.run.smithi097.stdout:(110/144): python3-kubernetes-11.0.0-6.el8.noar 12 MB/s | 1.2 MB 00:00 2023-07-06T03:40:32.789 INFO:teuthology.orchestra.run.smithi097.stdout:(111/144): python3-more-itertools-7.2.0-3.el8.n 1.1 MB/s | 59 kB 00:00 2023-07-06T03:40:32.815 INFO:teuthology.orchestra.run.smithi097.stdout:(112/144): python3-natsort-7.1.1-2.el8.noarch.r 1.4 MB/s | 60 kB 00:00 2023-07-06T03:40:32.856 INFO:teuthology.orchestra.run.smithi097.stdout:(113/144): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-07-06T03:40:32.874 INFO:teuthology.orchestra.run.smithi097.stdout:(114/144): python3-portend-2.6-1.el8.noarch.rpm 265 kB/s | 16 kB 00:00 2023-07-06T03:40:32.899 INFO:teuthology.orchestra.run.smithi097.stdout:(115/144): python3-repoze-lru-0.7-6.el8.noarch. 802 kB/s | 34 kB 00:00 2023-07-06T03:40:32.924 INFO:teuthology.orchestra.run.smithi097.stdout:(116/144): python3-routes-2.4.1-12.el8.noarch.r 3.8 MB/s | 196 kB 00:00 2023-07-06T03:40:32.941 INFO:teuthology.orchestra.run.smithi097.stdout:(117/144): python3-rsa-4.8-1.el8.noarch.rpm 1.4 MB/s | 61 kB 00:00 2023-07-06T03:40:32.966 INFO:teuthology.orchestra.run.smithi097.stdout:(118/144): python3-simplegeneric-0.8.1-17.el8.n 450 kB/s | 19 kB 00:00 2023-07-06T03:40:32.983 INFO:teuthology.orchestra.run.smithi097.stdout:(119/144): python3-singledispatch-3.4.0.3-18.el 573 kB/s | 24 kB 00:00 2023-07-06T03:40:33.009 INFO:teuthology.orchestra.run.smithi097.stdout:(120/144): python3-tempora-1.14.1-5.el8.noarch. 702 kB/s | 29 kB 00:00 2023-07-06T03:40:33.026 INFO:teuthology.orchestra.run.smithi097.stdout:(121/144): python3-trustme-0.6.0-4.el8.noarch.r 656 kB/s | 27 kB 00:00 2023-07-06T03:40:33.068 INFO:teuthology.orchestra.run.smithi097.stdout:(122/144): python3-waitress-1.4.3-1.el8.noarch. 4.2 MB/s | 255 kB 00:00 2023-07-06T03:40:33.093 INFO:teuthology.orchestra.run.smithi097.stdout:(123/144): python3-webob-1.8.5-1.el8.1.noarch.r 3.7 MB/s | 251 kB 00:00 2023-07-06T03:40:33.119 INFO:teuthology.orchestra.run.smithi097.stdout:(124/144): python3-websocket-client-0.56.0-5.el 1.2 MB/s | 61 kB 00:00 2023-07-06T03:40:33.144 INFO:teuthology.orchestra.run.smithi097.stdout:(125/144): python3-webtest-2.0.33-1.el8.noarch. 1.7 MB/s | 85 kB 00:00 2023-07-06T03:40:33.161 INFO:teuthology.orchestra.run.smithi097.stdout:(126/144): python3-zc-lockfile-2.0-2.el8.noarch 547 kB/s | 23 kB 00:00 2023-07-06T03:40:33.303 INFO:teuthology.orchestra.run.smithi097.stdout:(127/144): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2023-07-06T03:40:33.495 INFO:teuthology.orchestra.run.smithi097.stdout:(128/144): librados2-18.0.0-4742.g79365bb3.el8. 11 MB/s | 3.7 MB 00:00 2023-07-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi097.stdout:(129/144): librbd1-18.0.0-4742.g79365bb3.el8.x8 8.4 MB/s | 3.6 MB 00:00 2023-07-06T03:40:33.820 INFO:teuthology.orchestra.run.smithi097.stdout:(130/144): NetworkManager-initscripts-updown-1. 441 kB/s | 143 kB 00:00 2023-07-06T03:40:33.971 INFO:teuthology.orchestra.run.smithi097.stdout:(131/144): elfutils-debuginfod-client-0.189-2.e 314 kB/s | 76 kB 00:00 2023-07-06T03:40:33.996 INFO:teuthology.orchestra.run.smithi097.stdout:(132/144): elfutils-libelf-0.189-2.el8.x86_64.r 1.3 MB/s | 232 kB 00:00 2023-07-06T03:40:34.205 INFO:teuthology.orchestra.run.smithi097.stdout:(133/144): elfutils-libs-0.189-2.el8.x86_64.rpm 1.3 MB/s | 302 kB 00:00 2023-07-06T03:40:34.238 INFO:teuthology.orchestra.run.smithi097.stdout:(134/144): iproute-6.2.0-2.el8.x86_64.rpm 3.6 MB/s | 880 kB 00:00 2023-07-06T03:40:34.297 INFO:teuthology.orchestra.run.smithi097.stdout:(135/144): libblkid-2.32.1-42.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-07-06T03:40:34.347 INFO:teuthology.orchestra.run.smithi097.stdout:(136/144): libfdisk-2.32.1-42.el8.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2023-07-06T03:40:34.423 INFO:teuthology.orchestra.run.smithi097.stdout:(137/144): libmount-2.32.1-42.el8.x86_64.rpm 1.8 MB/s | 236 kB 00:00 2023-07-06T03:40:34.456 INFO:teuthology.orchestra.run.smithi097.stdout:(138/144): libsmartcols-2.32.1-42.el8.x86_64.rp 1.6 MB/s | 179 kB 00:00 2023-07-06T03:40:34.515 INFO:teuthology.orchestra.run.smithi097.stdout:(139/144): libuuid-2.32.1-42.el8.x86_64.rpm 1.0 MB/s | 98 kB 00:00 2023-07-06T03:40:34.607 INFO:teuthology.orchestra.run.smithi097.stdout:(140/144): util-linux-user-2.32.1-42.el8.x86_64 1.1 MB/s | 102 kB 00:00 2023-07-06T03:40:34.707 INFO:teuthology.orchestra.run.smithi097.stdout:(141/144): xfsdump-3.1.8-6.el8.x86_64.rpm 3.3 MB/s | 334 kB 00:00 2023-07-06T03:40:34.908 INFO:teuthology.orchestra.run.smithi097.stdout:(142/144): util-linux-2.32.1-42.el8.x86_64.rpm 5.8 MB/s | 2.6 MB 00:00 2023-07-06T03:40:34.950 INFO:teuthology.orchestra.run.smithi097.stdout:(143/144): xfsprogs-5.0.0-12.el8.x86_64.rpm 4.5 MB/s | 1.1 MB 00:00 2023-07-06T03:40:37.416 INFO:teuthology.orchestra.run.smithi097.stdout:(144/144): ceph-test-18.0.0-4742.g79365bb3.el8. 4.1 MB/s | 48 MB 00:11 2023-07-06T03:40:37.421 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2023-07-06T03:40:37.422 INFO:teuthology.orchestra.run.smithi097.stdout:Total 15 MB/s | 212 MB 00:13 2023-07-06T03:40:39.218 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2023-07-06T03:40:39.630 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2023-07-06T03:40:39.631 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2023-07-06T03:40:41.157 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2023-07-06T03:40:41.161 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2023-07-06T03:40:46.508 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2023-07-06T03:40:46.726 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: thrift-0.13.0-2.el8.x86_64 1/1 2023-07-06T03:40:46.881 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/160 2023-07-06T03:40:47.010 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-06T03:40:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-06T03:40:47.235 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-06T03:40:47.325 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-06T03:40:47.502 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/160 2023-07-06T03:40:47.690 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libelf-0.189-2.el8.x86_64 5/160 2023-07-06T03:40:47.899 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/160 2023-07-06T03:40:48.060 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-06T03:40:48.527 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-06T03:40:48.759 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T03:40:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T03:40:49.376 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T03:40:49.516 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T03:40:49.676 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T03:40:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T03:40:49.969 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 11/160 2023-07-06T03:40:50.092 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-06T03:40:50.328 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-06T03:40:50.515 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-06T03:40:50.608 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-06T03:40:50.800 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/160 2023-07-06T03:40:50.977 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/160 2023-07-06T03:40:51.411 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpq-13.5-1.el8.x86_64 16/160 2023-07-06T03:40:51.614 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/160 2023-07-06T03:40:51.734 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 18/160 2023-07-06T03:40:51.855 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 18/160 2023-07-06T03:40:51.967 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 19/160 2023-07-06T03:40:52.103 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 19/160 2023-07-06T03:40:52.303 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/160 2023-07-06T03:40:52.444 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 21/160 2023-07-06T03:40:52.559 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-06T03:40:52.730 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-06T03:40:52.859 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-06T03:40:53.543 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-06T03:40:54.059 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/160 2023-07-06T03:40:54.749 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/160 2023-07-06T03:40:54.908 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-06T03:40:55.740 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-06T03:40:56.093 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/160 2023-07-06T03:40:56.292 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/160 2023-07-06T03:40:56.449 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/160 2023-07-06T03:40:56.644 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/160 2023-07-06T03:40:56.866 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 30/160 2023-07-06T03:40:57.013 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 31/160 2023-07-06T03:40:57.240 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 32/160 2023-07-06T03:40:58.302 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/160 2023-07-06T03:40:58.654 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/160 2023-07-06T03:40:58.934 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/160 2023-07-06T03:40:59.133 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-06T03:41:01.216 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-06T03:41:01.721 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/160 2023-07-06T03:41:01.916 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/160 2023-07-06T03:41:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/160 2023-07-06T03:41:02.332 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 40/160 2023-07-06T03:41:02.528 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 41/160 2023-07-06T03:41:02.702 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libs-0.189-2.el8.x86_64 42/160 2023-07-06T03:41:02.891 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-debuginfod-client-0.189-2.el8.x86_64 43/160 2023-07-06T03:41:03.132 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-06T03:41:03.229 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-06T03:41:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libmount-2.32.1-42.el8.x86_64 45/160 2023-07-06T03:41:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libmount-2.32.1-42.el8.x86_64 45/160 2023-07-06T03:41:03.663 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.32.1-42.el8.x86_64 46/160 2023-07-06T03:41:03.841 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-06T03:41:04.417 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-06T03:41:04.666 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-06T03:41:04.695 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-06T03:41:04.890 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/160 2023-07-06T03:41:05.182 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/160 2023-07-06T03:41:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/160 2023-07-06T03:41:05.571 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/160 2023-07-06T03:41:05.728 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/160 2023-07-06T03:41:05.921 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/160 2023-07-06T03:41:06.123 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/160 2023-07-06T03:41:06.348 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/160 2023-07-06T03:41:06.536 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/160 2023-07-06T03:41:06.688 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/160 2023-07-06T03:41:06.854 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/160 2023-07-06T03:41:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/160 2023-07-06T03:41:07.191 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/160 2023-07-06T03:41:07.386 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/160 2023-07-06T03:41:07.626 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/160 2023-07-06T03:41:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/160 2023-07-06T03:41:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/160 2023-07-06T03:41:08.410 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/160 2023-07-06T03:41:08.701 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/160 2023-07-06T03:41:08.886 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/160 2023-07-06T03:41:09.698 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/160 2023-07-06T03:41:09.898 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 70/160 2023-07-06T03:41:09.976 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 70/160 2023-07-06T03:41:10.184 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 71/160 2023-07-06T03:41:10.427 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 72/160 2023-07-06T03:41:10.850 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 73/160 2023-07-06T03:41:11.147 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 74/160 2023-07-06T03:41:11.338 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 75/160 2023-07-06T03:41:11.602 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 76/160 2023-07-06T03:41:11.817 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 77/160 2023-07-06T03:41:13.082 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/160 2023-07-06T03:41:13.447 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/160 2023-07-06T03:41:13.675 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 80/160 2023-07-06T03:41:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 81/160 2023-07-06T03:41:14.222 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 82/160 2023-07-06T03:41:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/160 2023-07-06T03:41:14.719 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 84/160 2023-07-06T03:41:14.933 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 85/160 2023-07-06T03:41:15.260 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 86/160 2023-07-06T03:41:15.495 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 87/160 2023-07-06T03:41:15.658 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 88/160 2023-07-06T03:41:15.842 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/160 2023-07-06T03:41:16.180 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/160 2023-07-06T03:41:16.395 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 91/160 2023-07-06T03:41:16.546 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/160 2023-07-06T03:41:16.745 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/160 2023-07-06T03:41:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/160 2023-07-06T03:41:17.192 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.7-17.el8.x86_64 95/160 2023-07-06T03:41:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/160 2023-07-06T03:41:17.571 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/160 2023-07-06T03:41:17.694 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/160 2023-07-06T03:41:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 98/160 2023-07-06T03:41:18.022 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 99/160 2023-07-06T03:41:20.904 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T03:41:21.156 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T03:41:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T03:41:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 101/160 2023-07-06T03:41:22.331 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 101/160 2023-07-06T03:41:22.530 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 102/160 2023-07-06T03:41:42.895 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 102/160 2023-07-06T03:41:42.895 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2023-07-06T03:41:42.895 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2023-07-06T03:41:42.895 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /dev 2023-07-06T03:41:42.896 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /run 2023-07-06T03:41:42.896 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2023-07-06T03:41:42.896 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2023-07-06T03:41:42.897 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2023-07-06T03:41:42.897 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2023-07-06T03:41:42.897 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /dev 2023-07-06T03:41:42.897 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:41:43.294 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 103/160 2023-07-06T03:41:45.019 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 103/160 2023-07-06T03:41:45.249 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 104/160 2023-07-06T03:41:45.514 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 104/160 2023-07-06T03:41:45.719 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 105/160 2023-07-06T03:41:46.251 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 105/160 2023-07-06T03:41:46.469 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 106/160 2023-07-06T03:41:46.518 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 106/160 2023-07-06T03:41:46.719 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 107/160 2023-07-06T03:41:46.924 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 108/160 2023-07-06T03:41:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-06T03:41:47.164 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-06T03:41:47.479 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-7.el8.x86_64 110/160 2023-07-06T03:41:47.735 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.1-7.el8.noarch 111/160 2023-07-06T03:41:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 112/160 2023-07-06T03:41:48.591 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 112/160 2023-07-06T03:41:48.718 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 113/160 2023-07-06T03:41:48.924 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 114/160 2023-07-06T03:41:49.143 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 115/160 2023-07-06T03:41:49.402 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 115/160 2023-07-06T03:41:49.638 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 116/160 2023-07-06T03:41:50.748 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 116/160 2023-07-06T03:41:50.935 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 117/160 2023-07-06T03:41:51.063 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 117/160 2023-07-06T03:41:51.467 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 118/160 2023-07-06T03:41:51.671 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 119/160 2023-07-06T03:41:51.735 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 119/160 2023-07-06T03:41:51.901 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 120/160 2023-07-06T03:41:51.927 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 120/160 2023-07-06T03:41:52.130 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 121/160 2023-07-06T03:41:52.267 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-25.el8.x86_64 122/160 2023-07-06T03:41:57.235 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 122/160 2023-07-06T03:41:58.771 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 123/160 2023-07-06T03:41:58.988 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 124/160 2023-07-06T03:41:59.059 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 124/160 2023-07-06T03:41:59.234 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4742.g79365 125/160 2023-07-06T03:41:59.305 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4742.g79365 125/160 2023-07-06T03:41:59.509 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 126/160 2023-07-06T03:41:59.917 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 126/160 2023-07-06T03:42:00.134 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 127/160 2023-07-06T03:42:00.411 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 127/160 2023-07-06T03:42:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 128/160 2023-07-06T03:42:01.968 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 129/160 2023-07-06T03:42:02.297 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 130/160 2023-07-06T03:42:02.536 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.189-2.el8.x86_64 131/160 2023-07-06T03:42:02.722 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-user-2.32.1-42.el8.x86_64 132/160 2023-07-06T03:42:02.922 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/160 2023-07-06T03:42:03.118 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 134/160 2023-07-06T03:42:04.264 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T03:42:04.500 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T03:42:04.582 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T03:42:04.796 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 136/160 2023-07-06T03:42:05.059 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 137/160 2023-07-06T03:42:05.284 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 138/160 2023-07-06T03:42:05.623 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 139/160 2023-07-06T03:42:05.877 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 140/160 2023-07-06T03:42:06.074 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-06T03:42:06.098 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-06T03:42:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 142/160 2023-07-06T03:42:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/160 2023-07-06T03:42:06.628 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/160 2023-07-06T03:42:06.841 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/160 2023-07-06T03:42:06.866 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/160 2023-07-06T03:42:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 145/160 2023-07-06T03:42:07.229 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 146/160 2023-07-06T03:42:07.387 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 147/160 2023-07-06T03:42:07.387 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-06T03:42:07.637 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-06T03:42:07.637 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-06T03:42:07.852 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-06T03:42:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-06T03:42:07.972 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-06T03:42:07.972 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-06T03:42:08.256 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-06T03:42:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 152/160 2023-07-06T03:42:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 153/160 2023-07-06T03:42:08.784 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-06T03:42:08.785 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-06T03:42:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-06T03:42:08.994 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 156/160 2023-07-06T03:42:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 156/160 2023-07-06T03:42:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-06T03:42:09.369 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-06T03:42:09.369 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-06T03:42:09.568 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-06T03:42:09.569 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-06T03:42:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-06T03:42:09.992 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-06T03:42:10.228 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-06T03:42:10.228 INFO:teuthology.orchestra.run.smithi097.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-06T03:42:10.229 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:42:10.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 1/160 2023-07-06T03:42:10.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2/160 2023-07-06T03:42:10.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 3/160 2023-07-06T03:42:10.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 4/160 2023-07-06T03:42:10.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4742.g79365 5/160 2023-07-06T03:42:10.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 6/160 2023-07-06T03:42:10.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 7/160 2023-07-06T03:42:10.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T03:42:10.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T03:42:10.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T03:42:10.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 11/160 2023-07-06T03:42:10.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 12/160 2023-07-06T03:42:10.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 13/160 2023-07-06T03:42:10.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 14/160 2023-07-06T03:42:10.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 15/160 2023-07-06T03:42:10.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 16/160 2023-07-06T03:42:10.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 17/160 2023-07-06T03:42:10.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 18/160 2023-07-06T03:42:10.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 19/160 2023-07-06T03:42:10.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 20/160 2023-07-06T03:42:10.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 21/160 2023-07-06T03:42:10.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 22/160 2023-07-06T03:42:10.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 23/160 2023-07-06T03:42:10.938 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 24/160 2023-07-06T03:42:10.938 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 25/160 2023-07-06T03:42:10.938 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 26/160 2023-07-06T03:42:10.938 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 27/160 2023-07-06T03:42:10.938 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 28/160 2023-07-06T03:42:10.938 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 29/160 2023-07-06T03:42:10.947 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 30/160 2023-07-06T03:42:10.947 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 31/160 2023-07-06T03:42:10.948 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 32/160 2023-07-06T03:42:10.948 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 33/160 2023-07-06T03:42:10.948 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 34/160 2023-07-06T03:42:10.948 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 35/160 2023-07-06T03:42:10.948 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 36/160 2023-07-06T03:42:10.948 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 37/160 2023-07-06T03:42:10.948 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 38/160 2023-07-06T03:42:10.949 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/160 2023-07-06T03:42:10.949 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/160 2023-07-06T03:42:10.949 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/160 2023-07-06T03:42:10.949 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-7.el8.x86_64 42/160 2023-07-06T03:42:10.949 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/160 2023-07-06T03:42:10.949 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/160 2023-07-06T03:42:10.949 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/160 2023-07-06T03:42:10.950 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/160 2023-07-06T03:42:10.950 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/160 2023-07-06T03:42:10.950 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/160 2023-07-06T03:42:10.950 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/160 2023-07-06T03:42:10.950 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/160 2023-07-06T03:42:10.950 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/160 2023-07-06T03:42:10.950 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/160 2023-07-06T03:42:10.950 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/160 2023-07-06T03:42:10.951 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/160 2023-07-06T03:42:10.951 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/160 2023-07-06T03:42:10.951 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/160 2023-07-06T03:42:10.951 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/160 2023-07-06T03:42:10.951 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/160 2023-07-06T03:42:10.951 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/160 2023-07-06T03:42:10.951 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/160 2023-07-06T03:42:10.951 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/160 2023-07-06T03:42:10.952 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/160 2023-07-06T03:42:10.952 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/160 2023-07-06T03:42:10.952 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/160 2023-07-06T03:42:10.952 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/160 2023-07-06T03:42:10.952 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/160 2023-07-06T03:42:10.952 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/160 2023-07-06T03:42:10.952 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/160 2023-07-06T03:42:10.952 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/160 2023-07-06T03:42:10.953 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/160 2023-07-06T03:42:10.953 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.189-2.el8.x86_64 71/160 2023-07-06T03:42:10.953 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 72/160 2023-07-06T03:42:10.953 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/160 2023-07-06T03:42:10.953 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/160 2023-07-06T03:42:10.953 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/160 2023-07-06T03:42:10.953 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/160 2023-07-06T03:42:10.953 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 77/160 2023-07-06T03:42:10.954 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 78/160 2023-07-06T03:42:10.954 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 79/160 2023-07-06T03:42:10.954 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 80/160 2023-07-06T03:42:10.954 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/160 2023-07-06T03:42:10.954 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.32.1-42.el8.x86_64 82/160 2023-07-06T03:42:10.954 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/160 2023-07-06T03:42:10.954 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 84/160 2023-07-06T03:42:10.954 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 85/160 2023-07-06T03:42:10.955 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 86/160 2023-07-06T03:42:10.955 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 87/160 2023-07-06T03:42:10.955 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 88/160 2023-07-06T03:42:10.955 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 89/160 2023-07-06T03:42:10.955 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 90/160 2023-07-06T03:42:10.955 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 91/160 2023-07-06T03:42:10.955 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 92/160 2023-07-06T03:42:10.956 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/160 2023-07-06T03:42:10.956 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 94/160 2023-07-06T03:42:10.956 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/160 2023-07-06T03:42:10.956 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 96/160 2023-07-06T03:42:10.956 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 97/160 2023-07-06T03:42:10.956 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 98/160 2023-07-06T03:42:10.956 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 99/160 2023-07-06T03:42:10.956 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 100/160 2023-07-06T03:42:10.957 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 101/160 2023-07-06T03:42:10.957 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 102/160 2023-07-06T03:42:10.957 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 103/160 2023-07-06T03:42:10.957 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 104/160 2023-07-06T03:42:10.957 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 105/160 2023-07-06T03:42:10.957 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 106/160 2023-07-06T03:42:10.957 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 107/160 2023-07-06T03:42:10.957 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 108/160 2023-07-06T03:42:10.958 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 109/160 2023-07-06T03:42:10.958 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 110/160 2023-07-06T03:42:10.958 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 111/160 2023-07-06T03:42:10.958 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 112/160 2023-07-06T03:42:10.958 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 113/160 2023-07-06T03:42:10.958 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 114/160 2023-07-06T03:42:10.958 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-2.6-1.el8.noarch 115/160 2023-07-06T03:42:10.958 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 116/160 2023-07-06T03:42:10.959 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 117/160 2023-07-06T03:42:10.959 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 118/160 2023-07-06T03:42:10.959 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 119/160 2023-07-06T03:42:10.959 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 120/160 2023-07-06T03:42:10.959 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 121/160 2023-07-06T03:42:10.959 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 122/160 2023-07-06T03:42:10.959 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 123/160 2023-07-06T03:42:10.959 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 124/160 2023-07-06T03:42:10.960 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 125/160 2023-07-06T03:42:10.960 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 126/160 2023-07-06T03:42:10.960 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 127/160 2023-07-06T03:42:10.960 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 128/160 2023-07-06T03:42:10.960 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 129/160 2023-07-06T03:42:10.960 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/160 2023-07-06T03:42:10.960 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 131/160 2023-07-06T03:42:10.961 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/160 2023-07-06T03:42:10.961 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-8.el 133/160 2023-07-06T03:42:10.961 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 134/160 2023-07-06T03:42:10.961 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.189-2.el8.x86_64 135/160 2023-07-06T03:42:10.961 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 136/160 2023-07-06T03:42:10.961 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.189-2.el8.x86_64 137/160 2023-07-06T03:42:10.961 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 138/160 2023-07-06T03:42:10.962 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.189-2.el8.x86_64 139/160 2023-07-06T03:42:10.962 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 140/160 2023-07-06T03:42:10.962 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 141/160 2023-07-06T03:42:10.962 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 142/160 2023-07-06T03:42:10.962 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.32.1-42.el8.x86_64 143/160 2023-07-06T03:42:10.962 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 144/160 2023-07-06T03:42:10.962 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.32.1-42.el8.x86_64 145/160 2023-07-06T03:42:10.962 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 146/160 2023-07-06T03:42:10.963 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.32.1-42.el8.x86_64 147/160 2023-07-06T03:42:10.963 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 148/160 2023-07-06T03:42:10.963 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.32.1-42.el8.x86_64 149/160 2023-07-06T03:42:10.963 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 150/160 2023-07-06T03:42:10.963 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.32.1-42.el8.x86_64 151/160 2023-07-06T03:42:10.963 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 152/160 2023-07-06T03:42:10.963 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.32.1-42.el8.x86_64 153/160 2023-07-06T03:42:10.963 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-06T03:42:10.964 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.32.1-42.el8.x86_64 155/160 2023-07-06T03:42:10.964 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/160 2023-07-06T03:42:10.964 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 157/160 2023-07-06T03:42:10.964 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 158/160 2023-07-06T03:42:10.964 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 159/160 2023-07-06T03:42:13.003 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 160/160 2023-07-06T03:42:13.004 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:42:13.004 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2023-07-06T03:42:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-initscripts-updown-1:1.40.16-8.el8.noarch 2023-07-06T03:42:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client-0.189-2.el8.x86_64 2023-07-06T03:42:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-0.189-2.el8.x86_64 2023-07-06T03:42:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs-0.189-2.el8.x86_64 2023-07-06T03:42:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: iproute-6.2.0-2.el8.x86_64 2023-07-06T03:42:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid-2.32.1-42.el8.x86_64 2023-07-06T03:42:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk-2.32.1-42.el8.x86_64 2023-07-06T03:42:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: libmount-2.32.1-42.el8.x86_64 2023-07-06T03:42:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols-2.32.1-42.el8.x86_64 2023-07-06T03:42:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-2.32.1-42.el8.x86_64 2023-07-06T03:42:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-2.32.1-42.el8.x86_64 2023-07-06T03:42:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user-2.32.1-42.el8.x86_64 2023-07-06T03:42:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-07-06T03:42:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-07-06T03:42:13.007 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2023-07-06T03:42:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T03:42:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T03:42:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.007 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:42:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.008 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.010 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:42:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:42:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:42:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:42:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:42:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:42:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:42:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:42:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.012 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:42:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.189-2.el8.x86_64 2023-07-06T03:42:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T03:42:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.7-17.el8.x86_64 2023-07-06T03:42:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T03:42:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-7.el8.x86_64 2023-07-06T03:42:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T03:42:13.013 INFO:teuthology.orchestra.run.smithi097.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-07-06T03:42:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T03:42:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-07-06T03:42:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-07-06T03:42:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T03:42:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T03:42:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-07-06T03:42:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-07-06T03:42:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T03:42:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-07-06T03:42:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T03:42:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T03:42:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T03:42:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.32.1-42.el8.x86_64 2023-07-06T03:42:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-07-06T03:42:13.016 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-07-06T03:42:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T03:42:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T03:42:13.017 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.15-4.el8.x86_64 2023-07-06T03:42:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-07-06T03:42:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-07-06T03:42:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-07-06T03:42:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-07-06T03:42:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-07-06T03:42:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T03:42:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T03:42:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T03:42:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T03:42:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T03:42:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T03:42:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T03:42:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T03:42:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T03:42:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-07-06T03:42:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T03:42:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T03:42:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-07-06T03:42:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T03:42:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T03:42:13.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T03:42:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T03:42:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T03:42:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T03:42:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T03:42:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T03:42:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T03:42:13.022 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.0.6-14.el8.noarch 2023-07-06T03:42:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T03:42:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T03:42:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T03:42:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T03:42:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T03:42:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T03:42:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T03:42:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T03:42:13.024 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T03:42:13.024 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T03:42:13.024 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T03:42:13.024 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T03:42:13.024 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T03:42:13.024 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T03:42:13.024 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.025 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.025 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T03:42:13.025 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T03:42:13.025 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.025 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T03:42:13.025 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T03:42:13.025 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-07-06T03:42:13.025 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T03:42:13.026 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T03:42:13.026 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T03:42:13.026 INFO:teuthology.orchestra.run.smithi097.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T03:42:13.026 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T03:42:13.026 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T03:42:13.026 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T03:42:13.026 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T03:42:13.027 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T03:42:13.027 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T03:42:13.027 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.027 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.027 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:42:13.027 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T03:42:13.027 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T03:42:13.027 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T03:42:13.028 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-07-06T03:42:13.028 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T03:42:13.028 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-07-06T03:42:13.028 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:42:13.029 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-07-06T03:42:13.295 DEBUG:teuthology.parallel:result is None 2023-07-06T03:42:16.197 INFO:teuthology.orchestra.run.smithi146.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-06T03:42:16.198 INFO:teuthology.orchestra.run.smithi146.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-06T03:42:16.198 INFO:teuthology.orchestra.run.smithi146.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-06T03:42:16.421 INFO:teuthology.orchestra.run.smithi146.stdout:(97/144): liboath-2.6.2-3.el8.x86_64.rpm 464 B/s | 59 kB 02:09 2023-07-06T03:42:16.446 INFO:teuthology.orchestra.run.smithi146.stdout:(98/144): libunwind-1.3.1-3.el8.x86_64.rpm 592 B/s | 75 kB 02:09 2023-07-06T03:42:16.515 INFO:teuthology.orchestra.run.smithi146.stdout:(99/144): gperftools-libs-2.7-9.el8.x86_64.rpm 2.4 kB/s | 306 kB 02:09 2023-07-06T03:42:16.539 INFO:teuthology.orchestra.run.smithi146.stdout:(100/144): python3-bcrypt-3.1.6-2.el8.1.x86_64. 475 kB/s | 44 kB 00:00 2023-07-06T03:42:16.605 INFO:teuthology.orchestra.run.smithi146.stdout:(101/144): python3-asyncssh-2.7.0-2.el8.noarch. 2.5 MB/s | 476 kB 00:00 2023-07-06T03:42:16.631 INFO:teuthology.orchestra.run.smithi146.stdout:(102/144): python3-cachetools-3.1.1-4.el8.noarc 360 kB/s | 33 kB 00:00 2023-07-06T03:42:16.656 INFO:teuthology.orchestra.run.smithi146.stdout:(103/144): python3-beautifulsoup4-4.6.3-2.el8.1 1.3 MB/s | 185 kB 00:00 2023-07-06T03:42:16.681 INFO:teuthology.orchestra.run.smithi146.stdout:(104/144): python3-certifi-2018.10.15-7.el8.noa 216 kB/s | 16 kB 00:00 2023-07-06T03:42:16.757 INFO:teuthology.orchestra.run.smithi146.stdout:(105/144): python3-google-auth-1.1.1-10.el8.noa 1.1 MB/s | 81 kB 00:00 2023-07-06T03:42:16.782 INFO:teuthology.orchestra.run.smithi146.stdout:(106/144): python3-cheroot-8.5.2-1.el8.noarch.r 1.1 MB/s | 173 kB 00:00 2023-07-06T03:42:16.815 INFO:teuthology.orchestra.run.smithi146.stdout:(107/144): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-07-06T03:42:16.840 INFO:teuthology.orchestra.run.smithi146.stdout:(108/144): python3-jaraco-functools-2.0-4.el8.n 301 kB/s | 18 kB 00:00 2023-07-06T03:42:16.874 INFO:teuthology.orchestra.run.smithi146.stdout:(109/144): python3-cherrypy-18.4.0-1.el8.noarch 1.7 MB/s | 384 kB 00:00 2023-07-06T03:42:16.900 INFO:teuthology.orchestra.run.smithi146.stdout:(110/144): python3-logutils-0.3.5-11.el8.noarch 831 kB/s | 49 kB 00:00 2023-07-06T03:42:16.941 INFO:teuthology.orchestra.run.smithi146.stdout:(111/144): python3-more-itertools-7.2.0-3.el8.n 880 kB/s | 59 kB 00:00 2023-07-06T03:42:16.967 INFO:teuthology.orchestra.run.smithi146.stdout:(112/144): python3-natsort-7.1.1-2.el8.noarch.r 907 kB/s | 60 kB 00:00 2023-07-06T03:42:17.012 INFO:teuthology.orchestra.run.smithi146.stdout:(113/144): python3-kubernetes-11.0.0-6.el8.noar 6.1 MB/s | 1.2 MB 00:00 2023-07-06T03:42:17.034 INFO:teuthology.orchestra.run.smithi146.stdout:(114/144): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2023-07-06T03:42:17.067 INFO:teuthology.orchestra.run.smithi146.stdout:(115/144): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-07-06T03:42:17.093 INFO:teuthology.orchestra.run.smithi146.stdout:(116/144): python3-repoze-lru-0.7-6.el8.noarch. 403 kB/s | 34 kB 00:00 2023-07-06T03:42:17.118 INFO:teuthology.orchestra.run.smithi146.stdout:(117/144): python3-routes-2.4.1-12.el8.noarch.r 2.3 MB/s | 196 kB 00:00 2023-07-06T03:42:17.143 INFO:teuthology.orchestra.run.smithi146.stdout:(118/144): python3-rsa-4.8-1.el8.noarch.rpm 816 kB/s | 61 kB 00:00 2023-07-06T03:42:17.168 INFO:teuthology.orchestra.run.smithi146.stdout:(119/144): python3-simplegeneric-0.8.1-17.el8.n 248 kB/s | 19 kB 00:00 2023-07-06T03:42:17.193 INFO:teuthology.orchestra.run.smithi146.stdout:(120/144): python3-singledispatch-3.4.0.3-18.el 318 kB/s | 24 kB 00:00 2023-07-06T03:42:17.219 INFO:teuthology.orchestra.run.smithi146.stdout:(121/144): python3-tempora-1.14.1-5.el8.noarch. 388 kB/s | 29 kB 00:00 2023-07-06T03:42:17.244 INFO:teuthology.orchestra.run.smithi146.stdout:(122/144): python3-trustme-0.6.0-4.el8.noarch.r 365 kB/s | 27 kB 00:00 2023-07-06T03:42:17.294 INFO:teuthology.orchestra.run.smithi146.stdout:(123/144): python3-waitress-1.4.3-1.el8.noarch. 2.5 MB/s | 255 kB 00:00 2023-07-06T03:42:17.328 INFO:teuthology.orchestra.run.smithi146.stdout:(124/144): python3-websocket-client-0.56.0-5.el 733 kB/s | 61 kB 00:00 2023-07-06T03:42:17.353 INFO:teuthology.orchestra.run.smithi146.stdout:(125/144): python3-webtest-2.0.33-1.el8.noarch. 1.4 MB/s | 85 kB 00:00 2023-07-06T03:42:17.379 INFO:teuthology.orchestra.run.smithi146.stdout:(126/144): python3-webob-1.8.5-1.el8.1.noarch.r 1.5 MB/s | 251 kB 00:00 2023-07-06T03:42:17.404 INFO:teuthology.orchestra.run.smithi146.stdout:(127/144): python3-zc-lockfile-2.0-2.el8.noarch 305 kB/s | 23 kB 00:00 2023-07-06T03:42:17.512 INFO:teuthology.orchestra.run.smithi146.stdout:(128/144): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2023-07-06T03:42:17.888 INFO:teuthology.orchestra.run.smithi146.stdout:(129/144): NetworkManager-initscripts-updown-1. 383 kB/s | 143 kB 00:00 2023-07-06T03:42:17.996 INFO:teuthology.orchestra.run.smithi146.stdout:(130/144): elfutils-debuginfod-client-0.189-2.e 703 kB/s | 76 kB 00:00 2023-07-06T03:42:18.147 INFO:teuthology.orchestra.run.smithi146.stdout:(131/144): librbd1-18.0.0-4742.g79365bb3.el8.x8 4.8 MB/s | 3.6 MB 00:00 2023-07-06T03:42:18.172 INFO:teuthology.orchestra.run.smithi146.stdout:(132/144): elfutils-libelf-0.189-2.el8.x86_64.r 1.3 MB/s | 232 kB 00:00 2023-07-06T03:42:18.347 INFO:teuthology.orchestra.run.smithi146.stdout:(133/144): iproute-6.2.0-2.el8.x86_64.rpm 4.9 MB/s | 880 kB 00:00 2023-07-06T03:42:18.447 INFO:teuthology.orchestra.run.smithi146.stdout:(134/144): libblkid-2.32.1-42.el8.x86_64.rpm 2.2 MB/s | 220 kB 00:00 2023-07-06T03:42:18.548 INFO:teuthology.orchestra.run.smithi146.stdout:(135/144): libfdisk-2.32.1-42.el8.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2023-07-06T03:42:18.615 INFO:teuthology.orchestra.run.smithi146.stdout:(136/144): elfutils-libs-0.189-2.el8.x86_64.rpm 647 kB/s | 302 kB 00:00 2023-07-06T03:42:18.648 INFO:teuthology.orchestra.run.smithi146.stdout:(137/144): libmount-2.32.1-42.el8.x86_64.rpm 2.3 MB/s | 236 kB 00:00 2023-07-06T03:42:18.724 INFO:teuthology.orchestra.run.smithi146.stdout:(138/144): libsmartcols-2.32.1-42.el8.x86_64.rp 1.6 MB/s | 179 kB 00:00 2023-07-06T03:42:18.749 INFO:teuthology.orchestra.run.smithi146.stdout:(139/144): libuuid-2.32.1-42.el8.x86_64.rpm 983 kB/s | 98 kB 00:00 2023-07-06T03:42:18.874 INFO:teuthology.orchestra.run.smithi146.stdout:(140/144): util-linux-user-2.32.1-42.el8.x86_64 819 kB/s | 102 kB 00:00 2023-07-06T03:42:18.983 INFO:teuthology.orchestra.run.smithi146.stdout:(141/144): xfsdump-3.1.8-6.el8.x86_64.rpm 3.0 MB/s | 334 kB 00:00 2023-07-06T03:42:19.033 INFO:teuthology.orchestra.run.smithi146.stdout:(142/144): librados2-18.0.0-4742.g79365bb3.el8. 2.2 MB/s | 3.7 MB 00:01 2023-07-06T03:42:19.084 INFO:teuthology.orchestra.run.smithi146.stdout:(143/144): util-linux-2.32.1-42.el8.x86_64.rpm 7.2 MB/s | 2.6 MB 00:00 2023-07-06T03:42:19.125 INFO:teuthology.orchestra.run.smithi146.stdout:(144/144): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2023-07-06T03:42:19.128 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2023-07-06T03:42:19.129 INFO:teuthology.orchestra.run.smithi146.stdout:Total 1.5 MB/s | 212 MB 02:22 2023-07-06T03:42:20.937 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2023-07-06T03:42:21.356 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2023-07-06T03:42:21.357 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2023-07-06T03:42:22.967 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2023-07-06T03:42:22.971 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2023-07-06T03:42:28.248 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2023-07-06T03:42:28.486 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: thrift-0.13.0-2.el8.x86_64 1/1 2023-07-06T03:42:28.649 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/160 2023-07-06T03:42:28.773 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-06T03:42:28.878 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libuuid-2.32.1-42.el8.x86_64 2/160 2023-07-06T03:42:29.015 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-06T03:42:29.100 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libblkid-2.32.1-42.el8.x86_64 3/160 2023-07-06T03:42:29.288 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/160 2023-07-06T03:42:29.512 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : elfutils-libelf-0.189-2.el8.x86_64 5/160 2023-07-06T03:42:29.710 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/160 2023-07-06T03:42:29.873 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-06T03:42:30.351 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/160 2023-07-06T03:42:30.517 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T03:42:30.960 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T03:42:31.132 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T03:42:31.274 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T03:42:31.464 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T03:42:31.580 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T03:42:31.782 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 11/160 2023-07-06T03:42:31.931 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-06T03:42:32.166 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/160 2023-07-06T03:42:32.389 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-06T03:42:32.473 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/160 2023-07-06T03:42:32.680 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/160 2023-07-06T03:42:32.857 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/160 2023-07-06T03:42:33.279 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpq-13.5-1.el8.x86_64 16/160 2023-07-06T03:42:33.469 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/160 2023-07-06T03:42:33.623 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 18/160 2023-07-06T03:42:33.745 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 18/160 2023-07-06T03:42:33.914 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 19/160 2023-07-06T03:42:34.059 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 19/160 2023-07-06T03:42:34.277 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/160 2023-07-06T03:42:34.430 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 21/160 2023-07-06T03:42:34.572 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-06T03:42:34.733 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 22/160 2023-07-06T03:42:34.880 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-06T03:42:35.555 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 23/160 2023-07-06T03:42:35.751 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/160 2023-07-06T03:42:36.450 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/160 2023-07-06T03:42:36.622 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-06T03:42:37.443 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/160 2023-07-06T03:42:37.790 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/160 2023-07-06T03:42:37.989 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/160 2023-07-06T03:42:38.149 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/160 2023-07-06T03:42:38.341 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/160 2023-07-06T03:42:38.562 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 30/160 2023-07-06T03:42:38.702 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 31/160 2023-07-06T03:42:38.922 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 32/160 2023-07-06T03:42:39.995 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/160 2023-07-06T03:42:40.368 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/160 2023-07-06T03:42:40.681 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/160 2023-07-06T03:42:40.880 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-06T03:42:42.951 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/160 2023-07-06T03:42:43.293 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/160 2023-07-06T03:42:43.488 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/160 2023-07-06T03:42:43.684 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/160 2023-07-06T03:42:43.862 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 40/160 2023-07-06T03:42:44.044 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 41/160 2023-07-06T03:42:44.226 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : elfutils-libs-0.189-2.el8.x86_64 42/160 2023-07-06T03:42:44.408 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : elfutils-debuginfod-client-0.189-2.el8.x86_64 43/160 2023-07-06T03:42:44.621 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-06T03:42:44.701 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libfdisk-2.32.1-42.el8.x86_64 44/160 2023-07-06T03:42:44.862 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libmount-2.32.1-42.el8.x86_64 45/160 2023-07-06T03:42:44.935 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libmount-2.32.1-42.el8.x86_64 45/160 2023-07-06T03:42:45.100 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libuuid-devel-2.32.1-42.el8.x86_64 46/160 2023-07-06T03:42:45.288 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-06T03:42:45.878 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libsmartcols-2.32.1-42.el8.x86_64 47/160 2023-07-06T03:42:46.139 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-06T03:42:46.166 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: util-linux-2.32.1-42.el8.x86_64 48/160 2023-07-06T03:42:46.354 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/160 2023-07-06T03:42:46.629 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/160 2023-07-06T03:42:46.860 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/160 2023-07-06T03:42:47.009 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/160 2023-07-06T03:42:47.158 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/160 2023-07-06T03:42:47.367 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/160 2023-07-06T03:42:47.569 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/160 2023-07-06T03:42:47.792 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/160 2023-07-06T03:42:47.999 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/160 2023-07-06T03:42:48.152 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/160 2023-07-06T03:42:48.327 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/160 2023-07-06T03:42:48.520 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/160 2023-07-06T03:42:48.681 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/160 2023-07-06T03:42:48.850 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/160 2023-07-06T03:42:49.065 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/160 2023-07-06T03:42:49.301 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/160 2023-07-06T03:42:49.504 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/160 2023-07-06T03:42:49.858 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/160 2023-07-06T03:42:50.122 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/160 2023-07-06T03:42:50.317 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/160 2023-07-06T03:42:51.170 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/160 2023-07-06T03:42:51.370 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 70/160 2023-07-06T03:42:51.463 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 70/160 2023-07-06T03:42:51.680 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 71/160 2023-07-06T03:42:51.923 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 72/160 2023-07-06T03:42:52.312 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 73/160 2023-07-06T03:42:52.561 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 74/160 2023-07-06T03:42:52.801 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 75/160 2023-07-06T03:42:53.056 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 76/160 2023-07-06T03:42:53.280 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 77/160 2023-07-06T03:42:54.577 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/160 2023-07-06T03:42:54.962 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/160 2023-07-06T03:42:55.163 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 80/160 2023-07-06T03:42:55.410 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 81/160 2023-07-06T03:42:55.711 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 82/160 2023-07-06T03:42:55.947 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/160 2023-07-06T03:42:56.165 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 84/160 2023-07-06T03:42:56.358 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 85/160 2023-07-06T03:42:56.709 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 86/160 2023-07-06T03:42:56.908 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 87/160 2023-07-06T03:42:57.047 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 88/160 2023-07-06T03:42:57.229 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/160 2023-07-06T03:42:57.574 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/160 2023-07-06T03:42:57.865 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 91/160 2023-07-06T03:42:58.018 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/160 2023-07-06T03:42:58.211 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/160 2023-07-06T03:42:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/160 2023-07-06T03:42:58.638 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.7-17.el8.x86_64 95/160 2023-07-06T03:42:58.816 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/160 2023-07-06T03:42:58.982 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/160 2023-07-06T03:42:59.105 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/160 2023-07-06T03:42:59.286 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 98/160 2023-07-06T03:42:59.436 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 99/160 2023-07-06T03:43:02.284 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T03:43:02.502 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T03:43:03.340 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 100/160 2023-07-06T03:43:03.624 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 101/160 2023-07-06T03:43:03.801 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 101/160 2023-07-06T03:43:04.018 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 102/160 2023-07-06T03:43:24.415 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 102/160 2023-07-06T03:43:24.415 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2023-07-06T03:43:24.416 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2023-07-06T03:43:24.416 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /dev 2023-07-06T03:43:24.416 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /run 2023-07-06T03:43:24.416 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2023-07-06T03:43:24.416 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2023-07-06T03:43:24.417 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2023-07-06T03:43:24.417 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2023-07-06T03:43:24.417 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /dev 2023-07-06T03:43:24.417 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:43:24.782 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 103/160 2023-07-06T03:43:26.519 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 103/160 2023-07-06T03:43:26.728 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 104/160 2023-07-06T03:43:26.995 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 104/160 2023-07-06T03:43:27.175 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 105/160 2023-07-06T03:43:27.706 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 105/160 2023-07-06T03:43:27.964 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 106/160 2023-07-06T03:43:28.013 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 106/160 2023-07-06T03:43:28.198 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 107/160 2023-07-06T03:43:28.395 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 108/160 2023-07-06T03:43:28.547 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-06T03:43:28.643 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 109/160 2023-07-06T03:43:28.947 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : jq-1.6-7.el8.x86_64 110/160 2023-07-06T03:43:29.173 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : automake-1.16.1-7.el8.noarch 111/160 2023-07-06T03:43:29.781 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 112/160 2023-07-06T03:43:29.939 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 112/160 2023-07-06T03:43:30.080 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 113/160 2023-07-06T03:43:30.293 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 114/160 2023-07-06T03:43:30.514 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 115/160 2023-07-06T03:43:30.771 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 115/160 2023-07-06T03:43:30.977 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 116/160 2023-07-06T03:43:32.090 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 116/160 2023-07-06T03:43:32.289 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 117/160 2023-07-06T03:43:32.415 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 117/160 2023-07-06T03:43:32.811 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 118/160 2023-07-06T03:43:33.011 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 119/160 2023-07-06T03:43:33.075 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 119/160 2023-07-06T03:43:33.230 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 120/160 2023-07-06T03:43:33.257 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 120/160 2023-07-06T03:43:33.461 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 121/160 2023-07-06T03:43:33.605 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libtool-2.4.6-25.el8.x86_64 122/160 2023-07-06T03:43:38.561 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 122/160 2023-07-06T03:43:40.101 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 123/160 2023-07-06T03:43:40.351 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 124/160 2023-07-06T03:43:40.424 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 124/160 2023-07-06T03:43:40.571 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4742.g79365 125/160 2023-07-06T03:43:40.652 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4742.g79365 125/160 2023-07-06T03:43:40.862 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 126/160 2023-07-06T03:43:41.271 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 126/160 2023-07-06T03:43:41.513 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 127/160 2023-07-06T03:43:41.793 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 127/160 2023-07-06T03:43:42.101 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 128/160 2023-07-06T03:43:43.315 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 129/160 2023-07-06T03:43:43.901 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 130/160 2023-07-06T03:43:44.096 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : elfutils-libelf-devel-0.189-2.el8.x86_64 131/160 2023-07-06T03:43:44.293 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : util-linux-user-2.32.1-42.el8.x86_64 132/160 2023-07-06T03:43:44.484 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/160 2023-07-06T03:43:44.671 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 134/160 2023-07-06T03:43:45.843 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T03:43:46.120 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T03:43:46.210 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 135/160 2023-07-06T03:43:46.441 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 136/160 2023-07-06T03:43:46.704 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 137/160 2023-07-06T03:43:46.929 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 138/160 2023-07-06T03:43:47.295 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 139/160 2023-07-06T03:43:47.538 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 140/160 2023-07-06T03:43:47.686 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-06T03:43:47.709 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-8.el 141/160 2023-07-06T03:43:47.900 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 142/160 2023-07-06T03:43:48.136 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/160 2023-07-06T03:43:48.242 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/160 2023-07-06T03:43:48.478 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/160 2023-07-06T03:43:48.504 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/160 2023-07-06T03:43:48.679 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 145/160 2023-07-06T03:43:48.876 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 146/160 2023-07-06T03:43:49.048 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 147/160 2023-07-06T03:43:49.049 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-06T03:43:49.307 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 148/160 2023-07-06T03:43:49.308 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-06T03:43:49.547 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 149/160 2023-07-06T03:43:49.562 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-06T03:43:49.675 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 150/160 2023-07-06T03:43:49.675 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-06T03:43:49.926 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/160 2023-07-06T03:43:50.104 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 152/160 2023-07-06T03:43:50.269 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 153/160 2023-07-06T03:43:50.487 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-06T03:43:50.487 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-06T03:43:50.697 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 155/160 2023-07-06T03:43:50.697 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 156/160 2023-07-06T03:43:50.914 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 156/160 2023-07-06T03:43:50.914 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-06T03:43:51.130 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 157/160 2023-07-06T03:43:51.131 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-06T03:43:51.347 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 158/160 2023-07-06T03:43:51.348 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-06T03:43:51.564 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 159/160 2023-07-06T03:43:51.737 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-06T03:43:51.934 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 160/160 2023-07-06T03:43:51.934 INFO:teuthology.orchestra.run.smithi146.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-06T03:43:51.935 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:43:52.662 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 1/160 2023-07-06T03:43:52.663 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2/160 2023-07-06T03:43:52.663 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 3/160 2023-07-06T03:43:52.663 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 4/160 2023-07-06T03:43:52.663 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4742.g79365 5/160 2023-07-06T03:43:52.664 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 6/160 2023-07-06T03:43:52.664 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 7/160 2023-07-06T03:43:52.664 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 8/160 2023-07-06T03:43:52.664 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 9/160 2023-07-06T03:43:52.665 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 10/160 2023-07-06T03:43:52.665 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 11/160 2023-07-06T03:43:52.665 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 12/160 2023-07-06T03:43:52.665 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 13/160 2023-07-06T03:43:52.665 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 14/160 2023-07-06T03:43:52.665 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 15/160 2023-07-06T03:43:52.665 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 16/160 2023-07-06T03:43:52.665 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 17/160 2023-07-06T03:43:52.666 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 18/160 2023-07-06T03:43:52.666 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 19/160 2023-07-06T03:43:52.666 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 20/160 2023-07-06T03:43:52.666 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 21/160 2023-07-06T03:43:52.666 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 22/160 2023-07-06T03:43:52.666 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 23/160 2023-07-06T03:43:52.666 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 24/160 2023-07-06T03:43:52.666 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 25/160 2023-07-06T03:43:52.667 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 26/160 2023-07-06T03:43:52.667 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 27/160 2023-07-06T03:43:52.667 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 28/160 2023-07-06T03:43:52.667 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 29/160 2023-07-06T03:43:52.675 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 30/160 2023-07-06T03:43:52.676 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 31/160 2023-07-06T03:43:52.676 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 32/160 2023-07-06T03:43:52.676 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 33/160 2023-07-06T03:43:52.676 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 34/160 2023-07-06T03:43:52.676 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 35/160 2023-07-06T03:43:52.676 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 36/160 2023-07-06T03:43:52.677 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 37/160 2023-07-06T03:43:52.677 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 38/160 2023-07-06T03:43:52.677 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/160 2023-07-06T03:43:52.677 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/160 2023-07-06T03:43:52.677 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/160 2023-07-06T03:43:52.677 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : jq-1.6-7.el8.x86_64 42/160 2023-07-06T03:43:52.677 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/160 2023-07-06T03:43:52.677 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/160 2023-07-06T03:43:52.678 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/160 2023-07-06T03:43:52.678 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/160 2023-07-06T03:43:52.678 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/160 2023-07-06T03:43:52.678 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/160 2023-07-06T03:43:52.678 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/160 2023-07-06T03:43:52.678 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/160 2023-07-06T03:43:52.678 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/160 2023-07-06T03:43:52.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/160 2023-07-06T03:43:52.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/160 2023-07-06T03:43:52.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/160 2023-07-06T03:43:52.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/160 2023-07-06T03:43:52.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/160 2023-07-06T03:43:52.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/160 2023-07-06T03:43:52.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/160 2023-07-06T03:43:52.679 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/160 2023-07-06T03:43:52.680 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/160 2023-07-06T03:43:52.680 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/160 2023-07-06T03:43:52.680 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/160 2023-07-06T03:43:52.680 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/160 2023-07-06T03:43:52.680 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/160 2023-07-06T03:43:52.680 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/160 2023-07-06T03:43:52.680 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/160 2023-07-06T03:43:52.680 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/160 2023-07-06T03:43:52.681 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/160 2023-07-06T03:43:52.681 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/160 2023-07-06T03:43:52.681 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/160 2023-07-06T03:43:52.681 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-devel-0.189-2.el8.x86_64 71/160 2023-07-06T03:43:52.681 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 72/160 2023-07-06T03:43:52.681 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/160 2023-07-06T03:43:52.681 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/160 2023-07-06T03:43:52.682 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/160 2023-07-06T03:43:52.682 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/160 2023-07-06T03:43:52.682 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 77/160 2023-07-06T03:43:52.682 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 78/160 2023-07-06T03:43:52.682 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 79/160 2023-07-06T03:43:52.683 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 80/160 2023-07-06T03:43:52.683 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/160 2023-07-06T03:43:52.683 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-devel-2.32.1-42.el8.x86_64 82/160 2023-07-06T03:43:52.683 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/160 2023-07-06T03:43:52.683 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 84/160 2023-07-06T03:43:52.683 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 85/160 2023-07-06T03:43:52.683 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 86/160 2023-07-06T03:43:52.683 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 87/160 2023-07-06T03:43:52.684 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 88/160 2023-07-06T03:43:52.684 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 89/160 2023-07-06T03:43:52.684 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 90/160 2023-07-06T03:43:52.684 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 91/160 2023-07-06T03:43:52.684 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 92/160 2023-07-06T03:43:52.684 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/160 2023-07-06T03:43:52.684 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 94/160 2023-07-06T03:43:52.685 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/160 2023-07-06T03:43:52.685 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 96/160 2023-07-06T03:43:52.685 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 97/160 2023-07-06T03:43:52.685 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 98/160 2023-07-06T03:43:52.685 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 99/160 2023-07-06T03:43:52.685 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 100/160 2023-07-06T03:43:52.685 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 101/160 2023-07-06T03:43:52.685 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 102/160 2023-07-06T03:43:52.686 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 103/160 2023-07-06T03:43:52.686 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 104/160 2023-07-06T03:43:52.686 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 105/160 2023-07-06T03:43:52.686 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 106/160 2023-07-06T03:43:52.686 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 107/160 2023-07-06T03:43:52.686 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 108/160 2023-07-06T03:43:52.686 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 109/160 2023-07-06T03:43:52.686 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 110/160 2023-07-06T03:43:52.687 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 111/160 2023-07-06T03:43:52.687 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 112/160 2023-07-06T03:43:52.687 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 113/160 2023-07-06T03:43:52.687 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 114/160 2023-07-06T03:43:52.687 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-2.6-1.el8.noarch 115/160 2023-07-06T03:43:52.687 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 116/160 2023-07-06T03:43:52.687 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 117/160 2023-07-06T03:43:52.688 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 118/160 2023-07-06T03:43:52.688 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 119/160 2023-07-06T03:43:52.688 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 120/160 2023-07-06T03:43:52.688 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 121/160 2023-07-06T03:43:52.688 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 122/160 2023-07-06T03:43:52.688 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 123/160 2023-07-06T03:43:52.688 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 124/160 2023-07-06T03:43:52.688 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 125/160 2023-07-06T03:43:52.689 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 126/160 2023-07-06T03:43:52.689 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 127/160 2023-07-06T03:43:52.689 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 128/160 2023-07-06T03:43:52.689 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 129/160 2023-07-06T03:43:52.689 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/160 2023-07-06T03:43:52.689 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 131/160 2023-07-06T03:43:52.689 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/160 2023-07-06T03:43:52.690 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-8.el 133/160 2023-07-06T03:43:52.690 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 134/160 2023-07-06T03:43:52.690 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-debuginfod-client-0.189-2.el8.x86_64 135/160 2023-07-06T03:43:52.690 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 136/160 2023-07-06T03:43:52.690 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-0.189-2.el8.x86_64 137/160 2023-07-06T03:43:52.690 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 138/160 2023-07-06T03:43:52.690 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libs-0.189-2.el8.x86_64 139/160 2023-07-06T03:43:52.690 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 140/160 2023-07-06T03:43:52.691 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 141/160 2023-07-06T03:43:52.691 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 142/160 2023-07-06T03:43:52.691 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libblkid-2.32.1-42.el8.x86_64 143/160 2023-07-06T03:43:52.691 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 144/160 2023-07-06T03:43:52.691 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libfdisk-2.32.1-42.el8.x86_64 145/160 2023-07-06T03:43:52.691 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 146/160 2023-07-06T03:43:52.691 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libmount-2.32.1-42.el8.x86_64 147/160 2023-07-06T03:43:52.691 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 148/160 2023-07-06T03:43:52.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libsmartcols-2.32.1-42.el8.x86_64 149/160 2023-07-06T03:43:52.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 150/160 2023-07-06T03:43:52.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-2.32.1-42.el8.x86_64 151/160 2023-07-06T03:43:52.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 152/160 2023-07-06T03:43:52.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-2.32.1-42.el8.x86_64 153/160 2023-07-06T03:43:52.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/160 2023-07-06T03:43:52.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-user-2.32.1-42.el8.x86_64 155/160 2023-07-06T03:43:52.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/160 2023-07-06T03:43:52.693 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 157/160 2023-07-06T03:43:52.693 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 158/160 2023-07-06T03:43:52.693 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 159/160 2023-07-06T03:43:54.784 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 160/160 2023-07-06T03:43:54.785 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:43:54.785 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2023-07-06T03:43:54.785 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-initscripts-updown-1:1.40.16-8.el8.noarch 2023-07-06T03:43:54.785 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-debuginfod-client-0.189-2.el8.x86_64 2023-07-06T03:43:54.786 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-0.189-2.el8.x86_64 2023-07-06T03:43:54.786 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libs-0.189-2.el8.x86_64 2023-07-06T03:43:54.786 INFO:teuthology.orchestra.run.smithi146.stdout: iproute-6.2.0-2.el8.x86_64 2023-07-06T03:43:54.786 INFO:teuthology.orchestra.run.smithi146.stdout: libblkid-2.32.1-42.el8.x86_64 2023-07-06T03:43:54.786 INFO:teuthology.orchestra.run.smithi146.stdout: libfdisk-2.32.1-42.el8.x86_64 2023-07-06T03:43:54.786 INFO:teuthology.orchestra.run.smithi146.stdout: libmount-2.32.1-42.el8.x86_64 2023-07-06T03:43:54.787 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.787 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.787 INFO:teuthology.orchestra.run.smithi146.stdout: libsmartcols-2.32.1-42.el8.x86_64 2023-07-06T03:43:54.787 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-2.32.1-42.el8.x86_64 2023-07-06T03:43:54.787 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-2.32.1-42.el8.x86_64 2023-07-06T03:43:54.787 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-user-2.32.1-42.el8.x86_64 2023-07-06T03:43:54.787 INFO:teuthology.orchestra.run.smithi146.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-07-06T03:43:54.788 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-07-06T03:43:54.788 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2023-07-06T03:43:54.788 INFO:teuthology.orchestra.run.smithi146.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T03:43:54.788 INFO:teuthology.orchestra.run.smithi146.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T03:43:54.788 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.789 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.789 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.789 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.789 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:43:54.790 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.791 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.791 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.791 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:43:54.791 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:43:54.791 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:43:54.792 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:43:54.792 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:43:54.792 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.792 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.793 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:43:54.793 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.793 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.793 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.793 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:43:54.793 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:43:54.794 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.794 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T03:43:54.794 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel-0.189-2.el8.x86_64 2023-07-06T03:43:54.794 INFO:teuthology.orchestra.run.smithi146.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T03:43:54.794 INFO:teuthology.orchestra.run.smithi146.stdout: fuse-2.9.7-17.el8.x86_64 2023-07-06T03:43:54.794 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T03:43:54.795 INFO:teuthology.orchestra.run.smithi146.stdout: jq-1.6-7.el8.x86_64 2023-07-06T03:43:54.795 INFO:teuthology.orchestra.run.smithi146.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T03:43:54.795 INFO:teuthology.orchestra.run.smithi146.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-07-06T03:43:54.795 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T03:43:54.795 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.795 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.795 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.796 INFO:teuthology.orchestra.run.smithi146.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-07-06T03:43:54.796 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-07-06T03:43:54.796 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T03:43:54.796 INFO:teuthology.orchestra.run.smithi146.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T03:43:54.796 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-07-06T03:43:54.796 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-07-06T03:43:54.796 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.797 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.797 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T03:43:54.797 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.797 INFO:teuthology.orchestra.run.smithi146.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-07-06T03:43:54.797 INFO:teuthology.orchestra.run.smithi146.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T03:43:54.797 INFO:teuthology.orchestra.run.smithi146.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T03:43:54.797 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T03:43:54.797 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel-2.32.1-42.el8.x86_64 2023-07-06T03:43:54.798 INFO:teuthology.orchestra.run.smithi146.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-07-06T03:43:54.798 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-07-06T03:43:54.798 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T03:43:54.798 INFO:teuthology.orchestra.run.smithi146.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T03:43:54.798 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.15-4.el8.x86_64 2023-07-06T03:43:54.798 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-07-06T03:43:54.798 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-07-06T03:43:54.799 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-07-06T03:43:54.799 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-07-06T03:43:54.799 INFO:teuthology.orchestra.run.smithi146.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-07-06T03:43:54.799 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T03:43:54.799 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T03:43:54.799 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-07-06T03:43:54.799 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T03:43:54.800 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T03:43:54.800 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T03:43:54.800 INFO:teuthology.orchestra.run.smithi146.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T03:43:54.801 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T03:43:54.801 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.801 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.801 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.801 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T03:43:54.801 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-07-06T03:43:54.802 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T03:43:54.802 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T03:43:54.802 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-07-06T03:43:54.802 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T03:43:54.802 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T03:43:54.802 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T03:43:54.803 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T03:43:54.803 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T03:43:54.803 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T03:43:54.803 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T03:43:54.803 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T03:43:54.803 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T03:43:54.803 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.0.6-14.el8.noarch 2023-07-06T03:43:54.804 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T03:43:54.804 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T03:43:54.804 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T03:43:54.804 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T03:43:54.805 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T03:43:54.805 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T03:43:54.805 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T03:43:54.805 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T03:43:54.805 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T03:43:54.806 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T03:43:54.806 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T03:43:54.806 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T03:43:54.806 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T03:43:54.806 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T03:43:54.806 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.806 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.807 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T03:43:54.807 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T03:43:54.807 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.807 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T03:43:54.807 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T03:43:54.807 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-07-06T03:43:54.807 INFO:teuthology.orchestra.run.smithi146.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T03:43:54.807 INFO:teuthology.orchestra.run.smithi146.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T03:43:54.808 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T03:43:54.808 INFO:teuthology.orchestra.run.smithi146.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T03:43:54.808 INFO:teuthology.orchestra.run.smithi146.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T03:43:54.808 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T03:43:54.808 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T03:43:54.808 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T03:43:54.808 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T03:43:54.809 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T03:43:54.809 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.809 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.810 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T03:43:54.810 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T03:43:54.810 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T03:43:54.811 INFO:teuthology.orchestra.run.smithi146.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T03:43:54.811 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-07-06T03:43:54.811 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T03:43:54.812 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-07-06T03:43:54.812 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:43:54.812 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2023-07-06T03:43:55.098 DEBUG:teuthology.parallel:result is None 2023-07-06T03:43:55.099 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T03:43:55.238 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T03:43:55.298 INFO:teuthology.orchestra.run.smithi097.stdout:18.0.0-4742.g79365bb3.el8 2023-07-06T03:43:55.299 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4742.g79365bb3.el8 2023-07-06T03:43:55.299 INFO:teuthology.task.install:The correct ceph version 18.0.0-4742.g79365bb3 is installed. 2023-07-06T03:43:55.300 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T03:43:55.438 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T03:43:55.496 INFO:teuthology.orchestra.run.smithi146.stdout:18.0.0-4742.g79365bb3.el8 2023-07-06T03:43:55.496 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4742.g79365bb3.el8 2023-07-06T03:43:55.496 INFO:teuthology.task.install:The correct ceph version 18.0.0-4742.g79365bb3 is installed. 2023-07-06T03:43:55.497 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-06T03:43:55.498 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:43:55.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T03:43:55.575 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:43:55.576 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T03:43:55.651 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-06T03:43:55.652 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:43:55.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T03:43:55.722 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T03:43:55.831 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:43:55.832 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T03:43:55.908 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T03:43:55.980 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-06T03:43:55.981 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:43:55.981 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T03:43:56.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T03:43:56.156 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:43:56.156 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T03:43:56.233 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T03:43:56.347 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-06T03:43:56.348 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:43:56.348 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T03:43:56.416 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T03:43:56.524 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:43:56.524 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T03:43:56.596 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T03:43:56.706 INFO:teuthology.run_tasks:Running task ceph... 2023-07-06T03:43:56.790 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-07-06T03:43:56.790 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2023-07-06T03:43:56.795 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /var/log/ceph 2023-07-06T03:43:56.864 INFO:tasks.ceph:Disabling ceph logrotate... 2023-07-06T03:43:56.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-06T03:43:56.869 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-06T03:43:56.938 INFO:tasks.ceph:Creating extra log directories... 2023-07-06T03:43:56.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-06T03:43:56.943 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-06T03:43:57.027 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-07-06T03:43:57.028 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-07-06T03:43:57.028 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-07-06T03:43:57.029 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:43:57.029 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-06T03:43:57.074 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-06T03:44:02.277 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-06T03:44:02.353 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:02.353 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-06T03:44:02.400 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-06T03:44:02.621 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-06T03:44:02.696 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-07-06T03:44:02.696 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-07-06T03:44:02.697 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895', 'branch': 'wip-rishabh-2023Jul5', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits}', 'email': None, 'first_in_suite': False, 'job_id': '7327895', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-2023Jul5'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-2023Jul5', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-2023Jul5', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbLB4vrKtiUGEdyy5vFw5lZIMGJWLgfKu1FVCS9K8igDuk1xKRkTBDq+z0YIrYGDhUWIFXkqNli9BT6Jf9pD7c=', 'smithi146.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDL+5Bv2zzEExDF42A9wR5B2XOKQwNmFgKCpXESaCkKVHsS7/uHuMqg/F279BCxmbt3UfjWgbqyr8R85CKHiAWLMGdbGdEdN1LUxOuDr/gOjjWR3uDEr2B0nmU/wOV3XY8LVRSAaxFdRWeNXRG03U7BMS/1QnhbWodZqDIA39ALk9Oe9tTWjG5uGOqOceku4+Y+0VMJliy9E0qxzLFhbww9vm3UHlKnkjgORQ653YzeJHSsgKOwL09ivX42nvXn3bjMtKMpH0j4Fe9bB/laS9ame+JDRy1M5uGNcYBHtuaiUGx7rnRClnq9WZ6yKdcqnmjyjhkrRb41BpkZKbO4hXCqT83ulMnB3f96TiSHkRoA55yB+TUQeVgf7F804u3gHnAsT97iZYt+YZRe8/E8cSZBL37bXzwC9+vmHlEmd/Z5od4xpsjOIoJwDzq22a9akFO1cwywWpKUKdD4B1BZYSs7dwj7pZDAt74Im3rKGGoqyrG5B+QBodBQbIeYtYIxvhc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_limits']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b53f236c1077f1d295baafb9e0ca9a4b447019d7', 'timestamp': '2023-07-05_22:21:20', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-07-06T03:44:02.697 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-06T03:44:02.704 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-06T03:44:02.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2023-07-06T03:44:02.754 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/run/ceph 2023-07-06T03:44:02.858 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:02.859 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T03:44:02.904 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T03:44:02.905 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2023-07-06T03:44:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T03:44:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T03:44:02.990 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-07-06T03:44:02.990 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T03:44:02.991 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T03:44:02.991 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-07-06 03:42:10.436574914 +0000 2023-07-06T03:44:02.991 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-07-06 03:37:54.317980148 +0000 2023-07-06T03:44:02.991 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-07-06 03:37:54.317980148 +0000 2023-07-06T03:44:02.991 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2023-07-06 03:37:54.292980554 +0000 2023-07-06T03:44:02.991 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T03:44:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-07-06T03:44:03.096 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-07-06T03:44:03.096 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000290328 s, 1.8 MB/s 2023-07-06T03:44:03.097 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T03:44:03.144 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2023-07-06T03:44:03.191 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T03:44:03.191 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T03:44:03.191 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-07-06T03:44:03.191 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T03:44:03.192 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T03:44:03.192 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-07-06 03:42:10.436574914 +0000 2023-07-06T03:44:03.192 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-07-06 03:37:55.148966623 +0000 2023-07-06T03:44:03.192 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-07-06 03:37:55.148966623 +0000 2023-07-06T03:44:03.192 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2023-07-06 03:37:55.122967046 +0000 2023-07-06T03:44:03.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T03:44:03.295 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-07-06T03:44:03.296 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-07-06T03:44:03.296 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000259213 s, 2.0 MB/s 2023-07-06T03:44:03.298 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T03:44:03.344 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2023-07-06T03:44:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T03:44:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T03:44:03.430 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-07-06T03:44:03.430 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T03:44:03.430 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T03:44:03.430 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-07-06 03:42:10.436574914 +0000 2023-07-06T03:44:03.430 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-07-06 03:37:55.977953131 +0000 2023-07-06T03:44:03.431 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-07-06 03:37:55.977953131 +0000 2023-07-06T03:44:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2023-07-06 03:37:55.954953505 +0000 2023-07-06T03:44:03.431 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T03:44:03.534 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-07-06T03:44:03.534 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-07-06T03:44:03.534 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000302385 s, 1.7 MB/s 2023-07-06T03:44:03.536 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T03:44:03.583 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2023-07-06T03:44:03.629 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T03:44:03.630 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T03:44:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-07-06T03:44:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T03:44:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T03:44:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-07-06 03:42:10.436574914 +0000 2023-07-06T03:44:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-07-06 03:37:56.807939623 +0000 2023-07-06T03:44:03.630 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-07-06 03:37:56.807939623 +0000 2023-07-06T03:44:03.631 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2023-07-06 03:37:56.783940013 +0000 2023-07-06T03:44:03.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T03:44:03.734 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-07-06T03:44:03.735 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-07-06T03:44:03.735 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000303333 s, 1.7 MB/s 2023-07-06T03:44:03.736 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T03:44:03.784 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T03:44:03.784 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:03.785 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T03:44:03.835 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T03:44:03.835 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2023-07-06T03:44:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T03:44:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T03:44:03.883 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-07-06T03:44:03.884 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T03:44:03.884 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T03:44:03.884 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-07-06 03:43:52.153189051 +0000 2023-07-06T03:44:03.884 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-07-06 03:38:00.470323239 +0000 2023-07-06T03:44:03.884 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-07-06 03:38:00.470323239 +0000 2023-07-06T03:44:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2023-07-06 03:38:00.441323663 +0000 2023-07-06T03:44:03.884 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T03:44:03.951 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-07-06T03:44:03.951 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-07-06T03:44:03.951 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000310794 s, 1.6 MB/s 2023-07-06T03:44:03.953 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T03:44:04.000 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2023-07-06T03:44:04.045 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T03:44:04.046 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T03:44:04.046 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-07-06T03:44:04.046 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T03:44:04.046 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T03:44:04.047 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-07-06 03:43:52.153189051 +0000 2023-07-06T03:44:04.047 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-07-06 03:38:01.326310697 +0000 2023-07-06T03:44:04.047 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-07-06 03:38:01.326310697 +0000 2023-07-06T03:44:04.047 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2023-07-06 03:38:01.303311034 +0000 2023-07-06T03:44:04.047 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T03:44:04.151 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-07-06T03:44:04.152 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-07-06T03:44:04.152 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000301453 s, 1.7 MB/s 2023-07-06T03:44:04.154 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T03:44:04.202 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2023-07-06T03:44:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T03:44:04.249 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T03:44:04.249 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-07-06T03:44:04.249 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T03:44:04.249 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T03:44:04.250 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-07-06 03:43:52.153189051 +0000 2023-07-06T03:44:04.250 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-07-06 03:38:02.186298102 +0000 2023-07-06T03:44:04.250 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-07-06 03:38:02.186298102 +0000 2023-07-06T03:44:04.250 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2023-07-06 03:38:02.162298454 +0000 2023-07-06T03:44:04.250 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T03:44:04.355 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-07-06T03:44:04.355 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-07-06T03:44:04.356 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000285988 s, 1.8 MB/s 2023-07-06T03:44:04.357 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T03:44:04.405 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2023-07-06T03:44:04.451 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T03:44:04.452 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T03:44:04.452 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-07-06T03:44:04.452 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T03:44:04.452 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T03:44:04.452 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2023-07-06 03:43:52.153189051 +0000 2023-07-06T03:44:04.453 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2023-07-06 03:38:03.028285769 +0000 2023-07-06T03:44:04.453 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2023-07-06 03:38:03.028285769 +0000 2023-07-06T03:44:04.453 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2023-07-06 03:38:03.001286165 +0000 2023-07-06T03:44:04.453 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T03:44:04.559 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2023-07-06T03:44:04.560 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2023-07-06T03:44:04.560 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000296474 s, 1.7 MB/s 2023-07-06T03:44:04.561 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T03:44:04.619 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T03:44:04.619 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-06T03:44:04.619 INFO:tasks.ceph:Generating config... 2023-07-06T03:44:04.620 INFO:tasks.ceph:[client] client mount timeout = 600 2023-07-06T03:44:04.621 INFO:tasks.ceph:[client] debug client = 20 2023-07-06T03:44:04.621 INFO:tasks.ceph:[client] debug ms = 1 2023-07-06T03:44:04.621 INFO:tasks.ceph:[client] pid file = 2023-07-06T03:44:04.621 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-07-06T03:44:04.622 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-07-06T03:44:04.622 INFO:tasks.ceph:[mds] debug mds = 20 2023-07-06T03:44:04.622 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-07-06T03:44:04.622 INFO:tasks.ceph:[mds] debug ms = 1 2023-07-06T03:44:04.622 INFO:tasks.ceph:[mds] mds debug frag = True 2023-07-06T03:44:04.622 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-07-06T03:44:04.623 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-07-06T03:44:04.623 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-07-06T03:44:04.623 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-07-06T03:44:04.623 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-07-06T03:44:04.623 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-07-06T03:44:04.624 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-07-06T03:44:04.624 INFO:tasks.ceph:[mgr] debug ms = 1 2023-07-06T03:44:04.624 INFO:tasks.ceph:[mon] debug mon = 20 2023-07-06T03:44:04.624 INFO:tasks.ceph:[mon] debug ms = 1 2023-07-06T03:44:04.624 INFO:tasks.ceph:[mon] debug paxos = 20 2023-07-06T03:44:04.625 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-07-06T03:44:04.625 INFO:tasks.ceph:[osd] bdev async discard = True 2023-07-06T03:44:04.625 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-07-06T03:44:04.625 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-07-06T03:44:04.625 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-07-06T03:44:04.626 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-07-06T03:44:04.626 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-07-06T03:44:04.626 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-07-06T03:44:04.626 INFO:tasks.ceph:[osd] debug ms = 1 2023-07-06T03:44:04.626 INFO:tasks.ceph:[osd] debug osd = 20 2023-07-06T03:44:04.626 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-07-06T03:44:04.627 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-07-06T03:44:04.627 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-07-06T03:44:04.627 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-07-06T03:44:04.627 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-07-06T03:44:04.627 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-07-06T03:44:04.628 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-07-06T03:44:04.628 INFO:tasks.ceph:Setting up mon.a... 2023-07-06T03:44:04.628 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-07-06T03:44:04.748 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.keyring 2023-07-06T03:44:04.755 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-07-06T03:44:04.860 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T03:44:04.932 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.97'), ('mon.b', '172.21.15.146'), ('mon.c', '[v2:172.21.15.146:3301,v1:172.21.15.146:6790]')] 2023-07-06T03:44:04.934 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-07-06T03:44:04.935 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:04.936 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-07-06T03:44:04.985 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.97 --add b 172.21.15.146 --addv c '[v2:172.21.15.146:3301,v1:172.21.15.146:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:05.117 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:05.117 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: generated fsid eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:44:05.118 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = pacific 2023-07-06T03:44:05.118 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 0 2023-07-06T03:44:05.118 INFO:teuthology.orchestra.run.smithi097.stdout:fsid eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:44:05.118 INFO:teuthology.orchestra.run.smithi097.stdout:last_changed 2023-07-06T03:44:05.116395+0000 2023-07-06T03:44:05.118 INFO:teuthology.orchestra.run.smithi097.stdout:created 2023-07-06T03:44:05.116395+0000 2023-07-06T03:44:05.119 INFO:teuthology.orchestra.run.smithi097.stdout:min_mon_release 16 (pacific) 2023-07-06T03:44:05.119 INFO:teuthology.orchestra.run.smithi097.stdout:election_strategy: 1 2023-07-06T03:44:05.119 INFO:teuthology.orchestra.run.smithi097.stdout:0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2023-07-06T03:44:05.119 INFO:teuthology.orchestra.run.smithi097.stdout:1: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2023-07-06T03:44:05.120 INFO:teuthology.orchestra.run.smithi097.stdout:2: [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] mon.c 2023-07-06T03:44:05.120 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-07-06T03:44:05.122 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-07-06T03:44:05.167 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36... 2023-07-06T03:44:05.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:44:05.213 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:44:05.324 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2023-07-06T03:44:05.324 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2023-07-06T03:44:05.324 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:44:05.324 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2023-07-06T03:44:05.325 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.325 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2023-07-06T03:44:05.325 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.325 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2023-07-06T03:44:05.325 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.325 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2023-07-06T03:44:05.325 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2023-07-06T03:44:05.326 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.326 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2023-07-06T03:44:05.326 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2023-07-06T03:44:05.326 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.326 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:44:05.326 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2023-07-06T03:44:05.326 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.326 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:44:05.327 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:44:05.327 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.327 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2023-07-06T03:44:05.327 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.327 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2023-07-06T03:44:05.327 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2023-07-06T03:44:05.328 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:44:05.328 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:44:05.328 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2023-07-06T03:44:05.328 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:44:05.328 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2023-07-06T03:44:05.328 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:44:05.329 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:44:05.329 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2023-07-06T03:44:05.329 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.329 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:44:05.330 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.330 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2023-07-06T03:44:05.330 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.330 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2023-07-06T03:44:05.330 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.330 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2023-07-06T03:44:05.330 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2023-07-06T03:44:05.331 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2023-07-06T03:44:05.331 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:44:05.331 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:44:05.331 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.332 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2023-07-06T03:44:05.332 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2023-07-06T03:44:05.332 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.332 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2023-07-06T03:44:05.333 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2023-07-06T03:44:05.333 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:44:05.333 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.333 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2023-07-06T03:44:05.333 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2023-07-06T03:44:05.333 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.333 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2023-07-06T03:44:05.334 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2023-07-06T03:44:05.334 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2023-07-06T03:44:05.334 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.334 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2023-07-06T03:44:05.334 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2023-07-06T03:44:05.334 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.334 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:44:05.335 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.335 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2023-07-06T03:44:05.335 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.335 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2023-07-06T03:44:05.335 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.336 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:44:05.336 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2023-07-06T03:44:05.336 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.336 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2023-07-06T03:44:05.336 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2023-07-06T03:44:05.336 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2023-07-06T03:44:05.336 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2023-07-06T03:44:05.337 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2023-07-06T03:44:05.337 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2023-07-06T03:44:05.337 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2023-07-06T03:44:05.337 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2023-07-06T03:44:05.337 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2023-07-06T03:44:05.337 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:44:05.337 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2023-07-06T03:44:05.337 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2023-07-06T03:44:05.338 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:44:05.338 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2023-07-06T03:44:05.338 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:44:05.338 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:44:05.338 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2023-07-06T03:44:05.338 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:44:05.338 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.338 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2023-07-06T03:44:05.339 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:44:05.339 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2023-07-06T03:44:05.339 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:44:05.339 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:44:05.339 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:44:05.339 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:44:05.340 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2023-07-06T03:44:05.340 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.340 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2023-07-06T03:44:05.341 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:44:05.341 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:44:05.341 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2023-07-06T03:44:05.341 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:44:05.342 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2023-07-06T03:44:05.342 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2023-07-06T03:44:05.342 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:44:05.342 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2023-07-06T03:44:05.343 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2023-07-06T03:44:05.343 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:44:05.343 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.344 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:44:05.344 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:44:05.344 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:44:05.344 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.345 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:44:05.345 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:44:05.346 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:44:05.346 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.346 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2023-07-06T03:44:05.347 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:44:05.347 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.348 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:44:05.348 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2023-07-06T03:44:05.348 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:05.348 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2023-07-06T03:44:05.349 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2023-07-06T03:44:05.349 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2023-07-06T03:44:05.349 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2023-07-06T03:44:05.349 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:44:05.349 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:44:05.350 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2023-07-06T03:44:05.350 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2023-07-06T03:44:05.350 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:44:05.350 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2023-07-06T03:44:05.351 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:44:05.351 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:44:05.351 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2023-07-06T03:44:05.351 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2023-07-06T03:44:05.351 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2023-07-06T03:44:05.351 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-07-06T03:44:05.351 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-07-06T03:44:05.352 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-07-06T03:44:05.352 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-07-06T03:44:05.352 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2023-07-06T03:44:05.352 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2023-07-06T03:44:05.352 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2023-07-06T03:44:05.352 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:44:05.352 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2023-07-06T03:44:05.353 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2023-07-06T03:44:05.353 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2023-07-06T03:44:05.353 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2023-07-06T03:44:05.353 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:44:05.353 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:44:05.353 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:44:05.354 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2023-07-06T03:44:05.354 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2023-07-06T03:44:05.354 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:44:05.354 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2023-07-06T03:44:05.355 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.355 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2023-07-06T03:44:05.355 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.355 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2023-07-06T03:44:05.355 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.355 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2023-07-06T03:44:05.355 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2023-07-06T03:44:05.355 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.356 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2023-07-06T03:44:05.356 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2023-07-06T03:44:05.356 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.356 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:44:05.356 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2023-07-06T03:44:05.356 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.357 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:44:05.357 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:44:05.357 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.357 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2023-07-06T03:44:05.357 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.358 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2023-07-06T03:44:05.358 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2023-07-06T03:44:05.358 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:44:05.358 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:44:05.358 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2023-07-06T03:44:05.358 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:44:05.359 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2023-07-06T03:44:05.359 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:44:05.359 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:44:05.359 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2023-07-06T03:44:05.359 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.359 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:44:05.359 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.359 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2023-07-06T03:44:05.360 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.360 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2023-07-06T03:44:05.360 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.360 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2023-07-06T03:44:05.360 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2023-07-06T03:44:05.360 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2023-07-06T03:44:05.360 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:44:05.361 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:44:05.361 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.361 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2023-07-06T03:44:05.361 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2023-07-06T03:44:05.361 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.361 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2023-07-06T03:44:05.361 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2023-07-06T03:44:05.361 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:44:05.362 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.362 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2023-07-06T03:44:05.362 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2023-07-06T03:44:05.362 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.362 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2023-07-06T03:44:05.362 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2023-07-06T03:44:05.362 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2023-07-06T03:44:05.362 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.363 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2023-07-06T03:44:05.363 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2023-07-06T03:44:05.363 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.363 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:44:05.363 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.363 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2023-07-06T03:44:05.363 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.363 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2023-07-06T03:44:05.364 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.364 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:44:05.364 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2023-07-06T03:44:05.364 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.364 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2023-07-06T03:44:05.364 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2023-07-06T03:44:05.364 INFO:teuthology.orchestra.run.smithi146.stdout: bdev async discard = True 2023-07-06T03:44:05.365 INFO:teuthology.orchestra.run.smithi146.stdout: bdev enable discard = True 2023-07-06T03:44:05.365 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore allocator = bitmap 2023-07-06T03:44:05.365 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2023-07-06T03:44:05.365 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2023-07-06T03:44:05.365 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 1/20 2023-07-06T03:44:05.365 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 1/20 2023-07-06T03:44:05.365 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:44:05.365 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2023-07-06T03:44:05.366 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 4/10 2023-07-06T03:44:05.366 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:44:05.366 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2023-07-06T03:44:05.366 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:44:05.366 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:44:05.366 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2023-07-06T03:44:05.366 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:44:05.367 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.367 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2023-07-06T03:44:05.367 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:44:05.368 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2023-07-06T03:44:05.368 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:44:05.368 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:44:05.368 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:44:05.369 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:44:05.369 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2023-07-06T03:44:05.369 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.369 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2023-07-06T03:44:05.370 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:44:05.370 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:44:05.370 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2023-07-06T03:44:05.371 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:44:05.371 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2023-07-06T03:44:05.371 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2023-07-06T03:44:05.371 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:44:05.372 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2023-07-06T03:44:05.372 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2023-07-06T03:44:05.372 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:44:05.372 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.373 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:44:05.373 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:44:05.373 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:44:05.373 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.374 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:44:05.374 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:44:05.374 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:44:05.375 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.375 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2023-07-06T03:44:05.375 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:44:05.375 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.376 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:44:05.376 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2023-07-06T03:44:05.376 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:44:05.377 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2023-07-06T03:44:05.377 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2023-07-06T03:44:05.377 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2023-07-06T03:44:05.377 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2023-07-06T03:44:05.378 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:44:05.378 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:44:05.378 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2023-07-06T03:44:05.378 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2023-07-06T03:44:05.379 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:44:05.379 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2023-07-06T03:44:05.379 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:44:05.380 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:44:05.380 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2023-07-06T03:44:05.380 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2023-07-06T03:44:05.380 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2023-07-06T03:44:05.381 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2023-07-06T03:44:05.381 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2023-07-06T03:44:05.381 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2023-07-06T03:44:05.381 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2023-07-06T03:44:05.382 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2023-07-06T03:44:05.382 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2023-07-06T03:44:05.382 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2023-07-06T03:44:05.383 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:44:05.383 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2023-07-06T03:44:05.383 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2023-07-06T03:44:05.384 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2023-07-06T03:44:05.384 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2023-07-06T03:44:05.384 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:44:05.384 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:44:05.385 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:44:05.387 INFO:tasks.ceph:Creating admin key on mon.a... 2023-07-06T03:44:05.387 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-07-06T03:44:05.499 INFO:tasks.ceph:Copying monmap to all nodes... 2023-07-06T03:44:05.499 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:05.500 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-07-06T03:44:05.552 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:05.552 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-07-06T03:44:05.640 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:44:05.640 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:05.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-06T03:44:05.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T03:44:05.777 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:05.778 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:05.864 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:44:05.865 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:05.865 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-06T03:44:05.865 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T03:44:05.962 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:05.963 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:06.047 INFO:tasks.ceph:Setting up mon nodes... 2023-07-06T03:44:06.047 INFO:tasks.ceph:Setting up mgr nodes... 2023-07-06T03:44:06.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-07-06T03:44:06.169 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-07-06T03:44:06.176 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-07-06T03:44:06.303 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-07-06T03:44:06.310 INFO:tasks.ceph:Setting up mds nodes... 2023-07-06T03:44:06.311 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-07-06T03:44:06.432 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-07-06T03:44:06.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-07-06T03:44:06.505 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-07-06T03:44:06.625 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-07-06T03:44:06.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-07-06T03:44:06.699 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-07-06T03:44:06.820 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-07-06T03:44:06.825 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-07-06T03:44:06.893 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-07-06T03:44:07.014 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-07-06T03:44:07.019 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-07-06T03:44:07.088 INFO:tasks.ceph_client:Setting up client nodes... 2023-07-06T03:44:07.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-06T03:44:07.184 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-07-06T03:44:07.213 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-07-06T03:44:07.309 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-07-06T03:44:07.339 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-07-06T03:44:07.435 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-07-06T03:44:07.465 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-07-06T03:44:07.560 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-07-06T03:44:07.590 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-07-06T03:44:07.591 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-06T03:44:07.591 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-07-06T03:44:07.660 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T03:44:07.660 INFO:tasks.ceph:role: osd.0 2023-07-06T03:44:07.660 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:44:07.660 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-06T03:44:07.736 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T03:44:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T03:44:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T03:44:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T03:44:07.737 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T03:44:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-07-06T03:44:07.737 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T03:44:07.737 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T03:44:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T03:44:07.738 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T03:44:08.169 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-07-06T03:44:08.174 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-07-06T03:44:08.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-07-06T03:44:08.365 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-07-06T03:44:08.446 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-07-06T03:44:08.514 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T03:44:08.515 INFO:tasks.ceph:role: osd.1 2023-07-06T03:44:08.515 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:44:08.515 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-06T03:44:08.590 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T03:44:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T03:44:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T03:44:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T03:44:08.591 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T03:44:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-07-06T03:44:08.591 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T03:44:08.591 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T03:44:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T03:44:08.592 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T03:44:09.004 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-07-06T03:44:09.008 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-07-06T03:44:09.008 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-07-06T03:44:09.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-07-06T03:44:09.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-07-06T03:44:09.301 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T03:44:09.301 INFO:tasks.ceph:role: osd.2 2023-07-06T03:44:09.301 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:44:09.301 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-06T03:44:09.377 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T03:44:09.377 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T03:44:09.377 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T03:44:09.378 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T03:44:09.378 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T03:44:09.378 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-07-06T03:44:09.378 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T03:44:09.378 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T03:44:09.378 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T03:44:09.378 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T03:44:09.826 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-07-06T03:44:09.830 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-07-06T03:44:09.830 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-07-06T03:44:10.028 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-07-06T03:44:10.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-07-06T03:44:10.171 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T03:44:10.171 INFO:tasks.ceph:role: osd.3 2023-07-06T03:44:10.172 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:44:10.172 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-06T03:44:10.247 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T03:44:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T03:44:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T03:44:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T03:44:10.248 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T03:44:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-07-06T03:44:10.248 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T03:44:10.248 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T03:44:10.249 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T03:44:10.249 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T03:44:10.686 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-07-06T03:44:10.689 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-07-06T03:44:10.690 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-07-06T03:44:10.813 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-07-06T03:44:10.889 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:11.007+0000 7fcff8e56640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-07-06T03:44:11.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:11.007+0000 7fcff8e56640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-07-06T03:44:11.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:11.007+0000 7fcff8e56640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T03:44:11.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:11.007+0000 7fcff8e56640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T03:44:11.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:11.007+0000 7fcff8e56640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T03:44:11.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:11.010+0000 7fcff8e56640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-07-06T03:44:13.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-07-06T03:44:13.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:14.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:14.057+0000 7fdc30289640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-07-06T03:44:14.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:14.058+0000 7fdc30289640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-07-06T03:44:14.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:14.058+0000 7fdc30289640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T03:44:14.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:14.058+0000 7fdc30289640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T03:44:14.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:14.058+0000 7fdc30289640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T03:44:14.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:14.059+0000 7fdc30289640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-07-06T03:44:16.940 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-07-06T03:44:17.013 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:17.116+0000 7f0ce05f8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-07-06T03:44:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:17.116+0000 7f0ce05f8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-07-06T03:44:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:17.116+0000 7f0ce05f8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T03:44:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:17.116+0000 7f0ce05f8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T03:44:17.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:17.116+0000 7f0ce05f8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T03:44:17.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:17.118+0000 7f0ce05f8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-07-06T03:44:20.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-07-06T03:44:20.073 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:20.176+0000 7f62c08c6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-07-06T03:44:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:20.176+0000 7f62c08c6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-07-06T03:44:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:20.176+0000 7f62c08c6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T03:44:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:20.176+0000 7f62c08c6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T03:44:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:20.176+0000 7f62c08c6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T03:44:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:20.178+0000 7f62c08c6640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-07-06T03:44:23.063 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-07-06T03:44:23.131 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-07-06T03:44:23.208 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T03:44:23.209 INFO:tasks.ceph:role: osd.4 2023-07-06T03:44:23.209 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:44:23.209 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-06T03:44:23.287 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T03:44:23.287 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T03:44:23.288 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T03:44:23.288 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T03:44:23.288 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T03:44:23.288 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2023-07-06T03:44:23.288 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T03:44:23.288 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T03:44:23.288 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T03:44:23.289 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T03:44:23.722 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2023-07-06T03:44:23.727 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2023-07-06T03:44:23.727 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-07-06T03:44:23.919 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-07-06T03:44:23.999 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-07-06T03:44:24.068 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T03:44:24.068 INFO:tasks.ceph:role: osd.5 2023-07-06T03:44:24.069 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:44:24.069 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-06T03:44:24.144 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T03:44:24.145 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T03:44:24.145 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T03:44:24.145 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T03:44:24.145 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T03:44:24.145 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2023-07-06T03:44:24.145 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T03:44:24.145 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T03:44:24.146 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T03:44:24.146 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T03:44:24.555 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2023-07-06T03:44:24.559 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2023-07-06T03:44:24.559 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-07-06T03:44:24.701 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-07-06T03:44:24.779 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-07-06T03:44:24.849 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T03:44:24.849 INFO:tasks.ceph:role: osd.6 2023-07-06T03:44:24.849 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:44:24.850 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-06T03:44:24.926 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T03:44:24.926 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T03:44:24.926 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T03:44:24.926 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T03:44:24.926 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T03:44:24.927 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2023-07-06T03:44:24.927 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T03:44:24.927 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T03:44:24.927 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T03:44:24.927 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T03:44:25.375 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2023-07-06T03:44:25.379 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2023-07-06T03:44:25.379 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-07-06T03:44:25.547 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-07-06T03:44:25.622 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-07-06T03:44:25.691 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T03:44:25.691 INFO:tasks.ceph:role: osd.7 2023-07-06T03:44:25.692 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:44:25.692 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-06T03:44:25.770 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T03:44:25.770 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T03:44:25.770 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T03:44:25.770 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T03:44:25.771 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T03:44:25.771 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2023-07-06T03:44:25.771 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T03:44:25.771 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T03:44:25.771 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T03:44:25.771 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T03:44:26.191 INFO:teuthology.orchestra.run.smithi146.stdout:Discarding blocks...Done. 2023-07-06T03:44:26.195 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2023-07-06T03:44:26.196 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-07-06T03:44:26.343 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-07-06T03:44:26.419 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:26.544 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:26.541+0000 7f650e30d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-07-06T03:44:26.546 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:26.541+0000 7f650e30d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-07-06T03:44:26.546 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:26.541+0000 7f650e30d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T03:44:26.546 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:26.541+0000 7f650e30d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T03:44:26.546 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:26.541+0000 7f650e30d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T03:44:26.547 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:26.543+0000 7f650e30d640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-07-06T03:44:29.428 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-07-06T03:44:29.500 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:29.605 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:29.603+0000 7f8c1a4a7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-07-06T03:44:29.606 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:29.603+0000 7f8c1a4a7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-07-06T03:44:29.606 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:29.603+0000 7f8c1a4a7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T03:44:29.606 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:29.603+0000 7f8c1a4a7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T03:44:29.606 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:29.603+0000 7f8c1a4a7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T03:44:29.607 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:29.605+0000 7f8c1a4a7640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-07-06T03:44:32.490 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-07-06T03:44:32.563 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:32.668 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:32.666+0000 7fa291906640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-07-06T03:44:32.669 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:32.666+0000 7fa291906640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-07-06T03:44:32.669 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:32.666+0000 7fa291906640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T03:44:32.669 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:32.666+0000 7fa291906640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T03:44:32.669 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:32.666+0000 7fa291906640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T03:44:32.670 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:32.668+0000 7fa291906640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-07-06T03:44:32.699 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:44:32.704 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:44:35.554 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-07-06T03:44:35.625 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:35.731 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:35.728+0000 7f2d54c20640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-07-06T03:44:35.731 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:35.728+0000 7f2d54c20640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-07-06T03:44:35.731 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:35.728+0000 7f2d54c20640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T03:44:35.731 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:35.728+0000 7f2d54c20640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T03:44:35.731 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:35.728+0000 7f2d54c20640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T03:44:35.732 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:35.730+0000 7f2d54c20640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-07-06T03:44:38.607 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-07-06T03:44:38.678 INFO:tasks.ceph:Reading keys from all nodes... 2023-07-06T03:44:38.678 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:38.679 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-07-06T03:44:38.753 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:38.754 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-07-06T03:44:38.823 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:38.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-07-06T03:44:38.931 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:38.931 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-07-06T03:44:39.038 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:39.038 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-07-06T03:44:39.144 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:39.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-07-06T03:44:39.251 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:39.252 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-07-06T03:44:39.359 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:39.360 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-07-06T03:44:39.432 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:39.432 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-07-06T03:44:39.501 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:39.501 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-07-06T03:44:39.610 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:39.610 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-07-06T03:44:39.718 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:39.719 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-07-06T03:44:39.832 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:39.833 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-07-06T03:44:39.940 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:39.941 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-07-06T03:44:40.049 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:40.050 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-07-06T03:44:40.098 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:40.099 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-07-06T03:44:40.185 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:40.185 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-07-06T03:44:40.230 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:40.231 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-07-06T03:44:40.314 INFO:tasks.ceph:Adding keys to all mons... 2023-07-06T03:44:40.315 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-06T03:44:40.320 DEBUG:teuthology.orchestra.run.smithi146:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-06T03:44:40.381 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.y] 2023-07-06T03:44:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCGOKZkfnoEChAAuRxqXqPI33dgWkBJoUMh3w== 2023-07-06T03:44:40.382 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-07-06T03:44:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCGOKZkczuvGRAAUKjfmMyKOTCYzOQmtf3n3A== 2023-07-06T03:44:40.382 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-07-06T03:44:40.382 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCGOKZk0N4+JRAA3tjqoIBPpZ6C7TMmOOg7lw== 2023-07-06T03:44:40.383 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2023-07-06T03:44:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCLOKZkShOIABAAzIBHDcNkmnDOLSY7Qdb+FA== 2023-07-06T03:44:40.383 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2023-07-06T03:44:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCOOKZkq1t7AxAAwihuhimbjitZDeki1QuMug== 2023-07-06T03:44:40.383 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2023-07-06T03:44:40.383 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCROKZkdYv4BhAApTHg/pmlC0lO7hVTkimEgw== 2023-07-06T03:44:40.383 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2023-07-06T03:44:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCUOKZkqhuMChAAsV9xG6g1dofrpWt3F8tw5w== 2023-07-06T03:44:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2023-07-06T03:44:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCGOKZkT4ALEhAAXwwkcodQU9meJA0tfseJCg== 2023-07-06T03:44:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-07-06T03:44:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCGOKZkFHvUMBAA2WgMcUUf+Qi8T2ZjhfWaHA== 2023-07-06T03:44:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-07-06T03:44:40.384 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZkJnHUABAAwIMH2DmoHaQN3frJCgRiQw== 2023-07-06T03:44:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.4] 2023-07-06T03:44:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCaOKZk1Q5aIBAABU1+afwIihm+g4xylryz/A== 2023-07-06T03:44:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.5] 2023-07-06T03:44:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCdOKZkRskIJBAA+821aoG98aEV7h4FTINQFA== 2023-07-06T03:44:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.6] 2023-07-06T03:44:40.385 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCgOKZk9qnMJxAATcWG+a/9Hl1AuIL/TSrdEA== 2023-07-06T03:44:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.7] 2023-07-06T03:44:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCjOKZk6wJ/KxAAWnZxt7h7OJhQ+bqdel4+iQ== 2023-07-06T03:44:40.386 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:44:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg== 2023-07-06T03:44:40.386 INFO:teuthology.orchestra.run.smithi097.stdout:[client.2] 2023-07-06T03:44:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZkhL5fEhAAWhw4IbCPqTO2sXPIcpTVHA== 2023-07-06T03:44:40.386 INFO:teuthology.orchestra.run.smithi097.stdout:[client.1] 2023-07-06T03:44:40.386 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZkq4/pGRAAucHmR1eBA8MNMy3/l7tl0g== 2023-07-06T03:44:40.386 INFO:teuthology.orchestra.run.smithi097.stdout:[client.3] 2023-07-06T03:44:40.387 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZkO81dIRAAffjVyvsb/ygBtx8UrYQK0w== 2023-07-06T03:44:40.420 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.y] 2023-07-06T03:44:40.420 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCGOKZkfnoEChAAuRxqXqPI33dgWkBJoUMh3w== 2023-07-06T03:44:40.420 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2023-07-06T03:44:40.420 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCGOKZkczuvGRAAUKjfmMyKOTCYzOQmtf3n3A== 2023-07-06T03:44:40.420 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2023-07-06T03:44:40.421 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCGOKZk0N4+JRAA3tjqoIBPpZ6C7TMmOOg7lw== 2023-07-06T03:44:40.421 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.0] 2023-07-06T03:44:40.421 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCLOKZkShOIABAAzIBHDcNkmnDOLSY7Qdb+FA== 2023-07-06T03:44:40.421 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.1] 2023-07-06T03:44:40.421 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCOOKZkq1t7AxAAwihuhimbjitZDeki1QuMug== 2023-07-06T03:44:40.421 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.2] 2023-07-06T03:44:40.422 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCROKZkdYv4BhAApTHg/pmlC0lO7hVTkimEgw== 2023-07-06T03:44:40.422 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.3] 2023-07-06T03:44:40.422 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCUOKZkqhuMChAAsV9xG6g1dofrpWt3F8tw5w== 2023-07-06T03:44:40.422 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.x] 2023-07-06T03:44:40.422 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCGOKZkT4ALEhAAXwwkcodQU9meJA0tfseJCg== 2023-07-06T03:44:40.423 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2023-07-06T03:44:40.423 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCGOKZkFHvUMBAA2WgMcUUf+Qi8T2ZjhfWaHA== 2023-07-06T03:44:40.423 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2023-07-06T03:44:40.423 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCHOKZkJnHUABAAwIMH2DmoHaQN3frJCgRiQw== 2023-07-06T03:44:40.423 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.4] 2023-07-06T03:44:40.423 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCaOKZk1Q5aIBAABU1+afwIihm+g4xylryz/A== 2023-07-06T03:44:40.423 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.5] 2023-07-06T03:44:40.424 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCdOKZkRskIJBAA+821aoG98aEV7h4FTINQFA== 2023-07-06T03:44:40.424 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.6] 2023-07-06T03:44:40.424 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCgOKZk9qnMJxAATcWG+a/9Hl1AuIL/TSrdEA== 2023-07-06T03:44:40.424 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.7] 2023-07-06T03:44:40.424 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCjOKZk6wJ/KxAAWnZxt7h7OJhQ+bqdel4+iQ== 2023-07-06T03:44:40.425 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2023-07-06T03:44:40.425 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg== 2023-07-06T03:44:40.425 INFO:teuthology.orchestra.run.smithi146.stdout:[client.2] 2023-07-06T03:44:40.425 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCHOKZkhL5fEhAAWhw4IbCPqTO2sXPIcpTVHA== 2023-07-06T03:44:40.425 INFO:teuthology.orchestra.run.smithi146.stdout:[client.1] 2023-07-06T03:44:40.425 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCHOKZkq4/pGRAAucHmR1eBA8MNMy3/l7tl0g== 2023-07-06T03:44:40.425 INFO:teuthology.orchestra.run.smithi146.stdout:[client.3] 2023-07-06T03:44:40.425 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQCHOKZkO81dIRAAffjVyvsb/ygBtx8UrYQK0w== 2023-07-06T03:44:40.426 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T03:44:40.432 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T03:44:40.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T03:44:40.578 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T03:44:40.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T03:44:40.694 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T03:44:40.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:40.809 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:40.917 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:40.921 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.033 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.141 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.145 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T03:44:41.258 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T03:44:41.365 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T03:44:41.369 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T03:44:41.481 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T03:44:41.486 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T03:44:41.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.602 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.715 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.823 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.827 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:41.939 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T03:44:42.045 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T03:44:42.050 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T03:44:42.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T03:44:42.165 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T03:44:42.276 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T03:44:42.281 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T03:44:42.392 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T03:44:42.397 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T03:44:42.509 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-07-06T03:44:42.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-07-06T03:44:42.577 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T03:44:42.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-07-06T03:44:42.770 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-07-06T03:44:42.840 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T03:44:42.962 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-07-06T03:44:43.031 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-07-06T03:44:43.100 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T03:44:43.223 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-07-06T03:44:43.293 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:43.297 DEBUG:teuthology.orchestra.run.smithi146:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-06T03:44:43.342 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-07-06T03:44:43.343 INFO:tasks.ceph.mon.a:Restarting daemon 2023-07-06T03:44:43.343 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-07-06T03:44:43.385 INFO:tasks.ceph.mon.a:Started 2023-07-06T03:44:43.385 INFO:tasks.ceph.mon.b:Restarting daemon 2023-07-06T03:44:43.386 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-07-06T03:44:43.390 INFO:tasks.ceph.mon.b:Started 2023-07-06T03:44:43.390 INFO:tasks.ceph.mon.c:Restarting daemon 2023-07-06T03:44:43.390 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-07-06T03:44:43.394 INFO:tasks.ceph.mon.c:Started 2023-07-06T03:44:43.394 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-07-06T03:44:43.395 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-07-06T03:44:43.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-07-06T03:44:43.399 INFO:tasks.ceph.mgr.y:Started 2023-07-06T03:44:43.399 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-07-06T03:44:43.400 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-07-06T03:44:43.403 INFO:tasks.ceph.mgr.x:Started 2023-07-06T03:44:43.404 INFO:tasks.ceph:Setting crush tunables to default 2023-07-06T03:44:43.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd crush tunables default 2023-07-06T03:44:43.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.598+0000 7f73171ff700 1 Processor -- start 2023-07-06T03:44:43.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.598+0000 7f73171ff700 1 -- start start 2023-07-06T03:44:43.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.599+0000 7f73171ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7310159070 0x7f73100c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:43.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.599+0000 7f73171ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 0x7f73100cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:43.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.599+0000 7f73171ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73100ca260 con 0x7f7310159440 2023-07-06T03:44:43.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.599+0000 7f73171ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73100c99c0 con 0x7f7310159070 2023-07-06T03:44:43.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.599+0000 7f73171ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73100c9ca0 con 0x7f73100c3cd0 2023-07-06T03:44:43.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.599+0000 7f730ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 0x7f73100cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:43.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.599+0000 7f730ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 0x7f73100cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:42504/0 (socket says 172.21.15.97:42504) 2023-07-06T03:44:43.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.599+0000 7f730ffff700 1 -- 172.21.15.97:0/695196956 learned_addr learned my addr 172.21.15.97:0/695196956 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:44:43.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.600+0000 7f730ffff700 1 -- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 msgr2=0x7f73100cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-06T03:44:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.600+0000 7f730ffff700 1 -- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 msgr2=0x7f73100cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-06T03:44:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.600+0000 7f730ffff700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 0x7f73100cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-06T03:44:43.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.600+0000 7f730ffff700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 0x7f73100cc4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T03:44:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.600+0000 7f7314f9b700 1 --2- 172.21.15.97:0/695196956 >> v2:172.21.15.146:3300/0 conn(0x7f7310159070 0x7f73100c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.600+0000 7f7314f9b700 1 -- 172.21.15.97:0/695196956 >> v2:172.21.15.146:3300/0 conn(0x7f7310159070 msgr2=0x7f73100c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-07-06T03:44:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.600+0000 7f7314f9b700 1 -- 172.21.15.97:0/695196956 >> v2:172.21.15.146:3300/0 conn(0x7f7310159070 msgr2=0x7f73100c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-06T03:44:43.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.600+0000 7f7314f9b700 1 --2- 172.21.15.97:0/695196956 >> v2:172.21.15.146:3300/0 conn(0x7f7310159070 0x7f73100c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-06T03:44:43.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.600+0000 7f7314f9b700 1 --2- 172.21.15.97:0/695196956 >> v2:172.21.15.146:3300/0 conn(0x7f7310159070 0x7f73100c8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T03:44:43.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.790+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372725499 0 0) 0x7f73100ca260 con 0x7f7310159440 2023-07-06T03:44:43.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.790+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73000035b0 con 0x7f7310159440 2023-07-06T03:44:43.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.790+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3785629378 0 0) 0x7f73000035b0 con 0x7f7310159440 2023-07-06T03:44:43.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.790+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 msgr2=0x7f73100cc4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T03:44:43.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.790+0000 7f730f7fe700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 0x7f73100cc4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:43.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.790+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 >> v2:172.21.15.146:3300/0 conn(0x7f7310159070 msgr2=0x7f73100c8f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T03:44:43.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.790+0000 7f730f7fe700 1 --2- 172.21.15.97:0/695196956 >> v2:172.21.15.146:3300/0 conn(0x7f7310159070 0x7f73100c8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:43.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.790+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73100c9f80 con 0x7f7310159440 2023-07-06T03:44:43.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.790+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f73080019f0 con 0x7f7310159440 2023-07-06T03:44:43.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.790+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 >> v1:172.21.15.97:6789/0 conn(0x7f7310159440 legacy=0x7f73100b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:43.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f730f7fe700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7300003a00 0x7f7300003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:43.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f730f7fe700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7300004850 0x7f7300004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:43.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f730f7fe700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 0x7f7300005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:43.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73100c99c0 con 0x7f7300003a00 2023-07-06T03:44:43.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73100c9ca0 con 0x7f7300004850 2023-07-06T03:44:43.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730000bb40 con 0x7f73100c3cd0 2023-07-06T03:44:43.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f7314f9b700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7300003a00 0x7f7300003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:43.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f730ffff700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7300004850 0x7f7300004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:43.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f7314f9b700 1 -- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 msgr2=0x7f7300005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:43.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f7314f9b700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73100c3cd0 0x7f7300005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:43.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f7314f9b700 1 -- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7300004850 msgr2=0x7f7300004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:43.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f7314f9b700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7300004850 0x7f7300004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:43.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f7314f9b700 1 -- 172.21.15.97:0/695196956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000035b0 con 0x7f7300003a00 2023-07-06T03:44:43.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f730ffff700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7300004850 0x7f7300004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:44:43.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f7314f9b700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7300003a00 0x7f7300003e10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7304009890 tx=0x7f7300005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7027425d8250c2d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:43.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.791+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730400c070 con 0x7f7300003a00 2023-07-06T03:44:43.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.792+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7304004020 con 0x7f7300003a00 2023-07-06T03:44:43.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.792+0000 7f730f7fe700 1 -- 172.21.15.97:0/695196956 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73040043b0 con 0x7f7300003a00 2023-07-06T03:44:43.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.792+0000 7f73171ff700 1 -- 172.21.15.97:0/695196956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7300003a00 msgr2=0x7f7300003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:43.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.792+0000 7f73171ff700 1 --2- 172.21.15.97:0/695196956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7300003a00 0x7f7300003e10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7304009890 tx=0x7f7300005880 comp rx=0 tx=0).stop 2023-07-06T03:44:43.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.792+0000 7f731579c700 1 -- 172.21.15.97:0/695196956 reap_dead start 2023-07-06T03:44:43.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.792+0000 7f73171ff700 1 -- 172.21.15.97:0/695196956 shutdown_connections 2023-07-06T03:44:43.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.792+0000 7f73171ff700 1 -- 172.21.15.97:0/695196956 >> 172.21.15.97:0/695196956 conn(0x7f731006a060 msgr2=0x7f731006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:43.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.792+0000 7f73171ff700 1 -- 172.21.15.97:0/695196956 shutdown_connections 2023-07-06T03:44:43.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.792+0000 7f73171ff700 1 -- 172.21.15.97:0/695196956 wait complete. 2023-07-06T03:44:43.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.794+0000 7f73171ff700 1 Processor -- start 2023-07-06T03:44:43.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.795+0000 7f73171ff700 1 -- start start 2023-07-06T03:44:43.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f73171ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7300003a00 0x7f731014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:43.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f73171ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73100b2920 0x7f73100b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:43.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f73171ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7310159880 0x7f7310159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:43.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f73171ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73100b32d0 con 0x7f7310159880 2023-07-06T03:44:43.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f73171ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731015b050 con 0x7f73100b2920 2023-07-06T03:44:43.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f73171ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731015a210 con 0x7f7300003a00 2023-07-06T03:44:43.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f731579c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7310159880 0x7f7310159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:43.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f731579c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7310159880 0x7f7310159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39982/0 (socket says 172.21.15.97:39982) 2023-07-06T03:44:43.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f731579c700 1 -- 172.21.15.97:0/1274116028 learned_addr learned my addr 172.21.15.97:0/1274116028 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:44:43.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f7314f9b700 1 --2- 172.21.15.97:0/1274116028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7300003a00 0x7f731014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:43.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f730ffff700 1 --2- 172.21.15.97:0/1274116028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73100b2920 0x7f73100b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:43.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f731579c700 1 -- 172.21.15.97:0/1274116028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7300003a00 msgr2=0x7f731014d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:43.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f731579c700 1 --2- 172.21.15.97:0/1274116028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7300003a00 0x7f731014d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:43.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f731579c700 1 -- 172.21.15.97:0/1274116028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73100b2920 msgr2=0x7f73100b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f731579c700 1 --2- 172.21.15.97:0/1274116028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73100b2920 0x7f73100b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f731579c700 1 -- 172.21.15.97:0/1274116028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73100b1410 con 0x7f7310159880 2023-07-06T03:44:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.796+0000 7f730ffff700 1 --2- 172.21.15.97:0/1274116028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73100b2920 0x7f73100b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:44:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.797+0000 7f731579c700 1 --2- 172.21.15.97:0/1274116028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7310159880 0x7f7310159cd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f73100ca830 tx=0x7f730800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.797+0000 7f730d7fa700 1 -- 172.21.15.97:0/1274116028 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730800a760 con 0x7f7310159880 2023-07-06T03:44:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.797+0000 7f730d7fa700 1 -- 172.21.15.97:0/1274116028 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730800a8c0 con 0x7f7310159880 2023-07-06T03:44:43.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.797+0000 7f73171ff700 1 -- 172.21.15.97:0/1274116028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731015a460 con 0x7f7310159880 2023-07-06T03:44:43.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.797+0000 7f730d7fa700 1 -- 172.21.15.97:0/1274116028 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730800ac30 con 0x7f7310159880 2023-07-06T03:44:43.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.797+0000 7f73171ff700 1 -- 172.21.15.97:0/1274116028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731015ace0 con 0x7f7310159880 2023-07-06T03:44:43.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.797+0000 7f730d7fa700 1 -- 172.21.15.97:0/1274116028 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f730800ad90 con 0x7f7310159880 2023-07-06T03:44:43.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.797+0000 7f730d7fa700 1 -- 172.21.15.97:0/1274116028 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7308004870 con 0x7f7310159880 2023-07-06T03:44:43.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.798+0000 7f73171ff700 1 -- 172.21.15.97:0/1274116028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731015a980 con 0x7f7310159880 2023-07-06T03:44:43.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.801+0000 7f730d7fa700 1 -- 172.21.15.97:0/1274116028 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f731015a980 con 0x7f7310159880 2023-07-06T03:44:43.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.880+0000 7f73171ff700 1 -- 172.21.15.97:0/1274116028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f731015b1e0 con 0x7f7310159880 2023-07-06T03:44:43.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.883+0000 7f730d7fa700 1 -- 172.21.15.97:0/1274116028 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7308017070 con 0x7f7310159880 2023-07-06T03:44:43.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.884+0000 7f73171ff700 1 -- 172.21.15.97:0/1274116028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7310159880 msgr2=0x7f7310159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:43.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.884+0000 7f73171ff700 1 --2- 172.21.15.97:0/1274116028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7310159880 0x7f7310159cd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f73100ca830 tx=0x7f730800a060 comp rx=0 tx=0).stop 2023-07-06T03:44:43.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.884+0000 7f73171ff700 1 -- 172.21.15.97:0/1274116028 shutdown_connections 2023-07-06T03:44:43.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.884+0000 7f73171ff700 1 --2- 172.21.15.97:0/1274116028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7310159880 0x7f7310159cd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:43.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.884+0000 7f73171ff700 1 --2- 172.21.15.97:0/1274116028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7300003a00 0x7f731014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:43.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.884+0000 7f73171ff700 1 --2- 172.21.15.97:0/1274116028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73100b2920 0x7f73100b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:43.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.884+0000 7f73171ff700 1 -- 172.21.15.97:0/1274116028 >> 172.21.15.97:0/1274116028 conn(0x7f731006a060 msgr2=0x7f731006c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:43.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.885+0000 7f73171ff700 1 -- 172.21.15.97:0/1274116028 shutdown_connections 2023-07-06T03:44:43.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:43.885+0000 7f73171ff700 1 -- 172.21.15.97:0/1274116028 wait complete. 2023-07-06T03:44:43.888 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2023-07-06T03:44:43.903 INFO:tasks.ceph:check_enable_crimson: False 2023-07-06T03:44:43.904 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-07-06T03:44:43.904 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:43.904 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-07-06T03:44:43.967 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:43.965+0000 7ff302edf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T03:44:43.970 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:43.968+0000 7f51b8d48200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T03:44:43.972 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:43.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-07-06T03:44:44.083 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:44.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-07-06T03:44:44.179 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:44.177+0000 7f51b8d48200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T03:44:44.185 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:44.183+0000 7ff302edf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T03:44:44.206 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:44:44.206 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-07-06T03:44:44.275 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:44.276 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-07-06T03:44:44.344 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:44.345 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-07-06T03:44:44.457 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:44.457 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-07-06T03:44:44.572 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:44:44.573 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-07-06T03:44:44.622 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:44.621+0000 7ff302edf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T03:44:44.626 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:44.624+0000 7f51b8d48200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T03:44:44.647 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c 0 2023-07-06T03:44:44.841 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.838+0000 7fd57c9ff700 1 Processor -- start 2023-07-06T03:44:44.842 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.839+0000 7fd57c9ff700 1 -- start start 2023-07-06T03:44:44.844 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.840+0000 7fd57c9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd57815c610 0x7fd5780b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:44.845 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.840+0000 7fd57c9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd57806b360 0x7fd5780b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:44.845 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.840+0000 7fd57c9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 0x7fd5780d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:44.845 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.840+0000 7fd57c9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5780d0780 con 0x7fd57815c610 2023-07-06T03:44:44.845 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.840+0000 7fd57c9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57815c9e0 con 0x7fd57806b360 2023-07-06T03:44:44.845 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.840+0000 7fd57c9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780b5650 con 0x7fd5780d2210 2023-07-06T03:44:44.845 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.840+0000 7fd56ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd57806b360 0x7fd5780b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:44.845 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.840+0000 7fd576d9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 0x7fd5780d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:44.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.840+0000 7fd56ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd57806b360 0x7fd5780b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50648/0 (socket says 172.21.15.146:50648) 2023-07-06T03:44:44.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.840+0000 7fd56ffff700 1 -- 172.21.15.146:0/3698873008 learned_addr learned my addr 172.21.15.146:0/3698873008 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:44.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.840+0000 7fd57659c700 1 --2- 172.21.15.146:0/3698873008 >> v2:172.21.15.97:3300/0 conn(0x7fd57815c610 0x7fd5780b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:44.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.841+0000 7fd56ffff700 1 -- 172.21.15.146:0/3698873008 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 msgr2=0x7fd5780d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:44.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.841+0000 7fd56ffff700 1 --2- 172.21.15.146:0/3698873008 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 0x7fd5780d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:44.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.841+0000 7fd56ffff700 1 -- 172.21.15.146:0/3698873008 >> v2:172.21.15.97:3300/0 conn(0x7fd57815c610 msgr2=0x7fd5780b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:44.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.841+0000 7fd56ffff700 1 --2- 172.21.15.146:0/3698873008 >> v2:172.21.15.97:3300/0 conn(0x7fd57815c610 0x7fd5780b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:44.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.841+0000 7fd56ffff700 1 -- 172.21.15.146:0/3698873008 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780d01c0 con 0x7fd57806b360 2023-07-06T03:44:44.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.841+0000 7fd56ffff700 1 --2- 172.21.15.146:0/3698873008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57806b360 0x7fd5780b4cf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd57806a590 tx=0x7fd560009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e09f69f0820b27 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:44.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.841+0000 7fd575d9b700 1 -- 172.21.15.146:0/3698873008 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560011830 con 0x7fd57806b360 2023-07-06T03:44:44.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.841+0000 7fd575d9b700 1 -- 172.21.15.146:0/3698873008 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd560011990 con 0x7fd57806b360 2023-07-06T03:44:44.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.841+0000 7fd575d9b700 1 -- 172.21.15.146:0/3698873008 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560011d20 con 0x7fd57806b360 2023-07-06T03:44:44.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.842+0000 7fd57c9ff700 1 -- 172.21.15.146:0/3698873008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57806b360 msgr2=0x7fd5780b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:44.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.842+0000 7fd57c9ff700 1 --2- 172.21.15.146:0/3698873008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57806b360 0x7fd5780b4cf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd57806a590 tx=0x7fd560009bd0 comp rx=0 tx=0).stop 2023-07-06T03:44:44.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.842+0000 7fd57c9ff700 1 -- 172.21.15.146:0/3698873008 shutdown_connections 2023-07-06T03:44:44.848 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.842+0000 7fd57c9ff700 1 --2- 172.21.15.146:0/3698873008 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 0x7fd5780d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:44.848 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.842+0000 7fd57c9ff700 1 --2- 172.21.15.146:0/3698873008 >> v2:172.21.15.97:3300/0 conn(0x7fd57815c610 0x7fd5780b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:44.848 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.842+0000 7fd57c9ff700 1 --2- 172.21.15.146:0/3698873008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57806b360 0x7fd5780b4cf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:44.848 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.842+0000 7fd57c9ff700 1 -- 172.21.15.146:0/3698873008 >> 172.21.15.146:0/3698873008 conn(0x7fd57806a6c0 msgr2=0x7fd57806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:44.848 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.842+0000 7fd57c9ff700 1 -- 172.21.15.146:0/3698873008 shutdown_connections 2023-07-06T03:44:44.848 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.842+0000 7fd57c9ff700 1 -- 172.21.15.146:0/3698873008 wait complete. 2023-07-06T03:44:44.850 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.844+0000 7fd57c9ff700 1 Processor -- start 2023-07-06T03:44:44.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.844+0000 7fd57c9ff700 1 -- start start 2023-07-06T03:44:44.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.845+0000 7fd57c9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57806b360 0x7fd5781f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:44.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.845+0000 7fd57c9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 0x7fd57815d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:44.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.845+0000 7fd57c9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd57815c610 0x7fd57815e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:44.851 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.845+0000 7fd57c9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780c9f80 con 0x7fd57815c610 2023-07-06T03:44:44.852 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.845+0000 7fd57c9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780ca260 con 0x7fd57806b360 2023-07-06T03:44:44.852 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.845+0000 7fd57c9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780ca540 con 0x7fd5780d2210 2023-07-06T03:44:44.852 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.845+0000 7fd56ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 0x7fd57815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:44.852 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.845+0000 7fd56ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 0x7fd57815d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:55936/0 (socket says 172.21.15.146:55936) 2023-07-06T03:44:44.853 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.845+0000 7fd56ffff700 1 -- 172.21.15.146:0/1410237850 learned_addr learned my addr 172.21.15.146:0/1410237850 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:44.853 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.845+0000 7fd576d9d700 1 --2- 172.21.15.146:0/1410237850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd57815c610 0x7fd57815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:44.853 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.845+0000 7fd57659c700 1 --2- 172.21.15.146:0/1410237850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57806b360 0x7fd5781f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:44.854 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd576d9d700 1 -- 172.21.15.146:0/1410237850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 msgr2=0x7fd57815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:44.854 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd576d9d700 1 --2- 172.21.15.146:0/1410237850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 0x7fd57815d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:44.854 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd576d9d700 1 -- 172.21.15.146:0/1410237850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57806b360 msgr2=0x7fd5781f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:44.855 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd576d9d700 1 --2- 172.21.15.146:0/1410237850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57806b360 0x7fd5781f2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:44.855 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd576d9d700 1 -- 172.21.15.146:0/1410237850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd560009550 con 0x7fd57815c610 2023-07-06T03:44:44.855 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd56ffff700 1 --2- 172.21.15.146:0/1410237850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 0x7fd57815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:44:44.855 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd576d9d700 1 --2- 172.21.15.146:0/1410237850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd57815c610 0x7fd57815e280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd57000eed0 tx=0x7fd57000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:44.856 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd56effd700 1 -- 172.21.15.146:0/1410237850 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570010040 con 0x7fd57815c610 2023-07-06T03:44:44.856 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd56effd700 1 -- 172.21.15.146:0/1410237850 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57000ba80 con 0x7fd57815c610 2023-07-06T03:44:44.856 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd57c9ff700 1 -- 172.21.15.146:0/1410237850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5780ca820 con 0x7fd57815c610 2023-07-06T03:44:44.857 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd56effd700 1 -- 172.21.15.146:0/1410237850 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000be30 con 0x7fd57815c610 2023-07-06T03:44:44.857 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.846+0000 7fd57c9ff700 1 -- 172.21.15.146:0/1410237850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5781f3110 con 0x7fd57815c610 2023-07-06T03:44:44.857 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.847+0000 7fd56effd700 1 -- 172.21.15.146:0/1410237850 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd57001d700 con 0x7fd57815c610 2023-07-06T03:44:44.857 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.847+0000 7fd56effd700 1 -- 172.21.15.146:0/1410237850 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd570014830 con 0x7fd57815c610 2023-07-06T03:44:44.858 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.848+0000 7fd57c9ff700 1 -- 172.21.15.146:0/1410237850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5780ca0c0 con 0x7fd57815c610 2023-07-06T03:44:44.858 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.850+0000 7fd56effd700 1 -- 172.21.15.146:0/1410237850 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fd57003bb00 con 0x7fd57815c610 2023-07-06T03:44:44.938 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.936+0000 7fd57c9ff700 1 -- 172.21.15.146:0/1410237850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c", "id": 0} v 0) v1 -- 0x7fd5780caa10 con 0x7fd57815c610 2023-07-06T03:44:44.946 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.943+0000 7fd56effd700 1 -- 172.21.15.146:0/1410237850 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd57002a030 con 0x7fd57815c610 2023-07-06T03:44:44.946 INFO:teuthology.orchestra.run.smithi146.stdout:0 2023-07-06T03:44:44.947 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.944+0000 7fd57c9ff700 1 -- 172.21.15.146:0/1410237850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd57815c610 msgr2=0x7fd57815e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:44.947 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.944+0000 7fd57c9ff700 1 --2- 172.21.15.146:0/1410237850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd57815c610 0x7fd57815e280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd57000eed0 tx=0x7fd57000b390 comp rx=0 tx=0).stop 2023-07-06T03:44:44.947 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.945+0000 7fd57c9ff700 1 -- 172.21.15.146:0/1410237850 shutdown_connections 2023-07-06T03:44:44.947 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.945+0000 7fd57c9ff700 1 --2- 172.21.15.146:0/1410237850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd57815c610 0x7fd57815e280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:44.948 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.945+0000 7fd57c9ff700 1 --2- 172.21.15.146:0/1410237850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd57806b360 0x7fd5781f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:44.948 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.945+0000 7fd57c9ff700 1 --2- 172.21.15.146:0/1410237850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5780d2210 0x7fd57815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:44.948 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.945+0000 7fd57c9ff700 1 -- 172.21.15.146:0/1410237850 >> 172.21.15.146:0/1410237850 conn(0x7fd57806a6c0 msgr2=0x7fd57815ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:44.948 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.945+0000 7fd57c9ff700 1 -- 172.21.15.146:0/1410237850 shutdown_connections 2023-07-06T03:44:44.949 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:44.945+0000 7fd57c9ff700 1 -- 172.21.15.146:0/1410237850 wait complete. 2023-07-06T03:44:44.965 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new bfd65829-a141-4776-93fa-ec4a853e8b5a 1 2023-07-06T03:44:45.172 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.170+0000 7fced3fff700 1 Processor -- start 2023-07-06T03:44:45.173 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.170+0000 7fced3fff700 1 -- start start 2023-07-06T03:44:45.175 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.172+0000 7fced3fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b8890 0x7fcecc0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.175 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.172+0000 7fced3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcecc0cdb10 0x7fcecc0d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.175 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.172+0000 7fced3fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcecc0b5280 con 0x7fcecc0b4d70 2023-07-06T03:44:45.176 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.172+0000 7fced3fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcecc0d0de0 con 0x7fcecc0cdb10 2023-07-06T03:44:45.176 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.172+0000 7fced3fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0d10c0 con 0x7fcecc0b8890 2023-07-06T03:44:45.176 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.173+0000 7fced1d9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b8890 0x7fcecc0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.176 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.173+0000 7fced1d9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b8890 0x7fcecc0b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:55952/0 (socket says 172.21.15.146:55952) 2023-07-06T03:44:45.176 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.173+0000 7fced1d9b700 1 -- 172.21.15.146:0/2371026348 learned_addr learned my addr 172.21.15.146:0/2371026348 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:45.176 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.173+0000 7fced259c700 1 --2- 172.21.15.146:0/2371026348 >> v2:172.21.15.97:3300/0 conn(0x7fcecc0cdb10 0x7fcecc0d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.178 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.173+0000 7fced1d9b700 1 -- 172.21.15.146:0/2371026348 >> v2:172.21.15.97:3300/0 conn(0x7fcecc0cdb10 msgr2=0x7fcecc0d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:45.178 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.173+0000 7fced1d9b700 1 --2- 172.21.15.146:0/2371026348 >> v2:172.21.15.97:3300/0 conn(0x7fcecc0cdb10 0x7fcecc0d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.178 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.173+0000 7fced1d9b700 1 -- 172.21.15.146:0/2371026348 >> v1:172.21.15.97:6789/0 conn(0x7fcecc0b4d70 legacy=0x7fcecc0cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.178 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.173+0000 7fced1d9b700 1 -- 172.21.15.146:0/2371026348 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc0d13a0 con 0x7fcecc0b8890 2023-07-06T03:44:45.178 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.173+0000 7fced1d9b700 1 --2- 172.21.15.146:0/2371026348 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b8890 0x7fcecc0b4830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcec80098f0 tx=0x7fcec8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28a04b5244e168ac server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:45.178 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.174+0000 7fced0d99700 1 -- 172.21.15.146:0/2371026348 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8011830 con 0x7fcecc0b8890 2023-07-06T03:44:45.178 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.174+0000 7fced0d99700 1 -- 172.21.15.146:0/2371026348 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec8011990 con 0x7fcecc0b8890 2023-07-06T03:44:45.179 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.174+0000 7fced0d99700 1 -- 172.21.15.146:0/2371026348 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8011d20 con 0x7fcecc0b8890 2023-07-06T03:44:45.179 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.174+0000 7fced3fff700 1 -- 172.21.15.146:0/2371026348 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b8890 msgr2=0x7fcecc0b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.179 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.174+0000 7fced3fff700 1 --2- 172.21.15.146:0/2371026348 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b8890 0x7fcecc0b4830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcec80098f0 tx=0x7fcec8009c00 comp rx=0 tx=0).stop 2023-07-06T03:44:45.179 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.174+0000 7fced3fff700 1 -- 172.21.15.146:0/2371026348 shutdown_connections 2023-07-06T03:44:45.179 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.174+0000 7fced3fff700 1 --2- 172.21.15.146:0/2371026348 >> v2:172.21.15.97:3300/0 conn(0x7fcecc0cdb10 0x7fcecc0d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.179 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.174+0000 7fced3fff700 1 --2- 172.21.15.146:0/2371026348 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b8890 0x7fcecc0b4830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.179 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.174+0000 7fced3fff700 1 -- 172.21.15.146:0/2371026348 >> 172.21.15.146:0/2371026348 conn(0x7fcecc06b2d0 msgr2=0x7fcecc0cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:45.179 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.175+0000 7fced3fff700 1 -- 172.21.15.146:0/2371026348 shutdown_connections 2023-07-06T03:44:45.180 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.175+0000 7fced3fff700 1 -- 172.21.15.146:0/2371026348 wait complete. 2023-07-06T03:44:45.181 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.176+0000 7fced3fff700 1 Processor -- start 2023-07-06T03:44:45.182 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.176+0000 7fced3fff700 1 -- start start 2023-07-06T03:44:45.182 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.177+0000 7fced3fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b4d70 0x7fcecc15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.182 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.177+0000 7fced3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcecc0b8890 0x7fcecc0cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.182 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.177+0000 7fced3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcecc0cdb10 0x7fcecc14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.182 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.177+0000 7fced3fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc15d390 con 0x7fcecc0cdb10 2023-07-06T03:44:45.182 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.177+0000 7fced3fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc15d670 con 0x7fcecc0b8890 2023-07-06T03:44:45.182 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.177+0000 7fced3fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc15d950 con 0x7fcecc0b4d70 2023-07-06T03:44:45.183 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced159a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcecc0b8890 0x7fcecc0cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.183 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced1d9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b4d70 0x7fcecc15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.183 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced159a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcecc0b8890 0x7fcecc0cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50672/0 (socket says 172.21.15.146:50672) 2023-07-06T03:44:45.183 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced159a700 1 -- 172.21.15.146:0/171414387 learned_addr learned my addr 172.21.15.146:0/171414387 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:45.183 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced259c700 1 --2- 172.21.15.146:0/171414387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcecc0cdb10 0x7fcecc14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.183 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced159a700 1 -- 172.21.15.146:0/171414387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b4d70 msgr2=0x7fcecc15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.184 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced159a700 1 --2- 172.21.15.146:0/171414387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b4d70 0x7fcecc15cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.184 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced159a700 1 -- 172.21.15.146:0/171414387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcecc0cdb10 msgr2=0x7fcecc14ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.184 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced159a700 1 --2- 172.21.15.146:0/171414387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcecc0cdb10 0x7fcecc14ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.184 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced159a700 1 -- 172.21.15.146:0/171414387 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec8009550 con 0x7fcecc0b8890 2023-07-06T03:44:45.184 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced1d9b700 1 --2- 172.21.15.146:0/171414387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b4d70 0x7fcecc15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:44:45.185 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.178+0000 7fced159a700 1 --2- 172.21.15.146:0/171414387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcecc0b8890 0x7fcecc0cca40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00c450 tx=0x7fcebc00c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:45.185 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.179+0000 7fcec27fc700 1 -- 172.21.15.146:0/171414387 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0052d0 con 0x7fcecc0b8890 2023-07-06T03:44:45.185 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.179+0000 7fcec27fc700 1 -- 172.21.15.146:0/171414387 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc005430 con 0x7fcecc0b8890 2023-07-06T03:44:45.185 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.179+0000 7fced3fff700 1 -- 172.21.15.146:0/171414387 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcecc15dc30 con 0x7fcecc0b8890 2023-07-06T03:44:45.185 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.179+0000 7fcec27fc700 1 -- 172.21.15.146:0/171414387 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0038c0 con 0x7fcecc0b8890 2023-07-06T03:44:45.185 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.179+0000 7fced3fff700 1 -- 172.21.15.146:0/171414387 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcecc150510 con 0x7fcecc0b8890 2023-07-06T03:44:45.185 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.179+0000 7fcec27fc700 1 -- 172.21.15.146:0/171414387 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcebc003d20 con 0x7fcecc0b8890 2023-07-06T03:44:45.185 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.179+0000 7fcec27fc700 1 -- 172.21.15.146:0/171414387 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fcebc011590 con 0x7fcecc0b8890 2023-07-06T03:44:45.186 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.180+0000 7fced3fff700 1 -- 172.21.15.146:0/171414387 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcecc0b5be0 con 0x7fcecc0b8890 2023-07-06T03:44:45.186 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.183+0000 7fcec27fc700 1 -- 172.21.15.146:0/171414387 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fcecc0b5be0 con 0x7fcecc0b8890 2023-07-06T03:44:45.211 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:45.209+0000 7ff302edf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T03:44:45.286 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:45.283+0000 7f51b8d48200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T03:44:45.296 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.293+0000 7fced3fff700 1 -- 172.21.15.146:0/171414387 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bfd65829-a141-4776-93fa-ec4a853e8b5a", "id": 1} v 0) v1 -- 0x7fcecc04e340 con 0x7fcecc0b8890 2023-07-06T03:44:45.304 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.301+0000 7fcec27fc700 1 -- 172.21.15.146:0/171414387 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bfd65829-a141-4776-93fa-ec4a853e8b5a", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcebc021060 con 0x7fcecc0b8890 2023-07-06T03:44:45.304 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:44:45.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.302+0000 7fced3fff700 1 -- 172.21.15.146:0/171414387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcecc0b8890 msgr2=0x7fcecc0cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.302+0000 7fced3fff700 1 --2- 172.21.15.146:0/171414387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcecc0b8890 0x7fcecc0cca40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00c450 tx=0x7fcebc00c760 comp rx=0 tx=0).stop 2023-07-06T03:44:45.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.302+0000 7fced3fff700 1 -- 172.21.15.146:0/171414387 shutdown_connections 2023-07-06T03:44:45.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.302+0000 7fced3fff700 1 --2- 172.21.15.146:0/171414387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcecc0cdb10 0x7fcecc14ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.302+0000 7fced3fff700 1 --2- 172.21.15.146:0/171414387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcecc0b4d70 0x7fcecc15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.305 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.302+0000 7fced3fff700 1 --2- 172.21.15.146:0/171414387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcecc0b8890 0x7fcecc0cca40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.302+0000 7fced3fff700 1 -- 172.21.15.146:0/171414387 >> 172.21.15.146:0/171414387 conn(0x7fcecc06b2d0 msgr2=0x7fcecc0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.303+0000 7fced3fff700 1 -- 172.21.15.146:0/171414387 shutdown_connections 2023-07-06T03:44:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.303+0000 7fced3fff700 1 -- 172.21.15.146:0/171414387 wait complete. 2023-07-06T03:44:45.322 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 236bb1dc-a348-413d-aac2-91bfceb3d132 2 2023-07-06T03:44:45.446 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:45.445+0000 7ff302edf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T03:44:45.516 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.514+0000 7f7fb6bff700 1 Processor -- start 2023-07-06T03:44:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.514+0000 7f7fb6bff700 1 -- start start 2023-07-06T03:44:45.518 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.515+0000 7f7fb6bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7fb015c610 0x7f7fb00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.518 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.515+0000 7f7fb6bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb00d2210 0x7f7fb00d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.518 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.515+0000 7f7fb6bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb00cff90 con 0x7f7fb006b360 2023-07-06T03:44:45.518 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.515+0000 7f7fb6bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb00d0780 con 0x7f7fb015c610 2023-07-06T03:44:45.518 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.515+0000 7f7fb6bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb015c9e0 con 0x7f7fb00d2210 2023-07-06T03:44:45.518 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.516+0000 7f7fb519c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb00d2210 0x7f7fb00d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.519 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.516+0000 7f7fb499b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7fb015c610 0x7f7fb00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.519 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.516+0000 7f7fb519c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb00d2210 0x7f7fb00d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:55976/0 (socket says 172.21.15.146:55976) 2023-07-06T03:44:45.519 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.516+0000 7f7fb499b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7fb015c610 0x7f7fb00b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50678/0 (socket says 172.21.15.146:50678) 2023-07-06T03:44:45.519 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.516+0000 7f7fb519c700 1 -- 172.21.15.146:0/2453409289 learned_addr learned my addr 172.21.15.146:0/2453409289 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:45.520 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.516+0000 7f7fb519c700 1 -- 172.21.15.146:0/2453409289 >> v2:172.21.15.146:3300/0 conn(0x7f7fb015c610 msgr2=0x7f7fb00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:45.520 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.516+0000 7f7fb519c700 1 --2- 172.21.15.146:0/2453409289 >> v2:172.21.15.146:3300/0 conn(0x7f7fb015c610 0x7f7fb00b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.521 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.516+0000 7f7fb519c700 1 -- 172.21.15.146:0/2453409289 >> v1:172.21.15.97:6789/0 conn(0x7f7fb006b360 legacy=0x7f7fb00b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.521 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.516+0000 7f7fb519c700 1 -- 172.21.15.146:0/2453409289 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb00b5650 con 0x7f7fb00d2210 2023-07-06T03:44:45.521 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.516+0000 7f7fb519c700 1 --2- 172.21.15.146:0/2453409289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb00d2210 0x7f7fb00d1310 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009e50 tx=0x7f7fa800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2795c81716b0478d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:45.521 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.516+0000 7f7faf7fe700 1 -- 172.21.15.146:0/2453409289 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa800e070 con 0x7f7fb00d2210 2023-07-06T03:44:45.521 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.517+0000 7f7faf7fe700 1 -- 172.21.15.146:0/2453409289 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa800b910 con 0x7f7fb00d2210 2023-07-06T03:44:45.521 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.517+0000 7f7faf7fe700 1 -- 172.21.15.146:0/2453409289 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa800bcc0 con 0x7f7fb00d2210 2023-07-06T03:44:45.521 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.517+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2453409289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb00d2210 msgr2=0x7f7fb00d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.521 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.517+0000 7f7fb6bff700 1 --2- 172.21.15.146:0/2453409289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb00d2210 0x7f7fb00d1310 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009e50 tx=0x7f7fa800b330 comp rx=0 tx=0).stop 2023-07-06T03:44:45.522 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.517+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2453409289 shutdown_connections 2023-07-06T03:44:45.522 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.517+0000 7f7fb6bff700 1 --2- 172.21.15.146:0/2453409289 >> v2:172.21.15.146:3300/0 conn(0x7f7fb015c610 0x7f7fb00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.522 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.517+0000 7f7fb6bff700 1 --2- 172.21.15.146:0/2453409289 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb00d2210 0x7f7fb00d1310 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.522 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.517+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2453409289 >> 172.21.15.146:0/2453409289 conn(0x7f7fb006a6c0 msgr2=0x7f7fb006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:45.522 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.517+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2453409289 shutdown_connections 2023-07-06T03:44:45.522 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.517+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2453409289 wait complete. 2023-07-06T03:44:45.524 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.519+0000 7f7fb6bff700 1 Processor -- start 2023-07-06T03:44:45.524 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.519+0000 7f7fb6bff700 1 -- start start 2023-07-06T03:44:45.524 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.520+0000 7f7fb6bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb006b360 0x7f7fb00c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.520+0000 7f7fb6bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fb00d2210 0x7f7fb00b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.520+0000 7f7fb6bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb015c610 0x7f7fb00b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.520+0000 7f7fb6bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01f4f50 con 0x7f7fb006b360 2023-07-06T03:44:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.520+0000 7f7fb6bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01f51a0 con 0x7f7fb00d2210 2023-07-06T03:44:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.520+0000 7f7fb6bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01f5480 con 0x7f7fb015c610 2023-07-06T03:44:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.520+0000 7f7faffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fb00d2210 0x7f7fb00b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.520+0000 7f7fb519c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb015c610 0x7f7fb00b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fb499b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb006b360 0x7f7fb00c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.526 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fb519c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb015c610 0x7f7fb00b6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:55980/0 (socket says 172.21.15.146:55980) 2023-07-06T03:44:45.526 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fb519c700 1 -- 172.21.15.146:0/2199702781 learned_addr learned my addr 172.21.15.146:0/2199702781 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:45.526 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7faffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fb00d2210 0x7f7fb00b67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50690/0 (socket says 172.21.15.146:50690) 2023-07-06T03:44:45.526 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fb519c700 1 -- 172.21.15.146:0/2199702781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fb00d2210 msgr2=0x7f7fb00b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.526 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fb519c700 1 --2- 172.21.15.146:0/2199702781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fb00d2210 0x7f7fb00b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.526 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fb519c700 1 -- 172.21.15.146:0/2199702781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb006b360 msgr2=0x7f7fb00c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.526 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fb519c700 1 --2- 172.21.15.146:0/2199702781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb006b360 0x7f7fb00c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.527 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fb519c700 1 -- 172.21.15.146:0/2199702781 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8009b30 con 0x7f7fb015c610 2023-07-06T03:44:45.527 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fb519c700 1 --2- 172.21.15.146:0/2199702781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb015c610 0x7f7fb00b6d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00d2c10 tx=0x7f7fa8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:45.527 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fad7fa700 1 -- 172.21.15.146:0/2199702781 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8004800 con 0x7f7fb015c610 2023-07-06T03:44:45.527 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fad7fa700 1 -- 172.21.15.146:0/2199702781 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa8004960 con 0x7f7fb015c610 2023-07-06T03:44:45.527 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fad7fa700 1 -- 172.21.15.146:0/2199702781 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa801c680 con 0x7f7fb015c610 2023-07-06T03:44:45.527 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.521+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2199702781 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb01f5760 con 0x7f7fb015c610 2023-07-06T03:44:45.527 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.522+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2199702781 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb01f5d20 con 0x7f7fb015c610 2023-07-06T03:44:45.527 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.522+0000 7f7fad7fa700 1 -- 172.21.15.146:0/2199702781 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7fa801cae0 con 0x7f7fb015c610 2023-07-06T03:44:45.528 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.522+0000 7f7fad7fa700 1 -- 172.21.15.146:0/2199702781 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f7fa801cd90 con 0x7f7fb015c610 2023-07-06T03:44:45.528 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.523+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2199702781 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb004e340 con 0x7f7fb015c610 2023-07-06T03:44:45.528 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.525+0000 7f7fad7fa700 1 -- 172.21.15.146:0/2199702781 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f7fa8020070 con 0x7f7fb015c610 2023-07-06T03:44:45.540 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:45.537+0000 7f51b8d48200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T03:44:45.615 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.613+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2199702781 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "236bb1dc-a348-413d-aac2-91bfceb3d132", "id": 2} v 0) v1 -- 0x7f7fb01f5eb0 con 0x7f7fb015c610 2023-07-06T03:44:45.623 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.620+0000 7f7fad7fa700 1 -- 172.21.15.146:0/2199702781 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "236bb1dc-a348-413d-aac2-91bfceb3d132", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7fa8034020 con 0x7f7fb015c610 2023-07-06T03:44:45.623 INFO:teuthology.orchestra.run.smithi146.stdout:2 2023-07-06T03:44:45.624 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.621+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2199702781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb015c610 msgr2=0x7f7fb00b6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.624 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.621+0000 7f7fb6bff700 1 --2- 172.21.15.146:0/2199702781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb015c610 0x7f7fb00b6d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00d2c10 tx=0x7f7fa8004ad0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.624 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.621+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2199702781 shutdown_connections 2023-07-06T03:44:45.624 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.621+0000 7f7fb6bff700 1 --2- 172.21.15.146:0/2199702781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fb015c610 0x7f7fb00b6d30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.624 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.621+0000 7f7fb6bff700 1 --2- 172.21.15.146:0/2199702781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb006b360 0x7f7fb00c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.625 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.621+0000 7f7fb6bff700 1 --2- 172.21.15.146:0/2199702781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fb00d2210 0x7f7fb00b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.625 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.621+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2199702781 >> 172.21.15.146:0/2199702781 conn(0x7f7fb006a6c0 msgr2=0x7f7fb00b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:45.625 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.622+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2199702781 shutdown_connections 2023-07-06T03:44:45.625 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.622+0000 7f7fb6bff700 1 -- 172.21.15.146:0/2199702781 wait complete. 2023-07-06T03:44:45.641 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new d6f4576e-fddf-480f-8a3d-535999a6363d 3 2023-07-06T03:44:45.835 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.832+0000 7f5c3cbff700 1 Processor -- start 2023-07-06T03:44:45.836 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.832+0000 7f5c3cbff700 1 -- start start 2023-07-06T03:44:45.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.833+0000 7f5c3cbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c3815c610 0x7f5c380b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.837 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c3cbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5c380d2210 0x7f5c380d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c3cbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c380cff90 con 0x7f5c3806b360 2023-07-06T03:44:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c3cbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c380d0780 con 0x7f5c380d2210 2023-07-06T03:44:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c3cbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c3815c9e0 con 0x7f5c3815c610 2023-07-06T03:44:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c3659c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c3815c610 0x7f5c380b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c3659c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c3815c610 0x7f5c380b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50704/0 (socket says 172.21.15.146:50704) 2023-07-06T03:44:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c2ffff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5c3806b360 0x7f5c380b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:46312/0 (socket says 172.21.15.146:46312) 2023-07-06T03:44:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c3659c700 1 -- 172.21.15.146:0/491259637 learned_addr learned my addr 172.21.15.146:0/491259637 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c36d9d700 1 --2- 172.21.15.146:0/491259637 >> v2:172.21.15.97:3300/0 conn(0x7f5c380d2210 0x7f5c380d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.839 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c3659c700 1 -- 172.21.15.146:0/491259637 >> v2:172.21.15.97:3300/0 conn(0x7f5c380d2210 msgr2=0x7f5c380d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:45.839 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c3659c700 1 --2- 172.21.15.146:0/491259637 >> v2:172.21.15.97:3300/0 conn(0x7f5c380d2210 0x7f5c380d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.839 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c3659c700 1 -- 172.21.15.146:0/491259637 >> v1:172.21.15.146:6789/0 conn(0x7f5c3806b360 legacy=0x7f5c380b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.839 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c35d9b700 1 -- 172.21.15.146:0/491259637 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890692962 0 0) 0x7f5c380cff90 con 0x7f5c3806b360 2023-07-06T03:44:45.839 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.834+0000 7f5c3659c700 1 -- 172.21.15.146:0/491259637 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380b5650 con 0x7f5c3815c610 2023-07-06T03:44:45.839 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.835+0000 7f5c3659c700 1 --2- 172.21.15.146:0/491259637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3815c610 0x7f5c380b47b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5c3808b000 tx=0x7f5c28009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340f62d76c07917e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:45.839 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.835+0000 7f5c35d9b700 1 -- 172.21.15.146:0/491259637 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28011830 con 0x7f5c3815c610 2023-07-06T03:44:45.840 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.835+0000 7f5c35d9b700 1 -- 172.21.15.146:0/491259637 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c28011990 con 0x7f5c3815c610 2023-07-06T03:44:45.840 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.835+0000 7f5c35d9b700 1 -- 172.21.15.146:0/491259637 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28011d20 con 0x7f5c3815c610 2023-07-06T03:44:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.835+0000 7f5c3cbff700 1 -- 172.21.15.146:0/491259637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3815c610 msgr2=0x7f5c380b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.835+0000 7f5c3cbff700 1 --2- 172.21.15.146:0/491259637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3815c610 0x7f5c380b47b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5c3808b000 tx=0x7f5c28009bd0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.842 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.836+0000 7f5c3cbff700 1 -- 172.21.15.146:0/491259637 shutdown_connections 2023-07-06T03:44:45.842 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.836+0000 7f5c3cbff700 1 --2- 172.21.15.146:0/491259637 >> v2:172.21.15.97:3300/0 conn(0x7f5c380d2210 0x7f5c380d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.842 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.836+0000 7f5c3cbff700 1 --2- 172.21.15.146:0/491259637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3815c610 0x7f5c380b47b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.842 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.836+0000 7f5c3cbff700 1 -- 172.21.15.146:0/491259637 >> 172.21.15.146:0/491259637 conn(0x7f5c3806a6c0 msgr2=0x7f5c3806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:45.842 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.836+0000 7f5c3cbff700 1 -- 172.21.15.146:0/491259637 shutdown_connections 2023-07-06T03:44:45.842 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.836+0000 7f5c3cbff700 1 -- 172.21.15.146:0/491259637 wait complete. 2023-07-06T03:44:45.842 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.837+0000 7f5c3cbff700 1 Processor -- start 2023-07-06T03:44:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.838+0000 7f5c3cbff700 1 -- start start 2023-07-06T03:44:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.838+0000 7f5c3cbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3806b360 0x7f5c380cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.838+0000 7f5c3cbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c380d2210 0x7f5c380cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.838+0000 7f5c3cbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c3815c610 0x7f5c3815cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.838+0000 7f5c3cbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380cb810 con 0x7f5c3815c610 2023-07-06T03:44:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.838+0000 7f5c3cbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380cbaf0 con 0x7f5c3806b360 2023-07-06T03:44:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.838+0000 7f5c3cbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380cbdd0 con 0x7f5c380d2210 2023-07-06T03:44:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.839+0000 7f5c2ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c380d2210 0x7f5c380cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.839+0000 7f5c3659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3806b360 0x7f5c380cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.839+0000 7f5c3659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3806b360 0x7f5c380cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50720/0 (socket says 172.21.15.146:50720) 2023-07-06T03:44:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.839+0000 7f5c3659c700 1 -- 172.21.15.146:0/1017758825 learned_addr learned my addr 172.21.15.146:0/1017758825 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.839+0000 7f5c36d9d700 1 --2- 172.21.15.146:0/1017758825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c3815c610 0x7f5c3815cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.839+0000 7f5c3659c700 1 -- 172.21.15.146:0/1017758825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c380d2210 msgr2=0x7f5c380cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.839+0000 7f5c3659c700 1 --2- 172.21.15.146:0/1017758825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c380d2210 0x7f5c380cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.839+0000 7f5c3659c700 1 -- 172.21.15.146:0/1017758825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c3815c610 msgr2=0x7f5c3815cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.845 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.839+0000 7f5c3659c700 1 --2- 172.21.15.146:0/1017758825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c3815c610 0x7f5c3815cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.845 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.839+0000 7f5c3659c700 1 -- 172.21.15.146:0/1017758825 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c28009550 con 0x7f5c3806b360 2023-07-06T03:44:45.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.839+0000 7f5c3659c700 1 --2- 172.21.15.146:0/1017758825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3806b360 0x7f5c380cb230 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c28000c00 tx=0x7f5c28004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:45.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.840+0000 7f5c2effd700 1 -- 172.21.15.146:0/1017758825 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2801a8f0 con 0x7f5c3806b360 2023-07-06T03:44:45.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.840+0000 7f5c2effd700 1 -- 172.21.15.146:0/1017758825 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2801e070 con 0x7f5c3806b360 2023-07-06T03:44:45.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.840+0000 7f5c3cbff700 1 -- 172.21.15.146:0/1017758825 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c380cc0b0 con 0x7f5c3806b360 2023-07-06T03:44:45.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.840+0000 7f5c2effd700 1 -- 172.21.15.146:0/1017758825 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2801ac50 con 0x7f5c3806b360 2023-07-06T03:44:45.846 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.840+0000 7f5c3cbff700 1 -- 172.21.15.146:0/1017758825 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c381534e0 con 0x7f5c3806b360 2023-07-06T03:44:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.840+0000 7f5c2effd700 1 -- 172.21.15.146:0/1017758825 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5c2802d750 con 0x7f5c3806b360 2023-07-06T03:44:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.840+0000 7f5c2effd700 1 -- 172.21.15.146:0/1017758825 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f5c28024590 con 0x7f5c3806b360 2023-07-06T03:44:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.841+0000 7f5c3cbff700 1 -- 172.21.15.146:0/1017758825 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3804e340 con 0x7f5c3806b360 2023-07-06T03:44:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.843+0000 7f5c2effd700 1 -- 172.21.15.146:0/1017758825 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f5c28020070 con 0x7f5c3806b360 2023-07-06T03:44:45.890 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:45.888+0000 7ff302edf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T03:44:45.932 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.930+0000 7f5c3cbff700 1 -- 172.21.15.146:0/1017758825 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d6f4576e-fddf-480f-8a3d-535999a6363d", "id": 3} v 0) v1 -- 0x7f5c380cd4f0 con 0x7f5c3806b360 2023-07-06T03:44:45.940 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.937+0000 7f5c2effd700 1 -- 172.21.15.146:0/1017758825 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d6f4576e-fddf-480f-8a3d-535999a6363d", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5c28015020 con 0x7f5c3806b360 2023-07-06T03:44:45.940 INFO:teuthology.orchestra.run.smithi146.stdout:3 2023-07-06T03:44:45.941 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.939+0000 7f5c3cbff700 1 -- 172.21.15.146:0/1017758825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3806b360 msgr2=0x7f5c380cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:45.941 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.939+0000 7f5c3cbff700 1 --2- 172.21.15.146:0/1017758825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3806b360 0x7f5c380cb230 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c28000c00 tx=0x7f5c28004750 comp rx=0 tx=0).stop 2023-07-06T03:44:45.942 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.939+0000 7f5c3cbff700 1 -- 172.21.15.146:0/1017758825 shutdown_connections 2023-07-06T03:44:45.942 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.939+0000 7f5c3cbff700 1 --2- 172.21.15.146:0/1017758825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c3815c610 0x7f5c3815cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.942 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.939+0000 7f5c3cbff700 1 --2- 172.21.15.146:0/1017758825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c3806b360 0x7f5c380cb230 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.942 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.939+0000 7f5c3cbff700 1 --2- 172.21.15.146:0/1017758825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c380d2210 0x7f5c380cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:45.943 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.939+0000 7f5c3cbff700 1 -- 172.21.15.146:0/1017758825 >> 172.21.15.146:0/1017758825 conn(0x7f5c3806a6c0 msgr2=0x7f5c380b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:45.943 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.939+0000 7f5c3cbff700 1 -- 172.21.15.146:0/1017758825 shutdown_connections 2023-07-06T03:44:45.944 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:45.939+0000 7f5c3cbff700 1 -- 172.21.15.146:0/1017758825 wait complete. 2023-07-06T03:44:45.960 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new e5571845-c703-4465-9293-e87425028856 4 2023-07-06T03:44:46.012 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:46.010+0000 7f51b8d48200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T03:44:46.153 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.150+0000 7fa60b9ff700 1 Processor -- start 2023-07-06T03:44:46.153 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.151+0000 7fa60b9ff700 1 -- start start 2023-07-06T03:44:46.154 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.152+0000 7fa60b9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa60415c610 0x7fa6040b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.155 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.152+0000 7fa60b9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6040cff90 con 0x7fa6040d2210 2023-07-06T03:44:46.155 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.152+0000 7fa60b9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6040b5650 con 0x7fa60406b360 2023-07-06T03:44:46.155 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.152+0000 7fa60b9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6040d0780 con 0x7fa60415c610 2023-07-06T03:44:46.155 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.152+0000 7fa608f9a700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa60406b360 0x7fa6040b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:46320/0 (socket says 172.21.15.146:46320) 2023-07-06T03:44:46.155 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.152+0000 7fa608f9a700 1 -- 172.21.15.146:0/2636656521 learned_addr learned my addr 172.21.15.146:0/2636656521 (peer_addr_for_me v1:172.21.15.146:0/0) 2023-07-06T03:44:46.155 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.152+0000 7fa60979b700 1 --2- 172.21.15.146:0/2636656521 >> v2:172.21.15.97:3300/0 conn(0x7fa60415c610 0x7fa6040b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.158 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.153+0000 7fa5fbfff700 1 -- 172.21.15.146:0/2636656521 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923940835 0 0) 0x7fa6040b5650 con 0x7fa60406b360 2023-07-06T03:44:46.158 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.153+0000 7fa5fbfff700 1 -- 172.21.15.146:0/2636656521 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5f00035b0 con 0x7fa60406b360 2023-07-06T03:44:46.159 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.153+0000 7fa5fbfff700 1 -- 172.21.15.146:0/2636656521 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932148348 0 0) 0x7fa6040cff90 con 0x7fa6040d2210 2023-07-06T03:44:46.159 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.153+0000 7fa5fbfff700 1 -- 172.21.15.146:0/2636656521 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6040b5650 con 0x7fa6040d2210 2023-07-06T03:44:46.159 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.153+0000 7fa60979b700 1 -- 172.21.15.146:0/2636656521 >> v1:172.21.15.146:6789/0 conn(0x7fa60406b360 legacy=0x7fa6040b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.160 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.153+0000 7fa60979b700 1 -- 172.21.15.146:0/2636656521 >> v1:172.21.15.97:6789/0 conn(0x7fa6040d2210 legacy=0x7fa6040d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.160 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.153+0000 7fa60979b700 1 -- 172.21.15.146:0/2636656521 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040b5400 con 0x7fa60415c610 2023-07-06T03:44:46.160 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.153+0000 7fa60979b700 1 --2- 172.21.15.146:0/2636656521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60415c610 0x7fa6040b47b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa600009840 tx=0x7fa600009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2c26d566a4d691 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:46.160 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.153+0000 7fa5fbfff700 1 -- 172.21.15.146:0/2636656521 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600011830 con 0x7fa60415c610 2023-07-06T03:44:46.161 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.153+0000 7fa5fbfff700 1 -- 172.21.15.146:0/2636656521 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa600011990 con 0x7fa60415c610 2023-07-06T03:44:46.161 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.153+0000 7fa5fbfff700 1 -- 172.21.15.146:0/2636656521 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600011d20 con 0x7fa60415c610 2023-07-06T03:44:46.162 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.154+0000 7fa60b9ff700 1 -- 172.21.15.146:0/2636656521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60415c610 msgr2=0x7fa6040b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.162 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.154+0000 7fa60b9ff700 1 --2- 172.21.15.146:0/2636656521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60415c610 0x7fa6040b47b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa600009840 tx=0x7fa600009b50 comp rx=0 tx=0).stop 2023-07-06T03:44:46.162 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.154+0000 7fa60b9ff700 1 -- 172.21.15.146:0/2636656521 shutdown_connections 2023-07-06T03:44:46.162 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.154+0000 7fa60b9ff700 1 --2- 172.21.15.146:0/2636656521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60415c610 0x7fa6040b47b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.162 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.154+0000 7fa60b9ff700 1 -- 172.21.15.146:0/2636656521 >> 172.21.15.146:0/2636656521 conn(0x7fa60406a6c0 msgr2=0x7fa60406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:46.162 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.154+0000 7fa60b9ff700 1 -- 172.21.15.146:0/2636656521 shutdown_connections 2023-07-06T03:44:46.162 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.154+0000 7fa60b9ff700 1 -- 172.21.15.146:0/2636656521 wait complete. 2023-07-06T03:44:46.163 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.156+0000 7fa60b9ff700 1 Processor -- start 2023-07-06T03:44:46.163 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.156+0000 7fa60b9ff700 1 -- start start 2023-07-06T03:44:46.165 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa60b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa60406b360 0x7fa6040c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.165 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa60b9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6040d2210 0x7fa6041f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.165 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa60b9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60415c610 0x7fa60415f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.165 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa60b9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6041f1ff0 con 0x7fa60415c610 2023-07-06T03:44:46.165 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa60b9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60415e0d0 con 0x7fa6040d2210 2023-07-06T03:44:46.166 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa60b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60415e3b0 con 0x7fa60406b360 2023-07-06T03:44:46.166 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa60979b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa60406b360 0x7fa6040c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.166 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa608f9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6040d2210 0x7fa6041f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.166 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa60979b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa60406b360 0x7fa6040c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:56010/0 (socket says 172.21.15.146:56010) 2023-07-06T03:44:46.166 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa608f9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6040d2210 0x7fa6041f1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50722/0 (socket says 172.21.15.146:50722) 2023-07-06T03:44:46.166 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa60979b700 1 -- 172.21.15.146:0/3294687587 learned_addr learned my addr 172.21.15.146:0/3294687587 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:46.166 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.157+0000 7fa609f9c700 1 --2- 172.21.15.146:0/3294687587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60415c610 0x7fa60415f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.167 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa60979b700 1 -- 172.21.15.146:0/3294687587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6040d2210 msgr2=0x7fa6041f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.167 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa60979b700 1 --2- 172.21.15.146:0/3294687587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6040d2210 0x7fa6041f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.167 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa60979b700 1 -- 172.21.15.146:0/3294687587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60415c610 msgr2=0x7fa60415f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.167 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa60979b700 1 --2- 172.21.15.146:0/3294687587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60415c610 0x7fa60415f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.167 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa60979b700 1 -- 172.21.15.146:0/3294687587 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6000094a0 con 0x7fa60406b360 2023-07-06T03:44:46.167 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa608f9a700 1 --2- 172.21.15.146:0/3294687587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6040d2210 0x7fa6041f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:44:46.167 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa60979b700 1 --2- 172.21.15.146:0/3294687587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa60406b360 0x7fa6040c9e00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6000090b0 tx=0x7fa600004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:46.167 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa5f9ffb700 1 -- 172.21.15.146:0/3294687587 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60001a6e0 con 0x7fa60406b360 2023-07-06T03:44:46.168 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa5f9ffb700 1 -- 172.21.15.146:0/3294687587 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60001a840 con 0x7fa60406b360 2023-07-06T03:44:46.168 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa60b9ff700 1 -- 172.21.15.146:0/3294687587 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60415e690 con 0x7fa60406b360 2023-07-06T03:44:46.168 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa5f9ffb700 1 -- 172.21.15.146:0/3294687587 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60001abb0 con 0x7fa60406b360 2023-07-06T03:44:46.168 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.158+0000 7fa60b9ff700 1 -- 172.21.15.146:0/3294687587 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60415ec50 con 0x7fa60406b360 2023-07-06T03:44:46.168 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.159+0000 7fa5f9ffb700 1 -- 172.21.15.146:0/3294687587 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa600022070 con 0x7fa60406b360 2023-07-06T03:44:46.168 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.159+0000 7fa5f9ffb700 1 -- 172.21.15.146:0/3294687587 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fa600024c30 con 0x7fa60406b360 2023-07-06T03:44:46.169 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.160+0000 7fa60b9ff700 1 -- 172.21.15.146:0/3294687587 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60404e340 con 0x7fa60406b360 2023-07-06T03:44:46.169 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.162+0000 7fa5f9ffb700 1 -- 172.21.15.146:0/3294687587 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa600029070 con 0x7fa60406b360 2023-07-06T03:44:46.263 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.261+0000 7fa60b9ff700 1 -- 172.21.15.146:0/3294687587 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e5571845-c703-4465-9293-e87425028856", "id": 4} v 0) v1 -- 0x7fa6040cd040 con 0x7fa60406b360 2023-07-06T03:44:46.269 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.266+0000 7fa5f9ffb700 1 -- 172.21.15.146:0/3294687587 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e5571845-c703-4465-9293-e87425028856", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa600015020 con 0x7fa60406b360 2023-07-06T03:44:46.269 INFO:teuthology.orchestra.run.smithi146.stdout:4 2023-07-06T03:44:46.270 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.267+0000 7fa60b9ff700 1 -- 172.21.15.146:0/3294687587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa60406b360 msgr2=0x7fa6040c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.270 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.267+0000 7fa60b9ff700 1 --2- 172.21.15.146:0/3294687587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa60406b360 0x7fa6040c9e00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6000090b0 tx=0x7fa600004750 comp rx=0 tx=0).stop 2023-07-06T03:44:46.270 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.267+0000 7fa60b9ff700 1 -- 172.21.15.146:0/3294687587 shutdown_connections 2023-07-06T03:44:46.270 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.267+0000 7fa60b9ff700 1 --2- 172.21.15.146:0/3294687587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60415c610 0x7fa60415f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.270 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.267+0000 7fa60b9ff700 1 --2- 172.21.15.146:0/3294687587 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa60406b360 0x7fa6040c9e00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.271 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.267+0000 7fa60b9ff700 1 --2- 172.21.15.146:0/3294687587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6040d2210 0x7fa6041f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.271 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.267+0000 7fa60b9ff700 1 -- 172.21.15.146:0/3294687587 >> 172.21.15.146:0/3294687587 conn(0x7fa60406a6c0 msgr2=0x7fa6040b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:46.271 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.268+0000 7fa60b9ff700 1 -- 172.21.15.146:0/3294687587 shutdown_connections 2023-07-06T03:44:46.271 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.268+0000 7fa60b9ff700 1 -- 172.21.15.146:0/3294687587 wait complete. 2023-07-06T03:44:46.287 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new b45633e5-ab66-45fc-8b50-b90c1d5f38c9 5 2023-07-06T03:44:46.479 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.476+0000 7f912dfff700 1 Processor -- start 2023-07-06T03:44:46.479 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.476+0000 7f912dfff700 1 -- start start 2023-07-06T03:44:46.480 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.478+0000 7f912dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912815c610 0x7f91280b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.481 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.478+0000 7f912dfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280cff90 con 0x7f912806b360 2023-07-06T03:44:46.481 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.478+0000 7f912dfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280b5650 con 0x7f91280d2210 2023-07-06T03:44:46.481 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.478+0000 7f912dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91280d0780 con 0x7f912815c610 2023-07-06T03:44:46.481 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.478+0000 7f91277fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912815c610 0x7f91280b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.481 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.478+0000 7f91277fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912815c610 0x7f91280b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:56016/0 (socket says 172.21.15.146:56016) 2023-07-06T03:44:46.482 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.478+0000 7f91277fe700 1 -- 172.21.15.146:0/1941164714 learned_addr learned my addr 172.21.15.146:0/1941164714 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:46.482 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.479+0000 7f91277fe700 1 -- 172.21.15.146:0/1941164714 >> v1:172.21.15.146:6789/0 conn(0x7f91280d2210 legacy=0x7f91280d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.482 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.479+0000 7f91277fe700 1 -- 172.21.15.146:0/1941164714 >> v1:172.21.15.97:6789/0 conn(0x7f912806b360 legacy=0x7f91280b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.482 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.479+0000 7f91277fe700 1 -- 172.21.15.146:0/1941164714 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280b5400 con 0x7f912815c610 2023-07-06T03:44:46.483 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.479+0000 7f91277fe700 1 --2- 172.21.15.146:0/1941164714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912815c610 0x7f91280b47b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9118009920 tx=0x7f9118009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa8a74afc5bf9e01 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:46.483 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.479+0000 7f91267fc700 1 -- 172.21.15.146:0/1941164714 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911800ba60 con 0x7f912815c610 2023-07-06T03:44:46.483 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.479+0000 7f91267fc700 1 -- 172.21.15.146:0/1941164714 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911800bbc0 con 0x7f912815c610 2023-07-06T03:44:46.483 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.479+0000 7f91267fc700 1 -- 172.21.15.146:0/1941164714 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118009e70 con 0x7f912815c610 2023-07-06T03:44:46.484 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.480+0000 7f912dfff700 1 -- 172.21.15.146:0/1941164714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912815c610 msgr2=0x7f91280b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.484 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.480+0000 7f912dfff700 1 --2- 172.21.15.146:0/1941164714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912815c610 0x7f91280b47b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9118009920 tx=0x7f9118009c30 comp rx=0 tx=0).stop 2023-07-06T03:44:46.484 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.480+0000 7f912dfff700 1 -- 172.21.15.146:0/1941164714 shutdown_connections 2023-07-06T03:44:46.484 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.480+0000 7f912dfff700 1 --2- 172.21.15.146:0/1941164714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912815c610 0x7f91280b47b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.484 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.480+0000 7f912dfff700 1 -- 172.21.15.146:0/1941164714 >> 172.21.15.146:0/1941164714 conn(0x7f912806a6c0 msgr2=0x7f912806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:46.484 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.480+0000 7f912dfff700 1 -- 172.21.15.146:0/1941164714 shutdown_connections 2023-07-06T03:44:46.484 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.480+0000 7f912dfff700 1 -- 172.21.15.146:0/1941164714 wait complete. 2023-07-06T03:44:46.485 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.482+0000 7f912dfff700 1 Processor -- start 2023-07-06T03:44:46.485 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.482+0000 7f912dfff700 1 -- start start 2023-07-06T03:44:46.486 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.483+0000 7f912dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912806b360 0x7f9128156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.486 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.483+0000 7f912dfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91280d2210 0x7f91280b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.486 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.483+0000 7f912dfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f912815c610 0x7f91280b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.486 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.483+0000 7f912dfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912815d880 con 0x7f91280d2210 2023-07-06T03:44:46.486 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.483+0000 7f912dfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912815cc00 con 0x7f912815c610 2023-07-06T03:44:46.487 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.483+0000 7f912dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912815cee0 con 0x7f912806b360 2023-07-06T03:44:46.487 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.483+0000 7f91277fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912806b360 0x7f9128156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.487 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.483+0000 7f91277fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912806b360 0x7f9128156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:56024/0 (socket says 172.21.15.146:56024) 2023-07-06T03:44:46.487 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.483+0000 7f91277fe700 1 -- 172.21.15.146:0/1247735768 learned_addr learned my addr 172.21.15.146:0/1247735768 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:46.487 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.483+0000 7f9127fff700 1 --2- 172.21.15.146:0/1247735768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f912815c610 0x7f91280b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.487 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.483+0000 7f9126ffd700 1 --2- 172.21.15.146:0/1247735768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91280d2210 0x7f91280b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.488 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.484+0000 7f91277fe700 1 -- 172.21.15.146:0/1247735768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f912815c610 msgr2=0x7f91280b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.488 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.484+0000 7f91277fe700 1 --2- 172.21.15.146:0/1247735768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f912815c610 0x7f91280b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.489 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.484+0000 7f91277fe700 1 -- 172.21.15.146:0/1247735768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91280d2210 msgr2=0x7f91280b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.489 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.484+0000 7f91277fe700 1 --2- 172.21.15.146:0/1247735768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91280d2210 0x7f91280b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.489 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.484+0000 7f91277fe700 1 -- 172.21.15.146:0/1247735768 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118009550 con 0x7f912806b360 2023-07-06T03:44:46.489 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.484+0000 7f9127fff700 1 --2- 172.21.15.146:0/1247735768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f912815c610 0x7f91280b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:44:46.489 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.484+0000 7f91277fe700 1 --2- 172.21.15.146:0/1247735768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912806b360 0x7f9128156270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9118014040 tx=0x7f9118004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:46.489 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.485+0000 7f910ffff700 1 -- 172.21.15.146:0/1247735768 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118004040 con 0x7f912806b360 2023-07-06T03:44:46.489 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.485+0000 7f910ffff700 1 -- 172.21.15.146:0/1247735768 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91180041c0 con 0x7f912806b360 2023-07-06T03:44:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.485+0000 7f912dfff700 1 -- 172.21.15.146:0/1247735768 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912815d1c0 con 0x7f912806b360 2023-07-06T03:44:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.485+0000 7f910ffff700 1 -- 172.21.15.146:0/1247735768 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118004570 con 0x7f912806b360 2023-07-06T03:44:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.485+0000 7f912dfff700 1 -- 172.21.15.146:0/1247735768 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91281567b0 con 0x7f912806b360 2023-07-06T03:44:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.485+0000 7f910ffff700 1 -- 172.21.15.146:0/1247735768 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f911802a070 con 0x7f912806b360 2023-07-06T03:44:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.485+0000 7f910ffff700 1 -- 172.21.15.146:0/1247735768 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f911802cae0 con 0x7f912806b360 2023-07-06T03:44:46.490 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.486+0000 7f912dfff700 1 -- 172.21.15.146:0/1247735768 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912815d9c0 con 0x7f912806b360 2023-07-06T03:44:46.491 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.488+0000 7f910ffff700 1 -- 172.21.15.146:0/1247735768 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f9118031070 con 0x7f912806b360 2023-07-06T03:44:46.578 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.575+0000 7f912dfff700 1 -- 172.21.15.146:0/1247735768 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b45633e5-ab66-45fc-8b50-b90c1d5f38c9", "id": 5} v 0) v1 -- 0x7f912804e340 con 0x7f912806b360 2023-07-06T03:44:46.584 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.581+0000 7f910ffff700 1 -- 172.21.15.146:0/1247735768 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b45633e5-ab66-45fc-8b50-b90c1d5f38c9", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9118015020 con 0x7f912806b360 2023-07-06T03:44:46.585 INFO:teuthology.orchestra.run.smithi146.stdout:5 2023-07-06T03:44:46.586 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.582+0000 7f912dfff700 1 -- 172.21.15.146:0/1247735768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912806b360 msgr2=0x7f9128156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.586 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.582+0000 7f912dfff700 1 --2- 172.21.15.146:0/1247735768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912806b360 0x7f9128156270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9118014040 tx=0x7f9118004750 comp rx=0 tx=0).stop 2023-07-06T03:44:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.582+0000 7f912dfff700 1 -- 172.21.15.146:0/1247735768 shutdown_connections 2023-07-06T03:44:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.582+0000 7f912dfff700 1 --2- 172.21.15.146:0/1247735768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f912815c610 0x7f91280b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.582+0000 7f912dfff700 1 --2- 172.21.15.146:0/1247735768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f912806b360 0x7f9128156270 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.582+0000 7f912dfff700 1 --2- 172.21.15.146:0/1247735768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91280d2210 0x7f91280b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.582+0000 7f912dfff700 1 -- 172.21.15.146:0/1247735768 >> 172.21.15.146:0/1247735768 conn(0x7f912806a6c0 msgr2=0x7f91280b7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.582+0000 7f912dfff700 1 -- 172.21.15.146:0/1247735768 shutdown_connections 2023-07-06T03:44:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.582+0000 7f912dfff700 1 -- 172.21.15.146:0/1247735768 wait complete. 2023-07-06T03:44:46.602 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new d938c560-a4b1-4d1a-a6a1-422594d32887 6 2023-07-06T03:44:46.696 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:46.695+0000 7ff302edf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T03:44:46.789 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:46.787+0000 7ff302edf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T03:44:46.798 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.796+0000 7f5f335ff700 1 Processor -- start 2023-07-06T03:44:46.798 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.796+0000 7f5f335ff700 1 -- start start 2023-07-06T03:44:46.800 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.797+0000 7f5f335ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5f2c15c610 0x7f5f2c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.800 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.797+0000 7f5f335ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c0d2210 0x7f5f2c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.800 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.797+0000 7f5f335ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c0cff90 con 0x7f5f2c06b360 2023-07-06T03:44:46.800 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.797+0000 7f5f335ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c0d0780 con 0x7f5f2c15c610 2023-07-06T03:44:46.800 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.797+0000 7f5f335ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c15c9e0 con 0x7f5f2c0d2210 2023-07-06T03:44:46.800 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.797+0000 7f5f31b9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c0d2210 0x7f5f2c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.801 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.797+0000 7f5f3139b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5f2c15c610 0x7f5f2c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.801 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.797+0000 7f5f31b9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c0d2210 0x7f5f2c0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:56040/0 (socket says 172.21.15.146:56040) 2023-07-06T03:44:46.801 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.797+0000 7f5f31b9c700 1 -- 172.21.15.146:0/1399351494 learned_addr learned my addr 172.21.15.146:0/1399351494 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:46.802 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.798+0000 7f5f31b9c700 1 -- 172.21.15.146:0/1399351494 >> v2:172.21.15.97:3300/0 conn(0x7f5f2c15c610 msgr2=0x7f5f2c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:46.802 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.798+0000 7f5f31b9c700 1 --2- 172.21.15.146:0/1399351494 >> v2:172.21.15.97:3300/0 conn(0x7f5f2c15c610 0x7f5f2c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.803 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.798+0000 7f5f31b9c700 1 -- 172.21.15.146:0/1399351494 >> v1:172.21.15.97:6789/0 conn(0x7f5f2c06b360 legacy=0x7f5f2c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.803 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.798+0000 7f5f31b9c700 1 -- 172.21.15.146:0/1399351494 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0b5650 con 0x7f5f2c0d2210 2023-07-06T03:44:46.803 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.798+0000 7f5f31b9c700 1 --2- 172.21.15.146:0/1399351494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c0d2210 0x7f5f2c0d1310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009e50 tx=0x7f5f2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b3c2ea5cb45e294 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:46.804 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.798+0000 7f5f23fff700 1 -- 172.21.15.146:0/1399351494 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400e070 con 0x7f5f2c0d2210 2023-07-06T03:44:46.804 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.798+0000 7f5f23fff700 1 -- 172.21.15.146:0/1399351494 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2400b910 con 0x7f5f2c0d2210 2023-07-06T03:44:46.804 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.798+0000 7f5f23fff700 1 -- 172.21.15.146:0/1399351494 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400bcc0 con 0x7f5f2c0d2210 2023-07-06T03:44:46.804 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.799+0000 7f5f335ff700 1 -- 172.21.15.146:0/1399351494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c0d2210 msgr2=0x7f5f2c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.804 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.799+0000 7f5f335ff700 1 --2- 172.21.15.146:0/1399351494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c0d2210 0x7f5f2c0d1310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009e50 tx=0x7f5f2400b330 comp rx=0 tx=0).stop 2023-07-06T03:44:46.805 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.799+0000 7f5f335ff700 1 -- 172.21.15.146:0/1399351494 shutdown_connections 2023-07-06T03:44:46.805 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.799+0000 7f5f335ff700 1 --2- 172.21.15.146:0/1399351494 >> v2:172.21.15.97:3300/0 conn(0x7f5f2c15c610 0x7f5f2c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.805 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.799+0000 7f5f335ff700 1 --2- 172.21.15.146:0/1399351494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c0d2210 0x7f5f2c0d1310 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.805 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.799+0000 7f5f335ff700 1 -- 172.21.15.146:0/1399351494 >> 172.21.15.146:0/1399351494 conn(0x7f5f2c06a6c0 msgr2=0x7f5f2c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:46.805 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.799+0000 7f5f335ff700 1 -- 172.21.15.146:0/1399351494 shutdown_connections 2023-07-06T03:44:46.805 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.799+0000 7f5f335ff700 1 -- 172.21.15.146:0/1399351494 wait complete. 2023-07-06T03:44:46.807 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.801+0000 7f5f335ff700 1 Processor -- start 2023-07-06T03:44:46.807 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.801+0000 7f5f335ff700 1 -- start start 2023-07-06T03:44:46.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.802+0000 7f5f335ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2c06b360 0x7f5f2c0b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.802+0000 7f5f335ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f2c0d2210 0x7f5f2c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.802+0000 7f5f335ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c15c610 0x7f5f2c0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:46.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.802+0000 7f5f335ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c155840 con 0x7f5f2c06b360 2023-07-06T03:44:46.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.802+0000 7f5f335ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c15d5a0 con 0x7f5f2c0d2210 2023-07-06T03:44:46.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.802+0000 7f5f335ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c15d880 con 0x7f5f2c15c610 2023-07-06T03:44:46.808 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.802+0000 7f5f3139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2c06b360 0x7f5f2c0b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.809 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.802+0000 7f5f3139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2c06b360 0x7f5f2c0b6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.146:53200/0 (socket says 172.21.15.146:53200) 2023-07-06T03:44:46.809 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.802+0000 7f5f3139b700 1 -- 172.21.15.146:0/1524594194 learned_addr learned my addr 172.21.15.146:0/1524594194 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:46.809 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.802+0000 7f5f31b9c700 1 --2- 172.21.15.146:0/1524594194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c15c610 0x7f5f2c0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.809 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.802+0000 7f5f30b9a700 1 --2- 172.21.15.146:0/1524594194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f2c0d2210 0x7f5f2c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:46.809 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.803+0000 7f5f3139b700 1 -- 172.21.15.146:0/1524594194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c15c610 msgr2=0x7f5f2c0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.809 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.803+0000 7f5f3139b700 1 --2- 172.21.15.146:0/1524594194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c15c610 0x7f5f2c0cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.809 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.803+0000 7f5f3139b700 1 -- 172.21.15.146:0/1524594194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f2c0d2210 msgr2=0x7f5f2c15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.803+0000 7f5f3139b700 1 --2- 172.21.15.146:0/1524594194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f2c0d2210 0x7f5f2c15cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.803+0000 7f5f3139b700 1 -- 172.21.15.146:0/1524594194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f24009b30 con 0x7f5f2c06b360 2023-07-06T03:44:46.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.803+0000 7f5f31b9c700 1 --2- 172.21.15.146:0/1524594194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c15c610 0x7f5f2c0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:44:46.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.803+0000 7f5f3139b700 1 --2- 172.21.15.146:0/1524594194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2c06b360 0x7f5f2c0b6a60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e870 tx=0x7f5f2800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:46.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.803+0000 7f5f21ffb700 1 -- 172.21.15.146:0/1524594194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28009590 con 0x7f5f2c06b360 2023-07-06T03:44:46.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.803+0000 7f5f21ffb700 1 -- 172.21.15.146:0/1524594194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f28013680 con 0x7f5f2c06b360 2023-07-06T03:44:46.810 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.804+0000 7f5f335ff700 1 -- 172.21.15.146:0/1524594194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c15db60 con 0x7f5f2c06b360 2023-07-06T03:44:46.811 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.804+0000 7f5f21ffb700 1 -- 172.21.15.146:0/1524594194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280139f0 con 0x7f5f2c06b360 2023-07-06T03:44:46.811 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.804+0000 7f5f335ff700 1 -- 172.21.15.146:0/1524594194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c0cc1f0 con 0x7f5f2c06b360 2023-07-06T03:44:46.811 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.804+0000 7f5f21ffb700 1 -- 172.21.15.146:0/1524594194 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5f28005a20 con 0x7f5f2c06b360 2023-07-06T03:44:46.811 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.804+0000 7f5f21ffb700 1 -- 172.21.15.146:0/1524594194 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f5f28005cd0 con 0x7f5f2c06b360 2023-07-06T03:44:46.812 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.805+0000 7f5f335ff700 1 -- 172.21.15.146:0/1524594194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c155d40 con 0x7f5f2c06b360 2023-07-06T03:44:46.812 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.807+0000 7f5f21ffb700 1 -- 172.21.15.146:0/1524594194 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f5f2c155d40 con 0x7f5f2c06b360 2023-07-06T03:44:46.868 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:46.865+0000 7f51b8d48200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T03:44:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.893+0000 7f5f335ff700 1 -- 172.21.15.146:0/1524594194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d938c560-a4b1-4d1a-a6a1-422594d32887", "id": 6} v 0) v1 -- 0x7f5f2c04e340 con 0x7f5f2c06b360 2023-07-06T03:44:46.899 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.896+0000 7f5f21ffb700 1 -- 172.21.15.146:0/1524594194 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d938c560-a4b1-4d1a-a6a1-422594d32887", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f28029020 con 0x7f5f2c06b360 2023-07-06T03:44:46.899 INFO:teuthology.orchestra.run.smithi146.stdout:6 2023-07-06T03:44:46.900 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.898+0000 7f5f335ff700 1 -- 172.21.15.146:0/1524594194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2c06b360 msgr2=0x7f5f2c0b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:46.900 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.898+0000 7f5f335ff700 1 --2- 172.21.15.146:0/1524594194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2c06b360 0x7f5f2c0b6a60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e870 tx=0x7f5f2800eb80 comp rx=0 tx=0).stop 2023-07-06T03:44:46.900 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.898+0000 7f5f335ff700 1 -- 172.21.15.146:0/1524594194 shutdown_connections 2023-07-06T03:44:46.901 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.898+0000 7f5f335ff700 1 --2- 172.21.15.146:0/1524594194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f2c15c610 0x7f5f2c0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.901 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.898+0000 7f5f335ff700 1 --2- 172.21.15.146:0/1524594194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2c06b360 0x7f5f2c0b6a60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.901 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.898+0000 7f5f335ff700 1 --2- 172.21.15.146:0/1524594194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f2c0d2210 0x7f5f2c15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:46.901 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.898+0000 7f5f335ff700 1 -- 172.21.15.146:0/1524594194 >> 172.21.15.146:0/1524594194 conn(0x7f5f2c06a6c0 msgr2=0x7f5f2c0b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:46.901 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.898+0000 7f5f335ff700 1 -- 172.21.15.146:0/1524594194 shutdown_connections 2023-07-06T03:44:46.901 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:46.898+0000 7f5f335ff700 1 -- 172.21.15.146:0/1524594194 wait complete. 2023-07-06T03:44:46.917 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd new 18496b76-0c09-484a-b379-90e9c991a7db 7 2023-07-06T03:44:46.959 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:46.957+0000 7ff302edf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T03:44:46.968 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:46.966+0000 7f51b8d48200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T03:44:47.048 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:47.046+0000 7ff302edf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T03:44:47.110 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.108+0000 7f9a2ddff700 1 Processor -- start 2023-07-06T03:44:47.111 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.108+0000 7f9a2ddff700 1 -- start start 2023-07-06T03:44:47.114 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a2ddff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9a2815c610 0x7f9a280b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:47.115 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a2ddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:47.115 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a2ddff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9a280d2210 0x7f9a280d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:47.115 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a2ddff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a280d0780 con 0x7f9a280d2210 2023-07-06T03:44:47.115 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a2ddff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2815c9e0 con 0x7f9a2815c610 2023-07-06T03:44:47.116 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a2ddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280b5650 con 0x7f9a2806b360 2023-07-06T03:44:47.116 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a277fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9a2815c610 0x7f9a280b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:47.116 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:47.117 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:56056/0 (socket says 172.21.15.146:56056) 2023-07-06T03:44:47.117 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a277fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9a2815c610 0x7f9a280b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50750/0 (socket says 172.21.15.146:50750) 2023-07-06T03:44:47.117 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a26ffd700 1 -- 172.21.15.146:0/907552889 learned_addr learned my addr 172.21.15.146:0/907552889 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:47.117 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.109+0000 7f9a27fff700 1 --2- 172.21.15.146:0/907552889 >> v2:172.21.15.97:3300/0 conn(0x7f9a280d2210 0x7f9a280d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:47.118 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.110+0000 7f9a277fe700 1 -- 172.21.15.146:0/907552889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 msgr2=0x7f9a280b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:47.118 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.110+0000 7f9a277fe700 1 --2- 172.21.15.146:0/907552889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.118 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.110+0000 7f9a277fe700 1 -- 172.21.15.146:0/907552889 >> v2:172.21.15.97:3300/0 conn(0x7f9a280d2210 msgr2=0x7f9a280d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:47.118 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.110+0000 7f9a277fe700 1 --2- 172.21.15.146:0/907552889 >> v2:172.21.15.97:3300/0 conn(0x7f9a280d2210 0x7f9a280d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.119 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.110+0000 7f9a277fe700 1 -- 172.21.15.146:0/907552889 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a280d01c0 con 0x7f9a2815c610 2023-07-06T03:44:47.119 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.110+0000 7f9a26ffd700 1 --2- 172.21.15.146:0/907552889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:44:47.119 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.110+0000 7f9a277fe700 1 --2- 172.21.15.146:0/907552889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2815c610 0x7f9a280b47b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9a2808b000 tx=0x7f9a18009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f73d1fedbf29fd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:47.120 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.110+0000 7f9a267fc700 1 -- 172.21.15.146:0/907552889 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18011830 con 0x7f9a2815c610 2023-07-06T03:44:47.120 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.110+0000 7f9a267fc700 1 -- 172.21.15.146:0/907552889 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a18011990 con 0x7f9a2815c610 2023-07-06T03:44:47.120 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.110+0000 7f9a267fc700 1 -- 172.21.15.146:0/907552889 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18011d20 con 0x7f9a2815c610 2023-07-06T03:44:47.120 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.111+0000 7f9a2ddff700 1 -- 172.21.15.146:0/907552889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2815c610 msgr2=0x7f9a280b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:47.121 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.111+0000 7f9a2ddff700 1 --2- 172.21.15.146:0/907552889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2815c610 0x7f9a280b47b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9a2808b000 tx=0x7f9a18009b20 comp rx=0 tx=0).stop 2023-07-06T03:44:47.121 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.111+0000 7f9a2ddff700 1 -- 172.21.15.146:0/907552889 shutdown_connections 2023-07-06T03:44:47.121 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.111+0000 7f9a2ddff700 1 --2- 172.21.15.146:0/907552889 >> v2:172.21.15.97:3300/0 conn(0x7f9a280d2210 0x7f9a280d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.122 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.111+0000 7f9a2ddff700 1 --2- 172.21.15.146:0/907552889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2815c610 0x7f9a280b47b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.122 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.111+0000 7f9a2ddff700 1 --2- 172.21.15.146:0/907552889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.122 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.111+0000 7f9a2ddff700 1 -- 172.21.15.146:0/907552889 >> 172.21.15.146:0/907552889 conn(0x7f9a2806a6c0 msgr2=0x7f9a2806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:47.122 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.111+0000 7f9a2ddff700 1 -- 172.21.15.146:0/907552889 shutdown_connections 2023-07-06T03:44:47.123 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.111+0000 7f9a2ddff700 1 -- 172.21.15.146:0/907552889 wait complete. 2023-07-06T03:44:47.127 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.113+0000 7f9a2ddff700 1 Processor -- start 2023-07-06T03:44:47.127 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.113+0000 7f9a2ddff700 1 -- start start 2023-07-06T03:44:47.127 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.114+0000 7f9a2ddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:47.128 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.114+0000 7f9a2ddff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a280d2210 0x7f9a280cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:47.128 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.114+0000 7f9a2ddff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2815c610 0x7f9a280cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:47.128 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.114+0000 7f9a2ddff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280cc220 con 0x7f9a280d2210 2023-07-06T03:44:47.129 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.114+0000 7f9a2ddff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280cb390 con 0x7f9a2815c610 2023-07-06T03:44:47.129 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.114+0000 7f9a2ddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280ccbf0 con 0x7f9a2806b360 2023-07-06T03:44:47.129 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.114+0000 7f9a277fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:47.129 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.114+0000 7f9a27fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2815c610 0x7f9a280cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:47.130 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.114+0000 7f9a277fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.146:56058/0 (socket says 172.21.15.146:56058) 2023-07-06T03:44:47.130 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.114+0000 7f9a277fe700 1 -- 172.21.15.146:0/1220299629 learned_addr learned my addr 172.21.15.146:0/1220299629 (peer_addr_for_me v2:172.21.15.146:0/0) 2023-07-06T03:44:47.130 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.115+0000 7f9a26ffd700 1 --2- 172.21.15.146:0/1220299629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a280d2210 0x7f9a280cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:47.130 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.115+0000 7f9a277fe700 1 -- 172.21.15.146:0/1220299629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2815c610 msgr2=0x7f9a280cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:47.130 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.115+0000 7f9a277fe700 1 --2- 172.21.15.146:0/1220299629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2815c610 0x7f9a280cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.130 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.115+0000 7f9a277fe700 1 -- 172.21.15.146:0/1220299629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a280d2210 msgr2=0x7f9a280cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:47.130 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.115+0000 7f9a277fe700 1 --2- 172.21.15.146:0/1220299629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a280d2210 0x7f9a280cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.131 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.115+0000 7f9a277fe700 1 -- 172.21.15.146:0/1220299629 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c009550 con 0x7f9a2806b360 2023-07-06T03:44:47.131 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.115+0000 7f9a277fe700 1 --2- 172.21.15.146:0/1220299629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280c68d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9a180090b0 tx=0x7f9a18004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:47.131 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.115+0000 7f9a0ffff700 1 -- 172.21.15.146:0/1220299629 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18004450 con 0x7f9a2806b360 2023-07-06T03:44:47.131 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.115+0000 7f9a0ffff700 1 -- 172.21.15.146:0/1220299629 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1801e070 con 0x7f9a2806b360 2023-07-06T03:44:47.131 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.115+0000 7f9a2ddff700 1 -- 172.21.15.146:0/1220299629 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a180094a0 con 0x7f9a2806b360 2023-07-06T03:44:47.131 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.115+0000 7f9a0ffff700 1 -- 172.21.15.146:0/1220299629 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a180038c0 con 0x7f9a2806b360 2023-07-06T03:44:47.131 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.116+0000 7f9a2ddff700 1 -- 172.21.15.146:0/1220299629 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a280cb8f0 con 0x7f9a2806b360 2023-07-06T03:44:47.131 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.116+0000 7f9a0ffff700 1 -- 172.21.15.146:0/1220299629 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9a18003d20 con 0x7f9a2806b360 2023-07-06T03:44:47.132 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.116+0000 7f9a0ffff700 1 -- 172.21.15.146:0/1220299629 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f9a1801aba0 con 0x7f9a2806b360 2023-07-06T03:44:47.132 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.117+0000 7f9a2ddff700 1 -- 172.21.15.146:0/1220299629 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a280cc500 con 0x7f9a2806b360 2023-07-06T03:44:47.132 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.119+0000 7f9a0ffff700 1 -- 172.21.15.146:0/1220299629 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f9a18032060 con 0x7f9a2806b360 2023-07-06T03:44:47.145 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:47.143+0000 7f51b8d48200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T03:44:47.208 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.205+0000 7f9a2ddff700 1 -- 172.21.15.146:0/1220299629 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd new", "uuid": "18496b76-0c09-484a-b379-90e9c991a7db", "id": 7} v 0) v1 -- 0x7f9a280b5b80 con 0x7f9a2806b360 2023-07-06T03:44:47.214 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.212+0000 7f9a0ffff700 1 -- 172.21.15.146:0/1220299629 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "18496b76-0c09-484a-b379-90e9c991a7db", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9a18020070 con 0x7f9a2806b360 2023-07-06T03:44:47.215 INFO:teuthology.orchestra.run.smithi146.stdout:7 2023-07-06T03:44:47.216 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.213+0000 7f9a2ddff700 1 -- 172.21.15.146:0/1220299629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 msgr2=0x7f9a280c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:47.216 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.213+0000 7f9a2ddff700 1 --2- 172.21.15.146:0/1220299629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280c68d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9a180090b0 tx=0x7f9a18004750 comp rx=0 tx=0).stop 2023-07-06T03:44:47.216 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.213+0000 7f9a2ddff700 1 -- 172.21.15.146:0/1220299629 shutdown_connections 2023-07-06T03:44:47.216 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.213+0000 7f9a2ddff700 1 --2- 172.21.15.146:0/1220299629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a2815c610 0x7f9a280cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.216 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.213+0000 7f9a2ddff700 1 --2- 172.21.15.146:0/1220299629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a2806b360 0x7f9a280c68d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.217 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.213+0000 7f9a2ddff700 1 --2- 172.21.15.146:0/1220299629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a280d2210 0x7f9a280cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.217 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.213+0000 7f9a2ddff700 1 -- 172.21.15.146:0/1220299629 >> 172.21.15.146:0/1220299629 conn(0x7f9a2806a6c0 msgr2=0x7f9a280b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:47.217 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.213+0000 7f9a2ddff700 1 -- 172.21.15.146:0/1220299629 shutdown_connections 2023-07-06T03:44:47.217 INFO:teuthology.orchestra.run.smithi146.stderr:2023-07-06T03:44:47.213+0000 7f9a2ddff700 1 -- 172.21.15.146:0/1220299629 wait complete. 2023-07-06T03:44:47.233 INFO:tasks.ceph.osd.0:Restarting daemon 2023-07-06T03:44:47.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-07-06T03:44:47.238 INFO:tasks.ceph.osd.0:Started 2023-07-06T03:44:47.238 INFO:tasks.ceph.osd.1:Restarting daemon 2023-07-06T03:44:47.238 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-07-06T03:44:47.239 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:47.236+0000 7f51b8d48200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T03:44:47.243 INFO:tasks.ceph.osd.1:Started 2023-07-06T03:44:47.244 INFO:tasks.ceph.osd.2:Restarting daemon 2023-07-06T03:44:47.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-07-06T03:44:47.249 INFO:tasks.ceph.osd.2:Started 2023-07-06T03:44:47.250 INFO:tasks.ceph.osd.3:Restarting daemon 2023-07-06T03:44:47.250 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-07-06T03:44:47.255 INFO:tasks.ceph.osd.3:Started 2023-07-06T03:44:47.255 INFO:tasks.ceph.osd.4:Restarting daemon 2023-07-06T03:44:47.256 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-07-06T03:44:47.260 INFO:tasks.ceph.osd.4:Started 2023-07-06T03:44:47.260 INFO:tasks.ceph.osd.5:Restarting daemon 2023-07-06T03:44:47.260 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-07-06T03:44:47.264 INFO:tasks.ceph.osd.5:Started 2023-07-06T03:44:47.264 INFO:tasks.ceph.osd.6:Restarting daemon 2023-07-06T03:44:47.265 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-07-06T03:44:47.269 INFO:tasks.ceph.osd.6:Started 2023-07-06T03:44:47.269 INFO:tasks.ceph.osd.7:Restarting daemon 2023-07-06T03:44:47.269 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-07-06T03:44:47.273 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:47.271+0000 7ff302edf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T03:44:47.274 INFO:tasks.ceph.osd.7:Started 2023-07-06T03:44:47.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:44:47.433 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:47.430+0000 7ff302edf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T03:44:47.506 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:47.504+0000 7f51b8d48200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T03:44:47.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.525+0000 7f4c22bff700 1 Processor -- start 2023-07-06T03:44:47.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.525+0000 7f4c22bff700 1 -- start start 2023-07-06T03:44:47.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.526+0000 7f4c22bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c1c0cd390 0x7f4c1c160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:47.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.526+0000 7f4c22bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4c1c160dd0 0x7f4c1c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:47.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.526+0000 7f4c22bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1c0cdae0 con 0x7f4c1c15c520 2023-07-06T03:44:47.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.526+0000 7f4c22bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c0b6640 con 0x7f4c1c160dd0 2023-07-06T03:44:47.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.526+0000 7f4c22bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0b6920 con 0x7f4c1c0cd390 2023-07-06T03:44:47.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.526+0000 7f4c2119c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4c1c160dd0 0x7f4c1c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:47.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.526+0000 7f4c1bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c1c0cd390 0x7f4c1c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:47.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.526+0000 7f4c2119c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4c1c160dd0 0x7f4c1c0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:44596/0 (socket says 172.21.15.97:44596) 2023-07-06T03:44:47.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.526+0000 7f4c2119c700 1 -- 172.21.15.97:0/1724361766 learned_addr learned my addr 172.21.15.97:0/1724361766 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:44:47.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.526+0000 7f4c2099b700 1 --1- 172.21.15.97:0/1724361766 >> v1:172.21.15.146:6789/0 conn(0x7f4c1c15c520 0x7f4c1c160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:55646/0 (socket says 172.21.15.97:55646) 2023-07-06T03:44:47.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.527+0000 7f4c2119c700 1 -- 172.21.15.97:0/1724361766 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c1c0cd390 msgr2=0x7f4c1c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:47.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.527+0000 7f4c2119c700 1 --2- 172.21.15.97:0/1724361766 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c1c0cd390 0x7f4c1c160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.527+0000 7f4c2119c700 1 -- 172.21.15.97:0/1724361766 >> v1:172.21.15.146:6789/0 conn(0x7f4c1c15c520 legacy=0x7f4c1c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:47.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.527+0000 7f4c2119c700 1 -- 172.21.15.97:0/1724361766 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c0b6c00 con 0x7f4c1c160dd0 2023-07-06T03:44:47.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.527+0000 7f4c2119c700 1 --2- 172.21.15.97:0/1724361766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c1c160dd0 0x7f4c1c0b7400 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009e50 tx=0x7f4c1400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d917dde2f774bbca server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:47.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.527+0000 7f4c1b7fe700 1 -- 172.21.15.97:0/1724361766 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400e070 con 0x7f4c1c160dd0 2023-07-06T03:44:47.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.527+0000 7f4c1b7fe700 1 -- 172.21.15.97:0/1724361766 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1400b910 con 0x7f4c1c160dd0 2023-07-06T03:44:47.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.527+0000 7f4c1b7fe700 1 -- 172.21.15.97:0/1724361766 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400bcc0 con 0x7f4c1c160dd0 2023-07-06T03:44:47.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.528+0000 7f4c22bff700 1 -- 172.21.15.97:0/1724361766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c1c160dd0 msgr2=0x7f4c1c0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:47.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.528+0000 7f4c22bff700 1 --2- 172.21.15.97:0/1724361766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c1c160dd0 0x7f4c1c0b7400 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009e50 tx=0x7f4c1400b330 comp rx=0 tx=0).stop 2023-07-06T03:44:47.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.528+0000 7f4c22bff700 1 -- 172.21.15.97:0/1724361766 shutdown_connections 2023-07-06T03:44:47.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.528+0000 7f4c22bff700 1 --2- 172.21.15.97:0/1724361766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c1c160dd0 0x7f4c1c0b7400 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.528+0000 7f4c22bff700 1 --2- 172.21.15.97:0/1724361766 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c1c0cd390 0x7f4c1c160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.528+0000 7f4c22bff700 1 -- 172.21.15.97:0/1724361766 >> 172.21.15.97:0/1724361766 conn(0x7f4c1c071790 msgr2=0x7f4c1c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:47.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.528+0000 7f4c22bff700 1 -- 172.21.15.97:0/1724361766 shutdown_connections 2023-07-06T03:44:47.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.528+0000 7f4c22bff700 1 -- 172.21.15.97:0/1724361766 wait complete. 2023-07-06T03:44:47.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.530+0000 7f4c22bff700 1 Processor -- start 2023-07-06T03:44:47.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.530+0000 7f4c22bff700 1 -- start start 2023-07-06T03:44:47.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c22bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c1c0cd390 0x7f4c1c0d34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:47.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c22bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c1c15c520 0x7f4c1c159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:47.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c22bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1c160dd0 0x7f4c1c15a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c22bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0d3130 con 0x7f4c1c160dd0 2023-07-06T03:44:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c22bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c15b2c0 con 0x7f4c1c15c520 2023-07-06T03:44:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c22bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c15a6b0 con 0x7f4c1c0cd390 2023-07-06T03:44:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c2119c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1c160dd0 0x7f4c1c15a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c1bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c1c15c520 0x7f4c1c159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c2119c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1c160dd0 0x7f4c1c15a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40026/0 (socket says 172.21.15.97:40026) 2023-07-06T03:44:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c2099b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c1c0cd390 0x7f4c1c0d34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c1bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c1c15c520 0x7f4c1c159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:44602/0 (socket says 172.21.15.97:44602) 2023-07-06T03:44:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c2119c700 1 -- 172.21.15.97:0/2198209733 learned_addr learned my addr 172.21.15.97:0/2198209733 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:44:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c2119c700 1 -- 172.21.15.97:0/2198209733 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c1c0cd390 msgr2=0x7f4c1c0d34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c2119c700 1 --2- 172.21.15.97:0/2198209733 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c1c0cd390 0x7f4c1c0d34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c2119c700 1 -- 172.21.15.97:0/2198209733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c1c15c520 msgr2=0x7f4c1c159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c2119c700 1 --2- 172.21.15.97:0/2198209733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c1c15c520 0x7f4c1c159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.531+0000 7f4c2119c700 1 -- 172.21.15.97:0/2198209733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c14009b30 con 0x7f4c1c160dd0 2023-07-06T03:44:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.532+0000 7f4c2119c700 1 --2- 172.21.15.97:0/2198209733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1c160dd0 0x7f4c1c15a050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c06c240 tx=0x7f4c14004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.532+0000 7f4c197fa700 1 -- 172.21.15.97:0/2198209733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14004800 con 0x7f4c1c160dd0 2023-07-06T03:44:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.532+0000 7f4c197fa700 1 -- 172.21.15.97:0/2198209733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c14004960 con 0x7f4c1c160dd0 2023-07-06T03:44:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.532+0000 7f4c197fa700 1 -- 172.21.15.97:0/2198209733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1401c650 con 0x7f4c1c160dd0 2023-07-06T03:44:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.532+0000 7f4c22bff700 1 -- 172.21.15.97:0/2198209733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c15a990 con 0x7f4c1c160dd0 2023-07-06T03:44:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.532+0000 7f4c22bff700 1 -- 172.21.15.97:0/2198209733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c15af50 con 0x7f4c1c160dd0 2023-07-06T03:44:47.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.532+0000 7f4c197fa700 1 -- 172.21.15.97:0/2198209733 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4c1401cab0 con 0x7f4c1c160dd0 2023-07-06T03:44:47.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.533+0000 7f4c197fa700 1 -- 172.21.15.97:0/2198209733 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4c14024da0 con 0x7f4c1c160dd0 2023-07-06T03:44:47.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.533+0000 7f4c22bff700 1 -- 172.21.15.97:0/2198209733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c15b430 con 0x7f4c1c160dd0 2023-07-06T03:44:47.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.536+0000 7f4c197fa700 1 -- 172.21.15.97:0/2198209733 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4c14029060 con 0x7f4c1c160dd0 2023-07-06T03:44:47.543 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:47.541+0000 7ff302edf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T03:44:47.606 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:47.604+0000 7f51b8d48200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T03:44:47.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.607+0000 7f4c22bff700 1 -- 172.21.15.97:0/2198209733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c1c04e340 con 0x7f4c1c160dd0 2023-07-06T03:44:47.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.607+0000 7f4c197fa700 1 -- 172.21.15.97:0/2198209733 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4c1401cd60 con 0x7f4c1c160dd0 2023-07-06T03:44:47.610 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:44:47.610 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:44:47.208174+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:44:47.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.608+0000 7f4c22bff700 1 -- 172.21.15.97:0/2198209733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1c160dd0 msgr2=0x7f4c1c15a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:47.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.608+0000 7f4c22bff700 1 --2- 172.21.15.97:0/2198209733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1c160dd0 0x7f4c1c15a050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c06c240 tx=0x7f4c14004ad0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.608+0000 7f4c22bff700 1 -- 172.21.15.97:0/2198209733 shutdown_connections 2023-07-06T03:44:47.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.608+0000 7f4c22bff700 1 --2- 172.21.15.97:0/2198209733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1c160dd0 0x7f4c1c15a050 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.608+0000 7f4c22bff700 1 --2- 172.21.15.97:0/2198209733 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c1c0cd390 0x7f4c1c0d34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.608+0000 7f4c22bff700 1 --2- 172.21.15.97:0/2198209733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c1c15c520 0x7f4c1c159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:47.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.608+0000 7f4c22bff700 1 -- 172.21.15.97:0/2198209733 >> 172.21.15.97:0/2198209733 conn(0x7f4c1c071790 msgr2=0x7f4c1c0b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:47.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.608+0000 7f4c22bff700 1 -- 172.21.15.97:0/2198209733 shutdown_connections 2023-07-06T03:44:47.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:44:47.608+0000 7f4c22bff700 1 -- 172.21.15.97:0/2198209733 wait complete. 2023-07-06T03:44:47.628 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-07-06T03:44:47.629 INFO:tasks.ceph:Waiting for OSDs to come up 2023-07-06T03:44:47.655 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:47.652+0000 7ff302edf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T03:44:47.704 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:47.702+0000 7f51b8d48200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T03:44:47.806 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:47.804+0000 7f51b8d48200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T03:44:47.881 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:47.878+0000 7ff302edf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T03:44:47.944 INFO:tasks.ceph.osd.0.smithi097.stderr:2023-07-06T03:44:47.941+0000 7fed5a741640 -1 Falling back to public interface 2023-07-06T03:44:47.947 INFO:tasks.ceph.osd.4.smithi146.stderr:2023-07-06T03:44:47.945+0000 7f56afdbc640 -1 Falling back to public interface 2023-07-06T03:44:47.969 INFO:tasks.ceph.osd.5.smithi146.stderr:2023-07-06T03:44:47.967+0000 7fcb70a00640 -1 Falling back to public interface 2023-07-06T03:44:47.970 INFO:tasks.ceph.osd.7.smithi146.stderr:2023-07-06T03:44:47.967+0000 7f959a5fa640 -1 Falling back to public interface 2023-07-06T03:44:47.974 INFO:tasks.ceph.osd.1.smithi097.stderr:2023-07-06T03:44:47.972+0000 7f2f9b551640 -1 Falling back to public interface 2023-07-06T03:44:47.975 INFO:tasks.ceph.osd.3.smithi097.stderr:2023-07-06T03:44:47.972+0000 7ff4c14c6640 -1 Falling back to public interface 2023-07-06T03:44:47.984 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:47.981+0000 7ff302edf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T03:44:47.997 INFO:tasks.ceph.osd.2.smithi097.stderr:2023-07-06T03:44:47.994+0000 7f71fb7dc640 -1 Falling back to public interface 2023-07-06T03:44:48.000 INFO:tasks.ceph.osd.6.smithi146.stderr:2023-07-06T03:44:47.998+0000 7fcb31639640 -1 Falling back to public interface 2023-07-06T03:44:48.043 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:48.041+0000 7f51b8d48200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T03:44:48.145 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:48.143+0000 7f51b8d48200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T03:44:48.342 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:48.339+0000 7ff302edf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T03:44:48.432 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:48.430+0000 7ff302edf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T03:44:48.433 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T03:44:48.505 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:48.503+0000 7f51b8d48200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T03:44:48.598 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:48.595+0000 7f51b8d48200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T03:44:48.609 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.606+0000 7f88795ff700 1 Processor -- start 2023-07-06T03:44:48.609 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.607+0000 7f88795ff700 1 -- start start 2023-07-06T03:44:48.611 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.608+0000 7f88795ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88740cd390 0x7f8874160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:48.611 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.608+0000 7f88795ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88740cdae0 con 0x7f8874160dd0 2023-07-06T03:44:48.611 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.608+0000 7f88795ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88740b6640 con 0x7f887415c520 2023-07-06T03:44:48.611 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.608+0000 7f88795ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88740b6920 con 0x7f88740cd390 2023-07-06T03:44:48.611 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.608+0000 7f88737fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8874160dd0 0x7f88740b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54134/0 (socket says 172.21.15.97:54134) 2023-07-06T03:44:48.611 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.608+0000 7f88737fe700 1 -- 172.21.15.97:0/1761461163 learned_addr learned my addr 172.21.15.97:0/1761461163 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:44:48.612 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.608+0000 7f886bfff700 1 --2- 172.21.15.97:0/1761461163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88740cd390 0x7f8874160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:48.613 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.609+0000 7f88727fc700 1 -- 172.21.15.97:0/1761461163 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952958554 0 0) 0x7f88740cdae0 con 0x7f8874160dd0 2023-07-06T03:44:48.613 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.609+0000 7f88727fc700 1 -- 172.21.15.97:0/1761461163 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88600035b0 con 0x7f8874160dd0 2023-07-06T03:44:48.613 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.609+0000 7f88727fc700 1 -- 172.21.15.97:0/1761461163 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59457717 0 0) 0x7f88740b6640 con 0x7f887415c520 2023-07-06T03:44:48.613 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.609+0000 7f88727fc700 1 -- 172.21.15.97:0/1761461163 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88740cdae0 con 0x7f887415c520 2023-07-06T03:44:48.614 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.609+0000 7f886bfff700 1 -- 172.21.15.97:0/1761461163 >> v1:172.21.15.146:6789/0 conn(0x7f887415c520 legacy=0x7f8874160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:48.614 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.609+0000 7f886bfff700 1 -- 172.21.15.97:0/1761461163 >> v1:172.21.15.97:6789/0 conn(0x7f8874160dd0 legacy=0x7f88740b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:48.614 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.609+0000 7f886bfff700 1 -- 172.21.15.97:0/1761461163 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740b6c00 con 0x7f88740cd390 2023-07-06T03:44:48.614 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.609+0000 7f886bfff700 1 --2- 172.21.15.97:0/1761461163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88740cd390 0x7f8874160890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f885c009870 tx=0x7f885c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a085750eb185ed4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:48.614 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.609+0000 7f88727fc700 1 -- 172.21.15.97:0/1761461163 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c004230 con 0x7f88740cd390 2023-07-06T03:44:48.614 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.609+0000 7f88727fc700 1 -- 172.21.15.97:0/1761461163 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885c004390 con 0x7f88740cd390 2023-07-06T03:44:48.614 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.609+0000 7f88727fc700 1 -- 172.21.15.97:0/1761461163 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c0036a0 con 0x7f88740cd390 2023-07-06T03:44:48.614 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.610+0000 7f88795ff700 1 -- 172.21.15.97:0/1761461163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88740cd390 msgr2=0x7f8874160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:48.615 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.610+0000 7f88795ff700 1 --2- 172.21.15.97:0/1761461163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88740cd390 0x7f8874160890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f885c009870 tx=0x7f885c009b80 comp rx=0 tx=0).stop 2023-07-06T03:44:48.615 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.610+0000 7f88795ff700 1 -- 172.21.15.97:0/1761461163 shutdown_connections 2023-07-06T03:44:48.615 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.610+0000 7f88795ff700 1 --2- 172.21.15.97:0/1761461163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88740cd390 0x7f8874160890 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:48.615 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.610+0000 7f88795ff700 1 -- 172.21.15.97:0/1761461163 >> 172.21.15.97:0/1761461163 conn(0x7f8874071790 msgr2=0x7f8874071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:48.615 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.610+0000 7f88795ff700 1 -- 172.21.15.97:0/1761461163 shutdown_connections 2023-07-06T03:44:48.615 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.610+0000 7f88795ff700 1 -- 172.21.15.97:0/1761461163 wait complete. 2023-07-06T03:44:48.617 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.612+0000 7f88795ff700 1 Processor -- start 2023-07-06T03:44:48.617 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.612+0000 7f88795ff700 1 -- start start 2023-07-06T03:44:48.617 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f88795ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88740cd390 0x7f88740cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:48.618 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f88795ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887415c520 0x7f887415dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:48.618 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f88795ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8874160dd0 0x7f88740b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:48.618 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f88795ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88740d0e50 con 0x7f88740cd390 2023-07-06T03:44:48.618 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f88795ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88740ccb40 con 0x7f887415c520 2023-07-06T03:44:48.618 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f88795ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88740d2140 con 0x7f8874160dd0 2023-07-06T03:44:48.618 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f8872ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88740cd390 0x7f88740cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:48.618 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f886bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887415c520 0x7f887415dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:48.619 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f8872ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88740cd390 0x7f88740cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40030/0 (socket says 172.21.15.97:40030) 2023-07-06T03:44:48.619 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f886bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887415c520 0x7f887415dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:44616/0 (socket says 172.21.15.97:44616) 2023-07-06T03:44:48.619 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f8872ffd700 1 -- 172.21.15.97:0/2295705136 learned_addr learned my addr 172.21.15.97:0/2295705136 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:44:48.619 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.613+0000 7f88737fe700 1 --2- 172.21.15.97:0/2295705136 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8874160dd0 0x7f88740b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:48.619 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f886bfff700 1 -- 172.21.15.97:0/2295705136 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8874160dd0 msgr2=0x7f88740b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:48.619 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f886bfff700 1 --2- 172.21.15.97:0/2295705136 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8874160dd0 0x7f88740b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:48.619 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f886bfff700 1 -- 172.21.15.97:0/2295705136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88740cd390 msgr2=0x7f88740cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:48.620 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f886bfff700 1 --2- 172.21.15.97:0/2295705136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88740cd390 0x7f88740cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:48.620 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f886bfff700 1 -- 172.21.15.97:0/2295705136 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c0094a0 con 0x7f887415c520 2023-07-06T03:44:48.620 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f8872ffd700 1 --2- 172.21.15.97:0/2295705136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88740cd390 0x7f88740cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:44:48.620 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f886bfff700 1 --2- 172.21.15.97:0/2295705136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887415c520 0x7f887415dd60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f885c015000 tx=0x7f885c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:48.620 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f886b7fe700 1 -- 172.21.15.97:0/2295705136 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c003da0 con 0x7f887415c520 2023-07-06T03:44:48.620 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f886b7fe700 1 -- 172.21.15.97:0/2295705136 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885c01e070 con 0x7f887415c520 2023-07-06T03:44:48.620 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f88795ff700 1 -- 172.21.15.97:0/2295705136 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88740d10a0 con 0x7f887415c520 2023-07-06T03:44:48.620 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f886b7fe700 1 -- 172.21.15.97:0/2295705136 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c011870 con 0x7f887415c520 2023-07-06T03:44:48.621 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.614+0000 7f88795ff700 1 -- 172.21.15.97:0/2295705136 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88740d1660 con 0x7f887415c520 2023-07-06T03:44:48.621 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.615+0000 7f886b7fe700 1 -- 172.21.15.97:0/2295705136 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f885c011cd0 con 0x7f887415c520 2023-07-06T03:44:48.621 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.615+0000 7f886b7fe700 1 -- 172.21.15.97:0/2295705136 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f885c01ab00 con 0x7f887415c520 2023-07-06T03:44:48.621 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.615+0000 7f88795ff700 1 -- 172.21.15.97:0/2295705136 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887404e340 con 0x7f887415c520 2023-07-06T03:44:48.621 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.618+0000 7f886b7fe700 1 -- 172.21.15.97:0/2295705136 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f885c020070 con 0x7f887415c520 2023-07-06T03:44:48.693 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.691+0000 7f88795ff700 1 -- 172.21.15.97:0/2295705136 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88740cd800 con 0x7f887415c520 2023-07-06T03:44:48.694 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.692+0000 7f886b7fe700 1 -- 172.21.15.97:0/2295705136 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f885c01ade0 con 0x7f887415c520 2023-07-06T03:44:48.695 INFO:teuthology.misc.health.smithi097.stdout: 2023-07-06T03:44:48.695 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":10,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:44:47.208174+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:44:48.696 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.693+0000 7f88795ff700 1 -- 172.21.15.97:0/2295705136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887415c520 msgr2=0x7f887415dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:48.696 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.693+0000 7f88795ff700 1 --2- 172.21.15.97:0/2295705136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887415c520 0x7f887415dd60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f885c015000 tx=0x7f885c004750 comp rx=0 tx=0).stop 2023-07-06T03:44:48.696 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.693+0000 7f88795ff700 1 -- 172.21.15.97:0/2295705136 shutdown_connections 2023-07-06T03:44:48.696 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.693+0000 7f88795ff700 1 --2- 172.21.15.97:0/2295705136 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8874160dd0 0x7f88740b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:48.696 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.693+0000 7f88795ff700 1 --2- 172.21.15.97:0/2295705136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88740cd390 0x7f88740cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:48.697 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.693+0000 7f88795ff700 1 --2- 172.21.15.97:0/2295705136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f887415c520 0x7f887415dd60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:48.697 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.693+0000 7f88795ff700 1 -- 172.21.15.97:0/2295705136 >> 172.21.15.97:0/2295705136 conn(0x7f8874071790 msgr2=0x7f887415ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:48.697 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.693+0000 7f88795ff700 1 -- 172.21.15.97:0/2295705136 shutdown_connections 2023-07-06T03:44:48.697 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:48.693+0000 7f88795ff700 1 -- 172.21.15.97:0/2295705136 wait complete. 2023-07-06T03:44:48.710 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-06T03:44:49.068 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:49.065+0000 7ff302edf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T03:44:49.221 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:49.218+0000 7f51b8d48200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T03:44:49.451 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:49.448+0000 7ff302edf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T03:44:49.606 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:49.603+0000 7f51b8d48200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T03:44:49.632 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:49.630+0000 7ff302edf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T03:44:49.786 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:49.783+0000 7f51b8d48200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T03:44:50.077 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:50.074+0000 7ff302edf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T03:44:50.229 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:50.227+0000 7f51b8d48200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T03:44:50.263 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:50.260+0000 7ff302edf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T03:44:50.415 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:50.412+0000 7f51b8d48200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T03:44:50.478 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:50.475+0000 7ff302edf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T03:44:50.621 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:50.618+0000 7f51b8d48200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T03:44:50.866 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:50.864+0000 7ff302edf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T03:44:51.093 INFO:tasks.ceph.mgr.x.smithi146.stderr:2023-07-06T03:44:51.091+0000 7f51b8d48200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T03:44:51.496 INFO:tasks.ceph.osd.7.smithi146.stderr:2023-07-06T03:44:51.494+0000 7f959a5fa640 -1 osd.7 0 log_to_monitors true 2023-07-06T03:44:51.720 INFO:tasks.ceph.osd.0.smithi097.stderr:2023-07-06T03:44:51.718+0000 7fed5a741640 -1 osd.0 0 log_to_monitors true 2023-07-06T03:44:51.745 INFO:tasks.ceph.osd.4.smithi146.stderr:2023-07-06T03:44:51.743+0000 7f56afdbc640 -1 osd.4 0 log_to_monitors true 2023-07-06T03:44:51.791 INFO:tasks.ceph.osd.3.smithi097.stderr:2023-07-06T03:44:51.788+0000 7ff4c14c6640 -1 osd.3 0 log_to_monitors true 2023-07-06T03:44:51.793 INFO:tasks.ceph.osd.5.smithi146.stderr:2023-07-06T03:44:51.792+0000 7fcb70a00640 -1 osd.5 0 log_to_monitors true 2023-07-06T03:44:51.802 INFO:tasks.ceph.osd.2.smithi097.stderr:2023-07-06T03:44:51.799+0000 7f71fb7dc640 -1 osd.2 0 log_to_monitors true 2023-07-06T03:44:51.811 INFO:tasks.ceph.osd.1.smithi097.stderr:2023-07-06T03:44:51.808+0000 7f2f9b551640 -1 osd.1 0 log_to_monitors true 2023-07-06T03:44:51.815 INFO:tasks.ceph.osd.6.smithi146.stderr:2023-07-06T03:44:51.814+0000 7fcb31639640 -1 osd.6 0 log_to_monitors true 2023-07-06T03:44:51.888 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:51.885+0000 7ff2bc7fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-06T03:44:51.889 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-07-06T03:44:51.886+0000 7ff2bc7fa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-06T03:44:54.906 INFO:tasks.ceph.osd.4.smithi146.stderr:2023-07-06T03:44:54.904+0000 7f569e3f6700 -1 osd.4 0 waiting for initial osdmap 2023-07-06T03:44:54.924 INFO:tasks.ceph.osd.4.smithi146.stderr:2023-07-06T03:44:54.922+0000 7f56963fb700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T03:44:55.023 INFO:tasks.ceph.osd.2.smithi097.stderr:2023-07-06T03:44:55.020+0000 7f71eb5f9700 -1 osd.2 0 waiting for initial osdmap 2023-07-06T03:44:55.024 INFO:tasks.ceph.osd.6.smithi146.stderr:2023-07-06T03:44:55.023+0000 7fcb1fdf6700 -1 osd.6 0 waiting for initial osdmap 2023-07-06T03:44:55.041 INFO:tasks.ceph.osd.2.smithi097.stderr:2023-07-06T03:44:55.038+0000 7f71e1dfb700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T03:44:55.042 INFO:tasks.ceph.osd.6.smithi146.stderr:2023-07-06T03:44:55.040+0000 7fcb17dfb700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T03:44:55.073 INFO:tasks.ceph.osd.5.smithi146.stderr:2023-07-06T03:44:55.071+0000 7fcb607f9700 -1 osd.5 0 waiting for initial osdmap 2023-07-06T03:44:55.086 INFO:tasks.ceph.osd.5.smithi146.stderr:2023-07-06T03:44:55.085+0000 7fcb56ffb700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T03:44:55.159 INFO:tasks.ceph.osd.7.smithi146.stderr:2023-07-06T03:44:55.157+0000 7f9588bf6700 -1 osd.7 0 waiting for initial osdmap 2023-07-06T03:44:55.176 INFO:tasks.ceph.osd.7.smithi146.stderr:2023-07-06T03:44:55.175+0000 7f9580bfb700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T03:44:55.178 INFO:tasks.ceph.osd.3.smithi097.stderr:2023-07-06T03:44:55.176+0000 7ff4afbf6700 -1 osd.3 0 waiting for initial osdmap 2023-07-06T03:44:55.186 INFO:tasks.ceph.osd.1.smithi097.stderr:2023-07-06T03:44:55.183+0000 7f2f89bf6700 -1 osd.1 0 waiting for initial osdmap 2023-07-06T03:44:55.195 INFO:tasks.ceph.osd.3.smithi097.stderr:2023-07-06T03:44:55.192+0000 7ff4a7bfb700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T03:44:55.201 INFO:tasks.ceph.osd.1.smithi097.stderr:2023-07-06T03:44:55.198+0000 7f2f81bfb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T03:44:55.323 INFO:tasks.ceph.osd.0.smithi097.stderr:2023-07-06T03:44:55.320+0000 7fed48df6700 -1 osd.0 0 waiting for initial osdmap 2023-07-06T03:44:55.337 INFO:tasks.ceph.osd.0.smithi097.stderr:2023-07-06T03:44:55.334+0000 7fed40dfb700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T03:44:55.515 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T03:44:55.685 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.683+0000 7f0a549ff700 1 Processor -- start 2023-07-06T03:44:55.686 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.683+0000 7f0a549ff700 1 -- start start 2023-07-06T03:44:55.687 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.684+0000 7f0a549ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0a5015c520 0x7f0a500cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:55.687 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.684+0000 7f0a549ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0a50160d00 0x7f0a500b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:55.687 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.684+0000 7f0a549ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a500cdae0 con 0x7f0a50160180 2023-07-06T03:44:55.687 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.684+0000 7f0a549ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a500cd8d0 con 0x7f0a50160d00 2023-07-06T03:44:55.688 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.684+0000 7f0a549ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a500b6640 con 0x7f0a5015c520 2023-07-06T03:44:55.688 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.684+0000 7f0a4ed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0a50160d00 0x7f0a500b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:55.688 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.684+0000 7f0a4ed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0a50160d00 0x7f0a500b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43456/0 (socket says 172.21.15.97:43456) 2023-07-06T03:44:55.688 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.684+0000 7f0a4ed9d700 1 -- 172.21.15.97:0/2158315125 learned_addr learned my addr 172.21.15.97:0/2158315125 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:44:55.688 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.684+0000 7f0a47fff700 1 --1- 172.21.15.97:0/2158315125 >> v1:172.21.15.97:6789/0 conn(0x7f0a50160180 0x7f0a501605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46100/0 (socket says 172.21.15.97:46100) 2023-07-06T03:44:55.688 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.684+0000 7f0a4e59c700 1 --2- 172.21.15.97:0/2158315125 >> v2:172.21.15.146:3300/0 conn(0x7f0a5015c520 0x7f0a500cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:55.689 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.685+0000 7f0a4ed9d700 1 -- 172.21.15.97:0/2158315125 >> v2:172.21.15.146:3300/0 conn(0x7f0a5015c520 msgr2=0x7f0a500cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:44:55.690 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.685+0000 7f0a4ed9d700 1 --2- 172.21.15.97:0/2158315125 >> v2:172.21.15.146:3300/0 conn(0x7f0a5015c520 0x7f0a500cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:55.690 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.685+0000 7f0a4ed9d700 1 -- 172.21.15.97:0/2158315125 >> v1:172.21.15.97:6789/0 conn(0x7f0a50160180 legacy=0x7f0a501605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:55.690 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.685+0000 7f0a4ed9d700 1 -- 172.21.15.97:0/2158315125 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a500b6920 con 0x7f0a50160d00 2023-07-06T03:44:55.690 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.685+0000 7f0a4dd9b700 1 -- 172.21.15.97:0/2158315125 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769723545 0 0) 0x7f0a500cdae0 con 0x7f0a50160180 2023-07-06T03:44:55.690 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.685+0000 7f0a4ed9d700 1 --2- 172.21.15.97:0/2158315125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a50160d00 0x7f0a500b7400 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0a500b4c70 tx=0x7f0a4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08eba5775844 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:55.690 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.685+0000 7f0a4dd9b700 1 -- 172.21.15.97:0/2158315125 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800e070 con 0x7f0a50160d00 2023-07-06T03:44:55.690 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.685+0000 7f0a4dd9b700 1 -- 172.21.15.97:0/2158315125 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4800b910 con 0x7f0a50160d00 2023-07-06T03:44:55.691 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.685+0000 7f0a4dd9b700 1 -- 172.21.15.97:0/2158315125 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800bcc0 con 0x7f0a50160d00 2023-07-06T03:44:55.691 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.686+0000 7f0a549ff700 1 -- 172.21.15.97:0/2158315125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a50160d00 msgr2=0x7f0a500b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:55.691 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.686+0000 7f0a549ff700 1 --2- 172.21.15.97:0/2158315125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a50160d00 0x7f0a500b7400 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0a500b4c70 tx=0x7f0a4800b330 comp rx=0 tx=0).stop 2023-07-06T03:44:55.691 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.686+0000 7f0a549ff700 1 -- 172.21.15.97:0/2158315125 shutdown_connections 2023-07-06T03:44:55.691 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.686+0000 7f0a549ff700 1 --2- 172.21.15.97:0/2158315125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a50160d00 0x7f0a500b7400 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:55.691 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.686+0000 7f0a549ff700 1 --2- 172.21.15.97:0/2158315125 >> v2:172.21.15.146:3300/0 conn(0x7f0a5015c520 0x7f0a500cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:55.691 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.686+0000 7f0a549ff700 1 -- 172.21.15.97:0/2158315125 >> 172.21.15.97:0/2158315125 conn(0x7f0a50071790 msgr2=0x7f0a50071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:55.691 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.686+0000 7f0a549ff700 1 -- 172.21.15.97:0/2158315125 shutdown_connections 2023-07-06T03:44:55.692 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.686+0000 7f0a549ff700 1 -- 172.21.15.97:0/2158315125 wait complete. 2023-07-06T03:44:55.693 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.688+0000 7f0a549ff700 1 Processor -- start 2023-07-06T03:44:55.693 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.688+0000 7f0a549ff700 1 -- start start 2023-07-06T03:44:55.694 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a549ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a5015c520 0x7f0a50155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:55.694 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a549ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a50160180 0x7f0a5015cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:55.694 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a549ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a50160d00 0x7f0a5015d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:55.694 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a549ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a500cb920 con 0x7f0a5015c520 2023-07-06T03:44:55.694 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a549ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a50155ab0 con 0x7f0a50160180 2023-07-06T03:44:55.694 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a549ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5015dec0 con 0x7f0a50160d00 2023-07-06T03:44:55.694 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a4e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a5015c520 0x7f0a50155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:55.694 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a4e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a5015c520 0x7f0a50155570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43468/0 (socket says 172.21.15.97:43468) 2023-07-06T03:44:55.695 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a4e59c700 1 -- 172.21.15.97:0/2175076619 learned_addr learned my addr 172.21.15.97:0/2175076619 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:44:55.695 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a47fff700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a50160180 0x7f0a5015cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:55.695 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a4ed9d700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a50160d00 0x7f0a5015d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:55.695 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a4e59c700 1 -- 172.21.15.97:0/2175076619 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a50160d00 msgr2=0x7f0a5015d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:55.695 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a4e59c700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a50160d00 0x7f0a5015d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:55.695 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a4e59c700 1 -- 172.21.15.97:0/2175076619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a50160180 msgr2=0x7f0a5015cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:55.695 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a4e59c700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a50160180 0x7f0a5015cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:55.696 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.689+0000 7f0a4e59c700 1 -- 172.21.15.97:0/2175076619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a48009b30 con 0x7f0a5015c520 2023-07-06T03:44:55.696 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.690+0000 7f0a4e59c700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a5015c520 0x7f0a50155570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000e870 tx=0x7f0a4000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:55.696 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.690+0000 7f0a46ffd700 1 -- 172.21.15.97:0/2175076619 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40009590 con 0x7f0a5015c520 2023-07-06T03:44:55.696 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.690+0000 7f0a46ffd700 1 -- 172.21.15.97:0/2175076619 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a40013680 con 0x7f0a5015c520 2023-07-06T03:44:55.696 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.690+0000 7f0a549ff700 1 -- 172.21.15.97:0/2175076619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a500cbb00 con 0x7f0a5015c520 2023-07-06T03:44:55.696 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.690+0000 7f0a46ffd700 1 -- 172.21.15.97:0/2175076619 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a400139f0 con 0x7f0a5015c520 2023-07-06T03:44:55.697 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.690+0000 7f0a549ff700 1 -- 172.21.15.97:0/2175076619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a500cc0c0 con 0x7f0a5015c520 2023-07-06T03:44:55.697 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.691+0000 7f0a549ff700 1 -- 172.21.15.97:0/2175076619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5004e340 con 0x7f0a5015c520 2023-07-06T03:44:55.698 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.692+0000 7f0a46ffd700 1 -- 172.21.15.97:0/2175076619 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f0a40013b50 con 0x7f0a5015c520 2023-07-06T03:44:55.698 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.693+0000 7f0a46ffd700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0a3006ac50 0x7f0a3006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:44:55.698 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.693+0000 7f0a46ffd700 1 -- 172.21.15.97:0/2175076619 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f0a400883a0 con 0x7f0a5015c520 2023-07-06T03:44:55.698 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.694+0000 7f0a47fff700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0a3006ac50 0x7f0a3006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:44:55.698 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.694+0000 7f0a47fff700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0a3006ac50 0x7f0a3006d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0a500b63f0 tx=0x7f0a38007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:44:55.701 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.699+0000 7f0a46ffd700 1 -- 172.21.15.97:0/2175076619 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0a400543c0 con 0x7f0a5015c520 2023-07-06T03:44:55.898 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.895+0000 7f0a549ff700 1 -- 172.21.15.97:0/2175076619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a500b4f40 con 0x7f0a5015c520 2023-07-06T03:44:55.899 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.896+0000 7f0a46ffd700 1 -- 172.21.15.97:0/2175076619 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0a40021090 con 0x7f0a5015c520 2023-07-06T03:44:55.900 INFO:teuthology.misc.health.smithi097.stdout: 2023-07-06T03:44:55.900 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":12,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:44:52.937434+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:44:55.902 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 -- 172.21.15.97:0/2175076619 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0a3006ac50 msgr2=0x7f0a3006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:55.902 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0a3006ac50 0x7f0a3006d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0a500b63f0 tx=0x7f0a38007de0 comp rx=0 tx=0).stop 2023-07-06T03:44:55.902 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 -- 172.21.15.97:0/2175076619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a5015c520 msgr2=0x7f0a50155570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:44:55.902 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a5015c520 0x7f0a50155570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000e870 tx=0x7f0a4000eb80 comp rx=0 tx=0).stop 2023-07-06T03:44:55.903 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 -- 172.21.15.97:0/2175076619 shutdown_connections 2023-07-06T03:44:55.903 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0a3006ac50 0x7f0a3006d100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:55.903 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0a50160d00 0x7f0a5015d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:55.903 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a5015c520 0x7f0a50155570 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:55.903 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 --2- 172.21.15.97:0/2175076619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a50160180 0x7f0a5015cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:44:55.903 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 -- 172.21.15.97:0/2175076619 >> 172.21.15.97:0/2175076619 conn(0x7f0a50071790 msgr2=0x7f0a50071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:44:55.904 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 -- 172.21.15.97:0/2175076619 shutdown_connections 2023-07-06T03:44:55.904 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:44:55.898+0000 7f0a549ff700 1 -- 172.21.15.97:0/2175076619 wait complete. 2023-07-06T03:44:55.918 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-06T03:45:02.725 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:02.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:45:02.843 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:45:02.909 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.906+0000 7f7e8e7ff700 1 Processor -- start 2023-07-06T03:45:02.909 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.906+0000 7f7e8e7ff700 1 -- start start 2023-07-06T03:45:02.910 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.907+0000 7f7e8e7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7e8815c520 0x7f7e880cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:02.911 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.907+0000 7f7e8e7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7e88160180 0x7f7e881605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:02.911 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.907+0000 7f7e8e7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e880cdae0 con 0x7f7e88160b30 2023-07-06T03:45:02.911 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.907+0000 7f7e8e7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e880cd8d0 con 0x7f7e8815c520 2023-07-06T03:45:02.911 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.907+0000 7f7e8e7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e880b6640 con 0x7f7e88160180 2023-07-06T03:45:02.911 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.908+0000 7f7e87fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7e8815c520 0x7f7e880cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:02.911 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.908+0000 7f7e87fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7e8815c520 0x7f7e880cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39916/0 (socket says 172.21.15.97:39916) 2023-07-06T03:45:02.911 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.908+0000 7f7e87fff700 1 -- 172.21.15.97:0/1663314971 learned_addr learned my addr 172.21.15.97:0/1663314971 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:02.911 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.908+0000 7f7e877fe700 1 --2- 172.21.15.97:0/1663314971 >> v2:172.21.15.146:3300/0 conn(0x7f7e88160180 0x7f7e881605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:02.913 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.908+0000 7f7e87fff700 1 -- 172.21.15.97:0/1663314971 >> v2:172.21.15.146:3300/0 conn(0x7f7e88160180 msgr2=0x7f7e881605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:02.913 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.908+0000 7f7e87fff700 1 --2- 172.21.15.97:0/1663314971 >> v2:172.21.15.146:3300/0 conn(0x7f7e88160180 0x7f7e881605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:02.913 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.908+0000 7f7e87fff700 1 -- 172.21.15.97:0/1663314971 >> v1:172.21.15.146:6789/0 conn(0x7f7e88160b30 legacy=0x7f7e880cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:02.913 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.908+0000 7f7e87fff700 1 -- 172.21.15.97:0/1663314971 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e880b6920 con 0x7f7e8815c520 2023-07-06T03:45:02.913 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.908+0000 7f7e87fff700 1 --2- 172.21.15.97:0/1663314971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e8815c520 0x7f7e880cd390 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e880b5930 tx=0x7f7e78009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef06dd611b48685b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:02.914 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.909+0000 7f7e86ffd700 1 -- 172.21.15.97:0/1663314971 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78011830 con 0x7f7e8815c520 2023-07-06T03:45:02.914 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.909+0000 7f7e86ffd700 1 -- 172.21.15.97:0/1663314971 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e78011990 con 0x7f7e8815c520 2023-07-06T03:45:02.914 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.909+0000 7f7e86ffd700 1 -- 172.21.15.97:0/1663314971 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78011d20 con 0x7f7e8815c520 2023-07-06T03:45:02.914 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.909+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/1663314971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e8815c520 msgr2=0x7f7e880cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:02.914 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.909+0000 7f7e8e7ff700 1 --2- 172.21.15.97:0/1663314971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e8815c520 0x7f7e880cd390 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e880b5930 tx=0x7f7e78009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:02.914 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.909+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/1663314971 shutdown_connections 2023-07-06T03:45:02.914 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.909+0000 7f7e8e7ff700 1 --2- 172.21.15.97:0/1663314971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e8815c520 0x7f7e880cd390 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:02.914 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.909+0000 7f7e8e7ff700 1 --2- 172.21.15.97:0/1663314971 >> v2:172.21.15.146:3300/0 conn(0x7f7e88160180 0x7f7e881605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:02.915 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.909+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/1663314971 >> 172.21.15.97:0/1663314971 conn(0x7f7e88071790 msgr2=0x7f7e88071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:02.915 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.909+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/1663314971 shutdown_connections 2023-07-06T03:45:02.915 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.909+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/1663314971 wait complete. 2023-07-06T03:45:02.916 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.911+0000 7f7e8e7ff700 1 Processor -- start 2023-07-06T03:45:02.917 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.911+0000 7f7e8e7ff700 1 -- start start 2023-07-06T03:45:02.917 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.912+0000 7f7e8e7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8815c520 0x7f7e8815b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:02.917 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.912+0000 7f7e8e7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e88160180 0x7f7e881f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:02.917 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.912+0000 7f7e8e7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e88160b30 0x7f7e881f1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:02.917 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.912+0000 7f7e8e7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8815bd20 con 0x7f7e88160b30 2023-07-06T03:45:02.917 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.912+0000 7f7e8e7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e881f2fb0 con 0x7f7e8815c520 2023-07-06T03:45:02.918 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.912+0000 7f7e8e7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880d1710 con 0x7f7e88160180 2023-07-06T03:45:02.918 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.912+0000 7f7e8cd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e88160b30 0x7f7e881f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:02.918 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.912+0000 7f7e877fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e88160180 0x7f7e881f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:02.918 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.912+0000 7f7e8cd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e88160b30 0x7f7e881f1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39932/0 (socket says 172.21.15.97:39932) 2023-07-06T03:45:02.918 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.912+0000 7f7e8cd9c700 1 -- 172.21.15.97:0/2282918374 learned_addr learned my addr 172.21.15.97:0/2282918374 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:02.918 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.912+0000 7f7e87fff700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8815c520 0x7f7e8815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:02.918 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.913+0000 7f7e8cd9c700 1 -- 172.21.15.97:0/2282918374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e88160180 msgr2=0x7f7e881f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:02.919 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.913+0000 7f7e8cd9c700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e88160180 0x7f7e881f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:02.919 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.913+0000 7f7e8cd9c700 1 -- 172.21.15.97:0/2282918374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8815c520 msgr2=0x7f7e8815b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:02.919 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.913+0000 7f7e8cd9c700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8815c520 0x7f7e8815b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:02.919 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.913+0000 7f7e8cd9c700 1 -- 172.21.15.97:0/2282918374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e78009550 con 0x7f7e88160b30 2023-07-06T03:45:02.919 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.913+0000 7f7e8cd9c700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e88160b30 0x7f7e881f1760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000cc50 tx=0x7f7e8000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:02.919 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.913+0000 7f7e84ff9700 1 -- 172.21.15.97:0/2282918374 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80003c00 con 0x7f7e88160b30 2023-07-06T03:45:02.919 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.913+0000 7f7e84ff9700 1 -- 172.21.15.97:0/2282918374 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e80003d60 con 0x7f7e88160b30 2023-07-06T03:45:02.919 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.913+0000 7f7e84ff9700 1 -- 172.21.15.97:0/2282918374 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000aa50 con 0x7f7e88160b30 2023-07-06T03:45:02.920 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.913+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/2282918374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e880d0de0 con 0x7f7e88160b30 2023-07-06T03:45:02.920 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.913+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/2282918374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e880d13a0 con 0x7f7e88160b30 2023-07-06T03:45:02.920 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.914+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/2282918374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8804e340 con 0x7f7e88160b30 2023-07-06T03:45:02.921 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.915+0000 7f7e84ff9700 1 -- 172.21.15.97:0/2282918374 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e8000abb0 con 0x7f7e88160b30 2023-07-06T03:45:02.921 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.916+0000 7f7e84ff9700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7406ace0 0x7f7e7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:02.921 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.916+0000 7f7e84ff9700 1 -- 172.21.15.97:0/2282918374 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7e80088ef0 con 0x7f7e88160b30 2023-07-06T03:45:02.921 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.916+0000 7f7e87fff700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7406ace0 0x7f7e7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:02.921 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.917+0000 7f7e87fff700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7406ace0 0x7f7e7406d190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e78014040 tx=0x7f7e78008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:02.924 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:02.922+0000 7f7e84ff9700 1 -- 172.21.15.97:0/2282918374 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7e80054450 con 0x7f7e88160b30 2023-07-06T03:45:03.126 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.123+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/2282918374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e880b4f40 con 0x7f7e88160b30 2023-07-06T03:45:03.127 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.124+0000 7f7e84ff9700 1 -- 172.21.15.97:0/2282918374 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f7e8000ae60 con 0x7f7e88160b30 2023-07-06T03:45:03.127 INFO:teuthology.misc.health.smithi097.stdout: 2023-07-06T03:45:03.127 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":16,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:44:59.095312+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:03.130 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.126+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/2282918374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7406ace0 msgr2=0x7f7e7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:03.130 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.126+0000 7f7e8e7ff700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7406ace0 0x7f7e7406d190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e78014040 tx=0x7f7e78008030 comp rx=0 tx=0).stop 2023-07-06T03:45:03.131 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.126+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/2282918374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e88160b30 msgr2=0x7f7e881f1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:03.131 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.126+0000 7f7e8e7ff700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e88160b30 0x7f7e881f1760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000cc50 tx=0x7f7e8000cf60 comp rx=0 tx=0).stop 2023-07-06T03:45:03.131 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.126+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/2282918374 shutdown_connections 2023-07-06T03:45:03.131 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.126+0000 7f7e8e7ff700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7406ace0 0x7f7e7406d190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:03.132 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.126+0000 7f7e8e7ff700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e88160b30 0x7f7e881f1760 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:03.132 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.126+0000 7f7e8e7ff700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8815c520 0x7f7e8815b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:03.132 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.126+0000 7f7e8e7ff700 1 --2- 172.21.15.97:0/2282918374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e88160180 0x7f7e881f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:03.133 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.127+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/2282918374 >> 172.21.15.97:0/2282918374 conn(0x7f7e88071790 msgr2=0x7f7e880b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:03.133 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.127+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/2282918374 shutdown_connections 2023-07-06T03:45:03.133 INFO:teuthology.misc.health.smithi097.stderr:2023-07-06T03:45:03.127+0000 7f7e8e7ff700 1 -- 172.21.15.97:0/2282918374 wait complete. 2023-07-06T03:45:03.145 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-07-06T03:45:03.146 INFO:tasks.ceph:Creating RBD pool 2023-07-06T03:45:03.146 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd pool create rbd 8 2023-07-06T03:45:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.333+0000 7f8fca3ff700 1 Processor -- start 2023-07-06T03:45:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.333+0000 7f8fca3ff700 1 -- start start 2023-07-06T03:45:03.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.334+0000 7f8fca3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8fc415c610 0x7f8fc40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:03.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.334+0000 7f8fca3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8fc406b360 0x7f8fc40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:03.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.334+0000 7f8fca3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8fc40d2210 0x7f8fc40d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:03.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.334+0000 7f8fca3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc40d0780 con 0x7f8fc415c610 2023-07-06T03:45:03.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.334+0000 7f8fca3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc415c9e0 con 0x7f8fc406b360 2023-07-06T03:45:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.334+0000 7f8fca3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b5650 con 0x7f8fc40d2210 2023-07-06T03:45:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.334+0000 7f8fc3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8fc415c610 0x7f8fc40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.334+0000 7f8fc3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8fc415c610 0x7f8fc40b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39940/0 (socket says 172.21.15.97:39940) 2023-07-06T03:45:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.334+0000 7f8fc3fff700 1 -- 172.21.15.97:0/3288342538 learned_addr learned my addr 172.21.15.97:0/3288342538 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.335+0000 7f8fc3fff700 1 -- 172.21.15.97:0/3288342538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8fc40d2210 msgr2=0x7f8fc40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.335+0000 7f8fc3fff700 1 --2- 172.21.15.97:0/3288342538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8fc40d2210 0x7f8fc40d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.335+0000 7f8fc3fff700 1 -- 172.21.15.97:0/3288342538 >> v2:172.21.15.146:3300/0 conn(0x7f8fc406b360 msgr2=0x7f8fc40b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:03.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.335+0000 7f8fc37fe700 1 --2- 172.21.15.97:0/3288342538 >> v2:172.21.15.146:3300/0 conn(0x7f8fc406b360 0x7f8fc40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:03.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.335+0000 7f8fc3fff700 1 --2- 172.21.15.97:0/3288342538 >> v2:172.21.15.146:3300/0 conn(0x7f8fc406b360 0x7f8fc40b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:03.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.335+0000 7f8fc3fff700 1 -- 172.21.15.97:0/3288342538 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc40d01c0 con 0x7f8fc415c610 2023-07-06T03:45:03.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.335+0000 7f8fc37fe700 1 --2- 172.21.15.97:0/3288342538 >> v2:172.21.15.146:3300/0 conn(0x7f8fc406b360 0x7f8fc40b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:03.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.335+0000 7f8fc3fff700 1 --2- 172.21.15.97:0/3288342538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc415c610 0x7f8fc40b47b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8fc406a590 tx=0x7f8fb4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88807f7f3690029 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:03.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.335+0000 7f8fc2ffd700 1 -- 172.21.15.97:0/3288342538 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400b860 con 0x7f8fc415c610 2023-07-06T03:45:03.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.335+0000 7f8fc2ffd700 1 -- 172.21.15.97:0/3288342538 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb400b9c0 con 0x7f8fc415c610 2023-07-06T03:45:03.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.335+0000 7f8fc2ffd700 1 -- 172.21.15.97:0/3288342538 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400bd50 con 0x7f8fc415c610 2023-07-06T03:45:03.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.336+0000 7f8fca3ff700 1 -- 172.21.15.97:0/3288342538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc415c610 msgr2=0x7f8fc40b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:03.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.336+0000 7f8fca3ff700 1 --2- 172.21.15.97:0/3288342538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc415c610 0x7f8fc40b47b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8fc406a590 tx=0x7f8fb4009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:03.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.336+0000 7f8fca3ff700 1 -- 172.21.15.97:0/3288342538 shutdown_connections 2023-07-06T03:45:03.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.336+0000 7f8fca3ff700 1 --2- 172.21.15.97:0/3288342538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8fc40d2210 0x7f8fc40d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:03.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.336+0000 7f8fca3ff700 1 --2- 172.21.15.97:0/3288342538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc415c610 0x7f8fc40b47b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:03.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.336+0000 7f8fca3ff700 1 --2- 172.21.15.97:0/3288342538 >> v2:172.21.15.146:3300/0 conn(0x7f8fc406b360 0x7f8fc40b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:03.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.336+0000 7f8fca3ff700 1 -- 172.21.15.97:0/3288342538 >> 172.21.15.97:0/3288342538 conn(0x7f8fc406a6c0 msgr2=0x7f8fc406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:03.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.336+0000 7f8fca3ff700 1 -- 172.21.15.97:0/3288342538 shutdown_connections 2023-07-06T03:45:03.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.336+0000 7f8fca3ff700 1 -- 172.21.15.97:0/3288342538 wait complete. 2023-07-06T03:45:03.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.338+0000 7f8fca3ff700 1 Processor -- start 2023-07-06T03:45:03.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.338+0000 7f8fca3ff700 1 -- start start 2023-07-06T03:45:03.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fca3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc406b360 0x7f8fc414a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:03.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fca3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8fc40d2210 0x7f8fc40b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:03.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fca3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fc415c610 0x7f8fc415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:03.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fca3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b2790 con 0x7f8fc406b360 2023-07-06T03:45:03.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fca3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b35f0 con 0x7f8fc415c610 2023-07-06T03:45:03.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fca3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b29e0 con 0x7f8fc40d2210 2023-07-06T03:45:03.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc406b360 0x7f8fc414a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:03.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc406b360 0x7f8fc414a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39952/0 (socket says 172.21.15.97:39952) 2023-07-06T03:45:03.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc3fff700 1 -- 172.21.15.97:0/1339785073 learned_addr learned my addr 172.21.15.97:0/1339785073 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:03.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc37fe700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8fc40d2210 0x7f8fc40b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:03.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc899c700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fc415c610 0x7f8fc415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:03.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc3fff700 1 -- 172.21.15.97:0/1339785073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8fc40d2210 msgr2=0x7f8fc40b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:03.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc3fff700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8fc40d2210 0x7f8fc40b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:03.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc3fff700 1 -- 172.21.15.97:0/1339785073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fc415c610 msgr2=0x7f8fc415cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:03.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc3fff700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fc415c610 0x7f8fc415cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:03.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc3fff700 1 -- 172.21.15.97:0/1339785073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb4009550 con 0x7f8fc406b360 2023-07-06T03:45:03.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc37fe700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8fc40d2210 0x7f8fc40b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:03.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.339+0000 7f8fc899c700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fc415c610 0x7f8fc415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:03.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.340+0000 7f8fc3fff700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc406b360 0x7f8fc414a2d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4000c00 tx=0x7f8fb4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:03.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.340+0000 7f8fc0ff9700 1 -- 172.21.15.97:0/1339785073 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40036a0 con 0x7f8fc406b360 2023-07-06T03:45:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.340+0000 7f8fc0ff9700 1 -- 172.21.15.97:0/1339785073 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb401e070 con 0x7f8fc406b360 2023-07-06T03:45:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.340+0000 7f8fca3ff700 1 -- 172.21.15.97:0/1339785073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc40b2cc0 con 0x7f8fc406b360 2023-07-06T03:45:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.340+0000 7f8fc0ff9700 1 -- 172.21.15.97:0/1339785073 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4003a60 con 0x7f8fc406b360 2023-07-06T03:45:03.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.340+0000 7f8fca3ff700 1 -- 172.21.15.97:0/1339785073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc40b3280 con 0x7f8fc406b360 2023-07-06T03:45:03.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.341+0000 7f8fca3ff700 1 -- 172.21.15.97:0/1339785073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc40b3410 con 0x7f8fc406b360 2023-07-06T03:45:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.342+0000 7f8fc0ff9700 1 -- 172.21.15.97:0/1339785073 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8fb4003bc0 con 0x7f8fc406b360 2023-07-06T03:45:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.343+0000 7f8fc0ff9700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8fb006ad10 0x7f8fb006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.344+0000 7f8fc0ff9700 1 -- 172.21.15.97:0/1339785073 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8fb4020070 con 0x7f8fc406b360 2023-07-06T03:45:03.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.344+0000 7f8fc37fe700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8fb006ad10 0x7f8fb006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:03.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.344+0000 7f8fc37fe700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8fb006ad10 0x7f8fb006d1c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4153220 tx=0x7f8fb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:03.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.348+0000 7f8fc0ff9700 1 -- 172.21.15.97:0/1339785073 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8fb4055af0 con 0x7f8fc406b360 2023-07-06T03:45:03.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:03.543+0000 7f8fca3ff700 1 -- 172.21.15.97:0/1339785073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f8fc404e340 con 0x7f8fc406b360 2023-07-06T03:45:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.119+0000 7f8fc0ff9700 1 -- 172.21.15.97:0/1339785073 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f8fb4059110 con 0x7f8fc406b360 2023-07-06T03:45:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 -- 172.21.15.97:0/1339785073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8fb006ad10 msgr2=0x7f8fb006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8fb006ad10 0x7f8fb006d1c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4153220 tx=0x7f8fb8008040 comp rx=0 tx=0).stop 2023-07-06T03:45:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 -- 172.21.15.97:0/1339785073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc406b360 msgr2=0x7f8fc414a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc406b360 0x7f8fc414a2d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4000c00 tx=0x7f8fb4004750 comp rx=0 tx=0).stop 2023-07-06T03:45:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 -- 172.21.15.97:0/1339785073 shutdown_connections 2023-07-06T03:45:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8fb006ad10 0x7f8fb006d1c0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4153220 tx=0x7f8fb8008040 comp rx=0 tx=0).stop 2023-07-06T03:45:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fc415c610 0x7f8fc415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc406b360 0x7f8fc414a2d0 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4000c00 tx=0x7f8fb4004750 comp rx=0 tx=0).stop 2023-07-06T03:45:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 --2- 172.21.15.97:0/1339785073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8fc40d2210 0x7f8fc40b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 -- 172.21.15.97:0/1339785073 >> 172.21.15.97:0/1339785073 conn(0x7f8fc406a6c0 msgr2=0x7f8fc40d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 -- 172.21.15.97:0/1339785073 shutdown_connections 2023-07-06T03:45:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.123+0000 7f8fca3ff700 1 -- 172.21.15.97:0/1339785073 wait complete. 2023-07-06T03:45:04.128 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'rbd' created 2023-07-06T03:45:04.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-07-06T03:45:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.334+0000 7f619cdff700 1 Processor -- start 2023-07-06T03:45:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.334+0000 7f619cdff700 1 -- start start 2023-07-06T03:45:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.335+0000 7f619cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f619806b0c0 0x7f61980b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.335+0000 7f619cdff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f61980b4ca0 0x7f61980d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.335+0000 7f619cdff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61980cd330 con 0x7f61980b85f0 2023-07-06T03:45:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.335+0000 7f619cdff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61980b89c0 con 0x7f61980b4ca0 2023-07-06T03:45:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.335+0000 7f619cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619806b510 con 0x7f619806b0c0 2023-07-06T03:45:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.336+0000 7f6196d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f61980b4ca0 0x7f61980d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.336+0000 7f6196d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f61980b4ca0 0x7f61980d09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39968/0 (socket says 172.21.15.97:39968) 2023-07-06T03:45:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.336+0000 7f6196d9d700 1 -- 172.21.15.97:0/101905478 learned_addr learned my addr 172.21.15.97:0/101905478 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.336+0000 7f618ffff700 1 --2- 172.21.15.97:0/101905478 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f619806b0c0 0x7f61980b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.336+0000 7f6196d9d700 1 -- 172.21.15.97:0/101905478 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f619806b0c0 msgr2=0x7f61980b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.336+0000 7f6196d9d700 1 --2- 172.21.15.97:0/101905478 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f619806b0c0 0x7f61980b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.336+0000 7f6196d9d700 1 -- 172.21.15.97:0/101905478 >> v1:172.21.15.146:6789/0 conn(0x7f61980b85f0 legacy=0x7f61980cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.336+0000 7f6196d9d700 1 -- 172.21.15.97:0/101905478 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61980d02a0 con 0x7f61980b4ca0 2023-07-06T03:45:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.336+0000 7f618ffff700 1 --2- 172.21.15.97:0/101905478 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f619806b0c0 0x7f61980b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.336+0000 7f6196d9d700 1 --2- 172.21.15.97:0/101905478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61980b4ca0 0x7f61980d09b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6190009e50 tx=0x7f619000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4db82011b2ac51 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.337+0000 7f6195d9b700 1 -- 172.21.15.97:0/101905478 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000e070 con 0x7f61980b4ca0 2023-07-06T03:45:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.337+0000 7f6195d9b700 1 -- 172.21.15.97:0/101905478 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619000b9c0 con 0x7f61980b4ca0 2023-07-06T03:45:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.337+0000 7f6195d9b700 1 -- 172.21.15.97:0/101905478 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000bd70 con 0x7f61980b4ca0 2023-07-06T03:45:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.337+0000 7f619cdff700 1 -- 172.21.15.97:0/101905478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61980b4ca0 msgr2=0x7f61980d09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.337+0000 7f619cdff700 1 --2- 172.21.15.97:0/101905478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61980b4ca0 0x7f61980d09b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6190009e50 tx=0x7f619000b330 comp rx=0 tx=0).stop 2023-07-06T03:45:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.337+0000 7f619cdff700 1 -- 172.21.15.97:0/101905478 shutdown_connections 2023-07-06T03:45:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.337+0000 7f619cdff700 1 --2- 172.21.15.97:0/101905478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61980b4ca0 0x7f61980d09b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.337+0000 7f619cdff700 1 --2- 172.21.15.97:0/101905478 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f619806b0c0 0x7f61980b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.337+0000 7f619cdff700 1 -- 172.21.15.97:0/101905478 >> 172.21.15.97:0/101905478 conn(0x7f619806a5b0 msgr2=0x7f619806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.337+0000 7f619cdff700 1 -- 172.21.15.97:0/101905478 shutdown_connections 2023-07-06T03:45:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.337+0000 7f619cdff700 1 -- 172.21.15.97:0/101905478 wait complete. 2023-07-06T03:45:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.339+0000 7f619cdff700 1 Processor -- start 2023-07-06T03:45:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.339+0000 7f619cdff700 1 -- start start 2023-07-06T03:45:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.340+0000 7f619cdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f619806b0c0 0x7f619815f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.340+0000 7f619cdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61980b4ca0 0x7f61981f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.340+0000 7f619cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61980b85f0 0x7f61980b7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.340+0000 7f619cdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619815f980 con 0x7f61980b4ca0 2023-07-06T03:45:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.340+0000 7f619cdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61980cae30 con 0x7f619806b0c0 2023-07-06T03:45:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.340+0000 7f619cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61980ca2b0 con 0x7f61980b85f0 2023-07-06T03:45:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.340+0000 7f618ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61980b4ca0 0x7f61981f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.340+0000 7f618ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61980b4ca0 0x7f61981f12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39980/0 (socket says 172.21.15.97:39980) 2023-07-06T03:45:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.340+0000 7f618ffff700 1 -- 172.21.15.97:0/3290987054 learned_addr learned my addr 172.21.15.97:0/3290987054 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f6196d9d700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61980b85f0 0x7f61980b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f619659c700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f619806b0c0 0x7f619815f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f618ffff700 1 -- 172.21.15.97:0/3290987054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61980b85f0 msgr2=0x7f61980b7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f618ffff700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61980b85f0 0x7f61980b7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f618ffff700 1 -- 172.21.15.97:0/3290987054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f619806b0c0 msgr2=0x7f619815f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f618ffff700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f619806b0c0 0x7f619815f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f618ffff700 1 -- 172.21.15.97:0/3290987054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6190009b30 con 0x7f61980b4ca0 2023-07-06T03:45:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f6196d9d700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61980b85f0 0x7f61980b7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f618ffff700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61980b4ca0 0x7f61981f12b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f618000e810 tx=0x7f618000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f618effd700 1 -- 172.21.15.97:0/3290987054 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180013610 con 0x7f61980b4ca0 2023-07-06T03:45:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f618effd700 1 -- 172.21.15.97:0/3290987054 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6180013770 con 0x7f61980b4ca0 2023-07-06T03:45:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f618effd700 1 -- 172.21.15.97:0/3290987054 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180013b20 con 0x7f61980b4ca0 2023-07-06T03:45:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f619cdff700 1 -- 172.21.15.97:0/3290987054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61980ca590 con 0x7f61980b4ca0 2023-07-06T03:45:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.341+0000 7f619cdff700 1 -- 172.21.15.97:0/3290987054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61980cab50 con 0x7f61980b4ca0 2023-07-06T03:45:04.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.342+0000 7f619cdff700 1 -- 172.21.15.97:0/3290987054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619804e340 con 0x7f61980b4ca0 2023-07-06T03:45:04.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.343+0000 7f618effd700 1 -- 172.21.15.97:0/3290987054 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6180013c80 con 0x7f61980b4ca0 2023-07-06T03:45:04.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.344+0000 7f618effd700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f617806ace0 0x7f617806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:04.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.344+0000 7f618effd700 1 -- 172.21.15.97:0/3290987054 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f6180088e90 con 0x7f61980b4ca0 2023-07-06T03:45:04.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.345+0000 7f619659c700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f617806ace0 0x7f617806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:04.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.345+0000 7f619659c700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f617806ace0 0x7f617806d190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6188000f60 tx=0x7f6188009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:04.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.349+0000 7f618effd700 1 -- 172.21.15.97:0/3290987054 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6180054290 con 0x7f61980b4ca0 2023-07-06T03:45:04.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:04.545+0000 7f619cdff700 1 -- 172.21.15.97:0/3290987054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f61980d21f0 con 0x7f61980b4ca0 2023-07-06T03:45:05.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.125+0000 7f618effd700 1 -- 172.21.15.97:0/3290987054 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f61800210c0 con 0x7f61980b4ca0 2023-07-06T03:45:05.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 -- 172.21.15.97:0/3290987054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f617806ace0 msgr2=0x7f617806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f617806ace0 0x7f617806d190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6188000f60 tx=0x7f6188009040 comp rx=0 tx=0).stop 2023-07-06T03:45:05.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 -- 172.21.15.97:0/3290987054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61980b4ca0 msgr2=0x7f61981f12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61980b4ca0 0x7f61981f12b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f618000e810 tx=0x7f618000ebd0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 -- 172.21.15.97:0/3290987054 shutdown_connections 2023-07-06T03:45:05.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f617806ace0 0x7f617806d190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f61980b85f0 0x7f61980b7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f619806b0c0 0x7f619815f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 --2- 172.21.15.97:0/3290987054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61980b4ca0 0x7f61981f12b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 -- 172.21.15.97:0/3290987054 >> 172.21.15.97:0/3290987054 conn(0x7f619806a5b0 msgr2=0x7f61980d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:05.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 -- 172.21.15.97:0/3290987054 shutdown_connections 2023-07-06T03:45:05.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.129+0000 7f619cdff700 1 -- 172.21.15.97:0/3290987054 wait complete. 2023-07-06T03:45:05.135 INFO:teuthology.orchestra.run.smithi097.stderr:enabled application 'rbd' on pool 'rbd' 2023-07-06T03:45:05.151 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-07-06T03:45:05.151 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T03:45:05.152 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T03:45:05.156 INFO:tasks.ceph.mds.a:Started 2023-07-06T03:45:05.156 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T03:45:05.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T03:45:05.160 INFO:tasks.ceph.mds.c:Started 2023-07-06T03:45:05.161 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T03:45:05.161 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T03:45:05.166 INFO:tasks.ceph.mds.b:Started 2023-07-06T03:45:05.167 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T03:45:05.167 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T03:45:05.171 INFO:tasks.ceph.mds.d:Started 2023-07-06T03:45:05.172 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-07-06T03:45:05.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:05.309 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:45:05.310 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:45:05.322 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-07-06T03:45:05.325 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:45:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.380+0000 7f8ac49ff700 1 Processor -- start 2023-07-06T03:45:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.380+0000 7f8ac49ff700 1 -- start start 2023-07-06T03:45:05.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.381+0000 7f8ac49ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ac00b8410 0x7f8ac0167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.381+0000 7f8ac49ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac00cdae0 con 0x7f8ac015c520 2023-07-06T03:45:05.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.381+0000 7f8ac49ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac00b6640 con 0x7f8ac00cd390 2023-07-06T03:45:05.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.381+0000 7f8ac49ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac00b6920 con 0x7f8ac00b8410 2023-07-06T03:45:05.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abe59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8ac015c520 0x7f8ac0160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57724/0 (socket says 172.21.15.97:57724) 2023-07-06T03:45:05.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abe59c700 1 -- 172.21.15.97:0/3569230068 learned_addr learned my addr 172.21.15.97:0/3569230068 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:05.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059854927 0 0) 0x7f8ac00cdae0 con 0x7f8ac015c520 2023-07-06T03:45:05.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa80035b0 con 0x7f8ac015c520 2023-07-06T03:45:05.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abed9d700 1 --2- 172.21.15.97:0/3569230068 >> v2:172.21.15.146:3300/0 conn(0x7f8ac00b8410 0x7f8ac0167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2294593647 0 0) 0x7f8aa80035b0 con 0x7f8ac015c520 2023-07-06T03:45:05.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 >> v2:172.21.15.146:3300/0 conn(0x7f8ac00b8410 msgr2=0x7f8ac0167510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:05.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 --2- 172.21.15.97:0/3569230068 >> v2:172.21.15.146:3300/0 conn(0x7f8ac00b8410 0x7f8ac0167510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 >> v1:172.21.15.146:6789/0 conn(0x7f8ac00cd390 legacy=0x7f8ac0160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac00b6c00 con 0x7f8ac015c520 2023-07-06T03:45:05.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f8ab0001670 con 0x7f8ac015c520 2023-07-06T03:45:05.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 >> v1:172.21.15.97:6789/0 conn(0x7f8ac015c520 legacy=0x7f8ac0160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8aa8003a00 0x7f8aa8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 0x7f8aa80048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8aa80050b0 0x7f8aa800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00b6920 con 0x7f8aa80043f0 2023-07-06T03:45:05.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa800c000 con 0x7f8aa8003a00 2023-07-06T03:45:05.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.382+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa800c1c0 con 0x7f8aa80050b0 2023-07-06T03:45:05.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.383+0000 7f8abdd9b700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 0x7f8aa80048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.383+0000 7f8abe59c700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8aa8003a00 0x7f8aa8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.383+0000 7f8abed9d700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8aa80050b0 0x7f8aa800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.383+0000 7f8abdd9b700 1 -- 172.21.15.97:0/3569230068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8aa80050b0 msgr2=0x7f8aa800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:05.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.383+0000 7f8abdd9b700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8aa80050b0 0x7f8aa800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.383+0000 7f8abdd9b700 1 -- 172.21.15.97:0/3569230068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8aa8003a00 msgr2=0x7f8aa8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:05.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.383+0000 7f8abdd9b700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8aa8003a00 0x7f8aa8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.383+0000 7f8abed9d700 1 -- 172.21.15.97:0/3569230068 reap_dead start 2023-07-06T03:45:05.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.383+0000 7f8abdd9b700 1 -- 172.21.15.97:0/3569230068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa80035b0 con 0x7f8aa80043f0 2023-07-06T03:45:05.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.383+0000 7f8abe59c700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8aa8003a00 0x7f8aa8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:05.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.383+0000 7f8abdd9b700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 0x7f8aa80048b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40018d0 tx=0x7f8aa8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515693f2d0bd7746 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:05.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.384+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40052d0 con 0x7f8aa80043f0 2023-07-06T03:45:05.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.384+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab4005430 con 0x7f8aa80043f0 2023-07-06T03:45:05.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.384+0000 7f8abd59a700 1 -- 172.21.15.97:0/3569230068 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40038a0 con 0x7f8aa80043f0 2023-07-06T03:45:05.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.384+0000 7f8ac49ff700 1 -- 172.21.15.97:0/3569230068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 msgr2=0x7f8aa80048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.384+0000 7f8ac49ff700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 0x7f8aa80048b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40018d0 tx=0x7f8aa8005880 comp rx=0 tx=0).stop 2023-07-06T03:45:05.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.384+0000 7f8ac49ff700 1 -- 172.21.15.97:0/3569230068 shutdown_connections 2023-07-06T03:45:05.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.384+0000 7f8ac49ff700 1 --2- 172.21.15.97:0/3569230068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 0x7f8aa80048b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.384+0000 7f8ac49ff700 1 -- 172.21.15.97:0/3569230068 >> 172.21.15.97:0/3569230068 conn(0x7f8ac0071790 msgr2=0x7f8ac0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:05.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.384+0000 7f8ac49ff700 1 -- 172.21.15.97:0/3569230068 shutdown_connections 2023-07-06T03:45:05.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.384+0000 7f8ac49ff700 1 -- 172.21.15.97:0/3569230068 wait complete. 2023-07-06T03:45:05.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.386+0000 7f8ac49ff700 1 Processor -- start 2023-07-06T03:45:05.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.386+0000 7f8ac49ff700 1 -- start start 2023-07-06T03:45:05.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.387+0000 7f8ac49ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 0x7f8ac0146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.387+0000 7f8ac49ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ac01f4af0 0x7f8ac00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.387+0000 7f8ac49ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ac00c8a40 0x7f8ac00c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.387+0000 7f8ac49ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00c93f0 con 0x7f8aa80043f0 2023-07-06T03:45:05.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.387+0000 7f8ac49ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00d0c40 con 0x7f8ac01f4af0 2023-07-06T03:45:05.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.387+0000 7f8ac49ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac01470a0 con 0x7f8ac00c8a40 2023-07-06T03:45:05.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.387+0000 7f8abe59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 0x7f8ac0146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.387+0000 7f8abe59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 0x7f8ac0146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40016/0 (socket says 172.21.15.97:40016) 2023-07-06T03:45:05.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.387+0000 7f8abe59c700 1 -- 172.21.15.97:0/2545327594 learned_addr learned my addr 172.21.15.97:0/2545327594 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.387+0000 7f8abed9d700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ac00c8a40 0x7f8ac00c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.387+0000 7f8abdd9b700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ac01f4af0 0x7f8ac00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.388+0000 7f8abe59c700 1 -- 172.21.15.97:0/2545327594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ac00c8a40 msgr2=0x7f8ac00c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.388+0000 7f8abe59c700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ac00c8a40 0x7f8ac00c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.388+0000 7f8abe59c700 1 -- 172.21.15.97:0/2545327594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ac01f4af0 msgr2=0x7f8ac00cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.388+0000 7f8abe59c700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ac01f4af0 0x7f8ac00cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.388+0000 7f8abe59c700 1 -- 172.21.15.97:0/2545327594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac00d0e90 con 0x7f8aa80043f0 2023-07-06T03:45:05.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.388+0000 7f8abe59c700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 0x7f8ac0146b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00017a0 tx=0x7f8ab0007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:05.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.388+0000 7f8aaeffd700 1 -- 172.21.15.97:0/2545327594 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0011580 con 0x7f8aa80043f0 2023-07-06T03:45:05.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.388+0000 7f8aaeffd700 1 -- 172.21.15.97:0/2545327594 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab00116e0 con 0x7f8aa80043f0 2023-07-06T03:45:05.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.388+0000 7f8aaeffd700 1 -- 172.21.15.97:0/2545327594 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0011a50 con 0x7f8aa80043f0 2023-07-06T03:45:05.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.388+0000 7f8ac49ff700 1 -- 172.21.15.97:0/2545327594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac00d1170 con 0x7f8aa80043f0 2023-07-06T03:45:05.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.388+0000 7f8ac49ff700 1 -- 172.21.15.97:0/2545327594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac00d1730 con 0x7f8aa80043f0 2023-07-06T03:45:05.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.389+0000 7f8ac49ff700 1 -- 172.21.15.97:0/2545327594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac004e340 con 0x7f8aa80043f0 2023-07-06T03:45:05.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.390+0000 7f8aaeffd700 1 -- 172.21.15.97:0/2545327594 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8ab0011bb0 con 0x7f8aa80043f0 2023-07-06T03:45:05.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.391+0000 7f8aaeffd700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8aa006ace0 0x7f8aa006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.391+0000 7f8aaeffd700 1 -- 172.21.15.97:0/2545327594 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8ab0088f10 con 0x7f8aa80043f0 2023-07-06T03:45:05.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.391+0000 7f8abdd9b700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8aa006ace0 0x7f8aa006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.392+0000 7f8abdd9b700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8aa006ace0 0x7f8aa006d190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0149d10 tx=0x7f8ab4005000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:05.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.395+0000 7f8aaeffd700 1 -- 172.21.15.97:0/2545327594 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8ab0054320 con 0x7f8aa80043f0 2023-07-06T03:45:05.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.590+0000 7f8ac49ff700 1 -- 172.21.15.97:0/2545327594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ac00b4f40 con 0x7f8aa80043f0 2023-07-06T03:45:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.591+0000 7f8aaeffd700 1 -- 172.21.15.97:0/2545327594 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14675 (secure 0 0 0) 0x7f8ab0007da0 con 0x7f8aa80043f0 2023-07-06T03:45:05.594 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:05.594 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:05.119732+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:05.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 -- 172.21.15.97:0/2545327594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8aa006ace0 msgr2=0x7f8aa006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8aa006ace0 0x7f8aa006d190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0149d10 tx=0x7f8ab4005000 comp rx=0 tx=0).stop 2023-07-06T03:45:05.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 -- 172.21.15.97:0/2545327594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 msgr2=0x7f8ac0146b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 0x7f8ac0146b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00017a0 tx=0x7f8ab0007570 comp rx=0 tx=0).stop 2023-07-06T03:45:05.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 -- 172.21.15.97:0/2545327594 shutdown_connections 2023-07-06T03:45:05.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8aa006ace0 0x7f8aa006d190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ac00c8a40 0x7f8ac00c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa80043f0 0x7f8ac0146b60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 --2- 172.21.15.97:0/2545327594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ac01f4af0 0x7f8ac00cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 -- 172.21.15.97:0/2545327594 >> 172.21.15.97:0/2545327594 conn(0x7f8ac0071790 msgr2=0x7f8ac00b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:05.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 -- 172.21.15.97:0/2545327594 shutdown_connections 2023-07-06T03:45:05.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.594+0000 7f8ac49ff700 1 -- 172.21.15.97:0/2545327594 wait complete. 2023-07-06T03:45:05.615 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:45:05.616 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:45:05.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.812+0000 7f83795ff700 1 Processor -- start 2023-07-06T03:45:05.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.812+0000 7f83795ff700 1 -- start start 2023-07-06T03:45:05.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.813+0000 7f83795ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f837415c610 0x7f83740b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.813+0000 7f83795ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f837406b360 0x7f83740b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.813+0000 7f83795ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83740cff90 con 0x7f83740d2210 2023-07-06T03:45:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.813+0000 7f83795ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83740d0780 con 0x7f837415c610 2023-07-06T03:45:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.813+0000 7f83795ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837415c9e0 con 0x7f837406b360 2023-07-06T03:45:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.813+0000 7f8372ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f837415c610 0x7f83740b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.813+0000 7f8372ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f837415c610 0x7f83740b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40028/0 (socket says 172.21.15.97:40028) 2023-07-06T03:45:05.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.813+0000 7f8372ffd700 1 -- 172.21.15.97:0/3962232804 learned_addr learned my addr 172.21.15.97:0/3962232804 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:05.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.814+0000 7f83727fc700 1 --2- 172.21.15.97:0/3962232804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f837406b360 0x7f83740b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.814+0000 7f8372ffd700 1 -- 172.21.15.97:0/3962232804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f837406b360 msgr2=0x7f83740b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:05.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.814+0000 7f8372ffd700 1 --2- 172.21.15.97:0/3962232804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f837406b360 0x7f83740b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.814+0000 7f8372ffd700 1 -- 172.21.15.97:0/3962232804 >> v1:172.21.15.146:6789/0 conn(0x7f83740d2210 legacy=0x7f83740d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.814+0000 7f8372ffd700 1 -- 172.21.15.97:0/3962232804 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83740b5650 con 0x7f837415c610 2023-07-06T03:45:05.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.814+0000 7f83727fc700 1 --2- 172.21.15.97:0/3962232804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f837406b360 0x7f83740b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:05.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.814+0000 7f8372ffd700 1 --2- 172.21.15.97:0/3962232804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f837415c610 0x7f83740b47b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8364009920 tx=0x7f8364009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3d18a96b4997d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:05.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.814+0000 7f8371ffb700 1 -- 172.21.15.97:0/3962232804 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836400ba20 con 0x7f837415c610 2023-07-06T03:45:05.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.814+0000 7f8371ffb700 1 -- 172.21.15.97:0/3962232804 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836400bb80 con 0x7f837415c610 2023-07-06T03:45:05.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.814+0000 7f8371ffb700 1 -- 172.21.15.97:0/3962232804 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364009e70 con 0x7f837415c610 2023-07-06T03:45:05.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.815+0000 7f83795ff700 1 -- 172.21.15.97:0/3962232804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f837415c610 msgr2=0x7f83740b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.815+0000 7f83795ff700 1 --2- 172.21.15.97:0/3962232804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f837415c610 0x7f83740b47b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8364009920 tx=0x7f8364009c30 comp rx=0 tx=0).stop 2023-07-06T03:45:05.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.815+0000 7f83795ff700 1 -- 172.21.15.97:0/3962232804 shutdown_connections 2023-07-06T03:45:05.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.815+0000 7f83795ff700 1 --2- 172.21.15.97:0/3962232804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f837415c610 0x7f83740b47b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.815+0000 7f83795ff700 1 --2- 172.21.15.97:0/3962232804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f837406b360 0x7f83740b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.815+0000 7f83795ff700 1 -- 172.21.15.97:0/3962232804 >> 172.21.15.97:0/3962232804 conn(0x7f837406a6c0 msgr2=0x7f837406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:05.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.815+0000 7f83795ff700 1 -- 172.21.15.97:0/3962232804 shutdown_connections 2023-07-06T03:45:05.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.815+0000 7f83795ff700 1 -- 172.21.15.97:0/3962232804 wait complete. 2023-07-06T03:45:05.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.817+0000 7f83795ff700 1 Processor -- start 2023-07-06T03:45:05.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.818+0000 7f83795ff700 1 -- start start 2023-07-06T03:45:05.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.819+0000 7f83795ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f837406b360 0x7f83740ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.819+0000 7f83795ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83740d2210 0x7f837414f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.819+0000 7f83795ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f837415c610 0x7f83740cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.819+0000 7f83795ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740cbf20 con 0x7f83740d2210 2023-07-06T03:45:05.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.819+0000 7f83795ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740cad50 con 0x7f837415c610 2023-07-06T03:45:05.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.819+0000 7f83795ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83740cafc0 con 0x7f837406b360 2023-07-06T03:45:05.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.819+0000 7f83727fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83740d2210 0x7f837414f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.819+0000 7f83727fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83740d2210 0x7f837414f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40038/0 (socket says 172.21.15.97:40038) 2023-07-06T03:45:05.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.819+0000 7f83727fc700 1 -- 172.21.15.97:0/2456871858 learned_addr learned my addr 172.21.15.97:0/2456871858 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:05.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.819+0000 7f8372ffd700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f837406b360 0x7f83740ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.819+0000 7f83737fe700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f837415c610 0x7f83740cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.820+0000 7f83727fc700 1 -- 172.21.15.97:0/2456871858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f837406b360 msgr2=0x7f83740ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.820+0000 7f83727fc700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f837406b360 0x7f83740ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.820+0000 7f83727fc700 1 -- 172.21.15.97:0/2456871858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f837415c610 msgr2=0x7f83740cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:05.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.820+0000 7f83727fc700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f837415c610 0x7f83740cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:05.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.820+0000 7f83727fc700 1 -- 172.21.15.97:0/2456871858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8364009550 con 0x7f83740d2210 2023-07-06T03:45:05.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.820+0000 7f83727fc700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83740d2210 0x7f837414f5b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f836800c7f0 tx=0x7f836800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:05.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.820+0000 7f835b7fe700 1 -- 172.21.15.97:0/2456871858 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368011800 con 0x7f83740d2210 2023-07-06T03:45:05.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.820+0000 7f835b7fe700 1 -- 172.21.15.97:0/2456871858 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8368011960 con 0x7f83740d2210 2023-07-06T03:45:05.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.820+0000 7f835b7fe700 1 -- 172.21.15.97:0/2456871858 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368011cf0 con 0x7f83740d2210 2023-07-06T03:45:05.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.820+0000 7f83795ff700 1 -- 172.21.15.97:0/2456871858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83740cb2a0 con 0x7f83740d2210 2023-07-06T03:45:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.820+0000 7f83795ff700 1 -- 172.21.15.97:0/2456871858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83740cb860 con 0x7f83740d2210 2023-07-06T03:45:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.821+0000 7f83795ff700 1 -- 172.21.15.97:0/2456871858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837404e340 con 0x7f83740d2210 2023-07-06T03:45:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.822+0000 7f835b7fe700 1 -- 172.21.15.97:0/2456871858 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8368018360 con 0x7f83740d2210 2023-07-06T03:45:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.823+0000 7f835b7fe700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836006ace0 0x7f836006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.823+0000 7f835b7fe700 1 -- 172.21.15.97:0/2456871858 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8368089050 con 0x7f83740d2210 2023-07-06T03:45:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.823+0000 7f8372ffd700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836006ace0 0x7f836006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.824+0000 7f8372ffd700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836006ace0 0x7f836006d190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8364000c00 tx=0x7f8364008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:05.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:05.827+0000 7f835b7fe700 1 -- 172.21.15.97:0/2456871858 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8368054450 con 0x7f83740d2210 2023-07-06T03:45:06.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.024+0000 7f83795ff700 1 -- 172.21.15.97:0/2456871858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f837415a400 con 0x7f83740d2210 2023-07-06T03:45:06.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.025+0000 7f835b7fe700 1 -- 172.21.15.97:0/2456871858 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8368057a70 con 0x7f83740d2210 2023-07-06T03:45:06.028 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:45:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.027+0000 7f83795ff700 1 -- 172.21.15.97:0/2456871858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836006ace0 msgr2=0x7f836006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.027+0000 7f83795ff700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836006ace0 0x7f836006d190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8364000c00 tx=0x7f8364008030 comp rx=0 tx=0).stop 2023-07-06T03:45:06.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.027+0000 7f83795ff700 1 -- 172.21.15.97:0/2456871858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83740d2210 msgr2=0x7f837414f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.027+0000 7f83795ff700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83740d2210 0x7f837414f5b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f836800c7f0 tx=0x7f836800cb00 comp rx=0 tx=0).stop 2023-07-06T03:45:06.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.027+0000 7f83795ff700 1 -- 172.21.15.97:0/2456871858 shutdown_connections 2023-07-06T03:45:06.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.027+0000 7f83795ff700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836006ace0 0x7f836006d190 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.027+0000 7f83795ff700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f837415c610 0x7f83740cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.027+0000 7f83795ff700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f837406b360 0x7f83740ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.028+0000 7f83795ff700 1 --2- 172.21.15.97:0/2456871858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83740d2210 0x7f837414f5b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.028+0000 7f83795ff700 1 -- 172.21.15.97:0/2456871858 >> 172.21.15.97:0/2456871858 conn(0x7f837406a6c0 msgr2=0x7f83740b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:06.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.028+0000 7f83795ff700 1 -- 172.21.15.97:0/2456871858 shutdown_connections 2023-07-06T03:45:06.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.028+0000 7f83795ff700 1 -- 172.21.15.97:0/2456871858 wait complete. 2023-07-06T03:45:06.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:45:06.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.245+0000 7f5d7ebff700 1 Processor -- start 2023-07-06T03:45:06.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.245+0000 7f5d7ebff700 1 -- start start 2023-07-06T03:45:06.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.246+0000 7f5d7ebff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5d780d15a0 0x7f5d780cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:06.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.246+0000 7f5d7ebff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780d2d30 con 0x7f5d780b4a40 2023-07-06T03:45:06.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.246+0000 7f5d7ebff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780b6520 con 0x7f5d780cfed0 2023-07-06T03:45:06.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.246+0000 7f5d7ebff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d780d1a10 con 0x7f5d780d15a0 2023-07-06T03:45:06.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.246+0000 7f5d7c99b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5d780b4a40 0x7f5d780b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57732/0 (socket says 172.21.15.97:57732) 2023-07-06T03:45:06.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.246+0000 7f5d7c99b700 1 -- 172.21.15.97:0/890283253 learned_addr learned my addr 172.21.15.97:0/890283253 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:06.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.247+0000 7f5d77fff700 1 --2- 172.21.15.97:0/890283253 >> v2:172.21.15.146:3300/0 conn(0x7f5d780d15a0 0x7f5d780cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:06.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.247+0000 7f5d777fe700 1 -- 172.21.15.97:0/890283253 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776709566 0 0) 0x7f5d780d2d30 con 0x7f5d780b4a40 2023-07-06T03:45:06.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.247+0000 7f5d777fe700 1 -- 172.21.15.97:0/890283253 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d680035b0 con 0x7f5d780b4a40 2023-07-06T03:45:06.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.247+0000 7f5d777fe700 1 -- 172.21.15.97:0/890283253 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803155523 0 0) 0x7f5d780b6520 con 0x7f5d780cfed0 2023-07-06T03:45:06.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.247+0000 7f5d777fe700 1 -- 172.21.15.97:0/890283253 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d780d2d30 con 0x7f5d780cfed0 2023-07-06T03:45:06.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.247+0000 7f5d777fe700 1 -- 172.21.15.97:0/890283253 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2271470797 0 0) 0x7f5d680035b0 con 0x7f5d780b4a40 2023-07-06T03:45:06.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.247+0000 7f5d77fff700 1 -- 172.21.15.97:0/890283253 >> v1:172.21.15.146:6789/0 conn(0x7f5d780cfed0 legacy=0x7f5d780b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.247+0000 7f5d77fff700 1 -- 172.21.15.97:0/890283253 >> v1:172.21.15.97:6789/0 conn(0x7f5d780b4a40 legacy=0x7f5d780b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.247+0000 7f5d77fff700 1 -- 172.21.15.97:0/890283253 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d780cf210 con 0x7f5d780d15a0 2023-07-06T03:45:06.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.248+0000 7f5d77fff700 1 --2- 172.21.15.97:0/890283253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780d15a0 0x7f5d780cf990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009840 tx=0x7f5d64009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b9b395fa93b7631 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:06.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.248+0000 7f5d777fe700 1 -- 172.21.15.97:0/890283253 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c001670 con 0x7f5d780d15a0 2023-07-06T03:45:06.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.248+0000 7f5d777fe700 1 -- 172.21.15.97:0/890283253 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d64004030 con 0x7f5d780d15a0 2023-07-06T03:45:06.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.248+0000 7f5d777fe700 1 -- 172.21.15.97:0/890283253 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d640043c0 con 0x7f5d780d15a0 2023-07-06T03:45:06.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.248+0000 7f5d7ebff700 1 -- 172.21.15.97:0/890283253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780d15a0 msgr2=0x7f5d780cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.248+0000 7f5d7ebff700 1 --2- 172.21.15.97:0/890283253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780d15a0 0x7f5d780cf990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009840 tx=0x7f5d64009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:06.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.248+0000 7f5d7ebff700 1 -- 172.21.15.97:0/890283253 shutdown_connections 2023-07-06T03:45:06.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.248+0000 7f5d7ebff700 1 --2- 172.21.15.97:0/890283253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780d15a0 0x7f5d780cf990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.248+0000 7f5d7ebff700 1 -- 172.21.15.97:0/890283253 >> 172.21.15.97:0/890283253 conn(0x7f5d7806a900 msgr2=0x7f5d7806acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:06.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.248+0000 7f5d7ebff700 1 -- 172.21.15.97:0/890283253 shutdown_connections 2023-07-06T03:45:06.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.249+0000 7f5d7ebff700 1 -- 172.21.15.97:0/890283253 wait complete. 2023-07-06T03:45:06.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.250+0000 7f5d7ebff700 1 Processor -- start 2023-07-06T03:45:06.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.250+0000 7f5d7ebff700 1 -- start start 2023-07-06T03:45:06.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.251+0000 7f5d7ebff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d780b4a40 0x7f5d780c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:06.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.251+0000 7f5d7ebff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780cfed0 0x7f5d781f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:06.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.251+0000 7f5d7ebff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d780d15a0 0x7f5d780c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:06.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.251+0000 7f5d7ebff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780c7030 con 0x7f5d780b4a40 2023-07-06T03:45:06.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.251+0000 7f5d7ebff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d781f23f0 con 0x7f5d780cfed0 2023-07-06T03:45:06.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.251+0000 7f5d7ebff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d781f26d0 con 0x7f5d780d15a0 2023-07-06T03:45:06.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.251+0000 7f5d7c99b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d780b4a40 0x7f5d780c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:06.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.251+0000 7f5d7c99b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d780b4a40 0x7f5d780c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40048/0 (socket says 172.21.15.97:40048) 2023-07-06T03:45:06.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.251+0000 7f5d7c99b700 1 -- 172.21.15.97:0/3686577970 learned_addr learned my addr 172.21.15.97:0/3686577970 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:06.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d77fff700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780cfed0 0x7f5d781f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:06.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d7d19c700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d780d15a0 0x7f5d780c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:06.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d7c99b700 1 -- 172.21.15.97:0/3686577970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d780d15a0 msgr2=0x7f5d780c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d7c99b700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d780d15a0 0x7f5d780c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d7c99b700 1 -- 172.21.15.97:0/3686577970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780cfed0 msgr2=0x7f5d781f1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d7c99b700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780cfed0 0x7f5d781f1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d7c99b700 1 -- 172.21.15.97:0/3686577970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d640094a0 con 0x7f5d780b4a40 2023-07-06T03:45:06.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d7c99b700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d780b4a40 0x7f5d780c6af0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00c520 tx=0x7f5d6c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:06.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d757fa700 1 -- 172.21.15.97:0/3686577970 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c004c80 con 0x7f5d780b4a40 2023-07-06T03:45:06.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d757fa700 1 -- 172.21.15.97:0/3686577970 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c004de0 con 0x7f5d780b4a40 2023-07-06T03:45:06.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d757fa700 1 -- 172.21.15.97:0/3686577970 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c007310 con 0x7f5d780b4a40 2023-07-06T03:45:06.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.252+0000 7f5d7ebff700 1 -- 172.21.15.97:0/3686577970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d781f29b0 con 0x7f5d780b4a40 2023-07-06T03:45:06.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.253+0000 7f5d7ebff700 1 -- 172.21.15.97:0/3686577970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d781f2f70 con 0x7f5d780b4a40 2023-07-06T03:45:06.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.253+0000 7f5d7ebff700 1 -- 172.21.15.97:0/3686577970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d781f5670 con 0x7f5d780b4a40 2023-07-06T03:45:06.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.254+0000 7f5d757fa700 1 -- 172.21.15.97:0/3686577970 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d6c007470 con 0x7f5d780b4a40 2023-07-06T03:45:06.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.255+0000 7f5d757fa700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5c06ace0 0x7f5d5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:06.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.256+0000 7f5d757fa700 1 -- 172.21.15.97:0/3686577970 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5d6c088ea0 con 0x7f5d780b4a40 2023-07-06T03:45:06.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.256+0000 7f5d77fff700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5c06ace0 0x7f5d5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:06.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.256+0000 7f5d77fff700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5c06ace0 0x7f5d5c06d190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d640090b0 tx=0x7f5d64008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:06.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.259+0000 7f5d757fa700 1 -- 172.21.15.97:0/3686577970 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d781f5670 con 0x7f5d780b4a40 2023-07-06T03:45:06.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.461+0000 7f5d7ebff700 1 -- 172.21.15.97:0/3686577970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d7804e340 con 0x7f5d780b4a40 2023-07-06T03:45:06.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.462+0000 7f5d757fa700 1 -- 172.21.15.97:0/3686577970 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d6c054200 con 0x7f5d780b4a40 2023-07-06T03:45:06.465 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:45:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 -- 172.21.15.97:0/3686577970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5c06ace0 msgr2=0x7f5d5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5c06ace0 0x7f5d5c06d190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d640090b0 tx=0x7f5d64008030 comp rx=0 tx=0).stop 2023-07-06T03:45:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 -- 172.21.15.97:0/3686577970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d780b4a40 msgr2=0x7f5d780c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d780b4a40 0x7f5d780c6af0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00c520 tx=0x7f5d6c00c830 comp rx=0 tx=0).stop 2023-07-06T03:45:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 -- 172.21.15.97:0/3686577970 shutdown_connections 2023-07-06T03:45:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5c06ace0 0x7f5d5c06d190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d780d15a0 0x7f5d780c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d780b4a40 0x7f5d780c6af0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 --2- 172.21.15.97:0/3686577970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d780cfed0 0x7f5d781f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 -- 172.21.15.97:0/3686577970 >> 172.21.15.97:0/3686577970 conn(0x7f5d7806a900 msgr2=0x7f5d7806c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:06.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 -- 172.21.15.97:0/3686577970 shutdown_connections 2023-07-06T03:45:06.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.464+0000 7f5d7ebff700 1 -- 172.21.15.97:0/3686577970 wait complete. 2023-07-06T03:45:06.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:45:06.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.677+0000 7f49c4dff700 1 Processor -- start 2023-07-06T03:45:06.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.677+0000 7f49c4dff700 1 -- start start 2023-07-06T03:45:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.678+0000 7f49c4dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49c015c520 0x7f49c00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.678+0000 7f49c4dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f49c0160180 0x7f49c01605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.678+0000 7f49c4dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00cdae0 con 0x7f49c0160b30 2023-07-06T03:45:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.678+0000 7f49c4dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00cd8d0 con 0x7f49c015c520 2023-07-06T03:45:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.678+0000 7f49c4dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00b6640 con 0x7f49c0160180 2023-07-06T03:45:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.678+0000 7f49be59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49c015c520 0x7f49c00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.678+0000 7f49be59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49c015c520 0x7f49c00cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40052/0 (socket says 172.21.15.97:40052) 2023-07-06T03:45:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.678+0000 7f49bed9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f49c0160b30 0x7f49c00cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57744/0 (socket says 172.21.15.97:57744) 2023-07-06T03:45:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.678+0000 7f49be59c700 1 -- 172.21.15.97:0/599812842 learned_addr learned my addr 172.21.15.97:0/599812842 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.679+0000 7f49be59c700 1 -- 172.21.15.97:0/599812842 >> v2:172.21.15.146:3300/0 conn(0x7f49c0160180 msgr2=0x7f49c01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.679+0000 7f49be59c700 1 --2- 172.21.15.97:0/599812842 >> v2:172.21.15.146:3300/0 conn(0x7f49c0160180 0x7f49c01605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.679+0000 7f49be59c700 1 -- 172.21.15.97:0/599812842 >> v1:172.21.15.97:6789/0 conn(0x7f49c0160b30 legacy=0x7f49c00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.679+0000 7f49be59c700 1 -- 172.21.15.97:0/599812842 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00b6920 con 0x7f49c015c520 2023-07-06T03:45:06.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.679+0000 7f49bdd9b700 1 -- 172.21.15.97:0/599812842 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858257169 0 0) 0x7f49c00cdae0 con 0x7f49c0160b30 2023-07-06T03:45:06.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.679+0000 7f49be59c700 1 --2- 172.21.15.97:0/599812842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c015c520 0x7f49c00cd390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49c00b5930 tx=0x7f49b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb01a3350500ad0b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:06.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.679+0000 7f49bdd9b700 1 -- 172.21.15.97:0/599812842 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b000b870 con 0x7f49c015c520 2023-07-06T03:45:06.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.679+0000 7f49bdd9b700 1 -- 172.21.15.97:0/599812842 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b000b9d0 con 0x7f49c015c520 2023-07-06T03:45:06.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.679+0000 7f49bdd9b700 1 -- 172.21.15.97:0/599812842 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b000bd60 con 0x7f49c015c520 2023-07-06T03:45:06.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.680+0000 7f49c4dff700 1 -- 172.21.15.97:0/599812842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c015c520 msgr2=0x7f49c00cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.680+0000 7f49c4dff700 1 --2- 172.21.15.97:0/599812842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c015c520 0x7f49c00cd390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49c00b5930 tx=0x7f49b0009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.680+0000 7f49c4dff700 1 -- 172.21.15.97:0/599812842 shutdown_connections 2023-07-06T03:45:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.680+0000 7f49c4dff700 1 --2- 172.21.15.97:0/599812842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c015c520 0x7f49c00cd390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.680+0000 7f49c4dff700 1 --2- 172.21.15.97:0/599812842 >> v2:172.21.15.146:3300/0 conn(0x7f49c0160180 0x7f49c01605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.680+0000 7f49c4dff700 1 -- 172.21.15.97:0/599812842 >> 172.21.15.97:0/599812842 conn(0x7f49c0071790 msgr2=0x7f49c0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.680+0000 7f49c4dff700 1 -- 172.21.15.97:0/599812842 shutdown_connections 2023-07-06T03:45:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.680+0000 7f49c4dff700 1 -- 172.21.15.97:0/599812842 wait complete. 2023-07-06T03:45:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.681+0000 7f49c4dff700 1 Processor -- start 2023-07-06T03:45:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.682+0000 7f49c4dff700 1 -- start start 2023-07-06T03:45:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49c4dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49c015c520 0x7f49c00d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49c4dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c0160180 0x7f49c015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49c4dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49c0160b30 0x7f49c015b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49c4dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c015aa80 con 0x7f49c0160180 2023-07-06T03:45:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49c4dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c01f2ed0 con 0x7f49c015c520 2023-07-06T03:45:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49c4dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c015a430 con 0x7f49c0160b30 2023-07-06T03:45:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c0160180 0x7f49c015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c0160180 0x7f49c015af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40060/0 (socket says 172.21.15.97:40060) 2023-07-06T03:45:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49b7fff700 1 -- 172.21.15.97:0/1729096691 learned_addr learned my addr 172.21.15.97:0/1729096691 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49be59c700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49c015c520 0x7f49c00d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49bed9d700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49c0160b30 0x7f49c015b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49b7fff700 1 -- 172.21.15.97:0/1729096691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49c0160b30 msgr2=0x7f49c015b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49b7fff700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49c0160b30 0x7f49c015b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49b7fff700 1 -- 172.21.15.97:0/1729096691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49c015c520 msgr2=0x7f49c00d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49b7fff700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49c015c520 0x7f49c00d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49b7fff700 1 -- 172.21.15.97:0/1729096691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b0009550 con 0x7f49c0160180 2023-07-06T03:45:06.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.683+0000 7f49be59c700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49c015c520 0x7f49c00d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:06.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.684+0000 7f49b7fff700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c0160180 0x7f49c015af10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f49a800b530 tx=0x7f49a800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:06.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.684+0000 7f49b6ffd700 1 -- 172.21.15.97:0/1729096691 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8011830 con 0x7f49c0160180 2023-07-06T03:45:06.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.684+0000 7f49b6ffd700 1 -- 172.21.15.97:0/1729096691 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a80119b0 con 0x7f49c0160180 2023-07-06T03:45:06.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.684+0000 7f49c4dff700 1 -- 172.21.15.97:0/1729096691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c015a710 con 0x7f49c0160180 2023-07-06T03:45:06.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.684+0000 7f49b6ffd700 1 -- 172.21.15.97:0/1729096691 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8011d60 con 0x7f49c0160180 2023-07-06T03:45:06.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.684+0000 7f49c4dff700 1 -- 172.21.15.97:0/1729096691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c01f2b10 con 0x7f49c0160180 2023-07-06T03:45:06.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.685+0000 7f49c4dff700 1 -- 172.21.15.97:0/1729096691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c015a290 con 0x7f49c0160180 2023-07-06T03:45:06.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.686+0000 7f49b6ffd700 1 -- 172.21.15.97:0/1729096691 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f49a801a400 con 0x7f49c0160180 2023-07-06T03:45:06.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.686+0000 7f49b6ffd700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49a006ace0 0x7f49a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:06.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.687+0000 7f49b6ffd700 1 -- 172.21.15.97:0/1729096691 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f49a80892b0 con 0x7f49c0160180 2023-07-06T03:45:06.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.687+0000 7f49be59c700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49a006ace0 0x7f49a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:06.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.687+0000 7f49be59c700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49a006ace0 0x7f49a006d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f49b0014040 tx=0x7f49b0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:06.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.691+0000 7f49b6ffd700 1 -- 172.21.15.97:0/1729096691 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f49c015a290 con 0x7f49c0160180 2023-07-06T03:45:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.928+0000 7f49c4dff700 1 -- 172.21.15.97:0/1729096691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49c015a290 con 0x7f49c0160180 2023-07-06T03:45:06.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.928+0000 7f49b6ffd700 1 -- 172.21.15.97:0/1729096691 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7f49a8015de0 con 0x7f49c0160180 2023-07-06T03:45:06.931 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:06.932 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/4255451313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4255451313},{"type":"v1","addr":"172.21.15.97:6836","nonce":4255451313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/370322164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":370322164},{"type":"v1","addr":"172.21.15.97:6837","nonce":370322164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3764820847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3764820847},{"type":"v1","addr":"172.21.15.146:6835","nonce":3764820847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/186544257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":186544257},{"type":"v1","addr":"172.21.15.146:6833","nonce":186544257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-06T03:45:06.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.930+0000 7f49c4dff700 1 -- 172.21.15.97:0/1729096691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49a006ace0 msgr2=0x7f49a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.930+0000 7f49c4dff700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49a006ace0 0x7f49a006d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f49b0014040 tx=0x7f49b0008030 comp rx=0 tx=0).stop 2023-07-06T03:45:06.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.930+0000 7f49c4dff700 1 -- 172.21.15.97:0/1729096691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c0160180 msgr2=0x7f49c015af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:06.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.930+0000 7f49c4dff700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c0160180 0x7f49c015af10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f49a800b530 tx=0x7f49a800b8f0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.930+0000 7f49c4dff700 1 -- 172.21.15.97:0/1729096691 shutdown_connections 2023-07-06T03:45:06.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.930+0000 7f49c4dff700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49a006ace0 0x7f49a006d190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.930+0000 7f49c4dff700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49c0160b30 0x7f49c015b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.931+0000 7f49c4dff700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49c015c520 0x7f49c00d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.931+0000 7f49c4dff700 1 --2- 172.21.15.97:0/1729096691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c0160180 0x7f49c015af10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:06.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.931+0000 7f49c4dff700 1 -- 172.21.15.97:0/1729096691 >> 172.21.15.97:0/1729096691 conn(0x7f49c0071790 msgr2=0x7f49c0071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:06.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.931+0000 7f49c4dff700 1 -- 172.21.15.97:0/1729096691 shutdown_connections 2023-07-06T03:45:06.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:06.931+0000 7f49c4dff700 1 -- 172.21.15.97:0/1729096691 wait complete. 2023-07-06T03:45:06.936 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2023-07-06T03:45:06.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:07.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.144+0000 7fc71a3ff700 1 Processor -- start 2023-07-06T03:45:07.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.144+0000 7fc71a3ff700 1 -- start start 2023-07-06T03:45:07.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.145+0000 7fc71a3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc71415c520 0x7fc7140cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:07.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.146+0000 7fc71a3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7140cdae0 con 0x7fc714160180 2023-07-06T03:45:07.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.146+0000 7fc71a3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7140b6640 con 0x7fc714160d00 2023-07-06T03:45:07.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.146+0000 7fc71a3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7140cd8d0 con 0x7fc71415c520 2023-07-06T03:45:07.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.146+0000 7fc713fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc71415c520 0x7fc7140cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:07.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.146+0000 7fc7137fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc714160180 0x7fc7141605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57746/0 (socket says 172.21.15.97:57746) 2023-07-06T03:45:07.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.146+0000 7fc7137fe700 1 -- 172.21.15.97:0/367328239 learned_addr learned my addr 172.21.15.97:0/367328239 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:07.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.146+0000 7fc713fff700 1 -- 172.21.15.97:0/367328239 >> v1:172.21.15.146:6789/0 conn(0x7fc714160d00 legacy=0x7fc7140b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.146+0000 7fc713fff700 1 -- 172.21.15.97:0/367328239 >> v1:172.21.15.97:6789/0 conn(0x7fc714160180 legacy=0x7fc7141605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.146+0000 7fc713fff700 1 -- 172.21.15.97:0/367328239 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7140b6920 con 0x7fc71415c520 2023-07-06T03:45:07.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.146+0000 7fc712ffd700 1 -- 172.21.15.97:0/367328239 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114783654 0 0) 0x7fc7140cdae0 con 0x7fc714160180 2023-07-06T03:45:07.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.146+0000 7fc713fff700 1 --2- 172.21.15.97:0/367328239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc71415c520 0x7fc7140cd390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc704009840 tx=0x7fc704009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1dc21038ebff9f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:07.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.147+0000 7fc712ffd700 1 -- 172.21.15.97:0/367328239 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704011830 con 0x7fc71415c520 2023-07-06T03:45:07.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.147+0000 7fc712ffd700 1 -- 172.21.15.97:0/367328239 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc704011990 con 0x7fc71415c520 2023-07-06T03:45:07.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.147+0000 7fc712ffd700 1 -- 172.21.15.97:0/367328239 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704011d20 con 0x7fc71415c520 2023-07-06T03:45:07.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.147+0000 7fc71a3ff700 1 -- 172.21.15.97:0/367328239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc71415c520 msgr2=0x7fc7140cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.147+0000 7fc71a3ff700 1 --2- 172.21.15.97:0/367328239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc71415c520 0x7fc7140cd390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc704009840 tx=0x7fc704009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:07.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.147+0000 7fc71a3ff700 1 -- 172.21.15.97:0/367328239 shutdown_connections 2023-07-06T03:45:07.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.147+0000 7fc71a3ff700 1 --2- 172.21.15.97:0/367328239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc71415c520 0x7fc7140cd390 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.147+0000 7fc71a3ff700 1 -- 172.21.15.97:0/367328239 >> 172.21.15.97:0/367328239 conn(0x7fc714071790 msgr2=0x7fc714071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:07.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.147+0000 7fc71a3ff700 1 -- 172.21.15.97:0/367328239 shutdown_connections 2023-07-06T03:45:07.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.147+0000 7fc71a3ff700 1 -- 172.21.15.97:0/367328239 wait complete. 2023-07-06T03:45:07.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.149+0000 7fc71a3ff700 1 Processor -- start 2023-07-06T03:45:07.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.149+0000 7fc71a3ff700 1 -- start start 2023-07-06T03:45:07.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.150+0000 7fc71a3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc71415c520 0x7fc7141505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:07.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.150+0000 7fc71a3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc714160180 0x7fc71414d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:07.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.150+0000 7fc71a3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc714160d00 0x7fc7141f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:07.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.150+0000 7fc71a3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71415d060 con 0x7fc71415c520 2023-07-06T03:45:07.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.150+0000 7fc71a3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71415d340 con 0x7fc714160180 2023-07-06T03:45:07.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.150+0000 7fc71a3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71415d620 con 0x7fc714160d00 2023-07-06T03:45:07.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.150+0000 7fc713fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc71415c520 0x7fc7141505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:07.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.150+0000 7fc713fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc71415c520 0x7fc7141505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40076/0 (socket says 172.21.15.97:40076) 2023-07-06T03:45:07.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.150+0000 7fc713fff700 1 -- 172.21.15.97:0/4015189962 learned_addr learned my addr 172.21.15.97:0/4015189962 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:07.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc7137fe700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc714160180 0x7fc71414d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:07.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc71899c700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc714160d00 0x7fc7141f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:07.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc713fff700 1 -- 172.21.15.97:0/4015189962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc714160d00 msgr2=0x7fc7141f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc713fff700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc714160d00 0x7fc7141f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc713fff700 1 -- 172.21.15.97:0/4015189962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc714160180 msgr2=0x7fc71414d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc713fff700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc714160180 0x7fc71414d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc713fff700 1 -- 172.21.15.97:0/4015189962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7040094a0 con 0x7fc71415c520 2023-07-06T03:45:07.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc713fff700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc71415c520 0x7fc7141505e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc704009b90 tx=0x7fc704004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:07.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc710ff9700 1 -- 172.21.15.97:0/4015189962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7040036a0 con 0x7fc71415c520 2023-07-06T03:45:07.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc710ff9700 1 -- 172.21.15.97:0/4015189962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70401e070 con 0x7fc71415c520 2023-07-06T03:45:07.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc71a3ff700 1 -- 172.21.15.97:0/4015189962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7141f5830 con 0x7fc71415c520 2023-07-06T03:45:07.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc710ff9700 1 -- 172.21.15.97:0/4015189962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704003a00 con 0x7fc71415c520 2023-07-06T03:45:07.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.151+0000 7fc71a3ff700 1 -- 172.21.15.97:0/4015189962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc714150260 con 0x7fc71415c520 2023-07-06T03:45:07.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.152+0000 7fc71a3ff700 1 -- 172.21.15.97:0/4015189962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7141f5a40 con 0x7fc71415c520 2023-07-06T03:45:07.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.153+0000 7fc710ff9700 1 -- 172.21.15.97:0/4015189962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc704003b60 con 0x7fc71415c520 2023-07-06T03:45:07.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.154+0000 7fc710ff9700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc70006ace0 0x7fc70006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:07.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.154+0000 7fc710ff9700 1 -- 172.21.15.97:0/4015189962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc70408a280 con 0x7fc71415c520 2023-07-06T03:45:07.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.154+0000 7fc7137fe700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc70006ace0 0x7fc70006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:07.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.155+0000 7fc7137fe700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc70006ace0 0x7fc70006d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc708007fd0 tx=0x7fc708007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:07.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.159+0000 7fc710ff9700 1 -- 172.21.15.97:0/4015189962 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc704055710 con 0x7fc71415c520 2023-07-06T03:45:07.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.356+0000 7fc71a3ff700 1 -- 172.21.15.97:0/4015189962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc71404e340 con 0x7fc71415c520 2023-07-06T03:45:07.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.357+0000 7fc710ff9700 1 -- 172.21.15.97:0/4015189962 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14662 (secure 0 0 0) 0x7fc704058d30 con 0x7fc71415c520 2023-07-06T03:45:07.360 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:07.360 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":19,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:06.124806+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:07.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 -- 172.21.15.97:0/4015189962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc70006ace0 msgr2=0x7fc70006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc70006ace0 0x7fc70006d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc708007fd0 tx=0x7fc708007c20 comp rx=0 tx=0).stop 2023-07-06T03:45:07.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 -- 172.21.15.97:0/4015189962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc71415c520 msgr2=0x7fc7141505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc71415c520 0x7fc7141505e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc704009b90 tx=0x7fc704004750 comp rx=0 tx=0).stop 2023-07-06T03:45:07.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 -- 172.21.15.97:0/4015189962 shutdown_connections 2023-07-06T03:45:07.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc70006ace0 0x7fc70006d190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc714160d00 0x7fc7141f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc71415c520 0x7fc7141505e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 --2- 172.21.15.97:0/4015189962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc714160180 0x7fc71414d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 -- 172.21.15.97:0/4015189962 >> 172.21.15.97:0/4015189962 conn(0x7fc714071790 msgr2=0x7fc7140b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 -- 172.21.15.97:0/4015189962 shutdown_connections 2023-07-06T03:45:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.360+0000 7fc71a3ff700 1 -- 172.21.15.97:0/4015189962 wait complete. 2023-07-06T03:45:07.381 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:45:07.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:45:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.575+0000 7f79603ff700 1 Processor -- start 2023-07-06T03:45:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.575+0000 7f79603ff700 1 -- start start 2023-07-06T03:45:07.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.576+0000 7f79603ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f795815c610 0x7f79580b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:07.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.576+0000 7f79603ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79580d2210 0x7f79580d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:07.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.576+0000 7f79603ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79580cff90 con 0x7f795806b360 2023-07-06T03:45:07.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.576+0000 7f79603ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79580d0780 con 0x7f79580d2210 2023-07-06T03:45:07.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.576+0000 7f79603ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f795815c9e0 con 0x7f795815c610 2023-07-06T03:45:07.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795e99c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79580d2210 0x7f79580d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:07.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795d99a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f795806b360 0x7f79580b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57760/0 (socket says 172.21.15.97:57760) 2023-07-06T03:45:07.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795d99a700 1 -- 172.21.15.97:0/2882068488 learned_addr learned my addr 172.21.15.97:0/2882068488 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:07.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795e19b700 1 --2- 172.21.15.97:0/2882068488 >> v2:172.21.15.146:3300/0 conn(0x7f795815c610 0x7f79580b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795e99c700 1 -- 172.21.15.97:0/2882068488 >> v2:172.21.15.146:3300/0 conn(0x7f795815c610 msgr2=0x7f79580b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795e99c700 1 --2- 172.21.15.97:0/2882068488 >> v2:172.21.15.146:3300/0 conn(0x7f795815c610 0x7f79580b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795e99c700 1 -- 172.21.15.97:0/2882068488 >> v1:172.21.15.97:6789/0 conn(0x7f795806b360 legacy=0x7f79580b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795e99c700 1 -- 172.21.15.97:0/2882068488 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79580b5650 con 0x7f79580d2210 2023-07-06T03:45:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795d199700 1 -- 172.21.15.97:0/2882068488 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591511876 0 0) 0x7f79580cff90 con 0x7f795806b360 2023-07-06T03:45:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795e99c700 1 --2- 172.21.15.97:0/2882068488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79580d2210 0x7f79580d1310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f795808b000 tx=0x7f795000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d4b1abb06a2f0b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795d199700 1 -- 172.21.15.97:0/2882068488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795000e070 con 0x7f79580d2210 2023-07-06T03:45:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795d199700 1 -- 172.21.15.97:0/2882068488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795000bae0 con 0x7f79580d2210 2023-07-06T03:45:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.577+0000 7f795d199700 1 -- 172.21.15.97:0/2882068488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795000be90 con 0x7f79580d2210 2023-07-06T03:45:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.578+0000 7f79603ff700 1 -- 172.21.15.97:0/2882068488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79580d2210 msgr2=0x7f79580d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.578+0000 7f79603ff700 1 --2- 172.21.15.97:0/2882068488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79580d2210 0x7f79580d1310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f795808b000 tx=0x7f795000b330 comp rx=0 tx=0).stop 2023-07-06T03:45:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.578+0000 7f79603ff700 1 -- 172.21.15.97:0/2882068488 shutdown_connections 2023-07-06T03:45:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.578+0000 7f79603ff700 1 --2- 172.21.15.97:0/2882068488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79580d2210 0x7f79580d1310 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.578+0000 7f79603ff700 1 --2- 172.21.15.97:0/2882068488 >> v2:172.21.15.146:3300/0 conn(0x7f795815c610 0x7f79580b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.578+0000 7f79603ff700 1 -- 172.21.15.97:0/2882068488 >> 172.21.15.97:0/2882068488 conn(0x7f795806a6c0 msgr2=0x7f795806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.578+0000 7f79603ff700 1 -- 172.21.15.97:0/2882068488 shutdown_connections 2023-07-06T03:45:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.578+0000 7f79603ff700 1 -- 172.21.15.97:0/2882068488 wait complete. 2023-07-06T03:45:07.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.580+0000 7f79603ff700 1 Processor -- start 2023-07-06T03:45:07.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.580+0000 7f79603ff700 1 -- start start 2023-07-06T03:45:07.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f79603ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f795806b360 0x7f795815f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:07.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f79603ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79580d2210 0x7f79580b6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:07.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f79603ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f795815c610 0x7f79581f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:07.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f79603ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79580c9fa0 con 0x7f795806b360 2023-07-06T03:45:07.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f79603ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79580ca280 con 0x7f79580d2210 2023-07-06T03:45:07.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f79603ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79580ca560 con 0x7f795815c610 2023-07-06T03:45:07.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f795e19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f795806b360 0x7f795815f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:07.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f795e19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f795806b360 0x7f795815f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40106/0 (socket says 172.21.15.97:40106) 2023-07-06T03:45:07.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f795e19b700 1 -- 172.21.15.97:0/277696324 learned_addr learned my addr 172.21.15.97:0/277696324 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:07.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f795e99c700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f795815c610 0x7f79581f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:07.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f795d99a700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79580d2210 0x7f79580b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:07.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f795e19b700 1 -- 172.21.15.97:0/277696324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f795815c610 msgr2=0x7f79581f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f795e19b700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f795815c610 0x7f79581f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f795e19b700 1 -- 172.21.15.97:0/277696324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79580d2210 msgr2=0x7f79580b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f795e19b700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79580d2210 0x7f79580b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f795e19b700 1 -- 172.21.15.97:0/277696324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7950009a70 con 0x7f795806b360 2023-07-06T03:45:07.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.581+0000 7f795d99a700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79580d2210 0x7f79580b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:07.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.582+0000 7f795e19b700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f795806b360 0x7f795815f3a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f795400e870 tx=0x7f795400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:07.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.582+0000 7f794effd700 1 -- 172.21.15.97:0/277696324 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954009590 con 0x7f795806b360 2023-07-06T03:45:07.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.582+0000 7f794effd700 1 -- 172.21.15.97:0/277696324 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795400ee70 con 0x7f795806b360 2023-07-06T03:45:07.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.582+0000 7f794effd700 1 -- 172.21.15.97:0/277696324 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795400ca50 con 0x7f795806b360 2023-07-06T03:45:07.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.582+0000 7f79603ff700 1 -- 172.21.15.97:0/277696324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79580ca840 con 0x7f795806b360 2023-07-06T03:45:07.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.582+0000 7f79603ff700 1 -- 172.21.15.97:0/277696324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795815f8e0 con 0x7f795806b360 2023-07-06T03:45:07.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.583+0000 7f79603ff700 1 -- 172.21.15.97:0/277696324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79580cae90 con 0x7f795806b360 2023-07-06T03:45:07.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.584+0000 7f794effd700 1 -- 172.21.15.97:0/277696324 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f795400cbb0 con 0x7f795806b360 2023-07-06T03:45:07.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.585+0000 7f794effd700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f793c06ace0 0x7f793c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:07.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.585+0000 7f794effd700 1 -- 172.21.15.97:0/277696324 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7954089070 con 0x7f795806b360 2023-07-06T03:45:07.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.585+0000 7f795d99a700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f793c06ace0 0x7f793c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:07.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.586+0000 7f795d99a700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f793c06ace0 0x7f793c06d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7948000f60 tx=0x7f7948009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:07.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.589+0000 7f794effd700 1 -- 172.21.15.97:0/277696324 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7954057f90 con 0x7f795806b360 2023-07-06T03:45:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.781+0000 7f79603ff700 1 -- 172.21.15.97:0/277696324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f795804e340 con 0x7f795806b360 2023-07-06T03:45:07.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.781+0000 7f794effd700 1 -- 172.21.15.97:0/277696324 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79540210a0 con 0x7f795806b360 2023-07-06T03:45:07.785 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:45:07.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 -- 172.21.15.97:0/277696324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f793c06ace0 msgr2=0x7f793c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f793c06ace0 0x7f793c06d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7948000f60 tx=0x7f7948009040 comp rx=0 tx=0).stop 2023-07-06T03:45:07.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 -- 172.21.15.97:0/277696324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f795806b360 msgr2=0x7f795815f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:07.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f795806b360 0x7f795815f3a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f795400e870 tx=0x7f795400ec30 comp rx=0 tx=0).stop 2023-07-06T03:45:07.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 -- 172.21.15.97:0/277696324 shutdown_connections 2023-07-06T03:45:07.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f793c06ace0 0x7f793c06d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f795815c610 0x7f79581f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f795806b360 0x7f795815f3a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 --2- 172.21.15.97:0/277696324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79580d2210 0x7f79580b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:07.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 -- 172.21.15.97:0/277696324 >> 172.21.15.97:0/277696324 conn(0x7f795806a6c0 msgr2=0x7f79580b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:07.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 -- 172.21.15.97:0/277696324 shutdown_connections 2023-07-06T03:45:07.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.784+0000 7f79603ff700 1 -- 172.21.15.97:0/277696324 wait complete. 2023-07-06T03:45:07.804 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:45:08.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.999+0000 7facb9dff700 1 Processor -- start 2023-07-06T03:45:08.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:07.999+0000 7facb9dff700 1 -- start start 2023-07-06T03:45:08.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facb9dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7facb406b360 0x7facb40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facb9dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb40d2210 0x7facb40d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facb9dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb40cff90 con 0x7facb415c610 2023-07-06T03:45:08.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facb9dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb40d0780 con 0x7facb406b360 2023-07-06T03:45:08.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facb9dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb415c9e0 con 0x7facb40d2210 2023-07-06T03:45:08.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facabfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7facb406b360 0x7facb40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facabfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7facb406b360 0x7facb40b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40114/0 (socket says 172.21.15.97:40114) 2023-07-06T03:45:08.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facabfff700 1 -- 172.21.15.97:0/217953462 learned_addr learned my addr 172.21.15.97:0/217953462 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facb3fff700 1 --2- 172.21.15.97:0/217953462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb40d2210 0x7facb40d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facabfff700 1 -- 172.21.15.97:0/217953462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb40d2210 msgr2=0x7facb40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facabfff700 1 --2- 172.21.15.97:0/217953462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb40d2210 0x7facb40d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facabfff700 1 -- 172.21.15.97:0/217953462 >> v1:172.21.15.97:6789/0 conn(0x7facb415c610 legacy=0x7facb40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.001+0000 7facabfff700 1 -- 172.21.15.97:0/217953462 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb40b5650 con 0x7facb406b360 2023-07-06T03:45:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facabfff700 1 --2- 172.21.15.97:0/217953462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb406b360 0x7facb40b4ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0098f0 tx=0x7fac9c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c52f3a342a786e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:08.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facb2ffd700 1 -- 172.21.15.97:0/217953462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00b8a0 con 0x7facb406b360 2023-07-06T03:45:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facb2ffd700 1 -- 172.21.15.97:0/217953462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c00ba00 con 0x7facb406b360 2023-07-06T03:45:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facb2ffd700 1 -- 172.21.15.97:0/217953462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00bd90 con 0x7facb406b360 2023-07-06T03:45:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facb9dff700 1 -- 172.21.15.97:0/217953462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb406b360 msgr2=0x7facb40b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facb9dff700 1 --2- 172.21.15.97:0/217953462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb406b360 0x7facb40b4ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0098f0 tx=0x7fac9c009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facb9dff700 1 -- 172.21.15.97:0/217953462 shutdown_connections 2023-07-06T03:45:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facb9dff700 1 --2- 172.21.15.97:0/217953462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb40d2210 0x7facb40d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facb9dff700 1 --2- 172.21.15.97:0/217953462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb406b360 0x7facb40b4ec0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facb9dff700 1 -- 172.21.15.97:0/217953462 >> 172.21.15.97:0/217953462 conn(0x7facb406a6c0 msgr2=0x7facb406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:08.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facb9dff700 1 -- 172.21.15.97:0/217953462 shutdown_connections 2023-07-06T03:45:08.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.002+0000 7facb9dff700 1 -- 172.21.15.97:0/217953462 wait complete. 2023-07-06T03:45:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.004+0000 7facb9dff700 1 Processor -- start 2023-07-06T03:45:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.004+0000 7facb9dff700 1 -- start start 2023-07-06T03:45:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.005+0000 7facb9dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb406b360 0x7facb4153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.005+0000 7facb9dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb40d2210 0x7facb40ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.005+0000 7facb9dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb415c610 0x7facb415e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.005+0000 7facb9dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb415ea60 con 0x7facb40d2210 2023-07-06T03:45:08.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.005+0000 7facb9dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb40c9e00 con 0x7facb415c610 2023-07-06T03:45:08.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.005+0000 7facb9dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb415ece0 con 0x7facb406b360 2023-07-06T03:45:08.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.005+0000 7facabfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb40d2210 0x7facb40ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.005+0000 7facabfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb40d2210 0x7facb40ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40126/0 (socket says 172.21.15.97:40126) 2023-07-06T03:45:08.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.005+0000 7facabfff700 1 -- 172.21.15.97:0/2410394938 learned_addr learned my addr 172.21.15.97:0/2410394938 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:08.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facb37fe700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb406b360 0x7facb4153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facb3fff700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb415c610 0x7facb415e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facabfff700 1 -- 172.21.15.97:0/2410394938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb406b360 msgr2=0x7facb4153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facabfff700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb406b360 0x7facb4153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facabfff700 1 -- 172.21.15.97:0/2410394938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb415c610 msgr2=0x7facb415e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facabfff700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb415c610 0x7facb415e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facabfff700 1 -- 172.21.15.97:0/2410394938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c009550 con 0x7facb40d2210 2023-07-06T03:45:08.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facabfff700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb40d2210 0x7facb40ca8a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fac9c000c00 tx=0x7fac9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:08.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facb0ff9700 1 -- 172.21.15.97:0/2410394938 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c004480 con 0x7facb40d2210 2023-07-06T03:45:08.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facb0ff9700 1 -- 172.21.15.97:0/2410394938 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c01f070 con 0x7facb40d2210 2023-07-06T03:45:08.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facb9dff700 1 -- 172.21.15.97:0/2410394938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb415efc0 con 0x7facb40d2210 2023-07-06T03:45:08.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facb0ff9700 1 -- 172.21.15.97:0/2410394938 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0038c0 con 0x7facb40d2210 2023-07-06T03:45:08.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.006+0000 7facb9dff700 1 -- 172.21.15.97:0/2410394938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb40ca380 con 0x7facb40d2210 2023-07-06T03:45:08.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.007+0000 7facb9dff700 1 -- 172.21.15.97:0/2410394938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb40ca5a0 con 0x7facb40d2210 2023-07-06T03:45:08.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.008+0000 7facb0ff9700 1 -- 172.21.15.97:0/2410394938 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac9c021030 con 0x7facb40d2210 2023-07-06T03:45:08.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.009+0000 7facb0ff9700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac9406ace0 0x7fac9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.009+0000 7facb0ff9700 1 -- 172.21.15.97:0/2410394938 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fac9c08a710 con 0x7facb40d2210 2023-07-06T03:45:08.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.010+0000 7facb37fe700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac9406ace0 0x7fac9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.010+0000 7facb37fe700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac9406ace0 0x7fac9406d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7facb41f53d0 tx=0x7faca4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:08.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.014+0000 7facb0ff9700 1 -- 172.21.15.97:0/2410394938 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fac9c055c20 con 0x7facb40d2210 2023-07-06T03:45:08.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.209+0000 7facb9dff700 1 -- 172.21.15.97:0/2410394938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7facb404e340 con 0x7facb40d2210 2023-07-06T03:45:08.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.210+0000 7facb0ff9700 1 -- 172.21.15.97:0/2410394938 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fac9c059240 con 0x7facb40d2210 2023-07-06T03:45:08.213 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:45:08.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 -- 172.21.15.97:0/2410394938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac9406ace0 msgr2=0x7fac9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac9406ace0 0x7fac9406d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7facb41f53d0 tx=0x7faca4009040 comp rx=0 tx=0).stop 2023-07-06T03:45:08.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 -- 172.21.15.97:0/2410394938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb40d2210 msgr2=0x7facb40ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb40d2210 0x7facb40ca8a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fac9c000c00 tx=0x7fac9c004750 comp rx=0 tx=0).stop 2023-07-06T03:45:08.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 -- 172.21.15.97:0/2410394938 shutdown_connections 2023-07-06T03:45:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac9406ace0 0x7fac9406d190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facb415c610 0x7facb415e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7facb406b360 0x7facb4153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 --2- 172.21.15.97:0/2410394938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb40d2210 0x7facb40ca8a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 -- 172.21.15.97:0/2410394938 >> 172.21.15.97:0/2410394938 conn(0x7facb406a6c0 msgr2=0x7facb40d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 -- 172.21.15.97:0/2410394938 shutdown_connections 2023-07-06T03:45:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.212+0000 7facb9dff700 1 -- 172.21.15.97:0/2410394938 wait complete. 2023-07-06T03:45:08.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T03:45:08.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.425+0000 7f2d5b7ff700 1 Processor -- start 2023-07-06T03:45:08.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.426+0000 7f2d5b7ff700 1 -- start start 2023-07-06T03:45:08.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.427+0000 7f2d5b7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d540b8410 0x7f2d54167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.427+0000 7f2d5b7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d540cdae0 con 0x7f2d540cd390 2023-07-06T03:45:08.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.427+0000 7f2d5b7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d540b6640 con 0x7f2d5415c520 2023-07-06T03:45:08.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.427+0000 7f2d5b7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540b6920 con 0x7f2d540b8410 2023-07-06T03:45:08.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.428+0000 7f2d58d9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2d540cd390 0x7f2d54160890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57776/0 (socket says 172.21.15.97:57776) 2023-07-06T03:45:08.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.428+0000 7f2d58d9a700 1 -- 172.21.15.97:0/2454752933 learned_addr learned my addr 172.21.15.97:0/2454752933 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:08.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.428+0000 7f2d59d9c700 1 --2- 172.21.15.97:0/2454752933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d540b8410 0x7f2d54167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.428+0000 7f2d4bfff700 1 -- 172.21.15.97:0/2454752933 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 97793377 0 0) 0x7f2d540cdae0 con 0x7f2d540cd390 2023-07-06T03:45:08.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.428+0000 7f2d4bfff700 1 -- 172.21.15.97:0/2454752933 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d400035b0 con 0x7f2d540cd390 2023-07-06T03:45:08.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.428+0000 7f2d4bfff700 1 -- 172.21.15.97:0/2454752933 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934624316 0 0) 0x7f2d540b6640 con 0x7f2d5415c520 2023-07-06T03:45:08.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.428+0000 7f2d4bfff700 1 -- 172.21.15.97:0/2454752933 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d540cdae0 con 0x7f2d5415c520 2023-07-06T03:45:08.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.428+0000 7f2d59d9c700 1 -- 172.21.15.97:0/2454752933 >> v1:172.21.15.146:6789/0 conn(0x7f2d5415c520 legacy=0x7f2d54160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.428+0000 7f2d59d9c700 1 -- 172.21.15.97:0/2454752933 >> v1:172.21.15.97:6789/0 conn(0x7f2d540cd390 legacy=0x7f2d54160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.428+0000 7f2d59d9c700 1 -- 172.21.15.97:0/2454752933 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d540b6c00 con 0x7f2d540b8410 2023-07-06T03:45:08.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.428+0000 7f2d59d9c700 1 --2- 172.21.15.97:0/2454752933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d540b8410 0x7f2d54167510 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009dc0 tx=0x7f2d4c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c72d1261a372917a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:08.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.429+0000 7f2d4bfff700 1 -- 172.21.15.97:0/2454752933 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c00e070 con 0x7f2d540b8410 2023-07-06T03:45:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.429+0000 7f2d4bfff700 1 -- 172.21.15.97:0/2454752933 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c00bb20 con 0x7f2d540b8410 2023-07-06T03:45:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.429+0000 7f2d4bfff700 1 -- 172.21.15.97:0/2454752933 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c012430 con 0x7f2d540b8410 2023-07-06T03:45:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.429+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2454752933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d540b8410 msgr2=0x7f2d54167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.429+0000 7f2d5b7ff700 1 --2- 172.21.15.97:0/2454752933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d540b8410 0x7f2d54167510 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009dc0 tx=0x7f2d4c00b330 comp rx=0 tx=0).stop 2023-07-06T03:45:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.429+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2454752933 shutdown_connections 2023-07-06T03:45:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.429+0000 7f2d5b7ff700 1 --2- 172.21.15.97:0/2454752933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d540b8410 0x7f2d54167510 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.429+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2454752933 >> 172.21.15.97:0/2454752933 conn(0x7f2d54071790 msgr2=0x7f2d54071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.429+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2454752933 shutdown_connections 2023-07-06T03:45:08.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.429+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2454752933 wait complete. 2023-07-06T03:45:08.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.432+0000 7f2d5b7ff700 1 Processor -- start 2023-07-06T03:45:08.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.432+0000 7f2d5b7ff700 1 -- start start 2023-07-06T03:45:08.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.433+0000 7f2d5b7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d540b8410 0x7f2d540d2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.433+0000 7f2d5b7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d540cd390 0x7f2d5415db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.433+0000 7f2d5b7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d5415c520 0x7f2d5415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.433+0000 7f2d5b7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540ca2b0 con 0x7f2d540cd390 2023-07-06T03:45:08.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.433+0000 7f2d5b7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540d31e0 con 0x7f2d540b8410 2023-07-06T03:45:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.433+0000 7f2d5b7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540d2c20 con 0x7f2d5415c520 2023-07-06T03:45:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.433+0000 7f2d58d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d540cd390 0x7f2d5415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.433+0000 7f2d58d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d540cd390 0x7f2d5415db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40134/0 (socket says 172.21.15.97:40134) 2023-07-06T03:45:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.433+0000 7f2d58d9a700 1 -- 172.21.15.97:0/2824018049 learned_addr learned my addr 172.21.15.97:0/2824018049 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d5959b700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d540b8410 0x7f2d540d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d59d9c700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d5415c520 0x7f2d5415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d58d9a700 1 -- 172.21.15.97:0/2824018049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d5415c520 msgr2=0x7f2d5415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d58d9a700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d5415c520 0x7f2d5415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d58d9a700 1 -- 172.21.15.97:0/2824018049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d540b8410 msgr2=0x7f2d540d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d58d9a700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d540b8410 0x7f2d540d2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d58d9a700 1 -- 172.21.15.97:0/2824018049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c009a70 con 0x7f2d540cd390 2023-07-06T03:45:08.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d5959b700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d540b8410 0x7f2d540d2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:08.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d58d9a700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d540cd390 0x7f2d5415db70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400c700 tx=0x7f2d4400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:08.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d49ffb700 1 -- 172.21.15.97:0/2824018049 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d44005d70 con 0x7f2d540cd390 2023-07-06T03:45:08.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d49ffb700 1 -- 172.21.15.97:0/2824018049 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d440070d0 con 0x7f2d540cd390 2023-07-06T03:45:08.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2824018049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d540d2f00 con 0x7f2d540cd390 2023-07-06T03:45:08.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d49ffb700 1 -- 172.21.15.97:0/2824018049 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d44007460 con 0x7f2d540cd390 2023-07-06T03:45:08.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.434+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2824018049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5414d3a0 con 0x7f2d540cd390 2023-07-06T03:45:08.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.435+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2824018049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d540ca460 con 0x7f2d540cd390 2023-07-06T03:45:08.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.436+0000 7f2d49ffb700 1 -- 172.21.15.97:0/2824018049 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2d440075c0 con 0x7f2d540cd390 2023-07-06T03:45:08.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.437+0000 7f2d49ffb700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d3806ace0 0x7f2d3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.437+0000 7f2d49ffb700 1 -- 172.21.15.97:0/2824018049 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2d440887a0 con 0x7f2d540cd390 2023-07-06T03:45:08.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.437+0000 7f2d5959b700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d3806ace0 0x7f2d3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.438+0000 7f2d5959b700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d3806ace0 0x7f2d3806d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2d50007fd0 tx=0x7f2d50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:08.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.441+0000 7f2d49ffb700 1 -- 172.21.15.97:0/2824018049 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2d540ca460 con 0x7f2d540cd390 2023-07-06T03:45:08.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.642+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2824018049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2d5404e340 con 0x7f2d540cd390 2023-07-06T03:45:08.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.643+0000 7f2d49ffb700 1 -- 172.21.15.97:0/2824018049 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2d44053b10 con 0x7f2d540cd390 2023-07-06T03:45:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:08.646 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-07-06T03:45:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:45:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-07-06T03:45:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-07-06T03:45:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:45:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:45:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 2, 2023-07-06T03:45:08.647 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": "rbd" 2023-07-06T03:45:08.648 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:45:08.649 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-07-06T03:45:08.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2824018049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d3806ace0 msgr2=0x7f2d3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d3806ace0 0x7f2d3806d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2d50007fd0 tx=0x7f2d50009040 comp rx=0 tx=0).stop 2023-07-06T03:45:08.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2824018049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d540cd390 msgr2=0x7f2d5415db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d540cd390 0x7f2d5415db70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400c700 tx=0x7f2d4400cac0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2824018049 shutdown_connections 2023-07-06T03:45:08.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d3806ace0 0x7f2d3806d190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d5415c520 0x7f2d5415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d540b8410 0x7f2d540d2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 --2- 172.21.15.97:0/2824018049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d540cd390 0x7f2d5415db70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2824018049 >> 172.21.15.97:0/2824018049 conn(0x7f2d54071790 msgr2=0x7f2d54071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:08.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2824018049 shutdown_connections 2023-07-06T03:45:08.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.645+0000 7f2d5b7ff700 1 -- 172.21.15.97:0/2824018049 wait complete. 2023-07-06T03:45:08.666 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T03:45:08.666 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T03:45:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.861+0000 7f2c43bff700 1 Processor -- start 2023-07-06T03:45:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.862+0000 7f2c43bff700 1 -- start start 2023-07-06T03:45:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c43bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2c3c06b0c0 0x7f2c3c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c43bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2c3c0b4ca0 0x7f2c3c0d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c43bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3c0cd330 con 0x7f2c3c0b85f0 2023-07-06T03:45:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c43bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3c0b89c0 con 0x7f2c3c0b4ca0 2023-07-06T03:45:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c43bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3c06b510 con 0x7f2c3c06b0c0 2023-07-06T03:45:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c4219c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2c3c0b4ca0 0x7f2c3c0d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c4219c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2c3c0b4ca0 0x7f2c3c0d09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40150/0 (socket says 172.21.15.97:40150) 2023-07-06T03:45:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c4219c700 1 -- 172.21.15.97:0/2918113734 learned_addr learned my addr 172.21.15.97:0/2918113734 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c4119a700 1 --2- 172.21.15.97:0/2918113734 >> v2:172.21.15.146:3300/0 conn(0x7f2c3c06b0c0 0x7f2c3c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c4219c700 1 -- 172.21.15.97:0/2918113734 >> v2:172.21.15.146:3300/0 conn(0x7f2c3c06b0c0 msgr2=0x7f2c3c0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:08.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c4219c700 1 --2- 172.21.15.97:0/2918113734 >> v2:172.21.15.146:3300/0 conn(0x7f2c3c06b0c0 0x7f2c3c0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c4219c700 1 -- 172.21.15.97:0/2918113734 >> v1:172.21.15.146:6789/0 conn(0x7f2c3c0b85f0 legacy=0x7f2c3c0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c4219c700 1 -- 172.21.15.97:0/2918113734 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c0d02a0 con 0x7f2c3c0b4ca0 2023-07-06T03:45:08.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.863+0000 7f2c4119a700 1 --2- 172.21.15.97:0/2918113734 >> v2:172.21.15.146:3300/0 conn(0x7f2c3c06b0c0 0x7f2c3c0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:08.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.864+0000 7f2c4219c700 1 --2- 172.21.15.97:0/2918113734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c3c0b4ca0 0x7f2c3c0d09b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c08bf20 tx=0x7f2c3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5df605b40f9307 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:08.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.864+0000 7f2c40999700 1 -- 172.21.15.97:0/2918113734 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3400e070 con 0x7f2c3c0b4ca0 2023-07-06T03:45:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.864+0000 7f2c40999700 1 -- 172.21.15.97:0/2918113734 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3400b9c0 con 0x7f2c3c0b4ca0 2023-07-06T03:45:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.864+0000 7f2c40999700 1 -- 172.21.15.97:0/2918113734 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3400bd70 con 0x7f2c3c0b4ca0 2023-07-06T03:45:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.864+0000 7f2c43bff700 1 -- 172.21.15.97:0/2918113734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c3c0b4ca0 msgr2=0x7f2c3c0d09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.864+0000 7f2c43bff700 1 --2- 172.21.15.97:0/2918113734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c3c0b4ca0 0x7f2c3c0d09b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c08bf20 tx=0x7f2c3400b330 comp rx=0 tx=0).stop 2023-07-06T03:45:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.865+0000 7f2c43bff700 1 -- 172.21.15.97:0/2918113734 shutdown_connections 2023-07-06T03:45:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.865+0000 7f2c43bff700 1 --2- 172.21.15.97:0/2918113734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c3c0b4ca0 0x7f2c3c0d09b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.865+0000 7f2c43bff700 1 --2- 172.21.15.97:0/2918113734 >> v2:172.21.15.146:3300/0 conn(0x7f2c3c06b0c0 0x7f2c3c0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.865+0000 7f2c43bff700 1 -- 172.21.15.97:0/2918113734 >> 172.21.15.97:0/2918113734 conn(0x7f2c3c06a5b0 msgr2=0x7f2c3c06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.865+0000 7f2c43bff700 1 -- 172.21.15.97:0/2918113734 shutdown_connections 2023-07-06T03:45:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.865+0000 7f2c43bff700 1 -- 172.21.15.97:0/2918113734 wait complete. 2023-07-06T03:45:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.866+0000 7f2c43bff700 1 Processor -- start 2023-07-06T03:45:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.867+0000 7f2c43bff700 1 -- start start 2023-07-06T03:45:08.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.867+0000 7f2c43bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c3c06b0c0 0x7f2c3c159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.867+0000 7f2c43bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c3c0b4ca0 0x7f2c3c0b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.867+0000 7f2c43bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c3c0b85f0 0x7f2c3c0b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c43bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c159690 con 0x7f2c3c0b85f0 2023-07-06T03:45:08.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c43bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c15ca30 con 0x7f2c3c06b0c0 2023-07-06T03:45:08.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c43bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c15cc80 con 0x7f2c3c0b4ca0 2023-07-06T03:45:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c4219c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c3c0b85f0 0x7f2c3c0b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c4219c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c3c0b85f0 0x7f2c3c0b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40154/0 (socket says 172.21.15.97:40154) 2023-07-06T03:45:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c4219c700 1 -- 172.21.15.97:0/1437978340 learned_addr learned my addr 172.21.15.97:0/1437978340 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c4199b700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c3c06b0c0 0x7f2c3c159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c4119a700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c3c0b4ca0 0x7f2c3c0b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c4219c700 1 -- 172.21.15.97:0/1437978340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c3c0b4ca0 msgr2=0x7f2c3c0b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c4219c700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c3c0b4ca0 0x7f2c3c0b64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c4219c700 1 -- 172.21.15.97:0/1437978340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c3c06b0c0 msgr2=0x7f2c3c159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c4219c700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c3c06b0c0 0x7f2c3c159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c4219c700 1 -- 172.21.15.97:0/1437978340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c34009b30 con 0x7f2c3c0b85f0 2023-07-06T03:45:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.868+0000 7f2c4219c700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c3c0b85f0 0x7f2c3c0b6a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400b730 tx=0x7f2c34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.869+0000 7f2c327fc700 1 -- 172.21.15.97:0/1437978340 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34004870 con 0x7f2c3c0b85f0 2023-07-06T03:45:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.869+0000 7f2c327fc700 1 -- 172.21.15.97:0/1437978340 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c340092e0 con 0x7f2c3c0b85f0 2023-07-06T03:45:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.869+0000 7f2c327fc700 1 -- 172.21.15.97:0/1437978340 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3401c660 con 0x7f2c3c0b85f0 2023-07-06T03:45:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.869+0000 7f2c43bff700 1 -- 172.21.15.97:0/1437978340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3c15cf60 con 0x7f2c3c0b85f0 2023-07-06T03:45:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.869+0000 7f2c43bff700 1 -- 172.21.15.97:0/1437978340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c15d520 con 0x7f2c3c0b85f0 2023-07-06T03:45:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.870+0000 7f2c43bff700 1 -- 172.21.15.97:0/1437978340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3c04e340 con 0x7f2c3c0b85f0 2023-07-06T03:45:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.871+0000 7f2c327fc700 1 -- 172.21.15.97:0/1437978340 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2c3401c7c0 con 0x7f2c3c0b85f0 2023-07-06T03:45:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.872+0000 7f2c327fc700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c2006ace0 0x7f2c2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.872+0000 7f2c327fc700 1 -- 172.21.15.97:0/1437978340 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2c3408a970 con 0x7f2c3c0b85f0 2023-07-06T03:45:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.872+0000 7f2c4199b700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c2006ace0 0x7f2c2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.872+0000 7f2c4199b700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c2006ace0 0x7f2c2006d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c38007fd0 tx=0x7f2c38009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:08.876+0000 7f2c327fc700 1 -- 172.21.15.97:0/1437978340 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2c34055e00 con 0x7f2c3c0b85f0 2023-07-06T03:45:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.074+0000 7f2c43bff700 1 -- 172.21.15.97:0/1437978340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2c3c06abd0 con 0x7f2c3c0b85f0 2023-07-06T03:45:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.630+0000 7f2c327fc700 1 -- 172.21.15.97:0/1437978340 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2c34059420 con 0x7f2c3c0b85f0 2023-07-06T03:45:09.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.636+0000 7f2c43bff700 1 -- 172.21.15.97:0/1437978340 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c2006ace0 msgr2=0x7f2c2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:09.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.636+0000 7f2c43bff700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c2006ace0 0x7f2c2006d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c38007fd0 tx=0x7f2c38009040 comp rx=0 tx=0).stop 2023-07-06T03:45:09.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.636+0000 7f2c43bff700 1 -- 172.21.15.97:0/1437978340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c3c0b85f0 msgr2=0x7f2c3c0b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:09.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.636+0000 7f2c43bff700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c3c0b85f0 0x7f2c3c0b6a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400b730 tx=0x7f2c34004ad0 comp rx=0 tx=0).stop 2023-07-06T03:45:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.636+0000 7f2c43bff700 1 -- 172.21.15.97:0/1437978340 shutdown_connections 2023-07-06T03:45:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.636+0000 7f2c43bff700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c2006ace0 0x7f2c2006d190 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c38007fd0 tx=0x7f2c38009040 comp rx=0 tx=0).stop 2023-07-06T03:45:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.636+0000 7f2c43bff700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c3c0b85f0 0x7f2c3c0b6a00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.636+0000 7f2c43bff700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c3c06b0c0 0x7f2c3c159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.636+0000 7f2c43bff700 1 --2- 172.21.15.97:0/1437978340 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c3c0b4ca0 0x7f2c3c0b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.636+0000 7f2c43bff700 1 -- 172.21.15.97:0/1437978340 >> 172.21.15.97:0/1437978340 conn(0x7f2c3c06a5b0 msgr2=0x7f2c3c15c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.637+0000 7f2c43bff700 1 -- 172.21.15.97:0/1437978340 shutdown_connections 2023-07-06T03:45:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.637+0000 7f2c43bff700 1 -- 172.21.15.97:0/1437978340 wait complete. 2023-07-06T03:45:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-07-06T03:45:09.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T03:45:09.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.860+0000 7f55879ff700 1 Processor -- start 2023-07-06T03:45:09.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.861+0000 7f55879ff700 1 -- start start 2023-07-06T03:45:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.862+0000 7f55879ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f55800b4ea0 0x7f55800cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.862+0000 7f55879ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5580160a50 0x7f55800d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:09.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.862+0000 7f55879ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5580160eb0 con 0x7f55800b8590 2023-07-06T03:45:09.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.862+0000 7f55879ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55800d0c10 con 0x7f55800b4ea0 2023-07-06T03:45:09.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.862+0000 7f55879ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55800d08e0 con 0x7f5580160a50 2023-07-06T03:45:09.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.862+0000 7f558579b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f55800b8590 0x7f55800b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56016/0 (socket says 172.21.15.97:56016) 2023-07-06T03:45:09.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.862+0000 7f558579b700 1 -- 172.21.15.97:0/1423857980 learned_addr learned my addr 172.21.15.97:0/1423857980 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:09.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.862+0000 7f5584f9a700 1 --2- 172.21.15.97:0/1423857980 >> v2:172.21.15.146:3300/0 conn(0x7f55800b4ea0 0x7f55800cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:09.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.862+0000 7f5585f9c700 1 --2- 172.21.15.97:0/1423857980 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5580160a50 0x7f55800d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:09.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.862+0000 7f5577fff700 1 -- 172.21.15.97:0/1423857980 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079138283 0 0) 0x7f5580160eb0 con 0x7f55800b8590 2023-07-06T03:45:09.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.863+0000 7f5577fff700 1 -- 172.21.15.97:0/1423857980 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f556c0035b0 con 0x7f55800b8590 2023-07-06T03:45:09.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.863+0000 7f5584f9a700 1 -- 172.21.15.97:0/1423857980 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5580160a50 msgr2=0x7f55800d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:09.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.863+0000 7f5584f9a700 1 --2- 172.21.15.97:0/1423857980 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5580160a50 0x7f55800d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.863+0000 7f5584f9a700 1 -- 172.21.15.97:0/1423857980 >> v1:172.21.15.97:6789/0 conn(0x7f55800b8590 legacy=0x7f55800b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.863+0000 7f5577fff700 1 -- 172.21.15.97:0/1423857980 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2881370026 0 0) 0x7f556c0035b0 con 0x7f55800b8590 2023-07-06T03:45:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.863+0000 7f5584f9a700 1 -- 172.21.15.97:0/1423857980 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55800b2560 con 0x7f55800b4ea0 2023-07-06T03:45:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.863+0000 7f5585f9c700 1 --2- 172.21.15.97:0/1423857980 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5580160a50 0x7f55800d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:09.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.863+0000 7f5584f9a700 1 --2- 172.21.15.97:0/1423857980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55800b4ea0 0x7f55800cef60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5570009840 tx=0x7f5570009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=754812a5c861d4f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:09.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.863+0000 7f5577fff700 1 -- 172.21.15.97:0/1423857980 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c001670 con 0x7f55800b4ea0 2023-07-06T03:45:09.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.863+0000 7f5577fff700 1 -- 172.21.15.97:0/1423857980 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5570004030 con 0x7f55800b4ea0 2023-07-06T03:45:09.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.863+0000 7f5577fff700 1 -- 172.21.15.97:0/1423857980 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700043c0 con 0x7f55800b4ea0 2023-07-06T03:45:09.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.864+0000 7f55879ff700 1 -- 172.21.15.97:0/1423857980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55800b4ea0 msgr2=0x7f55800cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:09.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.864+0000 7f55879ff700 1 --2- 172.21.15.97:0/1423857980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55800b4ea0 0x7f55800cef60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5570009840 tx=0x7f5570009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:09.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.864+0000 7f55879ff700 1 -- 172.21.15.97:0/1423857980 shutdown_connections 2023-07-06T03:45:09.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.864+0000 7f55879ff700 1 --2- 172.21.15.97:0/1423857980 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5580160a50 0x7f55800d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:09.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.864+0000 7f55879ff700 1 --2- 172.21.15.97:0/1423857980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55800b4ea0 0x7f55800cef60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:09.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.864+0000 7f55879ff700 1 -- 172.21.15.97:0/1423857980 >> 172.21.15.97:0/1423857980 conn(0x7f558006b290 msgr2=0x7f55800cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:09.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.864+0000 7f55879ff700 1 -- 172.21.15.97:0/1423857980 shutdown_connections 2023-07-06T03:45:09.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.864+0000 7f55879ff700 1 -- 172.21.15.97:0/1423857980 wait complete. 2023-07-06T03:45:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.866+0000 7f55879ff700 1 Processor -- start 2023-07-06T03:45:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.866+0000 7f55879ff700 1 -- start start 2023-07-06T03:45:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f55879ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55800b4ea0 0x7f55800ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f55879ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55800b8590 0x7f55801e8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f55879ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5580160a50 0x7f55800c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f55879ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55801e8760 con 0x7f55800b8590 2023-07-06T03:45:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f55879ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55800cadd0 con 0x7f55800b4ea0 2023-07-06T03:45:09.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f55879ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55800cb0b0 con 0x7f5580160a50 2023-07-06T03:45:09.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f5584f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55800b8590 0x7f55801e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:09.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f5584f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55800b8590 0x7f55801e8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46764/0 (socket says 172.21.15.97:46764) 2023-07-06T03:45:09.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f5584f9a700 1 -- 172.21.15.97:0/390446796 learned_addr learned my addr 172.21.15.97:0/390446796 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:09.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f5584f9a700 1 -- 172.21.15.97:0/390446796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5580160a50 msgr2=0x7f55800c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:09.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f5584f9a700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5580160a50 0x7f55800c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:09.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f5584f9a700 1 -- 172.21.15.97:0/390446796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55800b4ea0 msgr2=0x7f55800ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:09.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f5584f9a700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55800b4ea0 0x7f55800ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:09.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.867+0000 7f5584f9a700 1 -- 172.21.15.97:0/390446796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5578009a70 con 0x7f55800b8590 2023-07-06T03:45:09.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.868+0000 7f5584f9a700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55800b8590 0x7f55801e8220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f557000a010 tx=0x7f5570004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:09.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.868+0000 7f5575ffb700 1 -- 172.21.15.97:0/390446796 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570003c40 con 0x7f55800b8590 2023-07-06T03:45:09.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.868+0000 7f5575ffb700 1 -- 172.21.15.97:0/390446796 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557001f070 con 0x7f55800b8590 2023-07-06T03:45:09.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.868+0000 7f5575ffb700 1 -- 172.21.15.97:0/390446796 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570011630 con 0x7f55800b8590 2023-07-06T03:45:09.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.868+0000 7f55879ff700 1 -- 172.21.15.97:0/390446796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55700094a0 con 0x7f55800b8590 2023-07-06T03:45:09.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.868+0000 7f55879ff700 1 -- 172.21.15.97:0/390446796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55800cb670 con 0x7f55800b8590 2023-07-06T03:45:09.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.869+0000 7f55879ff700 1 -- 172.21.15.97:0/390446796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55800d3ac0 con 0x7f55800b8590 2023-07-06T03:45:09.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.870+0000 7f5575ffb700 1 -- 172.21.15.97:0/390446796 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5570021030 con 0x7f55800b8590 2023-07-06T03:45:09.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.871+0000 7f5575ffb700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f556406ace0 0x7f556406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:09.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.871+0000 7f5575ffb700 1 -- 172.21.15.97:0/390446796 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f557008a610 con 0x7f55800b8590 2023-07-06T03:45:09.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.871+0000 7f558579b700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f556406ace0 0x7f556406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:09.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.872+0000 7f558579b700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f556406ace0 0x7f556406d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f557c0011e0 tx=0x7f557c0078a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:09.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:09.876+0000 7f5575ffb700 1 -- 172.21.15.97:0/390446796 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5570055890 con 0x7f55800b8590 2023-07-06T03:45:10.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.071+0000 7f55879ff700 1 -- 172.21.15.97:0/390446796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f558004e340 con 0x7f55800b8590 2023-07-06T03:45:10.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.633+0000 7f5575ffb700 1 -- 172.21.15.97:0/390446796 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5570058eb0 con 0x7f55800b8590 2023-07-06T03:45:10.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.640+0000 7f55879ff700 1 -- 172.21.15.97:0/390446796 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f556406ace0 msgr2=0x7f556406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:10.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.640+0000 7f55879ff700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f556406ace0 0x7f556406d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f557c0011e0 tx=0x7f557c0078a0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.640+0000 7f55879ff700 1 -- 172.21.15.97:0/390446796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55800b8590 msgr2=0x7f55801e8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:10.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.640+0000 7f55879ff700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55800b8590 0x7f55801e8220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f557000a010 tx=0x7f5570004750 comp rx=0 tx=0).stop 2023-07-06T03:45:10.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.641+0000 7f55879ff700 1 -- 172.21.15.97:0/390446796 shutdown_connections 2023-07-06T03:45:10.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.641+0000 7f55879ff700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f556406ace0 0x7f556406d190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.641+0000 7f55879ff700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5580160a50 0x7f55800c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.641+0000 7f55879ff700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55800b4ea0 0x7f55800ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.641+0000 7f55879ff700 1 --2- 172.21.15.97:0/390446796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55800b8590 0x7f55801e8220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.641+0000 7f55879ff700 1 -- 172.21.15.97:0/390446796 >> 172.21.15.97:0/390446796 conn(0x7f558006b290 msgr2=0x7f55800b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.641+0000 7f55879ff700 1 -- 172.21.15.97:0/390446796 shutdown_connections 2023-07-06T03:45:10.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.641+0000 7f55879ff700 1 -- 172.21.15.97:0/390446796 wait complete. 2023-07-06T03:45:10.656 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-07-06T03:45:10.673 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T03:45:10.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.867+0000 7f985ebff700 1 Processor -- start 2023-07-06T03:45:10.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.867+0000 7f985ebff700 1 -- start start 2023-07-06T03:45:10.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f985ebff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f985815c1c0 0x7f98580cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:10.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f985ebff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f985815c590 0x7f985815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:10.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f985ebff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98580719c0 0x7f985806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:10.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f985ebff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98580ccc30 con 0x7f985815c590 2023-07-06T03:45:10.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f985ebff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98580ccf10 con 0x7f985815c1c0 2023-07-06T03:45:10.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f985ebff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98580cd1f0 con 0x7f98580719c0 2023-07-06T03:45:10.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f9857fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f985815c590 0x7f985815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:10.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f9857fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f985815c590 0x7f985815d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46768/0 (socket says 172.21.15.97:46768) 2023-07-06T03:45:10.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f9857fff700 1 -- 172.21.15.97:0/102364263 learned_addr learned my addr 172.21.15.97:0/102364263 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f985c99b700 1 --2- 172.21.15.97:0/102364263 >> v2:172.21.15.146:3300/0 conn(0x7f985815c1c0 0x7f98580cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f9857fff700 1 -- 172.21.15.97:0/102364263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98580719c0 msgr2=0x7f985806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f985d19c700 1 --2- 172.21.15.97:0/102364263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98580719c0 0x7f985806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f9857fff700 1 --2- 172.21.15.97:0/102364263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98580719c0 0x7f985806a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f9857fff700 1 -- 172.21.15.97:0/102364263 >> v2:172.21.15.146:3300/0 conn(0x7f985815c1c0 msgr2=0x7f98580cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f9857fff700 1 --2- 172.21.15.97:0/102364263 >> v2:172.21.15.146:3300/0 conn(0x7f985815c1c0 0x7f98580cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f9857fff700 1 -- 172.21.15.97:0/102364263 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985815cb00 con 0x7f985815c590 2023-07-06T03:45:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f985c99b700 1 --2- 172.21.15.97:0/102364263 >> v2:172.21.15.146:3300/0 conn(0x7f985815c1c0 0x7f98580cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.869+0000 7f985d19c700 1 --2- 172.21.15.97:0/102364263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98580719c0 0x7f985806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.870+0000 7f9857fff700 1 --2- 172.21.15.97:0/102364263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f985815c590 0x7f985815d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9858072570 tx=0x7f9844009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f3a7c9c94e57ad server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.870+0000 7f98577fe700 1 -- 172.21.15.97:0/102364263 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984400b870 con 0x7f985815c590 2023-07-06T03:45:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.870+0000 7f98577fe700 1 -- 172.21.15.97:0/102364263 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984400b9d0 con 0x7f985815c590 2023-07-06T03:45:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.870+0000 7f98577fe700 1 -- 172.21.15.97:0/102364263 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984400bd60 con 0x7f985815c590 2023-07-06T03:45:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.870+0000 7f985ebff700 1 -- 172.21.15.97:0/102364263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f985815c590 msgr2=0x7f985815d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.870+0000 7f985ebff700 1 --2- 172.21.15.97:0/102364263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f985815c590 0x7f985815d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9858072570 tx=0x7f9844009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.870+0000 7f985ebff700 1 -- 172.21.15.97:0/102364263 shutdown_connections 2023-07-06T03:45:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.870+0000 7f985ebff700 1 --2- 172.21.15.97:0/102364263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98580719c0 0x7f985806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.870+0000 7f985ebff700 1 --2- 172.21.15.97:0/102364263 >> v2:172.21.15.146:3300/0 conn(0x7f985815c1c0 0x7f98580cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.870+0000 7f985ebff700 1 --2- 172.21.15.97:0/102364263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f985815c590 0x7f985815d300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.870+0000 7f985ebff700 1 -- 172.21.15.97:0/102364263 >> 172.21.15.97:0/102364263 conn(0x7f98580b1650 msgr2=0x7f98580b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.871+0000 7f985ebff700 1 -- 172.21.15.97:0/102364263 shutdown_connections 2023-07-06T03:45:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.871+0000 7f985ebff700 1 -- 172.21.15.97:0/102364263 wait complete. 2023-07-06T03:45:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.872+0000 7f985ebff700 1 Processor -- start 2023-07-06T03:45:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.873+0000 7f985ebff700 1 -- start start 2023-07-06T03:45:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.873+0000 7f985ebff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98580719c0 0x7f9858154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.873+0000 7f985ebff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f985815c1c0 0x7f9858158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.873+0000 7f985ebff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f985815c590 0x7f9858158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.873+0000 7f985ebff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858159390 con 0x7f98580719c0 2023-07-06T03:45:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.873+0000 7f985ebff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858159600 con 0x7f985815c590 2023-07-06T03:45:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.873+0000 7f985ebff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98581598e0 con 0x7f985815c1c0 2023-07-06T03:45:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f985c99b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98580719c0 0x7f9858154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f985c99b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98580719c0 0x7f9858154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46774/0 (socket says 172.21.15.97:46774) 2023-07-06T03:45:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f985c99b700 1 -- 172.21.15.97:0/2988739293 learned_addr learned my addr 172.21.15.97:0/2988739293 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f9857fff700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f985815c1c0 0x7f9858158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f985d19c700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f985815c590 0x7f9858158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f985c99b700 1 -- 172.21.15.97:0/2988739293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f985815c1c0 msgr2=0x7f9858158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f985c99b700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f985815c1c0 0x7f9858158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f985c99b700 1 -- 172.21.15.97:0/2988739293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f985815c590 msgr2=0x7f9858158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f985c99b700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f985815c590 0x7f9858158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f985c99b700 1 -- 172.21.15.97:0/2988739293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9844009550 con 0x7f98580719c0 2023-07-06T03:45:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f9857fff700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f985815c1c0 0x7f9858158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f985d19c700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f985815c590 0x7f9858158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.874+0000 7f985c99b700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98580719c0 0x7f9858154700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c750 tx=0x7f984c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.875+0000 7f98557fa700 1 -- 172.21.15.97:0/2988739293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c011800 con 0x7f98580719c0 2023-07-06T03:45:10.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.875+0000 7f98557fa700 1 -- 172.21.15.97:0/2988739293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c011960 con 0x7f98580719c0 2023-07-06T03:45:10.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.875+0000 7f98557fa700 1 -- 172.21.15.97:0/2988739293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c011cd0 con 0x7f98580719c0 2023-07-06T03:45:10.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.875+0000 7f985ebff700 1 -- 172.21.15.97:0/2988739293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9858159bc0 con 0x7f98580719c0 2023-07-06T03:45:10.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.875+0000 7f985ebff700 1 -- 172.21.15.97:0/2988739293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985815a180 con 0x7f98580719c0 2023-07-06T03:45:10.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.876+0000 7f985ebff700 1 -- 172.21.15.97:0/2988739293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985804e340 con 0x7f98580719c0 2023-07-06T03:45:10.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.877+0000 7f98557fa700 1 -- 172.21.15.97:0/2988739293 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f984c021350 con 0x7f98580719c0 2023-07-06T03:45:10.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.878+0000 7f98557fa700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f983c06ace0 0x7f983c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:10.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.878+0000 7f98557fa700 1 -- 172.21.15.97:0/2988739293 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f984c089410 con 0x7f98580719c0 2023-07-06T03:45:10.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.878+0000 7f9857fff700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f983c06ace0 0x7f983c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:10.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.879+0000 7f9857fff700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f983c06ace0 0x7f983c06d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9858072570 tx=0x7f9844008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:10.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:10.882+0000 7f98557fa700 1 -- 172.21.15.97:0/2988739293 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f984c057fa0 con 0x7f98580719c0 2023-07-06T03:45:11.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.124+0000 7f985ebff700 1 -- 172.21.15.97:0/2988739293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9858071290 con 0x7f98580719c0 2023-07-06T03:45:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.666+0000 7f98557fa700 1 -- 172.21.15.97:0/2988739293 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f984c057b30 con 0x7f98580719c0 2023-07-06T03:45:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 -- 172.21.15.97:0/2988739293 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f983c06ace0 msgr2=0x7f983c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f983c06ace0 0x7f983c06d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9858072570 tx=0x7f9844008030 comp rx=0 tx=0).stop 2023-07-06T03:45:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 -- 172.21.15.97:0/2988739293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98580719c0 msgr2=0x7f9858154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98580719c0 0x7f9858154700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c750 tx=0x7f984c00cb10 comp rx=0 tx=0).stop 2023-07-06T03:45:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 -- 172.21.15.97:0/2988739293 shutdown_connections 2023-07-06T03:45:11.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f983c06ace0 0x7f983c06d190 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9858072570 tx=0x7f9844008030 comp rx=0 tx=0).stop 2023-07-06T03:45:11.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f985815c590 0x7f9858158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:11.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98580719c0 0x7f9858154700 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:11.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 --2- 172.21.15.97:0/2988739293 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f985815c1c0 0x7f9858158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:11.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 -- 172.21.15.97:0/2988739293 >> 172.21.15.97:0/2988739293 conn(0x7f98580b1650 msgr2=0x7f98580c6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:11.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 -- 172.21.15.97:0/2988739293 shutdown_connections 2023-07-06T03:45:11.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.671+0000 7f985ebff700 1 -- 172.21.15.97:0/2988739293 wait complete. 2023-07-06T03:45:11.681 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 3 and data pool 4 2023-07-06T03:45:11.704 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:11.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.897+0000 7f741e9ff700 1 Processor -- start 2023-07-06T03:45:11.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.897+0000 7f741e9ff700 1 -- start start 2023-07-06T03:45:11.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.898+0000 7f741e9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f74180b8410 0x7f7418167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:11.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.898+0000 7f741e9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74180cdae0 con 0x7f741815c520 2023-07-06T03:45:11.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.898+0000 7f741e9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74180b6640 con 0x7f74180cd390 2023-07-06T03:45:11.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.898+0000 7f741e9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74180b6920 con 0x7f74180b8410 2023-07-06T03:45:11.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.898+0000 7f7417fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f741815c520 0x7f7418160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56020/0 (socket says 172.21.15.97:56020) 2023-07-06T03:45:11.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.898+0000 7f7417fff700 1 -- 172.21.15.97:0/120998953 learned_addr learned my addr 172.21.15.97:0/120998953 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:11.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.898+0000 7f741cf9c700 1 --2- 172.21.15.97:0/120998953 >> v2:172.21.15.146:3300/0 conn(0x7f74180b8410 0x7f7418167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:11.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.899+0000 7f7416ffd700 1 -- 172.21.15.97:0/120998953 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608845959 0 0) 0x7f74180cdae0 con 0x7f741815c520 2023-07-06T03:45:11.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.899+0000 7f7416ffd700 1 -- 172.21.15.97:0/120998953 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74000035b0 con 0x7f741815c520 2023-07-06T03:45:11.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.899+0000 7f7416ffd700 1 -- 172.21.15.97:0/120998953 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610487800 0 0) 0x7f74180b6640 con 0x7f74180cd390 2023-07-06T03:45:11.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.899+0000 7f7416ffd700 1 -- 172.21.15.97:0/120998953 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74180cdae0 con 0x7f74180cd390 2023-07-06T03:45:11.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.899+0000 7f741cf9c700 1 -- 172.21.15.97:0/120998953 >> v1:172.21.15.146:6789/0 conn(0x7f74180cd390 legacy=0x7f7418160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:11.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.899+0000 7f741cf9c700 1 -- 172.21.15.97:0/120998953 >> v1:172.21.15.97:6789/0 conn(0x7f741815c520 legacy=0x7f7418160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:11.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.899+0000 7f741cf9c700 1 -- 172.21.15.97:0/120998953 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74180b6c00 con 0x7f74180b8410 2023-07-06T03:45:11.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.899+0000 7f741cf9c700 1 --2- 172.21.15.97:0/120998953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74180b8410 0x7f7418167510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7410009d90 tx=0x7f741000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abf3f83e0c90397e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:11.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.899+0000 7f7416ffd700 1 -- 172.21.15.97:0/120998953 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741000e070 con 0x7f74180b8410 2023-07-06T03:45:11.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.899+0000 7f7416ffd700 1 -- 172.21.15.97:0/120998953 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741000bb20 con 0x7f74180b8410 2023-07-06T03:45:11.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.899+0000 7f7416ffd700 1 -- 172.21.15.97:0/120998953 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410012430 con 0x7f74180b8410 2023-07-06T03:45:11.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.900+0000 7f741e9ff700 1 -- 172.21.15.97:0/120998953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74180b8410 msgr2=0x7f7418167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:11.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.900+0000 7f741e9ff700 1 --2- 172.21.15.97:0/120998953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74180b8410 0x7f7418167510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7410009d90 tx=0x7f741000b330 comp rx=0 tx=0).stop 2023-07-06T03:45:11.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.900+0000 7f741e9ff700 1 -- 172.21.15.97:0/120998953 shutdown_connections 2023-07-06T03:45:11.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.900+0000 7f741e9ff700 1 --2- 172.21.15.97:0/120998953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74180b8410 0x7f7418167510 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:11.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.900+0000 7f741e9ff700 1 -- 172.21.15.97:0/120998953 >> 172.21.15.97:0/120998953 conn(0x7f7418071790 msgr2=0x7f7418071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:11.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.900+0000 7f741e9ff700 1 -- 172.21.15.97:0/120998953 shutdown_connections 2023-07-06T03:45:11.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.900+0000 7f741e9ff700 1 -- 172.21.15.97:0/120998953 wait complete. 2023-07-06T03:45:11.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.902+0000 7f741e9ff700 1 Processor -- start 2023-07-06T03:45:11.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.902+0000 7f741e9ff700 1 -- start start 2023-07-06T03:45:11.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741e9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74180b8410 0x7f741814c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:11.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741e9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74180cd390 0x7f74181539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:11.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741e9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741815c520 0x7f74180d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:11.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741e9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741814cdd0 con 0x7f741815c520 2023-07-06T03:45:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741e9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741815e4f0 con 0x7f74180cd390 2023-07-06T03:45:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741e9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741815e7d0 con 0x7f74180b8410 2023-07-06T03:45:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741cf9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741815c520 0x7f74180d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741cf9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741815c520 0x7f74180d30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46780/0 (socket says 172.21.15.97:46780) 2023-07-06T03:45:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741cf9c700 1 -- 172.21.15.97:0/2766715235 learned_addr learned my addr 172.21.15.97:0/2766715235 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f7417fff700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74180b8410 0x7f741814c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f74177fe700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74180cd390 0x7f74181539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741cf9c700 1 -- 172.21.15.97:0/2766715235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74180b8410 msgr2=0x7f741814c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741cf9c700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74180b8410 0x7f741814c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741cf9c700 1 -- 172.21.15.97:0/2766715235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74180cd390 msgr2=0x7f74181539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.903+0000 7f741cf9c700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74180cd390 0x7f74181539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.904+0000 7f741cf9c700 1 -- 172.21.15.97:0/2766715235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7410009a70 con 0x7f741815c520 2023-07-06T03:45:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.904+0000 7f74177fe700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74180cd390 0x7f74181539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.904+0000 7f741cf9c700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741815c520 0x7f74180d30b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f741806c240 tx=0x7f7410004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:11.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.904+0000 7f7414ff9700 1 -- 172.21.15.97:0/2766715235 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741001b810 con 0x7f741815c520 2023-07-06T03:45:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.904+0000 7f7414ff9700 1 -- 172.21.15.97:0/2766715235 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7410020070 con 0x7f741815c520 2023-07-06T03:45:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.904+0000 7f741e9ff700 1 -- 172.21.15.97:0/2766715235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741815eab0 con 0x7f741815c520 2023-07-06T03:45:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.904+0000 7f7414ff9700 1 -- 172.21.15.97:0/2766715235 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741001bbd0 con 0x7f741815c520 2023-07-06T03:45:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.904+0000 7f741e9ff700 1 -- 172.21.15.97:0/2766715235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741815f070 con 0x7f741815c520 2023-07-06T03:45:11.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.905+0000 7f741e9ff700 1 -- 172.21.15.97:0/2766715235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741804e340 con 0x7f741815c520 2023-07-06T03:45:11.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.906+0000 7f7414ff9700 1 -- 172.21.15.97:0/2766715235 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f741001bd30 con 0x7f741815c520 2023-07-06T03:45:11.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.907+0000 7f7414ff9700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740406ace0 0x7f740406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:11.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.907+0000 7f7414ff9700 1 -- 172.21.15.97:0/2766715235 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7410022070 con 0x7f741815c520 2023-07-06T03:45:11.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.907+0000 7f7417fff700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740406ace0 0x7f740406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:11.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.908+0000 7f7417fff700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740406ace0 0x7f740406d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7408000f60 tx=0x7f7408009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:11.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:11.911+0000 7f7414ff9700 1 -- 172.21.15.97:0/2766715235 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7410056930 con 0x7f741815c520 2023-07-06T03:45:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.108+0000 7f741e9ff700 1 -- 172.21.15.97:0/2766715235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f741814cf10 con 0x7f741815c520 2023-07-06T03:45:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.109+0000 7f7414ff9700 1 -- 172.21.15.97:0/2766715235 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18390 (secure 0 0 0) 0x7f7410059f50 con 0x7f741815c520 2023-07-06T03:45:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:12.111 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":22,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:11.125916+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:12.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.111+0000 7f741e9ff700 1 -- 172.21.15.97:0/2766715235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740406ace0 msgr2=0x7f740406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.111+0000 7f741e9ff700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740406ace0 0x7f740406d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7408000f60 tx=0x7f7408009040 comp rx=0 tx=0).stop 2023-07-06T03:45:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.111+0000 7f741e9ff700 1 -- 172.21.15.97:0/2766715235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741815c520 msgr2=0x7f74180d30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.111+0000 7f741e9ff700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741815c520 0x7f74180d30b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f741806c240 tx=0x7f7410004ad0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.111+0000 7f741e9ff700 1 -- 172.21.15.97:0/2766715235 shutdown_connections 2023-07-06T03:45:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.111+0000 7f741e9ff700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740406ace0 0x7f740406d190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.111+0000 7f741e9ff700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f741815c520 0x7f74180d30b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.111+0000 7f741e9ff700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74180b8410 0x7f741814c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.111+0000 7f741e9ff700 1 --2- 172.21.15.97:0/2766715235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74180cd390 0x7f74181539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.112+0000 7f741e9ff700 1 -- 172.21.15.97:0/2766715235 >> 172.21.15.97:0/2766715235 conn(0x7f7418071790 msgr2=0x7f74180b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:12.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.112+0000 7f741e9ff700 1 -- 172.21.15.97:0/2766715235 shutdown_connections 2023-07-06T03:45:12.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.112+0000 7f741e9ff700 1 -- 172.21.15.97:0/2766715235 wait complete. 2023-07-06T03:45:12.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.325+0000 7f1ffb9ff700 1 Processor -- start 2023-07-06T03:45:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.325+0000 7f1ffb9ff700 1 -- start start 2023-07-06T03:45:12.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ffb9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ff40cf8a0 0x7f1ff40cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ffb9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ff40d15e0 0x7f1ff40d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ffb9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff40b8a50 con 0x7f1ff40b8680 2023-07-06T03:45:12.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ffb9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff415cc70 con 0x7f1ff40d15e0 2023-07-06T03:45:12.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ffb9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40d0250 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ff9f9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ff40d15e0 0x7f1ff40d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ff9f9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ff40d15e0 0x7f1ff40d1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46782/0 (socket says 172.21.15.97:46782) 2023-07-06T03:45:12.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ff979b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1ff40b8680 0x7f1ff4160d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:53616/0 (socket says 172.21.15.97:53616) 2023-07-06T03:45:12.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ff9f9c700 1 -- 172.21.15.97:0/548443029 learned_addr learned my addr 172.21.15.97:0/548443029 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:12.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ff8f9a700 1 --2- 172.21.15.97:0/548443029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ff40cf8a0 0x7f1ff40cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ff9f9c700 1 -- 172.21.15.97:0/548443029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ff40cf8a0 msgr2=0x7f1ff40cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:12.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ff9f9c700 1 --2- 172.21.15.97:0/548443029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ff40cf8a0 0x7f1ff40cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ff9f9c700 1 -- 172.21.15.97:0/548443029 >> v1:172.21.15.146:6789/0 conn(0x7f1ff40b8680 legacy=0x7f1ff4160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.327+0000 7f1ff9f9c700 1 -- 172.21.15.97:0/548443029 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff40cd4e0 con 0x7f1ff40d15e0 2023-07-06T03:45:12.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1ff9f9c700 1 --2- 172.21.15.97:0/548443029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40d15e0 0x7f1ff40d1a60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009e50 tx=0x7f1fec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1a13076d4923c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:12.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1febfff700 1 -- 172.21.15.97:0/548443029 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00e070 con 0x7f1ff40d15e0 2023-07-06T03:45:12.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1febfff700 1 -- 172.21.15.97:0/548443029 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fec00b910 con 0x7f1ff40d15e0 2023-07-06T03:45:12.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1febfff700 1 -- 172.21.15.97:0/548443029 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00bcc0 con 0x7f1ff40d15e0 2023-07-06T03:45:12.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/548443029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40d15e0 msgr2=0x7f1ff40d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1ffb9ff700 1 --2- 172.21.15.97:0/548443029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40d15e0 0x7f1ff40d1a60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009e50 tx=0x7f1fec00b330 comp rx=0 tx=0).stop 2023-07-06T03:45:12.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/548443029 shutdown_connections 2023-07-06T03:45:12.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1ffb9ff700 1 --2- 172.21.15.97:0/548443029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40d15e0 0x7f1ff40d1a60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1ffb9ff700 1 --2- 172.21.15.97:0/548443029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ff40cf8a0 0x7f1ff40cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/548443029 >> 172.21.15.97:0/548443029 conn(0x7f1ff406b4d0 msgr2=0x7f1ff406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:12.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/548443029 shutdown_connections 2023-07-06T03:45:12.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.328+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/548443029 wait complete. 2023-07-06T03:45:12.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.330+0000 7f1ffb9ff700 1 Processor -- start 2023-07-06T03:45:12.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.330+0000 7f1ffb9ff700 1 -- start start 2023-07-06T03:45:12.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.331+0000 7f1ffb9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ff40b8680 0x7f1ff40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.331+0000 7f1ffb9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40cf8a0 0x7f1ff414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.331+0000 7f1ffb9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ff40d15e0 0x7f1ff40cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.331+0000 7f1ffb9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40ccdf0 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.331+0000 7f1ffb9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40cb0e0 con 0x7f1ff40d15e0 2023-07-06T03:45:12.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.331+0000 7f1ffb9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40ca4d0 con 0x7f1ff40b8680 2023-07-06T03:45:12.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.331+0000 7f1ff8f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40cf8a0 0x7f1ff414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.331+0000 7f1ff8f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40cf8a0 0x7f1ff414eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46784/0 (socket says 172.21.15.97:46784) 2023-07-06T03:45:12.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.331+0000 7f1ff8f9a700 1 -- 172.21.15.97:0/2264930477 learned_addr learned my addr 172.21.15.97:0/2264930477 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:12.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1ff979b700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ff40b8680 0x7f1ff40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1ff9f9c700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ff40d15e0 0x7f1ff40cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1ff8f9a700 1 -- 172.21.15.97:0/2264930477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ff40b8680 msgr2=0x7f1ff40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1ff8f9a700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ff40b8680 0x7f1ff40b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1ff8f9a700 1 -- 172.21.15.97:0/2264930477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ff40d15e0 msgr2=0x7f1ff40cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1ff8f9a700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ff40d15e0 0x7f1ff40cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1ff8f9a700 1 -- 172.21.15.97:0/2264930477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec009b30 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1ff8f9a700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40cf8a0 0x7f1ff414eea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e9b0 tx=0x7f1fe400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:12.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1fe9ffb700 1 -- 172.21.15.97:0/2264930477 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4009710 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1fe9ffb700 1 -- 172.21.15.97:0/2264930477 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe400ee50 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1fe9ffb700 1 -- 172.21.15.97:0/2264930477 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400ca50 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/2264930477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff40ca7b0 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.332+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/2264930477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff40cad70 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.333+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/2264930477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff404e340 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.334+0000 7f1fe9ffb700 1 -- 172.21.15.97:0/2264930477 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1fe400cbb0 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.335+0000 7f1fe9ffb700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fd806ace0 0x7f1fd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.335+0000 7f1fe9ffb700 1 -- 172.21.15.97:0/2264930477 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1fe40894b0 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.335+0000 7f1ff979b700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fd806ace0 0x7f1fd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.336+0000 7f1ff979b700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fd806ace0 0x7f1fd806d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0000f60 tx=0x7f1ff0007ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:12.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.339+0000 7f1fe9ffb700 1 -- 172.21.15.97:0/2264930477 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1fe4057fe0 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.536+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/2264930477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ff40cb220 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.537+0000 7f1fe9ffb700 1 -- 172.21.15.97:0/2264930477 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18390 (secure 0 0 0) 0x7f1fe400ce60 con 0x7f1ff40cf8a0 2023-07-06T03:45:12.539 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:12.540 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":22,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:11.125916+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/2264930477 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fd806ace0 msgr2=0x7f1fd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fd806ace0 0x7f1fd806d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0000f60 tx=0x7f1ff0007ed0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/2264930477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40cf8a0 msgr2=0x7f1ff414eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40cf8a0 0x7f1ff414eea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e9b0 tx=0x7f1fe400ecc0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/2264930477 shutdown_connections 2023-07-06T03:45:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fd806ace0 0x7f1fd806d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ff40d15e0 0x7f1ff40cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ff40b8680 0x7f1ff40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 --2- 172.21.15.97:0/2264930477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40cf8a0 0x7f1ff414eea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/2264930477 >> 172.21.15.97:0/2264930477 conn(0x7f1ff406b4d0 msgr2=0x7f1ff415b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:12.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/2264930477 shutdown_connections 2023-07-06T03:45:12.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.540+0000 7f1ffb9ff700 1 -- 172.21.15.97:0/2264930477 wait complete. 2023-07-06T03:45:12.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T03:45:12.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.761+0000 7f9d217ff700 1 Processor -- start 2023-07-06T03:45:12.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.761+0000 7f9d217ff700 1 -- start start 2023-07-06T03:45:12.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d217ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d1c15c610 0x7f9d1c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d217ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c06b360 0x7f9d1c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d217ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1c0cff90 con 0x7f9d1c0d2210 2023-07-06T03:45:12.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d217ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1c0d0780 con 0x7f9d1c15c610 2023-07-06T03:45:12.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d217ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c15c9e0 con 0x7f9d1c06b360 2023-07-06T03:45:12.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d1affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d1c15c610 0x7f9d1c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d1affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d1c15c610 0x7f9d1c0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46796/0 (socket says 172.21.15.97:46796) 2023-07-06T03:45:12.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d1affd700 1 -- 172.21.15.97:0/1983508888 learned_addr learned my addr 172.21.15.97:0/1983508888 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:12.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d13fff700 1 --2- 172.21.15.97:0/1983508888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c06b360 0x7f9d1c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d1affd700 1 -- 172.21.15.97:0/1983508888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c06b360 msgr2=0x7f9d1c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:12.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d1affd700 1 --2- 172.21.15.97:0/1983508888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c06b360 0x7f9d1c0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d1affd700 1 -- 172.21.15.97:0/1983508888 >> v1:172.21.15.146:6789/0 conn(0x7f9d1c0d2210 legacy=0x7f9d1c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.763+0000 7f9d1affd700 1 -- 172.21.15.97:0/1983508888 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c0b5650 con 0x7f9d1c15c610 2023-07-06T03:45:12.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.764+0000 7f9d1affd700 1 --2- 172.21.15.97:0/1983508888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c610 0x7f9d1c0b47b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009920 tx=0x7f9d0c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8256d6ee1ed9c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:12.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.764+0000 7f9d1a7fc700 1 -- 172.21.15.97:0/1983508888 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00ba20 con 0x7f9d1c15c610 2023-07-06T03:45:12.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.764+0000 7f9d1a7fc700 1 -- 172.21.15.97:0/1983508888 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c00bb80 con 0x7f9d1c15c610 2023-07-06T03:45:12.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.764+0000 7f9d1a7fc700 1 -- 172.21.15.97:0/1983508888 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c009e70 con 0x7f9d1c15c610 2023-07-06T03:45:12.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.764+0000 7f9d217ff700 1 -- 172.21.15.97:0/1983508888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c610 msgr2=0x7f9d1c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.764+0000 7f9d217ff700 1 --2- 172.21.15.97:0/1983508888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c610 0x7f9d1c0b47b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009920 tx=0x7f9d0c009c30 comp rx=0 tx=0).stop 2023-07-06T03:45:12.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.764+0000 7f9d217ff700 1 -- 172.21.15.97:0/1983508888 shutdown_connections 2023-07-06T03:45:12.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.764+0000 7f9d217ff700 1 --2- 172.21.15.97:0/1983508888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c610 0x7f9d1c0b47b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.764+0000 7f9d217ff700 1 --2- 172.21.15.97:0/1983508888 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c06b360 0x7f9d1c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.764+0000 7f9d217ff700 1 -- 172.21.15.97:0/1983508888 >> 172.21.15.97:0/1983508888 conn(0x7f9d1c06a6c0 msgr2=0x7f9d1c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:12.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.765+0000 7f9d217ff700 1 -- 172.21.15.97:0/1983508888 shutdown_connections 2023-07-06T03:45:12.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.765+0000 7f9d217ff700 1 -- 172.21.15.97:0/1983508888 wait complete. 2023-07-06T03:45:12.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.766+0000 7f9d217ff700 1 Processor -- start 2023-07-06T03:45:12.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.766+0000 7f9d217ff700 1 -- start start 2023-07-06T03:45:12.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.767+0000 7f9d217ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c06b360 0x7f9d1c0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.767+0000 7f9d217ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d1c0d2210 0x7f9d1c15b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.767+0000 7f9d217ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c610 0x7f9d1c15b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.767+0000 7f9d217ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0c7090 con 0x7f9d1c15c610 2023-07-06T03:45:12.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.767+0000 7f9d217ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0c7370 con 0x7f9d1c0d2210 2023-07-06T03:45:12.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.767+0000 7f9d217ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0c7650 con 0x7f9d1c06b360 2023-07-06T03:45:12.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.767+0000 7f9d1b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c610 0x7f9d1c15b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d1affd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c06b360 0x7f9d1c0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d1b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c610 0x7f9d1c15b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46800/0 (socket says 172.21.15.97:46800) 2023-07-06T03:45:12.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d1b7fe700 1 -- 172.21.15.97:0/1238473779 learned_addr learned my addr 172.21.15.97:0/1238473779 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:12.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d13fff700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d1c0d2210 0x7f9d1c15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d1b7fe700 1 -- 172.21.15.97:0/1238473779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c06b360 msgr2=0x7f9d1c0c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d1b7fe700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c06b360 0x7f9d1c0c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d1b7fe700 1 -- 172.21.15.97:0/1238473779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d1c0d2210 msgr2=0x7f9d1c15b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:12.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d1b7fe700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d1c0d2210 0x7f9d1c15b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:12.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d1b7fe700 1 -- 172.21.15.97:0/1238473779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c009550 con 0x7f9d1c15c610 2023-07-06T03:45:12.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d1b7fe700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c610 0x7f9d1c15b840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400cc50 tx=0x7f9d1400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:12.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d137fe700 1 -- 172.21.15.97:0/1238473779 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14003c00 con 0x7f9d1c15c610 2023-07-06T03:45:12.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d137fe700 1 -- 172.21.15.97:0/1238473779 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d14003d60 con 0x7f9d1c15c610 2023-07-06T03:45:12.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d217ff700 1 -- 172.21.15.97:0/1238473779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c0c7930 con 0x7f9d1c15c610 2023-07-06T03:45:12.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.768+0000 7f9d137fe700 1 -- 172.21.15.97:0/1238473779 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400aab0 con 0x7f9d1c15c610 2023-07-06T03:45:12.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.769+0000 7f9d217ff700 1 -- 172.21.15.97:0/1238473779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c0c6da0 con 0x7f9d1c15c610 2023-07-06T03:45:12.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.770+0000 7f9d217ff700 1 -- 172.21.15.97:0/1238473779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c0c7e50 con 0x7f9d1c15c610 2023-07-06T03:45:12.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.771+0000 7f9d137fe700 1 -- 172.21.15.97:0/1238473779 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d1400ac10 con 0x7f9d1c15c610 2023-07-06T03:45:12.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.771+0000 7f9d137fe700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 0x7f9cfc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:12.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.772+0000 7f9d137fe700 1 -- 172.21.15.97:0/1238473779 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9d140894d0 con 0x7f9d1c15c610 2023-07-06T03:45:12.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.772+0000 7f9d1affd700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 0x7f9cfc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:12.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.772+0000 7f9d1affd700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 0x7f9cfc06d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c000c00 tx=0x7f9d0c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:12.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:12.776+0000 7f9d137fe700 1 -- 172.21.15.97:0/1238473779 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9d1c0c7e50 con 0x7f9d1c15c610 2023-07-06T03:45:13.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.015+0000 7f9d217ff700 1 -- 172.21.15.97:0/1238473779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9d1c0c7e50 con 0x7f9d1c15c610 2023-07-06T03:45:13.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.687+0000 7f9d137fe700 1 -- 172.21.15.97:0/1238473779 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9d1c0c7e50 con 0x7f9d1c15c610 2023-07-06T03:45:13.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.690+0000 7f9d217ff700 1 -- 172.21.15.97:0/1238473779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 msgr2=0x7f9cfc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:13.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.690+0000 7f9d217ff700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 0x7f9cfc06d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c000c00 tx=0x7f9d0c008030 comp rx=0 tx=0).stop 2023-07-06T03:45:13.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.690+0000 7f9d217ff700 1 -- 172.21.15.97:0/1238473779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c610 msgr2=0x7f9d1c15b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:13.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.690+0000 7f9d217ff700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c610 0x7f9d1c15b840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400cc50 tx=0x7f9d1400cf60 comp rx=0 tx=0).stop 2023-07-06T03:45:13.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.691+0000 7f9d217ff700 1 -- 172.21.15.97:0/1238473779 shutdown_connections 2023-07-06T03:45:13.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.691+0000 7f9d217ff700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 0x7f9cfc06d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:13.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.691+0000 7f9d217ff700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c610 0x7f9d1c15b840 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:13.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.691+0000 7f9d217ff700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c06b360 0x7f9d1c0c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:13.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.691+0000 7f9d217ff700 1 --2- 172.21.15.97:0/1238473779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d1c0d2210 0x7f9d1c15b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.691+0000 7f9d217ff700 1 -- 172.21.15.97:0/1238473779 >> 172.21.15.97:0/1238473779 conn(0x7f9d1c06a6c0 msgr2=0x7f9d1c0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.691+0000 7f9d217ff700 1 -- 172.21.15.97:0/1238473779 shutdown_connections 2023-07-06T03:45:13.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.691+0000 7f9d217ff700 1 -- 172.21.15.97:0/1238473779 wait complete. 2023-07-06T03:45:13.714 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T03:45:13.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T03:45:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.916+0000 7fd8913ff700 1 Processor -- start 2023-07-06T03:45:13.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.916+0000 7fd8913ff700 1 -- start start 2023-07-06T03:45:13.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd8913ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd88c15c610 0x7fd88c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:13.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd8913ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd88c06b360 0x7fd88c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:13.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd8913ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88c0cff90 con 0x7fd88c0d2210 2023-07-06T03:45:13.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd8913ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88c0d0780 con 0x7fd88c06b360 2023-07-06T03:45:13.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd8913ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88c15c9e0 con 0x7fd88c15c610 2023-07-06T03:45:13.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd883fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd88c06b360 0x7fd88c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:13.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd883fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd88c06b360 0x7fd88c0b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46814/0 (socket says 172.21.15.97:46814) 2023-07-06T03:45:13.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd883fff700 1 -- 172.21.15.97:0/1989397627 learned_addr learned my addr 172.21.15.97:0/1989397627 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:13.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd88b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd88c0d2210 0x7fd88c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56028/0 (socket says 172.21.15.97:56028) 2023-07-06T03:45:13.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd88affd700 1 --2- 172.21.15.97:0/1989397627 >> v2:172.21.15.146:3300/0 conn(0x7fd88c15c610 0x7fd88c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:13.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd883fff700 1 -- 172.21.15.97:0/1989397627 >> v2:172.21.15.146:3300/0 conn(0x7fd88c15c610 msgr2=0x7fd88c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:13.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd883fff700 1 --2- 172.21.15.97:0/1989397627 >> v2:172.21.15.146:3300/0 conn(0x7fd88c15c610 0x7fd88c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd883fff700 1 -- 172.21.15.97:0/1989397627 >> v1:172.21.15.97:6789/0 conn(0x7fd88c0d2210 legacy=0x7fd88c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd88a7fc700 1 -- 172.21.15.97:0/1989397627 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759177086 0 0) 0x7fd88c0cff90 con 0x7fd88c0d2210 2023-07-06T03:45:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd883fff700 1 -- 172.21.15.97:0/1989397627 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c0b5650 con 0x7fd88c06b360 2023-07-06T03:45:13.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.918+0000 7fd88affd700 1 --2- 172.21.15.97:0/1989397627 >> v2:172.21.15.146:3300/0 conn(0x7fd88c15c610 0x7fd88c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.919+0000 7fd883fff700 1 --2- 172.21.15.97:0/1989397627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c06b360 0x7fd88c0b4cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd88c08b000 tx=0x7fd874009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7686aaab2cce4e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.919+0000 7fd88a7fc700 1 -- 172.21.15.97:0/1989397627 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87400b860 con 0x7fd88c06b360 2023-07-06T03:45:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.919+0000 7fd88a7fc700 1 -- 172.21.15.97:0/1989397627 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87400b9c0 con 0x7fd88c06b360 2023-07-06T03:45:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.919+0000 7fd88a7fc700 1 -- 172.21.15.97:0/1989397627 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87400bd50 con 0x7fd88c06b360 2023-07-06T03:45:13.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.919+0000 7fd8913ff700 1 -- 172.21.15.97:0/1989397627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c06b360 msgr2=0x7fd88c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.919+0000 7fd8913ff700 1 --2- 172.21.15.97:0/1989397627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c06b360 0x7fd88c0b4cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd88c08b000 tx=0x7fd874009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.919+0000 7fd8913ff700 1 -- 172.21.15.97:0/1989397627 shutdown_connections 2023-07-06T03:45:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.919+0000 7fd8913ff700 1 --2- 172.21.15.97:0/1989397627 >> v2:172.21.15.146:3300/0 conn(0x7fd88c15c610 0x7fd88c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.919+0000 7fd8913ff700 1 --2- 172.21.15.97:0/1989397627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c06b360 0x7fd88c0b4cf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:13.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.919+0000 7fd8913ff700 1 -- 172.21.15.97:0/1989397627 >> 172.21.15.97:0/1989397627 conn(0x7fd88c06a6c0 msgr2=0x7fd88c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:13.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.920+0000 7fd8913ff700 1 -- 172.21.15.97:0/1989397627 shutdown_connections 2023-07-06T03:45:13.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.920+0000 7fd8913ff700 1 -- 172.21.15.97:0/1989397627 wait complete. 2023-07-06T03:45:13.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.921+0000 7fd8913ff700 1 Processor -- start 2023-07-06T03:45:13.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.921+0000 7fd8913ff700 1 -- start start 2023-07-06T03:45:13.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.922+0000 7fd8913ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd88c06b360 0x7fd88c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:13.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.922+0000 7fd8913ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c0d2210 0x7fd88c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:13.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.922+0000 7fd8913ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd88c15c610 0x7fd88c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:13.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.922+0000 7fd8913ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0ca810 con 0x7fd88c0d2210 2023-07-06T03:45:13.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.922+0000 7fd8913ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0cb390 con 0x7fd88c06b360 2023-07-06T03:45:13.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.922+0000 7fd8913ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0caa60 con 0x7fd88c15c610 2023-07-06T03:45:13.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.922+0000 7fd883fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c0d2210 0x7fd88c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:13.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd88affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd88c06b360 0x7fd88c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:13.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd883fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c0d2210 0x7fd88c15cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46820/0 (socket says 172.21.15.97:46820) 2023-07-06T03:45:13.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd883fff700 1 -- 172.21.15.97:0/2411455860 learned_addr learned my addr 172.21.15.97:0/2411455860 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:13.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd88b7fe700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd88c15c610 0x7fd88c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:13.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd883fff700 1 -- 172.21.15.97:0/2411455860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd88c15c610 msgr2=0x7fd88c15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:13.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd883fff700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd88c15c610 0x7fd88c15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:13.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd883fff700 1 -- 172.21.15.97:0/2411455860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd88c06b360 msgr2=0x7fd88c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:13.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd883fff700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd88c06b360 0x7fd88c153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd883fff700 1 -- 172.21.15.97:0/2411455860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd874009550 con 0x7fd88c0d2210 2023-07-06T03:45:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd883fff700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c0d2210 0x7fd88c15cb20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd874000c00 tx=0x7fd874004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd8837fe700 1 -- 172.21.15.97:0/2411455860 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874004380 con 0x7fd88c0d2210 2023-07-06T03:45:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd8837fe700 1 -- 172.21.15.97:0/2411455860 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87401f070 con 0x7fd88c0d2210 2023-07-06T03:45:13.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd8913ff700 1 -- 172.21.15.97:0/2411455860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c0cad40 con 0x7fd88c0d2210 2023-07-06T03:45:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.923+0000 7fd8837fe700 1 -- 172.21.15.97:0/2411455860 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8740036a0 con 0x7fd88c0d2210 2023-07-06T03:45:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.924+0000 7fd8913ff700 1 -- 172.21.15.97:0/2411455860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c1544d0 con 0x7fd88c0d2210 2023-07-06T03:45:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.925+0000 7fd8913ff700 1 -- 172.21.15.97:0/2411455860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88c0cb520 con 0x7fd88c0d2210 2023-07-06T03:45:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.925+0000 7fd8837fe700 1 -- 172.21.15.97:0/2411455860 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd874021030 con 0x7fd88c0d2210 2023-07-06T03:45:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.926+0000 7fd8837fe700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd86c06ace0 0x7fd86c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.926+0000 7fd8837fe700 1 -- 172.21.15.97:0/2411455860 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd87408a9d0 con 0x7fd88c0d2210 2023-07-06T03:45:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.927+0000 7fd88affd700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd86c06ace0 0x7fd86c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:13.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.927+0000 7fd88affd700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd86c06ace0 0x7fd86c06d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd87c006fd0 tx=0x7fd87c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:13.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:13.931+0000 7fd8837fe700 1 -- 172.21.15.97:0/2411455860 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd874055b10 con 0x7fd88c0d2210 2023-07-06T03:45:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.169+0000 7fd8913ff700 1 -- 172.21.15.97:0/2411455860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd88c04e340 con 0x7fd88c0d2210 2023-07-06T03:45:14.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.692+0000 7fd8837fe700 1 -- 172.21.15.97:0/2411455860 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd874059130 con 0x7fd88c0d2210 2023-07-06T03:45:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.694+0000 7fd8913ff700 1 -- 172.21.15.97:0/2411455860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd86c06ace0 msgr2=0x7fd86c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.694+0000 7fd8913ff700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd86c06ace0 0x7fd86c06d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd87c006fd0 tx=0x7fd87c008040 comp rx=0 tx=0).stop 2023-07-06T03:45:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.694+0000 7fd8913ff700 1 -- 172.21.15.97:0/2411455860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c0d2210 msgr2=0x7fd88c15cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.694+0000 7fd8913ff700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c0d2210 0x7fd88c15cb20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd874000c00 tx=0x7fd874004750 comp rx=0 tx=0).stop 2023-07-06T03:45:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.694+0000 7fd8913ff700 1 -- 172.21.15.97:0/2411455860 shutdown_connections 2023-07-06T03:45:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.694+0000 7fd8913ff700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd86c06ace0 0x7fd86c06d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.695+0000 7fd8913ff700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd88c15c610 0x7fd88c15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.695+0000 7fd8913ff700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd88c06b360 0x7fd88c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:14.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.695+0000 7fd8913ff700 1 --2- 172.21.15.97:0/2411455860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88c0d2210 0x7fd88c15cb20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:14.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.695+0000 7fd8913ff700 1 -- 172.21.15.97:0/2411455860 >> 172.21.15.97:0/2411455860 conn(0x7fd88c06a6c0 msgr2=0x7fd88c15a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:14.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.695+0000 7fd8913ff700 1 -- 172.21.15.97:0/2411455860 shutdown_connections 2023-07-06T03:45:14.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.695+0000 7fd8913ff700 1 -- 172.21.15.97:0/2411455860 wait complete. 2023-07-06T03:45:14.716 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T03:45:14.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:45:14.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.911+0000 7f47f9fff700 1 Processor -- start 2023-07-06T03:45:14.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.911+0000 7f47f9fff700 1 -- start start 2023-07-06T03:45:14.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.912+0000 7f47f9fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f47f415ccf0 0x7f47f40d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:14.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.912+0000 7f47f9fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f40cfd00 con 0x7f47f415c920 2023-07-06T03:45:14.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.912+0000 7f47f9fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f40b28f0 con 0x7f47f406caa0 2023-07-06T03:45:14.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.912+0000 7f47f9fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f41610b0 con 0x7f47f415ccf0 2023-07-06T03:45:14.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.912+0000 7f47f37fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f47f415c920 0x7f47f406c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56036/0 (socket says 172.21.15.97:56036) 2023-07-06T03:45:14.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.912+0000 7f47f37fe700 1 -- 172.21.15.97:0/1227430362 learned_addr learned my addr 172.21.15.97:0/1227430362 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:14.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066090360 0 0) 0x7f47f40cfd00 con 0x7f47f415c920 2023-07-06T03:45:14.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47dc0035b0 con 0x7f47f415c920 2023-07-06T03:45:14.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f2ffd700 1 --2- 172.21.15.97:0/1227430362 >> v2:172.21.15.146:3300/0 conn(0x7f47f415ccf0 0x7f47f40d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:14.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2242720587 0 0) 0x7f47dc0035b0 con 0x7f47f415c920 2023-07-06T03:45:14.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 >> v2:172.21.15.146:3300/0 conn(0x7f47f415ccf0 msgr2=0x7f47f40d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:14.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 --2- 172.21.15.97:0/1227430362 >> v2:172.21.15.146:3300/0 conn(0x7f47f415ccf0 0x7f47f40d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:14.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 >> v1:172.21.15.146:6789/0 conn(0x7f47f406caa0 legacy=0x7f47f40d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:14.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f4161390 con 0x7f47f415c920 2023-07-06T03:45:14.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f47e4001670 con 0x7f47f415c920 2023-07-06T03:45:14.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 >> v1:172.21.15.97:6789/0 conn(0x7f47f415c920 legacy=0x7f47f406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:14.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 --2- 172.21.15.97:0/1227430362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47dc003a00 0x7f47dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:14.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 --2- 172.21.15.97:0/1227430362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47dc004850 0x7f47dc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:14.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 --2- 172.21.15.97:0/1227430362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47dc005200 0x7f47dc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:14.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41610b0 con 0x7f47dc003a00 2023-07-06T03:45:14.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc00c080 con 0x7f47dc004850 2023-07-06T03:45:14.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc00c240 con 0x7f47dc005200 2023-07-06T03:45:14.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.913+0000 7f47f37fe700 1 --2- 172.21.15.97:0/1227430362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47dc003a00 0x7f47dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:14.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f2ffd700 1 --2- 172.21.15.97:0/1227430362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47dc004850 0x7f47dc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:14.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f3fff700 1 --2- 172.21.15.97:0/1227430362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47dc005200 0x7f47dc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:14.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f37fe700 1 -- 172.21.15.97:0/1227430362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47dc005200 msgr2=0x7f47dc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:14.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f37fe700 1 --2- 172.21.15.97:0/1227430362 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47dc005200 0x7f47dc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:14.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f37fe700 1 -- 172.21.15.97:0/1227430362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47dc004850 msgr2=0x7f47dc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:14.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f37fe700 1 --2- 172.21.15.97:0/1227430362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47dc004850 0x7f47dc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:14.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f37fe700 1 -- 172.21.15.97:0/1227430362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc0035b0 con 0x7f47dc003a00 2023-07-06T03:45:14.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f3fff700 1 -- 172.21.15.97:0/1227430362 reap_dead start 2023-07-06T03:45:14.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f37fe700 1 --2- 172.21.15.97:0/1227430362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47dc003a00 0x7f47dc003e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47e400c240 tx=0x7f47dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504049e31865dd88 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:14.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4007c10 con 0x7f47dc003a00 2023-07-06T03:45:14.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e400ce00 con 0x7f47dc003a00 2023-07-06T03:45:14.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.914+0000 7f47f27fc700 1 -- 172.21.15.97:0/1227430362 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4005ba0 con 0x7f47dc003a00 2023-07-06T03:45:14.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.915+0000 7f47f9fff700 1 -- 172.21.15.97:0/1227430362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47dc003a00 msgr2=0x7f47dc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:14.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.915+0000 7f47f9fff700 1 --2- 172.21.15.97:0/1227430362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47dc003a00 0x7f47dc003e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47e400c240 tx=0x7f47dc005880 comp rx=0 tx=0).stop 2023-07-06T03:45:14.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.915+0000 7f47f9fff700 1 -- 172.21.15.97:0/1227430362 shutdown_connections 2023-07-06T03:45:14.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.915+0000 7f47f9fff700 1 --2- 172.21.15.97:0/1227430362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47dc003a00 0x7f47dc003e10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:14.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.915+0000 7f47f9fff700 1 -- 172.21.15.97:0/1227430362 >> 172.21.15.97:0/1227430362 conn(0x7f47f406ace0 msgr2=0x7f47f406dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:14.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.915+0000 7f47f9fff700 1 -- 172.21.15.97:0/1227430362 shutdown_connections 2023-07-06T03:45:14.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.915+0000 7f47f9fff700 1 -- 172.21.15.97:0/1227430362 wait complete. 2023-07-06T03:45:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.917+0000 7f47f9fff700 1 Processor -- start 2023-07-06T03:45:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.917+0000 7f47f9fff700 1 -- start start 2023-07-06T03:45:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.918+0000 7f47f9fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47f414d030 0x7f47f415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.918+0000 7f47f9fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40b3d80 0x7f47f40b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.918+0000 7f47f9fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f40b6360 0x7f47f41f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.918+0000 7f47f9fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f415a4d0 con 0x7f47f40b3d80 2023-07-06T03:45:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.918+0000 7f47f9fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f415a7b0 con 0x7f47f414d030 2023-07-06T03:45:14.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.918+0000 7f47f9fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f415aa90 con 0x7f47f40b6360 2023-07-06T03:45:14.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.918+0000 7f47f2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40b3d80 0x7f47f40b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:14.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.918+0000 7f47f2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40b3d80 0x7f47f40b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46836/0 (socket says 172.21.15.97:46836) 2023-07-06T03:45:14.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.918+0000 7f47f2ffd700 1 -- 172.21.15.97:0/1802421093 learned_addr learned my addr 172.21.15.97:0/1802421093 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:14.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.918+0000 7f47f37fe700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47f414d030 0x7f47f415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:14.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.918+0000 7f47f3fff700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f40b6360 0x7f47f41f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47f2ffd700 1 -- 172.21.15.97:0/1802421093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f40b6360 msgr2=0x7f47f41f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47f2ffd700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f40b6360 0x7f47f41f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47f2ffd700 1 -- 172.21.15.97:0/1802421093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47f414d030 msgr2=0x7f47f415b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47f2ffd700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47f414d030 0x7f47f415b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:14.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47f2ffd700 1 -- 172.21.15.97:0/1802421093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f415ad70 con 0x7f47f40b3d80 2023-07-06T03:45:14.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47f37fe700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47f414d030 0x7f47f415b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:14.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47f2ffd700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40b3d80 0x7f47f40b5e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f47e8011aa0 tx=0x7f47e8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47dbfff700 1 -- 172.21.15.97:0/1802421093 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e800f940 con 0x7f47f40b3d80 2023-07-06T03:45:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47dbfff700 1 -- 172.21.15.97:0/1802421093 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e800fac0 con 0x7f47f40b3d80 2023-07-06T03:45:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47f9fff700 1 -- 172.21.15.97:0/1802421093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f415ba00 con 0x7f47f40b3d80 2023-07-06T03:45:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47dbfff700 1 -- 172.21.15.97:0/1802421093 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e800fe70 con 0x7f47f40b3d80 2023-07-06T03:45:14.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.919+0000 7f47f9fff700 1 -- 172.21.15.97:0/1802421093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f414d400 con 0x7f47f40b3d80 2023-07-06T03:45:14.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.920+0000 7f47f9fff700 1 -- 172.21.15.97:0/1802421093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f4066a40 con 0x7f47f40b3d80 2023-07-06T03:45:14.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.921+0000 7f47dbfff700 1 -- 172.21.15.97:0/1802421093 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47e8014440 con 0x7f47f40b3d80 2023-07-06T03:45:14.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.922+0000 7f47dbfff700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f47e006ace0 0x7f47e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:14.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.922+0000 7f47dbfff700 1 -- 172.21.15.97:0/1802421093 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47e8089bd0 con 0x7f47f40b3d80 2023-07-06T03:45:14.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.922+0000 7f47f37fe700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f47e006ace0 0x7f47e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:14.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.923+0000 7f47f37fe700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f47e006ace0 0x7f47e006d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47e4000c00 tx=0x7f47e4007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:14.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:14.926+0000 7f47dbfff700 1 -- 172.21.15.97:0/1802421093 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f47e8054ca0 con 0x7f47f40b3d80 2023-07-06T03:45:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.162+0000 7f47f9fff700 1 -- 172.21.15.97:0/1802421093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47f40ad8d0 con 0x7f47f40b3d80 2023-07-06T03:45:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.163+0000 7f47dbfff700 1 -- 172.21.15.97:0/1802421093 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4665 (secure 0 0 0) 0x7f47f40ad8d0 con 0x7f47f40b3d80 2023-07-06T03:45:15.165 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:15.165 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/4255451313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4255451313},{"type":"v1","addr":"172.21.15.97:6836","nonce":4255451313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/370322164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":370322164},{"type":"v1","addr":"172.21.15.97:6837","nonce":370322164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3764820847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3764820847},{"type":"v1","addr":"172.21.15.146:6835","nonce":3764820847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:45:11.126281+0000","modified":"2023-07-06T03:45:14.685499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6833/186544257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":186544257},{"type":"v1","addr":"172.21.15.146:6833","nonce":186544257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T03:45:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 -- 172.21.15.97:0/1802421093 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f47e006ace0 msgr2=0x7f47e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f47e006ace0 0x7f47e006d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47e4000c00 tx=0x7f47e4007000 comp rx=0 tx=0).stop 2023-07-06T03:45:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 -- 172.21.15.97:0/1802421093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40b3d80 msgr2=0x7f47f40b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40b3d80 0x7f47f40b5e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f47e8011aa0 tx=0x7f47e8011db0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 -- 172.21.15.97:0/1802421093 shutdown_connections 2023-07-06T03:45:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f47e006ace0 0x7f47e006d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f40b6360 0x7f47f41f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47f414d030 0x7f47f415b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 --2- 172.21.15.97:0/1802421093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40b3d80 0x7f47f40b5e20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 -- 172.21.15.97:0/1802421093 >> 172.21.15.97:0/1802421093 conn(0x7f47dc003a00 msgr2=0x7f47f40ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 -- 172.21.15.97:0/1802421093 shutdown_connections 2023-07-06T03:45:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.165+0000 7f47f9fff700 1 -- 172.21.15.97:0/1802421093 wait complete. 2023-07-06T03:45:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-07-06T03:45:15.185 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:15.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.378+0000 7fb9c37ff700 1 Processor -- start 2023-07-06T03:45:15.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.378+0000 7fb9c37ff700 1 -- start start 2023-07-06T03:45:15.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.379+0000 7fb9c37ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9bc15c520 0x7fb9bc0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:15.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.379+0000 7fb9c37ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9bc0cdae0 con 0x7fb9bc160d00 2023-07-06T03:45:15.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.379+0000 7fb9c37ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9bc0b6640 con 0x7fb9bc160180 2023-07-06T03:45:15.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.379+0000 7fb9c37ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9bc0cd8d0 con 0x7fb9bc15c520 2023-07-06T03:45:15.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.380+0000 7fb9c159b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9bc15c520 0x7fb9bc0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:15.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.380+0000 7fb9c1d9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb9bc160d00 0x7fb9bc0b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56038/0 (socket says 172.21.15.97:56038) 2023-07-06T03:45:15.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.380+0000 7fb9c1d9c700 1 -- 172.21.15.97:0/3076621957 learned_addr learned my addr 172.21.15.97:0/3076621957 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:15.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.380+0000 7fb9c159b700 1 -- 172.21.15.97:0/3076621957 >> v1:172.21.15.146:6789/0 conn(0x7fb9bc160180 legacy=0x7fb9bc1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.380+0000 7fb9c159b700 1 -- 172.21.15.97:0/3076621957 >> v1:172.21.15.97:6789/0 conn(0x7fb9bc160d00 legacy=0x7fb9bc0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.380+0000 7fb9c159b700 1 -- 172.21.15.97:0/3076621957 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc0b6920 con 0x7fb9bc15c520 2023-07-06T03:45:15.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.380+0000 7fb9b3fff700 1 -- 172.21.15.97:0/3076621957 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319140694 0 0) 0x7fb9bc0cdae0 con 0x7fb9bc160d00 2023-07-06T03:45:15.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.380+0000 7fb9c159b700 1 --2- 172.21.15.97:0/3076621957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9bc15c520 0x7fb9bc0cd390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009840 tx=0x7fb9b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43068cd7b9f84824 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:15.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.380+0000 7fb9b3fff700 1 -- 172.21.15.97:0/3076621957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8011830 con 0x7fb9bc15c520 2023-07-06T03:45:15.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.380+0000 7fb9b3fff700 1 -- 172.21.15.97:0/3076621957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b8011990 con 0x7fb9bc15c520 2023-07-06T03:45:15.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.380+0000 7fb9b3fff700 1 -- 172.21.15.97:0/3076621957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8011d20 con 0x7fb9bc15c520 2023-07-06T03:45:15.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.381+0000 7fb9c37ff700 1 -- 172.21.15.97:0/3076621957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9bc15c520 msgr2=0x7fb9bc0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.381+0000 7fb9c37ff700 1 --2- 172.21.15.97:0/3076621957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9bc15c520 0x7fb9bc0cd390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009840 tx=0x7fb9b8009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:15.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.381+0000 7fb9c37ff700 1 -- 172.21.15.97:0/3076621957 shutdown_connections 2023-07-06T03:45:15.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.381+0000 7fb9c37ff700 1 --2- 172.21.15.97:0/3076621957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9bc15c520 0x7fb9bc0cd390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.381+0000 7fb9c37ff700 1 -- 172.21.15.97:0/3076621957 >> 172.21.15.97:0/3076621957 conn(0x7fb9bc071790 msgr2=0x7fb9bc071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:15.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.381+0000 7fb9c37ff700 1 -- 172.21.15.97:0/3076621957 shutdown_connections 2023-07-06T03:45:15.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.381+0000 7fb9c37ff700 1 -- 172.21.15.97:0/3076621957 wait complete. 2023-07-06T03:45:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.383+0000 7fb9c37ff700 1 Processor -- start 2023-07-06T03:45:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.383+0000 7fb9c37ff700 1 -- start start 2023-07-06T03:45:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c37ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9bc15c520 0x7fb9bc14e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c37ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9bc160180 0x7fb9bc1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c37ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9bc160d00 0x7fb9bc15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c37ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc153a00 con 0x7fb9bc160d00 2023-07-06T03:45:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c37ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc150400 con 0x7fb9bc160180 2023-07-06T03:45:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c37ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc15d0a0 con 0x7fb9bc15c520 2023-07-06T03:45:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c1d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9bc160d00 0x7fb9bc15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c1d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9bc160d00 0x7fb9bc15cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46860/0 (socket says 172.21.15.97:46860) 2023-07-06T03:45:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c1d9c700 1 -- 172.21.15.97:0/1626115062 learned_addr learned my addr 172.21.15.97:0/1626115062 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c0d9a700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9bc160180 0x7fb9bc1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:15.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c159b700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9bc15c520 0x7fb9bc14e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:15.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c1d9c700 1 -- 172.21.15.97:0/1626115062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9bc15c520 msgr2=0x7fb9bc14e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c1d9c700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9bc15c520 0x7fb9bc14e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c1d9c700 1 -- 172.21.15.97:0/1626115062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9bc160180 msgr2=0x7fb9bc1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c1d9c700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9bc160180 0x7fb9bc1534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.384+0000 7fb9c1d9c700 1 -- 172.21.15.97:0/1626115062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b80094a0 con 0x7fb9bc160d00 2023-07-06T03:45:15.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.385+0000 7fb9c1d9c700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9bc160d00 0x7fb9bc15cb60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400cca0 tx=0x7fb9b400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:15.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.385+0000 7fb9b1ffb700 1 -- 172.21.15.97:0/1626115062 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b400cea0 con 0x7fb9bc160d00 2023-07-06T03:45:15.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.385+0000 7fb9b1ffb700 1 -- 172.21.15.97:0/1626115062 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b400ad80 con 0x7fb9bc160d00 2023-07-06T03:45:15.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.385+0000 7fb9c37ff700 1 -- 172.21.15.97:0/1626115062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9bc14d780 con 0x7fb9bc160d00 2023-07-06T03:45:15.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.385+0000 7fb9b1ffb700 1 -- 172.21.15.97:0/1626115062 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4007310 con 0x7fb9bc160d00 2023-07-06T03:45:15.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.385+0000 7fb9c37ff700 1 -- 172.21.15.97:0/1626115062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc1e79d0 con 0x7fb9bc160d00 2023-07-06T03:45:15.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.386+0000 7fb9c37ff700 1 -- 172.21.15.97:0/1626115062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9bc150260 con 0x7fb9bc160d00 2023-07-06T03:45:15.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.387+0000 7fb9b1ffb700 1 -- 172.21.15.97:0/1626115062 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb9b4007470 con 0x7fb9bc160d00 2023-07-06T03:45:15.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.388+0000 7fb9b1ffb700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb9a006ace0 0x7fb9a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:15.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.388+0000 7fb9b1ffb700 1 -- 172.21.15.97:0/1626115062 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb9b4088ff0 con 0x7fb9bc160d00 2023-07-06T03:45:15.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.388+0000 7fb9c159b700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb9a006ace0 0x7fb9a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:15.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.388+0000 7fb9c159b700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb9a006ace0 0x7fb9a006d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8000c00 tx=0x7fb9b801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:15.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.392+0000 7fb9b1ffb700 1 -- 172.21.15.97:0/1626115062 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb9bc150260 con 0x7fb9bc160d00 2023-07-06T03:45:15.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.585+0000 7fb9c37ff700 1 -- 172.21.15.97:0/1626115062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9bc150260 con 0x7fb9bc160d00 2023-07-06T03:45:15.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.587+0000 7fb9b1ffb700 1 -- 172.21.15.97:0/1626115062 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7fb9bc150260 con 0x7fb9bc160d00 2023-07-06T03:45:15.589 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:15.589 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:15.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 -- 172.21.15.97:0/1626115062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb9a006ace0 msgr2=0x7fb9a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb9a006ace0 0x7fb9a006d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8000c00 tx=0x7fb9b801c040 comp rx=0 tx=0).stop 2023-07-06T03:45:15.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 -- 172.21.15.97:0/1626115062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9bc160d00 msgr2=0x7fb9bc15cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9bc160d00 0x7fb9bc15cb60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400cca0 tx=0x7fb9b400a340 comp rx=0 tx=0).stop 2023-07-06T03:45:15.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 -- 172.21.15.97:0/1626115062 shutdown_connections 2023-07-06T03:45:15.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb9a006ace0 0x7fb9a006d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9bc160d00 0x7fb9bc15cb60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9bc15c520 0x7fb9bc14e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 --2- 172.21.15.97:0/1626115062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9bc160180 0x7fb9bc1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 -- 172.21.15.97:0/1626115062 >> 172.21.15.97:0/1626115062 conn(0x7fb9bc071790 msgr2=0x7fb9bc156c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:15.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 -- 172.21.15.97:0/1626115062 shutdown_connections 2023-07-06T03:45:15.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.589+0000 7fb9c37ff700 1 -- 172.21.15.97:0/1626115062 wait complete. 2023-07-06T03:45:15.609 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T03:45:15.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:45:15.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.802+0000 7f90607ff700 1 Processor -- start 2023-07-06T03:45:15.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.802+0000 7f90607ff700 1 -- start start 2023-07-06T03:45:15.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.803+0000 7f90607ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f90580cd390 0x7f9058160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:15.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.803+0000 7f90607ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9058160dd0 0x7f90580b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:15.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.803+0000 7f90607ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90580cdae0 con 0x7f905815c520 2023-07-06T03:45:15.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.803+0000 7f90607ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90580b6640 con 0x7f90580cd390 2023-07-06T03:45:15.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.803+0000 7f90607ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90580b6920 con 0x7f9058160dd0 2023-07-06T03:45:15.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.803+0000 7f905dd9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f90580cd390 0x7f9058160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:15.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.803+0000 7f905e59b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f905815c520 0x7f9058160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56044/0 (socket says 172.21.15.97:56044) 2023-07-06T03:45:15.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.803+0000 7f905dd9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f90580cd390 0x7f9058160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46876/0 (socket says 172.21.15.97:46876) 2023-07-06T03:45:15.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.803+0000 7f905e59b700 1 -- 172.21.15.97:0/2509835951 learned_addr learned my addr 172.21.15.97:0/2509835951 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:15.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.804+0000 7f905ed9c700 1 --2- 172.21.15.97:0/2509835951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9058160dd0 0x7f90580b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:15.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.804+0000 7f905dd9a700 1 -- 172.21.15.97:0/2509835951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9058160dd0 msgr2=0x7f90580b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:15.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.804+0000 7f905dd9a700 1 --2- 172.21.15.97:0/2509835951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9058160dd0 0x7f90580b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.804+0000 7f905dd9a700 1 -- 172.21.15.97:0/2509835951 >> v1:172.21.15.97:6789/0 conn(0x7f905815c520 legacy=0x7f9058160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.804+0000 7f905dd9a700 1 -- 172.21.15.97:0/2509835951 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90580b6c00 con 0x7f90580cd390 2023-07-06T03:45:15.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.804+0000 7f905d599700 1 -- 172.21.15.97:0/2509835951 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4072316909 0 0) 0x7f90580cdae0 con 0x7f905815c520 2023-07-06T03:45:15.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.804+0000 7f905dd9a700 1 --2- 172.21.15.97:0/2509835951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90580cd390 0x7f9058160890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9048009840 tx=0x7f9048009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20be7572ed282f7b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:15.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.804+0000 7f905d599700 1 -- 172.21.15.97:0/2509835951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048011830 con 0x7f90580cd390 2023-07-06T03:45:15.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.804+0000 7f905d599700 1 -- 172.21.15.97:0/2509835951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9048011990 con 0x7f90580cd390 2023-07-06T03:45:15.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.804+0000 7f905d599700 1 -- 172.21.15.97:0/2509835951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048011d20 con 0x7f90580cd390 2023-07-06T03:45:15.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.805+0000 7f90607ff700 1 -- 172.21.15.97:0/2509835951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90580cd390 msgr2=0x7f9058160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.805+0000 7f90607ff700 1 --2- 172.21.15.97:0/2509835951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90580cd390 0x7f9058160890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9048009840 tx=0x7f9048009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:15.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.805+0000 7f90607ff700 1 -- 172.21.15.97:0/2509835951 shutdown_connections 2023-07-06T03:45:15.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.805+0000 7f90607ff700 1 --2- 172.21.15.97:0/2509835951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9058160dd0 0x7f90580b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.805+0000 7f90607ff700 1 --2- 172.21.15.97:0/2509835951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90580cd390 0x7f9058160890 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.805+0000 7f90607ff700 1 -- 172.21.15.97:0/2509835951 >> 172.21.15.97:0/2509835951 conn(0x7f9058071790 msgr2=0x7f9058071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.805+0000 7f90607ff700 1 -- 172.21.15.97:0/2509835951 shutdown_connections 2023-07-06T03:45:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.805+0000 7f90607ff700 1 -- 172.21.15.97:0/2509835951 wait complete. 2023-07-06T03:45:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.807+0000 7f90607ff700 1 Processor -- start 2023-07-06T03:45:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.807+0000 7f90607ff700 1 -- start start 2023-07-06T03:45:15.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f90607ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90580cd390 0x7f905815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:15.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f90607ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f905815c520 0x7f90580c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:15.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f90607ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9058160dd0 0x7f90581f5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f90607ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905815eaf0 con 0x7f9058160dd0 2023-07-06T03:45:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f90607ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905815bcc0 con 0x7f90580cd390 2023-07-06T03:45:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f90607ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905815d800 con 0x7f905815c520 2023-07-06T03:45:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f905ed9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9058160dd0 0x7f90581f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f905ed9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9058160dd0 0x7f90581f5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46884/0 (socket says 172.21.15.97:46884) 2023-07-06T03:45:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f905ed9c700 1 -- 172.21.15.97:0/4033834785 learned_addr learned my addr 172.21.15.97:0/4033834785 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f905dd9a700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f905815c520 0x7f90580c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f905e59b700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90580cd390 0x7f905815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f905ed9c700 1 -- 172.21.15.97:0/4033834785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f905815c520 msgr2=0x7f90580c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f905ed9c700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f905815c520 0x7f90580c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f905ed9c700 1 -- 172.21.15.97:0/4033834785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90580cd390 msgr2=0x7f905815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f905ed9c700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90580cd390 0x7f905815b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f905ed9c700 1 -- 172.21.15.97:0/4033834785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90480094a0 con 0x7f9058160dd0 2023-07-06T03:45:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.808+0000 7f905e59b700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90580cd390 0x7f905815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.809+0000 7f905ed9c700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9058160dd0 0x7f90581f5880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9050007e70 tx=0x7f905000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.809+0000 7f904effd700 1 -- 172.21.15.97:0/4033834785 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905000edf0 con 0x7f9058160dd0 2023-07-06T03:45:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.809+0000 7f904effd700 1 -- 172.21.15.97:0/4033834785 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050014420 con 0x7f9058160dd0 2023-07-06T03:45:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.809+0000 7f90607ff700 1 -- 172.21.15.97:0/4033834785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905815e660 con 0x7f9058160dd0 2023-07-06T03:45:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.809+0000 7f904effd700 1 -- 172.21.15.97:0/4033834785 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90500147d0 con 0x7f9058160dd0 2023-07-06T03:45:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.809+0000 7f90607ff700 1 -- 172.21.15.97:0/4033834785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905815dd30 con 0x7f9058160dd0 2023-07-06T03:45:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.810+0000 7f90607ff700 1 -- 172.21.15.97:0/4033834785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905815e7f0 con 0x7f9058160dd0 2023-07-06T03:45:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.811+0000 7f904effd700 1 -- 172.21.15.97:0/4033834785 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9050014930 con 0x7f9058160dd0 2023-07-06T03:45:15.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.812+0000 7f904effd700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f903c06ace0 0x7f903c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:15.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.812+0000 7f904effd700 1 -- 172.21.15.97:0/4033834785 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9050089c20 con 0x7f9058160dd0 2023-07-06T03:45:15.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.812+0000 7f905e59b700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f903c06ace0 0x7f903c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:15.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.813+0000 7f905e59b700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f903c06ace0 0x7f903c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9054000f60 tx=0x7f9054009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:15.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:15.816+0000 7f904effd700 1 -- 172.21.15.97:0/4033834785 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9050054cd0 con 0x7f9058160dd0 2023-07-06T03:45:16.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.009+0000 7f90607ff700 1 -- 172.21.15.97:0/4033834785 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90580cd800 con 0x7f903c06ace0 2023-07-06T03:45:16.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.019+0000 7f904effd700 1 -- 172.21.15.97:0/4033834785 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281423 (secure 0 0 0) 0x7f9054001cc0 con 0x7f903c06ace0 2023-07-06T03:45:16.026 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:16.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 -- 172.21.15.97:0/4033834785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f903c06ace0 msgr2=0x7f903c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f903c06ace0 0x7f903c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9054000f60 tx=0x7f9054009040 comp rx=0 tx=0).stop 2023-07-06T03:45:16.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 -- 172.21.15.97:0/4033834785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9058160dd0 msgr2=0x7f90581f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9058160dd0 0x7f90581f5880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9050007e70 tx=0x7f905000e3c0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 -- 172.21.15.97:0/4033834785 shutdown_connections 2023-07-06T03:45:16.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f903c06ace0 0x7f903c06d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9058160dd0 0x7f90581f5880 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90580cd390 0x7f905815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 --2- 172.21.15.97:0/4033834785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f905815c520 0x7f90580c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 -- 172.21.15.97:0/4033834785 >> 172.21.15.97:0/4033834785 conn(0x7f9058071790 msgr2=0x7f90580b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:16.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 -- 172.21.15.97:0/4033834785 shutdown_connections 2023-07-06T03:45:16.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.023+0000 7f90607ff700 1 -- 172.21.15.97:0/4033834785 wait complete. 2023-07-06T03:45:16.032 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:45:16.043 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-07-06T03:45:14.882854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218420,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756300,"statfs":{"total":773094113280,"available":772870451200,"internally_reserved":0,"allocated":2871296,"data_stored":1759749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.255727"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.646335+0000","last_change":"2023-07-06T03:45:10.646335+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.646335+0000","last_undegraded":"2023-07-06T03:45:10.646335+0000","last_fullsized":"2023-07-06T03:45:10.646335+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.636786+0000","last_change":"2023-07-06T03:45:10.636786+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.636786+0000","last_undegraded":"2023-07-06T03:45:10.636786+0000","last_fullsized":"2023-07-06T03:45:10.636786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640495+0000","last_change":"2023-07-06T03:45:10.640495+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640495+0000","last_undegraded":"2023-07-06T03:45:10.640495+0000","last_fullsized":"2023-07-06T03:45:10.640495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640262+0000","last_change":"2023-07-06T03:45:10.640262+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640262+0000","last_undegraded":"2023-07-06T03:45:10.640262+0000","last_fullsized":"2023-07-06T03:45:10.640262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640322+0000","last_change":"2023-07-06T03:45:10.640322+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640322+0000","last_undegraded":"2023-07-06T03:45:10.640322+0000","last_fullsized":"2023-07-06T03:45:10.640322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632337+0000","last_change":"2023-07-06T03:45:10.632337+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632337+0000","last_undegraded":"2023-07-06T03:45:10.632337+0000","last_fullsized":"2023-07-06T03:45:10.632337+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.633030+0000","last_change":"2023-07-06T03:45:10.633030+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.633030+0000","last_undegraded":"2023-07-06T03:45:10.633030+0000","last_fullsized":"2023-07-06T03:45:10.633030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.643381+0000","last_change":"2023-07-06T03:45:10.643381+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.643381+0000","last_undegraded":"2023-07-06T03:45:10.643381+0000","last_fullsized":"2023-07-06T03:45:10.643381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.644127+0000","last_change":"2023-07-06T03:45:10.644127+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.644127+0000","last_undegraded":"2023-07-06T03:45:10.644127+0000","last_fullsized":"2023-07-06T03:45:10.644127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.642582+0000","last_change":"2023-07-06T03:45:10.642582+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.642582+0000","last_undegraded":"2023-07-06T03:45:10.642582+0000","last_fullsized":"2023-07-06T03:45:10.642582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.642966+0000","last_change":"2023-07-06T03:45:10.642966+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.642966+0000","last_undegraded":"2023-07-06T03:45:10.642966+0000","last_fullsized":"2023-07-06T03:45:10.642966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.636404+0000","last_change":"2023-07-06T03:45:10.636404+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.636404+0000","last_undegraded":"2023-07-06T03:45:10.636404+0000","last_fullsized":"2023-07-06T03:45:10.636404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.642356+0000","last_change":"2023-07-06T03:45:10.642356+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.642356+0000","last_undegraded":"2023-07-06T03:45:10.642356+0000","last_fullsized":"2023-07-06T03:45:10.642356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.638948+0000","last_change":"2023-07-06T03:45:10.638948+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.638948+0000","last_undegraded":"2023-07-06T03:45:10.638948+0000","last_fullsized":"2023-07-06T03:45:10.638948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.642316+0000","last_change":"2023-07-06T03:45:10.642316+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.642316+0000","last_undegraded":"2023-07-06T03:45:10.642316+0000","last_fullsized":"2023-07-06T03:45:10.642316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632041+0000","last_change":"2023-07-06T03:45:10.632041+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632041+0000","last_undegraded":"2023-07-06T03:45:10.632041+0000","last_fullsized":"2023-07-06T03:45:10.632041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.637396+0000","last_change":"2023-07-06T03:45:10.637396+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.637396+0000","last_undegraded":"2023-07-06T03:45:10.637396+0000","last_fullsized":"2023-07-06T03:45:10.637396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640795+0000","last_change":"2023-07-06T03:45:10.640795+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640795+0000","last_undegraded":"2023-07-06T03:45:10.640795+0000","last_fullsized":"2023-07-06T03:45:10.640795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.636951+0000","last_change":"2023-07-06T03:45:10.636951+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.636951+0000","last_undegraded":"2023-07-06T03:45:10.636951+0000","last_fullsized":"2023-07-06T03:45:10.636951+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639619+0000","last_change":"2023-07-06T03:45:10.639619+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639619+0000","last_undegraded":"2023-07-06T03:45:10.639619+0000","last_fullsized":"2023-07-06T03:45:10.639619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641830+0000","last_change":"2023-07-06T03:45:10.641830+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641830+0000","last_undegraded":"2023-07-06T03:45:10.641830+0000","last_fullsized":"2023-07-06T03:45:10.641830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635782+0000","last_change":"2023-07-06T03:45:10.635782+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635782+0000","last_undegraded":"2023-07-06T03:45:10.635782+0000","last_fullsized":"2023-07-06T03:45:10.635782+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639649+0000","last_change":"2023-07-06T03:45:10.639649+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639649+0000","last_undegraded":"2023-07-06T03:45:10.639649+0000","last_fullsized":"2023-07-06T03:45:10.639649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641852+0000","last_change":"2023-07-06T03:45:10.641852+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641852+0000","last_undegraded":"2023-07-06T03:45:10.641852+0000","last_fullsized":"2023-07-06T03:45:10.641852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641582+0000","last_change":"2023-07-06T03:45:10.641582+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641582+0000","last_undegraded":"2023-07-06T03:45:10.641582+0000","last_fullsized":"2023-07-06T03:45:10.641582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635787+0000","last_change":"2023-07-06T03:45:10.635787+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635787+0000","last_undegraded":"2023-07-06T03:45:10.635787+0000","last_fullsized":"2023-07-06T03:45:10.635787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.634471+0000","last_change":"2023-07-06T03:45:10.634471+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.634471+0000","last_undegraded":"2023-07-06T03:45:10.634471+0000","last_fullsized":"2023-07-06T03:45:10.634471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.657693+0000","last_change":"2023-07-06T03:45:10.656464+0000","last_active":"2023-07-06T03:45:10.657693+0000","last_peered":"2023-07-06T03:45:10.657693+0000","last_clean":"2023-07-06T03:45:10.657693+0000","last_became_active":"2023-07-06T03:45:10.656116+0000","last_became_peered":"2023-07-06T03:45:10.656116+0000","last_unstale":"2023-07-06T03:45:10.657693+0000","last_undegraded":"2023-07-06T03:45:10.657693+0000","last_fullsized":"2023-07-06T03:45:10.657693+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:23.874792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635685+0000","last_change":"2023-07-06T03:45:10.635685+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635685+0000","last_undegraded":"2023-07-06T03:45:10.635685+0000","last_fullsized":"2023-07-06T03:45:10.635685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.662152+0000","last_change":"2023-07-06T03:45:10.657736+0000","last_active":"2023-07-06T03:45:10.662152+0000","last_peered":"2023-07-06T03:45:10.662152+0000","last_clean":"2023-07-06T03:45:10.662152+0000","last_became_active":"2023-07-06T03:45:10.657450+0000","last_became_peered":"2023-07-06T03:45:10.657450+0000","last_unstale":"2023-07-06T03:45:10.662152+0000","last_undegraded":"2023-07-06T03:45:10.662152+0000","last_fullsized":"2023-07-06T03:45:10.662152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:36.746033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639627+0000","last_change":"2023-07-06T03:45:10.639627+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639627+0000","last_undegraded":"2023-07-06T03:45:10.639627+0000","last_fullsized":"2023-07-06T03:45:10.639627+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.660151+0000","last_change":"2023-07-06T03:45:10.653666+0000","last_active":"2023-07-06T03:45:10.660151+0000","last_peered":"2023-07-06T03:45:10.660151+0000","last_clean":"2023-07-06T03:45:10.660151+0000","last_became_active":"2023-07-06T03:45:10.653357+0000","last_became_peered":"2023-07-06T03:45:10.653357+0000","last_unstale":"2023-07-06T03:45:10.660151+0000","last_undegraded":"2023-07-06T03:45:10.660151+0000","last_fullsized":"2023-07-06T03:45:10.660151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:05:07.859127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.633261+0000","last_change":"2023-07-06T03:45:10.633261+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.633261+0000","last_undegraded":"2023-07-06T03:45:10.633261+0000","last_fullsized":"2023-07-06T03:45:10.633261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.659914+0000","last_change":"2023-07-06T03:45:10.649734+0000","last_active":"2023-07-06T03:45:10.659914+0000","last_peered":"2023-07-06T03:45:10.659914+0000","last_clean":"2023-07-06T03:45:10.659914+0000","last_became_active":"2023-07-06T03:45:10.649401+0000","last_became_peered":"2023-07-06T03:45:10.649401+0000","last_unstale":"2023-07-06T03:45:10.659914+0000","last_undegraded":"2023-07-06T03:45:10.659914+0000","last_fullsized":"2023-07-06T03:45:10.659914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:11.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635901+0000","last_change":"2023-07-06T03:45:10.635901+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635901+0000","last_undegraded":"2023-07-06T03:45:10.635901+0000","last_fullsized":"2023-07-06T03:45:10.635901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.656704+0000","last_change":"2023-07-06T03:45:10.648660+0000","last_active":"2023-07-06T03:45:10.656704+0000","last_peered":"2023-07-06T03:45:10.656704+0000","last_clean":"2023-07-06T03:45:10.656704+0000","last_became_active":"2023-07-06T03:45:10.648254+0000","last_became_peered":"2023-07-06T03:45:10.648254+0000","last_unstale":"2023-07-06T03:45:10.656704+0000","last_undegraded":"2023-07-06T03:45:10.656704+0000","last_fullsized":"2023-07-06T03:45:10.656704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:43:52.346163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641858+0000","last_change":"2023-07-06T03:45:10.641858+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641858+0000","last_undegraded":"2023-07-06T03:45:10.641858+0000","last_fullsized":"2023-07-06T03:45:10.641858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.657255+0000","last_change":"2023-07-06T03:45:10.656590+0000","last_active":"2023-07-06T03:45:10.657255+0000","last_peered":"2023-07-06T03:45:10.657255+0000","last_clean":"2023-07-06T03:45:10.657255+0000","last_became_active":"2023-07-06T03:45:10.656328+0000","last_became_peered":"2023-07-06T03:45:10.656328+0000","last_unstale":"2023-07-06T03:45:10.657255+0000","last_undegraded":"2023-07-06T03:45:10.657255+0000","last_fullsized":"2023-07-06T03:45:10.657255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:10:28.622148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639726+0000","last_change":"2023-07-06T03:45:10.639726+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639726+0000","last_undegraded":"2023-07-06T03:45:10.639726+0000","last_fullsized":"2023-07-06T03:45:10.639726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.649618+0000","last_change":"2023-07-06T03:45:10.644219+0000","last_active":"2023-07-06T03:45:10.649618+0000","last_peered":"2023-07-06T03:45:10.649618+0000","last_clean":"2023-07-06T03:45:10.649618+0000","last_became_active":"2023-07-06T03:45:10.643854+0000","last_became_peered":"2023-07-06T03:45:10.643854+0000","last_unstale":"2023-07-06T03:45:10.649618+0000","last_undegraded":"2023-07-06T03:45:10.649618+0000","last_fullsized":"2023-07-06T03:45:10.649618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:57:15.170218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.631268+0000","last_change":"2023-07-06T03:45:10.631268+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.631268+0000","last_undegraded":"2023-07-06T03:45:10.631268+0000","last_fullsized":"2023-07-06T03:45:10.631268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.657824+0000","last_change":"2023-07-06T03:45:10.653534+0000","last_active":"2023-07-06T03:45:10.657824+0000","last_peered":"2023-07-06T03:45:10.657824+0000","last_clean":"2023-07-06T03:45:10.657824+0000","last_became_active":"2023-07-06T03:45:10.653232+0000","last_became_peered":"2023-07-06T03:45:10.653232+0000","last_unstale":"2023-07-06T03:45:10.657824+0000","last_undegraded":"2023-07-06T03:45:10.657824+0000","last_fullsized":"2023-07-06T03:45:10.657824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:22:54.149661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.643865+0000","last_change":"2023-07-06T03:45:10.643865+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.643865+0000","last_undegraded":"2023-07-06T03:45:10.643865+0000","last_fullsized":"2023-07-06T03:45:10.643865+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.666056+0000","last_change":"2023-07-06T03:45:10.655738+0000","last_active":"2023-07-06T03:45:10.666056+0000","last_peered":"2023-07-06T03:45:10.666056+0000","last_clean":"2023-07-06T03:45:10.666056+0000","last_became_active":"2023-07-06T03:45:10.655414+0000","last_became_peered":"2023-07-06T03:45:10.655414+0000","last_unstale":"2023-07-06T03:45:10.666056+0000","last_undegraded":"2023-07-06T03:45:10.666056+0000","last_fullsized":"2023-07-06T03:45:10.666056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:59.642411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.631296+0000","last_change":"2023-07-06T03:45:10.631296+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.631296+0000","last_undegraded":"2023-07-06T03:45:10.631296+0000","last_fullsized":"2023-07-06T03:45:10.631296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.649225+0000","last_change":"2023-07-06T03:45:10.644075+0000","last_active":"2023-07-06T03:45:10.649225+0000","last_peered":"2023-07-06T03:45:10.649225+0000","last_clean":"2023-07-06T03:45:10.649225+0000","last_became_active":"2023-07-06T03:45:10.643785+0000","last_became_peered":"2023-07-06T03:45:10.643785+0000","last_unstale":"2023-07-06T03:45:10.649225+0000","last_undegraded":"2023-07-06T03:45:10.649225+0000","last_fullsized":"2023-07-06T03:45:10.649225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:18:10.750680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.637415+0000","last_change":"2023-07-06T03:45:10.637415+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.637415+0000","last_undegraded":"2023-07-06T03:45:10.637415+0000","last_fullsized":"2023-07-06T03:45:10.637415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.655246+0000","last_change":"2023-07-06T03:45:10.652546+0000","last_active":"2023-07-06T03:45:10.655246+0000","last_peered":"2023-07-06T03:45:10.655246+0000","last_clean":"2023-07-06T03:45:10.655246+0000","last_became_active":"2023-07-06T03:45:10.652314+0000","last_became_peered":"2023-07-06T03:45:10.652314+0000","last_unstale":"2023-07-06T03:45:10.655246+0000","last_undegraded":"2023-07-06T03:45:10.655246+0000","last_fullsized":"2023-07-06T03:45:10.655246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:11.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639277+0000","last_change":"2023-07-06T03:45:10.639277+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639277+0000","last_undegraded":"2023-07-06T03:45:10.639277+0000","last_fullsized":"2023-07-06T03:45:10.639277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.655427+0000","last_change":"2023-07-06T03:45:10.654280+0000","last_active":"2023-07-06T03:45:10.655427+0000","last_peered":"2023-07-06T03:45:10.655427+0000","last_clean":"2023-07-06T03:45:10.655427+0000","last_became_active":"2023-07-06T03:45:10.654039+0000","last_became_peered":"2023-07-06T03:45:10.654039+0000","last_unstale":"2023-07-06T03:45:10.655427+0000","last_undegraded":"2023-07-06T03:45:10.655427+0000","last_fullsized":"2023-07-06T03:45:10.655427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:59.642411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.631273+0000","last_change":"2023-07-06T03:45:10.631273+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.631273+0000","last_undegraded":"2023-07-06T03:45:10.631273+0000","last_fullsized":"2023-07-06T03:45:10.631273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.656690+0000","last_change":"2023-07-06T03:45:10.647142+0000","last_active":"2023-07-06T03:45:10.656690+0000","last_peered":"2023-07-06T03:45:10.656690+0000","last_clean":"2023-07-06T03:45:10.656690+0000","last_became_active":"2023-07-06T03:45:10.646882+0000","last_became_peered":"2023-07-06T03:45:10.646882+0000","last_unstale":"2023-07-06T03:45:10.656690+0000","last_undegraded":"2023-07-06T03:45:10.656690+0000","last_fullsized":"2023-07-06T03:45:10.656690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:56:58.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640373+0000","last_change":"2023-07-06T03:45:10.640373+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640373+0000","last_undegraded":"2023-07-06T03:45:10.640373+0000","last_fullsized":"2023-07-06T03:45:10.640373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.654115+0000","last_change":"2023-07-06T03:45:10.647677+0000","last_active":"2023-07-06T03:45:10.654115+0000","last_peered":"2023-07-06T03:45:10.654115+0000","last_clean":"2023-07-06T03:45:10.654115+0000","last_became_active":"2023-07-06T03:45:10.647414+0000","last_became_peered":"2023-07-06T03:45:10.647414+0000","last_unstale":"2023-07-06T03:45:10.654115+0000","last_undegraded":"2023-07-06T03:45:10.654115+0000","last_fullsized":"2023-07-06T03:45:10.654115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:38.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.631999+0000","last_change":"2023-07-06T03:45:10.631999+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.631999+0000","last_undegraded":"2023-07-06T03:45:10.631999+0000","last_fullsized":"2023-07-06T03:45:10.631999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.653985+0000","last_change":"2023-07-06T03:45:10.651255+0000","last_active":"2023-07-06T03:45:10.653985+0000","last_peered":"2023-07-06T03:45:10.653985+0000","last_clean":"2023-07-06T03:45:10.653985+0000","last_became_active":"2023-07-06T03:45:10.651012+0000","last_became_peered":"2023-07-06T03:45:10.651012+0000","last_unstale":"2023-07-06T03:45:10.653985+0000","last_undegraded":"2023-07-06T03:45:10.653985+0000","last_fullsized":"2023-07-06T03:45:10.653985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:43:52.346163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.643645+0000","last_change":"2023-07-06T03:45:10.643645+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.643645+0000","last_undegraded":"2023-07-06T03:45:10.643645+0000","last_fullsized":"2023-07-06T03:45:10.643645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.659222+0000","last_change":"2023-07-06T03:45:10.653064+0000","last_active":"2023-07-06T03:45:10.659222+0000","last_peered":"2023-07-06T03:45:10.659222+0000","last_clean":"2023-07-06T03:45:10.659222+0000","last_became_active":"2023-07-06T03:45:10.652765+0000","last_became_peered":"2023-07-06T03:45:10.652765+0000","last_unstale":"2023-07-06T03:45:10.659222+0000","last_undegraded":"2023-07-06T03:45:10.659222+0000","last_fullsized":"2023-07-06T03:45:10.659222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:05:07.859127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639361+0000","last_change":"2023-07-06T03:45:10.639361+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639361+0000","last_undegraded":"2023-07-06T03:45:10.639361+0000","last_fullsized":"2023-07-06T03:45:10.639361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.657925+0000","last_change":"2023-07-06T03:45:10.650363+0000","last_active":"2023-07-06T03:45:10.657925+0000","last_peered":"2023-07-06T03:45:10.657925+0000","last_clean":"2023-07-06T03:45:10.657925+0000","last_became_active":"2023-07-06T03:45:10.650097+0000","last_became_peered":"2023-07-06T03:45:10.650097+0000","last_unstale":"2023-07-06T03:45:10.657925+0000","last_undegraded":"2023-07-06T03:45:10.657925+0000","last_fullsized":"2023-07-06T03:45:10.657925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:13:41.819003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632457+0000","last_change":"2023-07-06T03:45:10.632457+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632457+0000","last_undegraded":"2023-07-06T03:45:10.632457+0000","last_fullsized":"2023-07-06T03:45:10.632457+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.662018+0000","last_change":"2023-07-06T03:45:10.661165+0000","last_active":"2023-07-06T03:45:10.662018+0000","last_peered":"2023-07-06T03:45:10.662018+0000","last_clean":"2023-07-06T03:45:10.662018+0000","last_became_active":"2023-07-06T03:45:10.660843+0000","last_became_peered":"2023-07-06T03:45:10.660843+0000","last_unstale":"2023-07-06T03:45:10.662018+0000","last_undegraded":"2023-07-06T03:45:10.662018+0000","last_fullsized":"2023-07-06T03:45:10.662018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:43.801501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.636289+0000","last_change":"2023-07-06T03:45:10.636289+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.636289+0000","last_undegraded":"2023-07-06T03:45:10.636289+0000","last_fullsized":"2023-07-06T03:45:10.636289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.662189+0000","last_change":"2023-07-06T03:45:10.658201+0000","last_active":"2023-07-06T03:45:10.662189+0000","last_peered":"2023-07-06T03:45:10.662189+0000","last_clean":"2023-07-06T03:45:10.662189+0000","last_became_active":"2023-07-06T03:45:10.657688+0000","last_became_peered":"2023-07-06T03:45:10.657688+0000","last_unstale":"2023-07-06T03:45:10.662189+0000","last_undegraded":"2023-07-06T03:45:10.662189+0000","last_fullsized":"2023-07-06T03:45:10.662189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:01:48.414029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635680+0000","last_change":"2023-07-06T03:45:10.635680+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635680+0000","last_undegraded":"2023-07-06T03:45:10.635680+0000","last_fullsized":"2023-07-06T03:45:10.635680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.649954+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:45:10.649954+0000","last_peered":"2023-07-06T03:45:10.649954+0000","last_clean":"2023-07-06T03:45:10.649954+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:45:10.649954+0000","last_undegraded":"2023-07-06T03:45:10.649954+0000","last_fullsized":"2023-07-06T03:45:10.649954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:01:42.910402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.661492+0000","last_change":"2023-07-06T03:45:10.660307+0000","last_active":"2023-07-06T03:45:10.661492+0000","last_peered":"2023-07-06T03:45:10.661492+0000","last_clean":"2023-07-06T03:45:10.661492+0000","last_became_active":"2023-07-06T03:45:10.660032+0000","last_became_peered":"2023-07-06T03:45:10.660032+0000","last_unstale":"2023-07-06T03:45:10.661492+0000","last_undegraded":"2023-07-06T03:45:10.661492+0000","last_fullsized":"2023-07-06T03:45:10.661492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:59.642411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639898+0000","last_change":"2023-07-06T03:45:10.639898+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639898+0000","last_undegraded":"2023-07-06T03:45:10.639898+0000","last_fullsized":"2023-07-06T03:45:10.639898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.633750+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:45:10.633750+0000","last_peered":"2023-07-06T03:45:10.633750+0000","last_clean":"2023-07-06T03:45:10.633750+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:45:10.633750+0000","last_undegraded":"2023-07-06T03:45:10.633750+0000","last_fullsized":"2023-07-06T03:45:10.633750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:11.498012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.653941+0000","last_change":"2023-07-06T03:45:10.649547+0000","last_active":"2023-07-06T03:45:10.653941+0000","last_peered":"2023-07-06T03:45:10.653941+0000","last_clean":"2023-07-06T03:45:10.653941+0000","last_became_active":"2023-07-06T03:45:10.649231+0000","last_became_peered":"2023-07-06T03:45:10.649231+0000","last_unstale":"2023-07-06T03:45:10.653941+0000","last_undegraded":"2023-07-06T03:45:10.653941+0000","last_fullsized":"2023-07-06T03:45:10.653941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:56:58.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641932+0000","last_change":"2023-07-06T03:45:10.641932+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641932+0000","last_undegraded":"2023-07-06T03:45:10.641932+0000","last_fullsized":"2023-07-06T03:45:10.641932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.632269+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:45:10.632269+0000","last_peered":"2023-07-06T03:45:10.632269+0000","last_clean":"2023-07-06T03:45:10.632269+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:45:10.632269+0000","last_undegraded":"2023-07-06T03:45:10.632269+0000","last_fullsized":"2023-07-06T03:45:10.632269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:10:23.118520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.658148+0000","last_change":"2023-07-06T03:45:10.647981+0000","last_active":"2023-07-06T03:45:10.658148+0000","last_peered":"2023-07-06T03:45:10.658148+0000","last_clean":"2023-07-06T03:45:10.658148+0000","last_became_active":"2023-07-06T03:45:10.647678+0000","last_became_peered":"2023-07-06T03:45:10.647678+0000","last_unstale":"2023-07-06T03:45:10.658148+0000","last_undegraded":"2023-07-06T03:45:10.658148+0000","last_fullsized":"2023-07-06T03:45:10.658148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:36.746033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639203+0000","last_change":"2023-07-06T03:45:10.639203+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639203+0000","last_undegraded":"2023-07-06T03:45:10.639203+0000","last_fullsized":"2023-07-06T03:45:10.639203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.637829+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:45:10.637829+0000","last_peered":"2023-07-06T03:45:10.637829+0000","last_clean":"2023-07-06T03:45:10.637829+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:45:10.637829+0000","last_undegraded":"2023-07-06T03:45:10.637829+0000","last_fullsized":"2023-07-06T03:45:10.637829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:07.992862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.636647+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:45:10.636647+0000","last_peered":"2023-07-06T03:45:10.636647+0000","last_clean":"2023-07-06T03:45:10.636647+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:45:10.636647+0000","last_undegraded":"2023-07-06T03:45:10.636647+0000","last_fullsized":"2023-07-06T03:45:10.636647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:24.207282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.661537+0000","last_change":"2023-07-06T03:45:10.657235+0000","last_active":"2023-07-06T03:45:10.661537+0000","last_peered":"2023-07-06T03:45:10.661537+0000","last_clean":"2023-07-06T03:45:10.661537+0000","last_became_active":"2023-07-06T03:45:10.656945+0000","last_became_peered":"2023-07-06T03:45:10.656945+0000","last_unstale":"2023-07-06T03:45:10.661537+0000","last_undegraded":"2023-07-06T03:45:10.661537+0000","last_fullsized":"2023-07-06T03:45:10.661537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:05:07.859127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632056+0000","last_change":"2023-07-06T03:45:10.632056+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632056+0000","last_undegraded":"2023-07-06T03:45:10.632056+0000","last_fullsized":"2023-07-06T03:45:10.632056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.633553+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:45:10.633553+0000","last_peered":"2023-07-06T03:45:10.633553+0000","last_clean":"2023-07-06T03:45:10.633553+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:45:10.633553+0000","last_undegraded":"2023-07-06T03:45:10.633553+0000","last_fullsized":"2023-07-06T03:45:10.633553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:02:59.779305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.666141+0000","last_change":"2023-07-06T03:45:10.660900+0000","last_active":"2023-07-06T03:45:10.666141+0000","last_peered":"2023-07-06T03:45:10.666141+0000","last_clean":"2023-07-06T03:45:10.666141+0000","last_became_active":"2023-07-06T03:45:10.660490+0000","last_became_peered":"2023-07-06T03:45:10.660490+0000","last_unstale":"2023-07-06T03:45:10.666141+0000","last_undegraded":"2023-07-06T03:45:10.666141+0000","last_fullsized":"2023-07-06T03:45:10.666141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:43:52.346163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.636474+0000","last_change":"2023-07-06T03:45:10.636474+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.636474+0000","last_undegraded":"2023-07-06T03:45:10.636474+0000","last_fullsized":"2023-07-06T03:45:10.636474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.637506+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:45:10.637506+0000","last_peered":"2023-07-06T03:45:10.637506+0000","last_clean":"2023-07-06T03:45:10.637506+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:45:10.637506+0000","last_undegraded":"2023-07-06T03:45:10.637506+0000","last_fullsized":"2023-07-06T03:45:10.637506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:41.368994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.666172+0000","last_change":"2023-07-06T03:45:10.661812+0000","last_active":"2023-07-06T03:45:10.666172+0000","last_peered":"2023-07-06T03:45:10.666172+0000","last_clean":"2023-07-06T03:45:10.666172+0000","last_became_active":"2023-07-06T03:45:10.661499+0000","last_became_peered":"2023-07-06T03:45:10.661499+0000","last_unstale":"2023-07-06T03:45:10.666172+0000","last_undegraded":"2023-07-06T03:45:10.666172+0000","last_fullsized":"2023-07-06T03:45:10.666172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:56:58.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.638646+0000","last_change":"2023-07-06T03:45:10.638646+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.638646+0000","last_undegraded":"2023-07-06T03:45:10.638646+0000","last_fullsized":"2023-07-06T03:45:10.638646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.661732+0000","last_change":"2023-07-06T03:45:10.657429+0000","last_active":"2023-07-06T03:45:10.661732+0000","last_peered":"2023-07-06T03:45:10.661732+0000","last_clean":"2023-07-06T03:45:10.661732+0000","last_became_active":"2023-07-06T03:45:10.657085+0000","last_became_peered":"2023-07-06T03:45:10.657085+0000","last_unstale":"2023-07-06T03:45:10.661732+0000","last_undegraded":"2023-07-06T03:45:10.661732+0000","last_fullsized":"2023-07-06T03:45:10.661732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:23.874792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.648590+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:45:10.648590+0000","last_peered":"2023-07-06T03:45:10.648590+0000","last_clean":"2023-07-06T03:45:10.648590+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:45:10.648590+0000","last_undegraded":"2023-07-06T03:45:10.648590+0000","last_fullsized":"2023-07-06T03:45:10.648590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:39:41.393077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.638574+0000","last_change":"2023-07-06T03:45:10.638574+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.638574+0000","last_undegraded":"2023-07-06T03:45:10.638574+0000","last_fullsized":"2023-07-06T03:45:10.638574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.639518+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:45:10.639518+0000","last_peered":"2023-07-06T03:45:10.639518+0000","last_clean":"2023-07-06T03:45:10.639518+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:45:10.639518+0000","last_undegraded":"2023-07-06T03:45:10.639518+0000","last_fullsized":"2023-07-06T03:45:10.639518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:53.585611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.662161+0000","last_change":"2023-07-06T03:45:10.657912+0000","last_active":"2023-07-06T03:45:10.662161+0000","last_peered":"2023-07-06T03:45:10.662161+0000","last_clean":"2023-07-06T03:45:10.662161+0000","last_became_active":"2023-07-06T03:45:10.657615+0000","last_became_peered":"2023-07-06T03:45:10.657615+0000","last_unstale":"2023-07-06T03:45:10.662161+0000","last_undegraded":"2023-07-06T03:45:10.662161+0000","last_fullsized":"2023-07-06T03:45:10.662161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:10:28.622148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632279+0000","last_change":"2023-07-06T03:45:10.632279+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632279+0000","last_undegraded":"2023-07-06T03:45:10.632279+0000","last_fullsized":"2023-07-06T03:45:10.632279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.663451+0000","last_change":"2023-07-06T03:45:10.658517+0000","last_active":"2023-07-06T03:45:10.663451+0000","last_peered":"2023-07-06T03:45:10.663451+0000","last_clean":"2023-07-06T03:45:10.663451+0000","last_became_active":"2023-07-06T03:45:10.657946+0000","last_became_peered":"2023-07-06T03:45:10.657946+0000","last_unstale":"2023-07-06T03:45:10.663451+0000","last_undegraded":"2023-07-06T03:45:10.663451+0000","last_fullsized":"2023-07-06T03:45:10.663451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:11.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640062+0000","last_change":"2023-07-06T03:45:10.640062+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640062+0000","last_undegraded":"2023-07-06T03:45:10.640062+0000","last_fullsized":"2023-07-06T03:45:10.640062+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.654834+0000","last_change":"2023-07-06T03:45:10.649186+0000","last_active":"2023-07-06T03:45:10.654834+0000","last_peered":"2023-07-06T03:45:10.654834+0000","last_clean":"2023-07-06T03:45:10.654834+0000","last_became_active":"2023-07-06T03:45:10.648707+0000","last_became_peered":"2023-07-06T03:45:10.648707+0000","last_unstale":"2023-07-06T03:45:10.654834+0000","last_undegraded":"2023-07-06T03:45:10.654834+0000","last_fullsized":"2023-07-06T03:45:10.654834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:01:48.414029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632953+0000","last_change":"2023-07-06T03:45:10.632953+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632953+0000","last_undegraded":"2023-07-06T03:45:10.632953+0000","last_fullsized":"2023-07-06T03:45:10.632953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.657765+0000","last_change":"2023-07-06T03:45:10.653276+0000","last_active":"2023-07-06T03:45:10.657765+0000","last_peered":"2023-07-06T03:45:10.657765+0000","last_clean":"2023-07-06T03:45:10.657765+0000","last_became_active":"2023-07-06T03:45:10.652958+0000","last_became_peered":"2023-07-06T03:45:10.652958+0000","last_unstale":"2023-07-06T03:45:10.657765+0000","last_undegraded":"2023-07-06T03:45:10.657765+0000","last_fullsized":"2023-07-06T03:45:10.657765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:23.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632470+0000","last_change":"2023-07-06T03:45:10.632470+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632470+0000","last_undegraded":"2023-07-06T03:45:10.632470+0000","last_fullsized":"2023-07-06T03:45:10.632470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.648167+0000","last_change":"2023-07-06T03:45:10.645296+0000","last_active":"2023-07-06T03:45:10.648167+0000","last_peered":"2023-07-06T03:45:10.648167+0000","last_clean":"2023-07-06T03:45:10.648167+0000","last_became_active":"2023-07-06T03:45:10.644967+0000","last_became_peered":"2023-07-06T03:45:10.644967+0000","last_unstale":"2023-07-06T03:45:10.648167+0000","last_undegraded":"2023-07-06T03:45:10.648167+0000","last_fullsized":"2023-07-06T03:45:10.648167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:10:28.622148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635490+0000","last_change":"2023-07-06T03:45:10.635490+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635490+0000","last_undegraded":"2023-07-06T03:45:10.635490+0000","last_fullsized":"2023-07-06T03:45:10.635490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.660155+0000","last_change":"2023-07-06T03:45:10.654706+0000","last_active":"2023-07-06T03:45:10.660155+0000","last_peered":"2023-07-06T03:45:10.660155+0000","last_clean":"2023-07-06T03:45:10.660155+0000","last_became_active":"2023-07-06T03:45:10.653995+0000","last_became_peered":"2023-07-06T03:45:10.653995+0000","last_unstale":"2023-07-06T03:45:10.660155+0000","last_undegraded":"2023-07-06T03:45:10.660155+0000","last_fullsized":"2023-07-06T03:45:10.660155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:56:58.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.631366+0000","last_change":"2023-07-06T03:45:10.631366+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.631366+0000","last_undegraded":"2023-07-06T03:45:10.631366+0000","last_fullsized":"2023-07-06T03:45:10.631366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641388+0000","last_change":"2023-07-06T03:45:10.641388+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641388+0000","last_undegraded":"2023-07-06T03:45:10.641388+0000","last_fullsized":"2023-07-06T03:45:10.641388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641341+0000","last_change":"2023-07-06T03:45:10.641341+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641341+0000","last_undegraded":"2023-07-06T03:45:10.641341+0000","last_fullsized":"2023-07-06T03:45:10.641341+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639222+0000","last_change":"2023-07-06T03:45:10.639222+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639222+0000","last_undegraded":"2023-07-06T03:45:10.639222+0000","last_fullsized":"2023-07-06T03:45:10.639222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.637982+0000","last_change":"2023-07-06T03:45:10.637982+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.637982+0000","last_undegraded":"2023-07-06T03:45:10.637982+0000","last_fullsized":"2023-07-06T03:45:10.637982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.637307+0000","last_change":"2023-07-06T03:45:10.637307+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.637307+0000","last_undegraded":"2023-07-06T03:45:10.637307+0000","last_fullsized":"2023-07-06T03:45:10.637307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":811008,"data_stored":666186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6679999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":794624,"data_stored":658539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:45:16.054 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:45:16.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.276+0000 7fda1fbff700 1 Processor -- start 2023-07-06T03:45:16.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.276+0000 7fda1fbff700 1 -- start start 2023-07-06T03:45:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.277+0000 7fda1fbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda180cd390 0x7fda18160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.277+0000 7fda1fbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fda18160dd0 0x7fda180b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.277+0000 7fda1fbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda180cdae0 con 0x7fda1815c520 2023-07-06T03:45:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.277+0000 7fda1fbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda180b6640 con 0x7fda180cd390 2023-07-06T03:45:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.277+0000 7fda1fbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda180b6920 con 0x7fda18160dd0 2023-07-06T03:45:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.277+0000 7fda1d99b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fda1815c520 0x7fda18160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56050/0 (socket says 172.21.15.97:56050) 2023-07-06T03:45:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.277+0000 7fda1d99b700 1 -- 172.21.15.97:0/640800441 learned_addr learned my addr 172.21.15.97:0/640800441 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.277+0000 7fda1d19a700 1 --2- 172.21.15.97:0/640800441 >> v2:172.21.15.97:3300/0 conn(0x7fda180cd390 0x7fda18160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.277+0000 7fda1e19c700 1 --2- 172.21.15.97:0/640800441 >> v2:172.21.15.146:3300/0 conn(0x7fda18160dd0 0x7fda180b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.278+0000 7fda1d19a700 1 -- 172.21.15.97:0/640800441 >> v2:172.21.15.146:3300/0 conn(0x7fda18160dd0 msgr2=0x7fda180b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.278+0000 7fda1d19a700 1 --2- 172.21.15.97:0/640800441 >> v2:172.21.15.146:3300/0 conn(0x7fda18160dd0 0x7fda180b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.278+0000 7fda1d19a700 1 -- 172.21.15.97:0/640800441 >> v1:172.21.15.97:6789/0 conn(0x7fda1815c520 legacy=0x7fda18160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.278+0000 7fda1d19a700 1 -- 172.21.15.97:0/640800441 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda180b6c00 con 0x7fda180cd390 2023-07-06T03:45:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.278+0000 7fda1e19c700 1 --2- 172.21.15.97:0/640800441 >> v2:172.21.15.146:3300/0 conn(0x7fda18160dd0 0x7fda180b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.278+0000 7fda1d19a700 1 --2- 172.21.15.97:0/640800441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda180cd390 0x7fda18160890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fda180cd320 tx=0x7fda08009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce10e2d0b4f5a5a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.278+0000 7fda1c999700 1 -- 172.21.15.97:0/640800441 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08011830 con 0x7fda180cd390 2023-07-06T03:45:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.278+0000 7fda1c999700 1 -- 172.21.15.97:0/640800441 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda08011990 con 0x7fda180cd390 2023-07-06T03:45:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.278+0000 7fda1c999700 1 -- 172.21.15.97:0/640800441 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08011d20 con 0x7fda180cd390 2023-07-06T03:45:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.279+0000 7fda1fbff700 1 -- 172.21.15.97:0/640800441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda180cd390 msgr2=0x7fda18160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.279+0000 7fda1fbff700 1 --2- 172.21.15.97:0/640800441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda180cd390 0x7fda18160890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fda180cd320 tx=0x7fda08009b20 comp rx=0 tx=0).stop 2023-07-06T03:45:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.279+0000 7fda1fbff700 1 -- 172.21.15.97:0/640800441 shutdown_connections 2023-07-06T03:45:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.279+0000 7fda1fbff700 1 --2- 172.21.15.97:0/640800441 >> v2:172.21.15.146:3300/0 conn(0x7fda18160dd0 0x7fda180b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.279+0000 7fda1fbff700 1 --2- 172.21.15.97:0/640800441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda180cd390 0x7fda18160890 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.279+0000 7fda1fbff700 1 -- 172.21.15.97:0/640800441 >> 172.21.15.97:0/640800441 conn(0x7fda18071790 msgr2=0x7fda18071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.279+0000 7fda1fbff700 1 -- 172.21.15.97:0/640800441 shutdown_connections 2023-07-06T03:45:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.279+0000 7fda1fbff700 1 -- 172.21.15.97:0/640800441 wait complete. 2023-07-06T03:45:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.280+0000 7fda1fbff700 1 Processor -- start 2023-07-06T03:45:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.281+0000 7fda1fbff700 1 -- start start 2023-07-06T03:45:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.281+0000 7fda1fbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda180cd390 0x7fda1814d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.281+0000 7fda1fbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda1815c520 0x7fda180d35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:16.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1fbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda18160dd0 0x7fda18159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1fbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda180d31c0 con 0x7fda180cd390 2023-07-06T03:45:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1fbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda180d2840 con 0x7fda1815c520 2023-07-06T03:45:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1fbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda18159ca0 con 0x7fda18160dd0 2023-07-06T03:45:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1d99b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda180cd390 0x7fda1814d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1d99b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda180cd390 0x7fda1814d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46914/0 (socket says 172.21.15.97:46914) 2023-07-06T03:45:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1d99b700 1 -- 172.21.15.97:0/43451216 learned_addr learned my addr 172.21.15.97:0/43451216 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1d99b700 1 -- 172.21.15.97:0/43451216 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda18160dd0 msgr2=0x7fda18159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1d99b700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda18160dd0 0x7fda18159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1d19a700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda1815c520 0x7fda180d35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:16.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1d99b700 1 -- 172.21.15.97:0/43451216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda1815c520 msgr2=0x7fda180d35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1d99b700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda1815c520 0x7fda180d35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1d99b700 1 -- 172.21.15.97:0/43451216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda080094a0 con 0x7fda180cd390 2023-07-06T03:45:16.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1d19a700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda1815c520 0x7fda180d35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:16.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda1d99b700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda180cd390 0x7fda1814d390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fda1400c520 tx=0x7fda1400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:16.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.282+0000 7fda0e7fc700 1 -- 172.21.15.97:0/43451216 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14005d70 con 0x7fda180cd390 2023-07-06T03:45:16.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.283+0000 7fda0e7fc700 1 -- 172.21.15.97:0/43451216 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda140070d0 con 0x7fda180cd390 2023-07-06T03:45:16.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.283+0000 7fda1fbff700 1 -- 172.21.15.97:0/43451216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda180d2b00 con 0x7fda180cd390 2023-07-06T03:45:16.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.283+0000 7fda0e7fc700 1 -- 172.21.15.97:0/43451216 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14007440 con 0x7fda180cd390 2023-07-06T03:45:16.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.283+0000 7fda1fbff700 1 -- 172.21.15.97:0/43451216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda18158ed0 con 0x7fda180cd390 2023-07-06T03:45:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.284+0000 7fda1fbff700 1 -- 172.21.15.97:0/43451216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda180d3430 con 0x7fda180cd390 2023-07-06T03:45:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.285+0000 7fda0e7fc700 1 -- 172.21.15.97:0/43451216 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda140075a0 con 0x7fda180cd390 2023-07-06T03:45:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.285+0000 7fda0e7fc700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd9fc06ace0 0x7fd9fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.286+0000 7fda0e7fc700 1 -- 172.21.15.97:0/43451216 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fda14088930 con 0x7fda180cd390 2023-07-06T03:45:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.286+0000 7fda1d19a700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd9fc06ace0 0x7fd9fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.286+0000 7fda1d19a700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd9fc06ace0 0x7fd9fc06d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda180b3c80 tx=0x7fda08008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:16.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.290+0000 7fda0e7fc700 1 -- 172.21.15.97:0/43451216 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda140539f0 con 0x7fda180cd390 2023-07-06T03:45:16.330 INFO:tasks.ceph.mds.b.smithi146.stderr:2023-07-06T03:45:16.328+0000 7fe7673f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:45:16.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.483+0000 7fda1fbff700 1 -- 172.21.15.97:0/43451216 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fda180cd800 con 0x7fd9fc06ace0 2023-07-06T03:45:16.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.491+0000 7fda0e7fc700 1 -- 172.21.15.97:0/43451216 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281423 (secure 0 0 0) 0x7fda08011b00 con 0x7fd9fc06ace0 2023-07-06T03:45:16.499 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 -- 172.21.15.97:0/43451216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd9fc06ace0 msgr2=0x7fd9fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd9fc06ace0 0x7fd9fc06d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda180b3c80 tx=0x7fda08008030 comp rx=0 tx=0).stop 2023-07-06T03:45:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 -- 172.21.15.97:0/43451216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda180cd390 msgr2=0x7fda1814d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda180cd390 0x7fda1814d390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fda1400c520 tx=0x7fda1400c8e0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 -- 172.21.15.97:0/43451216 shutdown_connections 2023-07-06T03:45:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd9fc06ace0 0x7fd9fc06d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda18160dd0 0x7fda18159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda180cd390 0x7fda1814d390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 --2- 172.21.15.97:0/43451216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda1815c520 0x7fda180d35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 -- 172.21.15.97:0/43451216 >> 172.21.15.97:0/43451216 conn(0x7fda18071790 msgr2=0x7fda180b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 -- 172.21.15.97:0/43451216 shutdown_connections 2023-07-06T03:45:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.496+0000 7fda1fbff700 1 -- 172.21.15.97:0/43451216 wait complete. 2023-07-06T03:45:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:45:16.516 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-07-06T03:45:14.882854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218420,"kb_used_data":2804,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756300,"statfs":{"total":773094113280,"available":772870451200,"internally_reserved":0,"allocated":2871296,"data_stored":1759749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.255727"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.646335+0000","last_change":"2023-07-06T03:45:10.646335+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.646335+0000","last_undegraded":"2023-07-06T03:45:10.646335+0000","last_fullsized":"2023-07-06T03:45:10.646335+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.636786+0000","last_change":"2023-07-06T03:45:10.636786+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.636786+0000","last_undegraded":"2023-07-06T03:45:10.636786+0000","last_fullsized":"2023-07-06T03:45:10.636786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640495+0000","last_change":"2023-07-06T03:45:10.640495+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640495+0000","last_undegraded":"2023-07-06T03:45:10.640495+0000","last_fullsized":"2023-07-06T03:45:10.640495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640262+0000","last_change":"2023-07-06T03:45:10.640262+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640262+0000","last_undegraded":"2023-07-06T03:45:10.640262+0000","last_fullsized":"2023-07-06T03:45:10.640262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640322+0000","last_change":"2023-07-06T03:45:10.640322+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640322+0000","last_undegraded":"2023-07-06T03:45:10.640322+0000","last_fullsized":"2023-07-06T03:45:10.640322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632337+0000","last_change":"2023-07-06T03:45:10.632337+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632337+0000","last_undegraded":"2023-07-06T03:45:10.632337+0000","last_fullsized":"2023-07-06T03:45:10.632337+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.633030+0000","last_change":"2023-07-06T03:45:10.633030+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.633030+0000","last_undegraded":"2023-07-06T03:45:10.633030+0000","last_fullsized":"2023-07-06T03:45:10.633030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.643381+0000","last_change":"2023-07-06T03:45:10.643381+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.643381+0000","last_undegraded":"2023-07-06T03:45:10.643381+0000","last_fullsized":"2023-07-06T03:45:10.643381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.644127+0000","last_change":"2023-07-06T03:45:10.644127+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.644127+0000","last_undegraded":"2023-07-06T03:45:10.644127+0000","last_fullsized":"2023-07-06T03:45:10.644127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.642582+0000","last_change":"2023-07-06T03:45:10.642582+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.642582+0000","last_undegraded":"2023-07-06T03:45:10.642582+0000","last_fullsized":"2023-07-06T03:45:10.642582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.642966+0000","last_change":"2023-07-06T03:45:10.642966+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.642966+0000","last_undegraded":"2023-07-06T03:45:10.642966+0000","last_fullsized":"2023-07-06T03:45:10.642966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.636404+0000","last_change":"2023-07-06T03:45:10.636404+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.636404+0000","last_undegraded":"2023-07-06T03:45:10.636404+0000","last_fullsized":"2023-07-06T03:45:10.636404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.642356+0000","last_change":"2023-07-06T03:45:10.642356+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.642356+0000","last_undegraded":"2023-07-06T03:45:10.642356+0000","last_fullsized":"2023-07-06T03:45:10.642356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.638948+0000","last_change":"2023-07-06T03:45:10.638948+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.638948+0000","last_undegraded":"2023-07-06T03:45:10.638948+0000","last_fullsized":"2023-07-06T03:45:10.638948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.642316+0000","last_change":"2023-07-06T03:45:10.642316+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.642316+0000","last_undegraded":"2023-07-06T03:45:10.642316+0000","last_fullsized":"2023-07-06T03:45:10.642316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632041+0000","last_change":"2023-07-06T03:45:10.632041+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632041+0000","last_undegraded":"2023-07-06T03:45:10.632041+0000","last_fullsized":"2023-07-06T03:45:10.632041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.637396+0000","last_change":"2023-07-06T03:45:10.637396+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.637396+0000","last_undegraded":"2023-07-06T03:45:10.637396+0000","last_fullsized":"2023-07-06T03:45:10.637396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640795+0000","last_change":"2023-07-06T03:45:10.640795+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640795+0000","last_undegraded":"2023-07-06T03:45:10.640795+0000","last_fullsized":"2023-07-06T03:45:10.640795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.636951+0000","last_change":"2023-07-06T03:45:10.636951+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.636951+0000","last_undegraded":"2023-07-06T03:45:10.636951+0000","last_fullsized":"2023-07-06T03:45:10.636951+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639619+0000","last_change":"2023-07-06T03:45:10.639619+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639619+0000","last_undegraded":"2023-07-06T03:45:10.639619+0000","last_fullsized":"2023-07-06T03:45:10.639619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641830+0000","last_change":"2023-07-06T03:45:10.641830+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641830+0000","last_undegraded":"2023-07-06T03:45:10.641830+0000","last_fullsized":"2023-07-06T03:45:10.641830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635782+0000","last_change":"2023-07-06T03:45:10.635782+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635782+0000","last_undegraded":"2023-07-06T03:45:10.635782+0000","last_fullsized":"2023-07-06T03:45:10.635782+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639649+0000","last_change":"2023-07-06T03:45:10.639649+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639649+0000","last_undegraded":"2023-07-06T03:45:10.639649+0000","last_fullsized":"2023-07-06T03:45:10.639649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641852+0000","last_change":"2023-07-06T03:45:10.641852+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641852+0000","last_undegraded":"2023-07-06T03:45:10.641852+0000","last_fullsized":"2023-07-06T03:45:10.641852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641582+0000","last_change":"2023-07-06T03:45:10.641582+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641582+0000","last_undegraded":"2023-07-06T03:45:10.641582+0000","last_fullsized":"2023-07-06T03:45:10.641582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635787+0000","last_change":"2023-07-06T03:45:10.635787+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635787+0000","last_undegraded":"2023-07-06T03:45:10.635787+0000","last_fullsized":"2023-07-06T03:45:10.635787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.634471+0000","last_change":"2023-07-06T03:45:10.634471+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.634471+0000","last_undegraded":"2023-07-06T03:45:10.634471+0000","last_fullsized":"2023-07-06T03:45:10.634471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.657693+0000","last_change":"2023-07-06T03:45:10.656464+0000","last_active":"2023-07-06T03:45:10.657693+0000","last_peered":"2023-07-06T03:45:10.657693+0000","last_clean":"2023-07-06T03:45:10.657693+0000","last_became_active":"2023-07-06T03:45:10.656116+0000","last_became_peered":"2023-07-06T03:45:10.656116+0000","last_unstale":"2023-07-06T03:45:10.657693+0000","last_undegraded":"2023-07-06T03:45:10.657693+0000","last_fullsized":"2023-07-06T03:45:10.657693+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:23.874792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635685+0000","last_change":"2023-07-06T03:45:10.635685+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635685+0000","last_undegraded":"2023-07-06T03:45:10.635685+0000","last_fullsized":"2023-07-06T03:45:10.635685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.662152+0000","last_change":"2023-07-06T03:45:10.657736+0000","last_active":"2023-07-06T03:45:10.662152+0000","last_peered":"2023-07-06T03:45:10.662152+0000","last_clean":"2023-07-06T03:45:10.662152+0000","last_became_active":"2023-07-06T03:45:10.657450+0000","last_became_peered":"2023-07-06T03:45:10.657450+0000","last_unstale":"2023-07-06T03:45:10.662152+0000","last_undegraded":"2023-07-06T03:45:10.662152+0000","last_fullsized":"2023-07-06T03:45:10.662152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:36.746033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639627+0000","last_change":"2023-07-06T03:45:10.639627+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639627+0000","last_undegraded":"2023-07-06T03:45:10.639627+0000","last_fullsized":"2023-07-06T03:45:10.639627+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.660151+0000","last_change":"2023-07-06T03:45:10.653666+0000","last_active":"2023-07-06T03:45:10.660151+0000","last_peered":"2023-07-06T03:45:10.660151+0000","last_clean":"2023-07-06T03:45:10.660151+0000","last_became_active":"2023-07-06T03:45:10.653357+0000","last_became_peered":"2023-07-06T03:45:10.653357+0000","last_unstale":"2023-07-06T03:45:10.660151+0000","last_undegraded":"2023-07-06T03:45:10.660151+0000","last_fullsized":"2023-07-06T03:45:10.660151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:05:07.859127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.633261+0000","last_change":"2023-07-06T03:45:10.633261+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.633261+0000","last_undegraded":"2023-07-06T03:45:10.633261+0000","last_fullsized":"2023-07-06T03:45:10.633261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.659914+0000","last_change":"2023-07-06T03:45:10.649734+0000","last_active":"2023-07-06T03:45:10.659914+0000","last_peered":"2023-07-06T03:45:10.659914+0000","last_clean":"2023-07-06T03:45:10.659914+0000","last_became_active":"2023-07-06T03:45:10.649401+0000","last_became_peered":"2023-07-06T03:45:10.649401+0000","last_unstale":"2023-07-06T03:45:10.659914+0000","last_undegraded":"2023-07-06T03:45:10.659914+0000","last_fullsized":"2023-07-06T03:45:10.659914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:11.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635901+0000","last_change":"2023-07-06T03:45:10.635901+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635901+0000","last_undegraded":"2023-07-06T03:45:10.635901+0000","last_fullsized":"2023-07-06T03:45:10.635901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.656704+0000","last_change":"2023-07-06T03:45:10.648660+0000","last_active":"2023-07-06T03:45:10.656704+0000","last_peered":"2023-07-06T03:45:10.656704+0000","last_clean":"2023-07-06T03:45:10.656704+0000","last_became_active":"2023-07-06T03:45:10.648254+0000","last_became_peered":"2023-07-06T03:45:10.648254+0000","last_unstale":"2023-07-06T03:45:10.656704+0000","last_undegraded":"2023-07-06T03:45:10.656704+0000","last_fullsized":"2023-07-06T03:45:10.656704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:43:52.346163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641858+0000","last_change":"2023-07-06T03:45:10.641858+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641858+0000","last_undegraded":"2023-07-06T03:45:10.641858+0000","last_fullsized":"2023-07-06T03:45:10.641858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.657255+0000","last_change":"2023-07-06T03:45:10.656590+0000","last_active":"2023-07-06T03:45:10.657255+0000","last_peered":"2023-07-06T03:45:10.657255+0000","last_clean":"2023-07-06T03:45:10.657255+0000","last_became_active":"2023-07-06T03:45:10.656328+0000","last_became_peered":"2023-07-06T03:45:10.656328+0000","last_unstale":"2023-07-06T03:45:10.657255+0000","last_undegraded":"2023-07-06T03:45:10.657255+0000","last_fullsized":"2023-07-06T03:45:10.657255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:10:28.622148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639726+0000","last_change":"2023-07-06T03:45:10.639726+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639726+0000","last_undegraded":"2023-07-06T03:45:10.639726+0000","last_fullsized":"2023-07-06T03:45:10.639726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.649618+0000","last_change":"2023-07-06T03:45:10.644219+0000","last_active":"2023-07-06T03:45:10.649618+0000","last_peered":"2023-07-06T03:45:10.649618+0000","last_clean":"2023-07-06T03:45:10.649618+0000","last_became_active":"2023-07-06T03:45:10.643854+0000","last_became_peered":"2023-07-06T03:45:10.643854+0000","last_unstale":"2023-07-06T03:45:10.649618+0000","last_undegraded":"2023-07-06T03:45:10.649618+0000","last_fullsized":"2023-07-06T03:45:10.649618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:57:15.170218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.631268+0000","last_change":"2023-07-06T03:45:10.631268+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.631268+0000","last_undegraded":"2023-07-06T03:45:10.631268+0000","last_fullsized":"2023-07-06T03:45:10.631268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.657824+0000","last_change":"2023-07-06T03:45:10.653534+0000","last_active":"2023-07-06T03:45:10.657824+0000","last_peered":"2023-07-06T03:45:10.657824+0000","last_clean":"2023-07-06T03:45:10.657824+0000","last_became_active":"2023-07-06T03:45:10.653232+0000","last_became_peered":"2023-07-06T03:45:10.653232+0000","last_unstale":"2023-07-06T03:45:10.657824+0000","last_undegraded":"2023-07-06T03:45:10.657824+0000","last_fullsized":"2023-07-06T03:45:10.657824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:22:54.149661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.643865+0000","last_change":"2023-07-06T03:45:10.643865+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.643865+0000","last_undegraded":"2023-07-06T03:45:10.643865+0000","last_fullsized":"2023-07-06T03:45:10.643865+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.666056+0000","last_change":"2023-07-06T03:45:10.655738+0000","last_active":"2023-07-06T03:45:10.666056+0000","last_peered":"2023-07-06T03:45:10.666056+0000","last_clean":"2023-07-06T03:45:10.666056+0000","last_became_active":"2023-07-06T03:45:10.655414+0000","last_became_peered":"2023-07-06T03:45:10.655414+0000","last_unstale":"2023-07-06T03:45:10.666056+0000","last_undegraded":"2023-07-06T03:45:10.666056+0000","last_fullsized":"2023-07-06T03:45:10.666056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:59.642411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.631296+0000","last_change":"2023-07-06T03:45:10.631296+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.631296+0000","last_undegraded":"2023-07-06T03:45:10.631296+0000","last_fullsized":"2023-07-06T03:45:10.631296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.649225+0000","last_change":"2023-07-06T03:45:10.644075+0000","last_active":"2023-07-06T03:45:10.649225+0000","last_peered":"2023-07-06T03:45:10.649225+0000","last_clean":"2023-07-06T03:45:10.649225+0000","last_became_active":"2023-07-06T03:45:10.643785+0000","last_became_peered":"2023-07-06T03:45:10.643785+0000","last_unstale":"2023-07-06T03:45:10.649225+0000","last_undegraded":"2023-07-06T03:45:10.649225+0000","last_fullsized":"2023-07-06T03:45:10.649225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:18:10.750680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.637415+0000","last_change":"2023-07-06T03:45:10.637415+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.637415+0000","last_undegraded":"2023-07-06T03:45:10.637415+0000","last_fullsized":"2023-07-06T03:45:10.637415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.655246+0000","last_change":"2023-07-06T03:45:10.652546+0000","last_active":"2023-07-06T03:45:10.655246+0000","last_peered":"2023-07-06T03:45:10.655246+0000","last_clean":"2023-07-06T03:45:10.655246+0000","last_became_active":"2023-07-06T03:45:10.652314+0000","last_became_peered":"2023-07-06T03:45:10.652314+0000","last_unstale":"2023-07-06T03:45:10.655246+0000","last_undegraded":"2023-07-06T03:45:10.655246+0000","last_fullsized":"2023-07-06T03:45:10.655246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:11.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639277+0000","last_change":"2023-07-06T03:45:10.639277+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639277+0000","last_undegraded":"2023-07-06T03:45:10.639277+0000","last_fullsized":"2023-07-06T03:45:10.639277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.655427+0000","last_change":"2023-07-06T03:45:10.654280+0000","last_active":"2023-07-06T03:45:10.655427+0000","last_peered":"2023-07-06T03:45:10.655427+0000","last_clean":"2023-07-06T03:45:10.655427+0000","last_became_active":"2023-07-06T03:45:10.654039+0000","last_became_peered":"2023-07-06T03:45:10.654039+0000","last_unstale":"2023-07-06T03:45:10.655427+0000","last_undegraded":"2023-07-06T03:45:10.655427+0000","last_fullsized":"2023-07-06T03:45:10.655427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:59.642411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.631273+0000","last_change":"2023-07-06T03:45:10.631273+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.631273+0000","last_undegraded":"2023-07-06T03:45:10.631273+0000","last_fullsized":"2023-07-06T03:45:10.631273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.656690+0000","last_change":"2023-07-06T03:45:10.647142+0000","last_active":"2023-07-06T03:45:10.656690+0000","last_peered":"2023-07-06T03:45:10.656690+0000","last_clean":"2023-07-06T03:45:10.656690+0000","last_became_active":"2023-07-06T03:45:10.646882+0000","last_became_peered":"2023-07-06T03:45:10.646882+0000","last_unstale":"2023-07-06T03:45:10.656690+0000","last_undegraded":"2023-07-06T03:45:10.656690+0000","last_fullsized":"2023-07-06T03:45:10.656690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:56:58.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640373+0000","last_change":"2023-07-06T03:45:10.640373+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640373+0000","last_undegraded":"2023-07-06T03:45:10.640373+0000","last_fullsized":"2023-07-06T03:45:10.640373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.654115+0000","last_change":"2023-07-06T03:45:10.647677+0000","last_active":"2023-07-06T03:45:10.654115+0000","last_peered":"2023-07-06T03:45:10.654115+0000","last_clean":"2023-07-06T03:45:10.654115+0000","last_became_active":"2023-07-06T03:45:10.647414+0000","last_became_peered":"2023-07-06T03:45:10.647414+0000","last_unstale":"2023-07-06T03:45:10.654115+0000","last_undegraded":"2023-07-06T03:45:10.654115+0000","last_fullsized":"2023-07-06T03:45:10.654115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:38.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.631999+0000","last_change":"2023-07-06T03:45:10.631999+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.631999+0000","last_undegraded":"2023-07-06T03:45:10.631999+0000","last_fullsized":"2023-07-06T03:45:10.631999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.653985+0000","last_change":"2023-07-06T03:45:10.651255+0000","last_active":"2023-07-06T03:45:10.653985+0000","last_peered":"2023-07-06T03:45:10.653985+0000","last_clean":"2023-07-06T03:45:10.653985+0000","last_became_active":"2023-07-06T03:45:10.651012+0000","last_became_peered":"2023-07-06T03:45:10.651012+0000","last_unstale":"2023-07-06T03:45:10.653985+0000","last_undegraded":"2023-07-06T03:45:10.653985+0000","last_fullsized":"2023-07-06T03:45:10.653985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:43:52.346163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.643645+0000","last_change":"2023-07-06T03:45:10.643645+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.643645+0000","last_undegraded":"2023-07-06T03:45:10.643645+0000","last_fullsized":"2023-07-06T03:45:10.643645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.659222+0000","last_change":"2023-07-06T03:45:10.653064+0000","last_active":"2023-07-06T03:45:10.659222+0000","last_peered":"2023-07-06T03:45:10.659222+0000","last_clean":"2023-07-06T03:45:10.659222+0000","last_became_active":"2023-07-06T03:45:10.652765+0000","last_became_peered":"2023-07-06T03:45:10.652765+0000","last_unstale":"2023-07-06T03:45:10.659222+0000","last_undegraded":"2023-07-06T03:45:10.659222+0000","last_fullsized":"2023-07-06T03:45:10.659222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:05:07.859127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639361+0000","last_change":"2023-07-06T03:45:10.639361+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639361+0000","last_undegraded":"2023-07-06T03:45:10.639361+0000","last_fullsized":"2023-07-06T03:45:10.639361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.657925+0000","last_change":"2023-07-06T03:45:10.650363+0000","last_active":"2023-07-06T03:45:10.657925+0000","last_peered":"2023-07-06T03:45:10.657925+0000","last_clean":"2023-07-06T03:45:10.657925+0000","last_became_active":"2023-07-06T03:45:10.650097+0000","last_became_peered":"2023-07-06T03:45:10.650097+0000","last_unstale":"2023-07-06T03:45:10.657925+0000","last_undegraded":"2023-07-06T03:45:10.657925+0000","last_fullsized":"2023-07-06T03:45:10.657925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:13:41.819003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632457+0000","last_change":"2023-07-06T03:45:10.632457+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632457+0000","last_undegraded":"2023-07-06T03:45:10.632457+0000","last_fullsized":"2023-07-06T03:45:10.632457+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.662018+0000","last_change":"2023-07-06T03:45:10.661165+0000","last_active":"2023-07-06T03:45:10.662018+0000","last_peered":"2023-07-06T03:45:10.662018+0000","last_clean":"2023-07-06T03:45:10.662018+0000","last_became_active":"2023-07-06T03:45:10.660843+0000","last_became_peered":"2023-07-06T03:45:10.660843+0000","last_unstale":"2023-07-06T03:45:10.662018+0000","last_undegraded":"2023-07-06T03:45:10.662018+0000","last_fullsized":"2023-07-06T03:45:10.662018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:43.801501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.636289+0000","last_change":"2023-07-06T03:45:10.636289+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.636289+0000","last_undegraded":"2023-07-06T03:45:10.636289+0000","last_fullsized":"2023-07-06T03:45:10.636289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.662189+0000","last_change":"2023-07-06T03:45:10.658201+0000","last_active":"2023-07-06T03:45:10.662189+0000","last_peered":"2023-07-06T03:45:10.662189+0000","last_clean":"2023-07-06T03:45:10.662189+0000","last_became_active":"2023-07-06T03:45:10.657688+0000","last_became_peered":"2023-07-06T03:45:10.657688+0000","last_unstale":"2023-07-06T03:45:10.662189+0000","last_undegraded":"2023-07-06T03:45:10.662189+0000","last_fullsized":"2023-07-06T03:45:10.662189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:01:48.414029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635680+0000","last_change":"2023-07-06T03:45:10.635680+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635680+0000","last_undegraded":"2023-07-06T03:45:10.635680+0000","last_fullsized":"2023-07-06T03:45:10.635680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.649954+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:45:10.649954+0000","last_peered":"2023-07-06T03:45:10.649954+0000","last_clean":"2023-07-06T03:45:10.649954+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:45:10.649954+0000","last_undegraded":"2023-07-06T03:45:10.649954+0000","last_fullsized":"2023-07-06T03:45:10.649954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:01:42.910402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.661492+0000","last_change":"2023-07-06T03:45:10.660307+0000","last_active":"2023-07-06T03:45:10.661492+0000","last_peered":"2023-07-06T03:45:10.661492+0000","last_clean":"2023-07-06T03:45:10.661492+0000","last_became_active":"2023-07-06T03:45:10.660032+0000","last_became_peered":"2023-07-06T03:45:10.660032+0000","last_unstale":"2023-07-06T03:45:10.661492+0000","last_undegraded":"2023-07-06T03:45:10.661492+0000","last_fullsized":"2023-07-06T03:45:10.661492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:59.642411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639898+0000","last_change":"2023-07-06T03:45:10.639898+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639898+0000","last_undegraded":"2023-07-06T03:45:10.639898+0000","last_fullsized":"2023-07-06T03:45:10.639898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.633750+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:45:10.633750+0000","last_peered":"2023-07-06T03:45:10.633750+0000","last_clean":"2023-07-06T03:45:10.633750+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:45:10.633750+0000","last_undegraded":"2023-07-06T03:45:10.633750+0000","last_fullsized":"2023-07-06T03:45:10.633750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:11.498012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.653941+0000","last_change":"2023-07-06T03:45:10.649547+0000","last_active":"2023-07-06T03:45:10.653941+0000","last_peered":"2023-07-06T03:45:10.653941+0000","last_clean":"2023-07-06T03:45:10.653941+0000","last_became_active":"2023-07-06T03:45:10.649231+0000","last_became_peered":"2023-07-06T03:45:10.649231+0000","last_unstale":"2023-07-06T03:45:10.653941+0000","last_undegraded":"2023-07-06T03:45:10.653941+0000","last_fullsized":"2023-07-06T03:45:10.653941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:56:58.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641932+0000","last_change":"2023-07-06T03:45:10.641932+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641932+0000","last_undegraded":"2023-07-06T03:45:10.641932+0000","last_fullsized":"2023-07-06T03:45:10.641932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.632269+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:45:10.632269+0000","last_peered":"2023-07-06T03:45:10.632269+0000","last_clean":"2023-07-06T03:45:10.632269+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:45:10.632269+0000","last_undegraded":"2023-07-06T03:45:10.632269+0000","last_fullsized":"2023-07-06T03:45:10.632269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:10:23.118520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.658148+0000","last_change":"2023-07-06T03:45:10.647981+0000","last_active":"2023-07-06T03:45:10.658148+0000","last_peered":"2023-07-06T03:45:10.658148+0000","last_clean":"2023-07-06T03:45:10.658148+0000","last_became_active":"2023-07-06T03:45:10.647678+0000","last_became_peered":"2023-07-06T03:45:10.647678+0000","last_unstale":"2023-07-06T03:45:10.658148+0000","last_undegraded":"2023-07-06T03:45:10.658148+0000","last_fullsized":"2023-07-06T03:45:10.658148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:36.746033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639203+0000","last_change":"2023-07-06T03:45:10.639203+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639203+0000","last_undegraded":"2023-07-06T03:45:10.639203+0000","last_fullsized":"2023-07-06T03:45:10.639203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.637829+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:45:10.637829+0000","last_peered":"2023-07-06T03:45:10.637829+0000","last_clean":"2023-07-06T03:45:10.637829+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:45:10.637829+0000","last_undegraded":"2023-07-06T03:45:10.637829+0000","last_fullsized":"2023-07-06T03:45:10.637829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:07.992862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.636647+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:45:10.636647+0000","last_peered":"2023-07-06T03:45:10.636647+0000","last_clean":"2023-07-06T03:45:10.636647+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:45:10.636647+0000","last_undegraded":"2023-07-06T03:45:10.636647+0000","last_fullsized":"2023-07-06T03:45:10.636647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:24.207282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.661537+0000","last_change":"2023-07-06T03:45:10.657235+0000","last_active":"2023-07-06T03:45:10.661537+0000","last_peered":"2023-07-06T03:45:10.661537+0000","last_clean":"2023-07-06T03:45:10.661537+0000","last_became_active":"2023-07-06T03:45:10.656945+0000","last_became_peered":"2023-07-06T03:45:10.656945+0000","last_unstale":"2023-07-06T03:45:10.661537+0000","last_undegraded":"2023-07-06T03:45:10.661537+0000","last_fullsized":"2023-07-06T03:45:10.661537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:05:07.859127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632056+0000","last_change":"2023-07-06T03:45:10.632056+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632056+0000","last_undegraded":"2023-07-06T03:45:10.632056+0000","last_fullsized":"2023-07-06T03:45:10.632056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.633553+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:45:10.633553+0000","last_peered":"2023-07-06T03:45:10.633553+0000","last_clean":"2023-07-06T03:45:10.633553+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:45:10.633553+0000","last_undegraded":"2023-07-06T03:45:10.633553+0000","last_fullsized":"2023-07-06T03:45:10.633553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:02:59.779305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.666141+0000","last_change":"2023-07-06T03:45:10.660900+0000","last_active":"2023-07-06T03:45:10.666141+0000","last_peered":"2023-07-06T03:45:10.666141+0000","last_clean":"2023-07-06T03:45:10.666141+0000","last_became_active":"2023-07-06T03:45:10.660490+0000","last_became_peered":"2023-07-06T03:45:10.660490+0000","last_unstale":"2023-07-06T03:45:10.666141+0000","last_undegraded":"2023-07-06T03:45:10.666141+0000","last_fullsized":"2023-07-06T03:45:10.666141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:43:52.346163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.636474+0000","last_change":"2023-07-06T03:45:10.636474+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.636474+0000","last_undegraded":"2023-07-06T03:45:10.636474+0000","last_fullsized":"2023-07-06T03:45:10.636474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.637506+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:45:10.637506+0000","last_peered":"2023-07-06T03:45:10.637506+0000","last_clean":"2023-07-06T03:45:10.637506+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:45:10.637506+0000","last_undegraded":"2023-07-06T03:45:10.637506+0000","last_fullsized":"2023-07-06T03:45:10.637506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:41.368994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.666172+0000","last_change":"2023-07-06T03:45:10.661812+0000","last_active":"2023-07-06T03:45:10.666172+0000","last_peered":"2023-07-06T03:45:10.666172+0000","last_clean":"2023-07-06T03:45:10.666172+0000","last_became_active":"2023-07-06T03:45:10.661499+0000","last_became_peered":"2023-07-06T03:45:10.661499+0000","last_unstale":"2023-07-06T03:45:10.666172+0000","last_undegraded":"2023-07-06T03:45:10.666172+0000","last_fullsized":"2023-07-06T03:45:10.666172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:56:58.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.638646+0000","last_change":"2023-07-06T03:45:10.638646+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.638646+0000","last_undegraded":"2023-07-06T03:45:10.638646+0000","last_fullsized":"2023-07-06T03:45:10.638646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.661732+0000","last_change":"2023-07-06T03:45:10.657429+0000","last_active":"2023-07-06T03:45:10.661732+0000","last_peered":"2023-07-06T03:45:10.661732+0000","last_clean":"2023-07-06T03:45:10.661732+0000","last_became_active":"2023-07-06T03:45:10.657085+0000","last_became_peered":"2023-07-06T03:45:10.657085+0000","last_unstale":"2023-07-06T03:45:10.661732+0000","last_undegraded":"2023-07-06T03:45:10.661732+0000","last_fullsized":"2023-07-06T03:45:10.661732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:23.874792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.648590+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:45:10.648590+0000","last_peered":"2023-07-06T03:45:10.648590+0000","last_clean":"2023-07-06T03:45:10.648590+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:45:10.648590+0000","last_undegraded":"2023-07-06T03:45:10.648590+0000","last_fullsized":"2023-07-06T03:45:10.648590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:39:41.393077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.638574+0000","last_change":"2023-07-06T03:45:10.638574+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.638574+0000","last_undegraded":"2023-07-06T03:45:10.638574+0000","last_fullsized":"2023-07-06T03:45:10.638574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.639518+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:45:10.639518+0000","last_peered":"2023-07-06T03:45:10.639518+0000","last_clean":"2023-07-06T03:45:10.639518+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:45:10.639518+0000","last_undegraded":"2023-07-06T03:45:10.639518+0000","last_fullsized":"2023-07-06T03:45:10.639518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:53.585611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.662161+0000","last_change":"2023-07-06T03:45:10.657912+0000","last_active":"2023-07-06T03:45:10.662161+0000","last_peered":"2023-07-06T03:45:10.662161+0000","last_clean":"2023-07-06T03:45:10.662161+0000","last_became_active":"2023-07-06T03:45:10.657615+0000","last_became_peered":"2023-07-06T03:45:10.657615+0000","last_unstale":"2023-07-06T03:45:10.662161+0000","last_undegraded":"2023-07-06T03:45:10.662161+0000","last_fullsized":"2023-07-06T03:45:10.662161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:10:28.622148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632279+0000","last_change":"2023-07-06T03:45:10.632279+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632279+0000","last_undegraded":"2023-07-06T03:45:10.632279+0000","last_fullsized":"2023-07-06T03:45:10.632279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.663451+0000","last_change":"2023-07-06T03:45:10.658517+0000","last_active":"2023-07-06T03:45:10.663451+0000","last_peered":"2023-07-06T03:45:10.663451+0000","last_clean":"2023-07-06T03:45:10.663451+0000","last_became_active":"2023-07-06T03:45:10.657946+0000","last_became_peered":"2023-07-06T03:45:10.657946+0000","last_unstale":"2023-07-06T03:45:10.663451+0000","last_undegraded":"2023-07-06T03:45:10.663451+0000","last_fullsized":"2023-07-06T03:45:10.663451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:11.533017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.640062+0000","last_change":"2023-07-06T03:45:10.640062+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.640062+0000","last_undegraded":"2023-07-06T03:45:10.640062+0000","last_fullsized":"2023-07-06T03:45:10.640062+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.654834+0000","last_change":"2023-07-06T03:45:10.649186+0000","last_active":"2023-07-06T03:45:10.654834+0000","last_peered":"2023-07-06T03:45:10.654834+0000","last_clean":"2023-07-06T03:45:10.654834+0000","last_became_active":"2023-07-06T03:45:10.648707+0000","last_became_peered":"2023-07-06T03:45:10.648707+0000","last_unstale":"2023-07-06T03:45:10.654834+0000","last_undegraded":"2023-07-06T03:45:10.654834+0000","last_fullsized":"2023-07-06T03:45:10.654834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:01:48.414029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632953+0000","last_change":"2023-07-06T03:45:10.632953+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632953+0000","last_undegraded":"2023-07-06T03:45:10.632953+0000","last_fullsized":"2023-07-06T03:45:10.632953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.657765+0000","last_change":"2023-07-06T03:45:10.653276+0000","last_active":"2023-07-06T03:45:10.657765+0000","last_peered":"2023-07-06T03:45:10.657765+0000","last_clean":"2023-07-06T03:45:10.657765+0000","last_became_active":"2023-07-06T03:45:10.652958+0000","last_became_peered":"2023-07-06T03:45:10.652958+0000","last_unstale":"2023-07-06T03:45:10.657765+0000","last_undegraded":"2023-07-06T03:45:10.657765+0000","last_fullsized":"2023-07-06T03:45:10.657765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:23.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.632470+0000","last_change":"2023-07-06T03:45:10.632470+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.632470+0000","last_undegraded":"2023-07-06T03:45:10.632470+0000","last_fullsized":"2023-07-06T03:45:10.632470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.648167+0000","last_change":"2023-07-06T03:45:10.645296+0000","last_active":"2023-07-06T03:45:10.648167+0000","last_peered":"2023-07-06T03:45:10.648167+0000","last_clean":"2023-07-06T03:45:10.648167+0000","last_became_active":"2023-07-06T03:45:10.644967+0000","last_became_peered":"2023-07-06T03:45:10.644967+0000","last_unstale":"2023-07-06T03:45:10.648167+0000","last_undegraded":"2023-07-06T03:45:10.648167+0000","last_fullsized":"2023-07-06T03:45:10.648167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:10:28.622148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.635490+0000","last_change":"2023-07-06T03:45:10.635490+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.635490+0000","last_undegraded":"2023-07-06T03:45:10.635490+0000","last_fullsized":"2023-07-06T03:45:10.635490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T03:45:10.660155+0000","last_change":"2023-07-06T03:45:10.654706+0000","last_active":"2023-07-06T03:45:10.660155+0000","last_peered":"2023-07-06T03:45:10.660155+0000","last_clean":"2023-07-06T03:45:10.660155+0000","last_became_active":"2023-07-06T03:45:10.653995+0000","last_became_peered":"2023-07-06T03:45:10.653995+0000","last_unstale":"2023-07-06T03:45:10.660155+0000","last_undegraded":"2023-07-06T03:45:10.660155+0000","last_fullsized":"2023-07-06T03:45:10.660155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:56:58.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.631366+0000","last_change":"2023-07-06T03:45:10.631366+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.631366+0000","last_undegraded":"2023-07-06T03:45:10.631366+0000","last_fullsized":"2023-07-06T03:45:10.631366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641388+0000","last_change":"2023-07-06T03:45:10.641388+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641388+0000","last_undegraded":"2023-07-06T03:45:10.641388+0000","last_fullsized":"2023-07-06T03:45:10.641388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.641341+0000","last_change":"2023-07-06T03:45:10.641341+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.641341+0000","last_undegraded":"2023-07-06T03:45:10.641341+0000","last_fullsized":"2023-07-06T03:45:10.641341+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.639222+0000","last_change":"2023-07-06T03:45:10.639222+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.639222+0000","last_undegraded":"2023-07-06T03:45:10.639222+0000","last_fullsized":"2023-07-06T03:45:10.639222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.637982+0000","last_change":"2023-07-06T03:45:10.637982+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.637982+0000","last_undegraded":"2023-07-06T03:45:10.637982+0000","last_fullsized":"2023-07-06T03:45:10.637982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2023-07-06T03:45:10.637307+0000","last_change":"2023-07-06T03:45:10.637307+0000","last_active":"2023-07-06T03:45:10.624339+0000","last_peered":"2023-07-06T03:45:10.624339+0000","last_clean":"2023-07-06T03:45:10.624339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:45:10.637307+0000","last_undegraded":"2023-07-06T03:45:10.637307+0000","last_fullsized":"2023-07-06T03:45:10.637307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":811008,"data_stored":666186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6679999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":794624,"data_stored":658539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:45:16.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:45:16.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.748+0000 7fa5383ff700 1 Processor -- start 2023-07-06T03:45:16.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.748+0000 7fa5383ff700 1 -- start start 2023-07-06T03:45:16.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.749+0000 7fa5383ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa5300cd390 0x7fa530160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:16.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa5383ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5300cdae0 con 0x7fa53015c520 2023-07-06T03:45:16.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa5383ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5300b6640 con 0x7fa530160dd0 2023-07-06T03:45:16.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa5383ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5300b6920 con 0x7fa5300cd390 2023-07-06T03:45:16.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa53619b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa53015c520 0x7fa530160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56064/0 (socket says 172.21.15.97:56064) 2023-07-06T03:45:16.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa53619b700 1 -- 172.21.15.97:0/176728682 learned_addr learned my addr 172.21.15.97:0/176728682 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:16.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa53599a700 1 --2- 172.21.15.97:0/176728682 >> v2:172.21.15.146:3300/0 conn(0x7fa5300cd390 0x7fa530160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:16.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa535199700 1 -- 172.21.15.97:0/176728682 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950804952 0 0) 0x7fa5300cdae0 con 0x7fa53015c520 2023-07-06T03:45:16.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa535199700 1 -- 172.21.15.97:0/176728682 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa51c0035b0 con 0x7fa53015c520 2023-07-06T03:45:16.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa53599a700 1 -- 172.21.15.97:0/176728682 >> v1:172.21.15.146:6789/0 conn(0x7fa530160dd0 legacy=0x7fa5300b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa53599a700 1 -- 172.21.15.97:0/176728682 >> v1:172.21.15.97:6789/0 conn(0x7fa53015c520 legacy=0x7fa530160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa535199700 1 -- 172.21.15.97:0/176728682 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3558944860 0 0) 0x7fa51c0035b0 con 0x7fa53015c520 2023-07-06T03:45:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.750+0000 7fa53599a700 1 -- 172.21.15.97:0/176728682 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5300b6c00 con 0x7fa5300cd390 2023-07-06T03:45:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.751+0000 7fa53599a700 1 --2- 172.21.15.97:0/176728682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5300cd390 0x7fa530160890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa520009840 tx=0x7fa520009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f78e620f6b5ca79 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.751+0000 7fa535199700 1 -- 172.21.15.97:0/176728682 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520011830 con 0x7fa5300cd390 2023-07-06T03:45:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.751+0000 7fa535199700 1 -- 172.21.15.97:0/176728682 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa520011990 con 0x7fa5300cd390 2023-07-06T03:45:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.751+0000 7fa535199700 1 -- 172.21.15.97:0/176728682 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520011d20 con 0x7fa5300cd390 2023-07-06T03:45:16.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.752+0000 7fa5383ff700 1 -- 172.21.15.97:0/176728682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5300cd390 msgr2=0x7fa530160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.752+0000 7fa5383ff700 1 --2- 172.21.15.97:0/176728682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5300cd390 0x7fa530160890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa520009840 tx=0x7fa520009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:16.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.752+0000 7fa5383ff700 1 -- 172.21.15.97:0/176728682 shutdown_connections 2023-07-06T03:45:16.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.752+0000 7fa5383ff700 1 --2- 172.21.15.97:0/176728682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5300cd390 0x7fa530160890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.752+0000 7fa5383ff700 1 -- 172.21.15.97:0/176728682 >> 172.21.15.97:0/176728682 conn(0x7fa530071790 msgr2=0x7fa530071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:16.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.752+0000 7fa5383ff700 1 -- 172.21.15.97:0/176728682 shutdown_connections 2023-07-06T03:45:16.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.752+0000 7fa5383ff700 1 -- 172.21.15.97:0/176728682 wait complete. 2023-07-06T03:45:16.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.754+0000 7fa5383ff700 1 Processor -- start 2023-07-06T03:45:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.754+0000 7fa5383ff700 1 -- start start 2023-07-06T03:45:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa5383ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5300cd390 0x7fa53014cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa5383ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa53015c520 0x7fa530154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa5383ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa530160dd0 0x7fa5300d1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa5383ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa530156800 con 0x7fa5300cd390 2023-07-06T03:45:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa5383ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa530155760 con 0x7fa53015c520 2023-07-06T03:45:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa5383ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa530155a40 con 0x7fa530160dd0 2023-07-06T03:45:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa53619b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5300cd390 0x7fa53014cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa53619b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5300cd390 0x7fa53014cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46922/0 (socket says 172.21.15.97:46922) 2023-07-06T03:45:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa53619b700 1 -- 172.21.15.97:0/2464581839 learned_addr learned my addr 172.21.15.97:0/2464581839 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:16.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa53619b700 1 -- 172.21.15.97:0/2464581839 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa530160dd0 msgr2=0x7fa5300d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa53619b700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa530160dd0 0x7fa5300d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa53619b700 1 -- 172.21.15.97:0/2464581839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa53015c520 msgr2=0x7fa530154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa53619b700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa53015c520 0x7fa530154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa53619b700 1 -- 172.21.15.97:0/2464581839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200094a0 con 0x7fa5300cd390 2023-07-06T03:45:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.756+0000 7fa53619b700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5300cd390 0x7fa53014cfe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c780 tx=0x7fa52c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.757+0000 7fa526ffd700 1 -- 172.21.15.97:0/2464581839 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c005d70 con 0x7fa5300cd390 2023-07-06T03:45:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.757+0000 7fa526ffd700 1 -- 172.21.15.97:0/2464581839 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c0070d0 con 0x7fa5300cd390 2023-07-06T03:45:16.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.757+0000 7fa5383ff700 1 -- 172.21.15.97:0/2464581839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa530155d20 con 0x7fa5300cd390 2023-07-06T03:45:16.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.757+0000 7fa526ffd700 1 -- 172.21.15.97:0/2464581839 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0074c0 con 0x7fa5300cd390 2023-07-06T03:45:16.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.757+0000 7fa5383ff700 1 -- 172.21.15.97:0/2464581839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5301562e0 con 0x7fa5300cd390 2023-07-06T03:45:16.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.758+0000 7fa5383ff700 1 -- 172.21.15.97:0/2464581839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5300b6200 con 0x7fa5300cd390 2023-07-06T03:45:16.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.759+0000 7fa526ffd700 1 -- 172.21.15.97:0/2464581839 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa52c007620 con 0x7fa5300cd390 2023-07-06T03:45:16.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.760+0000 7fa526ffd700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa51406ace0 0x7fa51406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:16.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.760+0000 7fa526ffd700 1 -- 172.21.15.97:0/2464581839 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa52c088ba0 con 0x7fa5300cd390 2023-07-06T03:45:16.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.760+0000 7fa53599a700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa51406ace0 0x7fa51406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:16.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.761+0000 7fa53599a700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa51406ace0 0x7fa51406d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa53015cc30 tx=0x7fa520010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:16.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.764+0000 7fa526ffd700 1 -- 172.21.15.97:0/2464581839 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa52c053bb0 con 0x7fa5300cd390 2023-07-06T03:45:16.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.958+0000 7fa5383ff700 1 -- 172.21.15.97:0/2464581839 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5300cd800 con 0x7fa51406ace0 2023-07-06T03:45:16.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.966+0000 7fa526ffd700 1 -- 172.21.15.97:0/2464581839 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286825 (secure 0 0 0) 0x7fa5200117f0 con 0x7fa51406ace0 2023-07-06T03:45:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 -- 172.21.15.97:0/2464581839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa51406ace0 msgr2=0x7fa51406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa51406ace0 0x7fa51406d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa53015cc30 tx=0x7fa520010000 comp rx=0 tx=0).stop 2023-07-06T03:45:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 -- 172.21.15.97:0/2464581839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5300cd390 msgr2=0x7fa53014cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5300cd390 0x7fa53014cfe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c780 tx=0x7fa52c00cb40 comp rx=0 tx=0).stop 2023-07-06T03:45:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 -- 172.21.15.97:0/2464581839 shutdown_connections 2023-07-06T03:45:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa51406ace0 0x7fa51406d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa530160dd0 0x7fa5300d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5300cd390 0x7fa53014cfe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 --2- 172.21.15.97:0/2464581839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa53015c520 0x7fa530154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 -- 172.21.15.97:0/2464581839 >> 172.21.15.97:0/2464581839 conn(0x7fa530071790 msgr2=0x7fa5300c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 -- 172.21.15.97:0/2464581839 shutdown_connections 2023-07-06T03:45:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:16.970+0000 7fa5383ff700 1 -- 172.21.15.97:0/2464581839 wait complete. 2023-07-06T03:45:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:45:16.989 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-07-06T03:45:16.883203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218724,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.253071"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681463+0000","last_change":"2023-07-06T03:45:11.693177+0000","last_active":"2023-07-06T03:45:12.681463+0000","last_peered":"2023-07-06T03:45:12.681463+0000","last_clean":"2023-07-06T03:45:12.681463+0000","last_became_active":"2023-07-06T03:45:11.692851+0000","last_became_peered":"2023-07-06T03:45:11.692851+0000","last_unstale":"2023-07-06T03:45:12.681463+0000","last_undegraded":"2023-07-06T03:45:12.681463+0000","last_fullsized":"2023-07-06T03:45:12.681463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678856+0000","last_change":"2023-07-06T03:45:11.674359+0000","last_active":"2023-07-06T03:45:12.678856+0000","last_peered":"2023-07-06T03:45:12.678856+0000","last_clean":"2023-07-06T03:45:12.678856+0000","last_became_active":"2023-07-06T03:45:11.674035+0000","last_became_peered":"2023-07-06T03:45:11.674035+0000","last_unstale":"2023-07-06T03:45:12.678856+0000","last_undegraded":"2023-07-06T03:45:12.678856+0000","last_fullsized":"2023-07-06T03:45:12.678856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827906+0000","last_change":"2023-07-06T03:45:11.691806+0000","last_active":"2023-07-06T03:45:13.827906+0000","last_peered":"2023-07-06T03:45:13.827906+0000","last_clean":"2023-07-06T03:45:13.827906+0000","last_became_active":"2023-07-06T03:45:11.691532+0000","last_became_peered":"2023-07-06T03:45:11.691532+0000","last_unstale":"2023-07-06T03:45:13.827906+0000","last_undegraded":"2023-07-06T03:45:13.827906+0000","last_fullsized":"2023-07-06T03:45:13.827906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827578+0000","last_change":"2023-07-06T03:45:11.675300+0000","last_active":"2023-07-06T03:45:13.827578+0000","last_peered":"2023-07-06T03:45:13.827578+0000","last_clean":"2023-07-06T03:45:13.827578+0000","last_became_active":"2023-07-06T03:45:11.675018+0000","last_became_peered":"2023-07-06T03:45:11.675018+0000","last_unstale":"2023-07-06T03:45:13.827578+0000","last_undegraded":"2023-07-06T03:45:13.827578+0000","last_fullsized":"2023-07-06T03:45:13.827578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205854+0000","last_change":"2023-07-06T03:45:11.666099+0000","last_active":"2023-07-06T03:45:14.205854+0000","last_peered":"2023-07-06T03:45:14.205854+0000","last_clean":"2023-07-06T03:45:14.205854+0000","last_became_active":"2023-07-06T03:45:11.665759+0000","last_became_peered":"2023-07-06T03:45:11.665759+0000","last_unstale":"2023-07-06T03:45:14.205854+0000","last_undegraded":"2023-07-06T03:45:14.205854+0000","last_fullsized":"2023-07-06T03:45:14.205854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:52.666500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830215+0000","last_change":"2023-07-06T03:45:11.662251+0000","last_active":"2023-07-06T03:45:13.830215+0000","last_peered":"2023-07-06T03:45:13.830215+0000","last_clean":"2023-07-06T03:45:13.830215+0000","last_became_active":"2023-07-06T03:45:11.661916+0000","last_became_peered":"2023-07-06T03:45:11.661916+0000","last_unstale":"2023-07-06T03:45:13.830215+0000","last_undegraded":"2023-07-06T03:45:13.830215+0000","last_fullsized":"2023-07-06T03:45:13.830215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:11.287937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829687+0000","last_change":"2023-07-06T03:45:11.676465+0000","last_active":"2023-07-06T03:45:13.829687+0000","last_peered":"2023-07-06T03:45:13.829687+0000","last_clean":"2023-07-06T03:45:13.829687+0000","last_became_active":"2023-07-06T03:45:11.676180+0000","last_became_peered":"2023-07-06T03:45:11.676180+0000","last_unstale":"2023-07-06T03:45:13.829687+0000","last_undegraded":"2023-07-06T03:45:13.829687+0000","last_fullsized":"2023-07-06T03:45:13.829687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:38:24.542890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818440+0000","last_change":"2023-07-06T03:45:11.676387+0000","last_active":"2023-07-06T03:45:12.818440+0000","last_peered":"2023-07-06T03:45:12.818440+0000","last_clean":"2023-07-06T03:45:12.818440+0000","last_became_active":"2023-07-06T03:45:11.676129+0000","last_became_peered":"2023-07-06T03:45:11.676129+0000","last_unstale":"2023-07-06T03:45:12.818440+0000","last_undegraded":"2023-07-06T03:45:12.818440+0000","last_fullsized":"2023-07-06T03:45:12.818440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206076+0000","last_change":"2023-07-06T03:45:11.690977+0000","last_active":"2023-07-06T03:45:14.206076+0000","last_peered":"2023-07-06T03:45:14.206076+0000","last_clean":"2023-07-06T03:45:14.206076+0000","last_became_active":"2023-07-06T03:45:11.690423+0000","last_became_peered":"2023-07-06T03:45:11.690423+0000","last_unstale":"2023-07-06T03:45:14.206076+0000","last_undegraded":"2023-07-06T03:45:14.206076+0000","last_fullsized":"2023-07-06T03:45:14.206076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:14:59.597908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818347+0000","last_change":"2023-07-06T03:45:11.680924+0000","last_active":"2023-07-06T03:45:12.818347+0000","last_peered":"2023-07-06T03:45:12.818347+0000","last_clean":"2023-07-06T03:45:12.818347+0000","last_became_active":"2023-07-06T03:45:11.680607+0000","last_became_peered":"2023-07-06T03:45:11.680607+0000","last_unstale":"2023-07-06T03:45:12.818347+0000","last_undegraded":"2023-07-06T03:45:12.818347+0000","last_fullsized":"2023-07-06T03:45:12.818347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:13.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818149+0000","last_change":"2023-07-06T03:45:11.677946+0000","last_active":"2023-07-06T03:45:12.818149+0000","last_peered":"2023-07-06T03:45:12.818149+0000","last_clean":"2023-07-06T03:45:12.818149+0000","last_became_active":"2023-07-06T03:45:11.677684+0000","last_became_peered":"2023-07-06T03:45:11.677684+0000","last_unstale":"2023-07-06T03:45:12.818149+0000","last_undegraded":"2023-07-06T03:45:12.818149+0000","last_fullsized":"2023-07-06T03:45:12.818149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:27.384754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680941+0000","last_change":"2023-07-06T03:45:11.681178+0000","last_active":"2023-07-06T03:45:12.680941+0000","last_peered":"2023-07-06T03:45:12.680941+0000","last_clean":"2023-07-06T03:45:12.680941+0000","last_became_active":"2023-07-06T03:45:11.680841+0000","last_became_peered":"2023-07-06T03:45:11.680841+0000","last_unstale":"2023-07-06T03:45:12.680941+0000","last_undegraded":"2023-07-06T03:45:12.680941+0000","last_fullsized":"2023-07-06T03:45:12.680941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:58.087282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652673+0000","last_change":"2023-07-06T03:45:11.687461+0000","last_active":"2023-07-06T03:45:13.652673+0000","last_peered":"2023-07-06T03:45:13.652673+0000","last_clean":"2023-07-06T03:45:13.652673+0000","last_became_active":"2023-07-06T03:45:11.687182+0000","last_became_peered":"2023-07-06T03:45:11.687182+0000","last_unstale":"2023-07-06T03:45:13.652673+0000","last_undegraded":"2023-07-06T03:45:13.652673+0000","last_fullsized":"2023-07-06T03:45:13.652673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817899+0000","last_change":"2023-07-06T03:45:11.691496+0000","last_active":"2023-07-06T03:45:12.817899+0000","last_peered":"2023-07-06T03:45:12.817899+0000","last_clean":"2023-07-06T03:45:12.817899+0000","last_became_active":"2023-07-06T03:45:11.691206+0000","last_became_peered":"2023-07-06T03:45:11.691206+0000","last_unstale":"2023-07-06T03:45:12.817899+0000","last_undegraded":"2023-07-06T03:45:12.817899+0000","last_fullsized":"2023-07-06T03:45:12.817899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827487+0000","last_change":"2023-07-06T03:45:11.685001+0000","last_active":"2023-07-06T03:45:13.827487+0000","last_peered":"2023-07-06T03:45:13.827487+0000","last_clean":"2023-07-06T03:45:13.827487+0000","last_became_active":"2023-07-06T03:45:11.684695+0000","last_became_peered":"2023-07-06T03:45:11.684695+0000","last_unstale":"2023-07-06T03:45:13.827487+0000","last_undegraded":"2023-07-06T03:45:13.827487+0000","last_fullsized":"2023-07-06T03:45:13.827487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829742+0000","last_change":"2023-07-06T03:45:11.662096+0000","last_active":"2023-07-06T03:45:13.829742+0000","last_peered":"2023-07-06T03:45:13.829742+0000","last_clean":"2023-07-06T03:45:13.829742+0000","last_became_active":"2023-07-06T03:45:11.661834+0000","last_became_peered":"2023-07-06T03:45:11.661834+0000","last_unstale":"2023-07-06T03:45:13.829742+0000","last_undegraded":"2023-07-06T03:45:13.829742+0000","last_fullsized":"2023-07-06T03:45:13.829742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:36:49.534209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678504+0000","last_change":"2023-07-06T03:45:11.685162+0000","last_active":"2023-07-06T03:45:12.678504+0000","last_peered":"2023-07-06T03:45:12.678504+0000","last_clean":"2023-07-06T03:45:12.678504+0000","last_became_active":"2023-07-06T03:45:11.684977+0000","last_became_peered":"2023-07-06T03:45:11.684977+0000","last_unstale":"2023-07-06T03:45:12.678504+0000","last_undegraded":"2023-07-06T03:45:12.678504+0000","last_fullsized":"2023-07-06T03:45:12.678504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827827+0000","last_change":"2023-07-06T03:45:11.675611+0000","last_active":"2023-07-06T03:45:13.827827+0000","last_peered":"2023-07-06T03:45:13.827827+0000","last_clean":"2023-07-06T03:45:13.827827+0000","last_became_active":"2023-07-06T03:45:11.675348+0000","last_became_peered":"2023-07-06T03:45:11.675348+0000","last_unstale":"2023-07-06T03:45:13.827827+0000","last_undegraded":"2023-07-06T03:45:13.827827+0000","last_fullsized":"2023-07-06T03:45:13.827827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:00.096345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680922+0000","last_change":"2023-07-06T03:45:11.690182+0000","last_active":"2023-07-06T03:45:12.680922+0000","last_peered":"2023-07-06T03:45:12.680922+0000","last_clean":"2023-07-06T03:45:12.680922+0000","last_became_active":"2023-07-06T03:45:11.689744+0000","last_became_peered":"2023-07-06T03:45:11.689744+0000","last_unstale":"2023-07-06T03:45:12.680922+0000","last_undegraded":"2023-07-06T03:45:12.680922+0000","last_fullsized":"2023-07-06T03:45:12.680922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:32.898881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818137+0000","last_change":"2023-07-06T03:45:11.672749+0000","last_active":"2023-07-06T03:45:12.818137+0000","last_peered":"2023-07-06T03:45:12.818137+0000","last_clean":"2023-07-06T03:45:12.818137+0000","last_became_active":"2023-07-06T03:45:11.672401+0000","last_became_peered":"2023-07-06T03:45:11.672401+0000","last_unstale":"2023-07-06T03:45:12.818137+0000","last_undegraded":"2023-07-06T03:45:12.818137+0000","last_fullsized":"2023-07-06T03:45:12.818137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:15.627811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818032+0000","last_change":"2023-07-06T03:45:11.692060+0000","last_active":"2023-07-06T03:45:12.818032+0000","last_peered":"2023-07-06T03:45:12.818032+0000","last_clean":"2023-07-06T03:45:12.818032+0000","last_became_active":"2023-07-06T03:45:11.691699+0000","last_became_peered":"2023-07-06T03:45:11.691699+0000","last_unstale":"2023-07-06T03:45:12.818032+0000","last_undegraded":"2023-07-06T03:45:12.818032+0000","last_fullsized":"2023-07-06T03:45:12.818032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:32.898881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678195+0000","last_change":"2023-07-06T03:45:11.672926+0000","last_active":"2023-07-06T03:45:12.678195+0000","last_peered":"2023-07-06T03:45:12.678195+0000","last_clean":"2023-07-06T03:45:12.678195+0000","last_became_active":"2023-07-06T03:45:11.672627+0000","last_became_peered":"2023-07-06T03:45:11.672627+0000","last_unstale":"2023-07-06T03:45:12.678195+0000","last_undegraded":"2023-07-06T03:45:12.678195+0000","last_fullsized":"2023-07-06T03:45:12.678195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205814+0000","last_change":"2023-07-06T03:45:11.691373+0000","last_active":"2023-07-06T03:45:14.205814+0000","last_peered":"2023-07-06T03:45:14.205814+0000","last_clean":"2023-07-06T03:45:14.205814+0000","last_became_active":"2023-07-06T03:45:11.691046+0000","last_became_peered":"2023-07-06T03:45:11.691046+0000","last_unstale":"2023-07-06T03:45:14.205814+0000","last_undegraded":"2023-07-06T03:45:14.205814+0000","last_fullsized":"2023-07-06T03:45:14.205814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818372+0000","last_change":"2023-07-06T03:45:11.691643+0000","last_active":"2023-07-06T03:45:12.818372+0000","last_peered":"2023-07-06T03:45:12.818372+0000","last_clean":"2023-07-06T03:45:12.818372+0000","last_became_active":"2023-07-06T03:45:11.691382+0000","last_became_peered":"2023-07-06T03:45:11.691382+0000","last_unstale":"2023-07-06T03:45:12.818372+0000","last_undegraded":"2023-07-06T03:45:12.818372+0000","last_fullsized":"2023-07-06T03:45:12.818372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206243+0000","last_change":"2023-07-06T03:45:11.670205+0000","last_active":"2023-07-06T03:45:14.206243+0000","last_peered":"2023-07-06T03:45:14.206243+0000","last_clean":"2023-07-06T03:45:14.206243+0000","last_became_active":"2023-07-06T03:45:11.669935+0000","last_became_peered":"2023-07-06T03:45:11.669935+0000","last_unstale":"2023-07-06T03:45:14.206243+0000","last_undegraded":"2023-07-06T03:45:14.206243+0000","last_fullsized":"2023-07-06T03:45:14.206243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:11:07.089396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678592+0000","last_change":"2023-07-06T03:45:11.674601+0000","last_active":"2023-07-06T03:45:12.678592+0000","last_peered":"2023-07-06T03:45:12.678592+0000","last_clean":"2023-07-06T03:45:12.678592+0000","last_became_active":"2023-07-06T03:45:11.673533+0000","last_became_peered":"2023-07-06T03:45:11.673533+0000","last_unstale":"2023-07-06T03:45:12.678592+0000","last_undegraded":"2023-07-06T03:45:12.678592+0000","last_fullsized":"2023-07-06T03:45:12.678592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830285+0000","last_change":"2023-07-06T03:45:11.678777+0000","last_active":"2023-07-06T03:45:13.830285+0000","last_peered":"2023-07-06T03:45:13.830285+0000","last_clean":"2023-07-06T03:45:13.830285+0000","last_became_active":"2023-07-06T03:45:11.678437+0000","last_became_peered":"2023-07-06T03:45:11.678437+0000","last_unstale":"2023-07-06T03:45:13.830285+0000","last_undegraded":"2023-07-06T03:45:13.830285+0000","last_fullsized":"2023-07-06T03:45:13.830285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:51:53.330078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818105+0000","last_change":"2023-07-06T03:45:10.656464+0000","last_active":"2023-07-06T03:45:12.818105+0000","last_peered":"2023-07-06T03:45:12.818105+0000","last_clean":"2023-07-06T03:45:12.818105+0000","last_became_active":"2023-07-06T03:45:10.656116+0000","last_became_peered":"2023-07-06T03:45:10.656116+0000","last_unstale":"2023-07-06T03:45:12.818105+0000","last_undegraded":"2023-07-06T03:45:12.818105+0000","last_fullsized":"2023-07-06T03:45:12.818105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:57.080176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678331+0000","last_change":"2023-07-06T03:45:11.687243+0000","last_active":"2023-07-06T03:45:12.678331+0000","last_peered":"2023-07-06T03:45:12.678331+0000","last_clean":"2023-07-06T03:45:12.678331+0000","last_became_active":"2023-07-06T03:45:11.686947+0000","last_became_peered":"2023-07-06T03:45:11.686947+0000","last_unstale":"2023-07-06T03:45:12.678331+0000","last_undegraded":"2023-07-06T03:45:12.678331+0000","last_fullsized":"2023-07-06T03:45:12.678331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205529+0000","last_change":"2023-07-06T03:45:10.657736+0000","last_active":"2023-07-06T03:45:14.205529+0000","last_peered":"2023-07-06T03:45:14.205529+0000","last_clean":"2023-07-06T03:45:14.205529+0000","last_became_active":"2023-07-06T03:45:10.657450+0000","last_became_peered":"2023-07-06T03:45:10.657450+0000","last_unstale":"2023-07-06T03:45:14.205529+0000","last_undegraded":"2023-07-06T03:45:14.205529+0000","last_fullsized":"2023-07-06T03:45:14.205529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:14.620705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206125+0000","last_change":"2023-07-06T03:45:11.672564+0000","last_active":"2023-07-06T03:45:14.206125+0000","last_peered":"2023-07-06T03:45:14.206125+0000","last_clean":"2023-07-06T03:45:14.206125+0000","last_became_active":"2023-07-06T03:45:11.672150+0000","last_became_peered":"2023-07-06T03:45:11.672150+0000","last_unstale":"2023-07-06T03:45:14.206125+0000","last_undegraded":"2023-07-06T03:45:14.206125+0000","last_fullsized":"2023-07-06T03:45:14.206125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:59:01.536411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680441+0000","last_change":"2023-07-06T03:45:10.653666+0000","last_active":"2023-07-06T03:45:12.680441+0000","last_peered":"2023-07-06T03:45:12.680441+0000","last_clean":"2023-07-06T03:45:12.680441+0000","last_became_active":"2023-07-06T03:45:10.653357+0000","last_became_peered":"2023-07-06T03:45:10.653357+0000","last_unstale":"2023-07-06T03:45:12.680441+0000","last_undegraded":"2023-07-06T03:45:12.680441+0000","last_fullsized":"2023-07-06T03:45:12.680441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:25.838923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830215+0000","last_change":"2023-07-06T03:45:11.690985+0000","last_active":"2023-07-06T03:45:13.830215+0000","last_peered":"2023-07-06T03:45:13.830215+0000","last_clean":"2023-07-06T03:45:13.830215+0000","last_became_active":"2023-07-06T03:45:11.690692+0000","last_became_peered":"2023-07-06T03:45:11.690692+0000","last_unstale":"2023-07-06T03:45:13.830215+0000","last_undegraded":"2023-07-06T03:45:13.830215+0000","last_fullsized":"2023-07-06T03:45:13.830215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:27:20.751301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681704+0000","last_change":"2023-07-06T03:45:10.649734+0000","last_active":"2023-07-06T03:45:12.681704+0000","last_peered":"2023-07-06T03:45:12.681704+0000","last_clean":"2023-07-06T03:45:12.681704+0000","last_became_active":"2023-07-06T03:45:10.649401+0000","last_became_peered":"2023-07-06T03:45:10.649401+0000","last_unstale":"2023-07-06T03:45:12.681704+0000","last_undegraded":"2023-07-06T03:45:12.681704+0000","last_fullsized":"2023-07-06T03:45:12.681704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:24:42.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678483+0000","last_change":"2023-07-06T03:45:11.667398+0000","last_active":"2023-07-06T03:45:12.678483+0000","last_peered":"2023-07-06T03:45:12.678483+0000","last_clean":"2023-07-06T03:45:12.678483+0000","last_became_active":"2023-07-06T03:45:11.667124+0000","last_became_peered":"2023-07-06T03:45:11.667124+0000","last_unstale":"2023-07-06T03:45:12.678483+0000","last_undegraded":"2023-07-06T03:45:12.678483+0000","last_fullsized":"2023-07-06T03:45:12.678483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:00.096345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.679013+0000","last_change":"2023-07-06T03:45:10.648660+0000","last_active":"2023-07-06T03:45:12.679013+0000","last_peered":"2023-07-06T03:45:12.679013+0000","last_clean":"2023-07-06T03:45:12.679013+0000","last_became_active":"2023-07-06T03:45:10.648254+0000","last_became_peered":"2023-07-06T03:45:10.648254+0000","last_unstale":"2023-07-06T03:45:12.679013+0000","last_undegraded":"2023-07-06T03:45:12.679013+0000","last_fullsized":"2023-07-06T03:45:12.679013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:25.838923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205490+0000","last_change":"2023-07-06T03:45:11.682171+0000","last_active":"2023-07-06T03:45:14.205490+0000","last_peered":"2023-07-06T03:45:14.205490+0000","last_clean":"2023-07-06T03:45:14.205490+0000","last_became_active":"2023-07-06T03:45:11.681861+0000","last_became_peered":"2023-07-06T03:45:11.681861+0000","last_unstale":"2023-07-06T03:45:14.205490+0000","last_undegraded":"2023-07-06T03:45:14.205490+0000","last_fullsized":"2023-07-06T03:45:14.205490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827972+0000","last_change":"2023-07-06T03:45:10.656590+0000","last_active":"2023-07-06T03:45:13.827972+0000","last_peered":"2023-07-06T03:45:13.827972+0000","last_clean":"2023-07-06T03:45:13.827972+0000","last_became_active":"2023-07-06T03:45:10.656328+0000","last_became_peered":"2023-07-06T03:45:10.656328+0000","last_unstale":"2023-07-06T03:45:13.827972+0000","last_undegraded":"2023-07-06T03:45:13.827972+0000","last_fullsized":"2023-07-06T03:45:13.827972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205720+0000","last_change":"2023-07-06T03:45:11.671697+0000","last_active":"2023-07-06T03:45:14.205720+0000","last_peered":"2023-07-06T03:45:14.205720+0000","last_clean":"2023-07-06T03:45:14.205720+0000","last_became_active":"2023-07-06T03:45:11.671343+0000","last_became_peered":"2023-07-06T03:45:11.671343+0000","last_unstale":"2023-07-06T03:45:14.205720+0000","last_undegraded":"2023-07-06T03:45:14.205720+0000","last_fullsized":"2023-07-06T03:45:14.205720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:13.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829989+0000","last_change":"2023-07-06T03:45:10.644219+0000","last_active":"2023-07-06T03:45:13.829989+0000","last_peered":"2023-07-06T03:45:13.829989+0000","last_clean":"2023-07-06T03:45:13.829989+0000","last_became_active":"2023-07-06T03:45:10.643854+0000","last_became_peered":"2023-07-06T03:45:10.643854+0000","last_unstale":"2023-07-06T03:45:13.829989+0000","last_undegraded":"2023-07-06T03:45:13.829989+0000","last_fullsized":"2023-07-06T03:45:13.829989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:50:44.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678700+0000","last_change":"2023-07-06T03:45:11.665747+0000","last_active":"2023-07-06T03:45:12.678700+0000","last_peered":"2023-07-06T03:45:12.678700+0000","last_clean":"2023-07-06T03:45:12.678700+0000","last_became_active":"2023-07-06T03:45:11.665443+0000","last_became_peered":"2023-07-06T03:45:11.665443+0000","last_unstale":"2023-07-06T03:45:12.678700+0000","last_undegraded":"2023-07-06T03:45:12.678700+0000","last_fullsized":"2023-07-06T03:45:12.678700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829814+0000","last_change":"2023-07-06T03:45:10.653534+0000","last_active":"2023-07-06T03:45:13.829814+0000","last_peered":"2023-07-06T03:45:13.829814+0000","last_clean":"2023-07-06T03:45:13.829814+0000","last_became_active":"2023-07-06T03:45:10.653232+0000","last_became_peered":"2023-07-06T03:45:10.653232+0000","last_unstale":"2023-07-06T03:45:13.829814+0000","last_undegraded":"2023-07-06T03:45:13.829814+0000","last_fullsized":"2023-07-06T03:45:13.829814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:16.021365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652974+0000","last_change":"2023-07-06T03:45:11.683611+0000","last_active":"2023-07-06T03:45:13.652974+0000","last_peered":"2023-07-06T03:45:13.652974+0000","last_clean":"2023-07-06T03:45:13.652974+0000","last_became_active":"2023-07-06T03:45:11.683343+0000","last_became_peered":"2023-07-06T03:45:11.683343+0000","last_unstale":"2023-07-06T03:45:13.652974+0000","last_undegraded":"2023-07-06T03:45:13.652974+0000","last_fullsized":"2023-07-06T03:45:13.652974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652756+0000","last_change":"2023-07-06T03:45:10.655738+0000","last_active":"2023-07-06T03:45:13.652756+0000","last_peered":"2023-07-06T03:45:13.652756+0000","last_clean":"2023-07-06T03:45:13.652756+0000","last_became_active":"2023-07-06T03:45:10.655414+0000","last_became_peered":"2023-07-06T03:45:10.655414+0000","last_unstale":"2023-07-06T03:45:13.652756+0000","last_undegraded":"2023-07-06T03:45:13.652756+0000","last_fullsized":"2023-07-06T03:45:13.652756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:38.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830471+0000","last_change":"2023-07-06T03:45:11.678704+0000","last_active":"2023-07-06T03:45:13.830471+0000","last_peered":"2023-07-06T03:45:13.830471+0000","last_clean":"2023-07-06T03:45:13.830471+0000","last_became_active":"2023-07-06T03:45:11.678393+0000","last_became_peered":"2023-07-06T03:45:11.678393+0000","last_unstale":"2023-07-06T03:45:13.830471+0000","last_undegraded":"2023-07-06T03:45:13.830471+0000","last_fullsized":"2023-07-06T03:45:13.830471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:42.303647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830064+0000","last_change":"2023-07-06T03:45:10.644075+0000","last_active":"2023-07-06T03:45:13.830064+0000","last_peered":"2023-07-06T03:45:13.830064+0000","last_clean":"2023-07-06T03:45:13.830064+0000","last_became_active":"2023-07-06T03:45:10.643785+0000","last_became_peered":"2023-07-06T03:45:10.643785+0000","last_unstale":"2023-07-06T03:45:13.830064+0000","last_undegraded":"2023-07-06T03:45:13.830064+0000","last_fullsized":"2023-07-06T03:45:13.830064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:01:07.035467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681216+0000","last_change":"2023-07-06T03:45:11.687766+0000","last_active":"2023-07-06T03:45:12.681216+0000","last_peered":"2023-07-06T03:45:12.681216+0000","last_clean":"2023-07-06T03:45:12.681216+0000","last_became_active":"2023-07-06T03:45:11.687528+0000","last_became_peered":"2023-07-06T03:45:11.687528+0000","last_unstale":"2023-07-06T03:45:12.681216+0000","last_undegraded":"2023-07-06T03:45:12.681216+0000","last_fullsized":"2023-07-06T03:45:12.681216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817929+0000","last_change":"2023-07-06T03:45:10.652546+0000","last_active":"2023-07-06T03:45:12.817929+0000","last_peered":"2023-07-06T03:45:12.817929+0000","last_clean":"2023-07-06T03:45:12.817929+0000","last_became_active":"2023-07-06T03:45:10.652314+0000","last_became_peered":"2023-07-06T03:45:10.652314+0000","last_unstale":"2023-07-06T03:45:12.817929+0000","last_undegraded":"2023-07-06T03:45:12.817929+0000","last_fullsized":"2023-07-06T03:45:12.817929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681543+0000","last_change":"2023-07-06T03:45:11.695000+0000","last_active":"2023-07-06T03:45:12.681543+0000","last_peered":"2023-07-06T03:45:12.681543+0000","last_clean":"2023-07-06T03:45:12.681543+0000","last_became_active":"2023-07-06T03:45:11.694569+0000","last_became_peered":"2023-07-06T03:45:11.694569+0000","last_unstale":"2023-07-06T03:45:12.681543+0000","last_undegraded":"2023-07-06T03:45:12.681543+0000","last_fullsized":"2023-07-06T03:45:12.681543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:52.666500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.828097+0000","last_change":"2023-07-06T03:45:10.654280+0000","last_active":"2023-07-06T03:45:13.828097+0000","last_peered":"2023-07-06T03:45:13.828097+0000","last_clean":"2023-07-06T03:45:13.828097+0000","last_became_active":"2023-07-06T03:45:10.654039+0000","last_became_peered":"2023-07-06T03:45:10.654039+0000","last_unstale":"2023-07-06T03:45:13.828097+0000","last_undegraded":"2023-07-06T03:45:13.828097+0000","last_fullsized":"2023-07-06T03:45:13.828097+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:07.054998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829778+0000","last_change":"2023-07-06T03:45:11.667282+0000","last_active":"2023-07-06T03:45:13.829778+0000","last_peered":"2023-07-06T03:45:13.829778+0000","last_clean":"2023-07-06T03:45:13.829778+0000","last_became_active":"2023-07-06T03:45:11.667027+0000","last_became_peered":"2023-07-06T03:45:11.667027+0000","last_unstale":"2023-07-06T03:45:13.829778+0000","last_undegraded":"2023-07-06T03:45:13.829778+0000","last_fullsized":"2023-07-06T03:45:13.829778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:05:40.764560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678971+0000","last_change":"2023-07-06T03:45:10.647142+0000","last_active":"2023-07-06T03:45:12.678971+0000","last_peered":"2023-07-06T03:45:12.678971+0000","last_clean":"2023-07-06T03:45:12.678971+0000","last_became_active":"2023-07-06T03:45:10.646882+0000","last_became_peered":"2023-07-06T03:45:10.646882+0000","last_unstale":"2023-07-06T03:45:12.678971+0000","last_undegraded":"2023-07-06T03:45:12.678971+0000","last_fullsized":"2023-07-06T03:45:12.678971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:59.089238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817907+0000","last_change":"2023-07-06T03:45:11.691436+0000","last_active":"2023-07-06T03:45:12.817907+0000","last_peered":"2023-07-06T03:45:12.817907+0000","last_clean":"2023-07-06T03:45:12.817907+0000","last_became_active":"2023-07-06T03:45:11.691139+0000","last_became_peered":"2023-07-06T03:45:11.691139+0000","last_unstale":"2023-07-06T03:45:12.817907+0000","last_undegraded":"2023-07-06T03:45:12.817907+0000","last_fullsized":"2023-07-06T03:45:12.817907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829985+0000","last_change":"2023-07-06T03:45:10.647677+0000","last_active":"2023-07-06T03:45:13.829985+0000","last_peered":"2023-07-06T03:45:13.829985+0000","last_clean":"2023-07-06T03:45:13.829985+0000","last_became_active":"2023-07-06T03:45:10.647414+0000","last_became_peered":"2023-07-06T03:45:10.647414+0000","last_unstale":"2023-07-06T03:45:13.829985+0000","last_undegraded":"2023-07-06T03:45:13.829985+0000","last_fullsized":"2023-07-06T03:45:13.829985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:34:02.031995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678552+0000","last_change":"2023-07-06T03:45:11.663454+0000","last_active":"2023-07-06T03:45:12.678552+0000","last_peered":"2023-07-06T03:45:12.678552+0000","last_clean":"2023-07-06T03:45:12.678552+0000","last_became_active":"2023-07-06T03:45:11.663172+0000","last_became_peered":"2023-07-06T03:45:11.663172+0000","last_unstale":"2023-07-06T03:45:12.678552+0000","last_undegraded":"2023-07-06T03:45:12.678552+0000","last_fullsized":"2023-07-06T03:45:12.678552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827748+0000","last_change":"2023-07-06T03:45:10.651255+0000","last_active":"2023-07-06T03:45:13.827748+0000","last_peered":"2023-07-06T03:45:13.827748+0000","last_clean":"2023-07-06T03:45:13.827748+0000","last_became_active":"2023-07-06T03:45:10.651012+0000","last_became_peered":"2023-07-06T03:45:10.651012+0000","last_unstale":"2023-07-06T03:45:13.827748+0000","last_undegraded":"2023-07-06T03:45:13.827748+0000","last_fullsized":"2023-07-06T03:45:13.827748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:46.872621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680880+0000","last_change":"2023-07-06T03:45:11.688331+0000","last_active":"2023-07-06T03:45:12.680880+0000","last_peered":"2023-07-06T03:45:12.680880+0000","last_clean":"2023-07-06T03:45:12.680880+0000","last_became_active":"2023-07-06T03:45:11.686894+0000","last_became_peered":"2023-07-06T03:45:11.686894+0000","last_unstale":"2023-07-06T03:45:12.680880+0000","last_undegraded":"2023-07-06T03:45:12.680880+0000","last_fullsized":"2023-07-06T03:45:12.680880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:08.062104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678168+0000","last_change":"2023-07-06T03:45:10.653064+0000","last_active":"2023-07-06T03:45:12.678168+0000","last_peered":"2023-07-06T03:45:12.678168+0000","last_clean":"2023-07-06T03:45:12.678168+0000","last_became_active":"2023-07-06T03:45:10.652765+0000","last_became_peered":"2023-07-06T03:45:10.652765+0000","last_unstale":"2023-07-06T03:45:12.678168+0000","last_undegraded":"2023-07-06T03:45:12.678168+0000","last_fullsized":"2023-07-06T03:45:12.678168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:23.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827686+0000","last_change":"2023-07-06T03:45:11.683336+0000","last_active":"2023-07-06T03:45:13.827686+0000","last_peered":"2023-07-06T03:45:13.827686+0000","last_clean":"2023-07-06T03:45:13.827686+0000","last_became_active":"2023-07-06T03:45:11.683087+0000","last_became_peered":"2023-07-06T03:45:11.683087+0000","last_unstale":"2023-07-06T03:45:13.827686+0000","last_undegraded":"2023-07-06T03:45:13.827686+0000","last_fullsized":"2023-07-06T03:45:13.827686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830000+0000","last_change":"2023-07-06T03:45:10.650363+0000","last_active":"2023-07-06T03:45:13.830000+0000","last_peered":"2023-07-06T03:45:13.830000+0000","last_clean":"2023-07-06T03:45:13.830000+0000","last_became_active":"2023-07-06T03:45:10.650097+0000","last_became_peered":"2023-07-06T03:45:10.650097+0000","last_unstale":"2023-07-06T03:45:13.830000+0000","last_undegraded":"2023-07-06T03:45:13.830000+0000","last_fullsized":"2023-07-06T03:45:13.830000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:05.604877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830398+0000","last_change":"2023-07-06T03:45:11.668010+0000","last_active":"2023-07-06T03:45:13.830398+0000","last_peered":"2023-07-06T03:45:13.830398+0000","last_clean":"2023-07-06T03:45:13.830398+0000","last_became_active":"2023-07-06T03:45:11.667739+0000","last_became_peered":"2023-07-06T03:45:11.667739+0000","last_unstale":"2023-07-06T03:45:13.830398+0000","last_undegraded":"2023-07-06T03:45:13.830398+0000","last_fullsized":"2023-07-06T03:45:13.830398+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:06:53.685059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829747+0000","last_change":"2023-07-06T03:45:10.661165+0000","last_active":"2023-07-06T03:45:13.829747+0000","last_peered":"2023-07-06T03:45:13.829747+0000","last_clean":"2023-07-06T03:45:13.829747+0000","last_became_active":"2023-07-06T03:45:10.660843+0000","last_became_peered":"2023-07-06T03:45:10.660843+0000","last_unstale":"2023-07-06T03:45:13.829747+0000","last_undegraded":"2023-07-06T03:45:13.829747+0000","last_fullsized":"2023-07-06T03:45:13.829747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:02:20.704169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678908+0000","last_change":"2023-07-06T03:45:11.665692+0000","last_active":"2023-07-06T03:45:12.678908+0000","last_peered":"2023-07-06T03:45:12.678908+0000","last_clean":"2023-07-06T03:45:12.678908+0000","last_became_active":"2023-07-06T03:45:11.665425+0000","last_became_peered":"2023-07-06T03:45:11.665425+0000","last_unstale":"2023-07-06T03:45:12.678908+0000","last_undegraded":"2023-07-06T03:45:12.678908+0000","last_fullsized":"2023-07-06T03:45:12.678908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205473+0000","last_change":"2023-07-06T03:45:10.658201+0000","last_active":"2023-07-06T03:45:14.205473+0000","last_peered":"2023-07-06T03:45:14.205473+0000","last_clean":"2023-07-06T03:45:14.205473+0000","last_became_active":"2023-07-06T03:45:10.657688+0000","last_became_peered":"2023-07-06T03:45:10.657688+0000","last_unstale":"2023-07-06T03:45:14.205473+0000","last_undegraded":"2023-07-06T03:45:14.205473+0000","last_fullsized":"2023-07-06T03:45:14.205473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:26.377648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678497+0000","last_change":"2023-07-06T03:45:11.673905+0000","last_active":"2023-07-06T03:45:12.678497+0000","last_peered":"2023-07-06T03:45:12.678497+0000","last_clean":"2023-07-06T03:45:12.678497+0000","last_became_active":"2023-07-06T03:45:11.673609+0000","last_became_peered":"2023-07-06T03:45:11.673609+0000","last_unstale":"2023-07-06T03:45:12.678497+0000","last_undegraded":"2023-07-06T03:45:12.678497+0000","last_fullsized":"2023-07-06T03:45:12.678497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681110+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:45:12.681110+0000","last_peered":"2023-07-06T03:45:12.681110+0000","last_clean":"2023-07-06T03:45:12.681110+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:45:12.681110+0000","last_undegraded":"2023-07-06T03:45:12.681110+0000","last_fullsized":"2023-07-06T03:45:12.681110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:20.874020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678806+0000","last_change":"2023-07-06T03:45:10.660307+0000","last_active":"2023-07-06T03:45:12.678806+0000","last_peered":"2023-07-06T03:45:12.678806+0000","last_clean":"2023-07-06T03:45:12.678806+0000","last_became_active":"2023-07-06T03:45:10.660032+0000","last_became_peered":"2023-07-06T03:45:10.660032+0000","last_unstale":"2023-07-06T03:45:12.678806+0000","last_undegraded":"2023-07-06T03:45:12.678806+0000","last_fullsized":"2023-07-06T03:45:12.678806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652629+0000","last_change":"2023-07-06T03:45:11.679839+0000","last_active":"2023-07-06T03:45:13.652629+0000","last_peered":"2023-07-06T03:45:13.652629+0000","last_clean":"2023-07-06T03:45:13.652629+0000","last_became_active":"2023-07-06T03:45:11.679509+0000","last_became_peered":"2023-07-06T03:45:11.679509+0000","last_unstale":"2023-07-06T03:45:13.652629+0000","last_undegraded":"2023-07-06T03:45:13.652629+0000","last_fullsized":"2023-07-06T03:45:13.652629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830179+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:45:13.830179+0000","last_peered":"2023-07-06T03:45:13.830179+0000","last_clean":"2023-07-06T03:45:13.830179+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:45:13.830179+0000","last_undegraded":"2023-07-06T03:45:13.830179+0000","last_fullsized":"2023-07-06T03:45:13.830179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:03:06.569126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827774+0000","last_change":"2023-07-06T03:45:10.649547+0000","last_active":"2023-07-06T03:45:13.827774+0000","last_peered":"2023-07-06T03:45:13.827774+0000","last_clean":"2023-07-06T03:45:13.827774+0000","last_became_active":"2023-07-06T03:45:10.649231+0000","last_became_peered":"2023-07-06T03:45:10.649231+0000","last_unstale":"2023-07-06T03:45:13.827774+0000","last_undegraded":"2023-07-06T03:45:13.827774+0000","last_fullsized":"2023-07-06T03:45:13.827774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:13.496490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653155+0000","last_change":"2023-07-06T03:45:11.678151+0000","last_active":"2023-07-06T03:45:13.653155+0000","last_peered":"2023-07-06T03:45:13.653155+0000","last_clean":"2023-07-06T03:45:13.653155+0000","last_became_active":"2023-07-06T03:45:11.677841+0000","last_became_peered":"2023-07-06T03:45:11.677841+0000","last_unstale":"2023-07-06T03:45:13.653155+0000","last_undegraded":"2023-07-06T03:45:13.653155+0000","last_fullsized":"2023-07-06T03:45:13.653155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678308+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:45:12.678308+0000","last_peered":"2023-07-06T03:45:12.678308+0000","last_clean":"2023-07-06T03:45:12.678308+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:45:12.678308+0000","last_undegraded":"2023-07-06T03:45:12.678308+0000","last_fullsized":"2023-07-06T03:45:12.678308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:11.498012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680537+0000","last_change":"2023-07-06T03:45:10.647981+0000","last_active":"2023-07-06T03:45:12.680537+0000","last_peered":"2023-07-06T03:45:12.680537+0000","last_clean":"2023-07-06T03:45:12.680537+0000","last_became_active":"2023-07-06T03:45:10.647678+0000","last_became_peered":"2023-07-06T03:45:10.647678+0000","last_unstale":"2023-07-06T03:45:12.680537+0000","last_undegraded":"2023-07-06T03:45:12.680537+0000","last_fullsized":"2023-07-06T03:45:12.680537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653181+0000","last_change":"2023-07-06T03:45:11.683452+0000","last_active":"2023-07-06T03:45:13.653181+0000","last_peered":"2023-07-06T03:45:13.653181+0000","last_clean":"2023-07-06T03:45:13.653181+0000","last_became_active":"2023-07-06T03:45:11.683187+0000","last_became_peered":"2023-07-06T03:45:11.683187+0000","last_unstale":"2023-07-06T03:45:13.653181+0000","last_undegraded":"2023-07-06T03:45:13.653181+0000","last_fullsized":"2023-07-06T03:45:13.653181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:38.118602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829567+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:45:13.829567+0000","last_peered":"2023-07-06T03:45:13.829567+0000","last_clean":"2023-07-06T03:45:13.829567+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:45:13.829567+0000","last_undegraded":"2023-07-06T03:45:13.829567+0000","last_fullsized":"2023-07-06T03:45:13.829567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:49:25.728537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817899+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:45:12.817899+0000","last_peered":"2023-07-06T03:45:12.817899+0000","last_clean":"2023-07-06T03:45:12.817899+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:45:12.817899+0000","last_undegraded":"2023-07-06T03:45:12.817899+0000","last_fullsized":"2023-07-06T03:45:12.817899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:39.120643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205527+0000","last_change":"2023-07-06T03:45:10.657235+0000","last_active":"2023-07-06T03:45:14.205527+0000","last_peered":"2023-07-06T03:45:14.205527+0000","last_clean":"2023-07-06T03:45:14.205527+0000","last_became_active":"2023-07-06T03:45:10.656945+0000","last_became_peered":"2023-07-06T03:45:10.656945+0000","last_unstale":"2023-07-06T03:45:14.205527+0000","last_undegraded":"2023-07-06T03:45:14.205527+0000","last_fullsized":"2023-07-06T03:45:14.205527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:57.080176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830434+0000","last_change":"2023-07-06T03:45:11.668172+0000","last_active":"2023-07-06T03:45:13.830434+0000","last_peered":"2023-07-06T03:45:13.830434+0000","last_clean":"2023-07-06T03:45:13.830434+0000","last_became_active":"2023-07-06T03:45:11.667041+0000","last_became_peered":"2023-07-06T03:45:11.667041+0000","last_unstale":"2023-07-06T03:45:13.830434+0000","last_undegraded":"2023-07-06T03:45:13.830434+0000","last_fullsized":"2023-07-06T03:45:13.830434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:57.552603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829547+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:45:13.829547+0000","last_peered":"2023-07-06T03:45:13.829547+0000","last_clean":"2023-07-06T03:45:13.829547+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:45:13.829547+0000","last_undegraded":"2023-07-06T03:45:13.829547+0000","last_fullsized":"2023-07-06T03:45:13.829547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:25:39.144218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652538+0000","last_change":"2023-07-06T03:45:10.660900+0000","last_active":"2023-07-06T03:45:13.652538+0000","last_peered":"2023-07-06T03:45:13.652538+0000","last_clean":"2023-07-06T03:45:13.652538+0000","last_became_active":"2023-07-06T03:45:10.660490+0000","last_became_peered":"2023-07-06T03:45:10.660490+0000","last_unstale":"2023-07-06T03:45:13.652538+0000","last_undegraded":"2023-07-06T03:45:13.652538+0000","last_fullsized":"2023-07-06T03:45:13.652538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:43.801501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681363+0000","last_change":"2023-07-06T03:45:11.687991+0000","last_active":"2023-07-06T03:45:12.681363+0000","last_peered":"2023-07-06T03:45:12.681363+0000","last_clean":"2023-07-06T03:45:12.681363+0000","last_became_active":"2023-07-06T03:45:11.685096+0000","last_became_peered":"2023-07-06T03:45:11.685096+0000","last_unstale":"2023-07-06T03:45:12.681363+0000","last_undegraded":"2023-07-06T03:45:12.681363+0000","last_fullsized":"2023-07-06T03:45:12.681363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830169+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:45:13.830169+0000","last_peered":"2023-07-06T03:45:13.830169+0000","last_clean":"2023-07-06T03:45:13.830169+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:45:13.830169+0000","last_undegraded":"2023-07-06T03:45:13.830169+0000","last_fullsized":"2023-07-06T03:45:13.830169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:27:55.564390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653143+0000","last_change":"2023-07-06T03:45:10.661812+0000","last_active":"2023-07-06T03:45:13.653143+0000","last_peered":"2023-07-06T03:45:13.653143+0000","last_clean":"2023-07-06T03:45:13.653143+0000","last_became_active":"2023-07-06T03:45:10.661499+0000","last_became_peered":"2023-07-06T03:45:10.661499+0000","last_unstale":"2023-07-06T03:45:13.653143+0000","last_undegraded":"2023-07-06T03:45:13.653143+0000","last_fullsized":"2023-07-06T03:45:13.653143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:59.089238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652878+0000","last_change":"2023-07-06T03:45:11.678171+0000","last_active":"2023-07-06T03:45:13.652878+0000","last_peered":"2023-07-06T03:45:13.652878+0000","last_clean":"2023-07-06T03:45:13.652878+0000","last_became_active":"2023-07-06T03:45:11.677886+0000","last_became_peered":"2023-07-06T03:45:11.677886+0000","last_unstale":"2023-07-06T03:45:13.652878+0000","last_undegraded":"2023-07-06T03:45:13.652878+0000","last_fullsized":"2023-07-06T03:45:13.652878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206208+0000","last_change":"2023-07-06T03:45:10.657429+0000","last_active":"2023-07-06T03:45:14.206208+0000","last_peered":"2023-07-06T03:45:14.206208+0000","last_clean":"2023-07-06T03:45:14.206208+0000","last_became_active":"2023-07-06T03:45:10.657085+0000","last_became_peered":"2023-07-06T03:45:10.657085+0000","last_unstale":"2023-07-06T03:45:14.206208+0000","last_undegraded":"2023-07-06T03:45:14.206208+0000","last_fullsized":"2023-07-06T03:45:14.206208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:24:47.052321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206026+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:45:14.206026+0000","last_peered":"2023-07-06T03:45:14.206026+0000","last_clean":"2023-07-06T03:45:14.206026+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:45:14.206026+0000","last_undegraded":"2023-07-06T03:45:14.206026+0000","last_fullsized":"2023-07-06T03:45:14.206026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:05:14.989733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205502+0000","last_change":"2023-07-06T03:45:11.671780+0000","last_active":"2023-07-06T03:45:14.205502+0000","last_peered":"2023-07-06T03:45:14.205502+0000","last_clean":"2023-07-06T03:45:14.205502+0000","last_became_active":"2023-07-06T03:45:11.671390+0000","last_became_peered":"2023-07-06T03:45:11.671390+0000","last_unstale":"2023-07-06T03:45:14.205502+0000","last_undegraded":"2023-07-06T03:45:14.205502+0000","last_fullsized":"2023-07-06T03:45:14.205502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829963+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:45:13.829963+0000","last_peered":"2023-07-06T03:45:13.829963+0000","last_clean":"2023-07-06T03:45:13.829963+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:45:13.829963+0000","last_undegraded":"2023-07-06T03:45:13.829963+0000","last_fullsized":"2023-07-06T03:45:13.829963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:12.638307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206183+0000","last_change":"2023-07-06T03:45:10.657912+0000","last_active":"2023-07-06T03:45:14.206183+0000","last_peered":"2023-07-06T03:45:14.206183+0000","last_clean":"2023-07-06T03:45:14.206183+0000","last_became_active":"2023-07-06T03:45:10.657615+0000","last_became_peered":"2023-07-06T03:45:10.657615+0000","last_unstale":"2023-07-06T03:45:14.206183+0000","last_undegraded":"2023-07-06T03:45:14.206183+0000","last_fullsized":"2023-07-06T03:45:14.206183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:30:30.877422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678345+0000","last_change":"2023-07-06T03:45:11.665120+0000","last_active":"2023-07-06T03:45:12.678345+0000","last_peered":"2023-07-06T03:45:12.678345+0000","last_clean":"2023-07-06T03:45:12.678345+0000","last_became_active":"2023-07-06T03:45:11.664817+0000","last_became_peered":"2023-07-06T03:45:11.664817+0000","last_unstale":"2023-07-06T03:45:12.678345+0000","last_undegraded":"2023-07-06T03:45:12.678345+0000","last_fullsized":"2023-07-06T03:45:12.678345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:44.808607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205783+0000","last_change":"2023-07-06T03:45:10.658517+0000","last_active":"2023-07-06T03:45:14.205783+0000","last_peered":"2023-07-06T03:45:14.205783+0000","last_clean":"2023-07-06T03:45:14.205783+0000","last_became_active":"2023-07-06T03:45:10.657946+0000","last_became_peered":"2023-07-06T03:45:10.657946+0000","last_unstale":"2023-07-06T03:45:14.205783+0000","last_undegraded":"2023-07-06T03:45:14.205783+0000","last_fullsized":"2023-07-06T03:45:14.205783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:24:42.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205842+0000","last_change":"2023-07-06T03:45:11.683447+0000","last_active":"2023-07-06T03:45:14.205842+0000","last_peered":"2023-07-06T03:45:14.205842+0000","last_clean":"2023-07-06T03:45:14.205842+0000","last_became_active":"2023-07-06T03:45:11.683156+0000","last_became_peered":"2023-07-06T03:45:11.683156+0000","last_unstale":"2023-07-06T03:45:14.205842+0000","last_undegraded":"2023-07-06T03:45:14.205842+0000","last_fullsized":"2023-07-06T03:45:14.205842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818006+0000","last_change":"2023-07-06T03:45:10.649186+0000","last_active":"2023-07-06T03:45:12.818006+0000","last_peered":"2023-07-06T03:45:12.818006+0000","last_clean":"2023-07-06T03:45:12.818006+0000","last_became_active":"2023-07-06T03:45:10.648707+0000","last_became_peered":"2023-07-06T03:45:10.648707+0000","last_unstale":"2023-07-06T03:45:12.818006+0000","last_undegraded":"2023-07-06T03:45:12.818006+0000","last_fullsized":"2023-07-06T03:45:12.818006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:07.054998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678566+0000","last_change":"2023-07-06T03:45:11.690462+0000","last_active":"2023-07-06T03:45:12.678566+0000","last_peered":"2023-07-06T03:45:12.678566+0000","last_clean":"2023-07-06T03:45:12.678566+0000","last_became_active":"2023-07-06T03:45:11.690181+0000","last_became_peered":"2023-07-06T03:45:11.690181+0000","last_unstale":"2023-07-06T03:45:12.678566+0000","last_undegraded":"2023-07-06T03:45:12.678566+0000","last_fullsized":"2023-07-06T03:45:12.678566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830291+0000","last_change":"2023-07-06T03:45:10.653276+0000","last_active":"2023-07-06T03:45:13.830291+0000","last_peered":"2023-07-06T03:45:13.830291+0000","last_clean":"2023-07-06T03:45:13.830291+0000","last_became_active":"2023-07-06T03:45:10.652958+0000","last_became_peered":"2023-07-06T03:45:10.652958+0000","last_unstale":"2023-07-06T03:45:13.830291+0000","last_undegraded":"2023-07-06T03:45:13.830291+0000","last_fullsized":"2023-07-06T03:45:13.830291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:45.669370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678497+0000","last_change":"2023-07-06T03:45:11.684391+0000","last_active":"2023-07-06T03:45:12.678497+0000","last_peered":"2023-07-06T03:45:12.678497+0000","last_clean":"2023-07-06T03:45:12.678497+0000","last_became_active":"2023-07-06T03:45:11.684078+0000","last_became_peered":"2023-07-06T03:45:11.684078+0000","last_unstale":"2023-07-06T03:45:12.678497+0000","last_undegraded":"2023-07-06T03:45:12.678497+0000","last_fullsized":"2023-07-06T03:45:12.678497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681347+0000","last_change":"2023-07-06T03:45:10.645296+0000","last_active":"2023-07-06T03:45:12.681347+0000","last_peered":"2023-07-06T03:45:12.681347+0000","last_clean":"2023-07-06T03:45:12.681347+0000","last_became_active":"2023-07-06T03:45:10.644967+0000","last_became_peered":"2023-07-06T03:45:10.644967+0000","last_unstale":"2023-07-06T03:45:12.681347+0000","last_undegraded":"2023-07-06T03:45:12.681347+0000","last_fullsized":"2023-07-06T03:45:12.681347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:12.862578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678143+0000","last_change":"2023-07-06T03:45:11.683562+0000","last_active":"2023-07-06T03:45:12.678143+0000","last_peered":"2023-07-06T03:45:12.678143+0000","last_clean":"2023-07-06T03:45:12.678143+0000","last_became_active":"2023-07-06T03:45:11.683275+0000","last_became_peered":"2023-07-06T03:45:11.683275+0000","last_unstale":"2023-07-06T03:45:12.678143+0000","last_undegraded":"2023-07-06T03:45:12.678143+0000","last_fullsized":"2023-07-06T03:45:12.678143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:44.808607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681723+0000","last_change":"2023-07-06T03:45:10.654706+0000","last_active":"2023-07-06T03:45:12.681723+0000","last_peered":"2023-07-06T03:45:12.681723+0000","last_clean":"2023-07-06T03:45:12.681723+0000","last_became_active":"2023-07-06T03:45:10.653995+0000","last_became_peered":"2023-07-06T03:45:10.653995+0000","last_unstale":"2023-07-06T03:45:12.681723+0000","last_undegraded":"2023-07-06T03:45:12.681723+0000","last_fullsized":"2023-07-06T03:45:12.681723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:05:20.493361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678265+0000","last_change":"2023-07-06T03:45:11.670138+0000","last_active":"2023-07-06T03:45:12.678265+0000","last_peered":"2023-07-06T03:45:12.678265+0000","last_clean":"2023-07-06T03:45:12.678265+0000","last_became_active":"2023-07-06T03:45:11.669695+0000","last_became_peered":"2023-07-06T03:45:11.669695+0000","last_unstale":"2023-07-06T03:45:12.678265+0000","last_undegraded":"2023-07-06T03:45:12.678265+0000","last_fullsized":"2023-07-06T03:45:12.678265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:22:55.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818277+0000","last_change":"2023-07-06T03:45:11.673123+0000","last_active":"2023-07-06T03:45:12.818277+0000","last_peered":"2023-07-06T03:45:12.818277+0000","last_clean":"2023-07-06T03:45:12.818277+0000","last_became_active":"2023-07-06T03:45:11.672765+0000","last_became_peered":"2023-07-06T03:45:11.672765+0000","last_unstale":"2023-07-06T03:45:12.818277+0000","last_undegraded":"2023-07-06T03:45:12.818277+0000","last_fullsized":"2023-07-06T03:45:12.818277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652635+0000","last_change":"2023-07-06T03:45:11.689866+0000","last_active":"2023-07-06T03:45:13.652635+0000","last_peered":"2023-07-06T03:45:13.652635+0000","last_clean":"2023-07-06T03:45:13.652635+0000","last_became_active":"2023-07-06T03:45:11.689595+0000","last_became_peered":"2023-07-06T03:45:11.689595+0000","last_unstale":"2023-07-06T03:45:13.652635+0000","last_undegraded":"2023-07-06T03:45:13.652635+0000","last_fullsized":"2023-07-06T03:45:13.652635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827567+0000","last_change":"2023-07-06T03:45:11.675336+0000","last_active":"2023-07-06T03:45:13.827567+0000","last_peered":"2023-07-06T03:45:13.827567+0000","last_clean":"2023-07-06T03:45:13.827567+0000","last_became_active":"2023-07-06T03:45:11.675044+0000","last_became_peered":"2023-07-06T03:45:11.675044+0000","last_unstale":"2023-07-06T03:45:13.827567+0000","last_undegraded":"2023-07-06T03:45:13.827567+0000","last_fullsized":"2023-07-06T03:45:13.827567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680990+0000","last_change":"2023-07-06T03:45:11.689419+0000","last_active":"2023-07-06T03:45:12.680990+0000","last_peered":"2023-07-06T03:45:12.680990+0000","last_clean":"2023-07-06T03:45:12.680990+0000","last_became_active":"2023-07-06T03:45:11.688843+0000","last_became_peered":"2023-07-06T03:45:11.688843+0000","last_unstale":"2023-07-06T03:45:12.680990+0000","last_undegraded":"2023-07-06T03:45:12.680990+0000","last_fullsized":"2023-07-06T03:45:12.680990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:15.627811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680936+0000","last_change":"2023-07-06T03:45:11.687716+0000","last_active":"2023-07-06T03:45:12.680936+0000","last_peered":"2023-07-06T03:45:12.680936+0000","last_clean":"2023-07-06T03:45:12.680936+0000","last_became_active":"2023-07-06T03:45:11.685418+0000","last_became_peered":"2023-07-06T03:45:11.685418+0000","last_unstale":"2023-07-06T03:45:12.680936+0000","last_undegraded":"2023-07-06T03:45:12.680936+0000","last_fullsized":"2023-07-06T03:45:12.680936+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":847872,"data_stored":680641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":89745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":91163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:45:17.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-07-06T03:45:17.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.220+0000 7fafd25ff700 1 Processor -- start 2023-07-06T03:45:17.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.220+0000 7fafd25ff700 1 -- start start 2023-07-06T03:45:17.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.221+0000 7fafd25ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafcc0cd390 0x7fafcc160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.221+0000 7fafd25ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fafcc160dd0 0x7fafcc0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.221+0000 7fafd25ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc0cdae0 con 0x7fafcc15c520 2023-07-06T03:45:17.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.221+0000 7fafd25ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafcc0b6640 con 0x7fafcc160dd0 2023-07-06T03:45:17.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.221+0000 7fafd25ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0b6920 con 0x7fafcc0cd390 2023-07-06T03:45:17.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.221+0000 7fafcbfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fafcc15c520 0x7fafcc160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56068/0 (socket says 172.21.15.97:56068) 2023-07-06T03:45:17.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.221+0000 7fafcbfff700 1 -- 172.21.15.97:0/1250761157 learned_addr learned my addr 172.21.15.97:0/1250761157 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:17.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.222+0000 7fafcb7fe700 1 --2- 172.21.15.97:0/1250761157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafcc0cd390 0x7fafcc160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.222+0000 7fafd0b9c700 1 --2- 172.21.15.97:0/1250761157 >> v2:172.21.15.146:3300/0 conn(0x7fafcc160dd0 0x7fafcc0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.222+0000 7fafcaffd700 1 -- 172.21.15.97:0/1250761157 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174080774 0 0) 0x7fafcc0cdae0 con 0x7fafcc15c520 2023-07-06T03:45:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.222+0000 7fafcaffd700 1 -- 172.21.15.97:0/1250761157 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafb40035b0 con 0x7fafcc15c520 2023-07-06T03:45:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.222+0000 7fafcb7fe700 1 -- 172.21.15.97:0/1250761157 >> v2:172.21.15.146:3300/0 conn(0x7fafcc160dd0 msgr2=0x7fafcc0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.222+0000 7fafcb7fe700 1 --2- 172.21.15.97:0/1250761157 >> v2:172.21.15.146:3300/0 conn(0x7fafcc160dd0 0x7fafcc0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.222+0000 7fafcb7fe700 1 -- 172.21.15.97:0/1250761157 >> v1:172.21.15.97:6789/0 conn(0x7fafcc15c520 legacy=0x7fafcc160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.222+0000 7fafcaffd700 1 -- 172.21.15.97:0/1250761157 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4281866635 0 0) 0x7fafb40035b0 con 0x7fafcc15c520 2023-07-06T03:45:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.222+0000 7fafcb7fe700 1 -- 172.21.15.97:0/1250761157 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc0b6c00 con 0x7fafcc0cd390 2023-07-06T03:45:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.222+0000 7fafcb7fe700 1 --2- 172.21.15.97:0/1250761157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafcc0cd390 0x7fafcc160890 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009840 tx=0x7fafc0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7adbc2e8784516f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.223+0000 7fafcaffd700 1 -- 172.21.15.97:0/1250761157 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc001670 con 0x7fafcc0cd390 2023-07-06T03:45:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.223+0000 7fafcaffd700 1 -- 172.21.15.97:0/1250761157 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc0004030 con 0x7fafcc0cd390 2023-07-06T03:45:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.223+0000 7fafcaffd700 1 -- 172.21.15.97:0/1250761157 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00043c0 con 0x7fafcc0cd390 2023-07-06T03:45:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.223+0000 7fafd25ff700 1 -- 172.21.15.97:0/1250761157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafcc0cd390 msgr2=0x7fafcc160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.223+0000 7fafd25ff700 1 --2- 172.21.15.97:0/1250761157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafcc0cd390 0x7fafcc160890 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009840 tx=0x7fafc0009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.223+0000 7fafd25ff700 1 -- 172.21.15.97:0/1250761157 shutdown_connections 2023-07-06T03:45:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.223+0000 7fafd25ff700 1 --2- 172.21.15.97:0/1250761157 >> v2:172.21.15.146:3300/0 conn(0x7fafcc160dd0 0x7fafcc0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.223+0000 7fafd25ff700 1 --2- 172.21.15.97:0/1250761157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafcc0cd390 0x7fafcc160890 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.223+0000 7fafd25ff700 1 -- 172.21.15.97:0/1250761157 >> 172.21.15.97:0/1250761157 conn(0x7fafcc071790 msgr2=0x7fafcc071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:17.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.223+0000 7fafd25ff700 1 -- 172.21.15.97:0/1250761157 shutdown_connections 2023-07-06T03:45:17.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.223+0000 7fafd25ff700 1 -- 172.21.15.97:0/1250761157 wait complete. 2023-07-06T03:45:17.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.225+0000 7fafd25ff700 1 Processor -- start 2023-07-06T03:45:17.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.225+0000 7fafd25ff700 1 -- start start 2023-07-06T03:45:17.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.226+0000 7fafd25ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafcc0cd390 0x7fafcc0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.226+0000 7fafd25ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc15c520 0x7fafcc150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.226+0000 7fafd25ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafcc160dd0 0x7fafcc0d3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.226+0000 7fafd25ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc151190 con 0x7fafcc15c520 2023-07-06T03:45:17.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.226+0000 7fafd25ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0d0c40 con 0x7fafcc0cd390 2023-07-06T03:45:17.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.226+0000 7fafd25ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc15e150 con 0x7fafcc160dd0 2023-07-06T03:45:17.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.226+0000 7fafcb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc15c520 0x7fafcc150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.226+0000 7fafcb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc15c520 0x7fafcc150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46938/0 (socket says 172.21.15.97:46938) 2023-07-06T03:45:17.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.226+0000 7fafcb7fe700 1 -- 172.21.15.97:0/1004945724 learned_addr learned my addr 172.21.15.97:0/1004945724 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:17.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.226+0000 7fafcbfff700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafcc0cd390 0x7fafcc0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafd0b9c700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafcc160dd0 0x7fafcc0d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafcb7fe700 1 -- 172.21.15.97:0/1004945724 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafcc160dd0 msgr2=0x7fafcc0d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafcb7fe700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafcc160dd0 0x7fafcc0d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafcb7fe700 1 -- 172.21.15.97:0/1004945724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafcc0cd390 msgr2=0x7fafcc0c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafcb7fe700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafcc0cd390 0x7fafcc0c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafcb7fe700 1 -- 172.21.15.97:0/1004945724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc00094a0 con 0x7fafcc15c520 2023-07-06T03:45:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafcb7fe700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc15c520 0x7fafcc150c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fafc000a010 tx=0x7fafc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafc8ff9700 1 -- 172.21.15.97:0/1004945724 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0003ae0 con 0x7fafcc15c520 2023-07-06T03:45:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafc8ff9700 1 -- 172.21.15.97:0/1004945724 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc001f070 con 0x7fafcc15c520 2023-07-06T03:45:17.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafc8ff9700 1 -- 172.21.15.97:0/1004945724 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0003ea0 con 0x7fafcc15c520 2023-07-06T03:45:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafd25ff700 1 -- 172.21.15.97:0/1004945724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc15b060 con 0x7fafcc15c520 2023-07-06T03:45:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.227+0000 7fafd25ff700 1 -- 172.21.15.97:0/1004945724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc15b620 con 0x7fafcc15c520 2023-07-06T03:45:17.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.228+0000 7fafd25ff700 1 -- 172.21.15.97:0/1004945724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafcc04e340 con 0x7fafcc15c520 2023-07-06T03:45:17.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.229+0000 7fafc8ff9700 1 -- 172.21.15.97:0/1004945724 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fafc0021030 con 0x7fafcc15c520 2023-07-06T03:45:17.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.231+0000 7fafc8ff9700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafb806ace0 0x7fafb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.231+0000 7fafc8ff9700 1 -- 172.21.15.97:0/1004945724 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fafc008a6f0 con 0x7fafcc15c520 2023-07-06T03:45:17.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.231+0000 7fafcbfff700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafb806ace0 0x7fafb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.231+0000 7fafcbfff700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafb806ace0 0x7fafb806d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fafbc001eb0 tx=0x7fafbc001730 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:17.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.235+0000 7fafc8ff9700 1 -- 172.21.15.97:0/1004945724 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fafc0055790 con 0x7fafcc15c520 2023-07-06T03:45:17.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.512+0000 7fafd25ff700 1 -- 172.21.15.97:0/1004945724 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fafcc0b4f40 con 0x7fafcc15c520 2023-07-06T03:45:17.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.512+0000 7fafc8ff9700 1 -- 172.21.15.97:0/1004945724 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1032 (secure 0 0 0) 0x7fafc0058db0 con 0x7fafcc15c520 2023-07-06T03:45:17.514 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:17.515 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":33,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1688615095,"num_in_osds":8,"osd_in_since":1688615087,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":223662080,"bytes_avail":772870451200,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4290}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-07-06T03:44:43.771952+0000","services":{}},"progress_events":{}} 2023-07-06T03:45:17.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.514+0000 7fafd25ff700 1 -- 172.21.15.97:0/1004945724 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafb806ace0 msgr2=0x7fafb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.514+0000 7fafd25ff700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafb806ace0 0x7fafb806d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fafbc001eb0 tx=0x7fafbc001730 comp rx=0 tx=0).stop 2023-07-06T03:45:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.514+0000 7fafd25ff700 1 -- 172.21.15.97:0/1004945724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc15c520 msgr2=0x7fafcc150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.514+0000 7fafd25ff700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc15c520 0x7fafcc150c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fafc000a010 tx=0x7fafc0004750 comp rx=0 tx=0).stop 2023-07-06T03:45:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.515+0000 7fafd25ff700 1 -- 172.21.15.97:0/1004945724 shutdown_connections 2023-07-06T03:45:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.515+0000 7fafd25ff700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafb806ace0 0x7fafb806d190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.515+0000 7fafd25ff700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafcc160dd0 0x7fafcc0d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.515+0000 7fafd25ff700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafcc0cd390 0x7fafcc0c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.515+0000 7fafd25ff700 1 --2- 172.21.15.97:0/1004945724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafcc15c520 0x7fafcc150c50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.515+0000 7fafd25ff700 1 -- 172.21.15.97:0/1004945724 >> 172.21.15.97:0/1004945724 conn(0x7fafcc071790 msgr2=0x7fafcc0b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:17.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.515+0000 7fafd25ff700 1 -- 172.21.15.97:0/1004945724 shutdown_connections 2023-07-06T03:45:17.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.515+0000 7fafd25ff700 1 -- 172.21.15.97:0/1004945724 wait complete. 2023-07-06T03:45:17.534 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-07-06T03:45:17.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:45:17.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.729+0000 7f845cbff700 1 Processor -- start 2023-07-06T03:45:17.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.729+0000 7f845cbff700 1 -- start start 2023-07-06T03:45:17.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.730+0000 7f845cbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f845815c520 0x7f84580cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.730+0000 7f845cbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8458160180 0x7f84581605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.730+0000 7f845cbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8458160b30 0x7f84580cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.730+0000 7f845cbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84580cd8d0 con 0x7f8458160180 2023-07-06T03:45:17.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.730+0000 7f845cbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84580cdae0 con 0x7f8458160b30 2023-07-06T03:45:17.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.730+0000 7f845cbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84580b6640 con 0x7f845815c520 2023-07-06T03:45:17.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.730+0000 7f8455d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8458160180 0x7f84581605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.730+0000 7f8455d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8458160180 0x7f84581605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46942/0 (socket says 172.21.15.97:46942) 2023-07-06T03:45:17.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.730+0000 7f8455d9b700 1 -- 172.21.15.97:0/4015861823 learned_addr learned my addr 172.21.15.97:0/4015861823 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:17.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f845659c700 1 --2- 172.21.15.97:0/4015861823 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f845815c520 0x7f84580cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f8456d9d700 1 --2- 172.21.15.97:0/4015861823 >> v2:172.21.15.146:3300/0 conn(0x7f8458160b30 0x7f84580cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f8455d9b700 1 -- 172.21.15.97:0/4015861823 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f845815c520 msgr2=0x7f84580cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:17.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f8455d9b700 1 --2- 172.21.15.97:0/4015861823 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f845815c520 0x7f84580cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f8455d9b700 1 -- 172.21.15.97:0/4015861823 >> v2:172.21.15.146:3300/0 conn(0x7f8458160b30 msgr2=0x7f84580cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:17.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f8455d9b700 1 --2- 172.21.15.97:0/4015861823 >> v2:172.21.15.146:3300/0 conn(0x7f8458160b30 0x7f84580cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f8455d9b700 1 -- 172.21.15.97:0/4015861823 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84580b6920 con 0x7f8458160180 2023-07-06T03:45:17.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f845659c700 1 --2- 172.21.15.97:0/4015861823 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f845815c520 0x7f84580cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:17.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f8456d9d700 1 --2- 172.21.15.97:0/4015861823 >> v2:172.21.15.146:3300/0 conn(0x7f8458160b30 0x7f84580cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:17.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f8455d9b700 1 --2- 172.21.15.97:0/4015861823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8458160180 0x7f84581605f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f845808bfc0 tx=0x7f844c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad118afa0133dfa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:17.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f845559a700 1 -- 172.21.15.97:0/4015861823 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c00b870 con 0x7f8458160180 2023-07-06T03:45:17.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f845559a700 1 -- 172.21.15.97:0/4015861823 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c00b9d0 con 0x7f8458160180 2023-07-06T03:45:17.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.731+0000 7f845559a700 1 -- 172.21.15.97:0/4015861823 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c00bd60 con 0x7f8458160180 2023-07-06T03:45:17.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.732+0000 7f845cbff700 1 -- 172.21.15.97:0/4015861823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8458160180 msgr2=0x7f84581605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:17.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.732+0000 7f845cbff700 1 --2- 172.21.15.97:0/4015861823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8458160180 0x7f84581605f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f845808bfc0 tx=0x7f844c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.732+0000 7f845cbff700 1 -- 172.21.15.97:0/4015861823 shutdown_connections 2023-07-06T03:45:17.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.732+0000 7f845cbff700 1 --2- 172.21.15.97:0/4015861823 >> v2:172.21.15.146:3300/0 conn(0x7f8458160b30 0x7f84580cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.732+0000 7f845cbff700 1 --2- 172.21.15.97:0/4015861823 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f845815c520 0x7f84580cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.732+0000 7f845cbff700 1 --2- 172.21.15.97:0/4015861823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8458160180 0x7f84581605f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.732+0000 7f845cbff700 1 -- 172.21.15.97:0/4015861823 >> 172.21.15.97:0/4015861823 conn(0x7f8458071790 msgr2=0x7f8458071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:17.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.732+0000 7f845cbff700 1 -- 172.21.15.97:0/4015861823 shutdown_connections 2023-07-06T03:45:17.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.732+0000 7f845cbff700 1 -- 172.21.15.97:0/4015861823 wait complete. 2023-07-06T03:45:17.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.734+0000 7f845cbff700 1 Processor -- start 2023-07-06T03:45:17.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.734+0000 7f845cbff700 1 -- start start 2023-07-06T03:45:17.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.735+0000 7f845cbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f845815c520 0x7f845815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f845cbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8458160180 0x7f84580d2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f845cbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8458160b30 0x7f84580cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f845cbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84580d3c10 con 0x7f8458160b30 2023-07-06T03:45:17.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f845cbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84580d3780 con 0x7f845815c520 2023-07-06T03:45:17.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f845cbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84580d2b70 con 0x7f8458160180 2023-07-06T03:45:17.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f8456d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8458160b30 0x7f84580cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f8456d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8458160b30 0x7f84580cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46952/0 (socket says 172.21.15.97:46952) 2023-07-06T03:45:17.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f8456d9d700 1 -- 172.21.15.97:0/2988937606 learned_addr learned my addr 172.21.15.97:0/2988937606 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:17.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f845659c700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f845815c520 0x7f845815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f8455d9b700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8458160180 0x7f84580d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f8456d9d700 1 -- 172.21.15.97:0/2988937606 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8458160180 msgr2=0x7f84580d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:17.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f8456d9d700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8458160180 0x7f84580d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f8456d9d700 1 -- 172.21.15.97:0/2988937606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f845815c520 msgr2=0x7f845815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:17.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f8456d9d700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f845815c520 0x7f845815b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f8456d9d700 1 -- 172.21.15.97:0/2988937606 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844c009550 con 0x7f8458160b30 2023-07-06T03:45:17.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f845659c700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f845815c520 0x7f845815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:17.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.736+0000 7f8456d9d700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8458160b30 0x7f84580cbfe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f845000cef0 tx=0x7f845000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:17.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.737+0000 7f8446ffd700 1 -- 172.21.15.97:0/2988937606 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845000ae50 con 0x7f8458160b30 2023-07-06T03:45:17.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.737+0000 7f8446ffd700 1 -- 172.21.15.97:0/2988937606 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8450015440 con 0x7f8458160b30 2023-07-06T03:45:17.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.737+0000 7f845cbff700 1 -- 172.21.15.97:0/2988937606 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84580d2e50 con 0x7f8458160b30 2023-07-06T03:45:17.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.737+0000 7f8446ffd700 1 -- 172.21.15.97:0/2988937606 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500157f0 con 0x7f8458160b30 2023-07-06T03:45:17.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.737+0000 7f845cbff700 1 -- 172.21.15.97:0/2988937606 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84580d3410 con 0x7f8458160b30 2023-07-06T03:45:17.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.738+0000 7f845cbff700 1 -- 172.21.15.97:0/2988937606 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84580d38f0 con 0x7f8458160b30 2023-07-06T03:45:17.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.739+0000 7f8446ffd700 1 -- 172.21.15.97:0/2988937606 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f845000d430 con 0x7f8458160b30 2023-07-06T03:45:17.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.740+0000 7f8446ffd700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f843806ace0 0x7f843806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:17.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.740+0000 7f8446ffd700 1 -- 172.21.15.97:0/2988937606 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8450089a60 con 0x7f8458160b30 2023-07-06T03:45:17.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.740+0000 7f845659c700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f843806ace0 0x7f843806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:17.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.740+0000 7f845659c700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f843806ace0 0x7f843806d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f844800aa20 tx=0x7f8448008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:17.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.744+0000 7f8446ffd700 1 -- 172.21.15.97:0/2988937606 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8450054b00 con 0x7f8458160b30 2023-07-06T03:45:17.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.935+0000 7f845cbff700 1 -- 172.21.15.97:0/2988937606 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84580b6ee0 con 0x7f843806ace0 2023-07-06T03:45:17.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.943+0000 7f8446ffd700 1 -- 172.21.15.97:0/2988937606 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286825 (secure 0 0 0) 0x7f844800a6f0 con 0x7f843806ace0 2023-07-06T03:45:17.948 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:17.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.947+0000 7f845cbff700 1 -- 172.21.15.97:0/2988937606 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f843806ace0 msgr2=0x7f843806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:17.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.947+0000 7f845cbff700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f843806ace0 0x7f843806d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f844800aa20 tx=0x7f8448008040 comp rx=0 tx=0).stop 2023-07-06T03:45:17.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.947+0000 7f845cbff700 1 -- 172.21.15.97:0/2988937606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8458160b30 msgr2=0x7f84580cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:17.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.947+0000 7f845cbff700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8458160b30 0x7f84580cbfe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f845000cef0 tx=0x7f845000a340 comp rx=0 tx=0).stop 2023-07-06T03:45:17.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.947+0000 7f845cbff700 1 -- 172.21.15.97:0/2988937606 shutdown_connections 2023-07-06T03:45:17.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.947+0000 7f845cbff700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f843806ace0 0x7f843806d190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.947+0000 7f845cbff700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8458160b30 0x7f84580cbfe0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.947+0000 7f845cbff700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f845815c520 0x7f845815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.947+0000 7f845cbff700 1 --2- 172.21.15.97:0/2988937606 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8458160180 0x7f84580d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:17.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.947+0000 7f845cbff700 1 -- 172.21.15.97:0/2988937606 >> 172.21.15.97:0/2988937606 conn(0x7f8458071790 msgr2=0x7f84580b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:17.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.948+0000 7f845cbff700 1 -- 172.21.15.97:0/2988937606 shutdown_connections 2023-07-06T03:45:17.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:17.948+0000 7f845cbff700 1 -- 172.21.15.97:0/2988937606 wait complete. 2023-07-06T03:45:17.952 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:45:17.967 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-07-06T03:45:16.883203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218724,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.253071"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681463+0000","last_change":"2023-07-06T03:45:11.693177+0000","last_active":"2023-07-06T03:45:12.681463+0000","last_peered":"2023-07-06T03:45:12.681463+0000","last_clean":"2023-07-06T03:45:12.681463+0000","last_became_active":"2023-07-06T03:45:11.692851+0000","last_became_peered":"2023-07-06T03:45:11.692851+0000","last_unstale":"2023-07-06T03:45:12.681463+0000","last_undegraded":"2023-07-06T03:45:12.681463+0000","last_fullsized":"2023-07-06T03:45:12.681463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678856+0000","last_change":"2023-07-06T03:45:11.674359+0000","last_active":"2023-07-06T03:45:12.678856+0000","last_peered":"2023-07-06T03:45:12.678856+0000","last_clean":"2023-07-06T03:45:12.678856+0000","last_became_active":"2023-07-06T03:45:11.674035+0000","last_became_peered":"2023-07-06T03:45:11.674035+0000","last_unstale":"2023-07-06T03:45:12.678856+0000","last_undegraded":"2023-07-06T03:45:12.678856+0000","last_fullsized":"2023-07-06T03:45:12.678856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827906+0000","last_change":"2023-07-06T03:45:11.691806+0000","last_active":"2023-07-06T03:45:13.827906+0000","last_peered":"2023-07-06T03:45:13.827906+0000","last_clean":"2023-07-06T03:45:13.827906+0000","last_became_active":"2023-07-06T03:45:11.691532+0000","last_became_peered":"2023-07-06T03:45:11.691532+0000","last_unstale":"2023-07-06T03:45:13.827906+0000","last_undegraded":"2023-07-06T03:45:13.827906+0000","last_fullsized":"2023-07-06T03:45:13.827906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827578+0000","last_change":"2023-07-06T03:45:11.675300+0000","last_active":"2023-07-06T03:45:13.827578+0000","last_peered":"2023-07-06T03:45:13.827578+0000","last_clean":"2023-07-06T03:45:13.827578+0000","last_became_active":"2023-07-06T03:45:11.675018+0000","last_became_peered":"2023-07-06T03:45:11.675018+0000","last_unstale":"2023-07-06T03:45:13.827578+0000","last_undegraded":"2023-07-06T03:45:13.827578+0000","last_fullsized":"2023-07-06T03:45:13.827578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205854+0000","last_change":"2023-07-06T03:45:11.666099+0000","last_active":"2023-07-06T03:45:14.205854+0000","last_peered":"2023-07-06T03:45:14.205854+0000","last_clean":"2023-07-06T03:45:14.205854+0000","last_became_active":"2023-07-06T03:45:11.665759+0000","last_became_peered":"2023-07-06T03:45:11.665759+0000","last_unstale":"2023-07-06T03:45:14.205854+0000","last_undegraded":"2023-07-06T03:45:14.205854+0000","last_fullsized":"2023-07-06T03:45:14.205854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:52.666500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830215+0000","last_change":"2023-07-06T03:45:11.662251+0000","last_active":"2023-07-06T03:45:13.830215+0000","last_peered":"2023-07-06T03:45:13.830215+0000","last_clean":"2023-07-06T03:45:13.830215+0000","last_became_active":"2023-07-06T03:45:11.661916+0000","last_became_peered":"2023-07-06T03:45:11.661916+0000","last_unstale":"2023-07-06T03:45:13.830215+0000","last_undegraded":"2023-07-06T03:45:13.830215+0000","last_fullsized":"2023-07-06T03:45:13.830215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:11.287937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829687+0000","last_change":"2023-07-06T03:45:11.676465+0000","last_active":"2023-07-06T03:45:13.829687+0000","last_peered":"2023-07-06T03:45:13.829687+0000","last_clean":"2023-07-06T03:45:13.829687+0000","last_became_active":"2023-07-06T03:45:11.676180+0000","last_became_peered":"2023-07-06T03:45:11.676180+0000","last_unstale":"2023-07-06T03:45:13.829687+0000","last_undegraded":"2023-07-06T03:45:13.829687+0000","last_fullsized":"2023-07-06T03:45:13.829687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:38:24.542890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818440+0000","last_change":"2023-07-06T03:45:11.676387+0000","last_active":"2023-07-06T03:45:12.818440+0000","last_peered":"2023-07-06T03:45:12.818440+0000","last_clean":"2023-07-06T03:45:12.818440+0000","last_became_active":"2023-07-06T03:45:11.676129+0000","last_became_peered":"2023-07-06T03:45:11.676129+0000","last_unstale":"2023-07-06T03:45:12.818440+0000","last_undegraded":"2023-07-06T03:45:12.818440+0000","last_fullsized":"2023-07-06T03:45:12.818440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206076+0000","last_change":"2023-07-06T03:45:11.690977+0000","last_active":"2023-07-06T03:45:14.206076+0000","last_peered":"2023-07-06T03:45:14.206076+0000","last_clean":"2023-07-06T03:45:14.206076+0000","last_became_active":"2023-07-06T03:45:11.690423+0000","last_became_peered":"2023-07-06T03:45:11.690423+0000","last_unstale":"2023-07-06T03:45:14.206076+0000","last_undegraded":"2023-07-06T03:45:14.206076+0000","last_fullsized":"2023-07-06T03:45:14.206076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:14:59.597908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818347+0000","last_change":"2023-07-06T03:45:11.680924+0000","last_active":"2023-07-06T03:45:12.818347+0000","last_peered":"2023-07-06T03:45:12.818347+0000","last_clean":"2023-07-06T03:45:12.818347+0000","last_became_active":"2023-07-06T03:45:11.680607+0000","last_became_peered":"2023-07-06T03:45:11.680607+0000","last_unstale":"2023-07-06T03:45:12.818347+0000","last_undegraded":"2023-07-06T03:45:12.818347+0000","last_fullsized":"2023-07-06T03:45:12.818347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:13.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818149+0000","last_change":"2023-07-06T03:45:11.677946+0000","last_active":"2023-07-06T03:45:12.818149+0000","last_peered":"2023-07-06T03:45:12.818149+0000","last_clean":"2023-07-06T03:45:12.818149+0000","last_became_active":"2023-07-06T03:45:11.677684+0000","last_became_peered":"2023-07-06T03:45:11.677684+0000","last_unstale":"2023-07-06T03:45:12.818149+0000","last_undegraded":"2023-07-06T03:45:12.818149+0000","last_fullsized":"2023-07-06T03:45:12.818149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:27.384754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680941+0000","last_change":"2023-07-06T03:45:11.681178+0000","last_active":"2023-07-06T03:45:12.680941+0000","last_peered":"2023-07-06T03:45:12.680941+0000","last_clean":"2023-07-06T03:45:12.680941+0000","last_became_active":"2023-07-06T03:45:11.680841+0000","last_became_peered":"2023-07-06T03:45:11.680841+0000","last_unstale":"2023-07-06T03:45:12.680941+0000","last_undegraded":"2023-07-06T03:45:12.680941+0000","last_fullsized":"2023-07-06T03:45:12.680941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:58.087282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652673+0000","last_change":"2023-07-06T03:45:11.687461+0000","last_active":"2023-07-06T03:45:13.652673+0000","last_peered":"2023-07-06T03:45:13.652673+0000","last_clean":"2023-07-06T03:45:13.652673+0000","last_became_active":"2023-07-06T03:45:11.687182+0000","last_became_peered":"2023-07-06T03:45:11.687182+0000","last_unstale":"2023-07-06T03:45:13.652673+0000","last_undegraded":"2023-07-06T03:45:13.652673+0000","last_fullsized":"2023-07-06T03:45:13.652673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817899+0000","last_change":"2023-07-06T03:45:11.691496+0000","last_active":"2023-07-06T03:45:12.817899+0000","last_peered":"2023-07-06T03:45:12.817899+0000","last_clean":"2023-07-06T03:45:12.817899+0000","last_became_active":"2023-07-06T03:45:11.691206+0000","last_became_peered":"2023-07-06T03:45:11.691206+0000","last_unstale":"2023-07-06T03:45:12.817899+0000","last_undegraded":"2023-07-06T03:45:12.817899+0000","last_fullsized":"2023-07-06T03:45:12.817899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827487+0000","last_change":"2023-07-06T03:45:11.685001+0000","last_active":"2023-07-06T03:45:13.827487+0000","last_peered":"2023-07-06T03:45:13.827487+0000","last_clean":"2023-07-06T03:45:13.827487+0000","last_became_active":"2023-07-06T03:45:11.684695+0000","last_became_peered":"2023-07-06T03:45:11.684695+0000","last_unstale":"2023-07-06T03:45:13.827487+0000","last_undegraded":"2023-07-06T03:45:13.827487+0000","last_fullsized":"2023-07-06T03:45:13.827487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829742+0000","last_change":"2023-07-06T03:45:11.662096+0000","last_active":"2023-07-06T03:45:13.829742+0000","last_peered":"2023-07-06T03:45:13.829742+0000","last_clean":"2023-07-06T03:45:13.829742+0000","last_became_active":"2023-07-06T03:45:11.661834+0000","last_became_peered":"2023-07-06T03:45:11.661834+0000","last_unstale":"2023-07-06T03:45:13.829742+0000","last_undegraded":"2023-07-06T03:45:13.829742+0000","last_fullsized":"2023-07-06T03:45:13.829742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:36:49.534209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678504+0000","last_change":"2023-07-06T03:45:11.685162+0000","last_active":"2023-07-06T03:45:12.678504+0000","last_peered":"2023-07-06T03:45:12.678504+0000","last_clean":"2023-07-06T03:45:12.678504+0000","last_became_active":"2023-07-06T03:45:11.684977+0000","last_became_peered":"2023-07-06T03:45:11.684977+0000","last_unstale":"2023-07-06T03:45:12.678504+0000","last_undegraded":"2023-07-06T03:45:12.678504+0000","last_fullsized":"2023-07-06T03:45:12.678504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827827+0000","last_change":"2023-07-06T03:45:11.675611+0000","last_active":"2023-07-06T03:45:13.827827+0000","last_peered":"2023-07-06T03:45:13.827827+0000","last_clean":"2023-07-06T03:45:13.827827+0000","last_became_active":"2023-07-06T03:45:11.675348+0000","last_became_peered":"2023-07-06T03:45:11.675348+0000","last_unstale":"2023-07-06T03:45:13.827827+0000","last_undegraded":"2023-07-06T03:45:13.827827+0000","last_fullsized":"2023-07-06T03:45:13.827827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:00.096345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680922+0000","last_change":"2023-07-06T03:45:11.690182+0000","last_active":"2023-07-06T03:45:12.680922+0000","last_peered":"2023-07-06T03:45:12.680922+0000","last_clean":"2023-07-06T03:45:12.680922+0000","last_became_active":"2023-07-06T03:45:11.689744+0000","last_became_peered":"2023-07-06T03:45:11.689744+0000","last_unstale":"2023-07-06T03:45:12.680922+0000","last_undegraded":"2023-07-06T03:45:12.680922+0000","last_fullsized":"2023-07-06T03:45:12.680922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:32.898881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818137+0000","last_change":"2023-07-06T03:45:11.672749+0000","last_active":"2023-07-06T03:45:12.818137+0000","last_peered":"2023-07-06T03:45:12.818137+0000","last_clean":"2023-07-06T03:45:12.818137+0000","last_became_active":"2023-07-06T03:45:11.672401+0000","last_became_peered":"2023-07-06T03:45:11.672401+0000","last_unstale":"2023-07-06T03:45:12.818137+0000","last_undegraded":"2023-07-06T03:45:12.818137+0000","last_fullsized":"2023-07-06T03:45:12.818137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:15.627811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818032+0000","last_change":"2023-07-06T03:45:11.692060+0000","last_active":"2023-07-06T03:45:12.818032+0000","last_peered":"2023-07-06T03:45:12.818032+0000","last_clean":"2023-07-06T03:45:12.818032+0000","last_became_active":"2023-07-06T03:45:11.691699+0000","last_became_peered":"2023-07-06T03:45:11.691699+0000","last_unstale":"2023-07-06T03:45:12.818032+0000","last_undegraded":"2023-07-06T03:45:12.818032+0000","last_fullsized":"2023-07-06T03:45:12.818032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:32.898881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678195+0000","last_change":"2023-07-06T03:45:11.672926+0000","last_active":"2023-07-06T03:45:12.678195+0000","last_peered":"2023-07-06T03:45:12.678195+0000","last_clean":"2023-07-06T03:45:12.678195+0000","last_became_active":"2023-07-06T03:45:11.672627+0000","last_became_peered":"2023-07-06T03:45:11.672627+0000","last_unstale":"2023-07-06T03:45:12.678195+0000","last_undegraded":"2023-07-06T03:45:12.678195+0000","last_fullsized":"2023-07-06T03:45:12.678195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205814+0000","last_change":"2023-07-06T03:45:11.691373+0000","last_active":"2023-07-06T03:45:14.205814+0000","last_peered":"2023-07-06T03:45:14.205814+0000","last_clean":"2023-07-06T03:45:14.205814+0000","last_became_active":"2023-07-06T03:45:11.691046+0000","last_became_peered":"2023-07-06T03:45:11.691046+0000","last_unstale":"2023-07-06T03:45:14.205814+0000","last_undegraded":"2023-07-06T03:45:14.205814+0000","last_fullsized":"2023-07-06T03:45:14.205814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818372+0000","last_change":"2023-07-06T03:45:11.691643+0000","last_active":"2023-07-06T03:45:12.818372+0000","last_peered":"2023-07-06T03:45:12.818372+0000","last_clean":"2023-07-06T03:45:12.818372+0000","last_became_active":"2023-07-06T03:45:11.691382+0000","last_became_peered":"2023-07-06T03:45:11.691382+0000","last_unstale":"2023-07-06T03:45:12.818372+0000","last_undegraded":"2023-07-06T03:45:12.818372+0000","last_fullsized":"2023-07-06T03:45:12.818372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206243+0000","last_change":"2023-07-06T03:45:11.670205+0000","last_active":"2023-07-06T03:45:14.206243+0000","last_peered":"2023-07-06T03:45:14.206243+0000","last_clean":"2023-07-06T03:45:14.206243+0000","last_became_active":"2023-07-06T03:45:11.669935+0000","last_became_peered":"2023-07-06T03:45:11.669935+0000","last_unstale":"2023-07-06T03:45:14.206243+0000","last_undegraded":"2023-07-06T03:45:14.206243+0000","last_fullsized":"2023-07-06T03:45:14.206243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:11:07.089396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678592+0000","last_change":"2023-07-06T03:45:11.674601+0000","last_active":"2023-07-06T03:45:12.678592+0000","last_peered":"2023-07-06T03:45:12.678592+0000","last_clean":"2023-07-06T03:45:12.678592+0000","last_became_active":"2023-07-06T03:45:11.673533+0000","last_became_peered":"2023-07-06T03:45:11.673533+0000","last_unstale":"2023-07-06T03:45:12.678592+0000","last_undegraded":"2023-07-06T03:45:12.678592+0000","last_fullsized":"2023-07-06T03:45:12.678592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830285+0000","last_change":"2023-07-06T03:45:11.678777+0000","last_active":"2023-07-06T03:45:13.830285+0000","last_peered":"2023-07-06T03:45:13.830285+0000","last_clean":"2023-07-06T03:45:13.830285+0000","last_became_active":"2023-07-06T03:45:11.678437+0000","last_became_peered":"2023-07-06T03:45:11.678437+0000","last_unstale":"2023-07-06T03:45:13.830285+0000","last_undegraded":"2023-07-06T03:45:13.830285+0000","last_fullsized":"2023-07-06T03:45:13.830285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:51:53.330078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818105+0000","last_change":"2023-07-06T03:45:10.656464+0000","last_active":"2023-07-06T03:45:12.818105+0000","last_peered":"2023-07-06T03:45:12.818105+0000","last_clean":"2023-07-06T03:45:12.818105+0000","last_became_active":"2023-07-06T03:45:10.656116+0000","last_became_peered":"2023-07-06T03:45:10.656116+0000","last_unstale":"2023-07-06T03:45:12.818105+0000","last_undegraded":"2023-07-06T03:45:12.818105+0000","last_fullsized":"2023-07-06T03:45:12.818105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:57.080176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678331+0000","last_change":"2023-07-06T03:45:11.687243+0000","last_active":"2023-07-06T03:45:12.678331+0000","last_peered":"2023-07-06T03:45:12.678331+0000","last_clean":"2023-07-06T03:45:12.678331+0000","last_became_active":"2023-07-06T03:45:11.686947+0000","last_became_peered":"2023-07-06T03:45:11.686947+0000","last_unstale":"2023-07-06T03:45:12.678331+0000","last_undegraded":"2023-07-06T03:45:12.678331+0000","last_fullsized":"2023-07-06T03:45:12.678331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205529+0000","last_change":"2023-07-06T03:45:10.657736+0000","last_active":"2023-07-06T03:45:14.205529+0000","last_peered":"2023-07-06T03:45:14.205529+0000","last_clean":"2023-07-06T03:45:14.205529+0000","last_became_active":"2023-07-06T03:45:10.657450+0000","last_became_peered":"2023-07-06T03:45:10.657450+0000","last_unstale":"2023-07-06T03:45:14.205529+0000","last_undegraded":"2023-07-06T03:45:14.205529+0000","last_fullsized":"2023-07-06T03:45:14.205529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:14.620705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206125+0000","last_change":"2023-07-06T03:45:11.672564+0000","last_active":"2023-07-06T03:45:14.206125+0000","last_peered":"2023-07-06T03:45:14.206125+0000","last_clean":"2023-07-06T03:45:14.206125+0000","last_became_active":"2023-07-06T03:45:11.672150+0000","last_became_peered":"2023-07-06T03:45:11.672150+0000","last_unstale":"2023-07-06T03:45:14.206125+0000","last_undegraded":"2023-07-06T03:45:14.206125+0000","last_fullsized":"2023-07-06T03:45:14.206125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:59:01.536411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680441+0000","last_change":"2023-07-06T03:45:10.653666+0000","last_active":"2023-07-06T03:45:12.680441+0000","last_peered":"2023-07-06T03:45:12.680441+0000","last_clean":"2023-07-06T03:45:12.680441+0000","last_became_active":"2023-07-06T03:45:10.653357+0000","last_became_peered":"2023-07-06T03:45:10.653357+0000","last_unstale":"2023-07-06T03:45:12.680441+0000","last_undegraded":"2023-07-06T03:45:12.680441+0000","last_fullsized":"2023-07-06T03:45:12.680441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:25.838923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830215+0000","last_change":"2023-07-06T03:45:11.690985+0000","last_active":"2023-07-06T03:45:13.830215+0000","last_peered":"2023-07-06T03:45:13.830215+0000","last_clean":"2023-07-06T03:45:13.830215+0000","last_became_active":"2023-07-06T03:45:11.690692+0000","last_became_peered":"2023-07-06T03:45:11.690692+0000","last_unstale":"2023-07-06T03:45:13.830215+0000","last_undegraded":"2023-07-06T03:45:13.830215+0000","last_fullsized":"2023-07-06T03:45:13.830215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:27:20.751301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681704+0000","last_change":"2023-07-06T03:45:10.649734+0000","last_active":"2023-07-06T03:45:12.681704+0000","last_peered":"2023-07-06T03:45:12.681704+0000","last_clean":"2023-07-06T03:45:12.681704+0000","last_became_active":"2023-07-06T03:45:10.649401+0000","last_became_peered":"2023-07-06T03:45:10.649401+0000","last_unstale":"2023-07-06T03:45:12.681704+0000","last_undegraded":"2023-07-06T03:45:12.681704+0000","last_fullsized":"2023-07-06T03:45:12.681704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:24:42.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678483+0000","last_change":"2023-07-06T03:45:11.667398+0000","last_active":"2023-07-06T03:45:12.678483+0000","last_peered":"2023-07-06T03:45:12.678483+0000","last_clean":"2023-07-06T03:45:12.678483+0000","last_became_active":"2023-07-06T03:45:11.667124+0000","last_became_peered":"2023-07-06T03:45:11.667124+0000","last_unstale":"2023-07-06T03:45:12.678483+0000","last_undegraded":"2023-07-06T03:45:12.678483+0000","last_fullsized":"2023-07-06T03:45:12.678483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:00.096345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.679013+0000","last_change":"2023-07-06T03:45:10.648660+0000","last_active":"2023-07-06T03:45:12.679013+0000","last_peered":"2023-07-06T03:45:12.679013+0000","last_clean":"2023-07-06T03:45:12.679013+0000","last_became_active":"2023-07-06T03:45:10.648254+0000","last_became_peered":"2023-07-06T03:45:10.648254+0000","last_unstale":"2023-07-06T03:45:12.679013+0000","last_undegraded":"2023-07-06T03:45:12.679013+0000","last_fullsized":"2023-07-06T03:45:12.679013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:25.838923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205490+0000","last_change":"2023-07-06T03:45:11.682171+0000","last_active":"2023-07-06T03:45:14.205490+0000","last_peered":"2023-07-06T03:45:14.205490+0000","last_clean":"2023-07-06T03:45:14.205490+0000","last_became_active":"2023-07-06T03:45:11.681861+0000","last_became_peered":"2023-07-06T03:45:11.681861+0000","last_unstale":"2023-07-06T03:45:14.205490+0000","last_undegraded":"2023-07-06T03:45:14.205490+0000","last_fullsized":"2023-07-06T03:45:14.205490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827972+0000","last_change":"2023-07-06T03:45:10.656590+0000","last_active":"2023-07-06T03:45:13.827972+0000","last_peered":"2023-07-06T03:45:13.827972+0000","last_clean":"2023-07-06T03:45:13.827972+0000","last_became_active":"2023-07-06T03:45:10.656328+0000","last_became_peered":"2023-07-06T03:45:10.656328+0000","last_unstale":"2023-07-06T03:45:13.827972+0000","last_undegraded":"2023-07-06T03:45:13.827972+0000","last_fullsized":"2023-07-06T03:45:13.827972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205720+0000","last_change":"2023-07-06T03:45:11.671697+0000","last_active":"2023-07-06T03:45:14.205720+0000","last_peered":"2023-07-06T03:45:14.205720+0000","last_clean":"2023-07-06T03:45:14.205720+0000","last_became_active":"2023-07-06T03:45:11.671343+0000","last_became_peered":"2023-07-06T03:45:11.671343+0000","last_unstale":"2023-07-06T03:45:14.205720+0000","last_undegraded":"2023-07-06T03:45:14.205720+0000","last_fullsized":"2023-07-06T03:45:14.205720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:13.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829989+0000","last_change":"2023-07-06T03:45:10.644219+0000","last_active":"2023-07-06T03:45:13.829989+0000","last_peered":"2023-07-06T03:45:13.829989+0000","last_clean":"2023-07-06T03:45:13.829989+0000","last_became_active":"2023-07-06T03:45:10.643854+0000","last_became_peered":"2023-07-06T03:45:10.643854+0000","last_unstale":"2023-07-06T03:45:13.829989+0000","last_undegraded":"2023-07-06T03:45:13.829989+0000","last_fullsized":"2023-07-06T03:45:13.829989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:50:44.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678700+0000","last_change":"2023-07-06T03:45:11.665747+0000","last_active":"2023-07-06T03:45:12.678700+0000","last_peered":"2023-07-06T03:45:12.678700+0000","last_clean":"2023-07-06T03:45:12.678700+0000","last_became_active":"2023-07-06T03:45:11.665443+0000","last_became_peered":"2023-07-06T03:45:11.665443+0000","last_unstale":"2023-07-06T03:45:12.678700+0000","last_undegraded":"2023-07-06T03:45:12.678700+0000","last_fullsized":"2023-07-06T03:45:12.678700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829814+0000","last_change":"2023-07-06T03:45:10.653534+0000","last_active":"2023-07-06T03:45:13.829814+0000","last_peered":"2023-07-06T03:45:13.829814+0000","last_clean":"2023-07-06T03:45:13.829814+0000","last_became_active":"2023-07-06T03:45:10.653232+0000","last_became_peered":"2023-07-06T03:45:10.653232+0000","last_unstale":"2023-07-06T03:45:13.829814+0000","last_undegraded":"2023-07-06T03:45:13.829814+0000","last_fullsized":"2023-07-06T03:45:13.829814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:16.021365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652974+0000","last_change":"2023-07-06T03:45:11.683611+0000","last_active":"2023-07-06T03:45:13.652974+0000","last_peered":"2023-07-06T03:45:13.652974+0000","last_clean":"2023-07-06T03:45:13.652974+0000","last_became_active":"2023-07-06T03:45:11.683343+0000","last_became_peered":"2023-07-06T03:45:11.683343+0000","last_unstale":"2023-07-06T03:45:13.652974+0000","last_undegraded":"2023-07-06T03:45:13.652974+0000","last_fullsized":"2023-07-06T03:45:13.652974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652756+0000","last_change":"2023-07-06T03:45:10.655738+0000","last_active":"2023-07-06T03:45:13.652756+0000","last_peered":"2023-07-06T03:45:13.652756+0000","last_clean":"2023-07-06T03:45:13.652756+0000","last_became_active":"2023-07-06T03:45:10.655414+0000","last_became_peered":"2023-07-06T03:45:10.655414+0000","last_unstale":"2023-07-06T03:45:13.652756+0000","last_undegraded":"2023-07-06T03:45:13.652756+0000","last_fullsized":"2023-07-06T03:45:13.652756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:38.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830471+0000","last_change":"2023-07-06T03:45:11.678704+0000","last_active":"2023-07-06T03:45:13.830471+0000","last_peered":"2023-07-06T03:45:13.830471+0000","last_clean":"2023-07-06T03:45:13.830471+0000","last_became_active":"2023-07-06T03:45:11.678393+0000","last_became_peered":"2023-07-06T03:45:11.678393+0000","last_unstale":"2023-07-06T03:45:13.830471+0000","last_undegraded":"2023-07-06T03:45:13.830471+0000","last_fullsized":"2023-07-06T03:45:13.830471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:42.303647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830064+0000","last_change":"2023-07-06T03:45:10.644075+0000","last_active":"2023-07-06T03:45:13.830064+0000","last_peered":"2023-07-06T03:45:13.830064+0000","last_clean":"2023-07-06T03:45:13.830064+0000","last_became_active":"2023-07-06T03:45:10.643785+0000","last_became_peered":"2023-07-06T03:45:10.643785+0000","last_unstale":"2023-07-06T03:45:13.830064+0000","last_undegraded":"2023-07-06T03:45:13.830064+0000","last_fullsized":"2023-07-06T03:45:13.830064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:01:07.035467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681216+0000","last_change":"2023-07-06T03:45:11.687766+0000","last_active":"2023-07-06T03:45:12.681216+0000","last_peered":"2023-07-06T03:45:12.681216+0000","last_clean":"2023-07-06T03:45:12.681216+0000","last_became_active":"2023-07-06T03:45:11.687528+0000","last_became_peered":"2023-07-06T03:45:11.687528+0000","last_unstale":"2023-07-06T03:45:12.681216+0000","last_undegraded":"2023-07-06T03:45:12.681216+0000","last_fullsized":"2023-07-06T03:45:12.681216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817929+0000","last_change":"2023-07-06T03:45:10.652546+0000","last_active":"2023-07-06T03:45:12.817929+0000","last_peered":"2023-07-06T03:45:12.817929+0000","last_clean":"2023-07-06T03:45:12.817929+0000","last_became_active":"2023-07-06T03:45:10.652314+0000","last_became_peered":"2023-07-06T03:45:10.652314+0000","last_unstale":"2023-07-06T03:45:12.817929+0000","last_undegraded":"2023-07-06T03:45:12.817929+0000","last_fullsized":"2023-07-06T03:45:12.817929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681543+0000","last_change":"2023-07-06T03:45:11.695000+0000","last_active":"2023-07-06T03:45:12.681543+0000","last_peered":"2023-07-06T03:45:12.681543+0000","last_clean":"2023-07-06T03:45:12.681543+0000","last_became_active":"2023-07-06T03:45:11.694569+0000","last_became_peered":"2023-07-06T03:45:11.694569+0000","last_unstale":"2023-07-06T03:45:12.681543+0000","last_undegraded":"2023-07-06T03:45:12.681543+0000","last_fullsized":"2023-07-06T03:45:12.681543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:52.666500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.828097+0000","last_change":"2023-07-06T03:45:10.654280+0000","last_active":"2023-07-06T03:45:13.828097+0000","last_peered":"2023-07-06T03:45:13.828097+0000","last_clean":"2023-07-06T03:45:13.828097+0000","last_became_active":"2023-07-06T03:45:10.654039+0000","last_became_peered":"2023-07-06T03:45:10.654039+0000","last_unstale":"2023-07-06T03:45:13.828097+0000","last_undegraded":"2023-07-06T03:45:13.828097+0000","last_fullsized":"2023-07-06T03:45:13.828097+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:07.054998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829778+0000","last_change":"2023-07-06T03:45:11.667282+0000","last_active":"2023-07-06T03:45:13.829778+0000","last_peered":"2023-07-06T03:45:13.829778+0000","last_clean":"2023-07-06T03:45:13.829778+0000","last_became_active":"2023-07-06T03:45:11.667027+0000","last_became_peered":"2023-07-06T03:45:11.667027+0000","last_unstale":"2023-07-06T03:45:13.829778+0000","last_undegraded":"2023-07-06T03:45:13.829778+0000","last_fullsized":"2023-07-06T03:45:13.829778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:05:40.764560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678971+0000","last_change":"2023-07-06T03:45:10.647142+0000","last_active":"2023-07-06T03:45:12.678971+0000","last_peered":"2023-07-06T03:45:12.678971+0000","last_clean":"2023-07-06T03:45:12.678971+0000","last_became_active":"2023-07-06T03:45:10.646882+0000","last_became_peered":"2023-07-06T03:45:10.646882+0000","last_unstale":"2023-07-06T03:45:12.678971+0000","last_undegraded":"2023-07-06T03:45:12.678971+0000","last_fullsized":"2023-07-06T03:45:12.678971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:59.089238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817907+0000","last_change":"2023-07-06T03:45:11.691436+0000","last_active":"2023-07-06T03:45:12.817907+0000","last_peered":"2023-07-06T03:45:12.817907+0000","last_clean":"2023-07-06T03:45:12.817907+0000","last_became_active":"2023-07-06T03:45:11.691139+0000","last_became_peered":"2023-07-06T03:45:11.691139+0000","last_unstale":"2023-07-06T03:45:12.817907+0000","last_undegraded":"2023-07-06T03:45:12.817907+0000","last_fullsized":"2023-07-06T03:45:12.817907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829985+0000","last_change":"2023-07-06T03:45:10.647677+0000","last_active":"2023-07-06T03:45:13.829985+0000","last_peered":"2023-07-06T03:45:13.829985+0000","last_clean":"2023-07-06T03:45:13.829985+0000","last_became_active":"2023-07-06T03:45:10.647414+0000","last_became_peered":"2023-07-06T03:45:10.647414+0000","last_unstale":"2023-07-06T03:45:13.829985+0000","last_undegraded":"2023-07-06T03:45:13.829985+0000","last_fullsized":"2023-07-06T03:45:13.829985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:34:02.031995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678552+0000","last_change":"2023-07-06T03:45:11.663454+0000","last_active":"2023-07-06T03:45:12.678552+0000","last_peered":"2023-07-06T03:45:12.678552+0000","last_clean":"2023-07-06T03:45:12.678552+0000","last_became_active":"2023-07-06T03:45:11.663172+0000","last_became_peered":"2023-07-06T03:45:11.663172+0000","last_unstale":"2023-07-06T03:45:12.678552+0000","last_undegraded":"2023-07-06T03:45:12.678552+0000","last_fullsized":"2023-07-06T03:45:12.678552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827748+0000","last_change":"2023-07-06T03:45:10.651255+0000","last_active":"2023-07-06T03:45:13.827748+0000","last_peered":"2023-07-06T03:45:13.827748+0000","last_clean":"2023-07-06T03:45:13.827748+0000","last_became_active":"2023-07-06T03:45:10.651012+0000","last_became_peered":"2023-07-06T03:45:10.651012+0000","last_unstale":"2023-07-06T03:45:13.827748+0000","last_undegraded":"2023-07-06T03:45:13.827748+0000","last_fullsized":"2023-07-06T03:45:13.827748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:46.872621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680880+0000","last_change":"2023-07-06T03:45:11.688331+0000","last_active":"2023-07-06T03:45:12.680880+0000","last_peered":"2023-07-06T03:45:12.680880+0000","last_clean":"2023-07-06T03:45:12.680880+0000","last_became_active":"2023-07-06T03:45:11.686894+0000","last_became_peered":"2023-07-06T03:45:11.686894+0000","last_unstale":"2023-07-06T03:45:12.680880+0000","last_undegraded":"2023-07-06T03:45:12.680880+0000","last_fullsized":"2023-07-06T03:45:12.680880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:08.062104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678168+0000","last_change":"2023-07-06T03:45:10.653064+0000","last_active":"2023-07-06T03:45:12.678168+0000","last_peered":"2023-07-06T03:45:12.678168+0000","last_clean":"2023-07-06T03:45:12.678168+0000","last_became_active":"2023-07-06T03:45:10.652765+0000","last_became_peered":"2023-07-06T03:45:10.652765+0000","last_unstale":"2023-07-06T03:45:12.678168+0000","last_undegraded":"2023-07-06T03:45:12.678168+0000","last_fullsized":"2023-07-06T03:45:12.678168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:23.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827686+0000","last_change":"2023-07-06T03:45:11.683336+0000","last_active":"2023-07-06T03:45:13.827686+0000","last_peered":"2023-07-06T03:45:13.827686+0000","last_clean":"2023-07-06T03:45:13.827686+0000","last_became_active":"2023-07-06T03:45:11.683087+0000","last_became_peered":"2023-07-06T03:45:11.683087+0000","last_unstale":"2023-07-06T03:45:13.827686+0000","last_undegraded":"2023-07-06T03:45:13.827686+0000","last_fullsized":"2023-07-06T03:45:13.827686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830000+0000","last_change":"2023-07-06T03:45:10.650363+0000","last_active":"2023-07-06T03:45:13.830000+0000","last_peered":"2023-07-06T03:45:13.830000+0000","last_clean":"2023-07-06T03:45:13.830000+0000","last_became_active":"2023-07-06T03:45:10.650097+0000","last_became_peered":"2023-07-06T03:45:10.650097+0000","last_unstale":"2023-07-06T03:45:13.830000+0000","last_undegraded":"2023-07-06T03:45:13.830000+0000","last_fullsized":"2023-07-06T03:45:13.830000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:05.604877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830398+0000","last_change":"2023-07-06T03:45:11.668010+0000","last_active":"2023-07-06T03:45:13.830398+0000","last_peered":"2023-07-06T03:45:13.830398+0000","last_clean":"2023-07-06T03:45:13.830398+0000","last_became_active":"2023-07-06T03:45:11.667739+0000","last_became_peered":"2023-07-06T03:45:11.667739+0000","last_unstale":"2023-07-06T03:45:13.830398+0000","last_undegraded":"2023-07-06T03:45:13.830398+0000","last_fullsized":"2023-07-06T03:45:13.830398+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:06:53.685059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829747+0000","last_change":"2023-07-06T03:45:10.661165+0000","last_active":"2023-07-06T03:45:13.829747+0000","last_peered":"2023-07-06T03:45:13.829747+0000","last_clean":"2023-07-06T03:45:13.829747+0000","last_became_active":"2023-07-06T03:45:10.660843+0000","last_became_peered":"2023-07-06T03:45:10.660843+0000","last_unstale":"2023-07-06T03:45:13.829747+0000","last_undegraded":"2023-07-06T03:45:13.829747+0000","last_fullsized":"2023-07-06T03:45:13.829747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:02:20.704169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678908+0000","last_change":"2023-07-06T03:45:11.665692+0000","last_active":"2023-07-06T03:45:12.678908+0000","last_peered":"2023-07-06T03:45:12.678908+0000","last_clean":"2023-07-06T03:45:12.678908+0000","last_became_active":"2023-07-06T03:45:11.665425+0000","last_became_peered":"2023-07-06T03:45:11.665425+0000","last_unstale":"2023-07-06T03:45:12.678908+0000","last_undegraded":"2023-07-06T03:45:12.678908+0000","last_fullsized":"2023-07-06T03:45:12.678908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205473+0000","last_change":"2023-07-06T03:45:10.658201+0000","last_active":"2023-07-06T03:45:14.205473+0000","last_peered":"2023-07-06T03:45:14.205473+0000","last_clean":"2023-07-06T03:45:14.205473+0000","last_became_active":"2023-07-06T03:45:10.657688+0000","last_became_peered":"2023-07-06T03:45:10.657688+0000","last_unstale":"2023-07-06T03:45:14.205473+0000","last_undegraded":"2023-07-06T03:45:14.205473+0000","last_fullsized":"2023-07-06T03:45:14.205473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:26.377648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678497+0000","last_change":"2023-07-06T03:45:11.673905+0000","last_active":"2023-07-06T03:45:12.678497+0000","last_peered":"2023-07-06T03:45:12.678497+0000","last_clean":"2023-07-06T03:45:12.678497+0000","last_became_active":"2023-07-06T03:45:11.673609+0000","last_became_peered":"2023-07-06T03:45:11.673609+0000","last_unstale":"2023-07-06T03:45:12.678497+0000","last_undegraded":"2023-07-06T03:45:12.678497+0000","last_fullsized":"2023-07-06T03:45:12.678497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681110+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:45:12.681110+0000","last_peered":"2023-07-06T03:45:12.681110+0000","last_clean":"2023-07-06T03:45:12.681110+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:45:12.681110+0000","last_undegraded":"2023-07-06T03:45:12.681110+0000","last_fullsized":"2023-07-06T03:45:12.681110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:20.874020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678806+0000","last_change":"2023-07-06T03:45:10.660307+0000","last_active":"2023-07-06T03:45:12.678806+0000","last_peered":"2023-07-06T03:45:12.678806+0000","last_clean":"2023-07-06T03:45:12.678806+0000","last_became_active":"2023-07-06T03:45:10.660032+0000","last_became_peered":"2023-07-06T03:45:10.660032+0000","last_unstale":"2023-07-06T03:45:12.678806+0000","last_undegraded":"2023-07-06T03:45:12.678806+0000","last_fullsized":"2023-07-06T03:45:12.678806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652629+0000","last_change":"2023-07-06T03:45:11.679839+0000","last_active":"2023-07-06T03:45:13.652629+0000","last_peered":"2023-07-06T03:45:13.652629+0000","last_clean":"2023-07-06T03:45:13.652629+0000","last_became_active":"2023-07-06T03:45:11.679509+0000","last_became_peered":"2023-07-06T03:45:11.679509+0000","last_unstale":"2023-07-06T03:45:13.652629+0000","last_undegraded":"2023-07-06T03:45:13.652629+0000","last_fullsized":"2023-07-06T03:45:13.652629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830179+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:45:13.830179+0000","last_peered":"2023-07-06T03:45:13.830179+0000","last_clean":"2023-07-06T03:45:13.830179+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:45:13.830179+0000","last_undegraded":"2023-07-06T03:45:13.830179+0000","last_fullsized":"2023-07-06T03:45:13.830179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:03:06.569126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827774+0000","last_change":"2023-07-06T03:45:10.649547+0000","last_active":"2023-07-06T03:45:13.827774+0000","last_peered":"2023-07-06T03:45:13.827774+0000","last_clean":"2023-07-06T03:45:13.827774+0000","last_became_active":"2023-07-06T03:45:10.649231+0000","last_became_peered":"2023-07-06T03:45:10.649231+0000","last_unstale":"2023-07-06T03:45:13.827774+0000","last_undegraded":"2023-07-06T03:45:13.827774+0000","last_fullsized":"2023-07-06T03:45:13.827774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:13.496490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653155+0000","last_change":"2023-07-06T03:45:11.678151+0000","last_active":"2023-07-06T03:45:13.653155+0000","last_peered":"2023-07-06T03:45:13.653155+0000","last_clean":"2023-07-06T03:45:13.653155+0000","last_became_active":"2023-07-06T03:45:11.677841+0000","last_became_peered":"2023-07-06T03:45:11.677841+0000","last_unstale":"2023-07-06T03:45:13.653155+0000","last_undegraded":"2023-07-06T03:45:13.653155+0000","last_fullsized":"2023-07-06T03:45:13.653155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678308+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:45:12.678308+0000","last_peered":"2023-07-06T03:45:12.678308+0000","last_clean":"2023-07-06T03:45:12.678308+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:45:12.678308+0000","last_undegraded":"2023-07-06T03:45:12.678308+0000","last_fullsized":"2023-07-06T03:45:12.678308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:11.498012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680537+0000","last_change":"2023-07-06T03:45:10.647981+0000","last_active":"2023-07-06T03:45:12.680537+0000","last_peered":"2023-07-06T03:45:12.680537+0000","last_clean":"2023-07-06T03:45:12.680537+0000","last_became_active":"2023-07-06T03:45:10.647678+0000","last_became_peered":"2023-07-06T03:45:10.647678+0000","last_unstale":"2023-07-06T03:45:12.680537+0000","last_undegraded":"2023-07-06T03:45:12.680537+0000","last_fullsized":"2023-07-06T03:45:12.680537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653181+0000","last_change":"2023-07-06T03:45:11.683452+0000","last_active":"2023-07-06T03:45:13.653181+0000","last_peered":"2023-07-06T03:45:13.653181+0000","last_clean":"2023-07-06T03:45:13.653181+0000","last_became_active":"2023-07-06T03:45:11.683187+0000","last_became_peered":"2023-07-06T03:45:11.683187+0000","last_unstale":"2023-07-06T03:45:13.653181+0000","last_undegraded":"2023-07-06T03:45:13.653181+0000","last_fullsized":"2023-07-06T03:45:13.653181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:38.118602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829567+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:45:13.829567+0000","last_peered":"2023-07-06T03:45:13.829567+0000","last_clean":"2023-07-06T03:45:13.829567+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:45:13.829567+0000","last_undegraded":"2023-07-06T03:45:13.829567+0000","last_fullsized":"2023-07-06T03:45:13.829567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:49:25.728537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817899+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:45:12.817899+0000","last_peered":"2023-07-06T03:45:12.817899+0000","last_clean":"2023-07-06T03:45:12.817899+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:45:12.817899+0000","last_undegraded":"2023-07-06T03:45:12.817899+0000","last_fullsized":"2023-07-06T03:45:12.817899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:39.120643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205527+0000","last_change":"2023-07-06T03:45:10.657235+0000","last_active":"2023-07-06T03:45:14.205527+0000","last_peered":"2023-07-06T03:45:14.205527+0000","last_clean":"2023-07-06T03:45:14.205527+0000","last_became_active":"2023-07-06T03:45:10.656945+0000","last_became_peered":"2023-07-06T03:45:10.656945+0000","last_unstale":"2023-07-06T03:45:14.205527+0000","last_undegraded":"2023-07-06T03:45:14.205527+0000","last_fullsized":"2023-07-06T03:45:14.205527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:57.080176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830434+0000","last_change":"2023-07-06T03:45:11.668172+0000","last_active":"2023-07-06T03:45:13.830434+0000","last_peered":"2023-07-06T03:45:13.830434+0000","last_clean":"2023-07-06T03:45:13.830434+0000","last_became_active":"2023-07-06T03:45:11.667041+0000","last_became_peered":"2023-07-06T03:45:11.667041+0000","last_unstale":"2023-07-06T03:45:13.830434+0000","last_undegraded":"2023-07-06T03:45:13.830434+0000","last_fullsized":"2023-07-06T03:45:13.830434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:57.552603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829547+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:45:13.829547+0000","last_peered":"2023-07-06T03:45:13.829547+0000","last_clean":"2023-07-06T03:45:13.829547+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:45:13.829547+0000","last_undegraded":"2023-07-06T03:45:13.829547+0000","last_fullsized":"2023-07-06T03:45:13.829547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:25:39.144218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652538+0000","last_change":"2023-07-06T03:45:10.660900+0000","last_active":"2023-07-06T03:45:13.652538+0000","last_peered":"2023-07-06T03:45:13.652538+0000","last_clean":"2023-07-06T03:45:13.652538+0000","last_became_active":"2023-07-06T03:45:10.660490+0000","last_became_peered":"2023-07-06T03:45:10.660490+0000","last_unstale":"2023-07-06T03:45:13.652538+0000","last_undegraded":"2023-07-06T03:45:13.652538+0000","last_fullsized":"2023-07-06T03:45:13.652538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:43.801501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681363+0000","last_change":"2023-07-06T03:45:11.687991+0000","last_active":"2023-07-06T03:45:12.681363+0000","last_peered":"2023-07-06T03:45:12.681363+0000","last_clean":"2023-07-06T03:45:12.681363+0000","last_became_active":"2023-07-06T03:45:11.685096+0000","last_became_peered":"2023-07-06T03:45:11.685096+0000","last_unstale":"2023-07-06T03:45:12.681363+0000","last_undegraded":"2023-07-06T03:45:12.681363+0000","last_fullsized":"2023-07-06T03:45:12.681363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830169+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:45:13.830169+0000","last_peered":"2023-07-06T03:45:13.830169+0000","last_clean":"2023-07-06T03:45:13.830169+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:45:13.830169+0000","last_undegraded":"2023-07-06T03:45:13.830169+0000","last_fullsized":"2023-07-06T03:45:13.830169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:27:55.564390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653143+0000","last_change":"2023-07-06T03:45:10.661812+0000","last_active":"2023-07-06T03:45:13.653143+0000","last_peered":"2023-07-06T03:45:13.653143+0000","last_clean":"2023-07-06T03:45:13.653143+0000","last_became_active":"2023-07-06T03:45:10.661499+0000","last_became_peered":"2023-07-06T03:45:10.661499+0000","last_unstale":"2023-07-06T03:45:13.653143+0000","last_undegraded":"2023-07-06T03:45:13.653143+0000","last_fullsized":"2023-07-06T03:45:13.653143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:59.089238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652878+0000","last_change":"2023-07-06T03:45:11.678171+0000","last_active":"2023-07-06T03:45:13.652878+0000","last_peered":"2023-07-06T03:45:13.652878+0000","last_clean":"2023-07-06T03:45:13.652878+0000","last_became_active":"2023-07-06T03:45:11.677886+0000","last_became_peered":"2023-07-06T03:45:11.677886+0000","last_unstale":"2023-07-06T03:45:13.652878+0000","last_undegraded":"2023-07-06T03:45:13.652878+0000","last_fullsized":"2023-07-06T03:45:13.652878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206208+0000","last_change":"2023-07-06T03:45:10.657429+0000","last_active":"2023-07-06T03:45:14.206208+0000","last_peered":"2023-07-06T03:45:14.206208+0000","last_clean":"2023-07-06T03:45:14.206208+0000","last_became_active":"2023-07-06T03:45:10.657085+0000","last_became_peered":"2023-07-06T03:45:10.657085+0000","last_unstale":"2023-07-06T03:45:14.206208+0000","last_undegraded":"2023-07-06T03:45:14.206208+0000","last_fullsized":"2023-07-06T03:45:14.206208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:24:47.052321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206026+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:45:14.206026+0000","last_peered":"2023-07-06T03:45:14.206026+0000","last_clean":"2023-07-06T03:45:14.206026+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:45:14.206026+0000","last_undegraded":"2023-07-06T03:45:14.206026+0000","last_fullsized":"2023-07-06T03:45:14.206026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:05:14.989733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205502+0000","last_change":"2023-07-06T03:45:11.671780+0000","last_active":"2023-07-06T03:45:14.205502+0000","last_peered":"2023-07-06T03:45:14.205502+0000","last_clean":"2023-07-06T03:45:14.205502+0000","last_became_active":"2023-07-06T03:45:11.671390+0000","last_became_peered":"2023-07-06T03:45:11.671390+0000","last_unstale":"2023-07-06T03:45:14.205502+0000","last_undegraded":"2023-07-06T03:45:14.205502+0000","last_fullsized":"2023-07-06T03:45:14.205502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829963+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:45:13.829963+0000","last_peered":"2023-07-06T03:45:13.829963+0000","last_clean":"2023-07-06T03:45:13.829963+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:45:13.829963+0000","last_undegraded":"2023-07-06T03:45:13.829963+0000","last_fullsized":"2023-07-06T03:45:13.829963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:12.638307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206183+0000","last_change":"2023-07-06T03:45:10.657912+0000","last_active":"2023-07-06T03:45:14.206183+0000","last_peered":"2023-07-06T03:45:14.206183+0000","last_clean":"2023-07-06T03:45:14.206183+0000","last_became_active":"2023-07-06T03:45:10.657615+0000","last_became_peered":"2023-07-06T03:45:10.657615+0000","last_unstale":"2023-07-06T03:45:14.206183+0000","last_undegraded":"2023-07-06T03:45:14.206183+0000","last_fullsized":"2023-07-06T03:45:14.206183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:30:30.877422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678345+0000","last_change":"2023-07-06T03:45:11.665120+0000","last_active":"2023-07-06T03:45:12.678345+0000","last_peered":"2023-07-06T03:45:12.678345+0000","last_clean":"2023-07-06T03:45:12.678345+0000","last_became_active":"2023-07-06T03:45:11.664817+0000","last_became_peered":"2023-07-06T03:45:11.664817+0000","last_unstale":"2023-07-06T03:45:12.678345+0000","last_undegraded":"2023-07-06T03:45:12.678345+0000","last_fullsized":"2023-07-06T03:45:12.678345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:44.808607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205783+0000","last_change":"2023-07-06T03:45:10.658517+0000","last_active":"2023-07-06T03:45:14.205783+0000","last_peered":"2023-07-06T03:45:14.205783+0000","last_clean":"2023-07-06T03:45:14.205783+0000","last_became_active":"2023-07-06T03:45:10.657946+0000","last_became_peered":"2023-07-06T03:45:10.657946+0000","last_unstale":"2023-07-06T03:45:14.205783+0000","last_undegraded":"2023-07-06T03:45:14.205783+0000","last_fullsized":"2023-07-06T03:45:14.205783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:24:42.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205842+0000","last_change":"2023-07-06T03:45:11.683447+0000","last_active":"2023-07-06T03:45:14.205842+0000","last_peered":"2023-07-06T03:45:14.205842+0000","last_clean":"2023-07-06T03:45:14.205842+0000","last_became_active":"2023-07-06T03:45:11.683156+0000","last_became_peered":"2023-07-06T03:45:11.683156+0000","last_unstale":"2023-07-06T03:45:14.205842+0000","last_undegraded":"2023-07-06T03:45:14.205842+0000","last_fullsized":"2023-07-06T03:45:14.205842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818006+0000","last_change":"2023-07-06T03:45:10.649186+0000","last_active":"2023-07-06T03:45:12.818006+0000","last_peered":"2023-07-06T03:45:12.818006+0000","last_clean":"2023-07-06T03:45:12.818006+0000","last_became_active":"2023-07-06T03:45:10.648707+0000","last_became_peered":"2023-07-06T03:45:10.648707+0000","last_unstale":"2023-07-06T03:45:12.818006+0000","last_undegraded":"2023-07-06T03:45:12.818006+0000","last_fullsized":"2023-07-06T03:45:12.818006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:07.054998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678566+0000","last_change":"2023-07-06T03:45:11.690462+0000","last_active":"2023-07-06T03:45:12.678566+0000","last_peered":"2023-07-06T03:45:12.678566+0000","last_clean":"2023-07-06T03:45:12.678566+0000","last_became_active":"2023-07-06T03:45:11.690181+0000","last_became_peered":"2023-07-06T03:45:11.690181+0000","last_unstale":"2023-07-06T03:45:12.678566+0000","last_undegraded":"2023-07-06T03:45:12.678566+0000","last_fullsized":"2023-07-06T03:45:12.678566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830291+0000","last_change":"2023-07-06T03:45:10.653276+0000","last_active":"2023-07-06T03:45:13.830291+0000","last_peered":"2023-07-06T03:45:13.830291+0000","last_clean":"2023-07-06T03:45:13.830291+0000","last_became_active":"2023-07-06T03:45:10.652958+0000","last_became_peered":"2023-07-06T03:45:10.652958+0000","last_unstale":"2023-07-06T03:45:13.830291+0000","last_undegraded":"2023-07-06T03:45:13.830291+0000","last_fullsized":"2023-07-06T03:45:13.830291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:45.669370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678497+0000","last_change":"2023-07-06T03:45:11.684391+0000","last_active":"2023-07-06T03:45:12.678497+0000","last_peered":"2023-07-06T03:45:12.678497+0000","last_clean":"2023-07-06T03:45:12.678497+0000","last_became_active":"2023-07-06T03:45:11.684078+0000","last_became_peered":"2023-07-06T03:45:11.684078+0000","last_unstale":"2023-07-06T03:45:12.678497+0000","last_undegraded":"2023-07-06T03:45:12.678497+0000","last_fullsized":"2023-07-06T03:45:12.678497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681347+0000","last_change":"2023-07-06T03:45:10.645296+0000","last_active":"2023-07-06T03:45:12.681347+0000","last_peered":"2023-07-06T03:45:12.681347+0000","last_clean":"2023-07-06T03:45:12.681347+0000","last_became_active":"2023-07-06T03:45:10.644967+0000","last_became_peered":"2023-07-06T03:45:10.644967+0000","last_unstale":"2023-07-06T03:45:12.681347+0000","last_undegraded":"2023-07-06T03:45:12.681347+0000","last_fullsized":"2023-07-06T03:45:12.681347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:12.862578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678143+0000","last_change":"2023-07-06T03:45:11.683562+0000","last_active":"2023-07-06T03:45:12.678143+0000","last_peered":"2023-07-06T03:45:12.678143+0000","last_clean":"2023-07-06T03:45:12.678143+0000","last_became_active":"2023-07-06T03:45:11.683275+0000","last_became_peered":"2023-07-06T03:45:11.683275+0000","last_unstale":"2023-07-06T03:45:12.678143+0000","last_undegraded":"2023-07-06T03:45:12.678143+0000","last_fullsized":"2023-07-06T03:45:12.678143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:44.808607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681723+0000","last_change":"2023-07-06T03:45:10.654706+0000","last_active":"2023-07-06T03:45:12.681723+0000","last_peered":"2023-07-06T03:45:12.681723+0000","last_clean":"2023-07-06T03:45:12.681723+0000","last_became_active":"2023-07-06T03:45:10.653995+0000","last_became_peered":"2023-07-06T03:45:10.653995+0000","last_unstale":"2023-07-06T03:45:12.681723+0000","last_undegraded":"2023-07-06T03:45:12.681723+0000","last_fullsized":"2023-07-06T03:45:12.681723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:05:20.493361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678265+0000","last_change":"2023-07-06T03:45:11.670138+0000","last_active":"2023-07-06T03:45:12.678265+0000","last_peered":"2023-07-06T03:45:12.678265+0000","last_clean":"2023-07-06T03:45:12.678265+0000","last_became_active":"2023-07-06T03:45:11.669695+0000","last_became_peered":"2023-07-06T03:45:11.669695+0000","last_unstale":"2023-07-06T03:45:12.678265+0000","last_undegraded":"2023-07-06T03:45:12.678265+0000","last_fullsized":"2023-07-06T03:45:12.678265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:22:55.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818277+0000","last_change":"2023-07-06T03:45:11.673123+0000","last_active":"2023-07-06T03:45:12.818277+0000","last_peered":"2023-07-06T03:45:12.818277+0000","last_clean":"2023-07-06T03:45:12.818277+0000","last_became_active":"2023-07-06T03:45:11.672765+0000","last_became_peered":"2023-07-06T03:45:11.672765+0000","last_unstale":"2023-07-06T03:45:12.818277+0000","last_undegraded":"2023-07-06T03:45:12.818277+0000","last_fullsized":"2023-07-06T03:45:12.818277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652635+0000","last_change":"2023-07-06T03:45:11.689866+0000","last_active":"2023-07-06T03:45:13.652635+0000","last_peered":"2023-07-06T03:45:13.652635+0000","last_clean":"2023-07-06T03:45:13.652635+0000","last_became_active":"2023-07-06T03:45:11.689595+0000","last_became_peered":"2023-07-06T03:45:11.689595+0000","last_unstale":"2023-07-06T03:45:13.652635+0000","last_undegraded":"2023-07-06T03:45:13.652635+0000","last_fullsized":"2023-07-06T03:45:13.652635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827567+0000","last_change":"2023-07-06T03:45:11.675336+0000","last_active":"2023-07-06T03:45:13.827567+0000","last_peered":"2023-07-06T03:45:13.827567+0000","last_clean":"2023-07-06T03:45:13.827567+0000","last_became_active":"2023-07-06T03:45:11.675044+0000","last_became_peered":"2023-07-06T03:45:11.675044+0000","last_unstale":"2023-07-06T03:45:13.827567+0000","last_undegraded":"2023-07-06T03:45:13.827567+0000","last_fullsized":"2023-07-06T03:45:13.827567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680990+0000","last_change":"2023-07-06T03:45:11.689419+0000","last_active":"2023-07-06T03:45:12.680990+0000","last_peered":"2023-07-06T03:45:12.680990+0000","last_clean":"2023-07-06T03:45:12.680990+0000","last_became_active":"2023-07-06T03:45:11.688843+0000","last_became_peered":"2023-07-06T03:45:11.688843+0000","last_unstale":"2023-07-06T03:45:12.680990+0000","last_undegraded":"2023-07-06T03:45:12.680990+0000","last_fullsized":"2023-07-06T03:45:12.680990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:15.627811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680936+0000","last_change":"2023-07-06T03:45:11.687716+0000","last_active":"2023-07-06T03:45:12.680936+0000","last_peered":"2023-07-06T03:45:12.680936+0000","last_clean":"2023-07-06T03:45:12.680936+0000","last_became_active":"2023-07-06T03:45:11.685418+0000","last_became_peered":"2023-07-06T03:45:11.685418+0000","last_unstale":"2023-07-06T03:45:12.680936+0000","last_undegraded":"2023-07-06T03:45:12.680936+0000","last_fullsized":"2023-07-06T03:45:12.680936+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":847872,"data_stored":680641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":89745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":91163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:45:20.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:45:21.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.180+0000 7ff4593ff700 1 Processor -- start 2023-07-06T03:45:21.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.181+0000 7ff4593ff700 1 -- start start 2023-07-06T03:45:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff4593ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff45415c520 0x7ff4540cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff4593ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff454160180 0x7ff4541605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff4593ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff454160b30 0x7ff4540cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff4593ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4540cd8d0 con 0x7ff45415c520 2023-07-06T03:45:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff4593ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4540cdae0 con 0x7ff454160180 2023-07-06T03:45:21.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff4593ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540b6640 con 0x7ff454160b30 2023-07-06T03:45:21.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff452ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff45415c520 0x7ff4540cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff452ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff45415c520 0x7ff4540cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51354/0 (socket says 172.21.15.97:51354) 2023-07-06T03:45:21.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff452ffd700 1 -- 172.21.15.97:0/4061152688 learned_addr learned my addr 172.21.15.97:0/4061152688 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:21.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff4537fe700 1 --2- 172.21.15.97:0/4061152688 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff454160b30 0x7ff4540cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff44bfff700 1 --2- 172.21.15.97:0/4061152688 >> v2:172.21.15.146:3300/0 conn(0x7ff454160180 0x7ff4541605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff452ffd700 1 -- 172.21.15.97:0/4061152688 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff454160b30 msgr2=0x7ff4540cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:21.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff452ffd700 1 --2- 172.21.15.97:0/4061152688 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff454160b30 0x7ff4540cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff452ffd700 1 -- 172.21.15.97:0/4061152688 >> v2:172.21.15.146:3300/0 conn(0x7ff454160180 msgr2=0x7ff4541605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:21.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff452ffd700 1 --2- 172.21.15.97:0/4061152688 >> v2:172.21.15.146:3300/0 conn(0x7ff454160180 0x7ff4541605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.182+0000 7ff452ffd700 1 -- 172.21.15.97:0/4061152688 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4540b6920 con 0x7ff45415c520 2023-07-06T03:45:21.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.183+0000 7ff452ffd700 1 --2- 172.21.15.97:0/4061152688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45415c520 0x7ff4540cd390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff4540b5930 tx=0x7ff444009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60236221e5ac9e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:21.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.183+0000 7ff4527fc700 1 -- 172.21.15.97:0/4061152688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400b870 con 0x7ff45415c520 2023-07-06T03:45:21.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.183+0000 7ff4527fc700 1 -- 172.21.15.97:0/4061152688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44400b9d0 con 0x7ff45415c520 2023-07-06T03:45:21.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.183+0000 7ff4527fc700 1 -- 172.21.15.97:0/4061152688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400bd60 con 0x7ff45415c520 2023-07-06T03:45:21.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.183+0000 7ff4593ff700 1 -- 172.21.15.97:0/4061152688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45415c520 msgr2=0x7ff4540cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:21.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.183+0000 7ff4593ff700 1 --2- 172.21.15.97:0/4061152688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45415c520 0x7ff4540cd390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff4540b5930 tx=0x7ff444009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.183+0000 7ff4593ff700 1 -- 172.21.15.97:0/4061152688 shutdown_connections 2023-07-06T03:45:21.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.184+0000 7ff4593ff700 1 --2- 172.21.15.97:0/4061152688 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff454160b30 0x7ff4540cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.184+0000 7ff4593ff700 1 --2- 172.21.15.97:0/4061152688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45415c520 0x7ff4540cd390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.184+0000 7ff4593ff700 1 --2- 172.21.15.97:0/4061152688 >> v2:172.21.15.146:3300/0 conn(0x7ff454160180 0x7ff4541605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.184+0000 7ff4593ff700 1 -- 172.21.15.97:0/4061152688 >> 172.21.15.97:0/4061152688 conn(0x7ff454071790 msgr2=0x7ff454071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:21.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.184+0000 7ff4593ff700 1 -- 172.21.15.97:0/4061152688 shutdown_connections 2023-07-06T03:45:21.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.184+0000 7ff4593ff700 1 -- 172.21.15.97:0/4061152688 wait complete. 2023-07-06T03:45:21.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.185+0000 7ff4593ff700 1 Processor -- start 2023-07-06T03:45:21.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.185+0000 7ff4593ff700 1 -- start start 2023-07-06T03:45:21.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.186+0000 7ff4593ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45415c520 0x7ff45415f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.186+0000 7ff4593ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff454160180 0x7ff454159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.186+0000 7ff4593ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff454160b30 0x7ff454159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.186+0000 7ff4593ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540d2120 con 0x7ff45415c520 2023-07-06T03:45:21.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.186+0000 7ff4593ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45415a3d0 con 0x7ff454160180 2023-07-06T03:45:21.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.186+0000 7ff4593ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45415a6b0 con 0x7ff454160b30 2023-07-06T03:45:21.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff452ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45415c520 0x7ff45415f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff452ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45415c520 0x7ff45415f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51360/0 (socket says 172.21.15.97:51360) 2023-07-06T03:45:21.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff452ffd700 1 -- 172.21.15.97:0/2415795742 learned_addr learned my addr 172.21.15.97:0/2415795742 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:21.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff44bfff700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff454160180 0x7ff454159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff4537fe700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff454160b30 0x7ff454159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff452ffd700 1 -- 172.21.15.97:0/2415795742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff454160b30 msgr2=0x7ff454159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:21.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff452ffd700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff454160b30 0x7ff454159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff452ffd700 1 -- 172.21.15.97:0/2415795742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff454160180 msgr2=0x7ff454159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:21.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff452ffd700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff454160180 0x7ff454159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff452ffd700 1 -- 172.21.15.97:0/2415795742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff444009550 con 0x7ff45415c520 2023-07-06T03:45:21.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff4537fe700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff454160b30 0x7ff454159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:21.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff452ffd700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45415c520 0x7ff45415f0f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff444000c00 tx=0x7ff444004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:21.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff44b7fe700 1 -- 172.21.15.97:0/2415795742 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444004540 con 0x7ff45415c520 2023-07-06T03:45:21.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.187+0000 7ff44b7fe700 1 -- 172.21.15.97:0/2415795742 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4440036a0 con 0x7ff45415c520 2023-07-06T03:45:21.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.188+0000 7ff4593ff700 1 -- 172.21.15.97:0/2415795742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4540d1a60 con 0x7ff45415c520 2023-07-06T03:45:21.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.188+0000 7ff44b7fe700 1 -- 172.21.15.97:0/2415795742 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444003a50 con 0x7ff45415c520 2023-07-06T03:45:21.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.188+0000 7ff4593ff700 1 -- 172.21.15.97:0/2415795742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45415d3f0 con 0x7ff45415c520 2023-07-06T03:45:21.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.189+0000 7ff4593ff700 1 -- 172.21.15.97:0/2415795742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45404e340 con 0x7ff45415c520 2023-07-06T03:45:21.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.189+0000 7ff44b7fe700 1 -- 172.21.15.97:0/2415795742 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff444021070 con 0x7ff45415c520 2023-07-06T03:45:21.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.190+0000 7ff44b7fe700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff43406ace0 0x7ff43406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.190+0000 7ff44b7fe700 1 -- 172.21.15.97:0/2415795742 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff44408abc0 con 0x7ff45415c520 2023-07-06T03:45:21.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.191+0000 7ff44bfff700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff43406ace0 0x7ff43406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.191+0000 7ff44bfff700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff43406ace0 0x7ff43406d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff4541f6940 tx=0x7ff43c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:21.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.195+0000 7ff44b7fe700 1 -- 172.21.15.97:0/2415795742 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff444055c60 con 0x7ff45415c520 2023-07-06T03:45:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.388+0000 7ff4593ff700 1 -- 172.21.15.97:0/2415795742 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4540b6ee0 con 0x7ff43406ace0 2023-07-06T03:45:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.396+0000 7ff44b7fe700 1 -- 172.21.15.97:0/2415795742 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286825 (secure 0 0 0) 0x7ff43c009dd0 con 0x7ff43406ace0 2023-07-06T03:45:21.401 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 -- 172.21.15.97:0/2415795742 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff43406ace0 msgr2=0x7ff43406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff43406ace0 0x7ff43406d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff4541f6940 tx=0x7ff43c008040 comp rx=0 tx=0).stop 2023-07-06T03:45:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 -- 172.21.15.97:0/2415795742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45415c520 msgr2=0x7ff45415f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45415c520 0x7ff45415f0f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff444000c00 tx=0x7ff444004750 comp rx=0 tx=0).stop 2023-07-06T03:45:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 -- 172.21.15.97:0/2415795742 shutdown_connections 2023-07-06T03:45:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff43406ace0 0x7ff43406d190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff454160b30 0x7ff454159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff45415c520 0x7ff45415f0f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 --2- 172.21.15.97:0/2415795742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff454160180 0x7ff454159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 -- 172.21.15.97:0/2415795742 >> 172.21.15.97:0/2415795742 conn(0x7ff454071790 msgr2=0x7ff454072010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 -- 172.21.15.97:0/2415795742 shutdown_connections 2023-07-06T03:45:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.400+0000 7ff4593ff700 1 -- 172.21.15.97:0/2415795742 wait complete. 2023-07-06T03:45:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:45:21.420 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-07-06T03:45:20.884068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218724,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.230775"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681463+0000","last_change":"2023-07-06T03:45:11.693177+0000","last_active":"2023-07-06T03:45:12.681463+0000","last_peered":"2023-07-06T03:45:12.681463+0000","last_clean":"2023-07-06T03:45:12.681463+0000","last_became_active":"2023-07-06T03:45:11.692851+0000","last_became_peered":"2023-07-06T03:45:11.692851+0000","last_unstale":"2023-07-06T03:45:12.681463+0000","last_undegraded":"2023-07-06T03:45:12.681463+0000","last_fullsized":"2023-07-06T03:45:12.681463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678856+0000","last_change":"2023-07-06T03:45:11.674359+0000","last_active":"2023-07-06T03:45:12.678856+0000","last_peered":"2023-07-06T03:45:12.678856+0000","last_clean":"2023-07-06T03:45:12.678856+0000","last_became_active":"2023-07-06T03:45:11.674035+0000","last_became_peered":"2023-07-06T03:45:11.674035+0000","last_unstale":"2023-07-06T03:45:12.678856+0000","last_undegraded":"2023-07-06T03:45:12.678856+0000","last_fullsized":"2023-07-06T03:45:12.678856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827906+0000","last_change":"2023-07-06T03:45:11.691806+0000","last_active":"2023-07-06T03:45:13.827906+0000","last_peered":"2023-07-06T03:45:13.827906+0000","last_clean":"2023-07-06T03:45:13.827906+0000","last_became_active":"2023-07-06T03:45:11.691532+0000","last_became_peered":"2023-07-06T03:45:11.691532+0000","last_unstale":"2023-07-06T03:45:13.827906+0000","last_undegraded":"2023-07-06T03:45:13.827906+0000","last_fullsized":"2023-07-06T03:45:13.827906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827578+0000","last_change":"2023-07-06T03:45:11.675300+0000","last_active":"2023-07-06T03:45:13.827578+0000","last_peered":"2023-07-06T03:45:13.827578+0000","last_clean":"2023-07-06T03:45:13.827578+0000","last_became_active":"2023-07-06T03:45:11.675018+0000","last_became_peered":"2023-07-06T03:45:11.675018+0000","last_unstale":"2023-07-06T03:45:13.827578+0000","last_undegraded":"2023-07-06T03:45:13.827578+0000","last_fullsized":"2023-07-06T03:45:13.827578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205854+0000","last_change":"2023-07-06T03:45:11.666099+0000","last_active":"2023-07-06T03:45:14.205854+0000","last_peered":"2023-07-06T03:45:14.205854+0000","last_clean":"2023-07-06T03:45:14.205854+0000","last_became_active":"2023-07-06T03:45:11.665759+0000","last_became_peered":"2023-07-06T03:45:11.665759+0000","last_unstale":"2023-07-06T03:45:14.205854+0000","last_undegraded":"2023-07-06T03:45:14.205854+0000","last_fullsized":"2023-07-06T03:45:14.205854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:52.666500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830215+0000","last_change":"2023-07-06T03:45:11.662251+0000","last_active":"2023-07-06T03:45:13.830215+0000","last_peered":"2023-07-06T03:45:13.830215+0000","last_clean":"2023-07-06T03:45:13.830215+0000","last_became_active":"2023-07-06T03:45:11.661916+0000","last_became_peered":"2023-07-06T03:45:11.661916+0000","last_unstale":"2023-07-06T03:45:13.830215+0000","last_undegraded":"2023-07-06T03:45:13.830215+0000","last_fullsized":"2023-07-06T03:45:13.830215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:11.287937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829687+0000","last_change":"2023-07-06T03:45:11.676465+0000","last_active":"2023-07-06T03:45:13.829687+0000","last_peered":"2023-07-06T03:45:13.829687+0000","last_clean":"2023-07-06T03:45:13.829687+0000","last_became_active":"2023-07-06T03:45:11.676180+0000","last_became_peered":"2023-07-06T03:45:11.676180+0000","last_unstale":"2023-07-06T03:45:13.829687+0000","last_undegraded":"2023-07-06T03:45:13.829687+0000","last_fullsized":"2023-07-06T03:45:13.829687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:38:24.542890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818440+0000","last_change":"2023-07-06T03:45:11.676387+0000","last_active":"2023-07-06T03:45:12.818440+0000","last_peered":"2023-07-06T03:45:12.818440+0000","last_clean":"2023-07-06T03:45:12.818440+0000","last_became_active":"2023-07-06T03:45:11.676129+0000","last_became_peered":"2023-07-06T03:45:11.676129+0000","last_unstale":"2023-07-06T03:45:12.818440+0000","last_undegraded":"2023-07-06T03:45:12.818440+0000","last_fullsized":"2023-07-06T03:45:12.818440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206076+0000","last_change":"2023-07-06T03:45:11.690977+0000","last_active":"2023-07-06T03:45:14.206076+0000","last_peered":"2023-07-06T03:45:14.206076+0000","last_clean":"2023-07-06T03:45:14.206076+0000","last_became_active":"2023-07-06T03:45:11.690423+0000","last_became_peered":"2023-07-06T03:45:11.690423+0000","last_unstale":"2023-07-06T03:45:14.206076+0000","last_undegraded":"2023-07-06T03:45:14.206076+0000","last_fullsized":"2023-07-06T03:45:14.206076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:14:59.597908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818347+0000","last_change":"2023-07-06T03:45:11.680924+0000","last_active":"2023-07-06T03:45:12.818347+0000","last_peered":"2023-07-06T03:45:12.818347+0000","last_clean":"2023-07-06T03:45:12.818347+0000","last_became_active":"2023-07-06T03:45:11.680607+0000","last_became_peered":"2023-07-06T03:45:11.680607+0000","last_unstale":"2023-07-06T03:45:12.818347+0000","last_undegraded":"2023-07-06T03:45:12.818347+0000","last_fullsized":"2023-07-06T03:45:12.818347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:13.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818149+0000","last_change":"2023-07-06T03:45:11.677946+0000","last_active":"2023-07-06T03:45:12.818149+0000","last_peered":"2023-07-06T03:45:12.818149+0000","last_clean":"2023-07-06T03:45:12.818149+0000","last_became_active":"2023-07-06T03:45:11.677684+0000","last_became_peered":"2023-07-06T03:45:11.677684+0000","last_unstale":"2023-07-06T03:45:12.818149+0000","last_undegraded":"2023-07-06T03:45:12.818149+0000","last_fullsized":"2023-07-06T03:45:12.818149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:27.384754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680941+0000","last_change":"2023-07-06T03:45:11.681178+0000","last_active":"2023-07-06T03:45:12.680941+0000","last_peered":"2023-07-06T03:45:12.680941+0000","last_clean":"2023-07-06T03:45:12.680941+0000","last_became_active":"2023-07-06T03:45:11.680841+0000","last_became_peered":"2023-07-06T03:45:11.680841+0000","last_unstale":"2023-07-06T03:45:12.680941+0000","last_undegraded":"2023-07-06T03:45:12.680941+0000","last_fullsized":"2023-07-06T03:45:12.680941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:58.087282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652673+0000","last_change":"2023-07-06T03:45:11.687461+0000","last_active":"2023-07-06T03:45:13.652673+0000","last_peered":"2023-07-06T03:45:13.652673+0000","last_clean":"2023-07-06T03:45:13.652673+0000","last_became_active":"2023-07-06T03:45:11.687182+0000","last_became_peered":"2023-07-06T03:45:11.687182+0000","last_unstale":"2023-07-06T03:45:13.652673+0000","last_undegraded":"2023-07-06T03:45:13.652673+0000","last_fullsized":"2023-07-06T03:45:13.652673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817899+0000","last_change":"2023-07-06T03:45:11.691496+0000","last_active":"2023-07-06T03:45:12.817899+0000","last_peered":"2023-07-06T03:45:12.817899+0000","last_clean":"2023-07-06T03:45:12.817899+0000","last_became_active":"2023-07-06T03:45:11.691206+0000","last_became_peered":"2023-07-06T03:45:11.691206+0000","last_unstale":"2023-07-06T03:45:12.817899+0000","last_undegraded":"2023-07-06T03:45:12.817899+0000","last_fullsized":"2023-07-06T03:45:12.817899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827487+0000","last_change":"2023-07-06T03:45:11.685001+0000","last_active":"2023-07-06T03:45:13.827487+0000","last_peered":"2023-07-06T03:45:13.827487+0000","last_clean":"2023-07-06T03:45:13.827487+0000","last_became_active":"2023-07-06T03:45:11.684695+0000","last_became_peered":"2023-07-06T03:45:11.684695+0000","last_unstale":"2023-07-06T03:45:13.827487+0000","last_undegraded":"2023-07-06T03:45:13.827487+0000","last_fullsized":"2023-07-06T03:45:13.827487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829742+0000","last_change":"2023-07-06T03:45:11.662096+0000","last_active":"2023-07-06T03:45:13.829742+0000","last_peered":"2023-07-06T03:45:13.829742+0000","last_clean":"2023-07-06T03:45:13.829742+0000","last_became_active":"2023-07-06T03:45:11.661834+0000","last_became_peered":"2023-07-06T03:45:11.661834+0000","last_unstale":"2023-07-06T03:45:13.829742+0000","last_undegraded":"2023-07-06T03:45:13.829742+0000","last_fullsized":"2023-07-06T03:45:13.829742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:36:49.534209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678504+0000","last_change":"2023-07-06T03:45:11.685162+0000","last_active":"2023-07-06T03:45:12.678504+0000","last_peered":"2023-07-06T03:45:12.678504+0000","last_clean":"2023-07-06T03:45:12.678504+0000","last_became_active":"2023-07-06T03:45:11.684977+0000","last_became_peered":"2023-07-06T03:45:11.684977+0000","last_unstale":"2023-07-06T03:45:12.678504+0000","last_undegraded":"2023-07-06T03:45:12.678504+0000","last_fullsized":"2023-07-06T03:45:12.678504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827827+0000","last_change":"2023-07-06T03:45:11.675611+0000","last_active":"2023-07-06T03:45:13.827827+0000","last_peered":"2023-07-06T03:45:13.827827+0000","last_clean":"2023-07-06T03:45:13.827827+0000","last_became_active":"2023-07-06T03:45:11.675348+0000","last_became_peered":"2023-07-06T03:45:11.675348+0000","last_unstale":"2023-07-06T03:45:13.827827+0000","last_undegraded":"2023-07-06T03:45:13.827827+0000","last_fullsized":"2023-07-06T03:45:13.827827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:00.096345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680922+0000","last_change":"2023-07-06T03:45:11.690182+0000","last_active":"2023-07-06T03:45:12.680922+0000","last_peered":"2023-07-06T03:45:12.680922+0000","last_clean":"2023-07-06T03:45:12.680922+0000","last_became_active":"2023-07-06T03:45:11.689744+0000","last_became_peered":"2023-07-06T03:45:11.689744+0000","last_unstale":"2023-07-06T03:45:12.680922+0000","last_undegraded":"2023-07-06T03:45:12.680922+0000","last_fullsized":"2023-07-06T03:45:12.680922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:32.898881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818137+0000","last_change":"2023-07-06T03:45:11.672749+0000","last_active":"2023-07-06T03:45:12.818137+0000","last_peered":"2023-07-06T03:45:12.818137+0000","last_clean":"2023-07-06T03:45:12.818137+0000","last_became_active":"2023-07-06T03:45:11.672401+0000","last_became_peered":"2023-07-06T03:45:11.672401+0000","last_unstale":"2023-07-06T03:45:12.818137+0000","last_undegraded":"2023-07-06T03:45:12.818137+0000","last_fullsized":"2023-07-06T03:45:12.818137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:15.627811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818032+0000","last_change":"2023-07-06T03:45:11.692060+0000","last_active":"2023-07-06T03:45:12.818032+0000","last_peered":"2023-07-06T03:45:12.818032+0000","last_clean":"2023-07-06T03:45:12.818032+0000","last_became_active":"2023-07-06T03:45:11.691699+0000","last_became_peered":"2023-07-06T03:45:11.691699+0000","last_unstale":"2023-07-06T03:45:12.818032+0000","last_undegraded":"2023-07-06T03:45:12.818032+0000","last_fullsized":"2023-07-06T03:45:12.818032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:32.898881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678195+0000","last_change":"2023-07-06T03:45:11.672926+0000","last_active":"2023-07-06T03:45:12.678195+0000","last_peered":"2023-07-06T03:45:12.678195+0000","last_clean":"2023-07-06T03:45:12.678195+0000","last_became_active":"2023-07-06T03:45:11.672627+0000","last_became_peered":"2023-07-06T03:45:11.672627+0000","last_unstale":"2023-07-06T03:45:12.678195+0000","last_undegraded":"2023-07-06T03:45:12.678195+0000","last_fullsized":"2023-07-06T03:45:12.678195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205814+0000","last_change":"2023-07-06T03:45:11.691373+0000","last_active":"2023-07-06T03:45:14.205814+0000","last_peered":"2023-07-06T03:45:14.205814+0000","last_clean":"2023-07-06T03:45:14.205814+0000","last_became_active":"2023-07-06T03:45:11.691046+0000","last_became_peered":"2023-07-06T03:45:11.691046+0000","last_unstale":"2023-07-06T03:45:14.205814+0000","last_undegraded":"2023-07-06T03:45:14.205814+0000","last_fullsized":"2023-07-06T03:45:14.205814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818372+0000","last_change":"2023-07-06T03:45:11.691643+0000","last_active":"2023-07-06T03:45:12.818372+0000","last_peered":"2023-07-06T03:45:12.818372+0000","last_clean":"2023-07-06T03:45:12.818372+0000","last_became_active":"2023-07-06T03:45:11.691382+0000","last_became_peered":"2023-07-06T03:45:11.691382+0000","last_unstale":"2023-07-06T03:45:12.818372+0000","last_undegraded":"2023-07-06T03:45:12.818372+0000","last_fullsized":"2023-07-06T03:45:12.818372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206243+0000","last_change":"2023-07-06T03:45:11.670205+0000","last_active":"2023-07-06T03:45:14.206243+0000","last_peered":"2023-07-06T03:45:14.206243+0000","last_clean":"2023-07-06T03:45:14.206243+0000","last_became_active":"2023-07-06T03:45:11.669935+0000","last_became_peered":"2023-07-06T03:45:11.669935+0000","last_unstale":"2023-07-06T03:45:14.206243+0000","last_undegraded":"2023-07-06T03:45:14.206243+0000","last_fullsized":"2023-07-06T03:45:14.206243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:11:07.089396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678592+0000","last_change":"2023-07-06T03:45:11.674601+0000","last_active":"2023-07-06T03:45:12.678592+0000","last_peered":"2023-07-06T03:45:12.678592+0000","last_clean":"2023-07-06T03:45:12.678592+0000","last_became_active":"2023-07-06T03:45:11.673533+0000","last_became_peered":"2023-07-06T03:45:11.673533+0000","last_unstale":"2023-07-06T03:45:12.678592+0000","last_undegraded":"2023-07-06T03:45:12.678592+0000","last_fullsized":"2023-07-06T03:45:12.678592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830285+0000","last_change":"2023-07-06T03:45:11.678777+0000","last_active":"2023-07-06T03:45:13.830285+0000","last_peered":"2023-07-06T03:45:13.830285+0000","last_clean":"2023-07-06T03:45:13.830285+0000","last_became_active":"2023-07-06T03:45:11.678437+0000","last_became_peered":"2023-07-06T03:45:11.678437+0000","last_unstale":"2023-07-06T03:45:13.830285+0000","last_undegraded":"2023-07-06T03:45:13.830285+0000","last_fullsized":"2023-07-06T03:45:13.830285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:51:53.330078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818105+0000","last_change":"2023-07-06T03:45:10.656464+0000","last_active":"2023-07-06T03:45:12.818105+0000","last_peered":"2023-07-06T03:45:12.818105+0000","last_clean":"2023-07-06T03:45:12.818105+0000","last_became_active":"2023-07-06T03:45:10.656116+0000","last_became_peered":"2023-07-06T03:45:10.656116+0000","last_unstale":"2023-07-06T03:45:12.818105+0000","last_undegraded":"2023-07-06T03:45:12.818105+0000","last_fullsized":"2023-07-06T03:45:12.818105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:57.080176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678331+0000","last_change":"2023-07-06T03:45:11.687243+0000","last_active":"2023-07-06T03:45:12.678331+0000","last_peered":"2023-07-06T03:45:12.678331+0000","last_clean":"2023-07-06T03:45:12.678331+0000","last_became_active":"2023-07-06T03:45:11.686947+0000","last_became_peered":"2023-07-06T03:45:11.686947+0000","last_unstale":"2023-07-06T03:45:12.678331+0000","last_undegraded":"2023-07-06T03:45:12.678331+0000","last_fullsized":"2023-07-06T03:45:12.678331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205529+0000","last_change":"2023-07-06T03:45:10.657736+0000","last_active":"2023-07-06T03:45:14.205529+0000","last_peered":"2023-07-06T03:45:14.205529+0000","last_clean":"2023-07-06T03:45:14.205529+0000","last_became_active":"2023-07-06T03:45:10.657450+0000","last_became_peered":"2023-07-06T03:45:10.657450+0000","last_unstale":"2023-07-06T03:45:14.205529+0000","last_undegraded":"2023-07-06T03:45:14.205529+0000","last_fullsized":"2023-07-06T03:45:14.205529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:14.620705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206125+0000","last_change":"2023-07-06T03:45:11.672564+0000","last_active":"2023-07-06T03:45:14.206125+0000","last_peered":"2023-07-06T03:45:14.206125+0000","last_clean":"2023-07-06T03:45:14.206125+0000","last_became_active":"2023-07-06T03:45:11.672150+0000","last_became_peered":"2023-07-06T03:45:11.672150+0000","last_unstale":"2023-07-06T03:45:14.206125+0000","last_undegraded":"2023-07-06T03:45:14.206125+0000","last_fullsized":"2023-07-06T03:45:14.206125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:59:01.536411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680441+0000","last_change":"2023-07-06T03:45:10.653666+0000","last_active":"2023-07-06T03:45:12.680441+0000","last_peered":"2023-07-06T03:45:12.680441+0000","last_clean":"2023-07-06T03:45:12.680441+0000","last_became_active":"2023-07-06T03:45:10.653357+0000","last_became_peered":"2023-07-06T03:45:10.653357+0000","last_unstale":"2023-07-06T03:45:12.680441+0000","last_undegraded":"2023-07-06T03:45:12.680441+0000","last_fullsized":"2023-07-06T03:45:12.680441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:25.838923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830215+0000","last_change":"2023-07-06T03:45:11.690985+0000","last_active":"2023-07-06T03:45:13.830215+0000","last_peered":"2023-07-06T03:45:13.830215+0000","last_clean":"2023-07-06T03:45:13.830215+0000","last_became_active":"2023-07-06T03:45:11.690692+0000","last_became_peered":"2023-07-06T03:45:11.690692+0000","last_unstale":"2023-07-06T03:45:13.830215+0000","last_undegraded":"2023-07-06T03:45:13.830215+0000","last_fullsized":"2023-07-06T03:45:13.830215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:27:20.751301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681704+0000","last_change":"2023-07-06T03:45:10.649734+0000","last_active":"2023-07-06T03:45:12.681704+0000","last_peered":"2023-07-06T03:45:12.681704+0000","last_clean":"2023-07-06T03:45:12.681704+0000","last_became_active":"2023-07-06T03:45:10.649401+0000","last_became_peered":"2023-07-06T03:45:10.649401+0000","last_unstale":"2023-07-06T03:45:12.681704+0000","last_undegraded":"2023-07-06T03:45:12.681704+0000","last_fullsized":"2023-07-06T03:45:12.681704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:24:42.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678483+0000","last_change":"2023-07-06T03:45:11.667398+0000","last_active":"2023-07-06T03:45:12.678483+0000","last_peered":"2023-07-06T03:45:12.678483+0000","last_clean":"2023-07-06T03:45:12.678483+0000","last_became_active":"2023-07-06T03:45:11.667124+0000","last_became_peered":"2023-07-06T03:45:11.667124+0000","last_unstale":"2023-07-06T03:45:12.678483+0000","last_undegraded":"2023-07-06T03:45:12.678483+0000","last_fullsized":"2023-07-06T03:45:12.678483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:00.096345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.679013+0000","last_change":"2023-07-06T03:45:10.648660+0000","last_active":"2023-07-06T03:45:12.679013+0000","last_peered":"2023-07-06T03:45:12.679013+0000","last_clean":"2023-07-06T03:45:12.679013+0000","last_became_active":"2023-07-06T03:45:10.648254+0000","last_became_peered":"2023-07-06T03:45:10.648254+0000","last_unstale":"2023-07-06T03:45:12.679013+0000","last_undegraded":"2023-07-06T03:45:12.679013+0000","last_fullsized":"2023-07-06T03:45:12.679013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:25.838923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205490+0000","last_change":"2023-07-06T03:45:11.682171+0000","last_active":"2023-07-06T03:45:14.205490+0000","last_peered":"2023-07-06T03:45:14.205490+0000","last_clean":"2023-07-06T03:45:14.205490+0000","last_became_active":"2023-07-06T03:45:11.681861+0000","last_became_peered":"2023-07-06T03:45:11.681861+0000","last_unstale":"2023-07-06T03:45:14.205490+0000","last_undegraded":"2023-07-06T03:45:14.205490+0000","last_fullsized":"2023-07-06T03:45:14.205490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827972+0000","last_change":"2023-07-06T03:45:10.656590+0000","last_active":"2023-07-06T03:45:13.827972+0000","last_peered":"2023-07-06T03:45:13.827972+0000","last_clean":"2023-07-06T03:45:13.827972+0000","last_became_active":"2023-07-06T03:45:10.656328+0000","last_became_peered":"2023-07-06T03:45:10.656328+0000","last_unstale":"2023-07-06T03:45:13.827972+0000","last_undegraded":"2023-07-06T03:45:13.827972+0000","last_fullsized":"2023-07-06T03:45:13.827972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205720+0000","last_change":"2023-07-06T03:45:11.671697+0000","last_active":"2023-07-06T03:45:14.205720+0000","last_peered":"2023-07-06T03:45:14.205720+0000","last_clean":"2023-07-06T03:45:14.205720+0000","last_became_active":"2023-07-06T03:45:11.671343+0000","last_became_peered":"2023-07-06T03:45:11.671343+0000","last_unstale":"2023-07-06T03:45:14.205720+0000","last_undegraded":"2023-07-06T03:45:14.205720+0000","last_fullsized":"2023-07-06T03:45:14.205720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:13.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829989+0000","last_change":"2023-07-06T03:45:10.644219+0000","last_active":"2023-07-06T03:45:13.829989+0000","last_peered":"2023-07-06T03:45:13.829989+0000","last_clean":"2023-07-06T03:45:13.829989+0000","last_became_active":"2023-07-06T03:45:10.643854+0000","last_became_peered":"2023-07-06T03:45:10.643854+0000","last_unstale":"2023-07-06T03:45:13.829989+0000","last_undegraded":"2023-07-06T03:45:13.829989+0000","last_fullsized":"2023-07-06T03:45:13.829989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:50:44.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678700+0000","last_change":"2023-07-06T03:45:11.665747+0000","last_active":"2023-07-06T03:45:12.678700+0000","last_peered":"2023-07-06T03:45:12.678700+0000","last_clean":"2023-07-06T03:45:12.678700+0000","last_became_active":"2023-07-06T03:45:11.665443+0000","last_became_peered":"2023-07-06T03:45:11.665443+0000","last_unstale":"2023-07-06T03:45:12.678700+0000","last_undegraded":"2023-07-06T03:45:12.678700+0000","last_fullsized":"2023-07-06T03:45:12.678700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829814+0000","last_change":"2023-07-06T03:45:10.653534+0000","last_active":"2023-07-06T03:45:13.829814+0000","last_peered":"2023-07-06T03:45:13.829814+0000","last_clean":"2023-07-06T03:45:13.829814+0000","last_became_active":"2023-07-06T03:45:10.653232+0000","last_became_peered":"2023-07-06T03:45:10.653232+0000","last_unstale":"2023-07-06T03:45:13.829814+0000","last_undegraded":"2023-07-06T03:45:13.829814+0000","last_fullsized":"2023-07-06T03:45:13.829814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:16.021365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652974+0000","last_change":"2023-07-06T03:45:11.683611+0000","last_active":"2023-07-06T03:45:13.652974+0000","last_peered":"2023-07-06T03:45:13.652974+0000","last_clean":"2023-07-06T03:45:13.652974+0000","last_became_active":"2023-07-06T03:45:11.683343+0000","last_became_peered":"2023-07-06T03:45:11.683343+0000","last_unstale":"2023-07-06T03:45:13.652974+0000","last_undegraded":"2023-07-06T03:45:13.652974+0000","last_fullsized":"2023-07-06T03:45:13.652974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652756+0000","last_change":"2023-07-06T03:45:10.655738+0000","last_active":"2023-07-06T03:45:13.652756+0000","last_peered":"2023-07-06T03:45:13.652756+0000","last_clean":"2023-07-06T03:45:13.652756+0000","last_became_active":"2023-07-06T03:45:10.655414+0000","last_became_peered":"2023-07-06T03:45:10.655414+0000","last_unstale":"2023-07-06T03:45:13.652756+0000","last_undegraded":"2023-07-06T03:45:13.652756+0000","last_fullsized":"2023-07-06T03:45:13.652756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:38.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830471+0000","last_change":"2023-07-06T03:45:11.678704+0000","last_active":"2023-07-06T03:45:13.830471+0000","last_peered":"2023-07-06T03:45:13.830471+0000","last_clean":"2023-07-06T03:45:13.830471+0000","last_became_active":"2023-07-06T03:45:11.678393+0000","last_became_peered":"2023-07-06T03:45:11.678393+0000","last_unstale":"2023-07-06T03:45:13.830471+0000","last_undegraded":"2023-07-06T03:45:13.830471+0000","last_fullsized":"2023-07-06T03:45:13.830471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:42.303647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830064+0000","last_change":"2023-07-06T03:45:10.644075+0000","last_active":"2023-07-06T03:45:13.830064+0000","last_peered":"2023-07-06T03:45:13.830064+0000","last_clean":"2023-07-06T03:45:13.830064+0000","last_became_active":"2023-07-06T03:45:10.643785+0000","last_became_peered":"2023-07-06T03:45:10.643785+0000","last_unstale":"2023-07-06T03:45:13.830064+0000","last_undegraded":"2023-07-06T03:45:13.830064+0000","last_fullsized":"2023-07-06T03:45:13.830064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:01:07.035467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681216+0000","last_change":"2023-07-06T03:45:11.687766+0000","last_active":"2023-07-06T03:45:12.681216+0000","last_peered":"2023-07-06T03:45:12.681216+0000","last_clean":"2023-07-06T03:45:12.681216+0000","last_became_active":"2023-07-06T03:45:11.687528+0000","last_became_peered":"2023-07-06T03:45:11.687528+0000","last_unstale":"2023-07-06T03:45:12.681216+0000","last_undegraded":"2023-07-06T03:45:12.681216+0000","last_fullsized":"2023-07-06T03:45:12.681216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817929+0000","last_change":"2023-07-06T03:45:10.652546+0000","last_active":"2023-07-06T03:45:12.817929+0000","last_peered":"2023-07-06T03:45:12.817929+0000","last_clean":"2023-07-06T03:45:12.817929+0000","last_became_active":"2023-07-06T03:45:10.652314+0000","last_became_peered":"2023-07-06T03:45:10.652314+0000","last_unstale":"2023-07-06T03:45:12.817929+0000","last_undegraded":"2023-07-06T03:45:12.817929+0000","last_fullsized":"2023-07-06T03:45:12.817929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681543+0000","last_change":"2023-07-06T03:45:11.695000+0000","last_active":"2023-07-06T03:45:12.681543+0000","last_peered":"2023-07-06T03:45:12.681543+0000","last_clean":"2023-07-06T03:45:12.681543+0000","last_became_active":"2023-07-06T03:45:11.694569+0000","last_became_peered":"2023-07-06T03:45:11.694569+0000","last_unstale":"2023-07-06T03:45:12.681543+0000","last_undegraded":"2023-07-06T03:45:12.681543+0000","last_fullsized":"2023-07-06T03:45:12.681543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:52.666500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.828097+0000","last_change":"2023-07-06T03:45:10.654280+0000","last_active":"2023-07-06T03:45:13.828097+0000","last_peered":"2023-07-06T03:45:13.828097+0000","last_clean":"2023-07-06T03:45:13.828097+0000","last_became_active":"2023-07-06T03:45:10.654039+0000","last_became_peered":"2023-07-06T03:45:10.654039+0000","last_unstale":"2023-07-06T03:45:13.828097+0000","last_undegraded":"2023-07-06T03:45:13.828097+0000","last_fullsized":"2023-07-06T03:45:13.828097+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:07.054998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829778+0000","last_change":"2023-07-06T03:45:11.667282+0000","last_active":"2023-07-06T03:45:13.829778+0000","last_peered":"2023-07-06T03:45:13.829778+0000","last_clean":"2023-07-06T03:45:13.829778+0000","last_became_active":"2023-07-06T03:45:11.667027+0000","last_became_peered":"2023-07-06T03:45:11.667027+0000","last_unstale":"2023-07-06T03:45:13.829778+0000","last_undegraded":"2023-07-06T03:45:13.829778+0000","last_fullsized":"2023-07-06T03:45:13.829778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:05:40.764560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678971+0000","last_change":"2023-07-06T03:45:10.647142+0000","last_active":"2023-07-06T03:45:12.678971+0000","last_peered":"2023-07-06T03:45:12.678971+0000","last_clean":"2023-07-06T03:45:12.678971+0000","last_became_active":"2023-07-06T03:45:10.646882+0000","last_became_peered":"2023-07-06T03:45:10.646882+0000","last_unstale":"2023-07-06T03:45:12.678971+0000","last_undegraded":"2023-07-06T03:45:12.678971+0000","last_fullsized":"2023-07-06T03:45:12.678971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:59.089238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817907+0000","last_change":"2023-07-06T03:45:11.691436+0000","last_active":"2023-07-06T03:45:12.817907+0000","last_peered":"2023-07-06T03:45:12.817907+0000","last_clean":"2023-07-06T03:45:12.817907+0000","last_became_active":"2023-07-06T03:45:11.691139+0000","last_became_peered":"2023-07-06T03:45:11.691139+0000","last_unstale":"2023-07-06T03:45:12.817907+0000","last_undegraded":"2023-07-06T03:45:12.817907+0000","last_fullsized":"2023-07-06T03:45:12.817907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829985+0000","last_change":"2023-07-06T03:45:10.647677+0000","last_active":"2023-07-06T03:45:13.829985+0000","last_peered":"2023-07-06T03:45:13.829985+0000","last_clean":"2023-07-06T03:45:13.829985+0000","last_became_active":"2023-07-06T03:45:10.647414+0000","last_became_peered":"2023-07-06T03:45:10.647414+0000","last_unstale":"2023-07-06T03:45:13.829985+0000","last_undegraded":"2023-07-06T03:45:13.829985+0000","last_fullsized":"2023-07-06T03:45:13.829985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:34:02.031995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678552+0000","last_change":"2023-07-06T03:45:11.663454+0000","last_active":"2023-07-06T03:45:12.678552+0000","last_peered":"2023-07-06T03:45:12.678552+0000","last_clean":"2023-07-06T03:45:12.678552+0000","last_became_active":"2023-07-06T03:45:11.663172+0000","last_became_peered":"2023-07-06T03:45:11.663172+0000","last_unstale":"2023-07-06T03:45:12.678552+0000","last_undegraded":"2023-07-06T03:45:12.678552+0000","last_fullsized":"2023-07-06T03:45:12.678552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827748+0000","last_change":"2023-07-06T03:45:10.651255+0000","last_active":"2023-07-06T03:45:13.827748+0000","last_peered":"2023-07-06T03:45:13.827748+0000","last_clean":"2023-07-06T03:45:13.827748+0000","last_became_active":"2023-07-06T03:45:10.651012+0000","last_became_peered":"2023-07-06T03:45:10.651012+0000","last_unstale":"2023-07-06T03:45:13.827748+0000","last_undegraded":"2023-07-06T03:45:13.827748+0000","last_fullsized":"2023-07-06T03:45:13.827748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:46.872621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680880+0000","last_change":"2023-07-06T03:45:11.688331+0000","last_active":"2023-07-06T03:45:12.680880+0000","last_peered":"2023-07-06T03:45:12.680880+0000","last_clean":"2023-07-06T03:45:12.680880+0000","last_became_active":"2023-07-06T03:45:11.686894+0000","last_became_peered":"2023-07-06T03:45:11.686894+0000","last_unstale":"2023-07-06T03:45:12.680880+0000","last_undegraded":"2023-07-06T03:45:12.680880+0000","last_fullsized":"2023-07-06T03:45:12.680880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:08.062104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678168+0000","last_change":"2023-07-06T03:45:10.653064+0000","last_active":"2023-07-06T03:45:12.678168+0000","last_peered":"2023-07-06T03:45:12.678168+0000","last_clean":"2023-07-06T03:45:12.678168+0000","last_became_active":"2023-07-06T03:45:10.652765+0000","last_became_peered":"2023-07-06T03:45:10.652765+0000","last_unstale":"2023-07-06T03:45:12.678168+0000","last_undegraded":"2023-07-06T03:45:12.678168+0000","last_fullsized":"2023-07-06T03:45:12.678168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:23.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827686+0000","last_change":"2023-07-06T03:45:11.683336+0000","last_active":"2023-07-06T03:45:13.827686+0000","last_peered":"2023-07-06T03:45:13.827686+0000","last_clean":"2023-07-06T03:45:13.827686+0000","last_became_active":"2023-07-06T03:45:11.683087+0000","last_became_peered":"2023-07-06T03:45:11.683087+0000","last_unstale":"2023-07-06T03:45:13.827686+0000","last_undegraded":"2023-07-06T03:45:13.827686+0000","last_fullsized":"2023-07-06T03:45:13.827686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830000+0000","last_change":"2023-07-06T03:45:10.650363+0000","last_active":"2023-07-06T03:45:13.830000+0000","last_peered":"2023-07-06T03:45:13.830000+0000","last_clean":"2023-07-06T03:45:13.830000+0000","last_became_active":"2023-07-06T03:45:10.650097+0000","last_became_peered":"2023-07-06T03:45:10.650097+0000","last_unstale":"2023-07-06T03:45:13.830000+0000","last_undegraded":"2023-07-06T03:45:13.830000+0000","last_fullsized":"2023-07-06T03:45:13.830000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:05.604877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830398+0000","last_change":"2023-07-06T03:45:11.668010+0000","last_active":"2023-07-06T03:45:13.830398+0000","last_peered":"2023-07-06T03:45:13.830398+0000","last_clean":"2023-07-06T03:45:13.830398+0000","last_became_active":"2023-07-06T03:45:11.667739+0000","last_became_peered":"2023-07-06T03:45:11.667739+0000","last_unstale":"2023-07-06T03:45:13.830398+0000","last_undegraded":"2023-07-06T03:45:13.830398+0000","last_fullsized":"2023-07-06T03:45:13.830398+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:06:53.685059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829747+0000","last_change":"2023-07-06T03:45:10.661165+0000","last_active":"2023-07-06T03:45:13.829747+0000","last_peered":"2023-07-06T03:45:13.829747+0000","last_clean":"2023-07-06T03:45:13.829747+0000","last_became_active":"2023-07-06T03:45:10.660843+0000","last_became_peered":"2023-07-06T03:45:10.660843+0000","last_unstale":"2023-07-06T03:45:13.829747+0000","last_undegraded":"2023-07-06T03:45:13.829747+0000","last_fullsized":"2023-07-06T03:45:13.829747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:02:20.704169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678908+0000","last_change":"2023-07-06T03:45:11.665692+0000","last_active":"2023-07-06T03:45:12.678908+0000","last_peered":"2023-07-06T03:45:12.678908+0000","last_clean":"2023-07-06T03:45:12.678908+0000","last_became_active":"2023-07-06T03:45:11.665425+0000","last_became_peered":"2023-07-06T03:45:11.665425+0000","last_unstale":"2023-07-06T03:45:12.678908+0000","last_undegraded":"2023-07-06T03:45:12.678908+0000","last_fullsized":"2023-07-06T03:45:12.678908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205473+0000","last_change":"2023-07-06T03:45:10.658201+0000","last_active":"2023-07-06T03:45:14.205473+0000","last_peered":"2023-07-06T03:45:14.205473+0000","last_clean":"2023-07-06T03:45:14.205473+0000","last_became_active":"2023-07-06T03:45:10.657688+0000","last_became_peered":"2023-07-06T03:45:10.657688+0000","last_unstale":"2023-07-06T03:45:14.205473+0000","last_undegraded":"2023-07-06T03:45:14.205473+0000","last_fullsized":"2023-07-06T03:45:14.205473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:26.377648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678497+0000","last_change":"2023-07-06T03:45:11.673905+0000","last_active":"2023-07-06T03:45:12.678497+0000","last_peered":"2023-07-06T03:45:12.678497+0000","last_clean":"2023-07-06T03:45:12.678497+0000","last_became_active":"2023-07-06T03:45:11.673609+0000","last_became_peered":"2023-07-06T03:45:11.673609+0000","last_unstale":"2023-07-06T03:45:12.678497+0000","last_undegraded":"2023-07-06T03:45:12.678497+0000","last_fullsized":"2023-07-06T03:45:12.678497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681110+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:45:12.681110+0000","last_peered":"2023-07-06T03:45:12.681110+0000","last_clean":"2023-07-06T03:45:12.681110+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:45:12.681110+0000","last_undegraded":"2023-07-06T03:45:12.681110+0000","last_fullsized":"2023-07-06T03:45:12.681110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:20.874020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678806+0000","last_change":"2023-07-06T03:45:10.660307+0000","last_active":"2023-07-06T03:45:12.678806+0000","last_peered":"2023-07-06T03:45:12.678806+0000","last_clean":"2023-07-06T03:45:12.678806+0000","last_became_active":"2023-07-06T03:45:10.660032+0000","last_became_peered":"2023-07-06T03:45:10.660032+0000","last_unstale":"2023-07-06T03:45:12.678806+0000","last_undegraded":"2023-07-06T03:45:12.678806+0000","last_fullsized":"2023-07-06T03:45:12.678806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652629+0000","last_change":"2023-07-06T03:45:11.679839+0000","last_active":"2023-07-06T03:45:13.652629+0000","last_peered":"2023-07-06T03:45:13.652629+0000","last_clean":"2023-07-06T03:45:13.652629+0000","last_became_active":"2023-07-06T03:45:11.679509+0000","last_became_peered":"2023-07-06T03:45:11.679509+0000","last_unstale":"2023-07-06T03:45:13.652629+0000","last_undegraded":"2023-07-06T03:45:13.652629+0000","last_fullsized":"2023-07-06T03:45:13.652629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830179+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:45:13.830179+0000","last_peered":"2023-07-06T03:45:13.830179+0000","last_clean":"2023-07-06T03:45:13.830179+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:45:13.830179+0000","last_undegraded":"2023-07-06T03:45:13.830179+0000","last_fullsized":"2023-07-06T03:45:13.830179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:03:06.569126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827774+0000","last_change":"2023-07-06T03:45:10.649547+0000","last_active":"2023-07-06T03:45:13.827774+0000","last_peered":"2023-07-06T03:45:13.827774+0000","last_clean":"2023-07-06T03:45:13.827774+0000","last_became_active":"2023-07-06T03:45:10.649231+0000","last_became_peered":"2023-07-06T03:45:10.649231+0000","last_unstale":"2023-07-06T03:45:13.827774+0000","last_undegraded":"2023-07-06T03:45:13.827774+0000","last_fullsized":"2023-07-06T03:45:13.827774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:13.496490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653155+0000","last_change":"2023-07-06T03:45:11.678151+0000","last_active":"2023-07-06T03:45:13.653155+0000","last_peered":"2023-07-06T03:45:13.653155+0000","last_clean":"2023-07-06T03:45:13.653155+0000","last_became_active":"2023-07-06T03:45:11.677841+0000","last_became_peered":"2023-07-06T03:45:11.677841+0000","last_unstale":"2023-07-06T03:45:13.653155+0000","last_undegraded":"2023-07-06T03:45:13.653155+0000","last_fullsized":"2023-07-06T03:45:13.653155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678308+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:45:12.678308+0000","last_peered":"2023-07-06T03:45:12.678308+0000","last_clean":"2023-07-06T03:45:12.678308+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:45:12.678308+0000","last_undegraded":"2023-07-06T03:45:12.678308+0000","last_fullsized":"2023-07-06T03:45:12.678308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:11.498012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680537+0000","last_change":"2023-07-06T03:45:10.647981+0000","last_active":"2023-07-06T03:45:12.680537+0000","last_peered":"2023-07-06T03:45:12.680537+0000","last_clean":"2023-07-06T03:45:12.680537+0000","last_became_active":"2023-07-06T03:45:10.647678+0000","last_became_peered":"2023-07-06T03:45:10.647678+0000","last_unstale":"2023-07-06T03:45:12.680537+0000","last_undegraded":"2023-07-06T03:45:12.680537+0000","last_fullsized":"2023-07-06T03:45:12.680537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653181+0000","last_change":"2023-07-06T03:45:11.683452+0000","last_active":"2023-07-06T03:45:13.653181+0000","last_peered":"2023-07-06T03:45:13.653181+0000","last_clean":"2023-07-06T03:45:13.653181+0000","last_became_active":"2023-07-06T03:45:11.683187+0000","last_became_peered":"2023-07-06T03:45:11.683187+0000","last_unstale":"2023-07-06T03:45:13.653181+0000","last_undegraded":"2023-07-06T03:45:13.653181+0000","last_fullsized":"2023-07-06T03:45:13.653181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:38.118602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829567+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:45:13.829567+0000","last_peered":"2023-07-06T03:45:13.829567+0000","last_clean":"2023-07-06T03:45:13.829567+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:45:13.829567+0000","last_undegraded":"2023-07-06T03:45:13.829567+0000","last_fullsized":"2023-07-06T03:45:13.829567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:49:25.728537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817899+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:45:12.817899+0000","last_peered":"2023-07-06T03:45:12.817899+0000","last_clean":"2023-07-06T03:45:12.817899+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:45:12.817899+0000","last_undegraded":"2023-07-06T03:45:12.817899+0000","last_fullsized":"2023-07-06T03:45:12.817899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:39.120643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205527+0000","last_change":"2023-07-06T03:45:10.657235+0000","last_active":"2023-07-06T03:45:14.205527+0000","last_peered":"2023-07-06T03:45:14.205527+0000","last_clean":"2023-07-06T03:45:14.205527+0000","last_became_active":"2023-07-06T03:45:10.656945+0000","last_became_peered":"2023-07-06T03:45:10.656945+0000","last_unstale":"2023-07-06T03:45:14.205527+0000","last_undegraded":"2023-07-06T03:45:14.205527+0000","last_fullsized":"2023-07-06T03:45:14.205527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:57.080176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830434+0000","last_change":"2023-07-06T03:45:11.668172+0000","last_active":"2023-07-06T03:45:13.830434+0000","last_peered":"2023-07-06T03:45:13.830434+0000","last_clean":"2023-07-06T03:45:13.830434+0000","last_became_active":"2023-07-06T03:45:11.667041+0000","last_became_peered":"2023-07-06T03:45:11.667041+0000","last_unstale":"2023-07-06T03:45:13.830434+0000","last_undegraded":"2023-07-06T03:45:13.830434+0000","last_fullsized":"2023-07-06T03:45:13.830434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:57.552603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829547+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:45:13.829547+0000","last_peered":"2023-07-06T03:45:13.829547+0000","last_clean":"2023-07-06T03:45:13.829547+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:45:13.829547+0000","last_undegraded":"2023-07-06T03:45:13.829547+0000","last_fullsized":"2023-07-06T03:45:13.829547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:25:39.144218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652538+0000","last_change":"2023-07-06T03:45:10.660900+0000","last_active":"2023-07-06T03:45:13.652538+0000","last_peered":"2023-07-06T03:45:13.652538+0000","last_clean":"2023-07-06T03:45:13.652538+0000","last_became_active":"2023-07-06T03:45:10.660490+0000","last_became_peered":"2023-07-06T03:45:10.660490+0000","last_unstale":"2023-07-06T03:45:13.652538+0000","last_undegraded":"2023-07-06T03:45:13.652538+0000","last_fullsized":"2023-07-06T03:45:13.652538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:43.801501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681363+0000","last_change":"2023-07-06T03:45:11.687991+0000","last_active":"2023-07-06T03:45:12.681363+0000","last_peered":"2023-07-06T03:45:12.681363+0000","last_clean":"2023-07-06T03:45:12.681363+0000","last_became_active":"2023-07-06T03:45:11.685096+0000","last_became_peered":"2023-07-06T03:45:11.685096+0000","last_unstale":"2023-07-06T03:45:12.681363+0000","last_undegraded":"2023-07-06T03:45:12.681363+0000","last_fullsized":"2023-07-06T03:45:12.681363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830169+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:45:13.830169+0000","last_peered":"2023-07-06T03:45:13.830169+0000","last_clean":"2023-07-06T03:45:13.830169+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:45:13.830169+0000","last_undegraded":"2023-07-06T03:45:13.830169+0000","last_fullsized":"2023-07-06T03:45:13.830169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:27:55.564390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653143+0000","last_change":"2023-07-06T03:45:10.661812+0000","last_active":"2023-07-06T03:45:13.653143+0000","last_peered":"2023-07-06T03:45:13.653143+0000","last_clean":"2023-07-06T03:45:13.653143+0000","last_became_active":"2023-07-06T03:45:10.661499+0000","last_became_peered":"2023-07-06T03:45:10.661499+0000","last_unstale":"2023-07-06T03:45:13.653143+0000","last_undegraded":"2023-07-06T03:45:13.653143+0000","last_fullsized":"2023-07-06T03:45:13.653143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:59.089238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652878+0000","last_change":"2023-07-06T03:45:11.678171+0000","last_active":"2023-07-06T03:45:13.652878+0000","last_peered":"2023-07-06T03:45:13.652878+0000","last_clean":"2023-07-06T03:45:13.652878+0000","last_became_active":"2023-07-06T03:45:11.677886+0000","last_became_peered":"2023-07-06T03:45:11.677886+0000","last_unstale":"2023-07-06T03:45:13.652878+0000","last_undegraded":"2023-07-06T03:45:13.652878+0000","last_fullsized":"2023-07-06T03:45:13.652878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206208+0000","last_change":"2023-07-06T03:45:10.657429+0000","last_active":"2023-07-06T03:45:14.206208+0000","last_peered":"2023-07-06T03:45:14.206208+0000","last_clean":"2023-07-06T03:45:14.206208+0000","last_became_active":"2023-07-06T03:45:10.657085+0000","last_became_peered":"2023-07-06T03:45:10.657085+0000","last_unstale":"2023-07-06T03:45:14.206208+0000","last_undegraded":"2023-07-06T03:45:14.206208+0000","last_fullsized":"2023-07-06T03:45:14.206208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:24:47.052321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206026+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:45:14.206026+0000","last_peered":"2023-07-06T03:45:14.206026+0000","last_clean":"2023-07-06T03:45:14.206026+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:45:14.206026+0000","last_undegraded":"2023-07-06T03:45:14.206026+0000","last_fullsized":"2023-07-06T03:45:14.206026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:05:14.989733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205502+0000","last_change":"2023-07-06T03:45:11.671780+0000","last_active":"2023-07-06T03:45:14.205502+0000","last_peered":"2023-07-06T03:45:14.205502+0000","last_clean":"2023-07-06T03:45:14.205502+0000","last_became_active":"2023-07-06T03:45:11.671390+0000","last_became_peered":"2023-07-06T03:45:11.671390+0000","last_unstale":"2023-07-06T03:45:14.205502+0000","last_undegraded":"2023-07-06T03:45:14.205502+0000","last_fullsized":"2023-07-06T03:45:14.205502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829963+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:45:13.829963+0000","last_peered":"2023-07-06T03:45:13.829963+0000","last_clean":"2023-07-06T03:45:13.829963+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:45:13.829963+0000","last_undegraded":"2023-07-06T03:45:13.829963+0000","last_fullsized":"2023-07-06T03:45:13.829963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:12.638307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206183+0000","last_change":"2023-07-06T03:45:10.657912+0000","last_active":"2023-07-06T03:45:14.206183+0000","last_peered":"2023-07-06T03:45:14.206183+0000","last_clean":"2023-07-06T03:45:14.206183+0000","last_became_active":"2023-07-06T03:45:10.657615+0000","last_became_peered":"2023-07-06T03:45:10.657615+0000","last_unstale":"2023-07-06T03:45:14.206183+0000","last_undegraded":"2023-07-06T03:45:14.206183+0000","last_fullsized":"2023-07-06T03:45:14.206183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:30:30.877422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678345+0000","last_change":"2023-07-06T03:45:11.665120+0000","last_active":"2023-07-06T03:45:12.678345+0000","last_peered":"2023-07-06T03:45:12.678345+0000","last_clean":"2023-07-06T03:45:12.678345+0000","last_became_active":"2023-07-06T03:45:11.664817+0000","last_became_peered":"2023-07-06T03:45:11.664817+0000","last_unstale":"2023-07-06T03:45:12.678345+0000","last_undegraded":"2023-07-06T03:45:12.678345+0000","last_fullsized":"2023-07-06T03:45:12.678345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:44.808607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205783+0000","last_change":"2023-07-06T03:45:10.658517+0000","last_active":"2023-07-06T03:45:14.205783+0000","last_peered":"2023-07-06T03:45:14.205783+0000","last_clean":"2023-07-06T03:45:14.205783+0000","last_became_active":"2023-07-06T03:45:10.657946+0000","last_became_peered":"2023-07-06T03:45:10.657946+0000","last_unstale":"2023-07-06T03:45:14.205783+0000","last_undegraded":"2023-07-06T03:45:14.205783+0000","last_fullsized":"2023-07-06T03:45:14.205783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:24:42.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205842+0000","last_change":"2023-07-06T03:45:11.683447+0000","last_active":"2023-07-06T03:45:14.205842+0000","last_peered":"2023-07-06T03:45:14.205842+0000","last_clean":"2023-07-06T03:45:14.205842+0000","last_became_active":"2023-07-06T03:45:11.683156+0000","last_became_peered":"2023-07-06T03:45:11.683156+0000","last_unstale":"2023-07-06T03:45:14.205842+0000","last_undegraded":"2023-07-06T03:45:14.205842+0000","last_fullsized":"2023-07-06T03:45:14.205842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818006+0000","last_change":"2023-07-06T03:45:10.649186+0000","last_active":"2023-07-06T03:45:12.818006+0000","last_peered":"2023-07-06T03:45:12.818006+0000","last_clean":"2023-07-06T03:45:12.818006+0000","last_became_active":"2023-07-06T03:45:10.648707+0000","last_became_peered":"2023-07-06T03:45:10.648707+0000","last_unstale":"2023-07-06T03:45:12.818006+0000","last_undegraded":"2023-07-06T03:45:12.818006+0000","last_fullsized":"2023-07-06T03:45:12.818006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:07.054998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678566+0000","last_change":"2023-07-06T03:45:11.690462+0000","last_active":"2023-07-06T03:45:12.678566+0000","last_peered":"2023-07-06T03:45:12.678566+0000","last_clean":"2023-07-06T03:45:12.678566+0000","last_became_active":"2023-07-06T03:45:11.690181+0000","last_became_peered":"2023-07-06T03:45:11.690181+0000","last_unstale":"2023-07-06T03:45:12.678566+0000","last_undegraded":"2023-07-06T03:45:12.678566+0000","last_fullsized":"2023-07-06T03:45:12.678566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830291+0000","last_change":"2023-07-06T03:45:10.653276+0000","last_active":"2023-07-06T03:45:13.830291+0000","last_peered":"2023-07-06T03:45:13.830291+0000","last_clean":"2023-07-06T03:45:13.830291+0000","last_became_active":"2023-07-06T03:45:10.652958+0000","last_became_peered":"2023-07-06T03:45:10.652958+0000","last_unstale":"2023-07-06T03:45:13.830291+0000","last_undegraded":"2023-07-06T03:45:13.830291+0000","last_fullsized":"2023-07-06T03:45:13.830291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:45.669370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678497+0000","last_change":"2023-07-06T03:45:11.684391+0000","last_active":"2023-07-06T03:45:12.678497+0000","last_peered":"2023-07-06T03:45:12.678497+0000","last_clean":"2023-07-06T03:45:12.678497+0000","last_became_active":"2023-07-06T03:45:11.684078+0000","last_became_peered":"2023-07-06T03:45:11.684078+0000","last_unstale":"2023-07-06T03:45:12.678497+0000","last_undegraded":"2023-07-06T03:45:12.678497+0000","last_fullsized":"2023-07-06T03:45:12.678497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681347+0000","last_change":"2023-07-06T03:45:10.645296+0000","last_active":"2023-07-06T03:45:12.681347+0000","last_peered":"2023-07-06T03:45:12.681347+0000","last_clean":"2023-07-06T03:45:12.681347+0000","last_became_active":"2023-07-06T03:45:10.644967+0000","last_became_peered":"2023-07-06T03:45:10.644967+0000","last_unstale":"2023-07-06T03:45:12.681347+0000","last_undegraded":"2023-07-06T03:45:12.681347+0000","last_fullsized":"2023-07-06T03:45:12.681347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:12.862578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678143+0000","last_change":"2023-07-06T03:45:11.683562+0000","last_active":"2023-07-06T03:45:12.678143+0000","last_peered":"2023-07-06T03:45:12.678143+0000","last_clean":"2023-07-06T03:45:12.678143+0000","last_became_active":"2023-07-06T03:45:11.683275+0000","last_became_peered":"2023-07-06T03:45:11.683275+0000","last_unstale":"2023-07-06T03:45:12.678143+0000","last_undegraded":"2023-07-06T03:45:12.678143+0000","last_fullsized":"2023-07-06T03:45:12.678143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:44.808607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681723+0000","last_change":"2023-07-06T03:45:10.654706+0000","last_active":"2023-07-06T03:45:12.681723+0000","last_peered":"2023-07-06T03:45:12.681723+0000","last_clean":"2023-07-06T03:45:12.681723+0000","last_became_active":"2023-07-06T03:45:10.653995+0000","last_became_peered":"2023-07-06T03:45:10.653995+0000","last_unstale":"2023-07-06T03:45:12.681723+0000","last_undegraded":"2023-07-06T03:45:12.681723+0000","last_fullsized":"2023-07-06T03:45:12.681723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:05:20.493361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678265+0000","last_change":"2023-07-06T03:45:11.670138+0000","last_active":"2023-07-06T03:45:12.678265+0000","last_peered":"2023-07-06T03:45:12.678265+0000","last_clean":"2023-07-06T03:45:12.678265+0000","last_became_active":"2023-07-06T03:45:11.669695+0000","last_became_peered":"2023-07-06T03:45:11.669695+0000","last_unstale":"2023-07-06T03:45:12.678265+0000","last_undegraded":"2023-07-06T03:45:12.678265+0000","last_fullsized":"2023-07-06T03:45:12.678265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:22:55.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818277+0000","last_change":"2023-07-06T03:45:11.673123+0000","last_active":"2023-07-06T03:45:12.818277+0000","last_peered":"2023-07-06T03:45:12.818277+0000","last_clean":"2023-07-06T03:45:12.818277+0000","last_became_active":"2023-07-06T03:45:11.672765+0000","last_became_peered":"2023-07-06T03:45:11.672765+0000","last_unstale":"2023-07-06T03:45:12.818277+0000","last_undegraded":"2023-07-06T03:45:12.818277+0000","last_fullsized":"2023-07-06T03:45:12.818277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652635+0000","last_change":"2023-07-06T03:45:11.689866+0000","last_active":"2023-07-06T03:45:13.652635+0000","last_peered":"2023-07-06T03:45:13.652635+0000","last_clean":"2023-07-06T03:45:13.652635+0000","last_became_active":"2023-07-06T03:45:11.689595+0000","last_became_peered":"2023-07-06T03:45:11.689595+0000","last_unstale":"2023-07-06T03:45:13.652635+0000","last_undegraded":"2023-07-06T03:45:13.652635+0000","last_fullsized":"2023-07-06T03:45:13.652635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827567+0000","last_change":"2023-07-06T03:45:11.675336+0000","last_active":"2023-07-06T03:45:13.827567+0000","last_peered":"2023-07-06T03:45:13.827567+0000","last_clean":"2023-07-06T03:45:13.827567+0000","last_became_active":"2023-07-06T03:45:11.675044+0000","last_became_peered":"2023-07-06T03:45:11.675044+0000","last_unstale":"2023-07-06T03:45:13.827567+0000","last_undegraded":"2023-07-06T03:45:13.827567+0000","last_fullsized":"2023-07-06T03:45:13.827567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680990+0000","last_change":"2023-07-06T03:45:11.689419+0000","last_active":"2023-07-06T03:45:12.680990+0000","last_peered":"2023-07-06T03:45:12.680990+0000","last_clean":"2023-07-06T03:45:12.680990+0000","last_became_active":"2023-07-06T03:45:11.688843+0000","last_became_peered":"2023-07-06T03:45:11.688843+0000","last_unstale":"2023-07-06T03:45:12.680990+0000","last_undegraded":"2023-07-06T03:45:12.680990+0000","last_fullsized":"2023-07-06T03:45:12.680990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:15.627811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680936+0000","last_change":"2023-07-06T03:45:11.687716+0000","last_active":"2023-07-06T03:45:12.680936+0000","last_peered":"2023-07-06T03:45:12.680936+0000","last_clean":"2023-07-06T03:45:12.680936+0000","last_became_active":"2023-07-06T03:45:11.685418+0000","last_became_peered":"2023-07-06T03:45:11.685418+0000","last_unstale":"2023-07-06T03:45:12.680936+0000","last_undegraded":"2023-07-06T03:45:12.680936+0000","last_fullsized":"2023-07-06T03:45:12.680936+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":847872,"data_stored":680641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":89745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":91163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:45:21.428 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T03:45:21.428 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-06T03:45:21.429 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-07-06T03:45:21.429 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-07-06T03:45:21.430 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-07-06T03:45:21.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.651+0000 7fc505dff700 1 Processor -- start 2023-07-06T03:45:21.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.651+0000 7fc505dff700 1 -- start start 2023-07-06T03:45:21.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc505dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc50015c520 0x7fc5000cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc505dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc500160d00 0x7fc5000b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc505dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5000cdae0 con 0x7fc500160180 2023-07-06T03:45:21.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc505dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5000cd8d0 con 0x7fc50015c520 2023-07-06T03:45:21.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc505dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5000b6640 con 0x7fc500160d00 2023-07-06T03:45:21.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc4ff7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc50015c520 0x7fc5000cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc4ff7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc50015c520 0x7fc5000cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51366/0 (socket says 172.21.15.97:51366) 2023-07-06T03:45:21.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc4ff7fe700 1 -- 172.21.15.97:0/887045951 learned_addr learned my addr 172.21.15.97:0/887045951 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc4fffff700 1 --2- 172.21.15.97:0/887045951 >> v2:172.21.15.146:3300/0 conn(0x7fc500160d00 0x7fc5000b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc4ff7fe700 1 -- 172.21.15.97:0/887045951 >> v2:172.21.15.146:3300/0 conn(0x7fc500160d00 msgr2=0x7fc5000b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc4ff7fe700 1 --2- 172.21.15.97:0/887045951 >> v2:172.21.15.146:3300/0 conn(0x7fc500160d00 0x7fc5000b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc4ff7fe700 1 -- 172.21.15.97:0/887045951 >> v1:172.21.15.146:6789/0 conn(0x7fc500160180 legacy=0x7fc5001605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.653+0000 7fc4ff7fe700 1 -- 172.21.15.97:0/887045951 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5000b6920 con 0x7fc50015c520 2023-07-06T03:45:21.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.654+0000 7fc4ff7fe700 1 --2- 172.21.15.97:0/887045951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc50015c520 0x7fc5000cd390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc5000cd320 tx=0x7fc4f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a77a05fddaa34a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:21.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.654+0000 7fc4feffd700 1 -- 172.21.15.97:0/887045951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000b870 con 0x7fc50015c520 2023-07-06T03:45:21.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.654+0000 7fc4feffd700 1 -- 172.21.15.97:0/887045951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f000b9d0 con 0x7fc50015c520 2023-07-06T03:45:21.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.654+0000 7fc4feffd700 1 -- 172.21.15.97:0/887045951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000bd60 con 0x7fc50015c520 2023-07-06T03:45:21.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.654+0000 7fc505dff700 1 -- 172.21.15.97:0/887045951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc50015c520 msgr2=0x7fc5000cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:21.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.654+0000 7fc505dff700 1 --2- 172.21.15.97:0/887045951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc50015c520 0x7fc5000cd390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc5000cd320 tx=0x7fc4f0009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.655+0000 7fc505dff700 1 -- 172.21.15.97:0/887045951 shutdown_connections 2023-07-06T03:45:21.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.655+0000 7fc505dff700 1 --2- 172.21.15.97:0/887045951 >> v2:172.21.15.146:3300/0 conn(0x7fc500160d00 0x7fc5000b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.655+0000 7fc505dff700 1 --2- 172.21.15.97:0/887045951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc50015c520 0x7fc5000cd390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.655+0000 7fc505dff700 1 -- 172.21.15.97:0/887045951 >> 172.21.15.97:0/887045951 conn(0x7fc500071790 msgr2=0x7fc500071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.655+0000 7fc505dff700 1 -- 172.21.15.97:0/887045951 shutdown_connections 2023-07-06T03:45:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.655+0000 7fc505dff700 1 -- 172.21.15.97:0/887045951 wait complete. 2023-07-06T03:45:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.656+0000 7fc505dff700 1 Processor -- start 2023-07-06T03:45:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.656+0000 7fc505dff700 1 -- start start 2023-07-06T03:45:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.657+0000 7fc505dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc50015c520 0x7fc50014ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.657+0000 7fc505dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc500160180 0x7fc5001500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.657+0000 7fc505dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc500160d00 0x7fc50015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.657+0000 7fc505dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50015dd60 con 0x7fc500160d00 2023-07-06T03:45:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.657+0000 7fc505dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50014d3f0 con 0x7fc50015c520 2023-07-06T03:45:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.657+0000 7fc505dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5001e5fc0 con 0x7fc500160180 2023-07-06T03:45:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4fffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc500160d00 0x7fc50015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4ff7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc50015c520 0x7fc50014ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4ff7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc50015c520 0x7fc50014ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57484/0 (socket says 172.21.15.97:57484) 2023-07-06T03:45:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4fffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc500160d00 0x7fc50015e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51376/0 (socket says 172.21.15.97:51376) 2023-07-06T03:45:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4ff7fe700 1 -- 172.21.15.97:0/682722434 learned_addr learned my addr 172.21.15.97:0/682722434 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4f7fff700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc500160180 0x7fc5001500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4fffff700 1 -- 172.21.15.97:0/682722434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc500160180 msgr2=0x7fc5001500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4fffff700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc500160180 0x7fc5001500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4fffff700 1 -- 172.21.15.97:0/682722434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc50015c520 msgr2=0x7fc50014ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4fffff700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc50015c520 0x7fc50014ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4fffff700 1 -- 172.21.15.97:0/682722434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f0009550 con 0x7fc500160d00 2023-07-06T03:45:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4ff7fe700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc50015c520 0x7fc50014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4fffff700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc500160d00 0x7fc50015e1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8007dc0 tx=0x7fc4f800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.658+0000 7fc4fcff9700 1 -- 172.21.15.97:0/682722434 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8018070 con 0x7fc500160d00 2023-07-06T03:45:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.659+0000 7fc4fcff9700 1 -- 172.21.15.97:0/682722434 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f800eb30 con 0x7fc500160d00 2023-07-06T03:45:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.659+0000 7fc505dff700 1 -- 172.21.15.97:0/682722434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5001e62a0 con 0x7fc500160d00 2023-07-06T03:45:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.659+0000 7fc4fcff9700 1 -- 172.21.15.97:0/682722434 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f800eec0 con 0x7fc500160d00 2023-07-06T03:45:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.659+0000 7fc505dff700 1 -- 172.21.15.97:0/682722434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50014cb40 con 0x7fc500160d00 2023-07-06T03:45:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.660+0000 7fc505dff700 1 -- 172.21.15.97:0/682722434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50004e340 con 0x7fc500160d00 2023-07-06T03:45:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.660+0000 7fc4fcff9700 1 -- 172.21.15.97:0/682722434 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc4f8003a20 con 0x7fc500160d00 2023-07-06T03:45:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.661+0000 7fc4fcff9700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4e006ace0 0x7fc4e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.662+0000 7fc4fcff9700 1 -- 172.21.15.97:0/682722434 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4f80893c0 con 0x7fc500160d00 2023-07-06T03:45:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.662+0000 7fc4ff7fe700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4e006ace0 0x7fc4e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.662+0000 7fc4ff7fe700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4e006ace0 0x7fc4e006d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009c10 tx=0x7fc4f0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.666+0000 7fc4fcff9700 1 -- 172.21.15.97:0/682722434 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc4f8057ef0 con 0x7fc500160d00 2023-07-06T03:45:21.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.914+0000 7fc505dff700 1 -- 172.21.15.97:0/682722434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc500071e10 con 0x7fc500160d00 2023-07-06T03:45:21.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.918+0000 7fc4fcff9700 1 -- 172.21.15.97:0/682722434 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170841 (secure 0 0 0) 0x7fc4f8009950 con 0x7fc500160d00 2023-07-06T03:45:21.922 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 -- 172.21.15.97:0/682722434 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4e006ace0 msgr2=0x7fc4e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4e006ace0 0x7fc4e006d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009c10 tx=0x7fc4f0008030 comp rx=0 tx=0).stop 2023-07-06T03:45:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 -- 172.21.15.97:0/682722434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc500160d00 msgr2=0x7fc50015e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc500160d00 0x7fc50015e1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8007dc0 tx=0x7fc4f800e3c0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 -- 172.21.15.97:0/682722434 shutdown_connections 2023-07-06T03:45:21.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4e006ace0 0x7fc4e006d190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc500160d00 0x7fc50015e1b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc50015c520 0x7fc50014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 --2- 172.21.15.97:0/682722434 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc500160180 0x7fc5001500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:21.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 -- 172.21.15.97:0/682722434 >> 172.21.15.97:0/682722434 conn(0x7fc500071790 msgr2=0x7fc500156bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:21.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 -- 172.21.15.97:0/682722434 shutdown_connections 2023-07-06T03:45:21.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:21.921+0000 7fc505dff700 1 -- 172.21.15.97:0/682722434 wait complete. 2023-07-06T03:45:21.940 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1323531702},{"type":"v1","addr":"172.21.15.97:6833","nonce":1323531702}]},"active_addr":"172.21.15.97:6833/1323531702","active_change":"2023-07-06T03:44:50.867647+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2265140405}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":302368923}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2697636909}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3550144650}]}]} 2023-07-06T03:45:21.945 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-07-06T03:45:21.945 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-07-06T03:45:21.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.138+0000 7f80285ff700 1 Processor -- start 2023-07-06T03:45:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.138+0000 7f80285ff700 1 -- start start 2023-07-06T03:45:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.139+0000 7f80285ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80200cd390 0x7f8020160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.139+0000 7f80285ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8020160dd0 0x7f80200b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.139+0000 7f80285ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80200cdae0 con 0x7f802015c520 2023-07-06T03:45:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.139+0000 7f80285ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80200b6640 con 0x7f80200cd390 2023-07-06T03:45:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.139+0000 7f80285ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80200b6920 con 0x7f8020160dd0 2023-07-06T03:45:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8025b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80200cd390 0x7f8020160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8025b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80200cd390 0x7f8020160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51384/0 (socket says 172.21.15.97:51384) 2023-07-06T03:45:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8025b9a700 1 -- 172.21.15.97:0/3299817214 learned_addr learned my addr 172.21.15.97:0/3299817214 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f802639b700 1 --1- 172.21.15.97:0/3299817214 >> v1:172.21.15.146:6789/0 conn(0x7f802015c520 0x7f8020160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:35260/0 (socket says 172.21.15.97:35260) 2023-07-06T03:45:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8026b9c700 1 --2- 172.21.15.97:0/3299817214 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8020160dd0 0x7f80200b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8025b9a700 1 -- 172.21.15.97:0/3299817214 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8020160dd0 msgr2=0x7f80200b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8025b9a700 1 --2- 172.21.15.97:0/3299817214 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8020160dd0 0x7f80200b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8025b9a700 1 -- 172.21.15.97:0/3299817214 >> v1:172.21.15.146:6789/0 conn(0x7f802015c520 legacy=0x7f8020160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8025b9a700 1 -- 172.21.15.97:0/3299817214 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80200b6c00 con 0x7f80200cd390 2023-07-06T03:45:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8025b9a700 1 --2- 172.21.15.97:0/3299817214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80200cd390 0x7f8020160890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f80100098f0 tx=0x7f8010009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a43f83effe50b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8025399700 1 -- 172.21.15.97:0/3299817214 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000b8a0 con 0x7f80200cd390 2023-07-06T03:45:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8025399700 1 -- 172.21.15.97:0/3299817214 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801000ba00 con 0x7f80200cd390 2023-07-06T03:45:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.140+0000 7f8025399700 1 -- 172.21.15.97:0/3299817214 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000bd90 con 0x7f80200cd390 2023-07-06T03:45:22.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.141+0000 7f80285ff700 1 -- 172.21.15.97:0/3299817214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80200cd390 msgr2=0x7f8020160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.141+0000 7f80285ff700 1 --2- 172.21.15.97:0/3299817214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80200cd390 0x7f8020160890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f80100098f0 tx=0x7f8010009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:22.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.141+0000 7f80285ff700 1 -- 172.21.15.97:0/3299817214 shutdown_connections 2023-07-06T03:45:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.141+0000 7f80285ff700 1 --2- 172.21.15.97:0/3299817214 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8020160dd0 0x7f80200b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.141+0000 7f80285ff700 1 --2- 172.21.15.97:0/3299817214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80200cd390 0x7f8020160890 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.141+0000 7f80285ff700 1 -- 172.21.15.97:0/3299817214 >> 172.21.15.97:0/3299817214 conn(0x7f8020071790 msgr2=0x7f8020071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.141+0000 7f80285ff700 1 -- 172.21.15.97:0/3299817214 shutdown_connections 2023-07-06T03:45:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.141+0000 7f80285ff700 1 -- 172.21.15.97:0/3299817214 wait complete. 2023-07-06T03:45:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.143+0000 7f80285ff700 1 Processor -- start 2023-07-06T03:45:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.143+0000 7f80285ff700 1 -- start start 2023-07-06T03:45:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f80285ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80200cd390 0x7f802015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f80285ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f802015c520 0x7f80200d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f80285ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8020160dd0 0x7f802014e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f80285ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802014e100 con 0x7f802015c520 2023-07-06T03:45:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f80285ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80200d2200 con 0x7f8020160dd0 2023-07-06T03:45:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f80285ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802014ead0 con 0x7f80200cd390 2023-07-06T03:45:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f8025b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f802015c520 0x7f80200d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:22.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f8025b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f802015c520 0x7f80200d1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51390/0 (socket says 172.21.15.97:51390) 2023-07-06T03:45:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f8025b9a700 1 -- 172.21.15.97:0/2593003182 learned_addr learned my addr 172.21.15.97:0/2593003182 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f8025b9a700 1 -- 172.21.15.97:0/2593003182 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80200cd390 msgr2=0x7f802015f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f8025b9a700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80200cd390 0x7f802015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f8025b9a700 1 -- 172.21.15.97:0/2593003182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8020160dd0 msgr2=0x7f802014e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f8025b9a700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8020160dd0 0x7f802014e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.144+0000 7f8025b9a700 1 -- 172.21.15.97:0/2593003182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8010009550 con 0x7f802015c520 2023-07-06T03:45:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.145+0000 7f8025b9a700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f802015c520 0x7f80200d1cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8010000c00 tx=0x7f8010004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.145+0000 7f8016ffd700 1 -- 172.21.15.97:0/2593003182 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010004480 con 0x7f802015c520 2023-07-06T03:45:22.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.145+0000 7f8016ffd700 1 -- 172.21.15.97:0/2593003182 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801001f070 con 0x7f802015c520 2023-07-06T03:45:22.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.145+0000 7f80285ff700 1 -- 172.21.15.97:0/2593003182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802014daf0 con 0x7f802015c520 2023-07-06T03:45:22.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.145+0000 7f8016ffd700 1 -- 172.21.15.97:0/2593003182 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80100038c0 con 0x7f802015c520 2023-07-06T03:45:22.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.145+0000 7f80285ff700 1 -- 172.21.15.97:0/2593003182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802014dd90 con 0x7f802015c520 2023-07-06T03:45:22.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.146+0000 7f80285ff700 1 -- 172.21.15.97:0/2593003182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802004e340 con 0x7f802015c520 2023-07-06T03:45:22.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.147+0000 7f8016ffd700 1 -- 172.21.15.97:0/2593003182 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8010021030 con 0x7f802015c520 2023-07-06T03:45:22.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.148+0000 7f8016ffd700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f800406ace0 0x7f800406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.149+0000 7f8016ffd700 1 -- 172.21.15.97:0/2593003182 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f801008aa20 con 0x7f802015c520 2023-07-06T03:45:22.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.149+0000 7f802639b700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f800406ace0 0x7f800406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:22.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.149+0000 7f802639b700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f800406ace0 0x7f800406d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f801c001660 tx=0x7f801c007a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:22.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.153+0000 7f8016ffd700 1 -- 172.21.15.97:0/2593003182 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8010055be0 con 0x7f802015c520 2023-07-06T03:45:22.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.352+0000 7f80285ff700 1 -- 172.21.15.97:0/2593003182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f802014e240 con 0x7f802015c520 2023-07-06T03:45:22.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.353+0000 7f8016ffd700 1 -- 172.21.15.97:0/2593003182 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f802014e240 con 0x7f802015c520 2023-07-06T03:45:22.355 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:22.355 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.355+0000 7f80285ff700 1 -- 172.21.15.97:0/2593003182 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f800406ace0 msgr2=0x7f800406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.355+0000 7f80285ff700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f800406ace0 0x7f800406d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f801c001660 tx=0x7f801c007a50 comp rx=0 tx=0).stop 2023-07-06T03:45:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.355+0000 7f80285ff700 1 -- 172.21.15.97:0/2593003182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f802015c520 msgr2=0x7f80200d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.355+0000 7f80285ff700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f802015c520 0x7f80200d1cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8010000c00 tx=0x7f8010004750 comp rx=0 tx=0).stop 2023-07-06T03:45:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.356+0000 7f80285ff700 1 -- 172.21.15.97:0/2593003182 shutdown_connections 2023-07-06T03:45:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.356+0000 7f80285ff700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f800406ace0 0x7f800406d190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.356+0000 7f80285ff700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8020160dd0 0x7f802014e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.356+0000 7f80285ff700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80200cd390 0x7f802015f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.356+0000 7f80285ff700 1 --2- 172.21.15.97:0/2593003182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f802015c520 0x7f80200d1cc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.356+0000 7f80285ff700 1 -- 172.21.15.97:0/2593003182 >> 172.21.15.97:0/2593003182 conn(0x7f8020071790 msgr2=0x7f80200b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:22.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.356+0000 7f80285ff700 1 -- 172.21.15.97:0/2593003182 shutdown_connections 2023-07-06T03:45:22.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.356+0000 7f80285ff700 1 -- 172.21.15.97:0/2593003182 wait complete. 2023-07-06T03:45:22.376 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-07-06T03:45:22.377 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:22.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.568+0000 7f7b0f7ff700 1 Processor -- start 2023-07-06T03:45:22.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.568+0000 7f7b0f7ff700 1 -- start start 2023-07-06T03:45:22.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.569+0000 7f7b0f7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b08158f20 0x7f7b080caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.569+0000 7f7b0f7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b081592f0 0x7f7b080b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.569+0000 7f7b0f7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0815cc40 con 0x7f7b0806d860 2023-07-06T03:45:22.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.569+0000 7f7b0f7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0815cf20 con 0x7f7b08158f20 2023-07-06T03:45:22.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.569+0000 7f7b0f7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0815d200 con 0x7f7b081592f0 2023-07-06T03:45:22.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.570+0000 7f7b0d59b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b08158f20 0x7f7b080caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:22.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.570+0000 7f7b0d59b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b08158f20 0x7f7b080caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51392/0 (socket says 172.21.15.97:51392) 2023-07-06T03:45:22.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.570+0000 7f7b0d59b700 1 -- 172.21.15.97:0/4006502877 learned_addr learned my addr 172.21.15.97:0/4006502877 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:22.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.570+0000 7f7b0cd9a700 1 --2- 172.21.15.97:0/4006502877 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b081592f0 0x7f7b080b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:22.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.570+0000 7f7b0d59b700 1 -- 172.21.15.97:0/4006502877 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b081592f0 msgr2=0x7f7b080b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:22.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.570+0000 7f7b0d59b700 1 --2- 172.21.15.97:0/4006502877 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b081592f0 0x7f7b080b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.570+0000 7f7b0d59b700 1 -- 172.21.15.97:0/4006502877 >> v1:172.21.15.146:6789/0 conn(0x7f7b0806d860 legacy=0x7f7b080cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.570+0000 7f7b0d59b700 1 -- 172.21.15.97:0/4006502877 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0815d4e0 con 0x7f7b08158f20 2023-07-06T03:45:22.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.570+0000 7f7b0d59b700 1 --2- 172.21.15.97:0/4006502877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b08158f20 0x7f7b080caac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400bb90 tx=0x7f7b0400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7e0682a3c5685f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:22.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.570+0000 7f7afffff700 1 -- 172.21.15.97:0/4006502877 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400dd50 con 0x7f7b08158f20 2023-07-06T03:45:22.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.571+0000 7f7afffff700 1 -- 172.21.15.97:0/4006502877 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b04010470 con 0x7f7b08158f20 2023-07-06T03:45:22.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.571+0000 7f7afffff700 1 -- 172.21.15.97:0/4006502877 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04010800 con 0x7f7b08158f20 2023-07-06T03:45:22.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.571+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/4006502877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b08158f20 msgr2=0x7f7b080caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.571+0000 7f7b0f7ff700 1 --2- 172.21.15.97:0/4006502877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b08158f20 0x7f7b080caac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400bb90 tx=0x7f7b0400bea0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.571+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/4006502877 shutdown_connections 2023-07-06T03:45:22.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.571+0000 7f7b0f7ff700 1 --2- 172.21.15.97:0/4006502877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b08158f20 0x7f7b080caac0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.571+0000 7f7b0f7ff700 1 --2- 172.21.15.97:0/4006502877 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b081592f0 0x7f7b080b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.571+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/4006502877 >> 172.21.15.97:0/4006502877 conn(0x7f7b080ce8c0 msgr2=0x7f7b080cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:22.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.571+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/4006502877 shutdown_connections 2023-07-06T03:45:22.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.571+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/4006502877 wait complete. 2023-07-06T03:45:22.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.573+0000 7f7b0f7ff700 1 Processor -- start 2023-07-06T03:45:22.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.573+0000 7f7b0f7ff700 1 -- start start 2023-07-06T03:45:22.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.574+0000 7f7b0f7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b0806d860 0x7f7b08139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.574+0000 7f7b0f7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b08158f20 0x7f7b0813cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.574+0000 7f7b0f7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b081592f0 0x7f7b080b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.574+0000 7f7b0f7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080b4890 con 0x7f7b081592f0 2023-07-06T03:45:22.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.574+0000 7f7b0f7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0813d470 con 0x7f7b08158f20 2023-07-06T03:45:22.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.574+0000 7f7b0f7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080b3c80 con 0x7f7b0806d860 2023-07-06T03:45:22.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.574+0000 7f7b0dd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b081592f0 0x7f7b080b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:22.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.574+0000 7f7b0dd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b081592f0 0x7f7b080b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51408/0 (socket says 172.21.15.97:51408) 2023-07-06T03:45:22.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.574+0000 7f7b0dd9c700 1 -- 172.21.15.97:0/149391127 learned_addr learned my addr 172.21.15.97:0/149391127 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.574+0000 7f7b0d59b700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b0806d860 0x7f7b08139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.574+0000 7f7b0cd9a700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b08158f20 0x7f7b0813cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.575+0000 7f7b0dd9c700 1 -- 172.21.15.97:0/149391127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b0806d860 msgr2=0x7f7b08139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.575+0000 7f7b0dd9c700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b0806d860 0x7f7b08139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.575+0000 7f7b0dd9c700 1 -- 172.21.15.97:0/149391127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b08158f20 msgr2=0x7f7b0813cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.575+0000 7f7b0dd9c700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b08158f20 0x7f7b0813cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.575+0000 7f7b0dd9c700 1 -- 172.21.15.97:0/149391127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0400b7f0 con 0x7f7b081592f0 2023-07-06T03:45:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.575+0000 7f7b0dd9c700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b081592f0 0x7f7b080b18d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000cc50 tx=0x7f7b0000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.575+0000 7f7afdffb700 1 -- 172.21.15.97:0/149391127 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00003c00 con 0x7f7b081592f0 2023-07-06T03:45:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.575+0000 7f7afdffb700 1 -- 172.21.15.97:0/149391127 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b00003d60 con 0x7f7b081592f0 2023-07-06T03:45:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.575+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/149391127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b080b3f60 con 0x7f7b081592f0 2023-07-06T03:45:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.575+0000 7f7afdffb700 1 -- 172.21.15.97:0/149391127 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000aab0 con 0x7f7b081592f0 2023-07-06T03:45:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.575+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/149391127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b080b4520 con 0x7f7b081592f0 2023-07-06T03:45:22.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.576+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/149391127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b080caf80 con 0x7f7b081592f0 2023-07-06T03:45:22.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.577+0000 7f7afdffb700 1 -- 172.21.15.97:0/149391127 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b0000ac10 con 0x7f7b081592f0 2023-07-06T03:45:22.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.578+0000 7f7afdffb700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 0x7f7aec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:22.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.578+0000 7f7afdffb700 1 -- 172.21.15.97:0/149391127 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b000894b0 con 0x7f7b081592f0 2023-07-06T03:45:22.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.578+0000 7f7b0d59b700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 0x7f7aec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:22.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.579+0000 7f7b0d59b700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 0x7f7aec06d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400bee0 tx=0x7f7b04013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:22.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.582+0000 7f7afdffb700 1 -- 172.21.15.97:0/149391127 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7b000544c0 con 0x7f7b081592f0 2023-07-06T03:45:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.781+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/149391127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b0804e340 con 0x7f7b081592f0 2023-07-06T03:45:22.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.782+0000 7f7afdffb700 1 -- 172.21.15.97:0/149391127 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f7b00057ae0 con 0x7f7b081592f0 2023-07-06T03:45:22.784 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:22.785 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/149391127 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 msgr2=0x7f7aec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 0x7f7aec06d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400bee0 tx=0x7f7b04013000 comp rx=0 tx=0).stop 2023-07-06T03:45:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/149391127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b081592f0 msgr2=0x7f7b080b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b081592f0 0x7f7b080b18d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000cc50 tx=0x7f7b0000cf60 comp rx=0 tx=0).stop 2023-07-06T03:45:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/149391127 shutdown_connections 2023-07-06T03:45:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 0x7f7aec06d190 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b081592f0 0x7f7b080b18d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b0806d860 0x7f7b08139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 --2- 172.21.15.97:0/149391127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b08158f20 0x7f7b0813cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:22.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/149391127 >> 172.21.15.97:0/149391127 conn(0x7f7b080ce8c0 msgr2=0x7f7b080aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:22.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/149391127 shutdown_connections 2023-07-06T03:45:22.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:22.785+0000 7f7b0f7ff700 1 -- 172.21.15.97:0/149391127 wait complete. 2023-07-06T03:45:22.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-07-06T03:45:22.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-07-06T03:45:22.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-07-06T03:45:22.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-07-06T03:45:22.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-07-06T03:45:22.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-07-06T03:45:22.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-07-06T03:45:22.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-07-06T03:45:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.156+0000 7f67921ff700 1 Processor -- start 2023-07-06T03:45:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.157+0000 7f67921ff700 1 -- start start 2023-07-06T03:45:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.159+0000 7f67921ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f678c160a20 0x7f678c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.159+0000 7f67921ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f678c0ccbd0 0x7f678c0c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.159+0000 7f67921ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c0caa30 con 0x7f678c0ce3c0 2023-07-06T03:45:23.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.159+0000 7f67921ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c160df0 con 0x7f678c160a20 2023-07-06T03:45:23.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.159+0000 7f67921ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0cac60 con 0x7f678c0ccbd0 2023-07-06T03:45:23.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.160+0000 7f678bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f678c0ccbd0 0x7f678c0c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.160+0000 7f678bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f678c0ccbd0 0x7f678c0c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:53856/0 (socket says 172.21.15.97:53856) 2023-07-06T03:45:23.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.160+0000 7f678bfff700 1 -- 172.21.15.97:0/1416003906 learned_addr learned my addr 172.21.15.97:0/1416003906 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.162+0000 7f678bfff700 1 -- 172.21.15.97:0/1416003906 >> v2:172.21.15.97:3300/0 conn(0x7f678c160a20 msgr2=0x7f678c0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.162+0000 7f678bfff700 1 --2- 172.21.15.97:0/1416003906 >> v2:172.21.15.97:3300/0 conn(0x7f678c160a20 0x7f678c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.162+0000 7f678bfff700 1 -- 172.21.15.97:0/1416003906 >> v1:172.21.15.97:6789/0 conn(0x7f678c0ce3c0 legacy=0x7f678c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.162+0000 7f678bfff700 1 -- 172.21.15.97:0/1416003906 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c0cb1e0 con 0x7f678c0ccbd0 2023-07-06T03:45:23.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.162+0000 7f678bfff700 1 --2- 172.21.15.97:0/1416003906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f678c0ccbd0 0x7f678c0c4250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f67840098f0 tx=0x7f6784009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fbe92e43242cc5c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.163+0000 7f678a7fc700 1 -- 172.21.15.97:0/1416003906 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784004230 con 0x7f678c0ccbd0 2023-07-06T03:45:23.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.163+0000 7f678a7fc700 1 -- 172.21.15.97:0/1416003906 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6784004390 con 0x7f678c0ccbd0 2023-07-06T03:45:23.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.163+0000 7f678a7fc700 1 -- 172.21.15.97:0/1416003906 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840036a0 con 0x7f678c0ccbd0 2023-07-06T03:45:23.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.162+0000 7f9708dff700 1 Processor -- start 2023-07-06T03:45:23.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.163+0000 7f9708dff700 1 -- start start 2023-07-06T03:45:23.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.165+0000 7f9708dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f97040c8f40 0x7f97040ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.165+0000 7f9708dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97040caa30 con 0x7f97040ccbd0 2023-07-06T03:45:23.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.165+0000 7f9708dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9704160df0 con 0x7f9704160a20 2023-07-06T03:45:23.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.165+0000 7f9708dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97040c9340 con 0x7f97040c8f40 2023-07-06T03:45:23.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.166+0000 7f67921ff700 1 -- 172.21.15.97:0/1416003906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f678c0ccbd0 msgr2=0x7f678c0c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.166+0000 7f67921ff700 1 --2- 172.21.15.97:0/1416003906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f678c0ccbd0 0x7f678c0c4250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f67840098f0 tx=0x7f6784009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.166+0000 7f970259c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9704160a20 0x7f97040cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:35280/0 (socket says 172.21.15.97:35280) 2023-07-06T03:45:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.166+0000 7f970259c700 1 -- 172.21.15.97:0/4277607937 learned_addr learned my addr 172.21.15.97:0/4277607937 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.167+0000 7f9702d9d700 1 --2- 172.21.15.97:0/4277607937 >> v2:172.21.15.97:3300/0 conn(0x7f97040c8f40 0x7f97040ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.167+0000 7f970159a700 1 -- 172.21.15.97:0/4277607937 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964942380 0 0) 0x7f9704160df0 con 0x7f9704160a20 2023-07-06T03:45:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.167+0000 7f970159a700 1 -- 172.21.15.97:0/4277607937 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f00035b0 con 0x7f9704160a20 2023-07-06T03:45:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.167+0000 7f9702d9d700 1 -- 172.21.15.97:0/4277607937 >> v1:172.21.15.146:6789/0 conn(0x7f9704160a20 legacy=0x7f97040cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.167+0000 7f9702d9d700 1 -- 172.21.15.97:0/4277607937 >> v1:172.21.15.97:6789/0 conn(0x7f97040ccbd0 legacy=0x7f97040c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.167+0000 7f9702d9d700 1 -- 172.21.15.97:0/4277607937 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97040cac60 con 0x7f97040c8f40 2023-07-06T03:45:23.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.168+0000 7f9702d9d700 1 --2- 172.21.15.97:0/4277607937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97040c8f40 0x7f97040ce3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009d90 tx=0x7f96fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1dad55464aaeef server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.168+0000 7f970159a700 1 -- 172.21.15.97:0/4277607937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00e070 con 0x7f97040c8f40 2023-07-06T03:45:23.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.169+0000 7f970159a700 1 -- 172.21.15.97:0/4277607937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96fc00bb20 con 0x7f97040c8f40 2023-07-06T03:45:23.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.169+0000 7f970159a700 1 -- 172.21.15.97:0/4277607937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc012430 con 0x7f97040c8f40 2023-07-06T03:45:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.169+0000 7f67921ff700 1 -- 172.21.15.97:0/1416003906 shutdown_connections 2023-07-06T03:45:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.169+0000 7f67921ff700 1 --2- 172.21.15.97:0/1416003906 >> v2:172.21.15.97:3300/0 conn(0x7f678c160a20 0x7f678c0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.169+0000 7f67921ff700 1 --2- 172.21.15.97:0/1416003906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f678c0ccbd0 0x7f678c0c4250 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.169+0000 7f67921ff700 1 -- 172.21.15.97:0/1416003906 >> 172.21.15.97:0/1416003906 conn(0x7f678c0cf610 msgr2=0x7f678c0cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.169+0000 7f9708dff700 1 -- 172.21.15.97:0/4277607937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97040c8f40 msgr2=0x7f97040ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.169+0000 7f9708dff700 1 --2- 172.21.15.97:0/4277607937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97040c8f40 0x7f97040ce3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009d90 tx=0x7f96fc00b330 comp rx=0 tx=0).stop 2023-07-06T03:45:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.170+0000 7f67921ff700 1 -- 172.21.15.97:0/1416003906 shutdown_connections 2023-07-06T03:45:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.170+0000 7f67921ff700 1 -- 172.21.15.97:0/1416003906 wait complete. 2023-07-06T03:45:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.170+0000 7f9708dff700 1 -- 172.21.15.97:0/4277607937 shutdown_connections 2023-07-06T03:45:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.170+0000 7f9708dff700 1 --2- 172.21.15.97:0/4277607937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97040c8f40 0x7f97040ce3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.170+0000 7f9708dff700 1 -- 172.21.15.97:0/4277607937 >> 172.21.15.97:0/4277607937 conn(0x7f97040cf610 msgr2=0x7f97040cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.170+0000 7f9708dff700 1 -- 172.21.15.97:0/4277607937 shutdown_connections 2023-07-06T03:45:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.170+0000 7f9708dff700 1 -- 172.21.15.97:0/4277607937 wait complete. 2023-07-06T03:45:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.172+0000 7f82695ff700 1 Processor -- start 2023-07-06T03:45:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.173+0000 7f9708dff700 1 Processor -- start 2023-07-06T03:45:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.172+0000 7f67921ff700 1 Processor -- start 2023-07-06T03:45:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.173+0000 7f67921ff700 1 -- start start 2023-07-06T03:45:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.174+0000 7f67921ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f678c0ccbd0 0x7f678c15f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.174+0000 7f67921ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c0ce3c0 0x7f678c0c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.174+0000 7f67921ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f678c160a20 0x7f678c163260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.174+0000 7f67921ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0aff60 con 0x7f678c0ce3c0 2023-07-06T03:45:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.174+0000 7f67921ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0af930 con 0x7f678c0ccbd0 2023-07-06T03:45:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.174+0000 7f67921ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0c8760 con 0x7f678c160a20 2023-07-06T03:45:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.172+0000 7f82695ff700 1 -- start start 2023-07-06T03:45:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.174+0000 7f82695ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f826415c610 0x7f82640b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.174+0000 7f82695ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82640d2210 0x7f82640d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.174+0000 7f82695ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82640cff90 con 0x7f826406b360 2023-07-06T03:45:23.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.174+0000 7f82695ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82640d0780 con 0x7f826415c610 2023-07-06T03:45:23.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.174+0000 7f82695ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826415c9e0 con 0x7f82640d2210 2023-07-06T03:45:23.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.175+0000 7f678b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f678c0ccbd0 0x7f678c15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.175+0000 7f678b7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f678c0ccbd0 0x7f678c15f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57510/0 (socket says 172.21.15.97:57510) 2023-07-06T03:45:23.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.175+0000 7f678b7fe700 1 -- 172.21.15.97:0/239111363 learned_addr learned my addr 172.21.15.97:0/239111363 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.175+0000 7f82637fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82640d2210 0x7f82640d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.175+0000 7f82627fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f826406b360 0x7f82640b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35432/0 (socket says 172.21.15.97:35432) 2023-07-06T03:45:23.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.175+0000 7f82637fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82640d2210 0x7f82640d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:53878/0 (socket says 172.21.15.97:53878) 2023-07-06T03:45:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.175+0000 7f82627fc700 1 -- 172.21.15.97:0/1312606248 learned_addr learned my addr 172.21.15.97:0/1312606248 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.175+0000 7f8262ffd700 1 --2- 172.21.15.97:0/1312606248 >> v2:172.21.15.97:3300/0 conn(0x7f826415c610 0x7f82640b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.175+0000 7f678affd700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c0ce3c0 0x7f678c0c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.176+0000 7f678bfff700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f678c160a20 0x7f678c163260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.176+0000 7f82637fe700 1 -- 172.21.15.97:0/1312606248 >> v2:172.21.15.97:3300/0 conn(0x7f826415c610 msgr2=0x7f82640b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.176+0000 7f82637fe700 1 --2- 172.21.15.97:0/1312606248 >> v2:172.21.15.97:3300/0 conn(0x7f826415c610 0x7f82640b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.176+0000 7f82637fe700 1 -- 172.21.15.97:0/1312606248 >> v1:172.21.15.97:6789/0 conn(0x7f826406b360 legacy=0x7f82640b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.176+0000 7f82637fe700 1 -- 172.21.15.97:0/1312606248 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640b5650 con 0x7f82640d2210 2023-07-06T03:45:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.176+0000 7f678b7fe700 1 -- 172.21.15.97:0/239111363 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f678c160a20 msgr2=0x7f678c163260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.176+0000 7f678b7fe700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f678c160a20 0x7f678c163260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.176+0000 7f678b7fe700 1 -- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c0ce3c0 msgr2=0x7f678c0c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.176+0000 7f678b7fe700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c0ce3c0 0x7f678c0c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.176+0000 7f678b7fe700 1 -- 172.21.15.97:0/239111363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6784009550 con 0x7f678c0ccbd0 2023-07-06T03:45:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.177+0000 7f82637fe700 1 --2- 172.21.15.97:0/1312606248 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82640d2210 0x7f82640d1310 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8258009840 tx=0x7f8258009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a6f9bd1e109294 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.177+0000 7f678b7fe700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f678c0ccbd0 0x7f678c15f410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f677c00b530 tx=0x7f677c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.177+0000 7f9708dff700 1 -- start start 2023-07-06T03:45:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.177+0000 7f04da5ff700 1 Processor -- start 2023-07-06T03:45:23.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f8261ffb700 1 -- 172.21.15.97:0/1312606248 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258011830 con 0x7f82640d2210 2023-07-06T03:45:23.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f8261ffb700 1 -- 172.21.15.97:0/1312606248 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8258011990 con 0x7f82640d2210 2023-07-06T03:45:23.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f8261ffb700 1 -- 172.21.15.97:0/1312606248 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258011d20 con 0x7f82640d2210 2023-07-06T03:45:23.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7fb9e83ff700 1 Processor -- start 2023-07-06T03:45:23.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f04da5ff700 1 -- start start 2023-07-06T03:45:23.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f9708dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97040c8f40 0x7f97040b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f9708dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97040ccbd0 0x7f97040b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f9708dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9704160a20 0x7f97040b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f9708dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97040b0200 con 0x7f97040c8f40 2023-07-06T03:45:23.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f9708dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97040af660 con 0x7f9704160a20 2023-07-06T03:45:23.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f9708dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97040b1e10 con 0x7f97040ccbd0 2023-07-06T03:45:23.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.179+0000 7f82695ff700 1 -- 172.21.15.97:0/1312606248 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82640d2210 msgr2=0x7f82640d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.179+0000 7f82695ff700 1 --2- 172.21.15.97:0/1312606248 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82640d2210 0x7f82640d1310 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8258009840 tx=0x7f8258009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:23.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.179+0000 7f82695ff700 1 -- 172.21.15.97:0/1312606248 shutdown_connections 2023-07-06T03:45:23.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.179+0000 7f82695ff700 1 --2- 172.21.15.97:0/1312606248 >> v2:172.21.15.97:3300/0 conn(0x7f826415c610 0x7f82640b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.179+0000 7f82695ff700 1 --2- 172.21.15.97:0/1312606248 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82640d2210 0x7f82640d1310 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.179+0000 7f82695ff700 1 -- 172.21.15.97:0/1312606248 >> 172.21.15.97:0/1312606248 conn(0x7f826406a6c0 msgr2=0x7f826406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f6783fff700 1 -- 172.21.15.97:0/239111363 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c0118f0 con 0x7f678c0ccbd0 2023-07-06T03:45:23.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f6783fff700 1 -- 172.21.15.97:0/239111363 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c011a70 con 0x7f678c0ccbd0 2023-07-06T03:45:23.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.178+0000 7f6783fff700 1 -- 172.21.15.97:0/239111363 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c011e20 con 0x7f678c0ccbd0 2023-07-06T03:45:23.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.179+0000 7f67921ff700 1 -- 172.21.15.97:0/239111363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c0c8a40 con 0x7f678c0ccbd0 2023-07-06T03:45:23.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f67921ff700 1 -- 172.21.15.97:0/239111363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c0c9000 con 0x7f678c0ccbd0 2023-07-06T03:45:23.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f970259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97040c8f40 0x7f97040b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f9702d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9704160a20 0x7f97040b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f970259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97040c8f40 0x7f97040b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51444/0 (socket says 172.21.15.97:51444) 2023-07-06T03:45:23.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f970259c700 1 -- 172.21.15.97:0/2419863093 learned_addr learned my addr 172.21.15.97:0/2419863093 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.179+0000 7fd3aefff700 1 Processor -- start 2023-07-06T03:45:23.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7fd3aefff700 1 -- start start 2023-07-06T03:45:23.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f04da5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f04d406ab70 0x7f04d406cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f04da5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04d415c020 0x7f04d415c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f04da5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f04d40cca60 0x7f04d40ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f04da5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d415c9b0 con 0x7f04d406ab70 2023-07-06T03:45:23.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f04da5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d4070490 con 0x7f04d40cca60 2023-07-06T03:45:23.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f04da5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40cc5d0 con 0x7f04d415c020 2023-07-06T03:45:23.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f9702d9d700 1 -- 172.21.15.97:0/2419863093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97040ccbd0 msgr2=0x7f97040b38c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T03:45:23.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f9702d9d700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97040ccbd0 0x7f97040b38c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f9702d9d700 1 -- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97040c8f40 msgr2=0x7f97040b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f9702d9d700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97040c8f40 0x7f97040b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7f9702d9d700 1 -- 172.21.15.97:0/2419863093 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc009a70 con 0x7f9704160a20 2023-07-06T03:45:23.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.180+0000 7fb9e83ff700 1 -- start start 2023-07-06T03:45:23.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.181+0000 7f093f9ff700 1 Processor -- start 2023-07-06T03:45:23.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.181+0000 7f04d37fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04d415c020 0x7f04d415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.181+0000 7f04d37fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04d415c020 0x7f04d415c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:53896/0 (socket says 172.21.15.97:53896) 2023-07-06T03:45:23.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.181+0000 7f04d37fe700 1 -- 172.21.15.97:0/110038253 learned_addr learned my addr 172.21.15.97:0/110038253 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.181+0000 7f093f9ff700 1 -- start start 2023-07-06T03:45:23.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.181+0000 7f67921ff700 1 -- 172.21.15.97:0/239111363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f678c0cdad0 con 0x7f678c0ccbd0 2023-07-06T03:45:23.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.181+0000 7f04d8b9c700 1 --2- 172.21.15.97:0/110038253 >> v2:172.21.15.146:3300/0 conn(0x7f04d40cca60 0x7f04d40ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7f04d8b9c700 1 -- 172.21.15.97:0/110038253 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04d415c020 msgr2=0x7f04d415c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7f04d8b9c700 1 --2- 172.21.15.97:0/110038253 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04d415c020 0x7f04d415c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7f04d8b9c700 1 -- 172.21.15.97:0/110038253 >> v2:172.21.15.97:3300/0 conn(0x7f04d406ab70 msgr2=0x7f04d406cac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T03:45:23.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7f04d8b9c700 1 --2- 172.21.15.97:0/110038253 >> v2:172.21.15.97:3300/0 conn(0x7f04d406ab70 0x7f04d406cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7f04d8b9c700 1 -- 172.21.15.97:0/110038253 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d406d000 con 0x7f04d40cca60 2023-07-06T03:45:23.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7f82695ff700 1 -- 172.21.15.97:0/1312606248 shutdown_connections 2023-07-06T03:45:23.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7f04d8b9c700 1 --2- 172.21.15.97:0/110038253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d40cca60 0x7f04d40ccee0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04d406c360 tx=0x7f04c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d66cf909c0499f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7fd3aefff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd3a80b8890 0x7fd3a80b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7fd3aefff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3a80b4d70 0x7fd3a80cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7fd3aefff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a80cdda0 con 0x7fd3a80cd940 2023-07-06T03:45:23.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7fd3aefff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a80b5280 con 0x7fd3a80b4d70 2023-07-06T03:45:23.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.182+0000 7fd3aefff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a80d2bb0 con 0x7fd3a80b8890 2023-07-06T03:45:23.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fb9e83ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9e00ce3d0 0x7fb9e00c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fb9e83ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb9e015c0c0 0x7fb9e00c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fb9e83ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e00c3b70 con 0x7fb9e00add10 2023-07-06T03:45:23.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fb9e83ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e00c47f0 con 0x7fb9e015c0c0 2023-07-06T03:45:23.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fb9e83ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00c3e50 con 0x7fb9e00ce3d0 2023-07-06T03:45:23.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7f04d2ffd700 1 -- 172.21.15.97:0/110038253 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c400e070 con 0x7f04d40cca60 2023-07-06T03:45:23.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7f04d2ffd700 1 -- 172.21.15.97:0/110038253 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c400bbd0 con 0x7f04d40cca60 2023-07-06T03:45:23.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7f04d2ffd700 1 -- 172.21.15.97:0/110038253 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c4012430 con 0x7f04d40cca60 2023-07-06T03:45:23.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fd3ad59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd3a80cd940 0x7fd3a806c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:35284/0 (socket says 172.21.15.97:35284) 2023-07-06T03:45:23.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fd3ad59c700 1 -- 172.21.15.97:0/1890489617 learned_addr learned my addr 172.21.15.97:0/1890489617 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:23.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7f093f9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0938156cd0 0x7f09380c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7f093f9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f093806eeb0 0x7f09380cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fd3acd9b700 1 --2- 172.21.15.97:0/1890489617 >> v2:172.21.15.146:3300/0 conn(0x7fd3a80b8890 0x7fd3a80b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7f093f9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093815a680 con 0x7f093806f320 2023-07-06T03:45:23.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7f093f9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0938159860 con 0x7f0938156cd0 2023-07-06T03:45:23.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7f093f9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0938159a70 con 0x7f093806eeb0 2023-07-06T03:45:23.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7f093df9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f093806f320 0x7f093815d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35444/0 (socket says 172.21.15.97:35444) 2023-07-06T03:45:23.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7f093df9c700 1 -- 172.21.15.97:0/673418068 learned_addr learned my addr 172.21.15.97:0/673418068 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:23.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fb9e599a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9e00ce3d0 0x7fb9e00c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fb9e699c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb9e015c0c0 0x7fb9e00c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fb9e599a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9e00ce3d0 0x7fb9e00c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:53904/0 (socket says 172.21.15.97:53904) 2023-07-06T03:45:23.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fb9e699c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb9e015c0c0 0x7fb9e00c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57560/0 (socket says 172.21.15.97:57560) 2023-07-06T03:45:23.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fb9e599a700 1 -- 172.21.15.97:0/3200667282 learned_addr learned my addr 172.21.15.97:0/3200667282 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.183+0000 7fd39ffff700 1 --2- 172.21.15.97:0/1890489617 >> v2:172.21.15.97:3300/0 conn(0x7fd3a80b4d70 0x7fd3a80cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7fb9e599a700 1 -- 172.21.15.97:0/3200667282 >> v2:172.21.15.146:3300/0 conn(0x7fb9e015c0c0 msgr2=0x7fb9e00c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7fb9e599a700 1 --2- 172.21.15.97:0/3200667282 >> v2:172.21.15.146:3300/0 conn(0x7fb9e015c0c0 0x7fb9e00c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7fb9e599a700 1 -- 172.21.15.97:0/3200667282 >> v1:172.21.15.146:6789/0 conn(0x7fb9e00add10 legacy=0x7fb9e00c4c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T03:45:23.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7fd3acd9b700 1 -- 172.21.15.97:0/1890489617 >> v2:172.21.15.97:3300/0 conn(0x7fd3a80b4d70 msgr2=0x7fd3a80cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7fd3acd9b700 1 --2- 172.21.15.97:0/1890489617 >> v2:172.21.15.97:3300/0 conn(0x7fd3a80b4d70 0x7fd3a80cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7fd3acd9b700 1 -- 172.21.15.97:0/1890489617 >> v1:172.21.15.146:6789/0 conn(0x7fd3a80cd940 legacy=0x7fd3a806c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7fd3acd9b700 1 -- 172.21.15.97:0/1890489617 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a80d0de0 con 0x7fd3a80b8890 2023-07-06T03:45:23.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7fb9e599a700 1 -- 172.21.15.97:0/3200667282 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e00c4130 con 0x7fb9e00ce3d0 2023-07-06T03:45:23.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029839595 0 0) 0x7f093815a680 con 0x7f093806f320 2023-07-06T03:45:23.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09240035b0 con 0x7f093806f320 2023-07-06T03:45:23.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7f82695ff700 1 -- 172.21.15.97:0/1312606248 wait complete. 2023-07-06T03:45:23.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7fb9e599a700 1 --2- 172.21.15.97:0/3200667282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9e00ce3d0 0x7fb9e00c8f40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800cf60 tx=0x7fb9d800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=523bc461dac64547 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2104414004 0 0) 0x7f09240035b0 con 0x7f093806f320 2023-07-06T03:45:23.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f093806eeb0 msgr2=0x7f09380cc430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T03:45:23.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7f092ffff700 1 --2- 172.21.15.97:0/673418068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f093806eeb0 0x7f09380cc430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 >> v2:172.21.15.146:3300/0 conn(0x7f0938156cd0 msgr2=0x7f09380c5340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T03:45:23.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f092ffff700 1 --2- 172.21.15.97:0/673418068 >> v2:172.21.15.146:3300/0 conn(0x7f0938156cd0 0x7f09380c5340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0938159d50 con 0x7f093806f320 2023-07-06T03:45:23.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f0928001670 con 0x7f093806f320 2023-07-06T03:45:23.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7fb9e5199700 1 -- 172.21.15.97:0/3200667282 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8011070 con 0x7fb9e00ce3d0 2023-07-06T03:45:23.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7fb9e5199700 1 -- 172.21.15.97:0/3200667282 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d800c2e0 con 0x7fb9e00ce3d0 2023-07-06T03:45:23.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7fb9e5199700 1 -- 172.21.15.97:0/3200667282 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8007f50 con 0x7fb9e00ce3d0 2023-07-06T03:45:23.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7fd3acd9b700 1 --2- 172.21.15.97:0/1890489617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3a80b8890 0x7fd3a80b4830 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009840 tx=0x7fd3a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=147f024beae3e4a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 >> v1:172.21.15.97:6789/0 conn(0x7f093806f320 legacy=0x7f093815d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f092ffff700 1 --2- 172.21.15.97:0/673418068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0924003a00 0x7f0924003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f092ffff700 1 --2- 172.21.15.97:0/673418068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0924004850 0x7f0924004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f092ffff700 1 --2- 172.21.15.97:0/673418068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0924005250 0x7f092400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0938159860 con 0x7f0924005250 2023-07-06T03:45:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0938159a70 con 0x7f0924003a00 2023-07-06T03:45:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09240105d0 con 0x7f0924004850 2023-07-06T03:45:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f093df9c700 1 --2- 172.21.15.97:0/673418068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0924005250 0x7f092400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7f093df9c700 1 -- 172.21.15.97:0/673418068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0924004850 msgr2=0x7f0924004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T03:45:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7f093df9c700 1 --2- 172.21.15.97:0/673418068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0924004850 0x7f0924004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7f093df9c700 1 -- 172.21.15.97:0/673418068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0924003a00 msgr2=0x7f0924003e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T03:45:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7f093df9c700 1 --2- 172.21.15.97:0/673418068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0924003a00 0x7f0924003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7f093df9c700 1 -- 172.21.15.97:0/673418068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0938159d50 con 0x7f0924005250 2023-07-06T03:45:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.184+0000 7f6783fff700 1 -- 172.21.15.97:0/239111363 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f677c01a400 con 0x7f678c0ccbd0 2023-07-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.185+0000 7f6783fff700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f676c06adb0 0x7f676c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7f6783fff700 1 -- 172.21.15.97:0/239111363 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f677c089550 con 0x7f678c0ccbd0 2023-07-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7f6783fff700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] conn(0x7f676c0735e0 0x7f676c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7f6783fff700 1 -- 172.21.15.97:0/239111363 --> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f676c0760a0 con 0x7f676c0735e0 2023-07-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7f678affd700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f676c06adb0 0x7f676c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7f6783fff700 1 -- 172.21.15.97:0/239111363 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f677c0545e0 con 0x7f678c0ccbd0 2023-07-06T03:45:23.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7f093df9c700 1 --2- 172.21.15.97:0/673418068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0924005250 0x7f092400ba10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0928001eb0 tx=0x7f0924005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f15a708d0f7612 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.187+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800ce00 con 0x7f0924005250 2023-07-06T03:45:23.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.187+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0928005960 con 0x7f0924005250 2023-07-06T03:45:23.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.187+0000 7f092ffff700 1 -- 172.21.15.97:0/673418068 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928005cd0 con 0x7f0924005250 2023-07-06T03:45:23.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.187+0000 7f82695ff700 1 Processor -- start 2023-07-06T03:45:23.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.187+0000 7f093f9ff700 1 -- 172.21.15.97:0/673418068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0924005250 msgr2=0x7f092400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.187+0000 7f093f9ff700 1 --2- 172.21.15.97:0/673418068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0924005250 0x7f092400ba10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0928001eb0 tx=0x7f0924005880 comp rx=0 tx=0).stop 2023-07-06T03:45:23.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.187+0000 7f093df9c700 1 -- 172.21.15.97:0/673418068 reap_dead start 2023-07-06T03:45:23.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.187+0000 7f04da5ff700 1 -- 172.21.15.97:0/110038253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d40cca60 msgr2=0x7f04d40ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.187+0000 7f04da5ff700 1 --2- 172.21.15.97:0/110038253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d40cca60 0x7f04d40ccee0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04d406c360 tx=0x7f04c400b330 comp rx=0 tx=0).stop 2023-07-06T03:45:23.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f093f9ff700 1 -- 172.21.15.97:0/673418068 shutdown_connections 2023-07-06T03:45:23.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f093f9ff700 1 -- 172.21.15.97:0/673418068 >> 172.21.15.97:0/673418068 conn(0x7f09380b1650 msgr2=0x7f09380b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7fd39f7fe700 1 -- 172.21.15.97:0/1890489617 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4004230 con 0x7fd3a80b8890 2023-07-06T03:45:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.187+0000 7fd39f7fe700 1 -- 172.21.15.97:0/1890489617 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a4004390 con 0x7fd3a80b8890 2023-07-06T03:45:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.187+0000 7fd39f7fe700 1 -- 172.21.15.97:0/1890489617 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40036a0 con 0x7fd3a80b8890 2023-07-06T03:45:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7fd3aefff700 1 -- 172.21.15.97:0/1890489617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3a80b8890 msgr2=0x7fd3a80b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7fd3aefff700 1 --2- 172.21.15.97:0/1890489617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3a80b8890 0x7fd3a80b4830 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009840 tx=0x7fd3a4009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f678bfff700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] conn(0x7f676c0735e0 0x7f676c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f678affd700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f676c06adb0 0x7f676c06d260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f678c1ece10 tx=0x7f677400d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f04da5ff700 1 -- 172.21.15.97:0/110038253 shutdown_connections 2023-07-06T03:45:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f04da5ff700 1 --2- 172.21.15.97:0/110038253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d40cca60 0x7f04d40ccee0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f04da5ff700 1 --2- 172.21.15.97:0/110038253 >> v2:172.21.15.97:3300/0 conn(0x7f04d406ab70 0x7f04d406cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f04da5ff700 1 --2- 172.21.15.97:0/110038253 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04d415c020 0x7f04d415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f04da5ff700 1 -- 172.21.15.97:0/110038253 >> 172.21.15.97:0/110038253 conn(0x7f04d406a060 msgr2=0x7f04d406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f9702d9d700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9704160a20 0x7f97040b3e00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f96fc016000 tx=0x7f96fc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f82695ff700 1 -- start start 2023-07-06T03:45:23.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f96faffd700 1 -- 172.21.15.97:0/2419863093 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc012430 con 0x7f9704160a20 2023-07-06T03:45:23.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f96faffd700 1 -- 172.21.15.97:0/2419863093 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96fc020070 con 0x7f9704160a20 2023-07-06T03:45:23.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f96faffd700 1 -- 172.21.15.97:0/2419863093 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc004280 con 0x7f9704160a20 2023-07-06T03:45:23.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7fb9e83ff700 1 -- 172.21.15.97:0/3200667282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9e00ce3d0 msgr2=0x7fb9e00c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7fb9e83ff700 1 --2- 172.21.15.97:0/3200667282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9e00ce3d0 0x7fb9e00c8f40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800cf60 tx=0x7fb9d800cf90 comp rx=0 tx=0).stop 2023-07-06T03:45:23.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7fb9e83ff700 1 -- 172.21.15.97:0/3200667282 shutdown_connections 2023-07-06T03:45:23.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7fb9e83ff700 1 --2- 172.21.15.97:0/3200667282 >> v2:172.21.15.146:3300/0 conn(0x7fb9e015c0c0 0x7fb9e00c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7fb9e83ff700 1 --2- 172.21.15.97:0/3200667282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9e00ce3d0 0x7fb9e00c8f40 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800cf60 tx=0x7fb9d800cf90 comp rx=0 tx=0).stop 2023-07-06T03:45:23.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.186+0000 7fb9e83ff700 1 -- 172.21.15.97:0/3200667282 >> 172.21.15.97:0/3200667282 conn(0x7fb9e00cf610 msgr2=0x7fb9e00cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f04da5ff700 1 -- 172.21.15.97:0/110038253 shutdown_connections 2023-07-06T03:45:23.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f04da5ff700 1 -- 172.21.15.97:0/110038253 wait complete. 2023-07-06T03:45:23.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.189+0000 7f678bfff700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] conn(0x7f676c0735e0 0x7f676c0759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7f9708dff700 1 -- 172.21.15.97:0/2419863093 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97040af8d0 con 0x7f9704160a20 2023-07-06T03:45:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.189+0000 7f9708dff700 1 -- 172.21.15.97:0/2419863093 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97040afe90 con 0x7f9704160a20 2023-07-06T03:45:23.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.188+0000 7fb9e83ff700 1 -- 172.21.15.97:0/3200667282 shutdown_connections 2023-07-06T03:45:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.189+0000 7fd3aefff700 1 -- 172.21.15.97:0/1890489617 shutdown_connections 2023-07-06T03:45:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.189+0000 7fd3aefff700 1 --2- 172.21.15.97:0/1890489617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3a80b8890 0x7fd3a80b4830 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.189+0000 7fd3aefff700 1 --2- 172.21.15.97:0/1890489617 >> v2:172.21.15.97:3300/0 conn(0x7fd3a80b4d70 0x7fd3a80cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.189+0000 7fd3aefff700 1 -- 172.21.15.97:0/1890489617 >> 172.21.15.97:0/1890489617 conn(0x7fd3a806b2d0 msgr2=0x7fd3a80cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.189+0000 7f093f9ff700 1 -- 172.21.15.97:0/673418068 shutdown_connections 2023-07-06T03:45:23.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.189+0000 7f82695ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f826406b360 0x7f82640cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7f82695ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82640d2210 0x7f82641f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7f82695ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f826415c610 0x7f826415f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7f82695ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640cccc0 con 0x7f82640d2210 2023-07-06T03:45:23.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7f82695ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640cba40 con 0x7f826406b360 2023-07-06T03:45:23.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7f82695ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826415dec0 con 0x7f826415c610 2023-07-06T03:45:23.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7f9708dff700 1 -- 172.21.15.97:0/2419863093 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97040caf00 con 0x7f9704160a20 2023-07-06T03:45:23.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7f82627fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82640d2210 0x7f82641f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7f82627fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82640d2210 0x7f82641f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51464/0 (socket says 172.21.15.97:51464) 2023-07-06T03:45:23.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7f82627fc700 1 -- 172.21.15.97:0/862476165 learned_addr learned my addr 172.21.15.97:0/862476165 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7f8262ffd700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f826406b360 0x7f82640cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7f82637fe700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f826415c610 0x7f826415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.191+0000 7fd3aefff700 1 -- 172.21.15.97:0/1890489617 shutdown_connections 2023-07-06T03:45:23.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.191+0000 7f82627fc700 1 -- 172.21.15.97:0/862476165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f826415c610 msgr2=0x7f826415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.191+0000 7f82627fc700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f826415c610 0x7f826415f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.191+0000 7f82627fc700 1 -- 172.21.15.97:0/862476165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f826406b360 msgr2=0x7f82640cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.191+0000 7f82627fc700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f826406b360 0x7f82640cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.191+0000 7f82627fc700 1 -- 172.21.15.97:0/862476165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82580094a0 con 0x7f82640d2210 2023-07-06T03:45:23.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.191+0000 7f82627fc700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82640d2210 0x7f82641f6a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f825c00cbd0 tx=0x7f825c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.192+0000 7f824b7fe700 1 -- 172.21.15.97:0/862476165 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00aa60 con 0x7f82640d2210 2023-07-06T03:45:23.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.192+0000 7f824b7fe700 1 -- 172.21.15.97:0/862476165 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c00abc0 con 0x7f82640d2210 2023-07-06T03:45:23.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.192+0000 7f824b7fe700 1 -- 172.21.15.97:0/862476165 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c012400 con 0x7f82640d2210 2023-07-06T03:45:23.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.192+0000 7f96faffd700 1 -- 172.21.15.97:0/2419863093 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96fc0043e0 con 0x7f9704160a20 2023-07-06T03:45:23.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.191+0000 7f04da5ff700 1 Processor -- start 2023-07-06T03:45:23.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.191+0000 7f04da5ff700 1 -- start start 2023-07-06T03:45:23.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.192+0000 7f82695ff700 1 -- 172.21.15.97:0/862476165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82641f6fa0 con 0x7f82640d2210 2023-07-06T03:45:23.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.190+0000 7fb9e83ff700 1 -- 172.21.15.97:0/3200667282 wait complete. 2023-07-06T03:45:23.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7fb9e83ff700 1 Processor -- start 2023-07-06T03:45:23.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.191+0000 7fd3aefff700 1 -- 172.21.15.97:0/1890489617 wait complete. 2023-07-06T03:45:23.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7f04da5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d406ab70 0x7f04d416f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7f04da5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d40cca60 0x7f04d40b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7f04da5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04d415c020 0x7f04d41591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7f04da5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d4157ef0 con 0x7f04d406ab70 2023-07-06T03:45:23.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7f04da5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d4158d50 con 0x7f04d40cca60 2023-07-06T03:45:23.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7f04da5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d4158140 con 0x7f04d415c020 2023-07-06T03:45:23.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7fd3aefff700 1 Processor -- start 2023-07-06T03:45:23.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7f6783fff700 1 -- 172.21.15.97:0/239111363 <== osd.0 v2:172.21.15.97:6800/3690379650 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f676c0760a0 con 0x7f676c0735e0 2023-07-06T03:45:23.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7f82695ff700 1 -- 172.21.15.97:0/862476165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82640cc9e0 con 0x7f82640d2210 2023-07-06T03:45:23.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7f82695ff700 1 -- 172.21.15.97:0/862476165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82640d01c0 con 0x7f82640d2210 2023-07-06T03:45:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.194+0000 7f96faffd700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f96e406ace0 0x7f96e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.194+0000 7f96faffd700 1 -- 172.21.15.97:0/2419863093 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f96fc022070 con 0x7f9704160a20 2023-07-06T03:45:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.194+0000 7f96faffd700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] conn(0x7f96e4073510 0x7f96e4075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.194+0000 7f96faffd700 1 -- 172.21.15.97:0/2419863093 --> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96e4075fd0 con 0x7f96e4073510 2023-07-06T03:45:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.194+0000 7f96faffd700 1 -- 172.21.15.97:0/2419863093 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96fc08b7b0 con 0x7f9704160a20 2023-07-06T03:45:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7f093f9ff700 1 -- 172.21.15.97:0/673418068 wait complete. 2023-07-06T03:45:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.193+0000 7fb9e83ff700 1 -- start start 2023-07-06T03:45:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fb9e83ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e015c0c0 0x7fb9e01574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fb9e83ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9e01579f0 0x7fb9e00c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fb9e83ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9e00c77f0 0x7fb9e00c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fb9e83ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00c5d90 con 0x7fb9e015c0c0 2023-07-06T03:45:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fb9e83ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00c81a0 con 0x7fb9e00c77f0 2023-07-06T03:45:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fb9e83ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0157e60 con 0x7fb9e01579f0 2023-07-06T03:45:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.194+0000 7f04d8b9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04d415c020 0x7f04d41591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.194+0000 7f04d8b9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04d415c020 0x7f04d41591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:53944/0 (socket says 172.21.15.97:53944) 2023-07-06T03:45:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.194+0000 7f04d8b9c700 1 -- 172.21.15.97:0/2436314314 learned_addr learned my addr 172.21.15.97:0/2436314314 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7f04d37fe700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d40cca60 0x7f04d40b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.194+0000 7fd3aefff700 1 -- start start 2023-07-06T03:45:23.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fd3aefff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a80b4d70 0x7fd3a80cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fd3aefff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3a80b8890 0x7fd3a815f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fd3aefff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a80cd940 0x7fd3a8070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fd3aefff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a80c68e0 con 0x7fd3a80b4d70 2023-07-06T03:45:23.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fd3aefff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a81f60c0 con 0x7fd3a80b8890 2023-07-06T03:45:23.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7fd3aefff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a81f63a0 con 0x7fd3a80cd940 2023-07-06T03:45:23.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7f04d37fe700 1 -- 172.21.15.97:0/2436314314 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04d415c020 msgr2=0x7f04d41591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7f04d37fe700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04d415c020 0x7f04d41591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7f04d37fe700 1 -- 172.21.15.97:0/2436314314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d406ab70 msgr2=0x7f04d416f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7f04d37fe700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04d406ab70 0x7f04d416f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.195+0000 7f04d37fe700 1 -- 172.21.15.97:0/2436314314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c4009a70 con 0x7f04d40cca60 2023-07-06T03:45:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fb9e699c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9e00c77f0 0x7fb9e00c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fb9e699c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9e00c77f0 0x7fb9e00c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57578/0 (socket says 172.21.15.97:57578) 2023-07-06T03:45:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fb9e699c700 1 -- 172.21.15.97:0/2885145641 learned_addr learned my addr 172.21.15.97:0/2885145641 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fb9e599a700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9e01579f0 0x7fb9e00c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fb9e619b700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e015c0c0 0x7fb9e01574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7f04d37fe700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d40cca60 0x7f04d40b3b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f04c800e600 tx=0x7f04c800e910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fd3acd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a80b4d70 0x7fd3a80cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fd3acd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a80b4d70 0x7fd3a80cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51498/0 (socket says 172.21.15.97:51498) 2023-07-06T03:45:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fd3acd9b700 1 -- 172.21.15.97:0/1917346426 learned_addr learned my addr 172.21.15.97:0/1917346426 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fd3ad59c700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a80cd940 0x7fd3a8070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7f04d0ff9700 1 -- 172.21.15.97:0/2436314314 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8013610 con 0x7f04d40cca60 2023-07-06T03:45:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7f04d0ff9700 1 -- 172.21.15.97:0/2436314314 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c8013770 con 0x7f04d40cca60 2023-07-06T03:45:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7f04d0ff9700 1 -- 172.21.15.97:0/2436314314 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8013ae0 con 0x7f04d40cca60 2023-07-06T03:45:23.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7f04da5ff700 1 -- 172.21.15.97:0/2436314314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d4158420 con 0x7f04d40cca60 2023-07-06T03:45:23.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7f093f9ff700 1 Processor -- start 2023-07-06T03:45:23.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7f04da5ff700 1 -- 172.21.15.97:0/2436314314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d41589e0 con 0x7f04d40cca60 2023-07-06T03:45:23.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fd3ad59c700 1 -- 172.21.15.97:0/1917346426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3a80b8890 msgr2=0x7fd3a815f610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T03:45:23.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fd3ad59c700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3a80b8890 0x7fd3a815f610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fd3ad59c700 1 -- 172.21.15.97:0/1917346426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a80b4d70 msgr2=0x7fd3a80cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fd3ad59c700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a80b4d70 0x7fd3a80cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.196+0000 7fd3ad59c700 1 -- 172.21.15.97:0/1917346426 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a40094a0 con 0x7fd3a80cd940 2023-07-06T03:45:23.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.197+0000 7f9701d9b700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] conn(0x7f96e4073510 0x7f96e4075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.197+0000 7fb9e619b700 1 -- 172.21.15.97:0/2885145641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9e01579f0 msgr2=0x7fb9e00c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.197+0000 7fb9e619b700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb9e01579f0 0x7fb9e00c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.197+0000 7fb9e619b700 1 -- 172.21.15.97:0/2885145641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9e00c77f0 msgr2=0x7fb9e00c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.197+0000 7fb9e619b700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9e00c77f0 0x7fb9e00c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.197+0000 7fb9e619b700 1 -- 172.21.15.97:0/2885145641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d800cc40 con 0x7fb9e015c0c0 2023-07-06T03:45:23.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7fb9e619b700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e015c0c0 0x7fb9e01574b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0066b70 tx=0x7fb9d000b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7fd3ad59c700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a80cd940 0x7fd3a8070810 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000caf0 tx=0x7fd3a000ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f970259c700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f96e406ace0 0x7f96e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f9701d9b700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] conn(0x7f96e4073510 0x7f96e4075920 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.197+0000 7f093f9ff700 1 -- start start 2023-07-06T03:45:23.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f093f9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09381ee8e0 0x7f093806cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f093f9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f093806d1c0 0x7f093815b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f093f9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f093815b550 0x7f0938151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f093f9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093815b9f0 con 0x7f093815b550 2023-07-06T03:45:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f093f9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093806d640 con 0x7f093806d1c0 2023-07-06T03:45:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f093f9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093806d920 con 0x7f09381ee8e0 2023-07-06T03:45:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f093d79b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09381ee8e0 0x7f093806cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f093d79b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09381ee8e0 0x7f093806cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:53968/0 (socket says 172.21.15.97:53968) 2023-07-06T03:45:23.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f093cf9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f093806d1c0 0x7f093815b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7f093d79b700 1 -- 172.21.15.97:0/4099582719 learned_addr learned my addr 172.21.15.97:0/4099582719 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7f093d79b700 1 -- 172.21.15.97:0/4099582719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f093806d1c0 msgr2=0x7f093815b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7f093d79b700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f093806d1c0 0x7f093815b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7f093d79b700 1 -- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f093815b550 msgr2=0x7f0938151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7f093d79b700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f093815b550 0x7f0938151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7f093d79b700 1 -- 172.21.15.97:0/4099582719 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093806dc00 con 0x7f09381ee8e0 2023-07-06T03:45:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7f093d79b700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09381ee8e0 0x7f093806cc80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f093000b780 tx=0x7f093000bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7fb9d6ffd700 1 -- 172.21.15.97:0/2885145641 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0011830 con 0x7fb9e015c0c0 2023-07-06T03:45:23.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7fb9d6ffd700 1 -- 172.21.15.97:0/2885145641 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0011990 con 0x7fb9e015c0c0 2023-07-06T03:45:23.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7fb9d6ffd700 1 -- 172.21.15.97:0/2885145641 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0011d20 con 0x7fb9e015c0c0 2023-07-06T03:45:23.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.198+0000 7fd39d7fa700 1 -- 172.21.15.97:0/1917346426 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000a9e0 con 0x7fd3a80cd940 2023-07-06T03:45:23.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7fd39d7fa700 1 -- 172.21.15.97:0/1917346426 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a000ab40 con 0x7fd3a80cd940 2023-07-06T03:45:23.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7fd39d7fa700 1 -- 172.21.15.97:0/1917346426 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0012400 con 0x7fd3a80cd940 2023-07-06T03:45:23.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7fd3aefff700 1 -- 172.21.15.97:0/1917346426 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a81f6680 con 0x7fd3a80cd940 2023-07-06T03:45:23.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7f092dffb700 1 -- 172.21.15.97:0/4099582719 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000f840 con 0x7f09381ee8e0 2023-07-06T03:45:23.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7f092dffb700 1 -- 172.21.15.97:0/4099582719 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093000f9c0 con 0x7f09381ee8e0 2023-07-06T03:45:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7f092dffb700 1 -- 172.21.15.97:0/4099582719 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000fd70 con 0x7f09381ee8e0 2023-07-06T03:45:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.200+0000 7f093f9ff700 1 -- 172.21.15.97:0/4099582719 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093806dee0 con 0x7f09381ee8e0 2023-07-06T03:45:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.200+0000 7f093f9ff700 1 -- 172.21.15.97:0/4099582719 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09381eecb0 con 0x7f09381ee8e0 2023-07-06T03:45:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.199+0000 7f970259c700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f96e406ace0 0x7f96e406d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f96f40011e0 tx=0x7f96f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.200+0000 7f96faffd700 1 -- 172.21.15.97:0/2419863093 <== osd.2 v2:172.21.15.97:6824/3131825859 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f96e4075fd0 con 0x7f96e4073510 2023-07-06T03:45:23.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.200+0000 7f04da5ff700 1 -- 172.21.15.97:0/2436314314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f04d40706e0 con 0x7f04d40cca60 2023-07-06T03:45:23.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.200+0000 7fb9e83ff700 1 -- 172.21.15.97:0/2885145641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e0158130 con 0x7fb9e015c0c0 2023-07-06T03:45:23.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.200+0000 7f824b7fe700 1 -- 172.21.15.97:0/862476165 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f825c01b3b0 con 0x7f82640d2210 2023-07-06T03:45:23.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.200+0000 7fb9e83ff700 1 -- 172.21.15.97:0/2885145641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e01586f0 con 0x7fb9e015c0c0 2023-07-06T03:45:23.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.200+0000 7f04d0ff9700 1 -- 172.21.15.97:0/2436314314 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f04c8013c40 con 0x7f04d40cca60 2023-07-06T03:45:23.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.201+0000 7f093f9ff700 1 -- 172.21.15.97:0/4099582719 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f093815a310 con 0x7f09381ee8e0 2023-07-06T03:45:23.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.202+0000 7f824b7fe700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f825006adb0 0x7f825006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.202+0000 7f824b7fe700 1 -- 172.21.15.97:0/862476165 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f825c0897f0 con 0x7f82640d2210 2023-07-06T03:45:23.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.202+0000 7f824b7fe700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] conn(0x7f8250073660 0x7f8250075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.202+0000 7f824b7fe700 1 -- 172.21.15.97:0/862476165 --> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8250076120 con 0x7f8250073660 2023-07-06T03:45:23.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.202+0000 7f824b7fe700 1 -- 172.21.15.97:0/862476165 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f825c089ba0 con 0x7f82640d2210 2023-07-06T03:45:23.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.202+0000 7f04d0ff9700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f04c006adb0 0x7f04c006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.202+0000 7f04d0ff9700 1 -- 172.21.15.97:0/2436314314 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f04c8088e60 con 0x7f04d40cca60 2023-07-06T03:45:23.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.202+0000 7f04d0ff9700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] conn(0x7f04c0073660 0x7f04c0075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.202+0000 7f04d0ff9700 1 -- 172.21.15.97:0/2436314314 --> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04c0076120 con 0x7f04c0073660 2023-07-06T03:45:23.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.202+0000 7f04d0ff9700 1 -- 172.21.15.97:0/2436314314 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04c808b480 con 0x7f04d40cca60 2023-07-06T03:45:23.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.203+0000 7fb9e83ff700 1 -- 172.21.15.97:0/2885145641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9e00ae0e0 con 0x7fb9e015c0c0 2023-07-06T03:45:23.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.203+0000 7f8262ffd700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f825006adb0 0x7f825006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.203+0000 7f82637fe700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] conn(0x7f8250073660 0x7f8250075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.204+0000 7f8262ffd700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f825006adb0 0x7f825006d260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82640d19d0 tx=0x7f8254009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.203+0000 7f52d23ff700 1 Processor -- start 2023-07-06T03:45:23.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.204+0000 7f52d23ff700 1 -- start start 2023-07-06T03:45:23.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.204+0000 7f82637fe700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] conn(0x7f8250073660 0x7f8250075a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.205+0000 7f092dffb700 1 -- 172.21.15.97:0/4099582719 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0930017400 con 0x7f09381ee8e0 2023-07-06T03:45:23.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.203+0000 7f04d8b9c700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] conn(0x7f04c0073660 0x7f04c0075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.204+0000 7f04d8b9c700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] conn(0x7f04c0073660 0x7f04c0075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.205+0000 7f04d0ff9700 1 -- 172.21.15.97:0/2436314314 <== osd.5 v2:172.21.15.146:6809/3757512297 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f04c0076120 con 0x7f04c0073660 2023-07-06T03:45:23.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.203+0000 7fd3aefff700 1 -- 172.21.15.97:0/1917346426 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a80ccf60 con 0x7fd3a80cd940 2023-07-06T03:45:23.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.204+0000 7fd3aefff700 1 -- 172.21.15.97:0/1917346426 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd3a80d13a0 con 0x7fd3a80cd940 2023-07-06T03:45:23.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.206+0000 7fb9d6ffd700 1 -- 172.21.15.97:0/2885145641 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb9d001a400 con 0x7fb9e015c0c0 2023-07-06T03:45:23.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.206+0000 7f092dffb700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f091c06ace0 0x7f091c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.206+0000 7f092dffb700 1 -- 172.21.15.97:0/4099582719 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0930089d30 con 0x7f09381ee8e0 2023-07-06T03:45:23.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.207+0000 7f092dffb700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] conn(0x7f091c073590 0x7f091c0759a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.207+0000 7f092dffb700 1 -- 172.21.15.97:0/4099582719 --> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f091c076050 con 0x7f091c073590 2023-07-06T03:45:23.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.207+0000 7f092dffb700 1 -- 172.21.15.97:0/4099582719 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f093008a0e0 con 0x7f09381ee8e0 2023-07-06T03:45:23.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.206+0000 7f52d23ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc0ce290 0x7f52cc06c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.206+0000 7f52d23ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc0cadb0 con 0x7f52cc1583a0 2023-07-06T03:45:23.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.206+0000 7f52d23ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc0c52f0 con 0x7f52cc15ca80 2023-07-06T03:45:23.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.206+0000 7f52d23ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0c55d0 con 0x7f52cc0ce290 2023-07-06T03:45:23.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.207+0000 7f52cb7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f52cc15ca80 0x7f52cc0cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:35296/0 (socket says 172.21.15.97:35296) 2023-07-06T03:45:23.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.207+0000 7f52cb7fe700 1 -- 172.21.15.97:0/1738632544 learned_addr learned my addr 172.21.15.97:0/1738632544 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:23.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.207+0000 7f52d099c700 1 --2- 172.21.15.97:0/1738632544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc0ce290 0x7f52cc06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.207+0000 7f04d3fff700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f04c006adb0 0x7f04c006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.207+0000 7f824b7fe700 1 -- 172.21.15.97:0/862476165 <== osd.1 v2:172.21.15.97:6808/3448212434 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8250076120 con 0x7f8250073660 2023-07-06T03:45:23.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.207+0000 7f52d099c700 1 -- 172.21.15.97:0/1738632544 >> v1:172.21.15.146:6789/0 conn(0x7f52cc15ca80 legacy=0x7f52cc0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.207+0000 7f52d099c700 1 -- 172.21.15.97:0/1738632544 >> v1:172.21.15.97:6789/0 conn(0x7f52cc1583a0 legacy=0x7f52cc0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.207+0000 7f52d099c700 1 -- 172.21.15.97:0/1738632544 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc0cb000 con 0x7f52cc0ce290 2023-07-06T03:45:23.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.208+0000 7f52d099c700 1 --2- 172.21.15.97:0/1738632544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc0ce290 0x7f52cc06c300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f52b4008870 tx=0x7f52b400d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd487a036d667292 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.208+0000 7f093cf9a700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f091c06ace0 0x7f091c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.208+0000 7f093df9c700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] conn(0x7f091c073590 0x7f091c0759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.208+0000 7f04d3fff700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f04c006adb0 0x7f04c006d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009770 tx=0x7f04cc006cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.208+0000 7f52caffd700 1 -- 172.21.15.97:0/1738632544 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4010070 con 0x7f52cc0ce290 2023-07-06T03:45:23.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.208+0000 7f52caffd700 1 -- 172.21.15.97:0/1738632544 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4004030 con 0x7f52cc0ce290 2023-07-06T03:45:23.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.208+0000 7f52caffd700 1 -- 172.21.15.97:0/1738632544 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40043c0 con 0x7f52cc0ce290 2023-07-06T03:45:23.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.208+0000 7fb9d6ffd700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb9c406adb0 0x7fb9c406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.209+0000 7fb9d6ffd700 1 -- 172.21.15.97:0/2885145641 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb9d0089250 con 0x7fb9e015c0c0 2023-07-06T03:45:23.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.209+0000 7fb9d6ffd700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] conn(0x7fb9c4073660 0x7fb9c4075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.209+0000 7fb9d6ffd700 1 -- 172.21.15.97:0/2885145641 --> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9c4076120 con 0x7fb9c4073660 2023-07-06T03:45:23.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.209+0000 7fb9d6ffd700 1 -- 172.21.15.97:0/2885145641 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d008b870 con 0x7fb9e015c0c0 2023-07-06T03:45:23.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.209+0000 7f52d23ff700 1 -- 172.21.15.97:0/1738632544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc0ce290 msgr2=0x7f52cc06c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.209+0000 7f52d23ff700 1 --2- 172.21.15.97:0/1738632544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc0ce290 0x7f52cc06c300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f52b4008870 tx=0x7f52b400d330 comp rx=0 tx=0).stop 2023-07-06T03:45:23.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.209+0000 7fb9e699c700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] conn(0x7fb9c4073660 0x7fb9c4075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.210+0000 7fb9e599a700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb9c406adb0 0x7fb9c406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.210+0000 7f52d23ff700 1 -- 172.21.15.97:0/1738632544 shutdown_connections 2023-07-06T03:45:23.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.210+0000 7f52d23ff700 1 --2- 172.21.15.97:0/1738632544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc0ce290 0x7f52cc06c300 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.210+0000 7f52d23ff700 1 -- 172.21.15.97:0/1738632544 >> 172.21.15.97:0/1738632544 conn(0x7f52cc06a060 msgr2=0x7f52cc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.210+0000 7f52d23ff700 1 -- 172.21.15.97:0/1738632544 shutdown_connections 2023-07-06T03:45:23.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.210+0000 7f52d23ff700 1 -- 172.21.15.97:0/1738632544 wait complete. 2023-07-06T03:45:23.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.210+0000 7f093cf9a700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f091c06ace0 0x7f091c06d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09381f2510 tx=0x7f0934008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.210+0000 7f093df9c700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] conn(0x7f091c073590 0x7f091c0759a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.212+0000 7fd39d7fa700 1 -- 172.21.15.97:0/1917346426 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd3a0012560 con 0x7fd3a80cd940 2023-07-06T03:45:23.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.211+0000 7fb9e599a700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb9c406adb0 0x7fb9c406d260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0073d50 tx=0x7fb9d800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.211+0000 7fb9e699c700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] conn(0x7fb9c4073660 0x7fb9c4075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.213+0000 7f52d23ff700 1 Processor -- start 2023-07-06T03:45:23.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.213+0000 7f52d23ff700 1 -- start start 2023-07-06T03:45:23.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.213+0000 7fd39d7fa700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd38c06acb0 0x7fd38c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.213+0000 7fd39d7fa700 1 -- 172.21.15.97:0/1917346426 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd3a0088ed0 con 0x7fd3a80cd940 2023-07-06T03:45:23.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.214+0000 7fd39d7fa700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] conn(0x7fd38c073560 0x7fd38c075970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.214+0000 7fd39d7fa700 1 -- 172.21.15.97:0/1917346426 --> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd38c076020 con 0x7fd38c073560 2023-07-06T03:45:23.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.214+0000 7fd39d7fa700 1 -- 172.21.15.97:0/1917346426 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3a00891d0 con 0x7fd3a80cd940 2023-07-06T03:45:23.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.212+0000 7f092dffb700 1 -- 172.21.15.97:0/4099582719 <== osd.3 v2:172.21.15.97:6809/1358667479 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f091c076050 con 0x7f091c073590 2023-07-06T03:45:23.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.214+0000 7fd39ffff700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] conn(0x7fd38c073560 0x7fd38c075970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7f52d23ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52cc0ce290 0x7f52cc159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7f52d23ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc1583a0 0x7f52cc0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7f52d23ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52cc15ca80 0x7f52cc15a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7f52d23ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0c5ad0 con 0x7f52cc0ce290 2023-07-06T03:45:23.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7f52d23ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc159860 con 0x7f52cc15ca80 2023-07-06T03:45:23.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7f52d23ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc159b40 con 0x7f52cc1583a0 2023-07-06T03:45:23.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7fd39ffff700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] conn(0x7fd38c073560 0x7fd38c075970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7fd3acd9b700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd38c06acb0 0x7fd38c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7f52cb7fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc1583a0 0x7f52cc0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7f52cb7fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc1583a0 0x7f52cc0b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:53990/0 (socket says 172.21.15.97:53990) 2023-07-06T03:45:23.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7f52cb7fe700 1 -- 172.21.15.97:0/1642870294 learned_addr learned my addr 172.21.15.97:0/1642870294 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.215+0000 7f52d099c700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52cc15ca80 0x7f52cc15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.216+0000 7fb9d6ffd700 1 -- 172.21.15.97:0/2885145641 <== osd.4 v2:172.21.15.146:6800/663749144 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb9c4076120 con 0x7fb9c4073660 2023-07-06T03:45:23.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.216+0000 7f52cb7fe700 1 -- 172.21.15.97:0/1642870294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52cc15ca80 msgr2=0x7f52cc15a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.216+0000 7f52cb7fe700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52cc15ca80 0x7f52cc15a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.216+0000 7f52cb7fe700 1 -- 172.21.15.97:0/1642870294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52cc0ce290 msgr2=0x7f52cc159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.216+0000 7f52cb7fe700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52cc0ce290 0x7f52cc159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.216+0000 7f52cb7fe700 1 -- 172.21.15.97:0/1642870294 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b4008520 con 0x7f52cc1583a0 2023-07-06T03:45:23.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.216+0000 7fd3acd9b700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd38c06acb0 0x7fd38c06d160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009b90 tx=0x7fd3a4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.216+0000 7f52cb7fe700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc1583a0 0x7f52cc0b3290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f52bc001ff0 tx=0x7f52bc00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.218+0000 7fd39d7fa700 1 -- 172.21.15.97:0/1917346426 <== osd.6 v2:172.21.15.146:6824/1310854864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd38c076020 con 0x7fd38c073560 2023-07-06T03:45:23.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.217+0000 7f52c8ff9700 1 -- 172.21.15.97:0/1642870294 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00ad80 con 0x7f52cc1583a0 2023-07-06T03:45:23.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.217+0000 7f52c8ff9700 1 -- 172.21.15.97:0/1642870294 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc0070d0 con 0x7f52cc1583a0 2023-07-06T03:45:23.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.217+0000 7f52c8ff9700 1 -- 172.21.15.97:0/1642870294 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc007480 con 0x7f52cc1583a0 2023-07-06T03:45:23.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.218+0000 7f52d23ff700 1 -- 172.21.15.97:0/1642870294 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc159e20 con 0x7f52cc1583a0 2023-07-06T03:45:23.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.218+0000 7f52d23ff700 1 -- 172.21.15.97:0/1642870294 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc15a3e0 con 0x7f52cc1583a0 2023-07-06T03:45:23.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.222+0000 7f52d23ff700 1 -- 172.21.15.97:0/1642870294 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52cc0c4fc0 con 0x7f52cc1583a0 2023-07-06T03:45:23.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.222+0000 7f52c8ff9700 1 -- 172.21.15.97:0/1642870294 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f52bc0075e0 con 0x7f52cc1583a0 2023-07-06T03:45:23.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.224+0000 7f52c8ff9700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f52ac06adb0 0x7f52ac06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.224+0000 7f52c8ff9700 1 -- 172.21.15.97:0/1642870294 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52bc089550 con 0x7f52cc1583a0 2023-07-06T03:45:23.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.224+0000 7f52c8ff9700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] conn(0x7f52ac0735e0 0x7f52ac0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.225+0000 7f52c8ff9700 1 -- 172.21.15.97:0/1642870294 --> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52ac0760a0 con 0x7f52ac0735e0 2023-07-06T03:45:23.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.225+0000 7f52c8ff9700 1 -- 172.21.15.97:0/1642870294 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52bc001dc0 con 0x7f52cc1583a0 2023-07-06T03:45:23.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.231+0000 7f9708dff700 1 -- 172.21.15.97:0/2419863093 --> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9704148020 con 0x7f96e4073510 2023-07-06T03:45:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.225+0000 7f52cbfff700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f52ac06adb0 0x7f52ac06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.226+0000 7f52cbfff700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f52ac06adb0 0x7f52ac06d260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f52c4001600 tx=0x7f52c4007a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.227+0000 7f52d099c700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] conn(0x7f52ac0735e0 0x7f52ac0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.228+0000 7f52d099c700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] conn(0x7f52ac0735e0 0x7f52ac0759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.229+0000 7f52c8ff9700 1 -- 172.21.15.97:0/1642870294 <== osd.7 v2:172.21.15.146:6808/1299847983 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f52ac0760a0 con 0x7f52ac0735e0 2023-07-06T03:45:23.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.232+0000 7f67921ff700 1 -- 172.21.15.97:0/239111363 --> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f678c0caf00 con 0x7f676c0735e0 2023-07-06T03:45:23.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.232+0000 7f96faffd700 1 -- 172.21.15.97:0/2419863093 <== osd.2 v2:172.21.15.97:6824/3131825859 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9704148020 con 0x7f96e4073510 2023-07-06T03:45:23.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.233+0000 7f6783fff700 1 -- 172.21.15.97:0/239111363 <== osd.0 v2:172.21.15.97:6800/3690379650 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f67840041f0 con 0x7f676c0735e0 2023-07-06T03:45:23.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.233+0000 7f67921ff700 1 -- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] conn(0x7f676c0735e0 msgr2=0x7f676c0759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.233+0000 7f67921ff700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] conn(0x7f676c0735e0 0x7f676c0759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.233+0000 7f67921ff700 1 -- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f676c06adb0 msgr2=0x7f676c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.233+0000 7f67921ff700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f676c06adb0 0x7f676c06d260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f678c1ece10 tx=0x7f677400d040 comp rx=0 tx=0).stop 2023-07-06T03:45:23.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.233+0000 7f67921ff700 1 -- 172.21.15.97:0/239111363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f678c0ccbd0 msgr2=0x7f678c15f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.233+0000 7f67921ff700 1 --2- 172.21.15.97:0/239111363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f678c0ccbd0 0x7f678c15f410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f677c00b530 tx=0x7f677c00b840 comp rx=0 tx=0).stop 2023-07-06T03:45:23.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.234+0000 7f678bfff700 1 -- 172.21.15.97:0/239111363 reap_dead start 2023-07-06T03:45:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.234+0000 7f9708dff700 1 -- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] conn(0x7f96e4073510 msgr2=0x7f96e4075920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.234+0000 7f9708dff700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] conn(0x7f96e4073510 0x7f96e4075920 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.234+0000 7f9708dff700 1 -- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f96e406ace0 msgr2=0x7f96e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.234+0000 7f9708dff700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f96e406ace0 0x7f96e406d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f96f40011e0 tx=0x7f96f4009040 comp rx=0 tx=0).stop 2023-07-06T03:45:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.234+0000 7f9708dff700 1 -- 172.21.15.97:0/2419863093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9704160a20 msgr2=0x7f97040b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.234+0000 7f9708dff700 1 --2- 172.21.15.97:0/2419863093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9704160a20 0x7f97040b3e00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f96fc016000 tx=0x7f96fc004ad0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.235+0000 7f67921ff700 1 -- 172.21.15.97:0/239111363 shutdown_connections 2023-07-06T03:45:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.235+0000 7f67921ff700 1 -- 172.21.15.97:0/239111363 >> 172.21.15.97:0/239111363 conn(0x7f678c0cf610 msgr2=0x7f678c06cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.235+0000 7f67921ff700 1 -- 172.21.15.97:0/239111363 shutdown_connections 2023-07-06T03:45:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.235+0000 7f67921ff700 1 -- 172.21.15.97:0/239111363 wait complete. 2023-07-06T03:45:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.235+0000 7f9702d9d700 1 -- 172.21.15.97:0/2419863093 reap_dead start 2023-07-06T03:45:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.238+0000 7f9708dff700 1 -- 172.21.15.97:0/2419863093 shutdown_connections 2023-07-06T03:45:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.238+0000 7f9708dff700 1 -- 172.21.15.97:0/2419863093 >> 172.21.15.97:0/2419863093 conn(0x7f97040cf610 msgr2=0x7f970415ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.241+0000 7f04da5ff700 1 -- 172.21.15.97:0/2436314314 --> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f04d415f390 con 0x7f04c0073660 2023-07-06T03:45:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.241+0000 7f9708dff700 1 -- 172.21.15.97:0/2419863093 shutdown_connections 2023-07-06T03:45:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.241+0000 7f9708dff700 1 -- 172.21.15.97:0/2419863093 wait complete. 2023-07-06T03:45:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.241+0000 7f82695ff700 1 -- 172.21.15.97:0/862476165 --> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f82640d04a0 con 0x7f8250073660 2023-07-06T03:45:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.242+0000 7f824b7fe700 1 -- 172.21.15.97:0/862476165 <== osd.1 v2:172.21.15.97:6808/3448212434 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8258022040 con 0x7f8250073660 2023-07-06T03:45:23.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.243+0000 7f82695ff700 1 -- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] conn(0x7f8250073660 msgr2=0x7f8250075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.243+0000 7f82695ff700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] conn(0x7f8250073660 0x7f8250075a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.243+0000 7f82695ff700 1 -- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f825006adb0 msgr2=0x7f825006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.243+0000 7f82695ff700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f825006adb0 0x7f825006d260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82640d19d0 tx=0x7f8254009210 comp rx=0 tx=0).stop 2023-07-06T03:45:23.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.243+0000 7f82695ff700 1 -- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82640d2210 msgr2=0x7f82641f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.243+0000 7f82695ff700 1 --2- 172.21.15.97:0/862476165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82640d2210 0x7f82641f6a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f825c00cbd0 tx=0x7f825c00cee0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.243+0000 7f82637fe700 1 -- 172.21.15.97:0/862476165 reap_dead start 2023-07-06T03:45:23.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.243+0000 7f82695ff700 1 -- 172.21.15.97:0/862476165 shutdown_connections 2023-07-06T03:45:23.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.243+0000 7f82695ff700 1 -- 172.21.15.97:0/862476165 >> 172.21.15.97:0/862476165 conn(0x7f826406a6c0 msgr2=0x7f826415b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.246+0000 7f82695ff700 1 -- 172.21.15.97:0/862476165 shutdown_connections 2023-07-06T03:45:23.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.246+0000 7f82695ff700 1 -- 172.21.15.97:0/862476165 wait complete. 2023-07-06T03:45:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.246+0000 7f04d0ff9700 1 -- 172.21.15.97:0/2436314314 <== osd.5 v2:172.21.15.146:6809/3757512297 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f04c4003ce0 con 0x7f04c0073660 2023-07-06T03:45:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.245+0000 7f093f9ff700 1 -- 172.21.15.97:0/4099582719 --> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f093815a030 con 0x7f091c073590 2023-07-06T03:45:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.247+0000 7f092dffb700 1 -- 172.21.15.97:0/4099582719 <== osd.3 v2:172.21.15.97:6809/1358667479 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0928001550 con 0x7f091c073590 2023-07-06T03:45:23.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.247+0000 7f093f9ff700 1 -- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] conn(0x7f091c073590 msgr2=0x7f091c0759a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.247+0000 7f093f9ff700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] conn(0x7f091c073590 0x7f091c0759a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f093f9ff700 1 -- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f091c06ace0 msgr2=0x7f091c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f093f9ff700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f091c06ace0 0x7f091c06d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09381f2510 tx=0x7f0934008040 comp rx=0 tx=0).stop 2023-07-06T03:45:23.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f093f9ff700 1 -- 172.21.15.97:0/4099582719 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09381ee8e0 msgr2=0x7f093806cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f093f9ff700 1 --2- 172.21.15.97:0/4099582719 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09381ee8e0 0x7f093806cc80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f093000b780 tx=0x7f093000bb40 comp rx=0 tx=0).stop 2023-07-06T03:45:23.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f093df9c700 1 -- 172.21.15.97:0/4099582719 reap_dead start 2023-07-06T03:45:23.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f04da5ff700 1 -- 172.21.15.97:0/2436314314 >> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] conn(0x7f04c0073660 msgr2=0x7f04c0075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f04da5ff700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] conn(0x7f04c0073660 0x7f04c0075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f04da5ff700 1 -- 172.21.15.97:0/2436314314 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f04c006adb0 msgr2=0x7f04c006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f04da5ff700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f04c006adb0 0x7f04c006d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009770 tx=0x7f04cc006cd0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f04da5ff700 1 -- 172.21.15.97:0/2436314314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d40cca60 msgr2=0x7f04d40b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f04da5ff700 1 --2- 172.21.15.97:0/2436314314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04d40cca60 0x7f04d40b3b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f04c800e600 tx=0x7f04c800e910 comp rx=0 tx=0).stop 2023-07-06T03:45:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.248+0000 7f04d8b9c700 1 -- 172.21.15.97:0/2436314314 reap_dead start 2023-07-06T03:45:23.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.251+0000 7fb9e83ff700 1 -- 172.21.15.97:0/2885145641 --> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9e00c4410 con 0x7fb9c4073660 2023-07-06T03:45:23.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.251+0000 7f04da5ff700 1 -- 172.21.15.97:0/2436314314 shutdown_connections 2023-07-06T03:45:23.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.251+0000 7f04da5ff700 1 -- 172.21.15.97:0/2436314314 >> 172.21.15.97:0/2436314314 conn(0x7f04d406a060 msgr2=0x7f04d415d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.252+0000 7f093f9ff700 1 -- 172.21.15.97:0/4099582719 shutdown_connections 2023-07-06T03:45:23.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.252+0000 7f093f9ff700 1 -- 172.21.15.97:0/4099582719 >> 172.21.15.97:0/4099582719 conn(0x7f09380b1650 msgr2=0x7f09380c5930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.252+0000 7f04da5ff700 1 -- 172.21.15.97:0/2436314314 shutdown_connections 2023-07-06T03:45:23.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.252+0000 7f04da5ff700 1 -- 172.21.15.97:0/2436314314 wait complete. 2023-07-06T03:45:23.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.254+0000 7f093f9ff700 1 -- 172.21.15.97:0/4099582719 shutdown_connections 2023-07-06T03:45:23.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.254+0000 7f093f9ff700 1 -- 172.21.15.97:0/4099582719 wait complete. 2023-07-06T03:45:23.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.254+0000 7fb9d6ffd700 1 -- 172.21.15.97:0/2885145641 <== osd.4 v2:172.21.15.146:6800/663749144 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb9dc0078e0 con 0x7fb9c4073660 2023-07-06T03:45:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.255+0000 7fb9e83ff700 1 -- 172.21.15.97:0/2885145641 >> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] conn(0x7fb9c4073660 msgr2=0x7fb9c4075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.255+0000 7fb9e83ff700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] conn(0x7fb9c4073660 0x7fb9c4075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.255+0000 7fb9e83ff700 1 -- 172.21.15.97:0/2885145641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb9c406adb0 msgr2=0x7fb9c406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.255+0000 7fb9e83ff700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb9c406adb0 0x7fb9c406d260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0073d50 tx=0x7fb9d800b040 comp rx=0 tx=0).stop 2023-07-06T03:45:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.255+0000 7fb9e83ff700 1 -- 172.21.15.97:0/2885145641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e015c0c0 msgr2=0x7fb9e01574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.255+0000 7fb9e83ff700 1 --2- 172.21.15.97:0/2885145641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e015c0c0 0x7fb9e01574b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0066b70 tx=0x7fb9d000b750 comp rx=0 tx=0).stop 2023-07-06T03:45:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.256+0000 7fb9e699c700 1 -- 172.21.15.97:0/2885145641 reap_dead start 2023-07-06T03:45:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.256+0000 7fd3aefff700 1 -- 172.21.15.97:0/1917346426 --> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd3a80d10c0 con 0x7fd38c073560 2023-07-06T03:45:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.257+0000 7fb9e83ff700 1 -- 172.21.15.97:0/2885145641 shutdown_connections 2023-07-06T03:45:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.257+0000 7fb9e83ff700 1 -- 172.21.15.97:0/2885145641 >> 172.21.15.97:0/2885145641 conn(0x7fb9e00cf610 msgr2=0x7fb9e00ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.257+0000 7fb9e83ff700 1 -- 172.21.15.97:0/2885145641 shutdown_connections 2023-07-06T03:45:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.257+0000 7fb9e83ff700 1 -- 172.21.15.97:0/2885145641 wait complete. 2023-07-06T03:45:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.258+0000 7fd39d7fa700 1 -- 172.21.15.97:0/1917346426 <== osd.6 v2:172.21.15.146:6824/1310854864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd398009620 con 0x7fd38c073560 2023-07-06T03:45:23.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.259+0000 7fd3aefff700 1 -- 172.21.15.97:0/1917346426 >> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] conn(0x7fd38c073560 msgr2=0x7fd38c075970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.259+0000 7fd3aefff700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] conn(0x7fd38c073560 0x7fd38c075970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.260+0000 7fd3aefff700 1 -- 172.21.15.97:0/1917346426 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd38c06acb0 msgr2=0x7fd38c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.260+0000 7fd3aefff700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd38c06acb0 0x7fd38c06d160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009b90 tx=0x7fd3a4008030 comp rx=0 tx=0).stop 2023-07-06T03:45:23.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.260+0000 7fd3aefff700 1 -- 172.21.15.97:0/1917346426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a80cd940 msgr2=0x7fd3a8070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.260+0000 7fd3aefff700 1 --2- 172.21.15.97:0/1917346426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3a80cd940 0x7fd3a8070810 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000caf0 tx=0x7fd3a000ce00 comp rx=0 tx=0).stop 2023-07-06T03:45:23.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.262+0000 7fd3ad59c700 1 -- 172.21.15.97:0/1917346426 reap_dead start 2023-07-06T03:45:23.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.262+0000 7fd3aefff700 1 -- 172.21.15.97:0/1917346426 shutdown_connections 2023-07-06T03:45:23.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.262+0000 7fd3aefff700 1 -- 172.21.15.97:0/1917346426 >> 172.21.15.97:0/1917346426 conn(0x7fd3a806b2d0 msgr2=0x7fd3a81539e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.263+0000 7fd3aefff700 1 -- 172.21.15.97:0/1917346426 shutdown_connections 2023-07-06T03:45:23.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.263+0000 7fd3aefff700 1 -- 172.21.15.97:0/1917346426 wait complete. 2023-07-06T03:45:23.314 INFO:teuthology.orchestra.run.smithi097.stdout:55834574857 2023-07-06T03:45:23.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-06T03:45:23.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.267+0000 7f52d23ff700 1 -- 172.21.15.97:0/1642870294 --> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52cc0cb2e0 con 0x7f52ac0735e0 2023-07-06T03:45:23.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.267+0000 7f52c8ff9700 1 -- 172.21.15.97:0/1642870294 <== osd.7 v2:172.21.15.146:6808/1299847983 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f52b40045e0 con 0x7f52ac0735e0 2023-07-06T03:45:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.268+0000 7f52d23ff700 1 -- 172.21.15.97:0/1642870294 >> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] conn(0x7f52ac0735e0 msgr2=0x7f52ac0759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.268+0000 7f52d23ff700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] conn(0x7f52ac0735e0 0x7f52ac0759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.268+0000 7f52d23ff700 1 -- 172.21.15.97:0/1642870294 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f52ac06adb0 msgr2=0x7f52ac06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.268+0000 7f52d23ff700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f52ac06adb0 0x7f52ac06d260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f52c4001600 tx=0x7f52c4007a20 comp rx=0 tx=0).stop 2023-07-06T03:45:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.268+0000 7f52d23ff700 1 -- 172.21.15.97:0/1642870294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc1583a0 msgr2=0x7f52cc0b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.268+0000 7f52d23ff700 1 --2- 172.21.15.97:0/1642870294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f52cc1583a0 0x7f52cc0b3290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f52bc001ff0 tx=0x7f52bc00a340 comp rx=0 tx=0).stop 2023-07-06T03:45:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.269+0000 7f52d099c700 1 -- 172.21.15.97:0/1642870294 reap_dead start 2023-07-06T03:45:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.269+0000 7f52d23ff700 1 -- 172.21.15.97:0/1642870294 shutdown_connections 2023-07-06T03:45:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.269+0000 7f52d23ff700 1 -- 172.21.15.97:0/1642870294 >> 172.21.15.97:0/1642870294 conn(0x7f52cc06a060 msgr2=0x7f52cc072fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.269+0000 7f52d23ff700 1 -- 172.21.15.97:0/1642870294 shutdown_connections 2023-07-06T03:45:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.269+0000 7f52d23ff700 1 -- 172.21.15.97:0/1642870294 wait complete. 2023-07-06T03:45:23.319 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:23.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-06T03:45:23.320 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:23.320 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-06T03:45:23.321 INFO:teuthology.orchestra.run.smithi097.stdout:55834574857 2023-07-06T03:45:23.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-06T03:45:23.322 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:23.323 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-06T03:45:23.324 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-07-06T03:45:23.324 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-06T03:45:23.325 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:23.325 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-06T03:45:23.326 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:23.326 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-06T03:45:23.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.703+0000 7fa9c17ff700 1 Processor -- start 2023-07-06T03:45:23.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.703+0000 7fa9c17ff700 1 -- start start 2023-07-06T03:45:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.706+0000 7fa9c17ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa9bc160a20 0x7fa9bc0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.707+0000 7fa9c17ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9bc0ccbd0 0x7fa9bc0c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.707+0000 7fa9c17ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9bc0caa30 con 0x7fa9bc0ce3c0 2023-07-06T03:45:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.707+0000 7fa9c17ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc160df0 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.707+0000 7fa9c17ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc0cac60 con 0x7fa9bc160a20 2023-07-06T03:45:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.708+0000 7fa9baffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa9bc160a20 0x7fa9bc0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.708+0000 7fa9baffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa9bc160a20 0x7fa9bc0c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57598/0 (socket says 172.21.15.97:57598) 2023-07-06T03:45:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.708+0000 7fa9baffd700 1 -- 172.21.15.97:0/3632848204 learned_addr learned my addr 172.21.15.97:0/3632848204 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.708+0000 7fa9baffd700 1 -- 172.21.15.97:0/3632848204 >> v2:172.21.15.97:3300/0 conn(0x7fa9bc0ccbd0 msgr2=0x7fa9bc0c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.708+0000 7fa9baffd700 1 --2- 172.21.15.97:0/3632848204 >> v2:172.21.15.97:3300/0 conn(0x7fa9bc0ccbd0 0x7fa9bc0c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.708+0000 7fa9baffd700 1 -- 172.21.15.97:0/3632848204 >> v1:172.21.15.97:6789/0 conn(0x7fa9bc0ce3c0 legacy=0x7fa9bc0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.708+0000 7fa9baffd700 1 -- 172.21.15.97:0/3632848204 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0cb1e0 con 0x7fa9bc160a20 2023-07-06T03:45:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.709+0000 7fa9baffd700 1 --2- 172.21.15.97:0/3632848204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9bc160a20 0x7fa9bc0c8f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc06f210 tx=0x7fa9ac009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aafeef1c620f7054 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.709+0000 7fa9b9ffb700 1 -- 172.21.15.97:0/3632848204 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac004230 con 0x7fa9bc160a20 2023-07-06T03:45:23.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.710+0000 7fa9b9ffb700 1 -- 172.21.15.97:0/3632848204 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac004390 con 0x7fa9bc160a20 2023-07-06T03:45:23.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.710+0000 7fa9b9ffb700 1 -- 172.21.15.97:0/3632848204 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac0036a0 con 0x7fa9bc160a20 2023-07-06T03:45:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.711+0000 7fa9c17ff700 1 -- 172.21.15.97:0/3632848204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9bc160a20 msgr2=0x7fa9bc0c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.711+0000 7fa9c17ff700 1 --2- 172.21.15.97:0/3632848204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9bc160a20 0x7fa9bc0c8f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc06f210 tx=0x7fa9ac009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.711+0000 7fa9c17ff700 1 -- 172.21.15.97:0/3632848204 shutdown_connections 2023-07-06T03:45:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.711+0000 7fa9c17ff700 1 --2- 172.21.15.97:0/3632848204 >> v2:172.21.15.97:3300/0 conn(0x7fa9bc0ccbd0 0x7fa9bc0c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.711+0000 7fa9c17ff700 1 --2- 172.21.15.97:0/3632848204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9bc160a20 0x7fa9bc0c8f40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.711+0000 7fa9c17ff700 1 -- 172.21.15.97:0/3632848204 >> 172.21.15.97:0/3632848204 conn(0x7fa9bc0cf610 msgr2=0x7fa9bc0cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.712+0000 7fa9c17ff700 1 -- 172.21.15.97:0/3632848204 shutdown_connections 2023-07-06T03:45:23.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.712+0000 7fa9c17ff700 1 -- 172.21.15.97:0/3632848204 wait complete. 2023-07-06T03:45:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.714+0000 7fa9c17ff700 1 Processor -- start 2023-07-06T03:45:23.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.715+0000 7fa9c17ff700 1 -- start start 2023-07-06T03:45:23.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.716+0000 7fa9c17ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9bc0ccbd0 0x7fa9bc0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.716+0000 7fa9c17ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9bc0ce3c0 0x7fa9bc0b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.716+0000 7fa9c17ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc160a20 0x7fa9bc14b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.716+0000 7fa9c17ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0afa80 con 0x7fa9bc160a20 2023-07-06T03:45:23.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.716+0000 7fa9c17ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0aee00 con 0x7fa9bc0ce3c0 2023-07-06T03:45:23.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.716+0000 7fa9c17ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0af0e0 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.718+0000 7fa9ba7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9bc0ce3c0 0x7fa9bc0b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.718+0000 7fa9baffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9bc0ccbd0 0x7fa9bc0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.717+0000 7ff5cf5ff700 1 Processor -- start 2023-07-06T03:45:23.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.717+0000 7ff5cf5ff700 1 -- start start 2023-07-06T03:45:23.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.719+0000 7fa9ba7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9bc0ce3c0 0x7fa9bc0b3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57614/0 (socket says 172.21.15.97:57614) 2023-07-06T03:45:23.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.719+0000 7fa9ba7fc700 1 -- 172.21.15.97:0/719381177 learned_addr learned my addr 172.21.15.97:0/719381177 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.719+0000 7fa9bb7fe700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc160a20 0x7fa9bc14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.719+0000 7f49079ff700 1 Processor -- start 2023-07-06T03:45:23.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.720+0000 7fa9baffd700 1 -- 172.21.15.97:0/719381177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9bc0ce3c0 msgr2=0x7fa9bc0b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.720+0000 7fa9baffd700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9bc0ce3c0 0x7fa9bc0b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.721+0000 7fa9baffd700 1 -- 172.21.15.97:0/719381177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc160a20 msgr2=0x7fa9bc14b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.721+0000 7fa9baffd700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc160a20 0x7fa9bc14b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.721+0000 7fa9baffd700 1 -- 172.21.15.97:0/719381177 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac009550 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.720+0000 7ff5cf5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff5c80cd950 0x7ff5c8161930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.720+0000 7ff5cf5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5c81644a0 0x7ff5c80c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.720+0000 7ff5cf5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cdd20 0x7ff5c80ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.720+0000 7ff5cf5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c806c1a0 con 0x7ff5c81644a0 2023-07-06T03:45:23.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.721+0000 7ff5cf5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c8161e70 con 0x7ff5c80cd950 2023-07-06T03:45:23.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.721+0000 7ff5cf5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c8164930 con 0x7ff5c80cdd20 2023-07-06T03:45:23.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.721+0000 7fa9baffd700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9bc0ccbd0 0x7fa9bc0ae540 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac015000 tx=0x7fa9ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.720+0000 7f49079ff700 1 -- start start 2023-07-06T03:45:23.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.722+0000 7fa9a37fe700 1 -- 172.21.15.97:0/719381177 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac011630 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.722+0000 7fa9a37fe700 1 -- 172.21.15.97:0/719381177 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac011790 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.722+0000 7fa9a37fe700 1 -- 172.21.15.97:0/719381177 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac011b40 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.722+0000 7ff5cd39b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff5c80cd950 0x7ff5c8161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.723+0000 7ff5cdb9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cdd20 0x7ff5c80ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.722+0000 7f49079ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000ce3d0 0x7f49000adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.722+0000 7f49079ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000c3b70 con 0x7f49000cbb10 2023-07-06T03:45:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.722+0000 7f49079ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000c3e50 con 0x7f490015c0c0 2023-07-06T03:45:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.722+0000 7f49079ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49000c47f0 con 0x7f49000ce3d0 2023-07-06T03:45:23.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.723+0000 7f4904f9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000ce3d0 0x7f49000adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.723+0000 7f4904f9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000ce3d0 0x7f49000adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54020/0 (socket says 172.21.15.97:54020) 2023-07-06T03:45:23.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.723+0000 7f4904f9a700 1 -- 172.21.15.97:0/1439967770 learned_addr learned my addr 172.21.15.97:0/1439967770 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.723+0000 7ff5cdb9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cdd20 0x7ff5c80ad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54006/0 (socket says 172.21.15.97:54006) 2023-07-06T03:45:23.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.723+0000 7ff5cdb9c700 1 -- 172.21.15.97:0/1017489763 learned_addr learned my addr 172.21.15.97:0/1017489763 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.723+0000 7f4904f9a700 1 -- 172.21.15.97:0/1439967770 >> v1:172.21.15.146:6789/0 conn(0x7f490015c0c0 legacy=0x7f49000c8f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T03:45:23.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.723+0000 7f4904f9a700 1 -- 172.21.15.97:0/1439967770 >> v1:172.21.15.97:6789/0 conn(0x7f49000cbb10 legacy=0x7f49000c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.723+0000 7f4904f9a700 1 -- 172.21.15.97:0/1439967770 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49000c4130 con 0x7f49000ce3d0 2023-07-06T03:45:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.724+0000 7ff5cd39b700 1 -- 172.21.15.97:0/1017489763 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cdd20 msgr2=0x7ff5c80ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.724+0000 7ff5cd39b700 1 --2- 172.21.15.97:0/1017489763 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cdd20 0x7ff5c80ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.724+0000 7ff5cd39b700 1 -- 172.21.15.97:0/1017489763 >> v2:172.21.15.97:3300/0 conn(0x7ff5c81644a0 msgr2=0x7ff5c80c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.724+0000 7ff5cd39b700 1 --2- 172.21.15.97:0/1017489763 >> v2:172.21.15.97:3300/0 conn(0x7ff5c81644a0 0x7ff5c80c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.724+0000 7ff5cd39b700 1 -- 172.21.15.97:0/1017489763 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8164c10 con 0x7ff5c80cd950 2023-07-06T03:45:23.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.724+0000 7ff5cd39b700 1 --2- 172.21.15.97:0/1017489763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5c80cd950 0x7ff5c8161930 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80cf940 tx=0x7ff5bc009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76999e666477ff07 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.725+0000 7f4904f9a700 1 --2- 172.21.15.97:0/1439967770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000ce3d0 0x7f49000adc80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f48f0013210 tx=0x7f48f0013520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d87bc3d9690257eb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.725+0000 7f48f7fff700 1 -- 172.21.15.97:0/1439967770 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0016070 con 0x7f49000ce3d0 2023-07-06T03:45:23.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.725+0000 7f48f7fff700 1 -- 172.21.15.97:0/1439967770 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f0013e20 con 0x7f49000ce3d0 2023-07-06T03:45:23.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.725+0000 7f48f7fff700 1 -- 172.21.15.97:0/1439967770 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0004250 con 0x7f49000ce3d0 2023-07-06T03:45:23.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.726+0000 7f49079ff700 1 -- 172.21.15.97:0/1439967770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000ce3d0 msgr2=0x7f49000adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.726+0000 7f49079ff700 1 --2- 172.21.15.97:0/1439967770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000ce3d0 0x7f49000adc80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f48f0013210 tx=0x7f48f0013520 comp rx=0 tx=0).stop 2023-07-06T03:45:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.725+0000 7ff5c7fff700 1 -- 172.21.15.97:0/1017489763 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc011830 con 0x7ff5c80cd950 2023-07-06T03:45:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.725+0000 7ff5c7fff700 1 -- 172.21.15.97:0/1017489763 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc011990 con 0x7ff5c80cd950 2023-07-06T03:45:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.725+0000 7ff5c7fff700 1 -- 172.21.15.97:0/1017489763 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc011d20 con 0x7ff5c80cd950 2023-07-06T03:45:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.726+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/1017489763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5c80cd950 msgr2=0x7ff5c8161930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.726+0000 7ff5cf5ff700 1 --2- 172.21.15.97:0/1017489763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5c80cd950 0x7ff5c8161930 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80cf940 tx=0x7ff5bc009b20 comp rx=0 tx=0).stop 2023-07-06T03:45:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.727+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/1017489763 shutdown_connections 2023-07-06T03:45:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.727+0000 7ff5cf5ff700 1 --2- 172.21.15.97:0/1017489763 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cdd20 0x7ff5c80ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.727+0000 7ff5cf5ff700 1 --2- 172.21.15.97:0/1017489763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5c80cd950 0x7ff5c8161930 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.727+0000 7ff5cf5ff700 1 --2- 172.21.15.97:0/1017489763 >> v2:172.21.15.97:3300/0 conn(0x7ff5c81644a0 0x7ff5c80c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.727+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/1017489763 >> 172.21.15.97:0/1017489763 conn(0x7ff5c806a060 msgr2=0x7ff5c806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.727+0000 7fa9c17ff700 1 -- 172.21.15.97:0/719381177 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9bc0af3c0 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.727+0000 7fb4989ff700 1 Processor -- start 2023-07-06T03:45:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.727+0000 7fb4989ff700 1 -- start start 2023-07-06T03:45:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.728+0000 7fb8841ff700 1 Processor -- start 2023-07-06T03:45:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.727+0000 7fa9c17ff700 1 -- 172.21.15.97:0/719381177 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc0aea80 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.728+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/1017489763 shutdown_connections 2023-07-06T03:45:23.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.729+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/1017489763 wait complete. 2023-07-06T03:45:23.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.729+0000 7fa9c17ff700 1 -- 172.21.15.97:0/719381177 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9bc0adf10 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.728+0000 7fb8841ff700 1 -- start start 2023-07-06T03:45:23.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.730+0000 7fb8841ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb87c160770 0x7fb87c0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.728+0000 7f49079ff700 1 -- 172.21.15.97:0/1439967770 shutdown_connections 2023-07-06T03:45:23.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.728+0000 7f49079ff700 1 --2- 172.21.15.97:0/1439967770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000ce3d0 0x7f49000adc80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.728+0000 7f49079ff700 1 -- 172.21.15.97:0/1439967770 >> 172.21.15.97:0/1439967770 conn(0x7f49000cf610 msgr2=0x7f49000cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7ff5cf5ff700 1 Processor -- start 2023-07-06T03:45:23.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7f70a35ff700 1 Processor -- start 2023-07-06T03:45:23.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7fb4989ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb49415c610 0x7fb4940b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7fb4989ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49406b360 0x7fb4940b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7fb4989ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4940cff90 con 0x7fb4940d2210 2023-07-06T03:45:23.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7fb4989ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4940d0780 con 0x7fb49415c610 2023-07-06T03:45:23.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7fb4989ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49415c9e0 con 0x7fb49406b360 2023-07-06T03:45:23.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7fb8841ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87c0c4a60 0x7fb87c0c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7fb88179a700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb87c0adb80 0x7fb87c0ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:35312/0 (socket says 172.21.15.97:35312) 2023-07-06T03:45:23.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7fb88179a700 1 -- 172.21.15.97:0/2485630833 learned_addr learned my addr 172.21.15.97:0/2485630833 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:23.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7fb8841ff700 1 -- 172.21.15.97:0/2485630833 --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87c0c5400 con 0x7fb87c0adb80 2023-07-06T03:45:23.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7fb8841ff700 1 -- 172.21.15.97:0/2485630833 --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87c0b1480 con 0x7fb87c160770 2023-07-06T03:45:23.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7fb8841ff700 1 -- 172.21.15.97:0/2485630833 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c06c5c0 con 0x7fb87c0c4a60 2023-07-06T03:45:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.731+0000 7f70a35ff700 1 -- start start 2023-07-06T03:45:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7fb489d9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49406b360 0x7fb4940b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7fb489d9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49406b360 0x7fb4940b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54028/0 (socket says 172.21.15.97:54028) 2023-07-06T03:45:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7fb489d9b700 1 -- 172.21.15.97:0/226832959 learned_addr learned my addr 172.21.15.97:0/226832959 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7fb88279c700 1 --2- 172.21.15.97:0/2485630833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87c0c4a60 0x7fb87c0c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7fb881f9b700 1 --2- 172.21.15.97:0/2485630833 >> v2:172.21.15.146:3300/0 conn(0x7fb87c160770 0x7fb87c0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7fb88279c700 1 -- 172.21.15.97:0/2485630833 >> v2:172.21.15.146:3300/0 conn(0x7fb87c160770 msgr2=0x7fb87c0b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7fb88279c700 1 --2- 172.21.15.97:0/2485630833 >> v2:172.21.15.146:3300/0 conn(0x7fb87c160770 0x7fb87c0b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7fb489d9b700 1 -- 172.21.15.97:0/226832959 >> v2:172.21.15.146:3300/0 conn(0x7fb49415c610 msgr2=0x7fb4940b47b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T03:45:23.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7fb489d9b700 1 --2- 172.21.15.97:0/226832959 >> v2:172.21.15.146:3300/0 conn(0x7fb49415c610 0x7fb4940b47b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7fb489d9b700 1 -- 172.21.15.97:0/226832959 >> v1:172.21.15.146:6789/0 conn(0x7fb4940d2210 legacy=0x7fb4940d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7fb489d9b700 1 -- 172.21.15.97:0/226832959 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4940b5650 con 0x7fb49406b360 2023-07-06T03:45:23.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7fb489d9b700 1 --2- 172.21.15.97:0/226832959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49406b360 0x7fb4940b4cf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4840098f0 tx=0x7fb484009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2c9e5a3dee1bfee server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.732+0000 7ff5cf5ff700 1 -- start start 2023-07-06T03:45:23.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7ff5cf5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cd950 0x7ff5c80aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7ff5cf5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5c80cdd20 0x7ff5c81f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7ff5cf5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c81644a0 0x7ff5c80b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7ff5cf5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80aafa0 con 0x7ff5c81644a0 2023-07-06T03:45:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7ff5cf5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80b2950 con 0x7ff5c80cdd20 2023-07-06T03:45:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7ff5cf5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c81fa1a0 con 0x7ff5c80cd950 2023-07-06T03:45:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7f49079ff700 1 -- 172.21.15.97:0/1439967770 shutdown_connections 2023-07-06T03:45:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7f70a35ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f709c0ce3d0 0x7f709c0add10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7f70a35ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 0x7f709c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7f70a35ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709c0c3b70 con 0x7f709c0cbb10 2023-07-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7f70a35ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709c0c47f0 con 0x7f709c0ce3d0 2023-07-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7f70a35ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0c3e50 con 0x7f709c15c0c0 2023-07-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7fb88279c700 1 -- 172.21.15.97:0/2485630833 >> v1:172.21.15.146:6789/0 conn(0x7fb87c0adb80 legacy=0x7fb87c0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7fb88279c700 1 -- 172.21.15.97:0/2485630833 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0ce760 con 0x7fb87c0c4a60 2023-07-06T03:45:23.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7fb88279c700 1 --2- 172.21.15.97:0/2485630833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87c0c4a60 0x7fb87c0c4ec0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb878009d90 tx=0x7fb87800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b73e376bceaa0f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7fb491d9b700 1 -- 172.21.15.97:0/226832959 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400b8a0 con 0x7fb49406b360 2023-07-06T03:45:23.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7fb491d9b700 1 -- 172.21.15.97:0/226832959 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48400ba00 con 0x7fb49406b360 2023-07-06T03:45:23.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7fb491d9b700 1 -- 172.21.15.97:0/226832959 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400bd90 con 0x7fb49406b360 2023-07-06T03:45:23.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7f70a1b9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 0x7f709c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7f70a0b9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f709c0ce3d0 0x7f709c0add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7f70a1b9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 0x7f709c0c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54052/0 (socket says 172.21.15.97:54052) 2023-07-06T03:45:23.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7f70a1b9c700 1 -- 172.21.15.97:0/3119548890 learned_addr learned my addr 172.21.15.97:0/3119548890 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.733+0000 7f49079ff700 1 -- 172.21.15.97:0/1439967770 wait complete. 2023-07-06T03:45:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7f70a1b9c700 1 -- 172.21.15.97:0/3119548890 >> v2:172.21.15.146:3300/0 conn(0x7f709c0ce3d0 msgr2=0x7f709c0add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7f70a1b9c700 1 --2- 172.21.15.97:0/3119548890 >> v2:172.21.15.146:3300/0 conn(0x7f709c0ce3d0 0x7f709c0add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7f70a1b9c700 1 -- 172.21.15.97:0/3119548890 >> v1:172.21.15.146:6789/0 conn(0x7f709c0cbb10 legacy=0x7f709c0c4c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T03:45:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7f70a1b9c700 1 -- 172.21.15.97:0/3119548890 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0c4130 con 0x7f709c15c0c0 2023-07-06T03:45:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7ff5cd39b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cd950 0x7ff5c80aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7ff5cd39b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cd950 0x7ff5c80aaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54044/0 (socket says 172.21.15.97:54044) 2023-07-06T03:45:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.734+0000 7ff5cd39b700 1 -- 172.21.15.97:0/716760588 learned_addr learned my addr 172.21.15.97:0/716760588 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7f70a1b9c700 1 --2- 172.21.15.97:0/3119548890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 0x7f709c0c8f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7094009e50 tx=0x7f709400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd74d0f42abf6b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7ff5cd39b700 1 -- 172.21.15.97:0/716760588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5c80cdd20 msgr2=0x7ff5c81f9950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T03:45:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7ff5cd39b700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5c80cdd20 0x7ff5c81f9950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7ff5cd39b700 1 -- 172.21.15.97:0/716760588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c81644a0 msgr2=0x7ff5c80b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7ff5cd39b700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c81644a0 0x7ff5c80b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7ff5cd39b700 1 -- 172.21.15.97:0/716760588 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc0094a0 con 0x7ff5c80cd950 2023-07-06T03:45:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7fb4989ff700 1 -- 172.21.15.97:0/226832959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49406b360 msgr2=0x7fb4940b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7fb4989ff700 1 --2- 172.21.15.97:0/226832959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49406b360 0x7fb4940b4cf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4840098f0 tx=0x7fb484009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7fb4989ff700 1 -- 172.21.15.97:0/226832959 shutdown_connections 2023-07-06T03:45:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7fb4989ff700 1 --2- 172.21.15.97:0/226832959 >> v2:172.21.15.146:3300/0 conn(0x7fb49415c610 0x7fb4940b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7fb4989ff700 1 --2- 172.21.15.97:0/226832959 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49406b360 0x7fb4940b4cf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7fb4989ff700 1 -- 172.21.15.97:0/226832959 >> 172.21.15.97:0/226832959 conn(0x7fb49406a6c0 msgr2=0x7fb49406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7fb4989ff700 1 -- 172.21.15.97:0/226832959 shutdown_connections 2023-07-06T03:45:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.736+0000 7ff5cd39b700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cd950 0x7ff5c80aaa60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc000c00 tx=0x7ff5bc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.736+0000 7f32b6fff700 1 Processor -- start 2023-07-06T03:45:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7f49079ff700 1 Processor -- start 2023-07-06T03:45:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7f49079ff700 1 -- start start 2023-07-06T03:45:23.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.736+0000 7f32b6fff700 1 -- start start 2023-07-06T03:45:23.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.736+0000 7f49079ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000cbb10 0x7f490014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.736+0000 7f49079ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49000ce3d0 0x7f49001ed620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.737+0000 7f49079ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f490015c0c0 0x7f49000b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.737+0000 7f49079ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490014c2e0 con 0x7f490015c0c0 2023-07-06T03:45:23.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.737+0000 7f49079ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49000c6ab0 con 0x7f49000ce3d0 2023-07-06T03:45:23.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.737+0000 7f49079ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49000c6d90 con 0x7f49000cbb10 2023-07-06T03:45:23.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.736+0000 7fa9a37fe700 1 -- 172.21.15.97:0/719381177 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa9ac022070 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.737+0000 7f4904f9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49000ce3d0 0x7f49001ed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.737+0000 7f4904f9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49000ce3d0 0x7f49001ed620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57670/0 (socket says 172.21.15.97:57670) 2023-07-06T03:45:23.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.737+0000 7f4904f9a700 1 -- 172.21.15.97:0/2626026741 learned_addr learned my addr 172.21.15.97:0/2626026741 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.737+0000 7f490579b700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000cbb10 0x7f490014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7fa9a37fe700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa9a806ad30 0x7fa9a806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7f490579b700 1 -- 172.21.15.97:0/2626026741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49000ce3d0 msgr2=0x7f49001ed620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7f490579b700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49000ce3d0 0x7f49001ed620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7f490579b700 1 -- 172.21.15.97:0/2626026741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f490015c0c0 msgr2=0x7f49000b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7f490579b700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f490015c0c0 0x7f49000b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7f490579b700 1 -- 172.21.15.97:0/2626026741 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f0011df0 con 0x7f49000cbb10 2023-07-06T03:45:23.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7fa9a37fe700 1 -- 172.21.15.97:0/719381177 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa9ac08b300 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7f490579b700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000cbb10 0x7f490014bda0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00b410 tx=0x7f48fc009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.735+0000 7fb4989ff700 1 -- 172.21.15.97:0/226832959 wait complete. 2023-07-06T03:45:23.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7fb4989ff700 1 Processor -- start 2023-07-06T03:45:23.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7f32b6fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b00c5170 0x7f32b00b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7f32b6fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b006c5c0 con 0x7f32b00adb80 2023-07-06T03:45:23.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7f32b6fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b00ce760 con 0x7f32b0160770 2023-07-06T03:45:23.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7f32b6fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00b1480 con 0x7f32b00c5170 2023-07-06T03:45:23.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.738+0000 7fb4989ff700 1 -- start start 2023-07-06T03:45:23.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.739+0000 7f32affff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f32b00adb80 0x7f32b00c4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35494/0 (socket says 172.21.15.97:35494) 2023-07-06T03:45:23.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.739+0000 7f32affff700 1 -- 172.21.15.97:0/3203378611 learned_addr learned my addr 172.21.15.97:0/3203378611 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:23.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.739+0000 7fa9ba7fc700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa9a806ad30 0x7fa9a806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.739+0000 7f32b559c700 1 --2- 172.21.15.97:0/3203378611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b00c5170 0x7f32b00b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.739+0000 7fdd70bff700 1 Processor -- start 2023-07-06T03:45:23.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.739+0000 7f32b559c700 1 -- 172.21.15.97:0/3203378611 >> v1:172.21.15.146:6789/0 conn(0x7f32b0160770 legacy=0x7f32b00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.739+0000 7f32af7fe700 1 -- 172.21.15.97:0/3203378611 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879553581 0 0) 0x7f32b006c5c0 con 0x7f32b00adb80 2023-07-06T03:45:23.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.739+0000 7f32b559c700 1 -- 172.21.15.97:0/3203378611 >> v1:172.21.15.97:6789/0 conn(0x7f32b00adb80 legacy=0x7f32b00c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.739+0000 7f32b559c700 1 -- 172.21.15.97:0/3203378611 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b006c8e0 con 0x7f32b00c5170 2023-07-06T03:45:23.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fdd70bff700 1 -- start start 2023-07-06T03:45:23.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fa9ba7fc700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa9a806ad30 0x7fa9a806d1e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc15d350 tx=0x7fa9b0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7f32b559c700 1 --2- 172.21.15.97:0/3203378611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b00c5170 0x7f32b00b0f40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009dc0 tx=0x7f32a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd57106ae894e30c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fb4989ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49406b360 0x7fb4940b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fb4989ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4940d2210 0x7fb4940cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fb4989ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49415c610 0x7fb4940cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fb4989ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49415cb20 con 0x7fb49406b360 2023-07-06T03:45:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fb4989ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940b7aa0 con 0x7fb4940d2210 2023-07-06T03:45:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fb4989ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49415cd30 con 0x7fb49415c610 2023-07-06T03:45:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fb49259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49406b360 0x7fb4940b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fb49259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49406b360 0x7fb4940b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51594/0 (socket says 172.21.15.97:51594) 2023-07-06T03:45:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fb49259c700 1 -- 172.21.15.97:0/2821884657 learned_addr learned my addr 172.21.15.97:0/2821884657 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.740+0000 7fb489d9b700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4940d2210 0x7fb4940cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.741+0000 7fb492d9d700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49415c610 0x7fb4940cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.741+0000 7fa9a37fe700 1 -- 172.21.15.97:0/719381177 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa9ac056540 con 0x7fa9bc0ccbd0 2023-07-06T03:45:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.741+0000 7fb489d9b700 1 -- 172.21.15.97:0/2821884657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49415c610 msgr2=0x7fb4940cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.741+0000 7fb489d9b700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49415c610 0x7fb4940cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.741+0000 7fb489d9b700 1 -- 172.21.15.97:0/2821884657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49406b360 msgr2=0x7fb4940b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.741+0000 7fb489d9b700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49406b360 0x7fb4940b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.741+0000 7fb489d9b700 1 -- 172.21.15.97:0/2821884657 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb484009550 con 0x7fb4940d2210 2023-07-06T03:45:23.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.741+0000 7fb49259c700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49406b360 0x7fb4940b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:23.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.741+0000 7fb489d9b700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4940d2210 0x7fb4940cae50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb4840098f0 tx=0x7fb484004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.741+0000 7fdd70bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdd6c160770 0x7fdd6c0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd70bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd6c0c4a60 0x7fdd6c0c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd70bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6c0c53b0 con 0x7fdd6c0adb80 2023-07-06T03:45:23.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd70bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c0b1480 con 0x7fdd6c0c4a60 2023-07-06T03:45:23.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd70bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c06c5c0 con 0x7fdd6c160770 2023-07-06T03:45:23.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd6a59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdd6c160770 0x7fdd6c0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd63fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fdd6c0adb80 0x7fdd6c0ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:35338/0 (socket says 172.21.15.97:35338) 2023-07-06T03:45:23.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd63fff700 1 -- 172.21.15.97:0/540955350 learned_addr learned my addr 172.21.15.97:0/540955350 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:23.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd6ad9d700 1 --2- 172.21.15.97:0/540955350 >> v2:172.21.15.97:3300/0 conn(0x7fdd6c0c4a60 0x7fdd6c0c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fb48b7fe700 1 -- 172.21.15.97:0/2821884657 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484004510 con 0x7fb4940d2210 2023-07-06T03:45:23.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fb48b7fe700 1 -- 172.21.15.97:0/2821884657 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48401f070 con 0x7fb4940d2210 2023-07-06T03:45:23.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fb48b7fe700 1 -- 172.21.15.97:0/2821884657 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840038c0 con 0x7fb4940d2210 2023-07-06T03:45:23.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd69d9b700 1 -- 172.21.15.97:0/540955350 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663291542 0 0) 0x7fdd6c0c53b0 con 0x7fdd6c0adb80 2023-07-06T03:45:23.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd69d9b700 1 -- 172.21.15.97:0/540955350 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd580035b0 con 0x7fdd6c0adb80 2023-07-06T03:45:23.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd6a59c700 1 -- 172.21.15.97:0/540955350 >> v2:172.21.15.97:3300/0 conn(0x7fdd6c0c4a60 msgr2=0x7fdd6c0c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:23.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd6a59c700 1 --2- 172.21.15.97:0/540955350 >> v2:172.21.15.97:3300/0 conn(0x7fdd6c0c4a60 0x7fdd6c0c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd6a59c700 1 -- 172.21.15.97:0/540955350 >> v1:172.21.15.146:6789/0 conn(0x7fdd6c0adb80 legacy=0x7fdd6c0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.742+0000 7fdd6a59c700 1 -- 172.21.15.97:0/540955350 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0ce760 con 0x7fdd6c160770 2023-07-06T03:45:23.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7fb880f99700 1 -- 172.21.15.97:0/2485630833 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87800e070 con 0x7fb87c0c4a60 2023-07-06T03:45:23.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7f32af7fe700 1 -- 172.21.15.97:0/3203378611 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a800e070 con 0x7f32b00c5170 2023-07-06T03:45:23.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7f32af7fe700 1 -- 172.21.15.97:0/3203378611 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a800bb90 con 0x7f32b00c5170 2023-07-06T03:45:23.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7f32af7fe700 1 -- 172.21.15.97:0/3203378611 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8003a20 con 0x7f32b00c5170 2023-07-06T03:45:23.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7f7093fff700 1 -- 172.21.15.97:0/3119548890 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709400e070 con 0x7f709c15c0c0 2023-07-06T03:45:23.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7ff5c5ffb700 1 -- 172.21.15.97:0/716760588 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc0036a0 con 0x7ff5c80cd950 2023-07-06T03:45:23.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7ff5c5ffb700 1 -- 172.21.15.97:0/716760588 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc01e070 con 0x7ff5c80cd950 2023-07-06T03:45:23.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7ff5c5ffb700 1 -- 172.21.15.97:0/716760588 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc003a40 con 0x7ff5c80cd950 2023-07-06T03:45:23.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/716760588 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c81fa480 con 0x7ff5c80cd950 2023-07-06T03:45:23.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/716760588 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c81f9e90 con 0x7ff5c80cd950 2023-07-06T03:45:23.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.744+0000 7fb880f99700 1 -- 172.21.15.97:0/2485630833 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87800bbd0 con 0x7fb87c0c4a60 2023-07-06T03:45:23.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.744+0000 7fb880f99700 1 -- 172.21.15.97:0/2485630833 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878012430 con 0x7fb87c0c4a60 2023-07-06T03:45:23.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.744+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/716760588 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c80b2e00 con 0x7ff5c80cd950 2023-07-06T03:45:23.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7f48f5ffb700 1 -- 172.21.15.97:0/2626026741 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc011830 con 0x7f49000cbb10 2023-07-06T03:45:23.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7f48f5ffb700 1 -- 172.21.15.97:0/2626026741 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc011990 con 0x7f49000cbb10 2023-07-06T03:45:23.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7f48f5ffb700 1 -- 172.21.15.97:0/2626026741 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc011d40 con 0x7f49000cbb10 2023-07-06T03:45:23.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7f49079ff700 1 -- 172.21.15.97:0/2626026741 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49000c7070 con 0x7f49000cbb10 2023-07-06T03:45:23.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.744+0000 7f49079ff700 1 -- 172.21.15.97:0/2626026741 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49000b2d30 con 0x7f49000cbb10 2023-07-06T03:45:23.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7f49079ff700 1 -- 172.21.15.97:0/2626026741 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490006f060 con 0x7f49000cbb10 2023-07-06T03:45:23.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7fb4989ff700 1 -- 172.21.15.97:0/2821884657 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49415d010 con 0x7fb4940d2210 2023-07-06T03:45:23.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.743+0000 7fb4989ff700 1 -- 172.21.15.97:0/2821884657 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4940c6b40 con 0x7fb4940d2210 2023-07-06T03:45:23.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7fb4989ff700 1 -- 172.21.15.97:0/2821884657 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49415d530 con 0x7fb4940d2210 2023-07-06T03:45:23.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7fb8841ff700 1 -- 172.21.15.97:0/2485630833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87c0c4a60 msgr2=0x7fb87c0c4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7fb8841ff700 1 --2- 172.21.15.97:0/2485630833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87c0c4a60 0x7fb87c0c4ec0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb878009d90 tx=0x7fb87800b330 comp rx=0 tx=0).stop 2023-07-06T03:45:23.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7fb8841ff700 1 -- 172.21.15.97:0/2485630833 shutdown_connections 2023-07-06T03:45:23.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7fb8841ff700 1 --2- 172.21.15.97:0/2485630833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87c0c4a60 0x7fb87c0c4ec0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7fb8841ff700 1 --2- 172.21.15.97:0/2485630833 >> v2:172.21.15.146:3300/0 conn(0x7fb87c160770 0x7fb87c0b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7fb8841ff700 1 -- 172.21.15.97:0/2485630833 >> 172.21.15.97:0/2485630833 conn(0x7fb87c06a060 msgr2=0x7fb87c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7fb8841ff700 1 -- 172.21.15.97:0/2485630833 shutdown_connections 2023-07-06T03:45:23.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7fdd6a59c700 1 --2- 172.21.15.97:0/540955350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6c160770 0x7fdd6c0b0f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0cf9f0 tx=0x7fdd5c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=881c7ef56d36aaa0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7fdd6ad9d700 1 --2- 172.21.15.97:0/540955350 >> v2:172.21.15.97:3300/0 conn(0x7fdd6c0c4a60 0x7fdd6c0c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:23.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.746+0000 7fdd69d9b700 1 -- 172.21.15.97:0/540955350 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c011830 con 0x7fdd6c160770 2023-07-06T03:45:23.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.746+0000 7f32b6fff700 1 -- 172.21.15.97:0/3203378611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b00c5170 msgr2=0x7f32b00b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.746+0000 7f32b6fff700 1 --2- 172.21.15.97:0/3203378611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b00c5170 0x7f32b00b0f40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009dc0 tx=0x7f32a800b330 comp rx=0 tx=0).stop 2023-07-06T03:45:23.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.746+0000 7f32b6fff700 1 -- 172.21.15.97:0/3203378611 shutdown_connections 2023-07-06T03:45:23.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.746+0000 7f32b6fff700 1 --2- 172.21.15.97:0/3203378611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b00c5170 0x7f32b00b0f40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.746+0000 7f32b6fff700 1 -- 172.21.15.97:0/3203378611 >> 172.21.15.97:0/3203378611 conn(0x7f32b006a060 msgr2=0x7f32b006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7f7093fff700 1 -- 172.21.15.97:0/3119548890 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7094003a20 con 0x7f709c15c0c0 2023-07-06T03:45:23.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.745+0000 7f7093fff700 1 -- 172.21.15.97:0/3119548890 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094003dd0 con 0x7f709c15c0c0 2023-07-06T03:45:23.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.746+0000 7f70a35ff700 1 -- 172.21.15.97:0/3119548890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 msgr2=0x7f709c0c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.746+0000 7f70a35ff700 1 --2- 172.21.15.97:0/3119548890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 0x7f709c0c8f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7094009e50 tx=0x7f709400b330 comp rx=0 tx=0).stop 2023-07-06T03:45:23.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.746+0000 7fb8841ff700 1 -- 172.21.15.97:0/2485630833 wait complete. 2023-07-06T03:45:23.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.746+0000 7ff5c5ffb700 1 -- 172.21.15.97:0/716760588 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff5bc003ba0 con 0x7ff5c80cd950 2023-07-06T03:45:23.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.747+0000 7f32b6fff700 1 -- 172.21.15.97:0/3203378611 shutdown_connections 2023-07-06T03:45:23.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.748+0000 7ff5c5ffb700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5b406ace0 0x7ff5b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.748+0000 7ff5c5ffb700 1 -- 172.21.15.97:0/716760588 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5bc08a350 con 0x7ff5c80cd950 2023-07-06T03:45:23.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.748+0000 7ff5ccb9a700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5b406ace0 0x7ff5b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.748+0000 7fb8841ff700 1 Processor -- start 2023-07-06T03:45:23.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.748+0000 7f32b6fff700 1 -- 172.21.15.97:0/3203378611 wait complete. 2023-07-06T03:45:23.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.749+0000 7ff5ccb9a700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5b406ace0 0x7ff5b406d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80b3e90 tx=0x7ff5b8009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.748+0000 7f48f5ffb700 1 -- 172.21.15.97:0/2626026741 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f48fc01a400 con 0x7f49000cbb10 2023-07-06T03:45:23.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.749+0000 7f48f5ffb700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48e406ad30 0x7f48e406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.749+0000 7f48f5ffb700 1 -- 172.21.15.97:0/2626026741 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f48fc089900 con 0x7f49000cbb10 2023-07-06T03:45:23.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7f32b6fff700 1 Processor -- start 2023-07-06T03:45:23.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7f32b6fff700 1 -- start start 2023-07-06T03:45:23.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.748+0000 7fb48b7fe700 1 -- 172.21.15.97:0/2821884657 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb484003a20 con 0x7fb4940d2210 2023-07-06T03:45:23.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.749+0000 7fb48b7fe700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb47406adb0 0x7fb47406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7fb48b7fe700 1 -- 172.21.15.97:0/2821884657 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb484021070 con 0x7fb4940d2210 2023-07-06T03:45:23.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7f70a35ff700 1 -- 172.21.15.97:0/3119548890 shutdown_connections 2023-07-06T03:45:23.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7f70a35ff700 1 --2- 172.21.15.97:0/3119548890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 0x7f709c0c8f40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7f70a35ff700 1 --2- 172.21.15.97:0/3119548890 >> v2:172.21.15.146:3300/0 conn(0x7f709c0ce3d0 0x7f709c0add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7f70a35ff700 1 -- 172.21.15.97:0/3119548890 >> 172.21.15.97:0/3119548890 conn(0x7f709c0cf610 msgr2=0x7f709c0cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7f70a35ff700 1 -- 172.21.15.97:0/3119548890 shutdown_connections 2023-07-06T03:45:23.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7f70a35ff700 1 -- 172.21.15.97:0/3119548890 wait complete. 2023-07-06T03:45:23.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7f4904f9a700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48e406ad30 0x7f48e406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.749+0000 7fb8841ff700 1 -- start start 2023-07-06T03:45:23.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7fb8841ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87c0adb80 0x7fb87c1480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7fb8841ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87c0c4a60 0x7fb87c1f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7fb8841ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb87c160770 0x7fb87c1f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.750+0000 7fb8841ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c147670 con 0x7fb87c0c4a60 2023-07-06T03:45:23.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7fb8841ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c148600 con 0x7fb87c160770 2023-07-06T03:45:23.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7fb8841ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c1fa290 con 0x7fb87c0adb80 2023-07-06T03:45:23.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7f32b6fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b00adb80 0x7f32b00b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7f32b6fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b00c5170 0x7f32b0153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7f32b6fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b0160770 0x7f32b015edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7f32b6fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01faa80 con 0x7f32b00adb80 2023-07-06T03:45:23.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7f32b6fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01f9e70 con 0x7f32b00c5170 2023-07-06T03:45:23.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7f32b6fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01fa150 con 0x7f32b0160770 2023-07-06T03:45:23.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7fb88279c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb87c160770 0x7fb87c1f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7fb88279c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb87c160770 0x7fb87c1f97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57688/0 (socket says 172.21.15.97:57688) 2023-07-06T03:45:23.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7fb88279c700 1 -- 172.21.15.97:0/2443055952 learned_addr learned my addr 172.21.15.97:0/2443055952 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7f32affff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b00c5170 0x7f32b0153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7f32affff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b00c5170 0x7f32b0153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57704/0 (socket says 172.21.15.97:57704) 2023-07-06T03:45:23.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7f32affff700 1 -- 172.21.15.97:0/3076249782 learned_addr learned my addr 172.21.15.97:0/3076249782 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7fb88179a700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87c0c4a60 0x7fb87c1f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7fb881f9b700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87c0adb80 0x7fb87c1480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7f32b4d9b700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b00adb80 0x7f32b00b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7f32b559c700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b0160770 0x7f32b015edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.751+0000 7f4904f9a700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48e406ad30 0x7f48e406d1e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49000c9370 tx=0x7f48f0011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7f48f5ffb700 1 -- 172.21.15.97:0/2626026741 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f48fc054990 con 0x7f49000cbb10 2023-07-06T03:45:23.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7fb48b7fe700 1 -- 172.21.15.97:0/2821884657 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb484056120 con 0x7fb4940d2210 2023-07-06T03:45:23.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7f32affff700 1 -- 172.21.15.97:0/3076249782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b0160770 msgr2=0x7f32b015edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7f32affff700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b0160770 0x7f32b015edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7f32affff700 1 -- 172.21.15.97:0/3076249782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b00adb80 msgr2=0x7f32b00b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7f32affff700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b00adb80 0x7f32b00b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7f32affff700 1 -- 172.21.15.97:0/3076249782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a8009a70 con 0x7f32b00c5170 2023-07-06T03:45:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.753+0000 7f32affff700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b00c5170 0x7f32b0153c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f329800c550 tx=0x7f329800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7fb88279c700 1 -- 172.21.15.97:0/2443055952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87c0adb80 msgr2=0x7fb87c1480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7fb88279c700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87c0adb80 0x7fb87c1480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7fb88279c700 1 -- 172.21.15.97:0/2443055952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87c0c4a60 msgr2=0x7fb87c1f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.753+0000 7f70a35ff700 1 Processor -- start 2023-07-06T03:45:23.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7fb88279c700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87c0c4a60 0x7fb87c1f5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.752+0000 7fb88279c700 1 -- 172.21.15.97:0/2443055952 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878009a70 con 0x7fb87c160770 2023-07-06T03:45:23.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.753+0000 7fb49259c700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb47406adb0 0x7fb47406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.753+0000 7fdd69d9b700 1 -- 172.21.15.97:0/540955350 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c011990 con 0x7fdd6c160770 2023-07-06T03:45:23.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.753+0000 7fdd69d9b700 1 -- 172.21.15.97:0/540955350 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c011d00 con 0x7fdd6c160770 2023-07-06T03:45:23.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.754+0000 7f32ad7fa700 1 -- 172.21.15.97:0/3076249782 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3298004c80 con 0x7f32b00c5170 2023-07-06T03:45:23.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.754+0000 7f32ad7fa700 1 -- 172.21.15.97:0/3076249782 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3298004e00 con 0x7f32b00c5170 2023-07-06T03:45:23.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.754+0000 7f32ad7fa700 1 -- 172.21.15.97:0/3076249782 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3298007310 con 0x7f32b00c5170 2023-07-06T03:45:23.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.754+0000 7fb49259c700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb47406adb0 0x7fb47406d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb4940d3d10 tx=0x7fb47c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.754+0000 7fdd70bff700 1 -- 172.21.15.97:0/540955350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6c160770 msgr2=0x7fdd6c0b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.754+0000 7fdd70bff700 1 --2- 172.21.15.97:0/540955350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6c160770 0x7fdd6c0b0f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0cf9f0 tx=0x7fdd5c009b20 comp rx=0 tx=0).stop 2023-07-06T03:45:23.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f32b6fff700 1 -- 172.21.15.97:0/3076249782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b01fa430 con 0x7f32b00c5170 2023-07-06T03:45:23.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f32b6fff700 1 -- 172.21.15.97:0/3076249782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b01541c0 con 0x7f32b00c5170 2023-07-06T03:45:23.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.753+0000 7f70a35ff700 1 -- start start 2023-07-06T03:45:23.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f70a35ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f709c0cbb10 0x7f709c0c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f70a35ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f709c0ce3d0 0x7f709c0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f70a35ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 0x7f709c13ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f70a35ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0c7420 con 0x7f709c0cbb10 2023-07-06T03:45:23.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f70a35ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0c79e0 con 0x7f709c0ce3d0 2023-07-06T03:45:23.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f70a35ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0b2c50 con 0x7f709c15c0c0 2023-07-06T03:45:23.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.756+0000 7ff5c5ffb700 1 -- 172.21.15.97:0/716760588 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff5c80b2e00 con 0x7ff5c80cd950 2023-07-06T03:45:23.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.756+0000 7f32b6fff700 1 -- 172.21.15.97:0/3076249782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b015bd50 con 0x7f32b00c5170 2023-07-06T03:45:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f70a139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f709c0cbb10 0x7f709c0c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f70a1b9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 0x7f709c13ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f70a139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f709c0cbb10 0x7f709c0c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51628/0 (socket says 172.21.15.97:51628) 2023-07-06T03:45:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f70a139b700 1 -- 172.21.15.97:0/90029200 learned_addr learned my addr 172.21.15.97:0/90029200 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.755+0000 7f70a0b9a700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f709c0ce3d0 0x7f709c0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.756+0000 7f70a139b700 1 -- 172.21.15.97:0/90029200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 msgr2=0x7f709c13ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.756+0000 7f70a139b700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 0x7f709c13ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.756+0000 7f70a139b700 1 -- 172.21.15.97:0/90029200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f709c0ce3d0 msgr2=0x7f709c0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.756+0000 7f70a139b700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f709c0ce3d0 0x7f709c0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.756+0000 7f70a139b700 1 -- 172.21.15.97:0/90029200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7094009b30 con 0x7f709c0cbb10 2023-07-06T03:45:23.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.757+0000 7f70a139b700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f709c0cbb10 0x7f709c0c7e70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f709800b450 tx=0x7f709800b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.757+0000 7fb88279c700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb87c160770 0x7fb87c1f97d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb87800b370 tx=0x7fb878004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.758+0000 7fb8727fc700 1 -- 172.21.15.97:0/2443055952 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878009410 con 0x7fb87c160770 2023-07-06T03:45:23.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.758+0000 7fb8727fc700 1 -- 172.21.15.97:0/2443055952 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb878020070 con 0x7fb87c160770 2023-07-06T03:45:23.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.758+0000 7fb8727fc700 1 -- 172.21.15.97:0/2443055952 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878004280 con 0x7fb87c160770 2023-07-06T03:45:23.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.758+0000 7fb8841ff700 1 -- 172.21.15.97:0/2443055952 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c1fa570 con 0x7fb87c160770 2023-07-06T03:45:23.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.758+0000 7fb8841ff700 1 -- 172.21.15.97:0/2443055952 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c147ba0 con 0x7fb87c160770 2023-07-06T03:45:23.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.757+0000 7f7091ffb700 1 -- 172.21.15.97:0/90029200 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709800be60 con 0x7f709c0cbb10 2023-07-06T03:45:23.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.757+0000 7f7091ffb700 1 -- 172.21.15.97:0/90029200 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7098022640 con 0x7f709c0cbb10 2023-07-06T03:45:23.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.757+0000 7f7091ffb700 1 -- 172.21.15.97:0/90029200 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70980229d0 con 0x7f709c0cbb10 2023-07-06T03:45:23.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.759+0000 7f70a35ff700 1 -- 172.21.15.97:0/90029200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c0b2f30 con 0x7f709c0cbb10 2023-07-06T03:45:23.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.759+0000 7f70a35ff700 1 -- 172.21.15.97:0/90029200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c0c83b0 con 0x7f709c0cbb10 2023-07-06T03:45:23.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.759+0000 7f32ad7fa700 1 -- 172.21.15.97:0/3076249782 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3298007470 con 0x7f32b00c5170 2023-07-06T03:45:23.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.760+0000 7f32ad7fa700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f329006ad30 0x7f329006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.761+0000 7f32ad7fa700 1 -- 172.21.15.97:0/3076249782 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3298089010 con 0x7f32b00c5170 2023-07-06T03:45:23.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.762+0000 7f32b4d9b700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f329006ad30 0x7f329006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.762+0000 7f32b4d9b700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f329006ad30 0x7f329006d1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f32b0159df0 tx=0x7f32a0007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.762+0000 7fb8727fc700 1 -- 172.21.15.97:0/2443055952 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb8780043e0 con 0x7fb87c160770 2023-07-06T03:45:23.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.761+0000 7f70a35ff700 1 -- 172.21.15.97:0/90029200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c0c7cc0 con 0x7f709c0cbb10 2023-07-06T03:45:23.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.764+0000 7fdd70bff700 1 -- 172.21.15.97:0/540955350 shutdown_connections 2023-07-06T03:45:23.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.764+0000 7fdd70bff700 1 --2- 172.21.15.97:0/540955350 >> v2:172.21.15.97:3300/0 conn(0x7fdd6c0c4a60 0x7fdd6c0c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.764+0000 7fdd70bff700 1 --2- 172.21.15.97:0/540955350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6c160770 0x7fdd6c0b0f40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.764+0000 7fdd70bff700 1 -- 172.21.15.97:0/540955350 >> 172.21.15.97:0/540955350 conn(0x7fdd6c06a060 msgr2=0x7fdd6c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:23.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.763+0000 7fb8727fc700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86006adb0 0x7fb86006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.764+0000 7fb8727fc700 1 -- 172.21.15.97:0/2443055952 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb878022070 con 0x7fb87c160770 2023-07-06T03:45:23.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.765+0000 7f32ad7fa700 1 -- 172.21.15.97:0/3076249782 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f32980541a0 con 0x7f32b00c5170 2023-07-06T03:45:23.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.764+0000 7fb8841ff700 1 -- 172.21.15.97:0/2443055952 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8640052f0 con 0x7fb87c160770 2023-07-06T03:45:23.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.765+0000 7fb881f9b700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86006adb0 0x7fb86006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.766+0000 7fb881f9b700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86006adb0 0x7fb86006d260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009930 tx=0x7fb86c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.765+0000 7fdd70bff700 1 -- 172.21.15.97:0/540955350 shutdown_connections 2023-07-06T03:45:23.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.763+0000 7f7091ffb700 1 -- 172.21.15.97:0/90029200 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7098022b30 con 0x7f709c0cbb10 2023-07-06T03:45:23.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.765+0000 7f7091ffb700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f708006ad30 0x7f708006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.765+0000 7f7091ffb700 1 -- 172.21.15.97:0/90029200 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f709809a1a0 con 0x7f709c0cbb10 2023-07-06T03:45:23.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.766+0000 7f70a0b9a700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f708006ad30 0x7f708006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.766+0000 7f70a0b9a700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f708006ad30 0x7f708006d1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f709c0af2e0 tx=0x7f708c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.768+0000 7fdd70bff700 1 -- 172.21.15.97:0/540955350 wait complete. 2023-07-06T03:45:23.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.770+0000 7fdd70bff700 1 Processor -- start 2023-07-06T03:45:23.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.771+0000 7fdd70bff700 1 -- start start 2023-07-06T03:45:23.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.773+0000 7fdd70bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6c0adb80 0x7fdd6c15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.773+0000 7fdd70bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6c0c4a60 0x7fdd6c206d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.773+0000 7fdd70bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6c160770 0x7fdd6c163320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.773+0000 7fdd70bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c207240 con 0x7fdd6c0c4a60 2023-07-06T03:45:23.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.773+0000 7fdd70bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0b1e00 con 0x7fdd6c0adb80 2023-07-06T03:45:23.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.773+0000 7fdd70bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0b20e0 con 0x7fdd6c160770 2023-07-06T03:45:23.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.773+0000 7fb8727fc700 1 -- 172.21.15.97:0/2443055952 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb8780567a0 con 0x7fb87c160770 2023-07-06T03:45:23.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.774+0000 7fdd6ad9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6c160770 0x7fdd6c163320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.774+0000 7fdd6a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6c0adb80 0x7fdd6c15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.774+0000 7fdd6ad9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6c160770 0x7fdd6c163320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54132/0 (socket says 172.21.15.97:54132) 2023-07-06T03:45:23.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.774+0000 7fdd6ad9d700 1 -- 172.21.15.97:0/3665771297 learned_addr learned my addr 172.21.15.97:0/3665771297 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:23.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.774+0000 7fdd63fff700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6c0c4a60 0x7fdd6c206d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.774+0000 7fdd6ad9d700 1 -- 172.21.15.97:0/3665771297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6c0adb80 msgr2=0x7fdd6c15cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.774+0000 7fdd6ad9d700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6c0adb80 0x7fdd6c15cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.775+0000 7fdd6ad9d700 1 -- 172.21.15.97:0/3665771297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6c0c4a60 msgr2=0x7fdd6c206d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:23.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.775+0000 7fdd6ad9d700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6c0c4a60 0x7fdd6c206d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:23.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.775+0000 7fdd6ad9d700 1 -- 172.21.15.97:0/3665771297 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0094a0 con 0x7fdd6c160770 2023-07-06T03:45:23.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.775+0000 7f7091ffb700 1 -- 172.21.15.97:0/90029200 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f70980652c0 con 0x7f709c0cbb10 2023-07-06T03:45:23.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.775+0000 7fdd6ad9d700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6c160770 0x7fdd6c163320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400cef0 tx=0x7fdd6400cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.776+0000 7fdd62ffd700 1 -- 172.21.15.97:0/3665771297 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6400adb0 con 0x7fdd6c160770 2023-07-06T03:45:23.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.776+0000 7fdd62ffd700 1 -- 172.21.15.97:0/3665771297 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd64015440 con 0x7fdd6c160770 2023-07-06T03:45:23.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.776+0000 7fdd62ffd700 1 -- 172.21.15.97:0/3665771297 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd640157f0 con 0x7fdd6c160770 2023-07-06T03:45:23.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.776+0000 7fdd70bff700 1 -- 172.21.15.97:0/3665771297 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c0b23c0 con 0x7fdd6c160770 2023-07-06T03:45:23.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.776+0000 7fdd70bff700 1 -- 172.21.15.97:0/3665771297 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c0b2980 con 0x7fdd6c160770 2023-07-06T03:45:23.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.778+0000 7fdd70bff700 1 -- 172.21.15.97:0/3665771297 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c0b2ba0 con 0x7fdd6c160770 2023-07-06T03:45:23.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.779+0000 7fdd62ffd700 1 -- 172.21.15.97:0/3665771297 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd6400d430 con 0x7fdd6c160770 2023-07-06T03:45:23.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.781+0000 7fdd62ffd700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4c06ad30 0x7fdd4c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:23.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.781+0000 7fdd62ffd700 1 -- 172.21.15.97:0/3665771297 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdd64089aa0 con 0x7fdd6c160770 2023-07-06T03:45:23.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.781+0000 7fdd6a59c700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4c06ad30 0x7fdd4c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:23.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.783+0000 7fdd6a59c700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4c06ad30 0x7fdd4c06d1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0090b0 tx=0x7fdd5c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:23.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:23.785+0000 7fdd62ffd700 1 -- 172.21.15.97:0/3665771297 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdd64054b30 con 0x7fdd6c160770 2023-07-06T03:45:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.181+0000 7fa9c17ff700 1 -- 172.21.15.97:0/719381177 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa9bc0adf10 con 0x7fa9bc0ccbd0 2023-07-06T03:45:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.182+0000 7fa9a37fe700 1 -- 172.21.15.97:0/719381177 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa9bc0adf10 con 0x7fa9bc0ccbd0 2023-07-06T03:45:24.186 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:24.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.186+0000 7fa9c17ff700 1 -- 172.21.15.97:0/719381177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa9a806ad30 msgr2=0x7fa9a806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.186+0000 7fa9c17ff700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa9a806ad30 0x7fa9a806d1e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc15d350 tx=0x7fa9b0009190 comp rx=0 tx=0).stop 2023-07-06T03:45:24.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.186+0000 7fa9c17ff700 1 -- 172.21.15.97:0/719381177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9bc0ccbd0 msgr2=0x7fa9bc0ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.186+0000 7fa9c17ff700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9bc0ccbd0 0x7fa9bc0ae540 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac015000 tx=0x7fa9ac004750 comp rx=0 tx=0).stop 2023-07-06T03:45:24.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.186+0000 7fa9c17ff700 1 -- 172.21.15.97:0/719381177 shutdown_connections 2023-07-06T03:45:24.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.186+0000 7fa9c17ff700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa9a806ad30 0x7fa9a806d1e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.186+0000 7fa9c17ff700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9bc160a20 0x7fa9bc14b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.186+0000 7fa9c17ff700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9bc0ccbd0 0x7fa9bc0ae540 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.186+0000 7fa9c17ff700 1 --2- 172.21.15.97:0/719381177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9bc0ce3c0 0x7fa9bc0b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.186+0000 7fa9c17ff700 1 -- 172.21.15.97:0/719381177 >> 172.21.15.97:0/719381177 conn(0x7fa9bc0cf610 msgr2=0x7fa9bc0cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.187+0000 7fa9c17ff700 1 -- 172.21.15.97:0/719381177 shutdown_connections 2023-07-06T03:45:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.187+0000 7fa9c17ff700 1 -- 172.21.15.97:0/719381177 wait complete. 2023-07-06T03:45:24.202 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-07-06T03:45:24.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.199+0000 7f49079ff700 1 -- 172.21.15.97:0/2626026741 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f49000c6ed0 con 0x7f49000cbb10 2023-07-06T03:45:24.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.200+0000 7f48f5ffb700 1 -- 172.21.15.97:0/2626026741 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f49000c6ed0 con 0x7f49000cbb10 2023-07-06T03:45:24.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.202+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/716760588 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff5c80b2e00 con 0x7ff5c80cd950 2023-07-06T03:45:24.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.203+0000 7ff5c5ffb700 1 -- 172.21.15.97:0/716760588 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff5c80b2e00 con 0x7ff5c80cd950 2023-07-06T03:45:24.205 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-07-06T03:45:24.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.205+0000 7f49079ff700 1 -- 172.21.15.97:0/2626026741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48e406ad30 msgr2=0x7f48e406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.205+0000 7f49079ff700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48e406ad30 0x7f48e406d1e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49000c9370 tx=0x7f48f0011000 comp rx=0 tx=0).stop 2023-07-06T03:45:24.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.205+0000 7f49079ff700 1 -- 172.21.15.97:0/2626026741 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000cbb10 msgr2=0x7f490014bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.205+0000 7f49079ff700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000cbb10 0x7f490014bda0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00b410 tx=0x7f48fc009f90 comp rx=0 tx=0).stop 2023-07-06T03:45:24.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.205+0000 7f49079ff700 1 -- 172.21.15.97:0/2626026741 shutdown_connections 2023-07-06T03:45:24.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.205+0000 7f49079ff700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48e406ad30 0x7f48e406d1e0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49000c9370 tx=0x7f48f0011000 comp rx=0 tx=0).stop 2023-07-06T03:45:24.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.205+0000 7f49079ff700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f490015c0c0 0x7f49000b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.205+0000 7f49079ff700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49000cbb10 0x7f490014bda0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.205+0000 7f49079ff700 1 --2- 172.21.15.97:0/2626026741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49000ce3d0 0x7f49001ed620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.205+0000 7f49079ff700 1 -- 172.21.15.97:0/2626026741 >> 172.21.15.97:0/2626026741 conn(0x7f49000cf610 msgr2=0x7f49000af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:24.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.206+0000 7f49079ff700 1 -- 172.21.15.97:0/2626026741 shutdown_connections 2023-07-06T03:45:24.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.206+0000 7f49079ff700 1 -- 172.21.15.97:0/2626026741 wait complete. 2023-07-06T03:45:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.207+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/716760588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5b406ace0 msgr2=0x7ff5b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.207+0000 7ff5cf5ff700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5b406ace0 0x7ff5b406d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80b3e90 tx=0x7ff5b8009250 comp rx=0 tx=0).stop 2023-07-06T03:45:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.207+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/716760588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cd950 msgr2=0x7ff5c80aaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.207+0000 7ff5cf5ff700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cd950 0x7ff5c80aaa60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc000c00 tx=0x7ff5bc004750 comp rx=0 tx=0).stop 2023-07-06T03:45:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.210+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/716760588 shutdown_connections 2023-07-06T03:45:24.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.210+0000 7ff5cf5ff700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5b406ace0 0x7ff5b406d190 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.210+0000 7ff5cf5ff700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c81644a0 0x7ff5c80b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.210+0000 7ff5cf5ff700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5c80cd950 0x7ff5c80aaa60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.210+0000 7ff5cf5ff700 1 --2- 172.21.15.97:0/716760588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5c80cdd20 0x7ff5c81f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.210+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/716760588 >> 172.21.15.97:0/716760588 conn(0x7ff5c806a060 msgr2=0x7ff5c80cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:24.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.211+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/716760588 shutdown_connections 2023-07-06T03:45:24.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.212+0000 7ff5cf5ff700 1 -- 172.21.15.97:0/716760588 wait complete. 2023-07-06T03:45:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.213+0000 7f32b6fff700 1 -- 172.21.15.97:0/3076249782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f32b004e340 con 0x7f32b00c5170 2023-07-06T03:45:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.214+0000 7f32ad7fa700 1 -- 172.21.15.97:0/3076249782 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f32980577c0 con 0x7f32b00c5170 2023-07-06T03:45:24.217 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-07-06T03:45:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.217+0000 7fb4989ff700 1 -- 172.21.15.97:0/2821884657 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb49415d530 con 0x7fb4940d2210 2023-07-06T03:45:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.218+0000 7fb48b7fe700 1 -- 172.21.15.97:0/2821884657 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb49415d530 con 0x7fb4940d2210 2023-07-06T03:45:24.221 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-07-06T03:45:24.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-07-06T03:45:24.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.219+0000 7fb8841ff700 1 -- 172.21.15.97:0/2443055952 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb8640055a0 con 0x7fb87c160770 2023-07-06T03:45:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.219+0000 7f32b6fff700 1 -- 172.21.15.97:0/3076249782 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f329006ad30 msgr2=0x7f329006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.219+0000 7f32b6fff700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f329006ad30 0x7f329006d1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f32b0159df0 tx=0x7f32a0007ec0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.219+0000 7f32b6fff700 1 -- 172.21.15.97:0/3076249782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b00c5170 msgr2=0x7f32b0153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.219+0000 7f32b6fff700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b00c5170 0x7f32b0153c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f329800c550 tx=0x7f329800c860 comp rx=0 tx=0).stop 2023-07-06T03:45:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.220+0000 7f32b6fff700 1 -- 172.21.15.97:0/3076249782 shutdown_connections 2023-07-06T03:45:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.220+0000 7f32b6fff700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f329006ad30 0x7f329006d1e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.220+0000 7f32b6fff700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f32b0160770 0x7f32b015edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.220+0000 7f32b6fff700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b00adb80 0x7f32b00b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.220+0000 7f32b6fff700 1 --2- 172.21.15.97:0/3076249782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32b00c5170 0x7f32b0153c80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.220+0000 7f32b6fff700 1 -- 172.21.15.97:0/3076249782 >> 172.21.15.97:0/3076249782 conn(0x7f32b006a060 msgr2=0x7f32b00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:24.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.220+0000 7fb8727fc700 1 -- 172.21.15.97:0/2443055952 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb878059dc0 con 0x7fb87c160770 2023-07-06T03:45:24.227 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2023-07-06T03:45:24.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.220+0000 7f32b6fff700 1 -- 172.21.15.97:0/3076249782 shutdown_connections 2023-07-06T03:45:24.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.220+0000 7f32b6fff700 1 -- 172.21.15.97:0/3076249782 wait complete. 2023-07-06T03:45:24.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.222+0000 7fb4989ff700 1 -- 172.21.15.97:0/2821884657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb47406adb0 msgr2=0x7fb47406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.222+0000 7fb4989ff700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb47406adb0 0x7fb47406d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb4940d3d10 tx=0x7fb47c009210 comp rx=0 tx=0).stop 2023-07-06T03:45:24.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.222+0000 7fb4989ff700 1 -- 172.21.15.97:0/2821884657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4940d2210 msgr2=0x7fb4940cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.222+0000 7fb4989ff700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4940d2210 0x7fb4940cae50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb4840098f0 tx=0x7fb484004750 comp rx=0 tx=0).stop 2023-07-06T03:45:24.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.222+0000 7fb4989ff700 1 -- 172.21.15.97:0/2821884657 shutdown_connections 2023-07-06T03:45:24.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.222+0000 7fb4989ff700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb47406adb0 0x7fb47406d260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.222+0000 7fb4989ff700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49415c610 0x7fb4940cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.222+0000 7fb4989ff700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb49406b360 0x7fb4940b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.222+0000 7fb4989ff700 1 --2- 172.21.15.97:0/2821884657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4940d2210 0x7fb4940cae50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.222+0000 7fb4989ff700 1 -- 172.21.15.97:0/2821884657 >> 172.21.15.97:0/2821884657 conn(0x7fb49406a6c0 msgr2=0x7fb4940b5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:24.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.223+0000 7fb4989ff700 1 -- 172.21.15.97:0/2821884657 shutdown_connections 2023-07-06T03:45:24.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.223+0000 7fb4989ff700 1 -- 172.21.15.97:0/2821884657 wait complete. 2023-07-06T03:45:24.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.224+0000 7fb8841ff700 1 -- 172.21.15.97:0/2443055952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86006adb0 msgr2=0x7fb86006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.224+0000 7fb8841ff700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86006adb0 0x7fb86006d260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009930 tx=0x7fb86c008040 comp rx=0 tx=0).stop 2023-07-06T03:45:24.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.224+0000 7fb8841ff700 1 -- 172.21.15.97:0/2443055952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb87c160770 msgr2=0x7fb87c1f97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.224+0000 7fb8841ff700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb87c160770 0x7fb87c1f97d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb87800b370 tx=0x7fb878004ad0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.224+0000 7fb8841ff700 1 -- 172.21.15.97:0/2443055952 shutdown_connections 2023-07-06T03:45:24.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.224+0000 7fb8841ff700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86006adb0 0x7fb86006d260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.224+0000 7fb8841ff700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb87c160770 0x7fb87c1f97d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.224+0000 7fb8841ff700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87c0adb80 0x7fb87c1480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.224+0000 7fb8841ff700 1 --2- 172.21.15.97:0/2443055952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87c0c4a60 0x7fb87c1f5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.224+0000 7fb8841ff700 1 -- 172.21.15.97:0/2443055952 >> 172.21.15.97:0/2443055952 conn(0x7fb87c06a060 msgr2=0x7fb87c164aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:24.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.225+0000 7fb8841ff700 1 -- 172.21.15.97:0/2443055952 shutdown_connections 2023-07-06T03:45:24.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.225+0000 7fb8841ff700 1 -- 172.21.15.97:0/2443055952 wait complete. 2023-07-06T03:45:24.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.227+0000 7fdd70bff700 1 -- 172.21.15.97:0/3665771297 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fdd6c04e340 con 0x7fdd6c160770 2023-07-06T03:45:24.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.228+0000 7f70a35ff700 1 -- 172.21.15.97:0/90029200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f709c04e340 con 0x7f709c0cbb10 2023-07-06T03:45:24.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.228+0000 7f7091ffb700 1 -- 172.21.15.97:0/90029200 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70980688e0 con 0x7f709c0cbb10 2023-07-06T03:45:24.235 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:24.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.232+0000 7f70a35ff700 1 -- 172.21.15.97:0/90029200 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f708006ad30 msgr2=0x7f708006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.232+0000 7f70a35ff700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f708006ad30 0x7f708006d1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f709c0af2e0 tx=0x7f708c009210 comp rx=0 tx=0).stop 2023-07-06T03:45:24.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.232+0000 7f70a35ff700 1 -- 172.21.15.97:0/90029200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f709c0cbb10 msgr2=0x7f709c0c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.232+0000 7f70a35ff700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f709c0cbb10 0x7f709c0c7e70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f709800b450 tx=0x7f709800b760 comp rx=0 tx=0).stop 2023-07-06T03:45:24.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.233+0000 7f70a35ff700 1 -- 172.21.15.97:0/90029200 shutdown_connections 2023-07-06T03:45:24.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.233+0000 7f70a35ff700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f708006ad30 0x7f708006d1e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.233+0000 7f70a35ff700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f709c15c0c0 0x7f709c13ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.233+0000 7f70a35ff700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f709c0cbb10 0x7f709c0c7e70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.233+0000 7f70a35ff700 1 --2- 172.21.15.97:0/90029200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f709c0ce3d0 0x7f709c0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.233+0000 7f70a35ff700 1 -- 172.21.15.97:0/90029200 >> 172.21.15.97:0/90029200 conn(0x7f709c0cf610 msgr2=0x7f709c072fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:24.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.233+0000 7f70a35ff700 1 -- 172.21.15.97:0/90029200 shutdown_connections 2023-07-06T03:45:24.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.233+0000 7f70a35ff700 1 -- 172.21.15.97:0/90029200 wait complete. 2023-07-06T03:45:24.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.234+0000 7fdd62ffd700 1 -- 172.21.15.97:0/3665771297 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdd64058150 con 0x7fdd6c160770 2023-07-06T03:45:24.241 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-07-06T03:45:24.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2023-07-06T03:45:24.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.238+0000 7fdd70bff700 1 -- 172.21.15.97:0/3665771297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4c06ad30 msgr2=0x7fdd4c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.238+0000 7fdd70bff700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4c06ad30 0x7fdd4c06d1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0090b0 tx=0x7fdd5c01c040 comp rx=0 tx=0).stop 2023-07-06T03:45:24.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.238+0000 7fdd70bff700 1 -- 172.21.15.97:0/3665771297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6c160770 msgr2=0x7fdd6c163320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:24.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.238+0000 7fdd70bff700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6c160770 0x7fdd6c163320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400cef0 tx=0x7fdd6400cf20 comp rx=0 tx=0).stop 2023-07-06T03:45:24.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.239+0000 7fdd70bff700 1 -- 172.21.15.97:0/3665771297 shutdown_connections 2023-07-06T03:45:24.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.239+0000 7fdd70bff700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4c06ad30 0x7fdd4c06d1e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.239+0000 7fdd70bff700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd6c160770 0x7fdd6c163320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.239+0000 7fdd70bff700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6c0adb80 0x7fdd6c15cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.239+0000 7fdd70bff700 1 --2- 172.21.15.97:0/3665771297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6c0c4a60 0x7fdd6c206d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:24.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.239+0000 7fdd70bff700 1 -- 172.21.15.97:0/3665771297 >> 172.21.15.97:0/3665771297 conn(0x7fdd6c06a060 msgr2=0x7fdd6c06a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:24.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.240+0000 7fdd70bff700 1 -- 172.21.15.97:0/3665771297 shutdown_connections 2023-07-06T03:45:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:24.240+0000 7fdd70bff700 1 -- 172.21.15.97:0/3665771297 wait complete. 2023-07-06T03:45:24.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2023-07-06T03:45:24.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2023-07-06T03:45:24.257 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-07-06T03:45:24.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-07-06T03:45:24.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2023-07-06T03:45:24.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2023-07-06T03:45:25.224 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-06T03:45:25.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-06T03:45:25.255 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-06T03:45:25.257 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-06T03:45:25.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-06T03:45:25.261 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-06T03:45:25.266 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-06T03:45:25.268 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-06T03:45:25.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.530+0000 7fb6b2bff700 1 Processor -- start 2023-07-06T03:45:25.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.530+0000 7fb6b2bff700 1 -- start start 2023-07-06T03:45:25.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.532+0000 7fb6b2bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb6ac160a20 0x7fb6ac0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.532+0000 7fb6b2bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 0x7fb6ac0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.532+0000 7fb6b2bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ac0caa30 con 0x7fb6ac0cca00 2023-07-06T03:45:25.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.532+0000 7fb6b2bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ac0c5260 con 0x7fb6ac160a20 2023-07-06T03:45:25.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.532+0000 7fb6b2bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac160df0 con 0x7fb6ac0ce3c0 2023-07-06T03:45:25.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.534+0000 7fb6abfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 0x7fb6ac0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.534+0000 7fb6abfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 0x7fb6ac0cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54146/0 (socket says 172.21.15.97:54146) 2023-07-06T03:45:25.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.534+0000 7fb6abfff700 1 -- 172.21.15.97:0/4087906979 learned_addr learned my addr 172.21.15.97:0/4087906979 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.534+0000 7fb6b099b700 1 --2- 172.21.15.97:0/4087906979 >> v2:172.21.15.146:3300/0 conn(0x7fb6ac160a20 0x7fb6ac0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.534+0000 7fb6abfff700 1 -- 172.21.15.97:0/4087906979 >> v2:172.21.15.146:3300/0 conn(0x7fb6ac160a20 msgr2=0x7fb6ac0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:25.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.534+0000 7fb6abfff700 1 --2- 172.21.15.97:0/4087906979 >> v2:172.21.15.146:3300/0 conn(0x7fb6ac160a20 0x7fb6ac0c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.534+0000 7fb6abfff700 1 -- 172.21.15.97:0/4087906979 >> v1:172.21.15.146:6789/0 conn(0x7fb6ac0cca00 legacy=0x7fb6ac164520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T03:45:25.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.534+0000 7fb6abfff700 1 -- 172.21.15.97:0/4087906979 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac0cce60 con 0x7fb6ac0ce3c0 2023-07-06T03:45:25.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.536+0000 7fb6abfff700 1 --2- 172.21.15.97:0/4087906979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 0x7fb6ac0cc4c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0098f0 tx=0x7fb69c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8128f54a4b5e156b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.536+0000 7fb6ab7fe700 1 -- 172.21.15.97:0/4087906979 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c004230 con 0x7fb6ac0ce3c0 2023-07-06T03:45:25.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.536+0000 7fb6ab7fe700 1 -- 172.21.15.97:0/4087906979 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c004390 con 0x7fb6ac0ce3c0 2023-07-06T03:45:25.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.537+0000 7fb6ab7fe700 1 -- 172.21.15.97:0/4087906979 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c0036a0 con 0x7fb6ac0ce3c0 2023-07-06T03:45:25.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.537+0000 7fb6b2bff700 1 -- 172.21.15.97:0/4087906979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 msgr2=0x7fb6ac0cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.537+0000 7fb6b2bff700 1 --2- 172.21.15.97:0/4087906979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 0x7fb6ac0cc4c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0098f0 tx=0x7fb69c009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:25.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.538+0000 7fb6b2bff700 1 -- 172.21.15.97:0/4087906979 shutdown_connections 2023-07-06T03:45:25.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.538+0000 7fb6b2bff700 1 --2- 172.21.15.97:0/4087906979 >> v2:172.21.15.146:3300/0 conn(0x7fb6ac160a20 0x7fb6ac0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.538+0000 7fb6b2bff700 1 --2- 172.21.15.97:0/4087906979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 0x7fb6ac0cc4c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.538+0000 7fb6b2bff700 1 -- 172.21.15.97:0/4087906979 >> 172.21.15.97:0/4087906979 conn(0x7fb6ac0cf610 msgr2=0x7fb6ac0cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:25.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.539+0000 7fb6b2bff700 1 -- 172.21.15.97:0/4087906979 shutdown_connections 2023-07-06T03:45:25.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.539+0000 7fb6b2bff700 1 -- 172.21.15.97:0/4087906979 wait complete. 2023-07-06T03:45:25.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.541+0000 7fb6b2bff700 1 Processor -- start 2023-07-06T03:45:25.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.543+0000 7fb6b2bff700 1 -- start start 2023-07-06T03:45:25.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.544+0000 7fb6b2bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6ac0cca00 0x7fb6ac163400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.544+0000 7fb6b2bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 0x7fb6ac0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.544+0000 7fb6b2bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6ac160a20 0x7fb6ac0c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.544+0000 7fb6b2bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0b0f80 con 0x7fb6ac0cca00 2023-07-06T03:45:25.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.544+0000 7fb6b2bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0b0300 con 0x7fb6ac160a20 2023-07-06T03:45:25.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.544+0000 7fb6b2bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0b05e0 con 0x7fb6ac0ce3c0 2023-07-06T03:45:25.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.547+0000 7fb6b119c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6ac160a20 0x7fb6ac0c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.547+0000 7fb6b119c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6ac160a20 0x7fb6ac0c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57734/0 (socket says 172.21.15.97:57734) 2023-07-06T03:45:25.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.547+0000 7fb6b119c700 1 -- 172.21.15.97:0/1939942374 learned_addr learned my addr 172.21.15.97:0/1939942374 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.547+0000 7fb6abfff700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 0x7fb6ac0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.548+0000 7fb6b119c700 1 -- 172.21.15.97:0/1939942374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 msgr2=0x7fb6ac0af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.548+0000 7fb6b119c700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 0x7fb6ac0af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.548+0000 7fb6b119c700 1 -- 172.21.15.97:0/1939942374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6ac0cca00 msgr2=0x7fb6ac163400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.548+0000 7fb6b119c700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6ac0cca00 0x7fb6ac163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.548+0000 7fb6b119c700 1 -- 172.21.15.97:0/1939942374 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c009550 con 0x7fb6ac160a20 2023-07-06T03:45:25.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.549+0000 7fb6b119c700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6ac160a20 0x7fb6ac0c62d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400b990 tx=0x7fb6a400bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.549+0000 7fb6a97fa700 1 -- 172.21.15.97:0/1939942374 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a400c930 con 0x7fb6ac160a20 2023-07-06T03:45:25.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.549+0000 7fb6a97fa700 1 -- 172.21.15.97:0/1939942374 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a400ca90 con 0x7fb6ac160a20 2023-07-06T03:45:25.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.549+0000 7fb6a97fa700 1 -- 172.21.15.97:0/1939942374 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a400ce40 con 0x7fb6ac160a20 2023-07-06T03:45:25.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.549+0000 7fb6b2bff700 1 -- 172.21.15.97:0/1939942374 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac0b08c0 con 0x7fb6ac160a20 2023-07-06T03:45:25.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.550+0000 7fb6b2bff700 1 -- 172.21.15.97:0/1939942374 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac0ae700 con 0x7fb6ac160a20 2023-07-06T03:45:25.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.553+0000 7fb6b2bff700 1 -- 172.21.15.97:0/1939942374 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68c0052f0 con 0x7fb6ac160a20 2023-07-06T03:45:25.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.558+0000 7fb6a97fa700 1 -- 172.21.15.97:0/1939942374 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb6a4006d50 con 0x7fb6ac160a20 2023-07-06T03:45:25.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.560+0000 7fb6a97fa700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69806ad30 0x7fb69806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.560+0000 7fb6a97fa700 1 -- 172.21.15.97:0/1939942374 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb6a40891f0 con 0x7fb6ac160a20 2023-07-06T03:45:25.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.561+0000 7fb6b099b700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69806ad30 0x7fb69806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.562+0000 7fb6b099b700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69806ad30 0x7fb69806d1e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0006b20 tx=0x7fb6a0006ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.566+0000 7fb6a97fa700 1 -- 172.21.15.97:0/1939942374 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb6a4054310 con 0x7fb6ac160a20 2023-07-06T03:45:25.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.617+0000 7f9059fff700 1 Processor -- start 2023-07-06T03:45:25.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.617+0000 7f9059fff700 1 -- start start 2023-07-06T03:45:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.619+0000 7f9059fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f90540ce3c0 0x7f90540ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.620+0000 7f9059fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f90540ccbd0 0x7f90540c4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.620+0000 7f9059fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90540cda30 con 0x7f90540cd060 2023-07-06T03:45:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.620+0000 7f9059fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905408ae50 con 0x7f90540ccbd0 2023-07-06T03:45:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.620+0000 7f9059fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90540c9300 con 0x7f90540ce3c0 2023-07-06T03:45:25.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.622+0000 7f90537fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f90540ce3c0 0x7f90540ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.622+0000 7f90537fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f90540ce3c0 0x7f90540ad7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57750/0 (socket says 172.21.15.97:57750) 2023-07-06T03:45:25.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.622+0000 7f90537fe700 1 -- 172.21.15.97:0/4257157422 learned_addr learned my addr 172.21.15.97:0/4257157422 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.624+0000 7f90537fe700 1 -- 172.21.15.97:0/4257157422 >> v2:172.21.15.97:3300/0 conn(0x7f90540ccbd0 msgr2=0x7f90540c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.624+0000 7f90537fe700 1 --2- 172.21.15.97:0/4257157422 >> v2:172.21.15.97:3300/0 conn(0x7f90540ccbd0 0x7f90540c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.624+0000 7f90537fe700 1 -- 172.21.15.97:0/4257157422 >> v1:172.21.15.146:6789/0 conn(0x7f90540cd060 legacy=0x7f90540cc4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T03:45:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.624+0000 7f90537fe700 1 -- 172.21.15.97:0/4257157422 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90540cacb0 con 0x7f90540ce3c0 2023-07-06T03:45:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.624+0000 7f90537fe700 1 --2- 172.21.15.97:0/4257157422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90540ce3c0 0x7f90540ad7f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f905406f210 tx=0x7f904c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af8ff22868897a7b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.625+0000 7f90527fc700 1 -- 172.21.15.97:0/4257157422 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c00e070 con 0x7f90540ce3c0 2023-07-06T03:45:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.625+0000 7f90527fc700 1 -- 172.21.15.97:0/4257157422 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c003a20 con 0x7f90540ce3c0 2023-07-06T03:45:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.625+0000 7f90527fc700 1 -- 172.21.15.97:0/4257157422 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c003dd0 con 0x7f90540ce3c0 2023-07-06T03:45:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.626+0000 7f9059fff700 1 -- 172.21.15.97:0/4257157422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90540ce3c0 msgr2=0x7f90540ad7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.626+0000 7f9059fff700 1 --2- 172.21.15.97:0/4257157422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90540ce3c0 0x7f90540ad7f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f905406f210 tx=0x7f904c00b330 comp rx=0 tx=0).stop 2023-07-06T03:45:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.627+0000 7f9059fff700 1 -- 172.21.15.97:0/4257157422 shutdown_connections 2023-07-06T03:45:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.627+0000 7f9059fff700 1 --2- 172.21.15.97:0/4257157422 >> v2:172.21.15.97:3300/0 conn(0x7f90540ccbd0 0x7f90540c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.627+0000 7f9059fff700 1 --2- 172.21.15.97:0/4257157422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90540ce3c0 0x7f90540ad7f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.627+0000 7f9059fff700 1 -- 172.21.15.97:0/4257157422 >> 172.21.15.97:0/4257157422 conn(0x7f90540cf610 msgr2=0x7f90540cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:25.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.627+0000 7f9059fff700 1 -- 172.21.15.97:0/4257157422 shutdown_connections 2023-07-06T03:45:25.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.627+0000 7f9059fff700 1 -- 172.21.15.97:0/4257157422 wait complete. 2023-07-06T03:45:25.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.630+0000 7f9059fff700 1 Processor -- start 2023-07-06T03:45:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.630+0000 7f9059fff700 1 -- start start 2023-07-06T03:45:25.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.631+0000 7f9059fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90540ccbd0 0x7f90540b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.631+0000 7f9059fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90540cd060 0x7f9054088c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.631+0000 7f9059fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90540ce3c0 0x7f90540b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.631+0000 7f9059fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90540af680 con 0x7f90540ccbd0 2023-07-06T03:45:25.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.632+0000 7f9059fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90540afee0 con 0x7f90540cd060 2023-07-06T03:45:25.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.632+0000 7f9059fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905415f690 con 0x7f90540ce3c0 2023-07-06T03:45:25.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.632+0000 7f9053fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90540ce3c0 0x7f90540b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.633+0000 7f9053fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90540ce3c0 0x7f90540b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54164/0 (socket says 172.21.15.97:54164) 2023-07-06T03:45:25.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.633+0000 7f9053fff700 1 -- 172.21.15.97:0/993996622 learned_addr learned my addr 172.21.15.97:0/993996622 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.633+0000 7f90537fe700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90540ccbd0 0x7f90540b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.633+0000 7f9052ffd700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90540cd060 0x7f9054088c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.633+0000 7f9053fff700 1 -- 172.21.15.97:0/993996622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90540cd060 msgr2=0x7f9054088c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.633+0000 7f9053fff700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90540cd060 0x7f9054088c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.633+0000 7f9053fff700 1 -- 172.21.15.97:0/993996622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90540ccbd0 msgr2=0x7f90540b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.633+0000 7f9053fff700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90540ccbd0 0x7f90540b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.633+0000 7f9053fff700 1 -- 172.21.15.97:0/993996622 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c009b30 con 0x7f90540ce3c0 2023-07-06T03:45:25.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.634+0000 7f9053fff700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90540ce3c0 0x7f90540b0370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f904800b450 tx=0x7f904800b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.634+0000 7fbca3bff700 1 Processor -- start 2023-07-06T03:45:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.634+0000 7fbca3bff700 1 -- start start 2023-07-06T03:45:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.635+0000 7f903bfff700 1 -- 172.21.15.97:0/993996622 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048011830 con 0x7f90540ce3c0 2023-07-06T03:45:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.635+0000 7f903bfff700 1 -- 172.21.15.97:0/993996622 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9048011990 con 0x7f90540ce3c0 2023-07-06T03:45:25.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.635+0000 7f903bfff700 1 -- 172.21.15.97:0/993996622 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048011d40 con 0x7f90540ce3c0 2023-07-06T03:45:25.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.635+0000 7f9059fff700 1 -- 172.21.15.97:0/993996622 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90540b1410 con 0x7f90540ce3c0 2023-07-06T03:45:25.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.635+0000 7f9059fff700 1 -- 172.21.15.97:0/993996622 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90540afb70 con 0x7f90540ce3c0 2023-07-06T03:45:25.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.637+0000 7fbca3bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc9c160a20 0x7fbc9c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.637+0000 7fbca3bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbc9c0ce3c0 0x7fbc9c0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.637+0000 7fbca3bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9c0caa30 con 0x7fbc9c0cca00 2023-07-06T03:45:25.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.638+0000 7fbca3bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9c0c5260 con 0x7fbc9c160a20 2023-07-06T03:45:25.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.638+0000 7fbca3bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9c0cce20 con 0x7fbc9c0ce3c0 2023-07-06T03:45:25.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.638+0000 7f9f6a7ff700 1 Processor -- start 2023-07-06T03:45:25.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.638+0000 7fbca199b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc9c160a20 0x7fbc9c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.638+0000 7fbca199b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc9c160a20 0x7fbc9c0c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51672/0 (socket says 172.21.15.97:51672) 2023-07-06T03:45:25.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.638+0000 7fbca199b700 1 -- 172.21.15.97:0/4204985126 learned_addr learned my addr 172.21.15.97:0/4204985126 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.639+0000 7fbca119a700 1 --2- 172.21.15.97:0/4204985126 >> v2:172.21.15.146:3300/0 conn(0x7fbc9c0ce3c0 0x7fbc9c0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.633+0000 7fa26c5ff700 1 Processor -- start 2023-07-06T03:45:25.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.633+0000 7fa26c5ff700 1 -- start start 2023-07-06T03:45:25.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.635+0000 7fa26c5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa26408aa00 0x7fa2640ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.635+0000 7fa26c5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa2640cc4c0 0x7fa2640cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.635+0000 7fa26c5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2640caa30 con 0x7fa2640ce3c0 2023-07-06T03:45:25.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.635+0000 7fa26c5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2640cb040 con 0x7fa26408aa00 2023-07-06T03:45:25.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.635+0000 7fa26c5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26408add0 con 0x7fa2640cc4c0 2023-07-06T03:45:25.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.636+0000 7fa26a39b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa26408aa00 0x7fa2640ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.636+0000 7fa269b9a700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa2640ce3c0 0x7fa2640cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:35356/0 (socket says 172.21.15.97:35356) 2023-07-06T03:45:25.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.636+0000 7fa269b9a700 1 -- 172.21.15.97:0/3015817010 learned_addr learned my addr 172.21.15.97:0/3015817010 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:25.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.638+0000 7f9059fff700 1 -- 172.21.15.97:0/993996622 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90540b0070 con 0x7f90540ce3c0 2023-07-06T03:45:25.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.639+0000 7fbca0999700 1 -- 172.21.15.97:0/4204985126 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802832665 0 0) 0x7fbc9c0caa30 con 0x7fbc9c0cca00 2023-07-06T03:45:25.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.639+0000 7fbca199b700 1 -- 172.21.15.97:0/4204985126 >> v2:172.21.15.146:3300/0 conn(0x7fbc9c0ce3c0 msgr2=0x7fbc9c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:25.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.639+0000 7fbca199b700 1 --2- 172.21.15.97:0/4204985126 >> v2:172.21.15.146:3300/0 conn(0x7fbc9c0ce3c0 0x7fbc9c0cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.639+0000 7fbca199b700 1 -- 172.21.15.97:0/4204985126 >> v1:172.21.15.97:6789/0 conn(0x7fbc9c0cca00 legacy=0x7fbc9c164520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.639+0000 7fbca199b700 1 -- 172.21.15.97:0/4204985126 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c160df0 con 0x7fbc9c160a20 2023-07-06T03:45:25.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.639+0000 7fbca119a700 1 --2- 172.21.15.97:0/4204985126 >> v2:172.21.15.146:3300/0 conn(0x7fbc9c0ce3c0 0x7fbc9c0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:25.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.638+0000 7fa26ab9c700 1 --2- 172.21.15.97:0/3015817010 >> v2:172.21.15.146:3300/0 conn(0x7fa2640cc4c0 0x7fa2640cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.640+0000 7fbca199b700 1 --2- 172.21.15.97:0/4204985126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9c160a20 0x7fbc9c0c8f40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c06b640 tx=0x7fbc98009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1007aaa10e8807a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.640+0000 7fa26a39b700 1 -- 172.21.15.97:0/3015817010 >> v2:172.21.15.146:3300/0 conn(0x7fa2640cc4c0 msgr2=0x7fa2640cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:25.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.640+0000 7fa26a39b700 1 --2- 172.21.15.97:0/3015817010 >> v2:172.21.15.146:3300/0 conn(0x7fa2640cc4c0 0x7fa2640cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.640+0000 7fa26a39b700 1 -- 172.21.15.97:0/3015817010 >> v1:172.21.15.146:6789/0 conn(0x7fa2640ce3c0 legacy=0x7fa2640cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.640+0000 7fa26a39b700 1 -- 172.21.15.97:0/3015817010 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640cce20 con 0x7fa26408aa00 2023-07-06T03:45:25.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.638+0000 7f9f6a7ff700 1 -- start start 2023-07-06T03:45:25.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.640+0000 7fbca0999700 1 -- 172.21.15.97:0/4204985126 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9800b870 con 0x7fbc9c160a20 2023-07-06T03:45:25.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.640+0000 7fbca0999700 1 -- 172.21.15.97:0/4204985126 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9800b9d0 con 0x7fbc9c160a20 2023-07-06T03:45:25.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.640+0000 7fbca0999700 1 -- 172.21.15.97:0/4204985126 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9800bd60 con 0x7fbc9c160a20 2023-07-06T03:45:25.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.640+0000 7fa26a39b700 1 --2- 172.21.15.97:0/3015817010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26408aa00 0x7fa2640ad7d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa26406ad80 tx=0x7fa25c00c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb451b331f0f515 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7f9f6a7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9f640add10 0x7f9f640c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7f9f6a7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640c9650 0x7f9f6406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7f9f6a7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f640c3b70 con 0x7f9f640c5150 2023-07-06T03:45:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7f9f6a7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f640c47f0 con 0x7f9f640add10 2023-07-06T03:45:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7f9f6a7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640c3e50 con 0x7f9f640c9650 2023-07-06T03:45:25.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.640+0000 7f903bfff700 1 -- 172.21.15.97:0/993996622 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f904801a400 con 0x7f90540ce3c0 2023-07-06T03:45:25.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7f9f68d9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640c9650 0x7f9f6406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7f9f68d9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640c9650 0x7f9f6406f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54180/0 (socket says 172.21.15.97:54180) 2023-07-06T03:45:25.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7f9f68d9c700 1 -- 172.21.15.97:0/611745329 learned_addr learned my addr 172.21.15.97:0/611745329 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7f9f63fff700 1 --2- 172.21.15.97:0/611745329 >> v2:172.21.15.146:3300/0 conn(0x7f9f640add10 0x7f9f640c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7fbca3bff700 1 -- 172.21.15.97:0/4204985126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9c160a20 msgr2=0x7fbc9c0c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7fbca3bff700 1 --2- 172.21.15.97:0/4204985126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9c160a20 0x7fbc9c0c8f40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c06b640 tx=0x7fbc98009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.641+0000 7fbca3bff700 1 -- 172.21.15.97:0/4204985126 shutdown_connections 2023-07-06T03:45:25.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7fbca3bff700 1 --2- 172.21.15.97:0/4204985126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9c160a20 0x7fbc9c0c8f40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7fbca3bff700 1 --2- 172.21.15.97:0/4204985126 >> v2:172.21.15.146:3300/0 conn(0x7fbc9c0ce3c0 0x7fbc9c0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7fbca3bff700 1 -- 172.21.15.97:0/4204985126 >> 172.21.15.97:0/4204985126 conn(0x7fbc9c0cf610 msgr2=0x7fbc9c0cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:25.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7f9f68d9c700 1 -- 172.21.15.97:0/611745329 >> v2:172.21.15.146:3300/0 conn(0x7f9f640add10 msgr2=0x7f9f640c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:25.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7f9f68d9c700 1 --2- 172.21.15.97:0/611745329 >> v2:172.21.15.146:3300/0 conn(0x7f9f640add10 0x7f9f640c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7f9f68d9c700 1 -- 172.21.15.97:0/611745329 >> v1:172.21.15.146:6789/0 conn(0x7f9f640c5150 legacy=0x7f9f640c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7f9f68d9c700 1 -- 172.21.15.97:0/611745329 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f640c4130 con 0x7f9f640c9650 2023-07-06T03:45:25.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7f903bfff700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f904006adb0 0x7f904006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7f903bfff700 1 -- 172.21.15.97:0/993996622 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9048089780 con 0x7f90540ce3c0 2023-07-06T03:45:25.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7fbca3bff700 1 -- 172.21.15.97:0/4204985126 shutdown_connections 2023-07-06T03:45:25.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7fbca3bff700 1 -- 172.21.15.97:0/4204985126 wait complete. 2023-07-06T03:45:25.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7fa8c45ff700 1 Processor -- start 2023-07-06T03:45:25.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7fa8c45ff700 1 -- start start 2023-07-06T03:45:25.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.642+0000 7f9f68d9c700 1 --2- 172.21.15.97:0/611745329 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640c9650 0x7f9f6406f9e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400bb90 tx=0x7f9f5400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79a4efbd905d0a4c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.643+0000 7f126fdff700 1 Processor -- start 2023-07-06T03:45:25.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.643+0000 7fa269399700 1 -- 172.21.15.97:0/3015817010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00aea0 con 0x7fa26408aa00 2023-07-06T03:45:25.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.643+0000 7fa269399700 1 -- 172.21.15.97:0/3015817010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25c008e90 con 0x7fa26408aa00 2023-07-06T03:45:25.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.643+0000 7fa269399700 1 -- 172.21.15.97:0/3015817010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c003a20 con 0x7fa26408aa00 2023-07-06T03:45:25.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.643+0000 7f126fdff700 1 -- start start 2023-07-06T03:45:25.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.643+0000 7f90537fe700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f904006adb0 0x7f904006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.644+0000 7f90537fe700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f904006adb0 0x7f904006d260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f904c00b370 tx=0x7f904c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.644+0000 7fa26c5ff700 1 -- 172.21.15.97:0/3015817010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26408aa00 msgr2=0x7fa2640ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.644+0000 7fa26c5ff700 1 --2- 172.21.15.97:0/3015817010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26408aa00 0x7fa2640ad7d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa26406ad80 tx=0x7fa25c00c3b0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f903bfff700 1 -- 172.21.15.97:0/993996622 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f90480548a0 con 0x7f90540ce3c0 2023-07-06T03:45:25.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.644+0000 7fa8c45ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa8bc0cc4c0 0x7fa8bc171aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.644+0000 7fa8c45ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa8bc16d9e0 0x7fa8bc0c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7fa8c45ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8bc0cd480 con 0x7fa8bc16d610 2023-07-06T03:45:25.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7fa8c45ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8bc0c3f40 con 0x7fa8bc16d9e0 2023-07-06T03:45:25.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7fa8c45ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8bc0ccf00 con 0x7fa8bc0cc4c0 2023-07-06T03:45:25.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7fbca3bff700 1 Processor -- start 2023-07-06T03:45:25.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f2c559ff700 1 Processor -- start 2023-07-06T03:45:25.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.644+0000 7f9f62ffd700 1 -- 172.21.15.97:0/611745329 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400dd30 con 0x7f9f640c9650 2023-07-06T03:45:25.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.644+0000 7f9f62ffd700 1 -- 172.21.15.97:0/611745329 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5400de90 con 0x7f9f640c9650 2023-07-06T03:45:25.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.644+0000 7f9f62ffd700 1 -- 172.21.15.97:0/611745329 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54010690 con 0x7f9f640c9650 2023-07-06T03:45:25.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/611745329 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640c9650 msgr2=0x7f9f6406f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f9f6a7ff700 1 --2- 172.21.15.97:0/611745329 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640c9650 0x7f9f6406f9e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400bb90 tx=0x7f9f5400bea0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/611745329 shutdown_connections 2023-07-06T03:45:25.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f9f6a7ff700 1 --2- 172.21.15.97:0/611745329 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640c9650 0x7f9f6406f9e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f9f6a7ff700 1 --2- 172.21.15.97:0/611745329 >> v2:172.21.15.146:3300/0 conn(0x7f9f640add10 0x7f9f640c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/611745329 >> 172.21.15.97:0/611745329 conn(0x7f9f640cf610 msgr2=0x7f9f640cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:25.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/611745329 shutdown_connections 2023-07-06T03:45:25.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/611745329 wait complete. 2023-07-06T03:45:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.646+0000 7fa8c1b9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa8bc0cc4c0 0x7fa8bc171aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.646+0000 7fa8c1b9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa8bc0cc4c0 0x7fa8bc171aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57790/0 (socket says 172.21.15.97:57790) 2023-07-06T03:45:25.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.646+0000 7fa8c1b9a700 1 -- 172.21.15.97:0/2084389919 learned_addr learned my addr 172.21.15.97:0/2084389919 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f126fdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 0x7f12680cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f126fdff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f12680d1ba0 0x7f12680d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f126fdff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12680cdd90 con 0x7f12680b8890 2023-07-06T03:45:25.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f126fdff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12680b5280 con 0x7f12680d1ba0 2023-07-06T03:45:25.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f126fdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12680d0de0 con 0x7f12680cd400 2023-07-06T03:45:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7f2c559ff700 1 -- start start 2023-07-06T03:45:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.646+0000 7fa8c1b9a700 1 -- 172.21.15.97:0/2084389919 >> v2:172.21.15.97:3300/0 conn(0x7fa8bc16d9e0 msgr2=0x7fa8bc0c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.646+0000 7fa8c1b9a700 1 --2- 172.21.15.97:0/2084389919 >> v2:172.21.15.97:3300/0 conn(0x7fa8bc16d9e0 0x7fa8bc0c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.646+0000 7fa8c1b9a700 1 -- 172.21.15.97:0/2084389919 >> v1:172.21.15.146:6789/0 conn(0x7fa8bc16d610 legacy=0x7fa8bc171390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T03:45:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.646+0000 7fa8c1b9a700 1 -- 172.21.15.97:0/2084389919 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc0cd1a0 con 0x7fa8bc0cc4c0 2023-07-06T03:45:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7fa8c1b9a700 1 --2- 172.21.15.97:0/2084389919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8bc0cc4c0 0x7fa8bc171aa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0ae4d0 tx=0x7fa8ac009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de3ef033731f3dd8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.645+0000 7fbca3bff700 1 -- start start 2023-07-06T03:45:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7fbca3bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc9c0cca00 0x7fbc9c0af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7fbca3bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9c0ce3c0 0x7fbc9c0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7fbca3bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc9c160a20 0x7fbc9c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7fbca3bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c0af410 con 0x7fbc9c0ce3c0 2023-07-06T03:45:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7fbca3bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c0ae540 con 0x7fbc9c0cca00 2023-07-06T03:45:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7fbca3bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c0ae790 con 0x7fbc9c160a20 2023-07-06T03:45:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7fbca219c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc9c160a20 0x7fbc9c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7fbca219c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc9c160a20 0x7fbc9c0b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54196/0 (socket says 172.21.15.97:54196) 2023-07-06T03:45:25.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7fbca219c700 1 -- 172.21.15.97:0/167165919 learned_addr learned my addr 172.21.15.97:0/167165919 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7f126db9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f12680b8890 0x7f12680b4830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35522/0 (socket says 172.21.15.97:35522) 2023-07-06T03:45:25.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7f126db9b700 1 -- 172.21.15.97:0/1784493747 learned_addr learned my addr 172.21.15.97:0/1784493747 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:25.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7f126e39c700 1 --2- 172.21.15.97:0/1784493747 >> v2:172.21.15.97:3300/0 conn(0x7f12680d1ba0 0x7f12680d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.648+0000 7fa8c1399700 1 -- 172.21.15.97:0/2084389919 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac004230 con 0x7fa8bc0cc4c0 2023-07-06T03:45:25.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.648+0000 7fa8c1399700 1 -- 172.21.15.97:0/2084389919 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac004390 con 0x7fa8bc0cc4c0 2023-07-06T03:45:25.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.648+0000 7fa8c1399700 1 -- 172.21.15.97:0/2084389919 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0036a0 con 0x7fa8bc0cc4c0 2023-07-06T03:45:25.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7fbca199b700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc9c0cca00 0x7fbc9c0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7f2c559ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c50160770 0x7f2c500b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7f2c559ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5006c5c0 con 0x7f2c500c4a60 2023-07-06T03:45:25.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7f2c559ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c500ce760 con 0x7f2c500adb80 2023-07-06T03:45:25.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.647+0000 7f2c559ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500b1480 con 0x7f2c50160770 2023-07-06T03:45:25.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.648+0000 7f9f6a7ff700 1 Processor -- start 2023-07-06T03:45:25.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.646+0000 7fa26c5ff700 1 -- 172.21.15.97:0/3015817010 shutdown_connections 2023-07-06T03:45:25.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.646+0000 7fa26c5ff700 1 --2- 172.21.15.97:0/3015817010 >> v2:172.21.15.146:3300/0 conn(0x7fa2640cc4c0 0x7fa2640cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.646+0000 7fa26c5ff700 1 --2- 172.21.15.97:0/3015817010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26408aa00 0x7fa2640ad7d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.646+0000 7fa26c5ff700 1 -- 172.21.15.97:0/3015817010 >> 172.21.15.97:0/3015817010 conn(0x7fa2640cf610 msgr2=0x7fa2640cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:25.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.648+0000 7f2c4effd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c50160770 0x7f2c500b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.648+0000 7f2c4effd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c50160770 0x7f2c500b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54208/0 (socket says 172.21.15.97:54208) 2023-07-06T03:45:25.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.648+0000 7f2c4effd700 1 -- 172.21.15.97:0/3457744329 learned_addr learned my addr 172.21.15.97:0/3457744329 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7fa8c45ff700 1 -- 172.21.15.97:0/2084389919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8bc0cc4c0 msgr2=0x7fa8bc171aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7fa8c45ff700 1 --2- 172.21.15.97:0/2084389919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8bc0cc4c0 0x7fa8bc171aa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0ae4d0 tx=0x7fa8ac009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7f2c4effd700 1 -- 172.21.15.97:0/3457744329 >> v1:172.21.15.146:6789/0 conn(0x7f2c500adb80 legacy=0x7f2c500ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7f2c4effd700 1 -- 172.21.15.97:0/3457744329 >> v1:172.21.15.97:6789/0 conn(0x7f2c500c4a60 legacy=0x7f2c50164480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:25.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7f2c4effd700 1 -- 172.21.15.97:0/3457744329 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5006c8e0 con 0x7f2c50160770 2023-07-06T03:45:25.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.648+0000 7f126d39a700 1 --2- 172.21.15.97:0/1784493747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 0x7f12680cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7fa8c45ff700 1 -- 172.21.15.97:0/2084389919 shutdown_connections 2023-07-06T03:45:25.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7fa8c45ff700 1 --2- 172.21.15.97:0/2084389919 >> v2:172.21.15.97:3300/0 conn(0x7fa8bc16d9e0 0x7fa8bc0c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7fa8c45ff700 1 --2- 172.21.15.97:0/2084389919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8bc0cc4c0 0x7fa8bc171aa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7fa8c45ff700 1 -- 172.21.15.97:0/2084389919 >> 172.21.15.97:0/2084389919 conn(0x7fa8bc0cf610 msgr2=0x7fa8bc0cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:25.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.648+0000 7fbca119a700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9c0ce3c0 0x7fbc9c0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7fbca219c700 1 -- 172.21.15.97:0/167165919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc9c0cca00 msgr2=0x7fbc9c0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7fbca219c700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc9c0cca00 0x7fbc9c0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7fbca219c700 1 -- 172.21.15.97:0/167165919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9c0ce3c0 msgr2=0x7fbc9c0b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7fbca219c700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9c0ce3c0 0x7fbc9c0b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7fbca219c700 1 -- 172.21.15.97:0/167165919 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc98009550 con 0x7fbc9c160a20 2023-07-06T03:45:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7f126cb99700 1 -- 172.21.15.97:0/1784493747 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104177793 0 0) 0x7f125c001490 con 0x7f12680b8890 2023-07-06T03:45:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7f126cb99700 1 -- 172.21.15.97:0/1784493747 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12580035b0 con 0x7f12680b8890 2023-07-06T03:45:25.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.648+0000 7f9f6a7ff700 1 -- start start 2023-07-06T03:45:25.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f9f6a7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640add10 0x7f9f641506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f9f6a7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f640c5150 0x7f9f641fa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f9f6a7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f640c9650 0x7f9f641fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f9f6a7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f64151de0 con 0x7f9f640c5150 2023-07-06T03:45:25.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f9f6a7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f64151160 con 0x7f9f640c9650 2023-07-06T03:45:25.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f9f6a7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f64151440 con 0x7f9f640add10 2023-07-06T03:45:25.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7fbca219c700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc9c160a20 0x7fbc9c0b3aa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400c7d0 tx=0x7fbc9400cae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7f126d39a700 1 -- 172.21.15.97:0/1784493747 >> v2:172.21.15.97:3300/0 conn(0x7f12680d1ba0 msgr2=0x7f12680d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:25.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7f126d39a700 1 --2- 172.21.15.97:0/1784493747 >> v2:172.21.15.97:3300/0 conn(0x7f12680d1ba0 0x7f12680d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.649+0000 7f126d39a700 1 -- 172.21.15.97:0/1784493747 >> v1:172.21.15.97:6789/0 conn(0x7f12680b8890 legacy=0x7f12680b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f126d39a700 1 -- 172.21.15.97:0/1784493747 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12680d10c0 con 0x7f12680cd400 2023-07-06T03:45:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f126d39a700 1 --2- 172.21.15.97:0/1784493747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 0x7f12680cd850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1254009840 tx=0x7f1254009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a133a9ac08e5955 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f2c4effd700 1 --2- 172.21.15.97:0/3457744329 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c50160770 0x7f2c500b0f40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009e80 tx=0x7f2c4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa6623df25e59017 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7fa8c45ff700 1 -- 172.21.15.97:0/2084389919 shutdown_connections 2023-07-06T03:45:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f2c4dffb700 1 -- 172.21.15.97:0/3457744329 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4800e070 con 0x7f2c50160770 2023-07-06T03:45:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f2c4dffb700 1 -- 172.21.15.97:0/3457744329 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c48003a20 con 0x7f2c50160770 2023-07-06T03:45:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7f2c4dffb700 1 -- 172.21.15.97:0/3457744329 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c48003dd0 con 0x7f2c50160770 2023-07-06T03:45:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f2c559ff700 1 -- 172.21.15.97:0/3457744329 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c50160770 msgr2=0x7f2c500b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f2c559ff700 1 --2- 172.21.15.97:0/3457744329 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c50160770 0x7f2c500b0f40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009e80 tx=0x7f2c4800b330 comp rx=0 tx=0).stop 2023-07-06T03:45:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7fbc927fc700 1 -- 172.21.15.97:0/167165919 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94003e30 con 0x7fbc9c160a20 2023-07-06T03:45:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7fbc927fc700 1 -- 172.21.15.97:0/167165919 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc94005070 con 0x7fbc9c160a20 2023-07-06T03:45:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7fbc927fc700 1 -- 172.21.15.97:0/167165919 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94005420 con 0x7fbc9c160a20 2023-07-06T03:45:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7fbca3bff700 1 -- 172.21.15.97:0/167165919 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9c0aea70 con 0x7fbc9c160a20 2023-07-06T03:45:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f126cb99700 1 -- 172.21.15.97:0/1784493747 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254011830 con 0x7f12680cd400 2023-07-06T03:45:25.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.652+0000 7f126cb99700 1 -- 172.21.15.97:0/1784493747 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1254011990 con 0x7f12680cd400 2023-07-06T03:45:25.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.652+0000 7f126cb99700 1 -- 172.21.15.97:0/1784493747 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254011d20 con 0x7f12680cd400 2023-07-06T03:45:25.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.650+0000 7fa8c45ff700 1 -- 172.21.15.97:0/2084389919 wait complete. 2023-07-06T03:45:25.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.652+0000 7fa26c5ff700 1 -- 172.21.15.97:0/3015817010 shutdown_connections 2023-07-06T03:45:25.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.652+0000 7fa26c5ff700 1 -- 172.21.15.97:0/3015817010 wait complete. 2023-07-06T03:45:25.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.652+0000 7f2c559ff700 1 -- 172.21.15.97:0/3457744329 shutdown_connections 2023-07-06T03:45:25.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.652+0000 7f2c559ff700 1 --2- 172.21.15.97:0/3457744329 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c50160770 0x7f2c500b0f40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.652+0000 7f2c559ff700 1 -- 172.21.15.97:0/3457744329 >> 172.21.15.97:0/3457744329 conn(0x7f2c5006a060 msgr2=0x7f2c5006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:25.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f9f63fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640add10 0x7f9f641506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f9f637fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f640c5150 0x7f9f641fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f9f637fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f640c5150 0x7f9f641fa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51704/0 (socket says 172.21.15.97:51704) 2023-07-06T03:45:25.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f9f63fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640add10 0x7f9f641506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54224/0 (socket says 172.21.15.97:54224) 2023-07-06T03:45:25.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f9f637fe700 1 -- 172.21.15.97:0/1379382419 learned_addr learned my addr 172.21.15.97:0/1379382419 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f9f637fe700 1 -- 172.21.15.97:0/1379382419 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640add10 msgr2=0x7f9f641506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f9f637fe700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640add10 0x7f9f641506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f9f637fe700 1 -- 172.21.15.97:0/1379382419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f640c9650 msgr2=0x7f9f641fa680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T03:45:25.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f9f637fe700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f640c9650 0x7f9f641fa680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.651+0000 7f9f637fe700 1 -- 172.21.15.97:0/1379382419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5400b7f0 con 0x7f9f640c5150 2023-07-06T03:45:25.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.652+0000 7f2c559ff700 1 -- 172.21.15.97:0/3457744329 shutdown_connections 2023-07-06T03:45:25.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.653+0000 7fa8c45ff700 1 Processor -- start 2023-07-06T03:45:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.653+0000 7fa8c45ff700 1 -- start start 2023-07-06T03:45:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7fa8c45ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8bc0cc4c0 0x7fa8bc0b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7fa8c45ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8bc16d610 0x7fa8bc0c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7fa8c45ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8bc16d9e0 0x7fa8bc0b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7fa8c45ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc0c7030 con 0x7fa8bc16d9e0 2023-07-06T03:45:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7fa8c45ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc0c7280 con 0x7fa8bc16d610 2023-07-06T03:45:25.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7fa8c45ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc0c7560 con 0x7fa8bc0cc4c0 2023-07-06T03:45:25.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.653+0000 7fbca3bff700 1 -- 172.21.15.97:0/167165919 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9c0af030 con 0x7fbc9c160a20 2023-07-06T03:45:25.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.652+0000 7f2c559ff700 1 -- 172.21.15.97:0/3457744329 wait complete. 2023-07-06T03:45:25.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7f2c559ff700 1 Processor -- start 2023-07-06T03:45:25.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7f2c559ff700 1 -- start start 2023-07-06T03:45:25.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7fa26c5ff700 1 Processor -- start 2023-07-06T03:45:25.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7fa26c5ff700 1 -- start start 2023-07-06T03:45:25.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.656+0000 7fa26c5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26408aa00 0x7fa2640a8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.656+0000 7fa26c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2640cc4c0 0x7fa2640af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7fa26c5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2640ce3c0 0x7fa2640afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7fa26c5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640b0100 con 0x7fa26408aa00 2023-07-06T03:45:25.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7fa26c5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640b03e0 con 0x7fa2640ce3c0 2023-07-06T03:45:25.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7fa26c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640b06c0 con 0x7fa2640cc4c0 2023-07-06T03:45:25.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7fa269b9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2640cc4c0 0x7fa2640af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7fa269b9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2640cc4c0 0x7fa2640af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54246/0 (socket says 172.21.15.97:54246) 2023-07-06T03:45:25.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7fa269b9a700 1 -- 172.21.15.97:0/4270657388 learned_addr learned my addr 172.21.15.97:0/4270657388 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.654+0000 7f9f637fe700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f640c5150 0x7f9f641fa140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00cba0 tx=0x7f9f5c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7f9f60ff9700 1 -- 172.21.15.97:0/1379382419 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c003c00 con 0x7f9f640c5150 2023-07-06T03:45:25.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7f2c559ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c500adb80 0x7f2c50207470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7f2c559ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c500c4a60 0x7f2c501635d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7f9f60ff9700 1 -- 172.21.15.97:0/1379382419 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c003d60 con 0x7f9f640c5150 2023-07-06T03:45:25.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7f9f60ff9700 1 -- 172.21.15.97:0/1379382419 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00abc0 con 0x7f9f640c5150 2023-07-06T03:45:25.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/1379382419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f64151720 con 0x7f9f640c5150 2023-07-06T03:45:25.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/1379382419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f641fabc0 con 0x7f9f640c5150 2023-07-06T03:45:25.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/1379382419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f440052f0 con 0x7f9f640c5150 2023-07-06T03:45:25.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7f2c559ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c50160770 0x7f2c501620c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7f2c559ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5008ad50 con 0x7f2c50160770 2023-07-06T03:45:25.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7f2c559ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50161c30 con 0x7f2c500adb80 2023-07-06T03:45:25.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7f2c559ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5015fcb0 con 0x7f2c500c4a60 2023-07-06T03:45:25.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa8c2b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8bc16d9e0 0x7fa8bc0b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa8c1b9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8bc16d610 0x7fa8bc0c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa8c1b9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8bc16d610 0x7fa8bc0c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57812/0 (socket says 172.21.15.97:57812) 2023-07-06T03:45:25.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa8c1b9a700 1 -- 172.21.15.97:0/30957421 learned_addr learned my addr 172.21.15.97:0/30957421 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa8c2b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8bc16d9e0 0x7fa8bc0b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51714/0 (socket says 172.21.15.97:51714) 2023-07-06T03:45:25.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa8c239b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8bc0cc4c0 0x7fa8bc0b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7f2c4e7fc700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c500c4a60 0x7f2c501635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7f2c4f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c50160770 0x7f2c501620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7f2c4e7fc700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c500c4a60 0x7f2c501635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54250/0 (socket says 172.21.15.97:54250) 2023-07-06T03:45:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7f2c4e7fc700 1 -- 172.21.15.97:0/3245389792 learned_addr learned my addr 172.21.15.97:0/3245389792 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.659+0000 7f2c4f7fe700 1 -- 172.21.15.97:0/3245389792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c500c4a60 msgr2=0x7f2c501635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.659+0000 7f2c4f7fe700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c500c4a60 0x7f2c501635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.659+0000 7f2c4f7fe700 1 -- 172.21.15.97:0/3245389792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c500adb80 msgr2=0x7f2c50207470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T03:45:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.659+0000 7f2c4f7fe700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c500adb80 0x7f2c50207470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.659+0000 7f2c4f7fe700 1 -- 172.21.15.97:0/3245389792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c48009b30 con 0x7f2c50160770 2023-07-06T03:45:25.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.660+0000 7f2c4f7fe700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c50160770 0x7f2c501620c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000b440 tx=0x7f2c4000b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.660+0000 7fa8c1b9a700 1 -- 172.21.15.97:0/30957421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8bc0cc4c0 msgr2=0x7fa8bc0b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.660+0000 7fa8c1b9a700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8bc0cc4c0 0x7fa8bc0b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.660+0000 7fa8c1b9a700 1 -- 172.21.15.97:0/30957421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8bc16d9e0 msgr2=0x7fa8bc0b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.660+0000 7fa8c1b9a700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8bc16d9e0 0x7fa8bc0b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.660+0000 7fa8c1b9a700 1 -- 172.21.15.97:0/30957421 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac009550 con 0x7fa8bc16d610 2023-07-06T03:45:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.655+0000 7fbc927fc700 1 -- 172.21.15.97:0/167165919 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbc94005580 con 0x7fbc9c160a20 2023-07-06T03:45:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.659+0000 7fbca3bff700 1 -- 172.21.15.97:0/167165919 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9c0af550 con 0x7fbc9c160a20 2023-07-06T03:45:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.660+0000 7fbc927fc700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc8006adb0 0x7fbc8006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.660+0000 7fbc927fc700 1 -- 172.21.15.97:0/167165919 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbc940889c0 con 0x7fbc9c160a20 2023-07-06T03:45:25.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.661+0000 7fbca199b700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc8006adb0 0x7fbc8006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7f126fdff700 1 -- 172.21.15.97:0/1784493747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 msgr2=0x7f12680cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7f126fdff700 1 --2- 172.21.15.97:0/1784493747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 0x7f12680cd850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1254009840 tx=0x7f1254009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.661+0000 7fbca199b700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc8006adb0 0x7fbc8006d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc98014040 tx=0x7fbc98008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.662+0000 7fa8c1b9a700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8bc16d610 0x7fa8bc0c8040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0cd040 tx=0x7fa8ac004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.657+0000 7fa26ab9c700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2640ce3c0 0x7fa2640afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa269b9a700 1 -- 172.21.15.97:0/4270657388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2640ce3c0 msgr2=0x7fa2640afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa269b9a700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2640ce3c0 0x7fa2640afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa269b9a700 1 -- 172.21.15.97:0/4270657388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26408aa00 msgr2=0x7fa2640a8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa269b9a700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26408aa00 0x7fa2640a8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa269b9a700 1 -- 172.21.15.97:0/4270657388 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c0089d0 con 0x7fa2640cc4c0 2023-07-06T03:45:25.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa269b9a700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2640cc4c0 0x7fa2640af590 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa26000c780 tx=0x7fa26000cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa25affd700 1 -- 172.21.15.97:0/4270657388 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260004c80 con 0x7fa2640cc4c0 2023-07-06T03:45:25.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa25affd700 1 -- 172.21.15.97:0/4270657388 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa260004e00 con 0x7fa2640cc4c0 2023-07-06T03:45:25.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.658+0000 7fa25affd700 1 -- 172.21.15.97:0/4270657388 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260007310 con 0x7fa2640cc4c0 2023-07-06T03:45:25.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.662+0000 7fa26c5ff700 1 -- 172.21.15.97:0/4270657388 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2640b09a0 con 0x7fa2640cc4c0 2023-07-06T03:45:25.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.662+0000 7fa26c5ff700 1 -- 172.21.15.97:0/4270657388 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa264203b40 con 0x7fa2640cc4c0 2023-07-06T03:45:25.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.662+0000 7f126fdff700 1 -- 172.21.15.97:0/1784493747 shutdown_connections 2023-07-06T03:45:25.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.662+0000 7f126fdff700 1 --2- 172.21.15.97:0/1784493747 >> v2:172.21.15.97:3300/0 conn(0x7f12680d1ba0 0x7f12680d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.662+0000 7f126fdff700 1 --2- 172.21.15.97:0/1784493747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 0x7f12680cd850 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.662+0000 7f126fdff700 1 -- 172.21.15.97:0/1784493747 >> 172.21.15.97:0/1784493747 conn(0x7f126806b2d0 msgr2=0x7f12680cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:25.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.663+0000 7fa26c5ff700 1 -- 172.21.15.97:0/4270657388 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2640b0ec0 con 0x7fa2640cc4c0 2023-07-06T03:45:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.666+0000 7fbc927fc700 1 -- 172.21.15.97:0/167165919 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbc9c0af550 con 0x7fbc9c160a20 2023-07-06T03:45:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.666+0000 7fa8b2ffd700 1 -- 172.21.15.97:0/30957421 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac003e60 con 0x7fa8bc16d610 2023-07-06T03:45:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.666+0000 7fa8b2ffd700 1 -- 172.21.15.97:0/30957421 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac011630 con 0x7fa8bc16d610 2023-07-06T03:45:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.666+0000 7fa8b2ffd700 1 -- 172.21.15.97:0/30957421 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0119c0 con 0x7fa8bc16d610 2023-07-06T03:45:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.666+0000 7fa8c45ff700 1 -- 172.21.15.97:0/30957421 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8bc0c7840 con 0x7fa8bc16d610 2023-07-06T03:45:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.667+0000 7fa8c45ff700 1 -- 172.21.15.97:0/30957421 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8bc0c8580 con 0x7fa8bc16d610 2023-07-06T03:45:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.668+0000 7f126fdff700 1 -- 172.21.15.97:0/1784493747 shutdown_connections 2023-07-06T03:45:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.668+0000 7fa25affd700 1 -- 172.21.15.97:0/4270657388 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa260007470 con 0x7fa2640cc4c0 2023-07-06T03:45:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.668+0000 7fa8c45ff700 1 -- 172.21.15.97:0/30957421 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8bc0c7d60 con 0x7fa8bc16d610 2023-07-06T03:45:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.668+0000 7f126fdff700 1 -- 172.21.15.97:0/1784493747 wait complete. 2023-07-06T03:45:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.668+0000 7f2c37fff700 1 -- 172.21.15.97:0/3245389792 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40011830 con 0x7f2c50160770 2023-07-06T03:45:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.669+0000 7f2c37fff700 1 -- 172.21.15.97:0/3245389792 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c40011990 con 0x7f2c50160770 2023-07-06T03:45:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.669+0000 7f2c37fff700 1 -- 172.21.15.97:0/3245389792 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40011d20 con 0x7f2c50160770 2023-07-06T03:45:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.664+0000 7f9f60ff9700 1 -- 172.21.15.97:0/1379382419 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f5c00ad20 con 0x7f9f640c5150 2023-07-06T03:45:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.666+0000 7f9f60ff9700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f5006ad30 0x7f9f5006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.666+0000 7f9f60ff9700 1 -- 172.21.15.97:0/1379382419 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9f5c089f90 con 0x7f9f640c5150 2023-07-06T03:45:25.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.666+0000 7f9f63fff700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f5006ad30 0x7f9f5006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.668+0000 7f9f63fff700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f5006ad30 0x7f9f5006d1e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009ad0 tx=0x7f9f58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.670+0000 7fa25affd700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa24806ace0 0x7fa24806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.670+0000 7fa25affd700 1 -- 172.21.15.97:0/4270657388 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa260089760 con 0x7fa2640cc4c0 2023-07-06T03:45:25.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.671+0000 7f126fdff700 1 Processor -- start 2023-07-06T03:45:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.669+0000 7f2c559ff700 1 -- 172.21.15.97:0/3245389792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c50161020 con 0x7f2c50160770 2023-07-06T03:45:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.669+0000 7f2c559ff700 1 -- 172.21.15.97:0/3245389792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c501615e0 con 0x7f2c50160770 2023-07-06T03:45:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.671+0000 7f9f60ff9700 1 -- 172.21.15.97:0/1379382419 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9f5c055020 con 0x7f9f640c5150 2023-07-06T03:45:25.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.671+0000 7fa26a39b700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa24806ace0 0x7fa24806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.671+0000 7f126fdff700 1 -- start start 2023-07-06T03:45:25.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.671+0000 7f2c559ff700 1 -- 172.21.15.97:0/3245389792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5004e340 con 0x7f2c50160770 2023-07-06T03:45:25.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.671+0000 7fa8b2ffd700 1 -- 172.21.15.97:0/30957421 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa8ac021070 con 0x7fa8bc16d610 2023-07-06T03:45:25.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.673+0000 7fa26a39b700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa24806ace0 0x7fa24806d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa264073590 tx=0x7fa25c010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.673+0000 7fa8b2ffd700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a006adb0 0x7fa8a006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.673+0000 7fa8b2ffd700 1 -- 172.21.15.97:0/30957421 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8ac08ab50 con 0x7fa8bc16d610 2023-07-06T03:45:25.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.673+0000 7f126fdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12680b8890 0x7f126815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.673+0000 7f126fdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 0x7f12680caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.673+0000 7f126fdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12680d1ba0 0x7f12680caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.673+0000 7f126fdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12681558b0 con 0x7f12680d1ba0 2023-07-06T03:45:25.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.673+0000 7f126fdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12680b4fc0 con 0x7f12680b8890 2023-07-06T03:45:25.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.673+0000 7f126fdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1268155420 con 0x7f12680cd400 2023-07-06T03:45:25.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.674+0000 7f126d39a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 0x7f12680caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.674+0000 7f126d39a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 0x7f12680caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54258/0 (socket says 172.21.15.97:54258) 2023-07-06T03:45:25.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.674+0000 7f126d39a700 1 -- 172.21.15.97:0/2316023745 learned_addr learned my addr 172.21.15.97:0/2316023745 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:25.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.674+0000 7f126db9b700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12680b8890 0x7f126815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.674+0000 7f126e39c700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12680d1ba0 0x7f12680caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.674+0000 7f126d39a700 1 -- 172.21.15.97:0/2316023745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12680b8890 msgr2=0x7f126815a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.674+0000 7f126d39a700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12680b8890 0x7f126815a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.674+0000 7f126d39a700 1 -- 172.21.15.97:0/2316023745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12680d1ba0 msgr2=0x7f12680caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:25.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.674+0000 7f126d39a700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12680d1ba0 0x7f12680caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:25.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.674+0000 7f126d39a700 1 -- 172.21.15.97:0/2316023745 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12540094a0 con 0x7f12680cd400 2023-07-06T03:45:25.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.674+0000 7fa8c239b700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a006adb0 0x7fa8a006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.675+0000 7f126d39a700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 0x7f12680caa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f12540090b0 tx=0x7f1254004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.675+0000 7f12667fc700 1 -- 172.21.15.97:0/2316023745 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12540036a0 con 0x7f12680cd400 2023-07-06T03:45:25.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.675+0000 7f12667fc700 1 -- 172.21.15.97:0/2316023745 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125401e070 con 0x7f12680cd400 2023-07-06T03:45:25.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.675+0000 7f12667fc700 1 -- 172.21.15.97:0/2316023745 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254003a40 con 0x7f12680cd400 2023-07-06T03:45:25.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.675+0000 7f126fdff700 1 -- 172.21.15.97:0/2316023745 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1268154530 con 0x7f12680cd400 2023-07-06T03:45:25.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.675+0000 7fa25affd700 1 -- 172.21.15.97:0/4270657388 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa260054880 con 0x7fa2640cc4c0 2023-07-06T03:45:25.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.676+0000 7fa8c239b700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a006adb0 0x7fa8a006d260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800a510 tx=0x7fa8b800d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.676+0000 7f126fdff700 1 -- 172.21.15.97:0/2316023745 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1268154af0 con 0x7f12680cd400 2023-07-06T03:45:25.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.677+0000 7fa8b2ffd700 1 -- 172.21.15.97:0/30957421 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa8ac055d90 con 0x7fa8bc16d610 2023-07-06T03:45:25.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.681+0000 7f2c37fff700 1 -- 172.21.15.97:0/3245389792 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2c4001a400 con 0x7f2c50160770 2023-07-06T03:45:25.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.679+0000 7f12667fc700 1 -- 172.21.15.97:0/2316023745 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1254003ba0 con 0x7f12680cd400 2023-07-06T03:45:25.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.681+0000 7f12667fc700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f124c06adb0 0x7f124c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.681+0000 7f12667fc700 1 -- 172.21.15.97:0/2316023745 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1254089e50 con 0x7f12680cd400 2023-07-06T03:45:25.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.682+0000 7f126fdff700 1 -- 172.21.15.97:0/2316023745 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126804e340 con 0x7f12680cd400 2023-07-06T03:45:25.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.682+0000 7f126db9b700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f124c06adb0 0x7f124c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.682+0000 7f2c37fff700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c3c06adb0 0x7f2c3c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:25.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.683+0000 7f2c37fff700 1 -- 172.21.15.97:0/3245389792 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2c400898c0 con 0x7f2c50160770 2023-07-06T03:45:25.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.683+0000 7f126db9b700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f124c06adb0 0x7f124c06d260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f125c0015f0 tx=0x7f125c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.684+0000 7f2c4effd700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c3c06adb0 0x7f2c3c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:25.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.684+0000 7f2c37fff700 1 -- 172.21.15.97:0/3245389792 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2c400583e0 con 0x7f2c50160770 2023-07-06T03:45:25.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.685+0000 7f2c4effd700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c3c06adb0 0x7f2c3c06d260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800b370 tx=0x7f2c48009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:25.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:25.687+0000 7f12667fc700 1 -- 172.21.15.97:0/2316023745 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1254020080 con 0x7f12680cd400 2023-07-06T03:45:26.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.005+0000 7fb6b2bff700 1 -- 172.21.15.97:0/1939942374 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb68c005160 con 0x7fb6ac160a20 2023-07-06T03:45:26.011 INFO:teuthology.orchestra.run.smithi097.stdout:55834574857 2023-07-06T03:45:26.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.006+0000 7fb6a97fa700 1 -- 172.21.15.97:0/1939942374 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6a4019020 con 0x7fb6ac160a20 2023-07-06T03:45:26.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.010+0000 7fb6b2bff700 1 -- 172.21.15.97:0/1939942374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69806ad30 msgr2=0x7fb69806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.010+0000 7fb6b2bff700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69806ad30 0x7fb69806d1e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0006b20 tx=0x7fb6a0006ab0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.010+0000 7fb6b2bff700 1 -- 172.21.15.97:0/1939942374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6ac160a20 msgr2=0x7fb6ac0c62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.010+0000 7fb6b2bff700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6ac160a20 0x7fb6ac0c62d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400b990 tx=0x7fb6a400bca0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.010+0000 7fb6b2bff700 1 -- 172.21.15.97:0/1939942374 shutdown_connections 2023-07-06T03:45:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.010+0000 7fb6b2bff700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69806ad30 0x7fb69806d1e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.010+0000 7fb6b2bff700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6ac160a20 0x7fb6ac0c62d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.010+0000 7fb6b2bff700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6ac0cca00 0x7fb6ac163400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.010+0000 7fb6b2bff700 1 --2- 172.21.15.97:0/1939942374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6ac0ce3c0 0x7fb6ac0af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.010+0000 7fb6b2bff700 1 -- 172.21.15.97:0/1939942374 >> 172.21.15.97:0/1939942374 conn(0x7fb6ac0cf610 msgr2=0x7fb6ac070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.011+0000 7fb6b2bff700 1 -- 172.21.15.97:0/1939942374 shutdown_connections 2023-07-06T03:45:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.011+0000 7fb6b2bff700 1 -- 172.21.15.97:0/1939942374 wait complete. 2023-07-06T03:45:26.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-07-06T03:45:26.043 DEBUG:teuthology.parallel:result is None 2023-07-06T03:45:26.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.080+0000 7f2c559ff700 1 -- 172.21.15.97:0/3245389792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2c500cd870 con 0x7f2c50160770 2023-07-06T03:45:26.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.081+0000 7f2c37fff700 1 -- 172.21.15.97:0/3245389792 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2c4000be60 con 0x7f2c50160770 2023-07-06T03:45:26.083 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:26.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.083+0000 7f2c559ff700 1 -- 172.21.15.97:0/3245389792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c3c06adb0 msgr2=0x7f2c3c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.083+0000 7f2c559ff700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c3c06adb0 0x7f2c3c06d260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800b370 tx=0x7f2c48009000 comp rx=0 tx=0).stop 2023-07-06T03:45:26.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.083+0000 7f2c559ff700 1 -- 172.21.15.97:0/3245389792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c50160770 msgr2=0x7f2c501620c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.083+0000 7f2c559ff700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c50160770 0x7f2c501620c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000b440 tx=0x7f2c4000b750 comp rx=0 tx=0).stop 2023-07-06T03:45:26.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.083+0000 7f2c559ff700 1 -- 172.21.15.97:0/3245389792 shutdown_connections 2023-07-06T03:45:26.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.083+0000 7f2c559ff700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c3c06adb0 0x7f2c3c06d260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.083+0000 7f2c559ff700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c50160770 0x7f2c501620c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.083+0000 7f2c559ff700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c500adb80 0x7f2c50207470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.083+0000 7f2c559ff700 1 --2- 172.21.15.97:0/3245389792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c500c4a60 0x7f2c501635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.083+0000 7f2c559ff700 1 -- 172.21.15.97:0/3245389792 >> 172.21.15.97:0/3245389792 conn(0x7f2c5006a060 msgr2=0x7f2c500b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:26.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.084+0000 7f2c559ff700 1 -- 172.21.15.97:0/3245389792 shutdown_connections 2023-07-06T03:45:26.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.084+0000 7f2c559ff700 1 -- 172.21.15.97:0/3245389792 wait complete. 2023-07-06T03:45:26.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.091+0000 7f9059fff700 1 -- 172.21.15.97:0/993996622 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f905404e340 con 0x7f90540ce3c0 2023-07-06T03:45:26.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.092+0000 7f903bfff700 1 -- 172.21.15.97:0/993996622 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9048015020 con 0x7f90540ce3c0 2023-07-06T03:45:26.094 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:26.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 -- 172.21.15.97:0/993996622 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f904006adb0 msgr2=0x7f904006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f904006adb0 0x7f904006d260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f904c00b370 tx=0x7f904c009000 comp rx=0 tx=0).stop 2023-07-06T03:45:26.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 -- 172.21.15.97:0/993996622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90540ce3c0 msgr2=0x7f90540b0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90540ce3c0 0x7f90540b0370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f904800b450 tx=0x7f904800b810 comp rx=0 tx=0).stop 2023-07-06T03:45:26.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 -- 172.21.15.97:0/993996622 shutdown_connections 2023-07-06T03:45:26.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f904006adb0 0x7f904006d260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90540ce3c0 0x7f90540b0370 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90540ccbd0 0x7f90540b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 --2- 172.21.15.97:0/993996622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90540cd060 0x7f9054088c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 -- 172.21.15.97:0/993996622 >> 172.21.15.97:0/993996622 conn(0x7f90540cf610 msgr2=0x7f905408a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:26.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 -- 172.21.15.97:0/993996622 shutdown_connections 2023-07-06T03:45:26.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.096+0000 7f9059fff700 1 -- 172.21.15.97:0/993996622 wait complete. 2023-07-06T03:45:26.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2023-07-06T03:45:26.106 DEBUG:teuthology.parallel:result is None 2023-07-06T03:45:26.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.120+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/1379382419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9f44005870 con 0x7f9f640c5150 2023-07-06T03:45:26.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.120+0000 7f9f60ff9700 1 -- 172.21.15.97:0/1379382419 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f5c001c60 con 0x7f9f640c5150 2023-07-06T03:45:26.122 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:26.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.123+0000 7fa26c5ff700 1 -- 172.21.15.97:0/4270657388 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa2640b0ec0 con 0x7fa2640cc4c0 2023-07-06T03:45:26.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-07-06T03:45:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/1379382419 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f5006ad30 msgr2=0x7f9f5006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f5006ad30 0x7f9f5006d1e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009ad0 tx=0x7f9f58008040 comp rx=0 tx=0).stop 2023-07-06T03:45:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/1379382419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f640c5150 msgr2=0x7f9f641fa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f640c5150 0x7f9f641fa140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00cba0 tx=0x7f9f5c00cf60 comp rx=0 tx=0).stop 2023-07-06T03:45:26.127 DEBUG:teuthology.parallel:result is None 2023-07-06T03:45:26.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/1379382419 shutdown_connections 2023-07-06T03:45:26.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f5006ad30 0x7f9f5006d1e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f640c9650 0x7f9f641fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f640add10 0x7f9f641506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 --2- 172.21.15.97:0/1379382419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f640c5150 0x7f9f641fa140 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/1379382419 >> 172.21.15.97:0/1379382419 conn(0x7f9f640cf610 msgr2=0x7f9f640ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/1379382419 shutdown_connections 2023-07-06T03:45:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7f9f6a7ff700 1 -- 172.21.15.97:0/1379382419 wait complete. 2023-07-06T03:45:26.132 INFO:teuthology.orchestra.run.smithi097.stdout:55834574857 2023-07-06T03:45:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.124+0000 7fa25affd700 1 -- 172.21.15.97:0/4270657388 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa2640b0ec0 con 0x7fa2640cc4c0 2023-07-06T03:45:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.127+0000 7fa26c5ff700 1 -- 172.21.15.97:0/4270657388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa24806ace0 msgr2=0x7fa24806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.127+0000 7fa26c5ff700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa24806ace0 0x7fa24806d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa264073590 tx=0x7fa25c010000 comp rx=0 tx=0).stop 2023-07-06T03:45:26.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.127+0000 7fa26c5ff700 1 -- 172.21.15.97:0/4270657388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2640cc4c0 msgr2=0x7fa2640af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.127+0000 7fa26c5ff700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2640cc4c0 0x7fa2640af590 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa26000c780 tx=0x7fa26000cb40 comp rx=0 tx=0).stop 2023-07-06T03:45:26.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.128+0000 7fa26c5ff700 1 -- 172.21.15.97:0/4270657388 shutdown_connections 2023-07-06T03:45:26.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.128+0000 7fa26c5ff700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa24806ace0 0x7fa24806d190 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.128+0000 7fa26c5ff700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2640ce3c0 0x7fa2640afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.128+0000 7fa26c5ff700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26408aa00 0x7fa2640a8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.128+0000 7fa26c5ff700 1 --2- 172.21.15.97:0/4270657388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2640cc4c0 0x7fa2640af590 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.128+0000 7fa26c5ff700 1 -- 172.21.15.97:0/4270657388 >> 172.21.15.97:0/4270657388 conn(0x7fa2640cf610 msgr2=0x7fa26406c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:26.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.128+0000 7fa26c5ff700 1 -- 172.21.15.97:0/4270657388 shutdown_connections 2023-07-06T03:45:26.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.128+0000 7fa26c5ff700 1 -- 172.21.15.97:0/4270657388 wait complete. 2023-07-06T03:45:26.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.128+0000 7fbca3bff700 1 -- 172.21.15.97:0/167165919 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbc9c0af550 con 0x7fbc9c160a20 2023-07-06T03:45:26.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.129+0000 7fbc927fc700 1 -- 172.21.15.97:0/167165919 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbc9c0af550 con 0x7fbc9c160a20 2023-07-06T03:45:26.138 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-07-06T03:45:26.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.132+0000 7fbca3bff700 1 -- 172.21.15.97:0/167165919 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc8006adb0 msgr2=0x7fbc8006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.132+0000 7fbca3bff700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc8006adb0 0x7fbc8006d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc98014040 tx=0x7fbc98008030 comp rx=0 tx=0).stop 2023-07-06T03:45:26.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.132+0000 7fbca3bff700 1 -- 172.21.15.97:0/167165919 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc9c160a20 msgr2=0x7fbc9c0b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.132+0000 7fbca3bff700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc9c160a20 0x7fbc9c0b3aa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400c7d0 tx=0x7fbc9400cae0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.133+0000 7fbca3bff700 1 -- 172.21.15.97:0/167165919 shutdown_connections 2023-07-06T03:45:26.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.133+0000 7fbca3bff700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc8006adb0 0x7fbc8006d260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.133+0000 7fbca3bff700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc9c160a20 0x7fbc9c0b3aa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.133+0000 7fbca3bff700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc9c0cca00 0x7fbc9c0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.133+0000 7fbca3bff700 1 --2- 172.21.15.97:0/167165919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9c0ce3c0 0x7fbc9c0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.133+0000 7fbca3bff700 1 -- 172.21.15.97:0/167165919 >> 172.21.15.97:0/167165919 conn(0x7fbc9c0cf610 msgr2=0x7fbc9c15fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.133+0000 7fbca3bff700 1 -- 172.21.15.97:0/167165919 shutdown_connections 2023-07-06T03:45:26.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.133+0000 7fbca3bff700 1 -- 172.21.15.97:0/167165919 wait complete. 2023-07-06T03:45:26.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.134+0000 7fa8c45ff700 1 -- 172.21.15.97:0/30957421 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa8bc0cd480 con 0x7fa8bc16d610 2023-07-06T03:45:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.135+0000 7fa8b2ffd700 1 -- 172.21.15.97:0/30957421 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa8bc0cd480 con 0x7fa8bc16d610 2023-07-06T03:45:26.144 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.135+0000 7f126fdff700 1 -- 172.21.15.97:0/2316023745 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1268155560 con 0x7f12680cd400 2023-07-06T03:45:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.136+0000 7f12667fc700 1 -- 172.21.15.97:0/2316023745 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1254054f70 con 0x7f12680cd400 2023-07-06T03:45:26.145 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-07-06T03:45:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.138+0000 7fa8c45ff700 1 -- 172.21.15.97:0/30957421 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a006adb0 msgr2=0x7fa8a006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.138+0000 7fa8c45ff700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a006adb0 0x7fa8a006d260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800a510 tx=0x7fa8b800d040 comp rx=0 tx=0).stop 2023-07-06T03:45:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.138+0000 7fa8c45ff700 1 -- 172.21.15.97:0/30957421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8bc16d610 msgr2=0x7fa8bc0c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.138+0000 7fa8c45ff700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8bc16d610 0x7fa8bc0c8040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0cd040 tx=0x7fa8ac004750 comp rx=0 tx=0).stop 2023-07-06T03:45:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.138+0000 7fa8c45ff700 1 -- 172.21.15.97:0/30957421 shutdown_connections 2023-07-06T03:45:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.138+0000 7fa8c45ff700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a006adb0 0x7fa8a006d260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.138+0000 7fa8c45ff700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8bc16d9e0 0x7fa8bc0b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.138+0000 7fa8c45ff700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8bc0cc4c0 0x7fa8bc0b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.138+0000 7fa8c45ff700 1 --2- 172.21.15.97:0/30957421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8bc16d610 0x7fa8bc0c8040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.138+0000 7fa8c45ff700 1 -- 172.21.15.97:0/30957421 >> 172.21.15.97:0/30957421 conn(0x7fa8bc0cf610 msgr2=0x7fa8bc0c43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.139+0000 7fa8c45ff700 1 -- 172.21.15.97:0/30957421 shutdown_connections 2023-07-06T03:45:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.139+0000 7fa8c45ff700 1 -- 172.21.15.97:0/30957421 wait complete. 2023-07-06T03:45:26.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.139+0000 7f126fdff700 1 -- 172.21.15.97:0/2316023745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f124c06adb0 msgr2=0x7f124c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.139+0000 7f126fdff700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f124c06adb0 0x7f124c06d260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f125c0015f0 tx=0x7f125c009040 comp rx=0 tx=0).stop 2023-07-06T03:45:26.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.140+0000 7f126fdff700 1 -- 172.21.15.97:0/2316023745 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 msgr2=0x7f12680caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.140+0000 7f126fdff700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 0x7f12680caa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f12540090b0 tx=0x7f1254004750 comp rx=0 tx=0).stop 2023-07-06T03:45:26.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.140+0000 7f126fdff700 1 -- 172.21.15.97:0/2316023745 shutdown_connections 2023-07-06T03:45:26.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.140+0000 7f126fdff700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f124c06adb0 0x7f124c06d260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.140+0000 7f126fdff700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12680d1ba0 0x7f12680caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.140+0000 7f126fdff700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12680b8890 0x7f126815a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.140+0000 7f126fdff700 1 --2- 172.21.15.97:0/2316023745 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f12680cd400 0x7f12680caa00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.140+0000 7f126fdff700 1 -- 172.21.15.97:0/2316023745 >> 172.21.15.97:0/2316023745 conn(0x7f126806b2d0 msgr2=0x7f126815be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.140+0000 7f126fdff700 1 -- 172.21.15.97:0/2316023745 shutdown_connections 2023-07-06T03:45:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.140+0000 7f126fdff700 1 -- 172.21.15.97:0/2316023745 wait complete. 2023-07-06T03:45:26.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-07-06T03:45:26.150 DEBUG:teuthology.parallel:result is None 2023-07-06T03:45:26.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-07-06T03:45:26.159 DEBUG:teuthology.parallel:result is None 2023-07-06T03:45:26.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-07-06T03:45:26.161 DEBUG:teuthology.parallel:result is None 2023-07-06T03:45:26.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-07-06T03:45:26.165 DEBUG:teuthology.parallel:result is None 2023-07-06T03:45:26.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2023-07-06T03:45:26.168 DEBUG:teuthology.parallel:result is None 2023-07-06T03:45:26.168 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-06T03:45:26.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:45:26.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.401+0000 7f1a755ff700 1 Processor -- start 2023-07-06T03:45:26.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.402+0000 7f1a755ff700 1 -- start start 2023-07-06T03:45:26.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a755ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a7015c520 0x7f1a700cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a755ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a70160180 0x7f1a701605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a755ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1a70160b30 0x7f1a700cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a755ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a700cd8d0 con 0x7f1a70160180 2023-07-06T03:45:26.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a755ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a700cdae0 con 0x7f1a70160b30 2023-07-06T03:45:26.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a755ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700b6640 con 0x7f1a7015c520 2023-07-06T03:45:26.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a6e7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a70160180 0x7f1a701605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a6e7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a70160180 0x7f1a701605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51754/0 (socket says 172.21.15.97:51754) 2023-07-06T03:45:26.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a6e7fc700 1 -- 172.21.15.97:0/1155931915 learned_addr learned my addr 172.21.15.97:0/1155931915 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:26.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a6f7fe700 1 --2- 172.21.15.97:0/1155931915 >> v2:172.21.15.146:3300/0 conn(0x7f1a70160b30 0x7f1a700cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a6effd700 1 --2- 172.21.15.97:0/1155931915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a7015c520 0x7f1a700cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a6e7fc700 1 -- 172.21.15.97:0/1155931915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a7015c520 msgr2=0x7f1a700cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:26.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a6e7fc700 1 --2- 172.21.15.97:0/1155931915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a7015c520 0x7f1a700cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a6e7fc700 1 -- 172.21.15.97:0/1155931915 >> v2:172.21.15.146:3300/0 conn(0x7f1a70160b30 msgr2=0x7f1a700cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:26.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a6e7fc700 1 --2- 172.21.15.97:0/1155931915 >> v2:172.21.15.146:3300/0 conn(0x7f1a70160b30 0x7f1a700cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a6e7fc700 1 -- 172.21.15.97:0/1155931915 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a700b6920 con 0x7f1a70160180 2023-07-06T03:45:26.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.403+0000 7f1a6f7fe700 1 --2- 172.21.15.97:0/1155931915 >> v2:172.21.15.146:3300/0 conn(0x7f1a70160b30 0x7f1a700cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:26.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a6e7fc700 1 --2- 172.21.15.97:0/1155931915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70160180 0x7f1a701605f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a7008bfc0 tx=0x7f1a64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2749dd6d27a9741b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:26.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a6dffb700 1 -- 172.21.15.97:0/1155931915 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400b870 con 0x7f1a70160180 2023-07-06T03:45:26.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a6dffb700 1 -- 172.21.15.97:0/1155931915 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6400b9d0 con 0x7f1a70160180 2023-07-06T03:45:26.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a6dffb700 1 -- 172.21.15.97:0/1155931915 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400bd60 con 0x7f1a70160180 2023-07-06T03:45:26.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a755ff700 1 -- 172.21.15.97:0/1155931915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70160180 msgr2=0x7f1a701605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a755ff700 1 --2- 172.21.15.97:0/1155931915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70160180 0x7f1a701605f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a7008bfc0 tx=0x7f1a64009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a755ff700 1 -- 172.21.15.97:0/1155931915 shutdown_connections 2023-07-06T03:45:26.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a755ff700 1 --2- 172.21.15.97:0/1155931915 >> v2:172.21.15.146:3300/0 conn(0x7f1a70160b30 0x7f1a700cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a755ff700 1 --2- 172.21.15.97:0/1155931915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a7015c520 0x7f1a700cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a755ff700 1 --2- 172.21.15.97:0/1155931915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70160180 0x7f1a701605f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a755ff700 1 -- 172.21.15.97:0/1155931915 >> 172.21.15.97:0/1155931915 conn(0x7f1a70071790 msgr2=0x7f1a70071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:26.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.404+0000 7f1a755ff700 1 -- 172.21.15.97:0/1155931915 shutdown_connections 2023-07-06T03:45:26.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.405+0000 7f1a755ff700 1 -- 172.21.15.97:0/1155931915 wait complete. 2023-07-06T03:45:26.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.406+0000 7f1a755ff700 1 Processor -- start 2023-07-06T03:45:26.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.406+0000 7f1a755ff700 1 -- start start 2023-07-06T03:45:26.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.407+0000 7f1a755ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a7015c520 0x7f1a70150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.407+0000 7f1a755ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70160180 0x7f1a7015e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.407+0000 7f1a755ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a70160b30 0x7f1a7015b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.407+0000 7f1a755ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7015eb80 con 0x7f1a70160180 2023-07-06T03:45:26.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.407+0000 7f1a755ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7015ee60 con 0x7f1a70160b30 2023-07-06T03:45:26.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.407+0000 7f1a755ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7015f140 con 0x7f1a7015c520 2023-07-06T03:45:26.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a6e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70160180 0x7f1a7015e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a6e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70160180 0x7f1a7015e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51762/0 (socket says 172.21.15.97:51762) 2023-07-06T03:45:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a6e7fc700 1 -- 172.21.15.97:0/1987654898 learned_addr learned my addr 172.21.15.97:0/1987654898 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a6effd700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a7015c520 0x7f1a70150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a6f7fe700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a70160b30 0x7f1a7015b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a6e7fc700 1 -- 172.21.15.97:0/1987654898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a7015c520 msgr2=0x7f1a70150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a6e7fc700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a7015c520 0x7f1a70150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a6e7fc700 1 -- 172.21.15.97:0/1987654898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a70160b30 msgr2=0x7f1a7015b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a6e7fc700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a70160b30 0x7f1a7015b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a6e7fc700 1 -- 172.21.15.97:0/1987654898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a64009550 con 0x7f1a70160180 2023-07-06T03:45:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a6e7fc700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70160180 0x7f1a7015e350 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1a64000c00 tx=0x7f1a64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a577fe700 1 -- 172.21.15.97:0/1987654898 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64004480 con 0x7f1a70160180 2023-07-06T03:45:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a577fe700 1 -- 172.21.15.97:0/1987654898 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a640045e0 con 0x7f1a70160180 2023-07-06T03:45:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a755ff700 1 -- 172.21.15.97:0/1987654898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a700cac40 con 0x7f1a70160180 2023-07-06T03:45:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.408+0000 7f1a577fe700 1 -- 172.21.15.97:0/1987654898 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a640038e0 con 0x7f1a70160180 2023-07-06T03:45:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.409+0000 7f1a755ff700 1 -- 172.21.15.97:0/1987654898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7015e890 con 0x7f1a70160180 2023-07-06T03:45:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.410+0000 7f1a755ff700 1 -- 172.21.15.97:0/1987654898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a700cb160 con 0x7f1a70160180 2023-07-06T03:45:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.411+0000 7f1a577fe700 1 -- 172.21.15.97:0/1987654898 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a64021070 con 0x7f1a70160180 2023-07-06T03:45:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.411+0000 7f1a577fe700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5c06ad10 0x7f1a5c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.412+0000 7f1a577fe700 1 -- 172.21.15.97:0/1987654898 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a6408aac0 con 0x7f1a70160180 2023-07-06T03:45:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.412+0000 7f1a6effd700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5c06ad10 0x7f1a5c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.412+0000 7f1a6effd700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5c06ad10 0x7f1a5c06d1c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009950 tx=0x7f1a60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:26.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.416+0000 7f1a577fe700 1 -- 172.21.15.97:0/1987654898 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1a700cb160 con 0x7f1a70160180 2023-07-06T03:45:26.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.608+0000 7f1a755ff700 1 -- 172.21.15.97:0/1987654898 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a700b6ee0 con 0x7f1a5c06ad10 2023-07-06T03:45:26.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.616+0000 7f1a577fe700 1 -- 172.21.15.97:0/1987654898 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286650 (secure 0 0 0) 0x7f1a60009e70 con 0x7f1a5c06ad10 2023-07-06T03:45:26.621 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:26.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 -- 172.21.15.97:0/1987654898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5c06ad10 msgr2=0x7f1a5c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5c06ad10 0x7f1a5c06d1c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009950 tx=0x7f1a60008040 comp rx=0 tx=0).stop 2023-07-06T03:45:26.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 -- 172.21.15.97:0/1987654898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70160180 msgr2=0x7f1a7015e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70160180 0x7f1a7015e350 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1a64000c00 tx=0x7f1a64004750 comp rx=0 tx=0).stop 2023-07-06T03:45:26.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 -- 172.21.15.97:0/1987654898 shutdown_connections 2023-07-06T03:45:26.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5c06ad10 0x7f1a5c06d1c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a70160b30 0x7f1a7015b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a7015c520 0x7f1a70150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 --2- 172.21.15.97:0/1987654898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70160180 0x7f1a7015e350 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 -- 172.21.15.97:0/1987654898 >> 172.21.15.97:0/1987654898 conn(0x7f1a70071790 msgr2=0x7f1a70071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:26.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 -- 172.21.15.97:0/1987654898 shutdown_connections 2023-07-06T03:45:26.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.621+0000 7f1a755ff700 1 -- 172.21.15.97:0/1987654898 wait complete. 2023-07-06T03:45:26.625 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:45:26.641 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-07-06T03:45:24.885129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218724,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002643"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681463+0000","last_change":"2023-07-06T03:45:11.693177+0000","last_active":"2023-07-06T03:45:12.681463+0000","last_peered":"2023-07-06T03:45:12.681463+0000","last_clean":"2023-07-06T03:45:12.681463+0000","last_became_active":"2023-07-06T03:45:11.692851+0000","last_became_peered":"2023-07-06T03:45:11.692851+0000","last_unstale":"2023-07-06T03:45:12.681463+0000","last_undegraded":"2023-07-06T03:45:12.681463+0000","last_fullsized":"2023-07-06T03:45:12.681463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678856+0000","last_change":"2023-07-06T03:45:11.674359+0000","last_active":"2023-07-06T03:45:12.678856+0000","last_peered":"2023-07-06T03:45:12.678856+0000","last_clean":"2023-07-06T03:45:12.678856+0000","last_became_active":"2023-07-06T03:45:11.674035+0000","last_became_peered":"2023-07-06T03:45:11.674035+0000","last_unstale":"2023-07-06T03:45:12.678856+0000","last_undegraded":"2023-07-06T03:45:12.678856+0000","last_fullsized":"2023-07-06T03:45:12.678856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827906+0000","last_change":"2023-07-06T03:45:11.691806+0000","last_active":"2023-07-06T03:45:13.827906+0000","last_peered":"2023-07-06T03:45:13.827906+0000","last_clean":"2023-07-06T03:45:13.827906+0000","last_became_active":"2023-07-06T03:45:11.691532+0000","last_became_peered":"2023-07-06T03:45:11.691532+0000","last_unstale":"2023-07-06T03:45:13.827906+0000","last_undegraded":"2023-07-06T03:45:13.827906+0000","last_fullsized":"2023-07-06T03:45:13.827906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827578+0000","last_change":"2023-07-06T03:45:11.675300+0000","last_active":"2023-07-06T03:45:13.827578+0000","last_peered":"2023-07-06T03:45:13.827578+0000","last_clean":"2023-07-06T03:45:13.827578+0000","last_became_active":"2023-07-06T03:45:11.675018+0000","last_became_peered":"2023-07-06T03:45:11.675018+0000","last_unstale":"2023-07-06T03:45:13.827578+0000","last_undegraded":"2023-07-06T03:45:13.827578+0000","last_fullsized":"2023-07-06T03:45:13.827578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205854+0000","last_change":"2023-07-06T03:45:11.666099+0000","last_active":"2023-07-06T03:45:14.205854+0000","last_peered":"2023-07-06T03:45:14.205854+0000","last_clean":"2023-07-06T03:45:14.205854+0000","last_became_active":"2023-07-06T03:45:11.665759+0000","last_became_peered":"2023-07-06T03:45:11.665759+0000","last_unstale":"2023-07-06T03:45:14.205854+0000","last_undegraded":"2023-07-06T03:45:14.205854+0000","last_fullsized":"2023-07-06T03:45:14.205854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:52.666500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830215+0000","last_change":"2023-07-06T03:45:11.662251+0000","last_active":"2023-07-06T03:45:13.830215+0000","last_peered":"2023-07-06T03:45:13.830215+0000","last_clean":"2023-07-06T03:45:13.830215+0000","last_became_active":"2023-07-06T03:45:11.661916+0000","last_became_peered":"2023-07-06T03:45:11.661916+0000","last_unstale":"2023-07-06T03:45:13.830215+0000","last_undegraded":"2023-07-06T03:45:13.830215+0000","last_fullsized":"2023-07-06T03:45:13.830215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:11.287937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829687+0000","last_change":"2023-07-06T03:45:11.676465+0000","last_active":"2023-07-06T03:45:13.829687+0000","last_peered":"2023-07-06T03:45:13.829687+0000","last_clean":"2023-07-06T03:45:13.829687+0000","last_became_active":"2023-07-06T03:45:11.676180+0000","last_became_peered":"2023-07-06T03:45:11.676180+0000","last_unstale":"2023-07-06T03:45:13.829687+0000","last_undegraded":"2023-07-06T03:45:13.829687+0000","last_fullsized":"2023-07-06T03:45:13.829687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:38:24.542890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818440+0000","last_change":"2023-07-06T03:45:11.676387+0000","last_active":"2023-07-06T03:45:12.818440+0000","last_peered":"2023-07-06T03:45:12.818440+0000","last_clean":"2023-07-06T03:45:12.818440+0000","last_became_active":"2023-07-06T03:45:11.676129+0000","last_became_peered":"2023-07-06T03:45:11.676129+0000","last_unstale":"2023-07-06T03:45:12.818440+0000","last_undegraded":"2023-07-06T03:45:12.818440+0000","last_fullsized":"2023-07-06T03:45:12.818440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206076+0000","last_change":"2023-07-06T03:45:11.690977+0000","last_active":"2023-07-06T03:45:14.206076+0000","last_peered":"2023-07-06T03:45:14.206076+0000","last_clean":"2023-07-06T03:45:14.206076+0000","last_became_active":"2023-07-06T03:45:11.690423+0000","last_became_peered":"2023-07-06T03:45:11.690423+0000","last_unstale":"2023-07-06T03:45:14.206076+0000","last_undegraded":"2023-07-06T03:45:14.206076+0000","last_fullsized":"2023-07-06T03:45:14.206076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:14:59.597908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818347+0000","last_change":"2023-07-06T03:45:11.680924+0000","last_active":"2023-07-06T03:45:12.818347+0000","last_peered":"2023-07-06T03:45:12.818347+0000","last_clean":"2023-07-06T03:45:12.818347+0000","last_became_active":"2023-07-06T03:45:11.680607+0000","last_became_peered":"2023-07-06T03:45:11.680607+0000","last_unstale":"2023-07-06T03:45:12.818347+0000","last_undegraded":"2023-07-06T03:45:12.818347+0000","last_fullsized":"2023-07-06T03:45:12.818347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:13.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818149+0000","last_change":"2023-07-06T03:45:11.677946+0000","last_active":"2023-07-06T03:45:12.818149+0000","last_peered":"2023-07-06T03:45:12.818149+0000","last_clean":"2023-07-06T03:45:12.818149+0000","last_became_active":"2023-07-06T03:45:11.677684+0000","last_became_peered":"2023-07-06T03:45:11.677684+0000","last_unstale":"2023-07-06T03:45:12.818149+0000","last_undegraded":"2023-07-06T03:45:12.818149+0000","last_fullsized":"2023-07-06T03:45:12.818149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:27.384754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680941+0000","last_change":"2023-07-06T03:45:11.681178+0000","last_active":"2023-07-06T03:45:12.680941+0000","last_peered":"2023-07-06T03:45:12.680941+0000","last_clean":"2023-07-06T03:45:12.680941+0000","last_became_active":"2023-07-06T03:45:11.680841+0000","last_became_peered":"2023-07-06T03:45:11.680841+0000","last_unstale":"2023-07-06T03:45:12.680941+0000","last_undegraded":"2023-07-06T03:45:12.680941+0000","last_fullsized":"2023-07-06T03:45:12.680941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:58.087282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652673+0000","last_change":"2023-07-06T03:45:11.687461+0000","last_active":"2023-07-06T03:45:13.652673+0000","last_peered":"2023-07-06T03:45:13.652673+0000","last_clean":"2023-07-06T03:45:13.652673+0000","last_became_active":"2023-07-06T03:45:11.687182+0000","last_became_peered":"2023-07-06T03:45:11.687182+0000","last_unstale":"2023-07-06T03:45:13.652673+0000","last_undegraded":"2023-07-06T03:45:13.652673+0000","last_fullsized":"2023-07-06T03:45:13.652673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817899+0000","last_change":"2023-07-06T03:45:11.691496+0000","last_active":"2023-07-06T03:45:12.817899+0000","last_peered":"2023-07-06T03:45:12.817899+0000","last_clean":"2023-07-06T03:45:12.817899+0000","last_became_active":"2023-07-06T03:45:11.691206+0000","last_became_peered":"2023-07-06T03:45:11.691206+0000","last_unstale":"2023-07-06T03:45:12.817899+0000","last_undegraded":"2023-07-06T03:45:12.817899+0000","last_fullsized":"2023-07-06T03:45:12.817899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827487+0000","last_change":"2023-07-06T03:45:11.685001+0000","last_active":"2023-07-06T03:45:13.827487+0000","last_peered":"2023-07-06T03:45:13.827487+0000","last_clean":"2023-07-06T03:45:13.827487+0000","last_became_active":"2023-07-06T03:45:11.684695+0000","last_became_peered":"2023-07-06T03:45:11.684695+0000","last_unstale":"2023-07-06T03:45:13.827487+0000","last_undegraded":"2023-07-06T03:45:13.827487+0000","last_fullsized":"2023-07-06T03:45:13.827487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829742+0000","last_change":"2023-07-06T03:45:11.662096+0000","last_active":"2023-07-06T03:45:13.829742+0000","last_peered":"2023-07-06T03:45:13.829742+0000","last_clean":"2023-07-06T03:45:13.829742+0000","last_became_active":"2023-07-06T03:45:11.661834+0000","last_became_peered":"2023-07-06T03:45:11.661834+0000","last_unstale":"2023-07-06T03:45:13.829742+0000","last_undegraded":"2023-07-06T03:45:13.829742+0000","last_fullsized":"2023-07-06T03:45:13.829742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:36:49.534209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678504+0000","last_change":"2023-07-06T03:45:11.685162+0000","last_active":"2023-07-06T03:45:12.678504+0000","last_peered":"2023-07-06T03:45:12.678504+0000","last_clean":"2023-07-06T03:45:12.678504+0000","last_became_active":"2023-07-06T03:45:11.684977+0000","last_became_peered":"2023-07-06T03:45:11.684977+0000","last_unstale":"2023-07-06T03:45:12.678504+0000","last_undegraded":"2023-07-06T03:45:12.678504+0000","last_fullsized":"2023-07-06T03:45:12.678504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827827+0000","last_change":"2023-07-06T03:45:11.675611+0000","last_active":"2023-07-06T03:45:13.827827+0000","last_peered":"2023-07-06T03:45:13.827827+0000","last_clean":"2023-07-06T03:45:13.827827+0000","last_became_active":"2023-07-06T03:45:11.675348+0000","last_became_peered":"2023-07-06T03:45:11.675348+0000","last_unstale":"2023-07-06T03:45:13.827827+0000","last_undegraded":"2023-07-06T03:45:13.827827+0000","last_fullsized":"2023-07-06T03:45:13.827827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:00.096345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680922+0000","last_change":"2023-07-06T03:45:11.690182+0000","last_active":"2023-07-06T03:45:12.680922+0000","last_peered":"2023-07-06T03:45:12.680922+0000","last_clean":"2023-07-06T03:45:12.680922+0000","last_became_active":"2023-07-06T03:45:11.689744+0000","last_became_peered":"2023-07-06T03:45:11.689744+0000","last_unstale":"2023-07-06T03:45:12.680922+0000","last_undegraded":"2023-07-06T03:45:12.680922+0000","last_fullsized":"2023-07-06T03:45:12.680922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:32.898881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818137+0000","last_change":"2023-07-06T03:45:11.672749+0000","last_active":"2023-07-06T03:45:12.818137+0000","last_peered":"2023-07-06T03:45:12.818137+0000","last_clean":"2023-07-06T03:45:12.818137+0000","last_became_active":"2023-07-06T03:45:11.672401+0000","last_became_peered":"2023-07-06T03:45:11.672401+0000","last_unstale":"2023-07-06T03:45:12.818137+0000","last_undegraded":"2023-07-06T03:45:12.818137+0000","last_fullsized":"2023-07-06T03:45:12.818137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:15.627811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818032+0000","last_change":"2023-07-06T03:45:11.692060+0000","last_active":"2023-07-06T03:45:12.818032+0000","last_peered":"2023-07-06T03:45:12.818032+0000","last_clean":"2023-07-06T03:45:12.818032+0000","last_became_active":"2023-07-06T03:45:11.691699+0000","last_became_peered":"2023-07-06T03:45:11.691699+0000","last_unstale":"2023-07-06T03:45:12.818032+0000","last_undegraded":"2023-07-06T03:45:12.818032+0000","last_fullsized":"2023-07-06T03:45:12.818032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:32.898881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678195+0000","last_change":"2023-07-06T03:45:11.672926+0000","last_active":"2023-07-06T03:45:12.678195+0000","last_peered":"2023-07-06T03:45:12.678195+0000","last_clean":"2023-07-06T03:45:12.678195+0000","last_became_active":"2023-07-06T03:45:11.672627+0000","last_became_peered":"2023-07-06T03:45:11.672627+0000","last_unstale":"2023-07-06T03:45:12.678195+0000","last_undegraded":"2023-07-06T03:45:12.678195+0000","last_fullsized":"2023-07-06T03:45:12.678195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205814+0000","last_change":"2023-07-06T03:45:11.691373+0000","last_active":"2023-07-06T03:45:14.205814+0000","last_peered":"2023-07-06T03:45:14.205814+0000","last_clean":"2023-07-06T03:45:14.205814+0000","last_became_active":"2023-07-06T03:45:11.691046+0000","last_became_peered":"2023-07-06T03:45:11.691046+0000","last_unstale":"2023-07-06T03:45:14.205814+0000","last_undegraded":"2023-07-06T03:45:14.205814+0000","last_fullsized":"2023-07-06T03:45:14.205814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818372+0000","last_change":"2023-07-06T03:45:11.691643+0000","last_active":"2023-07-06T03:45:12.818372+0000","last_peered":"2023-07-06T03:45:12.818372+0000","last_clean":"2023-07-06T03:45:12.818372+0000","last_became_active":"2023-07-06T03:45:11.691382+0000","last_became_peered":"2023-07-06T03:45:11.691382+0000","last_unstale":"2023-07-06T03:45:12.818372+0000","last_undegraded":"2023-07-06T03:45:12.818372+0000","last_fullsized":"2023-07-06T03:45:12.818372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206243+0000","last_change":"2023-07-06T03:45:11.670205+0000","last_active":"2023-07-06T03:45:14.206243+0000","last_peered":"2023-07-06T03:45:14.206243+0000","last_clean":"2023-07-06T03:45:14.206243+0000","last_became_active":"2023-07-06T03:45:11.669935+0000","last_became_peered":"2023-07-06T03:45:11.669935+0000","last_unstale":"2023-07-06T03:45:14.206243+0000","last_undegraded":"2023-07-06T03:45:14.206243+0000","last_fullsized":"2023-07-06T03:45:14.206243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:11:07.089396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678592+0000","last_change":"2023-07-06T03:45:11.674601+0000","last_active":"2023-07-06T03:45:12.678592+0000","last_peered":"2023-07-06T03:45:12.678592+0000","last_clean":"2023-07-06T03:45:12.678592+0000","last_became_active":"2023-07-06T03:45:11.673533+0000","last_became_peered":"2023-07-06T03:45:11.673533+0000","last_unstale":"2023-07-06T03:45:12.678592+0000","last_undegraded":"2023-07-06T03:45:12.678592+0000","last_fullsized":"2023-07-06T03:45:12.678592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830285+0000","last_change":"2023-07-06T03:45:11.678777+0000","last_active":"2023-07-06T03:45:13.830285+0000","last_peered":"2023-07-06T03:45:13.830285+0000","last_clean":"2023-07-06T03:45:13.830285+0000","last_became_active":"2023-07-06T03:45:11.678437+0000","last_became_peered":"2023-07-06T03:45:11.678437+0000","last_unstale":"2023-07-06T03:45:13.830285+0000","last_undegraded":"2023-07-06T03:45:13.830285+0000","last_fullsized":"2023-07-06T03:45:13.830285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:51:53.330078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818105+0000","last_change":"2023-07-06T03:45:10.656464+0000","last_active":"2023-07-06T03:45:12.818105+0000","last_peered":"2023-07-06T03:45:12.818105+0000","last_clean":"2023-07-06T03:45:12.818105+0000","last_became_active":"2023-07-06T03:45:10.656116+0000","last_became_peered":"2023-07-06T03:45:10.656116+0000","last_unstale":"2023-07-06T03:45:12.818105+0000","last_undegraded":"2023-07-06T03:45:12.818105+0000","last_fullsized":"2023-07-06T03:45:12.818105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:57.080176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678331+0000","last_change":"2023-07-06T03:45:11.687243+0000","last_active":"2023-07-06T03:45:12.678331+0000","last_peered":"2023-07-06T03:45:12.678331+0000","last_clean":"2023-07-06T03:45:12.678331+0000","last_became_active":"2023-07-06T03:45:11.686947+0000","last_became_peered":"2023-07-06T03:45:11.686947+0000","last_unstale":"2023-07-06T03:45:12.678331+0000","last_undegraded":"2023-07-06T03:45:12.678331+0000","last_fullsized":"2023-07-06T03:45:12.678331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205529+0000","last_change":"2023-07-06T03:45:10.657736+0000","last_active":"2023-07-06T03:45:14.205529+0000","last_peered":"2023-07-06T03:45:14.205529+0000","last_clean":"2023-07-06T03:45:14.205529+0000","last_became_active":"2023-07-06T03:45:10.657450+0000","last_became_peered":"2023-07-06T03:45:10.657450+0000","last_unstale":"2023-07-06T03:45:14.205529+0000","last_undegraded":"2023-07-06T03:45:14.205529+0000","last_fullsized":"2023-07-06T03:45:14.205529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:14.620705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206125+0000","last_change":"2023-07-06T03:45:11.672564+0000","last_active":"2023-07-06T03:45:14.206125+0000","last_peered":"2023-07-06T03:45:14.206125+0000","last_clean":"2023-07-06T03:45:14.206125+0000","last_became_active":"2023-07-06T03:45:11.672150+0000","last_became_peered":"2023-07-06T03:45:11.672150+0000","last_unstale":"2023-07-06T03:45:14.206125+0000","last_undegraded":"2023-07-06T03:45:14.206125+0000","last_fullsized":"2023-07-06T03:45:14.206125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:59:01.536411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680441+0000","last_change":"2023-07-06T03:45:10.653666+0000","last_active":"2023-07-06T03:45:12.680441+0000","last_peered":"2023-07-06T03:45:12.680441+0000","last_clean":"2023-07-06T03:45:12.680441+0000","last_became_active":"2023-07-06T03:45:10.653357+0000","last_became_peered":"2023-07-06T03:45:10.653357+0000","last_unstale":"2023-07-06T03:45:12.680441+0000","last_undegraded":"2023-07-06T03:45:12.680441+0000","last_fullsized":"2023-07-06T03:45:12.680441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:25.838923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830215+0000","last_change":"2023-07-06T03:45:11.690985+0000","last_active":"2023-07-06T03:45:13.830215+0000","last_peered":"2023-07-06T03:45:13.830215+0000","last_clean":"2023-07-06T03:45:13.830215+0000","last_became_active":"2023-07-06T03:45:11.690692+0000","last_became_peered":"2023-07-06T03:45:11.690692+0000","last_unstale":"2023-07-06T03:45:13.830215+0000","last_undegraded":"2023-07-06T03:45:13.830215+0000","last_fullsized":"2023-07-06T03:45:13.830215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:27:20.751301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681704+0000","last_change":"2023-07-06T03:45:10.649734+0000","last_active":"2023-07-06T03:45:12.681704+0000","last_peered":"2023-07-06T03:45:12.681704+0000","last_clean":"2023-07-06T03:45:12.681704+0000","last_became_active":"2023-07-06T03:45:10.649401+0000","last_became_peered":"2023-07-06T03:45:10.649401+0000","last_unstale":"2023-07-06T03:45:12.681704+0000","last_undegraded":"2023-07-06T03:45:12.681704+0000","last_fullsized":"2023-07-06T03:45:12.681704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:24:42.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678483+0000","last_change":"2023-07-06T03:45:11.667398+0000","last_active":"2023-07-06T03:45:12.678483+0000","last_peered":"2023-07-06T03:45:12.678483+0000","last_clean":"2023-07-06T03:45:12.678483+0000","last_became_active":"2023-07-06T03:45:11.667124+0000","last_became_peered":"2023-07-06T03:45:11.667124+0000","last_unstale":"2023-07-06T03:45:12.678483+0000","last_undegraded":"2023-07-06T03:45:12.678483+0000","last_fullsized":"2023-07-06T03:45:12.678483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:00.096345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.679013+0000","last_change":"2023-07-06T03:45:10.648660+0000","last_active":"2023-07-06T03:45:12.679013+0000","last_peered":"2023-07-06T03:45:12.679013+0000","last_clean":"2023-07-06T03:45:12.679013+0000","last_became_active":"2023-07-06T03:45:10.648254+0000","last_became_peered":"2023-07-06T03:45:10.648254+0000","last_unstale":"2023-07-06T03:45:12.679013+0000","last_undegraded":"2023-07-06T03:45:12.679013+0000","last_fullsized":"2023-07-06T03:45:12.679013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:25.838923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205490+0000","last_change":"2023-07-06T03:45:11.682171+0000","last_active":"2023-07-06T03:45:14.205490+0000","last_peered":"2023-07-06T03:45:14.205490+0000","last_clean":"2023-07-06T03:45:14.205490+0000","last_became_active":"2023-07-06T03:45:11.681861+0000","last_became_peered":"2023-07-06T03:45:11.681861+0000","last_unstale":"2023-07-06T03:45:14.205490+0000","last_undegraded":"2023-07-06T03:45:14.205490+0000","last_fullsized":"2023-07-06T03:45:14.205490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827972+0000","last_change":"2023-07-06T03:45:10.656590+0000","last_active":"2023-07-06T03:45:13.827972+0000","last_peered":"2023-07-06T03:45:13.827972+0000","last_clean":"2023-07-06T03:45:13.827972+0000","last_became_active":"2023-07-06T03:45:10.656328+0000","last_became_peered":"2023-07-06T03:45:10.656328+0000","last_unstale":"2023-07-06T03:45:13.827972+0000","last_undegraded":"2023-07-06T03:45:13.827972+0000","last_fullsized":"2023-07-06T03:45:13.827972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205720+0000","last_change":"2023-07-06T03:45:11.671697+0000","last_active":"2023-07-06T03:45:14.205720+0000","last_peered":"2023-07-06T03:45:14.205720+0000","last_clean":"2023-07-06T03:45:14.205720+0000","last_became_active":"2023-07-06T03:45:11.671343+0000","last_became_peered":"2023-07-06T03:45:11.671343+0000","last_unstale":"2023-07-06T03:45:14.205720+0000","last_undegraded":"2023-07-06T03:45:14.205720+0000","last_fullsized":"2023-07-06T03:45:14.205720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:13.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829989+0000","last_change":"2023-07-06T03:45:10.644219+0000","last_active":"2023-07-06T03:45:13.829989+0000","last_peered":"2023-07-06T03:45:13.829989+0000","last_clean":"2023-07-06T03:45:13.829989+0000","last_became_active":"2023-07-06T03:45:10.643854+0000","last_became_peered":"2023-07-06T03:45:10.643854+0000","last_unstale":"2023-07-06T03:45:13.829989+0000","last_undegraded":"2023-07-06T03:45:13.829989+0000","last_fullsized":"2023-07-06T03:45:13.829989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:50:44.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678700+0000","last_change":"2023-07-06T03:45:11.665747+0000","last_active":"2023-07-06T03:45:12.678700+0000","last_peered":"2023-07-06T03:45:12.678700+0000","last_clean":"2023-07-06T03:45:12.678700+0000","last_became_active":"2023-07-06T03:45:11.665443+0000","last_became_peered":"2023-07-06T03:45:11.665443+0000","last_unstale":"2023-07-06T03:45:12.678700+0000","last_undegraded":"2023-07-06T03:45:12.678700+0000","last_fullsized":"2023-07-06T03:45:12.678700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829814+0000","last_change":"2023-07-06T03:45:10.653534+0000","last_active":"2023-07-06T03:45:13.829814+0000","last_peered":"2023-07-06T03:45:13.829814+0000","last_clean":"2023-07-06T03:45:13.829814+0000","last_became_active":"2023-07-06T03:45:10.653232+0000","last_became_peered":"2023-07-06T03:45:10.653232+0000","last_unstale":"2023-07-06T03:45:13.829814+0000","last_undegraded":"2023-07-06T03:45:13.829814+0000","last_fullsized":"2023-07-06T03:45:13.829814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:16.021365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652974+0000","last_change":"2023-07-06T03:45:11.683611+0000","last_active":"2023-07-06T03:45:13.652974+0000","last_peered":"2023-07-06T03:45:13.652974+0000","last_clean":"2023-07-06T03:45:13.652974+0000","last_became_active":"2023-07-06T03:45:11.683343+0000","last_became_peered":"2023-07-06T03:45:11.683343+0000","last_unstale":"2023-07-06T03:45:13.652974+0000","last_undegraded":"2023-07-06T03:45:13.652974+0000","last_fullsized":"2023-07-06T03:45:13.652974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652756+0000","last_change":"2023-07-06T03:45:10.655738+0000","last_active":"2023-07-06T03:45:13.652756+0000","last_peered":"2023-07-06T03:45:13.652756+0000","last_clean":"2023-07-06T03:45:13.652756+0000","last_became_active":"2023-07-06T03:45:10.655414+0000","last_became_peered":"2023-07-06T03:45:10.655414+0000","last_unstale":"2023-07-06T03:45:13.652756+0000","last_undegraded":"2023-07-06T03:45:13.652756+0000","last_fullsized":"2023-07-06T03:45:13.652756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:38.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830471+0000","last_change":"2023-07-06T03:45:11.678704+0000","last_active":"2023-07-06T03:45:13.830471+0000","last_peered":"2023-07-06T03:45:13.830471+0000","last_clean":"2023-07-06T03:45:13.830471+0000","last_became_active":"2023-07-06T03:45:11.678393+0000","last_became_peered":"2023-07-06T03:45:11.678393+0000","last_unstale":"2023-07-06T03:45:13.830471+0000","last_undegraded":"2023-07-06T03:45:13.830471+0000","last_fullsized":"2023-07-06T03:45:13.830471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:42.303647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830064+0000","last_change":"2023-07-06T03:45:10.644075+0000","last_active":"2023-07-06T03:45:13.830064+0000","last_peered":"2023-07-06T03:45:13.830064+0000","last_clean":"2023-07-06T03:45:13.830064+0000","last_became_active":"2023-07-06T03:45:10.643785+0000","last_became_peered":"2023-07-06T03:45:10.643785+0000","last_unstale":"2023-07-06T03:45:13.830064+0000","last_undegraded":"2023-07-06T03:45:13.830064+0000","last_fullsized":"2023-07-06T03:45:13.830064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:01:07.035467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681216+0000","last_change":"2023-07-06T03:45:11.687766+0000","last_active":"2023-07-06T03:45:12.681216+0000","last_peered":"2023-07-06T03:45:12.681216+0000","last_clean":"2023-07-06T03:45:12.681216+0000","last_became_active":"2023-07-06T03:45:11.687528+0000","last_became_peered":"2023-07-06T03:45:11.687528+0000","last_unstale":"2023-07-06T03:45:12.681216+0000","last_undegraded":"2023-07-06T03:45:12.681216+0000","last_fullsized":"2023-07-06T03:45:12.681216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817929+0000","last_change":"2023-07-06T03:45:10.652546+0000","last_active":"2023-07-06T03:45:12.817929+0000","last_peered":"2023-07-06T03:45:12.817929+0000","last_clean":"2023-07-06T03:45:12.817929+0000","last_became_active":"2023-07-06T03:45:10.652314+0000","last_became_peered":"2023-07-06T03:45:10.652314+0000","last_unstale":"2023-07-06T03:45:12.817929+0000","last_undegraded":"2023-07-06T03:45:12.817929+0000","last_fullsized":"2023-07-06T03:45:12.817929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681543+0000","last_change":"2023-07-06T03:45:11.695000+0000","last_active":"2023-07-06T03:45:12.681543+0000","last_peered":"2023-07-06T03:45:12.681543+0000","last_clean":"2023-07-06T03:45:12.681543+0000","last_became_active":"2023-07-06T03:45:11.694569+0000","last_became_peered":"2023-07-06T03:45:11.694569+0000","last_unstale":"2023-07-06T03:45:12.681543+0000","last_undegraded":"2023-07-06T03:45:12.681543+0000","last_fullsized":"2023-07-06T03:45:12.681543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:52.666500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.828097+0000","last_change":"2023-07-06T03:45:10.654280+0000","last_active":"2023-07-06T03:45:13.828097+0000","last_peered":"2023-07-06T03:45:13.828097+0000","last_clean":"2023-07-06T03:45:13.828097+0000","last_became_active":"2023-07-06T03:45:10.654039+0000","last_became_peered":"2023-07-06T03:45:10.654039+0000","last_unstale":"2023-07-06T03:45:13.828097+0000","last_undegraded":"2023-07-06T03:45:13.828097+0000","last_fullsized":"2023-07-06T03:45:13.828097+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:07.054998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829778+0000","last_change":"2023-07-06T03:45:11.667282+0000","last_active":"2023-07-06T03:45:13.829778+0000","last_peered":"2023-07-06T03:45:13.829778+0000","last_clean":"2023-07-06T03:45:13.829778+0000","last_became_active":"2023-07-06T03:45:11.667027+0000","last_became_peered":"2023-07-06T03:45:11.667027+0000","last_unstale":"2023-07-06T03:45:13.829778+0000","last_undegraded":"2023-07-06T03:45:13.829778+0000","last_fullsized":"2023-07-06T03:45:13.829778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:05:40.764560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678971+0000","last_change":"2023-07-06T03:45:10.647142+0000","last_active":"2023-07-06T03:45:12.678971+0000","last_peered":"2023-07-06T03:45:12.678971+0000","last_clean":"2023-07-06T03:45:12.678971+0000","last_became_active":"2023-07-06T03:45:10.646882+0000","last_became_peered":"2023-07-06T03:45:10.646882+0000","last_unstale":"2023-07-06T03:45:12.678971+0000","last_undegraded":"2023-07-06T03:45:12.678971+0000","last_fullsized":"2023-07-06T03:45:12.678971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:59.089238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817907+0000","last_change":"2023-07-06T03:45:11.691436+0000","last_active":"2023-07-06T03:45:12.817907+0000","last_peered":"2023-07-06T03:45:12.817907+0000","last_clean":"2023-07-06T03:45:12.817907+0000","last_became_active":"2023-07-06T03:45:11.691139+0000","last_became_peered":"2023-07-06T03:45:11.691139+0000","last_unstale":"2023-07-06T03:45:12.817907+0000","last_undegraded":"2023-07-06T03:45:12.817907+0000","last_fullsized":"2023-07-06T03:45:12.817907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829985+0000","last_change":"2023-07-06T03:45:10.647677+0000","last_active":"2023-07-06T03:45:13.829985+0000","last_peered":"2023-07-06T03:45:13.829985+0000","last_clean":"2023-07-06T03:45:13.829985+0000","last_became_active":"2023-07-06T03:45:10.647414+0000","last_became_peered":"2023-07-06T03:45:10.647414+0000","last_unstale":"2023-07-06T03:45:13.829985+0000","last_undegraded":"2023-07-06T03:45:13.829985+0000","last_fullsized":"2023-07-06T03:45:13.829985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:34:02.031995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678552+0000","last_change":"2023-07-06T03:45:11.663454+0000","last_active":"2023-07-06T03:45:12.678552+0000","last_peered":"2023-07-06T03:45:12.678552+0000","last_clean":"2023-07-06T03:45:12.678552+0000","last_became_active":"2023-07-06T03:45:11.663172+0000","last_became_peered":"2023-07-06T03:45:11.663172+0000","last_unstale":"2023-07-06T03:45:12.678552+0000","last_undegraded":"2023-07-06T03:45:12.678552+0000","last_fullsized":"2023-07-06T03:45:12.678552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827748+0000","last_change":"2023-07-06T03:45:10.651255+0000","last_active":"2023-07-06T03:45:13.827748+0000","last_peered":"2023-07-06T03:45:13.827748+0000","last_clean":"2023-07-06T03:45:13.827748+0000","last_became_active":"2023-07-06T03:45:10.651012+0000","last_became_peered":"2023-07-06T03:45:10.651012+0000","last_unstale":"2023-07-06T03:45:13.827748+0000","last_undegraded":"2023-07-06T03:45:13.827748+0000","last_fullsized":"2023-07-06T03:45:13.827748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:46.872621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680880+0000","last_change":"2023-07-06T03:45:11.688331+0000","last_active":"2023-07-06T03:45:12.680880+0000","last_peered":"2023-07-06T03:45:12.680880+0000","last_clean":"2023-07-06T03:45:12.680880+0000","last_became_active":"2023-07-06T03:45:11.686894+0000","last_became_peered":"2023-07-06T03:45:11.686894+0000","last_unstale":"2023-07-06T03:45:12.680880+0000","last_undegraded":"2023-07-06T03:45:12.680880+0000","last_fullsized":"2023-07-06T03:45:12.680880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:08.062104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678168+0000","last_change":"2023-07-06T03:45:10.653064+0000","last_active":"2023-07-06T03:45:12.678168+0000","last_peered":"2023-07-06T03:45:12.678168+0000","last_clean":"2023-07-06T03:45:12.678168+0000","last_became_active":"2023-07-06T03:45:10.652765+0000","last_became_peered":"2023-07-06T03:45:10.652765+0000","last_unstale":"2023-07-06T03:45:12.678168+0000","last_undegraded":"2023-07-06T03:45:12.678168+0000","last_fullsized":"2023-07-06T03:45:12.678168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:23.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827686+0000","last_change":"2023-07-06T03:45:11.683336+0000","last_active":"2023-07-06T03:45:13.827686+0000","last_peered":"2023-07-06T03:45:13.827686+0000","last_clean":"2023-07-06T03:45:13.827686+0000","last_became_active":"2023-07-06T03:45:11.683087+0000","last_became_peered":"2023-07-06T03:45:11.683087+0000","last_unstale":"2023-07-06T03:45:13.827686+0000","last_undegraded":"2023-07-06T03:45:13.827686+0000","last_fullsized":"2023-07-06T03:45:13.827686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830000+0000","last_change":"2023-07-06T03:45:10.650363+0000","last_active":"2023-07-06T03:45:13.830000+0000","last_peered":"2023-07-06T03:45:13.830000+0000","last_clean":"2023-07-06T03:45:13.830000+0000","last_became_active":"2023-07-06T03:45:10.650097+0000","last_became_peered":"2023-07-06T03:45:10.650097+0000","last_unstale":"2023-07-06T03:45:13.830000+0000","last_undegraded":"2023-07-06T03:45:13.830000+0000","last_fullsized":"2023-07-06T03:45:13.830000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:05.604877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830398+0000","last_change":"2023-07-06T03:45:11.668010+0000","last_active":"2023-07-06T03:45:13.830398+0000","last_peered":"2023-07-06T03:45:13.830398+0000","last_clean":"2023-07-06T03:45:13.830398+0000","last_became_active":"2023-07-06T03:45:11.667739+0000","last_became_peered":"2023-07-06T03:45:11.667739+0000","last_unstale":"2023-07-06T03:45:13.830398+0000","last_undegraded":"2023-07-06T03:45:13.830398+0000","last_fullsized":"2023-07-06T03:45:13.830398+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:06:53.685059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829747+0000","last_change":"2023-07-06T03:45:10.661165+0000","last_active":"2023-07-06T03:45:13.829747+0000","last_peered":"2023-07-06T03:45:13.829747+0000","last_clean":"2023-07-06T03:45:13.829747+0000","last_became_active":"2023-07-06T03:45:10.660843+0000","last_became_peered":"2023-07-06T03:45:10.660843+0000","last_unstale":"2023-07-06T03:45:13.829747+0000","last_undegraded":"2023-07-06T03:45:13.829747+0000","last_fullsized":"2023-07-06T03:45:13.829747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:02:20.704169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678908+0000","last_change":"2023-07-06T03:45:11.665692+0000","last_active":"2023-07-06T03:45:12.678908+0000","last_peered":"2023-07-06T03:45:12.678908+0000","last_clean":"2023-07-06T03:45:12.678908+0000","last_became_active":"2023-07-06T03:45:11.665425+0000","last_became_peered":"2023-07-06T03:45:11.665425+0000","last_unstale":"2023-07-06T03:45:12.678908+0000","last_undegraded":"2023-07-06T03:45:12.678908+0000","last_fullsized":"2023-07-06T03:45:12.678908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205473+0000","last_change":"2023-07-06T03:45:10.658201+0000","last_active":"2023-07-06T03:45:14.205473+0000","last_peered":"2023-07-06T03:45:14.205473+0000","last_clean":"2023-07-06T03:45:14.205473+0000","last_became_active":"2023-07-06T03:45:10.657688+0000","last_became_peered":"2023-07-06T03:45:10.657688+0000","last_unstale":"2023-07-06T03:45:14.205473+0000","last_undegraded":"2023-07-06T03:45:14.205473+0000","last_fullsized":"2023-07-06T03:45:14.205473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:26.377648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678497+0000","last_change":"2023-07-06T03:45:11.673905+0000","last_active":"2023-07-06T03:45:12.678497+0000","last_peered":"2023-07-06T03:45:12.678497+0000","last_clean":"2023-07-06T03:45:12.678497+0000","last_became_active":"2023-07-06T03:45:11.673609+0000","last_became_peered":"2023-07-06T03:45:11.673609+0000","last_unstale":"2023-07-06T03:45:12.678497+0000","last_undegraded":"2023-07-06T03:45:12.678497+0000","last_fullsized":"2023-07-06T03:45:12.678497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681110+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:45:12.681110+0000","last_peered":"2023-07-06T03:45:12.681110+0000","last_clean":"2023-07-06T03:45:12.681110+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:45:12.681110+0000","last_undegraded":"2023-07-06T03:45:12.681110+0000","last_fullsized":"2023-07-06T03:45:12.681110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:20.874020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678806+0000","last_change":"2023-07-06T03:45:10.660307+0000","last_active":"2023-07-06T03:45:12.678806+0000","last_peered":"2023-07-06T03:45:12.678806+0000","last_clean":"2023-07-06T03:45:12.678806+0000","last_became_active":"2023-07-06T03:45:10.660032+0000","last_became_peered":"2023-07-06T03:45:10.660032+0000","last_unstale":"2023-07-06T03:45:12.678806+0000","last_undegraded":"2023-07-06T03:45:12.678806+0000","last_fullsized":"2023-07-06T03:45:12.678806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652629+0000","last_change":"2023-07-06T03:45:11.679839+0000","last_active":"2023-07-06T03:45:13.652629+0000","last_peered":"2023-07-06T03:45:13.652629+0000","last_clean":"2023-07-06T03:45:13.652629+0000","last_became_active":"2023-07-06T03:45:11.679509+0000","last_became_peered":"2023-07-06T03:45:11.679509+0000","last_unstale":"2023-07-06T03:45:13.652629+0000","last_undegraded":"2023-07-06T03:45:13.652629+0000","last_fullsized":"2023-07-06T03:45:13.652629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830179+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:45:13.830179+0000","last_peered":"2023-07-06T03:45:13.830179+0000","last_clean":"2023-07-06T03:45:13.830179+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:45:13.830179+0000","last_undegraded":"2023-07-06T03:45:13.830179+0000","last_fullsized":"2023-07-06T03:45:13.830179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:03:06.569126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827774+0000","last_change":"2023-07-06T03:45:10.649547+0000","last_active":"2023-07-06T03:45:13.827774+0000","last_peered":"2023-07-06T03:45:13.827774+0000","last_clean":"2023-07-06T03:45:13.827774+0000","last_became_active":"2023-07-06T03:45:10.649231+0000","last_became_peered":"2023-07-06T03:45:10.649231+0000","last_unstale":"2023-07-06T03:45:13.827774+0000","last_undegraded":"2023-07-06T03:45:13.827774+0000","last_fullsized":"2023-07-06T03:45:13.827774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:13.496490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653155+0000","last_change":"2023-07-06T03:45:11.678151+0000","last_active":"2023-07-06T03:45:13.653155+0000","last_peered":"2023-07-06T03:45:13.653155+0000","last_clean":"2023-07-06T03:45:13.653155+0000","last_became_active":"2023-07-06T03:45:11.677841+0000","last_became_peered":"2023-07-06T03:45:11.677841+0000","last_unstale":"2023-07-06T03:45:13.653155+0000","last_undegraded":"2023-07-06T03:45:13.653155+0000","last_fullsized":"2023-07-06T03:45:13.653155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678308+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:45:12.678308+0000","last_peered":"2023-07-06T03:45:12.678308+0000","last_clean":"2023-07-06T03:45:12.678308+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:45:12.678308+0000","last_undegraded":"2023-07-06T03:45:12.678308+0000","last_fullsized":"2023-07-06T03:45:12.678308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:11.498012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680537+0000","last_change":"2023-07-06T03:45:10.647981+0000","last_active":"2023-07-06T03:45:12.680537+0000","last_peered":"2023-07-06T03:45:12.680537+0000","last_clean":"2023-07-06T03:45:12.680537+0000","last_became_active":"2023-07-06T03:45:10.647678+0000","last_became_peered":"2023-07-06T03:45:10.647678+0000","last_unstale":"2023-07-06T03:45:12.680537+0000","last_undegraded":"2023-07-06T03:45:12.680537+0000","last_fullsized":"2023-07-06T03:45:12.680537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653181+0000","last_change":"2023-07-06T03:45:11.683452+0000","last_active":"2023-07-06T03:45:13.653181+0000","last_peered":"2023-07-06T03:45:13.653181+0000","last_clean":"2023-07-06T03:45:13.653181+0000","last_became_active":"2023-07-06T03:45:11.683187+0000","last_became_peered":"2023-07-06T03:45:11.683187+0000","last_unstale":"2023-07-06T03:45:13.653181+0000","last_undegraded":"2023-07-06T03:45:13.653181+0000","last_fullsized":"2023-07-06T03:45:13.653181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:38.118602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829567+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:45:13.829567+0000","last_peered":"2023-07-06T03:45:13.829567+0000","last_clean":"2023-07-06T03:45:13.829567+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:45:13.829567+0000","last_undegraded":"2023-07-06T03:45:13.829567+0000","last_fullsized":"2023-07-06T03:45:13.829567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:49:25.728537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817899+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:45:12.817899+0000","last_peered":"2023-07-06T03:45:12.817899+0000","last_clean":"2023-07-06T03:45:12.817899+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:45:12.817899+0000","last_undegraded":"2023-07-06T03:45:12.817899+0000","last_fullsized":"2023-07-06T03:45:12.817899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:39.120643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205527+0000","last_change":"2023-07-06T03:45:10.657235+0000","last_active":"2023-07-06T03:45:14.205527+0000","last_peered":"2023-07-06T03:45:14.205527+0000","last_clean":"2023-07-06T03:45:14.205527+0000","last_became_active":"2023-07-06T03:45:10.656945+0000","last_became_peered":"2023-07-06T03:45:10.656945+0000","last_unstale":"2023-07-06T03:45:14.205527+0000","last_undegraded":"2023-07-06T03:45:14.205527+0000","last_fullsized":"2023-07-06T03:45:14.205527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:57.080176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830434+0000","last_change":"2023-07-06T03:45:11.668172+0000","last_active":"2023-07-06T03:45:13.830434+0000","last_peered":"2023-07-06T03:45:13.830434+0000","last_clean":"2023-07-06T03:45:13.830434+0000","last_became_active":"2023-07-06T03:45:11.667041+0000","last_became_peered":"2023-07-06T03:45:11.667041+0000","last_unstale":"2023-07-06T03:45:13.830434+0000","last_undegraded":"2023-07-06T03:45:13.830434+0000","last_fullsized":"2023-07-06T03:45:13.830434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:57.552603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829547+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:45:13.829547+0000","last_peered":"2023-07-06T03:45:13.829547+0000","last_clean":"2023-07-06T03:45:13.829547+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:45:13.829547+0000","last_undegraded":"2023-07-06T03:45:13.829547+0000","last_fullsized":"2023-07-06T03:45:13.829547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:25:39.144218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652538+0000","last_change":"2023-07-06T03:45:10.660900+0000","last_active":"2023-07-06T03:45:13.652538+0000","last_peered":"2023-07-06T03:45:13.652538+0000","last_clean":"2023-07-06T03:45:13.652538+0000","last_became_active":"2023-07-06T03:45:10.660490+0000","last_became_peered":"2023-07-06T03:45:10.660490+0000","last_unstale":"2023-07-06T03:45:13.652538+0000","last_undegraded":"2023-07-06T03:45:13.652538+0000","last_fullsized":"2023-07-06T03:45:13.652538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:43.801501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681363+0000","last_change":"2023-07-06T03:45:11.687991+0000","last_active":"2023-07-06T03:45:12.681363+0000","last_peered":"2023-07-06T03:45:12.681363+0000","last_clean":"2023-07-06T03:45:12.681363+0000","last_became_active":"2023-07-06T03:45:11.685096+0000","last_became_peered":"2023-07-06T03:45:11.685096+0000","last_unstale":"2023-07-06T03:45:12.681363+0000","last_undegraded":"2023-07-06T03:45:12.681363+0000","last_fullsized":"2023-07-06T03:45:12.681363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830169+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:45:13.830169+0000","last_peered":"2023-07-06T03:45:13.830169+0000","last_clean":"2023-07-06T03:45:13.830169+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:45:13.830169+0000","last_undegraded":"2023-07-06T03:45:13.830169+0000","last_fullsized":"2023-07-06T03:45:13.830169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:27:55.564390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653143+0000","last_change":"2023-07-06T03:45:10.661812+0000","last_active":"2023-07-06T03:45:13.653143+0000","last_peered":"2023-07-06T03:45:13.653143+0000","last_clean":"2023-07-06T03:45:13.653143+0000","last_became_active":"2023-07-06T03:45:10.661499+0000","last_became_peered":"2023-07-06T03:45:10.661499+0000","last_unstale":"2023-07-06T03:45:13.653143+0000","last_undegraded":"2023-07-06T03:45:13.653143+0000","last_fullsized":"2023-07-06T03:45:13.653143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:59.089238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652878+0000","last_change":"2023-07-06T03:45:11.678171+0000","last_active":"2023-07-06T03:45:13.652878+0000","last_peered":"2023-07-06T03:45:13.652878+0000","last_clean":"2023-07-06T03:45:13.652878+0000","last_became_active":"2023-07-06T03:45:11.677886+0000","last_became_peered":"2023-07-06T03:45:11.677886+0000","last_unstale":"2023-07-06T03:45:13.652878+0000","last_undegraded":"2023-07-06T03:45:13.652878+0000","last_fullsized":"2023-07-06T03:45:13.652878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206208+0000","last_change":"2023-07-06T03:45:10.657429+0000","last_active":"2023-07-06T03:45:14.206208+0000","last_peered":"2023-07-06T03:45:14.206208+0000","last_clean":"2023-07-06T03:45:14.206208+0000","last_became_active":"2023-07-06T03:45:10.657085+0000","last_became_peered":"2023-07-06T03:45:10.657085+0000","last_unstale":"2023-07-06T03:45:14.206208+0000","last_undegraded":"2023-07-06T03:45:14.206208+0000","last_fullsized":"2023-07-06T03:45:14.206208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:24:47.052321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206026+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:45:14.206026+0000","last_peered":"2023-07-06T03:45:14.206026+0000","last_clean":"2023-07-06T03:45:14.206026+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:45:14.206026+0000","last_undegraded":"2023-07-06T03:45:14.206026+0000","last_fullsized":"2023-07-06T03:45:14.206026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:05:14.989733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205502+0000","last_change":"2023-07-06T03:45:11.671780+0000","last_active":"2023-07-06T03:45:14.205502+0000","last_peered":"2023-07-06T03:45:14.205502+0000","last_clean":"2023-07-06T03:45:14.205502+0000","last_became_active":"2023-07-06T03:45:11.671390+0000","last_became_peered":"2023-07-06T03:45:11.671390+0000","last_unstale":"2023-07-06T03:45:14.205502+0000","last_undegraded":"2023-07-06T03:45:14.205502+0000","last_fullsized":"2023-07-06T03:45:14.205502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829963+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:45:13.829963+0000","last_peered":"2023-07-06T03:45:13.829963+0000","last_clean":"2023-07-06T03:45:13.829963+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:45:13.829963+0000","last_undegraded":"2023-07-06T03:45:13.829963+0000","last_fullsized":"2023-07-06T03:45:13.829963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:12.638307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206183+0000","last_change":"2023-07-06T03:45:10.657912+0000","last_active":"2023-07-06T03:45:14.206183+0000","last_peered":"2023-07-06T03:45:14.206183+0000","last_clean":"2023-07-06T03:45:14.206183+0000","last_became_active":"2023-07-06T03:45:10.657615+0000","last_became_peered":"2023-07-06T03:45:10.657615+0000","last_unstale":"2023-07-06T03:45:14.206183+0000","last_undegraded":"2023-07-06T03:45:14.206183+0000","last_fullsized":"2023-07-06T03:45:14.206183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:30:30.877422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678345+0000","last_change":"2023-07-06T03:45:11.665120+0000","last_active":"2023-07-06T03:45:12.678345+0000","last_peered":"2023-07-06T03:45:12.678345+0000","last_clean":"2023-07-06T03:45:12.678345+0000","last_became_active":"2023-07-06T03:45:11.664817+0000","last_became_peered":"2023-07-06T03:45:11.664817+0000","last_unstale":"2023-07-06T03:45:12.678345+0000","last_undegraded":"2023-07-06T03:45:12.678345+0000","last_fullsized":"2023-07-06T03:45:12.678345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:44.808607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205783+0000","last_change":"2023-07-06T03:45:10.658517+0000","last_active":"2023-07-06T03:45:14.205783+0000","last_peered":"2023-07-06T03:45:14.205783+0000","last_clean":"2023-07-06T03:45:14.205783+0000","last_became_active":"2023-07-06T03:45:10.657946+0000","last_became_peered":"2023-07-06T03:45:10.657946+0000","last_unstale":"2023-07-06T03:45:14.205783+0000","last_undegraded":"2023-07-06T03:45:14.205783+0000","last_fullsized":"2023-07-06T03:45:14.205783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:24:42.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205842+0000","last_change":"2023-07-06T03:45:11.683447+0000","last_active":"2023-07-06T03:45:14.205842+0000","last_peered":"2023-07-06T03:45:14.205842+0000","last_clean":"2023-07-06T03:45:14.205842+0000","last_became_active":"2023-07-06T03:45:11.683156+0000","last_became_peered":"2023-07-06T03:45:11.683156+0000","last_unstale":"2023-07-06T03:45:14.205842+0000","last_undegraded":"2023-07-06T03:45:14.205842+0000","last_fullsized":"2023-07-06T03:45:14.205842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818006+0000","last_change":"2023-07-06T03:45:10.649186+0000","last_active":"2023-07-06T03:45:12.818006+0000","last_peered":"2023-07-06T03:45:12.818006+0000","last_clean":"2023-07-06T03:45:12.818006+0000","last_became_active":"2023-07-06T03:45:10.648707+0000","last_became_peered":"2023-07-06T03:45:10.648707+0000","last_unstale":"2023-07-06T03:45:12.818006+0000","last_undegraded":"2023-07-06T03:45:12.818006+0000","last_fullsized":"2023-07-06T03:45:12.818006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:07.054998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678566+0000","last_change":"2023-07-06T03:45:11.690462+0000","last_active":"2023-07-06T03:45:12.678566+0000","last_peered":"2023-07-06T03:45:12.678566+0000","last_clean":"2023-07-06T03:45:12.678566+0000","last_became_active":"2023-07-06T03:45:11.690181+0000","last_became_peered":"2023-07-06T03:45:11.690181+0000","last_unstale":"2023-07-06T03:45:12.678566+0000","last_undegraded":"2023-07-06T03:45:12.678566+0000","last_fullsized":"2023-07-06T03:45:12.678566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830291+0000","last_change":"2023-07-06T03:45:10.653276+0000","last_active":"2023-07-06T03:45:13.830291+0000","last_peered":"2023-07-06T03:45:13.830291+0000","last_clean":"2023-07-06T03:45:13.830291+0000","last_became_active":"2023-07-06T03:45:10.652958+0000","last_became_peered":"2023-07-06T03:45:10.652958+0000","last_unstale":"2023-07-06T03:45:13.830291+0000","last_undegraded":"2023-07-06T03:45:13.830291+0000","last_fullsized":"2023-07-06T03:45:13.830291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:45.669370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678497+0000","last_change":"2023-07-06T03:45:11.684391+0000","last_active":"2023-07-06T03:45:12.678497+0000","last_peered":"2023-07-06T03:45:12.678497+0000","last_clean":"2023-07-06T03:45:12.678497+0000","last_became_active":"2023-07-06T03:45:11.684078+0000","last_became_peered":"2023-07-06T03:45:11.684078+0000","last_unstale":"2023-07-06T03:45:12.678497+0000","last_undegraded":"2023-07-06T03:45:12.678497+0000","last_fullsized":"2023-07-06T03:45:12.678497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681347+0000","last_change":"2023-07-06T03:45:10.645296+0000","last_active":"2023-07-06T03:45:12.681347+0000","last_peered":"2023-07-06T03:45:12.681347+0000","last_clean":"2023-07-06T03:45:12.681347+0000","last_became_active":"2023-07-06T03:45:10.644967+0000","last_became_peered":"2023-07-06T03:45:10.644967+0000","last_unstale":"2023-07-06T03:45:12.681347+0000","last_undegraded":"2023-07-06T03:45:12.681347+0000","last_fullsized":"2023-07-06T03:45:12.681347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:12.862578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678143+0000","last_change":"2023-07-06T03:45:11.683562+0000","last_active":"2023-07-06T03:45:12.678143+0000","last_peered":"2023-07-06T03:45:12.678143+0000","last_clean":"2023-07-06T03:45:12.678143+0000","last_became_active":"2023-07-06T03:45:11.683275+0000","last_became_peered":"2023-07-06T03:45:11.683275+0000","last_unstale":"2023-07-06T03:45:12.678143+0000","last_undegraded":"2023-07-06T03:45:12.678143+0000","last_fullsized":"2023-07-06T03:45:12.678143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:44.808607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681723+0000","last_change":"2023-07-06T03:45:10.654706+0000","last_active":"2023-07-06T03:45:12.681723+0000","last_peered":"2023-07-06T03:45:12.681723+0000","last_clean":"2023-07-06T03:45:12.681723+0000","last_became_active":"2023-07-06T03:45:10.653995+0000","last_became_peered":"2023-07-06T03:45:10.653995+0000","last_unstale":"2023-07-06T03:45:12.681723+0000","last_undegraded":"2023-07-06T03:45:12.681723+0000","last_fullsized":"2023-07-06T03:45:12.681723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:05:20.493361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678265+0000","last_change":"2023-07-06T03:45:11.670138+0000","last_active":"2023-07-06T03:45:12.678265+0000","last_peered":"2023-07-06T03:45:12.678265+0000","last_clean":"2023-07-06T03:45:12.678265+0000","last_became_active":"2023-07-06T03:45:11.669695+0000","last_became_peered":"2023-07-06T03:45:11.669695+0000","last_unstale":"2023-07-06T03:45:12.678265+0000","last_undegraded":"2023-07-06T03:45:12.678265+0000","last_fullsized":"2023-07-06T03:45:12.678265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:22:55.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818277+0000","last_change":"2023-07-06T03:45:11.673123+0000","last_active":"2023-07-06T03:45:12.818277+0000","last_peered":"2023-07-06T03:45:12.818277+0000","last_clean":"2023-07-06T03:45:12.818277+0000","last_became_active":"2023-07-06T03:45:11.672765+0000","last_became_peered":"2023-07-06T03:45:11.672765+0000","last_unstale":"2023-07-06T03:45:12.818277+0000","last_undegraded":"2023-07-06T03:45:12.818277+0000","last_fullsized":"2023-07-06T03:45:12.818277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652635+0000","last_change":"2023-07-06T03:45:11.689866+0000","last_active":"2023-07-06T03:45:13.652635+0000","last_peered":"2023-07-06T03:45:13.652635+0000","last_clean":"2023-07-06T03:45:13.652635+0000","last_became_active":"2023-07-06T03:45:11.689595+0000","last_became_peered":"2023-07-06T03:45:11.689595+0000","last_unstale":"2023-07-06T03:45:13.652635+0000","last_undegraded":"2023-07-06T03:45:13.652635+0000","last_fullsized":"2023-07-06T03:45:13.652635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827567+0000","last_change":"2023-07-06T03:45:11.675336+0000","last_active":"2023-07-06T03:45:13.827567+0000","last_peered":"2023-07-06T03:45:13.827567+0000","last_clean":"2023-07-06T03:45:13.827567+0000","last_became_active":"2023-07-06T03:45:11.675044+0000","last_became_peered":"2023-07-06T03:45:11.675044+0000","last_unstale":"2023-07-06T03:45:13.827567+0000","last_undegraded":"2023-07-06T03:45:13.827567+0000","last_fullsized":"2023-07-06T03:45:13.827567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680990+0000","last_change":"2023-07-06T03:45:11.689419+0000","last_active":"2023-07-06T03:45:12.680990+0000","last_peered":"2023-07-06T03:45:12.680990+0000","last_clean":"2023-07-06T03:45:12.680990+0000","last_became_active":"2023-07-06T03:45:11.688843+0000","last_became_peered":"2023-07-06T03:45:11.688843+0000","last_unstale":"2023-07-06T03:45:12.680990+0000","last_undegraded":"2023-07-06T03:45:12.680990+0000","last_fullsized":"2023-07-06T03:45:12.680990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:15.627811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680936+0000","last_change":"2023-07-06T03:45:11.687716+0000","last_active":"2023-07-06T03:45:12.680936+0000","last_peered":"2023-07-06T03:45:12.680936+0000","last_clean":"2023-07-06T03:45:12.680936+0000","last_became_active":"2023-07-06T03:45:11.685418+0000","last_became_peered":"2023-07-06T03:45:11.685418+0000","last_unstale":"2023-07-06T03:45:12.680936+0000","last_undegraded":"2023-07-06T03:45:12.680936+0000","last_fullsized":"2023-07-06T03:45:12.680936+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":847872,"data_stored":680641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8879999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.055999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9249999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.989000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.806}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":89745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":91163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:45:26.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:45:26.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.874+0000 7f8544dff700 1 Processor -- start 2023-07-06T03:45:26.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.874+0000 7f8544dff700 1 -- start start 2023-07-06T03:45:26.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.875+0000 7f8544dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f854015c520 0x7f85400cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.875+0000 7f8544dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8540160d00 0x7f85400b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.875+0000 7f8544dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85400cdae0 con 0x7f8540160180 2023-07-06T03:45:26.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.875+0000 7f8544dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85400cd8d0 con 0x7f8540160d00 2023-07-06T03:45:26.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.875+0000 7f8544dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85400b6640 con 0x7f854015c520 2023-07-06T03:45:26.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.875+0000 7f853dd9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8540160180 0x7f85401605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35538/0 (socket says 172.21.15.97:35538) 2023-07-06T03:45:26.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.875+0000 7f853dd9b700 1 -- 172.21.15.97:0/548008209 learned_addr learned my addr 172.21.15.97:0/548008209 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853e59c700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f854015c520 0x7f85400cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853ed9d700 1 --2- 172.21.15.97:0/548008209 >> v2:172.21.15.146:3300/0 conn(0x7f8540160d00 0x7f85400b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619340978 0 0) 0x7f85400cdae0 con 0x7f8540160180 2023-07-06T03:45:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85280035b0 con 0x7f8540160180 2023-07-06T03:45:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2077173863 0 0) 0x7f85280035b0 con 0x7f8540160180 2023-07-06T03:45:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f854015c520 msgr2=0x7f85400cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:26.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f854015c520 0x7f85400cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 >> v2:172.21.15.146:3300/0 conn(0x7f8540160d00 msgr2=0x7f85400b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:26.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 --2- 172.21.15.97:0/548008209 >> v2:172.21.15.146:3300/0 conn(0x7f8540160d00 0x7f85400b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85400b6920 con 0x7f8540160180 2023-07-06T03:45:26.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f8534001670 con 0x7f8540160180 2023-07-06T03:45:26.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853ed9d700 1 --2- 172.21.15.97:0/548008209 >> v2:172.21.15.146:3300/0 conn(0x7f8540160d00 0x7f85400b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:26.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 >> v1:172.21.15.97:6789/0 conn(0x7f8540160180 legacy=0x7f85401605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f854015c520 0x7f8528003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 0x7f8528004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8528004d90 0x7f8528005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85400cd8d0 con 0x7f8528003f80 2023-07-06T03:45:26.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85400b6640 con 0x7f8528004d90 2023-07-06T03:45:26.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.876+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852800ba10 con 0x7f854015c520 2023-07-06T03:45:26.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853dd9b700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 0x7f8528004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853e59c700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f854015c520 0x7f8528003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853ed9d700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8528004d90 0x7f8528005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853dd9b700 1 -- 172.21.15.97:0/548008209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f854015c520 msgr2=0x7f8528003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:26.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853dd9b700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f854015c520 0x7f8528003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853dd9b700 1 -- 172.21.15.97:0/548008209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8528004d90 msgr2=0x7f8528005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:26.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853dd9b700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8528004d90 0x7f8528005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853dd9b700 1 -- 172.21.15.97:0/548008209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85280035b0 con 0x7f8528003f80 2023-07-06T03:45:26.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853dd9b700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 0x7f8528004850 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f853400c320 tx=0x7f8528005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41bc20a8454e098 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:26.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853400ce60 con 0x7f8528003f80 2023-07-06T03:45:26.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8534005960 con 0x7f8528003f80 2023-07-06T03:45:26.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.877+0000 7f853d59a700 1 -- 172.21.15.97:0/548008209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8534005d10 con 0x7f8528003f80 2023-07-06T03:45:26.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.878+0000 7f8544dff700 1 -- 172.21.15.97:0/548008209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 msgr2=0x7f8528004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.878+0000 7f8544dff700 1 --2- 172.21.15.97:0/548008209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 0x7f8528004850 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f853400c320 tx=0x7f8528005880 comp rx=0 tx=0).stop 2023-07-06T03:45:26.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.878+0000 7f853ed9d700 1 -- 172.21.15.97:0/548008209 reap_dead start 2023-07-06T03:45:26.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.878+0000 7f8544dff700 1 -- 172.21.15.97:0/548008209 shutdown_connections 2023-07-06T03:45:26.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.878+0000 7f8544dff700 1 -- 172.21.15.97:0/548008209 >> 172.21.15.97:0/548008209 conn(0x7f8540071790 msgr2=0x7f8540071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:26.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.878+0000 7f8544dff700 1 -- 172.21.15.97:0/548008209 shutdown_connections 2023-07-06T03:45:26.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.878+0000 7f8544dff700 1 -- 172.21.15.97:0/548008209 wait complete. 2023-07-06T03:45:26.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.880+0000 7f8544dff700 1 Processor -- start 2023-07-06T03:45:26.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.880+0000 7f8544dff700 1 -- start start 2023-07-06T03:45:26.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.881+0000 7f8544dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 0x7f85400d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.881+0000 7f8544dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85400d3910 0x7f85400c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.881+0000 7f8544dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85401e73f0 0x7f85401f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.881+0000 7f8544dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85401e6fd0 con 0x7f8528003f80 2023-07-06T03:45:26.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.881+0000 7f8544dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85400cc570 con 0x7f85401e73f0 2023-07-06T03:45:26.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.881+0000 7f8544dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85400ccd40 con 0x7f85400d3910 2023-07-06T03:45:26.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.881+0000 7f853e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 0x7f85400d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.881+0000 7f853e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 0x7f85400d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51792/0 (socket says 172.21.15.97:51792) 2023-07-06T03:45:26.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.881+0000 7f853e59c700 1 -- 172.21.15.97:0/3278177194 learned_addr learned my addr 172.21.15.97:0/3278177194 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:26.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f853dd9b700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85400d3910 0x7f85400c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f853e59c700 1 -- 172.21.15.97:0/3278177194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85400d3910 msgr2=0x7f85400c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f853e59c700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85400d3910 0x7f85400c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f853ed9d700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85401e73f0 0x7f85401f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f853e59c700 1 -- 172.21.15.97:0/3278177194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85401e73f0 msgr2=0x7f85401f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:26.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f853e59c700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85401e73f0 0x7f85401f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:26.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f853e59c700 1 -- 172.21.15.97:0/3278177194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85400cc780 con 0x7f8528003f80 2023-07-06T03:45:26.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f853dd9b700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85400d3910 0x7f85400c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:26.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f853ed9d700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85401e73f0 0x7f85401f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:26.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f853e59c700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 0x7f85400d0c40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f85300119a0 tx=0x7f8530011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:26.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f852effd700 1 -- 172.21.15.97:0/3278177194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853000f940 con 0x7f8528003f80 2023-07-06T03:45:26.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f852effd700 1 -- 172.21.15.97:0/3278177194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853000faa0 con 0x7f8528003f80 2023-07-06T03:45:26.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f852effd700 1 -- 172.21.15.97:0/3278177194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853000fe30 con 0x7f8528003f80 2023-07-06T03:45:26.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f8544dff700 1 -- 172.21.15.97:0/3278177194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85400cca60 con 0x7f8528003f80 2023-07-06T03:45:26.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.882+0000 7f8544dff700 1 -- 172.21.15.97:0/3278177194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85400d1180 con 0x7f8528003f80 2023-07-06T03:45:26.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.883+0000 7f8544dff700 1 -- 172.21.15.97:0/3278177194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854004e340 con 0x7f8528003f80 2023-07-06T03:45:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.884+0000 7f852effd700 1 -- 172.21.15.97:0/3278177194 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8530014440 con 0x7f8528003f80 2023-07-06T03:45:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.885+0000 7f852effd700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f852006ace0 0x7f852006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.885+0000 7f852effd700 1 -- 172.21.15.97:0/3278177194 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8530089af0 con 0x7f8528003f80 2023-07-06T03:45:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.885+0000 7f853dd9b700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f852006ace0 0x7f852006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.886+0000 7f853dd9b700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f852006ace0 0x7f852006d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8534015040 tx=0x7f853400a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:26.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:26.890+0000 7f852effd700 1 -- 172.21.15.97:0/3278177194 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8530054c40 con 0x7f8528003f80 2023-07-06T03:45:27.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.082+0000 7f8544dff700 1 -- 172.21.15.97:0/3278177194 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85400b6ee0 con 0x7f852006ace0 2023-07-06T03:45:27.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.090+0000 7f852effd700 1 -- 172.21.15.97:0/3278177194 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286592 (secure 0 0 0) 0x7f853400cce0 con 0x7f852006ace0 2023-07-06T03:45:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:27.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 -- 172.21.15.97:0/3278177194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f852006ace0 msgr2=0x7f852006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f852006ace0 0x7f852006d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8534015040 tx=0x7f853400a000 comp rx=0 tx=0).stop 2023-07-06T03:45:27.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 -- 172.21.15.97:0/3278177194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 msgr2=0x7f85400d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 0x7f85400d0c40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f85300119a0 tx=0x7f8530011d60 comp rx=0 tx=0).stop 2023-07-06T03:45:27.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 -- 172.21.15.97:0/3278177194 shutdown_connections 2023-07-06T03:45:27.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f852006ace0 0x7f852006d190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85401e73f0 0x7f85401f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8528003f80 0x7f85400d0c40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 --2- 172.21.15.97:0/3278177194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f85400d3910 0x7f85400c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 -- 172.21.15.97:0/3278177194 >> 172.21.15.97:0/3278177194 conn(0x7f8540071790 msgr2=0x7f85400b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 -- 172.21.15.97:0/3278177194 shutdown_connections 2023-07-06T03:45:27.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.094+0000 7f8544dff700 1 -- 172.21.15.97:0/3278177194 wait complete. 2023-07-06T03:45:27.099 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:45:27.114 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-07-06T03:45:26.885644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218724,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002789"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681463+0000","last_change":"2023-07-06T03:45:11.693177+0000","last_active":"2023-07-06T03:45:12.681463+0000","last_peered":"2023-07-06T03:45:12.681463+0000","last_clean":"2023-07-06T03:45:12.681463+0000","last_became_active":"2023-07-06T03:45:11.692851+0000","last_became_peered":"2023-07-06T03:45:11.692851+0000","last_unstale":"2023-07-06T03:45:12.681463+0000","last_undegraded":"2023-07-06T03:45:12.681463+0000","last_fullsized":"2023-07-06T03:45:12.681463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678856+0000","last_change":"2023-07-06T03:45:11.674359+0000","last_active":"2023-07-06T03:45:12.678856+0000","last_peered":"2023-07-06T03:45:12.678856+0000","last_clean":"2023-07-06T03:45:12.678856+0000","last_became_active":"2023-07-06T03:45:11.674035+0000","last_became_peered":"2023-07-06T03:45:11.674035+0000","last_unstale":"2023-07-06T03:45:12.678856+0000","last_undegraded":"2023-07-06T03:45:12.678856+0000","last_fullsized":"2023-07-06T03:45:12.678856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827906+0000","last_change":"2023-07-06T03:45:11.691806+0000","last_active":"2023-07-06T03:45:13.827906+0000","last_peered":"2023-07-06T03:45:13.827906+0000","last_clean":"2023-07-06T03:45:13.827906+0000","last_became_active":"2023-07-06T03:45:11.691532+0000","last_became_peered":"2023-07-06T03:45:11.691532+0000","last_unstale":"2023-07-06T03:45:13.827906+0000","last_undegraded":"2023-07-06T03:45:13.827906+0000","last_fullsized":"2023-07-06T03:45:13.827906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827578+0000","last_change":"2023-07-06T03:45:11.675300+0000","last_active":"2023-07-06T03:45:13.827578+0000","last_peered":"2023-07-06T03:45:13.827578+0000","last_clean":"2023-07-06T03:45:13.827578+0000","last_became_active":"2023-07-06T03:45:11.675018+0000","last_became_peered":"2023-07-06T03:45:11.675018+0000","last_unstale":"2023-07-06T03:45:13.827578+0000","last_undegraded":"2023-07-06T03:45:13.827578+0000","last_fullsized":"2023-07-06T03:45:13.827578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205854+0000","last_change":"2023-07-06T03:45:11.666099+0000","last_active":"2023-07-06T03:45:14.205854+0000","last_peered":"2023-07-06T03:45:14.205854+0000","last_clean":"2023-07-06T03:45:14.205854+0000","last_became_active":"2023-07-06T03:45:11.665759+0000","last_became_peered":"2023-07-06T03:45:11.665759+0000","last_unstale":"2023-07-06T03:45:14.205854+0000","last_undegraded":"2023-07-06T03:45:14.205854+0000","last_fullsized":"2023-07-06T03:45:14.205854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:52.666500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830215+0000","last_change":"2023-07-06T03:45:11.662251+0000","last_active":"2023-07-06T03:45:13.830215+0000","last_peered":"2023-07-06T03:45:13.830215+0000","last_clean":"2023-07-06T03:45:13.830215+0000","last_became_active":"2023-07-06T03:45:11.661916+0000","last_became_peered":"2023-07-06T03:45:11.661916+0000","last_unstale":"2023-07-06T03:45:13.830215+0000","last_undegraded":"2023-07-06T03:45:13.830215+0000","last_fullsized":"2023-07-06T03:45:13.830215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:11.287937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829687+0000","last_change":"2023-07-06T03:45:11.676465+0000","last_active":"2023-07-06T03:45:13.829687+0000","last_peered":"2023-07-06T03:45:13.829687+0000","last_clean":"2023-07-06T03:45:13.829687+0000","last_became_active":"2023-07-06T03:45:11.676180+0000","last_became_peered":"2023-07-06T03:45:11.676180+0000","last_unstale":"2023-07-06T03:45:13.829687+0000","last_undegraded":"2023-07-06T03:45:13.829687+0000","last_fullsized":"2023-07-06T03:45:13.829687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:38:24.542890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818440+0000","last_change":"2023-07-06T03:45:11.676387+0000","last_active":"2023-07-06T03:45:12.818440+0000","last_peered":"2023-07-06T03:45:12.818440+0000","last_clean":"2023-07-06T03:45:12.818440+0000","last_became_active":"2023-07-06T03:45:11.676129+0000","last_became_peered":"2023-07-06T03:45:11.676129+0000","last_unstale":"2023-07-06T03:45:12.818440+0000","last_undegraded":"2023-07-06T03:45:12.818440+0000","last_fullsized":"2023-07-06T03:45:12.818440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206076+0000","last_change":"2023-07-06T03:45:11.690977+0000","last_active":"2023-07-06T03:45:14.206076+0000","last_peered":"2023-07-06T03:45:14.206076+0000","last_clean":"2023-07-06T03:45:14.206076+0000","last_became_active":"2023-07-06T03:45:11.690423+0000","last_became_peered":"2023-07-06T03:45:11.690423+0000","last_unstale":"2023-07-06T03:45:14.206076+0000","last_undegraded":"2023-07-06T03:45:14.206076+0000","last_fullsized":"2023-07-06T03:45:14.206076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:14:59.597908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818347+0000","last_change":"2023-07-06T03:45:11.680924+0000","last_active":"2023-07-06T03:45:12.818347+0000","last_peered":"2023-07-06T03:45:12.818347+0000","last_clean":"2023-07-06T03:45:12.818347+0000","last_became_active":"2023-07-06T03:45:11.680607+0000","last_became_peered":"2023-07-06T03:45:11.680607+0000","last_unstale":"2023-07-06T03:45:12.818347+0000","last_undegraded":"2023-07-06T03:45:12.818347+0000","last_fullsized":"2023-07-06T03:45:12.818347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:13.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818149+0000","last_change":"2023-07-06T03:45:11.677946+0000","last_active":"2023-07-06T03:45:12.818149+0000","last_peered":"2023-07-06T03:45:12.818149+0000","last_clean":"2023-07-06T03:45:12.818149+0000","last_became_active":"2023-07-06T03:45:11.677684+0000","last_became_peered":"2023-07-06T03:45:11.677684+0000","last_unstale":"2023-07-06T03:45:12.818149+0000","last_undegraded":"2023-07-06T03:45:12.818149+0000","last_fullsized":"2023-07-06T03:45:12.818149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:27.384754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680941+0000","last_change":"2023-07-06T03:45:11.681178+0000","last_active":"2023-07-06T03:45:12.680941+0000","last_peered":"2023-07-06T03:45:12.680941+0000","last_clean":"2023-07-06T03:45:12.680941+0000","last_became_active":"2023-07-06T03:45:11.680841+0000","last_became_peered":"2023-07-06T03:45:11.680841+0000","last_unstale":"2023-07-06T03:45:12.680941+0000","last_undegraded":"2023-07-06T03:45:12.680941+0000","last_fullsized":"2023-07-06T03:45:12.680941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:58.087282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652673+0000","last_change":"2023-07-06T03:45:11.687461+0000","last_active":"2023-07-06T03:45:13.652673+0000","last_peered":"2023-07-06T03:45:13.652673+0000","last_clean":"2023-07-06T03:45:13.652673+0000","last_became_active":"2023-07-06T03:45:11.687182+0000","last_became_peered":"2023-07-06T03:45:11.687182+0000","last_unstale":"2023-07-06T03:45:13.652673+0000","last_undegraded":"2023-07-06T03:45:13.652673+0000","last_fullsized":"2023-07-06T03:45:13.652673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817899+0000","last_change":"2023-07-06T03:45:11.691496+0000","last_active":"2023-07-06T03:45:12.817899+0000","last_peered":"2023-07-06T03:45:12.817899+0000","last_clean":"2023-07-06T03:45:12.817899+0000","last_became_active":"2023-07-06T03:45:11.691206+0000","last_became_peered":"2023-07-06T03:45:11.691206+0000","last_unstale":"2023-07-06T03:45:12.817899+0000","last_undegraded":"2023-07-06T03:45:12.817899+0000","last_fullsized":"2023-07-06T03:45:12.817899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827487+0000","last_change":"2023-07-06T03:45:11.685001+0000","last_active":"2023-07-06T03:45:13.827487+0000","last_peered":"2023-07-06T03:45:13.827487+0000","last_clean":"2023-07-06T03:45:13.827487+0000","last_became_active":"2023-07-06T03:45:11.684695+0000","last_became_peered":"2023-07-06T03:45:11.684695+0000","last_unstale":"2023-07-06T03:45:13.827487+0000","last_undegraded":"2023-07-06T03:45:13.827487+0000","last_fullsized":"2023-07-06T03:45:13.827487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829742+0000","last_change":"2023-07-06T03:45:11.662096+0000","last_active":"2023-07-06T03:45:13.829742+0000","last_peered":"2023-07-06T03:45:13.829742+0000","last_clean":"2023-07-06T03:45:13.829742+0000","last_became_active":"2023-07-06T03:45:11.661834+0000","last_became_peered":"2023-07-06T03:45:11.661834+0000","last_unstale":"2023-07-06T03:45:13.829742+0000","last_undegraded":"2023-07-06T03:45:13.829742+0000","last_fullsized":"2023-07-06T03:45:13.829742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:36:49.534209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678504+0000","last_change":"2023-07-06T03:45:11.685162+0000","last_active":"2023-07-06T03:45:12.678504+0000","last_peered":"2023-07-06T03:45:12.678504+0000","last_clean":"2023-07-06T03:45:12.678504+0000","last_became_active":"2023-07-06T03:45:11.684977+0000","last_became_peered":"2023-07-06T03:45:11.684977+0000","last_unstale":"2023-07-06T03:45:12.678504+0000","last_undegraded":"2023-07-06T03:45:12.678504+0000","last_fullsized":"2023-07-06T03:45:12.678504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827827+0000","last_change":"2023-07-06T03:45:11.675611+0000","last_active":"2023-07-06T03:45:13.827827+0000","last_peered":"2023-07-06T03:45:13.827827+0000","last_clean":"2023-07-06T03:45:13.827827+0000","last_became_active":"2023-07-06T03:45:11.675348+0000","last_became_peered":"2023-07-06T03:45:11.675348+0000","last_unstale":"2023-07-06T03:45:13.827827+0000","last_undegraded":"2023-07-06T03:45:13.827827+0000","last_fullsized":"2023-07-06T03:45:13.827827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:00.096345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680922+0000","last_change":"2023-07-06T03:45:11.690182+0000","last_active":"2023-07-06T03:45:12.680922+0000","last_peered":"2023-07-06T03:45:12.680922+0000","last_clean":"2023-07-06T03:45:12.680922+0000","last_became_active":"2023-07-06T03:45:11.689744+0000","last_became_peered":"2023-07-06T03:45:11.689744+0000","last_unstale":"2023-07-06T03:45:12.680922+0000","last_undegraded":"2023-07-06T03:45:12.680922+0000","last_fullsized":"2023-07-06T03:45:12.680922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:32.898881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818137+0000","last_change":"2023-07-06T03:45:11.672749+0000","last_active":"2023-07-06T03:45:12.818137+0000","last_peered":"2023-07-06T03:45:12.818137+0000","last_clean":"2023-07-06T03:45:12.818137+0000","last_became_active":"2023-07-06T03:45:11.672401+0000","last_became_peered":"2023-07-06T03:45:11.672401+0000","last_unstale":"2023-07-06T03:45:12.818137+0000","last_undegraded":"2023-07-06T03:45:12.818137+0000","last_fullsized":"2023-07-06T03:45:12.818137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:15.627811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818032+0000","last_change":"2023-07-06T03:45:11.692060+0000","last_active":"2023-07-06T03:45:12.818032+0000","last_peered":"2023-07-06T03:45:12.818032+0000","last_clean":"2023-07-06T03:45:12.818032+0000","last_became_active":"2023-07-06T03:45:11.691699+0000","last_became_peered":"2023-07-06T03:45:11.691699+0000","last_unstale":"2023-07-06T03:45:12.818032+0000","last_undegraded":"2023-07-06T03:45:12.818032+0000","last_fullsized":"2023-07-06T03:45:12.818032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:32.898881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678195+0000","last_change":"2023-07-06T03:45:11.672926+0000","last_active":"2023-07-06T03:45:12.678195+0000","last_peered":"2023-07-06T03:45:12.678195+0000","last_clean":"2023-07-06T03:45:12.678195+0000","last_became_active":"2023-07-06T03:45:11.672627+0000","last_became_peered":"2023-07-06T03:45:11.672627+0000","last_unstale":"2023-07-06T03:45:12.678195+0000","last_undegraded":"2023-07-06T03:45:12.678195+0000","last_fullsized":"2023-07-06T03:45:12.678195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205814+0000","last_change":"2023-07-06T03:45:11.691373+0000","last_active":"2023-07-06T03:45:14.205814+0000","last_peered":"2023-07-06T03:45:14.205814+0000","last_clean":"2023-07-06T03:45:14.205814+0000","last_became_active":"2023-07-06T03:45:11.691046+0000","last_became_peered":"2023-07-06T03:45:11.691046+0000","last_unstale":"2023-07-06T03:45:14.205814+0000","last_undegraded":"2023-07-06T03:45:14.205814+0000","last_fullsized":"2023-07-06T03:45:14.205814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818372+0000","last_change":"2023-07-06T03:45:11.691643+0000","last_active":"2023-07-06T03:45:12.818372+0000","last_peered":"2023-07-06T03:45:12.818372+0000","last_clean":"2023-07-06T03:45:12.818372+0000","last_became_active":"2023-07-06T03:45:11.691382+0000","last_became_peered":"2023-07-06T03:45:11.691382+0000","last_unstale":"2023-07-06T03:45:12.818372+0000","last_undegraded":"2023-07-06T03:45:12.818372+0000","last_fullsized":"2023-07-06T03:45:12.818372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206243+0000","last_change":"2023-07-06T03:45:11.670205+0000","last_active":"2023-07-06T03:45:14.206243+0000","last_peered":"2023-07-06T03:45:14.206243+0000","last_clean":"2023-07-06T03:45:14.206243+0000","last_became_active":"2023-07-06T03:45:11.669935+0000","last_became_peered":"2023-07-06T03:45:11.669935+0000","last_unstale":"2023-07-06T03:45:14.206243+0000","last_undegraded":"2023-07-06T03:45:14.206243+0000","last_fullsized":"2023-07-06T03:45:14.206243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:11:07.089396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678592+0000","last_change":"2023-07-06T03:45:11.674601+0000","last_active":"2023-07-06T03:45:12.678592+0000","last_peered":"2023-07-06T03:45:12.678592+0000","last_clean":"2023-07-06T03:45:12.678592+0000","last_became_active":"2023-07-06T03:45:11.673533+0000","last_became_peered":"2023-07-06T03:45:11.673533+0000","last_unstale":"2023-07-06T03:45:12.678592+0000","last_undegraded":"2023-07-06T03:45:12.678592+0000","last_fullsized":"2023-07-06T03:45:12.678592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830285+0000","last_change":"2023-07-06T03:45:11.678777+0000","last_active":"2023-07-06T03:45:13.830285+0000","last_peered":"2023-07-06T03:45:13.830285+0000","last_clean":"2023-07-06T03:45:13.830285+0000","last_became_active":"2023-07-06T03:45:11.678437+0000","last_became_peered":"2023-07-06T03:45:11.678437+0000","last_unstale":"2023-07-06T03:45:13.830285+0000","last_undegraded":"2023-07-06T03:45:13.830285+0000","last_fullsized":"2023-07-06T03:45:13.830285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:51:53.330078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818105+0000","last_change":"2023-07-06T03:45:10.656464+0000","last_active":"2023-07-06T03:45:12.818105+0000","last_peered":"2023-07-06T03:45:12.818105+0000","last_clean":"2023-07-06T03:45:12.818105+0000","last_became_active":"2023-07-06T03:45:10.656116+0000","last_became_peered":"2023-07-06T03:45:10.656116+0000","last_unstale":"2023-07-06T03:45:12.818105+0000","last_undegraded":"2023-07-06T03:45:12.818105+0000","last_fullsized":"2023-07-06T03:45:12.818105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:57.080176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678331+0000","last_change":"2023-07-06T03:45:11.687243+0000","last_active":"2023-07-06T03:45:12.678331+0000","last_peered":"2023-07-06T03:45:12.678331+0000","last_clean":"2023-07-06T03:45:12.678331+0000","last_became_active":"2023-07-06T03:45:11.686947+0000","last_became_peered":"2023-07-06T03:45:11.686947+0000","last_unstale":"2023-07-06T03:45:12.678331+0000","last_undegraded":"2023-07-06T03:45:12.678331+0000","last_fullsized":"2023-07-06T03:45:12.678331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205529+0000","last_change":"2023-07-06T03:45:10.657736+0000","last_active":"2023-07-06T03:45:14.205529+0000","last_peered":"2023-07-06T03:45:14.205529+0000","last_clean":"2023-07-06T03:45:14.205529+0000","last_became_active":"2023-07-06T03:45:10.657450+0000","last_became_peered":"2023-07-06T03:45:10.657450+0000","last_unstale":"2023-07-06T03:45:14.205529+0000","last_undegraded":"2023-07-06T03:45:14.205529+0000","last_fullsized":"2023-07-06T03:45:14.205529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:14.620705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206125+0000","last_change":"2023-07-06T03:45:11.672564+0000","last_active":"2023-07-06T03:45:14.206125+0000","last_peered":"2023-07-06T03:45:14.206125+0000","last_clean":"2023-07-06T03:45:14.206125+0000","last_became_active":"2023-07-06T03:45:11.672150+0000","last_became_peered":"2023-07-06T03:45:11.672150+0000","last_unstale":"2023-07-06T03:45:14.206125+0000","last_undegraded":"2023-07-06T03:45:14.206125+0000","last_fullsized":"2023-07-06T03:45:14.206125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:59:01.536411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680441+0000","last_change":"2023-07-06T03:45:10.653666+0000","last_active":"2023-07-06T03:45:12.680441+0000","last_peered":"2023-07-06T03:45:12.680441+0000","last_clean":"2023-07-06T03:45:12.680441+0000","last_became_active":"2023-07-06T03:45:10.653357+0000","last_became_peered":"2023-07-06T03:45:10.653357+0000","last_unstale":"2023-07-06T03:45:12.680441+0000","last_undegraded":"2023-07-06T03:45:12.680441+0000","last_fullsized":"2023-07-06T03:45:12.680441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:25.838923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830215+0000","last_change":"2023-07-06T03:45:11.690985+0000","last_active":"2023-07-06T03:45:13.830215+0000","last_peered":"2023-07-06T03:45:13.830215+0000","last_clean":"2023-07-06T03:45:13.830215+0000","last_became_active":"2023-07-06T03:45:11.690692+0000","last_became_peered":"2023-07-06T03:45:11.690692+0000","last_unstale":"2023-07-06T03:45:13.830215+0000","last_undegraded":"2023-07-06T03:45:13.830215+0000","last_fullsized":"2023-07-06T03:45:13.830215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:27:20.751301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681704+0000","last_change":"2023-07-06T03:45:10.649734+0000","last_active":"2023-07-06T03:45:12.681704+0000","last_peered":"2023-07-06T03:45:12.681704+0000","last_clean":"2023-07-06T03:45:12.681704+0000","last_became_active":"2023-07-06T03:45:10.649401+0000","last_became_peered":"2023-07-06T03:45:10.649401+0000","last_unstale":"2023-07-06T03:45:12.681704+0000","last_undegraded":"2023-07-06T03:45:12.681704+0000","last_fullsized":"2023-07-06T03:45:12.681704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:24:42.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678483+0000","last_change":"2023-07-06T03:45:11.667398+0000","last_active":"2023-07-06T03:45:12.678483+0000","last_peered":"2023-07-06T03:45:12.678483+0000","last_clean":"2023-07-06T03:45:12.678483+0000","last_became_active":"2023-07-06T03:45:11.667124+0000","last_became_peered":"2023-07-06T03:45:11.667124+0000","last_unstale":"2023-07-06T03:45:12.678483+0000","last_undegraded":"2023-07-06T03:45:12.678483+0000","last_fullsized":"2023-07-06T03:45:12.678483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:00.096345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.679013+0000","last_change":"2023-07-06T03:45:10.648660+0000","last_active":"2023-07-06T03:45:12.679013+0000","last_peered":"2023-07-06T03:45:12.679013+0000","last_clean":"2023-07-06T03:45:12.679013+0000","last_became_active":"2023-07-06T03:45:10.648254+0000","last_became_peered":"2023-07-06T03:45:10.648254+0000","last_unstale":"2023-07-06T03:45:12.679013+0000","last_undegraded":"2023-07-06T03:45:12.679013+0000","last_fullsized":"2023-07-06T03:45:12.679013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:25.838923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205490+0000","last_change":"2023-07-06T03:45:11.682171+0000","last_active":"2023-07-06T03:45:14.205490+0000","last_peered":"2023-07-06T03:45:14.205490+0000","last_clean":"2023-07-06T03:45:14.205490+0000","last_became_active":"2023-07-06T03:45:11.681861+0000","last_became_peered":"2023-07-06T03:45:11.681861+0000","last_unstale":"2023-07-06T03:45:14.205490+0000","last_undegraded":"2023-07-06T03:45:14.205490+0000","last_fullsized":"2023-07-06T03:45:14.205490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827972+0000","last_change":"2023-07-06T03:45:10.656590+0000","last_active":"2023-07-06T03:45:13.827972+0000","last_peered":"2023-07-06T03:45:13.827972+0000","last_clean":"2023-07-06T03:45:13.827972+0000","last_became_active":"2023-07-06T03:45:10.656328+0000","last_became_peered":"2023-07-06T03:45:10.656328+0000","last_unstale":"2023-07-06T03:45:13.827972+0000","last_undegraded":"2023-07-06T03:45:13.827972+0000","last_fullsized":"2023-07-06T03:45:13.827972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205720+0000","last_change":"2023-07-06T03:45:11.671697+0000","last_active":"2023-07-06T03:45:14.205720+0000","last_peered":"2023-07-06T03:45:14.205720+0000","last_clean":"2023-07-06T03:45:14.205720+0000","last_became_active":"2023-07-06T03:45:11.671343+0000","last_became_peered":"2023-07-06T03:45:11.671343+0000","last_unstale":"2023-07-06T03:45:14.205720+0000","last_undegraded":"2023-07-06T03:45:14.205720+0000","last_fullsized":"2023-07-06T03:45:14.205720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:13.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829989+0000","last_change":"2023-07-06T03:45:10.644219+0000","last_active":"2023-07-06T03:45:13.829989+0000","last_peered":"2023-07-06T03:45:13.829989+0000","last_clean":"2023-07-06T03:45:13.829989+0000","last_became_active":"2023-07-06T03:45:10.643854+0000","last_became_peered":"2023-07-06T03:45:10.643854+0000","last_unstale":"2023-07-06T03:45:13.829989+0000","last_undegraded":"2023-07-06T03:45:13.829989+0000","last_fullsized":"2023-07-06T03:45:13.829989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:50:44.307603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678700+0000","last_change":"2023-07-06T03:45:11.665747+0000","last_active":"2023-07-06T03:45:12.678700+0000","last_peered":"2023-07-06T03:45:12.678700+0000","last_clean":"2023-07-06T03:45:12.678700+0000","last_became_active":"2023-07-06T03:45:11.665443+0000","last_became_peered":"2023-07-06T03:45:11.665443+0000","last_unstale":"2023-07-06T03:45:12.678700+0000","last_undegraded":"2023-07-06T03:45:12.678700+0000","last_fullsized":"2023-07-06T03:45:12.678700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829814+0000","last_change":"2023-07-06T03:45:10.653534+0000","last_active":"2023-07-06T03:45:13.829814+0000","last_peered":"2023-07-06T03:45:13.829814+0000","last_clean":"2023-07-06T03:45:13.829814+0000","last_became_active":"2023-07-06T03:45:10.653232+0000","last_became_peered":"2023-07-06T03:45:10.653232+0000","last_unstale":"2023-07-06T03:45:13.829814+0000","last_undegraded":"2023-07-06T03:45:13.829814+0000","last_fullsized":"2023-07-06T03:45:13.829814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:16.021365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652974+0000","last_change":"2023-07-06T03:45:11.683611+0000","last_active":"2023-07-06T03:45:13.652974+0000","last_peered":"2023-07-06T03:45:13.652974+0000","last_clean":"2023-07-06T03:45:13.652974+0000","last_became_active":"2023-07-06T03:45:11.683343+0000","last_became_peered":"2023-07-06T03:45:11.683343+0000","last_unstale":"2023-07-06T03:45:13.652974+0000","last_undegraded":"2023-07-06T03:45:13.652974+0000","last_fullsized":"2023-07-06T03:45:13.652974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652756+0000","last_change":"2023-07-06T03:45:10.655738+0000","last_active":"2023-07-06T03:45:13.652756+0000","last_peered":"2023-07-06T03:45:13.652756+0000","last_clean":"2023-07-06T03:45:13.652756+0000","last_became_active":"2023-07-06T03:45:10.655414+0000","last_became_peered":"2023-07-06T03:45:10.655414+0000","last_unstale":"2023-07-06T03:45:13.652756+0000","last_undegraded":"2023-07-06T03:45:13.652756+0000","last_fullsized":"2023-07-06T03:45:13.652756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:38.735726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830471+0000","last_change":"2023-07-06T03:45:11.678704+0000","last_active":"2023-07-06T03:45:13.830471+0000","last_peered":"2023-07-06T03:45:13.830471+0000","last_clean":"2023-07-06T03:45:13.830471+0000","last_became_active":"2023-07-06T03:45:11.678393+0000","last_became_peered":"2023-07-06T03:45:11.678393+0000","last_unstale":"2023-07-06T03:45:13.830471+0000","last_undegraded":"2023-07-06T03:45:13.830471+0000","last_fullsized":"2023-07-06T03:45:13.830471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:42.303647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830064+0000","last_change":"2023-07-06T03:45:10.644075+0000","last_active":"2023-07-06T03:45:13.830064+0000","last_peered":"2023-07-06T03:45:13.830064+0000","last_clean":"2023-07-06T03:45:13.830064+0000","last_became_active":"2023-07-06T03:45:10.643785+0000","last_became_peered":"2023-07-06T03:45:10.643785+0000","last_unstale":"2023-07-06T03:45:13.830064+0000","last_undegraded":"2023-07-06T03:45:13.830064+0000","last_fullsized":"2023-07-06T03:45:13.830064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:01:07.035467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681216+0000","last_change":"2023-07-06T03:45:11.687766+0000","last_active":"2023-07-06T03:45:12.681216+0000","last_peered":"2023-07-06T03:45:12.681216+0000","last_clean":"2023-07-06T03:45:12.681216+0000","last_became_active":"2023-07-06T03:45:11.687528+0000","last_became_peered":"2023-07-06T03:45:11.687528+0000","last_unstale":"2023-07-06T03:45:12.681216+0000","last_undegraded":"2023-07-06T03:45:12.681216+0000","last_fullsized":"2023-07-06T03:45:12.681216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817929+0000","last_change":"2023-07-06T03:45:10.652546+0000","last_active":"2023-07-06T03:45:12.817929+0000","last_peered":"2023-07-06T03:45:12.817929+0000","last_clean":"2023-07-06T03:45:12.817929+0000","last_became_active":"2023-07-06T03:45:10.652314+0000","last_became_peered":"2023-07-06T03:45:10.652314+0000","last_unstale":"2023-07-06T03:45:12.817929+0000","last_undegraded":"2023-07-06T03:45:12.817929+0000","last_fullsized":"2023-07-06T03:45:12.817929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681543+0000","last_change":"2023-07-06T03:45:11.695000+0000","last_active":"2023-07-06T03:45:12.681543+0000","last_peered":"2023-07-06T03:45:12.681543+0000","last_clean":"2023-07-06T03:45:12.681543+0000","last_became_active":"2023-07-06T03:45:11.694569+0000","last_became_peered":"2023-07-06T03:45:11.694569+0000","last_unstale":"2023-07-06T03:45:12.681543+0000","last_undegraded":"2023-07-06T03:45:12.681543+0000","last_fullsized":"2023-07-06T03:45:12.681543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:52.666500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.828097+0000","last_change":"2023-07-06T03:45:10.654280+0000","last_active":"2023-07-06T03:45:13.828097+0000","last_peered":"2023-07-06T03:45:13.828097+0000","last_clean":"2023-07-06T03:45:13.828097+0000","last_became_active":"2023-07-06T03:45:10.654039+0000","last_became_peered":"2023-07-06T03:45:10.654039+0000","last_unstale":"2023-07-06T03:45:13.828097+0000","last_undegraded":"2023-07-06T03:45:13.828097+0000","last_fullsized":"2023-07-06T03:45:13.828097+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:07.054998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829778+0000","last_change":"2023-07-06T03:45:11.667282+0000","last_active":"2023-07-06T03:45:13.829778+0000","last_peered":"2023-07-06T03:45:13.829778+0000","last_clean":"2023-07-06T03:45:13.829778+0000","last_became_active":"2023-07-06T03:45:11.667027+0000","last_became_peered":"2023-07-06T03:45:11.667027+0000","last_unstale":"2023-07-06T03:45:13.829778+0000","last_undegraded":"2023-07-06T03:45:13.829778+0000","last_fullsized":"2023-07-06T03:45:13.829778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:05:40.764560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678971+0000","last_change":"2023-07-06T03:45:10.647142+0000","last_active":"2023-07-06T03:45:12.678971+0000","last_peered":"2023-07-06T03:45:12.678971+0000","last_clean":"2023-07-06T03:45:12.678971+0000","last_became_active":"2023-07-06T03:45:10.646882+0000","last_became_peered":"2023-07-06T03:45:10.646882+0000","last_unstale":"2023-07-06T03:45:12.678971+0000","last_undegraded":"2023-07-06T03:45:12.678971+0000","last_fullsized":"2023-07-06T03:45:12.678971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:59.089238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817907+0000","last_change":"2023-07-06T03:45:11.691436+0000","last_active":"2023-07-06T03:45:12.817907+0000","last_peered":"2023-07-06T03:45:12.817907+0000","last_clean":"2023-07-06T03:45:12.817907+0000","last_became_active":"2023-07-06T03:45:11.691139+0000","last_became_peered":"2023-07-06T03:45:11.691139+0000","last_unstale":"2023-07-06T03:45:12.817907+0000","last_undegraded":"2023-07-06T03:45:12.817907+0000","last_fullsized":"2023-07-06T03:45:12.817907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829985+0000","last_change":"2023-07-06T03:45:10.647677+0000","last_active":"2023-07-06T03:45:13.829985+0000","last_peered":"2023-07-06T03:45:13.829985+0000","last_clean":"2023-07-06T03:45:13.829985+0000","last_became_active":"2023-07-06T03:45:10.647414+0000","last_became_peered":"2023-07-06T03:45:10.647414+0000","last_unstale":"2023-07-06T03:45:13.829985+0000","last_undegraded":"2023-07-06T03:45:13.829985+0000","last_fullsized":"2023-07-06T03:45:13.829985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:34:02.031995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678552+0000","last_change":"2023-07-06T03:45:11.663454+0000","last_active":"2023-07-06T03:45:12.678552+0000","last_peered":"2023-07-06T03:45:12.678552+0000","last_clean":"2023-07-06T03:45:12.678552+0000","last_became_active":"2023-07-06T03:45:11.663172+0000","last_became_peered":"2023-07-06T03:45:11.663172+0000","last_unstale":"2023-07-06T03:45:12.678552+0000","last_undegraded":"2023-07-06T03:45:12.678552+0000","last_fullsized":"2023-07-06T03:45:12.678552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827748+0000","last_change":"2023-07-06T03:45:10.651255+0000","last_active":"2023-07-06T03:45:13.827748+0000","last_peered":"2023-07-06T03:45:13.827748+0000","last_clean":"2023-07-06T03:45:13.827748+0000","last_became_active":"2023-07-06T03:45:10.651012+0000","last_became_peered":"2023-07-06T03:45:10.651012+0000","last_unstale":"2023-07-06T03:45:13.827748+0000","last_undegraded":"2023-07-06T03:45:13.827748+0000","last_fullsized":"2023-07-06T03:45:13.827748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:46.872621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680880+0000","last_change":"2023-07-06T03:45:11.688331+0000","last_active":"2023-07-06T03:45:12.680880+0000","last_peered":"2023-07-06T03:45:12.680880+0000","last_clean":"2023-07-06T03:45:12.680880+0000","last_became_active":"2023-07-06T03:45:11.686894+0000","last_became_peered":"2023-07-06T03:45:11.686894+0000","last_unstale":"2023-07-06T03:45:12.680880+0000","last_undegraded":"2023-07-06T03:45:12.680880+0000","last_fullsized":"2023-07-06T03:45:12.680880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:08.062104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678168+0000","last_change":"2023-07-06T03:45:10.653064+0000","last_active":"2023-07-06T03:45:12.678168+0000","last_peered":"2023-07-06T03:45:12.678168+0000","last_clean":"2023-07-06T03:45:12.678168+0000","last_became_active":"2023-07-06T03:45:10.652765+0000","last_became_peered":"2023-07-06T03:45:10.652765+0000","last_unstale":"2023-07-06T03:45:12.678168+0000","last_undegraded":"2023-07-06T03:45:12.678168+0000","last_fullsized":"2023-07-06T03:45:12.678168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:23.545158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827686+0000","last_change":"2023-07-06T03:45:11.683336+0000","last_active":"2023-07-06T03:45:13.827686+0000","last_peered":"2023-07-06T03:45:13.827686+0000","last_clean":"2023-07-06T03:45:13.827686+0000","last_became_active":"2023-07-06T03:45:11.683087+0000","last_became_peered":"2023-07-06T03:45:11.683087+0000","last_unstale":"2023-07-06T03:45:13.827686+0000","last_undegraded":"2023-07-06T03:45:13.827686+0000","last_fullsized":"2023-07-06T03:45:13.827686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:39.742832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830000+0000","last_change":"2023-07-06T03:45:10.650363+0000","last_active":"2023-07-06T03:45:13.830000+0000","last_peered":"2023-07-06T03:45:13.830000+0000","last_clean":"2023-07-06T03:45:13.830000+0000","last_became_active":"2023-07-06T03:45:10.650097+0000","last_became_peered":"2023-07-06T03:45:10.650097+0000","last_unstale":"2023-07-06T03:45:13.830000+0000","last_undegraded":"2023-07-06T03:45:13.830000+0000","last_fullsized":"2023-07-06T03:45:13.830000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:28:05.604877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830398+0000","last_change":"2023-07-06T03:45:11.668010+0000","last_active":"2023-07-06T03:45:13.830398+0000","last_peered":"2023-07-06T03:45:13.830398+0000","last_clean":"2023-07-06T03:45:13.830398+0000","last_became_active":"2023-07-06T03:45:11.667739+0000","last_became_peered":"2023-07-06T03:45:11.667739+0000","last_unstale":"2023-07-06T03:45:13.830398+0000","last_undegraded":"2023-07-06T03:45:13.830398+0000","last_fullsized":"2023-07-06T03:45:13.830398+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:06:53.685059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829747+0000","last_change":"2023-07-06T03:45:10.661165+0000","last_active":"2023-07-06T03:45:13.829747+0000","last_peered":"2023-07-06T03:45:13.829747+0000","last_clean":"2023-07-06T03:45:13.829747+0000","last_became_active":"2023-07-06T03:45:10.660843+0000","last_became_peered":"2023-07-06T03:45:10.660843+0000","last_unstale":"2023-07-06T03:45:13.829747+0000","last_undegraded":"2023-07-06T03:45:13.829747+0000","last_fullsized":"2023-07-06T03:45:13.829747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:02:20.704169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678908+0000","last_change":"2023-07-06T03:45:11.665692+0000","last_active":"2023-07-06T03:45:12.678908+0000","last_peered":"2023-07-06T03:45:12.678908+0000","last_clean":"2023-07-06T03:45:12.678908+0000","last_became_active":"2023-07-06T03:45:11.665425+0000","last_became_peered":"2023-07-06T03:45:11.665425+0000","last_unstale":"2023-07-06T03:45:12.678908+0000","last_undegraded":"2023-07-06T03:45:12.678908+0000","last_fullsized":"2023-07-06T03:45:12.678908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:14.503596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205473+0000","last_change":"2023-07-06T03:45:10.658201+0000","last_active":"2023-07-06T03:45:14.205473+0000","last_peered":"2023-07-06T03:45:14.205473+0000","last_clean":"2023-07-06T03:45:14.205473+0000","last_became_active":"2023-07-06T03:45:10.657688+0000","last_became_peered":"2023-07-06T03:45:10.657688+0000","last_unstale":"2023-07-06T03:45:14.205473+0000","last_undegraded":"2023-07-06T03:45:14.205473+0000","last_fullsized":"2023-07-06T03:45:14.205473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:26.377648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678497+0000","last_change":"2023-07-06T03:45:11.673905+0000","last_active":"2023-07-06T03:45:12.678497+0000","last_peered":"2023-07-06T03:45:12.678497+0000","last_clean":"2023-07-06T03:45:12.678497+0000","last_became_active":"2023-07-06T03:45:11.673609+0000","last_became_peered":"2023-07-06T03:45:11.673609+0000","last_unstale":"2023-07-06T03:45:12.678497+0000","last_undegraded":"2023-07-06T03:45:12.678497+0000","last_fullsized":"2023-07-06T03:45:12.678497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681110+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:45:12.681110+0000","last_peered":"2023-07-06T03:45:12.681110+0000","last_clean":"2023-07-06T03:45:12.681110+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:45:12.681110+0000","last_undegraded":"2023-07-06T03:45:12.681110+0000","last_fullsized":"2023-07-06T03:45:12.681110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:22:20.874020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678806+0000","last_change":"2023-07-06T03:45:10.660307+0000","last_active":"2023-07-06T03:45:12.678806+0000","last_peered":"2023-07-06T03:45:12.678806+0000","last_clean":"2023-07-06T03:45:12.678806+0000","last_became_active":"2023-07-06T03:45:10.660032+0000","last_became_peered":"2023-07-06T03:45:10.660032+0000","last_unstale":"2023-07-06T03:45:12.678806+0000","last_undegraded":"2023-07-06T03:45:12.678806+0000","last_fullsized":"2023-07-06T03:45:12.678806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652629+0000","last_change":"2023-07-06T03:45:11.679839+0000","last_active":"2023-07-06T03:45:13.652629+0000","last_peered":"2023-07-06T03:45:13.652629+0000","last_clean":"2023-07-06T03:45:13.652629+0000","last_became_active":"2023-07-06T03:45:11.679509+0000","last_became_peered":"2023-07-06T03:45:11.679509+0000","last_unstale":"2023-07-06T03:45:13.652629+0000","last_undegraded":"2023-07-06T03:45:13.652629+0000","last_fullsized":"2023-07-06T03:45:13.652629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830179+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:45:13.830179+0000","last_peered":"2023-07-06T03:45:13.830179+0000","last_clean":"2023-07-06T03:45:13.830179+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:45:13.830179+0000","last_undegraded":"2023-07-06T03:45:13.830179+0000","last_fullsized":"2023-07-06T03:45:13.830179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:03:06.569126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827774+0000","last_change":"2023-07-06T03:45:10.649547+0000","last_active":"2023-07-06T03:45:13.827774+0000","last_peered":"2023-07-06T03:45:13.827774+0000","last_clean":"2023-07-06T03:45:13.827774+0000","last_became_active":"2023-07-06T03:45:10.649231+0000","last_became_peered":"2023-07-06T03:45:10.649231+0000","last_unstale":"2023-07-06T03:45:13.827774+0000","last_undegraded":"2023-07-06T03:45:13.827774+0000","last_fullsized":"2023-07-06T03:45:13.827774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:13.496490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653155+0000","last_change":"2023-07-06T03:45:11.678151+0000","last_active":"2023-07-06T03:45:13.653155+0000","last_peered":"2023-07-06T03:45:13.653155+0000","last_clean":"2023-07-06T03:45:13.653155+0000","last_became_active":"2023-07-06T03:45:11.677841+0000","last_became_peered":"2023-07-06T03:45:11.677841+0000","last_unstale":"2023-07-06T03:45:13.653155+0000","last_undegraded":"2023-07-06T03:45:13.653155+0000","last_fullsized":"2023-07-06T03:45:13.653155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:26.846029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678308+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:45:12.678308+0000","last_peered":"2023-07-06T03:45:12.678308+0000","last_clean":"2023-07-06T03:45:12.678308+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:45:12.678308+0000","last_undegraded":"2023-07-06T03:45:12.678308+0000","last_fullsized":"2023-07-06T03:45:12.678308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:11.498012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680537+0000","last_change":"2023-07-06T03:45:10.647981+0000","last_active":"2023-07-06T03:45:12.680537+0000","last_peered":"2023-07-06T03:45:12.680537+0000","last_clean":"2023-07-06T03:45:12.680537+0000","last_became_active":"2023-07-06T03:45:10.647678+0000","last_became_peered":"2023-07-06T03:45:10.647678+0000","last_unstale":"2023-07-06T03:45:12.680537+0000","last_undegraded":"2023-07-06T03:45:12.680537+0000","last_fullsized":"2023-07-06T03:45:12.680537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:37.111496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653181+0000","last_change":"2023-07-06T03:45:11.683452+0000","last_active":"2023-07-06T03:45:13.653181+0000","last_peered":"2023-07-06T03:45:13.653181+0000","last_clean":"2023-07-06T03:45:13.653181+0000","last_became_active":"2023-07-06T03:45:11.683187+0000","last_became_peered":"2023-07-06T03:45:11.683187+0000","last_unstale":"2023-07-06T03:45:13.653181+0000","last_undegraded":"2023-07-06T03:45:13.653181+0000","last_fullsized":"2023-07-06T03:45:13.653181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:38.118602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829567+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:45:13.829567+0000","last_peered":"2023-07-06T03:45:13.829567+0000","last_clean":"2023-07-06T03:45:13.829567+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:45:13.829567+0000","last_undegraded":"2023-07-06T03:45:13.829567+0000","last_fullsized":"2023-07-06T03:45:13.829567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:49:25.728537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.817899+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:45:12.817899+0000","last_peered":"2023-07-06T03:45:12.817899+0000","last_clean":"2023-07-06T03:45:12.817899+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:45:12.817899+0000","last_undegraded":"2023-07-06T03:45:12.817899+0000","last_fullsized":"2023-07-06T03:45:12.817899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:39.120643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205527+0000","last_change":"2023-07-06T03:45:10.657235+0000","last_active":"2023-07-06T03:45:14.205527+0000","last_peered":"2023-07-06T03:45:14.205527+0000","last_clean":"2023-07-06T03:45:14.205527+0000","last_became_active":"2023-07-06T03:45:10.656945+0000","last_became_peered":"2023-07-06T03:45:10.656945+0000","last_unstale":"2023-07-06T03:45:14.205527+0000","last_undegraded":"2023-07-06T03:45:14.205527+0000","last_fullsized":"2023-07-06T03:45:14.205527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:54:57.080176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830434+0000","last_change":"2023-07-06T03:45:11.668172+0000","last_active":"2023-07-06T03:45:13.830434+0000","last_peered":"2023-07-06T03:45:13.830434+0000","last_clean":"2023-07-06T03:45:13.830434+0000","last_became_active":"2023-07-06T03:45:11.667041+0000","last_became_peered":"2023-07-06T03:45:11.667041+0000","last_unstale":"2023-07-06T03:45:13.830434+0000","last_undegraded":"2023-07-06T03:45:13.830434+0000","last_fullsized":"2023-07-06T03:45:13.830434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:57.552603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829547+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:45:13.829547+0000","last_peered":"2023-07-06T03:45:13.829547+0000","last_clean":"2023-07-06T03:45:13.829547+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:45:13.829547+0000","last_undegraded":"2023-07-06T03:45:13.829547+0000","last_fullsized":"2023-07-06T03:45:13.829547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:25:39.144218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652538+0000","last_change":"2023-07-06T03:45:10.660900+0000","last_active":"2023-07-06T03:45:13.652538+0000","last_peered":"2023-07-06T03:45:13.652538+0000","last_clean":"2023-07-06T03:45:13.652538+0000","last_became_active":"2023-07-06T03:45:10.660490+0000","last_became_peered":"2023-07-06T03:45:10.660490+0000","last_unstale":"2023-07-06T03:45:13.652538+0000","last_undegraded":"2023-07-06T03:45:13.652538+0000","last_fullsized":"2023-07-06T03:45:13.652538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:43.801501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681363+0000","last_change":"2023-07-06T03:45:11.687991+0000","last_active":"2023-07-06T03:45:12.681363+0000","last_peered":"2023-07-06T03:45:12.681363+0000","last_clean":"2023-07-06T03:45:12.681363+0000","last_became_active":"2023-07-06T03:45:11.685096+0000","last_became_peered":"2023-07-06T03:45:11.685096+0000","last_unstale":"2023-07-06T03:45:12.681363+0000","last_undegraded":"2023-07-06T03:45:12.681363+0000","last_fullsized":"2023-07-06T03:45:12.681363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:29.300537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830169+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:45:13.830169+0000","last_peered":"2023-07-06T03:45:13.830169+0000","last_clean":"2023-07-06T03:45:13.830169+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:45:13.830169+0000","last_undegraded":"2023-07-06T03:45:13.830169+0000","last_fullsized":"2023-07-06T03:45:13.830169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:27:55.564390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.653143+0000","last_change":"2023-07-06T03:45:10.661812+0000","last_active":"2023-07-06T03:45:13.653143+0000","last_peered":"2023-07-06T03:45:13.653143+0000","last_clean":"2023-07-06T03:45:13.653143+0000","last_became_active":"2023-07-06T03:45:10.661499+0000","last_became_peered":"2023-07-06T03:45:10.661499+0000","last_unstale":"2023-07-06T03:45:13.653143+0000","last_undegraded":"2023-07-06T03:45:13.653143+0000","last_fullsized":"2023-07-06T03:45:13.653143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:57:59.089238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652878+0000","last_change":"2023-07-06T03:45:11.678171+0000","last_active":"2023-07-06T03:45:13.652878+0000","last_peered":"2023-07-06T03:45:13.652878+0000","last_clean":"2023-07-06T03:45:13.652878+0000","last_became_active":"2023-07-06T03:45:11.677886+0000","last_became_peered":"2023-07-06T03:45:11.677886+0000","last_unstale":"2023-07-06T03:45:13.652878+0000","last_undegraded":"2023-07-06T03:45:13.652878+0000","last_fullsized":"2023-07-06T03:45:13.652878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.879727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206208+0000","last_change":"2023-07-06T03:45:10.657429+0000","last_active":"2023-07-06T03:45:14.206208+0000","last_peered":"2023-07-06T03:45:14.206208+0000","last_clean":"2023-07-06T03:45:14.206208+0000","last_became_active":"2023-07-06T03:45:10.657085+0000","last_became_peered":"2023-07-06T03:45:10.657085+0000","last_unstale":"2023-07-06T03:45:14.206208+0000","last_undegraded":"2023-07-06T03:45:14.206208+0000","last_fullsized":"2023-07-06T03:45:14.206208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:24:47.052321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206026+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:45:14.206026+0000","last_peered":"2023-07-06T03:45:14.206026+0000","last_clean":"2023-07-06T03:45:14.206026+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:45:14.206026+0000","last_undegraded":"2023-07-06T03:45:14.206026+0000","last_fullsized":"2023-07-06T03:45:14.206026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:05:14.989733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205502+0000","last_change":"2023-07-06T03:45:11.671780+0000","last_active":"2023-07-06T03:45:14.205502+0000","last_peered":"2023-07-06T03:45:14.205502+0000","last_clean":"2023-07-06T03:45:14.205502+0000","last_became_active":"2023-07-06T03:45:11.671390+0000","last_became_peered":"2023-07-06T03:45:11.671390+0000","last_unstale":"2023-07-06T03:45:14.205502+0000","last_undegraded":"2023-07-06T03:45:14.205502+0000","last_fullsized":"2023-07-06T03:45:14.205502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.829963+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:45:13.829963+0000","last_peered":"2023-07-06T03:45:13.829963+0000","last_clean":"2023-07-06T03:45:13.829963+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:45:13.829963+0000","last_undegraded":"2023-07-06T03:45:13.829963+0000","last_fullsized":"2023-07-06T03:45:13.829963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:12.638307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.206183+0000","last_change":"2023-07-06T03:45:10.657912+0000","last_active":"2023-07-06T03:45:14.206183+0000","last_peered":"2023-07-06T03:45:14.206183+0000","last_clean":"2023-07-06T03:45:14.206183+0000","last_became_active":"2023-07-06T03:45:10.657615+0000","last_became_peered":"2023-07-06T03:45:10.657615+0000","last_unstale":"2023-07-06T03:45:14.206183+0000","last_undegraded":"2023-07-06T03:45:14.206183+0000","last_fullsized":"2023-07-06T03:45:14.206183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:30:30.877422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678345+0000","last_change":"2023-07-06T03:45:11.665120+0000","last_active":"2023-07-06T03:45:12.678345+0000","last_peered":"2023-07-06T03:45:12.678345+0000","last_clean":"2023-07-06T03:45:12.678345+0000","last_became_active":"2023-07-06T03:45:11.664817+0000","last_became_peered":"2023-07-06T03:45:11.664817+0000","last_unstale":"2023-07-06T03:45:12.678345+0000","last_undegraded":"2023-07-06T03:45:12.678345+0000","last_fullsized":"2023-07-06T03:45:12.678345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:44.808607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205783+0000","last_change":"2023-07-06T03:45:10.658517+0000","last_active":"2023-07-06T03:45:14.205783+0000","last_peered":"2023-07-06T03:45:14.205783+0000","last_clean":"2023-07-06T03:45:14.205783+0000","last_became_active":"2023-07-06T03:45:10.657946+0000","last_became_peered":"2023-07-06T03:45:10.657946+0000","last_unstale":"2023-07-06T03:45:14.205783+0000","last_undegraded":"2023-07-06T03:45:14.205783+0000","last_fullsized":"2023-07-06T03:45:14.205783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:24:42.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:14.205842+0000","last_change":"2023-07-06T03:45:11.683447+0000","last_active":"2023-07-06T03:45:14.205842+0000","last_peered":"2023-07-06T03:45:14.205842+0000","last_clean":"2023-07-06T03:45:14.205842+0000","last_became_active":"2023-07-06T03:45:11.683156+0000","last_became_peered":"2023-07-06T03:45:11.683156+0000","last_unstale":"2023-07-06T03:45:14.205842+0000","last_undegraded":"2023-07-06T03:45:14.205842+0000","last_fullsized":"2023-07-06T03:45:14.205842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:44.220976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818006+0000","last_change":"2023-07-06T03:45:10.649186+0000","last_active":"2023-07-06T03:45:12.818006+0000","last_peered":"2023-07-06T03:45:12.818006+0000","last_clean":"2023-07-06T03:45:12.818006+0000","last_became_active":"2023-07-06T03:45:10.648707+0000","last_became_peered":"2023-07-06T03:45:10.648707+0000","last_unstale":"2023-07-06T03:45:12.818006+0000","last_undegraded":"2023-07-06T03:45:12.818006+0000","last_fullsized":"2023-07-06T03:45:12.818006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:07.054998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678566+0000","last_change":"2023-07-06T03:45:11.690462+0000","last_active":"2023-07-06T03:45:12.678566+0000","last_peered":"2023-07-06T03:45:12.678566+0000","last_clean":"2023-07-06T03:45:12.678566+0000","last_became_active":"2023-07-06T03:45:11.690181+0000","last_became_peered":"2023-07-06T03:45:11.690181+0000","last_unstale":"2023-07-06T03:45:12.678566+0000","last_undegraded":"2023-07-06T03:45:12.678566+0000","last_fullsized":"2023-07-06T03:45:12.678566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:54:18.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.830291+0000","last_change":"2023-07-06T03:45:10.653276+0000","last_active":"2023-07-06T03:45:13.830291+0000","last_peered":"2023-07-06T03:45:13.830291+0000","last_clean":"2023-07-06T03:45:13.830291+0000","last_became_active":"2023-07-06T03:45:10.652958+0000","last_became_peered":"2023-07-06T03:45:10.652958+0000","last_unstale":"2023-07-06T03:45:13.830291+0000","last_undegraded":"2023-07-06T03:45:13.830291+0000","last_fullsized":"2023-07-06T03:45:13.830291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:45.669370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678497+0000","last_change":"2023-07-06T03:45:11.684391+0000","last_active":"2023-07-06T03:45:12.678497+0000","last_peered":"2023-07-06T03:45:12.678497+0000","last_clean":"2023-07-06T03:45:12.678497+0000","last_became_active":"2023-07-06T03:45:11.684078+0000","last_became_peered":"2023-07-06T03:45:11.684078+0000","last_unstale":"2023-07-06T03:45:12.678497+0000","last_undegraded":"2023-07-06T03:45:12.678497+0000","last_fullsized":"2023-07-06T03:45:12.678497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681347+0000","last_change":"2023-07-06T03:45:10.645296+0000","last_active":"2023-07-06T03:45:12.681347+0000","last_peered":"2023-07-06T03:45:12.681347+0000","last_clean":"2023-07-06T03:45:12.681347+0000","last_became_active":"2023-07-06T03:45:10.644967+0000","last_became_peered":"2023-07-06T03:45:10.644967+0000","last_unstale":"2023-07-06T03:45:12.681347+0000","last_undegraded":"2023-07-06T03:45:12.681347+0000","last_fullsized":"2023-07-06T03:45:12.681347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:12.862578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678143+0000","last_change":"2023-07-06T03:45:11.683562+0000","last_active":"2023-07-06T03:45:12.678143+0000","last_peered":"2023-07-06T03:45:12.678143+0000","last_clean":"2023-07-06T03:45:12.678143+0000","last_became_active":"2023-07-06T03:45:11.683275+0000","last_became_peered":"2023-07-06T03:45:11.683275+0000","last_unstale":"2023-07-06T03:45:12.678143+0000","last_undegraded":"2023-07-06T03:45:12.678143+0000","last_fullsized":"2023-07-06T03:45:12.678143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:44.808607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.681723+0000","last_change":"2023-07-06T03:45:10.654706+0000","last_active":"2023-07-06T03:45:12.681723+0000","last_peered":"2023-07-06T03:45:12.681723+0000","last_clean":"2023-07-06T03:45:12.681723+0000","last_became_active":"2023-07-06T03:45:10.653995+0000","last_became_peered":"2023-07-06T03:45:10.653995+0000","last_unstale":"2023-07-06T03:45:12.681723+0000","last_undegraded":"2023-07-06T03:45:12.681723+0000","last_fullsized":"2023-07-06T03:45:12.681723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:09.617233+0000","last_clean_scrub_stamp":"2023-07-06T03:45:09.617233+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:05:20.493361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.678265+0000","last_change":"2023-07-06T03:45:11.670138+0000","last_active":"2023-07-06T03:45:12.678265+0000","last_peered":"2023-07-06T03:45:12.678265+0000","last_clean":"2023-07-06T03:45:12.678265+0000","last_became_active":"2023-07-06T03:45:11.669695+0000","last_became_peered":"2023-07-06T03:45:11.669695+0000","last_unstale":"2023-07-06T03:45:12.678265+0000","last_undegraded":"2023-07-06T03:45:12.678265+0000","last_fullsized":"2023-07-06T03:45:12.678265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:22:55.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.818277+0000","last_change":"2023-07-06T03:45:11.673123+0000","last_active":"2023-07-06T03:45:12.818277+0000","last_peered":"2023-07-06T03:45:12.818277+0000","last_clean":"2023-07-06T03:45:12.818277+0000","last_became_active":"2023-07-06T03:45:11.672765+0000","last_became_peered":"2023-07-06T03:45:11.672765+0000","last_unstale":"2023-07-06T03:45:12.818277+0000","last_undegraded":"2023-07-06T03:45:12.818277+0000","last_fullsized":"2023-07-06T03:45:12.818277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:17.092196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.652635+0000","last_change":"2023-07-06T03:45:11.689866+0000","last_active":"2023-07-06T03:45:13.652635+0000","last_peered":"2023-07-06T03:45:13.652635+0000","last_clean":"2023-07-06T03:45:13.652635+0000","last_became_active":"2023-07-06T03:45:11.689595+0000","last_became_peered":"2023-07-06T03:45:11.689595+0000","last_unstale":"2023-07-06T03:45:13.652635+0000","last_undegraded":"2023-07-06T03:45:13.652635+0000","last_fullsized":"2023-07-06T03:45:13.652635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:24.552264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:13.827567+0000","last_change":"2023-07-06T03:45:11.675336+0000","last_active":"2023-07-06T03:45:13.827567+0000","last_peered":"2023-07-06T03:45:13.827567+0000","last_clean":"2023-07-06T03:45:13.827567+0000","last_became_active":"2023-07-06T03:45:11.675044+0000","last_became_peered":"2023-07-06T03:45:11.675044+0000","last_unstale":"2023-07-06T03:45:13.827567+0000","last_undegraded":"2023-07-06T03:45:13.827567+0000","last_fullsized":"2023-07-06T03:45:13.827567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680990+0000","last_change":"2023-07-06T03:45:11.689419+0000","last_active":"2023-07-06T03:45:12.680990+0000","last_peered":"2023-07-06T03:45:12.680990+0000","last_clean":"2023-07-06T03:45:12.680990+0000","last_became_active":"2023-07-06T03:45:11.688843+0000","last_became_peered":"2023-07-06T03:45:11.688843+0000","last_unstale":"2023-07-06T03:45:12.680990+0000","last_undegraded":"2023-07-06T03:45:12.680990+0000","last_fullsized":"2023-07-06T03:45:12.680990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:12:15.627811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T03:45:12.680936+0000","last_change":"2023-07-06T03:45:11.687716+0000","last_active":"2023-07-06T03:45:12.680936+0000","last_peered":"2023-07-06T03:45:12.680936+0000","last_clean":"2023-07-06T03:45:12.680936+0000","last_became_active":"2023-07-06T03:45:11.685418+0000","last_became_peered":"2023-07-06T03:45:11.685418+0000","last_unstale":"2023-07-06T03:45:12.680936+0000","last_undegraded":"2023-07-06T03:45:12.680936+0000","last_fullsized":"2023-07-06T03:45:12.680936+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:10.624339+0000","last_clean_scrub_stamp":"2023-07-06T03:45:10.624339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:38.910577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6219999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.77}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":847872,"data_stored":680641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9059999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9790000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0249999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":89745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":91163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.6550000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.911}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.077}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.258}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.4740000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:45:27.125 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-06T03:45:27.126 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-06T03:45:27.126 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-07-06T03:45:27.127 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:45:27.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.345+0000 7f79953ff700 1 Processor -- start 2023-07-06T03:45:27.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.345+0000 7f79953ff700 1 -- start start 2023-07-06T03:45:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.346+0000 7f79953ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f799015c520 0x7f79900cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.346+0000 7f79953ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7990160180 0x7f79901605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.346+0000 7f79953ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79900cdae0 con 0x7f7990160b30 2023-07-06T03:45:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.346+0000 7f79953ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79900cd8d0 con 0x7f7990160180 2023-07-06T03:45:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.346+0000 7f79953ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79900b6640 con 0x7f799015c520 2023-07-06T03:45:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.347+0000 7f798e7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7990160180 0x7f79901605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.347+0000 7f798f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7990160b30 0x7f79900cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35548/0 (socket says 172.21.15.97:35548) 2023-07-06T03:45:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.347+0000 7f798e7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7990160180 0x7f79901605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51804/0 (socket says 172.21.15.97:51804) 2023-07-06T03:45:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.347+0000 7f798f7fe700 1 -- 172.21.15.97:0/892074919 learned_addr learned my addr 172.21.15.97:0/892074919 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.347+0000 7f798effd700 1 --2- 172.21.15.97:0/892074919 >> v2:172.21.15.146:3300/0 conn(0x7f799015c520 0x7f79900cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.347+0000 7f798e7fc700 1 -- 172.21.15.97:0/892074919 >> v2:172.21.15.146:3300/0 conn(0x7f799015c520 msgr2=0x7f79900cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.347+0000 7f798e7fc700 1 --2- 172.21.15.97:0/892074919 >> v2:172.21.15.146:3300/0 conn(0x7f799015c520 0x7f79900cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.347+0000 7f798e7fc700 1 -- 172.21.15.97:0/892074919 >> v1:172.21.15.97:6789/0 conn(0x7f7990160b30 legacy=0x7f79900cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.347+0000 7f798e7fc700 1 -- 172.21.15.97:0/892074919 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79900b6920 con 0x7f7990160180 2023-07-06T03:45:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.347+0000 7f798dffb700 1 -- 172.21.15.97:0/892074919 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082658237 0 0) 0x7f79900cdae0 con 0x7f7990160b30 2023-07-06T03:45:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.347+0000 7f798e7fc700 1 --2- 172.21.15.97:0/892074919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990160180 0x7f79901605f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79900b4c70 tx=0x7f7984009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedcd583ad1c1bb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:27.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.348+0000 7f798dffb700 1 -- 172.21.15.97:0/892074919 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984011830 con 0x7f7990160180 2023-07-06T03:45:27.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.348+0000 7f798dffb700 1 -- 172.21.15.97:0/892074919 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7984011990 con 0x7f7990160180 2023-07-06T03:45:27.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.348+0000 7f798dffb700 1 -- 172.21.15.97:0/892074919 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984011d20 con 0x7f7990160180 2023-07-06T03:45:27.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.348+0000 7f79953ff700 1 -- 172.21.15.97:0/892074919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990160180 msgr2=0x7f79901605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.348+0000 7f79953ff700 1 --2- 172.21.15.97:0/892074919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990160180 0x7f79901605f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79900b4c70 tx=0x7f7984009b20 comp rx=0 tx=0).stop 2023-07-06T03:45:27.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.348+0000 7f79953ff700 1 -- 172.21.15.97:0/892074919 shutdown_connections 2023-07-06T03:45:27.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.348+0000 7f79953ff700 1 --2- 172.21.15.97:0/892074919 >> v2:172.21.15.146:3300/0 conn(0x7f799015c520 0x7f79900cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.348+0000 7f79953ff700 1 --2- 172.21.15.97:0/892074919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990160180 0x7f79901605f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.348+0000 7f79953ff700 1 -- 172.21.15.97:0/892074919 >> 172.21.15.97:0/892074919 conn(0x7f7990071790 msgr2=0x7f7990071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:27.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.348+0000 7f79953ff700 1 -- 172.21.15.97:0/892074919 shutdown_connections 2023-07-06T03:45:27.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.348+0000 7f79953ff700 1 -- 172.21.15.97:0/892074919 wait complete. 2023-07-06T03:45:27.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.350+0000 7f79953ff700 1 Processor -- start 2023-07-06T03:45:27.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.350+0000 7f79953ff700 1 -- start start 2023-07-06T03:45:27.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.351+0000 7f79953ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f799015c520 0x7f799014fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.351+0000 7f79953ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990160180 0x7f79900d2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.351+0000 7f79953ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7990160b30 0x7f799014f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.351+0000 7f79953ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79900d3bd0 con 0x7f7990160180 2023-07-06T03:45:27.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.351+0000 7f79953ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79900d3330 con 0x7f799015c520 2023-07-06T03:45:27.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.351+0000 7f79953ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79900d3610 con 0x7f7990160b30 2023-07-06T03:45:27.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.351+0000 7f798e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990160180 0x7f79900d2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.351+0000 7f798e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990160180 0x7f79900d2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51814/0 (socket says 172.21.15.97:51814) 2023-07-06T03:45:27.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.351+0000 7f798e7fc700 1 -- 172.21.15.97:0/2275197158 learned_addr learned my addr 172.21.15.97:0/2275197158 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:27.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.351+0000 7f798effd700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f799015c520 0x7f799014fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.351+0000 7f798f7fe700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7990160b30 0x7f799014f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.352+0000 7f798e7fc700 1 -- 172.21.15.97:0/2275197158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7990160b30 msgr2=0x7f799014f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.352+0000 7f798e7fc700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7990160b30 0x7f799014f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.352+0000 7f798e7fc700 1 -- 172.21.15.97:0/2275197158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f799015c520 msgr2=0x7f799014fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.352+0000 7f798e7fc700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f799015c520 0x7f799014fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.352+0000 7f798e7fc700 1 -- 172.21.15.97:0/2275197158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79840094a0 con 0x7f7990160180 2023-07-06T03:45:27.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.352+0000 7f798e7fc700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990160180 0x7f79900d2ce0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f79840090b0 tx=0x7f7984004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:27.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.352+0000 7f79777fe700 1 -- 172.21.15.97:0/2275197158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79840038a0 con 0x7f7990160180 2023-07-06T03:45:27.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.352+0000 7f79777fe700 1 -- 172.21.15.97:0/2275197158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798401f070 con 0x7f7990160180 2023-07-06T03:45:27.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.352+0000 7f79777fe700 1 -- 172.21.15.97:0/2275197158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984003c20 con 0x7f7990160180 2023-07-06T03:45:27.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.352+0000 7f79953ff700 1 -- 172.21.15.97:0/2275197158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79900d38f0 con 0x7f7990160180 2023-07-06T03:45:27.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.352+0000 7f79953ff700 1 -- 172.21.15.97:0/2275197158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7990150520 con 0x7f7990160180 2023-07-06T03:45:27.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.353+0000 7f79953ff700 1 -- 172.21.15.97:0/2275197158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79901f5ae0 con 0x7f7990160180 2023-07-06T03:45:27.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.354+0000 7f79777fe700 1 -- 172.21.15.97:0/2275197158 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f798401a430 con 0x7f7990160180 2023-07-06T03:45:27.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.355+0000 7f79777fe700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f797c06ace0 0x7f797c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.355+0000 7f79777fe700 1 -- 172.21.15.97:0/2275197158 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7984021070 con 0x7f7990160180 2023-07-06T03:45:27.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.355+0000 7f798effd700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f797c06ace0 0x7f797c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.356+0000 7f798effd700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f797c06ace0 0x7f797c06d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f79901e66a0 tx=0x7f7980008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:27.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.359+0000 7f79777fe700 1 -- 172.21.15.97:0/2275197158 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f79901f5ae0 con 0x7f7990160180 2023-07-06T03:45:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.596+0000 7f79953ff700 1 -- 172.21.15.97:0/2275197158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f79901f5ae0 con 0x7f7990160180 2023-07-06T03:45:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.597+0000 7f79777fe700 1 -- 172.21.15.97:0/2275197158 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7984015040 con 0x7f7990160180 2023-07-06T03:45:27.599 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:27.599 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:45:27.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 -- 172.21.15.97:0/2275197158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f797c06ace0 msgr2=0x7f797c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f797c06ace0 0x7f797c06d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f79901e66a0 tx=0x7f7980008040 comp rx=0 tx=0).stop 2023-07-06T03:45:27.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 -- 172.21.15.97:0/2275197158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990160180 msgr2=0x7f79900d2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990160180 0x7f79900d2ce0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f79840090b0 tx=0x7f7984004750 comp rx=0 tx=0).stop 2023-07-06T03:45:27.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 -- 172.21.15.97:0/2275197158 shutdown_connections 2023-07-06T03:45:27.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f797c06ace0 0x7f797c06d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7990160b30 0x7f799014f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f799015c520 0x7f799014fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 --2- 172.21.15.97:0/2275197158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990160180 0x7f79900d2ce0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 -- 172.21.15.97:0/2275197158 >> 172.21.15.97:0/2275197158 conn(0x7f7990071790 msgr2=0x7f79901580b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:27.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 -- 172.21.15.97:0/2275197158 shutdown_connections 2023-07-06T03:45:27.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.599+0000 7f79953ff700 1 -- 172.21.15.97:0/2275197158 wait complete. 2023-07-06T03:45:27.618 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-07-06T03:45:27.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:27.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.814+0000 7f5a599ff700 1 Processor -- start 2023-07-06T03:45:27.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.814+0000 7f5a599ff700 1 -- start start 2023-07-06T03:45:27.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.815+0000 7f5a599ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5a540cd390 0x7f5a54160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a599ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a540cdae0 con 0x7f5a5415c520 2023-07-06T03:45:27.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a599ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a540b6640 con 0x7f5a54160dd0 2023-07-06T03:45:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a599ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a540b6920 con 0x7f5a540cd390 2023-07-06T03:45:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a52ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5a5415c520 0x7f5a54160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35564/0 (socket says 172.21.15.97:35564) 2023-07-06T03:45:27.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a52ffd700 1 -- 172.21.15.97:0/293036188 learned_addr learned my addr 172.21.15.97:0/293036188 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:27.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a4bfff700 1 --2- 172.21.15.97:0/293036188 >> v2:172.21.15.146:3300/0 conn(0x7f5a540cd390 0x7f5a54160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93638522 0 0) 0x7f5a540cdae0 con 0x7f5a5415c520 2023-07-06T03:45:27.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a400035b0 con 0x7f5a5415c520 2023-07-06T03:45:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1190584653 0 0) 0x7f5a400035b0 con 0x7f5a5415c520 2023-07-06T03:45:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 >> v2:172.21.15.146:3300/0 conn(0x7f5a540cd390 msgr2=0x7f5a54160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a527fc700 1 --2- 172.21.15.97:0/293036188 >> v2:172.21.15.146:3300/0 conn(0x7f5a540cd390 0x7f5a54160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 >> v1:172.21.15.146:6789/0 conn(0x7f5a54160dd0 legacy=0x7f5a540b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a540b6c00 con 0x7f5a5415c520 2023-07-06T03:45:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f5a44001670 con 0x7f5a5415c520 2023-07-06T03:45:27.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.816+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 >> v1:172.21.15.97:6789/0 conn(0x7f5a5415c520 legacy=0x7f5a54160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a527fc700 1 --2- 172.21.15.97:0/293036188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a40003a50 0x7f5a40003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a527fc700 1 --2- 172.21.15.97:0/293036188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a40004850 0x7f5a40004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a527fc700 1 --2- 172.21.15.97:0/293036188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a40005200 0x7f5a4000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a540b6920 con 0x7f5a40003a50 2023-07-06T03:45:27.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4000c0e0 con 0x7f5a40004850 2023-07-06T03:45:27.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4000c2a0 con 0x7f5a40005200 2023-07-06T03:45:27.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a52ffd700 1 --2- 172.21.15.97:0/293036188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a40003a50 0x7f5a40003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a4bfff700 1 --2- 172.21.15.97:0/293036188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a40004850 0x7f5a40004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a537fe700 1 --2- 172.21.15.97:0/293036188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a40005200 0x7f5a4000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a52ffd700 1 -- 172.21.15.97:0/293036188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a40005200 msgr2=0x7f5a4000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:27.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a52ffd700 1 --2- 172.21.15.97:0/293036188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a40005200 0x7f5a4000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a52ffd700 1 -- 172.21.15.97:0/293036188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a40004850 msgr2=0x7f5a40004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:27.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a52ffd700 1 --2- 172.21.15.97:0/293036188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a40004850 0x7f5a40004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a52ffd700 1 -- 172.21.15.97:0/293036188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a400035b0 con 0x7f5a40003a50 2023-07-06T03:45:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a537fe700 1 -- 172.21.15.97:0/293036188 reap_dead start 2023-07-06T03:45:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a52ffd700 1 --2- 172.21.15.97:0/293036188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a40003a50 0x7f5a40003e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a44001eb0 tx=0x7f5a40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cab5c90f9e85a91 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.817+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44007b70 con 0x7f5a40003a50 2023-07-06T03:45:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.818+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4400ccf0 con 0x7f5a40003a50 2023-07-06T03:45:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.818+0000 7f5a527fc700 1 -- 172.21.15.97:0/293036188 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44005ba0 con 0x7f5a40003a50 2023-07-06T03:45:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.818+0000 7f5a599ff700 1 -- 172.21.15.97:0/293036188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a40003a50 msgr2=0x7f5a40003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.818+0000 7f5a599ff700 1 --2- 172.21.15.97:0/293036188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a40003a50 0x7f5a40003e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a44001eb0 tx=0x7f5a40005880 comp rx=0 tx=0).stop 2023-07-06T03:45:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.818+0000 7f5a599ff700 1 -- 172.21.15.97:0/293036188 shutdown_connections 2023-07-06T03:45:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.818+0000 7f5a599ff700 1 --2- 172.21.15.97:0/293036188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a40003a50 0x7f5a40003e60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.818+0000 7f5a599ff700 1 -- 172.21.15.97:0/293036188 >> 172.21.15.97:0/293036188 conn(0x7f5a54071790 msgr2=0x7f5a54071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.818+0000 7f5a599ff700 1 -- 172.21.15.97:0/293036188 shutdown_connections 2023-07-06T03:45:27.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.818+0000 7f5a599ff700 1 -- 172.21.15.97:0/293036188 wait complete. 2023-07-06T03:45:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.820+0000 7f5a599ff700 1 Processor -- start 2023-07-06T03:45:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.820+0000 7f5a599ff700 1 -- start start 2023-07-06T03:45:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.821+0000 7f5a599ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a40003a50 0x7f5a541f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.821+0000 7f5a599ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a5415de20 0x7f5a541f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.821+0000 7f5a599ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a5415e290 0x7f5a5414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.821+0000 7f5a599ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a541f5b50 con 0x7f5a5415de20 2023-07-06T03:45:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.821+0000 7f5a599ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a541f6aa0 con 0x7f5a40003a50 2023-07-06T03:45:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.821+0000 7f5a599ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a541f3070 con 0x7f5a5415e290 2023-07-06T03:45:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.821+0000 7f5a4bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a5415de20 0x7f5a541f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.821+0000 7f5a4bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a5415de20 0x7f5a541f2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51830/0 (socket says 172.21.15.97:51830) 2023-07-06T03:45:27.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.821+0000 7f5a4bfff700 1 -- 172.21.15.97:0/1058578471 learned_addr learned my addr 172.21.15.97:0/1058578471 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:27.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.821+0000 7f5a52ffd700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a40003a50 0x7f5a541f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.821+0000 7f5a537fe700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a5415e290 0x7f5a5414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.822+0000 7f5a4bfff700 1 -- 172.21.15.97:0/1058578471 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a5415e290 msgr2=0x7f5a5414cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.822+0000 7f5a4bfff700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a5415e290 0x7f5a5414cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.822+0000 7f5a4bfff700 1 -- 172.21.15.97:0/1058578471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a40003a50 msgr2=0x7f5a541f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:27.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.822+0000 7f5a4bfff700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a40003a50 0x7f5a541f6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:27.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.822+0000 7f5a4bfff700 1 -- 172.21.15.97:0/1058578471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a541f3350 con 0x7f5a5415de20 2023-07-06T03:45:27.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.822+0000 7f5a4bfff700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a5415de20 0x7f5a541f2580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c011ad0 tx=0x7f5a3c011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.822+0000 7f5a50ff9700 1 -- 172.21.15.97:0/1058578471 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00f940 con 0x7f5a5415de20 2023-07-06T03:45:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.822+0000 7f5a50ff9700 1 -- 172.21.15.97:0/1058578471 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c00faa0 con 0x7f5a5415de20 2023-07-06T03:45:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.822+0000 7f5a50ff9700 1 -- 172.21.15.97:0/1058578471 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00fe50 con 0x7f5a5415de20 2023-07-06T03:45:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.822+0000 7f5a599ff700 1 -- 172.21.15.97:0/1058578471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a541f5d60 con 0x7f5a5415de20 2023-07-06T03:45:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.822+0000 7f5a599ff700 1 -- 172.21.15.97:0/1058578471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5415b8c0 con 0x7f5a5415de20 2023-07-06T03:45:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.823+0000 7f5a599ff700 1 -- 172.21.15.97:0/1058578471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a541f63b0 con 0x7f5a5415de20 2023-07-06T03:45:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.824+0000 7f5a50ff9700 1 -- 172.21.15.97:0/1058578471 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a3c014440 con 0x7f5a5415de20 2023-07-06T03:45:27.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.825+0000 7f5a50ff9700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a3406ace0 0x7f5a3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:27.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.825+0000 7f5a50ff9700 1 -- 172.21.15.97:0/1058578471 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a3c089a70 con 0x7f5a5415de20 2023-07-06T03:45:27.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.825+0000 7f5a52ffd700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a3406ace0 0x7f5a3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:27.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.826+0000 7f5a52ffd700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a3406ace0 0x7f5a3406d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5a44015040 tx=0x7f5a44007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:27.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:27.830+0000 7f5a50ff9700 1 -- 172.21.15.97:0/1058578471 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a3c054bd0 con 0x7f5a5415de20 2023-07-06T03:45:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.025+0000 7f5a599ff700 1 -- 172.21.15.97:0/1058578471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a54156750 con 0x7f5a5415de20 2023-07-06T03:45:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.026+0000 7f5a50ff9700 1 -- 172.21.15.97:0/1058578471 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f5a3c022090 con 0x7f5a5415de20 2023-07-06T03:45:28.028 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:28.028 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:28.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 -- 172.21.15.97:0/1058578471 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a3406ace0 msgr2=0x7f5a3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a3406ace0 0x7f5a3406d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5a44015040 tx=0x7f5a44007000 comp rx=0 tx=0).stop 2023-07-06T03:45:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 -- 172.21.15.97:0/1058578471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a5415de20 msgr2=0x7f5a541f2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a5415de20 0x7f5a541f2580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c011ad0 tx=0x7f5a3c011de0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 -- 172.21.15.97:0/1058578471 shutdown_connections 2023-07-06T03:45:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a3406ace0 0x7f5a3406d190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a5415e290 0x7f5a5414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a40003a50 0x7f5a541f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 --2- 172.21.15.97:0/1058578471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a5415de20 0x7f5a541f2580 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 -- 172.21.15.97:0/1058578471 >> 172.21.15.97:0/1058578471 conn(0x7f5a54071790 msgr2=0x7f5a540d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:28.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 -- 172.21.15.97:0/1058578471 shutdown_connections 2023-07-06T03:45:28.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.029+0000 7f5a599ff700 1 -- 172.21.15.97:0/1058578471 wait complete. 2023-07-06T03:45:28.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:45:28.049 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:45:28.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.244+0000 7fb186dff700 1 Processor -- start 2023-07-06T03:45:28.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.245+0000 7fb186dff700 1 -- start start 2023-07-06T03:45:28.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.246+0000 7fb186dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb18015c610 0x7fb1800b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.246+0000 7fb186dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18006b360 0x7fb1800b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.246+0000 7fb186dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1800cff90 con 0x7fb1800d2210 2023-07-06T03:45:28.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.246+0000 7fb186dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1800d0780 con 0x7fb18015c610 2023-07-06T03:45:28.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.246+0000 7fb186dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18015c9e0 con 0x7fb18006b360 2023-07-06T03:45:28.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.246+0000 7fb17ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18006b360 0x7fb1800b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.246+0000 7fb17ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18006b360 0x7fb1800b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54344/0 (socket says 172.21.15.97:54344) 2023-07-06T03:45:28.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.246+0000 7fb184b9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb18015c610 0x7fb1800b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.246+0000 7fb17ffff700 1 -- 172.21.15.97:0/3412120010 learned_addr learned my addr 172.21.15.97:0/3412120010 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:28.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.246+0000 7fb18539c700 1 --1- 172.21.15.97:0/3412120010 >> v1:172.21.15.146:6789/0 conn(0x7fb1800d2210 0x7fb1800d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:35388/0 (socket says 172.21.15.97:35388) 2023-07-06T03:45:28.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.247+0000 7fb17ffff700 1 -- 172.21.15.97:0/3412120010 >> v2:172.21.15.146:3300/0 conn(0x7fb18015c610 msgr2=0x7fb1800b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:28.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.247+0000 7fb17ffff700 1 --2- 172.21.15.97:0/3412120010 >> v2:172.21.15.146:3300/0 conn(0x7fb18015c610 0x7fb1800b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.247+0000 7fb17ffff700 1 -- 172.21.15.97:0/3412120010 >> v1:172.21.15.146:6789/0 conn(0x7fb1800d2210 legacy=0x7fb1800d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.247+0000 7fb17ffff700 1 -- 172.21.15.97:0/3412120010 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1800b5650 con 0x7fb18006b360 2023-07-06T03:45:28.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.247+0000 7fb184b9b700 1 --2- 172.21.15.97:0/3412120010 >> v2:172.21.15.146:3300/0 conn(0x7fb18015c610 0x7fb1800b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:28.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.247+0000 7fb17ffff700 1 --2- 172.21.15.97:0/3412120010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18006b360 0x7fb1800b4cf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0098f0 tx=0x7fb16c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0026951bc9a7b80 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:28.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.247+0000 7fb17f7fe700 1 -- 172.21.15.97:0/3412120010 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c011830 con 0x7fb18006b360 2023-07-06T03:45:28.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.247+0000 7fb17f7fe700 1 -- 172.21.15.97:0/3412120010 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16c011990 con 0x7fb18006b360 2023-07-06T03:45:28.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.247+0000 7fb17f7fe700 1 -- 172.21.15.97:0/3412120010 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c011d20 con 0x7fb18006b360 2023-07-06T03:45:28.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.248+0000 7fb186dff700 1 -- 172.21.15.97:0/3412120010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18006b360 msgr2=0x7fb1800b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.248+0000 7fb186dff700 1 --2- 172.21.15.97:0/3412120010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18006b360 0x7fb1800b4cf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0098f0 tx=0x7fb16c009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:28.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.248+0000 7fb186dff700 1 -- 172.21.15.97:0/3412120010 shutdown_connections 2023-07-06T03:45:28.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.248+0000 7fb186dff700 1 --2- 172.21.15.97:0/3412120010 >> v2:172.21.15.146:3300/0 conn(0x7fb18015c610 0x7fb1800b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.248+0000 7fb186dff700 1 --2- 172.21.15.97:0/3412120010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18006b360 0x7fb1800b4cf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.248+0000 7fb186dff700 1 -- 172.21.15.97:0/3412120010 >> 172.21.15.97:0/3412120010 conn(0x7fb18006a6c0 msgr2=0x7fb18006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:28.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.248+0000 7fb186dff700 1 -- 172.21.15.97:0/3412120010 shutdown_connections 2023-07-06T03:45:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.248+0000 7fb186dff700 1 -- 172.21.15.97:0/3412120010 wait complete. 2023-07-06T03:45:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.250+0000 7fb186dff700 1 Processor -- start 2023-07-06T03:45:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.250+0000 7fb186dff700 1 -- start start 2023-07-06T03:45:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb186dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb18006b360 0x7fb1800c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb186dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1800d2210 0x7fb1800b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb186dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18015c610 0x7fb1800c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb186dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800cae60 con 0x7fb18006b360 2023-07-06T03:45:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb186dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800ca8a0 con 0x7fb1800d2210 2023-07-06T03:45:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb186dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800cab80 con 0x7fb18015c610 2023-07-06T03:45:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb184b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb18006b360 0x7fb1800c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb184b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb18006b360 0x7fb1800c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51836/0 (socket says 172.21.15.97:51836) 2023-07-06T03:45:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb184b9b700 1 -- 172.21.15.97:0/3224980808 learned_addr learned my addr 172.21.15.97:0/3224980808 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb18539c700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18015c610 0x7fb1800c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb184b9b700 1 -- 172.21.15.97:0/3224980808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18015c610 msgr2=0x7fb1800c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb17ffff700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1800d2210 0x7fb1800b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb184b9b700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18015c610 0x7fb1800c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb184b9b700 1 -- 172.21.15.97:0/3224980808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1800d2210 msgr2=0x7fb1800b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb184b9b700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1800d2210 0x7fb1800b3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.251+0000 7fb184b9b700 1 -- 172.21.15.97:0/3224980808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1740094a0 con 0x7fb18006b360 2023-07-06T03:45:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.252+0000 7fb184b9b700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb18006b360 0x7fb1800c6860 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb17400ebe0 tx=0x7fb174009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.252+0000 7fb17d7fa700 1 -- 172.21.15.97:0/3224980808 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17400c910 con 0x7fb18006b360 2023-07-06T03:45:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.252+0000 7fb17d7fa700 1 -- 172.21.15.97:0/3224980808 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17400ca70 con 0x7fb18006b360 2023-07-06T03:45:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.252+0000 7fb186dff700 1 -- 172.21.15.97:0/3224980808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c009550 con 0x7fb18006b360 2023-07-06T03:45:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.252+0000 7fb17d7fa700 1 -- 172.21.15.97:0/3224980808 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17400cde0 con 0x7fb18006b360 2023-07-06T03:45:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.252+0000 7fb186dff700 1 -- 172.21.15.97:0/3224980808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1800cb8f0 con 0x7fb18006b360 2023-07-06T03:45:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.253+0000 7fb186dff700 1 -- 172.21.15.97:0/3224980808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1800cb470 con 0x7fb18006b360 2023-07-06T03:45:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.254+0000 7fb17d7fa700 1 -- 172.21.15.97:0/3224980808 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb174013470 con 0x7fb18006b360 2023-07-06T03:45:28.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.255+0000 7fb17d7fa700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb16406ace0 0x7fb16406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.255+0000 7fb17d7fa700 1 -- 172.21.15.97:0/3224980808 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb1740897c0 con 0x7fb18006b360 2023-07-06T03:45:28.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.255+0000 7fb17ffff700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb16406ace0 0x7fb16406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.256+0000 7fb17ffff700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb16406ace0 0x7fb16406d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1800c9920 tx=0x7fb16c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:28.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.259+0000 7fb17d7fa700 1 -- 172.21.15.97:0/3224980808 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb1800cb470 con 0x7fb18006b360 2023-07-06T03:45:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.455+0000 7fb186dff700 1 -- 172.21.15.97:0/3224980808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1800cb470 con 0x7fb18006b360 2023-07-06T03:45:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.455+0000 7fb17d7fa700 1 -- 172.21.15.97:0/3224980808 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1800cb470 con 0x7fb18006b360 2023-07-06T03:45:28.457 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:45:28.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 -- 172.21.15.97:0/3224980808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb16406ace0 msgr2=0x7fb16406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb16406ace0 0x7fb16406d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1800c9920 tx=0x7fb16c008030 comp rx=0 tx=0).stop 2023-07-06T03:45:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 -- 172.21.15.97:0/3224980808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb18006b360 msgr2=0x7fb1800c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb18006b360 0x7fb1800c6860 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb17400ebe0 tx=0x7fb174009ec0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 -- 172.21.15.97:0/3224980808 shutdown_connections 2023-07-06T03:45:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb16406ace0 0x7fb16406d190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb18015c610 0x7fb1800c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb18006b360 0x7fb1800c6860 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 --2- 172.21.15.97:0/3224980808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1800d2210 0x7fb1800b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 -- 172.21.15.97:0/3224980808 >> 172.21.15.97:0/3224980808 conn(0x7fb18006a6c0 msgr2=0x7fb180159680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:28.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 -- 172.21.15.97:0/3224980808 shutdown_connections 2023-07-06T03:45:28.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.458+0000 7fb186dff700 1 -- 172.21.15.97:0/3224980808 wait complete. 2023-07-06T03:45:28.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:45:28.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.674+0000 7f175b1ff700 1 Processor -- start 2023-07-06T03:45:28.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.674+0000 7f175b1ff700 1 -- start start 2023-07-06T03:45:28.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.675+0000 7f175b1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f175415c610 0x7f17540b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.675+0000 7f175b1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f17540d2210 0x7f17540d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.675+0000 7f175b1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17540cff90 con 0x7f175406b360 2023-07-06T03:45:28.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.675+0000 7f175b1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17540d0780 con 0x7f17540d2210 2023-07-06T03:45:28.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.675+0000 7f175b1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175415c9e0 con 0x7f175415c610 2023-07-06T03:45:28.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.675+0000 7f1753fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f175406b360 0x7f17540b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35580/0 (socket says 172.21.15.97:35580) 2023-07-06T03:45:28.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.675+0000 7f1753fff700 1 -- 172.21.15.97:0/1928796058 learned_addr learned my addr 172.21.15.97:0/1928796058 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:28.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.675+0000 7f1758f9b700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f175415c610 0x7f17540b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.675+0000 7f175979c700 1 --2- 172.21.15.97:0/1928796058 >> v2:172.21.15.146:3300/0 conn(0x7f17540d2210 0x7f17540d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1776048972 0 0) 0x7f17540cff90 con 0x7f175406b360 2023-07-06T03:45:28.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17440035b0 con 0x7f175406b360 2023-07-06T03:45:28.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1677534010 0 0) 0x7f17440035b0 con 0x7f175406b360 2023-07-06T03:45:28.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f175415c610 msgr2=0x7f17540b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:28.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f175415c610 0x7f17540b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 >> v2:172.21.15.146:3300/0 conn(0x7f17540d2210 msgr2=0x7f17540d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 --2- 172.21.15.97:0/1928796058 >> v2:172.21.15.146:3300/0 conn(0x7f17540d2210 0x7f17540d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17540b5650 con 0x7f175406b360 2023-07-06T03:45:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f1740001670 con 0x7f175406b360 2023-07-06T03:45:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 >> v1:172.21.15.97:6789/0 conn(0x7f175406b360 legacy=0x7f17540b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1744003a30 0x7f1744003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1744004410 0x7f17440048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004f40 0x7f174400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17540d0780 con 0x7f1744004f40 2023-07-06T03:45:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175415c9e0 con 0x7f1744004410 2023-07-06T03:45:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174400bf50 con 0x7f1744003a30 2023-07-06T03:45:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f175979c700 1 --2- 172.21.15.97:0/1928796058 >> v2:172.21.15.146:3300/0 conn(0x7f17540d2210 0x7f17540d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:28.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f1758f9b700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f175415c610 0x7f17540b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:28.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f175979c700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004f40 0x7f174400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f1758f9b700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1744003a30 0x7f1744003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.676+0000 7f1753fff700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1744004410 0x7f17440048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.677+0000 7f175979c700 1 -- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1744003a30 msgr2=0x7f1744003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:28.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.677+0000 7f175979c700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1744003a30 0x7f1744003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.677+0000 7f175979c700 1 -- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1744004410 msgr2=0x7f17440048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:28.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.677+0000 7f175979c700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1744004410 0x7f17440048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.677+0000 7f175979c700 1 -- 172.21.15.97:0/1928796058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17440035b0 con 0x7f1744004f40 2023-07-06T03:45:28.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.677+0000 7f1758f9b700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1744003a30 0x7f1744003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:28.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.677+0000 7f175979c700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004f40 0x7f174400ba10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f174c00e380 tx=0x7f1744005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdddb32c54c966e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:28.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.677+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c00ee40 con 0x7f1744004f40 2023-07-06T03:45:28.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.677+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c00f040 con 0x7f1744004f40 2023-07-06T03:45:28.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.677+0000 7f17537fe700 1 -- 172.21.15.97:0/1928796058 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c003c40 con 0x7f1744004f40 2023-07-06T03:45:28.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.678+0000 7f175b1ff700 1 -- 172.21.15.97:0/1928796058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004f40 msgr2=0x7f174400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.678+0000 7f175b1ff700 1 --2- 172.21.15.97:0/1928796058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1744004f40 0x7f174400ba10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f174c00e380 tx=0x7f1744005880 comp rx=0 tx=0).stop 2023-07-06T03:45:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.678+0000 7f175979c700 1 -- 172.21.15.97:0/1928796058 reap_dead start 2023-07-06T03:45:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.678+0000 7f175b1ff700 1 -- 172.21.15.97:0/1928796058 shutdown_connections 2023-07-06T03:45:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.678+0000 7f175b1ff700 1 -- 172.21.15.97:0/1928796058 >> 172.21.15.97:0/1928796058 conn(0x7f175406a6c0 msgr2=0x7f175406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.678+0000 7f175b1ff700 1 -- 172.21.15.97:0/1928796058 shutdown_connections 2023-07-06T03:45:28.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.678+0000 7f175b1ff700 1 -- 172.21.15.97:0/1928796058 wait complete. 2023-07-06T03:45:28.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.680+0000 7f175b1ff700 1 Processor -- start 2023-07-06T03:45:28.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.680+0000 7f175b1ff700 1 -- start start 2023-07-06T03:45:28.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f175b1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1744004f40 0x7f175414eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f175b1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f175414f130 0x7f17540b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f175b1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17540b83a0 0x7f17540b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f175b1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17540b7970 con 0x7f175414f130 2023-07-06T03:45:28.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f175b1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175415ddf0 con 0x7f17540b83a0 2023-07-06T03:45:28.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f175b1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17541f5690 con 0x7f1744004f40 2023-07-06T03:45:28.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f1753fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f175414f130 0x7f17540b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f1753fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f175414f130 0x7f17540b7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51856/0 (socket says 172.21.15.97:51856) 2023-07-06T03:45:28.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f1753fff700 1 -- 172.21.15.97:0/3766343598 learned_addr learned my addr 172.21.15.97:0/3766343598 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:28.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f1758f9b700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1744004f40 0x7f175414eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f175979c700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17540b83a0 0x7f17540b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f1753fff700 1 -- 172.21.15.97:0/3766343598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1744004f40 msgr2=0x7f175414eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f1753fff700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1744004f40 0x7f175414eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f1753fff700 1 -- 172.21.15.97:0/3766343598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17540b83a0 msgr2=0x7f17540b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f1753fff700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17540b83a0 0x7f17540b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.681+0000 7f1753fff700 1 -- 172.21.15.97:0/3766343598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175415d4e0 con 0x7f175414f130 2023-07-06T03:45:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.682+0000 7f1753fff700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f175414f130 0x7f17540b7e60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f174000e8f0 tx=0x7f174000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.682+0000 7f17517fa700 1 -- 172.21.15.97:0/3766343598 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740012590 con 0x7f175414f130 2023-07-06T03:45:28.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.682+0000 7f17517fa700 1 -- 172.21.15.97:0/3766343598 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17400126f0 con 0x7f175414f130 2023-07-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.682+0000 7f17517fa700 1 -- 172.21.15.97:0/3766343598 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740012aa0 con 0x7f175414f130 2023-07-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.682+0000 7f175b1ff700 1 -- 172.21.15.97:0/3766343598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17540b88d0 con 0x7f175414f130 2023-07-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.682+0000 7f175b1ff700 1 -- 172.21.15.97:0/3766343598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175415da10 con 0x7f175414f130 2023-07-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.683+0000 7f175b1ff700 1 -- 172.21.15.97:0/3766343598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175415df30 con 0x7f175414f130 2023-07-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.684+0000 7f17517fa700 1 -- 172.21.15.97:0/3766343598 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1740012c00 con 0x7f175414f130 2023-07-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.685+0000 7f17517fa700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f173806ace0 0x7f173806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.685+0000 7f17517fa700 1 -- 172.21.15.97:0/3766343598 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1740089190 con 0x7f175414f130 2023-07-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.685+0000 7f1758f9b700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f173806ace0 0x7f173806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.686+0000 7f1758f9b700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f173806ace0 0x7f173806d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1748009470 tx=0x7f1748008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:28.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.689+0000 7f17517fa700 1 -- 172.21.15.97:0/3766343598 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f175415df30 con 0x7f175414f130 2023-07-06T03:45:28.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.885+0000 7f175b1ff700 1 -- 172.21.15.97:0/3766343598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f175415df30 con 0x7f175414f130 2023-07-06T03:45:28.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.886+0000 7f17517fa700 1 -- 172.21.15.97:0/3766343598 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f175415df30 con 0x7f175414f130 2023-07-06T03:45:28.888 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:45:28.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 -- 172.21.15.97:0/3766343598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f173806ace0 msgr2=0x7f173806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f173806ace0 0x7f173806d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1748009470 tx=0x7f1748008040 comp rx=0 tx=0).stop 2023-07-06T03:45:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 -- 172.21.15.97:0/3766343598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f175414f130 msgr2=0x7f17540b7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f175414f130 0x7f17540b7e60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f174000e8f0 tx=0x7f174000ec00 comp rx=0 tx=0).stop 2023-07-06T03:45:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 -- 172.21.15.97:0/3766343598 shutdown_connections 2023-07-06T03:45:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f173806ace0 0x7f173806d190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17540b83a0 0x7f17540b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1744004f40 0x7f175414eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 --2- 172.21.15.97:0/3766343598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f175414f130 0x7f17540b7e60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 -- 172.21.15.97:0/3766343598 >> 172.21.15.97:0/3766343598 conn(0x7f175406a6c0 msgr2=0x7f175415b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 -- 172.21.15.97:0/3766343598 shutdown_connections 2023-07-06T03:45:28.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:28.888+0000 7f175b1ff700 1 -- 172.21.15.97:0/3766343598 wait complete. 2023-07-06T03:45:28.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:45:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.103+0000 7fb95b1ff700 1 Processor -- start 2023-07-06T03:45:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.104+0000 7fb95b1ff700 1 -- start start 2023-07-06T03:45:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95b1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb95415c610 0x7fb9540b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95b1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb95406b360 0x7fb9540b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95b1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9540d2210 0x7fb9540d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95b1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9540d0780 con 0x7fb9540d2210 2023-07-06T03:45:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95b1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95415c9e0 con 0x7fb95415c610 2023-07-06T03:45:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95b1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540b5650 con 0x7fb95406b360 2023-07-06T03:45:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95979c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9540d2210 0x7fb9540d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95979c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9540d2210 0x7fb9540d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51862/0 (socket says 172.21.15.97:51862) 2023-07-06T03:45:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95979c700 1 -- 172.21.15.97:0/3587191266 learned_addr learned my addr 172.21.15.97:0/3587191266 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:29.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb953fff700 1 --2- 172.21.15.97:0/3587191266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb95406b360 0x7fb9540b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb958f9b700 1 --2- 172.21.15.97:0/3587191266 >> v2:172.21.15.146:3300/0 conn(0x7fb95415c610 0x7fb9540b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95979c700 1 -- 172.21.15.97:0/3587191266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb95406b360 msgr2=0x7fb9540b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:29.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95979c700 1 --2- 172.21.15.97:0/3587191266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb95406b360 0x7fb9540b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95979c700 1 -- 172.21.15.97:0/3587191266 >> v2:172.21.15.146:3300/0 conn(0x7fb95415c610 msgr2=0x7fb9540b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:29.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95979c700 1 --2- 172.21.15.97:0/3587191266 >> v2:172.21.15.146:3300/0 conn(0x7fb95415c610 0x7fb9540b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.105+0000 7fb95979c700 1 -- 172.21.15.97:0/3587191266 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9540d01c0 con 0x7fb9540d2210 2023-07-06T03:45:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.106+0000 7fb95979c700 1 --2- 172.21.15.97:0/3587191266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9540d2210 0x7fb9540d1310 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb95408b000 tx=0x7fb94c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aaf818f060903f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.106+0000 7fb9537fe700 1 -- 172.21.15.97:0/3587191266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00e070 con 0x7fb9540d2210 2023-07-06T03:45:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.106+0000 7fb9537fe700 1 -- 172.21.15.97:0/3587191266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c00b980 con 0x7fb9540d2210 2023-07-06T03:45:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.106+0000 7fb9537fe700 1 -- 172.21.15.97:0/3587191266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00bd30 con 0x7fb9540d2210 2023-07-06T03:45:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.106+0000 7fb95b1ff700 1 -- 172.21.15.97:0/3587191266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9540d2210 msgr2=0x7fb9540d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.106+0000 7fb95b1ff700 1 --2- 172.21.15.97:0/3587191266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9540d2210 0x7fb9540d1310 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb95408b000 tx=0x7fb94c00b330 comp rx=0 tx=0).stop 2023-07-06T03:45:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.107+0000 7fb95b1ff700 1 -- 172.21.15.97:0/3587191266 shutdown_connections 2023-07-06T03:45:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.107+0000 7fb95b1ff700 1 --2- 172.21.15.97:0/3587191266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9540d2210 0x7fb9540d1310 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.107+0000 7fb95b1ff700 1 --2- 172.21.15.97:0/3587191266 >> v2:172.21.15.146:3300/0 conn(0x7fb95415c610 0x7fb9540b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.107+0000 7fb95b1ff700 1 --2- 172.21.15.97:0/3587191266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb95406b360 0x7fb9540b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.107+0000 7fb95b1ff700 1 -- 172.21.15.97:0/3587191266 >> 172.21.15.97:0/3587191266 conn(0x7fb95406a6c0 msgr2=0x7fb95406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.107+0000 7fb95b1ff700 1 -- 172.21.15.97:0/3587191266 shutdown_connections 2023-07-06T03:45:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.107+0000 7fb95b1ff700 1 -- 172.21.15.97:0/3587191266 wait complete. 2023-07-06T03:45:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.109+0000 7fb95b1ff700 1 Processor -- start 2023-07-06T03:45:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.109+0000 7fb95b1ff700 1 -- start start 2023-07-06T03:45:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb95b1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95406b360 0x7fb954153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb95b1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9540d2210 0x7fb954153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb95b1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb95415c610 0x7fb9540cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb95b1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb954153ac0 con 0x7fb95406b360 2023-07-06T03:45:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb95b1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540cbb70 con 0x7fb9540d2210 2023-07-06T03:45:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb95b1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540cbe50 con 0x7fb95415c610 2023-07-06T03:45:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb958f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95406b360 0x7fb954153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb958f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95406b360 0x7fb954153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51876/0 (socket says 172.21.15.97:51876) 2023-07-06T03:45:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb958f9b700 1 -- 172.21.15.97:0/2788975358 learned_addr learned my addr 172.21.15.97:0/2788975358 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb958f9b700 1 -- 172.21.15.97:0/2788975358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb95415c610 msgr2=0x7fb9540cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb958f9b700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb95415c610 0x7fb9540cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb958f9b700 1 -- 172.21.15.97:0/2788975358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9540d2210 msgr2=0x7fb954153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb958f9b700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9540d2210 0x7fb954153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.110+0000 7fb958f9b700 1 -- 172.21.15.97:0/2788975358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c009b30 con 0x7fb95406b360 2023-07-06T03:45:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.111+0000 7fb958f9b700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95406b360 0x7fb954153cf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d800 tx=0x7fb94800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.111+0000 7fb9517fa700 1 -- 172.21.15.97:0/2788975358 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9480125f0 con 0x7fb95406b360 2023-07-06T03:45:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.111+0000 7fb9517fa700 1 -- 172.21.15.97:0/2788975358 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb948012750 con 0x7fb95406b360 2023-07-06T03:45:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.111+0000 7fb9517fa700 1 -- 172.21.15.97:0/2788975358 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948012b00 con 0x7fb95406b360 2023-07-06T03:45:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.111+0000 7fb95b1ff700 1 -- 172.21.15.97:0/2788975358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9540ca9b0 con 0x7fb95406b360 2023-07-06T03:45:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.111+0000 7fb95b1ff700 1 -- 172.21.15.97:0/2788975358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9540cb6f0 con 0x7fb95406b360 2023-07-06T03:45:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.112+0000 7fb95b1ff700 1 -- 172.21.15.97:0/2788975358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9540ca810 con 0x7fb95406b360 2023-07-06T03:45:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.113+0000 7fb9517fa700 1 -- 172.21.15.97:0/2788975358 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb948012c60 con 0x7fb95406b360 2023-07-06T03:45:29.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.114+0000 7fb9517fa700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb93806ace0 0x7fb93806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.114+0000 7fb9517fa700 1 -- 172.21.15.97:0/2788975358 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb948089860 con 0x7fb95406b360 2023-07-06T03:45:29.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.114+0000 7fb953fff700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb93806ace0 0x7fb93806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.115+0000 7fb953fff700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb93806ace0 0x7fb93806d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9541f51c0 tx=0x7fb940008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:29.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.118+0000 7fb9517fa700 1 -- 172.21.15.97:0/2788975358 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb9540ca810 con 0x7fb95406b360 2023-07-06T03:45:29.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.316+0000 7fb95b1ff700 1 -- 172.21.15.97:0/2788975358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9540ca810 con 0x7fb95406b360 2023-07-06T03:45:29.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.316+0000 7fb9517fa700 1 -- 172.21.15.97:0/2788975358 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9540ca810 con 0x7fb95406b360 2023-07-06T03:45:29.319 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:45:29.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 -- 172.21.15.97:0/2788975358 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb93806ace0 msgr2=0x7fb93806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb93806ace0 0x7fb93806d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9541f51c0 tx=0x7fb940008040 comp rx=0 tx=0).stop 2023-07-06T03:45:29.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 -- 172.21.15.97:0/2788975358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95406b360 msgr2=0x7fb954153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95406b360 0x7fb954153cf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d800 tx=0x7fb94800dbc0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 -- 172.21.15.97:0/2788975358 shutdown_connections 2023-07-06T03:45:29.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb93806ace0 0x7fb93806d190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb95415c610 0x7fb9540cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95406b360 0x7fb954153cf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 --2- 172.21.15.97:0/2788975358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9540d2210 0x7fb954153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 -- 172.21.15.97:0/2788975358 >> 172.21.15.97:0/2788975358 conn(0x7fb95406a6c0 msgr2=0x7fb9541599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:29.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 -- 172.21.15.97:0/2788975358 shutdown_connections 2023-07-06T03:45:29.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.319+0000 7fb95b1ff700 1 -- 172.21.15.97:0/2788975358 wait complete. 2023-07-06T03:45:29.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:45:29.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.535+0000 7f7345bff700 1 Processor -- start 2023-07-06T03:45:29.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.536+0000 7f7345bff700 1 -- start start 2023-07-06T03:45:29.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.537+0000 7f7345bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734006b360 0x7f73400b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.537+0000 7f7345bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73400cff90 con 0x7f734015c610 2023-07-06T03:45:29.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.538+0000 7f7345bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73400b5650 con 0x7f73400d2210 2023-07-06T03:45:29.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.538+0000 7f7345bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73400d0780 con 0x7f734006b360 2023-07-06T03:45:29.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.538+0000 7f733f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f734015c610 0x7f73400b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35588/0 (socket says 172.21.15.97:35588) 2023-07-06T03:45:29.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.538+0000 7f733f7fe700 1 -- 172.21.15.97:0/246654680 learned_addr learned my addr 172.21.15.97:0/246654680 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:29.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.538+0000 7f7337fff700 1 --2- 172.21.15.97:0/246654680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734006b360 0x7f73400b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.538+0000 7f733effd700 1 -- 172.21.15.97:0/246654680 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178891720 0 0) 0x7f73400cff90 con 0x7f734015c610 2023-07-06T03:45:29.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.538+0000 7f733effd700 1 -- 172.21.15.97:0/246654680 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f732c0035b0 con 0x7f734015c610 2023-07-06T03:45:29.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.538+0000 7f7337fff700 1 -- 172.21.15.97:0/246654680 >> v1:172.21.15.146:6789/0 conn(0x7f73400d2210 legacy=0x7f73400d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.538+0000 7f733effd700 1 -- 172.21.15.97:0/246654680 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668758096 0 0) 0x7f73400b5650 con 0x7f73400d2210 2023-07-06T03:45:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.538+0000 7f7337fff700 1 -- 172.21.15.97:0/246654680 >> v1:172.21.15.97:6789/0 conn(0x7f734015c610 legacy=0x7f73400b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.538+0000 7f7337fff700 1 -- 172.21.15.97:0/246654680 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73400b5400 con 0x7f734006b360 2023-07-06T03:45:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.539+0000 7f7337fff700 1 --2- 172.21.15.97:0/246654680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734006b360 0x7f73400b4ec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7328009870 tx=0x7f7328009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c4710edca4c845 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.539+0000 7f733effd700 1 -- 172.21.15.97:0/246654680 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328011830 con 0x7f734006b360 2023-07-06T03:45:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.539+0000 7f733effd700 1 -- 172.21.15.97:0/246654680 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7328011990 con 0x7f734006b360 2023-07-06T03:45:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.539+0000 7f733effd700 1 -- 172.21.15.97:0/246654680 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328011d20 con 0x7f734006b360 2023-07-06T03:45:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.540+0000 7f7345bff700 1 -- 172.21.15.97:0/246654680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734006b360 msgr2=0x7f73400b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.540+0000 7f7345bff700 1 --2- 172.21.15.97:0/246654680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734006b360 0x7f73400b4ec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7328009870 tx=0x7f7328009b80 comp rx=0 tx=0).stop 2023-07-06T03:45:29.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.540+0000 7f7345bff700 1 -- 172.21.15.97:0/246654680 shutdown_connections 2023-07-06T03:45:29.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.540+0000 7f7345bff700 1 --2- 172.21.15.97:0/246654680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734006b360 0x7f73400b4ec0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.540+0000 7f7345bff700 1 -- 172.21.15.97:0/246654680 >> 172.21.15.97:0/246654680 conn(0x7f734006a6c0 msgr2=0x7f734006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:29.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.540+0000 7f7345bff700 1 -- 172.21.15.97:0/246654680 shutdown_connections 2023-07-06T03:45:29.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.540+0000 7f7345bff700 1 -- 172.21.15.97:0/246654680 wait complete. 2023-07-06T03:45:29.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.542+0000 7f7345bff700 1 Processor -- start 2023-07-06T03:45:29.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.542+0000 7f7345bff700 1 -- start start 2023-07-06T03:45:29.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.543+0000 7f7345bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734006b360 0x7f73400c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.543+0000 7f7345bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73400d2210 0x7f73400c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.543+0000 7f7345bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f734015c610 0x7f734015e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.543+0000 7f7345bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734015dbb0 con 0x7f73400d2210 2023-07-06T03:45:29.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.543+0000 7f7345bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73400c9420 con 0x7f734015c610 2023-07-06T03:45:29.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.543+0000 7f7345bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73400cbb90 con 0x7f734006b360 2023-07-06T03:45:29.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.544+0000 7f7337fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73400d2210 0x7f73400c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.544+0000 7f7337fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73400d2210 0x7f73400c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51892/0 (socket says 172.21.15.97:51892) 2023-07-06T03:45:29.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.544+0000 7f7337fff700 1 -- 172.21.15.97:0/3094743366 learned_addr learned my addr 172.21.15.97:0/3094743366 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:29.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.544+0000 7f733f7fe700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734006b360 0x7f73400c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.544+0000 7f733ffff700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f734015c610 0x7f734015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.544+0000 7f7337fff700 1 -- 172.21.15.97:0/3094743366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734006b360 msgr2=0x7f73400c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.544+0000 7f7337fff700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734006b360 0x7f73400c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.544+0000 7f7337fff700 1 -- 172.21.15.97:0/3094743366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f734015c610 msgr2=0x7f734015e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.544+0000 7f7337fff700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f734015c610 0x7f734015e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.544+0000 7f7337fff700 1 -- 172.21.15.97:0/3094743366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73280094a0 con 0x7f73400d2210 2023-07-06T03:45:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.544+0000 7f7337fff700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73400d2210 0x7f73400c8ee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f73280097c0 tx=0x7f7328004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.545+0000 7f733cff9700 1 -- 172.21.15.97:0/3094743366 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732801ab20 con 0x7f73400d2210 2023-07-06T03:45:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.545+0000 7f733cff9700 1 -- 172.21.15.97:0/3094743366 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732801ac80 con 0x7f73400d2210 2023-07-06T03:45:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.545+0000 7f7345bff700 1 -- 172.21.15.97:0/3094743366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73400cbe70 con 0x7f73400d2210 2023-07-06T03:45:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.545+0000 7f733cff9700 1 -- 172.21.15.97:0/3094743366 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73280243f0 con 0x7f73400d2210 2023-07-06T03:45:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.545+0000 7f7345bff700 1 -- 172.21.15.97:0/3094743366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734015e0e0 con 0x7f73400d2210 2023-07-06T03:45:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.546+0000 7f7345bff700 1 -- 172.21.15.97:0/3094743366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734015e2d0 con 0x7f73400d2210 2023-07-06T03:45:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.547+0000 7f733cff9700 1 -- 172.21.15.97:0/3094743366 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f732802d450 con 0x7f73400d2210 2023-07-06T03:45:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.548+0000 7f733cff9700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 0x7f732006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.548+0000 7f733cff9700 1 -- 172.21.15.97:0/3094743366 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f732808aa20 con 0x7f73400d2210 2023-07-06T03:45:29.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.548+0000 7f733f7fe700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 0x7f732006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.548+0000 7f733f7fe700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 0x7f732006d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7330005910 tx=0x7f7330009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:29.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.552+0000 7f733cff9700 1 -- 172.21.15.97:0/3094743366 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7328055be0 con 0x7f73400d2210 2023-07-06T03:45:29.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.748+0000 7f7345bff700 1 -- 172.21.15.97:0/3094743366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f734004e340 con 0x7f73400d2210 2023-07-06T03:45:29.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.749+0000 7f733cff9700 1 -- 172.21.15.97:0/3094743366 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7328015020 con 0x7f73400d2210 2023-07-06T03:45:29.751 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:45:29.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 -- 172.21.15.97:0/3094743366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 msgr2=0x7f732006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 0x7f732006d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7330005910 tx=0x7f7330009040 comp rx=0 tx=0).stop 2023-07-06T03:45:29.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 -- 172.21.15.97:0/3094743366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73400d2210 msgr2=0x7f73400c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73400d2210 0x7f73400c8ee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f73280097c0 tx=0x7f7328004750 comp rx=0 tx=0).stop 2023-07-06T03:45:29.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 -- 172.21.15.97:0/3094743366 shutdown_connections 2023-07-06T03:45:29.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 0x7f732006d190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f734015c610 0x7f734015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f734006b360 0x7f73400c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 --2- 172.21.15.97:0/3094743366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73400d2210 0x7f73400c8ee0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 -- 172.21.15.97:0/3094743366 >> 172.21.15.97:0/3094743366 conn(0x7f734006a6c0 msgr2=0x7f73400cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:29.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 -- 172.21.15.97:0/3094743366 shutdown_connections 2023-07-06T03:45:29.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.751+0000 7f7345bff700 1 -- 172.21.15.97:0/3094743366 wait complete. 2023-07-06T03:45:29.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:45:29.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.964+0000 7f06471ff700 1 Processor -- start 2023-07-06T03:45:29.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.964+0000 7f06471ff700 1 -- start start 2023-07-06T03:45:29.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.965+0000 7f06471ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 0x7f06400cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.965+0000 7f06471ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0640160d00 0x7f06400b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.965+0000 7f06471ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06400cdae0 con 0x7f0640160180 2023-07-06T03:45:29.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.965+0000 7f06471ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06400cd8d0 con 0x7f0640160d00 2023-07-06T03:45:29.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.965+0000 7f06471ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06400b6640 con 0x7f064015c520 2023-07-06T03:45:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.965+0000 7f0644f9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 0x7f06400cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.966+0000 7f0644f9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 0x7f06400cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:34170/0 (socket says 172.21.15.97:34170) 2023-07-06T03:45:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.966+0000 7f0644f9b700 1 -- 172.21.15.97:0/3465373834 learned_addr learned my addr 172.21.15.97:0/3465373834 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.966+0000 7f064579c700 1 --2- 172.21.15.97:0/3465373834 >> v2:172.21.15.146:3300/0 conn(0x7f0640160d00 0x7f06400b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.966+0000 7f0644f9b700 1 -- 172.21.15.97:0/3465373834 >> v2:172.21.15.146:3300/0 conn(0x7f0640160d00 msgr2=0x7f06400b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.966+0000 7f0644f9b700 1 --2- 172.21.15.97:0/3465373834 >> v2:172.21.15.146:3300/0 conn(0x7f0640160d00 0x7f06400b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.966+0000 7f0644f9b700 1 -- 172.21.15.97:0/3465373834 >> v1:172.21.15.146:6789/0 conn(0x7f0640160180 legacy=0x7f06401605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.966+0000 7f063f7fe700 1 -- 172.21.15.97:0/3465373834 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468583931 0 0) 0x7f06400cdae0 con 0x7f0640160180 2023-07-06T03:45:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.966+0000 7f0644f9b700 1 -- 172.21.15.97:0/3465373834 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400b6920 con 0x7f064015c520 2023-07-06T03:45:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.966+0000 7f064579c700 1 --2- 172.21.15.97:0/3465373834 >> v2:172.21.15.146:3300/0 conn(0x7f0640160d00 0x7f06400b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.966+0000 7f0644f9b700 1 --2- 172.21.15.97:0/3465373834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 0x7f06400cd390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f06340098f0 tx=0x7f0634009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba846dbbae77e07 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.966+0000 7f063f7fe700 1 -- 172.21.15.97:0/3465373834 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634011830 con 0x7f064015c520 2023-07-06T03:45:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.967+0000 7f063f7fe700 1 -- 172.21.15.97:0/3465373834 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0634011990 con 0x7f064015c520 2023-07-06T03:45:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.967+0000 7f063f7fe700 1 -- 172.21.15.97:0/3465373834 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634011d20 con 0x7f064015c520 2023-07-06T03:45:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.967+0000 7f06471ff700 1 -- 172.21.15.97:0/3465373834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 msgr2=0x7f06400cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.967+0000 7f06471ff700 1 --2- 172.21.15.97:0/3465373834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 0x7f06400cd390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f06340098f0 tx=0x7f0634009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.967+0000 7f06471ff700 1 -- 172.21.15.97:0/3465373834 shutdown_connections 2023-07-06T03:45:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.967+0000 7f06471ff700 1 --2- 172.21.15.97:0/3465373834 >> v2:172.21.15.146:3300/0 conn(0x7f0640160d00 0x7f06400b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.967+0000 7f06471ff700 1 --2- 172.21.15.97:0/3465373834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 0x7f06400cd390 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.967+0000 7f06471ff700 1 -- 172.21.15.97:0/3465373834 >> 172.21.15.97:0/3465373834 conn(0x7f0640071790 msgr2=0x7f0640071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.967+0000 7f06471ff700 1 -- 172.21.15.97:0/3465373834 shutdown_connections 2023-07-06T03:45:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.967+0000 7f06471ff700 1 -- 172.21.15.97:0/3465373834 wait complete. 2023-07-06T03:45:29.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.969+0000 7f06471ff700 1 Processor -- start 2023-07-06T03:45:29.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.969+0000 7f06471ff700 1 -- start start 2023-07-06T03:45:29.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.970+0000 7f06471ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 0x7f064014cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.970+0000 7f06471ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0640160180 0x7f06400d3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.970+0000 7f06471ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0640160d00 0x7f06400c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.970+0000 7f06471ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06401589d0 con 0x7f0640160180 2023-07-06T03:45:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.970+0000 7f06471ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06401580a0 con 0x7f0640160d00 2023-07-06T03:45:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.970+0000 7f06471ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0640158380 con 0x7f064015c520 2023-07-06T03:45:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.970+0000 7f063ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0640160180 0x7f06400d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.970+0000 7f063ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0640160180 0x7f06400d3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52692/0 (socket says 172.21.15.97:52692) 2023-07-06T03:45:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.970+0000 7f063ffff700 1 -- 172.21.15.97:0/2836750778 learned_addr learned my addr 172.21.15.97:0/2836750778 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.970+0000 7f0644f9b700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 0x7f064014cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f064579c700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0640160d00 0x7f06400c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f063ffff700 1 -- 172.21.15.97:0/2836750778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 msgr2=0x7f064014cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f063ffff700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 0x7f064014cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f063ffff700 1 -- 172.21.15.97:0/2836750778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0640160d00 msgr2=0x7f06400c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f063ffff700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0640160d00 0x7f06400c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f063ffff700 1 -- 172.21.15.97:0/2836750778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638009a70 con 0x7f0640160180 2023-07-06T03:45:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f063ffff700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0640160180 0x7f06400d3130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f062c0011e0 tx=0x7f062c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f063d7fa700 1 -- 172.21.15.97:0/2836750778 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c0054e0 con 0x7f0640160180 2023-07-06T03:45:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f063d7fa700 1 -- 172.21.15.97:0/2836750778 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c003680 con 0x7f0640160180 2023-07-06T03:45:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f06471ff700 1 -- 172.21.15.97:0/2836750778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0634009550 con 0x7f0640160180 2023-07-06T03:45:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f063d7fa700 1 -- 172.21.15.97:0/2836750778 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c003a30 con 0x7f0640160180 2023-07-06T03:45:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.971+0000 7f06471ff700 1 -- 172.21.15.97:0/2836750778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064014d270 con 0x7f0640160180 2023-07-06T03:45:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.972+0000 7f06471ff700 1 -- 172.21.15.97:0/2836750778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0640158b80 con 0x7f0640160180 2023-07-06T03:45:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.973+0000 7f063d7fa700 1 -- 172.21.15.97:0/2836750778 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f062c003b90 con 0x7f0640160180 2023-07-06T03:45:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.974+0000 7f063d7fa700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 0x7f062406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.974+0000 7f063d7fa700 1 -- 172.21.15.97:0/2836750778 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f062c088a60 con 0x7f0640160180 2023-07-06T03:45:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.974+0000 7f0644f9b700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 0x7f062406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.975+0000 7f0644f9b700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 0x7f062406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0634014040 tx=0x7f0634008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:29.978+0000 7f063d7fa700 1 -- 172.21.15.97:0/2836750778 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0640158b80 con 0x7f0640160180 2023-07-06T03:45:30.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.212+0000 7f06471ff700 1 -- 172.21.15.97:0/2836750778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f064004e340 con 0x7f0640160180 2023-07-06T03:45:30.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.212+0000 7f063d7fa700 1 -- 172.21.15.97:0/2836750778 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4665 (secure 0 0 0) 0x7f062c017020 con 0x7f0640160180 2023-07-06T03:45:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:30.215 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/4255451313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4255451313},{"type":"v1","addr":"172.21.15.97:6836","nonce":4255451313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/370322164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":370322164},{"type":"v1","addr":"172.21.15.97:6837","nonce":370322164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3764820847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3764820847},{"type":"v1","addr":"172.21.15.146:6835","nonce":3764820847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:45:11.126281+0000","modified":"2023-07-06T03:45:14.685499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6833/186544257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":186544257},{"type":"v1","addr":"172.21.15.146:6833","nonce":186544257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T03:45:30.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.214+0000 7f06471ff700 1 -- 172.21.15.97:0/2836750778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 msgr2=0x7f062406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.214+0000 7f06471ff700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 0x7f062406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0634014040 tx=0x7f0634008030 comp rx=0 tx=0).stop 2023-07-06T03:45:30.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.214+0000 7f06471ff700 1 -- 172.21.15.97:0/2836750778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0640160180 msgr2=0x7f06400d3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.214+0000 7f06471ff700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0640160180 0x7f06400d3130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f062c0011e0 tx=0x7f062c00cae0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.214+0000 7f06471ff700 1 -- 172.21.15.97:0/2836750778 shutdown_connections 2023-07-06T03:45:30.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.214+0000 7f06471ff700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 0x7f062406d190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.214+0000 7f06471ff700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0640160d00 0x7f06400c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.214+0000 7f06471ff700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064015c520 0x7f064014cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.214+0000 7f06471ff700 1 --2- 172.21.15.97:0/2836750778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0640160180 0x7f06400d3130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.214+0000 7f06471ff700 1 -- 172.21.15.97:0/2836750778 >> 172.21.15.97:0/2836750778 conn(0x7f0640071790 msgr2=0x7f06401605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:30.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.215+0000 7f06471ff700 1 -- 172.21.15.97:0/2836750778 shutdown_connections 2023-07-06T03:45:30.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.215+0000 7f06471ff700 1 -- 172.21.15.97:0/2836750778 wait complete. 2023-07-06T03:45:30.219 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-07-06T03:45:30.234 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:30.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.426+0000 7fcbdd3ff700 1 Processor -- start 2023-07-06T03:45:30.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.426+0000 7fcbdd3ff700 1 -- start start 2023-07-06T03:45:30.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.427+0000 7fcbdd3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcbd815c520 0x7fcbd80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.427+0000 7fcbdd3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbd8160d00 0x7fcbd80b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.427+0000 7fcbdd3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd80cdae0 con 0x7fcbd8160180 2023-07-06T03:45:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.427+0000 7fcbdd3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd80cd8d0 con 0x7fcbd815c520 2023-07-06T03:45:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.427+0000 7fcbdd3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80b6640 con 0x7fcbd8160d00 2023-07-06T03:45:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.428+0000 7fcbcffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcbd8160180 0x7fcbd81605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60910/0 (socket says 172.21.15.97:60910) 2023-07-06T03:45:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.428+0000 7fcbcffff700 1 -- 172.21.15.97:0/1169361858 learned_addr learned my addr 172.21.15.97:0/1169361858 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:30.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.428+0000 7fcbd77fe700 1 --2- 172.21.15.97:0/1169361858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbd8160d00 0x7fcbd80b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.428+0000 7fcbd6ffd700 1 --2- 172.21.15.97:0/1169361858 >> v2:172.21.15.146:3300/0 conn(0x7fcbd815c520 0x7fcbd80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.428+0000 7fcbd67fc700 1 -- 172.21.15.97:0/1169361858 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415065440 0 0) 0x7fcbd80cdae0 con 0x7fcbd8160180 2023-07-06T03:45:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.428+0000 7fcbd67fc700 1 -- 172.21.15.97:0/1169361858 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbc40035b0 con 0x7fcbd8160180 2023-07-06T03:45:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.428+0000 7fcbd77fe700 1 -- 172.21.15.97:0/1169361858 >> v2:172.21.15.146:3300/0 conn(0x7fcbd815c520 msgr2=0x7fcbd80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.428+0000 7fcbd77fe700 1 --2- 172.21.15.97:0/1169361858 >> v2:172.21.15.146:3300/0 conn(0x7fcbd815c520 0x7fcbd80cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.428+0000 7fcbd77fe700 1 -- 172.21.15.97:0/1169361858 >> v1:172.21.15.97:6789/0 conn(0x7fcbd8160180 legacy=0x7fcbd81605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.428+0000 7fcbd77fe700 1 -- 172.21.15.97:0/1169361858 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd80b6920 con 0x7fcbd8160d00 2023-07-06T03:45:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.429+0000 7fcbd77fe700 1 --2- 172.21.15.97:0/1169361858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbd8160d00 0x7fcbd80b7400 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009d90 tx=0x7fcbd000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa6575762c721215 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.429+0000 7fcbd67fc700 1 -- 172.21.15.97:0/1169361858 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000e070 con 0x7fcbd8160d00 2023-07-06T03:45:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.429+0000 7fcbd67fc700 1 -- 172.21.15.97:0/1169361858 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd000bb20 con 0x7fcbd8160d00 2023-07-06T03:45:30.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.429+0000 7fcbd67fc700 1 -- 172.21.15.97:0/1169361858 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0012430 con 0x7fcbd8160d00 2023-07-06T03:45:30.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.429+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/1169361858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbd8160d00 msgr2=0x7fcbd80b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.429+0000 7fcbdd3ff700 1 --2- 172.21.15.97:0/1169361858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbd8160d00 0x7fcbd80b7400 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009d90 tx=0x7fcbd000b330 comp rx=0 tx=0).stop 2023-07-06T03:45:30.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.429+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/1169361858 shutdown_connections 2023-07-06T03:45:30.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.429+0000 7fcbdd3ff700 1 --2- 172.21.15.97:0/1169361858 >> v2:172.21.15.146:3300/0 conn(0x7fcbd815c520 0x7fcbd80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.429+0000 7fcbdd3ff700 1 --2- 172.21.15.97:0/1169361858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbd8160d00 0x7fcbd80b7400 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.429+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/1169361858 >> 172.21.15.97:0/1169361858 conn(0x7fcbd8071790 msgr2=0x7fcbd8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:30.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.430+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/1169361858 shutdown_connections 2023-07-06T03:45:30.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.430+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/1169361858 wait complete. 2023-07-06T03:45:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.431+0000 7fcbdd3ff700 1 Processor -- start 2023-07-06T03:45:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.431+0000 7fcbdd3ff700 1 -- start start 2023-07-06T03:45:30.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.432+0000 7fcbdd3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbd815c520 0x7fcbd81f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.432+0000 7fcbdd3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbd8160180 0x7fcbd80d1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.432+0000 7fcbdd3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbd8160d00 0x7fcbd80d3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.432+0000 7fcbdd3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80cad80 con 0x7fcbd8160d00 2023-07-06T03:45:30.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.432+0000 7fcbdd3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80cb060 con 0x7fcbd8160180 2023-07-06T03:45:30.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.432+0000 7fcbdd3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80cb340 con 0x7fcbd815c520 2023-07-06T03:45:30.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbd8160d00 0x7fcbd80d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbd6ffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbd815c520 0x7fcbd81f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbd8160d00 0x7fcbd80d3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52700/0 (socket says 172.21.15.97:52700) 2023-07-06T03:45:30.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbd77fe700 1 -- 172.21.15.97:0/3782538413 learned_addr learned my addr 172.21.15.97:0/3782538413 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:30.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbcffff700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbd8160180 0x7fcbd80d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbd77fe700 1 -- 172.21.15.97:0/3782538413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbd815c520 msgr2=0x7fcbd81f1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbd77fe700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbd815c520 0x7fcbd81f1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbd77fe700 1 -- 172.21.15.97:0/3782538413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbd8160180 msgr2=0x7fcbd80d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbd77fe700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbd8160180 0x7fcbd80d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbd77fe700 1 -- 172.21.15.97:0/3782538413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0009a70 con 0x7fcbd8160d00 2023-07-06T03:45:30.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbd77fe700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbd8160d00 0x7fcbd80d3020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0014040 tx=0x7fcbd0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:30.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.433+0000 7fcbcf7fe700 1 -- 172.21.15.97:0/3782538413 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd00129f0 con 0x7fcbd8160d00 2023-07-06T03:45:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.434+0000 7fcbcf7fe700 1 -- 172.21.15.97:0/3782538413 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd0020070 con 0x7fcbd8160d00 2023-07-06T03:45:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.434+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/3782538413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd80cb620 con 0x7fcbd8160d00 2023-07-06T03:45:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.434+0000 7fcbcf7fe700 1 -- 172.21.15.97:0/3782538413 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0012db0 con 0x7fcbd8160d00 2023-07-06T03:45:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.434+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/3782538413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd80c98c0 con 0x7fcbd8160d00 2023-07-06T03:45:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.435+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/3782538413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd80cbb40 con 0x7fcbd8160d00 2023-07-06T03:45:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.436+0000 7fcbcf7fe700 1 -- 172.21.15.97:0/3782538413 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcbd0004040 con 0x7fcbd8160d00 2023-07-06T03:45:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.437+0000 7fcbcf7fe700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbb806ad10 0x7fcbb806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.437+0000 7fcbcf7fe700 1 -- 172.21.15.97:0/3782538413 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbd0022070 con 0x7fcbd8160d00 2023-07-06T03:45:30.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.437+0000 7fcbd6ffd700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbb806ad10 0x7fcbb806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.437+0000 7fcbd6ffd700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbb806ad10 0x7fcbb806d1c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009be0 tx=0x7fcbc8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.441+0000 7fcbcf7fe700 1 -- 172.21.15.97:0/3782538413 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcbd80cbb40 con 0x7fcbd8160d00 2023-07-06T03:45:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.636+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/3782538413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbd80cbb40 con 0x7fcbd8160d00 2023-07-06T03:45:30.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.637+0000 7fcbcf7fe700 1 -- 172.21.15.97:0/3782538413 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7fcbd80cbb40 con 0x7fcbd8160d00 2023-07-06T03:45:30.639 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:30.639 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/3782538413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbb806ad10 msgr2=0x7fcbb806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbb806ad10 0x7fcbb806d1c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009be0 tx=0x7fcbc8009210 comp rx=0 tx=0).stop 2023-07-06T03:45:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/3782538413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbd8160d00 msgr2=0x7fcbd80d3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbd8160d00 0x7fcbd80d3020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0014040 tx=0x7fcbd0004ad0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/3782538413 shutdown_connections 2023-07-06T03:45:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbb806ad10 0x7fcbb806d1c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbd8160d00 0x7fcbd80d3020 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbd815c520 0x7fcbd81f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 --2- 172.21.15.97:0/3782538413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbd8160180 0x7fcbd80d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/3782538413 >> 172.21.15.97:0/3782538413 conn(0x7fcbd8071790 msgr2=0x7fcbd815ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:30.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/3782538413 shutdown_connections 2023-07-06T03:45:30.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.640+0000 7fcbdd3ff700 1 -- 172.21.15.97:0/3782538413 wait complete. 2023-07-06T03:45:30.660 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:45:30.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:45:30.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.857+0000 7fa80279d700 1 Processor -- start 2023-07-06T03:45:30.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.857+0000 7fa80279d700 1 -- start start 2023-07-06T03:45:30.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.858+0000 7fa80279d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7fc0b8470 0x7fa7fc0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.858+0000 7fa80279d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa7fc0cd660 0x7fa7fc0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.858+0000 7fa80279d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7fc15cb70 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.858+0000 7fa80279d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7fc0cfe10 con 0x7fa7fc0cd660 2023-07-06T03:45:30.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.858+0000 7fa80279d700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc0cd1d0 con 0x7fa7fc0b8470 2023-07-06T03:45:30.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.858+0000 7fa800f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa7fc0cd660 0x7fa7fc0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.858+0000 7fa800f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa7fc0cd660 0x7fa7fc0d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52708/0 (socket says 172.21.15.97:52708) 2023-07-06T03:45:30.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.858+0000 7fa800f9a700 1 -- 172.21.15.97:0/2650370350 learned_addr learned my addr 172.21.15.97:0/2650370350 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:30.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.858+0000 7fa80179b700 1 --2- 172.21.15.97:0/2650370350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7fc0b8470 0x7fa7fc0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.859+0000 7fa800f9a700 1 -- 172.21.15.97:0/2650370350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7fc0b8470 msgr2=0x7fa7fc0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:30.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.859+0000 7fa800f9a700 1 --2- 172.21.15.97:0/2650370350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7fc0b8470 0x7fa7fc0b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.859+0000 7fa800f9a700 1 -- 172.21.15.97:0/2650370350 >> v1:172.21.15.97:6789/0 conn(0x7fa7fc0b1b00 legacy=0x7fa7fc0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.859+0000 7fa800f9a700 1 -- 172.21.15.97:0/2650370350 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc0d2690 con 0x7fa7fc0cd660 2023-07-06T03:45:30.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.859+0000 7fa7f3fff700 1 -- 172.21.15.97:0/2650370350 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480699455 0 0) 0x7fa7f4001490 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.859+0000 7fa800f9a700 1 --2- 172.21.15.97:0/2650370350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7fc0cd660 0x7fa7fc0d0c20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0098f0 tx=0x7fa7ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab2515cd72f6f1a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:30.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.859+0000 7fa7f3fff700 1 -- 172.21.15.97:0/2650370350 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec011830 con 0x7fa7fc0cd660 2023-07-06T03:45:30.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.859+0000 7fa7f3fff700 1 -- 172.21.15.97:0/2650370350 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec011990 con 0x7fa7fc0cd660 2023-07-06T03:45:30.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.859+0000 7fa7f3fff700 1 -- 172.21.15.97:0/2650370350 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec011d20 con 0x7fa7fc0cd660 2023-07-06T03:45:30.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.860+0000 7fa80279d700 1 -- 172.21.15.97:0/2650370350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7fc0cd660 msgr2=0x7fa7fc0d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.860+0000 7fa80279d700 1 --2- 172.21.15.97:0/2650370350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7fc0cd660 0x7fa7fc0d0c20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0098f0 tx=0x7fa7ec009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:30.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.860+0000 7fa80279d700 1 -- 172.21.15.97:0/2650370350 shutdown_connections 2023-07-06T03:45:30.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.860+0000 7fa80279d700 1 --2- 172.21.15.97:0/2650370350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7fc0b8470 0x7fa7fc0b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.860+0000 7fa80279d700 1 --2- 172.21.15.97:0/2650370350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7fc0cd660 0x7fa7fc0d0c20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.860+0000 7fa80279d700 1 -- 172.21.15.97:0/2650370350 >> 172.21.15.97:0/2650370350 conn(0x7fa7fc06b160 msgr2=0x7fa7fc071790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:30.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.860+0000 7fa80279d700 1 -- 172.21.15.97:0/2650370350 shutdown_connections 2023-07-06T03:45:30.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.860+0000 7fa80279d700 1 -- 172.21.15.97:0/2650370350 wait complete. 2023-07-06T03:45:30.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.862+0000 7fa80279d700 1 Processor -- start 2023-07-06T03:45:30.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.862+0000 7fa80279d700 1 -- start start 2023-07-06T03:45:30.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80279d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7fc0b1b00 0x7fa7fc0cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80279d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7fc0b8470 0x7fa7fc154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80279d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7fc0cd660 0x7fa7fc15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80279d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc1571b0 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80279d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc15f350 con 0x7fa7fc0b8470 2023-07-06T03:45:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80279d700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc15f630 con 0x7fa7fc0cd660 2023-07-06T03:45:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80179b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7fc0b1b00 0x7fa7fc0cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80179b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7fc0b1b00 0x7fa7fc0cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52718/0 (socket says 172.21.15.97:52718) 2023-07-06T03:45:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80179b700 1 -- 172.21.15.97:0/4109925396 learned_addr learned my addr 172.21.15.97:0/4109925396 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa801f9c700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7fc0cd660 0x7fa7fc15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa800f9a700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7fc0b8470 0x7fa7fc154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80179b700 1 -- 172.21.15.97:0/4109925396 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7fc0cd660 msgr2=0x7fa7fc15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80179b700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7fc0cd660 0x7fa7fc15cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80179b700 1 -- 172.21.15.97:0/4109925396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7fc0b8470 msgr2=0x7fa7fc154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:30.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80179b700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7fc0b8470 0x7fa7fc154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:30.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa80179b700 1 -- 172.21.15.97:0/4109925396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec009550 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.863+0000 7fa800f9a700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7fc0b8470 0x7fa7fc154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:30.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.864+0000 7fa80179b700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7fc0b1b00 0x7fa7fc0cc800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e9b0 tx=0x7fa7f800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:30.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.864+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/4109925396 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8009710 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.864+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/4109925396 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f800c850 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.864+0000 7fa80279d700 1 -- 172.21.15.97:0/4109925396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7fc15f910 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.864+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/4109925396 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f800cbc0 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.864+0000 7fa80279d700 1 -- 172.21.15.97:0/4109925396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7fc155590 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.865+0000 7fa80279d700 1 -- 172.21.15.97:0/4109925396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7fc157420 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.866+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/4109925396 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7f800cd20 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.867+0000 7fa7f1ffb700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 0x7fa7e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:30.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.867+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/4109925396 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7f80895a0 con 0x7fa7fc0b1b00 2023-07-06T03:45:30.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.867+0000 7fa800f9a700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 0x7fa7e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:30.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.868+0000 7fa800f9a700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 0x7fa7e006d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc066130 tx=0x7fa7ec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:30.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:30.871+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/4109925396 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa7f80580d0 con 0x7fa7fc0b1b00 2023-07-06T03:45:31.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.067+0000 7fa80279d700 1 -- 172.21.15.97:0/4109925396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7fc0c9840 con 0x7fa7fc0b1b00 2023-07-06T03:45:31.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.068+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/4109925396 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7f8057c60 con 0x7fa7fc0b1b00 2023-07-06T03:45:31.070 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:45:31.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.070+0000 7fa80279d700 1 -- 172.21.15.97:0/4109925396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 msgr2=0x7fa7e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.070+0000 7fa80279d700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 0x7fa7e006d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc066130 tx=0x7fa7ec008030 comp rx=0 tx=0).stop 2023-07-06T03:45:31.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.070+0000 7fa80279d700 1 -- 172.21.15.97:0/4109925396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7fc0b1b00 msgr2=0x7fa7fc0cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.070+0000 7fa80279d700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7fc0b1b00 0x7fa7fc0cc800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e9b0 tx=0x7fa7f800ed70 comp rx=0 tx=0).stop 2023-07-06T03:45:31.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.070+0000 7fa80279d700 1 -- 172.21.15.97:0/4109925396 shutdown_connections 2023-07-06T03:45:31.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.070+0000 7fa80279d700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 0x7fa7e006d190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.070+0000 7fa80279d700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7fc0cd660 0x7fa7fc15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.070+0000 7fa80279d700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7fc0b1b00 0x7fa7fc0cc800 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.070+0000 7fa80279d700 1 --2- 172.21.15.97:0/4109925396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7fc0b8470 0x7fa7fc154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.070+0000 7fa80279d700 1 -- 172.21.15.97:0/4109925396 >> 172.21.15.97:0/4109925396 conn(0x7fa7fc06b160 msgr2=0x7fa7fc0d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:31.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.071+0000 7fa80279d700 1 -- 172.21.15.97:0/4109925396 shutdown_connections 2023-07-06T03:45:31.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.071+0000 7fa80279d700 1 -- 172.21.15.97:0/4109925396 wait complete. 2023-07-06T03:45:31.090 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:45:31.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.285+0000 7f39145ff700 1 Processor -- start 2023-07-06T03:45:31.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.285+0000 7f39145ff700 1 -- start start 2023-07-06T03:45:31.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.286+0000 7f39145ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f390c06b360 0x7f390c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.286+0000 7f39145ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f390c0d2210 0x7f390c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.286+0000 7f39145ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c0cff90 con 0x7f390c15c610 2023-07-06T03:45:31.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.286+0000 7f39145ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c0d0780 con 0x7f390c0d2210 2023-07-06T03:45:31.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.286+0000 7f39145ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c15c9e0 con 0x7f390c06b360 2023-07-06T03:45:31.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.287+0000 7f391239b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f390c15c610 0x7f390c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60928/0 (socket says 172.21.15.97:60928) 2023-07-06T03:45:31.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.287+0000 7f391239b700 1 -- 172.21.15.97:0/3026367198 learned_addr learned my addr 172.21.15.97:0/3026367198 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:31.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.287+0000 7f3912b9c700 1 --2- 172.21.15.97:0/3026367198 >> v2:172.21.15.97:3300/0 conn(0x7f390c0d2210 0x7f390c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.287+0000 7f3911b9a700 1 --2- 172.21.15.97:0/3026367198 >> v2:172.21.15.146:3300/0 conn(0x7f390c06b360 0x7f390c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.287+0000 7f3912b9c700 1 -- 172.21.15.97:0/3026367198 >> v2:172.21.15.146:3300/0 conn(0x7f390c06b360 msgr2=0x7f390c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:31.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.287+0000 7f3912b9c700 1 --2- 172.21.15.97:0/3026367198 >> v2:172.21.15.146:3300/0 conn(0x7f390c06b360 0x7f390c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.287+0000 7f3912b9c700 1 -- 172.21.15.97:0/3026367198 >> v1:172.21.15.97:6789/0 conn(0x7f390c15c610 legacy=0x7f390c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.287+0000 7f3912b9c700 1 -- 172.21.15.97:0/3026367198 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0b5650 con 0x7f390c0d2210 2023-07-06T03:45:31.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.287+0000 7f3911399700 1 -- 172.21.15.97:0/3026367198 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350461055 0 0) 0x7f390c0cff90 con 0x7f390c15c610 2023-07-06T03:45:31.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.287+0000 7f3912b9c700 1 --2- 172.21.15.97:0/3026367198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0d1310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f390c06a590 tx=0x7f390400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844273113c19491f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.288+0000 7f3911399700 1 -- 172.21.15.97:0/3026367198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390400e070 con 0x7f390c0d2210 2023-07-06T03:45:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.288+0000 7f3911399700 1 -- 172.21.15.97:0/3026367198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390400bb90 con 0x7f390c0d2210 2023-07-06T03:45:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.288+0000 7f3911399700 1 -- 172.21.15.97:0/3026367198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904012430 con 0x7f390c0d2210 2023-07-06T03:45:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.288+0000 7f39145ff700 1 -- 172.21.15.97:0/3026367198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 msgr2=0x7f390c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.288+0000 7f39145ff700 1 --2- 172.21.15.97:0/3026367198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0d1310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f390c06a590 tx=0x7f390400b330 comp rx=0 tx=0).stop 2023-07-06T03:45:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.288+0000 7f39145ff700 1 -- 172.21.15.97:0/3026367198 shutdown_connections 2023-07-06T03:45:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.288+0000 7f39145ff700 1 --2- 172.21.15.97:0/3026367198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0d1310 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.288+0000 7f39145ff700 1 --2- 172.21.15.97:0/3026367198 >> v2:172.21.15.146:3300/0 conn(0x7f390c06b360 0x7f390c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.288+0000 7f39145ff700 1 -- 172.21.15.97:0/3026367198 >> 172.21.15.97:0/3026367198 conn(0x7f390c06a6c0 msgr2=0x7f390c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.288+0000 7f39145ff700 1 -- 172.21.15.97:0/3026367198 shutdown_connections 2023-07-06T03:45:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.288+0000 7f39145ff700 1 -- 172.21.15.97:0/3026367198 wait complete. 2023-07-06T03:45:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.290+0000 7f39145ff700 1 Processor -- start 2023-07-06T03:45:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.290+0000 7f39145ff700 1 -- start start 2023-07-06T03:45:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.291+0000 7f39145ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f390c06b360 0x7f390c0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.291+0000 7f39145ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c0d2210 0x7f390c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.291+0000 7f39145ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c15c610 0x7f390c1f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.291+0000 7f39145ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c15f270 con 0x7f390c15c610 2023-07-06T03:45:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.291+0000 7f39145ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c154470 con 0x7f390c0d2210 2023-07-06T03:45:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.291+0000 7f39145ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0cb0a0 con 0x7f390c06b360 2023-07-06T03:45:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.291+0000 7f3912b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c15c610 0x7f390c1f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.291+0000 7f3912b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c15c610 0x7f390c1f17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52746/0 (socket says 172.21.15.97:52746) 2023-07-06T03:45:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.291+0000 7f3912b9c700 1 -- 172.21.15.97:0/875599663 learned_addr learned my addr 172.21.15.97:0/875599663 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.291+0000 7f391239b700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f390c06b360 0x7f390c0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.291+0000 7f3911b9a700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c0d2210 0x7f390c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f3912b9c700 1 -- 172.21.15.97:0/875599663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f390c06b360 msgr2=0x7f390c0c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f3912b9c700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f390c06b360 0x7f390c0c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f3912b9c700 1 -- 172.21.15.97:0/875599663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c0d2210 msgr2=0x7f390c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f3912b9c700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c0d2210 0x7f390c15ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f3912b9c700 1 -- 172.21.15.97:0/875599663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3904009a70 con 0x7f390c15c610 2023-07-06T03:45:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f3911b9a700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c0d2210 0x7f390c15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f3912b9c700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c15c610 0x7f390c1f17f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3904014040 tx=0x7f3904004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f3902ffd700 1 -- 172.21.15.97:0/875599663 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904012430 con 0x7f390c15c610 2023-07-06T03:45:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f3902ffd700 1 -- 172.21.15.97:0/875599663 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3904020070 con 0x7f390c15c610 2023-07-06T03:45:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f3902ffd700 1 -- 172.21.15.97:0/875599663 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904004280 con 0x7f390c15c610 2023-07-06T03:45:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f39145ff700 1 -- 172.21.15.97:0/875599663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c0ca490 con 0x7f390c15c610 2023-07-06T03:45:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.292+0000 7f39145ff700 1 -- 172.21.15.97:0/875599663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c0caa50 con 0x7f390c15c610 2023-07-06T03:45:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.293+0000 7f39145ff700 1 -- 172.21.15.97:0/875599663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c0cb250 con 0x7f390c15c610 2023-07-06T03:45:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.294+0000 7f3902ffd700 1 -- 172.21.15.97:0/875599663 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f39040043e0 con 0x7f390c15c610 2023-07-06T03:45:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.295+0000 7f3902ffd700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 0x7f38f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.295+0000 7f3902ffd700 1 -- 172.21.15.97:0/875599663 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3904022070 con 0x7f390c15c610 2023-07-06T03:45:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.295+0000 7f391239b700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 0x7f38f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.296+0000 7f391239b700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 0x7f38f006d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3908000f60 tx=0x7f3908007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.299+0000 7f3902ffd700 1 -- 172.21.15.97:0/875599663 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f390c0cb250 con 0x7f390c15c610 2023-07-06T03:45:31.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.495+0000 7f39145ff700 1 -- 172.21.15.97:0/875599663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f390c0cb210 con 0x7f390c15c610 2023-07-06T03:45:31.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.496+0000 7f3902ffd700 1 -- 172.21.15.97:0/875599663 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f390402c020 con 0x7f390c15c610 2023-07-06T03:45:31.498 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:45:31.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 -- 172.21.15.97:0/875599663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 msgr2=0x7f38f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 0x7f38f006d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3908000f60 tx=0x7f3908007e70 comp rx=0 tx=0).stop 2023-07-06T03:45:31.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 -- 172.21.15.97:0/875599663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c15c610 msgr2=0x7f390c1f17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c15c610 0x7f390c1f17f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3904014040 tx=0x7f3904004ad0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 -- 172.21.15.97:0/875599663 shutdown_connections 2023-07-06T03:45:31.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 0x7f38f006d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c15c610 0x7f390c1f17f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f390c06b360 0x7f390c0c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 --2- 172.21.15.97:0/875599663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c0d2210 0x7f390c15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 -- 172.21.15.97:0/875599663 >> 172.21.15.97:0/875599663 conn(0x7f390c06a6c0 msgr2=0x7f390c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:31.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 -- 172.21.15.97:0/875599663 shutdown_connections 2023-07-06T03:45:31.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.498+0000 7f39145ff700 1 -- 172.21.15.97:0/875599663 wait complete. 2023-07-06T03:45:31.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:45:31.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.713+0000 7f9337dff700 1 Processor -- start 2023-07-06T03:45:31.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.713+0000 7f9337dff700 1 -- start start 2023-07-06T03:45:31.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.714+0000 7f9337dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933006b360 0x7f93300b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.714+0000 7f9337dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f93300d2210 0x7f93300d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.714+0000 7f9337dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93300cff90 con 0x7f933015c610 2023-07-06T03:45:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.714+0000 7f9337dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93300d0780 con 0x7f93300d2210 2023-07-06T03:45:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.714+0000 7f9337dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933015c9e0 con 0x7f933006b360 2023-07-06T03:45:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.714+0000 7f9335b9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f933015c610 0x7f93300b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60944/0 (socket says 172.21.15.97:60944) 2023-07-06T03:45:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.714+0000 7f9335b9b700 1 -- 172.21.15.97:0/79291863 learned_addr learned my addr 172.21.15.97:0/79291863 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.714+0000 7f933539a700 1 --2- 172.21.15.97:0/79291863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933006b360 0x7f93300b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.714+0000 7f933639c700 1 --2- 172.21.15.97:0/79291863 >> v2:172.21.15.146:3300/0 conn(0x7f93300d2210 0x7f93300d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.715+0000 7f9334b99700 1 -- 172.21.15.97:0/79291863 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175626856 0 0) 0x7f93300cff90 con 0x7f933015c610 2023-07-06T03:45:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.715+0000 7f9334b99700 1 -- 172.21.15.97:0/79291863 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f931c0035b0 con 0x7f933015c610 2023-07-06T03:45:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.715+0000 7f933539a700 1 -- 172.21.15.97:0/79291863 >> v2:172.21.15.146:3300/0 conn(0x7f93300d2210 msgr2=0x7f93300d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.715+0000 7f933539a700 1 --2- 172.21.15.97:0/79291863 >> v2:172.21.15.146:3300/0 conn(0x7f93300d2210 0x7f93300d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.715+0000 7f933539a700 1 -- 172.21.15.97:0/79291863 >> v1:172.21.15.97:6789/0 conn(0x7f933015c610 legacy=0x7f93300b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.715+0000 7f933539a700 1 -- 172.21.15.97:0/79291863 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93300b5650 con 0x7f933006b360 2023-07-06T03:45:31.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.715+0000 7f933539a700 1 --2- 172.21.15.97:0/79291863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933006b360 0x7f93300b4ec0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9320009840 tx=0x7f9320009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ca1b20e04f1d3d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.715+0000 7f9334b99700 1 -- 172.21.15.97:0/79291863 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320011830 con 0x7f933006b360 2023-07-06T03:45:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.715+0000 7f9334b99700 1 -- 172.21.15.97:0/79291863 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9320011990 con 0x7f933006b360 2023-07-06T03:45:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.715+0000 7f9334b99700 1 -- 172.21.15.97:0/79291863 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320011d20 con 0x7f933006b360 2023-07-06T03:45:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.716+0000 7f9337dff700 1 -- 172.21.15.97:0/79291863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933006b360 msgr2=0x7f93300b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.716+0000 7f9337dff700 1 --2- 172.21.15.97:0/79291863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933006b360 0x7f93300b4ec0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9320009840 tx=0x7f9320009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.716+0000 7f9337dff700 1 -- 172.21.15.97:0/79291863 shutdown_connections 2023-07-06T03:45:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.716+0000 7f9337dff700 1 --2- 172.21.15.97:0/79291863 >> v2:172.21.15.146:3300/0 conn(0x7f93300d2210 0x7f93300d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.716+0000 7f9337dff700 1 --2- 172.21.15.97:0/79291863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933006b360 0x7f93300b4ec0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.716+0000 7f9337dff700 1 -- 172.21.15.97:0/79291863 >> 172.21.15.97:0/79291863 conn(0x7f933006a6c0 msgr2=0x7f933006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.716+0000 7f9337dff700 1 -- 172.21.15.97:0/79291863 shutdown_connections 2023-07-06T03:45:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.716+0000 7f9337dff700 1 -- 172.21.15.97:0/79291863 wait complete. 2023-07-06T03:45:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.718+0000 7f9337dff700 1 Processor -- start 2023-07-06T03:45:31.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.718+0000 7f9337dff700 1 -- start start 2023-07-06T03:45:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.719+0000 7f9337dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933006b360 0x7f93301f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.719+0000 7f9337dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93300d2210 0x7f93300cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.719+0000 7f9337dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f933015c610 0x7f933015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.719+0000 7f9337dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93301f60a0 con 0x7f933015c610 2023-07-06T03:45:31.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.719+0000 7f9337dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93301f5800 con 0x7f93300d2210 2023-07-06T03:45:31.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.719+0000 7f9337dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93301f5ae0 con 0x7f933006b360 2023-07-06T03:45:31.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.719+0000 7f933639c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f933015c610 0x7f933015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.719+0000 7f933639c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f933015c610 0x7f933015cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52760/0 (socket says 172.21.15.97:52760) 2023-07-06T03:45:31.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.719+0000 7f933639c700 1 -- 172.21.15.97:0/286702413 learned_addr learned my addr 172.21.15.97:0/286702413 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:31.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.719+0000 7f933539a700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93300d2210 0x7f93300cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.719+0000 7f9335b9b700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933006b360 0x7f93301f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.720+0000 7f933639c700 1 -- 172.21.15.97:0/286702413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933006b360 msgr2=0x7f93301f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.720+0000 7f933639c700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933006b360 0x7f93301f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.720+0000 7f933639c700 1 -- 172.21.15.97:0/286702413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93300d2210 msgr2=0x7f93300cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.720+0000 7f933639c700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93300d2210 0x7f93300cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.720+0000 7f933639c700 1 -- 172.21.15.97:0/286702413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93200094a0 con 0x7f933015c610 2023-07-06T03:45:31.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.720+0000 7f933639c700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f933015c610 0x7f933015cb20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f932800efc0 tx=0x7f932800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:31.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.720+0000 7f93267fc700 1 -- 172.21.15.97:0/286702413 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328009ea0 con 0x7f933015c610 2023-07-06T03:45:31.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.720+0000 7f93267fc700 1 -- 172.21.15.97:0/286702413 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932800f040 con 0x7f933015c610 2023-07-06T03:45:31.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.720+0000 7f9337dff700 1 -- 172.21.15.97:0/286702413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93301f5dc0 con 0x7f933015c610 2023-07-06T03:45:31.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.720+0000 7f93267fc700 1 -- 172.21.15.97:0/286702413 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932800ced0 con 0x7f933015c610 2023-07-06T03:45:31.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.720+0000 7f9337dff700 1 -- 172.21.15.97:0/286702413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933015f170 con 0x7f933015c610 2023-07-06T03:45:31.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.721+0000 7f9337dff700 1 -- 172.21.15.97:0/286702413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93300cae50 con 0x7f933015c610 2023-07-06T03:45:31.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.722+0000 7f93267fc700 1 -- 172.21.15.97:0/286702413 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9328014480 con 0x7f933015c610 2023-07-06T03:45:31.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.723+0000 7f93267fc700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f931406ace0 0x7f931406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:31.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.723+0000 7f93267fc700 1 -- 172.21.15.97:0/286702413 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9328089c90 con 0x7f933015c610 2023-07-06T03:45:31.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.723+0000 7f9335b9b700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f931406ace0 0x7f931406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:31.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.724+0000 7f9335b9b700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f931406ace0 0x7f931406d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f932c007fd0 tx=0x7f932c007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:31.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.727+0000 7f93267fc700 1 -- 172.21.15.97:0/286702413 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f93300cae50 con 0x7f933015c610 2023-07-06T03:45:31.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.931+0000 7f9337dff700 1 -- 172.21.15.97:0/286702413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93300cae50 con 0x7f933015c610 2023-07-06T03:45:31.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.931+0000 7f93267fc700 1 -- 172.21.15.97:0/286702413 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f93300cae50 con 0x7f933015c610 2023-07-06T03:45:31.934 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:45:31.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 -- 172.21.15.97:0/286702413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f931406ace0 msgr2=0x7f931406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f931406ace0 0x7f931406d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f932c007fd0 tx=0x7f932c007dc0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 -- 172.21.15.97:0/286702413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f933015c610 msgr2=0x7f933015cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:31.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f933015c610 0x7f933015cb20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f932800efc0 tx=0x7f932800c3a0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 -- 172.21.15.97:0/286702413 shutdown_connections 2023-07-06T03:45:31.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f931406ace0 0x7f931406d190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f933015c610 0x7f933015cb20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933006b360 0x7f93301f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 --2- 172.21.15.97:0/286702413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93300d2210 0x7f93300cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:31.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 -- 172.21.15.97:0/286702413 >> 172.21.15.97:0/286702413 conn(0x7f933006a6c0 msgr2=0x7f93300b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:31.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 -- 172.21.15.97:0/286702413 shutdown_connections 2023-07-06T03:45:31.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:31.934+0000 7f9337dff700 1 -- 172.21.15.97:0/286702413 wait complete. 2023-07-06T03:45:31.954 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:45:32.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.149+0000 7f815dfff700 1 Processor -- start 2023-07-06T03:45:32.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.149+0000 7f815dfff700 1 -- start start 2023-07-06T03:45:32.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.150+0000 7f815dfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8158159070 0x7f81580c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.150+0000 7f815dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81580c3cd0 0x7f81580cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.150+0000 7f815dfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81580ca260 con 0x7f8158159440 2023-07-06T03:45:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.150+0000 7f815dfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81580c99c0 con 0x7f8158159070 2023-07-06T03:45:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.150+0000 7f815dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81580c9ca0 con 0x7f81580c3cd0 2023-07-06T03:45:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.151+0000 7f81577fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8158159070 0x7f81580c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.151+0000 7f81577fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8158159070 0x7f81580c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52772/0 (socket says 172.21.15.97:52772) 2023-07-06T03:45:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.151+0000 7f81577fe700 1 -- 172.21.15.97:0/1011990651 learned_addr learned my addr 172.21.15.97:0/1011990651 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.151+0000 7f8156ffd700 1 --2- 172.21.15.97:0/1011990651 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81580c3cd0 0x7f81580cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.151+0000 7f81577fe700 1 -- 172.21.15.97:0/1011990651 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81580c3cd0 msgr2=0x7f81580cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:32.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.151+0000 7f81577fe700 1 --2- 172.21.15.97:0/1011990651 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81580c3cd0 0x7f81580cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.151+0000 7f81577fe700 1 -- 172.21.15.97:0/1011990651 >> v1:172.21.15.146:6789/0 conn(0x7f8158159440 legacy=0x7f81580b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.151+0000 7f81577fe700 1 -- 172.21.15.97:0/1011990651 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81580c9f80 con 0x7f8158159070 2023-07-06T03:45:32.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.151+0000 7f8156ffd700 1 --2- 172.21.15.97:0/1011990651 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81580c3cd0 0x7f81580cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:32.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.151+0000 7f81577fe700 1 --2- 172.21.15.97:0/1011990651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8158159070 0x7f81580c8f40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f814800bb90 tx=0x7f814800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74b7fd2be3f0743 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.151+0000 7f81567fc700 1 -- 172.21.15.97:0/1011990651 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814800dd50 con 0x7f8158159070 2023-07-06T03:45:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.152+0000 7f81567fc700 1 -- 172.21.15.97:0/1011990651 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8148010470 con 0x7f8158159070 2023-07-06T03:45:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.152+0000 7f81567fc700 1 -- 172.21.15.97:0/1011990651 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148010800 con 0x7f8158159070 2023-07-06T03:45:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.152+0000 7f815dfff700 1 -- 172.21.15.97:0/1011990651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8158159070 msgr2=0x7f81580c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.152+0000 7f815dfff700 1 --2- 172.21.15.97:0/1011990651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8158159070 0x7f81580c8f40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f814800bb90 tx=0x7f814800bea0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.152+0000 7f815dfff700 1 -- 172.21.15.97:0/1011990651 shutdown_connections 2023-07-06T03:45:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.152+0000 7f815dfff700 1 --2- 172.21.15.97:0/1011990651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8158159070 0x7f81580c8f40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.152+0000 7f815dfff700 1 --2- 172.21.15.97:0/1011990651 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81580c3cd0 0x7f81580cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.152+0000 7f815dfff700 1 -- 172.21.15.97:0/1011990651 >> 172.21.15.97:0/1011990651 conn(0x7f815806a060 msgr2=0x7f815806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.152+0000 7f815dfff700 1 -- 172.21.15.97:0/1011990651 shutdown_connections 2023-07-06T03:45:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.152+0000 7f815dfff700 1 -- 172.21.15.97:0/1011990651 wait complete. 2023-07-06T03:45:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.154+0000 7f815dfff700 1 Processor -- start 2023-07-06T03:45:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.154+0000 7f815dfff700 1 -- start start 2023-07-06T03:45:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.155+0000 7f815dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81580c3cd0 0x7f815815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.155+0000 7f815dfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8158159070 0x7f81581f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.155+0000 7f815dfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8158159440 0x7f81580b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.155+0000 7f815dfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815815c7b0 con 0x7f8158159440 2023-07-06T03:45:32.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.155+0000 7f815dfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81580b1fc0 con 0x7f8158159070 2023-07-06T03:45:32.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.155+0000 7f815dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815815bc90 con 0x7f81580c3cd0 2023-07-06T03:45:32.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.155+0000 7f8157fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8158159440 0x7f81580b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.155+0000 7f8157fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8158159440 0x7f81580b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52778/0 (socket says 172.21.15.97:52778) 2023-07-06T03:45:32.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.155+0000 7f8157fff700 1 -- 172.21.15.97:0/705967885 learned_addr learned my addr 172.21.15.97:0/705967885 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:32.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f81577fe700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81580c3cd0 0x7f815815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f8156ffd700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8158159070 0x7f81581f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f8157fff700 1 -- 172.21.15.97:0/705967885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81580c3cd0 msgr2=0x7f815815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f8157fff700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81580c3cd0 0x7f815815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f8157fff700 1 -- 172.21.15.97:0/705967885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8158159070 msgr2=0x7f81581f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f8157fff700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8158159070 0x7f81581f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f8157fff700 1 -- 172.21.15.97:0/705967885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814800b7f0 con 0x7f8158159440 2023-07-06T03:45:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f8156ffd700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8158159070 0x7f81581f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:32.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f8157fff700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8158159440 0x7f81580b42d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f815000cb50 tx=0x7f815000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:32.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f813ffff700 1 -- 172.21.15.97:0/705967885 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150003c00 con 0x7f8158159440 2023-07-06T03:45:32.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f813ffff700 1 -- 172.21.15.97:0/705967885 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8150003d60 con 0x7f8158159440 2023-07-06T03:45:32.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f815dfff700 1 -- 172.21.15.97:0/705967885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815815bf70 con 0x7f8158159440 2023-07-06T03:45:32.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f813ffff700 1 -- 172.21.15.97:0/705967885 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815000aac0 con 0x7f8158159440 2023-07-06T03:45:32.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.156+0000 7f815dfff700 1 -- 172.21.15.97:0/705967885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81580b1c50 con 0x7f8158159440 2023-07-06T03:45:32.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.157+0000 7f815dfff700 1 -- 172.21.15.97:0/705967885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81580b1e10 con 0x7f8158159440 2023-07-06T03:45:32.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.158+0000 7f813ffff700 1 -- 172.21.15.97:0/705967885 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f815000ac20 con 0x7f8158159440 2023-07-06T03:45:32.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.159+0000 7f813ffff700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f814406ace0 0x7f814406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.159+0000 7f813ffff700 1 -- 172.21.15.97:0/705967885 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f81500894e0 con 0x7f8158159440 2023-07-06T03:45:32.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.159+0000 7f81577fe700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f814406ace0 0x7f814406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.160+0000 7f81577fe700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f814406ace0 0x7f814406d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f814800bee0 tx=0x7f8148013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:32.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.163+0000 7f813ffff700 1 -- 172.21.15.97:0/705967885 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f81500544f0 con 0x7f8158159440 2023-07-06T03:45:32.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.357+0000 7f815dfff700 1 -- 172.21.15.97:0/705967885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81580b2130 con 0x7f8158159440 2023-07-06T03:45:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.358+0000 7f813ffff700 1 -- 172.21.15.97:0/705967885 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8150057b10 con 0x7f8158159440 2023-07-06T03:45:32.360 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:45:32.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.360+0000 7f815dfff700 1 -- 172.21.15.97:0/705967885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f814406ace0 msgr2=0x7f814406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.360+0000 7f815dfff700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f814406ace0 0x7f814406d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f814800bee0 tx=0x7f8148013000 comp rx=0 tx=0).stop 2023-07-06T03:45:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.360+0000 7f815dfff700 1 -- 172.21.15.97:0/705967885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8158159440 msgr2=0x7f81580b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.360+0000 7f815dfff700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8158159440 0x7f81580b42d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f815000cb50 tx=0x7f815000ce60 comp rx=0 tx=0).stop 2023-07-06T03:45:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.360+0000 7f815dfff700 1 -- 172.21.15.97:0/705967885 shutdown_connections 2023-07-06T03:45:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.360+0000 7f815dfff700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f814406ace0 0x7f814406d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.360+0000 7f815dfff700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8158159440 0x7f81580b42d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.360+0000 7f815dfff700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f81580c3cd0 0x7f815815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.360+0000 7f815dfff700 1 --2- 172.21.15.97:0/705967885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8158159070 0x7f81581f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.360+0000 7f815dfff700 1 -- 172.21.15.97:0/705967885 >> 172.21.15.97:0/705967885 conn(0x7f815806a060 msgr2=0x7f81580cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.361+0000 7f815dfff700 1 -- 172.21.15.97:0/705967885 shutdown_connections 2023-07-06T03:45:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.361+0000 7f815dfff700 1 -- 172.21.15.97:0/705967885 wait complete. 2023-07-06T03:45:32.380 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:45:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.572+0000 7fd287dff700 1 Processor -- start 2023-07-06T03:45:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.572+0000 7fd287dff700 1 -- start start 2023-07-06T03:45:32.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.573+0000 7fd287dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd28015ca10 0x7fd280160e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.573+0000 7fd287dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd2800cf3c0 0x7fd2800b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.573+0000 7fd287dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2800d1190 con 0x7fd2800b4ee0 2023-07-06T03:45:32.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.573+0000 7fd287dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2800b5360 con 0x7fd28015ca10 2023-07-06T03:45:32.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.573+0000 7fd287dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd280161340 con 0x7fd2800cf3c0 2023-07-06T03:45:32.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.574+0000 7fd285b9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd28015ca10 0x7fd280160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.574+0000 7fd285b9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd28015ca10 0x7fd280160e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52786/0 (socket says 172.21.15.97:52786) 2023-07-06T03:45:32.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.574+0000 7fd285b9b700 1 -- 172.21.15.97:0/772683404 learned_addr learned my addr 172.21.15.97:0/772683404 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:32.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.574+0000 7fd28539a700 1 --2- 172.21.15.97:0/772683404 >> v2:172.21.15.146:3300/0 conn(0x7fd2800cf3c0 0x7fd2800b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.574+0000 7fd285b9b700 1 -- 172.21.15.97:0/772683404 >> v2:172.21.15.146:3300/0 conn(0x7fd2800cf3c0 msgr2=0x7fd2800b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:32.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.574+0000 7fd285b9b700 1 --2- 172.21.15.97:0/772683404 >> v2:172.21.15.146:3300/0 conn(0x7fd2800cf3c0 0x7fd2800b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.574+0000 7fd285b9b700 1 -- 172.21.15.97:0/772683404 >> v1:172.21.15.97:6789/0 conn(0x7fd2800b4ee0 legacy=0x7fd2800cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.574+0000 7fd285b9b700 1 -- 172.21.15.97:0/772683404 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2800cd680 con 0x7fd28015ca10 2023-07-06T03:45:32.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.574+0000 7fd285b9b700 1 --2- 172.21.15.97:0/772683404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28015ca10 0x7fd280160e00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2800b1ec0 tx=0x7fd27c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34af54da731b1000 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:32.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.574+0000 7fd284b99700 1 -- 172.21.15.97:0/772683404 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c00b870 con 0x7fd28015ca10 2023-07-06T03:45:32.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.575+0000 7fd284b99700 1 -- 172.21.15.97:0/772683404 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c00b9d0 con 0x7fd28015ca10 2023-07-06T03:45:32.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.575+0000 7fd284b99700 1 -- 172.21.15.97:0/772683404 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c00bd60 con 0x7fd28015ca10 2023-07-06T03:45:32.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.575+0000 7fd287dff700 1 -- 172.21.15.97:0/772683404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28015ca10 msgr2=0x7fd280160e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.575+0000 7fd287dff700 1 --2- 172.21.15.97:0/772683404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28015ca10 0x7fd280160e00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2800b1ec0 tx=0x7fd27c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.575+0000 7fd287dff700 1 -- 172.21.15.97:0/772683404 shutdown_connections 2023-07-06T03:45:32.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.575+0000 7fd287dff700 1 --2- 172.21.15.97:0/772683404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28015ca10 0x7fd280160e00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.575+0000 7fd287dff700 1 --2- 172.21.15.97:0/772683404 >> v2:172.21.15.146:3300/0 conn(0x7fd2800cf3c0 0x7fd2800b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.575+0000 7fd287dff700 1 -- 172.21.15.97:0/772683404 >> 172.21.15.97:0/772683404 conn(0x7fd28006a920 msgr2=0x7fd28006ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:32.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.575+0000 7fd287dff700 1 -- 172.21.15.97:0/772683404 shutdown_connections 2023-07-06T03:45:32.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.575+0000 7fd287dff700 1 -- 172.21.15.97:0/772683404 wait complete. 2023-07-06T03:45:32.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.577+0000 7fd287dff700 1 Processor -- start 2023-07-06T03:45:32.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.577+0000 7fd287dff700 1 -- start start 2023-07-06T03:45:32.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.578+0000 7fd287dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2800b4ee0 0x7fd2800b8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.578+0000 7fd287dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2800cf3c0 0x7fd2801e6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.578+0000 7fd287dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd28015ca10 0x7fd2801e8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.578+0000 7fd287dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800c7a50 con 0x7fd2800cf3c0 2023-07-06T03:45:32.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.578+0000 7fd287dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800c7720 con 0x7fd2800b4ee0 2023-07-06T03:45:32.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.578+0000 7fd287dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2801e6cc0 con 0x7fd28015ca10 2023-07-06T03:45:32.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.578+0000 7fd28539a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2800cf3c0 0x7fd2801e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.578+0000 7fd28539a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2800cf3c0 0x7fd2801e6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52802/0 (socket says 172.21.15.97:52802) 2023-07-06T03:45:32.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.578+0000 7fd28539a700 1 -- 172.21.15.97:0/3556551646 learned_addr learned my addr 172.21.15.97:0/3556551646 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:32.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.578+0000 7fd285b9b700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2800b4ee0 0x7fd2800b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.578+0000 7fd28639c700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd28015ca10 0x7fd2801e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd28539a700 1 -- 172.21.15.97:0/3556551646 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd28015ca10 msgr2=0x7fd2801e8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd28539a700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd28015ca10 0x7fd2801e8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd28539a700 1 -- 172.21.15.97:0/3556551646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2800b4ee0 msgr2=0x7fd2800b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd28539a700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2800b4ee0 0x7fd2800b8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd28539a700 1 -- 172.21.15.97:0/3556551646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c009550 con 0x7fd2800cf3c0 2023-07-06T03:45:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd28639c700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd28015ca10 0x7fd2801e8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd28539a700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2800cf3c0 0x7fd2801e6220 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd27000d800 tx=0x7fd27000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:32.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd2767fc700 1 -- 172.21.15.97:0/3556551646 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700125f0 con 0x7fd2800cf3c0 2023-07-06T03:45:32.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd2767fc700 1 -- 172.21.15.97:0/3556551646 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd270012750 con 0x7fd2800cf3c0 2023-07-06T03:45:32.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd287dff700 1 -- 172.21.15.97:0/3556551646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2801e6fa0 con 0x7fd2800cf3c0 2023-07-06T03:45:32.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd2767fc700 1 -- 172.21.15.97:0/3556551646 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270012b00 con 0x7fd2800cf3c0 2023-07-06T03:45:32.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.579+0000 7fd287dff700 1 -- 172.21.15.97:0/3556551646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd280154520 con 0x7fd2800cf3c0 2023-07-06T03:45:32.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.580+0000 7fd287dff700 1 -- 172.21.15.97:0/3556551646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28006a790 con 0x7fd2800cf3c0 2023-07-06T03:45:32.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.581+0000 7fd2767fc700 1 -- 172.21.15.97:0/3556551646 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd270012c60 con 0x7fd2800cf3c0 2023-07-06T03:45:32.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.582+0000 7fd2767fc700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd26406ad10 0x7fd26406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:32.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.582+0000 7fd2767fc700 1 -- 172.21.15.97:0/3556551646 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd270089770 con 0x7fd2800cf3c0 2023-07-06T03:45:32.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.582+0000 7fd285b9b700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd26406ad10 0x7fd26406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:32.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.583+0000 7fd285b9b700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd26406ad10 0x7fd26406d1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd27c014040 tx=0x7fd27c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:32.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.586+0000 7fd2767fc700 1 -- 172.21.15.97:0/3556551646 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd270054810 con 0x7fd2800cf3c0 2023-07-06T03:45:32.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.828+0000 7fd287dff700 1 -- 172.21.15.97:0/3556551646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd28004e340 con 0x7fd2800cf3c0 2023-07-06T03:45:32.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.829+0000 7fd2767fc700 1 -- 172.21.15.97:0/3556551646 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4665 (secure 0 0 0) 0x7fd270057e30 con 0x7fd2800cf3c0 2023-07-06T03:45:32.831 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:32.831 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/4255451313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4255451313},{"type":"v1","addr":"172.21.15.97:6836","nonce":4255451313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/370322164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":370322164},{"type":"v1","addr":"172.21.15.97:6837","nonce":370322164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3764820847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3764820847},{"type":"v1","addr":"172.21.15.146:6835","nonce":3764820847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:45:11.126281+0000","modified":"2023-07-06T03:45:14.685499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6833/186544257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":186544257},{"type":"v1","addr":"172.21.15.146:6833","nonce":186544257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T03:45:32.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 -- 172.21.15.97:0/3556551646 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd26406ad10 msgr2=0x7fd26406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd26406ad10 0x7fd26406d1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd27c014040 tx=0x7fd27c008030 comp rx=0 tx=0).stop 2023-07-06T03:45:32.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 -- 172.21.15.97:0/3556551646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2800cf3c0 msgr2=0x7fd2801e6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:32.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2800cf3c0 0x7fd2801e6220 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd27000d800 tx=0x7fd27000db10 comp rx=0 tx=0).stop 2023-07-06T03:45:32.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 -- 172.21.15.97:0/3556551646 shutdown_connections 2023-07-06T03:45:32.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd26406ad10 0x7fd26406d1c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd28015ca10 0x7fd2801e8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd2800b4ee0 0x7fd2800b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 --2- 172.21.15.97:0/3556551646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2800cf3c0 0x7fd2801e6220 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:32.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 -- 172.21.15.97:0/3556551646 >> 172.21.15.97:0/3556551646 conn(0x7fd28006a920 msgr2=0x7fd2800b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:32.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 -- 172.21.15.97:0/3556551646 shutdown_connections 2023-07-06T03:45:32.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:32.831+0000 7fd287dff700 1 -- 172.21.15.97:0/3556551646 wait complete. 2023-07-06T03:45:32.835 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-07-06T03:45:32.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:33.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:45:33.014 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:45:33.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.045+0000 7fd58abff700 1 Processor -- start 2023-07-06T03:45:33.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.046+0000 7fd58abff700 1 -- start start 2023-07-06T03:45:33.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.047+0000 7fd58abff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5840cd390 0x7fd584160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.047+0000 7fd58abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd584160dd0 0x7fd5840b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.047+0000 7fd58abff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5840cdae0 con 0x7fd58415c520 2023-07-06T03:45:33.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.047+0000 7fd58abff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5840b6640 con 0x7fd5840cd390 2023-07-06T03:45:33.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.047+0000 7fd58abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840b6920 con 0x7fd584160dd0 2023-07-06T03:45:33.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.047+0000 7fd583fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5840cd390 0x7fd584160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.047+0000 7fd583fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5840cd390 0x7fd584160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52818/0 (socket says 172.21.15.97:52818) 2023-07-06T03:45:33.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.047+0000 7fd58899b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd58415c520 0x7fd584160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:34964/0 (socket says 172.21.15.97:34964) 2023-07-06T03:45:33.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.047+0000 7fd583fff700 1 -- 172.21.15.97:0/547070926 learned_addr learned my addr 172.21.15.97:0/547070926 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:33.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.048+0000 7fd58919c700 1 --2- 172.21.15.97:0/547070926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd584160dd0 0x7fd5840b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.048+0000 7fd583fff700 1 -- 172.21.15.97:0/547070926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd584160dd0 msgr2=0x7fd5840b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:33.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.048+0000 7fd583fff700 1 --2- 172.21.15.97:0/547070926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd584160dd0 0x7fd5840b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.048+0000 7fd583fff700 1 -- 172.21.15.97:0/547070926 >> v1:172.21.15.146:6789/0 conn(0x7fd58415c520 legacy=0x7fd584160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.048+0000 7fd583fff700 1 -- 172.21.15.97:0/547070926 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5840b6c00 con 0x7fd5840cd390 2023-07-06T03:45:33.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.048+0000 7fd58919c700 1 --2- 172.21.15.97:0/547070926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd584160dd0 0x7fd5840b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:33.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.048+0000 7fd583fff700 1 --2- 172.21.15.97:0/547070926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5840cd390 0x7fd584160890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098f0 tx=0x7fd570009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15be50bcac81c96 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:33.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.048+0000 7fd5837fe700 1 -- 172.21.15.97:0/547070926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000b8a0 con 0x7fd5840cd390 2023-07-06T03:45:33.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.048+0000 7fd5837fe700 1 -- 172.21.15.97:0/547070926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57000ba00 con 0x7fd5840cd390 2023-07-06T03:45:33.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.048+0000 7fd5837fe700 1 -- 172.21.15.97:0/547070926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000bd90 con 0x7fd5840cd390 2023-07-06T03:45:33.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.049+0000 7fd58abff700 1 -- 172.21.15.97:0/547070926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5840cd390 msgr2=0x7fd584160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.049+0000 7fd58abff700 1 --2- 172.21.15.97:0/547070926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5840cd390 0x7fd584160890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098f0 tx=0x7fd570009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:33.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.049+0000 7fd58abff700 1 -- 172.21.15.97:0/547070926 shutdown_connections 2023-07-06T03:45:33.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.049+0000 7fd58abff700 1 --2- 172.21.15.97:0/547070926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd584160dd0 0x7fd5840b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.049+0000 7fd58abff700 1 --2- 172.21.15.97:0/547070926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5840cd390 0x7fd584160890 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.049+0000 7fd58abff700 1 -- 172.21.15.97:0/547070926 >> 172.21.15.97:0/547070926 conn(0x7fd584071790 msgr2=0x7fd584071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:33.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.049+0000 7fd58abff700 1 -- 172.21.15.97:0/547070926 shutdown_connections 2023-07-06T03:45:33.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.049+0000 7fd58abff700 1 -- 172.21.15.97:0/547070926 wait complete. 2023-07-06T03:45:33.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.051+0000 7fd58abff700 1 Processor -- start 2023-07-06T03:45:33.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.051+0000 7fd58abff700 1 -- start start 2023-07-06T03:45:33.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58abff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5840cd390 0x7fd584156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd58415c520 0x7fd58415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58abff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584160dd0 0x7fd58415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58abff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58415e400 con 0x7fd5840cd390 2023-07-06T03:45:33.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58abff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58415db60 con 0x7fd584160dd0 2023-07-06T03:45:33.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58415de40 con 0x7fd58415c520 2023-07-06T03:45:33.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58899b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5840cd390 0x7fd584156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58899b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5840cd390 0x7fd584156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52820/0 (socket says 172.21.15.97:52820) 2023-07-06T03:45:33.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58899b700 1 -- 172.21.15.97:0/976666840 learned_addr learned my addr 172.21.15.97:0/976666840 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:33.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd583fff700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd58415c520 0x7fd58415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58919c700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584160dd0 0x7fd58415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58899b700 1 -- 172.21.15.97:0/976666840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd58415c520 msgr2=0x7fd58415cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58899b700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd58415c520 0x7fd58415cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58899b700 1 -- 172.21.15.97:0/976666840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584160dd0 msgr2=0x7fd58415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.052+0000 7fd58899b700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584160dd0 0x7fd58415d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.053+0000 7fd58899b700 1 -- 172.21.15.97:0/976666840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd570009550 con 0x7fd5840cd390 2023-07-06T03:45:33.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.053+0000 7fd58899b700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5840cd390 0x7fd584156050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd57800cc50 tx=0x7fd57800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:33.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.053+0000 7fd5817fa700 1 -- 172.21.15.97:0/976666840 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57800aad0 con 0x7fd5840cd390 2023-07-06T03:45:33.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.053+0000 7fd5817fa700 1 -- 172.21.15.97:0/976666840 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57800ac30 con 0x7fd5840cd390 2023-07-06T03:45:33.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.053+0000 7fd5817fa700 1 -- 172.21.15.97:0/976666840 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578007070 con 0x7fd5840cd390 2023-07-06T03:45:33.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.053+0000 7fd58abff700 1 -- 172.21.15.97:0/976666840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58415e120 con 0x7fd5840cd390 2023-07-06T03:45:33.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.053+0000 7fd58abff700 1 -- 172.21.15.97:0/976666840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd584156590 con 0x7fd5840cd390 2023-07-06T03:45:33.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.054+0000 7fd58abff700 1 -- 172.21.15.97:0/976666840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5840cb140 con 0x7fd5840cd390 2023-07-06T03:45:33.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.055+0000 7fd5817fa700 1 -- 172.21.15.97:0/976666840 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd5780071d0 con 0x7fd5840cd390 2023-07-06T03:45:33.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.056+0000 7fd5817fa700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd56806ace0 0x7fd56806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.057+0000 7fd5817fa700 1 -- 172.21.15.97:0/976666840 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd5780891d0 con 0x7fd5840cd390 2023-07-06T03:45:33.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.057+0000 7fd583fff700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd56806ace0 0x7fd56806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.057+0000 7fd583fff700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd56806ace0 0x7fd56806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd570009c40 tx=0x7fd570008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:33.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.061+0000 7fd5817fa700 1 -- 172.21.15.97:0/976666840 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd5840cb140 con 0x7fd5840cd390 2023-07-06T03:45:33.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.263+0000 7fd58abff700 1 -- 172.21.15.97:0/976666840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5840cb140 con 0x7fd5840cd390 2023-07-06T03:45:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.264+0000 7fd5817fa700 1 -- 172.21.15.97:0/976666840 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7fd5840cb140 con 0x7fd5840cd390 2023-07-06T03:45:33.266 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:33.266 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:33.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 -- 172.21.15.97:0/976666840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd56806ace0 msgr2=0x7fd56806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd56806ace0 0x7fd56806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd570009c40 tx=0x7fd570008030 comp rx=0 tx=0).stop 2023-07-06T03:45:33.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 -- 172.21.15.97:0/976666840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5840cd390 msgr2=0x7fd584156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5840cd390 0x7fd584156050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd57800cc50 tx=0x7fd57800cf60 comp rx=0 tx=0).stop 2023-07-06T03:45:33.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 -- 172.21.15.97:0/976666840 shutdown_connections 2023-07-06T03:45:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd56806ace0 0x7fd56806d190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd584160dd0 0x7fd58415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5840cd390 0x7fd584156050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 --2- 172.21.15.97:0/976666840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd58415c520 0x7fd58415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 -- 172.21.15.97:0/976666840 >> 172.21.15.97:0/976666840 conn(0x7fd584071790 msgr2=0x7fd5841582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 -- 172.21.15.97:0/976666840 shutdown_connections 2023-07-06T03:45:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.266+0000 7fd58abff700 1 -- 172.21.15.97:0/976666840 wait complete. 2023-07-06T03:45:33.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:45:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.478+0000 7fab735ff700 1 Processor -- start 2023-07-06T03:45:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.478+0000 7fab735ff700 1 -- start start 2023-07-06T03:45:33.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.479+0000 7fab735ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fab6c15c520 0x7fab6c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.479+0000 7fab735ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c0cdae0 con 0x7fab6c160d00 2023-07-06T03:45:33.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.479+0000 7fab735ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c0b6640 con 0x7fab6c160180 2023-07-06T03:45:33.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.479+0000 7fab735ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c0cd8d0 con 0x7fab6c15c520 2023-07-06T03:45:33.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab71b9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fab6c160d00 0x7fab6c0b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60962/0 (socket says 172.21.15.97:60962) 2023-07-06T03:45:33.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab71b9c700 1 -- 172.21.15.97:0/1608820611 learned_addr learned my addr 172.21.15.97:0/1608820611 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:33.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab7139b700 1 --2- 172.21.15.97:0/1608820611 >> v2:172.21.15.146:3300/0 conn(0x7fab6c15c520 0x7fab6c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970449418 0 0) 0x7fab6c0cdae0 con 0x7fab6c160d00 2023-07-06T03:45:33.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab580035b0 con 0x7fab6c160d00 2023-07-06T03:45:33.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2668034564 0 0) 0x7fab580035b0 con 0x7fab6c160d00 2023-07-06T03:45:33.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 >> v2:172.21.15.146:3300/0 conn(0x7fab6c15c520 msgr2=0x7fab6c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:33.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 --2- 172.21.15.97:0/1608820611 >> v2:172.21.15.146:3300/0 conn(0x7fab6c15c520 0x7fab6c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 >> v1:172.21.15.146:6789/0 conn(0x7fab6c160180 legacy=0x7fab6c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c0b6920 con 0x7fab6c160d00 2023-07-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fab640019f0 con 0x7fab6c160d00 2023-07-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 >> v1:172.21.15.97:6789/0 conn(0x7fab6c160d00 legacy=0x7fab6c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 --2- 172.21.15.97:0/1608820611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab58003a00 0x7fab58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 --2- 172.21.15.97:0/1608820611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab580043f0 0x7fab580048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 --2- 172.21.15.97:0/1608820611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab580050b0 0x7fab5800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0cd8d0 con 0x7fab58003a00 2023-07-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5800c000 con 0x7fab580043f0 2023-07-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.480+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5800c1c0 con 0x7fab580050b0 2023-07-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.481+0000 7fab7139b700 1 --2- 172.21.15.97:0/1608820611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab58003a00 0x7fab58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.481+0000 7fab70b9a700 1 --2- 172.21.15.97:0/1608820611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab580043f0 0x7fab580048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.481+0000 7fab71b9c700 1 --2- 172.21.15.97:0/1608820611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab580050b0 0x7fab5800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.481+0000 7fab7139b700 1 -- 172.21.15.97:0/1608820611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab580050b0 msgr2=0x7fab5800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.481+0000 7fab7139b700 1 --2- 172.21.15.97:0/1608820611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab580050b0 0x7fab5800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.481+0000 7fab7139b700 1 -- 172.21.15.97:0/1608820611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab580043f0 msgr2=0x7fab580048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:33.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.481+0000 7fab7139b700 1 --2- 172.21.15.97:0/1608820611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab580043f0 0x7fab580048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.481+0000 7fab7139b700 1 -- 172.21.15.97:0/1608820611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab580035b0 con 0x7fab58003a00 2023-07-06T03:45:33.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.481+0000 7fab71b9c700 1 -- 172.21.15.97:0/1608820611 reap_dead start 2023-07-06T03:45:33.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.481+0000 7fab7139b700 1 --2- 172.21.15.97:0/1608820611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab58003a00 0x7fab58003dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab68009b60 tx=0x7fab58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f079ad63c6f9b85 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:33.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.481+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6800e6c0 con 0x7fab58003a00 2023-07-06T03:45:33.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.482+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6800e820 con 0x7fab58003a00 2023-07-06T03:45:33.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.482+0000 7fab63fff700 1 -- 172.21.15.97:0/1608820611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6800eb90 con 0x7fab58003a00 2023-07-06T03:45:33.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.482+0000 7fab735ff700 1 -- 172.21.15.97:0/1608820611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab58003a00 msgr2=0x7fab58003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.482+0000 7fab735ff700 1 --2- 172.21.15.97:0/1608820611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab58003a00 0x7fab58003dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab68009b60 tx=0x7fab58005880 comp rx=0 tx=0).stop 2023-07-06T03:45:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.482+0000 7fab735ff700 1 -- 172.21.15.97:0/1608820611 shutdown_connections 2023-07-06T03:45:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.482+0000 7fab735ff700 1 --2- 172.21.15.97:0/1608820611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab58003a00 0x7fab58003dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.482+0000 7fab735ff700 1 -- 172.21.15.97:0/1608820611 >> 172.21.15.97:0/1608820611 conn(0x7fab6c071790 msgr2=0x7fab6c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.482+0000 7fab735ff700 1 -- 172.21.15.97:0/1608820611 shutdown_connections 2023-07-06T03:45:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.482+0000 7fab735ff700 1 -- 172.21.15.97:0/1608820611 wait complete. 2023-07-06T03:45:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.484+0000 7fab735ff700 1 Processor -- start 2023-07-06T03:45:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.484+0000 7fab735ff700 1 -- start start 2023-07-06T03:45:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.485+0000 7fab735ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab58003a00 0x7fab6c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.485+0000 7fab735ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab6c0c7dd0 0x7fab6c0d3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.485+0000 7fab735ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab6c0c93e0 0x7fab6c0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.485+0000 7fab735ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0c8340 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.485+0000 7fab735ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0c8620 con 0x7fab6c0c93e0 2023-07-06T03:45:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.485+0000 7fab735ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0c8900 con 0x7fab58003a00 2023-07-06T03:45:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.485+0000 7fab70b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab6c0c7dd0 0x7fab6c0d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.485+0000 7fab70b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab6c0c7dd0 0x7fab6c0d3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52848/0 (socket says 172.21.15.97:52848) 2023-07-06T03:45:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.485+0000 7fab70b9a700 1 -- 172.21.15.97:0/1325973609 learned_addr learned my addr 172.21.15.97:0/1325973609 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.485+0000 7fab7139b700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab58003a00 0x7fab6c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.485+0000 7fab71b9c700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab6c0c93e0 0x7fab6c0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.486+0000 7fab70b9a700 1 -- 172.21.15.97:0/1325973609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab58003a00 msgr2=0x7fab6c0c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.486+0000 7fab70b9a700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab58003a00 0x7fab6c0c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.486+0000 7fab70b9a700 1 -- 172.21.15.97:0/1325973609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab6c0c93e0 msgr2=0x7fab6c0cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.486+0000 7fab70b9a700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab6c0c93e0 0x7fab6c0cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.486+0000 7fab70b9a700 1 -- 172.21.15.97:0/1325973609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c0c8be0 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.486+0000 7fab70b9a700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab6c0c7dd0 0x7fab6c0d3570 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00f7a0 tx=0x7fab5c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.486+0000 7fab61ffb700 1 -- 172.21.15.97:0/1325973609 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0052e0 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.486+0000 7fab61ffb700 1 -- 172.21.15.97:0/1325973609 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c005460 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.486+0000 7fab61ffb700 1 -- 172.21.15.97:0/1325973609 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0038c0 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.486+0000 7fab735ff700 1 -- 172.21.15.97:0/1325973609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c0c8ec0 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.486+0000 7fab735ff700 1 -- 172.21.15.97:0/1325973609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c0d3ab0 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.487+0000 7fab735ff700 1 -- 172.21.15.97:0/1325973609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c04e340 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.488+0000 7fab61ffb700 1 -- 172.21.15.97:0/1325973609 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fab5c003a20 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.489+0000 7fab61ffb700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab5006ace0 0x7fab5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.489+0000 7fab61ffb700 1 -- 172.21.15.97:0/1325973609 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab5c0890e0 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.489+0000 7fab7139b700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab5006ace0 0x7fab5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.490+0000 7fab7139b700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab5006ace0 0x7fab5006d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fab680090b0 tx=0x7fab6801f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.493+0000 7fab61ffb700 1 -- 172.21.15.97:0/1325973609 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fab5c054250 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.731+0000 7fab735ff700 1 -- 172.21.15.97:0/1325973609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab6c0b4f40 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.731+0000 7fab61ffb700 1 -- 172.21.15.97:0/1325973609 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4665 (secure 0 0 0) 0x7fab5c057870 con 0x7fab6c0c7dd0 2023-07-06T03:45:33.733 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:33.733 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/4255451313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4255451313},{"type":"v1","addr":"172.21.15.97:6836","nonce":4255451313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/370322164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":370322164},{"type":"v1","addr":"172.21.15.97:6837","nonce":370322164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3764820847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3764820847},{"type":"v1","addr":"172.21.15.146:6835","nonce":3764820847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:45:11.126281+0000","modified":"2023-07-06T03:45:14.685499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6833/186544257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":186544257},{"type":"v1","addr":"172.21.15.146:6833","nonce":186544257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T03:45:33.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.733+0000 7fab735ff700 1 -- 172.21.15.97:0/1325973609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab5006ace0 msgr2=0x7fab5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.733+0000 7fab735ff700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab5006ace0 0x7fab5006d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fab680090b0 tx=0x7fab6801f040 comp rx=0 tx=0).stop 2023-07-06T03:45:33.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.733+0000 7fab735ff700 1 -- 172.21.15.97:0/1325973609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab6c0c7dd0 msgr2=0x7fab6c0d3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:33.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.733+0000 7fab735ff700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab6c0c7dd0 0x7fab6c0d3570 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00f7a0 tx=0x7fab5c00fab0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.734+0000 7fab735ff700 1 -- 172.21.15.97:0/1325973609 shutdown_connections 2023-07-06T03:45:33.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.734+0000 7fab735ff700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab5006ace0 0x7fab5006d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.734+0000 7fab735ff700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab6c0c93e0 0x7fab6c0cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.734+0000 7fab735ff700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab58003a00 0x7fab6c0c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.734+0000 7fab735ff700 1 --2- 172.21.15.97:0/1325973609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab6c0c7dd0 0x7fab6c0d3570 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:33.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.734+0000 7fab735ff700 1 -- 172.21.15.97:0/1325973609 >> 172.21.15.97:0/1325973609 conn(0x7fab6c071790 msgr2=0x7fab6c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:33.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.734+0000 7fab735ff700 1 -- 172.21.15.97:0/1325973609 shutdown_connections 2023-07-06T03:45:33.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:33.734+0000 7fab735ff700 1 -- 172.21.15.97:0/1325973609 wait complete. 2023-07-06T03:45:33.737 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-07-06T03:45:34.156 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T03:45:11.126281+0000', 'modified': '2023-07-06T03:45:14.685499+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6833/186544257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 186544257}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 186544257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T03:45:34.156 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T03:45:34.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-06T03:45:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.350+0000 7f635a9ff700 1 Processor -- start 2023-07-06T03:45:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.351+0000 7f635a9ff700 1 -- start start 2023-07-06T03:45:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f635a9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f635415c610 0x7f63540b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f635a9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f635406b360 0x7f63540b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f635a9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63540cff90 con 0x7f63540d2210 2023-07-06T03:45:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f635a9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63540d0780 con 0x7f635406b360 2023-07-06T03:45:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f635a9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635415c9e0 con 0x7f635415c610 2023-07-06T03:45:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f634bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f635406b360 0x7f63540b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f634bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f635406b360 0x7f63540b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52850/0 (socket says 172.21.15.97:52850) 2023-07-06T03:45:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f634bfff700 1 -- 172.21.15.97:0/3459582686 learned_addr learned my addr 172.21.15.97:0/3459582686 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f6353fff700 1 --2- 172.21.15.97:0/3459582686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f635415c610 0x7f63540b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f634bfff700 1 -- 172.21.15.97:0/3459582686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f635415c610 msgr2=0x7f63540b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f634bfff700 1 --2- 172.21.15.97:0/3459582686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f635415c610 0x7f63540b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f634bfff700 1 -- 172.21.15.97:0/3459582686 >> v1:172.21.15.146:6789/0 conn(0x7f63540d2210 legacy=0x7f63540d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.352+0000 7f634bfff700 1 -- 172.21.15.97:0/3459582686 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63540b5650 con 0x7f635406b360 2023-07-06T03:45:34.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.353+0000 7f634bfff700 1 --2- 172.21.15.97:0/3459582686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f635406b360 0x7f63540b4cf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f633c009920 tx=0x7f633c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c27b0e0f30767d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:34.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.353+0000 7f63537fe700 1 -- 172.21.15.97:0/3459582686 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c011830 con 0x7f635406b360 2023-07-06T03:45:34.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.353+0000 7f63537fe700 1 -- 172.21.15.97:0/3459582686 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c011990 con 0x7f635406b360 2023-07-06T03:45:34.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.353+0000 7f63537fe700 1 -- 172.21.15.97:0/3459582686 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c011d20 con 0x7f635406b360 2023-07-06T03:45:34.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.353+0000 7f635a9ff700 1 -- 172.21.15.97:0/3459582686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f635406b360 msgr2=0x7f63540b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.353+0000 7f635a9ff700 1 --2- 172.21.15.97:0/3459582686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f635406b360 0x7f63540b4cf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f633c009920 tx=0x7f633c009c30 comp rx=0 tx=0).stop 2023-07-06T03:45:34.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.353+0000 7f635a9ff700 1 -- 172.21.15.97:0/3459582686 shutdown_connections 2023-07-06T03:45:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.353+0000 7f635a9ff700 1 --2- 172.21.15.97:0/3459582686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f635415c610 0x7f63540b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.353+0000 7f635a9ff700 1 --2- 172.21.15.97:0/3459582686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f635406b360 0x7f63540b4cf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.353+0000 7f635a9ff700 1 -- 172.21.15.97:0/3459582686 >> 172.21.15.97:0/3459582686 conn(0x7f635406a6c0 msgr2=0x7f635406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.354+0000 7f635a9ff700 1 -- 172.21.15.97:0/3459582686 shutdown_connections 2023-07-06T03:45:34.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.354+0000 7f635a9ff700 1 -- 172.21.15.97:0/3459582686 wait complete. 2023-07-06T03:45:34.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.355+0000 7f635a9ff700 1 Processor -- start 2023-07-06T03:45:34.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.355+0000 7f635a9ff700 1 -- start start 2023-07-06T03:45:34.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.356+0000 7f635a9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f635406b360 0x7f635415f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.356+0000 7f635a9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63540d2210 0x7f6354155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.356+0000 7f635a9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f635415c610 0x7f63541f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.356+0000 7f635a9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540cbb20 con 0x7f635415c610 2023-07-06T03:45:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.356+0000 7f635a9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540caea0 con 0x7f635406b360 2023-07-06T03:45:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.356+0000 7f635a9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63540cb180 con 0x7f63540d2210 2023-07-06T03:45:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f6358f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f635415c610 0x7f63541f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f6358f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f635415c610 0x7f63541f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52852/0 (socket says 172.21.15.97:52852) 2023-07-06T03:45:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f6358f9c700 1 -- 172.21.15.97:0/55197450 learned_addr learned my addr 172.21.15.97:0/55197450 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f634bfff700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63540d2210 0x7f6354155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f6353fff700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f635406b360 0x7f635415f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f6358f9c700 1 -- 172.21.15.97:0/55197450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63540d2210 msgr2=0x7f6354155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f6358f9c700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63540d2210 0x7f6354155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f6358f9c700 1 -- 172.21.15.97:0/55197450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f635406b360 msgr2=0x7f635415f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f6358f9c700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f635406b360 0x7f635415f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f6358f9c700 1 -- 172.21.15.97:0/55197450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c009550 con 0x7f635415c610 2023-07-06T03:45:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f6358f9c700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f635415c610 0x7f63541f1f40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f634c00cc50 tx=0x7f634c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.357+0000 7f63517fa700 1 -- 172.21.15.97:0/55197450 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c003c00 con 0x7f635415c610 2023-07-06T03:45:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.358+0000 7f63517fa700 1 -- 172.21.15.97:0/55197450 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c003d60 con 0x7f635415c610 2023-07-06T03:45:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.358+0000 7f635a9ff700 1 -- 172.21.15.97:0/55197450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63540cb460 con 0x7f635415c610 2023-07-06T03:45:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.358+0000 7f63517fa700 1 -- 172.21.15.97:0/55197450 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c00aab0 con 0x7f635415c610 2023-07-06T03:45:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.358+0000 7f635a9ff700 1 -- 172.21.15.97:0/55197450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635415f940 con 0x7f635415c610 2023-07-06T03:45:34.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.360+0000 7f63517fa700 1 -- 172.21.15.97:0/55197450 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f634c00ac10 con 0x7f635415c610 2023-07-06T03:45:34.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.361+0000 7f63517fa700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f633406ace0 0x7f633406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.361+0000 7f63517fa700 1 -- 172.21.15.97:0/55197450 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f634c089420 con 0x7f635415c610 2023-07-06T03:45:34.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.361+0000 7f6353fff700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f633406ace0 0x7f633406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.361+0000 7f6353fff700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f633406ace0 0x7f633406d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6344009be0 tx=0x7f6344009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:34.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.363+0000 7f636e283b80 1 Processor -- start 2023-07-06T03:45:34.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.363+0000 7f636e283b80 1 -- start start 2023-07-06T03:45:34.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.364+0000 7f636e283b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c1084d50 0x55c6c1085120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.364+0000 7f636e283b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c108a190 0x55c6c10a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.364+0000 7f636e283b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c1062110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.364+0000 7f636e283b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0fd6ea0 con 0x55c6c105fb50 2023-07-06T03:45:34.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.364+0000 7f636e283b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0f4a600 con 0x55c6c108a190 2023-07-06T03:45:34.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.364+0000 7f636e283b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0fb9910 con 0x55c6c1084d50 2023-07-06T03:45:34.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.364+0000 7f6358f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c1062110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.364+0000 7f6358f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c1062110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52864/0 (socket says 172.21.15.97:52864) 2023-07-06T03:45:34.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.364+0000 7f6358f9c700 1 -- 172.21.15.97:0/2998213432 learned_addr learned my addr 172.21.15.97:0/2998213432 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:34.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.364+0000 7f634bfff700 1 --2- 172.21.15.97:0/2998213432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c108a190 0x55c6c10a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.364+0000 7f6353fff700 1 --2- 172.21.15.97:0/2998213432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c1084d50 0x55c6c1085120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.365+0000 7f6358f9c700 1 -- 172.21.15.97:0/2998213432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c1084d50 msgr2=0x55c6c1085120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:34.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.365+0000 7f6358f9c700 1 --2- 172.21.15.97:0/2998213432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c1084d50 0x55c6c1085120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.365+0000 7f6358f9c700 1 -- 172.21.15.97:0/2998213432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c108a190 msgr2=0x55c6c10a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.365+0000 7f6358f9c700 1 --2- 172.21.15.97:0/2998213432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c108a190 0x55c6c10a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.365+0000 7f6358f9c700 1 -- 172.21.15.97:0/2998213432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c0da6790 con 0x55c6c105fb50 2023-07-06T03:45:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.365+0000 7f6358f9c700 1 --2- 172.21.15.97:0/2998213432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c1062110 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f634c05ffa0 tx=0x7f634c057bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c59d2ec88e752c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.365+0000 7f6349ffb700 1 -- 172.21.15.97:0/2998213432 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c087860 con 0x55c6c105fb50 2023-07-06T03:45:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.365+0000 7f6349ffb700 1 -- 172.21.15.97:0/2998213432 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c0890c0 con 0x55c6c105fb50 2023-07-06T03:45:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.365+0000 7f6349ffb700 1 -- 172.21.15.97:0/2998213432 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c057870 con 0x55c6c105fb50 2023-07-06T03:45:34.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.366+0000 7f636e283b80 1 -- 172.21.15.97:0/2998213432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 msgr2=0x55c6c1062110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.366+0000 7f636e283b80 1 --2- 172.21.15.97:0/2998213432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c1062110 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f634c05ffa0 tx=0x7f634c057bb0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.366+0000 7f636e283b80 1 -- 172.21.15.97:0/2998213432 shutdown_connections 2023-07-06T03:45:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.366+0000 7f636e283b80 1 --2- 172.21.15.97:0/2998213432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c1062110 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.366+0000 7f636e283b80 1 --2- 172.21.15.97:0/2998213432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c1084d50 0x55c6c1085120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.366+0000 7f636e283b80 1 --2- 172.21.15.97:0/2998213432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c108a190 0x55c6c10a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.366+0000 7f636e283b80 1 -- 172.21.15.97:0/2998213432 >> 172.21.15.97:0/2998213432 conn(0x55c6c1057310 msgr2=0x55c6c10a75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.366+0000 7f636e283b80 1 -- 172.21.15.97:0/2998213432 shutdown_connections 2023-07-06T03:45:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.366+0000 7f636e283b80 1 -- 172.21.15.97:0/2998213432 wait complete. 2023-07-06T03:45:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.367+0000 7f636e283b80 1 -- start start 2023-07-06T03:45:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.367+0000 7f636e283b80 1 Processor -- start 2023-07-06T03:45:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f636e283b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c105fb50 0x55c6c1072770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f636e283b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c1084d50 0x55c6c1089aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f636e283b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c108a190 0x55c6c10b7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f636e283b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0fb2be0 con 0x55c6c1084d50 2023-07-06T03:45:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f636e283b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0f705b0 con 0x55c6c108a190 2023-07-06T03:45:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f636e283b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0f56910 con 0x55c6c105fb50 2023-07-06T03:45:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f636e283b80 10 client.0 _ll_register_callbacks cb 0x55c6c0f6eb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:45:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f634bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c1084d50 0x55c6c1089aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f634bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c1084d50 0x55c6c1089aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52874/0 (socket says 172.21.15.97:52874) 2023-07-06T03:45:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f634bfff700 1 -- 172.21.15.97:0/1889694030 learned_addr learned my addr 172.21.15.97:0/1889694030 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:34.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f6358f9c700 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c108a190 0x55c6c10b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.368+0000 7f6353fff700 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c105fb50 0x55c6c1072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f634bfff700 1 -- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c105fb50 msgr2=0x55c6c1072770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:34.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f634bfff700 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c105fb50 0x55c6c1072770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f634bfff700 1 -- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c108a190 msgr2=0x55c6c10b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:34.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f634bfff700 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c108a190 0x55c6c10b7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f634bfff700 1 -- 172.21.15.97:0/1889694030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635415f940 con 0x55c6c1084d50 2023-07-06T03:45:34.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f634bfff700 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c1084d50 0x55c6c1089aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f633c000c00 tx=0x7f633c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85dd573ef5a2ca11 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f6332ffd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:45:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f6332ffd700 1 -- 172.21.15.97:0/1889694030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c01f050 con 0x55c6c1084d50 2023-07-06T03:45:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f6332ffd700 1 -- 172.21.15.97:0/1889694030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f633c011e30 con 0x55c6c1084d50 2023-07-06T03:45:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f636e283b80 1 -- 172.21.15.97:0/1889694030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6c1054920 con 0x55c6c1084d50 2023-07-06T03:45:34.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f6332ffd700 1 -- 172.21.15.97:0/1889694030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f633c004450 con 0x55c6c1084d50 2023-07-06T03:45:34.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f636e283b80 10 client.4674 fetch_fsmap learned FSMap version 8 2023-07-06T03:45:34.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.369+0000 7f636e283b80 1 -- 172.21.15.97:0/1889694030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55c6c0e930a0 con 0x55c6c1084d50 2023-07-06T03:45:34.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.370+0000 7f6332ffd700 1 -- 172.21.15.97:0/1889694030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c6c1054920 con 0x55c6c1084d50 2023-07-06T03:45:34.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.370+0000 7f6332ffd700 1 -- 172.21.15.97:0/1889694030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f633c0036a0 con 0x55c6c1084d50 2023-07-06T03:45:34.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.370+0000 7f636e283b80 10 client.4674 fetch_fsmap finished waiting for FSMap version 8 2023-07-06T03:45:34.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.370+0000 7f636e283b80 10 client.4674 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-06T03:45:34.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.370+0000 7f636e283b80 20 client.4674 populate_metadata read hostname 'smithi097' 2023-07-06T03:45:34.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.370+0000 7f636e283b80 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:6832/186544257,v1:172.21.15.146:6833/186544257] conn(0x55c6c108b120 0x55c6c108b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.370+0000 7f636e283b80 4 client.4674 mds_command: new command op to 4290 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T03:45:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.370+0000 7f636e283b80 1 -- 172.21.15.97:0/1889694030 --> [v2:172.21.15.146:6832/186544257,v1:172.21.15.146:6833/186544257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6c10310d0 con 0x55c6c108b120 2023-07-06T03:45:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.370+0000 7f6353fff700 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:6832/186544257,v1:172.21.15.146:6833/186544257] conn(0x55c6c108b120 0x55c6c108b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.371+0000 7f6332ffd700 10 client.4674 ms_handle_connect on v2:172.21.15.146:6832/186544257 2023-07-06T03:45:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.371+0000 7f6353fff700 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:6832/186544257,v1:172.21.15.146:6833/186544257] conn(0x55c6c108b120 0x55c6c108b4f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe8351c8c50a78c server_cookie=9ad02cb8c71202d2 in_seq=0 out_seq=0 2023-07-06T03:45:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f6332ffd700 1 -- 172.21.15.97:0/1889694030 <== mds.0 v2:172.21.15.146:6832/186544257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c6c10310d0 con 0x55c6c108b120 2023-07-06T03:45:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f6332ffd700 10 client.4674 handle_command_reply: tid=0 2023-07-06T03:45:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 1 client.4674 shutdown 2023-07-06T03:45:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 1 -- 172.21.15.97:0/1889694030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c1084d50 msgr2=0x55c6c1089aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c1084d50 0x55c6c1089aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f633c000c00 tx=0x7f633c00e000 comp rx=0 tx=0).stop 2023-07-06T03:45:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 1 -- 172.21.15.97:0/1889694030 shutdown_connections 2023-07-06T03:45:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c108a190 0x55c6c10b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:6832/186544257,v1:172.21.15.146:6833/186544257] conn(0x55c6c108b120 0x55c6c108b4f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c105fb50 0x55c6c1072770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 1 --2- 172.21.15.97:0/1889694030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c1084d50 0x55c6c1089aa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f6332ffd700 0 client.4674 ms_handle_reset on v2:172.21.15.146:6832/186544257 2023-07-06T03:45:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 1 -- 172.21.15.97:0/1889694030 >> 172.21.15.97:0/1889694030 conn(0x55c6c1057310 msgr2=0x55c6c108a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 1 -- 172.21.15.97:0/1889694030 shutdown_connections 2023-07-06T03:45:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 1 -- 172.21.15.97:0/1889694030 wait complete. 2023-07-06T03:45:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.372+0000 7f636e283b80 20 client.4674 trim_cache size 0 max 16384 2023-07-06T03:45:34.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.390+0000 7f636e283b80 1 Processor -- start 2023-07-06T03:45:34.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.390+0000 7f636e283b80 1 -- start start 2023-07-06T03:45:34.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.391+0000 7f636e283b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c105ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.391+0000 7f636e283b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c1084d50 0x55c6c1072460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.391+0000 7f636e283b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c108a190 0x55c6c10729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.391+0000 7f636e283b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0da7390 con 0x55c6c105fb50 2023-07-06T03:45:34.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.391+0000 7f636e283b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c109e1a0 con 0x55c6c1084d50 2023-07-06T03:45:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.391+0000 7f636e283b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6c10a7df0 con 0x55c6c108a190 2023-07-06T03:45:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.391+0000 7f6353fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c105ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.391+0000 7f6353fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c105ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52876/0 (socket says 172.21.15.97:52876) 2023-07-06T03:45:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.391+0000 7f6353fff700 1 -- 172.21.15.97:0/2832675069 learned_addr learned my addr 172.21.15.97:0/2832675069 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.392+0000 7f6353fff700 1 -- 172.21.15.97:0/2832675069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c108a190 msgr2=0x55c6c10729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.392+0000 7f6353fff700 1 --2- 172.21.15.97:0/2832675069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c108a190 0x55c6c10729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.392+0000 7f6353fff700 1 -- 172.21.15.97:0/2832675069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c1084d50 msgr2=0x55c6c1072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.392+0000 7f6353fff700 1 --2- 172.21.15.97:0/2832675069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c1084d50 0x55c6c1072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.392+0000 7f6353fff700 1 -- 172.21.15.97:0/2832675069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c0f78b10 con 0x55c6c105fb50 2023-07-06T03:45:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.392+0000 7f6353fff700 1 --2- 172.21.15.97:0/2832675069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c105ed80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6344011b30 tx=0x7f6344019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927b3fc2e943992a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.392+0000 7f634a7fc700 1 -- 172.21.15.97:0/2832675069 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344030070 con 0x55c6c105fb50 2023-07-06T03:45:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.392+0000 7f634a7fc700 1 -- 172.21.15.97:0/2832675069 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634402c330 con 0x55c6c105fb50 2023-07-06T03:45:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.392+0000 7f634a7fc700 1 -- 172.21.15.97:0/2832675069 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344029040 con 0x55c6c105fb50 2023-07-06T03:45:34.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.393+0000 7f636e283b80 1 -- 172.21.15.97:0/2832675069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 msgr2=0x55c6c105ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.393+0000 7f636e283b80 1 --2- 172.21.15.97:0/2832675069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c105ed80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6344011b30 tx=0x7f6344019000 comp rx=0 tx=0).stop 2023-07-06T03:45:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.393+0000 7f636e283b80 1 -- 172.21.15.97:0/2832675069 shutdown_connections 2023-07-06T03:45:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.393+0000 7f636e283b80 1 --2- 172.21.15.97:0/2832675069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c108a190 0x55c6c10729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.393+0000 7f636e283b80 1 --2- 172.21.15.97:0/2832675069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c105fb50 0x55c6c105ed80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.393+0000 7f636e283b80 1 --2- 172.21.15.97:0/2832675069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c1084d50 0x55c6c1072460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.393+0000 7f636e283b80 1 -- 172.21.15.97:0/2832675069 >> 172.21.15.97:0/2832675069 conn(0x55c6c1057310 msgr2=0x55c6c105bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:34.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.393+0000 7f636e283b80 1 -- 172.21.15.97:0/2832675069 shutdown_connections 2023-07-06T03:45:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.393+0000 7f636e283b80 1 -- 172.21.15.97:0/2832675069 wait complete. 2023-07-06T03:45:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.393+0000 7f636e283b80 1 -- start start 2023-07-06T03:45:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.394+0000 7f636e283b80 1 Processor -- start 2023-07-06T03:45:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f636e283b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c105fb50 0x55c6c110de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f636e283b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c1084d50 0x55c6c109da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f636e283b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c108a190 0x55c6c105ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f636e283b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c109d910 con 0x55c6c108a190 2023-07-06T03:45:34.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f636e283b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c1082cd0 con 0x55c6c105fb50 2023-07-06T03:45:34.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f636e283b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6c108fc60 con 0x55c6c1084d50 2023-07-06T03:45:34.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f636e283b80 10 client.0 _ll_register_callbacks cb 0x7f633c0117b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:45:34.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6358f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c108a190 0x55c6c105ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6358f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c108a190 0x55c6c105ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52878/0 (socket says 172.21.15.97:52878) 2023-07-06T03:45:34.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6358f9c700 1 -- 172.21.15.97:0/1766999757 learned_addr learned my addr 172.21.15.97:0/1766999757 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:34.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6353fff700 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c105fb50 0x55c6c110de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f634bfff700 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c1084d50 0x55c6c109da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6358f9c700 1 -- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c1084d50 msgr2=0x55c6c109da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6358f9c700 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c1084d50 0x55c6c109da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6358f9c700 1 -- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c105fb50 msgr2=0x55c6c110de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6358f9c700 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c105fb50 0x55c6c110de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6358f9c700 1 -- 172.21.15.97:0/1766999757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6344011860 con 0x55c6c108a190 2023-07-06T03:45:34.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6353fff700 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c105fb50 0x55c6c110de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6358f9c700 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c108a190 0x55c6c105ce70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f634c00cfd0 tx=0x7f634c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7a95db161f133a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.395+0000 7f6332ffd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:45:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f6332ffd700 1 -- 172.21.15.97:0/1766999757 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c057870 con 0x55c6c108a190 2023-07-06T03:45:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f6332ffd700 1 -- 172.21.15.97:0/1766999757 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f634c087030 con 0x55c6c108a190 2023-07-06T03:45:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f636e283b80 1 -- 172.21.15.97:0/1766999757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f634c0890c0 con 0x55c6c108a190 2023-07-06T03:45:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f6332ffd700 1 -- 172.21.15.97:0/1766999757 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f634c089230 con 0x55c6c108a190 2023-07-06T03:45:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f636e283b80 10 client.4680 fetch_fsmap learned FSMap version 8 2023-07-06T03:45:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f636e283b80 1 -- 172.21.15.97:0/1766999757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55c6c0e071d0 con 0x55c6c108a190 2023-07-06T03:45:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f6332ffd700 1 -- 172.21.15.97:0/1766999757 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c0890c0 con 0x55c6c108a190 2023-07-06T03:45:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f6332ffd700 1 -- 172.21.15.97:0/1766999757 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f634c060040 con 0x55c6c108a190 2023-07-06T03:45:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f636e283b80 10 client.4680 fetch_fsmap finished waiting for FSMap version 8 2023-07-06T03:45:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f636e283b80 10 client.4680 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-06T03:45:34.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f636e283b80 20 client.4680 populate_metadata read hostname 'smithi097' 2023-07-06T03:45:34.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f636e283b80 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:6832/186544257,v1:172.21.15.146:6833/186544257] conn(0x55c6c10b5e90 0x55c6c10b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f636e283b80 4 client.4680 mds_command: new command op to 4290 tid=0[{"prefix": "status"}] 2023-07-06T03:45:34.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f636e283b80 1 -- 172.21.15.97:0/1766999757 --> [v2:172.21.15.146:6832/186544257,v1:172.21.15.146:6833/186544257] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c6c0fabd20 con 0x55c6c10b5e90 2023-07-06T03:45:34.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.396+0000 7f6353fff700 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:6832/186544257,v1:172.21.15.146:6833/186544257] conn(0x55c6c10b5e90 0x55c6c10b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.397+0000 7f6332ffd700 10 client.4680 ms_handle_connect on v2:172.21.15.146:6832/186544257 2023-07-06T03:45:34.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.397+0000 7f6353fff700 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:6832/186544257,v1:172.21.15.146:6833/186544257] conn(0x55c6c10b5e90 0x55c6c10b6260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee68f1609e05ef4 server_cookie=f83e59fd2a168c8d in_seq=0 out_seq=0 2023-07-06T03:45:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.397+0000 7f6332ffd700 1 -- 172.21.15.97:0/1766999757 <== mds.0 v2:172.21.15.146:6832/186544257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55c6c0fabd20 con 0x55c6c10b5e90 2023-07-06T03:45:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.397+0000 7f6332ffd700 10 client.4680 handle_command_reply: tid=0 2023-07-06T03:45:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.397+0000 7f636e283b80 1 client.4680 shutdown 2023-07-06T03:45:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.397+0000 7f636e283b80 1 -- 172.21.15.97:0/1766999757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c108a190 msgr2=0x55c6c105ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.397+0000 7f636e283b80 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c108a190 0x55c6c105ce70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f634c00cfd0 tx=0x7f634c0014a0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f636e283b80 1 -- 172.21.15.97:0/1766999757 shutdown_connections 2023-07-06T03:45:34.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f636e283b80 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6c108a190 0x55c6c105ce70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f636e283b80 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c6c105fb50 0x55c6c110de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f636e283b80 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:6832/186544257,v1:172.21.15.146:6833/186544257] conn(0x55c6c10b5e90 0x55c6c10b6260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f636e283b80 1 --2- 172.21.15.97:0/1766999757 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c6c1084d50 0x55c6c109da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f636e283b80 1 -- 172.21.15.97:0/1766999757 >> 172.21.15.97:0/1766999757 conn(0x55c6c1057310 msgr2=0x55c6c105ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:34.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f6332ffd700 0 client.4680 ms_handle_reset on v2:172.21.15.146:6832/186544257 2023-07-06T03:45:34.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f636e283b80 1 -- 172.21.15.97:0/1766999757 shutdown_connections 2023-07-06T03:45:34.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f636e283b80 1 -- 172.21.15.97:0/1766999757 wait complete. 2023-07-06T03:45:34.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f636e283b80 20 client.4680 trim_cache size 0 max 16384 2023-07-06T03:45:34.411 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:45:34.411 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36", 2023-07-06T03:45:34.412 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-07-06T03:45:34.412 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4290, 2023-07-06T03:45:34.412 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-07-06T03:45:34.412 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-07-06T03:45:34.412 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-07-06T03:45:34.413 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 22.735005183999998, 2023-07-06T03:45:34.413 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 8, 2023-07-06T03:45:34.413 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 23, 2023-07-06T03:45:34.413 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 23, 2023-07-06T03:45:34.413 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 29.072105049000001 2023-07-06T03:45:34.413 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:45:34.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f635a9ff700 1 -- 172.21.15.97:0/55197450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f633406ace0 msgr2=0x7f633406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f635a9ff700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f633406ace0 0x7f633406d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6344009be0 tx=0x7f6344009210 comp rx=0 tx=0).stop 2023-07-06T03:45:34.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f635a9ff700 1 -- 172.21.15.97:0/55197450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f635415c610 msgr2=0x7f63541f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.398+0000 7f635a9ff700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f635415c610 0x7f63541f1f40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f634c00cc50 tx=0x7f634c00cf60 comp rx=0 tx=0).stop 2023-07-06T03:45:34.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.399+0000 7f635a9ff700 1 -- 172.21.15.97:0/55197450 shutdown_connections 2023-07-06T03:45:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.399+0000 7f635a9ff700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f633406ace0 0x7f633406d190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.399+0000 7f635a9ff700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f635415c610 0x7f63541f1f40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.399+0000 7f635a9ff700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f635406b360 0x7f635415f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.399+0000 7f635a9ff700 1 --2- 172.21.15.97:0/55197450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63540d2210 0x7f6354155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.399+0000 7f635a9ff700 1 -- 172.21.15.97:0/55197450 >> 172.21.15.97:0/55197450 conn(0x7f635406a6c0 msgr2=0x7f63540d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.399+0000 7f635a9ff700 1 -- 172.21.15.97:0/55197450 shutdown_connections 2023-07-06T03:45:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.399+0000 7f635a9ff700 1 -- 172.21.15.97:0/55197450 wait complete. 2023-07-06T03:45:34.417 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-07-06T03:45:34.441 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-07-06T03:45:34.441 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-07-06T03:45:34.441 INFO:tasks.ceph_fuse:client.0 config is {} 2023-07-06T03:45:34.442 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T03:45:34.442 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:45:34.443 INFO:tasks.ceph_fuse:client.2 config is {} 2023-07-06T03:45:34.443 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T03:45:34.443 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-06T03:45:34.443 INFO:tasks.ceph_fuse:client.1 config is {} 2023-07-06T03:45:34.443 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T03:45:34.444 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T03:45:34.444 INFO:tasks.ceph_fuse:client.3 config is {} 2023-07-06T03:45:34.444 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T03:45:34.444 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-06T03:45:34.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:34.445 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:45:34.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:34.498 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2023-07-06T03:45:34.574 INFO:teuthology.orchestra.run.smithi146.stderr:Cannot find device "ceph-brx" 2023-07-06T03:45:34.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:45:34.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:34.578 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:45:34.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:34.625 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-07-06T03:45:34.698 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2023-07-06T03:45:34.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:45:34.701 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-07-06T03:45:34.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:45:34.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.895+0000 7ff5f39ff700 1 Processor -- start 2023-07-06T03:45:34.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.895+0000 7ff5f39ff700 1 -- start start 2023-07-06T03:45:34.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.896+0000 7ff5f39ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff5ec15b440 0x7ff5ec15e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.896+0000 7ff5f39ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5ec070c30 0x7ff5ec0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.896+0000 7ff5f39ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ec071330 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.896+0000 7ff5f39ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ec0ae3e0 con 0x7ff5ec070c30 2023-07-06T03:45:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.896+0000 7ff5f39ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ec071610 con 0x7ff5ec15b440 2023-07-06T03:45:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.896+0000 7ff5f1f9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5ec070c30 0x7ff5ec0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.896+0000 7ff5f179b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff5ec0cd8e0 0x7ff5ec0cdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:34982/0 (socket says 172.21.15.97:34982) 2023-07-06T03:45:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.897+0000 7ff5f179b700 1 -- 172.21.15.97:0/2658563827 learned_addr learned my addr 172.21.15.97:0/2658563827 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.897+0000 7ff5f0f9a700 1 --2- 172.21.15.97:0/2658563827 >> v2:172.21.15.146:3300/0 conn(0x7ff5ec15b440 0x7ff5ec15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.897+0000 7ff5f1f9c700 1 -- 172.21.15.97:0/2658563827 >> v2:172.21.15.146:3300/0 conn(0x7ff5ec15b440 msgr2=0x7ff5ec15e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.897+0000 7ff5f1f9c700 1 --2- 172.21.15.97:0/2658563827 >> v2:172.21.15.146:3300/0 conn(0x7ff5ec15b440 0x7ff5ec15e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.897+0000 7ff5f1f9c700 1 -- 172.21.15.97:0/2658563827 >> v1:172.21.15.146:6789/0 conn(0x7ff5ec0cd8e0 legacy=0x7ff5ec0cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.897+0000 7ff5e3fff700 1 -- 172.21.15.97:0/2658563827 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325575306 0 0) 0x7ff5ec071330 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.897+0000 7ff5f1f9c700 1 -- 172.21.15.97:0/2658563827 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec15dd80 con 0x7ff5ec070c30 2023-07-06T03:45:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.897+0000 7ff5f1f9c700 1 --2- 172.21.15.97:0/2658563827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec070c30 0x7ff5ec0adea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0b4970 tx=0x7ff5e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07653458ed46b61 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.897+0000 7ff5e3fff700 1 -- 172.21.15.97:0/2658563827 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400e070 con 0x7ff5ec070c30 2023-07-06T03:45:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.897+0000 7ff5e3fff700 1 -- 172.21.15.97:0/2658563827 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e400bb20 con 0x7ff5ec070c30 2023-07-06T03:45:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.897+0000 7ff5e3fff700 1 -- 172.21.15.97:0/2658563827 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4012430 con 0x7ff5ec070c30 2023-07-06T03:45:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.898+0000 7ff5f39ff700 1 -- 172.21.15.97:0/2658563827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec070c30 msgr2=0x7ff5ec0adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.898+0000 7ff5f39ff700 1 --2- 172.21.15.97:0/2658563827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec070c30 0x7ff5ec0adea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0b4970 tx=0x7ff5e400b330 comp rx=0 tx=0).stop 2023-07-06T03:45:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.898+0000 7ff5f39ff700 1 -- 172.21.15.97:0/2658563827 shutdown_connections 2023-07-06T03:45:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.898+0000 7ff5f39ff700 1 --2- 172.21.15.97:0/2658563827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec070c30 0x7ff5ec0adea0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.898+0000 7ff5f39ff700 1 --2- 172.21.15.97:0/2658563827 >> v2:172.21.15.146:3300/0 conn(0x7ff5ec15b440 0x7ff5ec15e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.898+0000 7ff5f39ff700 1 -- 172.21.15.97:0/2658563827 >> 172.21.15.97:0/2658563827 conn(0x7ff5ec0ce8c0 msgr2=0x7ff5ec0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.898+0000 7ff5f39ff700 1 -- 172.21.15.97:0/2658563827 shutdown_connections 2023-07-06T03:45:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.898+0000 7ff5f39ff700 1 -- 172.21.15.97:0/2658563827 wait complete. 2023-07-06T03:45:34.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.900+0000 7ff5f39ff700 1 Processor -- start 2023-07-06T03:45:34.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.900+0000 7ff5f39ff700 1 -- start start 2023-07-06T03:45:34.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.901+0000 7ff5f39ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5ec070c30 0x7ff5ec15a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.901+0000 7ff5f39ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec0cd8e0 0x7ff5ec14fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.901+0000 7ff5f39ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ec15b440 0x7ff5ec13df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.901+0000 7ff5f39ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec157820 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.901+0000 7ff5f39ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec156ba0 con 0x7ff5ec15b440 2023-07-06T03:45:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.901+0000 7ff5f39ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec156e80 con 0x7ff5ec070c30 2023-07-06T03:45:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.901+0000 7ff5f0f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec0cd8e0 0x7ff5ec14fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.901+0000 7ff5f0f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec0cd8e0 0x7ff5ec14fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52898/0 (socket says 172.21.15.97:52898) 2023-07-06T03:45:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.901+0000 7ff5f0f9a700 1 -- 172.21.15.97:0/1738558984 learned_addr learned my addr 172.21.15.97:0/1738558984 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.901+0000 7ff5f179b700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5ec070c30 0x7ff5ec15a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.901+0000 7ff5f1f9c700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ec15b440 0x7ff5ec13df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5f0f9a700 1 -- 172.21.15.97:0/1738558984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5ec070c30 msgr2=0x7ff5ec15a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5f0f9a700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5ec070c30 0x7ff5ec15a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5f0f9a700 1 -- 172.21.15.97:0/1738558984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ec15b440 msgr2=0x7ff5ec13df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5f0f9a700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ec15b440 0x7ff5ec13df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5f0f9a700 1 -- 172.21.15.97:0/1738558984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e4009a70 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5f1f9c700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ec15b440 0x7ff5ec13df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5f0f9a700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec0cd8e0 0x7ff5ec14fef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00e980 tx=0x7ff5dc00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5e1ffb700 1 -- 172.21.15.97:0/1738558984 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc009710 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5e1ffb700 1 -- 172.21.15.97:0/1738558984 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5dc00c690 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5e1ffb700 1 -- 172.21.15.97:0/1738558984 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00ca20 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5f39ff700 1 -- 172.21.15.97:0/1738558984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ec157160 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.902+0000 7ff5f39ff700 1 -- 172.21.15.97:0/1738558984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ec156830 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.903+0000 7ff5f39ff700 1 -- 172.21.15.97:0/1738558984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ec070aa0 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.904+0000 7ff5e1ffb700 1 -- 172.21.15.97:0/1738558984 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff5dc00cb80 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.905+0000 7ff5e1ffb700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5d006ace0 0x7ff5d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.905+0000 7ff5e1ffb700 1 -- 172.21.15.97:0/1738558984 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5dc0894e0 con 0x7ff5ec0cd8e0 2023-07-06T03:45:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.905+0000 7ff5f179b700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5d006ace0 0x7ff5d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:34.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.906+0000 7ff5f179b700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5d006ace0 0x7ff5d006d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8007b10 tx=0x7ff5e8007a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:34.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:34.909+0000 7ff5e1ffb700 1 -- 172.21.15.97:0/1738558984 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff5dc057f90 con 0x7ff5ec0cd8e0 2023-07-06T03:45:35.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.150+0000 7ff5f39ff700 1 -- 172.21.15.97:0/1738558984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff5ec157960 con 0x7ff5ec0cd8e0 2023-07-06T03:45:35.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.150+0000 7ff5e1ffb700 1 -- 172.21.15.97:0/1738558984 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff5ec157960 con 0x7ff5ec0cd8e0 2023-07-06T03:45:35.152 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:45:35.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.152+0000 7ff5f39ff700 1 -- 172.21.15.97:0/1738558984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5d006ace0 msgr2=0x7ff5d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.152+0000 7ff5f39ff700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5d006ace0 0x7ff5d006d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8007b10 tx=0x7ff5e8007a60 comp rx=0 tx=0).stop 2023-07-06T03:45:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.152+0000 7ff5f39ff700 1 -- 172.21.15.97:0/1738558984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec0cd8e0 msgr2=0x7ff5ec14fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.152+0000 7ff5f39ff700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec0cd8e0 0x7ff5ec14fef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00e980 tx=0x7ff5dc00ec90 comp rx=0 tx=0).stop 2023-07-06T03:45:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.153+0000 7ff5f39ff700 1 -- 172.21.15.97:0/1738558984 shutdown_connections 2023-07-06T03:45:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.153+0000 7ff5f39ff700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5d006ace0 0x7ff5d006d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.153+0000 7ff5f39ff700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ec15b440 0x7ff5ec13df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.153+0000 7ff5f39ff700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5ec070c30 0x7ff5ec15a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:35.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.153+0000 7ff5f39ff700 1 --2- 172.21.15.97:0/1738558984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5ec0cd8e0 0x7ff5ec14fef0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:35.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.153+0000 7ff5f39ff700 1 -- 172.21.15.97:0/1738558984 >> 172.21.15.97:0/1738558984 conn(0x7ff5ec0ce8c0 msgr2=0x7ff5ec0ad6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:35.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.153+0000 7ff5f39ff700 1 -- 172.21.15.97:0/1738558984 shutdown_connections 2023-07-06T03:45:35.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:35.153+0000 7ff5f39ff700 1 -- 172.21.15.97:0/1738558984 wait complete. 2023-07-06T03:45:35.172 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:45:35.172 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:45:35.173 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:45:35.173 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:45:35.173 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:45:35.173 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:45:35.173 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:45:35.174 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:45:35.174 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:45:35.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:35.174 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:45:35.220 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:45:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:45:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:45:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:45:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:35.221 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:45:35.221 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:45:35.222 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:35.222 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:45:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:45:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42134sec preferred_lft 42134sec 2023-07-06T03:45:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:45:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:35.224 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T03:45:35.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:35.225 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:45:35.225 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:45:35.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2023-07-06T03:45:35.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2023-07-06T03:45:35.226 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2023-07-06T03:45:35.226 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T03:45:35.227 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:45:35.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:35.674 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:45:35.774 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:45:35.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:35.777 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:45:35.844 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:45:35.844 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:45:35.845 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:45:35.845 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:45:35.845 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:45:35.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:35.846 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:45:35.847 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:45:35.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:45:35.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:45:35.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:45:35.848 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:45:36.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:36.514 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:45:36.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:36.561 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2023-07-06T03:45:36.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:36.648 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:45:36.648 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:45:36.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:45:36.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-06T03:45:36.649 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:45:36.829 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-06T03:45:36.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:36.831 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:45:36.831 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:45:36.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-06T03:45:36.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T03:45:36.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-06T03:45:36.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-06T03:45:36.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-06T03:45:36.833 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:45:37.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:37.098 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:45:37.098 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:45:37.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:45:37.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T03:45:37.099 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:45:37.277 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:45:37.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:45:37.278 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:45:37.323 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:45:37.324 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:45:37.324 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:45:37.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:45:37.517 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:45:37.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:45:37.585 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:45:37.588 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:45:37.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:37.589 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:45:37.635 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:45:37.636 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:45:37.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:45:37.751 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:45:37.752 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:45:37.820 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:45:37.822 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:45:37.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:37.823 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:45:37.848 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:45:37.846+0000 7f2b9ae0f5c0 -1 init, newargv = 0x556ab404c150 newargc=15 2023-07-06T03:45:37.848 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[122054]: starting ceph client 2023-07-06T03:45:37.857 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[122054]: starting fuse 2023-07-06T03:45:37.874 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:45:37.874 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:45:37.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:45:37.875 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:45:37.875 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:45:37.876 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:45:37.876 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:45:37.876 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:45:37.877 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:45:37.877 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:45:37.877 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:45:37.877 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:45:37.877 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:45:37.878 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:45:37.878 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:45:37.878 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:45:37.878 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:45:37.879 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:45:37.879 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:45:37.879 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:45:37.879 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:45:37.879 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:45:37.880 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:45:37.880 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:45:37.880 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:45:37.880 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:45:37.881 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:45:37.881 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:45:37.881 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:45:38.057 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.122054.asok 2023-07-06T03:45:38.235 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.122054.asok 2023-07-06T03:45:38.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:38.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.122054.asok status 2023-07-06T03:45:38.458 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:45:38.459 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:45:38.459 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:45:38.459 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:45:38.459 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:45:38.459 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:45:38.460 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:45:38.460 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "122054", 2023-07-06T03:45:38.460 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:45:38.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:45:38.460 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:45:38.460 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:45:38.460 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4692, 2023-07-06T03:45:38.460 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:45:38.461 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:45:38.461 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:45:38.461 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 4692 2023-07-06T03:45:38.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:45:38.461 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:45:38.461 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:45:38.461 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:45:38.461 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 607360895 2023-07-06T03:45:38.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:45:38.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:45:38.462 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:45:38.462 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:45:38.462 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:45:38.462 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 607360895 2023-07-06T03:45:38.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:45:38.462 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.4692 192.168.0.1:0/607360895", 2023-07-06T03:45:38.463 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/607360895", 2023-07-06T03:45:38.463 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:45:38.463 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 8, 2023-07-06T03:45:38.463 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 23, 2023-07-06T03:45:38.463 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:45:38.463 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:45:38.464 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:45:38.464 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:45:38.474 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:45:38.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.666+0000 7fa7f97ff700 1 Processor -- start 2023-07-06T03:45:38.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.667+0000 7fa7f97ff700 1 -- start start 2023-07-06T03:45:38.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.668+0000 7fa7f97ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 0x7fa7f4160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:38.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.668+0000 7fa7f97ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa7f4160dd0 0x7fa7f40b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:38.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.668+0000 7fa7f97ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f40cdae0 con 0x7fa7f415c520 2023-07-06T03:45:38.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.668+0000 7fa7f97ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f40b6640 con 0x7fa7f4160dd0 2023-07-06T03:45:38.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.668+0000 7fa7f97ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40b6920 con 0x7fa7f40cd390 2023-07-06T03:45:38.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.668+0000 7fa7f27fc700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 0x7fa7f4160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:38.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.668+0000 7fa7f2ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa7f415c520 0x7fa7f4160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:34992/0 (socket says 172.21.15.97:34992) 2023-07-06T03:45:38.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.669+0000 7fa7f27fc700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 0x7fa7f4160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:34368/0 (socket says 172.21.15.97:34368) 2023-07-06T03:45:38.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.669+0000 7fa7f2ffd700 1 -- 172.21.15.97:0/1212843600 learned_addr learned my addr 172.21.15.97:0/1212843600 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:38.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.669+0000 7fa7f37fe700 1 --2- 172.21.15.97:0/1212843600 >> v2:172.21.15.146:3300/0 conn(0x7fa7f4160dd0 0x7fa7f40b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:38.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.669+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/1212843600 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865479183 0 0) 0x7fa7f40cdae0 con 0x7fa7f415c520 2023-07-06T03:45:38.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.669+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/1212843600 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7dc0035b0 con 0x7fa7f415c520 2023-07-06T03:45:38.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.669+0000 7fa7f37fe700 1 -- 172.21.15.97:0/1212843600 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 msgr2=0x7fa7f4160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:38.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.669+0000 7fa7f37fe700 1 --2- 172.21.15.97:0/1212843600 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 0x7fa7f4160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:38.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.669+0000 7fa7f37fe700 1 -- 172.21.15.97:0/1212843600 >> v1:172.21.15.146:6789/0 conn(0x7fa7f415c520 legacy=0x7fa7f4160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:38.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.669+0000 7fa7f37fe700 1 -- 172.21.15.97:0/1212843600 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40b6c00 con 0x7fa7f4160dd0 2023-07-06T03:45:38.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.669+0000 7fa7f27fc700 1 --2- 172.21.15.97:0/1212843600 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 0x7fa7f4160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:38.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.669+0000 7fa7f37fe700 1 --2- 172.21.15.97:0/1212843600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f4160dd0 0x7fa7f40b7400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009d90 tx=0x7fa7ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b4cf0d7a37e3f39 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:38.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.670+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/1212843600 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00e070 con 0x7fa7f4160dd0 2023-07-06T03:45:38.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.670+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/1212843600 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec00bb20 con 0x7fa7f4160dd0 2023-07-06T03:45:38.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.670+0000 7fa7f1ffb700 1 -- 172.21.15.97:0/1212843600 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec012430 con 0x7fa7f4160dd0 2023-07-06T03:45:38.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.670+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1212843600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f4160dd0 msgr2=0x7fa7f40b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:38.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.670+0000 7fa7f97ff700 1 --2- 172.21.15.97:0/1212843600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f4160dd0 0x7fa7f40b7400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009d90 tx=0x7fa7ec00b330 comp rx=0 tx=0).stop 2023-07-06T03:45:38.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.670+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1212843600 shutdown_connections 2023-07-06T03:45:38.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.670+0000 7fa7f97ff700 1 --2- 172.21.15.97:0/1212843600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f4160dd0 0x7fa7f40b7400 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:38.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.670+0000 7fa7f97ff700 1 --2- 172.21.15.97:0/1212843600 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 0x7fa7f4160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:38.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.670+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1212843600 >> 172.21.15.97:0/1212843600 conn(0x7fa7f4071790 msgr2=0x7fa7f4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:38.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.670+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1212843600 shutdown_connections 2023-07-06T03:45:38.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.670+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1212843600 wait complete. 2023-07-06T03:45:38.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.672+0000 7fa7f97ff700 1 Processor -- start 2023-07-06T03:45:38.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.673+0000 7fa7f97ff700 1 -- start start 2023-07-06T03:45:38.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.674+0000 7fa7f97ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 0x7fa7f414c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:38.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.674+0000 7fa7f97ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f415c520 0x7fa7f415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:38.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.674+0000 7fa7f97ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f4160dd0 0x7fa7f41586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:38.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.674+0000 7fa7f97ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f415b240 con 0x7fa7f4160dd0 2023-07-06T03:45:38.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.674+0000 7fa7f97ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f415a910 con 0x7fa7f415c520 2023-07-06T03:45:38.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.674+0000 7fa7f97ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f415abf0 con 0x7fa7f40cd390 2023-07-06T03:45:38.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.674+0000 7fa7f37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f4160dd0 0x7fa7f41586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.674+0000 7fa7f37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f4160dd0 0x7fa7f41586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52902/0 (socket says 172.21.15.97:52902) 2023-07-06T03:45:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.674+0000 7fa7f37fe700 1 -- 172.21.15.97:0/1280128952 learned_addr learned my addr 172.21.15.97:0/1280128952 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.674+0000 7fa7f27fc700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f415c520 0x7fa7f415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.674+0000 7fa7f2ffd700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 0x7fa7f414c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.675+0000 7fa7f37fe700 1 -- 172.21.15.97:0/1280128952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 msgr2=0x7fa7f414c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.675+0000 7fa7f37fe700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 0x7fa7f414c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.675+0000 7fa7f37fe700 1 -- 172.21.15.97:0/1280128952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f415c520 msgr2=0x7fa7f415b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.675+0000 7fa7f37fe700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f415c520 0x7fa7f415b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:38.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.675+0000 7fa7f37fe700 1 -- 172.21.15.97:0/1280128952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e80094a0 con 0x7fa7f4160dd0 2023-07-06T03:45:38.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.675+0000 7fa7f37fe700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f4160dd0 0x7fa7f41586f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec014040 tx=0x7fa7ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:38.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.675+0000 7fa7db7fe700 1 -- 172.21.15.97:0/1280128952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0129f0 con 0x7fa7f4160dd0 2023-07-06T03:45:38.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.675+0000 7fa7db7fe700 1 -- 172.21.15.97:0/1280128952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec020070 con 0x7fa7f4160dd0 2023-07-06T03:45:38.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.675+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1280128952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ec009a70 con 0x7fa7f4160dd0 2023-07-06T03:45:38.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.675+0000 7fa7db7fe700 1 -- 172.21.15.97:0/1280128952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec012db0 con 0x7fa7f4160dd0 2023-07-06T03:45:38.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.675+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1280128952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f415bc10 con 0x7fa7f4160dd0 2023-07-06T03:45:38.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.677+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1280128952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f415b3b0 con 0x7fa7f4160dd0 2023-07-06T03:45:38.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.678+0000 7fa7db7fe700 1 -- 172.21.15.97:0/1280128952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7ec004040 con 0x7fa7f4160dd0 2023-07-06T03:45:38.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.678+0000 7fa7db7fe700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 0x7fa7e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:38.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.678+0000 7fa7db7fe700 1 -- 172.21.15.97:0/1280128952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7ec022070 con 0x7fa7f4160dd0 2023-07-06T03:45:38.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.679+0000 7fa7f2ffd700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 0x7fa7e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:38.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.679+0000 7fa7f2ffd700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 0x7fa7e006d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4007fd0 tx=0x7fa7e4007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:38.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.683+0000 7fa7db7fe700 1 -- 172.21.15.97:0/1280128952 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa7ec056510 con 0x7fa7f4160dd0 2023-07-06T03:45:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.922+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1280128952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa7f404e340 con 0x7fa7f4160dd0 2023-07-06T03:45:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.922+0000 7fa7db7fe700 1 -- 172.21.15.97:0/1280128952 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa7ec059b30 con 0x7fa7f4160dd0 2023-07-06T03:45:38.925 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:45:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1280128952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 msgr2=0x7fa7e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 0x7fa7e006d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4007fd0 tx=0x7fa7e4007dc0 comp rx=0 tx=0).stop 2023-07-06T03:45:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1280128952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f4160dd0 msgr2=0x7fa7f41586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f4160dd0 0x7fa7f41586f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec014040 tx=0x7fa7ec004ad0 comp rx=0 tx=0).stop 2023-07-06T03:45:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1280128952 shutdown_connections 2023-07-06T03:45:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7e006ace0 0x7fa7e006d190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f4160dd0 0x7fa7f41586f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7f40cd390 0x7fa7f414c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 --2- 172.21.15.97:0/1280128952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7f415c520 0x7fa7f415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1280128952 >> 172.21.15.97:0/1280128952 conn(0x7fa7f4071790 msgr2=0x7fa7f415eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1280128952 shutdown_connections 2023-07-06T03:45:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:38.925+0000 7fa7f97ff700 1 -- 172.21.15.97:0/1280128952 wait complete. 2023-07-06T03:45:38.945 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:45:38.945 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:45:38.945 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:45:38.945 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-07-06T03:45:38.946 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-06T03:45:38.946 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:45:38.946 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:45:38.946 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:45:38.946 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-07-06T03:45:38.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:38.947 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:45:38.993 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:45:38.993 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:45:38.993 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:45:38.993 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:38.993 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:45:38.994 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:38.994 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:38.994 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:38.994 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:45:38.994 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:38.994 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:38.994 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:45:38.995 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:38.995 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:38.995 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:45:38.995 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:38.995 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:45:38.995 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42130sec preferred_lft 42130sec 2023-07-06T03:45:38.995 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:45:38.996 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:38.996 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:45:38.996 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:38.997 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:45:38.997 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:38.997 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:45:38.998 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:38.998 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:45:38.998 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:45:38.999 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:45:38.999 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:39.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:39.001 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:45:39.068 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:45:39.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:39.071 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:45:39.119 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:45:39.119 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:45:39.120 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:45:39.120 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:45:39.120 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:45:39.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:39.121 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:45:39.122 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:45:39.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:45:39.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:45:39.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:45:39.123 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:45:39.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:39.290 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:45:39.337 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:45:39.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:39.338 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2023-07-06T03:45:39.383 INFO:teuthology.orchestra.run.smithi097.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-07-06T03:45:39.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:39.385 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:45:39.385 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:45:39.385 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:45:39.385 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-07-06T03:45:39.386 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:45:39.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:39.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-06T03:45:39.593 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:45:39.593 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:45:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:45:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:45:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:39.594 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:45:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 56:89:e1:99:ba:05 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T03:45:39.594 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T03:45:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5489:e1ff:fe99:ba05/64 scope link 2023-07-06T03:45:39.595 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:39.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:39.596 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-06T03:45:39.666 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:45:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:45:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:45:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:45:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:39.667 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:45:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 56:89:e1:99:ba:05 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T03:45:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T03:45:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5489:e1ff:fe99:ba05/64 scope link 2023-07-06T03:45:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:39.669 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-07-06T03:45:39.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:39.670 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:45:39.670 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:45:39.670 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-07-06T03:45:39.670 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-06T03:45:39.670 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-07-06T03:45:39.671 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-07-06T03:45:39.671 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-07-06T03:45:39.671 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:45:39.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:39.916 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:45:39.917 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:45:39.917 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.1 up 2023-07-06T03:45:39.917 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.1 master ceph-brx 2023-07-06T03:45:39.918 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:45:40.096 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-07-06T03:45:40.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:45:40.096 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-07-06T03:45:40.147 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-07-06T03:45:40.148 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:45:40.148 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-07-06T03:45:40.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:45:40.340 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:45:40.341 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:45:40.407 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:45:40.410 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:45:40.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:40.411 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:45:40.456 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:45:40.456 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-07-06T03:45:40.457 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-07-06T03:45:40.501 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:45:40.575 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:45:40.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:45:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:45:40.645 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:45:40.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:40.646 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:45:40.669 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi097.stderr:2023-07-06T03:45:40.667+0000 7fec1be805c0 -1 init, newargv = 0x5556062e5110 newargc=15 2023-07-06T03:45:40.669 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi097.stderr:ceph-fuse[122778]: starting ceph client 2023-07-06T03:45:40.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi097.stderr:ceph-fuse[122778]: starting fuse 2023-07-06T03:45:40.700 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:45:40.701 INFO:teuthology.orchestra.run.smithi097.stdout:44 2023-07-06T03:45:40.701 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-07-06T03:45:40.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:45:40.702 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:45:40.702 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:45:40.702 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:45:40.703 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:45:40.703 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:45:40.703 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:45:40.704 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-07-06T03:45:40.704 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:45:40.704 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-07-06T03:45:40.705 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:45:40.705 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:45:40.706 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:45:40.706 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:45:40.707 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:45:40.708 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:45:40.708 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:45:40.709 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:45:40.709 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:45:40.710 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:45:40.710 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:45:40.711 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:45:40.711 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:45:40.711 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:45:40.711 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:45:40.712 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.2")) 2023-07-06T03:45:40.712 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:45:40.883 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.2.122778.asok 2023-07-06T03:45:41.060 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.122778.asok 2023-07-06T03:45:41.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:41.061 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.122778.asok status 2023-07-06T03:45:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:45:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:45:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:45:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:45:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "2", 2023-07-06T03:45:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:45:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-07-06T03:45:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "122778", 2023-07-06T03:45:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:45:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:45:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:45:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:45:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4701, 2023-07-06T03:45:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:45:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:45:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:45:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 4701 2023-07-06T03:45:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:45:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:45:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:45:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.2:0", 2023-07-06T03:45:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1009652472 2023-07-06T03:45:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:45:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:45:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:45:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:45:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.2:0", 2023-07-06T03:45:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1009652472 2023-07-06T03:45:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:45:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.4701 192.168.0.2:0/1009652472", 2023-07-06T03:45:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.2:0/1009652472", 2023-07-06T03:45:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:45:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 8, 2023-07-06T03:45:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 23, 2023-07-06T03:45:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:45:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:45:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:45:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:45:41.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:45:41.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.490+0000 7f92c53ff700 1 Processor -- start 2023-07-06T03:45:41.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.490+0000 7f92c53ff700 1 -- start start 2023-07-06T03:45:41.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.491+0000 7f92c53ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f92c015c520 0x7f92c00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:41.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.491+0000 7f92c53ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92c0160d00 0x7f92c00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92c53ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c00cdae0 con 0x7f92c0160180 2023-07-06T03:45:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92c53ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c00cd8d0 con 0x7f92c0160d00 2023-07-06T03:45:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92c53ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c00b6640 con 0x7f92c015c520 2023-07-06T03:45:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92b7fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f92c0160180 0x7f92c01605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53878/0 (socket says 172.21.15.97:53878) 2023-07-06T03:45:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92b7fff700 1 -- 172.21.15.97:0/1033874432 learned_addr learned my addr 172.21.15.97:0/1033874432 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92bf7fe700 1 --2- 172.21.15.97:0/1033874432 >> v2:172.21.15.97:3300/0 conn(0x7f92c0160d00 0x7f92c00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:41.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92beffd700 1 --2- 172.21.15.97:0/1033874432 >> v2:172.21.15.146:3300/0 conn(0x7f92c015c520 0x7f92c00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92be7fc700 1 -- 172.21.15.97:0/1033874432 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432678090 0 0) 0x7f92c00cdae0 con 0x7f92c0160180 2023-07-06T03:45:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92be7fc700 1 -- 172.21.15.97:0/1033874432 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92ac0035b0 con 0x7f92c0160180 2023-07-06T03:45:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92bf7fe700 1 -- 172.21.15.97:0/1033874432 >> v2:172.21.15.146:3300/0 conn(0x7f92c015c520 msgr2=0x7f92c00cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92bf7fe700 1 --2- 172.21.15.97:0/1033874432 >> v2:172.21.15.146:3300/0 conn(0x7f92c015c520 0x7f92c00cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92bf7fe700 1 -- 172.21.15.97:0/1033874432 >> v1:172.21.15.97:6789/0 conn(0x7f92c0160180 legacy=0x7f92c01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.492+0000 7f92bf7fe700 1 -- 172.21.15.97:0/1033874432 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c00b6920 con 0x7f92c0160d00 2023-07-06T03:45:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.493+0000 7f92bf7fe700 1 --2- 172.21.15.97:0/1033874432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92c0160d00 0x7f92c00b7400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92c00b4c70 tx=0x7f92b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0cab84836fc4c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.493+0000 7f92be7fc700 1 -- 172.21.15.97:0/1033874432 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b800e070 con 0x7f92c0160d00 2023-07-06T03:45:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.493+0000 7f92be7fc700 1 -- 172.21.15.97:0/1033874432 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b800bb20 con 0x7f92c0160d00 2023-07-06T03:45:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.493+0000 7f92be7fc700 1 -- 172.21.15.97:0/1033874432 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8012430 con 0x7f92c0160d00 2023-07-06T03:45:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.493+0000 7f92c53ff700 1 -- 172.21.15.97:0/1033874432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92c0160d00 msgr2=0x7f92c00b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.493+0000 7f92c53ff700 1 --2- 172.21.15.97:0/1033874432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92c0160d00 0x7f92c00b7400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92c00b4c70 tx=0x7f92b800b330 comp rx=0 tx=0).stop 2023-07-06T03:45:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.493+0000 7f92c53ff700 1 -- 172.21.15.97:0/1033874432 shutdown_connections 2023-07-06T03:45:41.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.493+0000 7f92c53ff700 1 --2- 172.21.15.97:0/1033874432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92c0160d00 0x7f92c00b7400 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:41.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.493+0000 7f92c53ff700 1 --2- 172.21.15.97:0/1033874432 >> v2:172.21.15.146:3300/0 conn(0x7f92c015c520 0x7f92c00cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:41.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.493+0000 7f92c53ff700 1 -- 172.21.15.97:0/1033874432 >> 172.21.15.97:0/1033874432 conn(0x7f92c0071790 msgr2=0x7f92c0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:41.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.493+0000 7f92c53ff700 1 -- 172.21.15.97:0/1033874432 shutdown_connections 2023-07-06T03:45:41.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.494+0000 7f92c53ff700 1 -- 172.21.15.97:0/1033874432 wait complete. 2023-07-06T03:45:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.495+0000 7f92c53ff700 1 Processor -- start 2023-07-06T03:45:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.495+0000 7f92c53ff700 1 -- start start 2023-07-06T03:45:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.496+0000 7f92c53ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c015c520 0x7f92c0154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.496+0000 7f92c53ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92c0160180 0x7f92c00d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.496+0000 7f92c53ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c0160d00 0x7f92c00d2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:41.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.496+0000 7f92c53ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c015cd70 con 0x7f92c0160180 2023-07-06T03:45:41.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.496+0000 7f92c53ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00d1180 con 0x7f92c015c520 2023-07-06T03:45:41.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.496+0000 7f92c53ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c01f18d0 con 0x7f92c0160d00 2023-07-06T03:45:41.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.496+0000 7f92b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92c0160180 0x7f92c00d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:41.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.496+0000 7f92b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92c0160180 0x7f92c00d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44734/0 (socket says 172.21.15.97:44734) 2023-07-06T03:45:41.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.496+0000 7f92b7fff700 1 -- 172.21.15.97:0/72479735 learned_addr learned my addr 172.21.15.97:0/72479735 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:41.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92beffd700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c015c520 0x7f92c0154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:41.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92bf7fe700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c0160d00 0x7f92c00d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:41.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92b7fff700 1 -- 172.21.15.97:0/72479735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c0160d00 msgr2=0x7f92c00d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:41.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92b7fff700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c0160d00 0x7f92c00d2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:41.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92b7fff700 1 -- 172.21.15.97:0/72479735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c015c520 msgr2=0x7f92c0154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:41.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92b7fff700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c015c520 0x7f92c0154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:41.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92b7fff700 1 -- 172.21.15.97:0/72479735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b8009a70 con 0x7f92c0160180 2023-07-06T03:45:41.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92bf7fe700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c0160d00 0x7f92c00d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:41.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92b7fff700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92c0160180 0x7f92c00d0c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92a800c700 tx=0x7f92a800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:41.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92b77fe700 1 -- 172.21.15.97:0/72479735 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a8005d70 con 0x7f92c0160180 2023-07-06T03:45:41.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92b77fe700 1 -- 172.21.15.97:0/72479735 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a80070d0 con 0x7f92c0160180 2023-07-06T03:45:41.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92b77fe700 1 -- 172.21.15.97:0/72479735 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a8007460 con 0x7f92c0160180 2023-07-06T03:45:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.497+0000 7f92c53ff700 1 -- 172.21.15.97:0/72479735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c0155b50 con 0x7f92c0160180 2023-07-06T03:45:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.498+0000 7f92c53ff700 1 -- 172.21.15.97:0/72479735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c0155590 con 0x7f92c0160180 2023-07-06T03:45:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.498+0000 7f92c53ff700 1 -- 172.21.15.97:0/72479735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c015cee0 con 0x7f92c0160180 2023-07-06T03:45:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.499+0000 7f92b77fe700 1 -- 172.21.15.97:0/72479735 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f92a80075c0 con 0x7f92c0160180 2023-07-06T03:45:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.500+0000 7f92b77fe700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92a006ad10 0x7f92a006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.500+0000 7f92b77fe700 1 -- 172.21.15.97:0/72479735 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f92a8088b00 con 0x7f92c0160180 2023-07-06T03:45:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.501+0000 7f92beffd700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92a006ad10 0x7f92a006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.501+0000 7f92beffd700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92a006ad10 0x7f92a006d1c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009ad0 tx=0x7f92b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.505+0000 7f92b77fe700 1 -- 172.21.15.97:0/72479735 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f92a808ae60 con 0x7f92c0160180 2023-07-06T03:45:41.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.742+0000 7f92c53ff700 1 -- 172.21.15.97:0/72479735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f92c004e340 con 0x7f92c0160180 2023-07-06T03:45:41.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.743+0000 7f92b77fe700 1 -- 172.21.15.97:0/72479735 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f92a8053b20 con 0x7f92c0160180 2023-07-06T03:45:41.745 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:45:41.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 -- 172.21.15.97:0/72479735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92a006ad10 msgr2=0x7f92a006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:41.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92a006ad10 0x7f92a006d1c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009ad0 tx=0x7f92b0008040 comp rx=0 tx=0).stop 2023-07-06T03:45:41.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 -- 172.21.15.97:0/72479735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92c0160180 msgr2=0x7f92c00d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:41.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92c0160180 0x7f92c00d0c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92a800c700 tx=0x7f92a800cac0 comp rx=0 tx=0).stop 2023-07-06T03:45:41.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 -- 172.21.15.97:0/72479735 shutdown_connections 2023-07-06T03:45:41.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92a006ad10 0x7f92a006d1c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:41.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92c0160d00 0x7f92c00d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:41.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92c015c520 0x7f92c0154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:41.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 --2- 172.21.15.97:0/72479735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92c0160180 0x7f92c00d0c40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:41.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 -- 172.21.15.97:0/72479735 >> 172.21.15.97:0/72479735 conn(0x7f92c0071790 msgr2=0x7f92c00b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:41.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 -- 172.21.15.97:0/72479735 shutdown_connections 2023-07-06T03:45:41.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:41.745+0000 7f92c53ff700 1 -- 172.21.15.97:0/72479735 wait complete. 2023-07-06T03:45:41.765 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:45:41.765 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:45:41.765 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2023-07-06T03:45:41.766 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T03:45:41.766 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T03:45:41.766 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:45:41.766 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:45:41.766 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:45:41.766 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:45:41.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:41.767 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2023-07-06T03:45:41.820 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:45:41.820 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:45:41.820 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:45:41.821 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:41.821 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:45:41.821 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:41.821 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:41.821 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:41.821 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2023-07-06T03:45:41.821 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:41.822 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:41.822 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2023-07-06T03:45:41.822 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:41.822 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:41.822 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:45:41.822 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:41.822 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:45:41.823 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42143sec preferred_lft 42143sec 2023-07-06T03:45:41.823 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2023-07-06T03:45:41.823 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:41.824 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T03:45:41.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:41.824 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:45:41.825 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:45:41.825 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add name ceph-brx type bridge 2023-07-06T03:45:41.825 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr flush dev ceph-brx 2023-07-06T03:45:41.825 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set ceph-brx up 2023-07-06T03:45:41.826 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T03:45:41.826 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:45:42.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:42.305 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:45:42.407 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:45:42.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:42.410 DEBUG:teuthology.orchestra.run.smithi146:> route 2023-07-06T03:45:42.481 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2023-07-06T03:45:42.481 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:45:42.481 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:45:42.482 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:45:42.482 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:45:42.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:42.483 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:45:42.483 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:45:42.483 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:45:42.484 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:45:42.484 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:45:42.484 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:45:43.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:43.140 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:45:43.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:43.188 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2023-07-06T03:45:43.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:43.235 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:45:43.235 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:45:43.235 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:45:43.235 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-07-06T03:45:43.235 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:45:43.417 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-07-06T03:45:43.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:43.418 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:45:43.419 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:45:43.419 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-07-06T03:45:43.419 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T03:45:43.419 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-07-06T03:45:43.420 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-07-06T03:45:43.420 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-07-06T03:45:43.421 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:45:43.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:43.712 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:45:43.712 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:45:43.712 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2023-07-06T03:45:43.712 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T03:45:43.713 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:45:43.892 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-06T03:45:43.892 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:45:43.893 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T03:45:43.938 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:45:43.939 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:45:43.939 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:45:44.023 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:45:44.133 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:45:44.134 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:45:44.201 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:45:44.203 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:45:44.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:44.204 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:45:44.250 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:45:44.250 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-06T03:45:44.294 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:45:44.365 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:45:44.366 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:45:44.434 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:45:44.436 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:45:44.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:44.437 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:45:44.461 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2023-07-06T03:45:44.459+0000 7f6a3dc995c0 -1 init, newargv = 0x563990007c10 newargc=15 2023-07-06T03:45:44.462 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[118792]: starting ceph client 2023-07-06T03:45:44.470 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[118792]: starting fuse 2023-07-06T03:45:44.486 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:45:44.486 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T03:45:44.487 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:45:44.487 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2023-07-06T03:45:44.487 DEBUG:teuthology.orchestra.run.smithi146:> import re 2023-07-06T03:45:44.488 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:45:44.488 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2023-07-06T03:45:44.488 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:45:44.488 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2023-07-06T03:45:44.489 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-06T03:45:44.489 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2023-07-06T03:45:44.489 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-06T03:45:44.489 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:45:44.490 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2023-07-06T03:45:44.490 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2023-07-06T03:45:44.490 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2023-07-06T03:45:44.490 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2023-07-06T03:45:44.490 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:45:44.491 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2023-07-06T03:45:44.491 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:45:44.491 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:45:44.492 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:45:44.492 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2023-07-06T03:45:44.492 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2023-07-06T03:45:44.493 DEBUG:teuthology.orchestra.run.smithi146:> return f 2023-07-06T03:45:44.493 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:45:44.493 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:45:44.493 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2023-07-06T03:45:44.494 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:45:44.669 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.118792.asok 2023-07-06T03:45:44.847 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.118792.asok 2023-07-06T03:45:44.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:44.849 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.118792.asok status 2023-07-06T03:45:45.072 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-07-06T03:45:45.075 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2023-07-06T03:45:45.076 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:45:45.076 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:45:45.076 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2023-07-06T03:45:45.076 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2023-07-06T03:45:45.077 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-06T03:45:45.077 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "118792", 2023-07-06T03:45:45.077 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2023-07-06T03:45:45.077 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:45:45.077 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2023-07-06T03:45:45.077 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2023-07-06T03:45:45.078 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4681, 2023-07-06T03:45:45.078 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2023-07-06T03:45:45.078 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2023-07-06T03:45:45.078 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2023-07-06T03:45:45.078 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 4681 2023-07-06T03:45:45.078 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:45:45.079 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:45:45.079 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:45:45.079 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:45:45.079 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 923942252 2023-07-06T03:45:45.080 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-07-06T03:45:45.080 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:45:45.080 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:45:45.081 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:45:45.081 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:45:45.081 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 923942252 2023-07-06T03:45:45.082 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:45:45.082 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.4681 192.168.0.1:0/923942252", 2023-07-06T03:45:45.082 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/923942252", 2023-07-06T03:45:45.083 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2023-07-06T03:45:45.083 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 8, 2023-07-06T03:45:45.083 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 23, 2023-07-06T03:45:45.083 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:45:45.084 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2023-07-06T03:45:45.084 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2023-07-06T03:45:45.084 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-07-06T03:45:45.087 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:45:45.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.289+0000 7f2e8d1ff700 1 Processor -- start 2023-07-06T03:45:45.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.290+0000 7f2e8d1ff700 1 -- start start 2023-07-06T03:45:45.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e8d1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e8815c520 0x7f2e880cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:45.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e8d1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e88160180 0x7f2e881605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:45.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e8d1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e880cdae0 con 0x7f2e88160b30 2023-07-06T03:45:45.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e8d1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e880cd8d0 con 0x7f2e8815c520 2023-07-06T03:45:45.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e8d1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880b6640 con 0x7f2e88160180 2023-07-06T03:45:45.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e86d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e8815c520 0x7f2e880cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:45.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e86d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e8815c520 0x7f2e880cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44740/0 (socket says 172.21.15.97:44740) 2023-07-06T03:45:45.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e86d9d700 1 -- 172.21.15.97:0/2275933593 learned_addr learned my addr 172.21.15.97:0/2275933593 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:45.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e8659c700 1 --2- 172.21.15.97:0/2275933593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e88160180 0x7f2e881605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:45.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e86d9d700 1 -- 172.21.15.97:0/2275933593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e88160180 msgr2=0x7f2e881605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:45.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e86d9d700 1 --2- 172.21.15.97:0/2275933593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e88160180 0x7f2e881605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:45.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.291+0000 7f2e86d9d700 1 -- 172.21.15.97:0/2275933593 >> v1:172.21.15.146:6789/0 conn(0x7f2e88160b30 legacy=0x7f2e880cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:45.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.292+0000 7f2e86d9d700 1 -- 172.21.15.97:0/2275933593 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e880b6920 con 0x7f2e8815c520 2023-07-06T03:45:45.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.292+0000 7f2e86d9d700 1 --2- 172.21.15.97:0/2275933593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e8815c520 0x7f2e880cd390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e780098f0 tx=0x7f2e78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad8da8c1a8623d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:45.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.292+0000 7f2e85d9b700 1 -- 172.21.15.97:0/2275933593 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78011830 con 0x7f2e8815c520 2023-07-06T03:45:45.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.292+0000 7f2e85d9b700 1 -- 172.21.15.97:0/2275933593 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e78011990 con 0x7f2e8815c520 2023-07-06T03:45:45.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.292+0000 7f2e85d9b700 1 -- 172.21.15.97:0/2275933593 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78011d20 con 0x7f2e8815c520 2023-07-06T03:45:45.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.293+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/2275933593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e8815c520 msgr2=0x7f2e880cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:45.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.293+0000 7f2e8d1ff700 1 --2- 172.21.15.97:0/2275933593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e8815c520 0x7f2e880cd390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e780098f0 tx=0x7f2e78009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:45.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.293+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/2275933593 shutdown_connections 2023-07-06T03:45:45.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.293+0000 7f2e8d1ff700 1 --2- 172.21.15.97:0/2275933593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e8815c520 0x7f2e880cd390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:45.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.293+0000 7f2e8d1ff700 1 --2- 172.21.15.97:0/2275933593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e88160180 0x7f2e881605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:45.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.293+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/2275933593 >> 172.21.15.97:0/2275933593 conn(0x7f2e88071790 msgr2=0x7f2e88071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:45.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.293+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/2275933593 shutdown_connections 2023-07-06T03:45:45.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.293+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/2275933593 wait complete. 2023-07-06T03:45:45.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.294+0000 7f2e8d1ff700 1 Processor -- start 2023-07-06T03:45:45.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.295+0000 7f2e8d1ff700 1 -- start start 2023-07-06T03:45:45.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.295+0000 7f2e8d1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e8815c520 0x7f2e880ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:45.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.295+0000 7f2e8d1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e88160180 0x7f2e880d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:45.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.295+0000 7f2e8d1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e88160b30 0x7f2e8815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:45.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.295+0000 7f2e8d1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880cadf0 con 0x7f2e88160b30 2023-07-06T03:45:45.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.295+0000 7f2e8d1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880ca1e0 con 0x7f2e88160180 2023-07-06T03:45:45.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.295+0000 7f2e8d1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880ca4c0 con 0x7f2e8815c520 2023-07-06T03:45:45.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e8659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e88160180 0x7f2e880d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:45.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e86d9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e8815c520 0x7f2e880ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:45.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e8659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e88160180 0x7f2e880d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:33740/0 (socket says 172.21.15.97:33740) 2023-07-06T03:45:45.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e8659c700 1 -- 172.21.15.97:0/1216439221 learned_addr learned my addr 172.21.15.97:0/1216439221 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:45.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e8759e700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e88160b30 0x7f2e8815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:45.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e86d9d700 1 -- 172.21.15.97:0/1216439221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e88160180 msgr2=0x7f2e880d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:45.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e86d9d700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e88160180 0x7f2e880d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:45.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e86d9d700 1 -- 172.21.15.97:0/1216439221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e88160b30 msgr2=0x7f2e8815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:45.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e86d9d700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e88160b30 0x7f2e8815dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:45.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e86d9d700 1 -- 172.21.15.97:0/1216439221 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e78009550 con 0x7f2e8815c520 2023-07-06T03:45:45.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e8659c700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e88160180 0x7f2e880d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:45.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.296+0000 7f2e86d9d700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e8815c520 0x7f2e880ccb60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e78016000 tx=0x7f2e78004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:45.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.297+0000 7f2e6f7fe700 1 -- 172.21.15.97:0/1216439221 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78004510 con 0x7f2e8815c520 2023-07-06T03:45:45.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.297+0000 7f2e6f7fe700 1 -- 172.21.15.97:0/1216439221 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7801e070 con 0x7f2e8815c520 2023-07-06T03:45:45.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.297+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/1216439221 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e880ca7a0 con 0x7f2e8815c520 2023-07-06T03:45:45.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.297+0000 7f2e6f7fe700 1 -- 172.21.15.97:0/1216439221 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e780038c0 con 0x7f2e8815c520 2023-07-06T03:45:45.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.297+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/1216439221 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e880d19f0 con 0x7f2e8815c520 2023-07-06T03:45:45.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.298+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/1216439221 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e880d1be0 con 0x7f2e8815c520 2023-07-06T03:45:45.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.299+0000 7f2e6f7fe700 1 -- 172.21.15.97:0/1216439221 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2e78003a20 con 0x7f2e8815c520 2023-07-06T03:45:45.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.299+0000 7f2e6f7fe700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e7406ace0 0x7f2e7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:45.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.300+0000 7f2e6f7fe700 1 -- 172.21.15.97:0/1216439221 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e7808a2d0 con 0x7f2e8815c520 2023-07-06T03:45:45.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.300+0000 7f2e8659c700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e7406ace0 0x7f2e7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:45.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.300+0000 7f2e8659c700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e7406ace0 0x7f2e7406d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2e8815f290 tx=0x7f2e7c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:45.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.304+0000 7f2e6f7fe700 1 -- 172.21.15.97:0/1216439221 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2e780552f0 con 0x7f2e8815c520 2023-07-06T03:45:45.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.543+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/1216439221 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2e880b6460 con 0x7f2e8815c520 2023-07-06T03:45:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.544+0000 7f2e6f7fe700 1 -- 172.21.15.97:0/1216439221 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2e78058910 con 0x7f2e8815c520 2023-07-06T03:45:45.546 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:45:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.546+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/1216439221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e7406ace0 msgr2=0x7f2e7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:45.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.546+0000 7f2e8d1ff700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e7406ace0 0x7f2e7406d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2e8815f290 tx=0x7f2e7c009210 comp rx=0 tx=0).stop 2023-07-06T03:45:45.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.546+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/1216439221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e8815c520 msgr2=0x7f2e880ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:45.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.546+0000 7f2e8d1ff700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e8815c520 0x7f2e880ccb60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e78016000 tx=0x7f2e78004750 comp rx=0 tx=0).stop 2023-07-06T03:45:45.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.547+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/1216439221 shutdown_connections 2023-07-06T03:45:45.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.547+0000 7f2e8d1ff700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e7406ace0 0x7f2e7406d190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:45.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.547+0000 7f2e8d1ff700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e88160b30 0x7f2e8815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:45.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.547+0000 7f2e8d1ff700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e8815c520 0x7f2e880ccb60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:45.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.547+0000 7f2e8d1ff700 1 --2- 172.21.15.97:0/1216439221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e88160180 0x7f2e880d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:45.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.547+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/1216439221 >> 172.21.15.97:0/1216439221 conn(0x7f2e88071790 msgr2=0x7f2e880c9490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:45.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.547+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/1216439221 shutdown_connections 2023-07-06T03:45:45.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:45.547+0000 7f2e8d1ff700 1 -- 172.21.15.97:0/1216439221 wait complete. 2023-07-06T03:45:45.567 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:45:45.567 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:45:45.567 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2023-07-06T03:45:45.568 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-07-06T03:45:45.568 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-06T03:45:45.568 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:45:45.568 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:45:45.568 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:45:45.569 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-07-06T03:45:45.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:45.569 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2023-07-06T03:45:45.616 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:45:45.616 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:45:45.617 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:45:45.617 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:45.617 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:45:45.617 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:45.617 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:45.617 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:45.617 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2023-07-06T03:45:45.617 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:45.618 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:45.618 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2023-07-06T03:45:45.618 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:45:45.618 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:45.618 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:45:45.618 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:45.618 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:45:45.619 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42139sec preferred_lft 42139sec 2023-07-06T03:45:45.619 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2023-07-06T03:45:45.619 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:45.619 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:45:45.619 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:45:45.619 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:45:45.619 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:45.619 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::58f0:6eff:fe04:3be0/64 scope link 2023-07-06T03:45:45.620 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:45.620 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:45:45.620 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:45:45.620 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::106d:9cff:fed5:9d0f/64 scope link 2023-07-06T03:45:45.620 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:45.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:45.621 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:45:45.728 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:45:45.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:45.731 DEBUG:teuthology.orchestra.run.smithi146:> route 2023-07-06T03:45:45.779 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2023-07-06T03:45:45.779 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:45:45.779 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:45:45.779 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:45:45.780 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:45:45.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:45.780 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:45:45.780 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:45:45.781 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:45:45.781 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:45:45.781 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:45:45.781 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:45:45.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:45.951 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:45:45.997 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T03:45:45.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:45.998 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2023-07-06T03:45:46.084 INFO:teuthology.orchestra.run.smithi146.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-07-06T03:45:46.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:46.085 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:45:46.086 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:45:46.086 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T03:45:46.086 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-07-06T03:45:46.086 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:45:46.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:46.267 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-06T03:45:46.338 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:45:46.339 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:45:46.339 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:45:46.339 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:46.339 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:45:46.339 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:46.340 INFO:teuthology.orchestra.run.smithi146.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:45:46.340 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 06:40:d1:9a:27:6d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T03:45:46.340 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T03:45:46.340 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:46.340 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::440:d1ff:fe9a:276d/64 scope link 2023-07-06T03:45:46.340 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:46.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:46.342 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-06T03:45:46.412 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:45:46.413 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:45:46.413 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:45:46.413 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:46.414 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:45:46.414 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:46.414 INFO:teuthology.orchestra.run.smithi146.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:45:46.414 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 06:40:d1:9a:27:6d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T03:45:46.415 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T03:45:46.415 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:46.415 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::440:d1ff:fe9a:276d/64 scope link 2023-07-06T03:45:46.415 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:45:46.417 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-07-06T03:45:46.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:46.418 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:45:46.418 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:45:46.418 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-07-06T03:45:46.418 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-06T03:45:46.419 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-07-06T03:45:46.419 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-07-06T03:45:46.419 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-07-06T03:45:46.419 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:45:46.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:46.665 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:45:46.665 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:45:46.665 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.1 up 2023-07-06T03:45:46.666 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.1 master ceph-brx 2023-07-06T03:45:46.666 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:45:46.845 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-07-06T03:45:46.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:45:46.846 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-07-06T03:45:46.892 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-07-06T03:45:46.892 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:45:46.893 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-07-06T03:45:46.977 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:45:47.087 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:45:47.087 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:45:47.156 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:45:47.158 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:45:47.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:47.159 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:45:47.205 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:45:47.205 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2023-07-06T03:45:47.206 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-07-06T03:45:47.250 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:45:47.320 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:45:47.321 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:45:47.389 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:45:47.392 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:45:47.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:47.393 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:45:47.417 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi146.stderr:2023-07-06T03:45:47.415+0000 7f614f9db5c0 -1 init, newargv = 0x558912a641f0 newargc=15 2023-07-06T03:45:47.418 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi146.stderr:ceph-fuse[119467]: starting ceph client 2023-07-06T03:45:47.425 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi146.stderr:ceph-fuse[119467]: starting fuse 2023-07-06T03:45:47.440 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:45:47.441 INFO:teuthology.orchestra.run.smithi146.stdout:44 2023-07-06T03:45:47.441 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2023-07-06T03:45:47.441 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:45:47.441 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2023-07-06T03:45:47.442 DEBUG:teuthology.orchestra.run.smithi146:> import re 2023-07-06T03:45:47.442 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:45:47.442 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2023-07-06T03:45:47.442 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:45:47.443 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2023-07-06T03:45:47.443 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-07-06T03:45:47.443 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2023-07-06T03:45:47.443 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-07-06T03:45:47.443 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:45:47.444 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2023-07-06T03:45:47.444 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2023-07-06T03:45:47.444 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2023-07-06T03:45:47.444 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2023-07-06T03:45:47.445 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:45:47.445 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2023-07-06T03:45:47.445 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:45:47.445 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:45:47.446 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:45:47.446 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2023-07-06T03:45:47.446 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2023-07-06T03:45:47.446 DEBUG:teuthology.orchestra.run.smithi146:> return f 2023-07-06T03:45:47.446 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:45:47.447 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:45:47.447 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.3")) 2023-07-06T03:45:47.447 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:45:47.624 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.3.119467.asok 2023-07-06T03:45:47.801 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.119467.asok 2023-07-06T03:45:47.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:47.802 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.119467.asok status 2023-07-06T03:45:48.029 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-07-06T03:45:48.029 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2023-07-06T03:45:48.029 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:45:48.029 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:45:48.029 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "3", 2023-07-06T03:45:48.030 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2023-07-06T03:45:48.030 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-07-06T03:45:48.030 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "119467", 2023-07-06T03:45:48.030 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2023-07-06T03:45:48.030 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:45:48.030 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2023-07-06T03:45:48.030 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2023-07-06T03:45:48.031 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4693, 2023-07-06T03:45:48.031 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2023-07-06T03:45:48.031 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2023-07-06T03:45:48.031 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2023-07-06T03:45:48.031 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 4693 2023-07-06T03:45:48.031 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:45:48.031 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:45:48.031 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:45:48.032 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.2:0", 2023-07-06T03:45:48.032 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 647784440 2023-07-06T03:45:48.032 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-07-06T03:45:48.032 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:45:48.032 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:45:48.032 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:45:48.032 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.2:0", 2023-07-06T03:45:48.033 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 647784440 2023-07-06T03:45:48.033 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:45:48.033 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.4693 192.168.0.2:0/647784440", 2023-07-06T03:45:48.033 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.2:0/647784440", 2023-07-06T03:45:48.033 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2023-07-06T03:45:48.033 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 8, 2023-07-06T03:45:48.033 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 23, 2023-07-06T03:45:48.033 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:45:48.034 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2023-07-06T03:45:48.034 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2023-07-06T03:45:48.034 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-07-06T03:45:48.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:48.043 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:45:48.043 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:45:48.095 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:45:48.095 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:45:48.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:48.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:45:48.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:48.171 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:45:48.171 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.2 2023-07-06T03:45:48.217 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:45:48.217 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-07-06T03:45:48.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:48.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-07-06T03:45:48.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:48.290 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2023-07-06T03:45:48.290 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-06T03:45:48.337 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2023-07-06T03:45:48.338 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-06T03:45:48.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:48.339 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:45:48.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:48.408 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2023-07-06T03:45:48.409 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.3 2023-07-06T03:45:48.455 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2023-07-06T03:45:48.455 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-07-06T03:45:48.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:45:48.456 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-07-06T03:45:48.528 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-07-06T03:45:48.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:48.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.752+0000 7f077cbff700 1 Processor -- start 2023-07-06T03:45:48.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.752+0000 7f077cbff700 1 -- start start 2023-07-06T03:45:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.754+0000 7f077cbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f07780cd390 0x7f0778160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.754+0000 7f077cbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07780cdae0 con 0x7f0778160dd0 2023-07-06T03:45:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.754+0000 7f077cbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07780b6640 con 0x7f077815c520 2023-07-06T03:45:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.754+0000 7f077cbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07780b6920 con 0x7f07780cd390 2023-07-06T03:45:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.754+0000 7f0776d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0778160dd0 0x7f07780b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53882/0 (socket says 172.21.15.97:53882) 2023-07-06T03:45:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.754+0000 7f0776d9d700 1 -- 172.21.15.97:0/3439549807 learned_addr learned my addr 172.21.15.97:0/3439549807 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:48.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.754+0000 7f076ffff700 1 --2- 172.21.15.97:0/3439549807 >> v2:172.21.15.146:3300/0 conn(0x7f07780cd390 0x7f0778160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:48.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.754+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886253282 0 0) 0x7f07780cdae0 con 0x7f0778160dd0 2023-07-06T03:45:48.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.754+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07640035b0 con 0x7f0778160dd0 2023-07-06T03:45:48.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.754+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 118716423 0 0) 0x7f07640035b0 con 0x7f0778160dd0 2023-07-06T03:45:48.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 >> v2:172.21.15.146:3300/0 conn(0x7f07780cd390 msgr2=0x7f0778160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:48.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 --2- 172.21.15.97:0/3439549807 >> v2:172.21.15.146:3300/0 conn(0x7f07780cd390 0x7f0778160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:48.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 >> v1:172.21.15.146:6789/0 conn(0x7f077815c520 legacy=0x7f0778160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:48.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07780b6c00 con 0x7f0778160dd0 2023-07-06T03:45:48.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f07700019f0 con 0x7f0778160dd0 2023-07-06T03:45:48.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 >> v1:172.21.15.97:6789/0 conn(0x7f0778160dd0 legacy=0x7f07780b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:48.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 --2- 172.21.15.97:0/3439549807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f0764003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 --2- 172.21.15.97:0/3439549807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0764004850 0x7f0764004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 --2- 172.21.15.97:0/3439549807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0764005200 0x7f076400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07780b6920 con 0x7f0764004850 2023-07-06T03:45:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076400c080 con 0x7f0764003a00 2023-07-06T03:45:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076400c240 con 0x7f0764005200 2023-07-06T03:45:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f076ffff700 1 --2- 172.21.15.97:0/3439549807 >> v2:172.21.15.146:3300/0 conn(0x7f07780cd390 0x7f0778160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f077659c700 1 --2- 172.21.15.97:0/3439549807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f0764003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f076ffff700 1 --2- 172.21.15.97:0/3439549807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0764004850 0x7f0764004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0776d9d700 1 --2- 172.21.15.97:0/3439549807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0764005200 0x7f076400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f077659c700 1 -- 172.21.15.97:0/3439549807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0764005200 msgr2=0x7f076400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f077659c700 1 --2- 172.21.15.97:0/3439549807 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0764005200 0x7f076400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f077659c700 1 -- 172.21.15.97:0/3439549807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0764004850 msgr2=0x7f0764004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f077659c700 1 --2- 172.21.15.97:0/3439549807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0764004850 0x7f0764004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f077659c700 1 -- 172.21.15.97:0/3439549807 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07640035b0 con 0x7f0764003a00 2023-07-06T03:45:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.755+0000 7f0776d9d700 1 -- 172.21.15.97:0/3439549807 reap_dead start 2023-07-06T03:45:48.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.756+0000 7f077659c700 1 --2- 172.21.15.97:0/3439549807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f0764003e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f07680017c0 tx=0x7f0764005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc16b5503d5c0d04 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:48.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.756+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680052d0 con 0x7f0764003a00 2023-07-06T03:45:48.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.756+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0768005450 con 0x7f0764003a00 2023-07-06T03:45:48.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.756+0000 7f0775d9b700 1 -- 172.21.15.97:0/3439549807 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680038c0 con 0x7f0764003a00 2023-07-06T03:45:48.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.756+0000 7f077cbff700 1 -- 172.21.15.97:0/3439549807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 msgr2=0x7f0764003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:48.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.756+0000 7f077cbff700 1 --2- 172.21.15.97:0/3439549807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f0764003e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f07680017c0 tx=0x7f0764005880 comp rx=0 tx=0).stop 2023-07-06T03:45:48.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.756+0000 7f077cbff700 1 -- 172.21.15.97:0/3439549807 shutdown_connections 2023-07-06T03:45:48.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.756+0000 7f077cbff700 1 --2- 172.21.15.97:0/3439549807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f0764003e10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:48.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.756+0000 7f077cbff700 1 -- 172.21.15.97:0/3439549807 >> 172.21.15.97:0/3439549807 conn(0x7f0778071790 msgr2=0x7f0778071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:48.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.757+0000 7f077cbff700 1 -- 172.21.15.97:0/3439549807 shutdown_connections 2023-07-06T03:45:48.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.757+0000 7f077cbff700 1 -- 172.21.15.97:0/3439549807 wait complete. 2023-07-06T03:45:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.758+0000 7f077cbff700 1 Processor -- start 2023-07-06T03:45:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.758+0000 7f077cbff700 1 -- start start 2023-07-06T03:45:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.759+0000 7f077cbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f07780c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.759+0000 7f077cbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07780c7c00 0x7f077815dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.759+0000 7f077cbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f077815e0f0 0x7f07780d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.759+0000 7f077cbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077815e560 con 0x7f07780c7c00 2023-07-06T03:45:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.759+0000 7f077cbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07780d29a0 con 0x7f0764003a00 2023-07-06T03:45:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.759+0000 7f077cbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07780c8070 con 0x7f077815e0f0 2023-07-06T03:45:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f076ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07780c7c00 0x7f077815dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f076ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07780c7c00 0x7f077815dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44768/0 (socket says 172.21.15.97:44768) 2023-07-06T03:45:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f076ffff700 1 -- 172.21.15.97:0/1744703353 learned_addr learned my addr 172.21.15.97:0/1744703353 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f077659c700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f07780c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f0776d9d700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f077815e0f0 0x7f07780d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f076ffff700 1 -- 172.21.15.97:0/1744703353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f077815e0f0 msgr2=0x7f07780d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f076ffff700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f077815e0f0 0x7f07780d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f076ffff700 1 -- 172.21.15.97:0/1744703353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 msgr2=0x7f07780c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f076ffff700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f07780c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f076ffff700 1 -- 172.21.15.97:0/1744703353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07780d2070 con 0x7f07780c7c00 2023-07-06T03:45:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f076ffff700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07780c7c00 0x7f077815dbb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0760011c00 tx=0x7f0760011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f076effd700 1 -- 172.21.15.97:0/1744703353 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076000f980 con 0x7f07780c7c00 2023-07-06T03:45:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f076effd700 1 -- 172.21.15.97:0/1744703353 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076000fae0 con 0x7f07780c7c00 2023-07-06T03:45:48.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.760+0000 7f077cbff700 1 -- 172.21.15.97:0/1744703353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07780d2350 con 0x7f07780c7c00 2023-07-06T03:45:48.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.761+0000 7f076effd700 1 -- 172.21.15.97:0/1744703353 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076000fe70 con 0x7f07780c7c00 2023-07-06T03:45:48.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.761+0000 7f077cbff700 1 -- 172.21.15.97:0/1744703353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07780d1b00 con 0x7f07780c7c00 2023-07-06T03:45:48.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.762+0000 7f077cbff700 1 -- 172.21.15.97:0/1744703353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077804e340 con 0x7f07780c7c00 2023-07-06T03:45:48.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.762+0000 7f076effd700 1 -- 172.21.15.97:0/1744703353 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0760014440 con 0x7f07780c7c00 2023-07-06T03:45:48.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.763+0000 7f076effd700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f075806ace0 0x7f075806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:48.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.763+0000 7f076effd700 1 -- 172.21.15.97:0/1744703353 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0760089b80 con 0x7f07780c7c00 2023-07-06T03:45:48.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.764+0000 7f077659c700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f075806ace0 0x7f075806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:48.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.764+0000 7f077659c700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f075806ace0 0x7f075806d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0768000c00 tx=0x7f0768009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:48.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.768+0000 7f076effd700 1 -- 172.21.15.97:0/1744703353 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0760054b90 con 0x7f07780c7c00 2023-07-06T03:45:48.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.965+0000 7f077cbff700 1 -- 172.21.15.97:0/1744703353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07780b4f40 con 0x7f07780c7c00 2023-07-06T03:45:48.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.966+0000 7f076effd700 1 -- 172.21.15.97:0/1744703353 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f0760022090 con 0x7f07780c7c00 2023-07-06T03:45:48.969 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:48.969 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:48.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.968+0000 7f077cbff700 1 -- 172.21.15.97:0/1744703353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f075806ace0 msgr2=0x7f075806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:48.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.968+0000 7f077cbff700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f075806ace0 0x7f075806d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0768000c00 tx=0x7f0768009030 comp rx=0 tx=0).stop 2023-07-06T03:45:48.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.969+0000 7f077cbff700 1 -- 172.21.15.97:0/1744703353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07780c7c00 msgr2=0x7f077815dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:48.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.969+0000 7f077cbff700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07780c7c00 0x7f077815dbb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0760011c00 tx=0x7f0760011f10 comp rx=0 tx=0).stop 2023-07-06T03:45:48.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.969+0000 7f077cbff700 1 -- 172.21.15.97:0/1744703353 shutdown_connections 2023-07-06T03:45:48.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.969+0000 7f077cbff700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f075806ace0 0x7f075806d190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:48.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.969+0000 7f077cbff700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f077815e0f0 0x7f07780d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:48.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.969+0000 7f077cbff700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f07780c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:48.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.969+0000 7f077cbff700 1 --2- 172.21.15.97:0/1744703353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07780c7c00 0x7f077815dbb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:48.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.969+0000 7f077cbff700 1 -- 172.21.15.97:0/1744703353 >> 172.21.15.97:0/1744703353 conn(0x7f0778071790 msgr2=0x7f07780b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:48.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.969+0000 7f077cbff700 1 -- 172.21.15.97:0/1744703353 shutdown_connections 2023-07-06T03:45:48.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:48.969+0000 7f077cbff700 1 -- 172.21.15.97:0/1744703353 wait complete. 2023-07-06T03:45:48.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:45:48.990 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:45:49.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.186+0000 7f024dbff700 1 Processor -- start 2023-07-06T03:45:49.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.187+0000 7f024dbff700 1 -- start start 2023-07-06T03:45:49.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.188+0000 7f024dbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f02480d2210 0x7f02480b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.188+0000 7f024dbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02480cff90 con 0x7f024806b360 2023-07-06T03:45:49.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.188+0000 7f024dbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02480b5650 con 0x7f024815c610 2023-07-06T03:45:49.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.188+0000 7f024dbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480d0780 con 0x7f02480d2210 2023-07-06T03:45:49.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.188+0000 7f0246ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f024806b360 0x7f02480d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53890/0 (socket says 172.21.15.97:53890) 2023-07-06T03:45:49.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.188+0000 7f0246ffd700 1 -- 172.21.15.97:0/2380608066 learned_addr learned my addr 172.21.15.97:0/2380608066 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:49.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.188+0000 7f0247fff700 1 --2- 172.21.15.97:0/2380608066 >> v2:172.21.15.146:3300/0 conn(0x7f02480d2210 0x7f02480b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.188+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544514573 0 0) 0x7f02480cff90 con 0x7f024806b360 2023-07-06T03:45:49.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.188+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02300035b0 con 0x7f024806b360 2023-07-06T03:45:49.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3726822187 0 0) 0x7f02300035b0 con 0x7f024806b360 2023-07-06T03:45:49.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 >> v2:172.21.15.146:3300/0 conn(0x7f02480d2210 msgr2=0x7f02480b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:49.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 --2- 172.21.15.97:0/2380608066 >> v2:172.21.15.146:3300/0 conn(0x7f02480d2210 0x7f02480b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 >> v1:172.21.15.146:6789/0 conn(0x7f024815c610 legacy=0x7f02480b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480b5400 con 0x7f024806b360 2023-07-06T03:45:49.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f023c001670 con 0x7f024806b360 2023-07-06T03:45:49.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 >> v1:172.21.15.97:6789/0 conn(0x7f024806b360 legacy=0x7f02480d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0230003a50 0x7f0230003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0230004850 0x7f0230004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0230005200 0x7f023000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02480d0780 con 0x7f0230004850 2023-07-06T03:45:49.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023000c0e0 con 0x7f0230003a50 2023-07-06T03:45:49.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023000c2a0 con 0x7f0230005200 2023-07-06T03:45:49.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f0246ffd700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0230004850 0x7f0230004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f02477fe700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0230003a50 0x7f0230003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f0247fff700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0230005200 0x7f023000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f0246ffd700 1 -- 172.21.15.97:0/2380608066 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0230005200 msgr2=0x7f023000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:49.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f0246ffd700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0230005200 0x7f023000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f0246ffd700 1 -- 172.21.15.97:0/2380608066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0230003a50 msgr2=0x7f0230003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:49.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f0246ffd700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0230003a50 0x7f0230003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.189+0000 7f0246ffd700 1 -- 172.21.15.97:0/2380608066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02300035b0 con 0x7f0230004850 2023-07-06T03:45:49.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.190+0000 7f02477fe700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0230003a50 0x7f0230003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:49.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.190+0000 7f0247fff700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0230005200 0x7f023000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:49.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.190+0000 7f0247fff700 1 -- 172.21.15.97:0/2380608066 reap_dead start 2023-07-06T03:45:49.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.190+0000 7f0246ffd700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0230004850 0x7f0230004cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f023c001eb0 tx=0x7f0230005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60af83f5c34470e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:49.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.190+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c007b70 con 0x7f0230004850 2023-07-06T03:45:49.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.190+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c00ccf0 con 0x7f0230004850 2023-07-06T03:45:49.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.190+0000 7f02467fc700 1 -- 172.21.15.97:0/2380608066 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c005ba0 con 0x7f0230004850 2023-07-06T03:45:49.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.190+0000 7f024dbff700 1 -- 172.21.15.97:0/2380608066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0230004850 msgr2=0x7f0230004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.190+0000 7f024dbff700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0230004850 0x7f0230004cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f023c001eb0 tx=0x7f0230005880 comp rx=0 tx=0).stop 2023-07-06T03:45:49.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.190+0000 7f024dbff700 1 -- 172.21.15.97:0/2380608066 shutdown_connections 2023-07-06T03:45:49.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.191+0000 7f024dbff700 1 --2- 172.21.15.97:0/2380608066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0230004850 0x7f0230004cc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.191+0000 7f024dbff700 1 -- 172.21.15.97:0/2380608066 >> 172.21.15.97:0/2380608066 conn(0x7f024806a6c0 msgr2=0x7f024806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:49.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.191+0000 7f024dbff700 1 -- 172.21.15.97:0/2380608066 shutdown_connections 2023-07-06T03:45:49.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.191+0000 7f024dbff700 1 -- 172.21.15.97:0/2380608066 wait complete. 2023-07-06T03:45:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.192+0000 7f024dbff700 1 Processor -- start 2023-07-06T03:45:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.192+0000 7f024dbff700 1 -- start start 2023-07-06T03:45:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.193+0000 7f024dbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0230004850 0x7f02480caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.193+0000 7f024dbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02480ca6d0 0x7f02481566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.193+0000 7f024dbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f024815ee60 0x7f024815f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.193+0000 7f024dbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02480b5f40 con 0x7f02480ca6d0 2023-07-06T03:45:49.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.193+0000 7f024dbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02480cb400 con 0x7f0230004850 2023-07-06T03:45:49.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.193+0000 7f024dbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02480cb6a0 con 0x7f024815ee60 2023-07-06T03:45:49.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f0246ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02480ca6d0 0x7f02481566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f0246ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02480ca6d0 0x7f02481566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44792/0 (socket says 172.21.15.97:44792) 2023-07-06T03:45:49.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f0246ffd700 1 -- 172.21.15.97:0/850285617 learned_addr learned my addr 172.21.15.97:0/850285617 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f02477fe700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0230004850 0x7f02480caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f0247fff700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f024815ee60 0x7f024815f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f0246ffd700 1 -- 172.21.15.97:0/850285617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f024815ee60 msgr2=0x7f024815f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f0246ffd700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f024815ee60 0x7f024815f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f0246ffd700 1 -- 172.21.15.97:0/850285617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0230004850 msgr2=0x7f02480caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f0246ffd700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0230004850 0x7f02480caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f0246ffd700 1 -- 172.21.15.97:0/850285617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480cb980 con 0x7f02480ca6d0 2023-07-06T03:45:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f02477fe700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0230004850 0x7f02480caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f0246ffd700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02480ca6d0 0x7f02481566c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f023c00c720 tx=0x7f023c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.194+0000 7f022ffff700 1 -- 172.21.15.97:0/850285617 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c004c80 con 0x7f02480ca6d0 2023-07-06T03:45:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.195+0000 7f022ffff700 1 -- 172.21.15.97:0/850285617 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c004de0 con 0x7f02480ca6d0 2023-07-06T03:45:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.195+0000 7f024dbff700 1 -- 172.21.15.97:0/850285617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02480cbc60 con 0x7f02480ca6d0 2023-07-06T03:45:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.195+0000 7f022ffff700 1 -- 172.21.15.97:0/850285617 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c007310 con 0x7f02480ca6d0 2023-07-06T03:45:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.195+0000 7f024dbff700 1 -- 172.21.15.97:0/850285617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0248156c00 con 0x7f02480ca6d0 2023-07-06T03:45:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.196+0000 7f024dbff700 1 -- 172.21.15.97:0/850285617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024804e340 con 0x7f02480ca6d0 2023-07-06T03:45:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.197+0000 7f022ffff700 1 -- 172.21.15.97:0/850285617 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f023c007470 con 0x7f02480ca6d0 2023-07-06T03:45:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.198+0000 7f022ffff700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f023406ace0 0x7f023406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.198+0000 7f022ffff700 1 -- 172.21.15.97:0/850285617 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f023c089fa0 con 0x7f02480ca6d0 2023-07-06T03:45:49.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.198+0000 7f02477fe700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f023406ace0 0x7f023406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.198+0000 7f02477fe700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f023406ace0 0x7f023406d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0238000c00 tx=0x7f0238009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:49.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.202+0000 7f022ffff700 1 -- 172.21.15.97:0/850285617 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f023c055040 con 0x7f02480ca6d0 2023-07-06T03:45:49.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.397+0000 7f024dbff700 1 -- 172.21.15.97:0/850285617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02480b6220 con 0x7f02480ca6d0 2023-07-06T03:45:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.398+0000 7f022ffff700 1 -- 172.21.15.97:0/850285617 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f023c0170a0 con 0x7f02480ca6d0 2023-07-06T03:45:49.401 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:45:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 -- 172.21.15.97:0/850285617 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f023406ace0 msgr2=0x7f023406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f023406ace0 0x7f023406d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0238000c00 tx=0x7f0238009040 comp rx=0 tx=0).stop 2023-07-06T03:45:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 -- 172.21.15.97:0/850285617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02480ca6d0 msgr2=0x7f02481566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02480ca6d0 0x7f02481566c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f023c00c720 tx=0x7f023c001930 comp rx=0 tx=0).stop 2023-07-06T03:45:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 -- 172.21.15.97:0/850285617 shutdown_connections 2023-07-06T03:45:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f023406ace0 0x7f023406d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f024815ee60 0x7f024815f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0230004850 0x7f02480caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 --2- 172.21.15.97:0/850285617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02480ca6d0 0x7f02481566c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 -- 172.21.15.97:0/850285617 >> 172.21.15.97:0/850285617 conn(0x7f024806a6c0 msgr2=0x7f0248159040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 -- 172.21.15.97:0/850285617 shutdown_connections 2023-07-06T03:45:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.400+0000 7f024dbff700 1 -- 172.21.15.97:0/850285617 wait complete. 2023-07-06T03:45:49.421 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:45:49.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.615+0000 7f4ab4bff700 1 Processor -- start 2023-07-06T03:45:49.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.616+0000 7f4ab4bff700 1 -- start start 2023-07-06T03:45:49.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4ab4bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4ab015c610 0x7f4ab00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4ab4bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab006b360 0x7f4ab00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4ab4bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ab00d2210 0x7f4ab00d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4ab4bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab00d0780 con 0x7f4ab015c610 2023-07-06T03:45:49.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4ab4bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab015c9e0 con 0x7f4ab00d2210 2023-07-06T03:45:49.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4ab4bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00b5650 con 0x7f4ab006b360 2023-07-06T03:45:49.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aae59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4ab015c610 0x7f4ab00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aae59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4ab015c610 0x7f4ab00b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44800/0 (socket says 172.21.15.97:44800) 2023-07-06T03:45:49.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aae59c700 1 -- 172.21.15.97:0/3465289889 learned_addr learned my addr 172.21.15.97:0/3465289889 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:49.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aa7fff700 1 --2- 172.21.15.97:0/3465289889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab006b360 0x7f4ab00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aaed9d700 1 --2- 172.21.15.97:0/3465289889 >> v2:172.21.15.146:3300/0 conn(0x7f4ab00d2210 0x7f4ab00d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aae59c700 1 -- 172.21.15.97:0/3465289889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab006b360 msgr2=0x7f4ab00b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:49.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aae59c700 1 --2- 172.21.15.97:0/3465289889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab006b360 0x7f4ab00b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aae59c700 1 -- 172.21.15.97:0/3465289889 >> v2:172.21.15.146:3300/0 conn(0x7f4ab00d2210 msgr2=0x7f4ab00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:49.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aae59c700 1 --2- 172.21.15.97:0/3465289889 >> v2:172.21.15.146:3300/0 conn(0x7f4ab00d2210 0x7f4ab00d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aae59c700 1 -- 172.21.15.97:0/3465289889 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00d01c0 con 0x7f4ab015c610 2023-07-06T03:45:49.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aa7fff700 1 --2- 172.21.15.97:0/3465289889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab006b360 0x7f4ab00b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:49.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.617+0000 7f4aaed9d700 1 --2- 172.21.15.97:0/3465289889 >> v2:172.21.15.146:3300/0 conn(0x7f4ab00d2210 0x7f4ab00d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:49.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.618+0000 7f4aae59c700 1 --2- 172.21.15.97:0/3465289889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab015c610 0x7f4ab00b47b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00b1b70 tx=0x7f4aa0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7cfe151c29f6cf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:49.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.618+0000 7f4aadd9b700 1 -- 172.21.15.97:0/3465289889 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000b860 con 0x7f4ab015c610 2023-07-06T03:45:49.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.618+0000 7f4aadd9b700 1 -- 172.21.15.97:0/3465289889 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa000b9c0 con 0x7f4ab015c610 2023-07-06T03:45:49.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.618+0000 7f4aadd9b700 1 -- 172.21.15.97:0/3465289889 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000bd50 con 0x7f4ab015c610 2023-07-06T03:45:49.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.618+0000 7f4ab4bff700 1 -- 172.21.15.97:0/3465289889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab015c610 msgr2=0x7f4ab00b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.618+0000 7f4ab4bff700 1 --2- 172.21.15.97:0/3465289889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab015c610 0x7f4ab00b47b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00b1b70 tx=0x7f4aa0009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:49.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.619+0000 7f4ab4bff700 1 -- 172.21.15.97:0/3465289889 shutdown_connections 2023-07-06T03:45:49.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.619+0000 7f4ab4bff700 1 --2- 172.21.15.97:0/3465289889 >> v2:172.21.15.146:3300/0 conn(0x7f4ab00d2210 0x7f4ab00d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.619+0000 7f4ab4bff700 1 --2- 172.21.15.97:0/3465289889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab015c610 0x7f4ab00b47b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.619+0000 7f4ab4bff700 1 --2- 172.21.15.97:0/3465289889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab006b360 0x7f4ab00b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.619+0000 7f4ab4bff700 1 -- 172.21.15.97:0/3465289889 >> 172.21.15.97:0/3465289889 conn(0x7f4ab006a6c0 msgr2=0x7f4ab006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:49.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.619+0000 7f4ab4bff700 1 -- 172.21.15.97:0/3465289889 shutdown_connections 2023-07-06T03:45:49.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.619+0000 7f4ab4bff700 1 -- 172.21.15.97:0/3465289889 wait complete. 2023-07-06T03:45:49.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.620+0000 7f4ab4bff700 1 Processor -- start 2023-07-06T03:45:49.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.621+0000 7f4ab4bff700 1 -- start start 2023-07-06T03:45:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.621+0000 7f4ab4bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab006b360 0x7f4ab015d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4ab4bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00d2210 0x7f4ab00cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4ab4bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab015c610 0x7f4ab0150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4ab4bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00cbf80 con 0x7f4ab006b360 2023-07-06T03:45:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4ab4bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00c6a00 con 0x7f4ab00d2210 2023-07-06T03:45:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4ab4bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab015e7b0 con 0x7f4ab015c610 2023-07-06T03:45:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4aae59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab006b360 0x7f4ab015d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4aae59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab006b360 0x7f4ab015d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44804/0 (socket says 172.21.15.97:44804) 2023-07-06T03:45:49.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4aae59c700 1 -- 172.21.15.97:0/1629532886 learned_addr learned my addr 172.21.15.97:0/1629532886 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:49.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4aaed9d700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab015c610 0x7f4ab0150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4aa7fff700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00d2210 0x7f4ab00cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4aae59c700 1 -- 172.21.15.97:0/1629532886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab015c610 msgr2=0x7f4ab0150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4aae59c700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab015c610 0x7f4ab0150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4aae59c700 1 -- 172.21.15.97:0/1629532886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00d2210 msgr2=0x7f4ab00cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4aae59c700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00d2210 0x7f4ab00cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4aae59c700 1 -- 172.21.15.97:0/1629532886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa0009550 con 0x7f4ab006b360 2023-07-06T03:45:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.622+0000 7f4aae59c700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab006b360 0x7f4ab015d530 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009130 tx=0x7f4aa0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.623+0000 7f4aa6ffd700 1 -- 172.21.15.97:0/1629532886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa00044d0 con 0x7f4ab006b360 2023-07-06T03:45:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.623+0000 7f4aa6ffd700 1 -- 172.21.15.97:0/1629532886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa001e070 con 0x7f4ab006b360 2023-07-06T03:45:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.623+0000 7f4ab4bff700 1 -- 172.21.15.97:0/1629532886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab01512a0 con 0x7f4ab006b360 2023-07-06T03:45:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.623+0000 7f4aa6ffd700 1 -- 172.21.15.97:0/1629532886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa00038e0 con 0x7f4ab006b360 2023-07-06T03:45:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.623+0000 7f4ab4bff700 1 -- 172.21.15.97:0/1629532886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab015de80 con 0x7f4ab006b360 2023-07-06T03:45:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.624+0000 7f4ab4bff700 1 -- 172.21.15.97:0/1629532886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab00ce5d0 con 0x7f4ab006b360 2023-07-06T03:45:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.625+0000 7f4aa6ffd700 1 -- 172.21.15.97:0/1629532886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4aa0003a40 con 0x7f4ab006b360 2023-07-06T03:45:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.626+0000 7f4aa6ffd700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a9006ace0 0x7f4a9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.626+0000 7f4aa6ffd700 1 -- 172.21.15.97:0/1629532886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4aa0020070 con 0x7f4ab006b360 2023-07-06T03:45:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.626+0000 7f4aa7fff700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a9006ace0 0x7f4a9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.627+0000 7f4aa7fff700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a9006ace0 0x7f4a9006d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00cc620 tx=0x7f4a98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.629+0000 7f4aa6ffd700 1 -- 172.21.15.97:0/1629532886 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4aa0056050 con 0x7f4ab006b360 2023-07-06T03:45:49.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.825+0000 7f4ab4bff700 1 -- 172.21.15.97:0/1629532886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4ab004e340 con 0x7f4ab006b360 2023-07-06T03:45:49.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.826+0000 7f4aa6ffd700 1 -- 172.21.15.97:0/1629532886 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4aa002a0a0 con 0x7f4ab006b360 2023-07-06T03:45:49.829 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:45:49.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 -- 172.21.15.97:0/1629532886 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a9006ace0 msgr2=0x7f4a9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a9006ace0 0x7f4a9006d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00cc620 tx=0x7f4a98008040 comp rx=0 tx=0).stop 2023-07-06T03:45:49.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 -- 172.21.15.97:0/1629532886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab006b360 msgr2=0x7f4ab015d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:49.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab006b360 0x7f4ab015d530 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009130 tx=0x7f4aa0004750 comp rx=0 tx=0).stop 2023-07-06T03:45:49.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 -- 172.21.15.97:0/1629532886 shutdown_connections 2023-07-06T03:45:49.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a9006ace0 0x7f4a9006d190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ab015c610 0x7f4ab0150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab006b360 0x7f4ab015d530 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 --2- 172.21.15.97:0/1629532886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00d2210 0x7f4ab00cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:49.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 -- 172.21.15.97:0/1629532886 >> 172.21.15.97:0/1629532886 conn(0x7f4ab006a6c0 msgr2=0x7f4ab00b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:49.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 -- 172.21.15.97:0/1629532886 shutdown_connections 2023-07-06T03:45:49.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:49.828+0000 7f4ab4bff700 1 -- 172.21.15.97:0/1629532886 wait complete. 2023-07-06T03:45:49.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:45:50.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.044+0000 7fc7e03ff700 1 Processor -- start 2023-07-06T03:45:50.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.044+0000 7fc7e03ff700 1 -- start start 2023-07-06T03:45:50.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.045+0000 7fc7e03ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7d8159070 0x7fc7d80c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.045+0000 7fc7e03ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7d80c3cd0 0x7fc7d80cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.045+0000 7fc7e03ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d80ca260 con 0x7fc7d8159440 2023-07-06T03:45:50.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.045+0000 7fc7e03ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d80c99c0 con 0x7fc7d8159070 2023-07-06T03:45:50.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.045+0000 7fc7e03ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d80c9ca0 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.045+0000 7fc7de19b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7d8159070 0x7fc7d80c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.045+0000 7fc7de99c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc7d8159440 0x7fc7d80b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58786/0 (socket says 172.21.15.97:58786) 2023-07-06T03:45:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.045+0000 7fc7de99c700 1 -- 172.21.15.97:0/3155530543 learned_addr learned my addr 172.21.15.97:0/3155530543 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.046+0000 7fc7dd99a700 1 --2- 172.21.15.97:0/3155530543 >> v2:172.21.15.146:3300/0 conn(0x7fc7d80c3cd0 0x7fc7d80cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.046+0000 7fc7de19b700 1 -- 172.21.15.97:0/3155530543 >> v2:172.21.15.146:3300/0 conn(0x7fc7d80c3cd0 msgr2=0x7fc7d80cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.046+0000 7fc7de19b700 1 --2- 172.21.15.97:0/3155530543 >> v2:172.21.15.146:3300/0 conn(0x7fc7d80c3cd0 0x7fc7d80cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.046+0000 7fc7de19b700 1 -- 172.21.15.97:0/3155530543 >> v1:172.21.15.97:6789/0 conn(0x7fc7d8159440 legacy=0x7fc7d80b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.046+0000 7fc7de19b700 1 -- 172.21.15.97:0/3155530543 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d80c9f80 con 0x7fc7d8159070 2023-07-06T03:45:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.046+0000 7fc7de19b700 1 --2- 172.21.15.97:0/3155530543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d8159070 0x7fc7d80c8f40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80cf9f0 tx=0x7fc7d4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55394e3df59bb5f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.046+0000 7fc7dd199700 1 -- 172.21.15.97:0/3155530543 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d400f8a0 con 0x7fc7d8159070 2023-07-06T03:45:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.046+0000 7fc7dd199700 1 -- 172.21.15.97:0/3155530543 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d400fa00 con 0x7fc7d8159070 2023-07-06T03:45:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.046+0000 7fc7dd199700 1 -- 172.21.15.97:0/3155530543 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d400fd90 con 0x7fc7d8159070 2023-07-06T03:45:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.047+0000 7fc7e03ff700 1 -- 172.21.15.97:0/3155530543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d8159070 msgr2=0x7fc7d80c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.047+0000 7fc7e03ff700 1 --2- 172.21.15.97:0/3155530543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d8159070 0x7fc7d80c8f40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80cf9f0 tx=0x7fc7d4009b20 comp rx=0 tx=0).stop 2023-07-06T03:45:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.047+0000 7fc7e03ff700 1 -- 172.21.15.97:0/3155530543 shutdown_connections 2023-07-06T03:45:50.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.047+0000 7fc7e03ff700 1 --2- 172.21.15.97:0/3155530543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d8159070 0x7fc7d80c8f40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.047+0000 7fc7e03ff700 1 --2- 172.21.15.97:0/3155530543 >> v2:172.21.15.146:3300/0 conn(0x7fc7d80c3cd0 0x7fc7d80cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.047+0000 7fc7e03ff700 1 -- 172.21.15.97:0/3155530543 >> 172.21.15.97:0/3155530543 conn(0x7fc7d806a060 msgr2=0x7fc7d806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:50.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.047+0000 7fc7e03ff700 1 -- 172.21.15.97:0/3155530543 shutdown_connections 2023-07-06T03:45:50.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.047+0000 7fc7e03ff700 1 -- 172.21.15.97:0/3155530543 wait complete. 2023-07-06T03:45:50.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.049+0000 7fc7e03ff700 1 Processor -- start 2023-07-06T03:45:50.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.049+0000 7fc7e03ff700 1 -- start start 2023-07-06T03:45:50.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7e03ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d80c3cd0 0x7fc7d80b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7e03ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7d8159070 0x7fc7d8155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7e03ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7d8159440 0x7fc7d815aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7e03ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8157d90 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7e03ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80b0330 con 0x7fc7d8159070 2023-07-06T03:45:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7e03ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80af450 con 0x7fc7d8159440 2023-07-06T03:45:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7de19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d80c3cd0 0x7fc7d80b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7de19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d80c3cd0 0x7fc7d80b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36936/0 (socket says 172.21.15.97:36936) 2023-07-06T03:45:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7de19b700 1 -- 172.21.15.97:0/1181326462 learned_addr learned my addr 172.21.15.97:0/1181326462 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7de19b700 1 -- 172.21.15.97:0/1181326462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7d8159440 msgr2=0x7fc7d815aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7de19b700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7d8159440 0x7fc7d815aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7de19b700 1 -- 172.21.15.97:0/1181326462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7d8159070 msgr2=0x7fc7d8155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7de19b700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7d8159070 0x7fc7d8155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.050+0000 7fc7de19b700 1 -- 172.21.15.97:0/1181326462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d40094a0 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.051+0000 7fc7de19b700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d80c3cd0 0x7fc7d80b2b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000c00 tx=0x7fc7d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.051+0000 7fc7ceffd700 1 -- 172.21.15.97:0/1181326462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d401e070 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.051+0000 7fc7ceffd700 1 -- 172.21.15.97:0/1181326462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d40045d0 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.051+0000 7fc7e03ff700 1 -- 172.21.15.97:0/1181326462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d80af170 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.051+0000 7fc7ceffd700 1 -- 172.21.15.97:0/1181326462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40038c0 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.051+0000 7fc7e03ff700 1 -- 172.21.15.97:0/1181326462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d80affc0 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.052+0000 7fc7e03ff700 1 -- 172.21.15.97:0/1181326462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d804e340 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.053+0000 7fc7ceffd700 1 -- 172.21.15.97:0/1181326462 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc7d4022070 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.054+0000 7fc7ceffd700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc7bc06ad10 0x7fc7bc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.054+0000 7fc7ceffd700 1 -- 172.21.15.97:0/1181326462 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7d408be10 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.054+0000 7fc7dd99a700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc7bc06ad10 0x7fc7bc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.055+0000 7fc7dd99a700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc7bc06ad10 0x7fc7bc06d1c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc7d81f1790 tx=0x7fc7c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:50.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.058+0000 7fc7ceffd700 1 -- 172.21.15.97:0/1181326462 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc7d4056f50 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.251+0000 7fc7e03ff700 1 -- 172.21.15.97:0/1181326462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7d806d220 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.252+0000 7fc7ceffd700 1 -- 172.21.15.97:0/1181326462 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7d405a570 con 0x7fc7d80c3cd0 2023-07-06T03:45:50.255 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:45:50.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 -- 172.21.15.97:0/1181326462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc7bc06ad10 msgr2=0x7fc7bc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc7bc06ad10 0x7fc7bc06d1c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc7d81f1790 tx=0x7fc7c8008040 comp rx=0 tx=0).stop 2023-07-06T03:45:50.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 -- 172.21.15.97:0/1181326462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d80c3cd0 msgr2=0x7fc7d80b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d80c3cd0 0x7fc7d80b2b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000c00 tx=0x7fc7d4004750 comp rx=0 tx=0).stop 2023-07-06T03:45:50.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 -- 172.21.15.97:0/1181326462 shutdown_connections 2023-07-06T03:45:50.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc7bc06ad10 0x7fc7bc06d1c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7d8159440 0x7fc7d815aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d80c3cd0 0x7fc7d80b2b90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 --2- 172.21.15.97:0/1181326462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7d8159070 0x7fc7d8155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 -- 172.21.15.97:0/1181326462 >> 172.21.15.97:0/1181326462 conn(0x7fc7d806a060 msgr2=0x7fc7d806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:50.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 -- 172.21.15.97:0/1181326462 shutdown_connections 2023-07-06T03:45:50.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.254+0000 7fc7e03ff700 1 -- 172.21.15.97:0/1181326462 wait complete. 2023-07-06T03:45:50.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:45:50.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.468+0000 7fdca21ff700 1 Processor -- start 2023-07-06T03:45:50.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.468+0000 7fdca21ff700 1 -- start start 2023-07-06T03:45:50.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.469+0000 7fdca21ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdc9c15c610 0x7fdc9c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.469+0000 7fdca21ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdc9c0d2210 0x7fdc9c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.469+0000 7fdca21ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc9c0cff90 con 0x7fdc9c06b360 2023-07-06T03:45:50.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.469+0000 7fdca21ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9c0d0780 con 0x7fdc9c0d2210 2023-07-06T03:45:50.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.469+0000 7fdca21ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9c15c9e0 con 0x7fdc9c15c610 2023-07-06T03:45:50.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.470+0000 7fdc9bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdc9c0d2210 0x7fdc9c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.470+0000 7fdc9bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdc9c0d2210 0x7fdc9c0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36944/0 (socket says 172.21.15.97:36944) 2023-07-06T03:45:50.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.470+0000 7fdc9bfff700 1 -- 172.21.15.97:0/4243789534 learned_addr learned my addr 172.21.15.97:0/4243789534 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:50.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.470+0000 7fdc9b7fe700 1 --2- 172.21.15.97:0/4243789534 >> v2:172.21.15.146:3300/0 conn(0x7fdc9c15c610 0x7fdc9c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.470+0000 7fdc9bfff700 1 -- 172.21.15.97:0/4243789534 >> v2:172.21.15.146:3300/0 conn(0x7fdc9c15c610 msgr2=0x7fdc9c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:50.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.470+0000 7fdc9bfff700 1 --2- 172.21.15.97:0/4243789534 >> v2:172.21.15.146:3300/0 conn(0x7fdc9c15c610 0x7fdc9c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.470+0000 7fdc9bfff700 1 -- 172.21.15.97:0/4243789534 >> v1:172.21.15.146:6789/0 conn(0x7fdc9c06b360 legacy=0x7fdc9c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.470+0000 7fdc9bfff700 1 -- 172.21.15.97:0/4243789534 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c0b5650 con 0x7fdc9c0d2210 2023-07-06T03:45:50.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.470+0000 7fdc9bfff700 1 --2- 172.21.15.97:0/4243789534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9c0d2210 0x7fdc9c0d1310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c08b000 tx=0x7fdc9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54628bfcd2204b4a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:50.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.471+0000 7fdc9a7fc700 1 -- 172.21.15.97:0/4243789534 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400e070 con 0x7fdc9c0d2210 2023-07-06T03:45:50.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.471+0000 7fdc9a7fc700 1 -- 172.21.15.97:0/4243789534 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9400b8d0 con 0x7fdc9c0d2210 2023-07-06T03:45:50.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.471+0000 7fdc9a7fc700 1 -- 172.21.15.97:0/4243789534 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400bc80 con 0x7fdc9c0d2210 2023-07-06T03:45:50.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.471+0000 7fdca21ff700 1 -- 172.21.15.97:0/4243789534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9c0d2210 msgr2=0x7fdc9c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.471+0000 7fdca21ff700 1 --2- 172.21.15.97:0/4243789534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9c0d2210 0x7fdc9c0d1310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c08b000 tx=0x7fdc9400b330 comp rx=0 tx=0).stop 2023-07-06T03:45:50.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.471+0000 7fdca21ff700 1 -- 172.21.15.97:0/4243789534 shutdown_connections 2023-07-06T03:45:50.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.471+0000 7fdca21ff700 1 --2- 172.21.15.97:0/4243789534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9c0d2210 0x7fdc9c0d1310 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.471+0000 7fdca21ff700 1 --2- 172.21.15.97:0/4243789534 >> v2:172.21.15.146:3300/0 conn(0x7fdc9c15c610 0x7fdc9c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.471+0000 7fdca21ff700 1 -- 172.21.15.97:0/4243789534 >> 172.21.15.97:0/4243789534 conn(0x7fdc9c06a6c0 msgr2=0x7fdc9c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:50.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.471+0000 7fdca21ff700 1 -- 172.21.15.97:0/4243789534 shutdown_connections 2023-07-06T03:45:50.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.471+0000 7fdca21ff700 1 -- 172.21.15.97:0/4243789534 wait complete. 2023-07-06T03:45:50.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.473+0000 7fdca21ff700 1 Processor -- start 2023-07-06T03:45:50.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.473+0000 7fdca21ff700 1 -- start start 2023-07-06T03:45:50.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.474+0000 7fdca21ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc9c06b360 0x7fdc9c14b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.474+0000 7fdca21ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9c0d2210 0x7fdc9c0c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.474+0000 7fdca21ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc9c15c610 0x7fdc9c0b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.474+0000 7fdca21ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0c9510 con 0x7fdc9c0d2210 2023-07-06T03:45:50.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.474+0000 7fdca21ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0c8890 con 0x7fdc9c15c610 2023-07-06T03:45:50.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.474+0000 7fdca21ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0c8b70 con 0x7fdc9c06b360 2023-07-06T03:45:50.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.474+0000 7fdc9affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9c0d2210 0x7fdc9c0c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.474+0000 7fdc9affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9c0d2210 0x7fdc9c0c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36948/0 (socket says 172.21.15.97:36948) 2023-07-06T03:45:50.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.474+0000 7fdc9affd700 1 -- 172.21.15.97:0/4135450255 learned_addr learned my addr 172.21.15.97:0/4135450255 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:50.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc9bfff700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc9c15c610 0x7fdc9c0b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc9b7fe700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc9c06b360 0x7fdc9c14b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc9affd700 1 -- 172.21.15.97:0/4135450255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc9c06b360 msgr2=0x7fdc9c14b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc9affd700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc9c06b360 0x7fdc9c14b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc9affd700 1 -- 172.21.15.97:0/4135450255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc9c15c610 msgr2=0x7fdc9c0b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc9affd700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc9c15c610 0x7fdc9c0b6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc9affd700 1 -- 172.21.15.97:0/4135450255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc94009b30 con 0x7fdc9c0d2210 2023-07-06T03:45:50.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc9b7fe700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc9c06b360 0x7fdc9c14b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:50.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc9affd700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9c0d2210 0x7fdc9c0c7e40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000c490 tx=0x7fdc9000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:50.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc83fff700 1 -- 172.21.15.97:0/4135450255 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900052d0 con 0x7fdc9c0d2210 2023-07-06T03:45:50.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc83fff700 1 -- 172.21.15.97:0/4135450255 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc90005450 con 0x7fdc9c0d2210 2023-07-06T03:45:50.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdc83fff700 1 -- 172.21.15.97:0/4135450255 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90006aa0 con 0x7fdc9c0d2210 2023-07-06T03:45:50.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.475+0000 7fdca21ff700 1 -- 172.21.15.97:0/4135450255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c0c8e50 con 0x7fdc9c0d2210 2023-07-06T03:45:50.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.476+0000 7fdca21ff700 1 -- 172.21.15.97:0/4135450255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c0c9930 con 0x7fdc9c0d2210 2023-07-06T03:45:50.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.476+0000 7fdca21ff700 1 -- 172.21.15.97:0/4135450255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9c0c9650 con 0x7fdc9c0d2210 2023-07-06T03:45:50.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.477+0000 7fdc83fff700 1 -- 172.21.15.97:0/4135450255 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc90003680 con 0x7fdc9c0d2210 2023-07-06T03:45:50.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.478+0000 7fdc83fff700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ace0 0x7fdc8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.478+0000 7fdc83fff700 1 -- 172.21.15.97:0/4135450255 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc90088700 con 0x7fdc9c0d2210 2023-07-06T03:45:50.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.479+0000 7fdc9b7fe700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ace0 0x7fdc8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.479+0000 7fdc9b7fe700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ace0 0x7fdc8806d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c1581e0 tx=0x7fdc8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:50.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.483+0000 7fdc83fff700 1 -- 172.21.15.97:0/4135450255 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdc9c0c9650 con 0x7fdc9c0d2210 2023-07-06T03:45:50.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.679+0000 7fdca21ff700 1 -- 172.21.15.97:0/4135450255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc9c0c9650 con 0x7fdc9c0d2210 2023-07-06T03:45:50.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.679+0000 7fdc83fff700 1 -- 172.21.15.97:0/4135450255 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc9c0c9650 con 0x7fdc9c0d2210 2023-07-06T03:45:50.682 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:45:50.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.681+0000 7fdca21ff700 1 -- 172.21.15.97:0/4135450255 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ace0 msgr2=0x7fdc8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.681+0000 7fdca21ff700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ace0 0x7fdc8806d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c1581e0 tx=0x7fdc8c008040 comp rx=0 tx=0).stop 2023-07-06T03:45:50.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.681+0000 7fdca21ff700 1 -- 172.21.15.97:0/4135450255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9c0d2210 msgr2=0x7fdc9c0c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.681+0000 7fdca21ff700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9c0d2210 0x7fdc9c0c7e40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000c490 tx=0x7fdc9000c7a0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.681+0000 7fdca21ff700 1 -- 172.21.15.97:0/4135450255 shutdown_connections 2023-07-06T03:45:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.681+0000 7fdca21ff700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ace0 0x7fdc8806d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.681+0000 7fdca21ff700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc9c15c610 0x7fdc9c0b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.681+0000 7fdca21ff700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc9c06b360 0x7fdc9c14b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.681+0000 7fdca21ff700 1 --2- 172.21.15.97:0/4135450255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9c0d2210 0x7fdc9c0c7e40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.681+0000 7fdca21ff700 1 -- 172.21.15.97:0/4135450255 >> 172.21.15.97:0/4135450255 conn(0x7fdc9c06a6c0 msgr2=0x7fdc9c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:50.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.681+0000 7fdca21ff700 1 -- 172.21.15.97:0/4135450255 shutdown_connections 2023-07-06T03:45:50.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.682+0000 7fdca21ff700 1 -- 172.21.15.97:0/4135450255 wait complete. 2023-07-06T03:45:50.702 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.896+0000 7f337d7ff700 1 Processor -- start 2023-07-06T03:45:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.896+0000 7f337d7ff700 1 -- start start 2023-07-06T03:45:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.897+0000 7f337d7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337815c520 0x7f33780cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.897+0000 7f337d7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33780cdae0 con 0x7f3378160d00 2023-07-06T03:45:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.897+0000 7f337d7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33780b6640 con 0x7f3378160180 2023-07-06T03:45:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.897+0000 7f337d7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33780cd8d0 con 0x7f337815c520 2023-07-06T03:45:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.897+0000 7f33777fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3378160d00 0x7f33780b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58794/0 (socket says 172.21.15.97:58794) 2023-07-06T03:45:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.897+0000 7f33777fe700 1 -- 172.21.15.97:0/2964359737 learned_addr learned my addr 172.21.15.97:0/2964359737 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f3376ffd700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337815c520 0x7f33780cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877988320 0 0) 0x7f33780cdae0 con 0x7f3378160d00 2023-07-06T03:45:50.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33640035b0 con 0x7f3378160d00 2023-07-06T03:45:50.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824974890 0 0) 0x7f33780b6640 con 0x7f3378160180 2023-07-06T03:45:50.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33780cdae0 con 0x7f3378160180 2023-07-06T03:45:50.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3856327248 0 0) 0x7f33640035b0 con 0x7f3378160d00 2023-07-06T03:45:50.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337815c520 msgr2=0x7f33780cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:50.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337815c520 0x7f33780cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 >> v1:172.21.15.146:6789/0 conn(0x7f3378160180 legacy=0x7f33781605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33780b6920 con 0x7f3378160d00 2023-07-06T03:45:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f33700019f0 con 0x7f3378160d00 2023-07-06T03:45:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f3376ffd700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337815c520 0x7f33780cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 >> v1:172.21.15.97:6789/0 conn(0x7f3378160d00 legacy=0x7f33780b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337815c520 0x7f3364003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3364004150 0x7f3364004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3364004d90 0x7f3364005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33780cd8d0 con 0x7f3364004d90 2023-07-06T03:45:50.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336400baf0 con 0x7f3364004150 2023-07-06T03:45:50.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336400bcb0 con 0x7f337815c520 2023-07-06T03:45:50.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f33777fe700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3364004d90 0x7f3364005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.898+0000 7f3376ffd700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337815c520 0x7f3364003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.899+0000 7f336ffff700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3364004150 0x7f3364004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.899+0000 7f33777fe700 1 -- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337815c520 msgr2=0x7f3364003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.899+0000 7f33777fe700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337815c520 0x7f3364003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.899+0000 7f33777fe700 1 -- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3364004150 msgr2=0x7f3364004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:50.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.899+0000 7f33777fe700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3364004150 0x7f3364004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.899+0000 7f33777fe700 1 -- 172.21.15.97:0/2964359737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33680094a0 con 0x7f3364004d90 2023-07-06T03:45:50.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.899+0000 7f3376ffd700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337815c520 0x7f3364003bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:45:50.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.899+0000 7f33777fe700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3364004d90 0x7f3364005200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f337000a710 tx=0x7f3364005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f31c0f99a232eec server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:50.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.899+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33700072d0 con 0x7f3364004d90 2023-07-06T03:45:50.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.899+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3370007430 con 0x7f3364004d90 2023-07-06T03:45:50.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.899+0000 7f33767fc700 1 -- 172.21.15.97:0/2964359737 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33700077e0 con 0x7f3364004d90 2023-07-06T03:45:50.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.900+0000 7f337d7ff700 1 -- 172.21.15.97:0/2964359737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3364004d90 msgr2=0x7f3364005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.900+0000 7f337d7ff700 1 --2- 172.21.15.97:0/2964359737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3364004d90 0x7f3364005200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f337000a710 tx=0x7f3364005880 comp rx=0 tx=0).stop 2023-07-06T03:45:50.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.900+0000 7f33777fe700 1 -- 172.21.15.97:0/2964359737 reap_dead start 2023-07-06T03:45:50.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.900+0000 7f337d7ff700 1 -- 172.21.15.97:0/2964359737 shutdown_connections 2023-07-06T03:45:50.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.900+0000 7f337d7ff700 1 -- 172.21.15.97:0/2964359737 >> 172.21.15.97:0/2964359737 conn(0x7f3378071790 msgr2=0x7f3378071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:50.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.900+0000 7f337d7ff700 1 -- 172.21.15.97:0/2964359737 shutdown_connections 2023-07-06T03:45:50.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.900+0000 7f337d7ff700 1 -- 172.21.15.97:0/2964359737 wait complete. 2023-07-06T03:45:50.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.902+0000 7f337d7ff700 1 Processor -- start 2023-07-06T03:45:50.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.902+0000 7f337d7ff700 1 -- start start 2023-07-06T03:45:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.903+0000 7f337d7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3364004d90 0x7f33780c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.903+0000 7f337d7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337814d530 0x7f337815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.903+0000 7f337d7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780d23f0 0x7f33780d2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.903+0000 7f337d7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33780d2da0 con 0x7f33780d23f0 2023-07-06T03:45:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.903+0000 7f337d7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33780c8d60 con 0x7f3364004d90 2023-07-06T03:45:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.903+0000 7f337d7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33780c9730 con 0x7f337814d530 2023-07-06T03:45:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.903+0000 7f33777fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780d23f0 0x7f33780d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.903+0000 7f33777fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780d23f0 0x7f33780d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36968/0 (socket says 172.21.15.97:36968) 2023-07-06T03:45:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.903+0000 7f33777fe700 1 -- 172.21.15.97:0/1287189787 learned_addr learned my addr 172.21.15.97:0/1287189787 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.903+0000 7f3376ffd700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3364004d90 0x7f33780c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.903+0000 7f336ffff700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337814d530 0x7f337815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.904+0000 7f33777fe700 1 -- 172.21.15.97:0/1287189787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337814d530 msgr2=0x7f337815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.904+0000 7f33777fe700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337814d530 0x7f337815dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.904+0000 7f33777fe700 1 -- 172.21.15.97:0/1287189787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3364004d90 msgr2=0x7f33780c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.904+0000 7f33777fe700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3364004d90 0x7f33780c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.904+0000 7f33777fe700 1 -- 172.21.15.97:0/1287189787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33780c7e70 con 0x7f33780d23f0 2023-07-06T03:45:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.904+0000 7f33777fe700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780d23f0 0x7f33780d2860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3370002230 tx=0x7f3370001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.904+0000 7f336f7fe700 1 -- 172.21.15.97:0/1287189787 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370025030 con 0x7f33780d23f0 2023-07-06T03:45:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.904+0000 7f336f7fe700 1 -- 172.21.15.97:0/1287189787 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33700079b0 con 0x7f33780d23f0 2023-07-06T03:45:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.904+0000 7f336f7fe700 1 -- 172.21.15.97:0/1287189787 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370012620 con 0x7f33780d23f0 2023-07-06T03:45:50.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.904+0000 7f337d7ff700 1 -- 172.21.15.97:0/1287189787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33780c8150 con 0x7f33780d23f0 2023-07-06T03:45:50.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.904+0000 7f337d7ff700 1 -- 172.21.15.97:0/1287189787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33780c8710 con 0x7f33780d23f0 2023-07-06T03:45:50.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.906+0000 7f336d7fa700 1 -- 172.21.15.97:0/1287189787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33500052f0 con 0x7f33780d23f0 2023-07-06T03:45:50.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.906+0000 7f336f7fe700 1 -- 172.21.15.97:0/1287189787 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3370028070 con 0x7f33780d23f0 2023-07-06T03:45:50.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.907+0000 7f336f7fe700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f335806ace0 0x7f335806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:50.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.907+0000 7f336f7fe700 1 -- 172.21.15.97:0/1287189787 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3370026070 con 0x7f33780d23f0 2023-07-06T03:45:50.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.907+0000 7f3376ffd700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f335806ace0 0x7f335806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:50.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.908+0000 7f3376ffd700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f335806ace0 0x7f335806d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f33680098b0 tx=0x7f3368009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:50.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:50.912+0000 7f336f7fe700 1 -- 172.21.15.97:0/1287189787 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3370056060 con 0x7f33780d23f0 2023-07-06T03:45:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.111+0000 7f336d7fa700 1 -- 172.21.15.97:0/1287189787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33500055a0 con 0x7f33780d23f0 2023-07-06T03:45:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.112+0000 7f336f7fe700 1 -- 172.21.15.97:0/1287189787 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f3370059680 con 0x7f33780d23f0 2023-07-06T03:45:51.115 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:51.115 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:51.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 -- 172.21.15.97:0/1287189787 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f335806ace0 msgr2=0x7f335806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f335806ace0 0x7f335806d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f33680098b0 tx=0x7f3368009210 comp rx=0 tx=0).stop 2023-07-06T03:45:51.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 -- 172.21.15.97:0/1287189787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780d23f0 msgr2=0x7f33780d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780d23f0 0x7f33780d2860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3370002230 tx=0x7f3370001cb0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 -- 172.21.15.97:0/1287189787 shutdown_connections 2023-07-06T03:45:51.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f335806ace0 0x7f335806d190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780d23f0 0x7f33780d2860 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3364004d90 0x7f33780c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 --2- 172.21.15.97:0/1287189787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f337814d530 0x7f337815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 -- 172.21.15.97:0/1287189787 >> 172.21.15.97:0/1287189787 conn(0x7f3378071790 msgr2=0x7f33780b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:51.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 -- 172.21.15.97:0/1287189787 shutdown_connections 2023-07-06T03:45:51.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.114+0000 7f336d7fa700 1 -- 172.21.15.97:0/1287189787 wait complete. 2023-07-06T03:45:51.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:45:51.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:45:51.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.331+0000 7eff0759e700 1 Processor -- start 2023-07-06T03:45:51.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.331+0000 7eff0759e700 1 -- start start 2023-07-06T03:45:51.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.332+0000 7eff0759e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7eff080b8470 0x7eff080b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.332+0000 7eff0759e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff080cd660 0x7eff080d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.332+0000 7eff0759e700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff080b1b00 0x7eff080d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.332+0000 7eff0759e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0815cb70 con 0x7eff080cd660 2023-07-06T03:45:51.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.332+0000 7eff0759e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff080cd1d0 con 0x7eff080b8470 2023-07-06T03:45:51.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.332+0000 7eff0759e700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff080d2690 con 0x7eff080b1b00 2023-07-06T03:45:51.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.332+0000 7eff05d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff080cd660 0x7eff080d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.332+0000 7eff05d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff080cd660 0x7eff080d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36982/0 (socket says 172.21.15.97:36982) 2023-07-06T03:45:51.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.333+0000 7eff05d9b700 1 -- 172.21.15.97:0/1852116090 learned_addr learned my addr 172.21.15.97:0/1852116090 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:51.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.333+0000 7eff0659c700 1 --2- 172.21.15.97:0/1852116090 >> v2:172.21.15.146:3300/0 conn(0x7eff080b8470 0x7eff080b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.333+0000 7eff06d9d700 1 --2- 172.21.15.97:0/1852116090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff080b1b00 0x7eff080d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.333+0000 7eff05d9b700 1 -- 172.21.15.97:0/1852116090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff080b1b00 msgr2=0x7eff080d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.333+0000 7eff05d9b700 1 --2- 172.21.15.97:0/1852116090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff080b1b00 0x7eff080d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.333+0000 7eff05d9b700 1 -- 172.21.15.97:0/1852116090 >> v2:172.21.15.146:3300/0 conn(0x7eff080b8470 msgr2=0x7eff080b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.333+0000 7eff05d9b700 1 --2- 172.21.15.97:0/1852116090 >> v2:172.21.15.146:3300/0 conn(0x7eff080b8470 0x7eff080b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.333+0000 7eff05d9b700 1 -- 172.21.15.97:0/1852116090 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff08160cc0 con 0x7eff080cd660 2023-07-06T03:45:51.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.333+0000 7eff05d9b700 1 --2- 172.21.15.97:0/1852116090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff080cd660 0x7eff080d0c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff080b5710 tx=0x7efefc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564d26826d034858 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:51.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.333+0000 7eff0559a700 1 -- 172.21.15.97:0/1852116090 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc011830 con 0x7eff080cd660 2023-07-06T03:45:51.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.333+0000 7eff0559a700 1 -- 172.21.15.97:0/1852116090 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efefc011990 con 0x7eff080cd660 2023-07-06T03:45:51.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.334+0000 7eff0559a700 1 -- 172.21.15.97:0/1852116090 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc011d20 con 0x7eff080cd660 2023-07-06T03:45:51.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.334+0000 7eff0759e700 1 -- 172.21.15.97:0/1852116090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff080cd660 msgr2=0x7eff080d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.334+0000 7eff0759e700 1 --2- 172.21.15.97:0/1852116090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff080cd660 0x7eff080d0c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff080b5710 tx=0x7efefc009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.334+0000 7eff0759e700 1 -- 172.21.15.97:0/1852116090 shutdown_connections 2023-07-06T03:45:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.334+0000 7eff0759e700 1 --2- 172.21.15.97:0/1852116090 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff080b1b00 0x7eff080d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.334+0000 7eff0759e700 1 --2- 172.21.15.97:0/1852116090 >> v2:172.21.15.146:3300/0 conn(0x7eff080b8470 0x7eff080b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.334+0000 7eff0759e700 1 --2- 172.21.15.97:0/1852116090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff080cd660 0x7eff080d0c20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.334+0000 7eff0759e700 1 -- 172.21.15.97:0/1852116090 >> 172.21.15.97:0/1852116090 conn(0x7eff0806b160 msgr2=0x7eff08071790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.334+0000 7eff0759e700 1 -- 172.21.15.97:0/1852116090 shutdown_connections 2023-07-06T03:45:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.334+0000 7eff0759e700 1 -- 172.21.15.97:0/1852116090 wait complete. 2023-07-06T03:45:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.336+0000 7eff0759e700 1 Processor -- start 2023-07-06T03:45:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.336+0000 7eff0759e700 1 -- start start 2023-07-06T03:45:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff080b1b00 0x7eff0814e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0759e700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff080b8470 0x7eff080c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff080cd660 0x7eff080c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0759e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0815f5f0 con 0x7eff080b1b00 2023-07-06T03:45:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0815e9e0 con 0x7eff080cd660 2023-07-06T03:45:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0759e700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0815ecc0 con 0x7eff080b8470 2023-07-06T03:45:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff080b1b00 0x7eff0814e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff080b1b00 0x7eff0814e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36994/0 (socket says 172.21.15.97:36994) 2023-07-06T03:45:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0659c700 1 -- 172.21.15.97:0/3812833530 learned_addr learned my addr 172.21.15.97:0/3812833530 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0659c700 1 -- 172.21.15.97:0/3812833530 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff080b8470 msgr2=0x7eff080c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff06d9d700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff080cd660 0x7eff080c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0659c700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff080b8470 0x7eff080c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0659c700 1 -- 172.21.15.97:0/3812833530 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff080cd660 msgr2=0x7eff080c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0659c700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff080cd660 0x7eff080c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff0659c700 1 -- 172.21.15.97:0/3812833530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc009550 con 0x7eff080b1b00 2023-07-06T03:45:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.337+0000 7eff06d9d700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff080cd660 0x7eff080c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.338+0000 7eff0659c700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff080b1b00 0x7eff0814e090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efef800e9b0 tx=0x7efef800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.338+0000 7efef6ffd700 1 -- 172.21.15.97:0/3812833530 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef8009710 con 0x7eff080b1b00 2023-07-06T03:45:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.338+0000 7efef6ffd700 1 -- 172.21.15.97:0/3812833530 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef800c850 con 0x7eff080b1b00 2023-07-06T03:45:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.338+0000 7efef6ffd700 1 -- 172.21.15.97:0/3812833530 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef800cbc0 con 0x7eff080b1b00 2023-07-06T03:45:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.338+0000 7eff0759e700 1 -- 172.21.15.97:0/3812833530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0815efa0 con 0x7eff080b1b00 2023-07-06T03:45:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.338+0000 7eff0759e700 1 -- 172.21.15.97:0/3812833530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0814e5d0 con 0x7eff080b1b00 2023-07-06T03:45:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.339+0000 7eff0759e700 1 -- 172.21.15.97:0/3812833530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff08068440 con 0x7eff080b1b00 2023-07-06T03:45:51.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.340+0000 7efef6ffd700 1 -- 172.21.15.97:0/3812833530 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efef800cd20 con 0x7eff080b1b00 2023-07-06T03:45:51.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.341+0000 7efef6ffd700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee806ace0 0x7efee806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.341+0000 7efef6ffd700 1 -- 172.21.15.97:0/3812833530 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efef8089620 con 0x7eff080b1b00 2023-07-06T03:45:51.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.341+0000 7eff05d9b700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee806ace0 0x7efee806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.342+0000 7eff05d9b700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee806ace0 0x7efee806d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7eff0815b280 tx=0x7efefc010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.345+0000 7efef6ffd700 1 -- 172.21.15.97:0/3812833530 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efef80580d0 con 0x7eff080b1b00 2023-07-06T03:45:51.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.538+0000 7eff0759e700 1 -- 172.21.15.97:0/3812833530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff080ccfb0 con 0x7eff080b1b00 2023-07-06T03:45:51.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.539+0000 7efef6ffd700 1 -- 172.21.15.97:0/3812833530 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7efef8057c60 con 0x7eff080b1b00 2023-07-06T03:45:51.542 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:45:51.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 -- 172.21.15.97:0/3812833530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee806ace0 msgr2=0x7efee806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee806ace0 0x7efee806d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7eff0815b280 tx=0x7efefc010000 comp rx=0 tx=0).stop 2023-07-06T03:45:51.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 -- 172.21.15.97:0/3812833530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff080b1b00 msgr2=0x7eff0814e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff080b1b00 0x7eff0814e090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efef800e9b0 tx=0x7efef800ed70 comp rx=0 tx=0).stop 2023-07-06T03:45:51.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 -- 172.21.15.97:0/3812833530 shutdown_connections 2023-07-06T03:45:51.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee806ace0 0x7efee806d190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff080cd660 0x7eff080c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff080b1b00 0x7eff0814e090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 --2- 172.21.15.97:0/3812833530 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff080b8470 0x7eff080c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 -- 172.21.15.97:0/3812833530 >> 172.21.15.97:0/3812833530 conn(0x7eff0806b160 msgr2=0x7eff080cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 -- 172.21.15.97:0/3812833530 shutdown_connections 2023-07-06T03:45:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.541+0000 7eff0759e700 1 -- 172.21.15.97:0/3812833530 wait complete. 2023-07-06T03:45:51.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:45:51.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.760+0000 7f38c25ff700 1 Processor -- start 2023-07-06T03:45:51.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.760+0000 7f38c25ff700 1 -- start start 2023-07-06T03:45:51.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.761+0000 7f38c25ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38bc15c610 0x7f38bc0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.761+0000 7f38c25ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38bc06b360 0x7f38bc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38c25ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38bc0cff90 con 0x7f38bc0d2210 2023-07-06T03:45:51.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38c25ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38bc0d0780 con 0x7f38bc06b360 2023-07-06T03:45:51.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38c25ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc15c9e0 con 0x7f38bc15c610 2023-07-06T03:45:51.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38b3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38bc06b360 0x7f38bc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38b3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38bc06b360 0x7f38bc0b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37006/0 (socket says 172.21.15.97:37006) 2023-07-06T03:45:51.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38b3fff700 1 -- 172.21.15.97:0/1537592918 learned_addr learned my addr 172.21.15.97:0/1537592918 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:51.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38b3fff700 1 -- 172.21.15.97:0/1537592918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38bc15c610 msgr2=0x7f38bc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:51.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38bbfff700 1 --2- 172.21.15.97:0/1537592918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38bc15c610 0x7f38bc0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38b3fff700 1 --2- 172.21.15.97:0/1537592918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38bc15c610 0x7f38bc0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38b3fff700 1 -- 172.21.15.97:0/1537592918 >> v1:172.21.15.146:6789/0 conn(0x7f38bc0d2210 legacy=0x7f38bc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:51.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38b3fff700 1 -- 172.21.15.97:0/1537592918 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc0b5650 con 0x7f38bc06b360 2023-07-06T03:45:51.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.762+0000 7f38bbfff700 1 --2- 172.21.15.97:0/1537592918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38bc15c610 0x7f38bc0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:51.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.763+0000 7f38b3fff700 1 --2- 172.21.15.97:0/1537592918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38bc06b360 0x7f38bc0b4cf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009920 tx=0x7f38a4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cce3c0c3c6565a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.763+0000 7f38bb7fe700 1 -- 172.21.15.97:0/1537592918 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4004230 con 0x7f38bc06b360 2023-07-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.763+0000 7f38bb7fe700 1 -- 172.21.15.97:0/1537592918 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a4004390 con 0x7f38bc06b360 2023-07-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.763+0000 7f38bb7fe700 1 -- 172.21.15.97:0/1537592918 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4009e70 con 0x7f38bc06b360 2023-07-06T03:45:51.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.763+0000 7f38c25ff700 1 -- 172.21.15.97:0/1537592918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38bc06b360 msgr2=0x7f38bc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.763+0000 7f38c25ff700 1 --2- 172.21.15.97:0/1537592918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38bc06b360 0x7f38bc0b4cf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009920 tx=0x7f38a4009c30 comp rx=0 tx=0).stop 2023-07-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.763+0000 7f38c25ff700 1 -- 172.21.15.97:0/1537592918 shutdown_connections 2023-07-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.763+0000 7f38c25ff700 1 --2- 172.21.15.97:0/1537592918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38bc15c610 0x7f38bc0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.763+0000 7f38c25ff700 1 --2- 172.21.15.97:0/1537592918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38bc06b360 0x7f38bc0b4cf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.763+0000 7f38c25ff700 1 -- 172.21.15.97:0/1537592918 >> 172.21.15.97:0/1537592918 conn(0x7f38bc06a6c0 msgr2=0x7f38bc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:51.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.763+0000 7f38c25ff700 1 -- 172.21.15.97:0/1537592918 shutdown_connections 2023-07-06T03:45:51.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.764+0000 7f38c25ff700 1 -- 172.21.15.97:0/1537592918 wait complete. 2023-07-06T03:45:51.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.765+0000 7f38c25ff700 1 Processor -- start 2023-07-06T03:45:51.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.765+0000 7f38c25ff700 1 -- start start 2023-07-06T03:45:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.766+0000 7f38c25ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38bc06b360 0x7f38bc0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.766+0000 7f38c25ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38bc0d2210 0x7f38bc1f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.766+0000 7f38c25ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38bc15c610 0x7f38bc15dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.766+0000 7f38c25ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0cb960 con 0x7f38bc06b360 2023-07-06T03:45:51.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.766+0000 7f38c25ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0ccc20 con 0x7f38bc15c610 2023-07-06T03:45:51.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.766+0000 7f38c25ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0caf90 con 0x7f38bc0d2210 2023-07-06T03:45:51.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.766+0000 7f38bbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38bc06b360 0x7f38bc0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.766+0000 7f38bbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38bc06b360 0x7f38bc0cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37014/0 (socket says 172.21.15.97:37014) 2023-07-06T03:45:51.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.766+0000 7f38bbfff700 1 -- 172.21.15.97:0/1404750838 learned_addr learned my addr 172.21.15.97:0/1404750838 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:51.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38b3fff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38bc0d2210 0x7f38bc1f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38c0b9c700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38bc15c610 0x7f38bc15dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38bbfff700 1 -- 172.21.15.97:0/1404750838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38bc0d2210 msgr2=0x7f38bc1f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38bbfff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38bc0d2210 0x7f38bc1f1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38bbfff700 1 -- 172.21.15.97:0/1404750838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38bc15c610 msgr2=0x7f38bc15dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38bbfff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38bc15c610 0x7f38bc15dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38bbfff700 1 -- 172.21.15.97:0/1404750838 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a4009550 con 0x7f38bc06b360 2023-07-06T03:45:51.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38bbfff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38bc06b360 0x7f38bc0cb420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00c780 tx=0x7f38ac00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:51.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38b97fa700 1 -- 172.21.15.97:0/1404750838 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac011800 con 0x7f38bc06b360 2023-07-06T03:45:51.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38b97fa700 1 -- 172.21.15.97:0/1404750838 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac011960 con 0x7f38bc06b360 2023-07-06T03:45:51.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38b97fa700 1 -- 172.21.15.97:0/1404750838 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac011cd0 con 0x7f38bc06b360 2023-07-06T03:45:51.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38c25ff700 1 -- 172.21.15.97:0/1404750838 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38bc0cc940 con 0x7f38bc06b360 2023-07-06T03:45:51.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.767+0000 7f38c25ff700 1 -- 172.21.15.97:0/1404750838 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc0cac20 con 0x7f38bc06b360 2023-07-06T03:45:51.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.768+0000 7f38c25ff700 1 -- 172.21.15.97:0/1404750838 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38bc0cb270 con 0x7f38bc06b360 2023-07-06T03:45:51.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.769+0000 7f38b97fa700 1 -- 172.21.15.97:0/1404750838 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38ac021350 con 0x7f38bc06b360 2023-07-06T03:45:51.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.770+0000 7f38b97fa700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f389c06ace0 0x7f389c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:51.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.770+0000 7f38b97fa700 1 -- 172.21.15.97:0/1404750838 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38ac089500 con 0x7f38bc06b360 2023-07-06T03:45:51.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.771+0000 7f38b3fff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f389c06ace0 0x7f389c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:51.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.771+0000 7f38b3fff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f389c06ace0 0x7f389c06d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f38bc06b790 tx=0x7f38a4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:51.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.774+0000 7f38b97fa700 1 -- 172.21.15.97:0/1404750838 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f38ac057fb0 con 0x7f38bc06b360 2023-07-06T03:45:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.972+0000 7f38c25ff700 1 -- 172.21.15.97:0/1404750838 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f38bc04e340 con 0x7f38bc06b360 2023-07-06T03:45:51.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.972+0000 7f38b97fa700 1 -- 172.21.15.97:0/1404750838 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f38ac057b40 con 0x7f38bc06b360 2023-07-06T03:45:51.976 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:45:51.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.974+0000 7f38c25ff700 1 -- 172.21.15.97:0/1404750838 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f389c06ace0 msgr2=0x7f389c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.974+0000 7f38c25ff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f389c06ace0 0x7f389c06d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f38bc06b790 tx=0x7f38a4008030 comp rx=0 tx=0).stop 2023-07-06T03:45:51.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.974+0000 7f38c25ff700 1 -- 172.21.15.97:0/1404750838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38bc06b360 msgr2=0x7f38bc0cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:51.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.974+0000 7f38c25ff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38bc06b360 0x7f38bc0cb420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00c780 tx=0x7f38ac00cb40 comp rx=0 tx=0).stop 2023-07-06T03:45:51.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.975+0000 7f38c25ff700 1 -- 172.21.15.97:0/1404750838 shutdown_connections 2023-07-06T03:45:51.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.975+0000 7f38c25ff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f389c06ace0 0x7f389c06d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.975+0000 7f38c25ff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38bc15c610 0x7f38bc15dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.975+0000 7f38c25ff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38bc06b360 0x7f38bc0cb420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.975+0000 7f38c25ff700 1 --2- 172.21.15.97:0/1404750838 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38bc0d2210 0x7f38bc1f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:51.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.975+0000 7f38c25ff700 1 -- 172.21.15.97:0/1404750838 >> 172.21.15.97:0/1404750838 conn(0x7f38bc06a6c0 msgr2=0x7f38bc0b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:51.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.975+0000 7f38c25ff700 1 -- 172.21.15.97:0/1404750838 shutdown_connections 2023-07-06T03:45:51.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:51.975+0000 7f38c25ff700 1 -- 172.21.15.97:0/1404750838 wait complete. 2023-07-06T03:45:51.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:45:52.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.191+0000 7f55135ff700 1 Processor -- start 2023-07-06T03:45:52.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.191+0000 7f55135ff700 1 -- start start 2023-07-06T03:45:52.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.192+0000 7f55135ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f550c0d00a0 0x7f550c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.192+0000 7f55135ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550c0d2d30 con 0x7f550c0b4a40 2023-07-06T03:45:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.192+0000 7f55135ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550c0b6520 con 0x7f550c0d15a0 2023-07-06T03:45:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.192+0000 7f55135ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0d1a10 con 0x7f550c0d00a0 2023-07-06T03:45:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.192+0000 7f551139b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f550c0b4a40 0x7f550c0b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58806/0 (socket says 172.21.15.97:58806) 2023-07-06T03:45:52.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.192+0000 7f551139b700 1 -- 172.21.15.97:0/3070993477 learned_addr learned my addr 172.21.15.97:0/3070993477 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:52.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.192+0000 7f5511b9c700 1 --2- 172.21.15.97:0/3070993477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f550c0d00a0 0x7f550c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.193+0000 7f5503fff700 1 -- 172.21.15.97:0/3070993477 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975676979 0 0) 0x7f550c0d2d30 con 0x7f550c0b4a40 2023-07-06T03:45:52.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.193+0000 7f5503fff700 1 -- 172.21.15.97:0/3070993477 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f80035b0 con 0x7f550c0b4a40 2023-07-06T03:45:52.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.193+0000 7f5503fff700 1 -- 172.21.15.97:0/3070993477 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288745959 0 0) 0x7f550c0b6520 con 0x7f550c0d15a0 2023-07-06T03:45:52.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.193+0000 7f5503fff700 1 -- 172.21.15.97:0/3070993477 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f550c0d2d30 con 0x7f550c0d15a0 2023-07-06T03:45:52.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.193+0000 7f5511b9c700 1 -- 172.21.15.97:0/3070993477 >> v1:172.21.15.146:6789/0 conn(0x7f550c0d15a0 legacy=0x7f550c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.193+0000 7f5511b9c700 1 -- 172.21.15.97:0/3070993477 >> v1:172.21.15.97:6789/0 conn(0x7f550c0b4a40 legacy=0x7f550c0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.193+0000 7f5511b9c700 1 -- 172.21.15.97:0/3070993477 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c0cf210 con 0x7f550c0d00a0 2023-07-06T03:45:52.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.193+0000 7f5511b9c700 1 --2- 172.21.15.97:0/3070993477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f550c0d00a0 0x7f550c0b4e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5504009dc0 tx=0x7f550400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d4cfa9abc7ed60c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:52.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.193+0000 7f5503fff700 1 -- 172.21.15.97:0/3070993477 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550400e070 con 0x7f550c0d00a0 2023-07-06T03:45:52.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.193+0000 7f5503fff700 1 -- 172.21.15.97:0/3070993477 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550400bb20 con 0x7f550c0d00a0 2023-07-06T03:45:52.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.193+0000 7f5503fff700 1 -- 172.21.15.97:0/3070993477 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504012430 con 0x7f550c0d00a0 2023-07-06T03:45:52.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.194+0000 7f55135ff700 1 -- 172.21.15.97:0/3070993477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f550c0d00a0 msgr2=0x7f550c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.194+0000 7f55135ff700 1 --2- 172.21.15.97:0/3070993477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f550c0d00a0 0x7f550c0b4e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5504009dc0 tx=0x7f550400b330 comp rx=0 tx=0).stop 2023-07-06T03:45:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.194+0000 7f55135ff700 1 -- 172.21.15.97:0/3070993477 shutdown_connections 2023-07-06T03:45:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.194+0000 7f55135ff700 1 --2- 172.21.15.97:0/3070993477 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f550c0d00a0 0x7f550c0b4e10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.194+0000 7f55135ff700 1 -- 172.21.15.97:0/3070993477 >> 172.21.15.97:0/3070993477 conn(0x7f550c06a900 msgr2=0x7f550c06acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:52.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.194+0000 7f55135ff700 1 -- 172.21.15.97:0/3070993477 shutdown_connections 2023-07-06T03:45:52.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.194+0000 7f55135ff700 1 -- 172.21.15.97:0/3070993477 wait complete. 2023-07-06T03:45:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.196+0000 7f55135ff700 1 Processor -- start 2023-07-06T03:45:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.196+0000 7f55135ff700 1 -- start start 2023-07-06T03:45:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.197+0000 7f55135ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f550c0b4a40 0x7f550c1e6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.197+0000 7f55135ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f550c0d00a0 0x7f550c0b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.197+0000 7f55135ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f550c0d15a0 0x7f550c0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.197+0000 7f55135ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c15d130 con 0x7f550c0d15a0 2023-07-06T03:45:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.197+0000 7f55135ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c15d6f0 con 0x7f550c0d00a0 2023-07-06T03:45:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.197+0000 7f55135ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c152820 con 0x7f550c0b4a40 2023-07-06T03:45:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.197+0000 7f5511b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f550c0d15a0 0x7f550c0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.197+0000 7f5511b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f550c0d15a0 0x7f550c0b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37030/0 (socket says 172.21.15.97:37030) 2023-07-06T03:45:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.197+0000 7f5511b9c700 1 -- 172.21.15.97:0/1342196771 learned_addr learned my addr 172.21.15.97:0/1342196771 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.197+0000 7f5510b9a700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f550c0d00a0 0x7f550c0b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.197+0000 7f551139b700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f550c0b4a40 0x7f550c1e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.198+0000 7f5511b9c700 1 -- 172.21.15.97:0/1342196771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f550c0b4a40 msgr2=0x7f550c1e6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.198+0000 7f5511b9c700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f550c0b4a40 0x7f550c1e6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.198+0000 7f5511b9c700 1 -- 172.21.15.97:0/1342196771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f550c0d00a0 msgr2=0x7f550c0b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.198+0000 7f5511b9c700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f550c0d00a0 0x7f550c0b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.198+0000 7f5511b9c700 1 -- 172.21.15.97:0/1342196771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5504009a70 con 0x7f550c0d15a0 2023-07-06T03:45:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.198+0000 7f5511b9c700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f550c0d15a0 0x7f550c0b3fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f550c06b7a0 tx=0x7f5504004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.198+0000 7f5501ffb700 1 -- 172.21.15.97:0/1342196771 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550401c810 con 0x7f550c0d15a0 2023-07-06T03:45:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.198+0000 7f5501ffb700 1 -- 172.21.15.97:0/1342196771 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5504020070 con 0x7f550c0d15a0 2023-07-06T03:45:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.198+0000 7f55135ff700 1 -- 172.21.15.97:0/1342196771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550c0b8340 con 0x7f550c0d15a0 2023-07-06T03:45:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.198+0000 7f5501ffb700 1 -- 172.21.15.97:0/1342196771 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550401cbd0 con 0x7f550c0d15a0 2023-07-06T03:45:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.198+0000 7f55135ff700 1 -- 172.21.15.97:0/1342196771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550c15d380 con 0x7f550c0d15a0 2023-07-06T03:45:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.199+0000 7f55135ff700 1 -- 172.21.15.97:0/1342196771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f550c0727a0 con 0x7f550c0d15a0 2023-07-06T03:45:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.200+0000 7f5501ffb700 1 -- 172.21.15.97:0/1342196771 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f550401cd30 con 0x7f550c0d15a0 2023-07-06T03:45:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.201+0000 7f5501ffb700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f54f006ace0 0x7f54f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.201+0000 7f5501ffb700 1 -- 172.21.15.97:0/1342196771 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5504022070 con 0x7f550c0d15a0 2023-07-06T03:45:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.201+0000 7f551139b700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f54f006ace0 0x7f54f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.202+0000 7f551139b700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f54f006ace0 0x7f54f006d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5508007fd0 tx=0x7f5508007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:52.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.205+0000 7f5501ffb700 1 -- 172.21.15.97:0/1342196771 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5504056490 con 0x7f550c0d15a0 2023-07-06T03:45:52.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.398+0000 7f55135ff700 1 -- 172.21.15.97:0/1342196771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f550c0d1330 con 0x7f550c0d15a0 2023-07-06T03:45:52.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.399+0000 7f5501ffb700 1 -- 172.21.15.97:0/1342196771 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5504059ab0 con 0x7f550c0d15a0 2023-07-06T03:45:52.402 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:45:52.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 -- 172.21.15.97:0/1342196771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f54f006ace0 msgr2=0x7f54f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f54f006ace0 0x7f54f006d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5508007fd0 tx=0x7f5508007de0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 -- 172.21.15.97:0/1342196771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f550c0d15a0 msgr2=0x7f550c0b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f550c0d15a0 0x7f550c0b3fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f550c06b7a0 tx=0x7f5504004ad0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 -- 172.21.15.97:0/1342196771 shutdown_connections 2023-07-06T03:45:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f54f006ace0 0x7f54f006d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f550c0d15a0 0x7f550c0b3fc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f550c0b4a40 0x7f550c1e6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 --2- 172.21.15.97:0/1342196771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f550c0d00a0 0x7f550c0b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 -- 172.21.15.97:0/1342196771 >> 172.21.15.97:0/1342196771 conn(0x7f550c06a900 msgr2=0x7f550c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 -- 172.21.15.97:0/1342196771 shutdown_connections 2023-07-06T03:45:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.401+0000 7f55135ff700 1 -- 172.21.15.97:0/1342196771 wait complete. 2023-07-06T03:45:52.422 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:45:52.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.617+0000 7f7100fff700 1 Processor -- start 2023-07-06T03:45:52.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.617+0000 7f7100fff700 1 -- start start 2023-07-06T03:45:52.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.618+0000 7f7100fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc15c610 0x7f70fc0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.619+0000 7f7100fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f70fc06b360 0x7f70fc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.619+0000 7f7100fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70fc0cff90 con 0x7f70fc0d2210 2023-07-06T03:45:52.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.619+0000 7f7100fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70fc0d0780 con 0x7f70fc06b360 2023-07-06T03:45:52.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.619+0000 7f7100fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc15c9e0 con 0x7f70fc15c610 2023-07-06T03:45:52.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.619+0000 7f70fad9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f70fc0d2210 0x7f70fc0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58808/0 (socket says 172.21.15.97:58808) 2023-07-06T03:45:52.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.619+0000 7f70fad9d700 1 -- 172.21.15.97:0/1199186534 learned_addr learned my addr 172.21.15.97:0/1199186534 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:52.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.619+0000 7f70fa59c700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc15c610 0x7f70fc0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.619+0000 7f70f3fff700 1 --2- 172.21.15.97:0/1199186534 >> v2:172.21.15.146:3300/0 conn(0x7f70fc06b360 0x7f70fc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.619+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529324470 0 0) 0x7f70fc0cff90 con 0x7f70fc0d2210 2023-07-06T03:45:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.619+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70e80035b0 con 0x7f70fc0d2210 2023-07-06T03:45:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1910913329 0 0) 0x7f70e80035b0 con 0x7f70fc0d2210 2023-07-06T03:45:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc15c610 msgr2=0x7f70fc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc15c610 0x7f70fc0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 >> v2:172.21.15.146:3300/0 conn(0x7f70fc06b360 msgr2=0x7f70fc0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 --2- 172.21.15.97:0/1199186534 >> v2:172.21.15.146:3300/0 conn(0x7f70fc06b360 0x7f70fc0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc0b5650 con 0x7f70fc0d2210 2023-07-06T03:45:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f70f40019f0 con 0x7f70fc0d2210 2023-07-06T03:45:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 >> v1:172.21.15.97:6789/0 conn(0x7f70fc0d2210 legacy=0x7f70fc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70e8003a30 0x7f70e8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 0x7f70e8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc15c610 0x7f70e8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc0d0780 con 0x7f70e8004850 2023-07-06T03:45:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc15c9e0 con 0x7f70e8003a30 2023-07-06T03:45:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e800ba10 con 0x7f70fc15c610 2023-07-06T03:45:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f3fff700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 0x7f70e8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70fa59c700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70e8003a30 0x7f70e8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70fad9d700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc15c610 0x7f70e8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f3fff700 1 -- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc15c610 msgr2=0x7f70e8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f3fff700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc15c610 0x7f70e8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f3fff700 1 -- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70e8003a30 msgr2=0x7f70e8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:52.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f3fff700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70e8003a30 0x7f70e8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70f3fff700 1 -- 172.21.15.97:0/1199186534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e80035b0 con 0x7f70e8004850 2023-07-06T03:45:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.620+0000 7f70fad9d700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc15c610 0x7f70e8005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.621+0000 7f70f3fff700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 0x7f70e8004cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009b30 tx=0x7f70e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b8c4a2e20ba6b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.621+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e400e6c0 con 0x7f70e8004850 2023-07-06T03:45:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.621+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e400e820 con 0x7f70e8004850 2023-07-06T03:45:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.621+0000 7f70f9d9b700 1 -- 172.21.15.97:0/1199186534 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e400eb90 con 0x7f70e8004850 2023-07-06T03:45:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.621+0000 7f7100fff700 1 -- 172.21.15.97:0/1199186534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 msgr2=0x7f70e8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.621+0000 7f7100fff700 1 --2- 172.21.15.97:0/1199186534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 0x7f70e8004cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009b30 tx=0x7f70e8005880 comp rx=0 tx=0).stop 2023-07-06T03:45:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.621+0000 7f70fad9d700 1 -- 172.21.15.97:0/1199186534 reap_dead start 2023-07-06T03:45:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.621+0000 7f7100fff700 1 -- 172.21.15.97:0/1199186534 shutdown_connections 2023-07-06T03:45:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.621+0000 7f7100fff700 1 -- 172.21.15.97:0/1199186534 >> 172.21.15.97:0/1199186534 conn(0x7f70fc06a6c0 msgr2=0x7f70fc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.622+0000 7f7100fff700 1 -- 172.21.15.97:0/1199186534 shutdown_connections 2023-07-06T03:45:52.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.622+0000 7f7100fff700 1 -- 172.21.15.97:0/1199186534 wait complete. 2023-07-06T03:45:52.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.623+0000 7f7100fff700 1 Processor -- start 2023-07-06T03:45:52.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.623+0000 7f7100fff700 1 -- start start 2023-07-06T03:45:52.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.624+0000 7f7100fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 0x7f70fc0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.624+0000 7f7100fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc14f4c0 0x7f70fc0c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.624+0000 7f7100fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70fc0c6f70 0x7f70fc15e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.624+0000 7f7100fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc1f1fd0 con 0x7f70e8004850 2023-07-06T03:45:52.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.624+0000 7f7100fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc15df80 con 0x7f70fc0c6f70 2023-07-06T03:45:52.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.624+0000 7f7100fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc0c73e0 con 0x7f70fc14f4c0 2023-07-06T03:45:52.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70fa59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 0x7f70fc0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70fa59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 0x7f70fc0c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37056/0 (socket says 172.21.15.97:37056) 2023-07-06T03:45:52.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70fa59c700 1 -- 172.21.15.97:0/2422789342 learned_addr learned my addr 172.21.15.97:0/2422789342 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:52.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70f3fff700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc14f4c0 0x7f70fc0c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70fad9d700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70fc0c6f70 0x7f70fc15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70fa59c700 1 -- 172.21.15.97:0/2422789342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc14f4c0 msgr2=0x7f70fc0c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70fa59c700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc14f4c0 0x7f70fc0c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70fa59c700 1 -- 172.21.15.97:0/2422789342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70fc0c6f70 msgr2=0x7f70fc15e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70fa59c700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70fc0c6f70 0x7f70fc15e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70fa59c700 1 -- 172.21.15.97:0/2422789342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc0cc440 con 0x7f70e8004850 2023-07-06T03:45:52.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70fa59c700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 0x7f70fc0c9180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f70ec011ad0 tx=0x7f70ec011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:52.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70f2ffd700 1 -- 172.21.15.97:0/2422789342 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00f940 con 0x7f70e8004850 2023-07-06T03:45:52.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f70f2ffd700 1 -- 172.21.15.97:0/2422789342 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec00faa0 con 0x7f70e8004850 2023-07-06T03:45:52.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.625+0000 7f7100fff700 1 -- 172.21.15.97:0/2422789342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc15d930 con 0x7f70e8004850 2023-07-06T03:45:52.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.626+0000 7f70f2ffd700 1 -- 172.21.15.97:0/2422789342 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00fe50 con 0x7f70e8004850 2023-07-06T03:45:52.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.626+0000 7f7100fff700 1 -- 172.21.15.97:0/2422789342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc0cc720 con 0x7f70e8004850 2023-07-06T03:45:52.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.627+0000 7f7100fff700 1 -- 172.21.15.97:0/2422789342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70fc15e260 con 0x7f70e8004850 2023-07-06T03:45:52.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.627+0000 7f70f2ffd700 1 -- 172.21.15.97:0/2422789342 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70ec014440 con 0x7f70e8004850 2023-07-06T03:45:52.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.629+0000 7f70f2ffd700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f70dc06ace0 0x7f70dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:52.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.629+0000 7f70f2ffd700 1 -- 172.21.15.97:0/2422789342 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70ec089af0 con 0x7f70e8004850 2023-07-06T03:45:52.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.629+0000 7f70f3fff700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f70dc06ace0 0x7f70dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:52.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.630+0000 7f70f3fff700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f70dc06ace0 0x7f70dc06d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70e40090b0 tx=0x7f70e401f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:52.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.633+0000 7f70f2ffd700 1 -- 172.21.15.97:0/2422789342 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f70ec054bc0 con 0x7f70e8004850 2023-07-06T03:45:52.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.831+0000 7f7100fff700 1 -- 172.21.15.97:0/2422789342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70fc0cd4e0 con 0x7f70e8004850 2023-07-06T03:45:52.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.832+0000 7f70f2ffd700 1 -- 172.21.15.97:0/2422789342 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70fc0cd4e0 con 0x7f70e8004850 2023-07-06T03:45:52.835 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:45:52.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 -- 172.21.15.97:0/2422789342 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f70dc06ace0 msgr2=0x7f70dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f70dc06ace0 0x7f70dc06d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70e40090b0 tx=0x7f70e401f040 comp rx=0 tx=0).stop 2023-07-06T03:45:52.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 -- 172.21.15.97:0/2422789342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 msgr2=0x7f70fc0c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:52.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 0x7f70fc0c9180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f70ec011ad0 tx=0x7f70ec011de0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 -- 172.21.15.97:0/2422789342 shutdown_connections 2023-07-06T03:45:52.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f70dc06ace0 0x7f70dc06d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70fc0c6f70 0x7f70fc15e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70e8004850 0x7f70fc0c9180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 --2- 172.21.15.97:0/2422789342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70fc14f4c0 0x7f70fc0c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:52.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 -- 172.21.15.97:0/2422789342 >> 172.21.15.97:0/2422789342 conn(0x7f70fc06a6c0 msgr2=0x7f70fc0cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:52.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 -- 172.21.15.97:0/2422789342 shutdown_connections 2023-07-06T03:45:52.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:52.834+0000 7f7100fff700 1 -- 172.21.15.97:0/2422789342 wait complete. 2023-07-06T03:45:52.855 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:53.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.046+0000 7f23d2bff700 1 Processor -- start 2023-07-06T03:45:53.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.046+0000 7f23d2bff700 1 -- start start 2023-07-06T03:45:53.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.048+0000 7f23d2bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23cc0cd390 0x7f23cc160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.048+0000 7f23d2bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc160dd0 0x7f23cc0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.048+0000 7f23d2bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc0cdae0 con 0x7f23cc15c520 2023-07-06T03:45:53.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.048+0000 7f23d2bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23cc0b6640 con 0x7f23cc0cd390 2023-07-06T03:45:53.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.048+0000 7f23d2bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0b6920 con 0x7f23cc160dd0 2023-07-06T03:45:53.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.048+0000 7f23d119c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc160dd0 0x7f23cc0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.048+0000 7f23cbfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23cc0cd390 0x7f23cc160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.048+0000 7f23d119c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc160dd0 0x7f23cc0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:39982/0 (socket says 172.21.15.97:39982) 2023-07-06T03:45:53.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.048+0000 7f23d119c700 1 -- 172.21.15.97:0/2780594399 learned_addr learned my addr 172.21.15.97:0/2780594399 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:53.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.049+0000 7f23d119c700 1 -- 172.21.15.97:0/2780594399 >> v2:172.21.15.146:3300/0 conn(0x7f23cc0cd390 msgr2=0x7f23cc160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:53.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.049+0000 7f23d119c700 1 --2- 172.21.15.97:0/2780594399 >> v2:172.21.15.146:3300/0 conn(0x7f23cc0cd390 0x7f23cc160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.049+0000 7f23d119c700 1 -- 172.21.15.97:0/2780594399 >> v1:172.21.15.146:6789/0 conn(0x7f23cc15c520 legacy=0x7f23cc160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.049+0000 7f23d119c700 1 -- 172.21.15.97:0/2780594399 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc0b6c00 con 0x7f23cc160dd0 2023-07-06T03:45:53.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.049+0000 7f23d119c700 1 --2- 172.21.15.97:0/2780594399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc160dd0 0x7f23cc0b7400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009e50 tx=0x7f23c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c83defda13a740e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:53.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.049+0000 7f23cb7fe700 1 -- 172.21.15.97:0/2780594399 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c400e070 con 0x7f23cc160dd0 2023-07-06T03:45:53.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.049+0000 7f23cb7fe700 1 -- 172.21.15.97:0/2780594399 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c400b910 con 0x7f23cc160dd0 2023-07-06T03:45:53.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.049+0000 7f23cb7fe700 1 -- 172.21.15.97:0/2780594399 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c400bcc0 con 0x7f23cc160dd0 2023-07-06T03:45:53.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.050+0000 7f23d2bff700 1 -- 172.21.15.97:0/2780594399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc160dd0 msgr2=0x7f23cc0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.050+0000 7f23d2bff700 1 --2- 172.21.15.97:0/2780594399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc160dd0 0x7f23cc0b7400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009e50 tx=0x7f23c400b330 comp rx=0 tx=0).stop 2023-07-06T03:45:53.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.050+0000 7f23d2bff700 1 -- 172.21.15.97:0/2780594399 shutdown_connections 2023-07-06T03:45:53.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.050+0000 7f23d2bff700 1 --2- 172.21.15.97:0/2780594399 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc160dd0 0x7f23cc0b7400 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.050+0000 7f23d2bff700 1 --2- 172.21.15.97:0/2780594399 >> v2:172.21.15.146:3300/0 conn(0x7f23cc0cd390 0x7f23cc160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.050+0000 7f23d2bff700 1 -- 172.21.15.97:0/2780594399 >> 172.21.15.97:0/2780594399 conn(0x7f23cc071790 msgr2=0x7f23cc071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:53.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.050+0000 7f23d2bff700 1 -- 172.21.15.97:0/2780594399 shutdown_connections 2023-07-06T03:45:53.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.050+0000 7f23d2bff700 1 -- 172.21.15.97:0/2780594399 wait complete. 2023-07-06T03:45:53.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.052+0000 7f23d2bff700 1 Processor -- start 2023-07-06T03:45:53.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.052+0000 7f23d2bff700 1 -- start start 2023-07-06T03:45:53.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23d2bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc0cd390 0x7f23cc0d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23d2bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc15c520 0x7f23cc14cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23d2bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23cc160dd0 0x7f23cc0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23d2bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0d3c20 con 0x7f23cc15c520 2023-07-06T03:45:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23d2bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc15dcf0 con 0x7f23cc160dd0 2023-07-06T03:45:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23d2bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0d3130 con 0x7f23cc0cd390 2023-07-06T03:45:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23cbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc15c520 0x7f23cc14cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23cbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc15c520 0x7f23cc14cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37058/0 (socket says 172.21.15.97:37058) 2023-07-06T03:45:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23cbfff700 1 -- 172.21.15.97:0/1037844085 learned_addr learned my addr 172.21.15.97:0/1037844085 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23d099b700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc0cd390 0x7f23cc0d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23d119c700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23cc160dd0 0x7f23cc0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23cbfff700 1 -- 172.21.15.97:0/1037844085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc0cd390 msgr2=0x7f23cc0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23cbfff700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc0cd390 0x7f23cc0d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23cbfff700 1 -- 172.21.15.97:0/1037844085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23cc160dd0 msgr2=0x7f23cc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23cbfff700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23cc160dd0 0x7f23cc0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.053+0000 7f23cbfff700 1 -- 172.21.15.97:0/1037844085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4009b30 con 0x7f23cc15c520 2023-07-06T03:45:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.054+0000 7f23cbfff700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc15c520 0x7f23cc14cac0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f23b800ea10 tx=0x7f23b800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.054+0000 7f23c97fa700 1 -- 172.21.15.97:0/1037844085 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8009710 con 0x7f23cc15c520 2023-07-06T03:45:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.054+0000 7f23c97fa700 1 -- 172.21.15.97:0/1037844085 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b800eeb0 con 0x7f23cc15c520 2023-07-06T03:45:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.054+0000 7f23d2bff700 1 -- 172.21.15.97:0/1037844085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc14d6f0 con 0x7f23cc15c520 2023-07-06T03:45:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.054+0000 7f23c97fa700 1 -- 172.21.15.97:0/1037844085 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b800ca50 con 0x7f23cc15c520 2023-07-06T03:45:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.054+0000 7f23d2bff700 1 -- 172.21.15.97:0/1037844085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc0d3380 con 0x7f23cc15c520 2023-07-06T03:45:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.055+0000 7f23d2bff700 1 -- 172.21.15.97:0/1037844085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc15de80 con 0x7f23cc15c520 2023-07-06T03:45:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.056+0000 7f23c97fa700 1 -- 172.21.15.97:0/1037844085 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f23b800cbb0 con 0x7f23cc15c520 2023-07-06T03:45:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.057+0000 7f23c97fa700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f23b006ace0 0x7f23b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.057+0000 7f23c97fa700 1 -- 172.21.15.97:0/1037844085 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23b8089430 con 0x7f23cc15c520 2023-07-06T03:45:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.057+0000 7f23d099b700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f23b006ace0 0x7f23b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.058+0000 7f23d099b700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f23b006ace0 0x7f23b006d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f23c0007fd0 tx=0x7f23c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:53.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.061+0000 7f23c97fa700 1 -- 172.21.15.97:0/1037844085 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f23b8057f60 con 0x7f23cc15c520 2023-07-06T03:45:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.257+0000 7f23d2bff700 1 -- 172.21.15.97:0/1037844085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23cc04e340 con 0x7f23cc15c520 2023-07-06T03:45:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.259+0000 7f23c97fa700 1 -- 172.21.15.97:0/1037844085 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f23b8057af0 con 0x7f23cc15c520 2023-07-06T03:45:53.262 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:53.262 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:53.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 -- 172.21.15.97:0/1037844085 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f23b006ace0 msgr2=0x7f23b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f23b006ace0 0x7f23b006d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f23c0007fd0 tx=0x7f23c0009040 comp rx=0 tx=0).stop 2023-07-06T03:45:53.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 -- 172.21.15.97:0/1037844085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc15c520 msgr2=0x7f23cc14cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc15c520 0x7f23cc14cac0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f23b800ea10 tx=0x7f23b800ed20 comp rx=0 tx=0).stop 2023-07-06T03:45:53.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 -- 172.21.15.97:0/1037844085 shutdown_connections 2023-07-06T03:45:53.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f23b006ace0 0x7f23b006d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23cc160dd0 0x7f23cc0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f23cc0cd390 0x7f23cc0d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 --2- 172.21.15.97:0/1037844085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23cc15c520 0x7f23cc14cac0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 -- 172.21.15.97:0/1037844085 >> 172.21.15.97:0/1037844085 conn(0x7f23cc071790 msgr2=0x7f23cc071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:53.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 -- 172.21.15.97:0/1037844085 shutdown_connections 2023-07-06T03:45:53.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.261+0000 7f23d2bff700 1 -- 172.21.15.97:0/1037844085 wait complete. 2023-07-06T03:45:53.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:45:53.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:45:53.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.477+0000 7f742dfff700 1 Processor -- start 2023-07-06T03:45:53.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.478+0000 7f742dfff700 1 -- start start 2023-07-06T03:45:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.479+0000 7f742dfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f742815c1c0 0x7f74280cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.479+0000 7f742dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74280719c0 0x7f742806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.479+0000 7f742dfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742815da10 con 0x7f742815c590 2023-07-06T03:45:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.479+0000 7f742dfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74280ccc30 con 0x7f742815c1c0 2023-07-06T03:45:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.479+0000 7f742dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74280ccf10 con 0x7f74280719c0 2023-07-06T03:45:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.479+0000 7f74277fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f742815c1c0 0x7f74280cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.479+0000 7f74277fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f742815c1c0 0x7f74280cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37074/0 (socket says 172.21.15.97:37074) 2023-07-06T03:45:53.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.479+0000 7f74277fe700 1 -- 172.21.15.97:0/1336213507 learned_addr learned my addr 172.21.15.97:0/1336213507 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:53.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.479+0000 7f7427fff700 1 --2- 172.21.15.97:0/1336213507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74280719c0 0x7f742806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.480+0000 7f74277fe700 1 -- 172.21.15.97:0/1336213507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74280719c0 msgr2=0x7f742806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:53.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.480+0000 7f74277fe700 1 --2- 172.21.15.97:0/1336213507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74280719c0 0x7f742806a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.480+0000 7f74277fe700 1 -- 172.21.15.97:0/1336213507 >> v1:172.21.15.146:6789/0 conn(0x7f742815c590 legacy=0x7f742815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:53.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.480+0000 7f74277fe700 1 -- 172.21.15.97:0/1336213507 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74280cd1f0 con 0x7f742815c1c0 2023-07-06T03:45:53.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.480+0000 7f74277fe700 1 --2- 172.21.15.97:0/1336213507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742815c1c0 0x7f74280cc3e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f74180098f0 tx=0x7f7418009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58220d5ae6b0584 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:53.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.480+0000 7f7426ffd700 1 -- 172.21.15.97:0/1336213507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418004230 con 0x7f742815c1c0 2023-07-06T03:45:53.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.480+0000 7f7426ffd700 1 -- 172.21.15.97:0/1336213507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7418004390 con 0x7f742815c1c0 2023-07-06T03:45:53.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.480+0000 7f7426ffd700 1 -- 172.21.15.97:0/1336213507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74180036a0 con 0x7f742815c1c0 2023-07-06T03:45:53.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.481+0000 7f742dfff700 1 -- 172.21.15.97:0/1336213507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742815c1c0 msgr2=0x7f74280cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.481+0000 7f742dfff700 1 --2- 172.21.15.97:0/1336213507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742815c1c0 0x7f74280cc3e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f74180098f0 tx=0x7f7418009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:53.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.481+0000 7f742dfff700 1 -- 172.21.15.97:0/1336213507 shutdown_connections 2023-07-06T03:45:53.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.481+0000 7f742dfff700 1 --2- 172.21.15.97:0/1336213507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74280719c0 0x7f742806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.481+0000 7f742dfff700 1 --2- 172.21.15.97:0/1336213507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742815c1c0 0x7f74280cc3e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.481+0000 7f742dfff700 1 -- 172.21.15.97:0/1336213507 >> 172.21.15.97:0/1336213507 conn(0x7f74280b1650 msgr2=0x7f74280b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:53.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.481+0000 7f742dfff700 1 -- 172.21.15.97:0/1336213507 shutdown_connections 2023-07-06T03:45:53.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.481+0000 7f742dfff700 1 -- 172.21.15.97:0/1336213507 wait complete. 2023-07-06T03:45:53.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.482+0000 7f742dfff700 1 Processor -- start 2023-07-06T03:45:53.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.483+0000 7f742dfff700 1 -- start start 2023-07-06T03:45:53.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.483+0000 7f742dfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280719c0 0x7f74281499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.483+0000 7f742dfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f742815c1c0 0x7f7428158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.483+0000 7f742dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742815c590 0x7f74280b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.483+0000 7f742dfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7428160120 con 0x7f74280719c0 2023-07-06T03:45:53.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.483+0000 7f742dfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74280b2950 con 0x7f742815c1c0 2023-07-06T03:45:53.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.483+0000 7f742dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74281f6410 con 0x7f742815c590 2023-07-06T03:45:53.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.484+0000 7f74277fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280719c0 0x7f74281499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.484+0000 7f74277fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280719c0 0x7f74281499a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37088/0 (socket says 172.21.15.97:37088) 2023-07-06T03:45:53.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.484+0000 7f74277fe700 1 -- 172.21.15.97:0/3602709599 learned_addr learned my addr 172.21.15.97:0/3602709599 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:53.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.484+0000 7f74277fe700 1 -- 172.21.15.97:0/3602709599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742815c590 msgr2=0x7f74280b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.484+0000 7f74277fe700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742815c590 0x7f74280b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.484+0000 7f74277fe700 1 -- 172.21.15.97:0/3602709599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f742815c1c0 msgr2=0x7f7428158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.484+0000 7f74277fe700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f742815c1c0 0x7f7428158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.484+0000 7f74277fe700 1 -- 172.21.15.97:0/3602709599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7418009550 con 0x7f74280719c0 2023-07-06T03:45:53.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.484+0000 7f74277fe700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280719c0 0x7f74281499a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f74280b26f0 tx=0x7f7418004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:53.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.484+0000 7f7424ff9700 1 -- 172.21.15.97:0/3602709599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418003cc0 con 0x7f74280719c0 2023-07-06T03:45:53.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.485+0000 7f7424ff9700 1 -- 172.21.15.97:0/3602709599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741801f070 con 0x7f74280719c0 2023-07-06T03:45:53.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.485+0000 7f7424ff9700 1 -- 172.21.15.97:0/3602709599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741800b8a0 con 0x7f74280719c0 2023-07-06T03:45:53.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.485+0000 7f742dfff700 1 -- 172.21.15.97:0/3602709599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74280b2ba0 con 0x7f74280719c0 2023-07-06T03:45:53.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.485+0000 7f742dfff700 1 -- 172.21.15.97:0/3602709599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74280b3160 con 0x7f74280719c0 2023-07-06T03:45:53.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.486+0000 7f742dfff700 1 -- 172.21.15.97:0/3602709599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742804e340 con 0x7f74280719c0 2023-07-06T03:45:53.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.487+0000 7f7424ff9700 1 -- 172.21.15.97:0/3602709599 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7418021030 con 0x7f74280719c0 2023-07-06T03:45:53.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.487+0000 7f7424ff9700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 0x7f740806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.488+0000 7f7424ff9700 1 -- 172.21.15.97:0/3602709599 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f741808a8c0 con 0x7f74280719c0 2023-07-06T03:45:53.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.488+0000 7f741ffff700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 0x7f740806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.488+0000 7f741ffff700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 0x7f740806d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7428158140 tx=0x7f7410006c50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:53.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.492+0000 7f7424ff9700 1 -- 172.21.15.97:0/3602709599 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7418055a00 con 0x7f74280719c0 2023-07-06T03:45:53.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.689+0000 7f742dfff700 1 -- 172.21.15.97:0/3602709599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7428071290 con 0x7f74280719c0 2023-07-06T03:45:53.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.689+0000 7f7424ff9700 1 -- 172.21.15.97:0/3602709599 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7418059020 con 0x7f74280719c0 2023-07-06T03:45:53.693 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:45:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 -- 172.21.15.97:0/3602709599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 msgr2=0x7f740806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 0x7f740806d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7428158140 tx=0x7f7410006c50 comp rx=0 tx=0).stop 2023-07-06T03:45:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 -- 172.21.15.97:0/3602709599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280719c0 msgr2=0x7f74281499a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280719c0 0x7f74281499a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f74280b26f0 tx=0x7f7418004750 comp rx=0 tx=0).stop 2023-07-06T03:45:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 -- 172.21.15.97:0/3602709599 shutdown_connections 2023-07-06T03:45:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 0x7f740806d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f742815c590 0x7f74280b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74280719c0 0x7f74281499a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 --2- 172.21.15.97:0/3602709599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f742815c1c0 0x7f7428158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 -- 172.21.15.97:0/3602709599 >> 172.21.15.97:0/3602709599 conn(0x7f74280b1650 msgr2=0x7f74280af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 -- 172.21.15.97:0/3602709599 shutdown_connections 2023-07-06T03:45:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.692+0000 7f742dfff700 1 -- 172.21.15.97:0/3602709599 wait complete. 2023-07-06T03:45:53.712 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:45:53.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.909+0000 7f4e471ff700 1 Processor -- start 2023-07-06T03:45:53.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.909+0000 7f4e471ff700 1 -- start start 2023-07-06T03:45:53.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.910+0000 7f4e471ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4e4015c610 0x7f4e400b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.910+0000 7f4e471ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e4006b360 0x7f4e400b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e471ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e400cff90 con 0x7f4e400d2210 2023-07-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e471ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e400d0780 con 0x7f4e4015c610 2023-07-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e471ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4015c9e0 con 0x7f4e4006b360 2023-07-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e4579c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4e400d2210 0x7f4e400d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58812/0 (socket says 172.21.15.97:58812) 2023-07-06T03:45:53.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e4579c700 1 -- 172.21.15.97:0/2087052610 learned_addr learned my addr 172.21.15.97:0/2087052610 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:53.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e44f9b700 1 --2- 172.21.15.97:0/2087052610 >> v2:172.21.15.146:3300/0 conn(0x7f4e4015c610 0x7f4e400b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e3ffff700 1 --2- 172.21.15.97:0/2087052610 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e4006b360 0x7f4e400b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e3f7fe700 1 -- 172.21.15.97:0/2087052610 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392231076 0 0) 0x7f4e400cff90 con 0x7f4e400d2210 2023-07-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e3f7fe700 1 -- 172.21.15.97:0/2087052610 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e300035b0 con 0x7f4e400d2210 2023-07-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e44f9b700 1 -- 172.21.15.97:0/2087052610 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e4006b360 msgr2=0x7f4e400b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e44f9b700 1 --2- 172.21.15.97:0/2087052610 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e4006b360 0x7f4e400b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.911+0000 7f4e44f9b700 1 -- 172.21.15.97:0/2087052610 >> v1:172.21.15.97:6789/0 conn(0x7f4e400d2210 legacy=0x7f4e400d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.912+0000 7f4e44f9b700 1 -- 172.21.15.97:0/2087052610 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e400b5650 con 0x7f4e4015c610 2023-07-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.912+0000 7f4e44f9b700 1 --2- 172.21.15.97:0/2087052610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e4015c610 0x7f4e400b47b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009870 tx=0x7f4e34009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae5aefbfeefb4d89 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:53.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.912+0000 7f4e3f7fe700 1 -- 172.21.15.97:0/2087052610 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34011830 con 0x7f4e4015c610 2023-07-06T03:45:53.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.912+0000 7f4e3f7fe700 1 -- 172.21.15.97:0/2087052610 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e34011990 con 0x7f4e4015c610 2023-07-06T03:45:53.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.912+0000 7f4e3f7fe700 1 -- 172.21.15.97:0/2087052610 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34011d20 con 0x7f4e4015c610 2023-07-06T03:45:53.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.912+0000 7f4e471ff700 1 -- 172.21.15.97:0/2087052610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e4015c610 msgr2=0x7f4e400b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.912+0000 7f4e471ff700 1 --2- 172.21.15.97:0/2087052610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e4015c610 0x7f4e400b47b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009870 tx=0x7f4e34009b80 comp rx=0 tx=0).stop 2023-07-06T03:45:53.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.913+0000 7f4e471ff700 1 -- 172.21.15.97:0/2087052610 shutdown_connections 2023-07-06T03:45:53.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.913+0000 7f4e471ff700 1 --2- 172.21.15.97:0/2087052610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e4015c610 0x7f4e400b47b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.913+0000 7f4e471ff700 1 --2- 172.21.15.97:0/2087052610 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e4006b360 0x7f4e400b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.913+0000 7f4e471ff700 1 -- 172.21.15.97:0/2087052610 >> 172.21.15.97:0/2087052610 conn(0x7f4e4006a6c0 msgr2=0x7f4e4006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:53.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.913+0000 7f4e471ff700 1 -- 172.21.15.97:0/2087052610 shutdown_connections 2023-07-06T03:45:53.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.913+0000 7f4e471ff700 1 -- 172.21.15.97:0/2087052610 wait complete. 2023-07-06T03:45:53.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.914+0000 7f4e471ff700 1 Processor -- start 2023-07-06T03:45:53.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.915+0000 7f4e471ff700 1 -- start start 2023-07-06T03:45:53.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.915+0000 7f4e471ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e4006b360 0x7f4e401f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.915+0000 7f4e471ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e400d2210 0x7f4e400c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.915+0000 7f4e471ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4015c610 0x7f4e40157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.915+0000 7f4e471ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4015d710 con 0x7f4e4015c610 2023-07-06T03:45:53.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.915+0000 7f4e471ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e40158380 con 0x7f4e400d2210 2023-07-06T03:45:53.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.915+0000 7f4e471ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e401f2b30 con 0x7f4e4006b360 2023-07-06T03:45:53.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.916+0000 7f4e4579c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4015c610 0x7f4e40157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.916+0000 7f4e4579c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4015c610 0x7f4e40157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37100/0 (socket says 172.21.15.97:37100) 2023-07-06T03:45:53.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.916+0000 7f4e4579c700 1 -- 172.21.15.97:0/3280920743 learned_addr learned my addr 172.21.15.97:0/3280920743 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:53.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.916+0000 7f4e44f9b700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e4006b360 0x7f4e401f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.916+0000 7f4e3ffff700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e400d2210 0x7f4e400c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.916+0000 7f4e4579c700 1 -- 172.21.15.97:0/3280920743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e4006b360 msgr2=0x7f4e401f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.916+0000 7f4e4579c700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e4006b360 0x7f4e401f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.916+0000 7f4e4579c700 1 -- 172.21.15.97:0/3280920743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e400d2210 msgr2=0x7f4e400c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.916+0000 7f4e4579c700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e400d2210 0x7f4e400c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.916+0000 7f4e4579c700 1 -- 172.21.15.97:0/3280920743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e340094a0 con 0x7f4e4015c610 2023-07-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.916+0000 7f4e4579c700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4015c610 0x7f4e40157e40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800cb40 tx=0x7f4e3800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.917+0000 7f4e3d7fa700 1 -- 172.21.15.97:0/3280920743 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800a980 con 0x7f4e4015c610 2023-07-06T03:45:53.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.917+0000 7f4e3d7fa700 1 -- 172.21.15.97:0/3280920743 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3800aae0 con 0x7f4e4015c610 2023-07-06T03:45:53.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.917+0000 7f4e471ff700 1 -- 172.21.15.97:0/3280920743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e400d1de0 con 0x7f4e4015c610 2023-07-06T03:45:53.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.917+0000 7f4e3d7fa700 1 -- 172.21.15.97:0/3280920743 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800ae50 con 0x7f4e4015c610 2023-07-06T03:45:53.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.917+0000 7f4e471ff700 1 -- 172.21.15.97:0/3280920743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e401f2570 con 0x7f4e4015c610 2023-07-06T03:45:53.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.918+0000 7f4e471ff700 1 -- 172.21.15.97:0/3280920743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4004e340 con 0x7f4e4015c610 2023-07-06T03:45:53.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.919+0000 7f4e3d7fa700 1 -- 172.21.15.97:0/3280920743 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e3801b3b0 con 0x7f4e4015c610 2023-07-06T03:45:53.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.920+0000 7f4e3d7fa700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e2406ace0 0x7f4e2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:53.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.920+0000 7f4e3d7fa700 1 -- 172.21.15.97:0/3280920743 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e38088ff0 con 0x7f4e4015c610 2023-07-06T03:45:53.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.920+0000 7f4e44f9b700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e2406ace0 0x7f4e2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:53.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.921+0000 7f4e44f9b700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e2406ace0 0x7f4e2406d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e340090b0 tx=0x7f4e3401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:53.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:53.924+0000 7f4e3d7fa700 1 -- 172.21.15.97:0/3280920743 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4e38054140 con 0x7f4e4015c610 2023-07-06T03:45:54.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.117+0000 7f4e471ff700 1 -- 172.21.15.97:0/3280920743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4e4015b5e0 con 0x7f4e4015c610 2023-07-06T03:45:54.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.118+0000 7f4e3d7fa700 1 -- 172.21.15.97:0/3280920743 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4e38017020 con 0x7f4e4015c610 2023-07-06T03:45:54.121 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:45:54.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 -- 172.21.15.97:0/3280920743 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e2406ace0 msgr2=0x7f4e2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e2406ace0 0x7f4e2406d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e340090b0 tx=0x7f4e3401c040 comp rx=0 tx=0).stop 2023-07-06T03:45:54.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 -- 172.21.15.97:0/3280920743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4015c610 msgr2=0x7f4e40157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4015c610 0x7f4e40157e40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800cb40 tx=0x7f4e3800ce50 comp rx=0 tx=0).stop 2023-07-06T03:45:54.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 -- 172.21.15.97:0/3280920743 shutdown_connections 2023-07-06T03:45:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e2406ace0 0x7f4e2406d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4015c610 0x7f4e40157e40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e4006b360 0x7f4e401f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 --2- 172.21.15.97:0/3280920743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e400d2210 0x7f4e400c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 -- 172.21.15.97:0/3280920743 >> 172.21.15.97:0/3280920743 conn(0x7f4e4006a6c0 msgr2=0x7f4e4015aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:54.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 -- 172.21.15.97:0/3280920743 shutdown_connections 2023-07-06T03:45:54.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.120+0000 7f4e471ff700 1 -- 172.21.15.97:0/3280920743 wait complete. 2023-07-06T03:45:54.141 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:45:54.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.336+0000 7ff2659ff700 1 Processor -- start 2023-07-06T03:45:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.336+0000 7ff2659ff700 1 -- start start 2023-07-06T03:45:54.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.337+0000 7ff2659ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff26015c610 0x7ff2600b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:54.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.337+0000 7ff2659ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff26006b360 0x7ff2600b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:54.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.337+0000 7ff2659ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2600cff90 con 0x7ff2600d2210 2023-07-06T03:45:54.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.337+0000 7ff2659ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2600d0780 con 0x7ff26015c610 2023-07-06T03:45:54.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.337+0000 7ff2659ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26015c9e0 con 0x7ff26006b360 2023-07-06T03:45:54.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.337+0000 7ff25f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff2600d2210 0x7ff2600d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58826/0 (socket says 172.21.15.97:58826) 2023-07-06T03:45:54.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.337+0000 7ff25f7fe700 1 -- 172.21.15.97:0/1386198272 learned_addr learned my addr 172.21.15.97:0/1386198272 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:54.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.337+0000 7ff25effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff26015c610 0x7ff2600b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:54.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.337+0000 7ff25e7fc700 1 --2- 172.21.15.97:0/1386198272 >> v2:172.21.15.146:3300/0 conn(0x7ff26006b360 0x7ff2600b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:54.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.338+0000 7ff25effd700 1 -- 172.21.15.97:0/1386198272 >> v2:172.21.15.146:3300/0 conn(0x7ff26006b360 msgr2=0x7ff2600b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:54.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.338+0000 7ff25effd700 1 --2- 172.21.15.97:0/1386198272 >> v2:172.21.15.146:3300/0 conn(0x7ff26006b360 0x7ff2600b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.338+0000 7ff25effd700 1 -- 172.21.15.97:0/1386198272 >> v1:172.21.15.97:6789/0 conn(0x7ff2600d2210 legacy=0x7ff2600d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.338+0000 7ff25effd700 1 -- 172.21.15.97:0/1386198272 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2600b5650 con 0x7ff26015c610 2023-07-06T03:45:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.338+0000 7ff25dffb700 1 -- 172.21.15.97:0/1386198272 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437795728 0 0) 0x7ff2600cff90 con 0x7ff2600d2210 2023-07-06T03:45:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.338+0000 7ff25effd700 1 --2- 172.21.15.97:0/1386198272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff26015c610 0x7ff2600b47b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff26006a590 tx=0x7ff250009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f8bc22ec2eb185 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.338+0000 7ff25dffb700 1 -- 172.21.15.97:0/1386198272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250011830 con 0x7ff26015c610 2023-07-06T03:45:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.338+0000 7ff25dffb700 1 -- 172.21.15.97:0/1386198272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff250011990 con 0x7ff26015c610 2023-07-06T03:45:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.338+0000 7ff25dffb700 1 -- 172.21.15.97:0/1386198272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250011d20 con 0x7ff26015c610 2023-07-06T03:45:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.339+0000 7ff2659ff700 1 -- 172.21.15.97:0/1386198272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff26015c610 msgr2=0x7ff2600b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.339+0000 7ff2659ff700 1 --2- 172.21.15.97:0/1386198272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff26015c610 0x7ff2600b47b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff26006a590 tx=0x7ff250009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.339+0000 7ff2659ff700 1 -- 172.21.15.97:0/1386198272 shutdown_connections 2023-07-06T03:45:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.339+0000 7ff2659ff700 1 --2- 172.21.15.97:0/1386198272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff26015c610 0x7ff2600b47b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.339+0000 7ff2659ff700 1 --2- 172.21.15.97:0/1386198272 >> v2:172.21.15.146:3300/0 conn(0x7ff26006b360 0x7ff2600b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.339+0000 7ff2659ff700 1 -- 172.21.15.97:0/1386198272 >> 172.21.15.97:0/1386198272 conn(0x7ff26006a6c0 msgr2=0x7ff26006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.339+0000 7ff2659ff700 1 -- 172.21.15.97:0/1386198272 shutdown_connections 2023-07-06T03:45:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.339+0000 7ff2659ff700 1 -- 172.21.15.97:0/1386198272 wait complete. 2023-07-06T03:45:54.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.341+0000 7ff2659ff700 1 Processor -- start 2023-07-06T03:45:54.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.341+0000 7ff2659ff700 1 -- start start 2023-07-06T03:45:54.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff2659ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff26006b360 0x7ff2601f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:54.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff2659ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2600d2210 0x7ff2600b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:54.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff2659ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff26015c610 0x7ff2600b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:54.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff2659ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600cb820 con 0x7ff26015c610 2023-07-06T03:45:54.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff2659ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600cb210 con 0x7ff26006b360 2023-07-06T03:45:54.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff2659ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600c7cf0 con 0x7ff2600d2210 2023-07-06T03:45:54.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff25f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff26015c610 0x7ff2600b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:54.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff25f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff26015c610 0x7ff2600b7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37114/0 (socket says 172.21.15.97:37114) 2023-07-06T03:45:54.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff25f7fe700 1 -- 172.21.15.97:0/500315783 learned_addr learned my addr 172.21.15.97:0/500315783 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff25e7fc700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2600d2210 0x7ff2600b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff25effd700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff26006b360 0x7ff2601f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff25f7fe700 1 -- 172.21.15.97:0/500315783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2600d2210 msgr2=0x7ff2600b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff25f7fe700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2600d2210 0x7ff2600b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff25f7fe700 1 -- 172.21.15.97:0/500315783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff26006b360 msgr2=0x7ff2601f11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff25f7fe700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff26006b360 0x7ff2601f11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff25f7fe700 1 -- 172.21.15.97:0/500315783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2500094a0 con 0x7ff26015c610 2023-07-06T03:45:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.342+0000 7ff25effd700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff26006b360 0x7ff2601f11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.343+0000 7ff25f7fe700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff26015c610 0x7ff2600b7580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff258002230 tx=0x7ff25800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.343+0000 7ff247fff700 1 -- 172.21.15.97:0/500315783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25800ac20 con 0x7ff26015c610 2023-07-06T03:45:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.343+0000 7ff247fff700 1 -- 172.21.15.97:0/500315783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25800ada0 con 0x7ff26015c610 2023-07-06T03:45:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.343+0000 7ff247fff700 1 -- 172.21.15.97:0/500315783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258007310 con 0x7ff26015c610 2023-07-06T03:45:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.343+0000 7ff2659ff700 1 -- 172.21.15.97:0/500315783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2600cb4b0 con 0x7ff26015c610 2023-07-06T03:45:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.343+0000 7ff2659ff700 1 -- 172.21.15.97:0/500315783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26015cb20 con 0x7ff26015c610 2023-07-06T03:45:54.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.344+0000 7ff2659ff700 1 -- 172.21.15.97:0/500315783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26004e340 con 0x7ff26015c610 2023-07-06T03:45:54.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.345+0000 7ff247fff700 1 -- 172.21.15.97:0/500315783 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff258007470 con 0x7ff26015c610 2023-07-06T03:45:54.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.346+0000 7ff247fff700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff24c06ad10 0x7ff24c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:54.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.346+0000 7ff247fff700 1 -- 172.21.15.97:0/500315783 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff258088ff0 con 0x7ff26015c610 2023-07-06T03:45:54.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.346+0000 7ff25effd700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff24c06ad10 0x7ff24c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:54.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.347+0000 7ff25effd700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff24c06ad10 0x7ff24c06d1c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b5520 tx=0x7ff250010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:54.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.350+0000 7ff247fff700 1 -- 172.21.15.97:0/500315783 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff258054060 con 0x7ff26015c610 2023-07-06T03:45:54.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.543+0000 7ff2659ff700 1 -- 172.21.15.97:0/500315783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff260159e70 con 0x7ff26015c610 2023-07-06T03:45:54.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.544+0000 7ff247fff700 1 -- 172.21.15.97:0/500315783 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff258057680 con 0x7ff26015c610 2023-07-06T03:45:54.547 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:45:54.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 -- 172.21.15.97:0/500315783 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff24c06ad10 msgr2=0x7ff24c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff24c06ad10 0x7ff24c06d1c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b5520 tx=0x7ff250010000 comp rx=0 tx=0).stop 2023-07-06T03:45:54.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 -- 172.21.15.97:0/500315783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff26015c610 msgr2=0x7ff2600b7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff26015c610 0x7ff2600b7580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff258002230 tx=0x7ff25800a060 comp rx=0 tx=0).stop 2023-07-06T03:45:54.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 -- 172.21.15.97:0/500315783 shutdown_connections 2023-07-06T03:45:54.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff24c06ad10 0x7ff24c06d1c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff26015c610 0x7ff2600b7580 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff26006b360 0x7ff2601f11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 --2- 172.21.15.97:0/500315783 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2600d2210 0x7ff2600b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 -- 172.21.15.97:0/500315783 >> 172.21.15.97:0/500315783 conn(0x7ff26006a6c0 msgr2=0x7ff2600d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:54.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 -- 172.21.15.97:0/500315783 shutdown_connections 2023-07-06T03:45:54.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.546+0000 7ff2659ff700 1 -- 172.21.15.97:0/500315783 wait complete. 2023-07-06T03:45:54.566 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:45:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.762+0000 7fcc31bff700 1 Processor -- start 2023-07-06T03:45:54.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.762+0000 7fcc31bff700 1 -- start start 2023-07-06T03:45:54.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.763+0000 7fcc31bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c0d2210 0x7fcc2c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:54.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.763+0000 7fcc31bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2c0cff90 con 0x7fcc2c15c610 2023-07-06T03:45:54.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.763+0000 7fcc31bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2c0b5650 con 0x7fcc2c06b360 2023-07-06T03:45:54.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.763+0000 7fcc31bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0d0780 con 0x7fcc2c0d2210 2023-07-06T03:45:54.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.763+0000 7fcc2b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcc2c15c610 0x7fcc2c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58834/0 (socket says 172.21.15.97:58834) 2023-07-06T03:45:54.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.763+0000 7fcc2b7fe700 1 -- 172.21.15.97:0/2702682209 learned_addr learned my addr 172.21.15.97:0/2702682209 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:54.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.763+0000 7fcc2bfff700 1 --2- 172.21.15.97:0/2702682209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c0d2210 0x7fcc2c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:54.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.764+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2702682209 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355218467 0 0) 0x7fcc2c0cff90 con 0x7fcc2c15c610 2023-07-06T03:45:54.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.764+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2702682209 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc140035b0 con 0x7fcc2c15c610 2023-07-06T03:45:54.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.764+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2702682209 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395098413 0 0) 0x7fcc2c0b5650 con 0x7fcc2c06b360 2023-07-06T03:45:54.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.764+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2702682209 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc2c0cff90 con 0x7fcc2c06b360 2023-07-06T03:45:54.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.764+0000 7fcc2bfff700 1 -- 172.21.15.97:0/2702682209 >> v1:172.21.15.146:6789/0 conn(0x7fcc2c06b360 legacy=0x7fcc2c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.764+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2702682209 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1793715075 0 0) 0x7fcc140035b0 con 0x7fcc2c15c610 2023-07-06T03:45:54.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.764+0000 7fcc2bfff700 1 -- 172.21.15.97:0/2702682209 >> v1:172.21.15.97:6789/0 conn(0x7fcc2c15c610 legacy=0x7fcc2c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.764+0000 7fcc2bfff700 1 -- 172.21.15.97:0/2702682209 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c0b5400 con 0x7fcc2c0d2210 2023-07-06T03:45:54.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.764+0000 7fcc2bfff700 1 --2- 172.21.15.97:0/2702682209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c0d2210 0x7fcc2c0b4ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009dc0 tx=0x7fcc2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82703ad3228d4b55 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:54.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.764+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2702682209 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400e070 con 0x7fcc2c0d2210 2023-07-06T03:45:54.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.765+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2702682209 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc24003a20 con 0x7fcc2c0d2210 2023-07-06T03:45:54.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.765+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2702682209 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c001670 con 0x7fcc2c0d2210 2023-07-06T03:45:54.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.765+0000 7fcc31bff700 1 -- 172.21.15.97:0/2702682209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c0d2210 msgr2=0x7fcc2c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.765+0000 7fcc31bff700 1 --2- 172.21.15.97:0/2702682209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c0d2210 0x7fcc2c0b4ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009dc0 tx=0x7fcc2400b330 comp rx=0 tx=0).stop 2023-07-06T03:45:54.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.765+0000 7fcc31bff700 1 -- 172.21.15.97:0/2702682209 shutdown_connections 2023-07-06T03:45:54.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.765+0000 7fcc31bff700 1 --2- 172.21.15.97:0/2702682209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c0d2210 0x7fcc2c0b4ec0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.765+0000 7fcc31bff700 1 -- 172.21.15.97:0/2702682209 >> 172.21.15.97:0/2702682209 conn(0x7fcc2c06a6c0 msgr2=0x7fcc2c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:54.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.765+0000 7fcc31bff700 1 -- 172.21.15.97:0/2702682209 shutdown_connections 2023-07-06T03:45:54.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.765+0000 7fcc31bff700 1 -- 172.21.15.97:0/2702682209 wait complete. 2023-07-06T03:45:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.767+0000 7fcc31bff700 1 Processor -- start 2023-07-06T03:45:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.767+0000 7fcc31bff700 1 -- start start 2023-07-06T03:45:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.768+0000 7fcc31bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c06b360 0x7fcc2c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.768+0000 7fcc31bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.768+0000 7fcc31bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c15c610 0x7fcc2c0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.768+0000 7fcc31bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0cb700 con 0x7fcc2c0d2210 2023-07-06T03:45:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.768+0000 7fcc31bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0caaf0 con 0x7fcc2c15c610 2023-07-06T03:45:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.768+0000 7fcc31bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0cadd0 con 0x7fcc2c06b360 2023-07-06T03:45:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.768+0000 7fcc2affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.768+0000 7fcc2affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37130/0 (socket says 172.21.15.97:37130) 2023-07-06T03:45:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.768+0000 7fcc2affd700 1 -- 172.21.15.97:0/3889132589 learned_addr learned my addr 172.21.15.97:0/3889132589 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.768+0000 7fcc2b7fe700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c06b360 0x7fcc2c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.768+0000 7fcc2bfff700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c15c610 0x7fcc2c0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.769+0000 7fcc2affd700 1 -- 172.21.15.97:0/3889132589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c06b360 msgr2=0x7fcc2c0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.769+0000 7fcc2affd700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c06b360 0x7fcc2c0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.769+0000 7fcc2affd700 1 -- 172.21.15.97:0/3889132589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c15c610 msgr2=0x7fcc2c0c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.769+0000 7fcc2affd700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c15c610 0x7fcc2c0c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.769+0000 7fcc2affd700 1 -- 172.21.15.97:0/3889132589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc24009a70 con 0x7fcc2c0d2210 2023-07-06T03:45:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.769+0000 7fcc2affd700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15ebb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000c500 tx=0x7fcc2000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.769+0000 7fcc13fff700 1 -- 172.21.15.97:0/3889132589 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc200052d0 con 0x7fcc2c0d2210 2023-07-06T03:45:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.769+0000 7fcc13fff700 1 -- 172.21.15.97:0/3889132589 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc20005450 con 0x7fcc2c0d2210 2023-07-06T03:45:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.769+0000 7fcc31bff700 1 -- 172.21.15.97:0/3889132589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2c0cb0b0 con 0x7fcc2c0d2210 2023-07-06T03:45:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.769+0000 7fcc13fff700 1 -- 172.21.15.97:0/3889132589 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc200038c0 con 0x7fcc2c0d2210 2023-07-06T03:45:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.769+0000 7fcc31bff700 1 -- 172.21.15.97:0/3889132589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c15f0f0 con 0x7fcc2c0d2210 2023-07-06T03:45:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.770+0000 7fcc31bff700 1 -- 172.21.15.97:0/3889132589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2c0cc0d0 con 0x7fcc2c0d2210 2023-07-06T03:45:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.771+0000 7fcc13fff700 1 -- 172.21.15.97:0/3889132589 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc20003a20 con 0x7fcc2c0d2210 2023-07-06T03:45:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.772+0000 7fcc13fff700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1806ace0 0x7fcc1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.772+0000 7fcc13fff700 1 -- 172.21.15.97:0/3889132589 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc20088860 con 0x7fcc2c0d2210 2023-07-06T03:45:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.772+0000 7fcc2b7fe700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1806ace0 0x7fcc1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.773+0000 7fcc2b7fe700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1806ace0 0x7fcc1806d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000f60 tx=0x7fcc1c000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.776+0000 7fcc13fff700 1 -- 172.21.15.97:0/3889132589 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcc20053900 con 0x7fcc2c0d2210 2023-07-06T03:45:54.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.973+0000 7fcc31bff700 1 -- 172.21.15.97:0/3889132589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc2c06ba30 con 0x7fcc2c0d2210 2023-07-06T03:45:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.973+0000 7fcc13fff700 1 -- 172.21.15.97:0/3889132589 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc20056f20 con 0x7fcc2c0d2210 2023-07-06T03:45:54.976 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:45:54.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.975+0000 7fcc31bff700 1 -- 172.21.15.97:0/3889132589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1806ace0 msgr2=0x7fcc1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.975+0000 7fcc31bff700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1806ace0 0x7fcc1806d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000f60 tx=0x7fcc1c000bc0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.975+0000 7fcc31bff700 1 -- 172.21.15.97:0/3889132589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 msgr2=0x7fcc2c15ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.975+0000 7fcc31bff700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15ebb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000c500 tx=0x7fcc2000c810 comp rx=0 tx=0).stop 2023-07-06T03:45:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.975+0000 7fcc31bff700 1 -- 172.21.15.97:0/3889132589 shutdown_connections 2023-07-06T03:45:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.975+0000 7fcc31bff700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1806ace0 0x7fcc1806d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.975+0000 7fcc31bff700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c15c610 0x7fcc2c0c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.975+0000 7fcc31bff700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c06b360 0x7fcc2c0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.975+0000 7fcc31bff700 1 --2- 172.21.15.97:0/3889132589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15ebb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.975+0000 7fcc31bff700 1 -- 172.21.15.97:0/3889132589 >> 172.21.15.97:0/3889132589 conn(0x7fcc2c06a6c0 msgr2=0x7fcc2c0d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.975+0000 7fcc31bff700 1 -- 172.21.15.97:0/3889132589 shutdown_connections 2023-07-06T03:45:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:54.976+0000 7fcc31bff700 1 -- 172.21.15.97:0/3889132589 wait complete. 2023-07-06T03:45:54.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:55.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.188+0000 7f29f17ff700 1 Processor -- start 2023-07-06T03:45:55.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.188+0000 7f29f17ff700 1 -- start start 2023-07-06T03:45:55.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.189+0000 7f29f17ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f29ec15c520 0x7f29ec0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:55.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.189+0000 7f29f17ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ec0cdae0 con 0x7f29ec160d00 2023-07-06T03:45:55.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.189+0000 7f29f17ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ec0b6640 con 0x7f29ec160180 2023-07-06T03:45:55.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.189+0000 7f29f17ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec0cd8d0 con 0x7f29ec15c520 2023-07-06T03:45:55.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.189+0000 7f29eb7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f29ec160d00 0x7f29ec0b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58846/0 (socket says 172.21.15.97:58846) 2023-07-06T03:45:55.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.189+0000 7f29eb7fe700 1 -- 172.21.15.97:0/3602762044 learned_addr learned my addr 172.21.15.97:0/3602762044 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:55.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.189+0000 7f29eaffd700 1 --2- 172.21.15.97:0/3602762044 >> v2:172.21.15.146:3300/0 conn(0x7f29ec15c520 0x7f29ec0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:55.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.190+0000 7f29ea7fc700 1 -- 172.21.15.97:0/3602762044 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522119335 0 0) 0x7f29ec0cdae0 con 0x7f29ec160d00 2023-07-06T03:45:55.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.190+0000 7f29ea7fc700 1 -- 172.21.15.97:0/3602762044 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29d80035b0 con 0x7f29ec160d00 2023-07-06T03:45:55.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.190+0000 7f29ea7fc700 1 -- 172.21.15.97:0/3602762044 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55344531 0 0) 0x7f29ec0b6640 con 0x7f29ec160180 2023-07-06T03:45:55.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.190+0000 7f29ea7fc700 1 -- 172.21.15.97:0/3602762044 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29ec0cdae0 con 0x7f29ec160180 2023-07-06T03:45:55.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.190+0000 7f29eaffd700 1 -- 172.21.15.97:0/3602762044 >> v1:172.21.15.146:6789/0 conn(0x7f29ec160180 legacy=0x7f29ec1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.190+0000 7f29eaffd700 1 -- 172.21.15.97:0/3602762044 >> v1:172.21.15.97:6789/0 conn(0x7f29ec160d00 legacy=0x7f29ec0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.190+0000 7f29eaffd700 1 -- 172.21.15.97:0/3602762044 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec0b6920 con 0x7f29ec15c520 2023-07-06T03:45:55.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.190+0000 7f29eaffd700 1 --2- 172.21.15.97:0/3602762044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29ec15c520 0x7f29ec0cd390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009840 tx=0x7f29dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8abc1a61dbe2202e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:55.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.190+0000 7f29ea7fc700 1 -- 172.21.15.97:0/3602762044 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc011830 con 0x7f29ec15c520 2023-07-06T03:45:55.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.190+0000 7f29ea7fc700 1 -- 172.21.15.97:0/3602762044 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc011990 con 0x7f29ec15c520 2023-07-06T03:45:55.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.190+0000 7f29ea7fc700 1 -- 172.21.15.97:0/3602762044 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc011d20 con 0x7f29ec15c520 2023-07-06T03:45:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.191+0000 7f29f17ff700 1 -- 172.21.15.97:0/3602762044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29ec15c520 msgr2=0x7f29ec0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.191+0000 7f29f17ff700 1 --2- 172.21.15.97:0/3602762044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29ec15c520 0x7f29ec0cd390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009840 tx=0x7f29dc009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.191+0000 7f29f17ff700 1 -- 172.21.15.97:0/3602762044 shutdown_connections 2023-07-06T03:45:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.191+0000 7f29f17ff700 1 --2- 172.21.15.97:0/3602762044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29ec15c520 0x7f29ec0cd390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.191+0000 7f29f17ff700 1 -- 172.21.15.97:0/3602762044 >> 172.21.15.97:0/3602762044 conn(0x7f29ec071790 msgr2=0x7f29ec071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:55.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.191+0000 7f29f17ff700 1 -- 172.21.15.97:0/3602762044 shutdown_connections 2023-07-06T03:45:55.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.191+0000 7f29f17ff700 1 -- 172.21.15.97:0/3602762044 wait complete. 2023-07-06T03:45:55.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.193+0000 7f29f17ff700 1 Processor -- start 2023-07-06T03:45:55.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.193+0000 7f29f17ff700 1 -- start start 2023-07-06T03:45:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.194+0000 7f29f17ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29ec15c520 0x7f29ec1ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.194+0000 7f29f17ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29ec160180 0x7f29ec0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.194+0000 7f29f17ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29ec160d00 0x7f29ec1f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.194+0000 7f29f17ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec14da50 con 0x7f29ec160d00 2023-07-06T03:45:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.194+0000 7f29f17ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0cc470 con 0x7f29ec15c520 2023-07-06T03:45:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.194+0000 7f29f17ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0cc710 con 0x7f29ec160180 2023-07-06T03:45:55.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.194+0000 7f29eb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29ec160d00 0x7f29ec1f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:55.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.194+0000 7f29eb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29ec160d00 0x7f29ec1f5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37134/0 (socket says 172.21.15.97:37134) 2023-07-06T03:45:55.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.194+0000 7f29eb7fe700 1 -- 172.21.15.97:0/2203823298 learned_addr learned my addr 172.21.15.97:0/2203823298 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:55.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.194+0000 7f29eaffd700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29ec15c520 0x7f29ec1ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:55.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.194+0000 7f29e3fff700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29ec160180 0x7f29ec0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:55.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29eb7fe700 1 -- 172.21.15.97:0/2203823298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29ec160180 msgr2=0x7f29ec0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29eb7fe700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29ec160180 0x7f29ec0ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29eb7fe700 1 -- 172.21.15.97:0/2203823298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29ec15c520 msgr2=0x7f29ec1ef6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29eb7fe700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29ec15c520 0x7f29ec1ef6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29eb7fe700 1 -- 172.21.15.97:0/2203823298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc0094a0 con 0x7f29ec160d00 2023-07-06T03:45:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29e3fff700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29ec160180 0x7f29ec0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29eb7fe700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29ec160d00 0x7f29ec1f5650 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f29e400cba0 tx=0x7f29e400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29e37fe700 1 -- 172.21.15.97:0/2203823298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e400aa70 con 0x7f29ec160d00 2023-07-06T03:45:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29e37fe700 1 -- 172.21.15.97:0/2203823298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e400abd0 con 0x7f29ec160d00 2023-07-06T03:45:55.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29f17ff700 1 -- 172.21.15.97:0/2203823298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec0cc9f0 con 0x7f29ec160d00 2023-07-06T03:45:55.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29e37fe700 1 -- 172.21.15.97:0/2203823298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4012400 con 0x7f29ec160d00 2023-07-06T03:45:55.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.195+0000 7f29f17ff700 1 -- 172.21.15.97:0/2203823298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec0d1d90 con 0x7f29ec160d00 2023-07-06T03:45:55.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.196+0000 7f29f17ff700 1 -- 172.21.15.97:0/2203823298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec0d1fb0 con 0x7f29ec160d00 2023-07-06T03:45:55.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.197+0000 7f29e37fe700 1 -- 172.21.15.97:0/2203823298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f29e401b3b0 con 0x7f29ec160d00 2023-07-06T03:45:55.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.198+0000 7f29e37fe700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29cc06ace0 0x7f29cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:55.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.198+0000 7f29e37fe700 1 -- 172.21.15.97:0/2203823298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29e4089140 con 0x7f29ec160d00 2023-07-06T03:45:55.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.198+0000 7f29eaffd700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29cc06ace0 0x7f29cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:55.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.199+0000 7f29eaffd700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29cc06ace0 0x7f29cc06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009fd0 tx=0x7f29dc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:55.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.202+0000 7f29e37fe700 1 -- 172.21.15.97:0/2203823298 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f29e4054260 con 0x7f29ec160d00 2023-07-06T03:45:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.397+0000 7f29f17ff700 1 -- 172.21.15.97:0/2203823298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29ec04e340 con 0x7f29ec160d00 2023-07-06T03:45:55.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.398+0000 7f29e37fe700 1 -- 172.21.15.97:0/2203823298 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f29e4057880 con 0x7f29ec160d00 2023-07-06T03:45:55.402 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:55.402 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:55.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 -- 172.21.15.97:0/2203823298 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29cc06ace0 msgr2=0x7f29cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29cc06ace0 0x7f29cc06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009fd0 tx=0x7f29dc01c040 comp rx=0 tx=0).stop 2023-07-06T03:45:55.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 -- 172.21.15.97:0/2203823298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29ec160d00 msgr2=0x7f29ec1f5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29ec160d00 0x7f29ec1f5650 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f29e400cba0 tx=0x7f29e400ceb0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 -- 172.21.15.97:0/2203823298 shutdown_connections 2023-07-06T03:45:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29cc06ace0 0x7f29cc06d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29ec160d00 0x7f29ec1f5650 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29ec15c520 0x7f29ec1ef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 --2- 172.21.15.97:0/2203823298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29ec160180 0x7f29ec0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 -- 172.21.15.97:0/2203823298 >> 172.21.15.97:0/2203823298 conn(0x7f29ec071790 msgr2=0x7f29ec0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:55.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 -- 172.21.15.97:0/2203823298 shutdown_connections 2023-07-06T03:45:55.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.401+0000 7f29f17ff700 1 -- 172.21.15.97:0/2203823298 wait complete. 2023-07-06T03:45:55.421 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:45:55.422 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:45:55.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.617+0000 7fca467ff700 1 Processor -- start 2023-07-06T03:45:55.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.617+0000 7fca467ff700 1 -- start start 2023-07-06T03:45:55.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.618+0000 7fca467ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fca4015c610 0x7fca400b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:55.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.618+0000 7fca467ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca400d2210 0x7fca400d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:55.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.618+0000 7fca467ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca400cff90 con 0x7fca4006b360 2023-07-06T03:45:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.618+0000 7fca467ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca400d0780 con 0x7fca4015c610 2023-07-06T03:45:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.618+0000 7fca467ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4015c9e0 con 0x7fca400d2210 2023-07-06T03:45:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.619+0000 7fca3f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fca4006b360 0x7fca400b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58852/0 (socket says 172.21.15.97:58852) 2023-07-06T03:45:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.619+0000 7fca3f7fe700 1 -- 172.21.15.97:0/327331402 learned_addr learned my addr 172.21.15.97:0/327331402 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:55.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.619+0000 7fca3ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fca4015c610 0x7fca400b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.619+0000 7fca44d9c700 1 --2- 172.21.15.97:0/327331402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca400d2210 0x7fca400d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.619+0000 7fca3ffff700 1 -- 172.21.15.97:0/327331402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca400d2210 msgr2=0x7fca400d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.619+0000 7fca3ffff700 1 --2- 172.21.15.97:0/327331402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca400d2210 0x7fca400d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.619+0000 7fca3ffff700 1 -- 172.21.15.97:0/327331402 >> v1:172.21.15.97:6789/0 conn(0x7fca4006b360 legacy=0x7fca400b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.619+0000 7fca3ffff700 1 -- 172.21.15.97:0/327331402 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca400b5650 con 0x7fca4015c610 2023-07-06T03:45:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.619+0000 7fca3effd700 1 -- 172.21.15.97:0/327331402 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601894978 0 0) 0x7fca400cff90 con 0x7fca4006b360 2023-07-06T03:45:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.619+0000 7fca3ffff700 1 --2- 172.21.15.97:0/327331402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4015c610 0x7fca400b47b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fca30009840 tx=0x7fca30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2c1929c9a33886 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.620+0000 7fca3effd700 1 -- 172.21.15.97:0/327331402 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30011830 con 0x7fca4015c610 2023-07-06T03:45:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.620+0000 7fca3effd700 1 -- 172.21.15.97:0/327331402 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca30011990 con 0x7fca4015c610 2023-07-06T03:45:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.620+0000 7fca3effd700 1 -- 172.21.15.97:0/327331402 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30011d20 con 0x7fca4015c610 2023-07-06T03:45:55.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.620+0000 7fca467ff700 1 -- 172.21.15.97:0/327331402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4015c610 msgr2=0x7fca400b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.620+0000 7fca467ff700 1 --2- 172.21.15.97:0/327331402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4015c610 0x7fca400b47b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fca30009840 tx=0x7fca30009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:55.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.620+0000 7fca467ff700 1 -- 172.21.15.97:0/327331402 shutdown_connections 2023-07-06T03:45:55.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.620+0000 7fca467ff700 1 --2- 172.21.15.97:0/327331402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4015c610 0x7fca400b47b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.620+0000 7fca467ff700 1 --2- 172.21.15.97:0/327331402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca400d2210 0x7fca400d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.620+0000 7fca467ff700 1 -- 172.21.15.97:0/327331402 >> 172.21.15.97:0/327331402 conn(0x7fca4006a6c0 msgr2=0x7fca4006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:55.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.620+0000 7fca467ff700 1 -- 172.21.15.97:0/327331402 shutdown_connections 2023-07-06T03:45:55.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.620+0000 7fca467ff700 1 -- 172.21.15.97:0/327331402 wait complete. 2023-07-06T03:45:55.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.622+0000 7fca467ff700 1 Processor -- start 2023-07-06T03:45:55.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.622+0000 7fca467ff700 1 -- start start 2023-07-06T03:45:55.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.623+0000 7fca467ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4006b360 0x7fca40157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:55.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.623+0000 7fca467ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca400d2210 0x7fca4015ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:55.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.623+0000 7fca467ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca4015c610 0x7fca400cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:55.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.623+0000 7fca467ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4015cb20 con 0x7fca4006b360 2023-07-06T03:45:55.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.623+0000 7fca467ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4015cd30 con 0x7fca400d2210 2023-07-06T03:45:55.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.623+0000 7fca467ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4015d010 con 0x7fca4015c610 2023-07-06T03:45:55.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.623+0000 7fca3ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4006b360 0x7fca40157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:55.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.623+0000 7fca3ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4006b360 0x7fca40157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37150/0 (socket says 172.21.15.97:37150) 2023-07-06T03:45:55.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.623+0000 7fca3ffff700 1 -- 172.21.15.97:0/1443783430 learned_addr learned my addr 172.21.15.97:0/1443783430 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:55.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca44d9c700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca4015c610 0x7fca400cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:55.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca3f7fe700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca400d2210 0x7fca4015ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:55.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca3ffff700 1 -- 172.21.15.97:0/1443783430 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca4015c610 msgr2=0x7fca400cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca3ffff700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca4015c610 0x7fca400cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca3ffff700 1 -- 172.21.15.97:0/1443783430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca400d2210 msgr2=0x7fca4015ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca3ffff700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca400d2210 0x7fca4015ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca3ffff700 1 -- 172.21.15.97:0/1443783430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300094a0 con 0x7fca4006b360 2023-07-06T03:45:55.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca3f7fe700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca400d2210 0x7fca4015ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:55.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca3ffff700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4006b360 0x7fca40157030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca30009b90 tx=0x7fca30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:55.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca3cff9700 1 -- 172.21.15.97:0/1443783430 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300038a0 con 0x7fca4006b360 2023-07-06T03:45:55.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca3cff9700 1 -- 172.21.15.97:0/1443783430 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3001e070 con 0x7fca4006b360 2023-07-06T03:45:55.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca467ff700 1 -- 172.21.15.97:0/1443783430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4015d2f0 con 0x7fca4006b360 2023-07-06T03:45:55.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca3cff9700 1 -- 172.21.15.97:0/1443783430 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30003c40 con 0x7fca4006b360 2023-07-06T03:45:55.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.624+0000 7fca467ff700 1 -- 172.21.15.97:0/1443783430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4015d8b0 con 0x7fca4006b360 2023-07-06T03:45:55.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.625+0000 7fca467ff700 1 -- 172.21.15.97:0/1443783430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca400ce600 con 0x7fca4006b360 2023-07-06T03:45:55.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.626+0000 7fca3cff9700 1 -- 172.21.15.97:0/1443783430 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fca3002d450 con 0x7fca4006b360 2023-07-06T03:45:55.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.627+0000 7fca3cff9700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca2c06ace0 0x7fca2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:55.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.627+0000 7fca3cff9700 1 -- 172.21.15.97:0/1443783430 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fca3008a530 con 0x7fca4006b360 2023-07-06T03:45:55.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.627+0000 7fca3f7fe700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca2c06ace0 0x7fca2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:55.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.628+0000 7fca3f7fe700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca2c06ace0 0x7fca2c06d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fca40158430 tx=0x7fca34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:55.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.631+0000 7fca3cff9700 1 -- 172.21.15.97:0/1443783430 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fca300555d0 con 0x7fca4006b360 2023-07-06T03:45:55.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.826+0000 7fca467ff700 1 -- 172.21.15.97:0/1443783430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca4004e340 con 0x7fca4006b360 2023-07-06T03:45:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.826+0000 7fca3cff9700 1 -- 172.21.15.97:0/1443783430 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca30058bf0 con 0x7fca4006b360 2023-07-06T03:45:55.829 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:45:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 -- 172.21.15.97:0/1443783430 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca2c06ace0 msgr2=0x7fca2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca2c06ace0 0x7fca2c06d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fca40158430 tx=0x7fca34009040 comp rx=0 tx=0).stop 2023-07-06T03:45:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 -- 172.21.15.97:0/1443783430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4006b360 msgr2=0x7fca40157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:55.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4006b360 0x7fca40157030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca30009b90 tx=0x7fca30004750 comp rx=0 tx=0).stop 2023-07-06T03:45:55.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 -- 172.21.15.97:0/1443783430 shutdown_connections 2023-07-06T03:45:55.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca2c06ace0 0x7fca2c06d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca4015c610 0x7fca400cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4006b360 0x7fca40157030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 --2- 172.21.15.97:0/1443783430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca400d2210 0x7fca4015ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:55.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 -- 172.21.15.97:0/1443783430 >> 172.21.15.97:0/1443783430 conn(0x7fca4006a6c0 msgr2=0x7fca400b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:55.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 -- 172.21.15.97:0/1443783430 shutdown_connections 2023-07-06T03:45:55.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:55.828+0000 7fca467ff700 1 -- 172.21.15.97:0/1443783430 wait complete. 2023-07-06T03:45:55.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:45:56.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.045+0000 7f1922bff700 1 Processor -- start 2023-07-06T03:45:56.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.045+0000 7f1922bff700 1 -- start start 2023-07-06T03:45:56.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.046+0000 7f1922bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f191c15c610 0x7f191c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.046+0000 7f1922bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f191c06b360 0x7f191c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.046+0000 7f1922bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191c0cff90 con 0x7f191c0d2210 2023-07-06T03:45:56.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.046+0000 7f1922bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191c0d0780 con 0x7f191c15c610 2023-07-06T03:45:56.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.046+0000 7f1922bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c15c9e0 con 0x7f191c06b360 2023-07-06T03:45:56.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.047+0000 7f192099b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f191c15c610 0x7f191c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.047+0000 7f192099b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f191c15c610 0x7f191c0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37152/0 (socket says 172.21.15.97:37152) 2023-07-06T03:45:56.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.047+0000 7f192099b700 1 -- 172.21.15.97:0/1855079870 learned_addr learned my addr 172.21.15.97:0/1855079870 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.047+0000 7f191bfff700 1 --2- 172.21.15.97:0/1855079870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f191c06b360 0x7f191c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.047+0000 7f192099b700 1 -- 172.21.15.97:0/1855079870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f191c06b360 msgr2=0x7f191c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.047+0000 7f192099b700 1 --2- 172.21.15.97:0/1855079870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f191c06b360 0x7f191c0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.047+0000 7f192099b700 1 -- 172.21.15.97:0/1855079870 >> v1:172.21.15.146:6789/0 conn(0x7f191c0d2210 legacy=0x7f191c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.047+0000 7f192099b700 1 -- 172.21.15.97:0/1855079870 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c0b5650 con 0x7f191c15c610 2023-07-06T03:45:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.048+0000 7f192099b700 1 --2- 172.21.15.97:0/1855079870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c15c610 0x7f191c0b47b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1910009920 tx=0x7f1910009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152ea96c802fd63e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.048+0000 7f191b7fe700 1 -- 172.21.15.97:0/1855079870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910011830 con 0x7f191c15c610 2023-07-06T03:45:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.048+0000 7f191b7fe700 1 -- 172.21.15.97:0/1855079870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1910011990 con 0x7f191c15c610 2023-07-06T03:45:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.048+0000 7f191b7fe700 1 -- 172.21.15.97:0/1855079870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910011d20 con 0x7f191c15c610 2023-07-06T03:45:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.048+0000 7f1922bff700 1 -- 172.21.15.97:0/1855079870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c15c610 msgr2=0x7f191c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.048+0000 7f1922bff700 1 --2- 172.21.15.97:0/1855079870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c15c610 0x7f191c0b47b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1910009920 tx=0x7f1910009c30 comp rx=0 tx=0).stop 2023-07-06T03:45:56.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.048+0000 7f1922bff700 1 -- 172.21.15.97:0/1855079870 shutdown_connections 2023-07-06T03:45:56.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.048+0000 7f1922bff700 1 --2- 172.21.15.97:0/1855079870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c15c610 0x7f191c0b47b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.048+0000 7f1922bff700 1 --2- 172.21.15.97:0/1855079870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f191c06b360 0x7f191c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.048+0000 7f1922bff700 1 -- 172.21.15.97:0/1855079870 >> 172.21.15.97:0/1855079870 conn(0x7f191c06a6c0 msgr2=0x7f191c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:56.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.049+0000 7f1922bff700 1 -- 172.21.15.97:0/1855079870 shutdown_connections 2023-07-06T03:45:56.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.049+0000 7f1922bff700 1 -- 172.21.15.97:0/1855079870 wait complete. 2023-07-06T03:45:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.050+0000 7f1922bff700 1 Processor -- start 2023-07-06T03:45:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.050+0000 7f1922bff700 1 -- start start 2023-07-06T03:45:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.051+0000 7f1922bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c06b360 0x7f191c1f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.051+0000 7f1922bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f191c0d2210 0x7f191c0c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.051+0000 7f1922bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f191c15c610 0x7f191c0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.051+0000 7f1922bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0cb860 con 0x7f191c06b360 2023-07-06T03:45:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.051+0000 7f1922bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0c6d20 con 0x7f191c0d2210 2023-07-06T03:45:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.051+0000 7f1922bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0cbab0 con 0x7f191c15c610 2023-07-06T03:45:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f192099b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c06b360 0x7f191c1f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f192099b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c06b360 0x7f191c1f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37160/0 (socket says 172.21.15.97:37160) 2023-07-06T03:45:56.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f191bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f191c0d2210 0x7f191c0c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f192119c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f191c15c610 0x7f191c0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f192099b700 1 -- 172.21.15.97:0/3344086907 learned_addr learned my addr 172.21.15.97:0/3344086907 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:56.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f192099b700 1 -- 172.21.15.97:0/3344086907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f191c15c610 msgr2=0x7f191c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f192099b700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f191c15c610 0x7f191c0c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f192099b700 1 -- 172.21.15.97:0/3344086907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f191c0d2210 msgr2=0x7f191c0c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f192099b700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f191c0d2210 0x7f191c0c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f192099b700 1 -- 172.21.15.97:0/3344086907 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1910009550 con 0x7f191c06b360 2023-07-06T03:45:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f192099b700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c06b360 0x7f191c1f1db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1910009870 tx=0x7f1910004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.052+0000 7f19197fa700 1 -- 172.21.15.97:0/3344086907 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910004300 con 0x7f191c06b360 2023-07-06T03:45:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.053+0000 7f19197fa700 1 -- 172.21.15.97:0/3344086907 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191001f070 con 0x7f191c06b360 2023-07-06T03:45:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.053+0000 7f19197fa700 1 -- 172.21.15.97:0/3344086907 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100036a0 con 0x7f191c06b360 2023-07-06T03:45:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.053+0000 7f1922bff700 1 -- 172.21.15.97:0/3344086907 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191c0cbd90 con 0x7f191c06b360 2023-07-06T03:45:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.053+0000 7f1922bff700 1 -- 172.21.15.97:0/3344086907 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191c158650 con 0x7f191c06b360 2023-07-06T03:45:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.054+0000 7f1922bff700 1 -- 172.21.15.97:0/3344086907 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191c04e340 con 0x7f191c06b360 2023-07-06T03:45:56.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.055+0000 7f19197fa700 1 -- 172.21.15.97:0/3344086907 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1910021030 con 0x7f191c06b360 2023-07-06T03:45:56.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.056+0000 7f19197fa700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190006ace0 0x7f190006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.056+0000 7f19197fa700 1 -- 172.21.15.97:0/3344086907 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f191008a850 con 0x7f191c06b360 2023-07-06T03:45:56.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.056+0000 7f191bfff700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190006ace0 0x7f190006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.057+0000 7f191bfff700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190006ace0 0x7f190006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f191c0c64b0 tx=0x7f1908009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:56.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.060+0000 7f19197fa700 1 -- 172.21.15.97:0/3344086907 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1910055a10 con 0x7f191c06b360 2023-07-06T03:45:56.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.252+0000 7f1922bff700 1 -- 172.21.15.97:0/3344086907 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f191c0cdfe0 con 0x7f191c06b360 2023-07-06T03:45:56.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.252+0000 7f19197fa700 1 -- 172.21.15.97:0/3344086907 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1910059030 con 0x7f191c06b360 2023-07-06T03:45:56.255 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:45:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 -- 172.21.15.97:0/3344086907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190006ace0 msgr2=0x7f190006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190006ace0 0x7f190006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f191c0c64b0 tx=0x7f1908009210 comp rx=0 tx=0).stop 2023-07-06T03:45:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 -- 172.21.15.97:0/3344086907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c06b360 msgr2=0x7f191c1f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c06b360 0x7f191c1f1db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1910009870 tx=0x7f1910004750 comp rx=0 tx=0).stop 2023-07-06T03:45:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 -- 172.21.15.97:0/3344086907 shutdown_connections 2023-07-06T03:45:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190006ace0 0x7f190006d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f191c15c610 0x7f191c0c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f191c06b360 0x7f191c1f1db0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 --2- 172.21.15.97:0/3344086907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f191c0d2210 0x7f191c0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 -- 172.21.15.97:0/3344086907 >> 172.21.15.97:0/3344086907 conn(0x7f191c06a6c0 msgr2=0x7f191c0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 -- 172.21.15.97:0/3344086907 shutdown_connections 2023-07-06T03:45:56.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.254+0000 7f1922bff700 1 -- 172.21.15.97:0/3344086907 wait complete. 2023-07-06T03:45:56.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:45:56.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.469+0000 7f71673ff700 1 Processor -- start 2023-07-06T03:45:56.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.469+0000 7f71673ff700 1 -- start start 2023-07-06T03:45:56.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.470+0000 7f71673ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f716006b360 0x7f71600b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.470+0000 7f71673ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d2210 0x7f71600d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.470+0000 7f71673ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71600cff90 con 0x7f716015c610 2023-07-06T03:45:56.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.470+0000 7f71673ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71600d0780 con 0x7f716006b360 2023-07-06T03:45:56.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.470+0000 7f71673ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716015c9e0 con 0x7f71600d2210 2023-07-06T03:45:56.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.470+0000 7f716499a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f716006b360 0x7f71600b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.470+0000 7f716599c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d2210 0x7f71600d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.470+0000 7f716499a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f716006b360 0x7f71600b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:41850/0 (socket says 172.21.15.97:41850) 2023-07-06T03:45:56.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.470+0000 7f716499a700 1 -- 172.21.15.97:0/3991329529 learned_addr learned my addr 172.21.15.97:0/3991329529 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:56.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.470+0000 7f716519b700 1 --1- 172.21.15.97:0/3991329529 >> v1:172.21.15.146:6789/0 conn(0x7f716015c610 0x7f71600b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:59416/0 (socket says 172.21.15.97:59416) 2023-07-06T03:45:56.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.471+0000 7f716499a700 1 -- 172.21.15.97:0/3991329529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d2210 msgr2=0x7f71600d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:56.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.471+0000 7f716499a700 1 --2- 172.21.15.97:0/3991329529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d2210 0x7f71600d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.471+0000 7f716499a700 1 -- 172.21.15.97:0/3991329529 >> v1:172.21.15.146:6789/0 conn(0x7f716015c610 legacy=0x7f71600b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.471+0000 7f716499a700 1 -- 172.21.15.97:0/3991329529 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71600b5650 con 0x7f716006b360 2023-07-06T03:45:56.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.471+0000 7f716499a700 1 --2- 172.21.15.97:0/3991329529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716006b360 0x7f71600b4ec0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71500098f0 tx=0x7f7150009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f920cd3dcc85df9a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:56.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.471+0000 7f7157fff700 1 -- 172.21.15.97:0/3991329529 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150011830 con 0x7f716006b360 2023-07-06T03:45:56.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.471+0000 7f7157fff700 1 -- 172.21.15.97:0/3991329529 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7150011990 con 0x7f716006b360 2023-07-06T03:45:56.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.472+0000 7f7157fff700 1 -- 172.21.15.97:0/3991329529 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150011d20 con 0x7f716006b360 2023-07-06T03:45:56.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.472+0000 7f71673ff700 1 -- 172.21.15.97:0/3991329529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716006b360 msgr2=0x7f71600b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.472+0000 7f71673ff700 1 --2- 172.21.15.97:0/3991329529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716006b360 0x7f71600b4ec0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71500098f0 tx=0x7f7150009c00 comp rx=0 tx=0).stop 2023-07-06T03:45:56.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.472+0000 7f71673ff700 1 -- 172.21.15.97:0/3991329529 shutdown_connections 2023-07-06T03:45:56.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.472+0000 7f71673ff700 1 --2- 172.21.15.97:0/3991329529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71600d2210 0x7f71600d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.472+0000 7f71673ff700 1 --2- 172.21.15.97:0/3991329529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716006b360 0x7f71600b4ec0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.472+0000 7f71673ff700 1 -- 172.21.15.97:0/3991329529 >> 172.21.15.97:0/3991329529 conn(0x7f716006a6c0 msgr2=0x7f716006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:56.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.472+0000 7f71673ff700 1 -- 172.21.15.97:0/3991329529 shutdown_connections 2023-07-06T03:45:56.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.472+0000 7f71673ff700 1 -- 172.21.15.97:0/3991329529 wait complete. 2023-07-06T03:45:56.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.474+0000 7f71673ff700 1 Processor -- start 2023-07-06T03:45:56.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.474+0000 7f71673ff700 1 -- start start 2023-07-06T03:45:56.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f71673ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f716006b360 0x7f71601575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f71673ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71600d2210 0x7f71600c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f71673ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716015c610 0x7f71600c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f71673ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71600c79e0 con 0x7f71600d2210 2023-07-06T03:45:56.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f71673ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71600c6d60 con 0x7f716015c610 2023-07-06T03:45:56.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f71673ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71600c7040 con 0x7f716006b360 2023-07-06T03:45:56.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f716499a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71600d2210 0x7f71600c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f716499a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71600d2210 0x7f71600c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37170/0 (socket says 172.21.15.97:37170) 2023-07-06T03:45:56.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f716499a700 1 -- 172.21.15.97:0/1365532472 learned_addr learned my addr 172.21.15.97:0/1365532472 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:56.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f716519b700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f716006b360 0x7f71601575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f716599c700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716015c610 0x7f71600c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.475+0000 7f716499a700 1 -- 172.21.15.97:0/1365532472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f716006b360 msgr2=0x7f71601575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.476+0000 7f716499a700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f716006b360 0x7f71601575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.476+0000 7f716499a700 1 -- 172.21.15.97:0/1365532472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716015c610 msgr2=0x7f71600c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.476+0000 7f716499a700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716015c610 0x7f71600c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.476+0000 7f716499a700 1 -- 172.21.15.97:0/1365532472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7150009550 con 0x7f71600d2210 2023-07-06T03:45:56.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.476+0000 7f716499a700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71600d2210 0x7f71600c85b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7150000c00 tx=0x7f7150004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:56.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.476+0000 7f7155ffb700 1 -- 172.21.15.97:0/1365532472 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150004400 con 0x7f71600d2210 2023-07-06T03:45:56.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.476+0000 7f7155ffb700 1 -- 172.21.15.97:0/1365532472 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715001f070 con 0x7f71600d2210 2023-07-06T03:45:56.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.476+0000 7f7155ffb700 1 -- 172.21.15.97:0/1365532472 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71500038e0 con 0x7f71600d2210 2023-07-06T03:45:56.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.476+0000 7f71673ff700 1 -- 172.21.15.97:0/1365532472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71600c7320 con 0x7f71600d2210 2023-07-06T03:45:56.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.476+0000 7f71673ff700 1 -- 172.21.15.97:0/1365532472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7160157ae0 con 0x7f71600d2210 2023-07-06T03:45:56.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.477+0000 7f71673ff700 1 -- 172.21.15.97:0/1365532472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7160066fe0 con 0x7f71600d2210 2023-07-06T03:45:56.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.478+0000 7f7155ffb700 1 -- 172.21.15.97:0/1365532472 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7150021030 con 0x7f71600d2210 2023-07-06T03:45:56.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.479+0000 7f7155ffb700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f714406ace0 0x7f714406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.479+0000 7f7155ffb700 1 -- 172.21.15.97:0/1365532472 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f715008a900 con 0x7f71600d2210 2023-07-06T03:45:56.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.479+0000 7f716519b700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f714406ace0 0x7f714406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.480+0000 7f716519b700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f714406ace0 0x7f714406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f715c000f60 tx=0x7f715c007ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:56.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.483+0000 7f7155ffb700 1 -- 172.21.15.97:0/1365532472 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7150055920 con 0x7f71600d2210 2023-07-06T03:45:56.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.680+0000 7f71673ff700 1 -- 172.21.15.97:0/1365532472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f716004e340 con 0x7f71600d2210 2023-07-06T03:45:56.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.680+0000 7f7155ffb700 1 -- 172.21.15.97:0/1365532472 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7150058f40 con 0x7f71600d2210 2023-07-06T03:45:56.683 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:45:56.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.682+0000 7f71673ff700 1 -- 172.21.15.97:0/1365532472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f714406ace0 msgr2=0x7f714406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.682+0000 7f71673ff700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f714406ace0 0x7f714406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f715c000f60 tx=0x7f715c007ed0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.682+0000 7f71673ff700 1 -- 172.21.15.97:0/1365532472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71600d2210 msgr2=0x7f71600c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.682+0000 7f71673ff700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71600d2210 0x7f71600c85b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7150000c00 tx=0x7f7150004750 comp rx=0 tx=0).stop 2023-07-06T03:45:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.683+0000 7f71673ff700 1 -- 172.21.15.97:0/1365532472 shutdown_connections 2023-07-06T03:45:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.683+0000 7f71673ff700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f714406ace0 0x7f714406d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.683+0000 7f71673ff700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f716015c610 0x7f71600c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.683+0000 7f71673ff700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f716006b360 0x7f71601575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.683+0000 7f71673ff700 1 --2- 172.21.15.97:0/1365532472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71600d2210 0x7f71600c85b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.683+0000 7f71673ff700 1 -- 172.21.15.97:0/1365532472 >> 172.21.15.97:0/1365532472 conn(0x7f716006a6c0 msgr2=0x7f71600d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:56.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.683+0000 7f71673ff700 1 -- 172.21.15.97:0/1365532472 shutdown_connections 2023-07-06T03:45:56.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.683+0000 7f71673ff700 1 -- 172.21.15.97:0/1365532472 wait complete. 2023-07-06T03:45:56.703 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:45:56.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.899+0000 7ff7badff700 1 Processor -- start 2023-07-06T03:45:56.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.899+0000 7ff7badff700 1 -- start start 2023-07-06T03:45:56.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7badff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7b40cbaa0 0x7ff7b415a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7badff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff7b415eb90 0x7ff7b415ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7badff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b415b6e0 con 0x7ff7b4154890 2023-07-06T03:45:56.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7badff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b415e0b0 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7badff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b415e390 con 0x7ff7b415eb90 2023-07-06T03:45:56.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7b3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7b40cbaa0 0x7ff7b415a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7b3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7b40cbaa0 0x7ff7b415a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37174/0 (socket says 172.21.15.97:37174) 2023-07-06T03:45:56.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7b3fff700 1 -- 172.21.15.97:0/1866899680 learned_addr learned my addr 172.21.15.97:0/1866899680 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:56.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7b939c700 1 --2- 172.21.15.97:0/1866899680 >> v2:172.21.15.146:3300/0 conn(0x7ff7b415eb90 0x7ff7b415ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7b3fff700 1 -- 172.21.15.97:0/1866899680 >> v2:172.21.15.146:3300/0 conn(0x7ff7b415eb90 msgr2=0x7ff7b415ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:56.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7b3fff700 1 --2- 172.21.15.97:0/1866899680 >> v2:172.21.15.146:3300/0 conn(0x7ff7b415eb90 0x7ff7b415ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7b3fff700 1 -- 172.21.15.97:0/1866899680 >> v1:172.21.15.146:6789/0 conn(0x7ff7b4154890 legacy=0x7ff7b40cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7b3fff700 1 -- 172.21.15.97:0/1866899680 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b415e670 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.901+0000 7ff7b3fff700 1 --2- 172.21.15.97:0/1866899680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7b40cbaa0 0x7ff7b415a940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4158500 tx=0x7ff7a0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1838b17005a16a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:56.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.902+0000 7ff7b37fe700 1 -- 172.21.15.97:0/1866899680 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000b870 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.902+0000 7ff7b37fe700 1 -- 172.21.15.97:0/1866899680 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a000b9d0 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.902+0000 7ff7b37fe700 1 -- 172.21.15.97:0/1866899680 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000bd60 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.902+0000 7ff7badff700 1 -- 172.21.15.97:0/1866899680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7b40cbaa0 msgr2=0x7ff7b415a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.902+0000 7ff7badff700 1 --2- 172.21.15.97:0/1866899680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7b40cbaa0 0x7ff7b415a940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4158500 tx=0x7ff7a0009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.902+0000 7ff7badff700 1 -- 172.21.15.97:0/1866899680 shutdown_connections 2023-07-06T03:45:56.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.902+0000 7ff7badff700 1 --2- 172.21.15.97:0/1866899680 >> v2:172.21.15.146:3300/0 conn(0x7ff7b415eb90 0x7ff7b415ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.902+0000 7ff7badff700 1 --2- 172.21.15.97:0/1866899680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7b40cbaa0 0x7ff7b415a940 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.902+0000 7ff7badff700 1 -- 172.21.15.97:0/1866899680 >> 172.21.15.97:0/1866899680 conn(0x7ff7b406a060 msgr2=0x7ff7b406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:56.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.902+0000 7ff7badff700 1 -- 172.21.15.97:0/1866899680 shutdown_connections 2023-07-06T03:45:56.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.902+0000 7ff7badff700 1 -- 172.21.15.97:0/1866899680 wait complete. 2023-07-06T03:45:56.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.904+0000 7ff7badff700 1 Processor -- start 2023-07-06T03:45:56.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.904+0000 7ff7badff700 1 -- start start 2023-07-06T03:45:56.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.905+0000 7ff7badff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7b40cbaa0 0x7ff7b41f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.905+0000 7ff7badff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7b4154890 0x7ff7b41ef360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.905+0000 7ff7badff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b415eb90 0x7ff7b41ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.905+0000 7ff7badff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b406f9e0 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.905+0000 7ff7badff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4070280 con 0x7ff7b415eb90 2023-07-06T03:45:56.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.905+0000 7ff7badff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b41f42c0 con 0x7ff7b4154890 2023-07-06T03:45:56.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.905+0000 7ff7b8b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7b40cbaa0 0x7ff7b41f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.905+0000 7ff7b8b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7b40cbaa0 0x7ff7b41f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37186/0 (socket says 172.21.15.97:37186) 2023-07-06T03:45:56.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.905+0000 7ff7b8b9b700 1 -- 172.21.15.97:0/3675612346 learned_addr learned my addr 172.21.15.97:0/3675612346 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:56.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.905+0000 7ff7b939c700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b415eb90 0x7ff7b41ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b3fff700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7b4154890 0x7ff7b41ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b8b9b700 1 -- 172.21.15.97:0/3675612346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7b4154890 msgr2=0x7ff7b41ef360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b8b9b700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7b4154890 0x7ff7b41ef360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b8b9b700 1 -- 172.21.15.97:0/3675612346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b415eb90 msgr2=0x7ff7b41ef8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:56.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b8b9b700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b415eb90 0x7ff7b41ef8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:56.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b8b9b700 1 -- 172.21.15.97:0/3675612346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a0009550 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b3fff700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7b4154890 0x7ff7b41ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:56.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b939c700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b415eb90 0x7ff7b41ef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:56.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b8b9b700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7b40cbaa0 0x7ff7b41f3d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800c510 tx=0x7ff7a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:56.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b17fa700 1 -- 172.21.15.97:0/3675612346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80052d0 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b17fa700 1 -- 172.21.15.97:0/3675612346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a8005430 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7badff700 1 -- 172.21.15.97:0/3675612346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b4153e20 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7b17fa700 1 -- 172.21.15.97:0/3675612346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80038c0 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.906+0000 7ff7badff700 1 -- 172.21.15.97:0/3675612346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b406ff10 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.907+0000 7ff7badff700 1 -- 172.21.15.97:0/3675612346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b404e340 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.908+0000 7ff7b17fa700 1 -- 172.21.15.97:0/3675612346 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff7a8003a20 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.909+0000 7ff7b17fa700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff79806ace0 0x7ff79806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:56.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.910+0000 7ff7b17fa700 1 -- 172.21.15.97:0/3675612346 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff7a8088a40 con 0x7ff7b40cbaa0 2023-07-06T03:45:56.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.910+0000 7ff7b3fff700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff79806ace0 0x7ff79806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:56.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.910+0000 7ff7b3fff700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff79806ace0 0x7ff79806d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40adb60 tx=0x7ff7a0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:56.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:56.913+0000 7ff7b17fa700 1 -- 172.21.15.97:0/3675612346 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff7a8053a50 con 0x7ff7b40cbaa0 2023-07-06T03:45:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.106+0000 7ff7badff700 1 -- 172.21.15.97:0/3675612346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff7b40c55d0 con 0x7ff7b40cbaa0 2023-07-06T03:45:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.106+0000 7ff7b17fa700 1 -- 172.21.15.97:0/3675612346 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff7a8017020 con 0x7ff7b40cbaa0 2023-07-06T03:45:57.109 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:45:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.108+0000 7ff7badff700 1 -- 172.21.15.97:0/3675612346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff79806ace0 msgr2=0x7ff79806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.108+0000 7ff7badff700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff79806ace0 0x7ff79806d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40adb60 tx=0x7ff7a0008030 comp rx=0 tx=0).stop 2023-07-06T03:45:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.108+0000 7ff7badff700 1 -- 172.21.15.97:0/3675612346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7b40cbaa0 msgr2=0x7ff7b41f3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.109+0000 7ff7badff700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7b40cbaa0 0x7ff7b41f3d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800c510 tx=0x7ff7a800c8d0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.109+0000 7ff7badff700 1 -- 172.21.15.97:0/3675612346 shutdown_connections 2023-07-06T03:45:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.109+0000 7ff7badff700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff79806ace0 0x7ff79806d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.109+0000 7ff7badff700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff7b415eb90 0x7ff7b41ef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.109+0000 7ff7badff700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7b40cbaa0 0x7ff7b41f3d80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.109+0000 7ff7badff700 1 --2- 172.21.15.97:0/3675612346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff7b4154890 0x7ff7b41ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.109+0000 7ff7badff700 1 -- 172.21.15.97:0/3675612346 >> 172.21.15.97:0/3675612346 conn(0x7ff7b406a060 msgr2=0x7ff7b4151690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:57.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.109+0000 7ff7badff700 1 -- 172.21.15.97:0/3675612346 shutdown_connections 2023-07-06T03:45:57.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.109+0000 7ff7badff700 1 -- 172.21.15.97:0/3675612346 wait complete. 2023-07-06T03:45:57.166 INFO:tasks.cephfs_test_runner:Starting test: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:45:57.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-07-06T03:45:57.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.361+0000 7f4cd21ff700 1 Processor -- start 2023-07-06T03:45:57.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.361+0000 7f4cd21ff700 1 -- start start 2023-07-06T03:45:57.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.362+0000 7f4cd21ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4ccc0cb830 0x7f4ccc0718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:57.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.362+0000 7f4cd21ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ccc070fc0 con 0x7f4ccc071450 2023-07-06T03:45:57.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.362+0000 7f4cd21ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ccc071e30 con 0x7f4ccc15a300 2023-07-06T03:45:57.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.362+0000 7f4cd21ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ccc0c96f0 con 0x7f4ccc0cb830 2023-07-06T03:45:57.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.363+0000 7f4ccaffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4ccc071450 0x7f4ccc0cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58862/0 (socket says 172.21.15.97:58862) 2023-07-06T03:45:57.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.363+0000 7f4ccaffd700 1 -- 172.21.15.97:0/3950391010 learned_addr learned my addr 172.21.15.97:0/3950391010 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:57.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.363+0000 7f4ccbfff700 1 --2- 172.21.15.97:0/3950391010 >> v2:172.21.15.97:3300/0 conn(0x7f4ccc0cb830 0x7f4ccc0718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:57.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.363+0000 7f4cca7fc700 1 -- 172.21.15.97:0/3950391010 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638625329 0 0) 0x7f4ccc070fc0 con 0x7f4ccc071450 2023-07-06T03:45:57.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.363+0000 7f4cca7fc700 1 -- 172.21.15.97:0/3950391010 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cb40035b0 con 0x7f4ccc071450 2023-07-06T03:45:57.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.363+0000 7f4ccbfff700 1 -- 172.21.15.97:0/3950391010 >> v1:172.21.15.146:6789/0 conn(0x7f4ccc15a300 legacy=0x7f4ccc0c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.363+0000 7f4ccbfff700 1 -- 172.21.15.97:0/3950391010 >> v1:172.21.15.97:6789/0 conn(0x7f4ccc071450 legacy=0x7f4ccc0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.363+0000 7f4ccbfff700 1 -- 172.21.15.97:0/3950391010 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc06c470 con 0x7f4ccc0cb830 2023-07-06T03:45:57.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.364+0000 7f4ccbfff700 1 --2- 172.21.15.97:0/3950391010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ccc0cb830 0x7f4ccc0718f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009d90 tx=0x7f4cc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15c73c773fc05d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:57.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.364+0000 7f4cca7fc700 1 -- 172.21.15.97:0/3950391010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400e070 con 0x7f4ccc0cb830 2023-07-06T03:45:57.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.364+0000 7f4cca7fc700 1 -- 172.21.15.97:0/3950391010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc400bb20 con 0x7f4ccc0cb830 2023-07-06T03:45:57.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.364+0000 7f4cca7fc700 1 -- 172.21.15.97:0/3950391010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4012430 con 0x7f4ccc0cb830 2023-07-06T03:45:57.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.364+0000 7f4cd21ff700 1 -- 172.21.15.97:0/3950391010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ccc0cb830 msgr2=0x7f4ccc0718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.364+0000 7f4cd21ff700 1 --2- 172.21.15.97:0/3950391010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ccc0cb830 0x7f4ccc0718f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009d90 tx=0x7f4cc400b330 comp rx=0 tx=0).stop 2023-07-06T03:45:57.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.364+0000 7f4cd21ff700 1 -- 172.21.15.97:0/3950391010 shutdown_connections 2023-07-06T03:45:57.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.364+0000 7f4cd21ff700 1 --2- 172.21.15.97:0/3950391010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ccc0cb830 0x7f4ccc0718f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.364+0000 7f4cd21ff700 1 -- 172.21.15.97:0/3950391010 >> 172.21.15.97:0/3950391010 conn(0x7f4ccc0b04a0 msgr2=0x7f4ccc0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:57.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.364+0000 7f4cd21ff700 1 -- 172.21.15.97:0/3950391010 shutdown_connections 2023-07-06T03:45:57.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.365+0000 7f4cd21ff700 1 -- 172.21.15.97:0/3950391010 wait complete. 2023-07-06T03:45:57.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.366+0000 7f4cd21ff700 1 Processor -- start 2023-07-06T03:45:57.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.366+0000 7f4cd21ff700 1 -- start start 2023-07-06T03:45:57.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.367+0000 7f4cd21ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ccc071450 0x7f4ccc0acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:57.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.367+0000 7f4cd21ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ccc0cb830 0x7f4ccc1f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:57.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.367+0000 7f4cd21ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ccc15a300 0x7f4ccc1f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:57.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.367+0000 7f4cd21ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc0cca90 con 0x7f4ccc071450 2023-07-06T03:45:57.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.367+0000 7f4cd21ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc0ccd70 con 0x7f4ccc0cb830 2023-07-06T03:45:57.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.367+0000 7f4cd21ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc0cd050 con 0x7f4ccc15a300 2023-07-06T03:45:57.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.367+0000 7f4ccb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ccc071450 0x7f4ccc0acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:57.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4ccb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ccc071450 0x7f4ccc0acaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37204/0 (socket says 172.21.15.97:37204) 2023-07-06T03:45:57.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4ccb7fe700 1 -- 172.21.15.97:0/1756056595 learned_addr learned my addr 172.21.15.97:0/1756056595 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:57.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4ccaffd700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ccc0cb830 0x7f4ccc1f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:57.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4ccbfff700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ccc15a300 0x7f4ccc1f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:57.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4ccb7fe700 1 -- 172.21.15.97:0/1756056595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ccc15a300 msgr2=0x7f4ccc1f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4ccb7fe700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ccc15a300 0x7f4ccc1f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4ccb7fe700 1 -- 172.21.15.97:0/1756056595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ccc0cb830 msgr2=0x7f4ccc1f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4ccb7fe700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ccc0cb830 0x7f4ccc1f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4ccb7fe700 1 -- 172.21.15.97:0/1756056595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc4009a70 con 0x7f4ccc071450 2023-07-06T03:45:57.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4ccb7fe700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ccc071450 0x7f4ccc0acaf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00c4e0 tx=0x7f4cbc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:57.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4cb3fff700 1 -- 172.21.15.97:0/1756056595 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0052d0 con 0x7f4ccc071450 2023-07-06T03:45:57.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4cb3fff700 1 -- 172.21.15.97:0/1756056595 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cbc005430 con 0x7f4ccc071450 2023-07-06T03:45:57.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4cb3fff700 1 -- 172.21.15.97:0/1756056595 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0038c0 con 0x7f4ccc071450 2023-07-06T03:45:57.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.368+0000 7f4cd21ff700 1 -- 172.21.15.97:0/1756056595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ccc0cd330 con 0x7f4ccc071450 2023-07-06T03:45:57.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.369+0000 7f4cd21ff700 1 -- 172.21.15.97:0/1756056595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ccc155ea0 con 0x7f4ccc071450 2023-07-06T03:45:57.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.370+0000 7f4cd21ff700 1 -- 172.21.15.97:0/1756056595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ccc156030 con 0x7f4ccc071450 2023-07-06T03:45:57.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.370+0000 7f4cb3fff700 1 -- 172.21.15.97:0/1756056595 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4cbc003a20 con 0x7f4ccc071450 2023-07-06T03:45:57.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.371+0000 7f4cb3fff700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cb806ace0 0x7f4cb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:57.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.371+0000 7f4cb3fff700 1 -- 172.21.15.97:0/1756056595 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4cbc088830 con 0x7f4ccc071450 2023-07-06T03:45:57.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.372+0000 7f4ccaffd700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cb806ace0 0x7f4cb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:57.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.372+0000 7f4ccaffd700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cb806ace0 0x7f4cb806d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc155ca0 tx=0x7f4cc0007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:57.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.376+0000 7f4cb3fff700 1 -- 172.21.15.97:0/1756056595 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4cbc0538d0 con 0x7f4ccc071450 2023-07-06T03:45:57.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.650+0000 7f4cd21ff700 1 -- 172.21.15.97:0/1756056595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7f4ccc04e340 con 0x7f4ccc071450 2023-07-06T03:45:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.666+0000 7f4cb3fff700 1 -- 172.21.15.97:0/1756056595 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v67) v1 ==== 168+0+0 (secure 0 0 0) 0x7f4cbc056ef0 con 0x7f4ccc071450 2023-07-06T03:45:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 -- 172.21.15.97:0/1756056595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cb806ace0 msgr2=0x7f4cb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cb806ace0 0x7f4cb806d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc155ca0 tx=0x7f4cc0007dc0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 -- 172.21.15.97:0/1756056595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ccc071450 msgr2=0x7f4ccc0acaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ccc071450 0x7f4ccc0acaf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00c4e0 tx=0x7f4cbc00c7f0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 -- 172.21.15.97:0/1756056595 shutdown_connections 2023-07-06T03:45:57.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cb806ace0 0x7f4cb806d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ccc15a300 0x7f4ccc1f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ccc071450 0x7f4ccc0acaf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 --2- 172.21.15.97:0/1756056595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ccc0cb830 0x7f4ccc1f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 -- 172.21.15.97:0/1756056595 >> 172.21.15.97:0/1756056595 conn(0x7f4ccc0b04a0 msgr2=0x7f4ccc15b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:57.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 -- 172.21.15.97:0/1756056595 shutdown_connections 2023-07-06T03:45:57.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.668+0000 7f4cd21ff700 1 -- 172.21.15.97:0/1756056595 wait complete. 2023-07-06T03:45:57.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T03:45:57.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.884+0000 7f4d781ff700 1 Processor -- start 2023-07-06T03:45:57.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.885+0000 7f4d781ff700 1 -- start start 2023-07-06T03:45:57.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.886+0000 7f4d781ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d7015c610 0x7f4d700b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:57.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.886+0000 7f4d781ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4d700d2210 0x7f4d700d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:57.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.886+0000 7f4d781ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d700cff90 con 0x7f4d7006b360 2023-07-06T03:45:57.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.886+0000 7f4d781ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d700d0780 con 0x7f4d7015c610 2023-07-06T03:45:57.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.886+0000 7f4d781ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7015c9e0 con 0x7f4d700d2210 2023-07-06T03:45:57.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.886+0000 7f4d7579a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4d7006b360 0x7f4d700b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58868/0 (socket says 172.21.15.97:58868) 2023-07-06T03:45:57.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.886+0000 7f4d75f9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d7015c610 0x7f4d700b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:57.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.886+0000 7f4d7579a700 1 -- 172.21.15.97:0/27984819 learned_addr learned my addr 172.21.15.97:0/27984819 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:57.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.886+0000 7f4d7679c700 1 --2- 172.21.15.97:0/27984819 >> v2:172.21.15.146:3300/0 conn(0x7f4d700d2210 0x7f4d700d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:57.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.886+0000 7f4d75f9b700 1 -- 172.21.15.97:0/27984819 >> v2:172.21.15.146:3300/0 conn(0x7f4d700d2210 msgr2=0x7f4d700d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:57.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.886+0000 7f4d75f9b700 1 --2- 172.21.15.97:0/27984819 >> v2:172.21.15.146:3300/0 conn(0x7f4d700d2210 0x7f4d700d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.887+0000 7f4d75f9b700 1 -- 172.21.15.97:0/27984819 >> v1:172.21.15.97:6789/0 conn(0x7f4d7006b360 legacy=0x7f4d700b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.887+0000 7f4d75f9b700 1 -- 172.21.15.97:0/27984819 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d700b5650 con 0x7f4d7015c610 2023-07-06T03:45:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.887+0000 7f4d7679c700 1 --2- 172.21.15.97:0/27984819 >> v2:172.21.15.146:3300/0 conn(0x7f4d700d2210 0x7f4d700d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.887+0000 7f4d74f99700 1 -- 172.21.15.97:0/27984819 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616100363 0 0) 0x7f4d700cff90 con 0x7f4d7006b360 2023-07-06T03:45:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.887+0000 7f4d75f9b700 1 --2- 172.21.15.97:0/27984819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d7015c610 0x7f4d700b47b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4d700d2bc0 tx=0x7f4d6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33ed4e4b71e225f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.887+0000 7f4d74f99700 1 -- 172.21.15.97:0/27984819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c011830 con 0x7f4d7015c610 2023-07-06T03:45:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.887+0000 7f4d74f99700 1 -- 172.21.15.97:0/27984819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c011990 con 0x7f4d7015c610 2023-07-06T03:45:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.887+0000 7f4d74f99700 1 -- 172.21.15.97:0/27984819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c011d20 con 0x7f4d7015c610 2023-07-06T03:45:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.888+0000 7f4d781ff700 1 -- 172.21.15.97:0/27984819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d7015c610 msgr2=0x7f4d700b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.888+0000 7f4d781ff700 1 --2- 172.21.15.97:0/27984819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d7015c610 0x7f4d700b47b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4d700d2bc0 tx=0x7f4d6c009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.888+0000 7f4d781ff700 1 -- 172.21.15.97:0/27984819 shutdown_connections 2023-07-06T03:45:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.888+0000 7f4d781ff700 1 --2- 172.21.15.97:0/27984819 >> v2:172.21.15.146:3300/0 conn(0x7f4d700d2210 0x7f4d700d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.888+0000 7f4d781ff700 1 --2- 172.21.15.97:0/27984819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d7015c610 0x7f4d700b47b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.888+0000 7f4d781ff700 1 -- 172.21.15.97:0/27984819 >> 172.21.15.97:0/27984819 conn(0x7f4d7006a6c0 msgr2=0x7f4d7006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:57.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.888+0000 7f4d781ff700 1 -- 172.21.15.97:0/27984819 shutdown_connections 2023-07-06T03:45:57.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.888+0000 7f4d781ff700 1 -- 172.21.15.97:0/27984819 wait complete. 2023-07-06T03:45:57.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.889+0000 7f4d781ff700 1 Processor -- start 2023-07-06T03:45:57.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.890+0000 7f4d781ff700 1 -- start start 2023-07-06T03:45:57.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.890+0000 7f4d781ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d7006b360 0x7f4d700c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:57.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.890+0000 7f4d781ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d700d2210 0x7f4d70157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.890+0000 7f4d781ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d7015c610 0x7f4d700cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.890+0000 7f4d781ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7015ce70 con 0x7f4d7015c610 2023-07-06T03:45:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.890+0000 7f4d781ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7015cb20 con 0x7f4d7006b360 2023-07-06T03:45:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.890+0000 7f4d781ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d700c7ee0 con 0x7f4d700d2210 2023-07-06T03:45:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.891+0000 7f4d7679c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d7015c610 0x7f4d700cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.891+0000 7f4d7679c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d7015c610 0x7f4d700cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37230/0 (socket says 172.21.15.97:37230) 2023-07-06T03:45:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.891+0000 7f4d7679c700 1 -- 172.21.15.97:0/3577394328 learned_addr learned my addr 172.21.15.97:0/3577394328 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.891+0000 7f4d75f9b700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d7006b360 0x7f4d700c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:57.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.891+0000 7f4d7579a700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d700d2210 0x7f4d70157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:57.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.891+0000 7f4d7679c700 1 -- 172.21.15.97:0/3577394328 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d700d2210 msgr2=0x7f4d70157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.891+0000 7f4d7679c700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d700d2210 0x7f4d70157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.891+0000 7f4d7679c700 1 -- 172.21.15.97:0/3577394328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d7006b360 msgr2=0x7f4d700c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:57.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.891+0000 7f4d7679c700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d7006b360 0x7f4d700c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:57.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.891+0000 7f4d7679c700 1 -- 172.21.15.97:0/3577394328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c0094a0 con 0x7f4d7015c610 2023-07-06T03:45:57.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.891+0000 7f4d7679c700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d7015c610 0x7f4d700cae50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800cef0 tx=0x7f4d6800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.892+0000 7f4d667fc700 1 -- 172.21.15.97:0/3577394328 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800adb0 con 0x7f4d7015c610 2023-07-06T03:45:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.892+0000 7f4d667fc700 1 -- 172.21.15.97:0/3577394328 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d68015440 con 0x7f4d7015c610 2023-07-06T03:45:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.892+0000 7f4d667fc700 1 -- 172.21.15.97:0/3577394328 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d680157f0 con 0x7f4d7015c610 2023-07-06T03:45:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.892+0000 7f4d781ff700 1 -- 172.21.15.97:0/3577394328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d700cb7c0 con 0x7f4d7015c610 2023-07-06T03:45:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.892+0000 7f4d781ff700 1 -- 172.21.15.97:0/3577394328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d700cbd80 con 0x7f4d7015c610 2023-07-06T03:45:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.893+0000 7f4d781ff700 1 -- 172.21.15.97:0/3577394328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7015cfb0 con 0x7f4d7015c610 2023-07-06T03:45:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.894+0000 7f4d667fc700 1 -- 172.21.15.97:0/3577394328 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d6800d430 con 0x7f4d7015c610 2023-07-06T03:45:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.895+0000 7f4d667fc700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d5406ace0 0x7f4d5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.895+0000 7f4d667fc700 1 -- 172.21.15.97:0/3577394328 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d68089a00 con 0x7f4d7015c610 2023-07-06T03:45:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.895+0000 7f4d75f9b700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d5406ace0 0x7f4d5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:57.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.895+0000 7f4d75f9b700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d5406ace0 0x7f4d5406d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0097c0 tx=0x7f4d6c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:57.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:57.899+0000 7f4d667fc700 1 -- 172.21.15.97:0/3577394328 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4d68054aa0 con 0x7f4d7015c610 2023-07-06T03:45:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.096+0000 7f4d781ff700 1 -- 172.21.15.97:0/3577394328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4d7004e340 con 0x7f4d7015c610 2023-07-06T03:45:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.101+0000 7f4d667fc700 1 -- 172.21.15.97:0/3577394328 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4d6801a0c0 con 0x7f4d7015c610 2023-07-06T03:45:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 -- 172.21.15.97:0/3577394328 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d5406ace0 msgr2=0x7f4d5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d5406ace0 0x7f4d5406d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0097c0 tx=0x7f4d6c01c040 comp rx=0 tx=0).stop 2023-07-06T03:45:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 -- 172.21.15.97:0/3577394328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d7015c610 msgr2=0x7f4d700cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d7015c610 0x7f4d700cae50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800cef0 tx=0x7f4d6800cf20 comp rx=0 tx=0).stop 2023-07-06T03:45:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 -- 172.21.15.97:0/3577394328 shutdown_connections 2023-07-06T03:45:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d5406ace0 0x7f4d5406d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d7015c610 0x7f4d700cae50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d7006b360 0x7f4d700c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 --2- 172.21.15.97:0/3577394328 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d700d2210 0x7f4d70157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 -- 172.21.15.97:0/3577394328 >> 172.21.15.97:0/3577394328 conn(0x7f4d7006a6c0 msgr2=0x7f4d700b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 -- 172.21.15.97:0/3577394328 shutdown_connections 2023-07-06T03:45:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.103+0000 7f4d781ff700 1 -- 172.21.15.97:0/3577394328 wait complete. 2023-07-06T03:45:58.124 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:45:58.124 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:45:58.222 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:45:58.333 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:45:58.334 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:45:58.334 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:45:58.384 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:45:58.384 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:45:58.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:45:58.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.705+0000 7f82f75ff700 1 Processor -- start 2023-07-06T03:45:58.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.705+0000 7f82f75ff700 1 -- start start 2023-07-06T03:45:58.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.706+0000 7f82f75ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f82f00cd390 0x7f82f0160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:58.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.706+0000 7f82f75ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f00cdae0 con 0x7f82f015c520 2023-07-06T03:45:58.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.706+0000 7f82f75ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f00b6640 con 0x7f82f0160dd0 2023-07-06T03:45:58.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.706+0000 7f82f75ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f00b6920 con 0x7f82f00cd390 2023-07-06T03:45:58.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.706+0000 7f82f539b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f82f015c520 0x7f82f0160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58882/0 (socket says 172.21.15.97:58882) 2023-07-06T03:45:58.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.706+0000 7f82f539b700 1 -- 172.21.15.97:0/3870457213 learned_addr learned my addr 172.21.15.97:0/3870457213 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:45:58.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.706+0000 7f82f4b9a700 1 --2- 172.21.15.97:0/3870457213 >> v2:172.21.15.146:3300/0 conn(0x7f82f00cd390 0x7f82f0160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:58.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.707+0000 7f82e7fff700 1 -- 172.21.15.97:0/3870457213 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649416289 0 0) 0x7f82f00cdae0 con 0x7f82f015c520 2023-07-06T03:45:58.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.707+0000 7f82e7fff700 1 -- 172.21.15.97:0/3870457213 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82dc0035b0 con 0x7f82f015c520 2023-07-06T03:45:58.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.707+0000 7f82e7fff700 1 -- 172.21.15.97:0/3870457213 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036966965 0 0) 0x7f82f00b6640 con 0x7f82f0160dd0 2023-07-06T03:45:58.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.707+0000 7f82e7fff700 1 -- 172.21.15.97:0/3870457213 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82f00cdae0 con 0x7f82f0160dd0 2023-07-06T03:45:58.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.707+0000 7f82f4b9a700 1 -- 172.21.15.97:0/3870457213 >> v1:172.21.15.146:6789/0 conn(0x7f82f0160dd0 legacy=0x7f82f00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:58.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.707+0000 7f82f4b9a700 1 -- 172.21.15.97:0/3870457213 >> v1:172.21.15.97:6789/0 conn(0x7f82f015c520 legacy=0x7f82f0160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:58.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.707+0000 7f82f4b9a700 1 -- 172.21.15.97:0/3870457213 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f00b6c00 con 0x7f82f00cd390 2023-07-06T03:45:58.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.707+0000 7f82f4b9a700 1 --2- 172.21.15.97:0/3870457213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82f00cd390 0x7f82f0160890 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009840 tx=0x7f82e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e51500a277691397 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:58.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.708+0000 7f82e7fff700 1 -- 172.21.15.97:0/3870457213 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0011830 con 0x7f82f00cd390 2023-07-06T03:45:58.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.708+0000 7f82e7fff700 1 -- 172.21.15.97:0/3870457213 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e0011990 con 0x7f82f00cd390 2023-07-06T03:45:58.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.708+0000 7f82e7fff700 1 -- 172.21.15.97:0/3870457213 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0011d20 con 0x7f82f00cd390 2023-07-06T03:45:58.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.708+0000 7f82f75ff700 1 -- 172.21.15.97:0/3870457213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82f00cd390 msgr2=0x7f82f0160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:58.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.708+0000 7f82f75ff700 1 --2- 172.21.15.97:0/3870457213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82f00cd390 0x7f82f0160890 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009840 tx=0x7f82e0009b50 comp rx=0 tx=0).stop 2023-07-06T03:45:58.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.708+0000 7f82f75ff700 1 -- 172.21.15.97:0/3870457213 shutdown_connections 2023-07-06T03:45:58.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.708+0000 7f82f75ff700 1 --2- 172.21.15.97:0/3870457213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82f00cd390 0x7f82f0160890 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.708+0000 7f82f75ff700 1 -- 172.21.15.97:0/3870457213 >> 172.21.15.97:0/3870457213 conn(0x7f82f0071790 msgr2=0x7f82f0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:58.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.708+0000 7f82f75ff700 1 -- 172.21.15.97:0/3870457213 shutdown_connections 2023-07-06T03:45:58.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.708+0000 7f82f75ff700 1 -- 172.21.15.97:0/3870457213 wait complete. 2023-07-06T03:45:58.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.710+0000 7f82f75ff700 1 Processor -- start 2023-07-06T03:45:58.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.710+0000 7f82f75ff700 1 -- start start 2023-07-06T03:45:58.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.711+0000 7f82f75ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f00cd390 0x7f82f01ef490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:58.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.711+0000 7f82f75ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f015c520 0x7f82f01f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:58.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.711+0000 7f82f75ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82f0160dd0 0x7f82f00d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:58.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.711+0000 7f82f75ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f013f900 con 0x7f82f00cd390 2023-07-06T03:45:58.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.711+0000 7f82f75ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00c91f0 con 0x7f82f0160dd0 2023-07-06T03:45:58.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.711+0000 7f82f75ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f013ec80 con 0x7f82f015c520 2023-07-06T03:45:58.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.711+0000 7f82f539b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f00cd390 0x7f82f01ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:58.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.711+0000 7f82f539b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f00cd390 0x7f82f01ef490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37246/0 (socket says 172.21.15.97:37246) 2023-07-06T03:45:58.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.711+0000 7f82f539b700 1 -- 172.21.15.97:0/390236636 learned_addr learned my addr 172.21.15.97:0/390236636 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:58.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.711+0000 7f82f4b9a700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f015c520 0x7f82f01f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:58.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.711+0000 7f82f5b9c700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82f0160dd0 0x7f82f00d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:58.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82f539b700 1 -- 172.21.15.97:0/390236636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f015c520 msgr2=0x7f82f01f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:58.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82f539b700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f015c520 0x7f82f01f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82f539b700 1 -- 172.21.15.97:0/390236636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82f0160dd0 msgr2=0x7f82f00d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:58.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82f539b700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82f0160dd0 0x7f82f00d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82f539b700 1 -- 172.21.15.97:0/390236636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00094a0 con 0x7f82f00cd390 2023-07-06T03:45:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82f4b9a700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f015c520 0x7f82f01f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:45:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82f539b700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f00cd390 0x7f82f01ef490 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c700 tx=0x7f82ec00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82e5ffb700 1 -- 172.21.15.97:0/390236636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec005d70 con 0x7f82f00cd390 2023-07-06T03:45:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82e5ffb700 1 -- 172.21.15.97:0/390236636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec0070d0 con 0x7f82f00cd390 2023-07-06T03:45:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82e5ffb700 1 -- 172.21.15.97:0/390236636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec007460 con 0x7f82f00cd390 2023-07-06T03:45:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82f75ff700 1 -- 172.21.15.97:0/390236636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f013ef60 con 0x7f82f00cd390 2023-07-06T03:45:58.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.712+0000 7f82f75ff700 1 -- 172.21.15.97:0/390236636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f013f520 con 0x7f82f00cd390 2023-07-06T03:45:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.713+0000 7f82f75ff700 1 -- 172.21.15.97:0/390236636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f013fa40 con 0x7f82f00cd390 2023-07-06T03:45:58.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.714+0000 7f82e5ffb700 1 -- 172.21.15.97:0/390236636 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f82ec0075c0 con 0x7f82f00cd390 2023-07-06T03:45:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.715+0000 7f82e5ffb700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f82d406ace0 0x7f82d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.715+0000 7f82e5ffb700 1 -- 172.21.15.97:0/390236636 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82ec088af0 con 0x7f82f00cd390 2023-07-06T03:45:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.715+0000 7f82f4b9a700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f82d406ace0 0x7f82d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:58.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.716+0000 7f82f4b9a700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f82d406ace0 0x7f82d406d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009fd0 tx=0x7f82e001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:58.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.720+0000 7f82e5ffb700 1 -- 172.21.15.97:0/390236636 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f82f013fa40 con 0x7f82f00cd390 2023-07-06T03:45:58.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.916+0000 7f82f75ff700 1 -- 172.21.15.97:0/390236636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82f013fa40 con 0x7f82f00cd390 2023-07-06T03:45:58.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.918+0000 7f82e5ffb700 1 -- 172.21.15.97:0/390236636 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f82ec053b10 con 0x7f82f00cd390 2023-07-06T03:45:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:45:58.921 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:45:58.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.920+0000 7f82f75ff700 1 -- 172.21.15.97:0/390236636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f82d406ace0 msgr2=0x7f82d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:58.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.920+0000 7f82f75ff700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f82d406ace0 0x7f82d406d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009fd0 tx=0x7f82e001c040 comp rx=0 tx=0).stop 2023-07-06T03:45:58.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.920+0000 7f82f75ff700 1 -- 172.21.15.97:0/390236636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f00cd390 msgr2=0x7f82f01ef490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:58.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.920+0000 7f82f75ff700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f00cd390 0x7f82f01ef490 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c700 tx=0x7f82ec00cac0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.920+0000 7f82f75ff700 1 -- 172.21.15.97:0/390236636 shutdown_connections 2023-07-06T03:45:58.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.920+0000 7f82f75ff700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f82d406ace0 0x7f82d406d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.920+0000 7f82f75ff700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f82f0160dd0 0x7f82f00d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.920+0000 7f82f75ff700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f00cd390 0x7f82f01ef490 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.920+0000 7f82f75ff700 1 --2- 172.21.15.97:0/390236636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82f015c520 0x7f82f01f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:58.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.920+0000 7f82f75ff700 1 -- 172.21.15.97:0/390236636 >> 172.21.15.97:0/390236636 conn(0x7f82f0071790 msgr2=0x7f82f00c95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:58.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.921+0000 7f82f75ff700 1 -- 172.21.15.97:0/390236636 shutdown_connections 2023-07-06T03:45:58.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:58.921+0000 7f82f75ff700 1 -- 172.21.15.97:0/390236636 wait complete. 2023-07-06T03:45:58.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:45:58.942 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:45:59.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.137+0000 7ff8215ff700 1 Processor -- start 2023-07-06T03:45:59.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.138+0000 7ff8215ff700 1 -- start start 2023-07-06T03:45:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff8215ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff81c159070 0x7ff81c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff8215ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff81c0c3cd0 0x7ff81c0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff8215ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff81c159440 0x7ff81c0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff8215ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c0ca260 con 0x7ff81c0c3cd0 2023-07-06T03:45:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff8215ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c0c99c0 con 0x7ff81c159440 2023-07-06T03:45:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff8215ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0c9ca0 con 0x7ff81c159070 2023-07-06T03:45:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff813fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff81c0c3cd0 0x7ff81c0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff813fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff81c0c3cd0 0x7ff81c0cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37262/0 (socket says 172.21.15.97:37262) 2023-07-06T03:45:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff813fff700 1 -- 172.21.15.97:0/1159184874 learned_addr learned my addr 172.21.15.97:0/1159184874 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:59.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff81b7fe700 1 --2- 172.21.15.97:0/1159184874 >> v2:172.21.15.146:3300/0 conn(0x7ff81c159440 0x7ff81c0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff813fff700 1 -- 172.21.15.97:0/1159184874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff81c159070 msgr2=0x7ff81c0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.139+0000 7ff81affd700 1 --2- 172.21.15.97:0/1159184874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff81c159070 0x7ff81c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.140+0000 7ff813fff700 1 --2- 172.21.15.97:0/1159184874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff81c159070 0x7ff81c0c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.140+0000 7ff813fff700 1 -- 172.21.15.97:0/1159184874 >> v2:172.21.15.146:3300/0 conn(0x7ff81c159440 msgr2=0x7ff81c0cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.140+0000 7ff813fff700 1 --2- 172.21.15.97:0/1159184874 >> v2:172.21.15.146:3300/0 conn(0x7ff81c159440 0x7ff81c0cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.140+0000 7ff813fff700 1 -- 172.21.15.97:0/1159184874 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0c9f80 con 0x7ff81c0c3cd0 2023-07-06T03:45:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.140+0000 7ff81b7fe700 1 --2- 172.21.15.97:0/1159184874 >> v2:172.21.15.146:3300/0 conn(0x7ff81c159440 0x7ff81c0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.140+0000 7ff81affd700 1 --2- 172.21.15.97:0/1159184874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff81c159070 0x7ff81c0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:45:59.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.140+0000 7ff813fff700 1 --2- 172.21.15.97:0/1159184874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff81c0c3cd0 0x7ff81c0cc4c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0ce370 tx=0x7ff804009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0593c0403851d6b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:59.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.140+0000 7ff81a7fc700 1 -- 172.21.15.97:0/1159184874 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80400b870 con 0x7ff81c0c3cd0 2023-07-06T03:45:59.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.140+0000 7ff81a7fc700 1 -- 172.21.15.97:0/1159184874 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80400b9d0 con 0x7ff81c0c3cd0 2023-07-06T03:45:59.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.140+0000 7ff81a7fc700 1 -- 172.21.15.97:0/1159184874 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80400bd60 con 0x7ff81c0c3cd0 2023-07-06T03:45:59.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.141+0000 7ff8215ff700 1 -- 172.21.15.97:0/1159184874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff81c0c3cd0 msgr2=0x7ff81c0cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:59.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.141+0000 7ff8215ff700 1 --2- 172.21.15.97:0/1159184874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff81c0c3cd0 0x7ff81c0cc4c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0ce370 tx=0x7ff804009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.141+0000 7ff8215ff700 1 -- 172.21.15.97:0/1159184874 shutdown_connections 2023-07-06T03:45:59.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.141+0000 7ff8215ff700 1 --2- 172.21.15.97:0/1159184874 >> v2:172.21.15.146:3300/0 conn(0x7ff81c159440 0x7ff81c0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.141+0000 7ff8215ff700 1 --2- 172.21.15.97:0/1159184874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff81c159070 0x7ff81c0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.141+0000 7ff8215ff700 1 --2- 172.21.15.97:0/1159184874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff81c0c3cd0 0x7ff81c0cc4c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.141+0000 7ff8215ff700 1 -- 172.21.15.97:0/1159184874 >> 172.21.15.97:0/1159184874 conn(0x7ff81c06a060 msgr2=0x7ff81c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.141+0000 7ff8215ff700 1 -- 172.21.15.97:0/1159184874 shutdown_connections 2023-07-06T03:45:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.141+0000 7ff8215ff700 1 -- 172.21.15.97:0/1159184874 wait complete. 2023-07-06T03:45:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.142+0000 7ff8215ff700 1 Processor -- start 2023-07-06T03:45:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.143+0000 7ff8215ff700 1 -- start start 2023-07-06T03:45:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.143+0000 7ff8215ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff81c0c3cd0 0x7ff81c14ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.143+0000 7ff8215ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff81c159070 0x7ff81c15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff8215ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff81c159440 0x7ff81c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff8215ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0b4390 con 0x7ff81c159070 2023-07-06T03:45:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff8215ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c15c110 con 0x7ff81c0c3cd0 2023-07-06T03:45:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff8215ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c15b500 con 0x7ff81c159440 2023-07-06T03:45:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff813fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff81c159070 0x7ff81c15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff813fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff81c159070 0x7ff81c15ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37274/0 (socket says 172.21.15.97:37274) 2023-07-06T03:45:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff813fff700 1 -- 172.21.15.97:0/1099967209 learned_addr learned my addr 172.21.15.97:0/1099967209 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff81b7fe700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff81c159440 0x7ff81c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff81affd700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff81c0c3cd0 0x7ff81c14ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff813fff700 1 -- 172.21.15.97:0/1099967209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff81c159440 msgr2=0x7ff81c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff813fff700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff81c159440 0x7ff81c157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff813fff700 1 -- 172.21.15.97:0/1099967209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff81c0c3cd0 msgr2=0x7ff81c14ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff813fff700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff81c0c3cd0 0x7ff81c14ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff813fff700 1 -- 172.21.15.97:0/1099967209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804009550 con 0x7ff81c159070 2023-07-06T03:45:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.144+0000 7ff813fff700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff81c159070 0x7ff81c15ae60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff81c06de50 tx=0x7ff804004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.145+0000 7ff8137fe700 1 -- 172.21.15.97:0/1099967209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804004310 con 0x7ff81c159070 2023-07-06T03:45:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.145+0000 7ff8137fe700 1 -- 172.21.15.97:0/1099967209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80401f070 con 0x7ff81c159070 2023-07-06T03:45:59.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.145+0000 7ff8215ff700 1 -- 172.21.15.97:0/1099967209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c15b7e0 con 0x7ff81c159070 2023-07-06T03:45:59.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.145+0000 7ff8137fe700 1 -- 172.21.15.97:0/1099967209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040036a0 con 0x7ff81c159070 2023-07-06T03:45:59.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.145+0000 7ff8215ff700 1 -- 172.21.15.97:0/1099967209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c15bda0 con 0x7ff81c159070 2023-07-06T03:45:59.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.146+0000 7ff8215ff700 1 -- 172.21.15.97:0/1099967209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c04e340 con 0x7ff81c159070 2023-07-06T03:45:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.147+0000 7ff8137fe700 1 -- 172.21.15.97:0/1099967209 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff804021030 con 0x7ff81c159070 2023-07-06T03:45:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.148+0000 7ff8137fe700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff7fc06ad10 0x7ff7fc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.148+0000 7ff8137fe700 1 -- 172.21.15.97:0/1099967209 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff80408a850 con 0x7ff81c159070 2023-07-06T03:45:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.148+0000 7ff81affd700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff7fc06ad10 0x7ff7fc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.149+0000 7ff81affd700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff7fc06ad10 0x7ff7fc06d1c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff80c006fd0 tx=0x7ff80c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:59.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.152+0000 7ff8137fe700 1 -- 172.21.15.97:0/1099967209 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff804055990 con 0x7ff81c159070 2023-07-06T03:45:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.346+0000 7ff8215ff700 1 -- 172.21.15.97:0/1099967209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff81c15bf30 con 0x7ff81c159070 2023-07-06T03:45:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.347+0000 7ff8137fe700 1 -- 172.21.15.97:0/1099967209 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff804058fb0 con 0x7ff81c159070 2023-07-06T03:45:59.350 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:45:59.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.349+0000 7ff8215ff700 1 -- 172.21.15.97:0/1099967209 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff7fc06ad10 msgr2=0x7ff7fc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:59.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.349+0000 7ff8215ff700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff7fc06ad10 0x7ff7fc06d1c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff80c006fd0 tx=0x7ff80c008040 comp rx=0 tx=0).stop 2023-07-06T03:45:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.349+0000 7ff8215ff700 1 -- 172.21.15.97:0/1099967209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff81c159070 msgr2=0x7ff81c15ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.349+0000 7ff8215ff700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff81c159070 0x7ff81c15ae60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff81c06de50 tx=0x7ff804004750 comp rx=0 tx=0).stop 2023-07-06T03:45:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.349+0000 7ff8215ff700 1 -- 172.21.15.97:0/1099967209 shutdown_connections 2023-07-06T03:45:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.349+0000 7ff8215ff700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff7fc06ad10 0x7ff7fc06d1c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.349+0000 7ff8215ff700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff81c159440 0x7ff81c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.349+0000 7ff8215ff700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff81c0c3cd0 0x7ff81c14ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.349+0000 7ff8215ff700 1 --2- 172.21.15.97:0/1099967209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff81c159070 0x7ff81c15ae60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.349+0000 7ff8215ff700 1 -- 172.21.15.97:0/1099967209 >> 172.21.15.97:0/1099967209 conn(0x7ff81c06a060 msgr2=0x7ff81c152840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.350+0000 7ff8215ff700 1 -- 172.21.15.97:0/1099967209 shutdown_connections 2023-07-06T03:45:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.350+0000 7ff8215ff700 1 -- 172.21.15.97:0/1099967209 wait complete. 2023-07-06T03:45:59.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:45:59.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.563+0000 7f9200bff700 1 Processor -- start 2023-07-06T03:45:59.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.563+0000 7f9200bff700 1 -- start start 2023-07-06T03:45:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f9200bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91fc159070 0x7f91fc0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f9200bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f91fc0c3cd0 0x7f91fc0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f9200bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91fc0ca260 con 0x7f91fc159440 2023-07-06T03:45:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f9200bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91fc0c99c0 con 0x7f91fc159070 2023-07-06T03:45:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f9200bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91fc0c9ca0 con 0x7f91fc0c3cd0 2023-07-06T03:45:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f91fa59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91fc159070 0x7f91fc0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f91fa59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91fc159070 0x7f91fc0c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37284/0 (socket says 172.21.15.97:37284) 2023-07-06T03:45:59.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f91fa59c700 1 -- 172.21.15.97:0/2472295374 learned_addr learned my addr 172.21.15.97:0/2472295374 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:59.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f91f1d9b700 1 --2- 172.21.15.97:0/2472295374 >> v2:172.21.15.146:3300/0 conn(0x7f91fc0c3cd0 0x7f91fc0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f91fa59c700 1 -- 172.21.15.97:0/2472295374 >> v2:172.21.15.146:3300/0 conn(0x7f91fc0c3cd0 msgr2=0x7f91fc0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:45:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f91fa59c700 1 --2- 172.21.15.97:0/2472295374 >> v2:172.21.15.146:3300/0 conn(0x7f91fc0c3cd0 0x7f91fc0cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f91fa59c700 1 -- 172.21.15.97:0/2472295374 >> v1:172.21.15.97:6789/0 conn(0x7f91fc159440 legacy=0x7f91fc0b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f91fa59c700 1 -- 172.21.15.97:0/2472295374 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc0c9f80 con 0x7f91fc159070 2023-07-06T03:45:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.565+0000 7f91f9d9b700 1 -- 172.21.15.97:0/2472295374 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572583309 0 0) 0x7f91fc0ca260 con 0x7f91fc159440 2023-07-06T03:45:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.566+0000 7f91fa59c700 1 --2- 172.21.15.97:0/2472295374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc159070 0x7f91fc0c8f40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0cf9f0 tx=0x7f91e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea3aa219b5ef4e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.566+0000 7f91f9d9b700 1 -- 172.21.15.97:0/2472295374 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4011830 con 0x7f91fc159070 2023-07-06T03:45:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.566+0000 7f91f9d9b700 1 -- 172.21.15.97:0/2472295374 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e4011990 con 0x7f91fc159070 2023-07-06T03:45:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.566+0000 7f91f9d9b700 1 -- 172.21.15.97:0/2472295374 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4011d20 con 0x7f91fc159070 2023-07-06T03:45:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.566+0000 7f9200bff700 1 -- 172.21.15.97:0/2472295374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc159070 msgr2=0x7f91fc0c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.566+0000 7f9200bff700 1 --2- 172.21.15.97:0/2472295374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc159070 0x7f91fc0c8f40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0cf9f0 tx=0x7f91e4009bd0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.567+0000 7f9200bff700 1 -- 172.21.15.97:0/2472295374 shutdown_connections 2023-07-06T03:45:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.567+0000 7f9200bff700 1 --2- 172.21.15.97:0/2472295374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc159070 0x7f91fc0c8f40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.567+0000 7f9200bff700 1 --2- 172.21.15.97:0/2472295374 >> v2:172.21.15.146:3300/0 conn(0x7f91fc0c3cd0 0x7f91fc0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.567+0000 7f9200bff700 1 -- 172.21.15.97:0/2472295374 >> 172.21.15.97:0/2472295374 conn(0x7f91fc06a060 msgr2=0x7f91fc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.567+0000 7f9200bff700 1 -- 172.21.15.97:0/2472295374 shutdown_connections 2023-07-06T03:45:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.567+0000 7f9200bff700 1 -- 172.21.15.97:0/2472295374 wait complete. 2023-07-06T03:45:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.568+0000 7f9200bff700 1 Processor -- start 2023-07-06T03:45:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.569+0000 7f9200bff700 1 -- start start 2023-07-06T03:45:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.569+0000 7f9200bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91fc0c3cd0 0x7f91fc0aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f9200bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91fc159070 0x7f91fc151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f9200bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc159440 0x7f91fc14f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f9200bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc151fb0 con 0x7f91fc159440 2023-07-06T03:45:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f9200bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0af530 con 0x7f91fc0c3cd0 2023-07-06T03:45:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f9200bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0af810 con 0x7f91fc159070 2023-07-06T03:45:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f91fad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc159440 0x7f91fc14f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f91fad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc159440 0x7f91fc14f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37294/0 (socket says 172.21.15.97:37294) 2023-07-06T03:45:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f91fad9d700 1 -- 172.21.15.97:0/3457746931 learned_addr learned my addr 172.21.15.97:0/3457746931 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:45:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f91fa59c700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91fc0c3cd0 0x7f91fc0aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f91f1d9b700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91fc159070 0x7f91fc151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f91fad9d700 1 -- 172.21.15.97:0/3457746931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91fc159070 msgr2=0x7f91fc151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f91fad9d700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91fc159070 0x7f91fc151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f91fad9d700 1 -- 172.21.15.97:0/3457746931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91fc0c3cd0 msgr2=0x7f91fc0aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f91fad9d700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91fc0c3cd0 0x7f91fc0aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f91fad9d700 1 -- 172.21.15.97:0/3457746931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e4009550 con 0x7f91fc159440 2023-07-06T03:45:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.570+0000 7f91fad9d700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc159440 0x7f91fc14f640 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c7a0 tx=0x7f91f400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.571+0000 7f91f37fe700 1 -- 172.21.15.97:0/3457746931 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4003e30 con 0x7f91fc159440 2023-07-06T03:45:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.571+0000 7f91f37fe700 1 -- 172.21.15.97:0/3457746931 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f4005070 con 0x7f91fc159440 2023-07-06T03:45:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.571+0000 7f91f37fe700 1 -- 172.21.15.97:0/3457746931 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4005420 con 0x7f91fc159440 2023-07-06T03:45:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.571+0000 7f9200bff700 1 -- 172.21.15.97:0/3457746931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91fc0afaf0 con 0x7f91fc159440 2023-07-06T03:45:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.571+0000 7f9200bff700 1 -- 172.21.15.97:0/3457746931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91fc0b00b0 con 0x7f91fc159440 2023-07-06T03:45:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.572+0000 7f9200bff700 1 -- 172.21.15.97:0/3457746931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91fc04e340 con 0x7f91fc159440 2023-07-06T03:45:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.573+0000 7f91f37fe700 1 -- 172.21.15.97:0/3457746931 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f91f4005580 con 0x7f91fc159440 2023-07-06T03:45:59.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.574+0000 7f91f37fe700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91dc06ace0 0x7f91dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:45:59.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.574+0000 7f91f37fe700 1 -- 172.21.15.97:0/3457746931 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f91f4088a30 con 0x7f91fc159440 2023-07-06T03:45:59.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.574+0000 7f91fa59c700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91dc06ace0 0x7f91dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:45:59.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.574+0000 7f91fa59c700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91dc06ace0 0x7f91dc06d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009c10 tx=0x7f91e4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:45:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.578+0000 7f91f37fe700 1 -- 172.21.15.97:0/3457746931 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f91f4057560 con 0x7f91fc159440 2023-07-06T03:45:59.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.778+0000 7f9200bff700 1 -- 172.21.15.97:0/3457746931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f91fc06d220 con 0x7f91fc159440 2023-07-06T03:45:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.778+0000 7f91f37fe700 1 -- 172.21.15.97:0/3457746931 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f91f4017020 con 0x7f91fc159440 2023-07-06T03:45:59.781 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:45:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.780+0000 7f9200bff700 1 -- 172.21.15.97:0/3457746931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91dc06ace0 msgr2=0x7f91dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:59.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.780+0000 7f9200bff700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91dc06ace0 0x7f91dc06d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009c10 tx=0x7f91e4008030 comp rx=0 tx=0).stop 2023-07-06T03:45:59.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.780+0000 7f9200bff700 1 -- 172.21.15.97:0/3457746931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc159440 msgr2=0x7f91fc14f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:45:59.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.780+0000 7f9200bff700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc159440 0x7f91fc14f640 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c7a0 tx=0x7f91f400cab0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.781+0000 7f9200bff700 1 -- 172.21.15.97:0/3457746931 shutdown_connections 2023-07-06T03:45:59.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.781+0000 7f9200bff700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91dc06ace0 0x7f91dc06d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.781+0000 7f9200bff700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc159440 0x7f91fc14f640 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.781+0000 7f9200bff700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91fc0c3cd0 0x7f91fc0aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.781+0000 7f9200bff700 1 --2- 172.21.15.97:0/3457746931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91fc159070 0x7f91fc151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:45:59.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.781+0000 7f9200bff700 1 -- 172.21.15.97:0/3457746931 >> 172.21.15.97:0/3457746931 conn(0x7f91fc06a060 msgr2=0x7f91fc0ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:45:59.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.781+0000 7f9200bff700 1 -- 172.21.15.97:0/3457746931 shutdown_connections 2023-07-06T03:45:59.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.781+0000 7f9200bff700 1 -- 172.21.15.97:0/3457746931 wait complete. 2023-07-06T03:45:59.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:45:59.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.996+0000 7f72c4bff700 1 Processor -- start 2023-07-06T03:45:59.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.996+0000 7f72c4bff700 1 -- start start 2023-07-06T03:46:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.997+0000 7f72c4bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f72c015c610 0x7f72c00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.997+0000 7f72c4bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72c006b360 0x7f72c00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.997+0000 7f72c4bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c00cff90 con 0x7f72c00d2210 2023-07-06T03:46:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.997+0000 7f72c4bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c00d0780 con 0x7f72c006b360 2023-07-06T03:46:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.997+0000 7f72c4bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c015c9e0 con 0x7f72c015c610 2023-07-06T03:46:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.997+0000 7f72b7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72c006b360 0x7f72c00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.997+0000 7f72bed9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f72c00d2210 0x7f72c00d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56710/0 (socket says 172.21.15.97:56710) 2023-07-06T03:46:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.997+0000 7f72bed9d700 1 -- 172.21.15.97:0/1466787854 learned_addr learned my addr 172.21.15.97:0/1466787854 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.997+0000 7f72be59c700 1 --2- 172.21.15.97:0/1466787854 >> v2:172.21.15.146:3300/0 conn(0x7f72c015c610 0x7f72c00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.998+0000 7f72b7fff700 1 -- 172.21.15.97:0/1466787854 >> v2:172.21.15.146:3300/0 conn(0x7f72c015c610 msgr2=0x7f72c00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.998+0000 7f72b7fff700 1 --2- 172.21.15.97:0/1466787854 >> v2:172.21.15.146:3300/0 conn(0x7f72c015c610 0x7f72c00b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.998+0000 7f72b7fff700 1 -- 172.21.15.97:0/1466787854 >> v1:172.21.15.97:6789/0 conn(0x7f72c00d2210 legacy=0x7f72c00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.998+0000 7f72b7fff700 1 -- 172.21.15.97:0/1466787854 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c00b5650 con 0x7f72c006b360 2023-07-06T03:46:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.998+0000 7f72bdd9b700 1 -- 172.21.15.97:0/1466787854 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727045299 0 0) 0x7f72c00cff90 con 0x7f72c00d2210 2023-07-06T03:46:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.998+0000 7f72b7fff700 1 --2- 172.21.15.97:0/1466787854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72c006b360 0x7f72c00b4cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f72c008b000 tx=0x7f72a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946fb3201124fd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.998+0000 7f72bdd9b700 1 -- 172.21.15.97:0/1466787854 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8011830 con 0x7f72c006b360 2023-07-06T03:46:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.998+0000 7f72bdd9b700 1 -- 172.21.15.97:0/1466787854 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a8011990 con 0x7f72c006b360 2023-07-06T03:46:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.998+0000 7f72bdd9b700 1 -- 172.21.15.97:0/1466787854 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8011d20 con 0x7f72c006b360 2023-07-06T03:46:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.999+0000 7f72c4bff700 1 -- 172.21.15.97:0/1466787854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72c006b360 msgr2=0x7f72c00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.999+0000 7f72c4bff700 1 --2- 172.21.15.97:0/1466787854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72c006b360 0x7f72c00b4cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f72c008b000 tx=0x7f72a8009b50 comp rx=0 tx=0).stop 2023-07-06T03:46:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.999+0000 7f72c4bff700 1 -- 172.21.15.97:0/1466787854 shutdown_connections 2023-07-06T03:46:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.999+0000 7f72c4bff700 1 --2- 172.21.15.97:0/1466787854 >> v2:172.21.15.146:3300/0 conn(0x7f72c015c610 0x7f72c00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.999+0000 7f72c4bff700 1 --2- 172.21.15.97:0/1466787854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72c006b360 0x7f72c00b4cf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.999+0000 7f72c4bff700 1 -- 172.21.15.97:0/1466787854 >> 172.21.15.97:0/1466787854 conn(0x7f72c006a6c0 msgr2=0x7f72c006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.999+0000 7f72c4bff700 1 -- 172.21.15.97:0/1466787854 shutdown_connections 2023-07-06T03:46:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:45:59.999+0000 7f72c4bff700 1 -- 172.21.15.97:0/1466787854 wait complete. 2023-07-06T03:46:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.001+0000 7f72c4bff700 1 Processor -- start 2023-07-06T03:46:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.001+0000 7f72c4bff700 1 -- start start 2023-07-06T03:46:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72c4bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72c006b360 0x7f72c015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72c4bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72c00d2210 0x7f72c00b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72c4bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72c015c610 0x7f72c00b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72c4bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c01f5540 con 0x7f72c015c610 2023-07-06T03:46:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72c4bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c01f5790 con 0x7f72c00d2210 2023-07-06T03:46:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72c4bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c01f5a70 con 0x7f72c006b360 2023-07-06T03:46:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72bed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72c015c610 0x7f72c00b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72bed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72c015c610 0x7f72c00b7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54706/0 (socket says 172.21.15.97:54706) 2023-07-06T03:46:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72bed9d700 1 -- 172.21.15.97:0/3654529292 learned_addr learned my addr 172.21.15.97:0/3654529292 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72be59c700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72c006b360 0x7f72c015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72b7fff700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72c00d2210 0x7f72c00b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72bed9d700 1 -- 172.21.15.97:0/3654529292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72c006b360 msgr2=0x7f72c015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72bed9d700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72c006b360 0x7f72c015cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72bed9d700 1 -- 172.21.15.97:0/3654529292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72c00d2210 msgr2=0x7f72c00b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72bed9d700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72c00d2210 0x7f72c00b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72bed9d700 1 -- 172.21.15.97:0/3654529292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a80094a0 con 0x7f72c015c610 2023-07-06T03:46:00.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.002+0000 7f72bed9d700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72c015c610 0x7f72c00b7070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f72b8002230 tx=0x7f72b800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:00.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.003+0000 7f72b6ffd700 1 -- 172.21.15.97:0/3654529292 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800ac20 con 0x7f72c015c610 2023-07-06T03:46:00.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.003+0000 7f72b6ffd700 1 -- 172.21.15.97:0/3654529292 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800ada0 con 0x7f72c015c610 2023-07-06T03:46:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.003+0000 7f72b6ffd700 1 -- 172.21.15.97:0/3654529292 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8007310 con 0x7f72c015c610 2023-07-06T03:46:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.003+0000 7f72c4bff700 1 -- 172.21.15.97:0/3654529292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c01f5d50 con 0x7f72c015c610 2023-07-06T03:46:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.003+0000 7f72c4bff700 1 -- 172.21.15.97:0/3654529292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c00cb320 con 0x7f72c015c610 2023-07-06T03:46:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.004+0000 7f72c4bff700 1 -- 172.21.15.97:0/3654529292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c01f6270 con 0x7f72c015c610 2023-07-06T03:46:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.005+0000 7f72b6ffd700 1 -- 172.21.15.97:0/3654529292 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72b8007470 con 0x7f72c015c610 2023-07-06T03:46:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.006+0000 7f72b6ffd700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f72a006ace0 0x7f72a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.006+0000 7f72b6ffd700 1 -- 172.21.15.97:0/3654529292 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72b8088f50 con 0x7f72c015c610 2023-07-06T03:46:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.006+0000 7f72be59c700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f72a006ace0 0x7f72a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.007+0000 7f72be59c700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f72a006ace0 0x7f72a006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f72c00b84a0 tx=0x7f72b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.010+0000 7f72b6ffd700 1 -- 172.21.15.97:0/3654529292 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f72c01f6270 con 0x7f72c015c610 2023-07-06T03:46:00.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.207+0000 7f72c4bff700 1 -- 172.21.15.97:0/3654529292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72c01f6270 con 0x7f72c015c610 2023-07-06T03:46:00.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.208+0000 7f72b6ffd700 1 -- 172.21.15.97:0/3654529292 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f72b8018020 con 0x7f72c015c610 2023-07-06T03:46:00.211 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:46:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 -- 172.21.15.97:0/3654529292 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f72a006ace0 msgr2=0x7f72a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f72a006ace0 0x7f72a006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f72c00b84a0 tx=0x7f72b0008040 comp rx=0 tx=0).stop 2023-07-06T03:46:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 -- 172.21.15.97:0/3654529292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72c015c610 msgr2=0x7f72c00b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72c015c610 0x7f72c00b7070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f72b8002230 tx=0x7f72b800a060 comp rx=0 tx=0).stop 2023-07-06T03:46:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 -- 172.21.15.97:0/3654529292 shutdown_connections 2023-07-06T03:46:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f72a006ace0 0x7f72a006d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72c015c610 0x7f72c00b7070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72c006b360 0x7f72c015cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 --2- 172.21.15.97:0/3654529292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72c00d2210 0x7f72c00b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 -- 172.21.15.97:0/3654529292 >> 172.21.15.97:0/3654529292 conn(0x7f72c006a6c0 msgr2=0x7f72c015b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:00.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 -- 172.21.15.97:0/3654529292 shutdown_connections 2023-07-06T03:46:00.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.210+0000 7f72c4bff700 1 -- 172.21.15.97:0/3654529292 wait complete. 2023-07-06T03:46:00.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:46:00.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.425+0000 7f8888fff700 1 Processor -- start 2023-07-06T03:46:00.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.425+0000 7f8888fff700 1 -- start start 2023-07-06T03:46:00.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.426+0000 7f8888fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f888415c610 0x7f88840b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.426+0000 7f8888fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88840d2210 0x7f88840d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.426+0000 7f8888fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88840cff90 con 0x7f888406b360 2023-07-06T03:46:00.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.426+0000 7f8888fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88840d0780 con 0x7f888415c610 2023-07-06T03:46:00.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.426+0000 7f8888fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888415c9e0 con 0x7f88840d2210 2023-07-06T03:46:00.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.427+0000 7f8881d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f888406b360 0x7f88840b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56724/0 (socket says 172.21.15.97:56724) 2023-07-06T03:46:00.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.427+0000 7f8881d9b700 1 -- 172.21.15.97:0/1012759435 learned_addr learned my addr 172.21.15.97:0/1012759435 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:00.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.427+0000 7f8882d9d700 1 --2- 172.21.15.97:0/1012759435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88840d2210 0x7f88840d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.427+0000 7f888259c700 1 --2- 172.21.15.97:0/1012759435 >> v2:172.21.15.146:3300/0 conn(0x7f888415c610 0x7f88840b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.427+0000 7f888159a700 1 -- 172.21.15.97:0/1012759435 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303362943 0 0) 0x7f88840cff90 con 0x7f888406b360 2023-07-06T03:46:00.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.427+0000 7f888159a700 1 -- 172.21.15.97:0/1012759435 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f886c0035b0 con 0x7f888406b360 2023-07-06T03:46:00.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.427+0000 7f8882d9d700 1 -- 172.21.15.97:0/1012759435 >> v2:172.21.15.146:3300/0 conn(0x7f888415c610 msgr2=0x7f88840b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:00.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.427+0000 7f8882d9d700 1 --2- 172.21.15.97:0/1012759435 >> v2:172.21.15.146:3300/0 conn(0x7f888415c610 0x7f88840b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.427+0000 7f8882d9d700 1 -- 172.21.15.97:0/1012759435 >> v1:172.21.15.97:6789/0 conn(0x7f888406b360 legacy=0x7f88840b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.427+0000 7f8882d9d700 1 -- 172.21.15.97:0/1012759435 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840b5650 con 0x7f88840d2210 2023-07-06T03:46:00.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.428+0000 7f8882d9d700 1 --2- 172.21.15.97:0/1012759435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88840d2210 0x7f88840d1310 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f887c009d90 tx=0x7f887c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf67af2ee2a0821c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:00.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.428+0000 7f888159a700 1 -- 172.21.15.97:0/1012759435 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c00e070 con 0x7f88840d2210 2023-07-06T03:46:00.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.428+0000 7f888159a700 1 -- 172.21.15.97:0/1012759435 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c00bb20 con 0x7f88840d2210 2023-07-06T03:46:00.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.428+0000 7f888159a700 1 -- 172.21.15.97:0/1012759435 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c012430 con 0x7f88840d2210 2023-07-06T03:46:00.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.428+0000 7f8888fff700 1 -- 172.21.15.97:0/1012759435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88840d2210 msgr2=0x7f88840d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.428+0000 7f8888fff700 1 --2- 172.21.15.97:0/1012759435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88840d2210 0x7f88840d1310 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f887c009d90 tx=0x7f887c00b330 comp rx=0 tx=0).stop 2023-07-06T03:46:00.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.429+0000 7f8888fff700 1 -- 172.21.15.97:0/1012759435 shutdown_connections 2023-07-06T03:46:00.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.429+0000 7f8888fff700 1 --2- 172.21.15.97:0/1012759435 >> v2:172.21.15.146:3300/0 conn(0x7f888415c610 0x7f88840b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.429+0000 7f8888fff700 1 --2- 172.21.15.97:0/1012759435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88840d2210 0x7f88840d1310 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.429+0000 7f8888fff700 1 -- 172.21.15.97:0/1012759435 >> 172.21.15.97:0/1012759435 conn(0x7f888406a6c0 msgr2=0x7f888406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:00.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.429+0000 7f8888fff700 1 -- 172.21.15.97:0/1012759435 shutdown_connections 2023-07-06T03:46:00.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.429+0000 7f8888fff700 1 -- 172.21.15.97:0/1012759435 wait complete. 2023-07-06T03:46:00.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.430+0000 7f8888fff700 1 Processor -- start 2023-07-06T03:46:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.431+0000 7f8888fff700 1 -- start start 2023-07-06T03:46:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.431+0000 7f8888fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f888406b360 0x7f88840caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.431+0000 7f8888fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88840d2210 0x7f88840cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.431+0000 7f8888fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f888415c610 0x7f88840cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.431+0000 7f8888fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888415d9d0 con 0x7f888415c610 2023-07-06T03:46:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.431+0000 7f8888fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888415cb20 con 0x7f88840d2210 2023-07-06T03:46:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.431+0000 7f8888fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888415cdc0 con 0x7f888406b360 2023-07-06T03:46:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.432+0000 7f8882d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f888415c610 0x7f88840cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.432+0000 7f888259c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f888406b360 0x7f88840caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.432+0000 7f8882d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f888415c610 0x7f88840cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54716/0 (socket says 172.21.15.97:54716) 2023-07-06T03:46:00.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.432+0000 7f8882d9d700 1 -- 172.21.15.97:0/1311597771 learned_addr learned my addr 172.21.15.97:0/1311597771 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:00.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.432+0000 7f8881d9b700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88840d2210 0x7f88840cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.432+0000 7f8882d9d700 1 -- 172.21.15.97:0/1311597771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f888406b360 msgr2=0x7f88840caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.432+0000 7f8882d9d700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f888406b360 0x7f88840caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.432+0000 7f8882d9d700 1 -- 172.21.15.97:0/1311597771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88840d2210 msgr2=0x7f88840cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.432+0000 7f8882d9d700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88840d2210 0x7f88840cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.432+0000 7f8882d9d700 1 -- 172.21.15.97:0/1311597771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c009a70 con 0x7f888415c610 2023-07-06T03:46:00.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.432+0000 7f8882d9d700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f888415c610 0x7f88840cbc30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f887c014040 tx=0x7f887c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:00.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.433+0000 7f8872ffd700 1 -- 172.21.15.97:0/1311597771 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c0129f0 con 0x7f888415c610 2023-07-06T03:46:00.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.433+0000 7f8872ffd700 1 -- 172.21.15.97:0/1311597771 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c020070 con 0x7f888415c610 2023-07-06T03:46:00.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.433+0000 7f8872ffd700 1 -- 172.21.15.97:0/1311597771 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c012db0 con 0x7f888415c610 2023-07-06T03:46:00.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.433+0000 7f8888fff700 1 -- 172.21.15.97:0/1311597771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888415d0a0 con 0x7f888415c610 2023-07-06T03:46:00.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.433+0000 7f8888fff700 1 -- 172.21.15.97:0/1311597771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888415d660 con 0x7f888415c610 2023-07-06T03:46:00.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.434+0000 7f8888fff700 1 -- 172.21.15.97:0/1311597771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888404e340 con 0x7f888415c610 2023-07-06T03:46:00.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.435+0000 7f8872ffd700 1 -- 172.21.15.97:0/1311597771 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f887c004040 con 0x7f888415c610 2023-07-06T03:46:00.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.436+0000 7f8872ffd700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f886406ace0 0x7f886406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.436+0000 7f8872ffd700 1 -- 172.21.15.97:0/1311597771 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f887c022070 con 0x7f888415c610 2023-07-06T03:46:00.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.436+0000 7f888259c700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f886406ace0 0x7f886406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.436+0000 7f888259c700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f886406ace0 0x7f886406d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8874009b40 tx=0x7f8874009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:00.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.440+0000 7f8872ffd700 1 -- 172.21.15.97:0/1311597771 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f887c0560b0 con 0x7f888415c610 2023-07-06T03:46:00.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.632+0000 7f8888fff700 1 -- 172.21.15.97:0/1311597771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f888415b080 con 0x7f888415c610 2023-07-06T03:46:00.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.633+0000 7f8872ffd700 1 -- 172.21.15.97:0/1311597771 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f887c02c0a0 con 0x7f888415c610 2023-07-06T03:46:00.637 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:46:00.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.635+0000 7f8888fff700 1 -- 172.21.15.97:0/1311597771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f886406ace0 msgr2=0x7f886406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.635+0000 7f8888fff700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f886406ace0 0x7f886406d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8874009b40 tx=0x7f8874009210 comp rx=0 tx=0).stop 2023-07-06T03:46:00.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.635+0000 7f8888fff700 1 -- 172.21.15.97:0/1311597771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f888415c610 msgr2=0x7f88840cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.635+0000 7f8888fff700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f888415c610 0x7f88840cbc30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f887c014040 tx=0x7f887c004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.635+0000 7f8888fff700 1 -- 172.21.15.97:0/1311597771 shutdown_connections 2023-07-06T03:46:00.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.635+0000 7f8888fff700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f886406ace0 0x7f886406d190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.635+0000 7f8888fff700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f888415c610 0x7f88840cbc30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.635+0000 7f8888fff700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f888406b360 0x7f88840caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.635+0000 7f8888fff700 1 --2- 172.21.15.97:0/1311597771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88840d2210 0x7f88840cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.635+0000 7f8888fff700 1 -- 172.21.15.97:0/1311597771 >> 172.21.15.97:0/1311597771 conn(0x7f888406a6c0 msgr2=0x7f88840b7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:00.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.636+0000 7f8888fff700 1 -- 172.21.15.97:0/1311597771 shutdown_connections 2023-07-06T03:46:00.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.636+0000 7f8888fff700 1 -- 172.21.15.97:0/1311597771 wait complete. 2023-07-06T03:46:00.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:46:00.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.849+0000 7f7412dff700 1 Processor -- start 2023-07-06T03:46:00.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.850+0000 7f7412dff700 1 -- start start 2023-07-06T03:46:00.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.851+0000 7f7412dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f740c06b360 0x7f740c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.851+0000 7f7412dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f740c0d2210 0x7f740c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.851+0000 7f7412dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c0cff90 con 0x7f740c15c610 2023-07-06T03:46:00.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.851+0000 7f7412dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c0d0780 con 0x7f740c0d2210 2023-07-06T03:46:00.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.851+0000 7f7412dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c15c9e0 con 0x7f740c06b360 2023-07-06T03:46:00.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.851+0000 7f741139c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f740c0d2210 0x7f740c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.851+0000 7f741139c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f740c0d2210 0x7f740c0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54726/0 (socket says 172.21.15.97:54726) 2023-07-06T03:46:00.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.851+0000 7f741139c700 1 -- 172.21.15.97:0/1620906882 learned_addr learned my addr 172.21.15.97:0/1620906882 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:00.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.851+0000 7f740bfff700 1 --2- 172.21.15.97:0/1620906882 >> v2:172.21.15.146:3300/0 conn(0x7f740c06b360 0x7f740c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.852+0000 7f741139c700 1 -- 172.21.15.97:0/1620906882 >> v2:172.21.15.146:3300/0 conn(0x7f740c06b360 msgr2=0x7f740c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:00.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.852+0000 7f741139c700 1 --2- 172.21.15.97:0/1620906882 >> v2:172.21.15.146:3300/0 conn(0x7f740c06b360 0x7f740c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.852+0000 7f741139c700 1 -- 172.21.15.97:0/1620906882 >> v1:172.21.15.146:6789/0 conn(0x7f740c15c610 legacy=0x7f740c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.852+0000 7f741139c700 1 -- 172.21.15.97:0/1620906882 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0b5650 con 0x7f740c0d2210 2023-07-06T03:46:00.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.852+0000 7f740bfff700 1 --2- 172.21.15.97:0/1620906882 >> v2:172.21.15.146:3300/0 conn(0x7f740c06b360 0x7f740c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:00.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.852+0000 7f741139c700 1 --2- 172.21.15.97:0/1620906882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f740c0d2210 0x7f740c0d1310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f740c06a590 tx=0x7f740400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54125638af6d32aa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:00.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.852+0000 7f740b7fe700 1 -- 172.21.15.97:0/1620906882 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400e070 con 0x7f740c0d2210 2023-07-06T03:46:00.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.852+0000 7f740b7fe700 1 -- 172.21.15.97:0/1620906882 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740400b980 con 0x7f740c0d2210 2023-07-06T03:46:00.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.852+0000 7f740b7fe700 1 -- 172.21.15.97:0/1620906882 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400bd30 con 0x7f740c0d2210 2023-07-06T03:46:00.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.853+0000 7f7412dff700 1 -- 172.21.15.97:0/1620906882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f740c0d2210 msgr2=0x7f740c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.853+0000 7f7412dff700 1 --2- 172.21.15.97:0/1620906882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f740c0d2210 0x7f740c0d1310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f740c06a590 tx=0x7f740400b330 comp rx=0 tx=0).stop 2023-07-06T03:46:00.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.853+0000 7f7412dff700 1 -- 172.21.15.97:0/1620906882 shutdown_connections 2023-07-06T03:46:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.853+0000 7f7412dff700 1 --2- 172.21.15.97:0/1620906882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f740c0d2210 0x7f740c0d1310 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.853+0000 7f7412dff700 1 --2- 172.21.15.97:0/1620906882 >> v2:172.21.15.146:3300/0 conn(0x7f740c06b360 0x7f740c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.853+0000 7f7412dff700 1 -- 172.21.15.97:0/1620906882 >> 172.21.15.97:0/1620906882 conn(0x7f740c06a6c0 msgr2=0x7f740c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.853+0000 7f7412dff700 1 -- 172.21.15.97:0/1620906882 shutdown_connections 2023-07-06T03:46:00.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.853+0000 7f7412dff700 1 -- 172.21.15.97:0/1620906882 wait complete. 2023-07-06T03:46:00.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.854+0000 7f7412dff700 1 Processor -- start 2023-07-06T03:46:00.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.855+0000 7f7412dff700 1 -- start start 2023-07-06T03:46:00.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.855+0000 7f7412dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f740c06b360 0x7f740c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.855+0000 7f7412dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f740c0d2210 0x7f740c146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.855+0000 7f7412dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f740c15c610 0x7f740c15f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7412dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c146ff0 con 0x7f740c06b360 2023-07-06T03:46:00.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7412dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c15f9b0 con 0x7f740c15c610 2023-07-06T03:46:00.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7412dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0c7b10 con 0x7f740c0d2210 2023-07-06T03:46:00.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7410b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f740c06b360 0x7f740c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7410b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f740c06b360 0x7f740c0cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54730/0 (socket says 172.21.15.97:54730) 2023-07-06T03:46:00.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7410b9b700 1 -- 172.21.15.97:0/1126322376 learned_addr learned my addr 172.21.15.97:0/1126322376 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:00.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f741139c700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f740c15c610 0x7f740c15f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f740bfff700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f740c0d2210 0x7f740c146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7410b9b700 1 -- 172.21.15.97:0/1126322376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f740c0d2210 msgr2=0x7f740c146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7410b9b700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f740c0d2210 0x7f740c146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7410b9b700 1 -- 172.21.15.97:0/1126322376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f740c15c610 msgr2=0x7f740c15f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:00.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7410b9b700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f740c15c610 0x7f740c15f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:00.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7410b9b700 1 -- 172.21.15.97:0/1126322376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7404009b30 con 0x7f740c06b360 2023-07-06T03:46:00.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f741139c700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f740c15c610 0x7f740c15f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:00.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.856+0000 7f7410b9b700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f740c06b360 0x7f740c0cb620 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f740000c430 tx=0x7f740000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:00.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.857+0000 7f74097fa700 1 -- 172.21.15.97:0/1126322376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74000052d0 con 0x7f740c06b360 2023-07-06T03:46:00.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.857+0000 7f74097fa700 1 -- 172.21.15.97:0/1126322376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7400005450 con 0x7f740c06b360 2023-07-06T03:46:00.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.857+0000 7f74097fa700 1 -- 172.21.15.97:0/1126322376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400006aa0 con 0x7f740c06b360 2023-07-06T03:46:00.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.857+0000 7f7412dff700 1 -- 172.21.15.97:0/1126322376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c0cbd10 con 0x7f740c06b360 2023-07-06T03:46:00.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.857+0000 7f7412dff700 1 -- 172.21.15.97:0/1126322376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c0cc2d0 con 0x7f740c06b360 2023-07-06T03:46:00.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.858+0000 7f7412dff700 1 -- 172.21.15.97:0/1126322376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c04e340 con 0x7f740c06b360 2023-07-06T03:46:00.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.859+0000 7f74097fa700 1 -- 172.21.15.97:0/1126322376 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7400003680 con 0x7f740c06b360 2023-07-06T03:46:00.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.860+0000 7f74097fa700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73f006ace0 0x7f73f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:00.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.860+0000 7f74097fa700 1 -- 172.21.15.97:0/1126322376 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7400088640 con 0x7f740c06b360 2023-07-06T03:46:00.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.860+0000 7f740bfff700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73f006ace0 0x7f73f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:00.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.861+0000 7f740bfff700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73f006ace0 0x7f73f006d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009930 tx=0x7f73f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:00.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:00.864+0000 7f74097fa700 1 -- 172.21.15.97:0/1126322376 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7400053780 con 0x7f740c06b360 2023-07-06T03:46:01.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.077+0000 7f7412dff700 1 -- 172.21.15.97:0/1126322376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f740c0b24f0 con 0x7f740c06b360 2023-07-06T03:46:01.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.077+0000 7f74097fa700 1 -- 172.21.15.97:0/1126322376 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7400056da0 con 0x7f740c06b360 2023-07-06T03:46:01.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.079+0000 7f7412dff700 1 -- 172.21.15.97:0/1126322376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73f006ace0 msgr2=0x7f73f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:01.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.079+0000 7f7412dff700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73f006ace0 0x7f73f006d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009930 tx=0x7f73f8008040 comp rx=0 tx=0).stop 2023-07-06T03:46:01.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.079+0000 7f7412dff700 1 -- 172.21.15.97:0/1126322376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f740c06b360 msgr2=0x7f740c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:01.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.079+0000 7f7412dff700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f740c06b360 0x7f740c0cb620 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f740000c430 tx=0x7f740000c7f0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.079+0000 7f7412dff700 1 -- 172.21.15.97:0/1126322376 shutdown_connections 2023-07-06T03:46:01.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.079+0000 7f7412dff700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73f006ace0 0x7f73f006d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.079+0000 7f7412dff700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f740c15c610 0x7f740c15f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.079+0000 7f7412dff700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f740c06b360 0x7f740c0cb620 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.079+0000 7f7412dff700 1 --2- 172.21.15.97:0/1126322376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f740c0d2210 0x7f740c146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.080+0000 7f7412dff700 1 -- 172.21.15.97:0/1126322376 >> 172.21.15.97:0/1126322376 conn(0x7f740c06a6c0 msgr2=0x7f740c15b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:01.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.080+0000 7f7412dff700 1 -- 172.21.15.97:0/1126322376 shutdown_connections 2023-07-06T03:46:01.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.080+0000 7f7412dff700 1 -- 172.21.15.97:0/1126322376 wait complete. 2023-07-06T03:46:01.084 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:46:01.100 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:46:01.100 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:01.146 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.379+0000 7ff20cdff700 1 Processor -- start 2023-07-06T03:46:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.379+0000 7ff20cdff700 1 -- start start 2023-07-06T03:46:01.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.380+0000 7ff20cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2080cd390 0x7ff208160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.380+0000 7ff20cdff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff208160dd0 0x7ff2080b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.380+0000 7ff20cdff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2080cdae0 con 0x7ff20815c520 2023-07-06T03:46:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.380+0000 7ff20cdff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2080b6640 con 0x7ff208160dd0 2023-07-06T03:46:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.380+0000 7ff20cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080b6920 con 0x7ff2080cd390 2023-07-06T03:46:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.381+0000 7ff206d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff208160dd0 0x7ff2080b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.381+0000 7ff205d9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2080cd390 0x7ff208160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.381+0000 7ff206d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff208160dd0 0x7ff2080b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:40642/0 (socket says 172.21.15.97:40642) 2023-07-06T03:46:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.381+0000 7ff206d9d700 1 -- 172.21.15.97:0/1382672338 learned_addr learned my addr 172.21.15.97:0/1382672338 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.381+0000 7ff206d9d700 1 -- 172.21.15.97:0/1382672338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2080cd390 msgr2=0x7ff208160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.381+0000 7ff206d9d700 1 --2- 172.21.15.97:0/1382672338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2080cd390 0x7ff208160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.381+0000 7ff206d9d700 1 -- 172.21.15.97:0/1382672338 >> v1:172.21.15.146:6789/0 conn(0x7ff20815c520 legacy=0x7ff208160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.381+0000 7ff206d9d700 1 -- 172.21.15.97:0/1382672338 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080b6c00 con 0x7ff208160dd0 2023-07-06T03:46:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.381+0000 7ff205d9b700 1 --2- 172.21.15.97:0/1382672338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2080cd390 0x7ff208160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:46:01.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.382+0000 7ff206d9d700 1 --2- 172.21.15.97:0/1382672338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff208160dd0 0x7ff2080b7400 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff200009e50 tx=0x7ff20000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92fa60ee9bfc72bf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:01.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.382+0000 7ff20559a700 1 -- 172.21.15.97:0/1382672338 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20000e070 con 0x7ff208160dd0 2023-07-06T03:46:01.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.382+0000 7ff20559a700 1 -- 172.21.15.97:0/1382672338 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20000b910 con 0x7ff208160dd0 2023-07-06T03:46:01.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.382+0000 7ff20559a700 1 -- 172.21.15.97:0/1382672338 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20000bcc0 con 0x7ff208160dd0 2023-07-06T03:46:01.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.382+0000 7ff20cdff700 1 -- 172.21.15.97:0/1382672338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff208160dd0 msgr2=0x7ff2080b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:01.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.382+0000 7ff20cdff700 1 --2- 172.21.15.97:0/1382672338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff208160dd0 0x7ff2080b7400 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff200009e50 tx=0x7ff20000b330 comp rx=0 tx=0).stop 2023-07-06T03:46:01.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.383+0000 7ff20cdff700 1 -- 172.21.15.97:0/1382672338 shutdown_connections 2023-07-06T03:46:01.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.383+0000 7ff20cdff700 1 --2- 172.21.15.97:0/1382672338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff208160dd0 0x7ff2080b7400 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.383+0000 7ff20cdff700 1 --2- 172.21.15.97:0/1382672338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2080cd390 0x7ff208160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.383+0000 7ff20cdff700 1 -- 172.21.15.97:0/1382672338 >> 172.21.15.97:0/1382672338 conn(0x7ff208071790 msgr2=0x7ff208071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:01.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.383+0000 7ff20cdff700 1 -- 172.21.15.97:0/1382672338 shutdown_connections 2023-07-06T03:46:01.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.383+0000 7ff20cdff700 1 -- 172.21.15.97:0/1382672338 wait complete. 2023-07-06T03:46:01.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.384+0000 7ff20cdff700 1 Processor -- start 2023-07-06T03:46:01.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.385+0000 7ff20cdff700 1 -- start start 2023-07-06T03:46:01.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.385+0000 7ff20cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2080cd390 0x7ff20814dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.385+0000 7ff20cdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff20815c520 0x7ff2081509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.385+0000 7ff20cdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff208160dd0 0x7ff2080cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff20cdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20814cdc0 con 0x7ff208160dd0 2023-07-06T03:46:01.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff20cdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080cbc00 con 0x7ff20815c520 2023-07-06T03:46:01.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff20cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff208150ee0 con 0x7ff2080cd390 2023-07-06T03:46:01.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff206d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff208160dd0 0x7ff2080cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff206d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff208160dd0 0x7ff2080cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54734/0 (socket says 172.21.15.97:54734) 2023-07-06T03:46:01.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff206d9d700 1 -- 172.21.15.97:0/4152029153 learned_addr learned my addr 172.21.15.97:0/4152029153 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:01.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff20659c700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2080cd390 0x7ff20814dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff205d9b700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff20815c520 0x7ff2081509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff206d9d700 1 -- 172.21.15.97:0/4152029153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2080cd390 msgr2=0x7ff20814dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:01.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff206d9d700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2080cd390 0x7ff20814dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff206d9d700 1 -- 172.21.15.97:0/4152029153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff20815c520 msgr2=0x7ff2081509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:01.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff206d9d700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff20815c520 0x7ff2081509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff206d9d700 1 -- 172.21.15.97:0/4152029153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc0094a0 con 0x7ff208160dd0 2023-07-06T03:46:01.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.386+0000 7ff206d9d700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff208160dd0 0x7ff2080cf010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff20806c240 tx=0x7ff200004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:01.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.387+0000 7ff1f6ffd700 1 -- 172.21.15.97:0/4152029153 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff200004800 con 0x7ff208160dd0 2023-07-06T03:46:01.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.387+0000 7ff1f6ffd700 1 -- 172.21.15.97:0/4152029153 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff200004960 con 0x7ff208160dd0 2023-07-06T03:46:01.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.387+0000 7ff20cdff700 1 -- 172.21.15.97:0/4152029153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff200009b30 con 0x7ff208160dd0 2023-07-06T03:46:01.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.387+0000 7ff1f6ffd700 1 -- 172.21.15.97:0/4152029153 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20001c650 con 0x7ff208160dd0 2023-07-06T03:46:01.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.387+0000 7ff20cdff700 1 -- 172.21.15.97:0/4152029153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2080cb5b0 con 0x7ff208160dd0 2023-07-06T03:46:01.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.388+0000 7ff20cdff700 1 -- 172.21.15.97:0/4152029153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2080cbdb0 con 0x7ff208160dd0 2023-07-06T03:46:01.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.389+0000 7ff1f6ffd700 1 -- 172.21.15.97:0/4152029153 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff20001c7b0 con 0x7ff208160dd0 2023-07-06T03:46:01.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.390+0000 7ff1f6ffd700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1e806ace0 0x7ff1e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.390+0000 7ff1f6ffd700 1 -- 172.21.15.97:0/4152029153 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff20008afb0 con 0x7ff208160dd0 2023-07-06T03:46:01.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.390+0000 7ff20659c700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1e806ace0 0x7ff1e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.391+0000 7ff20659c700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1e806ace0 0x7ff1e806d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8007fd0 tx=0x7ff1f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.394+0000 7ff1f6ffd700 1 -- 172.21.15.97:0/4152029153 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff2080cbdb0 con 0x7ff208160dd0 2023-07-06T03:46:01.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.593+0000 7ff20cdff700 1 -- 172.21.15.97:0/4152029153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2080cd170 con 0x7ff208160dd0 2023-07-06T03:46:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.594+0000 7ff1f6ffd700 1 -- 172.21.15.97:0/4152029153 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7ff200056060 con 0x7ff208160dd0 2023-07-06T03:46:01.597 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:01.597 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:01.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.596+0000 7ff20cdff700 1 -- 172.21.15.97:0/4152029153 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1e806ace0 msgr2=0x7ff1e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:01.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.596+0000 7ff20cdff700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1e806ace0 0x7ff1e806d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8007fd0 tx=0x7ff1f8009040 comp rx=0 tx=0).stop 2023-07-06T03:46:01.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.596+0000 7ff20cdff700 1 -- 172.21.15.97:0/4152029153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff208160dd0 msgr2=0x7ff2080cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.596+0000 7ff20cdff700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff208160dd0 0x7ff2080cf010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff20806c240 tx=0x7ff200004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.596+0000 7ff20cdff700 1 -- 172.21.15.97:0/4152029153 shutdown_connections 2023-07-06T03:46:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.596+0000 7ff20cdff700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1e806ace0 0x7ff1e806d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.596+0000 7ff20cdff700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff208160dd0 0x7ff2080cf010 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.596+0000 7ff20cdff700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2080cd390 0x7ff20814dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.596+0000 7ff20cdff700 1 --2- 172.21.15.97:0/4152029153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff20815c520 0x7ff2081509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.596+0000 7ff20cdff700 1 -- 172.21.15.97:0/4152029153 >> 172.21.15.97:0/4152029153 conn(0x7ff208071790 msgr2=0x7ff2080b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.597+0000 7ff20cdff700 1 -- 172.21.15.97:0/4152029153 shutdown_connections 2023-07-06T03:46:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.597+0000 7ff20cdff700 1 -- 172.21.15.97:0/4152029153 wait complete. 2023-07-06T03:46:01.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:46:01.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:46:01.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.813+0000 7fe58bdff700 1 Processor -- start 2023-07-06T03:46:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.814+0000 7fe58bdff700 1 -- start start 2023-07-06T03:46:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58bdff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe58406b360 0x7fe5840b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58bdff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe5840d2210 0x7fe5840d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58bdff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5840cff90 con 0x7fe58415c610 2023-07-06T03:46:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58bdff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5840d0780 con 0x7fe58406b360 2023-07-06T03:46:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58bdff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58415c9e0 con 0x7fe5840d2210 2023-07-06T03:46:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58939a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe58406b360 0x7fe5840b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58939a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe58406b360 0x7fe5840b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54738/0 (socket says 172.21.15.97:54738) 2023-07-06T03:46:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58939a700 1 -- 172.21.15.97:0/2277579535 learned_addr learned my addr 172.21.15.97:0/2277579535 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58a39c700 1 --2- 172.21.15.97:0/2277579535 >> v2:172.21.15.146:3300/0 conn(0x7fe5840d2210 0x7fe5840d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58939a700 1 -- 172.21.15.97:0/2277579535 >> v2:172.21.15.146:3300/0 conn(0x7fe5840d2210 msgr2=0x7fe5840d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:01.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58939a700 1 --2- 172.21.15.97:0/2277579535 >> v2:172.21.15.146:3300/0 conn(0x7fe5840d2210 0x7fe5840d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.815+0000 7fe58939a700 1 -- 172.21.15.97:0/2277579535 >> v1:172.21.15.146:6789/0 conn(0x7fe58415c610 legacy=0x7fe5840b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:01.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.816+0000 7fe58939a700 1 -- 172.21.15.97:0/2277579535 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5840b5650 con 0x7fe58406b360 2023-07-06T03:46:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.816+0000 7fe58939a700 1 --2- 172.21.15.97:0/2277579535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58406b360 0x7fe5840b4ec0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe5840d2bc0 tx=0x7fe574009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc549a9c3bc0aa1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.816+0000 7fe588b99700 1 -- 172.21.15.97:0/2277579535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574004230 con 0x7fe58406b360 2023-07-06T03:46:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.816+0000 7fe588b99700 1 -- 172.21.15.97:0/2277579535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe574004390 con 0x7fe58406b360 2023-07-06T03:46:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.816+0000 7fe588b99700 1 -- 172.21.15.97:0/2277579535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5740036a0 con 0x7fe58406b360 2023-07-06T03:46:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.816+0000 7fe58bdff700 1 -- 172.21.15.97:0/2277579535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58406b360 msgr2=0x7fe5840b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.816+0000 7fe58bdff700 1 --2- 172.21.15.97:0/2277579535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58406b360 0x7fe5840b4ec0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe5840d2bc0 tx=0x7fe574009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.817+0000 7fe58bdff700 1 -- 172.21.15.97:0/2277579535 shutdown_connections 2023-07-06T03:46:01.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.817+0000 7fe58bdff700 1 --2- 172.21.15.97:0/2277579535 >> v2:172.21.15.146:3300/0 conn(0x7fe5840d2210 0x7fe5840d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.817+0000 7fe58bdff700 1 --2- 172.21.15.97:0/2277579535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58406b360 0x7fe5840b4ec0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.817+0000 7fe58bdff700 1 -- 172.21.15.97:0/2277579535 >> 172.21.15.97:0/2277579535 conn(0x7fe58406a6c0 msgr2=0x7fe58406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:01.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.817+0000 7fe58bdff700 1 -- 172.21.15.97:0/2277579535 shutdown_connections 2023-07-06T03:46:01.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.817+0000 7fe58bdff700 1 -- 172.21.15.97:0/2277579535 wait complete. 2023-07-06T03:46:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.818+0000 7fe58bdff700 1 Processor -- start 2023-07-06T03:46:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.818+0000 7fe58bdff700 1 -- start start 2023-07-06T03:46:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.819+0000 7fe58bdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe58406b360 0x7fe5840c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.819+0000 7fe58bdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840d2210 0x7fe584140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.819+0000 7fe58bdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe58415c610 0x7fe584141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.819+0000 7fe58bdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840c6860 con 0x7fe5840d2210 2023-07-06T03:46:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.819+0000 7fe58bdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840c7070 con 0x7fe58415c610 2023-07-06T03:46:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.819+0000 7fe58bdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840c6ab0 con 0x7fe58406b360 2023-07-06T03:46:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe58939a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840d2210 0x7fe584140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe58939a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840d2210 0x7fe584140e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54752/0 (socket says 172.21.15.97:54752) 2023-07-06T03:46:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe58939a700 1 -- 172.21.15.97:0/3940325457 learned_addr learned my addr 172.21.15.97:0/3940325457 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe58a39c700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe58415c610 0x7fe584141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe589b9b700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe58406b360 0x7fe5840c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe58939a700 1 -- 172.21.15.97:0/3940325457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe58406b360 msgr2=0x7fe5840c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe58939a700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe58406b360 0x7fe5840c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe58939a700 1 -- 172.21.15.97:0/3940325457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe58415c610 msgr2=0x7fe584141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe58939a700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe58415c610 0x7fe584141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe58939a700 1 -- 172.21.15.97:0/3940325457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574009550 con 0x7fe5840d2210 2023-07-06T03:46:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe58a39c700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe58415c610 0x7fe584141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.820+0000 7fe58939a700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840d2210 0x7fe584140e40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe574009870 tx=0x7fe574004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.821+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3940325457 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574003d80 con 0x7fe5840d2210 2023-07-06T03:46:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.821+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3940325457 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57401f070 con 0x7fe5840d2210 2023-07-06T03:46:01.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.821+0000 7fe58bdff700 1 -- 172.21.15.97:0/3940325457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5840c6d90 con 0x7fe5840d2210 2023-07-06T03:46:01.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.821+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3940325457 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574011850 con 0x7fe5840d2210 2023-07-06T03:46:01.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.821+0000 7fe58bdff700 1 -- 172.21.15.97:0/3940325457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5841418c0 con 0x7fe5840d2210 2023-07-06T03:46:01.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.822+0000 7fe58bdff700 1 -- 172.21.15.97:0/3940325457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5840c9380 con 0x7fe5840d2210 2023-07-06T03:46:01.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.823+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3940325457 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe574021030 con 0x7fe5840d2210 2023-07-06T03:46:01.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.824+0000 7fe57a7fc700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 0x7fe56806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:01.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.824+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3940325457 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe57408ac20 con 0x7fe5840d2210 2023-07-06T03:46:01.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.824+0000 7fe589b9b700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 0x7fe56806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:01.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.825+0000 7fe589b9b700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 0x7fe56806d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe580009870 tx=0x7fe580008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:01.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:01.828+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3940325457 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe5840c9380 con 0x7fe5840d2210 2023-07-06T03:46:02.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.023+0000 7fe58bdff700 1 -- 172.21.15.97:0/3940325457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5840c9380 con 0x7fe5840d2210 2023-07-06T03:46:02.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.024+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3940325457 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5840c9380 con 0x7fe5840d2210 2023-07-06T03:46:02.027 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:46:02.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 -- 172.21.15.97:0/3940325457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 msgr2=0x7fe56806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 0x7fe56806d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe580009870 tx=0x7fe580008040 comp rx=0 tx=0).stop 2023-07-06T03:46:02.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 -- 172.21.15.97:0/3940325457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840d2210 msgr2=0x7fe584140e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840d2210 0x7fe584140e40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe574009870 tx=0x7fe574004750 comp rx=0 tx=0).stop 2023-07-06T03:46:02.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 -- 172.21.15.97:0/3940325457 shutdown_connections 2023-07-06T03:46:02.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 0x7fe56806d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe58415c610 0x7fe584141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe58406b360 0x7fe5840c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 --2- 172.21.15.97:0/3940325457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5840d2210 0x7fe584140e40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 -- 172.21.15.97:0/3940325457 >> 172.21.15.97:0/3940325457 conn(0x7fe58406a6c0 msgr2=0x7fe58415ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:02.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 -- 172.21.15.97:0/3940325457 shutdown_connections 2023-07-06T03:46:02.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.026+0000 7fe58bdff700 1 -- 172.21.15.97:0/3940325457 wait complete. 2023-07-06T03:46:02.047 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:46:02.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.243+0000 7f0d1dbff700 1 Processor -- start 2023-07-06T03:46:02.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.243+0000 7f0d1dbff700 1 -- start start 2023-07-06T03:46:02.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.244+0000 7f0d1dbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d1815c610 0x7f0d180b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d1dbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0d180d2210 0x7f0d180d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d1dbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d180cff90 con 0x7f0d1806b360 2023-07-06T03:46:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d1dbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d180d0780 con 0x7f0d1815c610 2023-07-06T03:46:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d1dbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1815c9e0 con 0x7f0d180d2210 2023-07-06T03:46:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d177fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d1815c610 0x7f0d180b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d177fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d1815c610 0x7f0d180b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54758/0 (socket says 172.21.15.97:54758) 2023-07-06T03:46:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d177fe700 1 -- 172.21.15.97:0/4262941590 learned_addr learned my addr 172.21.15.97:0/4262941590 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d0ffff700 1 --1- 172.21.15.97:0/4262941590 >> v1:172.21.15.97:6789/0 conn(0x7f0d1806b360 0x7f0d180b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56738/0 (socket says 172.21.15.97:56738) 2023-07-06T03:46:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d17fff700 1 --2- 172.21.15.97:0/4262941590 >> v2:172.21.15.146:3300/0 conn(0x7f0d180d2210 0x7f0d180d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d177fe700 1 -- 172.21.15.97:0/4262941590 >> v2:172.21.15.146:3300/0 conn(0x7f0d180d2210 msgr2=0x7f0d180d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:02.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d177fe700 1 --2- 172.21.15.97:0/4262941590 >> v2:172.21.15.146:3300/0 conn(0x7f0d180d2210 0x7f0d180d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d177fe700 1 -- 172.21.15.97:0/4262941590 >> v1:172.21.15.97:6789/0 conn(0x7f0d1806b360 legacy=0x7f0d180b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d177fe700 1 -- 172.21.15.97:0/4262941590 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180b5650 con 0x7f0d1815c610 2023-07-06T03:46:02.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d17fff700 1 --2- 172.21.15.97:0/4262941590 >> v2:172.21.15.146:3300/0 conn(0x7f0d180d2210 0x7f0d180d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:02.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.245+0000 7f0d16ffd700 1 -- 172.21.15.97:0/4262941590 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356629329 0 0) 0x7f0d180cff90 con 0x7f0d1806b360 2023-07-06T03:46:02.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d177fe700 1 --2- 172.21.15.97:0/4262941590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1815c610 0x7f0d180b47b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0d180d2bc0 tx=0x7f0d08009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e5ebf450350ed0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:02.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d16ffd700 1 -- 172.21.15.97:0/4262941590 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0800b860 con 0x7f0d1815c610 2023-07-06T03:46:02.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d16ffd700 1 -- 172.21.15.97:0/4262941590 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0800b9c0 con 0x7f0d1815c610 2023-07-06T03:46:02.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d16ffd700 1 -- 172.21.15.97:0/4262941590 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0800bd50 con 0x7f0d1815c610 2023-07-06T03:46:02.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d1dbff700 1 -- 172.21.15.97:0/4262941590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1815c610 msgr2=0x7f0d180b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d1dbff700 1 --2- 172.21.15.97:0/4262941590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1815c610 0x7f0d180b47b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0d180d2bc0 tx=0x7f0d08009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d1dbff700 1 -- 172.21.15.97:0/4262941590 shutdown_connections 2023-07-06T03:46:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d1dbff700 1 --2- 172.21.15.97:0/4262941590 >> v2:172.21.15.146:3300/0 conn(0x7f0d180d2210 0x7f0d180d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d1dbff700 1 --2- 172.21.15.97:0/4262941590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1815c610 0x7f0d180b47b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d1dbff700 1 -- 172.21.15.97:0/4262941590 >> 172.21.15.97:0/4262941590 conn(0x7f0d1806a6c0 msgr2=0x7f0d1806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d1dbff700 1 -- 172.21.15.97:0/4262941590 shutdown_connections 2023-07-06T03:46:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.246+0000 7f0d1dbff700 1 -- 172.21.15.97:0/4262941590 wait complete. 2023-07-06T03:46:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.249+0000 7f0d1dbff700 1 Processor -- start 2023-07-06T03:46:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.249+0000 7f0d1dbff700 1 -- start start 2023-07-06T03:46:02.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.250+0000 7f0d1dbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1806b360 0x7f0d1815da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.250+0000 7f0d1dbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d180d2210 0x7f0d1815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.250+0000 7f0d1dbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d1815c610 0x7f0d181f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.250+0000 7f0d1dbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181f3610 con 0x7f0d1806b360 2023-07-06T03:46:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.250+0000 7f0d1dbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181f38f0 con 0x7f0d180d2210 2023-07-06T03:46:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.250+0000 7f0d1dbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181f3bd0 con 0x7f0d1815c610 2023-07-06T03:46:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.250+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1806b360 0x7f0d1815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.250+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1806b360 0x7f0d1815da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54768/0 (socket says 172.21.15.97:54768) 2023-07-06T03:46:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.250+0000 7f0d177fe700 1 -- 172.21.15.97:0/690029416 learned_addr learned my addr 172.21.15.97:0/690029416 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d0ffff700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d180d2210 0x7f0d1815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d17fff700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d1815c610 0x7f0d181f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d177fe700 1 -- 172.21.15.97:0/690029416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d1815c610 msgr2=0x7f0d181f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d177fe700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d1815c610 0x7f0d181f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d177fe700 1 -- 172.21.15.97:0/690029416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d180d2210 msgr2=0x7f0d1815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d177fe700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d180d2210 0x7f0d1815dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d177fe700 1 -- 172.21.15.97:0/690029416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08009550 con 0x7f0d1806b360 2023-07-06T03:46:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d0ffff700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d180d2210 0x7f0d1815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d177fe700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1806b360 0x7f0d1815da60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d08000c00 tx=0x7f0d08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d14ff9700 1 -- 172.21.15.97:0/690029416 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080036a0 con 0x7f0d1806b360 2023-07-06T03:46:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d14ff9700 1 -- 172.21.15.97:0/690029416 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0801e070 con 0x7f0d1806b360 2023-07-06T03:46:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d1dbff700 1 -- 172.21.15.97:0/690029416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d181f3eb0 con 0x7f0d1806b360 2023-07-06T03:46:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.251+0000 7f0d14ff9700 1 -- 172.21.15.97:0/690029416 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08003a20 con 0x7f0d1806b360 2023-07-06T03:46:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.252+0000 7f0d1dbff700 1 -- 172.21.15.97:0/690029416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d180c6860 con 0x7f0d1806b360 2023-07-06T03:46:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.253+0000 7f0d1dbff700 1 -- 172.21.15.97:0/690029416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1804e340 con 0x7f0d1806b360 2023-07-06T03:46:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.254+0000 7f0d14ff9700 1 -- 172.21.15.97:0/690029416 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d08003b80 con 0x7f0d1806b360 2023-07-06T03:46:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.254+0000 7f0d14ff9700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cf806ace0 0x7f0cf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.255+0000 7f0d14ff9700 1 -- 172.21.15.97:0/690029416 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d08020070 con 0x7f0d1806b360 2023-07-06T03:46:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.255+0000 7f0d0ffff700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cf806ace0 0x7f0cf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.255+0000 7f0d0ffff700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cf806ace0 0x7f0cf806d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0d00000f60 tx=0x7f0d00007cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.259+0000 7f0d14ff9700 1 -- 172.21.15.97:0/690029416 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0d08055ac0 con 0x7f0d1806b360 2023-07-06T03:46:02.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.453+0000 7f0d1dbff700 1 -- 172.21.15.97:0/690029416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0d180cd4f0 con 0x7f0d1806b360 2023-07-06T03:46:02.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.454+0000 7f0d14ff9700 1 -- 172.21.15.97:0/690029416 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0d08015020 con 0x7f0d1806b360 2023-07-06T03:46:02.457 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:46:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 -- 172.21.15.97:0/690029416 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cf806ace0 msgr2=0x7f0cf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cf806ace0 0x7f0cf806d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0d00000f60 tx=0x7f0d00007cc0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 -- 172.21.15.97:0/690029416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1806b360 msgr2=0x7f0d1815da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1806b360 0x7f0d1815da60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d08000c00 tx=0x7f0d08004750 comp rx=0 tx=0).stop 2023-07-06T03:46:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 -- 172.21.15.97:0/690029416 shutdown_connections 2023-07-06T03:46:02.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cf806ace0 0x7f0cf806d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d1815c610 0x7f0d181f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d1806b360 0x7f0d1815da60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 --2- 172.21.15.97:0/690029416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d180d2210 0x7f0d1815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 -- 172.21.15.97:0/690029416 >> 172.21.15.97:0/690029416 conn(0x7f0d1806a6c0 msgr2=0x7f0d180b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 -- 172.21.15.97:0/690029416 shutdown_connections 2023-07-06T03:46:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.456+0000 7f0d1dbff700 1 -- 172.21.15.97:0/690029416 wait complete. 2023-07-06T03:46:02.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:46:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.671+0000 7ff23a9ff700 1 Processor -- start 2023-07-06T03:46:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.671+0000 7ff23a9ff700 1 -- start start 2023-07-06T03:46:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.672+0000 7ff23a9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff23415c610 0x7ff2340b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.672+0000 7ff23a9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff2340d2210 0x7ff2340d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.672+0000 7ff23a9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2340cff90 con 0x7ff23406b360 2023-07-06T03:46:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.672+0000 7ff23a9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2340d0780 con 0x7ff23415c610 2023-07-06T03:46:02.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.672+0000 7ff23a9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23415c9e0 con 0x7ff2340d2210 2023-07-06T03:46:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.672+0000 7ff233fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff23415c610 0x7ff2340b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.672+0000 7ff233fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff23415c610 0x7ff2340b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54772/0 (socket says 172.21.15.97:54772) 2023-07-06T03:46:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.672+0000 7ff233fff700 1 -- 172.21.15.97:0/1280282034 learned_addr learned my addr 172.21.15.97:0/1280282034 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.673+0000 7ff238f9c700 1 --2- 172.21.15.97:0/1280282034 >> v2:172.21.15.146:3300/0 conn(0x7ff2340d2210 0x7ff2340d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.673+0000 7ff233fff700 1 -- 172.21.15.97:0/1280282034 >> v2:172.21.15.146:3300/0 conn(0x7ff2340d2210 msgr2=0x7ff2340d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.673+0000 7ff233fff700 1 --2- 172.21.15.97:0/1280282034 >> v2:172.21.15.146:3300/0 conn(0x7ff2340d2210 0x7ff2340d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.673+0000 7ff233fff700 1 -- 172.21.15.97:0/1280282034 >> v1:172.21.15.97:6789/0 conn(0x7ff23406b360 legacy=0x7ff2340b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.673+0000 7ff233fff700 1 -- 172.21.15.97:0/1280282034 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2340b5650 con 0x7ff23415c610 2023-07-06T03:46:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.673+0000 7ff233fff700 1 --2- 172.21.15.97:0/1280282034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff23415c610 0x7ff2340b47b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff2340d2bc0 tx=0x7ff224009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f0c242ed0127eb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.673+0000 7ff232ffd700 1 -- 172.21.15.97:0/1280282034 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22400b860 con 0x7ff23415c610 2023-07-06T03:46:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.673+0000 7ff232ffd700 1 -- 172.21.15.97:0/1280282034 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22400b9c0 con 0x7ff23415c610 2023-07-06T03:46:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.673+0000 7ff232ffd700 1 -- 172.21.15.97:0/1280282034 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22400bd50 con 0x7ff23415c610 2023-07-06T03:46:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.674+0000 7ff23a9ff700 1 -- 172.21.15.97:0/1280282034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff23415c610 msgr2=0x7ff2340b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.674+0000 7ff23a9ff700 1 --2- 172.21.15.97:0/1280282034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff23415c610 0x7ff2340b47b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff2340d2bc0 tx=0x7ff224009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.674+0000 7ff23a9ff700 1 -- 172.21.15.97:0/1280282034 shutdown_connections 2023-07-06T03:46:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.674+0000 7ff23a9ff700 1 --2- 172.21.15.97:0/1280282034 >> v2:172.21.15.146:3300/0 conn(0x7ff2340d2210 0x7ff2340d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.674+0000 7ff23a9ff700 1 --2- 172.21.15.97:0/1280282034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff23415c610 0x7ff2340b47b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.674+0000 7ff23a9ff700 1 -- 172.21.15.97:0/1280282034 >> 172.21.15.97:0/1280282034 conn(0x7ff23406a6c0 msgr2=0x7ff23406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.674+0000 7ff23a9ff700 1 -- 172.21.15.97:0/1280282034 shutdown_connections 2023-07-06T03:46:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.674+0000 7ff23a9ff700 1 -- 172.21.15.97:0/1280282034 wait complete. 2023-07-06T03:46:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.675+0000 7ff23a9ff700 1 Processor -- start 2023-07-06T03:46:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.676+0000 7ff23a9ff700 1 -- start start 2023-07-06T03:46:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.676+0000 7ff23a9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff23406b360 0x7ff23415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.676+0000 7ff23a9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2340d2210 0x7ff23415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff23a9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff23415c610 0x7ff23415f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff23a9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2340c7750 con 0x7ff23406b360 2023-07-06T03:46:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff23a9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2340c96a0 con 0x7ff2340d2210 2023-07-06T03:46:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff23a9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23415d200 con 0x7ff23415c610 2023-07-06T03:46:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff233fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff23406b360 0x7ff23415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff233fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff23406b360 0x7ff23415cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54782/0 (socket says 172.21.15.97:54782) 2023-07-06T03:46:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff233fff700 1 -- 172.21.15.97:0/3294812261 learned_addr learned my addr 172.21.15.97:0/3294812261 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff238f9c700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff23415c610 0x7ff23415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff2337fe700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2340d2210 0x7ff23415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff233fff700 1 -- 172.21.15.97:0/3294812261 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff23415c610 msgr2=0x7ff23415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff233fff700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff23415c610 0x7ff23415f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff233fff700 1 -- 172.21.15.97:0/3294812261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2340d2210 msgr2=0x7ff23415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff233fff700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2340d2210 0x7ff23415eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff233fff700 1 -- 172.21.15.97:0/3294812261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224009550 con 0x7ff23406b360 2023-07-06T03:46:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.677+0000 7ff233fff700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff23406b360 0x7ff23415cb20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff224009130 tx=0x7ff224004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.678+0000 7ff230ff9700 1 -- 172.21.15.97:0/3294812261 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2240045d0 con 0x7ff23406b360 2023-07-06T03:46:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.678+0000 7ff230ff9700 1 -- 172.21.15.97:0/3294812261 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22401e070 con 0x7ff23406b360 2023-07-06T03:46:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.678+0000 7ff23a9ff700 1 -- 172.21.15.97:0/3294812261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2340c6db0 con 0x7ff23406b360 2023-07-06T03:46:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.678+0000 7ff230ff9700 1 -- 172.21.15.97:0/3294812261 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2240038a0 con 0x7ff23406b360 2023-07-06T03:46:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.678+0000 7ff23a9ff700 1 -- 172.21.15.97:0/3294812261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2340c7370 con 0x7ff23406b360 2023-07-06T03:46:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.679+0000 7ff23a9ff700 1 -- 172.21.15.97:0/3294812261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23404e340 con 0x7ff23406b360 2023-07-06T03:46:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.680+0000 7ff230ff9700 1 -- 172.21.15.97:0/3294812261 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff224003a00 con 0x7ff23406b360 2023-07-06T03:46:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.681+0000 7ff230ff9700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff22006ace0 0x7ff22006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.681+0000 7ff230ff9700 1 -- 172.21.15.97:0/3294812261 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff224020070 con 0x7ff23406b360 2023-07-06T03:46:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.681+0000 7ff2337fe700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff22006ace0 0x7ff22006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:02.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.681+0000 7ff2337fe700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff22006ace0 0x7ff22006d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff228000f60 tx=0x7ff228009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:02.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.685+0000 7ff230ff9700 1 -- 172.21.15.97:0/3294812261 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff224055a40 con 0x7ff23406b360 2023-07-06T03:46:02.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.880+0000 7ff23a9ff700 1 -- 172.21.15.97:0/3294812261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2340cd4f0 con 0x7ff23406b360 2023-07-06T03:46:02.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.881+0000 7ff230ff9700 1 -- 172.21.15.97:0/3294812261 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff224003cb0 con 0x7ff23406b360 2023-07-06T03:46:02.884 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:46:02.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 -- 172.21.15.97:0/3294812261 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff22006ace0 msgr2=0x7ff22006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff22006ace0 0x7ff22006d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff228000f60 tx=0x7ff228009040 comp rx=0 tx=0).stop 2023-07-06T03:46:02.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 -- 172.21.15.97:0/3294812261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff23406b360 msgr2=0x7ff23415cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:02.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff23406b360 0x7ff23415cb20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff224009130 tx=0x7ff224004750 comp rx=0 tx=0).stop 2023-07-06T03:46:02.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 -- 172.21.15.97:0/3294812261 shutdown_connections 2023-07-06T03:46:02.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff22006ace0 0x7ff22006d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff23415c610 0x7ff23415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff23406b360 0x7ff23415cb20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 --2- 172.21.15.97:0/3294812261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2340d2210 0x7ff23415eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:02.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 -- 172.21.15.97:0/3294812261 >> 172.21.15.97:0/3294812261 conn(0x7ff23406a6c0 msgr2=0x7ff2340b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:02.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 -- 172.21.15.97:0/3294812261 shutdown_connections 2023-07-06T03:46:02.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:02.883+0000 7ff23a9ff700 1 -- 172.21.15.97:0/3294812261 wait complete. 2023-07-06T03:46:02.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:46:03.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.099+0000 7fd07d3ff700 1 Processor -- start 2023-07-06T03:46:03.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.099+0000 7fd07d3ff700 1 -- start start 2023-07-06T03:46:03.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.100+0000 7fd07d3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0780d2210 0x7fd0780b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.100+0000 7fd07d3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0780cff90 con 0x7fd07815c610 2023-07-06T03:46:03.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.100+0000 7fd07d3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0780b5650 con 0x7fd07806b360 2023-07-06T03:46:03.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.100+0000 7fd07d3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780d0780 con 0x7fd0780d2210 2023-07-06T03:46:03.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.100+0000 7fd076ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd07815c610 0x7fd0780b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56764/0 (socket says 172.21.15.97:56764) 2023-07-06T03:46:03.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd076ffd700 1 -- 172.21.15.97:0/1455959887 learned_addr learned my addr 172.21.15.97:0/1455959887 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:03.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841112350 0 0) 0x7fd0780cff90 con 0x7fd07815c610 2023-07-06T03:46:03.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0600035b0 con 0x7fd07815c610 2023-07-06T03:46:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd0777fe700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0780d2210 0x7fd0780b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 637513635 0 0) 0x7fd0600035b0 con 0x7fd07815c610 2023-07-06T03:46:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0780d2210 msgr2=0x7fd0780b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0780d2210 0x7fd0780b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 >> v1:172.21.15.146:6789/0 conn(0x7fd07806b360 legacy=0x7fd0780d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780b5400 con 0x7fd07815c610 2023-07-06T03:46:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fd068001670 con 0x7fd07815c610 2023-07-06T03:46:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 >> v1:172.21.15.97:6789/0 conn(0x7fd07815c610 legacy=0x7fd0780b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd060003a30 0x7fd060003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0780d2210 0x7fd060004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd060004d90 0x7fd060005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780d0780 con 0x7fd060004d90 2023-07-06T03:46:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06000bac0 con 0x7fd060003a30 2023-07-06T03:46:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.101+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06000bc00 con 0x7fd0780d2210 2023-07-06T03:46:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.102+0000 7fd0777fe700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd060004d90 0x7fd060005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.102+0000 7fd0767fc700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0780d2210 0x7fd060004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.102+0000 7fd076ffd700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd060003a30 0x7fd060003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.102+0000 7fd0777fe700 1 -- 172.21.15.97:0/1455959887 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0780d2210 msgr2=0x7fd060004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:03.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.102+0000 7fd0777fe700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0780d2210 0x7fd060004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.102+0000 7fd0777fe700 1 -- 172.21.15.97:0/1455959887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd060003a30 msgr2=0x7fd060003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:03.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.102+0000 7fd0777fe700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd060003a30 0x7fd060003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.102+0000 7fd0777fe700 1 -- 172.21.15.97:0/1455959887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0600035b0 con 0x7fd060004d90 2023-07-06T03:46:03.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.102+0000 7fd0777fe700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd060004d90 0x7fd060005200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd070009fc0 tx=0x7fd060005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf4eae99f0516ba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:03.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.103+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07000ec30 con 0x7fd060004d90 2023-07-06T03:46:03.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.103+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07000f040 con 0x7fd060004d90 2023-07-06T03:46:03.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.103+0000 7fd075ffb700 1 -- 172.21.15.97:0/1455959887 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070014480 con 0x7fd060004d90 2023-07-06T03:46:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.103+0000 7fd07d3ff700 1 -- 172.21.15.97:0/1455959887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd060004d90 msgr2=0x7fd060005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.103+0000 7fd07d3ff700 1 --2- 172.21.15.97:0/1455959887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd060004d90 0x7fd060005200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd070009fc0 tx=0x7fd060005880 comp rx=0 tx=0).stop 2023-07-06T03:46:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.103+0000 7fd0777fe700 1 -- 172.21.15.97:0/1455959887 reap_dead start 2023-07-06T03:46:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.103+0000 7fd07d3ff700 1 -- 172.21.15.97:0/1455959887 shutdown_connections 2023-07-06T03:46:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.103+0000 7fd07d3ff700 1 -- 172.21.15.97:0/1455959887 >> 172.21.15.97:0/1455959887 conn(0x7fd07806a6c0 msgr2=0x7fd07806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.103+0000 7fd07d3ff700 1 -- 172.21.15.97:0/1455959887 shutdown_connections 2023-07-06T03:46:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.103+0000 7fd07d3ff700 1 -- 172.21.15.97:0/1455959887 wait complete. 2023-07-06T03:46:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.105+0000 7fd07d3ff700 1 Processor -- start 2023-07-06T03:46:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.105+0000 7fd07d3ff700 1 -- start start 2023-07-06T03:46:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.106+0000 7fd07d3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd060004d90 0x7fd07815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.106+0000 7fd07d3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078157160 0x7fd078157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.106+0000 7fd07d3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd078157dc0 0x7fd07815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.106+0000 7fd07d3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780c9e00 con 0x7fd078157160 2023-07-06T03:46:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.106+0000 7fd07d3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780c9290 con 0x7fd078157dc0 2023-07-06T03:46:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.106+0000 7fd07d3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780c9530 con 0x7fd060004d90 2023-07-06T03:46:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.106+0000 7fd0767fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078157160 0x7fd078157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd0767fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078157160 0x7fd078157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54790/0 (socket says 172.21.15.97:54790) 2023-07-06T03:46:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd0767fc700 1 -- 172.21.15.97:0/2021221615 learned_addr learned my addr 172.21.15.97:0/2021221615 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd076ffd700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd060004d90 0x7fd07815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd0777fe700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd078157dc0 0x7fd07815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd0767fc700 1 -- 172.21.15.97:0/2021221615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd060004d90 msgr2=0x7fd07815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd0767fc700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd060004d90 0x7fd07815cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd0767fc700 1 -- 172.21.15.97:0/2021221615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd078157dc0 msgr2=0x7fd07815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd0767fc700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd078157dc0 0x7fd07815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd0767fc700 1 -- 172.21.15.97:0/2021221615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780ca050 con 0x7fd078157160 2023-07-06T03:46:03.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd0767fc700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078157160 0x7fd078157880 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00f770 tx=0x7fd06c00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:03.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd05f7fe700 1 -- 172.21.15.97:0/2021221615 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c001460 con 0x7fd078157160 2023-07-06T03:46:03.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd05f7fe700 1 -- 172.21.15.97:0/2021221615 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c0052e0 con 0x7fd078157160 2023-07-06T03:46:03.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd07d3ff700 1 -- 172.21.15.97:0/2021221615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0780ca330 con 0x7fd078157160 2023-07-06T03:46:03.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.107+0000 7fd05f7fe700 1 -- 172.21.15.97:0/2021221615 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c003680 con 0x7fd078157160 2023-07-06T03:46:03.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.108+0000 7fd07d3ff700 1 -- 172.21.15.97:0/2021221615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0780cc470 con 0x7fd078157160 2023-07-06T03:46:03.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.109+0000 7fd07d3ff700 1 -- 172.21.15.97:0/2021221615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0780ca540 con 0x7fd078157160 2023-07-06T03:46:03.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.109+0000 7fd05f7fe700 1 -- 172.21.15.97:0/2021221615 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd06c0037e0 con 0x7fd078157160 2023-07-06T03:46:03.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.110+0000 7fd05f7fe700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd06406ace0 0x7fd06406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.110+0000 7fd05f7fe700 1 -- 172.21.15.97:0/2021221615 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd06c089290 con 0x7fd078157160 2023-07-06T03:46:03.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.111+0000 7fd076ffd700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd06406ace0 0x7fd06406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.111+0000 7fd076ffd700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd06406ace0 0x7fd06406d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd0680017a0 tx=0x7fd068001690 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:03.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.115+0000 7fd05f7fe700 1 -- 172.21.15.97:0/2021221615 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd06c0543e0 con 0x7fd078157160 2023-07-06T03:46:03.124 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:46:03.128 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:46:03.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.317+0000 7fd07d3ff700 1 -- 172.21.15.97:0/2021221615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd07804e340 con 0x7fd078157160 2023-07-06T03:46:03.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.318+0000 7fd05f7fe700 1 -- 172.21.15.97:0/2021221615 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd06c057a00 con 0x7fd078157160 2023-07-06T03:46:03.323 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:46:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 -- 172.21.15.97:0/2021221615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd06406ace0 msgr2=0x7fd06406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd06406ace0 0x7fd06406d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd0680017a0 tx=0x7fd068001690 comp rx=0 tx=0).stop 2023-07-06T03:46:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 -- 172.21.15.97:0/2021221615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078157160 msgr2=0x7fd078157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078157160 0x7fd078157880 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00f770 tx=0x7fd06c00fa80 comp rx=0 tx=0).stop 2023-07-06T03:46:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 -- 172.21.15.97:0/2021221615 shutdown_connections 2023-07-06T03:46:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd06406ace0 0x7fd06406d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd078157dc0 0x7fd07815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd060004d90 0x7fd07815cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 --2- 172.21.15.97:0/2021221615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd078157160 0x7fd078157880 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 -- 172.21.15.97:0/2021221615 >> 172.21.15.97:0/2021221615 conn(0x7fd07806a6c0 msgr2=0x7fd0780d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 -- 172.21.15.97:0/2021221615 shutdown_connections 2023-07-06T03:46:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.320+0000 7fd07d3ff700 1 -- 172.21.15.97:0/2021221615 wait complete. 2023-07-06T03:46:03.341 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:46:03.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.534+0000 7f443b9ff700 1 Processor -- start 2023-07-06T03:46:03.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.534+0000 7f443b9ff700 1 -- start start 2023-07-06T03:46:03.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f443b9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f443406cab0 0x7f44340ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f443b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44340b0560 0x7f44340b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f443b9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44340ce560 con 0x7f4434158b20 2023-07-06T03:46:03.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f443b9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44340ce820 con 0x7f443406cab0 2023-07-06T03:46:03.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f443b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44340b0f20 con 0x7f44340b0560 2023-07-06T03:46:03.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f4438f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f443406cab0 0x7f44340ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f4438f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f443406cab0 0x7f44340ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54796/0 (socket says 172.21.15.97:54796) 2023-07-06T03:46:03.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f4438f9a700 1 -- 172.21.15.97:0/1790891369 learned_addr learned my addr 172.21.15.97:0/1790891369 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:03.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f4438f9a700 1 -- 172.21.15.97:0/1790891369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44340b0560 msgr2=0x7f44340b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:03.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f4439f9c700 1 --2- 172.21.15.97:0/1790891369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44340b0560 0x7f44340b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f4438f9a700 1 --2- 172.21.15.97:0/1790891369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44340b0560 0x7f44340b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.536+0000 7f4438f9a700 1 -- 172.21.15.97:0/1790891369 >> v1:172.21.15.146:6789/0 conn(0x7f4434158b20 legacy=0x7f44340cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:03.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.537+0000 7f4438f9a700 1 -- 172.21.15.97:0/1790891369 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44340adcf0 con 0x7f443406cab0 2023-07-06T03:46:03.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.537+0000 7f4439f9c700 1 --2- 172.21.15.97:0/1790891369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44340b0560 0x7f44340b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:03.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.537+0000 7f4438f9a700 1 --2- 172.21.15.97:0/1790891369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443406cab0 0x7f44340ce020 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f44240098f0 tx=0x7f4424009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95886e1a09c98671 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:03.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.537+0000 7f442bfff700 1 -- 172.21.15.97:0/1790891369 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424004230 con 0x7f443406cab0 2023-07-06T03:46:03.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.537+0000 7f442bfff700 1 -- 172.21.15.97:0/1790891369 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4424004390 con 0x7f443406cab0 2023-07-06T03:46:03.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.537+0000 7f442bfff700 1 -- 172.21.15.97:0/1790891369 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44240036a0 con 0x7f443406cab0 2023-07-06T03:46:03.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.537+0000 7f443b9ff700 1 -- 172.21.15.97:0/1790891369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443406cab0 msgr2=0x7f44340ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.537+0000 7f443b9ff700 1 --2- 172.21.15.97:0/1790891369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443406cab0 0x7f44340ce020 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f44240098f0 tx=0x7f4424009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:03.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.538+0000 7f443b9ff700 1 -- 172.21.15.97:0/1790891369 shutdown_connections 2023-07-06T03:46:03.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.538+0000 7f443b9ff700 1 --2- 172.21.15.97:0/1790891369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44340b0560 0x7f44340b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.538+0000 7f443b9ff700 1 --2- 172.21.15.97:0/1790891369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443406cab0 0x7f44340ce020 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.538+0000 7f443b9ff700 1 -- 172.21.15.97:0/1790891369 >> 172.21.15.97:0/1790891369 conn(0x7f443406a060 msgr2=0x7f443406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:03.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.538+0000 7f443b9ff700 1 -- 172.21.15.97:0/1790891369 shutdown_connections 2023-07-06T03:46:03.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.538+0000 7f443b9ff700 1 -- 172.21.15.97:0/1790891369 wait complete. 2023-07-06T03:46:03.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.539+0000 7f443b9ff700 1 Processor -- start 2023-07-06T03:46:03.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.540+0000 7f443b9ff700 1 -- start start 2023-07-06T03:46:03.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.540+0000 7f443b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f443406cab0 0x7f4434151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.540+0000 7f443b9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44340b0560 0x7f443415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.540+0000 7f443b9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4434158b20 0x7f44341598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.540+0000 7f443b9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4434151da0 con 0x7f44340b0560 2023-07-06T03:46:03.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.540+0000 7f443b9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44340ae520 con 0x7f4434158b20 2023-07-06T03:46:03.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.540+0000 7f443b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4434153570 con 0x7f443406cab0 2023-07-06T03:46:03.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f4438f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44340b0560 0x7f443415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f4438f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44340b0560 0x7f443415b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54798/0 (socket says 172.21.15.97:54798) 2023-07-06T03:46:03.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f4438f9a700 1 -- 172.21.15.97:0/764326394 learned_addr learned my addr 172.21.15.97:0/764326394 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:03.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f443979b700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f443406cab0 0x7f4434151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f4439f9c700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4434158b20 0x7f44341598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f4438f9a700 1 -- 172.21.15.97:0/764326394 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f443406cab0 msgr2=0x7f4434151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f4438f9a700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f443406cab0 0x7f4434151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f4438f9a700 1 -- 172.21.15.97:0/764326394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4434158b20 msgr2=0x7f44341598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f4438f9a700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4434158b20 0x7f44341598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f4438f9a700 1 -- 172.21.15.97:0/764326394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4424009550 con 0x7f44340b0560 2023-07-06T03:46:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f4438f9a700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44340b0560 0x7f443415b780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4424009130 tx=0x7f4424004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.541+0000 7f4429ffb700 1 -- 172.21.15.97:0/764326394 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400b660 con 0x7f44340b0560 2023-07-06T03:46:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.542+0000 7f4429ffb700 1 -- 172.21.15.97:0/764326394 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442400b7c0 con 0x7f44340b0560 2023-07-06T03:46:03.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.542+0000 7f443b9ff700 1 -- 172.21.15.97:0/764326394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4434153850 con 0x7f44340b0560 2023-07-06T03:46:03.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.542+0000 7f4429ffb700 1 -- 172.21.15.97:0/764326394 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400bb70 con 0x7f44340b0560 2023-07-06T03:46:03.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.542+0000 7f443b9ff700 1 -- 172.21.15.97:0/764326394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4434153e10 con 0x7f44340b0560 2023-07-06T03:46:03.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.543+0000 7f443b9ff700 1 -- 172.21.15.97:0/764326394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44340c4af0 con 0x7f44340b0560 2023-07-06T03:46:03.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.544+0000 7f4429ffb700 1 -- 172.21.15.97:0/764326394 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f442400bcd0 con 0x7f44340b0560 2023-07-06T03:46:03.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.545+0000 7f4429ffb700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f441806ace0 0x7f441806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:03.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.545+0000 7f4429ffb700 1 -- 172.21.15.97:0/764326394 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f442408adc0 con 0x7f44340b0560 2023-07-06T03:46:03.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.545+0000 7f443979b700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f441806ace0 0x7f441806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:03.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.546+0000 7f443979b700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f441806ace0 0x7f441806d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4430007fd0 tx=0x7f4430007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:03.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.549+0000 7f4429ffb700 1 -- 172.21.15.97:0/764326394 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4424055dd0 con 0x7f44340b0560 2023-07-06T03:46:03.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.763+0000 7f443b9ff700 1 -- 172.21.15.97:0/764326394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4434071250 con 0x7f44340b0560 2023-07-06T03:46:03.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.763+0000 7f4429ffb700 1 -- 172.21.15.97:0/764326394 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f44240593f0 con 0x7f44340b0560 2023-07-06T03:46:03.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 -- 172.21.15.97:0/764326394 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f441806ace0 msgr2=0x7f441806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f441806ace0 0x7f441806d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4430007fd0 tx=0x7f4430007ec0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 -- 172.21.15.97:0/764326394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44340b0560 msgr2=0x7f443415b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:03.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44340b0560 0x7f443415b780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4424009130 tx=0x7f4424004750 comp rx=0 tx=0).stop 2023-07-06T03:46:03.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 -- 172.21.15.97:0/764326394 shutdown_connections 2023-07-06T03:46:03.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f441806ace0 0x7f441806d190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4434158b20 0x7f44341598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f443406cab0 0x7f4434151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 --2- 172.21.15.97:0/764326394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44340b0560 0x7f443415b780 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:03.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 -- 172.21.15.97:0/764326394 >> 172.21.15.97:0/764326394 conn(0x7f443406a060 msgr2=0x7f44340c42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:03.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 -- 172.21.15.97:0/764326394 shutdown_connections 2023-07-06T03:46:03.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:03.766+0000 7f443b9ff700 1 -- 172.21.15.97:0/764326394 wait complete. 2023-07-06T03:46:03.771 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:46:03.786 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:46:03.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:03.787 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:46:03.856 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:46:03.857 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:46:03.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:03.858 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:46:03.906 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:46:03.907 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:03.958 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:46:03.958 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:04.006 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.199+0000 7fe99efff700 1 Processor -- start 2023-07-06T03:46:04.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.199+0000 7fe99efff700 1 -- start start 2023-07-06T03:46:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.200+0000 7fe99efff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9980cd390 0x7fe998160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.200+0000 7fe99efff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe998160dd0 0x7fe9980b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.200+0000 7fe99efff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9980cdae0 con 0x7fe99815c520 2023-07-06T03:46:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.200+0000 7fe99efff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9980b6640 con 0x7fe998160dd0 2023-07-06T03:46:04.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.200+0000 7fe99efff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980b6920 con 0x7fe9980cd390 2023-07-06T03:46:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.200+0000 7fe99cd9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe99815c520 0x7fe998160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56778/0 (socket says 172.21.15.97:56778) 2023-07-06T03:46:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.200+0000 7fe99cd9b700 1 -- 172.21.15.97:0/1032191020 learned_addr learned my addr 172.21.15.97:0/1032191020 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.200+0000 7fe99d59c700 1 --2- 172.21.15.97:0/1032191020 >> v2:172.21.15.97:3300/0 conn(0x7fe998160dd0 0x7fe9980b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.201+0000 7fe997fff700 1 --2- 172.21.15.97:0/1032191020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9980cd390 0x7fe998160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.201+0000 7fe9977fe700 1 -- 172.21.15.97:0/1032191020 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919998770 0 0) 0x7fe9980cdae0 con 0x7fe99815c520 2023-07-06T03:46:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.201+0000 7fe9977fe700 1 -- 172.21.15.97:0/1032191020 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9880035b0 con 0x7fe99815c520 2023-07-06T03:46:04.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.201+0000 7fe99d59c700 1 -- 172.21.15.97:0/1032191020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9980cd390 msgr2=0x7fe998160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.201+0000 7fe99d59c700 1 --2- 172.21.15.97:0/1032191020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9980cd390 0x7fe998160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.201+0000 7fe99d59c700 1 -- 172.21.15.97:0/1032191020 >> v1:172.21.15.97:6789/0 conn(0x7fe99815c520 legacy=0x7fe998160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.201+0000 7fe99d59c700 1 -- 172.21.15.97:0/1032191020 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9980b6c00 con 0x7fe998160dd0 2023-07-06T03:46:04.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.201+0000 7fe99d59c700 1 --2- 172.21.15.97:0/1032191020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe998160dd0 0x7fe9980b7400 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe990009d90 tx=0x7fe99000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc2ff726f193291 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.201+0000 7fe9977fe700 1 -- 172.21.15.97:0/1032191020 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99000e070 con 0x7fe998160dd0 2023-07-06T03:46:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.201+0000 7fe9977fe700 1 -- 172.21.15.97:0/1032191020 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99000bb20 con 0x7fe998160dd0 2023-07-06T03:46:04.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.201+0000 7fe9977fe700 1 -- 172.21.15.97:0/1032191020 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990012430 con 0x7fe998160dd0 2023-07-06T03:46:04.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.202+0000 7fe99efff700 1 -- 172.21.15.97:0/1032191020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe998160dd0 msgr2=0x7fe9980b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:04.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.202+0000 7fe99efff700 1 --2- 172.21.15.97:0/1032191020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe998160dd0 0x7fe9980b7400 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe990009d90 tx=0x7fe99000b330 comp rx=0 tx=0).stop 2023-07-06T03:46:04.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.202+0000 7fe99efff700 1 -- 172.21.15.97:0/1032191020 shutdown_connections 2023-07-06T03:46:04.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.202+0000 7fe99efff700 1 --2- 172.21.15.97:0/1032191020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe998160dd0 0x7fe9980b7400 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.202+0000 7fe99efff700 1 --2- 172.21.15.97:0/1032191020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9980cd390 0x7fe998160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.202+0000 7fe99efff700 1 -- 172.21.15.97:0/1032191020 >> 172.21.15.97:0/1032191020 conn(0x7fe998071790 msgr2=0x7fe998071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:04.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.202+0000 7fe99efff700 1 -- 172.21.15.97:0/1032191020 shutdown_connections 2023-07-06T03:46:04.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.202+0000 7fe99efff700 1 -- 172.21.15.97:0/1032191020 wait complete. 2023-07-06T03:46:04.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.204+0000 7fe99efff700 1 Processor -- start 2023-07-06T03:46:04.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.204+0000 7fe99efff700 1 -- start start 2023-07-06T03:46:04.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.205+0000 7fe99efff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9980cd390 0x7fe99815f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.205+0000 7fe99efff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe99815c520 0x7fe9980cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.205+0000 7fe99efff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe998160dd0 0x7fe9980cf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.205+0000 7fe99efff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980d0c40 con 0x7fe99815c520 2023-07-06T03:46:04.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.205+0000 7fe99efff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980d0f20 con 0x7fe998160dd0 2023-07-06T03:46:04.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.205+0000 7fe99efff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980d1200 con 0x7fe9980cd390 2023-07-06T03:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.205+0000 7fe997fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe99815c520 0x7fe9980cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.205+0000 7fe997fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe99815c520 0x7fe9980cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54818/0 (socket says 172.21.15.97:54818) 2023-07-06T03:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.205+0000 7fe997fff700 1 -- 172.21.15.97:0/3603191445 learned_addr learned my addr 172.21.15.97:0/3603191445 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe99cd9b700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9980cd390 0x7fe99815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe99d59c700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe998160dd0 0x7fe9980cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe997fff700 1 -- 172.21.15.97:0/3603191445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9980cd390 msgr2=0x7fe99815f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:04.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe997fff700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9980cd390 0x7fe99815f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe997fff700 1 -- 172.21.15.97:0/3603191445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe998160dd0 msgr2=0x7fe9980cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:04.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe997fff700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe998160dd0 0x7fe9980cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe997fff700 1 -- 172.21.15.97:0/3603191445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe990009a70 con 0x7fe99815c520 2023-07-06T03:46:04.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe99cd9b700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9980cd390 0x7fe99815f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:04.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe99d59c700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe998160dd0 0x7fe9980cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:04.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe997fff700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe99815c520 0x7fe9980cf010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe98400d800 tx=0x7fe98400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:04.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe9957fa700 1 -- 172.21.15.97:0/3603191445 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9840125f0 con 0x7fe99815c520 2023-07-06T03:46:04.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe9957fa700 1 -- 172.21.15.97:0/3603191445 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe984012750 con 0x7fe99815c520 2023-07-06T03:46:04.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe99efff700 1 -- 172.21.15.97:0/3603191445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9980d14e0 con 0x7fe99815c520 2023-07-06T03:46:04.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe9957fa700 1 -- 172.21.15.97:0/3603191445 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984012b00 con 0x7fe99815c520 2023-07-06T03:46:04.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.206+0000 7fe99efff700 1 -- 172.21.15.97:0/3603191445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99815f9b0 con 0x7fe99815c520 2023-07-06T03:46:04.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.207+0000 7fe99efff700 1 -- 172.21.15.97:0/3603191445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99804e340 con 0x7fe99815c520 2023-07-06T03:46:04.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.208+0000 7fe9957fa700 1 -- 172.21.15.97:0/3603191445 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe984012c60 con 0x7fe99815c520 2023-07-06T03:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.209+0000 7fe9957fa700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe97c06ace0 0x7fe97c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.209+0000 7fe9957fa700 1 -- 172.21.15.97:0/3603191445 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe984089900 con 0x7fe99815c520 2023-07-06T03:46:04.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.209+0000 7fe99cd9b700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe97c06ace0 0x7fe97c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.210+0000 7fe99cd9b700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe97c06ace0 0x7fe97c06d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0011e0 tx=0x7fe98c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:04.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.214+0000 7fe9957fa700 1 -- 172.21.15.97:0/3603191445 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe984054910 con 0x7fe99815c520 2023-07-06T03:46:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.409+0000 7fe99efff700 1 -- 172.21.15.97:0/3603191445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9980b4f40 con 0x7fe99815c520 2023-07-06T03:46:04.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.410+0000 7fe9957fa700 1 -- 172.21.15.97:0/3603191445 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7fe984057f30 con 0x7fe99815c520 2023-07-06T03:46:04.413 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:04.414 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:04.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 -- 172.21.15.97:0/3603191445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe97c06ace0 msgr2=0x7fe97c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:04.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe97c06ace0 0x7fe97c06d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0011e0 tx=0x7fe98c009040 comp rx=0 tx=0).stop 2023-07-06T03:46:04.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 -- 172.21.15.97:0/3603191445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe99815c520 msgr2=0x7fe9980cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:04.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe99815c520 0x7fe9980cf010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe98400d800 tx=0x7fe98400dbc0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 -- 172.21.15.97:0/3603191445 shutdown_connections 2023-07-06T03:46:04.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe97c06ace0 0x7fe97c06d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe998160dd0 0x7fe9980cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9980cd390 0x7fe99815f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 --2- 172.21.15.97:0/3603191445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe99815c520 0x7fe9980cf010 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 -- 172.21.15.97:0/3603191445 >> 172.21.15.97:0/3603191445 conn(0x7fe998071790 msgr2=0x7fe998158f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:04.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 -- 172.21.15.97:0/3603191445 shutdown_connections 2023-07-06T03:46:04.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.413+0000 7fe99efff700 1 -- 172.21.15.97:0/3603191445 wait complete. 2023-07-06T03:46:04.434 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:46:04.435 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:46:04.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.629+0000 7fc4f57ff700 1 Processor -- start 2023-07-06T03:46:04.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.629+0000 7fc4f57ff700 1 -- start start 2023-07-06T03:46:04.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.630+0000 7fc4f57ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4f015c610 0x7fc4f00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.630+0000 7fc4f57ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc4f006b360 0x7fc4f00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.630+0000 7fc4f57ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 0x7fc4f00d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.630+0000 7fc4f57ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f00d0780 con 0x7fc4f015c610 2023-07-06T03:46:04.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.630+0000 7fc4f57ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f015c9e0 con 0x7fc4f006b360 2023-07-06T03:46:04.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.630+0000 7fc4f57ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00b5650 con 0x7fc4f00d2210 2023-07-06T03:46:04.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.630+0000 7fc4eeffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4f015c610 0x7fc4f00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.630+0000 7fc4eeffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4f015c610 0x7fc4f00b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54822/0 (socket says 172.21.15.97:54822) 2023-07-06T03:46:04.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.630+0000 7fc4eeffd700 1 -- 172.21.15.97:0/575023503 learned_addr learned my addr 172.21.15.97:0/575023503 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:04.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4ef7fe700 1 --2- 172.21.15.97:0/575023503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 0x7fc4f00d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4ee7fc700 1 --2- 172.21.15.97:0/575023503 >> v2:172.21.15.146:3300/0 conn(0x7fc4f006b360 0x7fc4f00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4eeffd700 1 -- 172.21.15.97:0/575023503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 msgr2=0x7fc4f00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:04.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4eeffd700 1 --2- 172.21.15.97:0/575023503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 0x7fc4f00d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4eeffd700 1 -- 172.21.15.97:0/575023503 >> v2:172.21.15.146:3300/0 conn(0x7fc4f006b360 msgr2=0x7fc4f00b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:04.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4eeffd700 1 --2- 172.21.15.97:0/575023503 >> v2:172.21.15.146:3300/0 conn(0x7fc4f006b360 0x7fc4f00b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4eeffd700 1 -- 172.21.15.97:0/575023503 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f00d01c0 con 0x7fc4f015c610 2023-07-06T03:46:04.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4ef7fe700 1 --2- 172.21.15.97:0/575023503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 0x7fc4f00d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:04.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4eeffd700 1 --2- 172.21.15.97:0/575023503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f015c610 0x7fc4f00b47b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4f006a590 tx=0x7fc4e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578b4c701662c8c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:04.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4edffb700 1 -- 172.21.15.97:0/575023503 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0011830 con 0x7fc4f015c610 2023-07-06T03:46:04.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4edffb700 1 -- 172.21.15.97:0/575023503 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e0011990 con 0x7fc4f015c610 2023-07-06T03:46:04.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.631+0000 7fc4edffb700 1 -- 172.21.15.97:0/575023503 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0011d20 con 0x7fc4f015c610 2023-07-06T03:46:04.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.632+0000 7fc4f57ff700 1 -- 172.21.15.97:0/575023503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f015c610 msgr2=0x7fc4f00b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:04.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.632+0000 7fc4f57ff700 1 --2- 172.21.15.97:0/575023503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f015c610 0x7fc4f00b47b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4f006a590 tx=0x7fc4e0009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:04.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.632+0000 7fc4f57ff700 1 -- 172.21.15.97:0/575023503 shutdown_connections 2023-07-06T03:46:04.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.632+0000 7fc4f57ff700 1 --2- 172.21.15.97:0/575023503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 0x7fc4f00d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.632+0000 7fc4f57ff700 1 --2- 172.21.15.97:0/575023503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f015c610 0x7fc4f00b47b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.632+0000 7fc4f57ff700 1 --2- 172.21.15.97:0/575023503 >> v2:172.21.15.146:3300/0 conn(0x7fc4f006b360 0x7fc4f00b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.632+0000 7fc4f57ff700 1 -- 172.21.15.97:0/575023503 >> 172.21.15.97:0/575023503 conn(0x7fc4f006a6c0 msgr2=0x7fc4f006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:04.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.632+0000 7fc4f57ff700 1 -- 172.21.15.97:0/575023503 shutdown_connections 2023-07-06T03:46:04.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.632+0000 7fc4f57ff700 1 -- 172.21.15.97:0/575023503 wait complete. 2023-07-06T03:46:04.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.634+0000 7fc4f57ff700 1 Processor -- start 2023-07-06T03:46:04.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.634+0000 7fc4f57ff700 1 -- start start 2023-07-06T03:46:04.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4f57ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f006b360 0x7fc4f00b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4f57ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 0x7fc4f00cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4f57ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f015c610 0x7fc4f015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4f57ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f015cb20 con 0x7fc4f015c610 2023-07-06T03:46:04.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4f57ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00cb910 con 0x7fc4f006b360 2023-07-06T03:46:04.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4f57ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00cbbf0 con 0x7fc4f00d2210 2023-07-06T03:46:04.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4ef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f015c610 0x7fc4f015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4ef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f015c610 0x7fc4f015d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54824/0 (socket says 172.21.15.97:54824) 2023-07-06T03:46:04.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4ef7fe700 1 -- 172.21.15.97:0/3217846053 learned_addr learned my addr 172.21.15.97:0/3217846053 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:04.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4ee7fc700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 0x7fc4f00cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4eeffd700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f006b360 0x7fc4f00b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4ef7fe700 1 -- 172.21.15.97:0/3217846053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 msgr2=0x7fc4f00cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:04.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.635+0000 7fc4ef7fe700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 0x7fc4f00cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.636+0000 7fc4ef7fe700 1 -- 172.21.15.97:0/3217846053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f006b360 msgr2=0x7fc4f00b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:04.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.636+0000 7fc4ef7fe700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f006b360 0x7fc4f00b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.636+0000 7fc4ef7fe700 1 -- 172.21.15.97:0/3217846053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e0009550 con 0x7fc4f015c610 2023-07-06T03:46:04.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.636+0000 7fc4ee7fc700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 0x7fc4f00cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:04.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.636+0000 7fc4eeffd700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f006b360 0x7fc4f00b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:04.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.636+0000 7fc4ef7fe700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f015c610 0x7fc4f015d570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8007ef0 tx=0x7fc4e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:04.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.636+0000 7fc4d77fe700 1 -- 172.21.15.97:0/3217846053 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e800de60 con 0x7fc4f015c610 2023-07-06T03:46:04.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.636+0000 7fc4d77fe700 1 -- 172.21.15.97:0/3217846053 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e800f040 con 0x7fc4f015c610 2023-07-06T03:46:04.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.636+0000 7fc4f57ff700 1 -- 172.21.15.97:0/3217846053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f015cd70 con 0x7fc4f015c610 2023-07-06T03:46:04.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.636+0000 7fc4d77fe700 1 -- 172.21.15.97:0/3217846053 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8003c60 con 0x7fc4f015c610 2023-07-06T03:46:04.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.636+0000 7fc4f57ff700 1 -- 172.21.15.97:0/3217846053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f00cc8a0 con 0x7fc4f015c610 2023-07-06T03:46:04.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.637+0000 7fc4f57ff700 1 -- 172.21.15.97:0/3217846053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f004e340 con 0x7fc4f015c610 2023-07-06T03:46:04.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.638+0000 7fc4d77fe700 1 -- 172.21.15.97:0/3217846053 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc4e8004030 con 0x7fc4f015c610 2023-07-06T03:46:04.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.639+0000 7fc4d77fe700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4dc06ace0 0x7fc4dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:04.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.639+0000 7fc4d77fe700 1 -- 172.21.15.97:0/3217846053 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4e8089650 con 0x7fc4f015c610 2023-07-06T03:46:04.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.639+0000 7fc4eeffd700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4dc06ace0 0x7fc4dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:04.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.640+0000 7fc4eeffd700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4dc06ace0 0x7fc4dc06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0014040 tx=0x7fc4e0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:04.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.643+0000 7fc4d77fe700 1 -- 172.21.15.97:0/3217846053 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc4e80546f0 con 0x7fc4f015c610 2023-07-06T03:46:04.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.838+0000 7fc4f57ff700 1 -- 172.21.15.97:0/3217846053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4f00d0a60 con 0x7fc4f015c610 2023-07-06T03:46:04.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.838+0000 7fc4d77fe700 1 -- 172.21.15.97:0/3217846053 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4e8057d10 con 0x7fc4f015c610 2023-07-06T03:46:04.842 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:46:04.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.840+0000 7fc4f57ff700 1 -- 172.21.15.97:0/3217846053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4dc06ace0 msgr2=0x7fc4dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.840+0000 7fc4f57ff700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4dc06ace0 0x7fc4dc06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0014040 tx=0x7fc4e0008030 comp rx=0 tx=0).stop 2023-07-06T03:46:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.841+0000 7fc4f57ff700 1 -- 172.21.15.97:0/3217846053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f015c610 msgr2=0x7fc4f015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.841+0000 7fc4f57ff700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f015c610 0x7fc4f015d570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8007ef0 tx=0x7fc4e800d3b0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.841+0000 7fc4f57ff700 1 -- 172.21.15.97:0/3217846053 shutdown_connections 2023-07-06T03:46:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.841+0000 7fc4f57ff700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4dc06ace0 0x7fc4dc06d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.841+0000 7fc4f57ff700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f015c610 0x7fc4f015d570 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.841+0000 7fc4f57ff700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f006b360 0x7fc4f00b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.841+0000 7fc4f57ff700 1 --2- 172.21.15.97:0/3217846053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4f00d2210 0x7fc4f00cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.841+0000 7fc4f57ff700 1 -- 172.21.15.97:0/3217846053 >> 172.21.15.97:0/3217846053 conn(0x7fc4f006a6c0 msgr2=0x7fc4f00b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.841+0000 7fc4f57ff700 1 -- 172.21.15.97:0/3217846053 shutdown_connections 2023-07-06T03:46:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:04.841+0000 7fc4f57ff700 1 -- 172.21.15.97:0/3217846053 wait complete. 2023-07-06T03:46:04.862 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:46:05.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.056+0000 7f57e6fff700 1 Processor -- start 2023-07-06T03:46:05.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.056+0000 7f57e6fff700 1 -- start start 2023-07-06T03:46:05.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e6fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57e0154890 0x7f57e00cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e6fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f57e015eb90 0x7f57e00cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e6fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57e015a940 0x7f57e015ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e6fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e015b6e0 con 0x7f57e0154890 2023-07-06T03:46:05.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e6fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e015e0b0 con 0x7f57e015eb90 2023-07-06T03:46:05.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e6fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e015e390 con 0x7f57e015a940 2023-07-06T03:46:05.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e4d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57e0154890 0x7f57e00cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e4d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57e0154890 0x7f57e00cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54830/0 (socket says 172.21.15.97:54830) 2023-07-06T03:46:05.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e4d9b700 1 -- 172.21.15.97:0/3792734603 learned_addr learned my addr 172.21.15.97:0/3792734603 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:05.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e559c700 1 --2- 172.21.15.97:0/3792734603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57e015a940 0x7f57e015ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57dffff700 1 --2- 172.21.15.97:0/3792734603 >> v2:172.21.15.146:3300/0 conn(0x7f57e015eb90 0x7f57e00cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e4d9b700 1 -- 172.21.15.97:0/3792734603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57e015a940 msgr2=0x7f57e015ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:05.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e4d9b700 1 --2- 172.21.15.97:0/3792734603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57e015a940 0x7f57e015ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e4d9b700 1 -- 172.21.15.97:0/3792734603 >> v2:172.21.15.146:3300/0 conn(0x7f57e015eb90 msgr2=0x7f57e00cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:05.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.058+0000 7f57e4d9b700 1 --2- 172.21.15.97:0/3792734603 >> v2:172.21.15.146:3300/0 conn(0x7f57e015eb90 0x7f57e00cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.059+0000 7f57e4d9b700 1 -- 172.21.15.97:0/3792734603 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e015e670 con 0x7f57e0154890 2023-07-06T03:46:05.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.059+0000 7f57e4d9b700 1 --2- 172.21.15.97:0/3792734603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e0154890 0x7f57e00cb390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f57e00cf9f0 tx=0x7f57d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b335f12e680547f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:05.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.059+0000 7f57df7fe700 1 -- 172.21.15.97:0/3792734603 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d400b870 con 0x7f57e0154890 2023-07-06T03:46:05.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.059+0000 7f57df7fe700 1 -- 172.21.15.97:0/3792734603 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d400b9d0 con 0x7f57e0154890 2023-07-06T03:46:05.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.059+0000 7f57df7fe700 1 -- 172.21.15.97:0/3792734603 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d400bd60 con 0x7f57e0154890 2023-07-06T03:46:05.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.059+0000 7f57e6fff700 1 -- 172.21.15.97:0/3792734603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e0154890 msgr2=0x7f57e00cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.059+0000 7f57e6fff700 1 --2- 172.21.15.97:0/3792734603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e0154890 0x7f57e00cb390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f57e00cf9f0 tx=0x7f57d4009bd0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.060+0000 7f57e6fff700 1 -- 172.21.15.97:0/3792734603 shutdown_connections 2023-07-06T03:46:05.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.060+0000 7f57e6fff700 1 --2- 172.21.15.97:0/3792734603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57e015a940 0x7f57e015ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.060+0000 7f57e6fff700 1 --2- 172.21.15.97:0/3792734603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e0154890 0x7f57e00cb390 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.060+0000 7f57e6fff700 1 --2- 172.21.15.97:0/3792734603 >> v2:172.21.15.146:3300/0 conn(0x7f57e015eb90 0x7f57e00cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.060+0000 7f57e6fff700 1 -- 172.21.15.97:0/3792734603 >> 172.21.15.97:0/3792734603 conn(0x7f57e006a060 msgr2=0x7f57e006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:05.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.060+0000 7f57e6fff700 1 -- 172.21.15.97:0/3792734603 shutdown_connections 2023-07-06T03:46:05.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.060+0000 7f57e6fff700 1 -- 172.21.15.97:0/3792734603 wait complete. 2023-07-06T03:46:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.061+0000 7f57e6fff700 1 Processor -- start 2023-07-06T03:46:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.062+0000 7f57e6fff700 1 -- start start 2023-07-06T03:46:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.062+0000 7f57e6fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57e0154890 0x7f57e00ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.062+0000 7f57e6fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57e015a940 0x7f57e01453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.062+0000 7f57e6fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e015eb90 0x7f57e014f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.062+0000 7f57e6fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00ce590 con 0x7f57e015eb90 2023-07-06T03:46:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.062+0000 7f57e6fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0155010 con 0x7f57e0154890 2023-07-06T03:46:05.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.062+0000 7f57e6fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e01f56b0 con 0x7f57e015a940 2023-07-06T03:46:05.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.063+0000 7f57e559c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e015eb90 0x7f57e014f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.063+0000 7f57e559c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e015eb90 0x7f57e014f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54840/0 (socket says 172.21.15.97:54840) 2023-07-06T03:46:05.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.063+0000 7f57e559c700 1 -- 172.21.15.97:0/1407082525 learned_addr learned my addr 172.21.15.97:0/1407082525 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:05.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.063+0000 7f57dffff700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57e015a940 0x7f57e01453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.063+0000 7f57e4d9b700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57e0154890 0x7f57e00ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.063+0000 7f57e559c700 1 -- 172.21.15.97:0/1407082525 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57e015a940 msgr2=0x7f57e01453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.063+0000 7f57e559c700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57e015a940 0x7f57e01453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.063+0000 7f57e559c700 1 -- 172.21.15.97:0/1407082525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57e0154890 msgr2=0x7f57e00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.063+0000 7f57e559c700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57e0154890 0x7f57e00ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.063+0000 7f57e559c700 1 -- 172.21.15.97:0/1407082525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4009550 con 0x7f57e015eb90 2023-07-06T03:46:05.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.063+0000 7f57e559c700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e015eb90 0x7f57e014f3b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f57d8007e60 tx=0x7f57d800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:05.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.064+0000 7f57dd7fa700 1 -- 172.21.15.97:0/1407082525 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d800ede0 con 0x7f57e015eb90 2023-07-06T03:46:05.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.064+0000 7f57dd7fa700 1 -- 172.21.15.97:0/1407082525 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d8014420 con 0x7f57e015eb90 2023-07-06T03:46:05.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.064+0000 7f57dd7fa700 1 -- 172.21.15.97:0/1407082525 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d80147d0 con 0x7f57e015eb90 2023-07-06T03:46:05.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.064+0000 7f57e6fff700 1 -- 172.21.15.97:0/1407082525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e01f5990 con 0x7f57e015eb90 2023-07-06T03:46:05.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.064+0000 7f57e6fff700 1 -- 172.21.15.97:0/1407082525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e01f5f50 con 0x7f57e015eb90 2023-07-06T03:46:05.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.065+0000 7f57e6fff700 1 -- 172.21.15.97:0/1407082525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e00cc4c0 con 0x7f57e015eb90 2023-07-06T03:46:05.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.066+0000 7f57dd7fa700 1 -- 172.21.15.97:0/1407082525 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57d8014930 con 0x7f57e015eb90 2023-07-06T03:46:05.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.067+0000 7f57dd7fa700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f57c406ace0 0x7f57c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.067+0000 7f57dd7fa700 1 -- 172.21.15.97:0/1407082525 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57d8089cc0 con 0x7f57e015eb90 2023-07-06T03:46:05.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.067+0000 7f57e4d9b700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f57c406ace0 0x7f57c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.068+0000 7f57e4d9b700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f57c406ace0 0x7f57c406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57d4000c00 tx=0x7f57d4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:05.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.071+0000 7f57dd7fa700 1 -- 172.21.15.97:0/1407082525 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f57e00cc4c0 con 0x7f57e015eb90 2023-07-06T03:46:05.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.263+0000 7f57e6fff700 1 -- 172.21.15.97:0/1407082525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f57e00cc4c0 con 0x7f57e015eb90 2023-07-06T03:46:05.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.263+0000 7f57dd7fa700 1 -- 172.21.15.97:0/1407082525 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f57e00cc4c0 con 0x7f57e015eb90 2023-07-06T03:46:05.266 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:46:05.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.265+0000 7f57e6fff700 1 -- 172.21.15.97:0/1407082525 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f57c406ace0 msgr2=0x7f57c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.265+0000 7f57e6fff700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f57c406ace0 0x7f57c406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57d4000c00 tx=0x7f57d4008030 comp rx=0 tx=0).stop 2023-07-06T03:46:05.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.265+0000 7f57e6fff700 1 -- 172.21.15.97:0/1407082525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e015eb90 msgr2=0x7f57e014f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.265+0000 7f57e6fff700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e015eb90 0x7f57e014f3b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f57d8007e60 tx=0x7f57d800e3c0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.266+0000 7f57e6fff700 1 -- 172.21.15.97:0/1407082525 shutdown_connections 2023-07-06T03:46:05.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.266+0000 7f57e6fff700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f57c406ace0 0x7f57c406d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.266+0000 7f57e6fff700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e015eb90 0x7f57e014f3b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.266+0000 7f57e6fff700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57e0154890 0x7f57e00ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.266+0000 7f57e6fff700 1 --2- 172.21.15.97:0/1407082525 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57e015a940 0x7f57e01453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.266+0000 7f57e6fff700 1 -- 172.21.15.97:0/1407082525 >> 172.21.15.97:0/1407082525 conn(0x7f57e006a060 msgr2=0x7f57e00caae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:05.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.266+0000 7f57e6fff700 1 -- 172.21.15.97:0/1407082525 shutdown_connections 2023-07-06T03:46:05.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.266+0000 7f57e6fff700 1 -- 172.21.15.97:0/1407082525 wait complete. 2023-07-06T03:46:05.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:46:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.482+0000 7f192abff700 1 Processor -- start 2023-07-06T03:46:05.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.482+0000 7f192abff700 1 -- start start 2023-07-06T03:46:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.483+0000 7f192abff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1924159070 0x7f19240c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.483+0000 7f192abff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19240ca260 con 0x7f19240ccbd0 2023-07-06T03:46:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.484+0000 7f192abff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19240c99c0 con 0x7f19240c3cd0 2023-07-06T03:46:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.484+0000 7f192abff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19240c9ca0 con 0x7f1924159070 2023-07-06T03:46:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.484+0000 7f192919c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f19240ccbd0 0x7f19240cb0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56782/0 (socket says 172.21.15.97:56782) 2023-07-06T03:46:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.484+0000 7f192919c700 1 -- 172.21.15.97:0/2809124762 learned_addr learned my addr 172.21.15.97:0/2809124762 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:05.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.484+0000 7f192899b700 1 --2- 172.21.15.97:0/2809124762 >> v2:172.21.15.146:3300/0 conn(0x7f1924159070 0x7f19240c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.484+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185125287 0 0) 0x7f19240ca260 con 0x7f19240ccbd0 2023-07-06T03:46:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.484+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19140057c0 con 0x7f19240ccbd0 2023-07-06T03:46:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.484+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 141461234 0 0) 0x7f19140057c0 con 0x7f19240ccbd0 2023-07-06T03:46:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.484+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 >> v2:172.21.15.146:3300/0 conn(0x7f1924159070 msgr2=0x7f19240c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.484+0000 7f19237fe700 1 --2- 172.21.15.97:0/2809124762 >> v2:172.21.15.146:3300/0 conn(0x7f1924159070 0x7f19240c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.484+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 >> v1:172.21.15.146:6789/0 conn(0x7f19240c3cd0 legacy=0x7f19240cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19240c9f80 con 0x7f19240ccbd0 2023-07-06T03:46:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f191c0019f0 con 0x7f19240ccbd0 2023-07-06T03:46:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 >> v1:172.21.15.97:6789/0 conn(0x7f19240ccbd0 legacy=0x7f19240cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f19237fe700 1 --2- 172.21.15.97:0/2809124762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1914005c60 0x7f1914006070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f19237fe700 1 --2- 172.21.15.97:0/2809124762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914006a60 0x7f1914006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f19237fe700 1 --2- 172.21.15.97:0/2809124762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1914007460 0x7f191400dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c9ca0 con 0x7f1914006a60 2023-07-06T03:46:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191400e370 con 0x7f1914007460 2023-07-06T03:46:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191400e530 con 0x7f1914005c60 2023-07-06T03:46:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f1923fff700 1 --2- 172.21.15.97:0/2809124762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914006a60 0x7f1914006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f192899b700 1 --2- 172.21.15.97:0/2809124762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1914005c60 0x7f1914006070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f192919c700 1 --2- 172.21.15.97:0/2809124762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1914007460 0x7f191400dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f1923fff700 1 -- 172.21.15.97:0/2809124762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1914005c60 msgr2=0x7f1914006070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f1923fff700 1 --2- 172.21.15.97:0/2809124762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1914005c60 0x7f1914006070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f1923fff700 1 -- 172.21.15.97:0/2809124762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1914007460 msgr2=0x7f191400dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f1923fff700 1 --2- 172.21.15.97:0/2809124762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1914007460 0x7f191400dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f1923fff700 1 -- 172.21.15.97:0/2809124762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19140057c0 con 0x7f1914006a60 2023-07-06T03:46:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.485+0000 7f192919c700 1 -- 172.21.15.97:0/2809124762 reap_dead start 2023-07-06T03:46:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.486+0000 7f1923fff700 1 --2- 172.21.15.97:0/2809124762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914006a60 0x7f1914006ed0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f19240c40d0 tx=0x7f1914007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5c7294755e8097 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.486+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100052d0 con 0x7f1914006a60 2023-07-06T03:46:05.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.486+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1910005430 con 0x7f1914006a60 2023-07-06T03:46:05.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.486+0000 7f19237fe700 1 -- 172.21.15.97:0/2809124762 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100038c0 con 0x7f1914006a60 2023-07-06T03:46:05.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.486+0000 7f192abff700 1 -- 172.21.15.97:0/2809124762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914006a60 msgr2=0x7f1914006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.486+0000 7f192abff700 1 --2- 172.21.15.97:0/2809124762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914006a60 0x7f1914006ed0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f19240c40d0 tx=0x7f1914007a90 comp rx=0 tx=0).stop 2023-07-06T03:46:05.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.487+0000 7f192abff700 1 -- 172.21.15.97:0/2809124762 shutdown_connections 2023-07-06T03:46:05.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.487+0000 7f192abff700 1 --2- 172.21.15.97:0/2809124762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914006a60 0x7f1914006ed0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.487+0000 7f192abff700 1 -- 172.21.15.97:0/2809124762 >> 172.21.15.97:0/2809124762 conn(0x7f192406a060 msgr2=0x7f192406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:05.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.487+0000 7f192abff700 1 -- 172.21.15.97:0/2809124762 shutdown_connections 2023-07-06T03:46:05.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.487+0000 7f192abff700 1 -- 172.21.15.97:0/2809124762 wait complete. 2023-07-06T03:46:05.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.488+0000 7f192abff700 1 Processor -- start 2023-07-06T03:46:05.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.489+0000 7f192abff700 1 -- start start 2023-07-06T03:46:05.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.489+0000 7f192abff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1914006a60 0x7f19240c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.489+0000 7f192abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19241493c0 0x7f1924149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192abff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19240c59c0 0x7f19240c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192abff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c7700 con 0x7f19240c59c0 2023-07-06T03:46:05.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192abff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1924149d70 con 0x7f1914006a60 2023-07-06T03:46:05.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c6680 con 0x7f19241493c0 2023-07-06T03:46:05.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192919c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19240c59c0 0x7f19240c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192919c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19240c59c0 0x7f19240c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54856/0 (socket says 172.21.15.97:54856) 2023-07-06T03:46:05.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192919c700 1 -- 172.21.15.97:0/3886499091 learned_addr learned my addr 172.21.15.97:0/3886499091 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:05.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192899b700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1914006a60 0x7f19240c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f1923fff700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19241493c0 0x7f1924149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192919c700 1 -- 172.21.15.97:0/3886499091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19241493c0 msgr2=0x7f1924149830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192919c700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19241493c0 0x7f1924149830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192919c700 1 -- 172.21.15.97:0/3886499091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1914006a60 msgr2=0x7f19240c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192919c700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1914006a60 0x7f19240c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192919c700 1 -- 172.21.15.97:0/3886499091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19240c6960 con 0x7f19240c59c0 2023-07-06T03:46:05.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f1923fff700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19241493c0 0x7f1924149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:05.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.490+0000 7f192919c700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19240c59c0 0x7f19240c5e60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f19240ce090 tx=0x7f191c00a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:05.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.491+0000 7f19217fa700 1 -- 172.21.15.97:0/3886499091 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c00ad20 con 0x7f19240c59c0 2023-07-06T03:46:05.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.491+0000 7f19217fa700 1 -- 172.21.15.97:0/3886499091 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c00ae80 con 0x7f19240c59c0 2023-07-06T03:46:05.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.491+0000 7f19217fa700 1 -- 172.21.15.97:0/3886499091 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c013630 con 0x7f19240c59c0 2023-07-06T03:46:05.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.491+0000 7f192abff700 1 -- 172.21.15.97:0/3886499091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19240c6c40 con 0x7f19240c59c0 2023-07-06T03:46:05.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.491+0000 7f192abff700 1 -- 172.21.15.97:0/3886499091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19240c7200 con 0x7f19240c59c0 2023-07-06T03:46:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.492+0000 7f192abff700 1 -- 172.21.15.97:0/3886499091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19240c7390 con 0x7f19240c59c0 2023-07-06T03:46:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.493+0000 7f19217fa700 1 -- 172.21.15.97:0/3886499091 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f191c013790 con 0x7f19240c59c0 2023-07-06T03:46:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.495+0000 7f19217fa700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190806ac70 0x7f190806d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.495+0000 7f19217fa700 1 -- 172.21.15.97:0/3886499091 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f191c089df0 con 0x7f19240c59c0 2023-07-06T03:46:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.495+0000 7f192899b700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190806ac70 0x7f190806d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.495+0000 7f192899b700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190806ac70 0x7f190806d120 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1924063c30 tx=0x7f1918008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.499+0000 7f19217fa700 1 -- 172.21.15.97:0/3886499091 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f191c054ea0 con 0x7f19240c59c0 2023-07-06T03:46:05.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.692+0000 7f192abff700 1 -- 172.21.15.97:0/3886499091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f192404e340 con 0x7f19240c59c0 2023-07-06T03:46:05.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.692+0000 7f19217fa700 1 -- 172.21.15.97:0/3886499091 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f191c0584c0 con 0x7f19240c59c0 2023-07-06T03:46:05.696 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:46:05.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 -- 172.21.15.97:0/3886499091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190806ac70 msgr2=0x7f190806d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190806ac70 0x7f190806d120 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1924063c30 tx=0x7f1918008040 comp rx=0 tx=0).stop 2023-07-06T03:46:05.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 -- 172.21.15.97:0/3886499091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19240c59c0 msgr2=0x7f19240c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19240c59c0 0x7f19240c5e60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f19240ce090 tx=0x7f191c00a550 comp rx=0 tx=0).stop 2023-07-06T03:46:05.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 -- 172.21.15.97:0/3886499091 shutdown_connections 2023-07-06T03:46:05.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f190806ac70 0x7f190806d120 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19240c59c0 0x7f19240c5e60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1914006a60 0x7f19240c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 --2- 172.21.15.97:0/3886499091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19241493c0 0x7f1924149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 -- 172.21.15.97:0/3886499091 >> 172.21.15.97:0/3886499091 conn(0x7f192406a060 msgr2=0x7f19240b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 -- 172.21.15.97:0/3886499091 shutdown_connections 2023-07-06T03:46:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.695+0000 7f192abff700 1 -- 172.21.15.97:0/3886499091 wait complete. 2023-07-06T03:46:05.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:46:05.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.911+0000 7f6265dff700 1 Processor -- start 2023-07-06T03:46:05.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.911+0000 7f6265dff700 1 -- start start 2023-07-06T03:46:05.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.912+0000 7f6265dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f626015c610 0x7f62600b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.912+0000 7f6265dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f626006b360 0x7f62600b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.912+0000 7f6265dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62600cff90 con 0x7f62600d2210 2023-07-06T03:46:05.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.912+0000 7f6265dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62600d0780 con 0x7f626006b360 2023-07-06T03:46:05.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.912+0000 7f6265dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626015c9e0 con 0x7f626015c610 2023-07-06T03:46:05.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.912+0000 7f6257fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f626006b360 0x7f62600b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.912+0000 7f6257fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f626006b360 0x7f62600b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54872/0 (socket says 172.21.15.97:54872) 2023-07-06T03:46:05.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.912+0000 7f6257fff700 1 -- 172.21.15.97:0/3361706990 learned_addr learned my addr 172.21.15.97:0/3361706990 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:05.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.913+0000 7f6257fff700 1 -- 172.21.15.97:0/3361706990 >> v2:172.21.15.146:3300/0 conn(0x7f626015c610 msgr2=0x7f62600b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:05.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.913+0000 7f6257fff700 1 --2- 172.21.15.97:0/3361706990 >> v2:172.21.15.146:3300/0 conn(0x7f626015c610 0x7f62600b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.913+0000 7f6257fff700 1 -- 172.21.15.97:0/3361706990 >> v1:172.21.15.146:6789/0 conn(0x7f62600d2210 legacy=0x7f62600d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.913+0000 7f6257fff700 1 -- 172.21.15.97:0/3361706990 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62600b5650 con 0x7f626006b360 2023-07-06T03:46:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.913+0000 7f6257fff700 1 --2- 172.21.15.97:0/3361706990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626006b360 0x7f62600b4cf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f626008b000 tx=0x7f6248009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad575583cc040327 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.913+0000 7f625effd700 1 -- 172.21.15.97:0/3361706990 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248004230 con 0x7f626006b360 2023-07-06T03:46:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.913+0000 7f625effd700 1 -- 172.21.15.97:0/3361706990 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6248004390 con 0x7f626006b360 2023-07-06T03:46:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.913+0000 7f625effd700 1 -- 172.21.15.97:0/3361706990 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62480036a0 con 0x7f626006b360 2023-07-06T03:46:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.914+0000 7f6265dff700 1 -- 172.21.15.97:0/3361706990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626006b360 msgr2=0x7f62600b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.914+0000 7f6265dff700 1 --2- 172.21.15.97:0/3361706990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626006b360 0x7f62600b4cf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f626008b000 tx=0x7f6248009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.914+0000 7f6265dff700 1 -- 172.21.15.97:0/3361706990 shutdown_connections 2023-07-06T03:46:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.914+0000 7f6265dff700 1 --2- 172.21.15.97:0/3361706990 >> v2:172.21.15.146:3300/0 conn(0x7f626015c610 0x7f62600b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.914+0000 7f6265dff700 1 --2- 172.21.15.97:0/3361706990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626006b360 0x7f62600b4cf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.914+0000 7f6265dff700 1 -- 172.21.15.97:0/3361706990 >> 172.21.15.97:0/3361706990 conn(0x7f626006a6c0 msgr2=0x7f626006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.914+0000 7f6265dff700 1 -- 172.21.15.97:0/3361706990 shutdown_connections 2023-07-06T03:46:05.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.914+0000 7f6265dff700 1 -- 172.21.15.97:0/3361706990 wait complete. 2023-07-06T03:46:05.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.916+0000 7f6265dff700 1 Processor -- start 2023-07-06T03:46:05.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.916+0000 7f6265dff700 1 -- start start 2023-07-06T03:46:05.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6265dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f626006b360 0x7f626015e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6265dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62600d2210 0x7f62600b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6265dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f626015c610 0x7f626014e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6265dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62601f24e0 con 0x7f62600d2210 2023-07-06T03:46:05.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6265dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62600c9ea0 con 0x7f626006b360 2023-07-06T03:46:05.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6265dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62600b2cd0 con 0x7f626015c610 2023-07-06T03:46:05.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6257fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62600d2210 0x7f62600b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6257fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62600d2210 0x7f62600b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54886/0 (socket says 172.21.15.97:54886) 2023-07-06T03:46:05.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6257fff700 1 -- 172.21.15.97:0/3279643438 learned_addr learned my addr 172.21.15.97:0/3279643438 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:05.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6257fff700 1 -- 172.21.15.97:0/3279643438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f626015c610 msgr2=0x7f626014e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6257fff700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f626015c610 0x7f626014e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6257fff700 1 -- 172.21.15.97:0/3279643438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f626006b360 msgr2=0x7f626015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f625f7fe700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f626006b360 0x7f626015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6257fff700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f626006b360 0x7f626015e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f6257fff700 1 -- 172.21.15.97:0/3279643438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248009550 con 0x7f62600d2210 2023-07-06T03:46:05.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.917+0000 7f625f7fe700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f626006b360 0x7f626015e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:05.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.918+0000 7f6257fff700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62600d2210 0x7f62600b2790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f626008b120 tx=0x7f6248004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:05.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.918+0000 7f625cff9700 1 -- 172.21.15.97:0/3279643438 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248003df0 con 0x7f62600d2210 2023-07-06T03:46:05.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.918+0000 7f625cff9700 1 -- 172.21.15.97:0/3279643438 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624801f070 con 0x7f62600d2210 2023-07-06T03:46:05.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.918+0000 7f6265dff700 1 -- 172.21.15.97:0/3279643438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626015f600 con 0x7f62600d2210 2023-07-06T03:46:05.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.918+0000 7f625cff9700 1 -- 172.21.15.97:0/3279643438 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248011870 con 0x7f62600d2210 2023-07-06T03:46:05.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.918+0000 7f6265dff700 1 -- 172.21.15.97:0/3279643438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62600ca3f0 con 0x7f62600d2210 2023-07-06T03:46:05.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.919+0000 7f6265dff700 1 -- 172.21.15.97:0/3279643438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62600ca5e0 con 0x7f62600d2210 2023-07-06T03:46:05.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.920+0000 7f625cff9700 1 -- 172.21.15.97:0/3279643438 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6248021030 con 0x7f62600d2210 2023-07-06T03:46:05.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.921+0000 7f625cff9700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f624006ace0 0x7f624006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.921+0000 7f625cff9700 1 -- 172.21.15.97:0/3279643438 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f624808abc0 con 0x7f62600d2210 2023-07-06T03:46:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.921+0000 7f625f7fe700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f624006ace0 0x7f624006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.922+0000 7f625f7fe700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f624006ace0 0x7f624006d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f625000a8f0 tx=0x7f6250008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:05.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:05.926+0000 7f625cff9700 1 -- 172.21.15.97:0/3279643438 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6248055c60 con 0x7f62600d2210 2023-07-06T03:46:06.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.121+0000 7f6265dff700 1 -- 172.21.15.97:0/3279643438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62600ca5e0 con 0x7f62600d2210 2023-07-06T03:46:06.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.122+0000 7f625cff9700 1 -- 172.21.15.97:0/3279643438 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6248059280 con 0x7f62600d2210 2023-07-06T03:46:06.125 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:46:06.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 -- 172.21.15.97:0/3279643438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f624006ace0 msgr2=0x7f624006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f624006ace0 0x7f624006d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f625000a8f0 tx=0x7f6250008040 comp rx=0 tx=0).stop 2023-07-06T03:46:06.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 -- 172.21.15.97:0/3279643438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62600d2210 msgr2=0x7f62600b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62600d2210 0x7f62600b2790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f626008b120 tx=0x7f6248004750 comp rx=0 tx=0).stop 2023-07-06T03:46:06.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 -- 172.21.15.97:0/3279643438 shutdown_connections 2023-07-06T03:46:06.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f624006ace0 0x7f624006d190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f626015c610 0x7f626014e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f626006b360 0x7f626015e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 --2- 172.21.15.97:0/3279643438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62600d2210 0x7f62600b2790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 -- 172.21.15.97:0/3279643438 >> 172.21.15.97:0/3279643438 conn(0x7f626006a6c0 msgr2=0x7f626015a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:06.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 -- 172.21.15.97:0/3279643438 shutdown_connections 2023-07-06T03:46:06.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.124+0000 7f6265dff700 1 -- 172.21.15.97:0/3279643438 wait complete. 2023-07-06T03:46:06.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:46:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.337+0000 7fcd077ff700 1 Processor -- start 2023-07-06T03:46:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.337+0000 7fcd077ff700 1 -- start start 2023-07-06T03:46:06.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.338+0000 7fcd077ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 0x7fcd000b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:06.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.338+0000 7fcd077ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcd0006b360 0x7fcd000b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:06.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.338+0000 7fcd077ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd000cff90 con 0x7fcd000d2210 2023-07-06T03:46:06.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.338+0000 7fcd077ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd000d0780 con 0x7fcd0006b360 2023-07-06T03:46:06.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.338+0000 7fcd077ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0015c9e0 con 0x7fcd0015c610 2023-07-06T03:46:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fcd0559b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 0x7fcd000b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fcd04d9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcd0006b360 0x7fcd000b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fcd0559b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 0x7fcd000b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:36744/0 (socket says 172.21.15.97:36744) 2023-07-06T03:46:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fcd05d9c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fcd000d2210 0x7fcd000d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:42012/0 (socket says 172.21.15.97:42012) 2023-07-06T03:46:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fcd0559b700 1 -- 172.21.15.97:0/3218640286 learned_addr learned my addr 172.21.15.97:0/3218640286 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fccf7fff700 1 -- 172.21.15.97:0/3218640286 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230075533 0 0) 0x7fcd000cff90 con 0x7fcd000d2210 2023-07-06T03:46:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fccf7fff700 1 -- 172.21.15.97:0/3218640286 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccec0035b0 con 0x7fcd000d2210 2023-07-06T03:46:06.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fcd0559b700 1 -- 172.21.15.97:0/3218640286 >> v2:172.21.15.146:3300/0 conn(0x7fcd0006b360 msgr2=0x7fcd000b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:06.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fcd0559b700 1 --2- 172.21.15.97:0/3218640286 >> v2:172.21.15.146:3300/0 conn(0x7fcd0006b360 0x7fcd000b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fcd0559b700 1 -- 172.21.15.97:0/3218640286 >> v1:172.21.15.146:6789/0 conn(0x7fcd000d2210 legacy=0x7fcd000d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fcd0559b700 1 -- 172.21.15.97:0/3218640286 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd000b5650 con 0x7fcd0015c610 2023-07-06T03:46:06.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.339+0000 7fcd04d9a700 1 --2- 172.21.15.97:0/3218640286 >> v2:172.21.15.146:3300/0 conn(0x7fcd0006b360 0x7fcd000b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:46:06.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.340+0000 7fcd0559b700 1 --2- 172.21.15.97:0/3218640286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 0x7fcd000b47b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0098f0 tx=0x7fccfc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc819b164fd72d26 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:06.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.340+0000 7fccf7fff700 1 -- 172.21.15.97:0/3218640286 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc011830 con 0x7fcd0015c610 2023-07-06T03:46:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.340+0000 7fccf7fff700 1 -- 172.21.15.97:0/3218640286 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc011990 con 0x7fcd0015c610 2023-07-06T03:46:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.340+0000 7fccf7fff700 1 -- 172.21.15.97:0/3218640286 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc011d20 con 0x7fcd0015c610 2023-07-06T03:46:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.341+0000 7fcd077ff700 1 -- 172.21.15.97:0/3218640286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 msgr2=0x7fcd000b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.341+0000 7fcd077ff700 1 --2- 172.21.15.97:0/3218640286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 0x7fcd000b47b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0098f0 tx=0x7fccfc009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.341+0000 7fcd077ff700 1 -- 172.21.15.97:0/3218640286 shutdown_connections 2023-07-06T03:46:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.341+0000 7fcd077ff700 1 --2- 172.21.15.97:0/3218640286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 0x7fcd000b47b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.341+0000 7fcd077ff700 1 --2- 172.21.15.97:0/3218640286 >> v2:172.21.15.146:3300/0 conn(0x7fcd0006b360 0x7fcd000b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.341+0000 7fcd077ff700 1 -- 172.21.15.97:0/3218640286 >> 172.21.15.97:0/3218640286 conn(0x7fcd0006a6c0 msgr2=0x7fcd0006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.341+0000 7fcd077ff700 1 -- 172.21.15.97:0/3218640286 shutdown_connections 2023-07-06T03:46:06.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.341+0000 7fcd077ff700 1 -- 172.21.15.97:0/3218640286 wait complete. 2023-07-06T03:46:06.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.343+0000 7fcd077ff700 1 Processor -- start 2023-07-06T03:46:06.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.343+0000 7fcd077ff700 1 -- start start 2023-07-06T03:46:06.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd077ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd0006b360 0x7fcd0014d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:06.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd077ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd000d2210 0x7fcd001f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:06.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd077ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 0x7fcd000d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:06.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd077ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd001f2e00 con 0x7fcd0006b360 2023-07-06T03:46:06.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd077ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0014d690 con 0x7fcd000d2210 2023-07-06T03:46:06.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd077ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd001f5cb0 con 0x7fcd0015c610 2023-07-06T03:46:06.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd0559b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd0006b360 0x7fcd0014d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:06.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd0559b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd0006b360 0x7fcd0014d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54892/0 (socket says 172.21.15.97:54892) 2023-07-06T03:46:06.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd0559b700 1 -- 172.21.15.97:0/1149536472 learned_addr learned my addr 172.21.15.97:0/1149536472 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd04d9a700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd000d2210 0x7fcd001f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd05d9c700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 0x7fcd000d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd0559b700 1 -- 172.21.15.97:0/1149536472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 msgr2=0x7fcd000d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd0559b700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 0x7fcd000d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd0559b700 1 -- 172.21.15.97:0/1149536472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd000d2210 msgr2=0x7fcd001f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd0559b700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd000d2210 0x7fcd001f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.344+0000 7fcd0559b700 1 -- 172.21.15.97:0/1149536472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf00094a0 con 0x7fcd0006b360 2023-07-06T03:46:06.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.345+0000 7fcd0559b700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd0006b360 0x7fcd0014d150 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fccfc014040 tx=0x7fccfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:06.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.345+0000 7fccf5ffb700 1 -- 172.21.15.97:0/1149536472 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc004300 con 0x7fcd0006b360 2023-07-06T03:46:06.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.345+0000 7fccf5ffb700 1 -- 172.21.15.97:0/1149536472 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc01f070 con 0x7fcd0006b360 2023-07-06T03:46:06.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.345+0000 7fcd077ff700 1 -- 172.21.15.97:0/1149536472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccfc009550 con 0x7fcd0006b360 2023-07-06T03:46:06.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.345+0000 7fccf5ffb700 1 -- 172.21.15.97:0/1149536472 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0036a0 con 0x7fcd0006b360 2023-07-06T03:46:06.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.345+0000 7fcd077ff700 1 -- 172.21.15.97:0/1149536472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd001f3050 con 0x7fcd0006b360 2023-07-06T03:46:06.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.346+0000 7fcd077ff700 1 -- 172.21.15.97:0/1149536472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0004e340 con 0x7fcd0006b360 2023-07-06T03:46:06.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.347+0000 7fccf5ffb700 1 -- 172.21.15.97:0/1149536472 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fccfc021030 con 0x7fcd0006b360 2023-07-06T03:46:06.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.348+0000 7fccf5ffb700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcce406ace0 0x7fcce406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:06.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.349+0000 7fccf5ffb700 1 -- 172.21.15.97:0/1149536472 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fccfc08a670 con 0x7fcd0006b360 2023-07-06T03:46:06.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.349+0000 7fcd04d9a700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcce406ace0 0x7fcce406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:06.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.349+0000 7fcd04d9a700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcce406ace0 0x7fcce406d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcd00072500 tx=0x7fccf0009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:06.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.353+0000 7fccf5ffb700 1 -- 172.21.15.97:0/1149536472 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fccfc0557b0 con 0x7fcd0006b360 2023-07-06T03:46:06.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.560+0000 7fcd077ff700 1 -- 172.21.15.97:0/1149536472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd000d10d0 con 0x7fcd0006b360 2023-07-06T03:46:06.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.560+0000 7fccf5ffb700 1 -- 172.21.15.97:0/1149536472 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fccfc088e60 con 0x7fcd0006b360 2023-07-06T03:46:06.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 -- 172.21.15.97:0/1149536472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcce406ace0 msgr2=0x7fcce406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcce406ace0 0x7fcce406d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcd00072500 tx=0x7fccf0009210 comp rx=0 tx=0).stop 2023-07-06T03:46:06.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 -- 172.21.15.97:0/1149536472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd0006b360 msgr2=0x7fcd0014d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd0006b360 0x7fcd0014d150 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fccfc014040 tx=0x7fccfc004750 comp rx=0 tx=0).stop 2023-07-06T03:46:06.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 -- 172.21.15.97:0/1149536472 shutdown_connections 2023-07-06T03:46:06.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcce406ace0 0x7fcce406d190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd0015c610 0x7fcd000d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd0006b360 0x7fcd0014d150 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 --2- 172.21.15.97:0/1149536472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd000d2210 0x7fcd001f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 -- 172.21.15.97:0/1149536472 >> 172.21.15.97:0/1149536472 conn(0x7fcd0006a6c0 msgr2=0x7fcd0015ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:06.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 -- 172.21.15.97:0/1149536472 shutdown_connections 2023-07-06T03:46:06.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.562+0000 7fcd077ff700 1 -- 172.21.15.97:0/1149536472 wait complete. 2023-07-06T03:46:06.568 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:46:06.582 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:46:06.583 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:06.634 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:06.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.827+0000 7f68543ff700 1 Processor -- start 2023-07-06T03:46:06.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.828+0000 7f68543ff700 1 -- start start 2023-07-06T03:46:06.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.829+0000 7f68543ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f684c0cd8e0 0x7f684c15e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:06.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.829+0000 7f68543ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0710d0 con 0x7f684c15b440 2023-07-06T03:46:06.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.829+0000 7f68543ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c071330 con 0x7f684c070c30 2023-07-06T03:46:06.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.829+0000 7f68543ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684c071610 con 0x7f684c0cd8e0 2023-07-06T03:46:06.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.829+0000 7f685299c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f684c15b440 0x7f684c06f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56792/0 (socket says 172.21.15.97:56792) 2023-07-06T03:46:06.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.829+0000 7f685299c700 1 -- 172.21.15.97:0/293710210 learned_addr learned my addr 172.21.15.97:0/293710210 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:06.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.829+0000 7f685219b700 1 --2- 172.21.15.97:0/293710210 >> v2:172.21.15.146:3300/0 conn(0x7f684c0cd8e0 0x7f684c15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:06.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f6851199700 1 -- 172.21.15.97:0/293710210 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233006479 0 0) 0x7f684c0710d0 con 0x7f684c15b440 2023-07-06T03:46:06.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f6851199700 1 -- 172.21.15.97:0/293710210 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68380035b0 con 0x7f684c15b440 2023-07-06T03:46:06.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f6851199700 1 -- 172.21.15.97:0/293710210 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596447226 0 0) 0x7f684c071330 con 0x7f684c070c30 2023-07-06T03:46:06.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f6851199700 1 -- 172.21.15.97:0/293710210 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f684c0710d0 con 0x7f684c070c30 2023-07-06T03:46:06.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f685219b700 1 -- 172.21.15.97:0/293710210 >> v1:172.21.15.146:6789/0 conn(0x7f684c070c30 legacy=0x7f684c0cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f685219b700 1 -- 172.21.15.97:0/293710210 >> v1:172.21.15.97:6789/0 conn(0x7f684c15b440 legacy=0x7f684c06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f6851199700 1 -- 172.21.15.97:0/293710210 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3224524632 0 0) 0x7f68380035b0 con 0x7f684c15b440 2023-07-06T03:46:06.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f685219b700 1 -- 172.21.15.97:0/293710210 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c15dd80 con 0x7f684c0cd8e0 2023-07-06T03:46:06.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f685219b700 1 --2- 172.21.15.97:0/293710210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f684c0cd8e0 0x7f684c15e860 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6848009840 tx=0x7f6848009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ec72989cfd60e4d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:06.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f6851199700 1 -- 172.21.15.97:0/293710210 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68440019f0 con 0x7f684c0cd8e0 2023-07-06T03:46:06.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f6851199700 1 -- 172.21.15.97:0/293710210 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6848004030 con 0x7f684c0cd8e0 2023-07-06T03:46:06.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.830+0000 7f6851199700 1 -- 172.21.15.97:0/293710210 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68480043c0 con 0x7f684c0cd8e0 2023-07-06T03:46:06.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.831+0000 7f68543ff700 1 -- 172.21.15.97:0/293710210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f684c0cd8e0 msgr2=0x7f684c15e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.831+0000 7f68543ff700 1 --2- 172.21.15.97:0/293710210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f684c0cd8e0 0x7f684c15e860 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6848009840 tx=0x7f6848009b50 comp rx=0 tx=0).stop 2023-07-06T03:46:06.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.831+0000 7f68543ff700 1 -- 172.21.15.97:0/293710210 shutdown_connections 2023-07-06T03:46:06.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.831+0000 7f68543ff700 1 --2- 172.21.15.97:0/293710210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f684c0cd8e0 0x7f684c15e860 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.831+0000 7f68543ff700 1 -- 172.21.15.97:0/293710210 >> 172.21.15.97:0/293710210 conn(0x7f684c0ce8c0 msgr2=0x7f684c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:06.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.831+0000 7f68543ff700 1 -- 172.21.15.97:0/293710210 shutdown_connections 2023-07-06T03:46:06.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.831+0000 7f68543ff700 1 -- 172.21.15.97:0/293710210 wait complete. 2023-07-06T03:46:06.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.833+0000 7f68543ff700 1 Processor -- start 2023-07-06T03:46:06.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.833+0000 7f68543ff700 1 -- start start 2023-07-06T03:46:06.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f68543ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c070c30 0x7f684c1f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:06.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f68543ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f684c0cd8e0 0x7f684c1e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:06.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f68543ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f684c15b440 0x7f684c1e5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:06.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f68543ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c140480 con 0x7f684c070c30 2023-07-06T03:46:06.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f68543ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c06db50 con 0x7f684c0cd8e0 2023-07-06T03:46:06.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f68543ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c13fe70 con 0x7f684c15b440 2023-07-06T03:46:06.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685219b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c070c30 0x7f684c1f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:06.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685219b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c070c30 0x7f684c1f2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54902/0 (socket says 172.21.15.97:54902) 2023-07-06T03:46:06.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685219b700 1 -- 172.21.15.97:0/222083204 learned_addr learned my addr 172.21.15.97:0/222083204 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:06.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685299c700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f684c15b440 0x7f684c1e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:06.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685199a700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f684c0cd8e0 0x7f684c1e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:06.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685219b700 1 -- 172.21.15.97:0/222083204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f684c15b440 msgr2=0x7f684c1e5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685219b700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f684c15b440 0x7f684c1e5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685219b700 1 -- 172.21.15.97:0/222083204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f684c0cd8e0 msgr2=0x7f684c1e4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:06.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685219b700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f684c0cd8e0 0x7f684c1e4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:06.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685219b700 1 -- 172.21.15.97:0/222083204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68480094a0 con 0x7f684c070c30 2023-07-06T03:46:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685199a700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f684c0cd8e0 0x7f684c1e4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.834+0000 7f685299c700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f684c15b440 0x7f684c1e5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.835+0000 7f685219b700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c070c30 0x7f684c1f2010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f68480090b0 tx=0x7f6848004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.835+0000 7f6842ffd700 1 -- 172.21.15.97:0/222083204 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848011830 con 0x7f684c070c30 2023-07-06T03:46:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.835+0000 7f6842ffd700 1 -- 172.21.15.97:0/222083204 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684801f070 con 0x7f684c070c30 2023-07-06T03:46:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.835+0000 7f68543ff700 1 -- 172.21.15.97:0/222083204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c06d870 con 0x7f684c070c30 2023-07-06T03:46:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.835+0000 7f6842ffd700 1 -- 172.21.15.97:0/222083204 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848011bf0 con 0x7f684c070c30 2023-07-06T03:46:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.835+0000 7f68543ff700 1 -- 172.21.15.97:0/222083204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c1f2550 con 0x7f684c070c30 2023-07-06T03:46:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.836+0000 7f68543ff700 1 -- 172.21.15.97:0/222083204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c070aa0 con 0x7f684c070c30 2023-07-06T03:46:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.837+0000 7f6842ffd700 1 -- 172.21.15.97:0/222083204 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6848011d50 con 0x7f684c070c30 2023-07-06T03:46:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.838+0000 7f6842ffd700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f683006ace0 0x7f683006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.838+0000 7f6842ffd700 1 -- 172.21.15.97:0/222083204 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6848021070 con 0x7f684c070c30 2023-07-06T03:46:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.838+0000 7f685199a700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f683006ace0 0x7f683006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.839+0000 7f685199a700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f683006ace0 0x7f683006d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f684c1e5760 tx=0x7f683c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:06.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:06.843+0000 7f6842ffd700 1 -- 172.21.15.97:0/222083204 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6848055ef0 con 0x7f684c070c30 2023-07-06T03:46:07.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.039+0000 7f68543ff700 1 -- 172.21.15.97:0/222083204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f684c04e340 con 0x7f684c070c30 2023-07-06T03:46:07.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.040+0000 7f6842ffd700 1 -- 172.21.15.97:0/222083204 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f684802b090 con 0x7f684c070c30 2023-07-06T03:46:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:07.045 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:07.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 -- 172.21.15.97:0/222083204 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f683006ace0 msgr2=0x7f683006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f683006ace0 0x7f683006d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f684c1e5760 tx=0x7f683c009040 comp rx=0 tx=0).stop 2023-07-06T03:46:07.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 -- 172.21.15.97:0/222083204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c070c30 msgr2=0x7f684c1f2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c070c30 0x7f684c1f2010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f68480090b0 tx=0x7f6848004750 comp rx=0 tx=0).stop 2023-07-06T03:46:07.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 -- 172.21.15.97:0/222083204 shutdown_connections 2023-07-06T03:46:07.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f683006ace0 0x7f683006d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f684c15b440 0x7f684c1e5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c070c30 0x7f684c1f2010 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 --2- 172.21.15.97:0/222083204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f684c0cd8e0 0x7f684c1e4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 -- 172.21.15.97:0/222083204 >> 172.21.15.97:0/222083204 conn(0x7f684c0ce8c0 msgr2=0x7f684c15c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:07.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 -- 172.21.15.97:0/222083204 shutdown_connections 2023-07-06T03:46:07.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.043+0000 7f68543ff700 1 -- 172.21.15.97:0/222083204 wait complete. 2023-07-06T03:46:07.064 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:46:07.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:46:07.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.259+0000 7f1fed1ff700 1 Processor -- start 2023-07-06T03:46:07.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.259+0000 7f1fed1ff700 1 -- start start 2023-07-06T03:46:07.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.260+0000 7f1fed1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fe815c610 0x7f1fe80b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:07.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.260+0000 7f1fed1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe80cff90 con 0x7f1fe806b360 2023-07-06T03:46:07.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.260+0000 7f1fed1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe80b5650 con 0x7f1fe80d2210 2023-07-06T03:46:07.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.260+0000 7f1fed1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80d0780 con 0x7f1fe815c610 2023-07-06T03:46:07.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.260+0000 7f1fdffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1fe806b360 0x7f1fe80b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56804/0 (socket says 172.21.15.97:56804) 2023-07-06T03:46:07.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.260+0000 7f1fdffff700 1 -- 172.21.15.97:0/658885866 learned_addr learned my addr 172.21.15.97:0/658885866 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:07.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.261+0000 7f1fe6d9d700 1 --2- 172.21.15.97:0/658885866 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fe815c610 0x7f1fe80b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:07.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.261+0000 7f1fe659c700 1 -- 172.21.15.97:0/658885866 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828057860 0 0) 0x7f1fe80cff90 con 0x7f1fe806b360 2023-07-06T03:46:07.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.261+0000 7f1fe659c700 1 -- 172.21.15.97:0/658885866 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fd40035b0 con 0x7f1fe806b360 2023-07-06T03:46:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.261+0000 7f1fe659c700 1 -- 172.21.15.97:0/658885866 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409521610 0 0) 0x7f1fe80b5650 con 0x7f1fe80d2210 2023-07-06T03:46:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.261+0000 7f1fe659c700 1 -- 172.21.15.97:0/658885866 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe80cff90 con 0x7f1fe80d2210 2023-07-06T03:46:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.261+0000 7f1fe6d9d700 1 -- 172.21.15.97:0/658885866 >> v1:172.21.15.146:6789/0 conn(0x7f1fe80d2210 legacy=0x7f1fe80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.261+0000 7f1fe6d9d700 1 -- 172.21.15.97:0/658885866 >> v1:172.21.15.97:6789/0 conn(0x7f1fe806b360 legacy=0x7f1fe80b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.261+0000 7f1fe6d9d700 1 -- 172.21.15.97:0/658885866 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80b5400 con 0x7f1fe815c610 2023-07-06T03:46:07.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.261+0000 7f1fe6d9d700 1 --2- 172.21.15.97:0/658885866 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fe815c610 0x7f1fe80b47b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009870 tx=0x7f1fd8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d29a8c83f6bd036 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:07.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.262+0000 7f1fe659c700 1 -- 172.21.15.97:0/658885866 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011830 con 0x7f1fe815c610 2023-07-06T03:46:07.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.262+0000 7f1fe659c700 1 -- 172.21.15.97:0/658885866 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8011990 con 0x7f1fe815c610 2023-07-06T03:46:07.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.262+0000 7f1fe659c700 1 -- 172.21.15.97:0/658885866 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011d20 con 0x7f1fe815c610 2023-07-06T03:46:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.262+0000 7f1fed1ff700 1 -- 172.21.15.97:0/658885866 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fe815c610 msgr2=0x7f1fe80b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.262+0000 7f1fed1ff700 1 --2- 172.21.15.97:0/658885866 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fe815c610 0x7f1fe80b47b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009870 tx=0x7f1fd8009b80 comp rx=0 tx=0).stop 2023-07-06T03:46:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.262+0000 7f1fed1ff700 1 -- 172.21.15.97:0/658885866 shutdown_connections 2023-07-06T03:46:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.262+0000 7f1fed1ff700 1 --2- 172.21.15.97:0/658885866 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fe815c610 0x7f1fe80b47b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.262+0000 7f1fed1ff700 1 -- 172.21.15.97:0/658885866 >> 172.21.15.97:0/658885866 conn(0x7f1fe806a6c0 msgr2=0x7f1fe806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.262+0000 7f1fed1ff700 1 -- 172.21.15.97:0/658885866 shutdown_connections 2023-07-06T03:46:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.262+0000 7f1fed1ff700 1 -- 172.21.15.97:0/658885866 wait complete. 2023-07-06T03:46:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.264+0000 7f1fed1ff700 1 Processor -- start 2023-07-06T03:46:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.264+0000 7f1fed1ff700 1 -- start start 2023-07-06T03:46:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.265+0000 7f1fed1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fe806b360 0x7f1fe80cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.265+0000 7f1fed1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe80d2210 0x7f1fe80c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.265+0000 7f1fed1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fe815c610 0x7f1fe81e5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:07.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.265+0000 7f1fed1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c9fc0 con 0x7f1fe80d2210 2023-07-06T03:46:07.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.265+0000 7f1fed1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80ca2a0 con 0x7f1fe806b360 2023-07-06T03:46:07.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.265+0000 7f1fed1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80ca580 con 0x7f1fe815c610 2023-07-06T03:46:07.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.265+0000 7f1fdffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe80d2210 0x7f1fe80c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:07.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.265+0000 7f1fdffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe80d2210 0x7f1fe80c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54912/0 (socket says 172.21.15.97:54912) 2023-07-06T03:46:07.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.265+0000 7f1fdffff700 1 -- 172.21.15.97:0/624180905 learned_addr learned my addr 172.21.15.97:0/624180905 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:07.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fe6d9d700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fe806b360 0x7f1fe80cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:07.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fe759e700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fe815c610 0x7f1fe81e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:07.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fdffff700 1 -- 172.21.15.97:0/624180905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fe815c610 msgr2=0x7f1fe81e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fdffff700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fe815c610 0x7f1fe81e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fdffff700 1 -- 172.21.15.97:0/624180905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fe806b360 msgr2=0x7f1fe80cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fdffff700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fe806b360 0x7f1fe80cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fdffff700 1 -- 172.21.15.97:0/624180905 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80094a0 con 0x7f1fe80d2210 2023-07-06T03:46:07.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fdffff700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe80d2210 0x7f1fe80c58d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0005910 tx=0x7f1fd000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:07.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fdf7fe700 1 -- 172.21.15.97:0/624180905 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0005d70 con 0x7f1fe80d2210 2023-07-06T03:46:07.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fdf7fe700 1 -- 172.21.15.97:0/624180905 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd00070d0 con 0x7f1fe80d2210 2023-07-06T03:46:07.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fdf7fe700 1 -- 172.21.15.97:0/624180905 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0007460 con 0x7f1fe80d2210 2023-07-06T03:46:07.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fed1ff700 1 -- 172.21.15.97:0/624180905 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe80ca860 con 0x7f1fe80d2210 2023-07-06T03:46:07.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.266+0000 7f1fed1ff700 1 -- 172.21.15.97:0/624180905 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe80cbbc0 con 0x7f1fe80d2210 2023-07-06T03:46:07.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.267+0000 7f1fed1ff700 1 -- 172.21.15.97:0/624180905 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe804e340 con 0x7f1fe80d2210 2023-07-06T03:46:07.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.268+0000 7f1fdf7fe700 1 -- 172.21.15.97:0/624180905 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1fd00075c0 con 0x7f1fe80d2210 2023-07-06T03:46:07.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.269+0000 7f1fdf7fe700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fc806ace0 0x7f1fc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:07.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.269+0000 7f1fdf7fe700 1 -- 172.21.15.97:0/624180905 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1fd0088940 con 0x7f1fe80d2210 2023-07-06T03:46:07.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.269+0000 7f1fe6d9d700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fc806ace0 0x7f1fc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:07.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.270+0000 7f1fe6d9d700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fc806ace0 0x7f1fc806d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80090b0 tx=0x7f1fd801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:07.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.274+0000 7f1fdf7fe700 1 -- 172.21.15.97:0/624180905 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1fd00539e0 con 0x7f1fe80d2210 2023-07-06T03:46:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.469+0000 7f1fed1ff700 1 -- 172.21.15.97:0/624180905 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1fe815a140 con 0x7f1fe80d2210 2023-07-06T03:46:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.469+0000 7f1fdf7fe700 1 -- 172.21.15.97:0/624180905 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1fd0057000 con 0x7f1fe80d2210 2023-07-06T03:46:07.473 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:46:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.471+0000 7f1fed1ff700 1 -- 172.21.15.97:0/624180905 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fc806ace0 msgr2=0x7f1fc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.471+0000 7f1fed1ff700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fc806ace0 0x7f1fc806d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80090b0 tx=0x7f1fd801c040 comp rx=0 tx=0).stop 2023-07-06T03:46:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.471+0000 7f1fed1ff700 1 -- 172.21.15.97:0/624180905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe80d2210 msgr2=0x7f1fe80c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.471+0000 7f1fed1ff700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe80d2210 0x7f1fe80c58d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0005910 tx=0x7f1fd000c9e0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.472+0000 7f1fed1ff700 1 -- 172.21.15.97:0/624180905 shutdown_connections 2023-07-06T03:46:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.472+0000 7f1fed1ff700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fc806ace0 0x7f1fc806d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.472+0000 7f1fed1ff700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fe815c610 0x7f1fe81e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.472+0000 7f1fed1ff700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fe806b360 0x7f1fe80cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.472+0000 7f1fed1ff700 1 --2- 172.21.15.97:0/624180905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe80d2210 0x7f1fe80c58d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.472+0000 7f1fed1ff700 1 -- 172.21.15.97:0/624180905 >> 172.21.15.97:0/624180905 conn(0x7f1fe806a6c0 msgr2=0x7f1fe815b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.472+0000 7f1fed1ff700 1 -- 172.21.15.97:0/624180905 shutdown_connections 2023-07-06T03:46:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.472+0000 7f1fed1ff700 1 -- 172.21.15.97:0/624180905 wait complete. 2023-07-06T03:46:07.493 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:46:07.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.689+0000 7fbeffdff700 1 Processor -- start 2023-07-06T03:46:07.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.689+0000 7fbeffdff700 1 -- start start 2023-07-06T03:46:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.691+0000 7fbeffdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbef80cbaa0 0x7fbef815a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.691+0000 7fbeffdff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef815b6e0 con 0x7fbef8154890 2023-07-06T03:46:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.691+0000 7fbeffdff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef815e0b0 con 0x7fbef815eb90 2023-07-06T03:46:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.691+0000 7fbeffdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef815e390 con 0x7fbef80cbaa0 2023-07-06T03:46:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.691+0000 7fbefdb9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbef8154890 0x7fbef80cb390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56810/0 (socket says 172.21.15.97:56810) 2023-07-06T03:46:07.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.691+0000 7fbefdb9b700 1 -- 172.21.15.97:0/2852023665 learned_addr learned my addr 172.21.15.97:0/2852023665 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:07.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.691+0000 7fbefd39a700 1 --2- 172.21.15.97:0/2852023665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbef80cbaa0 0x7fbef815a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:07.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.692+0000 7fbefcb99700 1 -- 172.21.15.97:0/2852023665 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428519411 0 0) 0x7fbef815b6e0 con 0x7fbef8154890 2023-07-06T03:46:07.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.692+0000 7fbefcb99700 1 -- 172.21.15.97:0/2852023665 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee40035b0 con 0x7fbef8154890 2023-07-06T03:46:07.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.692+0000 7fbefcb99700 1 -- 172.21.15.97:0/2852023665 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224383628 0 0) 0x7fbef815e0b0 con 0x7fbef815eb90 2023-07-06T03:46:07.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.692+0000 7fbefd39a700 1 -- 172.21.15.97:0/2852023665 >> v1:172.21.15.146:6789/0 conn(0x7fbef815eb90 legacy=0x7fbef815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.692+0000 7fbefd39a700 1 -- 172.21.15.97:0/2852023665 >> v1:172.21.15.97:6789/0 conn(0x7fbef8154890 legacy=0x7fbef80cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.692+0000 7fbefd39a700 1 -- 172.21.15.97:0/2852023665 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef815e670 con 0x7fbef80cbaa0 2023-07-06T03:46:07.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.692+0000 7fbefd39a700 1 --2- 172.21.15.97:0/2852023665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbef80cbaa0 0x7fbef815a940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009870 tx=0x7fbee8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638ebd6c5232b395 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:07.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.692+0000 7fbefcb99700 1 -- 172.21.15.97:0/2852023665 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8011830 con 0x7fbef80cbaa0 2023-07-06T03:46:07.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.693+0000 7fbefcb99700 1 -- 172.21.15.97:0/2852023665 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee8011990 con 0x7fbef80cbaa0 2023-07-06T03:46:07.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.693+0000 7fbefcb99700 1 -- 172.21.15.97:0/2852023665 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8011d20 con 0x7fbef80cbaa0 2023-07-06T03:46:07.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.693+0000 7fbeffdff700 1 -- 172.21.15.97:0/2852023665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbef80cbaa0 msgr2=0x7fbef815a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.693+0000 7fbeffdff700 1 --2- 172.21.15.97:0/2852023665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbef80cbaa0 0x7fbef815a940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009870 tx=0x7fbee8009b80 comp rx=0 tx=0).stop 2023-07-06T03:46:07.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.693+0000 7fbeffdff700 1 -- 172.21.15.97:0/2852023665 shutdown_connections 2023-07-06T03:46:07.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.693+0000 7fbeffdff700 1 --2- 172.21.15.97:0/2852023665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbef80cbaa0 0x7fbef815a940 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.693+0000 7fbeffdff700 1 -- 172.21.15.97:0/2852023665 >> 172.21.15.97:0/2852023665 conn(0x7fbef806a060 msgr2=0x7fbef806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.693+0000 7fbeffdff700 1 -- 172.21.15.97:0/2852023665 shutdown_connections 2023-07-06T03:46:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.693+0000 7fbeffdff700 1 -- 172.21.15.97:0/2852023665 wait complete. 2023-07-06T03:46:07.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.695+0000 7fbeffdff700 1 Processor -- start 2023-07-06T03:46:07.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.695+0000 7fbeffdff700 1 -- start start 2023-07-06T03:46:07.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.696+0000 7fbeffdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbef80cbaa0 0x7fbef806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:07.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.696+0000 7fbeffdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef8154890 0x7fbef8071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:07.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.696+0000 7fbeffdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbef815eb90 0x7fbef80cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:07.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.696+0000 7fbeffdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef81463a0 con 0x7fbef8154890 2023-07-06T03:46:07.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.696+0000 7fbeffdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef81465f0 con 0x7fbef80cbaa0 2023-07-06T03:46:07.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.696+0000 7fbeffdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef81468d0 con 0x7fbef815eb90 2023-07-06T03:46:07.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.696+0000 7fbefd39a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef8154890 0x7fbef8071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:07.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.696+0000 7fbefd39a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef8154890 0x7fbef8071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54926/0 (socket says 172.21.15.97:54926) 2023-07-06T03:46:07.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.696+0000 7fbefd39a700 1 -- 172.21.15.97:0/4222177648 learned_addr learned my addr 172.21.15.97:0/4222177648 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:07.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.696+0000 7fbefdb9b700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbef80cbaa0 0x7fbef806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:07.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.696+0000 7fbefe39c700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbef815eb90 0x7fbef80cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:07.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbefd39a700 1 -- 172.21.15.97:0/4222177648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbef815eb90 msgr2=0x7fbef80cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbefd39a700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbef815eb90 0x7fbef80cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbefd39a700 1 -- 172.21.15.97:0/4222177648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbef80cbaa0 msgr2=0x7fbef806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbefd39a700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbef80cbaa0 0x7fbef806f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbefd39a700 1 -- 172.21.15.97:0/4222177648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee80094a0 con 0x7fbef8154890 2023-07-06T03:46:07.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbefdb9b700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbef80cbaa0 0x7fbef806f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:07.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbefd39a700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef8154890 0x7fbef8071740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbee80090b0 tx=0x7fbee8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:07.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbeee7fc700 1 -- 172.21.15.97:0/4222177648 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee801a910 con 0x7fbef8154890 2023-07-06T03:46:07.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbeee7fc700 1 -- 172.21.15.97:0/4222177648 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee801aa70 con 0x7fbef8154890 2023-07-06T03:46:07.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbeffdff700 1 -- 172.21.15.97:0/4222177648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef8146bb0 con 0x7fbef8154890 2023-07-06T03:46:07.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbeee7fc700 1 -- 172.21.15.97:0/4222177648 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee801ade0 con 0x7fbef8154890 2023-07-06T03:46:07.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.697+0000 7fbeffdff700 1 -- 172.21.15.97:0/4222177648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef8147170 con 0x7fbef8154890 2023-07-06T03:46:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.698+0000 7fbeffdff700 1 -- 172.21.15.97:0/4222177648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef806bcd0 con 0x7fbef8154890 2023-07-06T03:46:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.699+0000 7fbeee7fc700 1 -- 172.21.15.97:0/4222177648 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbee8022070 con 0x7fbef8154890 2023-07-06T03:46:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.700+0000 7fbeee7fc700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbedc06ace0 0x7fbedc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.700+0000 7fbeee7fc700 1 -- 172.21.15.97:0/4222177648 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbee808a250 con 0x7fbef8154890 2023-07-06T03:46:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.701+0000 7fbefdb9b700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbedc06ace0 0x7fbedc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.701+0000 7fbefdb9b700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbedc06ace0 0x7fbedc06d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbef4005910 tx=0x7fbef4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.705+0000 7fbeee7fc700 1 -- 172.21.15.97:0/4222177648 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbee80553a0 con 0x7fbef8154890 2023-07-06T03:46:07.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.898+0000 7fbeffdff700 1 -- 172.21.15.97:0/4222177648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbef804e340 con 0x7fbef8154890 2023-07-06T03:46:07.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.898+0000 7fbeee7fc700 1 -- 172.21.15.97:0/4222177648 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbee80150a0 con 0x7fbef8154890 2023-07-06T03:46:07.902 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:46:07.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 -- 172.21.15.97:0/4222177648 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbedc06ace0 msgr2=0x7fbedc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbedc06ace0 0x7fbedc06d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbef4005910 tx=0x7fbef4009040 comp rx=0 tx=0).stop 2023-07-06T03:46:07.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 -- 172.21.15.97:0/4222177648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef8154890 msgr2=0x7fbef8071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:07.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef8154890 0x7fbef8071740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbee80090b0 tx=0x7fbee8004750 comp rx=0 tx=0).stop 2023-07-06T03:46:07.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 -- 172.21.15.97:0/4222177648 shutdown_connections 2023-07-06T03:46:07.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbedc06ace0 0x7fbedc06d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbef815eb90 0x7fbef80cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbef80cbaa0 0x7fbef806f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 --2- 172.21.15.97:0/4222177648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef8154890 0x7fbef8071740 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:07.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 -- 172.21.15.97:0/4222177648 >> 172.21.15.97:0/4222177648 conn(0x7fbef806a060 msgr2=0x7fbef80cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:07.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 -- 172.21.15.97:0/4222177648 shutdown_connections 2023-07-06T03:46:07.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:07.901+0000 7fbeffdff700 1 -- 172.21.15.97:0/4222177648 wait complete. 2023-07-06T03:46:07.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:46:08.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.116+0000 7f9537dff700 1 Processor -- start 2023-07-06T03:46:08.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.116+0000 7f9537dff700 1 -- start start 2023-07-06T03:46:08.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.117+0000 7f9537dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f953006b360 0x7f95300b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.117+0000 7f9537dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f95300d2210 0x7f95300d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.117+0000 7f9537dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95300cff90 con 0x7f953015c610 2023-07-06T03:46:08.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.117+0000 7f9537dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95300d0780 con 0x7f953006b360 2023-07-06T03:46:08.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.117+0000 7f9537dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f953015c9e0 con 0x7f95300d2210 2023-07-06T03:46:08.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f953539a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f953006b360 0x7f95300b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f953539a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f953006b360 0x7f95300b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54930/0 (socket says 172.21.15.97:54930) 2023-07-06T03:46:08.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f953539a700 1 -- 172.21.15.97:0/3323729015 learned_addr learned my addr 172.21.15.97:0/3323729015 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:08.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f9535b9b700 1 --1- 172.21.15.97:0/3323729015 >> v1:172.21.15.97:6789/0 conn(0x7f953015c610 0x7f95300b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56824/0 (socket says 172.21.15.97:56824) 2023-07-06T03:46:08.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f953539a700 1 -- 172.21.15.97:0/3323729015 >> v2:172.21.15.146:3300/0 conn(0x7f95300d2210 msgr2=0x7f95300d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:08.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f953639c700 1 --2- 172.21.15.97:0/3323729015 >> v2:172.21.15.146:3300/0 conn(0x7f95300d2210 0x7f95300d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f953539a700 1 --2- 172.21.15.97:0/3323729015 >> v2:172.21.15.146:3300/0 conn(0x7f95300d2210 0x7f95300d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f953539a700 1 -- 172.21.15.97:0/3323729015 >> v1:172.21.15.97:6789/0 conn(0x7f953015c610 legacy=0x7f95300b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f953539a700 1 -- 172.21.15.97:0/3323729015 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95300b5650 con 0x7f953006b360 2023-07-06T03:46:08.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f9534b99700 1 -- 172.21.15.97:0/3323729015 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527986077 0 0) 0x7f95300cff90 con 0x7f953015c610 2023-07-06T03:46:08.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f953639c700 1 --2- 172.21.15.97:0/3323729015 >> v2:172.21.15.146:3300/0 conn(0x7f95300d2210 0x7f95300d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:08.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.118+0000 7f953539a700 1 --2- 172.21.15.97:0/3323729015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953006b360 0x7f95300b4ec0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f95300d2bc0 tx=0x7f9520009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8008303c330ef437 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:08.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.119+0000 7f9534b99700 1 -- 172.21.15.97:0/3323729015 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000b860 con 0x7f953006b360 2023-07-06T03:46:08.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.119+0000 7f9534b99700 1 -- 172.21.15.97:0/3323729015 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952000b9c0 con 0x7f953006b360 2023-07-06T03:46:08.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.119+0000 7f9534b99700 1 -- 172.21.15.97:0/3323729015 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000bd50 con 0x7f953006b360 2023-07-06T03:46:08.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.119+0000 7f9537dff700 1 -- 172.21.15.97:0/3323729015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953006b360 msgr2=0x7f95300b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.119+0000 7f9537dff700 1 --2- 172.21.15.97:0/3323729015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953006b360 0x7f95300b4ec0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f95300d2bc0 tx=0x7f9520009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:08.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.119+0000 7f9537dff700 1 -- 172.21.15.97:0/3323729015 shutdown_connections 2023-07-06T03:46:08.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.119+0000 7f9537dff700 1 --2- 172.21.15.97:0/3323729015 >> v2:172.21.15.146:3300/0 conn(0x7f95300d2210 0x7f95300d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.119+0000 7f9537dff700 1 --2- 172.21.15.97:0/3323729015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953006b360 0x7f95300b4ec0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.119+0000 7f9537dff700 1 -- 172.21.15.97:0/3323729015 >> 172.21.15.97:0/3323729015 conn(0x7f953006a6c0 msgr2=0x7f953006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:08.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.119+0000 7f9537dff700 1 -- 172.21.15.97:0/3323729015 shutdown_connections 2023-07-06T03:46:08.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.119+0000 7f9537dff700 1 -- 172.21.15.97:0/3323729015 wait complete. 2023-07-06T03:46:08.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.121+0000 7f9537dff700 1 Processor -- start 2023-07-06T03:46:08.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.121+0000 7f9537dff700 1 -- start start 2023-07-06T03:46:08.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.122+0000 7f9537dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f953006b360 0x7f9530158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.122+0000 7f9537dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95300d2210 0x7f95300c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.122+0000 7f9537dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953015c610 0x7f95300cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.122+0000 7f9537dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9530157f20 con 0x7f953015c610 2023-07-06T03:46:08.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.122+0000 7f9537dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953015d6a0 con 0x7f953006b360 2023-07-06T03:46:08.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.122+0000 7f9537dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95300ccc50 con 0x7f95300d2210 2023-07-06T03:46:08.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.122+0000 7f953639c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953015c610 0x7f95300cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.122+0000 7f953639c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953015c610 0x7f95300cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54940/0 (socket says 172.21.15.97:54940) 2023-07-06T03:46:08.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.122+0000 7f953639c700 1 -- 172.21.15.97:0/479345956 learned_addr learned my addr 172.21.15.97:0/479345956 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:08.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f9535b9b700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f953006b360 0x7f9530158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f953539a700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95300d2210 0x7f95300c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f953639c700 1 -- 172.21.15.97:0/479345956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95300d2210 msgr2=0x7f95300c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f953639c700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95300d2210 0x7f95300c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f953639c700 1 -- 172.21.15.97:0/479345956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f953006b360 msgr2=0x7f9530158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f953639c700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f953006b360 0x7f9530158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f953639c700 1 -- 172.21.15.97:0/479345956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9520009550 con 0x7f953015c610 2023-07-06T03:46:08.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f953539a700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95300d2210 0x7f95300c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:08.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f953639c700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953015c610 0x7f95300cbf80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f952800cf20 tx=0x7f952800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:08.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f95267fc700 1 -- 172.21.15.97:0/479345956 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952800ae80 con 0x7f953015c610 2023-07-06T03:46:08.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f95267fc700 1 -- 172.21.15.97:0/479345956 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9528015440 con 0x7f953015c610 2023-07-06T03:46:08.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f95267fc700 1 -- 172.21.15.97:0/479345956 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95280157d0 con 0x7f953015c610 2023-07-06T03:46:08.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f9537dff700 1 -- 172.21.15.97:0/479345956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953015dc40 con 0x7f953015c610 2023-07-06T03:46:08.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.123+0000 7f9537dff700 1 -- 172.21.15.97:0/479345956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95300cc970 con 0x7f953015c610 2023-07-06T03:46:08.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.124+0000 7f9537dff700 1 -- 172.21.15.97:0/479345956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953015ddd0 con 0x7f953015c610 2023-07-06T03:46:08.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.125+0000 7f95267fc700 1 -- 172.21.15.97:0/479345956 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f952800d430 con 0x7f953015c610 2023-07-06T03:46:08.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.126+0000 7f95267fc700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f951406ace0 0x7f951406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.126+0000 7f95267fc700 1 -- 172.21.15.97:0/479345956 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9528089bf0 con 0x7f953015c610 2023-07-06T03:46:08.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.127+0000 7f9535b9b700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f951406ace0 0x7f951406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.127+0000 7f9535b9b700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f951406ace0 0x7f951406d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f952c000f60 tx=0x7f952c007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:08.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.131+0000 7f95267fc700 1 -- 172.21.15.97:0/479345956 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9528054b70 con 0x7f953015c610 2023-07-06T03:46:08.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.321+0000 7f9537dff700 1 -- 172.21.15.97:0/479345956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f953004e340 con 0x7f953015c610 2023-07-06T03:46:08.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.321+0000 7f95267fc700 1 -- 172.21.15.97:0/479345956 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f952801a020 con 0x7f953015c610 2023-07-06T03:46:08.324 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:46:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 -- 172.21.15.97:0/479345956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f951406ace0 msgr2=0x7f951406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f951406ace0 0x7f951406d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f952c000f60 tx=0x7f952c007c90 comp rx=0 tx=0).stop 2023-07-06T03:46:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 -- 172.21.15.97:0/479345956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953015c610 msgr2=0x7f95300cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953015c610 0x7f95300cbf80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f952800cf20 tx=0x7f952800a340 comp rx=0 tx=0).stop 2023-07-06T03:46:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 -- 172.21.15.97:0/479345956 shutdown_connections 2023-07-06T03:46:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f951406ace0 0x7f951406d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953015c610 0x7f95300cbf80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f953006b360 0x7f9530158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 --2- 172.21.15.97:0/479345956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95300d2210 0x7f95300c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 -- 172.21.15.97:0/479345956 >> 172.21.15.97:0/479345956 conn(0x7f953006a6c0 msgr2=0x7f95300d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 -- 172.21.15.97:0/479345956 shutdown_connections 2023-07-06T03:46:08.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.324+0000 7f9537dff700 1 -- 172.21.15.97:0/479345956 wait complete. 2023-07-06T03:46:08.344 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:46:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.539+0000 7f6096fff700 1 Processor -- start 2023-07-06T03:46:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.539+0000 7f6096fff700 1 -- start start 2023-07-06T03:46:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.540+0000 7f6096fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60900719c0 0x7f60900cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.540+0000 7f6096fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609015da10 con 0x7f609015c590 2023-07-06T03:46:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.540+0000 7f6096fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60900ccc30 con 0x7f609015c1c0 2023-07-06T03:46:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.540+0000 7f6096fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60900ccf10 con 0x7f60900719c0 2023-07-06T03:46:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.541+0000 7f608ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f609015c590 0x7f609015d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56830/0 (socket says 172.21.15.97:56830) 2023-07-06T03:46:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.541+0000 7f608ffff700 1 -- 172.21.15.97:0/651789488 learned_addr learned my addr 172.21.15.97:0/651789488 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.541+0000 7f609559c700 1 --2- 172.21.15.97:0/651789488 >> v2:172.21.15.97:3300/0 conn(0x7f60900719c0 0x7f60900cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.541+0000 7f608f7fe700 1 -- 172.21.15.97:0/651789488 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611145144 0 0) 0x7f607c001110 con 0x7f609015c590 2023-07-06T03:46:08.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.541+0000 7f608f7fe700 1 -- 172.21.15.97:0/651789488 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60800035b0 con 0x7f609015c590 2023-07-06T03:46:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.541+0000 7f608f7fe700 1 -- 172.21.15.97:0/651789488 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744754202 0 0) 0x7f60900ccc30 con 0x7f609015c1c0 2023-07-06T03:46:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.541+0000 7f608f7fe700 1 -- 172.21.15.97:0/651789488 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607c001110 con 0x7f609015c1c0 2023-07-06T03:46:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.541+0000 7f609559c700 1 -- 172.21.15.97:0/651789488 >> v1:172.21.15.146:6789/0 conn(0x7f609015c1c0 legacy=0x7f60900cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.541+0000 7f609559c700 1 -- 172.21.15.97:0/651789488 >> v1:172.21.15.97:6789/0 conn(0x7f609015c590 legacy=0x7f609015d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.541+0000 7f609559c700 1 -- 172.21.15.97:0/651789488 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60900cd1f0 con 0x7f60900719c0 2023-07-06T03:46:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.541+0000 7f609559c700 1 --2- 172.21.15.97:0/651789488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60900719c0 0x7f60900cb080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6088009d90 tx=0x7f608800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac77f53b3d2d558 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.542+0000 7f608f7fe700 1 -- 172.21.15.97:0/651789488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608800e070 con 0x7f60900719c0 2023-07-06T03:46:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.542+0000 7f608f7fe700 1 -- 172.21.15.97:0/651789488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608800bb20 con 0x7f60900719c0 2023-07-06T03:46:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.542+0000 7f608f7fe700 1 -- 172.21.15.97:0/651789488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088012430 con 0x7f60900719c0 2023-07-06T03:46:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.542+0000 7f6096fff700 1 -- 172.21.15.97:0/651789488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60900719c0 msgr2=0x7f60900cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.542+0000 7f6096fff700 1 --2- 172.21.15.97:0/651789488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60900719c0 0x7f60900cb080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6088009d90 tx=0x7f608800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.542+0000 7f6096fff700 1 -- 172.21.15.97:0/651789488 shutdown_connections 2023-07-06T03:46:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.542+0000 7f6096fff700 1 --2- 172.21.15.97:0/651789488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60900719c0 0x7f60900cb080 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.542+0000 7f6096fff700 1 -- 172.21.15.97:0/651789488 >> 172.21.15.97:0/651789488 conn(0x7f60900b1650 msgr2=0x7f60900b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:08.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.542+0000 7f6096fff700 1 -- 172.21.15.97:0/651789488 shutdown_connections 2023-07-06T03:46:08.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.542+0000 7f6096fff700 1 -- 172.21.15.97:0/651789488 wait complete. 2023-07-06T03:46:08.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.544+0000 7f6096fff700 1 Processor -- start 2023-07-06T03:46:08.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.544+0000 7f6096fff700 1 -- start start 2023-07-06T03:46:08.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.545+0000 7f6096fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60900719c0 0x7f609015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.545+0000 7f6096fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f609015c1c0 0x7f609015b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.545+0000 7f6096fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f609015c590 0x7f60901f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.545+0000 7f6096fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60901f3020 con 0x7f60900719c0 2023-07-06T03:46:08.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.545+0000 7f6096fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60901f2a60 con 0x7f609015c590 2023-07-06T03:46:08.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.545+0000 7f6096fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60901f2d40 con 0x7f609015c1c0 2023-07-06T03:46:08.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.545+0000 7f6094d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60900719c0 0x7f609015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.545+0000 7f6094d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60900719c0 0x7f609015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54954/0 (socket says 172.21.15.97:54954) 2023-07-06T03:46:08.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.545+0000 7f6094d9b700 1 -- 172.21.15.97:0/1229144700 learned_addr learned my addr 172.21.15.97:0/1229144700 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:08.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f609559c700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f609015c590 0x7f60901f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f608ffff700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f609015c1c0 0x7f609015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f6094d9b700 1 -- 172.21.15.97:0/1229144700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f609015c1c0 msgr2=0x7f609015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f6094d9b700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f609015c1c0 0x7f609015b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f6094d9b700 1 -- 172.21.15.97:0/1229144700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f609015c590 msgr2=0x7f60901f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f6094d9b700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f609015c590 0x7f60901f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f6094d9b700 1 -- 172.21.15.97:0/1229144700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6088009a70 con 0x7f60900719c0 2023-07-06T03:46:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f6094d9b700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60900719c0 0x7f609015f5a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f608400c520 tx=0x7f608400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f608d7fa700 1 -- 172.21.15.97:0/1229144700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60840052d0 con 0x7f60900719c0 2023-07-06T03:46:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f608d7fa700 1 -- 172.21.15.97:0/1229144700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6084005430 con 0x7f60900719c0 2023-07-06T03:46:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f608d7fa700 1 -- 172.21.15.97:0/1229144700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60840038c0 con 0x7f60900719c0 2023-07-06T03:46:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.546+0000 7f6096fff700 1 -- 172.21.15.97:0/1229144700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609015eda0 con 0x7f60900719c0 2023-07-06T03:46:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.547+0000 7f6096fff700 1 -- 172.21.15.97:0/1229144700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609015fae0 con 0x7f60900719c0 2023-07-06T03:46:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.547+0000 7f6096fff700 1 -- 172.21.15.97:0/1229144700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609015ec00 con 0x7f60900719c0 2023-07-06T03:46:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.548+0000 7f608d7fa700 1 -- 172.21.15.97:0/1229144700 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6084003a20 con 0x7f60900719c0 2023-07-06T03:46:08.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.549+0000 7f608d7fa700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f607406ace0 0x7f607406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.550+0000 7f608d7fa700 1 -- 172.21.15.97:0/1229144700 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f60840888e0 con 0x7f60900719c0 2023-07-06T03:46:08.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.550+0000 7f608ffff700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f607406ace0 0x7f607406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.550+0000 7f608ffff700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f607406ace0 0x7f607406d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f607c007fd0 tx=0x7f607c007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.554+0000 7f608d7fa700 1 -- 172.21.15.97:0/1229144700 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f609015ec00 con 0x7f60900719c0 2023-07-06T03:46:08.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.751+0000 7f6096fff700 1 -- 172.21.15.97:0/1229144700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f609015ec00 con 0x7f60900719c0 2023-07-06T03:46:08.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.752+0000 7f608d7fa700 1 -- 172.21.15.97:0/1229144700 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6084017020 con 0x7f60900719c0 2023-07-06T03:46:08.755 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:46:08.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 -- 172.21.15.97:0/1229144700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f607406ace0 msgr2=0x7f607406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f607406ace0 0x7f607406d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f607c007fd0 tx=0x7f607c007dc0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 -- 172.21.15.97:0/1229144700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60900719c0 msgr2=0x7f609015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60900719c0 0x7f609015f5a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f608400c520 tx=0x7f608400c830 comp rx=0 tx=0).stop 2023-07-06T03:46:08.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 -- 172.21.15.97:0/1229144700 shutdown_connections 2023-07-06T03:46:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f607406ace0 0x7f607406d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f609015c590 0x7f60901f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60900719c0 0x7f609015f5a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 --2- 172.21.15.97:0/1229144700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f609015c1c0 0x7f609015b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 -- 172.21.15.97:0/1229144700 >> 172.21.15.97:0/1229144700 conn(0x7f60900b1650 msgr2=0x7f609006f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 -- 172.21.15.97:0/1229144700 shutdown_connections 2023-07-06T03:46:08.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.754+0000 7f6096fff700 1 -- 172.21.15.97:0/1229144700 wait complete. 2023-07-06T03:46:08.774 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:46:08.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.968+0000 7fa28a3ff700 1 Processor -- start 2023-07-06T03:46:08.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.968+0000 7fa28a3ff700 1 -- start start 2023-07-06T03:46:08.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.969+0000 7fa28a3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28406b360 0x7fa2840b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.969+0000 7fa28a3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2840cff90 con 0x7fa2840d2210 2023-07-06T03:46:08.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.969+0000 7fa28a3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2840b5650 con 0x7fa28415c610 2023-07-06T03:46:08.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.969+0000 7fa28a3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840d0780 con 0x7fa28406b360 2023-07-06T03:46:08.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.969+0000 7fa28899c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa2840d2210 0x7fa2840d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56844/0 (socket says 172.21.15.97:56844) 2023-07-06T03:46:08.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.969+0000 7fa28899c700 1 -- 172.21.15.97:0/3504546308 learned_addr learned my addr 172.21.15.97:0/3504546308 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:08.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa27bfff700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28406b360 0x7fa2840b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406145569 0 0) 0x7fa2840cff90 con 0x7fa2840d2210 2023-07-06T03:46:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2700035b0 con 0x7fa2840d2210 2023-07-06T03:46:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2530484004 0 0) 0x7fa2700035b0 con 0x7fa2840d2210 2023-07-06T03:46:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28406b360 msgr2=0x7fa2840b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28406b360 0x7fa2840b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 >> v1:172.21.15.146:6789/0 conn(0x7fa28415c610 legacy=0x7fa2840b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2840b5400 con 0x7fa2840d2210 2023-07-06T03:46:08.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fa27c0019f0 con 0x7fa2840d2210 2023-07-06T03:46:08.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 >> v1:172.21.15.97:6789/0 conn(0x7fa2840d2210 legacy=0x7fa2840d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa270003a30 0x7fa270003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2700043d0 0x7fa270004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28406b360 0x7fa2700050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840d0780 con 0x7fa270003a30 2023-07-06T03:46:08.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27000ba10 con 0x7fa2700043d0 2023-07-06T03:46:08.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.970+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27000bba0 con 0x7fa28406b360 2023-07-06T03:46:08.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa283fff700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa270003a30 0x7fa270003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa28899c700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28406b360 0x7fa2700050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa27bfff700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2700043d0 0x7fa270004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa283fff700 1 -- 172.21.15.97:0/3504546308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28406b360 msgr2=0x7fa2700050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:08.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa283fff700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28406b360 0x7fa2700050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa283fff700 1 -- 172.21.15.97:0/3504546308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2700043d0 msgr2=0x7fa270004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:08.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa283fff700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2700043d0 0x7fa270004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa283fff700 1 -- 172.21.15.97:0/3504546308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2700035b0 con 0x7fa270003a30 2023-07-06T03:46:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa283fff700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa270003a30 0x7fa270003e00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa2740018b0 tx=0x7fa270005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d6ea0887e457dd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274001370 con 0x7fa270003a30 2023-07-06T03:46:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27400c6b0 con 0x7fa270003a30 2023-07-06T03:46:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.971+0000 7fa2837fe700 1 -- 172.21.15.97:0/3504546308 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400ca60 con 0x7fa270003a30 2023-07-06T03:46:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.972+0000 7fa28a3ff700 1 -- 172.21.15.97:0/3504546308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa270003a30 msgr2=0x7fa270003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.972+0000 7fa28a3ff700 1 --2- 172.21.15.97:0/3504546308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa270003a30 0x7fa270003e00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa2740018b0 tx=0x7fa270005880 comp rx=0 tx=0).stop 2023-07-06T03:46:08.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.972+0000 7fa28899c700 1 -- 172.21.15.97:0/3504546308 reap_dead start 2023-07-06T03:46:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.972+0000 7fa28a3ff700 1 -- 172.21.15.97:0/3504546308 shutdown_connections 2023-07-06T03:46:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.972+0000 7fa28a3ff700 1 -- 172.21.15.97:0/3504546308 >> 172.21.15.97:0/3504546308 conn(0x7fa28406a6c0 msgr2=0x7fa28406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.972+0000 7fa28a3ff700 1 -- 172.21.15.97:0/3504546308 shutdown_connections 2023-07-06T03:46:08.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.972+0000 7fa28a3ff700 1 -- 172.21.15.97:0/3504546308 wait complete. 2023-07-06T03:46:08.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.974+0000 7fa28a3ff700 1 Processor -- start 2023-07-06T03:46:08.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.975+0000 7fa28a3ff700 1 -- start start 2023-07-06T03:46:08.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.976+0000 7fa28a3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa270003a30 0x7fa284158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.976+0000 7fa28a3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28414ad00 0x7fa28414b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.976+0000 7fa28a3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28415d540 0x7fa28415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.976+0000 7fa28a3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840ca500 con 0x7fa28415d540 2023-07-06T03:46:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.976+0000 7fa28a3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28414b6b0 con 0x7fa270003a30 2023-07-06T03:46:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.976+0000 7fa28a3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2841e5e70 con 0x7fa28414ad00 2023-07-06T03:46:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.976+0000 7fa28899c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28415d540 0x7fa28415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.976+0000 7fa28899c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28415d540 0x7fa28415d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54978/0 (socket says 172.21.15.97:54978) 2023-07-06T03:46:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.976+0000 7fa28899c700 1 -- 172.21.15.97:0/4159543087 learned_addr learned my addr 172.21.15.97:0/4159543087 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.976+0000 7fa283fff700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa270003a30 0x7fa284158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.976+0000 7fa27bfff700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28414ad00 0x7fa28414b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.977+0000 7fa28899c700 1 -- 172.21.15.97:0/4159543087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28414ad00 msgr2=0x7fa28414b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.977+0000 7fa28899c700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28414ad00 0x7fa28414b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.977+0000 7fa28899c700 1 -- 172.21.15.97:0/4159543087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa270003a30 msgr2=0x7fa284158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:08.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.977+0000 7fa28899c700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa270003a30 0x7fa284158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:08.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.977+0000 7fa28899c700 1 -- 172.21.15.97:0/4159543087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28415df20 con 0x7fa28415d540 2023-07-06T03:46:08.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.977+0000 7fa28899c700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28415d540 0x7fa28415d9e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa2840cea10 tx=0x7fa27c00a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:08.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.977+0000 7fa2817fa700 1 -- 172.21.15.97:0/4159543087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00ad50 con 0x7fa28415d540 2023-07-06T03:46:08.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.977+0000 7fa2817fa700 1 -- 172.21.15.97:0/4159543087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c0133f0 con 0x7fa28415d540 2023-07-06T03:46:08.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.977+0000 7fa28a3ff700 1 -- 172.21.15.97:0/4159543087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28415e1c0 con 0x7fa28415d540 2023-07-06T03:46:08.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.977+0000 7fa2817fa700 1 -- 172.21.15.97:0/4159543087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0137a0 con 0x7fa28415d540 2023-07-06T03:46:08.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.977+0000 7fa28a3ff700 1 -- 172.21.15.97:0/4159543087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28415e780 con 0x7fa28415d540 2023-07-06T03:46:08.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.978+0000 7fa28a3ff700 1 -- 172.21.15.97:0/4159543087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2840ca670 con 0x7fa28415d540 2023-07-06T03:46:08.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.979+0000 7fa2817fa700 1 -- 172.21.15.97:0/4159543087 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa27c013900 con 0x7fa28415d540 2023-07-06T03:46:08.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.980+0000 7fa2817fa700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa26406ace0 0x7fa26406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:08.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.980+0000 7fa2817fa700 1 -- 172.21.15.97:0/4159543087 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa27c089e30 con 0x7fa28415d540 2023-07-06T03:46:08.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.980+0000 7fa283fff700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa26406ace0 0x7fa26406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:08.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.981+0000 7fa283fff700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa26406ace0 0x7fa26406d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa2740018b0 tx=0x7fa274005040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:08.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:08.985+0000 7fa2817fa700 1 -- 172.21.15.97:0/4159543087 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa2840ca670 con 0x7fa28415d540 2023-07-06T03:46:09.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.197+0000 7fa28a3ff700 1 -- 172.21.15.97:0/4159543087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa2840ca640 con 0x7fa28415d540 2023-07-06T03:46:09.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.198+0000 7fa2817fa700 1 -- 172.21.15.97:0/4159543087 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa27c054e40 con 0x7fa28415d540 2023-07-06T03:46:09.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 -- 172.21.15.97:0/4159543087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa26406ace0 msgr2=0x7fa26406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:09.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa26406ace0 0x7fa26406d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa2740018b0 tx=0x7fa274005040 comp rx=0 tx=0).stop 2023-07-06T03:46:09.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 -- 172.21.15.97:0/4159543087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28415d540 msgr2=0x7fa28415d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:09.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28415d540 0x7fa28415d9e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa2840cea10 tx=0x7fa27c00a640 comp rx=0 tx=0).stop 2023-07-06T03:46:09.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 -- 172.21.15.97:0/4159543087 shutdown_connections 2023-07-06T03:46:09.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa26406ace0 0x7fa26406d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa28415d540 0x7fa28415d9e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa270003a30 0x7fa284158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 --2- 172.21.15.97:0/4159543087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa28414ad00 0x7fa28414b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 -- 172.21.15.97:0/4159543087 >> 172.21.15.97:0/4159543087 conn(0x7fa28406a6c0 msgr2=0x7fa28406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:09.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 -- 172.21.15.97:0/4159543087 shutdown_connections 2023-07-06T03:46:09.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.200+0000 7fa28a3ff700 1 -- 172.21.15.97:0/4159543087 wait complete. 2023-07-06T03:46:09.205 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:46:09.220 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2023-07-06T03:46:09.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:09.221 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-06T03:46:09.299 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:46:09.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:46:09.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:09.300 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T03:46:09.355 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:46:09.355 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:09.401 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:46:09.402 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:09.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.718+0000 7f3a157ff700 1 Processor -- start 2023-07-06T03:46:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.719+0000 7f3a157ff700 1 -- start start 2023-07-06T03:46:09.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a157ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a1015c520 0x7f3a100cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a157ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3a10160d00 0x7f3a100b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a157ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a100cdae0 con 0x7f3a10160180 2023-07-06T03:46:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a157ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a100cd8d0 con 0x7f3a10160d00 2023-07-06T03:46:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a157ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100b6640 con 0x7f3a1015c520 2023-07-06T03:46:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a0f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3a10160d00 0x7f3a100b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a0e7fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3a10160180 0x7f3a101605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56860/0 (socket says 172.21.15.97:56860) 2023-07-06T03:46:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a0e7fc700 1 -- 172.21.15.97:0/2179485829 learned_addr learned my addr 172.21.15.97:0/2179485829 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a0effd700 1 --2- 172.21.15.97:0/2179485829 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a1015c520 0x7f3a100cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a0f7fe700 1 -- 172.21.15.97:0/2179485829 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a1015c520 msgr2=0x7f3a100cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a0f7fe700 1 --2- 172.21.15.97:0/2179485829 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a1015c520 0x7f3a100cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a0f7fe700 1 -- 172.21.15.97:0/2179485829 >> v1:172.21.15.97:6789/0 conn(0x7f3a10160180 legacy=0x7f3a101605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a0dffb700 1 -- 172.21.15.97:0/2179485829 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546017083 0 0) 0x7f3a100cdae0 con 0x7f3a10160180 2023-07-06T03:46:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.720+0000 7f3a0f7fe700 1 -- 172.21.15.97:0/2179485829 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100b6920 con 0x7f3a10160d00 2023-07-06T03:46:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.721+0000 7f3a0f7fe700 1 --2- 172.21.15.97:0/2179485829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a10160d00 0x7f3a100b7400 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009d90 tx=0x7f3a0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe076fa63c4c5056 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.721+0000 7f3a0dffb700 1 -- 172.21.15.97:0/2179485829 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0800e070 con 0x7f3a10160d00 2023-07-06T03:46:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.721+0000 7f3a0dffb700 1 -- 172.21.15.97:0/2179485829 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0800bb20 con 0x7f3a10160d00 2023-07-06T03:46:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.721+0000 7f3a0dffb700 1 -- 172.21.15.97:0/2179485829 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08012430 con 0x7f3a10160d00 2023-07-06T03:46:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.721+0000 7f3a157ff700 1 -- 172.21.15.97:0/2179485829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a10160d00 msgr2=0x7f3a100b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.721+0000 7f3a157ff700 1 --2- 172.21.15.97:0/2179485829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a10160d00 0x7f3a100b7400 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009d90 tx=0x7f3a0800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:09.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.722+0000 7f3a157ff700 1 -- 172.21.15.97:0/2179485829 shutdown_connections 2023-07-06T03:46:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.722+0000 7f3a157ff700 1 --2- 172.21.15.97:0/2179485829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a10160d00 0x7f3a100b7400 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.722+0000 7f3a157ff700 1 --2- 172.21.15.97:0/2179485829 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a1015c520 0x7f3a100cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.722+0000 7f3a157ff700 1 -- 172.21.15.97:0/2179485829 >> 172.21.15.97:0/2179485829 conn(0x7f3a10071790 msgr2=0x7f3a10071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.722+0000 7f3a157ff700 1 -- 172.21.15.97:0/2179485829 shutdown_connections 2023-07-06T03:46:09.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.722+0000 7f3a157ff700 1 -- 172.21.15.97:0/2179485829 wait complete. 2023-07-06T03:46:09.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.723+0000 7f3a157ff700 1 Processor -- start 2023-07-06T03:46:09.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.723+0000 7f3a157ff700 1 -- start start 2023-07-06T03:46:09.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.724+0000 7f3a157ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1015c520 0x7f3a1014b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:09.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.724+0000 7f3a157ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a10160180 0x7f3a100cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:09.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.724+0000 7f3a157ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a10160d00 0x7f3a10158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:09.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.724+0000 7f3a157ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100d13e0 con 0x7f3a1015c520 2023-07-06T03:46:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.724+0000 7f3a157ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1014b6a0 con 0x7f3a10160180 2023-07-06T03:46:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.724+0000 7f3a157ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100cb8e0 con 0x7f3a10160d00 2023-07-06T03:46:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f3a0effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1015c520 0x7f3a1014b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f3a0effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1015c520 0x7f3a1014b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54994/0 (socket says 172.21.15.97:54994) 2023-07-06T03:46:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f3a0effd700 1 -- 172.21.15.97:0/3665699325 learned_addr learned my addr 172.21.15.97:0/3665699325 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f3a0e7fc700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a10160180 0x7f3a100cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f3a0f7fe700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a10160d00 0x7f3a10158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f3a0effd700 1 -- 172.21.15.97:0/3665699325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a10160d00 msgr2=0x7f3a10158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f3a0effd700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a10160d00 0x7f3a10158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f3a0effd700 1 -- 172.21.15.97:0/3665699325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a10160180 msgr2=0x7f3a100cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f3a0effd700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a10160180 0x7f3a100cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f3a0effd700 1 -- 172.21.15.97:0/3665699325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a08009a70 con 0x7f3a1015c520 2023-07-06T03:46:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f3a0effd700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1015c520 0x7f3a1014b160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000e870 tx=0x7f3a0000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f39f77fe700 1 -- 172.21.15.97:0/3665699325 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00009590 con 0x7f3a1015c520 2023-07-06T03:46:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.725+0000 7f39f77fe700 1 -- 172.21.15.97:0/3665699325 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a00013680 con 0x7f3a1015c520 2023-07-06T03:46:09.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.726+0000 7f3a157ff700 1 -- 172.21.15.97:0/3665699325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a100cbbc0 con 0x7f3a1015c520 2023-07-06T03:46:09.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.726+0000 7f39f77fe700 1 -- 172.21.15.97:0/3665699325 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a000139f0 con 0x7f3a1015c520 2023-07-06T03:46:09.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.726+0000 7f3a157ff700 1 -- 172.21.15.97:0/3665699325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a100cc180 con 0x7f3a1015c520 2023-07-06T03:46:09.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.727+0000 7f3a157ff700 1 -- 172.21.15.97:0/3665699325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1004e340 con 0x7f3a1015c520 2023-07-06T03:46:09.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.728+0000 7f39f77fe700 1 -- 172.21.15.97:0/3665699325 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a00013b50 con 0x7f3a1015c520 2023-07-06T03:46:09.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.729+0000 7f39f77fe700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f39fc06ace0 0x7f39fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:09.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.729+0000 7f39f77fe700 1 -- 172.21.15.97:0/3665699325 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a000893e0 con 0x7f3a1015c520 2023-07-06T03:46:09.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.729+0000 7f3a0e7fc700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f39fc06ace0 0x7f39fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:09.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.729+0000 7f3a0e7fc700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f39fc06ace0 0x7f39fc06d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3a100d1170 tx=0x7f3a04007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.733+0000 7f39f77fe700 1 -- 172.21.15.97:0/3665699325 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3a00057e80 con 0x7f3a1015c520 2023-07-06T03:46:09.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.928+0000 7f3a157ff700 1 -- 172.21.15.97:0/3665699325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a100b4f40 con 0x7f3a1015c520 2023-07-06T03:46:09.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.929+0000 7f39f77fe700 1 -- 172.21.15.97:0/3665699325 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f3a00057a10 con 0x7f3a1015c520 2023-07-06T03:46:09.933 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:09.934 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 -- 172.21.15.97:0/3665699325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f39fc06ace0 msgr2=0x7f39fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:09.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f39fc06ace0 0x7f39fc06d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3a100d1170 tx=0x7f3a04007e70 comp rx=0 tx=0).stop 2023-07-06T03:46:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 -- 172.21.15.97:0/3665699325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1015c520 msgr2=0x7f3a1014b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1015c520 0x7f3a1014b160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000e870 tx=0x7f3a0000eb80 comp rx=0 tx=0).stop 2023-07-06T03:46:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 -- 172.21.15.97:0/3665699325 shutdown_connections 2023-07-06T03:46:09.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f39fc06ace0 0x7f39fc06d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a10160d00 0x7f3a10158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1015c520 0x7f3a1014b160 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 --2- 172.21.15.97:0/3665699325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a10160180 0x7f3a100cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 -- 172.21.15.97:0/3665699325 >> 172.21.15.97:0/3665699325 conn(0x7f3a10071790 msgr2=0x7f3a10071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 -- 172.21.15.97:0/3665699325 shutdown_connections 2023-07-06T03:46:09.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:09.932+0000 7f3a157ff700 1 -- 172.21.15.97:0/3665699325 wait complete. 2023-07-06T03:46:09.953 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:46:09.954 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:46:10.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.149+0000 7f47869ff700 1 Processor -- start 2023-07-06T03:46:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.149+0000 7f47869ff700 1 -- start start 2023-07-06T03:46:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.150+0000 7f47869ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f478015c610 0x7f47800b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.150+0000 7f47869ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47800d2210 0x7f47800d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.150+0000 7f47869ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47800cff90 con 0x7f478006b360 2023-07-06T03:46:10.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.150+0000 7f47869ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47800d0780 con 0x7f47800d2210 2023-07-06T03:46:10.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.150+0000 7f47869ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478015c9e0 con 0x7f478015c610 2023-07-06T03:46:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.150+0000 7f4784f9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47800d2210 0x7f47800d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.150+0000 7f4784f9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47800d2210 0x7f47800d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40642/0 (socket says 172.21.15.97:40642) 2023-07-06T03:46:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.150+0000 7f4784f9c700 1 -- 172.21.15.97:0/692020367 learned_addr learned my addr 172.21.15.97:0/692020367 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.151+0000 7f477ffff700 1 --2- 172.21.15.97:0/692020367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f478015c610 0x7f47800b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.151+0000 7f4784f9c700 1 -- 172.21.15.97:0/692020367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f478015c610 msgr2=0x7f47800b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.151+0000 7f4784f9c700 1 --2- 172.21.15.97:0/692020367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f478015c610 0x7f47800b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.151+0000 7f4784f9c700 1 -- 172.21.15.97:0/692020367 >> v1:172.21.15.146:6789/0 conn(0x7f478006b360 legacy=0x7f47800b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.151+0000 7f4784f9c700 1 -- 172.21.15.97:0/692020367 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47800b5650 con 0x7f47800d2210 2023-07-06T03:46:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.151+0000 7f477ffff700 1 --2- 172.21.15.97:0/692020367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f478015c610 0x7f47800b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.151+0000 7f4784f9c700 1 --2- 172.21.15.97:0/692020367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47800d2210 0x7f47800d1310 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4778009e80 tx=0x7f477800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8473b5e57f88d50 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.151+0000 7f477f7fe700 1 -- 172.21.15.97:0/692020367 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477800e070 con 0x7f47800d2210 2023-07-06T03:46:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.151+0000 7f477f7fe700 1 -- 172.21.15.97:0/692020367 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477800b980 con 0x7f47800d2210 2023-07-06T03:46:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.151+0000 7f477f7fe700 1 -- 172.21.15.97:0/692020367 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477800bd30 con 0x7f47800d2210 2023-07-06T03:46:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.152+0000 7f47869ff700 1 -- 172.21.15.97:0/692020367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47800d2210 msgr2=0x7f47800d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:10.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.152+0000 7f47869ff700 1 --2- 172.21.15.97:0/692020367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47800d2210 0x7f47800d1310 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4778009e80 tx=0x7f477800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:10.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.152+0000 7f47869ff700 1 -- 172.21.15.97:0/692020367 shutdown_connections 2023-07-06T03:46:10.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.152+0000 7f47869ff700 1 --2- 172.21.15.97:0/692020367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47800d2210 0x7f47800d1310 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.152+0000 7f47869ff700 1 --2- 172.21.15.97:0/692020367 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f478015c610 0x7f47800b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.152+0000 7f47869ff700 1 -- 172.21.15.97:0/692020367 >> 172.21.15.97:0/692020367 conn(0x7f478006a6c0 msgr2=0x7f478006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:10.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.152+0000 7f47869ff700 1 -- 172.21.15.97:0/692020367 shutdown_connections 2023-07-06T03:46:10.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.152+0000 7f47869ff700 1 -- 172.21.15.97:0/692020367 wait complete. 2023-07-06T03:46:10.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.154+0000 7f47869ff700 1 Processor -- start 2023-07-06T03:46:10.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.154+0000 7f47869ff700 1 -- start start 2023-07-06T03:46:10.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f47869ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f478006b360 0x7f478014b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f47869ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47800d2210 0x7f478015f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f47869ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478015c610 0x7f478015f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f47869ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4780158b40 con 0x7f478015c610 2023-07-06T03:46:10.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f47869ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47800c96c0 con 0x7f47800d2210 2023-07-06T03:46:10.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f47869ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47800ca8b0 con 0x7f478006b360 2023-07-06T03:46:10.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f4784f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478015c610 0x7f478015f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f4784f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478015c610 0x7f478015f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40650/0 (socket says 172.21.15.97:40650) 2023-07-06T03:46:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f4784f9c700 1 -- 172.21.15.97:0/3900918911 learned_addr learned my addr 172.21.15.97:0/3900918911 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f4777fff700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47800d2210 0x7f478015f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f477ffff700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f478006b360 0x7f478014b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f4784f9c700 1 -- 172.21.15.97:0/3900918911 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f478006b360 msgr2=0x7f478014b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f4784f9c700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f478006b360 0x7f478014b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f4784f9c700 1 -- 172.21.15.97:0/3900918911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47800d2210 msgr2=0x7f478015f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f4784f9c700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47800d2210 0x7f478015f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.155+0000 7f4784f9c700 1 -- 172.21.15.97:0/3900918911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778009b30 con 0x7f478015c610 2023-07-06T03:46:10.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.156+0000 7f4784f9c700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478015c610 0x7f478015f540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f478008b120 tx=0x7f4778004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:10.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.156+0000 7f477d7fa700 1 -- 172.21.15.97:0/3900918911 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778004960 con 0x7f478015c610 2023-07-06T03:46:10.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.156+0000 7f477d7fa700 1 -- 172.21.15.97:0/3900918911 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47780099b0 con 0x7f478015c610 2023-07-06T03:46:10.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.156+0000 7f47869ff700 1 -- 172.21.15.97:0/3900918911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47800cab90 con 0x7f478015c610 2023-07-06T03:46:10.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.156+0000 7f477d7fa700 1 -- 172.21.15.97:0/3900918911 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477801c680 con 0x7f478015c610 2023-07-06T03:46:10.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.156+0000 7f47869ff700 1 -- 172.21.15.97:0/3900918911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47800cb150 con 0x7f478015c610 2023-07-06T03:46:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.157+0000 7f47869ff700 1 -- 172.21.15.97:0/3900918911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47800cb340 con 0x7f478015c610 2023-07-06T03:46:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.158+0000 7f477d7fa700 1 -- 172.21.15.97:0/3900918911 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f477801c7e0 con 0x7f478015c610 2023-07-06T03:46:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.159+0000 7f477d7fa700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f476006ace0 0x7f476006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.159+0000 7f477d7fa700 1 -- 172.21.15.97:0/3900918911 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f477808abd0 con 0x7f478015c610 2023-07-06T03:46:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.159+0000 7f477ffff700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f476006ace0 0x7f476006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.160+0000 7f477ffff700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f476006ace0 0x7f476006d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4770006fd0 tx=0x7f4770008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:10.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.163+0000 7f477d7fa700 1 -- 172.21.15.97:0/3900918911 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4778059670 con 0x7f478015c610 2023-07-06T03:46:10.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.356+0000 7f47869ff700 1 -- 172.21.15.97:0/3900918911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f478004e340 con 0x7f478015c610 2023-07-06T03:46:10.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.356+0000 7f477d7fa700 1 -- 172.21.15.97:0/3900918911 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4778059200 con 0x7f478015c610 2023-07-06T03:46:10.359 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:46:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 -- 172.21.15.97:0/3900918911 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f476006ace0 msgr2=0x7f476006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f476006ace0 0x7f476006d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4770006fd0 tx=0x7f4770008040 comp rx=0 tx=0).stop 2023-07-06T03:46:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 -- 172.21.15.97:0/3900918911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478015c610 msgr2=0x7f478015f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478015c610 0x7f478015f540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f478008b120 tx=0x7f4778004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 -- 172.21.15.97:0/3900918911 shutdown_connections 2023-07-06T03:46:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f476006ace0 0x7f476006d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478015c610 0x7f478015f540 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f478006b360 0x7f478014b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 --2- 172.21.15.97:0/3900918911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47800d2210 0x7f478015f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 -- 172.21.15.97:0/3900918911 >> 172.21.15.97:0/3900918911 conn(0x7f478006a6c0 msgr2=0x7f478015ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:10.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 -- 172.21.15.97:0/3900918911 shutdown_connections 2023-07-06T03:46:10.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.359+0000 7f47869ff700 1 -- 172.21.15.97:0/3900918911 wait complete. 2023-07-06T03:46:10.380 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:46:10.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.573+0000 7fe8ab7ff700 1 Processor -- start 2023-07-06T03:46:10.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.573+0000 7fe8ab7ff700 1 -- start start 2023-07-06T03:46:10.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.574+0000 7fe8ab7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a415c610 0x7fe8a40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.574+0000 7fe8ab7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe8a40d2210 0x7fe8a40d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.574+0000 7fe8ab7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a40cff90 con 0x7fe8a406b360 2023-07-06T03:46:10.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.574+0000 7fe8ab7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a40d0780 con 0x7fe8a40d2210 2023-07-06T03:46:10.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.574+0000 7fe8ab7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a415c9e0 con 0x7fe8a415c610 2023-07-06T03:46:10.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.575+0000 7fe8a959b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a415c610 0x7fe8a40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.575+0000 7fe8a959b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a415c610 0x7fe8a40b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:35212/0 (socket says 172.21.15.97:35212) 2023-07-06T03:46:10.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.575+0000 7fe8a9d9c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe8a40d2210 0x7fe8a40d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.575+0000 7fe8a959b700 1 -- 172.21.15.97:0/2379477925 learned_addr learned my addr 172.21.15.97:0/2379477925 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:10.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.575+0000 7fe8a959b700 1 -- 172.21.15.97:0/2379477925 >> v2:172.21.15.146:3300/0 conn(0x7fe8a40d2210 msgr2=0x7fe8a40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:10.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.575+0000 7fe8a959b700 1 --2- 172.21.15.97:0/2379477925 >> v2:172.21.15.146:3300/0 conn(0x7fe8a40d2210 0x7fe8a40d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.575+0000 7fe8a959b700 1 -- 172.21.15.97:0/2379477925 >> v1:172.21.15.146:6789/0 conn(0x7fe8a406b360 legacy=0x7fe8a40b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:10.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.575+0000 7fe8a959b700 1 -- 172.21.15.97:0/2379477925 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40b5650 con 0x7fe8a415c610 2023-07-06T03:46:10.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.576+0000 7fe8a959b700 1 --2- 172.21.15.97:0/2379477925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a415c610 0x7fe8a40b47b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00098f0 tx=0x7fe8a0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b3a42afeefbae1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:10.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.576+0000 7fe89bfff700 1 -- 172.21.15.97:0/2379477925 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0011830 con 0x7fe8a415c610 2023-07-06T03:46:10.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.576+0000 7fe89bfff700 1 -- 172.21.15.97:0/2379477925 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0011990 con 0x7fe8a415c610 2023-07-06T03:46:10.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.576+0000 7fe89bfff700 1 -- 172.21.15.97:0/2379477925 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0011d20 con 0x7fe8a415c610 2023-07-06T03:46:10.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.576+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/2379477925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a415c610 msgr2=0x7fe8a40b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:10.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.576+0000 7fe8ab7ff700 1 --2- 172.21.15.97:0/2379477925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a415c610 0x7fe8a40b47b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00098f0 tx=0x7fe8a0009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:10.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.577+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/2379477925 shutdown_connections 2023-07-06T03:46:10.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.577+0000 7fe8ab7ff700 1 --2- 172.21.15.97:0/2379477925 >> v2:172.21.15.146:3300/0 conn(0x7fe8a40d2210 0x7fe8a40d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.577+0000 7fe8ab7ff700 1 --2- 172.21.15.97:0/2379477925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a415c610 0x7fe8a40b47b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.577+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/2379477925 >> 172.21.15.97:0/2379477925 conn(0x7fe8a406a6c0 msgr2=0x7fe8a406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:10.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.577+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/2379477925 shutdown_connections 2023-07-06T03:46:10.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.577+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/2379477925 wait complete. 2023-07-06T03:46:10.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.578+0000 7fe8ab7ff700 1 Processor -- start 2023-07-06T03:46:10.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.578+0000 7fe8ab7ff700 1 -- start start 2023-07-06T03:46:10.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.579+0000 7fe8ab7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a406b360 0x7fe8a40ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.579+0000 7fe8ab7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8a40d2210 0x7fe8a40c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.579+0000 7fe8ab7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8a415c610 0x7fe8a415e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.579+0000 7fe8ab7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40c6910 con 0x7fe8a40d2210 2023-07-06T03:46:10.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.579+0000 7fe8ab7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40cadf0 con 0x7fe8a415c610 2023-07-06T03:46:10.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.579+0000 7fe8ab7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40cb0d0 con 0x7fe8a406b360 2023-07-06T03:46:10.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a8d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8a40d2210 0x7fe8a40c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a8d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8a40d2210 0x7fe8a40c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40654/0 (socket says 172.21.15.97:40654) 2023-07-06T03:46:10.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a8d9a700 1 -- 172.21.15.97:0/3477937574 learned_addr learned my addr 172.21.15.97:0/3477937574 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:10.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a959b700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a406b360 0x7fe8a40ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a9d9c700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8a415c610 0x7fe8a415e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a8d9a700 1 -- 172.21.15.97:0/3477937574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a406b360 msgr2=0x7fe8a40ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:10.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a8d9a700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a406b360 0x7fe8a40ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a8d9a700 1 -- 172.21.15.97:0/3477937574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8a415c610 msgr2=0x7fe8a415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:10.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a8d9a700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8a415c610 0x7fe8a415e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a8d9a700 1 -- 172.21.15.97:0/3477937574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a0009550 con 0x7fe8a40d2210 2023-07-06T03:46:10.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a959b700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a406b360 0x7fe8a40ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:46:10.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe8a8d9a700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8a40d2210 0x7fe8a40c90a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c420 tx=0x7fe89400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:10.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.580+0000 7fe899ffb700 1 -- 172.21.15.97:0/3477937574 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940052d0 con 0x7fe8a40d2210 2023-07-06T03:46:10.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.581+0000 7fe899ffb700 1 -- 172.21.15.97:0/3477937574 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe894005430 con 0x7fe8a40d2210 2023-07-06T03:46:10.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.581+0000 7fe899ffb700 1 -- 172.21.15.97:0/3477937574 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940038c0 con 0x7fe8a40d2210 2023-07-06T03:46:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.581+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/3477937574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a40cb3b0 con 0x7fe8a40d2210 2023-07-06T03:46:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.581+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/3477937574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a40c95e0 con 0x7fe8a40d2210 2023-07-06T03:46:10.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.582+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/3477937574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a40cb8d0 con 0x7fe8a40d2210 2023-07-06T03:46:10.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.583+0000 7fe899ffb700 1 -- 172.21.15.97:0/3477937574 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe894003a20 con 0x7fe8a40d2210 2023-07-06T03:46:10.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.584+0000 7fe899ffb700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe88806ace0 0x7fe88806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:10.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.584+0000 7fe899ffb700 1 -- 172.21.15.97:0/3477937574 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe8940887e0 con 0x7fe8a40d2210 2023-07-06T03:46:10.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.584+0000 7fe8a959b700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe88806ace0 0x7fe88806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:10.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.585+0000 7fe8a959b700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe88806ace0 0x7fe88806d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0000c00 tx=0x7fe8a0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:10.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.588+0000 7fe899ffb700 1 -- 172.21.15.97:0/3477937574 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe8a40cb8d0 con 0x7fe8a40d2210 2023-07-06T03:46:10.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.782+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/3477937574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe8a40cb8d0 con 0x7fe8a40d2210 2023-07-06T03:46:10.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.783+0000 7fe899ffb700 1 -- 172.21.15.97:0/3477937574 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe8a40cb8d0 con 0x7fe8a40d2210 2023-07-06T03:46:10.786 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:46:10.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/3477937574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe88806ace0 msgr2=0x7fe88806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe88806ace0 0x7fe88806d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0000c00 tx=0x7fe8a0008030 comp rx=0 tx=0).stop 2023-07-06T03:46:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/3477937574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8a40d2210 msgr2=0x7fe8a40c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8a40d2210 0x7fe8a40c90a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c420 tx=0x7fe89400c730 comp rx=0 tx=0).stop 2023-07-06T03:46:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/3477937574 shutdown_connections 2023-07-06T03:46:10.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe88806ace0 0x7fe88806d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8a415c610 0x7fe8a415e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8a406b360 0x7fe8a40ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 --2- 172.21.15.97:0/3477937574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8a40d2210 0x7fe8a40c90a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/3477937574 >> 172.21.15.97:0/3477937574 conn(0x7fe8a406a6c0 msgr2=0x7fe8a40d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/3477937574 shutdown_connections 2023-07-06T03:46:10.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:10.785+0000 7fe8ab7ff700 1 -- 172.21.15.97:0/3477937574 wait complete. 2023-07-06T03:46:10.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:46:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.002+0000 7f6804dff700 1 Processor -- start 2023-07-06T03:46:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.003+0000 7f6804dff700 1 -- start start 2023-07-06T03:46:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f6804dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f680006b360 0x7f68000b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f6804dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f68000d2210 0x7f68000d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f6804dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68000cff90 con 0x7f680015c610 2023-07-06T03:46:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f6804dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68000d0780 con 0x7f680006b360 2023-07-06T03:46:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f6804dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680015c9e0 con 0x7f68000d2210 2023-07-06T03:46:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f67f7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f680006b360 0x7f68000b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f67f7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f680006b360 0x7f68000b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40664/0 (socket says 172.21.15.97:40664) 2023-07-06T03:46:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f67f7fff700 1 -- 172.21.15.97:0/4184123934 learned_addr learned my addr 172.21.15.97:0/4184123934 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f67fed9d700 1 --2- 172.21.15.97:0/4184123934 >> v2:172.21.15.146:3300/0 conn(0x7f68000d2210 0x7f68000d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f67f7fff700 1 -- 172.21.15.97:0/4184123934 >> v2:172.21.15.146:3300/0 conn(0x7f68000d2210 msgr2=0x7f68000d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f67f7fff700 1 --2- 172.21.15.97:0/4184123934 >> v2:172.21.15.146:3300/0 conn(0x7f68000d2210 0x7f68000d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f67f7fff700 1 -- 172.21.15.97:0/4184123934 >> v1:172.21.15.146:6789/0 conn(0x7f680015c610 legacy=0x7f68000b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f67f7fff700 1 -- 172.21.15.97:0/4184123934 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000b5650 con 0x7f680006b360 2023-07-06T03:46:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.004+0000 7f67fed9d700 1 --2- 172.21.15.97:0/4184123934 >> v2:172.21.15.146:3300/0 conn(0x7f68000d2210 0x7f68000d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.005+0000 7f67f7fff700 1 --2- 172.21.15.97:0/4184123934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680006b360 0x7f68000b4ec0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f68000d2bc0 tx=0x7f67e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48286bbc0ad1a8f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.005+0000 7f67fdd9b700 1 -- 172.21.15.97:0/4184123934 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8004230 con 0x7f680006b360 2023-07-06T03:46:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.005+0000 7f67fdd9b700 1 -- 172.21.15.97:0/4184123934 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e8004390 con 0x7f680006b360 2023-07-06T03:46:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.005+0000 7f67fdd9b700 1 -- 172.21.15.97:0/4184123934 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e80036a0 con 0x7f680006b360 2023-07-06T03:46:11.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.005+0000 7f6804dff700 1 -- 172.21.15.97:0/4184123934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680006b360 msgr2=0x7f68000b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.005+0000 7f6804dff700 1 --2- 172.21.15.97:0/4184123934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680006b360 0x7f68000b4ec0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f68000d2bc0 tx=0x7f67e8009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:11.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.006+0000 7f6804dff700 1 -- 172.21.15.97:0/4184123934 shutdown_connections 2023-07-06T03:46:11.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.006+0000 7f6804dff700 1 --2- 172.21.15.97:0/4184123934 >> v2:172.21.15.146:3300/0 conn(0x7f68000d2210 0x7f68000d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.006+0000 7f6804dff700 1 --2- 172.21.15.97:0/4184123934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680006b360 0x7f68000b4ec0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.006+0000 7f6804dff700 1 -- 172.21.15.97:0/4184123934 >> 172.21.15.97:0/4184123934 conn(0x7f680006a6c0 msgr2=0x7f680006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:11.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.006+0000 7f6804dff700 1 -- 172.21.15.97:0/4184123934 shutdown_connections 2023-07-06T03:46:11.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.006+0000 7f6804dff700 1 -- 172.21.15.97:0/4184123934 wait complete. 2023-07-06T03:46:11.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.007+0000 7f6804dff700 1 Processor -- start 2023-07-06T03:46:11.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.008+0000 7f6804dff700 1 -- start start 2023-07-06T03:46:11.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.008+0000 7f6804dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680006b360 0x7f68001f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.008+0000 7f6804dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68000d2210 0x7f6800156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.008+0000 7f6804dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f680015c610 0x7f6800147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f6804dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6800148e90 con 0x7f680006b360 2023-07-06T03:46:11.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f6804dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001488d0 con 0x7f680015c610 2023-07-06T03:46:11.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f6804dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6800148bb0 con 0x7f68000d2210 2023-07-06T03:46:11.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f67fe59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680006b360 0x7f68001f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f67fe59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680006b360 0x7f68001f5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40680/0 (socket says 172.21.15.97:40680) 2023-07-06T03:46:11.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f67fe59c700 1 -- 172.21.15.97:0/401055631 learned_addr learned my addr 172.21.15.97:0/401055631 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:11.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f67fed9d700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f680015c610 0x7f6800147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f67f7fff700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68000d2210 0x7f6800156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f67fe59c700 1 -- 172.21.15.97:0/401055631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68000d2210 msgr2=0x7f6800156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f67fe59c700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68000d2210 0x7f6800156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f67fe59c700 1 -- 172.21.15.97:0/401055631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f680015c610 msgr2=0x7f6800147ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f67fe59c700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f680015c610 0x7f6800147ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f67fe59c700 1 -- 172.21.15.97:0/401055631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e8009550 con 0x7f680006b360 2023-07-06T03:46:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.009+0000 7f67fe59c700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680006b360 0x7f68001f5b60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f67f000b4a0 tx=0x7f67f000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.010+0000 7f67f6ffd700 1 -- 172.21.15.97:0/401055631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0011830 con 0x7f680006b360 2023-07-06T03:46:11.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.010+0000 7f67f6ffd700 1 -- 172.21.15.97:0/401055631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f0011990 con 0x7f680006b360 2023-07-06T03:46:11.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.010+0000 7f6804dff700 1 -- 172.21.15.97:0/401055631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68001f5360 con 0x7f680006b360 2023-07-06T03:46:11.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.010+0000 7f67f6ffd700 1 -- 172.21.15.97:0/401055631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0011d40 con 0x7f680006b360 2023-07-06T03:46:11.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.010+0000 7f6804dff700 1 -- 172.21.15.97:0/401055631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68001f60a0 con 0x7f680006b360 2023-07-06T03:46:11.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.011+0000 7f6804dff700 1 -- 172.21.15.97:0/401055631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68001f5880 con 0x7f680006b360 2023-07-06T03:46:11.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.012+0000 7f67f6ffd700 1 -- 172.21.15.97:0/401055631 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f67f001a400 con 0x7f680006b360 2023-07-06T03:46:11.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.012+0000 7f67f6ffd700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f67e006ace0 0x7f67e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.013+0000 7f67f6ffd700 1 -- 172.21.15.97:0/401055631 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67f0089210 con 0x7f680006b360 2023-07-06T03:46:11.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.013+0000 7f67f7fff700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f67e006ace0 0x7f67e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.013+0000 7f67f7fff700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f67e006ace0 0x7f67e006d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009130 tx=0x7f67e8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.017+0000 7f67f6ffd700 1 -- 172.21.15.97:0/401055631 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f68001f5880 con 0x7f680006b360 2023-07-06T03:46:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.211+0000 7f6804dff700 1 -- 172.21.15.97:0/401055631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68001f5880 con 0x7f680006b360 2023-07-06T03:46:11.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.212+0000 7f67f6ffd700 1 -- 172.21.15.97:0/401055631 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67f00542b0 con 0x7f680006b360 2023-07-06T03:46:11.215 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:46:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 -- 172.21.15.97:0/401055631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f67e006ace0 msgr2=0x7f67e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f67e006ace0 0x7f67e006d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009130 tx=0x7f67e8008030 comp rx=0 tx=0).stop 2023-07-06T03:46:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 -- 172.21.15.97:0/401055631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680006b360 msgr2=0x7f68001f5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680006b360 0x7f68001f5b60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f67f000b4a0 tx=0x7f67f000b860 comp rx=0 tx=0).stop 2023-07-06T03:46:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 -- 172.21.15.97:0/401055631 shutdown_connections 2023-07-06T03:46:11.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f67e006ace0 0x7f67e006d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f680015c610 0x7f6800147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680006b360 0x7f68001f5b60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 --2- 172.21.15.97:0/401055631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f68000d2210 0x7f6800156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 -- 172.21.15.97:0/401055631 >> 172.21.15.97:0/401055631 conn(0x7f680006a6c0 msgr2=0x7f68000b7190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:11.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 -- 172.21.15.97:0/401055631 shutdown_connections 2023-07-06T03:46:11.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.214+0000 7f6804dff700 1 -- 172.21.15.97:0/401055631 wait complete. 2023-07-06T03:46:11.235 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:46:11.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.429+0000 7f6df45ff700 1 Processor -- start 2023-07-06T03:46:11.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.430+0000 7f6df45ff700 1 -- start start 2023-07-06T03:46:11.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df45ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dec15c610 0x7f6dec0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df45ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6dec06b360 0x7f6dec0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df45ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dec0cff90 con 0x7f6dec0d2210 2023-07-06T03:46:11.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df45ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec0d0780 con 0x7f6dec06b360 2023-07-06T03:46:11.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df45ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec15c9e0 con 0x7f6dec15c610 2023-07-06T03:46:11.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df1b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6dec06b360 0x7f6dec0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df1b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6dec06b360 0x7f6dec0b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40682/0 (socket says 172.21.15.97:40682) 2023-07-06T03:46:11.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df1b9a700 1 -- 172.21.15.97:0/2379926510 learned_addr learned my addr 172.21.15.97:0/2379926510 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:11.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df239b700 1 --2- 172.21.15.97:0/2379926510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dec15c610 0x7f6dec0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df1b9a700 1 -- 172.21.15.97:0/2379926510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dec15c610 msgr2=0x7f6dec0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:11.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df1b9a700 1 --2- 172.21.15.97:0/2379926510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dec15c610 0x7f6dec0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df1b9a700 1 -- 172.21.15.97:0/2379926510 >> v1:172.21.15.146:6789/0 conn(0x7f6dec0d2210 legacy=0x7f6dec0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.431+0000 7f6df1b9a700 1 -- 172.21.15.97:0/2379926510 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec0b5650 con 0x7f6dec06b360 2023-07-06T03:46:11.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.432+0000 7f6df1b9a700 1 --2- 172.21.15.97:0/2379926510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dec06b360 0x7f6dec0b4cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009920 tx=0x7f6ddc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7803c8c620d7603b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:11.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.432+0000 7f6df1399700 1 -- 172.21.15.97:0/2379926510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00ba20 con 0x7f6dec06b360 2023-07-06T03:46:11.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.432+0000 7f6df1399700 1 -- 172.21.15.97:0/2379926510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc00bb80 con 0x7f6dec06b360 2023-07-06T03:46:11.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.432+0000 7f6df1399700 1 -- 172.21.15.97:0/2379926510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc009e70 con 0x7f6dec06b360 2023-07-06T03:46:11.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.432+0000 7f6df45ff700 1 -- 172.21.15.97:0/2379926510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dec06b360 msgr2=0x7f6dec0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.432+0000 7f6df45ff700 1 --2- 172.21.15.97:0/2379926510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dec06b360 0x7f6dec0b4cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009920 tx=0x7f6ddc009c30 comp rx=0 tx=0).stop 2023-07-06T03:46:11.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.433+0000 7f6df45ff700 1 -- 172.21.15.97:0/2379926510 shutdown_connections 2023-07-06T03:46:11.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.433+0000 7f6df45ff700 1 --2- 172.21.15.97:0/2379926510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dec15c610 0x7f6dec0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.433+0000 7f6df45ff700 1 --2- 172.21.15.97:0/2379926510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dec06b360 0x7f6dec0b4cf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.433+0000 7f6df45ff700 1 -- 172.21.15.97:0/2379926510 >> 172.21.15.97:0/2379926510 conn(0x7f6dec06a6c0 msgr2=0x7f6dec06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:11.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.433+0000 7f6df45ff700 1 -- 172.21.15.97:0/2379926510 shutdown_connections 2023-07-06T03:46:11.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.433+0000 7f6df45ff700 1 -- 172.21.15.97:0/2379926510 wait complete. 2023-07-06T03:46:11.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.434+0000 7f6df45ff700 1 Processor -- start 2023-07-06T03:46:11.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.434+0000 7f6df45ff700 1 -- start start 2023-07-06T03:46:11.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.435+0000 7f6df45ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dec06b360 0x7f6dec0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.435+0000 7f6df45ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dec0d2210 0x7f6dec157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.435+0000 7f6df45ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dec15c610 0x7f6dec15eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.435+0000 7f6df45ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0cccb0 con 0x7f6dec06b360 2023-07-06T03:46:11.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.435+0000 7f6df45ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec158830 con 0x7f6dec0d2210 2023-07-06T03:46:11.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.435+0000 7f6df45ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0cb9d0 con 0x7f6dec15c610 2023-07-06T03:46:11.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df239b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dec06b360 0x7f6dec0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df239b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dec06b360 0x7f6dec0cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40692/0 (socket says 172.21.15.97:40692) 2023-07-06T03:46:11.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df239b700 1 -- 172.21.15.97:0/118049623 learned_addr learned my addr 172.21.15.97:0/118049623 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:11.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df1b9a700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dec0d2210 0x7f6dec157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df2b9c700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dec15c610 0x7f6dec15eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df239b700 1 -- 172.21.15.97:0/118049623 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dec15c610 msgr2=0x7f6dec15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df239b700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dec15c610 0x7f6dec15eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df239b700 1 -- 172.21.15.97:0/118049623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dec0d2210 msgr2=0x7f6dec157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df239b700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dec0d2210 0x7f6dec157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df239b700 1 -- 172.21.15.97:0/118049623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc009550 con 0x7f6dec06b360 2023-07-06T03:46:11.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df1b9a700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dec0d2210 0x7f6dec157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:11.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df239b700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dec06b360 0x7f6dec0cb490 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6de800d830 tx=0x7f6de800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:11.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6de2ffd700 1 -- 172.21.15.97:0/118049623 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de80125f0 con 0x7f6dec06b360 2023-07-06T03:46:11.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6de2ffd700 1 -- 172.21.15.97:0/118049623 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de8012750 con 0x7f6dec06b360 2023-07-06T03:46:11.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6de2ffd700 1 -- 172.21.15.97:0/118049623 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8012b00 con 0x7f6dec06b360 2023-07-06T03:46:11.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.436+0000 7f6df45ff700 1 -- 172.21.15.97:0/118049623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dec0cb000 con 0x7f6dec06b360 2023-07-06T03:46:11.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.437+0000 7f6df45ff700 1 -- 172.21.15.97:0/118049623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dec0ca9b0 con 0x7f6dec06b360 2023-07-06T03:46:11.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.438+0000 7f6df45ff700 1 -- 172.21.15.97:0/118049623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dec0ca810 con 0x7f6dec06b360 2023-07-06T03:46:11.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.439+0000 7f6de2ffd700 1 -- 172.21.15.97:0/118049623 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6de8012c60 con 0x7f6dec06b360 2023-07-06T03:46:11.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.439+0000 7f6de2ffd700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6dd006ace0 0x7f6dd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.440+0000 7f6de2ffd700 1 -- 172.21.15.97:0/118049623 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6de80897e0 con 0x7f6dec06b360 2023-07-06T03:46:11.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.440+0000 7f6df1b9a700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6dd006ace0 0x7f6dd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.440+0000 7f6df1b9a700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6dd006ace0 0x7f6dd006d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6dec06b790 tx=0x7f6ddc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:11.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.444+0000 7f6de2ffd700 1 -- 172.21.15.97:0/118049623 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6dec0ca810 con 0x7f6dec06b360 2023-07-06T03:46:11.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.635+0000 7f6df45ff700 1 -- 172.21.15.97:0/118049623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6dec0ca810 con 0x7f6dec06b360 2023-07-06T03:46:11.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.635+0000 7f6de2ffd700 1 -- 172.21.15.97:0/118049623 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6dec0ca810 con 0x7f6dec06b360 2023-07-06T03:46:11.639 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:46:11.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.637+0000 7f6df45ff700 1 -- 172.21.15.97:0/118049623 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6dd006ace0 msgr2=0x7f6dd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.637+0000 7f6df45ff700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6dd006ace0 0x7f6dd006d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6dec06b790 tx=0x7f6ddc008030 comp rx=0 tx=0).stop 2023-07-06T03:46:11.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.637+0000 7f6df45ff700 1 -- 172.21.15.97:0/118049623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dec06b360 msgr2=0x7f6dec0cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.637+0000 7f6df45ff700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dec06b360 0x7f6dec0cb490 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6de800d830 tx=0x7f6de800dbf0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.637+0000 7f6df45ff700 1 -- 172.21.15.97:0/118049623 shutdown_connections 2023-07-06T03:46:11.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.638+0000 7f6df45ff700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6dd006ace0 0x7f6dd006d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.638+0000 7f6df45ff700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6dec15c610 0x7f6dec15eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.638+0000 7f6df45ff700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dec06b360 0x7f6dec0cb490 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.638+0000 7f6df45ff700 1 --2- 172.21.15.97:0/118049623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6dec0d2210 0x7f6dec157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.638+0000 7f6df45ff700 1 -- 172.21.15.97:0/118049623 >> 172.21.15.97:0/118049623 conn(0x7f6dec06a6c0 msgr2=0x7f6dec15b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:11.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.638+0000 7f6df45ff700 1 -- 172.21.15.97:0/118049623 shutdown_connections 2023-07-06T03:46:11.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.638+0000 7f6df45ff700 1 -- 172.21.15.97:0/118049623 wait complete. 2023-07-06T03:46:11.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:46:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.853+0000 7f33a91ff700 1 Processor -- start 2023-07-06T03:46:11.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.853+0000 7f33a91ff700 1 -- start start 2023-07-06T03:46:11.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a91ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f33a415c610 0x7f33a40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a91ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 0x7f33a40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a91ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a40cff90 con 0x7f33a40d2210 2023-07-06T03:46:11.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a91ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a40d0780 con 0x7f33a415c610 2023-07-06T03:46:11.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a91ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a415c9e0 con 0x7f33a406b360 2023-07-06T03:46:11.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f339bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 0x7f33a40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a2d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f33a415c610 0x7f33a40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a359e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f33a40d2210 0x7f33a40d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:41890/0 (socket says 172.21.15.97:41890) 2023-07-06T03:46:11.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f339bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 0x7f33a40b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:35250/0 (socket says 172.21.15.97:35250) 2023-07-06T03:46:11.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a359e700 1 -- 172.21.15.97:0/1304186556 learned_addr learned my addr 172.21.15.97:0/1304186556 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:11.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a259c700 1 -- 172.21.15.97:0/1304186556 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769000072 0 0) 0x7f33a40cff90 con 0x7f33a40d2210 2023-07-06T03:46:11.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f339bfff700 1 -- 172.21.15.97:0/1304186556 >> v2:172.21.15.146:3300/0 conn(0x7f33a415c610 msgr2=0x7f33a40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:11.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f339bfff700 1 --2- 172.21.15.97:0/1304186556 >> v2:172.21.15.146:3300/0 conn(0x7f33a415c610 0x7f33a40b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f339bfff700 1 -- 172.21.15.97:0/1304186556 >> v1:172.21.15.146:6789/0 conn(0x7f33a40d2210 legacy=0x7f33a40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f339bfff700 1 -- 172.21.15.97:0/1304186556 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a40b5650 con 0x7f33a406b360 2023-07-06T03:46:11.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a2d9d700 1 --2- 172.21.15.97:0/1304186556 >> v2:172.21.15.146:3300/0 conn(0x7f33a415c610 0x7f33a40b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:46:11.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f339bfff700 1 --2- 172.21.15.97:0/1304186556 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 0x7f33a40b4cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f338c009840 tx=0x7f338c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd43a9d6d57aec7a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:11.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a259c700 1 -- 172.21.15.97:0/1304186556 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c011830 con 0x7f33a406b360 2023-07-06T03:46:11.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a259c700 1 -- 172.21.15.97:0/1304186556 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c011990 con 0x7f33a406b360 2023-07-06T03:46:11.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.856+0000 7f33a259c700 1 -- 172.21.15.97:0/1304186556 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c011d20 con 0x7f33a406b360 2023-07-06T03:46:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.857+0000 7f33a91ff700 1 -- 172.21.15.97:0/1304186556 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 msgr2=0x7f33a40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.857+0000 7f33a91ff700 1 --2- 172.21.15.97:0/1304186556 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 0x7f33a40b4cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f338c009840 tx=0x7f338c009b50 comp rx=0 tx=0).stop 2023-07-06T03:46:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.857+0000 7f33a91ff700 1 -- 172.21.15.97:0/1304186556 shutdown_connections 2023-07-06T03:46:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.857+0000 7f33a91ff700 1 --2- 172.21.15.97:0/1304186556 >> v2:172.21.15.146:3300/0 conn(0x7f33a415c610 0x7f33a40b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.857+0000 7f33a91ff700 1 --2- 172.21.15.97:0/1304186556 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 0x7f33a40b4cf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.857+0000 7f33a91ff700 1 -- 172.21.15.97:0/1304186556 >> 172.21.15.97:0/1304186556 conn(0x7f33a406a6c0 msgr2=0x7f33a406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.857+0000 7f33a91ff700 1 -- 172.21.15.97:0/1304186556 shutdown_connections 2023-07-06T03:46:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.857+0000 7f33a91ff700 1 -- 172.21.15.97:0/1304186556 wait complete. 2023-07-06T03:46:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.859+0000 7f33a91ff700 1 Processor -- start 2023-07-06T03:46:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.859+0000 7f33a91ff700 1 -- start start 2023-07-06T03:46:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.860+0000 7f33a91ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 0x7f33a4148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.860+0000 7f33a91ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a40d2210 0x7f33a40cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.860+0000 7f33a91ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33a415c610 0x7f33a40c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.860+0000 7f33a91ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40cc4b0 con 0x7f33a40d2210 2023-07-06T03:46:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.860+0000 7f33a91ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40cb8a0 con 0x7f33a415c610 2023-07-06T03:46:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.860+0000 7f33a91ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40cbb80 con 0x7f33a406b360 2023-07-06T03:46:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.860+0000 7f339bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a40d2210 0x7f33a40cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.860+0000 7f339bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a40d2210 0x7f33a40cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40708/0 (socket says 172.21.15.97:40708) 2023-07-06T03:46:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.860+0000 7f33a2d9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 0x7f33a4148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.860+0000 7f339bfff700 1 -- 172.21.15.97:0/2872474349 learned_addr learned my addr 172.21.15.97:0/2872474349 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.860+0000 7f33a359e700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33a415c610 0x7f33a40c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.861+0000 7f339bfff700 1 -- 172.21.15.97:0/2872474349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 msgr2=0x7f33a4148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.861+0000 7f339bfff700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 0x7f33a4148ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.861+0000 7f339bfff700 1 -- 172.21.15.97:0/2872474349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33a415c610 msgr2=0x7f33a40c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.861+0000 7f339bfff700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33a415c610 0x7f33a40c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.861+0000 7f339bfff700 1 -- 172.21.15.97:0/2872474349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33940094a0 con 0x7f33a40d2210 2023-07-06T03:46:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.861+0000 7f339bfff700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a40d2210 0x7f33a40cae50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f338c009fd0 tx=0x7f338c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.861+0000 7f339b7fe700 1 -- 172.21.15.97:0/2872474349 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c0045d0 con 0x7f33a40d2210 2023-07-06T03:46:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.861+0000 7f339b7fe700 1 -- 172.21.15.97:0/2872474349 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c01f070 con 0x7f33a40d2210 2023-07-06T03:46:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.861+0000 7f339b7fe700 1 -- 172.21.15.97:0/2872474349 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c0038e0 con 0x7f33a40d2210 2023-07-06T03:46:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.861+0000 7f33a91ff700 1 -- 172.21.15.97:0/2872474349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338c0094a0 con 0x7f33a40d2210 2023-07-06T03:46:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.862+0000 7f33a91ff700 1 -- 172.21.15.97:0/2872474349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a40cc140 con 0x7f33a40d2210 2023-07-06T03:46:11.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.862+0000 7f33a91ff700 1 -- 172.21.15.97:0/2872474349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a40b7710 con 0x7f33a40d2210 2023-07-06T03:46:11.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.863+0000 7f339b7fe700 1 -- 172.21.15.97:0/2872474349 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f338c003a40 con 0x7f33a40d2210 2023-07-06T03:46:11.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.864+0000 7f339b7fe700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f338406ace0 0x7f338406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:11.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.864+0000 7f339b7fe700 1 -- 172.21.15.97:0/2872474349 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f338c021070 con 0x7f33a40d2210 2023-07-06T03:46:11.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.865+0000 7f33a2d9d700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f338406ace0 0x7f338406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.865+0000 7f33a2d9d700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f338406ace0 0x7f338406d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3394009cd0 tx=0x7f3394009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:11.869+0000 7f339b7fe700 1 -- 172.21.15.97:0/2872474349 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f338c055d90 con 0x7f33a40d2210 2023-07-06T03:46:12.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.078+0000 7f33a91ff700 1 -- 172.21.15.97:0/2872474349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f33a404e340 con 0x7f33a40d2210 2023-07-06T03:46:12.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.079+0000 7f339b7fe700 1 -- 172.21.15.97:0/2872474349 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f338c0593b0 con 0x7f33a40d2210 2023-07-06T03:46:12.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 -- 172.21.15.97:0/2872474349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f338406ace0 msgr2=0x7f338406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f338406ace0 0x7f338406d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3394009cd0 tx=0x7f3394009210 comp rx=0 tx=0).stop 2023-07-06T03:46:12.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 -- 172.21.15.97:0/2872474349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a40d2210 msgr2=0x7f33a40cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a40d2210 0x7f33a40cae50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f338c009fd0 tx=0x7f338c004750 comp rx=0 tx=0).stop 2023-07-06T03:46:12.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 -- 172.21.15.97:0/2872474349 shutdown_connections 2023-07-06T03:46:12.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f338406ace0 0x7f338406d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33a415c610 0x7f33a40c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a406b360 0x7f33a4148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 --2- 172.21.15.97:0/2872474349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a40d2210 0x7f33a40cae50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 -- 172.21.15.97:0/2872474349 >> 172.21.15.97:0/2872474349 conn(0x7f33a406a6c0 msgr2=0x7f33a40b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 -- 172.21.15.97:0/2872474349 shutdown_connections 2023-07-06T03:46:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.081+0000 7f33a91ff700 1 -- 172.21.15.97:0/2872474349 wait complete. 2023-07-06T03:46:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:46:12.101 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:46:12.102 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:12.147 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:12.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.378+0000 7ff84c5ff700 1 Processor -- start 2023-07-06T03:46:12.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.379+0000 7ff84c5ff700 1 -- start start 2023-07-06T03:46:12.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.380+0000 7ff84c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff84415c520 0x7ff8440cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.380+0000 7ff84c5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff844160180 0x7ff8441605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.380+0000 7ff84c5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8440cdae0 con 0x7ff844160b30 2023-07-06T03:46:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.380+0000 7ff84c5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8440cd8d0 con 0x7ff844160180 2023-07-06T03:46:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.380+0000 7ff84c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440b6640 con 0x7ff84415c520 2023-07-06T03:46:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.380+0000 7ff84ab9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff844160b30 0x7ff8440cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60850/0 (socket says 172.21.15.97:60850) 2023-07-06T03:46:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.380+0000 7ff84ab9c700 1 -- 172.21.15.97:0/4069195449 learned_addr learned my addr 172.21.15.97:0/4069195449 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:12.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.380+0000 7ff84a39b700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff84415c520 0x7ff8440cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.380+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295898251 0 0) 0x7ff8440cdae0 con 0x7ff844160b30 2023-07-06T03:46:12.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.380+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8300035b0 con 0x7ff844160b30 2023-07-06T03:46:12.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.380+0000 7ff849b9a700 1 --2- 172.21.15.97:0/4069195449 >> v2:172.21.15.146:3300/0 conn(0x7ff844160180 0x7ff8441605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2340161636 0 0) 0x7ff8300035b0 con 0x7ff844160b30 2023-07-06T03:46:12.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff84415c520 msgr2=0x7ff8440cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:12.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff84415c520 0x7ff8440cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 >> v2:172.21.15.146:3300/0 conn(0x7ff844160180 msgr2=0x7ff8441605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 --2- 172.21.15.97:0/4069195449 >> v2:172.21.15.146:3300/0 conn(0x7ff844160180 0x7ff8441605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8440b6920 con 0x7ff844160b30 2023-07-06T03:46:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7ff83c0019f0 con 0x7ff844160b30 2023-07-06T03:46:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 >> v1:172.21.15.97:6789/0 conn(0x7ff844160b30 legacy=0x7ff8440cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff830003a00 0x7ff830003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff830004850 0x7ff830004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff84415c520 0x7ff830005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440cd8d0 con 0x7ff830003a00 2023-07-06T03:46:12.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440b6640 con 0x7ff830004850 2023-07-06T03:46:12.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83000ba10 con 0x7ff84415c520 2023-07-06T03:46:12.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff84a39b700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff830003a00 0x7ff830003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff84ab9c700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff84415c520 0x7ff830005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849b9a700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff830004850 0x7ff830004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff84a39b700 1 -- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff84415c520 msgr2=0x7ff830005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff84a39b700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff84415c520 0x7ff830005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff84a39b700 1 -- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff830004850 msgr2=0x7ff830004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff84a39b700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff830004850 0x7ff830004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff84a39b700 1 -- 172.21.15.97:0/4069195449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8300035b0 con 0x7ff830003a00 2023-07-06T03:46:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.381+0000 7ff849b9a700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff830004850 0x7ff830004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.382+0000 7ff84a39b700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff830003a00 0x7ff830003e10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff840009b90 tx=0x7ff830005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65216d4b56d675f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.382+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84000e6c0 con 0x7ff830003a00 2023-07-06T03:46:12.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.382+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84000e820 con 0x7ff830003a00 2023-07-06T03:46:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.382+0000 7ff849399700 1 -- 172.21.15.97:0/4069195449 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84000eb90 con 0x7ff830003a00 2023-07-06T03:46:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.382+0000 7ff84c5ff700 1 -- 172.21.15.97:0/4069195449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff830003a00 msgr2=0x7ff830003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.382+0000 7ff84c5ff700 1 --2- 172.21.15.97:0/4069195449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff830003a00 0x7ff830003e10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff840009b90 tx=0x7ff830005880 comp rx=0 tx=0).stop 2023-07-06T03:46:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.382+0000 7ff84ab9c700 1 -- 172.21.15.97:0/4069195449 reap_dead start 2023-07-06T03:46:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.382+0000 7ff84c5ff700 1 -- 172.21.15.97:0/4069195449 shutdown_connections 2023-07-06T03:46:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.382+0000 7ff84c5ff700 1 -- 172.21.15.97:0/4069195449 >> 172.21.15.97:0/4069195449 conn(0x7ff844071790 msgr2=0x7ff844071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.383+0000 7ff84c5ff700 1 -- 172.21.15.97:0/4069195449 shutdown_connections 2023-07-06T03:46:12.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.383+0000 7ff84c5ff700 1 -- 172.21.15.97:0/4069195449 wait complete. 2023-07-06T03:46:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.384+0000 7ff84c5ff700 1 Processor -- start 2023-07-06T03:46:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.384+0000 7ff84c5ff700 1 -- start start 2023-07-06T03:46:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.385+0000 7ff84c5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff830003a00 0x7ff8441591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.385+0000 7ff84c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8441562a0 0x7ff84415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.385+0000 7ff84c5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8441f5a40 0x7ff8441f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.385+0000 7ff84c5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8441f5620 con 0x7ff8441f5a40 2023-07-06T03:46:12.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.385+0000 7ff84c5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84415a060 con 0x7ff830003a00 2023-07-06T03:46:12.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.385+0000 7ff84c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8441597c0 con 0x7ff8441562a0 2023-07-06T03:46:12.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff84ab9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8441f5a40 0x7ff8441f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff84ab9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8441f5a40 0x7ff8441f5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40720/0 (socket says 172.21.15.97:40720) 2023-07-06T03:46:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff84ab9c700 1 -- 172.21.15.97:0/106068368 learned_addr learned my addr 172.21.15.97:0/106068368 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff849b9a700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8441562a0 0x7ff84415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff84a39b700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff830003a00 0x7ff8441591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff84ab9c700 1 -- 172.21.15.97:0/106068368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8441562a0 msgr2=0x7ff84415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff84ab9c700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8441562a0 0x7ff84415a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff84ab9c700 1 -- 172.21.15.97:0/106068368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff830003a00 msgr2=0x7ff8441591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff84ab9c700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff830003a00 0x7ff8441591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff84ab9c700 1 -- 172.21.15.97:0/106068368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff844159aa0 con 0x7ff8441f5a40 2023-07-06T03:46:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff84ab9c700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8441f5a40 0x7ff8441f5ee0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0013d0 tx=0x7ff83c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff83affd700 1 -- 172.21.15.97:0/106068368 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c00ebb0 con 0x7ff8441f5a40 2023-07-06T03:46:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff83affd700 1 -- 172.21.15.97:0/106068368 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83c00ed30 con 0x7ff8441f5a40 2023-07-06T03:46:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff83affd700 1 -- 172.21.15.97:0/106068368 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c013660 con 0x7ff8441f5a40 2023-07-06T03:46:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.386+0000 7ff84c5ff700 1 -- 172.21.15.97:0/106068368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff844159d80 con 0x7ff8441f5a40 2023-07-06T03:46:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.387+0000 7ff84c5ff700 1 -- 172.21.15.97:0/106068368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff844156710 con 0x7ff8441f5a40 2023-07-06T03:46:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.387+0000 7ff84c5ff700 1 -- 172.21.15.97:0/106068368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84415aee0 con 0x7ff8441f5a40 2023-07-06T03:46:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.388+0000 7ff83affd700 1 -- 172.21.15.97:0/106068368 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff83c0137c0 con 0x7ff8441f5a40 2023-07-06T03:46:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.389+0000 7ff83affd700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82806ace0 0x7ff82806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.389+0000 7ff83affd700 1 -- 172.21.15.97:0/106068368 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff83c089970 con 0x7ff8441f5a40 2023-07-06T03:46:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.390+0000 7ff84a39b700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82806ace0 0x7ff82806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.390+0000 7ff84a39b700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82806ace0 0x7ff82806d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff8400090b0 tx=0x7ff840008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.393+0000 7ff83affd700 1 -- 172.21.15.97:0/106068368 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff83c054a10 con 0x7ff8441f5a40 2023-07-06T03:46:12.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.587+0000 7ff84c5ff700 1 -- 172.21.15.97:0/106068368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8441f5760 con 0x7ff8441f5a40 2023-07-06T03:46:12.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.589+0000 7ff83affd700 1 -- 172.21.15.97:0/106068368 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7ff8441f5760 con 0x7ff8441f5a40 2023-07-06T03:46:12.592 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:12.592 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 -- 172.21.15.97:0/106068368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82806ace0 msgr2=0x7ff82806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82806ace0 0x7ff82806d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff8400090b0 tx=0x7ff840008030 comp rx=0 tx=0).stop 2023-07-06T03:46:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 -- 172.21.15.97:0/106068368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8441f5a40 msgr2=0x7ff8441f5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8441f5a40 0x7ff8441f5ee0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0013d0 tx=0x7ff83c001ae0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 -- 172.21.15.97:0/106068368 shutdown_connections 2023-07-06T03:46:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82806ace0 0x7ff82806d190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8441f5a40 0x7ff8441f5ee0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff830003a00 0x7ff8441591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 --2- 172.21.15.97:0/106068368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8441562a0 0x7ff84415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 -- 172.21.15.97:0/106068368 >> 172.21.15.97:0/106068368 conn(0x7ff844071790 msgr2=0x7ff84415b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 -- 172.21.15.97:0/106068368 shutdown_connections 2023-07-06T03:46:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.592+0000 7ff84c5ff700 1 -- 172.21.15.97:0/106068368 wait complete. 2023-07-06T03:46:12.613 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:46:12.614 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:46:12.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.813+0000 7fe5781ff700 1 Processor -- start 2023-07-06T03:46:12.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.813+0000 7fe5781ff700 1 -- start start 2023-07-06T03:46:12.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.815+0000 7fe5781ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe57015d490 0x7fe57006c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.815+0000 7fe5781ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5701606a0 0x7fe570160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.815+0000 7fe5781ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe570156f40 con 0x7fe570156b70 2023-07-06T03:46:12.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.815+0000 7fe5781ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5700c5840 con 0x7fe5701606a0 2023-07-06T03:46:12.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.815+0000 7fe5781ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe570160210 con 0x7fe57015d490 2023-07-06T03:46:12.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.815+0000 7fe57679c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5701606a0 0x7fe570160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.815+0000 7fe575f9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fe570156b70 0x7fe5700ac870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:41896/0 (socket says 172.21.15.97:41896) 2023-07-06T03:46:12.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.815+0000 7fe575f9b700 1 -- 172.21.15.97:0/2641101124 learned_addr learned my addr 172.21.15.97:0/2641101124 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:12.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.815+0000 7fe57579a700 1 --2- 172.21.15.97:0/2641101124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe57015d490 0x7fe57006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.816+0000 7fe57679c700 1 -- 172.21.15.97:0/2641101124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe57015d490 msgr2=0x7fe57006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.816+0000 7fe57679c700 1 --2- 172.21.15.97:0/2641101124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe57015d490 0x7fe57006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.816+0000 7fe57679c700 1 -- 172.21.15.97:0/2641101124 >> v1:172.21.15.146:6789/0 conn(0x7fe570156b70 legacy=0x7fe5700ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.816+0000 7fe57679c700 1 -- 172.21.15.97:0/2641101124 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5700c5090 con 0x7fe5701606a0 2023-07-06T03:46:12.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.816+0000 7fe574f99700 1 -- 172.21.15.97:0/2641101124 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344199222 0 0) 0x7fe570156f40 con 0x7fe570156b70 2023-07-06T03:46:12.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.816+0000 7fe57679c700 1 --2- 172.21.15.97:0/2641101124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5701606a0 0x7fe570160b00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe568009d90 tx=0x7fe56800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b5568d9db9291c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:12.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.816+0000 7fe574f99700 1 -- 172.21.15.97:0/2641101124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56800e070 con 0x7fe5701606a0 2023-07-06T03:46:12.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.816+0000 7fe574f99700 1 -- 172.21.15.97:0/2641101124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56800bb20 con 0x7fe5701606a0 2023-07-06T03:46:12.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.816+0000 7fe574f99700 1 -- 172.21.15.97:0/2641101124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568012430 con 0x7fe5701606a0 2023-07-06T03:46:12.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.817+0000 7fe5781ff700 1 -- 172.21.15.97:0/2641101124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5701606a0 msgr2=0x7fe570160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.817+0000 7fe5781ff700 1 --2- 172.21.15.97:0/2641101124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5701606a0 0x7fe570160b00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe568009d90 tx=0x7fe56800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:12.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.817+0000 7fe5781ff700 1 -- 172.21.15.97:0/2641101124 shutdown_connections 2023-07-06T03:46:12.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.817+0000 7fe5781ff700 1 --2- 172.21.15.97:0/2641101124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5701606a0 0x7fe570160b00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.817+0000 7fe5781ff700 1 --2- 172.21.15.97:0/2641101124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe57015d490 0x7fe57006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.817+0000 7fe5781ff700 1 -- 172.21.15.97:0/2641101124 >> 172.21.15.97:0/2641101124 conn(0x7fe57006a060 msgr2=0x7fe57006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:12.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.817+0000 7fe5781ff700 1 -- 172.21.15.97:0/2641101124 shutdown_connections 2023-07-06T03:46:12.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.817+0000 7fe5781ff700 1 -- 172.21.15.97:0/2641101124 wait complete. 2023-07-06T03:46:12.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.819+0000 7fe5781ff700 1 Processor -- start 2023-07-06T03:46:12.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.819+0000 7fe5781ff700 1 -- start start 2023-07-06T03:46:12.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.820+0000 7fe5781ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe570156b70 0x7fe57014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.820+0000 7fe5781ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57015d490 0x7fe5700700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.820+0000 7fe5781ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe5701606a0 0x7fe570070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.820+0000 7fe5781ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5700afc30 con 0x7fe57015d490 2023-07-06T03:46:12.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.820+0000 7fe5781ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5700b01e0 con 0x7fe570156b70 2023-07-06T03:46:12.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.820+0000 7fe5781ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5700b0670 con 0x7fe5701606a0 2023-07-06T03:46:12.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.820+0000 7fe57579a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57015d490 0x7fe5700700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.820+0000 7fe57579a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57015d490 0x7fe5700700f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40728/0 (socket says 172.21.15.97:40728) 2023-07-06T03:46:12.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.820+0000 7fe57579a700 1 -- 172.21.15.97:0/940653373 learned_addr learned my addr 172.21.15.97:0/940653373 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:12.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.820+0000 7fe575f9b700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe570156b70 0x7fe57014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.820+0000 7fe57679c700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe5701606a0 0x7fe570070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.821+0000 7fe57579a700 1 -- 172.21.15.97:0/940653373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe5701606a0 msgr2=0x7fe570070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.821+0000 7fe57579a700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe5701606a0 0x7fe570070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.821+0000 7fe57579a700 1 -- 172.21.15.97:0/940653373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe570156b70 msgr2=0x7fe57014b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:12.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.821+0000 7fe57579a700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe570156b70 0x7fe57014b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:12.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.821+0000 7fe57579a700 1 -- 172.21.15.97:0/940653373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe568009a70 con 0x7fe57015d490 2023-07-06T03:46:12.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.821+0000 7fe57579a700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57015d490 0x7fe5700700f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe56000e9b0 tx=0x7fe56000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.821+0000 7fe5667fc700 1 -- 172.21.15.97:0/940653373 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560009710 con 0x7fe57015d490 2023-07-06T03:46:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.821+0000 7fe5667fc700 1 -- 172.21.15.97:0/940653373 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56000c850 con 0x7fe57015d490 2023-07-06T03:46:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.821+0000 7fe5667fc700 1 -- 172.21.15.97:0/940653373 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56000cbc0 con 0x7fe57015d490 2023-07-06T03:46:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.821+0000 7fe5781ff700 1 -- 172.21.15.97:0/940653373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5700cde30 con 0x7fe57015d490 2023-07-06T03:46:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.821+0000 7fe5781ff700 1 -- 172.21.15.97:0/940653373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5700ce3f0 con 0x7fe57015d490 2023-07-06T03:46:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.822+0000 7fe5781ff700 1 -- 172.21.15.97:0/940653373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57004e340 con 0x7fe57015d490 2023-07-06T03:46:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.823+0000 7fe5667fc700 1 -- 172.21.15.97:0/940653373 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe56000cd20 con 0x7fe57015d490 2023-07-06T03:46:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.824+0000 7fe5667fc700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe55406ace0 0x7fe55406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:12.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.824+0000 7fe5667fc700 1 -- 172.21.15.97:0/940653373 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe560089560 con 0x7fe57015d490 2023-07-06T03:46:12.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.824+0000 7fe575f9b700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe55406ace0 0x7fe55406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:12.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.825+0000 7fe575f9b700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe55406ace0 0x7fe55406d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000f60 tx=0x7fe56c007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:12.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:12.829+0000 7fe5667fc700 1 -- 172.21.15.97:0/940653373 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe560058130 con 0x7fe57015d490 2023-07-06T03:46:13.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.027+0000 7fe5781ff700 1 -- 172.21.15.97:0/940653373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe57006c110 con 0x7fe57015d490 2023-07-06T03:46:13.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.027+0000 7fe5667fc700 1 -- 172.21.15.97:0/940653373 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe560057cc0 con 0x7fe57015d490 2023-07-06T03:46:13.029 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:46:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.029+0000 7fe5781ff700 1 -- 172.21.15.97:0/940653373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe55406ace0 msgr2=0x7fe55406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.029+0000 7fe5781ff700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe55406ace0 0x7fe55406d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000f60 tx=0x7fe56c007be0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.029+0000 7fe5781ff700 1 -- 172.21.15.97:0/940653373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57015d490 msgr2=0x7fe5700700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.029+0000 7fe5781ff700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57015d490 0x7fe5700700f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe56000e9b0 tx=0x7fe56000ed70 comp rx=0 tx=0).stop 2023-07-06T03:46:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.029+0000 7fe5781ff700 1 -- 172.21.15.97:0/940653373 shutdown_connections 2023-07-06T03:46:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.029+0000 7fe5781ff700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe55406ace0 0x7fe55406d190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.029+0000 7fe5781ff700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe5701606a0 0x7fe570070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.029+0000 7fe5781ff700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe570156b70 0x7fe57014b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.029+0000 7fe5781ff700 1 --2- 172.21.15.97:0/940653373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57015d490 0x7fe5700700f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.030+0000 7fe5781ff700 1 -- 172.21.15.97:0/940653373 >> 172.21.15.97:0/940653373 conn(0x7fe57006a060 msgr2=0x7fe5700c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:13.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.030+0000 7fe5781ff700 1 -- 172.21.15.97:0/940653373 shutdown_connections 2023-07-06T03:46:13.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.030+0000 7fe5781ff700 1 -- 172.21.15.97:0/940653373 wait complete. 2023-07-06T03:46:13.049 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:46:13.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.246+0000 7fd73b9ff700 1 Processor -- start 2023-07-06T03:46:13.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.246+0000 7fd73b9ff700 1 -- start start 2023-07-06T03:46:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.247+0000 7fd73b9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd73415c610 0x7fd7340b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.247+0000 7fd73b9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd7340d2210 0x7fd7340d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd73b9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7340cff90 con 0x7fd73406b360 2023-07-06T03:46:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd73b9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7340d0780 con 0x7fd7340d2210 2023-07-06T03:46:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd73b9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd73415c9e0 con 0x7fd73415c610 2023-07-06T03:46:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd738f9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd73406b360 0x7fd7340b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60852/0 (socket says 172.21.15.97:60852) 2023-07-06T03:46:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd738f9a700 1 -- 172.21.15.97:0/1869527784 learned_addr learned my addr 172.21.15.97:0/1869527784 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd739f9c700 1 --2- 172.21.15.97:0/1869527784 >> v2:172.21.15.97:3300/0 conn(0x7fd7340d2210 0x7fd7340d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd73979b700 1 --2- 172.21.15.97:0/1869527784 >> v2:172.21.15.146:3300/0 conn(0x7fd73415c610 0x7fd7340b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd739f9c700 1 -- 172.21.15.97:0/1869527784 >> v2:172.21.15.146:3300/0 conn(0x7fd73415c610 msgr2=0x7fd7340b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd72bfff700 1 -- 172.21.15.97:0/1869527784 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476378164 0 0) 0x7fd7340cff90 con 0x7fd73406b360 2023-07-06T03:46:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd739f9c700 1 --2- 172.21.15.97:0/1869527784 >> v2:172.21.15.146:3300/0 conn(0x7fd73415c610 0x7fd7340b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd739f9c700 1 -- 172.21.15.97:0/1869527784 >> v1:172.21.15.97:6789/0 conn(0x7fd73406b360 legacy=0x7fd7340b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.248+0000 7fd739f9c700 1 -- 172.21.15.97:0/1869527784 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7340b5650 con 0x7fd7340d2210 2023-07-06T03:46:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd739f9c700 1 --2- 172.21.15.97:0/1869527784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7340d2210 0x7fd7340d1310 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd73408b000 tx=0x7fd72c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5bddf9ab26df0c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd72bfff700 1 -- 172.21.15.97:0/1869527784 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c00e070 con 0x7fd7340d2210 2023-07-06T03:46:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd72bfff700 1 -- 172.21.15.97:0/1869527784 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c00bb90 con 0x7fd7340d2210 2023-07-06T03:46:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd72bfff700 1 -- 172.21.15.97:0/1869527784 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c012430 con 0x7fd7340d2210 2023-07-06T03:46:13.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd73b9ff700 1 -- 172.21.15.97:0/1869527784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7340d2210 msgr2=0x7fd7340d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd73b9ff700 1 --2- 172.21.15.97:0/1869527784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7340d2210 0x7fd7340d1310 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd73408b000 tx=0x7fd72c00b330 comp rx=0 tx=0).stop 2023-07-06T03:46:13.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd73b9ff700 1 -- 172.21.15.97:0/1869527784 shutdown_connections 2023-07-06T03:46:13.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd73b9ff700 1 --2- 172.21.15.97:0/1869527784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7340d2210 0x7fd7340d1310 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd73b9ff700 1 --2- 172.21.15.97:0/1869527784 >> v2:172.21.15.146:3300/0 conn(0x7fd73415c610 0x7fd7340b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd73b9ff700 1 -- 172.21.15.97:0/1869527784 >> 172.21.15.97:0/1869527784 conn(0x7fd73406a6c0 msgr2=0x7fd73406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:13.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd73b9ff700 1 -- 172.21.15.97:0/1869527784 shutdown_connections 2023-07-06T03:46:13.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.249+0000 7fd73b9ff700 1 -- 172.21.15.97:0/1869527784 wait complete. 2023-07-06T03:46:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.251+0000 7fd73b9ff700 1 Processor -- start 2023-07-06T03:46:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.251+0000 7fd73b9ff700 1 -- start start 2023-07-06T03:46:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.252+0000 7fd73b9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73406b360 0x7fd7340c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.252+0000 7fd73b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7340d2210 0x7fd73415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.252+0000 7fd73b9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd73415c610 0x7fd7341f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.252+0000 7fd73b9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73415f270 con 0x7fd73406b360 2023-07-06T03:46:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.252+0000 7fd73b9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd734154470 con 0x7fd73415c610 2023-07-06T03:46:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.252+0000 7fd73b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7340cb0a0 con 0x7fd7340d2210 2023-07-06T03:46:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.252+0000 7fd73979b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73406b360 0x7fd7340c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.252+0000 7fd73979b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73406b360 0x7fd7340c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40750/0 (socket says 172.21.15.97:40750) 2023-07-06T03:46:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.252+0000 7fd73979b700 1 -- 172.21.15.97:0/103625866 learned_addr learned my addr 172.21.15.97:0/103625866 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.253+0000 7fd73979b700 1 -- 172.21.15.97:0/103625866 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7340d2210 msgr2=0x7fd73415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.253+0000 7fd73979b700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7340d2210 0x7fd73415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.253+0000 7fd73979b700 1 -- 172.21.15.97:0/103625866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd73415c610 msgr2=0x7fd7341f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.253+0000 7fd73979b700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd73415c610 0x7fd7341f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.253+0000 7fd73979b700 1 -- 172.21.15.97:0/103625866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c009a70 con 0x7fd73406b360 2023-07-06T03:46:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.253+0000 7fd73979b700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73406b360 0x7fd7340c9e00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd73000d800 tx=0x7fd73000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.253+0000 7fd729ffb700 1 -- 172.21.15.97:0/103625866 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7300125f0 con 0x7fd73406b360 2023-07-06T03:46:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.253+0000 7fd729ffb700 1 -- 172.21.15.97:0/103625866 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd730012750 con 0x7fd73406b360 2023-07-06T03:46:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.253+0000 7fd73b9ff700 1 -- 172.21.15.97:0/103625866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7340ca490 con 0x7fd73406b360 2023-07-06T03:46:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.253+0000 7fd729ffb700 1 -- 172.21.15.97:0/103625866 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd730012b00 con 0x7fd73406b360 2023-07-06T03:46:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.253+0000 7fd73b9ff700 1 -- 172.21.15.97:0/103625866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7340caa50 con 0x7fd73406b360 2023-07-06T03:46:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.254+0000 7fd73b9ff700 1 -- 172.21.15.97:0/103625866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73404e340 con 0x7fd73406b360 2023-07-06T03:46:13.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.255+0000 7fd729ffb700 1 -- 172.21.15.97:0/103625866 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd730012c60 con 0x7fd73406b360 2023-07-06T03:46:13.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.256+0000 7fd729ffb700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd71806ace0 0x7fd71806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.256+0000 7fd729ffb700 1 -- 172.21.15.97:0/103625866 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd7300898e0 con 0x7fd73406b360 2023-07-06T03:46:13.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.256+0000 7fd738f9a700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd71806ace0 0x7fd71806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:13.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.257+0000 7fd738f9a700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd71806ace0 0x7fd71806d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd7240011e0 tx=0x7fd7240079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:13.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.261+0000 7fd729ffb700 1 -- 172.21.15.97:0/103625866 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd7300548f0 con 0x7fd73406b360 2023-07-06T03:46:13.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.456+0000 7fd73b9ff700 1 -- 172.21.15.97:0/103625866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd7340cd4f0 con 0x7fd73406b360 2023-07-06T03:46:13.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.456+0000 7fd729ffb700 1 -- 172.21.15.97:0/103625866 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd730057f10 con 0x7fd73406b360 2023-07-06T03:46:13.458 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:46:13.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.458+0000 7fd73b9ff700 1 -- 172.21.15.97:0/103625866 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd71806ace0 msgr2=0x7fd71806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.458+0000 7fd73b9ff700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd71806ace0 0x7fd71806d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd7240011e0 tx=0x7fd7240079b0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.458+0000 7fd73b9ff700 1 -- 172.21.15.97:0/103625866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73406b360 msgr2=0x7fd7340c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.458+0000 7fd73b9ff700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73406b360 0x7fd7340c9e00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd73000d800 tx=0x7fd73000dbc0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.458+0000 7fd73b9ff700 1 -- 172.21.15.97:0/103625866 shutdown_connections 2023-07-06T03:46:13.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.458+0000 7fd73b9ff700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd71806ace0 0x7fd71806d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.458+0000 7fd73b9ff700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd73415c610 0x7fd7341f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.458+0000 7fd73b9ff700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd73406b360 0x7fd7340c9e00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.458+0000 7fd73b9ff700 1 --2- 172.21.15.97:0/103625866 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7340d2210 0x7fd73415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.459+0000 7fd73b9ff700 1 -- 172.21.15.97:0/103625866 >> 172.21.15.97:0/103625866 conn(0x7fd73406a6c0 msgr2=0x7fd73406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:13.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.459+0000 7fd73b9ff700 1 -- 172.21.15.97:0/103625866 shutdown_connections 2023-07-06T03:46:13.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.459+0000 7fd73b9ff700 1 -- 172.21.15.97:0/103625866 wait complete. 2023-07-06T03:46:13.478 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:46:13.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.674+0000 7f9e89dff700 1 Processor -- start 2023-07-06T03:46:13.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.674+0000 7f9e89dff700 1 -- start start 2023-07-06T03:46:13.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.675+0000 7f9e89dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8415c610 0x7f9e840b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.675+0000 7f9e89dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9e8406b360 0x7f9e840b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.675+0000 7f9e89dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e840cff90 con 0x7f9e840d2210 2023-07-06T03:46:13.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.675+0000 7f9e89dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e840d0780 con 0x7f9e8406b360 2023-07-06T03:46:13.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.675+0000 7f9e89dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8415c9e0 con 0x7f9e8415c610 2023-07-06T03:46:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.676+0000 7f9e82ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9e8406b360 0x7f9e840b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.676+0000 7f9e82ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9e8406b360 0x7f9e840b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40764/0 (socket says 172.21.15.97:40764) 2023-07-06T03:46:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.676+0000 7f9e82ffd700 1 -- 172.21.15.97:0/3892963145 learned_addr learned my addr 172.21.15.97:0/3892963145 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.676+0000 7f9e83fff700 1 --1- 172.21.15.97:0/3892963145 >> v1:172.21.15.97:6789/0 conn(0x7f9e840d2210 0x7f9e840d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60862/0 (socket says 172.21.15.97:60862) 2023-07-06T03:46:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.676+0000 7f9e837fe700 1 --2- 172.21.15.97:0/3892963145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8415c610 0x7f9e840b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.676+0000 7f9e82ffd700 1 -- 172.21.15.97:0/3892963145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8415c610 msgr2=0x7f9e840b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.676+0000 7f9e82ffd700 1 --2- 172.21.15.97:0/3892963145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8415c610 0x7f9e840b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.676+0000 7f9e82ffd700 1 -- 172.21.15.97:0/3892963145 >> v1:172.21.15.97:6789/0 conn(0x7f9e840d2210 legacy=0x7f9e840d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.676+0000 7f9e82ffd700 1 -- 172.21.15.97:0/3892963145 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e840b5650 con 0x7f9e8406b360 2023-07-06T03:46:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.676+0000 7f9e827fc700 1 -- 172.21.15.97:0/3892963145 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1701034739 0 0) 0x7f9e840cff90 con 0x7f9e840d2210 2023-07-06T03:46:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.676+0000 7f9e82ffd700 1 --2- 172.21.15.97:0/3892963145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8406b360 0x7f9e840b4cf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009920 tx=0x7f9e78009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b24d9f99445bab server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.677+0000 7f9e827fc700 1 -- 172.21.15.97:0/3892963145 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800ba20 con 0x7f9e8406b360 2023-07-06T03:46:13.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.677+0000 7f9e827fc700 1 -- 172.21.15.97:0/3892963145 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7800bb80 con 0x7f9e8406b360 2023-07-06T03:46:13.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.677+0000 7f9e827fc700 1 -- 172.21.15.97:0/3892963145 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78009e70 con 0x7f9e8406b360 2023-07-06T03:46:13.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.677+0000 7f9e89dff700 1 -- 172.21.15.97:0/3892963145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8406b360 msgr2=0x7f9e840b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.677+0000 7f9e89dff700 1 --2- 172.21.15.97:0/3892963145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8406b360 0x7f9e840b4cf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009920 tx=0x7f9e78009c30 comp rx=0 tx=0).stop 2023-07-06T03:46:13.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.677+0000 7f9e89dff700 1 -- 172.21.15.97:0/3892963145 shutdown_connections 2023-07-06T03:46:13.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.677+0000 7f9e89dff700 1 --2- 172.21.15.97:0/3892963145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8415c610 0x7f9e840b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.677+0000 7f9e89dff700 1 --2- 172.21.15.97:0/3892963145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8406b360 0x7f9e840b4cf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.677+0000 7f9e89dff700 1 -- 172.21.15.97:0/3892963145 >> 172.21.15.97:0/3892963145 conn(0x7f9e8406a6c0 msgr2=0x7f9e8406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:13.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.677+0000 7f9e89dff700 1 -- 172.21.15.97:0/3892963145 shutdown_connections 2023-07-06T03:46:13.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.677+0000 7f9e89dff700 1 -- 172.21.15.97:0/3892963145 wait complete. 2023-07-06T03:46:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.679+0000 7f9e89dff700 1 Processor -- start 2023-07-06T03:46:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.679+0000 7f9e89dff700 1 -- start start 2023-07-06T03:46:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.680+0000 7f9e89dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8406b360 0x7f9e840b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.680+0000 7f9e89dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e840d2210 0x7f9e841f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.680+0000 7f9e89dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8415c610 0x7f9e841585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.680+0000 7f9e89dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e840b4390 con 0x7f9e840d2210 2023-07-06T03:46:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.680+0000 7f9e89dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e841f28c0 con 0x7f9e8406b360 2023-07-06T03:46:13.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.680+0000 7f9e89dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e841f1dc0 con 0x7f9e8415c610 2023-07-06T03:46:13.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e82ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e840d2210 0x7f9e841f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:13.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e82ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e840d2210 0x7f9e841f1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40774/0 (socket says 172.21.15.97:40774) 2023-07-06T03:46:13.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e82ffd700 1 -- 172.21.15.97:0/963562106 learned_addr learned my addr 172.21.15.97:0/963562106 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:13.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e837fe700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8406b360 0x7f9e840b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:13.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e83fff700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8415c610 0x7f9e841585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:13.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e82ffd700 1 -- 172.21.15.97:0/963562106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8415c610 msgr2=0x7f9e841585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e82ffd700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8415c610 0x7f9e841585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e82ffd700 1 -- 172.21.15.97:0/963562106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8406b360 msgr2=0x7f9e840b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e82ffd700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8406b360 0x7f9e840b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e82ffd700 1 -- 172.21.15.97:0/963562106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e78009550 con 0x7f9e840d2210 2023-07-06T03:46:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e837fe700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8406b360 0x7f9e840b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.681+0000 7f9e82ffd700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e840d2210 0x7f9e841f1880 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009130 tx=0x7f9e78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.682+0000 7f9e6bfff700 1 -- 172.21.15.97:0/963562106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78004480 con 0x7f9e840d2210 2023-07-06T03:46:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.682+0000 7f9e6bfff700 1 -- 172.21.15.97:0/963562106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7801f070 con 0x7f9e840d2210 2023-07-06T03:46:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.682+0000 7f9e6bfff700 1 -- 172.21.15.97:0/963562106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e780038c0 con 0x7f9e840d2210 2023-07-06T03:46:13.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.682+0000 7f9e89dff700 1 -- 172.21.15.97:0/963562106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e84158af0 con 0x7f9e840d2210 2023-07-06T03:46:13.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.682+0000 7f9e89dff700 1 -- 172.21.15.97:0/963562106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e841f2300 con 0x7f9e840d2210 2023-07-06T03:46:13.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.683+0000 7f9e89dff700 1 -- 172.21.15.97:0/963562106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8404e340 con 0x7f9e840d2210 2023-07-06T03:46:13.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.684+0000 7f9e6bfff700 1 -- 172.21.15.97:0/963562106 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9e78021030 con 0x7f9e840d2210 2023-07-06T03:46:13.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.684+0000 7f9e6bfff700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9e7006ad10 0x7f9e7006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:13.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.685+0000 7f9e6bfff700 1 -- 172.21.15.97:0/963562106 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e7808a9c0 con 0x7f9e840d2210 2023-07-06T03:46:13.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.685+0000 7f9e837fe700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9e7006ad10 0x7f9e7006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:13.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.685+0000 7f9e837fe700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9e7006ad10 0x7f9e7006d1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e840c64b0 tx=0x7f9e74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:13.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.689+0000 7f9e6bfff700 1 -- 172.21.15.97:0/963562106 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9e78055c20 con 0x7f9e840d2210 2023-07-06T03:46:13.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.891+0000 7f9e89dff700 1 -- 172.21.15.97:0/963562106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e8406afc0 con 0x7f9e840d2210 2023-07-06T03:46:13.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.891+0000 7f9e6bfff700 1 -- 172.21.15.97:0/963562106 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e78015020 con 0x7f9e840d2210 2023-07-06T03:46:13.893 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:46:13.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.893+0000 7f9e89dff700 1 -- 172.21.15.97:0/963562106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9e7006ad10 msgr2=0x7f9e7006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.893+0000 7f9e89dff700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9e7006ad10 0x7f9e7006d1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e840c64b0 tx=0x7f9e74008040 comp rx=0 tx=0).stop 2023-07-06T03:46:13.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.893+0000 7f9e89dff700 1 -- 172.21.15.97:0/963562106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e840d2210 msgr2=0x7f9e841f1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:13.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.893+0000 7f9e89dff700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e840d2210 0x7f9e841f1880 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009130 tx=0x7f9e78004750 comp rx=0 tx=0).stop 2023-07-06T03:46:13.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.893+0000 7f9e89dff700 1 -- 172.21.15.97:0/963562106 shutdown_connections 2023-07-06T03:46:13.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.893+0000 7f9e89dff700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9e7006ad10 0x7f9e7006d1c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.893+0000 7f9e89dff700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9e8415c610 0x7f9e841585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.894+0000 7f9e89dff700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8406b360 0x7f9e840b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.894+0000 7f9e89dff700 1 --2- 172.21.15.97:0/963562106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e840d2210 0x7f9e841f1880 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:13.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.894+0000 7f9e89dff700 1 -- 172.21.15.97:0/963562106 >> 172.21.15.97:0/963562106 conn(0x7f9e8406a6c0 msgr2=0x7f9e8415b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.894+0000 7f9e89dff700 1 -- 172.21.15.97:0/963562106 shutdown_connections 2023-07-06T03:46:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:13.894+0000 7f9e89dff700 1 -- 172.21.15.97:0/963562106 wait complete. 2023-07-06T03:46:13.913 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:46:14.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.108+0000 7f4e8f7ff700 1 Processor -- start 2023-07-06T03:46:14.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.108+0000 7f4e8f7ff700 1 -- start start 2023-07-06T03:46:14.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8f7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4e8815c610 0x7f4e880b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8f7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4e8806b360 0x7f4e880b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8f7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e880d2210 0x7f4e880d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8f7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e880d0780 con 0x7f4e8815c610 2023-07-06T03:46:14.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8f7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8815c9e0 con 0x7f4e8806b360 2023-07-06T03:46:14.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8f7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880b5650 con 0x7f4e880d2210 2023-07-06T03:46:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8d59b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4e8815c610 0x7f4e880b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8d59b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4e8815c610 0x7f4e880b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40790/0 (socket says 172.21.15.97:40790) 2023-07-06T03:46:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8d59b700 1 -- 172.21.15.97:0/1982386531 learned_addr learned my addr 172.21.15.97:0/1982386531 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8cd9a700 1 --2- 172.21.15.97:0/1982386531 >> v2:172.21.15.146:3300/0 conn(0x7f4e8806b360 0x7f4e880b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8dd9c700 1 --2- 172.21.15.97:0/1982386531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e880d2210 0x7f4e880d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8d59b700 1 -- 172.21.15.97:0/1982386531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e880d2210 msgr2=0x7f4e880d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8d59b700 1 --2- 172.21.15.97:0/1982386531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e880d2210 0x7f4e880d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8d59b700 1 -- 172.21.15.97:0/1982386531 >> v2:172.21.15.146:3300/0 conn(0x7f4e8806b360 msgr2=0x7f4e880b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:14.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8d59b700 1 --2- 172.21.15.97:0/1982386531 >> v2:172.21.15.146:3300/0 conn(0x7f4e8806b360 0x7f4e880b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8d59b700 1 -- 172.21.15.97:0/1982386531 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e880d01c0 con 0x7f4e8815c610 2023-07-06T03:46:14.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.110+0000 7f4e8dd9c700 1 --2- 172.21.15.97:0/1982386531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e880d2210 0x7f4e880d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:14.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.111+0000 7f4e8d59b700 1 --2- 172.21.15.97:0/1982386531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e8815c610 0x7f4e880b47b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4e8806a590 tx=0x7f4e84009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca0a1643905d60a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:14.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.111+0000 7f4e7ffff700 1 -- 172.21.15.97:0/1982386531 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84011830 con 0x7f4e8815c610 2023-07-06T03:46:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.111+0000 7f4e7ffff700 1 -- 172.21.15.97:0/1982386531 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e84011990 con 0x7f4e8815c610 2023-07-06T03:46:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.111+0000 7f4e7ffff700 1 -- 172.21.15.97:0/1982386531 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84011d20 con 0x7f4e8815c610 2023-07-06T03:46:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.111+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/1982386531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e8815c610 msgr2=0x7f4e880b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.111+0000 7f4e8f7ff700 1 --2- 172.21.15.97:0/1982386531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e8815c610 0x7f4e880b47b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4e8806a590 tx=0x7f4e84009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.112+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/1982386531 shutdown_connections 2023-07-06T03:46:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.112+0000 7f4e8f7ff700 1 --2- 172.21.15.97:0/1982386531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e880d2210 0x7f4e880d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.112+0000 7f4e8f7ff700 1 --2- 172.21.15.97:0/1982386531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e8815c610 0x7f4e880b47b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.112+0000 7f4e8f7ff700 1 --2- 172.21.15.97:0/1982386531 >> v2:172.21.15.146:3300/0 conn(0x7f4e8806b360 0x7f4e880b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.112+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/1982386531 >> 172.21.15.97:0/1982386531 conn(0x7f4e8806a6c0 msgr2=0x7f4e8806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.112+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/1982386531 shutdown_connections 2023-07-06T03:46:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.112+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/1982386531 wait complete. 2023-07-06T03:46:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.113+0000 7f4e8f7ff700 1 Processor -- start 2023-07-06T03:46:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.114+0000 7f4e8f7ff700 1 -- start start 2023-07-06T03:46:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.114+0000 7f4e8f7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e8806b360 0x7f4e880cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.114+0000 7f4e8f7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e880d2210 0x7f4e8815cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.114+0000 7f4e8f7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e8815c610 0x7f4e8815a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8f7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880cc310 con 0x7f4e880d2210 2023-07-06T03:46:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8f7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e881536b0 con 0x7f4e8806b360 2023-07-06T03:46:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8f7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8815d060 con 0x7f4e8815c610 2023-07-06T03:46:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8cd9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e880d2210 0x7f4e8815cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8cd9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e880d2210 0x7f4e8815cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40804/0 (socket says 172.21.15.97:40804) 2023-07-06T03:46:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8cd9a700 1 -- 172.21.15.97:0/2061881915 learned_addr learned my addr 172.21.15.97:0/2061881915 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8dd9c700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e8815c610 0x7f4e8815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8d59b700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e8806b360 0x7f4e880cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8cd9a700 1 -- 172.21.15.97:0/2061881915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e8815c610 msgr2=0x7f4e8815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8cd9a700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e8815c610 0x7f4e8815a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8cd9a700 1 -- 172.21.15.97:0/2061881915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e8806b360 msgr2=0x7f4e880cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8cd9a700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e8806b360 0x7f4e880cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8cd9a700 1 -- 172.21.15.97:0/2061881915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e84009550 con 0x7f4e880d2210 2023-07-06T03:46:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.115+0000 7f4e8cd9a700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e880d2210 0x7f4e8815cb20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800e930 tx=0x7f4e7800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.116+0000 7f4e7dffb700 1 -- 172.21.15.97:0/2061881915 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78013680 con 0x7f4e880d2210 2023-07-06T03:46:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.116+0000 7f4e7dffb700 1 -- 172.21.15.97:0/2061881915 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e780137e0 con 0x7f4e880d2210 2023-07-06T03:46:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.116+0000 7f4e7dffb700 1 -- 172.21.15.97:0/2061881915 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78013b70 con 0x7f4e880d2210 2023-07-06T03:46:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.116+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/2061881915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e880cb700 con 0x7f4e880d2210 2023-07-06T03:46:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.116+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/2061881915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e880cbcc0 con 0x7f4e880d2210 2023-07-06T03:46:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.117+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/2061881915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e880cc480 con 0x7f4e880d2210 2023-07-06T03:46:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.118+0000 7f4e7dffb700 1 -- 172.21.15.97:0/2061881915 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e78013cd0 con 0x7f4e880d2210 2023-07-06T03:46:14.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.119+0000 7f4e7dffb700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e6c06ace0 0x7f4e6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.119+0000 7f4e7dffb700 1 -- 172.21.15.97:0/2061881915 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e780893d0 con 0x7f4e880d2210 2023-07-06T03:46:14.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.119+0000 7f4e8d59b700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e6c06ace0 0x7f4e6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.119+0000 7f4e8d59b700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e6c06ace0 0x7f4e6c06d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4e84014040 tx=0x7f4e84008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:14.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.123+0000 7f4e7dffb700 1 -- 172.21.15.97:0/2061881915 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4e78054470 con 0x7f4e880d2210 2023-07-06T03:46:14.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.317+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/2061881915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e8804e340 con 0x7f4e880d2210 2023-07-06T03:46:14.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.318+0000 7f4e7dffb700 1 -- 172.21.15.97:0/2061881915 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e78057a90 con 0x7f4e880d2210 2023-07-06T03:46:14.320 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:46:14.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.320+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/2061881915 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e6c06ace0 msgr2=0x7f4e6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:14.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.320+0000 7f4e8f7ff700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e6c06ace0 0x7f4e6c06d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4e84014040 tx=0x7f4e84008030 comp rx=0 tx=0).stop 2023-07-06T03:46:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.320+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/2061881915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e880d2210 msgr2=0x7f4e8815cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.320+0000 7f4e8f7ff700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e880d2210 0x7f4e8815cb20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800e930 tx=0x7f4e7800ec40 comp rx=0 tx=0).stop 2023-07-06T03:46:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.320+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/2061881915 shutdown_connections 2023-07-06T03:46:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.320+0000 7f4e8f7ff700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e6c06ace0 0x7f4e6c06d190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.320+0000 7f4e8f7ff700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e8815c610 0x7f4e8815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.320+0000 7f4e8f7ff700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e8806b360 0x7f4e880cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.320+0000 7f4e8f7ff700 1 --2- 172.21.15.97:0/2061881915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e880d2210 0x7f4e8815cb20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.320+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/2061881915 >> 172.21.15.97:0/2061881915 conn(0x7f4e8806a6c0 msgr2=0x7f4e880b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:14.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.321+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/2061881915 shutdown_connections 2023-07-06T03:46:14.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.321+0000 7f4e8f7ff700 1 -- 172.21.15.97:0/2061881915 wait complete. 2023-07-06T03:46:14.340 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:46:14.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.533+0000 7fb2365ff700 1 Processor -- start 2023-07-06T03:46:14.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.533+0000 7fb2365ff700 1 -- start start 2023-07-06T03:46:14.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.534+0000 7fb2365ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb23015c610 0x7fb2300b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.534+0000 7fb2365ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2300d2210 0x7fb2300d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.535+0000 7fb2365ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2300cff90 con 0x7fb23006b360 2023-07-06T03:46:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.535+0000 7fb2365ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2300d0780 con 0x7fb23015c610 2023-07-06T03:46:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.535+0000 7fb2365ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23015c9e0 con 0x7fb2300d2210 2023-07-06T03:46:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.535+0000 7fb22ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb23015c610 0x7fb2300b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.535+0000 7fb22ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb23015c610 0x7fb2300b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40812/0 (socket says 172.21.15.97:40812) 2023-07-06T03:46:14.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.535+0000 7fb22ffff700 1 -- 172.21.15.97:0/14124950 learned_addr learned my addr 172.21.15.97:0/14124950 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:14.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.535+0000 7fb234b9c700 1 --2- 172.21.15.97:0/14124950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2300d2210 0x7fb2300d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.535+0000 7fb22ffff700 1 -- 172.21.15.97:0/14124950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2300d2210 msgr2=0x7fb2300d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.535+0000 7fb22ffff700 1 --2- 172.21.15.97:0/14124950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2300d2210 0x7fb2300d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.535+0000 7fb22ffff700 1 -- 172.21.15.97:0/14124950 >> v1:172.21.15.146:6789/0 conn(0x7fb23006b360 legacy=0x7fb2300b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.535+0000 7fb22ffff700 1 -- 172.21.15.97:0/14124950 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2300b5650 con 0x7fb23015c610 2023-07-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb22ffff700 1 --2- 172.21.15.97:0/14124950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23015c610 0x7fb2300b47b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb2200098f0 tx=0x7fb220009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8943f73075259ddd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb22effd700 1 -- 172.21.15.97:0/14124950 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220011830 con 0x7fb23015c610 2023-07-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb22effd700 1 -- 172.21.15.97:0/14124950 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb220011990 con 0x7fb23015c610 2023-07-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb22effd700 1 -- 172.21.15.97:0/14124950 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220011d20 con 0x7fb23015c610 2023-07-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb2365ff700 1 -- 172.21.15.97:0/14124950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23015c610 msgr2=0x7fb2300b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb2365ff700 1 --2- 172.21.15.97:0/14124950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23015c610 0x7fb2300b47b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb2200098f0 tx=0x7fb220009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb2365ff700 1 -- 172.21.15.97:0/14124950 shutdown_connections 2023-07-06T03:46:14.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb2365ff700 1 --2- 172.21.15.97:0/14124950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2300d2210 0x7fb2300d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb2365ff700 1 --2- 172.21.15.97:0/14124950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23015c610 0x7fb2300b47b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb2365ff700 1 -- 172.21.15.97:0/14124950 >> 172.21.15.97:0/14124950 conn(0x7fb23006a6c0 msgr2=0x7fb23006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:14.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb2365ff700 1 -- 172.21.15.97:0/14124950 shutdown_connections 2023-07-06T03:46:14.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.536+0000 7fb2365ff700 1 -- 172.21.15.97:0/14124950 wait complete. 2023-07-06T03:46:14.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.538+0000 7fb2365ff700 1 Processor -- start 2023-07-06T03:46:14.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.538+0000 7fb2365ff700 1 -- start start 2023-07-06T03:46:14.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.539+0000 7fb2365ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23006b360 0x7fb23015d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.539+0000 7fb2365ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2300d2210 0x7fb230158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.539+0000 7fb2365ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb23015c610 0x7fb2300caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.539+0000 7fb2365ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c9ea0 con 0x7fb23006b360 2023-07-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.539+0000 7fb2365ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300ca180 con 0x7fb23015c610 2023-07-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.539+0000 7fb2365ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300ca460 con 0x7fb2300d2210 2023-07-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.539+0000 7fb22ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23006b360 0x7fb23015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.539+0000 7fb22ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23006b360 0x7fb23015d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40824/0 (socket says 172.21.15.97:40824) 2023-07-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.539+0000 7fb22ffff700 1 -- 172.21.15.97:0/2711010515 learned_addr learned my addr 172.21.15.97:0/2711010515 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb234b9c700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb23015c610 0x7fb2300caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb22f7fe700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2300d2210 0x7fb230158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb22ffff700 1 -- 172.21.15.97:0/2711010515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2300d2210 msgr2=0x7fb230158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb22ffff700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2300d2210 0x7fb230158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb22ffff700 1 -- 172.21.15.97:0/2711010515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb23015c610 msgr2=0x7fb2300caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb22ffff700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb23015c610 0x7fb2300caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb22ffff700 1 -- 172.21.15.97:0/2711010515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb220009550 con 0x7fb23006b360 2023-07-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb22f7fe700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2300d2210 0x7fb230158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb22ffff700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23006b360 0x7fb23015d980 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb220009c40 tx=0x7fb220004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb22cff9700 1 -- 172.21.15.97:0/2711010515 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220004590 con 0x7fb23006b360 2023-07-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb22cff9700 1 -- 172.21.15.97:0/2711010515 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22001e070 con 0x7fb23006b360 2023-07-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb2365ff700 1 -- 172.21.15.97:0/2711010515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2300ca740 con 0x7fb23006b360 2023-07-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.540+0000 7fb22cff9700 1 -- 172.21.15.97:0/2711010515 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2200038e0 con 0x7fb23006b360 2023-07-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.541+0000 7fb2365ff700 1 -- 172.21.15.97:0/2711010515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23015dec0 con 0x7fb23006b360 2023-07-06T03:46:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.542+0000 7fb2365ff700 1 -- 172.21.15.97:0/2711010515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23004e340 con 0x7fb23006b360 2023-07-06T03:46:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.542+0000 7fb22cff9700 1 -- 172.21.15.97:0/2711010515 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb220003a40 con 0x7fb23006b360 2023-07-06T03:46:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.543+0000 7fb22cff9700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb21c06ace0 0x7fb21c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.543+0000 7fb22cff9700 1 -- 172.21.15.97:0/2711010515 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb22008a3b0 con 0x7fb23006b360 2023-07-06T03:46:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.543+0000 7fb22f7fe700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb21c06ace0 0x7fb21c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.544+0000 7fb22f7fe700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb21c06ace0 0x7fb21c06d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb2300b4fd0 tx=0x7fb224009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:14.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.548+0000 7fb22cff9700 1 -- 172.21.15.97:0/2711010515 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb220055470 con 0x7fb23006b360 2023-07-06T03:46:14.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.760+0000 7fb2365ff700 1 -- 172.21.15.97:0/2711010515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb230066010 con 0x7fb23006b360 2023-07-06T03:46:14.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.761+0000 7fb22cff9700 1 -- 172.21.15.97:0/2711010515 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb220058a90 con 0x7fb23006b360 2023-07-06T03:46:14.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 -- 172.21.15.97:0/2711010515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb21c06ace0 msgr2=0x7fb21c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:14.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb21c06ace0 0x7fb21c06d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb2300b4fd0 tx=0x7fb224009040 comp rx=0 tx=0).stop 2023-07-06T03:46:14.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 -- 172.21.15.97:0/2711010515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23006b360 msgr2=0x7fb23015d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:14.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23006b360 0x7fb23015d980 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb220009c40 tx=0x7fb220004750 comp rx=0 tx=0).stop 2023-07-06T03:46:14.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 -- 172.21.15.97:0/2711010515 shutdown_connections 2023-07-06T03:46:14.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb21c06ace0 0x7fb21c06d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb23015c610 0x7fb2300caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23006b360 0x7fb23015d980 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 --2- 172.21.15.97:0/2711010515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb2300d2210 0x7fb230158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:14.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 -- 172.21.15.97:0/2711010515 >> 172.21.15.97:0/2711010515 conn(0x7fb23006a6c0 msgr2=0x7fb2300b5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:14.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 -- 172.21.15.97:0/2711010515 shutdown_connections 2023-07-06T03:46:14.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:14.763+0000 7fb2365ff700 1 -- 172.21.15.97:0/2711010515 wait complete. 2023-07-06T03:46:14.767 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:46:14.783 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:46:14.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:14.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-07-06T03:46:14.852 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:46:14.853 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:46:14.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:14.854 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T03:46:14.900 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:46:14.901 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:14.951 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:46:14.952 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:15.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:15.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.193+0000 7fd71e1ff700 1 Processor -- start 2023-07-06T03:46:15.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.193+0000 7fd71e1ff700 1 -- start start 2023-07-06T03:46:15.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd71e1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd7180cd390 0x7fd718160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:15.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd71e1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd718160dd0 0x7fd7180b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:15.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd71e1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7180cdae0 con 0x7fd71815c520 2023-07-06T03:46:15.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd71e1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7180b6640 con 0x7fd7180cd390 2023-07-06T03:46:15.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd71e1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7180b6920 con 0x7fd718160dd0 2023-07-06T03:46:15.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd70ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd7180cd390 0x7fd718160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:15.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd70ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd7180cd390 0x7fd718160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40838/0 (socket says 172.21.15.97:40838) 2023-07-06T03:46:15.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd70ffff700 1 -- 172.21.15.97:0/1537631591 learned_addr learned my addr 172.21.15.97:0/1537631591 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:15.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd717fff700 1 --2- 172.21.15.97:0/1537631591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd718160dd0 0x7fd7180b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:15.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd70ffff700 1 -- 172.21.15.97:0/1537631591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd718160dd0 msgr2=0x7fd7180b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:15.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd70ffff700 1 --2- 172.21.15.97:0/1537631591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd718160dd0 0x7fd7180b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd70ffff700 1 -- 172.21.15.97:0/1537631591 >> v1:172.21.15.146:6789/0 conn(0x7fd71815c520 legacy=0x7fd718160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.195+0000 7fd70ffff700 1 -- 172.21.15.97:0/1537631591 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7180b6c00 con 0x7fd7180cd390 2023-07-06T03:46:15.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.196+0000 7fd70ffff700 1 --2- 172.21.15.97:0/1537631591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7180cd390 0x7fd718160890 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7000098f0 tx=0x7fd700009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed4c7c94bbe3e06 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:15.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.196+0000 7fd716ffd700 1 -- 172.21.15.97:0/1537631591 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70000b8a0 con 0x7fd7180cd390 2023-07-06T03:46:15.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.196+0000 7fd716ffd700 1 -- 172.21.15.97:0/1537631591 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70000ba00 con 0x7fd7180cd390 2023-07-06T03:46:15.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.196+0000 7fd716ffd700 1 -- 172.21.15.97:0/1537631591 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70000bd90 con 0x7fd7180cd390 2023-07-06T03:46:15.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.196+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1537631591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7180cd390 msgr2=0x7fd718160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.196+0000 7fd71e1ff700 1 --2- 172.21.15.97:0/1537631591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7180cd390 0x7fd718160890 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7000098f0 tx=0x7fd700009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:15.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.196+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1537631591 shutdown_connections 2023-07-06T03:46:15.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.196+0000 7fd71e1ff700 1 --2- 172.21.15.97:0/1537631591 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd718160dd0 0x7fd7180b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.196+0000 7fd71e1ff700 1 --2- 172.21.15.97:0/1537631591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7180cd390 0x7fd718160890 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.196+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1537631591 >> 172.21.15.97:0/1537631591 conn(0x7fd718071790 msgr2=0x7fd718071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:15.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.197+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1537631591 shutdown_connections 2023-07-06T03:46:15.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.197+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1537631591 wait complete. 2023-07-06T03:46:15.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.198+0000 7fd71e1ff700 1 Processor -- start 2023-07-06T03:46:15.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.198+0000 7fd71e1ff700 1 -- start start 2023-07-06T03:46:15.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.199+0000 7fd71e1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7180cd390 0x7fd7180cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:15.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.199+0000 7fd71e1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd71815c520 0x7fd7180d3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:15.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.199+0000 7fd71e1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd718160dd0 0x7fd71815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:15.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.199+0000 7fd71e1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7180d3a70 con 0x7fd71815c520 2023-07-06T03:46:15.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.199+0000 7fd71e1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71815f1d0 con 0x7fd718160dd0 2023-07-06T03:46:15.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.199+0000 7fd71e1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71815fa30 con 0x7fd7180cd390 2023-07-06T03:46:15.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd70ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd71815c520 0x7fd7180d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:15.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd7177fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7180cd390 0x7fd7180cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:15.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd70ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd71815c520 0x7fd7180d3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40840/0 (socket says 172.21.15.97:40840) 2023-07-06T03:46:15.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd70ffff700 1 -- 172.21.15.97:0/1499255103 learned_addr learned my addr 172.21.15.97:0/1499255103 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd717fff700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd718160dd0 0x7fd71815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd70ffff700 1 -- 172.21.15.97:0/1499255103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7180cd390 msgr2=0x7fd7180cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd70ffff700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7180cd390 0x7fd7180cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd70ffff700 1 -- 172.21.15.97:0/1499255103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd718160dd0 msgr2=0x7fd71815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd70ffff700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd718160dd0 0x7fd71815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd70ffff700 1 -- 172.21.15.97:0/1499255103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700009550 con 0x7fd71815c520 2023-07-06T03:46:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd717fff700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd718160dd0 0x7fd71815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd70ffff700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd71815c520 0x7fd7180d3530 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd700000c00 tx=0x7fd700004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.200+0000 7fd714ff9700 1 -- 172.21.15.97:0/1499255103 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700004380 con 0x7fd71815c520 2023-07-06T03:46:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.201+0000 7fd714ff9700 1 -- 172.21.15.97:0/1499255103 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70001f070 con 0x7fd71815c520 2023-07-06T03:46:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.201+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1499255103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71815f470 con 0x7fd71815c520 2023-07-06T03:46:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.201+0000 7fd714ff9700 1 -- 172.21.15.97:0/1499255103 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7000036a0 con 0x7fd71815c520 2023-07-06T03:46:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.201+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1499255103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7180cab00 con 0x7fd71815c520 2023-07-06T03:46:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.202+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1499255103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71804e340 con 0x7fd71815c520 2023-07-06T03:46:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.203+0000 7fd714ff9700 1 -- 172.21.15.97:0/1499255103 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd700021030 con 0x7fd71815c520 2023-07-06T03:46:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.203+0000 7fd714ff9700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd6f806ad10 0x7fd6f806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.204+0000 7fd714ff9700 1 -- 172.21.15.97:0/1499255103 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd70008a850 con 0x7fd71815c520 2023-07-06T03:46:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.204+0000 7fd7177fe700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd6f806ad10 0x7fd6f806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.204+0000 7fd7177fe700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd6f806ad10 0x7fd6f806d1c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd708007fd0 tx=0x7fd708009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:15.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.208+0000 7fd714ff9700 1 -- 172.21.15.97:0/1499255103 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd700055860 con 0x7fd71815c520 2023-07-06T03:46:15.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.408+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1499255103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7180b4f40 con 0x7fd71815c520 2023-07-06T03:46:15.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.409+0000 7fd714ff9700 1 -- 172.21.15.97:0/1499255103 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7fd700058e80 con 0x7fd71815c520 2023-07-06T03:46:15.411 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:15.411 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:15.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.411+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1499255103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd6f806ad10 msgr2=0x7fd6f806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.411+0000 7fd71e1ff700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd6f806ad10 0x7fd6f806d1c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd708007fd0 tx=0x7fd708009040 comp rx=0 tx=0).stop 2023-07-06T03:46:15.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.411+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1499255103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd71815c520 msgr2=0x7fd7180d3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.411+0000 7fd71e1ff700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd71815c520 0x7fd7180d3530 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd700000c00 tx=0x7fd700004750 comp rx=0 tx=0).stop 2023-07-06T03:46:15.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.411+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1499255103 shutdown_connections 2023-07-06T03:46:15.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.411+0000 7fd71e1ff700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd6f806ad10 0x7fd6f806d1c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.412+0000 7fd71e1ff700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd718160dd0 0x7fd71815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.412+0000 7fd71e1ff700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd7180cd390 0x7fd7180cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.412+0000 7fd71e1ff700 1 --2- 172.21.15.97:0/1499255103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd71815c520 0x7fd7180d3530 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.412+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1499255103 >> 172.21.15.97:0/1499255103 conn(0x7fd718071790 msgr2=0x7fd718071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:15.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.412+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1499255103 shutdown_connections 2023-07-06T03:46:15.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.412+0000 7fd71e1ff700 1 -- 172.21.15.97:0/1499255103 wait complete. 2023-07-06T03:46:15.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:46:15.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:46:15.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.628+0000 7fb4b65ff700 1 Processor -- start 2023-07-06T03:46:15.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.629+0000 7fb4b65ff700 1 -- start start 2023-07-06T03:46:15.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.630+0000 7fb4b65ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb4b00d2210 0x7fb4b00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:15.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.630+0000 7fb4b65ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b00cff90 con 0x7fb4b006b360 2023-07-06T03:46:15.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.630+0000 7fb4b65ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b00b5650 con 0x7fb4b015c610 2023-07-06T03:46:15.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.630+0000 7fb4b65ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b00d0780 con 0x7fb4b00d2210 2023-07-06T03:46:15.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.630+0000 7fb4af7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb4b006b360 0x7fb4b00d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60872/0 (socket says 172.21.15.97:60872) 2023-07-06T03:46:15.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.630+0000 7fb4af7fe700 1 -- 172.21.15.97:0/1955535240 learned_addr learned my addr 172.21.15.97:0/1955535240 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:15.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.630+0000 7fb4b4b9c700 1 --2- 172.21.15.97:0/1955535240 >> v2:172.21.15.146:3300/0 conn(0x7fb4b00d2210 0x7fb4b00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:15.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.630+0000 7fb4aeffd700 1 -- 172.21.15.97:0/1955535240 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656795948 0 0) 0x7fb4b00cff90 con 0x7fb4b006b360 2023-07-06T03:46:15.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.631+0000 7fb4aeffd700 1 -- 172.21.15.97:0/1955535240 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4980035b0 con 0x7fb4b006b360 2023-07-06T03:46:15.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.631+0000 7fb4aeffd700 1 -- 172.21.15.97:0/1955535240 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069673063 0 0) 0x7fb4b00b5650 con 0x7fb4b015c610 2023-07-06T03:46:15.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.631+0000 7fb4aeffd700 1 -- 172.21.15.97:0/1955535240 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b00cff90 con 0x7fb4b015c610 2023-07-06T03:46:15.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.631+0000 7fb4b4b9c700 1 -- 172.21.15.97:0/1955535240 >> v1:172.21.15.146:6789/0 conn(0x7fb4b015c610 legacy=0x7fb4b00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.631+0000 7fb4aeffd700 1 -- 172.21.15.97:0/1955535240 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926027553 0 0) 0x7fb4980035b0 con 0x7fb4b006b360 2023-07-06T03:46:15.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.631+0000 7fb4b4b9c700 1 -- 172.21.15.97:0/1955535240 >> v1:172.21.15.97:6789/0 conn(0x7fb4b006b360 legacy=0x7fb4b00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.631+0000 7fb4b4b9c700 1 -- 172.21.15.97:0/1955535240 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b00b5400 con 0x7fb4b00d2210 2023-07-06T03:46:15.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.631+0000 7fb4b4b9c700 1 --2- 172.21.15.97:0/1955535240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4b00d2210 0x7fb4b00b4ec0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009d90 tx=0x7fb4a800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7684887f142a3386 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:15.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.631+0000 7fb4aeffd700 1 -- 172.21.15.97:0/1955535240 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a800e070 con 0x7fb4b00d2210 2023-07-06T03:46:15.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.631+0000 7fb4aeffd700 1 -- 172.21.15.97:0/1955535240 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a8003a20 con 0x7fb4b00d2210 2023-07-06T03:46:15.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.631+0000 7fb4aeffd700 1 -- 172.21.15.97:0/1955535240 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4001670 con 0x7fb4b00d2210 2023-07-06T03:46:15.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.632+0000 7fb4b65ff700 1 -- 172.21.15.97:0/1955535240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4b00d2210 msgr2=0x7fb4b00b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.632+0000 7fb4b65ff700 1 --2- 172.21.15.97:0/1955535240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4b00d2210 0x7fb4b00b4ec0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009d90 tx=0x7fb4a800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:15.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.632+0000 7fb4b65ff700 1 -- 172.21.15.97:0/1955535240 shutdown_connections 2023-07-06T03:46:15.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.632+0000 7fb4b65ff700 1 --2- 172.21.15.97:0/1955535240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4b00d2210 0x7fb4b00b4ec0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.632+0000 7fb4b65ff700 1 -- 172.21.15.97:0/1955535240 >> 172.21.15.97:0/1955535240 conn(0x7fb4b006a6c0 msgr2=0x7fb4b006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:15.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.632+0000 7fb4b65ff700 1 -- 172.21.15.97:0/1955535240 shutdown_connections 2023-07-06T03:46:15.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.632+0000 7fb4b65ff700 1 -- 172.21.15.97:0/1955535240 wait complete. 2023-07-06T03:46:15.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.634+0000 7fb4b65ff700 1 Processor -- start 2023-07-06T03:46:15.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.634+0000 7fb4b65ff700 1 -- start start 2023-07-06T03:46:15.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.635+0000 7fb4b65ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4b006b360 0x7fb4b00c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:15.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.635+0000 7fb4b65ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4b00d2210 0x7fb4b0153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:15.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.635+0000 7fb4b65ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4b015c610 0x7fb4b015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:15.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.635+0000 7fb4b65ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c7bc0 con 0x7fb4b015c610 2023-07-06T03:46:15.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.635+0000 7fb4b65ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c7ea0 con 0x7fb4b00d2210 2023-07-06T03:46:15.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.635+0000 7fb4b65ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c8180 con 0x7fb4b006b360 2023-07-06T03:46:15.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.635+0000 7fb4b4b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4b015c610 0x7fb4b015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:15.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.635+0000 7fb4b4b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4b015c610 0x7fb4b015ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40856/0 (socket says 172.21.15.97:40856) 2023-07-06T03:46:15.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.635+0000 7fb4b4b9c700 1 -- 172.21.15.97:0/3757799594 learned_addr learned my addr 172.21.15.97:0/3757799594 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:15.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.635+0000 7fb4af7fe700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4b00d2210 0x7fb4b0153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:15.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.635+0000 7fb4affff700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4b006b360 0x7fb4b00c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:15.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.636+0000 7fb4b4b9c700 1 -- 172.21.15.97:0/3757799594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4b006b360 msgr2=0x7fb4b00c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.636+0000 7fb4b4b9c700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4b006b360 0x7fb4b00c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.636+0000 7fb4b4b9c700 1 -- 172.21.15.97:0/3757799594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4b00d2210 msgr2=0x7fb4b0153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.636+0000 7fb4b4b9c700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4b00d2210 0x7fb4b0153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.636+0000 7fb4b4b9c700 1 -- 172.21.15.97:0/3757799594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a8009a70 con 0x7fb4b015c610 2023-07-06T03:46:15.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.636+0000 7fb4b4b9c700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4b015c610 0x7fb4b015ec90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800b370 tx=0x7fb4a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:15.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.636+0000 7fb4acff9700 1 -- 172.21.15.97:0/3757799594 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8004440 con 0x7fb4b015c610 2023-07-06T03:46:15.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.636+0000 7fb4acff9700 1 -- 172.21.15.97:0/3757799594 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a80045a0 con 0x7fb4b015c610 2023-07-06T03:46:15.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.636+0000 7fb4b65ff700 1 -- 172.21.15.97:0/3757799594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b00c8460 con 0x7fb4b015c610 2023-07-06T03:46:15.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.636+0000 7fb4acff9700 1 -- 172.21.15.97:0/3757799594 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8004950 con 0x7fb4b015c610 2023-07-06T03:46:15.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.636+0000 7fb4b65ff700 1 -- 172.21.15.97:0/3757799594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b00c7650 con 0x7fb4b015c610 2023-07-06T03:46:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.637+0000 7fb4b65ff700 1 -- 172.21.15.97:0/3757799594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b00c8980 con 0x7fb4b015c610 2023-07-06T03:46:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.638+0000 7fb4acff9700 1 -- 172.21.15.97:0/3757799594 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb4a801d470 con 0x7fb4b015c610 2023-07-06T03:46:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.639+0000 7fb4acff9700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb49c06ace0 0x7fb49c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.639+0000 7fb4acff9700 1 -- 172.21.15.97:0/3757799594 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4a808aa70 con 0x7fb4b015c610 2023-07-06T03:46:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.639+0000 7fb4affff700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb49c06ace0 0x7fb49c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.640+0000 7fb4affff700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb49c06ace0 0x7fb49c06d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00011e0 tx=0x7fb4a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:15.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.644+0000 7fb4acff9700 1 -- 172.21.15.97:0/3757799594 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb4b00c8980 con 0x7fb4b015c610 2023-07-06T03:46:15.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.845+0000 7fb4b65ff700 1 -- 172.21.15.97:0/3757799594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4b00c8980 con 0x7fb4b015c610 2023-07-06T03:46:15.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.845+0000 7fb4acff9700 1 -- 172.21.15.97:0/3757799594 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4b00c8980 con 0x7fb4b015c610 2023-07-06T03:46:15.848 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:46:15.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.847+0000 7fb4b65ff700 1 -- 172.21.15.97:0/3757799594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb49c06ace0 msgr2=0x7fb49c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.847+0000 7fb4b65ff700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb49c06ace0 0x7fb49c06d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00011e0 tx=0x7fb4a0009040 comp rx=0 tx=0).stop 2023-07-06T03:46:15.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.847+0000 7fb4b65ff700 1 -- 172.21.15.97:0/3757799594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4b015c610 msgr2=0x7fb4b015ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:15.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.847+0000 7fb4b65ff700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4b015c610 0x7fb4b015ec90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800b370 tx=0x7fb4a8004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.848+0000 7fb4b65ff700 1 -- 172.21.15.97:0/3757799594 shutdown_connections 2023-07-06T03:46:15.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.848+0000 7fb4b65ff700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb49c06ace0 0x7fb49c06d190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.848+0000 7fb4b65ff700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4b015c610 0x7fb4b015ec90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.848+0000 7fb4b65ff700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4b006b360 0x7fb4b00c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.848+0000 7fb4b65ff700 1 --2- 172.21.15.97:0/3757799594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4b00d2210 0x7fb4b0153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.848+0000 7fb4b65ff700 1 -- 172.21.15.97:0/3757799594 >> 172.21.15.97:0/3757799594 conn(0x7fb4b006a6c0 msgr2=0x7fb4b00d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.848+0000 7fb4b65ff700 1 -- 172.21.15.97:0/3757799594 shutdown_connections 2023-07-06T03:46:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:15.848+0000 7fb4b65ff700 1 -- 172.21.15.97:0/3757799594 wait complete. 2023-07-06T03:46:15.867 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:46:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.064+0000 7f85341ff700 1 Processor -- start 2023-07-06T03:46:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.064+0000 7f85341ff700 1 -- start start 2023-07-06T03:46:16.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.066+0000 7f85341ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c15c1c0 0x7f852c0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.066+0000 7f85341ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f852c15c590 0x7f852c15d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.066+0000 7f85341ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c0ccc30 con 0x7f852c0719c0 2023-07-06T03:46:16.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.066+0000 7f85341ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852c0ccf10 con 0x7f852c15c590 2023-07-06T03:46:16.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.066+0000 7f85341ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0cd1f0 con 0x7f852c15c1c0 2023-07-06T03:46:16.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.066+0000 7f8531f9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c15c1c0 0x7f852c0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.066+0000 7f853179a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f852c15c590 0x7f852c15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.066+0000 7f8531f9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c15c1c0 0x7f852c0cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:35434/0 (socket says 172.21.15.97:35434) 2023-07-06T03:46:16.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.066+0000 7f853279c700 1 --1- 172.21.15.97:0/2504720650 >> v1:172.21.15.146:6789/0 conn(0x7f852c0719c0 0x7f852c06a060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:41946/0 (socket says 172.21.15.97:41946) 2023-07-06T03:46:16.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.066+0000 7f8531f9b700 1 -- 172.21.15.97:0/2504720650 learned_addr learned my addr 172.21.15.97:0/2504720650 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.067+0000 7f8530f99700 1 -- 172.21.15.97:0/2504720650 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612797617 0 0) 0x7f852c0ccc30 con 0x7f852c0719c0 2023-07-06T03:46:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.067+0000 7f8530f99700 1 -- 172.21.15.97:0/2504720650 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85180035b0 con 0x7f852c0719c0 2023-07-06T03:46:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.067+0000 7f8531f9b700 1 -- 172.21.15.97:0/2504720650 >> v2:172.21.15.146:3300/0 conn(0x7f852c15c590 msgr2=0x7f852c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.067+0000 7f8531f9b700 1 --2- 172.21.15.97:0/2504720650 >> v2:172.21.15.146:3300/0 conn(0x7f852c15c590 0x7f852c15d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.067+0000 7f8531f9b700 1 -- 172.21.15.97:0/2504720650 >> v1:172.21.15.146:6789/0 conn(0x7f852c0719c0 legacy=0x7f852c06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.067+0000 7f8531f9b700 1 -- 172.21.15.97:0/2504720650 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c15cb00 con 0x7f852c15c1c0 2023-07-06T03:46:16.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.067+0000 7f853179a700 1 --2- 172.21.15.97:0/2504720650 >> v2:172.21.15.146:3300/0 conn(0x7f852c15c590 0x7f852c15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:46:16.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.067+0000 7f8531f9b700 1 --2- 172.21.15.97:0/2504720650 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c15c1c0 0x7f852c0cc3e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85280098f0 tx=0x7f8528009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4a41be8d1dd803 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:16.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.068+0000 7f8530f99700 1 -- 172.21.15.97:0/2504720650 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528011830 con 0x7f852c15c1c0 2023-07-06T03:46:16.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.068+0000 7f8530f99700 1 -- 172.21.15.97:0/2504720650 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8528011990 con 0x7f852c15c1c0 2023-07-06T03:46:16.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.068+0000 7f8530f99700 1 -- 172.21.15.97:0/2504720650 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528011d20 con 0x7f852c15c1c0 2023-07-06T03:46:16.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.068+0000 7f85341ff700 1 -- 172.21.15.97:0/2504720650 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c15c1c0 msgr2=0x7f852c0cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.068+0000 7f85341ff700 1 --2- 172.21.15.97:0/2504720650 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c15c1c0 0x7f852c0cc3e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85280098f0 tx=0x7f8528009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:16.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.068+0000 7f85341ff700 1 -- 172.21.15.97:0/2504720650 shutdown_connections 2023-07-06T03:46:16.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.068+0000 7f85341ff700 1 --2- 172.21.15.97:0/2504720650 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c15c1c0 0x7f852c0cc3e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.068+0000 7f85341ff700 1 --2- 172.21.15.97:0/2504720650 >> v2:172.21.15.146:3300/0 conn(0x7f852c15c590 0x7f852c15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.068+0000 7f85341ff700 1 -- 172.21.15.97:0/2504720650 >> 172.21.15.97:0/2504720650 conn(0x7f852c0b1650 msgr2=0x7f852c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:16.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.068+0000 7f85341ff700 1 -- 172.21.15.97:0/2504720650 shutdown_connections 2023-07-06T03:46:16.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.068+0000 7f85341ff700 1 -- 172.21.15.97:0/2504720650 wait complete. 2023-07-06T03:46:16.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.070+0000 7f85341ff700 1 Processor -- start 2023-07-06T03:46:16.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.070+0000 7f85341ff700 1 -- start start 2023-07-06T03:46:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.071+0000 7f85341ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c0719c0 0x7f852c0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.071+0000 7f85341ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c15c1c0 0x7f852c0b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.071+0000 7f85341ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f852c15c590 0x7f852c0b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.071+0000 7f85341ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c13fd00 con 0x7f852c15c1c0 2023-07-06T03:46:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.071+0000 7f85341ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c13ffa0 con 0x7f852c15c590 2023-07-06T03:46:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.071+0000 7f85341ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c140280 con 0x7f852c0719c0 2023-07-06T03:46:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.071+0000 7f853179a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c15c1c0 0x7f852c0b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.071+0000 7f853179a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c15c1c0 0x7f852c0b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40870/0 (socket says 172.21.15.97:40870) 2023-07-06T03:46:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.071+0000 7f853179a700 1 -- 172.21.15.97:0/3617520357 learned_addr learned my addr 172.21.15.97:0/3617520357 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f853279c700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f852c15c590 0x7f852c0b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f8531f9b700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c0719c0 0x7f852c0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f853179a700 1 -- 172.21.15.97:0/3617520357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c0719c0 msgr2=0x7f852c0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f853179a700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c0719c0 0x7f852c0b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f853179a700 1 -- 172.21.15.97:0/3617520357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f852c15c590 msgr2=0x7f852c0b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f853179a700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f852c15c590 0x7f852c0b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f853179a700 1 -- 172.21.15.97:0/3617520357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0094a0 con 0x7f852c15c1c0 2023-07-06T03:46:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f853179a700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c15c1c0 0x7f852c0b4290 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f851c00ebe0 tx=0x7f851c009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f85227fc700 1 -- 172.21.15.97:0/3617520357 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00c8b0 con 0x7f852c15c1c0 2023-07-06T03:46:16.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f85227fc700 1 -- 172.21.15.97:0/3617520357 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c00ca10 con 0x7f852c15c1c0 2023-07-06T03:46:16.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f85341ff700 1 -- 172.21.15.97:0/3617520357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8528009550 con 0x7f852c15c1c0 2023-07-06T03:46:16.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f85227fc700 1 -- 172.21.15.97:0/3617520357 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00cda0 con 0x7f852c15c1c0 2023-07-06T03:46:16.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.072+0000 7f85341ff700 1 -- 172.21.15.97:0/3617520357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c0b38b0 con 0x7f852c15c1c0 2023-07-06T03:46:16.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.073+0000 7f85341ff700 1 -- 172.21.15.97:0/3617520357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c04e340 con 0x7f852c15c1c0 2023-07-06T03:46:16.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.074+0000 7f85227fc700 1 -- 172.21.15.97:0/3617520357 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f851c013470 con 0x7f852c15c1c0 2023-07-06T03:46:16.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.075+0000 7f85227fc700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f851006ace0 0x7f851006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.075+0000 7f85227fc700 1 -- 172.21.15.97:0/3617520357 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f851c0895d0 con 0x7f852c15c1c0 2023-07-06T03:46:16.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.076+0000 7f8531f9b700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f851006ace0 0x7f851006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.076+0000 7f8531f9b700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f851006ace0 0x7f851006d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8528009c40 tx=0x7f8528008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:16.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.080+0000 7f85227fc700 1 -- 172.21.15.97:0/3617520357 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f851c054680 con 0x7f852c15c1c0 2023-07-06T03:46:16.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.275+0000 7f85341ff700 1 -- 172.21.15.97:0/3617520357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f852c071290 con 0x7f852c15c1c0 2023-07-06T03:46:16.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.276+0000 7f85227fc700 1 -- 172.21.15.97:0/3617520357 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f851c057ca0 con 0x7f852c15c1c0 2023-07-06T03:46:16.278 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:46:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.278+0000 7f85341ff700 1 -- 172.21.15.97:0/3617520357 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f851006ace0 msgr2=0x7f851006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.278+0000 7f85341ff700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f851006ace0 0x7f851006d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8528009c40 tx=0x7f8528008030 comp rx=0 tx=0).stop 2023-07-06T03:46:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.278+0000 7f85341ff700 1 -- 172.21.15.97:0/3617520357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c15c1c0 msgr2=0x7f852c0b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.278+0000 7f85341ff700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c15c1c0 0x7f852c0b4290 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f851c00ebe0 tx=0x7f851c009e20 comp rx=0 tx=0).stop 2023-07-06T03:46:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.279+0000 7f85341ff700 1 -- 172.21.15.97:0/3617520357 shutdown_connections 2023-07-06T03:46:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.279+0000 7f85341ff700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f851006ace0 0x7f851006d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.279+0000 7f85341ff700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f852c15c590 0x7f852c0b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.279+0000 7f85341ff700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f852c0719c0 0x7f852c0b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.279+0000 7f85341ff700 1 --2- 172.21.15.97:0/3617520357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f852c15c1c0 0x7f852c0b4290 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.279+0000 7f85341ff700 1 -- 172.21.15.97:0/3617520357 >> 172.21.15.97:0/3617520357 conn(0x7f852c0b1650 msgr2=0x7f852c06a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.279+0000 7f85341ff700 1 -- 172.21.15.97:0/3617520357 shutdown_connections 2023-07-06T03:46:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.279+0000 7f85341ff700 1 -- 172.21.15.97:0/3617520357 wait complete. 2023-07-06T03:46:16.298 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:46:16.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.494+0000 7fd915fff700 1 Processor -- start 2023-07-06T03:46:16.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.494+0000 7fd915fff700 1 -- start start 2023-07-06T03:46:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.495+0000 7fd915fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91015c610 0x7fd9100b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.495+0000 7fd915fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd91006b360 0x7fd9100b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.495+0000 7fd915fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9100d2210 0x7fd9100d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.495+0000 7fd915fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9100d0780 con 0x7fd9100d2210 2023-07-06T03:46:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.495+0000 7fd915fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91015c9e0 con 0x7fd91006b360 2023-07-06T03:46:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.495+0000 7fd915fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100b5650 con 0x7fd91015c610 2023-07-06T03:46:16.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.495+0000 7fd90ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9100d2210 0x7fd9100d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.495+0000 7fd90ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9100d2210 0x7fd9100d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40878/0 (socket says 172.21.15.97:40878) 2023-07-06T03:46:16.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.495+0000 7fd90ffff700 1 -- 172.21.15.97:0/2777660069 learned_addr learned my addr 172.21.15.97:0/2777660069 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90f7fe700 1 --2- 172.21.15.97:0/2777660069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91015c610 0x7fd9100b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90effd700 1 --2- 172.21.15.97:0/2777660069 >> v2:172.21.15.146:3300/0 conn(0x7fd91006b360 0x7fd9100b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90ffff700 1 -- 172.21.15.97:0/2777660069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91015c610 msgr2=0x7fd9100b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:16.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90ffff700 1 --2- 172.21.15.97:0/2777660069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91015c610 0x7fd9100b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90ffff700 1 -- 172.21.15.97:0/2777660069 >> v2:172.21.15.146:3300/0 conn(0x7fd91006b360 msgr2=0x7fd9100b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:16.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90ffff700 1 --2- 172.21.15.97:0/2777660069 >> v2:172.21.15.146:3300/0 conn(0x7fd91006b360 0x7fd9100b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90ffff700 1 -- 172.21.15.97:0/2777660069 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9100d01c0 con 0x7fd9100d2210 2023-07-06T03:46:16.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90f7fe700 1 --2- 172.21.15.97:0/2777660069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91015c610 0x7fd9100b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:16.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90effd700 1 --2- 172.21.15.97:0/2777660069 >> v2:172.21.15.146:3300/0 conn(0x7fd91006b360 0x7fd9100b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90ffff700 1 --2- 172.21.15.97:0/2777660069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9100d2210 0x7fd9100d1310 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd9100b58b0 tx=0x7fd90800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0a6db09126bd96 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90e7fc700 1 -- 172.21.15.97:0/2777660069 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90800e070 con 0x7fd9100d2210 2023-07-06T03:46:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90e7fc700 1 -- 172.21.15.97:0/2777660069 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90800b980 con 0x7fd9100d2210 2023-07-06T03:46:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.496+0000 7fd90e7fc700 1 -- 172.21.15.97:0/2777660069 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90800bd30 con 0x7fd9100d2210 2023-07-06T03:46:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.497+0000 7fd915fff700 1 -- 172.21.15.97:0/2777660069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9100d2210 msgr2=0x7fd9100d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.497+0000 7fd915fff700 1 --2- 172.21.15.97:0/2777660069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9100d2210 0x7fd9100d1310 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd9100b58b0 tx=0x7fd90800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.497+0000 7fd915fff700 1 -- 172.21.15.97:0/2777660069 shutdown_connections 2023-07-06T03:46:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.497+0000 7fd915fff700 1 --2- 172.21.15.97:0/2777660069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9100d2210 0x7fd9100d1310 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.497+0000 7fd915fff700 1 --2- 172.21.15.97:0/2777660069 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91015c610 0x7fd9100b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.497+0000 7fd915fff700 1 --2- 172.21.15.97:0/2777660069 >> v2:172.21.15.146:3300/0 conn(0x7fd91006b360 0x7fd9100b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.497+0000 7fd915fff700 1 -- 172.21.15.97:0/2777660069 >> 172.21.15.97:0/2777660069 conn(0x7fd91006a6c0 msgr2=0x7fd91006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.497+0000 7fd915fff700 1 -- 172.21.15.97:0/2777660069 shutdown_connections 2023-07-06T03:46:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.497+0000 7fd915fff700 1 -- 172.21.15.97:0/2777660069 wait complete. 2023-07-06T03:46:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.499+0000 7fd915fff700 1 Processor -- start 2023-07-06T03:46:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.499+0000 7fd915fff700 1 -- start start 2023-07-06T03:46:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd915fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91006b360 0x7fd910155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd915fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9100d2210 0x7fd9100cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd915fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91015c610 0x7fd910148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd915fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100cccc0 con 0x7fd91006b360 2023-07-06T03:46:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd915fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9101f6970 con 0x7fd9100d2210 2023-07-06T03:46:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd915fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd910148640 con 0x7fd91015c610 2023-07-06T03:46:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd90f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91006b360 0x7fd910155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd90f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91006b360 0x7fd910155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40880/0 (socket says 172.21.15.97:40880) 2023-07-06T03:46:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd90f7fe700 1 -- 172.21.15.97:0/2353182462 learned_addr learned my addr 172.21.15.97:0/2353182462 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd90f7fe700 1 -- 172.21.15.97:0/2353182462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91015c610 msgr2=0x7fd910148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd90f7fe700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91015c610 0x7fd910148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd90f7fe700 1 -- 172.21.15.97:0/2353182462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9100d2210 msgr2=0x7fd9100cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd90f7fe700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9100d2210 0x7fd9100cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.500+0000 7fd90f7fe700 1 -- 172.21.15.97:0/2353182462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd908009b30 con 0x7fd91006b360 2023-07-06T03:46:16.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.501+0000 7fd90f7fe700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91006b360 0x7fd910155b40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd90000e840 tx=0x7fd90000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:16.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.501+0000 7fd8f7fff700 1 -- 172.21.15.97:0/2353182462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90000c020 con 0x7fd91006b360 2023-07-06T03:46:16.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.501+0000 7fd8f7fff700 1 -- 172.21.15.97:0/2353182462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd900004250 con 0x7fd91006b360 2023-07-06T03:46:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.501+0000 7fd915fff700 1 -- 172.21.15.97:0/2353182462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd910156200 con 0x7fd91006b360 2023-07-06T03:46:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.501+0000 7fd8f7fff700 1 -- 172.21.15.97:0/2353182462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9000036a0 con 0x7fd91006b360 2023-07-06T03:46:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.501+0000 7fd915fff700 1 -- 172.21.15.97:0/2353182462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9101567c0 con 0x7fd91006b360 2023-07-06T03:46:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.502+0000 7fd915fff700 1 -- 172.21.15.97:0/2353182462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91004e340 con 0x7fd91006b360 2023-07-06T03:46:16.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.503+0000 7fd8f7fff700 1 -- 172.21.15.97:0/2353182462 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd900003800 con 0x7fd91006b360 2023-07-06T03:46:16.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.504+0000 7fd8f7fff700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8fc06ace0 0x7fd8fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.504+0000 7fd8f7fff700 1 -- 172.21.15.97:0/2353182462 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd900088f10 con 0x7fd91006b360 2023-07-06T03:46:16.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.504+0000 7fd90effd700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8fc06ace0 0x7fd8fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.505+0000 7fd90effd700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8fc06ace0 0x7fd8fc06d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd904006fd0 tx=0x7fd904008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:16.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.508+0000 7fd8f7fff700 1 -- 172.21.15.97:0/2353182462 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd900053fc0 con 0x7fd91006b360 2023-07-06T03:46:16.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.703+0000 7fd915fff700 1 -- 172.21.15.97:0/2353182462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9100c63c0 con 0x7fd91006b360 2023-07-06T03:46:16.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.703+0000 7fd8f7fff700 1 -- 172.21.15.97:0/2353182462 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9000575e0 con 0x7fd91006b360 2023-07-06T03:46:16.705 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:46:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 -- 172.21.15.97:0/2353182462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8fc06ace0 msgr2=0x7fd8fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8fc06ace0 0x7fd8fc06d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd904006fd0 tx=0x7fd904008040 comp rx=0 tx=0).stop 2023-07-06T03:46:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 -- 172.21.15.97:0/2353182462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91006b360 msgr2=0x7fd910155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91006b360 0x7fd910155b40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd90000e840 tx=0x7fd90000ec00 comp rx=0 tx=0).stop 2023-07-06T03:46:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 -- 172.21.15.97:0/2353182462 shutdown_connections 2023-07-06T03:46:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8fc06ace0 0x7fd8fc06d190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91015c610 0x7fd910148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91006b360 0x7fd910155b40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 --2- 172.21.15.97:0/2353182462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9100d2210 0x7fd9100cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 -- 172.21.15.97:0/2353182462 >> 172.21.15.97:0/2353182462 conn(0x7fd91006a6c0 msgr2=0x7fd9100b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 -- 172.21.15.97:0/2353182462 shutdown_connections 2023-07-06T03:46:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.706+0000 7fd915fff700 1 -- 172.21.15.97:0/2353182462 wait complete. 2023-07-06T03:46:16.726 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:46:16.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.922+0000 7fd04a5ff700 1 Processor -- start 2023-07-06T03:46:16.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.922+0000 7fd04a5ff700 1 -- start start 2023-07-06T03:46:16.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.924+0000 7fd04a5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415c1c0 0x7fd0440cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.924+0000 7fd04a5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04415da10 con 0x7fd04415c590 2023-07-06T03:46:16.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.924+0000 7fd04a5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0440ccc30 con 0x7fd0440719c0 2023-07-06T03:46:16.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.924+0000 7fd04a5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0440ccf10 con 0x7fd04415c1c0 2023-07-06T03:46:16.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.924+0000 7fd0437fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd04415c590 0x7fd04415d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60882/0 (socket says 172.21.15.97:60882) 2023-07-06T03:46:16.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.924+0000 7fd0437fe700 1 -- 172.21.15.97:0/3872568418 learned_addr learned my addr 172.21.15.97:0/3872568418 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:16.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.924+0000 7fd043fff700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415c1c0 0x7fd0440cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.924+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216140445 0 0) 0x7fd038001110 con 0x7fd04415c590 2023-07-06T03:46:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.924+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd02c0035b0 con 0x7fd04415c590 2023-07-06T03:46:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 313835840 0 0) 0x7fd02c0035b0 con 0x7fd04415c590 2023-07-06T03:46:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415c1c0 msgr2=0x7fd0440cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415c1c0 0x7fd0440cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 >> v1:172.21.15.146:6789/0 conn(0x7fd0440719c0 legacy=0x7fd04406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0440cd1f0 con 0x7fd04415c590 2023-07-06T03:46:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fd038001800 con 0x7fd04415c590 2023-07-06T03:46:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 >> v1:172.21.15.97:6789/0 conn(0x7fd04415c590 legacy=0x7fd04415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd02c003a30 0x7fd02c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415c1c0 0x7fd02c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd02c004d90 0x7fd02c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0440ccf10 con 0x7fd02c004d90 2023-07-06T03:46:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c00bac0 con 0x7fd02c003a30 2023-07-06T03:46:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c00bc00 con 0x7fd04415c1c0 2023-07-06T03:46:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd048b9c700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd02c004d90 0x7fd02c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd0437fe700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415c1c0 0x7fd02c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.925+0000 7fd043fff700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd02c003a30 0x7fd02c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.926+0000 7fd048b9c700 1 -- 172.21.15.97:0/3872568418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415c1c0 msgr2=0x7fd02c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.926+0000 7fd048b9c700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415c1c0 0x7fd02c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.926+0000 7fd048b9c700 1 -- 172.21.15.97:0/3872568418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd02c003a30 msgr2=0x7fd02c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.926+0000 7fd048b9c700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd02c003a30 0x7fd02c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.926+0000 7fd048b9c700 1 -- 172.21.15.97:0/3872568418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c0035b0 con 0x7fd02c004d90 2023-07-06T03:46:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.926+0000 7fd048b9c700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd02c004d90 0x7fd02c005200 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00c290 tx=0x7fd02c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf5ef30587ead88 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:16.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.926+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c001e70 con 0x7fd02c004d90 2023-07-06T03:46:16.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.926+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c00ce50 con 0x7fd02c004d90 2023-07-06T03:46:16.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.926+0000 7fd042ffd700 1 -- 172.21.15.97:0/3872568418 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c012660 con 0x7fd02c004d90 2023-07-06T03:46:16.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.927+0000 7fd04a5ff700 1 -- 172.21.15.97:0/3872568418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd02c004d90 msgr2=0x7fd02c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.927+0000 7fd04a5ff700 1 --2- 172.21.15.97:0/3872568418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd02c004d90 0x7fd02c005200 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00c290 tx=0x7fd02c005880 comp rx=0 tx=0).stop 2023-07-06T03:46:16.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.927+0000 7fd048b9c700 1 -- 172.21.15.97:0/3872568418 reap_dead start 2023-07-06T03:46:16.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.927+0000 7fd04a5ff700 1 -- 172.21.15.97:0/3872568418 shutdown_connections 2023-07-06T03:46:16.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.927+0000 7fd04a5ff700 1 -- 172.21.15.97:0/3872568418 >> 172.21.15.97:0/3872568418 conn(0x7fd0440b1650 msgr2=0x7fd0440b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:16.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.927+0000 7fd04a5ff700 1 -- 172.21.15.97:0/3872568418 shutdown_connections 2023-07-06T03:46:16.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.927+0000 7fd04a5ff700 1 -- 172.21.15.97:0/3872568418 wait complete. 2023-07-06T03:46:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.928+0000 7fd04a5ff700 1 Processor -- start 2023-07-06T03:46:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.929+0000 7fd04a5ff700 1 -- start start 2023-07-06T03:46:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.929+0000 7fd04a5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd02c004d90 0x7fd04406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.929+0000 7fd04a5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04406cd10 0x7fd044150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.929+0000 7fd04a5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415e3e0 0x7fd04415e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.929+0000 7fd04a5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04406e700 con 0x7fd04406cd10 2023-07-06T03:46:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.929+0000 7fd04a5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0441f5640 con 0x7fd02c004d90 2023-07-06T03:46:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.929+0000 7fd04a5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04406ebe0 con 0x7fd04415e3e0 2023-07-06T03:46:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd0437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04406cd10 0x7fd044150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd0437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04406cd10 0x7fd044150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40906/0 (socket says 172.21.15.97:40906) 2023-07-06T03:46:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd0437fe700 1 -- 172.21.15.97:0/2491848840 learned_addr learned my addr 172.21.15.97:0/2491848840 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd043fff700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd02c004d90 0x7fd04406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd048b9c700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415e3e0 0x7fd04415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd0437fe700 1 -- 172.21.15.97:0/2491848840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415e3e0 msgr2=0x7fd04415e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd0437fe700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415e3e0 0x7fd04415e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd0437fe700 1 -- 172.21.15.97:0/2491848840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd02c004d90 msgr2=0x7fd04406e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:16.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd0437fe700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd02c004d90 0x7fd04406e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:16.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd0437fe700 1 -- 172.21.15.97:0/2491848840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04406d1a0 con 0x7fd04406cd10 2023-07-06T03:46:16.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd043fff700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd02c004d90 0x7fd04406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:16.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.930+0000 7fd0437fe700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04406cd10 0x7fd044150550 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd038007290 tx=0x7fd038007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:16.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.931+0000 7fd040ff9700 1 -- 172.21.15.97:0/2491848840 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03800f660 con 0x7fd04406cd10 2023-07-06T03:46:16.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.931+0000 7fd040ff9700 1 -- 172.21.15.97:0/2491848840 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03800f7e0 con 0x7fd04406cd10 2023-07-06T03:46:16.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.931+0000 7fd04a5ff700 1 -- 172.21.15.97:0/2491848840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04406d480 con 0x7fd04406cd10 2023-07-06T03:46:16.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.931+0000 7fd040ff9700 1 -- 172.21.15.97:0/2491848840 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03800fb90 con 0x7fd04406cd10 2023-07-06T03:46:16.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.931+0000 7fd04a5ff700 1 -- 172.21.15.97:0/2491848840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04406da40 con 0x7fd04406cd10 2023-07-06T03:46:16.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.932+0000 7fd04a5ff700 1 -- 172.21.15.97:0/2491848840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04404e340 con 0x7fd04406cd10 2023-07-06T03:46:16.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.933+0000 7fd040ff9700 1 -- 172.21.15.97:0/2491848840 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd03800fcf0 con 0x7fd04406cd10 2023-07-06T03:46:16.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.934+0000 7fd040ff9700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd03006ace0 0x7fd03006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:16.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.934+0000 7fd040ff9700 1 -- 172.21.15.97:0/2491848840 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd038089360 con 0x7fd04406cd10 2023-07-06T03:46:16.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.934+0000 7fd043fff700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd03006ace0 0x7fd03006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:16.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.934+0000 7fd043fff700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd03006ace0 0x7fd03006d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd034009c60 tx=0x7fd034008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:16.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:16.938+0000 7fd040ff9700 1 -- 172.21.15.97:0/2491848840 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd038054370 con 0x7fd04406cd10 2023-07-06T03:46:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.133+0000 7fd04a5ff700 1 -- 172.21.15.97:0/2491848840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd04415da10 con 0x7fd04406cd10 2023-07-06T03:46:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.133+0000 7fd040ff9700 1 -- 172.21.15.97:0/2491848840 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd04415da10 con 0x7fd04406cd10 2023-07-06T03:46:17.135 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:46:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 -- 172.21.15.97:0/2491848840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd03006ace0 msgr2=0x7fd03006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd03006ace0 0x7fd03006d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd034009c60 tx=0x7fd034008040 comp rx=0 tx=0).stop 2023-07-06T03:46:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 -- 172.21.15.97:0/2491848840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04406cd10 msgr2=0x7fd044150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04406cd10 0x7fd044150550 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd038007290 tx=0x7fd038007650 comp rx=0 tx=0).stop 2023-07-06T03:46:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 -- 172.21.15.97:0/2491848840 shutdown_connections 2023-07-06T03:46:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd03006ace0 0x7fd03006d190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd04415e3e0 0x7fd04415e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd02c004d90 0x7fd04406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 --2- 172.21.15.97:0/2491848840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04406cd10 0x7fd044150550 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 -- 172.21.15.97:0/2491848840 >> 172.21.15.97:0/2491848840 conn(0x7fd0440b1650 msgr2=0x7fd0440cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 -- 172.21.15.97:0/2491848840 shutdown_connections 2023-07-06T03:46:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.136+0000 7fd04a5ff700 1 -- 172.21.15.97:0/2491848840 wait complete. 2023-07-06T03:46:17.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.351+0000 7f8b3c9ff700 1 Processor -- start 2023-07-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.351+0000 7f8b3c9ff700 1 -- start start 2023-07-06T03:46:17.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.352+0000 7f8b3c9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8b38154890 0x7f8b380cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.352+0000 7f8b3c9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3815b450 con 0x7f8b3815eb90 2023-07-06T03:46:17.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.352+0000 7f8b3c9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3815b6e0 con 0x7f8b3815b050 2023-07-06T03:46:17.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.352+0000 7f8b3c9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3815e0b0 con 0x7f8b38154890 2023-07-06T03:46:17.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b35d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8b3815eb90 0x7f8b3815a940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60886/0 (socket says 172.21.15.97:60886) 2023-07-06T03:46:17.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b35d9b700 1 -- 172.21.15.97:0/2655689532 learned_addr learned my addr 172.21.15.97:0/2655689532 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:17.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187248707 0 0) 0x7f8b3815b450 con 0x7f8b3815eb90 2023-07-06T03:46:17.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b200035b0 con 0x7f8b3815eb90 2023-07-06T03:46:17.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3659c700 1 --2- 172.21.15.97:0/2655689532 >> v2:172.21.15.146:3300/0 conn(0x7f8b38154890 0x7f8b380cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4106467520 0 0) 0x7f8b200035b0 con 0x7f8b3815eb90 2023-07-06T03:46:17.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 >> v2:172.21.15.146:3300/0 conn(0x7f8b38154890 msgr2=0x7f8b380cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:17.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 --2- 172.21.15.97:0/2655689532 >> v2:172.21.15.146:3300/0 conn(0x7f8b38154890 0x7f8b380cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 >> v1:172.21.15.146:6789/0 conn(0x7f8b3815b050 legacy=0x7f8b380c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3815e390 con 0x7f8b3815eb90 2023-07-06T03:46:17.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f8b2c001670 con 0x7f8b3815eb90 2023-07-06T03:46:17.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 >> v1:172.21.15.97:6789/0 conn(0x7f8b3815eb90 legacy=0x7f8b3815a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b20003a00 0x7f8b20003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 0x7f8b20004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.353+0000 7f8b3559a700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b20005200 0x7f8b2000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3815e0b0 con 0x7f8b20004850 2023-07-06T03:46:17.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2000c190 con 0x7f8b20005200 2023-07-06T03:46:17.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2000c350 con 0x7f8b20003a00 2023-07-06T03:46:17.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b35d9b700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 0x7f8b20004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b3659c700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b20003a00 0x7f8b20003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b36d9d700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b20005200 0x7f8b2000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b35d9b700 1 -- 172.21.15.97:0/2655689532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b20003a00 msgr2=0x7f8b20003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:17.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b35d9b700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b20003a00 0x7f8b20003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b35d9b700 1 -- 172.21.15.97:0/2655689532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b20005200 msgr2=0x7f8b2000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:17.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b35d9b700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b20005200 0x7f8b2000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b35d9b700 1 -- 172.21.15.97:0/2655689532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b200035b0 con 0x7f8b20004850 2023-07-06T03:46:17.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b36d9d700 1 -- 172.21.15.97:0/2655689532 reap_dead start 2023-07-06T03:46:17.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b3659c700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b20003a00 0x7f8b20003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:17.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b35d9b700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 0x7f8b20004cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c210 tx=0x7f8b20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97907b3ebed74439 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:17.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c007c10 con 0x7f8b20004850 2023-07-06T03:46:17.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.354+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c00cdd0 con 0x7f8b20004850 2023-07-06T03:46:17.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.355+0000 7f8b3559a700 1 -- 172.21.15.97:0/2655689532 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c005ba0 con 0x7f8b20004850 2023-07-06T03:46:17.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.355+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/2655689532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 msgr2=0x7f8b20004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.355+0000 7f8b3c9ff700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 0x7f8b20004cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c210 tx=0x7f8b20005880 comp rx=0 tx=0).stop 2023-07-06T03:46:17.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.355+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/2655689532 shutdown_connections 2023-07-06T03:46:17.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.355+0000 7f8b3c9ff700 1 --2- 172.21.15.97:0/2655689532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 0x7f8b20004cc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.355+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/2655689532 >> 172.21.15.97:0/2655689532 conn(0x7f8b3806a060 msgr2=0x7f8b3806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:17.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.355+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/2655689532 shutdown_connections 2023-07-06T03:46:17.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.355+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/2655689532 wait complete. 2023-07-06T03:46:17.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.357+0000 7f8b3c9ff700 1 Processor -- start 2023-07-06T03:46:17.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.357+0000 7f8b3c9ff700 1 -- start start 2023-07-06T03:46:17.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.358+0000 7f8b3c9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 0x7f8b38145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.358+0000 7f8b3c9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b380afd10 0x7f8b380b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.358+0000 7f8b3c9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b380b0260 0x7f8b380b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.358+0000 7f8b3c9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380b40d0 con 0x7f8b20004850 2023-07-06T03:46:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.358+0000 7f8b3c9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38071000 con 0x7f8b380afd10 2023-07-06T03:46:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.358+0000 7f8b3c9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380aef50 con 0x7f8b380b0260 2023-07-06T03:46:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.358+0000 7f8b3659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 0x7f8b38145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.358+0000 7f8b3659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 0x7f8b38145410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40916/0 (socket says 172.21.15.97:40916) 2023-07-06T03:46:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.358+0000 7f8b3659c700 1 -- 172.21.15.97:0/4173001120 learned_addr learned my addr 172.21.15.97:0/4173001120 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.358+0000 7f8b35d9b700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b380afd10 0x7f8b380b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b36d9d700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b380b0260 0x7f8b380b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b3659c700 1 -- 172.21.15.97:0/4173001120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b380b0260 msgr2=0x7f8b380b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b3659c700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b380b0260 0x7f8b380b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b3659c700 1 -- 172.21.15.97:0/4173001120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b380afd10 msgr2=0x7f8b380b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b3659c700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b380afd10 0x7f8b380b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b3659c700 1 -- 172.21.15.97:0/4173001120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38071250 con 0x7f8b20004850 2023-07-06T03:46:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b3659c700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 0x7f8b38145410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8b280108a0 tx=0x7f8b28010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b26ffd700 1 -- 172.21.15.97:0/4173001120 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800e860 con 0x7f8b20004850 2023-07-06T03:46:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b26ffd700 1 -- 172.21.15.97:0/4173001120 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2800c050 con 0x7f8b20004850 2023-07-06T03:46:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/4173001120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b380af1a0 con 0x7f8b20004850 2023-07-06T03:46:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b26ffd700 1 -- 172.21.15.97:0/4173001120 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800ec00 con 0x7f8b20004850 2023-07-06T03:46:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.359+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/4173001120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b380af760 con 0x7f8b20004850 2023-07-06T03:46:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.360+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/4173001120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b380af980 con 0x7f8b20004850 2023-07-06T03:46:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.361+0000 7f8b26ffd700 1 -- 172.21.15.97:0/4173001120 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8b2800ed60 con 0x7f8b20004850 2023-07-06T03:46:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.362+0000 7f8b26ffd700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1806ace0 0x7f8b1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.362+0000 7f8b26ffd700 1 -- 172.21.15.97:0/4173001120 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b28089b70 con 0x7f8b20004850 2023-07-06T03:46:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.362+0000 7f8b35d9b700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1806ace0 0x7f8b1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.363+0000 7f8b35d9b700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1806ace0 0x7f8b1806d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c001930 tx=0x7f8b2c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.366+0000 7f8b26ffd700 1 -- 172.21.15.97:0/4173001120 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8b28054cb0 con 0x7f8b20004850 2023-07-06T03:46:17.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.575+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/4173001120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8b3804e340 con 0x7f8b20004850 2023-07-06T03:46:17.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.576+0000 7f8b26ffd700 1 -- 172.21.15.97:0/4173001120 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8b280582d0 con 0x7f8b20004850 2023-07-06T03:46:17.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/4173001120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1806ace0 msgr2=0x7f8b1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1806ace0 0x7f8b1806d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c001930 tx=0x7f8b2c007000 comp rx=0 tx=0).stop 2023-07-06T03:46:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/4173001120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 msgr2=0x7f8b38145410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 0x7f8b38145410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8b280108a0 tx=0x7f8b28010c60 comp rx=0 tx=0).stop 2023-07-06T03:46:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/4173001120 shutdown_connections 2023-07-06T03:46:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1806ace0 0x7f8b1806d190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b380b0260 0x7f8b380b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b20004850 0x7f8b38145410 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 --2- 172.21.15.97:0/4173001120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b380afd10 0x7f8b380b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/4173001120 >> 172.21.15.97:0/4173001120 conn(0x7f8b3806a060 msgr2=0x7f8b380c4600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:17.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/4173001120 shutdown_connections 2023-07-06T03:46:17.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.578+0000 7f8b3c9ff700 1 -- 172.21.15.97:0/4173001120 wait complete. 2023-07-06T03:46:17.582 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:46:17.598 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:46:17.598 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:17.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:17.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.844+0000 7f9339dff700 1 Processor -- start 2023-07-06T03:46:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.844+0000 7f9339dff700 1 -- start start 2023-07-06T03:46:17.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.845+0000 7f9339dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f933415c520 0x7f93340cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.845+0000 7f9339dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9334160180 0x7f93341605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.845+0000 7f9339dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93340cdae0 con 0x7f9334160b30 2023-07-06T03:46:17.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.845+0000 7f9339dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93340cd8d0 con 0x7f933415c520 2023-07-06T03:46:17.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.845+0000 7f9339dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340b6640 con 0x7f9334160180 2023-07-06T03:46:17.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.845+0000 7f93337fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f933415c520 0x7f93340cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.845+0000 7f93337fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f933415c520 0x7f93340cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40928/0 (socket says 172.21.15.97:40928) 2023-07-06T03:46:17.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.845+0000 7f9333fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9334160b30 0x7f93340cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60892/0 (socket says 172.21.15.97:60892) 2023-07-06T03:46:17.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.845+0000 7f93337fe700 1 -- 172.21.15.97:0/1012966802 learned_addr learned my addr 172.21.15.97:0/1012966802 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:17.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.846+0000 7f9332ffd700 1 --2- 172.21.15.97:0/1012966802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9334160180 0x7f93341605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.846+0000 7f93337fe700 1 -- 172.21.15.97:0/1012966802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9334160180 msgr2=0x7f93341605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:17.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.846+0000 7f93337fe700 1 --2- 172.21.15.97:0/1012966802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9334160180 0x7f93341605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.846+0000 7f93337fe700 1 -- 172.21.15.97:0/1012966802 >> v1:172.21.15.97:6789/0 conn(0x7f9334160b30 legacy=0x7f93340cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.846+0000 7f93337fe700 1 -- 172.21.15.97:0/1012966802 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93340b6920 con 0x7f933415c520 2023-07-06T03:46:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.846+0000 7f93327fc700 1 -- 172.21.15.97:0/1012966802 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131385739 0 0) 0x7f93340cdae0 con 0x7f9334160b30 2023-07-06T03:46:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.846+0000 7f93337fe700 1 --2- 172.21.15.97:0/1012966802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f933415c520 0x7f93340cd390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f93240098f0 tx=0x7f9324009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacb22ff51a45daf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.846+0000 7f93327fc700 1 -- 172.21.15.97:0/1012966802 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932400b8a0 con 0x7f933415c520 2023-07-06T03:46:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.846+0000 7f93327fc700 1 -- 172.21.15.97:0/1012966802 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932400ba00 con 0x7f933415c520 2023-07-06T03:46:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.846+0000 7f93327fc700 1 -- 172.21.15.97:0/1012966802 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932400bd90 con 0x7f933415c520 2023-07-06T03:46:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.847+0000 7f9339dff700 1 -- 172.21.15.97:0/1012966802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f933415c520 msgr2=0x7f93340cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.847+0000 7f9339dff700 1 --2- 172.21.15.97:0/1012966802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f933415c520 0x7f93340cd390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f93240098f0 tx=0x7f9324009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:17.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.847+0000 7f9339dff700 1 -- 172.21.15.97:0/1012966802 shutdown_connections 2023-07-06T03:46:17.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.847+0000 7f9339dff700 1 --2- 172.21.15.97:0/1012966802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f933415c520 0x7f93340cd390 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.847+0000 7f9339dff700 1 --2- 172.21.15.97:0/1012966802 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9334160180 0x7f93341605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.847+0000 7f9339dff700 1 -- 172.21.15.97:0/1012966802 >> 172.21.15.97:0/1012966802 conn(0x7f9334071790 msgr2=0x7f9334071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:17.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.847+0000 7f9339dff700 1 -- 172.21.15.97:0/1012966802 shutdown_connections 2023-07-06T03:46:17.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.847+0000 7f9339dff700 1 -- 172.21.15.97:0/1012966802 wait complete. 2023-07-06T03:46:17.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.849+0000 7f9339dff700 1 Processor -- start 2023-07-06T03:46:17.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.849+0000 7f9339dff700 1 -- start start 2023-07-06T03:46:17.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9339dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933415c520 0x7f933414d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9339dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9334160180 0x7f93340cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9339dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9334160b30 0x7f93340cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9339dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340ccd40 con 0x7f9334160180 2023-07-06T03:46:17.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9339dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340cbca0 con 0x7f9334160b30 2023-07-06T03:46:17.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9339dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340cbf80 con 0x7f933415c520 2023-07-06T03:46:17.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9332ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9334160180 0x7f93340cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9332ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9334160180 0x7f93340cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40944/0 (socket says 172.21.15.97:40944) 2023-07-06T03:46:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9332ffd700 1 -- 172.21.15.97:0/43069545 learned_addr learned my addr 172.21.15.97:0/43069545 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f93337fe700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933415c520 0x7f933414d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9333fff700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9334160b30 0x7f93340cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9332ffd700 1 -- 172.21.15.97:0/43069545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933415c520 msgr2=0x7f933414d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9332ffd700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933415c520 0x7f933414d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9332ffd700 1 -- 172.21.15.97:0/43069545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9334160b30 msgr2=0x7f93340cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9332ffd700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9334160b30 0x7f93340cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:17.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9332ffd700 1 -- 172.21.15.97:0/43069545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9324009550 con 0x7f9334160180 2023-07-06T03:46:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f93337fe700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933415c520 0x7f933414d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.850+0000 7f9333fff700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9334160b30 0x7f93340cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.851+0000 7f9332ffd700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9334160180 0x7f93340cf560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f932800d800 tx=0x7f932800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.851+0000 7f931bfff700 1 -- 172.21.15.97:0/43069545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93280125f0 con 0x7f9334160180 2023-07-06T03:46:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.851+0000 7f931bfff700 1 -- 172.21.15.97:0/43069545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9328012750 con 0x7f9334160180 2023-07-06T03:46:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.851+0000 7f9339dff700 1 -- 172.21.15.97:0/43069545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93340cc260 con 0x7f9334160180 2023-07-06T03:46:17.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.851+0000 7f931bfff700 1 -- 172.21.15.97:0/43069545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328012b00 con 0x7f9334160180 2023-07-06T03:46:17.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.851+0000 7f9339dff700 1 -- 172.21.15.97:0/43069545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93340cc820 con 0x7f9334160180 2023-07-06T03:46:17.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.852+0000 7f9339dff700 1 -- 172.21.15.97:0/43069545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933404e340 con 0x7f9334160180 2023-07-06T03:46:17.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.853+0000 7f931bfff700 1 -- 172.21.15.97:0/43069545 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9328012c60 con 0x7f9334160180 2023-07-06T03:46:17.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.854+0000 7f931bfff700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f932006ace0 0x7f932006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:17.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.854+0000 7f931bfff700 1 -- 172.21.15.97:0/43069545 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9328089900 con 0x7f9334160180 2023-07-06T03:46:17.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.855+0000 7f93337fe700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f932006ace0 0x7f932006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:17.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.855+0000 7f93337fe700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f932006ace0 0x7f932006d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9324014040 tx=0x7f9324008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:17.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:17.859+0000 7f931bfff700 1 -- 172.21.15.97:0/43069545 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9328054910 con 0x7f9334160180 2023-07-06T03:46:18.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.055+0000 7f9339dff700 1 -- 172.21.15.97:0/43069545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93340cd100 con 0x7f9334160180 2023-07-06T03:46:18.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.056+0000 7f931bfff700 1 -- 172.21.15.97:0/43069545 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f9328057f30 con 0x7f9334160180 2023-07-06T03:46:18.058 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:18.058 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.058+0000 7f9339dff700 1 -- 172.21.15.97:0/43069545 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f932006ace0 msgr2=0x7f932006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.058+0000 7f9339dff700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f932006ace0 0x7f932006d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9324014040 tx=0x7f9324008030 comp rx=0 tx=0).stop 2023-07-06T03:46:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.058+0000 7f9339dff700 1 -- 172.21.15.97:0/43069545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9334160180 msgr2=0x7f93340cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.058+0000 7f9339dff700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9334160180 0x7f93340cf560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f932800d800 tx=0x7f932800dbc0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.058+0000 7f9339dff700 1 -- 172.21.15.97:0/43069545 shutdown_connections 2023-07-06T03:46:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.058+0000 7f9339dff700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f932006ace0 0x7f932006d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.058+0000 7f9339dff700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9334160b30 0x7f93340cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.058+0000 7f9339dff700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f933415c520 0x7f933414d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.058+0000 7f9339dff700 1 --2- 172.21.15.97:0/43069545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9334160180 0x7f93340cf560 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.058+0000 7f9339dff700 1 -- 172.21.15.97:0/43069545 >> 172.21.15.97:0/43069545 conn(0x7f9334071790 msgr2=0x7f9334072150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.059+0000 7f9339dff700 1 -- 172.21.15.97:0/43069545 shutdown_connections 2023-07-06T03:46:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.059+0000 7f9339dff700 1 -- 172.21.15.97:0/43069545 wait complete. 2023-07-06T03:46:18.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:46:18.080 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:46:18.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.276+0000 7fae027ff700 1 Processor -- start 2023-07-06T03:46:18.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.276+0000 7fae027ff700 1 -- start start 2023-07-06T03:46:18.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.277+0000 7fae027ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc0d2210 0x7fadfc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.277+0000 7fae027ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc0cff90 con 0x7fadfc15c610 2023-07-06T03:46:18.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.277+0000 7fae027ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc0b5650 con 0x7fadfc06b360 2023-07-06T03:46:18.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.277+0000 7fae027ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0d0780 con 0x7fadfc0d2210 2023-07-06T03:46:18.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.278+0000 7fadfbfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fadfc15c610 0x7fadfc0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60898/0 (socket says 172.21.15.97:60898) 2023-07-06T03:46:18.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.278+0000 7fadfbfff700 1 -- 172.21.15.97:0/1980664428 learned_addr learned my addr 172.21.15.97:0/1980664428 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:18.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.278+0000 7fae00d9c700 1 --2- 172.21.15.97:0/1980664428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc0d2210 0x7fadfc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.278+0000 7fadfb7fe700 1 -- 172.21.15.97:0/1980664428 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868568525 0 0) 0x7fadfc0cff90 con 0x7fadfc15c610 2023-07-06T03:46:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.278+0000 7fadfb7fe700 1 -- 172.21.15.97:0/1980664428 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade80035b0 con 0x7fadfc15c610 2023-07-06T03:46:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.278+0000 7fadfb7fe700 1 -- 172.21.15.97:0/1980664428 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625222458 0 0) 0x7fadfc0b5650 con 0x7fadfc06b360 2023-07-06T03:46:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.278+0000 7fadfb7fe700 1 -- 172.21.15.97:0/1980664428 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadfc0cff90 con 0x7fadfc06b360 2023-07-06T03:46:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.278+0000 7fadfb7fe700 1 -- 172.21.15.97:0/1980664428 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 347543699 0 0) 0x7fade80035b0 con 0x7fadfc15c610 2023-07-06T03:46:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.278+0000 7fae00d9c700 1 -- 172.21.15.97:0/1980664428 >> v1:172.21.15.146:6789/0 conn(0x7fadfc06b360 legacy=0x7fadfc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.278+0000 7fae00d9c700 1 -- 172.21.15.97:0/1980664428 >> v1:172.21.15.97:6789/0 conn(0x7fadfc15c610 legacy=0x7fadfc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.279+0000 7fae00d9c700 1 -- 172.21.15.97:0/1980664428 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc0b5400 con 0x7fadfc0d2210 2023-07-06T03:46:18.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.279+0000 7fae00d9c700 1 --2- 172.21.15.97:0/1980664428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc0d2210 0x7fadfc0b4ec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009870 tx=0x7fadf4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b89ffdf73d37da9d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:18.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.279+0000 7fadfb7fe700 1 -- 172.21.15.97:0/1980664428 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec0019f0 con 0x7fadfc0d2210 2023-07-06T03:46:18.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.279+0000 7fadfb7fe700 1 -- 172.21.15.97:0/1980664428 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf4004030 con 0x7fadfc0d2210 2023-07-06T03:46:18.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.279+0000 7fadfb7fe700 1 -- 172.21.15.97:0/1980664428 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf40043c0 con 0x7fadfc0d2210 2023-07-06T03:46:18.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.280+0000 7fae027ff700 1 -- 172.21.15.97:0/1980664428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc0d2210 msgr2=0x7fadfc0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.280+0000 7fae027ff700 1 --2- 172.21.15.97:0/1980664428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc0d2210 0x7fadfc0b4ec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009870 tx=0x7fadf4009b80 comp rx=0 tx=0).stop 2023-07-06T03:46:18.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.280+0000 7fae027ff700 1 -- 172.21.15.97:0/1980664428 shutdown_connections 2023-07-06T03:46:18.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.280+0000 7fae027ff700 1 --2- 172.21.15.97:0/1980664428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc0d2210 0x7fadfc0b4ec0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.280+0000 7fae027ff700 1 -- 172.21.15.97:0/1980664428 >> 172.21.15.97:0/1980664428 conn(0x7fadfc06a6c0 msgr2=0x7fadfc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:18.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.280+0000 7fae027ff700 1 -- 172.21.15.97:0/1980664428 shutdown_connections 2023-07-06T03:46:18.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.280+0000 7fae027ff700 1 -- 172.21.15.97:0/1980664428 wait complete. 2023-07-06T03:46:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.281+0000 7fae027ff700 1 Processor -- start 2023-07-06T03:46:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.282+0000 7fae027ff700 1 -- start start 2023-07-06T03:46:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fae027ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadfc06b360 0x7fadfc1f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fae027ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc0d2210 0x7fadfc0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fae027ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc15c610 0x7fadfc0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fae027ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0cb8f0 con 0x7fadfc06b360 2023-07-06T03:46:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fae027ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1f5340 con 0x7fadfc0d2210 2023-07-06T03:46:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fae027ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0ca850 con 0x7fadfc15c610 2023-07-06T03:46:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fadfbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadfc06b360 0x7fadfc1f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fadfbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadfc06b360 0x7fadfc1f4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40948/0 (socket says 172.21.15.97:40948) 2023-07-06T03:46:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fadfbfff700 1 -- 172.21.15.97:0/3901427372 learned_addr learned my addr 172.21.15.97:0/3901427372 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fadf37fe700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc0d2210 0x7fadfc0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fae00d9c700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc15c610 0x7fadfc0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fadfbfff700 1 -- 172.21.15.97:0/3901427372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc15c610 msgr2=0x7fadfc0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fadfbfff700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc15c610 0x7fadfc0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fadfbfff700 1 -- 172.21.15.97:0/3901427372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc0d2210 msgr2=0x7fadfc0c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fadfbfff700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc0d2210 0x7fadfc0c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fadfbfff700 1 -- 172.21.15.97:0/3901427372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf40094a0 con 0x7fadfc06b360 2023-07-06T03:46:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fae00d9c700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc15c610 0x7fadfc0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.283+0000 7fadf37fe700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc0d2210 0x7fadfc0c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.284+0000 7fadfbfff700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadfc06b360 0x7fadfc1f4e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fadec001f40 tx=0x7fadec00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.284+0000 7fadf97fa700 1 -- 172.21.15.97:0/3901427372 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec00ade0 con 0x7fadfc06b360 2023-07-06T03:46:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.284+0000 7fadf97fa700 1 -- 172.21.15.97:0/3901427372 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec0070d0 con 0x7fadfc06b360 2023-07-06T03:46:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.284+0000 7fadf97fa700 1 -- 172.21.15.97:0/3901427372 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec007480 con 0x7fadfc06b360 2023-07-06T03:46:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.284+0000 7fae027ff700 1 -- 172.21.15.97:0/3901427372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc0cab30 con 0x7fadfc06b360 2023-07-06T03:46:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.284+0000 7fae027ff700 1 -- 172.21.15.97:0/3901427372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc0cb0f0 con 0x7fadfc06b360 2023-07-06T03:46:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.285+0000 7fae027ff700 1 -- 172.21.15.97:0/3901427372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc04e340 con 0x7fadfc06b360 2023-07-06T03:46:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.286+0000 7fadf97fa700 1 -- 172.21.15.97:0/3901427372 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fadec0075e0 con 0x7fadfc06b360 2023-07-06T03:46:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.287+0000 7fadf97fa700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faddc06ad10 0x7faddc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.287+0000 7fadf97fa700 1 -- 172.21.15.97:0/3901427372 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadec0891e0 con 0x7fadfc06b360 2023-07-06T03:46:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.287+0000 7fadf37fe700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faddc06ad10 0x7faddc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.288+0000 7fadf37fe700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faddc06ad10 0x7faddc06d1c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fadfc158050 tx=0x7fade4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.291+0000 7fadf97fa700 1 -- 172.21.15.97:0/3901427372 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fadec054200 con 0x7fadfc06b360 2023-07-06T03:46:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.484+0000 7fae027ff700 1 -- 172.21.15.97:0/3901427372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fadfc066010 con 0x7fadfc06b360 2023-07-06T03:46:18.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.485+0000 7fadf97fa700 1 -- 172.21.15.97:0/3901427372 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadec0180a0 con 0x7fadfc06b360 2023-07-06T03:46:18.487 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:46:18.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 -- 172.21.15.97:0/3901427372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faddc06ad10 msgr2=0x7faddc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faddc06ad10 0x7faddc06d1c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fadfc158050 tx=0x7fade4009040 comp rx=0 tx=0).stop 2023-07-06T03:46:18.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 -- 172.21.15.97:0/3901427372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadfc06b360 msgr2=0x7fadfc1f4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadfc06b360 0x7fadfc1f4e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fadec001f40 tx=0x7fadec00a340 comp rx=0 tx=0).stop 2023-07-06T03:46:18.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 -- 172.21.15.97:0/3901427372 shutdown_connections 2023-07-06T03:46:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faddc06ad10 0x7faddc06d1c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadfc15c610 0x7fadfc0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadfc06b360 0x7fadfc1f4e00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 --2- 172.21.15.97:0/3901427372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadfc0d2210 0x7fadfc0c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 -- 172.21.15.97:0/3901427372 >> 172.21.15.97:0/3901427372 conn(0x7fadfc06a6c0 msgr2=0x7fadfc0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 -- 172.21.15.97:0/3901427372 shutdown_connections 2023-07-06T03:46:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.487+0000 7fae027ff700 1 -- 172.21.15.97:0/3901427372 wait complete. 2023-07-06T03:46:18.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:46:18.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.703+0000 7f93965ff700 1 Processor -- start 2023-07-06T03:46:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.704+0000 7f93965ff700 1 -- start start 2023-07-06T03:46:18.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.705+0000 7f93965ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939006b360 0x7f93900b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.705+0000 7f93965ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f93900d2210 0x7f93900d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.705+0000 7f93965ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93900cff90 con 0x7f939015c610 2023-07-06T03:46:18.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.705+0000 7f93965ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93900d0780 con 0x7f93900d2210 2023-07-06T03:46:18.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.705+0000 7f93965ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939015c9e0 con 0x7f939006b360 2023-07-06T03:46:18.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.705+0000 7f938ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f939015c610 0x7f93900b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60904/0 (socket says 172.21.15.97:60904) 2023-07-06T03:46:18.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.705+0000 7f938ffff700 1 -- 172.21.15.97:0/2625775663 learned_addr learned my addr 172.21.15.97:0/2625775663 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:18.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.705+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723051972 0 0) 0x7f93900cff90 con 0x7f939015c610 2023-07-06T03:46:18.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f937c0035b0 con 0x7f939015c610 2023-07-06T03:46:18.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f9394b9c700 1 --2- 172.21.15.97:0/2625775663 >> v2:172.21.15.146:3300/0 conn(0x7f93900d2210 0x7f93900d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f9387fff700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939006b360 0x7f93900b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 651934633 0 0) 0x7f937c0035b0 con 0x7f939015c610 2023-07-06T03:46:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939006b360 msgr2=0x7f93900b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939006b360 0x7f93900b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 >> v2:172.21.15.146:3300/0 conn(0x7f93900d2210 msgr2=0x7f93900d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 --2- 172.21.15.97:0/2625775663 >> v2:172.21.15.146:3300/0 conn(0x7f93900d2210 0x7f93900d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93900b5650 con 0x7f939015c610 2023-07-06T03:46:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f9380001670 con 0x7f939015c610 2023-07-06T03:46:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 >> v1:172.21.15.97:6789/0 conn(0x7f939015c610 legacy=0x7f93900b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939006b360 0x7f937c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f937c004000 0x7f937c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f937c004d90 0x7f937c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93900d0780 con 0x7f937c004000 2023-07-06T03:46:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939015c9e0 con 0x7f937c004d90 2023-07-06T03:46:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937c00ba10 con 0x7f939006b360 2023-07-06T03:46:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f9387fff700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f937c004000 0x7f937c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f938ffff700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939006b360 0x7f937c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.706+0000 7f9394b9c700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f937c004d90 0x7f937c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.707+0000 7f9387fff700 1 -- 172.21.15.97:0/2625775663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939006b360 msgr2=0x7f937c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.707+0000 7f9387fff700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f939006b360 0x7f937c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.707+0000 7f9387fff700 1 -- 172.21.15.97:0/2625775663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f937c004d90 msgr2=0x7f937c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.707+0000 7f9387fff700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f937c004d90 0x7f937c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.707+0000 7f9387fff700 1 -- 172.21.15.97:0/2625775663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c0035b0 con 0x7f937c004000 2023-07-06T03:46:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.707+0000 7f9387fff700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f937c004000 0x7f937c004850 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9378009b30 tx=0x7f937c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a08230fd04bbf64 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.707+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937800e6c0 con 0x7f937c004000 2023-07-06T03:46:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.707+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937800e820 con 0x7f937c004000 2023-07-06T03:46:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.707+0000 7f938f7fe700 1 -- 172.21.15.97:0/2625775663 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937800eb90 con 0x7f937c004000 2023-07-06T03:46:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.708+0000 7f93965ff700 1 -- 172.21.15.97:0/2625775663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f937c004000 msgr2=0x7f937c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.708+0000 7f93965ff700 1 --2- 172.21.15.97:0/2625775663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f937c004000 0x7f937c004850 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9378009b30 tx=0x7f937c005880 comp rx=0 tx=0).stop 2023-07-06T03:46:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.708+0000 7f9394b9c700 1 -- 172.21.15.97:0/2625775663 reap_dead start 2023-07-06T03:46:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.708+0000 7f93965ff700 1 -- 172.21.15.97:0/2625775663 shutdown_connections 2023-07-06T03:46:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.708+0000 7f93965ff700 1 -- 172.21.15.97:0/2625775663 >> 172.21.15.97:0/2625775663 conn(0x7f939006a6c0 msgr2=0x7f939006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.708+0000 7f93965ff700 1 -- 172.21.15.97:0/2625775663 shutdown_connections 2023-07-06T03:46:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.708+0000 7f93965ff700 1 -- 172.21.15.97:0/2625775663 wait complete. 2023-07-06T03:46:18.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.710+0000 7f93965ff700 1 Processor -- start 2023-07-06T03:46:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.710+0000 7f93965ff700 1 -- start start 2023-07-06T03:46:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f93965ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f937c004000 0x7f93901f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f93965ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93901f4670 0x7f939015a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f93965ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93901f4ae0 0x7f939015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f93965ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93900cafa0 con 0x7f93901f4670 2023-07-06T03:46:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f93965ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93901f1ee0 con 0x7f937c004000 2023-07-06T03:46:18.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f93965ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939015dd10 con 0x7f93901f4ae0 2023-07-06T03:46:18.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f9387fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93901f4670 0x7f939015a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f9387fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93901f4670 0x7f939015a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40970/0 (socket says 172.21.15.97:40970) 2023-07-06T03:46:18.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f9387fff700 1 -- 172.21.15.97:0/1848396518 learned_addr learned my addr 172.21.15.97:0/1848396518 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:18.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f9394b9c700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93901f4ae0 0x7f939015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f938ffff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f937c004000 0x7f93901f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f9387fff700 1 -- 172.21.15.97:0/1848396518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93901f4ae0 msgr2=0x7f939015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f9387fff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93901f4ae0 0x7f939015ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f9387fff700 1 -- 172.21.15.97:0/1848396518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f937c004000 msgr2=0x7f93901f19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f9387fff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f937c004000 0x7f93901f19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.712+0000 7f9387fff700 1 -- 172.21.15.97:0/1848396518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939015dff0 con 0x7f93901f4670 2023-07-06T03:46:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.713+0000 7f9387fff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93901f4670 0x7f939015a620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f937800e4b0 tx=0x7f93780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.713+0000 7f938d7fa700 1 -- 172.21.15.97:0/1848396518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937800e6c0 con 0x7f93901f4670 2023-07-06T03:46:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.713+0000 7f938d7fa700 1 -- 172.21.15.97:0/1848396518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9378003ab0 con 0x7f93901f4670 2023-07-06T03:46:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.713+0000 7f93965ff700 1 -- 172.21.15.97:0/1848396518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93900cb1f0 con 0x7f93901f4670 2023-07-06T03:46:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.713+0000 7f938d7fa700 1 -- 172.21.15.97:0/1848396518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378003e60 con 0x7f93901f4670 2023-07-06T03:46:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.713+0000 7f93965ff700 1 -- 172.21.15.97:0/1848396518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939015d020 con 0x7f93901f4670 2023-07-06T03:46:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.714+0000 7f93965ff700 1 -- 172.21.15.97:0/1848396518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93900cb840 con 0x7f93901f4670 2023-07-06T03:46:18.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.715+0000 7f938d7fa700 1 -- 172.21.15.97:0/1848396518 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9378013470 con 0x7f93901f4670 2023-07-06T03:46:18.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.716+0000 7f938d7fa700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f937006ace0 0x7f937006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:18.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.716+0000 7f938d7fa700 1 -- 172.21.15.97:0/1848396518 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f937808ad90 con 0x7f93901f4670 2023-07-06T03:46:18.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.716+0000 7f938ffff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f937006ace0 0x7f937006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:18.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.717+0000 7f938ffff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f937006ace0 0x7f937006d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9380001050 tx=0x7f9380009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:18.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.720+0000 7f938d7fa700 1 -- 172.21.15.97:0/1848396518 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9378055f50 con 0x7f93901f4670 2023-07-06T03:46:18.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.920+0000 7f93965ff700 1 -- 172.21.15.97:0/1848396518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f939004e340 con 0x7f93901f4670 2023-07-06T03:46:18.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.920+0000 7f938d7fa700 1 -- 172.21.15.97:0/1848396518 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9378059570 con 0x7f93901f4670 2023-07-06T03:46:18.923 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:46:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 -- 172.21.15.97:0/1848396518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f937006ace0 msgr2=0x7f937006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f937006ace0 0x7f937006d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9380001050 tx=0x7f9380009040 comp rx=0 tx=0).stop 2023-07-06T03:46:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 -- 172.21.15.97:0/1848396518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93901f4670 msgr2=0x7f939015a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93901f4670 0x7f939015a620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f937800e4b0 tx=0x7f93780047c0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 -- 172.21.15.97:0/1848396518 shutdown_connections 2023-07-06T03:46:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f937006ace0 0x7f937006d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93901f4ae0 0x7f939015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f937c004000 0x7f93901f19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 --2- 172.21.15.97:0/1848396518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93901f4670 0x7f939015a620 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 -- 172.21.15.97:0/1848396518 >> 172.21.15.97:0/1848396518 conn(0x7f939006a6c0 msgr2=0x7f93900d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:18.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 -- 172.21.15.97:0/1848396518 shutdown_connections 2023-07-06T03:46:18.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:18.923+0000 7f93965ff700 1 -- 172.21.15.97:0/1848396518 wait complete. 2023-07-06T03:46:18.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:46:19.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.139+0000 7fceab7ff700 1 Processor -- start 2023-07-06T03:46:19.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.139+0000 7fceab7ff700 1 -- start start 2023-07-06T03:46:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fceab7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcea415c610 0x7fcea40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fceab7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcea406b360 0x7fcea40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fceab7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcea40d2210 0x7fcea40d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fceab7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40d0780 con 0x7fcea415c610 2023-07-06T03:46:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fceab7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea415c9e0 con 0x7fcea40d2210 2023-07-06T03:46:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fceab7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40b5650 con 0x7fcea406b360 2023-07-06T03:46:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fcea959b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcea415c610 0x7fcea40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fcea959b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcea415c610 0x7fcea40b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40980/0 (socket says 172.21.15.97:40980) 2023-07-06T03:46:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fcea959b700 1 -- 172.21.15.97:0/356155411 learned_addr learned my addr 172.21.15.97:0/356155411 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fcea8d9a700 1 --2- 172.21.15.97:0/356155411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcea406b360 0x7fcea40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fcea9d9c700 1 --2- 172.21.15.97:0/356155411 >> v2:172.21.15.146:3300/0 conn(0x7fcea40d2210 0x7fcea40d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fcea959b700 1 -- 172.21.15.97:0/356155411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcea406b360 msgr2=0x7fcea40b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fcea959b700 1 --2- 172.21.15.97:0/356155411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcea406b360 0x7fcea40b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fcea959b700 1 -- 172.21.15.97:0/356155411 >> v2:172.21.15.146:3300/0 conn(0x7fcea40d2210 msgr2=0x7fcea40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.140+0000 7fcea959b700 1 --2- 172.21.15.97:0/356155411 >> v2:172.21.15.146:3300/0 conn(0x7fcea40d2210 0x7fcea40d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.141+0000 7fcea959b700 1 -- 172.21.15.97:0/356155411 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea40d01c0 con 0x7fcea415c610 2023-07-06T03:46:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.141+0000 7fcea9d9c700 1 --2- 172.21.15.97:0/356155411 >> v2:172.21.15.146:3300/0 conn(0x7fcea40d2210 0x7fcea40d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.141+0000 7fcea8d9a700 1 --2- 172.21.15.97:0/356155411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcea406b360 0x7fcea40b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.141+0000 7fcea959b700 1 --2- 172.21.15.97:0/356155411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea415c610 0x7fcea40b47b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcea40b1b70 tx=0x7fcea0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72349765ecaeb2e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.141+0000 7fce9bfff700 1 -- 172.21.15.97:0/356155411 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea000b860 con 0x7fcea415c610 2023-07-06T03:46:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.141+0000 7fce9bfff700 1 -- 172.21.15.97:0/356155411 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea000b9c0 con 0x7fcea415c610 2023-07-06T03:46:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.141+0000 7fce9bfff700 1 -- 172.21.15.97:0/356155411 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea000bd50 con 0x7fcea415c610 2023-07-06T03:46:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.141+0000 7fceab7ff700 1 -- 172.21.15.97:0/356155411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea415c610 msgr2=0x7fcea40b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.141+0000 7fceab7ff700 1 --2- 172.21.15.97:0/356155411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea415c610 0x7fcea40b47b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcea40b1b70 tx=0x7fcea0009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.142+0000 7fceab7ff700 1 -- 172.21.15.97:0/356155411 shutdown_connections 2023-07-06T03:46:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.142+0000 7fceab7ff700 1 --2- 172.21.15.97:0/356155411 >> v2:172.21.15.146:3300/0 conn(0x7fcea40d2210 0x7fcea40d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.142+0000 7fceab7ff700 1 --2- 172.21.15.97:0/356155411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea415c610 0x7fcea40b47b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.142+0000 7fceab7ff700 1 --2- 172.21.15.97:0/356155411 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcea406b360 0x7fcea40b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.142+0000 7fceab7ff700 1 -- 172.21.15.97:0/356155411 >> 172.21.15.97:0/356155411 conn(0x7fcea406a6c0 msgr2=0x7fcea406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.142+0000 7fceab7ff700 1 -- 172.21.15.97:0/356155411 shutdown_connections 2023-07-06T03:46:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.142+0000 7fceab7ff700 1 -- 172.21.15.97:0/356155411 wait complete. 2023-07-06T03:46:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.144+0000 7fceab7ff700 1 Processor -- start 2023-07-06T03:46:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.144+0000 7fceab7ff700 1 -- start start 2023-07-06T03:46:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fceab7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea406b360 0x7fcea41f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fceab7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcea40d2210 0x7fcea40b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fceab7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcea415c610 0x7fcea40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fceab7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40ca1a0 con 0x7fcea406b360 2023-07-06T03:46:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fceab7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40ca480 con 0x7fcea40d2210 2023-07-06T03:46:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fceab7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40ca760 con 0x7fcea415c610 2023-07-06T03:46:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fcea959b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea406b360 0x7fcea41f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fcea959b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea406b360 0x7fcea41f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40994/0 (socket says 172.21.15.97:40994) 2023-07-06T03:46:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fcea959b700 1 -- 172.21.15.97:0/1221808333 learned_addr learned my addr 172.21.15.97:0/1221808333 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fcea8d9a700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcea40d2210 0x7fcea40b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fcea9d9c700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcea415c610 0x7fcea40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fcea959b700 1 -- 172.21.15.97:0/1221808333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcea415c610 msgr2=0x7fcea40b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fcea959b700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcea415c610 0x7fcea40b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fcea959b700 1 -- 172.21.15.97:0/1221808333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcea40d2210 msgr2=0x7fcea40b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fcea959b700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcea40d2210 0x7fcea40b73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.145+0000 7fcea959b700 1 -- 172.21.15.97:0/1221808333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea0009550 con 0x7fcea406b360 2023-07-06T03:46:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.146+0000 7fcea959b700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea406b360 0x7fcea41f4b70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcea0014040 tx=0x7fcea0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.146+0000 7fce99ffb700 1 -- 172.21.15.97:0/1221808333 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea00036a0 con 0x7fcea406b360 2023-07-06T03:46:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.146+0000 7fce99ffb700 1 -- 172.21.15.97:0/1221808333 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea001e070 con 0x7fcea406b360 2023-07-06T03:46:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.146+0000 7fceab7ff700 1 -- 172.21.15.97:0/1221808333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea40caa40 con 0x7fcea406b360 2023-07-06T03:46:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.146+0000 7fce99ffb700 1 -- 172.21.15.97:0/1221808333 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0003a60 con 0x7fcea406b360 2023-07-06T03:46:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.146+0000 7fceab7ff700 1 -- 172.21.15.97:0/1221808333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea40c9540 con 0x7fcea406b360 2023-07-06T03:46:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.147+0000 7fce8f7fe700 1 -- 172.21.15.97:0/1221808333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea404e340 con 0x7fcea406b360 2023-07-06T03:46:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.148+0000 7fce99ffb700 1 -- 172.21.15.97:0/1221808333 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcea0003bc0 con 0x7fcea406b360 2023-07-06T03:46:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.149+0000 7fce99ffb700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fce8806ace0 0x7fce8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.149+0000 7fce99ffb700 1 -- 172.21.15.97:0/1221808333 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcea0020070 con 0x7fcea406b360 2023-07-06T03:46:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.149+0000 7fcea8d9a700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fce8806ace0 0x7fce8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.150+0000 7fcea8d9a700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fce8806ace0 0x7fce8806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce94006ba0 tx=0x7fce94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.153+0000 7fce99ffb700 1 -- 172.21.15.97:0/1221808333 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcea0055a80 con 0x7fcea406b360 2023-07-06T03:46:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.347+0000 7fce8f7fe700 1 -- 172.21.15.97:0/1221808333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcea40b6770 con 0x7fcea406b360 2023-07-06T03:46:19.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.348+0000 7fce99ffb700 1 -- 172.21.15.97:0/1221808333 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcea00590a0 con 0x7fcea406b360 2023-07-06T03:46:19.350 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:46:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 -- 172.21.15.97:0/1221808333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fce8806ace0 msgr2=0x7fce8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fce8806ace0 0x7fce8806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce94006ba0 tx=0x7fce94008040 comp rx=0 tx=0).stop 2023-07-06T03:46:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 -- 172.21.15.97:0/1221808333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea406b360 msgr2=0x7fcea41f4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea406b360 0x7fcea41f4b70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcea0014040 tx=0x7fcea0004750 comp rx=0 tx=0).stop 2023-07-06T03:46:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 -- 172.21.15.97:0/1221808333 shutdown_connections 2023-07-06T03:46:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fce8806ace0 0x7fce8806d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcea415c610 0x7fcea40b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea406b360 0x7fcea41f4b70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 --2- 172.21.15.97:0/1221808333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcea40d2210 0x7fcea40b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 -- 172.21.15.97:0/1221808333 >> 172.21.15.97:0/1221808333 conn(0x7fcea406a6c0 msgr2=0x7fcea406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:19.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 -- 172.21.15.97:0/1221808333 shutdown_connections 2023-07-06T03:46:19.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.350+0000 7fce8f7fe700 1 -- 172.21.15.97:0/1221808333 wait complete. 2023-07-06T03:46:19.369 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:46:19.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.564+0000 7f05a3dff700 1 Processor -- start 2023-07-06T03:46:19.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.564+0000 7f05a3dff700 1 -- start start 2023-07-06T03:46:19.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.565+0000 7f05a3dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f059c15c610 0x7f059c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.565+0000 7f05a3dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f059c0d2210 0x7f059c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.565+0000 7f05a3dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c0cff90 con 0x7f059c06b360 2023-07-06T03:46:19.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.565+0000 7f05a3dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c0d0780 con 0x7f059c15c610 2023-07-06T03:46:19.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.565+0000 7f05a3dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c15c9e0 con 0x7f059c0d2210 2023-07-06T03:46:19.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.565+0000 7f05a1b9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f059c15c610 0x7f059c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.565+0000 7f05a1b9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f059c15c610 0x7f059c0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41002/0 (socket says 172.21.15.97:41002) 2023-07-06T03:46:19.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.565+0000 7f05a139a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f059c06b360 0x7f059c0b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60912/0 (socket says 172.21.15.97:60912) 2023-07-06T03:46:19.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.565+0000 7f05a1b9b700 1 -- 172.21.15.97:0/4264339016 learned_addr learned my addr 172.21.15.97:0/4264339016 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:19.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.566+0000 7f05a239c700 1 --2- 172.21.15.97:0/4264339016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f059c0d2210 0x7f059c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.566+0000 7f05a0b99700 1 -- 172.21.15.97:0/4264339016 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258426754 0 0) 0x7f059c0cff90 con 0x7f059c06b360 2023-07-06T03:46:19.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.566+0000 7f05a1b9b700 1 -- 172.21.15.97:0/4264339016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f059c0d2210 msgr2=0x7f059c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:19.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.566+0000 7f05a1b9b700 1 --2- 172.21.15.97:0/4264339016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f059c0d2210 0x7f059c0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.566+0000 7f05a1b9b700 1 -- 172.21.15.97:0/4264339016 >> v1:172.21.15.97:6789/0 conn(0x7f059c06b360 legacy=0x7f059c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.566+0000 7f05a1b9b700 1 -- 172.21.15.97:0/4264339016 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c0b5650 con 0x7f059c15c610 2023-07-06T03:46:19.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.566+0000 7f05a1b9b700 1 --2- 172.21.15.97:0/4264339016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c15c610 0x7f059c0b47b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f05980098f0 tx=0x7f0598009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9531bfc595bbbe6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:19.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.566+0000 7f05a0b99700 1 -- 172.21.15.97:0/4264339016 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800b8a0 con 0x7f059c15c610 2023-07-06T03:46:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.566+0000 7f05a0b99700 1 -- 172.21.15.97:0/4264339016 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059800ba00 con 0x7f059c15c610 2023-07-06T03:46:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.566+0000 7f05a0b99700 1 -- 172.21.15.97:0/4264339016 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800bd90 con 0x7f059c15c610 2023-07-06T03:46:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.567+0000 7f05a3dff700 1 -- 172.21.15.97:0/4264339016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c15c610 msgr2=0x7f059c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.567+0000 7f05a3dff700 1 --2- 172.21.15.97:0/4264339016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c15c610 0x7f059c0b47b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f05980098f0 tx=0x7f0598009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.567+0000 7f05a3dff700 1 -- 172.21.15.97:0/4264339016 shutdown_connections 2023-07-06T03:46:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.567+0000 7f05a3dff700 1 --2- 172.21.15.97:0/4264339016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c15c610 0x7f059c0b47b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.567+0000 7f05a3dff700 1 --2- 172.21.15.97:0/4264339016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f059c0d2210 0x7f059c0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.567+0000 7f05a3dff700 1 -- 172.21.15.97:0/4264339016 >> 172.21.15.97:0/4264339016 conn(0x7f059c06a6c0 msgr2=0x7f059c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.567+0000 7f05a3dff700 1 -- 172.21.15.97:0/4264339016 shutdown_connections 2023-07-06T03:46:19.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.567+0000 7f05a3dff700 1 -- 172.21.15.97:0/4264339016 wait complete. 2023-07-06T03:46:19.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.569+0000 7f05a3dff700 1 Processor -- start 2023-07-06T03:46:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.569+0000 7f05a3dff700 1 -- start start 2023-07-06T03:46:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.570+0000 7f05a3dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f059c06b360 0x7f059c157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.570+0000 7f05a3dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c0d2210 0x7f059c15ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.570+0000 7f05a3dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f059c15c610 0x7f059c0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.570+0000 7f05a3dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c15cb20 con 0x7f059c0d2210 2023-07-06T03:46:19.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.570+0000 7f05a3dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c15cd30 con 0x7f059c06b360 2023-07-06T03:46:19.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.570+0000 7f05a3dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c15d010 con 0x7f059c15c610 2023-07-06T03:46:19.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.570+0000 7f05a139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c0d2210 0x7f059c15ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.570+0000 7f05a139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c0d2210 0x7f059c15ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41010/0 (socket says 172.21.15.97:41010) 2023-07-06T03:46:19.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.570+0000 7f05a139a700 1 -- 172.21.15.97:0/4008739912 learned_addr learned my addr 172.21.15.97:0/4008739912 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:19.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a239c700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f059c15c610 0x7f059c0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a1b9b700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f059c06b360 0x7f059c157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a139a700 1 -- 172.21.15.97:0/4008739912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f059c15c610 msgr2=0x7f059c0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a139a700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f059c15c610 0x7f059c0cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a139a700 1 -- 172.21.15.97:0/4008739912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f059c06b360 msgr2=0x7f059c157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a139a700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f059c06b360 0x7f059c157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a139a700 1 -- 172.21.15.97:0/4008739912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598009550 con 0x7f059c0d2210 2023-07-06T03:46:19.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a239c700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f059c15c610 0x7f059c0cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:19.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a1b9b700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f059c06b360 0x7f059c157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:19.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a139a700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c0d2210 0x7f059c15ddd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f058c0011e0 tx=0x7f058c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:19.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05927fc700 1 -- 172.21.15.97:0/4008739912 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c004c80 con 0x7f059c0d2210 2023-07-06T03:46:19.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05927fc700 1 -- 172.21.15.97:0/4008739912 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c004e00 con 0x7f059c0d2210 2023-07-06T03:46:19.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a3dff700 1 -- 172.21.15.97:0/4008739912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c15d2f0 con 0x7f059c0d2210 2023-07-06T03:46:19.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05927fc700 1 -- 172.21.15.97:0/4008739912 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c007310 con 0x7f059c0d2210 2023-07-06T03:46:19.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.571+0000 7f05a3dff700 1 -- 172.21.15.97:0/4008739912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c15d8b0 con 0x7f059c0d2210 2023-07-06T03:46:19.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.572+0000 7f05a3dff700 1 -- 172.21.15.97:0/4008739912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c15dad0 con 0x7f059c0d2210 2023-07-06T03:46:19.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.573+0000 7f05927fc700 1 -- 172.21.15.97:0/4008739912 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f058c007470 con 0x7f059c0d2210 2023-07-06T03:46:19.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.575+0000 7f05927fc700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f058006ace0 0x7f058006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.575+0000 7f05927fc700 1 -- 172.21.15.97:0/4008739912 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f058c089210 con 0x7f059c0d2210 2023-07-06T03:46:19.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.575+0000 7f05a1b9b700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f058006ace0 0x7f058006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.575+0000 7f05a1b9b700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f058006ace0 0x7f058006d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0598000c00 tx=0x7f0598008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:19.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.579+0000 7f05927fc700 1 -- 172.21.15.97:0/4008739912 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f058c0542b0 con 0x7f059c0d2210 2023-07-06T03:46:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.774+0000 7f05a3dff700 1 -- 172.21.15.97:0/4008739912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f059c0ce600 con 0x7f059c0d2210 2023-07-06T03:46:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.775+0000 7f05927fc700 1 -- 172.21.15.97:0/4008739912 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f058c0578d0 con 0x7f059c0d2210 2023-07-06T03:46:19.777 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:46:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.777+0000 7f05a3dff700 1 -- 172.21.15.97:0/4008739912 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f058006ace0 msgr2=0x7f058006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.777+0000 7f05a3dff700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f058006ace0 0x7f058006d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0598000c00 tx=0x7f0598008030 comp rx=0 tx=0).stop 2023-07-06T03:46:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.777+0000 7f05a3dff700 1 -- 172.21.15.97:0/4008739912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c0d2210 msgr2=0x7f059c15ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.777+0000 7f05a3dff700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c0d2210 0x7f059c15ddd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f058c0011e0 tx=0x7f058c00cae0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.777+0000 7f05a3dff700 1 -- 172.21.15.97:0/4008739912 shutdown_connections 2023-07-06T03:46:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.777+0000 7f05a3dff700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f058006ace0 0x7f058006d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.777+0000 7f05a3dff700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f059c15c610 0x7f059c0cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.777+0000 7f05a3dff700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f059c06b360 0x7f059c157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.777+0000 7f05a3dff700 1 --2- 172.21.15.97:0/4008739912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c0d2210 0x7f059c15ddd0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.777+0000 7f05a3dff700 1 -- 172.21.15.97:0/4008739912 >> 172.21.15.97:0/4008739912 conn(0x7f059c06a6c0 msgr2=0x7f059c0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.778+0000 7f05a3dff700 1 -- 172.21.15.97:0/4008739912 shutdown_connections 2023-07-06T03:46:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.778+0000 7f05a3dff700 1 -- 172.21.15.97:0/4008739912 wait complete. 2023-07-06T03:46:19.797 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:46:19.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.991+0000 7fd62f5ff700 1 Processor -- start 2023-07-06T03:46:19.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.992+0000 7fd62f5ff700 1 -- start start 2023-07-06T03:46:19.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.993+0000 7fd62f5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd6280cb2c0 0x7fd6280cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.993+0000 7fd62f5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd6280b26f0 0x7fd62806c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:19.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.993+0000 7fd62f5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6280cb690 con 0x7fd62806b760 2023-07-06T03:46:19.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.993+0000 7fd62f5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62806aa70 con 0x7fd6280b26f0 2023-07-06T03:46:19.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.993+0000 7fd62f5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62806dde0 con 0x7fd6280cb2c0 2023-07-06T03:46:19.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.993+0000 7fd62db9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd6280b26f0 0x7fd62806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.993+0000 7fd62db9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd6280b26f0 0x7fd62806c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45180/0 (socket says 172.21.15.97:45180) 2023-07-06T03:46:19.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.993+0000 7fd62db9c700 1 -- 172.21.15.97:0/2335333665 learned_addr learned my addr 172.21.15.97:0/2335333665 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:19.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.993+0000 7fd62d39b700 1 --2- 172.21.15.97:0/2335333665 >> v2:172.21.15.146:3300/0 conn(0x7fd6280cb2c0 0x7fd6280cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:19.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.994+0000 7fd62db9c700 1 -- 172.21.15.97:0/2335333665 >> v2:172.21.15.146:3300/0 conn(0x7fd6280cb2c0 msgr2=0x7fd6280cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:19.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.994+0000 7fd62db9c700 1 --2- 172.21.15.97:0/2335333665 >> v2:172.21.15.146:3300/0 conn(0x7fd6280cb2c0 0x7fd6280cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.994+0000 7fd62db9c700 1 -- 172.21.15.97:0/2335333665 >> v1:172.21.15.146:6789/0 conn(0x7fd62806b760 legacy=0x7fd6280b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.994+0000 7fd62db9c700 1 -- 172.21.15.97:0/2335333665 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6280cff50 con 0x7fd6280b26f0 2023-07-06T03:46:19.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.994+0000 7fd62d39b700 1 --2- 172.21.15.97:0/2335333665 >> v2:172.21.15.146:3300/0 conn(0x7fd6280cb2c0 0x7fd6280cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:19.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.994+0000 7fd62db9c700 1 --2- 172.21.15.97:0/2335333665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6280b26f0 0x7fd62806c360 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd628066310 tx=0x7fd62000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d12899c83c78e45 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:19.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.994+0000 7fd61ffff700 1 -- 172.21.15.97:0/2335333665 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62000e070 con 0x7fd6280b26f0 2023-07-06T03:46:19.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.994+0000 7fd61ffff700 1 -- 172.21.15.97:0/2335333665 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62000b9c0 con 0x7fd6280b26f0 2023-07-06T03:46:19.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.994+0000 7fd61ffff700 1 -- 172.21.15.97:0/2335333665 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62000bd70 con 0x7fd6280b26f0 2023-07-06T03:46:19.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.995+0000 7fd62f5ff700 1 -- 172.21.15.97:0/2335333665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6280b26f0 msgr2=0x7fd62806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:19.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.995+0000 7fd62f5ff700 1 --2- 172.21.15.97:0/2335333665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6280b26f0 0x7fd62806c360 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd628066310 tx=0x7fd62000b330 comp rx=0 tx=0).stop 2023-07-06T03:46:19.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.995+0000 7fd62f5ff700 1 -- 172.21.15.97:0/2335333665 shutdown_connections 2023-07-06T03:46:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.995+0000 7fd62f5ff700 1 --2- 172.21.15.97:0/2335333665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6280b26f0 0x7fd62806c360 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.995+0000 7fd62f5ff700 1 --2- 172.21.15.97:0/2335333665 >> v2:172.21.15.146:3300/0 conn(0x7fd6280cb2c0 0x7fd6280cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.995+0000 7fd62f5ff700 1 -- 172.21.15.97:0/2335333665 >> 172.21.15.97:0/2335333665 conn(0x7fd6280cc890 msgr2=0x7fd6280cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.995+0000 7fd62f5ff700 1 -- 172.21.15.97:0/2335333665 shutdown_connections 2023-07-06T03:46:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.995+0000 7fd62f5ff700 1 -- 172.21.15.97:0/2335333665 wait complete. 2023-07-06T03:46:20.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.996+0000 7fd62f5ff700 1 Processor -- start 2023-07-06T03:46:20.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.997+0000 7fd62f5ff700 1 -- start start 2023-07-06T03:46:20.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.997+0000 7fd62f5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62806b760 0x7fd6280b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:20.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62f5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6280b26f0 0x7fd6280ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:20.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62f5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6280cb2c0 0x7fd6280b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:20.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62f5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280c41a0 con 0x7fd6280b26f0 2023-07-06T03:46:20.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62f5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280aec10 con 0x7fd62806b760 2023-07-06T03:46:20.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62f5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280c35b0 con 0x7fd6280cb2c0 2023-07-06T03:46:20.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62cb9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6280b26f0 0x7fd6280ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:20.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62cb9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6280b26f0 0x7fd6280ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45186/0 (socket says 172.21.15.97:45186) 2023-07-06T03:46:20.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62cb9a700 1 -- 172.21.15.97:0/4098698198 learned_addr learned my addr 172.21.15.97:0/4098698198 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:20.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62d39b700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62806b760 0x7fd6280b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:20.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62db9c700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6280cb2c0 0x7fd6280b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:20.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62cb9a700 1 -- 172.21.15.97:0/4098698198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6280cb2c0 msgr2=0x7fd6280b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:20.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62cb9a700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6280cb2c0 0x7fd6280b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62cb9a700 1 -- 172.21.15.97:0/4098698198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62806b760 msgr2=0x7fd6280b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:20.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62cb9a700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62806b760 0x7fd6280b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.998+0000 7fd62cb9a700 1 -- 172.21.15.97:0/4098698198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd620009b30 con 0x7fd6280b26f0 2023-07-06T03:46:20.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.999+0000 7fd62cb9a700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6280b26f0 0x7fd6280ce080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c510 tx=0x7fd61800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:20.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.999+0000 7fd61dffb700 1 -- 172.21.15.97:0/4098698198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180052d0 con 0x7fd6280b26f0 2023-07-06T03:46:20.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.999+0000 7fd61dffb700 1 -- 172.21.15.97:0/4098698198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd618005430 con 0x7fd6280b26f0 2023-07-06T03:46:20.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.999+0000 7fd62f5ff700 1 -- 172.21.15.97:0/4098698198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6280b1970 con 0x7fd6280b26f0 2023-07-06T03:46:20.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.999+0000 7fd61dffb700 1 -- 172.21.15.97:0/4098698198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180038c0 con 0x7fd6280b26f0 2023-07-06T03:46:20.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:19.999+0000 7fd62f5ff700 1 -- 172.21.15.97:0/4098698198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6280c3ae0 con 0x7fd6280b26f0 2023-07-06T03:46:20.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.000+0000 7fd62f5ff700 1 -- 172.21.15.97:0/4098698198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6280c42e0 con 0x7fd6280b26f0 2023-07-06T03:46:20.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.001+0000 7fd61dffb700 1 -- 172.21.15.97:0/4098698198 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd618003a20 con 0x7fd6280b26f0 2023-07-06T03:46:20.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.002+0000 7fd61dffb700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd60c06ad10 0x7fd60c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:20.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.002+0000 7fd61dffb700 1 -- 172.21.15.97:0/4098698198 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd6180888c0 con 0x7fd6280b26f0 2023-07-06T03:46:20.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.002+0000 7fd62d39b700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd60c06ad10 0x7fd60c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:20.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.003+0000 7fd62d39b700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd60c06ad10 0x7fd60c06d1c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd62400c940 tx=0x7fd624008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:20.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.006+0000 7fd61dffb700 1 -- 172.21.15.97:0/4098698198 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd6180538d0 con 0x7fd6280b26f0 2023-07-06T03:46:20.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.215+0000 7fd62f5ff700 1 -- 172.21.15.97:0/4098698198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd62804e340 con 0x7fd6280b26f0 2023-07-06T03:46:20.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.215+0000 7fd61dffb700 1 -- 172.21.15.97:0/4098698198 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd618056ef0 con 0x7fd6280b26f0 2023-07-06T03:46:20.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.217+0000 7fd62f5ff700 1 -- 172.21.15.97:0/4098698198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd60c06ad10 msgr2=0x7fd60c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:20.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.217+0000 7fd62f5ff700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd60c06ad10 0x7fd60c06d1c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd62400c940 tx=0x7fd624008040 comp rx=0 tx=0).stop 2023-07-06T03:46:20.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.218+0000 7fd62f5ff700 1 -- 172.21.15.97:0/4098698198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6280b26f0 msgr2=0x7fd6280ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:20.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.218+0000 7fd62f5ff700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6280b26f0 0x7fd6280ce080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c510 tx=0x7fd61800c820 comp rx=0 tx=0).stop 2023-07-06T03:46:20.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.218+0000 7fd62f5ff700 1 -- 172.21.15.97:0/4098698198 shutdown_connections 2023-07-06T03:46:20.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.218+0000 7fd62f5ff700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd60c06ad10 0x7fd60c06d1c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.218+0000 7fd62f5ff700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd6280cb2c0 0x7fd6280b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.218+0000 7fd62f5ff700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd62806b760 0x7fd6280b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.218+0000 7fd62f5ff700 1 --2- 172.21.15.97:0/4098698198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6280b26f0 0x7fd6280ce080 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.218+0000 7fd62f5ff700 1 -- 172.21.15.97:0/4098698198 >> 172.21.15.97:0/4098698198 conn(0x7fd6280cc890 msgr2=0x7fd62806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.218+0000 7fd62f5ff700 1 -- 172.21.15.97:0/4098698198 shutdown_connections 2023-07-06T03:46:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.218+0000 7fd62f5ff700 1 -- 172.21.15.97:0/4098698198 wait complete. 2023-07-06T03:46:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:46:20.237 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2023-07-06T03:46:20.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:20.238 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-07-06T03:46:20.314 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:46:20.315 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:46:20.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:20.316 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T03:46:20.363 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T03:46:20.364 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:46:20.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.556+0000 7f57bf7ff700 1 Processor -- start 2023-07-06T03:46:20.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.556+0000 7f57bf7ff700 1 -- start start 2023-07-06T03:46:20.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.557+0000 7f57bf7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f57b815c520 0x7f57b80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:20.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.557+0000 7f57bf7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b8160180 0x7f57b81605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:20.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bf7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57b8160b30 0x7f57b80cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:20.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bf7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b80cd8d0 con 0x7f57b8160b30 2023-07-06T03:46:20.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bf7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b80cdae0 con 0x7f57b815c520 2023-07-06T03:46:20.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bf7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80b6640 con 0x7f57b8160180 2023-07-06T03:46:20.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bdd9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57b8160b30 0x7f57b80cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:20.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bdd9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57b8160b30 0x7f57b80cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45190/0 (socket says 172.21.15.97:45190) 2023-07-06T03:46:20.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bdd9c700 1 -- 172.21.15.97:0/3412657831 learned_addr learned my addr 172.21.15.97:0/3412657831 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:20.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bcd9a700 1 --2- 172.21.15.97:0/3412657831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b8160180 0x7f57b81605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:20.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bd59b700 1 --2- 172.21.15.97:0/3412657831 >> v2:172.21.15.146:3300/0 conn(0x7f57b815c520 0x7f57b80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:20.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bdd9c700 1 -- 172.21.15.97:0/3412657831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b8160180 msgr2=0x7f57b81605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:20.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bdd9c700 1 --2- 172.21.15.97:0/3412657831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b8160180 0x7f57b81605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bdd9c700 1 -- 172.21.15.97:0/3412657831 >> v2:172.21.15.146:3300/0 conn(0x7f57b815c520 msgr2=0x7f57b80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:20.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bdd9c700 1 --2- 172.21.15.97:0/3412657831 >> v2:172.21.15.146:3300/0 conn(0x7f57b815c520 0x7f57b80cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.558+0000 7f57bdd9c700 1 -- 172.21.15.97:0/3412657831 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b80b6920 con 0x7f57b8160b30 2023-07-06T03:46:20.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.559+0000 7f57bdd9c700 1 --2- 172.21.15.97:0/3412657831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57b8160b30 0x7f57b80cf050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f57b80b4c70 tx=0x7f57b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db5a7eceb732e69 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:20.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.559+0000 7f57affff700 1 -- 172.21.15.97:0/3412657831 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000e070 con 0x7f57b8160b30 2023-07-06T03:46:20.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.559+0000 7f57affff700 1 -- 172.21.15.97:0/3412657831 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b000b910 con 0x7f57b8160b30 2023-07-06T03:46:20.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.559+0000 7f57affff700 1 -- 172.21.15.97:0/3412657831 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000bcc0 con 0x7f57b8160b30 2023-07-06T03:46:20.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.559+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3412657831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57b8160b30 msgr2=0x7f57b80cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:20.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.559+0000 7f57bf7ff700 1 --2- 172.21.15.97:0/3412657831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57b8160b30 0x7f57b80cf050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f57b80b4c70 tx=0x7f57b000b330 comp rx=0 tx=0).stop 2023-07-06T03:46:20.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.559+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3412657831 shutdown_connections 2023-07-06T03:46:20.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.559+0000 7f57bf7ff700 1 --2- 172.21.15.97:0/3412657831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57b8160b30 0x7f57b80cf050 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.559+0000 7f57bf7ff700 1 --2- 172.21.15.97:0/3412657831 >> v2:172.21.15.146:3300/0 conn(0x7f57b815c520 0x7f57b80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.559+0000 7f57bf7ff700 1 --2- 172.21.15.97:0/3412657831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b8160180 0x7f57b81605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.559+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3412657831 >> 172.21.15.97:0/3412657831 conn(0x7f57b8071790 msgr2=0x7f57b8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:20.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.560+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3412657831 shutdown_connections 2023-07-06T03:46:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.560+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3412657831 wait complete. 2023-07-06T03:46:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.561+0000 7f57bf7ff700 1 Processor -- start 2023-07-06T03:46:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.561+0000 7f57bf7ff700 1 -- start start 2023-07-06T03:46:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.562+0000 7f57bf7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57b815c520 0x7f57b81f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.562+0000 7f57bf7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b8160180 0x7f57b815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.562+0000 7f57bf7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57b8160b30 0x7f57b815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.562+0000 7f57bf7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b81f3170 con 0x7f57b815c520 2023-07-06T03:46:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.562+0000 7f57bf7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80ca2b0 con 0x7f57b8160b30 2023-07-06T03:46:20.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.562+0000 7f57bf7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b81f2bb0 con 0x7f57b8160180 2023-07-06T03:46:20.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bd59b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57b815c520 0x7f57b81f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:20.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bd59b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57b815c520 0x7f57b81f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45204/0 (socket says 172.21.15.97:45204) 2023-07-06T03:46:20.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bd59b700 1 -- 172.21.15.97:0/3007607374 learned_addr learned my addr 172.21.15.97:0/3007607374 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:20.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bcd9a700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b8160180 0x7f57b815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:20.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bdd9c700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57b8160b30 0x7f57b815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:20.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bd59b700 1 -- 172.21.15.97:0/3007607374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b8160180 msgr2=0x7f57b815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bd59b700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b8160180 0x7f57b815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bd59b700 1 -- 172.21.15.97:0/3007607374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57b8160b30 msgr2=0x7f57b815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bd59b700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57b8160b30 0x7f57b815b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bd59b700 1 -- 172.21.15.97:0/3007607374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b0009b30 con 0x7f57b815c520 2023-07-06T03:46:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bd59b700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57b815c520 0x7f57b81f21d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f57b400e870 tx=0x7f57b400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57adffb700 1 -- 172.21.15.97:0/3007607374 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4009590 con 0x7f57b815c520 2023-07-06T03:46:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57adffb700 1 -- 172.21.15.97:0/3007607374 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b4013680 con 0x7f57b815c520 2023-07-06T03:46:20.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57adffb700 1 -- 172.21.15.97:0/3007607374 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b40139f0 con 0x7f57b815c520 2023-07-06T03:46:20.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.563+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3007607374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b81f2e90 con 0x7f57b815c520 2023-07-06T03:46:20.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.564+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3007607374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b815f630 con 0x7f57b815c520 2023-07-06T03:46:20.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.565+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3007607374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b804e340 con 0x7f57b815c520 2023-07-06T03:46:20.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.565+0000 7f57adffb700 1 -- 172.21.15.97:0/3007607374 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57b4013b50 con 0x7f57b815c520 2023-07-06T03:46:20.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.566+0000 7f57adffb700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f579c06ace0 0x7f579c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:20.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.566+0000 7f57adffb700 1 -- 172.21.15.97:0/3007607374 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57b40896c0 con 0x7f57b815c520 2023-07-06T03:46:20.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.567+0000 7f57bcd9a700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f579c06ace0 0x7f579c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:20.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.567+0000 7f57bcd9a700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f579c06ace0 0x7f579c06d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009b60 tx=0x7f57a8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:20.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.571+0000 7f57adffb700 1 -- 172.21.15.97:0/3007607374 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f57b40581f0 con 0x7f57b815c520 2023-07-06T03:46:20.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.808+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3007607374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57b815ea20 con 0x7f57b815c520 2023-07-06T03:46:20.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.809+0000 7f57adffb700 1 -- 172.21.15.97:0/3007607374 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4665 (secure 0 0 0) 0x7f57b4057d80 con 0x7f57b815c520 2023-07-06T03:46:20.811 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:20.811 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/4255451313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4255451313},{"type":"v1","addr":"172.21.15.97:6836","nonce":4255451313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/370322164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":370322164},{"type":"v1","addr":"172.21.15.97:6837","nonce":370322164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3764820847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3764820847},{"type":"v1","addr":"172.21.15.146:6835","nonce":3764820847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:45:11.126281+0000","modified":"2023-07-06T03:45:14.685499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6833/186544257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":186544257},{"type":"v1","addr":"172.21.15.146:6833","nonce":186544257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T03:46:20.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3007607374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f579c06ace0 msgr2=0x7f579c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:20.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f579c06ace0 0x7f579c06d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009b60 tx=0x7f57a8009210 comp rx=0 tx=0).stop 2023-07-06T03:46:20.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3007607374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57b815c520 msgr2=0x7f57b81f21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:20.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57b815c520 0x7f57b81f21d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f57b400e870 tx=0x7f57b400eb80 comp rx=0 tx=0).stop 2023-07-06T03:46:20.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3007607374 shutdown_connections 2023-07-06T03:46:20.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f579c06ace0 0x7f579c06d190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57b8160b30 0x7f57b815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57b815c520 0x7f57b81f21d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 --2- 172.21.15.97:0/3007607374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57b8160180 0x7f57b815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:20.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3007607374 >> 172.21.15.97:0/3007607374 conn(0x7f57b8071790 msgr2=0x7f57b8157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:20.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3007607374 shutdown_connections 2023-07-06T03:46:20.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:20.812+0000 7f57bf7ff700 1 -- 172.21.15.97:0/3007607374 wait complete. 2023-07-06T03:46:20.816 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-07-06T03:46:20.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:21.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.027+0000 7fc7ad19d700 1 Processor -- start 2023-07-06T03:46:21.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.027+0000 7fc7ad19d700 1 -- start start 2023-07-06T03:46:21.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.028+0000 7fc7ad19d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7a815c4b0 0x7fc7a80cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.028+0000 7fc7ad19d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a80d08b0 con 0x7fc7a815c0e0 2023-07-06T03:46:21.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.028+0000 7fc7ad19d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a80d2700 con 0x7fc7a80cd7d0 2023-07-06T03:46:21.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.028+0000 7fc7ad19d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a80d0b50 con 0x7fc7a815c4b0 2023-07-06T03:46:21.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.028+0000 7fc7a77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7a815c4b0 0x7fc7a80cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.028+0000 7fc7a7fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc7a815c0e0 0x7fc7a80d1330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54484/0 (socket says 172.21.15.97:54484) 2023-07-06T03:46:21.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.029+0000 7fc7a7fff700 1 -- 172.21.15.97:0/1797418042 learned_addr learned my addr 172.21.15.97:0/1797418042 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:21.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.029+0000 7fc7a77fe700 1 -- 172.21.15.97:0/1797418042 >> v1:172.21.15.146:6789/0 conn(0x7fc7a80cd7d0 legacy=0x7fc7a80b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.029+0000 7fc7a77fe700 1 -- 172.21.15.97:0/1797418042 >> v1:172.21.15.97:6789/0 conn(0x7fc7a815c0e0 legacy=0x7fc7a80d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.029+0000 7fc7a77fe700 1 -- 172.21.15.97:0/1797418042 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a80d0e30 con 0x7fc7a815c4b0 2023-07-06T03:46:21.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.029+0000 7fc7a6ffd700 1 -- 172.21.15.97:0/1797418042 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396415479 0 0) 0x7fc7a80d08b0 con 0x7fc7a815c0e0 2023-07-06T03:46:21.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.029+0000 7fc7a77fe700 1 --2- 172.21.15.97:0/1797418042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7a815c4b0 0x7fc7a80cd290 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009870 tx=0x7fc79c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd98debdc82f10b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:21.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.029+0000 7fc7a6ffd700 1 -- 172.21.15.97:0/1797418042 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c011830 con 0x7fc7a815c4b0 2023-07-06T03:46:21.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.030+0000 7fc7a6ffd700 1 -- 172.21.15.97:0/1797418042 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c011990 con 0x7fc7a815c4b0 2023-07-06T03:46:21.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.030+0000 7fc7a6ffd700 1 -- 172.21.15.97:0/1797418042 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c011d20 con 0x7fc7a815c4b0 2023-07-06T03:46:21.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.030+0000 7fc7ad19d700 1 -- 172.21.15.97:0/1797418042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7a815c4b0 msgr2=0x7fc7a80cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.030+0000 7fc7ad19d700 1 --2- 172.21.15.97:0/1797418042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7a815c4b0 0x7fc7a80cd290 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009870 tx=0x7fc79c009b80 comp rx=0 tx=0).stop 2023-07-06T03:46:21.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.030+0000 7fc7ad19d700 1 -- 172.21.15.97:0/1797418042 shutdown_connections 2023-07-06T03:46:21.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.030+0000 7fc7ad19d700 1 --2- 172.21.15.97:0/1797418042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7a815c4b0 0x7fc7a80cd290 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.030+0000 7fc7ad19d700 1 -- 172.21.15.97:0/1797418042 >> 172.21.15.97:0/1797418042 conn(0x7fc7a80cf6c0 msgr2=0x7fc7a80cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:21.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.030+0000 7fc7ad19d700 1 -- 172.21.15.97:0/1797418042 shutdown_connections 2023-07-06T03:46:21.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.030+0000 7fc7ad19d700 1 -- 172.21.15.97:0/1797418042 wait complete. 2023-07-06T03:46:21.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.032+0000 7fc7ad19d700 1 Processor -- start 2023-07-06T03:46:21.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.032+0000 7fc7ad19d700 1 -- start start 2023-07-06T03:46:21.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.033+0000 7fc7ad19d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7a80cd7d0 0x7fc7a814f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.033+0000 7fc7ad19d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7a815c0e0 0x7fc7a80c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.033+0000 7fc7ad19d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7a815c4b0 0x7fc7a80ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.033+0000 7fc7ad19d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a814faa0 con 0x7fc7a815c4b0 2023-07-06T03:46:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.033+0000 7fc7ad19d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80cc330 con 0x7fc7a80cd7d0 2023-07-06T03:46:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.033+0000 7fc7ad19d700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80cc610 con 0x7fc7a815c0e0 2023-07-06T03:46:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.033+0000 7fc7ac99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7a815c4b0 0x7fc7a80ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.033+0000 7fc7ac99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7a815c4b0 0x7fc7a80ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45216/0 (socket says 172.21.15.97:45216) 2023-07-06T03:46:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.033+0000 7fc7ac99c700 1 -- 172.21.15.97:0/2504183378 learned_addr learned my addr 172.21.15.97:0/2504183378 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.033+0000 7fc7a77fe700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7a815c0e0 0x7fc7a80c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.033+0000 7fc7a7fff700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7a80cd7d0 0x7fc7a814f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7ac99c700 1 -- 172.21.15.97:0/2504183378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7a815c0e0 msgr2=0x7fc7a80c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7ac99c700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7a815c0e0 0x7fc7a80c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7ac99c700 1 -- 172.21.15.97:0/2504183378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7a80cd7d0 msgr2=0x7fc7a814f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7ac99c700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7a80cd7d0 0x7fc7a814f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7ac99c700 1 -- 172.21.15.97:0/2504183378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c0094a0 con 0x7fc7a815c4b0 2023-07-06T03:46:21.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7a7fff700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7a80cd7d0 0x7fc7a814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:21.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7ac99c700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7a815c4b0 0x7fc7a80ca170 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000cc50 tx=0x7fc7a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:21.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7a4ff9700 1 -- 172.21.15.97:0/2504183378 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000ad80 con 0x7fc7a815c4b0 2023-07-06T03:46:21.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7a4ff9700 1 -- 172.21.15.97:0/2504183378 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a000ce50 con 0x7fc7a815c4b0 2023-07-06T03:46:21.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7ad19d700 1 -- 172.21.15.97:0/2504183378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a80cc8f0 con 0x7fc7a815c4b0 2023-07-06T03:46:21.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7a4ff9700 1 -- 172.21.15.97:0/2504183378 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0003c40 con 0x7fc7a815c4b0 2023-07-06T03:46:21.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.034+0000 7fc7ad19d700 1 -- 172.21.15.97:0/2504183378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a80b6c90 con 0x7fc7a815c4b0 2023-07-06T03:46:21.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.035+0000 7fc7ad19d700 1 -- 172.21.15.97:0/2504183378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a80728f0 con 0x7fc7a815c4b0 2023-07-06T03:46:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.036+0000 7fc7a4ff9700 1 -- 172.21.15.97:0/2504183378 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc7a00050d0 con 0x7fc7a815c4b0 2023-07-06T03:46:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.037+0000 7fc7a4ff9700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc79406ace0 0x7fc79406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.037+0000 7fc7a4ff9700 1 -- 172.21.15.97:0/2504183378 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7a00895a0 con 0x7fc7a815c4b0 2023-07-06T03:46:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.037+0000 7fc7a7fff700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc79406ace0 0x7fc79406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.038+0000 7fc7a7fff700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc79406ace0 0x7fc79406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc798000f60 tx=0x7fc798009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:21.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.041+0000 7fc7a4ff9700 1 -- 172.21.15.97:0/2504183378 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc7a0054640 con 0x7fc7a815c4b0 2023-07-06T03:46:21.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.237+0000 7fc7ad19d700 1 -- 172.21.15.97:0/2504183378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7a80c63f0 con 0x7fc7a815c4b0 2023-07-06T03:46:21.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.239+0000 7fc7a4ff9700 1 -- 172.21.15.97:0/2504183378 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7fc7a0057c60 con 0x7fc7a815c4b0 2023-07-06T03:46:21.241 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:21.241 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:21.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 -- 172.21.15.97:0/2504183378 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc79406ace0 msgr2=0x7fc79406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc79406ace0 0x7fc79406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc798000f60 tx=0x7fc798009040 comp rx=0 tx=0).stop 2023-07-06T03:46:21.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 -- 172.21.15.97:0/2504183378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7a815c4b0 msgr2=0x7fc7a80ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7a815c4b0 0x7fc7a80ca170 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000cc50 tx=0x7fc7a000a340 comp rx=0 tx=0).stop 2023-07-06T03:46:21.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 -- 172.21.15.97:0/2504183378 shutdown_connections 2023-07-06T03:46:21.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc79406ace0 0x7fc79406d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7a815c4b0 0x7fc7a80ca170 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7a80cd7d0 0x7fc7a814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 --2- 172.21.15.97:0/2504183378 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc7a815c0e0 0x7fc7a80c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 -- 172.21.15.97:0/2504183378 >> 172.21.15.97:0/2504183378 conn(0x7fc7a80cf6c0 msgr2=0x7fc7a8150260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:21.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 -- 172.21.15.97:0/2504183378 shutdown_connections 2023-07-06T03:46:21.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.241+0000 7fc7ad19d700 1 -- 172.21.15.97:0/2504183378 wait complete. 2023-07-06T03:46:21.261 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:45:09.115113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:45:10.111206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:46:21.262 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:46:21.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.457+0000 7fe5021ff700 1 Processor -- start 2023-07-06T03:46:21.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.457+0000 7fe5021ff700 1 -- start start 2023-07-06T03:46:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.459+0000 7fe5021ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4fc06b360 0x7fe4fc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.459+0000 7fe5021ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc0cff90 con 0x7fe4fc0d2210 2023-07-06T03:46:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.459+0000 7fe5021ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc0b5650 con 0x7fe4fc15c610 2023-07-06T03:46:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.459+0000 7fe5021ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4fc0d0780 con 0x7fe4fc06b360 2023-07-06T03:46:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.459+0000 7fe4f3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4fc06b360 0x7fe4fc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.459+0000 7fe4f3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4fc06b360 0x7fe4fc0b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45226/0 (socket says 172.21.15.97:45226) 2023-07-06T03:46:21.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.459+0000 7fe4f3fff700 1 -- 172.21.15.97:0/1367577773 learned_addr learned my addr 172.21.15.97:0/1367577773 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:21.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.459+0000 7fe4fbfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe4fc0d2210 0x7fe4fc0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54498/0 (socket says 172.21.15.97:54498) 2023-07-06T03:46:21.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.459+0000 7fe4fb7fe700 1 --1- 172.21.15.97:0/1367577773 >> v1:172.21.15.146:6789/0 conn(0x7fe4fc15c610 0x7fe4fc0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:37104/0 (socket says 172.21.15.97:37104) 2023-07-06T03:46:21.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.460+0000 7fe4faffd700 1 -- 172.21.15.97:0/1367577773 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630274922 0 0) 0x7fe4fc0b5650 con 0x7fe4fc15c610 2023-07-06T03:46:21.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.460+0000 7fe4faffd700 1 -- 172.21.15.97:0/1367577773 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4e80035b0 con 0x7fe4fc15c610 2023-07-06T03:46:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.460+0000 7fe4f3fff700 1 -- 172.21.15.97:0/1367577773 >> v1:172.21.15.146:6789/0 conn(0x7fe4fc15c610 legacy=0x7fe4fc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.460+0000 7fe4faffd700 1 -- 172.21.15.97:0/1367577773 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557607489 0 0) 0x7fe4fc0cff90 con 0x7fe4fc0d2210 2023-07-06T03:46:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.460+0000 7fe4f3fff700 1 -- 172.21.15.97:0/1367577773 >> v1:172.21.15.97:6789/0 conn(0x7fe4fc0d2210 legacy=0x7fe4fc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.460+0000 7fe4f3fff700 1 -- 172.21.15.97:0/1367577773 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc0b5400 con 0x7fe4fc06b360 2023-07-06T03:46:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.460+0000 7fe4f3fff700 1 --2- 172.21.15.97:0/1367577773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4fc06b360 0x7fe4fc0b4ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40098f0 tx=0x7fe4e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b365dfcafd1064b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.460+0000 7fe4faffd700 1 -- 172.21.15.97:0/1367577773 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4011830 con 0x7fe4fc06b360 2023-07-06T03:46:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.460+0000 7fe4faffd700 1 -- 172.21.15.97:0/1367577773 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e4011990 con 0x7fe4fc06b360 2023-07-06T03:46:21.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.460+0000 7fe4faffd700 1 -- 172.21.15.97:0/1367577773 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4011d20 con 0x7fe4fc06b360 2023-07-06T03:46:21.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.461+0000 7fe5021ff700 1 -- 172.21.15.97:0/1367577773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4fc06b360 msgr2=0x7fe4fc0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.461+0000 7fe5021ff700 1 --2- 172.21.15.97:0/1367577773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4fc06b360 0x7fe4fc0b4ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40098f0 tx=0x7fe4e4009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:21.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.461+0000 7fe5021ff700 1 -- 172.21.15.97:0/1367577773 shutdown_connections 2023-07-06T03:46:21.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.461+0000 7fe5021ff700 1 --2- 172.21.15.97:0/1367577773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4fc06b360 0x7fe4fc0b4ec0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.461+0000 7fe5021ff700 1 -- 172.21.15.97:0/1367577773 >> 172.21.15.97:0/1367577773 conn(0x7fe4fc06a6c0 msgr2=0x7fe4fc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:21.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.461+0000 7fe5021ff700 1 -- 172.21.15.97:0/1367577773 shutdown_connections 2023-07-06T03:46:21.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.461+0000 7fe5021ff700 1 -- 172.21.15.97:0/1367577773 wait complete. 2023-07-06T03:46:21.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.463+0000 7fe5021ff700 1 Processor -- start 2023-07-06T03:46:21.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.463+0000 7fe5021ff700 1 -- start start 2023-07-06T03:46:21.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe5021ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4fc06b360 0x7fe4fc0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe5021ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4fc0d2210 0x7fe4fc0c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe5021ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4fc15c610 0x7fe4fc1f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe5021ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0c9490 con 0x7fe4fc0d2210 2023-07-06T03:46:21.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe5021ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0c8b60 con 0x7fe4fc15c610 2023-07-06T03:46:21.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe5021ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0c8e40 con 0x7fe4fc06b360 2023-07-06T03:46:21.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe4f3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4fc0d2210 0x7fe4fc0c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe4f3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4fc0d2210 0x7fe4fc0c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45236/0 (socket says 172.21.15.97:45236) 2023-07-06T03:46:21.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe4f3fff700 1 -- 172.21.15.97:0/2134562482 learned_addr learned my addr 172.21.15.97:0/2134562482 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:21.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe4fb7fe700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4fc06b360 0x7fe4fc0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe4fbfff700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4fc15c610 0x7fe4fc1f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.464+0000 7fe4f3fff700 1 -- 172.21.15.97:0/2134562482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4fc06b360 msgr2=0x7fe4fc0c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.465+0000 7fe4f3fff700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4fc06b360 0x7fe4fc0c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.465+0000 7fe4f3fff700 1 -- 172.21.15.97:0/2134562482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4fc15c610 msgr2=0x7fe4fc1f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.465+0000 7fe4f3fff700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4fc15c610 0x7fe4fc1f5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.465+0000 7fe4f3fff700 1 -- 172.21.15.97:0/2134562482 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e4009550 con 0x7fe4fc0d2210 2023-07-06T03:46:21.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.465+0000 7fe4f3fff700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4fc0d2210 0x7fe4fc0c6da0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4014040 tx=0x7fe4e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:21.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.465+0000 7fe4f8ff9700 1 -- 172.21.15.97:0/2134562482 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4019ab0 con 0x7fe4fc0d2210 2023-07-06T03:46:21.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.465+0000 7fe4f8ff9700 1 -- 172.21.15.97:0/2134562482 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e401e070 con 0x7fe4fc0d2210 2023-07-06T03:46:21.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.465+0000 7fe5021ff700 1 -- 172.21.15.97:0/2134562482 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc0c9120 con 0x7fe4fc0d2210 2023-07-06T03:46:21.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.465+0000 7fe4f8ff9700 1 -- 172.21.15.97:0/2134562482 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4019e30 con 0x7fe4fc0d2210 2023-07-06T03:46:21.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.465+0000 7fe5021ff700 1 -- 172.21.15.97:0/2134562482 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc0c9920 con 0x7fe4fc0d2210 2023-07-06T03:46:21.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.466+0000 7fe5021ff700 1 -- 172.21.15.97:0/2134562482 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc06ba30 con 0x7fe4fc0d2210 2023-07-06T03:46:21.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.467+0000 7fe4f8ff9700 1 -- 172.21.15.97:0/2134562482 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe4e402d450 con 0x7fe4fc0d2210 2023-07-06T03:46:21.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.468+0000 7fe4f8ff9700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4dc06ace0 0x7fe4dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.468+0000 7fe4f8ff9700 1 -- 172.21.15.97:0/2134562482 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4e408a5b0 con 0x7fe4fc0d2210 2023-07-06T03:46:21.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.468+0000 7fe4fb7fe700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4dc06ace0 0x7fe4dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.469+0000 7fe4fb7fe700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4dc06ace0 0x7fe4dc06d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0011e0 tx=0x7fe4ec007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:21.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.472+0000 7fe4f8ff9700 1 -- 172.21.15.97:0/2134562482 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe4e4055650 con 0x7fe4fc0d2210 2023-07-06T03:46:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.665+0000 7fe5021ff700 1 -- 172.21.15.97:0/2134562482 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4fc04e340 con 0x7fe4fc0d2210 2023-07-06T03:46:21.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.666+0000 7fe4f8ff9700 1 -- 172.21.15.97:0/2134562482 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4e4058c70 con 0x7fe4fc0d2210 2023-07-06T03:46:21.668 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:46:21.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 -- 172.21.15.97:0/2134562482 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4dc06ace0 msgr2=0x7fe4dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4dc06ace0 0x7fe4dc06d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0011e0 tx=0x7fe4ec007be0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 -- 172.21.15.97:0/2134562482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4fc0d2210 msgr2=0x7fe4fc0c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4fc0d2210 0x7fe4fc0c6da0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4014040 tx=0x7fe4e4004750 comp rx=0 tx=0).stop 2023-07-06T03:46:21.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 -- 172.21.15.97:0/2134562482 shutdown_connections 2023-07-06T03:46:21.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4dc06ace0 0x7fe4dc06d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4fc15c610 0x7fe4fc1f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4fc06b360 0x7fe4fc0c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 --2- 172.21.15.97:0/2134562482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4fc0d2210 0x7fe4fc0c6da0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 -- 172.21.15.97:0/2134562482 >> 172.21.15.97:0/2134562482 conn(0x7fe4fc06a6c0 msgr2=0x7fe4fc0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:21.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 -- 172.21.15.97:0/2134562482 shutdown_connections 2023-07-06T03:46:21.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.668+0000 7fe5021ff700 1 -- 172.21.15.97:0/2134562482 wait complete. 2023-07-06T03:46:21.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:46:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.884+0000 7f3913bff700 1 Processor -- start 2023-07-06T03:46:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.884+0000 7f3913bff700 1 -- start start 2023-07-06T03:46:21.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.885+0000 7f3913bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f390c06b360 0x7f390c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.885+0000 7f3913bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f390c0d2210 0x7f390c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.885+0000 7f3913bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c0cff90 con 0x7f390c15c610 2023-07-06T03:46:21.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.885+0000 7f3913bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c0d0780 con 0x7f390c0d2210 2023-07-06T03:46:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.885+0000 7f3913bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c15c9e0 con 0x7f390c06b360 2023-07-06T03:46:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f391219c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f390c0d2210 0x7f390c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f391219c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f390c0d2210 0x7f390c0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45248/0 (socket says 172.21.15.97:45248) 2023-07-06T03:46:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f391219c700 1 -- 172.21.15.97:0/4177164795 learned_addr learned my addr 172.21.15.97:0/4177164795 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:21.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f391119a700 1 --2- 172.21.15.97:0/4177164795 >> v2:172.21.15.146:3300/0 conn(0x7f390c06b360 0x7f390c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f391219c700 1 -- 172.21.15.97:0/4177164795 >> v2:172.21.15.146:3300/0 conn(0x7f390c06b360 msgr2=0x7f390c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:21.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f391219c700 1 --2- 172.21.15.97:0/4177164795 >> v2:172.21.15.146:3300/0 conn(0x7f390c06b360 0x7f390c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f391219c700 1 -- 172.21.15.97:0/4177164795 >> v1:172.21.15.146:6789/0 conn(0x7f390c15c610 legacy=0x7f390c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f391219c700 1 -- 172.21.15.97:0/4177164795 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0b5650 con 0x7f390c0d2210 2023-07-06T03:46:21.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f391219c700 1 --2- 172.21.15.97:0/4177164795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0d1310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f390c06a590 tx=0x7f390400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fedb39e1da0cc31 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:21.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f3910999700 1 -- 172.21.15.97:0/4177164795 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390400e070 con 0x7f390c0d2210 2023-07-06T03:46:21.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f3910999700 1 -- 172.21.15.97:0/4177164795 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390400b8d0 con 0x7f390c0d2210 2023-07-06T03:46:21.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.886+0000 7f3910999700 1 -- 172.21.15.97:0/4177164795 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390400bc80 con 0x7f390c0d2210 2023-07-06T03:46:21.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.887+0000 7f3913bff700 1 -- 172.21.15.97:0/4177164795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 msgr2=0x7f390c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.887+0000 7f3913bff700 1 --2- 172.21.15.97:0/4177164795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0d1310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f390c06a590 tx=0x7f390400b330 comp rx=0 tx=0).stop 2023-07-06T03:46:21.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.887+0000 7f3913bff700 1 -- 172.21.15.97:0/4177164795 shutdown_connections 2023-07-06T03:46:21.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.887+0000 7f3913bff700 1 --2- 172.21.15.97:0/4177164795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0d1310 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.887+0000 7f3913bff700 1 --2- 172.21.15.97:0/4177164795 >> v2:172.21.15.146:3300/0 conn(0x7f390c06b360 0x7f390c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.887+0000 7f3913bff700 1 -- 172.21.15.97:0/4177164795 >> 172.21.15.97:0/4177164795 conn(0x7f390c06a6c0 msgr2=0x7f390c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:21.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.887+0000 7f3913bff700 1 -- 172.21.15.97:0/4177164795 shutdown_connections 2023-07-06T03:46:21.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.887+0000 7f3913bff700 1 -- 172.21.15.97:0/4177164795 wait complete. 2023-07-06T03:46:21.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.889+0000 7f3913bff700 1 Processor -- start 2023-07-06T03:46:21.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.889+0000 7f3913bff700 1 -- start start 2023-07-06T03:46:21.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f3913bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f390c06b360 0x7f390c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f3913bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f3913bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c15c610 0x7f390c0b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f3913bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c1545a0 con 0x7f390c0d2210 2023-07-06T03:46:21.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f3913bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c1f1fd0 con 0x7f390c15c610 2023-07-06T03:46:21.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f3913bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c15ef30 con 0x7f390c06b360 2023-07-06T03:46:21.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f391119a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f391119a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45252/0 (socket says 172.21.15.97:45252) 2023-07-06T03:46:21.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f391119a700 1 -- 172.21.15.97:0/3790960689 learned_addr learned my addr 172.21.15.97:0/3790960689 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:21.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f391199b700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f390c06b360 0x7f390c1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f391219c700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c15c610 0x7f390c0b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.890+0000 7f391119a700 1 -- 172.21.15.97:0/3790960689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f390c06b360 msgr2=0x7f390c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f391119a700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f390c06b360 0x7f390c1470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f391119a700 1 -- 172.21.15.97:0/3790960689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c15c610 msgr2=0x7f390c0b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:21.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f391119a700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c15c610 0x7f390c0b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:21.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f391119a700 1 -- 172.21.15.97:0/3790960689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3904009b30 con 0x7f390c0d2210 2023-07-06T03:46:21.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f391219c700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c15c610 0x7f390c0b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:21.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f391199b700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f390c06b360 0x7f390c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:21.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f391119a700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0b6660 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00e9e0 tx=0x7f38fc00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:21.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f39027fc700 1 -- 172.21.15.97:0/3790960689 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc009710 con 0x7f390c0d2210 2023-07-06T03:46:21.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f39027fc700 1 -- 172.21.15.97:0/3790960689 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc00c850 con 0x7f390c0d2210 2023-07-06T03:46:21.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f3913bff700 1 -- 172.21.15.97:0/3790960689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c15cb20 con 0x7f390c0d2210 2023-07-06T03:46:21.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f39027fc700 1 -- 172.21.15.97:0/3790960689 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00cbc0 con 0x7f390c0d2210 2023-07-06T03:46:21.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.891+0000 7f3913bff700 1 -- 172.21.15.97:0/3790960689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c15d0e0 con 0x7f390c0d2210 2023-07-06T03:46:21.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.892+0000 7f3913bff700 1 -- 172.21.15.97:0/3790960689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c04e340 con 0x7f390c0d2210 2023-07-06T03:46:21.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.893+0000 7f39027fc700 1 -- 172.21.15.97:0/3790960689 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38fc00cd20 con 0x7f390c0d2210 2023-07-06T03:46:21.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.894+0000 7f39027fc700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 0x7f38f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:21.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.894+0000 7f39027fc700 1 -- 172.21.15.97:0/3790960689 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38fc089630 con 0x7f390c0d2210 2023-07-06T03:46:21.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.894+0000 7f391199b700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 0x7f38f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:21.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.895+0000 7f391199b700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 0x7f38f006d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3908000f60 tx=0x7f3908009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:21.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:21.899+0000 7f39027fc700 1 -- 172.21.15.97:0/3790960689 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f38fc0580d0 con 0x7f390c0d2210 2023-07-06T03:46:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.094+0000 7f3913bff700 1 -- 172.21.15.97:0/3790960689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f390c0b8a50 con 0x7f390c0d2210 2023-07-06T03:46:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.095+0000 7f39027fc700 1 -- 172.21.15.97:0/3790960689 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f38fc057c60 con 0x7f390c0d2210 2023-07-06T03:46:22.097 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:46:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 -- 172.21.15.97:0/3790960689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 msgr2=0x7f38f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 0x7f38f006d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3908000f60 tx=0x7f3908009040 comp rx=0 tx=0).stop 2023-07-06T03:46:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 -- 172.21.15.97:0/3790960689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 msgr2=0x7f390c0b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0b6660 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00e9e0 tx=0x7f38fc00eda0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 -- 172.21.15.97:0/3790960689 shutdown_connections 2023-07-06T03:46:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f38f006ace0 0x7f38f006d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f390c15c610 0x7f390c0b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f390c06b360 0x7f390c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 --2- 172.21.15.97:0/3790960689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390c0d2210 0x7f390c0b6660 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 -- 172.21.15.97:0/3790960689 >> 172.21.15.97:0/3790960689 conn(0x7f390c06a6c0 msgr2=0x7f390c0d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:22.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 -- 172.21.15.97:0/3790960689 shutdown_connections 2023-07-06T03:46:22.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.097+0000 7f3913bff700 1 -- 172.21.15.97:0/3790960689 wait complete. 2023-07-06T03:46:22.117 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:46:22.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.312+0000 7efff95ff700 1 Processor -- start 2023-07-06T03:46:22.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.312+0000 7efff95ff700 1 -- start start 2023-07-06T03:46:22.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.314+0000 7efff95ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff406cab0 0x7efff40ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.314+0000 7efff95ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff40ce560 con 0x7efff4158b20 2023-07-06T03:46:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.314+0000 7efff95ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff40adcf0 con 0x7efff40b0560 2023-07-06T03:46:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.314+0000 7efff95ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff40ce820 con 0x7efff406cab0 2023-07-06T03:46:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.314+0000 7efff2ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efff4158b20 0x7efff40cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54502/0 (socket says 172.21.15.97:54502) 2023-07-06T03:46:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.314+0000 7efff2ffd700 1 -- 172.21.15.97:0/745332665 learned_addr learned my addr 172.21.15.97:0/745332665 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.314+0000 7efff27fc700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff406cab0 0x7efff40ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.314+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708904489 0 0) 0x7efff40ce560 con 0x7efff4158b20 2023-07-06T03:46:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.314+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effdc0057c0 con 0x7efff4158b20 2023-07-06T03:46:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180717590 0 0) 0x7efff40adcf0 con 0x7efff40b0560 2023-07-06T03:46:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff40ce560 con 0x7efff40b0560 2023-07-06T03:46:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 57251759 0 0) 0x7effdc0057c0 con 0x7efff4158b20 2023-07-06T03:46:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff406cab0 msgr2=0x7efff40ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff406cab0 0x7efff40ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 >> v1:172.21.15.146:6789/0 conn(0x7efff40b0560 legacy=0x7efff406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff40adfd0 con 0x7efff4158b20 2023-07-06T03:46:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7effe4001670 con 0x7efff4158b20 2023-07-06T03:46:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 >> v1:172.21.15.97:6789/0 conn(0x7efff4158b20 legacy=0x7efff40cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff406cab0 0x7effdc005dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 0x7effdc006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7effdc006fa0 0x7effdc007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff40ce820 con 0x7effdc006330 2023-07-06T03:46:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc00dcd0 con 0x7effdc006fa0 2023-07-06T03:46:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc00de90 con 0x7efff406cab0 2023-07-06T03:46:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff27fc700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 0x7effdc006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff2ffd700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff406cab0 0x7effdc005dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff37fe700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7effdc006fa0 0x7effdc007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff27fc700 1 -- 172.21.15.97:0/745332665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff406cab0 msgr2=0x7effdc005dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff27fc700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff406cab0 0x7effdc005dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff27fc700 1 -- 172.21.15.97:0/745332665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7effdc006fa0 msgr2=0x7effdc007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff27fc700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7effdc006fa0 0x7effdc007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.315+0000 7efff27fc700 1 -- 172.21.15.97:0/745332665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc0057c0 con 0x7effdc006330 2023-07-06T03:46:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.316+0000 7efff27fc700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 0x7effdc006a60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7effe8009b30 tx=0x7effdc007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed2e38013649de server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.316+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe800e6c0 con 0x7effdc006330 2023-07-06T03:46:22.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.316+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe800e820 con 0x7effdc006330 2023-07-06T03:46:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.316+0000 7efff1ffb700 1 -- 172.21.15.97:0/745332665 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe800eb90 con 0x7effdc006330 2023-07-06T03:46:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.316+0000 7efff95ff700 1 -- 172.21.15.97:0/745332665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 msgr2=0x7effdc006a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.316+0000 7efff95ff700 1 --2- 172.21.15.97:0/745332665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 0x7effdc006a60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7effe8009b30 tx=0x7effdc007a90 comp rx=0 tx=0).stop 2023-07-06T03:46:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.316+0000 7efff37fe700 1 -- 172.21.15.97:0/745332665 reap_dead start 2023-07-06T03:46:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.317+0000 7efff95ff700 1 -- 172.21.15.97:0/745332665 shutdown_connections 2023-07-06T03:46:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.317+0000 7efff95ff700 1 -- 172.21.15.97:0/745332665 >> 172.21.15.97:0/745332665 conn(0x7efff406a060 msgr2=0x7efff406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:22.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.317+0000 7efff95ff700 1 -- 172.21.15.97:0/745332665 shutdown_connections 2023-07-06T03:46:22.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.317+0000 7efff95ff700 1 -- 172.21.15.97:0/745332665 wait complete. 2023-07-06T03:46:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.318+0000 7efff95ff700 1 Processor -- start 2023-07-06T03:46:22.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.318+0000 7efff95ff700 1 -- start start 2023-07-06T03:46:22.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.319+0000 7efff95ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 0x7efff4155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.319+0000 7efff95ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff414b570 0x7efff4153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.319+0000 7efff95ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff4154fe0 0x7efff4159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.319+0000 7efff95ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff4154440 con 0x7effdc006330 2023-07-06T03:46:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.319+0000 7efff95ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff41539b0 con 0x7efff4154fe0 2023-07-06T03:46:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.319+0000 7efff95ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff414d740 con 0x7efff414b570 2023-07-06T03:46:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 0x7efff4155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 0x7efff4155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45262/0 (socket says 172.21.15.97:45262) 2023-07-06T03:46:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff2ffd700 1 -- 172.21.15.97:0/3606397858 learned_addr learned my addr 172.21.15.97:0/3606397858 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff27fc700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff414b570 0x7efff4153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff37fe700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff4154fe0 0x7efff4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff2ffd700 1 -- 172.21.15.97:0/3606397858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff414b570 msgr2=0x7efff4153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff2ffd700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff414b570 0x7efff4153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff2ffd700 1 -- 172.21.15.97:0/3606397858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff4154fe0 msgr2=0x7efff4159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff2ffd700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff4154fe0 0x7efff4159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff2ffd700 1 -- 172.21.15.97:0/3606397858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff4154b50 con 0x7effdc006330 2023-07-06T03:46:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff37fe700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff4154fe0 0x7efff4159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff2ffd700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 0x7efff4155f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7effe4007340 tx=0x7effe4007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7effdb7fe700 1 -- 172.21.15.97:0/3606397858 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4001b10 con 0x7effdc006330 2023-07-06T03:46:22.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7effdb7fe700 1 -- 172.21.15.97:0/3606397858 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe400f660 con 0x7effdc006330 2023-07-06T03:46:22.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.320+0000 7efff95ff700 1 -- 172.21.15.97:0/3606397858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff414b9e0 con 0x7effdc006330 2023-07-06T03:46:22.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.321+0000 7effdb7fe700 1 -- 172.21.15.97:0/3606397858 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe400fa30 con 0x7effdc006330 2023-07-06T03:46:22.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.321+0000 7efff95ff700 1 -- 172.21.15.97:0/3606397858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff4153e80 con 0x7effdc006330 2023-07-06T03:46:22.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.321+0000 7efff95ff700 1 -- 172.21.15.97:0/3606397858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff41548b0 con 0x7effdc006330 2023-07-06T03:46:22.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.322+0000 7effdb7fe700 1 -- 172.21.15.97:0/3606397858 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effe400fb90 con 0x7effdc006330 2023-07-06T03:46:22.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.323+0000 7effdb7fe700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7effe006ace0 0x7effe006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.323+0000 7effdb7fe700 1 -- 172.21.15.97:0/3606397858 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effe4089360 con 0x7effdc006330 2023-07-06T03:46:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.324+0000 7efff27fc700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7effe006ace0 0x7effe006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.324+0000 7efff27fc700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7effe006ace0 0x7effe006d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7effe80090b0 tx=0x7effe8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:22.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.328+0000 7effdb7fe700 1 -- 172.21.15.97:0/3606397858 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7effe40544a0 con 0x7effdc006330 2023-07-06T03:46:22.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.522+0000 7efff95ff700 1 -- 172.21.15.97:0/3606397858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efff404e340 con 0x7effdc006330 2023-07-06T03:46:22.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.523+0000 7effdb7fe700 1 -- 172.21.15.97:0/3606397858 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7effe4057ac0 con 0x7effdc006330 2023-07-06T03:46:22.525 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:46:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 -- 172.21.15.97:0/3606397858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7effe006ace0 msgr2=0x7effe006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7effe006ace0 0x7effe006d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7effe80090b0 tx=0x7effe8008030 comp rx=0 tx=0).stop 2023-07-06T03:46:22.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 -- 172.21.15.97:0/3606397858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 msgr2=0x7efff4155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 0x7efff4155f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7effe4007340 tx=0x7effe4007650 comp rx=0 tx=0).stop 2023-07-06T03:46:22.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 -- 172.21.15.97:0/3606397858 shutdown_connections 2023-07-06T03:46:22.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7effe006ace0 0x7effe006d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff4154fe0 0x7efff4159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effdc006330 0x7efff4155f30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 --2- 172.21.15.97:0/3606397858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efff414b570 0x7efff4153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 -- 172.21.15.97:0/3606397858 >> 172.21.15.97:0/3606397858 conn(0x7efff406a060 msgr2=0x7efff40cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:22.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 -- 172.21.15.97:0/3606397858 shutdown_connections 2023-07-06T03:46:22.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.525+0000 7efff95ff700 1 -- 172.21.15.97:0/3606397858 wait complete. 2023-07-06T03:46:22.545 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:46:22.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.743+0000 7f7f65bff700 1 Processor -- start 2023-07-06T03:46:22.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.743+0000 7f7f65bff700 1 -- start start 2023-07-06T03:46:22.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f65bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7f6015c610 0x7f7f600b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f65bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7f600d2210 0x7f7f600d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f65bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f600cff90 con 0x7f7f6006b360 2023-07-06T03:46:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f65bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f600d0780 con 0x7f7f600d2210 2023-07-06T03:46:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f65bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6015c9e0 con 0x7f7f6015c610 2023-07-06T03:46:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f5ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7f600d2210 0x7f7f600d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f5ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7f600d2210 0x7f7f600d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45264/0 (socket says 172.21.15.97:45264) 2023-07-06T03:46:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f5ffff700 1 -- 172.21.15.97:0/177005631 learned_addr learned my addr 172.21.15.97:0/177005631 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:22.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f5f7fe700 1 --2- 172.21.15.97:0/177005631 >> v2:172.21.15.146:3300/0 conn(0x7f7f6015c610 0x7f7f600b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f5ffff700 1 -- 172.21.15.97:0/177005631 >> v2:172.21.15.146:3300/0 conn(0x7f7f6015c610 msgr2=0x7f7f600b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:22.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f5ffff700 1 --2- 172.21.15.97:0/177005631 >> v2:172.21.15.146:3300/0 conn(0x7f7f6015c610 0x7f7f600b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f5ffff700 1 -- 172.21.15.97:0/177005631 >> v1:172.21.15.146:6789/0 conn(0x7f7f6006b360 legacy=0x7f7f600b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.745+0000 7f7f5ffff700 1 -- 172.21.15.97:0/177005631 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f600b5650 con 0x7f7f600d2210 2023-07-06T03:46:22.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.746+0000 7f7f5ffff700 1 --2- 172.21.15.97:0/177005631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f600d2210 0x7f7f600d1310 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7f6008b000 tx=0x7f7f5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a395fc7b09acd6e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.746+0000 7f7f5e7fc700 1 -- 172.21.15.97:0/177005631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800e070 con 0x7f7f600d2210 2023-07-06T03:46:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.746+0000 7f7f5e7fc700 1 -- 172.21.15.97:0/177005631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5800b8d0 con 0x7f7f600d2210 2023-07-06T03:46:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.746+0000 7f7f5e7fc700 1 -- 172.21.15.97:0/177005631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800bc80 con 0x7f7f600d2210 2023-07-06T03:46:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.746+0000 7f7f65bff700 1 -- 172.21.15.97:0/177005631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f600d2210 msgr2=0x7f7f600d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.746+0000 7f7f65bff700 1 --2- 172.21.15.97:0/177005631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f600d2210 0x7f7f600d1310 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7f6008b000 tx=0x7f7f5800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.746+0000 7f7f65bff700 1 -- 172.21.15.97:0/177005631 shutdown_connections 2023-07-06T03:46:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.746+0000 7f7f65bff700 1 --2- 172.21.15.97:0/177005631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f600d2210 0x7f7f600d1310 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.747+0000 7f7f65bff700 1 --2- 172.21.15.97:0/177005631 >> v2:172.21.15.146:3300/0 conn(0x7f7f6015c610 0x7f7f600b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.747+0000 7f7f65bff700 1 -- 172.21.15.97:0/177005631 >> 172.21.15.97:0/177005631 conn(0x7f7f6006a6c0 msgr2=0x7f7f6006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.747+0000 7f7f65bff700 1 -- 172.21.15.97:0/177005631 shutdown_connections 2023-07-06T03:46:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.747+0000 7f7f65bff700 1 -- 172.21.15.97:0/177005631 wait complete. 2023-07-06T03:46:22.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.748+0000 7f7f65bff700 1 Processor -- start 2023-07-06T03:46:22.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.748+0000 7f7f65bff700 1 -- start start 2023-07-06T03:46:22.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.749+0000 7f7f65bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6006b360 0x7f7f6014b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.749+0000 7f7f65bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f600d2210 0x7f7f600c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.749+0000 7f7f65bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f6015c610 0x7f7f600b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.749+0000 7f7f65bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600c9510 con 0x7f7f6006b360 2023-07-06T03:46:22.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.749+0000 7f7f65bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600c8890 con 0x7f7f6015c610 2023-07-06T03:46:22.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.749+0000 7f7f65bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600c8b70 con 0x7f7f600d2210 2023-07-06T03:46:22.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6006b360 0x7f7f6014b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6006b360 0x7f7f6014b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45272/0 (socket says 172.21.15.97:45272) 2023-07-06T03:46:22.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5f7fe700 1 -- 172.21.15.97:0/41276739 learned_addr learned my addr 172.21.15.97:0/41276739 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:22.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5ffff700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f6015c610 0x7f7f600b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5effd700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f600d2210 0x7f7f600c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5f7fe700 1 -- 172.21.15.97:0/41276739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f600d2210 msgr2=0x7f7f600c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5f7fe700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f600d2210 0x7f7f600c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5f7fe700 1 -- 172.21.15.97:0/41276739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f6015c610 msgr2=0x7f7f600b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5f7fe700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f6015c610 0x7f7f600b6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5f7fe700 1 -- 172.21.15.97:0/41276739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f58009b30 con 0x7f7f6006b360 2023-07-06T03:46:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5ffff700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f6015c610 0x7f7f600b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5effd700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f600d2210 0x7f7f600c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f5f7fe700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6006b360 0x7f7f6014b0c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000e870 tx=0x7f7f5000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.750+0000 7f7f47fff700 1 -- 172.21.15.97:0/41276739 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50009590 con 0x7f7f6006b360 2023-07-06T03:46:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.751+0000 7f7f47fff700 1 -- 172.21.15.97:0/41276739 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5000ee70 con 0x7f7f6006b360 2023-07-06T03:46:22.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.751+0000 7f7f47fff700 1 -- 172.21.15.97:0/41276739 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5000ca50 con 0x7f7f6006b360 2023-07-06T03:46:22.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.751+0000 7f7f65bff700 1 -- 172.21.15.97:0/41276739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f600c8e50 con 0x7f7f6006b360 2023-07-06T03:46:22.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.751+0000 7f7f65bff700 1 -- 172.21.15.97:0/41276739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f600c9930 con 0x7f7f6006b360 2023-07-06T03:46:22.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.752+0000 7f7f65bff700 1 -- 172.21.15.97:0/41276739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6004e340 con 0x7f7f6006b360 2023-07-06T03:46:22.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.753+0000 7f7f47fff700 1 -- 172.21.15.97:0/41276739 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f5000cbb0 con 0x7f7f6006b360 2023-07-06T03:46:22.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.753+0000 7f7f47fff700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f4c06ace0 0x7f7f4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:22.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.754+0000 7f7f47fff700 1 -- 172.21.15.97:0/41276739 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7f500893f0 con 0x7f7f6006b360 2023-07-06T03:46:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.754+0000 7f7f5effd700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f4c06ace0 0x7f7f4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.754+0000 7f7f5effd700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f4c06ace0 0x7f7f4c06d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7f60159a10 tx=0x7f7f54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.758+0000 7f7f47fff700 1 -- 172.21.15.97:0/41276739 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7f50057f20 con 0x7f7f6006b360 2023-07-06T03:46:22.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.957+0000 7f7f65bff700 1 -- 172.21.15.97:0/41276739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f60159a40 con 0x7f7f6006b360 2023-07-06T03:46:22.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.958+0000 7f7f47fff700 1 -- 172.21.15.97:0/41276739 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f5000ce60 con 0x7f7f6006b360 2023-07-06T03:46:22.960 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:46:22.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 -- 172.21.15.97:0/41276739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f4c06ace0 msgr2=0x7f7f4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f4c06ace0 0x7f7f4c06d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7f60159a10 tx=0x7f7f54009040 comp rx=0 tx=0).stop 2023-07-06T03:46:22.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 -- 172.21.15.97:0/41276739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6006b360 msgr2=0x7f7f6014b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:22.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6006b360 0x7f7f6014b0c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000e870 tx=0x7f7f5000ec30 comp rx=0 tx=0).stop 2023-07-06T03:46:22.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 -- 172.21.15.97:0/41276739 shutdown_connections 2023-07-06T03:46:22.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f4c06ace0 0x7f7f4c06d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f6015c610 0x7f7f600b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6006b360 0x7f7f6014b0c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 --2- 172.21.15.97:0/41276739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f600d2210 0x7f7f600c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:22.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 -- 172.21.15.97:0/41276739 >> 172.21.15.97:0/41276739 conn(0x7f7f6006a6c0 msgr2=0x7f7f600b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:22.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 -- 172.21.15.97:0/41276739 shutdown_connections 2023-07-06T03:46:22.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:22.960+0000 7f7f65bff700 1 -- 172.21.15.97:0/41276739 wait complete. 2023-07-06T03:46:22.981 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:46:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.174+0000 7f00355ff700 1 Processor -- start 2023-07-06T03:46:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.174+0000 7f00355ff700 1 -- start start 2023-07-06T03:46:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.176+0000 7f00355ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f003015c520 0x7f00300cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.176+0000 7f00355ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0030160d00 0x7f00300b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.176+0000 7f00355ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00300cdae0 con 0x7f0030160180 2023-07-06T03:46:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.176+0000 7f00355ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00300cd8d0 con 0x7f003015c520 2023-07-06T03:46:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.176+0000 7f00355ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00300b6640 con 0x7f0030160d00 2023-07-06T03:46:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.176+0000 7f0027fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0030160180 0x7f00301605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54510/0 (socket says 172.21.15.97:54510) 2023-07-06T03:46:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.176+0000 7f0027fff700 1 -- 172.21.15.97:0/1966283638 learned_addr learned my addr 172.21.15.97:0/1966283638 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.176+0000 7f002effd700 1 --2- 172.21.15.97:0/1966283638 >> v2:172.21.15.146:3300/0 conn(0x7f003015c520 0x7f00300cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.176+0000 7f002f7fe700 1 --2- 172.21.15.97:0/1966283638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0030160d00 0x7f00300b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.176+0000 7f002e7fc700 1 -- 172.21.15.97:0/1966283638 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763310099 0 0) 0x7f00300cdae0 con 0x7f0030160180 2023-07-06T03:46:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.176+0000 7f002e7fc700 1 -- 172.21.15.97:0/1966283638 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f001c0035b0 con 0x7f0030160180 2023-07-06T03:46:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.177+0000 7f002effd700 1 -- 172.21.15.97:0/1966283638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0030160d00 msgr2=0x7f00300b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.177+0000 7f002effd700 1 --2- 172.21.15.97:0/1966283638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0030160d00 0x7f00300b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.177+0000 7f002e7fc700 1 -- 172.21.15.97:0/1966283638 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4290199657 0 0) 0x7f001c0035b0 con 0x7f0030160180 2023-07-06T03:46:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.177+0000 7f002effd700 1 -- 172.21.15.97:0/1966283638 >> v1:172.21.15.97:6789/0 conn(0x7f0030160180 legacy=0x7f00301605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.177+0000 7f002effd700 1 -- 172.21.15.97:0/1966283638 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00300b6920 con 0x7f003015c520 2023-07-06T03:46:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.177+0000 7f002f7fe700 1 --2- 172.21.15.97:0/1966283638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0030160d00 0x7f00300b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:46:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.177+0000 7f002effd700 1 --2- 172.21.15.97:0/1966283638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f003015c520 0x7f00300cd390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0020009840 tx=0x7f0020009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5dd2fd7216e9711 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.177+0000 7f002e7fc700 1 -- 172.21.15.97:0/1966283638 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018001670 con 0x7f003015c520 2023-07-06T03:46:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.177+0000 7f002e7fc700 1 -- 172.21.15.97:0/1966283638 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0020004030 con 0x7f003015c520 2023-07-06T03:46:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.177+0000 7f002e7fc700 1 -- 172.21.15.97:0/1966283638 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00200043a0 con 0x7f003015c520 2023-07-06T03:46:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.178+0000 7f00355ff700 1 -- 172.21.15.97:0/1966283638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f003015c520 msgr2=0x7f00300cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.178+0000 7f00355ff700 1 --2- 172.21.15.97:0/1966283638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f003015c520 0x7f00300cd390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0020009840 tx=0x7f0020009b50 comp rx=0 tx=0).stop 2023-07-06T03:46:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.178+0000 7f00355ff700 1 -- 172.21.15.97:0/1966283638 shutdown_connections 2023-07-06T03:46:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.178+0000 7f00355ff700 1 --2- 172.21.15.97:0/1966283638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f003015c520 0x7f00300cd390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.178+0000 7f00355ff700 1 --2- 172.21.15.97:0/1966283638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0030160d00 0x7f00300b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.178+0000 7f00355ff700 1 -- 172.21.15.97:0/1966283638 >> 172.21.15.97:0/1966283638 conn(0x7f0030071790 msgr2=0x7f0030071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:23.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.178+0000 7f00355ff700 1 -- 172.21.15.97:0/1966283638 shutdown_connections 2023-07-06T03:46:23.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.178+0000 7f00355ff700 1 -- 172.21.15.97:0/1966283638 wait complete. 2023-07-06T03:46:23.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.180+0000 7f00355ff700 1 Processor -- start 2023-07-06T03:46:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.180+0000 7f00355ff700 1 -- start start 2023-07-06T03:46:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f00355ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f003015c520 0x7f003014d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f00355ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0030160180 0x7f00300d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f00355ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0030160d00 0x7f00300d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f00355ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003014da30 con 0x7f0030160180 2023-07-06T03:46:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f00355ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00300d17b0 con 0x7f0030160d00 2023-07-06T03:46:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f00355ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00300d1a90 con 0x7f003015c520 2023-07-06T03:46:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f0027fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0030160180 0x7f00300d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f0027fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0030160180 0x7f00300d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45288/0 (socket says 172.21.15.97:45288) 2023-07-06T03:46:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f0027fff700 1 -- 172.21.15.97:0/2661457176 learned_addr learned my addr 172.21.15.97:0/2661457176 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f002effd700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f003015c520 0x7f003014d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f002f7fe700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0030160d00 0x7f00300d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f0027fff700 1 -- 172.21.15.97:0/2661457176 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f003015c520 msgr2=0x7f003014d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f0027fff700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f003015c520 0x7f003014d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f0027fff700 1 -- 172.21.15.97:0/2661457176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0030160d00 msgr2=0x7f00300d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f0027fff700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0030160d00 0x7f00300d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.181+0000 7f0027fff700 1 -- 172.21.15.97:0/2661457176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0028009a70 con 0x7f0030160180 2023-07-06T03:46:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.182+0000 7f0027fff700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0030160180 0x7f00300d0c40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f001800c520 tx=0x7f001800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.182+0000 7f00277fe700 1 -- 172.21.15.97:0/2661457176 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018004c80 con 0x7f0030160180 2023-07-06T03:46:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.182+0000 7f00277fe700 1 -- 172.21.15.97:0/2661457176 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0018004e20 con 0x7f0030160180 2023-07-06T03:46:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.182+0000 7f00355ff700 1 -- 172.21.15.97:0/2661457176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00200094a0 con 0x7f0030160180 2023-07-06T03:46:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.182+0000 7f00277fe700 1 -- 172.21.15.97:0/2661457176 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018007310 con 0x7f0030160180 2023-07-06T03:46:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.182+0000 7f00355ff700 1 -- 172.21.15.97:0/2661457176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00300ca190 con 0x7f0030160180 2023-07-06T03:46:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.183+0000 7f00355ff700 1 -- 172.21.15.97:0/2661457176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00300ca3b0 con 0x7f0030160180 2023-07-06T03:46:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.184+0000 7f00277fe700 1 -- 172.21.15.97:0/2661457176 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0018007470 con 0x7f0030160180 2023-07-06T03:46:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.185+0000 7f00277fe700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f001006ace0 0x7f001006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.185+0000 7f00277fe700 1 -- 172.21.15.97:0/2661457176 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00180892e0 con 0x7f0030160180 2023-07-06T03:46:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.185+0000 7f002effd700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f001006ace0 0x7f001006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.186+0000 7f002effd700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f001006ace0 0x7f001006d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0020000c00 tx=0x7f0020013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:23.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.189+0000 7f00277fe700 1 -- 172.21.15.97:0/2661457176 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f00180542f0 con 0x7f0030160180 2023-07-06T03:46:23.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.425+0000 7f00355ff700 1 -- 172.21.15.97:0/2661457176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f003004e340 con 0x7f0030160180 2023-07-06T03:46:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.425+0000 7f00277fe700 1 -- 172.21.15.97:0/2661457176 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4665 (secure 0 0 0) 0x7f0018057910 con 0x7f0030160180 2023-07-06T03:46:23.427 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:23.428 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/4255451313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4255451313},{"type":"v1","addr":"172.21.15.97:6836","nonce":4255451313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/370322164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":370322164},{"type":"v1","addr":"172.21.15.97:6837","nonce":370322164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3764820847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3764820847},{"type":"v1","addr":"172.21.15.146:6835","nonce":3764820847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:45:11.126281+0000","modified":"2023-07-06T03:45:14.685499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6833/186544257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":186544257},{"type":"v1","addr":"172.21.15.146:6833","nonce":186544257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T03:46:23.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 -- 172.21.15.97:0/2661457176 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f001006ace0 msgr2=0x7f001006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f001006ace0 0x7f001006d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0020000c00 tx=0x7f0020013040 comp rx=0 tx=0).stop 2023-07-06T03:46:23.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 -- 172.21.15.97:0/2661457176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0030160180 msgr2=0x7f00300d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0030160180 0x7f00300d0c40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f001800c520 tx=0x7f001800c8e0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 -- 172.21.15.97:0/2661457176 shutdown_connections 2023-07-06T03:46:23.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f001006ace0 0x7f001006d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0030160d00 0x7f00300d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f003015c520 0x7f003014d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 --2- 172.21.15.97:0/2661457176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0030160180 0x7f00300d0c40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 -- 172.21.15.97:0/2661457176 >> 172.21.15.97:0/2661457176 conn(0x7f0030071790 msgr2=0x7f0030071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:23.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 -- 172.21.15.97:0/2661457176 shutdown_connections 2023-07-06T03:46:23.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.428+0000 7f00355ff700 1 -- 172.21.15.97:0/2661457176 wait complete. 2023-07-06T03:46:23.432 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-07-06T03:46:23.448 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.641+0000 7fc226fff700 1 Processor -- start 2023-07-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.642+0000 7fc226fff700 1 -- start start 2023-07-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc226fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc2200b8410 0x7fc220167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:23.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc226fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2200cdae0 con 0x7fc22015c520 2023-07-06T03:46:23.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc226fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2200b6640 con 0x7fc2200cd390 2023-07-06T03:46:23.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc226fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2200b6920 con 0x7fc2200b8410 2023-07-06T03:46:23.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc224d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc22015c520 0x7fc220160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54514/0 (socket says 172.21.15.97:54514) 2023-07-06T03:46:23.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc224d9b700 1 -- 172.21.15.97:0/3295133700 learned_addr learned my addr 172.21.15.97:0/3295133700 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:23.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc22559c700 1 --2- 172.21.15.97:0/3295133700 >> v2:172.21.15.97:3300/0 conn(0x7fc2200b8410 0x7fc220167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:23.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc21f7fe700 1 -- 172.21.15.97:0/3295133700 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724428759 0 0) 0x7fc2200cdae0 con 0x7fc22015c520 2023-07-06T03:46:23.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc21f7fe700 1 -- 172.21.15.97:0/3295133700 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2100035b0 con 0x7fc22015c520 2023-07-06T03:46:23.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc22559c700 1 -- 172.21.15.97:0/3295133700 >> v1:172.21.15.146:6789/0 conn(0x7fc2200cd390 legacy=0x7fc220160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc22559c700 1 -- 172.21.15.97:0/3295133700 >> v1:172.21.15.97:6789/0 conn(0x7fc22015c520 legacy=0x7fc220160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.643+0000 7fc22559c700 1 -- 172.21.15.97:0/3295133700 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2200b6c00 con 0x7fc2200b8410 2023-07-06T03:46:23.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.644+0000 7fc22559c700 1 --2- 172.21.15.97:0/3295133700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2200b8410 0x7fc220167510 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc218009d90 tx=0x7fc21800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc9ae6a248eb59c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:23.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.644+0000 7fc21f7fe700 1 -- 172.21.15.97:0/3295133700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21800e070 con 0x7fc2200b8410 2023-07-06T03:46:23.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.644+0000 7fc21f7fe700 1 -- 172.21.15.97:0/3295133700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21800bb20 con 0x7fc2200b8410 2023-07-06T03:46:23.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.644+0000 7fc21f7fe700 1 -- 172.21.15.97:0/3295133700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218012430 con 0x7fc2200b8410 2023-07-06T03:46:23.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.644+0000 7fc226fff700 1 -- 172.21.15.97:0/3295133700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2200b8410 msgr2=0x7fc220167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.644+0000 7fc226fff700 1 --2- 172.21.15.97:0/3295133700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2200b8410 0x7fc220167510 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc218009d90 tx=0x7fc21800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:23.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.645+0000 7fc226fff700 1 -- 172.21.15.97:0/3295133700 shutdown_connections 2023-07-06T03:46:23.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.645+0000 7fc226fff700 1 --2- 172.21.15.97:0/3295133700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2200b8410 0x7fc220167510 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.645+0000 7fc226fff700 1 -- 172.21.15.97:0/3295133700 >> 172.21.15.97:0/3295133700 conn(0x7fc220071790 msgr2=0x7fc220071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:23.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.645+0000 7fc226fff700 1 -- 172.21.15.97:0/3295133700 shutdown_connections 2023-07-06T03:46:23.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.645+0000 7fc226fff700 1 -- 172.21.15.97:0/3295133700 wait complete. 2023-07-06T03:46:23.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.646+0000 7fc226fff700 1 Processor -- start 2023-07-06T03:46:23.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.647+0000 7fc226fff700 1 -- start start 2023-07-06T03:46:23.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.647+0000 7fc226fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2200b8410 0x7fc2201f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:23.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.647+0000 7fc226fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2200cd390 0x7fc2200d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:23.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc226fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc22015c520 0x7fc22014cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:23.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc226fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200cb5c0 con 0x7fc2200b8410 2023-07-06T03:46:23.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc226fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2201f2370 con 0x7fc2200cd390 2023-07-06T03:46:23.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc226fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2201f2650 con 0x7fc22015c520 2023-07-06T03:46:23.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc224d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2200b8410 0x7fc2201f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:23.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc224d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2200b8410 0x7fc2201f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45314/0 (socket says 172.21.15.97:45314) 2023-07-06T03:46:23.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc224d9b700 1 -- 172.21.15.97:0/3859878505 learned_addr learned my addr 172.21.15.97:0/3859878505 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:23.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc21ffff700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2200cd390 0x7fc2200d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:23.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc22559c700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc22015c520 0x7fc22014cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:23.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc224d9b700 1 -- 172.21.15.97:0/3859878505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc22015c520 msgr2=0x7fc22014cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc224d9b700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc22015c520 0x7fc22014cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc224d9b700 1 -- 172.21.15.97:0/3859878505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2200cd390 msgr2=0x7fc2200d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc224d9b700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2200cd390 0x7fc2200d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc224d9b700 1 -- 172.21.15.97:0/3859878505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc218009a70 con 0x7fc2200b8410 2023-07-06T03:46:23.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.648+0000 7fc224d9b700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2200b8410 0x7fc2201f2e50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc214001eb0 tx=0x7fc21400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:23.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.649+0000 7fc21d7fa700 1 -- 172.21.15.97:0/3859878505 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214005d70 con 0x7fc2200b8410 2023-07-06T03:46:23.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.649+0000 7fc21d7fa700 1 -- 172.21.15.97:0/3859878505 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2140070d0 con 0x7fc2200b8410 2023-07-06T03:46:23.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.649+0000 7fc21d7fa700 1 -- 172.21.15.97:0/3859878505 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214007460 con 0x7fc2200b8410 2023-07-06T03:46:23.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.649+0000 7fc226fff700 1 -- 172.21.15.97:0/3859878505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2201f2930 con 0x7fc2200b8410 2023-07-06T03:46:23.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.649+0000 7fc226fff700 1 -- 172.21.15.97:0/3859878505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2200cb8a0 con 0x7fc2200b8410 2023-07-06T03:46:23.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.650+0000 7fc226fff700 1 -- 172.21.15.97:0/3859878505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2200b6460 con 0x7fc2200b8410 2023-07-06T03:46:23.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.651+0000 7fc21d7fa700 1 -- 172.21.15.97:0/3859878505 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2140075c0 con 0x7fc2200b8410 2023-07-06T03:46:23.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.652+0000 7fc21d7fa700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc20406ace0 0x7fc20406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:23.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.653+0000 7fc21d7fa700 1 -- 172.21.15.97:0/3859878505 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc214088a40 con 0x7fc2200b8410 2023-07-06T03:46:23.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.653+0000 7fc21ffff700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc20406ace0 0x7fc20406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:23.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.653+0000 7fc21ffff700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc20406ace0 0x7fc20406d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc20c000f60 tx=0x7fc20c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:23.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.657+0000 7fc21d7fa700 1 -- 172.21.15.97:0/3859878505 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc21408ae60 con 0x7fc2200b8410 2023-07-06T03:46:23.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.860+0000 7fc226fff700 1 -- 172.21.15.97:0/3859878505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc22004e340 con 0x7fc2200b8410 2023-07-06T03:46:23.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.861+0000 7fc21d7fa700 1 -- 172.21.15.97:0/3859878505 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7fc214053af0 con 0x7fc2200b8410 2023-07-06T03:46:23.863 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:23.863 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:23.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.863+0000 7fc226fff700 1 -- 172.21.15.97:0/3859878505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc20406ace0 msgr2=0x7fc20406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.863+0000 7fc226fff700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc20406ace0 0x7fc20406d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc20c000f60 tx=0x7fc20c009040 comp rx=0 tx=0).stop 2023-07-06T03:46:23.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.863+0000 7fc226fff700 1 -- 172.21.15.97:0/3859878505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2200b8410 msgr2=0x7fc2201f2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.863+0000 7fc226fff700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2200b8410 0x7fc2201f2e50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc214001eb0 tx=0x7fc21400c860 comp rx=0 tx=0).stop 2023-07-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.863+0000 7fc226fff700 1 -- 172.21.15.97:0/3859878505 shutdown_connections 2023-07-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.863+0000 7fc226fff700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc20406ace0 0x7fc20406d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.863+0000 7fc226fff700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc22015c520 0x7fc22014cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.863+0000 7fc226fff700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2200b8410 0x7fc2201f2e50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.863+0000 7fc226fff700 1 --2- 172.21.15.97:0/3859878505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2200cd390 0x7fc2200d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.863+0000 7fc226fff700 1 -- 172.21.15.97:0/3859878505 >> 172.21.15.97:0/3859878505 conn(0x7fc220071790 msgr2=0x7fc2200c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:23.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.864+0000 7fc226fff700 1 -- 172.21.15.97:0/3859878505 shutdown_connections 2023-07-06T03:46:23.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:23.864+0000 7fc226fff700 1 -- 172.21.15.97:0/3859878505 wait complete. 2023-07-06T03:46:23.883 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T03:46:23.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:46:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.075+0000 7fbbff3ff700 1 Processor -- start 2023-07-06T03:46:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.076+0000 7fbbff3ff700 1 -- start start 2023-07-06T03:46:24.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbff3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbbf80cd390 0x7fbbf8160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:24.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbff3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbbf8160dd0 0x7fbbf80b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:24.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbff3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf80cdae0 con 0x7fbbf815c520 2023-07-06T03:46:24.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbff3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80b6640 con 0x7fbbf8160dd0 2023-07-06T03:46:24.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbff3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80b6920 con 0x7fbbf80cd390 2023-07-06T03:46:24.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbfd99c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbbf8160dd0 0x7fbbf80b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:24.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbfd99c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbbf8160dd0 0x7fbbf80b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45320/0 (socket says 172.21.15.97:45320) 2023-07-06T03:46:24.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbfd99c700 1 -- 172.21.15.97:0/2219065445 learned_addr learned my addr 172.21.15.97:0/2219065445 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:24.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbfd99c700 1 -- 172.21.15.97:0/2219065445 >> v2:172.21.15.146:3300/0 conn(0x7fbbf80cd390 msgr2=0x7fbbf8160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:24.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbfd99c700 1 --2- 172.21.15.97:0/2219065445 >> v2:172.21.15.146:3300/0 conn(0x7fbbf80cd390 0x7fbbf8160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbfd99c700 1 -- 172.21.15.97:0/2219065445 >> v1:172.21.15.146:6789/0 conn(0x7fbbf815c520 legacy=0x7fbbf8160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:24.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.077+0000 7fbbfd99c700 1 -- 172.21.15.97:0/2219065445 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf80b6c00 con 0x7fbbf8160dd0 2023-07-06T03:46:24.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.078+0000 7fbbfd99c700 1 --2- 172.21.15.97:0/2219065445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf8160dd0 0x7fbbf80b7400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80b5930 tx=0x7fbbf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51586ab3422a0c6d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:24.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.078+0000 7fbbeffff700 1 -- 172.21.15.97:0/2219065445 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000e070 con 0x7fbbf8160dd0 2023-07-06T03:46:24.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.078+0000 7fbbeffff700 1 -- 172.21.15.97:0/2219065445 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf0003a20 con 0x7fbbf8160dd0 2023-07-06T03:46:24.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.078+0000 7fbbeffff700 1 -- 172.21.15.97:0/2219065445 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0003dd0 con 0x7fbbf8160dd0 2023-07-06T03:46:24.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.078+0000 7fbbff3ff700 1 -- 172.21.15.97:0/2219065445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf8160dd0 msgr2=0x7fbbf80b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.078+0000 7fbbff3ff700 1 --2- 172.21.15.97:0/2219065445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf8160dd0 0x7fbbf80b7400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80b5930 tx=0x7fbbf000b330 comp rx=0 tx=0).stop 2023-07-06T03:46:24.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.079+0000 7fbbff3ff700 1 -- 172.21.15.97:0/2219065445 shutdown_connections 2023-07-06T03:46:24.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.079+0000 7fbbff3ff700 1 --2- 172.21.15.97:0/2219065445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf8160dd0 0x7fbbf80b7400 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.079+0000 7fbbff3ff700 1 --2- 172.21.15.97:0/2219065445 >> v2:172.21.15.146:3300/0 conn(0x7fbbf80cd390 0x7fbbf8160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.079+0000 7fbbff3ff700 1 -- 172.21.15.97:0/2219065445 >> 172.21.15.97:0/2219065445 conn(0x7fbbf8071790 msgr2=0x7fbbf8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.079+0000 7fbbff3ff700 1 -- 172.21.15.97:0/2219065445 shutdown_connections 2023-07-06T03:46:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.079+0000 7fbbff3ff700 1 -- 172.21.15.97:0/2219065445 wait complete. 2023-07-06T03:46:24.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.080+0000 7fbbff3ff700 1 Processor -- start 2023-07-06T03:46:24.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.080+0000 7fbbff3ff700 1 -- start start 2023-07-06T03:46:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.081+0000 7fbbff3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf80cd390 0x7fbbf815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.081+0000 7fbbff3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbf815c520 0x7fbbf815f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.081+0000 7fbbff3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbf8160dd0 0x7fbbf8159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.081+0000 7fbbff3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80ccd40 con 0x7fbbf80cd390 2023-07-06T03:46:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.081+0000 7fbbff3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80cc410 con 0x7fbbf8160dd0 2023-07-06T03:46:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.081+0000 7fbbff3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80cc6f0 con 0x7fbbf815c520 2023-07-06T03:46:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.082+0000 7fbbfd19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf80cd390 0x7fbbf815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.082+0000 7fbbfd19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf80cd390 0x7fbbf815efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45336/0 (socket says 172.21.15.97:45336) 2023-07-06T03:46:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.082+0000 7fbbfd19b700 1 -- 172.21.15.97:0/16015624 learned_addr learned my addr 172.21.15.97:0/16015624 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.082+0000 7fbbfd19b700 1 -- 172.21.15.97:0/16015624 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbf815c520 msgr2=0x7fbbf815f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.082+0000 7fbbfd19b700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbf815c520 0x7fbbf815f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.082+0000 7fbbfd19b700 1 -- 172.21.15.97:0/16015624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbf8160dd0 msgr2=0x7fbbf8159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.082+0000 7fbbfd19b700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbf8160dd0 0x7fbbf8159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.082+0000 7fbbfd19b700 1 -- 172.21.15.97:0/16015624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf0009b30 con 0x7fbbf80cd390 2023-07-06T03:46:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.082+0000 7fbbfd19b700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf80cd390 0x7fbbf815efd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400c4a0 tx=0x7fbbf400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.082+0000 7fbbedffb700 1 -- 172.21.15.97:0/16015624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40054e0 con 0x7fbbf80cd390 2023-07-06T03:46:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.082+0000 7fbbedffb700 1 -- 172.21.15.97:0/16015624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4003680 con 0x7fbbf80cd390 2023-07-06T03:46:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.083+0000 7fbbedffb700 1 -- 172.21.15.97:0/16015624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4003a30 con 0x7fbbf80cd390 2023-07-06T03:46:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.083+0000 7fbbff3ff700 1 -- 172.21.15.97:0/16015624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf80cc9d0 con 0x7fbbf80cd390 2023-07-06T03:46:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.083+0000 7fbbff3ff700 1 -- 172.21.15.97:0/16015624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf814d330 con 0x7fbbf80cd390 2023-07-06T03:46:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.084+0000 7fbbff3ff700 1 -- 172.21.15.97:0/16015624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf804e340 con 0x7fbbf80cd390 2023-07-06T03:46:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.085+0000 7fbbedffb700 1 -- 172.21.15.97:0/16015624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbbf4003b90 con 0x7fbbf80cd390 2023-07-06T03:46:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.085+0000 7fbbedffb700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbbdc06ace0 0x7fbbdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.085+0000 7fbbedffb700 1 -- 172.21.15.97:0/16015624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbbf4088650 con 0x7fbbf80cd390 2023-07-06T03:46:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.086+0000 7fbbfc99a700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbbdc06ace0 0x7fbbdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.086+0000 7fbbfc99a700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbbdc06ace0 0x7fbbdc06d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbbf815fc00 tx=0x7fbbe8006c80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.090+0000 7fbbedffb700 1 -- 172.21.15.97:0/16015624 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbbf40535d0 con 0x7fbbf80cd390 2023-07-06T03:46:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.324+0000 7fbbff3ff700 1 -- 172.21.15.97:0/16015624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbf80b4f40 con 0x7fbbf80cd390 2023-07-06T03:46:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.324+0000 7fbbedffb700 1 -- 172.21.15.97:0/16015624 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4665 (secure 0 0 0) 0x7fbbf4056bf0 con 0x7fbbf80cd390 2023-07-06T03:46:24.327 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:24.327 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/4255451313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4255451313},{"type":"v1","addr":"172.21.15.97:6836","nonce":4255451313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/370322164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":370322164},{"type":"v1","addr":"172.21.15.97:6837","nonce":370322164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3764820847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3764820847},{"type":"v1","addr":"172.21.15.146:6835","nonce":3764820847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:45:11.126281+0000","modified":"2023-07-06T03:45:14.685499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6833/186544257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":186544257},{"type":"v1","addr":"172.21.15.146:6833","nonce":186544257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T03:46:24.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 -- 172.21.15.97:0/16015624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbbdc06ace0 msgr2=0x7fbbdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbbdc06ace0 0x7fbbdc06d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbbf815fc00 tx=0x7fbbe8006c80 comp rx=0 tx=0).stop 2023-07-06T03:46:24.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 -- 172.21.15.97:0/16015624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf80cd390 msgr2=0x7fbbf815efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf80cd390 0x7fbbf815efd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400c4a0 tx=0x7fbbf400c860 comp rx=0 tx=0).stop 2023-07-06T03:46:24.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 -- 172.21.15.97:0/16015624 shutdown_connections 2023-07-06T03:46:24.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbbdc06ace0 0x7fbbdc06d190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbf8160dd0 0x7fbbf8159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf80cd390 0x7fbbf815efd0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 --2- 172.21.15.97:0/16015624 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbf815c520 0x7fbbf815f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 -- 172.21.15.97:0/16015624 >> 172.21.15.97:0/16015624 conn(0x7fbbf8071790 msgr2=0x7fbbf8071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:24.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 -- 172.21.15.97:0/16015624 shutdown_connections 2023-07-06T03:46:24.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.327+0000 7fbbff3ff700 1 -- 172.21.15.97:0/16015624 wait complete. 2023-07-06T03:46:24.331 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-07-06T03:46:24.346 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:46:24.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.539+0000 7fb0835ff700 1 Processor -- start 2023-07-06T03:46:24.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.539+0000 7fb0835ff700 1 -- start start 2023-07-06T03:46:24.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.540+0000 7fb0835ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb07c15c520 0x7fb07c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:24.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.540+0000 7fb0835ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07c0cdae0 con 0x7fb07c160180 2023-07-06T03:46:24.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.540+0000 7fb0835ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07c0b6640 con 0x7fb07c160d00 2023-07-06T03:46:24.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.540+0000 7fb0835ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07c0cd8d0 con 0x7fb07c15c520 2023-07-06T03:46:24.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.540+0000 7fb08139b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb07c15c520 0x7fb07c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:24.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.541+0000 7fb08139b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb07c15c520 0x7fb07c0cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45346/0 (socket says 172.21.15.97:45346) 2023-07-06T03:46:24.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.541+0000 7fb08139b700 1 -- 172.21.15.97:0/4072495626 learned_addr learned my addr 172.21.15.97:0/4072495626 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.541+0000 7fb080b9a700 1 --1- 172.21.15.97:0/4072495626 >> v1:172.21.15.97:6789/0 conn(0x7fb07c160180 0x7fb07c1605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54522/0 (socket says 172.21.15.97:54522) 2023-07-06T03:46:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.541+0000 7fb08139b700 1 -- 172.21.15.97:0/4072495626 >> v1:172.21.15.146:6789/0 conn(0x7fb07c160d00 legacy=0x7fb07c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.541+0000 7fb08139b700 1 -- 172.21.15.97:0/4072495626 >> v1:172.21.15.97:6789/0 conn(0x7fb07c160180 legacy=0x7fb07c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.541+0000 7fb073fff700 1 -- 172.21.15.97:0/4072495626 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231570435 0 0) 0x7fb07c0cdae0 con 0x7fb07c160180 2023-07-06T03:46:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.541+0000 7fb08139b700 1 -- 172.21.15.97:0/4072495626 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c0b6920 con 0x7fb07c15c520 2023-07-06T03:46:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.541+0000 7fb08139b700 1 --2- 172.21.15.97:0/4072495626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c15c520 0x7fb07c0cd390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb0780098f0 tx=0x7fb078009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a47b2e2a957668 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.541+0000 7fb073fff700 1 -- 172.21.15.97:0/4072495626 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07800b8a0 con 0x7fb07c15c520 2023-07-06T03:46:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.542+0000 7fb073fff700 1 -- 172.21.15.97:0/4072495626 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07800ba00 con 0x7fb07c15c520 2023-07-06T03:46:24.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.542+0000 7fb073fff700 1 -- 172.21.15.97:0/4072495626 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07800bd90 con 0x7fb07c15c520 2023-07-06T03:46:24.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.542+0000 7fb0835ff700 1 -- 172.21.15.97:0/4072495626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c15c520 msgr2=0x7fb07c0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.542+0000 7fb0835ff700 1 --2- 172.21.15.97:0/4072495626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c15c520 0x7fb07c0cd390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb0780098f0 tx=0x7fb078009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.542+0000 7fb0835ff700 1 -- 172.21.15.97:0/4072495626 shutdown_connections 2023-07-06T03:46:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.542+0000 7fb0835ff700 1 --2- 172.21.15.97:0/4072495626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c15c520 0x7fb07c0cd390 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.542+0000 7fb0835ff700 1 -- 172.21.15.97:0/4072495626 >> 172.21.15.97:0/4072495626 conn(0x7fb07c071790 msgr2=0x7fb07c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.542+0000 7fb0835ff700 1 -- 172.21.15.97:0/4072495626 shutdown_connections 2023-07-06T03:46:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.542+0000 7fb0835ff700 1 -- 172.21.15.97:0/4072495626 wait complete. 2023-07-06T03:46:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.544+0000 7fb0835ff700 1 Processor -- start 2023-07-06T03:46:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.545+0000 7fb0835ff700 1 -- start start 2023-07-06T03:46:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb0835ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c15c520 0x7fb07c1505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb0835ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb07c160180 0x7fb07c14d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb0835ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07c160d00 0x7fb07c1f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb0835ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c15d060 con 0x7fb07c15c520 2023-07-06T03:46:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb0835ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c15d340 con 0x7fb07c160180 2023-07-06T03:46:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb0835ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c15d620 con 0x7fb07c160d00 2023-07-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb08139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c15c520 0x7fb07c1505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb08139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c15c520 0x7fb07c1505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45360/0 (socket says 172.21.15.97:45360) 2023-07-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb08139b700 1 -- 172.21.15.97:0/521522091 learned_addr learned my addr 172.21.15.97:0/521522091 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb080b9a700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb07c160180 0x7fb07c14d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb081b9c700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07c160d00 0x7fb07c1f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb08139b700 1 -- 172.21.15.97:0/521522091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07c160d00 msgr2=0x7fb07c1f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb08139b700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07c160d00 0x7fb07c1f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb08139b700 1 -- 172.21.15.97:0/521522091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb07c160180 msgr2=0x7fb07c14d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb08139b700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb07c160180 0x7fb07c14d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb08139b700 1 -- 172.21.15.97:0/521522091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078009550 con 0x7fb07c15c520 2023-07-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.546+0000 7fb080b9a700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb07c160180 0x7fb07c14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.547+0000 7fb081b9c700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07c160d00 0x7fb07c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.547+0000 7fb08139b700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c15c520 0x7fb07c1505e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb078014040 tx=0x7fb078004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.547+0000 7fb071ffb700 1 -- 172.21.15.97:0/521522091 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780038a0 con 0x7fb07c15c520 2023-07-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.547+0000 7fb071ffb700 1 -- 172.21.15.97:0/521522091 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07801e070 con 0x7fb07c15c520 2023-07-06T03:46:24.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.547+0000 7fb0835ff700 1 -- 172.21.15.97:0/521522091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c1f5830 con 0x7fb07c15c520 2023-07-06T03:46:24.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.547+0000 7fb071ffb700 1 -- 172.21.15.97:0/521522091 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb078003c20 con 0x7fb07c15c520 2023-07-06T03:46:24.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.547+0000 7fb0835ff700 1 -- 172.21.15.97:0/521522091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c150260 con 0x7fb07c15c520 2023-07-06T03:46:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.548+0000 7fb0835ff700 1 -- 172.21.15.97:0/521522091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c1f5a40 con 0x7fb07c15c520 2023-07-06T03:46:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.549+0000 7fb071ffb700 1 -- 172.21.15.97:0/521522091 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb078011420 con 0x7fb07c15c520 2023-07-06T03:46:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.550+0000 7fb071ffb700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb06006ace0 0x7fb06006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.550+0000 7fb071ffb700 1 -- 172.21.15.97:0/521522091 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb078020070 con 0x7fb07c15c520 2023-07-06T03:46:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.550+0000 7fb080b9a700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb06006ace0 0x7fb06006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.551+0000 7fb080b9a700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb06006ace0 0x7fb06006d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0d33b0 tx=0x7fb06c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.554+0000 7fb071ffb700 1 -- 172.21.15.97:0/521522091 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb078055ba0 con 0x7fb07c15c520 2023-07-06T03:46:24.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.789+0000 7fb0835ff700 1 -- 172.21.15.97:0/521522091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb07c04e340 con 0x7fb07c15c520 2023-07-06T03:46:24.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.790+0000 7fb071ffb700 1 -- 172.21.15.97:0/521522091 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4665 (secure 0 0 0) 0x7fb0780591c0 con 0x7fb07c15c520 2023-07-06T03:46:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:24.792 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/4255451313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4255451313},{"type":"v1","addr":"172.21.15.97:6836","nonce":4255451313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/370322164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":370322164},{"type":"v1","addr":"172.21.15.97:6837","nonce":370322164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3764820847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3764820847},{"type":"v1","addr":"172.21.15.146:6835","nonce":3764820847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:45:11.126281+0000","modified":"2023-07-06T03:45:14.685499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6833/186544257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":186544257},{"type":"v1","addr":"172.21.15.146:6833","nonce":186544257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T03:46:24.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 -- 172.21.15.97:0/521522091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb06006ace0 msgr2=0x7fb06006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb06006ace0 0x7fb06006d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0d33b0 tx=0x7fb06c009040 comp rx=0 tx=0).stop 2023-07-06T03:46:24.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 -- 172.21.15.97:0/521522091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c15c520 msgr2=0x7fb07c1505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:24.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c15c520 0x7fb07c1505e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb078014040 tx=0x7fb078004750 comp rx=0 tx=0).stop 2023-07-06T03:46:24.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 -- 172.21.15.97:0/521522091 shutdown_connections 2023-07-06T03:46:24.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb06006ace0 0x7fb06006d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb07c160d00 0x7fb07c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb07c15c520 0x7fb07c1505e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 --2- 172.21.15.97:0/521522091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb07c160180 0x7fb07c14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:24.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 -- 172.21.15.97:0/521522091 >> 172.21.15.97:0/521522091 conn(0x7fb07c071790 msgr2=0x7fb07c0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:24.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 -- 172.21.15.97:0/521522091 shutdown_connections 2023-07-06T03:46:24.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:24.792+0000 7fb0835ff700 1 -- 172.21.15.97:0/521522091 wait complete. 2023-07-06T03:46:24.796 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-07-06T03:46:24.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:25.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.005+0000 7fa676fff700 1 Processor -- start 2023-07-06T03:46:25.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.005+0000 7fa676fff700 1 -- start start 2023-07-06T03:46:25.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.006+0000 7fa676fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6700cd390 0x7fa670160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.006+0000 7fa676fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa670160dd0 0x7fa6700b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.006+0000 7fa676fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700cdae0 con 0x7fa67015c520 2023-07-06T03:46:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.006+0000 7fa676fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6700b6640 con 0x7fa6700cd390 2023-07-06T03:46:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.006+0000 7fa676fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6700b6920 con 0x7fa670160dd0 2023-07-06T03:46:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.006+0000 7fa66ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6700cd390 0x7fa670160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.007+0000 7fa66ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6700cd390 0x7fa670160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45374/0 (socket says 172.21.15.97:45374) 2023-07-06T03:46:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.007+0000 7fa66ffff700 1 -- 172.21.15.97:0/2355781580 learned_addr learned my addr 172.21.15.97:0/2355781580 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.007+0000 7fa67559c700 1 --2- 172.21.15.97:0/2355781580 >> v2:172.21.15.146:3300/0 conn(0x7fa670160dd0 0x7fa6700b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.007+0000 7fa66ffff700 1 -- 172.21.15.97:0/2355781580 >> v2:172.21.15.146:3300/0 conn(0x7fa670160dd0 msgr2=0x7fa6700b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:25.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.007+0000 7fa66ffff700 1 --2- 172.21.15.97:0/2355781580 >> v2:172.21.15.146:3300/0 conn(0x7fa670160dd0 0x7fa6700b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.007+0000 7fa66ffff700 1 -- 172.21.15.97:0/2355781580 >> v1:172.21.15.146:6789/0 conn(0x7fa67015c520 legacy=0x7fa670160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.007+0000 7fa66ffff700 1 -- 172.21.15.97:0/2355781580 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700b6c00 con 0x7fa6700cd390 2023-07-06T03:46:25.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.007+0000 7fa66ffff700 1 --2- 172.21.15.97:0/2355781580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6700cd390 0x7fa670160890 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa6700cd320 tx=0x7fa65c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc42f0ef4cb08f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:25.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.007+0000 7fa66f7fe700 1 -- 172.21.15.97:0/2355781580 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c00b870 con 0x7fa6700cd390 2023-07-06T03:46:25.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.007+0000 7fa66f7fe700 1 -- 172.21.15.97:0/2355781580 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65c00b9d0 con 0x7fa6700cd390 2023-07-06T03:46:25.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.007+0000 7fa66f7fe700 1 -- 172.21.15.97:0/2355781580 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c00bd60 con 0x7fa6700cd390 2023-07-06T03:46:25.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.008+0000 7fa676fff700 1 -- 172.21.15.97:0/2355781580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6700cd390 msgr2=0x7fa670160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.008+0000 7fa676fff700 1 --2- 172.21.15.97:0/2355781580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6700cd390 0x7fa670160890 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa6700cd320 tx=0x7fa65c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.008+0000 7fa676fff700 1 -- 172.21.15.97:0/2355781580 shutdown_connections 2023-07-06T03:46:25.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.008+0000 7fa676fff700 1 --2- 172.21.15.97:0/2355781580 >> v2:172.21.15.146:3300/0 conn(0x7fa670160dd0 0x7fa6700b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.008+0000 7fa676fff700 1 --2- 172.21.15.97:0/2355781580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6700cd390 0x7fa670160890 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.008+0000 7fa676fff700 1 -- 172.21.15.97:0/2355781580 >> 172.21.15.97:0/2355781580 conn(0x7fa670071790 msgr2=0x7fa670071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:25.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.008+0000 7fa676fff700 1 -- 172.21.15.97:0/2355781580 shutdown_connections 2023-07-06T03:46:25.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.008+0000 7fa676fff700 1 -- 172.21.15.97:0/2355781580 wait complete. 2023-07-06T03:46:25.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.010+0000 7fa676fff700 1 Processor -- start 2023-07-06T03:46:25.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.010+0000 7fa676fff700 1 -- start start 2023-07-06T03:46:25.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.011+0000 7fa676fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6700cd390 0x7fa6700d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.011+0000 7fa676fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa67015c520 0x7fa670158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.011+0000 7fa676fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa670160dd0 0x7fa670158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.011+0000 7fa676fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700d2ab0 con 0x7fa670160dd0 2023-07-06T03:46:25.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.011+0000 7fa676fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700c9380 con 0x7fa6700cd390 2023-07-06T03:46:25.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.011+0000 7fa676fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700d1e30 con 0x7fa67015c520 2023-07-06T03:46:25.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.011+0000 7fa67559c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa670160dd0 0x7fa670158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.011+0000 7fa67559c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa670160dd0 0x7fa670158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45384/0 (socket says 172.21.15.97:45384) 2023-07-06T03:46:25.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.011+0000 7fa67559c700 1 -- 172.21.15.97:0/1095664373 learned_addr learned my addr 172.21.15.97:0/1095664373 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:25.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.011+0000 7fa674d9b700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6700cd390 0x7fa6700d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.011+0000 7fa66ffff700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa67015c520 0x7fa670158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.012+0000 7fa67559c700 1 -- 172.21.15.97:0/1095664373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa67015c520 msgr2=0x7fa670158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.012+0000 7fa67559c700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa67015c520 0x7fa670158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.012+0000 7fa67559c700 1 -- 172.21.15.97:0/1095664373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6700cd390 msgr2=0x7fa6700d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.012+0000 7fa67559c700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6700cd390 0x7fa6700d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.012+0000 7fa67559c700 1 -- 172.21.15.97:0/1095664373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c009550 con 0x7fa670160dd0 2023-07-06T03:46:25.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.012+0000 7fa67559c700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa670160dd0 0x7fa670158640 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa668007e40 tx=0x7fa66800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:25.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.012+0000 7fa66d7fa700 1 -- 172.21.15.97:0/1095664373 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800ed30 con 0x7fa670160dd0 2023-07-06T03:46:25.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.012+0000 7fa66d7fa700 1 -- 172.21.15.97:0/1095664373 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa668014420 con 0x7fa670160dd0 2023-07-06T03:46:25.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.012+0000 7fa66d7fa700 1 -- 172.21.15.97:0/1095664373 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680147d0 con 0x7fa670160dd0 2023-07-06T03:46:25.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.012+0000 7fa676fff700 1 -- 172.21.15.97:0/1095664373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6700d2110 con 0x7fa670160dd0 2023-07-06T03:46:25.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.012+0000 7fa676fff700 1 -- 172.21.15.97:0/1095664373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6700d26d0 con 0x7fa670160dd0 2023-07-06T03:46:25.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.013+0000 7fa676fff700 1 -- 172.21.15.97:0/1095664373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6700d2bf0 con 0x7fa670160dd0 2023-07-06T03:46:25.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.014+0000 7fa66d7fa700 1 -- 172.21.15.97:0/1095664373 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa668014930 con 0x7fa670160dd0 2023-07-06T03:46:25.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.016+0000 7fa66d7fa700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa65406ace0 0x7fa65406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.016+0000 7fa66d7fa700 1 -- 172.21.15.97:0/1095664373 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa668089c10 con 0x7fa670160dd0 2023-07-06T03:46:25.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.016+0000 7fa674d9b700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa65406ace0 0x7fa65406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.017+0000 7fa674d9b700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa65406ace0 0x7fa65406d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa664007fd0 tx=0x7fa664009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:25.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.020+0000 7fa66d7fa700 1 -- 172.21.15.97:0/1095664373 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa6700d2bf0 con 0x7fa670160dd0 2023-07-06T03:46:25.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.216+0000 7fa676fff700 1 -- 172.21.15.97:0/1095664373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6700d2bf0 con 0x7fa670160dd0 2023-07-06T03:46:25.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.217+0000 7fa66d7fa700 1 -- 172.21.15.97:0/1095664373 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18377 (secure 0 0 0) 0x7fa6700d2bf0 con 0x7fa670160dd0 2023-07-06T03:46:25.219 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:25.219 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:45:12.662440+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:45:09.115113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T03:45:10.111206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:25.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.219+0000 7fa676fff700 1 -- 172.21.15.97:0/1095664373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa65406ace0 msgr2=0x7fa65406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.219+0000 7fa676fff700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa65406ace0 0x7fa65406d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa664007fd0 tx=0x7fa664009040 comp rx=0 tx=0).stop 2023-07-06T03:46:25.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.219+0000 7fa676fff700 1 -- 172.21.15.97:0/1095664373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa670160dd0 msgr2=0x7fa670158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.219+0000 7fa676fff700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa670160dd0 0x7fa670158640 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa668007e40 tx=0x7fa66800e3c0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.219+0000 7fa676fff700 1 -- 172.21.15.97:0/1095664373 shutdown_connections 2023-07-06T03:46:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.219+0000 7fa676fff700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa65406ace0 0x7fa65406d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.219+0000 7fa676fff700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa670160dd0 0x7fa670158640 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.219+0000 7fa676fff700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6700cd390 0x7fa6700d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.219+0000 7fa676fff700 1 --2- 172.21.15.97:0/1095664373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa67015c520 0x7fa670158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.220+0000 7fa676fff700 1 -- 172.21.15.97:0/1095664373 >> 172.21.15.97:0/1095664373 conn(0x7fa670071790 msgr2=0x7fa6700d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:25.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.220+0000 7fa676fff700 1 -- 172.21.15.97:0/1095664373 shutdown_connections 2023-07-06T03:46:25.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.220+0000 7fa676fff700 1 -- 172.21.15.97:0/1095664373 wait complete. 2023-07-06T03:46:25.240 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T03:46:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.433+0000 7f7cb9dff700 1 Processor -- start 2023-07-06T03:46:25.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.433+0000 7f7cb9dff700 1 -- start start 2023-07-06T03:46:25.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.434+0000 7f7cb9dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7cb4159070 0x7f7cb40c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.434+0000 7f7cb9dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb40ccbd0 0x7f7cb40cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.434+0000 7f7cb9dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40ca260 con 0x7f7cb40c3cd0 2023-07-06T03:46:25.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.434+0000 7f7cb9dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40c99c0 con 0x7f7cb4159070 2023-07-06T03:46:25.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.434+0000 7f7cb9dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40c9ca0 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.434+0000 7f7cb37fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7cb4159070 0x7f7cb40c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.434+0000 7f7cb37fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7cb4159070 0x7f7cb40c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45390/0 (socket says 172.21.15.97:45390) 2023-07-06T03:46:25.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.435+0000 7f7cb37fe700 1 -- 172.21.15.97:0/4134611369 learned_addr learned my addr 172.21.15.97:0/4134611369 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:25.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.435+0000 7f7cb3fff700 1 --2- 172.21.15.97:0/4134611369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb40ccbd0 0x7f7cb40cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.435+0000 7f7cb37fe700 1 -- 172.21.15.97:0/4134611369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb40ccbd0 msgr2=0x7f7cb40cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:25.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.435+0000 7f7cb37fe700 1 --2- 172.21.15.97:0/4134611369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb40ccbd0 0x7f7cb40cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.435+0000 7f7cb37fe700 1 -- 172.21.15.97:0/4134611369 >> v1:172.21.15.146:6789/0 conn(0x7f7cb40c3cd0 legacy=0x7f7cb40cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.435+0000 7f7cb37fe700 1 -- 172.21.15.97:0/4134611369 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40c9f80 con 0x7f7cb4159070 2023-07-06T03:46:25.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.435+0000 7f7cb37fe700 1 --2- 172.21.15.97:0/4134611369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb4159070 0x7f7cb40c8f40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400bb90 tx=0x7f7ca400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9d7cb5085977ee server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:25.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.435+0000 7f7cb27fc700 1 -- 172.21.15.97:0/4134611369 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400dd50 con 0x7f7cb4159070 2023-07-06T03:46:25.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.435+0000 7f7cb27fc700 1 -- 172.21.15.97:0/4134611369 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca4010470 con 0x7f7cb4159070 2023-07-06T03:46:25.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.435+0000 7f7cb27fc700 1 -- 172.21.15.97:0/4134611369 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4010800 con 0x7f7cb4159070 2023-07-06T03:46:25.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.436+0000 7f7cb9dff700 1 -- 172.21.15.97:0/4134611369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb4159070 msgr2=0x7f7cb40c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.436+0000 7f7cb9dff700 1 --2- 172.21.15.97:0/4134611369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb4159070 0x7f7cb40c8f40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400bb90 tx=0x7f7ca400bea0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.436+0000 7f7cb9dff700 1 -- 172.21.15.97:0/4134611369 shutdown_connections 2023-07-06T03:46:25.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.436+0000 7f7cb9dff700 1 --2- 172.21.15.97:0/4134611369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb40ccbd0 0x7f7cb40cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.436+0000 7f7cb9dff700 1 --2- 172.21.15.97:0/4134611369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb4159070 0x7f7cb40c8f40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.436+0000 7f7cb9dff700 1 -- 172.21.15.97:0/4134611369 >> 172.21.15.97:0/4134611369 conn(0x7f7cb406a060 msgr2=0x7f7cb406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:25.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.436+0000 7f7cb9dff700 1 -- 172.21.15.97:0/4134611369 shutdown_connections 2023-07-06T03:46:25.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.436+0000 7f7cb9dff700 1 -- 172.21.15.97:0/4134611369 wait complete. 2023-07-06T03:46:25.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.438+0000 7f7cb9dff700 1 Processor -- start 2023-07-06T03:46:25.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.439+0000 7f7cb9dff700 1 -- start start 2023-07-06T03:46:25.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.440+0000 7f7cb9dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb40c3cd0 0x7f7cb4155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.440+0000 7f7cb9dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb40ccbd0 0x7f7cb41f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.440+0000 7f7cb9dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb4159070 0x7f7cb4153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.440+0000 7f7cb9dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40b01c0 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.440+0000 7f7cb9dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40af470 con 0x7f7cb40c3cd0 2023-07-06T03:46:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.440+0000 7f7cb9dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40aeeb0 con 0x7f7cb4159070 2023-07-06T03:46:25.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.440+0000 7f7cb2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb40ccbd0 0x7f7cb41f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.440+0000 7f7cb2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb40ccbd0 0x7f7cb41f2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45396/0 (socket says 172.21.15.97:45396) 2023-07-06T03:46:25.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.440+0000 7f7cb2ffd700 1 -- 172.21.15.97:0/1292937902 learned_addr learned my addr 172.21.15.97:0/1292937902 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:25.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.440+0000 7f7cb37fe700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb40c3cd0 0x7f7cb4155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.440+0000 7f7cb3fff700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb4159070 0x7f7cb4153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7cb2ffd700 1 -- 172.21.15.97:0/1292937902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb4159070 msgr2=0x7f7cb4153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7cb2ffd700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb4159070 0x7f7cb4153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7cb2ffd700 1 -- 172.21.15.97:0/1292937902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb40c3cd0 msgr2=0x7f7cb4155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7cb2ffd700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb40c3cd0 0x7f7cb4155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7cb2ffd700 1 -- 172.21.15.97:0/1292937902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca400b7f0 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7cb3fff700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb4159070 0x7f7cb4153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:25.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7cb37fe700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb40c3cd0 0x7f7cb4155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:25.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7cb2ffd700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb40ccbd0 0x7f7cb41f2290 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800c4b0 tx=0x7f7ca800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:25.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7c9bfff700 1 -- 172.21.15.97:0/1292937902 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80052d0 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7c9bfff700 1 -- 172.21.15.97:0/1292937902 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca8005430 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7cb9dff700 1 -- 172.21.15.97:0/1292937902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb40af190 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7c9bfff700 1 -- 172.21.15.97:0/1292937902 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80038c0 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.441+0000 7f7cb9dff700 1 -- 172.21.15.97:0/1292937902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb4154150 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.442+0000 7f7cb9dff700 1 -- 172.21.15.97:0/1292937902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb404e340 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.443+0000 7f7c9bfff700 1 -- 172.21.15.97:0/1292937902 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ca8003a20 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.444+0000 7f7c9bfff700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ca006ace0 0x7f7ca006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.444+0000 7f7c9bfff700 1 -- 172.21.15.97:0/1292937902 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7ca80889b0 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.444+0000 7f7cb37fe700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ca006ace0 0x7f7ca006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.445+0000 7f7cb37fe700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ca006ace0 0x7f7ca006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400bee0 tx=0x7f7ca4013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:25.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.448+0000 7f7c9bfff700 1 -- 172.21.15.97:0/1292937902 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7ca80539c0 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.689+0000 7f7cb9dff700 1 -- 172.21.15.97:0/1292937902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7cb406d220 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.691 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-07-06T03:46:25.689+0000 7fb463ff4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T03:46:25.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.697+0000 7f7c9bfff700 1 -- 172.21.15.97:0/1292937902 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7ca8056fe0 con 0x7f7cb40ccbd0 2023-07-06T03:46:25.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.699+0000 7f7cb9dff700 1 -- 172.21.15.97:0/1292937902 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ca006ace0 msgr2=0x7f7ca006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.699+0000 7f7cb9dff700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ca006ace0 0x7f7ca006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400bee0 tx=0x7f7ca4013000 comp rx=0 tx=0).stop 2023-07-06T03:46:25.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.699+0000 7f7cb9dff700 1 -- 172.21.15.97:0/1292937902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb40ccbd0 msgr2=0x7f7cb41f2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.699+0000 7f7cb9dff700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb40ccbd0 0x7f7cb41f2290 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800c4b0 tx=0x7f7ca800c870 comp rx=0 tx=0).stop 2023-07-06T03:46:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.699+0000 7f7cb9dff700 1 -- 172.21.15.97:0/1292937902 shutdown_connections 2023-07-06T03:46:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.699+0000 7f7cb9dff700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ca006ace0 0x7f7ca006d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.699+0000 7f7cb9dff700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cb4159070 0x7f7cb4153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.699+0000 7f7cb9dff700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cb40c3cd0 0x7f7cb4155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.699+0000 7f7cb9dff700 1 --2- 172.21.15.97:0/1292937902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cb40ccbd0 0x7f7cb41f2290 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.699+0000 7f7cb9dff700 1 -- 172.21.15.97:0/1292937902 >> 172.21.15.97:0/1292937902 conn(0x7f7cb406a060 msgr2=0x7f7cb4070920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.699+0000 7f7cb9dff700 1 -- 172.21.15.97:0/1292937902 shutdown_connections 2023-07-06T03:46:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.700+0000 7f7cb9dff700 1 -- 172.21.15.97:0/1292937902 wait complete. 2023-07-06T03:46:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T03:46:25.704 INFO:tasks.ceph.mds.b.smithi146.stderr: -3737> 2023-07-06T03:45:16.328+0000 7fe7673f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:46:25.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T03:46:25.759 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-07-06T03:46:25.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.915+0000 7f51869ff700 1 Processor -- start 2023-07-06T03:46:25.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.915+0000 7f51869ff700 1 -- start start 2023-07-06T03:46:25.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f51869ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f518015c610 0x7f51800b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f51869ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51800d2210 0x7f51800d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f51869ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51800cff90 con 0x7f518006b360 2023-07-06T03:46:25.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f51869ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51800d0780 con 0x7f518015c610 2023-07-06T03:46:25.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f51869ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518015c9e0 con 0x7f51800d2210 2023-07-06T03:46:25.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f517ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f518015c610 0x7f51800b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f517ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f518015c610 0x7f51800b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45406/0 (socket says 172.21.15.97:45406) 2023-07-06T03:46:25.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f517ffff700 1 -- 172.21.15.97:0/1326010421 learned_addr learned my addr 172.21.15.97:0/1326010421 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:25.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f517ffff700 1 -- 172.21.15.97:0/1326010421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51800d2210 msgr2=0x7f51800d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:25.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f517ffff700 1 --2- 172.21.15.97:0/1326010421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51800d2210 0x7f51800d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f517ffff700 1 -- 172.21.15.97:0/1326010421 >> v1:172.21.15.146:6789/0 conn(0x7f518006b360 legacy=0x7f51800b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:25.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.917+0000 7f517ffff700 1 -- 172.21.15.97:0/1326010421 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800b5650 con 0x7f518015c610 2023-07-06T03:46:25.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.918+0000 7f517ffff700 1 --2- 172.21.15.97:0/1326010421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518015c610 0x7f51800b47b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f51700098f0 tx=0x7f5170009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62a56c032a40b22 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:25.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.918+0000 7f517effd700 1 -- 172.21.15.97:0/1326010421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170004230 con 0x7f518015c610 2023-07-06T03:46:25.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.918+0000 7f517effd700 1 -- 172.21.15.97:0/1326010421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5170004390 con 0x7f518015c610 2023-07-06T03:46:25.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.918+0000 7f517effd700 1 -- 172.21.15.97:0/1326010421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51700036a0 con 0x7f518015c610 2023-07-06T03:46:25.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.918+0000 7f51869ff700 1 -- 172.21.15.97:0/1326010421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518015c610 msgr2=0x7f51800b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.918+0000 7f51869ff700 1 --2- 172.21.15.97:0/1326010421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518015c610 0x7f51800b47b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f51700098f0 tx=0x7f5170009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:25.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.918+0000 7f51869ff700 1 -- 172.21.15.97:0/1326010421 shutdown_connections 2023-07-06T03:46:25.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.918+0000 7f51869ff700 1 --2- 172.21.15.97:0/1326010421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51800d2210 0x7f51800d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.918+0000 7f51869ff700 1 --2- 172.21.15.97:0/1326010421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518015c610 0x7f51800b47b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.918+0000 7f51869ff700 1 -- 172.21.15.97:0/1326010421 >> 172.21.15.97:0/1326010421 conn(0x7f518006a6c0 msgr2=0x7f518006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:25.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.918+0000 7f51869ff700 1 -- 172.21.15.97:0/1326010421 shutdown_connections 2023-07-06T03:46:25.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.919+0000 7f51869ff700 1 -- 172.21.15.97:0/1326010421 wait complete. 2023-07-06T03:46:25.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.920+0000 7f51869ff700 1 Processor -- start 2023-07-06T03:46:25.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.920+0000 7f51869ff700 1 -- start start 2023-07-06T03:46:25.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.921+0000 7f51869ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f518006b360 0x7f51801f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.921+0000 7f51869ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51800d2210 0x7f51801454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.921+0000 7f51869ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f518015c610 0x7f5180145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.921+0000 7f51869ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51800cb140 con 0x7f51800d2210 2023-07-06T03:46:25.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.921+0000 7f51869ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51800c95f0 con 0x7f518006b360 2023-07-06T03:46:25.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.921+0000 7f51869ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51800caaf0 con 0x7f518015c610 2023-07-06T03:46:25.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.921+0000 7f517f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51800d2210 0x7f51801454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.921+0000 7f517f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51800d2210 0x7f51801454c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45414/0 (socket says 172.21.15.97:45414) 2023-07-06T03:46:25.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.921+0000 7f517f7fe700 1 -- 172.21.15.97:0/1273949062 learned_addr learned my addr 172.21.15.97:0/1273949062 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:25.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f517ffff700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f518006b360 0x7f51801f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f517f7fe700 1 -- 172.21.15.97:0/1273949062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f518015c610 msgr2=0x7f5180145a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f517f7fe700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f518015c610 0x7f5180145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f517f7fe700 1 -- 172.21.15.97:0/1273949062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f518006b360 msgr2=0x7f51801f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:25.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f517f7fe700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f518006b360 0x7f51801f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:25.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f517f7fe700 1 -- 172.21.15.97:0/1273949062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5170009550 con 0x7f51800d2210 2023-07-06T03:46:25.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f517ffff700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f518006b360 0x7f51801f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:25.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f517f7fe700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51800d2210 0x7f51801454c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f517400c4a0 tx=0x7f517400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:25.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f517cff9700 1 -- 172.21.15.97:0/1273949062 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740054e0 con 0x7f51800d2210 2023-07-06T03:46:25.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f517cff9700 1 -- 172.21.15.97:0/1273949062 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5174003680 con 0x7f51800d2210 2023-07-06T03:46:25.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f517cff9700 1 -- 172.21.15.97:0/1273949062 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174003a30 con 0x7f51800d2210 2023-07-06T03:46:25.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f51869ff700 1 -- 172.21.15.97:0/1273949062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51800cadd0 con 0x7f51800d2210 2023-07-06T03:46:25.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.922+0000 7f51869ff700 1 -- 172.21.15.97:0/1273949062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51801f5590 con 0x7f51800d2210 2023-07-06T03:46:25.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.923+0000 7f51869ff700 1 -- 172.21.15.97:0/1273949062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51800cb2f0 con 0x7f51800d2210 2023-07-06T03:46:25.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.924+0000 7f517cff9700 1 -- 172.21.15.97:0/1273949062 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5174003b90 con 0x7f51800d2210 2023-07-06T03:46:25.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.926+0000 7f517cff9700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516c06ace0 0x7f516c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:25.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.926+0000 7f517cff9700 1 -- 172.21.15.97:0/1273949062 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f5174088720 con 0x7f51800d2210 2023-07-06T03:46:25.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.926+0000 7f517ffff700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516c06ace0 0x7f516c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:25.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.926+0000 7f517ffff700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516c06ace0 0x7f516c06d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5170009c40 tx=0x7f5170008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:25.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:25.930+0000 7f517cff9700 1 -- 172.21.15.97:0/1273949062 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f51800cb2f0 con 0x7f51800d2210 2023-07-06T03:46:26.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.168+0000 7f51869ff700 1 -- 172.21.15.97:0/1273949062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f518004e340 con 0x7f51800d2210 2023-07-06T03:46:26.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.696+0000 7f517cff9700 1 -- 172.21.15.97:0/1273949062 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5174053640 con 0x7f51800d2210 2023-07-06T03:46:26.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 -- 172.21.15.97:0/1273949062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516c06ace0 msgr2=0x7f516c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:26.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516c06ace0 0x7f516c06d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5170009c40 tx=0x7f5170008030 comp rx=0 tx=0).stop 2023-07-06T03:46:26.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 -- 172.21.15.97:0/1273949062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51800d2210 msgr2=0x7f51801454c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:26.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51800d2210 0x7f51801454c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f517400c4a0 tx=0x7f517400c860 comp rx=0 tx=0).stop 2023-07-06T03:46:26.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 -- 172.21.15.97:0/1273949062 shutdown_connections 2023-07-06T03:46:26.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516c06ace0 0x7f516c06d190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:26.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f518015c610 0x7f5180145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f518006b360 0x7f51801f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 --2- 172.21.15.97:0/1273949062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51800d2210 0x7f51801454c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 -- 172.21.15.97:0/1273949062 >> 172.21.15.97:0/1273949062 conn(0x7f518006a6c0 msgr2=0x7f51800d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 -- 172.21.15.97:0/1273949062 shutdown_connections 2023-07-06T03:46:26.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.699+0000 7f51869ff700 1 -- 172.21.15.97:0/1273949062 wait complete. 2023-07-06T03:46:26.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T03:46:26.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.920+0000 7fa5d63ff700 1 Processor -- start 2023-07-06T03:46:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.921+0000 7fa5d63ff700 1 -- start start 2023-07-06T03:46:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.922+0000 7fa5d63ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5d015c640 0x7fa5d00cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:26.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.922+0000 7fa5d63ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa5d00cfde0 0x7fa5d00b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:26.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.922+0000 7fa5d63ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d015ca10 con 0x7fa5d00b5270 2023-07-06T03:46:26.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.922+0000 7fa5d63ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d00b56e0 con 0x7fa5d00cfde0 2023-07-06T03:46:26.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.922+0000 7fa5d63ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00cda40 con 0x7fa5d015c640 2023-07-06T03:46:26.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.922+0000 7fa5cf7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa5d00b5270 0x7fa5d00cf6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54538/0 (socket says 172.21.15.97:54538) 2023-07-06T03:46:26.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.922+0000 7fa5cf7fe700 1 -- 172.21.15.97:0/61746950 learned_addr learned my addr 172.21.15.97:0/61746950 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.922+0000 7fa5d499c700 1 --2- 172.21.15.97:0/61746950 >> v2:172.21.15.146:3300/0 conn(0x7fa5d00cfde0 0x7fa5d00b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.922+0000 7fa5cffff700 1 --2- 172.21.15.97:0/61746950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5d015c640 0x7fa5d00cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.922+0000 7fa5ceffd700 1 -- 172.21.15.97:0/61746950 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907034245 0 0) 0x7fa5d015ca10 con 0x7fa5d00b5270 2023-07-06T03:46:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.922+0000 7fa5ceffd700 1 -- 172.21.15.97:0/61746950 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b80035b0 con 0x7fa5d00b5270 2023-07-06T03:46:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.923+0000 7fa5ceffd700 1 -- 172.21.15.97:0/61746950 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3625996592 0 0) 0x7fa5b80035b0 con 0x7fa5d00b5270 2023-07-06T03:46:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.923+0000 7fa5d499c700 1 -- 172.21.15.97:0/61746950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5d015c640 msgr2=0x7fa5d00cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.923+0000 7fa5d499c700 1 --2- 172.21.15.97:0/61746950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5d015c640 0x7fa5d00cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.923+0000 7fa5d499c700 1 -- 172.21.15.97:0/61746950 >> v1:172.21.15.97:6789/0 conn(0x7fa5d00b5270 legacy=0x7fa5d00cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.923+0000 7fa5d499c700 1 -- 172.21.15.97:0/61746950 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d00b65f0 con 0x7fa5d00cfde0 2023-07-06T03:46:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.923+0000 7fa5d499c700 1 --2- 172.21.15.97:0/61746950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5d00cfde0 0x7fa5d00b73b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009d90 tx=0x7fa5c800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9880c054f7a57f8b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.923+0000 7fa5ceffd700 1 -- 172.21.15.97:0/61746950 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c800e070 con 0x7fa5d00cfde0 2023-07-06T03:46:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.923+0000 7fa5ceffd700 1 -- 172.21.15.97:0/61746950 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c8003a20 con 0x7fa5d00cfde0 2023-07-06T03:46:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.923+0000 7fa5ceffd700 1 -- 172.21.15.97:0/61746950 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40017f0 con 0x7fa5d00cfde0 2023-07-06T03:46:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.924+0000 7fa5d63ff700 1 -- 172.21.15.97:0/61746950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5d00cfde0 msgr2=0x7fa5d00b73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.924+0000 7fa5d63ff700 1 --2- 172.21.15.97:0/61746950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5d00cfde0 0x7fa5d00b73b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009d90 tx=0x7fa5c800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.924+0000 7fa5d63ff700 1 -- 172.21.15.97:0/61746950 shutdown_connections 2023-07-06T03:46:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.924+0000 7fa5d63ff700 1 --2- 172.21.15.97:0/61746950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5d00cfde0 0x7fa5d00b73b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.924+0000 7fa5d63ff700 1 --2- 172.21.15.97:0/61746950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5d015c640 0x7fa5d00cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:26.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.924+0000 7fa5d63ff700 1 -- 172.21.15.97:0/61746950 >> 172.21.15.97:0/61746950 conn(0x7fa5d0071790 msgr2=0x7fa5d0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:26.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.924+0000 7fa5d63ff700 1 -- 172.21.15.97:0/61746950 shutdown_connections 2023-07-06T03:46:26.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.924+0000 7fa5d63ff700 1 -- 172.21.15.97:0/61746950 wait complete. 2023-07-06T03:46:26.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.926+0000 7fa5d63ff700 1 Processor -- start 2023-07-06T03:46:26.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.926+0000 7fa5d63ff700 1 -- start start 2023-07-06T03:46:26.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d63ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5d00b5270 0x7fa5d00c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:26.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d63ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5d00cfde0 0x7fa5d0148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:26.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d63ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5d015c640 0x7fa5d00b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:26.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d63ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00d0bf0 con 0x7fa5d015c640 2023-07-06T03:46:26.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d63ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00d0e40 con 0x7fa5d00b5270 2023-07-06T03:46:26.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d63ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00d1120 con 0x7fa5d00cfde0 2023-07-06T03:46:26.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d499c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5d015c640 0x7fa5d00b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:26.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d499c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5d015c640 0x7fa5d00b72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45416/0 (socket says 172.21.15.97:45416) 2023-07-06T03:46:26.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d499c700 1 -- 172.21.15.97:0/2778210945 learned_addr learned my addr 172.21.15.97:0/2778210945 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:26.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5cf7fe700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5d00cfde0 0x7fa5d0148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:26.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d499c700 1 -- 172.21.15.97:0/2778210945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5d00cfde0 msgr2=0x7fa5d0148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:26.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d499c700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5d00cfde0 0x7fa5d0148e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d499c700 1 -- 172.21.15.97:0/2778210945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5d00b5270 msgr2=0x7fa5d00c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d499c700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5d00b5270 0x7fa5d00c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.927+0000 7fa5d499c700 1 -- 172.21.15.97:0/2778210945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c8009a70 con 0x7fa5d015c640 2023-07-06T03:46:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.928+0000 7fa5d499c700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5d015c640 0x7fa5d00b72b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8004000 tx=0x7fa5c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.928+0000 7fa5ccff9700 1 -- 172.21.15.97:0/2778210945 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80045f0 con 0x7fa5d015c640 2023-07-06T03:46:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.928+0000 7fa5ccff9700 1 -- 172.21.15.97:0/2778210945 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c8004750 con 0x7fa5d015c640 2023-07-06T03:46:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.928+0000 7fa5d63ff700 1 -- 172.21.15.97:0/2778210945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d00d1400 con 0x7fa5d015c640 2023-07-06T03:46:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.928+0000 7fa5ccff9700 1 -- 172.21.15.97:0/2778210945 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8009450 con 0x7fa5d015c640 2023-07-06T03:46:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.928+0000 7fa5d63ff700 1 -- 172.21.15.97:0/2778210945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d00ca340 con 0x7fa5d015c640 2023-07-06T03:46:26.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.929+0000 7fa5d63ff700 1 -- 172.21.15.97:0/2778210945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d00cd110 con 0x7fa5d015c640 2023-07-06T03:46:26.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.930+0000 7fa5ccff9700 1 -- 172.21.15.97:0/2778210945 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5c80124b0 con 0x7fa5d015c640 2023-07-06T03:46:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.931+0000 7fa5ccff9700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5bc06ace0 0x7fa5bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.931+0000 7fa5ccff9700 1 -- 172.21.15.97:0/2778210945 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa5c808b600 con 0x7fa5d015c640 2023-07-06T03:46:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.931+0000 7fa5cffff700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5bc06ace0 0x7fa5bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.931+0000 7fa5cffff700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5bc06ace0 0x7fa5bc06d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0006b20 tx=0x7fa5c0006ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:26.935+0000 7fa5ccff9700 1 -- 172.21.15.97:0/2778210945 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa5c805a0d0 con 0x7fa5d015c640 2023-07-06T03:46:27.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.132+0000 7fa5d63ff700 1 -- 172.21.15.97:0/2778210945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa5d004e340 con 0x7fa5d015c640 2023-07-06T03:46:27.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.795+0000 7fa5ccff9700 1 -- 172.21.15.97:0/2778210945 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa5c8059c60 con 0x7fa5d015c640 2023-07-06T03:46:27.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.797+0000 7fa5d63ff700 1 -- 172.21.15.97:0/2778210945 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5bc06ace0 msgr2=0x7fa5bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:27.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.797+0000 7fa5d63ff700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5bc06ace0 0x7fa5bc06d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0006b20 tx=0x7fa5c0006ab0 comp rx=0 tx=0).stop 2023-07-06T03:46:27.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.797+0000 7fa5d63ff700 1 -- 172.21.15.97:0/2778210945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5d015c640 msgr2=0x7fa5d00b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:27.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.797+0000 7fa5d63ff700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5d015c640 0x7fa5d00b72b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8004000 tx=0x7fa5c8004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:27.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.798+0000 7fa5d63ff700 1 -- 172.21.15.97:0/2778210945 shutdown_connections 2023-07-06T03:46:27.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.798+0000 7fa5d63ff700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5bc06ace0 0x7fa5bc06d190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:27.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.798+0000 7fa5d63ff700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5d015c640 0x7fa5d00b72b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:27.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.798+0000 7fa5d63ff700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5d00b5270 0x7fa5d00c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:27.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.798+0000 7fa5d63ff700 1 --2- 172.21.15.97:0/2778210945 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5d00cfde0 0x7fa5d0148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:27.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.798+0000 7fa5d63ff700 1 -- 172.21.15.97:0/2778210945 >> 172.21.15.97:0/2778210945 conn(0x7fa5d0071790 msgr2=0x7fa5d0156b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:27.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.798+0000 7fa5d63ff700 1 -- 172.21.15.97:0/2778210945 shutdown_connections 2023-07-06T03:46:27.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:27.798+0000 7fa5d63ff700 1 -- 172.21.15.97:0/2778210945 wait complete. 2023-07-06T03:46:27.802 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-07-06T03:46:27.820 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T03:46:28.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.020+0000 7f7ce79ff700 1 Processor -- start 2023-07-06T03:46:28.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.020+0000 7f7ce79ff700 1 -- start start 2023-07-06T03:46:28.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.021+0000 7f7ce79ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7ce00b85f0 0x7f7ce006b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:28.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.021+0000 7f7ce79ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ce00cefa0 0x7f7ce00b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:28.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.022+0000 7f7ce79ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce00cd330 con 0x7f7ce00b4ca0 2023-07-06T03:46:28.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.022+0000 7f7ce79ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce00b89c0 con 0x7f7ce00b85f0 2023-07-06T03:46:28.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.022+0000 7f7ce79ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00b5120 con 0x7f7ce00cefa0 2023-07-06T03:46:28.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.022+0000 7f7ce5f9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7ce00b4ca0 0x7f7ce015ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54554/0 (socket says 172.21.15.97:54554) 2023-07-06T03:46:28.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.022+0000 7f7ce5f9c700 1 -- 172.21.15.97:0/1918603950 learned_addr learned my addr 172.21.15.97:0/1918603950 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.022+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1180064055 0 0) 0x7f7ce00cd330 con 0x7f7ce00b4ca0 2023-07-06T03:46:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.022+0000 7f7ce579b700 1 --2- 172.21.15.97:0/1918603950 >> v2:172.21.15.146:3300/0 conn(0x7f7ce00b85f0 0x7f7ce006b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.022+0000 7f7ce4f9a700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ce00cefa0 0x7f7ce00b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:28.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.022+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ccc0035b0 con 0x7f7ce00b4ca0 2023-07-06T03:46:28.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.022+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3170107148 0 0) 0x7f7ccc0035b0 con 0x7f7ce00b4ca0 2023-07-06T03:46:28.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ce00cefa0 msgr2=0x7f7ce00b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:28.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ce00cefa0 0x7f7ce00b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:28.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 >> v2:172.21.15.146:3300/0 conn(0x7f7ce00b85f0 msgr2=0x7f7ce006b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:28.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 --2- 172.21.15.97:0/1918603950 >> v2:172.21.15.146:3300/0 conn(0x7f7ce00b85f0 0x7f7ce006b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:28.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce00d02a0 con 0x7f7ce00b4ca0 2023-07-06T03:46:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f7cd80019f0 con 0x7f7ce00b4ca0 2023-07-06T03:46:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 >> v1:172.21.15.97:6789/0 conn(0x7f7ce00b4ca0 legacy=0x7f7ce015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ce00cefa0 0x7f7ccc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ccc003fd0 0x7f7ccc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc004d90 0x7f7ccc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00b89c0 con 0x7f7ccc004d90 2023-07-06T03:46:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00b5120 con 0x7f7ccc003fd0 2023-07-06T03:46:28.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc00ba10 con 0x7f7ce00cefa0 2023-07-06T03:46:28.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7ce5f9c700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc004d90 0x7f7ccc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:28.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7ce579b700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ce00cefa0 0x7f7ccc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:28.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7ce4f9a700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ccc003fd0 0x7f7ccc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:28.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7ce5f9c700 1 -- 172.21.15.97:0/1918603950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ce00cefa0 msgr2=0x7f7ccc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:28.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7ce5f9c700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ce00cefa0 0x7f7ccc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:28.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7ce5f9c700 1 -- 172.21.15.97:0/1918603950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ccc003fd0 msgr2=0x7f7ccc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:28.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7ce5f9c700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ccc003fd0 0x7f7ccc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:28.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.023+0000 7f7ce5f9c700 1 -- 172.21.15.97:0/1918603950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc0035b0 con 0x7f7ccc004d90 2023-07-06T03:46:28.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.024+0000 7f7ce5f9c700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc004d90 0x7f7ccc005200 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800a780 tx=0x7f7ccc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fc05dadbdb8b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:28.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.024+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80072d0 con 0x7f7ccc004d90 2023-07-06T03:46:28.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.024+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd8007430 con 0x7f7ccc004d90 2023-07-06T03:46:28.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.024+0000 7f7cd7fff700 1 -- 172.21.15.97:0/1918603950 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80077e0 con 0x7f7ccc004d90 2023-07-06T03:46:28.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.024+0000 7f7ce79ff700 1 -- 172.21.15.97:0/1918603950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc004d90 msgr2=0x7f7ccc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:28.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.024+0000 7f7ce79ff700 1 --2- 172.21.15.97:0/1918603950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc004d90 0x7f7ccc005200 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800a780 tx=0x7f7ccc005880 comp rx=0 tx=0).stop 2023-07-06T03:46:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.024+0000 7f7ce5f9c700 1 -- 172.21.15.97:0/1918603950 reap_dead start 2023-07-06T03:46:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.024+0000 7f7ce79ff700 1 -- 172.21.15.97:0/1918603950 shutdown_connections 2023-07-06T03:46:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.024+0000 7f7ce79ff700 1 -- 172.21.15.97:0/1918603950 >> 172.21.15.97:0/1918603950 conn(0x7f7ce006a5b0 msgr2=0x7f7ce006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.024+0000 7f7ce79ff700 1 -- 172.21.15.97:0/1918603950 shutdown_connections 2023-07-06T03:46:28.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.024+0000 7f7ce79ff700 1 -- 172.21.15.97:0/1918603950 wait complete. 2023-07-06T03:46:28.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.026+0000 7f7ce79ff700 1 Processor -- start 2023-07-06T03:46:28.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.026+0000 7f7ce79ff700 1 -- start start 2023-07-06T03:46:28.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.027+0000 7f7ce79ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc004d90 0x7f7ce01f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:28.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.027+0000 7f7ce79ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ce015d5b0 0x7f7ce015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:28.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.027+0000 7f7ce79ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ce015aaf0 0x7f7ce015e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:28.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.027+0000 7f7ce79ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce015b000 con 0x7f7ce015aaf0 2023-07-06T03:46:28.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.027+0000 7f7ce79ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce015b2e0 con 0x7f7ce015d5b0 2023-07-06T03:46:28.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.027+0000 7f7ce79ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce015cad0 con 0x7f7ccc004d90 2023-07-06T03:46:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.027+0000 7f7ce5f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ce015aaf0 0x7f7ce015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.027+0000 7f7ce5f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ce015aaf0 0x7f7ce015e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45426/0 (socket says 172.21.15.97:45426) 2023-07-06T03:46:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7ce5f9c700 1 -- 172.21.15.97:0/3253816048 learned_addr learned my addr 172.21.15.97:0/3253816048 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7ce579b700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc004d90 0x7f7ce01f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7ce4f9a700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ce015d5b0 0x7f7ce015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7ce5f9c700 1 -- 172.21.15.97:0/3253816048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc004d90 msgr2=0x7f7ce01f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7ce5f9c700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc004d90 0x7f7ce01f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:28.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7ce5f9c700 1 -- 172.21.15.97:0/3253816048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ce015d5b0 msgr2=0x7f7ce015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:28.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7ce5f9c700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ce015d5b0 0x7f7ce015a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:28.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7ce5f9c700 1 -- 172.21.15.97:0/3253816048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce015cdb0 con 0x7f7ce015aaf0 2023-07-06T03:46:28.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7ce579b700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc004d90 0x7f7ce01f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:28.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7ce5f9c700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ce015aaf0 0x7f7ce015e250 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8002230 tx=0x7f7cd8001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:28.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7cd5ffb700 1 -- 172.21.15.97:0/3253816048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8025070 con 0x7f7ce015aaf0 2023-07-06T03:46:28.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7cd5ffb700 1 -- 172.21.15.97:0/3253816048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd80075a0 con 0x7f7ce015aaf0 2023-07-06T03:46:28.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7ce79ff700 1 -- 172.21.15.97:0/3253816048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce015d090 con 0x7f7ce015aaf0 2023-07-06T03:46:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.028+0000 7f7cd5ffb700 1 -- 172.21.15.97:0/3253816048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8012620 con 0x7f7ce015aaf0 2023-07-06T03:46:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.029+0000 7f7ce79ff700 1 -- 172.21.15.97:0/3253816048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce0148390 con 0x7f7ce015aaf0 2023-07-06T03:46:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.030+0000 7f7ce79ff700 1 -- 172.21.15.97:0/3253816048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce00cf3f0 con 0x7f7ce015aaf0 2023-07-06T03:46:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.030+0000 7f7cd5ffb700 1 -- 172.21.15.97:0/3253816048 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7cd801a070 con 0x7f7ce015aaf0 2023-07-06T03:46:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.031+0000 7f7cd5ffb700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc406ace0 0x7f7cc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.032+0000 7f7cd5ffb700 1 -- 172.21.15.97:0/3253816048 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f7cd808add0 con 0x7f7ce015aaf0 2023-07-06T03:46:28.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.032+0000 7f7ce579b700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc406ace0 0x7f7cc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:28.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.032+0000 7f7ce579b700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc406ace0 0x7f7cc406d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc009ce0 tx=0x7f7cdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:28.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.035+0000 7f7cd5ffb700 1 -- 172.21.15.97:0/3253816048 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7cd80560f0 con 0x7f7ce015aaf0 2023-07-06T03:46:28.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.230+0000 7f7ce79ff700 1 -- 172.21.15.97:0/3253816048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7ce015d280 con 0x7f7ce015aaf0 2023-07-06T03:46:28.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.796+0000 7f7cd5ffb700 1 -- 172.21.15.97:0/3253816048 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7cd8059710 con 0x7f7ce015aaf0 2023-07-06T03:46:28.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 -- 172.21.15.97:0/3253816048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc406ace0 msgr2=0x7f7cc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:28.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc406ace0 0x7f7cc406d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc009ce0 tx=0x7f7cdc008040 comp rx=0 tx=0).stop 2023-07-06T03:46:28.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 -- 172.21.15.97:0/3253816048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ce015aaf0 msgr2=0x7f7ce015e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:28.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ce015aaf0 0x7f7ce015e250 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8002230 tx=0x7f7cd8001360 comp rx=0 tx=0).stop 2023-07-06T03:46:28.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 -- 172.21.15.97:0/3253816048 shutdown_connections 2023-07-06T03:46:28.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc406ace0 0x7f7cc406d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:28.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ce015aaf0 0x7f7ce015e250 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:28.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc004d90 0x7f7ce01f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:28.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 --2- 172.21.15.97:0/3253816048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ce015d5b0 0x7f7ce015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:28.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 -- 172.21.15.97:0/3253816048 >> 172.21.15.97:0/3253816048 conn(0x7f7ce006a5b0 msgr2=0x7f7ce00b7430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:28.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 -- 172.21.15.97:0/3253816048 shutdown_connections 2023-07-06T03:46:28.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:28.799+0000 7f7ce79ff700 1 -- 172.21.15.97:0/3253816048 wait complete. 2023-07-06T03:46:28.803 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-07-06T03:46:28.821 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T03:46:28.821 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T03:46:28.822 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T03:46:28.822 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:46:28.822 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T03:46:28.823 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T03:46:28.823 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T03:46:28.823 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:46:28.823 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T03:46:28.824 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T03:46:28.825 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T03:46:28.825 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:46:28.825 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T03:46:28.825 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T03:46:28.826 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T03:46:28.826 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:46:28.835 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T03:46:28.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T03:46:28.837 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T03:46:28.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T03:46:28.845 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T03:46:28.845 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T03:46:28.846 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T03:46:28.846 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T03:46:28.880 INFO:tasks.ceph.mds.a:Started 2023-07-06T03:46:28.881 DEBUG:teuthology.parallel:result is None 2023-07-06T03:46:28.883 INFO:tasks.ceph.mds.c:Started 2023-07-06T03:46:28.884 DEBUG:teuthology.parallel:result is None 2023-07-06T03:46:28.890 INFO:tasks.ceph.mds.b:Started 2023-07-06T03:46:28.890 DEBUG:teuthology.parallel:result is None 2023-07-06T03:46:28.893 INFO:tasks.ceph.mds.d:Started 2023-07-06T03:46:28.894 DEBUG:teuthology.parallel:result is None 2023-07-06T03:46:28.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T03:46:29.028 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:46:29.032 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:46:29.040 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-07-06T03:46:29.041 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:46:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.103+0000 7ff9e6fff700 1 Processor -- start 2023-07-06T03:46:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.104+0000 7ff9e6fff700 1 -- start start 2023-07-06T03:46:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.105+0000 7ff9e6fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff9e00c3cd0 0x7ff9e00cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.105+0000 7ff9e6fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff9e0159440 0x7ff9e00cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.105+0000 7ff9e6fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e00ca260 con 0x7ff9e0159070 2023-07-06T03:46:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.105+0000 7ff9e6fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e00c99c0 con 0x7ff9e0159440 2023-07-06T03:46:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.105+0000 7ff9e6fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e00c9ca0 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.105+0000 7ff9e559c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff9e0159440 0x7ff9e00cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.105+0000 7ff9e559c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff9e0159440 0x7ff9e00cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45438/0 (socket says 172.21.15.97:45438) 2023-07-06T03:46:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.105+0000 7ff9e4d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff9e0159070 0x7ff9e00c8f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54580/0 (socket says 172.21.15.97:54580) 2023-07-06T03:46:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.105+0000 7ff9e559c700 1 -- 172.21.15.97:0/1942361774 learned_addr learned my addr 172.21.15.97:0/1942361774 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.105+0000 7ff9dffff700 1 --2- 172.21.15.97:0/1942361774 >> v2:172.21.15.146:3300/0 conn(0x7ff9e00c3cd0 0x7ff9e00cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:29.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.106+0000 7ff9e559c700 1 -- 172.21.15.97:0/1942361774 >> v2:172.21.15.146:3300/0 conn(0x7ff9e00c3cd0 msgr2=0x7ff9e00cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:29.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.106+0000 7ff9e559c700 1 --2- 172.21.15.97:0/1942361774 >> v2:172.21.15.146:3300/0 conn(0x7ff9e00c3cd0 0x7ff9e00cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:29.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.106+0000 7ff9e559c700 1 -- 172.21.15.97:0/1942361774 >> v1:172.21.15.97:6789/0 conn(0x7ff9e0159070 legacy=0x7ff9e00c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:29.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.106+0000 7ff9e559c700 1 -- 172.21.15.97:0/1942361774 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00c9f80 con 0x7ff9e0159440 2023-07-06T03:46:29.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.106+0000 7ff9df7fe700 1 -- 172.21.15.97:0/1942361774 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287043431 0 0) 0x7ff9e00ca260 con 0x7ff9e0159070 2023-07-06T03:46:29.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.106+0000 7ff9e559c700 1 --2- 172.21.15.97:0/1942361774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e0159440 0x7ff9e00cca00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00cf9f0 tx=0x7ff9d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5662c54983ac4c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:29.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.106+0000 7ff9df7fe700 1 -- 172.21.15.97:0/1942361774 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800e070 con 0x7ff9e0159440 2023-07-06T03:46:29.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.106+0000 7ff9df7fe700 1 -- 172.21.15.97:0/1942361774 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d800b910 con 0x7ff9e0159440 2023-07-06T03:46:29.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.106+0000 7ff9df7fe700 1 -- 172.21.15.97:0/1942361774 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800bcc0 con 0x7ff9e0159440 2023-07-06T03:46:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.107+0000 7ff9e6fff700 1 -- 172.21.15.97:0/1942361774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e0159440 msgr2=0x7ff9e00cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.107+0000 7ff9e6fff700 1 --2- 172.21.15.97:0/1942361774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e0159440 0x7ff9e00cca00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00cf9f0 tx=0x7ff9d800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.107+0000 7ff9e6fff700 1 -- 172.21.15.97:0/1942361774 shutdown_connections 2023-07-06T03:46:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.107+0000 7ff9e6fff700 1 --2- 172.21.15.97:0/1942361774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e0159440 0x7ff9e00cca00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.107+0000 7ff9e6fff700 1 --2- 172.21.15.97:0/1942361774 >> v2:172.21.15.146:3300/0 conn(0x7ff9e00c3cd0 0x7ff9e00cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.107+0000 7ff9e6fff700 1 -- 172.21.15.97:0/1942361774 >> 172.21.15.97:0/1942361774 conn(0x7ff9e006a060 msgr2=0x7ff9e006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.107+0000 7ff9e6fff700 1 -- 172.21.15.97:0/1942361774 shutdown_connections 2023-07-06T03:46:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.107+0000 7ff9e6fff700 1 -- 172.21.15.97:0/1942361774 wait complete. 2023-07-06T03:46:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.108+0000 7ff9e6fff700 1 Processor -- start 2023-07-06T03:46:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.109+0000 7ff9e6fff700 1 -- start start 2023-07-06T03:46:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e6fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e00c3cd0 0x7ff9e00accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e6fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e0159070 0x7ff9e00b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e6fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e0159440 0x7ff9e00b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e6fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e014bea0 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e6fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0155da0 con 0x7ff9e0159070 2023-07-06T03:46:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e6fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00b1ec0 con 0x7ff9e0159440 2023-07-06T03:46:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e4d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e00c3cd0 0x7ff9e00accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e4d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e00c3cd0 0x7ff9e00accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45440/0 (socket says 172.21.15.97:45440) 2023-07-06T03:46:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e4d9b700 1 -- 172.21.15.97:0/3235255188 learned_addr learned my addr 172.21.15.97:0/3235255188 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e4d9b700 1 -- 172.21.15.97:0/3235255188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e0159440 msgr2=0x7ff9e00b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e4d9b700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e0159440 0x7ff9e00b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9dffff700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e0159070 0x7ff9e00b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e4d9b700 1 -- 172.21.15.97:0/3235255188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e0159070 msgr2=0x7ff9e00b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e4d9b700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e0159070 0x7ff9e00b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9e4d9b700 1 -- 172.21.15.97:0/3235255188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d8009b30 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.110+0000 7ff9dffff700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e0159070 0x7ff9e00b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.111+0000 7ff9e4d9b700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e00c3cd0 0x7ff9e00accf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40011e0 tx=0x7ff9d400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.111+0000 7ff9dd7fa700 1 -- 172.21.15.97:0/3235255188 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4004c80 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.111+0000 7ff9dd7fa700 1 -- 172.21.15.97:0/3235255188 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d4004e00 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.111+0000 7ff9dd7fa700 1 -- 172.21.15.97:0/3235255188 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4007310 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.111+0000 7ff9e6fff700 1 -- 172.21.15.97:0/3235255188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e00b21a0 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.111+0000 7ff9e6fff700 1 -- 172.21.15.97:0/3235255188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e00b2760 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.112+0000 7ff9e6fff700 1 -- 172.21.15.97:0/3235255188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e00b2980 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.113+0000 7ff9dd7fa700 1 -- 172.21.15.97:0/3235255188 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff9d4007470 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.114+0000 7ff9dd7fa700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff9c406ace0 0x7ff9c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.114+0000 7ff9dd7fa700 1 -- 172.21.15.97:0/3235255188 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff9d4088f70 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.114+0000 7ff9dffff700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff9c406ace0 0x7ff9c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:29.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.115+0000 7ff9dffff700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff9c406ace0 0x7ff9c406d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9e01f5080 tx=0x7ff9cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:29.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.118+0000 7ff9dd7fa700 1 -- 172.21.15.97:0/3235255188 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff9d40bb310 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.329+0000 7ff9e6fff700 1 -- 172.21.15.97:0/3235255188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff9e004e340 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.817+0000 7ff9dd7fa700 1 -- 172.21.15.97:0/3235255188 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff9d4054320 con 0x7ff9e00c3cd0 2023-07-06T03:46:29.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 -- 172.21.15.97:0/3235255188 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff9c406ace0 msgr2=0x7ff9c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff9c406ace0 0x7ff9c406d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9e01f5080 tx=0x7ff9cc008040 comp rx=0 tx=0).stop 2023-07-06T03:46:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 -- 172.21.15.97:0/3235255188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e00c3cd0 msgr2=0x7ff9e00accf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e00c3cd0 0x7ff9e00accf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40011e0 tx=0x7ff9d400cae0 comp rx=0 tx=0).stop 2023-07-06T03:46:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 -- 172.21.15.97:0/3235255188 shutdown_connections 2023-07-06T03:46:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff9c406ace0 0x7ff9c406d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9e0159440 0x7ff9e00b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e00c3cd0 0x7ff9e00accf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 --2- 172.21.15.97:0/3235255188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9e0159070 0x7ff9e00b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:29.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 -- 172.21.15.97:0/3235255188 >> 172.21.15.97:0/3235255188 conn(0x7ff9e006a060 msgr2=0x7ff9e00cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:29.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 -- 172.21.15.97:0/3235255188 shutdown_connections 2023-07-06T03:46:29.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:29.820+0000 7ff9e6fff700 1 -- 172.21.15.97:0/3235255188 wait complete. 2023-07-06T03:46:29.826 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-07-06T03:46:29.842 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T03:46:30.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.036+0000 7fc10b1ff700 1 Processor -- start 2023-07-06T03:46:30.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.036+0000 7fc10b1ff700 1 -- start start 2023-07-06T03:46:30.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.037+0000 7fc10b1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc1040b8410 0x7fc104167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:30.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.037+0000 7fc10b1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1040cdae0 con 0x7fc1040cd390 2023-07-06T03:46:30.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.037+0000 7fc10b1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1040b6640 con 0x7fc10415c520 2023-07-06T03:46:30.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.037+0000 7fc10b1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1040b6920 con 0x7fc1040b8410 2023-07-06T03:46:30.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.037+0000 7fc103fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc1040cd390 0x7fc104160890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54560/0 (socket says 172.21.15.97:54560) 2023-07-06T03:46:30.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.037+0000 7fc103fff700 1 -- 172.21.15.97:0/585564555 learned_addr learned my addr 172.21.15.97:0/585564555 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:30.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.038+0000 7fc10979c700 1 --2- 172.21.15.97:0/585564555 >> v2:172.21.15.146:3300/0 conn(0x7fc1040b8410 0x7fc104167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:30.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.038+0000 7fc1037fe700 1 -- 172.21.15.97:0/585564555 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065983495 0 0) 0x7fc1040cdae0 con 0x7fc1040cd390 2023-07-06T03:46:30.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.038+0000 7fc1037fe700 1 -- 172.21.15.97:0/585564555 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f40035b0 con 0x7fc1040cd390 2023-07-06T03:46:30.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.038+0000 7fc1037fe700 1 -- 172.21.15.97:0/585564555 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843289891 0 0) 0x7fc1040b6640 con 0x7fc10415c520 2023-07-06T03:46:30.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.038+0000 7fc1037fe700 1 -- 172.21.15.97:0/585564555 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1040cdae0 con 0x7fc10415c520 2023-07-06T03:46:30.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.038+0000 7fc10979c700 1 -- 172.21.15.97:0/585564555 >> v1:172.21.15.146:6789/0 conn(0x7fc10415c520 legacy=0x7fc104160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.038+0000 7fc10979c700 1 -- 172.21.15.97:0/585564555 >> v1:172.21.15.97:6789/0 conn(0x7fc1040cd390 legacy=0x7fc104160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.038+0000 7fc10979c700 1 -- 172.21.15.97:0/585564555 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1040b6c00 con 0x7fc1040b8410 2023-07-06T03:46:30.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.038+0000 7fc10979c700 1 --2- 172.21.15.97:0/585564555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1040b8410 0x7fc104167510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009d90 tx=0x7fc0fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40f7c691923f72b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:30.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.039+0000 7fc1037fe700 1 -- 172.21.15.97:0/585564555 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00e070 con 0x7fc1040b8410 2023-07-06T03:46:30.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.039+0000 7fc1037fe700 1 -- 172.21.15.97:0/585564555 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc00bb20 con 0x7fc1040b8410 2023-07-06T03:46:30.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.039+0000 7fc1037fe700 1 -- 172.21.15.97:0/585564555 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc012430 con 0x7fc1040b8410 2023-07-06T03:46:30.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.039+0000 7fc10b1ff700 1 -- 172.21.15.97:0/585564555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1040b8410 msgr2=0x7fc104167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.039+0000 7fc10b1ff700 1 --2- 172.21.15.97:0/585564555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1040b8410 0x7fc104167510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009d90 tx=0x7fc0fc00b330 comp rx=0 tx=0).stop 2023-07-06T03:46:30.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.039+0000 7fc10b1ff700 1 -- 172.21.15.97:0/585564555 shutdown_connections 2023-07-06T03:46:30.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.039+0000 7fc10b1ff700 1 --2- 172.21.15.97:0/585564555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1040b8410 0x7fc104167510 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.039+0000 7fc10b1ff700 1 -- 172.21.15.97:0/585564555 >> 172.21.15.97:0/585564555 conn(0x7fc104071790 msgr2=0x7fc104071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:30.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.039+0000 7fc10b1ff700 1 -- 172.21.15.97:0/585564555 shutdown_connections 2023-07-06T03:46:30.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.039+0000 7fc10b1ff700 1 -- 172.21.15.97:0/585564555 wait complete. 2023-07-06T03:46:30.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.041+0000 7fc10b1ff700 1 Processor -- start 2023-07-06T03:46:30.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.041+0000 7fc10b1ff700 1 -- start start 2023-07-06T03:46:30.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.042+0000 7fc10b1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1040b8410 0x7fc10414d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:30.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.042+0000 7fc10b1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1040cd390 0x7fc1041f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:30.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.042+0000 7fc10b1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10415c520 0x7fc10415d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:30.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.042+0000 7fc10b1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040d1a60 con 0x7fc10415c520 2023-07-06T03:46:30.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.042+0000 7fc10b1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10415e8e0 con 0x7fc1040b8410 2023-07-06T03:46:30.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.042+0000 7fc10b1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040d0de0 con 0x7fc1040cd390 2023-07-06T03:46:30.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.042+0000 7fc10979c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10415c520 0x7fc10415d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:30.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.042+0000 7fc10979c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10415c520 0x7fc10415d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47878/0 (socket says 172.21.15.97:47878) 2023-07-06T03:46:30.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.042+0000 7fc10979c700 1 -- 172.21.15.97:0/1483294949 learned_addr learned my addr 172.21.15.97:0/1483294949 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:30.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.042+0000 7fc108f9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1040b8410 0x7fc10414d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:30.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.043+0000 7fc103fff700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1040cd390 0x7fc1041f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:30.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.043+0000 7fc10979c700 1 -- 172.21.15.97:0/1483294949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1040cd390 msgr2=0x7fc1041f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.043+0000 7fc10979c700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1040cd390 0x7fc1041f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.043+0000 7fc10979c700 1 -- 172.21.15.97:0/1483294949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1040b8410 msgr2=0x7fc10414d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.043+0000 7fc10979c700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1040b8410 0x7fc10414d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.043+0000 7fc10979c700 1 -- 172.21.15.97:0/1483294949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc009a70 con 0x7fc10415c520 2023-07-06T03:46:30.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.043+0000 7fc10979c700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10415c520 0x7fc10415d940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc10406c240 tx=0x7fc0fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.043+0000 7fc1017fa700 1 -- 172.21.15.97:0/1483294949 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc012430 con 0x7fc10415c520 2023-07-06T03:46:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.043+0000 7fc1017fa700 1 -- 172.21.15.97:0/1483294949 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc020070 con 0x7fc10415c520 2023-07-06T03:46:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.043+0000 7fc1017fa700 1 -- 172.21.15.97:0/1483294949 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc01ba70 con 0x7fc10415c520 2023-07-06T03:46:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.044+0000 7fc10b1ff700 1 -- 172.21.15.97:0/1483294949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1040d10c0 con 0x7fc10415c520 2023-07-06T03:46:30.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.044+0000 7fc10b1ff700 1 -- 172.21.15.97:0/1483294949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1040d1680 con 0x7fc10415c520 2023-07-06T03:46:30.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.045+0000 7fc10b1ff700 1 -- 172.21.15.97:0/1483294949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10404e340 con 0x7fc10415c520 2023-07-06T03:46:30.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.046+0000 7fc1017fa700 1 -- 172.21.15.97:0/1483294949 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc0fc01bbd0 con 0x7fc10415c520 2023-07-06T03:46:30.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.047+0000 7fc1017fa700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0e806ad10 0x7fc0e806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:30.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.047+0000 7fc1017fa700 1 -- 172.21.15.97:0/1483294949 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc0fc022070 con 0x7fc10415c520 2023-07-06T03:46:30.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.048+0000 7fc108f9b700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0e806ad10 0x7fc0e806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:30.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.048+0000 7fc108f9b700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0e806ad10 0x7fc0e806d1c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80011e0 tx=0x7fc0f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:30.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.052+0000 7fc1017fa700 1 -- 172.21.15.97:0/1483294949 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc0fc056450 con 0x7fc10415c520 2023-07-06T03:46:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.324+0000 7fc10b1ff700 1 -- 172.21.15.97:0/1483294949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc1040d1840 con 0x7fc10415c520 2023-07-06T03:46:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.325+0000 7fc1017fa700 1 -- 172.21.15.97:0/1483294949 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fc0fc02c020 con 0x7fc10415c520 2023-07-06T03:46:30.327 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:30.328 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:46:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-07-06T03:46:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.329 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-07-06T03:46:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkczuvGRAAUKjfmMyKOTCYzOQmtf3n3A==", 2023-07-06T03:46:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:46:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:46:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:46:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-07-06T03:46:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkFHvUMBAA2WgMcUUf+Qi8T2ZjhfWaHA==", 2023-07-06T03:46:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:46:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:46:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:46:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-07-06T03:46:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZk0N4+JRAA3tjqoIBPpZ6C7TMmOOg7lw==", 2023-07-06T03:46:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:46:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:46:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:46:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.335 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-07-06T03:46:30.335 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkJnHUABAAwIMH2DmoHaQN3frJCgRiQw==", 2023-07-06T03:46:30.335 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:46:30.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:46:30.336 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:46:30.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.336 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.336 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-07-06T03:46:30.336 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCLOKZkShOIABAAzIBHDcNkmnDOLSY7Qdb+FA==", 2023-07-06T03:46:30.336 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.337 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:46:30.337 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:46:30.337 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.337 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-07-06T03:46:30.337 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCOOKZkq1t7AxAAwihuhimbjitZDeki1QuMug==", 2023-07-06T03:46:30.338 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.338 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:46:30.338 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:46:30.338 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.339 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-07-06T03:46:30.339 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCROKZkdYv4BhAApTHg/pmlC0lO7hVTkimEgw==", 2023-07-06T03:46:30.339 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.339 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:46:30.339 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:46:30.339 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.340 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-07-06T03:46:30.340 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCUOKZkqhuMChAAsV9xG6g1dofrpWt3F8tw5w==", 2023-07-06T03:46:30.340 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.340 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:46:30.341 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:46:30.341 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.341 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.341 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-07-06T03:46:30.341 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCaOKZk1Q5aIBAABU1+afwIihm+g4xylryz/A==", 2023-07-06T03:46:30.341 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.342 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:46:30.342 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:46:30.342 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.342 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-07-06T03:46:30.343 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCdOKZkRskIJBAA+821aoG98aEV7h4FTINQFA==", 2023-07-06T03:46:30.343 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.343 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:46:30.343 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:46:30.343 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.343 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.344 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-07-06T03:46:30.344 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCgOKZk9qnMJxAATcWG+a/9Hl1AuIL/TSrdEA==", 2023-07-06T03:46:30.344 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.344 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:46:30.344 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:46:30.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.345 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-07-06T03:46:30.345 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCjOKZk6wJ/KxAAWnZxt7h7OJhQ+bqdel4+iQ==", 2023-07-06T03:46:30.345 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.346 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:46:30.346 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:46:30.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.346 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-07-06T03:46:30.346 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg==", 2023-07-06T03:46:30.347 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.347 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:46:30.347 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:46:30.347 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:46:30.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:46:30.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.348 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-07-06T03:46:30.348 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkq4/pGRAAucHmR1eBA8MNMy3/l7tl0g==", 2023-07-06T03:46:30.348 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.348 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:46:30.348 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:46:30.348 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:46:30.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:46:30.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.349 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-07-06T03:46:30.349 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkhL5fEhAAWhw4IbCPqTO2sXPIcpTVHA==", 2023-07-06T03:46:30.349 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.349 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:46:30.349 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:46:30.350 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:46:30.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:46:30.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.350 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-07-06T03:46:30.350 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkO81dIRAAffjVyvsb/ygBtx8UrYQK0w==", 2023-07-06T03:46:30.350 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.351 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:46:30.351 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:46:30.351 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:46:30.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:46:30.351 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.352 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-07-06T03:46:30.352 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCFOKZk6uVlHRAA92RDhdDKW/q65SEOQMWgiA==", 2023-07-06T03:46:30.352 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.352 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:46:30.352 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:46:30.352 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:46:30.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.353 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.353 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mds", 2023-07-06T03:46:30.353 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCrOKZkusz4LRAAtj78623PURCTwQWfkKuspA==", 2023-07-06T03:46:30.353 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.353 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mds" 2023-07-06T03:46:30.353 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.354 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mgr", 2023-07-06T03:46:30.354 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCrOKZk4tf4LRAAfcSkxHXuSQUEwvY3NYo1DA==", 2023-07-06T03:46:30.354 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.354 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mgr" 2023-07-06T03:46:30.354 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.355 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-osd", 2023-07-06T03:46:30.355 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCrOKZkAeH4LRAAvdMGpbxGfqOHjeN6Bla4gg==", 2023-07-06T03:46:30.355 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.355 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-osd" 2023-07-06T03:46:30.355 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.356 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd", 2023-07-06T03:46:30.356 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCrOKZknOn4LRAABwNvCsVHwS2RwPaLutJw/A==", 2023-07-06T03:46:30.356 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.356 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd" 2023-07-06T03:46:30.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.357 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-07-06T03:46:30.357 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCrOKZkxvL4LRAAB75B0eFUX5EhiE+wkAZ4rw==", 2023-07-06T03:46:30.357 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.357 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-07-06T03:46:30.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.358 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rgw", 2023-07-06T03:46:30.358 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCrOKZkhvv4LRAAz6xwAKxfyMrEkpKeBNhKQA==", 2023-07-06T03:46:30.358 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.358 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rgw" 2023-07-06T03:46:30.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.358 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-07-06T03:46:30.359 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkT4ALEhAAXwwkcodQU9meJA0tfseJCg==", 2023-07-06T03:46:30.359 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.359 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:46:30.359 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:46:30.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:30.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:30.360 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-07-06T03:46:30.360 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkfnoEChAAuRxqXqPI33dgWkBJoUMh3w==", 2023-07-06T03:46:30.360 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:46:30.360 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:46:30.360 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:46:30.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:46:30.360 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.360 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:30.361 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:46:30.361 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:46:30.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.327+0000 7fc10b1ff700 1 -- 172.21.15.97:0/1483294949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0e806ad10 msgr2=0x7fc0e806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.327+0000 7fc10b1ff700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0e806ad10 0x7fc0e806d1c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80011e0 tx=0x7fc0f8009040 comp rx=0 tx=0).stop 2023-07-06T03:46:30.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.327+0000 7fc10b1ff700 1 -- 172.21.15.97:0/1483294949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10415c520 msgr2=0x7fc10415d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.327+0000 7fc10b1ff700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10415c520 0x7fc10415d940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc10406c240 tx=0x7fc0fc004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.327+0000 7fc10b1ff700 1 -- 172.21.15.97:0/1483294949 shutdown_connections 2023-07-06T03:46:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.327+0000 7fc10b1ff700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0e806ad10 0x7fc0e806d1c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.327+0000 7fc10b1ff700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10415c520 0x7fc10415d940 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.327+0000 7fc10b1ff700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1040b8410 0x7fc10414d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.327+0000 7fc10b1ff700 1 --2- 172.21.15.97:0/1483294949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1040cd390 0x7fc1041f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.327+0000 7fc10b1ff700 1 -- 172.21.15.97:0/1483294949 >> 172.21.15.97:0/1483294949 conn(0x7fc104071790 msgr2=0x7fc1040c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:30.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.328+0000 7fc10b1ff700 1 -- 172.21.15.97:0/1483294949 shutdown_connections 2023-07-06T03:46:30.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.328+0000 7fc10b1ff700 1 -- 172.21.15.97:0/1483294949 wait complete. 2023-07-06T03:46:30.364 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-07-06T03:46:30.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.556+0000 7f93015ff700 1 Processor -- start 2023-07-06T03:46:30.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.556+0000 7f93015ff700 1 -- start start 2023-07-06T03:46:30.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.557+0000 7f93015ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92fc06b360 0x7f92fc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:30.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.557+0000 7f93015ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92fc0cff90 con 0x7f92fc15c610 2023-07-06T03:46:30.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.557+0000 7f93015ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92fc0b5650 con 0x7f92fc0d2210 2023-07-06T03:46:30.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.557+0000 7f93015ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92fc0d0780 con 0x7f92fc06b360 2023-07-06T03:46:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.558+0000 7f92fa7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92fc06b360 0x7f92fc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.558+0000 7f92faffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f92fc15c610 0x7f92fc0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54574/0 (socket says 172.21.15.97:54574) 2023-07-06T03:46:30.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.558+0000 7f92faffd700 1 -- 172.21.15.97:0/1892241039 learned_addr learned my addr 172.21.15.97:0/1892241039 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.558+0000 7f92fa7fc700 1 -- 172.21.15.97:0/1892241039 >> v1:172.21.15.146:6789/0 conn(0x7f92fc0d2210 legacy=0x7f92fc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.558+0000 7f92fa7fc700 1 -- 172.21.15.97:0/1892241039 >> v1:172.21.15.97:6789/0 conn(0x7f92fc15c610 legacy=0x7f92fc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.558+0000 7f92fa7fc700 1 -- 172.21.15.97:0/1892241039 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc0b5400 con 0x7f92fc06b360 2023-07-06T03:46:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.558+0000 7f92f9ffb700 1 -- 172.21.15.97:0/1892241039 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240588145 0 0) 0x7f92fc0cff90 con 0x7f92fc15c610 2023-07-06T03:46:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.558+0000 7f92fa7fc700 1 --2- 172.21.15.97:0/1892241039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc06b360 0x7f92fc0b4ec0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009840 tx=0x7f92f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9b054ec28910fc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.558+0000 7f92f9ffb700 1 -- 172.21.15.97:0/1892241039 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0011830 con 0x7f92fc06b360 2023-07-06T03:46:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.559+0000 7f92f9ffb700 1 -- 172.21.15.97:0/1892241039 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f0011990 con 0x7f92fc06b360 2023-07-06T03:46:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.559+0000 7f92f9ffb700 1 -- 172.21.15.97:0/1892241039 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0011d20 con 0x7f92fc06b360 2023-07-06T03:46:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.559+0000 7f93015ff700 1 -- 172.21.15.97:0/1892241039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc06b360 msgr2=0x7f92fc0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.559+0000 7f93015ff700 1 --2- 172.21.15.97:0/1892241039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc06b360 0x7f92fc0b4ec0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009840 tx=0x7f92f0009b50 comp rx=0 tx=0).stop 2023-07-06T03:46:30.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.559+0000 7f93015ff700 1 -- 172.21.15.97:0/1892241039 shutdown_connections 2023-07-06T03:46:30.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.559+0000 7f93015ff700 1 --2- 172.21.15.97:0/1892241039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc06b360 0x7f92fc0b4ec0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.559+0000 7f93015ff700 1 -- 172.21.15.97:0/1892241039 >> 172.21.15.97:0/1892241039 conn(0x7f92fc06a6c0 msgr2=0x7f92fc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:30.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.559+0000 7f93015ff700 1 -- 172.21.15.97:0/1892241039 shutdown_connections 2023-07-06T03:46:30.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.559+0000 7f93015ff700 1 -- 172.21.15.97:0/1892241039 wait complete. 2023-07-06T03:46:30.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.561+0000 7f93015ff700 1 Processor -- start 2023-07-06T03:46:30.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.561+0000 7f93015ff700 1 -- start start 2023-07-06T03:46:30.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.562+0000 7f93015ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc06b360 0x7f92fc0cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:30.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.562+0000 7f93015ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92fc0d2210 0x7f92fc0ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:30.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.562+0000 7f93015ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92fc15c610 0x7f92fc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:30.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.562+0000 7f93015ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc151790 con 0x7f92fc06b360 2023-07-06T03:46:30.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.562+0000 7f93015ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc0cad50 con 0x7f92fc0d2210 2023-07-06T03:46:30.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.562+0000 7f93015ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc15cb60 con 0x7f92fc15c610 2023-07-06T03:46:30.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.562+0000 7f92faffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc06b360 0x7f92fc0cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:30.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.562+0000 7f92faffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc06b360 0x7f92fc0cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47904/0 (socket says 172.21.15.97:47904) 2023-07-06T03:46:30.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.562+0000 7f92faffd700 1 -- 172.21.15.97:0/1358500074 learned_addr learned my addr 172.21.15.97:0/1358500074 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:30.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.562+0000 7f92fa7fc700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92fc0d2210 0x7f92fc0ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:30.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.562+0000 7f92fb7fe700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92fc15c610 0x7f92fc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:30.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f92faffd700 1 -- 172.21.15.97:0/1358500074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92fc15c610 msgr2=0x7f92fc0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f92faffd700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92fc15c610 0x7f92fc0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f92faffd700 1 -- 172.21.15.97:0/1358500074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92fc0d2210 msgr2=0x7f92fc0ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f92faffd700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92fc0d2210 0x7f92fc0ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f92faffd700 1 -- 172.21.15.97:0/1358500074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00094a0 con 0x7f92fc06b360 2023-07-06T03:46:30.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f92fa7fc700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92fc0d2210 0x7f92fc0ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:30.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f92faffd700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc06b360 0x7f92fc0cc840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00c520 tx=0x7f92ec00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:30.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f92e37fe700 1 -- 172.21.15.97:0/1358500074 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec004c80 con 0x7f92fc06b360 2023-07-06T03:46:30.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f92e37fe700 1 -- 172.21.15.97:0/1358500074 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ec004e00 con 0x7f92fc06b360 2023-07-06T03:46:30.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f92e37fe700 1 -- 172.21.15.97:0/1358500074 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec007310 con 0x7f92fc06b360 2023-07-06T03:46:30.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f93015ff700 1 -- 172.21.15.97:0/1358500074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92fc0cafd0 con 0x7f92fc06b360 2023-07-06T03:46:30.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.563+0000 7f93015ff700 1 -- 172.21.15.97:0/1358500074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92fc0cb590 con 0x7f92fc06b360 2023-07-06T03:46:30.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.564+0000 7f93015ff700 1 -- 172.21.15.97:0/1358500074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92fc15cd80 con 0x7f92fc06b360 2023-07-06T03:46:30.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.565+0000 7f92e37fe700 1 -- 172.21.15.97:0/1358500074 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f92ec007470 con 0x7f92fc06b360 2023-07-06T03:46:30.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.566+0000 7f92e37fe700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92e806ace0 0x7f92e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:30.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.566+0000 7f92e37fe700 1 -- 172.21.15.97:0/1358500074 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f92ec088f00 con 0x7f92fc06b360 2023-07-06T03:46:30.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.566+0000 7f92fa7fc700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92e806ace0 0x7f92e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:30.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.567+0000 7f92fa7fc700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92e806ace0 0x7f92e806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f92f00090b0 tx=0x7f92f001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:30.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.570+0000 7f92e37fe700 1 -- 172.21.15.97:0/1358500074 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f92fc15cd80 con 0x7f92fc06b360 2023-07-06T03:46:30.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.848+0000 7f93015ff700 1 -- 172.21.15.97:0/1358500074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f92fc0b19c0 con 0x7f92fc06b360 2023-07-06T03:46:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.852+0000 7f92e37fe700 1 -- 172.21.15.97:0/1358500074 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f92ec054260 con 0x7f92fc06b360 2023-07-06T03:46:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 -- 172.21.15.97:0/1358500074 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92e806ace0 msgr2=0x7f92e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92e806ace0 0x7f92e806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f92f00090b0 tx=0x7f92f001c040 comp rx=0 tx=0).stop 2023-07-06T03:46:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 -- 172.21.15.97:0/1358500074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc06b360 msgr2=0x7f92fc0cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc06b360 0x7f92fc0cc840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00c520 tx=0x7f92ec00c8e0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 -- 172.21.15.97:0/1358500074 shutdown_connections 2023-07-06T03:46:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f92e806ace0 0x7f92e806d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92fc15c610 0x7f92fc0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc06b360 0x7f92fc0cc840 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 --2- 172.21.15.97:0/1358500074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92fc0d2210 0x7f92fc0ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 -- 172.21.15.97:0/1358500074 >> 172.21.15.97:0/1358500074 conn(0x7f92fc06a6c0 msgr2=0x7f92fc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 -- 172.21.15.97:0/1358500074 shutdown_connections 2023-07-06T03:46:30.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:30.854+0000 7f93015ff700 1 -- 172.21.15.97:0/1358500074 wait complete. 2023-07-06T03:46:30.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-07-06T03:46:31.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.070+0000 7f0593dff700 1 Processor -- start 2023-07-06T03:46:31.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.070+0000 7f0593dff700 1 -- start start 2023-07-06T03:46:31.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f0593dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f058c0b8890 0x7f058c0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f0593dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f058c0cdb10 0x7f058c0d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f0593dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058c0b5280 con 0x7f058c0b4d70 2023-07-06T03:46:31.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f0593dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058c0d0de0 con 0x7f058c0b8890 2023-07-06T03:46:31.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f0593dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0d10c0 con 0x7f058c0cdb10 2023-07-06T03:46:31.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f0591b9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f058c0b8890 0x7f058c0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f0591b9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f058c0b8890 0x7f058c0b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47912/0 (socket says 172.21.15.97:47912) 2023-07-06T03:46:31.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f0591b9b700 1 -- 172.21.15.97:0/2515666264 learned_addr learned my addr 172.21.15.97:0/2515666264 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:31.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f059239c700 1 --2- 172.21.15.97:0/2515666264 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f058c0cdb10 0x7f058c0d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f0591b9b700 1 -- 172.21.15.97:0/2515666264 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f058c0cdb10 msgr2=0x7f058c0d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:31.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f0591b9b700 1 --2- 172.21.15.97:0/2515666264 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f058c0cdb10 0x7f058c0d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.072+0000 7f0591b9b700 1 -- 172.21.15.97:0/2515666264 >> v1:172.21.15.146:6789/0 conn(0x7f058c0b4d70 legacy=0x7f058c0cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:31.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.073+0000 7f0591b9b700 1 -- 172.21.15.97:0/2515666264 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c0d13a0 con 0x7f058c0b8890 2023-07-06T03:46:31.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.073+0000 7f059239c700 1 --2- 172.21.15.97:0/2515666264 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f058c0cdb10 0x7f058c0d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:31.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.073+0000 7f0591b9b700 1 --2- 172.21.15.97:0/2515666264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058c0b8890 0x7f058c0b4830 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0588009920 tx=0x7f0588009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c41b6b5abb5a4f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:31.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.073+0000 7f0590b99700 1 -- 172.21.15.97:0/2515666264 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588004230 con 0x7f058c0b8890 2023-07-06T03:46:31.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.073+0000 7f0590b99700 1 -- 172.21.15.97:0/2515666264 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0588004390 con 0x7f058c0b8890 2023-07-06T03:46:31.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.073+0000 7f0590b99700 1 -- 172.21.15.97:0/2515666264 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588009e70 con 0x7f058c0b8890 2023-07-06T03:46:31.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.073+0000 7f0593dff700 1 -- 172.21.15.97:0/2515666264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058c0b8890 msgr2=0x7f058c0b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:31.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.073+0000 7f0593dff700 1 --2- 172.21.15.97:0/2515666264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058c0b8890 0x7f058c0b4830 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0588009920 tx=0x7f0588009c30 comp rx=0 tx=0).stop 2023-07-06T03:46:31.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.074+0000 7f0593dff700 1 -- 172.21.15.97:0/2515666264 shutdown_connections 2023-07-06T03:46:31.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.074+0000 7f0593dff700 1 --2- 172.21.15.97:0/2515666264 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f058c0cdb10 0x7f058c0d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.074+0000 7f0593dff700 1 --2- 172.21.15.97:0/2515666264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058c0b8890 0x7f058c0b4830 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.074+0000 7f0593dff700 1 -- 172.21.15.97:0/2515666264 >> 172.21.15.97:0/2515666264 conn(0x7f058c06b2d0 msgr2=0x7f058c0cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:31.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.074+0000 7f0593dff700 1 -- 172.21.15.97:0/2515666264 shutdown_connections 2023-07-06T03:46:31.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.074+0000 7f0593dff700 1 -- 172.21.15.97:0/2515666264 wait complete. 2023-07-06T03:46:31.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.075+0000 7f0593dff700 1 Processor -- start 2023-07-06T03:46:31.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.075+0000 7f0593dff700 1 -- start start 2023-07-06T03:46:31.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.076+0000 7f0593dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f058c0b4d70 0x7f058c155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.076+0000 7f0593dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058c0b8890 0x7f058c0c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.076+0000 7f0593dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f058c0cdb10 0x7f058c0c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.076+0000 7f0593dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c1559b0 con 0x7f058c0b8890 2023-07-06T03:46:31.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.076+0000 7f0593dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0c9e80 con 0x7f058c0b4d70 2023-07-06T03:46:31.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.076+0000 7f0593dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0ca0d0 con 0x7f058c0cdb10 2023-07-06T03:46:31.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.077+0000 7f059139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058c0b8890 0x7f058c0c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.077+0000 7f059139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058c0b8890 0x7f058c0c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47918/0 (socket says 172.21.15.97:47918) 2023-07-06T03:46:31.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.077+0000 7f059139a700 1 -- 172.21.15.97:0/1456164763 learned_addr learned my addr 172.21.15.97:0/1456164763 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:31.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.077+0000 7f0591b9b700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f058c0b4d70 0x7f058c155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.077+0000 7f059239c700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f058c0cdb10 0x7f058c0c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.077+0000 7f059139a700 1 -- 172.21.15.97:0/1456164763 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f058c0cdb10 msgr2=0x7f058c0c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:31.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.077+0000 7f059139a700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f058c0cdb10 0x7f058c0c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.077+0000 7f059139a700 1 -- 172.21.15.97:0/1456164763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f058c0b4d70 msgr2=0x7f058c155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:31.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.077+0000 7f059139a700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f058c0b4d70 0x7f058c155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.077+0000 7f059139a700 1 -- 172.21.15.97:0/1456164763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0588009550 con 0x7f058c0b8890 2023-07-06T03:46:31.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.077+0000 7f059139a700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058c0b8890 0x7f058c0c8da0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f057c00b450 tx=0x7f057c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:31.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.078+0000 7f05827fc700 1 -- 172.21.15.97:0/1456164763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c011630 con 0x7f058c0b8890 2023-07-06T03:46:31.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.078+0000 7f05827fc700 1 -- 172.21.15.97:0/1456164763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c011790 con 0x7f058c0b8890 2023-07-06T03:46:31.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.078+0000 7f05827fc700 1 -- 172.21.15.97:0/1456164763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c011b40 con 0x7f058c0b8890 2023-07-06T03:46:31.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.078+0000 7f0593dff700 1 -- 172.21.15.97:0/1456164763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058c0ca3b0 con 0x7f058c0b8890 2023-07-06T03:46:31.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.078+0000 7f0593dff700 1 -- 172.21.15.97:0/1456164763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058c0ca970 con 0x7f058c0b8890 2023-07-06T03:46:31.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.079+0000 7f0593dff700 1 -- 172.21.15.97:0/1456164763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058c0b5be0 con 0x7f058c0b8890 2023-07-06T03:46:31.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.080+0000 7f05827fc700 1 -- 172.21.15.97:0/1456164763 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f057c011ca0 con 0x7f058c0b8890 2023-07-06T03:46:31.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.081+0000 7f05827fc700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f057006ace0 0x7f057006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.081+0000 7f05827fc700 1 -- 172.21.15.97:0/1456164763 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f057c088d00 con 0x7f058c0b8890 2023-07-06T03:46:31.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.081+0000 7f0591b9b700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f057006ace0 0x7f057006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.082+0000 7f0591b9b700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f057006ace0 0x7f057006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0588009130 tx=0x7f0588008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:31.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.085+0000 7f05827fc700 1 -- 172.21.15.97:0/1456164763 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f058c0b5be0 con 0x7f058c0b8890 2023-07-06T03:46:31.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.361+0000 7f0593dff700 1 -- 172.21.15.97:0/1456164763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f058c0b5be0 con 0x7f058c0b8890 2023-07-06T03:46:31.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.365+0000 7f05827fc700 1 -- 172.21.15.97:0/1456164763 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f058c0b5be0 con 0x7f058c0b8890 2023-07-06T03:46:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 -- 172.21.15.97:0/1456164763 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f057006ace0 msgr2=0x7f057006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f057006ace0 0x7f057006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0588009130 tx=0x7f0588008030 comp rx=0 tx=0).stop 2023-07-06T03:46:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 -- 172.21.15.97:0/1456164763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058c0b8890 msgr2=0x7f058c0c8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058c0b8890 0x7f058c0c8da0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f057c00b450 tx=0x7f057c00b760 comp rx=0 tx=0).stop 2023-07-06T03:46:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 -- 172.21.15.97:0/1456164763 shutdown_connections 2023-07-06T03:46:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f057006ace0 0x7f057006d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f058c0cdb10 0x7f058c0c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f058c0b4d70 0x7f058c155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 --2- 172.21.15.97:0/1456164763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f058c0b8890 0x7f058c0c8da0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 -- 172.21.15.97:0/1456164763 >> 172.21.15.97:0/1456164763 conn(0x7f058c06b2d0 msgr2=0x7f058c0b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 -- 172.21.15.97:0/1456164763 shutdown_connections 2023-07-06T03:46:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.367+0000 7f0593dff700 1 -- 172.21.15.97:0/1456164763 wait complete. 2023-07-06T03:46:31.386 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-07-06T03:46:31.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.579+0000 7ff710dff700 1 Processor -- start 2023-07-06T03:46:31.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.579+0000 7ff710dff700 1 -- start start 2023-07-06T03:46:31.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.581+0000 7ff710dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff70c15c610 0x7ff70c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.581+0000 7ff710dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff70c06b360 0x7ff70c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.581+0000 7ff710dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70c0cff90 con 0x7ff70c0d2210 2023-07-06T03:46:31.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.581+0000 7ff710dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70c0d0780 con 0x7ff70c06b360 2023-07-06T03:46:31.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.581+0000 7ff710dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c15c9e0 con 0x7ff70c15c610 2023-07-06T03:46:31.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.581+0000 7ff70ad9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff70c0d2210 0x7ff70c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54584/0 (socket says 172.21.15.97:54584) 2023-07-06T03:46:31.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.581+0000 7ff70ad9d700 1 -- 172.21.15.97:0/2204496604 learned_addr learned my addr 172.21.15.97:0/2204496604 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.581+0000 7ff70a59c700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff70c15c610 0x7ff70c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.581+0000 7ff709d9b700 1 --2- 172.21.15.97:0/2204496604 >> v2:172.21.15.146:3300/0 conn(0x7ff70c06b360 0x7ff70c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.581+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290030797 0 0) 0x7ff70c0cff90 con 0x7ff70c0d2210 2023-07-06T03:46:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.581+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f40035b0 con 0x7ff70c0d2210 2023-07-06T03:46:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3657875191 0 0) 0x7ff6f40035b0 con 0x7ff70c0d2210 2023-07-06T03:46:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff70c15c610 msgr2=0x7ff70c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff70c15c610 0x7ff70c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 >> v2:172.21.15.146:3300/0 conn(0x7ff70c06b360 msgr2=0x7ff70c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 --2- 172.21.15.97:0/2204496604 >> v2:172.21.15.146:3300/0 conn(0x7ff70c06b360 0x7ff70c0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c0b5650 con 0x7ff70c0d2210 2023-07-06T03:46:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7ff7040019f0 con 0x7ff70c0d2210 2023-07-06T03:46:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 >> v1:172.21.15.97:6789/0 conn(0x7ff70c0d2210 legacy=0x7ff70c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6f4003a30 0x7ff6f4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff70c15c610 0x7ff6f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6f4004d90 0x7ff6f4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0d0780 con 0x7ff6f4003a30 2023-07-06T03:46:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c15c9e0 con 0x7ff6f4004d90 2023-07-06T03:46:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f400ba10 con 0x7ff70c15c610 2023-07-06T03:46:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70a59c700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6f4003a30 0x7ff6f4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff709d9b700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff70c15c610 0x7ff6f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.582+0000 7ff70ad9d700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6f4004d90 0x7ff6f4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff70a59c700 1 -- 172.21.15.97:0/2204496604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff70c15c610 msgr2=0x7ff6f4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:31.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff70a59c700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff70c15c610 0x7ff6f4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff70a59c700 1 -- 172.21.15.97:0/2204496604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6f4004d90 msgr2=0x7ff6f4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:31.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff70a59c700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6f4004d90 0x7ff6f4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff70a59c700 1 -- 172.21.15.97:0/2204496604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f40035b0 con 0x7ff6f4003a30 2023-07-06T03:46:31.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff70a59c700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6f4003a30 0x7ff6f4003e40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009b60 tx=0x7ff6f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6459602cbe3ae06 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:31.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00e660 con 0x7ff6f4003a30 2023-07-06T03:46:31.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc00e7c0 con 0x7ff6f4003a30 2023-07-06T03:46:31.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff70959a700 1 -- 172.21.15.97:0/2204496604 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00eb30 con 0x7ff6f4003a30 2023-07-06T03:46:31.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff710dff700 1 -- 172.21.15.97:0/2204496604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6f4003a30 msgr2=0x7ff6f4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:31.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff710dff700 1 --2- 172.21.15.97:0/2204496604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6f4003a30 0x7ff6f4003e40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009b60 tx=0x7ff6f4005880 comp rx=0 tx=0).stop 2023-07-06T03:46:31.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.583+0000 7ff70ad9d700 1 -- 172.21.15.97:0/2204496604 reap_dead start 2023-07-06T03:46:31.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.584+0000 7ff710dff700 1 -- 172.21.15.97:0/2204496604 shutdown_connections 2023-07-06T03:46:31.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.584+0000 7ff710dff700 1 -- 172.21.15.97:0/2204496604 >> 172.21.15.97:0/2204496604 conn(0x7ff70c06a6c0 msgr2=0x7ff70c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:31.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.584+0000 7ff710dff700 1 -- 172.21.15.97:0/2204496604 shutdown_connections 2023-07-06T03:46:31.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.584+0000 7ff710dff700 1 -- 172.21.15.97:0/2204496604 wait complete. 2023-07-06T03:46:31.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.585+0000 7ff710dff700 1 Processor -- start 2023-07-06T03:46:31.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.586+0000 7ff710dff700 1 -- start start 2023-07-06T03:46:31.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.586+0000 7ff710dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6f4003a30 0x7ff70c14f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.586+0000 7ff710dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff70c0cb130 0x7ff70c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.586+0000 7ff710dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff70c0b8560 0x7ff70c0b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.586+0000 7ff710dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0ccba0 con 0x7ff70c0b8560 2023-07-06T03:46:31.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.586+0000 7ff710dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0cb5c0 con 0x7ff70c0cb130 2023-07-06T03:46:31.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.586+0000 7ff710dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0cb8a0 con 0x7ff6f4003a30 2023-07-06T03:46:31.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff70ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff70c0b8560 0x7ff70c0b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff70ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff70c0b8560 0x7ff70c0b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47942/0 (socket says 172.21.15.97:47942) 2023-07-06T03:46:31.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff70ad9d700 1 -- 172.21.15.97:0/4242673414 learned_addr learned my addr 172.21.15.97:0/4242673414 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:31.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff70a59c700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6f4003a30 0x7ff70c14f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff709d9b700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff70c0cb130 0x7ff70c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff70ad9d700 1 -- 172.21.15.97:0/4242673414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6f4003a30 msgr2=0x7ff70c14f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:31.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff70ad9d700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6f4003a30 0x7ff70c14f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff70ad9d700 1 -- 172.21.15.97:0/4242673414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff70c0cb130 msgr2=0x7ff70c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:31.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff70ad9d700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff70c0cb130 0x7ff70c0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff70ad9d700 1 -- 172.21.15.97:0/4242673414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c0cbb80 con 0x7ff70c0b8560 2023-07-06T03:46:31.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff709d9b700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff70c0cb130 0x7ff70c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:31.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.587+0000 7ff70ad9d700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff70c0b8560 0x7ff70c0b6010 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0b5830 tx=0x7ff704007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:31.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.588+0000 7ff6faffd700 1 -- 172.21.15.97:0/4242673414 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70400ec60 con 0x7ff70c0b8560 2023-07-06T03:46:31.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.588+0000 7ff6faffd700 1 -- 172.21.15.97:0/4242673414 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70400edc0 con 0x7ff70c0b8560 2023-07-06T03:46:31.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.588+0000 7ff6faffd700 1 -- 172.21.15.97:0/4242673414 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704013660 con 0x7ff70c0b8560 2023-07-06T03:46:31.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.588+0000 7ff710dff700 1 -- 172.21.15.97:0/4242673414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70c0cbe60 con 0x7ff70c0b8560 2023-07-06T03:46:31.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.588+0000 7ff710dff700 1 -- 172.21.15.97:0/4242673414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70c0b6e00 con 0x7ff70c0b8560 2023-07-06T03:46:31.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.589+0000 7ff710dff700 1 -- 172.21.15.97:0/4242673414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70c0d0480 con 0x7ff70c0b8560 2023-07-06T03:46:31.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.590+0000 7ff6faffd700 1 -- 172.21.15.97:0/4242673414 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff7040137c0 con 0x7ff70c0b8560 2023-07-06T03:46:31.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.591+0000 7ff6faffd700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff6ec06ace0 0x7ff6ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:31.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.591+0000 7ff6faffd700 1 -- 172.21.15.97:0/4242673414 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff704089730 con 0x7ff70c0b8560 2023-07-06T03:46:31.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.591+0000 7ff70a59c700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff6ec06ace0 0x7ff6ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:31.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.591+0000 7ff70a59c700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff6ec06ace0 0x7ff6ec06d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0090b0 tx=0x7ff6fc01f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:31.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.595+0000 7ff6faffd700 1 -- 172.21.15.97:0/4242673414 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff70c0d0480 con 0x7ff70c0b8560 2023-07-06T03:46:31.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.868+0000 7ff710dff700 1 -- 172.21.15.97:0/4242673414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7ff70c0d0480 con 0x7ff70c0b8560 2023-07-06T03:46:31.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.871+0000 7ff6faffd700 1 -- 172.21.15.97:0/4242673414 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff70c0d0480 con 0x7ff70c0b8560 2023-07-06T03:46:31.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 -- 172.21.15.97:0/4242673414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff6ec06ace0 msgr2=0x7ff6ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:31.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff6ec06ace0 0x7ff6ec06d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0090b0 tx=0x7ff6fc01f040 comp rx=0 tx=0).stop 2023-07-06T03:46:31.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 -- 172.21.15.97:0/4242673414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff70c0b8560 msgr2=0x7ff70c0b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:31.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff70c0b8560 0x7ff70c0b6010 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0b5830 tx=0x7ff704007c90 comp rx=0 tx=0).stop 2023-07-06T03:46:31.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 -- 172.21.15.97:0/4242673414 shutdown_connections 2023-07-06T03:46:31.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff6ec06ace0 0x7ff6ec06d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff70c0b8560 0x7ff70c0b6010 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6f4003a30 0x7ff70c14f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 --2- 172.21.15.97:0/4242673414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff70c0cb130 0x7ff70c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:31.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 -- 172.21.15.97:0/4242673414 >> 172.21.15.97:0/4242673414 conn(0x7ff70c06a6c0 msgr2=0x7ff70c0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:31.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 -- 172.21.15.97:0/4242673414 shutdown_connections 2023-07-06T03:46:31.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:31.874+0000 7ff710dff700 1 -- 172.21.15.97:0/4242673414 wait complete. 2023-07-06T03:46:31.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-07-06T03:46:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.089+0000 7fbf1abff700 1 Processor -- start 2023-07-06T03:46:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.089+0000 7fbf1abff700 1 -- start start 2023-07-06T03:46:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.090+0000 7fbf1abff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf1415c610 0x7fbf140b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.090+0000 7fbf1abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1406b360 0x7fbf140b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.090+0000 7fbf1abff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbf140d2210 0x7fbf140d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.090+0000 7fbf1abff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf140d0780 con 0x7fbf1415c610 2023-07-06T03:46:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.090+0000 7fbf1abff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1415c9e0 con 0x7fbf140d2210 2023-07-06T03:46:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.090+0000 7fbf1abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf140b5650 con 0x7fbf1406b360 2023-07-06T03:46:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.090+0000 7fbf1899b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf1415c610 0x7fbf140b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.090+0000 7fbf1899b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf1415c610 0x7fbf140b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47944/0 (socket says 172.21.15.97:47944) 2023-07-06T03:46:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.090+0000 7fbf1899b700 1 -- 172.21.15.97:0/3592697318 learned_addr learned my addr 172.21.15.97:0/3592697318 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.090+0000 7fbf13fff700 1 --2- 172.21.15.97:0/3592697318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1406b360 0x7fbf140b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.090+0000 7fbf1919c700 1 --2- 172.21.15.97:0/3592697318 >> v2:172.21.15.146:3300/0 conn(0x7fbf140d2210 0x7fbf140d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.091+0000 7fbf1899b700 1 -- 172.21.15.97:0/3592697318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1406b360 msgr2=0x7fbf140b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.091+0000 7fbf1899b700 1 --2- 172.21.15.97:0/3592697318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1406b360 0x7fbf140b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.091+0000 7fbf1899b700 1 -- 172.21.15.97:0/3592697318 >> v2:172.21.15.146:3300/0 conn(0x7fbf140d2210 msgr2=0x7fbf140d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.091+0000 7fbf1899b700 1 --2- 172.21.15.97:0/3592697318 >> v2:172.21.15.146:3300/0 conn(0x7fbf140d2210 0x7fbf140d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.091+0000 7fbf1899b700 1 -- 172.21.15.97:0/3592697318 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf140d01c0 con 0x7fbf1415c610 2023-07-06T03:46:32.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.091+0000 7fbf1899b700 1 --2- 172.21.15.97:0/3592697318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf1415c610 0x7fbf140b47b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbf140b1b70 tx=0x7fbf08009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf6befd630faf9c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:32.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.091+0000 7fbf137fe700 1 -- 172.21.15.97:0/3592697318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0800b860 con 0x7fbf1415c610 2023-07-06T03:46:32.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.091+0000 7fbf137fe700 1 -- 172.21.15.97:0/3592697318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0800b9c0 con 0x7fbf1415c610 2023-07-06T03:46:32.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.091+0000 7fbf137fe700 1 -- 172.21.15.97:0/3592697318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0800bd50 con 0x7fbf1415c610 2023-07-06T03:46:32.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.092+0000 7fbf1abff700 1 -- 172.21.15.97:0/3592697318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf1415c610 msgr2=0x7fbf140b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:32.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.092+0000 7fbf1abff700 1 --2- 172.21.15.97:0/3592697318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf1415c610 0x7fbf140b47b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbf140b1b70 tx=0x7fbf08009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.092+0000 7fbf1abff700 1 -- 172.21.15.97:0/3592697318 shutdown_connections 2023-07-06T03:46:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.092+0000 7fbf1abff700 1 --2- 172.21.15.97:0/3592697318 >> v2:172.21.15.146:3300/0 conn(0x7fbf140d2210 0x7fbf140d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.092+0000 7fbf1abff700 1 --2- 172.21.15.97:0/3592697318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf1415c610 0x7fbf140b47b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.092+0000 7fbf1abff700 1 --2- 172.21.15.97:0/3592697318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1406b360 0x7fbf140b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.092+0000 7fbf1abff700 1 -- 172.21.15.97:0/3592697318 >> 172.21.15.97:0/3592697318 conn(0x7fbf1406a6c0 msgr2=0x7fbf1406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.092+0000 7fbf1abff700 1 -- 172.21.15.97:0/3592697318 shutdown_connections 2023-07-06T03:46:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.092+0000 7fbf1abff700 1 -- 172.21.15.97:0/3592697318 wait complete. 2023-07-06T03:46:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.094+0000 7fbf1abff700 1 Processor -- start 2023-07-06T03:46:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.094+0000 7fbf1abff700 1 -- start start 2023-07-06T03:46:32.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1abff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf1406b360 0x7fbf140cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1abff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf140d2210 0x7fbf140cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1415c610 0x7fbf1415ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1abff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf140cb630 con 0x7fbf1406b360 2023-07-06T03:46:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1abff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf140cb910 con 0x7fbf140d2210 2023-07-06T03:46:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf140cbbf0 con 0x7fbf1415c610 2023-07-06T03:46:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1899b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf1406b360 0x7fbf140cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1899b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf1406b360 0x7fbf140cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47950/0 (socket says 172.21.15.97:47950) 2023-07-06T03:46:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1899b700 1 -- 172.21.15.97:0/392005788 learned_addr learned my addr 172.21.15.97:0/392005788 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1899b700 1 -- 172.21.15.97:0/392005788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1415c610 msgr2=0x7fbf1415ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1899b700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1415c610 0x7fbf1415ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1899b700 1 -- 172.21.15.97:0/392005788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf140d2210 msgr2=0x7fbf140cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1899b700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf140d2210 0x7fbf140cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.096+0000 7fbf1899b700 1 -- 172.21.15.97:0/392005788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf08009550 con 0x7fbf1406b360 2023-07-06T03:46:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.097+0000 7fbf1899b700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf1406b360 0x7fbf140cc6d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009130 tx=0x7fbf08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.097+0000 7fbf117fa700 1 -- 172.21.15.97:0/392005788 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf080044d0 con 0x7fbf1406b360 2023-07-06T03:46:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.097+0000 7fbf117fa700 1 -- 172.21.15.97:0/392005788 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0801e070 con 0x7fbf1406b360 2023-07-06T03:46:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.097+0000 7fbf117fa700 1 -- 172.21.15.97:0/392005788 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf080038c0 con 0x7fbf1406b360 2023-07-06T03:46:32.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.097+0000 7fbf1abff700 1 -- 172.21.15.97:0/392005788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf140cbed0 con 0x7fbf1406b360 2023-07-06T03:46:32.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.097+0000 7fbf1abff700 1 -- 172.21.15.97:0/392005788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf140c7eb0 con 0x7fbf1406b360 2023-07-06T03:46:32.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.098+0000 7fbf1abff700 1 -- 172.21.15.97:0/392005788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf140cba50 con 0x7fbf1406b360 2023-07-06T03:46:32.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.099+0000 7fbf117fa700 1 -- 172.21.15.97:0/392005788 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf08003a20 con 0x7fbf1406b360 2023-07-06T03:46:32.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.101+0000 7fbf117fa700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbef806ace0 0x7fbef806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.101+0000 7fbf117fa700 1 -- 172.21.15.97:0/392005788 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbf08020070 con 0x7fbf1406b360 2023-07-06T03:46:32.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.101+0000 7fbf13fff700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbef806ace0 0x7fbef806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:32.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.101+0000 7fbf13fff700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbef806ace0 0x7fbef806d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf140caf30 tx=0x7fbf00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:32.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.105+0000 7fbf117fa700 1 -- 172.21.15.97:0/392005788 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbf140cba50 con 0x7fbf1406b360 2023-07-06T03:46:32.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.377+0000 7fbf1abff700 1 -- 172.21.15.97:0/392005788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fbf140cba50 con 0x7fbf1406b360 2023-07-06T03:46:32.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.381+0000 7fbf117fa700 1 -- 172.21.15.97:0/392005788 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbf140cba50 con 0x7fbf1406b360 2023-07-06T03:46:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 -- 172.21.15.97:0/392005788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbef806ace0 msgr2=0x7fbef806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbef806ace0 0x7fbef806d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf140caf30 tx=0x7fbf00008040 comp rx=0 tx=0).stop 2023-07-06T03:46:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 -- 172.21.15.97:0/392005788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf1406b360 msgr2=0x7fbf140cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf1406b360 0x7fbf140cc6d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009130 tx=0x7fbf08004750 comp rx=0 tx=0).stop 2023-07-06T03:46:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 -- 172.21.15.97:0/392005788 shutdown_connections 2023-07-06T03:46:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbef806ace0 0x7fbef806d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbf1415c610 0x7fbf1415ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf1406b360 0x7fbf140cc6d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 --2- 172.21.15.97:0/392005788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf140d2210 0x7fbf140cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 -- 172.21.15.97:0/392005788 >> 172.21.15.97:0/392005788 conn(0x7fbf1406a6c0 msgr2=0x7fbf140d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 -- 172.21.15.97:0/392005788 shutdown_connections 2023-07-06T03:46:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.383+0000 7fbf1abff700 1 -- 172.21.15.97:0/392005788 wait complete. 2023-07-06T03:46:32.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-07-06T03:46:32.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.596+0000 7f79ecdff700 1 Processor -- start 2023-07-06T03:46:32.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.596+0000 7f79ecdff700 1 -- start start 2023-07-06T03:46:32.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.597+0000 7f79ecdff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79e815c610 0x7f79e80b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.597+0000 7f79ecdff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f79e806b360 0x7f79e80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.597+0000 7f79ecdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79e80d2210 0x7f79e80d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.597+0000 7f79ecdff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e80d0780 con 0x7f79e815c610 2023-07-06T03:46:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.597+0000 7f79ecdff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e815c9e0 con 0x7f79e806b360 2023-07-06T03:46:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.597+0000 7f79ecdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e80b5650 con 0x7f79e80d2210 2023-07-06T03:46:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.597+0000 7f79e659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79e815c610 0x7f79e80b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.597+0000 7f79e659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79e815c610 0x7f79e80b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47964/0 (socket says 172.21.15.97:47964) 2023-07-06T03:46:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.597+0000 7f79e659c700 1 -- 172.21.15.97:0/674525764 learned_addr learned my addr 172.21.15.97:0/674525764 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.598+0000 7f79e659c700 1 -- 172.21.15.97:0/674525764 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79e80d2210 msgr2=0x7f79e80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.598+0000 7f79e659c700 1 --2- 172.21.15.97:0/674525764 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79e80d2210 0x7f79e80d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.598+0000 7f79e659c700 1 -- 172.21.15.97:0/674525764 >> v2:172.21.15.146:3300/0 conn(0x7f79e806b360 msgr2=0x7f79e80b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:32.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.598+0000 7f79e5d9b700 1 --2- 172.21.15.97:0/674525764 >> v2:172.21.15.146:3300/0 conn(0x7f79e806b360 0x7f79e80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:32.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.598+0000 7f79e659c700 1 --2- 172.21.15.97:0/674525764 >> v2:172.21.15.146:3300/0 conn(0x7f79e806b360 0x7f79e80b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.598+0000 7f79e659c700 1 -- 172.21.15.97:0/674525764 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e80d01c0 con 0x7f79e815c610 2023-07-06T03:46:32.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.598+0000 7f79e5d9b700 1 --2- 172.21.15.97:0/674525764 >> v2:172.21.15.146:3300/0 conn(0x7f79e806b360 0x7f79e80b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:32.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.598+0000 7f79e659c700 1 --2- 172.21.15.97:0/674525764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e815c610 0x7f79e80b47b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f79e806a590 tx=0x7f79d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f5d0fdcb01c64c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:32.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.598+0000 7f79e559a700 1 -- 172.21.15.97:0/674525764 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d800b860 con 0x7f79e815c610 2023-07-06T03:46:32.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.598+0000 7f79e559a700 1 -- 172.21.15.97:0/674525764 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d800b9c0 con 0x7f79e815c610 2023-07-06T03:46:32.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.598+0000 7f79e559a700 1 -- 172.21.15.97:0/674525764 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d800bd50 con 0x7f79e815c610 2023-07-06T03:46:32.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.599+0000 7f79ecdff700 1 -- 172.21.15.97:0/674525764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e815c610 msgr2=0x7f79e80b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:32.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.599+0000 7f79ecdff700 1 --2- 172.21.15.97:0/674525764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e815c610 0x7f79e80b47b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f79e806a590 tx=0x7f79d8009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:32.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.599+0000 7f79ecdff700 1 -- 172.21.15.97:0/674525764 shutdown_connections 2023-07-06T03:46:32.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.599+0000 7f79ecdff700 1 --2- 172.21.15.97:0/674525764 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79e80d2210 0x7f79e80d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.599+0000 7f79ecdff700 1 --2- 172.21.15.97:0/674525764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e815c610 0x7f79e80b47b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.599+0000 7f79ecdff700 1 --2- 172.21.15.97:0/674525764 >> v2:172.21.15.146:3300/0 conn(0x7f79e806b360 0x7f79e80b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.599+0000 7f79ecdff700 1 -- 172.21.15.97:0/674525764 >> 172.21.15.97:0/674525764 conn(0x7f79e806a6c0 msgr2=0x7f79e806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:32.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.599+0000 7f79ecdff700 1 -- 172.21.15.97:0/674525764 shutdown_connections 2023-07-06T03:46:32.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.599+0000 7f79ecdff700 1 -- 172.21.15.97:0/674525764 wait complete. 2023-07-06T03:46:32.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.601+0000 7f79ecdff700 1 Processor -- start 2023-07-06T03:46:32.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.601+0000 7f79ecdff700 1 -- start start 2023-07-06T03:46:32.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79ecdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79e806b360 0x7f79e8157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79ecdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79e80d2210 0x7f79e80cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79ecdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e815c610 0x7f79e815ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79ecdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e80c8870 con 0x7f79e815c610 2023-07-06T03:46:32.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79ecdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e80c8b50 con 0x7f79e806b360 2023-07-06T03:46:32.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79ecdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e80c8e30 con 0x7f79e80d2210 2023-07-06T03:46:32.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79e6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e815c610 0x7f79e815ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:32.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79e6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e815c610 0x7f79e815ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47968/0 (socket says 172.21.15.97:47968) 2023-07-06T03:46:32.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79e6d9d700 1 -- 172.21.15.97:0/398952297 learned_addr learned my addr 172.21.15.97:0/398952297 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79e5d9b700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79e80d2210 0x7f79e80cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79e659c700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79e806b360 0x7f79e8157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79e6d9d700 1 -- 172.21.15.97:0/398952297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79e80d2210 msgr2=0x7f79e80cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79e6d9d700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79e80d2210 0x7f79e80cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79e6d9d700 1 -- 172.21.15.97:0/398952297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79e806b360 msgr2=0x7f79e8157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79e6d9d700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79e806b360 0x7f79e8157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.602+0000 7f79e6d9d700 1 -- 172.21.15.97:0/398952297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d8009550 con 0x7f79e815c610 2023-07-06T03:46:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.603+0000 7f79e6d9d700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e815c610 0x7f79e815ea10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f79e000b970 tx=0x7f79e000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.603+0000 7f79d6ffd700 1 -- 172.21.15.97:0/398952297 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000c9b0 con 0x7f79e815c610 2023-07-06T03:46:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.603+0000 7f79d6ffd700 1 -- 172.21.15.97:0/398952297 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e000cb30 con 0x7f79e815c610 2023-07-06T03:46:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.603+0000 7f79d6ffd700 1 -- 172.21.15.97:0/398952297 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0014440 con 0x7f79e815c610 2023-07-06T03:46:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.603+0000 7f79ecdff700 1 -- 172.21.15.97:0/398952297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e80c9110 con 0x7f79e815c610 2023-07-06T03:46:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.603+0000 7f79ecdff700 1 -- 172.21.15.97:0/398952297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e80ccb40 con 0x7f79e815c610 2023-07-06T03:46:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.604+0000 7f79ecdff700 1 -- 172.21.15.97:0/398952297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e80c89b0 con 0x7f79e815c610 2023-07-06T03:46:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.605+0000 7f79d6ffd700 1 -- 172.21.15.97:0/398952297 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79e001d3b0 con 0x7f79e815c610 2023-07-06T03:46:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.606+0000 7f79d6ffd700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79c806ace0 0x7f79c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.607+0000 7f79d6ffd700 1 -- 172.21.15.97:0/398952297 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f79e0006e40 con 0x7f79e815c610 2023-07-06T03:46:32.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.607+0000 7f79e659c700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79c806ace0 0x7f79c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:32.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.607+0000 7f79e659c700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79c806ace0 0x7f79c806d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f79d8000c00 tx=0x7f79d8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.611+0000 7f79d6ffd700 1 -- 172.21.15.97:0/398952297 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f79e80c89b0 con 0x7f79e815c610 2023-07-06T03:46:32.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.883+0000 7f79ecdff700 1 -- 172.21.15.97:0/398952297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f79e80c89b0 con 0x7f79e815c610 2023-07-06T03:46:32.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.887+0000 7f79d6ffd700 1 -- 172.21.15.97:0/398952297 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f79e80c89b0 con 0x7f79e815c610 2023-07-06T03:46:32.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 -- 172.21.15.97:0/398952297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79c806ace0 msgr2=0x7f79c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:32.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79c806ace0 0x7f79c806d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f79d8000c00 tx=0x7f79d8008030 comp rx=0 tx=0).stop 2023-07-06T03:46:32.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 -- 172.21.15.97:0/398952297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e815c610 msgr2=0x7f79e815ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e815c610 0x7f79e815ea10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f79e000b970 tx=0x7f79e000bd30 comp rx=0 tx=0).stop 2023-07-06T03:46:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 -- 172.21.15.97:0/398952297 shutdown_connections 2023-07-06T03:46:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79c806ace0 0x7f79c806d190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79e815c610 0x7f79e815ea10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79e806b360 0x7f79e8157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 --2- 172.21.15.97:0/398952297 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79e80d2210 0x7f79e80cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 -- 172.21.15.97:0/398952297 >> 172.21.15.97:0/398952297 conn(0x7f79e806a6c0 msgr2=0x7f79e80b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:32.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 -- 172.21.15.97:0/398952297 shutdown_connections 2023-07-06T03:46:32.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:32.889+0000 7f79ecdff700 1 -- 172.21.15.97:0/398952297 wait complete. 2023-07-06T03:46:32.909 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-07-06T03:46:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.104+0000 7febb11ff700 1 Processor -- start 2023-07-06T03:46:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.104+0000 7febb11ff700 1 -- start start 2023-07-06T03:46:33.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.105+0000 7febb11ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7febac158b20 0x7febac0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.105+0000 7febb11ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7febac06cab0 0x7febac0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.105+0000 7febb11ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febac0b0560 0x7febac0b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.105+0000 7febb11ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac0ce820 con 0x7febac158b20 2023-07-06T03:46:33.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.105+0000 7febb11ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac0b0f20 con 0x7febac06cab0 2023-07-06T03:46:33.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.105+0000 7febb11ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac158ef0 con 0x7febac0b0560 2023-07-06T03:46:33.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.105+0000 7febaad9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7febac158b20 0x7febac0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.105+0000 7febaad9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7febac158b20 0x7febac0cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47980/0 (socket says 172.21.15.97:47980) 2023-07-06T03:46:33.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.105+0000 7febaad9d700 1 -- 172.21.15.97:0/1211728310 learned_addr learned my addr 172.21.15.97:0/1211728310 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:33.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.105+0000 7febab59e700 1 --2- 172.21.15.97:0/1211728310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febac0b0560 0x7febac0b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.106+0000 7febaa59c700 1 --2- 172.21.15.97:0/1211728310 >> v2:172.21.15.146:3300/0 conn(0x7febac06cab0 0x7febac0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.106+0000 7febaad9d700 1 -- 172.21.15.97:0/1211728310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febac0b0560 msgr2=0x7febac0b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:33.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.106+0000 7febaad9d700 1 --2- 172.21.15.97:0/1211728310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febac0b0560 0x7febac0b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.106+0000 7febaad9d700 1 -- 172.21.15.97:0/1211728310 >> v2:172.21.15.146:3300/0 conn(0x7febac06cab0 msgr2=0x7febac0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:33.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.106+0000 7febaad9d700 1 --2- 172.21.15.97:0/1211728310 >> v2:172.21.15.146:3300/0 conn(0x7febac06cab0 0x7febac0ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.106+0000 7febaad9d700 1 -- 172.21.15.97:0/1211728310 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac0ccbc0 con 0x7febac158b20 2023-07-06T03:46:33.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.106+0000 7febaa59c700 1 --2- 172.21.15.97:0/1211728310 >> v2:172.21.15.146:3300/0 conn(0x7febac06cab0 0x7febac0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:33.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.106+0000 7febaad9d700 1 --2- 172.21.15.97:0/1211728310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac158b20 0x7febac0cc680 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7febac06b040 tx=0x7feb9c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d2e5ae187ec969 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:33.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.106+0000 7feba9d9b700 1 -- 172.21.15.97:0/1211728310 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00f8a0 con 0x7febac158b20 2023-07-06T03:46:33.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.106+0000 7feba9d9b700 1 -- 172.21.15.97:0/1211728310 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c00fa00 con 0x7febac158b20 2023-07-06T03:46:33.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.106+0000 7feba9d9b700 1 -- 172.21.15.97:0/1211728310 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00fd90 con 0x7febac158b20 2023-07-06T03:46:33.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.107+0000 7febb11ff700 1 -- 172.21.15.97:0/1211728310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac158b20 msgr2=0x7febac0cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:33.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.107+0000 7febb11ff700 1 --2- 172.21.15.97:0/1211728310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac158b20 0x7febac0cc680 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7febac06b040 tx=0x7feb9c009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.107+0000 7febb11ff700 1 -- 172.21.15.97:0/1211728310 shutdown_connections 2023-07-06T03:46:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.107+0000 7febb11ff700 1 --2- 172.21.15.97:0/1211728310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febac0b0560 0x7febac0b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.107+0000 7febb11ff700 1 --2- 172.21.15.97:0/1211728310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac158b20 0x7febac0cc680 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.107+0000 7febb11ff700 1 --2- 172.21.15.97:0/1211728310 >> v2:172.21.15.146:3300/0 conn(0x7febac06cab0 0x7febac0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.107+0000 7febb11ff700 1 -- 172.21.15.97:0/1211728310 >> 172.21.15.97:0/1211728310 conn(0x7febac06a060 msgr2=0x7febac06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:33.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.107+0000 7febb11ff700 1 -- 172.21.15.97:0/1211728310 shutdown_connections 2023-07-06T03:46:33.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.107+0000 7febb11ff700 1 -- 172.21.15.97:0/1211728310 wait complete. 2023-07-06T03:46:33.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.109+0000 7febb11ff700 1 Processor -- start 2023-07-06T03:46:33.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.109+0000 7febb11ff700 1 -- start start 2023-07-06T03:46:33.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febb11ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febac06cab0 0x7febac150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febb11ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febac0b0560 0x7febac159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febb11ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac158b20 0x7febac0c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febb11ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0c6c70 con 0x7febac158b20 2023-07-06T03:46:33.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febb11ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0c6660 con 0x7febac06cab0 2023-07-06T03:46:33.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febb11ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac159d20 con 0x7febac0b0560 2023-07-06T03:46:33.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febab59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac158b20 0x7febac0c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febab59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac158b20 0x7febac0c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47992/0 (socket says 172.21.15.97:47992) 2023-07-06T03:46:33.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febab59e700 1 -- 172.21.15.97:0/1147486158 learned_addr learned my addr 172.21.15.97:0/1147486158 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:33.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febaad9d700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febac06cab0 0x7febac150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febaa59c700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febac0b0560 0x7febac159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febab59e700 1 -- 172.21.15.97:0/1147486158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febac0b0560 msgr2=0x7febac159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:33.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febab59e700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febac0b0560 0x7febac159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febab59e700 1 -- 172.21.15.97:0/1147486158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febac06cab0 msgr2=0x7febac150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:33.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febab59e700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febac06cab0 0x7febac150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.110+0000 7febab59e700 1 -- 172.21.15.97:0/1147486158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c009550 con 0x7febac158b20 2023-07-06T03:46:33.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.111+0000 7febaa59c700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febac0b0560 0x7febac159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:33.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.111+0000 7febaad9d700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febac06cab0 0x7febac150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:33.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.111+0000 7febab59e700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac158b20 0x7febac0c9fd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feba400efc0 tx=0x7feba400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:33.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.111+0000 7feb937fe700 1 -- 172.21.15.97:0/1147486158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba400cd40 con 0x7febac158b20 2023-07-06T03:46:33.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.111+0000 7feb937fe700 1 -- 172.21.15.97:0/1147486158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba400f040 con 0x7febac158b20 2023-07-06T03:46:33.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.111+0000 7febb11ff700 1 -- 172.21.15.97:0/1147486158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac0c6900 con 0x7febac158b20 2023-07-06T03:46:33.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.111+0000 7feb937fe700 1 -- 172.21.15.97:0/1147486158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba40146c0 con 0x7febac158b20 2023-07-06T03:46:33.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.111+0000 7febb11ff700 1 -- 172.21.15.97:0/1147486158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac15a000 con 0x7febac158b20 2023-07-06T03:46:33.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.112+0000 7febb11ff700 1 -- 172.21.15.97:0/1147486158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac04e340 con 0x7febac158b20 2023-07-06T03:46:33.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.113+0000 7feb937fe700 1 -- 172.21.15.97:0/1147486158 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feba4014820 con 0x7febac158b20 2023-07-06T03:46:33.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.114+0000 7feb937fe700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb9806ace0 0x7feb9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.114+0000 7feb937fe700 1 -- 172.21.15.97:0/1147486158 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7feba4089ac0 con 0x7febac158b20 2023-07-06T03:46:33.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.114+0000 7febaad9d700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb9806ace0 0x7feb9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.115+0000 7febaad9d700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb9806ace0 0x7feb9806d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb9c000c00 tx=0x7feb9c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:33.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.118+0000 7feb937fe700 1 -- 172.21.15.97:0/1147486158 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feba4058940 con 0x7febac158b20 2023-07-06T03:46:33.234 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:46:33.238 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:46:33.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.395+0000 7febb11ff700 1 -- 172.21.15.97:0/1147486158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7febac0c6db0 con 0x7febac158b20 2023-07-06T03:46:33.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.399+0000 7feb937fe700 1 -- 172.21.15.97:0/1147486158 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7feba400cea0 con 0x7febac158b20 2023-07-06T03:46:33.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.401+0000 7febb11ff700 1 -- 172.21.15.97:0/1147486158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb9806ace0 msgr2=0x7feb9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:33.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.401+0000 7febb11ff700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb9806ace0 0x7feb9806d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb9c000c00 tx=0x7feb9c008030 comp rx=0 tx=0).stop 2023-07-06T03:46:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.401+0000 7febb11ff700 1 -- 172.21.15.97:0/1147486158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac158b20 msgr2=0x7febac0c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.401+0000 7febb11ff700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac158b20 0x7febac0c9fd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feba400efc0 tx=0x7feba400c3a0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.402+0000 7febb11ff700 1 -- 172.21.15.97:0/1147486158 shutdown_connections 2023-07-06T03:46:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.402+0000 7febb11ff700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb9806ace0 0x7feb9806d190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.402+0000 7febb11ff700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac158b20 0x7febac0c9fd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.402+0000 7febb11ff700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7febac06cab0 0x7febac150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.402+0000 7febb11ff700 1 --2- 172.21.15.97:0/1147486158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7febac0b0560 0x7febac159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.402+0000 7febb11ff700 1 -- 172.21.15.97:0/1147486158 >> 172.21.15.97:0/1147486158 conn(0x7febac06a060 msgr2=0x7febac0c4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:33.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.402+0000 7febb11ff700 1 -- 172.21.15.97:0/1147486158 shutdown_connections 2023-07-06T03:46:33.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.402+0000 7febb11ff700 1 -- 172.21.15.97:0/1147486158 wait complete. 2023-07-06T03:46:33.422 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:33.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.615+0000 7fa7de3ff700 1 Processor -- start 2023-07-06T03:46:33.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.615+0000 7fa7de3ff700 1 -- start start 2023-07-06T03:46:33.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.616+0000 7fa7de3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7d8071a00 0x7fa7d806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.616+0000 7fa7de3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa7d815ce90 0x7fa7d8158d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.616+0000 7fa7de3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d806ddb0 con 0x7fa7d8158960 2023-07-06T03:46:33.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.616+0000 7fa7de3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d806bbf0 con 0x7fa7d815ce90 2023-07-06T03:46:33.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.616+0000 7fa7de3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d806bed0 con 0x7fa7d8071a00 2023-07-06T03:46:33.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.616+0000 7fa7d7fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa7d8158960 0x7fa7d80712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54596/0 (socket says 172.21.15.97:54596) 2023-07-06T03:46:33.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.616+0000 7fa7d7fff700 1 -- 172.21.15.97:0/3164420401 learned_addr learned my addr 172.21.15.97:0/3164420401 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:33.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.616+0000 7fa7d77fe700 1 --2- 172.21.15.97:0/3164420401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7d8071a00 0x7fa7d806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.616+0000 7fa7dc99c700 1 --2- 172.21.15.97:0/3164420401 >> v2:172.21.15.146:3300/0 conn(0x7fa7d815ce90 0x7fa7d8158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.617+0000 7fa7d6ffd700 1 -- 172.21.15.97:0/3164420401 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032720190 0 0) 0x7fa7d806ddb0 con 0x7fa7d8158960 2023-07-06T03:46:33.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.617+0000 7fa7d6ffd700 1 -- 172.21.15.97:0/3164420401 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c00035b0 con 0x7fa7d8158960 2023-07-06T03:46:33.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.617+0000 7fa7dc99c700 1 -- 172.21.15.97:0/3164420401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7d8071a00 msgr2=0x7fa7d806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:33.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.617+0000 7fa7dc99c700 1 --2- 172.21.15.97:0/3164420401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7d8071a00 0x7fa7d806d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.617+0000 7fa7dc99c700 1 -- 172.21.15.97:0/3164420401 >> v1:172.21.15.97:6789/0 conn(0x7fa7d8158960 legacy=0x7fa7d80712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:33.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.617+0000 7fa7dc99c700 1 -- 172.21.15.97:0/3164420401 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d806c1b0 con 0x7fa7d815ce90 2023-07-06T03:46:33.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.617+0000 7fa7d77fe700 1 --2- 172.21.15.97:0/3164420401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7d8071a00 0x7fa7d806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:46:33.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.617+0000 7fa7dc99c700 1 --2- 172.21.15.97:0/3164420401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7d815ce90 0x7fa7d8158d30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009d90 tx=0x7fa7d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4584ff80a731ea8d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:33.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.617+0000 7fa7d6ffd700 1 -- 172.21.15.97:0/3164420401 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d000e070 con 0x7fa7d815ce90 2023-07-06T03:46:33.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.617+0000 7fa7d6ffd700 1 -- 172.21.15.97:0/3164420401 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d000bb20 con 0x7fa7d815ce90 2023-07-06T03:46:33.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.617+0000 7fa7d6ffd700 1 -- 172.21.15.97:0/3164420401 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0012430 con 0x7fa7d815ce90 2023-07-06T03:46:33.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.618+0000 7fa7de3ff700 1 -- 172.21.15.97:0/3164420401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7d815ce90 msgr2=0x7fa7d8158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:33.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.618+0000 7fa7de3ff700 1 --2- 172.21.15.97:0/3164420401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7d815ce90 0x7fa7d8158d30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009d90 tx=0x7fa7d000b330 comp rx=0 tx=0).stop 2023-07-06T03:46:33.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.618+0000 7fa7de3ff700 1 -- 172.21.15.97:0/3164420401 shutdown_connections 2023-07-06T03:46:33.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.618+0000 7fa7de3ff700 1 --2- 172.21.15.97:0/3164420401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7d815ce90 0x7fa7d8158d30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.618+0000 7fa7de3ff700 1 --2- 172.21.15.97:0/3164420401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7d8071a00 0x7fa7d806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.618+0000 7fa7de3ff700 1 -- 172.21.15.97:0/3164420401 >> 172.21.15.97:0/3164420401 conn(0x7fa7d80ce8c0 msgr2=0x7fa7d80cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:33.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.618+0000 7fa7de3ff700 1 -- 172.21.15.97:0/3164420401 shutdown_connections 2023-07-06T03:46:33.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.618+0000 7fa7de3ff700 1 -- 172.21.15.97:0/3164420401 wait complete. 2023-07-06T03:46:33.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.620+0000 7fa7de3ff700 1 Processor -- start 2023-07-06T03:46:33.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.620+0000 7fa7de3ff700 1 -- start start 2023-07-06T03:46:33.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7de3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7d8071a00 0x7fa7d81f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7de3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7d8158960 0x7fa7d814ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7de3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7d815ce90 0x7fa7d80aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7de3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d814f2b0 con 0x7fa7d8158960 2023-07-06T03:46:33.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7de3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d81f53c0 con 0x7fa7d815ce90 2023-07-06T03:46:33.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7de3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80aef90 con 0x7fa7d8071a00 2023-07-06T03:46:33.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7d77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7d8158960 0x7fa7d814ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7d77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7d8158960 0x7fa7d814ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47994/0 (socket says 172.21.15.97:47994) 2023-07-06T03:46:33.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7d77fe700 1 -- 172.21.15.97:0/171083023 learned_addr learned my addr 172.21.15.97:0/171083023 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:33.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7d7fff700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7d8071a00 0x7fa7d81f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7dc99c700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7d815ce90 0x7fa7d80aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7d77fe700 1 -- 172.21.15.97:0/171083023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7d8071a00 msgr2=0x7fa7d81f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:33.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7d77fe700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7d8071a00 0x7fa7d81f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7d77fe700 1 -- 172.21.15.97:0/171083023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7d815ce90 msgr2=0x7fa7d80aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:33.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7d77fe700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7d815ce90 0x7fa7d80aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7d77fe700 1 -- 172.21.15.97:0/171083023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc0094a0 con 0x7fa7d8158960 2023-07-06T03:46:33.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.621+0000 7fa7dc99c700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7d815ce90 0x7fa7d80aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:33.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.622+0000 7fa7d77fe700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7d8158960 0x7fa7d814ed70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00ebe0 tx=0x7fa7cc009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:33.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.622+0000 7fa7d4ff9700 1 -- 172.21.15.97:0/171083023 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00c910 con 0x7fa7d8158960 2023-07-06T03:46:33.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.622+0000 7fa7d4ff9700 1 -- 172.21.15.97:0/171083023 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc00ca70 con 0x7fa7d8158960 2023-07-06T03:46:33.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.622+0000 7fa7de3ff700 1 -- 172.21.15.97:0/171083023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d0009a70 con 0x7fa7d8158960 2023-07-06T03:46:33.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.622+0000 7fa7d4ff9700 1 -- 172.21.15.97:0/171083023 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00cde0 con 0x7fa7d8158960 2023-07-06T03:46:33.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.622+0000 7fa7de3ff700 1 -- 172.21.15.97:0/171083023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d80af540 con 0x7fa7d8158960 2023-07-06T03:46:33.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.623+0000 7fa7de3ff700 1 -- 172.21.15.97:0/171083023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d81e2620 con 0x7fa7d8158960 2023-07-06T03:46:33.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.624+0000 7fa7d4ff9700 1 -- 172.21.15.97:0/171083023 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7cc013470 con 0x7fa7d8158960 2023-07-06T03:46:33.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.625+0000 7fa7d4ff9700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7c406ace0 0x7fa7c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:33.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.625+0000 7fa7d4ff9700 1 -- 172.21.15.97:0/171083023 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa7cc089690 con 0x7fa7d8158960 2023-07-06T03:46:33.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.625+0000 7fa7d7fff700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7c406ace0 0x7fa7c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:33.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.626+0000 7fa7d7fff700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7c406ace0 0x7fa7c406d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80011e0 tx=0x7fa7c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:33.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.629+0000 7fa7d4ff9700 1 -- 172.21.15.97:0/171083023 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa7d81e2620 con 0x7fa7d8158960 2023-07-06T03:46:33.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.826+0000 7fa7de3ff700 1 -- 172.21.15.97:0/171083023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7d81e2620 con 0x7fa7d8158960 2023-07-06T03:46:33.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.827+0000 7fa7d4ff9700 1 -- 172.21.15.97:0/171083023 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+14665 (secure 0 0 0) 0x7fa7d81e2620 con 0x7fa7d8158960 2023-07-06T03:46:33.829 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:33.829 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":30,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:46:29.826769+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.829+0000 7fa7de3ff700 1 -- 172.21.15.97:0/171083023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7c406ace0 msgr2=0x7fa7c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.829+0000 7fa7de3ff700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7c406ace0 0x7fa7c406d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80011e0 tx=0x7fa7c8009040 comp rx=0 tx=0).stop 2023-07-06T03:46:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.829+0000 7fa7de3ff700 1 -- 172.21.15.97:0/171083023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7d8158960 msgr2=0x7fa7d814ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.829+0000 7fa7de3ff700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7d8158960 0x7fa7d814ed70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00ebe0 tx=0x7fa7cc009ec0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.830+0000 7fa7de3ff700 1 -- 172.21.15.97:0/171083023 shutdown_connections 2023-07-06T03:46:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.830+0000 7fa7de3ff700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa7c406ace0 0x7fa7c406d190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.830+0000 7fa7de3ff700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa7d815ce90 0x7fa7d80aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.830+0000 7fa7de3ff700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa7d8071a00 0x7fa7d81f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.830+0000 7fa7de3ff700 1 --2- 172.21.15.97:0/171083023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7d8158960 0x7fa7d814ed70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:33.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.830+0000 7fa7de3ff700 1 -- 172.21.15.97:0/171083023 >> 172.21.15.97:0/171083023 conn(0x7fa7d80ce8c0 msgr2=0x7fa7d815a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:33.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.830+0000 7fa7de3ff700 1 -- 172.21.15.97:0/171083023 shutdown_connections 2023-07-06T03:46:33.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:33.830+0000 7fa7de3ff700 1 -- 172.21.15.97:0/171083023 wait complete. 2023-07-06T03:46:33.850 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:46:33.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:46:34.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.047+0000 7f2ab69ff700 1 Processor -- start 2023-07-06T03:46:34.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.047+0000 7f2ab69ff700 1 -- start start 2023-07-06T03:46:34.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.048+0000 7f2ab69ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab015c610 0x7f2ab00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.048+0000 7f2ab69ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ab006b360 0x7f2ab00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.048+0000 7f2ab69ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab00cff90 con 0x7f2ab00d2210 2023-07-06T03:46:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.048+0000 7f2ab69ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00d0780 con 0x7f2ab006b360 2023-07-06T03:46:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.048+0000 7f2ab69ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab015c9e0 con 0x7f2ab015c610 2023-07-06T03:46:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.049+0000 7f2aaf7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ab006b360 0x7f2ab00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.049+0000 7f2aaf7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ab006b360 0x7f2ab00b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48010/0 (socket says 172.21.15.97:48010) 2023-07-06T03:46:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.049+0000 7f2aaf7fe700 1 -- 172.21.15.97:0/637995688 learned_addr learned my addr 172.21.15.97:0/637995688 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:34.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.049+0000 7f2aaffff700 1 --2- 172.21.15.97:0/637995688 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab015c610 0x7f2ab00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.049+0000 7f2aaf7fe700 1 -- 172.21.15.97:0/637995688 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab015c610 msgr2=0x7f2ab00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.049+0000 7f2aaf7fe700 1 --2- 172.21.15.97:0/637995688 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab015c610 0x7f2ab00b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.049+0000 7f2aaf7fe700 1 -- 172.21.15.97:0/637995688 >> v1:172.21.15.146:6789/0 conn(0x7f2ab00d2210 legacy=0x7f2ab00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.049+0000 7f2aaf7fe700 1 -- 172.21.15.97:0/637995688 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00b5650 con 0x7f2ab006b360 2023-07-06T03:46:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.049+0000 7f2aaf7fe700 1 --2- 172.21.15.97:0/637995688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab006b360 0x7f2ab00b4cf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009920 tx=0x7f2aa4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2abeb13feffe3d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.050+0000 7f2aaeffd700 1 -- 172.21.15.97:0/637995688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4011830 con 0x7f2ab006b360 2023-07-06T03:46:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.050+0000 7f2aaeffd700 1 -- 172.21.15.97:0/637995688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa4011990 con 0x7f2ab006b360 2023-07-06T03:46:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.050+0000 7f2aaeffd700 1 -- 172.21.15.97:0/637995688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4011d20 con 0x7f2ab006b360 2023-07-06T03:46:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.050+0000 7f2ab69ff700 1 -- 172.21.15.97:0/637995688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab006b360 msgr2=0x7f2ab00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.050+0000 7f2ab69ff700 1 --2- 172.21.15.97:0/637995688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab006b360 0x7f2ab00b4cf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009920 tx=0x7f2aa4009c30 comp rx=0 tx=0).stop 2023-07-06T03:46:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.050+0000 7f2ab69ff700 1 -- 172.21.15.97:0/637995688 shutdown_connections 2023-07-06T03:46:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.050+0000 7f2ab69ff700 1 --2- 172.21.15.97:0/637995688 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab015c610 0x7f2ab00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.050+0000 7f2ab69ff700 1 --2- 172.21.15.97:0/637995688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab006b360 0x7f2ab00b4cf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.050+0000 7f2ab69ff700 1 -- 172.21.15.97:0/637995688 >> 172.21.15.97:0/637995688 conn(0x7f2ab006a6c0 msgr2=0x7f2ab006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.050+0000 7f2ab69ff700 1 -- 172.21.15.97:0/637995688 shutdown_connections 2023-07-06T03:46:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.050+0000 7f2ab69ff700 1 -- 172.21.15.97:0/637995688 wait complete. 2023-07-06T03:46:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.052+0000 7f2ab69ff700 1 Processor -- start 2023-07-06T03:46:34.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.052+0000 7f2ab69ff700 1 -- start start 2023-07-06T03:46:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.053+0000 7f2ab69ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab006b360 0x7f2ab015f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.053+0000 7f2ab69ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab00d2210 0x7f2ab0155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.053+0000 7f2ab69ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab015c610 0x7f2ab01f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.053+0000 7f2ab69ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00cbb20 con 0x7f2ab015c610 2023-07-06T03:46:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.053+0000 7f2ab69ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00caea0 con 0x7f2ab006b360 2023-07-06T03:46:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.053+0000 7f2ab69ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00cb180 con 0x7f2ab00d2210 2023-07-06T03:46:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.053+0000 7f2ab4f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab015c610 0x7f2ab01f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2ab4f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab015c610 0x7f2ab01f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48016/0 (socket says 172.21.15.97:48016) 2023-07-06T03:46:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2ab4f9c700 1 -- 172.21.15.97:0/1218509507 learned_addr learned my addr 172.21.15.97:0/1218509507 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2aaf7fe700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab00d2210 0x7f2ab0155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2aaffff700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab006b360 0x7f2ab015f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2ab4f9c700 1 -- 172.21.15.97:0/1218509507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab00d2210 msgr2=0x7f2ab0155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2ab4f9c700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab00d2210 0x7f2ab0155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2ab4f9c700 1 -- 172.21.15.97:0/1218509507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab006b360 msgr2=0x7f2ab015f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2ab4f9c700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab006b360 0x7f2ab015f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2ab4f9c700 1 -- 172.21.15.97:0/1218509507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa4009550 con 0x7f2ab015c610 2023-07-06T03:46:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2ab4f9c700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab015c610 0x7f2ab01f1f40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800cc50 tx=0x7f2aa800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:34.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2aacff9700 1 -- 172.21.15.97:0/1218509507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8003c00 con 0x7f2ab015c610 2023-07-06T03:46:34.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2aacff9700 1 -- 172.21.15.97:0/1218509507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa8003d60 con 0x7f2ab015c610 2023-07-06T03:46:34.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.054+0000 7f2aacff9700 1 -- 172.21.15.97:0/1218509507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800aab0 con 0x7f2ab015c610 2023-07-06T03:46:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.055+0000 7f2ab69ff700 1 -- 172.21.15.97:0/1218509507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab00cb460 con 0x7f2ab015c610 2023-07-06T03:46:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.055+0000 7f2ab69ff700 1 -- 172.21.15.97:0/1218509507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab015f940 con 0x7f2ab015c610 2023-07-06T03:46:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.056+0000 7f2ab69ff700 1 -- 172.21.15.97:0/1218509507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab00cbc60 con 0x7f2ab015c610 2023-07-06T03:46:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.057+0000 7f2aacff9700 1 -- 172.21.15.97:0/1218509507 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2aa800ac10 con 0x7f2ab015c610 2023-07-06T03:46:34.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.058+0000 7f2aacff9700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a9c06ace0 0x7f2a9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.058+0000 7f2aacff9700 1 -- 172.21.15.97:0/1218509507 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2aa8089030 con 0x7f2ab015c610 2023-07-06T03:46:34.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.058+0000 7f2aaffff700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a9c06ace0 0x7f2a9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.058+0000 7f2aaffff700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a9c06ace0 0x7f2a9c06d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009be0 tx=0x7f2aa0009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:34.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.062+0000 7f2aacff9700 1 -- 172.21.15.97:0/1218509507 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2ab00cbc60 con 0x7f2ab015c610 2023-07-06T03:46:34.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.255+0000 7f2ab69ff700 1 -- 172.21.15.97:0/1218509507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ab00cbc60 con 0x7f2ab015c610 2023-07-06T03:46:34.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.256+0000 7f2aacff9700 1 -- 172.21.15.97:0/1218509507 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ab00cbc60 con 0x7f2ab015c610 2023-07-06T03:46:34.258 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:46:34.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 -- 172.21.15.97:0/1218509507 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a9c06ace0 msgr2=0x7f2a9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a9c06ace0 0x7f2a9c06d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009be0 tx=0x7f2aa0009210 comp rx=0 tx=0).stop 2023-07-06T03:46:34.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 -- 172.21.15.97:0/1218509507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab015c610 msgr2=0x7f2ab01f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab015c610 0x7f2ab01f1f40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800cc50 tx=0x7f2aa800cf60 comp rx=0 tx=0).stop 2023-07-06T03:46:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 -- 172.21.15.97:0/1218509507 shutdown_connections 2023-07-06T03:46:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a9c06ace0 0x7f2a9c06d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab015c610 0x7f2ab01f1f40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ab006b360 0x7f2ab015f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 --2- 172.21.15.97:0/1218509507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ab00d2210 0x7f2ab0155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 -- 172.21.15.97:0/1218509507 >> 172.21.15.97:0/1218509507 conn(0x7f2ab006a6c0 msgr2=0x7f2ab00d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:34.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 -- 172.21.15.97:0/1218509507 shutdown_connections 2023-07-06T03:46:34.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.258+0000 7f2ab69ff700 1 -- 172.21.15.97:0/1218509507 wait complete. 2023-07-06T03:46:34.278 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:46:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.472+0000 7fe908fff700 1 Processor -- start 2023-07-06T03:46:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.472+0000 7fe908fff700 1 -- start start 2023-07-06T03:46:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.473+0000 7fe908fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe9040d2210 0x7fe9040b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.473+0000 7fe908fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9040cff90 con 0x7fe90415c610 2023-07-06T03:46:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.473+0000 7fe908fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9040b5650 con 0x7fe90406b360 2023-07-06T03:46:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.473+0000 7fe908fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9040d0780 con 0x7fe9040d2210 2023-07-06T03:46:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.473+0000 7fe90259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe90415c610 0x7fe9040b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54610/0 (socket says 172.21.15.97:54610) 2023-07-06T03:46:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.473+0000 7fe90259c700 1 -- 172.21.15.97:0/993275820 learned_addr learned my addr 172.21.15.97:0/993275820 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.474+0000 7fe902d9d700 1 --2- 172.21.15.97:0/993275820 >> v2:172.21.15.146:3300/0 conn(0x7fe9040d2210 0x7fe9040b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.474+0000 7fe90159a700 1 -- 172.21.15.97:0/993275820 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630437721 0 0) 0x7fe9040cff90 con 0x7fe90415c610 2023-07-06T03:46:34.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.474+0000 7fe90159a700 1 -- 172.21.15.97:0/993275820 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8ec0035b0 con 0x7fe90415c610 2023-07-06T03:46:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.474+0000 7fe902d9d700 1 -- 172.21.15.97:0/993275820 >> v1:172.21.15.146:6789/0 conn(0x7fe90406b360 legacy=0x7fe9040d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.474+0000 7fe90159a700 1 -- 172.21.15.97:0/993275820 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309145027 0 0) 0x7fe9040b5650 con 0x7fe90406b360 2023-07-06T03:46:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.474+0000 7fe902d9d700 1 -- 172.21.15.97:0/993275820 >> v1:172.21.15.97:6789/0 conn(0x7fe90415c610 legacy=0x7fe9040b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.474+0000 7fe902d9d700 1 -- 172.21.15.97:0/993275820 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9040b5400 con 0x7fe9040d2210 2023-07-06T03:46:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.474+0000 7fe902d9d700 1 --2- 172.21.15.97:0/993275820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9040d2210 0x7fe9040b4ec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009d90 tx=0x7fe8fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1896d39ac9e1d6ae server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.475+0000 7fe90159a700 1 -- 172.21.15.97:0/993275820 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc00e070 con 0x7fe9040d2210 2023-07-06T03:46:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.475+0000 7fe90159a700 1 -- 172.21.15.97:0/993275820 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc00bb20 con 0x7fe9040d2210 2023-07-06T03:46:34.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.475+0000 7fe90159a700 1 -- 172.21.15.97:0/993275820 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc00bed0 con 0x7fe9040d2210 2023-07-06T03:46:34.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.475+0000 7fe908fff700 1 -- 172.21.15.97:0/993275820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9040d2210 msgr2=0x7fe9040b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.475+0000 7fe908fff700 1 --2- 172.21.15.97:0/993275820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9040d2210 0x7fe9040b4ec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009d90 tx=0x7fe8fc00b330 comp rx=0 tx=0).stop 2023-07-06T03:46:34.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.475+0000 7fe908fff700 1 -- 172.21.15.97:0/993275820 shutdown_connections 2023-07-06T03:46:34.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.475+0000 7fe908fff700 1 --2- 172.21.15.97:0/993275820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9040d2210 0x7fe9040b4ec0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.475+0000 7fe908fff700 1 -- 172.21.15.97:0/993275820 >> 172.21.15.97:0/993275820 conn(0x7fe90406a6c0 msgr2=0x7fe90406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:34.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.475+0000 7fe908fff700 1 -- 172.21.15.97:0/993275820 shutdown_connections 2023-07-06T03:46:34.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.475+0000 7fe908fff700 1 -- 172.21.15.97:0/993275820 wait complete. 2023-07-06T03:46:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.478+0000 7fe908fff700 1 Processor -- start 2023-07-06T03:46:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.478+0000 7fe908fff700 1 -- start start 2023-07-06T03:46:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.479+0000 7fe908fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe90406b360 0x7fe9040c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.479+0000 7fe908fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9040d2210 0x7fe90415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.479+0000 7fe908fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90415c610 0x7fe90415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.479+0000 7fe908fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90415e5f0 con 0x7fe90415c610 2023-07-06T03:46:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.479+0000 7fe908fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040ca470 con 0x7fe90406b360 2023-07-06T03:46:34.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.479+0000 7fe908fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040ca750 con 0x7fe9040d2210 2023-07-06T03:46:34.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.479+0000 7fe902d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90415c610 0x7fe90415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.479+0000 7fe902d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90415c610 0x7fe90415eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48028/0 (socket says 172.21.15.97:48028) 2023-07-06T03:46:34.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.479+0000 7fe902d9d700 1 -- 172.21.15.97:0/475902173 learned_addr learned my addr 172.21.15.97:0/475902173 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:34.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe90259c700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe90406b360 0x7fe9040c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe901d9b700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9040d2210 0x7fe90415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe902d9d700 1 -- 172.21.15.97:0/475902173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9040d2210 msgr2=0x7fe90415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe902d9d700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9040d2210 0x7fe90415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe902d9d700 1 -- 172.21.15.97:0/475902173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe90406b360 msgr2=0x7fe9040c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe902d9d700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe90406b360 0x7fe9040c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe902d9d700 1 -- 172.21.15.97:0/475902173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc009a70 con 0x7fe90415c610 2023-07-06T03:46:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe902d9d700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90415c610 0x7fe90415eaa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc014040 tx=0x7fe8fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe8f2ffd700 1 -- 172.21.15.97:0/475902173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc01b810 con 0x7fe90415c610 2023-07-06T03:46:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe8f2ffd700 1 -- 172.21.15.97:0/475902173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc020070 con 0x7fe90415c610 2023-07-06T03:46:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe8f2ffd700 1 -- 172.21.15.97:0/475902173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc01bbb0 con 0x7fe90415c610 2023-07-06T03:46:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.480+0000 7fe908fff700 1 -- 172.21.15.97:0/475902173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9040caa30 con 0x7fe90415c610 2023-07-06T03:46:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.481+0000 7fe908fff700 1 -- 172.21.15.97:0/475902173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9040caff0 con 0x7fe90415c610 2023-07-06T03:46:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.481+0000 7fe908fff700 1 -- 172.21.15.97:0/475902173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90404e340 con 0x7fe90415c610 2023-07-06T03:46:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.482+0000 7fe8f2ffd700 1 -- 172.21.15.97:0/475902173 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe8fc01bd10 con 0x7fe90415c610 2023-07-06T03:46:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.484+0000 7fe8f2ffd700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe8e406ad10 0x7fe8e406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.484+0000 7fe8f2ffd700 1 -- 172.21.15.97:0/475902173 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe8fc022070 con 0x7fe90415c610 2023-07-06T03:46:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.484+0000 7fe90259c700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe8e406ad10 0x7fe8e406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.484+0000 7fe90259c700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe8e406ad10 0x7fe8e406d1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007fd0 tx=0x7fe8f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.488+0000 7fe8f2ffd700 1 -- 172.21.15.97:0/475902173 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe8fc056530 con 0x7fe90415c610 2023-07-06T03:46:34.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.683+0000 7fe908fff700 1 -- 172.21.15.97:0/475902173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe9040b8a40 con 0x7fe90415c610 2023-07-06T03:46:34.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.683+0000 7fe8f2ffd700 1 -- 172.21.15.97:0/475902173 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe8fc059b50 con 0x7fe90415c610 2023-07-06T03:46:34.685 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:46:34.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.685+0000 7fe908fff700 1 -- 172.21.15.97:0/475902173 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe8e406ad10 msgr2=0x7fe8e406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.685+0000 7fe908fff700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe8e406ad10 0x7fe8e406d1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007fd0 tx=0x7fe8f4009040 comp rx=0 tx=0).stop 2023-07-06T03:46:34.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.686+0000 7fe908fff700 1 -- 172.21.15.97:0/475902173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90415c610 msgr2=0x7fe90415eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.686+0000 7fe908fff700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90415c610 0x7fe90415eaa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc014040 tx=0x7fe8fc004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.686+0000 7fe908fff700 1 -- 172.21.15.97:0/475902173 shutdown_connections 2023-07-06T03:46:34.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.686+0000 7fe908fff700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe8e406ad10 0x7fe8e406d1c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.686+0000 7fe908fff700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90415c610 0x7fe90415eaa0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.686+0000 7fe908fff700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe90406b360 0x7fe9040c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.686+0000 7fe908fff700 1 --2- 172.21.15.97:0/475902173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9040d2210 0x7fe90415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.686+0000 7fe908fff700 1 -- 172.21.15.97:0/475902173 >> 172.21.15.97:0/475902173 conn(0x7fe90406a6c0 msgr2=0x7fe90415ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:34.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.686+0000 7fe908fff700 1 -- 172.21.15.97:0/475902173 shutdown_connections 2023-07-06T03:46:34.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.686+0000 7fe908fff700 1 -- 172.21.15.97:0/475902173 wait complete. 2023-07-06T03:46:34.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T03:46:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.900+0000 7efe40fff700 1 Processor -- start 2023-07-06T03:46:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.900+0000 7efe40fff700 1 -- start start 2023-07-06T03:46:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe40fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efe3c15b440 0x7efe3c15e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe40fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efe3c070c30 0x7efe3c0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe40fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c071330 con 0x7efe3c0cd8e0 2023-07-06T03:46:34.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe40fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c0ae3e0 con 0x7efe3c15b440 2023-07-06T03:46:34.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe40fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c071610 con 0x7efe3c070c30 2023-07-06T03:46:34.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe33fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efe3c15b440 0x7efe3c15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe3a59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efe3c0cd8e0 0x7efe3c0cdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54622/0 (socket says 172.21.15.97:54622) 2023-07-06T03:46:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe3a59c700 1 -- 172.21.15.97:0/346386061 learned_addr learned my addr 172.21.15.97:0/346386061 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe3ad9d700 1 --2- 172.21.15.97:0/346386061 >> v2:172.21.15.146:3300/0 conn(0x7efe3c070c30 0x7efe3c0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe33fff700 1 -- 172.21.15.97:0/346386061 >> v2:172.21.15.146:3300/0 conn(0x7efe3c070c30 msgr2=0x7efe3c0adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe33fff700 1 --2- 172.21.15.97:0/346386061 >> v2:172.21.15.146:3300/0 conn(0x7efe3c070c30 0x7efe3c0adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.902+0000 7efe33fff700 1 -- 172.21.15.97:0/346386061 >> v1:172.21.15.97:6789/0 conn(0x7efe3c0cd8e0 legacy=0x7efe3c0cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.903+0000 7efe33fff700 1 -- 172.21.15.97:0/346386061 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c15dd80 con 0x7efe3c15b440 2023-07-06T03:46:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.903+0000 7efe39d9b700 1 -- 172.21.15.97:0/346386061 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198453706 0 0) 0x7efe3c071330 con 0x7efe3c0cd8e0 2023-07-06T03:46:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.903+0000 7efe33fff700 1 --2- 172.21.15.97:0/346386061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3c15b440 0x7efe3c15e860 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0ce3c0 tx=0x7efe24009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b34f25af845c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.903+0000 7efe39d9b700 1 -- 172.21.15.97:0/346386061 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24011830 con 0x7efe3c15b440 2023-07-06T03:46:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.903+0000 7efe39d9b700 1 -- 172.21.15.97:0/346386061 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe24011990 con 0x7efe3c15b440 2023-07-06T03:46:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.903+0000 7efe39d9b700 1 -- 172.21.15.97:0/346386061 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24011d20 con 0x7efe3c15b440 2023-07-06T03:46:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.903+0000 7efe40fff700 1 -- 172.21.15.97:0/346386061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3c15b440 msgr2=0x7efe3c15e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.903+0000 7efe40fff700 1 --2- 172.21.15.97:0/346386061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3c15b440 0x7efe3c15e860 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0ce3c0 tx=0x7efe24009b20 comp rx=0 tx=0).stop 2023-07-06T03:46:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.904+0000 7efe40fff700 1 -- 172.21.15.97:0/346386061 shutdown_connections 2023-07-06T03:46:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.904+0000 7efe40fff700 1 --2- 172.21.15.97:0/346386061 >> v2:172.21.15.146:3300/0 conn(0x7efe3c070c30 0x7efe3c0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.904+0000 7efe40fff700 1 --2- 172.21.15.97:0/346386061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3c15b440 0x7efe3c15e860 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.904+0000 7efe40fff700 1 -- 172.21.15.97:0/346386061 >> 172.21.15.97:0/346386061 conn(0x7efe3c0ce8c0 msgr2=0x7efe3c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.904+0000 7efe40fff700 1 -- 172.21.15.97:0/346386061 shutdown_connections 2023-07-06T03:46:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.904+0000 7efe40fff700 1 -- 172.21.15.97:0/346386061 wait complete. 2023-07-06T03:46:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.905+0000 7efe40fff700 1 Processor -- start 2023-07-06T03:46:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.906+0000 7efe40fff700 1 -- start start 2023-07-06T03:46:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.906+0000 7efe40fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe3c070c30 0x7efe3c140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.906+0000 7efe40fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe3c0cd8e0 0x7efe3c0af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.906+0000 7efe40fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3c15b440 0x7efe3c1e4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.906+0000 7efe40fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c1500f0 con 0x7efe3c15b440 2023-07-06T03:46:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.906+0000 7efe40fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c14fda0 con 0x7efe3c0cd8e0 2023-07-06T03:46:34.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.906+0000 7efe40fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0b02e0 con 0x7efe3c070c30 2023-07-06T03:46:34.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe3ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3c15b440 0x7efe3c1e4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe3ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3c15b440 0x7efe3c1e4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48044/0 (socket says 172.21.15.97:48044) 2023-07-06T03:46:34.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe3ad9d700 1 -- 172.21.15.97:0/3157161992 learned_addr learned my addr 172.21.15.97:0/3157161992 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:34.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe33fff700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe3c0cd8e0 0x7efe3c0af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe3a59c700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe3c070c30 0x7efe3c140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe3ad9d700 1 -- 172.21.15.97:0/3157161992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe3c070c30 msgr2=0x7efe3c140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe3ad9d700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe3c070c30 0x7efe3c140420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe3ad9d700 1 -- 172.21.15.97:0/3157161992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe3c0cd8e0 msgr2=0x7efe3c0af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:34.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe3ad9d700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe3c0cd8e0 0x7efe3c0af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:34.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe3ad9d700 1 -- 172.21.15.97:0/3157161992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe240094a0 con 0x7efe3c15b440 2023-07-06T03:46:34.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe3a59c700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe3c070c30 0x7efe3c140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:34.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.907+0000 7efe3ad9d700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3c15b440 0x7efe3c1e4530 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe34007e90 tx=0x7efe3400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:34.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.908+0000 7efe32ffd700 1 -- 172.21.15.97:0/3157161992 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3400ede0 con 0x7efe3c15b440 2023-07-06T03:46:34.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.908+0000 7efe32ffd700 1 -- 172.21.15.97:0/3157161992 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe34014420 con 0x7efe3c15b440 2023-07-06T03:46:34.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.908+0000 7efe40fff700 1 -- 172.21.15.97:0/3157161992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c0b05c0 con 0x7efe3c15b440 2023-07-06T03:46:34.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.908+0000 7efe32ffd700 1 -- 172.21.15.97:0/3157161992 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe340147d0 con 0x7efe3c15b440 2023-07-06T03:46:34.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.908+0000 7efe40fff700 1 -- 172.21.15.97:0/3157161992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c140960 con 0x7efe3c15b440 2023-07-06T03:46:34.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.909+0000 7efe40fff700 1 -- 172.21.15.97:0/3157161992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c070aa0 con 0x7efe3c15b440 2023-07-06T03:46:34.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.910+0000 7efe32ffd700 1 -- 172.21.15.97:0/3157161992 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efe34014930 con 0x7efe3c15b440 2023-07-06T03:46:34.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.910+0000 7efe32ffd700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efe1c06ace0 0x7efe1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:34.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.910+0000 7efe32ffd700 1 -- 172.21.15.97:0/3157161992 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7efe34089830 con 0x7efe3c15b440 2023-07-06T03:46:34.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.911+0000 7efe3a59c700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efe1c06ace0 0x7efe1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:34.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.911+0000 7efe3a59c700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efe1c06ace0 0x7efe1c06d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efe2c000f60 tx=0x7efe2c007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:34.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:34.915+0000 7efe32ffd700 1 -- 172.21.15.97:0/3157161992 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efe34054c20 con 0x7efe3c15b440 2023-07-06T03:46:35.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.112+0000 7efe40fff700 1 -- 172.21.15.97:0/3157161992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efe3c067830 con 0x7efe3c15b440 2023-07-06T03:46:35.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.113+0000 7efe32ffd700 1 -- 172.21.15.97:0/3157161992 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+126 (secure 0 0 0) 0x7efe34058240 con 0x7efe3c15b440 2023-07-06T03:46:35.115 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:35.115 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-07-06T03:46:35.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:35.115 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-07-06T03:46:35.116 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-07-06T03:46:35.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:35.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:46:35.116 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 2, 2023-07-06T03:46:35.116 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": "rbd" 2023-07-06T03:46:35.116 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:35.117 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-07-06T03:46:35.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 -- 172.21.15.97:0/3157161992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efe1c06ace0 msgr2=0x7efe1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:35.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efe1c06ace0 0x7efe1c06d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efe2c000f60 tx=0x7efe2c007ca0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 -- 172.21.15.97:0/3157161992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3c15b440 msgr2=0x7efe3c1e4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:35.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3c15b440 0x7efe3c1e4530 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe34007e90 tx=0x7efe3400e3c0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 -- 172.21.15.97:0/3157161992 shutdown_connections 2023-07-06T03:46:35.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efe1c06ace0 0x7efe1c06d190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3c15b440 0x7efe3c1e4530 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efe3c070c30 0x7efe3c140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 --2- 172.21.15.97:0/3157161992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe3c0cd8e0 0x7efe3c0af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 -- 172.21.15.97:0/3157161992 >> 172.21.15.97:0/3157161992 conn(0x7efe3c0ce8c0 msgr2=0x7efe3c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 -- 172.21.15.97:0/3157161992 shutdown_connections 2023-07-06T03:46:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.115+0000 7efe40fff700 1 -- 172.21.15.97:0/3157161992 wait complete. 2023-07-06T03:46:35.135 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T03:46:35.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T03:46:35.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.333+0000 7f8d695ff700 1 Processor -- start 2023-07-06T03:46:35.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.334+0000 7f8d695ff700 1 -- start start 2023-07-06T03:46:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.335+0000 7f8d695ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d64160b70 0x7f8d640b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.335+0000 7f8d695ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8d640b4cb0 0x7f8d640d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.335+0000 7f8d695ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d6415cad0 con 0x7f8d6415c700 2023-07-06T03:46:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.335+0000 7f8d695ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d640b5120 con 0x7f8d640b4cb0 2023-07-06T03:46:35.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.335+0000 7f8d695ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d64161000 con 0x7f8d64160b70 2023-07-06T03:46:35.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.335+0000 7f8d62ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8d6415c700 0x7f8d640cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54632/0 (socket says 172.21.15.97:54632) 2023-07-06T03:46:35.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.335+0000 7f8d62ffd700 1 -- 172.21.15.97:0/3358471927 learned_addr learned my addr 172.21.15.97:0/3358471927 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:35.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.335+0000 7f8d637fe700 1 --2- 172.21.15.97:0/3358471927 >> v2:172.21.15.146:3300/0 conn(0x7f8d640b4cb0 0x7f8d640d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.335+0000 7f8d627fc700 1 --2- 172.21.15.97:0/3358471927 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d64160b70 0x7f8d640b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.335+0000 7f8d61ffb700 1 -- 172.21.15.97:0/3358471927 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784540651 0 0) 0x7f8d6415cad0 con 0x7f8d6415c700 2023-07-06T03:46:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.336+0000 7f8d61ffb700 1 -- 172.21.15.97:0/3358471927 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d4c0035b0 con 0x7f8d6415c700 2023-07-06T03:46:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.336+0000 7f8d637fe700 1 -- 172.21.15.97:0/3358471927 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d64160b70 msgr2=0x7f8d640b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.336+0000 7f8d637fe700 1 --2- 172.21.15.97:0/3358471927 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d64160b70 0x7f8d640b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.336+0000 7f8d637fe700 1 -- 172.21.15.97:0/3358471927 >> v1:172.21.15.97:6789/0 conn(0x7f8d6415c700 legacy=0x7f8d640cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.336+0000 7f8d61ffb700 1 -- 172.21.15.97:0/3358471927 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 652427406 0 0) 0x7f8d4c0035b0 con 0x7f8d6415c700 2023-07-06T03:46:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.336+0000 7f8d637fe700 1 -- 172.21.15.97:0/3358471927 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d640cd3e0 con 0x7f8d640b4cb0 2023-07-06T03:46:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.336+0000 7f8d637fe700 1 --2- 172.21.15.97:0/3358471927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d640b4cb0 0x7f8d640d1110 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009d90 tx=0x7f8d5c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a832cbd99d01f317 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.336+0000 7f8d61ffb700 1 -- 172.21.15.97:0/3358471927 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c00e070 con 0x7f8d640b4cb0 2023-07-06T03:46:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.336+0000 7f8d61ffb700 1 -- 172.21.15.97:0/3358471927 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5c003a20 con 0x7f8d640b4cb0 2023-07-06T03:46:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.336+0000 7f8d61ffb700 1 -- 172.21.15.97:0/3358471927 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54001670 con 0x7f8d640b4cb0 2023-07-06T03:46:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.337+0000 7f8d695ff700 1 -- 172.21.15.97:0/3358471927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d640b4cb0 msgr2=0x7f8d640d1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.337+0000 7f8d695ff700 1 --2- 172.21.15.97:0/3358471927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d640b4cb0 0x7f8d640d1110 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009d90 tx=0x7f8d5c00b330 comp rx=0 tx=0).stop 2023-07-06T03:46:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.337+0000 7f8d695ff700 1 -- 172.21.15.97:0/3358471927 shutdown_connections 2023-07-06T03:46:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.337+0000 7f8d695ff700 1 --2- 172.21.15.97:0/3358471927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d640b4cb0 0x7f8d640d1110 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.337+0000 7f8d695ff700 1 --2- 172.21.15.97:0/3358471927 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d64160b70 0x7f8d640b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.337+0000 7f8d695ff700 1 -- 172.21.15.97:0/3358471927 >> 172.21.15.97:0/3358471927 conn(0x7f8d640cf770 msgr2=0x7f8d640cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.337+0000 7f8d695ff700 1 -- 172.21.15.97:0/3358471927 shutdown_connections 2023-07-06T03:46:35.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.337+0000 7f8d695ff700 1 -- 172.21.15.97:0/3358471927 wait complete. 2023-07-06T03:46:35.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.339+0000 7f8d695ff700 1 Processor -- start 2023-07-06T03:46:35.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.339+0000 7f8d695ff700 1 -- start start 2023-07-06T03:46:35.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.340+0000 7f8d695ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d640b4cb0 0x7f8d640cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:35.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.340+0000 7f8d695ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d6415c700 0x7f8d640ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:35.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.340+0000 7f8d695ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d64160b70 0x7f8d640d1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:35.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.340+0000 7f8d695ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d640b7280 con 0x7f8d640b4cb0 2023-07-06T03:46:35.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.340+0000 7f8d695ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d640b6950 con 0x7f8d6415c700 2023-07-06T03:46:35.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.340+0000 7f8d695ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d640b6c30 con 0x7f8d64160b70 2023-07-06T03:46:35.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.340+0000 7f8d62ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d640b4cb0 0x7f8d640cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:35.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.340+0000 7f8d62ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d640b4cb0 0x7f8d640cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48054/0 (socket says 172.21.15.97:48054) 2023-07-06T03:46:35.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.340+0000 7f8d62ffd700 1 -- 172.21.15.97:0/2469024833 learned_addr learned my addr 172.21.15.97:0/2469024833 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:35.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.340+0000 7f8d637fe700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d64160b70 0x7f8d640d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:35.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.340+0000 7f8d627fc700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d6415c700 0x7f8d640ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:35.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.341+0000 7f8d62ffd700 1 -- 172.21.15.97:0/2469024833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d64160b70 msgr2=0x7f8d640d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:35.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.341+0000 7f8d62ffd700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d64160b70 0x7f8d640d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.341+0000 7f8d62ffd700 1 -- 172.21.15.97:0/2469024833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d6415c700 msgr2=0x7f8d640ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.341+0000 7f8d62ffd700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d6415c700 0x7f8d640ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.341+0000 7f8d62ffd700 1 -- 172.21.15.97:0/2469024833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c009a70 con 0x7f8d640b4cb0 2023-07-06T03:46:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.341+0000 7f8d62ffd700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d640b4cb0 0x7f8d640cc5d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8d540011e0 tx=0x7f8d5400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.341+0000 7f8d4b7fe700 1 -- 172.21.15.97:0/2469024833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54004c80 con 0x7f8d640b4cb0 2023-07-06T03:46:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.341+0000 7f8d4b7fe700 1 -- 172.21.15.97:0/2469024833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d54004e00 con 0x7f8d640b4cb0 2023-07-06T03:46:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.341+0000 7f8d695ff700 1 -- 172.21.15.97:0/2469024833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d640b6f10 con 0x7f8d640b4cb0 2023-07-06T03:46:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.341+0000 7f8d4b7fe700 1 -- 172.21.15.97:0/2469024833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54007310 con 0x7f8d640b4cb0 2023-07-06T03:46:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.341+0000 7f8d695ff700 1 -- 172.21.15.97:0/2469024833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d640ccc50 con 0x7f8d640b4cb0 2023-07-06T03:46:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.342+0000 7f8d695ff700 1 -- 172.21.15.97:0/2469024833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6404e340 con 0x7f8d640b4cb0 2023-07-06T03:46:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.343+0000 7f8d4b7fe700 1 -- 172.21.15.97:0/2469024833 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d54007470 con 0x7f8d640b4cb0 2023-07-06T03:46:35.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.345+0000 7f8d4b7fe700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 0x7f8d5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:35.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.345+0000 7f8d4b7fe700 1 -- 172.21.15.97:0/2469024833 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8d54088fe0 con 0x7f8d640b4cb0 2023-07-06T03:46:35.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.345+0000 7f8d627fc700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 0x7f8d5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:35.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.345+0000 7f8d627fc700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 0x7f8d5006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d64067140 tx=0x7f8d58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:35.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.349+0000 7f8d4b7fe700 1 -- 172.21.15.97:0/2469024833 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8d540543e0 con 0x7f8d640b4cb0 2023-07-06T03:46:35.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.545+0000 7f8d695ff700 1 -- 172.21.15.97:0/2469024833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8d640b2190 con 0x7f8d640b4cb0 2023-07-06T03:46:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.827+0000 7f8d4b7fe700 1 -- 172.21.15.97:0/2469024833 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8d54057a00 con 0x7f8d640b4cb0 2023-07-06T03:46:35.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.831+0000 7f8d695ff700 1 -- 172.21.15.97:0/2469024833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 msgr2=0x7f8d5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:35.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.831+0000 7f8d695ff700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 0x7f8d5006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d64067140 tx=0x7f8d58008040 comp rx=0 tx=0).stop 2023-07-06T03:46:35.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.831+0000 7f8d695ff700 1 -- 172.21.15.97:0/2469024833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d640b4cb0 msgr2=0x7f8d640cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:35.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.831+0000 7f8d695ff700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d640b4cb0 0x7f8d640cc5d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8d540011e0 tx=0x7f8d5400ca90 comp rx=0 tx=0).stop 2023-07-06T03:46:35.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.831+0000 7f8d695ff700 1 -- 172.21.15.97:0/2469024833 shutdown_connections 2023-07-06T03:46:35.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.831+0000 7f8d695ff700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 0x7f8d5006d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.832+0000 7f8d695ff700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d64160b70 0x7f8d640d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.832+0000 7f8d695ff700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d640b4cb0 0x7f8d640cc5d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.832+0000 7f8d695ff700 1 --2- 172.21.15.97:0/2469024833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d6415c700 0x7f8d640ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:35.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.832+0000 7f8d695ff700 1 -- 172.21.15.97:0/2469024833 >> 172.21.15.97:0/2469024833 conn(0x7f8d640cf770 msgr2=0x7f8d640cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:35.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.832+0000 7f8d695ff700 1 -- 172.21.15.97:0/2469024833 shutdown_connections 2023-07-06T03:46:35.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:35.832+0000 7f8d695ff700 1 -- 172.21.15.97:0/2469024833 wait complete. 2023-07-06T03:46:35.836 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-07-06T03:46:35.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T03:46:36.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.055+0000 7f71837ff700 1 Processor -- start 2023-07-06T03:46:36.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.055+0000 7f71837ff700 1 -- start start 2023-07-06T03:46:36.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.056+0000 7f71837ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f717c0b8590 0x7f717c0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:36.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.056+0000 7f71837ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c160a50 0x7f717c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.056+0000 7f71837ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717c0d0c10 con 0x7f717c0b4cd0 2023-07-06T03:46:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.057+0000 7f71837ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717c0b5150 con 0x7f717c0b8590 2023-07-06T03:46:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.057+0000 7f71837ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c160ec0 con 0x7f717c160a50 2023-07-06T03:46:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.057+0000 7f718159b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f717c0b8590 0x7f717c0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.057+0000 7f718159b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f717c0b8590 0x7f717c0cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48068/0 (socket says 172.21.15.97:48068) 2023-07-06T03:46:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.057+0000 7f718159b700 1 -- 172.21.15.97:0/673934440 learned_addr learned my addr 172.21.15.97:0/673934440 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.057+0000 7f7180d9a700 1 --2- 172.21.15.97:0/673934440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c160a50 0x7f717c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.057+0000 7f718159b700 1 -- 172.21.15.97:0/673934440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c160a50 msgr2=0x7f717c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.057+0000 7f718159b700 1 --2- 172.21.15.97:0/673934440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c160a50 0x7f717c0b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.057+0000 7f718159b700 1 -- 172.21.15.97:0/673934440 >> v1:172.21.15.146:6789/0 conn(0x7f717c0b4cd0 legacy=0x7f717c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.057+0000 7f718159b700 1 -- 172.21.15.97:0/673934440 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0d08e0 con 0x7f717c0b8590 2023-07-06T03:46:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.058+0000 7f718159b700 1 --2- 172.21.15.97:0/673934440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c0b8590 0x7f717c0cef60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f71780098f0 tx=0x7f7178009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133b1e66cfa5d427 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.058+0000 7f7173fff700 1 -- 172.21.15.97:0/673934440 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800b8a0 con 0x7f717c0b8590 2023-07-06T03:46:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.058+0000 7f7173fff700 1 -- 172.21.15.97:0/673934440 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717800ba00 con 0x7f717c0b8590 2023-07-06T03:46:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.058+0000 7f7173fff700 1 -- 172.21.15.97:0/673934440 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800bd90 con 0x7f717c0b8590 2023-07-06T03:46:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.058+0000 7f71837ff700 1 -- 172.21.15.97:0/673934440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c0b8590 msgr2=0x7f717c0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.058+0000 7f71837ff700 1 --2- 172.21.15.97:0/673934440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c0b8590 0x7f717c0cef60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f71780098f0 tx=0x7f7178009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.059+0000 7f71837ff700 1 -- 172.21.15.97:0/673934440 shutdown_connections 2023-07-06T03:46:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.059+0000 7f71837ff700 1 --2- 172.21.15.97:0/673934440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c0b8590 0x7f717c0cef60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.059+0000 7f71837ff700 1 --2- 172.21.15.97:0/673934440 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c160a50 0x7f717c0b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.059+0000 7f71837ff700 1 -- 172.21.15.97:0/673934440 >> 172.21.15.97:0/673934440 conn(0x7f717c06b290 msgr2=0x7f717c0cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.059+0000 7f71837ff700 1 -- 172.21.15.97:0/673934440 shutdown_connections 2023-07-06T03:46:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.059+0000 7f71837ff700 1 -- 172.21.15.97:0/673934440 wait complete. 2023-07-06T03:46:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.060+0000 7f71837ff700 1 Processor -- start 2023-07-06T03:46:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.060+0000 7f71837ff700 1 -- start start 2023-07-06T03:46:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.061+0000 7f71837ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c0b4cd0 0x7f717c0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.061+0000 7f71837ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c0b8590 0x7f717c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.061+0000 7f71837ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f717c160a50 0x7f717c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.061+0000 7f71837ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c151760 con 0x7f717c0b8590 2023-07-06T03:46:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.061+0000 7f71837ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0caf70 con 0x7f717c160a50 2023-07-06T03:46:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.061+0000 7f71837ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0cbcc0 con 0x7f717c0b4cd0 2023-07-06T03:46:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.062+0000 7f7180d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c0b8590 0x7f717c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.062+0000 7f7180d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c0b8590 0x7f717c151220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48076/0 (socket says 172.21.15.97:48076) 2023-07-06T03:46:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.062+0000 7f7180d9a700 1 -- 172.21.15.97:0/731109641 learned_addr learned my addr 172.21.15.97:0/731109641 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.062+0000 7f718159b700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c0b4cd0 0x7f717c0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.062+0000 7f7181d9c700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f717c160a50 0x7f717c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.062+0000 7f7180d9a700 1 -- 172.21.15.97:0/731109641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c0b4cd0 msgr2=0x7f717c0cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.062+0000 7f7180d9a700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c0b4cd0 0x7f717c0cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.062+0000 7f7180d9a700 1 -- 172.21.15.97:0/731109641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f717c160a50 msgr2=0x7f717c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.062+0000 7f7180d9a700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f717c160a50 0x7f717c15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.062+0000 7f7180d9a700 1 -- 172.21.15.97:0/731109641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7178009550 con 0x7f717c0b8590 2023-07-06T03:46:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.062+0000 7f7180d9a700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c0b8590 0x7f717c151220 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f716c00d800 tx=0x7f716c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.063+0000 7f7171ffb700 1 -- 172.21.15.97:0/731109641 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c0125f0 con 0x7f717c0b8590 2023-07-06T03:46:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.063+0000 7f7171ffb700 1 -- 172.21.15.97:0/731109641 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c012750 con 0x7f717c0b8590 2023-07-06T03:46:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.063+0000 7f7171ffb700 1 -- 172.21.15.97:0/731109641 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c012ae0 con 0x7f717c0b8590 2023-07-06T03:46:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.063+0000 7f71837ff700 1 -- 172.21.15.97:0/731109641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717c0cc240 con 0x7f717c0b8590 2023-07-06T03:46:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.063+0000 7f71837ff700 1 -- 172.21.15.97:0/731109641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717c0ca920 con 0x7f717c0b8590 2023-07-06T03:46:36.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.064+0000 7f71837ff700 1 -- 172.21.15.97:0/731109641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717c0cb250 con 0x7f717c0b8590 2023-07-06T03:46:36.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.066+0000 7f7171ffb700 1 -- 172.21.15.97:0/731109641 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f716c012c40 con 0x7f717c0b8590 2023-07-06T03:46:36.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.067+0000 7f7171ffb700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716006ace0 0x7f716006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:36.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.067+0000 7f7171ffb700 1 -- 172.21.15.97:0/731109641 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f716c089fd0 con 0x7f717c0b8590 2023-07-06T03:46:36.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.067+0000 7f718159b700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716006ace0 0x7f716006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:36.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.067+0000 7f718159b700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716006ace0 0x7f716006d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7178000c00 tx=0x7f7178008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:36.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.071+0000 7f7171ffb700 1 -- 172.21.15.97:0/731109641 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f716c0553d0 con 0x7f717c0b8590 2023-07-06T03:46:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.264+0000 7f71837ff700 1 -- 172.21.15.97:0/731109641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f717c04e340 con 0x7f717c0b8590 2023-07-06T03:46:36.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.839+0000 7f7171ffb700 1 -- 172.21.15.97:0/731109641 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f716c0589f0 con 0x7f717c0b8590 2023-07-06T03:46:36.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.849+0000 7f71837ff700 1 -- 172.21.15.97:0/731109641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716006ace0 msgr2=0x7f716006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:36.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.849+0000 7f71837ff700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716006ace0 0x7f716006d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7178000c00 tx=0x7f7178008030 comp rx=0 tx=0).stop 2023-07-06T03:46:36.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.849+0000 7f71837ff700 1 -- 172.21.15.97:0/731109641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c0b8590 msgr2=0x7f717c151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:36.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.849+0000 7f71837ff700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c0b8590 0x7f717c151220 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f716c00d800 tx=0x7f716c00dbc0 comp rx=0 tx=0).stop 2023-07-06T03:46:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.850+0000 7f71837ff700 1 -- 172.21.15.97:0/731109641 shutdown_connections 2023-07-06T03:46:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.850+0000 7f71837ff700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716006ace0 0x7f716006d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.850+0000 7f71837ff700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f717c160a50 0x7f717c15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.850+0000 7f71837ff700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c0b4cd0 0x7f717c0cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.850+0000 7f71837ff700 1 --2- 172.21.15.97:0/731109641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c0b8590 0x7f717c151220 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:36.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.850+0000 7f71837ff700 1 -- 172.21.15.97:0/731109641 >> 172.21.15.97:0/731109641 conn(0x7f717c06b290 msgr2=0x7f717c0b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:36.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.854+0000 7f71837ff700 1 -- 172.21.15.97:0/731109641 shutdown_connections 2023-07-06T03:46:36.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:36.854+0000 7f71837ff700 1 -- 172.21.15.97:0/731109641 wait complete. 2023-07-06T03:46:36.858 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-07-06T03:46:36.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T03:46:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.075+0000 7f49b41ff700 1 Processor -- start 2023-07-06T03:46:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.075+0000 7f49b41ff700 1 -- start start 2023-07-06T03:46:37.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.076+0000 7f49b41ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49ac15c1c0 0x7f49ac0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:37.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.076+0000 7f49b41ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49ac0719c0 0x7f49ac06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:37.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.076+0000 7f49b41ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ac15da10 con 0x7f49ac15c590 2023-07-06T03:46:37.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.076+0000 7f49b41ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac0ccc30 con 0x7f49ac15c1c0 2023-07-06T03:46:37.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.076+0000 7f49b41ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0ccf10 con 0x7f49ac0719c0 2023-07-06T03:46:37.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.077+0000 7f49b1f9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49ac15c1c0 0x7f49ac0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:37.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.077+0000 7f49b1f9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49ac15c1c0 0x7f49ac0cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48080/0 (socket says 172.21.15.97:48080) 2023-07-06T03:46:37.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.077+0000 7f49b1f9b700 1 -- 172.21.15.97:0/349486910 learned_addr learned my addr 172.21.15.97:0/349486910 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:37.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.077+0000 7f49b279c700 1 --2- 172.21.15.97:0/349486910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49ac0719c0 0x7f49ac06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:37.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.077+0000 7f49b1f9b700 1 -- 172.21.15.97:0/349486910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49ac0719c0 msgr2=0x7f49ac06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:37.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.077+0000 7f49b1f9b700 1 --2- 172.21.15.97:0/349486910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49ac0719c0 0x7f49ac06a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:37.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.077+0000 7f49b1f9b700 1 -- 172.21.15.97:0/349486910 >> v1:172.21.15.146:6789/0 conn(0x7f49ac15c590 legacy=0x7f49ac15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:37.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.077+0000 7f49b1f9b700 1 -- 172.21.15.97:0/349486910 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac0cd1f0 con 0x7f49ac15c1c0 2023-07-06T03:46:37.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.077+0000 7f49b1f9b700 1 --2- 172.21.15.97:0/349486910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49ac15c1c0 0x7f49ac0cc3e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f49a80098f0 tx=0x7f49a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1a360ab33e6168 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:37.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.078+0000 7f49b0f99700 1 -- 172.21.15.97:0/349486910 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a800b8a0 con 0x7f49ac15c1c0 2023-07-06T03:46:37.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.078+0000 7f49b0f99700 1 -- 172.21.15.97:0/349486910 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a800ba00 con 0x7f49ac15c1c0 2023-07-06T03:46:37.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.078+0000 7f49b0f99700 1 -- 172.21.15.97:0/349486910 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a800bd90 con 0x7f49ac15c1c0 2023-07-06T03:46:37.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.078+0000 7f49b41ff700 1 -- 172.21.15.97:0/349486910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49ac15c1c0 msgr2=0x7f49ac0cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:37.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.078+0000 7f49b41ff700 1 --2- 172.21.15.97:0/349486910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49ac15c1c0 0x7f49ac0cc3e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f49a80098f0 tx=0x7f49a8009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:37.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.078+0000 7f49b41ff700 1 -- 172.21.15.97:0/349486910 shutdown_connections 2023-07-06T03:46:37.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.078+0000 7f49b41ff700 1 --2- 172.21.15.97:0/349486910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49ac0719c0 0x7f49ac06a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:37.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.078+0000 7f49b41ff700 1 --2- 172.21.15.97:0/349486910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49ac15c1c0 0x7f49ac0cc3e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:37.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.078+0000 7f49b41ff700 1 -- 172.21.15.97:0/349486910 >> 172.21.15.97:0/349486910 conn(0x7f49ac0b1650 msgr2=0x7f49ac0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:37.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.078+0000 7f49b41ff700 1 -- 172.21.15.97:0/349486910 shutdown_connections 2023-07-06T03:46:37.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.078+0000 7f49b41ff700 1 -- 172.21.15.97:0/349486910 wait complete. 2023-07-06T03:46:37.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.080+0000 7f49b41ff700 1 Processor -- start 2023-07-06T03:46:37.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.080+0000 7f49b41ff700 1 -- start start 2023-07-06T03:46:37.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.081+0000 7f49b41ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49ac0719c0 0x7f49ac1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:37.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.081+0000 7f49b41ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49ac15c1c0 0x7f49ac06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.081+0000 7f49b41ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49ac15c590 0x7f49ac0c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.081+0000 7f49b41ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0b3cf0 con 0x7f49ac0719c0 2023-07-06T03:46:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.081+0000 7f49b41ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0b2e20 con 0x7f49ac15c1c0 2023-07-06T03:46:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.081+0000 7f49b41ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0b3070 con 0x7f49ac15c590 2023-07-06T03:46:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.081+0000 7f49b1f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49ac0719c0 0x7f49ac1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.081+0000 7f49b1f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49ac0719c0 0x7f49ac1504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48096/0 (socket says 172.21.15.97:48096) 2023-07-06T03:46:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.081+0000 7f49b1f9b700 1 -- 172.21.15.97:0/3554132835 learned_addr learned my addr 172.21.15.97:0/3554132835 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.081+0000 7f49b179a700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49ac15c1c0 0x7f49ac06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.081+0000 7f49b279c700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49ac15c590 0x7f49ac0c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.082+0000 7f49b1f9b700 1 -- 172.21.15.97:0/3554132835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49ac15c590 msgr2=0x7f49ac0c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.082+0000 7f49b1f9b700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49ac15c590 0x7f49ac0c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.082+0000 7f49b1f9b700 1 -- 172.21.15.97:0/3554132835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49ac15c1c0 msgr2=0x7f49ac06ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.082+0000 7f49b1f9b700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49ac15c1c0 0x7f49ac06ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.082+0000 7f49b1f9b700 1 -- 172.21.15.97:0/3554132835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a8009550 con 0x7f49ac0719c0 2023-07-06T03:46:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.082+0000 7f49b1f9b700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49ac0719c0 0x7f49ac1504d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f49a8000c00 tx=0x7f49a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.082+0000 7f49a27fc700 1 -- 172.21.15.97:0/3554132835 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8004480 con 0x7f49ac0719c0 2023-07-06T03:46:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.082+0000 7f49a27fc700 1 -- 172.21.15.97:0/3554132835 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a801f070 con 0x7f49ac0719c0 2023-07-06T03:46:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.082+0000 7f49b41ff700 1 -- 172.21.15.97:0/3554132835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac0b3350 con 0x7f49ac0719c0 2023-07-06T03:46:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.082+0000 7f49a27fc700 1 -- 172.21.15.97:0/3554132835 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80038c0 con 0x7f49ac0719c0 2023-07-06T03:46:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.082+0000 7f49b41ff700 1 -- 172.21.15.97:0/3554132835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac0b3910 con 0x7f49ac0719c0 2023-07-06T03:46:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.083+0000 7f49b41ff700 1 -- 172.21.15.97:0/3554132835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac0b3aa0 con 0x7f49ac0719c0 2023-07-06T03:46:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.084+0000 7f49a27fc700 1 -- 172.21.15.97:0/3554132835 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f49a8021030 con 0x7f49ac0719c0 2023-07-06T03:46:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.085+0000 7f49a27fc700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f499006ace0 0x7f499006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.085+0000 7f49a27fc700 1 -- 172.21.15.97:0/3554132835 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f49a808a940 con 0x7f49ac0719c0 2023-07-06T03:46:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.085+0000 7f49b179a700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f499006ace0 0x7f499006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.086+0000 7f49b179a700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f499006ace0 0x7f499006d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f499c000f60 tx=0x7f499c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:37.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.090+0000 7f49a27fc700 1 -- 172.21.15.97:0/3554132835 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f49a8055c00 con 0x7f49ac0719c0 2023-07-06T03:46:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.329+0000 7f49b41ff700 1 -- 172.21.15.97:0/3554132835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f49ac04e340 con 0x7f49ac0719c0 2023-07-06T03:46:37.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.853+0000 7f49a27fc700 1 -- 172.21.15.97:0/3554132835 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f49a8059220 con 0x7f49ac0719c0 2023-07-06T03:46:37.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.858+0000 7f49b41ff700 1 -- 172.21.15.97:0/3554132835 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f499006ace0 msgr2=0x7f499006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:37.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.858+0000 7f49b41ff700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f499006ace0 0x7f499006d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f499c000f60 tx=0x7f499c009040 comp rx=0 tx=0).stop 2023-07-06T03:46:37.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.858+0000 7f49b41ff700 1 -- 172.21.15.97:0/3554132835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49ac0719c0 msgr2=0x7f49ac1504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:37.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.858+0000 7f49b41ff700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49ac0719c0 0x7f49ac1504d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f49a8000c00 tx=0x7f49a8004750 comp rx=0 tx=0).stop 2023-07-06T03:46:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.860+0000 7f49b41ff700 1 -- 172.21.15.97:0/3554132835 shutdown_connections 2023-07-06T03:46:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.860+0000 7f49b41ff700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f499006ace0 0x7f499006d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.860+0000 7f49b41ff700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49ac15c590 0x7f49ac0c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.860+0000 7f49b41ff700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49ac0719c0 0x7f49ac1504d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.860+0000 7f49b41ff700 1 --2- 172.21.15.97:0/3554132835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49ac15c1c0 0x7f49ac06ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.860+0000 7f49b41ff700 1 -- 172.21.15.97:0/3554132835 >> 172.21.15.97:0/3554132835 conn(0x7f49ac0b1650 msgr2=0x7f49ac0b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.867+0000 7f49b41ff700 1 -- 172.21.15.97:0/3554132835 shutdown_connections 2023-07-06T03:46:37.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:37.870+0000 7f49b41ff700 1 -- 172.21.15.97:0/3554132835 wait complete. 2023-07-06T03:46:37.875 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 5 and data pool 6 2023-07-06T03:46:37.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:38.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.085+0000 7fc3847ff700 1 Processor -- start 2023-07-06T03:46:38.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.086+0000 7fc3847ff700 1 -- start start 2023-07-06T03:46:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc3847ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc37c15c520 0x7fc37c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc3847ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc37c160d00 0x7fc37c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc3847ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37c0cdae0 con 0x7fc37c160180 2023-07-06T03:46:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc3847ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37c0cd8d0 con 0x7fc37c160d00 2023-07-06T03:46:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc3847ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c0b6640 con 0x7fc37c15c520 2023-07-06T03:46:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc382d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc37c160d00 0x7fc37c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc382d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc37c160d00 0x7fc37c0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48108/0 (socket says 172.21.15.97:48108) 2023-07-06T03:46:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc382d9c700 1 -- 172.21.15.97:0/3441378747 learned_addr learned my addr 172.21.15.97:0/3441378747 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc38259b700 1 --2- 172.21.15.97:0/3441378747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc37c15c520 0x7fc37c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc382d9c700 1 -- 172.21.15.97:0/3441378747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc37c15c520 msgr2=0x7fc37c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc382d9c700 1 --2- 172.21.15.97:0/3441378747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc37c15c520 0x7fc37c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc382d9c700 1 -- 172.21.15.97:0/3441378747 >> v1:172.21.15.146:6789/0 conn(0x7fc37c160180 legacy=0x7fc37c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc382d9c700 1 -- 172.21.15.97:0/3441378747 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c0b6920 con 0x7fc37c160d00 2023-07-06T03:46:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.087+0000 7fc381599700 1 -- 172.21.15.97:0/3441378747 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684421974 0 0) 0x7fc37c0cdae0 con 0x7fc37c160180 2023-07-06T03:46:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.088+0000 7fc382d9c700 1 --2- 172.21.15.97:0/3441378747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc37c160d00 0x7fc37c0b7400 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc374009e50 tx=0x7fc37400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1282acca385b71c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.088+0000 7fc381599700 1 -- 172.21.15.97:0/3441378747 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37400e070 con 0x7fc37c160d00 2023-07-06T03:46:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.088+0000 7fc381599700 1 -- 172.21.15.97:0/3441378747 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37400b910 con 0x7fc37c160d00 2023-07-06T03:46:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.088+0000 7fc381599700 1 -- 172.21.15.97:0/3441378747 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37400bcc0 con 0x7fc37c160d00 2023-07-06T03:46:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.088+0000 7fc3847ff700 1 -- 172.21.15.97:0/3441378747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc37c160d00 msgr2=0x7fc37c0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.088+0000 7fc3847ff700 1 --2- 172.21.15.97:0/3441378747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc37c160d00 0x7fc37c0b7400 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc374009e50 tx=0x7fc37400b330 comp rx=0 tx=0).stop 2023-07-06T03:46:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.088+0000 7fc3847ff700 1 -- 172.21.15.97:0/3441378747 shutdown_connections 2023-07-06T03:46:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.088+0000 7fc3847ff700 1 --2- 172.21.15.97:0/3441378747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc37c160d00 0x7fc37c0b7400 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.089+0000 7fc3847ff700 1 --2- 172.21.15.97:0/3441378747 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc37c15c520 0x7fc37c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.089+0000 7fc3847ff700 1 -- 172.21.15.97:0/3441378747 >> 172.21.15.97:0/3441378747 conn(0x7fc37c071790 msgr2=0x7fc37c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.089+0000 7fc3847ff700 1 -- 172.21.15.97:0/3441378747 shutdown_connections 2023-07-06T03:46:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.089+0000 7fc3847ff700 1 -- 172.21.15.97:0/3441378747 wait complete. 2023-07-06T03:46:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.090+0000 7fc3847ff700 1 Processor -- start 2023-07-06T03:46:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.090+0000 7fc3847ff700 1 -- start start 2023-07-06T03:46:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.091+0000 7fc3847ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c15c520 0x7fc37c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.091+0000 7fc3847ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc37c160180 0x7fc37c0d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.091+0000 7fc3847ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc37c160d00 0x7fc37c15f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.091+0000 7fc3847ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c156150 con 0x7fc37c160d00 2023-07-06T03:46:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.091+0000 7fc3847ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c1e6ea0 con 0x7fc37c15c520 2023-07-06T03:46:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.091+0000 7fc3847ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c1e7180 con 0x7fc37c160180 2023-07-06T03:46:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.091+0000 7fc382d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc37c160d00 0x7fc37c15f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc382d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc37c160d00 0x7fc37c15f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48118/0 (socket says 172.21.15.97:48118) 2023-07-06T03:46:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc382d9c700 1 -- 172.21.15.97:0/3554641149 learned_addr learned my addr 172.21.15.97:0/3554641149 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc38259b700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c15c520 0x7fc37c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc381d9a700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc37c160180 0x7fc37c0d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc382d9c700 1 -- 172.21.15.97:0/3554641149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc37c160180 msgr2=0x7fc37c0d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc382d9c700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc37c160180 0x7fc37c0d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc382d9c700 1 -- 172.21.15.97:0/3554641149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c15c520 msgr2=0x7fc37c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc382d9c700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c15c520 0x7fc37c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc382d9c700 1 -- 172.21.15.97:0/3554641149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc374009b30 con 0x7fc37c160d00 2023-07-06T03:46:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc381d9a700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc37c160180 0x7fc37c0d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc382d9c700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc37c160d00 0x7fc37c15f0f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc37c06c240 tx=0x7fc374004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc372ffd700 1 -- 172.21.15.97:0/3554641149 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374003ea0 con 0x7fc37c160d00 2023-07-06T03:46:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc372ffd700 1 -- 172.21.15.97:0/3554641149 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3740099b0 con 0x7fc37c160d00 2023-07-06T03:46:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc3847ff700 1 -- 172.21.15.97:0/3554641149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37c1e7460 con 0x7fc37c160d00 2023-07-06T03:46:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.092+0000 7fc372ffd700 1 -- 172.21.15.97:0/3554641149 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37401c650 con 0x7fc37c160d00 2023-07-06T03:46:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.093+0000 7fc3847ff700 1 -- 172.21.15.97:0/3554641149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37c1e7a20 con 0x7fc37c160d00 2023-07-06T03:46:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.093+0000 7fc3847ff700 1 -- 172.21.15.97:0/3554641149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37c1e7c10 con 0x7fc37c160d00 2023-07-06T03:46:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.094+0000 7fc372ffd700 1 -- 172.21.15.97:0/3554641149 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc37401c7b0 con 0x7fc37c160d00 2023-07-06T03:46:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.095+0000 7fc372ffd700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc36006ace0 0x7fc36006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.096+0000 7fc372ffd700 1 -- 172.21.15.97:0/3554641149 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc37408a660 con 0x7fc37c160d00 2023-07-06T03:46:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.096+0000 7fc38259b700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc36006ace0 0x7fc36006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.096+0000 7fc38259b700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc36006ace0 0x7fc36006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc378009b40 tx=0x7fc378009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:38.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.100+0000 7fc372ffd700 1 -- 172.21.15.97:0/3554641149 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc374055700 con 0x7fc37c160d00 2023-07-06T03:46:38.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.294+0000 7fc3847ff700 1 -- 172.21.15.97:0/3554641149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc37c04e340 con 0x7fc37c160d00 2023-07-06T03:46:38.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.295+0000 7fc372ffd700 1 -- 172.21.15.97:0/3554641149 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+18390 (secure 0 0 0) 0x7fc374058d20 con 0x7fc37c160d00 2023-07-06T03:46:38.297 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:38.297 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:46:37.330848+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:46:35.584013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T03:46:36.303248+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:38.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.297+0000 7fc3847ff700 1 -- 172.21.15.97:0/3554641149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc36006ace0 msgr2=0x7fc36006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.297+0000 7fc3847ff700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc36006ace0 0x7fc36006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc378009b40 tx=0x7fc378009210 comp rx=0 tx=0).stop 2023-07-06T03:46:38.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.297+0000 7fc3847ff700 1 -- 172.21.15.97:0/3554641149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc37c160d00 msgr2=0x7fc37c15f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.297+0000 7fc3847ff700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc37c160d00 0x7fc37c15f0f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc37c06c240 tx=0x7fc374004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.297+0000 7fc3847ff700 1 -- 172.21.15.97:0/3554641149 shutdown_connections 2023-07-06T03:46:38.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.297+0000 7fc3847ff700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc36006ace0 0x7fc36006d190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.297+0000 7fc3847ff700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc37c160d00 0x7fc37c15f0f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.297+0000 7fc3847ff700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc37c15c520 0x7fc37c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.297+0000 7fc3847ff700 1 --2- 172.21.15.97:0/3554641149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc37c160180 0x7fc37c0d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.297+0000 7fc3847ff700 1 -- 172.21.15.97:0/3554641149 >> 172.21.15.97:0/3554641149 conn(0x7fc37c071790 msgr2=0x7fc37c0b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:38.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.298+0000 7fc3847ff700 1 -- 172.21.15.97:0/3554641149 shutdown_connections 2023-07-06T03:46:38.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.298+0000 7fc3847ff700 1 -- 172.21.15.97:0/3554641149 wait complete. 2023-07-06T03:46:38.317 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:38.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.508+0000 7ff552bff700 1 Processor -- start 2023-07-06T03:46:38.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.508+0000 7ff552bff700 1 -- start start 2023-07-06T03:46:38.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.509+0000 7ff552bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff54c158f20 0x7ff54c0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.509+0000 7ff552bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff54c1592f0 0x7ff54c0b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.509+0000 7ff552bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c15cc40 con 0x7ff54c06d860 2023-07-06T03:46:38.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.509+0000 7ff552bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c15cf20 con 0x7ff54c1592f0 2023-07-06T03:46:38.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.509+0000 7ff552bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c15d200 con 0x7ff54c158f20 2023-07-06T03:46:38.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.510+0000 7ff54bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff54c1592f0 0x7ff54c0b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.510+0000 7ff54bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff54c1592f0 0x7ff54c0b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48130/0 (socket says 172.21.15.97:48130) 2023-07-06T03:46:38.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.510+0000 7ff54bfff700 1 -- 172.21.15.97:0/2268874047 learned_addr learned my addr 172.21.15.97:0/2268874047 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:38.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.510+0000 7ff55099b700 1 --2- 172.21.15.97:0/2268874047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff54c158f20 0x7ff54c0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.510+0000 7ff54bfff700 1 -- 172.21.15.97:0/2268874047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff54c158f20 msgr2=0x7ff54c0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:38.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.510+0000 7ff54bfff700 1 --2- 172.21.15.97:0/2268874047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff54c158f20 0x7ff54c0caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.510+0000 7ff54bfff700 1 -- 172.21.15.97:0/2268874047 >> v1:172.21.15.97:6789/0 conn(0x7ff54c06d860 legacy=0x7ff54c0cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.510+0000 7ff54bfff700 1 -- 172.21.15.97:0/2268874047 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c15d4e0 con 0x7ff54c1592f0 2023-07-06T03:46:38.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.510+0000 7ff54bfff700 1 --2- 172.21.15.97:0/2268874047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54c1592f0 0x7ff54c0b0560 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff5380098f0 tx=0x7ff538009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5d2948f30fb17e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:38.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.510+0000 7ff54b7fe700 1 -- 172.21.15.97:0/2268874047 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53800b8a0 con 0x7ff54c1592f0 2023-07-06T03:46:38.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.511+0000 7ff54b7fe700 1 -- 172.21.15.97:0/2268874047 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53800ba00 con 0x7ff54c1592f0 2023-07-06T03:46:38.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.511+0000 7ff54b7fe700 1 -- 172.21.15.97:0/2268874047 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53800bd90 con 0x7ff54c1592f0 2023-07-06T03:46:38.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.511+0000 7ff552bff700 1 -- 172.21.15.97:0/2268874047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54c1592f0 msgr2=0x7ff54c0b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.511+0000 7ff552bff700 1 --2- 172.21.15.97:0/2268874047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54c1592f0 0x7ff54c0b0560 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff5380098f0 tx=0x7ff538009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:38.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.511+0000 7ff552bff700 1 -- 172.21.15.97:0/2268874047 shutdown_connections 2023-07-06T03:46:38.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.511+0000 7ff552bff700 1 --2- 172.21.15.97:0/2268874047 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff54c158f20 0x7ff54c0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.511+0000 7ff552bff700 1 --2- 172.21.15.97:0/2268874047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54c1592f0 0x7ff54c0b0560 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.511+0000 7ff552bff700 1 -- 172.21.15.97:0/2268874047 >> 172.21.15.97:0/2268874047 conn(0x7ff54c0ce8c0 msgr2=0x7ff54c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:38.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.511+0000 7ff552bff700 1 -- 172.21.15.97:0/2268874047 shutdown_connections 2023-07-06T03:46:38.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.511+0000 7ff552bff700 1 -- 172.21.15.97:0/2268874047 wait complete. 2023-07-06T03:46:38.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.513+0000 7ff552bff700 1 Processor -- start 2023-07-06T03:46:38.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.513+0000 7ff552bff700 1 -- start start 2023-07-06T03:46:38.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.514+0000 7ff552bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54c06d860 0x7ff54c0ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.514+0000 7ff552bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54c158f20 0x7ff54c0addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.514+0000 7ff552bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff54c1592f0 0x7ff54c147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.514+0000 7ff552bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c148ca0 con 0x7ff54c06d860 2023-07-06T03:46:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.514+0000 7ff552bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1554f0 con 0x7ff54c158f20 2023-07-06T03:46:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.514+0000 7ff552bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1557d0 con 0x7ff54c1592f0 2023-07-06T03:46:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.514+0000 7ff55099b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54c06d860 0x7ff54c0ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.514+0000 7ff55099b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54c06d860 0x7ff54c0ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48142/0 (socket says 172.21.15.97:48142) 2023-07-06T03:46:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.514+0000 7ff55099b700 1 -- 172.21.15.97:0/1556731202 learned_addr learned my addr 172.21.15.97:0/1556731202 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.514+0000 7ff54bfff700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54c158f20 0x7ff54c0addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.514+0000 7ff55119c700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff54c1592f0 0x7ff54c147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.515+0000 7ff55099b700 1 -- 172.21.15.97:0/1556731202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff54c1592f0 msgr2=0x7ff54c147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.515+0000 7ff55099b700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff54c1592f0 0x7ff54c147690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.515+0000 7ff55099b700 1 -- 172.21.15.97:0/1556731202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54c158f20 msgr2=0x7ff54c0addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.515+0000 7ff55099b700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54c158f20 0x7ff54c0addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.515+0000 7ff55099b700 1 -- 172.21.15.97:0/1556731202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff538009550 con 0x7ff54c06d860 2023-07-06T03:46:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.515+0000 7ff55099b700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54c06d860 0x7ff54c0ad8b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff54000e810 tx=0x7ff54000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.515+0000 7ff5497fa700 1 -- 172.21.15.97:0/1556731202 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54000c070 con 0x7ff54c06d860 2023-07-06T03:46:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.515+0000 7ff5497fa700 1 -- 172.21.15.97:0/1556731202 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff540013610 con 0x7ff54c06d860 2023-07-06T03:46:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.515+0000 7ff552bff700 1 -- 172.21.15.97:0/1556731202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c155ab0 con 0x7ff54c06d860 2023-07-06T03:46:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.515+0000 7ff5497fa700 1 -- 172.21.15.97:0/1556731202 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400139c0 con 0x7ff54c06d860 2023-07-06T03:46:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.515+0000 7ff552bff700 1 -- 172.21.15.97:0/1556731202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c15be70 con 0x7ff54c06d860 2023-07-06T03:46:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.516+0000 7ff552bff700 1 -- 172.21.15.97:0/1556731202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c15b9f0 con 0x7ff54c06d860 2023-07-06T03:46:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.517+0000 7ff5497fa700 1 -- 172.21.15.97:0/1556731202 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff540013b20 con 0x7ff54c06d860 2023-07-06T03:46:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.518+0000 7ff5497fa700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53006ace0 0x7ff53006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.518+0000 7ff5497fa700 1 -- 172.21.15.97:0/1556731202 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5400892c0 con 0x7ff54c06d860 2023-07-06T03:46:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.519+0000 7ff54bfff700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53006ace0 0x7ff53006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.519+0000 7ff54bfff700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53006ace0 0x7ff53006d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff538014040 tx=0x7ff538008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:38.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.522+0000 7ff5497fa700 1 -- 172.21.15.97:0/1556731202 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff54c15b9f0 con 0x7ff54c06d860 2023-07-06T03:46:38.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.720+0000 7ff552bff700 1 -- 172.21.15.97:0/1556731202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff54c15c1e0 con 0x7ff54c06d860 2023-07-06T03:46:38.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.722+0000 7ff5497fa700 1 -- 172.21.15.97:0/1556731202 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+18390 (secure 0 0 0) 0x7ff540057d70 con 0x7ff54c06d860 2023-07-06T03:46:38.724 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:38.725 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:46:37.330848+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:46:35.584013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T03:46:36.303248+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 -- 172.21.15.97:0/1556731202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53006ace0 msgr2=0x7ff53006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53006ace0 0x7ff53006d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff538014040 tx=0x7ff538008030 comp rx=0 tx=0).stop 2023-07-06T03:46:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 -- 172.21.15.97:0/1556731202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54c06d860 msgr2=0x7ff54c0ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54c06d860 0x7ff54c0ad8b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff54000e810 tx=0x7ff54000eb20 comp rx=0 tx=0).stop 2023-07-06T03:46:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 -- 172.21.15.97:0/1556731202 shutdown_connections 2023-07-06T03:46:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53006ace0 0x7ff53006d190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff54c1592f0 0x7ff54c147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff54c06d860 0x7ff54c0ad8b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 --2- 172.21.15.97:0/1556731202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff54c158f20 0x7ff54c0addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 -- 172.21.15.97:0/1556731202 >> 172.21.15.97:0/1556731202 conn(0x7ff54c0ce8c0 msgr2=0x7ff54c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 -- 172.21.15.97:0/1556731202 shutdown_connections 2023-07-06T03:46:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.724+0000 7ff552bff700 1 -- 172.21.15.97:0/1556731202 wait complete. 2023-07-06T03:46:38.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T03:46:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.943+0000 7fa8e27ff700 1 Processor -- start 2023-07-06T03:46:38.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.944+0000 7fa8e27ff700 1 -- start start 2023-07-06T03:46:38.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.945+0000 7fa8e27ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0d2210 0x7fa8dc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.945+0000 7fa8e27ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc0cff90 con 0x7fa8dc06b360 2023-07-06T03:46:38.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.945+0000 7fa8e27ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc0b5650 con 0x7fa8dc15c610 2023-07-06T03:46:38.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.945+0000 7fa8e27ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0d0780 con 0x7fa8dc0d2210 2023-07-06T03:46:38.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.945+0000 7fa8db7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa8dc06b360 0x7fa8dc0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54648/0 (socket says 172.21.15.97:54648) 2023-07-06T03:46:38.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.945+0000 7fa8db7fe700 1 -- 172.21.15.97:0/515776450 learned_addr learned my addr 172.21.15.97:0/515776450 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:38.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.945+0000 7fa8e0d9c700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0d2210 0x7fa8dc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.945+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264102101 0 0) 0x7fa8dc0cff90 con 0x7fa8dc06b360 2023-07-06T03:46:38.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.945+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8c40035b0 con 0x7fa8dc06b360 2023-07-06T03:46:38.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020946137 0 0) 0x7fa8dc0b5650 con 0x7fa8dc15c610 2023-07-06T03:46:38.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8dc0cff90 con 0x7fa8dc15c610 2023-07-06T03:46:38.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355335364 0 0) 0x7fa8c40035b0 con 0x7fa8dc06b360 2023-07-06T03:46:38.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0d2210 msgr2=0x7fa8dc0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:38.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0d2210 0x7fa8dc0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 >> v1:172.21.15.146:6789/0 conn(0x7fa8dc15c610 legacy=0x7fa8dc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0b5400 con 0x7fa8dc06b360 2023-07-06T03:46:38.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fa8d0001670 con 0x7fa8dc06b360 2023-07-06T03:46:38.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 >> v1:172.21.15.97:6789/0 conn(0x7fa8dc06b360 legacy=0x7fa8dc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0d2210 0x7fa8c4003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8c4004170 0x7fa8c4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 0x7fa8c4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0d0780 con 0x7fa8c4004d90 2023-07-06T03:46:38.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c400bb20 con 0x7fa8c4004170 2023-07-06T03:46:38.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c400bce0 con 0x7fa8dc0d2210 2023-07-06T03:46:38.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8e0d9c700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 0x7fa8c4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8db7fe700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8c4004170 0x7fa8c4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.946+0000 7fa8dbfff700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0d2210 0x7fa8c4003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8e0d9c700 1 -- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0d2210 msgr2=0x7fa8c4003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8e0d9c700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0d2210 0x7fa8c4003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8e0d9c700 1 -- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8c4004170 msgr2=0x7fa8c4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8e0d9c700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8c4004170 0x7fa8c4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8e0d9c700 1 -- 172.21.15.97:0/515776450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40035b0 con 0x7fa8c4004d90 2023-07-06T03:46:38.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8db7fe700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8c4004170 0x7fa8c4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:38.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8dbfff700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0d2210 0x7fa8c4003bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:38.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8e0d9c700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 0x7fa8c4005200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009fc0 tx=0x7fa8c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a33cdd2d54d36fe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:38.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400ec30 con 0x7fa8c4004d90 2023-07-06T03:46:38.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d400f040 con 0x7fa8c4004d90 2023-07-06T03:46:38.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8daffd700 1 -- 172.21.15.97:0/515776450 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4014480 con 0x7fa8c4004d90 2023-07-06T03:46:38.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8e27ff700 1 -- 172.21.15.97:0/515776450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 msgr2=0x7fa8c4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8e27ff700 1 --2- 172.21.15.97:0/515776450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 0x7fa8c4005200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009fc0 tx=0x7fa8c4005880 comp rx=0 tx=0).stop 2023-07-06T03:46:38.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.947+0000 7fa8e0d9c700 1 -- 172.21.15.97:0/515776450 reap_dead start 2023-07-06T03:46:38.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.948+0000 7fa8e27ff700 1 -- 172.21.15.97:0/515776450 shutdown_connections 2023-07-06T03:46:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.948+0000 7fa8e27ff700 1 -- 172.21.15.97:0/515776450 >> 172.21.15.97:0/515776450 conn(0x7fa8dc06a6c0 msgr2=0x7fa8dc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.948+0000 7fa8e27ff700 1 -- 172.21.15.97:0/515776450 shutdown_connections 2023-07-06T03:46:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.948+0000 7fa8e27ff700 1 -- 172.21.15.97:0/515776450 wait complete. 2023-07-06T03:46:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.950+0000 7fa8e27ff700 1 Processor -- start 2023-07-06T03:46:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.950+0000 7fa8e27ff700 1 -- start start 2023-07-06T03:46:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.951+0000 7fa8e27ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 0x7fa8dc14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.951+0000 7fa8e27ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8dc0cc370 0x7fa8dc0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.951+0000 7fa8e27ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0c6da0 0x7fa8dc0ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8e27ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0cbe70 con 0x7fa8c4004d90 2023-07-06T03:46:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8e27ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc15eb90 con 0x7fa8dc0cc370 2023-07-06T03:46:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8e27ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc15e580 con 0x7fa8dc0c6da0 2023-07-06T03:46:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8dbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 0x7fa8dc14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8dbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 0x7fa8dc14d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48150/0 (socket says 172.21.15.97:48150) 2023-07-06T03:46:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8dbfff700 1 -- 172.21.15.97:0/320465795 learned_addr learned my addr 172.21.15.97:0/320465795 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8db7fe700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8dc0cc370 0x7fa8dc0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8e0d9c700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0c6da0 0x7fa8dc0ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8dbfff700 1 -- 172.21.15.97:0/320465795 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0c6da0 msgr2=0x7fa8dc0ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8dbfff700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0c6da0 0x7fa8dc0ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8dbfff700 1 -- 172.21.15.97:0/320465795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8dc0cc370 msgr2=0x7fa8dc0c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8dbfff700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8dc0cc370 0x7fa8dc0c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8dbfff700 1 -- 172.21.15.97:0/320465795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0cc830 con 0x7fa8c4004d90 2023-07-06T03:46:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8db7fe700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8dc0cc370 0x7fa8dc0c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.952+0000 7fa8dbfff700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 0x7fa8dc14d200 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e7f0 tx=0x7fa8cc00ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.953+0000 7fa8d8ff9700 1 -- 172.21.15.97:0/320465795 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc017070 con 0x7fa8c4004d90 2023-07-06T03:46:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.953+0000 7fa8d8ff9700 1 -- 172.21.15.97:0/320465795 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc0052e0 con 0x7fa8c4004d90 2023-07-06T03:46:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.953+0000 7fa8e27ff700 1 -- 172.21.15.97:0/320465795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc0ccb10 con 0x7fa8c4004d90 2023-07-06T03:46:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.953+0000 7fa8d8ff9700 1 -- 172.21.15.97:0/320465795 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc005450 con 0x7fa8c4004d90 2023-07-06T03:46:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.953+0000 7fa8e27ff700 1 -- 172.21.15.97:0/320465795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc14d740 con 0x7fa8c4004d90 2023-07-06T03:46:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.954+0000 7fa8e27ff700 1 -- 172.21.15.97:0/320465795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc04e340 con 0x7fa8c4004d90 2023-07-06T03:46:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.955+0000 7fa8d8ff9700 1 -- 172.21.15.97:0/320465795 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa8cc006860 con 0x7fa8c4004d90 2023-07-06T03:46:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.955+0000 7fa8d8ff9700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8c806ace0 0x7fa8c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.956+0000 7fa8d8ff9700 1 -- 172.21.15.97:0/320465795 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8cc088d80 con 0x7fa8c4004d90 2023-07-06T03:46:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.956+0000 7fa8db7fe700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8c806ace0 0x7fa8c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.956+0000 7fa8db7fe700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8c806ace0 0x7fa8c806d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00017a0 tx=0x7fa8d0000ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:38.960+0000 7fa8d8ff9700 1 -- 172.21.15.97:0/320465795 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa8cc053d90 con 0x7fa8c4004d90 2023-07-06T03:46:39.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.229+0000 7fa8e27ff700 1 -- 172.21.15.97:0/320465795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa8dc0cd4f0 con 0x7fa8c4004d90 2023-07-06T03:46:39.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.865+0000 7fa8d8ff9700 1 -- 172.21.15.97:0/320465795 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa8cc0573b0 con 0x7fa8c4004d90 2023-07-06T03:46:39.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 -- 172.21.15.97:0/320465795 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8c806ace0 msgr2=0x7fa8c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:39.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8c806ace0 0x7fa8c806d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00017a0 tx=0x7fa8d0000ef0 comp rx=0 tx=0).stop 2023-07-06T03:46:39.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 -- 172.21.15.97:0/320465795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 msgr2=0x7fa8dc14d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:39.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 0x7fa8dc14d200 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e7f0 tx=0x7fa8cc00ebb0 comp rx=0 tx=0).stop 2023-07-06T03:46:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 -- 172.21.15.97:0/320465795 shutdown_connections 2023-07-06T03:46:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8c806ace0 0x7fa8c806d190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8dc0c6da0 0x7fa8dc0ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c4004d90 0x7fa8dc14d200 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 --2- 172.21.15.97:0/320465795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8dc0cc370 0x7fa8dc0c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 -- 172.21.15.97:0/320465795 >> 172.21.15.97:0/320465795 conn(0x7fa8dc06a6c0 msgr2=0x7fa8dc0b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 -- 172.21.15.97:0/320465795 shutdown_connections 2023-07-06T03:46:39.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:39.868+0000 7fa8e27ff700 1 -- 172.21.15.97:0/320465795 wait complete. 2023-07-06T03:46:39.889 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T03:46:39.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T03:46:40.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.086+0000 7f080ddff700 1 Processor -- start 2023-07-06T03:46:40.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.086+0000 7f080ddff700 1 -- start start 2023-07-06T03:46:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.087+0000 7f080ddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f080815c610 0x7f08080b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.087+0000 7f080ddff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f08080d2210 0x7f08080d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.087+0000 7f080ddff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08080cff90 con 0x7f080806b360 2023-07-06T03:46:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.087+0000 7f080ddff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08080d0780 con 0x7f08080d2210 2023-07-06T03:46:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.087+0000 7f080ddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080815c9e0 con 0x7f080815c610 2023-07-06T03:46:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.088+0000 7f0807fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f08080d2210 0x7f08080d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.088+0000 7f0806ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f080806b360 0x7f08080b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:48776/0 (socket says 172.21.15.97:48776) 2023-07-06T03:46:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.088+0000 7f0806ffd700 1 -- 172.21.15.97:0/1855139339 learned_addr learned my addr 172.21.15.97:0/1855139339 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.088+0000 7f08077fe700 1 --2- 172.21.15.97:0/1855139339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f080815c610 0x7f08080b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.088+0000 7f0807fff700 1 -- 172.21.15.97:0/1855139339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f080815c610 msgr2=0x7f08080b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:40.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.088+0000 7f0807fff700 1 --2- 172.21.15.97:0/1855139339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f080815c610 0x7f08080b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:40.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.088+0000 7f0807fff700 1 -- 172.21.15.97:0/1855139339 >> v1:172.21.15.146:6789/0 conn(0x7f080806b360 legacy=0x7f08080b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:40.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.088+0000 7f0807fff700 1 -- 172.21.15.97:0/1855139339 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08080b5650 con 0x7f08080d2210 2023-07-06T03:46:40.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.089+0000 7f0807fff700 1 --2- 172.21.15.97:0/1855139339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08080d2210 0x7f08080d1310 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0800009dc0 tx=0x7f080000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccae83f677fcab32 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:40.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.089+0000 7f08067fc700 1 -- 172.21.15.97:0/1855139339 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080000e070 con 0x7f08080d2210 2023-07-06T03:46:40.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.089+0000 7f08067fc700 1 -- 172.21.15.97:0/1855139339 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080000bae0 con 0x7f08080d2210 2023-07-06T03:46:40.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.089+0000 7f08067fc700 1 -- 172.21.15.97:0/1855139339 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080000be90 con 0x7f08080d2210 2023-07-06T03:46:40.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.089+0000 7f080ddff700 1 -- 172.21.15.97:0/1855139339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08080d2210 msgr2=0x7f08080d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:40.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.089+0000 7f080ddff700 1 --2- 172.21.15.97:0/1855139339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08080d2210 0x7f08080d1310 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0800009dc0 tx=0x7f080000b330 comp rx=0 tx=0).stop 2023-07-06T03:46:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.089+0000 7f080ddff700 1 -- 172.21.15.97:0/1855139339 shutdown_connections 2023-07-06T03:46:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.090+0000 7f080ddff700 1 --2- 172.21.15.97:0/1855139339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08080d2210 0x7f08080d1310 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.090+0000 7f080ddff700 1 --2- 172.21.15.97:0/1855139339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f080815c610 0x7f08080b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.090+0000 7f080ddff700 1 -- 172.21.15.97:0/1855139339 >> 172.21.15.97:0/1855139339 conn(0x7f080806a6c0 msgr2=0x7f080806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.090+0000 7f080ddff700 1 -- 172.21.15.97:0/1855139339 shutdown_connections 2023-07-06T03:46:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.090+0000 7f080ddff700 1 -- 172.21.15.97:0/1855139339 wait complete. 2023-07-06T03:46:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.091+0000 7f080ddff700 1 Processor -- start 2023-07-06T03:46:40.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.091+0000 7f080ddff700 1 -- start start 2023-07-06T03:46:40.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.092+0000 7f080ddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f080806b360 0x7f08080cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:40.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.092+0000 7f080ddff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08080d2210 0x7f08080b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:40.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.092+0000 7f080ddff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f080815c610 0x7f08080b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:40.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.092+0000 7f080ddff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080815e040 con 0x7f080815c610 2023-07-06T03:46:40.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.092+0000 7f080ddff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08080cbdb0 con 0x7f08080d2210 2023-07-06T03:46:40.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.092+0000 7f080ddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08080cb7f0 con 0x7f080806b360 2023-07-06T03:46:40.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0807fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f080815c610 0x7f08080b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:40.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0806ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08080d2210 0x7f08080b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:40.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f08077fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f080806b360 0x7f08080cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:40.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0807fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f080815c610 0x7f08080b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58046/0 (socket says 172.21.15.97:58046) 2023-07-06T03:46:40.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0806ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08080d2210 0x7f08080b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:54594/0 (socket says 172.21.15.97:54594) 2023-07-06T03:46:40.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0807fff700 1 -- 172.21.15.97:0/823133618 learned_addr learned my addr 172.21.15.97:0/823133618 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:40.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0807fff700 1 -- 172.21.15.97:0/823133618 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f080806b360 msgr2=0x7f08080cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:40.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0807fff700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f080806b360 0x7f08080cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:40.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0807fff700 1 -- 172.21.15.97:0/823133618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08080d2210 msgr2=0x7f08080b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:40.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0807fff700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08080d2210 0x7f08080b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:40.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0807fff700 1 -- 172.21.15.97:0/823133618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800009a70 con 0x7f080815c610 2023-07-06T03:46:40.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0806ffd700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08080d2210 0x7f08080b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:46:40.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.093+0000 7f0807fff700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f080815c610 0x7f08080b4ce0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f080000b370 tx=0x7f0800004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:40.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.094+0000 7f07effff700 1 -- 172.21.15.97:0/823133618 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800004550 con 0x7f080815c610 2023-07-06T03:46:40.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.094+0000 7f07effff700 1 -- 172.21.15.97:0/823133618 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0800020070 con 0x7f080815c610 2023-07-06T03:46:40.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.094+0000 7f080ddff700 1 -- 172.21.15.97:0/823133618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08080cbad0 con 0x7f080815c610 2023-07-06T03:46:40.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.094+0000 7f07effff700 1 -- 172.21.15.97:0/823133618 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800004910 con 0x7f080815c610 2023-07-06T03:46:40.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.094+0000 7f080ddff700 1 -- 172.21.15.97:0/823133618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080815e550 con 0x7f080815c610 2023-07-06T03:46:40.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.095+0000 7f080ddff700 1 -- 172.21.15.97:0/823133618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08080b53a0 con 0x7f080815c610 2023-07-06T03:46:40.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.096+0000 7f07effff700 1 -- 172.21.15.97:0/823133618 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0800012430 con 0x7f080815c610 2023-07-06T03:46:40.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.097+0000 7f07effff700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f07f406ace0 0x7f07f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:40.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.097+0000 7f07effff700 1 -- 172.21.15.97:0/823133618 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0800022070 con 0x7f080815c610 2023-07-06T03:46:40.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.097+0000 7f08077fe700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f07f406ace0 0x7f07f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:40.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.097+0000 7f08077fe700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f07f406ace0 0x7f07f406d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009b20 tx=0x7f07f8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:40.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.101+0000 7f07effff700 1 -- 172.21.15.97:0/823133618 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0800059f60 con 0x7f080815c610 2023-07-06T03:46:40.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.343+0000 7f080ddff700 1 -- 172.21.15.97:0/823133618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f080815e770 con 0x7f080815c610 2023-07-06T03:46:40.819 INFO:tasks.ceph.mds.c.smithi097.stderr:2023-07-06T03:46:40.817+0000 7f5a8cdf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:46:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.867+0000 7f07effff700 1 -- 172.21.15.97:0/823133618 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0800059af0 con 0x7f080815c610 2023-07-06T03:46:40.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 -- 172.21.15.97:0/823133618 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f07f406ace0 msgr2=0x7f07f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:40.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f07f406ace0 0x7f07f406d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009b20 tx=0x7f07f8009210 comp rx=0 tx=0).stop 2023-07-06T03:46:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 -- 172.21.15.97:0/823133618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f080815c610 msgr2=0x7f08080b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f080815c610 0x7f08080b4ce0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f080000b370 tx=0x7f0800004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 -- 172.21.15.97:0/823133618 shutdown_connections 2023-07-06T03:46:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f07f406ace0 0x7f07f406d190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f080815c610 0x7f08080b4ce0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f080806b360 0x7f08080cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:40.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 --2- 172.21.15.97:0/823133618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08080d2210 0x7f08080b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:40.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 -- 172.21.15.97:0/823133618 >> 172.21.15.97:0/823133618 conn(0x7f080806a6c0 msgr2=0x7f08080b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:40.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 -- 172.21.15.97:0/823133618 shutdown_connections 2023-07-06T03:46:40.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:40.870+0000 7f080ddff700 1 -- 172.21.15.97:0/823133618 wait complete. 2023-07-06T03:46:40.892 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T03:46:40.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:46:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.088+0000 7f08b15ff700 1 Processor -- start 2023-07-06T03:46:41.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.088+0000 7f08b15ff700 1 -- start start 2023-07-06T03:46:41.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.089+0000 7f08b15ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08ac15c520 0x7f08ac0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.089+0000 7f08b15ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f08ac160180 0x7f08ac1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.089+0000 7f08b15ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08ac160b30 0x7f08ac0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.089+0000 7f08b15ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac0cd8d0 con 0x7f08ac160b30 2023-07-06T03:46:41.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.089+0000 7f08b15ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac0cdae0 con 0x7f08ac160180 2023-07-06T03:46:41.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.089+0000 7f08b15ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0b6640 con 0x7f08ac15c520 2023-07-06T03:46:41.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.090+0000 7f08ab7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08ac160b30 0x7f08ac0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.090+0000 7f08ab7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08ac160b30 0x7f08ac0cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58058/0 (socket says 172.21.15.97:58058) 2023-07-06T03:46:41.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.090+0000 7f08ab7fe700 1 -- 172.21.15.97:0/3814188900 learned_addr learned my addr 172.21.15.97:0/3814188900 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.090+0000 7f08aaffd700 1 --2- 172.21.15.97:0/3814188900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08ac15c520 0x7f08ac0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.090+0000 7f08ab7fe700 1 -- 172.21.15.97:0/3814188900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08ac15c520 msgr2=0x7f08ac0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.090+0000 7f08ab7fe700 1 --2- 172.21.15.97:0/3814188900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08ac15c520 0x7f08ac0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.090+0000 7f08ab7fe700 1 -- 172.21.15.97:0/3814188900 >> v2:172.21.15.146:3300/0 conn(0x7f08ac160180 msgr2=0x7f08ac1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.090+0000 7f08a3fff700 1 --2- 172.21.15.97:0/3814188900 >> v2:172.21.15.146:3300/0 conn(0x7f08ac160180 0x7f08ac1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.090+0000 7f08ab7fe700 1 --2- 172.21.15.97:0/3814188900 >> v2:172.21.15.146:3300/0 conn(0x7f08ac160180 0x7f08ac1605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.090+0000 7f08ab7fe700 1 -- 172.21.15.97:0/3814188900 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac0b6920 con 0x7f08ac160b30 2023-07-06T03:46:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.090+0000 7f08a3fff700 1 --2- 172.21.15.97:0/3814188900 >> v2:172.21.15.146:3300/0 conn(0x7f08ac160180 0x7f08ac1605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08ab7fe700 1 --2- 172.21.15.97:0/3814188900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac160b30 0x7f08ac0cf050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f08ac072770 tx=0x7f08a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e92f7026ae3056b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08aa7fc700 1 -- 172.21.15.97:0/3814188900 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400e070 con 0x7f08ac160b30 2023-07-06T03:46:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08aa7fc700 1 -- 172.21.15.97:0/3814188900 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400b9c0 con 0x7f08ac160b30 2023-07-06T03:46:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08aa7fc700 1 -- 172.21.15.97:0/3814188900 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400bd70 con 0x7f08ac160b30 2023-07-06T03:46:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08b15ff700 1 -- 172.21.15.97:0/3814188900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac160b30 msgr2=0x7f08ac0cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08b15ff700 1 --2- 172.21.15.97:0/3814188900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac160b30 0x7f08ac0cf050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f08ac072770 tx=0x7f08a400b330 comp rx=0 tx=0).stop 2023-07-06T03:46:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08b15ff700 1 -- 172.21.15.97:0/3814188900 shutdown_connections 2023-07-06T03:46:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08b15ff700 1 --2- 172.21.15.97:0/3814188900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac160b30 0x7f08ac0cf050 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08b15ff700 1 --2- 172.21.15.97:0/3814188900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08ac15c520 0x7f08ac0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08b15ff700 1 --2- 172.21.15.97:0/3814188900 >> v2:172.21.15.146:3300/0 conn(0x7f08ac160180 0x7f08ac1605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08b15ff700 1 -- 172.21.15.97:0/3814188900 >> 172.21.15.97:0/3814188900 conn(0x7f08ac071790 msgr2=0x7f08ac071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08b15ff700 1 -- 172.21.15.97:0/3814188900 shutdown_connections 2023-07-06T03:46:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.091+0000 7f08b15ff700 1 -- 172.21.15.97:0/3814188900 wait complete. 2023-07-06T03:46:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.093+0000 7f08b15ff700 1 Processor -- start 2023-07-06T03:46:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.093+0000 7f08b15ff700 1 -- start start 2023-07-06T03:46:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.094+0000 7f08b15ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08ac15c520 0x7f08ac150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.094+0000 7f08b15ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08ac160180 0x7f08ac0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.094+0000 7f08b15ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac160b30 0x7f08ac0d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.094+0000 7f08b15ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac14ef80 con 0x7f08ac160b30 2023-07-06T03:46:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.094+0000 7f08b15ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac14f260 con 0x7f08ac160180 2023-07-06T03:46:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.094+0000 7f08b15ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac14f540 con 0x7f08ac15c520 2023-07-06T03:46:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08ab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac160b30 0x7f08ac0d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08aaffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08ac15c520 0x7f08ac150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08aaffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08ac15c520 0x7f08ac150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:37824/0 (socket says 172.21.15.97:37824) 2023-07-06T03:46:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08aaffd700 1 -- 172.21.15.97:0/3500302792 learned_addr learned my addr 172.21.15.97:0/3500302792 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08a3fff700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08ac160180 0x7f08ac0d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08ab7fe700 1 -- 172.21.15.97:0/3500302792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08ac15c520 msgr2=0x7f08ac150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08ab7fe700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08ac15c520 0x7f08ac150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08ab7fe700 1 -- 172.21.15.97:0/3500302792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08ac160180 msgr2=0x7f08ac0d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08ab7fe700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08ac160180 0x7f08ac0d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08ab7fe700 1 -- 172.21.15.97:0/3500302792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009b30 con 0x7f08ac160b30 2023-07-06T03:46:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08ab7fe700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac160b30 0x7f08ac0d1180 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f08a4015000 tx=0x7f08a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.095+0000 7f08a37fe700 1 -- 172.21.15.97:0/3500302792 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40099b0 con 0x7f08ac160b30 2023-07-06T03:46:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.096+0000 7f08a37fe700 1 -- 172.21.15.97:0/3500302792 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a4004250 con 0x7f08ac160b30 2023-07-06T03:46:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.096+0000 7f08b15ff700 1 -- 172.21.15.97:0/3500302792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac14f820 con 0x7f08ac160b30 2023-07-06T03:46:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.096+0000 7f08a37fe700 1 -- 172.21.15.97:0/3500302792 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4004600 con 0x7f08ac160b30 2023-07-06T03:46:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.096+0000 7f08b15ff700 1 -- 172.21.15.97:0/3500302792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac0c96b0 con 0x7f08ac160b30 2023-07-06T03:46:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.097+0000 7f08b15ff700 1 -- 172.21.15.97:0/3500302792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac14fce0 con 0x7f08ac160b30 2023-07-06T03:46:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.098+0000 7f08a37fe700 1 -- 172.21.15.97:0/3500302792 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f08a4004760 con 0x7f08ac160b30 2023-07-06T03:46:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.098+0000 7f08a37fe700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088c06ace0 0x7f088c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.099+0000 7f08a37fe700 1 -- 172.21.15.97:0/3500302792 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f08a408a860 con 0x7f08ac160b30 2023-07-06T03:46:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.099+0000 7f08aaffd700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088c06ace0 0x7f088c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.099+0000 7f08aaffd700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088c06ace0 0x7f088c06d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f089c009a90 tx=0x7f089c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:41.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.103+0000 7f08a37fe700 1 -- 172.21.15.97:0/3500302792 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f08a40559b0 con 0x7f08ac160b30 2023-07-06T03:46:41.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.340+0000 7f08b15ff700 1 -- 172.21.15.97:0/3500302792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08ac04e340 con 0x7f08ac160b30 2023-07-06T03:46:41.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.341+0000 7f08a37fe700 1 -- 172.21.15.97:0/3500302792 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f08a4058fd0 con 0x7f08ac160b30 2023-07-06T03:46:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:41.343 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/1839628004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1839628004},{"type":"v1","addr":"172.21.15.146:6834","nonce":1839628004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4937,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2131262285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2131262285},{"type":"v1","addr":"172.21.15.146:6835","nonce":2131262285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1367937004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367937004},{"type":"v1","addr":"172.21.15.97:6835","nonce":1367937004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:46:37.331190+0000","modified":"2023-07-06T03:46:40.864523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/663657264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":663657264},{"type":"v1","addr":"172.21.15.97:6837","nonce":663657264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T03:46:41.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.343+0000 7f08b15ff700 1 -- 172.21.15.97:0/3500302792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088c06ace0 msgr2=0x7f088c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.343+0000 7f08b15ff700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088c06ace0 0x7f088c06d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f089c009a90 tx=0x7f089c008040 comp rx=0 tx=0).stop 2023-07-06T03:46:41.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.343+0000 7f08b15ff700 1 -- 172.21.15.97:0/3500302792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac160b30 msgr2=0x7f08ac0d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.343+0000 7f08b15ff700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac160b30 0x7f08ac0d1180 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f08a4015000 tx=0x7f08a4004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.343+0000 7f08b15ff700 1 -- 172.21.15.97:0/3500302792 shutdown_connections 2023-07-06T03:46:41.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.343+0000 7f08b15ff700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088c06ace0 0x7f088c06d190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.343+0000 7f08b15ff700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac160b30 0x7f08ac0d1180 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.343+0000 7f08b15ff700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08ac15c520 0x7f08ac150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.343+0000 7f08b15ff700 1 --2- 172.21.15.97:0/3500302792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08ac160180 0x7f08ac0d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.343+0000 7f08b15ff700 1 -- 172.21.15.97:0/3500302792 >> 172.21.15.97:0/3500302792 conn(0x7f08ac071790 msgr2=0x7f08ac15b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:41.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.344+0000 7f08b15ff700 1 -- 172.21.15.97:0/3500302792 shutdown_connections 2023-07-06T03:46:41.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.344+0000 7f08b15ff700 1 -- 172.21.15.97:0/3500302792 wait complete. 2023-07-06T03:46:41.348 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2023-07-06T03:46:41.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:46:41.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.559+0000 7f5437bff700 1 Processor -- start 2023-07-06T03:46:41.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.560+0000 7f5437bff700 1 -- start start 2023-07-06T03:46:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f5437bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f543015c520 0x7f54300cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f5437bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5430160d00 0x7f54300b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f5437bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54300cdae0 con 0x7f5430160180 2023-07-06T03:46:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f5437bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54300cd8d0 con 0x7f5430160d00 2023-07-06T03:46:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f5437bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54300b6640 con 0x7f543015c520 2023-07-06T03:46:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f543619c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5430160d00 0x7f54300b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f543619c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5430160d00 0x7f54300b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58090/0 (socket says 172.21.15.97:58090) 2023-07-06T03:46:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f543619c700 1 -- 172.21.15.97:0/2848697465 learned_addr learned my addr 172.21.15.97:0/2848697465 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:41.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f543599b700 1 --2- 172.21.15.97:0/2848697465 >> v2:172.21.15.146:3300/0 conn(0x7f543015c520 0x7f54300cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f543619c700 1 -- 172.21.15.97:0/2848697465 >> v2:172.21.15.146:3300/0 conn(0x7f543015c520 msgr2=0x7f54300cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:41.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f543619c700 1 --2- 172.21.15.97:0/2848697465 >> v2:172.21.15.146:3300/0 conn(0x7f543015c520 0x7f54300cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.561+0000 7f543619c700 1 -- 172.21.15.97:0/2848697465 >> v1:172.21.15.146:6789/0 conn(0x7f5430160180 legacy=0x7f54301605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:41.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.562+0000 7f543619c700 1 -- 172.21.15.97:0/2848697465 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54300b6920 con 0x7f5430160d00 2023-07-06T03:46:41.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.562+0000 7f543619c700 1 --2- 172.21.15.97:0/2848697465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430160d00 0x7f54300b7400 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f54300b4c70 tx=0x7f542800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cd44fc01a64141 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:41.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.562+0000 7f5434999700 1 -- 172.21.15.97:0/2848697465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800e070 con 0x7f5430160d00 2023-07-06T03:46:41.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.562+0000 7f5434999700 1 -- 172.21.15.97:0/2848697465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5428003a20 con 0x7f5430160d00 2023-07-06T03:46:41.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.562+0000 7f5434999700 1 -- 172.21.15.97:0/2848697465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428003dd0 con 0x7f5430160d00 2023-07-06T03:46:41.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.562+0000 7f5437bff700 1 -- 172.21.15.97:0/2848697465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430160d00 msgr2=0x7f54300b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.562+0000 7f5437bff700 1 --2- 172.21.15.97:0/2848697465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430160d00 0x7f54300b7400 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f54300b4c70 tx=0x7f542800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:41.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.563+0000 7f5437bff700 1 -- 172.21.15.97:0/2848697465 shutdown_connections 2023-07-06T03:46:41.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.563+0000 7f5437bff700 1 --2- 172.21.15.97:0/2848697465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430160d00 0x7f54300b7400 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.563+0000 7f5437bff700 1 --2- 172.21.15.97:0/2848697465 >> v2:172.21.15.146:3300/0 conn(0x7f543015c520 0x7f54300cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.563+0000 7f5437bff700 1 -- 172.21.15.97:0/2848697465 >> 172.21.15.97:0/2848697465 conn(0x7f5430071790 msgr2=0x7f5430071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:41.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.563+0000 7f5437bff700 1 -- 172.21.15.97:0/2848697465 shutdown_connections 2023-07-06T03:46:41.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.563+0000 7f5437bff700 1 -- 172.21.15.97:0/2848697465 wait complete. 2023-07-06T03:46:41.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.564+0000 7f5437bff700 1 Processor -- start 2023-07-06T03:46:41.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.565+0000 7f5437bff700 1 -- start start 2023-07-06T03:46:41.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.565+0000 7f5437bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f543015c520 0x7f543015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f5437bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5430160180 0x7f543015b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f5437bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430160d00 0x7f543015e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f5437bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543015bac0 con 0x7f5430160d00 2023-07-06T03:46:41.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f5437bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543015e940 con 0x7f543015c520 2023-07-06T03:46:41.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f5437bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543015aae0 con 0x7f5430160180 2023-07-06T03:46:41.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543619c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430160d00 0x7f543015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543619c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430160d00 0x7f543015e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58092/0 (socket says 172.21.15.97:58092) 2023-07-06T03:46:41.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543619c700 1 -- 172.21.15.97:0/1311432672 learned_addr learned my addr 172.21.15.97:0/1311432672 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543519a700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5430160180 0x7f543015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543599b700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f543015c520 0x7f543015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543619c700 1 -- 172.21.15.97:0/1311432672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5430160180 msgr2=0x7f543015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543619c700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5430160180 0x7f543015b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543619c700 1 -- 172.21.15.97:0/1311432672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f543015c520 msgr2=0x7f543015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543619c700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f543015c520 0x7f543015a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543619c700 1 -- 172.21.15.97:0/1311432672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5428009b30 con 0x7f5430160d00 2023-07-06T03:46:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543599b700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f543015c520 0x7f543015a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.566+0000 7f543619c700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430160d00 0x7f543015e400 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f542800b370 tx=0x7f5428004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.567+0000 7f54267fc700 1 -- 172.21.15.97:0/1311432672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428004440 con 0x7f5430160d00 2023-07-06T03:46:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.567+0000 7f54267fc700 1 -- 172.21.15.97:0/1311432672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54280045c0 con 0x7f5430160d00 2023-07-06T03:46:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.567+0000 7f5437bff700 1 -- 172.21.15.97:0/1311432672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543015d640 con 0x7f5430160d00 2023-07-06T03:46:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.567+0000 7f54267fc700 1 -- 172.21.15.97:0/1311432672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428004970 con 0x7f5430160d00 2023-07-06T03:46:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.567+0000 7f5437bff700 1 -- 172.21.15.97:0/1311432672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543015dc00 con 0x7f5430160d00 2023-07-06T03:46:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.568+0000 7f5437bff700 1 -- 172.21.15.97:0/1311432672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543004e340 con 0x7f5430160d00 2023-07-06T03:46:41.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.569+0000 7f54267fc700 1 -- 172.21.15.97:0/1311432672 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5428012430 con 0x7f5430160d00 2023-07-06T03:46:41.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.570+0000 7f54267fc700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541406ace0 0x7f541406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.570+0000 7f54267fc700 1 -- 172.21.15.97:0/1311432672 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f542808a850 con 0x7f5430160d00 2023-07-06T03:46:41.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.570+0000 7f543599b700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541406ace0 0x7f541406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.570+0000 7f543599b700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541406ace0 0x7f541406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f54300b2d00 tx=0x7f542c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:41.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.574+0000 7f54267fc700 1 -- 172.21.15.97:0/1311432672 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5428059380 con 0x7f5430160d00 2023-07-06T03:46:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.769+0000 7f5437bff700 1 -- 172.21.15.97:0/1311432672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f543015bc50 con 0x7f5430160d00 2023-07-06T03:46:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.770+0000 7f54267fc700 1 -- 172.21.15.97:0/1311432672 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18377 (secure 0 0 0) 0x7f5428058f10 con 0x7f5430160d00 2023-07-06T03:46:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:41.773 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":34,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:46:38.844552+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:46:35.584013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T03:46:36.303248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:46:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.772+0000 7f5437bff700 1 -- 172.21.15.97:0/1311432672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541406ace0 msgr2=0x7f541406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.772+0000 7f5437bff700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541406ace0 0x7f541406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f54300b2d00 tx=0x7f542c008040 comp rx=0 tx=0).stop 2023-07-06T03:46:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.773+0000 7f5437bff700 1 -- 172.21.15.97:0/1311432672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430160d00 msgr2=0x7f543015e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.773+0000 7f5437bff700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430160d00 0x7f543015e400 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f542800b370 tx=0x7f5428004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.773+0000 7f5437bff700 1 -- 172.21.15.97:0/1311432672 shutdown_connections 2023-07-06T03:46:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.773+0000 7f5437bff700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541406ace0 0x7f541406d190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.773+0000 7f5437bff700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430160d00 0x7f543015e400 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.773+0000 7f5437bff700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f543015c520 0x7f543015a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.773+0000 7f5437bff700 1 --2- 172.21.15.97:0/1311432672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5430160180 0x7f543015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.773+0000 7f5437bff700 1 -- 172.21.15.97:0/1311432672 >> 172.21.15.97:0/1311432672 conn(0x7f5430071790 msgr2=0x7f54300b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:41.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.773+0000 7f5437bff700 1 -- 172.21.15.97:0/1311432672 shutdown_connections 2023-07-06T03:46:41.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.773+0000 7f5437bff700 1 -- 172.21.15.97:0/1311432672 wait complete. 2023-07-06T03:46:41.793 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T03:46:41.793 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:46:41.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.987+0000 7f9c48fff700 1 Processor -- start 2023-07-06T03:46:41.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.987+0000 7f9c48fff700 1 -- start start 2023-07-06T03:46:41.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.989+0000 7f9c48fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9c44158960 0x7f9c4406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.989+0000 7f9c48fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4406ddb0 con 0x7f9c4415ce90 2023-07-06T03:46:41.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.989+0000 7f9c48fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4406bbf0 con 0x7f9c44071a00 2023-07-06T03:46:41.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.989+0000 7f9c48fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4406bed0 con 0x7f9c44158960 2023-07-06T03:46:41.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.989+0000 7f9c3bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9c4415ce90 0x7f9c440712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50844/0 (socket says 172.21.15.97:50844) 2023-07-06T03:46:41.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.989+0000 7f9c3bfff700 1 -- 172.21.15.97:0/3159176531 learned_addr learned my addr 172.21.15.97:0/3159176531 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:41.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.989+0000 7f9c4259c700 1 --2- 172.21.15.97:0/3159176531 >> v2:172.21.15.146:3300/0 conn(0x7f9c44158960 0x7f9c4406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.989+0000 7f9c41d9b700 1 -- 172.21.15.97:0/3159176531 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76672733 0 0) 0x7f9c4406ddb0 con 0x7f9c4415ce90 2023-07-06T03:46:41.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.989+0000 7f9c41d9b700 1 -- 172.21.15.97:0/3159176531 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c300035b0 con 0x7f9c4415ce90 2023-07-06T03:46:41.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.990+0000 7f9c4259c700 1 -- 172.21.15.97:0/3159176531 >> v1:172.21.15.146:6789/0 conn(0x7f9c44071a00 legacy=0x7f9c44158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.990+0000 7f9c4259c700 1 -- 172.21.15.97:0/3159176531 >> v1:172.21.15.97:6789/0 conn(0x7f9c4415ce90 legacy=0x7f9c440712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.990+0000 7f9c41d9b700 1 -- 172.21.15.97:0/3159176531 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1916061173 0 0) 0x7f9c300035b0 con 0x7f9c4415ce90 2023-07-06T03:46:41.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.990+0000 7f9c4259c700 1 -- 172.21.15.97:0/3159176531 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4406c1b0 con 0x7f9c44158960 2023-07-06T03:46:41.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.990+0000 7f9c4259c700 1 --2- 172.21.15.97:0/3159176531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c44158960 0x7f9c4406d6f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009840 tx=0x7f9c34009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b3306ce993d5965 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:41.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.990+0000 7f9c41d9b700 1 -- 172.21.15.97:0/3159176531 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400f8a0 con 0x7f9c44158960 2023-07-06T03:46:41.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.990+0000 7f9c41d9b700 1 -- 172.21.15.97:0/3159176531 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3400fa00 con 0x7f9c44158960 2023-07-06T03:46:41.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.990+0000 7f9c41d9b700 1 -- 172.21.15.97:0/3159176531 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400fd70 con 0x7f9c44158960 2023-07-06T03:46:41.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.991+0000 7f9c48fff700 1 -- 172.21.15.97:0/3159176531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c44158960 msgr2=0x7f9c4406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.991+0000 7f9c48fff700 1 --2- 172.21.15.97:0/3159176531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c44158960 0x7f9c4406d6f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009840 tx=0x7f9c34009b50 comp rx=0 tx=0).stop 2023-07-06T03:46:41.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.991+0000 7f9c48fff700 1 -- 172.21.15.97:0/3159176531 shutdown_connections 2023-07-06T03:46:41.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.991+0000 7f9c48fff700 1 --2- 172.21.15.97:0/3159176531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c44158960 0x7f9c4406d6f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.991+0000 7f9c48fff700 1 -- 172.21.15.97:0/3159176531 >> 172.21.15.97:0/3159176531 conn(0x7f9c440ce8c0 msgr2=0x7f9c440cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:41.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.991+0000 7f9c48fff700 1 -- 172.21.15.97:0/3159176531 shutdown_connections 2023-07-06T03:46:41.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.991+0000 7f9c48fff700 1 -- 172.21.15.97:0/3159176531 wait complete. 2023-07-06T03:46:41.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.993+0000 7f9c48fff700 1 Processor -- start 2023-07-06T03:46:41.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.993+0000 7f9c48fff700 1 -- start start 2023-07-06T03:46:41.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c48fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c44071a00 0x7f9c441ed910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c48fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c44158960 0x7f9c4414ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c48fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c4415ce90 0x7f9c44152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:41.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c48fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440b18d0 con 0x7f9c4415ce90 2023-07-06T03:46:41.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c48fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440b1b70 con 0x7f9c44071a00 2023-07-06T03:46:41.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c48fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c441ede50 con 0x7f9c44158960 2023-07-06T03:46:41.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c42d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c4415ce90 0x7f9c44152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c42d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c4415ce90 0x7f9c44152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58104/0 (socket says 172.21.15.97:58104) 2023-07-06T03:46:41.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c42d9d700 1 -- 172.21.15.97:0/2717820589 learned_addr learned my addr 172.21.15.97:0/2717820589 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:41.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c4259c700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c44071a00 0x7f9c441ed910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c3bfff700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c44158960 0x7f9c4414ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:41.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c42d9d700 1 -- 172.21.15.97:0/2717820589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c44158960 msgr2=0x7f9c4414ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c42d9d700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c44158960 0x7f9c4414ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c42d9d700 1 -- 172.21.15.97:0/2717820589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c44071a00 msgr2=0x7f9c441ed910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:41.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c42d9d700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c44071a00 0x7f9c441ed910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:41.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.994+0000 7f9c42d9d700 1 -- 172.21.15.97:0/2717820589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340094a0 con 0x7f9c4415ce90 2023-07-06T03:46:42.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.995+0000 7f9c42d9d700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c4415ce90 0x7f9c44152640 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00cc50 tx=0x7f9c3c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:42.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.995+0000 7f9c3affd700 1 -- 172.21.15.97:0/2717820589 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c003c00 con 0x7f9c4415ce90 2023-07-06T03:46:42.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.995+0000 7f9c3affd700 1 -- 172.21.15.97:0/2717820589 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c003d60 con 0x7f9c4415ce90 2023-07-06T03:46:42.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.995+0000 7f9c3affd700 1 -- 172.21.15.97:0/2717820589 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00aab0 con 0x7f9c4415ce90 2023-07-06T03:46:42.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.995+0000 7f9c48fff700 1 -- 172.21.15.97:0/2717820589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4414f5c0 con 0x7f9c4415ce90 2023-07-06T03:46:42.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.995+0000 7f9c48fff700 1 -- 172.21.15.97:0/2717820589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4414fb80 con 0x7f9c4415ce90 2023-07-06T03:46:42.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.996+0000 7f9c48fff700 1 -- 172.21.15.97:0/2717820589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c440b1d00 con 0x7f9c4415ce90 2023-07-06T03:46:42.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.997+0000 7f9c3affd700 1 -- 172.21.15.97:0/2717820589 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9c3c00ac10 con 0x7f9c4415ce90 2023-07-06T03:46:42.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.998+0000 7f9c3affd700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9c2406ace0 0x7f9c2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.998+0000 7f9c3affd700 1 -- 172.21.15.97:0/2717820589 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9c3c0894b0 con 0x7f9c4415ce90 2023-07-06T03:46:42.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.999+0000 7f9c4259c700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9c2406ace0 0x7f9c2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:41.999+0000 7f9c4259c700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9c2406ace0 0x7f9c2406d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9c340090b0 tx=0x7f9c34008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:42.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.003+0000 7f9c3affd700 1 -- 172.21.15.97:0/2717820589 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9c3c0544c0 con 0x7f9c4415ce90 2023-07-06T03:46:42.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.193+0000 7f9c48fff700 1 -- 172.21.15.97:0/2717820589 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c4415d300 con 0x7f9c2406ace0 2023-07-06T03:46:42.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.202+0000 7f9c3affd700 1 -- 172.21.15.97:0/2717820589 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299010 (secure 0 0 0) 0x7f9c3400f720 con 0x7f9c2406ace0 2023-07-06T03:46:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:42.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 -- 172.21.15.97:0/2717820589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9c2406ace0 msgr2=0x7f9c2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9c2406ace0 0x7f9c2406d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9c340090b0 tx=0x7f9c34008030 comp rx=0 tx=0).stop 2023-07-06T03:46:42.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 -- 172.21.15.97:0/2717820589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c4415ce90 msgr2=0x7f9c44152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c4415ce90 0x7f9c44152640 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00cc50 tx=0x7f9c3c00cf60 comp rx=0 tx=0).stop 2023-07-06T03:46:42.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 -- 172.21.15.97:0/2717820589 shutdown_connections 2023-07-06T03:46:42.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9c2406ace0 0x7f9c2406d190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c4415ce90 0x7f9c44152640 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c44071a00 0x7f9c441ed910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 --2- 172.21.15.97:0/2717820589 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9c44158960 0x7f9c4414ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 -- 172.21.15.97:0/2717820589 >> 172.21.15.97:0/2717820589 conn(0x7f9c440ce8c0 msgr2=0x7f9c4415b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 -- 172.21.15.97:0/2717820589 shutdown_connections 2023-07-06T03:46:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.207+0000 7f9c48fff700 1 -- 172.21.15.97:0/2717820589 wait complete. 2023-07-06T03:46:42.211 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:46:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-07-06T03:46:40.899512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219408,"kb_used_data":3792,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755312,"statfs":{"total":773094113280,"available":772869439488,"internally_reserved":0,"allocated":3883008,"data_stored":2226711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.073321"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.830173+0000","last_change":"2023-07-06T03:46:35.830173+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.830173+0000","last_undegraded":"2023-07-06T03:46:35.830173+0000","last_fullsized":"2023-07-06T03:46:35.830173+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.831990+0000","last_change":"2023-07-06T03:46:35.831990+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.831990+0000","last_undegraded":"2023-07-06T03:46:35.831990+0000","last_fullsized":"2023-07-06T03:46:35.831990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832506+0000","last_change":"2023-07-06T03:46:35.832506+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832506+0000","last_undegraded":"2023-07-06T03:46:35.832506+0000","last_fullsized":"2023-07-06T03:46:35.832506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.828829+0000","last_change":"2023-07-06T03:46:35.828829+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.828829+0000","last_undegraded":"2023-07-06T03:46:35.828829+0000","last_fullsized":"2023-07-06T03:46:35.828829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829789+0000","last_change":"2023-07-06T03:46:35.829789+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829789+0000","last_undegraded":"2023-07-06T03:46:35.829789+0000","last_fullsized":"2023-07-06T03:46:35.829789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.828825+0000","last_change":"2023-07-06T03:46:35.828825+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.828825+0000","last_undegraded":"2023-07-06T03:46:35.828825+0000","last_fullsized":"2023-07-06T03:46:35.828825+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.833925+0000","last_change":"2023-07-06T03:46:35.833925+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.833925+0000","last_undegraded":"2023-07-06T03:46:35.833925+0000","last_fullsized":"2023-07-06T03:46:35.833925+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.828738+0000","last_change":"2023-07-06T03:46:35.828738+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.828738+0000","last_undegraded":"2023-07-06T03:46:35.828738+0000","last_fullsized":"2023-07-06T03:46:35.828738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832906+0000","last_change":"2023-07-06T03:46:35.832906+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832906+0000","last_undegraded":"2023-07-06T03:46:35.832906+0000","last_fullsized":"2023-07-06T03:46:35.832906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829217+0000","last_change":"2023-07-06T03:46:35.829217+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829217+0000","last_undegraded":"2023-07-06T03:46:35.829217+0000","last_fullsized":"2023-07-06T03:46:35.829217+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829845+0000","last_change":"2023-07-06T03:46:35.829845+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829845+0000","last_undegraded":"2023-07-06T03:46:35.829845+0000","last_fullsized":"2023-07-06T03:46:35.829845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.833452+0000","last_change":"2023-07-06T03:46:35.833452+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.833452+0000","last_undegraded":"2023-07-06T03:46:35.833452+0000","last_fullsized":"2023-07-06T03:46:35.833452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832453+0000","last_change":"2023-07-06T03:46:35.832453+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832453+0000","last_undegraded":"2023-07-06T03:46:35.832453+0000","last_fullsized":"2023-07-06T03:46:35.832453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832823+0000","last_change":"2023-07-06T03:46:35.832823+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832823+0000","last_undegraded":"2023-07-06T03:46:35.832823+0000","last_fullsized":"2023-07-06T03:46:35.832823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.830021+0000","last_change":"2023-07-06T03:46:35.830021+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.830021+0000","last_undegraded":"2023-07-06T03:46:35.830021+0000","last_fullsized":"2023-07-06T03:46:35.830021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829310+0000","last_change":"2023-07-06T03:46:35.829310+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829310+0000","last_undegraded":"2023-07-06T03:46:35.829310+0000","last_fullsized":"2023-07-06T03:46:35.829310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829283+0000","last_change":"2023-07-06T03:46:35.829283+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829283+0000","last_undegraded":"2023-07-06T03:46:35.829283+0000","last_fullsized":"2023-07-06T03:46:35.829283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.828778+0000","last_change":"2023-07-06T03:46:35.828778+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.828778+0000","last_undegraded":"2023-07-06T03:46:35.828778+0000","last_fullsized":"2023-07-06T03:46:35.828778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.833883+0000","last_change":"2023-07-06T03:46:35.833883+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.833883+0000","last_undegraded":"2023-07-06T03:46:35.833883+0000","last_fullsized":"2023-07-06T03:46:35.833883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.835339+0000","last_change":"2023-07-06T03:46:35.835339+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.835339+0000","last_undegraded":"2023-07-06T03:46:35.835339+0000","last_fullsized":"2023-07-06T03:46:35.835339+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.835354+0000","last_change":"2023-07-06T03:46:35.835354+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.835354+0000","last_undegraded":"2023-07-06T03:46:35.835354+0000","last_fullsized":"2023-07-06T03:46:35.835354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.831939+0000","last_change":"2023-07-06T03:46:35.831939+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.831939+0000","last_undegraded":"2023-07-06T03:46:35.831939+0000","last_fullsized":"2023-07-06T03:46:35.831939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.828872+0000","last_change":"2023-07-06T03:46:35.828872+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.828872+0000","last_undegraded":"2023-07-06T03:46:35.828872+0000","last_fullsized":"2023-07-06T03:46:35.828872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.831923+0000","last_change":"2023-07-06T03:46:35.831923+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.831923+0000","last_undegraded":"2023-07-06T03:46:35.831923+0000","last_fullsized":"2023-07-06T03:46:35.831923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.831865+0000","last_change":"2023-07-06T03:46:35.831865+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.831865+0000","last_undegraded":"2023-07-06T03:46:35.831865+0000","last_fullsized":"2023-07-06T03:46:35.831865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.836043+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:46:35.836043+0000","last_peered":"2023-07-06T03:46:35.836043+0000","last_clean":"2023-07-06T03:46:35.836043+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:46:35.836043+0000","last_undegraded":"2023-07-06T03:46:35.836043+0000","last_fullsized":"2023-07-06T03:46:35.836043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:27:35.968144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.833419+0000","last_change":"2023-07-06T03:46:35.833419+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.833419+0000","last_undegraded":"2023-07-06T03:46:35.833419+0000","last_fullsized":"2023-07-06T03:46:35.833419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.830482+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:46:35.830482+0000","last_peered":"2023-07-06T03:46:35.830482+0000","last_clean":"2023-07-06T03:46:35.830482+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:46:35.830482+0000","last_undegraded":"2023-07-06T03:46:35.830482+0000","last_fullsized":"2023-07-06T03:46:35.830482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:46:36.750598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.831862+0000","last_change":"2023-07-06T03:46:35.831862+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.831862+0000","last_undegraded":"2023-07-06T03:46:35.831862+0000","last_fullsized":"2023-07-06T03:46:35.831862+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.832065+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:46:35.832065+0000","last_peered":"2023-07-06T03:46:35.832065+0000","last_clean":"2023-07-06T03:46:35.832065+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:46:35.832065+0000","last_undegraded":"2023-07-06T03:46:35.832065+0000","last_fullsized":"2023-07-06T03:46:35.832065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:29:58.909470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832615+0000","last_change":"2023-07-06T03:46:35.832615+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832615+0000","last_undegraded":"2023-07-06T03:46:35.832615+0000","last_fullsized":"2023-07-06T03:46:35.832615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.830327+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:46:35.830327+0000","last_peered":"2023-07-06T03:46:35.830327+0000","last_clean":"2023-07-06T03:46:35.830327+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:46:35.830327+0000","last_undegraded":"2023-07-06T03:46:35.830327+0000","last_fullsized":"2023-07-06T03:46:35.830327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:42:18.501755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.827846+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:46:35.827846+0000","last_peered":"2023-07-06T03:46:35.827846+0000","last_clean":"2023-07-06T03:46:35.827846+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:46:35.827846+0000","last_undegraded":"2023-07-06T03:46:35.827846+0000","last_fullsized":"2023-07-06T03:46:35.827846+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:32:19.213621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829275+0000","last_change":"2023-07-06T03:46:35.829275+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829275+0000","last_undegraded":"2023-07-06T03:46:35.829275+0000","last_fullsized":"2023-07-06T03:46:35.829275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.830347+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:46:35.830347+0000","last_peered":"2023-07-06T03:46:35.830347+0000","last_clean":"2023-07-06T03:46:35.830347+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:46:35.830347+0000","last_undegraded":"2023-07-06T03:46:35.830347+0000","last_fullsized":"2023-07-06T03:46:35.830347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:18:17.607630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832209+0000","last_change":"2023-07-06T03:46:35.832209+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832209+0000","last_undegraded":"2023-07-06T03:46:35.832209+0000","last_fullsized":"2023-07-06T03:46:35.832209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.830763+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:46:35.830763+0000","last_peered":"2023-07-06T03:46:35.830763+0000","last_clean":"2023-07-06T03:46:35.830763+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:46:35.830763+0000","last_undegraded":"2023-07-06T03:46:35.830763+0000","last_fullsized":"2023-07-06T03:46:35.830763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:59:14.187429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829651+0000","last_change":"2023-07-06T03:46:35.829651+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829651+0000","last_undegraded":"2023-07-06T03:46:35.829651+0000","last_fullsized":"2023-07-06T03:46:35.829651+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.834544+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:46:35.834544+0000","last_peered":"2023-07-06T03:46:35.834544+0000","last_clean":"2023-07-06T03:46:35.834544+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:46:35.834544+0000","last_undegraded":"2023-07-06T03:46:35.834544+0000","last_fullsized":"2023-07-06T03:46:35.834544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:31:47.232229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832042+0000","last_change":"2023-07-06T03:46:35.832042+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832042+0000","last_undegraded":"2023-07-06T03:46:35.832042+0000","last_fullsized":"2023-07-06T03:46:35.832042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.830671+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:46:35.830671+0000","last_peered":"2023-07-06T03:46:35.830671+0000","last_clean":"2023-07-06T03:46:35.830671+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:46:35.830671+0000","last_undegraded":"2023-07-06T03:46:35.830671+0000","last_fullsized":"2023-07-06T03:46:35.830671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:45:00.398297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":130103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.623},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.8660000000000001,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.77,"5min":3.77,"15min":3.77},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69799999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63900000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.73999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":130103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.79400000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.745}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.72999999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":929792,"data_stored":720471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":13.055999999999999,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.503}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":12.989000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":13.023999999999999,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.874}]},{"osd":6,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.488,"5min":1.488,"15min":1.488},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":12.941000000000001,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.751}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.93000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":130103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":9.5960000000000001,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.54300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":130103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.93899999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.86899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":130103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":5.6550000000000002,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.443}]},{"osd":1,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":15.911,"5min":15.911,"15min":15.911},"last":0.34599999999999997},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":16.024000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":16.068999999999999,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":5.7300000000000004,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.077,"5min":5.077,"15min":5.077},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":5.6980000000000004,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.70299999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":5.157,"5min":5.157,"15min":5.157},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":5.1189999999999998,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.55200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":348160,"data_stored":136273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":6.258,"5min":6.258,"15min":6.258},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":6.4740000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":6.4859999999999998,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.73199999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.747}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.624},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.81399999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79200000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":719452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":6.4059999999999997,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.79100000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.626},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.80800000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.751},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:46:42.235 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:46:42.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.460+0000 7fc15dbff700 1 Processor -- start 2023-07-06T03:46:42.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.460+0000 7fc15dbff700 1 -- start start 2023-07-06T03:46:42.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.461+0000 7fc15dbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc1580cd390 0x7fc158160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.461+0000 7fc15dbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc158160dd0 0x7fc1580b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.462+0000 7fc15dbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1580cdae0 con 0x7fc15815c520 2023-07-06T03:46:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.462+0000 7fc15dbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1580b6640 con 0x7fc158160dd0 2023-07-06T03:46:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.462+0000 7fc15dbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1580b6920 con 0x7fc1580cd390 2023-07-06T03:46:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.462+0000 7fc157fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc158160dd0 0x7fc1580b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.462+0000 7fc157fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc158160dd0 0x7fc1580b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58116/0 (socket says 172.21.15.97:58116) 2023-07-06T03:46:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.462+0000 7fc157fff700 1 -- 172.21.15.97:0/3256867011 learned_addr learned my addr 172.21.15.97:0/3256867011 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.462+0000 7fc156ffd700 1 --2- 172.21.15.97:0/3256867011 >> v2:172.21.15.146:3300/0 conn(0x7fc1580cd390 0x7fc158160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.462+0000 7fc157fff700 1 -- 172.21.15.97:0/3256867011 >> v2:172.21.15.146:3300/0 conn(0x7fc1580cd390 msgr2=0x7fc158160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.462+0000 7fc157fff700 1 --2- 172.21.15.97:0/3256867011 >> v2:172.21.15.146:3300/0 conn(0x7fc1580cd390 0x7fc158160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.462+0000 7fc157fff700 1 -- 172.21.15.97:0/3256867011 >> v1:172.21.15.146:6789/0 conn(0x7fc15815c520 legacy=0x7fc158160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.462+0000 7fc157fff700 1 -- 172.21.15.97:0/3256867011 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1580b6c00 con 0x7fc158160dd0 2023-07-06T03:46:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.463+0000 7fc157fff700 1 --2- 172.21.15.97:0/3256867011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc158160dd0 0x7fc1580b7400 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc1580b5930 tx=0x7fc15000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2b9f4deb224d54 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:42.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.463+0000 7fc1567fc700 1 -- 172.21.15.97:0/3256867011 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15000e070 con 0x7fc158160dd0 2023-07-06T03:46:42.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.463+0000 7fc1567fc700 1 -- 172.21.15.97:0/3256867011 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15000b9c0 con 0x7fc158160dd0 2023-07-06T03:46:42.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.463+0000 7fc1567fc700 1 -- 172.21.15.97:0/3256867011 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15000bd70 con 0x7fc158160dd0 2023-07-06T03:46:42.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.463+0000 7fc15dbff700 1 -- 172.21.15.97:0/3256867011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc158160dd0 msgr2=0x7fc1580b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.463+0000 7fc15dbff700 1 --2- 172.21.15.97:0/3256867011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc158160dd0 0x7fc1580b7400 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc1580b5930 tx=0x7fc15000b330 comp rx=0 tx=0).stop 2023-07-06T03:46:42.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.463+0000 7fc15dbff700 1 -- 172.21.15.97:0/3256867011 shutdown_connections 2023-07-06T03:46:42.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.463+0000 7fc15dbff700 1 --2- 172.21.15.97:0/3256867011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc158160dd0 0x7fc1580b7400 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.463+0000 7fc15dbff700 1 --2- 172.21.15.97:0/3256867011 >> v2:172.21.15.146:3300/0 conn(0x7fc1580cd390 0x7fc158160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.463+0000 7fc15dbff700 1 -- 172.21.15.97:0/3256867011 >> 172.21.15.97:0/3256867011 conn(0x7fc158071790 msgr2=0x7fc158071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:42.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.463+0000 7fc15dbff700 1 -- 172.21.15.97:0/3256867011 shutdown_connections 2023-07-06T03:46:42.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.464+0000 7fc15dbff700 1 -- 172.21.15.97:0/3256867011 wait complete. 2023-07-06T03:46:42.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.466+0000 7fc15dbff700 1 Processor -- start 2023-07-06T03:46:42.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.466+0000 7fc15dbff700 1 -- start start 2023-07-06T03:46:42.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.467+0000 7fc15dbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1580cd390 0x7fc1580cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.467+0000 7fc15dbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc15815c520 0x7fc158158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.467+0000 7fc15dbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc158160dd0 0x7fc158155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.467+0000 7fc15dbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580cb7f0 con 0x7fc15815c520 2023-07-06T03:46:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.467+0000 7fc15dbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580cb590 con 0x7fc1580cd390 2023-07-06T03:46:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.467+0000 7fc15dbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580d2350 con 0x7fc158160dd0 2023-07-06T03:46:42.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.467+0000 7fc156ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc15815c520 0x7fc158158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.467+0000 7fc156ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc15815c520 0x7fc158158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58132/0 (socket says 172.21.15.97:58132) 2023-07-06T03:46:42.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.467+0000 7fc156ffd700 1 -- 172.21.15.97:0/3992064713 learned_addr learned my addr 172.21.15.97:0/3992064713 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:42.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc1577fe700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1580cd390 0x7fc1580cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc157fff700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc158160dd0 0x7fc158155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc156ffd700 1 -- 172.21.15.97:0/3992064713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc158160dd0 msgr2=0x7fc158155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc156ffd700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc158160dd0 0x7fc158155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc156ffd700 1 -- 172.21.15.97:0/3992064713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1580cd390 msgr2=0x7fc1580cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc156ffd700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1580cd390 0x7fc1580cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc156ffd700 1 -- 172.21.15.97:0/3992064713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc150009b30 con 0x7fc15815c520 2023-07-06T03:46:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc1577fe700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1580cd390 0x7fc1580cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc156ffd700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc15815c520 0x7fc158158190 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00d800 tx=0x7fc14c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc13ffff700 1 -- 172.21.15.97:0/3992064713 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0125f0 con 0x7fc15815c520 2023-07-06T03:46:42.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc13ffff700 1 -- 172.21.15.97:0/3992064713 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c012750 con 0x7fc15815c520 2023-07-06T03:46:42.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc13ffff700 1 -- 172.21.15.97:0/3992064713 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c012b00 con 0x7fc15815c520 2023-07-06T03:46:42.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.468+0000 7fc15dbff700 1 -- 172.21.15.97:0/3992064713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1580d2630 con 0x7fc15815c520 2023-07-06T03:46:42.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.469+0000 7fc15dbff700 1 -- 172.21.15.97:0/3992064713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1580d2bf0 con 0x7fc15815c520 2023-07-06T03:46:42.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.470+0000 7fc15dbff700 1 -- 172.21.15.97:0/3992064713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc158160180 con 0x7fc15815c520 2023-07-06T03:46:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.470+0000 7fc13ffff700 1 -- 172.21.15.97:0/3992064713 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc14c012c60 con 0x7fc15815c520 2023-07-06T03:46:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.471+0000 7fc13ffff700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc14406ace0 0x7fc14406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.471+0000 7fc13ffff700 1 -- 172.21.15.97:0/3992064713 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc14c089840 con 0x7fc15815c520 2023-07-06T03:46:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.472+0000 7fc1577fe700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc14406ace0 0x7fc14406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.472+0000 7fc1577fe700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc14406ace0 0x7fc14406d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc148000f60 tx=0x7fc148009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:42.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.475+0000 7fc13ffff700 1 -- 172.21.15.97:0/3992064713 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc14c054980 con 0x7fc15815c520 2023-07-06T03:46:42.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.666+0000 7fc15dbff700 1 -- 172.21.15.97:0/3992064713 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc1580cd800 con 0x7fc14406ace0 2023-07-06T03:46:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.674+0000 7fc13ffff700 1 -- 172.21.15.97:0/3992064713 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299010 (secure 0 0 0) 0x7fc1480014f0 con 0x7fc14406ace0 2023-07-06T03:46:42.679 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:42.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.678+0000 7fc15dbff700 1 -- 172.21.15.97:0/3992064713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc14406ace0 msgr2=0x7fc14406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.678+0000 7fc15dbff700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc14406ace0 0x7fc14406d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc148000f60 tx=0x7fc148009040 comp rx=0 tx=0).stop 2023-07-06T03:46:42.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.678+0000 7fc15dbff700 1 -- 172.21.15.97:0/3992064713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc15815c520 msgr2=0x7fc158158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.678+0000 7fc15dbff700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc15815c520 0x7fc158158190 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00d800 tx=0x7fc14c00dbc0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.678+0000 7fc15dbff700 1 -- 172.21.15.97:0/3992064713 shutdown_connections 2023-07-06T03:46:42.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.678+0000 7fc15dbff700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc14406ace0 0x7fc14406d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.678+0000 7fc15dbff700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc158160dd0 0x7fc158155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.678+0000 7fc15dbff700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1580cd390 0x7fc1580cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.678+0000 7fc15dbff700 1 --2- 172.21.15.97:0/3992064713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc15815c520 0x7fc158158190 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.678+0000 7fc15dbff700 1 -- 172.21.15.97:0/3992064713 >> 172.21.15.97:0/3992064713 conn(0x7fc158071790 msgr2=0x7fc1581605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:42.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.679+0000 7fc15dbff700 1 -- 172.21.15.97:0/3992064713 shutdown_connections 2023-07-06T03:46:42.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.679+0000 7fc15dbff700 1 -- 172.21.15.97:0/3992064713 wait complete. 2023-07-06T03:46:42.683 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:46:42.698 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-07-06T03:46:40.899512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219408,"kb_used_data":3792,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755312,"statfs":{"total":773094113280,"available":772869439488,"internally_reserved":0,"allocated":3883008,"data_stored":2226711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.073321"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.830173+0000","last_change":"2023-07-06T03:46:35.830173+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.830173+0000","last_undegraded":"2023-07-06T03:46:35.830173+0000","last_fullsized":"2023-07-06T03:46:35.830173+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.831990+0000","last_change":"2023-07-06T03:46:35.831990+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.831990+0000","last_undegraded":"2023-07-06T03:46:35.831990+0000","last_fullsized":"2023-07-06T03:46:35.831990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832506+0000","last_change":"2023-07-06T03:46:35.832506+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832506+0000","last_undegraded":"2023-07-06T03:46:35.832506+0000","last_fullsized":"2023-07-06T03:46:35.832506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.828829+0000","last_change":"2023-07-06T03:46:35.828829+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.828829+0000","last_undegraded":"2023-07-06T03:46:35.828829+0000","last_fullsized":"2023-07-06T03:46:35.828829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829789+0000","last_change":"2023-07-06T03:46:35.829789+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829789+0000","last_undegraded":"2023-07-06T03:46:35.829789+0000","last_fullsized":"2023-07-06T03:46:35.829789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.828825+0000","last_change":"2023-07-06T03:46:35.828825+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.828825+0000","last_undegraded":"2023-07-06T03:46:35.828825+0000","last_fullsized":"2023-07-06T03:46:35.828825+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.833925+0000","last_change":"2023-07-06T03:46:35.833925+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.833925+0000","last_undegraded":"2023-07-06T03:46:35.833925+0000","last_fullsized":"2023-07-06T03:46:35.833925+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.828738+0000","last_change":"2023-07-06T03:46:35.828738+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.828738+0000","last_undegraded":"2023-07-06T03:46:35.828738+0000","last_fullsized":"2023-07-06T03:46:35.828738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832906+0000","last_change":"2023-07-06T03:46:35.832906+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832906+0000","last_undegraded":"2023-07-06T03:46:35.832906+0000","last_fullsized":"2023-07-06T03:46:35.832906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829217+0000","last_change":"2023-07-06T03:46:35.829217+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829217+0000","last_undegraded":"2023-07-06T03:46:35.829217+0000","last_fullsized":"2023-07-06T03:46:35.829217+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829845+0000","last_change":"2023-07-06T03:46:35.829845+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829845+0000","last_undegraded":"2023-07-06T03:46:35.829845+0000","last_fullsized":"2023-07-06T03:46:35.829845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.833452+0000","last_change":"2023-07-06T03:46:35.833452+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.833452+0000","last_undegraded":"2023-07-06T03:46:35.833452+0000","last_fullsized":"2023-07-06T03:46:35.833452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832453+0000","last_change":"2023-07-06T03:46:35.832453+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832453+0000","last_undegraded":"2023-07-06T03:46:35.832453+0000","last_fullsized":"2023-07-06T03:46:35.832453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832823+0000","last_change":"2023-07-06T03:46:35.832823+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832823+0000","last_undegraded":"2023-07-06T03:46:35.832823+0000","last_fullsized":"2023-07-06T03:46:35.832823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.830021+0000","last_change":"2023-07-06T03:46:35.830021+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.830021+0000","last_undegraded":"2023-07-06T03:46:35.830021+0000","last_fullsized":"2023-07-06T03:46:35.830021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829310+0000","last_change":"2023-07-06T03:46:35.829310+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829310+0000","last_undegraded":"2023-07-06T03:46:35.829310+0000","last_fullsized":"2023-07-06T03:46:35.829310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829283+0000","last_change":"2023-07-06T03:46:35.829283+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829283+0000","last_undegraded":"2023-07-06T03:46:35.829283+0000","last_fullsized":"2023-07-06T03:46:35.829283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.828778+0000","last_change":"2023-07-06T03:46:35.828778+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.828778+0000","last_undegraded":"2023-07-06T03:46:35.828778+0000","last_fullsized":"2023-07-06T03:46:35.828778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.833883+0000","last_change":"2023-07-06T03:46:35.833883+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.833883+0000","last_undegraded":"2023-07-06T03:46:35.833883+0000","last_fullsized":"2023-07-06T03:46:35.833883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.835339+0000","last_change":"2023-07-06T03:46:35.835339+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.835339+0000","last_undegraded":"2023-07-06T03:46:35.835339+0000","last_fullsized":"2023-07-06T03:46:35.835339+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.835354+0000","last_change":"2023-07-06T03:46:35.835354+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.835354+0000","last_undegraded":"2023-07-06T03:46:35.835354+0000","last_fullsized":"2023-07-06T03:46:35.835354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.831939+0000","last_change":"2023-07-06T03:46:35.831939+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.831939+0000","last_undegraded":"2023-07-06T03:46:35.831939+0000","last_fullsized":"2023-07-06T03:46:35.831939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.828872+0000","last_change":"2023-07-06T03:46:35.828872+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.828872+0000","last_undegraded":"2023-07-06T03:46:35.828872+0000","last_fullsized":"2023-07-06T03:46:35.828872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.831923+0000","last_change":"2023-07-06T03:46:35.831923+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.831923+0000","last_undegraded":"2023-07-06T03:46:35.831923+0000","last_fullsized":"2023-07-06T03:46:35.831923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.831865+0000","last_change":"2023-07-06T03:46:35.831865+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.831865+0000","last_undegraded":"2023-07-06T03:46:35.831865+0000","last_fullsized":"2023-07-06T03:46:35.831865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.836043+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:46:35.836043+0000","last_peered":"2023-07-06T03:46:35.836043+0000","last_clean":"2023-07-06T03:46:35.836043+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:46:35.836043+0000","last_undegraded":"2023-07-06T03:46:35.836043+0000","last_fullsized":"2023-07-06T03:46:35.836043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:27:35.968144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.833419+0000","last_change":"2023-07-06T03:46:35.833419+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.833419+0000","last_undegraded":"2023-07-06T03:46:35.833419+0000","last_fullsized":"2023-07-06T03:46:35.833419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.830482+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:46:35.830482+0000","last_peered":"2023-07-06T03:46:35.830482+0000","last_clean":"2023-07-06T03:46:35.830482+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:46:35.830482+0000","last_undegraded":"2023-07-06T03:46:35.830482+0000","last_fullsized":"2023-07-06T03:46:35.830482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:46:36.750598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.831862+0000","last_change":"2023-07-06T03:46:35.831862+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.831862+0000","last_undegraded":"2023-07-06T03:46:35.831862+0000","last_fullsized":"2023-07-06T03:46:35.831862+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.832065+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:46:35.832065+0000","last_peered":"2023-07-06T03:46:35.832065+0000","last_clean":"2023-07-06T03:46:35.832065+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:46:35.832065+0000","last_undegraded":"2023-07-06T03:46:35.832065+0000","last_fullsized":"2023-07-06T03:46:35.832065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:29:58.909470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832615+0000","last_change":"2023-07-06T03:46:35.832615+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832615+0000","last_undegraded":"2023-07-06T03:46:35.832615+0000","last_fullsized":"2023-07-06T03:46:35.832615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.830327+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:46:35.830327+0000","last_peered":"2023-07-06T03:46:35.830327+0000","last_clean":"2023-07-06T03:46:35.830327+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:46:35.830327+0000","last_undegraded":"2023-07-06T03:46:35.830327+0000","last_fullsized":"2023-07-06T03:46:35.830327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:42:18.501755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.827846+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:46:35.827846+0000","last_peered":"2023-07-06T03:46:35.827846+0000","last_clean":"2023-07-06T03:46:35.827846+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:46:35.827846+0000","last_undegraded":"2023-07-06T03:46:35.827846+0000","last_fullsized":"2023-07-06T03:46:35.827846+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:32:19.213621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829275+0000","last_change":"2023-07-06T03:46:35.829275+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829275+0000","last_undegraded":"2023-07-06T03:46:35.829275+0000","last_fullsized":"2023-07-06T03:46:35.829275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.830347+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:46:35.830347+0000","last_peered":"2023-07-06T03:46:35.830347+0000","last_clean":"2023-07-06T03:46:35.830347+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:46:35.830347+0000","last_undegraded":"2023-07-06T03:46:35.830347+0000","last_fullsized":"2023-07-06T03:46:35.830347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:18:17.607630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832209+0000","last_change":"2023-07-06T03:46:35.832209+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832209+0000","last_undegraded":"2023-07-06T03:46:35.832209+0000","last_fullsized":"2023-07-06T03:46:35.832209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.830763+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:46:35.830763+0000","last_peered":"2023-07-06T03:46:35.830763+0000","last_clean":"2023-07-06T03:46:35.830763+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:46:35.830763+0000","last_undegraded":"2023-07-06T03:46:35.830763+0000","last_fullsized":"2023-07-06T03:46:35.830763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:59:14.187429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.829651+0000","last_change":"2023-07-06T03:46:35.829651+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.829651+0000","last_undegraded":"2023-07-06T03:46:35.829651+0000","last_fullsized":"2023-07-06T03:46:35.829651+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.834544+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:46:35.834544+0000","last_peered":"2023-07-06T03:46:35.834544+0000","last_clean":"2023-07-06T03:46:35.834544+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:46:35.834544+0000","last_undegraded":"2023-07-06T03:46:35.834544+0000","last_fullsized":"2023-07-06T03:46:35.834544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:31:47.232229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T03:46:36.826176+0000","last_change":"2023-07-06T03:46:36.826176+0000","last_active":"2023-07-06T03:46:36.826176+0000","last_peered":"2023-07-06T03:46:36.826176+0000","last_clean":"2023-07-06T03:46:36.826176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:36.826176+0000","last_undegraded":"2023-07-06T03:46:36.826176+0000","last_fullsized":"2023-07-06T03:46:36.826176+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T03:46:35.832042+0000","last_change":"2023-07-06T03:46:35.832042+0000","last_active":"2023-07-06T03:46:35.819445+0000","last_peered":"2023-07-06T03:46:35.819445+0000","last_clean":"2023-07-06T03:46:35.819445+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:46:35.832042+0000","last_undegraded":"2023-07-06T03:46:35.832042+0000","last_fullsized":"2023-07-06T03:46:35.832042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T03:46:35.830671+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:46:35.830671+0000","last_peered":"2023-07-06T03:46:35.830671+0000","last_clean":"2023-07-06T03:46:35.830671+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:46:35.830671+0000","last_undegraded":"2023-07-06T03:46:35.830671+0000","last_fullsized":"2023-07-06T03:46:35.830671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:45:00.398297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":130103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.623},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.8660000000000001,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.77,"5min":3.77,"15min":3.77},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69799999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63900000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.73999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":130103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.79400000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.745}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.72999999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":929792,"data_stored":720471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":13.055999999999999,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.503}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":12.989000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":13.023999999999999,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.874}]},{"osd":6,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.488,"5min":1.488,"15min":1.488},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":12.941000000000001,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.751}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.93000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":130103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":9.5960000000000001,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.54300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":130103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.93899999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.86899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":335872,"data_stored":130103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":5.6550000000000002,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.443}]},{"osd":1,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":15.911,"5min":15.911,"15min":15.911},"last":0.34599999999999997},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":16.024000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":16.068999999999999,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":5.7300000000000004,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.077,"5min":5.077,"15min":5.077},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":5.6980000000000004,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.70299999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":5.157,"5min":5.157,"15min":5.157},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":5.1189999999999998,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.55200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":348160,"data_stored":136273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":6.258,"5min":6.258,"15min":6.258},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":6.4740000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":6.4859999999999998,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.73199999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.747}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.624},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.81399999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79200000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":719452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":6.4059999999999997,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.79100000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.626},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.80800000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.751},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:46:42.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:46:42.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.934+0000 7ff846fff700 1 Processor -- start 2023-07-06T03:46:42.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.934+0000 7ff846fff700 1 -- start start 2023-07-06T03:46:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.935+0000 7ff846fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff84015c520 0x7ff8400cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.935+0000 7ff846fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff840160d00 0x7ff8400b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.935+0000 7ff846fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8400cdae0 con 0x7ff840160180 2023-07-06T03:46:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.935+0000 7ff846fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8400cd8d0 con 0x7ff84015c520 2023-07-06T03:46:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.935+0000 7ff846fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8400b6640 con 0x7ff840160d00 2023-07-06T03:46:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.936+0000 7ff844d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff84015c520 0x7ff8400cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.936+0000 7ff844d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff84015c520 0x7ff8400cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58144/0 (socket says 172.21.15.97:58144) 2023-07-06T03:46:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.936+0000 7ff844d9b700 1 -- 172.21.15.97:0/43492025 learned_addr learned my addr 172.21.15.97:0/43492025 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.936+0000 7ff84559c700 1 --2- 172.21.15.97:0/43492025 >> v2:172.21.15.146:3300/0 conn(0x7ff840160d00 0x7ff8400b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.936+0000 7ff844d9b700 1 -- 172.21.15.97:0/43492025 >> v2:172.21.15.146:3300/0 conn(0x7ff840160d00 msgr2=0x7ff8400b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.936+0000 7ff844d9b700 1 --2- 172.21.15.97:0/43492025 >> v2:172.21.15.146:3300/0 conn(0x7ff840160d00 0x7ff8400b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.936+0000 7ff844d9b700 1 -- 172.21.15.97:0/43492025 >> v1:172.21.15.97:6789/0 conn(0x7ff840160180 legacy=0x7ff8401605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.936+0000 7ff844d9b700 1 -- 172.21.15.97:0/43492025 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8400b6920 con 0x7ff84015c520 2023-07-06T03:46:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.936+0000 7ff844d9b700 1 --2- 172.21.15.97:0/43492025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff84015c520 0x7ff8400cd390 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff8400cd320 tx=0x7ff834009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576daa9a6401c070 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:42.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.937+0000 7ff83f7fe700 1 -- 172.21.15.97:0/43492025 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83400b870 con 0x7ff84015c520 2023-07-06T03:46:42.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.937+0000 7ff83f7fe700 1 -- 172.21.15.97:0/43492025 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83400b9d0 con 0x7ff84015c520 2023-07-06T03:46:42.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.937+0000 7ff83f7fe700 1 -- 172.21.15.97:0/43492025 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83400bd60 con 0x7ff84015c520 2023-07-06T03:46:42.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.937+0000 7ff846fff700 1 -- 172.21.15.97:0/43492025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff84015c520 msgr2=0x7ff8400cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.937+0000 7ff846fff700 1 --2- 172.21.15.97:0/43492025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff84015c520 0x7ff8400cd390 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff8400cd320 tx=0x7ff834009bd0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.937+0000 7ff846fff700 1 -- 172.21.15.97:0/43492025 shutdown_connections 2023-07-06T03:46:42.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.937+0000 7ff846fff700 1 --2- 172.21.15.97:0/43492025 >> v2:172.21.15.146:3300/0 conn(0x7ff840160d00 0x7ff8400b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.937+0000 7ff846fff700 1 --2- 172.21.15.97:0/43492025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff84015c520 0x7ff8400cd390 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.937+0000 7ff846fff700 1 -- 172.21.15.97:0/43492025 >> 172.21.15.97:0/43492025 conn(0x7ff840071790 msgr2=0x7ff840071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:42.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.937+0000 7ff846fff700 1 -- 172.21.15.97:0/43492025 shutdown_connections 2023-07-06T03:46:42.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.937+0000 7ff846fff700 1 -- 172.21.15.97:0/43492025 wait complete. 2023-07-06T03:46:42.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.939+0000 7ff846fff700 1 Processor -- start 2023-07-06T03:46:42.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.939+0000 7ff846fff700 1 -- start start 2023-07-06T03:46:42.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.940+0000 7ff846fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84015c520 0x7ff84014ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.940+0000 7ff846fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840160180 0x7ff8401500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.940+0000 7ff846fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff840160d00 0x7ff84015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.940+0000 7ff846fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84015dd60 con 0x7ff840160180 2023-07-06T03:46:42.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.940+0000 7ff846fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84014d3f0 con 0x7ff84015c520 2023-07-06T03:46:42.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.940+0000 7ff846fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8401e5fc0 con 0x7ff840160d00 2023-07-06T03:46:42.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.940+0000 7ff83ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840160180 0x7ff8401500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.940+0000 7ff83ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840160180 0x7ff8401500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58158/0 (socket says 172.21.15.97:58158) 2023-07-06T03:46:42.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.940+0000 7ff83ffff700 1 -- 172.21.15.97:0/1685802219 learned_addr learned my addr 172.21.15.97:0/1685802219 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:42.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.940+0000 7ff84559c700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff840160d00 0x7ff84015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.940+0000 7ff844d9b700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84015c520 0x7ff84014ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff83ffff700 1 -- 172.21.15.97:0/1685802219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff840160d00 msgr2=0x7ff84015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff83ffff700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff840160d00 0x7ff84015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff83ffff700 1 -- 172.21.15.97:0/1685802219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84015c520 msgr2=0x7ff84014ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:42.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff83ffff700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84015c520 0x7ff84014ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:42.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff83ffff700 1 -- 172.21.15.97:0/1685802219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff834009550 con 0x7ff840160180 2023-07-06T03:46:42.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff84559c700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff840160d00 0x7ff84015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:42.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff844d9b700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84015c520 0x7ff84014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:42.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff83ffff700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840160180 0x7ff8401500c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00c510 tx=0x7ff82c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:42.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff83d7fa700 1 -- 172.21.15.97:0/1685802219 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0052d0 con 0x7ff840160180 2023-07-06T03:46:42.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff83d7fa700 1 -- 172.21.15.97:0/1685802219 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c005430 con 0x7ff840160180 2023-07-06T03:46:42.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff846fff700 1 -- 172.21.15.97:0/1685802219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8401e62a0 con 0x7ff840160180 2023-07-06T03:46:42.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff83d7fa700 1 -- 172.21.15.97:0/1685802219 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0038c0 con 0x7ff840160180 2023-07-06T03:46:42.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.941+0000 7ff846fff700 1 -- 172.21.15.97:0/1685802219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84014cb40 con 0x7ff840160180 2023-07-06T03:46:42.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.942+0000 7ff846fff700 1 -- 172.21.15.97:0/1685802219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84015ded0 con 0x7ff840160180 2023-07-06T03:46:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.943+0000 7ff83d7fa700 1 -- 172.21.15.97:0/1685802219 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff82c003a20 con 0x7ff840160180 2023-07-06T03:46:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.944+0000 7ff83d7fa700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82406ace0 0x7ff82406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.944+0000 7ff83d7fa700 1 -- 172.21.15.97:0/1685802219 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff82c0889b0 con 0x7ff840160180 2023-07-06T03:46:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.945+0000 7ff844d9b700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82406ace0 0x7ff82406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.945+0000 7ff844d9b700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82406ace0 0x7ff82406d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff834014040 tx=0x7ff834008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:42.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:42.948+0000 7ff83d7fa700 1 -- 172.21.15.97:0/1685802219 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff84015ded0 con 0x7ff840160180 2023-07-06T03:46:43.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.141+0000 7ff846fff700 1 -- 172.21.15.97:0/1685802219 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8401e6130 con 0x7ff82406ace0 2023-07-06T03:46:43.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.150+0000 7ff83d7fa700 1 -- 172.21.15.97:0/1685802219 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310874 (secure 0 0 0) 0x7ff8401e6130 con 0x7ff82406ace0 2023-07-06T03:46:43.155 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:43.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 -- 172.21.15.97:0/1685802219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82406ace0 msgr2=0x7ff82406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82406ace0 0x7ff82406d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff834014040 tx=0x7ff834008030 comp rx=0 tx=0).stop 2023-07-06T03:46:43.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 -- 172.21.15.97:0/1685802219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840160180 msgr2=0x7ff8401500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840160180 0x7ff8401500c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00c510 tx=0x7ff82c00c8d0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 -- 172.21.15.97:0/1685802219 shutdown_connections 2023-07-06T03:46:43.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff82406ace0 0x7ff82406d190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff840160d00 0x7ff84015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff84015c520 0x7ff84014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 --2- 172.21.15.97:0/1685802219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840160180 0x7ff8401500c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 -- 172.21.15.97:0/1685802219 >> 172.21.15.97:0/1685802219 conn(0x7ff840071790 msgr2=0x7ff840156bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:43.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 -- 172.21.15.97:0/1685802219 shutdown_connections 2023-07-06T03:46:43.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.155+0000 7ff846fff700 1 -- 172.21.15.97:0/1685802219 wait complete. 2023-07-06T03:46:43.159 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:46:43.174 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-07-06T03:46:42.899899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219844,"kb_used_data":4228,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754876,"statfs":{"total":773094113280,"available":772868993024,"internally_reserved":0,"allocated":4329472,"data_stored":2437897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.064157"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419924+0000","last_change":"2023-07-06T03:46:37.856906+0000","last_active":"2023-07-06T03:46:39.419924+0000","last_peered":"2023-07-06T03:46:39.419924+0000","last_clean":"2023-07-06T03:46:39.419924+0000","last_became_active":"2023-07-06T03:46:37.856640+0000","last_became_peered":"2023-07-06T03:46:37.856640+0000","last_unstale":"2023-07-06T03:46:39.419924+0000","last_undegraded":"2023-07-06T03:46:39.419924+0000","last_fullsized":"2023-07-06T03:46:39.419924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:00:51.494589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419658+0000","last_change":"2023-07-06T03:46:37.857685+0000","last_active":"2023-07-06T03:46:39.419658+0000","last_peered":"2023-07-06T03:46:39.419658+0000","last_clean":"2023-07-06T03:46:39.419658+0000","last_became_active":"2023-07-06T03:46:37.857373+0000","last_became_peered":"2023-07-06T03:46:37.857373+0000","last_unstale":"2023-07-06T03:46:39.419658+0000","last_undegraded":"2023-07-06T03:46:39.419658+0000","last_fullsized":"2023-07-06T03:46:39.419658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:16.795837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420131+0000","last_change":"2023-07-06T03:46:37.862846+0000","last_active":"2023-07-06T03:46:39.420131+0000","last_peered":"2023-07-06T03:46:39.420131+0000","last_clean":"2023-07-06T03:46:39.420131+0000","last_became_active":"2023-07-06T03:46:37.862554+0000","last_became_peered":"2023-07-06T03:46:37.862554+0000","last_unstale":"2023-07-06T03:46:39.420131+0000","last_undegraded":"2023-07-06T03:46:39.420131+0000","last_fullsized":"2023-07-06T03:46:39.420131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:08:02.995580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420078+0000","last_change":"2023-07-06T03:46:37.866028+0000","last_active":"2023-07-06T03:46:39.420078+0000","last_peered":"2023-07-06T03:46:39.420078+0000","last_clean":"2023-07-06T03:46:39.420078+0000","last_became_active":"2023-07-06T03:46:37.865346+0000","last_became_peered":"2023-07-06T03:46:37.865346+0000","last_unstale":"2023-07-06T03:46:39.420078+0000","last_undegraded":"2023-07-06T03:46:39.420078+0000","last_fullsized":"2023-07-06T03:46:39.420078+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:59:06.270128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853587+0000","last_change":"2023-07-06T03:46:37.864390+0000","last_active":"2023-07-06T03:46:38.853587+0000","last_peered":"2023-07-06T03:46:38.853587+0000","last_clean":"2023-07-06T03:46:38.853587+0000","last_became_active":"2023-07-06T03:46:37.864064+0000","last_became_peered":"2023-07-06T03:46:37.864064+0000","last_unstale":"2023-07-06T03:46:38.853587+0000","last_undegraded":"2023-07-06T03:46:38.853587+0000","last_fullsized":"2023-07-06T03:46:38.853587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:16.795837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419343+0000","last_change":"2023-07-06T03:46:37.861836+0000","last_active":"2023-07-06T03:46:39.419343+0000","last_peered":"2023-07-06T03:46:39.419343+0000","last_clean":"2023-07-06T03:46:39.419343+0000","last_became_active":"2023-07-06T03:46:37.861568+0000","last_became_peered":"2023-07-06T03:46:37.861568+0000","last_unstale":"2023-07-06T03:46:39.419343+0000","last_undegraded":"2023-07-06T03:46:39.419343+0000","last_fullsized":"2023-07-06T03:46:39.419343+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.598122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418487+0000","last_change":"2023-07-06T03:46:37.844697+0000","last_active":"2023-07-06T03:46:39.418487+0000","last_peered":"2023-07-06T03:46:39.418487+0000","last_clean":"2023-07-06T03:46:39.418487+0000","last_became_active":"2023-07-06T03:46:37.844412+0000","last_became_peered":"2023-07-06T03:46:37.844412+0000","last_unstale":"2023-07-06T03:46:39.418487+0000","last_undegraded":"2023-07-06T03:46:39.418487+0000","last_fullsized":"2023-07-06T03:46:39.418487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:40:38.338794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419209+0000","last_change":"2023-07-06T03:46:37.851682+0000","last_active":"2023-07-06T03:46:39.419209+0000","last_peered":"2023-07-06T03:46:39.419209+0000","last_clean":"2023-07-06T03:46:39.419209+0000","last_became_active":"2023-07-06T03:46:37.851342+0000","last_became_peered":"2023-07-06T03:46:37.851342+0000","last_unstale":"2023-07-06T03:46:39.419209+0000","last_undegraded":"2023-07-06T03:46:39.419209+0000","last_fullsized":"2023-07-06T03:46:39.419209+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:31:39.275779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.854169+0000","last_change":"2023-07-06T03:46:37.860538+0000","last_active":"2023-07-06T03:46:38.854169+0000","last_peered":"2023-07-06T03:46:38.854169+0000","last_clean":"2023-07-06T03:46:38.854169+0000","last_became_active":"2023-07-06T03:46:37.860253+0000","last_became_peered":"2023-07-06T03:46:37.860253+0000","last_unstale":"2023-07-06T03:46:38.854169+0000","last_undegraded":"2023-07-06T03:46:38.854169+0000","last_fullsized":"2023-07-06T03:46:38.854169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:40:38.338794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418842+0000","last_change":"2023-07-06T03:46:37.861893+0000","last_active":"2023-07-06T03:46:39.418842+0000","last_peered":"2023-07-06T03:46:39.418842+0000","last_clean":"2023-07-06T03:46:39.418842+0000","last_became_active":"2023-07-06T03:46:37.861562+0000","last_became_peered":"2023-07-06T03:46:37.861562+0000","last_unstale":"2023-07-06T03:46:39.418842+0000","last_undegraded":"2023-07-06T03:46:39.418842+0000","last_fullsized":"2023-07-06T03:46:39.418842+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:10:32.090820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.042093+0000","last_change":"2023-07-06T03:46:37.869263+0000","last_active":"2023-07-06T03:46:39.042093+0000","last_peered":"2023-07-06T03:46:39.042093+0000","last_clean":"2023-07-06T03:46:39.042093+0000","last_became_active":"2023-07-06T03:46:37.868621+0000","last_became_peered":"2023-07-06T03:46:37.868621+0000","last_unstale":"2023-07-06T03:46:39.042093+0000","last_undegraded":"2023-07-06T03:46:39.042093+0000","last_fullsized":"2023-07-06T03:46:39.042093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:02:10.182517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853685+0000","last_change":"2023-07-06T03:46:37.858638+0000","last_active":"2023-07-06T03:46:38.853685+0000","last_peered":"2023-07-06T03:46:38.853685+0000","last_clean":"2023-07-06T03:46:38.853685+0000","last_became_active":"2023-07-06T03:46:37.858354+0000","last_became_peered":"2023-07-06T03:46:37.858354+0000","last_unstale":"2023-07-06T03:46:38.853685+0000","last_undegraded":"2023-07-06T03:46:38.853685+0000","last_fullsized":"2023-07-06T03:46:38.853685+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:53.824359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418613+0000","last_change":"2023-07-06T03:46:37.864649+0000","last_active":"2023-07-06T03:46:39.418613+0000","last_peered":"2023-07-06T03:46:39.418613+0000","last_clean":"2023-07-06T03:46:39.418613+0000","last_became_active":"2023-07-06T03:46:37.864204+0000","last_became_peered":"2023-07-06T03:46:37.864204+0000","last_unstale":"2023-07-06T03:46:39.418613+0000","last_undegraded":"2023-07-06T03:46:39.418613+0000","last_fullsized":"2023-07-06T03:46:39.418613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:29:38.817047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852484+0000","last_change":"2023-07-06T03:46:37.862356+0000","last_active":"2023-07-06T03:46:38.852484+0000","last_peered":"2023-07-06T03:46:38.852484+0000","last_clean":"2023-07-06T03:46:38.852484+0000","last_became_active":"2023-07-06T03:46:37.862123+0000","last_became_peered":"2023-07-06T03:46:37.862123+0000","last_unstale":"2023-07-06T03:46:38.852484+0000","last_undegraded":"2023-07-06T03:46:38.852484+0000","last_fullsized":"2023-07-06T03:46:38.852484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:37:12.375776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419159+0000","last_change":"2023-07-06T03:46:37.859689+0000","last_active":"2023-07-06T03:46:39.419159+0000","last_peered":"2023-07-06T03:46:39.419159+0000","last_clean":"2023-07-06T03:46:39.419159+0000","last_became_active":"2023-07-06T03:46:37.859356+0000","last_became_peered":"2023-07-06T03:46:37.859356+0000","last_unstale":"2023-07-06T03:46:39.419159+0000","last_undegraded":"2023-07-06T03:46:39.419159+0000","last_fullsized":"2023-07-06T03:46:39.419159+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:44.223992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418293+0000","last_change":"2023-07-06T03:46:37.860022+0000","last_active":"2023-07-06T03:46:39.418293+0000","last_peered":"2023-07-06T03:46:39.418293+0000","last_clean":"2023-07-06T03:46:39.418293+0000","last_became_active":"2023-07-06T03:46:37.859715+0000","last_became_peered":"2023-07-06T03:46:37.859715+0000","last_unstale":"2023-07-06T03:46:39.418293+0000","last_undegraded":"2023-07-06T03:46:39.418293+0000","last_fullsized":"2023-07-06T03:46:39.418293+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:44:39.103954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419760+0000","last_change":"2023-07-06T03:46:37.861889+0000","last_active":"2023-07-06T03:46:39.419760+0000","last_peered":"2023-07-06T03:46:39.419760+0000","last_clean":"2023-07-06T03:46:39.419760+0000","last_became_active":"2023-07-06T03:46:37.860756+0000","last_became_peered":"2023-07-06T03:46:37.860756+0000","last_unstale":"2023-07-06T03:46:39.419760+0000","last_undegraded":"2023-07-06T03:46:39.419760+0000","last_fullsized":"2023-07-06T03:46:39.419760+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:50:46.498928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419563+0000","last_change":"2023-07-06T03:46:37.860737+0000","last_active":"2023-07-06T03:46:39.419563+0000","last_peered":"2023-07-06T03:46:39.419563+0000","last_clean":"2023-07-06T03:46:39.419563+0000","last_became_active":"2023-07-06T03:46:37.860452+0000","last_became_peered":"2023-07-06T03:46:37.860452+0000","last_unstale":"2023-07-06T03:46:39.419563+0000","last_undegraded":"2023-07-06T03:46:39.419563+0000","last_fullsized":"2023-07-06T03:46:39.419563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:50.812126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851546+0000","last_change":"2023-07-06T03:46:37.864336+0000","last_active":"2023-07-06T03:46:38.851546+0000","last_peered":"2023-07-06T03:46:38.851546+0000","last_clean":"2023-07-06T03:46:38.851546+0000","last_became_active":"2023-07-06T03:46:37.863953+0000","last_became_peered":"2023-07-06T03:46:37.863953+0000","last_unstale":"2023-07-06T03:46:38.851546+0000","last_undegraded":"2023-07-06T03:46:38.851546+0000","last_fullsized":"2023-07-06T03:46:38.851546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:28.578365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041710+0000","last_change":"2023-07-06T03:46:37.862894+0000","last_active":"2023-07-06T03:46:39.041710+0000","last_peered":"2023-07-06T03:46:39.041710+0000","last_clean":"2023-07-06T03:46:39.041710+0000","last_became_active":"2023-07-06T03:46:37.862643+0000","last_became_peered":"2023-07-06T03:46:37.862643+0000","last_unstale":"2023-07-06T03:46:39.041710+0000","last_undegraded":"2023-07-06T03:46:39.041710+0000","last_fullsized":"2023-07-06T03:46:39.041710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:25.306376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041686+0000","last_change":"2023-07-06T03:46:37.863993+0000","last_active":"2023-07-06T03:46:39.041686+0000","last_peered":"2023-07-06T03:46:39.041686+0000","last_clean":"2023-07-06T03:46:39.041686+0000","last_became_active":"2023-07-06T03:46:37.863618+0000","last_became_peered":"2023-07-06T03:46:37.863618+0000","last_unstale":"2023-07-06T03:46:39.041686+0000","last_undegraded":"2023-07-06T03:46:39.041686+0000","last_fullsized":"2023-07-06T03:46:39.041686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:26.836331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041669+0000","last_change":"2023-07-06T03:46:37.864654+0000","last_active":"2023-07-06T03:46:39.041669+0000","last_peered":"2023-07-06T03:46:39.041669+0000","last_clean":"2023-07-06T03:46:39.041669+0000","last_became_active":"2023-07-06T03:46:37.864340+0000","last_became_peered":"2023-07-06T03:46:37.864340+0000","last_unstale":"2023-07-06T03:46:39.041669+0000","last_undegraded":"2023-07-06T03:46:39.041669+0000","last_fullsized":"2023-07-06T03:46:39.041669+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:28.578365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419184+0000","last_change":"2023-07-06T03:46:37.853945+0000","last_active":"2023-07-06T03:46:39.419184+0000","last_peered":"2023-07-06T03:46:39.419184+0000","last_clean":"2023-07-06T03:46:39.419184+0000","last_became_active":"2023-07-06T03:46:37.853672+0000","last_became_peered":"2023-07-06T03:46:37.853672+0000","last_unstale":"2023-07-06T03:46:39.419184+0000","last_undegraded":"2023-07-06T03:46:39.419184+0000","last_fullsized":"2023-07-06T03:46:39.419184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:46:28.663423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419858+0000","last_change":"2023-07-06T03:46:37.852498+0000","last_active":"2023-07-06T03:46:39.419858+0000","last_peered":"2023-07-06T03:46:39.419858+0000","last_clean":"2023-07-06T03:46:39.419858+0000","last_became_active":"2023-07-06T03:46:37.852155+0000","last_became_peered":"2023-07-06T03:46:37.852155+0000","last_unstale":"2023-07-06T03:46:39.419858+0000","last_undegraded":"2023-07-06T03:46:39.419858+0000","last_fullsized":"2023-07-06T03:46:39.419858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:40:57.846988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852750+0000","last_change":"2023-07-06T03:46:37.863217+0000","last_active":"2023-07-06T03:46:38.852750+0000","last_peered":"2023-07-06T03:46:38.852750+0000","last_clean":"2023-07-06T03:46:38.852750+0000","last_became_active":"2023-07-06T03:46:37.862977+0000","last_became_peered":"2023-07-06T03:46:37.862977+0000","last_unstale":"2023-07-06T03:46:38.852750+0000","last_undegraded":"2023-07-06T03:46:38.852750+0000","last_fullsized":"2023-07-06T03:46:38.852750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:16:10.345229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420251+0000","last_change":"2023-07-06T03:46:37.853812+0000","last_active":"2023-07-06T03:46:39.420251+0000","last_peered":"2023-07-06T03:46:39.420251+0000","last_clean":"2023-07-06T03:46:39.420251+0000","last_became_active":"2023-07-06T03:46:37.852250+0000","last_became_peered":"2023-07-06T03:46:37.852250+0000","last_unstale":"2023-07-06T03:46:39.420251+0000","last_undegraded":"2023-07-06T03:46:39.420251+0000","last_fullsized":"2023-07-06T03:46:39.420251+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:11:36.909511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.042092+0000","last_change":"2023-07-06T03:46:37.861070+0000","last_active":"2023-07-06T03:46:39.042092+0000","last_peered":"2023-07-06T03:46:39.042092+0000","last_clean":"2023-07-06T03:46:39.042092+0000","last_became_active":"2023-07-06T03:46:37.860738+0000","last_became_peered":"2023-07-06T03:46:37.860738+0000","last_unstale":"2023-07-06T03:46:39.042092+0000","last_undegraded":"2023-07-06T03:46:39.042092+0000","last_fullsized":"2023-07-06T03:46:39.042092+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:53:51.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418678+0000","last_change":"2023-07-06T03:46:37.848284+0000","last_active":"2023-07-06T03:46:39.418678+0000","last_peered":"2023-07-06T03:46:39.418678+0000","last_clean":"2023-07-06T03:46:39.418678+0000","last_became_active":"2023-07-06T03:46:37.848010+0000","last_became_peered":"2023-07-06T03:46:37.848010+0000","last_unstale":"2023-07-06T03:46:39.418678+0000","last_undegraded":"2023-07-06T03:46:39.418678+0000","last_fullsized":"2023-07-06T03:46:39.418678+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:22:19.485193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419230+0000","last_change":"2023-07-06T03:46:37.856579+0000","last_active":"2023-07-06T03:46:39.419230+0000","last_peered":"2023-07-06T03:46:39.419230+0000","last_clean":"2023-07-06T03:46:39.419230+0000","last_became_active":"2023-07-06T03:46:37.856331+0000","last_became_peered":"2023-07-06T03:46:37.856331+0000","last_unstale":"2023-07-06T03:46:39.419230+0000","last_undegraded":"2023-07-06T03:46:39.419230+0000","last_fullsized":"2023-07-06T03:46:39.419230+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:12:57.228030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419212+0000","last_change":"2023-07-06T03:46:37.858786+0000","last_active":"2023-07-06T03:46:39.419212+0000","last_peered":"2023-07-06T03:46:39.419212+0000","last_clean":"2023-07-06T03:46:39.419212+0000","last_became_active":"2023-07-06T03:46:37.858458+0000","last_became_peered":"2023-07-06T03:46:37.858458+0000","last_unstale":"2023-07-06T03:46:39.419212+0000","last_undegraded":"2023-07-06T03:46:39.419212+0000","last_fullsized":"2023-07-06T03:46:39.419212+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:31:04.637693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418659+0000","last_change":"2023-07-06T03:46:37.864684+0000","last_active":"2023-07-06T03:46:39.418659+0000","last_peered":"2023-07-06T03:46:39.418659+0000","last_clean":"2023-07-06T03:46:39.418659+0000","last_became_active":"2023-07-06T03:46:37.864342+0000","last_became_peered":"2023-07-06T03:46:37.864342+0000","last_unstale":"2023-07-06T03:46:39.418659+0000","last_undegraded":"2023-07-06T03:46:39.418659+0000","last_fullsized":"2023-07-06T03:46:39.418659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:12:55.269776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419718+0000","last_change":"2023-07-06T03:46:37.851130+0000","last_active":"2023-07-06T03:46:39.419718+0000","last_peered":"2023-07-06T03:46:39.419718+0000","last_clean":"2023-07-06T03:46:39.419718+0000","last_became_active":"2023-07-06T03:46:37.850783+0000","last_became_peered":"2023-07-06T03:46:37.850783+0000","last_unstale":"2023-07-06T03:46:39.419718+0000","last_undegraded":"2023-07-06T03:46:39.419718+0000","last_fullsized":"2023-07-06T03:46:39.419718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:00:21.691539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419611+0000","last_change":"2023-07-06T03:46:37.853339+0000","last_active":"2023-07-06T03:46:39.419611+0000","last_peered":"2023-07-06T03:46:39.419611+0000","last_clean":"2023-07-06T03:46:39.419611+0000","last_became_active":"2023-07-06T03:46:37.852923+0000","last_became_peered":"2023-07-06T03:46:37.852923+0000","last_unstale":"2023-07-06T03:46:39.419611+0000","last_undegraded":"2023-07-06T03:46:39.419611+0000","last_fullsized":"2023-07-06T03:46:39.419611+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:04.156561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852627+0000","last_change":"2023-07-06T03:46:36.854615+0000","last_active":"2023-07-06T03:46:38.852627+0000","last_peered":"2023-07-06T03:46:38.852627+0000","last_clean":"2023-07-06T03:46:38.852627+0000","last_became_active":"2023-07-06T03:46:36.854325+0000","last_became_peered":"2023-07-06T03:46:36.854325+0000","last_unstale":"2023-07-06T03:46:38.852627+0000","last_undegraded":"2023-07-06T03:46:38.852627+0000","last_fullsized":"2023-07-06T03:46:38.852627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:18:30.511652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420053+0000","last_change":"2023-07-06T03:46:37.865092+0000","last_active":"2023-07-06T03:46:39.420053+0000","last_peered":"2023-07-06T03:46:39.420053+0000","last_clean":"2023-07-06T03:46:39.420053+0000","last_became_active":"2023-07-06T03:46:37.864810+0000","last_became_peered":"2023-07-06T03:46:37.864810+0000","last_unstale":"2023-07-06T03:46:39.420053+0000","last_undegraded":"2023-07-06T03:46:39.420053+0000","last_fullsized":"2023-07-06T03:46:39.420053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:02:10.182517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419775+0000","last_change":"2023-07-06T03:46:36.857512+0000","last_active":"2023-07-06T03:46:39.419775+0000","last_peered":"2023-07-06T03:46:39.419775+0000","last_clean":"2023-07-06T03:46:39.419775+0000","last_became_active":"2023-07-06T03:46:36.857177+0000","last_became_peered":"2023-07-06T03:46:36.857177+0000","last_unstale":"2023-07-06T03:46:39.419775+0000","last_undegraded":"2023-07-06T03:46:39.419775+0000","last_fullsized":"2023-07-06T03:46:39.419775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:33:18.938068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419718+0000","last_change":"2023-07-06T03:46:37.854234+0000","last_active":"2023-07-06T03:46:39.419718+0000","last_peered":"2023-07-06T03:46:39.419718+0000","last_clean":"2023-07-06T03:46:39.419718+0000","last_became_active":"2023-07-06T03:46:37.853973+0000","last_became_peered":"2023-07-06T03:46:37.853973+0000","last_unstale":"2023-07-06T03:46:39.419718+0000","last_undegraded":"2023-07-06T03:46:39.419718+0000","last_fullsized":"2023-07-06T03:46:39.419718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:17:49.210528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420214+0000","last_change":"2023-07-06T03:46:36.860082+0000","last_active":"2023-07-06T03:46:39.420214+0000","last_peered":"2023-07-06T03:46:39.420214+0000","last_clean":"2023-07-06T03:46:39.420214+0000","last_became_active":"2023-07-06T03:46:36.859761+0000","last_became_peered":"2023-07-06T03:46:36.859761+0000","last_unstale":"2023-07-06T03:46:39.420214+0000","last_undegraded":"2023-07-06T03:46:39.420214+0000","last_fullsized":"2023-07-06T03:46:39.420214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:06:29.939063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419942+0000","last_change":"2023-07-06T03:46:37.861372+0000","last_active":"2023-07-06T03:46:39.419942+0000","last_peered":"2023-07-06T03:46:39.419942+0000","last_clean":"2023-07-06T03:46:39.419942+0000","last_became_active":"2023-07-06T03:46:37.861062+0000","last_became_peered":"2023-07-06T03:46:37.861062+0000","last_unstale":"2023-07-06T03:46:39.419942+0000","last_undegraded":"2023-07-06T03:46:39.419942+0000","last_fullsized":"2023-07-06T03:46:39.419942+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:53:51.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852628+0000","last_change":"2023-07-06T03:46:36.860395+0000","last_active":"2023-07-06T03:46:38.852628+0000","last_peered":"2023-07-06T03:46:38.852628+0000","last_clean":"2023-07-06T03:46:38.852628+0000","last_became_active":"2023-07-06T03:46:36.860128+0000","last_became_peered":"2023-07-06T03:46:36.860128+0000","last_unstale":"2023-07-06T03:46:38.852628+0000","last_undegraded":"2023-07-06T03:46:38.852628+0000","last_fullsized":"2023-07-06T03:46:38.852628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:24:21.195033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419703+0000","last_change":"2023-07-06T03:46:37.856482+0000","last_active":"2023-07-06T03:46:39.419703+0000","last_peered":"2023-07-06T03:46:39.419703+0000","last_clean":"2023-07-06T03:46:39.419703+0000","last_became_active":"2023-07-06T03:46:37.856161+0000","last_became_peered":"2023-07-06T03:46:37.856161+0000","last_unstale":"2023-07-06T03:46:39.419703+0000","last_undegraded":"2023-07-06T03:46:39.419703+0000","last_fullsized":"2023-07-06T03:46:39.419703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:36:54.027820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852417+0000","last_change":"2023-07-06T03:46:36.844040+0000","last_active":"2023-07-06T03:46:38.852417+0000","last_peered":"2023-07-06T03:46:38.852417+0000","last_clean":"2023-07-06T03:46:38.852417+0000","last_became_active":"2023-07-06T03:46:36.843778+0000","last_became_peered":"2023-07-06T03:46:36.843778+0000","last_unstale":"2023-07-06T03:46:38.852417+0000","last_undegraded":"2023-07-06T03:46:38.852417+0000","last_fullsized":"2023-07-06T03:46:38.852417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:33:57.954583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041627+0000","last_change":"2023-07-06T03:46:37.858452+0000","last_active":"2023-07-06T03:46:39.041627+0000","last_peered":"2023-07-06T03:46:39.041627+0000","last_clean":"2023-07-06T03:46:39.041627+0000","last_became_active":"2023-07-06T03:46:37.858175+0000","last_became_peered":"2023-07-06T03:46:37.858175+0000","last_unstale":"2023-07-06T03:46:39.041627+0000","last_undegraded":"2023-07-06T03:46:39.041627+0000","last_fullsized":"2023-07-06T03:46:39.041627+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:15.115431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418480+0000","last_change":"2023-07-06T03:46:36.860493+0000","last_active":"2023-07-06T03:46:39.418480+0000","last_peered":"2023-07-06T03:46:39.418480+0000","last_clean":"2023-07-06T03:46:39.418480+0000","last_became_active":"2023-07-06T03:46:36.860183+0000","last_became_peered":"2023-07-06T03:46:36.860183+0000","last_unstale":"2023-07-06T03:46:39.418480+0000","last_undegraded":"2023-07-06T03:46:39.418480+0000","last_fullsized":"2023-07-06T03:46:39.418480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:25:32.127906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419308+0000","last_change":"2023-07-06T03:46:37.863671+0000","last_active":"2023-07-06T03:46:39.419308+0000","last_peered":"2023-07-06T03:46:39.419308+0000","last_clean":"2023-07-06T03:46:39.419308+0000","last_became_active":"2023-07-06T03:46:37.863398+0000","last_became_peered":"2023-07-06T03:46:37.863398+0000","last_unstale":"2023-07-06T03:46:39.419308+0000","last_undegraded":"2023-07-06T03:46:39.419308+0000","last_fullsized":"2023-07-06T03:46:39.419308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:17:26.517544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419441+0000","last_change":"2023-07-06T03:46:36.858921+0000","last_active":"2023-07-06T03:46:39.419441+0000","last_peered":"2023-07-06T03:46:39.419441+0000","last_clean":"2023-07-06T03:46:39.419441+0000","last_became_active":"2023-07-06T03:46:36.858622+0000","last_became_peered":"2023-07-06T03:46:36.858622+0000","last_unstale":"2023-07-06T03:46:39.419441+0000","last_undegraded":"2023-07-06T03:46:39.419441+0000","last_fullsized":"2023-07-06T03:46:39.419441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:34.577488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419682+0000","last_change":"2023-07-06T03:46:37.863461+0000","last_active":"2023-07-06T03:46:39.419682+0000","last_peered":"2023-07-06T03:46:39.419682+0000","last_clean":"2023-07-06T03:46:39.419682+0000","last_became_active":"2023-07-06T03:46:37.862923+0000","last_became_peered":"2023-07-06T03:46:37.862923+0000","last_unstale":"2023-07-06T03:46:39.419682+0000","last_undegraded":"2023-07-06T03:46:39.419682+0000","last_fullsized":"2023-07-06T03:46:39.419682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:04.849320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418404+0000","last_change":"2023-07-06T03:46:36.853092+0000","last_active":"2023-07-06T03:46:39.418404+0000","last_peered":"2023-07-06T03:46:39.418404+0000","last_clean":"2023-07-06T03:46:39.418404+0000","last_became_active":"2023-07-06T03:46:36.852795+0000","last_became_peered":"2023-07-06T03:46:36.852795+0000","last_unstale":"2023-07-06T03:46:39.418404+0000","last_undegraded":"2023-07-06T03:46:39.418404+0000","last_fullsized":"2023-07-06T03:46:39.418404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:49:11.074324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420266+0000","last_change":"2023-07-06T03:46:37.865852+0000","last_active":"2023-07-06T03:46:39.420266+0000","last_peered":"2023-07-06T03:46:39.420266+0000","last_clean":"2023-07-06T03:46:39.420266+0000","last_became_active":"2023-07-06T03:46:37.864988+0000","last_became_peered":"2023-07-06T03:46:37.864988+0000","last_unstale":"2023-07-06T03:46:39.420266+0000","last_undegraded":"2023-07-06T03:46:39.420266+0000","last_fullsized":"2023-07-06T03:46:39.420266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:47:11.749793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420238+0000","last_change":"2023-07-06T03:46:36.854389+0000","last_active":"2023-07-06T03:46:39.420238+0000","last_peered":"2023-07-06T03:46:39.420238+0000","last_clean":"2023-07-06T03:46:39.420238+0000","last_became_active":"2023-07-06T03:46:36.854083+0000","last_became_peered":"2023-07-06T03:46:36.854083+0000","last_unstale":"2023-07-06T03:46:39.420238+0000","last_undegraded":"2023-07-06T03:46:39.420238+0000","last_fullsized":"2023-07-06T03:46:39.420238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:07:57.989913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852508+0000","last_change":"2023-07-06T03:46:37.863905+0000","last_active":"2023-07-06T03:46:38.852508+0000","last_peered":"2023-07-06T03:46:38.852508+0000","last_clean":"2023-07-06T03:46:38.852508+0000","last_became_active":"2023-07-06T03:46:37.863639+0000","last_became_peered":"2023-07-06T03:46:37.863639+0000","last_unstale":"2023-07-06T03:46:38.852508+0000","last_undegraded":"2023-07-06T03:46:38.852508+0000","last_fullsized":"2023-07-06T03:46:38.852508+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:29:08.680714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419465+0000","last_change":"2023-07-06T03:46:36.848400+0000","last_active":"2023-07-06T03:46:39.419465+0000","last_peered":"2023-07-06T03:46:39.419465+0000","last_clean":"2023-07-06T03:46:39.419465+0000","last_became_active":"2023-07-06T03:46:36.848127+0000","last_became_peered":"2023-07-06T03:46:36.848127+0000","last_unstale":"2023-07-06T03:46:39.419465+0000","last_undegraded":"2023-07-06T03:46:39.419465+0000","last_fullsized":"2023-07-06T03:46:39.419465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:30:34.301467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420013+0000","last_change":"2023-07-06T03:46:37.849656+0000","last_active":"2023-07-06T03:46:39.420013+0000","last_peered":"2023-07-06T03:46:39.420013+0000","last_clean":"2023-07-06T03:46:39.420013+0000","last_became_active":"2023-07-06T03:46:37.849395+0000","last_became_peered":"2023-07-06T03:46:37.849395+0000","last_unstale":"2023-07-06T03:46:39.420013+0000","last_undegraded":"2023-07-06T03:46:39.420013+0000","last_fullsized":"2023-07-06T03:46:39.420013+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:04:53.111652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419205+0000","last_change":"2023-07-06T03:46:36.860726+0000","last_active":"2023-07-06T03:46:39.419205+0000","last_peered":"2023-07-06T03:46:39.419205+0000","last_clean":"2023-07-06T03:46:39.419205+0000","last_became_active":"2023-07-06T03:46:36.859745+0000","last_became_peered":"2023-07-06T03:46:36.859745+0000","last_unstale":"2023-07-06T03:46:39.419205+0000","last_undegraded":"2023-07-06T03:46:39.419205+0000","last_fullsized":"2023-07-06T03:46:39.419205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:24:19.605298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418784+0000","last_change":"2023-07-06T03:46:37.845211+0000","last_active":"2023-07-06T03:46:39.418784+0000","last_peered":"2023-07-06T03:46:39.418784+0000","last_clean":"2023-07-06T03:46:39.418784+0000","last_became_active":"2023-07-06T03:46:37.844944+0000","last_became_peered":"2023-07-06T03:46:37.844944+0000","last_unstale":"2023-07-06T03:46:39.418784+0000","last_undegraded":"2023-07-06T03:46:39.418784+0000","last_fullsized":"2023-07-06T03:46:39.418784+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:01:33.509255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420347+0000","last_change":"2023-07-06T03:46:36.858688+0000","last_active":"2023-07-06T03:46:39.420347+0000","last_peered":"2023-07-06T03:46:39.420347+0000","last_clean":"2023-07-06T03:46:39.420347+0000","last_became_active":"2023-07-06T03:46:36.858031+0000","last_became_peered":"2023-07-06T03:46:36.858031+0000","last_unstale":"2023-07-06T03:46:39.420347+0000","last_undegraded":"2023-07-06T03:46:39.420347+0000","last_fullsized":"2023-07-06T03:46:39.420347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:39:49.641446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419557+0000","last_change":"2023-07-06T03:46:37.852588+0000","last_active":"2023-07-06T03:46:39.419557+0000","last_peered":"2023-07-06T03:46:39.419557+0000","last_clean":"2023-07-06T03:46:39.419557+0000","last_became_active":"2023-07-06T03:46:37.852256+0000","last_became_peered":"2023-07-06T03:46:37.852256+0000","last_unstale":"2023-07-06T03:46:39.419557+0000","last_undegraded":"2023-07-06T03:46:39.419557+0000","last_fullsized":"2023-07-06T03:46:39.419557+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:24:39.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419552+0000","last_change":"2023-07-06T03:46:36.859170+0000","last_active":"2023-07-06T03:46:39.419552+0000","last_peered":"2023-07-06T03:46:39.419552+0000","last_clean":"2023-07-06T03:46:39.419552+0000","last_became_active":"2023-07-06T03:46:36.858764+0000","last_became_peered":"2023-07-06T03:46:36.858764+0000","last_unstale":"2023-07-06T03:46:39.419552+0000","last_undegraded":"2023-07-06T03:46:39.419552+0000","last_fullsized":"2023-07-06T03:46:39.419552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:07:48.265940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419356+0000","last_change":"2023-07-06T03:46:37.865476+0000","last_active":"2023-07-06T03:46:39.419356+0000","last_peered":"2023-07-06T03:46:39.419356+0000","last_clean":"2023-07-06T03:46:39.419356+0000","last_became_active":"2023-07-06T03:46:37.865057+0000","last_became_peered":"2023-07-06T03:46:37.865057+0000","last_unstale":"2023-07-06T03:46:39.419356+0000","last_undegraded":"2023-07-06T03:46:39.419356+0000","last_fullsized":"2023-07-06T03:46:39.419356+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:28.578365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419534+0000","last_change":"2023-07-06T03:46:36.856232+0000","last_active":"2023-07-06T03:46:39.419534+0000","last_peered":"2023-07-06T03:46:39.419534+0000","last_clean":"2023-07-06T03:46:39.419534+0000","last_became_active":"2023-07-06T03:46:36.855057+0000","last_became_peered":"2023-07-06T03:46:36.855057+0000","last_unstale":"2023-07-06T03:46:39.419534+0000","last_undegraded":"2023-07-06T03:46:39.419534+0000","last_fullsized":"2023-07-06T03:46:39.419534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:24.299644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041626+0000","last_change":"2023-07-06T03:46:37.869250+0000","last_active":"2023-07-06T03:46:39.041626+0000","last_peered":"2023-07-06T03:46:39.041626+0000","last_clean":"2023-07-06T03:46:39.041626+0000","last_became_active":"2023-07-06T03:46:37.868585+0000","last_became_peered":"2023-07-06T03:46:37.868585+0000","last_unstale":"2023-07-06T03:46:39.041626+0000","last_undegraded":"2023-07-06T03:46:39.041626+0000","last_fullsized":"2023-07-06T03:46:39.041626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.598122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419252+0000","last_change":"2023-07-06T03:46:36.848489+0000","last_active":"2023-07-06T03:46:39.419252+0000","last_peered":"2023-07-06T03:46:39.419252+0000","last_clean":"2023-07-06T03:46:39.419252+0000","last_became_active":"2023-07-06T03:46:36.848195+0000","last_became_peered":"2023-07-06T03:46:36.848195+0000","last_unstale":"2023-07-06T03:46:39.419252+0000","last_undegraded":"2023-07-06T03:46:39.419252+0000","last_fullsized":"2023-07-06T03:46:39.419252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:49:58.368321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853875+0000","last_change":"2023-07-06T03:46:37.857742+0000","last_active":"2023-07-06T03:46:38.853875+0000","last_peered":"2023-07-06T03:46:38.853875+0000","last_clean":"2023-07-06T03:46:38.853875+0000","last_became_active":"2023-07-06T03:46:37.857431+0000","last_became_peered":"2023-07-06T03:46:37.857431+0000","last_unstale":"2023-07-06T03:46:38.853875+0000","last_undegraded":"2023-07-06T03:46:38.853875+0000","last_fullsized":"2023-07-06T03:46:38.853875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:44:39.103954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852415+0000","last_change":"2023-07-06T03:46:36.850609+0000","last_active":"2023-07-06T03:46:38.852415+0000","last_peered":"2023-07-06T03:46:38.852415+0000","last_clean":"2023-07-06T03:46:38.852415+0000","last_became_active":"2023-07-06T03:46:36.850316+0000","last_became_peered":"2023-07-06T03:46:36.850316+0000","last_unstale":"2023-07-06T03:46:38.852415+0000","last_undegraded":"2023-07-06T03:46:38.852415+0000","last_fullsized":"2023-07-06T03:46:38.852415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:06:25.318103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419909+0000","last_change":"2023-07-06T03:46:37.858603+0000","last_active":"2023-07-06T03:46:39.419909+0000","last_peered":"2023-07-06T03:46:39.419909+0000","last_clean":"2023-07-06T03:46:39.419909+0000","last_became_active":"2023-07-06T03:46:37.858343+0000","last_became_peered":"2023-07-06T03:46:37.858343+0000","last_unstale":"2023-07-06T03:46:39.419909+0000","last_undegraded":"2023-07-06T03:46:39.419909+0000","last_fullsized":"2023-07-06T03:46:39.419909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:52.391457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418737+0000","last_change":"2023-07-06T03:46:36.852893+0000","last_active":"2023-07-06T03:46:39.418737+0000","last_peered":"2023-07-06T03:46:39.418737+0000","last_clean":"2023-07-06T03:46:39.418737+0000","last_became_active":"2023-07-06T03:46:36.852559+0000","last_became_peered":"2023-07-06T03:46:36.852559+0000","last_unstale":"2023-07-06T03:46:39.418737+0000","last_undegraded":"2023-07-06T03:46:39.418737+0000","last_fullsized":"2023-07-06T03:46:39.418737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:50:39.638633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418795+0000","last_change":"2023-07-06T03:46:37.847891+0000","last_active":"2023-07-06T03:46:39.418795+0000","last_peered":"2023-07-06T03:46:39.418795+0000","last_clean":"2023-07-06T03:46:39.418795+0000","last_became_active":"2023-07-06T03:46:37.847633+0000","last_became_peered":"2023-07-06T03:46:37.847633+0000","last_unstale":"2023-07-06T03:46:39.418795+0000","last_undegraded":"2023-07-06T03:46:39.418795+0000","last_fullsized":"2023-07-06T03:46:39.418795+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:51:53.004246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419605+0000","last_change":"2023-07-06T03:46:36.861088+0000","last_active":"2023-07-06T03:46:39.419605+0000","last_peered":"2023-07-06T03:46:39.419605+0000","last_clean":"2023-07-06T03:46:39.419605+0000","last_became_active":"2023-07-06T03:46:36.860765+0000","last_became_peered":"2023-07-06T03:46:36.860765+0000","last_unstale":"2023-07-06T03:46:39.419605+0000","last_undegraded":"2023-07-06T03:46:39.419605+0000","last_fullsized":"2023-07-06T03:46:39.419605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:01:12.237971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418844+0000","last_change":"2023-07-06T03:46:37.865269+0000","last_active":"2023-07-06T03:46:39.418844+0000","last_peered":"2023-07-06T03:46:39.418844+0000","last_clean":"2023-07-06T03:46:39.418844+0000","last_became_active":"2023-07-06T03:46:37.864958+0000","last_became_peered":"2023-07-06T03:46:37.864958+0000","last_unstale":"2023-07-06T03:46:39.418844+0000","last_undegraded":"2023-07-06T03:46:39.418844+0000","last_fullsized":"2023-07-06T03:46:39.418844+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:24:42.365036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853575+0000","last_change":"2023-07-06T03:46:36.856749+0000","last_active":"2023-07-06T03:46:38.853575+0000","last_peered":"2023-07-06T03:46:38.853575+0000","last_clean":"2023-07-06T03:46:38.853575+0000","last_became_active":"2023-07-06T03:46:36.856419+0000","last_became_peered":"2023-07-06T03:46:36.856419+0000","last_unstale":"2023-07-06T03:46:38.853575+0000","last_undegraded":"2023-07-06T03:46:38.853575+0000","last_fullsized":"2023-07-06T03:46:38.853575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:03.842589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851579+0000","last_change":"2023-07-06T03:46:37.864167+0000","last_active":"2023-07-06T03:46:38.851579+0000","last_peered":"2023-07-06T03:46:38.851579+0000","last_clean":"2023-07-06T03:46:38.851579+0000","last_became_active":"2023-07-06T03:46:37.863865+0000","last_became_peered":"2023-07-06T03:46:37.863865+0000","last_unstale":"2023-07-06T03:46:38.851579+0000","last_undegraded":"2023-07-06T03:46:38.851579+0000","last_fullsized":"2023-07-06T03:46:38.851579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:04.849320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851484+0000","last_change":"2023-07-06T03:46:36.855799+0000","last_active":"2023-07-06T03:46:38.851484+0000","last_peered":"2023-07-06T03:46:38.851484+0000","last_clean":"2023-07-06T03:46:38.851484+0000","last_became_active":"2023-07-06T03:46:36.855494+0000","last_became_peered":"2023-07-06T03:46:36.855494+0000","last_unstale":"2023-07-06T03:46:38.851484+0000","last_undegraded":"2023-07-06T03:46:38.851484+0000","last_fullsized":"2023-07-06T03:46:38.851484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:24.299644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419061+0000","last_change":"2023-07-06T03:46:37.858828+0000","last_active":"2023-07-06T03:46:39.419061+0000","last_peered":"2023-07-06T03:46:39.419061+0000","last_clean":"2023-07-06T03:46:39.419061+0000","last_became_active":"2023-07-06T03:46:37.858546+0000","last_became_peered":"2023-07-06T03:46:37.858546+0000","last_unstale":"2023-07-06T03:46:39.419061+0000","last_undegraded":"2023-07-06T03:46:39.419061+0000","last_fullsized":"2023-07-06T03:46:39.419061+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:26:24.568746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851488+0000","last_change":"2023-07-06T03:46:36.849693+0000","last_active":"2023-07-06T03:46:38.851488+0000","last_peered":"2023-07-06T03:46:38.851488+0000","last_clean":"2023-07-06T03:46:38.851488+0000","last_became_active":"2023-07-06T03:46:36.849420+0000","last_became_peered":"2023-07-06T03:46:36.849420+0000","last_unstale":"2023-07-06T03:46:38.851488+0000","last_undegraded":"2023-07-06T03:46:38.851488+0000","last_fullsized":"2023-07-06T03:46:38.851488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:15.789105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419350+0000","last_change":"2023-07-06T03:46:37.858086+0000","last_active":"2023-07-06T03:46:39.419350+0000","last_peered":"2023-07-06T03:46:39.419350+0000","last_clean":"2023-07-06T03:46:39.419350+0000","last_became_active":"2023-07-06T03:46:37.857821+0000","last_became_peered":"2023-07-06T03:46:37.857821+0000","last_unstale":"2023-07-06T03:46:39.419350+0000","last_undegraded":"2023-07-06T03:46:39.419350+0000","last_fullsized":"2023-07-06T03:46:39.419350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:31:28.228023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419674+0000","last_change":"2023-07-06T03:46:36.852875+0000","last_active":"2023-07-06T03:46:39.419674+0000","last_peered":"2023-07-06T03:46:39.419674+0000","last_clean":"2023-07-06T03:46:39.419674+0000","last_became_active":"2023-07-06T03:46:36.852400+0000","last_became_peered":"2023-07-06T03:46:36.852400+0000","last_unstale":"2023-07-06T03:46:39.419674+0000","last_undegraded":"2023-07-06T03:46:39.419674+0000","last_fullsized":"2023-07-06T03:46:39.419674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:52.817628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041760+0000","last_change":"2023-07-06T03:46:37.869442+0000","last_active":"2023-07-06T03:46:39.041760+0000","last_peered":"2023-07-06T03:46:39.041760+0000","last_clean":"2023-07-06T03:46:39.041760+0000","last_became_active":"2023-07-06T03:46:37.869000+0000","last_became_peered":"2023-07-06T03:46:37.869000+0000","last_unstale":"2023-07-06T03:46:39.041760+0000","last_undegraded":"2023-07-06T03:46:39.041760+0000","last_fullsized":"2023-07-06T03:46:39.041760+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:16.795837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418545+0000","last_change":"2023-07-06T03:46:36.854933+0000","last_active":"2023-07-06T03:46:39.418545+0000","last_peered":"2023-07-06T03:46:39.418545+0000","last_clean":"2023-07-06T03:46:39.418545+0000","last_became_active":"2023-07-06T03:46:36.854557+0000","last_became_peered":"2023-07-06T03:46:36.854557+0000","last_unstale":"2023-07-06T03:46:39.418545+0000","last_undegraded":"2023-07-06T03:46:39.418545+0000","last_fullsized":"2023-07-06T03:46:39.418545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:39:46.623924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419773+0000","last_change":"2023-07-06T03:46:37.848797+0000","last_active":"2023-07-06T03:46:39.419773+0000","last_peered":"2023-07-06T03:46:39.419773+0000","last_clean":"2023-07-06T03:46:39.419773+0000","last_became_active":"2023-07-06T03:46:37.848516+0000","last_became_peered":"2023-07-06T03:46:37.848516+0000","last_unstale":"2023-07-06T03:46:39.419773+0000","last_undegraded":"2023-07-06T03:46:39.419773+0000","last_fullsized":"2023-07-06T03:46:39.419773+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:22:45.955750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419042+0000","last_change":"2023-07-06T03:46:36.861470+0000","last_active":"2023-07-06T03:46:39.419042+0000","last_peered":"2023-07-06T03:46:39.419042+0000","last_clean":"2023-07-06T03:46:39.419042+0000","last_became_active":"2023-07-06T03:46:36.861150+0000","last_became_peered":"2023-07-06T03:46:36.861150+0000","last_unstale":"2023-07-06T03:46:39.419042+0000","last_undegraded":"2023-07-06T03:46:39.419042+0000","last_fullsized":"2023-07-06T03:46:39.419042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:14.108700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853519+0000","last_change":"2023-07-06T03:46:37.859672+0000","last_active":"2023-07-06T03:46:38.853519+0000","last_peered":"2023-07-06T03:46:38.853519+0000","last_clean":"2023-07-06T03:46:38.853519+0000","last_became_active":"2023-07-06T03:46:37.859405+0000","last_became_peered":"2023-07-06T03:46:37.859405+0000","last_unstale":"2023-07-06T03:46:38.853519+0000","last_undegraded":"2023-07-06T03:46:38.853519+0000","last_fullsized":"2023-07-06T03:46:38.853519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:25.306376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419877+0000","last_change":"2023-07-06T03:46:36.854707+0000","last_active":"2023-07-06T03:46:39.419877+0000","last_peered":"2023-07-06T03:46:39.419877+0000","last_clean":"2023-07-06T03:46:39.419877+0000","last_became_active":"2023-07-06T03:46:36.854435+0000","last_became_peered":"2023-07-06T03:46:36.854435+0000","last_unstale":"2023-07-06T03:46:39.419877+0000","last_undegraded":"2023-07-06T03:46:39.419877+0000","last_fullsized":"2023-07-06T03:46:39.419877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:05:36.862857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851576+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:46:38.851576+0000","last_peered":"2023-07-06T03:46:38.851576+0000","last_clean":"2023-07-06T03:46:38.851576+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:46:38.851576+0000","last_undegraded":"2023-07-06T03:46:38.851576+0000","last_fullsized":"2023-07-06T03:46:38.851576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:17:21.111789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851549+0000","last_change":"2023-07-06T03:46:37.862798+0000","last_active":"2023-07-06T03:46:38.851549+0000","last_peered":"2023-07-06T03:46:38.851549+0000","last_clean":"2023-07-06T03:46:38.851549+0000","last_became_active":"2023-07-06T03:46:37.862509+0000","last_became_peered":"2023-07-06T03:46:37.862509+0000","last_unstale":"2023-07-06T03:46:38.851549+0000","last_undegraded":"2023-07-06T03:46:38.851549+0000","last_fullsized":"2023-07-06T03:46:38.851549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:26.836331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853722+0000","last_change":"2023-07-06T03:46:36.859132+0000","last_active":"2023-07-06T03:46:38.853722+0000","last_peered":"2023-07-06T03:46:38.853722+0000","last_clean":"2023-07-06T03:46:38.853722+0000","last_became_active":"2023-07-06T03:46:36.858862+0000","last_became_peered":"2023-07-06T03:46:36.858862+0000","last_unstale":"2023-07-06T03:46:38.853722+0000","last_undegraded":"2023-07-06T03:46:38.853722+0000","last_fullsized":"2023-07-06T03:46:38.853722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:51.384726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419437+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:46:39.419437+0000","last_peered":"2023-07-06T03:46:39.419437+0000","last_clean":"2023-07-06T03:46:39.419437+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:46:39.419437+0000","last_undegraded":"2023-07-06T03:46:39.419437+0000","last_fullsized":"2023-07-06T03:46:39.419437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:04:05.608173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041981+0000","last_change":"2023-07-06T03:46:37.870020+0000","last_active":"2023-07-06T03:46:39.041981+0000","last_peered":"2023-07-06T03:46:39.041981+0000","last_clean":"2023-07-06T03:46:39.041981+0000","last_became_active":"2023-07-06T03:46:37.869716+0000","last_became_peered":"2023-07-06T03:46:37.869716+0000","last_unstale":"2023-07-06T03:46:39.041981+0000","last_undegraded":"2023-07-06T03:46:39.041981+0000","last_fullsized":"2023-07-06T03:46:39.041981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:52.391457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853769+0000","last_change":"2023-07-06T03:46:36.857376+0000","last_active":"2023-07-06T03:46:38.853769+0000","last_peered":"2023-07-06T03:46:38.853769+0000","last_clean":"2023-07-06T03:46:38.853769+0000","last_became_active":"2023-07-06T03:46:36.856783+0000","last_became_peered":"2023-07-06T03:46:36.856783+0000","last_unstale":"2023-07-06T03:46:38.853769+0000","last_undegraded":"2023-07-06T03:46:38.853769+0000","last_fullsized":"2023-07-06T03:46:38.853769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:53:50.026476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853513+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:46:38.853513+0000","last_peered":"2023-07-06T03:46:38.853513+0000","last_clean":"2023-07-06T03:46:38.853513+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:46:38.853513+0000","last_undegraded":"2023-07-06T03:46:38.853513+0000","last_fullsized":"2023-07-06T03:46:38.853513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:31:47.232229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419801+0000","last_change":"2023-07-06T03:46:37.859320+0000","last_active":"2023-07-06T03:46:39.419801+0000","last_peered":"2023-07-06T03:46:39.419801+0000","last_clean":"2023-07-06T03:46:39.419801+0000","last_became_active":"2023-07-06T03:46:37.858687+0000","last_became_peered":"2023-07-06T03:46:37.858687+0000","last_unstale":"2023-07-06T03:46:39.419801+0000","last_undegraded":"2023-07-06T03:46:39.419801+0000","last_fullsized":"2023-07-06T03:46:39.419801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:53.443866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041830+0000","last_change":"2023-07-06T03:46:36.858102+0000","last_active":"2023-07-06T03:46:39.041830+0000","last_peered":"2023-07-06T03:46:39.041830+0000","last_clean":"2023-07-06T03:46:39.041830+0000","last_became_active":"2023-07-06T03:46:36.857729+0000","last_became_peered":"2023-07-06T03:46:36.857729+0000","last_unstale":"2023-07-06T03:46:39.041830+0000","last_undegraded":"2023-07-06T03:46:39.041830+0000","last_fullsized":"2023-07-06T03:46:39.041830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:52.817628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419309+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:46:39.419309+0000","last_peered":"2023-07-06T03:46:39.419309+0000","last_clean":"2023-07-06T03:46:39.419309+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:46:39.419309+0000","last_undegraded":"2023-07-06T03:46:39.419309+0000","last_fullsized":"2023-07-06T03:46:39.419309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:28:47.569266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418463+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:46:39.418463+0000","last_peered":"2023-07-06T03:46:39.418463+0000","last_clean":"2023-07-06T03:46:39.418463+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:46:39.418463+0000","last_undegraded":"2023-07-06T03:46:39.418463+0000","last_fullsized":"2023-07-06T03:46:39.418463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:08:09.570675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853891+0000","last_change":"2023-07-06T03:46:37.853652+0000","last_active":"2023-07-06T03:46:38.853891+0000","last_peered":"2023-07-06T03:46:38.853891+0000","last_clean":"2023-07-06T03:46:38.853891+0000","last_became_active":"2023-07-06T03:46:37.853323+0000","last_became_peered":"2023-07-06T03:46:37.853323+0000","last_unstale":"2023-07-06T03:46:38.853891+0000","last_undegraded":"2023-07-06T03:46:38.853891+0000","last_fullsized":"2023-07-06T03:46:38.853891+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:02:10.182517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419816+0000","last_change":"2023-07-06T03:46:36.847001+0000","last_active":"2023-07-06T03:46:39.419816+0000","last_peered":"2023-07-06T03:46:39.419816+0000","last_clean":"2023-07-06T03:46:39.419816+0000","last_became_active":"2023-07-06T03:46:36.846669+0000","last_became_peered":"2023-07-06T03:46:36.846669+0000","last_unstale":"2023-07-06T03:46:39.419816+0000","last_undegraded":"2023-07-06T03:46:39.419816+0000","last_fullsized":"2023-07-06T03:46:39.419816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:19.239209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419000+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:46:39.419000+0000","last_peered":"2023-07-06T03:46:39.419000+0000","last_clean":"2023-07-06T03:46:39.419000+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:46:39.419000+0000","last_undegraded":"2023-07-06T03:46:39.419000+0000","last_fullsized":"2023-07-06T03:46:39.419000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:26:05.045659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419530+0000","last_change":"2023-07-06T03:46:37.861516+0000","last_active":"2023-07-06T03:46:39.419530+0000","last_peered":"2023-07-06T03:46:39.419530+0000","last_clean":"2023-07-06T03:46:39.419530+0000","last_became_active":"2023-07-06T03:46:37.860991+0000","last_became_peered":"2023-07-06T03:46:37.860991+0000","last_unstale":"2023-07-06T03:46:39.419530+0000","last_undegraded":"2023-07-06T03:46:39.419530+0000","last_fullsized":"2023-07-06T03:46:39.419530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:26.836331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041886+0000","last_change":"2023-07-06T03:46:36.855134+0000","last_active":"2023-07-06T03:46:39.041886+0000","last_peered":"2023-07-06T03:46:39.041886+0000","last_clean":"2023-07-06T03:46:39.041886+0000","last_became_active":"2023-07-06T03:46:36.854885+0000","last_became_peered":"2023-07-06T03:46:36.854885+0000","last_unstale":"2023-07-06T03:46:39.041886+0000","last_undegraded":"2023-07-06T03:46:39.041886+0000","last_fullsized":"2023-07-06T03:46:39.041886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:33:18.938068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419516+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:46:39.419516+0000","last_peered":"2023-07-06T03:46:39.419516+0000","last_clean":"2023-07-06T03:46:39.419516+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:46:39.419516+0000","last_undegraded":"2023-07-06T03:46:39.419516+0000","last_fullsized":"2023-07-06T03:46:39.419516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:36:14.721099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419354+0000","last_change":"2023-07-06T03:46:37.861288+0000","last_active":"2023-07-06T03:46:39.419354+0000","last_peered":"2023-07-06T03:46:39.419354+0000","last_clean":"2023-07-06T03:46:39.419354+0000","last_became_active":"2023-07-06T03:46:37.860989+0000","last_became_peered":"2023-07-06T03:46:37.860989+0000","last_unstale":"2023-07-06T03:46:39.419354+0000","last_undegraded":"2023-07-06T03:46:39.419354+0000","last_fullsized":"2023-07-06T03:46:39.419354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:46:55.890372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419276+0000","last_change":"2023-07-06T03:46:36.847922+0000","last_active":"2023-07-06T03:46:39.419276+0000","last_peered":"2023-07-06T03:46:39.419276+0000","last_clean":"2023-07-06T03:46:39.419276+0000","last_became_active":"2023-07-06T03:46:36.847302+0000","last_became_peered":"2023-07-06T03:46:36.847302+0000","last_unstale":"2023-07-06T03:46:39.419276+0000","last_undegraded":"2023-07-06T03:46:39.419276+0000","last_fullsized":"2023-07-06T03:46:39.419276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:14:40.997856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420277+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:46:39.420277+0000","last_peered":"2023-07-06T03:46:39.420277+0000","last_clean":"2023-07-06T03:46:39.420277+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:46:39.420277+0000","last_undegraded":"2023-07-06T03:46:39.420277+0000","last_fullsized":"2023-07-06T03:46:39.420277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:51:20.288814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041768+0000","last_change":"2023-07-06T03:46:37.862349+0000","last_active":"2023-07-06T03:46:39.041768+0000","last_peered":"2023-07-06T03:46:39.041768+0000","last_clean":"2023-07-06T03:46:39.041768+0000","last_became_active":"2023-07-06T03:46:37.862010+0000","last_became_peered":"2023-07-06T03:46:37.862010+0000","last_unstale":"2023-07-06T03:46:39.041768+0000","last_undegraded":"2023-07-06T03:46:39.041768+0000","last_fullsized":"2023-07-06T03:46:39.041768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:04.849320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419332+0000","last_change":"2023-07-06T03:46:36.861193+0000","last_active":"2023-07-06T03:46:39.419332+0000","last_peered":"2023-07-06T03:46:39.419332+0000","last_clean":"2023-07-06T03:46:39.419332+0000","last_became_active":"2023-07-06T03:46:36.860876+0000","last_became_peered":"2023-07-06T03:46:36.860876+0000","last_unstale":"2023-07-06T03:46:39.419332+0000","last_undegraded":"2023-07-06T03:46:39.419332+0000","last_fullsized":"2023-07-06T03:46:39.419332+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:24:46.275089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419845+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:46:39.419845+0000","last_peered":"2023-07-06T03:46:39.419845+0000","last_clean":"2023-07-06T03:46:39.419845+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:46:39.419845+0000","last_undegraded":"2023-07-06T03:46:39.419845+0000","last_fullsized":"2023-07-06T03:46:39.419845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:15:52.413417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":156752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.8660000000000001,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.77,"5min":3.77,"15min":3.77},"last":0.622},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.90300000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.81999999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.92500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":156716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.622}]},{"osd":1,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.36299999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.55200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":983040,"data_stored":747072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":13.055999999999999,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.503}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":12.989000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":13.023999999999999,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.874}]},{"osd":6,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.488,"5min":1.488,"15min":1.488},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":12.941000000000001,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.751}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.93000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":393216,"data_stored":156774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.65800000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":9.5960000000000001,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":158828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.628}]},{"osd":5,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71699999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.80200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":156772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":5.6550000000000002,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.442},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.498}]},{"osd":1,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":15.911,"5min":15.911,"15min":15.911},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":16.024000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":16.068999999999999,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":5.7300000000000004,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.73399999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.077,"5min":5.077,"15min":5.077},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":5.6980000000000004,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":5.157,"5min":5.157,"15min":5.157},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":5.1189999999999998,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.93000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":158370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":6.258,"5min":6.258,"15min":6.258},"last":0.496},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":6.4740000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":6.4859999999999998,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.74199999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.83899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":978944,"data_stored":746613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":6.4059999999999997,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.624}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.90600000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.879}]},{"osd":7,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.83199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:46:43.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-07-06T03:46:43.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.405+0000 7fd0fcfff700 1 Processor -- start 2023-07-06T03:46:43.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.406+0000 7fd0fcfff700 1 -- start start 2023-07-06T03:46:43.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.407+0000 7fd0fcfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd0f80cd8e0 0x7fd0f815e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:43.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.407+0000 7fd0fcfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f8070c30 0x7fd0f80adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:43.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.407+0000 7fd0fcfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f80710a0 con 0x7fd0f80cdcb0 2023-07-06T03:46:43.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.407+0000 7fd0fcfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f80ae3e0 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.407+0000 7fd0fcfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8071330 con 0x7fd0f8070c30 2023-07-06T03:46:43.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.407+0000 7fd0f6d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd0f80cdcb0 0x7fd0f806f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50854/0 (socket says 172.21.15.97:50854) 2023-07-06T03:46:43.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.407+0000 7fd0f6d9d700 1 -- 172.21.15.97:0/2502967275 learned_addr learned my addr 172.21.15.97:0/2502967275 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:43.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.407+0000 7fd0f5d9b700 1 --2- 172.21.15.97:0/2502967275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f8070c30 0x7fd0f80adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:43.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.407+0000 7fd0f659c700 1 --2- 172.21.15.97:0/2502967275 >> v2:172.21.15.146:3300/0 conn(0x7fd0f80cd8e0 0x7fd0f815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:43.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.407+0000 7fd0f559a700 1 -- 172.21.15.97:0/2502967275 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326984261 0 0) 0x7fd0f80710a0 con 0x7fd0f80cdcb0 2023-07-06T03:46:43.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.408+0000 7fd0f559a700 1 -- 172.21.15.97:0/2502967275 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0e00035b0 con 0x7fd0f80cdcb0 2023-07-06T03:46:43.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.408+0000 7fd0f5d9b700 1 -- 172.21.15.97:0/2502967275 >> v2:172.21.15.146:3300/0 conn(0x7fd0f80cd8e0 msgr2=0x7fd0f815e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:43.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.408+0000 7fd0f5d9b700 1 --2- 172.21.15.97:0/2502967275 >> v2:172.21.15.146:3300/0 conn(0x7fd0f80cd8e0 0x7fd0f815e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.408+0000 7fd0f5d9b700 1 -- 172.21.15.97:0/2502967275 >> v1:172.21.15.97:6789/0 conn(0x7fd0f80cdcb0 legacy=0x7fd0f806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.408+0000 7fd0f559a700 1 -- 172.21.15.97:0/2502967275 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613851610 0 0) 0x7fd0e00035b0 con 0x7fd0f80cdcb0 2023-07-06T03:46:43.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.408+0000 7fd0f5d9b700 1 -- 172.21.15.97:0/2502967275 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f8071610 con 0x7fd0f8070c30 2023-07-06T03:46:43.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.408+0000 7fd0f5d9b700 1 --2- 172.21.15.97:0/2502967275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f8070c30 0x7fd0f80adea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009840 tx=0x7fd0ec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa24dbe72b5c653 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:43.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.408+0000 7fd0f559a700 1 -- 172.21.15.97:0/2502967275 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00019f0 con 0x7fd0f8070c30 2023-07-06T03:46:43.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.408+0000 7fd0f559a700 1 -- 172.21.15.97:0/2502967275 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ec004030 con 0x7fd0f8070c30 2023-07-06T03:46:43.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.408+0000 7fd0f559a700 1 -- 172.21.15.97:0/2502967275 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec0043c0 con 0x7fd0f8070c30 2023-07-06T03:46:43.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.409+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2502967275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f8070c30 msgr2=0x7fd0f80adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.409+0000 7fd0fcfff700 1 --2- 172.21.15.97:0/2502967275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f8070c30 0x7fd0f80adea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009840 tx=0x7fd0ec009b50 comp rx=0 tx=0).stop 2023-07-06T03:46:43.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.409+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2502967275 shutdown_connections 2023-07-06T03:46:43.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.409+0000 7fd0fcfff700 1 --2- 172.21.15.97:0/2502967275 >> v2:172.21.15.146:3300/0 conn(0x7fd0f80cd8e0 0x7fd0f815e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.409+0000 7fd0fcfff700 1 --2- 172.21.15.97:0/2502967275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f8070c30 0x7fd0f80adea0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.409+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2502967275 >> 172.21.15.97:0/2502967275 conn(0x7fd0f80ce8c0 msgr2=0x7fd0f80cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:43.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.409+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2502967275 shutdown_connections 2023-07-06T03:46:43.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.409+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2502967275 wait complete. 2023-07-06T03:46:43.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.411+0000 7fd0fcfff700 1 Processor -- start 2023-07-06T03:46:43.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.411+0000 7fd0fcfff700 1 -- start start 2023-07-06T03:46:43.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.412+0000 7fd0fcfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f8070c30 0x7fd0f8149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:43.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.412+0000 7fd0fcfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f80cd8e0 0x7fd0f814ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:43.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.412+0000 7fd0fcfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0f80cdcb0 0x7fd0f80ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:43.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.412+0000 7fd0fcfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8147550 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.412+0000 7fd0fcfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8149ab0 con 0x7fd0f80cdcb0 2023-07-06T03:46:43.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.412+0000 7fd0fcfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f814c370 con 0x7fd0f8070c30 2023-07-06T03:46:43.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.412+0000 7fd0f5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f80cd8e0 0x7fd0f814ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:43.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.412+0000 7fd0f5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f80cd8e0 0x7fd0f814ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58174/0 (socket says 172.21.15.97:58174) 2023-07-06T03:46:43.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.412+0000 7fd0f5d9b700 1 -- 172.21.15.97:0/2659039640 learned_addr learned my addr 172.21.15.97:0/2659039640 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:43.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.412+0000 7fd0f659c700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f8070c30 0x7fd0f8149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:43.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0f6d9d700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0f80cdcb0 0x7fd0f80ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:43.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0f5d9b700 1 -- 172.21.15.97:0/2659039640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f8070c30 msgr2=0x7fd0f8149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0f5d9b700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f8070c30 0x7fd0f8149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0f5d9b700 1 -- 172.21.15.97:0/2659039640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0f80cdcb0 msgr2=0x7fd0f80ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0f5d9b700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0f80cdcb0 0x7fd0f80ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0f5d9b700 1 -- 172.21.15.97:0/2659039640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec0094a0 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0f6d9d700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0f80cdcb0 0x7fd0f80ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0f5d9b700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f80cd8e0 0x7fd0f814ba20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00a010 tx=0x7fd0ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0e6ffd700 1 -- 172.21.15.97:0/2659039640 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec003c00 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0e6ffd700 1 -- 172.21.15.97:0/2659039640 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ec01f070 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0e6ffd700 1 -- 172.21.15.97:0/2659039640 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec011630 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2659039640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f8146c20 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.413+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2659039640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f81471e0 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.414+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2659039640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f8070aa0 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.415+0000 7fd0e6ffd700 1 -- 172.21.15.97:0/2659039640 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd0ec021030 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.417+0000 7fd0e6ffd700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 0x7fd0d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:43.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.417+0000 7fd0e6ffd700 1 -- 172.21.15.97:0/2659039640 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd0ec08a770 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.417+0000 7fd0f659c700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 0x7fd0d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:43.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.417+0000 7fd0f659c700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 0x7fd0d806d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009b70 tx=0x7fd0e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:43.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.421+0000 7fd0e6ffd700 1 -- 172.21.15.97:0/2659039640 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd0ec055810 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.692+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2659039640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd0f8067830 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.692+0000 7fd0e6ffd700 1 -- 172.21.15.97:0/2659039640 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1109 (secure 0 0 0) 0x7fd0ec058e30 con 0x7fd0f80cd8e0 2023-07-06T03:46:43.694 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:43.694 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":119,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1688615095,"num_in_osds":8,"osd_in_since":1688615087,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":224673792,"bytes_avail":772869439488,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5301}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-07-06T03:46:16.895526+0000","services":{}},"progress_events":{}} 2023-07-06T03:46:43.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.694+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2659039640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 msgr2=0x7fd0d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.694+0000 7fd0fcfff700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 0x7fd0d806d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009b70 tx=0x7fd0e8008040 comp rx=0 tx=0).stop 2023-07-06T03:46:43.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.694+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2659039640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f80cd8e0 msgr2=0x7fd0f814ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.694+0000 7fd0fcfff700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f80cd8e0 0x7fd0f814ba20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00a010 tx=0x7fd0ec004750 comp rx=0 tx=0).stop 2023-07-06T03:46:43.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.694+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2659039640 shutdown_connections 2023-07-06T03:46:43.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.694+0000 7fd0fcfff700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 0x7fd0d806d190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.694+0000 7fd0fcfff700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0f80cdcb0 0x7fd0f80ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.694+0000 7fd0fcfff700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f8070c30 0x7fd0f8149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.694+0000 7fd0fcfff700 1 --2- 172.21.15.97:0/2659039640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0f80cd8e0 0x7fd0f814ba20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.694+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2659039640 >> 172.21.15.97:0/2659039640 conn(0x7fd0f80ce8c0 msgr2=0x7fd0f80ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:43.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.694+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2659039640 shutdown_connections 2023-07-06T03:46:43.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.695+0000 7fd0fcfff700 1 -- 172.21.15.97:0/2659039640 wait complete. 2023-07-06T03:46:43.714 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-07-06T03:46:43.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:46:43.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.909+0000 7f4bb75ff700 1 Processor -- start 2023-07-06T03:46:43.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.909+0000 7f4bb75ff700 1 -- start start 2023-07-06T03:46:43.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.910+0000 7f4bb75ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4bb00cd390 0x7f4bb0160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:43.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.910+0000 7f4bb75ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb00cdae0 con 0x7f4bb0160dd0 2023-07-06T03:46:43.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.910+0000 7f4bb75ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb00b6640 con 0x7f4bb015c520 2023-07-06T03:46:43.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.910+0000 7f4bb75ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb00b6920 con 0x7f4bb00cd390 2023-07-06T03:46:43.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.911+0000 7f4bb5b9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4bb0160dd0 0x7f4bb00b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50864/0 (socket says 172.21.15.97:50864) 2023-07-06T03:46:43.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.911+0000 7f4bb5b9c700 1 -- 172.21.15.97:0/2752142653 learned_addr learned my addr 172.21.15.97:0/2752142653 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.911+0000 7f4bb4b9a700 1 --2- 172.21.15.97:0/2752142653 >> v2:172.21.15.146:3300/0 conn(0x7f4bb00cd390 0x7f4bb0160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.911+0000 7f4ba7fff700 1 -- 172.21.15.97:0/2752142653 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029604911 0 0) 0x7f4bb00cdae0 con 0x7f4bb0160dd0 2023-07-06T03:46:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.911+0000 7f4ba7fff700 1 -- 172.21.15.97:0/2752142653 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b9c0035b0 con 0x7f4bb0160dd0 2023-07-06T03:46:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.911+0000 7f4ba7fff700 1 -- 172.21.15.97:0/2752142653 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791983920 0 0) 0x7f4bb00b6640 con 0x7f4bb015c520 2023-07-06T03:46:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.911+0000 7f4ba7fff700 1 -- 172.21.15.97:0/2752142653 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bb00cdae0 con 0x7f4bb015c520 2023-07-06T03:46:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.911+0000 7f4bb4b9a700 1 -- 172.21.15.97:0/2752142653 >> v1:172.21.15.146:6789/0 conn(0x7f4bb015c520 legacy=0x7f4bb0160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.911+0000 7f4bb4b9a700 1 -- 172.21.15.97:0/2752142653 >> v1:172.21.15.97:6789/0 conn(0x7f4bb0160dd0 legacy=0x7f4bb00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.911+0000 7f4bb4b9a700 1 -- 172.21.15.97:0/2752142653 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb00b6c00 con 0x7f4bb00cd390 2023-07-06T03:46:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.912+0000 7f4bb4b9a700 1 --2- 172.21.15.97:0/2752142653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bb00cd390 0x7f4bb0160890 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009840 tx=0x7f4ba0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f36e288a3c9a635f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.912+0000 7f4ba7fff700 1 -- 172.21.15.97:0/2752142653 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0011830 con 0x7f4bb00cd390 2023-07-06T03:46:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.912+0000 7f4ba7fff700 1 -- 172.21.15.97:0/2752142653 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba0011990 con 0x7f4bb00cd390 2023-07-06T03:46:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.912+0000 7f4ba7fff700 1 -- 172.21.15.97:0/2752142653 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0011d00 con 0x7f4bb00cd390 2023-07-06T03:46:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.912+0000 7f4bb75ff700 1 -- 172.21.15.97:0/2752142653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bb00cd390 msgr2=0x7f4bb0160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.912+0000 7f4bb75ff700 1 --2- 172.21.15.97:0/2752142653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bb00cd390 0x7f4bb0160890 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009840 tx=0x7f4ba0009b50 comp rx=0 tx=0).stop 2023-07-06T03:46:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.913+0000 7f4bb75ff700 1 -- 172.21.15.97:0/2752142653 shutdown_connections 2023-07-06T03:46:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.913+0000 7f4bb75ff700 1 --2- 172.21.15.97:0/2752142653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bb00cd390 0x7f4bb0160890 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.913+0000 7f4bb75ff700 1 -- 172.21.15.97:0/2752142653 >> 172.21.15.97:0/2752142653 conn(0x7f4bb0071790 msgr2=0x7f4bb0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.913+0000 7f4bb75ff700 1 -- 172.21.15.97:0/2752142653 shutdown_connections 2023-07-06T03:46:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.913+0000 7f4bb75ff700 1 -- 172.21.15.97:0/2752142653 wait complete. 2023-07-06T03:46:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.914+0000 7f4bb75ff700 1 Processor -- start 2023-07-06T03:46:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.914+0000 7f4bb75ff700 1 -- start start 2023-07-06T03:46:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.915+0000 7f4bb75ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bb00cd390 0x7f4bb0157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.915+0000 7f4bb75ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb015c520 0x7f4bb00cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.915+0000 7f4bb75ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bb0160dd0 0x7f4bb00cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.915+0000 7f4bb75ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb00d3c90 con 0x7f4bb015c520 2023-07-06T03:46:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.915+0000 7f4bb75ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb00d33f0 con 0x7f4bb00cd390 2023-07-06T03:46:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.915+0000 7f4bb75ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb0157840 con 0x7f4bb0160dd0 2023-07-06T03:46:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.916+0000 7f4bb4b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb015c520 0x7f4bb00cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.916+0000 7f4bb4b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb015c520 0x7f4bb00cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58180/0 (socket says 172.21.15.97:58180) 2023-07-06T03:46:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.916+0000 7f4bb4b9a700 1 -- 172.21.15.97:0/4202783082 learned_addr learned my addr 172.21.15.97:0/4202783082 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.916+0000 7f4bb539b700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bb00cd390 0x7f4bb0157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.916+0000 7f4bb4b9a700 1 -- 172.21.15.97:0/4202783082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bb0160dd0 msgr2=0x7f4bb00cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.916+0000 7f4bb4b9a700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bb0160dd0 0x7f4bb00cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.916+0000 7f4bb4b9a700 1 -- 172.21.15.97:0/4202783082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bb00cd390 msgr2=0x7f4bb0157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.916+0000 7f4bb4b9a700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bb00cd390 0x7f4bb0157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.916+0000 7f4bb4b9a700 1 -- 172.21.15.97:0/4202783082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba00094a0 con 0x7f4bb015c520 2023-07-06T03:46:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.916+0000 7f4bb539b700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bb00cd390 0x7f4bb0157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.916+0000 7f4bb4b9a700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb015c520 0x7f4bb00cbfe0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009b90 tx=0x7f4ba0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.917+0000 7f4ba5ffb700 1 -- 172.21.15.97:0/4202783082 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00045c0 con 0x7f4bb015c520 2023-07-06T03:46:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.917+0000 7f4ba5ffb700 1 -- 172.21.15.97:0/4202783082 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba001e070 con 0x7f4bb015c520 2023-07-06T03:46:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.917+0000 7f4bb75ff700 1 -- 172.21.15.97:0/4202783082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb01585a0 con 0x7f4bb015c520 2023-07-06T03:46:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.917+0000 7f4ba5ffb700 1 -- 172.21.15.97:0/4202783082 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0003900 con 0x7f4bb015c520 2023-07-06T03:46:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.917+0000 7f4bb75ff700 1 -- 172.21.15.97:0/4202783082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb00d3920 con 0x7f4bb015c520 2023-07-06T03:46:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.918+0000 7f4bb75ff700 1 -- 172.21.15.97:0/4202783082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb004e340 con 0x7f4bb015c520 2023-07-06T03:46:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.919+0000 7f4ba5ffb700 1 -- 172.21.15.97:0/4202783082 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ba0003a60 con 0x7f4bb015c520 2023-07-06T03:46:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.920+0000 7f4ba5ffb700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b9406ace0 0x7f4b9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.920+0000 7f4ba5ffb700 1 -- 172.21.15.97:0/4202783082 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ba008a490 con 0x7f4bb015c520 2023-07-06T03:46:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.920+0000 7f4bb539b700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b9406ace0 0x7f4b9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.921+0000 7f4bb539b700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b9406ace0 0x7f4b9406d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0011e0 tx=0x7f4bac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:43.925+0000 7f4ba5ffb700 1 -- 172.21.15.97:0/4202783082 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4ba00554a0 con 0x7f4bb015c520 2023-07-06T03:46:44.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.115+0000 7f4bb75ff700 1 -- 172.21.15.97:0/4202783082 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4bb00cd800 con 0x7f4b9406ace0 2023-07-06T03:46:44.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.124+0000 7f4ba5ffb700 1 -- 172.21.15.97:0/4202783082 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310874 (secure 0 0 0) 0x7f4bac001c70 con 0x7f4b9406ace0 2023-07-06T03:46:44.130 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:44.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 -- 172.21.15.97:0/4202783082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b9406ace0 msgr2=0x7f4b9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:44.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b9406ace0 0x7f4b9406d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0011e0 tx=0x7f4bac009040 comp rx=0 tx=0).stop 2023-07-06T03:46:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 -- 172.21.15.97:0/4202783082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb015c520 msgr2=0x7f4bb00cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb015c520 0x7f4bb00cbfe0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009b90 tx=0x7f4ba0004750 comp rx=0 tx=0).stop 2023-07-06T03:46:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 -- 172.21.15.97:0/4202783082 shutdown_connections 2023-07-06T03:46:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b9406ace0 0x7f4b9406d190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:44.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4bb0160dd0 0x7f4bb00cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:44.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4bb00cd390 0x7f4bb0157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:44.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 --2- 172.21.15.97:0/4202783082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb015c520 0x7f4bb00cbfe0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:44.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 -- 172.21.15.97:0/4202783082 >> 172.21.15.97:0/4202783082 conn(0x7f4bb0071790 msgr2=0x7f4bb00c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:44.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 -- 172.21.15.97:0/4202783082 shutdown_connections 2023-07-06T03:46:44.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:44.129+0000 7f4bb75ff700 1 -- 172.21.15.97:0/4202783082 wait complete. 2023-07-06T03:46:44.136 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:46:44.149 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-07-06T03:46:42.899899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219844,"kb_used_data":4228,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754876,"statfs":{"total":773094113280,"available":772868993024,"internally_reserved":0,"allocated":4329472,"data_stored":2437897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.064157"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419924+0000","last_change":"2023-07-06T03:46:37.856906+0000","last_active":"2023-07-06T03:46:39.419924+0000","last_peered":"2023-07-06T03:46:39.419924+0000","last_clean":"2023-07-06T03:46:39.419924+0000","last_became_active":"2023-07-06T03:46:37.856640+0000","last_became_peered":"2023-07-06T03:46:37.856640+0000","last_unstale":"2023-07-06T03:46:39.419924+0000","last_undegraded":"2023-07-06T03:46:39.419924+0000","last_fullsized":"2023-07-06T03:46:39.419924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:00:51.494589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419658+0000","last_change":"2023-07-06T03:46:37.857685+0000","last_active":"2023-07-06T03:46:39.419658+0000","last_peered":"2023-07-06T03:46:39.419658+0000","last_clean":"2023-07-06T03:46:39.419658+0000","last_became_active":"2023-07-06T03:46:37.857373+0000","last_became_peered":"2023-07-06T03:46:37.857373+0000","last_unstale":"2023-07-06T03:46:39.419658+0000","last_undegraded":"2023-07-06T03:46:39.419658+0000","last_fullsized":"2023-07-06T03:46:39.419658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:16.795837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420131+0000","last_change":"2023-07-06T03:46:37.862846+0000","last_active":"2023-07-06T03:46:39.420131+0000","last_peered":"2023-07-06T03:46:39.420131+0000","last_clean":"2023-07-06T03:46:39.420131+0000","last_became_active":"2023-07-06T03:46:37.862554+0000","last_became_peered":"2023-07-06T03:46:37.862554+0000","last_unstale":"2023-07-06T03:46:39.420131+0000","last_undegraded":"2023-07-06T03:46:39.420131+0000","last_fullsized":"2023-07-06T03:46:39.420131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:08:02.995580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420078+0000","last_change":"2023-07-06T03:46:37.866028+0000","last_active":"2023-07-06T03:46:39.420078+0000","last_peered":"2023-07-06T03:46:39.420078+0000","last_clean":"2023-07-06T03:46:39.420078+0000","last_became_active":"2023-07-06T03:46:37.865346+0000","last_became_peered":"2023-07-06T03:46:37.865346+0000","last_unstale":"2023-07-06T03:46:39.420078+0000","last_undegraded":"2023-07-06T03:46:39.420078+0000","last_fullsized":"2023-07-06T03:46:39.420078+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:59:06.270128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853587+0000","last_change":"2023-07-06T03:46:37.864390+0000","last_active":"2023-07-06T03:46:38.853587+0000","last_peered":"2023-07-06T03:46:38.853587+0000","last_clean":"2023-07-06T03:46:38.853587+0000","last_became_active":"2023-07-06T03:46:37.864064+0000","last_became_peered":"2023-07-06T03:46:37.864064+0000","last_unstale":"2023-07-06T03:46:38.853587+0000","last_undegraded":"2023-07-06T03:46:38.853587+0000","last_fullsized":"2023-07-06T03:46:38.853587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:16.795837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419343+0000","last_change":"2023-07-06T03:46:37.861836+0000","last_active":"2023-07-06T03:46:39.419343+0000","last_peered":"2023-07-06T03:46:39.419343+0000","last_clean":"2023-07-06T03:46:39.419343+0000","last_became_active":"2023-07-06T03:46:37.861568+0000","last_became_peered":"2023-07-06T03:46:37.861568+0000","last_unstale":"2023-07-06T03:46:39.419343+0000","last_undegraded":"2023-07-06T03:46:39.419343+0000","last_fullsized":"2023-07-06T03:46:39.419343+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.598122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418487+0000","last_change":"2023-07-06T03:46:37.844697+0000","last_active":"2023-07-06T03:46:39.418487+0000","last_peered":"2023-07-06T03:46:39.418487+0000","last_clean":"2023-07-06T03:46:39.418487+0000","last_became_active":"2023-07-06T03:46:37.844412+0000","last_became_peered":"2023-07-06T03:46:37.844412+0000","last_unstale":"2023-07-06T03:46:39.418487+0000","last_undegraded":"2023-07-06T03:46:39.418487+0000","last_fullsized":"2023-07-06T03:46:39.418487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:40:38.338794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419209+0000","last_change":"2023-07-06T03:46:37.851682+0000","last_active":"2023-07-06T03:46:39.419209+0000","last_peered":"2023-07-06T03:46:39.419209+0000","last_clean":"2023-07-06T03:46:39.419209+0000","last_became_active":"2023-07-06T03:46:37.851342+0000","last_became_peered":"2023-07-06T03:46:37.851342+0000","last_unstale":"2023-07-06T03:46:39.419209+0000","last_undegraded":"2023-07-06T03:46:39.419209+0000","last_fullsized":"2023-07-06T03:46:39.419209+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:31:39.275779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.854169+0000","last_change":"2023-07-06T03:46:37.860538+0000","last_active":"2023-07-06T03:46:38.854169+0000","last_peered":"2023-07-06T03:46:38.854169+0000","last_clean":"2023-07-06T03:46:38.854169+0000","last_became_active":"2023-07-06T03:46:37.860253+0000","last_became_peered":"2023-07-06T03:46:37.860253+0000","last_unstale":"2023-07-06T03:46:38.854169+0000","last_undegraded":"2023-07-06T03:46:38.854169+0000","last_fullsized":"2023-07-06T03:46:38.854169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:40:38.338794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418842+0000","last_change":"2023-07-06T03:46:37.861893+0000","last_active":"2023-07-06T03:46:39.418842+0000","last_peered":"2023-07-06T03:46:39.418842+0000","last_clean":"2023-07-06T03:46:39.418842+0000","last_became_active":"2023-07-06T03:46:37.861562+0000","last_became_peered":"2023-07-06T03:46:37.861562+0000","last_unstale":"2023-07-06T03:46:39.418842+0000","last_undegraded":"2023-07-06T03:46:39.418842+0000","last_fullsized":"2023-07-06T03:46:39.418842+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:10:32.090820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.042093+0000","last_change":"2023-07-06T03:46:37.869263+0000","last_active":"2023-07-06T03:46:39.042093+0000","last_peered":"2023-07-06T03:46:39.042093+0000","last_clean":"2023-07-06T03:46:39.042093+0000","last_became_active":"2023-07-06T03:46:37.868621+0000","last_became_peered":"2023-07-06T03:46:37.868621+0000","last_unstale":"2023-07-06T03:46:39.042093+0000","last_undegraded":"2023-07-06T03:46:39.042093+0000","last_fullsized":"2023-07-06T03:46:39.042093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:02:10.182517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853685+0000","last_change":"2023-07-06T03:46:37.858638+0000","last_active":"2023-07-06T03:46:38.853685+0000","last_peered":"2023-07-06T03:46:38.853685+0000","last_clean":"2023-07-06T03:46:38.853685+0000","last_became_active":"2023-07-06T03:46:37.858354+0000","last_became_peered":"2023-07-06T03:46:37.858354+0000","last_unstale":"2023-07-06T03:46:38.853685+0000","last_undegraded":"2023-07-06T03:46:38.853685+0000","last_fullsized":"2023-07-06T03:46:38.853685+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:53.824359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418613+0000","last_change":"2023-07-06T03:46:37.864649+0000","last_active":"2023-07-06T03:46:39.418613+0000","last_peered":"2023-07-06T03:46:39.418613+0000","last_clean":"2023-07-06T03:46:39.418613+0000","last_became_active":"2023-07-06T03:46:37.864204+0000","last_became_peered":"2023-07-06T03:46:37.864204+0000","last_unstale":"2023-07-06T03:46:39.418613+0000","last_undegraded":"2023-07-06T03:46:39.418613+0000","last_fullsized":"2023-07-06T03:46:39.418613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:29:38.817047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852484+0000","last_change":"2023-07-06T03:46:37.862356+0000","last_active":"2023-07-06T03:46:38.852484+0000","last_peered":"2023-07-06T03:46:38.852484+0000","last_clean":"2023-07-06T03:46:38.852484+0000","last_became_active":"2023-07-06T03:46:37.862123+0000","last_became_peered":"2023-07-06T03:46:37.862123+0000","last_unstale":"2023-07-06T03:46:38.852484+0000","last_undegraded":"2023-07-06T03:46:38.852484+0000","last_fullsized":"2023-07-06T03:46:38.852484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:37:12.375776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419159+0000","last_change":"2023-07-06T03:46:37.859689+0000","last_active":"2023-07-06T03:46:39.419159+0000","last_peered":"2023-07-06T03:46:39.419159+0000","last_clean":"2023-07-06T03:46:39.419159+0000","last_became_active":"2023-07-06T03:46:37.859356+0000","last_became_peered":"2023-07-06T03:46:37.859356+0000","last_unstale":"2023-07-06T03:46:39.419159+0000","last_undegraded":"2023-07-06T03:46:39.419159+0000","last_fullsized":"2023-07-06T03:46:39.419159+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:44.223992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418293+0000","last_change":"2023-07-06T03:46:37.860022+0000","last_active":"2023-07-06T03:46:39.418293+0000","last_peered":"2023-07-06T03:46:39.418293+0000","last_clean":"2023-07-06T03:46:39.418293+0000","last_became_active":"2023-07-06T03:46:37.859715+0000","last_became_peered":"2023-07-06T03:46:37.859715+0000","last_unstale":"2023-07-06T03:46:39.418293+0000","last_undegraded":"2023-07-06T03:46:39.418293+0000","last_fullsized":"2023-07-06T03:46:39.418293+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:44:39.103954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419760+0000","last_change":"2023-07-06T03:46:37.861889+0000","last_active":"2023-07-06T03:46:39.419760+0000","last_peered":"2023-07-06T03:46:39.419760+0000","last_clean":"2023-07-06T03:46:39.419760+0000","last_became_active":"2023-07-06T03:46:37.860756+0000","last_became_peered":"2023-07-06T03:46:37.860756+0000","last_unstale":"2023-07-06T03:46:39.419760+0000","last_undegraded":"2023-07-06T03:46:39.419760+0000","last_fullsized":"2023-07-06T03:46:39.419760+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:50:46.498928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419563+0000","last_change":"2023-07-06T03:46:37.860737+0000","last_active":"2023-07-06T03:46:39.419563+0000","last_peered":"2023-07-06T03:46:39.419563+0000","last_clean":"2023-07-06T03:46:39.419563+0000","last_became_active":"2023-07-06T03:46:37.860452+0000","last_became_peered":"2023-07-06T03:46:37.860452+0000","last_unstale":"2023-07-06T03:46:39.419563+0000","last_undegraded":"2023-07-06T03:46:39.419563+0000","last_fullsized":"2023-07-06T03:46:39.419563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:50.812126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851546+0000","last_change":"2023-07-06T03:46:37.864336+0000","last_active":"2023-07-06T03:46:38.851546+0000","last_peered":"2023-07-06T03:46:38.851546+0000","last_clean":"2023-07-06T03:46:38.851546+0000","last_became_active":"2023-07-06T03:46:37.863953+0000","last_became_peered":"2023-07-06T03:46:37.863953+0000","last_unstale":"2023-07-06T03:46:38.851546+0000","last_undegraded":"2023-07-06T03:46:38.851546+0000","last_fullsized":"2023-07-06T03:46:38.851546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:28.578365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041710+0000","last_change":"2023-07-06T03:46:37.862894+0000","last_active":"2023-07-06T03:46:39.041710+0000","last_peered":"2023-07-06T03:46:39.041710+0000","last_clean":"2023-07-06T03:46:39.041710+0000","last_became_active":"2023-07-06T03:46:37.862643+0000","last_became_peered":"2023-07-06T03:46:37.862643+0000","last_unstale":"2023-07-06T03:46:39.041710+0000","last_undegraded":"2023-07-06T03:46:39.041710+0000","last_fullsized":"2023-07-06T03:46:39.041710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:25.306376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041686+0000","last_change":"2023-07-06T03:46:37.863993+0000","last_active":"2023-07-06T03:46:39.041686+0000","last_peered":"2023-07-06T03:46:39.041686+0000","last_clean":"2023-07-06T03:46:39.041686+0000","last_became_active":"2023-07-06T03:46:37.863618+0000","last_became_peered":"2023-07-06T03:46:37.863618+0000","last_unstale":"2023-07-06T03:46:39.041686+0000","last_undegraded":"2023-07-06T03:46:39.041686+0000","last_fullsized":"2023-07-06T03:46:39.041686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:26.836331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041669+0000","last_change":"2023-07-06T03:46:37.864654+0000","last_active":"2023-07-06T03:46:39.041669+0000","last_peered":"2023-07-06T03:46:39.041669+0000","last_clean":"2023-07-06T03:46:39.041669+0000","last_became_active":"2023-07-06T03:46:37.864340+0000","last_became_peered":"2023-07-06T03:46:37.864340+0000","last_unstale":"2023-07-06T03:46:39.041669+0000","last_undegraded":"2023-07-06T03:46:39.041669+0000","last_fullsized":"2023-07-06T03:46:39.041669+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:28.578365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419184+0000","last_change":"2023-07-06T03:46:37.853945+0000","last_active":"2023-07-06T03:46:39.419184+0000","last_peered":"2023-07-06T03:46:39.419184+0000","last_clean":"2023-07-06T03:46:39.419184+0000","last_became_active":"2023-07-06T03:46:37.853672+0000","last_became_peered":"2023-07-06T03:46:37.853672+0000","last_unstale":"2023-07-06T03:46:39.419184+0000","last_undegraded":"2023-07-06T03:46:39.419184+0000","last_fullsized":"2023-07-06T03:46:39.419184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:46:28.663423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419858+0000","last_change":"2023-07-06T03:46:37.852498+0000","last_active":"2023-07-06T03:46:39.419858+0000","last_peered":"2023-07-06T03:46:39.419858+0000","last_clean":"2023-07-06T03:46:39.419858+0000","last_became_active":"2023-07-06T03:46:37.852155+0000","last_became_peered":"2023-07-06T03:46:37.852155+0000","last_unstale":"2023-07-06T03:46:39.419858+0000","last_undegraded":"2023-07-06T03:46:39.419858+0000","last_fullsized":"2023-07-06T03:46:39.419858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:40:57.846988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852750+0000","last_change":"2023-07-06T03:46:37.863217+0000","last_active":"2023-07-06T03:46:38.852750+0000","last_peered":"2023-07-06T03:46:38.852750+0000","last_clean":"2023-07-06T03:46:38.852750+0000","last_became_active":"2023-07-06T03:46:37.862977+0000","last_became_peered":"2023-07-06T03:46:37.862977+0000","last_unstale":"2023-07-06T03:46:38.852750+0000","last_undegraded":"2023-07-06T03:46:38.852750+0000","last_fullsized":"2023-07-06T03:46:38.852750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:16:10.345229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420251+0000","last_change":"2023-07-06T03:46:37.853812+0000","last_active":"2023-07-06T03:46:39.420251+0000","last_peered":"2023-07-06T03:46:39.420251+0000","last_clean":"2023-07-06T03:46:39.420251+0000","last_became_active":"2023-07-06T03:46:37.852250+0000","last_became_peered":"2023-07-06T03:46:37.852250+0000","last_unstale":"2023-07-06T03:46:39.420251+0000","last_undegraded":"2023-07-06T03:46:39.420251+0000","last_fullsized":"2023-07-06T03:46:39.420251+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:11:36.909511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.042092+0000","last_change":"2023-07-06T03:46:37.861070+0000","last_active":"2023-07-06T03:46:39.042092+0000","last_peered":"2023-07-06T03:46:39.042092+0000","last_clean":"2023-07-06T03:46:39.042092+0000","last_became_active":"2023-07-06T03:46:37.860738+0000","last_became_peered":"2023-07-06T03:46:37.860738+0000","last_unstale":"2023-07-06T03:46:39.042092+0000","last_undegraded":"2023-07-06T03:46:39.042092+0000","last_fullsized":"2023-07-06T03:46:39.042092+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:53:51.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418678+0000","last_change":"2023-07-06T03:46:37.848284+0000","last_active":"2023-07-06T03:46:39.418678+0000","last_peered":"2023-07-06T03:46:39.418678+0000","last_clean":"2023-07-06T03:46:39.418678+0000","last_became_active":"2023-07-06T03:46:37.848010+0000","last_became_peered":"2023-07-06T03:46:37.848010+0000","last_unstale":"2023-07-06T03:46:39.418678+0000","last_undegraded":"2023-07-06T03:46:39.418678+0000","last_fullsized":"2023-07-06T03:46:39.418678+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:22:19.485193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419230+0000","last_change":"2023-07-06T03:46:37.856579+0000","last_active":"2023-07-06T03:46:39.419230+0000","last_peered":"2023-07-06T03:46:39.419230+0000","last_clean":"2023-07-06T03:46:39.419230+0000","last_became_active":"2023-07-06T03:46:37.856331+0000","last_became_peered":"2023-07-06T03:46:37.856331+0000","last_unstale":"2023-07-06T03:46:39.419230+0000","last_undegraded":"2023-07-06T03:46:39.419230+0000","last_fullsized":"2023-07-06T03:46:39.419230+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:12:57.228030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419212+0000","last_change":"2023-07-06T03:46:37.858786+0000","last_active":"2023-07-06T03:46:39.419212+0000","last_peered":"2023-07-06T03:46:39.419212+0000","last_clean":"2023-07-06T03:46:39.419212+0000","last_became_active":"2023-07-06T03:46:37.858458+0000","last_became_peered":"2023-07-06T03:46:37.858458+0000","last_unstale":"2023-07-06T03:46:39.419212+0000","last_undegraded":"2023-07-06T03:46:39.419212+0000","last_fullsized":"2023-07-06T03:46:39.419212+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:31:04.637693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418659+0000","last_change":"2023-07-06T03:46:37.864684+0000","last_active":"2023-07-06T03:46:39.418659+0000","last_peered":"2023-07-06T03:46:39.418659+0000","last_clean":"2023-07-06T03:46:39.418659+0000","last_became_active":"2023-07-06T03:46:37.864342+0000","last_became_peered":"2023-07-06T03:46:37.864342+0000","last_unstale":"2023-07-06T03:46:39.418659+0000","last_undegraded":"2023-07-06T03:46:39.418659+0000","last_fullsized":"2023-07-06T03:46:39.418659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:12:55.269776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419718+0000","last_change":"2023-07-06T03:46:37.851130+0000","last_active":"2023-07-06T03:46:39.419718+0000","last_peered":"2023-07-06T03:46:39.419718+0000","last_clean":"2023-07-06T03:46:39.419718+0000","last_became_active":"2023-07-06T03:46:37.850783+0000","last_became_peered":"2023-07-06T03:46:37.850783+0000","last_unstale":"2023-07-06T03:46:39.419718+0000","last_undegraded":"2023-07-06T03:46:39.419718+0000","last_fullsized":"2023-07-06T03:46:39.419718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:00:21.691539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419611+0000","last_change":"2023-07-06T03:46:37.853339+0000","last_active":"2023-07-06T03:46:39.419611+0000","last_peered":"2023-07-06T03:46:39.419611+0000","last_clean":"2023-07-06T03:46:39.419611+0000","last_became_active":"2023-07-06T03:46:37.852923+0000","last_became_peered":"2023-07-06T03:46:37.852923+0000","last_unstale":"2023-07-06T03:46:39.419611+0000","last_undegraded":"2023-07-06T03:46:39.419611+0000","last_fullsized":"2023-07-06T03:46:39.419611+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:04.156561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852627+0000","last_change":"2023-07-06T03:46:36.854615+0000","last_active":"2023-07-06T03:46:38.852627+0000","last_peered":"2023-07-06T03:46:38.852627+0000","last_clean":"2023-07-06T03:46:38.852627+0000","last_became_active":"2023-07-06T03:46:36.854325+0000","last_became_peered":"2023-07-06T03:46:36.854325+0000","last_unstale":"2023-07-06T03:46:38.852627+0000","last_undegraded":"2023-07-06T03:46:38.852627+0000","last_fullsized":"2023-07-06T03:46:38.852627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:18:30.511652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420053+0000","last_change":"2023-07-06T03:46:37.865092+0000","last_active":"2023-07-06T03:46:39.420053+0000","last_peered":"2023-07-06T03:46:39.420053+0000","last_clean":"2023-07-06T03:46:39.420053+0000","last_became_active":"2023-07-06T03:46:37.864810+0000","last_became_peered":"2023-07-06T03:46:37.864810+0000","last_unstale":"2023-07-06T03:46:39.420053+0000","last_undegraded":"2023-07-06T03:46:39.420053+0000","last_fullsized":"2023-07-06T03:46:39.420053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:02:10.182517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419775+0000","last_change":"2023-07-06T03:46:36.857512+0000","last_active":"2023-07-06T03:46:39.419775+0000","last_peered":"2023-07-06T03:46:39.419775+0000","last_clean":"2023-07-06T03:46:39.419775+0000","last_became_active":"2023-07-06T03:46:36.857177+0000","last_became_peered":"2023-07-06T03:46:36.857177+0000","last_unstale":"2023-07-06T03:46:39.419775+0000","last_undegraded":"2023-07-06T03:46:39.419775+0000","last_fullsized":"2023-07-06T03:46:39.419775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:33:18.938068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419718+0000","last_change":"2023-07-06T03:46:37.854234+0000","last_active":"2023-07-06T03:46:39.419718+0000","last_peered":"2023-07-06T03:46:39.419718+0000","last_clean":"2023-07-06T03:46:39.419718+0000","last_became_active":"2023-07-06T03:46:37.853973+0000","last_became_peered":"2023-07-06T03:46:37.853973+0000","last_unstale":"2023-07-06T03:46:39.419718+0000","last_undegraded":"2023-07-06T03:46:39.419718+0000","last_fullsized":"2023-07-06T03:46:39.419718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:17:49.210528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420214+0000","last_change":"2023-07-06T03:46:36.860082+0000","last_active":"2023-07-06T03:46:39.420214+0000","last_peered":"2023-07-06T03:46:39.420214+0000","last_clean":"2023-07-06T03:46:39.420214+0000","last_became_active":"2023-07-06T03:46:36.859761+0000","last_became_peered":"2023-07-06T03:46:36.859761+0000","last_unstale":"2023-07-06T03:46:39.420214+0000","last_undegraded":"2023-07-06T03:46:39.420214+0000","last_fullsized":"2023-07-06T03:46:39.420214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:06:29.939063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419942+0000","last_change":"2023-07-06T03:46:37.861372+0000","last_active":"2023-07-06T03:46:39.419942+0000","last_peered":"2023-07-06T03:46:39.419942+0000","last_clean":"2023-07-06T03:46:39.419942+0000","last_became_active":"2023-07-06T03:46:37.861062+0000","last_became_peered":"2023-07-06T03:46:37.861062+0000","last_unstale":"2023-07-06T03:46:39.419942+0000","last_undegraded":"2023-07-06T03:46:39.419942+0000","last_fullsized":"2023-07-06T03:46:39.419942+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:53:51.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852628+0000","last_change":"2023-07-06T03:46:36.860395+0000","last_active":"2023-07-06T03:46:38.852628+0000","last_peered":"2023-07-06T03:46:38.852628+0000","last_clean":"2023-07-06T03:46:38.852628+0000","last_became_active":"2023-07-06T03:46:36.860128+0000","last_became_peered":"2023-07-06T03:46:36.860128+0000","last_unstale":"2023-07-06T03:46:38.852628+0000","last_undegraded":"2023-07-06T03:46:38.852628+0000","last_fullsized":"2023-07-06T03:46:38.852628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:24:21.195033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419703+0000","last_change":"2023-07-06T03:46:37.856482+0000","last_active":"2023-07-06T03:46:39.419703+0000","last_peered":"2023-07-06T03:46:39.419703+0000","last_clean":"2023-07-06T03:46:39.419703+0000","last_became_active":"2023-07-06T03:46:37.856161+0000","last_became_peered":"2023-07-06T03:46:37.856161+0000","last_unstale":"2023-07-06T03:46:39.419703+0000","last_undegraded":"2023-07-06T03:46:39.419703+0000","last_fullsized":"2023-07-06T03:46:39.419703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:36:54.027820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852417+0000","last_change":"2023-07-06T03:46:36.844040+0000","last_active":"2023-07-06T03:46:38.852417+0000","last_peered":"2023-07-06T03:46:38.852417+0000","last_clean":"2023-07-06T03:46:38.852417+0000","last_became_active":"2023-07-06T03:46:36.843778+0000","last_became_peered":"2023-07-06T03:46:36.843778+0000","last_unstale":"2023-07-06T03:46:38.852417+0000","last_undegraded":"2023-07-06T03:46:38.852417+0000","last_fullsized":"2023-07-06T03:46:38.852417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:33:57.954583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041627+0000","last_change":"2023-07-06T03:46:37.858452+0000","last_active":"2023-07-06T03:46:39.041627+0000","last_peered":"2023-07-06T03:46:39.041627+0000","last_clean":"2023-07-06T03:46:39.041627+0000","last_became_active":"2023-07-06T03:46:37.858175+0000","last_became_peered":"2023-07-06T03:46:37.858175+0000","last_unstale":"2023-07-06T03:46:39.041627+0000","last_undegraded":"2023-07-06T03:46:39.041627+0000","last_fullsized":"2023-07-06T03:46:39.041627+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:15.115431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418480+0000","last_change":"2023-07-06T03:46:36.860493+0000","last_active":"2023-07-06T03:46:39.418480+0000","last_peered":"2023-07-06T03:46:39.418480+0000","last_clean":"2023-07-06T03:46:39.418480+0000","last_became_active":"2023-07-06T03:46:36.860183+0000","last_became_peered":"2023-07-06T03:46:36.860183+0000","last_unstale":"2023-07-06T03:46:39.418480+0000","last_undegraded":"2023-07-06T03:46:39.418480+0000","last_fullsized":"2023-07-06T03:46:39.418480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:25:32.127906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419308+0000","last_change":"2023-07-06T03:46:37.863671+0000","last_active":"2023-07-06T03:46:39.419308+0000","last_peered":"2023-07-06T03:46:39.419308+0000","last_clean":"2023-07-06T03:46:39.419308+0000","last_became_active":"2023-07-06T03:46:37.863398+0000","last_became_peered":"2023-07-06T03:46:37.863398+0000","last_unstale":"2023-07-06T03:46:39.419308+0000","last_undegraded":"2023-07-06T03:46:39.419308+0000","last_fullsized":"2023-07-06T03:46:39.419308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:17:26.517544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419441+0000","last_change":"2023-07-06T03:46:36.858921+0000","last_active":"2023-07-06T03:46:39.419441+0000","last_peered":"2023-07-06T03:46:39.419441+0000","last_clean":"2023-07-06T03:46:39.419441+0000","last_became_active":"2023-07-06T03:46:36.858622+0000","last_became_peered":"2023-07-06T03:46:36.858622+0000","last_unstale":"2023-07-06T03:46:39.419441+0000","last_undegraded":"2023-07-06T03:46:39.419441+0000","last_fullsized":"2023-07-06T03:46:39.419441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:34.577488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419682+0000","last_change":"2023-07-06T03:46:37.863461+0000","last_active":"2023-07-06T03:46:39.419682+0000","last_peered":"2023-07-06T03:46:39.419682+0000","last_clean":"2023-07-06T03:46:39.419682+0000","last_became_active":"2023-07-06T03:46:37.862923+0000","last_became_peered":"2023-07-06T03:46:37.862923+0000","last_unstale":"2023-07-06T03:46:39.419682+0000","last_undegraded":"2023-07-06T03:46:39.419682+0000","last_fullsized":"2023-07-06T03:46:39.419682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:04.849320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418404+0000","last_change":"2023-07-06T03:46:36.853092+0000","last_active":"2023-07-06T03:46:39.418404+0000","last_peered":"2023-07-06T03:46:39.418404+0000","last_clean":"2023-07-06T03:46:39.418404+0000","last_became_active":"2023-07-06T03:46:36.852795+0000","last_became_peered":"2023-07-06T03:46:36.852795+0000","last_unstale":"2023-07-06T03:46:39.418404+0000","last_undegraded":"2023-07-06T03:46:39.418404+0000","last_fullsized":"2023-07-06T03:46:39.418404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:49:11.074324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420266+0000","last_change":"2023-07-06T03:46:37.865852+0000","last_active":"2023-07-06T03:46:39.420266+0000","last_peered":"2023-07-06T03:46:39.420266+0000","last_clean":"2023-07-06T03:46:39.420266+0000","last_became_active":"2023-07-06T03:46:37.864988+0000","last_became_peered":"2023-07-06T03:46:37.864988+0000","last_unstale":"2023-07-06T03:46:39.420266+0000","last_undegraded":"2023-07-06T03:46:39.420266+0000","last_fullsized":"2023-07-06T03:46:39.420266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:47:11.749793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420238+0000","last_change":"2023-07-06T03:46:36.854389+0000","last_active":"2023-07-06T03:46:39.420238+0000","last_peered":"2023-07-06T03:46:39.420238+0000","last_clean":"2023-07-06T03:46:39.420238+0000","last_became_active":"2023-07-06T03:46:36.854083+0000","last_became_peered":"2023-07-06T03:46:36.854083+0000","last_unstale":"2023-07-06T03:46:39.420238+0000","last_undegraded":"2023-07-06T03:46:39.420238+0000","last_fullsized":"2023-07-06T03:46:39.420238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:07:57.989913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852508+0000","last_change":"2023-07-06T03:46:37.863905+0000","last_active":"2023-07-06T03:46:38.852508+0000","last_peered":"2023-07-06T03:46:38.852508+0000","last_clean":"2023-07-06T03:46:38.852508+0000","last_became_active":"2023-07-06T03:46:37.863639+0000","last_became_peered":"2023-07-06T03:46:37.863639+0000","last_unstale":"2023-07-06T03:46:38.852508+0000","last_undegraded":"2023-07-06T03:46:38.852508+0000","last_fullsized":"2023-07-06T03:46:38.852508+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:29:08.680714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419465+0000","last_change":"2023-07-06T03:46:36.848400+0000","last_active":"2023-07-06T03:46:39.419465+0000","last_peered":"2023-07-06T03:46:39.419465+0000","last_clean":"2023-07-06T03:46:39.419465+0000","last_became_active":"2023-07-06T03:46:36.848127+0000","last_became_peered":"2023-07-06T03:46:36.848127+0000","last_unstale":"2023-07-06T03:46:39.419465+0000","last_undegraded":"2023-07-06T03:46:39.419465+0000","last_fullsized":"2023-07-06T03:46:39.419465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:30:34.301467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420013+0000","last_change":"2023-07-06T03:46:37.849656+0000","last_active":"2023-07-06T03:46:39.420013+0000","last_peered":"2023-07-06T03:46:39.420013+0000","last_clean":"2023-07-06T03:46:39.420013+0000","last_became_active":"2023-07-06T03:46:37.849395+0000","last_became_peered":"2023-07-06T03:46:37.849395+0000","last_unstale":"2023-07-06T03:46:39.420013+0000","last_undegraded":"2023-07-06T03:46:39.420013+0000","last_fullsized":"2023-07-06T03:46:39.420013+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:04:53.111652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419205+0000","last_change":"2023-07-06T03:46:36.860726+0000","last_active":"2023-07-06T03:46:39.419205+0000","last_peered":"2023-07-06T03:46:39.419205+0000","last_clean":"2023-07-06T03:46:39.419205+0000","last_became_active":"2023-07-06T03:46:36.859745+0000","last_became_peered":"2023-07-06T03:46:36.859745+0000","last_unstale":"2023-07-06T03:46:39.419205+0000","last_undegraded":"2023-07-06T03:46:39.419205+0000","last_fullsized":"2023-07-06T03:46:39.419205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:24:19.605298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418784+0000","last_change":"2023-07-06T03:46:37.845211+0000","last_active":"2023-07-06T03:46:39.418784+0000","last_peered":"2023-07-06T03:46:39.418784+0000","last_clean":"2023-07-06T03:46:39.418784+0000","last_became_active":"2023-07-06T03:46:37.844944+0000","last_became_peered":"2023-07-06T03:46:37.844944+0000","last_unstale":"2023-07-06T03:46:39.418784+0000","last_undegraded":"2023-07-06T03:46:39.418784+0000","last_fullsized":"2023-07-06T03:46:39.418784+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:01:33.509255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420347+0000","last_change":"2023-07-06T03:46:36.858688+0000","last_active":"2023-07-06T03:46:39.420347+0000","last_peered":"2023-07-06T03:46:39.420347+0000","last_clean":"2023-07-06T03:46:39.420347+0000","last_became_active":"2023-07-06T03:46:36.858031+0000","last_became_peered":"2023-07-06T03:46:36.858031+0000","last_unstale":"2023-07-06T03:46:39.420347+0000","last_undegraded":"2023-07-06T03:46:39.420347+0000","last_fullsized":"2023-07-06T03:46:39.420347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:39:49.641446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419557+0000","last_change":"2023-07-06T03:46:37.852588+0000","last_active":"2023-07-06T03:46:39.419557+0000","last_peered":"2023-07-06T03:46:39.419557+0000","last_clean":"2023-07-06T03:46:39.419557+0000","last_became_active":"2023-07-06T03:46:37.852256+0000","last_became_peered":"2023-07-06T03:46:37.852256+0000","last_unstale":"2023-07-06T03:46:39.419557+0000","last_undegraded":"2023-07-06T03:46:39.419557+0000","last_fullsized":"2023-07-06T03:46:39.419557+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:24:39.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419552+0000","last_change":"2023-07-06T03:46:36.859170+0000","last_active":"2023-07-06T03:46:39.419552+0000","last_peered":"2023-07-06T03:46:39.419552+0000","last_clean":"2023-07-06T03:46:39.419552+0000","last_became_active":"2023-07-06T03:46:36.858764+0000","last_became_peered":"2023-07-06T03:46:36.858764+0000","last_unstale":"2023-07-06T03:46:39.419552+0000","last_undegraded":"2023-07-06T03:46:39.419552+0000","last_fullsized":"2023-07-06T03:46:39.419552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:07:48.265940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419356+0000","last_change":"2023-07-06T03:46:37.865476+0000","last_active":"2023-07-06T03:46:39.419356+0000","last_peered":"2023-07-06T03:46:39.419356+0000","last_clean":"2023-07-06T03:46:39.419356+0000","last_became_active":"2023-07-06T03:46:37.865057+0000","last_became_peered":"2023-07-06T03:46:37.865057+0000","last_unstale":"2023-07-06T03:46:39.419356+0000","last_undegraded":"2023-07-06T03:46:39.419356+0000","last_fullsized":"2023-07-06T03:46:39.419356+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:28.578365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419534+0000","last_change":"2023-07-06T03:46:36.856232+0000","last_active":"2023-07-06T03:46:39.419534+0000","last_peered":"2023-07-06T03:46:39.419534+0000","last_clean":"2023-07-06T03:46:39.419534+0000","last_became_active":"2023-07-06T03:46:36.855057+0000","last_became_peered":"2023-07-06T03:46:36.855057+0000","last_unstale":"2023-07-06T03:46:39.419534+0000","last_undegraded":"2023-07-06T03:46:39.419534+0000","last_fullsized":"2023-07-06T03:46:39.419534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:24.299644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041626+0000","last_change":"2023-07-06T03:46:37.869250+0000","last_active":"2023-07-06T03:46:39.041626+0000","last_peered":"2023-07-06T03:46:39.041626+0000","last_clean":"2023-07-06T03:46:39.041626+0000","last_became_active":"2023-07-06T03:46:37.868585+0000","last_became_peered":"2023-07-06T03:46:37.868585+0000","last_unstale":"2023-07-06T03:46:39.041626+0000","last_undegraded":"2023-07-06T03:46:39.041626+0000","last_fullsized":"2023-07-06T03:46:39.041626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.598122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419252+0000","last_change":"2023-07-06T03:46:36.848489+0000","last_active":"2023-07-06T03:46:39.419252+0000","last_peered":"2023-07-06T03:46:39.419252+0000","last_clean":"2023-07-06T03:46:39.419252+0000","last_became_active":"2023-07-06T03:46:36.848195+0000","last_became_peered":"2023-07-06T03:46:36.848195+0000","last_unstale":"2023-07-06T03:46:39.419252+0000","last_undegraded":"2023-07-06T03:46:39.419252+0000","last_fullsized":"2023-07-06T03:46:39.419252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:49:58.368321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853875+0000","last_change":"2023-07-06T03:46:37.857742+0000","last_active":"2023-07-06T03:46:38.853875+0000","last_peered":"2023-07-06T03:46:38.853875+0000","last_clean":"2023-07-06T03:46:38.853875+0000","last_became_active":"2023-07-06T03:46:37.857431+0000","last_became_peered":"2023-07-06T03:46:37.857431+0000","last_unstale":"2023-07-06T03:46:38.853875+0000","last_undegraded":"2023-07-06T03:46:38.853875+0000","last_fullsized":"2023-07-06T03:46:38.853875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:44:39.103954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852415+0000","last_change":"2023-07-06T03:46:36.850609+0000","last_active":"2023-07-06T03:46:38.852415+0000","last_peered":"2023-07-06T03:46:38.852415+0000","last_clean":"2023-07-06T03:46:38.852415+0000","last_became_active":"2023-07-06T03:46:36.850316+0000","last_became_peered":"2023-07-06T03:46:36.850316+0000","last_unstale":"2023-07-06T03:46:38.852415+0000","last_undegraded":"2023-07-06T03:46:38.852415+0000","last_fullsized":"2023-07-06T03:46:38.852415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:06:25.318103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419909+0000","last_change":"2023-07-06T03:46:37.858603+0000","last_active":"2023-07-06T03:46:39.419909+0000","last_peered":"2023-07-06T03:46:39.419909+0000","last_clean":"2023-07-06T03:46:39.419909+0000","last_became_active":"2023-07-06T03:46:37.858343+0000","last_became_peered":"2023-07-06T03:46:37.858343+0000","last_unstale":"2023-07-06T03:46:39.419909+0000","last_undegraded":"2023-07-06T03:46:39.419909+0000","last_fullsized":"2023-07-06T03:46:39.419909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:52.391457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418737+0000","last_change":"2023-07-06T03:46:36.852893+0000","last_active":"2023-07-06T03:46:39.418737+0000","last_peered":"2023-07-06T03:46:39.418737+0000","last_clean":"2023-07-06T03:46:39.418737+0000","last_became_active":"2023-07-06T03:46:36.852559+0000","last_became_peered":"2023-07-06T03:46:36.852559+0000","last_unstale":"2023-07-06T03:46:39.418737+0000","last_undegraded":"2023-07-06T03:46:39.418737+0000","last_fullsized":"2023-07-06T03:46:39.418737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:50:39.638633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418795+0000","last_change":"2023-07-06T03:46:37.847891+0000","last_active":"2023-07-06T03:46:39.418795+0000","last_peered":"2023-07-06T03:46:39.418795+0000","last_clean":"2023-07-06T03:46:39.418795+0000","last_became_active":"2023-07-06T03:46:37.847633+0000","last_became_peered":"2023-07-06T03:46:37.847633+0000","last_unstale":"2023-07-06T03:46:39.418795+0000","last_undegraded":"2023-07-06T03:46:39.418795+0000","last_fullsized":"2023-07-06T03:46:39.418795+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:51:53.004246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419605+0000","last_change":"2023-07-06T03:46:36.861088+0000","last_active":"2023-07-06T03:46:39.419605+0000","last_peered":"2023-07-06T03:46:39.419605+0000","last_clean":"2023-07-06T03:46:39.419605+0000","last_became_active":"2023-07-06T03:46:36.860765+0000","last_became_peered":"2023-07-06T03:46:36.860765+0000","last_unstale":"2023-07-06T03:46:39.419605+0000","last_undegraded":"2023-07-06T03:46:39.419605+0000","last_fullsized":"2023-07-06T03:46:39.419605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:01:12.237971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418844+0000","last_change":"2023-07-06T03:46:37.865269+0000","last_active":"2023-07-06T03:46:39.418844+0000","last_peered":"2023-07-06T03:46:39.418844+0000","last_clean":"2023-07-06T03:46:39.418844+0000","last_became_active":"2023-07-06T03:46:37.864958+0000","last_became_peered":"2023-07-06T03:46:37.864958+0000","last_unstale":"2023-07-06T03:46:39.418844+0000","last_undegraded":"2023-07-06T03:46:39.418844+0000","last_fullsized":"2023-07-06T03:46:39.418844+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:24:42.365036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853575+0000","last_change":"2023-07-06T03:46:36.856749+0000","last_active":"2023-07-06T03:46:38.853575+0000","last_peered":"2023-07-06T03:46:38.853575+0000","last_clean":"2023-07-06T03:46:38.853575+0000","last_became_active":"2023-07-06T03:46:36.856419+0000","last_became_peered":"2023-07-06T03:46:36.856419+0000","last_unstale":"2023-07-06T03:46:38.853575+0000","last_undegraded":"2023-07-06T03:46:38.853575+0000","last_fullsized":"2023-07-06T03:46:38.853575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:03.842589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851579+0000","last_change":"2023-07-06T03:46:37.864167+0000","last_active":"2023-07-06T03:46:38.851579+0000","last_peered":"2023-07-06T03:46:38.851579+0000","last_clean":"2023-07-06T03:46:38.851579+0000","last_became_active":"2023-07-06T03:46:37.863865+0000","last_became_peered":"2023-07-06T03:46:37.863865+0000","last_unstale":"2023-07-06T03:46:38.851579+0000","last_undegraded":"2023-07-06T03:46:38.851579+0000","last_fullsized":"2023-07-06T03:46:38.851579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:04.849320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851484+0000","last_change":"2023-07-06T03:46:36.855799+0000","last_active":"2023-07-06T03:46:38.851484+0000","last_peered":"2023-07-06T03:46:38.851484+0000","last_clean":"2023-07-06T03:46:38.851484+0000","last_became_active":"2023-07-06T03:46:36.855494+0000","last_became_peered":"2023-07-06T03:46:36.855494+0000","last_unstale":"2023-07-06T03:46:38.851484+0000","last_undegraded":"2023-07-06T03:46:38.851484+0000","last_fullsized":"2023-07-06T03:46:38.851484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:24.299644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419061+0000","last_change":"2023-07-06T03:46:37.858828+0000","last_active":"2023-07-06T03:46:39.419061+0000","last_peered":"2023-07-06T03:46:39.419061+0000","last_clean":"2023-07-06T03:46:39.419061+0000","last_became_active":"2023-07-06T03:46:37.858546+0000","last_became_peered":"2023-07-06T03:46:37.858546+0000","last_unstale":"2023-07-06T03:46:39.419061+0000","last_undegraded":"2023-07-06T03:46:39.419061+0000","last_fullsized":"2023-07-06T03:46:39.419061+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:26:24.568746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851488+0000","last_change":"2023-07-06T03:46:36.849693+0000","last_active":"2023-07-06T03:46:38.851488+0000","last_peered":"2023-07-06T03:46:38.851488+0000","last_clean":"2023-07-06T03:46:38.851488+0000","last_became_active":"2023-07-06T03:46:36.849420+0000","last_became_peered":"2023-07-06T03:46:36.849420+0000","last_unstale":"2023-07-06T03:46:38.851488+0000","last_undegraded":"2023-07-06T03:46:38.851488+0000","last_fullsized":"2023-07-06T03:46:38.851488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:15.789105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419350+0000","last_change":"2023-07-06T03:46:37.858086+0000","last_active":"2023-07-06T03:46:39.419350+0000","last_peered":"2023-07-06T03:46:39.419350+0000","last_clean":"2023-07-06T03:46:39.419350+0000","last_became_active":"2023-07-06T03:46:37.857821+0000","last_became_peered":"2023-07-06T03:46:37.857821+0000","last_unstale":"2023-07-06T03:46:39.419350+0000","last_undegraded":"2023-07-06T03:46:39.419350+0000","last_fullsized":"2023-07-06T03:46:39.419350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:31:28.228023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419674+0000","last_change":"2023-07-06T03:46:36.852875+0000","last_active":"2023-07-06T03:46:39.419674+0000","last_peered":"2023-07-06T03:46:39.419674+0000","last_clean":"2023-07-06T03:46:39.419674+0000","last_became_active":"2023-07-06T03:46:36.852400+0000","last_became_peered":"2023-07-06T03:46:36.852400+0000","last_unstale":"2023-07-06T03:46:39.419674+0000","last_undegraded":"2023-07-06T03:46:39.419674+0000","last_fullsized":"2023-07-06T03:46:39.419674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:52.817628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041760+0000","last_change":"2023-07-06T03:46:37.869442+0000","last_active":"2023-07-06T03:46:39.041760+0000","last_peered":"2023-07-06T03:46:39.041760+0000","last_clean":"2023-07-06T03:46:39.041760+0000","last_became_active":"2023-07-06T03:46:37.869000+0000","last_became_peered":"2023-07-06T03:46:37.869000+0000","last_unstale":"2023-07-06T03:46:39.041760+0000","last_undegraded":"2023-07-06T03:46:39.041760+0000","last_fullsized":"2023-07-06T03:46:39.041760+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:16.795837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418545+0000","last_change":"2023-07-06T03:46:36.854933+0000","last_active":"2023-07-06T03:46:39.418545+0000","last_peered":"2023-07-06T03:46:39.418545+0000","last_clean":"2023-07-06T03:46:39.418545+0000","last_became_active":"2023-07-06T03:46:36.854557+0000","last_became_peered":"2023-07-06T03:46:36.854557+0000","last_unstale":"2023-07-06T03:46:39.418545+0000","last_undegraded":"2023-07-06T03:46:39.418545+0000","last_fullsized":"2023-07-06T03:46:39.418545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:39:46.623924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419773+0000","last_change":"2023-07-06T03:46:37.848797+0000","last_active":"2023-07-06T03:46:39.419773+0000","last_peered":"2023-07-06T03:46:39.419773+0000","last_clean":"2023-07-06T03:46:39.419773+0000","last_became_active":"2023-07-06T03:46:37.848516+0000","last_became_peered":"2023-07-06T03:46:37.848516+0000","last_unstale":"2023-07-06T03:46:39.419773+0000","last_undegraded":"2023-07-06T03:46:39.419773+0000","last_fullsized":"2023-07-06T03:46:39.419773+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:22:45.955750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419042+0000","last_change":"2023-07-06T03:46:36.861470+0000","last_active":"2023-07-06T03:46:39.419042+0000","last_peered":"2023-07-06T03:46:39.419042+0000","last_clean":"2023-07-06T03:46:39.419042+0000","last_became_active":"2023-07-06T03:46:36.861150+0000","last_became_peered":"2023-07-06T03:46:36.861150+0000","last_unstale":"2023-07-06T03:46:39.419042+0000","last_undegraded":"2023-07-06T03:46:39.419042+0000","last_fullsized":"2023-07-06T03:46:39.419042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:14.108700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853519+0000","last_change":"2023-07-06T03:46:37.859672+0000","last_active":"2023-07-06T03:46:38.853519+0000","last_peered":"2023-07-06T03:46:38.853519+0000","last_clean":"2023-07-06T03:46:38.853519+0000","last_became_active":"2023-07-06T03:46:37.859405+0000","last_became_peered":"2023-07-06T03:46:37.859405+0000","last_unstale":"2023-07-06T03:46:38.853519+0000","last_undegraded":"2023-07-06T03:46:38.853519+0000","last_fullsized":"2023-07-06T03:46:38.853519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:25.306376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419877+0000","last_change":"2023-07-06T03:46:36.854707+0000","last_active":"2023-07-06T03:46:39.419877+0000","last_peered":"2023-07-06T03:46:39.419877+0000","last_clean":"2023-07-06T03:46:39.419877+0000","last_became_active":"2023-07-06T03:46:36.854435+0000","last_became_peered":"2023-07-06T03:46:36.854435+0000","last_unstale":"2023-07-06T03:46:39.419877+0000","last_undegraded":"2023-07-06T03:46:39.419877+0000","last_fullsized":"2023-07-06T03:46:39.419877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:05:36.862857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851576+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:46:38.851576+0000","last_peered":"2023-07-06T03:46:38.851576+0000","last_clean":"2023-07-06T03:46:38.851576+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:46:38.851576+0000","last_undegraded":"2023-07-06T03:46:38.851576+0000","last_fullsized":"2023-07-06T03:46:38.851576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:17:21.111789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851549+0000","last_change":"2023-07-06T03:46:37.862798+0000","last_active":"2023-07-06T03:46:38.851549+0000","last_peered":"2023-07-06T03:46:38.851549+0000","last_clean":"2023-07-06T03:46:38.851549+0000","last_became_active":"2023-07-06T03:46:37.862509+0000","last_became_peered":"2023-07-06T03:46:37.862509+0000","last_unstale":"2023-07-06T03:46:38.851549+0000","last_undegraded":"2023-07-06T03:46:38.851549+0000","last_fullsized":"2023-07-06T03:46:38.851549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:26.836331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853722+0000","last_change":"2023-07-06T03:46:36.859132+0000","last_active":"2023-07-06T03:46:38.853722+0000","last_peered":"2023-07-06T03:46:38.853722+0000","last_clean":"2023-07-06T03:46:38.853722+0000","last_became_active":"2023-07-06T03:46:36.858862+0000","last_became_peered":"2023-07-06T03:46:36.858862+0000","last_unstale":"2023-07-06T03:46:38.853722+0000","last_undegraded":"2023-07-06T03:46:38.853722+0000","last_fullsized":"2023-07-06T03:46:38.853722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:51.384726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419437+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:46:39.419437+0000","last_peered":"2023-07-06T03:46:39.419437+0000","last_clean":"2023-07-06T03:46:39.419437+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:46:39.419437+0000","last_undegraded":"2023-07-06T03:46:39.419437+0000","last_fullsized":"2023-07-06T03:46:39.419437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:04:05.608173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041981+0000","last_change":"2023-07-06T03:46:37.870020+0000","last_active":"2023-07-06T03:46:39.041981+0000","last_peered":"2023-07-06T03:46:39.041981+0000","last_clean":"2023-07-06T03:46:39.041981+0000","last_became_active":"2023-07-06T03:46:37.869716+0000","last_became_peered":"2023-07-06T03:46:37.869716+0000","last_unstale":"2023-07-06T03:46:39.041981+0000","last_undegraded":"2023-07-06T03:46:39.041981+0000","last_fullsized":"2023-07-06T03:46:39.041981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:52.391457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853769+0000","last_change":"2023-07-06T03:46:36.857376+0000","last_active":"2023-07-06T03:46:38.853769+0000","last_peered":"2023-07-06T03:46:38.853769+0000","last_clean":"2023-07-06T03:46:38.853769+0000","last_became_active":"2023-07-06T03:46:36.856783+0000","last_became_peered":"2023-07-06T03:46:36.856783+0000","last_unstale":"2023-07-06T03:46:38.853769+0000","last_undegraded":"2023-07-06T03:46:38.853769+0000","last_fullsized":"2023-07-06T03:46:38.853769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:53:50.026476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853513+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:46:38.853513+0000","last_peered":"2023-07-06T03:46:38.853513+0000","last_clean":"2023-07-06T03:46:38.853513+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:46:38.853513+0000","last_undegraded":"2023-07-06T03:46:38.853513+0000","last_fullsized":"2023-07-06T03:46:38.853513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:31:47.232229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419801+0000","last_change":"2023-07-06T03:46:37.859320+0000","last_active":"2023-07-06T03:46:39.419801+0000","last_peered":"2023-07-06T03:46:39.419801+0000","last_clean":"2023-07-06T03:46:39.419801+0000","last_became_active":"2023-07-06T03:46:37.858687+0000","last_became_peered":"2023-07-06T03:46:37.858687+0000","last_unstale":"2023-07-06T03:46:39.419801+0000","last_undegraded":"2023-07-06T03:46:39.419801+0000","last_fullsized":"2023-07-06T03:46:39.419801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:53.443866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041830+0000","last_change":"2023-07-06T03:46:36.858102+0000","last_active":"2023-07-06T03:46:39.041830+0000","last_peered":"2023-07-06T03:46:39.041830+0000","last_clean":"2023-07-06T03:46:39.041830+0000","last_became_active":"2023-07-06T03:46:36.857729+0000","last_became_peered":"2023-07-06T03:46:36.857729+0000","last_unstale":"2023-07-06T03:46:39.041830+0000","last_undegraded":"2023-07-06T03:46:39.041830+0000","last_fullsized":"2023-07-06T03:46:39.041830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:52.817628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419309+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:46:39.419309+0000","last_peered":"2023-07-06T03:46:39.419309+0000","last_clean":"2023-07-06T03:46:39.419309+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:46:39.419309+0000","last_undegraded":"2023-07-06T03:46:39.419309+0000","last_fullsized":"2023-07-06T03:46:39.419309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:28:47.569266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418463+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:46:39.418463+0000","last_peered":"2023-07-06T03:46:39.418463+0000","last_clean":"2023-07-06T03:46:39.418463+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:46:39.418463+0000","last_undegraded":"2023-07-06T03:46:39.418463+0000","last_fullsized":"2023-07-06T03:46:39.418463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:08:09.570675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853891+0000","last_change":"2023-07-06T03:46:37.853652+0000","last_active":"2023-07-06T03:46:38.853891+0000","last_peered":"2023-07-06T03:46:38.853891+0000","last_clean":"2023-07-06T03:46:38.853891+0000","last_became_active":"2023-07-06T03:46:37.853323+0000","last_became_peered":"2023-07-06T03:46:37.853323+0000","last_unstale":"2023-07-06T03:46:38.853891+0000","last_undegraded":"2023-07-06T03:46:38.853891+0000","last_fullsized":"2023-07-06T03:46:38.853891+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:02:10.182517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419816+0000","last_change":"2023-07-06T03:46:36.847001+0000","last_active":"2023-07-06T03:46:39.419816+0000","last_peered":"2023-07-06T03:46:39.419816+0000","last_clean":"2023-07-06T03:46:39.419816+0000","last_became_active":"2023-07-06T03:46:36.846669+0000","last_became_peered":"2023-07-06T03:46:36.846669+0000","last_unstale":"2023-07-06T03:46:39.419816+0000","last_undegraded":"2023-07-06T03:46:39.419816+0000","last_fullsized":"2023-07-06T03:46:39.419816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:19.239209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419000+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:46:39.419000+0000","last_peered":"2023-07-06T03:46:39.419000+0000","last_clean":"2023-07-06T03:46:39.419000+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:46:39.419000+0000","last_undegraded":"2023-07-06T03:46:39.419000+0000","last_fullsized":"2023-07-06T03:46:39.419000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:26:05.045659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419530+0000","last_change":"2023-07-06T03:46:37.861516+0000","last_active":"2023-07-06T03:46:39.419530+0000","last_peered":"2023-07-06T03:46:39.419530+0000","last_clean":"2023-07-06T03:46:39.419530+0000","last_became_active":"2023-07-06T03:46:37.860991+0000","last_became_peered":"2023-07-06T03:46:37.860991+0000","last_unstale":"2023-07-06T03:46:39.419530+0000","last_undegraded":"2023-07-06T03:46:39.419530+0000","last_fullsized":"2023-07-06T03:46:39.419530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:26.836331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041886+0000","last_change":"2023-07-06T03:46:36.855134+0000","last_active":"2023-07-06T03:46:39.041886+0000","last_peered":"2023-07-06T03:46:39.041886+0000","last_clean":"2023-07-06T03:46:39.041886+0000","last_became_active":"2023-07-06T03:46:36.854885+0000","last_became_peered":"2023-07-06T03:46:36.854885+0000","last_unstale":"2023-07-06T03:46:39.041886+0000","last_undegraded":"2023-07-06T03:46:39.041886+0000","last_fullsized":"2023-07-06T03:46:39.041886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:33:18.938068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419516+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:46:39.419516+0000","last_peered":"2023-07-06T03:46:39.419516+0000","last_clean":"2023-07-06T03:46:39.419516+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:46:39.419516+0000","last_undegraded":"2023-07-06T03:46:39.419516+0000","last_fullsized":"2023-07-06T03:46:39.419516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:36:14.721099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419354+0000","last_change":"2023-07-06T03:46:37.861288+0000","last_active":"2023-07-06T03:46:39.419354+0000","last_peered":"2023-07-06T03:46:39.419354+0000","last_clean":"2023-07-06T03:46:39.419354+0000","last_became_active":"2023-07-06T03:46:37.860989+0000","last_became_peered":"2023-07-06T03:46:37.860989+0000","last_unstale":"2023-07-06T03:46:39.419354+0000","last_undegraded":"2023-07-06T03:46:39.419354+0000","last_fullsized":"2023-07-06T03:46:39.419354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:46:55.890372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419276+0000","last_change":"2023-07-06T03:46:36.847922+0000","last_active":"2023-07-06T03:46:39.419276+0000","last_peered":"2023-07-06T03:46:39.419276+0000","last_clean":"2023-07-06T03:46:39.419276+0000","last_became_active":"2023-07-06T03:46:36.847302+0000","last_became_peered":"2023-07-06T03:46:36.847302+0000","last_unstale":"2023-07-06T03:46:39.419276+0000","last_undegraded":"2023-07-06T03:46:39.419276+0000","last_fullsized":"2023-07-06T03:46:39.419276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:14:40.997856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420277+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:46:39.420277+0000","last_peered":"2023-07-06T03:46:39.420277+0000","last_clean":"2023-07-06T03:46:39.420277+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:46:39.420277+0000","last_undegraded":"2023-07-06T03:46:39.420277+0000","last_fullsized":"2023-07-06T03:46:39.420277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:51:20.288814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041768+0000","last_change":"2023-07-06T03:46:37.862349+0000","last_active":"2023-07-06T03:46:39.041768+0000","last_peered":"2023-07-06T03:46:39.041768+0000","last_clean":"2023-07-06T03:46:39.041768+0000","last_became_active":"2023-07-06T03:46:37.862010+0000","last_became_peered":"2023-07-06T03:46:37.862010+0000","last_unstale":"2023-07-06T03:46:39.041768+0000","last_undegraded":"2023-07-06T03:46:39.041768+0000","last_fullsized":"2023-07-06T03:46:39.041768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:04.849320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419332+0000","last_change":"2023-07-06T03:46:36.861193+0000","last_active":"2023-07-06T03:46:39.419332+0000","last_peered":"2023-07-06T03:46:39.419332+0000","last_clean":"2023-07-06T03:46:39.419332+0000","last_became_active":"2023-07-06T03:46:36.860876+0000","last_became_peered":"2023-07-06T03:46:36.860876+0000","last_unstale":"2023-07-06T03:46:39.419332+0000","last_undegraded":"2023-07-06T03:46:39.419332+0000","last_fullsized":"2023-07-06T03:46:39.419332+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:24:46.275089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419845+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:46:39.419845+0000","last_peered":"2023-07-06T03:46:39.419845+0000","last_clean":"2023-07-06T03:46:39.419845+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:46:39.419845+0000","last_undegraded":"2023-07-06T03:46:39.419845+0000","last_fullsized":"2023-07-06T03:46:39.419845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:15:52.413417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":156752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.8660000000000001,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.77,"5min":3.77,"15min":3.77},"last":0.622},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.90300000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.81999999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.92500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":156716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.622}]},{"osd":1,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.36299999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.55200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":983040,"data_stored":747072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":13.055999999999999,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.503}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":12.989000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":13.023999999999999,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.874}]},{"osd":6,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.488,"5min":1.488,"15min":1.488},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":12.941000000000001,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.751}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.93000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":393216,"data_stored":156774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.65800000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":9.5960000000000001,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":158828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.628}]},{"osd":5,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71699999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.80200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":156772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":5.6550000000000002,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.442},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.498}]},{"osd":1,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":15.911,"5min":15.911,"15min":15.911},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":16.024000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":16.068999999999999,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":5.7300000000000004,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.73399999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.077,"5min":5.077,"15min":5.077},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":5.6980000000000004,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":5.157,"5min":5.157,"15min":5.157},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":5.1189999999999998,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.93000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":158370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":6.258,"5min":6.258,"15min":6.258},"last":0.496},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":6.4740000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":6.4859999999999998,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.74199999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.83899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":978944,"data_stored":746613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":6.4059999999999997,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.624}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.90600000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.879}]},{"osd":7,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.83199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:46:47.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:46:47.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.360+0000 7fd6131ff700 1 Processor -- start 2023-07-06T03:46:47.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.360+0000 7fd6131ff700 1 -- start start 2023-07-06T03:46:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.361+0000 7fd6131ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 0x7fd60c160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.361+0000 7fd6131ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd60c160dd0 0x7fd60c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.361+0000 7fd6131ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60c0cdae0 con 0x7fd60c15c520 2023-07-06T03:46:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.361+0000 7fd6131ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd60c0b6640 con 0x7fd60c160dd0 2023-07-06T03:46:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.361+0000 7fd6131ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c0b6920 con 0x7fd60c0cd390 2023-07-06T03:46:47.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.362+0000 7fd60bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 0x7fd60c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.362+0000 7fd60bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 0x7fd60c160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:37896/0 (socket says 172.21.15.97:37896) 2023-07-06T03:46:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.362+0000 7fd61179c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd60c160dd0 0x7fd60c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.362+0000 7fd60bfff700 1 -- 172.21.15.97:0/2943071790 learned_addr learned my addr 172.21.15.97:0/2943071790 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:47.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.362+0000 7fd610f9b700 1 --1- 172.21.15.97:0/2943071790 >> v1:172.21.15.146:6789/0 conn(0x7fd60c15c520 0x7fd60c160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:48810/0 (socket says 172.21.15.97:48810) 2023-07-06T03:46:47.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.362+0000 7fd60b7fe700 1 -- 172.21.15.97:0/2943071790 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803855802 0 0) 0x7fd60c0cdae0 con 0x7fd60c15c520 2023-07-06T03:46:47.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.362+0000 7fd60bfff700 1 -- 172.21.15.97:0/2943071790 >> v2:172.21.15.146:3300/0 conn(0x7fd60c160dd0 msgr2=0x7fd60c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:47.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.362+0000 7fd60bfff700 1 --2- 172.21.15.97:0/2943071790 >> v2:172.21.15.146:3300/0 conn(0x7fd60c160dd0 0x7fd60c0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.362+0000 7fd60bfff700 1 -- 172.21.15.97:0/2943071790 >> v1:172.21.15.146:6789/0 conn(0x7fd60c15c520 legacy=0x7fd60c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:47.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.362+0000 7fd60bfff700 1 -- 172.21.15.97:0/2943071790 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c0b6c00 con 0x7fd60c0cd390 2023-07-06T03:46:47.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.362+0000 7fd60bfff700 1 --2- 172.21.15.97:0/2943071790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 0x7fd60c160890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80098f0 tx=0x7fd5f8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0004064a1095ffc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.363+0000 7fd60b7fe700 1 -- 172.21.15.97:0/2943071790 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8011830 con 0x7fd60c0cd390 2023-07-06T03:46:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.363+0000 7fd60b7fe700 1 -- 172.21.15.97:0/2943071790 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f8011990 con 0x7fd60c0cd390 2023-07-06T03:46:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.363+0000 7fd60b7fe700 1 -- 172.21.15.97:0/2943071790 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8011d20 con 0x7fd60c0cd390 2023-07-06T03:46:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.363+0000 7fd6131ff700 1 -- 172.21.15.97:0/2943071790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 msgr2=0x7fd60c160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.363+0000 7fd6131ff700 1 --2- 172.21.15.97:0/2943071790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 0x7fd60c160890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80098f0 tx=0x7fd5f8009c00 comp rx=0 tx=0).stop 2023-07-06T03:46:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.363+0000 7fd6131ff700 1 -- 172.21.15.97:0/2943071790 shutdown_connections 2023-07-06T03:46:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.363+0000 7fd6131ff700 1 --2- 172.21.15.97:0/2943071790 >> v2:172.21.15.146:3300/0 conn(0x7fd60c160dd0 0x7fd60c0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.363+0000 7fd6131ff700 1 --2- 172.21.15.97:0/2943071790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 0x7fd60c160890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.363+0000 7fd6131ff700 1 -- 172.21.15.97:0/2943071790 >> 172.21.15.97:0/2943071790 conn(0x7fd60c071790 msgr2=0x7fd60c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.363+0000 7fd6131ff700 1 -- 172.21.15.97:0/2943071790 shutdown_connections 2023-07-06T03:46:47.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.363+0000 7fd6131ff700 1 -- 172.21.15.97:0/2943071790 wait complete. 2023-07-06T03:46:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.365+0000 7fd6131ff700 1 Processor -- start 2023-07-06T03:46:47.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.365+0000 7fd6131ff700 1 -- start start 2023-07-06T03:46:47.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.366+0000 7fd6131ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 0x7fd60c15f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.366+0000 7fd6131ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd60c15c520 0x7fd60c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.366+0000 7fd6131ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd60c160dd0 0x7fd60c0b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.366+0000 7fd6131ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c1f65e0 con 0x7fd60c160dd0 2023-07-06T03:46:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.366+0000 7fd6131ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c15fad0 con 0x7fd60c15c520 2023-07-06T03:46:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.366+0000 7fd6131ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c0d2a40 con 0x7fd60c0cd390 2023-07-06T03:46:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.366+0000 7fd61179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd60c160dd0 0x7fd60c0b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.366+0000 7fd61179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd60c160dd0 0x7fd60c0b3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58186/0 (socket says 172.21.15.97:58186) 2023-07-06T03:46:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.366+0000 7fd61179c700 1 -- 172.21.15.97:0/3848254914 learned_addr learned my addr 172.21.15.97:0/3848254914 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd610f9b700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 0x7fd60c15f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd60bfff700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd60c15c520 0x7fd60c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd61179c700 1 -- 172.21.15.97:0/3848254914 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 msgr2=0x7fd60c15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd61179c700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 0x7fd60c15f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd61179c700 1 -- 172.21.15.97:0/3848254914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd60c15c520 msgr2=0x7fd60c0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd61179c700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd60c15c520 0x7fd60c0b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd61179c700 1 -- 172.21.15.97:0/3848254914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f8009550 con 0x7fd60c160dd0 2023-07-06T03:46:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd610f9b700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 0x7fd60c15f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd61179c700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd60c160dd0 0x7fd60c0b3bd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd60400efc0 tx=0x7fd60400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd6097fa700 1 -- 172.21.15.97:0/3848254914 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60400cd40 con 0x7fd60c160dd0 2023-07-06T03:46:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd6097fa700 1 -- 172.21.15.97:0/3848254914 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60400f040 con 0x7fd60c160dd0 2023-07-06T03:46:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd6131ff700 1 -- 172.21.15.97:0/3848254914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60c0d2ce0 con 0x7fd60c160dd0 2023-07-06T03:46:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.367+0000 7fd6097fa700 1 -- 172.21.15.97:0/3848254914 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6040146c0 con 0x7fd60c160dd0 2023-07-06T03:46:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.368+0000 7fd6131ff700 1 -- 172.21.15.97:0/3848254914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60c0d32a0 con 0x7fd60c160dd0 2023-07-06T03:46:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.369+0000 7fd6131ff700 1 -- 172.21.15.97:0/3848254914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd60c04e340 con 0x7fd60c160dd0 2023-07-06T03:46:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.370+0000 7fd6097fa700 1 -- 172.21.15.97:0/3848254914 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd604014820 con 0x7fd60c160dd0 2023-07-06T03:46:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.371+0000 7fd6097fa700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd5f006ace0 0x7fd5f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.371+0000 7fd6097fa700 1 -- 172.21.15.97:0/3848254914 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd604089d50 con 0x7fd60c160dd0 2023-07-06T03:46:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.372+0000 7fd610f9b700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd5f006ace0 0x7fd5f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.372+0000 7fd610f9b700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd5f006ace0 0x7fd5f006d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd600000f60 tx=0x7fd600007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.376+0000 7fd6097fa700 1 -- 172.21.15.97:0/3848254914 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd60408d030 con 0x7fd60c160dd0 2023-07-06T03:46:47.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.568+0000 7fd6131ff700 1 -- 172.21.15.97:0/3848254914 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd60c0cd800 con 0x7fd5f006ace0 2023-07-06T03:46:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.577+0000 7fd6097fa700 1 -- 172.21.15.97:0/3848254914 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310772 (secure 0 0 0) 0x7fd600005bb0 con 0x7fd5f006ace0 2023-07-06T03:46:47.582 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 -- 172.21.15.97:0/3848254914 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd5f006ace0 msgr2=0x7fd5f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd5f006ace0 0x7fd5f006d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd600000f60 tx=0x7fd600007ca0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 -- 172.21.15.97:0/3848254914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd60c160dd0 msgr2=0x7fd60c0b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd60c160dd0 0x7fd60c0b3bd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd60400efc0 tx=0x7fd60400c3a0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 -- 172.21.15.97:0/3848254914 shutdown_connections 2023-07-06T03:46:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd5f006ace0 0x7fd5f006d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd60c160dd0 0x7fd60c0b3bd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd60c0cd390 0x7fd60c15f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 --2- 172.21.15.97:0/3848254914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd60c15c520 0x7fd60c0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 -- 172.21.15.97:0/3848254914 >> 172.21.15.97:0/3848254914 conn(0x7fd60c071790 msgr2=0x7fd60c0b8250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 -- 172.21.15.97:0/3848254914 shutdown_connections 2023-07-06T03:46:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.581+0000 7fd6131ff700 1 -- 172.21.15.97:0/3848254914 wait complete. 2023-07-06T03:46:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:46:47.601 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-07-06T03:46:46.901079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219844,"kb_used_data":4228,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754876,"statfs":{"total":773094113280,"available":772868993024,"internally_reserved":0,"allocated":4329472,"data_stored":2437897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.065511"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419924+0000","last_change":"2023-07-06T03:46:37.856906+0000","last_active":"2023-07-06T03:46:39.419924+0000","last_peered":"2023-07-06T03:46:39.419924+0000","last_clean":"2023-07-06T03:46:39.419924+0000","last_became_active":"2023-07-06T03:46:37.856640+0000","last_became_peered":"2023-07-06T03:46:37.856640+0000","last_unstale":"2023-07-06T03:46:39.419924+0000","last_undegraded":"2023-07-06T03:46:39.419924+0000","last_fullsized":"2023-07-06T03:46:39.419924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:00:51.494589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419658+0000","last_change":"2023-07-06T03:46:37.857685+0000","last_active":"2023-07-06T03:46:39.419658+0000","last_peered":"2023-07-06T03:46:39.419658+0000","last_clean":"2023-07-06T03:46:39.419658+0000","last_became_active":"2023-07-06T03:46:37.857373+0000","last_became_peered":"2023-07-06T03:46:37.857373+0000","last_unstale":"2023-07-06T03:46:39.419658+0000","last_undegraded":"2023-07-06T03:46:39.419658+0000","last_fullsized":"2023-07-06T03:46:39.419658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:16.795837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420131+0000","last_change":"2023-07-06T03:46:37.862846+0000","last_active":"2023-07-06T03:46:39.420131+0000","last_peered":"2023-07-06T03:46:39.420131+0000","last_clean":"2023-07-06T03:46:39.420131+0000","last_became_active":"2023-07-06T03:46:37.862554+0000","last_became_peered":"2023-07-06T03:46:37.862554+0000","last_unstale":"2023-07-06T03:46:39.420131+0000","last_undegraded":"2023-07-06T03:46:39.420131+0000","last_fullsized":"2023-07-06T03:46:39.420131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:08:02.995580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420078+0000","last_change":"2023-07-06T03:46:37.866028+0000","last_active":"2023-07-06T03:46:39.420078+0000","last_peered":"2023-07-06T03:46:39.420078+0000","last_clean":"2023-07-06T03:46:39.420078+0000","last_became_active":"2023-07-06T03:46:37.865346+0000","last_became_peered":"2023-07-06T03:46:37.865346+0000","last_unstale":"2023-07-06T03:46:39.420078+0000","last_undegraded":"2023-07-06T03:46:39.420078+0000","last_fullsized":"2023-07-06T03:46:39.420078+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:59:06.270128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853587+0000","last_change":"2023-07-06T03:46:37.864390+0000","last_active":"2023-07-06T03:46:38.853587+0000","last_peered":"2023-07-06T03:46:38.853587+0000","last_clean":"2023-07-06T03:46:38.853587+0000","last_became_active":"2023-07-06T03:46:37.864064+0000","last_became_peered":"2023-07-06T03:46:37.864064+0000","last_unstale":"2023-07-06T03:46:38.853587+0000","last_undegraded":"2023-07-06T03:46:38.853587+0000","last_fullsized":"2023-07-06T03:46:38.853587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:16.795837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419343+0000","last_change":"2023-07-06T03:46:37.861836+0000","last_active":"2023-07-06T03:46:39.419343+0000","last_peered":"2023-07-06T03:46:39.419343+0000","last_clean":"2023-07-06T03:46:39.419343+0000","last_became_active":"2023-07-06T03:46:37.861568+0000","last_became_peered":"2023-07-06T03:46:37.861568+0000","last_unstale":"2023-07-06T03:46:39.419343+0000","last_undegraded":"2023-07-06T03:46:39.419343+0000","last_fullsized":"2023-07-06T03:46:39.419343+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.598122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418487+0000","last_change":"2023-07-06T03:46:37.844697+0000","last_active":"2023-07-06T03:46:39.418487+0000","last_peered":"2023-07-06T03:46:39.418487+0000","last_clean":"2023-07-06T03:46:39.418487+0000","last_became_active":"2023-07-06T03:46:37.844412+0000","last_became_peered":"2023-07-06T03:46:37.844412+0000","last_unstale":"2023-07-06T03:46:39.418487+0000","last_undegraded":"2023-07-06T03:46:39.418487+0000","last_fullsized":"2023-07-06T03:46:39.418487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:40:38.338794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419209+0000","last_change":"2023-07-06T03:46:37.851682+0000","last_active":"2023-07-06T03:46:39.419209+0000","last_peered":"2023-07-06T03:46:39.419209+0000","last_clean":"2023-07-06T03:46:39.419209+0000","last_became_active":"2023-07-06T03:46:37.851342+0000","last_became_peered":"2023-07-06T03:46:37.851342+0000","last_unstale":"2023-07-06T03:46:39.419209+0000","last_undegraded":"2023-07-06T03:46:39.419209+0000","last_fullsized":"2023-07-06T03:46:39.419209+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:31:39.275779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.854169+0000","last_change":"2023-07-06T03:46:37.860538+0000","last_active":"2023-07-06T03:46:38.854169+0000","last_peered":"2023-07-06T03:46:38.854169+0000","last_clean":"2023-07-06T03:46:38.854169+0000","last_became_active":"2023-07-06T03:46:37.860253+0000","last_became_peered":"2023-07-06T03:46:37.860253+0000","last_unstale":"2023-07-06T03:46:38.854169+0000","last_undegraded":"2023-07-06T03:46:38.854169+0000","last_fullsized":"2023-07-06T03:46:38.854169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:40:38.338794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418842+0000","last_change":"2023-07-06T03:46:37.861893+0000","last_active":"2023-07-06T03:46:39.418842+0000","last_peered":"2023-07-06T03:46:39.418842+0000","last_clean":"2023-07-06T03:46:39.418842+0000","last_became_active":"2023-07-06T03:46:37.861562+0000","last_became_peered":"2023-07-06T03:46:37.861562+0000","last_unstale":"2023-07-06T03:46:39.418842+0000","last_undegraded":"2023-07-06T03:46:39.418842+0000","last_fullsized":"2023-07-06T03:46:39.418842+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:10:32.090820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.042093+0000","last_change":"2023-07-06T03:46:37.869263+0000","last_active":"2023-07-06T03:46:39.042093+0000","last_peered":"2023-07-06T03:46:39.042093+0000","last_clean":"2023-07-06T03:46:39.042093+0000","last_became_active":"2023-07-06T03:46:37.868621+0000","last_became_peered":"2023-07-06T03:46:37.868621+0000","last_unstale":"2023-07-06T03:46:39.042093+0000","last_undegraded":"2023-07-06T03:46:39.042093+0000","last_fullsized":"2023-07-06T03:46:39.042093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:02:10.182517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853685+0000","last_change":"2023-07-06T03:46:37.858638+0000","last_active":"2023-07-06T03:46:38.853685+0000","last_peered":"2023-07-06T03:46:38.853685+0000","last_clean":"2023-07-06T03:46:38.853685+0000","last_became_active":"2023-07-06T03:46:37.858354+0000","last_became_peered":"2023-07-06T03:46:37.858354+0000","last_unstale":"2023-07-06T03:46:38.853685+0000","last_undegraded":"2023-07-06T03:46:38.853685+0000","last_fullsized":"2023-07-06T03:46:38.853685+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:53.824359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418613+0000","last_change":"2023-07-06T03:46:37.864649+0000","last_active":"2023-07-06T03:46:39.418613+0000","last_peered":"2023-07-06T03:46:39.418613+0000","last_clean":"2023-07-06T03:46:39.418613+0000","last_became_active":"2023-07-06T03:46:37.864204+0000","last_became_peered":"2023-07-06T03:46:37.864204+0000","last_unstale":"2023-07-06T03:46:39.418613+0000","last_undegraded":"2023-07-06T03:46:39.418613+0000","last_fullsized":"2023-07-06T03:46:39.418613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:29:38.817047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852484+0000","last_change":"2023-07-06T03:46:37.862356+0000","last_active":"2023-07-06T03:46:38.852484+0000","last_peered":"2023-07-06T03:46:38.852484+0000","last_clean":"2023-07-06T03:46:38.852484+0000","last_became_active":"2023-07-06T03:46:37.862123+0000","last_became_peered":"2023-07-06T03:46:37.862123+0000","last_unstale":"2023-07-06T03:46:38.852484+0000","last_undegraded":"2023-07-06T03:46:38.852484+0000","last_fullsized":"2023-07-06T03:46:38.852484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:37:12.375776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419159+0000","last_change":"2023-07-06T03:46:37.859689+0000","last_active":"2023-07-06T03:46:39.419159+0000","last_peered":"2023-07-06T03:46:39.419159+0000","last_clean":"2023-07-06T03:46:39.419159+0000","last_became_active":"2023-07-06T03:46:37.859356+0000","last_became_peered":"2023-07-06T03:46:37.859356+0000","last_unstale":"2023-07-06T03:46:39.419159+0000","last_undegraded":"2023-07-06T03:46:39.419159+0000","last_fullsized":"2023-07-06T03:46:39.419159+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:44.223992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418293+0000","last_change":"2023-07-06T03:46:37.860022+0000","last_active":"2023-07-06T03:46:39.418293+0000","last_peered":"2023-07-06T03:46:39.418293+0000","last_clean":"2023-07-06T03:46:39.418293+0000","last_became_active":"2023-07-06T03:46:37.859715+0000","last_became_peered":"2023-07-06T03:46:37.859715+0000","last_unstale":"2023-07-06T03:46:39.418293+0000","last_undegraded":"2023-07-06T03:46:39.418293+0000","last_fullsized":"2023-07-06T03:46:39.418293+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:44:39.103954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419760+0000","last_change":"2023-07-06T03:46:37.861889+0000","last_active":"2023-07-06T03:46:39.419760+0000","last_peered":"2023-07-06T03:46:39.419760+0000","last_clean":"2023-07-06T03:46:39.419760+0000","last_became_active":"2023-07-06T03:46:37.860756+0000","last_became_peered":"2023-07-06T03:46:37.860756+0000","last_unstale":"2023-07-06T03:46:39.419760+0000","last_undegraded":"2023-07-06T03:46:39.419760+0000","last_fullsized":"2023-07-06T03:46:39.419760+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:50:46.498928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419563+0000","last_change":"2023-07-06T03:46:37.860737+0000","last_active":"2023-07-06T03:46:39.419563+0000","last_peered":"2023-07-06T03:46:39.419563+0000","last_clean":"2023-07-06T03:46:39.419563+0000","last_became_active":"2023-07-06T03:46:37.860452+0000","last_became_peered":"2023-07-06T03:46:37.860452+0000","last_unstale":"2023-07-06T03:46:39.419563+0000","last_undegraded":"2023-07-06T03:46:39.419563+0000","last_fullsized":"2023-07-06T03:46:39.419563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:02:50.812126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851546+0000","last_change":"2023-07-06T03:46:37.864336+0000","last_active":"2023-07-06T03:46:38.851546+0000","last_peered":"2023-07-06T03:46:38.851546+0000","last_clean":"2023-07-06T03:46:38.851546+0000","last_became_active":"2023-07-06T03:46:37.863953+0000","last_became_peered":"2023-07-06T03:46:37.863953+0000","last_unstale":"2023-07-06T03:46:38.851546+0000","last_undegraded":"2023-07-06T03:46:38.851546+0000","last_fullsized":"2023-07-06T03:46:38.851546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:28.578365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041710+0000","last_change":"2023-07-06T03:46:37.862894+0000","last_active":"2023-07-06T03:46:39.041710+0000","last_peered":"2023-07-06T03:46:39.041710+0000","last_clean":"2023-07-06T03:46:39.041710+0000","last_became_active":"2023-07-06T03:46:37.862643+0000","last_became_peered":"2023-07-06T03:46:37.862643+0000","last_unstale":"2023-07-06T03:46:39.041710+0000","last_undegraded":"2023-07-06T03:46:39.041710+0000","last_fullsized":"2023-07-06T03:46:39.041710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:25.306376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041686+0000","last_change":"2023-07-06T03:46:37.863993+0000","last_active":"2023-07-06T03:46:39.041686+0000","last_peered":"2023-07-06T03:46:39.041686+0000","last_clean":"2023-07-06T03:46:39.041686+0000","last_became_active":"2023-07-06T03:46:37.863618+0000","last_became_peered":"2023-07-06T03:46:37.863618+0000","last_unstale":"2023-07-06T03:46:39.041686+0000","last_undegraded":"2023-07-06T03:46:39.041686+0000","last_fullsized":"2023-07-06T03:46:39.041686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:26.836331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041669+0000","last_change":"2023-07-06T03:46:37.864654+0000","last_active":"2023-07-06T03:46:39.041669+0000","last_peered":"2023-07-06T03:46:39.041669+0000","last_clean":"2023-07-06T03:46:39.041669+0000","last_became_active":"2023-07-06T03:46:37.864340+0000","last_became_peered":"2023-07-06T03:46:37.864340+0000","last_unstale":"2023-07-06T03:46:39.041669+0000","last_undegraded":"2023-07-06T03:46:39.041669+0000","last_fullsized":"2023-07-06T03:46:39.041669+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:28.578365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419184+0000","last_change":"2023-07-06T03:46:37.853945+0000","last_active":"2023-07-06T03:46:39.419184+0000","last_peered":"2023-07-06T03:46:39.419184+0000","last_clean":"2023-07-06T03:46:39.419184+0000","last_became_active":"2023-07-06T03:46:37.853672+0000","last_became_peered":"2023-07-06T03:46:37.853672+0000","last_unstale":"2023-07-06T03:46:39.419184+0000","last_undegraded":"2023-07-06T03:46:39.419184+0000","last_fullsized":"2023-07-06T03:46:39.419184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:46:28.663423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419858+0000","last_change":"2023-07-06T03:46:37.852498+0000","last_active":"2023-07-06T03:46:39.419858+0000","last_peered":"2023-07-06T03:46:39.419858+0000","last_clean":"2023-07-06T03:46:39.419858+0000","last_became_active":"2023-07-06T03:46:37.852155+0000","last_became_peered":"2023-07-06T03:46:37.852155+0000","last_unstale":"2023-07-06T03:46:39.419858+0000","last_undegraded":"2023-07-06T03:46:39.419858+0000","last_fullsized":"2023-07-06T03:46:39.419858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:40:57.846988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852750+0000","last_change":"2023-07-06T03:46:37.863217+0000","last_active":"2023-07-06T03:46:38.852750+0000","last_peered":"2023-07-06T03:46:38.852750+0000","last_clean":"2023-07-06T03:46:38.852750+0000","last_became_active":"2023-07-06T03:46:37.862977+0000","last_became_peered":"2023-07-06T03:46:37.862977+0000","last_unstale":"2023-07-06T03:46:38.852750+0000","last_undegraded":"2023-07-06T03:46:38.852750+0000","last_fullsized":"2023-07-06T03:46:38.852750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:16:10.345229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420251+0000","last_change":"2023-07-06T03:46:37.853812+0000","last_active":"2023-07-06T03:46:39.420251+0000","last_peered":"2023-07-06T03:46:39.420251+0000","last_clean":"2023-07-06T03:46:39.420251+0000","last_became_active":"2023-07-06T03:46:37.852250+0000","last_became_peered":"2023-07-06T03:46:37.852250+0000","last_unstale":"2023-07-06T03:46:39.420251+0000","last_undegraded":"2023-07-06T03:46:39.420251+0000","last_fullsized":"2023-07-06T03:46:39.420251+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:11:36.909511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.042092+0000","last_change":"2023-07-06T03:46:37.861070+0000","last_active":"2023-07-06T03:46:39.042092+0000","last_peered":"2023-07-06T03:46:39.042092+0000","last_clean":"2023-07-06T03:46:39.042092+0000","last_became_active":"2023-07-06T03:46:37.860738+0000","last_became_peered":"2023-07-06T03:46:37.860738+0000","last_unstale":"2023-07-06T03:46:39.042092+0000","last_undegraded":"2023-07-06T03:46:39.042092+0000","last_fullsized":"2023-07-06T03:46:39.042092+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:53:51.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418678+0000","last_change":"2023-07-06T03:46:37.848284+0000","last_active":"2023-07-06T03:46:39.418678+0000","last_peered":"2023-07-06T03:46:39.418678+0000","last_clean":"2023-07-06T03:46:39.418678+0000","last_became_active":"2023-07-06T03:46:37.848010+0000","last_became_peered":"2023-07-06T03:46:37.848010+0000","last_unstale":"2023-07-06T03:46:39.418678+0000","last_undegraded":"2023-07-06T03:46:39.418678+0000","last_fullsized":"2023-07-06T03:46:39.418678+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:22:19.485193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419230+0000","last_change":"2023-07-06T03:46:37.856579+0000","last_active":"2023-07-06T03:46:39.419230+0000","last_peered":"2023-07-06T03:46:39.419230+0000","last_clean":"2023-07-06T03:46:39.419230+0000","last_became_active":"2023-07-06T03:46:37.856331+0000","last_became_peered":"2023-07-06T03:46:37.856331+0000","last_unstale":"2023-07-06T03:46:39.419230+0000","last_undegraded":"2023-07-06T03:46:39.419230+0000","last_fullsized":"2023-07-06T03:46:39.419230+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:12:57.228030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419212+0000","last_change":"2023-07-06T03:46:37.858786+0000","last_active":"2023-07-06T03:46:39.419212+0000","last_peered":"2023-07-06T03:46:39.419212+0000","last_clean":"2023-07-06T03:46:39.419212+0000","last_became_active":"2023-07-06T03:46:37.858458+0000","last_became_peered":"2023-07-06T03:46:37.858458+0000","last_unstale":"2023-07-06T03:46:39.419212+0000","last_undegraded":"2023-07-06T03:46:39.419212+0000","last_fullsized":"2023-07-06T03:46:39.419212+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:31:04.637693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418659+0000","last_change":"2023-07-06T03:46:37.864684+0000","last_active":"2023-07-06T03:46:39.418659+0000","last_peered":"2023-07-06T03:46:39.418659+0000","last_clean":"2023-07-06T03:46:39.418659+0000","last_became_active":"2023-07-06T03:46:37.864342+0000","last_became_peered":"2023-07-06T03:46:37.864342+0000","last_unstale":"2023-07-06T03:46:39.418659+0000","last_undegraded":"2023-07-06T03:46:39.418659+0000","last_fullsized":"2023-07-06T03:46:39.418659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:12:55.269776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419718+0000","last_change":"2023-07-06T03:46:37.851130+0000","last_active":"2023-07-06T03:46:39.419718+0000","last_peered":"2023-07-06T03:46:39.419718+0000","last_clean":"2023-07-06T03:46:39.419718+0000","last_became_active":"2023-07-06T03:46:37.850783+0000","last_became_peered":"2023-07-06T03:46:37.850783+0000","last_unstale":"2023-07-06T03:46:39.419718+0000","last_undegraded":"2023-07-06T03:46:39.419718+0000","last_fullsized":"2023-07-06T03:46:39.419718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:00:21.691539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419611+0000","last_change":"2023-07-06T03:46:37.853339+0000","last_active":"2023-07-06T03:46:39.419611+0000","last_peered":"2023-07-06T03:46:39.419611+0000","last_clean":"2023-07-06T03:46:39.419611+0000","last_became_active":"2023-07-06T03:46:37.852923+0000","last_became_peered":"2023-07-06T03:46:37.852923+0000","last_unstale":"2023-07-06T03:46:39.419611+0000","last_undegraded":"2023-07-06T03:46:39.419611+0000","last_fullsized":"2023-07-06T03:46:39.419611+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:04.156561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852627+0000","last_change":"2023-07-06T03:46:36.854615+0000","last_active":"2023-07-06T03:46:38.852627+0000","last_peered":"2023-07-06T03:46:38.852627+0000","last_clean":"2023-07-06T03:46:38.852627+0000","last_became_active":"2023-07-06T03:46:36.854325+0000","last_became_peered":"2023-07-06T03:46:36.854325+0000","last_unstale":"2023-07-06T03:46:38.852627+0000","last_undegraded":"2023-07-06T03:46:38.852627+0000","last_fullsized":"2023-07-06T03:46:38.852627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:18:30.511652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420053+0000","last_change":"2023-07-06T03:46:37.865092+0000","last_active":"2023-07-06T03:46:39.420053+0000","last_peered":"2023-07-06T03:46:39.420053+0000","last_clean":"2023-07-06T03:46:39.420053+0000","last_became_active":"2023-07-06T03:46:37.864810+0000","last_became_peered":"2023-07-06T03:46:37.864810+0000","last_unstale":"2023-07-06T03:46:39.420053+0000","last_undegraded":"2023-07-06T03:46:39.420053+0000","last_fullsized":"2023-07-06T03:46:39.420053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:02:10.182517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419775+0000","last_change":"2023-07-06T03:46:36.857512+0000","last_active":"2023-07-06T03:46:39.419775+0000","last_peered":"2023-07-06T03:46:39.419775+0000","last_clean":"2023-07-06T03:46:39.419775+0000","last_became_active":"2023-07-06T03:46:36.857177+0000","last_became_peered":"2023-07-06T03:46:36.857177+0000","last_unstale":"2023-07-06T03:46:39.419775+0000","last_undegraded":"2023-07-06T03:46:39.419775+0000","last_fullsized":"2023-07-06T03:46:39.419775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:33:18.938068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419718+0000","last_change":"2023-07-06T03:46:37.854234+0000","last_active":"2023-07-06T03:46:39.419718+0000","last_peered":"2023-07-06T03:46:39.419718+0000","last_clean":"2023-07-06T03:46:39.419718+0000","last_became_active":"2023-07-06T03:46:37.853973+0000","last_became_peered":"2023-07-06T03:46:37.853973+0000","last_unstale":"2023-07-06T03:46:39.419718+0000","last_undegraded":"2023-07-06T03:46:39.419718+0000","last_fullsized":"2023-07-06T03:46:39.419718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:17:49.210528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420214+0000","last_change":"2023-07-06T03:46:36.860082+0000","last_active":"2023-07-06T03:46:39.420214+0000","last_peered":"2023-07-06T03:46:39.420214+0000","last_clean":"2023-07-06T03:46:39.420214+0000","last_became_active":"2023-07-06T03:46:36.859761+0000","last_became_peered":"2023-07-06T03:46:36.859761+0000","last_unstale":"2023-07-06T03:46:39.420214+0000","last_undegraded":"2023-07-06T03:46:39.420214+0000","last_fullsized":"2023-07-06T03:46:39.420214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:06:29.939063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419942+0000","last_change":"2023-07-06T03:46:37.861372+0000","last_active":"2023-07-06T03:46:39.419942+0000","last_peered":"2023-07-06T03:46:39.419942+0000","last_clean":"2023-07-06T03:46:39.419942+0000","last_became_active":"2023-07-06T03:46:37.861062+0000","last_became_peered":"2023-07-06T03:46:37.861062+0000","last_unstale":"2023-07-06T03:46:39.419942+0000","last_undegraded":"2023-07-06T03:46:39.419942+0000","last_fullsized":"2023-07-06T03:46:39.419942+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:53:51.033207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852628+0000","last_change":"2023-07-06T03:46:36.860395+0000","last_active":"2023-07-06T03:46:38.852628+0000","last_peered":"2023-07-06T03:46:38.852628+0000","last_clean":"2023-07-06T03:46:38.852628+0000","last_became_active":"2023-07-06T03:46:36.860128+0000","last_became_peered":"2023-07-06T03:46:36.860128+0000","last_unstale":"2023-07-06T03:46:38.852628+0000","last_undegraded":"2023-07-06T03:46:38.852628+0000","last_fullsized":"2023-07-06T03:46:38.852628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:24:21.195033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419703+0000","last_change":"2023-07-06T03:46:37.856482+0000","last_active":"2023-07-06T03:46:39.419703+0000","last_peered":"2023-07-06T03:46:39.419703+0000","last_clean":"2023-07-06T03:46:39.419703+0000","last_became_active":"2023-07-06T03:46:37.856161+0000","last_became_peered":"2023-07-06T03:46:37.856161+0000","last_unstale":"2023-07-06T03:46:39.419703+0000","last_undegraded":"2023-07-06T03:46:39.419703+0000","last_fullsized":"2023-07-06T03:46:39.419703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:36:54.027820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852417+0000","last_change":"2023-07-06T03:46:36.844040+0000","last_active":"2023-07-06T03:46:38.852417+0000","last_peered":"2023-07-06T03:46:38.852417+0000","last_clean":"2023-07-06T03:46:38.852417+0000","last_became_active":"2023-07-06T03:46:36.843778+0000","last_became_peered":"2023-07-06T03:46:36.843778+0000","last_unstale":"2023-07-06T03:46:38.852417+0000","last_undegraded":"2023-07-06T03:46:38.852417+0000","last_fullsized":"2023-07-06T03:46:38.852417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:33:57.954583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041627+0000","last_change":"2023-07-06T03:46:37.858452+0000","last_active":"2023-07-06T03:46:39.041627+0000","last_peered":"2023-07-06T03:46:39.041627+0000","last_clean":"2023-07-06T03:46:39.041627+0000","last_became_active":"2023-07-06T03:46:37.858175+0000","last_became_peered":"2023-07-06T03:46:37.858175+0000","last_unstale":"2023-07-06T03:46:39.041627+0000","last_undegraded":"2023-07-06T03:46:39.041627+0000","last_fullsized":"2023-07-06T03:46:39.041627+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:15.115431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418480+0000","last_change":"2023-07-06T03:46:36.860493+0000","last_active":"2023-07-06T03:46:39.418480+0000","last_peered":"2023-07-06T03:46:39.418480+0000","last_clean":"2023-07-06T03:46:39.418480+0000","last_became_active":"2023-07-06T03:46:36.860183+0000","last_became_peered":"2023-07-06T03:46:36.860183+0000","last_unstale":"2023-07-06T03:46:39.418480+0000","last_undegraded":"2023-07-06T03:46:39.418480+0000","last_fullsized":"2023-07-06T03:46:39.418480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:25:32.127906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419308+0000","last_change":"2023-07-06T03:46:37.863671+0000","last_active":"2023-07-06T03:46:39.419308+0000","last_peered":"2023-07-06T03:46:39.419308+0000","last_clean":"2023-07-06T03:46:39.419308+0000","last_became_active":"2023-07-06T03:46:37.863398+0000","last_became_peered":"2023-07-06T03:46:37.863398+0000","last_unstale":"2023-07-06T03:46:39.419308+0000","last_undegraded":"2023-07-06T03:46:39.419308+0000","last_fullsized":"2023-07-06T03:46:39.419308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:17:26.517544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419441+0000","last_change":"2023-07-06T03:46:36.858921+0000","last_active":"2023-07-06T03:46:39.419441+0000","last_peered":"2023-07-06T03:46:39.419441+0000","last_clean":"2023-07-06T03:46:39.419441+0000","last_became_active":"2023-07-06T03:46:36.858622+0000","last_became_peered":"2023-07-06T03:46:36.858622+0000","last_unstale":"2023-07-06T03:46:39.419441+0000","last_undegraded":"2023-07-06T03:46:39.419441+0000","last_fullsized":"2023-07-06T03:46:39.419441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:10:34.577488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419682+0000","last_change":"2023-07-06T03:46:37.863461+0000","last_active":"2023-07-06T03:46:39.419682+0000","last_peered":"2023-07-06T03:46:39.419682+0000","last_clean":"2023-07-06T03:46:39.419682+0000","last_became_active":"2023-07-06T03:46:37.862923+0000","last_became_peered":"2023-07-06T03:46:37.862923+0000","last_unstale":"2023-07-06T03:46:39.419682+0000","last_undegraded":"2023-07-06T03:46:39.419682+0000","last_fullsized":"2023-07-06T03:46:39.419682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:04.849320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418404+0000","last_change":"2023-07-06T03:46:36.853092+0000","last_active":"2023-07-06T03:46:39.418404+0000","last_peered":"2023-07-06T03:46:39.418404+0000","last_clean":"2023-07-06T03:46:39.418404+0000","last_became_active":"2023-07-06T03:46:36.852795+0000","last_became_peered":"2023-07-06T03:46:36.852795+0000","last_unstale":"2023-07-06T03:46:39.418404+0000","last_undegraded":"2023-07-06T03:46:39.418404+0000","last_fullsized":"2023-07-06T03:46:39.418404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:49:11.074324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420266+0000","last_change":"2023-07-06T03:46:37.865852+0000","last_active":"2023-07-06T03:46:39.420266+0000","last_peered":"2023-07-06T03:46:39.420266+0000","last_clean":"2023-07-06T03:46:39.420266+0000","last_became_active":"2023-07-06T03:46:37.864988+0000","last_became_peered":"2023-07-06T03:46:37.864988+0000","last_unstale":"2023-07-06T03:46:39.420266+0000","last_undegraded":"2023-07-06T03:46:39.420266+0000","last_fullsized":"2023-07-06T03:46:39.420266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:47:11.749793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420238+0000","last_change":"2023-07-06T03:46:36.854389+0000","last_active":"2023-07-06T03:46:39.420238+0000","last_peered":"2023-07-06T03:46:39.420238+0000","last_clean":"2023-07-06T03:46:39.420238+0000","last_became_active":"2023-07-06T03:46:36.854083+0000","last_became_peered":"2023-07-06T03:46:36.854083+0000","last_unstale":"2023-07-06T03:46:39.420238+0000","last_undegraded":"2023-07-06T03:46:39.420238+0000","last_fullsized":"2023-07-06T03:46:39.420238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:07:57.989913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852508+0000","last_change":"2023-07-06T03:46:37.863905+0000","last_active":"2023-07-06T03:46:38.852508+0000","last_peered":"2023-07-06T03:46:38.852508+0000","last_clean":"2023-07-06T03:46:38.852508+0000","last_became_active":"2023-07-06T03:46:37.863639+0000","last_became_peered":"2023-07-06T03:46:37.863639+0000","last_unstale":"2023-07-06T03:46:38.852508+0000","last_undegraded":"2023-07-06T03:46:38.852508+0000","last_fullsized":"2023-07-06T03:46:38.852508+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:29:08.680714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419465+0000","last_change":"2023-07-06T03:46:36.848400+0000","last_active":"2023-07-06T03:46:39.419465+0000","last_peered":"2023-07-06T03:46:39.419465+0000","last_clean":"2023-07-06T03:46:39.419465+0000","last_became_active":"2023-07-06T03:46:36.848127+0000","last_became_peered":"2023-07-06T03:46:36.848127+0000","last_unstale":"2023-07-06T03:46:39.419465+0000","last_undegraded":"2023-07-06T03:46:39.419465+0000","last_fullsized":"2023-07-06T03:46:39.419465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:30:34.301467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420013+0000","last_change":"2023-07-06T03:46:37.849656+0000","last_active":"2023-07-06T03:46:39.420013+0000","last_peered":"2023-07-06T03:46:39.420013+0000","last_clean":"2023-07-06T03:46:39.420013+0000","last_became_active":"2023-07-06T03:46:37.849395+0000","last_became_peered":"2023-07-06T03:46:37.849395+0000","last_unstale":"2023-07-06T03:46:39.420013+0000","last_undegraded":"2023-07-06T03:46:39.420013+0000","last_fullsized":"2023-07-06T03:46:39.420013+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:04:53.111652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419205+0000","last_change":"2023-07-06T03:46:36.860726+0000","last_active":"2023-07-06T03:46:39.419205+0000","last_peered":"2023-07-06T03:46:39.419205+0000","last_clean":"2023-07-06T03:46:39.419205+0000","last_became_active":"2023-07-06T03:46:36.859745+0000","last_became_peered":"2023-07-06T03:46:36.859745+0000","last_unstale":"2023-07-06T03:46:39.419205+0000","last_undegraded":"2023-07-06T03:46:39.419205+0000","last_fullsized":"2023-07-06T03:46:39.419205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:24:19.605298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418784+0000","last_change":"2023-07-06T03:46:37.845211+0000","last_active":"2023-07-06T03:46:39.418784+0000","last_peered":"2023-07-06T03:46:39.418784+0000","last_clean":"2023-07-06T03:46:39.418784+0000","last_became_active":"2023-07-06T03:46:37.844944+0000","last_became_peered":"2023-07-06T03:46:37.844944+0000","last_unstale":"2023-07-06T03:46:39.418784+0000","last_undegraded":"2023-07-06T03:46:39.418784+0000","last_fullsized":"2023-07-06T03:46:39.418784+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:01:33.509255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420347+0000","last_change":"2023-07-06T03:46:36.858688+0000","last_active":"2023-07-06T03:46:39.420347+0000","last_peered":"2023-07-06T03:46:39.420347+0000","last_clean":"2023-07-06T03:46:39.420347+0000","last_became_active":"2023-07-06T03:46:36.858031+0000","last_became_peered":"2023-07-06T03:46:36.858031+0000","last_unstale":"2023-07-06T03:46:39.420347+0000","last_undegraded":"2023-07-06T03:46:39.420347+0000","last_fullsized":"2023-07-06T03:46:39.420347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:39:49.641446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419557+0000","last_change":"2023-07-06T03:46:37.852588+0000","last_active":"2023-07-06T03:46:39.419557+0000","last_peered":"2023-07-06T03:46:39.419557+0000","last_clean":"2023-07-06T03:46:39.419557+0000","last_became_active":"2023-07-06T03:46:37.852256+0000","last_became_peered":"2023-07-06T03:46:37.852256+0000","last_unstale":"2023-07-06T03:46:39.419557+0000","last_undegraded":"2023-07-06T03:46:39.419557+0000","last_fullsized":"2023-07-06T03:46:39.419557+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:24:39.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419552+0000","last_change":"2023-07-06T03:46:36.859170+0000","last_active":"2023-07-06T03:46:39.419552+0000","last_peered":"2023-07-06T03:46:39.419552+0000","last_clean":"2023-07-06T03:46:39.419552+0000","last_became_active":"2023-07-06T03:46:36.858764+0000","last_became_peered":"2023-07-06T03:46:36.858764+0000","last_unstale":"2023-07-06T03:46:39.419552+0000","last_undegraded":"2023-07-06T03:46:39.419552+0000","last_fullsized":"2023-07-06T03:46:39.419552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:07:48.265940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419356+0000","last_change":"2023-07-06T03:46:37.865476+0000","last_active":"2023-07-06T03:46:39.419356+0000","last_peered":"2023-07-06T03:46:39.419356+0000","last_clean":"2023-07-06T03:46:39.419356+0000","last_became_active":"2023-07-06T03:46:37.865057+0000","last_became_peered":"2023-07-06T03:46:37.865057+0000","last_unstale":"2023-07-06T03:46:39.419356+0000","last_undegraded":"2023-07-06T03:46:39.419356+0000","last_fullsized":"2023-07-06T03:46:39.419356+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:28.578365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419534+0000","last_change":"2023-07-06T03:46:36.856232+0000","last_active":"2023-07-06T03:46:39.419534+0000","last_peered":"2023-07-06T03:46:39.419534+0000","last_clean":"2023-07-06T03:46:39.419534+0000","last_became_active":"2023-07-06T03:46:36.855057+0000","last_became_peered":"2023-07-06T03:46:36.855057+0000","last_unstale":"2023-07-06T03:46:39.419534+0000","last_undegraded":"2023-07-06T03:46:39.419534+0000","last_fullsized":"2023-07-06T03:46:39.419534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:24.299644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041626+0000","last_change":"2023-07-06T03:46:37.869250+0000","last_active":"2023-07-06T03:46:39.041626+0000","last_peered":"2023-07-06T03:46:39.041626+0000","last_clean":"2023-07-06T03:46:39.041626+0000","last_became_active":"2023-07-06T03:46:37.868585+0000","last_became_peered":"2023-07-06T03:46:37.868585+0000","last_unstale":"2023-07-06T03:46:39.041626+0000","last_undegraded":"2023-07-06T03:46:39.041626+0000","last_fullsized":"2023-07-06T03:46:39.041626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.598122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419252+0000","last_change":"2023-07-06T03:46:36.848489+0000","last_active":"2023-07-06T03:46:39.419252+0000","last_peered":"2023-07-06T03:46:39.419252+0000","last_clean":"2023-07-06T03:46:39.419252+0000","last_became_active":"2023-07-06T03:46:36.848195+0000","last_became_peered":"2023-07-06T03:46:36.848195+0000","last_unstale":"2023-07-06T03:46:39.419252+0000","last_undegraded":"2023-07-06T03:46:39.419252+0000","last_fullsized":"2023-07-06T03:46:39.419252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:49:58.368321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853875+0000","last_change":"2023-07-06T03:46:37.857742+0000","last_active":"2023-07-06T03:46:38.853875+0000","last_peered":"2023-07-06T03:46:38.853875+0000","last_clean":"2023-07-06T03:46:38.853875+0000","last_became_active":"2023-07-06T03:46:37.857431+0000","last_became_peered":"2023-07-06T03:46:37.857431+0000","last_unstale":"2023-07-06T03:46:38.853875+0000","last_undegraded":"2023-07-06T03:46:38.853875+0000","last_fullsized":"2023-07-06T03:46:38.853875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:44:39.103954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.852415+0000","last_change":"2023-07-06T03:46:36.850609+0000","last_active":"2023-07-06T03:46:38.852415+0000","last_peered":"2023-07-06T03:46:38.852415+0000","last_clean":"2023-07-06T03:46:38.852415+0000","last_became_active":"2023-07-06T03:46:36.850316+0000","last_became_peered":"2023-07-06T03:46:36.850316+0000","last_unstale":"2023-07-06T03:46:38.852415+0000","last_undegraded":"2023-07-06T03:46:38.852415+0000","last_fullsized":"2023-07-06T03:46:38.852415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:06:25.318103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419909+0000","last_change":"2023-07-06T03:46:37.858603+0000","last_active":"2023-07-06T03:46:39.419909+0000","last_peered":"2023-07-06T03:46:39.419909+0000","last_clean":"2023-07-06T03:46:39.419909+0000","last_became_active":"2023-07-06T03:46:37.858343+0000","last_became_peered":"2023-07-06T03:46:37.858343+0000","last_unstale":"2023-07-06T03:46:39.419909+0000","last_undegraded":"2023-07-06T03:46:39.419909+0000","last_fullsized":"2023-07-06T03:46:39.419909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:52.391457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418737+0000","last_change":"2023-07-06T03:46:36.852893+0000","last_active":"2023-07-06T03:46:39.418737+0000","last_peered":"2023-07-06T03:46:39.418737+0000","last_clean":"2023-07-06T03:46:39.418737+0000","last_became_active":"2023-07-06T03:46:36.852559+0000","last_became_peered":"2023-07-06T03:46:36.852559+0000","last_unstale":"2023-07-06T03:46:39.418737+0000","last_undegraded":"2023-07-06T03:46:39.418737+0000","last_fullsized":"2023-07-06T03:46:39.418737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:50:39.638633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418795+0000","last_change":"2023-07-06T03:46:37.847891+0000","last_active":"2023-07-06T03:46:39.418795+0000","last_peered":"2023-07-06T03:46:39.418795+0000","last_clean":"2023-07-06T03:46:39.418795+0000","last_became_active":"2023-07-06T03:46:37.847633+0000","last_became_peered":"2023-07-06T03:46:37.847633+0000","last_unstale":"2023-07-06T03:46:39.418795+0000","last_undegraded":"2023-07-06T03:46:39.418795+0000","last_fullsized":"2023-07-06T03:46:39.418795+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:51:53.004246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419605+0000","last_change":"2023-07-06T03:46:36.861088+0000","last_active":"2023-07-06T03:46:39.419605+0000","last_peered":"2023-07-06T03:46:39.419605+0000","last_clean":"2023-07-06T03:46:39.419605+0000","last_became_active":"2023-07-06T03:46:36.860765+0000","last_became_peered":"2023-07-06T03:46:36.860765+0000","last_unstale":"2023-07-06T03:46:39.419605+0000","last_undegraded":"2023-07-06T03:46:39.419605+0000","last_fullsized":"2023-07-06T03:46:39.419605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:01:12.237971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418844+0000","last_change":"2023-07-06T03:46:37.865269+0000","last_active":"2023-07-06T03:46:39.418844+0000","last_peered":"2023-07-06T03:46:39.418844+0000","last_clean":"2023-07-06T03:46:39.418844+0000","last_became_active":"2023-07-06T03:46:37.864958+0000","last_became_peered":"2023-07-06T03:46:37.864958+0000","last_unstale":"2023-07-06T03:46:39.418844+0000","last_undegraded":"2023-07-06T03:46:39.418844+0000","last_fullsized":"2023-07-06T03:46:39.418844+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:24:42.365036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853575+0000","last_change":"2023-07-06T03:46:36.856749+0000","last_active":"2023-07-06T03:46:38.853575+0000","last_peered":"2023-07-06T03:46:38.853575+0000","last_clean":"2023-07-06T03:46:38.853575+0000","last_became_active":"2023-07-06T03:46:36.856419+0000","last_became_peered":"2023-07-06T03:46:36.856419+0000","last_unstale":"2023-07-06T03:46:38.853575+0000","last_undegraded":"2023-07-06T03:46:38.853575+0000","last_fullsized":"2023-07-06T03:46:38.853575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:03.842589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851579+0000","last_change":"2023-07-06T03:46:37.864167+0000","last_active":"2023-07-06T03:46:38.851579+0000","last_peered":"2023-07-06T03:46:38.851579+0000","last_clean":"2023-07-06T03:46:38.851579+0000","last_became_active":"2023-07-06T03:46:37.863865+0000","last_became_peered":"2023-07-06T03:46:37.863865+0000","last_unstale":"2023-07-06T03:46:38.851579+0000","last_undegraded":"2023-07-06T03:46:38.851579+0000","last_fullsized":"2023-07-06T03:46:38.851579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:04.849320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851484+0000","last_change":"2023-07-06T03:46:36.855799+0000","last_active":"2023-07-06T03:46:38.851484+0000","last_peered":"2023-07-06T03:46:38.851484+0000","last_clean":"2023-07-06T03:46:38.851484+0000","last_became_active":"2023-07-06T03:46:36.855494+0000","last_became_peered":"2023-07-06T03:46:36.855494+0000","last_unstale":"2023-07-06T03:46:38.851484+0000","last_undegraded":"2023-07-06T03:46:38.851484+0000","last_fullsized":"2023-07-06T03:46:38.851484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:24.299644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419061+0000","last_change":"2023-07-06T03:46:37.858828+0000","last_active":"2023-07-06T03:46:39.419061+0000","last_peered":"2023-07-06T03:46:39.419061+0000","last_clean":"2023-07-06T03:46:39.419061+0000","last_became_active":"2023-07-06T03:46:37.858546+0000","last_became_peered":"2023-07-06T03:46:37.858546+0000","last_unstale":"2023-07-06T03:46:39.419061+0000","last_undegraded":"2023-07-06T03:46:39.419061+0000","last_fullsized":"2023-07-06T03:46:39.419061+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:26:24.568746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851488+0000","last_change":"2023-07-06T03:46:36.849693+0000","last_active":"2023-07-06T03:46:38.851488+0000","last_peered":"2023-07-06T03:46:38.851488+0000","last_clean":"2023-07-06T03:46:38.851488+0000","last_became_active":"2023-07-06T03:46:36.849420+0000","last_became_peered":"2023-07-06T03:46:36.849420+0000","last_unstale":"2023-07-06T03:46:38.851488+0000","last_undegraded":"2023-07-06T03:46:38.851488+0000","last_fullsized":"2023-07-06T03:46:38.851488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:15.789105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419350+0000","last_change":"2023-07-06T03:46:37.858086+0000","last_active":"2023-07-06T03:46:39.419350+0000","last_peered":"2023-07-06T03:46:39.419350+0000","last_clean":"2023-07-06T03:46:39.419350+0000","last_became_active":"2023-07-06T03:46:37.857821+0000","last_became_peered":"2023-07-06T03:46:37.857821+0000","last_unstale":"2023-07-06T03:46:39.419350+0000","last_undegraded":"2023-07-06T03:46:39.419350+0000","last_fullsized":"2023-07-06T03:46:39.419350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:31:28.228023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419674+0000","last_change":"2023-07-06T03:46:36.852875+0000","last_active":"2023-07-06T03:46:39.419674+0000","last_peered":"2023-07-06T03:46:39.419674+0000","last_clean":"2023-07-06T03:46:39.419674+0000","last_became_active":"2023-07-06T03:46:36.852400+0000","last_became_peered":"2023-07-06T03:46:36.852400+0000","last_unstale":"2023-07-06T03:46:39.419674+0000","last_undegraded":"2023-07-06T03:46:39.419674+0000","last_fullsized":"2023-07-06T03:46:39.419674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:52.817628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041760+0000","last_change":"2023-07-06T03:46:37.869442+0000","last_active":"2023-07-06T03:46:39.041760+0000","last_peered":"2023-07-06T03:46:39.041760+0000","last_clean":"2023-07-06T03:46:39.041760+0000","last_became_active":"2023-07-06T03:46:37.869000+0000","last_became_peered":"2023-07-06T03:46:37.869000+0000","last_unstale":"2023-07-06T03:46:39.041760+0000","last_undegraded":"2023-07-06T03:46:39.041760+0000","last_fullsized":"2023-07-06T03:46:39.041760+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:16.795837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418545+0000","last_change":"2023-07-06T03:46:36.854933+0000","last_active":"2023-07-06T03:46:39.418545+0000","last_peered":"2023-07-06T03:46:39.418545+0000","last_clean":"2023-07-06T03:46:39.418545+0000","last_became_active":"2023-07-06T03:46:36.854557+0000","last_became_peered":"2023-07-06T03:46:36.854557+0000","last_unstale":"2023-07-06T03:46:39.418545+0000","last_undegraded":"2023-07-06T03:46:39.418545+0000","last_fullsized":"2023-07-06T03:46:39.418545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:39:46.623924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419773+0000","last_change":"2023-07-06T03:46:37.848797+0000","last_active":"2023-07-06T03:46:39.419773+0000","last_peered":"2023-07-06T03:46:39.419773+0000","last_clean":"2023-07-06T03:46:39.419773+0000","last_became_active":"2023-07-06T03:46:37.848516+0000","last_became_peered":"2023-07-06T03:46:37.848516+0000","last_unstale":"2023-07-06T03:46:39.419773+0000","last_undegraded":"2023-07-06T03:46:39.419773+0000","last_fullsized":"2023-07-06T03:46:39.419773+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:22:45.955750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419042+0000","last_change":"2023-07-06T03:46:36.861470+0000","last_active":"2023-07-06T03:46:39.419042+0000","last_peered":"2023-07-06T03:46:39.419042+0000","last_clean":"2023-07-06T03:46:39.419042+0000","last_became_active":"2023-07-06T03:46:36.861150+0000","last_became_peered":"2023-07-06T03:46:36.861150+0000","last_unstale":"2023-07-06T03:46:39.419042+0000","last_undegraded":"2023-07-06T03:46:39.419042+0000","last_fullsized":"2023-07-06T03:46:39.419042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:14.108700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853519+0000","last_change":"2023-07-06T03:46:37.859672+0000","last_active":"2023-07-06T03:46:38.853519+0000","last_peered":"2023-07-06T03:46:38.853519+0000","last_clean":"2023-07-06T03:46:38.853519+0000","last_became_active":"2023-07-06T03:46:37.859405+0000","last_became_peered":"2023-07-06T03:46:37.859405+0000","last_unstale":"2023-07-06T03:46:38.853519+0000","last_undegraded":"2023-07-06T03:46:38.853519+0000","last_fullsized":"2023-07-06T03:46:38.853519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:25.306376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419877+0000","last_change":"2023-07-06T03:46:36.854707+0000","last_active":"2023-07-06T03:46:39.419877+0000","last_peered":"2023-07-06T03:46:39.419877+0000","last_clean":"2023-07-06T03:46:39.419877+0000","last_became_active":"2023-07-06T03:46:36.854435+0000","last_became_peered":"2023-07-06T03:46:36.854435+0000","last_unstale":"2023-07-06T03:46:39.419877+0000","last_undegraded":"2023-07-06T03:46:39.419877+0000","last_fullsized":"2023-07-06T03:46:39.419877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:05:36.862857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851576+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:46:38.851576+0000","last_peered":"2023-07-06T03:46:38.851576+0000","last_clean":"2023-07-06T03:46:38.851576+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:46:38.851576+0000","last_undegraded":"2023-07-06T03:46:38.851576+0000","last_fullsized":"2023-07-06T03:46:38.851576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:17:21.111789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.851549+0000","last_change":"2023-07-06T03:46:37.862798+0000","last_active":"2023-07-06T03:46:38.851549+0000","last_peered":"2023-07-06T03:46:38.851549+0000","last_clean":"2023-07-06T03:46:38.851549+0000","last_became_active":"2023-07-06T03:46:37.862509+0000","last_became_peered":"2023-07-06T03:46:37.862509+0000","last_unstale":"2023-07-06T03:46:38.851549+0000","last_undegraded":"2023-07-06T03:46:38.851549+0000","last_fullsized":"2023-07-06T03:46:38.851549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:26.836331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853722+0000","last_change":"2023-07-06T03:46:36.859132+0000","last_active":"2023-07-06T03:46:38.853722+0000","last_peered":"2023-07-06T03:46:38.853722+0000","last_clean":"2023-07-06T03:46:38.853722+0000","last_became_active":"2023-07-06T03:46:36.858862+0000","last_became_peered":"2023-07-06T03:46:36.858862+0000","last_unstale":"2023-07-06T03:46:38.853722+0000","last_undegraded":"2023-07-06T03:46:38.853722+0000","last_fullsized":"2023-07-06T03:46:38.853722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:51.384726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419437+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:46:39.419437+0000","last_peered":"2023-07-06T03:46:39.419437+0000","last_clean":"2023-07-06T03:46:39.419437+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:46:39.419437+0000","last_undegraded":"2023-07-06T03:46:39.419437+0000","last_fullsized":"2023-07-06T03:46:39.419437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:04:05.608173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041981+0000","last_change":"2023-07-06T03:46:37.870020+0000","last_active":"2023-07-06T03:46:39.041981+0000","last_peered":"2023-07-06T03:46:39.041981+0000","last_clean":"2023-07-06T03:46:39.041981+0000","last_became_active":"2023-07-06T03:46:37.869716+0000","last_became_peered":"2023-07-06T03:46:37.869716+0000","last_unstale":"2023-07-06T03:46:39.041981+0000","last_undegraded":"2023-07-06T03:46:39.041981+0000","last_fullsized":"2023-07-06T03:46:39.041981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:52.391457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853769+0000","last_change":"2023-07-06T03:46:36.857376+0000","last_active":"2023-07-06T03:46:38.853769+0000","last_peered":"2023-07-06T03:46:38.853769+0000","last_clean":"2023-07-06T03:46:38.853769+0000","last_became_active":"2023-07-06T03:46:36.856783+0000","last_became_peered":"2023-07-06T03:46:36.856783+0000","last_unstale":"2023-07-06T03:46:38.853769+0000","last_undegraded":"2023-07-06T03:46:38.853769+0000","last_fullsized":"2023-07-06T03:46:38.853769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:53:50.026476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853513+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:46:38.853513+0000","last_peered":"2023-07-06T03:46:38.853513+0000","last_clean":"2023-07-06T03:46:38.853513+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:46:38.853513+0000","last_undegraded":"2023-07-06T03:46:38.853513+0000","last_fullsized":"2023-07-06T03:46:38.853513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:31:47.232229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419801+0000","last_change":"2023-07-06T03:46:37.859320+0000","last_active":"2023-07-06T03:46:39.419801+0000","last_peered":"2023-07-06T03:46:39.419801+0000","last_clean":"2023-07-06T03:46:39.419801+0000","last_became_active":"2023-07-06T03:46:37.858687+0000","last_became_peered":"2023-07-06T03:46:37.858687+0000","last_unstale":"2023-07-06T03:46:39.419801+0000","last_undegraded":"2023-07-06T03:46:39.419801+0000","last_fullsized":"2023-07-06T03:46:39.419801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:53.443866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041830+0000","last_change":"2023-07-06T03:46:36.858102+0000","last_active":"2023-07-06T03:46:39.041830+0000","last_peered":"2023-07-06T03:46:39.041830+0000","last_clean":"2023-07-06T03:46:39.041830+0000","last_became_active":"2023-07-06T03:46:36.857729+0000","last_became_peered":"2023-07-06T03:46:36.857729+0000","last_unstale":"2023-07-06T03:46:39.041830+0000","last_undegraded":"2023-07-06T03:46:39.041830+0000","last_fullsized":"2023-07-06T03:46:39.041830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:52.817628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419309+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:46:39.419309+0000","last_peered":"2023-07-06T03:46:39.419309+0000","last_clean":"2023-07-06T03:46:39.419309+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:46:39.419309+0000","last_undegraded":"2023-07-06T03:46:39.419309+0000","last_fullsized":"2023-07-06T03:46:39.419309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:28:47.569266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.418463+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:46:39.418463+0000","last_peered":"2023-07-06T03:46:39.418463+0000","last_clean":"2023-07-06T03:46:39.418463+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:46:39.418463+0000","last_undegraded":"2023-07-06T03:46:39.418463+0000","last_fullsized":"2023-07-06T03:46:39.418463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:08:09.570675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:38.853891+0000","last_change":"2023-07-06T03:46:37.853652+0000","last_active":"2023-07-06T03:46:38.853891+0000","last_peered":"2023-07-06T03:46:38.853891+0000","last_clean":"2023-07-06T03:46:38.853891+0000","last_became_active":"2023-07-06T03:46:37.853323+0000","last_became_peered":"2023-07-06T03:46:37.853323+0000","last_unstale":"2023-07-06T03:46:38.853891+0000","last_undegraded":"2023-07-06T03:46:38.853891+0000","last_fullsized":"2023-07-06T03:46:38.853891+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:02:10.182517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419816+0000","last_change":"2023-07-06T03:46:36.847001+0000","last_active":"2023-07-06T03:46:39.419816+0000","last_peered":"2023-07-06T03:46:39.419816+0000","last_clean":"2023-07-06T03:46:39.419816+0000","last_became_active":"2023-07-06T03:46:36.846669+0000","last_became_peered":"2023-07-06T03:46:36.846669+0000","last_unstale":"2023-07-06T03:46:39.419816+0000","last_undegraded":"2023-07-06T03:46:39.419816+0000","last_fullsized":"2023-07-06T03:46:39.419816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:19.239209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419000+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:46:39.419000+0000","last_peered":"2023-07-06T03:46:39.419000+0000","last_clean":"2023-07-06T03:46:39.419000+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:46:39.419000+0000","last_undegraded":"2023-07-06T03:46:39.419000+0000","last_fullsized":"2023-07-06T03:46:39.419000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:26:05.045659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419530+0000","last_change":"2023-07-06T03:46:37.861516+0000","last_active":"2023-07-06T03:46:39.419530+0000","last_peered":"2023-07-06T03:46:39.419530+0000","last_clean":"2023-07-06T03:46:39.419530+0000","last_became_active":"2023-07-06T03:46:37.860991+0000","last_became_peered":"2023-07-06T03:46:37.860991+0000","last_unstale":"2023-07-06T03:46:39.419530+0000","last_undegraded":"2023-07-06T03:46:39.419530+0000","last_fullsized":"2023-07-06T03:46:39.419530+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:26.836331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041886+0000","last_change":"2023-07-06T03:46:36.855134+0000","last_active":"2023-07-06T03:46:39.041886+0000","last_peered":"2023-07-06T03:46:39.041886+0000","last_clean":"2023-07-06T03:46:39.041886+0000","last_became_active":"2023-07-06T03:46:36.854885+0000","last_became_peered":"2023-07-06T03:46:36.854885+0000","last_unstale":"2023-07-06T03:46:39.041886+0000","last_undegraded":"2023-07-06T03:46:39.041886+0000","last_fullsized":"2023-07-06T03:46:39.041886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:33:18.938068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419516+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:46:39.419516+0000","last_peered":"2023-07-06T03:46:39.419516+0000","last_clean":"2023-07-06T03:46:39.419516+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:46:39.419516+0000","last_undegraded":"2023-07-06T03:46:39.419516+0000","last_fullsized":"2023-07-06T03:46:39.419516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:36:14.721099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419354+0000","last_change":"2023-07-06T03:46:37.861288+0000","last_active":"2023-07-06T03:46:39.419354+0000","last_peered":"2023-07-06T03:46:39.419354+0000","last_clean":"2023-07-06T03:46:39.419354+0000","last_became_active":"2023-07-06T03:46:37.860989+0000","last_became_peered":"2023-07-06T03:46:37.860989+0000","last_unstale":"2023-07-06T03:46:39.419354+0000","last_undegraded":"2023-07-06T03:46:39.419354+0000","last_fullsized":"2023-07-06T03:46:39.419354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:46:55.890372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419276+0000","last_change":"2023-07-06T03:46:36.847922+0000","last_active":"2023-07-06T03:46:39.419276+0000","last_peered":"2023-07-06T03:46:39.419276+0000","last_clean":"2023-07-06T03:46:39.419276+0000","last_became_active":"2023-07-06T03:46:36.847302+0000","last_became_peered":"2023-07-06T03:46:36.847302+0000","last_unstale":"2023-07-06T03:46:39.419276+0000","last_undegraded":"2023-07-06T03:46:39.419276+0000","last_fullsized":"2023-07-06T03:46:39.419276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:14:40.997856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.420277+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:46:39.420277+0000","last_peered":"2023-07-06T03:46:39.420277+0000","last_clean":"2023-07-06T03:46:39.420277+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:46:39.420277+0000","last_undegraded":"2023-07-06T03:46:39.420277+0000","last_fullsized":"2023-07-06T03:46:39.420277+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:51:20.288814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.041768+0000","last_change":"2023-07-06T03:46:37.862349+0000","last_active":"2023-07-06T03:46:39.041768+0000","last_peered":"2023-07-06T03:46:39.041768+0000","last_clean":"2023-07-06T03:46:39.041768+0000","last_became_active":"2023-07-06T03:46:37.862010+0000","last_became_peered":"2023-07-06T03:46:37.862010+0000","last_unstale":"2023-07-06T03:46:39.041768+0000","last_undegraded":"2023-07-06T03:46:39.041768+0000","last_fullsized":"2023-07-06T03:46:39.041768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:36.826176+0000","last_clean_scrub_stamp":"2023-07-06T03:46:36.826176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:00:04.849320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419332+0000","last_change":"2023-07-06T03:46:36.861193+0000","last_active":"2023-07-06T03:46:39.419332+0000","last_peered":"2023-07-06T03:46:39.419332+0000","last_clean":"2023-07-06T03:46:39.419332+0000","last_became_active":"2023-07-06T03:46:36.860876+0000","last_became_peered":"2023-07-06T03:46:36.860876+0000","last_unstale":"2023-07-06T03:46:39.419332+0000","last_undegraded":"2023-07-06T03:46:39.419332+0000","last_fullsized":"2023-07-06T03:46:39.419332+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:46:35.819445+0000","last_clean_scrub_stamp":"2023-07-06T03:46:35.819445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:24:46.275089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T03:46:39.419845+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:46:39.419845+0000","last_peered":"2023-07-06T03:46:39.419845+0000","last_clean":"2023-07-06T03:46:39.419845+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:46:39.419845+0000","last_undegraded":"2023-07-06T03:46:39.419845+0000","last_fullsized":"2023-07-06T03:46:39.419845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:15:52.413417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":156752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.629},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.8660000000000001,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.77400000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.80400000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.77,"5min":3.77,"15min":3.77},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.82299999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.57199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":156716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65400000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.5}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.628}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.74399999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.76300000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.78400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":983040,"data_stored":747072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":13.055999999999999,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.503}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":12.989000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":13.023999999999999,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.488,"5min":1.488,"15min":1.488},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":12.941000000000001,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.55000000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.57099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":393216,"data_stored":156774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54400000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.52400000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":9.5960000000000001,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.63}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":158828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.497}]},{"osd":3,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":2.157},"last":0.75},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.69099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":156772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":5.6550000000000002,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":15.911,"5min":15.911,"15min":15.911},"last":0.47999999999999998},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":16.024000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":16.068999999999999,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":5.7300000000000004,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.077,"5min":5.077,"15min":5.077},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":5.6980000000000004,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.84899999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":5.157,"5min":5.157,"15min":5.157},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":5.1189999999999998,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.70299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":158370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.628},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.60699999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":6.258,"5min":6.258,"15min":6.258},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":6.4740000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":6.4859999999999998,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.501}]},{"osd":6,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.79000000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.77200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":978944,"data_stored":746613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":6.4059999999999997,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.78100000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.84499999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.90400000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.95399999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.63},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.74199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:46:47.612 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T03:46:47.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T03:46:47.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.841+0000 7f3b989ff700 1 Processor -- start 2023-07-06T03:46:47.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.841+0000 7f3b989ff700 1 -- start start 2023-07-06T03:46:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b989ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3b940b4ea0 0x7f3b940cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b989ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b94160a50 0x7f3b940d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b989ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b940d0c10 con 0x7f3b940b8590 2023-07-06T03:46:47.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b989ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b94160e70 con 0x7f3b94160a50 2023-07-06T03:46:47.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b989ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b940d08e0 con 0x7f3b940b4ea0 2023-07-06T03:46:47.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b9259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3b940b8590 0x7f3b940b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50872/0 (socket says 172.21.15.97:50872) 2023-07-06T03:46:47.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b9259c700 1 -- 172.21.15.97:0/1523735248 learned_addr learned my addr 172.21.15.97:0/1523735248 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:47.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b92d9d700 1 --2- 172.21.15.97:0/1523735248 >> v2:172.21.15.97:3300/0 conn(0x7f3b94160a50 0x7f3b940d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b91d9b700 1 -- 172.21.15.97:0/1523735248 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908265781 0 0) 0x7f3b940d0c10 con 0x7f3b940b8590 2023-07-06T03:46:47.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b8bfff700 1 --2- 172.21.15.97:0/1523735248 >> v2:172.21.15.146:3300/0 conn(0x7f3b940b4ea0 0x7f3b940cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b91d9b700 1 -- 172.21.15.97:0/1523735248 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b800035b0 con 0x7f3b940b8590 2023-07-06T03:46:47.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b92d9d700 1 -- 172.21.15.97:0/1523735248 >> v2:172.21.15.146:3300/0 conn(0x7f3b940b4ea0 msgr2=0x7f3b940cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:47.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b92d9d700 1 --2- 172.21.15.97:0/1523735248 >> v2:172.21.15.146:3300/0 conn(0x7f3b940b4ea0 0x7f3b940cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b92d9d700 1 -- 172.21.15.97:0/1523735248 >> v1:172.21.15.97:6789/0 conn(0x7f3b940b8590 legacy=0x7f3b940b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b92d9d700 1 -- 172.21.15.97:0/1523735248 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b940b2560 con 0x7f3b94160a50 2023-07-06T03:46:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.843+0000 7f3b8bfff700 1 --2- 172.21.15.97:0/1523735248 >> v2:172.21.15.146:3300/0 conn(0x7f3b940b4ea0 0x7f3b940cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.844+0000 7f3b92d9d700 1 --2- 172.21.15.97:0/1523735248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b94160a50 0x7f3b940d10a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3b940b1980 tx=0x7f3b8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a412733bde4f772f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.844+0000 7f3b91d9b700 1 -- 172.21.15.97:0/1523735248 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00e070 con 0x7f3b94160a50 2023-07-06T03:46:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.844+0000 7f3b91d9b700 1 -- 172.21.15.97:0/1523735248 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c00bb20 con 0x7f3b94160a50 2023-07-06T03:46:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.844+0000 7f3b91d9b700 1 -- 172.21.15.97:0/1523735248 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00be90 con 0x7f3b94160a50 2023-07-06T03:46:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.844+0000 7f3b989ff700 1 -- 172.21.15.97:0/1523735248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b94160a50 msgr2=0x7f3b940d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.844+0000 7f3b989ff700 1 --2- 172.21.15.97:0/1523735248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b94160a50 0x7f3b940d10a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3b940b1980 tx=0x7f3b8c00b330 comp rx=0 tx=0).stop 2023-07-06T03:46:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.844+0000 7f3b989ff700 1 -- 172.21.15.97:0/1523735248 shutdown_connections 2023-07-06T03:46:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.844+0000 7f3b989ff700 1 --2- 172.21.15.97:0/1523735248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b94160a50 0x7f3b940d10a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.844+0000 7f3b989ff700 1 --2- 172.21.15.97:0/1523735248 >> v2:172.21.15.146:3300/0 conn(0x7f3b940b4ea0 0x7f3b940cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.844+0000 7f3b989ff700 1 -- 172.21.15.97:0/1523735248 >> 172.21.15.97:0/1523735248 conn(0x7f3b9406b290 msgr2=0x7f3b940cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.845+0000 7f3b989ff700 1 -- 172.21.15.97:0/1523735248 shutdown_connections 2023-07-06T03:46:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.845+0000 7f3b989ff700 1 -- 172.21.15.97:0/1523735248 wait complete. 2023-07-06T03:46:47.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.846+0000 7f3b989ff700 1 Processor -- start 2023-07-06T03:46:47.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.846+0000 7f3b989ff700 1 -- start start 2023-07-06T03:46:47.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.847+0000 7f3b989ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940b4ea0 0x7f3b9415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.847+0000 7f3b989ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b940b8590 0x7f3b940ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.847+0000 7f3b989ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b94160a50 0x7f3b940c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.847+0000 7f3b989ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9415d300 con 0x7f3b940b4ea0 2023-07-06T03:46:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.847+0000 7f3b989ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941f6560 con 0x7f3b940b8590 2023-07-06T03:46:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.847+0000 7f3b989ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9415d060 con 0x7f3b94160a50 2023-07-06T03:46:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b9259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940b4ea0 0x7f3b9415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b9259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940b4ea0 0x7f3b9415cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58194/0 (socket says 172.21.15.97:58194) 2023-07-06T03:46:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b9259c700 1 -- 172.21.15.97:0/1012188354 learned_addr learned my addr 172.21.15.97:0/1012188354 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b8bfff700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b940b8590 0x7f3b940ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b92d9d700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b94160a50 0x7f3b940c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b9259c700 1 -- 172.21.15.97:0/1012188354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b94160a50 msgr2=0x7f3b940c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b9259c700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b94160a50 0x7f3b940c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b9259c700 1 -- 172.21.15.97:0/1012188354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b940b8590 msgr2=0x7f3b940ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b9259c700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b940b8590 0x7f3b940ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b9259c700 1 -- 172.21.15.97:0/1012188354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c009a70 con 0x7f3b940b4ea0 2023-07-06T03:46:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b8bfff700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b940b8590 0x7f3b940ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:47.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b9259c700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940b4ea0 0x7f3b9415cb20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400c450 tx=0x7f3b8400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:47.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b8affd700 1 -- 172.21.15.97:0/1012188354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b84005d70 con 0x7f3b940b4ea0 2023-07-06T03:46:47.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b8affd700 1 -- 172.21.15.97:0/1012188354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b840070d0 con 0x7f3b940b4ea0 2023-07-06T03:46:47.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b8affd700 1 -- 172.21.15.97:0/1012188354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b84007440 con 0x7f3b940b4ea0 2023-07-06T03:46:47.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.848+0000 7f3b989ff700 1 -- 172.21.15.97:0/1012188354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b941f57a0 con 0x7f3b940b4ea0 2023-07-06T03:46:47.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.849+0000 7f3b989ff700 1 -- 172.21.15.97:0/1012188354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b941f5d60 con 0x7f3b940b4ea0 2023-07-06T03:46:47.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.850+0000 7f3b989ff700 1 -- 172.21.15.97:0/1012188354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b940d21d0 con 0x7f3b940b4ea0 2023-07-06T03:46:47.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.850+0000 7f3b8affd700 1 -- 172.21.15.97:0/1012188354 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3b840075a0 con 0x7f3b940b4ea0 2023-07-06T03:46:47.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.851+0000 7f3b8affd700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3b7406ace0 0x7f3b7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:47.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.851+0000 7f3b8affd700 1 -- 172.21.15.97:0/1012188354 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3b84088890 con 0x7f3b940b4ea0 2023-07-06T03:46:47.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.852+0000 7f3b8bfff700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3b7406ace0 0x7f3b7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:47.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.852+0000 7f3b8bfff700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3b7406ace0 0x7f3b7406d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c006fd0 tx=0x7f3b7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:47.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:47.856+0000 7f3b8affd700 1 -- 172.21.15.97:0/1012188354 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3b84053950 con 0x7f3b940b4ea0 2023-07-06T03:46:48.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.128+0000 7f3b989ff700 1 -- 172.21.15.97:0/1012188354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3b940ce5e0 con 0x7f3b940b4ea0 2023-07-06T03:46:48.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.132+0000 7f3b8affd700 1 -- 172.21.15.97:0/1012188354 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3b840170b0 con 0x7f3b940b4ea0 2023-07-06T03:46:48.134 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:46:48.134 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg== 2023-07-06T03:46:48.134 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-07-06T03:46:48.135 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-07-06T03:46:48.135 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T03:46:48.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.134+0000 7f3b989ff700 1 -- 172.21.15.97:0/1012188354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3b7406ace0 msgr2=0x7f3b7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:48.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.134+0000 7f3b989ff700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3b7406ace0 0x7f3b7406d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c006fd0 tx=0x7f3b7c008040 comp rx=0 tx=0).stop 2023-07-06T03:46:48.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.134+0000 7f3b989ff700 1 -- 172.21.15.97:0/1012188354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940b4ea0 msgr2=0x7f3b9415cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:48.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.134+0000 7f3b989ff700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940b4ea0 0x7f3b9415cb20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400c450 tx=0x7f3b8400c810 comp rx=0 tx=0).stop 2023-07-06T03:46:48.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.135+0000 7f3b989ff700 1 -- 172.21.15.97:0/1012188354 shutdown_connections 2023-07-06T03:46:48.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.135+0000 7f3b989ff700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3b7406ace0 0x7f3b7406d190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.135+0000 7f3b989ff700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3b94160a50 0x7f3b940c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.135+0000 7f3b989ff700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940b4ea0 0x7f3b9415cb20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.135+0000 7f3b989ff700 1 --2- 172.21.15.97:0/1012188354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3b940b8590 0x7f3b940ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.135+0000 7f3b989ff700 1 -- 172.21.15.97:0/1012188354 >> 172.21.15.97:0/1012188354 conn(0x7f3b9406b290 msgr2=0x7f3b9415aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:48.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.135+0000 7f3b989ff700 1 -- 172.21.15.97:0/1012188354 shutdown_connections 2023-07-06T03:46:48.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.135+0000 7f3b989ff700 1 -- 172.21.15.97:0/1012188354 wait complete. 2023-07-06T03:46:48.138 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-07-06T03:46:48.154 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:46:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.346+0000 7fdd255ff700 1 Processor -- start 2023-07-06T03:46:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.346+0000 7fdd255ff700 1 -- start start 2023-07-06T03:46:48.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.347+0000 7fdd255ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdd2015ca10 0x7fdd20160e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:48.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.348+0000 7fdd255ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd200b50b0 0x7fdd200d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:48.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.348+0000 7fdd255ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd200d1190 con 0x7fdd200cf3c0 2023-07-06T03:46:48.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.348+0000 7fdd255ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd20161340 con 0x7fdd200b50b0 2023-07-06T03:46:48.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.348+0000 7fdd255ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd200cd680 con 0x7fdd2015ca10 2023-07-06T03:46:48.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.348+0000 7fdd1f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd200b50b0 0x7fdd200d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:48.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.348+0000 7fdd1f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd200b50b0 0x7fdd200d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58196/0 (socket says 172.21.15.97:58196) 2023-07-06T03:46:48.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.348+0000 7fdd1f7fe700 1 -- 172.21.15.97:0/3789813831 learned_addr learned my addr 172.21.15.97:0/3789813831 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:48.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.348+0000 7fdd1f7fe700 1 -- 172.21.15.97:0/3789813831 >> v2:172.21.15.146:3300/0 conn(0x7fdd2015ca10 msgr2=0x7fdd20160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:48.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.348+0000 7fdd1f7fe700 1 --2- 172.21.15.97:0/3789813831 >> v2:172.21.15.146:3300/0 conn(0x7fdd2015ca10 0x7fdd20160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.348+0000 7fdd1f7fe700 1 -- 172.21.15.97:0/3789813831 >> v1:172.21.15.146:6789/0 conn(0x7fdd200cf3c0 legacy=0x7fdd200b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:48.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.348+0000 7fdd1f7fe700 1 -- 172.21.15.97:0/3789813831 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd200cd960 con 0x7fdd200b50b0 2023-07-06T03:46:48.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.349+0000 7fdd1f7fe700 1 --2- 172.21.15.97:0/3789813831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd200b50b0 0x7fdd200d1620 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdd2008c210 tx=0x7fdd1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cc2d6bb6446ca3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:48.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.349+0000 7fdd1e7fc700 1 -- 172.21.15.97:0/3789813831 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1800e070 con 0x7fdd200b50b0 2023-07-06T03:46:48.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.349+0000 7fdd1e7fc700 1 -- 172.21.15.97:0/3789813831 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd18003a20 con 0x7fdd200b50b0 2023-07-06T03:46:48.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.349+0000 7fdd1e7fc700 1 -- 172.21.15.97:0/3789813831 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18003dd0 con 0x7fdd200b50b0 2023-07-06T03:46:48.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.349+0000 7fdd255ff700 1 -- 172.21.15.97:0/3789813831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd200b50b0 msgr2=0x7fdd200d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:48.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.349+0000 7fdd255ff700 1 --2- 172.21.15.97:0/3789813831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd200b50b0 0x7fdd200d1620 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdd2008c210 tx=0x7fdd1800b330 comp rx=0 tx=0).stop 2023-07-06T03:46:48.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.349+0000 7fdd255ff700 1 -- 172.21.15.97:0/3789813831 shutdown_connections 2023-07-06T03:46:48.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.349+0000 7fdd255ff700 1 --2- 172.21.15.97:0/3789813831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd200b50b0 0x7fdd200d1620 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.349+0000 7fdd255ff700 1 --2- 172.21.15.97:0/3789813831 >> v2:172.21.15.146:3300/0 conn(0x7fdd2015ca10 0x7fdd20160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.349+0000 7fdd255ff700 1 -- 172.21.15.97:0/3789813831 >> 172.21.15.97:0/3789813831 conn(0x7fdd2006a920 msgr2=0x7fdd2006ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:48.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.350+0000 7fdd255ff700 1 -- 172.21.15.97:0/3789813831 shutdown_connections 2023-07-06T03:46:48.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.350+0000 7fdd255ff700 1 -- 172.21.15.97:0/3789813831 wait complete. 2023-07-06T03:46:48.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.351+0000 7fdd255ff700 1 Processor -- start 2023-07-06T03:46:48.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.351+0000 7fdd255ff700 1 -- start start 2023-07-06T03:46:48.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.352+0000 7fdd255ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd200b50b0 0x7fdd20158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.352+0000 7fdd255ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd200cf3c0 0x7fdd2014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.352+0000 7fdd255ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2015ca10 0x7fdd2015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.352+0000 7fdd255ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd20159100 con 0x7fdd200b50b0 2023-07-06T03:46:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.352+0000 7fdd255ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd20159330 con 0x7fdd2015ca10 2023-07-06T03:46:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.352+0000 7fdd255ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd20159610 con 0x7fdd200cf3c0 2023-07-06T03:46:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd1effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd200b50b0 0x7fdd20158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd1effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd200b50b0 0x7fdd20158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58200/0 (socket says 172.21.15.97:58200) 2023-07-06T03:46:48.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd1effd700 1 -- 172.21.15.97:0/2550909829 learned_addr learned my addr 172.21.15.97:0/2550909829 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:48.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd1f7fe700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2015ca10 0x7fdd2015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:48.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd17fff700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd200cf3c0 0x7fdd2014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:48.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd1effd700 1 -- 172.21.15.97:0/2550909829 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd200cf3c0 msgr2=0x7fdd2014b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:48.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd1effd700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd200cf3c0 0x7fdd2014b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd1effd700 1 -- 172.21.15.97:0/2550909829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2015ca10 msgr2=0x7fdd2015afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:48.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd1effd700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2015ca10 0x7fdd2015afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd1effd700 1 -- 172.21.15.97:0/2550909829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd18009b30 con 0x7fdd200b50b0 2023-07-06T03:46:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd1effd700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd200b50b0 0x7fdd20158bc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000b4a0 tx=0x7fdd1000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.353+0000 7fdd177fe700 1 -- 172.21.15.97:0/2550909829 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10011830 con 0x7fdd200b50b0 2023-07-06T03:46:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.354+0000 7fdd177fe700 1 -- 172.21.15.97:0/2550909829 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd10011990 con 0x7fdd200b50b0 2023-07-06T03:46:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.354+0000 7fdd255ff700 1 -- 172.21.15.97:0/2550909829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd201598f0 con 0x7fdd200b50b0 2023-07-06T03:46:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.354+0000 7fdd177fe700 1 -- 172.21.15.97:0/2550909829 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10011d40 con 0x7fdd200b50b0 2023-07-06T03:46:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.354+0000 7fdd255ff700 1 -- 172.21.15.97:0/2550909829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd20159eb0 con 0x7fdd200b50b0 2023-07-06T03:46:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.355+0000 7fdd255ff700 1 -- 172.21.15.97:0/2550909829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2006a790 con 0x7fdd200b50b0 2023-07-06T03:46:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.355+0000 7fdd177fe700 1 -- 172.21.15.97:0/2550909829 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd1001a400 con 0x7fdd200b50b0 2023-07-06T03:46:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.356+0000 7fdd177fe700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd0006ace0 0x7fdd0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.356+0000 7fdd177fe700 1 -- 172.21.15.97:0/2550909829 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdd10089300 con 0x7fdd200b50b0 2023-07-06T03:46:48.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.357+0000 7fdd17fff700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd0006ace0 0x7fdd0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:48.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.357+0000 7fdd17fff700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd0006ace0 0x7fdd0006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd2015eaf0 tx=0x7fdd08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:48.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.361+0000 7fdd177fe700 1 -- 172.21.15.97:0/2550909829 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdd100543c0 con 0x7fdd200b50b0 2023-07-06T03:46:48.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.599+0000 7fdd255ff700 1 -- 172.21.15.97:0/2550909829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd200b2340 con 0x7fdd200b50b0 2023-07-06T03:46:48.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.599+0000 7fdd177fe700 1 -- 172.21.15.97:0/2550909829 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fdd100579e0 con 0x7fdd200b50b0 2023-07-06T03:46:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:46:48.601 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/1839628004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1839628004},{"type":"v1","addr":"172.21.15.146:6834","nonce":1839628004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4937,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2131262285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2131262285},{"type":"v1","addr":"172.21.15.146:6835","nonce":2131262285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1367937004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367937004},{"type":"v1","addr":"172.21.15.97:6835","nonce":1367937004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:46:37.331190+0000","modified":"2023-07-06T03:46:40.864523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/663657264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":663657264},{"type":"v1","addr":"172.21.15.97:6837","nonce":663657264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T03:46:48.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 -- 172.21.15.97:0/2550909829 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd0006ace0 msgr2=0x7fdd0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:48.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd0006ace0 0x7fdd0006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd2015eaf0 tx=0x7fdd08009040 comp rx=0 tx=0).stop 2023-07-06T03:46:48.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 -- 172.21.15.97:0/2550909829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd200b50b0 msgr2=0x7fdd20158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:48.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd200b50b0 0x7fdd20158bc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000b4a0 tx=0x7fdd1000b7b0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 -- 172.21.15.97:0/2550909829 shutdown_connections 2023-07-06T03:46:48.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd0006ace0 0x7fdd0006d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd2015ca10 0x7fdd2015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd200b50b0 0x7fdd20158bc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 --2- 172.21.15.97:0/2550909829 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd200cf3c0 0x7fdd2014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:48.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 -- 172.21.15.97:0/2550909829 >> 172.21.15.97:0/2550909829 conn(0x7fdd2006a920 msgr2=0x7fdd2015f770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:48.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 -- 172.21.15.97:0/2550909829 shutdown_connections 2023-07-06T03:46:48.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:48.602+0000 7fdd255ff700 1 -- 172.21.15.97:0/2550909829 wait complete. 2023-07-06T03:46:48.605 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2023-07-06T03:46:49.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T03:46:37.331190+0000', 'modified': '2023-07-06T03:46:40.864523+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5301}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5301': {'gid': 5301, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6837/663657264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 663657264}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 663657264}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T03:46:49.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T03:46:49.026 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-06T03:46:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.220+0000 7f9a031ff700 1 Processor -- start 2023-07-06T03:46:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.220+0000 7f9a031ff700 1 -- start start 2023-07-06T03:46:49.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.221+0000 7f9a031ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f99fc15c610 0x7f99fc0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.221+0000 7f9a031ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99fc06b360 0x7f99fc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.221+0000 7f9a031ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99fc0cff90 con 0x7f99fc0d2210 2023-07-06T03:46:49.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.221+0000 7f9a031ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99fc0d0780 con 0x7f99fc15c610 2023-07-06T03:46:49.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.221+0000 7f9a031ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc15c9e0 con 0x7f99fc06b360 2023-07-06T03:46:49.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.222+0000 7f9a00f9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f99fc15c610 0x7f99fc0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.222+0000 7f9a00f9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f99fc15c610 0x7f99fc0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58214/0 (socket says 172.21.15.97:58214) 2023-07-06T03:46:49.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.222+0000 7f9a00f9b700 1 -- 172.21.15.97:0/1009217531 learned_addr learned my addr 172.21.15.97:0/1009217531 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:49.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.222+0000 7f99fbfff700 1 --2- 172.21.15.97:0/1009217531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99fc06b360 0x7f99fc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.222+0000 7f9a00f9b700 1 -- 172.21.15.97:0/1009217531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99fc06b360 msgr2=0x7f99fc0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:49.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.222+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/1009217531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99fc06b360 0x7f99fc0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.222+0000 7f9a00f9b700 1 -- 172.21.15.97:0/1009217531 >> v1:172.21.15.146:6789/0 conn(0x7f99fc0d2210 legacy=0x7f99fc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:49.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.222+0000 7f9a00f9b700 1 -- 172.21.15.97:0/1009217531 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc0b5650 con 0x7f99fc15c610 2023-07-06T03:46:49.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.222+0000 7f99fbfff700 1 --2- 172.21.15.97:0/1009217531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99fc06b360 0x7f99fc0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:49.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.222+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/1009217531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99fc15c610 0x7f99fc0b47b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009920 tx=0x7f99f0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756c4d2e5ccedba2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:49.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.222+0000 7f99fb7fe700 1 -- 172.21.15.97:0/1009217531 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0004230 con 0x7f99fc15c610 2023-07-06T03:46:49.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.223+0000 7f99fb7fe700 1 -- 172.21.15.97:0/1009217531 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f0004390 con 0x7f99fc15c610 2023-07-06T03:46:49.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.223+0000 7f99fb7fe700 1 -- 172.21.15.97:0/1009217531 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0009e70 con 0x7f99fc15c610 2023-07-06T03:46:49.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.223+0000 7f9a031ff700 1 -- 172.21.15.97:0/1009217531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99fc15c610 msgr2=0x7f99fc0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.223+0000 7f9a031ff700 1 --2- 172.21.15.97:0/1009217531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99fc15c610 0x7f99fc0b47b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009920 tx=0x7f99f0009c30 comp rx=0 tx=0).stop 2023-07-06T03:46:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.223+0000 7f9a031ff700 1 -- 172.21.15.97:0/1009217531 shutdown_connections 2023-07-06T03:46:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.223+0000 7f9a031ff700 1 --2- 172.21.15.97:0/1009217531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99fc15c610 0x7f99fc0b47b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.223+0000 7f9a031ff700 1 --2- 172.21.15.97:0/1009217531 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99fc06b360 0x7f99fc0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.223+0000 7f9a031ff700 1 -- 172.21.15.97:0/1009217531 >> 172.21.15.97:0/1009217531 conn(0x7f99fc06a6c0 msgr2=0x7f99fc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.223+0000 7f9a031ff700 1 -- 172.21.15.97:0/1009217531 shutdown_connections 2023-07-06T03:46:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.223+0000 7f9a031ff700 1 -- 172.21.15.97:0/1009217531 wait complete. 2023-07-06T03:46:49.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.225+0000 7f9a031ff700 1 Processor -- start 2023-07-06T03:46:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.225+0000 7f9a031ff700 1 -- start start 2023-07-06T03:46:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.226+0000 7f9a031ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99fc06b360 0x7f99fc14b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.226+0000 7f9a031ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99fc0d2210 0x7f99fc0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.226+0000 7f9a031ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99fc15c610 0x7f99fc0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.226+0000 7f9a031ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc14b700 con 0x7f99fc15c610 2023-07-06T03:46:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.226+0000 7f9a031ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0caf40 con 0x7f99fc0d2210 2023-07-06T03:46:49.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.226+0000 7f9a031ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0c9ea0 con 0x7f99fc06b360 2023-07-06T03:46:49.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.226+0000 7f9a0179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99fc15c610 0x7f99fc0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.226+0000 7f9a0179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99fc15c610 0x7f99fc0b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58226/0 (socket says 172.21.15.97:58226) 2023-07-06T03:46:49.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.226+0000 7f9a0179c700 1 -- 172.21.15.97:0/830847849 learned_addr learned my addr 172.21.15.97:0/830847849 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:49.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.226+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99fc06b360 0x7f99fc14b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f99fbfff700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99fc0d2210 0x7f99fc0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f9a0179c700 1 -- 172.21.15.97:0/830847849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99fc06b360 msgr2=0x7f99fc14b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f9a0179c700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99fc06b360 0x7f99fc14b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f9a0179c700 1 -- 172.21.15.97:0/830847849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99fc0d2210 msgr2=0x7f99fc0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f9a0179c700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99fc0d2210 0x7f99fc0b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f9a0179c700 1 -- 172.21.15.97:0/830847849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f0009550 con 0x7f99fc15c610 2023-07-06T03:46:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f9a0179c700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99fc15c610 0x7f99fc0b3d20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f99f400cc10 tx=0x7f99f400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f99f97fa700 1 -- 172.21.15.97:0/830847849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f400aa60 con 0x7f99fc15c610 2023-07-06T03:46:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f99f97fa700 1 -- 172.21.15.97:0/830847849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f400abc0 con 0x7f99fc15c610 2023-07-06T03:46:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f9a031ff700 1 -- 172.21.15.97:0/830847849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc0ca180 con 0x7f99fc15c610 2023-07-06T03:46:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f99f97fa700 1 -- 172.21.15.97:0/830847849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4012420 con 0x7f99fc15c610 2023-07-06T03:46:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.227+0000 7f9a031ff700 1 -- 172.21.15.97:0/830847849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc0ca740 con 0x7f99fc15c610 2023-07-06T03:46:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.229+0000 7f99f97fa700 1 -- 172.21.15.97:0/830847849 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f99f4012580 con 0x7f99fc15c610 2023-07-06T03:46:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.231+0000 7f99f97fa700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99e006ace0 0x7f99e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.231+0000 7f99f97fa700 1 -- 172.21.15.97:0/830847849 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99f40897e0 con 0x7f99fc15c610 2023-07-06T03:46:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.231+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99e006ace0 0x7f99e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.231+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99e006ace0 0x7f99e006d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009870 tx=0x7f99f0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.232+0000 7f9a16b14b80 1 Processor -- start 2023-07-06T03:46:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.232+0000 7f9a16b14b80 1 -- start start 2023-07-06T03:46:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.233+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c02ad50 0x55cc5c02b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.233+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c030190 0x55cc5c04ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.233+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c008110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.233+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5bf7cea0 con 0x55cc5c005b50 2023-07-06T03:46:49.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.233+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5bef0600 con 0x55cc5c030190 2023-07-06T03:46:49.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.233+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc5bf5f910 con 0x55cc5c02ad50 2023-07-06T03:46:49.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.233+0000 7f9a0179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c008110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f9a0179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c008110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58242/0 (socket says 172.21.15.97:58242) 2023-07-06T03:46:49.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f9a00f9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c02ad50 0x55cc5c02b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f9a0179c700 1 -- 172.21.15.97:0/1819257015 learned_addr learned my addr 172.21.15.97:0/1819257015 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f99fbfff700 1 --2- 172.21.15.97:0/1819257015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c030190 0x55cc5c04ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f9a0179c700 1 -- 172.21.15.97:0/1819257015 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c02ad50 msgr2=0x55cc5c02b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f9a0179c700 1 --2- 172.21.15.97:0/1819257015 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c02ad50 0x55cc5c02b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f9a0179c700 1 -- 172.21.15.97:0/1819257015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c030190 msgr2=0x55cc5c04ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f9a0179c700 1 --2- 172.21.15.97:0/1819257015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c030190 0x55cc5c04ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f9a0179c700 1 -- 172.21.15.97:0/1819257015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc5bd4c790 con 0x55cc5c005b50 2023-07-06T03:46:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f9a0179c700 1 --2- 172.21.15.97:0/1819257015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c008110 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f99f4087a50 tx=0x7f99f400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7a4d5dc6f54b43 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f99e67fc700 1 -- 172.21.15.97:0/1819257015 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4056850 con 0x55cc5c005b50 2023-07-06T03:46:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f99e67fc700 1 -- 172.21.15.97:0/1819257015 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f4010df0 con 0x55cc5c005b50 2023-07-06T03:46:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.234+0000 7f99e67fc700 1 -- 172.21.15.97:0/1819257015 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f405e0b0 con 0x55cc5c005b50 2023-07-06T03:46:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.235+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1819257015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 msgr2=0x55cc5c008110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.235+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/1819257015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c008110 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f99f4087a50 tx=0x7f99f400cf90 comp rx=0 tx=0).stop 2023-07-06T03:46:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.235+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1819257015 shutdown_connections 2023-07-06T03:46:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.235+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/1819257015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c008110 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.235+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/1819257015 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c02ad50 0x55cc5c02b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.235+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/1819257015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c030190 0x55cc5c04ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.235+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1819257015 >> 172.21.15.97:0/1819257015 conn(0x55cc5bffd310 msgr2=0x55cc5c04d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.235+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1819257015 shutdown_connections 2023-07-06T03:46:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.235+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1819257015 wait complete. 2023-07-06T03:46:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.236+0000 7f9a16b14b80 1 -- start start 2023-07-06T03:46:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.236+0000 7f9a16b14b80 1 Processor -- start 2023-07-06T03:46:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c005b50 0x55cc5c0448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c02ad50 0x55cc5c044e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c030190 0x55cc5c0315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5bf58be0 con 0x55cc5c030190 2023-07-06T03:46:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5bf165b0 con 0x55cc5c005b50 2023-07-06T03:46:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc5befc910 con 0x55cc5c02ad50 2023-07-06T03:46:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f9a16b14b80 10 client.0 _ll_register_callbacks cb 0x55cc5bf14b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:46:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f9a0179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c030190 0x55cc5c0315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f9a0179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c030190 0x55cc5c0315e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58250/0 (socket says 172.21.15.97:58250) 2023-07-06T03:46:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f9a0179c700 1 -- 172.21.15.97:0/1375612669 learned_addr learned my addr 172.21.15.97:0/1375612669 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c005b50 0x55cc5c0448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.238+0000 7f99fbfff700 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c02ad50 0x55cc5c044e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f9a0179c700 1 -- 172.21.15.97:0/1375612669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c02ad50 msgr2=0x55cc5c044e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f9a0179c700 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c02ad50 0x55cc5c044e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f9a0179c700 1 -- 172.21.15.97:0/1375612669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c005b50 msgr2=0x55cc5c0448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f9a0179c700 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c005b50 0x55cc5c0448c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f9a0179c700 1 -- 172.21.15.97:0/1375612669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc0ca740 con 0x55cc5c030190 2023-07-06T03:46:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f9a0179c700 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c030190 0x55cc5c0315e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f99f4058440 tx=0x7f99f4003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4ac60550f362f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f99db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:46:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f99db7fe700 1 -- 172.21.15.97:0/1375612669 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f405c170 con 0x55cc5c030190 2023-07-06T03:46:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1375612669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc5bffa920 con 0x55cc5c030190 2023-07-06T03:46:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f99db7fe700 1 -- 172.21.15.97:0/1375612669 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99f405e0b0 con 0x55cc5c030190 2023-07-06T03:46:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f99db7fe700 1 -- 172.21.15.97:0/1375612669 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f405e270 con 0x55cc5c030190 2023-07-06T03:46:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f9a16b14b80 10 client.5508 fetch_fsmap learned FSMap version 16 2023-07-06T03:46:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1375612669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55cc5be390a0 con 0x55cc5c030190 2023-07-06T03:46:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.239+0000 7f99db7fe700 1 -- 172.21.15.97:0/1375612669 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cc5bffa920 con 0x55cc5c030190 2023-07-06T03:46:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.240+0000 7f99db7fe700 1 -- 172.21.15.97:0/1375612669 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f99f407a020 con 0x55cc5c030190 2023-07-06T03:46:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.240+0000 7f9a16b14b80 10 client.5508 fetch_fsmap finished waiting for FSMap version 16 2023-07-06T03:46:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.240+0000 7f9a16b14b80 10 client.5508 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T03:46:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.240+0000 7f9a16b14b80 20 client.5508 populate_metadata read hostname 'smithi097' 2023-07-06T03:46:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.240+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.97:6836/663657264,v1:172.21.15.97:6837/663657264] conn(0x55cc5c060170 0x55cc5c060540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.240+0000 7f9a16b14b80 4 client.5508 mds_command: new command op to 5301 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T03:46:49.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.240+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1375612669 --> [v2:172.21.15.97:6836/663657264,v1:172.21.15.97:6837/663657264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc5bfd70d0 con 0x55cc5c060170 2023-07-06T03:46:49.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.240+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.97:6836/663657264,v1:172.21.15.97:6837/663657264] conn(0x55cc5c060170 0x55cc5c060540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.240+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.97:6836/663657264,v1:172.21.15.97:6837/663657264] conn(0x55cc5c060170 0x55cc5c060540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a934f38181ec62b0 server_cookie=264398d226be4d40 in_seq=0 out_seq=0 2023-07-06T03:46:49.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.240+0000 7f99db7fe700 10 client.5508 ms_handle_connect on v2:172.21.15.97:6836/663657264 2023-07-06T03:46:49.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.241+0000 7f99db7fe700 1 -- 172.21.15.97:0/1375612669 <== mds.0 v2:172.21.15.97:6836/663657264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cc5bfd70d0 con 0x55cc5c060170 2023-07-06T03:46:49.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.241+0000 7f99db7fe700 10 client.5508 handle_command_reply: tid=0 2023-07-06T03:46:49.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.241+0000 7f9a16b14b80 1 client.5508 shutdown 2023-07-06T03:46:49.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.241+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1375612669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c030190 msgr2=0x55cc5c0315e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.241+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c030190 0x55cc5c0315e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f99f4058440 tx=0x7f99f4003b70 comp rx=0 tx=0).stop 2023-07-06T03:46:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.242+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1375612669 shutdown_connections 2023-07-06T03:46:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.242+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.97:6836/663657264,v1:172.21.15.97:6837/663657264] conn(0x55cc5c060170 0x55cc5c060540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.242+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c030190 0x55cc5c0315e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.242+0000 7f99db7fe700 0 client.5508 ms_handle_reset on v2:172.21.15.97:6836/663657264 2023-07-06T03:46:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.242+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c005b50 0x55cc5c0448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.242+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/1375612669 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c02ad50 0x55cc5c044e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.242+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1375612669 >> 172.21.15.97:0/1375612669 conn(0x55cc5bffd310 msgr2=0x55cc5c028b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:49.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.242+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1375612669 shutdown_connections 2023-07-06T03:46:49.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.242+0000 7f9a16b14b80 1 -- 172.21.15.97:0/1375612669 wait complete. 2023-07-06T03:46:49.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.242+0000 7f9a16b14b80 20 client.5508 trim_cache size 0 max 16384 2023-07-06T03:46:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.260+0000 7f9a16b14b80 1 Processor -- start 2023-07-06T03:46:49.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.260+0000 7f9a16b14b80 1 -- start start 2023-07-06T03:46:49.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c02ad50 0x55cc5c045f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c030190 0x55cc5c046490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5bd4d390 con 0x55cc5c005b50 2023-07-06T03:46:49.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5c0092d0 con 0x55cc5c02ad50 2023-07-06T03:46:49.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc5c002570 con 0x55cc5c030190 2023-07-06T03:46:49.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a00f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a00f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0362d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58262/0 (socket says 172.21.15.97:58262) 2023-07-06T03:46:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a00f9b700 1 -- 172.21.15.97:0/948506302 learned_addr learned my addr 172.21.15.97:0/948506302 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a0179c700 1 --2- 172.21.15.97:0/948506302 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c030190 0x55cc5c046490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f99fbfff700 1 --2- 172.21.15.97:0/948506302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c02ad50 0x55cc5c045f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a00f9b700 1 -- 172.21.15.97:0/948506302 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c030190 msgr2=0x55cc5c046490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/948506302 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c030190 0x55cc5c046490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a00f9b700 1 -- 172.21.15.97:0/948506302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c02ad50 msgr2=0x55cc5c045f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/948506302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c02ad50 0x55cc5c045f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.261+0000 7f9a00f9b700 1 -- 172.21.15.97:0/948506302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc5bf1eb10 con 0x55cc5c005b50 2023-07-06T03:46:49.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.262+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/948506302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0362d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009300 tx=0x7f99f0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61632981e4e8f012 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:49.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.262+0000 7f99e6ffd700 1 -- 172.21.15.97:0/948506302 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f002c040 con 0x55cc5c005b50 2023-07-06T03:46:49.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.262+0000 7f99e6ffd700 1 -- 172.21.15.97:0/948506302 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f0031950 con 0x55cc5c005b50 2023-07-06T03:46:49.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.262+0000 7f99e6ffd700 1 -- 172.21.15.97:0/948506302 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0031cc0 con 0x55cc5c005b50 2023-07-06T03:46:49.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.262+0000 7f9a16b14b80 1 -- 172.21.15.97:0/948506302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 msgr2=0x55cc5c0362d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.262+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/948506302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0362d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009300 tx=0x7f99f0010000 comp rx=0 tx=0).stop 2023-07-06T03:46:49.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.262+0000 7f9a16b14b80 1 -- 172.21.15.97:0/948506302 shutdown_connections 2023-07-06T03:46:49.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.262+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/948506302 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c030190 0x55cc5c046490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.262+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/948506302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0362d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.262+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/948506302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c02ad50 0x55cc5c045f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.262+0000 7f9a16b14b80 1 -- 172.21.15.97:0/948506302 >> 172.21.15.97:0/948506302 conn(0x55cc5bffd310 msgr2=0x55cc5c02f980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.263+0000 7f9a16b14b80 1 -- 172.21.15.97:0/948506302 shutdown_connections 2023-07-06T03:46:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.263+0000 7f9a16b14b80 1 -- 172.21.15.97:0/948506302 wait complete. 2023-07-06T03:46:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.263+0000 7f9a16b14b80 1 -- start start 2023-07-06T03:46:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.263+0000 7f9a16b14b80 1 Processor -- start 2023-07-06T03:46:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.264+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.264+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c02ad50 0x55cc5c030e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.264+0000 7f9a16b14b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c030190 0x55cc5c0313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.264+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5c0245d0 con 0x55cc5c005b50 2023-07-06T03:46:49.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.264+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5c002dc0 con 0x55cc5c02ad50 2023-07-06T03:46:49.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.264+0000 7f9a16b14b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc5c05eca0 con 0x55cc5c030190 2023-07-06T03:46:49.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.264+0000 7f9a16b14b80 10 client.0 _ll_register_callbacks cb 0x7f99e800a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:46:49.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f9a00f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f9a00f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58266/0 (socket says 172.21.15.97:58266) 2023-07-06T03:46:49.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f9a00f9b700 1 -- 172.21.15.97:0/844170575 learned_addr learned my addr 172.21.15.97:0/844170575 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:49.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f99fbfff700 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c02ad50 0x55cc5c030e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f9a0179c700 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c030190 0x55cc5c0313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f9a00f9b700 1 -- 172.21.15.97:0/844170575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c030190 msgr2=0x55cc5c0313c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c030190 0x55cc5c0313c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f9a00f9b700 1 -- 172.21.15.97:0/844170575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c02ad50 msgr2=0x55cc5c030e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c02ad50 0x55cc5c030e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f9a00f9b700 1 -- 172.21.15.97:0/844170575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc0b5650 con 0x55cc5c005b50 2023-07-06T03:46:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f9a0179c700 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c030190 0x55cc5c0313c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:49.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f99fbfff700 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c02ad50 0x55cc5c030e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:46:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f9a00f9b700 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0b5050 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c010 tx=0x7f99f000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453dbf4d9cbf4286 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.265+0000 7f99db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:46:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f99db7fe700 1 -- 172.21.15.97:0/844170575 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f002c070 con 0x55cc5c005b50 2023-07-06T03:46:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f99db7fe700 1 -- 172.21.15.97:0/844170575 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99f00036a0 con 0x55cc5c005b50 2023-07-06T03:46:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f9a16b14b80 1 -- 172.21.15.97:0/844170575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99f4010df0 con 0x55cc5c005b50 2023-07-06T03:46:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f99db7fe700 1 -- 172.21.15.97:0/844170575 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f0003860 con 0x55cc5c005b50 2023-07-06T03:46:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f9a16b14b80 10 client.5514 fetch_fsmap learned FSMap version 16 2023-07-06T03:46:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f9a16b14b80 1 -- 172.21.15.97:0/844170575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55cc5bdad1d0 con 0x55cc5c005b50 2023-07-06T03:46:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f99db7fe700 1 -- 172.21.15.97:0/844170575 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f0044020 con 0x55cc5c005b50 2023-07-06T03:46:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f99db7fe700 1 -- 172.21.15.97:0/844170575 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f99f0038070 con 0x55cc5c005b50 2023-07-06T03:46:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f9a16b14b80 10 client.5514 fetch_fsmap finished waiting for FSMap version 16 2023-07-06T03:46:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f9a16b14b80 10 client.5514 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T03:46:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f9a16b14b80 20 client.5514 populate_metadata read hostname 'smithi097' 2023-07-06T03:46:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.97:6836/663657264,v1:172.21.15.97:6837/663657264] conn(0x55cc5c028790 0x55cc5c028b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f9a16b14b80 4 client.5514 mds_command: new command op to 5301 tid=0[{"prefix": "status"}] 2023-07-06T03:46:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f9a16b14b80 1 -- 172.21.15.97:0/844170575 --> [v2:172.21.15.97:6836/663657264,v1:172.21.15.97:6837/663657264] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cc5bf51d20 con 0x55cc5c028790 2023-07-06T03:46:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.266+0000 7f99fbfff700 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.97:6836/663657264,v1:172.21.15.97:6837/663657264] conn(0x55cc5c028790 0x55cc5c028b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f99fbfff700 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.97:6836/663657264,v1:172.21.15.97:6837/663657264] conn(0x55cc5c028790 0x55cc5c028b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef57ad470d3a3b4 server_cookie=15b8e9c8e715c709 in_seq=0 out_seq=0 2023-07-06T03:46:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f99db7fe700 10 client.5514 ms_handle_connect on v2:172.21.15.97:6836/663657264 2023-07-06T03:46:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f99db7fe700 1 -- 172.21.15.97:0/844170575 <== mds.0 v2:172.21.15.97:6836/663657264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+317 (crc 0 0 0) 0x55cc5bf51d20 con 0x55cc5c028790 2023-07-06T03:46:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f99db7fe700 10 client.5514 handle_command_reply: tid=0 2023-07-06T03:46:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f9a16b14b80 1 client.5514 shutdown 2023-07-06T03:46:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f9a16b14b80 1 -- 172.21.15.97:0/844170575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 msgr2=0x55cc5c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0b5050 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c010 tx=0x7f99f000e000 comp rx=0 tx=0).stop 2023-07-06T03:46:49.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f9a16b14b80 1 -- 172.21.15.97:0/844170575 shutdown_connections 2023-07-06T03:46:49.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.97:6836/663657264,v1:172.21.15.97:6837/663657264] conn(0x55cc5c028790 0x55cc5c028b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55cc5c030190 0x55cc5c0313c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc5c005b50 0x55cc5c0b5050 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f99db7fe700 0 client.5514 ms_handle_reset on v2:172.21.15.97:6836/663657264 2023-07-06T03:46:49.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f9a16b14b80 1 --2- 172.21.15.97:0/844170575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cc5c02ad50 0x55cc5c030e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.267+0000 7f9a16b14b80 1 -- 172.21.15.97:0/844170575 >> 172.21.15.97:0/844170575 conn(0x55cc5bffd310 msgr2=0x55cc5c066570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:49.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a16b14b80 1 -- 172.21.15.97:0/844170575 shutdown_connections 2023-07-06T03:46:49.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a16b14b80 1 -- 172.21.15.97:0/844170575 wait complete. 2023-07-06T03:46:49.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a16b14b80 20 client.5514 trim_cache size 0 max 16384 2023-07-06T03:46:49.279 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:46:49.279 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36", 2023-07-06T03:46:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-07-06T03:46:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5301, 2023-07-06T03:46:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-07-06T03:46:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-07-06T03:46:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-07-06T03:46:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 11.422929688, 2023-07-06T03:46:49.280 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 16, 2023-07-06T03:46:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 34, 2023-07-06T03:46:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 34, 2023-07-06T03:46:49.281 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 20.2333666 2023-07-06T03:46:49.281 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:46:49.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a031ff700 1 -- 172.21.15.97:0/830847849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99e006ace0 msgr2=0x7f99e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a031ff700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99e006ace0 0x7f99e006d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009870 tx=0x7f99f0008030 comp rx=0 tx=0).stop 2023-07-06T03:46:49.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a031ff700 1 -- 172.21.15.97:0/830847849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99fc15c610 msgr2=0x7f99fc0b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a031ff700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99fc15c610 0x7f99fc0b3d20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f99f400cc10 tx=0x7f99f400cf20 comp rx=0 tx=0).stop 2023-07-06T03:46:49.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a031ff700 1 -- 172.21.15.97:0/830847849 shutdown_connections 2023-07-06T03:46:49.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a031ff700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99e006ace0 0x7f99e006d190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a031ff700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99fc15c610 0x7f99fc0b3d20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a031ff700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99fc06b360 0x7f99fc14b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a031ff700 1 --2- 172.21.15.97:0/830847849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99fc0d2210 0x7f99fc0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a031ff700 1 -- 172.21.15.97:0/830847849 >> 172.21.15.97:0/830847849 conn(0x7f99fc06a6c0 msgr2=0x7f99fc0b6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.268+0000 7f9a031ff700 1 -- 172.21.15.97:0/830847849 shutdown_connections 2023-07-06T03:46:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.269+0000 7f9a031ff700 1 -- 172.21.15.97:0/830847849 wait complete. 2023-07-06T03:46:49.288 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:46:49.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.479+0000 7f8c143ff700 1 Processor -- start 2023-07-06T03:46:49.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.479+0000 7f8c143ff700 1 -- start start 2023-07-06T03:46:49.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.480+0000 7f8c143ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c0c158f20 0x7f8c0c0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.480+0000 7f8c143ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c0c1592f0 0x7f8c0c0b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.480+0000 7f8c143ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0c15cc40 con 0x7f8c0c06d860 2023-07-06T03:46:49.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.480+0000 7f8c143ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c0c15cf20 con 0x7f8c0c158f20 2023-07-06T03:46:49.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.480+0000 7f8c143ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c0c15d200 con 0x7f8c0c1592f0 2023-07-06T03:46:49.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.480+0000 7f8c1219b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c0c158f20 0x7f8c0c0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.480+0000 7f8c1299c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8c0c06d860 0x7f8c0c0cd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50874/0 (socket says 172.21.15.97:50874) 2023-07-06T03:46:49.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.480+0000 7f8c1299c700 1 -- 172.21.15.97:0/2894284294 learned_addr learned my addr 172.21.15.97:0/2894284294 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:46:49.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.480+0000 7f8c1219b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c0c158f20 0x7f8c0c0caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58282/0 (socket says 172.21.15.97:58282) 2023-07-06T03:46:49.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.480+0000 7f8c1199a700 1 --2- 172.21.15.97:0/2894284294 >> v2:172.21.15.146:3300/0 conn(0x7f8c0c1592f0 0x7f8c0c0b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.481+0000 7f8c1219b700 1 -- 172.21.15.97:0/2894284294 >> v2:172.21.15.146:3300/0 conn(0x7f8c0c1592f0 msgr2=0x7f8c0c0b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.481+0000 7f8c1219b700 1 --2- 172.21.15.97:0/2894284294 >> v2:172.21.15.146:3300/0 conn(0x7f8c0c1592f0 0x7f8c0c0b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.481+0000 7f8c1219b700 1 -- 172.21.15.97:0/2894284294 >> v1:172.21.15.97:6789/0 conn(0x7f8c0c06d860 legacy=0x7f8c0c0cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.481+0000 7f8c1219b700 1 -- 172.21.15.97:0/2894284294 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c15d4e0 con 0x7f8c0c158f20 2023-07-06T03:46:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.481+0000 7f8c11199700 1 -- 172.21.15.97:0/2894284294 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919540398 0 0) 0x7f8c0c15cc40 con 0x7f8c0c06d860 2023-07-06T03:46:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.481+0000 7f8c1219b700 1 --2- 172.21.15.97:0/2894284294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c0c158f20 0x7f8c0c0caac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0b4b90 tx=0x7f8c08009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a360145964547e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.481+0000 7f8c11199700 1 -- 172.21.15.97:0/2894284294 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08011830 con 0x7f8c0c158f20 2023-07-06T03:46:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.481+0000 7f8c11199700 1 -- 172.21.15.97:0/2894284294 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c08011990 con 0x7f8c0c158f20 2023-07-06T03:46:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.481+0000 7f8c11199700 1 -- 172.21.15.97:0/2894284294 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08011d20 con 0x7f8c0c158f20 2023-07-06T03:46:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.482+0000 7f8c143ff700 1 -- 172.21.15.97:0/2894284294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c0c158f20 msgr2=0x7f8c0c0caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.482+0000 7f8c143ff700 1 --2- 172.21.15.97:0/2894284294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c0c158f20 0x7f8c0c0caac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0b4b90 tx=0x7f8c08009b20 comp rx=0 tx=0).stop 2023-07-06T03:46:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.482+0000 7f8c143ff700 1 -- 172.21.15.97:0/2894284294 shutdown_connections 2023-07-06T03:46:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.482+0000 7f8c143ff700 1 --2- 172.21.15.97:0/2894284294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c0c158f20 0x7f8c0c0caac0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.482+0000 7f8c143ff700 1 --2- 172.21.15.97:0/2894284294 >> v2:172.21.15.146:3300/0 conn(0x7f8c0c1592f0 0x7f8c0c0b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.482+0000 7f8c143ff700 1 -- 172.21.15.97:0/2894284294 >> 172.21.15.97:0/2894284294 conn(0x7f8c0c0ce8c0 msgr2=0x7f8c0c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.482+0000 7f8c143ff700 1 -- 172.21.15.97:0/2894284294 shutdown_connections 2023-07-06T03:46:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.482+0000 7f8c143ff700 1 -- 172.21.15.97:0/2894284294 wait complete. 2023-07-06T03:46:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.484+0000 7f8c143ff700 1 Processor -- start 2023-07-06T03:46:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.484+0000 7f8c143ff700 1 -- start start 2023-07-06T03:46:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c143ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c0c06d860 0x7f8c0c150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c143ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c0c158f20 0x7f8c0c1486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c143ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c1592f0 0x7f8c0c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c143ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c14d2a0 con 0x7f8c0c06d860 2023-07-06T03:46:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c143ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c150aa0 con 0x7f8c0c1592f0 2023-07-06T03:46:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c143ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c148be0 con 0x7f8c0c158f20 2023-07-06T03:46:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c1219b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c0c06d860 0x7f8c0c150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c1219b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c0c06d860 0x7f8c0c150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58286/0 (socket says 172.21.15.97:58286) 2023-07-06T03:46:49.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c1219b700 1 -- 172.21.15.97:0/53609219 learned_addr learned my addr 172.21.15.97:0/53609219 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:49.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c1199a700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c0c158f20 0x7f8c0c1486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c1219b700 1 -- 172.21.15.97:0/53609219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c0c158f20 msgr2=0x7f8c0c1486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c1299c700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c1592f0 0x7f8c0c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c1219b700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c0c158f20 0x7f8c0c1486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c1219b700 1 -- 172.21.15.97:0/53609219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c1592f0 msgr2=0x7f8c0c0b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c1219b700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c1592f0 0x7f8c0c0b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c1219b700 1 -- 172.21.15.97:0/53609219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c080094a0 con 0x7f8c0c06d860 2023-07-06T03:46:49.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.485+0000 7f8c1299c700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c1592f0 0x7f8c0c0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:49.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.486+0000 7f8c1219b700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c0c06d860 0x7f8c0c150560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8c080090b0 tx=0x7f8c08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:49.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.486+0000 7f8c02ffd700 1 -- 172.21.15.97:0/53609219 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c080038a0 con 0x7f8c0c06d860 2023-07-06T03:46:49.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.486+0000 7f8c02ffd700 1 -- 172.21.15.97:0/53609219 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0801f070 con 0x7f8c0c06d860 2023-07-06T03:46:49.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.486+0000 7f8c02ffd700 1 -- 172.21.15.97:0/53609219 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08003c20 con 0x7f8c0c06d860 2023-07-06T03:46:49.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.486+0000 7f8c143ff700 1 -- 172.21.15.97:0/53609219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0c14ea40 con 0x7f8c0c06d860 2023-07-06T03:46:49.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.486+0000 7f8c143ff700 1 -- 172.21.15.97:0/53609219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0c14cc50 con 0x7f8c0c06d860 2023-07-06T03:46:49.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.487+0000 7f8c143ff700 1 -- 172.21.15.97:0/53609219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0c0b33c0 con 0x7f8c0c06d860 2023-07-06T03:46:49.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.488+0000 7f8c02ffd700 1 -- 172.21.15.97:0/53609219 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c0801a430 con 0x7f8c0c06d860 2023-07-06T03:46:49.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.489+0000 7f8c02ffd700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8bf006ace0 0x7f8bf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:49.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.489+0000 7f8c02ffd700 1 -- 172.21.15.97:0/53609219 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8c08021070 con 0x7f8c0c06d860 2023-07-06T03:46:49.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.489+0000 7f8c1199a700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8bf006ace0 0x7f8bf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:49.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.490+0000 7f8c1199a700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8bf006ace0 0x7f8bf006d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c154480 tx=0x7f8bfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:49.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.494+0000 7f8c02ffd700 1 -- 172.21.15.97:0/53609219 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8c08055eb0 con 0x7f8c0c06d860 2023-07-06T03:46:49.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.732+0000 7f8c143ff700 1 -- 172.21.15.97:0/53609219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8c0c04e340 con 0x7f8c0c06d860 2023-07-06T03:46:49.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.732+0000 7f8c02ffd700 1 -- 172.21.15.97:0/53609219 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8c08015070 con 0x7f8c0c06d860 2023-07-06T03:46:49.735 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:46:49.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.734+0000 7f8c143ff700 1 -- 172.21.15.97:0/53609219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8bf006ace0 msgr2=0x7f8bf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.734+0000 7f8c143ff700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8bf006ace0 0x7f8bf006d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c154480 tx=0x7f8bfc008040 comp rx=0 tx=0).stop 2023-07-06T03:46:49.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.734+0000 7f8c143ff700 1 -- 172.21.15.97:0/53609219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c0c06d860 msgr2=0x7f8c0c150560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:49.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.734+0000 7f8c143ff700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c0c06d860 0x7f8c0c150560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8c080090b0 tx=0x7f8c08004750 comp rx=0 tx=0).stop 2023-07-06T03:46:49.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.734+0000 7f8c143ff700 1 -- 172.21.15.97:0/53609219 shutdown_connections 2023-07-06T03:46:49.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.734+0000 7f8c143ff700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8bf006ace0 0x7f8bf006d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.734+0000 7f8c143ff700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c0c1592f0 0x7f8c0c0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.734+0000 7f8c143ff700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c0c06d860 0x7f8c0c150560 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.734+0000 7f8c143ff700 1 --2- 172.21.15.97:0/53609219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c0c158f20 0x7f8c0c1486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:49.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.734+0000 7f8c143ff700 1 -- 172.21.15.97:0/53609219 >> 172.21.15.97:0/53609219 conn(0x7f8c0c0ce8c0 msgr2=0x7f8c0c0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:49.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.735+0000 7f8c143ff700 1 -- 172.21.15.97:0/53609219 shutdown_connections 2023-07-06T03:46:49.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:49.735+0000 7f8c143ff700 1 -- 172.21.15.97:0/53609219 wait complete. 2023-07-06T03:46:49.755 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:46:49.755 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:46:49.755 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:46:49.756 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:46:49.756 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:46:49.756 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:46:49.757 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:46:49.757 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:46:49.757 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:46:49.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:49.758 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:46:49.804 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:46:49.804 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:46:49.805 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:46:49.805 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:49.805 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:46:49.805 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:49.805 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:46:49.805 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:46:49.805 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:46:49.805 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:46:49.806 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:46:49.806 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:46:49.806 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:46:49.806 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:46:49.806 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:46:49.806 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:46:49.806 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:46:49.807 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42059sec preferred_lft 42059sec 2023-07-06T03:46:49.807 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:46:49.807 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:49.807 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:46:49.807 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:46:49.807 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:46:49.808 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:49.808 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:46:49.808 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:49.808 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:46:49.808 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:46:49.808 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:46:49.808 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:49.809 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:46:49.809 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:46:49.809 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:46:49.809 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:49.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:49.810 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:46:49.917 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:46:49.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:49.920 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:46:49.969 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:46:49.969 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:46:49.969 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:46:49.970 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:46:49.970 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:46:49.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:49.970 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:46:49.971 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:46:49.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:46:49.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:46:49.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:46:49.972 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:46:50.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:50.139 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:46:50.184 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:46:50.184 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:46:50.185 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:46:50.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:50.186 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:46:50.295 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:46:50.295 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:46:50.296 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:46:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:46:50.342 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:46:50.342 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:46:50.426 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:46:50.535 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:46:50.536 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:46:50.603 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:46:50.606 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:46:50.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:50.607 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:46:50.652 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:46:50.653 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:46:50.697 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:46:50.766 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:46:50.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:46:50.836 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:46:50.839 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:46:50.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:50.840 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:46:50.862 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:46:50.860+0000 7f1330cac5c0 -1 init, newargv = 0x556bd5f35350 newargc=15 2023-07-06T03:46:50.863 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[129554]: starting ceph client 2023-07-06T03:46:50.870 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[129554]: starting fuse 2023-07-06T03:46:50.889 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:46:50.889 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:46:50.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:46:50.890 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:46:50.890 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:46:50.891 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:46:50.891 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:46:50.891 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:50.892 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:46:50.892 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:46:50.892 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:46:50.893 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:46:50.894 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:50.894 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:46:50.894 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:46:50.895 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:46:50.895 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:46:50.895 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:50.896 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:46:50.896 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:46:50.896 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:46:50.897 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:46:50.897 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:46:50.897 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:46:50.898 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:46:50.898 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:46:50.898 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:50.899 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:46:50.899 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:46:51.073 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.129554.asok 2023-07-06T03:46:51.251 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.129554.asok 2023-07-06T03:46:51.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:51.252 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.129554.asok status 2023-07-06T03:46:51.474 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:46:51.475 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:46:51.475 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:46:51.475 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:46:51.475 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:46:51.475 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:46:51.475 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:46:51.475 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "129554", 2023-07-06T03:46:51.476 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:46:51.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:51.476 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:46:51.476 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:46:51.476 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5529, 2023-07-06T03:46:51.476 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:46:51.477 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:46:51.477 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:46:51.477 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 5529 2023-07-06T03:46:51.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:51.477 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:46:51.477 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:46:51.477 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:46:51.477 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1407883079 2023-07-06T03:46:51.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:46:51.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:51.478 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:46:51.478 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:46:51.478 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:46:51.478 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1407883079 2023-07-06T03:46:51.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:46:51.479 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.5529 192.168.0.1:0/1407883079", 2023-07-06T03:46:51.479 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/1407883079", 2023-07-06T03:46:51.479 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:46:51.479 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 16, 2023-07-06T03:46:51.479 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 34, 2023-07-06T03:46:51.479 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:46:51.479 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:46:51.479 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:46:51.480 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:46:51.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:51.489 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:46:51.489 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:46:51.535 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:46:51.535 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:46:51.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:51.536 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:46:51.645 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:46:51.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.840+0000 7f1b509ff700 1 Processor -- start 2023-07-06T03:46:51.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.840+0000 7f1b509ff700 1 -- start start 2023-07-06T03:46:51.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.841+0000 7f1b509ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b4c158960 0x7f1b4c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:51.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.841+0000 7f1b509ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1b4c15ce90 0x7f1b4c0712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:51.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.841+0000 7f1b509ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b4c071830 0x7f1b4c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:51.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.841+0000 7f1b509ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c06ddb0 con 0x7f1b4c071830 2023-07-06T03:46:51.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.841+0000 7f1b509ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c06bbf0 con 0x7f1b4c15ce90 2023-07-06T03:46:51.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.841+0000 7f1b509ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c06bed0 con 0x7f1b4c158960 2023-07-06T03:46:51.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.842+0000 7f1b4ad9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b4c071830 0x7f1b4c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:51.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.842+0000 7f1b4ad9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b4c071830 0x7f1b4c0cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58030/0 (socket says 172.21.15.97:58030) 2023-07-06T03:46:51.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.842+0000 7f1b4ad9d700 1 -- 172.21.15.97:0/3753974793 learned_addr learned my addr 172.21.15.97:0/3753974793 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:51.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.842+0000 7f1b4a59c700 1 --2- 172.21.15.97:0/3753974793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b4c158960 0x7f1b4c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:51.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.842+0000 7f1b49d9b700 1 --2- 172.21.15.97:0/3753974793 >> v2:172.21.15.146:3300/0 conn(0x7f1b4c15ce90 0x7f1b4c0712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:51.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.842+0000 7f1b4ad9d700 1 -- 172.21.15.97:0/3753974793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b4c158960 msgr2=0x7f1b4c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:51.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.842+0000 7f1b4ad9d700 1 --2- 172.21.15.97:0/3753974793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b4c158960 0x7f1b4c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:51.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.842+0000 7f1b4ad9d700 1 -- 172.21.15.97:0/3753974793 >> v2:172.21.15.146:3300/0 conn(0x7f1b4c15ce90 msgr2=0x7f1b4c0712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:51.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.842+0000 7f1b4ad9d700 1 --2- 172.21.15.97:0/3753974793 >> v2:172.21.15.146:3300/0 conn(0x7f1b4c15ce90 0x7f1b4c0712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:51.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.842+0000 7f1b4ad9d700 1 -- 172.21.15.97:0/3753974793 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c06c1b0 con 0x7f1b4c071830 2023-07-06T03:46:51.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.842+0000 7f1b4ad9d700 1 --2- 172.21.15.97:0/3753974793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4c071830 0x7f1b4c0cc400 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0b1690 tx=0x7f1b4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f66f962c5642eab server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:51.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b4959a700 1 -- 172.21.15.97:0/3753974793 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400e070 con 0x7f1b4c071830 2023-07-06T03:46:51.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b4959a700 1 -- 172.21.15.97:0/3753974793 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4400b910 con 0x7f1b4c071830 2023-07-06T03:46:51.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b4959a700 1 -- 172.21.15.97:0/3753974793 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400bcc0 con 0x7f1b4c071830 2023-07-06T03:46:51.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b509ff700 1 -- 172.21.15.97:0/3753974793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4c071830 msgr2=0x7f1b4c0cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:51.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b509ff700 1 --2- 172.21.15.97:0/3753974793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4c071830 0x7f1b4c0cc400 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0b1690 tx=0x7f1b4400b330 comp rx=0 tx=0).stop 2023-07-06T03:46:51.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b509ff700 1 -- 172.21.15.97:0/3753974793 shutdown_connections 2023-07-06T03:46:51.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b509ff700 1 --2- 172.21.15.97:0/3753974793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4c071830 0x7f1b4c0cc400 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:51.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b509ff700 1 --2- 172.21.15.97:0/3753974793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b4c158960 0x7f1b4c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:51.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b509ff700 1 --2- 172.21.15.97:0/3753974793 >> v2:172.21.15.146:3300/0 conn(0x7f1b4c15ce90 0x7f1b4c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:51.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b509ff700 1 -- 172.21.15.97:0/3753974793 >> 172.21.15.97:0/3753974793 conn(0x7f1b4c0ce8c0 msgr2=0x7f1b4c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:51.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b509ff700 1 -- 172.21.15.97:0/3753974793 shutdown_connections 2023-07-06T03:46:51.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.843+0000 7f1b509ff700 1 -- 172.21.15.97:0/3753974793 wait complete. 2023-07-06T03:46:51.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.845+0000 7f1b509ff700 1 Processor -- start 2023-07-06T03:46:51.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.845+0000 7f1b509ff700 1 -- start start 2023-07-06T03:46:51.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.846+0000 7f1b509ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4c071830 0x7f1b4c0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:51.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.846+0000 7f1b509ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b4c158960 0x7f1b4c0b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:51.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.846+0000 7f1b509ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b4c15ce90 0x7f1b4c14b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:51.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.846+0000 7f1b509ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c147c20 con 0x7f1b4c071830 2023-07-06T03:46:51.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.846+0000 7f1b509ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0ae860 con 0x7f1b4c15ce90 2023-07-06T03:46:51.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.846+0000 7f1b509ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0aeeb0 con 0x7f1b4c158960 2023-07-06T03:46:51.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.846+0000 7f1b4a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4c071830 0x7f1b4c0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:51.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.846+0000 7f1b4a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4c071830 0x7f1b4c0b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58036/0 (socket says 172.21.15.97:58036) 2023-07-06T03:46:51.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.846+0000 7f1b4a59c700 1 -- 172.21.15.97:0/1476525811 learned_addr learned my addr 172.21.15.97:0/1476525811 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:51.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b49d9b700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b4c158960 0x7f1b4c0b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:51.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b4ad9d700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b4c15ce90 0x7f1b4c14b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:51.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b4a59c700 1 -- 172.21.15.97:0/1476525811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b4c158960 msgr2=0x7f1b4c0b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:51.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b4a59c700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b4c158960 0x7f1b4c0b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:51.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b4a59c700 1 -- 172.21.15.97:0/1476525811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b4c15ce90 msgr2=0x7f1b4c14b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:51.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b4a59c700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b4c15ce90 0x7f1b4c14b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:51.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b4a59c700 1 -- 172.21.15.97:0/1476525811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b44009b30 con 0x7f1b4c071830 2023-07-06T03:46:51.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b4a59c700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4c071830 0x7f1b4c0b3490 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00e930 tx=0x7f1b3c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:51.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b3affd700 1 -- 172.21.15.97:0/1476525811 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c013680 con 0x7f1b4c071830 2023-07-06T03:46:51.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b3affd700 1 -- 172.21.15.97:0/1476525811 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c0137e0 con 0x7f1b4c071830 2023-07-06T03:46:51.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b509ff700 1 -- 172.21.15.97:0/1476525811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c0af190 con 0x7f1b4c071830 2023-07-06T03:46:51.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.847+0000 7f1b3affd700 1 -- 172.21.15.97:0/1476525811 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c013b70 con 0x7f1b4c071830 2023-07-06T03:46:51.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.848+0000 7f1b509ff700 1 -- 172.21.15.97:0/1476525811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c15b200 con 0x7f1b4c071830 2023-07-06T03:46:51.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.848+0000 7f1b509ff700 1 -- 172.21.15.97:0/1476525811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c15b850 con 0x7f1b4c071830 2023-07-06T03:46:51.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.849+0000 7f1b3affd700 1 -- 172.21.15.97:0/1476525811 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b3c013cd0 con 0x7f1b4c071830 2023-07-06T03:46:51.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.850+0000 7f1b3affd700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2c06ace0 0x7f1b2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:51.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.850+0000 7f1b3affd700 1 -- 172.21.15.97:0/1476525811 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f1b3c0894c0 con 0x7f1b4c071830 2023-07-06T03:46:51.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.850+0000 7f1b49d9b700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2c06ace0 0x7f1b2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:51.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.851+0000 7f1b49d9b700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2c06ace0 0x7f1b2c06d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c155670 tx=0x7f1b40009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:51.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:51.855+0000 7f1b3affd700 1 -- 172.21.15.97:0/1476525811 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1b3c054500 con 0x7f1b4c071830 2023-07-06T03:46:52.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.097+0000 7f1b509ff700 1 -- 172.21.15.97:0/1476525811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1b4c067830 con 0x7f1b4c071830 2023-07-06T03:46:52.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.097+0000 7f1b3affd700 1 -- 172.21.15.97:0/1476525811 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1b3c057b20 con 0x7f1b4c071830 2023-07-06T03:46:52.101 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:46:52.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 -- 172.21.15.97:0/1476525811 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2c06ace0 msgr2=0x7f1b2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2c06ace0 0x7f1b2c06d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c155670 tx=0x7f1b40009210 comp rx=0 tx=0).stop 2023-07-06T03:46:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 -- 172.21.15.97:0/1476525811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4c071830 msgr2=0x7f1b4c0b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4c071830 0x7f1b4c0b3490 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00e930 tx=0x7f1b3c00ec40 comp rx=0 tx=0).stop 2023-07-06T03:46:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 -- 172.21.15.97:0/1476525811 shutdown_connections 2023-07-06T03:46:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2c06ace0 0x7f1b2c06d190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:52.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b4c15ce90 0x7f1b4c14b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:52.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4c071830 0x7f1b4c0b3490 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:52.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 --2- 172.21.15.97:0/1476525811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b4c158960 0x7f1b4c0b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:52.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 -- 172.21.15.97:0/1476525811 >> 172.21.15.97:0/1476525811 conn(0x7f1b4c0ce8c0 msgr2=0x7f1b4c150740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:52.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 -- 172.21.15.97:0/1476525811 shutdown_connections 2023-07-06T03:46:52.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:52.100+0000 7f1b509ff700 1 -- 172.21.15.97:0/1476525811 wait complete. 2023-07-06T03:46:52.121 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:46:52.121 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:46:52.121 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2023-07-06T03:46:52.122 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T03:46:52.122 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T03:46:52.122 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:46:52.122 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:46:52.123 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:46:52.123 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:46:52.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:52.124 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2023-07-06T03:46:52.175 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:46:52.176 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:46:52.176 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:46:52.176 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:52.176 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:46:52.176 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:52.176 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:46:52.177 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:46:52.177 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2023-07-06T03:46:52.177 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:46:52.177 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:46:52.177 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2023-07-06T03:46:52.177 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:46:52.177 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2023-07-06T03:46:52.178 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:46:52.178 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2023-07-06T03:46:52.178 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:46:52.178 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42073sec preferred_lft 42073sec 2023-07-06T03:46:52.178 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2023-07-06T03:46:52.178 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:52.178 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:46:52.178 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:46:52.179 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:46:52.179 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:52.179 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::58f0:6eff:fe04:3be0/64 scope link 2023-07-06T03:46:52.179 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:52.179 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:46:52.179 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:46:52.179 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::106d:9cff:fed5:9d0f/64 scope link 2023-07-06T03:46:52.179 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:52.180 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:46:52.180 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether a2:c5:04:8e:a2:a7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T03:46:52.180 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::a0c5:4ff:fe8e:a2a7/64 scope link 2023-07-06T03:46:52.180 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:46:52.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:52.181 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:46:52.251 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:46:52.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:52.254 DEBUG:teuthology.orchestra.run.smithi146:> route 2023-07-06T03:46:52.303 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2023-07-06T03:46:52.303 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:46:52.303 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:46:52.304 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:46:52.304 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:46:52.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:52.305 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:46:52.305 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:46:52.306 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:46:52.306 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:46:52.306 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:46:52.307 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:46:52.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:52.473 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:46:52.518 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T03:46:52.519 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T03:46:52.520 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:46:52.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:52.520 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2023-07-06T03:46:52.629 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-06T03:46:52.630 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:46:52.630 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T03:46:52.678 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:46:52.678 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:46:52.679 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:46:52.724 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:46:52.833 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:46:52.834 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:46:52.904 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:46:52.907 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:46:52.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:52.908 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:46:52.954 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:46:52.955 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-06T03:46:52.999 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:46:53.069 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:46:53.070 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:46:53.138 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:46:53.141 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:46:53.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:53.142 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:46:53.164 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2023-07-06T03:46:53.162+0000 7f105ba715c0 -1 init, newargv = 0x55bf62d54040 newargc=15 2023-07-06T03:46:53.165 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[120442]: starting ceph client 2023-07-06T03:46:53.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[120442]: starting fuse 2023-07-06T03:46:53.196 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:46:53.196 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T03:46:53.197 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:46:53.197 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2023-07-06T03:46:53.197 DEBUG:teuthology.orchestra.run.smithi146:> import re 2023-07-06T03:46:53.198 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:46:53.198 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2023-07-06T03:46:53.198 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:46:53.199 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2023-07-06T03:46:53.199 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-06T03:46:53.199 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2023-07-06T03:46:53.200 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-06T03:46:53.200 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:46:53.200 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2023-07-06T03:46:53.201 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2023-07-06T03:46:53.201 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2023-07-06T03:46:53.201 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2023-07-06T03:46:53.202 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:46:53.202 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2023-07-06T03:46:53.202 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:46:53.202 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:46:53.203 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:46:53.203 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2023-07-06T03:46:53.203 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2023-07-06T03:46:53.204 DEBUG:teuthology.orchestra.run.smithi146:> return f 2023-07-06T03:46:53.204 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:46:53.204 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:46:53.205 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2023-07-06T03:46:53.205 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:46:53.380 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.120442.asok 2023-07-06T03:46:53.557 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.120442.asok 2023-07-06T03:46:53.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:53.558 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.120442.asok status 2023-07-06T03:46:53.783 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-07-06T03:46:53.783 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2023-07-06T03:46:53.784 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:46:53.784 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:46:53.784 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2023-07-06T03:46:53.784 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2023-07-06T03:46:53.784 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-06T03:46:53.784 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "120442", 2023-07-06T03:46:53.784 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2023-07-06T03:46:53.784 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:46:53.785 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2023-07-06T03:46:53.785 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2023-07-06T03:46:53.785 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 5380, 2023-07-06T03:46:53.785 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2023-07-06T03:46:53.785 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2023-07-06T03:46:53.785 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2023-07-06T03:46:53.785 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 5380 2023-07-06T03:46:53.786 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:46:53.786 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:46:53.786 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:46:53.786 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:46:53.786 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 4270566397 2023-07-06T03:46:53.786 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-07-06T03:46:53.786 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:46:53.786 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:46:53.787 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:46:53.787 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:46:53.787 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 4270566397 2023-07-06T03:46:53.787 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:46:53.787 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.5380 192.168.0.1:0/4270566397", 2023-07-06T03:46:53.787 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/4270566397", 2023-07-06T03:46:53.787 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2023-07-06T03:46:53.787 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 16, 2023-07-06T03:46:53.788 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 36, 2023-07-06T03:46:53.788 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:46:53.788 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2023-07-06T03:46:53.788 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2023-07-06T03:46:53.788 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-07-06T03:46:53.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:53.798 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2023-07-06T03:46:53.798 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-06T03:46:53.845 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2023-07-06T03:46:53.845 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-06T03:46:53.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:46:53.846 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:46:53.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 500 2023-07-06T03:46:54.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.122+0000 7f45543ff700 1 Processor -- start 2023-07-06T03:46:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.122+0000 7f45543ff700 1 -- start start 2023-07-06T03:46:54.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.123+0000 7f45543ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f454c15c610 0x7f454c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.123+0000 7f45543ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f454c06b360 0x7f454c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.123+0000 7f45543ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454c0cff90 con 0x7f454c0d2210 2023-07-06T03:46:54.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.123+0000 7f45543ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c0d0780 con 0x7f454c15c610 2023-07-06T03:46:54.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.123+0000 7f45543ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c15c9e0 con 0x7f454c06b360 2023-07-06T03:46:54.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.123+0000 7f455219b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f454c15c610 0x7f454c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:54.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.124+0000 7f455219b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f454c15c610 0x7f454c0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58044/0 (socket says 172.21.15.97:58044) 2023-07-06T03:46:54.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.124+0000 7f455219b700 1 -- 172.21.15.97:0/3398664979 learned_addr learned my addr 172.21.15.97:0/3398664979 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:54.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.124+0000 7f455219b700 1 -- 172.21.15.97:0/3398664979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f454c06b360 msgr2=0x7f454c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:54.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.124+0000 7f455219b700 1 --2- 172.21.15.97:0/3398664979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f454c06b360 0x7f454c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.124+0000 7f455219b700 1 -- 172.21.15.97:0/3398664979 >> v1:172.21.15.146:6789/0 conn(0x7f454c0d2210 legacy=0x7f454c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.124+0000 7f455219b700 1 -- 172.21.15.97:0/3398664979 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c0b5650 con 0x7f454c15c610 2023-07-06T03:46:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.124+0000 7f455219b700 1 --2- 172.21.15.97:0/3398664979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c15c610 0x7f454c0b47b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4548009920 tx=0x7f4548009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff24aa0983d19af server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.124+0000 7f4551199700 1 -- 172.21.15.97:0/3398664979 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548004230 con 0x7f454c15c610 2023-07-06T03:46:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.124+0000 7f4551199700 1 -- 172.21.15.97:0/3398664979 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4548004390 con 0x7f454c15c610 2023-07-06T03:46:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.124+0000 7f4551199700 1 -- 172.21.15.97:0/3398664979 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548009e70 con 0x7f454c15c610 2023-07-06T03:46:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.125+0000 7f45543ff700 1 -- 172.21.15.97:0/3398664979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c15c610 msgr2=0x7f454c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.125+0000 7f45543ff700 1 --2- 172.21.15.97:0/3398664979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c15c610 0x7f454c0b47b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4548009920 tx=0x7f4548009c30 comp rx=0 tx=0).stop 2023-07-06T03:46:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.125+0000 7f45543ff700 1 -- 172.21.15.97:0/3398664979 shutdown_connections 2023-07-06T03:46:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.125+0000 7f45543ff700 1 --2- 172.21.15.97:0/3398664979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c15c610 0x7f454c0b47b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.125+0000 7f45543ff700 1 --2- 172.21.15.97:0/3398664979 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f454c06b360 0x7f454c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.125+0000 7f45543ff700 1 -- 172.21.15.97:0/3398664979 >> 172.21.15.97:0/3398664979 conn(0x7f454c06a6c0 msgr2=0x7f454c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.125+0000 7f45543ff700 1 -- 172.21.15.97:0/3398664979 shutdown_connections 2023-07-06T03:46:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.125+0000 7f45543ff700 1 -- 172.21.15.97:0/3398664979 wait complete. 2023-07-06T03:46:54.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.127+0000 7f45543ff700 1 Processor -- start 2023-07-06T03:46:54.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.127+0000 7f45543ff700 1 -- start start 2023-07-06T03:46:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f45543ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f454c06b360 0x7f454c14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f45543ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c0d2210 0x7f454c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f45543ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f454c15c610 0x7f454c15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f45543ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0c8e70 con 0x7f454c0d2210 2023-07-06T03:46:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f45543ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0cb020 con 0x7f454c15c610 2023-07-06T03:46:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f45543ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0caaa0 con 0x7f454c06b360 2023-07-06T03:46:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f455199a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c0d2210 0x7f454c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f455199a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c0d2210 0x7f454c15cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58060/0 (socket says 172.21.15.97:58060) 2023-07-06T03:46:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f455199a700 1 -- 172.21.15.97:0/3620192822 learned_addr learned my addr 172.21.15.97:0/3620192822 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f455299c700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f454c15c610 0x7f454c15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f455219b700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f454c06b360 0x7f454c14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.128+0000 7f455199a700 1 -- 172.21.15.97:0/3620192822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f454c06b360 msgr2=0x7f454c14fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f455199a700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f454c06b360 0x7f454c14fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f455199a700 1 -- 172.21.15.97:0/3620192822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f454c15c610 msgr2=0x7f454c15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f455199a700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f454c15c610 0x7f454c15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f455199a700 1 -- 172.21.15.97:0/3620192822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4548009550 con 0x7f454c0d2210 2023-07-06T03:46:54.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f455299c700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f454c15c610 0x7f454c15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:54.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f455219b700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f454c06b360 0x7f454c14fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:46:54.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f455199a700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c0d2210 0x7f454c15cb20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f453c00b500 tx=0x7f453c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:54.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f4542ffd700 1 -- 172.21.15.97:0/3620192822 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c011830 con 0x7f454c0d2210 2023-07-06T03:46:54.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f4542ffd700 1 -- 172.21.15.97:0/3620192822 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c011990 con 0x7f454c0d2210 2023-07-06T03:46:54.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f45543ff700 1 -- 172.21.15.97:0/3620192822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454c0c9080 con 0x7f454c0d2210 2023-07-06T03:46:54.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f4542ffd700 1 -- 172.21.15.97:0/3620192822 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c011d40 con 0x7f454c0d2210 2023-07-06T03:46:54.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.129+0000 7f45543ff700 1 -- 172.21.15.97:0/3620192822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454c0c9320 con 0x7f454c0d2210 2023-07-06T03:46:54.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.130+0000 7f45543ff700 1 -- 172.21.15.97:0/3620192822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c159dd0 con 0x7f454c0d2210 2023-07-06T03:46:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.131+0000 7f4542ffd700 1 -- 172.21.15.97:0/3620192822 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f453c01a400 con 0x7f454c0d2210 2023-07-06T03:46:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.132+0000 7f4542ffd700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f453006ace0 0x7f453006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.132+0000 7f4542ffd700 1 -- 172.21.15.97:0/3620192822 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f453c0894d0 con 0x7f454c0d2210 2023-07-06T03:46:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.133+0000 7f455219b700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f453006ace0 0x7f453006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.133+0000 7f455219b700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f453006ace0 0x7f453006d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4548009c70 tx=0x7f4548008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:54.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.137+0000 7f4542ffd700 1 -- 172.21.15.97:0/3620192822 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f453c0545b0 con 0x7f454c0d2210 2023-07-06T03:46:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.334+0000 7f45543ff700 1 -- 172.21.15.97:0/3620192822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) v1 -- 0x7f454c04e340 con 0x7f454c0d2210 2023-07-06T03:46:54.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.338+0000 7f4542ffd700 1 -- 172.21.15.97:0/3620192822 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v11)=0 v11) v1 ==== 123+0+0 (secure 0 0 0) 0x7f453c057bd0 con 0x7f454c0d2210 2023-07-06T03:46:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 -- 172.21.15.97:0/3620192822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f453006ace0 msgr2=0x7f453006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f453006ace0 0x7f453006d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4548009c70 tx=0x7f4548008030 comp rx=0 tx=0).stop 2023-07-06T03:46:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 -- 172.21.15.97:0/3620192822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c0d2210 msgr2=0x7f454c15cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c0d2210 0x7f454c15cb20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f453c00b500 tx=0x7f453c00b8c0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 -- 172.21.15.97:0/3620192822 shutdown_connections 2023-07-06T03:46:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f453006ace0 0x7f453006d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f454c15c610 0x7f454c15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f454c06b360 0x7f454c14fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 --2- 172.21.15.97:0/3620192822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c0d2210 0x7f454c15cb20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 -- 172.21.15.97:0/3620192822 >> 172.21.15.97:0/3620192822 conn(0x7f454c06a6c0 msgr2=0x7f454c15b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:54.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 -- 172.21.15.97:0/3620192822 shutdown_connections 2023-07-06T03:46:54.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.340+0000 7f45543ff700 1 -- 172.21.15.97:0/3620192822 wait complete. 2023-07-06T03:46:54.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_cap_acquisition_throttle 250 2023-07-06T03:46:54.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.557+0000 7fcc297ff700 1 Processor -- start 2023-07-06T03:46:54.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.557+0000 7fcc297ff700 1 -- start start 2023-07-06T03:46:54.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.558+0000 7fcc297ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2406b170 0x7fcc2415d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.558+0000 7fcc297ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc240cfbc0 0x7fcc2415e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.558+0000 7fcc297ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2415de00 con 0x7fcc2415cae0 2023-07-06T03:46:54.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.558+0000 7fcc297ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc240b25a0 con 0x7fcc240cfbc0 2023-07-06T03:46:54.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.558+0000 7fcc297ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240ceb20 con 0x7fcc2406b170 2023-07-06T03:46:54.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.559+0000 7fcc237fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc240cfbc0 0x7fcc2415e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:54.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.559+0000 7fcc237fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc240cfbc0 0x7fcc2415e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58064/0 (socket says 172.21.15.97:58064) 2023-07-06T03:46:54.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.559+0000 7fcc237fe700 1 -- 172.21.15.97:0/3227975538 learned_addr learned my addr 172.21.15.97:0/3227975538 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.559+0000 7fcc22ffd700 1 --2- 172.21.15.97:0/3227975538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2406b170 0x7fcc2415d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.559+0000 7fcc237fe700 1 -- 172.21.15.97:0/3227975538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2406b170 msgr2=0x7fcc2415d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:46:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.559+0000 7fcc237fe700 1 --2- 172.21.15.97:0/3227975538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2406b170 0x7fcc2415d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.559+0000 7fcc237fe700 1 -- 172.21.15.97:0/3227975538 >> v1:172.21.15.146:6789/0 conn(0x7fcc2415cae0 legacy=0x7fcc2406b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.559+0000 7fcc237fe700 1 -- 172.21.15.97:0/3227975538 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc240cee00 con 0x7fcc240cfbc0 2023-07-06T03:46:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.559+0000 7fcc237fe700 1 --2- 172.21.15.97:0/3227975538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc240cfbc0 0x7fcc2415e1b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009e50 tx=0x7fcc1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78730fe7b26814dc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.560+0000 7fcc21ffb700 1 -- 172.21.15.97:0/3227975538 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00e070 con 0x7fcc240cfbc0 2023-07-06T03:46:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.560+0000 7fcc21ffb700 1 -- 172.21.15.97:0/3227975538 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00b9c0 con 0x7fcc240cfbc0 2023-07-06T03:46:54.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.560+0000 7fcc21ffb700 1 -- 172.21.15.97:0/3227975538 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00bd70 con 0x7fcc240cfbc0 2023-07-06T03:46:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.560+0000 7fcc297ff700 1 -- 172.21.15.97:0/3227975538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc240cfbc0 msgr2=0x7fcc2415e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.560+0000 7fcc297ff700 1 --2- 172.21.15.97:0/3227975538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc240cfbc0 0x7fcc2415e1b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009e50 tx=0x7fcc1c00b330 comp rx=0 tx=0).stop 2023-07-06T03:46:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.560+0000 7fcc297ff700 1 -- 172.21.15.97:0/3227975538 shutdown_connections 2023-07-06T03:46:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.560+0000 7fcc297ff700 1 --2- 172.21.15.97:0/3227975538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc240cfbc0 0x7fcc2415e1b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.560+0000 7fcc297ff700 1 --2- 172.21.15.97:0/3227975538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2406b170 0x7fcc2415d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.560+0000 7fcc297ff700 1 -- 172.21.15.97:0/3227975538 >> 172.21.15.97:0/3227975538 conn(0x7fcc24099f10 msgr2=0x7fcc240d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.560+0000 7fcc297ff700 1 -- 172.21.15.97:0/3227975538 shutdown_connections 2023-07-06T03:46:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.560+0000 7fcc297ff700 1 -- 172.21.15.97:0/3227975538 wait complete. 2023-07-06T03:46:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.562+0000 7fcc297ff700 1 Processor -- start 2023-07-06T03:46:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.562+0000 7fcc297ff700 1 -- start start 2023-07-06T03:46:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.563+0000 7fcc297ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2406b170 0x7fcc2409e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.563+0000 7fcc297ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc240cfbc0 0x7fcc240c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.563+0000 7fcc297ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2415cae0 0x7fcc24153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.563+0000 7fcc297ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc241570c0 con 0x7fcc2415cae0 2023-07-06T03:46:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.563+0000 7fcc297ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc241573a0 con 0x7fcc2406b170 2023-07-06T03:46:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.563+0000 7fcc297ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc24157680 con 0x7fcc240cfbc0 2023-07-06T03:46:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.563+0000 7fcc237fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2415cae0 0x7fcc24153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc237fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2415cae0 0x7fcc24153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58066/0 (socket says 172.21.15.97:58066) 2023-07-06T03:46:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc237fe700 1 -- 172.21.15.97:0/2397702384 learned_addr learned my addr 172.21.15.97:0/2397702384 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:46:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc227fc700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc240cfbc0 0x7fcc240c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc22ffd700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2406b170 0x7fcc2409e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc237fe700 1 -- 172.21.15.97:0/2397702384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc240cfbc0 msgr2=0x7fcc240c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc237fe700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc240cfbc0 0x7fcc240c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc237fe700 1 -- 172.21.15.97:0/2397702384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2406b170 msgr2=0x7fcc2409e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc237fe700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2406b170 0x7fcc2409e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc237fe700 1 -- 172.21.15.97:0/2397702384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c009b30 con 0x7fcc2415cae0 2023-07-06T03:46:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc237fe700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2415cae0 0x7fcc24153ff0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcc24066da0 tx=0x7fcc1c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc0b7fe700 1 -- 172.21.15.97:0/2397702384 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c004800 con 0x7fcc2415cae0 2023-07-06T03:46:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc0b7fe700 1 -- 172.21.15.97:0/2397702384 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c004960 con 0x7fcc2415cae0 2023-07-06T03:46:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc297ff700 1 -- 172.21.15.97:0/2397702384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc24157960 con 0x7fcc2415cae0 2023-07-06T03:46:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.564+0000 7fcc0b7fe700 1 -- 172.21.15.97:0/2397702384 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c01c680 con 0x7fcc2415cae0 2023-07-06T03:46:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.565+0000 7fcc297ff700 1 -- 172.21.15.97:0/2397702384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2409e830 con 0x7fcc2415cae0 2023-07-06T03:46:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.566+0000 7fcc297ff700 1 -- 172.21.15.97:0/2397702384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2404e340 con 0x7fcc2415cae0 2023-07-06T03:46:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.567+0000 7fcc0b7fe700 1 -- 172.21.15.97:0/2397702384 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc1c01c7e0 con 0x7fcc2415cae0 2023-07-06T03:46:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.568+0000 7fcc0b7fe700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 0x7fcc1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:46:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.568+0000 7fcc0b7fe700 1 -- 172.21.15.97:0/2397702384 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fcc1c08b090 con 0x7fcc2415cae0 2023-07-06T03:46:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.568+0000 7fcc22ffd700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 0x7fcc1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:46:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.569+0000 7fcc22ffd700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 0x7fcc1006d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006fd0 tx=0x7fcc14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:46:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.572+0000 7fcc0b7fe700 1 -- 172.21.15.97:0/2397702384 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcc1c056170 con 0x7fcc2415cae0 2023-07-06T03:46:54.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.768+0000 7fcc297ff700 1 -- 172.21.15.97:0/2397702384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_session_cap_acquisition_throttle}] v 0) v1 -- 0x7fcc2415b7b0 con 0x7fcc2415cae0 2023-07-06T03:46:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.772+0000 7fcc0b7fe700 1 -- 172.21.15.97:0/2397702384 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_cap_acquisition_throttle}]=0 v12)=0 v12) v1 ==== 136+0+0 (secure 0 0 0) 0x7fcc1c059790 con 0x7fcc2415cae0 2023-07-06T03:46:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 -- 172.21.15.97:0/2397702384 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 msgr2=0x7fcc1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 0x7fcc1006d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006fd0 tx=0x7fcc14008040 comp rx=0 tx=0).stop 2023-07-06T03:46:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 -- 172.21.15.97:0/2397702384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2415cae0 msgr2=0x7fcc24153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:46:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2415cae0 0x7fcc24153ff0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcc24066da0 tx=0x7fcc1c004ad0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 -- 172.21.15.97:0/2397702384 shutdown_connections 2023-07-06T03:46:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 0x7fcc1006d190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2415cae0 0x7fcc24153ff0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2406b170 0x7fcc2409e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 --2- 172.21.15.97:0/2397702384 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc240cfbc0 0x7fcc240c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:46:54.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 -- 172.21.15.97:0/2397702384 >> 172.21.15.97:0/2397702384 conn(0x7fcc24099f10 msgr2=0x7fcc240b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:46:54.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 -- 172.21.15.97:0/2397702384 shutdown_connections 2023-07-06T03:46:54.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:46:54.775+0000 7fcc297ff700 1 -- 172.21.15.97:0/2397702384 wait complete. 2023-07-06T03:46:54.796 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:46:54.796 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:54.842 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:46:54.842 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:46:54.843 DEBUG:teuthology.orchestra.run.smithi097:> import uuid 2023-07-06T03:46:54.843 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:54.843 DEBUG:teuthology.orchestra.run.smithi097:> n = 250 2023-07-06T03:46:54.844 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = False 2023-07-06T03:46:54.844 DEBUG:teuthology.orchestra.run.smithi097:> if 0 > 0: 2023-07-06T03:46:54.844 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = True 2023-07-06T03:46:54.845 DEBUG:teuthology.orchestra.run.smithi097:> path = "/home/ubuntu/cephtest/mnt.0/dir1/file" 2023-07-06T03:46:54.845 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:54.845 DEBUG:teuthology.orchestra.run.smithi097:> dpath = os.path.dirname(path) 2023-07-06T03:46:54.846 DEBUG:teuthology.orchestra.run.smithi097:> fnameprefix = os.path.basename(path) 2023-07-06T03:46:54.846 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(dpath, exist_ok=True) 2023-07-06T03:46:54.846 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:54.846 DEBUG:teuthology.orchestra.run.smithi097:> try: 2023-07-06T03:46:54.847 DEBUG:teuthology.orchestra.run.smithi097:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-06T03:46:54.847 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:54.847 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(n): 2023-07-06T03:46:54.848 DEBUG:teuthology.orchestra.run.smithi097:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-06T03:46:54.848 DEBUG:teuthology.orchestra.run.smithi097:> with open(fpath, '"'"'w'"'"') as f: 2023-07-06T03:46:54.848 DEBUG:teuthology.orchestra.run.smithi097:> f.write(f"{i}") 2023-07-06T03:46:54.849 DEBUG:teuthology.orchestra.run.smithi097:> if True: 2023-07-06T03:46:54.849 DEBUG:teuthology.orchestra.run.smithi097:> f.flush() 2023-07-06T03:46:54.849 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(f.fileno()) 2023-07-06T03:46:54.850 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:54.850 DEBUG:teuthology.orchestra.run.smithi097:> os.unlink(fpath) 2023-07-06T03:46:54.850 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:54.850 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:46:54.851 DEBUG:teuthology.orchestra.run.smithi097:> if create_hard_links: 2023-07-06T03:46:54.851 DEBUG:teuthology.orchestra.run.smithi097:> for j in range(0): 2023-07-06T03:46:54.851 DEBUG:teuthology.orchestra.run.smithi097:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-06T03:46:54.852 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:54.852 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:46:54.852 DEBUG:teuthology.orchestra.run.smithi097:> finally: 2023-07-06T03:46:54.852 DEBUG:teuthology.orchestra.run.smithi097:> os.close(dirfd) 2023-07-06T03:46:54.853 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:46:56.581 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:46:56.582 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:56.667 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:46:56.667 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:46:56.668 DEBUG:teuthology.orchestra.run.smithi097:> import uuid 2023-07-06T03:46:56.668 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:56.668 DEBUG:teuthology.orchestra.run.smithi097:> n = 250 2023-07-06T03:46:56.669 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = False 2023-07-06T03:46:56.669 DEBUG:teuthology.orchestra.run.smithi097:> if 0 > 0: 2023-07-06T03:46:56.669 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = True 2023-07-06T03:46:56.669 DEBUG:teuthology.orchestra.run.smithi097:> path = "/home/ubuntu/cephtest/mnt.0/dir2/file" 2023-07-06T03:46:56.670 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:56.670 DEBUG:teuthology.orchestra.run.smithi097:> dpath = os.path.dirname(path) 2023-07-06T03:46:56.670 DEBUG:teuthology.orchestra.run.smithi097:> fnameprefix = os.path.basename(path) 2023-07-06T03:46:56.671 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(dpath, exist_ok=True) 2023-07-06T03:46:56.671 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:56.671 DEBUG:teuthology.orchestra.run.smithi097:> try: 2023-07-06T03:46:56.671 DEBUG:teuthology.orchestra.run.smithi097:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-06T03:46:56.672 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:56.672 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(n): 2023-07-06T03:46:56.672 DEBUG:teuthology.orchestra.run.smithi097:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-06T03:46:56.673 DEBUG:teuthology.orchestra.run.smithi097:> with open(fpath, '"'"'w'"'"') as f: 2023-07-06T03:46:56.673 DEBUG:teuthology.orchestra.run.smithi097:> f.write(f"{i}") 2023-07-06T03:46:56.673 DEBUG:teuthology.orchestra.run.smithi097:> if True: 2023-07-06T03:46:56.673 DEBUG:teuthology.orchestra.run.smithi097:> f.flush() 2023-07-06T03:46:56.674 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(f.fileno()) 2023-07-06T03:46:56.674 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:56.674 DEBUG:teuthology.orchestra.run.smithi097:> os.unlink(fpath) 2023-07-06T03:46:56.674 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:56.675 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:46:56.675 DEBUG:teuthology.orchestra.run.smithi097:> if create_hard_links: 2023-07-06T03:46:56.675 DEBUG:teuthology.orchestra.run.smithi097:> for j in range(0): 2023-07-06T03:46:56.676 DEBUG:teuthology.orchestra.run.smithi097:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-06T03:46:56.676 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:56.676 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:46:56.676 DEBUG:teuthology.orchestra.run.smithi097:> finally: 2023-07-06T03:46:56.677 DEBUG:teuthology.orchestra.run.smithi097:> os.close(dirfd) 2023-07-06T03:46:56.677 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:46:58.206 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:46:58.206 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:58.291 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:46:58.291 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:46:58.292 DEBUG:teuthology.orchestra.run.smithi097:> import uuid 2023-07-06T03:46:58.292 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:58.292 DEBUG:teuthology.orchestra.run.smithi097:> n = 250 2023-07-06T03:46:58.293 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = False 2023-07-06T03:46:58.293 DEBUG:teuthology.orchestra.run.smithi097:> if 0 > 0: 2023-07-06T03:46:58.293 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = True 2023-07-06T03:46:58.293 DEBUG:teuthology.orchestra.run.smithi097:> path = "/home/ubuntu/cephtest/mnt.0/dir3/file" 2023-07-06T03:46:58.294 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:58.294 DEBUG:teuthology.orchestra.run.smithi097:> dpath = os.path.dirname(path) 2023-07-06T03:46:58.294 DEBUG:teuthology.orchestra.run.smithi097:> fnameprefix = os.path.basename(path) 2023-07-06T03:46:58.294 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(dpath, exist_ok=True) 2023-07-06T03:46:58.295 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:58.295 DEBUG:teuthology.orchestra.run.smithi097:> try: 2023-07-06T03:46:58.295 DEBUG:teuthology.orchestra.run.smithi097:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-06T03:46:58.296 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:58.296 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(n): 2023-07-06T03:46:58.296 DEBUG:teuthology.orchestra.run.smithi097:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-06T03:46:58.296 DEBUG:teuthology.orchestra.run.smithi097:> with open(fpath, '"'"'w'"'"') as f: 2023-07-06T03:46:58.297 DEBUG:teuthology.orchestra.run.smithi097:> f.write(f"{i}") 2023-07-06T03:46:58.297 DEBUG:teuthology.orchestra.run.smithi097:> if True: 2023-07-06T03:46:58.297 DEBUG:teuthology.orchestra.run.smithi097:> f.flush() 2023-07-06T03:46:58.297 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(f.fileno()) 2023-07-06T03:46:58.298 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:58.298 DEBUG:teuthology.orchestra.run.smithi097:> os.unlink(fpath) 2023-07-06T03:46:58.298 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:58.298 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:46:58.299 DEBUG:teuthology.orchestra.run.smithi097:> if create_hard_links: 2023-07-06T03:46:58.299 DEBUG:teuthology.orchestra.run.smithi097:> for j in range(0): 2023-07-06T03:46:58.299 DEBUG:teuthology.orchestra.run.smithi097:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-06T03:46:58.300 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:58.300 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:46:58.300 DEBUG:teuthology.orchestra.run.smithi097:> finally: 2023-07-06T03:46:58.300 DEBUG:teuthology.orchestra.run.smithi097:> os.close(dirfd) 2023-07-06T03:46:58.301 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:46:59.829 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:46:59.829 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:46:59.915 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:46:59.915 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:46:59.915 DEBUG:teuthology.orchestra.run.smithi097:> import uuid 2023-07-06T03:46:59.916 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:59.916 DEBUG:teuthology.orchestra.run.smithi097:> n = 250 2023-07-06T03:46:59.916 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = False 2023-07-06T03:46:59.917 DEBUG:teuthology.orchestra.run.smithi097:> if 0 > 0: 2023-07-06T03:46:59.917 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = True 2023-07-06T03:46:59.918 DEBUG:teuthology.orchestra.run.smithi097:> path = "/home/ubuntu/cephtest/mnt.0/dir4/file" 2023-07-06T03:46:59.918 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:59.918 DEBUG:teuthology.orchestra.run.smithi097:> dpath = os.path.dirname(path) 2023-07-06T03:46:59.919 DEBUG:teuthology.orchestra.run.smithi097:> fnameprefix = os.path.basename(path) 2023-07-06T03:46:59.919 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(dpath, exist_ok=True) 2023-07-06T03:46:59.919 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:59.919 DEBUG:teuthology.orchestra.run.smithi097:> try: 2023-07-06T03:46:59.920 DEBUG:teuthology.orchestra.run.smithi097:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-06T03:46:59.921 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:46:59.921 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(n): 2023-07-06T03:46:59.922 DEBUG:teuthology.orchestra.run.smithi097:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-06T03:46:59.922 DEBUG:teuthology.orchestra.run.smithi097:> with open(fpath, '"'"'w'"'"') as f: 2023-07-06T03:46:59.922 DEBUG:teuthology.orchestra.run.smithi097:> f.write(f"{i}") 2023-07-06T03:46:59.922 DEBUG:teuthology.orchestra.run.smithi097:> if True: 2023-07-06T03:46:59.923 DEBUG:teuthology.orchestra.run.smithi097:> f.flush() 2023-07-06T03:46:59.923 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(f.fileno()) 2023-07-06T03:46:59.923 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:59.924 DEBUG:teuthology.orchestra.run.smithi097:> os.unlink(fpath) 2023-07-06T03:46:59.924 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:59.924 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:46:59.925 DEBUG:teuthology.orchestra.run.smithi097:> if create_hard_links: 2023-07-06T03:46:59.925 DEBUG:teuthology.orchestra.run.smithi097:> for j in range(0): 2023-07-06T03:46:59.925 DEBUG:teuthology.orchestra.run.smithi097:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-06T03:46:59.925 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:46:59.926 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:46:59.926 DEBUG:teuthology.orchestra.run.smithi097:> finally: 2023-07-06T03:46:59.926 DEBUG:teuthology.orchestra.run.smithi097:> os.close(dirfd) 2023-07-06T03:46:59.927 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:47:01.853 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:47:01.854 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:47:01.941 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:47:01.942 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:47:01.942 DEBUG:teuthology.orchestra.run.smithi097:> import uuid 2023-07-06T03:47:01.942 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:01.943 DEBUG:teuthology.orchestra.run.smithi097:> n = 250 2023-07-06T03:47:01.943 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = False 2023-07-06T03:47:01.943 DEBUG:teuthology.orchestra.run.smithi097:> if 0 > 0: 2023-07-06T03:47:01.944 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = True 2023-07-06T03:47:01.944 DEBUG:teuthology.orchestra.run.smithi097:> path = "/home/ubuntu/cephtest/mnt.0/dir5/file" 2023-07-06T03:47:01.944 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:01.945 DEBUG:teuthology.orchestra.run.smithi097:> dpath = os.path.dirname(path) 2023-07-06T03:47:01.945 DEBUG:teuthology.orchestra.run.smithi097:> fnameprefix = os.path.basename(path) 2023-07-06T03:47:01.945 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(dpath, exist_ok=True) 2023-07-06T03:47:01.946 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:01.946 DEBUG:teuthology.orchestra.run.smithi097:> try: 2023-07-06T03:47:01.946 DEBUG:teuthology.orchestra.run.smithi097:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-06T03:47:01.946 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:01.947 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(n): 2023-07-06T03:47:01.947 DEBUG:teuthology.orchestra.run.smithi097:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-06T03:47:01.947 DEBUG:teuthology.orchestra.run.smithi097:> with open(fpath, '"'"'w'"'"') as f: 2023-07-06T03:47:01.947 DEBUG:teuthology.orchestra.run.smithi097:> f.write(f"{i}") 2023-07-06T03:47:01.948 DEBUG:teuthology.orchestra.run.smithi097:> if True: 2023-07-06T03:47:01.948 DEBUG:teuthology.orchestra.run.smithi097:> f.flush() 2023-07-06T03:47:01.948 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(f.fileno()) 2023-07-06T03:47:01.949 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:47:01.949 DEBUG:teuthology.orchestra.run.smithi097:> os.unlink(fpath) 2023-07-06T03:47:01.949 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:47:01.950 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:47:01.951 DEBUG:teuthology.orchestra.run.smithi097:> if create_hard_links: 2023-07-06T03:47:01.951 DEBUG:teuthology.orchestra.run.smithi097:> for j in range(0): 2023-07-06T03:47:01.951 DEBUG:teuthology.orchestra.run.smithi097:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-06T03:47:01.952 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:47:01.952 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:47:01.952 DEBUG:teuthology.orchestra.run.smithi097:> finally: 2023-07-06T03:47:01.953 DEBUG:teuthology.orchestra.run.smithi097:> os.close(dirfd) 2023-07-06T03:47:01.953 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:47:03.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:47:03.361 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:47:03.480 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:47:03.480 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:47:03.526 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:47:03.527 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:47:03.527 DEBUG:teuthology.orchestra.run.smithi097:> import uuid 2023-07-06T03:47:03.528 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:03.528 DEBUG:teuthology.orchestra.run.smithi097:> n = 250 2023-07-06T03:47:03.529 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = False 2023-07-06T03:47:03.529 DEBUG:teuthology.orchestra.run.smithi097:> if 0 > 0: 2023-07-06T03:47:03.529 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = True 2023-07-06T03:47:03.529 DEBUG:teuthology.orchestra.run.smithi097:> path = "/home/ubuntu/cephtest/mnt.0/dir6/file" 2023-07-06T03:47:03.530 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:03.530 DEBUG:teuthology.orchestra.run.smithi097:> dpath = os.path.dirname(path) 2023-07-06T03:47:03.530 DEBUG:teuthology.orchestra.run.smithi097:> fnameprefix = os.path.basename(path) 2023-07-06T03:47:03.530 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(dpath, exist_ok=True) 2023-07-06T03:47:03.531 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:03.531 DEBUG:teuthology.orchestra.run.smithi097:> try: 2023-07-06T03:47:03.531 DEBUG:teuthology.orchestra.run.smithi097:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-06T03:47:03.532 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:03.532 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(n): 2023-07-06T03:47:03.532 DEBUG:teuthology.orchestra.run.smithi097:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-06T03:47:03.533 DEBUG:teuthology.orchestra.run.smithi097:> with open(fpath, '"'"'w'"'"') as f: 2023-07-06T03:47:03.533 DEBUG:teuthology.orchestra.run.smithi097:> f.write(f"{i}") 2023-07-06T03:47:03.533 DEBUG:teuthology.orchestra.run.smithi097:> if True: 2023-07-06T03:47:03.533 DEBUG:teuthology.orchestra.run.smithi097:> f.flush() 2023-07-06T03:47:03.534 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(f.fileno()) 2023-07-06T03:47:03.534 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:47:03.534 DEBUG:teuthology.orchestra.run.smithi097:> os.unlink(fpath) 2023-07-06T03:47:03.535 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:47:03.535 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:47:03.535 DEBUG:teuthology.orchestra.run.smithi097:> if create_hard_links: 2023-07-06T03:47:03.535 DEBUG:teuthology.orchestra.run.smithi097:> for j in range(0): 2023-07-06T03:47:03.536 DEBUG:teuthology.orchestra.run.smithi097:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-06T03:47:03.536 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:47:03.536 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:47:03.536 DEBUG:teuthology.orchestra.run.smithi097:> finally: 2023-07-06T03:47:03.537 DEBUG:teuthology.orchestra.run.smithi097:> os.close(dirfd) 2023-07-06T03:47:03.537 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:47:05.065 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:47:05.065 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:47:05.066 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:47:05.066 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:47:05.066 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:47:05.067 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:05.067 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:47:05.067 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:47:05.067 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:47:05.068 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:47:05.068 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:05.069 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:47:05.069 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:47:05.069 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:47:05.070 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:47:05.070 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:05.070 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:47:05.071 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:47:05.071 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:47:05.071 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:47:05.072 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:47:05.072 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:47:05.072 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:47:05.073 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:47:05.073 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:05.073 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:47:05.074 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:47:05.248 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.129554.asok 2023-07-06T03:47:05.426 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.129554.asok 2023-07-06T03:47:05.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:05.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.129554.asok mds_sessions 2023-07-06T03:47:05.649 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:47:05.649 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5529, 2023-07-06T03:47:05.649 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:47:05.649 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:47:05.650 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:47:05.650 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 5529 2023-07-06T03:47:05.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:05.650 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:47:05.650 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:47:05.650 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:47:05.650 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1407883079 2023-07-06T03:47:05.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:05.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:05.651 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.5529 v1:192.168.0.1:0/1407883079", 2023-07-06T03:47:05.651 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "v1:192.168.0.1:0/1407883079", 2023-07-06T03:47:05.651 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions": [ 2023-07-06T03:47:05.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:05.651 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": 0, 2023-07-06T03:47:05.651 INFO:teuthology.orchestra.run.smithi097.stdout: "addrs": { 2023-07-06T03:47:05.652 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-07-06T03:47:05.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:05.652 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-07-06T03:47:05.652 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6836", 2023-07-06T03:47:05.652 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 663657264 2023-07-06T03:47:05.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:05.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:05.653 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:47:05.653 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6837", 2023-07-06T03:47:05.653 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 663657264 2023-07-06T03:47:05.653 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:05.653 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:47:05.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:05.653 INFO:teuthology.orchestra.run.smithi097.stdout: "seq": 3006, 2023-07-06T03:47:05.653 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_gen": 0, 2023-07-06T03:47:05.654 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_ttl": "2023-07-06T03:47:51.864924+0000", 2023-07-06T03:47:05.654 INFO:teuthology.orchestra.run.smithi097.stdout: "last_cap_renew_request": "2023-07-06T03:46:51.864924+0000", 2023-07-06T03:47:05.654 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_renew_seq": 2, 2023-07-06T03:47:05.654 INFO:teuthology.orchestra.run.smithi097.stdout: "num_caps": 109, 2023-07-06T03:47:05.654 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "open" 2023-07-06T03:47:05.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:05.654 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-07-06T03:47:05.654 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 16 2023-07-06T03:47:05.655 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:47:05.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:05.663 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'find | wc') 2023-07-06T03:47:09.938 INFO:teuthology.orchestra.run.smithi097.stdout: 1507 1507 23384 2023-07-06T03:47:09.940 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:47:10.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.137+0000 7f90225ff700 1 Processor -- start 2023-07-06T03:47:10.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.137+0000 7f90225ff700 1 -- start start 2023-07-06T03:47:10.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.138+0000 7f90225ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f901c15c520 0x7f901c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:10.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.138+0000 7f90225ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 0x7f901c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:10.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.138+0000 7f90225ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901c0cdae0 con 0x7f901c160180 2023-07-06T03:47:10.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.138+0000 7f90225ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901c0cd8d0 con 0x7f901c15c520 2023-07-06T03:47:10.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.138+0000 7f90225ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0b6640 con 0x7f901c160d00 2023-07-06T03:47:10.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.139+0000 7f9013fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f901c160180 0x7f901c1605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41750/0 (socket says 172.21.15.97:41750) 2023-07-06T03:47:10.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.139+0000 7f9013fff700 1 -- 172.21.15.97:0/393080124 learned_addr learned my addr 172.21.15.97:0/393080124 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:10.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.139+0000 7f9020b9c700 1 --2- 172.21.15.97:0/393080124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 0x7f901c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:10.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.139+0000 7f901bfff700 1 --2- 172.21.15.97:0/393080124 >> v2:172.21.15.146:3300/0 conn(0x7f901c15c520 0x7f901c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:10.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.139+0000 7f901b7fe700 1 -- 172.21.15.97:0/393080124 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678785147 0 0) 0x7f901c0cdae0 con 0x7f901c160180 2023-07-06T03:47:10.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.139+0000 7f901b7fe700 1 -- 172.21.15.97:0/393080124 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90080035b0 con 0x7f901c160180 2023-07-06T03:47:10.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.139+0000 7f901bfff700 1 -- 172.21.15.97:0/393080124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 msgr2=0x7f901c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:10.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.139+0000 7f901bfff700 1 --2- 172.21.15.97:0/393080124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 0x7f901c0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.139+0000 7f901bfff700 1 -- 172.21.15.97:0/393080124 >> v1:172.21.15.97:6789/0 conn(0x7f901c160180 legacy=0x7f901c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:10.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.139+0000 7f901bfff700 1 -- 172.21.15.97:0/393080124 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c0b6920 con 0x7f901c15c520 2023-07-06T03:47:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.139+0000 7f9020b9c700 1 --2- 172.21.15.97:0/393080124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 0x7f901c0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:47:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.140+0000 7f901bfff700 1 --2- 172.21.15.97:0/393080124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f901c15c520 0x7f901c0cd390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f900c009840 tx=0x7f900c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fd9b9941ee704c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.140+0000 7f901b7fe700 1 -- 172.21.15.97:0/393080124 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c011830 con 0x7f901c15c520 2023-07-06T03:47:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.140+0000 7f901b7fe700 1 -- 172.21.15.97:0/393080124 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c011990 con 0x7f901c15c520 2023-07-06T03:47:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.140+0000 7f901b7fe700 1 -- 172.21.15.97:0/393080124 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c011d00 con 0x7f901c15c520 2023-07-06T03:47:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.140+0000 7f90225ff700 1 -- 172.21.15.97:0/393080124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f901c15c520 msgr2=0x7f901c0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:10.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.140+0000 7f90225ff700 1 --2- 172.21.15.97:0/393080124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f901c15c520 0x7f901c0cd390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f900c009840 tx=0x7f900c009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:10.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.141+0000 7f90225ff700 1 -- 172.21.15.97:0/393080124 shutdown_connections 2023-07-06T03:47:10.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.141+0000 7f90225ff700 1 --2- 172.21.15.97:0/393080124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f901c15c520 0x7f901c0cd390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.141+0000 7f90225ff700 1 --2- 172.21.15.97:0/393080124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 0x7f901c0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.141+0000 7f90225ff700 1 -- 172.21.15.97:0/393080124 >> 172.21.15.97:0/393080124 conn(0x7f901c071790 msgr2=0x7f901c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.141+0000 7f90225ff700 1 -- 172.21.15.97:0/393080124 shutdown_connections 2023-07-06T03:47:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.141+0000 7f90225ff700 1 -- 172.21.15.97:0/393080124 wait complete. 2023-07-06T03:47:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.142+0000 7f90225ff700 1 Processor -- start 2023-07-06T03:47:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.142+0000 7f90225ff700 1 -- start start 2023-07-06T03:47:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.143+0000 7f90225ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f901c15c520 0x7f901c0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:10.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.143+0000 7f90225ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c160180 0x7f901c14d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:10.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.143+0000 7f90225ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 0x7f901c15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:10.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.143+0000 7f90225ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c15db60 con 0x7f901c160180 2023-07-06T03:47:10.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.143+0000 7f90225ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c14d7f0 con 0x7f901c15c520 2023-07-06T03:47:10.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.143+0000 7f90225ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c15d5a0 con 0x7f901c160d00 2023-07-06T03:47:10.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.143+0000 7f9013fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c160180 0x7f901c14d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:10.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f9013fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c160180 0x7f901c14d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34792/0 (socket says 172.21.15.97:34792) 2023-07-06T03:47:10.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f9013fff700 1 -- 172.21.15.97:0/4220786778 learned_addr learned my addr 172.21.15.97:0/4220786778 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:10.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f901bfff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f901c15c520 0x7f901c0d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:10.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f9020b9c700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 0x7f901c15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:10.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f9013fff700 1 -- 172.21.15.97:0/4220786778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 msgr2=0x7f901c15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:10.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f9013fff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 0x7f901c15cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f9013fff700 1 -- 172.21.15.97:0/4220786778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f901c15c520 msgr2=0x7f901c0d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f9013fff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f901c15c520 0x7f901c0d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f9013fff700 1 -- 172.21.15.97:0/4220786778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9014009a70 con 0x7f901c160180 2023-07-06T03:47:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f9020b9c700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 0x7f901c15cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f9013fff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c160180 0x7f901c14d2b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f900400c420 tx=0x7f900400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f90197fa700 1 -- 172.21.15.97:0/4220786778 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004005d70 con 0x7f901c160180 2023-07-06T03:47:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f90197fa700 1 -- 172.21.15.97:0/4220786778 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90040070d0 con 0x7f901c160180 2023-07-06T03:47:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f90197fa700 1 -- 172.21.15.97:0/4220786778 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004007440 con 0x7f901c160180 2023-07-06T03:47:10.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.144+0000 7f90225ff700 1 -- 172.21.15.97:0/4220786778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c0094a0 con 0x7f901c160180 2023-07-06T03:47:10.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.145+0000 7f90225ff700 1 -- 172.21.15.97:0/4220786778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c15e540 con 0x7f901c160180 2023-07-06T03:47:10.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.145+0000 7f90225ff700 1 -- 172.21.15.97:0/4220786778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c15e3a0 con 0x7f901c160180 2023-07-06T03:47:10.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.147+0000 7f90197fa700 1 -- 172.21.15.97:0/4220786778 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f90040075a0 con 0x7f901c160180 2023-07-06T03:47:10.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.148+0000 7f90197fa700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ffc06ace0 0x7f8ffc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.148+0000 7f90197fa700 1 -- 172.21.15.97:0/4220786778 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f9004088820 con 0x7f901c160180 2023-07-06T03:47:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.148+0000 7f901bfff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ffc06ace0 0x7f8ffc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.149+0000 7f901bfff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ffc06ace0 0x7f8ffc06d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f900c000c00 tx=0x7f900c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.153+0000 7f90197fa700 1 -- 172.21.15.97:0/4220786778 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f901c15e3a0 con 0x7f901c160180 2023-07-06T03:47:10.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.391+0000 7f90225ff700 1 -- 172.21.15.97:0/4220786778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f901c15e3a0 con 0x7f901c160180 2023-07-06T03:47:10.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.392+0000 7f90197fa700 1 -- 172.21.15.97:0/4220786778 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f901c15e3a0 con 0x7f901c160180 2023-07-06T03:47:10.394 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:10.395 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/1839628004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1839628004},{"type":"v1","addr":"172.21.15.146:6834","nonce":1839628004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4937,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2131262285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2131262285},{"type":"v1","addr":"172.21.15.146:6835","nonce":2131262285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1367937004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367937004},{"type":"v1","addr":"172.21.15.97:6835","nonce":1367937004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:46:37.331190+0000","modified":"2023-07-06T03:46:40.864523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/663657264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":663657264},{"type":"v1","addr":"172.21.15.97:6837","nonce":663657264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T03:47:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 -- 172.21.15.97:0/4220786778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ffc06ace0 msgr2=0x7f8ffc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ffc06ace0 0x7f8ffc06d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f900c000c00 tx=0x7f900c01c040 comp rx=0 tx=0).stop 2023-07-06T03:47:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 -- 172.21.15.97:0/4220786778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c160180 msgr2=0x7f901c14d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c160180 0x7f901c14d2b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f900400c420 tx=0x7f900400c730 comp rx=0 tx=0).stop 2023-07-06T03:47:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 -- 172.21.15.97:0/4220786778 shutdown_connections 2023-07-06T03:47:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ffc06ace0 0x7f8ffc06d190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f901c160d00 0x7f901c15cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f901c15c520 0x7f901c0d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 --2- 172.21.15.97:0/4220786778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901c160180 0x7f901c14d2b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 -- 172.21.15.97:0/4220786778 >> 172.21.15.97:0/4220786778 conn(0x7f901c071790 msgr2=0x7f901c071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:10.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 -- 172.21.15.97:0/4220786778 shutdown_connections 2023-07-06T03:47:10.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.394+0000 7f90225ff700 1 -- 172.21.15.97:0/4220786778 wait complete. 2023-07-06T03:47:10.399 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2023-07-06T03:47:10.415 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-07-06T03:47:10.631 INFO:teuthology.orchestra.run.smithi097.stdout:[{"id":5380,"entity":{"name":{"type":"client","num":5380},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":4270566397}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":17.446713451000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.8897064847082432,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5380 192.168.0.1:0/4270566397","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"120442","root":"/"}},{"id":5529,"entity":{"name":{"type":"client","num":5529},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1407883079}},"state":"open","num_leases":6,"num_caps":350,"request_load_avg":1552,"uptime":19.749675626999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":2516.0132595478813,"halflife":60},"release_caps":{"value":2420.3414706263379,"halflife":60},"recall_caps_throttle":{"value":960.49577986858196,"halflife":1.5},"recall_caps_throttle2o":{"value":239.23209673154273,"halflife":0.5},"session_cache_liveness":{"value":4561.9960612154755,"halflife":300},"cap_acquisition":{"value":1218.8883848141745,"halflife":10},"last_trim_completed_requests_tid":1746,"last_trim_completed_flushes_tid":1501,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5529 192.168.0.1:0/1407883079","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000007d6","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"129554","root":"/"}}] 2023-07-06T03:47:10.632 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "120442", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 4270566397, "type": "any" }, "name": { "num": 5380, "type": "client" } }, "id": 5380, "inst": "client.5380 192.168.0.1:0/4270566397", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.889706484708243 }, "state": "open", "uptime": 17.446713451 }, { "cap_acquisition": { "halflife": 10, "value": 1218.8883848141745 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "129554", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1407883079, "type": "any" }, "name": { "num": 5529, "type": "client" } }, "id": 5529, "inst": "client.5529 192.168.0.1:0/1407883079", "last_trim_completed_flushes_tid": 1501, "last_trim_completed_requests_tid": 1746, "num_caps": 350, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 6, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 1000, "start": "0x100000007d6" } ], "recall_caps": { "halflife": 60, "value": 2516.0132595478813 }, "recall_caps_throttle": { "halflife": 1.5, "value": 960.495779868582 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 239.23209673154273 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 2420.341470626338 }, "request_load_avg": 1552, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 4561.9960612154755 }, "state": "open", "uptime": 19.749675627 } ] 2023-07-06T03:47:10.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:47:10.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.824+0000 7f985ddff700 1 Processor -- start 2023-07-06T03:47:10.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.825+0000 7f985ddff700 1 -- start start 2023-07-06T03:47:10.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.826+0000 7f985ddff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f98580cd390 0x7f9858160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:10.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.826+0000 7f985ddff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98580cdae0 con 0x7f985815c520 2023-07-06T03:47:10.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.826+0000 7f985ddff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98580b6640 con 0x7f9858160dd0 2023-07-06T03:47:10.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.826+0000 7f985ddff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98580b6920 con 0x7f98580cd390 2023-07-06T03:47:10.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.826+0000 7f9856ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f98580cd390 0x7f9858160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:10.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.826+0000 7f98577fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f985815c520 0x7f9858160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41754/0 (socket says 172.21.15.97:41754) 2023-07-06T03:47:10.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.826+0000 7f98577fe700 1 -- 172.21.15.97:0/1426296058 learned_addr learned my addr 172.21.15.97:0/1426296058 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:10.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.826+0000 7f98567fc700 1 -- 172.21.15.97:0/1426296058 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087405937 0 0) 0x7f98580cdae0 con 0x7f985815c520 2023-07-06T03:47:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.826+0000 7f98567fc700 1 -- 172.21.15.97:0/1426296058 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98400035b0 con 0x7f985815c520 2023-07-06T03:47:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.827+0000 7f9856ffd700 1 -- 172.21.15.97:0/1426296058 >> v1:172.21.15.146:6789/0 conn(0x7f9858160dd0 legacy=0x7f98580b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.827+0000 7f9856ffd700 1 -- 172.21.15.97:0/1426296058 >> v1:172.21.15.97:6789/0 conn(0x7f985815c520 legacy=0x7f9858160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.827+0000 7f9856ffd700 1 -- 172.21.15.97:0/1426296058 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98580b6c00 con 0x7f98580cd390 2023-07-06T03:47:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.827+0000 7f9856ffd700 1 --2- 172.21.15.97:0/1426296058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98580cd390 0x7f9858160890 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f984c009840 tx=0x7f984c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e92a474e0a31fe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.827+0000 7f98567fc700 1 -- 172.21.15.97:0/1426296058 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c011830 con 0x7f98580cd390 2023-07-06T03:47:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.827+0000 7f98567fc700 1 -- 172.21.15.97:0/1426296058 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c011990 con 0x7f98580cd390 2023-07-06T03:47:10.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.827+0000 7f98567fc700 1 -- 172.21.15.97:0/1426296058 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c011d20 con 0x7f98580cd390 2023-07-06T03:47:10.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.828+0000 7f985ddff700 1 -- 172.21.15.97:0/1426296058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98580cd390 msgr2=0x7f9858160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.828+0000 7f985ddff700 1 --2- 172.21.15.97:0/1426296058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98580cd390 0x7f9858160890 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f984c009840 tx=0x7f984c009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.828+0000 7f985ddff700 1 -- 172.21.15.97:0/1426296058 shutdown_connections 2023-07-06T03:47:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.828+0000 7f985ddff700 1 --2- 172.21.15.97:0/1426296058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98580cd390 0x7f9858160890 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.828+0000 7f985ddff700 1 -- 172.21.15.97:0/1426296058 >> 172.21.15.97:0/1426296058 conn(0x7f9858071790 msgr2=0x7f9858071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.828+0000 7f985ddff700 1 -- 172.21.15.97:0/1426296058 shutdown_connections 2023-07-06T03:47:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.828+0000 7f985ddff700 1 -- 172.21.15.97:0/1426296058 wait complete. 2023-07-06T03:47:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.829+0000 7f985ddff700 1 Processor -- start 2023-07-06T03:47:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.830+0000 7f985ddff700 1 -- start start 2023-07-06T03:47:10.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.830+0000 7f985ddff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98580cd390 0x7f98580d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:10.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.830+0000 7f985ddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f985815c520 0x7f98580d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:10.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.830+0000 7f985ddff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858160dd0 0x7f98581427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:10.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.830+0000 7f985ddff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858150500 con 0x7f9858160dd0 2023-07-06T03:47:10.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.830+0000 7f985ddff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98580d3c60 con 0x7f98580cd390 2023-07-06T03:47:10.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.830+0000 7f985ddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98580d36a0 con 0x7f985815c520 2023-07-06T03:47:10.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.831+0000 7f9857fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858160dd0 0x7f98581427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:10.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.831+0000 7f9857fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858160dd0 0x7f98581427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34818/0 (socket says 172.21.15.97:34818) 2023-07-06T03:47:10.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.831+0000 7f9857fff700 1 -- 172.21.15.97:0/1872866212 learned_addr learned my addr 172.21.15.97:0/1872866212 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:10.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.831+0000 7f98577fe700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98580cd390 0x7f98580d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:10.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.831+0000 7f9856ffd700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f985815c520 0x7f98580d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.831+0000 7f9857fff700 1 -- 172.21.15.97:0/1872866212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f985815c520 msgr2=0x7f98580d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.831+0000 7f9857fff700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f985815c520 0x7f98580d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.831+0000 7f9857fff700 1 -- 172.21.15.97:0/1872866212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98580cd390 msgr2=0x7f98580d2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.831+0000 7f9857fff700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98580cd390 0x7f98580d2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.831+0000 7f9857fff700 1 -- 172.21.15.97:0/1872866212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c0094a0 con 0x7f9858160dd0 2023-07-06T03:47:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.831+0000 7f9857fff700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858160dd0 0x7f98581427f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f985000cc50 tx=0x7f985000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.832+0000 7f983ffff700 1 -- 172.21.15.97:0/1872866212 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850003c00 con 0x7f9858160dd0 2023-07-06T03:47:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.832+0000 7f983ffff700 1 -- 172.21.15.97:0/1872866212 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9850003d60 con 0x7f9858160dd0 2023-07-06T03:47:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.832+0000 7f983ffff700 1 -- 172.21.15.97:0/1872866212 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000aab0 con 0x7f9858160dd0 2023-07-06T03:47:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.832+0000 7f985ddff700 1 -- 172.21.15.97:0/1872866212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98580d3980 con 0x7f9858160dd0 2023-07-06T03:47:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.832+0000 7f985ddff700 1 -- 172.21.15.97:0/1872866212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985815d850 con 0x7f9858160dd0 2023-07-06T03:47:10.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.833+0000 7f985ddff700 1 -- 172.21.15.97:0/1872866212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985815d3d0 con 0x7f9858160dd0 2023-07-06T03:47:10.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.834+0000 7f983ffff700 1 -- 172.21.15.97:0/1872866212 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f985000ac10 con 0x7f9858160dd0 2023-07-06T03:47:10.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.835+0000 7f983ffff700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f984406ace0 0x7f984406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:10.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.835+0000 7f983ffff700 1 -- 172.21.15.97:0/1872866212 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f9850089480 con 0x7f9858160dd0 2023-07-06T03:47:10.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.835+0000 7f98577fe700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f984406ace0 0x7f984406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:10.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.836+0000 7f98577fe700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f984406ace0 0x7f984406d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9848007fd0 tx=0x7f9848007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:10.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:10.840+0000 7f983ffff700 1 -- 172.21.15.97:0/1872866212 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f985815d3d0 con 0x7f9858160dd0 2023-07-06T03:47:11.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.082+0000 7f985ddff700 1 -- 172.21.15.97:0/1872866212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f985815d3d0 con 0x7f9858160dd0 2023-07-06T03:47:11.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.082+0000 7f983ffff700 1 -- 172.21.15.97:0/1872866212 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f985815d3d0 con 0x7f9858160dd0 2023-07-06T03:47:11.086 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:11.086 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/1839628004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1839628004},{"type":"v1","addr":"172.21.15.146:6834","nonce":1839628004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4937,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2131262285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2131262285},{"type":"v1","addr":"172.21.15.146:6835","nonce":2131262285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1367937004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367937004},{"type":"v1","addr":"172.21.15.97:6835","nonce":1367937004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:46:37.331190+0000","modified":"2023-07-06T03:46:40.864523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/663657264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":663657264},{"type":"v1","addr":"172.21.15.97:6837","nonce":663657264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T03:47:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 -- 172.21.15.97:0/1872866212 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f984406ace0 msgr2=0x7f984406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f984406ace0 0x7f984406d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9848007fd0 tx=0x7f9848007dc0 comp rx=0 tx=0).stop 2023-07-06T03:47:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 -- 172.21.15.97:0/1872866212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858160dd0 msgr2=0x7f98581427f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:11.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858160dd0 0x7f98581427f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f985000cc50 tx=0x7f985000cf60 comp rx=0 tx=0).stop 2023-07-06T03:47:11.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 -- 172.21.15.97:0/1872866212 shutdown_connections 2023-07-06T03:47:11.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f984406ace0 0x7f984406d190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:11.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858160dd0 0x7f98581427f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:11.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98580cd390 0x7f98580d2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:11.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 --2- 172.21.15.97:0/1872866212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f985815c520 0x7f98580d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:11.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 -- 172.21.15.97:0/1872866212 >> 172.21.15.97:0/1872866212 conn(0x7f9858071790 msgr2=0x7f98580c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:11.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 -- 172.21.15.97:0/1872866212 shutdown_connections 2023-07-06T03:47:11.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.085+0000 7f985ddff700 1 -- 172.21.15.97:0/1872866212 wait complete. 2023-07-06T03:47:11.091 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2023-07-06T03:47:11.105 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2023-07-06T03:47:11.321 INFO:teuthology.orchestra.run.smithi097.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":938,"msgr_send_messages":939,"msgr_recv_bytes":231076,"msgr_send_bytes":3926022,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":0.073347699,"msgr_running_send_time":0.027911659,"msgr_running_recv_time":0.016323690,"msgr_running_fast_dispatch_time":0.018955199,"msgr_send_messages_queue_lat":{"avgcount":938,"sum":0.268763750,"avgtime":0.000286528},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":4974,"msgr_send_messages":6472,"msgr_recv_bytes":1405482,"msgr_send_bytes":3423997,"msgr_created_connections":8,"msgr_active_connections":7,"msgr_running_total_time":0.529065931,"msgr_running_send_time":0.278547625,"msgr_running_recv_time":0.354286848,"msgr_running_fast_dispatch_time":0.001367949,"msgr_send_messages_queue_lat":{"avgcount":6471,"sum":0.162912795,"avgtime":0.000025175},"msgr_handle_ack_lat":{"avgcount":11394,"sum":0.002093083,"avgtime":0.000000183},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":99376},"AsyncMessenger::Worker-2":{"msgr_recv_messages":709,"msgr_send_messages":682,"msgr_recv_bytes":277341,"msgr_send_bytes":4491950,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.063053825,"msgr_running_send_time":0.029084756,"msgr_running_recv_time":0.022757541,"msgr_running_fast_dispatch_time":0.014012326,"msgr_send_messages_queue_lat":{"avgcount":682,"sum":0.216281055,"avgtime":0.000317127},"msgr_handle_ack_lat":{"avgcount":28,"sum":0.000004732,"avgtime":0.000000169},"msgr_recv_encrypted_bytes":116480,"msgr_send_encrypted_bytes":15760},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":3063,"sum":1.202441738,"avgtime":0.000392569}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000014883,"avgtime":0.000007441}},"mds":{"request":1748,"reply":1748,"reply_latency":{"avgcount":1748,"sum":5.259326094,"avgtime":0.003008767},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":1516,"inodes_top":0,"inodes_bottom":1516,"inodes_pin_tail":0,"inodes_pinned":1516,"inodes_expired":0,"inodes_with_caps":350,"caps":351,"subtrees":2,"traverse":1989,"traverse_hit":1568,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":427341,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1500,"root_rbytes":3840,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":188,"ceph_cap_op_grant":1500,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1688,"handle_client_caps_dirty":1500,"handle_client_cap_release":7,"process_request_cap_release":1507},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4521,"evex":0,"evtrm":0,"ev":4521,"evexg":0,"evexd":0,"segadd":6,"segex":0,"segtrm":0,"seg":6,"segexg":0,"segexd":0,"expos":4194304,"wrpos":13744862,"rdpos":0,"jlat":{"avgcount":1513,"sum":3.006776580,"avgtime":0.001987294},"replayed":0},"mds_mem":{"ino":1519,"ino+":1519,"ino-":0,"dir":18,"dir+":18,"dir-":0,"dn":1516,"dn+":1516,"dn-":0,"cap":351,"cap+":2762,"cap-":2411,"rss":63836,"heap":207196},"mds_server":{"dispatch_client_request":1768,"dispatch_server_request":0,"handle_client_request":1748,"handle_client_session":1506,"handle_peer_request":0,"req_create_latency":{"avgcount":1500,"sum":0.797759491,"avgtime":0.000531839},"req_getattr_latency":{"avgcount":23,"sum":0.194781152,"avgtime":0.008468745},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":211,"sum":0.208648105,"avgtime":0.000988853},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":6,"sum":0.006965927,"avgtime":0.001160987},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":7,"sum":4.050721022,"avgtime":0.578674431},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000450397,"avgtime":0.000450397},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":8,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":2,"session_add":2,"session_remove":0,"sessions_open":2,"sessions_stale":0,"total_load":1565,"average_load":782,"avg_session_uptime":17},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":233912,"buffer_anon_items":91,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":6464,"osdmap_items":84,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":4494240,"mds_co_items":29471,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":1558,"op_send_bytes":9647583,"op_resend":37,"op_reply":1558,"op_latency":{"avgcount":1558,"sum":3.769938112,"avgtime":0.002419729},"op_inflight":0,"oplen_avg":{"avgcount":1558,"sum":1587},"op":1558,"op_r":0,"op_w":1558,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":1513,"osdop_writefull":9,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":8,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":36,"map_full":0,"map_inc":3,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":21,"omap_rd":0,"omap_del":3},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":6615,"get_sum":1313581,"get_or_fail_fail":0,"get_or_fail_success":6615,"take":0,"take_sum":0,"put":6615,"put_sum":1313581,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":1558,"take_sum":9647583,"put":1558,"put_sum":9647583,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":1558,"take_sum":1558,"put":1558,"put_sum":1558,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x555cb5a99cc0":{"val":0,"max":3758096384,"get_started":0,"get":4521,"get_sum":9550558,"get_or_fail_fail":0,"get_or_fail_success":4521,"take":0,"take_sum":0,"put":1513,"put_sum":9550558,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2023-07-06T03:47:11.323 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 231076, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 938, "msgr_running_fast_dispatch_time": 0.018955199, "msgr_running_recv_time": 0.01632369, "msgr_running_send_time": 0.027911659, "msgr_running_total_time": 0.073347699, "msgr_send_bytes": 3926022, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 939, "msgr_send_messages_queue_lat": { "avgcount": 938, "avgtime": 0.000286528, "sum": 0.26876375 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 7, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 11394, "avgtime": 1.83e-07, "sum": 0.002093083 }, "msgr_recv_bytes": 1405482, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 4974, "msgr_running_fast_dispatch_time": 0.001367949, "msgr_running_recv_time": 0.354286848, "msgr_running_send_time": 0.278547625, "msgr_running_total_time": 0.529065931, "msgr_send_bytes": 3423997, "msgr_send_encrypted_bytes": 99376, "msgr_send_messages": 6472, "msgr_send_messages_queue_lat": { "avgcount": 6471, "avgtime": 2.5175e-05, "sum": 0.162912795 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 28, "avgtime": 1.69e-07, "sum": 4.732e-06 }, "msgr_recv_bytes": 277341, "msgr_recv_encrypted_bytes": 116480, "msgr_recv_messages": 709, "msgr_running_fast_dispatch_time": 0.014012326, "msgr_running_recv_time": 0.022757541, "msgr_running_send_time": 0.029084756, "msgr_running_total_time": 0.063053825, "msgr_send_bytes": 4491950, "msgr_send_encrypted_bytes": 15760, "msgr_send_messages": 682, "msgr_send_messages_queue_lat": { "avgcount": 682, "avgtime": 0.000317127, "sum": 0.216281055 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 3063, "avgtime": 0.000392569, "sum": 1.202441738 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 7.441e-06, "sum": 1.4883e-05 }, "queue_len": 0 }, "mds": { "caps": 351, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 1500, "ceph_cap_op_revoke": 188, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 7, "handle_client_caps": 1688, "handle_client_caps_dirty": 1500, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 1516, "inodes_bottom": 1516, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 1516, "inodes_top": 0, "inodes_with_caps": 350, "load_cent": 427341, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1507, "q": 0, "reply": 1748, "reply_latency": { "avgcount": 1748, "avgtime": 0.003008767, "sum": 5.259326094 }, "request": 1748, "root_rbytes": 3840, "root_rfiles": 1500, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 1989, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 1568, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 4521, "evadd": 4521, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 1513, "avgtime": 0.001987294, "sum": 3.00677658 }, "rdpos": 0, "replayed": 0, "seg": 6, "segadd": 6, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 13744862 }, "mds_mem": { "cap": 351, "cap+": 2762, "cap-": 2411, "dir": 18, "dir+": 18, "dir-": 0, "dn": 1516, "dn+": 1516, "dn-": 0, "heap": 207196, "ino": 1519, "ino+": 1519, "ino-": 0, "rss": 63836 }, "mds_server": { "cap_acquisition_throttle": 8, "cap_revoke_eviction": 0, "dispatch_client_request": 1768, "dispatch_server_request": 0, "handle_client_request": 1748, "handle_client_session": 1506, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1500, "avgtime": 0.000531839, "sum": 0.797759491 }, "req_getattr_latency": { "avgcount": 23, "avgtime": 0.008468745, "sum": 0.194781152 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 211, "avgtime": 0.000988853, "sum": 0.208648105 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 6, "avgtime": 0.001160987, "sum": 0.006965927 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 7, "avgtime": 0.578674431, "sum": 4.050721022 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000450397, "sum": 0.000450397 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 782, "avg_session_uptime": 17, "session_add": 2, "session_count": 2, "session_remove": 0, "sessions_open": 2, "sessions_stale": 0, "total_load": 1565 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 233912, "buffer_anon_items": 91, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 4494240, "mds_co_items": 29471, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 6464, "osdmap_items": 84, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 36, "map_full": 0, "map_inc": 3, "omap_del": 3, "omap_rd": 0, "omap_wr": 21, "op": 1558, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 1558, "avgtime": 0.002419729, "sum": 3.769938112 }, "op_pg": 0, "op_r": 0, "op_reply": 1558, "op_resend": 37, "op_rmw": 0, "op_send": 1558, "op_send_bytes": 9647583, "op_w": 1558, "oplen_avg": { "avgcount": 1558, "sum": 1587 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 8, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 1513, "osdop_writefull": 9, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 6615, "get_or_fail_fail": 0, "get_or_fail_success": 6615, "get_started": 0, "get_sum": 1313581, "max": 104857600, "put": 6615, "put_sum": 1313581, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 1558, "put_sum": 9647583, "take": 1558, "take_sum": 9647583, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 1558, "put_sum": 1558, "take": 1558, "take_sum": 1558, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x555cb5a99cc0": { "get": 4521, "get_or_fail_fail": 0, "get_or_fail_success": 4521, "get_started": 0, "get_sum": 9550558, "max": 3758096384, "put": 1513, "put_sum": 9550558, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2023-07-06T03:47:11.324 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:47:11.324 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:47:11.422 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:47:11.528 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:47:11.529 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:47:11.529 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:47:11.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:47:11.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.805+0000 7f33315ff700 1 Processor -- start 2023-07-06T03:47:11.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.805+0000 7f33315ff700 1 -- start start 2023-07-06T03:47:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.806+0000 7f33315ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f332c071a00 0x7f332c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.806+0000 7f33315ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c06ddb0 con 0x7f332c158960 2023-07-06T03:47:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.806+0000 7f33315ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c15d310 con 0x7f332c15ce90 2023-07-06T03:47:11.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.806+0000 7f33315ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332c06bbf0 con 0x7f332c071a00 2023-07-06T03:47:11.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.807+0000 7f332a7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f332c071a00 0x7f332c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:11.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.807+0000 7f332affd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f332c158960 0x7f332c0712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41766/0 (socket says 172.21.15.97:41766) 2023-07-06T03:47:11.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.807+0000 7f332affd700 1 -- 172.21.15.97:0/3061953026 learned_addr learned my addr 172.21.15.97:0/3061953026 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:11.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.807+0000 7f332a7fc700 1 -- 172.21.15.97:0/3061953026 >> v1:172.21.15.146:6789/0 conn(0x7f332c15ce90 legacy=0x7f332c158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:11.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.807+0000 7f332a7fc700 1 -- 172.21.15.97:0/3061953026 >> v1:172.21.15.97:6789/0 conn(0x7f332c158960 legacy=0x7f332c0712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.807+0000 7f332a7fc700 1 -- 172.21.15.97:0/3061953026 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c06bed0 con 0x7f332c071a00 2023-07-06T03:47:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.807+0000 7f3329ffb700 1 -- 172.21.15.97:0/3061953026 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533467771 0 0) 0x7f332c06ddb0 con 0x7f332c158960 2023-07-06T03:47:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.807+0000 7f332a7fc700 1 --2- 172.21.15.97:0/3061953026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c071a00 0x7f332c06d6f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3320009840 tx=0x7f3320009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b451ef7a0f4ba5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.807+0000 7f3329ffb700 1 -- 172.21.15.97:0/3061953026 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320004230 con 0x7f332c071a00 2023-07-06T03:47:11.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.808+0000 7f3329ffb700 1 -- 172.21.15.97:0/3061953026 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3320004390 con 0x7f332c071a00 2023-07-06T03:47:11.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.808+0000 7f3329ffb700 1 -- 172.21.15.97:0/3061953026 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33200036a0 con 0x7f332c071a00 2023-07-06T03:47:11.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.808+0000 7f33315ff700 1 -- 172.21.15.97:0/3061953026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c071a00 msgr2=0x7f332c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:11.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.808+0000 7f33315ff700 1 --2- 172.21.15.97:0/3061953026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c071a00 0x7f332c06d6f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3320009840 tx=0x7f3320009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:11.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.808+0000 7f33315ff700 1 -- 172.21.15.97:0/3061953026 shutdown_connections 2023-07-06T03:47:11.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.808+0000 7f33315ff700 1 --2- 172.21.15.97:0/3061953026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c071a00 0x7f332c06d6f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:11.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.808+0000 7f33315ff700 1 -- 172.21.15.97:0/3061953026 >> 172.21.15.97:0/3061953026 conn(0x7f332c0ce8c0 msgr2=0x7f332c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:11.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.808+0000 7f33315ff700 1 -- 172.21.15.97:0/3061953026 shutdown_connections 2023-07-06T03:47:11.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.808+0000 7f33315ff700 1 -- 172.21.15.97:0/3061953026 wait complete. 2023-07-06T03:47:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.810+0000 7f33315ff700 1 Processor -- start 2023-07-06T03:47:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.810+0000 7f33315ff700 1 -- start start 2023-07-06T03:47:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f33315ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c071a00 0x7f332c14f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:11.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f33315ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f332c158960 0x7f332c14f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f33315ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f332c15ce90 0x7f332c1f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f33315ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c14b1a0 con 0x7f332c071a00 2023-07-06T03:47:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f33315ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c0b2720 con 0x7f332c158960 2023-07-06T03:47:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f33315ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c0b2a00 con 0x7f332c15ce90 2023-07-06T03:47:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f332affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c071a00 0x7f332c14f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f332affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c071a00 0x7f332c14f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34826/0 (socket says 172.21.15.97:34826) 2023-07-06T03:47:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f332affd700 1 -- 172.21.15.97:0/1480739562 learned_addr learned my addr 172.21.15.97:0/1480739562 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:11.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f332affd700 1 -- 172.21.15.97:0/1480739562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f332c15ce90 msgr2=0x7f332c1f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:11.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f332affd700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f332c15ce90 0x7f332c1f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:11.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f332affd700 1 -- 172.21.15.97:0/1480739562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f332c158960 msgr2=0x7f332c14f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:11.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f332affd700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f332c158960 0x7f332c14f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:11.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.811+0000 7f332affd700 1 -- 172.21.15.97:0/1480739562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200094a0 con 0x7f332c071a00 2023-07-06T03:47:11.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.812+0000 7f332affd700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c071a00 0x7f332c14f3a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f331c0011e0 tx=0x7f331c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:11.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.812+0000 7f33137fe700 1 -- 172.21.15.97:0/1480739562 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c005d70 con 0x7f332c071a00 2023-07-06T03:47:11.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.812+0000 7f33137fe700 1 -- 172.21.15.97:0/1480739562 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331c0070d0 con 0x7f332c071a00 2023-07-06T03:47:11.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.812+0000 7f33137fe700 1 -- 172.21.15.97:0/1480739562 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c007480 con 0x7f332c071a00 2023-07-06T03:47:11.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.812+0000 7f33315ff700 1 -- 172.21.15.97:0/1480739562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c0b2ce0 con 0x7f332c071a00 2023-07-06T03:47:11.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.812+0000 7f33315ff700 1 -- 172.21.15.97:0/1480739562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c143910 con 0x7f332c071a00 2023-07-06T03:47:11.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.813+0000 7f33315ff700 1 -- 172.21.15.97:0/1480739562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c067830 con 0x7f332c071a00 2023-07-06T03:47:11.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.814+0000 7f33137fe700 1 -- 172.21.15.97:0/1480739562 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f331c0075e0 con 0x7f332c071a00 2023-07-06T03:47:11.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.815+0000 7f33137fe700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f331806ace0 0x7f331806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:11.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.815+0000 7f33137fe700 1 -- 172.21.15.97:0/1480739562 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f331c088a90 con 0x7f332c071a00 2023-07-06T03:47:11.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.815+0000 7f332a7fc700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f331806ace0 0x7f331806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:11.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.816+0000 7f332a7fc700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f331806ace0 0x7f331806d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f332c0b19a0 tx=0x7f3320013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:11.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:11.819+0000 7f33137fe700 1 -- 172.21.15.97:0/1480739562 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f331c053ad0 con 0x7f332c071a00 2023-07-06T03:47:12.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.015+0000 7f33315ff700 1 -- 172.21.15.97:0/1480739562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f332c04e340 con 0x7f332c071a00 2023-07-06T03:47:12.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.016+0000 7f33137fe700 1 -- 172.21.15.97:0/1480739562 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18460 (secure 0 0 0) 0x7f331c0570f0 con 0x7f332c071a00 2023-07-06T03:47:12.019 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:46:52.661343+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:46:35.584013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T03:46:36.303248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.28","mappings":[{"from":4,"to":7}]},{"pgid":"6.36","mappings":[{"from":1,"to":2}]},{"pgid":"6.38","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:47:12.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.018+0000 7f33315ff700 1 -- 172.21.15.97:0/1480739562 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f331806ace0 msgr2=0x7f331806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.018+0000 7f33315ff700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f331806ace0 0x7f331806d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f332c0b19a0 tx=0x7f3320013040 comp rx=0 tx=0).stop 2023-07-06T03:47:12.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.018+0000 7f33315ff700 1 -- 172.21.15.97:0/1480739562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c071a00 msgr2=0x7f332c14f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.018+0000 7f33315ff700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c071a00 0x7f332c14f3a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f331c0011e0 tx=0x7f331c00cae0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.018+0000 7f33315ff700 1 -- 172.21.15.97:0/1480739562 shutdown_connections 2023-07-06T03:47:12.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.018+0000 7f33315ff700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f331806ace0 0x7f331806d190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.018+0000 7f33315ff700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f332c15ce90 0x7f332c1f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.018+0000 7f33315ff700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c071a00 0x7f332c14f3a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.018+0000 7f33315ff700 1 --2- 172.21.15.97:0/1480739562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f332c158960 0x7f332c14f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.018+0000 7f33315ff700 1 -- 172.21.15.97:0/1480739562 >> 172.21.15.97:0/1480739562 conn(0x7f332c0ce8c0 msgr2=0x7f332c0b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:12.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.019+0000 7f33315ff700 1 -- 172.21.15.97:0/1480739562 shutdown_connections 2023-07-06T03:47:12.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.019+0000 7f33315ff700 1 -- 172.21.15.97:0/1480739562 wait complete. 2023-07-06T03:47:12.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:46:35.584013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:46:36.303248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:47:12.040 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:47:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.235+0000 7f7f847ff700 1 Processor -- start 2023-07-06T03:47:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.236+0000 7f7f847ff700 1 -- start start 2023-07-06T03:47:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.237+0000 7f7f847ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7f7c0cbaa0 0x7f7f7c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.237+0000 7f7f847ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c15b6e0 con 0x7f7f7c15eb90 2023-07-06T03:47:12.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.237+0000 7f7f847ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c15e0b0 con 0x7f7f7c154890 2023-07-06T03:47:12.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.237+0000 7f7f847ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7c15e390 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.237+0000 7f7f82d9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7f7c15eb90 0x7f7f7c15ae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41782/0 (socket says 172.21.15.97:41782) 2023-07-06T03:47:12.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.237+0000 7f7f82d9c700 1 -- 172.21.15.97:0/3112646404 learned_addr learned my addr 172.21.15.97:0/3112646404 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:12.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.237+0000 7f7f81d9a700 1 --2- 172.21.15.97:0/3112646404 >> v2:172.21.15.146:3300/0 conn(0x7f7f7c0cbaa0 0x7f7f7c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:12.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.238+0000 7f7f81599700 1 -- 172.21.15.97:0/3112646404 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011720793 0 0) 0x7f7f7c15b6e0 con 0x7f7f7c15eb90 2023-07-06T03:47:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.238+0000 7f7f81d9a700 1 -- 172.21.15.97:0/3112646404 >> v1:172.21.15.146:6789/0 conn(0x7f7f7c154890 legacy=0x7f7f7c0cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.238+0000 7f7f81d9a700 1 -- 172.21.15.97:0/3112646404 >> v1:172.21.15.97:6789/0 conn(0x7f7f7c15eb90 legacy=0x7f7f7c15ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.238+0000 7f7f81d9a700 1 -- 172.21.15.97:0/3112646404 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c15e670 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.238+0000 7f7f81d9a700 1 --2- 172.21.15.97:0/3112646404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7c0cbaa0 0x7f7f7c15a940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009840 tx=0x7f7f6c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f849f65d1e769eb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.239+0000 7f7f81599700 1 -- 172.21.15.97:0/3112646404 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c011830 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.239+0000 7f7f81599700 1 -- 172.21.15.97:0/3112646404 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c011990 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.239+0000 7f7f81599700 1 -- 172.21.15.97:0/3112646404 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c011d20 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.239+0000 7f7f847ff700 1 -- 172.21.15.97:0/3112646404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7c0cbaa0 msgr2=0x7f7f7c15a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.239+0000 7f7f847ff700 1 --2- 172.21.15.97:0/3112646404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7c0cbaa0 0x7f7f7c15a940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009840 tx=0x7f7f6c009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:12.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.239+0000 7f7f847ff700 1 -- 172.21.15.97:0/3112646404 shutdown_connections 2023-07-06T03:47:12.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.239+0000 7f7f847ff700 1 --2- 172.21.15.97:0/3112646404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7c0cbaa0 0x7f7f7c15a940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.239+0000 7f7f847ff700 1 -- 172.21.15.97:0/3112646404 >> 172.21.15.97:0/3112646404 conn(0x7f7f7c06a060 msgr2=0x7f7f7c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:12.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.239+0000 7f7f847ff700 1 -- 172.21.15.97:0/3112646404 shutdown_connections 2023-07-06T03:47:12.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.239+0000 7f7f847ff700 1 -- 172.21.15.97:0/3112646404 wait complete. 2023-07-06T03:47:12.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.241+0000 7f7f847ff700 1 Processor -- start 2023-07-06T03:47:12.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.241+0000 7f7f847ff700 1 -- start start 2023-07-06T03:47:12.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.242+0000 7f7f847ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f7c0cbaa0 0x7f7f7c158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:12.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.242+0000 7f7f847ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f7c154890 0x7f7f7c14e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:12.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.242+0000 7f7f847ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7c15eb90 0x7f7f7c15c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:12.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.242+0000 7f7f847ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c15c230 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.242+0000 7f7f847ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c15b920 con 0x7f7f7c15eb90 2023-07-06T03:47:12.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.242+0000 7f7f847ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c158b40 con 0x7f7f7c154890 2023-07-06T03:47:12.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.242+0000 7f7f8259b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f7c0cbaa0 0x7f7f7c158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:12.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.242+0000 7f7f8259b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f7c0cbaa0 0x7f7f7c158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34836/0 (socket says 172.21.15.97:34836) 2023-07-06T03:47:12.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.242+0000 7f7f8259b700 1 -- 172.21.15.97:0/1946409809 learned_addr learned my addr 172.21.15.97:0/1946409809 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:12.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.242+0000 7f7f82d9c700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7c15eb90 0x7f7f7c15c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:12.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.242+0000 7f7f81d9a700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f7c154890 0x7f7f7c14e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:12.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f8259b700 1 -- 172.21.15.97:0/1946409809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f7c154890 msgr2=0x7f7f7c14e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f8259b700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f7c154890 0x7f7f7c14e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f8259b700 1 -- 172.21.15.97:0/1946409809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7c15eb90 msgr2=0x7f7f7c15c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f8259b700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7c15eb90 0x7f7f7c15c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f8259b700 1 -- 172.21.15.97:0/1946409809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0094a0 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f81d9a700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f7c154890 0x7f7f7c14e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:12.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f8259b700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f7c0cbaa0 0x7f7f7c158560 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800c4e0 tx=0x7f7f7800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:12.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f72ffd700 1 -- 172.21.15.97:0/1946409809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780052d0 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f72ffd700 1 -- 172.21.15.97:0/1946409809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f78005430 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f847ff700 1 -- 172.21.15.97:0/1946409809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c15bbe0 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f72ffd700 1 -- 172.21.15.97:0/1946409809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780038c0 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.243+0000 7f7f847ff700 1 -- 172.21.15.97:0/1946409809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c15a3f0 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.244+0000 7f7f847ff700 1 -- 172.21.15.97:0/1946409809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c15c510 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.245+0000 7f7f72ffd700 1 -- 172.21.15.97:0/1946409809 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f78003a20 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.246+0000 7f7f72ffd700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f6006ad10 0x7f7f6006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:12.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.246+0000 7f7f72ffd700 1 -- 172.21.15.97:0/1946409809 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7f78088a10 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.246+0000 7f7f81d9a700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f6006ad10 0x7f7f6006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:12.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.247+0000 7f7f81d9a700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f6006ad10 0x7f7f6006d1c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0adb60 tx=0x7f7f6c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:12.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.250+0000 7f7f72ffd700 1 -- 172.21.15.97:0/1946409809 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7f78053a50 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.445+0000 7f7f847ff700 1 -- 172.21.15.97:0/1946409809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7f7c04e340 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.446+0000 7f7f72ffd700 1 -- 172.21.15.97:0/1946409809 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f78017020 con 0x7f7f7c0cbaa0 2023-07-06T03:47:12.449 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:47:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.448+0000 7f7f847ff700 1 -- 172.21.15.97:0/1946409809 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f6006ad10 msgr2=0x7f7f6006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.448+0000 7f7f847ff700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f6006ad10 0x7f7f6006d1c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0adb60 tx=0x7f7f6c01c040 comp rx=0 tx=0).stop 2023-07-06T03:47:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.448+0000 7f7f847ff700 1 -- 172.21.15.97:0/1946409809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f7c0cbaa0 msgr2=0x7f7f7c158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.448+0000 7f7f847ff700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f7c0cbaa0 0x7f7f7c158560 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800c4e0 tx=0x7f7f7800c8a0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.448+0000 7f7f847ff700 1 -- 172.21.15.97:0/1946409809 shutdown_connections 2023-07-06T03:47:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.448+0000 7f7f847ff700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f6006ad10 0x7f7f6006d1c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.448+0000 7f7f847ff700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f7c15eb90 0x7f7f7c15c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.448+0000 7f7f847ff700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f7c0cbaa0 0x7f7f7c158560 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.448+0000 7f7f847ff700 1 --2- 172.21.15.97:0/1946409809 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f7c154890 0x7f7f7c14e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.448+0000 7f7f847ff700 1 -- 172.21.15.97:0/1946409809 >> 172.21.15.97:0/1946409809 conn(0x7f7f7c06a060 msgr2=0x7f7f7c06cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.449+0000 7f7f847ff700 1 -- 172.21.15.97:0/1946409809 shutdown_connections 2023-07-06T03:47:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.449+0000 7f7f847ff700 1 -- 172.21.15.97:0/1946409809 wait complete. 2023-07-06T03:47:12.469 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:47:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.665+0000 7f7b0fbff700 1 Processor -- start 2023-07-06T03:47:12.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.665+0000 7f7b0fbff700 1 -- start start 2023-07-06T03:47:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.665+0000 7f7b0fbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b0815c610 0x7f7b080b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.665+0000 7f7b0fbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b080cff90 con 0x7f7b0806b360 2023-07-06T03:47:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.665+0000 7f7b0fbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b080b5650 con 0x7f7b080d2210 2023-07-06T03:47:12.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.665+0000 7f7b0fbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b080d0780 con 0x7f7b0815c610 2023-07-06T03:47:12.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.665+0000 7f7b0d99b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b0815c610 0x7f7b080b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.665+0000 7f7b0d19a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7b0806b360 0x7f7b080b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41798/0 (socket says 172.21.15.97:41798) 2023-07-06T03:47:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.665+0000 7f7b0d19a700 1 -- 172.21.15.97:0/2262420024 learned_addr learned my addr 172.21.15.97:0/2262420024 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.666+0000 7f7b0d99b700 1 -- 172.21.15.97:0/2262420024 >> v1:172.21.15.146:6789/0 conn(0x7f7b080d2210 legacy=0x7f7b080d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.666+0000 7f7b0d99b700 1 -- 172.21.15.97:0/2262420024 >> v1:172.21.15.97:6789/0 conn(0x7f7b0806b360 legacy=0x7f7b080b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.666+0000 7f7b0d99b700 1 -- 172.21.15.97:0/2262420024 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b080b5400 con 0x7f7b0815c610 2023-07-06T03:47:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.666+0000 7f7b0c999700 1 -- 172.21.15.97:0/2262420024 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662838452 0 0) 0x7f7b080cff90 con 0x7f7b0806b360 2023-07-06T03:47:12.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.666+0000 7f7b0d99b700 1 --2- 172.21.15.97:0/2262420024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0815c610 0x7f7b080b47b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009840 tx=0x7f7b04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c56f8a5f2683e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:12.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.666+0000 7f7b0c999700 1 -- 172.21.15.97:0/2262420024 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04011830 con 0x7f7b0815c610 2023-07-06T03:47:12.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.666+0000 7f7b0c999700 1 -- 172.21.15.97:0/2262420024 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b04011990 con 0x7f7b0815c610 2023-07-06T03:47:12.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.666+0000 7f7b0c999700 1 -- 172.21.15.97:0/2262420024 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04011d20 con 0x7f7b0815c610 2023-07-06T03:47:12.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.667+0000 7f7b0fbff700 1 -- 172.21.15.97:0/2262420024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0815c610 msgr2=0x7f7b080b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.667+0000 7f7b0fbff700 1 --2- 172.21.15.97:0/2262420024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0815c610 0x7f7b080b47b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009840 tx=0x7f7b04009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:12.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.667+0000 7f7b0fbff700 1 -- 172.21.15.97:0/2262420024 shutdown_connections 2023-07-06T03:47:12.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.667+0000 7f7b0fbff700 1 --2- 172.21.15.97:0/2262420024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0815c610 0x7f7b080b47b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.667+0000 7f7b0fbff700 1 -- 172.21.15.97:0/2262420024 >> 172.21.15.97:0/2262420024 conn(0x7f7b0806a6c0 msgr2=0x7f7b0806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:12.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.667+0000 7f7b0fbff700 1 -- 172.21.15.97:0/2262420024 shutdown_connections 2023-07-06T03:47:12.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.667+0000 7f7b0fbff700 1 -- 172.21.15.97:0/2262420024 wait complete. 2023-07-06T03:47:12.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.669+0000 7f7b0fbff700 1 Processor -- start 2023-07-06T03:47:12.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.670+0000 7f7b0fbff700 1 -- start start 2023-07-06T03:47:12.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.671+0000 7f7b0fbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b0806b360 0x7f7b080b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:12.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.671+0000 7f7b0fbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b080d2210 0x7f7b0814fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:12.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.671+0000 7f7b0fbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0815c610 0x7f7b08150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:12.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.671+0000 7f7b0fbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0815e850 con 0x7f7b0815c610 2023-07-06T03:47:12.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.671+0000 7f7b0fbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0815eb30 con 0x7f7b0806b360 2023-07-06T03:47:12.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.671+0000 7f7b0fbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0815ee10 con 0x7f7b080d2210 2023-07-06T03:47:12.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.671+0000 7f7b0e19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0815c610 0x7f7b08150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:12.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.671+0000 7f7b0e19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0815c610 0x7f7b08150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34856/0 (socket says 172.21.15.97:34856) 2023-07-06T03:47:12.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.671+0000 7f7b0e19c700 1 -- 172.21.15.97:0/4097825815 learned_addr learned my addr 172.21.15.97:0/4097825815 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:12.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.671+0000 7f7b0d99b700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b0806b360 0x7f7b080b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:12.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.671+0000 7f7b0d19a700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b080d2210 0x7f7b0814fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:12.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7b0e19c700 1 -- 172.21.15.97:0/4097825815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b080d2210 msgr2=0x7f7b0814fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7b0e19c700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b080d2210 0x7f7b0814fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7b0e19c700 1 -- 172.21.15.97:0/4097825815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b0806b360 msgr2=0x7f7b080b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7b0e19c700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b0806b360 0x7f7b080b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7b0e19c700 1 -- 172.21.15.97:0/4097825815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040094a0 con 0x7f7b0815c610 2023-07-06T03:47:12.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7b0d19a700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b080d2210 0x7f7b0814fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:12.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7b0e19c700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0815c610 0x7f7b08150020 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000cc50 tx=0x7f7b0000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:12.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7afe7fc700 1 -- 172.21.15.97:0/4097825815 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00003c00 con 0x7f7b0815c610 2023-07-06T03:47:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7afe7fc700 1 -- 172.21.15.97:0/4097825815 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b00003d60 con 0x7f7b0815c610 2023-07-06T03:47:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7b0fbff700 1 -- 172.21.15.97:0/4097825815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0815f0f0 con 0x7f7b0815c610 2023-07-06T03:47:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7afe7fc700 1 -- 172.21.15.97:0/4097825815 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000ab60 con 0x7f7b0815c610 2023-07-06T03:47:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.672+0000 7f7b0fbff700 1 -- 172.21.15.97:0/4097825815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0815f8f0 con 0x7f7b0815c610 2023-07-06T03:47:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.673+0000 7f7b0fbff700 1 -- 172.21.15.97:0/4097825815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0815f610 con 0x7f7b0815c610 2023-07-06T03:47:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.674+0000 7f7afe7fc700 1 -- 172.21.15.97:0/4097825815 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b0000acc0 con 0x7f7b0815c610 2023-07-06T03:47:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.675+0000 7f7afe7fc700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 0x7f7aec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:12.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.675+0000 7f7afe7fc700 1 -- 172.21.15.97:0/4097825815 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7b00089650 con 0x7f7b0815c610 2023-07-06T03:47:12.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.675+0000 7f7b0d99b700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 0x7f7aec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:12.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.676+0000 7f7b0d99b700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 0x7f7aec06d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b040090b0 tx=0x7f7b04008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:12.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.679+0000 7f7afe7fc700 1 -- 172.21.15.97:0/4097825815 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7b0815f610 con 0x7f7b0815c610 2023-07-06T03:47:12.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.877+0000 7f7b0fbff700 1 -- 172.21.15.97:0/4097825815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7b0815f610 con 0x7f7b0815c610 2023-07-06T03:47:12.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.878+0000 7f7afe7fc700 1 -- 172.21.15.97:0/4097825815 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7b0815f610 con 0x7f7b0815c610 2023-07-06T03:47:12.881 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:47:12.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 -- 172.21.15.97:0/4097825815 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 msgr2=0x7f7aec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 0x7f7aec06d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b040090b0 tx=0x7f7b04008030 comp rx=0 tx=0).stop 2023-07-06T03:47:12.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 -- 172.21.15.97:0/4097825815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0815c610 msgr2=0x7f7b08150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:12.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0815c610 0x7f7b08150020 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000cc50 tx=0x7f7b0000cf60 comp rx=0 tx=0).stop 2023-07-06T03:47:12.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 -- 172.21.15.97:0/4097825815 shutdown_connections 2023-07-06T03:47:12.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7aec06ace0 0x7f7aec06d190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b0815c610 0x7f7b08150020 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b0806b360 0x7f7b080b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 --2- 172.21.15.97:0/4097825815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7b080d2210 0x7f7b0814fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:12.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 -- 172.21.15.97:0/4097825815 >> 172.21.15.97:0/4097825815 conn(0x7f7b0806a6c0 msgr2=0x7f7b080b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:12.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 -- 172.21.15.97:0/4097825815 shutdown_connections 2023-07-06T03:47:12.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:12.880+0000 7f7b0fbff700 1 -- 172.21.15.97:0/4097825815 wait complete. 2023-07-06T03:47:12.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:47:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.097+0000 7fd1f83ff700 1 Processor -- start 2023-07-06T03:47:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.097+0000 7fd1f83ff700 1 -- start start 2023-07-06T03:47:13.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.097+0000 7fd1f83ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1f006b360 0x7fd1f00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.097+0000 7fd1f83ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1f00d2210 0x7fd1f00d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.097+0000 7fd1f83ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f00cff90 con 0x7fd1f015c610 2023-07-06T03:47:13.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.097+0000 7fd1f83ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f00d0780 con 0x7fd1f00d2210 2023-07-06T03:47:13.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.097+0000 7fd1f83ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f015c9e0 con 0x7fd1f006b360 2023-07-06T03:47:13.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.098+0000 7fd1f699c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1f00d2210 0x7fd1f00d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.098+0000 7fd1f619b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd1f015c610 0x7fd1f00b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41808/0 (socket says 172.21.15.97:41808) 2023-07-06T03:47:13.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.098+0000 7fd1f699c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1f00d2210 0x7fd1f00d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34862/0 (socket says 172.21.15.97:34862) 2023-07-06T03:47:13.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.098+0000 7fd1f619b700 1 -- 172.21.15.97:0/2013071549 learned_addr learned my addr 172.21.15.97:0/2013071549 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:13.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.098+0000 7fd1f599a700 1 --2- 172.21.15.97:0/2013071549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1f006b360 0x7fd1f00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.098+0000 7fd1f699c700 1 -- 172.21.15.97:0/2013071549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1f006b360 msgr2=0x7fd1f00b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.098+0000 7fd1f699c700 1 --2- 172.21.15.97:0/2013071549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1f006b360 0x7fd1f00b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.098+0000 7fd1f699c700 1 -- 172.21.15.97:0/2013071549 >> v1:172.21.15.97:6789/0 conn(0x7fd1f015c610 legacy=0x7fd1f00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.098+0000 7fd1f699c700 1 -- 172.21.15.97:0/2013071549 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f00b5650 con 0x7fd1f00d2210 2023-07-06T03:47:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.098+0000 7fd1f5199700 1 -- 172.21.15.97:0/2013071549 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634043309 0 0) 0x7fd1f00cff90 con 0x7fd1f015c610 2023-07-06T03:47:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.098+0000 7fd1f699c700 1 --2- 172.21.15.97:0/2013071549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f00d2210 0x7fd1f00d1310 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009dc0 tx=0x7fd1e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a661563067407360 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.099+0000 7fd1f5199700 1 -- 172.21.15.97:0/2013071549 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800e070 con 0x7fd1f00d2210 2023-07-06T03:47:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.099+0000 7fd1f5199700 1 -- 172.21.15.97:0/2013071549 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e800bae0 con 0x7fd1f00d2210 2023-07-06T03:47:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.099+0000 7fd1f5199700 1 -- 172.21.15.97:0/2013071549 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800be90 con 0x7fd1f00d2210 2023-07-06T03:47:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.099+0000 7fd1f83ff700 1 -- 172.21.15.97:0/2013071549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f00d2210 msgr2=0x7fd1f00d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.099+0000 7fd1f83ff700 1 --2- 172.21.15.97:0/2013071549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f00d2210 0x7fd1f00d1310 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009dc0 tx=0x7fd1e800b330 comp rx=0 tx=0).stop 2023-07-06T03:47:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.099+0000 7fd1f83ff700 1 -- 172.21.15.97:0/2013071549 shutdown_connections 2023-07-06T03:47:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.099+0000 7fd1f83ff700 1 --2- 172.21.15.97:0/2013071549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f00d2210 0x7fd1f00d1310 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.099+0000 7fd1f83ff700 1 --2- 172.21.15.97:0/2013071549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1f006b360 0x7fd1f00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.099+0000 7fd1f83ff700 1 -- 172.21.15.97:0/2013071549 >> 172.21.15.97:0/2013071549 conn(0x7fd1f006a6c0 msgr2=0x7fd1f006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.099+0000 7fd1f83ff700 1 -- 172.21.15.97:0/2013071549 shutdown_connections 2023-07-06T03:47:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.099+0000 7fd1f83ff700 1 -- 172.21.15.97:0/2013071549 wait complete. 2023-07-06T03:47:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.102+0000 7fd1f83ff700 1 Processor -- start 2023-07-06T03:47:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.102+0000 7fd1f83ff700 1 -- start start 2023-07-06T03:47:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.103+0000 7fd1f83ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1f006b360 0x7fd1f015b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.103+0000 7fd1f83ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f00d2210 0x7fd1f01571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.103+0000 7fd1f83ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1f015c610 0x7fd1f01576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.103+0000 7fd1f83ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f015d380 con 0x7fd1f00d2210 2023-07-06T03:47:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.103+0000 7fd1f83ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00cac30 con 0x7fd1f006b360 2023-07-06T03:47:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.103+0000 7fd1f83ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f015ccc0 con 0x7fd1f015c610 2023-07-06T03:47:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.103+0000 7fd1f599a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f00d2210 0x7fd1f01571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.103+0000 7fd1f599a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f00d2210 0x7fd1f01571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34866/0 (socket says 172.21.15.97:34866) 2023-07-06T03:47:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.103+0000 7fd1f599a700 1 -- 172.21.15.97:0/3876906158 learned_addr learned my addr 172.21.15.97:0/3876906158 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1f699c700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1f015c610 0x7fd1f01576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1f619b700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1f006b360 0x7fd1f015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1f599a700 1 -- 172.21.15.97:0/3876906158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1f015c610 msgr2=0x7fd1f01576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1f599a700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1f015c610 0x7fd1f01576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1f599a700 1 -- 172.21.15.97:0/3876906158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1f006b360 msgr2=0x7fd1f015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1f599a700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1f006b360 0x7fd1f015b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1f599a700 1 -- 172.21.15.97:0/3876906158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e8009a70 con 0x7fd1f00d2210 2023-07-06T03:47:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1f699c700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1f015c610 0x7fd1f01576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1f599a700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f00d2210 0x7fd1f01571a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000e840 tx=0x7fd1e000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1e6ffd700 1 -- 172.21.15.97:0/3876906158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0009590 con 0x7fd1f00d2210 2023-07-06T03:47:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1e6ffd700 1 -- 172.21.15.97:0/3876906158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e0013680 con 0x7fd1f00d2210 2023-07-06T03:47:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1f83ff700 1 -- 172.21.15.97:0/3876906158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f015cfa0 con 0x7fd1f00d2210 2023-07-06T03:47:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1e6ffd700 1 -- 172.21.15.97:0/3876906158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0013a30 con 0x7fd1f00d2210 2023-07-06T03:47:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.104+0000 7fd1f83ff700 1 -- 172.21.15.97:0/3876906158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f015b860 con 0x7fd1f00d2210 2023-07-06T03:47:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.106+0000 7fd1f83ff700 1 -- 172.21.15.97:0/3876906158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f015cb20 con 0x7fd1f00d2210 2023-07-06T03:47:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.106+0000 7fd1e6ffd700 1 -- 172.21.15.97:0/3876906158 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd1e0013b90 con 0x7fd1f00d2210 2023-07-06T03:47:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.107+0000 7fd1e6ffd700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd1d406ad10 0x7fd1d406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.108+0000 7fd1e6ffd700 1 -- 172.21.15.97:0/3876906158 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fd1e00891b0 con 0x7fd1f00d2210 2023-07-06T03:47:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.108+0000 7fd1f619b700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd1d406ad10 0x7fd1d406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.108+0000 7fd1f619b700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd1d406ad10 0x7fd1d406d1c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec000f60 tx=0x7fd1ec007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.112+0000 7fd1e6ffd700 1 -- 172.21.15.97:0/3876906158 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd1f015cb20 con 0x7fd1f00d2210 2023-07-06T03:47:13.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.307+0000 7fd1f83ff700 1 -- 172.21.15.97:0/3876906158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1f015cb20 con 0x7fd1f00d2210 2023-07-06T03:47:13.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.308+0000 7fd1e6ffd700 1 -- 172.21.15.97:0/3876906158 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd1f015cb20 con 0x7fd1f00d2210 2023-07-06T03:47:13.311 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:47:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 -- 172.21.15.97:0/3876906158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd1d406ad10 msgr2=0x7fd1d406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd1d406ad10 0x7fd1d406d1c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec000f60 tx=0x7fd1ec007be0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 -- 172.21.15.97:0/3876906158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f00d2210 msgr2=0x7fd1f01571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f00d2210 0x7fd1f01571a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000e840 tx=0x7fd1e000ec00 comp rx=0 tx=0).stop 2023-07-06T03:47:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 -- 172.21.15.97:0/3876906158 shutdown_connections 2023-07-06T03:47:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd1d406ad10 0x7fd1d406d1c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd1f015c610 0x7fd1f01576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1f006b360 0x7fd1f015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 --2- 172.21.15.97:0/3876906158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f00d2210 0x7fd1f01571a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 -- 172.21.15.97:0/3876906158 >> 172.21.15.97:0/3876906158 conn(0x7fd1f006a6c0 msgr2=0x7fd1f00b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 -- 172.21.15.97:0/3876906158 shutdown_connections 2023-07-06T03:47:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.311+0000 7fd1f83ff700 1 -- 172.21.15.97:0/3876906158 wait complete. 2023-07-06T03:47:13.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:47:13.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.524+0000 7fe7d5bff700 1 Processor -- start 2023-07-06T03:47:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.525+0000 7fe7d5bff700 1 -- start start 2023-07-06T03:47:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.526+0000 7fe7d5bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe7d0159070 0x7fe7d00c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.526+0000 7fe7d5bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7d00c3cd0 0x7fe7d00cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.526+0000 7fe7d5bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d00ca260 con 0x7fe7d0159440 2023-07-06T03:47:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.526+0000 7fe7d5bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d00c99c0 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.526+0000 7fe7d5bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d00c9ca0 con 0x7fe7d0159070 2023-07-06T03:47:13.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.526+0000 7fe7c7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7d00c3cd0 0x7fe7d00cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.526+0000 7fe7cffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe7d0159440 0x7fe7d00b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41814/0 (socket says 172.21.15.97:41814) 2023-07-06T03:47:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.526+0000 7fe7cffff700 1 -- 172.21.15.97:0/2731846213 learned_addr learned my addr 172.21.15.97:0/2731846213 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.526+0000 7fe7cf7fe700 1 --2- 172.21.15.97:0/2731846213 >> v2:172.21.15.146:3300/0 conn(0x7fe7d0159070 0x7fe7d00c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.527+0000 7fe7ceffd700 1 -- 172.21.15.97:0/2731846213 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643366867 0 0) 0x7fe7d00ca260 con 0x7fe7d0159440 2023-07-06T03:47:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.527+0000 7fe7ceffd700 1 -- 172.21.15.97:0/2731846213 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7bc0035b0 con 0x7fe7d0159440 2023-07-06T03:47:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.527+0000 7fe7c7fff700 1 -- 172.21.15.97:0/2731846213 >> v2:172.21.15.146:3300/0 conn(0x7fe7d0159070 msgr2=0x7fe7d00c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.527+0000 7fe7c7fff700 1 --2- 172.21.15.97:0/2731846213 >> v2:172.21.15.146:3300/0 conn(0x7fe7d0159070 0x7fe7d00c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.527+0000 7fe7c7fff700 1 -- 172.21.15.97:0/2731846213 >> v1:172.21.15.97:6789/0 conn(0x7fe7d0159440 legacy=0x7fe7d00b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.527+0000 7fe7c7fff700 1 -- 172.21.15.97:0/2731846213 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d00c9f80 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.527+0000 7fe7c7fff700 1 --2- 172.21.15.97:0/2731846213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d00c3cd0 0x7fe7d00cc4c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe7d006f8e0 tx=0x7fe7b8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157b73b261e420c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.527+0000 7fe7ceffd700 1 -- 172.21.15.97:0/2731846213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8011830 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.527+0000 7fe7ceffd700 1 -- 172.21.15.97:0/2731846213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8011990 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.527+0000 7fe7ceffd700 1 -- 172.21.15.97:0/2731846213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8011d20 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.528+0000 7fe7d5bff700 1 -- 172.21.15.97:0/2731846213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d00c3cd0 msgr2=0x7fe7d00cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.528+0000 7fe7d5bff700 1 --2- 172.21.15.97:0/2731846213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d00c3cd0 0x7fe7d00cc4c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe7d006f8e0 tx=0x7fe7b8009b20 comp rx=0 tx=0).stop 2023-07-06T03:47:13.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.528+0000 7fe7d5bff700 1 -- 172.21.15.97:0/2731846213 shutdown_connections 2023-07-06T03:47:13.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.528+0000 7fe7d5bff700 1 --2- 172.21.15.97:0/2731846213 >> v2:172.21.15.146:3300/0 conn(0x7fe7d0159070 0x7fe7d00c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.528+0000 7fe7d5bff700 1 --2- 172.21.15.97:0/2731846213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d00c3cd0 0x7fe7d00cc4c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.528+0000 7fe7d5bff700 1 -- 172.21.15.97:0/2731846213 >> 172.21.15.97:0/2731846213 conn(0x7fe7d006a060 msgr2=0x7fe7d006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:13.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.528+0000 7fe7d5bff700 1 -- 172.21.15.97:0/2731846213 shutdown_connections 2023-07-06T03:47:13.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.528+0000 7fe7d5bff700 1 -- 172.21.15.97:0/2731846213 wait complete. 2023-07-06T03:47:13.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.530+0000 7fe7d5bff700 1 Processor -- start 2023-07-06T03:47:13.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.530+0000 7fe7d5bff700 1 -- start start 2023-07-06T03:47:13.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7d5bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d00c3cd0 0x7fe7d014dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7d5bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7d0159070 0x7fe7d00b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7d5bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7d0159440 0x7fe7d00b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7d5bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d00cbbb0 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7d5bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d014e400 con 0x7fe7d0159070 2023-07-06T03:47:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7d5bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d00cafc0 con 0x7fe7d0159440 2023-07-06T03:47:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d00c3cd0 0x7fe7d014dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d00c3cd0 0x7fe7d014dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34880/0 (socket says 172.21.15.97:34880) 2023-07-06T03:47:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7cf7fe700 1 -- 172.21.15.97:0/3017414571 learned_addr learned my addr 172.21.15.97:0/3017414571 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7c7fff700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7d0159070 0x7fe7d00b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7cffff700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7d0159440 0x7fe7d00b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7cf7fe700 1 -- 172.21.15.97:0/3017414571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7d0159440 msgr2=0x7fe7d00b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7cf7fe700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7d0159440 0x7fe7d00b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7cf7fe700 1 -- 172.21.15.97:0/3017414571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7d0159070 msgr2=0x7fe7d00b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7cf7fe700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7d0159070 0x7fe7d00b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7cf7fe700 1 -- 172.21.15.97:0/3017414571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b80094a0 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.531+0000 7fe7c7fff700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7d0159070 0x7fe7d00b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:13.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.532+0000 7fe7cf7fe700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d00c3cd0 0x7fe7d014dec0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000e9e0 tx=0x7fe7c000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:13.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.532+0000 7fe7ccff9700 1 -- 172.21.15.97:0/3017414571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0009710 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.532+0000 7fe7ccff9700 1 -- 172.21.15.97:0/3017414571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c000c850 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.532+0000 7fe7ccff9700 1 -- 172.21.15.97:0/3017414571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c000cbc0 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.532+0000 7fe7d5bff700 1 -- 172.21.15.97:0/3017414571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d00cb210 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.532+0000 7fe7d5bff700 1 -- 172.21.15.97:0/3017414571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d00cb7d0 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.533+0000 7fe7d5bff700 1 -- 172.21.15.97:0/3017414571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d00cbcf0 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.534+0000 7fe7ccff9700 1 -- 172.21.15.97:0/3017414571 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe7c000cd20 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.535+0000 7fe7ccff9700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe7b006ace0 0x7fe7b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.535+0000 7fe7ccff9700 1 -- 172.21.15.97:0/3017414571 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fe7c0089600 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.535+0000 7fe7c7fff700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe7b006ace0 0x7fe7b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.536+0000 7fe7c7fff700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe7b006ace0 0x7fe7b006d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00c40d0 tx=0x7fe7b8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:13.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.540+0000 7fe7ccff9700 1 -- 172.21.15.97:0/3017414571 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe7c00580d0 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.735+0000 7fe7d5bff700 1 -- 172.21.15.97:0/3017414571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7d004e340 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.735+0000 7fe7ccff9700 1 -- 172.21.15.97:0/3017414571 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7c0057c60 con 0x7fe7d00c3cd0 2023-07-06T03:47:13.739 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:47:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.737+0000 7fe7d5bff700 1 -- 172.21.15.97:0/3017414571 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe7b006ace0 msgr2=0x7fe7b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.737+0000 7fe7d5bff700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe7b006ace0 0x7fe7b006d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00c40d0 tx=0x7fe7b8008030 comp rx=0 tx=0).stop 2023-07-06T03:47:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.737+0000 7fe7d5bff700 1 -- 172.21.15.97:0/3017414571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d00c3cd0 msgr2=0x7fe7d014dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.737+0000 7fe7d5bff700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d00c3cd0 0x7fe7d014dec0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000e9e0 tx=0x7fe7c000eda0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.738+0000 7fe7d5bff700 1 -- 172.21.15.97:0/3017414571 shutdown_connections 2023-07-06T03:47:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.738+0000 7fe7d5bff700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe7b006ace0 0x7fe7b006d190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.738+0000 7fe7d5bff700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7d0159440 0x7fe7d00b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.738+0000 7fe7d5bff700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d00c3cd0 0x7fe7d014dec0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.738+0000 7fe7d5bff700 1 --2- 172.21.15.97:0/3017414571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7d0159070 0x7fe7d00b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.738+0000 7fe7d5bff700 1 -- 172.21.15.97:0/3017414571 >> 172.21.15.97:0/3017414571 conn(0x7fe7d006a060 msgr2=0x7fe7d01578a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.738+0000 7fe7d5bff700 1 -- 172.21.15.97:0/3017414571 shutdown_connections 2023-07-06T03:47:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.738+0000 7fe7d5bff700 1 -- 172.21.15.97:0/3017414571 wait complete. 2023-07-06T03:47:13.759 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:47:13.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.953+0000 7fdd615ff700 1 Processor -- start 2023-07-06T03:47:13.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.953+0000 7fdd615ff700 1 -- start start 2023-07-06T03:47:13.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.954+0000 7fdd615ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd5c15c1c0 0x7fdd5c0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.954+0000 7fdd615ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdd5c0719c0 0x7fdd5c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.954+0000 7fdd615ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5c15da10 con 0x7fdd5c15c590 2023-07-06T03:47:13.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.954+0000 7fdd615ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c0ccc30 con 0x7fdd5c15c1c0 2023-07-06T03:47:13.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.954+0000 7fdd615ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c0ccf10 con 0x7fdd5c0719c0 2023-07-06T03:47:13.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.954+0000 7fdd5affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd5c15c1c0 0x7fdd5c0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.955+0000 7fdd5affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd5c15c1c0 0x7fdd5c0cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34892/0 (socket says 172.21.15.97:34892) 2023-07-06T03:47:13.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.955+0000 7fdd5affd700 1 -- 172.21.15.97:0/4157448617 learned_addr learned my addr 172.21.15.97:0/4157448617 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:13.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.955+0000 7fdd5b7fe700 1 --2- 172.21.15.97:0/4157448617 >> v2:172.21.15.146:3300/0 conn(0x7fdd5c0719c0 0x7fdd5c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.955+0000 7fdd5affd700 1 -- 172.21.15.97:0/4157448617 >> v2:172.21.15.146:3300/0 conn(0x7fdd5c0719c0 msgr2=0x7fdd5c06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.955+0000 7fdd5affd700 1 --2- 172.21.15.97:0/4157448617 >> v2:172.21.15.146:3300/0 conn(0x7fdd5c0719c0 0x7fdd5c06a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.955+0000 7fdd5affd700 1 -- 172.21.15.97:0/4157448617 >> v1:172.21.15.146:6789/0 conn(0x7fdd5c15c590 legacy=0x7fdd5c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.955+0000 7fdd5affd700 1 -- 172.21.15.97:0/4157448617 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0cd1f0 con 0x7fdd5c15c1c0 2023-07-06T03:47:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.955+0000 7fdd5b7fe700 1 --2- 172.21.15.97:0/4157448617 >> v2:172.21.15.146:3300/0 conn(0x7fdd5c0719c0 0x7fdd5c06a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:13.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.955+0000 7fdd5affd700 1 --2- 172.21.15.97:0/4157448617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5c15c1c0 0x7fdd5c0cc3e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0b26c0 tx=0x7fdd4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f1ea5feedff795 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:13.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.955+0000 7fdd59ffb700 1 -- 172.21.15.97:0/4157448617 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00b870 con 0x7fdd5c15c1c0 2023-07-06T03:47:13.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.956+0000 7fdd59ffb700 1 -- 172.21.15.97:0/4157448617 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c00b9d0 con 0x7fdd5c15c1c0 2023-07-06T03:47:13.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.956+0000 7fdd59ffb700 1 -- 172.21.15.97:0/4157448617 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00bd60 con 0x7fdd5c15c1c0 2023-07-06T03:47:13.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.956+0000 7fdd615ff700 1 -- 172.21.15.97:0/4157448617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5c15c1c0 msgr2=0x7fdd5c0cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.956+0000 7fdd615ff700 1 --2- 172.21.15.97:0/4157448617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5c15c1c0 0x7fdd5c0cc3e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0b26c0 tx=0x7fdd4c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.956+0000 7fdd615ff700 1 -- 172.21.15.97:0/4157448617 shutdown_connections 2023-07-06T03:47:13.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.956+0000 7fdd615ff700 1 --2- 172.21.15.97:0/4157448617 >> v2:172.21.15.146:3300/0 conn(0x7fdd5c0719c0 0x7fdd5c06a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.956+0000 7fdd615ff700 1 --2- 172.21.15.97:0/4157448617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5c15c1c0 0x7fdd5c0cc3e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.956+0000 7fdd615ff700 1 -- 172.21.15.97:0/4157448617 >> 172.21.15.97:0/4157448617 conn(0x7fdd5c0b1650 msgr2=0x7fdd5c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:13.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.956+0000 7fdd615ff700 1 -- 172.21.15.97:0/4157448617 shutdown_connections 2023-07-06T03:47:13.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.956+0000 7fdd615ff700 1 -- 172.21.15.97:0/4157448617 wait complete. 2023-07-06T03:47:13.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.958+0000 7fdd615ff700 1 Processor -- start 2023-07-06T03:47:13.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.958+0000 7fdd615ff700 1 -- start start 2023-07-06T03:47:13.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd615ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5c0719c0 0x7fdd5c1f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd615ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd5c15c1c0 0x7fdd5c1f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd615ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd5c15c590 0x7fdd5c1f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd615ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1f5aa0 con 0x7fdd5c0719c0 2023-07-06T03:47:13.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd615ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1f5d80 con 0x7fdd5c15c1c0 2023-07-06T03:47:13.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd615ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1f6060 con 0x7fdd5c15c590 2023-07-06T03:47:13.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd5affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5c0719c0 0x7fdd5c1f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd5affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5c0719c0 0x7fdd5c1f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34900/0 (socket says 172.21.15.97:34900) 2023-07-06T03:47:13.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd5affd700 1 -- 172.21.15.97:0/1005759020 learned_addr learned my addr 172.21.15.97:0/1005759020 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:13.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd5affd700 1 -- 172.21.15.97:0/1005759020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd5c15c590 msgr2=0x7fdd5c1f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd5affd700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd5c15c590 0x7fdd5c1f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd5affd700 1 -- 172.21.15.97:0/1005759020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd5c15c1c0 msgr2=0x7fdd5c1f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:13.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd5affd700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd5c15c1c0 0x7fdd5c1f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:13.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.959+0000 7fdd5affd700 1 -- 172.21.15.97:0/1005759020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c009550 con 0x7fdd5c0719c0 2023-07-06T03:47:13.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.960+0000 7fdd5affd700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5c0719c0 0x7fdd5c1f1600 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c000c00 tx=0x7fdd4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:13.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.960+0000 7fdd437fe700 1 -- 172.21.15.97:0/1005759020 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004540 con 0x7fdd5c0719c0 2023-07-06T03:47:13.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.960+0000 7fdd437fe700 1 -- 172.21.15.97:0/1005759020 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c0036a0 con 0x7fdd5c0719c0 2023-07-06T03:47:13.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.960+0000 7fdd615ff700 1 -- 172.21.15.97:0/1005759020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c1f1ce0 con 0x7fdd5c0719c0 2023-07-06T03:47:13.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.960+0000 7fdd437fe700 1 -- 172.21.15.97:0/1005759020 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c003a30 con 0x7fdd5c0719c0 2023-07-06T03:47:13.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.960+0000 7fdd615ff700 1 -- 172.21.15.97:0/1005759020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c1f5700 con 0x7fdd5c0719c0 2023-07-06T03:47:13.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.961+0000 7fdd615ff700 1 -- 172.21.15.97:0/1005759020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c1f1b40 con 0x7fdd5c0719c0 2023-07-06T03:47:13.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.962+0000 7fdd437fe700 1 -- 172.21.15.97:0/1005759020 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd4c021070 con 0x7fdd5c0719c0 2023-07-06T03:47:13.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.963+0000 7fdd437fe700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4806ace0 0x7fdd4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:13.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.964+0000 7fdd437fe700 1 -- 172.21.15.97:0/1005759020 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fdd4c08acb0 con 0x7fdd5c0719c0 2023-07-06T03:47:13.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.964+0000 7fdd5a7fc700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4806ace0 0x7fdd4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:13.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.964+0000 7fdd5a7fc700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4806ace0 0x7fdd4806d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000f60 tx=0x7fdd50007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:13.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:13.968+0000 7fdd437fe700 1 -- 172.21.15.97:0/1005759020 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdd5c1f1b40 con 0x7fdd5c0719c0 2023-07-06T03:47:14.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.178+0000 7fdd615ff700 1 -- 172.21.15.97:0/1005759020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdd5c1f1b40 con 0x7fdd5c0719c0 2023-07-06T03:47:14.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.179+0000 7fdd437fe700 1 -- 172.21.15.97:0/1005759020 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdd5c1f1b40 con 0x7fdd5c0719c0 2023-07-06T03:47:14.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 -- 172.21.15.97:0/1005759020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4806ace0 msgr2=0x7fdd4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:14.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4806ace0 0x7fdd4806d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000f60 tx=0x7fdd50007a40 comp rx=0 tx=0).stop 2023-07-06T03:47:14.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 -- 172.21.15.97:0/1005759020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5c0719c0 msgr2=0x7fdd5c1f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:14.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5c0719c0 0x7fdd5c1f1600 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c000c00 tx=0x7fdd4c004750 comp rx=0 tx=0).stop 2023-07-06T03:47:14.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 -- 172.21.15.97:0/1005759020 shutdown_connections 2023-07-06T03:47:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4806ace0 0x7fdd4806d190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd5c15c590 0x7fdd5c1f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5c0719c0 0x7fdd5c1f1600 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 --2- 172.21.15.97:0/1005759020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd5c15c1c0 0x7fdd5c1f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 -- 172.21.15.97:0/1005759020 >> 172.21.15.97:0/1005759020 conn(0x7fdd5c0b1650 msgr2=0x7fdd5c0af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 -- 172.21.15.97:0/1005759020 shutdown_connections 2023-07-06T03:47:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.181+0000 7fdd615ff700 1 -- 172.21.15.97:0/1005759020 wait complete. 2023-07-06T03:47:14.197 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:47:14.200 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:47:14.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:14.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:47:14.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[129554]: fuse finished with error 0 and tester_r 0 2023-07-06T03:47:14.271 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:47:14.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:14.272 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:47:14.469 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:47:14.470 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:47:14.520 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:47:14.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.712+0000 7f2ffdbff700 1 Processor -- start 2023-07-06T03:47:14.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.712+0000 7f2ffdbff700 1 -- start start 2023-07-06T03:47:14.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.713+0000 7f2ffdbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ff80b8410 0x7f2ff8167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ffdbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff80cdae0 con 0x7f2ff815c520 2023-07-06T03:47:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ffdbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff80b6640 con 0x7f2ff80cd390 2023-07-06T03:47:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ffdbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff80b6920 con 0x7f2ff80b8410 2023-07-06T03:47:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ff77fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2ff815c520 0x7f2ff8160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41830/0 (socket says 172.21.15.97:41830) 2023-07-06T03:47:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ff77fe700 1 -- 172.21.15.97:0/2583982620 learned_addr learned my addr 172.21.15.97:0/2583982620 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:14.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ff7fff700 1 --2- 172.21.15.97:0/2583982620 >> v2:172.21.15.146:3300/0 conn(0x7f2ff80b8410 0x7f2ff8167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:14.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ff67fc700 1 -- 172.21.15.97:0/2583982620 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085718159 0 0) 0x7f2ff80cdae0 con 0x7f2ff815c520 2023-07-06T03:47:14.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ff67fc700 1 -- 172.21.15.97:0/2583982620 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe00035b0 con 0x7f2ff815c520 2023-07-06T03:47:14.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ff67fc700 1 -- 172.21.15.97:0/2583982620 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223881846 0 0) 0x7f2ff80b6640 con 0x7f2ff80cd390 2023-07-06T03:47:14.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ff67fc700 1 -- 172.21.15.97:0/2583982620 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff80cdae0 con 0x7f2ff80cd390 2023-07-06T03:47:14.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ff7fff700 1 -- 172.21.15.97:0/2583982620 >> v1:172.21.15.146:6789/0 conn(0x7f2ff80cd390 legacy=0x7f2ff8160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:14.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ff7fff700 1 -- 172.21.15.97:0/2583982620 >> v1:172.21.15.97:6789/0 conn(0x7f2ff815c520 legacy=0x7f2ff8160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:14.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.714+0000 7f2ff7fff700 1 -- 172.21.15.97:0/2583982620 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff80b6c00 con 0x7f2ff80b8410 2023-07-06T03:47:14.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.715+0000 7f2ff7fff700 1 --2- 172.21.15.97:0/2583982620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff80b8410 0x7f2ff8167510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009d90 tx=0x7f2ff000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58fa4f01ceca3e72 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:14.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.715+0000 7f2ff67fc700 1 -- 172.21.15.97:0/2583982620 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff000e070 con 0x7f2ff80b8410 2023-07-06T03:47:14.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.715+0000 7f2ff67fc700 1 -- 172.21.15.97:0/2583982620 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff000bb20 con 0x7f2ff80b8410 2023-07-06T03:47:14.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.715+0000 7f2ff67fc700 1 -- 172.21.15.97:0/2583982620 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0012430 con 0x7f2ff80b8410 2023-07-06T03:47:14.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.715+0000 7f2ffdbff700 1 -- 172.21.15.97:0/2583982620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff80b8410 msgr2=0x7f2ff8167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:14.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.716+0000 7f2ffdbff700 1 --2- 172.21.15.97:0/2583982620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff80b8410 0x7f2ff8167510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009d90 tx=0x7f2ff000b330 comp rx=0 tx=0).stop 2023-07-06T03:47:14.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.716+0000 7f2ffdbff700 1 -- 172.21.15.97:0/2583982620 shutdown_connections 2023-07-06T03:47:14.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.716+0000 7f2ffdbff700 1 --2- 172.21.15.97:0/2583982620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff80b8410 0x7f2ff8167510 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:14.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.716+0000 7f2ffdbff700 1 -- 172.21.15.97:0/2583982620 >> 172.21.15.97:0/2583982620 conn(0x7f2ff8071790 msgr2=0x7f2ff8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:14.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.716+0000 7f2ffdbff700 1 -- 172.21.15.97:0/2583982620 shutdown_connections 2023-07-06T03:47:14.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.716+0000 7f2ffdbff700 1 -- 172.21.15.97:0/2583982620 wait complete. 2023-07-06T03:47:14.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.717+0000 7f2ffdbff700 1 Processor -- start 2023-07-06T03:47:14.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.718+0000 7f2ffdbff700 1 -- start start 2023-07-06T03:47:14.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.718+0000 7f2ffdbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80b8410 0x7f2ff814c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:14.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.718+0000 7f2ffdbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff80cd390 0x7f2ff81539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:14.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.718+0000 7f2ffdbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff815c520 0x7f2ff80d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:14.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.718+0000 7f2ffdbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff814cdd0 con 0x7f2ff80b8410 2023-07-06T03:47:14.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.718+0000 7f2ffdbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff815e4f0 con 0x7f2ff80cd390 2023-07-06T03:47:14.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.718+0000 7f2ffdbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff815e7d0 con 0x7f2ff815c520 2023-07-06T03:47:14.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2ff77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80b8410 0x7f2ff814c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:14.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2ff77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80b8410 0x7f2ff814c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34916/0 (socket says 172.21.15.97:34916) 2023-07-06T03:47:14.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2ff77fe700 1 -- 172.21.15.97:0/1856822049 learned_addr learned my addr 172.21.15.97:0/1856822049 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:14.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2ff7fff700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff815c520 0x7f2ff80d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:14.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2ff6ffd700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff80cd390 0x7f2ff81539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:14.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2ff77fe700 1 -- 172.21.15.97:0/1856822049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff815c520 msgr2=0x7f2ff80d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:14.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2ff77fe700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff815c520 0x7f2ff80d30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:14.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2ff77fe700 1 -- 172.21.15.97:0/1856822049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff80cd390 msgr2=0x7f2ff81539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:14.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2ff77fe700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff80cd390 0x7f2ff81539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:14.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2ff77fe700 1 -- 172.21.15.97:0/1856822049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0009a70 con 0x7f2ff80b8410 2023-07-06T03:47:14.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2ff77fe700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80b8410 0x7f2ff814c890 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8001eb0 tx=0x7f2fe800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.719+0000 7f2fdffff700 1 -- 172.21.15.97:0/1856822049 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8005d70 con 0x7f2ff80b8410 2023-07-06T03:47:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.720+0000 7f2fdffff700 1 -- 172.21.15.97:0/1856822049 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe80070d0 con 0x7f2ff80b8410 2023-07-06T03:47:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.720+0000 7f2ffdbff700 1 -- 172.21.15.97:0/1856822049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff815eab0 con 0x7f2ff80b8410 2023-07-06T03:47:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.720+0000 7f2fdffff700 1 -- 172.21.15.97:0/1856822049 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8007460 con 0x7f2ff80b8410 2023-07-06T03:47:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.720+0000 7f2ffdbff700 1 -- 172.21.15.97:0/1856822049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff815f070 con 0x7f2ff80b8410 2023-07-06T03:47:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.721+0000 7f2ffdbff700 1 -- 172.21.15.97:0/1856822049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff804e340 con 0x7f2ff80b8410 2023-07-06T03:47:14.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.722+0000 7f2fdffff700 1 -- 172.21.15.97:0/1856822049 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2fe80075c0 con 0x7f2ff80b8410 2023-07-06T03:47:14.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.723+0000 7f2fdffff700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fe406ace0 0x7f2fe406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:14.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.723+0000 7f2fdffff700 1 -- 172.21.15.97:0/1856822049 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f2fe8088a90 con 0x7f2ff80b8410 2023-07-06T03:47:14.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.723+0000 7f2ff6ffd700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fe406ace0 0x7f2fe406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:14.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.724+0000 7f2ff6ffd700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fe406ace0 0x7f2fe406d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2fec007fd0 tx=0x7f2fec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:14.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.727+0000 7f2fdffff700 1 -- 172.21.15.97:0/1856822049 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2fe8053ad0 con 0x7f2ff80b8410 2023-07-06T03:47:14.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.926+0000 7f2ffdbff700 1 -- 172.21.15.97:0/1856822049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ff814cf10 con 0x7f2ff80b8410 2023-07-06T03:47:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.928+0000 7f2fdffff700 1 -- 172.21.15.97:0/1856822049 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18460 (secure 0 0 0) 0x7f2fe80570f0 con 0x7f2ff80b8410 2023-07-06T03:47:14.930 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:14.930 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:46:52.661343+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:46:35.584013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T03:46:36.303248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.28","mappings":[{"from":4,"to":7}]},{"pgid":"6.36","mappings":[{"from":1,"to":2}]},{"pgid":"6.38","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:47:14.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 -- 172.21.15.97:0/1856822049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fe406ace0 msgr2=0x7f2fe406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:14.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fe406ace0 0x7f2fe406d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2fec007fd0 tx=0x7f2fec009040 comp rx=0 tx=0).stop 2023-07-06T03:47:14.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 -- 172.21.15.97:0/1856822049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80b8410 msgr2=0x7f2ff814c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:14.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80b8410 0x7f2ff814c890 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8001eb0 tx=0x7f2fe800c860 comp rx=0 tx=0).stop 2023-07-06T03:47:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 -- 172.21.15.97:0/1856822049 shutdown_connections 2023-07-06T03:47:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fe406ace0 0x7f2fe406d190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff815c520 0x7f2ff80d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80b8410 0x7f2ff814c890 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 --2- 172.21.15.97:0/1856822049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff80cd390 0x7f2ff81539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 -- 172.21.15.97:0/1856822049 >> 172.21.15.97:0/1856822049 conn(0x7f2ff8071790 msgr2=0x7f2ff80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 -- 172.21.15.97:0/1856822049 shutdown_connections 2023-07-06T03:47:14.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:14.930+0000 7f2ffdbff700 1 -- 172.21.15.97:0/1856822049 wait complete. 2023-07-06T03:47:14.951 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:46:35.584013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:46:36.303248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:47:14.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:47:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.145+0000 7f2cd89ff700 1 Processor -- start 2023-07-06T03:47:15.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.146+0000 7f2cd89ff700 1 -- start start 2023-07-06T03:47:15.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.147+0000 7f2cd89ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2cd415c610 0x7f2cd40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.147+0000 7f2cd89ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd40cff90 con 0x7f2cd406b360 2023-07-06T03:47:15.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.147+0000 7f2cd89ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd40b5650 con 0x7f2cd40d2210 2023-07-06T03:47:15.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.147+0000 7f2cd89ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd40d0780 con 0x7f2cd415c610 2023-07-06T03:47:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.147+0000 7f2ccbfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2cd406b360 0x7f2cd40b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41842/0 (socket says 172.21.15.97:41842) 2023-07-06T03:47:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.147+0000 7f2ccbfff700 1 -- 172.21.15.97:0/3176558823 learned_addr learned my addr 172.21.15.97:0/3176558823 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.147+0000 7f2cd259c700 1 --2- 172.21.15.97:0/3176558823 >> v2:172.21.15.146:3300/0 conn(0x7f2cd415c610 0x7f2cd40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.147+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654270884 0 0) 0x7f2cd40cff90 con 0x7f2cd406b360 2023-07-06T03:47:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cc00035b0 con 0x7f2cd406b360 2023-07-06T03:47:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1053396612 0 0) 0x7f2cc00035b0 con 0x7f2cd406b360 2023-07-06T03:47:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 >> v2:172.21.15.146:3300/0 conn(0x7f2cd415c610 msgr2=0x7f2cd40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 --2- 172.21.15.97:0/3176558823 >> v2:172.21.15.146:3300/0 conn(0x7f2cd415c610 0x7f2cd40b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 >> v1:172.21.15.146:6789/0 conn(0x7f2cd40d2210 legacy=0x7f2cd40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40b5400 con 0x7f2cd406b360 2023-07-06T03:47:15.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f2cbc001670 con 0x7f2cd406b360 2023-07-06T03:47:15.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd259c700 1 --2- 172.21.15.97:0/3176558823 >> v2:172.21.15.146:3300/0 conn(0x7f2cd415c610 0x7f2cd40b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:47:15.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 >> v1:172.21.15.97:6789/0 conn(0x7f2cd406b360 legacy=0x7f2cd40b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc0003a00 0x7f2cc0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cc0004850 0x7f2cc0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cc0005200 0x7f2cc000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd40d0780 con 0x7f2cc0003a00 2023-07-06T03:47:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc000c080 con 0x7f2cc0004850 2023-07-06T03:47:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc000c240 con 0x7f2cc0005200 2023-07-06T03:47:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2cd259c700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc0003a00 0x7f2cc0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.148+0000 7f2ccbfff700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cc0004850 0x7f2cc0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2cd2d9d700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cc0005200 0x7f2cc000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2cd259c700 1 -- 172.21.15.97:0/3176558823 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cc0005200 msgr2=0x7f2cc000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:15.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2cd259c700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cc0005200 0x7f2cc000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2cd259c700 1 -- 172.21.15.97:0/3176558823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cc0004850 msgr2=0x7f2cc0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:15.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2cd259c700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cc0004850 0x7f2cc0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2cd259c700 1 -- 172.21.15.97:0/3176558823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc00035b0 con 0x7f2cc0003a00 2023-07-06T03:47:15.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2ccbfff700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cc0004850 0x7f2cc0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:15.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2cd2d9d700 1 -- 172.21.15.97:0/3176558823 reap_dead start 2023-07-06T03:47:15.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2cd259c700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc0003a00 0x7f2cc0003e10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009d30 tx=0x7f2cc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a870ad0859397b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:15.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4009e80 con 0x7f2cc0003a00 2023-07-06T03:47:15.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc400e9a0 con 0x7f2cc0003a00 2023-07-06T03:47:15.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.149+0000 7f2cd1d9b700 1 -- 172.21.15.97:0/3176558823 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400ed10 con 0x7f2cc0003a00 2023-07-06T03:47:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.150+0000 7f2cd89ff700 1 -- 172.21.15.97:0/3176558823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc0003a00 msgr2=0x7f2cc0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.150+0000 7f2cd89ff700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc0003a00 0x7f2cc0003e10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009d30 tx=0x7f2cc0005880 comp rx=0 tx=0).stop 2023-07-06T03:47:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.150+0000 7f2cd89ff700 1 -- 172.21.15.97:0/3176558823 shutdown_connections 2023-07-06T03:47:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.150+0000 7f2cd89ff700 1 --2- 172.21.15.97:0/3176558823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cc0003a00 0x7f2cc0003e10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.150+0000 7f2cd89ff700 1 -- 172.21.15.97:0/3176558823 >> 172.21.15.97:0/3176558823 conn(0x7f2cd406a6c0 msgr2=0x7f2cd406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.150+0000 7f2cd89ff700 1 -- 172.21.15.97:0/3176558823 shutdown_connections 2023-07-06T03:47:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.150+0000 7f2cd89ff700 1 -- 172.21.15.97:0/3176558823 wait complete. 2023-07-06T03:47:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.151+0000 7f2cd89ff700 1 Processor -- start 2023-07-06T03:47:15.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.152+0000 7f2cd89ff700 1 -- start start 2023-07-06T03:47:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.152+0000 7f2cd89ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cc0003a00 0x7f2cd414edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2cd89ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cd414f440 0x7f2cd40c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2cd89ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd40b5b60 0x7f2cd40b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2cd89ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd40b6540 con 0x7f2cd414f440 2023-07-06T03:47:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2cd89ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd40c6da0 con 0x7f2cd40b5b60 2023-07-06T03:47:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2cd89ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd415f500 con 0x7f2cc0003a00 2023-07-06T03:47:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2ccbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cd414f440 0x7f2cd40c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2ccbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cd414f440 0x7f2cd40c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34936/0 (socket says 172.21.15.97:34936) 2023-07-06T03:47:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2ccbfff700 1 -- 172.21.15.97:0/1267682567 learned_addr learned my addr 172.21.15.97:0/1267682567 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2cd259c700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cc0003a00 0x7f2cd414edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2cd2d9d700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd40b5b60 0x7f2cd40b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2ccbfff700 1 -- 172.21.15.97:0/1267682567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cc0003a00 msgr2=0x7f2cd414edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2ccbfff700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cc0003a00 0x7f2cd414edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2ccbfff700 1 -- 172.21.15.97:0/1267682567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd40b5b60 msgr2=0x7f2cd40b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2ccbfff700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd40b5b60 0x7f2cd40b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2ccbfff700 1 -- 172.21.15.97:0/1267682567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40cafc0 con 0x7f2cd414f440 2023-07-06T03:47:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.153+0000 7f2cd2d9d700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd40b5b60 0x7f2cd40b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.154+0000 7f2ccbfff700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cd414f440 0x7f2cd40c6860 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00e760 tx=0x7f2cbc00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.154+0000 7f2ccaffd700 1 -- 172.21.15.97:0/1267682567 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc012590 con 0x7f2cd414f440 2023-07-06T03:47:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.154+0000 7f2ccaffd700 1 -- 172.21.15.97:0/1267682567 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc0126f0 con 0x7f2cd414f440 2023-07-06T03:47:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.154+0000 7f2cd89ff700 1 -- 172.21.15.97:0/1267682567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd40cb2a0 con 0x7f2cd414f440 2023-07-06T03:47:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.154+0000 7f2ccaffd700 1 -- 172.21.15.97:0/1267682567 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc012a60 con 0x7f2cd414f440 2023-07-06T03:47:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.154+0000 7f2cd89ff700 1 -- 172.21.15.97:0/1267682567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd40cb860 con 0x7f2cd414f440 2023-07-06T03:47:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.155+0000 7f2cd89ff700 1 -- 172.21.15.97:0/1267682567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd415c9e0 con 0x7f2cd414f440 2023-07-06T03:47:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.156+0000 7f2ccaffd700 1 -- 172.21.15.97:0/1267682567 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2cbc012bc0 con 0x7f2cd414f440 2023-07-06T03:47:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.157+0000 7f2ccaffd700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cb406ace0 0x7f2cb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.157+0000 7f2ccaffd700 1 -- 172.21.15.97:0/1267682567 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f2cbc089250 con 0x7f2cd414f440 2023-07-06T03:47:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.157+0000 7f2cd259c700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cb406ace0 0x7f2cb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.157+0000 7f2cd259c700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cb406ace0 0x7f2cb406d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4000c00 tx=0x7f2cc401e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.161+0000 7f2ccaffd700 1 -- 172.21.15.97:0/1267682567 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2cbc054290 con 0x7f2cd414f440 2023-07-06T03:47:15.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.356+0000 7f2cd89ff700 1 -- 172.21.15.97:0/1267682567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cd404e340 con 0x7f2cd414f440 2023-07-06T03:47:15.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.357+0000 7f2ccaffd700 1 -- 172.21.15.97:0/1267682567 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cbc05d020 con 0x7f2cd414f440 2023-07-06T03:47:15.359 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:47:15.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 -- 172.21.15.97:0/1267682567 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cb406ace0 msgr2=0x7f2cb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cb406ace0 0x7f2cb406d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4000c00 tx=0x7f2cc401e040 comp rx=0 tx=0).stop 2023-07-06T03:47:15.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 -- 172.21.15.97:0/1267682567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cd414f440 msgr2=0x7f2cd40c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cd414f440 0x7f2cd40c6860 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00e760 tx=0x7f2cbc00ea70 comp rx=0 tx=0).stop 2023-07-06T03:47:15.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 -- 172.21.15.97:0/1267682567 shutdown_connections 2023-07-06T03:47:15.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cb406ace0 0x7f2cb406d190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd40b5b60 0x7f2cd40b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2cc0003a00 0x7f2cd414edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 --2- 172.21.15.97:0/1267682567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cd414f440 0x7f2cd40c6860 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 -- 172.21.15.97:0/1267682567 >> 172.21.15.97:0/1267682567 conn(0x7f2cd406a6c0 msgr2=0x7f2cd40d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:15.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 -- 172.21.15.97:0/1267682567 shutdown_connections 2023-07-06T03:47:15.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.359+0000 7f2cd89ff700 1 -- 172.21.15.97:0/1267682567 wait complete. 2023-07-06T03:47:15.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:47:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.575+0000 7f1de11ff700 1 Processor -- start 2023-07-06T03:47:15.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.575+0000 7f1de11ff700 1 -- start start 2023-07-06T03:47:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.576+0000 7f1de11ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ddc0c3cd0 0x7f1ddc0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.576+0000 7f1de11ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ddc159440 0x7f1ddc0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.576+0000 7f1de11ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ddc0ca260 con 0x7f1ddc159070 2023-07-06T03:47:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.576+0000 7f1de11ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ddc0c99c0 con 0x7f1ddc159440 2023-07-06T03:47:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.576+0000 7f1de11ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0c9ca0 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.576+0000 7f1ddb59e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ddc159440 0x7f1ddc0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.577+0000 7f1ddb59e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ddc159440 0x7f1ddc0cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34948/0 (socket says 172.21.15.97:34948) 2023-07-06T03:47:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.577+0000 7f1ddb59e700 1 -- 172.21.15.97:0/3837250972 learned_addr learned my addr 172.21.15.97:0/3837250972 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.577+0000 7f1dda59c700 1 --2- 172.21.15.97:0/3837250972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ddc0c3cd0 0x7f1ddc0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.577+0000 7f1ddb59e700 1 -- 172.21.15.97:0/3837250972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ddc0c3cd0 msgr2=0x7f1ddc0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.577+0000 7f1ddb59e700 1 --2- 172.21.15.97:0/3837250972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ddc0c3cd0 0x7f1ddc0cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.577+0000 7f1ddb59e700 1 -- 172.21.15.97:0/3837250972 >> v1:172.21.15.146:6789/0 conn(0x7f1ddc159070 legacy=0x7f1ddc0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.577+0000 7f1ddb59e700 1 -- 172.21.15.97:0/3837250972 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc0c9f80 con 0x7f1ddc159440 2023-07-06T03:47:15.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.577+0000 7f1ddb59e700 1 --2- 172.21.15.97:0/3837250972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ddc159440 0x7f1ddc0cca00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400d150 tx=0x7f1dd400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cc9747568da514 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:15.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.577+0000 7f1dd9d9b700 1 -- 172.21.15.97:0/3837250972 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4010070 con 0x7f1ddc159440 2023-07-06T03:47:15.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.577+0000 7f1dd9d9b700 1 -- 172.21.15.97:0/3837250972 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd400dd10 con 0x7f1ddc159440 2023-07-06T03:47:15.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.577+0000 7f1dd9d9b700 1 -- 172.21.15.97:0/3837250972 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd400b310 con 0x7f1ddc159440 2023-07-06T03:47:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.578+0000 7f1de11ff700 1 -- 172.21.15.97:0/3837250972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ddc159440 msgr2=0x7f1ddc0cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.578+0000 7f1de11ff700 1 --2- 172.21.15.97:0/3837250972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ddc159440 0x7f1ddc0cca00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400d150 tx=0x7f1dd400d460 comp rx=0 tx=0).stop 2023-07-06T03:47:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.578+0000 7f1de11ff700 1 -- 172.21.15.97:0/3837250972 shutdown_connections 2023-07-06T03:47:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.578+0000 7f1de11ff700 1 --2- 172.21.15.97:0/3837250972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ddc159440 0x7f1ddc0cca00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.578+0000 7f1de11ff700 1 --2- 172.21.15.97:0/3837250972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ddc0c3cd0 0x7f1ddc0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.578+0000 7f1de11ff700 1 -- 172.21.15.97:0/3837250972 >> 172.21.15.97:0/3837250972 conn(0x7f1ddc06a060 msgr2=0x7f1ddc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.578+0000 7f1de11ff700 1 -- 172.21.15.97:0/3837250972 shutdown_connections 2023-07-06T03:47:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.578+0000 7f1de11ff700 1 -- 172.21.15.97:0/3837250972 wait complete. 2023-07-06T03:47:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.580+0000 7f1de11ff700 1 Processor -- start 2023-07-06T03:47:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.580+0000 7f1de11ff700 1 -- start start 2023-07-06T03:47:15.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.581+0000 7f1de11ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ddc0c3cd0 0x7f1ddc150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.581+0000 7f1de11ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ddc159070 0x7f1ddc0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.581+0000 7f1de11ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ddc159440 0x7f1ddc0b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.581+0000 7f1de11ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc15bf10 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.581+0000 7f1de11ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc15c1f0 con 0x7f1ddc159070 2023-07-06T03:47:15.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.581+0000 7f1de11ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc15c4d0 con 0x7f1ddc159440 2023-07-06T03:47:15.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.581+0000 7f1ddad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ddc0c3cd0 0x7f1ddc150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.581+0000 7f1ddad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ddc0c3cd0 0x7f1ddc150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34964/0 (socket says 172.21.15.97:34964) 2023-07-06T03:47:15.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.581+0000 7f1ddad9d700 1 -- 172.21.15.97:0/2043500923 learned_addr learned my addr 172.21.15.97:0/2043500923 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:15.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.581+0000 7f1ddb59e700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ddc159440 0x7f1ddc0b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.581+0000 7f1dda59c700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ddc159070 0x7f1ddc0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1ddad9d700 1 -- 172.21.15.97:0/2043500923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ddc159440 msgr2=0x7f1ddc0b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1ddad9d700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ddc159440 0x7f1ddc0b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1ddad9d700 1 -- 172.21.15.97:0/2043500923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ddc159070 msgr2=0x7f1ddc0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1ddad9d700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ddc159070 0x7f1ddc0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1ddad9d700 1 -- 172.21.15.97:0/2043500923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd400bd40 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1ddb59e700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ddc159440 0x7f1ddc0b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:15.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1ddad9d700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ddc0c3cd0 0x7f1ddc150870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c4e0 tx=0x7f1dcc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:15.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1dc37fe700 1 -- 172.21.15.97:0/2043500923 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0052d0 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1dc37fe700 1 -- 172.21.15.97:0/2043500923 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dcc005430 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1de11ff700 1 -- 172.21.15.97:0/2043500923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ddc15c7b0 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1dc37fe700 1 -- 172.21.15.97:0/2043500923 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0038c0 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.582+0000 7f1de11ff700 1 -- 172.21.15.97:0/2043500923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ddc150db0 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.583+0000 7f1de11ff700 1 -- 172.21.15.97:0/2043500923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ddc0b1bb0 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.584+0000 7f1dc37fe700 1 -- 172.21.15.97:0/2043500923 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1dcc003a20 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.585+0000 7f1dc37fe700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dc806ace0 0x7f1dc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:15.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.585+0000 7f1dc37fe700 1 -- 172.21.15.97:0/2043500923 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f1dcc0889d0 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.585+0000 7f1dda59c700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dc806ace0 0x7f1dc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:15.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.586+0000 7f1dda59c700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dc806ace0 0x7f1dc806d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0c8e10 tx=0x7f1dd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:15.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.589+0000 7f1dc37fe700 1 -- 172.21.15.97:0/2043500923 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1ddc0b1bb0 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.782+0000 7f1de11ff700 1 -- 172.21.15.97:0/2043500923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1ddc0b1bb0 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.782+0000 7f1dc37fe700 1 -- 172.21.15.97:0/2043500923 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1dcc017020 con 0x7f1ddc0c3cd0 2023-07-06T03:47:15.784 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:47:15.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.784+0000 7f1de11ff700 1 -- 172.21.15.97:0/2043500923 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dc806ace0 msgr2=0x7f1dc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.784+0000 7f1de11ff700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dc806ace0 0x7f1dc806d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0c8e10 tx=0x7f1dd0008040 comp rx=0 tx=0).stop 2023-07-06T03:47:15.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.784+0000 7f1de11ff700 1 -- 172.21.15.97:0/2043500923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ddc0c3cd0 msgr2=0x7f1ddc150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:15.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.784+0000 7f1de11ff700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ddc0c3cd0 0x7f1ddc150870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c4e0 tx=0x7f1dcc00c8a0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.784+0000 7f1de11ff700 1 -- 172.21.15.97:0/2043500923 shutdown_connections 2023-07-06T03:47:15.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.784+0000 7f1de11ff700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dc806ace0 0x7f1dc806d190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.784+0000 7f1de11ff700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ddc159440 0x7f1ddc0b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.784+0000 7f1de11ff700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ddc0c3cd0 0x7f1ddc150870 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.784+0000 7f1de11ff700 1 --2- 172.21.15.97:0/2043500923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ddc159070 0x7f1ddc0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:15.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.784+0000 7f1de11ff700 1 -- 172.21.15.97:0/2043500923 >> 172.21.15.97:0/2043500923 conn(0x7f1ddc06a060 msgr2=0x7f1ddc0ad480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:15.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.785+0000 7f1de11ff700 1 -- 172.21.15.97:0/2043500923 shutdown_connections 2023-07-06T03:47:15.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:15.785+0000 7f1de11ff700 1 -- 172.21.15.97:0/2043500923 wait complete. 2023-07-06T03:47:15.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:47:16.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.002+0000 7f3e7c5ff700 1 Processor -- start 2023-07-06T03:47:16.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.002+0000 7f3e7c5ff700 1 -- start start 2023-07-06T03:47:16.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.003+0000 7f3e7c5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e7415c610 0x7f3e740b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.003+0000 7f3e7c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 0x7f3e740b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.003+0000 7f3e7c5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e740cff90 con 0x7f3e740d2210 2023-07-06T03:47:16.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.003+0000 7f3e7c5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e740d0780 con 0x7f3e7415c610 2023-07-06T03:47:16.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.003+0000 7f3e7c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7415c9e0 con 0x7f3e7406b360 2023-07-06T03:47:16.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.004+0000 7f3e79b9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 0x7f3e740b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.004+0000 7f3e7a39b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e7415c610 0x7f3e740b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.004+0000 7f3e79b9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 0x7f3e740b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:60220/0 (socket says 172.21.15.97:60220) 2023-07-06T03:47:16.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.004+0000 7f3e7a39b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e7415c610 0x7f3e740b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:58816/0 (socket says 172.21.15.97:58816) 2023-07-06T03:47:16.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.004+0000 7f3e79b9a700 1 -- 172.21.15.97:0/3200431566 learned_addr learned my addr 172.21.15.97:0/3200431566 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:16.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.004+0000 7f3e79b9a700 1 -- 172.21.15.97:0/3200431566 >> v2:172.21.15.146:3300/0 conn(0x7f3e7415c610 msgr2=0x7f3e740b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:16.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.004+0000 7f3e79b9a700 1 --2- 172.21.15.97:0/3200431566 >> v2:172.21.15.146:3300/0 conn(0x7f3e7415c610 0x7f3e740b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.004+0000 7f3e79b9a700 1 -- 172.21.15.97:0/3200431566 >> v1:172.21.15.146:6789/0 conn(0x7f3e740d2210 legacy=0x7f3e740d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.005+0000 7f3e79b9a700 1 -- 172.21.15.97:0/3200431566 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e740b5650 con 0x7f3e7406b360 2023-07-06T03:47:16.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.005+0000 7f3e7a39b700 1 --2- 172.21.15.97:0/3200431566 >> v2:172.21.15.146:3300/0 conn(0x7f3e7415c610 0x7f3e740b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:47:16.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.004+0000 7f3e79399700 1 -- 172.21.15.97:0/3200431566 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310264075 0 0) 0x7f3e740cff90 con 0x7f3e740d2210 2023-07-06T03:47:16.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.005+0000 7f3e79b9a700 1 --2- 172.21.15.97:0/3200431566 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 0x7f3e740b4cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e640098f0 tx=0x7f3e64009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be3b205f3ead595 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:16.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.005+0000 7f3e79399700 1 -- 172.21.15.97:0/3200431566 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64011830 con 0x7f3e7406b360 2023-07-06T03:47:16.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.005+0000 7f3e79399700 1 -- 172.21.15.97:0/3200431566 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e64011990 con 0x7f3e7406b360 2023-07-06T03:47:16.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.005+0000 7f3e79399700 1 -- 172.21.15.97:0/3200431566 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64011d20 con 0x7f3e7406b360 2023-07-06T03:47:16.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.006+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/3200431566 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 msgr2=0x7f3e740b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.006+0000 7f3e7c5ff700 1 --2- 172.21.15.97:0/3200431566 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 0x7f3e740b4cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e640098f0 tx=0x7f3e64009c00 comp rx=0 tx=0).stop 2023-07-06T03:47:16.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.006+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/3200431566 shutdown_connections 2023-07-06T03:47:16.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.006+0000 7f3e7c5ff700 1 --2- 172.21.15.97:0/3200431566 >> v2:172.21.15.146:3300/0 conn(0x7f3e7415c610 0x7f3e740b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.006+0000 7f3e7c5ff700 1 --2- 172.21.15.97:0/3200431566 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 0x7f3e740b4cf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.006+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/3200431566 >> 172.21.15.97:0/3200431566 conn(0x7f3e7406a6c0 msgr2=0x7f3e7406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:16.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.006+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/3200431566 shutdown_connections 2023-07-06T03:47:16.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.006+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/3200431566 wait complete. 2023-07-06T03:47:16.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.007+0000 7f3e7c5ff700 1 Processor -- start 2023-07-06T03:47:16.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.008+0000 7f3e7c5ff700 1 -- start start 2023-07-06T03:47:16.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.008+0000 7f3e7c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 0x7f3e740ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.008+0000 7f3e7c5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e740d2210 0x7f3e741542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e7c5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e7415c610 0x7f3e740cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e7c5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7415d730 con 0x7f3e740d2210 2023-07-06T03:47:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e7c5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7415d8c0 con 0x7f3e7415c610 2023-07-06T03:47:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e7c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e740cadf0 con 0x7f3e7406b360 2023-07-06T03:47:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e79b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e740d2210 0x7f3e741542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e79b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e740d2210 0x7f3e741542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34970/0 (socket says 172.21.15.97:34970) 2023-07-06T03:47:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e79b9a700 1 -- 172.21.15.97:0/2842771827 learned_addr learned my addr 172.21.15.97:0/2842771827 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:16.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e7a39b700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 0x7f3e740ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e7ab9c700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e7415c610 0x7f3e740cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e79b9a700 1 -- 172.21.15.97:0/2842771827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 msgr2=0x7f3e740ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e79b9a700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 0x7f3e740ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e79b9a700 1 -- 172.21.15.97:0/2842771827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e7415c610 msgr2=0x7f3e740cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e79b9a700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e7415c610 0x7f3e740cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e79b9a700 1 -- 172.21.15.97:0/2842771827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e700094a0 con 0x7f3e740d2210 2023-07-06T03:47:16.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e7a39b700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 0x7f3e740ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e7ab9c700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e7415c610 0x7f3e740cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.009+0000 7f3e79b9a700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e740d2210 0x7f3e741542e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.010+0000 7f3e6affd700 1 -- 172.21.15.97:0/2842771827 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6401aa10 con 0x7f3e740d2210 2023-07-06T03:47:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.010+0000 7f3e6affd700 1 -- 172.21.15.97:0/2842771827 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6401f070 con 0x7f3e740d2210 2023-07-06T03:47:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.010+0000 7f3e6affd700 1 -- 172.21.15.97:0/2842771827 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6401ad90 con 0x7f3e740d2210 2023-07-06T03:47:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.010+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/2842771827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e64009550 con 0x7f3e740d2210 2023-07-06T03:47:16.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.010+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/2842771827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e740cb3b0 con 0x7f3e740d2210 2023-07-06T03:47:16.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.011+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/2842771827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7415da30 con 0x7f3e740d2210 2023-07-06T03:47:16.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.012+0000 7f3e6affd700 1 -- 172.21.15.97:0/2842771827 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e6401b430 con 0x7f3e740d2210 2023-07-06T03:47:16.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.013+0000 7f3e6affd700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e5806ace0 0x7f3e5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.013+0000 7f3e6affd700 1 -- 172.21.15.97:0/2842771827 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f3e6408a720 con 0x7f3e740d2210 2023-07-06T03:47:16.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.013+0000 7f3e7a39b700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e5806ace0 0x7f3e5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.013+0000 7f3e7a39b700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e5806ace0 0x7f3e5806d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009c50 tx=0x7f3e70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:16.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.017+0000 7f3e6affd700 1 -- 172.21.15.97:0/2842771827 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3e7415da30 con 0x7f3e740d2210 2023-07-06T03:47:16.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.209+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/2842771827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e7415da30 con 0x7f3e740d2210 2023-07-06T03:47:16.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.209+0000 7f3e6affd700 1 -- 172.21.15.97:0/2842771827 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e7415da30 con 0x7f3e740d2210 2023-07-06T03:47:16.212 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:47:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/2842771827 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e5806ace0 msgr2=0x7f3e5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e5806ace0 0x7f3e5806d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009c50 tx=0x7f3e70008040 comp rx=0 tx=0).stop 2023-07-06T03:47:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/2842771827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e740d2210 msgr2=0x7f3e741542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e740d2210 0x7f3e741542e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e64004750 comp rx=0 tx=0).stop 2023-07-06T03:47:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/2842771827 shutdown_connections 2023-07-06T03:47:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e5806ace0 0x7f3e5806d190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e7415c610 0x7f3e740cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e7406b360 0x7f3e740ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 --2- 172.21.15.97:0/2842771827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e740d2210 0x7f3e741542e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/2842771827 >> 172.21.15.97:0/2842771827 conn(0x7f3e7406a6c0 msgr2=0x7f3e7415b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:16.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/2842771827 shutdown_connections 2023-07-06T03:47:16.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.212+0000 7f3e7c5ff700 1 -- 172.21.15.97:0/2842771827 wait complete. 2023-07-06T03:47:16.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:47:16.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.426+0000 7f7a189ff700 1 Processor -- start 2023-07-06T03:47:16.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.427+0000 7f7a189ff700 1 -- start start 2023-07-06T03:47:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.428+0000 7f7a189ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a1406b360 0x7f7a140b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.428+0000 7f7a189ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a140d2210 0x7f7a140d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.428+0000 7f7a189ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a140cff90 con 0x7f7a1415c610 2023-07-06T03:47:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.428+0000 7f7a189ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a140d0780 con 0x7f7a140d2210 2023-07-06T03:47:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.428+0000 7f7a189ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1415c9e0 con 0x7f7a1406b360 2023-07-06T03:47:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.428+0000 7f7a12d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a140d2210 0x7f7a140d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.428+0000 7f7a12d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a140d2210 0x7f7a140d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34976/0 (socket says 172.21.15.97:34976) 2023-07-06T03:47:16.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.428+0000 7f7a12d9d700 1 -- 172.21.15.97:0/215375330 learned_addr learned my addr 172.21.15.97:0/215375330 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.428+0000 7f7a0bfff700 1 --2- 172.21.15.97:0/215375330 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a1406b360 0x7f7a140b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.429+0000 7f7a12d9d700 1 -- 172.21.15.97:0/215375330 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a1406b360 msgr2=0x7f7a140b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.429+0000 7f7a12d9d700 1 --2- 172.21.15.97:0/215375330 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a1406b360 0x7f7a140b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.429+0000 7f7a12d9d700 1 -- 172.21.15.97:0/215375330 >> v1:172.21.15.146:6789/0 conn(0x7f7a1415c610 legacy=0x7f7a140b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.429+0000 7f7a12d9d700 1 -- 172.21.15.97:0/215375330 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a140b5650 con 0x7f7a140d2210 2023-07-06T03:47:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.429+0000 7f7a12d9d700 1 --2- 172.21.15.97:0/215375330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a140d2210 0x7f7a140d1310 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009e80 tx=0x7f7a0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf52e0c456fceb74 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.429+0000 7f7a11d9b700 1 -- 172.21.15.97:0/215375330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00e070 con 0x7f7a140d2210 2023-07-06T03:47:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.429+0000 7f7a11d9b700 1 -- 172.21.15.97:0/215375330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c00b980 con 0x7f7a140d2210 2023-07-06T03:47:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.429+0000 7f7a11d9b700 1 -- 172.21.15.97:0/215375330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00bd30 con 0x7f7a140d2210 2023-07-06T03:47:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.430+0000 7f7a189ff700 1 -- 172.21.15.97:0/215375330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a140d2210 msgr2=0x7f7a140d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.430+0000 7f7a189ff700 1 --2- 172.21.15.97:0/215375330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a140d2210 0x7f7a140d1310 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009e80 tx=0x7f7a0c00b330 comp rx=0 tx=0).stop 2023-07-06T03:47:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.430+0000 7f7a189ff700 1 -- 172.21.15.97:0/215375330 shutdown_connections 2023-07-06T03:47:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.430+0000 7f7a189ff700 1 --2- 172.21.15.97:0/215375330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a140d2210 0x7f7a140d1310 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.430+0000 7f7a189ff700 1 --2- 172.21.15.97:0/215375330 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a1406b360 0x7f7a140b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.430+0000 7f7a189ff700 1 -- 172.21.15.97:0/215375330 >> 172.21.15.97:0/215375330 conn(0x7f7a1406a6c0 msgr2=0x7f7a1406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:16.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.430+0000 7f7a189ff700 1 -- 172.21.15.97:0/215375330 shutdown_connections 2023-07-06T03:47:16.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.430+0000 7f7a189ff700 1 -- 172.21.15.97:0/215375330 wait complete. 2023-07-06T03:47:16.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.431+0000 7f7a189ff700 1 Processor -- start 2023-07-06T03:47:16.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.432+0000 7f7a189ff700 1 -- start start 2023-07-06T03:47:16.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.432+0000 7f7a189ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a1406b360 0x7f7a141f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a189ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a140d2210 0x7f7a140c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a189ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a1415c610 0x7f7a140c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a189ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140cbad0 con 0x7f7a140d2210 2023-07-06T03:47:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a189ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140cbdb0 con 0x7f7a1415c610 2023-07-06T03:47:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a189ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140cc090 con 0x7f7a1406b360 2023-07-06T03:47:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a0bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a140d2210 0x7f7a140c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a0bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a140d2210 0x7f7a140c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34978/0 (socket says 172.21.15.97:34978) 2023-07-06T03:47:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a0bfff700 1 -- 172.21.15.97:0/1754630949 learned_addr learned my addr 172.21.15.97:0/1754630949 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a12d9d700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a1415c610 0x7f7a140c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a1259c700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a1406b360 0x7f7a141f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a0bfff700 1 -- 172.21.15.97:0/1754630949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a1406b360 msgr2=0x7f7a141f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a0bfff700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a1406b360 0x7f7a141f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a0bfff700 1 -- 172.21.15.97:0/1754630949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a1415c610 msgr2=0x7f7a140c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a0bfff700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a1415c610 0x7f7a140c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a0bfff700 1 -- 172.21.15.97:0/1754630949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c009b30 con 0x7f7a140d2210 2023-07-06T03:47:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a1259c700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a1406b360 0x7f7a141f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.433+0000 7f7a0bfff700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a140d2210 0x7f7a140c6860 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00e810 tx=0x7f79fc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.434+0000 7f7a0affd700 1 -- 172.21.15.97:0/1754630949 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc013610 con 0x7f7a140d2210 2023-07-06T03:47:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.434+0000 7f7a0affd700 1 -- 172.21.15.97:0/1754630949 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc013790 con 0x7f7a140d2210 2023-07-06T03:47:16.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.434+0000 7f7a189ff700 1 -- 172.21.15.97:0/1754630949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a140cc370 con 0x7f7a140d2210 2023-07-06T03:47:16.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.434+0000 7f7a0affd700 1 -- 172.21.15.97:0/1754630949 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc013b40 con 0x7f7a140d2210 2023-07-06T03:47:16.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.434+0000 7f7a189ff700 1 -- 172.21.15.97:0/1754630949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a141f1dc0 con 0x7f7a140d2210 2023-07-06T03:47:16.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.435+0000 7f7a189ff700 1 -- 172.21.15.97:0/1754630949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1404e340 con 0x7f7a140d2210 2023-07-06T03:47:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.436+0000 7f7a0affd700 1 -- 172.21.15.97:0/1754630949 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79fc013ca0 con 0x7f7a140d2210 2023-07-06T03:47:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.437+0000 7f7a0affd700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79f406ace0 0x7f79f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.437+0000 7f7a0affd700 1 -- 172.21.15.97:0/1754630949 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f79fc089280 con 0x7f7a140d2210 2023-07-06T03:47:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.437+0000 7f7a1259c700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79f406ace0 0x7f79f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.437+0000 7f7a1259c700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79f406ace0 0x7f79f406d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000f60 tx=0x7f7a04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:16.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.441+0000 7f7a0affd700 1 -- 172.21.15.97:0/1754630949 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f79fc057c30 con 0x7f7a140d2210 2023-07-06T03:47:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.634+0000 7f7a189ff700 1 -- 172.21.15.97:0/1754630949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a140b7de0 con 0x7f7a140d2210 2023-07-06T03:47:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.634+0000 7f7a0affd700 1 -- 172.21.15.97:0/1754630949 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f79fc0577c0 con 0x7f7a140d2210 2023-07-06T03:47:16.636 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:47:16.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 -- 172.21.15.97:0/1754630949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79f406ace0 msgr2=0x7f79f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79f406ace0 0x7f79f406d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000f60 tx=0x7f7a04009040 comp rx=0 tx=0).stop 2023-07-06T03:47:16.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 -- 172.21.15.97:0/1754630949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a140d2210 msgr2=0x7f7a140c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a140d2210 0x7f7a140c6860 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00e810 tx=0x7f79fc00eb20 comp rx=0 tx=0).stop 2023-07-06T03:47:16.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 -- 172.21.15.97:0/1754630949 shutdown_connections 2023-07-06T03:47:16.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79f406ace0 0x7f79f406d190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a1415c610 0x7f7a140c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a1406b360 0x7f7a141f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 --2- 172.21.15.97:0/1754630949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a140d2210 0x7f7a140c6860 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 -- 172.21.15.97:0/1754630949 >> 172.21.15.97:0/1754630949 conn(0x7f7a1406a6c0 msgr2=0x7f7a140d2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:16.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 -- 172.21.15.97:0/1754630949 shutdown_connections 2023-07-06T03:47:16.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.637+0000 7f7a189ff700 1 -- 172.21.15.97:0/1754630949 wait complete. 2023-07-06T03:47:16.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:47:16.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.852+0000 7fc499bff700 1 Processor -- start 2023-07-06T03:47:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.852+0000 7fc499bff700 1 -- start start 2023-07-06T03:47:16.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.854+0000 7fc499bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc494158b20 0x7fc4940cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.854+0000 7fc499bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4940ccbc0 con 0x7fc4940b0560 2023-07-06T03:47:16.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.854+0000 7fc499bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4940adcf0 con 0x7fc49406cab0 2023-07-06T03:47:16.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.854+0000 7fc499bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940ce820 con 0x7fc494158b20 2023-07-06T03:47:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.854+0000 7fc493fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc4940b0560 0x7fc49406f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41844/0 (socket says 172.21.15.97:41844) 2023-07-06T03:47:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.854+0000 7fc493fff700 1 -- 172.21.15.97:0/3659916811 learned_addr learned my addr 172.21.15.97:0/3659916811 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.854+0000 7fc4937fe700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc494158b20 0x7fc4940cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.854+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131897403 0 0) 0x7fc4940ccbc0 con 0x7fc4940b0560 2023-07-06T03:47:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.854+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc47c0035b0 con 0x7fc4940b0560 2023-07-06T03:47:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2413917430 0 0) 0x7fc47c0035b0 con 0x7fc4940b0560 2023-07-06T03:47:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc494158b20 msgr2=0x7fc4940cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc494158b20 0x7fc4940cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 >> v1:172.21.15.146:6789/0 conn(0x7fc49406cab0 legacy=0x7fc4940ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4940adfd0 con 0x7fc4940b0560 2023-07-06T03:47:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fc48c0019f0 con 0x7fc4940b0560 2023-07-06T03:47:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4937fe700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc494158b20 0x7fc4940cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:47:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 >> v1:172.21.15.97:6789/0 conn(0x7fc4940b0560 legacy=0x7fc49406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc494158b20 0x7fc47c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c003f70 0x7fc47c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc47c004d90 0x7fc47c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940ce820 con 0x7fc47c003f70 2023-07-06T03:47:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c010020 con 0x7fc47c004d90 2023-07-06T03:47:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c010160 con 0x7fc494158b20 2023-07-06T03:47:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.855+0000 7fc492ffd700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c003f70 0x7fc47c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc4937fe700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc494158b20 0x7fc47c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc493fff700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc47c004d90 0x7fc47c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc492ffd700 1 -- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc494158b20 msgr2=0x7fc47c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc492ffd700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc494158b20 0x7fc47c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc492ffd700 1 -- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc47c004d90 msgr2=0x7fc47c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc492ffd700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc47c004d90 0x7fc47c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc492ffd700 1 -- 172.21.15.97:0/3659916811 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4840094a0 con 0x7fc47c003f70 2023-07-06T03:47:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc493fff700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc47c004d90 0x7fc47c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc492ffd700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c003f70 0x7fc47c004850 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc4880018d0 tx=0x7fc47c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c34f774e1f21e0f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48800c690 con 0x7fc47c003f70 2023-07-06T03:47:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48800c7f0 con 0x7fc47c003f70 2023-07-06T03:47:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.856+0000 7fc4927fc700 1 -- 172.21.15.97:0/3659916811 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48800cba0 con 0x7fc47c003f70 2023-07-06T03:47:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.857+0000 7fc499bff700 1 -- 172.21.15.97:0/3659916811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c003f70 msgr2=0x7fc47c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.857+0000 7fc499bff700 1 --2- 172.21.15.97:0/3659916811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c003f70 0x7fc47c004850 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc4880018d0 tx=0x7fc47c005880 comp rx=0 tx=0).stop 2023-07-06T03:47:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.857+0000 7fc493fff700 1 -- 172.21.15.97:0/3659916811 reap_dead start 2023-07-06T03:47:16.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.857+0000 7fc499bff700 1 -- 172.21.15.97:0/3659916811 shutdown_connections 2023-07-06T03:47:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.857+0000 7fc499bff700 1 -- 172.21.15.97:0/3659916811 >> 172.21.15.97:0/3659916811 conn(0x7fc49406a060 msgr2=0x7fc49406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.857+0000 7fc499bff700 1 -- 172.21.15.97:0/3659916811 shutdown_connections 2023-07-06T03:47:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.857+0000 7fc499bff700 1 -- 172.21.15.97:0/3659916811 wait complete. 2023-07-06T03:47:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.859+0000 7fc499bff700 1 Processor -- start 2023-07-06T03:47:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.859+0000 7fc499bff700 1 -- start start 2023-07-06T03:47:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.860+0000 7fc499bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc47c003f70 0x7fc494155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.860+0000 7fc499bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc49415abb0 0x7fc494155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.860+0000 7fc499bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4940aeca0 0x7fc4940af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.860+0000 7fc499bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4941578a0 con 0x7fc49415abb0 2023-07-06T03:47:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.860+0000 7fc499bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940af650 con 0x7fc47c003f70 2023-07-06T03:47:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.860+0000 7fc499bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49415b020 con 0x7fc4940aeca0 2023-07-06T03:47:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.860+0000 7fc492ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc49415abb0 0x7fc494155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.860+0000 7fc492ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc49415abb0 0x7fc494155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34996/0 (socket says 172.21.15.97:34996) 2023-07-06T03:47:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.860+0000 7fc492ffd700 1 -- 172.21.15.97:0/247840376 learned_addr learned my addr 172.21.15.97:0/247840376 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.860+0000 7fc4937fe700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc47c003f70 0x7fc494155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.861+0000 7fc493fff700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4940aeca0 0x7fc4940af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.861+0000 7fc492ffd700 1 -- 172.21.15.97:0/247840376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4940aeca0 msgr2=0x7fc4940af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.861+0000 7fc492ffd700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4940aeca0 0x7fc4940af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.861+0000 7fc492ffd700 1 -- 172.21.15.97:0/247840376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc47c003f70 msgr2=0x7fc494155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.861+0000 7fc492ffd700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc47c003f70 0x7fc494155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.861+0000 7fc492ffd700 1 -- 172.21.15.97:0/247840376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494156350 con 0x7fc49415abb0 2023-07-06T03:47:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.861+0000 7fc492ffd700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc49415abb0 0x7fc494155480 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc48800c0e0 tx=0x7fc488001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.861+0000 7fc47bfff700 1 -- 172.21.15.97:0/247840376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488001330 con 0x7fc49415abb0 2023-07-06T03:47:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.861+0000 7fc47bfff700 1 -- 172.21.15.97:0/247840376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4880052d0 con 0x7fc49415abb0 2023-07-06T03:47:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.861+0000 7fc47bfff700 1 -- 172.21.15.97:0/247840376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488003680 con 0x7fc49415abb0 2023-07-06T03:47:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.861+0000 7fc499bff700 1 -- 172.21.15.97:0/247840376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc494156630 con 0x7fc49415abb0 2023-07-06T03:47:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.862+0000 7fc499bff700 1 -- 172.21.15.97:0/247840376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc494156bf0 con 0x7fc49415abb0 2023-07-06T03:47:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.863+0000 7fc499bff700 1 -- 172.21.15.97:0/247840376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49404e340 con 0x7fc49415abb0 2023-07-06T03:47:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.863+0000 7fc47bfff700 1 -- 172.21.15.97:0/247840376 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc488023070 con 0x7fc49415abb0 2023-07-06T03:47:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.864+0000 7fc47bfff700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc48006ace0 0x7fc48006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:16.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.864+0000 7fc47bfff700 1 -- 172.21.15.97:0/247840376 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fc48808a8d0 con 0x7fc49415abb0 2023-07-06T03:47:16.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.865+0000 7fc4937fe700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc48006ace0 0x7fc48006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:16.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.865+0000 7fc4937fe700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc48006ace0 0x7fc48006d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc484009df0 tx=0x7fc484009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:16.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:16.869+0000 7fc47bfff700 1 -- 172.21.15.97:0/247840376 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc488055930 con 0x7fc49415abb0 2023-07-06T03:47:17.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.082+0000 7fc499bff700 1 -- 172.21.15.97:0/247840376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc4940c9b80 con 0x7fc49415abb0 2023-07-06T03:47:17.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.082+0000 7fc47bfff700 1 -- 172.21.15.97:0/247840376 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc488058f50 con 0x7fc49415abb0 2023-07-06T03:47:17.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.084+0000 7fc499bff700 1 -- 172.21.15.97:0/247840376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc48006ace0 msgr2=0x7fc48006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:17.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.084+0000 7fc499bff700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc48006ace0 0x7fc48006d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc484009df0 tx=0x7fc484009210 comp rx=0 tx=0).stop 2023-07-06T03:47:17.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.084+0000 7fc499bff700 1 -- 172.21.15.97:0/247840376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc49415abb0 msgr2=0x7fc494155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:17.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.084+0000 7fc499bff700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc49415abb0 0x7fc494155480 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc48800c0e0 tx=0x7fc488001230 comp rx=0 tx=0).stop 2023-07-06T03:47:17.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.085+0000 7fc499bff700 1 -- 172.21.15.97:0/247840376 shutdown_connections 2023-07-06T03:47:17.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.085+0000 7fc499bff700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc48006ace0 0x7fc48006d190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:17.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.085+0000 7fc499bff700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4940aeca0 0x7fc4940af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:17.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.085+0000 7fc499bff700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc47c003f70 0x7fc494155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:17.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.085+0000 7fc499bff700 1 --2- 172.21.15.97:0/247840376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc49415abb0 0x7fc494155480 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:17.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.085+0000 7fc499bff700 1 -- 172.21.15.97:0/247840376 >> 172.21.15.97:0/247840376 conn(0x7fc49406a060 msgr2=0x7fc4940c93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:17.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.085+0000 7fc499bff700 1 -- 172.21.15.97:0/247840376 shutdown_connections 2023-07-06T03:47:17.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.085+0000 7fc499bff700 1 -- 172.21.15.97:0/247840376 wait complete. 2023-07-06T03:47:17.088 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:47:17.105 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2023-07-06T03:47:17.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:17.105 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-06T03:47:17.179 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[120442]: fuse finished with error 0 and tester_r 0 2023-07-06T03:47:17.183 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:47:17.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:17.184 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T03:47:17.345 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:47:17.345 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:47:17.391 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:47:17.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:47:17.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:17.392 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T03:47:17.476 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T03:47:17.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:47:17.478 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:47:17.478 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:47:17.525 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:47:17.526 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:47:17.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:17.527 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T03:47:17.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:47:17.614 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T03:47:17.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:47:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.808+0000 7f8089bff700 1 Processor -- start 2023-07-06T03:47:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.808+0000 7f8089bff700 1 -- start start 2023-07-06T03:47:17.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.810+0000 7f8089bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80840cd390 0x7f8084160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.810+0000 7f8089bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840cdae0 con 0x7f8084160dd0 2023-07-06T03:47:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.810+0000 7f8089bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840b6640 con 0x7f808415c520 2023-07-06T03:47:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.810+0000 7f8089bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80840b6920 con 0x7f80840cd390 2023-07-06T03:47:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.810+0000 7f8083fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8084160dd0 0x7f80840b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41852/0 (socket says 172.21.15.97:41852) 2023-07-06T03:47:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.810+0000 7f8083fff700 1 -- 172.21.15.97:0/3539020271 learned_addr learned my addr 172.21.15.97:0/3539020271 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.810+0000 7f8082ffd700 1 --2- 172.21.15.97:0/3539020271 >> v2:172.21.15.146:3300/0 conn(0x7f80840cd390 0x7f8084160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:17.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.811+0000 7f80827fc700 1 -- 172.21.15.97:0/3539020271 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105517506 0 0) 0x7f80840cdae0 con 0x7f8084160dd0 2023-07-06T03:47:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.811+0000 7f80827fc700 1 -- 172.21.15.97:0/3539020271 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f806c0035b0 con 0x7f8084160dd0 2023-07-06T03:47:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.811+0000 7f80827fc700 1 -- 172.21.15.97:0/3539020271 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855016617 0 0) 0x7f80840b6640 con 0x7f808415c520 2023-07-06T03:47:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.811+0000 7f80827fc700 1 -- 172.21.15.97:0/3539020271 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80840cdae0 con 0x7f808415c520 2023-07-06T03:47:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.811+0000 7f8082ffd700 1 -- 172.21.15.97:0/3539020271 >> v1:172.21.15.146:6789/0 conn(0x7f808415c520 legacy=0x7f8084160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.811+0000 7f8082ffd700 1 -- 172.21.15.97:0/3539020271 >> v1:172.21.15.97:6789/0 conn(0x7f8084160dd0 legacy=0x7f80840b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.811+0000 7f8082ffd700 1 -- 172.21.15.97:0/3539020271 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840b6c00 con 0x7f80840cd390 2023-07-06T03:47:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.811+0000 7f8082ffd700 1 --2- 172.21.15.97:0/3539020271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80840cd390 0x7f8084160890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8078009840 tx=0x7f8078009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f96b63f00f8ded server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.811+0000 7f80827fc700 1 -- 172.21.15.97:0/3539020271 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078011830 con 0x7f80840cd390 2023-07-06T03:47:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.811+0000 7f80827fc700 1 -- 172.21.15.97:0/3539020271 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8078011990 con 0x7f80840cd390 2023-07-06T03:47:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.811+0000 7f80827fc700 1 -- 172.21.15.97:0/3539020271 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078011d20 con 0x7f80840cd390 2023-07-06T03:47:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.812+0000 7f8089bff700 1 -- 172.21.15.97:0/3539020271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80840cd390 msgr2=0x7f8084160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.812+0000 7f8089bff700 1 --2- 172.21.15.97:0/3539020271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80840cd390 0x7f8084160890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8078009840 tx=0x7f8078009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.812+0000 7f8089bff700 1 -- 172.21.15.97:0/3539020271 shutdown_connections 2023-07-06T03:47:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.812+0000 7f8089bff700 1 --2- 172.21.15.97:0/3539020271 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80840cd390 0x7f8084160890 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.812+0000 7f8089bff700 1 -- 172.21.15.97:0/3539020271 >> 172.21.15.97:0/3539020271 conn(0x7f8084071790 msgr2=0x7f8084071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.812+0000 7f8089bff700 1 -- 172.21.15.97:0/3539020271 shutdown_connections 2023-07-06T03:47:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.812+0000 7f8089bff700 1 -- 172.21.15.97:0/3539020271 wait complete. 2023-07-06T03:47:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.814+0000 7f8089bff700 1 Processor -- start 2023-07-06T03:47:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.814+0000 7f8089bff700 1 -- start start 2023-07-06T03:47:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8089bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80840cd390 0x7f80841f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8089bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f808415c520 0x7f80840cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8089bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084160dd0 0x7f808414fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8089bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808415ce00 con 0x7f8084160dd0 2023-07-06T03:47:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8089bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80840cb800 con 0x7f808415c520 2023-07-06T03:47:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8089bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80841f22c0 con 0x7f80840cd390 2023-07-06T03:47:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8083fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084160dd0 0x7f808414fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8083fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084160dd0 0x7f808414fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34998/0 (socket says 172.21.15.97:34998) 2023-07-06T03:47:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8083fff700 1 -- 172.21.15.97:0/3987647023 learned_addr learned my addr 172.21.15.97:0/3987647023 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8082ffd700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f808415c520 0x7f80840cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f80837fe700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80840cd390 0x7f80841f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8083fff700 1 -- 172.21.15.97:0/3987647023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80840cd390 msgr2=0x7f80841f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8083fff700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80840cd390 0x7f80841f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8083fff700 1 -- 172.21.15.97:0/3987647023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f808415c520 msgr2=0x7f80840cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8083fff700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f808415c520 0x7f80840cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.815+0000 7f8083fff700 1 -- 172.21.15.97:0/3987647023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80780094a0 con 0x7f8084160dd0 2023-07-06T03:47:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.816+0000 7f8083fff700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084160dd0 0x7f808414fc50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f807c00cba0 tx=0x7f807c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.816+0000 7f806bfff700 1 -- 172.21.15.97:0/3987647023 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00aa40 con 0x7f8084160dd0 2023-07-06T03:47:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.816+0000 7f806bfff700 1 -- 172.21.15.97:0/3987647023 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c00aba0 con 0x7f8084160dd0 2023-07-06T03:47:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.816+0000 7f8089bff700 1 -- 172.21.15.97:0/3987647023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80841f25a0 con 0x7f8084160dd0 2023-07-06T03:47:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.816+0000 7f806bfff700 1 -- 172.21.15.97:0/3987647023 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c012400 con 0x7f8084160dd0 2023-07-06T03:47:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.816+0000 7f8089bff700 1 -- 172.21.15.97:0/3987647023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80841f2b60 con 0x7f8084160dd0 2023-07-06T03:47:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.817+0000 7f8089bff700 1 -- 172.21.15.97:0/3987647023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808404e340 con 0x7f8084160dd0 2023-07-06T03:47:17.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.818+0000 7f806bfff700 1 -- 172.21.15.97:0/3987647023 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f807c01b3b0 con 0x7f8084160dd0 2023-07-06T03:47:17.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.819+0000 7f806bfff700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f807006ace0 0x7f807006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:17.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.819+0000 7f806bfff700 1 -- 172.21.15.97:0/3987647023 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f807c089130 con 0x7f8084160dd0 2023-07-06T03:47:17.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.819+0000 7f80837fe700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f807006ace0 0x7f807006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:17.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.820+0000 7f80837fe700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f807006ace0 0x7f807006d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f80740011e0 tx=0x7f8074009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:17.826+0000 7f806bfff700 1 -- 172.21.15.97:0/3987647023 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f807c054210 con 0x7f8084160dd0 2023-07-06T03:47:18.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.069+0000 7f8089bff700 1 -- 172.21.15.97:0/3987647023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80840c91f0 con 0x7f8084160dd0 2023-07-06T03:47:18.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.070+0000 7f806bfff700 1 -- 172.21.15.97:0/3987647023 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f807c057830 con 0x7f8084160dd0 2023-07-06T03:47:18.072 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:18.072 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/1839628004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1839628004},{"type":"v1","addr":"172.21.15.146:6834","nonce":1839628004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4937,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2131262285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2131262285},{"type":"v1","addr":"172.21.15.146:6835","nonce":2131262285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1367937004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367937004},{"type":"v1","addr":"172.21.15.97:6835","nonce":1367937004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:46:37.331190+0000","modified":"2023-07-06T03:46:40.864523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/663657264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":663657264},{"type":"v1","addr":"172.21.15.97:6837","nonce":663657264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T03:47:18.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.072+0000 7f8089bff700 1 -- 172.21.15.97:0/3987647023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f807006ace0 msgr2=0x7f807006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.072+0000 7f8089bff700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f807006ace0 0x7f807006d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f80740011e0 tx=0x7f8074009040 comp rx=0 tx=0).stop 2023-07-06T03:47:18.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.072+0000 7f8089bff700 1 -- 172.21.15.97:0/3987647023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084160dd0 msgr2=0x7f808414fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.072+0000 7f8089bff700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084160dd0 0x7f808414fc50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f807c00cba0 tx=0x7f807c00ceb0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.073+0000 7f8089bff700 1 -- 172.21.15.97:0/3987647023 shutdown_connections 2023-07-06T03:47:18.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.073+0000 7f8089bff700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f807006ace0 0x7f807006d190 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.073+0000 7f8089bff700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084160dd0 0x7f808414fc50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.073+0000 7f8089bff700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80840cd390 0x7f80841f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.073+0000 7f8089bff700 1 --2- 172.21.15.97:0/3987647023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f808415c520 0x7f80840cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.073+0000 7f8089bff700 1 -- 172.21.15.97:0/3987647023 >> 172.21.15.97:0/3987647023 conn(0x7f8084071790 msgr2=0x7f80840c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.073+0000 7f8089bff700 1 -- 172.21.15.97:0/3987647023 shutdown_connections 2023-07-06T03:47:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.073+0000 7f8089bff700 1 -- 172.21.15.97:0/3987647023 wait complete. 2023-07-06T03:47:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2023-07-06T03:47:18.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:47:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.286+0000 7f6b2d5ff700 1 Processor -- start 2023-07-06T03:47:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.287+0000 7f6b2d5ff700 1 -- start start 2023-07-06T03:47:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.288+0000 7f6b2d5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 0x7f6b280d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.288+0000 7f6b2d5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6b28071820 0x7f6b2806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.288+0000 7f6b2d5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b28071c90 0x7f6b2806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.288+0000 7f6b2d5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b280d0610 con 0x7f6b28071c90 2023-07-06T03:47:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.288+0000 7f6b2d5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b280cdc80 con 0x7f6b28071820 2023-07-06T03:47:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.288+0000 7f6b2d5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b280ce380 con 0x7f6b2815c740 2023-07-06T03:47:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.288+0000 7f6b277fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b28071c90 0x7f6b2806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.288+0000 7f6b277fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b28071c90 0x7f6b2806a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35000/0 (socket says 172.21.15.97:35000) 2023-07-06T03:47:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.288+0000 7f6b277fe700 1 -- 172.21.15.97:0/945887064 learned_addr learned my addr 172.21.15.97:0/945887064 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.288+0000 7f6b26ffd700 1 --2- 172.21.15.97:0/945887064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 0x7f6b280d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.288+0000 7f6b1ffff700 1 --2- 172.21.15.97:0/945887064 >> v2:172.21.15.146:3300/0 conn(0x7f6b28071820 0x7f6b2806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.289+0000 7f6b277fe700 1 -- 172.21.15.97:0/945887064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 msgr2=0x7f6b280d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.289+0000 7f6b277fe700 1 --2- 172.21.15.97:0/945887064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 0x7f6b280d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.289+0000 7f6b277fe700 1 -- 172.21.15.97:0/945887064 >> v2:172.21.15.146:3300/0 conn(0x7f6b28071820 msgr2=0x7f6b2806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.289+0000 7f6b277fe700 1 --2- 172.21.15.97:0/945887064 >> v2:172.21.15.146:3300/0 conn(0x7f6b28071820 0x7f6b2806a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.289+0000 7f6b277fe700 1 -- 172.21.15.97:0/945887064 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b280cd630 con 0x7f6b28071c90 2023-07-06T03:47:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.289+0000 7f6b26ffd700 1 --2- 172.21.15.97:0/945887064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 0x7f6b280d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.289+0000 7f6b277fe700 1 --2- 172.21.15.97:0/945887064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28071c90 0x7f6b2806a5a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6b28066ce0 tx=0x7f6b2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1fd1d4df946d1c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.289+0000 7f6b267fc700 1 -- 172.21.15.97:0/945887064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000e070 con 0x7f6b28071c90 2023-07-06T03:47:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.289+0000 7f6b267fc700 1 -- 172.21.15.97:0/945887064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2000b9c0 con 0x7f6b28071c90 2023-07-06T03:47:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.289+0000 7f6b267fc700 1 -- 172.21.15.97:0/945887064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000bd70 con 0x7f6b28071c90 2023-07-06T03:47:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.290+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/945887064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28071c90 msgr2=0x7f6b2806a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.290+0000 7f6b2d5ff700 1 --2- 172.21.15.97:0/945887064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28071c90 0x7f6b2806a5a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6b28066ce0 tx=0x7f6b2000b330 comp rx=0 tx=0).stop 2023-07-06T03:47:18.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.290+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/945887064 shutdown_connections 2023-07-06T03:47:18.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.290+0000 7f6b2d5ff700 1 --2- 172.21.15.97:0/945887064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28071c90 0x7f6b2806a5a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.290+0000 7f6b2d5ff700 1 --2- 172.21.15.97:0/945887064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 0x7f6b280d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.290+0000 7f6b2d5ff700 1 --2- 172.21.15.97:0/945887064 >> v2:172.21.15.146:3300/0 conn(0x7f6b28071820 0x7f6b2806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.290+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/945887064 >> 172.21.15.97:0/945887064 conn(0x7f6b280b2280 msgr2=0x7f6b280cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:18.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.290+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/945887064 shutdown_connections 2023-07-06T03:47:18.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.290+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/945887064 wait complete. 2023-07-06T03:47:18.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.292+0000 7f6b2d5ff700 1 Processor -- start 2023-07-06T03:47:18.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.292+0000 7f6b2d5ff700 1 -- start start 2023-07-06T03:47:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b2d5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28071820 0x7f6b281f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b2d5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 0x7f6b2815e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b2d5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b2815e910 0x7f6b280b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b2d5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b280b44a0 con 0x7f6b28071820 2023-07-06T03:47:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b2d5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2815edb0 con 0x7f6b2815e910 2023-07-06T03:47:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b2d5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b280b3920 con 0x7f6b2815c740 2023-07-06T03:47:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b26ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28071820 0x7f6b281f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b26ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28071820 0x7f6b281f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35002/0 (socket says 172.21.15.97:35002) 2023-07-06T03:47:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b26ffd700 1 -- 172.21.15.97:0/472880383 learned_addr learned my addr 172.21.15.97:0/472880383 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b277fe700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b2815e910 0x7f6b280b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b1ffff700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 0x7f6b2815e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b26ffd700 1 -- 172.21.15.97:0/472880383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 msgr2=0x7f6b2815e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b26ffd700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 0x7f6b2815e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b26ffd700 1 -- 172.21.15.97:0/472880383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b2815e910 msgr2=0x7f6b280b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b26ffd700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b2815e910 0x7f6b280b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b26ffd700 1 -- 172.21.15.97:0/472880383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b20009b30 con 0x7f6b28071820 2023-07-06T03:47:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b277fe700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b2815e910 0x7f6b280b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.293+0000 7f6b1ffff700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 0x7f6b2815e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.294+0000 7f6b26ffd700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28071820 0x7f6b281f1650 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800e840 tx=0x7f6b1800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.294+0000 7f6b1f7fe700 1 -- 172.21.15.97:0/472880383 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18005c20 con 0x7f6b28071820 2023-07-06T03:47:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.294+0000 7f6b1f7fe700 1 -- 172.21.15.97:0/472880383 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b18005d80 con 0x7f6b28071820 2023-07-06T03:47:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.294+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/472880383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b280b3c00 con 0x7f6b28071820 2023-07-06T03:47:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.294+0000 7f6b1f7fe700 1 -- 172.21.15.97:0/472880383 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18006250 con 0x7f6b28071820 2023-07-06T03:47:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.294+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/472880383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b280b41c0 con 0x7f6b28071820 2023-07-06T03:47:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.295+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/472880383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b280b1400 con 0x7f6b28071820 2023-07-06T03:47:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.296+0000 7f6b1f7fe700 1 -- 172.21.15.97:0/472880383 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b180063b0 con 0x7f6b28071820 2023-07-06T03:47:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.297+0000 7f6b1f7fe700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b0806ace0 0x7f6b0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.297+0000 7f6b1f7fe700 1 -- 172.21.15.97:0/472880383 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f6b18088f60 con 0x7f6b28071820 2023-07-06T03:47:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.297+0000 7f6b1ffff700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b0806ace0 0x7f6b0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.297+0000 7f6b1ffff700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b0806ace0 0x7f6b0806d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6b28158a10 tx=0x7f6b10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.301+0000 7f6b1f7fe700 1 -- 172.21.15.97:0/472880383 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6b18053fb0 con 0x7f6b28071820 2023-07-06T03:47:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.497+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/472880383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b2804e340 con 0x7f6b28071820 2023-07-06T03:47:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.498+0000 7f6b1f7fe700 1 -- 172.21.15.97:0/472880383 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18460 (secure 0 0 0) 0x7f6b18020090 con 0x7f6b28071820 2023-07-06T03:47:18.500 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:18.500 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:46:52.661343+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:46:35.584013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T03:46:36.303248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.28","mappings":[{"from":4,"to":7}]},{"pgid":"6.36","mappings":[{"from":1,"to":2}]},{"pgid":"6.38","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:47:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.500+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/472880383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b0806ace0 msgr2=0x7f6b0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.500+0000 7f6b2d5ff700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b0806ace0 0x7f6b0806d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6b28158a10 tx=0x7f6b10008040 comp rx=0 tx=0).stop 2023-07-06T03:47:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.500+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/472880383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28071820 msgr2=0x7f6b281f1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.500+0000 7f6b2d5ff700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28071820 0x7f6b281f1650 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800e840 tx=0x7f6b1800ec00 comp rx=0 tx=0).stop 2023-07-06T03:47:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.500+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/472880383 shutdown_connections 2023-07-06T03:47:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.500+0000 7f6b2d5ff700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b0806ace0 0x7f6b0806d190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.500+0000 7f6b2d5ff700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b2815e910 0x7f6b280b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.500+0000 7f6b2d5ff700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b28071820 0x7f6b281f1650 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.500+0000 7f6b2d5ff700 1 --2- 172.21.15.97:0/472880383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b2815c740 0x7f6b2815e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.500+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/472880383 >> 172.21.15.97:0/472880383 conn(0x7f6b280b2280 msgr2=0x7f6b280cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.501+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/472880383 shutdown_connections 2023-07-06T03:47:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.501+0000 7f6b2d5ff700 1 -- 172.21.15.97:0/472880383 wait complete. 2023-07-06T03:47:18.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:46:35.584013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:46:36.303248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:47:18.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:47:18.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.720+0000 7f9bf5dff700 1 Processor -- start 2023-07-06T03:47:18.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.720+0000 7f9bf5dff700 1 -- start start 2023-07-06T03:47:18.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.721+0000 7f9bf5dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bf015c610 0x7f9bf00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.721+0000 7f9bf5dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9bf00d2210 0x7f9bf00d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.721+0000 7f9bf5dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf00cff90 con 0x7f9bf006b360 2023-07-06T03:47:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9bf5dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf00d0780 con 0x7f9bf015c610 2023-07-06T03:47:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9bf5dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf015c9e0 con 0x7f9bf00d2210 2023-07-06T03:47:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9bef7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bf015c610 0x7f9bf00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9bef7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bf015c610 0x7f9bf00b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35012/0 (socket says 172.21.15.97:35012) 2023-07-06T03:47:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9beeffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9bf006b360 0x7f9bf00b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41866/0 (socket says 172.21.15.97:41866) 2023-07-06T03:47:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9bef7fe700 1 -- 172.21.15.97:0/106704905 learned_addr learned my addr 172.21.15.97:0/106704905 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:18.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9beffff700 1 --2- 172.21.15.97:0/106704905 >> v2:172.21.15.146:3300/0 conn(0x7f9bf00d2210 0x7f9bf00d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:18.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9bef7fe700 1 -- 172.21.15.97:0/106704905 >> v2:172.21.15.146:3300/0 conn(0x7f9bf00d2210 msgr2=0x7f9bf00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:18.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9bef7fe700 1 --2- 172.21.15.97:0/106704905 >> v2:172.21.15.146:3300/0 conn(0x7f9bf00d2210 0x7f9bf00d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9bef7fe700 1 -- 172.21.15.97:0/106704905 >> v1:172.21.15.97:6789/0 conn(0x7f9bf006b360 legacy=0x7f9bf00b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9bef7fe700 1 -- 172.21.15.97:0/106704905 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00b5650 con 0x7f9bf015c610 2023-07-06T03:47:18.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.722+0000 7f9bee7fc700 1 -- 172.21.15.97:0/106704905 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 935742791 0 0) 0x7f9bf00cff90 con 0x7f9bf006b360 2023-07-06T03:47:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bef7fe700 1 --2- 172.21.15.97:0/106704905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf015c610 0x7f9bf00b47b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00d2bc0 tx=0x7f9be0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb2adc56495af13 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bee7fc700 1 -- 172.21.15.97:0/106704905 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be000b860 con 0x7f9bf015c610 2023-07-06T03:47:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bee7fc700 1 -- 172.21.15.97:0/106704905 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be000b9c0 con 0x7f9bf015c610 2023-07-06T03:47:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bee7fc700 1 -- 172.21.15.97:0/106704905 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be000bd50 con 0x7f9bf015c610 2023-07-06T03:47:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bf5dff700 1 -- 172.21.15.97:0/106704905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf015c610 msgr2=0x7f9bf00b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bf5dff700 1 --2- 172.21.15.97:0/106704905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf015c610 0x7f9bf00b47b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00d2bc0 tx=0x7f9be0009c00 comp rx=0 tx=0).stop 2023-07-06T03:47:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bf5dff700 1 -- 172.21.15.97:0/106704905 shutdown_connections 2023-07-06T03:47:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bf5dff700 1 --2- 172.21.15.97:0/106704905 >> v2:172.21.15.146:3300/0 conn(0x7f9bf00d2210 0x7f9bf00d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bf5dff700 1 --2- 172.21.15.97:0/106704905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf015c610 0x7f9bf00b47b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bf5dff700 1 -- 172.21.15.97:0/106704905 >> 172.21.15.97:0/106704905 conn(0x7f9bf006a6c0 msgr2=0x7f9bf006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:18.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bf5dff700 1 -- 172.21.15.97:0/106704905 shutdown_connections 2023-07-06T03:47:18.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.723+0000 7f9bf5dff700 1 -- 172.21.15.97:0/106704905 wait complete. 2023-07-06T03:47:18.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.725+0000 7f9bf5dff700 1 Processor -- start 2023-07-06T03:47:18.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.725+0000 7f9bf5dff700 1 -- start start 2023-07-06T03:47:18.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.726+0000 7f9bf5dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bf006b360 0x7f9bf01511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.726+0000 7f9bf5dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d2210 0x7f9bf00cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.726+0000 7f9bf5dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bf015c610 0x7f9bf00cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.726+0000 7f9bf5dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf014dab0 con 0x7f9bf00d2210 2023-07-06T03:47:18.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.726+0000 7f9bf5dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00ca560 con 0x7f9bf015c610 2023-07-06T03:47:18.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.726+0000 7f9bf5dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00ca840 con 0x7f9bf006b360 2023-07-06T03:47:18.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.726+0000 7f9beeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d2210 0x7f9bf00cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:18.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.726+0000 7f9beeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d2210 0x7f9bf00cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35018/0 (socket says 172.21.15.97:35018) 2023-07-06T03:47:18.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.726+0000 7f9beeffd700 1 -- 172.21.15.97:0/3363522450 learned_addr learned my addr 172.21.15.97:0/3363522450 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:18.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.727+0000 7f9beeffd700 1 -- 172.21.15.97:0/3363522450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bf006b360 msgr2=0x7f9bf01511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.727+0000 7f9beeffd700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bf006b360 0x7f9bf01511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.727+0000 7f9beeffd700 1 -- 172.21.15.97:0/3363522450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bf015c610 msgr2=0x7f9bf00cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.727+0000 7f9beeffd700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bf015c610 0x7f9bf00cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.727+0000 7f9beeffd700 1 -- 172.21.15.97:0/3363522450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be0009550 con 0x7f9bf00d2210 2023-07-06T03:47:18.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.727+0000 7f9beeffd700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d2210 0x7f9bf00cb600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9be40011e0 tx=0x7f9be400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:18.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.727+0000 7f9bd7fff700 1 -- 172.21.15.97:0/3363522450 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4004c80 con 0x7f9bf00d2210 2023-07-06T03:47:18.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.727+0000 7f9bd7fff700 1 -- 172.21.15.97:0/3363522450 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be4004e00 con 0x7f9bf00d2210 2023-07-06T03:47:18.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.727+0000 7f9bd7fff700 1 -- 172.21.15.97:0/3363522450 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4007310 con 0x7f9bf00d2210 2023-07-06T03:47:18.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.727+0000 7f9bf5dff700 1 -- 172.21.15.97:0/3363522450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf00cab20 con 0x7f9bf00d2210 2023-07-06T03:47:18.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.727+0000 7f9bf5dff700 1 -- 172.21.15.97:0/3363522450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf00cb0e0 con 0x7f9bf00d2210 2023-07-06T03:47:18.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.728+0000 7f9bf5dff700 1 -- 172.21.15.97:0/3363522450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf00cb300 con 0x7f9bf00d2210 2023-07-06T03:47:18.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.729+0000 7f9bd7fff700 1 -- 172.21.15.97:0/3363522450 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9be4007470 con 0x7f9bf00d2210 2023-07-06T03:47:18.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.730+0000 7f9bd7fff700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bdc06ace0 0x7f9bdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:18.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.730+0000 7f9bd7fff700 1 -- 172.21.15.97:0/3363522450 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f9be40892b0 con 0x7f9bf00d2210 2023-07-06T03:47:18.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.731+0000 7f9bef7fe700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bdc06ace0 0x7f9bdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:18.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.731+0000 7f9bef7fe700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bdc06ace0 0x7f9bdc06d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9be0014040 tx=0x7f9be0010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:18.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.735+0000 7f9bd7fff700 1 -- 172.21.15.97:0/3363522450 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9be4054300 con 0x7f9bf00d2210 2023-07-06T03:47:18.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.931+0000 7f9bf5dff700 1 -- 172.21.15.97:0/3363522450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9bf00b19c0 con 0x7f9bf00d2210 2023-07-06T03:47:18.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.931+0000 7f9bd7fff700 1 -- 172.21.15.97:0/3363522450 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9be4057920 con 0x7f9bf00d2210 2023-07-06T03:47:18.933 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:47:18.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.933+0000 7f9bf5dff700 1 -- 172.21.15.97:0/3363522450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bdc06ace0 msgr2=0x7f9bdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.933+0000 7f9bf5dff700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bdc06ace0 0x7f9bdc06d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9be0014040 tx=0x7f9be0010000 comp rx=0 tx=0).stop 2023-07-06T03:47:18.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.933+0000 7f9bf5dff700 1 -- 172.21.15.97:0/3363522450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d2210 msgr2=0x7f9bf00cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:18.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.933+0000 7f9bf5dff700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d2210 0x7f9bf00cb600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9be40011e0 tx=0x7f9be400cae0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.934+0000 7f9bf5dff700 1 -- 172.21.15.97:0/3363522450 shutdown_connections 2023-07-06T03:47:18.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.934+0000 7f9bf5dff700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bdc06ace0 0x7f9bdc06d190 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.934+0000 7f9bf5dff700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bf015c610 0x7f9bf00cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.934+0000 7f9bf5dff700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bf006b360 0x7f9bf01511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.934+0000 7f9bf5dff700 1 --2- 172.21.15.97:0/3363522450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf00d2210 0x7f9bf00cb600 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:18.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.934+0000 7f9bf5dff700 1 -- 172.21.15.97:0/3363522450 >> 172.21.15.97:0/3363522450 conn(0x7f9bf006a6c0 msgr2=0x7f9bf00b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:18.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.934+0000 7f9bf5dff700 1 -- 172.21.15.97:0/3363522450 shutdown_connections 2023-07-06T03:47:18.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:18.934+0000 7f9bf5dff700 1 -- 172.21.15.97:0/3363522450 wait complete. 2023-07-06T03:47:18.953 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:47:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.149+0000 7ff58c5ff700 1 Processor -- start 2023-07-06T03:47:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.149+0000 7ff58c5ff700 1 -- start start 2023-07-06T03:47:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.150+0000 7ff58c5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff58406b170 0x7ff58415d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.150+0000 7ff58c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58415cae0 0x7ff58406b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.150+0000 7ff58c5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5840cfbc0 0x7ff58415e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.150+0000 7ff58c5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5840b25a0 con 0x7ff5840cfbc0 2023-07-06T03:47:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.150+0000 7ff58c5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58415de00 con 0x7ff58406b170 2023-07-06T03:47:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.150+0000 7ff58c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58406ba80 con 0x7ff58415cae0 2023-07-06T03:47:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.151+0000 7ff58ab9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5840cfbc0 0x7ff58415e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.151+0000 7ff58ab9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5840cfbc0 0x7ff58415e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35026/0 (socket says 172.21.15.97:35026) 2023-07-06T03:47:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.151+0000 7ff58ab9c700 1 -- 172.21.15.97:0/2257272603 learned_addr learned my addr 172.21.15.97:0/2257272603 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.151+0000 7ff58a39b700 1 --2- 172.21.15.97:0/2257272603 >> v2:172.21.15.146:3300/0 conn(0x7ff58406b170 0x7ff58415d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.151+0000 7ff589b9a700 1 --2- 172.21.15.97:0/2257272603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58415cae0 0x7ff58406b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.151+0000 7ff58ab9c700 1 -- 172.21.15.97:0/2257272603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58415cae0 msgr2=0x7ff58406b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.151+0000 7ff58ab9c700 1 --2- 172.21.15.97:0/2257272603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58415cae0 0x7ff58406b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.151+0000 7ff58ab9c700 1 -- 172.21.15.97:0/2257272603 >> v2:172.21.15.146:3300/0 conn(0x7ff58406b170 msgr2=0x7ff58415d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.151+0000 7ff58ab9c700 1 --2- 172.21.15.97:0/2257272603 >> v2:172.21.15.146:3300/0 conn(0x7ff58406b170 0x7ff58415d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.151+0000 7ff58ab9c700 1 -- 172.21.15.97:0/2257272603 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5840ceb20 con 0x7ff5840cfbc0 2023-07-06T03:47:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.151+0000 7ff58ab9c700 1 --2- 172.21.15.97:0/2257272603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5840cfbc0 0x7ff58415e1b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff5840ce700 tx=0x7ff57c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a60df9bb2856b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:19.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff589399700 1 -- 172.21.15.97:0/2257272603 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00e070 con 0x7ff5840cfbc0 2023-07-06T03:47:19.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff589399700 1 -- 172.21.15.97:0/2257272603 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c00b9c0 con 0x7ff5840cfbc0 2023-07-06T03:47:19.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff589399700 1 -- 172.21.15.97:0/2257272603 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00bd70 con 0x7ff5840cfbc0 2023-07-06T03:47:19.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff58c5ff700 1 -- 172.21.15.97:0/2257272603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5840cfbc0 msgr2=0x7ff58415e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:19.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff58c5ff700 1 --2- 172.21.15.97:0/2257272603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5840cfbc0 0x7ff58415e1b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff5840ce700 tx=0x7ff57c00b330 comp rx=0 tx=0).stop 2023-07-06T03:47:19.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff58c5ff700 1 -- 172.21.15.97:0/2257272603 shutdown_connections 2023-07-06T03:47:19.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff58c5ff700 1 --2- 172.21.15.97:0/2257272603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5840cfbc0 0x7ff58415e1b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff58c5ff700 1 --2- 172.21.15.97:0/2257272603 >> v2:172.21.15.146:3300/0 conn(0x7ff58406b170 0x7ff58415d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff58c5ff700 1 --2- 172.21.15.97:0/2257272603 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58415cae0 0x7ff58406b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff58c5ff700 1 -- 172.21.15.97:0/2257272603 >> 172.21.15.97:0/2257272603 conn(0x7ff584099f10 msgr2=0x7ff5840d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:19.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff58c5ff700 1 -- 172.21.15.97:0/2257272603 shutdown_connections 2023-07-06T03:47:19.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.152+0000 7ff58c5ff700 1 -- 172.21.15.97:0/2257272603 wait complete. 2023-07-06T03:47:19.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.154+0000 7ff58c5ff700 1 Processor -- start 2023-07-06T03:47:19.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.154+0000 7ff58c5ff700 1 -- start start 2023-07-06T03:47:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.155+0000 7ff58c5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58406b170 0x7ff584153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.155+0000 7ff58c5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5840cfbc0 0x7ff58409b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.155+0000 7ff58c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58415cae0 0x7ff584154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.155+0000 7ff58c5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff584155610 con 0x7ff58406b170 2023-07-06T03:47:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.155+0000 7ff58c5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff584159c90 con 0x7ff5840cfbc0 2023-07-06T03:47:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.155+0000 7ff58c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58409bb20 con 0x7ff58415cae0 2023-07-06T03:47:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.155+0000 7ff58a39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58406b170 0x7ff584153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.155+0000 7ff58a39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58406b170 0x7ff584153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35030/0 (socket says 172.21.15.97:35030) 2023-07-06T03:47:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.155+0000 7ff58a39b700 1 -- 172.21.15.97:0/4114112891 learned_addr learned my addr 172.21.15.97:0/4114112891 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.155+0000 7ff589b9a700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5840cfbc0 0x7ff58409b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.155+0000 7ff58ab9c700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58415cae0 0x7ff584154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff58a39b700 1 -- 172.21.15.97:0/4114112891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58415cae0 msgr2=0x7ff584154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff58a39b700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58415cae0 0x7ff584154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff58a39b700 1 -- 172.21.15.97:0/4114112891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5840cfbc0 msgr2=0x7ff58409b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff58a39b700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5840cfbc0 0x7ff58409b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff58a39b700 1 -- 172.21.15.97:0/4114112891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c009b30 con 0x7ff58406b170 2023-07-06T03:47:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff589b9a700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5840cfbc0 0x7ff58409b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff58a39b700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58406b170 0x7ff584153da0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff58000e810 tx=0x7ff58000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff57affd700 1 -- 172.21.15.97:0/4114112891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580013610 con 0x7ff58406b170 2023-07-06T03:47:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff57affd700 1 -- 172.21.15.97:0/4114112891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff580013770 con 0x7ff58406b170 2023-07-06T03:47:19.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff57affd700 1 -- 172.21.15.97:0/4114112891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580013b00 con 0x7ff58406b170 2023-07-06T03:47:19.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff58c5ff700 1 -- 172.21.15.97:0/4114112891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58415b0b0 con 0x7ff58406b170 2023-07-06T03:47:19.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.156+0000 7ff58c5ff700 1 -- 172.21.15.97:0/4114112891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff584154f50 con 0x7ff58406b170 2023-07-06T03:47:19.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.157+0000 7ff58c5ff700 1 -- 172.21.15.97:0/4114112891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff584155880 con 0x7ff58406b170 2023-07-06T03:47:19.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.158+0000 7ff57affd700 1 -- 172.21.15.97:0/4114112891 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff580013c60 con 0x7ff58406b170 2023-07-06T03:47:19.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.159+0000 7ff57affd700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff56806ace0 0x7ff56806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.159+0000 7ff57affd700 1 -- 172.21.15.97:0/4114112891 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7ff5800890f0 con 0x7ff58406b170 2023-07-06T03:47:19.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.159+0000 7ff589b9a700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff56806ace0 0x7ff56806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.160+0000 7ff589b9a700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff56806ace0 0x7ff56806d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff58409b3d0 tx=0x7ff574008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:19.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.163+0000 7ff57affd700 1 -- 172.21.15.97:0/4114112891 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff580054130 con 0x7ff58406b170 2023-07-06T03:47:19.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.361+0000 7ff58c5ff700 1 -- 172.21.15.97:0/4114112891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff584154db0 con 0x7ff58406b170 2023-07-06T03:47:19.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.361+0000 7ff57affd700 1 -- 172.21.15.97:0/4114112891 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff584154db0 con 0x7ff58406b170 2023-07-06T03:47:19.363 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:47:19.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 -- 172.21.15.97:0/4114112891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff56806ace0 msgr2=0x7ff56806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:19.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff56806ace0 0x7ff56806d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff58409b3d0 tx=0x7ff574008040 comp rx=0 tx=0).stop 2023-07-06T03:47:19.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 -- 172.21.15.97:0/4114112891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58406b170 msgr2=0x7ff584153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:19.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58406b170 0x7ff584153da0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff58000e810 tx=0x7ff58000ebd0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 -- 172.21.15.97:0/4114112891 shutdown_connections 2023-07-06T03:47:19.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff56806ace0 0x7ff56806d190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58415cae0 0x7ff584154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58406b170 0x7ff584153da0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 --2- 172.21.15.97:0/4114112891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5840cfbc0 0x7ff58409b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 -- 172.21.15.97:0/4114112891 >> 172.21.15.97:0/4114112891 conn(0x7ff584099f10 msgr2=0x7ff5840d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:19.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 -- 172.21.15.97:0/4114112891 shutdown_connections 2023-07-06T03:47:19.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.364+0000 7ff58c5ff700 1 -- 172.21.15.97:0/4114112891 wait complete. 2023-07-06T03:47:19.384 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:47:19.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.579+0000 7fe4c9dff700 1 Processor -- start 2023-07-06T03:47:19.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.579+0000 7fe4c9dff700 1 -- start start 2023-07-06T03:47:19.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.580+0000 7fe4c9dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4c406b360 0x7fe4c40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4c9dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4c40d2210 0x7fe4c40d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4c9dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c40cff90 con 0x7fe4c415c610 2023-07-06T03:47:19.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4c9dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c40d0780 con 0x7fe4c40d2210 2023-07-06T03:47:19.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4c9dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c415c9e0 con 0x7fe4c406b360 2023-07-06T03:47:19.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4c3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4c40d2210 0x7fe4c40d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4c37fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe4c415c610 0x7fe4c40b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41882/0 (socket says 172.21.15.97:41882) 2023-07-06T03:47:19.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4c37fe700 1 -- 172.21.15.97:0/2035318027 learned_addr learned my addr 172.21.15.97:0/2035318027 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:19.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4bbfff700 1 --2- 172.21.15.97:0/2035318027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4c406b360 0x7fe4c40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4c3fff700 1 -- 172.21.15.97:0/2035318027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4c406b360 msgr2=0x7fe4c40b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:19.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4c3fff700 1 --2- 172.21.15.97:0/2035318027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4c406b360 0x7fe4c40b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4c3fff700 1 -- 172.21.15.97:0/2035318027 >> v1:172.21.15.97:6789/0 conn(0x7fe4c415c610 legacy=0x7fe4c40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:19.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.581+0000 7fe4c3fff700 1 -- 172.21.15.97:0/2035318027 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c40b5650 con 0x7fe4c40d2210 2023-07-06T03:47:19.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.582+0000 7fe4c3fff700 1 --2- 172.21.15.97:0/2035318027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4c40d2210 0x7fe4c40d1310 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009dc0 tx=0x7fe4bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f813fe39f0e33d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:19.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.582+0000 7fe4c2ffd700 1 -- 172.21.15.97:0/2035318027 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00e070 con 0x7fe4c40d2210 2023-07-06T03:47:19.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.582+0000 7fe4c2ffd700 1 -- 172.21.15.97:0/2035318027 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc00bae0 con 0x7fe4c40d2210 2023-07-06T03:47:19.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.582+0000 7fe4c2ffd700 1 -- 172.21.15.97:0/2035318027 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00be90 con 0x7fe4c40d2210 2023-07-06T03:47:19.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.582+0000 7fe4c9dff700 1 -- 172.21.15.97:0/2035318027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4c40d2210 msgr2=0x7fe4c40d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:19.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.582+0000 7fe4c9dff700 1 --2- 172.21.15.97:0/2035318027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4c40d2210 0x7fe4c40d1310 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009dc0 tx=0x7fe4bc00b330 comp rx=0 tx=0).stop 2023-07-06T03:47:19.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.582+0000 7fe4c9dff700 1 -- 172.21.15.97:0/2035318027 shutdown_connections 2023-07-06T03:47:19.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.582+0000 7fe4c9dff700 1 --2- 172.21.15.97:0/2035318027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4c40d2210 0x7fe4c40d1310 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.582+0000 7fe4c9dff700 1 --2- 172.21.15.97:0/2035318027 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4c406b360 0x7fe4c40b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.582+0000 7fe4c9dff700 1 -- 172.21.15.97:0/2035318027 >> 172.21.15.97:0/2035318027 conn(0x7fe4c406a6c0 msgr2=0x7fe4c406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:19.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.582+0000 7fe4c9dff700 1 -- 172.21.15.97:0/2035318027 shutdown_connections 2023-07-06T03:47:19.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.583+0000 7fe4c9dff700 1 -- 172.21.15.97:0/2035318027 wait complete. 2023-07-06T03:47:19.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.584+0000 7fe4c9dff700 1 Processor -- start 2023-07-06T03:47:19.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.584+0000 7fe4c9dff700 1 -- start start 2023-07-06T03:47:19.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.585+0000 7fe4c9dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c406b360 0x7fe4c41f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.585+0000 7fe4c9dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4c40d2210 0x7fe4c40c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.585+0000 7fe4c9dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4c415c610 0x7fe4c40c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.585+0000 7fe4c9dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c40cbad0 con 0x7fe4c40d2210 2023-07-06T03:47:19.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.585+0000 7fe4c9dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c40cbdb0 con 0x7fe4c406b360 2023-07-06T03:47:19.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.585+0000 7fe4c9dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c40cc090 con 0x7fe4c415c610 2023-07-06T03:47:19.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.585+0000 7fe4bbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4c40d2210 0x7fe4c40c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.585+0000 7fe4bbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4c40d2210 0x7fe4c40c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35056/0 (socket says 172.21.15.97:35056) 2023-07-06T03:47:19.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.585+0000 7fe4bbfff700 1 -- 172.21.15.97:0/631099006 learned_addr learned my addr 172.21.15.97:0/631099006 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:19.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4c37fe700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c406b360 0x7fe4c41f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4c3fff700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4c415c610 0x7fe4c40c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4bbfff700 1 -- 172.21.15.97:0/631099006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4c415c610 msgr2=0x7fe4c40c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:19.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4bbfff700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4c415c610 0x7fe4c40c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4bbfff700 1 -- 172.21.15.97:0/631099006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c406b360 msgr2=0x7fe4c41f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:19.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4bbfff700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c406b360 0x7fe4c41f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4bbfff700 1 -- 172.21.15.97:0/631099006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc009a70 con 0x7fe4c40d2210 2023-07-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4c37fe700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c406b360 0x7fe4c41f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4bbfff700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4c40d2210 0x7fe4c40c6860 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00e980 tx=0x7fe4ac00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4c0ff9700 1 -- 172.21.15.97:0/631099006 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac009710 con 0x7fe4c40d2210 2023-07-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4c0ff9700 1 -- 172.21.15.97:0/631099006 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac00c850 con 0x7fe4c40d2210 2023-07-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4c9dff700 1 -- 172.21.15.97:0/631099006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c40cc370 con 0x7fe4c40d2210 2023-07-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4c0ff9700 1 -- 172.21.15.97:0/631099006 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac00cc00 con 0x7fe4c40d2210 2023-07-06T03:47:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.586+0000 7fe4c9dff700 1 -- 172.21.15.97:0/631099006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c41f1dc0 con 0x7fe4c40d2210 2023-07-06T03:47:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.588+0000 7fe4c9dff700 1 -- 172.21.15.97:0/631099006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c41f1f50 con 0x7fe4c40d2210 2023-07-06T03:47:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.588+0000 7fe4c0ff9700 1 -- 172.21.15.97:0/631099006 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe4ac013470 con 0x7fe4c40d2210 2023-07-06T03:47:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.590+0000 7fe4c0ff9700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4a406ace0 0x7fe4a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.590+0000 7fe4c0ff9700 1 -- 172.21.15.97:0/631099006 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fe4ac089680 con 0x7fe4c40d2210 2023-07-06T03:47:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.590+0000 7fe4c37fe700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4a406ace0 0x7fe4a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.591+0000 7fe4c37fe700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4a406ace0 0x7fe4a406d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4007fd0 tx=0x7fe4b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.594+0000 7fe4c0ff9700 1 -- 172.21.15.97:0/631099006 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe4ac058190 con 0x7fe4c40d2210 2023-07-06T03:47:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.788+0000 7fe4c9dff700 1 -- 172.21.15.97:0/631099006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe4c404e340 con 0x7fe4c40d2210 2023-07-06T03:47:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.788+0000 7fe4c0ff9700 1 -- 172.21.15.97:0/631099006 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe4ac057d20 con 0x7fe4c40d2210 2023-07-06T03:47:19.790 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:47:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.790+0000 7fe4c9dff700 1 -- 172.21.15.97:0/631099006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4a406ace0 msgr2=0x7fe4a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.790+0000 7fe4c9dff700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4a406ace0 0x7fe4a406d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4007fd0 tx=0x7fe4b4009040 comp rx=0 tx=0).stop 2023-07-06T03:47:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.790+0000 7fe4c9dff700 1 -- 172.21.15.97:0/631099006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4c40d2210 msgr2=0x7fe4c40c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.790+0000 7fe4c9dff700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4c40d2210 0x7fe4c40c6860 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00e980 tx=0x7fe4ac00ed40 comp rx=0 tx=0).stop 2023-07-06T03:47:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.790+0000 7fe4c9dff700 1 -- 172.21.15.97:0/631099006 shutdown_connections 2023-07-06T03:47:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.790+0000 7fe4c9dff700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe4a406ace0 0x7fe4a406d190 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.790+0000 7fe4c9dff700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4c415c610 0x7fe4c40c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.790+0000 7fe4c9dff700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c406b360 0x7fe4c41f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.790+0000 7fe4c9dff700 1 --2- 172.21.15.97:0/631099006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4c40d2210 0x7fe4c40c6860 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.790+0000 7fe4c9dff700 1 -- 172.21.15.97:0/631099006 >> 172.21.15.97:0/631099006 conn(0x7fe4c406a6c0 msgr2=0x7fe4c40d2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.791+0000 7fe4c9dff700 1 -- 172.21.15.97:0/631099006 shutdown_connections 2023-07-06T03:47:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:19.791+0000 7fe4c9dff700 1 -- 172.21.15.97:0/631099006 wait complete. 2023-07-06T03:47:19.810 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:47:20.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.006+0000 7f91be5ff700 1 Processor -- start 2023-07-06T03:47:20.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.006+0000 7f91be5ff700 1 -- start start 2023-07-06T03:47:20.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.007+0000 7f91be5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f91b815c610 0x7f91b80b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.007+0000 7f91be5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91b806b360 0x7f91b80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.007+0000 7f91be5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91b80d2210 0x7f91b80d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.007+0000 7f91be5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b80d0780 con 0x7f91b806b360 2023-07-06T03:47:20.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.007+0000 7f91be5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b815c9e0 con 0x7f91b815c610 2023-07-06T03:47:20.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.007+0000 7f91be5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80b5650 con 0x7f91b80d2210 2023-07-06T03:47:20.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91b77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91b806b360 0x7f91b80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91b77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91b806b360 0x7f91b80b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51492/0 (socket says 172.21.15.97:51492) 2023-07-06T03:47:20.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91b77fe700 1 -- 172.21.15.97:0/3763056962 learned_addr learned my addr 172.21.15.97:0/3763056962 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:20.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91b7fff700 1 --2- 172.21.15.97:0/3763056962 >> v2:172.21.15.146:3300/0 conn(0x7f91b815c610 0x7f91b80b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91bcb9c700 1 --2- 172.21.15.97:0/3763056962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91b80d2210 0x7f91b80d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91b77fe700 1 -- 172.21.15.97:0/3763056962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91b80d2210 msgr2=0x7f91b80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:20.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91b77fe700 1 --2- 172.21.15.97:0/3763056962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91b80d2210 0x7f91b80d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91b77fe700 1 -- 172.21.15.97:0/3763056962 >> v2:172.21.15.146:3300/0 conn(0x7f91b815c610 msgr2=0x7f91b80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:20.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91b77fe700 1 --2- 172.21.15.97:0/3763056962 >> v2:172.21.15.146:3300/0 conn(0x7f91b815c610 0x7f91b80b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91b77fe700 1 -- 172.21.15.97:0/3763056962 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b80d01c0 con 0x7f91b806b360 2023-07-06T03:47:20.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91bcb9c700 1 --2- 172.21.15.97:0/3763056962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91b80d2210 0x7f91b80d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:20.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.008+0000 7f91b77fe700 1 --2- 172.21.15.97:0/3763056962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b806b360 0x7f91b80b4cf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f91b808b000 tx=0x7f91ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1a24ef4a77a86c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:20.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91b6ffd700 1 -- 172.21.15.97:0/3763056962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac011830 con 0x7f91b806b360 2023-07-06T03:47:20.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91b6ffd700 1 -- 172.21.15.97:0/3763056962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac011990 con 0x7f91b806b360 2023-07-06T03:47:20.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91b6ffd700 1 -- 172.21.15.97:0/3763056962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac011d20 con 0x7f91b806b360 2023-07-06T03:47:20.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91be5ff700 1 -- 172.21.15.97:0/3763056962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b806b360 msgr2=0x7f91b80b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91be5ff700 1 --2- 172.21.15.97:0/3763056962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b806b360 0x7f91b80b4cf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f91b808b000 tx=0x7f91ac009c00 comp rx=0 tx=0).stop 2023-07-06T03:47:20.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91be5ff700 1 -- 172.21.15.97:0/3763056962 shutdown_connections 2023-07-06T03:47:20.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91be5ff700 1 --2- 172.21.15.97:0/3763056962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91b80d2210 0x7f91b80d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91be5ff700 1 --2- 172.21.15.97:0/3763056962 >> v2:172.21.15.146:3300/0 conn(0x7f91b815c610 0x7f91b80b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91be5ff700 1 --2- 172.21.15.97:0/3763056962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b806b360 0x7f91b80b4cf0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91be5ff700 1 -- 172.21.15.97:0/3763056962 >> 172.21.15.97:0/3763056962 conn(0x7f91b806a6c0 msgr2=0x7f91b806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:20.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91be5ff700 1 -- 172.21.15.97:0/3763056962 shutdown_connections 2023-07-06T03:47:20.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.009+0000 7f91be5ff700 1 -- 172.21.15.97:0/3763056962 wait complete. 2023-07-06T03:47:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.011+0000 7f91be5ff700 1 Processor -- start 2023-07-06T03:47:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.011+0000 7f91be5ff700 1 -- start start 2023-07-06T03:47:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.012+0000 7f91be5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91b806b360 0x7f91b815e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.012+0000 7f91be5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91b80d2210 0x7f91b815f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.012+0000 7f91be5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b815c610 0x7f91b814eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.012+0000 7f91be5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b815f9b0 con 0x7f91b815c610 2023-07-06T03:47:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.012+0000 7f91be5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80cb010 con 0x7f91b806b360 2023-07-06T03:47:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.012+0000 7f91be5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b815f100 con 0x7f91b80d2210 2023-07-06T03:47:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.012+0000 7f91bcb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b815c610 0x7f91b814eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91bcb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b815c610 0x7f91b814eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51498/0 (socket says 172.21.15.97:51498) 2023-07-06T03:47:20.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91bcb9c700 1 -- 172.21.15.97:0/355258934 learned_addr learned my addr 172.21.15.97:0/355258934 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91b7fff700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91b806b360 0x7f91b815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91b77fe700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91b80d2210 0x7f91b815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91bcb9c700 1 -- 172.21.15.97:0/355258934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91b80d2210 msgr2=0x7f91b815f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91bcb9c700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91b80d2210 0x7f91b815f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91bcb9c700 1 -- 172.21.15.97:0/355258934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91b806b360 msgr2=0x7f91b815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91bcb9c700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91b806b360 0x7f91b815e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91bcb9c700 1 -- 172.21.15.97:0/355258934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac009550 con 0x7f91b815c610 2023-07-06T03:47:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91bcb9c700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b815c610 0x7f91b814eb10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f91b0007e10 tx=0x7f91b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91b4ff9700 1 -- 172.21.15.97:0/355258934 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b000de90 con 0x7f91b815c610 2023-07-06T03:47:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91b4ff9700 1 -- 172.21.15.97:0/355258934 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b000f040 con 0x7f91b815c610 2023-07-06T03:47:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91b4ff9700 1 -- 172.21.15.97:0/355258934 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0003c60 con 0x7f91b815c610 2023-07-06T03:47:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.013+0000 7f91be5ff700 1 -- 172.21.15.97:0/355258934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b815e860 con 0x7f91b815c610 2023-07-06T03:47:20.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.014+0000 7f91be5ff700 1 -- 172.21.15.97:0/355258934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b815ee20 con 0x7f91b815c610 2023-07-06T03:47:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.015+0000 7f91be5ff700 1 -- 172.21.15.97:0/355258934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b804e340 con 0x7f91b815c610 2023-07-06T03:47:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.016+0000 7f91b4ff9700 1 -- 172.21.15.97:0/355258934 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f91b0004030 con 0x7f91b815c610 2023-07-06T03:47:20.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.017+0000 7f91b4ff9700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91a406ace0 0x7f91a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.017+0000 7f91b4ff9700 1 -- 172.21.15.97:0/355258934 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f91b0089680 con 0x7f91b815c610 2023-07-06T03:47:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.017+0000 7f91b7fff700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91a406ace0 0x7f91a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.018+0000 7f91b7fff700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91a406ace0 0x7f91a406d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009ad0 tx=0x7f91a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:20.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.021+0000 7f91b4ff9700 1 -- 172.21.15.97:0/355258934 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f91b00546c0 con 0x7f91b815c610 2023-07-06T03:47:20.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.213+0000 7f91be5ff700 1 -- 172.21.15.97:0/355258934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91b80cb150 con 0x7f91b815c610 2023-07-06T03:47:20.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.213+0000 7f91b4ff9700 1 -- 172.21.15.97:0/355258934 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91b0057ce0 con 0x7f91b815c610 2023-07-06T03:47:20.215 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:47:20.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 -- 172.21.15.97:0/355258934 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91a406ace0 msgr2=0x7f91a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91a406ace0 0x7f91a406d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009ad0 tx=0x7f91a8008040 comp rx=0 tx=0).stop 2023-07-06T03:47:20.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 -- 172.21.15.97:0/355258934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b815c610 msgr2=0x7f91b814eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b815c610 0x7f91b814eb10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f91b0007e10 tx=0x7f91b000d3b0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 -- 172.21.15.97:0/355258934 shutdown_connections 2023-07-06T03:47:20.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91a406ace0 0x7f91a406d190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91b815c610 0x7f91b814eb10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91b806b360 0x7f91b815e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 --2- 172.21.15.97:0/355258934 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91b80d2210 0x7f91b815f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 -- 172.21.15.97:0/355258934 >> 172.21.15.97:0/355258934 conn(0x7f91b806a6c0 msgr2=0x7f91b8159170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:20.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 -- 172.21.15.97:0/355258934 shutdown_connections 2023-07-06T03:47:20.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.216+0000 7f91be5ff700 1 -- 172.21.15.97:0/355258934 wait complete. 2023-07-06T03:47:20.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:47:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.429+0000 7fd3583ff700 1 Processor -- start 2023-07-06T03:47:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.429+0000 7fd3583ff700 1 -- start start 2023-07-06T03:47:20.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd3583ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd35015c520 0x7fd3500cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd3583ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd350160180 0x7fd3501605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd3583ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3500cdae0 con 0x7fd350160b30 2023-07-06T03:47:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd3583ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3500cd8d0 con 0x7fd350160180 2023-07-06T03:47:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd3583ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3500b6640 con 0x7fd35015c520 2023-07-06T03:47:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd35599a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd350160180 0x7fd3501605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd35599a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd350160180 0x7fd3501605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51500/0 (socket says 172.21.15.97:51500) 2023-07-06T03:47:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd35599a700 1 -- 172.21.15.97:0/3853566162 learned_addr learned my addr 172.21.15.97:0/3853566162 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd35619b700 1 --2- 172.21.15.97:0/3853566162 >> v2:172.21.15.146:3300/0 conn(0x7fd35015c520 0x7fd3500cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd35599a700 1 -- 172.21.15.97:0/3853566162 >> v2:172.21.15.146:3300/0 conn(0x7fd35015c520 msgr2=0x7fd3500cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:20.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd35599a700 1 --2- 172.21.15.97:0/3853566162 >> v2:172.21.15.146:3300/0 conn(0x7fd35015c520 0x7fd3500cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd35599a700 1 -- 172.21.15.97:0/3853566162 >> v1:172.21.15.146:6789/0 conn(0x7fd350160b30 legacy=0x7fd3500cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.431+0000 7fd35599a700 1 -- 172.21.15.97:0/3853566162 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3500b6920 con 0x7fd350160180 2023-07-06T03:47:20.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.432+0000 7fd35599a700 1 --2- 172.21.15.97:0/3853566162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd350160180 0x7fd3501605f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd3500b4c70 tx=0x7fd340009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db03a294ab42af37 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:20.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.432+0000 7fd355199700 1 -- 172.21.15.97:0/3853566162 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd340011830 con 0x7fd350160180 2023-07-06T03:47:20.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.432+0000 7fd355199700 1 -- 172.21.15.97:0/3853566162 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd340011990 con 0x7fd350160180 2023-07-06T03:47:20.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.432+0000 7fd355199700 1 -- 172.21.15.97:0/3853566162 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd340011d20 con 0x7fd350160180 2023-07-06T03:47:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.432+0000 7fd3583ff700 1 -- 172.21.15.97:0/3853566162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd350160180 msgr2=0x7fd3501605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.432+0000 7fd3583ff700 1 --2- 172.21.15.97:0/3853566162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd350160180 0x7fd3501605f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd3500b4c70 tx=0x7fd340009bd0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.433+0000 7fd3583ff700 1 -- 172.21.15.97:0/3853566162 shutdown_connections 2023-07-06T03:47:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.433+0000 7fd3583ff700 1 --2- 172.21.15.97:0/3853566162 >> v2:172.21.15.146:3300/0 conn(0x7fd35015c520 0x7fd3500cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.433+0000 7fd3583ff700 1 --2- 172.21.15.97:0/3853566162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd350160180 0x7fd3501605f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.433+0000 7fd3583ff700 1 -- 172.21.15.97:0/3853566162 >> 172.21.15.97:0/3853566162 conn(0x7fd350071790 msgr2=0x7fd350071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.433+0000 7fd3583ff700 1 -- 172.21.15.97:0/3853566162 shutdown_connections 2023-07-06T03:47:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.433+0000 7fd3583ff700 1 -- 172.21.15.97:0/3853566162 wait complete. 2023-07-06T03:47:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.434+0000 7fd3583ff700 1 Processor -- start 2023-07-06T03:47:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.435+0000 7fd3583ff700 1 -- start start 2023-07-06T03:47:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.435+0000 7fd3583ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd35015c520 0x7fd350155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd3583ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350160180 0x7fd35015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd3583ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd350160b30 0x7fd35015cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd3583ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35015f700 con 0x7fd350160b30 2023-07-06T03:47:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd3583ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3500ca120 con 0x7fd350160180 2023-07-06T03:47:20.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd3583ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd350156520 con 0x7fd35015c520 2023-07-06T03:47:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd35699c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd350160b30 0x7fd35015cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd35619b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd35015c520 0x7fd350155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd35599a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350160180 0x7fd35015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd35699c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd350160b30 0x7fd35015cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51504/0 (socket says 172.21.15.97:51504) 2023-07-06T03:47:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd35619b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd35015c520 0x7fd350155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:45940/0 (socket says 172.21.15.97:45940) 2023-07-06T03:47:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd35699c700 1 -- 172.21.15.97:0/2895242738 learned_addr learned my addr 172.21.15.97:0/2895242738 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd35699c700 1 -- 172.21.15.97:0/2895242738 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd35015c520 msgr2=0x7fd350155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd35699c700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd35015c520 0x7fd350155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd35699c700 1 -- 172.21.15.97:0/2895242738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350160180 msgr2=0x7fd35015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd35699c700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350160180 0x7fd35015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.436+0000 7fd35699c700 1 -- 172.21.15.97:0/2895242738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd340009550 con 0x7fd350160b30 2023-07-06T03:47:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.437+0000 7fd35699c700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd350160b30 0x7fd35015cf50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd34800cc50 tx=0x7fd34800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.437+0000 7fd346ffd700 1 -- 172.21.15.97:0/2895242738 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348003c00 con 0x7fd350160b30 2023-07-06T03:47:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.437+0000 7fd346ffd700 1 -- 172.21.15.97:0/2895242738 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348003d60 con 0x7fd350160b30 2023-07-06T03:47:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.437+0000 7fd3583ff700 1 -- 172.21.15.97:0/2895242738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35015eaf0 con 0x7fd350160b30 2023-07-06T03:47:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.437+0000 7fd346ffd700 1 -- 172.21.15.97:0/2895242738 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34800aa50 con 0x7fd350160b30 2023-07-06T03:47:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.437+0000 7fd3583ff700 1 -- 172.21.15.97:0/2895242738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35015f040 con 0x7fd350160b30 2023-07-06T03:47:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.438+0000 7fd3583ff700 1 -- 172.21.15.97:0/2895242738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35015f840 con 0x7fd350160b30 2023-07-06T03:47:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.439+0000 7fd346ffd700 1 -- 172.21.15.97:0/2895242738 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd34800abb0 con 0x7fd350160b30 2023-07-06T03:47:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.440+0000 7fd346ffd700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd33406ace0 0x7fd33406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.440+0000 7fd346ffd700 1 -- 172.21.15.97:0/2895242738 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fd3480893c0 con 0x7fd350160b30 2023-07-06T03:47:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.440+0000 7fd35619b700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd33406ace0 0x7fd33406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.441+0000 7fd35619b700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd33406ace0 0x7fd33406d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009b40 tx=0x7fd34c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:20.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.444+0000 7fd346ffd700 1 -- 172.21.15.97:0/2895242738 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd35015f840 con 0x7fd350160b30 2023-07-06T03:47:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.679+0000 7fd3583ff700 1 -- 172.21.15.97:0/2895242738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd35015f840 con 0x7fd350160b30 2023-07-06T03:47:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.679+0000 7fd346ffd700 1 -- 172.21.15.97:0/2895242738 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fd35015f840 con 0x7fd350160b30 2023-07-06T03:47:20.681 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:20.681 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/1839628004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1839628004},{"type":"v1","addr":"172.21.15.146:6834","nonce":1839628004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4937,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2131262285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2131262285},{"type":"v1","addr":"172.21.15.146:6835","nonce":2131262285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1367937004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367937004},{"type":"v1","addr":"172.21.15.97:6835","nonce":1367937004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:46:37.331190+0000","modified":"2023-07-06T03:46:40.864523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/663657264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":663657264},{"type":"v1","addr":"172.21.15.97:6837","nonce":663657264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T03:47:20.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 -- 172.21.15.97:0/2895242738 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd33406ace0 msgr2=0x7fd33406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd33406ace0 0x7fd33406d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009b40 tx=0x7fd34c009210 comp rx=0 tx=0).stop 2023-07-06T03:47:20.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 -- 172.21.15.97:0/2895242738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd350160b30 msgr2=0x7fd35015cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd350160b30 0x7fd35015cf50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd34800cc50 tx=0x7fd34800cf60 comp rx=0 tx=0).stop 2023-07-06T03:47:20.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 -- 172.21.15.97:0/2895242738 shutdown_connections 2023-07-06T03:47:20.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd33406ace0 0x7fd33406d190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd350160b30 0x7fd35015cf50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd35015c520 0x7fd350155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 --2- 172.21.15.97:0/2895242738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd350160180 0x7fd35015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 -- 172.21.15.97:0/2895242738 >> 172.21.15.97:0/2895242738 conn(0x7fd350071790 msgr2=0x7fd3500b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 -- 172.21.15.97:0/2895242738 shutdown_connections 2023-07-06T03:47:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.682+0000 7fd3583ff700 1 -- 172.21.15.97:0/2895242738 wait complete. 2023-07-06T03:47:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2023-07-06T03:47:20.702 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:47:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.896+0000 7f2ef8fff700 1 Processor -- start 2023-07-06T03:47:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.897+0000 7f2ef8fff700 1 -- start start 2023-07-06T03:47:20.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2ef8fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef415c520 0x7f2ef40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2ef8fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ef4160180 0x7f2ef41605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2ef8fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ef4160b30 0x7f2ef40cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2ef8fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef40cd8d0 con 0x7f2ef4160180 2023-07-06T03:47:20.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2ef8fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef40cdae0 con 0x7f2ef4160b30 2023-07-06T03:47:20.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2ef8fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40b6640 con 0x7f2ef415c520 2023-07-06T03:47:20.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2eebfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ef4160180 0x7f2ef41605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2eebfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ef4160180 0x7f2ef41605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51506/0 (socket says 172.21.15.97:51506) 2023-07-06T03:47:20.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2eebfff700 1 -- 172.21.15.97:0/2306085144 learned_addr learned my addr 172.21.15.97:0/2306085144 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:20.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2eebfff700 1 -- 172.21.15.97:0/2306085144 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef415c520 msgr2=0x7f2ef40cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:20.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2eebfff700 1 --2- 172.21.15.97:0/2306085144 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef415c520 0x7f2ef40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2eebfff700 1 -- 172.21.15.97:0/2306085144 >> v2:172.21.15.146:3300/0 conn(0x7f2ef4160b30 msgr2=0x7f2ef40cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:20.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2eebfff700 1 --2- 172.21.15.97:0/2306085144 >> v2:172.21.15.146:3300/0 conn(0x7f2ef4160b30 0x7f2ef40cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.898+0000 7f2eebfff700 1 -- 172.21.15.97:0/2306085144 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef40b6920 con 0x7f2ef4160180 2023-07-06T03:47:20.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2eebfff700 1 --2- 172.21.15.97:0/2306085144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef4160180 0x7f2ef41605f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2ef408bfc0 tx=0x7f2edc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b0edcfd6d84542 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:20.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef1d9b700 1 -- 172.21.15.97:0/2306085144 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00b870 con 0x7f2ef4160180 2023-07-06T03:47:20.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef1d9b700 1 -- 172.21.15.97:0/2306085144 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc00b9d0 con 0x7f2ef4160180 2023-07-06T03:47:20.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef1d9b700 1 -- 172.21.15.97:0/2306085144 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00bd60 con 0x7f2ef4160180 2023-07-06T03:47:20.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef8fff700 1 -- 172.21.15.97:0/2306085144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef4160180 msgr2=0x7f2ef41605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef8fff700 1 --2- 172.21.15.97:0/2306085144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef4160180 0x7f2ef41605f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2ef408bfc0 tx=0x7f2edc009bd0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef8fff700 1 -- 172.21.15.97:0/2306085144 shutdown_connections 2023-07-06T03:47:20.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef8fff700 1 --2- 172.21.15.97:0/2306085144 >> v2:172.21.15.146:3300/0 conn(0x7f2ef4160b30 0x7f2ef40cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef8fff700 1 --2- 172.21.15.97:0/2306085144 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef415c520 0x7f2ef40cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef8fff700 1 --2- 172.21.15.97:0/2306085144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef4160180 0x7f2ef41605f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef8fff700 1 -- 172.21.15.97:0/2306085144 >> 172.21.15.97:0/2306085144 conn(0x7f2ef4071790 msgr2=0x7f2ef4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef8fff700 1 -- 172.21.15.97:0/2306085144 shutdown_connections 2023-07-06T03:47:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.899+0000 7f2ef8fff700 1 -- 172.21.15.97:0/2306085144 wait complete. 2023-07-06T03:47:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.901+0000 7f2ef8fff700 1 Processor -- start 2023-07-06T03:47:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.901+0000 7f2ef8fff700 1 -- start start 2023-07-06T03:47:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.902+0000 7f2ef8fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ef415c520 0x7f2ef41e8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.902+0000 7f2ef8fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef4160180 0x7f2ef40cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.902+0000 7f2ef8fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef4160b30 0x7f2ef41e7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.902+0000 7f2ef8fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef41e8600 con 0x7f2ef4160b30 2023-07-06T03:47:20.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.902+0000 7f2ef8fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40c9410 con 0x7f2ef415c520 2023-07-06T03:47:20.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.902+0000 7f2ef8fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef41e7980 con 0x7f2ef4160180 2023-07-06T03:47:20.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.902+0000 7f2ef2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef4160b30 0x7f2ef41e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2eebfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef4160180 0x7f2ef40cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2ef2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef4160b30 0x7f2ef41e7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51512/0 (socket says 172.21.15.97:51512) 2023-07-06T03:47:20.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2ef2d9d700 1 -- 172.21.15.97:0/168474552 learned_addr learned my addr 172.21.15.97:0/168474552 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:20.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2ef259c700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ef415c520 0x7f2ef41e8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2ef2d9d700 1 -- 172.21.15.97:0/168474552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef4160180 msgr2=0x7f2ef40cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2ef2d9d700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef4160180 0x7f2ef40cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2ef2d9d700 1 -- 172.21.15.97:0/168474552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ef415c520 msgr2=0x7f2ef41e8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2ef2d9d700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ef415c520 0x7f2ef41e8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2ef2d9d700 1 -- 172.21.15.97:0/168474552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc009550 con 0x7f2ef4160b30 2023-07-06T03:47:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2ef2d9d700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef4160b30 0x7f2ef41e7110 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b970 tx=0x7f2eec00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2eeaffd700 1 -- 172.21.15.97:0/168474552 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec00f860 con 0x7f2ef4160b30 2023-07-06T03:47:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2eeaffd700 1 -- 172.21.15.97:0/168474552 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec00f9e0 con 0x7f2ef4160b30 2023-07-06T03:47:20.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2ef8fff700 1 -- 172.21.15.97:0/168474552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef41e7c60 con 0x7f2ef4160b30 2023-07-06T03:47:20.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.903+0000 7f2eeaffd700 1 -- 172.21.15.97:0/168474552 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec00fd90 con 0x7f2ef4160b30 2023-07-06T03:47:20.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.904+0000 7f2ef8fff700 1 -- 172.21.15.97:0/168474552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef41e8220 con 0x7f2ef4160b30 2023-07-06T03:47:20.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.904+0000 7f2ef8fff700 1 -- 172.21.15.97:0/168474552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef404e340 con 0x7f2ef4160b30 2023-07-06T03:47:20.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.905+0000 7f2eeaffd700 1 -- 172.21.15.97:0/168474552 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2eec00d3f0 con 0x7f2ef4160b30 2023-07-06T03:47:20.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.906+0000 7f2eeaffd700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed406ace0 0x7f2ed406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.906+0000 7f2eeaffd700 1 -- 172.21.15.97:0/168474552 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f2eec0898e0 con 0x7f2ef4160b30 2023-07-06T03:47:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.907+0000 7f2ef259c700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed406ace0 0x7f2ed406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.907+0000 7f2ef259c700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed406ace0 0x7f2ed406d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009a10 tx=0x7f2ee4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:20.911+0000 7f2eeaffd700 1 -- 172.21.15.97:0/168474552 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2eec054920 con 0x7f2ef4160b30 2023-07-06T03:47:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.109+0000 7f2ef8fff700 1 -- 172.21.15.97:0/168474552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ef41598d0 con 0x7f2ef4160b30 2023-07-06T03:47:21.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.110+0000 7f2eeaffd700 1 -- 172.21.15.97:0/168474552 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18460 (secure 0 0 0) 0x7f2eec057f40 con 0x7f2ef4160b30 2023-07-06T03:47:21.112 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:21.113 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:46:52.661343+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:46:35.584013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T03:46:36.303248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.28","mappings":[{"from":4,"to":7}]},{"pgid":"6.36","mappings":[{"from":1,"to":2}]},{"pgid":"6.38","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:47:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 -- 172.21.15.97:0/168474552 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed406ace0 msgr2=0x7f2ed406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed406ace0 0x7f2ed406d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009a10 tx=0x7f2ee4009190 comp rx=0 tx=0).stop 2023-07-06T03:47:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 -- 172.21.15.97:0/168474552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef4160b30 msgr2=0x7f2ef41e7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef4160b30 0x7f2ef41e7110 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b970 tx=0x7f2eec00bc80 comp rx=0 tx=0).stop 2023-07-06T03:47:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 -- 172.21.15.97:0/168474552 shutdown_connections 2023-07-06T03:47:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed406ace0 0x7f2ed406d190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef4160b30 0x7f2ef41e7110 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ef415c520 0x7f2ef41e8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 --2- 172.21.15.97:0/168474552 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef4160180 0x7f2ef40cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 -- 172.21.15.97:0/168474552 >> 172.21.15.97:0/168474552 conn(0x7f2ef4071790 msgr2=0x7f2ef40b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 -- 172.21.15.97:0/168474552 shutdown_connections 2023-07-06T03:47:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.113+0000 7f2ef8fff700 1 -- 172.21.15.97:0/168474552 wait complete. 2023-07-06T03:47:21.133 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T03:47:21.134 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:47:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.327+0000 7fb5dd7ff700 1 Processor -- start 2023-07-06T03:47:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.327+0000 7fb5dd7ff700 1 -- start start 2023-07-06T03:47:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.328+0000 7fb5dd7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5d815c520 0x7fb5d80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.328+0000 7fb5dd7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb5d8160180 0x7fb5d81605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5dd7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d80cdae0 con 0x7fb5d8160b30 2023-07-06T03:47:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5dd7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d80cd8d0 con 0x7fb5d815c520 2023-07-06T03:47:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5dd7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d80b6640 con 0x7fb5d8160180 2023-07-06T03:47:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5d6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5d815c520 0x7fb5d80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5d77fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb5d8160b30 0x7fb5d80cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42110/0 (socket says 172.21.15.97:42110) 2023-07-06T03:47:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5d77fe700 1 -- 172.21.15.97:0/3765897762 learned_addr learned my addr 172.21.15.97:0/3765897762 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:21.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5d6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5d815c520 0x7fb5d80cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51516/0 (socket says 172.21.15.97:51516) 2023-07-06T03:47:21.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5cffff700 1 --2- 172.21.15.97:0/3765897762 >> v2:172.21.15.146:3300/0 conn(0x7fb5d8160180 0x7fb5d81605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:21.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5d6ffd700 1 -- 172.21.15.97:0/3765897762 >> v2:172.21.15.146:3300/0 conn(0x7fb5d8160180 msgr2=0x7fb5d81605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:21.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5d6ffd700 1 --2- 172.21.15.97:0/3765897762 >> v2:172.21.15.146:3300/0 conn(0x7fb5d8160180 0x7fb5d81605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5d6ffd700 1 -- 172.21.15.97:0/3765897762 >> v1:172.21.15.97:6789/0 conn(0x7fb5d8160b30 legacy=0x7fb5d80cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5d6ffd700 1 -- 172.21.15.97:0/3765897762 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d80b6920 con 0x7fb5d815c520 2023-07-06T03:47:21.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.329+0000 7fb5d6ffd700 1 --2- 172.21.15.97:0/3765897762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d815c520 0x7fb5d80cd390 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80b5930 tx=0x7fb5c8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aad49fd42f52d54 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:21.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.330+0000 7fb5d67fc700 1 -- 172.21.15.97:0/3765897762 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8011830 con 0x7fb5d815c520 2023-07-06T03:47:21.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.330+0000 7fb5d67fc700 1 -- 172.21.15.97:0/3765897762 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8011990 con 0x7fb5d815c520 2023-07-06T03:47:21.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.330+0000 7fb5d67fc700 1 -- 172.21.15.97:0/3765897762 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8011d20 con 0x7fb5d815c520 2023-07-06T03:47:21.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.330+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3765897762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d815c520 msgr2=0x7fb5d80cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.330+0000 7fb5dd7ff700 1 --2- 172.21.15.97:0/3765897762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d815c520 0x7fb5d80cd390 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80b5930 tx=0x7fb5c8009b20 comp rx=0 tx=0).stop 2023-07-06T03:47:21.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.330+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3765897762 shutdown_connections 2023-07-06T03:47:21.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.330+0000 7fb5dd7ff700 1 --2- 172.21.15.97:0/3765897762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d815c520 0x7fb5d80cd390 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.330+0000 7fb5dd7ff700 1 --2- 172.21.15.97:0/3765897762 >> v2:172.21.15.146:3300/0 conn(0x7fb5d8160180 0x7fb5d81605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.330+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3765897762 >> 172.21.15.97:0/3765897762 conn(0x7fb5d8071790 msgr2=0x7fb5d8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:21.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.330+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3765897762 shutdown_connections 2023-07-06T03:47:21.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.330+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3765897762 wait complete. 2023-07-06T03:47:21.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.332+0000 7fb5dd7ff700 1 Processor -- start 2023-07-06T03:47:21.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.332+0000 7fb5dd7ff700 1 -- start start 2023-07-06T03:47:21.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.333+0000 7fb5dd7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5d815c520 0x7fb5d815b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:21.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.333+0000 7fb5dd7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d8160180 0x7fb5d81f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:21.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.333+0000 7fb5dd7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5d8160b30 0x7fb5d81f1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:21.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.333+0000 7fb5dd7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815bd20 con 0x7fb5d8160180 2023-07-06T03:47:21.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.333+0000 7fb5dd7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d81f2fb0 con 0x7fb5d8160b30 2023-07-06T03:47:21.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.333+0000 7fb5dd7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80d1710 con 0x7fb5d815c520 2023-07-06T03:47:21.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d8160180 0x7fb5d81f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:21.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d8160180 0x7fb5d81f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51532/0 (socket says 172.21.15.97:51532) 2023-07-06T03:47:21.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cffff700 1 -- 172.21.15.97:0/3003881749 learned_addr learned my addr 172.21.15.97:0/3003881749 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:21.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5d6ffd700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5d815c520 0x7fb5d815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:21.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5d77fe700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5d8160b30 0x7fb5d81f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:21.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cffff700 1 -- 172.21.15.97:0/3003881749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5d815c520 msgr2=0x7fb5d815b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cffff700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5d815c520 0x7fb5d815b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cffff700 1 -- 172.21.15.97:0/3003881749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5d8160b30 msgr2=0x7fb5d81f1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cffff700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5d8160b30 0x7fb5d81f1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cffff700 1 -- 172.21.15.97:0/3003881749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c80094a0 con 0x7fb5d8160180 2023-07-06T03:47:21.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5d77fe700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5d8160b30 0x7fb5d81f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:21.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cffff700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d8160180 0x7fb5d81f2a70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000e810 tx=0x7fb5c000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:21.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cf7fe700 1 -- 172.21.15.97:0/3003881749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0013610 con 0x7fb5d8160180 2023-07-06T03:47:21.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cf7fe700 1 -- 172.21.15.97:0/3003881749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c0013770 con 0x7fb5d8160180 2023-07-06T03:47:21.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.334+0000 7fb5cf7fe700 1 -- 172.21.15.97:0/3003881749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0013b20 con 0x7fb5d8160180 2023-07-06T03:47:21.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.335+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3003881749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d80d0de0 con 0x7fb5d8160180 2023-07-06T03:47:21.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.335+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3003881749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d80d13a0 con 0x7fb5d8160180 2023-07-06T03:47:21.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.336+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3003881749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d80d18c0 con 0x7fb5d8160180 2023-07-06T03:47:21.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.336+0000 7fb5cf7fe700 1 -- 172.21.15.97:0/3003881749 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb5c0013c80 con 0x7fb5d8160180 2023-07-06T03:47:21.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.338+0000 7fb5cf7fe700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5b806ace0 0x7fb5b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:21.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.338+0000 7fb5cf7fe700 1 -- 172.21.15.97:0/3003881749 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fb5c00890f0 con 0x7fb5d8160180 2023-07-06T03:47:21.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.338+0000 7fb5d6ffd700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5b806ace0 0x7fb5b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:21.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.339+0000 7fb5d6ffd700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5b806ace0 0x7fb5b806d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80090b0 tx=0x7fb5c801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:21.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.342+0000 7fb5cf7fe700 1 -- 172.21.15.97:0/3003881749 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb5d80d18c0 con 0x7fb5d8160180 2023-07-06T03:47:21.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.579+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3003881749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5d80d1880 con 0x7fb5d8160180 2023-07-06T03:47:21.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.580+0000 7fb5cf7fe700 1 -- 172.21.15.97:0/3003881749 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fb5c0057b40 con 0x7fb5d8160180 2023-07-06T03:47:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:21.583 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/1839628004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1839628004},{"type":"v1","addr":"172.21.15.146:6834","nonce":1839628004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4937,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2131262285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2131262285},{"type":"v1","addr":"172.21.15.146:6835","nonce":2131262285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1367937004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367937004},{"type":"v1","addr":"172.21.15.97:6835","nonce":1367937004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:46:37.331190+0000","modified":"2023-07-06T03:46:40.864523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/663657264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":663657264},{"type":"v1","addr":"172.21.15.97:6837","nonce":663657264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T03:47:21.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.582+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3003881749 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5b806ace0 msgr2=0x7fb5b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.582+0000 7fb5dd7ff700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5b806ace0 0x7fb5b806d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80090b0 tx=0x7fb5c801c040 comp rx=0 tx=0).stop 2023-07-06T03:47:21.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.582+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3003881749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d8160180 msgr2=0x7fb5d81f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.582+0000 7fb5dd7ff700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d8160180 0x7fb5d81f2a70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000e810 tx=0x7fb5c000eb20 comp rx=0 tx=0).stop 2023-07-06T03:47:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.582+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3003881749 shutdown_connections 2023-07-06T03:47:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.582+0000 7fb5dd7ff700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5b806ace0 0x7fb5b806d190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.582+0000 7fb5dd7ff700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5d8160b30 0x7fb5d81f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.582+0000 7fb5dd7ff700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5d815c520 0x7fb5d815b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.582+0000 7fb5dd7ff700 1 --2- 172.21.15.97:0/3003881749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d8160180 0x7fb5d81f2a70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.582+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3003881749 >> 172.21.15.97:0/3003881749 conn(0x7fb5d8071790 msgr2=0x7fb5d80b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.582+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3003881749 shutdown_connections 2023-07-06T03:47:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.583+0000 7fb5dd7ff700 1 -- 172.21.15.97:0/3003881749 wait complete. 2023-07-06T03:47:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2023-07-06T03:47:21.602 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:47:21.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.795+0000 7fa5c89ff700 1 Processor -- start 2023-07-06T03:47:21.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.795+0000 7fa5c89ff700 1 -- start start 2023-07-06T03:47:21.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.796+0000 7fa5c89ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa5c415b440 0x7fa5c415e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:21.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.796+0000 7fa5c89ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c4071330 con 0x7fa5c4070c30 2023-07-06T03:47:21.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.796+0000 7fa5c89ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c4071610 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.796+0000 7fa5c89ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c415dd80 con 0x7fa5c415b440 2023-07-06T03:47:21.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.797+0000 7fa5c2d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa5c4070c30 0x7fa5c406f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42126/0 (socket says 172.21.15.97:42126) 2023-07-06T03:47:21.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.797+0000 7fa5c2d9d700 1 -- 172.21.15.97:0/62442086 learned_addr learned my addr 172.21.15.97:0/62442086 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:21.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.797+0000 7fa5c1d9b700 1 --2- 172.21.15.97:0/62442086 >> v2:172.21.15.146:3300/0 conn(0x7fa5c415b440 0x7fa5c415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:21.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.797+0000 7fa5c159a700 1 -- 172.21.15.97:0/62442086 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541397174 0 0) 0x7fa5c4071330 con 0x7fa5c4070c30 2023-07-06T03:47:21.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.797+0000 7fa5c159a700 1 -- 172.21.15.97:0/62442086 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5ac0035b0 con 0x7fa5c4070c30 2023-07-06T03:47:21.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.797+0000 7fa5c159a700 1 -- 172.21.15.97:0/62442086 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405864527 0 0) 0x7fa5c4071610 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.797+0000 7fa5c159a700 1 -- 172.21.15.97:0/62442086 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5c4071330 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.797+0000 7fa5c1d9b700 1 -- 172.21.15.97:0/62442086 >> v1:172.21.15.146:6789/0 conn(0x7fa5c40cd8e0 legacy=0x7fa5c40cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.797+0000 7fa5c1d9b700 1 -- 172.21.15.97:0/62442086 >> v1:172.21.15.97:6789/0 conn(0x7fa5c4070c30 legacy=0x7fa5c406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.797+0000 7fa5c159a700 1 -- 172.21.15.97:0/62442086 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2232922303 0 0) 0x7fa5ac0035b0 con 0x7fa5c4070c30 2023-07-06T03:47:21.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.797+0000 7fa5c1d9b700 1 -- 172.21.15.97:0/62442086 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c415e060 con 0x7fa5c415b440 2023-07-06T03:47:21.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.798+0000 7fa5c1d9b700 1 --2- 172.21.15.97:0/62442086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5c415b440 0x7fa5c415e860 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009840 tx=0x7fa5b8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf30e794aacd684 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.798+0000 7fa5c159a700 1 -- 172.21.15.97:0/62442086 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8011830 con 0x7fa5c415b440 2023-07-06T03:47:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.798+0000 7fa5c159a700 1 -- 172.21.15.97:0/62442086 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b8011990 con 0x7fa5c415b440 2023-07-06T03:47:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.798+0000 7fa5c159a700 1 -- 172.21.15.97:0/62442086 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8011d20 con 0x7fa5c415b440 2023-07-06T03:47:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.798+0000 7fa5c89ff700 1 -- 172.21.15.97:0/62442086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5c415b440 msgr2=0x7fa5c415e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.798+0000 7fa5c89ff700 1 --2- 172.21.15.97:0/62442086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5c415b440 0x7fa5c415e860 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009840 tx=0x7fa5b8009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.799+0000 7fa5c89ff700 1 -- 172.21.15.97:0/62442086 shutdown_connections 2023-07-06T03:47:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.799+0000 7fa5c89ff700 1 --2- 172.21.15.97:0/62442086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5c415b440 0x7fa5c415e860 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.799+0000 7fa5c89ff700 1 -- 172.21.15.97:0/62442086 >> 172.21.15.97:0/62442086 conn(0x7fa5c40ce8c0 msgr2=0x7fa5c40cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.799+0000 7fa5c89ff700 1 -- 172.21.15.97:0/62442086 shutdown_connections 2023-07-06T03:47:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.799+0000 7fa5c89ff700 1 -- 172.21.15.97:0/62442086 wait complete. 2023-07-06T03:47:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.800+0000 7fa5c89ff700 1 Processor -- start 2023-07-06T03:47:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.800+0000 7fa5c89ff700 1 -- start start 2023-07-06T03:47:21.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.801+0000 7fa5c89ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5c4070c30 0x7fa5c415b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:21.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.801+0000 7fa5c89ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c40cd8e0 0x7fa5c40afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:21.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.801+0000 7fa5c89ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5c415b440 0x7fa5c41f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:21.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.801+0000 7fa5c89ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c41e4920 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.801+0000 7fa5c89ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c41557c0 con 0x7fa5c4070c30 2023-07-06T03:47:21.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.801+0000 7fa5c89ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c4155aa0 con 0x7fa5c415b440 2023-07-06T03:47:21.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c1d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c40cd8e0 0x7fa5c40afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:21.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c1d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c40cd8e0 0x7fa5c40afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51546/0 (socket says 172.21.15.97:51546) 2023-07-06T03:47:21.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c1d9b700 1 -- 172.21.15.97:0/4197271294 learned_addr learned my addr 172.21.15.97:0/4197271294 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:21.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c259c700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5c4070c30 0x7fa5c415b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:21.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c2d9d700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5c415b440 0x7fa5c41f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c1d9b700 1 -- 172.21.15.97:0/4197271294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5c415b440 msgr2=0x7fa5c41f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c1d9b700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5c415b440 0x7fa5c41f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c1d9b700 1 -- 172.21.15.97:0/4197271294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5c4070c30 msgr2=0x7fa5c415b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c1d9b700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5c4070c30 0x7fa5c415b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c1d9b700 1 -- 172.21.15.97:0/4197271294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b80094a0 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c2d9d700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5c415b440 0x7fa5c41f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5c1d9b700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c40cd8e0 0x7fa5c40afea0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80090b0 tx=0x7fa5b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.802+0000 7fa5b2ffd700 1 -- 172.21.15.97:0/4197271294 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b801a760 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.803+0000 7fa5b2ffd700 1 -- 172.21.15.97:0/4197271294 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b801a8c0 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.803+0000 7fa5c89ff700 1 -- 172.21.15.97:0/4197271294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c41e6380 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.803+0000 7fa5b2ffd700 1 -- 172.21.15.97:0/4197271294 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b801ac30 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.803+0000 7fa5c89ff700 1 -- 172.21.15.97:0/4197271294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c41554b0 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.804+0000 7fa5c89ff700 1 -- 172.21.15.97:0/4197271294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c4070aa0 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.805+0000 7fa5b2ffd700 1 -- 172.21.15.97:0/4197271294 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5b8022070 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.805+0000 7fa5b2ffd700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5a406ace0 0x7fa5a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:21.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.806+0000 7fa5b2ffd700 1 -- 172.21.15.97:0/4197271294 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fa5b808a4b0 con 0x7fa5c40cd8e0 2023-07-06T03:47:21.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.806+0000 7fa5c259c700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5a406ace0 0x7fa5a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:21.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.806+0000 7fa5c259c700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5a406ace0 0x7fa5a406d190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40011e0 tx=0x7fa5b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:21.810+0000 7fa5b2ffd700 1 -- 172.21.15.97:0/4197271294 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa5b80556b0 con 0x7fa5c40cd8e0 2023-07-06T03:47:22.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.048+0000 7fa5c89ff700 1 -- 172.21.15.97:0/4197271294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5c40ce1e0 con 0x7fa5c40cd8e0 2023-07-06T03:47:22.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.048+0000 7fa5b2ffd700 1 -- 172.21.15.97:0/4197271294 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fa5b8058cd0 con 0x7fa5c40cd8e0 2023-07-06T03:47:22.050 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:22.050 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/1839628004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1839628004},{"type":"v1","addr":"172.21.15.146:6834","nonce":1839628004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4937,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2131262285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2131262285},{"type":"v1","addr":"172.21.15.146:6835","nonce":2131262285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1367937004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367937004},{"type":"v1","addr":"172.21.15.97:6835","nonce":1367937004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:46:37.331190+0000","modified":"2023-07-06T03:46:40.864523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5301":{"gid":5301,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6837/663657264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":663657264},{"type":"v1","addr":"172.21.15.97:6837","nonce":663657264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T03:47:22.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.050+0000 7fa5c89ff700 1 -- 172.21.15.97:0/4197271294 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5a406ace0 msgr2=0x7fa5a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.050+0000 7fa5c89ff700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5a406ace0 0x7fa5a406d190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40011e0 tx=0x7fa5b4009040 comp rx=0 tx=0).stop 2023-07-06T03:47:22.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.050+0000 7fa5c89ff700 1 -- 172.21.15.97:0/4197271294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c40cd8e0 msgr2=0x7fa5c40afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.050+0000 7fa5c89ff700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c40cd8e0 0x7fa5c40afea0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80090b0 tx=0x7fa5b8004750 comp rx=0 tx=0).stop 2023-07-06T03:47:22.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.050+0000 7fa5c89ff700 1 -- 172.21.15.97:0/4197271294 shutdown_connections 2023-07-06T03:47:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.050+0000 7fa5c89ff700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5a406ace0 0x7fa5a406d190 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.050+0000 7fa5c89ff700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5c415b440 0x7fa5c41f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.050+0000 7fa5c89ff700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5c4070c30 0x7fa5c415b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.050+0000 7fa5c89ff700 1 --2- 172.21.15.97:0/4197271294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c40cd8e0 0x7fa5c40afea0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.050+0000 7fa5c89ff700 1 -- 172.21.15.97:0/4197271294 >> 172.21.15.97:0/4197271294 conn(0x7fa5c40ce8c0 msgr2=0x7fa5c40cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.051+0000 7fa5c89ff700 1 -- 172.21.15.97:0/4197271294 shutdown_connections 2023-07-06T03:47:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.051+0000 7fa5c89ff700 1 -- 172.21.15.97:0/4197271294 wait complete. 2023-07-06T03:47:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2023-07-06T03:47:22.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:47:22.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.263+0000 7f4d65dff700 1 Processor -- start 2023-07-06T03:47:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.263+0000 7f4d65dff700 1 -- start start 2023-07-06T03:47:22.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.264+0000 7f4d65dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cd8e0 0x7f4d6015e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.264+0000 7f4d65dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d60070c30 0x7f4d600adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.264+0000 7f4d65dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d600710a0 con 0x7f4d600cdcb0 2023-07-06T03:47:22.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.264+0000 7f4d65dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d600ae3e0 con 0x7f4d60070c30 2023-07-06T03:47:22.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.264+0000 7f4d65dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60071330 con 0x7f4d600cd8e0 2023-07-06T03:47:22.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.264+0000 7f4d5effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d60070c30 0x7f4d600adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.264+0000 7f4d5effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d60070c30 0x7f4d600adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51556/0 (socket says 172.21.15.97:51556) 2023-07-06T03:47:22.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.264+0000 7f4d5effd700 1 -- 172.21.15.97:0/3306706116 learned_addr learned my addr 172.21.15.97:0/3306706116 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:22.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.265+0000 7f4d5f7fe700 1 --2- 172.21.15.97:0/3306706116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cd8e0 0x7f4d6015e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.265+0000 7f4d5effd700 1 -- 172.21.15.97:0/3306706116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cd8e0 msgr2=0x7f4d6015e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:22.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.265+0000 7f4d5effd700 1 --2- 172.21.15.97:0/3306706116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cd8e0 0x7f4d6015e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.265+0000 7f4d5effd700 1 -- 172.21.15.97:0/3306706116 >> v1:172.21.15.146:6789/0 conn(0x7f4d600cdcb0 legacy=0x7f4d6006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:22.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.265+0000 7f4d5effd700 1 -- 172.21.15.97:0/3306706116 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d60071610 con 0x7f4d60070c30 2023-07-06T03:47:22.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.265+0000 7f4d5f7fe700 1 --2- 172.21.15.97:0/3306706116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cd8e0 0x7f4d6015e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:22.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.265+0000 7f4d5effd700 1 --2- 172.21.15.97:0/3306706116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d60070c30 0x7f4d600adea0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098f0 tx=0x7f4d54009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fe51f3377660b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:22.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.265+0000 7f4d5e7fc700 1 -- 172.21.15.97:0/3306706116 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54004230 con 0x7f4d60070c30 2023-07-06T03:47:22.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.265+0000 7f4d5e7fc700 1 -- 172.21.15.97:0/3306706116 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54004390 con 0x7f4d60070c30 2023-07-06T03:47:22.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.265+0000 7f4d5e7fc700 1 -- 172.21.15.97:0/3306706116 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d540036a0 con 0x7f4d60070c30 2023-07-06T03:47:22.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.266+0000 7f4d65dff700 1 -- 172.21.15.97:0/3306706116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d60070c30 msgr2=0x7f4d600adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.266+0000 7f4d65dff700 1 --2- 172.21.15.97:0/3306706116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d60070c30 0x7f4d600adea0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098f0 tx=0x7f4d54009c00 comp rx=0 tx=0).stop 2023-07-06T03:47:22.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.266+0000 7f4d65dff700 1 -- 172.21.15.97:0/3306706116 shutdown_connections 2023-07-06T03:47:22.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.266+0000 7f4d65dff700 1 --2- 172.21.15.97:0/3306706116 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cd8e0 0x7f4d6015e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.266+0000 7f4d65dff700 1 --2- 172.21.15.97:0/3306706116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d60070c30 0x7f4d600adea0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.266+0000 7f4d65dff700 1 -- 172.21.15.97:0/3306706116 >> 172.21.15.97:0/3306706116 conn(0x7f4d600ce8c0 msgr2=0x7f4d600cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:22.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.266+0000 7f4d65dff700 1 -- 172.21.15.97:0/3306706116 shutdown_connections 2023-07-06T03:47:22.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.266+0000 7f4d65dff700 1 -- 172.21.15.97:0/3306706116 wait complete. 2023-07-06T03:47:22.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.268+0000 7f4d65dff700 1 Processor -- start 2023-07-06T03:47:22.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.268+0000 7f4d65dff700 1 -- start start 2023-07-06T03:47:22.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d65dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d60070c30 0x7f4d60149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d65dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d600cd8e0 0x7f4d60155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d65dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cdcb0 0x7f4d60155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d65dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6015a430 con 0x7f4d600cd8e0 2023-07-06T03:47:22.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d65dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6015be00 con 0x7f4d60070c30 2023-07-06T03:47:22.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d65dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6014a750 con 0x7f4d600cdcb0 2023-07-06T03:47:22.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d5effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d600cd8e0 0x7f4d60155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d5effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d600cd8e0 0x7f4d60155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51560/0 (socket says 172.21.15.97:51560) 2023-07-06T03:47:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d5effd700 1 -- 172.21.15.97:0/781115859 learned_addr learned my addr 172.21.15.97:0/781115859 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d5f7fe700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d60070c30 0x7f4d60149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d5ffff700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cdcb0 0x7f4d60155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d5effd700 1 -- 172.21.15.97:0/781115859 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cdcb0 msgr2=0x7f4d60155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d5effd700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cdcb0 0x7f4d60155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d5effd700 1 -- 172.21.15.97:0/781115859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d60070c30 msgr2=0x7f4d60149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d5effd700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d60070c30 0x7f4d60149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d5effd700 1 -- 172.21.15.97:0/781115859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009550 con 0x7f4d600cd8e0 2023-07-06T03:47:22.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.269+0000 7f4d5ffff700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cdcb0 0x7f4d60155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:22.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.270+0000 7f4d5effd700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d600cd8e0 0x7f4d60155340 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4d6006b980 tx=0x7f4d54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:22.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.270+0000 7f4d47fff700 1 -- 172.21.15.97:0/781115859 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400b860 con 0x7f4d600cd8e0 2023-07-06T03:47:22.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.270+0000 7f4d47fff700 1 -- 172.21.15.97:0/781115859 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5400b9c0 con 0x7f4d600cd8e0 2023-07-06T03:47:22.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.270+0000 7f4d65dff700 1 -- 172.21.15.97:0/781115859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6015a640 con 0x7f4d600cd8e0 2023-07-06T03:47:22.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.270+0000 7f4d47fff700 1 -- 172.21.15.97:0/781115859 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400bd70 con 0x7f4d600cd8e0 2023-07-06T03:47:22.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.270+0000 7f4d65dff700 1 -- 172.21.15.97:0/781115859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6015c310 con 0x7f4d600cd8e0 2023-07-06T03:47:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.271+0000 7f4d65dff700 1 -- 172.21.15.97:0/781115859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d60070aa0 con 0x7f4d600cd8e0 2023-07-06T03:47:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.272+0000 7f4d47fff700 1 -- 172.21.15.97:0/781115859 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d5401b440 con 0x7f4d600cd8e0 2023-07-06T03:47:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.273+0000 7f4d47fff700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4c06ace0 0x7f4d4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.273+0000 7f4d47fff700 1 -- 172.21.15.97:0/781115859 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f4d5408af00 con 0x7f4d600cd8e0 2023-07-06T03:47:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.273+0000 7f4d5f7fe700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4c06ace0 0x7f4d4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.274+0000 7f4d5f7fe700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4c06ace0 0x7f4d4c06d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d50006fd0 tx=0x7f4d50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.278+0000 7f4d47fff700 1 -- 172.21.15.97:0/781115859 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4d54055f40 con 0x7f4d600cd8e0 2023-07-06T03:47:22.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.471+0000 7f4d65dff700 1 -- 172.21.15.97:0/781115859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d60067830 con 0x7f4d600cd8e0 2023-07-06T03:47:22.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.473+0000 7f4d47fff700 1 -- 172.21.15.97:0/781115859 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18460 (secure 0 0 0) 0x7f4d54059560 con 0x7f4d600cd8e0 2023-07-06T03:47:22.475 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:22.475 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:46:52.661343+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:46:35.584013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T03:46:36.303248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.28","mappings":[{"from":4,"to":7}]},{"pgid":"6.36","mappings":[{"from":1,"to":2}]},{"pgid":"6.38","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:47:22.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.475+0000 7f4d65dff700 1 -- 172.21.15.97:0/781115859 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4c06ace0 msgr2=0x7f4d4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.475+0000 7f4d65dff700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4c06ace0 0x7f4d4c06d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d50006fd0 tx=0x7f4d50008040 comp rx=0 tx=0).stop 2023-07-06T03:47:22.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.475+0000 7f4d65dff700 1 -- 172.21.15.97:0/781115859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d600cd8e0 msgr2=0x7f4d60155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.475+0000 7f4d65dff700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d600cd8e0 0x7f4d60155340 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4d6006b980 tx=0x7f4d54004750 comp rx=0 tx=0).stop 2023-07-06T03:47:22.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.475+0000 7f4d65dff700 1 -- 172.21.15.97:0/781115859 shutdown_connections 2023-07-06T03:47:22.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.475+0000 7f4d65dff700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4c06ace0 0x7f4d4c06d190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.475+0000 7f4d65dff700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d600cdcb0 0x7f4d60155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.475+0000 7f4d65dff700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d60070c30 0x7f4d60149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.475+0000 7f4d65dff700 1 --2- 172.21.15.97:0/781115859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d600cd8e0 0x7f4d60155340 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.475+0000 7f4d65dff700 1 -- 172.21.15.97:0/781115859 >> 172.21.15.97:0/781115859 conn(0x7f4d600ce8c0 msgr2=0x7f4d600acdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:22.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.475+0000 7f4d65dff700 1 -- 172.21.15.97:0/781115859 shutdown_connections 2023-07-06T03:47:22.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.476+0000 7f4d65dff700 1 -- 172.21.15.97:0/781115859 wait complete. 2023-07-06T03:47:22.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T03:47:22.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.689+0000 7f5cc59ff700 1 Processor -- start 2023-07-06T03:47:22.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.689+0000 7f5cc59ff700 1 -- start start 2023-07-06T03:47:22.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.690+0000 7f5cc59ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5cc015c610 0x7f5cc00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.690+0000 7f5cc59ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5cc006b360 0x7f5cc00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.690+0000 7f5cc59ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc00cff90 con 0x7f5cc00d2210 2023-07-06T03:47:22.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.690+0000 7f5cc59ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc00d0780 con 0x7f5cc015c610 2023-07-06T03:47:22.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.690+0000 7f5cc59ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc015c9e0 con 0x7f5cc006b360 2023-07-06T03:47:22.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.690+0000 7f5cbeffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5cc015c610 0x7f5cc00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.690+0000 7f5cbf7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5cc00d2210 0x7f5cc00d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42140/0 (socket says 172.21.15.97:42140) 2023-07-06T03:47:22.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.690+0000 7f5cbf7fe700 1 -- 172.21.15.97:0/108686316 learned_addr learned my addr 172.21.15.97:0/108686316 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:22.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.690+0000 7f5cbe7fc700 1 --2- 172.21.15.97:0/108686316 >> v2:172.21.15.146:3300/0 conn(0x7f5cc006b360 0x7f5cc00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.691+0000 7f5cbeffd700 1 -- 172.21.15.97:0/108686316 >> v2:172.21.15.146:3300/0 conn(0x7f5cc006b360 msgr2=0x7f5cc00b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:22.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.691+0000 7f5cbeffd700 1 --2- 172.21.15.97:0/108686316 >> v2:172.21.15.146:3300/0 conn(0x7f5cc006b360 0x7f5cc00b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.691+0000 7f5cbeffd700 1 -- 172.21.15.97:0/108686316 >> v1:172.21.15.97:6789/0 conn(0x7f5cc00d2210 legacy=0x7f5cc00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.691+0000 7f5cbeffd700 1 -- 172.21.15.97:0/108686316 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc00b5650 con 0x7f5cc015c610 2023-07-06T03:47:22.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.691+0000 7f5cbdffb700 1 -- 172.21.15.97:0/108686316 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879993814 0 0) 0x7f5cc00cff90 con 0x7f5cc00d2210 2023-07-06T03:47:22.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.691+0000 7f5cbeffd700 1 --2- 172.21.15.97:0/108686316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc015c610 0x7f5cc00b47b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5cc006a590 tx=0x7f5cb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffba67784eeefbe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:22.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.691+0000 7f5cbdffb700 1 -- 172.21.15.97:0/108686316 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0011830 con 0x7f5cc015c610 2023-07-06T03:47:22.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.691+0000 7f5cbdffb700 1 -- 172.21.15.97:0/108686316 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb0011990 con 0x7f5cc015c610 2023-07-06T03:47:22.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.691+0000 7f5cbdffb700 1 -- 172.21.15.97:0/108686316 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0011d20 con 0x7f5cc015c610 2023-07-06T03:47:22.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.692+0000 7f5cc59ff700 1 -- 172.21.15.97:0/108686316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc015c610 msgr2=0x7f5cc00b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.692+0000 7f5cc59ff700 1 --2- 172.21.15.97:0/108686316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc015c610 0x7f5cc00b47b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5cc006a590 tx=0x7f5cb0009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:22.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.692+0000 7f5cc59ff700 1 -- 172.21.15.97:0/108686316 shutdown_connections 2023-07-06T03:47:22.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.692+0000 7f5cc59ff700 1 --2- 172.21.15.97:0/108686316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc015c610 0x7f5cc00b47b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.692+0000 7f5cc59ff700 1 --2- 172.21.15.97:0/108686316 >> v2:172.21.15.146:3300/0 conn(0x7f5cc006b360 0x7f5cc00b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.692+0000 7f5cc59ff700 1 -- 172.21.15.97:0/108686316 >> 172.21.15.97:0/108686316 conn(0x7f5cc006a6c0 msgr2=0x7f5cc006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:22.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.692+0000 7f5cc59ff700 1 -- 172.21.15.97:0/108686316 shutdown_connections 2023-07-06T03:47:22.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.692+0000 7f5cc59ff700 1 -- 172.21.15.97:0/108686316 wait complete. 2023-07-06T03:47:22.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.693+0000 7f5cc59ff700 1 Processor -- start 2023-07-06T03:47:22.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.694+0000 7f5cc59ff700 1 -- start start 2023-07-06T03:47:22.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.694+0000 7f5cc59ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc006b360 0x7f5cc01f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.694+0000 7f5cc59ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cc00d2210 0x7f5cc00caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cc59ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc015c610 0x7f5cc015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cc59ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00cc440 con 0x7f5cc006b360 2023-07-06T03:47:22.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cc59ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00cb830 con 0x7f5cc00d2210 2023-07-06T03:47:22.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cc59ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00cbb10 con 0x7f5cc015c610 2023-07-06T03:47:22.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cbeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc006b360 0x7f5cc01f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cbeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc006b360 0x7f5cc01f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51566/0 (socket says 172.21.15.97:51566) 2023-07-06T03:47:22.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cbeffd700 1 -- 172.21.15.97:0/517321693 learned_addr learned my addr 172.21.15.97:0/517321693 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:22.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cbf7fe700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc015c610 0x7f5cc015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cbe7fc700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cc00d2210 0x7f5cc00caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cbeffd700 1 -- 172.21.15.97:0/517321693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc015c610 msgr2=0x7f5cc015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cbeffd700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc015c610 0x7f5cc015d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cbeffd700 1 -- 172.21.15.97:0/517321693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cc00d2210 msgr2=0x7f5cc00caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cbeffd700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cc00d2210 0x7f5cc00caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cbeffd700 1 -- 172.21.15.97:0/517321693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb00094a0 con 0x7f5cc006b360 2023-07-06T03:47:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.695+0000 7f5cbeffd700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc006b360 0x7f5cc01f4c90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00090b0 tx=0x7f5cb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.696+0000 7f5ca7fff700 1 -- 172.21.15.97:0/517321693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb00038a0 con 0x7f5cc006b360 2023-07-06T03:47:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.696+0000 7f5ca7fff700 1 -- 172.21.15.97:0/517321693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb001e070 con 0x7f5cc006b360 2023-07-06T03:47:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.696+0000 7f5ca7fff700 1 -- 172.21.15.97:0/517321693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0003c20 con 0x7f5cc006b360 2023-07-06T03:47:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.696+0000 7f5cc59ff700 1 -- 172.21.15.97:0/517321693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc00cbdf0 con 0x7f5cc006b360 2023-07-06T03:47:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.696+0000 7f5cc59ff700 1 -- 172.21.15.97:0/517321693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc01f51d0 con 0x7f5cc006b360 2023-07-06T03:47:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.697+0000 7f5cc59ff700 1 -- 172.21.15.97:0/517321693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc00b19c0 con 0x7f5cc006b360 2023-07-06T03:47:22.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.698+0000 7f5ca7fff700 1 -- 172.21.15.97:0/517321693 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5cb002d450 con 0x7f5cc006b360 2023-07-06T03:47:22.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.698+0000 7f5ca7fff700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cac06ace0 0x7f5cac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:22.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.699+0000 7f5ca7fff700 1 -- 172.21.15.97:0/517321693 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f5cb008a520 con 0x7f5cc006b360 2023-07-06T03:47:22.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.699+0000 7f5cbe7fc700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cac06ace0 0x7f5cac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:22.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.699+0000 7f5cbe7fc700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cac06ace0 0x7f5cac06d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0153ac0 tx=0x7f5cb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:22.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.703+0000 7f5ca7fff700 1 -- 172.21.15.97:0/517321693 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5cb0055600 con 0x7f5cc006b360 2023-07-06T03:47:22.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.944+0000 7f5cc59ff700 1 -- 172.21.15.97:0/517321693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5cc004e340 con 0x7f5cc006b360 2023-07-06T03:47:22.946 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-07-06T03:47:22.945+0000 7fb463ff4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T03:47:22.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.949+0000 7f5ca7fff700 1 -- 172.21.15.97:0/517321693 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5cb0058c20 con 0x7f5cc006b360 2023-07-06T03:47:22.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 -- 172.21.15.97:0/517321693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cac06ace0 msgr2=0x7f5cac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cac06ace0 0x7f5cac06d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0153ac0 tx=0x7f5cb4008040 comp rx=0 tx=0).stop 2023-07-06T03:47:22.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 -- 172.21.15.97:0/517321693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc006b360 msgr2=0x7f5cc01f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:22.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc006b360 0x7f5cc01f4c90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00090b0 tx=0x7f5cb0004750 comp rx=0 tx=0).stop 2023-07-06T03:47:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 -- 172.21.15.97:0/517321693 shutdown_connections 2023-07-06T03:47:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cac06ace0 0x7f5cac06d190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc015c610 0x7f5cc015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc006b360 0x7f5cc01f4c90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 --2- 172.21.15.97:0/517321693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cc00d2210 0x7f5cc00caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 -- 172.21.15.97:0/517321693 >> 172.21.15.97:0/517321693 conn(0x7f5cc006a6c0 msgr2=0x7f5cc00b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 -- 172.21.15.97:0/517321693 shutdown_connections 2023-07-06T03:47:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:22.952+0000 7f5cc59ff700 1 -- 172.21.15.97:0/517321693 wait complete. 2023-07-06T03:47:22.956 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T03:47:22.975 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T03:47:23.024 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:47:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.175+0000 7f76b2dff700 1 Processor -- start 2023-07-06T03:47:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.175+0000 7f76b2dff700 1 -- start start 2023-07-06T03:47:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.176+0000 7f76b2dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76ac06b170 0x7f76ac15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.176+0000 7f76b2dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f76ac15cae0 0x7f76ac06b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.176+0000 7f76b2dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ac15de00 con 0x7f76ac0cfbc0 2023-07-06T03:47:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.176+0000 7f76b2dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ac0b25a0 con 0x7f76ac06b170 2023-07-06T03:47:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.176+0000 7f76b2dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ac06ba80 con 0x7f76ac15cae0 2023-07-06T03:47:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.176+0000 7f76b0b9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76ac06b170 0x7f76ac15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.176+0000 7f76b139c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f76ac0cfbc0 0x7f76ac15e1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42146/0 (socket says 172.21.15.97:42146) 2023-07-06T03:47:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.176+0000 7f76b139c700 1 -- 172.21.15.97:0/1996405809 learned_addr learned my addr 172.21.15.97:0/1996405809 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.176+0000 7f76abfff700 1 --2- 172.21.15.97:0/1996405809 >> v2:172.21.15.146:3300/0 conn(0x7f76ac15cae0 0x7f76ac06b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.176+0000 7f76ab7fe700 1 -- 172.21.15.97:0/1996405809 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146938982 0 0) 0x7f76a4001490 con 0x7f76ac0cfbc0 2023-07-06T03:47:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.177+0000 7f76ab7fe700 1 -- 172.21.15.97:0/1996405809 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f769c0035b0 con 0x7f76ac0cfbc0 2023-07-06T03:47:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.177+0000 7f76b0b9b700 1 -- 172.21.15.97:0/1996405809 >> v2:172.21.15.146:3300/0 conn(0x7f76ac15cae0 msgr2=0x7f76ac06b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.177+0000 7f76b0b9b700 1 --2- 172.21.15.97:0/1996405809 >> v2:172.21.15.146:3300/0 conn(0x7f76ac15cae0 0x7f76ac06b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.177+0000 7f76b0b9b700 1 -- 172.21.15.97:0/1996405809 >> v1:172.21.15.97:6789/0 conn(0x7f76ac0cfbc0 legacy=0x7f76ac15e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.177+0000 7f76b0b9b700 1 -- 172.21.15.97:0/1996405809 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac0ceb20 con 0x7f76ac06b170 2023-07-06T03:47:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.177+0000 7f76abfff700 1 --2- 172.21.15.97:0/1996405809 >> v2:172.21.15.146:3300/0 conn(0x7f76ac15cae0 0x7f76ac06b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.177+0000 7f76b0b9b700 1 --2- 172.21.15.97:0/1996405809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac06b170 0x7f76ac15d8c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f76ac06dbf0 tx=0x7f76a0009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4f3d246534bd23 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.177+0000 7f76ab7fe700 1 -- 172.21.15.97:0/1996405809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0005e30 con 0x7f76ac06b170 2023-07-06T03:47:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.177+0000 7f76ab7fe700 1 -- 172.21.15.97:0/1996405809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a0006030 con 0x7f76ac06b170 2023-07-06T03:47:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.177+0000 7f76ab7fe700 1 -- 172.21.15.97:0/1996405809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a00063c0 con 0x7f76ac06b170 2023-07-06T03:47:23.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.178+0000 7f76b2dff700 1 -- 172.21.15.97:0/1996405809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac06b170 msgr2=0x7f76ac15d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:23.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.178+0000 7f76b2dff700 1 --2- 172.21.15.97:0/1996405809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac06b170 0x7f76ac15d8c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f76ac06dbf0 tx=0x7f76a0009b00 comp rx=0 tx=0).stop 2023-07-06T03:47:23.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.178+0000 7f76b2dff700 1 -- 172.21.15.97:0/1996405809 shutdown_connections 2023-07-06T03:47:23.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.178+0000 7f76b2dff700 1 --2- 172.21.15.97:0/1996405809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac06b170 0x7f76ac15d8c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:23.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.178+0000 7f76b2dff700 1 --2- 172.21.15.97:0/1996405809 >> v2:172.21.15.146:3300/0 conn(0x7f76ac15cae0 0x7f76ac06b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:23.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.178+0000 7f76b2dff700 1 -- 172.21.15.97:0/1996405809 >> 172.21.15.97:0/1996405809 conn(0x7f76ac099f10 msgr2=0x7f76ac0d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:23.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.178+0000 7f76b2dff700 1 -- 172.21.15.97:0/1996405809 shutdown_connections 2023-07-06T03:47:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.178+0000 7f76b2dff700 1 -- 172.21.15.97:0/1996405809 wait complete. 2023-07-06T03:47:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.180+0000 7f76b2dff700 1 Processor -- start 2023-07-06T03:47:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.180+0000 7f76b2dff700 1 -- start start 2023-07-06T03:47:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b2dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76ac06b170 0x7f76ac0aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b2dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76ac0cfbc0 0x7f76ac09a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b2dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac15cae0 0x7f76ac09adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b2dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0adae0 con 0x7f76ac15cae0 2023-07-06T03:47:23.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b2dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0addc0 con 0x7f76ac06b170 2023-07-06T03:47:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b2dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0ae0a0 con 0x7f76ac0cfbc0 2023-07-06T03:47:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b139c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac15cae0 0x7f76ac09adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b139c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac15cae0 0x7f76ac09adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51612/0 (socket says 172.21.15.97:51612) 2023-07-06T03:47:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b139c700 1 -- 172.21.15.97:0/2689172897 learned_addr learned my addr 172.21.15.97:0/2689172897 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b0b9b700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76ac06b170 0x7f76ac0aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76abfff700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76ac0cfbc0 0x7f76ac09a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b139c700 1 -- 172.21.15.97:0/2689172897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76ac0cfbc0 msgr2=0x7f76ac09a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:23.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b139c700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76ac0cfbc0 0x7f76ac09a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b139c700 1 -- 172.21.15.97:0/2689172897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76ac06b170 msgr2=0x7f76ac0aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b139c700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76ac06b170 0x7f76ac0aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76b139c700 1 -- 172.21.15.97:0/2689172897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a0009430 con 0x7f76ac15cae0 2023-07-06T03:47:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.181+0000 7f76abfff700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76ac0cfbc0 0x7f76ac09a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.182+0000 7f76b139c700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac15cae0 0x7f76ac09adc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f76a400cb40 tx=0x7f76a400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.182+0000 7f76a97fa700 1 -- 172.21.15.97:0/2689172897 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a400ab50 con 0x7f76ac15cae0 2023-07-06T03:47:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.182+0000 7f76a97fa700 1 -- 172.21.15.97:0/2689172897 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a400acb0 con 0x7f76ac15cae0 2023-07-06T03:47:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.182+0000 7f76b2dff700 1 -- 172.21.15.97:0/2689172897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ac0ae380 con 0x7f76ac15cae0 2023-07-06T03:47:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.182+0000 7f76a97fa700 1 -- 172.21.15.97:0/2689172897 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a4012620 con 0x7f76ac15cae0 2023-07-06T03:47:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.182+0000 7f76b2dff700 1 -- 172.21.15.97:0/2689172897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ac0af0c0 con 0x7f76ac15cae0 2023-07-06T03:47:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.183+0000 7f76b2dff700 1 -- 172.21.15.97:0/2689172897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ac04e340 con 0x7f76ac15cae0 2023-07-06T03:47:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.184+0000 7f76a97fa700 1 -- 172.21.15.97:0/2689172897 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f76a401b3b0 con 0x7f76ac15cae0 2023-07-06T03:47:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.185+0000 7f76a97fa700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f769006ace0 0x7f769006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.185+0000 7f76a97fa700 1 -- 172.21.15.97:0/2689172897 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f76a4089230 con 0x7f76ac15cae0 2023-07-06T03:47:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.186+0000 7f76b0b9b700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f769006ace0 0x7f769006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:23.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.186+0000 7f76b0b9b700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f769006ace0 0x7f769006d190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000c00 tx=0x7f76a0008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:23.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.190+0000 7f76a97fa700 1 -- 172.21.15.97:0/2689172897 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f76a4054190 con 0x7f76ac15cae0 2023-07-06T03:47:23.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.427+0000 7f76b2dff700 1 -- 172.21.15.97:0/2689172897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f76ac15ddd0 con 0x7f76ac15cae0 2023-07-06T03:47:23.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.950+0000 7f76a97fa700 1 -- 172.21.15.97:0/2689172897 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f76a40577b0 con 0x7f76ac15cae0 2023-07-06T03:47:23.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 -- 172.21.15.97:0/2689172897 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f769006ace0 msgr2=0x7f769006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:23.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f769006ace0 0x7f769006d190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000c00 tx=0x7f76a0008070 comp rx=0 tx=0).stop 2023-07-06T03:47:23.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 -- 172.21.15.97:0/2689172897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac15cae0 msgr2=0x7f76ac09adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac15cae0 0x7f76ac09adc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f76a400cb40 tx=0x7f76a400cf00 comp rx=0 tx=0).stop 2023-07-06T03:47:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 -- 172.21.15.97:0/2689172897 shutdown_connections 2023-07-06T03:47:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f769006ace0 0x7f769006d190 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76ac15cae0 0x7f76ac09adc0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76ac06b170 0x7f76ac0aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:23.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 --2- 172.21.15.97:0/2689172897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76ac0cfbc0 0x7f76ac09a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:23.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 -- 172.21.15.97:0/2689172897 >> 172.21.15.97:0/2689172897 conn(0x7f76ac099f10 msgr2=0x7f76ac0c6740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:23.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 -- 172.21.15.97:0/2689172897 shutdown_connections 2023-07-06T03:47:23.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:23.953+0000 7f76b2dff700 1 -- 172.21.15.97:0/2689172897 wait complete. 2023-07-06T03:47:23.975 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T03:47:24.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.175+0000 7fb17abff700 1 Processor -- start 2023-07-06T03:47:24.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.176+0000 7fb17abff700 1 -- start start 2023-07-06T03:47:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.177+0000 7fb17abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb17406b0c0 0x7fb1740b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.177+0000 7fb17abff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb1740b4ca0 0x7fb1740d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.177+0000 7fb17abff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1740cd330 con 0x7fb1740b85f0 2023-07-06T03:47:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.177+0000 7fb17abff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1740b89c0 con 0x7fb1740b4ca0 2023-07-06T03:47:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.177+0000 7fb17abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17406b510 con 0x7fb17406b0c0 2023-07-06T03:47:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.177+0000 7fb17899b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb1740b85f0 0x7fb1740cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42154/0 (socket says 172.21.15.97:42154) 2023-07-06T03:47:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.177+0000 7fb17899b700 1 -- 172.21.15.97:0/3465544690 learned_addr learned my addr 172.21.15.97:0/3465544690 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.177+0000 7fb17919c700 1 --2- 172.21.15.97:0/3465544690 >> v2:172.21.15.97:3300/0 conn(0x7fb1740b4ca0 0x7fb1740d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.177+0000 7fb173fff700 1 --2- 172.21.15.97:0/3465544690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb17406b0c0 0x7fb1740b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.178+0000 7fb17919c700 1 -- 172.21.15.97:0/3465544690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb17406b0c0 msgr2=0x7fb1740b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.178+0000 7fb17919c700 1 --2- 172.21.15.97:0/3465544690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb17406b0c0 0x7fb1740b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.178+0000 7fb17919c700 1 -- 172.21.15.97:0/3465544690 >> v1:172.21.15.97:6789/0 conn(0x7fb1740b85f0 legacy=0x7fb1740cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.178+0000 7fb17919c700 1 -- 172.21.15.97:0/3465544690 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1740d02a0 con 0x7fb1740b4ca0 2023-07-06T03:47:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.178+0000 7fb17919c700 1 --2- 172.21.15.97:0/3465544690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1740b4ca0 0x7fb1740d09b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009d90 tx=0x7fb16c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb9ed3d122705c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.178+0000 7fb1737fe700 1 -- 172.21.15.97:0/3465544690 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00e070 con 0x7fb1740b4ca0 2023-07-06T03:47:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.178+0000 7fb1737fe700 1 -- 172.21.15.97:0/3465544690 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16c00bbd0 con 0x7fb1740b4ca0 2023-07-06T03:47:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.178+0000 7fb1737fe700 1 -- 172.21.15.97:0/3465544690 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c012430 con 0x7fb1740b4ca0 2023-07-06T03:47:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.178+0000 7fb17abff700 1 -- 172.21.15.97:0/3465544690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1740b4ca0 msgr2=0x7fb1740d09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.178+0000 7fb17abff700 1 --2- 172.21.15.97:0/3465544690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1740b4ca0 0x7fb1740d09b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009d90 tx=0x7fb16c00b330 comp rx=0 tx=0).stop 2023-07-06T03:47:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.179+0000 7fb17abff700 1 -- 172.21.15.97:0/3465544690 shutdown_connections 2023-07-06T03:47:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.179+0000 7fb17abff700 1 --2- 172.21.15.97:0/3465544690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1740b4ca0 0x7fb1740d09b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.179+0000 7fb17abff700 1 --2- 172.21.15.97:0/3465544690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb17406b0c0 0x7fb1740b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.179+0000 7fb17abff700 1 -- 172.21.15.97:0/3465544690 >> 172.21.15.97:0/3465544690 conn(0x7fb17406a5b0 msgr2=0x7fb17406a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.179+0000 7fb17abff700 1 -- 172.21.15.97:0/3465544690 shutdown_connections 2023-07-06T03:47:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.179+0000 7fb17abff700 1 -- 172.21.15.97:0/3465544690 wait complete. 2023-07-06T03:47:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.180+0000 7fb17abff700 1 Processor -- start 2023-07-06T03:47:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.181+0000 7fb17abff700 1 -- start start 2023-07-06T03:47:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17abff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb17406b0c0 0x7fb17415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17abff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1740b4ca0 0x7fb1740c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb1740b85f0 0x7fb1741f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17abff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740c8090 con 0x7fb17406b0c0 2023-07-06T03:47:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17abff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741f6200 con 0x7fb1740b4ca0 2023-07-06T03:47:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741f64e0 con 0x7fb1740b85f0 2023-07-06T03:47:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17899b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb17406b0c0 0x7fb17415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17899b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb17406b0c0 0x7fb17415a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51622/0 (socket says 172.21.15.97:51622) 2023-07-06T03:47:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17899b700 1 -- 172.21.15.97:0/1673961987 learned_addr learned my addr 172.21.15.97:0/1673961987 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb173fff700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1740b4ca0 0x7fb1740c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17919c700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb1740b85f0 0x7fb1741f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17899b700 1 -- 172.21.15.97:0/1673961987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb1740b85f0 msgr2=0x7fb1741f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17899b700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb1740b85f0 0x7fb1741f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:24.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17899b700 1 -- 172.21.15.97:0/1673961987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1740b4ca0 msgr2=0x7fb1740c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:24.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17899b700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1740b4ca0 0x7fb1740c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:24.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.182+0000 7fb17899b700 1 -- 172.21.15.97:0/1673961987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c009a70 con 0x7fb17406b0c0 2023-07-06T03:47:24.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.183+0000 7fb17899b700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb17406b0c0 0x7fb17415a8f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c480 tx=0x7fb16800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:24.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.183+0000 7fb1717fa700 1 -- 172.21.15.97:0/1673961987 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168005d70 con 0x7fb17406b0c0 2023-07-06T03:47:24.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.183+0000 7fb1717fa700 1 -- 172.21.15.97:0/1673961987 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1680070d0 con 0x7fb17406b0c0 2023-07-06T03:47:24.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.183+0000 7fb17abff700 1 -- 172.21.15.97:0/1673961987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1740c82e0 con 0x7fb17406b0c0 2023-07-06T03:47:24.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.183+0000 7fb1717fa700 1 -- 172.21.15.97:0/1673961987 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168007440 con 0x7fb17406b0c0 2023-07-06T03:47:24.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.183+0000 7fb17abff700 1 -- 172.21.15.97:0/1673961987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb174156600 con 0x7fb17406b0c0 2023-07-06T03:47:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.184+0000 7fb17abff700 1 -- 172.21.15.97:0/1673961987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17404e340 con 0x7fb17406b0c0 2023-07-06T03:47:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.185+0000 7fb1717fa700 1 -- 172.21.15.97:0/1673961987 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb1680075a0 con 0x7fb17406b0c0 2023-07-06T03:47:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.186+0000 7fb1717fa700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb15806ace0 0x7fb15806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.186+0000 7fb1717fa700 1 -- 172.21.15.97:0/1673961987 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fb1680889a0 con 0x7fb17406b0c0 2023-07-06T03:47:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.187+0000 7fb173fff700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb15806ace0 0x7fb15806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:24.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.187+0000 7fb173fff700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb15806ace0 0x7fb15806d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb17415b560 tx=0x7fb160008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:24.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.191+0000 7fb1717fa700 1 -- 172.21.15.97:0/1673961987 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb168053a40 con 0x7fb17406b0c0 2023-07-06T03:47:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.388+0000 7fb17abff700 1 -- 172.21.15.97:0/1673961987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb1740c6260 con 0x7fb17406b0c0 2023-07-06T03:47:24.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.952+0000 7fb1717fa700 1 -- 172.21.15.97:0/1673961987 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb168057060 con 0x7fb17406b0c0 2023-07-06T03:47:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.956+0000 7fb17abff700 1 -- 172.21.15.97:0/1673961987 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb15806ace0 msgr2=0x7fb15806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.956+0000 7fb17abff700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb15806ace0 0x7fb15806d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb17415b560 tx=0x7fb160008040 comp rx=0 tx=0).stop 2023-07-06T03:47:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.957+0000 7fb17abff700 1 -- 172.21.15.97:0/1673961987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb17406b0c0 msgr2=0x7fb17415a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.957+0000 7fb17abff700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb17406b0c0 0x7fb17415a8f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c480 tx=0x7fb16800c840 comp rx=0 tx=0).stop 2023-07-06T03:47:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.957+0000 7fb17abff700 1 -- 172.21.15.97:0/1673961987 shutdown_connections 2023-07-06T03:47:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.957+0000 7fb17abff700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb15806ace0 0x7fb15806d190 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.957+0000 7fb17abff700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb1740b85f0 0x7fb1741f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:24.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.957+0000 7fb17abff700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb17406b0c0 0x7fb17415a8f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:24.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.957+0000 7fb17abff700 1 --2- 172.21.15.97:0/1673961987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1740b4ca0 0x7fb1740c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:24.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.957+0000 7fb17abff700 1 -- 172.21.15.97:0/1673961987 >> 172.21.15.97:0/1673961987 conn(0x7fb17406a5b0 msgr2=0x7fb17415c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:24.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.957+0000 7fb17abff700 1 -- 172.21.15.97:0/1673961987 shutdown_connections 2023-07-06T03:47:24.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:24.957+0000 7fb17abff700 1 -- 172.21.15.97:0/1673961987 wait complete. 2023-07-06T03:47:24.960 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-07-06T03:47:24.981 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T03:47:25.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.179+0000 7faf319ff700 1 Processor -- start 2023-07-06T03:47:25.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.180+0000 7faf319ff700 1 -- start start 2023-07-06T03:47:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.181+0000 7faf319ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf2c160b40 0x7faf2c0cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.181+0000 7faf319ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faf2c160f10 0x7faf2c0cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.181+0000 7faf319ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf2c0ce1a0 0x7faf2c0ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.181+0000 7faf319ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf2c0cd140 con 0x7faf2c160b40 2023-07-06T03:47:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.181+0000 7faf319ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf2c0cd3e0 con 0x7faf2c160f10 2023-07-06T03:47:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.181+0000 7faf319ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0cd6c0 con 0x7faf2c0ce1a0 2023-07-06T03:47:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.181+0000 7faf2affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf2c160b40 0x7faf2c0cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.181+0000 7faf2affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf2c160b40 0x7faf2c0cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51626/0 (socket says 172.21.15.97:51626) 2023-07-06T03:47:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.181+0000 7faf2affd700 1 -- 172.21.15.97:0/2473201562 learned_addr learned my addr 172.21.15.97:0/2473201562 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:25.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.181+0000 7faf2b7fe700 1 --2- 172.21.15.97:0/2473201562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf2c0ce1a0 0x7faf2c0ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:25.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.181+0000 7faf23fff700 1 --2- 172.21.15.97:0/2473201562 >> v2:172.21.15.146:3300/0 conn(0x7faf2c160f10 0x7faf2c0cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:25.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.182+0000 7faf2affd700 1 -- 172.21.15.97:0/2473201562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf2c0ce1a0 msgr2=0x7faf2c0ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.182+0000 7faf2affd700 1 --2- 172.21.15.97:0/2473201562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf2c0ce1a0 0x7faf2c0ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.182+0000 7faf2affd700 1 -- 172.21.15.97:0/2473201562 >> v2:172.21.15.146:3300/0 conn(0x7faf2c160f10 msgr2=0x7faf2c0cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.182+0000 7faf2affd700 1 --2- 172.21.15.97:0/2473201562 >> v2:172.21.15.146:3300/0 conn(0x7faf2c160f10 0x7faf2c0cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.182+0000 7faf2affd700 1 -- 172.21.15.97:0/2473201562 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c0cd9a0 con 0x7faf2c160b40 2023-07-06T03:47:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.182+0000 7faf2affd700 1 --2- 172.21.15.97:0/2473201562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c160b40 0x7faf2c0cb010 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7faf2c066170 tx=0x7faf1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b3b33f46abb53 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.182+0000 7faf2a7fc700 1 -- 172.21.15.97:0/2473201562 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00b870 con 0x7faf2c160b40 2023-07-06T03:47:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.182+0000 7faf2a7fc700 1 -- 172.21.15.97:0/2473201562 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c00b9d0 con 0x7faf2c160b40 2023-07-06T03:47:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.182+0000 7faf2a7fc700 1 -- 172.21.15.97:0/2473201562 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00bd60 con 0x7faf2c160b40 2023-07-06T03:47:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.183+0000 7faf319ff700 1 -- 172.21.15.97:0/2473201562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c160b40 msgr2=0x7faf2c0cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.183+0000 7faf319ff700 1 --2- 172.21.15.97:0/2473201562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c160b40 0x7faf2c0cb010 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7faf2c066170 tx=0x7faf1c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.183+0000 7faf319ff700 1 -- 172.21.15.97:0/2473201562 shutdown_connections 2023-07-06T03:47:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.183+0000 7faf319ff700 1 --2- 172.21.15.97:0/2473201562 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf2c0ce1a0 0x7faf2c0ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.183+0000 7faf319ff700 1 --2- 172.21.15.97:0/2473201562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c160b40 0x7faf2c0cb010 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.183+0000 7faf319ff700 1 --2- 172.21.15.97:0/2473201562 >> v2:172.21.15.146:3300/0 conn(0x7faf2c160f10 0x7faf2c0cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.183+0000 7faf319ff700 1 -- 172.21.15.97:0/2473201562 >> 172.21.15.97:0/2473201562 conn(0x7faf2c0cbc20 msgr2=0x7faf2c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.183+0000 7faf319ff700 1 -- 172.21.15.97:0/2473201562 shutdown_connections 2023-07-06T03:47:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.183+0000 7faf319ff700 1 -- 172.21.15.97:0/2473201562 wait complete. 2023-07-06T03:47:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.184+0000 7faf319ff700 1 Processor -- start 2023-07-06T03:47:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.185+0000 7faf319ff700 1 -- start start 2023-07-06T03:47:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.185+0000 7faf319ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf2c0ce1a0 0x7faf2c1f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.185+0000 7faf319ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf2c160b40 0x7faf2c14a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf319ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c160f10 0x7faf2c14a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf319ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c1f1f40 con 0x7faf2c160f10 2023-07-06T03:47:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf319ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0b4820 con 0x7faf2c160b40 2023-07-06T03:47:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf319ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c1484f0 con 0x7faf2c0ce1a0 2023-07-06T03:47:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf2b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c160f10 0x7faf2c14a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf2b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c160f10 0x7faf2c14a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51634/0 (socket says 172.21.15.97:51634) 2023-07-06T03:47:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf2b7fe700 1 -- 172.21.15.97:0/3858192536 learned_addr learned my addr 172.21.15.97:0/3858192536 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf2affd700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf2c0ce1a0 0x7faf2c1f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf23fff700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf2c160b40 0x7faf2c14a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf2b7fe700 1 -- 172.21.15.97:0/3858192536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf2c0ce1a0 msgr2=0x7faf2c1f1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf2b7fe700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf2c0ce1a0 0x7faf2c1f1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf2b7fe700 1 -- 172.21.15.97:0/3858192536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf2c160b40 msgr2=0x7faf2c14a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf2b7fe700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf2c160b40 0x7faf2c14a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf2b7fe700 1 -- 172.21.15.97:0/3858192536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c009550 con 0x7faf2c160f10 2023-07-06T03:47:25.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.186+0000 7faf2b7fe700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c160f10 0x7faf2c14a940 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faf24007e60 tx=0x7faf2400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:25.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.187+0000 7faf28ff9700 1 -- 172.21.15.97:0/3858192536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2400ed30 con 0x7faf2c160f10 2023-07-06T03:47:25.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.187+0000 7faf28ff9700 1 -- 172.21.15.97:0/3858192536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf24014420 con 0x7faf2c160f10 2023-07-06T03:47:25.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.187+0000 7faf28ff9700 1 -- 172.21.15.97:0/3858192536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf240147d0 con 0x7faf2c160f10 2023-07-06T03:47:25.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.187+0000 7faf319ff700 1 -- 172.21.15.97:0/3858192536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2c1487d0 con 0x7faf2c160f10 2023-07-06T03:47:25.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.187+0000 7faf319ff700 1 -- 172.21.15.97:0/3858192536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2c0b3b00 con 0x7faf2c160f10 2023-07-06T03:47:25.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.188+0000 7faf319ff700 1 -- 172.21.15.97:0/3858192536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c04e340 con 0x7faf2c160f10 2023-07-06T03:47:25.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.189+0000 7faf28ff9700 1 -- 172.21.15.97:0/3858192536 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf24014930 con 0x7faf2c160f10 2023-07-06T03:47:25.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.190+0000 7faf28ff9700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf0c06ace0 0x7faf0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:25.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.190+0000 7faf28ff9700 1 -- 172.21.15.97:0/3858192536 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5797+0+0 (secure 0 0 0) 0x7faf24089980 con 0x7faf2c160f10 2023-07-06T03:47:25.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.190+0000 7faf2affd700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf0c06ace0 0x7faf0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:25.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.191+0000 7faf2affd700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf0c06ace0 0x7faf0c06d190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faf1c000c00 tx=0x7faf1c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:25.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.194+0000 7faf28ff9700 1 -- 172.21.15.97:0/3858192536 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faf24054b10 con 0x7faf2c160f10 2023-07-06T03:47:25.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.400+0000 7faf319ff700 1 -- 172.21.15.97:0/3858192536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faf2c148cf0 con 0x7faf2c160f10 2023-07-06T03:47:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.954+0000 7faf28ff9700 1 -- 172.21.15.97:0/3858192536 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7faf2c148cf0 con 0x7faf2c160f10 2023-07-06T03:47:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.957+0000 7faf319ff700 1 -- 172.21.15.97:0/3858192536 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf0c06ace0 msgr2=0x7faf0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.957+0000 7faf319ff700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf0c06ace0 0x7faf0c06d190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faf1c000c00 tx=0x7faf1c008030 comp rx=0 tx=0).stop 2023-07-06T03:47:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.957+0000 7faf319ff700 1 -- 172.21.15.97:0/3858192536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c160f10 msgr2=0x7faf2c14a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.957+0000 7faf319ff700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c160f10 0x7faf2c14a940 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faf24007e60 tx=0x7faf2400e3c0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.957+0000 7faf319ff700 1 -- 172.21.15.97:0/3858192536 shutdown_connections 2023-07-06T03:47:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.957+0000 7faf319ff700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf0c06ace0 0x7faf0c06d190 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.957+0000 7faf319ff700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c160f10 0x7faf2c14a940 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.957+0000 7faf319ff700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf2c0ce1a0 0x7faf2c1f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.957+0000 7faf319ff700 1 --2- 172.21.15.97:0/3858192536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf2c160b40 0x7faf2c14a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.957+0000 7faf319ff700 1 -- 172.21.15.97:0/3858192536 >> 172.21.15.97:0/3858192536 conn(0x7faf2c0cbc20 msgr2=0x7faf2c159960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.957+0000 7faf319ff700 1 -- 172.21.15.97:0/3858192536 shutdown_connections 2023-07-06T03:47:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:25.958+0000 7faf319ff700 1 -- 172.21.15.97:0/3858192536 wait complete. 2023-07-06T03:47:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-07-06T03:47:25.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2023-07-06T03:47:26.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.192+0000 7fa8cc1ff700 1 Processor -- start 2023-07-06T03:47:26.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.193+0000 7fa8cc1ff700 1 -- start start 2023-07-06T03:47:26.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.194+0000 7fa8cc1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 0x7fa8c40cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.194+0000 7fa8cc1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa8c415c590 0x7fa8c415d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.194+0000 7fa8cc1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa8c40719c0 0x7fa8c406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.194+0000 7fa8cc1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c40ccc30 con 0x7fa8c40719c0 2023-07-06T03:47:26.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.194+0000 7fa8cc1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c40ccf10 con 0x7fa8c415c590 2023-07-06T03:47:26.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.194+0000 7fa8cc1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40cd1f0 con 0x7fa8c415c1c0 2023-07-06T03:47:26.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.194+0000 7fa8ca79c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa8c40719c0 0x7fa8c406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.194+0000 7fa8ca79c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa8c40719c0 0x7fa8c406a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51642/0 (socket says 172.21.15.97:51642) 2023-07-06T03:47:26.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.194+0000 7fa8ca79c700 1 -- 172.21.15.97:0/3724458122 learned_addr learned my addr 172.21.15.97:0/3724458122 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:26.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.194+0000 7fa8c9f9b700 1 --2- 172.21.15.97:0/3724458122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 0x7fa8c40cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.194+0000 7fa8c979a700 1 --2- 172.21.15.97:0/3724458122 >> v2:172.21.15.146:3300/0 conn(0x7fa8c415c590 0x7fa8c415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.195+0000 7fa8ca79c700 1 -- 172.21.15.97:0/3724458122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 msgr2=0x7fa8c40cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:26.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.195+0000 7fa8ca79c700 1 --2- 172.21.15.97:0/3724458122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 0x7fa8c40cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.195+0000 7fa8ca79c700 1 -- 172.21.15.97:0/3724458122 >> v2:172.21.15.146:3300/0 conn(0x7fa8c415c590 msgr2=0x7fa8c415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:26.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.195+0000 7fa8ca79c700 1 --2- 172.21.15.97:0/3724458122 >> v2:172.21.15.146:3300/0 conn(0x7fa8c415c590 0x7fa8c415d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.195+0000 7fa8ca79c700 1 -- 172.21.15.97:0/3724458122 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c415cb00 con 0x7fa8c40719c0 2023-07-06T03:47:26.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.195+0000 7fa8c9f9b700 1 --2- 172.21.15.97:0/3724458122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 0x7fa8c40cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:26.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.195+0000 7fa8ca79c700 1 --2- 172.21.15.97:0/3724458122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c40719c0 0x7fa8c406a060 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40cd4b0 tx=0x7fa8bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf481d7bd22e60b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:26.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.195+0000 7fa8c8f99700 1 -- 172.21.15.97:0/3724458122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00e070 con 0x7fa8c40719c0 2023-07-06T03:47:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.195+0000 7fa8c8f99700 1 -- 172.21.15.97:0/3724458122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc00b9c0 con 0x7fa8c40719c0 2023-07-06T03:47:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.195+0000 7fa8c8f99700 1 -- 172.21.15.97:0/3724458122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00bd70 con 0x7fa8c40719c0 2023-07-06T03:47:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.196+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/3724458122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c40719c0 msgr2=0x7fa8c406a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.196+0000 7fa8cc1ff700 1 --2- 172.21.15.97:0/3724458122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c40719c0 0x7fa8c406a060 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40cd4b0 tx=0x7fa8bc00b330 comp rx=0 tx=0).stop 2023-07-06T03:47:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.196+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/3724458122 shutdown_connections 2023-07-06T03:47:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.196+0000 7fa8cc1ff700 1 --2- 172.21.15.97:0/3724458122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c40719c0 0x7fa8c406a060 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.196+0000 7fa8cc1ff700 1 --2- 172.21.15.97:0/3724458122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 0x7fa8c40cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.196+0000 7fa8cc1ff700 1 --2- 172.21.15.97:0/3724458122 >> v2:172.21.15.146:3300/0 conn(0x7fa8c415c590 0x7fa8c415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.196+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/3724458122 >> 172.21.15.97:0/3724458122 conn(0x7fa8c40b1650 msgr2=0x7fa8c40b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:26.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.196+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/3724458122 shutdown_connections 2023-07-06T03:47:26.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.196+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/3724458122 wait complete. 2023-07-06T03:47:26.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.198+0000 7fa8cc1ff700 1 Processor -- start 2023-07-06T03:47:26.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.198+0000 7fa8cc1ff700 1 -- start start 2023-07-06T03:47:26.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8cc1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c40719c0 0x7fa8c40b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8cc1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 0x7fa8c40b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8cc1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8c415c590 0x7fa8c414d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8cc1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40c6af0 con 0x7fa8c40719c0 2023-07-06T03:47:26.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8cc1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40b40a0 con 0x7fa8c415c590 2023-07-06T03:47:26.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8cc1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40b4380 con 0x7fa8c415c1c0 2023-07-06T03:47:26.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8c9f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c40719c0 0x7fa8c40b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8c9f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c40719c0 0x7fa8c40b53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51650/0 (socket says 172.21.15.97:51650) 2023-07-06T03:47:26.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8c9f9b700 1 -- 172.21.15.97:0/1687675736 learned_addr learned my addr 172.21.15.97:0/1687675736 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:26.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8ca79c700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8c415c590 0x7fa8c414d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8c9f9b700 1 -- 172.21.15.97:0/1687675736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 msgr2=0x7fa8c40b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8c979a700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 0x7fa8c40b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8c9f9b700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 0x7fa8c40b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8c9f9b700 1 -- 172.21.15.97:0/1687675736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8c415c590 msgr2=0x7fa8c414d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8c9f9b700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8c415c590 0x7fa8c414d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8c9f9b700 1 -- 172.21.15.97:0/1687675736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc009b30 con 0x7fa8c40719c0 2023-07-06T03:47:26.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8ca79c700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8c415c590 0x7fa8c414d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:26.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8c979a700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 0x7fa8c40b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:26.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.200+0000 7fa8c9f9b700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c40719c0 0x7fa8c40b53d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000d800 tx=0x7fa8c000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:26.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.201+0000 7fa8ba7fc700 1 -- 172.21.15.97:0/1687675736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0004030 con 0x7fa8c40719c0 2023-07-06T03:47:26.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.201+0000 7fa8ba7fc700 1 -- 172.21.15.97:0/1687675736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c0004190 con 0x7fa8c40719c0 2023-07-06T03:47:26.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.201+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/1687675736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c40b4660 con 0x7fa8c40719c0 2023-07-06T03:47:26.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.201+0000 7fa8ba7fc700 1 -- 172.21.15.97:0/1687675736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0004520 con 0x7fa8c40719c0 2023-07-06T03:47:26.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.201+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/1687675736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c414d910 con 0x7fa8c40719c0 2023-07-06T03:47:26.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.202+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/1687675736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c404e340 con 0x7fa8c40719c0 2023-07-06T03:47:26.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.203+0000 7fa8ba7fc700 1 -- 172.21.15.97:0/1687675736 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa8c00036a0 con 0x7fa8c40719c0 2023-07-06T03:47:26.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.204+0000 7fa8ba7fc700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a806ace0 0x7fa8a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.204+0000 7fa8ba7fc700 1 -- 172.21.15.97:0/1687675736 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa8c00897b0 con 0x7fa8c40719c0 2023-07-06T03:47:26.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.204+0000 7fa8c979a700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a806ace0 0x7fa8a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.204+0000 7fa8c979a700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a806ace0 0x7fa8a806d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40b2090 tx=0x7fa8b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:26.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.208+0000 7fa8ba7fc700 1 -- 172.21.15.97:0/1687675736 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa8c0058570 con 0x7fa8c40719c0 2023-07-06T03:47:26.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.406+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/1687675736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) v1 -- 0x7fa8c4071290 con 0x7fa8c40719c0 2023-07-06T03:47:26.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.410+0000 7fa8ba7fc700 1 -- 172.21.15.97:0/1687675736 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v13) v1 ==== 106+0+0 (secure 0 0 0) 0x7fa8c0058100 con 0x7fa8c40719c0 2023-07-06T03:47:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/1687675736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a806ace0 msgr2=0x7fa8a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a806ace0 0x7fa8a806d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40b2090 tx=0x7fa8b4008040 comp rx=0 tx=0).stop 2023-07-06T03:47:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/1687675736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c40719c0 msgr2=0x7fa8c40b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c40719c0 0x7fa8c40b53d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000d800 tx=0x7fa8c000dbc0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/1687675736 shutdown_connections 2023-07-06T03:47:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8a806ace0 0x7fa8a806d190 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8c415c590 0x7fa8c414d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c40719c0 0x7fa8c40b53d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 --2- 172.21.15.97:0/1687675736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8c415c1c0 0x7fa8c40b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/1687675736 >> 172.21.15.97:0/1687675736 conn(0x7fa8c40b1650 msgr2=0x7fa8c40c4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/1687675736 shutdown_connections 2023-07-06T03:47:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.412+0000 7fa8cc1ff700 1 -- 172.21.15.97:0/1687675736 wait complete. 2023-07-06T03:47:26.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T03:47:26.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.626+0000 7fef1e9ff700 1 Processor -- start 2023-07-06T03:47:26.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.626+0000 7fef1e9ff700 1 -- start start 2023-07-06T03:47:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.627+0000 7fef1e9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef1815c610 0x7fef180b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.627+0000 7fef1e9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef180cff90 con 0x7fef180d2210 2023-07-06T03:47:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.627+0000 7fef1e9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef180b5650 con 0x7fef1806b360 2023-07-06T03:47:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.627+0000 7fef1e9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef180d0780 con 0x7fef1815c610 2023-07-06T03:47:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.627+0000 7fef1cf9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fef180d2210 0x7fef180d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42156/0 (socket says 172.21.15.97:42156) 2023-07-06T03:47:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.627+0000 7fef1cf9c700 1 -- 172.21.15.97:0/2341649942 learned_addr learned my addr 172.21.15.97:0/2341649942 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.628+0000 7fef17fff700 1 --2- 172.21.15.97:0/2341649942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef1815c610 0x7fef180b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.628+0000 7fef177fe700 1 -- 172.21.15.97:0/2341649942 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791650514 0 0) 0x7fef180cff90 con 0x7fef180d2210 2023-07-06T03:47:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.628+0000 7fef177fe700 1 -- 172.21.15.97:0/2341649942 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef040035b0 con 0x7fef180d2210 2023-07-06T03:47:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.628+0000 7fef177fe700 1 -- 172.21.15.97:0/2341649942 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466595731 0 0) 0x7fef180b5650 con 0x7fef1806b360 2023-07-06T03:47:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.628+0000 7fef177fe700 1 -- 172.21.15.97:0/2341649942 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef180cff90 con 0x7fef1806b360 2023-07-06T03:47:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.628+0000 7fef17fff700 1 -- 172.21.15.97:0/2341649942 >> v1:172.21.15.146:6789/0 conn(0x7fef1806b360 legacy=0x7fef180b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.628+0000 7fef17fff700 1 -- 172.21.15.97:0/2341649942 >> v1:172.21.15.97:6789/0 conn(0x7fef180d2210 legacy=0x7fef180d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.628+0000 7fef17fff700 1 -- 172.21.15.97:0/2341649942 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef180b5400 con 0x7fef1815c610 2023-07-06T03:47:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.628+0000 7fef17fff700 1 --2- 172.21.15.97:0/2341649942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef1815c610 0x7fef180b47b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef08009870 tx=0x7fef08009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593f6b4d403f50c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.629+0000 7fef177fe700 1 -- 172.21.15.97:0/2341649942 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08011830 con 0x7fef1815c610 2023-07-06T03:47:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.629+0000 7fef177fe700 1 -- 172.21.15.97:0/2341649942 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef08011990 con 0x7fef1815c610 2023-07-06T03:47:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.629+0000 7fef177fe700 1 -- 172.21.15.97:0/2341649942 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08011d20 con 0x7fef1815c610 2023-07-06T03:47:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.629+0000 7fef1e9ff700 1 -- 172.21.15.97:0/2341649942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef1815c610 msgr2=0x7fef180b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.629+0000 7fef1e9ff700 1 --2- 172.21.15.97:0/2341649942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef1815c610 0x7fef180b47b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef08009870 tx=0x7fef08009b80 comp rx=0 tx=0).stop 2023-07-06T03:47:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.629+0000 7fef1e9ff700 1 -- 172.21.15.97:0/2341649942 shutdown_connections 2023-07-06T03:47:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.629+0000 7fef1e9ff700 1 --2- 172.21.15.97:0/2341649942 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef1815c610 0x7fef180b47b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.629+0000 7fef1e9ff700 1 -- 172.21.15.97:0/2341649942 >> 172.21.15.97:0/2341649942 conn(0x7fef1806a6c0 msgr2=0x7fef1806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.629+0000 7fef1e9ff700 1 -- 172.21.15.97:0/2341649942 shutdown_connections 2023-07-06T03:47:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.630+0000 7fef1e9ff700 1 -- 172.21.15.97:0/2341649942 wait complete. 2023-07-06T03:47:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.631+0000 7fef1e9ff700 1 Processor -- start 2023-07-06T03:47:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.631+0000 7fef1e9ff700 1 -- start start 2023-07-06T03:47:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.632+0000 7fef1e9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef1806b360 0x7fef1814b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.632+0000 7fef1e9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef180d2210 0x7fef1815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.632+0000 7fef1e9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef1815c610 0x7fef180c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.632+0000 7fef1e9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef180c91e0 con 0x7fef180d2210 2023-07-06T03:47:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.632+0000 7fef1e9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1814be00 con 0x7fef1815c610 2023-07-06T03:47:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.632+0000 7fef1e9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef180c8560 con 0x7fef1806b360 2023-07-06T03:47:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef0ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef180d2210 0x7fef1815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef17fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef1806b360 0x7fef1814b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef0ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef180d2210 0x7fef1815f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51664/0 (socket says 172.21.15.97:51664) 2023-07-06T03:47:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef0ffff700 1 -- 172.21.15.97:0/3491969266 learned_addr learned my addr 172.21.15.97:0/3491969266 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef1cf9c700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef1815c610 0x7fef180c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef0ffff700 1 -- 172.21.15.97:0/3491969266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef1806b360 msgr2=0x7fef1814b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef0ffff700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef1806b360 0x7fef1814b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef0ffff700 1 -- 172.21.15.97:0/3491969266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef1815c610 msgr2=0x7fef180c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef0ffff700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef1815c610 0x7fef180c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef0ffff700 1 -- 172.21.15.97:0/3491969266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef080094a0 con 0x7fef180d2210 2023-07-06T03:47:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef0ffff700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef180d2210 0x7fef1815f3e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fef0000c500 tx=0x7fef0000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef157fa700 1 -- 172.21.15.97:0/3491969266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef000052d0 con 0x7fef180d2210 2023-07-06T03:47:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef157fa700 1 -- 172.21.15.97:0/3491969266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef00005430 con 0x7fef180d2210 2023-07-06T03:47:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.633+0000 7fef157fa700 1 -- 172.21.15.97:0/3491969266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef000038c0 con 0x7fef180d2210 2023-07-06T03:47:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.634+0000 7fef1e9ff700 1 -- 172.21.15.97:0/3491969266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef180c8840 con 0x7fef180d2210 2023-07-06T03:47:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.634+0000 7fef1e9ff700 1 -- 172.21.15.97:0/3491969266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef180c8e00 con 0x7fef180d2210 2023-07-06T03:47:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.635+0000 7fef1e9ff700 1 -- 172.21.15.97:0/3491969266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef180d01a0 con 0x7fef180d2210 2023-07-06T03:47:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.636+0000 7fef157fa700 1 -- 172.21.15.97:0/3491969266 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef00003a20 con 0x7fef180d2210 2023-07-06T03:47:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.638+0000 7fef157fa700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 0x7feef806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.638+0000 7fef157fa700 1 -- 172.21.15.97:0/3491969266 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fef000886a0 con 0x7fef180d2210 2023-07-06T03:47:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.638+0000 7fef17fff700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 0x7feef806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.639+0000 7fef17fff700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 0x7feef806d190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fef080090b0 tx=0x7fef0801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:26.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.641+0000 7fef157fa700 1 -- 172.21.15.97:0/3491969266 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fef00053ca0 con 0x7fef180d2210 2023-07-06T03:47:26.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.838+0000 7fef1e9ff700 1 -- 172.21.15.97:0/3491969266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fef180c9320 con 0x7fef180d2210 2023-07-06T03:47:26.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.842+0000 7fef157fa700 1 -- 172.21.15.97:0/3491969266 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7fef180c9320 con 0x7fef180d2210 2023-07-06T03:47:26.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 -- 172.21.15.97:0/3491969266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 msgr2=0x7feef806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 0x7feef806d190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fef080090b0 tx=0x7fef0801c040 comp rx=0 tx=0).stop 2023-07-06T03:47:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 -- 172.21.15.97:0/3491969266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef180d2210 msgr2=0x7fef1815f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef180d2210 0x7fef1815f3e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fef0000c500 tx=0x7fef0000c810 comp rx=0 tx=0).stop 2023-07-06T03:47:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 -- 172.21.15.97:0/3491969266 shutdown_connections 2023-07-06T03:47:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 0x7feef806d190 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef1815c610 0x7fef180c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef1806b360 0x7fef1814b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 --2- 172.21.15.97:0/3491969266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef180d2210 0x7fef1815f3e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 -- 172.21.15.97:0/3491969266 >> 172.21.15.97:0/3491969266 conn(0x7fef1806a6c0 msgr2=0x7fef1815b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 -- 172.21.15.97:0/3491969266 shutdown_connections 2023-07-06T03:47:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:26.845+0000 7fef1e9ff700 1 -- 172.21.15.97:0/3491969266 wait complete. 2023-07-06T03:47:26.864 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_cap_acquisition_throttle 2023-07-06T03:47:27.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.060+0000 7f26bafff700 1 Processor -- start 2023-07-06T03:47:27.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.061+0000 7f26bafff700 1 -- start start 2023-07-06T03:47:27.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.062+0000 7f26bafff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f26b40b4a40 0x7f26b40d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:27.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.062+0000 7f26bafff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b40d2d30 con 0x7f26b40b5e10 2023-07-06T03:47:27.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.062+0000 7f26bafff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b40cf210 con 0x7f26b40b6280 2023-07-06T03:47:27.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.062+0000 7f26bafff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b40cf4b0 con 0x7f26b40b4a40 2023-07-06T03:47:27.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.062+0000 7f26b8d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f26b40b4a40 0x7f26b40d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:27.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.062+0000 7f26b8d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f26b40b4a40 0x7f26b40d15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51668/0 (socket says 172.21.15.97:51668) 2023-07-06T03:47:27.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.062+0000 7f26b8d9b700 1 -- 172.21.15.97:0/3395433834 learned_addr learned my addr 172.21.15.97:0/3395433834 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:27.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.062+0000 7f26b3fff700 1 --1- 172.21.15.97:0/3395433834 >> v1:172.21.15.97:6789/0 conn(0x7f26b40b5e10 0x7f26b40cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42166/0 (socket says 172.21.15.97:42166) 2023-07-06T03:47:27.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.063+0000 7f26b8d9b700 1 -- 172.21.15.97:0/3395433834 >> v1:172.21.15.146:6789/0 conn(0x7f26b40b6280 legacy=0x7f26b40b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:27.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.063+0000 7f26b8d9b700 1 -- 172.21.15.97:0/3395433834 >> v1:172.21.15.97:6789/0 conn(0x7f26b40b5e10 legacy=0x7f26b40cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.063+0000 7f26b8d9b700 1 -- 172.21.15.97:0/3395433834 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b40cdd70 con 0x7f26b40b4a40 2023-07-06T03:47:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.063+0000 7f26b8d9b700 1 --2- 172.21.15.97:0/3395433834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40b4a40 0x7f26b40d15a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f26a80098f0 tx=0x7f26a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d273aaa785c67347 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.063+0000 7f26b37fe700 1 -- 172.21.15.97:0/3395433834 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8004230 con 0x7f26b40b4a40 2023-07-06T03:47:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.063+0000 7f26b37fe700 1 -- 172.21.15.97:0/3395433834 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a8004390 con 0x7f26b40b4a40 2023-07-06T03:47:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.063+0000 7f26b37fe700 1 -- 172.21.15.97:0/3395433834 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a80036a0 con 0x7f26b40b4a40 2023-07-06T03:47:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.064+0000 7f26bafff700 1 -- 172.21.15.97:0/3395433834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40b4a40 msgr2=0x7f26b40d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.064+0000 7f26bafff700 1 --2- 172.21.15.97:0/3395433834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40b4a40 0x7f26b40d15a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f26a80098f0 tx=0x7f26a8009c00 comp rx=0 tx=0).stop 2023-07-06T03:47:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.064+0000 7f26bafff700 1 -- 172.21.15.97:0/3395433834 shutdown_connections 2023-07-06T03:47:27.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.064+0000 7f26bafff700 1 --2- 172.21.15.97:0/3395433834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40b4a40 0x7f26b40d15a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.064+0000 7f26bafff700 1 -- 172.21.15.97:0/3395433834 >> 172.21.15.97:0/3395433834 conn(0x7f26b406a900 msgr2=0x7f26b406acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:27.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.064+0000 7f26bafff700 1 -- 172.21.15.97:0/3395433834 shutdown_connections 2023-07-06T03:47:27.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.064+0000 7f26bafff700 1 -- 172.21.15.97:0/3395433834 wait complete. 2023-07-06T03:47:27.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.065+0000 7f26bafff700 1 Processor -- start 2023-07-06T03:47:27.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.066+0000 7f26bafff700 1 -- start start 2023-07-06T03:47:27.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.066+0000 7f26bafff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26b40b4a40 0x7f26b40cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:27.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.066+0000 7f26bafff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26b40b5e10 0x7f26b40ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:27.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.066+0000 7f26bafff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40b6280 0x7f26b40ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:27.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.066+0000 7f26bafff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40cd250 con 0x7f26b40b6280 2023-07-06T03:47:27.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.066+0000 7f26bafff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b414b560 con 0x7f26b40b4a40 2023-07-06T03:47:27.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26bafff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40cc640 con 0x7f26b40b5e10 2023-07-06T03:47:27.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b959c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40b6280 0x7f26b40ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:27.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b959c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40b6280 0x7f26b40ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51680/0 (socket says 172.21.15.97:51680) 2023-07-06T03:47:27.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b959c700 1 -- 172.21.15.97:0/912510962 learned_addr learned my addr 172.21.15.97:0/912510962 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:27.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b8d9b700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26b40b4a40 0x7f26b40cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b3fff700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26b40b5e10 0x7f26b40ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b959c700 1 -- 172.21.15.97:0/912510962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26b40b5e10 msgr2=0x7f26b40ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b959c700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26b40b5e10 0x7f26b40ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b959c700 1 -- 172.21.15.97:0/912510962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26b40b4a40 msgr2=0x7f26b40cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b959c700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26b40b4a40 0x7f26b40cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b959c700 1 -- 172.21.15.97:0/912510962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a8009550 con 0x7f26b40b6280 2023-07-06T03:47:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b8d9b700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26b40b4a40 0x7f26b40cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b3fff700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26b40b5e10 0x7f26b40ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.067+0000 7f26b959c700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40b6280 0x7f26b40ca5d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00c780 tx=0x7f26ac00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.068+0000 7f26b17fa700 1 -- 172.21.15.97:0/912510962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac003e10 con 0x7f26b40b6280 2023-07-06T03:47:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.068+0000 7f26b17fa700 1 -- 172.21.15.97:0/912510962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac005070 con 0x7f26b40b6280 2023-07-06T03:47:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.068+0000 7f26bafff700 1 -- 172.21.15.97:0/912510962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b40cc920 con 0x7f26b40b6280 2023-07-06T03:47:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.068+0000 7f26b17fa700 1 -- 172.21.15.97:0/912510962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac005420 con 0x7f26b40b6280 2023-07-06T03:47:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.068+0000 7f26bafff700 1 -- 172.21.15.97:0/912510962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b40ccee0 con 0x7f26b40b6280 2023-07-06T03:47:27.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.069+0000 7f26bafff700 1 -- 172.21.15.97:0/912510962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b40cd5e0 con 0x7f26b40b6280 2023-07-06T03:47:27.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.070+0000 7f26b17fa700 1 -- 172.21.15.97:0/912510962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f26ac005580 con 0x7f26b40b6280 2023-07-06T03:47:27.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.071+0000 7f26b17fa700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f269806ace0 0x7f269806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:27.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.071+0000 7f26b17fa700 1 -- 172.21.15.97:0/912510962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f26ac088550 con 0x7f26b40b6280 2023-07-06T03:47:27.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.071+0000 7f26b8d9b700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f269806ace0 0x7f269806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:27.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.072+0000 7f26b8d9b700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f269806ace0 0x7f269806d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009c40 tx=0x7f26a8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:27.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.075+0000 7f26b17fa700 1 -- 172.21.15.97:0/912510962 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f26b40cd5e0 con 0x7f26b40b6280 2023-07-06T03:47:27.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.275+0000 7f26bafff700 1 -- 172.21.15.97:0/912510962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"} v 0) v1 -- 0x7f26b40cd5e0 con 0x7f26b40b6280 2023-07-06T03:47:27.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.279+0000 7f26b17fa700 1 -- 172.21.15.97:0/912510962 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"}]=0 v15) v1 ==== 119+0+0 (secure 0 0 0) 0x7f26b40cd5e0 con 0x7f26b40b6280 2023-07-06T03:47:27.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 -- 172.21.15.97:0/912510962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f269806ace0 msgr2=0x7f269806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f269806ace0 0x7f269806d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009c40 tx=0x7f26a8008030 comp rx=0 tx=0).stop 2023-07-06T03:47:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 -- 172.21.15.97:0/912510962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40b6280 msgr2=0x7f26b40ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40b6280 0x7f26b40ca5d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00c780 tx=0x7f26ac00cb40 comp rx=0 tx=0).stop 2023-07-06T03:47:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 -- 172.21.15.97:0/912510962 shutdown_connections 2023-07-06T03:47:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f269806ace0 0x7f269806d190 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b40b6280 0x7f26b40ca5d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26b40b4a40 0x7f26b40cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 --2- 172.21.15.97:0/912510962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f26b40b5e10 0x7f26b40ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 -- 172.21.15.97:0/912510962 >> 172.21.15.97:0/912510962 conn(0x7f26b406a900 msgr2=0x7f26b40b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:27.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 -- 172.21.15.97:0/912510962 shutdown_connections 2023-07-06T03:47:27.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.282+0000 7f26bafff700 1 -- 172.21.15.97:0/912510962 wait complete. 2023-07-06T03:47:27.302 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-07-06T03:47:27.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.493+0000 7f00b23ff700 1 Processor -- start 2023-07-06T03:47:27.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.493+0000 7f00b23ff700 1 -- start start 2023-07-06T03:47:27.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.494+0000 7f00b23ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f00ac15ff00 0x7f00ac0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:27.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.494+0000 7f00b23ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f00ac0c8d70 0x7f00ac0d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00b23ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ac0b4740 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00b23ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac0d1370 con 0x7f00ac0c8d70 2023-07-06T03:47:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00b23ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac1602d0 con 0x7f00ac15ff00 2023-07-06T03:47:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00ab7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f00ac0c8d70 0x7f00ac0d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00b099c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f00ac0d0ef0 0x7f00ac0c6930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42172/0 (socket says 172.21.15.97:42172) 2023-07-06T03:47:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00ab7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f00ac0c8d70 0x7f00ac0d09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51684/0 (socket says 172.21.15.97:51684) 2023-07-06T03:47:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00b099c700 1 -- 172.21.15.97:0/1989810124 learned_addr learned my addr 172.21.15.97:0/1989810124 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00abfff700 1 --2- 172.21.15.97:0/1989810124 >> v2:172.21.15.146:3300/0 conn(0x7f00ac15ff00 0x7f00ac0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00ab7fe700 1 -- 172.21.15.97:0/1989810124 >> v2:172.21.15.146:3300/0 conn(0x7f00ac15ff00 msgr2=0x7f00ac0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00ab7fe700 1 --2- 172.21.15.97:0/1989810124 >> v2:172.21.15.146:3300/0 conn(0x7f00ac15ff00 0x7f00ac0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00ab7fe700 1 -- 172.21.15.97:0/1989810124 >> v1:172.21.15.97:6789/0 conn(0x7f00ac0d0ef0 legacy=0x7f00ac0c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00ab7fe700 1 -- 172.21.15.97:0/1989810124 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0b60b0 con 0x7f00ac0c8d70 2023-07-06T03:47:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00aaffd700 1 -- 172.21.15.97:0/1989810124 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766742616 0 0) 0x7f00ac0b4740 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.495+0000 7f00abfff700 1 --2- 172.21.15.97:0/1989810124 >> v2:172.21.15.146:3300/0 conn(0x7f00ac15ff00 0x7f00ac0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00ab7fe700 1 --2- 172.21.15.97:0/1989810124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00ac0c8d70 0x7f00ac0d09b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f00ac08b410 tx=0x7f00a0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57256342d29b5a5b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00aaffd700 1 -- 172.21.15.97:0/1989810124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0011830 con 0x7f00ac0c8d70 2023-07-06T03:47:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00aaffd700 1 -- 172.21.15.97:0/1989810124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a0011990 con 0x7f00ac0c8d70 2023-07-06T03:47:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00aaffd700 1 -- 172.21.15.97:0/1989810124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0011d20 con 0x7f00ac0c8d70 2023-07-06T03:47:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00b23ff700 1 -- 172.21.15.97:0/1989810124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00ac0c8d70 msgr2=0x7f00ac0d09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00b23ff700 1 --2- 172.21.15.97:0/1989810124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00ac0c8d70 0x7f00ac0d09b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f00ac08b410 tx=0x7f00a0009b20 comp rx=0 tx=0).stop 2023-07-06T03:47:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00b23ff700 1 -- 172.21.15.97:0/1989810124 shutdown_connections 2023-07-06T03:47:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00b23ff700 1 --2- 172.21.15.97:0/1989810124 >> v2:172.21.15.146:3300/0 conn(0x7f00ac15ff00 0x7f00ac0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00b23ff700 1 --2- 172.21.15.97:0/1989810124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00ac0c8d70 0x7f00ac0d09b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00b23ff700 1 -- 172.21.15.97:0/1989810124 >> 172.21.15.97:0/1989810124 conn(0x7f00ac09c130 msgr2=0x7f00ac0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:27.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00b23ff700 1 -- 172.21.15.97:0/1989810124 shutdown_connections 2023-07-06T03:47:27.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.496+0000 7f00b23ff700 1 -- 172.21.15.97:0/1989810124 wait complete. 2023-07-06T03:47:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.498+0000 7f00b23ff700 1 Processor -- start 2023-07-06T03:47:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.498+0000 7f00b23ff700 1 -- start start 2023-07-06T03:47:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.499+0000 7f00b23ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac0c8d70 0x7f00ac0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.499+0000 7f00b23ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00ac0d0ef0 0x7f00ac1f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:27.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.499+0000 7f00b23ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00ac15ff00 0x7f00ac0b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:27.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.499+0000 7f00b23ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac09d6c0 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.499+0000 7f00b23ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac1f2ba0 con 0x7f00ac15ff00 2023-07-06T03:47:27.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.499+0000 7f00b23ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0a0330 con 0x7f00ac0c8d70 2023-07-06T03:47:27.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.499+0000 7f00ab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00ac0d0ef0 0x7f00ac1f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:27.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.499+0000 7f00ab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00ac0d0ef0 0x7f00ac1f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51690/0 (socket says 172.21.15.97:51690) 2023-07-06T03:47:27.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.499+0000 7f00ab7fe700 1 -- 172.21.15.97:0/2782936515 learned_addr learned my addr 172.21.15.97:0/2782936515 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:27.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00b099c700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00ac15ff00 0x7f00ac0b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:27.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00abfff700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac0c8d70 0x7f00ac0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:27.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00ab7fe700 1 -- 172.21.15.97:0/2782936515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac0c8d70 msgr2=0x7f00ac0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00ab7fe700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac0c8d70 0x7f00ac0cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00ab7fe700 1 -- 172.21.15.97:0/2782936515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00ac15ff00 msgr2=0x7f00ac0b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00ab7fe700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00ac15ff00 0x7f00ac0b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00ab7fe700 1 -- 172.21.15.97:0/2782936515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a00094a0 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00ab7fe700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00ac0d0ef0 0x7f00ac1f2660 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f00a0000c00 tx=0x7f00a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:27.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00a8ff9700 1 -- 172.21.15.97:0/2782936515 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a00038a0 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00a8ff9700 1 -- 172.21.15.97:0/2782936515 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a001f070 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00a8ff9700 1 -- 172.21.15.97:0/2782936515 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0003c60 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00b23ff700 1 -- 172.21.15.97:0/2782936515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac09f400 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.500+0000 7f00b23ff700 1 -- 172.21.15.97:0/2782936515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac09d000 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.501+0000 7f00b23ff700 1 -- 172.21.15.97:0/2782936515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ac09d930 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.502+0000 7f00a8ff9700 1 -- 172.21.15.97:0/2782936515 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f00a001a430 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.503+0000 7f00a8ff9700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f009806ace0 0x7f009806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:27.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.503+0000 7f00a8ff9700 1 -- 172.21.15.97:0/2782936515 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f00a0021070 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.503+0000 7f00abfff700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f009806ace0 0x7f009806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:27.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.504+0000 7f00abfff700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f009806ace0 0x7f009806d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f00ac1f6290 tx=0x7f009c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:27.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.508+0000 7f00a8ff9700 1 -- 172.21.15.97:0/2782936515 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f00a00560a0 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.782+0000 7f00b23ff700 1 -- 172.21.15.97:0/2782936515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7f00ac04e340 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.966+0000 7f00a8ff9700 1 -- 172.21.15.97:0/2782936515 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v147) v1 ==== 165+0+0 (secure 0 0 0) 0x7f00a00045b0 con 0x7f00ac0d0ef0 2023-07-06T03:47:27.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.968+0000 7f00b23ff700 1 -- 172.21.15.97:0/2782936515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f009806ace0 msgr2=0x7f009806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.968+0000 7f00b23ff700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f009806ace0 0x7f009806d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f00ac1f6290 tx=0x7f009c008040 comp rx=0 tx=0).stop 2023-07-06T03:47:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.968+0000 7f00b23ff700 1 -- 172.21.15.97:0/2782936515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00ac0d0ef0 msgr2=0x7f00ac1f2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.968+0000 7f00b23ff700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00ac0d0ef0 0x7f00ac1f2660 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f00a0000c00 tx=0x7f00a0004750 comp rx=0 tx=0).stop 2023-07-06T03:47:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.968+0000 7f00b23ff700 1 -- 172.21.15.97:0/2782936515 shutdown_connections 2023-07-06T03:47:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.969+0000 7f00b23ff700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f009806ace0 0x7f009806d190 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.969+0000 7f00b23ff700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f00ac15ff00 0x7f00ac0b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.969+0000 7f00b23ff700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f00ac0c8d70 0x7f00ac0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.969+0000 7f00b23ff700 1 --2- 172.21.15.97:0/2782936515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00ac0d0ef0 0x7f00ac1f2660 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.969+0000 7f00b23ff700 1 -- 172.21.15.97:0/2782936515 >> 172.21.15.97:0/2782936515 conn(0x7f00ac09c130 msgr2=0x7f00ac0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.969+0000 7f00b23ff700 1 -- 172.21.15.97:0/2782936515 shutdown_connections 2023-07-06T03:47:27.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:27.969+0000 7f00b23ff700 1 -- 172.21.15.97:0/2782936515 wait complete. 2023-07-06T03:47:27.988 INFO:tasks.cephfs_test_runner:test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-06T03:47:27.989 INFO:tasks.cephfs_test_runner:Starting test: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:47:27.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2023-07-06T03:47:28.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.183+0000 7f167c1ff700 1 Processor -- start 2023-07-06T03:47:28.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.183+0000 7f167c1ff700 1 -- start start 2023-07-06T03:47:28.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.184+0000 7f167c1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f167415cd70 0x7f16740c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:28.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.184+0000 7f167c1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16740ad960 con 0x7f1674158c10 2023-07-06T03:47:28.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.184+0000 7f167c1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f167415d5a0 con 0x7f16740adb30 2023-07-06T03:47:28.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.185+0000 7f167c1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1674159ff0 con 0x7f167415cd70 2023-07-06T03:47:28.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.185+0000 7f167979a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f167415cd70 0x7f16740c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:28.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.185+0000 7f1679f9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1674158c10 0x7f167415c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42176/0 (socket says 172.21.15.97:42176) 2023-07-06T03:47:28.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.185+0000 7f1679f9b700 1 -- 172.21.15.97:0/3184481478 learned_addr learned my addr 172.21.15.97:0/3184481478 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:28.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.185+0000 7f167979a700 1 -- 172.21.15.97:0/3184481478 >> v1:172.21.15.146:6789/0 conn(0x7f16740adb30 legacy=0x7f1674070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:28.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.185+0000 7f167979a700 1 -- 172.21.15.97:0/3184481478 >> v1:172.21.15.97:6789/0 conn(0x7f1674158c10 legacy=0x7f167415c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:28.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.185+0000 7f167979a700 1 -- 172.21.15.97:0/3184481478 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16740ad680 con 0x7f167415cd70 2023-07-06T03:47:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.185+0000 7f1678f99700 1 -- 172.21.15.97:0/3184481478 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267710341 0 0) 0x7f16740ad960 con 0x7f1674158c10 2023-07-06T03:47:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.185+0000 7f167979a700 1 --2- 172.21.15.97:0/3184481478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167415cd70 0x7f16740c8fe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1664009840 tx=0x7f1664009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9729dbd69d9ab61a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.186+0000 7f1678f99700 1 -- 172.21.15.97:0/3184481478 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664011830 con 0x7f167415cd70 2023-07-06T03:47:28.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.186+0000 7f1678f99700 1 -- 172.21.15.97:0/3184481478 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1664011990 con 0x7f167415cd70 2023-07-06T03:47:28.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.186+0000 7f1678f99700 1 -- 172.21.15.97:0/3184481478 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664011d20 con 0x7f167415cd70 2023-07-06T03:47:28.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.186+0000 7f167c1ff700 1 -- 172.21.15.97:0/3184481478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167415cd70 msgr2=0x7f16740c8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:28.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.186+0000 7f167c1ff700 1 --2- 172.21.15.97:0/3184481478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167415cd70 0x7f16740c8fe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1664009840 tx=0x7f1664009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:28.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.186+0000 7f167c1ff700 1 -- 172.21.15.97:0/3184481478 shutdown_connections 2023-07-06T03:47:28.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.186+0000 7f167c1ff700 1 --2- 172.21.15.97:0/3184481478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167415cd70 0x7f16740c8fe0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:28.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.186+0000 7f167c1ff700 1 -- 172.21.15.97:0/3184481478 >> 172.21.15.97:0/3184481478 conn(0x7f16740b04a0 msgr2=0x7f16740b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:28.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.186+0000 7f167c1ff700 1 -- 172.21.15.97:0/3184481478 shutdown_connections 2023-07-06T03:47:28.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.186+0000 7f167c1ff700 1 -- 172.21.15.97:0/3184481478 wait complete. 2023-07-06T03:47:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.188+0000 7f167c1ff700 1 Processor -- start 2023-07-06T03:47:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.188+0000 7f167c1ff700 1 -- start start 2023-07-06T03:47:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.189+0000 7f167c1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16740adb30 0x7f1674148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.189+0000 7f167c1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1674158c10 0x7f1674151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.189+0000 7f167c1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f167415cd70 0x7f167414ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.189+0000 7f167c1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1674158510 con 0x7f16740adb30 2023-07-06T03:47:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.189+0000 7f167c1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1674152300 con 0x7f1674158c10 2023-07-06T03:47:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.189+0000 7f167c1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1674157f50 con 0x7f167415cd70 2023-07-06T03:47:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.189+0000 7f1679f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16740adb30 0x7f1674148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.189+0000 7f1679f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16740adb30 0x7f1674148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51714/0 (socket says 172.21.15.97:51714) 2023-07-06T03:47:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.189+0000 7f1679f9b700 1 -- 172.21.15.97:0/3500035660 learned_addr learned my addr 172.21.15.97:0/3500035660 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f167979a700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1674158c10 0x7f1674151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f167a79c700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f167415cd70 0x7f167414ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f1679f9b700 1 -- 172.21.15.97:0/3500035660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f167415cd70 msgr2=0x7f167414ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f1679f9b700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f167415cd70 0x7f167414ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f1679f9b700 1 -- 172.21.15.97:0/3500035660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1674158c10 msgr2=0x7f1674151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f1679f9b700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1674158c10 0x7f1674151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f1679f9b700 1 -- 172.21.15.97:0/3500035660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16640094a0 con 0x7f16740adb30 2023-07-06T03:47:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f1679f9b700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16740adb30 0x7f1674148ae0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f167000c520 tx=0x7f167000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f166a7fc700 1 -- 172.21.15.97:0/3500035660 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670004c80 con 0x7f16740adb30 2023-07-06T03:47:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f166a7fc700 1 -- 172.21.15.97:0/3500035660 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670004e20 con 0x7f16740adb30 2023-07-06T03:47:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f167c1ff700 1 -- 172.21.15.97:0/3500035660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1674158230 con 0x7f16740adb30 2023-07-06T03:47:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f166a7fc700 1 -- 172.21.15.97:0/3500035660 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670007310 con 0x7f16740adb30 2023-07-06T03:47:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.190+0000 7f167c1ff700 1 -- 172.21.15.97:0/3500035660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16741517b0 con 0x7f16740adb30 2023-07-06T03:47:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.191+0000 7f167c1ff700 1 -- 172.21.15.97:0/3500035660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1674150bb0 con 0x7f16740adb30 2023-07-06T03:47:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.192+0000 7f166a7fc700 1 -- 172.21.15.97:0/3500035660 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1670007470 con 0x7f16740adb30 2023-07-06T03:47:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.194+0000 7f166a7fc700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165806ace0 0x7f165806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.194+0000 7f166a7fc700 1 -- 172.21.15.97:0/3500035660 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1670088ed0 con 0x7f16740adb30 2023-07-06T03:47:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.194+0000 7f167979a700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165806ace0 0x7f165806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.195+0000 7f167979a700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165806ace0 0x7f165806d190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f167415d1f0 tx=0x7f166401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.198+0000 7f166a7fc700 1 -- 172.21.15.97:0/3500035660 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1674150bb0 con 0x7f16740adb30 2023-07-06T03:47:28.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.473+0000 7f167c1ff700 1 -- 172.21.15.97:0/3500035660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) v1 -- 0x7f1674150bb0 con 0x7f16740adb30 2023-07-06T03:47:28.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.972+0000 7f166a7fc700 1 -- 172.21.15.97:0/3500035660 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v148) v1 ==== 153+0+0 (secure 0 0 0) 0x7f1674150bb0 con 0x7f16740adb30 2023-07-06T03:47:28.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 -- 172.21.15.97:0/3500035660 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165806ace0 msgr2=0x7f165806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:28.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165806ace0 0x7f165806d190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f167415d1f0 tx=0x7f166401c040 comp rx=0 tx=0).stop 2023-07-06T03:47:28.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 -- 172.21.15.97:0/3500035660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16740adb30 msgr2=0x7f1674148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:28.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16740adb30 0x7f1674148ae0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f167000c520 tx=0x7f167000c8e0 comp rx=0 tx=0).stop 2023-07-06T03:47:28.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 -- 172.21.15.97:0/3500035660 shutdown_connections 2023-07-06T03:47:28.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165806ace0 0x7f165806d190 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:28.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f167415cd70 0x7f167414ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:28.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16740adb30 0x7f1674148ae0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:28.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 --2- 172.21.15.97:0/3500035660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1674158c10 0x7f1674151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:28.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 -- 172.21.15.97:0/3500035660 >> 172.21.15.97:0/3500035660 conn(0x7f16740b04a0 msgr2=0x7f1674152f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:28.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 -- 172.21.15.97:0/3500035660 shutdown_connections 2023-07-06T03:47:28.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:28.975+0000 7f167c1ff700 1 -- 172.21.15.97:0/3500035660 wait complete. 2023-07-06T03:47:28.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T03:47:29.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.194+0000 7f483c5ff700 1 Processor -- start 2023-07-06T03:47:29.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.194+0000 7f483c5ff700 1 -- start start 2023-07-06T03:47:29.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.195+0000 7f483c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4834158b20 0x7f48340cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:29.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.195+0000 7f483c5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f48340b0560 0x7f48340b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:29.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.195+0000 7f483c5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48340ccbc0 con 0x7f483406cab0 2023-07-06T03:47:29.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.195+0000 7f483c5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48340ce820 con 0x7f48340b0560 2023-07-06T03:47:29.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.195+0000 7f483c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340b0f20 con 0x7f4834158b20 2023-07-06T03:47:29.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.195+0000 7f4839b9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f483406cab0 0x7f48340ce020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42184/0 (socket says 172.21.15.97:42184) 2023-07-06T03:47:29.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.195+0000 7f4839b9a700 1 -- 172.21.15.97:0/4143272678 learned_addr learned my addr 172.21.15.97:0/4143272678 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:29.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f483a39b700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4834158b20 0x7f48340cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:29.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f483ab9c700 1 --2- 172.21.15.97:0/4143272678 >> v2:172.21.15.146:3300/0 conn(0x7f48340b0560 0x7f48340b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:29.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534185651 0 0) 0x7f48340ccbc0 con 0x7f483406cab0 2023-07-06T03:47:29.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48200035b0 con 0x7f483406cab0 2023-07-06T03:47:29.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1598208782 0 0) 0x7f48200035b0 con 0x7f483406cab0 2023-07-06T03:47:29.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4834158b20 msgr2=0x7f48340cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:29.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4834158b20 0x7f48340cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:29.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 >> v2:172.21.15.146:3300/0 conn(0x7f48340b0560 msgr2=0x7f48340b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:29.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 --2- 172.21.15.97:0/4143272678 >> v2:172.21.15.146:3300/0 conn(0x7f48340b0560 0x7f48340b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:29.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48340adcf0 con 0x7f483406cab0 2023-07-06T03:47:29.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f4824001670 con 0x7f483406cab0 2023-07-06T03:47:29.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 >> v1:172.21.15.97:6789/0 conn(0x7f483406cab0 legacy=0x7f48340ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:29.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4834158b20 0x7f4820003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:29.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4820003f80 0x7f4820004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:29.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4820004d90 0x7f4820005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340ce820 con 0x7f4820003f80 2023-07-06T03:47:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340b0f20 con 0x7f4820004d90 2023-07-06T03:47:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.196+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482000ba40 con 0x7f4834158b20 2023-07-06T03:47:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.197+0000 7f4839b9a700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4820003f80 0x7f4820004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.197+0000 7f483a39b700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4834158b20 0x7f4820003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.197+0000 7f483ab9c700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4820004d90 0x7f4820005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.197+0000 7f4839b9a700 1 -- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4834158b20 msgr2=0x7f4820003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:29.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.197+0000 7f4839b9a700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4834158b20 0x7f4820003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:29.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.197+0000 7f4839b9a700 1 -- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4820004d90 msgr2=0x7f4820005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:29.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.197+0000 7f4839b9a700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4820004d90 0x7f4820005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:29.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.197+0000 7f4839b9a700 1 -- 172.21.15.97:0/4143272678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48200035b0 con 0x7f4820003f80 2023-07-06T03:47:29.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.197+0000 7f483ab9c700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4820004d90 0x7f4820005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:29.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.197+0000 7f4839b9a700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4820003f80 0x7f4820004850 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f482400c370 tx=0x7f4820005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff85584c18be83da server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.198+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824007c60 con 0x7f4820003f80 2023-07-06T03:47:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.198+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824005960 con 0x7f4820003f80 2023-07-06T03:47:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.198+0000 7f4839399700 1 -- 172.21.15.97:0/4143272678 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824005d10 con 0x7f4820003f80 2023-07-06T03:47:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.198+0000 7f483c5ff700 1 -- 172.21.15.97:0/4143272678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4820003f80 msgr2=0x7f4820004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.198+0000 7f483c5ff700 1 --2- 172.21.15.97:0/4143272678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4820003f80 0x7f4820004850 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f482400c370 tx=0x7f4820005880 comp rx=0 tx=0).stop 2023-07-06T03:47:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.198+0000 7f483ab9c700 1 -- 172.21.15.97:0/4143272678 reap_dead start 2023-07-06T03:47:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.198+0000 7f483c5ff700 1 -- 172.21.15.97:0/4143272678 shutdown_connections 2023-07-06T03:47:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.198+0000 7f483c5ff700 1 -- 172.21.15.97:0/4143272678 >> 172.21.15.97:0/4143272678 conn(0x7f483406a060 msgr2=0x7f483406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.198+0000 7f483c5ff700 1 -- 172.21.15.97:0/4143272678 shutdown_connections 2023-07-06T03:47:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.198+0000 7f483c5ff700 1 -- 172.21.15.97:0/4143272678 wait complete. 2023-07-06T03:47:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.200+0000 7f483c5ff700 1 Processor -- start 2023-07-06T03:47:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.200+0000 7f483c5ff700 1 -- start start 2023-07-06T03:47:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.201+0000 7f483c5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4820003f80 0x7f48340b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.201+0000 7f483c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48340aecf0 0x7f4834146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.201+0000 7f483c5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483415b280 0x7f483415b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.201+0000 7f483c5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340b2d60 con 0x7f483415b280 2023-07-06T03:47:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.201+0000 7f483c5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483415bc30 con 0x7f4820003f80 2023-07-06T03:47:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.201+0000 7f483c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340b1e10 con 0x7f48340aecf0 2023-07-06T03:47:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.201+0000 7f483ab9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483415b280 0x7f483415b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.201+0000 7f483ab9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483415b280 0x7f483415b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51734/0 (socket says 172.21.15.97:51734) 2023-07-06T03:47:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.201+0000 7f483ab9c700 1 -- 172.21.15.97:0/2675796713 learned_addr learned my addr 172.21.15.97:0/2675796713 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.201+0000 7f483a39b700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4820003f80 0x7f48340b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f4839b9a700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48340aecf0 0x7f4834146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f483ab9c700 1 -- 172.21.15.97:0/2675796713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48340aecf0 msgr2=0x7f4834146d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f483ab9c700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48340aecf0 0x7f4834146d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f483ab9c700 1 -- 172.21.15.97:0/2675796713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4820003f80 msgr2=0x7f48340b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f483ab9c700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4820003f80 0x7f48340b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f483ab9c700 1 -- 172.21.15.97:0/2675796713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48340b20e0 con 0x7f483415b280 2023-07-06T03:47:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f483ab9c700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483415b280 0x7f483415b6f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f48340c5230 tx=0x7f482c0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f482affd700 1 -- 172.21.15.97:0/2675796713 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c00e050 con 0x7f483415b280 2023-07-06T03:47:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f482affd700 1 -- 172.21.15.97:0/2675796713 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c00f040 con 0x7f483415b280 2023-07-06T03:47:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f483c5ff700 1 -- 172.21.15.97:0/2675796713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48340b23c0 con 0x7f483415b280 2023-07-06T03:47:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f482affd700 1 -- 172.21.15.97:0/2675796713 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c003c60 con 0x7f483415b280 2023-07-06T03:47:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.202+0000 7f483c5ff700 1 -- 172.21.15.97:0/2675796713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48340b2980 con 0x7f483415b280 2023-07-06T03:47:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.204+0000 7f483c5ff700 1 -- 172.21.15.97:0/2675796713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48340b2ea0 con 0x7f483415b280 2023-07-06T03:47:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.204+0000 7f482affd700 1 -- 172.21.15.97:0/2675796713 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f482c010920 con 0x7f483415b280 2023-07-06T03:47:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.206+0000 7f482affd700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f481806ace0 0x7f481806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.206+0000 7f482affd700 1 -- 172.21.15.97:0/2675796713 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f482c089810 con 0x7f483415b280 2023-07-06T03:47:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.206+0000 7f483a39b700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f481806ace0 0x7f481806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.207+0000 7f483a39b700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f481806ace0 0x7f481806d190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4830009b80 tx=0x7f4830009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.211+0000 7f482affd700 1 -- 172.21.15.97:0/2675796713 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f48340b2ea0 con 0x7f483415b280 2023-07-06T03:47:29.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.405+0000 7f483c5ff700 1 -- 172.21.15.97:0/2675796713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f48341ef9b0 con 0x7f483415b280 2023-07-06T03:47:29.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.411+0000 7f482affd700 1 -- 172.21.15.97:0/2675796713 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f48341ef9b0 con 0x7f483415b280 2023-07-06T03:47:29.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.413+0000 7f483c5ff700 1 -- 172.21.15.97:0/2675796713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f481806ace0 msgr2=0x7f481806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:29.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.413+0000 7f483c5ff700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f481806ace0 0x7f481806d190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4830009b80 tx=0x7f4830009210 comp rx=0 tx=0).stop 2023-07-06T03:47:29.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.413+0000 7f483c5ff700 1 -- 172.21.15.97:0/2675796713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483415b280 msgr2=0x7f483415b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:29.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.413+0000 7f483c5ff700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483415b280 0x7f483415b6f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f48340c5230 tx=0x7f482c0104a0 comp rx=0 tx=0).stop 2023-07-06T03:47:29.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.413+0000 7f483c5ff700 1 -- 172.21.15.97:0/2675796713 shutdown_connections 2023-07-06T03:47:29.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.413+0000 7f483c5ff700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f481806ace0 0x7f481806d190 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:29.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.413+0000 7f483c5ff700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483415b280 0x7f483415b6f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:29.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.413+0000 7f483c5ff700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4820003f80 0x7f48340b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:29.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.413+0000 7f483c5ff700 1 --2- 172.21.15.97:0/2675796713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48340aecf0 0x7f4834146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:29.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.413+0000 7f483c5ff700 1 -- 172.21.15.97:0/2675796713 >> 172.21.15.97:0/2675796713 conn(0x7f483406a060 msgr2=0x7f48340cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:29.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.414+0000 7f483c5ff700 1 -- 172.21.15.97:0/2675796713 shutdown_connections 2023-07-06T03:47:29.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:29.414+0000 7f483c5ff700 1 -- 172.21.15.97:0/2675796713 wait complete. 2023-07-06T03:47:29.433 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:47:29.433 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:47:29.537 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:47:29.684 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:47:29.684 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:47:29.685 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:47:29.730 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:47:29.731 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:47:29.731 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:47:29.778 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:47:29.779 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:47:29.780 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:47:29.826 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:47:29.826 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:47:29.827 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:47:29.877 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:47:29.878 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T03:47:29.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:47:30.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.073+0000 7f3419dff700 1 Processor -- start 2023-07-06T03:47:30.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.073+0000 7f3419dff700 1 -- start start 2023-07-06T03:47:30.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.074+0000 7f3419dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3414158960 0x7f341406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.074+0000 7f3419dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f341415ce90 0x7f34140712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.074+0000 7f3419dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3414159d40 con 0x7f3414071830 2023-07-06T03:47:30.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.074+0000 7f3419dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341406ddb0 con 0x7f3414158960 2023-07-06T03:47:30.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.074+0000 7f3419dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341406bbf0 con 0x7f341415ce90 2023-07-06T03:47:30.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.075+0000 7f34137fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3414158960 0x7f341406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.075+0000 7f3413fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3414071830 0x7f34140cc400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34078/0 (socket says 172.21.15.97:34078) 2023-07-06T03:47:30.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.075+0000 7f34137fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3414158960 0x7f341406d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40706/0 (socket says 172.21.15.97:40706) 2023-07-06T03:47:30.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.075+0000 7f3413fff700 1 -- 172.21.15.97:0/3534910098 learned_addr learned my addr 172.21.15.97:0/3534910098 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:30.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.075+0000 7f3412ffd700 1 --2- 172.21.15.97:0/3534910098 >> v2:172.21.15.146:3300/0 conn(0x7f341415ce90 0x7f34140712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.075+0000 7f34137fe700 1 -- 172.21.15.97:0/3534910098 >> v2:172.21.15.146:3300/0 conn(0x7f341415ce90 msgr2=0x7f34140712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:30.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.075+0000 7f34137fe700 1 --2- 172.21.15.97:0/3534910098 >> v2:172.21.15.146:3300/0 conn(0x7f341415ce90 0x7f34140712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.075+0000 7f34137fe700 1 -- 172.21.15.97:0/3534910098 >> v1:172.21.15.97:6789/0 conn(0x7f3414071830 legacy=0x7f34140cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:30.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.075+0000 7f34137fe700 1 -- 172.21.15.97:0/3534910098 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341406bed0 con 0x7f3414158960 2023-07-06T03:47:30.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.075+0000 7f34137fe700 1 --2- 172.21.15.97:0/3534910098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3414158960 0x7f341406d6f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f341406f570 tx=0x7f340400bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599a97ee827058f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:30.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.076+0000 7f34127fc700 1 -- 172.21.15.97:0/3534910098 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400db70 con 0x7f3414158960 2023-07-06T03:47:30.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.076+0000 7f34127fc700 1 -- 172.21.15.97:0/3534910098 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340400dcd0 con 0x7f3414158960 2023-07-06T03:47:30.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.076+0000 7f34127fc700 1 -- 172.21.15.97:0/3534910098 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404004030 con 0x7f3414158960 2023-07-06T03:47:30.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.076+0000 7f3419dff700 1 -- 172.21.15.97:0/3534910098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3414158960 msgr2=0x7f341406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:30.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.076+0000 7f3419dff700 1 --2- 172.21.15.97:0/3534910098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3414158960 0x7f341406d6f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f341406f570 tx=0x7f340400bdc0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.076+0000 7f3419dff700 1 -- 172.21.15.97:0/3534910098 shutdown_connections 2023-07-06T03:47:30.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.076+0000 7f3419dff700 1 --2- 172.21.15.97:0/3534910098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3414158960 0x7f341406d6f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.076+0000 7f3419dff700 1 --2- 172.21.15.97:0/3534910098 >> v2:172.21.15.146:3300/0 conn(0x7f341415ce90 0x7f34140712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.076+0000 7f3419dff700 1 -- 172.21.15.97:0/3534910098 >> 172.21.15.97:0/3534910098 conn(0x7f34140ce8c0 msgr2=0x7f34140cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:30.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.076+0000 7f3419dff700 1 -- 172.21.15.97:0/3534910098 shutdown_connections 2023-07-06T03:47:30.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.076+0000 7f3419dff700 1 -- 172.21.15.97:0/3534910098 wait complete. 2023-07-06T03:47:30.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.078+0000 7f3419dff700 1 Processor -- start 2023-07-06T03:47:30.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.078+0000 7f3419dff700 1 -- start start 2023-07-06T03:47:30.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.079+0000 7f3419dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3414071830 0x7f34141f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.079+0000 7f3419dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3414158960 0x7f34141f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.079+0000 7f3419dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f341415ce90 0x7f3414156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.079+0000 7f3419dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3414155050 con 0x7f3414071830 2023-07-06T03:47:30.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.079+0000 7f3419dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3414151f30 con 0x7f3414158960 2023-07-06T03:47:30.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.079+0000 7f3419dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34141f62e0 con 0x7f341415ce90 2023-07-06T03:47:30.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.079+0000 7f34137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3414071830 0x7f34141f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.079+0000 7f34137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3414071830 0x7f34141f1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40712/0 (socket says 172.21.15.97:40712) 2023-07-06T03:47:30.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.079+0000 7f34137fe700 1 -- 172.21.15.97:0/4190367607 learned_addr learned my addr 172.21.15.97:0/4190367607 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:30.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.079+0000 7f3412ffd700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3414158960 0x7f34141f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f3413fff700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f341415ce90 0x7f3414156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f34137fe700 1 -- 172.21.15.97:0/4190367607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f341415ce90 msgr2=0x7f3414156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:30.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f34137fe700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f341415ce90 0x7f3414156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f34137fe700 1 -- 172.21.15.97:0/4190367607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3414158960 msgr2=0x7f34141f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:30.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f34137fe700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3414158960 0x7f34141f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f34137fe700 1 -- 172.21.15.97:0/4190367607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340400b740 con 0x7f3414071830 2023-07-06T03:47:30.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f34137fe700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3414071830 0x7f34141f1af0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f340400b2c0 tx=0x7f34040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:30.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f33fbfff700 1 -- 172.21.15.97:0/4190367607 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340401e070 con 0x7f3414071830 2023-07-06T03:47:30.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f33fbfff700 1 -- 172.21.15.97:0/4190367607 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340400de40 con 0x7f3414071830 2023-07-06T03:47:30.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f3419dff700 1 -- 172.21.15.97:0/4190367607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34141f65c0 con 0x7f3414071830 2023-07-06T03:47:30.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f33fbfff700 1 -- 172.21.15.97:0/4190367607 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404004030 con 0x7f3414071830 2023-07-06T03:47:30.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.080+0000 7f3419dff700 1 -- 172.21.15.97:0/4190367607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3414152420 con 0x7f3414071830 2023-07-06T03:47:30.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.081+0000 7f3419dff700 1 -- 172.21.15.97:0/4190367607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3414067830 con 0x7f3414071830 2023-07-06T03:47:30.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.082+0000 7f33fbfff700 1 -- 172.21.15.97:0/4190367607 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f340402c3f0 con 0x7f3414071830 2023-07-06T03:47:30.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.084+0000 7f33fbfff700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340006ad10 0x7f340006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.084+0000 7f33fbfff700 1 -- 172.21.15.97:0/4190367607 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f340408a430 con 0x7f3414071830 2023-07-06T03:47:30.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.084+0000 7f3412ffd700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340006ad10 0x7f340006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.084+0000 7f3412ffd700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340006ad10 0x7f340006d1c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3414153eb0 tx=0x7f3408009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:30.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.088+0000 7f33fbfff700 1 -- 172.21.15.97:0/4190367607 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3404055910 con 0x7f3414071830 2023-07-06T03:47:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.328+0000 7f3419dff700 1 -- 172.21.15.97:0/4190367607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f341404e340 con 0x7f3414071830 2023-07-06T03:47:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.328+0000 7f33fbfff700 1 -- 172.21.15.97:0/4190367607 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f3404028de0 con 0x7f3414071830 2023-07-06T03:47:30.331 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:30.331 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/1839628004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1839628004},{"type":"v1","addr":"172.21.15.146:6834","nonce":1839628004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4937,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2131262285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":2131262285},{"type":"v1","addr":"172.21.15.146:6835","nonce":2131262285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1367937004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367937004},{"type":"v1","addr":"172.21.15.97:6835","nonce":1367937004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5733,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2232612695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2232612695},{"type":"v1","addr":"172.21.15.97:6837","nonce":2232612695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-07-06T03:47:30.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 -- 172.21.15.97:0/4190367607 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340006ad10 msgr2=0x7f340006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:30.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340006ad10 0x7f340006d1c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3414153eb0 tx=0x7f3408009210 comp rx=0 tx=0).stop 2023-07-06T03:47:30.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 -- 172.21.15.97:0/4190367607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3414071830 msgr2=0x7f34141f1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:30.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3414071830 0x7f34141f1af0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f340400b2c0 tx=0x7f34040047c0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 -- 172.21.15.97:0/4190367607 shutdown_connections 2023-07-06T03:47:30.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340006ad10 0x7f340006d1c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f341415ce90 0x7f3414156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3414071830 0x7f34141f1af0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 --2- 172.21.15.97:0/4190367607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3414158960 0x7f34141f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 -- 172.21.15.97:0/4190367607 >> 172.21.15.97:0/4190367607 conn(0x7f34140ce8c0 msgr2=0x7f34140ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 -- 172.21.15.97:0/4190367607 shutdown_connections 2023-07-06T03:47:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.331+0000 7f3419dff700 1 -- 172.21.15.97:0/4190367607 wait complete. 2023-07-06T03:47:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2023-07-06T03:47:30.351 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T03:47:30.351 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T03:47:30.352 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T03:47:30.352 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:47:30.352 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T03:47:30.352 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T03:47:30.353 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T03:47:30.353 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:47:30.353 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T03:47:30.353 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T03:47:30.353 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T03:47:30.354 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:47:30.354 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T03:47:30.354 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T03:47:30.354 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T03:47:30.354 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:47:30.365 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T03:47:30.365 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T03:47:30.367 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T03:47:30.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T03:47:30.368 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T03:47:30.368 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T03:47:30.378 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T03:47:30.379 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T03:47:30.410 INFO:tasks.ceph.mds.a:Started 2023-07-06T03:47:30.411 DEBUG:teuthology.parallel:result is None 2023-07-06T03:47:30.414 INFO:tasks.ceph.mds.c:Started 2023-07-06T03:47:30.414 DEBUG:teuthology.parallel:result is None 2023-07-06T03:47:30.422 INFO:tasks.ceph.mds.b:Started 2023-07-06T03:47:30.422 DEBUG:teuthology.parallel:result is None 2023-07-06T03:47:30.425 INFO:tasks.ceph.mds.d:Started 2023-07-06T03:47:30.426 DEBUG:teuthology.parallel:result is None 2023-07-06T03:47:30.426 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T03:47:30.558 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:47:30.559 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:47:30.560 INFO:tasks.ceph.mds.c.smithi097.stderr:2023-07-06T03:47:30.559+0000 7f4f5effd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-07-06T03:47:30.569 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-07-06T03:47:30.572 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:47:30.580 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-07-06T03:47:30.581 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-07-06T03:47:30.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.634+0000 7f91f85ff700 1 Processor -- start 2023-07-06T03:47:30.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.634+0000 7f91f85ff700 1 -- start start 2023-07-06T03:47:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.635+0000 7f91f85ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91f00b8890 0x7f91f00b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.635+0000 7f91f85ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f91f00b4d70 0x7f91f00cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.635+0000 7f91f85ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91f00cd940 0x7f91f006c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.635+0000 7f91f85ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00cdde0 con 0x7f91f00cd940 2023-07-06T03:47:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.635+0000 7f91f85ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00b5280 con 0x7f91f00b4d70 2023-07-06T03:47:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.635+0000 7f91f85ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f00d0de0 con 0x7f91f00b8890 2023-07-06T03:47:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.635+0000 7f91f6b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91f00cd940 0x7f91f006c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.635+0000 7f91f6b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91f00cd940 0x7f91f006c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40754/0 (socket says 172.21.15.97:40754) 2023-07-06T03:47:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.635+0000 7f91f6b9c700 1 -- 172.21.15.97:0/3495289608 learned_addr learned my addr 172.21.15.97:0/3495289608 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:30.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.635+0000 7f91f5b9a700 1 --2- 172.21.15.97:0/3495289608 >> v2:172.21.15.146:3300/0 conn(0x7f91f00b4d70 0x7f91f00cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.635+0000 7f91f639b700 1 --2- 172.21.15.97:0/3495289608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91f00b8890 0x7f91f00b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.636+0000 7f91f6b9c700 1 -- 172.21.15.97:0/3495289608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91f00b8890 msgr2=0x7f91f00b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.636+0000 7f91f6b9c700 1 --2- 172.21.15.97:0/3495289608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91f00b8890 0x7f91f00b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.636+0000 7f91f6b9c700 1 -- 172.21.15.97:0/3495289608 >> v2:172.21.15.146:3300/0 conn(0x7f91f00b4d70 msgr2=0x7f91f00cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.636+0000 7f91f6b9c700 1 --2- 172.21.15.97:0/3495289608 >> v2:172.21.15.146:3300/0 conn(0x7f91f00b4d70 0x7f91f00cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.636+0000 7f91f6b9c700 1 -- 172.21.15.97:0/3495289608 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00d10c0 con 0x7f91f00cd940 2023-07-06T03:47:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.636+0000 7f91f639b700 1 --2- 172.21.15.97:0/3495289608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91f00b8890 0x7f91f00b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.636+0000 7f91f6b9c700 1 --2- 172.21.15.97:0/3495289608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f00cd940 0x7f91f006c060 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f91f0067bf0 tx=0x7f91e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9ae0f465e5868a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.636+0000 7f91f5399700 1 -- 172.21.15.97:0/3495289608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e800e070 con 0x7f91f00cd940 2023-07-06T03:47:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.636+0000 7f91f5399700 1 -- 172.21.15.97:0/3495289608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e800b910 con 0x7f91f00cd940 2023-07-06T03:47:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.636+0000 7f91f5399700 1 -- 172.21.15.97:0/3495289608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e800bcc0 con 0x7f91f00cd940 2023-07-06T03:47:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.637+0000 7f91f85ff700 1 -- 172.21.15.97:0/3495289608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f00cd940 msgr2=0x7f91f006c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.637+0000 7f91f85ff700 1 --2- 172.21.15.97:0/3495289608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f00cd940 0x7f91f006c060 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f91f0067bf0 tx=0x7f91e800b330 comp rx=0 tx=0).stop 2023-07-06T03:47:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.637+0000 7f91f85ff700 1 -- 172.21.15.97:0/3495289608 shutdown_connections 2023-07-06T03:47:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.637+0000 7f91f85ff700 1 --2- 172.21.15.97:0/3495289608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f00cd940 0x7f91f006c060 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.637+0000 7f91f85ff700 1 --2- 172.21.15.97:0/3495289608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91f00b8890 0x7f91f00b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.637+0000 7f91f85ff700 1 --2- 172.21.15.97:0/3495289608 >> v2:172.21.15.146:3300/0 conn(0x7f91f00b4d70 0x7f91f00cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.637+0000 7f91f85ff700 1 -- 172.21.15.97:0/3495289608 >> 172.21.15.97:0/3495289608 conn(0x7f91f006b2d0 msgr2=0x7f91f00cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:30.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.637+0000 7f91f85ff700 1 -- 172.21.15.97:0/3495289608 shutdown_connections 2023-07-06T03:47:30.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.637+0000 7f91f85ff700 1 -- 172.21.15.97:0/3495289608 wait complete. 2023-07-06T03:47:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.639+0000 7f91f85ff700 1 Processor -- start 2023-07-06T03:47:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.639+0000 7f91f85ff700 1 -- start start 2023-07-06T03:47:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f85ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91f00b4d70 0x7f91f014f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f85ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91f00b8890 0x7f91f015f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f85ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f00cd940 0x7f91f015f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f85ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01e5cd0 con 0x7f91f00cd940 2023-07-06T03:47:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f85ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01e5f40 con 0x7f91f00b8890 2023-07-06T03:47:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f85ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01e6220 con 0x7f91f00b4d70 2023-07-06T03:47:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f6b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f00cd940 0x7f91f015f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f6b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f00cd940 0x7f91f015f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40764/0 (socket says 172.21.15.97:40764) 2023-07-06T03:47:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f6b9c700 1 -- 172.21.15.97:0/2375402197 learned_addr learned my addr 172.21.15.97:0/2375402197 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f5b9a700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91f00b8890 0x7f91f015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f639b700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91f00b4d70 0x7f91f014f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f6b9c700 1 -- 172.21.15.97:0/2375402197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91f00b4d70 msgr2=0x7f91f014f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:30.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f6b9c700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91f00b4d70 0x7f91f014f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f6b9c700 1 -- 172.21.15.97:0/2375402197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91f00b8890 msgr2=0x7f91f015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:30.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f6b9c700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91f00b8890 0x7f91f015f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f6b9c700 1 -- 172.21.15.97:0/2375402197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e8009b30 con 0x7f91f00cd940 2023-07-06T03:47:30.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.640+0000 7f91f5b9a700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91f00b8890 0x7f91f015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:30.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.641+0000 7f91f6b9c700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f00cd940 0x7f91f015f790 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f91f00677e0 tx=0x7f91e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.641+0000 7f91e6ffd700 1 -- 172.21.15.97:0/2375402197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8004450 con 0x7f91f00cd940 2023-07-06T03:47:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.641+0000 7f91e6ffd700 1 -- 172.21.15.97:0/2375402197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e80045d0 con 0x7f91f00cd940 2023-07-06T03:47:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.641+0000 7f91e6ffd700 1 -- 172.21.15.97:0/2375402197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e80099b0 con 0x7f91f00cd940 2023-07-06T03:47:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.641+0000 7f91f85ff700 1 -- 172.21.15.97:0/2375402197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f01e6500 con 0x7f91f00cd940 2023-07-06T03:47:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.641+0000 7f91f85ff700 1 -- 172.21.15.97:0/2375402197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f015c5f0 con 0x7f91f00cd940 2023-07-06T03:47:30.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.642+0000 7f91f85ff700 1 -- 172.21.15.97:0/2375402197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f01e6720 con 0x7f91f00cd940 2023-07-06T03:47:30.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.643+0000 7f91e6ffd700 1 -- 172.21.15.97:0/2375402197 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f91e801c410 con 0x7f91f00cd940 2023-07-06T03:47:30.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.644+0000 7f91e6ffd700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91d406ace0 0x7f91d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:30.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.644+0000 7f91e6ffd700 1 -- 172.21.15.97:0/2375402197 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f91e808ad20 con 0x7f91f00cd940 2023-07-06T03:47:30.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.644+0000 7f91f639b700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91d406ace0 0x7f91d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:30.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.645+0000 7f91f639b700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91d406ace0 0x7f91d406d190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00c900 tx=0x7f91ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:30.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.649+0000 7f91e6ffd700 1 -- 172.21.15.97:0/2375402197 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f91e8056160 con 0x7f91f00cd940 2023-07-06T03:47:30.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.859+0000 7f91f85ff700 1 -- 172.21.15.97:0/2375402197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f91f004e340 con 0x7f91f00cd940 2023-07-06T03:47:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.990+0000 7f91e6ffd700 1 -- 172.21.15.97:0/2375402197 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7f91e802a020 con 0x7f91f00cd940 2023-07-06T03:47:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 -- 172.21.15.97:0/2375402197 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91d406ace0 msgr2=0x7f91d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91d406ace0 0x7f91d406d190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00c900 tx=0x7f91ec008040 comp rx=0 tx=0).stop 2023-07-06T03:47:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 -- 172.21.15.97:0/2375402197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f00cd940 msgr2=0x7f91f015f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f00cd940 0x7f91f015f790 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f91f00677e0 tx=0x7f91e8004ad0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 -- 172.21.15.97:0/2375402197 shutdown_connections 2023-07-06T03:47:30.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f91d406ace0 0x7f91d406d190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91f00cd940 0x7f91f015f790 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f91f00b4d70 0x7f91f014f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 --2- 172.21.15.97:0/2375402197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91f00b8890 0x7f91f015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 -- 172.21.15.97:0/2375402197 >> 172.21.15.97:0/2375402197 conn(0x7f91f006b2d0 msgr2=0x7f91f00d1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 -- 172.21.15.97:0/2375402197 shutdown_connections 2023-07-06T03:47:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:30.993+0000 7f91f85ff700 1 -- 172.21.15.97:0/2375402197 wait complete. 2023-07-06T03:47:30.997 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-07-06T03:47:31.014 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T03:47:31.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.207+0000 7f9a7f9ff700 1 Processor -- start 2023-07-06T03:47:31.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.207+0000 7f9a7f9ff700 1 -- start start 2023-07-06T03:47:31.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.208+0000 7f9a7f9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9a780cd390 0x7f9a78160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.208+0000 7f9a7f9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 0x7f9a780b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.208+0000 7f9a7f9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a780cdae0 con 0x7f9a7815c520 2023-07-06T03:47:31.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.208+0000 7f9a7f9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a780b6640 con 0x7f9a780cd390 2023-07-06T03:47:31.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.208+0000 7f9a7f9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780b6920 con 0x7f9a78160dd0 2023-07-06T03:47:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.209+0000 7f9a7df9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 0x7f9a780b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.209+0000 7f9a7df9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 0x7f9a780b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:43782/0 (socket says 172.21.15.97:43782) 2023-07-06T03:47:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.209+0000 7f9a7df9c700 1 -- 172.21.15.97:0/1487934099 learned_addr learned my addr 172.21.15.97:0/1487934099 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.209+0000 7f9a7cf9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9a780cd390 0x7f9a78160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.209+0000 7f9a7d79b700 1 --1- 172.21.15.97:0/1487934099 >> v1:172.21.15.146:6789/0 conn(0x7f9a7815c520 0x7f9a78160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:41398/0 (socket says 172.21.15.97:41398) 2023-07-06T03:47:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.209+0000 7f9a7df9c700 1 -- 172.21.15.97:0/1487934099 >> v2:172.21.15.146:3300/0 conn(0x7f9a780cd390 msgr2=0x7f9a78160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.209+0000 7f9a7df9c700 1 --2- 172.21.15.97:0/1487934099 >> v2:172.21.15.146:3300/0 conn(0x7f9a780cd390 0x7f9a78160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.209+0000 7f9a7df9c700 1 -- 172.21.15.97:0/1487934099 >> v1:172.21.15.146:6789/0 conn(0x7f9a7815c520 legacy=0x7f9a78160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.209+0000 7f9a7df9c700 1 -- 172.21.15.97:0/1487934099 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780b6c00 con 0x7f9a78160dd0 2023-07-06T03:47:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.209+0000 7f9a7df9c700 1 --2- 172.21.15.97:0/1487934099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 0x7f9a780b7400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009e50 tx=0x7f9a7000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a15eb86a1dd3af8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.210+0000 7f9a6ffff700 1 -- 172.21.15.97:0/1487934099 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7000e070 con 0x7f9a78160dd0 2023-07-06T03:47:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.210+0000 7f9a6ffff700 1 -- 172.21.15.97:0/1487934099 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7000b910 con 0x7f9a78160dd0 2023-07-06T03:47:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.210+0000 7f9a6ffff700 1 -- 172.21.15.97:0/1487934099 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7000bcc0 con 0x7f9a78160dd0 2023-07-06T03:47:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.210+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/1487934099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 msgr2=0x7f9a780b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.210+0000 7f9a7f9ff700 1 --2- 172.21.15.97:0/1487934099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 0x7f9a780b7400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009e50 tx=0x7f9a7000b330 comp rx=0 tx=0).stop 2023-07-06T03:47:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.210+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/1487934099 shutdown_connections 2023-07-06T03:47:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.210+0000 7f9a7f9ff700 1 --2- 172.21.15.97:0/1487934099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 0x7f9a780b7400 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.210+0000 7f9a7f9ff700 1 --2- 172.21.15.97:0/1487934099 >> v2:172.21.15.146:3300/0 conn(0x7f9a780cd390 0x7f9a78160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.210+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/1487934099 >> 172.21.15.97:0/1487934099 conn(0x7f9a78071790 msgr2=0x7f9a78071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:31.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.210+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/1487934099 shutdown_connections 2023-07-06T03:47:31.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.210+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/1487934099 wait complete. 2023-07-06T03:47:31.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.212+0000 7f9a7f9ff700 1 Processor -- start 2023-07-06T03:47:31.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.212+0000 7f9a7f9ff700 1 -- start start 2023-07-06T03:47:31.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.213+0000 7f9a7f9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a780cd390 0x7f9a78150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.213+0000 7f9a7f9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a7815c520 0x7f9a781f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.213+0000 7f9a7f9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 0x7f9a780cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.213+0000 7f9a7f9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7815bc60 con 0x7f9a780cd390 2023-07-06T03:47:31.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.213+0000 7f9a7f9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780cca60 con 0x7f9a7815c520 2023-07-06T03:47:31.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.213+0000 7f9a7f9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780ccd00 con 0x7f9a78160dd0 2023-07-06T03:47:31.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.213+0000 7f9a7d79b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a780cd390 0x7f9a78150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.213+0000 7f9a7d79b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a780cd390 0x7f9a78150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40778/0 (socket says 172.21.15.97:40778) 2023-07-06T03:47:31.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.213+0000 7f9a7d79b700 1 -- 172.21.15.97:0/3653722157 learned_addr learned my addr 172.21.15.97:0/3653722157 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:31.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.213+0000 7f9a7df9c700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 0x7f9a780cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.213+0000 7f9a7cf9a700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a7815c520 0x7f9a781f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.214+0000 7f9a7d79b700 1 -- 172.21.15.97:0/3653722157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 msgr2=0x7f9a780cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.214+0000 7f9a7d79b700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 0x7f9a780cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.214+0000 7f9a7d79b700 1 -- 172.21.15.97:0/3653722157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a7815c520 msgr2=0x7f9a781f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.214+0000 7f9a7d79b700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a7815c520 0x7f9a781f17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.214+0000 7f9a7d79b700 1 -- 172.21.15.97:0/3653722157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a70009b30 con 0x7f9a780cd390 2023-07-06T03:47:31.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.214+0000 7f9a7d79b700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a780cd390 0x7f9a78150b80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400c520 tx=0x7f9a7400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:31.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.214+0000 7f9a6dffb700 1 -- 172.21.15.97:0/3653722157 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a74005cb0 con 0x7f9a780cd390 2023-07-06T03:47:31.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.214+0000 7f9a6dffb700 1 -- 172.21.15.97:0/3653722157 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a74005e10 con 0x7f9a780cd390 2023-07-06T03:47:31.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.214+0000 7f9a6dffb700 1 -- 172.21.15.97:0/3653722157 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a740072d0 con 0x7f9a780cd390 2023-07-06T03:47:31.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.214+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/3653722157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a780d1b40 con 0x7f9a780cd390 2023-07-06T03:47:31.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.214+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/3653722157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a780d2100 con 0x7f9a780cd390 2023-07-06T03:47:31.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.215+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/3653722157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7804e340 con 0x7f9a780cd390 2023-07-06T03:47:31.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.216+0000 7f9a6dffb700 1 -- 172.21.15.97:0/3653722157 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a74007430 con 0x7f9a780cd390 2023-07-06T03:47:31.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.217+0000 7f9a6dffb700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 0x7f9a5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.217+0000 7f9a6dffb700 1 -- 172.21.15.97:0/3653722157 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9a74088330 con 0x7f9a780cd390 2023-07-06T03:47:31.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.218+0000 7f9a7cf9a700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 0x7f9a5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.218+0000 7f9a7cf9a700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 0x7f9a5c06d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9a780b24f0 tx=0x7f9a68009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:31.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.222+0000 7f9a6dffb700 1 -- 172.21.15.97:0/3653722157 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9a74053870 con 0x7f9a780cd390 2023-07-06T03:47:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.490+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/3653722157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9a780b4f40 con 0x7f9a780cd390 2023-07-06T03:47:31.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.491+0000 7f9a6dffb700 1 -- 172.21.15.97:0/3653722157 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9a74056e90 con 0x7f9a780cd390 2023-07-06T03:47:31.493 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:31.493 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:47:31.493 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-07-06T03:47:31.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.494 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-07-06T03:47:31.494 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkczuvGRAAUKjfmMyKOTCYzOQmtf3n3A==", 2023-07-06T03:47:31.494 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.494 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:47:31.494 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:47:31.494 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:47:31.494 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.494 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.495 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-07-06T03:47:31.495 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkFHvUMBAA2WgMcUUf+Qi8T2ZjhfWaHA==", 2023-07-06T03:47:31.495 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.495 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:47:31.495 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:47:31.496 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:47:31.496 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.496 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.496 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-07-06T03:47:31.496 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZk0N4+JRAA3tjqoIBPpZ6C7TMmOOg7lw==", 2023-07-06T03:47:31.496 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.497 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:47:31.497 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:47:31.497 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:47:31.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.497 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.498 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-07-06T03:47:31.498 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkJnHUABAAwIMH2DmoHaQN3frJCgRiQw==", 2023-07-06T03:47:31.498 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.498 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:47:31.498 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:47:31.498 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:47:31.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.498 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.499 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-07-06T03:47:31.499 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCLOKZkShOIABAAzIBHDcNkmnDOLSY7Qdb+FA==", 2023-07-06T03:47:31.499 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.499 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:47:31.499 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:47:31.499 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.500 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.500 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-07-06T03:47:31.500 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCOOKZkq1t7AxAAwihuhimbjitZDeki1QuMug==", 2023-07-06T03:47:31.500 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.500 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:47:31.500 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:47:31.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.501 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.501 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-07-06T03:47:31.501 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCROKZkdYv4BhAApTHg/pmlC0lO7hVTkimEgw==", 2023-07-06T03:47:31.501 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.502 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:47:31.502 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:47:31.502 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.502 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.502 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-07-06T03:47:31.502 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCUOKZkqhuMChAAsV9xG6g1dofrpWt3F8tw5w==", 2023-07-06T03:47:31.503 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.503 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:47:31.503 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:47:31.503 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.503 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.503 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-07-06T03:47:31.504 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCaOKZk1Q5aIBAABU1+afwIihm+g4xylryz/A==", 2023-07-06T03:47:31.504 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.504 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:47:31.504 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:47:31.504 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.504 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.505 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-07-06T03:47:31.505 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCdOKZkRskIJBAA+821aoG98aEV7h4FTINQFA==", 2023-07-06T03:47:31.505 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.505 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:47:31.505 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:47:31.505 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.505 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.506 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-07-06T03:47:31.506 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCgOKZk9qnMJxAATcWG+a/9Hl1AuIL/TSrdEA==", 2023-07-06T03:47:31.506 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.506 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:47:31.506 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:47:31.506 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.507 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.507 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-07-06T03:47:31.507 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCjOKZk6wJ/KxAAWnZxt7h7OJhQ+bqdel4+iQ==", 2023-07-06T03:47:31.507 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.507 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:47:31.507 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:47:31.508 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.508 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.508 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-07-06T03:47:31.508 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg==", 2023-07-06T03:47:31.508 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.508 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:47:31.509 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-07-06T03:47:31.509 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-06T03:47:31.509 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.509 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-07-06T03:47:31.509 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkq4/pGRAAucHmR1eBA8MNMy3/l7tl0g==", 2023-07-06T03:47:31.510 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.510 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:47:31.510 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:47:31.510 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:47:31.510 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:47:31.510 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.511 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-07-06T03:47:31.511 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkhL5fEhAAWhw4IbCPqTO2sXPIcpTVHA==", 2023-07-06T03:47:31.511 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.511 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:47:31.511 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:47:31.511 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:47:31.511 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:47:31.511 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.512 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-07-06T03:47:31.512 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkO81dIRAAffjVyvsb/ygBtx8UrYQK0w==", 2023-07-06T03:47:31.512 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.512 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:47:31.512 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:47:31.513 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:47:31.513 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:47:31.513 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.513 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-07-06T03:47:31.513 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCFOKZk6uVlHRAA92RDhdDKW/q65SEOQMWgiA==", 2023-07-06T03:47:31.513 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.514 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:47:31.514 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:47:31.514 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:47:31.514 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.514 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-07-06T03:47:31.515 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkT4ALEhAAXwwkcodQU9meJA0tfseJCg==", 2023-07-06T03:47:31.515 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.515 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:47:31.515 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:47:31.515 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.515 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:31.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:31.516 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-07-06T03:47:31.516 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkfnoEChAAuRxqXqPI33dgWkBJoUMh3w==", 2023-07-06T03:47:31.516 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:47:31.516 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:47:31.516 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:47:31.516 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:47:31.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:31.517 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:47:31.517 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:47:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/3653722157 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 msgr2=0x7f9a5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 0x7f9a5c06d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9a780b24f0 tx=0x7f9a68009210 comp rx=0 tx=0).stop 2023-07-06T03:47:31.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/3653722157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a780cd390 msgr2=0x7f9a78150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a780cd390 0x7f9a78150b80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400c520 tx=0x7f9a7400c830 comp rx=0 tx=0).stop 2023-07-06T03:47:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/3653722157 shutdown_connections 2023-07-06T03:47:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 0x7f9a5c06d190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78160dd0 0x7f9a780cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a780cd390 0x7f9a78150b80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 --2- 172.21.15.97:0/3653722157 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a7815c520 0x7f9a781f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/3653722157 >> 172.21.15.97:0/3653722157 conn(0x7f9a78071790 msgr2=0x7f9a780b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:31.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/3653722157 shutdown_connections 2023-07-06T03:47:31.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.493+0000 7f9a7f9ff700 1 -- 172.21.15.97:0/3653722157 wait complete. 2023-07-06T03:47:31.520 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:47:31.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.713+0000 7fec195ff700 1 Processor -- start 2023-07-06T03:47:31.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.713+0000 7fec195ff700 1 -- start start 2023-07-06T03:47:31.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.714+0000 7fec195ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec1415c520 0x7fec140cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.714+0000 7fec195ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fec14160d00 0x7fec140b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.714+0000 7fec195ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec140cdae0 con 0x7fec14160180 2023-07-06T03:47:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.714+0000 7fec195ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec140cd8d0 con 0x7fec14160d00 2023-07-06T03:47:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.714+0000 7fec195ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec140b6640 con 0x7fec1415c520 2023-07-06T03:47:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.715+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec1415c520 0x7fec140cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.715+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec1415c520 0x7fec140cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:43800/0 (socket says 172.21.15.97:43800) 2023-07-06T03:47:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.715+0000 7fec12ffd700 1 -- 172.21.15.97:0/432565324 learned_addr learned my addr 172.21.15.97:0/432565324 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.715+0000 7fec137fe700 1 --2- 172.21.15.97:0/432565324 >> v2:172.21.15.146:3300/0 conn(0x7fec14160d00 0x7fec140b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.715+0000 7fec127fc700 1 --1- 172.21.15.97:0/432565324 >> v1:172.21.15.146:6789/0 conn(0x7fec14160180 0x7fec141605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:41410/0 (socket says 172.21.15.97:41410) 2023-07-06T03:47:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.715+0000 7fec137fe700 1 -- 172.21.15.97:0/432565324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec1415c520 msgr2=0x7fec140cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.715+0000 7fec137fe700 1 --2- 172.21.15.97:0/432565324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec1415c520 0x7fec140cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.715+0000 7fec137fe700 1 -- 172.21.15.97:0/432565324 >> v1:172.21.15.146:6789/0 conn(0x7fec14160180 legacy=0x7fec141605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.715+0000 7fec137fe700 1 -- 172.21.15.97:0/432565324 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec140b6920 con 0x7fec14160d00 2023-07-06T03:47:31.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.715+0000 7fec12ffd700 1 --2- 172.21.15.97:0/432565324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec1415c520 0x7fec140cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:47:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.715+0000 7fec137fe700 1 --2- 172.21.15.97:0/432565324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec14160d00 0x7fec140b7400 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009d90 tx=0x7fec0c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d7fdf6df64c3fb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.716+0000 7fec11ffb700 1 -- 172.21.15.97:0/432565324 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00e070 con 0x7fec14160d00 2023-07-06T03:47:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.716+0000 7fec11ffb700 1 -- 172.21.15.97:0/432565324 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c00bb20 con 0x7fec14160d00 2023-07-06T03:47:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.716+0000 7fec11ffb700 1 -- 172.21.15.97:0/432565324 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c012430 con 0x7fec14160d00 2023-07-06T03:47:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.716+0000 7fec195ff700 1 -- 172.21.15.97:0/432565324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec14160d00 msgr2=0x7fec140b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.716+0000 7fec195ff700 1 --2- 172.21.15.97:0/432565324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec14160d00 0x7fec140b7400 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009d90 tx=0x7fec0c00b330 comp rx=0 tx=0).stop 2023-07-06T03:47:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.716+0000 7fec195ff700 1 -- 172.21.15.97:0/432565324 shutdown_connections 2023-07-06T03:47:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.716+0000 7fec195ff700 1 --2- 172.21.15.97:0/432565324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec14160d00 0x7fec140b7400 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.716+0000 7fec195ff700 1 --2- 172.21.15.97:0/432565324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec1415c520 0x7fec140cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.716+0000 7fec195ff700 1 -- 172.21.15.97:0/432565324 >> 172.21.15.97:0/432565324 conn(0x7fec14071790 msgr2=0x7fec14071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.716+0000 7fec195ff700 1 -- 172.21.15.97:0/432565324 shutdown_connections 2023-07-06T03:47:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.716+0000 7fec195ff700 1 -- 172.21.15.97:0/432565324 wait complete. 2023-07-06T03:47:31.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.718+0000 7fec195ff700 1 Processor -- start 2023-07-06T03:47:31.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.718+0000 7fec195ff700 1 -- start start 2023-07-06T03:47:31.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.719+0000 7fec195ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec1415c520 0x7fec14157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.719+0000 7fec195ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec14160180 0x7fec140c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.719+0000 7fec195ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec14160d00 0x7fec141f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.719+0000 7fec195ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec14157a60 con 0x7fec1415c520 2023-07-06T03:47:31.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.719+0000 7fec195ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec140ca710 con 0x7fec14160180 2023-07-06T03:47:31.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.719+0000 7fec195ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec140ca9f0 con 0x7fec14160d00 2023-07-06T03:47:31.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.719+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec1415c520 0x7fec14157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.719+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec1415c520 0x7fec14157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40780/0 (socket says 172.21.15.97:40780) 2023-07-06T03:47:31.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.719+0000 7fec12ffd700 1 -- 172.21.15.97:0/3159327435 learned_addr learned my addr 172.21.15.97:0/3159327435 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:31.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.719+0000 7fec137fe700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec14160d00 0x7fec141f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7fec127fc700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec14160180 0x7fec140c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7fec12ffd700 1 -- 172.21.15.97:0/3159327435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec14160d00 msgr2=0x7fec141f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7fec12ffd700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec14160d00 0x7fec141f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7fec12ffd700 1 -- 172.21.15.97:0/3159327435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec14160180 msgr2=0x7fec140c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7fec12ffd700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec14160180 0x7fec140c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7fec12ffd700 1 -- 172.21.15.97:0/3159327435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec04009550 con 0x7fec1415c520 2023-07-06T03:47:31.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7fec12ffd700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec1415c520 0x7fec14157520 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fec0400e9b0 tx=0x7fec0400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:31.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7febfb7fe700 1 -- 172.21.15.97:0/3159327435 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0400c850 con 0x7fec1415c520 2023-07-06T03:47:31.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7febfb7fe700 1 -- 172.21.15.97:0/3159327435 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0400c9b0 con 0x7fec1415c520 2023-07-06T03:47:31.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7febfb7fe700 1 -- 172.21.15.97:0/3159327435 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0400cd20 con 0x7fec1415c520 2023-07-06T03:47:31.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7fec195ff700 1 -- 172.21.15.97:0/3159327435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c009a70 con 0x7fec1415c520 2023-07-06T03:47:31.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.720+0000 7fec195ff700 1 -- 172.21.15.97:0/3159327435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec140cafb0 con 0x7fec1415c520 2023-07-06T03:47:31.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.721+0000 7fec195ff700 1 -- 172.21.15.97:0/3159327435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec140cb600 con 0x7fec1415c520 2023-07-06T03:47:31.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.722+0000 7febfb7fe700 1 -- 172.21.15.97:0/3159327435 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec04013470 con 0x7fec1415c520 2023-07-06T03:47:31.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.724+0000 7febfb7fe700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec0006ace0 0x7fec0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:31.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.724+0000 7febfb7fe700 1 -- 172.21.15.97:0/3159327435 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fec04089250 con 0x7fec1415c520 2023-07-06T03:47:31.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.724+0000 7fec127fc700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec0006ace0 0x7fec0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:31.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.724+0000 7fec127fc700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec0006ace0 0x7fec0006d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec140b47e0 tx=0x7fec08007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:31.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.728+0000 7febfb7fe700 1 -- 172.21.15.97:0/3159327435 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fec040546e0 con 0x7fec1415c520 2023-07-06T03:47:31.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.929+0000 7fec195ff700 1 -- 172.21.15.97:0/3159327435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec1404e340 con 0x7fec1415c520 2023-07-06T03:47:31.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.930+0000 7febfb7fe700 1 -- 172.21.15.97:0/3159327435 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+14665 (secure 0 0 0) 0x7fec04057d00 con 0x7fec1415c520 2023-07-06T03:47:31.932 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:31.932 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:47:30.994946+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:47:31.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 -- 172.21.15.97:0/3159327435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec0006ace0 msgr2=0x7fec0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec0006ace0 0x7fec0006d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec140b47e0 tx=0x7fec08007d30 comp rx=0 tx=0).stop 2023-07-06T03:47:31.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 -- 172.21.15.97:0/3159327435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec1415c520 msgr2=0x7fec14157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:31.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec1415c520 0x7fec14157520 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fec0400e9b0 tx=0x7fec0400ed70 comp rx=0 tx=0).stop 2023-07-06T03:47:31.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 -- 172.21.15.97:0/3159327435 shutdown_connections 2023-07-06T03:47:31.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec0006ace0 0x7fec0006d190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec14160d00 0x7fec141f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec1415c520 0x7fec14157520 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 --2- 172.21.15.97:0/3159327435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec14160180 0x7fec140c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:31.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 -- 172.21.15.97:0/3159327435 >> 172.21.15.97:0/3159327435 conn(0x7fec14071790 msgr2=0x7fec140b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:31.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 -- 172.21.15.97:0/3159327435 shutdown_connections 2023-07-06T03:47:31.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:31.932+0000 7fec195ff700 1 -- 172.21.15.97:0/3159327435 wait complete. 2023-07-06T03:47:31.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:47:31.953 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:47:32.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.149+0000 7f3a921ff700 1 Processor -- start 2023-07-06T03:47:32.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.149+0000 7f3a921ff700 1 -- start start 2023-07-06T03:47:32.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.150+0000 7f3a921ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3a8c06b360 0x7f3a8c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.150+0000 7f3a921ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c0d2210 0x7f3a8c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a921ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8c0cff90 con 0x7f3a8c15c610 2023-07-06T03:47:32.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a921ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8c0d0780 con 0x7f3a8c06b360 2023-07-06T03:47:32.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a921ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c15c9e0 con 0x7f3a8c0d2210 2023-07-06T03:47:32.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a8b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3a8c15c610 0x7f3a8c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34090/0 (socket says 172.21.15.97:34090) 2023-07-06T03:47:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a8b7fe700 1 -- 172.21.15.97:0/2440773305 learned_addr learned my addr 172.21.15.97:0/2440773305 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a8bfff700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c0d2210 0x7f3a8c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a83fff700 1 --2- 172.21.15.97:0/2440773305 >> v2:172.21.15.146:3300/0 conn(0x7f3a8c06b360 0x7f3a8c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216615120 0 0) 0x7f3a8c0cff90 con 0x7f3a8c15c610 2023-07-06T03:47:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a780035b0 con 0x7f3a8c15c610 2023-07-06T03:47:32.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1326407384 0 0) 0x7f3a780035b0 con 0x7f3a8c15c610 2023-07-06T03:47:32.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c0d2210 msgr2=0x7f3a8c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a8affd700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c0d2210 0x7f3a8c0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 >> v2:172.21.15.146:3300/0 conn(0x7f3a8c06b360 msgr2=0x7f3a8c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.151+0000 7f3a8affd700 1 --2- 172.21.15.97:0/2440773305 >> v2:172.21.15.146:3300/0 conn(0x7f3a8c06b360 0x7f3a8c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c0b5650 con 0x7f3a8c15c610 2023-07-06T03:47:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f3a7c001670 con 0x7f3a8c15c610 2023-07-06T03:47:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 >> v1:172.21.15.97:6789/0 conn(0x7f3a8c15c610 legacy=0x7f3a8c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a8affd700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c0d2210 0x7f3a78003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a8affd700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 0x7f3a78004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a8affd700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a78004d90 0x7f3a78005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c0d0780 con 0x7f3a78003f80 2023-07-06T03:47:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c15c9e0 con 0x7f3a78004d90 2023-07-06T03:47:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7800ba10 con 0x7f3a8c0d2210 2023-07-06T03:47:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a83fff700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 0x7f3a78004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a8b7fe700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c0d2210 0x7f3a78003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a8bfff700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a78004d90 0x7f3a78005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a83fff700 1 -- 172.21.15.97:0/2440773305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c0d2210 msgr2=0x7f3a78003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a83fff700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c0d2210 0x7f3a78003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a83fff700 1 -- 172.21.15.97:0/2440773305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a78004d90 msgr2=0x7f3a78005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a83fff700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a78004d90 0x7f3a78005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a83fff700 1 -- 172.21.15.97:0/2440773305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780035b0 con 0x7f3a78003f80 2023-07-06T03:47:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.152+0000 7f3a83fff700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 0x7f3a78004850 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009b60 tx=0x7f3a78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd38ded84c7c035e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.153+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400e6c0 con 0x7f3a78003f80 2023-07-06T03:47:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.153+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7400e820 con 0x7f3a78003f80 2023-07-06T03:47:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.153+0000 7f3a8affd700 1 -- 172.21.15.97:0/2440773305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400eb90 con 0x7f3a78003f80 2023-07-06T03:47:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.153+0000 7f3a921ff700 1 -- 172.21.15.97:0/2440773305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 msgr2=0x7f3a78004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.153+0000 7f3a921ff700 1 --2- 172.21.15.97:0/2440773305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 0x7f3a78004850 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009b60 tx=0x7f3a78005880 comp rx=0 tx=0).stop 2023-07-06T03:47:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.153+0000 7f3a8bfff700 1 -- 172.21.15.97:0/2440773305 reap_dead start 2023-07-06T03:47:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.153+0000 7f3a921ff700 1 -- 172.21.15.97:0/2440773305 shutdown_connections 2023-07-06T03:47:32.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.153+0000 7f3a921ff700 1 -- 172.21.15.97:0/2440773305 >> 172.21.15.97:0/2440773305 conn(0x7f3a8c06a6c0 msgr2=0x7f3a8c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:32.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.153+0000 7f3a921ff700 1 -- 172.21.15.97:0/2440773305 shutdown_connections 2023-07-06T03:47:32.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.153+0000 7f3a921ff700 1 -- 172.21.15.97:0/2440773305 wait complete. 2023-07-06T03:47:32.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.155+0000 7f3a921ff700 1 Processor -- start 2023-07-06T03:47:32.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.155+0000 7f3a921ff700 1 -- start start 2023-07-06T03:47:32.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.156+0000 7f3a921ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 0x7f3a8c14a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.156+0000 7f3a921ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a8c0cbd20 0x7f3a8c15d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.156+0000 7f3a921ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c15dab0 0x7f3a8c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.156+0000 7f3a921ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c14a710 con 0x7f3a78003f80 2023-07-06T03:47:32.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.156+0000 7f3a921ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c15dff0 con 0x7f3a8c0cbd20 2023-07-06T03:47:32.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.156+0000 7f3a921ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c0caf60 con 0x7f3a8c15dab0 2023-07-06T03:47:32.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.156+0000 7f3a8b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 0x7f3a8c14a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.156+0000 7f3a8b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 0x7f3a8c14a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40800/0 (socket says 172.21.15.97:40800) 2023-07-06T03:47:32.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.156+0000 7f3a8b7fe700 1 -- 172.21.15.97:0/2711724540 learned_addr learned my addr 172.21.15.97:0/2711724540 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:32.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a8bfff700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c15dab0 0x7f3a8c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a83fff700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a8c0cbd20 0x7f3a8c15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a8b7fe700 1 -- 172.21.15.97:0/2711724540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c15dab0 msgr2=0x7f3a8c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:32.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a8b7fe700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c15dab0 0x7f3a8c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a8b7fe700 1 -- 172.21.15.97:0/2711724540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a8c0cbd20 msgr2=0x7f3a8c15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:32.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a8b7fe700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a8c0cbd20 0x7f3a8c15d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a8b7fe700 1 -- 172.21.15.97:0/2711724540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c0cb240 con 0x7f3a78003f80 2023-07-06T03:47:32.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a8bfff700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c15dab0 0x7f3a8c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:32.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a8b7fe700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 0x7f3a8c14a1d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00e820 tx=0x7f3a7c00ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:32.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a88ff9700 1 -- 172.21.15.97:0/2711724540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c00c620 con 0x7f3a78003f80 2023-07-06T03:47:32.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a88ff9700 1 -- 172.21.15.97:0/2711724540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c00c7a0 con 0x7f3a78003f80 2023-07-06T03:47:32.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a921ff700 1 -- 172.21.15.97:0/2711724540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8c0cb520 con 0x7f3a78003f80 2023-07-06T03:47:32.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a88ff9700 1 -- 172.21.15.97:0/2711724540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c00cb50 con 0x7f3a78003f80 2023-07-06T03:47:32.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.157+0000 7f3a921ff700 1 -- 172.21.15.97:0/2711724540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c153490 con 0x7f3a78003f80 2023-07-06T03:47:32.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.158+0000 7f3a921ff700 1 -- 172.21.15.97:0/2711724540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8c0ce040 con 0x7f3a78003f80 2023-07-06T03:47:32.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.159+0000 7f3a88ff9700 1 -- 172.21.15.97:0/2711724540 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a7c00ccb0 con 0x7f3a78003f80 2023-07-06T03:47:32.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.161+0000 7f3a88ff9700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3a6c06ad10 0x7f3a6c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.161+0000 7f3a88ff9700 1 -- 172.21.15.97:0/2711724540 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3a7c089080 con 0x7f3a78003f80 2023-07-06T03:47:32.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.161+0000 7f3a83fff700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3a6c06ad10 0x7f3a6c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.161+0000 7f3a83fff700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3a6c06ad10 0x7f3a6c06d1c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0c9c00 tx=0x7f3a74008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:32.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.165+0000 7f3a88ff9700 1 -- 172.21.15.97:0/2711724540 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3a7c0bb310 con 0x7f3a78003f80 2023-07-06T03:47:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.363+0000 7f3a921ff700 1 -- 172.21.15.97:0/2711724540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a8c04e340 con 0x7f3a78003f80 2023-07-06T03:47:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.363+0000 7f3a88ff9700 1 -- 172.21.15.97:0/2711724540 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a7c0543e0 con 0x7f3a78003f80 2023-07-06T03:47:32.365 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:47:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.365+0000 7f3a921ff700 1 -- 172.21.15.97:0/2711724540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3a6c06ad10 msgr2=0x7f3a6c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.365+0000 7f3a921ff700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3a6c06ad10 0x7f3a6c06d1c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0c9c00 tx=0x7f3a74008030 comp rx=0 tx=0).stop 2023-07-06T03:47:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.365+0000 7f3a921ff700 1 -- 172.21.15.97:0/2711724540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 msgr2=0x7f3a8c14a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.365+0000 7f3a921ff700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 0x7f3a8c14a1d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00e820 tx=0x7f3a7c00ebe0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.365+0000 7f3a921ff700 1 -- 172.21.15.97:0/2711724540 shutdown_connections 2023-07-06T03:47:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.365+0000 7f3a921ff700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3a6c06ad10 0x7f3a6c06d1c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.365+0000 7f3a921ff700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3a8c15dab0 0x7f3a8c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.365+0000 7f3a921ff700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a78003f80 0x7f3a8c14a1d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.365+0000 7f3a921ff700 1 --2- 172.21.15.97:0/2711724540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a8c0cbd20 0x7f3a8c15d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.365+0000 7f3a921ff700 1 -- 172.21.15.97:0/2711724540 >> 172.21.15.97:0/2711724540 conn(0x7f3a8c06a6c0 msgr2=0x7f3a8c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.366+0000 7f3a921ff700 1 -- 172.21.15.97:0/2711724540 shutdown_connections 2023-07-06T03:47:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.366+0000 7f3a921ff700 1 -- 172.21.15.97:0/2711724540 wait complete. 2023-07-06T03:47:32.385 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:47:32.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.580+0000 7f5c0d1ff700 1 Processor -- start 2023-07-06T03:47:32.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.581+0000 7f5c0d1ff700 1 -- start start 2023-07-06T03:47:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.582+0000 7f5c0d1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c0815c610 0x7f5c080b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.582+0000 7f5c0d1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c0806b360 0x7f5c080b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.582+0000 7f5c0d1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5c080d2210 0x7f5c080d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.582+0000 7f5c0d1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c080d0780 con 0x7f5c080d2210 2023-07-06T03:47:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.582+0000 7f5c0d1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0815c9e0 con 0x7f5c0806b360 2023-07-06T03:47:32.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.582+0000 7f5c0d1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080b5650 con 0x7f5c0815c610 2023-07-06T03:47:32.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.582+0000 7f5c0759e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5c080d2210 0x7f5c080d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.582+0000 7f5c0759e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5c080d2210 0x7f5c080d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40816/0 (socket says 172.21.15.97:40816) 2023-07-06T03:47:32.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.582+0000 7f5c0759e700 1 -- 172.21.15.97:0/2460138634 learned_addr learned my addr 172.21.15.97:0/2460138634 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:32.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.582+0000 7f5c06d9d700 1 --2- 172.21.15.97:0/2460138634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c0815c610 0x7f5c080b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.582+0000 7f5bfffff700 1 --2- 172.21.15.97:0/2460138634 >> v2:172.21.15.146:3300/0 conn(0x7f5c0806b360 0x7f5c080b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.583+0000 7f5c0759e700 1 -- 172.21.15.97:0/2460138634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c0815c610 msgr2=0x7f5c080b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:32.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.583+0000 7f5c0759e700 1 --2- 172.21.15.97:0/2460138634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c0815c610 0x7f5c080b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.583+0000 7f5c0759e700 1 -- 172.21.15.97:0/2460138634 >> v2:172.21.15.146:3300/0 conn(0x7f5c0806b360 msgr2=0x7f5c080b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:32.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.583+0000 7f5c0759e700 1 --2- 172.21.15.97:0/2460138634 >> v2:172.21.15.146:3300/0 conn(0x7f5c0806b360 0x7f5c080b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.583+0000 7f5c0759e700 1 -- 172.21.15.97:0/2460138634 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c080d01c0 con 0x7f5c080d2210 2023-07-06T03:47:32.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.583+0000 7f5c0759e700 1 --2- 172.21.15.97:0/2460138634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c080d2210 0x7f5c080d1310 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5c080b58b0 tx=0x7f5c0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403e572af8a13a47 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:32.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.583+0000 7f5c0659c700 1 -- 172.21.15.97:0/2460138634 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000e070 con 0x7f5c080d2210 2023-07-06T03:47:32.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.583+0000 7f5c0659c700 1 -- 172.21.15.97:0/2460138634 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0000b8d0 con 0x7f5c080d2210 2023-07-06T03:47:32.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.583+0000 7f5c0659c700 1 -- 172.21.15.97:0/2460138634 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000bc80 con 0x7f5c080d2210 2023-07-06T03:47:32.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.584+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/2460138634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c080d2210 msgr2=0x7f5c080d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:32.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.584+0000 7f5c0d1ff700 1 --2- 172.21.15.97:0/2460138634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c080d2210 0x7f5c080d1310 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5c080b58b0 tx=0x7f5c0000b330 comp rx=0 tx=0).stop 2023-07-06T03:47:32.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.584+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/2460138634 shutdown_connections 2023-07-06T03:47:32.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.584+0000 7f5c0d1ff700 1 --2- 172.21.15.97:0/2460138634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c080d2210 0x7f5c080d1310 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.584+0000 7f5c0d1ff700 1 --2- 172.21.15.97:0/2460138634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c0815c610 0x7f5c080b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.584+0000 7f5c0d1ff700 1 --2- 172.21.15.97:0/2460138634 >> v2:172.21.15.146:3300/0 conn(0x7f5c0806b360 0x7f5c080b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.584+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/2460138634 >> 172.21.15.97:0/2460138634 conn(0x7f5c0806a6c0 msgr2=0x7f5c0806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.584+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/2460138634 shutdown_connections 2023-07-06T03:47:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.584+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/2460138634 wait complete. 2023-07-06T03:47:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.585+0000 7f5c0d1ff700 1 Processor -- start 2023-07-06T03:47:32.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.586+0000 7f5c0d1ff700 1 -- start start 2023-07-06T03:47:32.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.586+0000 7f5c0d1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c0806b360 0x7f5c0814b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.586+0000 7f5c0d1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c080d2210 0x7f5c080cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c0d1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c0815c610 0x7f5c080b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c0d1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0814b690 con 0x7f5c0806b360 2023-07-06T03:47:32.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c0d1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080cb8d0 con 0x7f5c080d2210 2023-07-06T03:47:32.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c0d1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080cbbb0 con 0x7f5c0815c610 2023-07-06T03:47:32.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c06d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c0806b360 0x7f5c0814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c06d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c0806b360 0x7f5c0814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40820/0 (socket says 172.21.15.97:40820) 2023-07-06T03:47:32.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c06d9d700 1 -- 172.21.15.97:0/1053392822 learned_addr learned my addr 172.21.15.97:0/1053392822 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:32.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c06d9d700 1 -- 172.21.15.97:0/1053392822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c0815c610 msgr2=0x7f5c080b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:32.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c06d9d700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c0815c610 0x7f5c080b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c06d9d700 1 -- 172.21.15.97:0/1053392822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c080d2210 msgr2=0x7f5c080cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:32.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c06d9d700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c080d2210 0x7f5c080cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c06d9d700 1 -- 172.21.15.97:0/1053392822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c00009b30 con 0x7f5c0806b360 2023-07-06T03:47:32.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5c06d9d700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c0806b360 0x7f5c0814b150 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800e870 tx=0x7f5bf800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:32.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.587+0000 7f5bff7fe700 1 -- 172.21.15.97:0/1053392822 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8009590 con 0x7f5c0806b360 2023-07-06T03:47:32.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.588+0000 7f5bff7fe700 1 -- 172.21.15.97:0/1053392822 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf800ee70 con 0x7f5c0806b360 2023-07-06T03:47:32.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.588+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/1053392822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c080cbe90 con 0x7f5c0806b360 2023-07-06T03:47:32.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.588+0000 7f5bff7fe700 1 -- 172.21.15.97:0/1053392822 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf800ca50 con 0x7f5c0806b360 2023-07-06T03:47:32.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.588+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/1053392822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c080cc450 con 0x7f5c0806b360 2023-07-06T03:47:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.589+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/1053392822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0804e340 con 0x7f5c0806b360 2023-07-06T03:47:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.590+0000 7f5bff7fe700 1 -- 172.21.15.97:0/1053392822 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5bf800cbb0 con 0x7f5c0806b360 2023-07-06T03:47:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.590+0000 7f5bff7fe700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5be806ace0 0x7f5be806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:32.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.591+0000 7f5bff7fe700 1 -- 172.21.15.97:0/1053392822 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5bf8089470 con 0x7f5c0806b360 2023-07-06T03:47:32.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.591+0000 7f5bfffff700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5be806ace0 0x7f5be806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:32.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.591+0000 7f5bfffff700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5be806ace0 0x7f5be806d190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5c080c99f0 tx=0x7f5bf00095a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:32.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.595+0000 7f5bff7fe700 1 -- 172.21.15.97:0/1053392822 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5bf80582f0 con 0x7f5c0806b360 2023-07-06T03:47:32.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.790+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/1053392822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5c080b47a0 con 0x7f5c0806b360 2023-07-06T03:47:32.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.790+0000 7f5bff7fe700 1 -- 172.21.15.97:0/1053392822 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5bf800ce60 con 0x7f5c0806b360 2023-07-06T03:47:32.792 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:47:32.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.792+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/1053392822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5be806ace0 msgr2=0x7f5be806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:32.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.792+0000 7f5c0d1ff700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5be806ace0 0x7f5be806d190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5c080c99f0 tx=0x7f5bf00095a0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.792+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/1053392822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c0806b360 msgr2=0x7f5c0814b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:32.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.792+0000 7f5c0d1ff700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c0806b360 0x7f5c0814b150 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800e870 tx=0x7f5bf800ec30 comp rx=0 tx=0).stop 2023-07-06T03:47:32.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.792+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/1053392822 shutdown_connections 2023-07-06T03:47:32.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.793+0000 7f5c0d1ff700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5be806ace0 0x7f5be806d190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.793+0000 7f5c0d1ff700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c0815c610 0x7f5c080b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.793+0000 7f5c0d1ff700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c0806b360 0x7f5c0814b150 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.793+0000 7f5c0d1ff700 1 --2- 172.21.15.97:0/1053392822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c080d2210 0x7f5c080cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:32.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.793+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/1053392822 >> 172.21.15.97:0/1053392822 conn(0x7f5c0806a6c0 msgr2=0x7f5c080b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:32.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.793+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/1053392822 shutdown_connections 2023-07-06T03:47:32.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:32.793+0000 7f5c0d1ff700 1 -- 172.21.15.97:0/1053392822 wait complete. 2023-07-06T03:47:32.813 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T03:47:33.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.007+0000 7f867d7ff700 1 Processor -- start 2023-07-06T03:47:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.008+0000 7f867d7ff700 1 -- start start 2023-07-06T03:47:33.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.009+0000 7f867d7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780b8410 0x7f8678167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:33.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.009+0000 7f867d7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780cdae0 con 0x7f867815c520 2023-07-06T03:47:33.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.009+0000 7f867d7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780b6640 con 0x7f86780cd390 2023-07-06T03:47:33.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.009+0000 7f867d7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86780b6920 con 0x7f86780b8410 2023-07-06T03:47:33.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.009+0000 7f8676ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f867815c520 0x7f8678160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34098/0 (socket says 172.21.15.97:34098) 2023-07-06T03:47:33.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.009+0000 7f8676ffd700 1 -- 172.21.15.97:0/1785423143 learned_addr learned my addr 172.21.15.97:0/1785423143 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.009+0000 7f86777fe700 1 --2- 172.21.15.97:0/1785423143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780b8410 0x7f8678167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.009+0000 7f8675ffb700 1 -- 172.21.15.97:0/1785423143 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979647074 0 0) 0x7f86780cdae0 con 0x7f867815c520 2023-07-06T03:47:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.010+0000 7f8675ffb700 1 -- 172.21.15.97:0/1785423143 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86600035b0 con 0x7f867815c520 2023-07-06T03:47:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.010+0000 7f8675ffb700 1 -- 172.21.15.97:0/1785423143 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955598454 0 0) 0x7f86780b6640 con 0x7f86780cd390 2023-07-06T03:47:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.010+0000 7f8675ffb700 1 -- 172.21.15.97:0/1785423143 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86780cdae0 con 0x7f86780cd390 2023-07-06T03:47:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.010+0000 7f86777fe700 1 -- 172.21.15.97:0/1785423143 >> v1:172.21.15.146:6789/0 conn(0x7f86780cd390 legacy=0x7f8678160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.010+0000 7f86777fe700 1 -- 172.21.15.97:0/1785423143 >> v1:172.21.15.97:6789/0 conn(0x7f867815c520 legacy=0x7f8678160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.010+0000 7f86777fe700 1 -- 172.21.15.97:0/1785423143 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780b6c00 con 0x7f86780b8410 2023-07-06T03:47:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.010+0000 7f86777fe700 1 --2- 172.21.15.97:0/1785423143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780b8410 0x7f8678167510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8670009dc0 tx=0x7f867000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23eee56d2275c0aa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:33.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.010+0000 7f8675ffb700 1 -- 172.21.15.97:0/1785423143 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867000e070 con 0x7f86780b8410 2023-07-06T03:47:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.010+0000 7f8675ffb700 1 -- 172.21.15.97:0/1785423143 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867000bb20 con 0x7f86780b8410 2023-07-06T03:47:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.010+0000 7f8675ffb700 1 -- 172.21.15.97:0/1785423143 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867000be90 con 0x7f86780b8410 2023-07-06T03:47:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.011+0000 7f867d7ff700 1 -- 172.21.15.97:0/1785423143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780b8410 msgr2=0x7f8678167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:33.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.011+0000 7f867d7ff700 1 --2- 172.21.15.97:0/1785423143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780b8410 0x7f8678167510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8670009dc0 tx=0x7f867000b330 comp rx=0 tx=0).stop 2023-07-06T03:47:33.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.011+0000 7f867d7ff700 1 -- 172.21.15.97:0/1785423143 shutdown_connections 2023-07-06T03:47:33.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.011+0000 7f867d7ff700 1 --2- 172.21.15.97:0/1785423143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780b8410 0x7f8678167510 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.011+0000 7f867d7ff700 1 -- 172.21.15.97:0/1785423143 >> 172.21.15.97:0/1785423143 conn(0x7f8678071790 msgr2=0x7f8678071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.011+0000 7f867d7ff700 1 -- 172.21.15.97:0/1785423143 shutdown_connections 2023-07-06T03:47:33.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.011+0000 7f867d7ff700 1 -- 172.21.15.97:0/1785423143 wait complete. 2023-07-06T03:47:33.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.013+0000 7f867d7ff700 1 Processor -- start 2023-07-06T03:47:33.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.013+0000 7f867d7ff700 1 -- start start 2023-07-06T03:47:33.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f867d7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86780b8410 0x7f86780cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:33.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f867d7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780cd390 0x7f86780ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:33.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f867d7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867815c520 0x7f86780d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:33.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f867d7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780d2670 con 0x7f86780b8410 2023-07-06T03:47:33.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f867d7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780caa20 con 0x7f867815c520 2023-07-06T03:47:33.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f867d7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8678154fe0 con 0x7f86780cd390 2023-07-06T03:47:33.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f8676ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86780b8410 0x7f86780cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:33.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f8676ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86780b8410 0x7f86780cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40836/0 (socket says 172.21.15.97:40836) 2023-07-06T03:47:33.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f8676ffd700 1 -- 172.21.15.97:0/3537813414 learned_addr learned my addr 172.21.15.97:0/3537813414 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:33.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f86767fc700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780cd390 0x7f86780ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:33.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f86777fe700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867815c520 0x7f86780d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:33.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f8676ffd700 1 -- 172.21.15.97:0/3537813414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780cd390 msgr2=0x7f86780ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:33.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f8676ffd700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780cd390 0x7f86780ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f8676ffd700 1 -- 172.21.15.97:0/3537813414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867815c520 msgr2=0x7f86780d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:33.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f8676ffd700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867815c520 0x7f86780d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f8676ffd700 1 -- 172.21.15.97:0/3537813414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670009a70 con 0x7f86780b8410 2023-07-06T03:47:33.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f86767fc700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780cd390 0x7f86780ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:33.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.014+0000 7f8676ffd700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86780b8410 0x7f86780cc890 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f866800c700 tx=0x7f866800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:33.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.015+0000 7f865f7fe700 1 -- 172.21.15.97:0/3537813414 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668005d70 con 0x7f86780b8410 2023-07-06T03:47:33.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.015+0000 7f865f7fe700 1 -- 172.21.15.97:0/3537813414 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86680070d0 con 0x7f86780b8410 2023-07-06T03:47:33.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.015+0000 7f867d7ff700 1 -- 172.21.15.97:0/3537813414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86781552c0 con 0x7f86780b8410 2023-07-06T03:47:33.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.015+0000 7f865f7fe700 1 -- 172.21.15.97:0/3537813414 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668007460 con 0x7f86780b8410 2023-07-06T03:47:33.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.015+0000 7f867d7ff700 1 -- 172.21.15.97:0/3537813414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8678155880 con 0x7f86780b8410 2023-07-06T03:47:33.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.016+0000 7f867d7ff700 1 -- 172.21.15.97:0/3537813414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867804e340 con 0x7f86780b8410 2023-07-06T03:47:33.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.017+0000 7f865f7fe700 1 -- 172.21.15.97:0/3537813414 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f86680075c0 con 0x7f86780b8410 2023-07-06T03:47:33.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.018+0000 7f865f7fe700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f866406ace0 0x7f866406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:33.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.018+0000 7f865f7fe700 1 -- 172.21.15.97:0/3537813414 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8668088770 con 0x7f86780b8410 2023-07-06T03:47:33.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.018+0000 7f86767fc700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f866406ace0 0x7f866406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:33.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.018+0000 7f86767fc700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f866406ace0 0x7f866406d190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f867814d8e0 tx=0x7f866c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:33.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.022+0000 7f865f7fe700 1 -- 172.21.15.97:0/3537813414 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8668053b80 con 0x7f86780b8410 2023-07-06T03:47:33.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.221+0000 7f867d7ff700 1 -- 172.21.15.97:0/3537813414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f86780b4f40 con 0x7f86780b8410 2023-07-06T03:47:33.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.221+0000 7f865f7fe700 1 -- 172.21.15.97:0/3537813414 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8668074e60 con 0x7f86780b8410 2023-07-06T03:47:33.223 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:33.223 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-07-06T03:47:33.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:33.224 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-07-06T03:47:33.224 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-07-06T03:47:33.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:33.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:47:33.224 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 2, 2023-07-06T03:47:33.224 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": "rbd" 2023-07-06T03:47:33.225 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:33.225 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-07-06T03:47:33.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.223+0000 7f867d7ff700 1 -- 172.21.15.97:0/3537813414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f866406ace0 msgr2=0x7f866406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:33.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.223+0000 7f867d7ff700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f866406ace0 0x7f866406d190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f867814d8e0 tx=0x7f866c009040 comp rx=0 tx=0).stop 2023-07-06T03:47:33.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.224+0000 7f867d7ff700 1 -- 172.21.15.97:0/3537813414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86780b8410 msgr2=0x7f86780cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:33.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.224+0000 7f867d7ff700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86780b8410 0x7f86780cc890 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f866800c700 tx=0x7f866800cac0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.224+0000 7f867d7ff700 1 -- 172.21.15.97:0/3537813414 shutdown_connections 2023-07-06T03:47:33.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.224+0000 7f867d7ff700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f866406ace0 0x7f866406d190 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.224+0000 7f867d7ff700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f867815c520 0x7f86780d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.224+0000 7f867d7ff700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86780b8410 0x7f86780cc890 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.224+0000 7f867d7ff700 1 --2- 172.21.15.97:0/3537813414 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86780cd390 0x7f86780ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.224+0000 7f867d7ff700 1 -- 172.21.15.97:0/3537813414 >> 172.21.15.97:0/3537813414 conn(0x7f8678071790 msgr2=0x7f8678071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:33.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.224+0000 7f867d7ff700 1 -- 172.21.15.97:0/3537813414 shutdown_connections 2023-07-06T03:47:33.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.224+0000 7f867d7ff700 1 -- 172.21.15.97:0/3537813414 wait complete. 2023-07-06T03:47:33.243 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T03:47:33.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T03:47:33.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.440+0000 7f80867ff700 1 Processor -- start 2023-07-06T03:47:33.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.440+0000 7f80867ff700 1 -- start start 2023-07-06T03:47:33.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.441+0000 7f80867ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80800b85f0 0x7f808006b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:33.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.441+0000 7f80867ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80800b4e70 0x7f80800d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:33.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.441+0000 7f80867ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80800cd330 con 0x7f80800cefa0 2023-07-06T03:47:33.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.441+0000 7f80867ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80800b89c0 con 0x7f80800b85f0 2023-07-06T03:47:33.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.441+0000 7f80867ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80800d02a0 con 0x7f80800b4e70 2023-07-06T03:47:33.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.441+0000 7f807ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80800b85f0 0x7f808006b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:33.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.441+0000 7f807f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f80800cefa0 0x7f80800b4760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34102/0 (socket says 172.21.15.97:34102) 2023-07-06T03:47:33.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.441+0000 7f807f7fe700 1 -- 172.21.15.97:0/1644600983 learned_addr learned my addr 172.21.15.97:0/1644600983 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:33.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.442+0000 7f8084d9c700 1 --2- 172.21.15.97:0/1644600983 >> v2:172.21.15.146:3300/0 conn(0x7f80800b4e70 0x7f80800d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:33.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.442+0000 7f807ffff700 1 -- 172.21.15.97:0/1644600983 >> v2:172.21.15.146:3300/0 conn(0x7f80800b4e70 msgr2=0x7f80800d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:33.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.442+0000 7f807ffff700 1 --2- 172.21.15.97:0/1644600983 >> v2:172.21.15.146:3300/0 conn(0x7f80800b4e70 0x7f80800d09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.442+0000 7f807ffff700 1 -- 172.21.15.97:0/1644600983 >> v1:172.21.15.97:6789/0 conn(0x7f80800cefa0 legacy=0x7f80800b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:33.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.442+0000 7f807effd700 1 -- 172.21.15.97:0/1644600983 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700892787 0 0) 0x7f80800cd330 con 0x7f80800cefa0 2023-07-06T03:47:33.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.442+0000 7f807ffff700 1 -- 172.21.15.97:0/1644600983 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800d0540 con 0x7f80800b85f0 2023-07-06T03:47:33.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.442+0000 7f807ffff700 1 --2- 172.21.15.97:0/1644600983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80800b85f0 0x7f808006b0c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f80800b2710 tx=0x7f8070009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e186ea313bb8d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:33.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.442+0000 7f807effd700 1 -- 172.21.15.97:0/1644600983 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070011830 con 0x7f80800b85f0 2023-07-06T03:47:33.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.442+0000 7f807effd700 1 -- 172.21.15.97:0/1644600983 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8070011990 con 0x7f80800b85f0 2023-07-06T03:47:33.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.442+0000 7f807effd700 1 -- 172.21.15.97:0/1644600983 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070011d20 con 0x7f80800b85f0 2023-07-06T03:47:33.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.443+0000 7f80867ff700 1 -- 172.21.15.97:0/1644600983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80800b85f0 msgr2=0x7f808006b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:33.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.443+0000 7f80867ff700 1 --2- 172.21.15.97:0/1644600983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80800b85f0 0x7f808006b0c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f80800b2710 tx=0x7f8070009b20 comp rx=0 tx=0).stop 2023-07-06T03:47:33.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.443+0000 7f80867ff700 1 -- 172.21.15.97:0/1644600983 shutdown_connections 2023-07-06T03:47:33.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.443+0000 7f80867ff700 1 --2- 172.21.15.97:0/1644600983 >> v2:172.21.15.146:3300/0 conn(0x7f80800b4e70 0x7f80800d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.443+0000 7f80867ff700 1 --2- 172.21.15.97:0/1644600983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80800b85f0 0x7f808006b0c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.443+0000 7f80867ff700 1 -- 172.21.15.97:0/1644600983 >> 172.21.15.97:0/1644600983 conn(0x7f808006a5b0 msgr2=0x7f808006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:33.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.443+0000 7f80867ff700 1 -- 172.21.15.97:0/1644600983 shutdown_connections 2023-07-06T03:47:33.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.443+0000 7f80867ff700 1 -- 172.21.15.97:0/1644600983 wait complete. 2023-07-06T03:47:33.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.445+0000 7f80867ff700 1 Processor -- start 2023-07-06T03:47:33.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.445+0000 7f80867ff700 1 -- start start 2023-07-06T03:47:33.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f80867ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80800b4e70 0x7f80801f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:33.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f80867ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80800b85f0 0x7f80801f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:33.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f80867ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80800cefa0 0x7f808015ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:33.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f80867ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808015f410 con 0x7f80800b4e70 2023-07-06T03:47:33.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f80867ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808015eb70 con 0x7f80800b85f0 2023-07-06T03:47:33.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f80867ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808015ee50 con 0x7f80800cefa0 2023-07-06T03:47:33.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f807ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80800b4e70 0x7f80801f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:33.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f807ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80800b4e70 0x7f80801f5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40856/0 (socket says 172.21.15.97:40856) 2023-07-06T03:47:33.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f807ffff700 1 -- 172.21.15.97:0/347128818 learned_addr learned my addr 172.21.15.97:0/347128818 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:33.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f8084d9c700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80800cefa0 0x7f808015ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:33.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f807f7fe700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80800b85f0 0x7f80801f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:33.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f807ffff700 1 -- 172.21.15.97:0/347128818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80800cefa0 msgr2=0x7f808015ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:33.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f807ffff700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80800cefa0 0x7f808015ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f807ffff700 1 -- 172.21.15.97:0/347128818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80800b85f0 msgr2=0x7f80801f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:33.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f807ffff700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80800b85f0 0x7f80801f5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:33.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f807ffff700 1 -- 172.21.15.97:0/347128818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80700094a0 con 0x7f80800b4e70 2023-07-06T03:47:33.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.446+0000 7f807f7fe700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80800b85f0 0x7f80801f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:33.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.447+0000 7f807ffff700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80800b4e70 0x7f80801f5100 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8070000c00 tx=0x7f8070004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:33.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.447+0000 7f807cff9700 1 -- 172.21.15.97:0/347128818 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80700036a0 con 0x7f80800b4e70 2023-07-06T03:47:33.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.447+0000 7f807cff9700 1 -- 172.21.15.97:0/347128818 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807001f070 con 0x7f80800b4e70 2023-07-06T03:47:33.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.447+0000 7f80867ff700 1 -- 172.21.15.97:0/347128818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808015f130 con 0x7f80800b4e70 2023-07-06T03:47:33.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.447+0000 7f807cff9700 1 -- 172.21.15.97:0/347128818 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070003a60 con 0x7f80800b4e70 2023-07-06T03:47:33.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.447+0000 7f80867ff700 1 -- 172.21.15.97:0/347128818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80801592b0 con 0x7f80800b4e70 2023-07-06T03:47:33.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.448+0000 7f80867ff700 1 -- 172.21.15.97:0/347128818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8080159440 con 0x7f80800b4e70 2023-07-06T03:47:33.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.449+0000 7f807cff9700 1 -- 172.21.15.97:0/347128818 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8070003bc0 con 0x7f80800b4e70 2023-07-06T03:47:33.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.450+0000 7f807cff9700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f806c06ace0 0x7f806c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:33.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.450+0000 7f807cff9700 1 -- 172.21.15.97:0/347128818 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8070021070 con 0x7f80800b4e70 2023-07-06T03:47:33.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.450+0000 7f807f7fe700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f806c06ace0 0x7f806c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:33.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.450+0000 7f807f7fe700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f806c06ace0 0x7f806c06d190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8074007fd0 tx=0x7f8074009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:33.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.454+0000 7f807cff9700 1 -- 172.21.15.97:0/347128818 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8070055f90 con 0x7f80800b4e70 2023-07-06T03:47:33.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:47:33.526 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:47:33.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:33.656+0000 7f80867ff700 1 -- 172.21.15.97:0/347128818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f808004e340 con 0x7f80800b4e70 2023-07-06T03:47:34.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.654+0000 7f807cff9700 1 -- 172.21.15.97:0/347128818 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7f80700595b0 con 0x7f80800b4e70 2023-07-06T03:47:34.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.659+0000 7f80867ff700 1 -- 172.21.15.97:0/347128818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f806c06ace0 msgr2=0x7f806c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:34.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.659+0000 7f80867ff700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f806c06ace0 0x7f806c06d190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8074007fd0 tx=0x7f8074009040 comp rx=0 tx=0).stop 2023-07-06T03:47:34.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.660+0000 7f80867ff700 1 -- 172.21.15.97:0/347128818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80800b4e70 msgr2=0x7f80801f5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:34.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.660+0000 7f80867ff700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80800b4e70 0x7f80801f5100 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8070000c00 tx=0x7f8070004750 comp rx=0 tx=0).stop 2023-07-06T03:47:34.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.660+0000 7f80867ff700 1 -- 172.21.15.97:0/347128818 shutdown_connections 2023-07-06T03:47:34.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.660+0000 7f80867ff700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f806c06ace0 0x7f806c06d190 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:34.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.660+0000 7f80867ff700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80800cefa0 0x7f808015ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:34.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.660+0000 7f80867ff700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80800b4e70 0x7f80801f5100 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:34.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.660+0000 7f80867ff700 1 --2- 172.21.15.97:0/347128818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80800b85f0 0x7f80801f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:34.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.660+0000 7f80867ff700 1 -- 172.21.15.97:0/347128818 >> 172.21.15.97:0/347128818 conn(0x7f808006a5b0 msgr2=0x7f80800b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:34.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.660+0000 7f80867ff700 1 -- 172.21.15.97:0/347128818 shutdown_connections 2023-07-06T03:47:34.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.660+0000 7f80867ff700 1 -- 172.21.15.97:0/347128818 wait complete. 2023-07-06T03:47:34.664 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-07-06T03:47:34.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T03:47:34.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.885+0000 7efda49ff700 1 Processor -- start 2023-07-06T03:47:34.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.886+0000 7efda49ff700 1 -- start start 2023-07-06T03:47:34.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.887+0000 7efda49ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efda00d0920 0x7efda00d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:34.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.887+0000 7efda49ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 0x7efda00ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:34.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.887+0000 7efda49ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda00d0080 con 0x7efda00d2970 2023-07-06T03:47:34.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.887+0000 7efda49ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda00d0360 con 0x7efda00d0920 2023-07-06T03:47:34.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.887+0000 7efda49ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda00d0640 con 0x7efda00b20c0 2023-07-06T03:47:34.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.887+0000 7efd9ed9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 0x7efda00ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:34.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.887+0000 7efd9ed9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 0x7efda00ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:43890/0 (socket says 172.21.15.97:43890) 2023-07-06T03:47:34.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.887+0000 7efd9ed9d700 1 -- 172.21.15.97:0/1924578308 learned_addr learned my addr 172.21.15.97:0/1924578308 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:34.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.887+0000 7efd97fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7efda00d2970 0x7efda00cdc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:41432/0 (socket says 172.21.15.97:41432) 2023-07-06T03:47:34.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.887+0000 7efd9e59c700 1 --2- 172.21.15.97:0/1924578308 >> v2:172.21.15.146:3300/0 conn(0x7efda00d0920 0x7efda00d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:34.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.888+0000 7efd9ed9d700 1 -- 172.21.15.97:0/1924578308 >> v2:172.21.15.146:3300/0 conn(0x7efda00d0920 msgr2=0x7efda00d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:34.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.888+0000 7efd9ed9d700 1 --2- 172.21.15.97:0/1924578308 >> v2:172.21.15.146:3300/0 conn(0x7efda00d0920 0x7efda00d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:34.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.888+0000 7efd9ed9d700 1 -- 172.21.15.97:0/1924578308 >> v1:172.21.15.146:6789/0 conn(0x7efda00d2970 legacy=0x7efda00cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:34.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.888+0000 7efd9dd9b700 1 -- 172.21.15.97:0/1924578308 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621124032 0 0) 0x7efda00d0080 con 0x7efda00d2970 2023-07-06T03:47:34.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.888+0000 7efd9ed9d700 1 -- 172.21.15.97:0/1924578308 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda00cd440 con 0x7efda00b20c0 2023-07-06T03:47:34.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.888+0000 7efd9ed9d700 1 --2- 172.21.15.97:0/1924578308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 0x7efda00ce350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efd98009e50 tx=0x7efd9800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28dcce424d60124a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:34.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.888+0000 7efd9dd9b700 1 -- 172.21.15.97:0/1924578308 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9800e070 con 0x7efda00b20c0 2023-07-06T03:47:34.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.888+0000 7efd9dd9b700 1 -- 172.21.15.97:0/1924578308 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9800b910 con 0x7efda00b20c0 2023-07-06T03:47:34.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.888+0000 7efd9dd9b700 1 -- 172.21.15.97:0/1924578308 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9800bcc0 con 0x7efda00b20c0 2023-07-06T03:47:34.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.889+0000 7efda49ff700 1 -- 172.21.15.97:0/1924578308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 msgr2=0x7efda00ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:34.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.889+0000 7efda49ff700 1 --2- 172.21.15.97:0/1924578308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 0x7efda00ce350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efd98009e50 tx=0x7efd9800b330 comp rx=0 tx=0).stop 2023-07-06T03:47:34.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.889+0000 7efda49ff700 1 -- 172.21.15.97:0/1924578308 shutdown_connections 2023-07-06T03:47:34.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.889+0000 7efda49ff700 1 --2- 172.21.15.97:0/1924578308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 0x7efda00ce350 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:34.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.889+0000 7efda49ff700 1 --2- 172.21.15.97:0/1924578308 >> v2:172.21.15.146:3300/0 conn(0x7efda00d0920 0x7efda00d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:34.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.889+0000 7efda49ff700 1 -- 172.21.15.97:0/1924578308 >> 172.21.15.97:0/1924578308 conn(0x7efda009c0d0 msgr2=0x7efda00d2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.889+0000 7efda49ff700 1 -- 172.21.15.97:0/1924578308 shutdown_connections 2023-07-06T03:47:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.889+0000 7efda49ff700 1 -- 172.21.15.97:0/1924578308 wait complete. 2023-07-06T03:47:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.891+0000 7efda49ff700 1 Processor -- start 2023-07-06T03:47:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.891+0000 7efda49ff700 1 -- start start 2023-07-06T03:47:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efda49ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 0x7efda015dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efda49ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda00d0920 0x7efda00b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efda49ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efda00b3880 0x7efda01574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efda49ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda00b7270 con 0x7efda00d0920 2023-07-06T03:47:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efda49ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda00b7550 con 0x7efda00b3880 2023-07-06T03:47:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efda49ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda00b7830 con 0x7efda00b20c0 2023-07-06T03:47:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd97fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda00d0920 0x7efda00b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd97fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda00d0920 0x7efda00b8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40862/0 (socket says 172.21.15.97:40862) 2023-07-06T03:47:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd97fff700 1 -- 172.21.15.97:0/513454601 learned_addr learned my addr 172.21.15.97:0/513454601 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd9ed9d700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efda00b3880 0x7efda01574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd9e59c700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 0x7efda015dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd97fff700 1 -- 172.21.15.97:0/513454601 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 msgr2=0x7efda015dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd97fff700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 0x7efda015dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd97fff700 1 -- 172.21.15.97:0/513454601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efda00b3880 msgr2=0x7efda01574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd97fff700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efda00b3880 0x7efda01574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd97fff700 1 -- 172.21.15.97:0/513454601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd98009b30 con 0x7efda00d0920 2023-07-06T03:47:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd9ed9d700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efda00b3880 0x7efda01574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.892+0000 7efd9e59c700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 0x7efda015dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.893+0000 7efd97fff700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda00d0920 0x7efda00b8310 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efd8800c550 tx=0x7efd8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.893+0000 7efd96ffd700 1 -- 172.21.15.97:0/513454601 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88004c80 con 0x7efda00d0920 2023-07-06T03:47:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.893+0000 7efd96ffd700 1 -- 172.21.15.97:0/513454601 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd88004e00 con 0x7efda00d0920 2023-07-06T03:47:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.893+0000 7efd96ffd700 1 -- 172.21.15.97:0/513454601 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88007310 con 0x7efda00d0920 2023-07-06T03:47:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.893+0000 7efda49ff700 1 -- 172.21.15.97:0/513454601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda00b7b10 con 0x7efda00d0920 2023-07-06T03:47:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.893+0000 7efda49ff700 1 -- 172.21.15.97:0/513454601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda015e2a0 con 0x7efda00d0920 2023-07-06T03:47:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.894+0000 7efda49ff700 1 -- 172.21.15.97:0/513454601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda004e340 con 0x7efda00d0920 2023-07-06T03:47:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.895+0000 7efd96ffd700 1 -- 172.21.15.97:0/513454601 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efd88007470 con 0x7efda00d0920 2023-07-06T03:47:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.896+0000 7efd96ffd700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd8006ace0 0x7efd8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.896+0000 7efd96ffd700 1 -- 172.21.15.97:0/513454601 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5580+0+0 (secure 0 0 0) 0x7efd880896c0 con 0x7efda00d0920 2023-07-06T03:47:34.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.896+0000 7efd9e59c700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd8006ace0 0x7efd8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:34.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.897+0000 7efd9e59c700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd8006ace0 0x7efd8006d190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efda01e8ea0 tx=0x7efd90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:34.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:34.900+0000 7efd96ffd700 1 -- 172.21.15.97:0/513454601 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efd88054930 con 0x7efda00d0920 2023-07-06T03:47:35.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.093+0000 7efda49ff700 1 -- 172.21.15.97:0/513454601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efda00ceff0 con 0x7efda00d0920 2023-07-06T03:47:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.661+0000 7efd96ffd700 1 -- 172.21.15.97:0/513454601 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) v1 ==== 170+0+0 (secure 0 0 0) 0x7efd88057f50 con 0x7efda00d0920 2023-07-06T03:47:35.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.667+0000 7efda49ff700 1 -- 172.21.15.97:0/513454601 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd8006ace0 msgr2=0x7efd8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:35.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.667+0000 7efda49ff700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd8006ace0 0x7efd8006d190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efda01e8ea0 tx=0x7efd90008040 comp rx=0 tx=0).stop 2023-07-06T03:47:35.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.668+0000 7efda49ff700 1 -- 172.21.15.97:0/513454601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda00d0920 msgr2=0x7efda00b8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:35.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.668+0000 7efda49ff700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda00d0920 0x7efda00b8310 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efd8800c550 tx=0x7efd8800c910 comp rx=0 tx=0).stop 2023-07-06T03:47:35.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.669+0000 7efda49ff700 1 -- 172.21.15.97:0/513454601 shutdown_connections 2023-07-06T03:47:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.669+0000 7efda49ff700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd8006ace0 0x7efd8006d190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.669+0000 7efda49ff700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efda00b3880 0x7efda01574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.669+0000 7efda49ff700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efda00b20c0 0x7efda015dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.669+0000 7efda49ff700 1 --2- 172.21.15.97:0/513454601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda00d0920 0x7efda00b8310 secure :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efd8800c550 tx=0x7efd8800c910 comp rx=0 tx=0).stop 2023-07-06T03:47:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.669+0000 7efda49ff700 1 -- 172.21.15.97:0/513454601 >> 172.21.15.97:0/513454601 conn(0x7efda009c0d0 msgr2=0x7efda01e8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:35.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.674+0000 7efda49ff700 1 -- 172.21.15.97:0/513454601 shutdown_connections 2023-07-06T03:47:35.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.674+0000 7efda49ff700 1 -- 172.21.15.97:0/513454601 wait complete. 2023-07-06T03:47:35.677 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-07-06T03:47:35.698 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T03:47:35.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.893+0000 7f54115ff700 1 Processor -- start 2023-07-06T03:47:35.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.893+0000 7f54115ff700 1 -- start start 2023-07-06T03:47:35.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.895+0000 7f54115ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f540c15c1c0 0x7f540c0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:35.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.895+0000 7f54115ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f540c15c590 0x7f540c15d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:35.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.895+0000 7f54115ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f540c0719c0 0x7f540c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:35.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.895+0000 7f54115ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c0ccc30 con 0x7f540c15c590 2023-07-06T03:47:35.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.895+0000 7f54115ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c0ccf10 con 0x7f540c15c1c0 2023-07-06T03:47:35.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.895+0000 7f54115ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c0cd1f0 con 0x7f540c0719c0 2023-07-06T03:47:35.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.895+0000 7f5403fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f540c15c590 0x7f540c15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:35.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.895+0000 7f5403fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f540c15c590 0x7f540c15d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40870/0 (socket says 172.21.15.97:40870) 2023-07-06T03:47:35.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.895+0000 7f5403fff700 1 -- 172.21.15.97:0/2711429209 learned_addr learned my addr 172.21.15.97:0/2711429209 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:35.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.895+0000 7f540affd700 1 --2- 172.21.15.97:0/2711429209 >> v2:172.21.15.146:3300/0 conn(0x7f540c15c1c0 0x7f540c0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:35.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.895+0000 7f540b7fe700 1 --2- 172.21.15.97:0/2711429209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f540c0719c0 0x7f540c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:35.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.896+0000 7f5403fff700 1 -- 172.21.15.97:0/2711429209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f540c0719c0 msgr2=0x7f540c06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:35.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.896+0000 7f5403fff700 1 --2- 172.21.15.97:0/2711429209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f540c0719c0 0x7f540c06a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:35.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.896+0000 7f5403fff700 1 -- 172.21.15.97:0/2711429209 >> v2:172.21.15.146:3300/0 conn(0x7f540c15c1c0 msgr2=0x7f540c0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:35.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.896+0000 7f5403fff700 1 --2- 172.21.15.97:0/2711429209 >> v2:172.21.15.146:3300/0 conn(0x7f540c15c1c0 0x7f540c0cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:35.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.896+0000 7f5403fff700 1 -- 172.21.15.97:0/2711429209 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c15cb00 con 0x7f540c15c590 2023-07-06T03:47:35.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.896+0000 7f5403fff700 1 --2- 172.21.15.97:0/2711429209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f540c15c590 0x7f540c15d300 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f540c072570 tx=0x7f53f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ddb7146ec8d2cc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:35.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.896+0000 7f540a7fc700 1 -- 172.21.15.97:0/2711429209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f400b870 con 0x7f540c15c590 2023-07-06T03:47:35.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.896+0000 7f540a7fc700 1 -- 172.21.15.97:0/2711429209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f400b9d0 con 0x7f540c15c590 2023-07-06T03:47:35.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.896+0000 7f540a7fc700 1 -- 172.21.15.97:0/2711429209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f400bd60 con 0x7f540c15c590 2023-07-06T03:47:35.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.897+0000 7f54115ff700 1 -- 172.21.15.97:0/2711429209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f540c15c590 msgr2=0x7f540c15d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:35.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.897+0000 7f54115ff700 1 --2- 172.21.15.97:0/2711429209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f540c15c590 0x7f540c15d300 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f540c072570 tx=0x7f53f4009bd0 comp rx=0 tx=0).stop 2023-07-06T03:47:35.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.897+0000 7f54115ff700 1 -- 172.21.15.97:0/2711429209 shutdown_connections 2023-07-06T03:47:35.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.897+0000 7f54115ff700 1 --2- 172.21.15.97:0/2711429209 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f540c0719c0 0x7f540c06a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:35.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.897+0000 7f54115ff700 1 --2- 172.21.15.97:0/2711429209 >> v2:172.21.15.146:3300/0 conn(0x7f540c15c1c0 0x7f540c0cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:35.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.897+0000 7f54115ff700 1 --2- 172.21.15.97:0/2711429209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f540c15c590 0x7f540c15d300 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:35.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.897+0000 7f54115ff700 1 -- 172.21.15.97:0/2711429209 >> 172.21.15.97:0/2711429209 conn(0x7f540c0b1650 msgr2=0x7f540c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:35.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.897+0000 7f54115ff700 1 -- 172.21.15.97:0/2711429209 shutdown_connections 2023-07-06T03:47:35.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.897+0000 7f54115ff700 1 -- 172.21.15.97:0/2711429209 wait complete. 2023-07-06T03:47:35.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.898+0000 7f54115ff700 1 Processor -- start 2023-07-06T03:47:35.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.899+0000 7f54115ff700 1 -- start start 2023-07-06T03:47:35.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.899+0000 7f54115ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f540c0719c0 0x7f540c14a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:35.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f54115ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f540c15c1c0 0x7f540c06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:35.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f54115ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f540c15c590 0x7f540c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:35.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f54115ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c06e910 con 0x7f540c15c1c0 2023-07-06T03:47:35.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f54115ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c14aa30 con 0x7f540c0719c0 2023-07-06T03:47:35.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f54115ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c0b56d0 con 0x7f540c15c590 2023-07-06T03:47:35.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f5403fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f540c15c1c0 0x7f540c06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:35.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f5403fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f540c15c1c0 0x7f540c06cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40886/0 (socket says 172.21.15.97:40886) 2023-07-06T03:47:35.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f5403fff700 1 -- 172.21.15.97:0/3835903663 learned_addr learned my addr 172.21.15.97:0/3835903663 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:35.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f540b7fe700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f540c15c590 0x7f540c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:35.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f540affd700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f540c0719c0 0x7f540c14a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:35.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f5403fff700 1 -- 172.21.15.97:0/3835903663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f540c15c590 msgr2=0x7f540c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:35.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f5403fff700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f540c15c590 0x7f540c06d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:35.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f5403fff700 1 -- 172.21.15.97:0/3835903663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f540c0719c0 msgr2=0x7f540c14a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:35.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f5403fff700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f540c0719c0 0x7f540c14a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:35.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.900+0000 7f5403fff700 1 -- 172.21.15.97:0/3835903663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f4009550 con 0x7f540c15c1c0 2023-07-06T03:47:35.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.901+0000 7f5403fff700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f540c15c1c0 0x7f540c06cb80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009c10 tx=0x7f53f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:35.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.901+0000 7f54037fe700 1 -- 172.21.15.97:0/3835903663 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f40036a0 con 0x7f540c15c1c0 2023-07-06T03:47:35.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.901+0000 7f54037fe700 1 -- 172.21.15.97:0/3835903663 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f4003800 con 0x7f540c15c1c0 2023-07-06T03:47:35.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.901+0000 7f54115ff700 1 -- 172.21.15.97:0/3835903663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540c06de30 con 0x7f540c15c1c0 2023-07-06T03:47:35.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.901+0000 7f54037fe700 1 -- 172.21.15.97:0/3835903663 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4003bb0 con 0x7f540c15c1c0 2023-07-06T03:47:35.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.901+0000 7f54115ff700 1 -- 172.21.15.97:0/3835903663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540c06e3f0 con 0x7f540c15c1c0 2023-07-06T03:47:35.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.902+0000 7f54115ff700 1 -- 172.21.15.97:0/3835903663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540c04e340 con 0x7f540c15c1c0 2023-07-06T03:47:35.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.903+0000 7f54037fe700 1 -- 172.21.15.97:0/3835903663 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f53f4003d10 con 0x7f540c15c1c0 2023-07-06T03:47:35.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.904+0000 7f54037fe700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53ec06ace0 0x7f53ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:35.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.904+0000 7f54037fe700 1 -- 172.21.15.97:0/3835903663 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f53f408a890 con 0x7f540c15c1c0 2023-07-06T03:47:35.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.904+0000 7f540affd700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53ec06ace0 0x7f53ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.905+0000 7f540affd700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53ec06ace0 0x7f53ec06d190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f540c06f310 tx=0x7f53fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:35.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:35.909+0000 7f54037fe700 1 -- 172.21.15.97:0/3835903663 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f53f4055a30 con 0x7f540c15c1c0 2023-07-06T03:47:36.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.147+0000 7f54115ff700 1 -- 172.21.15.97:0/3835903663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f540c0c65a0 con 0x7f540c15c1c0 2023-07-06T03:47:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.663+0000 7f54037fe700 1 -- 172.21.15.97:0/3835903663 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f53f4059050 con 0x7f540c15c1c0 2023-07-06T03:47:36.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.668+0000 7f54115ff700 1 -- 172.21.15.97:0/3835903663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53ec06ace0 msgr2=0x7f53ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:36.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.668+0000 7f54115ff700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53ec06ace0 0x7f53ec06d190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f540c06f310 tx=0x7f53fc008040 comp rx=0 tx=0).stop 2023-07-06T03:47:36.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.668+0000 7f54115ff700 1 -- 172.21.15.97:0/3835903663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f540c15c1c0 msgr2=0x7f540c06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:36.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.668+0000 7f54115ff700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f540c15c1c0 0x7f540c06cb80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009c10 tx=0x7f53f4004750 comp rx=0 tx=0).stop 2023-07-06T03:47:36.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.669+0000 7f54115ff700 1 -- 172.21.15.97:0/3835903663 shutdown_connections 2023-07-06T03:47:36.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.669+0000 7f54115ff700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53ec06ace0 0x7f53ec06d190 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:36.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.669+0000 7f54115ff700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f540c15c590 0x7f540c06d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:36.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.669+0000 7f54115ff700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f540c0719c0 0x7f540c14a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.669+0000 7f54115ff700 1 --2- 172.21.15.97:0/3835903663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f540c15c1c0 0x7f540c06cb80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.669+0000 7f54115ff700 1 -- 172.21.15.97:0/3835903663 >> 172.21.15.97:0/3835903663 conn(0x7f540c0b1650 msgr2=0x7f540c0ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.671+0000 7f54115ff700 1 -- 172.21.15.97:0/3835903663 shutdown_connections 2023-07-06T03:47:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.676+0000 7f54115ff700 1 -- 172.21.15.97:0/3835903663 wait complete. 2023-07-06T03:47:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 7 and data pool 8 2023-07-06T03:47:36.711 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:47:36.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.907+0000 7fd30bbff700 1 Processor -- start 2023-07-06T03:47:36.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.907+0000 7fd30bbff700 1 -- start start 2023-07-06T03:47:36.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.908+0000 7fd30bbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd30415b440 0x7fd30415e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:36.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.908+0000 7fd30bbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd304070c30 0x7fd3040adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:36.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.908+0000 7fd30bbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd304071330 con 0x7fd3040cd8e0 2023-07-06T03:47:36.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.908+0000 7fd30bbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3040ae3e0 con 0x7fd30415b440 2023-07-06T03:47:36.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.908+0000 7fd30bbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd304071610 con 0x7fd304070c30 2023-07-06T03:47:36.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.908+0000 7fd30919a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd30415b440 0x7fd30415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:36.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.909+0000 7fd30919a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd30415b440 0x7fd30415e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40888/0 (socket says 172.21.15.97:40888) 2023-07-06T03:47:36.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.909+0000 7fd30919a700 1 -- 172.21.15.97:0/29474291 learned_addr learned my addr 172.21.15.97:0/29474291 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:36.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.909+0000 7fd30a19c700 1 --2- 172.21.15.97:0/29474291 >> v2:172.21.15.146:3300/0 conn(0x7fd304070c30 0x7fd3040adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:36.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.909+0000 7fd30919a700 1 -- 172.21.15.97:0/29474291 >> v2:172.21.15.146:3300/0 conn(0x7fd304070c30 msgr2=0x7fd3040adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:36.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.909+0000 7fd30919a700 1 --2- 172.21.15.97:0/29474291 >> v2:172.21.15.146:3300/0 conn(0x7fd304070c30 0x7fd3040adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:36.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.909+0000 7fd30919a700 1 -- 172.21.15.97:0/29474291 >> v1:172.21.15.97:6789/0 conn(0x7fd3040cd8e0 legacy=0x7fd3040cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:36.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.909+0000 7fd308999700 1 -- 172.21.15.97:0/29474291 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145164397 0 0) 0x7fd304071330 con 0x7fd3040cd8e0 2023-07-06T03:47:36.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.909+0000 7fd30919a700 1 -- 172.21.15.97:0/29474291 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30415dd80 con 0x7fd30415b440 2023-07-06T03:47:36.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.909+0000 7fd30919a700 1 --2- 172.21.15.97:0/29474291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30415b440 0x7fd30415e860 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd3040ce3c0 tx=0x7fd2f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e831f2ac2da1ea07 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:36.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.909+0000 7fd308999700 1 -- 172.21.15.97:0/29474291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f400b870 con 0x7fd30415b440 2023-07-06T03:47:36.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.910+0000 7fd308999700 1 -- 172.21.15.97:0/29474291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f400b9d0 con 0x7fd30415b440 2023-07-06T03:47:36.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.910+0000 7fd308999700 1 -- 172.21.15.97:0/29474291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f400bd60 con 0x7fd30415b440 2023-07-06T03:47:36.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.910+0000 7fd30bbff700 1 -- 172.21.15.97:0/29474291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30415b440 msgr2=0x7fd30415e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:36.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.910+0000 7fd30bbff700 1 --2- 172.21.15.97:0/29474291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30415b440 0x7fd30415e860 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd3040ce3c0 tx=0x7fd2f4009bd0 comp rx=0 tx=0).stop 2023-07-06T03:47:36.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.910+0000 7fd30bbff700 1 -- 172.21.15.97:0/29474291 shutdown_connections 2023-07-06T03:47:36.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.910+0000 7fd30bbff700 1 --2- 172.21.15.97:0/29474291 >> v2:172.21.15.146:3300/0 conn(0x7fd304070c30 0x7fd3040adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:36.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.910+0000 7fd30bbff700 1 --2- 172.21.15.97:0/29474291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30415b440 0x7fd30415e860 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:36.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.910+0000 7fd30bbff700 1 -- 172.21.15.97:0/29474291 >> 172.21.15.97:0/29474291 conn(0x7fd3040ce8c0 msgr2=0x7fd3040cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:36.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.910+0000 7fd30bbff700 1 -- 172.21.15.97:0/29474291 shutdown_connections 2023-07-06T03:47:36.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.910+0000 7fd30bbff700 1 -- 172.21.15.97:0/29474291 wait complete. 2023-07-06T03:47:36.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.912+0000 7fd30bbff700 1 Processor -- start 2023-07-06T03:47:36.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.912+0000 7fd30bbff700 1 -- start start 2023-07-06T03:47:36.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.913+0000 7fd30bbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd304070c30 0x7fd304151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:36.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.913+0000 7fd30bbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3040cd8e0 0x7fd3040aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:36.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.913+0000 7fd30bbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30415b440 0x7fd3041e6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:36.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.913+0000 7fd30bbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040af980 con 0x7fd30415b440 2023-07-06T03:47:36.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.913+0000 7fd30bbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040afc60 con 0x7fd304070c30 2023-07-06T03:47:36.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.913+0000 7fd30bbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040aff40 con 0x7fd3040cd8e0 2023-07-06T03:47:36.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.913+0000 7fd30a19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30415b440 0x7fd3041e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:36.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.913+0000 7fd30a19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30415b440 0x7fd3041e6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40892/0 (socket says 172.21.15.97:40892) 2023-07-06T03:47:36.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.913+0000 7fd30a19c700 1 -- 172.21.15.97:0/2993912053 learned_addr learned my addr 172.21.15.97:0/2993912053 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:36.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.913+0000 7fd30999b700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd304070c30 0x7fd304151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:36.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.913+0000 7fd30919a700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3040cd8e0 0x7fd3040aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:36.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd30a19c700 1 -- 172.21.15.97:0/2993912053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3040cd8e0 msgr2=0x7fd3040aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:36.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd30a19c700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3040cd8e0 0x7fd3040aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:36.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd30a19c700 1 -- 172.21.15.97:0/2993912053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd304070c30 msgr2=0x7fd304151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:36.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd30a19c700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd304070c30 0x7fd304151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:36.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd30a19c700 1 -- 172.21.15.97:0/2993912053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4009550 con 0x7fd30415b440 2023-07-06T03:47:36.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd30919a700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3040cd8e0 0x7fd3040aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:36.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd30999b700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd304070c30 0x7fd304151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:36.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd30a19c700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30415b440 0x7fd3041e6880 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc007e60 tx=0x7fd2fc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:36.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd2fa7fc700 1 -- 172.21.15.97:0/2993912053 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00ede0 con 0x7fd30415b440 2023-07-06T03:47:36.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd2fa7fc700 1 -- 172.21.15.97:0/2993912053 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc014420 con 0x7fd30415b440 2023-07-06T03:47:36.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd30bbff700 1 -- 172.21.15.97:0/2993912053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3040b0220 con 0x7fd30415b440 2023-07-06T03:47:36.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd2fa7fc700 1 -- 172.21.15.97:0/2993912053 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0147d0 con 0x7fd30415b440 2023-07-06T03:47:36.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.914+0000 7fd30bbff700 1 -- 172.21.15.97:0/2993912053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30415d2b0 con 0x7fd30415b440 2023-07-06T03:47:36.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.915+0000 7fd30bbff700 1 -- 172.21.15.97:0/2993912053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd304070aa0 con 0x7fd30415b440 2023-07-06T03:47:36.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.916+0000 7fd2fa7fc700 1 -- 172.21.15.97:0/2993912053 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd2fc014930 con 0x7fd30415b440 2023-07-06T03:47:36.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.917+0000 7fd2fa7fc700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2e806ace0 0x7fd2e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:36.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.917+0000 7fd2fa7fc700 1 -- 172.21.15.97:0/2993912053 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd2fc089c70 con 0x7fd30415b440 2023-07-06T03:47:36.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.918+0000 7fd30999b700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2e806ace0 0x7fd2e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:36.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.918+0000 7fd30999b700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2e806ace0 0x7fd2e806d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd300000f60 tx=0x7fd300009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:36.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:36.922+0000 7fd2fa7fc700 1 -- 172.21.15.97:0/2993912053 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd2fc054dc0 con 0x7fd30415b440 2023-07-06T03:47:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.116+0000 7fd30bbff700 1 -- 172.21.15.97:0/2993912053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd304067830 con 0x7fd30415b440 2023-07-06T03:47:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.118+0000 7fd2fa7fc700 1 -- 172.21.15.97:0/2993912053 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+18330 (secure 0 0 0) 0x7fd2fc0877b0 con 0x7fd30415b440 2023-07-06T03:47:37.120 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:37.120 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":46,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:47:36.149014+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:47:33.695348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-06T03:47:35.132699+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:47:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 -- 172.21.15.97:0/2993912053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2e806ace0 msgr2=0x7fd2e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2e806ace0 0x7fd2e806d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd300000f60 tx=0x7fd300009040 comp rx=0 tx=0).stop 2023-07-06T03:47:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 -- 172.21.15.97:0/2993912053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30415b440 msgr2=0x7fd3041e6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30415b440 0x7fd3041e6880 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc007e60 tx=0x7fd2fc00e3c0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 -- 172.21.15.97:0/2993912053 shutdown_connections 2023-07-06T03:47:37.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2e806ace0 0x7fd2e806d190 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30415b440 0x7fd3041e6880 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd304070c30 0x7fd304151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 --2- 172.21.15.97:0/2993912053 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3040cd8e0 0x7fd3040aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 -- 172.21.15.97:0/2993912053 >> 172.21.15.97:0/2993912053 conn(0x7fd3040ce8c0 msgr2=0x7fd3040b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 -- 172.21.15.97:0/2993912053 shutdown_connections 2023-07-06T03:47:37.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.120+0000 7fd30bbff700 1 -- 172.21.15.97:0/2993912053 wait complete. 2023-07-06T03:47:37.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:47:37.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-07-06T03:47:37.294 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-07-06T03:47:37.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.333+0000 7f379bfff700 1 Processor -- start 2023-07-06T03:47:37.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.333+0000 7f379bfff700 1 -- start start 2023-07-06T03:47:37.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.334+0000 7f379bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37940cd390 0x7f3794160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:37.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.334+0000 7f379bfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37940cdae0 con 0x7f3794160dd0 2023-07-06T03:47:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.334+0000 7f379bfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37940b6640 con 0x7f379415c520 2023-07-06T03:47:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.334+0000 7f379bfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37940b6920 con 0x7f37940cd390 2023-07-06T03:47:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.335+0000 7f379959a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37940cd390 0x7f3794160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.335+0000 7f379959a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37940cd390 0x7f3794160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40906/0 (socket says 172.21.15.97:40906) 2023-07-06T03:47:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.335+0000 7f379959a700 1 -- 172.21.15.97:0/3165023723 learned_addr learned my addr 172.21.15.97:0/3165023723 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.335+0000 7f379959a700 1 -- 172.21.15.97:0/3165023723 >> v1:172.21.15.146:6789/0 conn(0x7f379415c520 legacy=0x7f3794160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:37.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.335+0000 7f379959a700 1 -- 172.21.15.97:0/3165023723 >> v1:172.21.15.97:6789/0 conn(0x7f3794160dd0 legacy=0x7f37940b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.335+0000 7f379959a700 1 -- 172.21.15.97:0/3165023723 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37940b6c00 con 0x7f37940cd390 2023-07-06T03:47:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.335+0000 7f379959a700 1 --2- 172.21.15.97:0/3165023723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37940cd390 0x7f3794160890 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f37840098f0 tx=0x7f3784009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27285a7ef8302eb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.336+0000 7f3798d99700 1 -- 172.21.15.97:0/3165023723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784004230 con 0x7f37940cd390 2023-07-06T03:47:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.336+0000 7f3798d99700 1 -- 172.21.15.97:0/3165023723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3784004390 con 0x7f37940cd390 2023-07-06T03:47:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.336+0000 7f3798d99700 1 -- 172.21.15.97:0/3165023723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37840036a0 con 0x7f37940cd390 2023-07-06T03:47:37.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.336+0000 7f379bfff700 1 -- 172.21.15.97:0/3165023723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37940cd390 msgr2=0x7f3794160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.336+0000 7f379bfff700 1 --2- 172.21.15.97:0/3165023723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37940cd390 0x7f3794160890 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f37840098f0 tx=0x7f3784009c00 comp rx=0 tx=0).stop 2023-07-06T03:47:37.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.336+0000 7f379bfff700 1 -- 172.21.15.97:0/3165023723 shutdown_connections 2023-07-06T03:47:37.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.336+0000 7f379bfff700 1 --2- 172.21.15.97:0/3165023723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37940cd390 0x7f3794160890 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.336+0000 7f379bfff700 1 -- 172.21.15.97:0/3165023723 >> 172.21.15.97:0/3165023723 conn(0x7f3794071790 msgr2=0x7f3794071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:37.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.336+0000 7f379bfff700 1 -- 172.21.15.97:0/3165023723 shutdown_connections 2023-07-06T03:47:37.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.336+0000 7f379bfff700 1 -- 172.21.15.97:0/3165023723 wait complete. 2023-07-06T03:47:37.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.338+0000 7f379bfff700 1 Processor -- start 2023-07-06T03:47:37.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.338+0000 7f379bfff700 1 -- start start 2023-07-06T03:47:37.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.339+0000 7f379bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37940cd390 0x7f379414cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:37.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.339+0000 7f379bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379415c520 0x7f3794159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:37.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.339+0000 7f379bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3794160dd0 0x7f37940cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:37.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.339+0000 7f379bfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37941f6ba0 con 0x7f379415c520 2023-07-06T03:47:37.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.339+0000 7f379bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37940ca8a0 con 0x7f3794160dd0 2023-07-06T03:47:37.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.339+0000 7f379bfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37940cab80 con 0x7f37940cd390 2023-07-06T03:47:37.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.339+0000 7f379959a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379415c520 0x7f3794159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:37.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.339+0000 7f379959a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379415c520 0x7f3794159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40916/0 (socket says 172.21.15.97:40916) 2023-07-06T03:47:37.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.339+0000 7f379959a700 1 -- 172.21.15.97:0/547995202 learned_addr learned my addr 172.21.15.97:0/547995202 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:37.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.339+0000 7f3799d9b700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37940cd390 0x7f379414cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.339+0000 7f379a59c700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3794160dd0 0x7f37940cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f379959a700 1 -- 172.21.15.97:0/547995202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37940cd390 msgr2=0x7f379414cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f379959a700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37940cd390 0x7f379414cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f379959a700 1 -- 172.21.15.97:0/547995202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3794160dd0 msgr2=0x7f37940cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f379959a700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3794160dd0 0x7f37940cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f379959a700 1 -- 172.21.15.97:0/547995202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784009550 con 0x7f379415c520 2023-07-06T03:47:37.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f3799d9b700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37940cd390 0x7f379414cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:37.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f379959a700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379415c520 0x7f3794159b10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3784000c00 tx=0x7f3784004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:37.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f378a7fc700 1 -- 172.21.15.97:0/547995202 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400b660 con 0x7f379415c520 2023-07-06T03:47:37.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f378a7fc700 1 -- 172.21.15.97:0/547995202 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378400b7e0 con 0x7f379415c520 2023-07-06T03:47:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f378a7fc700 1 -- 172.21.15.97:0/547995202 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400bb90 con 0x7f379415c520 2023-07-06T03:47:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f379bfff700 1 -- 172.21.15.97:0/547995202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37940cae60 con 0x7f379415c520 2023-07-06T03:47:37.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.340+0000 7f379bfff700 1 -- 172.21.15.97:0/547995202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379414d1e0 con 0x7f379415c520 2023-07-06T03:47:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.341+0000 7f379bfff700 1 -- 172.21.15.97:0/547995202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37940cb380 con 0x7f379415c520 2023-07-06T03:47:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.342+0000 7f378a7fc700 1 -- 172.21.15.97:0/547995202 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f378400bcf0 con 0x7f379415c520 2023-07-06T03:47:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.343+0000 7f378a7fc700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f377806ace0 0x7f377806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.343+0000 7f378a7fc700 1 -- 172.21.15.97:0/547995202 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f378408aee0 con 0x7f379415c520 2023-07-06T03:47:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.343+0000 7f3799d9b700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f377806ace0 0x7f377806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.344+0000 7f3799d9b700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f377806ace0 0x7f377806d190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3790003fc0 tx=0x7f3790009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:37.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.347+0000 7f378a7fc700 1 -- 172.21.15.97:0/547995202 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f37940cb380 con 0x7f379415c520 2023-07-06T03:47:37.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.544+0000 7f379bfff700 1 -- 172.21.15.97:0/547995202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37940cb380 con 0x7f379415c520 2023-07-06T03:47:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.546+0000 7f378a7fc700 1 -- 172.21.15.97:0/547995202 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+18330 (secure 0 0 0) 0x7f37940cb380 con 0x7f379415c520 2023-07-06T03:47:37.548 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:37.548 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":46,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:47:36.149014+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:47:33.695348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-06T03:47:35.132699+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:47:37.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.548+0000 7f379bfff700 1 -- 172.21.15.97:0/547995202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f377806ace0 msgr2=0x7f377806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.548+0000 7f379bfff700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f377806ace0 0x7f377806d190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3790003fc0 tx=0x7f3790009040 comp rx=0 tx=0).stop 2023-07-06T03:47:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.548+0000 7f379bfff700 1 -- 172.21.15.97:0/547995202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379415c520 msgr2=0x7f3794159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.548+0000 7f379bfff700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379415c520 0x7f3794159b10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3784000c00 tx=0x7f3784004750 comp rx=0 tx=0).stop 2023-07-06T03:47:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.549+0000 7f379bfff700 1 -- 172.21.15.97:0/547995202 shutdown_connections 2023-07-06T03:47:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.549+0000 7f379bfff700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f377806ace0 0x7f377806d190 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.549+0000 7f379bfff700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3794160dd0 0x7f37940cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.549+0000 7f379bfff700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f37940cd390 0x7f379414cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.549+0000 7f379bfff700 1 --2- 172.21.15.97:0/547995202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379415c520 0x7f3794159b10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.549+0000 7f379bfff700 1 -- 172.21.15.97:0/547995202 >> 172.21.15.97:0/547995202 conn(0x7f3794071790 msgr2=0x7f37940c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:37.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.549+0000 7f379bfff700 1 -- 172.21.15.97:0/547995202 shutdown_connections 2023-07-06T03:47:37.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.549+0000 7f379bfff700 1 -- 172.21.15.97:0/547995202 wait complete. 2023-07-06T03:47:37.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T03:47:37.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.771+0000 7f7dbebff700 1 Processor -- start 2023-07-06T03:47:37.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.771+0000 7f7dbebff700 1 -- start start 2023-07-06T03:47:37.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.772+0000 7f7dbebff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7db806b360 0x7f7db80b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:37.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.772+0000 7f7dbebff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db80cff90 con 0x7f7db815c610 2023-07-06T03:47:37.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.772+0000 7f7dbebff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db80b5650 con 0x7f7db80d2210 2023-07-06T03:47:37.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.772+0000 7f7dbebff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db80d0780 con 0x7f7db806b360 2023-07-06T03:47:37.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.772+0000 7f7dbc99b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7db815c610 0x7f7db80b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34134/0 (socket says 172.21.15.97:34134) 2023-07-06T03:47:37.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.772+0000 7f7dbc99b700 1 -- 172.21.15.97:0/46532199 learned_addr learned my addr 172.21.15.97:0/46532199 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:37.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.772+0000 7f7db7fff700 1 --2- 172.21.15.97:0/46532199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7db806b360 0x7f7db80b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:37.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.772+0000 7f7db77fe700 1 -- 172.21.15.97:0/46532199 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607678705 0 0) 0x7f7db80cff90 con 0x7f7db815c610 2023-07-06T03:47:37.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.773+0000 7f7db77fe700 1 -- 172.21.15.97:0/46532199 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7da80035b0 con 0x7f7db815c610 2023-07-06T03:47:37.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.773+0000 7f7db77fe700 1 -- 172.21.15.97:0/46532199 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427686719 0 0) 0x7f7db80b5650 con 0x7f7db80d2210 2023-07-06T03:47:37.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.773+0000 7f7db7fff700 1 -- 172.21.15.97:0/46532199 >> v1:172.21.15.146:6789/0 conn(0x7f7db80d2210 legacy=0x7f7db80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.773+0000 7f7db7fff700 1 -- 172.21.15.97:0/46532199 >> v1:172.21.15.97:6789/0 conn(0x7f7db815c610 legacy=0x7f7db80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.773+0000 7f7db7fff700 1 -- 172.21.15.97:0/46532199 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db80b5400 con 0x7f7db806b360 2023-07-06T03:47:37.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.773+0000 7f7db7fff700 1 --2- 172.21.15.97:0/46532199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7db806b360 0x7f7db80b4ec0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009870 tx=0x7f7da4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1305684143e9fc99 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:37.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.773+0000 7f7db77fe700 1 -- 172.21.15.97:0/46532199 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4011830 con 0x7f7db806b360 2023-07-06T03:47:37.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.773+0000 7f7db77fe700 1 -- 172.21.15.97:0/46532199 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da4011990 con 0x7f7db806b360 2023-07-06T03:47:37.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.773+0000 7f7db77fe700 1 -- 172.21.15.97:0/46532199 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4011d20 con 0x7f7db806b360 2023-07-06T03:47:37.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.774+0000 7f7dbebff700 1 -- 172.21.15.97:0/46532199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7db806b360 msgr2=0x7f7db80b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.774+0000 7f7dbebff700 1 --2- 172.21.15.97:0/46532199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7db806b360 0x7f7db80b4ec0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009870 tx=0x7f7da4009b80 comp rx=0 tx=0).stop 2023-07-06T03:47:37.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.774+0000 7f7dbebff700 1 -- 172.21.15.97:0/46532199 shutdown_connections 2023-07-06T03:47:37.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.774+0000 7f7dbebff700 1 --2- 172.21.15.97:0/46532199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7db806b360 0x7f7db80b4ec0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.774+0000 7f7dbebff700 1 -- 172.21.15.97:0/46532199 >> 172.21.15.97:0/46532199 conn(0x7f7db806a6c0 msgr2=0x7f7db806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:37.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.774+0000 7f7dbebff700 1 -- 172.21.15.97:0/46532199 shutdown_connections 2023-07-06T03:47:37.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.774+0000 7f7dbebff700 1 -- 172.21.15.97:0/46532199 wait complete. 2023-07-06T03:47:37.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.776+0000 7f7dbebff700 1 Processor -- start 2023-07-06T03:47:37.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.776+0000 7f7dbebff700 1 -- start start 2023-07-06T03:47:37.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7dbebff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7db806b360 0x7f7db80c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:37.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7dbebff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7db80d2210 0x7f7db80c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:37.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7dbebff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db815c610 0x7f7db815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:37.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7dbebff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db815dbb0 con 0x7f7db80d2210 2023-07-06T03:47:37.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7dbebff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db80c9420 con 0x7f7db815c610 2023-07-06T03:47:37.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7dbebff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db80cbb90 con 0x7f7db806b360 2023-07-06T03:47:37.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7db7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7db80d2210 0x7f7db80c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:37.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7db7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7db80d2210 0x7f7db80c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40932/0 (socket says 172.21.15.97:40932) 2023-07-06T03:47:37.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7db7fff700 1 -- 172.21.15.97:0/3568656512 learned_addr learned my addr 172.21.15.97:0/3568656512 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:37.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7dbc99b700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7db806b360 0x7f7db80c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:37.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7dbd19c700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db815c610 0x7f7db815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:37.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7db7fff700 1 -- 172.21.15.97:0/3568656512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7db806b360 msgr2=0x7f7db80c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7db7fff700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7db806b360 0x7f7db80c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7db7fff700 1 -- 172.21.15.97:0/3568656512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db815c610 msgr2=0x7f7db815e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:37.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7db7fff700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db815c610 0x7f7db815e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:37.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.777+0000 7f7db7fff700 1 -- 172.21.15.97:0/3568656512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da40094a0 con 0x7f7db80d2210 2023-07-06T03:47:37.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.778+0000 7f7db7fff700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7db80d2210 0x7f7db80c8ee0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7da40097c0 tx=0x7f7da4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:37.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.778+0000 7f7db57fa700 1 -- 172.21.15.97:0/3568656512 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da401aa70 con 0x7f7db80d2210 2023-07-06T03:47:37.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.778+0000 7f7db57fa700 1 -- 172.21.15.97:0/3568656512 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da401abd0 con 0x7f7db80d2210 2023-07-06T03:47:37.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.778+0000 7f7db57fa700 1 -- 172.21.15.97:0/3568656512 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da40243f0 con 0x7f7db80d2210 2023-07-06T03:47:37.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.778+0000 7f7dbebff700 1 -- 172.21.15.97:0/3568656512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db80cbe70 con 0x7f7db80d2210 2023-07-06T03:47:37.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.778+0000 7f7dbebff700 1 -- 172.21.15.97:0/3568656512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db815e0e0 con 0x7f7db80d2210 2023-07-06T03:47:37.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.779+0000 7f7dbebff700 1 -- 172.21.15.97:0/3568656512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db815e2d0 con 0x7f7db80d2210 2023-07-06T03:47:37.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.780+0000 7f7db57fa700 1 -- 172.21.15.97:0/3568656512 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7da402d450 con 0x7f7db80d2210 2023-07-06T03:47:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.781+0000 7f7db57fa700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d9c06ace0 0x7f7d9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.781+0000 7f7db57fa700 1 -- 172.21.15.97:0/3568656512 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7da408a8c0 con 0x7f7db80d2210 2023-07-06T03:47:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.781+0000 7f7dbc99b700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d9c06ace0 0x7f7d9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:37.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.782+0000 7f7dbc99b700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d9c06ace0 0x7f7d9c06d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7dac007fd0 tx=0x7f7dac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:37.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:37.785+0000 7f7db57fa700 1 -- 172.21.15.97:0/3568656512 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7da4055b20 con 0x7f7db80d2210 2023-07-06T03:47:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.023+0000 7f7dbebff700 1 -- 172.21.15.97:0/3568656512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7db804e340 con 0x7f7db80d2210 2023-07-06T03:47:38.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.684+0000 7f7db57fa700 1 -- 172.21.15.97:0/3568656512 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7da4015020 con 0x7f7db80d2210 2023-07-06T03:47:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 -- 172.21.15.97:0/3568656512 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d9c06ace0 msgr2=0x7f7d9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d9c06ace0 0x7f7d9c06d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7dac007fd0 tx=0x7f7dac009040 comp rx=0 tx=0).stop 2023-07-06T03:47:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 -- 172.21.15.97:0/3568656512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7db80d2210 msgr2=0x7f7db80c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7db80d2210 0x7f7db80c8ee0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7da40097c0 tx=0x7f7da4004750 comp rx=0 tx=0).stop 2023-07-06T03:47:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 -- 172.21.15.97:0/3568656512 shutdown_connections 2023-07-06T03:47:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d9c06ace0 0x7f7d9c06d190 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db815c610 0x7f7db815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7db806b360 0x7f7db80c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 --2- 172.21.15.97:0/3568656512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7db80d2210 0x7f7db80c8ee0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 -- 172.21.15.97:0/3568656512 >> 172.21.15.97:0/3568656512 conn(0x7f7db806a6c0 msgr2=0x7f7db80cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 -- 172.21.15.97:0/3568656512 shutdown_connections 2023-07-06T03:47:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.687+0000 7f7dbebff700 1 -- 172.21.15.97:0/3568656512 wait complete. 2023-07-06T03:47:38.709 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T03:47:38.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T03:47:38.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.906+0000 7fd9751ff700 1 Processor -- start 2023-07-06T03:47:38.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.907+0000 7fd9751ff700 1 -- start start 2023-07-06T03:47:38.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.908+0000 7fd9751ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd9700ccbd0 0x7fd9701640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:38.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.908+0000 7fd9751ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9700ca260 con 0x7fd9700c3cd0 2023-07-06T03:47:38.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.908+0000 7fd9751ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9700c99c0 con 0x7fd970159070 2023-07-06T03:47:38.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.908+0000 7fd9751ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9700c9ca0 con 0x7fd9700ccbd0 2023-07-06T03:47:38.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.908+0000 7fd967fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd9700c3cd0 0x7fd9700cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34146/0 (socket says 172.21.15.97:34146) 2023-07-06T03:47:38.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.908+0000 7fd967fff700 1 -- 172.21.15.97:0/2626842680 learned_addr learned my addr 172.21.15.97:0/2626842680 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:38.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96f59e700 1 --2- 172.21.15.97:0/2626842680 >> v2:172.21.15.146:3300/0 conn(0x7fd9700ccbd0 0x7fd9701640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:38.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441524942 0 0) 0x7fd9700ca260 con 0x7fd9700c3cd0 2023-07-06T03:47:38.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd95c0057c0 con 0x7fd9700c3cd0 2023-07-06T03:47:38.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922296707 0 0) 0x7fd9700c99c0 con 0x7fd970159070 2023-07-06T03:47:38.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9700ca260 con 0x7fd970159070 2023-07-06T03:47:38.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4056573011 0 0) 0x7fd95c0057c0 con 0x7fd9700c3cd0 2023-07-06T03:47:38.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 >> v2:172.21.15.146:3300/0 conn(0x7fd9700ccbd0 msgr2=0x7fd9701640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:38.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 --2- 172.21.15.97:0/2626842680 >> v2:172.21.15.146:3300/0 conn(0x7fd9700ccbd0 0x7fd9701640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:38.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 >> v1:172.21.15.146:6789/0 conn(0x7fd970159070 legacy=0x7fd9700c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:38.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9700c9f80 con 0x7fd9700c3cd0 2023-07-06T03:47:38.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fd958001670 con 0x7fd9700c3cd0 2023-07-06T03:47:38.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 >> v1:172.21.15.97:6789/0 conn(0x7fd9700c3cd0 legacy=0x7fd9700cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:38.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 --2- 172.21.15.97:0/2626842680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd95c005df0 0x7fd95c0061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:38.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 --2- 172.21.15.97:0/2626842680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 0x7fd95c006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:38.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 --2- 172.21.15.97:0/2626842680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd95c007460 0x7fd95c00dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:38.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9700c9ca0 con 0x7fd95c006a60 2023-07-06T03:47:38.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c00e370 con 0x7fd95c005df0 2023-07-06T03:47:38.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c00e530 con 0x7fd95c007460 2023-07-06T03:47:38.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd96f59e700 1 --2- 172.21.15.97:0/2626842680 >> v2:172.21.15.146:3300/0 conn(0x7fd9700ccbd0 0x7fd9701640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:47:38.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.909+0000 7fd967fff700 1 --2- 172.21.15.97:0/2626842680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 0x7fd95c006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:38.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd96ed9d700 1 --2- 172.21.15.97:0/2626842680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd95c005df0 0x7fd95c0061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd96f59e700 1 --2- 172.21.15.97:0/2626842680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd95c007460 0x7fd95c00dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd967fff700 1 -- 172.21.15.97:0/2626842680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd95c007460 msgr2=0x7fd95c00dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd967fff700 1 --2- 172.21.15.97:0/2626842680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd95c007460 0x7fd95c00dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd967fff700 1 -- 172.21.15.97:0/2626842680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd95c005df0 msgr2=0x7fd95c0061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd967fff700 1 --2- 172.21.15.97:0/2626842680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd95c005df0 0x7fd95c0061e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd967fff700 1 -- 172.21.15.97:0/2626842680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c0057c0 con 0x7fd95c006a60 2023-07-06T03:47:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd96f59e700 1 -- 172.21.15.97:0/2626842680 reap_dead start 2023-07-06T03:47:38.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd967fff700 1 --2- 172.21.15.97:0/2626842680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 0x7fd95c006ed0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd958001eb0 tx=0x7fd95c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29929e528b66a94a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95800cc40 con 0x7fd95c006a60 2023-07-06T03:47:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95800cda0 con 0x7fd95c006a60 2023-07-06T03:47:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.910+0000 7fd96e59c700 1 -- 172.21.15.97:0/2626842680 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd958005ba0 con 0x7fd95c006a60 2023-07-06T03:47:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.911+0000 7fd9751ff700 1 -- 172.21.15.97:0/2626842680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 msgr2=0x7fd95c006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.911+0000 7fd9751ff700 1 --2- 172.21.15.97:0/2626842680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 0x7fd95c006ed0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd958001eb0 tx=0x7fd95c007a90 comp rx=0 tx=0).stop 2023-07-06T03:47:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.911+0000 7fd9751ff700 1 -- 172.21.15.97:0/2626842680 shutdown_connections 2023-07-06T03:47:38.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.911+0000 7fd9751ff700 1 --2- 172.21.15.97:0/2626842680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 0x7fd95c006ed0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.911+0000 7fd9751ff700 1 -- 172.21.15.97:0/2626842680 >> 172.21.15.97:0/2626842680 conn(0x7fd97006a060 msgr2=0x7fd97006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.911+0000 7fd9751ff700 1 -- 172.21.15.97:0/2626842680 shutdown_connections 2023-07-06T03:47:38.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.911+0000 7fd9751ff700 1 -- 172.21.15.97:0/2626842680 wait complete. 2023-07-06T03:47:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.913+0000 7fd9751ff700 1 Processor -- start 2023-07-06T03:47:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.913+0000 7fd9751ff700 1 -- start start 2023-07-06T03:47:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.914+0000 7fd9751ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 0x7fd97014bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.914+0000 7fd9751ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd970159440 0x7fd9700cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.914+0000 7fd9751ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9700cb710 0x7fd9700c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:38.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.914+0000 7fd9751ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9700cbb90 con 0x7fd95c006a60 2023-07-06T03:47:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.914+0000 7fd9751ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97014d6d0 con 0x7fd9700cb710 2023-07-06T03:47:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.914+0000 7fd9751ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97014d9b0 con 0x7fd970159440 2023-07-06T03:47:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.914+0000 7fd96ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 0x7fd97014bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.914+0000 7fd96ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 0x7fd97014bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40956/0 (socket says 172.21.15.97:40956) 2023-07-06T03:47:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.914+0000 7fd96ed9d700 1 -- 172.21.15.97:0/521805750 learned_addr learned my addr 172.21.15.97:0/521805750 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd967fff700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd970159440 0x7fd9700cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd96f59e700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9700cb710 0x7fd9700c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:38.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd96ed9d700 1 -- 172.21.15.97:0/521805750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd970159440 msgr2=0x7fd9700cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd96ed9d700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd970159440 0x7fd9700cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd96ed9d700 1 -- 172.21.15.97:0/521805750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9700cb710 msgr2=0x7fd9700c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd96ed9d700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9700cb710 0x7fd9700c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd96ed9d700 1 -- 172.21.15.97:0/521805750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97014dc90 con 0x7fd95c006a60 2023-07-06T03:47:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd96ed9d700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 0x7fd97014bb50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd96000f770 tx=0x7fd96000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd9677fe700 1 -- 172.21.15.97:0/521805750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960005320 con 0x7fd95c006a60 2023-07-06T03:47:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd9677fe700 1 -- 172.21.15.97:0/521805750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9600054a0 con 0x7fd95c006a60 2023-07-06T03:47:38.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd9751ff700 1 -- 172.21.15.97:0/521805750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97014df70 con 0x7fd95c006a60 2023-07-06T03:47:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd9677fe700 1 -- 172.21.15.97:0/521805750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960006aa0 con 0x7fd95c006a60 2023-07-06T03:47:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.915+0000 7fd9751ff700 1 -- 172.21.15.97:0/521805750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9700c8980 con 0x7fd95c006a60 2023-07-06T03:47:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.916+0000 7fd9751ff700 1 -- 172.21.15.97:0/521805750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97014d810 con 0x7fd95c006a60 2023-07-06T03:47:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.917+0000 7fd9677fe700 1 -- 172.21.15.97:0/521805750 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd960006c00 con 0x7fd95c006a60 2023-07-06T03:47:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.918+0000 7fd9677fe700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd95006ace0 0x7fd95006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.918+0000 7fd9677fe700 1 -- 172.21.15.97:0/521805750 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd960088a10 con 0x7fd95c006a60 2023-07-06T03:47:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.918+0000 7fd967fff700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd95006ace0 0x7fd95006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.919+0000 7fd967fff700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd95006ace0 0x7fd95006d190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd958015040 tx=0x7fd958007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:38.923+0000 7fd9677fe700 1 -- 172.21.15.97:0/521805750 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd97014d810 con 0x7fd95c006a60 2023-07-06T03:47:39.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.170+0000 7fd9751ff700 1 -- 172.21.15.97:0/521805750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd97014d810 con 0x7fd95c006a60 2023-07-06T03:47:39.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.690+0000 7fd9677fe700 1 -- 172.21.15.97:0/521805750 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd97014d810 con 0x7fd95c006a60 2023-07-06T03:47:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.692+0000 7fd9751ff700 1 -- 172.21.15.97:0/521805750 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd95006ace0 msgr2=0x7fd95006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.692+0000 7fd9751ff700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd95006ace0 0x7fd95006d190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd958015040 tx=0x7fd958007000 comp rx=0 tx=0).stop 2023-07-06T03:47:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.692+0000 7fd9751ff700 1 -- 172.21.15.97:0/521805750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 msgr2=0x7fd97014bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.692+0000 7fd9751ff700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 0x7fd97014bb50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd96000f770 tx=0x7fd96000fb30 comp rx=0 tx=0).stop 2023-07-06T03:47:39.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.692+0000 7fd9751ff700 1 -- 172.21.15.97:0/521805750 shutdown_connections 2023-07-06T03:47:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.692+0000 7fd9751ff700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd95006ace0 0x7fd95006d190 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.692+0000 7fd9751ff700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9700cb710 0x7fd9700c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.692+0000 7fd9751ff700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd95c006a60 0x7fd97014bb50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.692+0000 7fd9751ff700 1 --2- 172.21.15.97:0/521805750 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd970159440 0x7fd9700cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:39.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.692+0000 7fd9751ff700 1 -- 172.21.15.97:0/521805750 >> 172.21.15.97:0/521805750 conn(0x7fd97006a060 msgr2=0x7fd9700710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:39.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.692+0000 7fd9751ff700 1 -- 172.21.15.97:0/521805750 shutdown_connections 2023-07-06T03:47:39.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.693+0000 7fd9751ff700 1 -- 172.21.15.97:0/521805750 wait complete. 2023-07-06T03:47:39.712 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T03:47:39.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:47:39.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.906+0000 7f78207ff700 1 Processor -- start 2023-07-06T03:47:39.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.906+0000 7f78207ff700 1 -- start start 2023-07-06T03:47:39.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.907+0000 7f78207ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781806caa0 0x7f78180d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:39.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.907+0000 7f78207ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78180cfd00 con 0x7f781815ccf0 2023-07-06T03:47:39.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.907+0000 7f78207ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78180b28f0 con 0x7f781815c920 2023-07-06T03:47:39.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.907+0000 7f78207ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78181610b0 con 0x7f781806caa0 2023-07-06T03:47:39.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781dd9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f781815ccf0 0x7f78180d0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46568/0 (socket says 172.21.15.97:46568) 2023-07-06T03:47:39.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781dd9a700 1 -- 172.21.15.97:0/3791804657 learned_addr learned my addr 172.21.15.97:0/3791804657 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781ed9c700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781806caa0 0x7f78180d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203881099 0 0) 0x7f78180cfd00 con 0x7f781815ccf0 2023-07-06T03:47:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78040035b0 con 0x7f781815ccf0 2023-07-06T03:47:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432583554 0 0) 0x7f78180b28f0 con 0x7f781815c920 2023-07-06T03:47:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78180cfd00 con 0x7f781815c920 2023-07-06T03:47:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1987036320 0 0) 0x7f78040035b0 con 0x7f781815ccf0 2023-07-06T03:47:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781806caa0 msgr2=0x7f78180d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:39.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781806caa0 0x7f78180d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:39.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 >> v1:172.21.15.146:6789/0 conn(0x7f781815c920 legacy=0x7f781806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:39.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7818161390 con 0x7f781815ccf0 2023-07-06T03:47:39.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f7808001670 con 0x7f781815ccf0 2023-07-06T03:47:39.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 >> v1:172.21.15.97:6789/0 conn(0x7f781815ccf0 legacy=0x7f78180d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:39.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7804003bc0 0x7f7804003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:39.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781806caa0 0x7f7804004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:39.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7804004d90 0x7f7804005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:39.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78181610b0 con 0x7f7804004d90 2023-07-06T03:47:39.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780400bac0 con 0x7f7804003bc0 2023-07-06T03:47:39.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.908+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780400bc30 con 0x7f781806caa0 2023-07-06T03:47:39.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.909+0000 7f781ed9c700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7804004d90 0x7f7804005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:39.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.909+0000 7f781dd9a700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781806caa0 0x7f7804004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:39.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.909+0000 7f781e59b700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7804003bc0 0x7f7804003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:39.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.909+0000 7f781ed9c700 1 -- 172.21.15.97:0/3791804657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781806caa0 msgr2=0x7f7804004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:39.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.909+0000 7f781ed9c700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781806caa0 0x7f7804004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:39.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.909+0000 7f781ed9c700 1 -- 172.21.15.97:0/3791804657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7804003bc0 msgr2=0x7f7804003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:39.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.909+0000 7f781ed9c700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7804003bc0 0x7f7804003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:39.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.909+0000 7f781ed9c700 1 -- 172.21.15.97:0/3791804657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78040035b0 con 0x7f7804004d90 2023-07-06T03:47:39.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.909+0000 7f781ed9c700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7804004d90 0x7f7804005200 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7810009f90 tx=0x7f7804005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4e5c9ad20debc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:39.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.910+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781000ebd0 con 0x7f7804004d90 2023-07-06T03:47:39.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.910+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781000f040 con 0x7f7804004d90 2023-07-06T03:47:39.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.910+0000 7f781d599700 1 -- 172.21.15.97:0/3791804657 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810014480 con 0x7f7804004d90 2023-07-06T03:47:39.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.910+0000 7f78207ff700 1 -- 172.21.15.97:0/3791804657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7804004d90 msgr2=0x7f7804005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:39.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.910+0000 7f78207ff700 1 --2- 172.21.15.97:0/3791804657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7804004d90 0x7f7804005200 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7810009f90 tx=0x7f7804005880 comp rx=0 tx=0).stop 2023-07-06T03:47:39.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.910+0000 7f781ed9c700 1 -- 172.21.15.97:0/3791804657 reap_dead start 2023-07-06T03:47:39.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.910+0000 7f78207ff700 1 -- 172.21.15.97:0/3791804657 shutdown_connections 2023-07-06T03:47:39.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.910+0000 7f78207ff700 1 -- 172.21.15.97:0/3791804657 >> 172.21.15.97:0/3791804657 conn(0x7f781806ace0 msgr2=0x7f781806dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:39.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.910+0000 7f78207ff700 1 -- 172.21.15.97:0/3791804657 shutdown_connections 2023-07-06T03:47:39.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.910+0000 7f78207ff700 1 -- 172.21.15.97:0/3791804657 wait complete. 2023-07-06T03:47:39.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.912+0000 7f78207ff700 1 Processor -- start 2023-07-06T03:47:39.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.912+0000 7f78207ff700 1 -- start start 2023-07-06T03:47:39.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.913+0000 7f78207ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f781814afa0 0x7f781815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:39.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.913+0000 7f78207ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781815e860 0x7f78180b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:39.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.913+0000 7f78207ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f781815f350 0x7f781815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:39.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.913+0000 7f78207ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78180b4a70 con 0x7f781815f350 2023-07-06T03:47:39.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.913+0000 7f78207ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781815fd00 con 0x7f781814afa0 2023-07-06T03:47:39.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.913+0000 7f78207ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781815e3d0 con 0x7f781815e860 2023-07-06T03:47:39.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.913+0000 7f781ed9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f781815f350 0x7f781815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.913+0000 7f781ed9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f781815f350 0x7f781815f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33104/0 (socket says 172.21.15.97:33104) 2023-07-06T03:47:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.913+0000 7f781ed9c700 1 -- 172.21.15.97:0/566667677 learned_addr learned my addr 172.21.15.97:0/566667677 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.913+0000 7f781e59b700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f781814afa0 0x7f781815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.913+0000 7f781dd9a700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781815e860 0x7f78180b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.914+0000 7f781ed9c700 1 -- 172.21.15.97:0/566667677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781815e860 msgr2=0x7f78180b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.914+0000 7f781ed9c700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781815e860 0x7f78180b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.914+0000 7f781ed9c700 1 -- 172.21.15.97:0/566667677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f781814afa0 msgr2=0x7f781815d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.914+0000 7f781ed9c700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f781814afa0 0x7f781815d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.914+0000 7f781ed9c700 1 -- 172.21.15.97:0/566667677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781815d7c0 con 0x7f781815f350 2023-07-06T03:47:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.914+0000 7f781ed9c700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f781815f350 0x7f781815f7c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f781000f010 tx=0x7f7810009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.914+0000 7f780effd700 1 -- 172.21.15.97:0/566667677 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810007000 con 0x7f781815f350 2023-07-06T03:47:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.914+0000 7f780effd700 1 -- 172.21.15.97:0/566667677 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7810003c30 con 0x7f781815f350 2023-07-06T03:47:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.914+0000 7f78207ff700 1 -- 172.21.15.97:0/566667677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781815daa0 con 0x7f781815f350 2023-07-06T03:47:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.914+0000 7f780effd700 1 -- 172.21.15.97:0/566667677 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810003de0 con 0x7f781815f350 2023-07-06T03:47:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.914+0000 7f78207ff700 1 -- 172.21.15.97:0/566667677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781815e060 con 0x7f781815f350 2023-07-06T03:47:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.915+0000 7f78207ff700 1 -- 172.21.15.97:0/566667677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7818066a40 con 0x7f781815f350 2023-07-06T03:47:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.916+0000 7f780effd700 1 -- 172.21.15.97:0/566667677 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7810014940 con 0x7f781815f350 2023-07-06T03:47:39.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.917+0000 7f780effd700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f77fc06ace0 0x7f77fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:39.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.917+0000 7f780effd700 1 -- 172.21.15.97:0/566667677 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f781008a6d0 con 0x7f781815f350 2023-07-06T03:47:39.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.917+0000 7f781e59b700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f77fc06ace0 0x7f77fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:39.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.918+0000 7f781e59b700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f77fc06ace0 0x7f77fc06d190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7814007fd0 tx=0x7f7814009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:39.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:39.921+0000 7f780effd700 1 -- 172.21.15.97:0/566667677 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7810055770 con 0x7f781815f350 2023-07-06T03:47:40.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.156+0000 7f78207ff700 1 -- 172.21.15.97:0/566667677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f781804e340 con 0x7f781815f350 2023-07-06T03:47:40.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.157+0000 7f780effd700 1 -- 172.21.15.97:0/566667677 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f7810058d90 con 0x7f781815f350 2023-07-06T03:47:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:40.159 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1299982704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1299982704},{"type":"v1","addr":"172.21.15.146:6833","nonce":1299982704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5602,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/640394657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":640394657},{"type":"v1","addr":"172.21.15.97:6837","nonce":640394657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2363940464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2363940464},{"type":"v1","addr":"172.21.15.97:6836","nonce":2363940464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:47:36.149445+0000","modified":"2023-07-06T03:47:39.683749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5838},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5838":{"gid":5838,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/3623506860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3623506860},{"type":"v1","addr":"172.21.15.146:6835","nonce":3623506860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T03:47:40.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 -- 172.21.15.97:0/566667677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f77fc06ace0 msgr2=0x7f77fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f77fc06ace0 0x7f77fc06d190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7814007fd0 tx=0x7f7814009040 comp rx=0 tx=0).stop 2023-07-06T03:47:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 -- 172.21.15.97:0/566667677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f781815f350 msgr2=0x7f781815f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f781815f350 0x7f781815f7c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f781000f010 tx=0x7f7810009750 comp rx=0 tx=0).stop 2023-07-06T03:47:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 -- 172.21.15.97:0/566667677 shutdown_connections 2023-07-06T03:47:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f77fc06ace0 0x7f77fc06d190 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f781815f350 0x7f781815f7c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f781814afa0 0x7f781815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 --2- 172.21.15.97:0/566667677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f781815e860 0x7f78180b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 -- 172.21.15.97:0/566667677 >> 172.21.15.97:0/566667677 conn(0x7f7804004d90 msgr2=0x7f781806b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:40.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 -- 172.21.15.97:0/566667677 shutdown_connections 2023-07-06T03:47:40.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.160+0000 7f78207ff700 1 -- 172.21.15.97:0/566667677 wait complete. 2023-07-06T03:47:40.163 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2023-07-06T03:47:40.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:47:40.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.371+0000 7f5c15dff700 1 Processor -- start 2023-07-06T03:47:40.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.371+0000 7f5c15dff700 1 -- start start 2023-07-06T03:47:40.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.372+0000 7f5c15dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c1015c520 0x7f5c100cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c15dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5c10160180 0x7f5c101605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c15dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c100cdae0 con 0x7f5c10160b30 2023-07-06T03:47:40.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c15dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c100cd8d0 con 0x7f5c10160180 2023-07-06T03:47:40.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c15dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c100b6640 con 0x7f5c1015c520 2023-07-06T03:47:40.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c0effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5c10160180 0x7f5c101605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c0effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5c10160180 0x7f5c101605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33114/0 (socket says 172.21.15.97:33114) 2023-07-06T03:47:40.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c0effd700 1 -- 172.21.15.97:0/328028234 learned_addr learned my addr 172.21.15.97:0/328028234 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:40.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c0f7fe700 1 --2- 172.21.15.97:0/328028234 >> v2:172.21.15.146:3300/0 conn(0x7f5c1015c520 0x7f5c100cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c0effd700 1 -- 172.21.15.97:0/328028234 >> v2:172.21.15.146:3300/0 conn(0x7f5c1015c520 msgr2=0x7f5c100cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:40.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c0effd700 1 --2- 172.21.15.97:0/328028234 >> v2:172.21.15.146:3300/0 conn(0x7f5c1015c520 0x7f5c100cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c0effd700 1 -- 172.21.15.97:0/328028234 >> v1:172.21.15.146:6789/0 conn(0x7f5c10160b30 legacy=0x7f5c100cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:40.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.373+0000 7f5c0effd700 1 -- 172.21.15.97:0/328028234 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c100b6920 con 0x7f5c10160180 2023-07-06T03:47:40.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c0effd700 1 --2- 172.21.15.97:0/328028234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c10160180 0x7f5c101605f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5c100b4c70 tx=0x7f5c04009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaf77126aa9a025 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:40.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c0e7fc700 1 -- 172.21.15.97:0/328028234 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400b870 con 0x7f5c10160180 2023-07-06T03:47:40.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c0e7fc700 1 -- 172.21.15.97:0/328028234 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0400b9d0 con 0x7f5c10160180 2023-07-06T03:47:40.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c0e7fc700 1 -- 172.21.15.97:0/328028234 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400bd60 con 0x7f5c10160180 2023-07-06T03:47:40.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c15dff700 1 -- 172.21.15.97:0/328028234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c10160180 msgr2=0x7f5c101605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:40.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c15dff700 1 --2- 172.21.15.97:0/328028234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c10160180 0x7f5c101605f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5c100b4c70 tx=0x7f5c04009bd0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c15dff700 1 -- 172.21.15.97:0/328028234 shutdown_connections 2023-07-06T03:47:40.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c15dff700 1 --2- 172.21.15.97:0/328028234 >> v2:172.21.15.146:3300/0 conn(0x7f5c1015c520 0x7f5c100cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c15dff700 1 --2- 172.21.15.97:0/328028234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c10160180 0x7f5c101605f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c15dff700 1 -- 172.21.15.97:0/328028234 >> 172.21.15.97:0/328028234 conn(0x7f5c10071790 msgr2=0x7f5c10071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:40.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c15dff700 1 -- 172.21.15.97:0/328028234 shutdown_connections 2023-07-06T03:47:40.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.374+0000 7f5c15dff700 1 -- 172.21.15.97:0/328028234 wait complete. 2023-07-06T03:47:40.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.377+0000 7f5c15dff700 1 Processor -- start 2023-07-06T03:47:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.377+0000 7f5c15dff700 1 -- start start 2023-07-06T03:47:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.378+0000 7f5c15dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c1015c520 0x7f5c10155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.378+0000 7f5c15dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c10160180 0x7f5c1015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.378+0000 7f5c15dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c10160b30 0x7f5c1015cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.378+0000 7f5c15dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1015f700 con 0x7f5c10160180 2023-07-06T03:47:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.378+0000 7f5c15dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100ca120 con 0x7f5c1015c520 2023-07-06T03:47:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.378+0000 7f5c15dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10156520 con 0x7f5c10160b30 2023-07-06T03:47:40.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.378+0000 7f5c0effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c10160180 0x7f5c1015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.378+0000 7f5c0effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c10160180 0x7f5c1015ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33126/0 (socket says 172.21.15.97:33126) 2023-07-06T03:47:40.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.378+0000 7f5c0effd700 1 -- 172.21.15.97:0/3196755680 learned_addr learned my addr 172.21.15.97:0/3196755680 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:40.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.378+0000 7f5c0f7fe700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c1015c520 0x7f5c10155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.379+0000 7f5c0ffff700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c10160b30 0x7f5c1015cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.379+0000 7f5c0effd700 1 -- 172.21.15.97:0/3196755680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c10160b30 msgr2=0x7f5c1015cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:40.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.379+0000 7f5c0effd700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c10160b30 0x7f5c1015cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.379+0000 7f5c0effd700 1 -- 172.21.15.97:0/3196755680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c1015c520 msgr2=0x7f5c10155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:40.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.379+0000 7f5c0effd700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c1015c520 0x7f5c10155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.379+0000 7f5c0effd700 1 -- 172.21.15.97:0/3196755680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c04009550 con 0x7f5c10160180 2023-07-06T03:47:40.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.379+0000 7f5c0effd700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c10160180 0x7f5c1015ca10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5c04000c00 tx=0x7f5c04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.379+0000 7f5bf7fff700 1 -- 172.21.15.97:0/3196755680 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04004480 con 0x7f5c10160180 2023-07-06T03:47:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.379+0000 7f5bf7fff700 1 -- 172.21.15.97:0/3196755680 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c040045e0 con 0x7f5c10160180 2023-07-06T03:47:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.379+0000 7f5c15dff700 1 -- 172.21.15.97:0/3196755680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1015eaf0 con 0x7f5c10160180 2023-07-06T03:47:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.379+0000 7f5bf7fff700 1 -- 172.21.15.97:0/3196755680 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040038e0 con 0x7f5c10160180 2023-07-06T03:47:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.380+0000 7f5c15dff700 1 -- 172.21.15.97:0/3196755680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1015f040 con 0x7f5c10160180 2023-07-06T03:47:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.380+0000 7f5c15dff700 1 -- 172.21.15.97:0/3196755680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1015f840 con 0x7f5c10160180 2023-07-06T03:47:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.382+0000 7f5bf7fff700 1 -- 172.21.15.97:0/3196755680 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c04021070 con 0x7f5c10160180 2023-07-06T03:47:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.382+0000 7f5bf7fff700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5bfc06ace0 0x7f5bfc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.382+0000 7f5bf7fff700 1 -- 172.21.15.97:0/3196755680 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c0408aab0 con 0x7f5c10160180 2023-07-06T03:47:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.383+0000 7f5c0f7fe700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5bfc06ace0 0x7f5bfc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.383+0000 7f5c0f7fe700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5bfc06ace0 0x7f5bfc06d190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5c00006b20 tx=0x7f5c00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:40.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.387+0000 7f5bf7fff700 1 -- 172.21.15.97:0/3196755680 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5c1015f840 con 0x7f5c10160180 2023-07-06T03:47:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.582+0000 7f5c15dff700 1 -- 172.21.15.97:0/3196755680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c1015f840 con 0x7f5c10160180 2023-07-06T03:47:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.584+0000 7f5bf7fff700 1 -- 172.21.15.97:0/3196755680 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+18317 (secure 0 0 0) 0x7f5c1015f840 con 0x7f5c10160180 2023-07-06T03:47:40.586 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:40.586 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":47,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:47:37.668975+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:47:33.695348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-06T03:47:35.132699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:47:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 -- 172.21.15.97:0/3196755680 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5bfc06ace0 msgr2=0x7f5bfc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5bfc06ace0 0x7f5bfc06d190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5c00006b20 tx=0x7f5c00008040 comp rx=0 tx=0).stop 2023-07-06T03:47:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 -- 172.21.15.97:0/3196755680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c10160180 msgr2=0x7f5c1015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c10160180 0x7f5c1015ca10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5c04000c00 tx=0x7f5c04004750 comp rx=0 tx=0).stop 2023-07-06T03:47:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 -- 172.21.15.97:0/3196755680 shutdown_connections 2023-07-06T03:47:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5bfc06ace0 0x7f5bfc06d190 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c10160b30 0x7f5c1015cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c1015c520 0x7f5c10155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 --2- 172.21.15.97:0/3196755680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c10160180 0x7f5c1015ca10 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 -- 172.21.15.97:0/3196755680 >> 172.21.15.97:0/3196755680 conn(0x7f5c10071790 msgr2=0x7f5c100b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 -- 172.21.15.97:0/3196755680 shutdown_connections 2023-07-06T03:47:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.587+0000 7f5c15dff700 1 -- 172.21.15.97:0/3196755680 wait complete. 2023-07-06T03:47:40.615 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T03:47:40.616 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:47:40.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.810+0000 7f869e9ff700 1 Processor -- start 2023-07-06T03:47:40.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.810+0000 7f869e9ff700 1 -- start start 2023-07-06T03:47:40.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.811+0000 7f869e9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86980cd390 0x7f8698160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.811+0000 7f869e9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8698160dd0 0x7f86980b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.811+0000 7f869e9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86980cdae0 con 0x7f869815c520 2023-07-06T03:47:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.811+0000 7f869e9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86980b6640 con 0x7f86980cd390 2023-07-06T03:47:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.811+0000 7f869e9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86980b6920 con 0x7f8698160dd0 2023-07-06T03:47:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.811+0000 7f868ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86980cd390 0x7f8698160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.811+0000 7f868ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86980cd390 0x7f8698160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33142/0 (socket says 172.21.15.97:33142) 2023-07-06T03:47:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.811+0000 7f868ffff700 1 -- 172.21.15.97:0/1371432002 learned_addr learned my addr 172.21.15.97:0/1371432002 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.811+0000 7f869cf9c700 1 --2- 172.21.15.97:0/1371432002 >> v2:172.21.15.146:3300/0 conn(0x7f8698160dd0 0x7f86980b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.812+0000 7f868ffff700 1 -- 172.21.15.97:0/1371432002 >> v2:172.21.15.146:3300/0 conn(0x7f8698160dd0 msgr2=0x7f86980b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:40.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.812+0000 7f868ffff700 1 --2- 172.21.15.97:0/1371432002 >> v2:172.21.15.146:3300/0 conn(0x7f8698160dd0 0x7f86980b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.812+0000 7f868ffff700 1 -- 172.21.15.97:0/1371432002 >> v1:172.21.15.146:6789/0 conn(0x7f869815c520 legacy=0x7f8698160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:40.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.812+0000 7f868ffff700 1 -- 172.21.15.97:0/1371432002 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86980b6c00 con 0x7f86980cd390 2023-07-06T03:47:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.812+0000 7f868ffff700 1 --2- 172.21.15.97:0/1371432002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd390 0x7f8698160890 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f86980cd320 tx=0x7f8680009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5349477812c26cb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.812+0000 7f86977fe700 1 -- 172.21.15.97:0/1371432002 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868000b870 con 0x7f86980cd390 2023-07-06T03:47:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.812+0000 7f86977fe700 1 -- 172.21.15.97:0/1371432002 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868000b9d0 con 0x7f86980cd390 2023-07-06T03:47:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.812+0000 7f86977fe700 1 -- 172.21.15.97:0/1371432002 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868000bd60 con 0x7f86980cd390 2023-07-06T03:47:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.813+0000 7f869e9ff700 1 -- 172.21.15.97:0/1371432002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd390 msgr2=0x7f8698160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.813+0000 7f869e9ff700 1 --2- 172.21.15.97:0/1371432002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd390 0x7f8698160890 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f86980cd320 tx=0x7f8680009bd0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.813+0000 7f869e9ff700 1 -- 172.21.15.97:0/1371432002 shutdown_connections 2023-07-06T03:47:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.813+0000 7f869e9ff700 1 --2- 172.21.15.97:0/1371432002 >> v2:172.21.15.146:3300/0 conn(0x7f8698160dd0 0x7f86980b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.813+0000 7f869e9ff700 1 --2- 172.21.15.97:0/1371432002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd390 0x7f8698160890 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.813+0000 7f869e9ff700 1 -- 172.21.15.97:0/1371432002 >> 172.21.15.97:0/1371432002 conn(0x7f8698071790 msgr2=0x7f8698071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:40.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.813+0000 7f869e9ff700 1 -- 172.21.15.97:0/1371432002 shutdown_connections 2023-07-06T03:47:40.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.813+0000 7f869e9ff700 1 -- 172.21.15.97:0/1371432002 wait complete. 2023-07-06T03:47:40.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.815+0000 7f869e9ff700 1 Processor -- start 2023-07-06T03:47:40.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.815+0000 7f869e9ff700 1 -- start start 2023-07-06T03:47:40.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.816+0000 7f869e9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86980cd390 0x7f86980d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869e9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f869815c520 0x7f8698158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869e9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8698160dd0 0x7f8698158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869e9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980d2ab0 con 0x7f8698160dd0 2023-07-06T03:47:40.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869e9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980c9380 con 0x7f86980cd390 2023-07-06T03:47:40.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869e9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86980d1e30 con 0x7f869815c520 2023-07-06T03:47:40.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869cf9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8698160dd0 0x7f8698158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869cf9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8698160dd0 0x7f8698158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33158/0 (socket says 172.21.15.97:33158) 2023-07-06T03:47:40.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869cf9c700 1 -- 172.21.15.97:0/4212853627 learned_addr learned my addr 172.21.15.97:0/4212853627 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f868ffff700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f869815c520 0x7f8698158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f8697fff700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86980cd390 0x7f86980d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869cf9c700 1 -- 172.21.15.97:0/4212853627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f869815c520 msgr2=0x7f8698158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869cf9c700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f869815c520 0x7f8698158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869cf9c700 1 -- 172.21.15.97:0/4212853627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86980cd390 msgr2=0x7f86980d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869cf9c700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86980cd390 0x7f86980d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.817+0000 7f869cf9c700 1 -- 172.21.15.97:0/4212853627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8680009550 con 0x7f8698160dd0 2023-07-06T03:47:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.818+0000 7f869cf9c700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8698160dd0 0x7f8698158640 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8690007e40 tx=0x7f869000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.818+0000 7f86957fa700 1 -- 172.21.15.97:0/4212853627 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000ed30 con 0x7f8698160dd0 2023-07-06T03:47:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.818+0000 7f86957fa700 1 -- 172.21.15.97:0/4212853627 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8690014420 con 0x7f8698160dd0 2023-07-06T03:47:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.818+0000 7f869e9ff700 1 -- 172.21.15.97:0/4212853627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86980d2110 con 0x7f8698160dd0 2023-07-06T03:47:40.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.818+0000 7f86957fa700 1 -- 172.21.15.97:0/4212853627 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86900147d0 con 0x7f8698160dd0 2023-07-06T03:47:40.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.818+0000 7f869e9ff700 1 -- 172.21.15.97:0/4212853627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86980d26d0 con 0x7f8698160dd0 2023-07-06T03:47:40.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.819+0000 7f869e9ff700 1 -- 172.21.15.97:0/4212853627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86980d2bf0 con 0x7f8698160dd0 2023-07-06T03:47:40.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.820+0000 7f86957fa700 1 -- 172.21.15.97:0/4212853627 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8690014930 con 0x7f8698160dd0 2023-07-06T03:47:40.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.821+0000 7f86957fa700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 0x7f867806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:40.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.821+0000 7f86957fa700 1 -- 172.21.15.97:0/4212853627 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8690089c10 con 0x7f8698160dd0 2023-07-06T03:47:40.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.821+0000 7f8697fff700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 0x7f867806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:40.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.822+0000 7f8697fff700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 0x7f867806d190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8688007fd0 tx=0x7f8688009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:40.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:40.825+0000 7f86957fa700 1 -- 172.21.15.97:0/4212853627 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86980d2bf0 con 0x7f8698160dd0 2023-07-06T03:47:41.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.021+0000 7f869e9ff700 1 -- 172.21.15.97:0/4212853627 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f86980cd800 con 0x7f867806ace0 2023-07-06T03:47:41.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.031+0000 7f86957fa700 1 -- 172.21.15.97:0/4212853627 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+305841 (secure 0 0 0) 0x7f8688001560 con 0x7f867806ace0 2023-07-06T03:47:41.035 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:41.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 -- 172.21.15.97:0/4212853627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 msgr2=0x7f867806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 0x7f867806d190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8688007fd0 tx=0x7f8688009040 comp rx=0 tx=0).stop 2023-07-06T03:47:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 -- 172.21.15.97:0/4212853627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8698160dd0 msgr2=0x7f8698158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8698160dd0 0x7f8698158640 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8690007e40 tx=0x7f869000e3c0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 -- 172.21.15.97:0/4212853627 shutdown_connections 2023-07-06T03:47:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 0x7f867806d190 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8698160dd0 0x7f8698158640 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86980cd390 0x7f86980d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 --2- 172.21.15.97:0/4212853627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f869815c520 0x7f8698158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 -- 172.21.15.97:0/4212853627 >> 172.21.15.97:0/4212853627 conn(0x7f8698071790 msgr2=0x7f86980d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:41.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 -- 172.21.15.97:0/4212853627 shutdown_connections 2023-07-06T03:47:41.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.035+0000 7f869e9ff700 1 -- 172.21.15.97:0/4212853627 wait complete. 2023-07-06T03:47:41.039 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:47:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-07-06T03:47:40.912045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249488,"kb_used_data":5200,"kb_used_omap":0,"kb_used_meta":244224,"kb_avail":754725232,"statfs":{"total":773094113280,"available":772838637568,"internally_reserved":0,"allocated":5324800,"data_stored":2919489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":250085376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001014"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671120+0000","last_change":"2023-07-06T03:47:35.671120+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671120+0000","last_undegraded":"2023-07-06T03:47:35.671120+0000","last_fullsized":"2023-07-06T03:47:35.671120+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668727+0000","last_change":"2023-07-06T03:47:35.668727+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668727+0000","last_undegraded":"2023-07-06T03:47:35.668727+0000","last_fullsized":"2023-07-06T03:47:35.668727+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664247+0000","last_change":"2023-07-06T03:47:35.664247+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664247+0000","last_undegraded":"2023-07-06T03:47:35.664247+0000","last_fullsized":"2023-07-06T03:47:35.664247+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664482+0000","last_change":"2023-07-06T03:47:35.664482+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664482+0000","last_undegraded":"2023-07-06T03:47:35.664482+0000","last_fullsized":"2023-07-06T03:47:35.664482+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668927+0000","last_change":"2023-07-06T03:47:35.668927+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668927+0000","last_undegraded":"2023-07-06T03:47:35.668927+0000","last_fullsized":"2023-07-06T03:47:35.668927+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665583+0000","last_change":"2023-07-06T03:47:35.665583+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665583+0000","last_undegraded":"2023-07-06T03:47:35.665583+0000","last_fullsized":"2023-07-06T03:47:35.665583+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664339+0000","last_change":"2023-07-06T03:47:35.664339+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664339+0000","last_undegraded":"2023-07-06T03:47:35.664339+0000","last_fullsized":"2023-07-06T03:47:35.664339+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665496+0000","last_change":"2023-07-06T03:47:35.665496+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665496+0000","last_undegraded":"2023-07-06T03:47:35.665496+0000","last_fullsized":"2023-07-06T03:47:35.665496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663507+0000","last_change":"2023-07-06T03:47:35.663507+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663507+0000","last_undegraded":"2023-07-06T03:47:35.663507+0000","last_fullsized":"2023-07-06T03:47:35.663507+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664563+0000","last_change":"2023-07-06T03:47:35.664563+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664563+0000","last_undegraded":"2023-07-06T03:47:35.664563+0000","last_fullsized":"2023-07-06T03:47:35.664563+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669176+0000","last_change":"2023-07-06T03:47:35.669176+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669176+0000","last_undegraded":"2023-07-06T03:47:35.669176+0000","last_fullsized":"2023-07-06T03:47:35.669176+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667342+0000","last_change":"2023-07-06T03:47:35.667342+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667342+0000","last_undegraded":"2023-07-06T03:47:35.667342+0000","last_fullsized":"2023-07-06T03:47:35.667342+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666058+0000","last_change":"2023-07-06T03:47:35.666058+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666058+0000","last_undegraded":"2023-07-06T03:47:35.666058+0000","last_fullsized":"2023-07-06T03:47:35.666058+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670675+0000","last_change":"2023-07-06T03:47:35.670675+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670675+0000","last_undegraded":"2023-07-06T03:47:35.670675+0000","last_fullsized":"2023-07-06T03:47:35.670675+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665278+0000","last_change":"2023-07-06T03:47:35.665278+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665278+0000","last_undegraded":"2023-07-06T03:47:35.665278+0000","last_fullsized":"2023-07-06T03:47:35.665278+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664921+0000","last_change":"2023-07-06T03:47:35.664921+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664921+0000","last_undegraded":"2023-07-06T03:47:35.664921+0000","last_fullsized":"2023-07-06T03:47:35.664921+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670602+0000","last_change":"2023-07-06T03:47:35.670602+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670602+0000","last_undegraded":"2023-07-06T03:47:35.670602+0000","last_fullsized":"2023-07-06T03:47:35.670602+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664731+0000","last_change":"2023-07-06T03:47:35.664731+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664731+0000","last_undegraded":"2023-07-06T03:47:35.664731+0000","last_fullsized":"2023-07-06T03:47:35.664731+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663759+0000","last_change":"2023-07-06T03:47:35.663759+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663759+0000","last_undegraded":"2023-07-06T03:47:35.663759+0000","last_fullsized":"2023-07-06T03:47:35.663759+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670330+0000","last_change":"2023-07-06T03:47:35.670330+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670330+0000","last_undegraded":"2023-07-06T03:47:35.670330+0000","last_fullsized":"2023-07-06T03:47:35.670330+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664418+0000","last_change":"2023-07-06T03:47:35.664418+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664418+0000","last_undegraded":"2023-07-06T03:47:35.664418+0000","last_fullsized":"2023-07-06T03:47:35.664418+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663209+0000","last_change":"2023-07-06T03:47:35.663209+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663209+0000","last_undegraded":"2023-07-06T03:47:35.663209+0000","last_fullsized":"2023-07-06T03:47:35.663209+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662599+0000","last_change":"2023-07-06T03:47:35.662599+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662599+0000","last_undegraded":"2023-07-06T03:47:35.662599+0000","last_fullsized":"2023-07-06T03:47:35.662599+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666150+0000","last_change":"2023-07-06T03:47:35.666150+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666150+0000","last_undegraded":"2023-07-06T03:47:35.666150+0000","last_fullsized":"2023-07-06T03:47:35.666150+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670081+0000","last_change":"2023-07-06T03:47:35.670081+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670081+0000","last_undegraded":"2023-07-06T03:47:35.670081+0000","last_fullsized":"2023-07-06T03:47:35.670081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664042+0000","last_change":"2023-07-06T03:47:35.664042+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664042+0000","last_undegraded":"2023-07-06T03:47:35.664042+0000","last_fullsized":"2023-07-06T03:47:35.664042+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667395+0000","last_change":"2023-07-06T03:47:35.667395+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667395+0000","last_undegraded":"2023-07-06T03:47:35.667395+0000","last_fullsized":"2023-07-06T03:47:35.667395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669999+0000","last_change":"2023-07-06T03:47:35.669999+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669999+0000","last_undegraded":"2023-07-06T03:47:35.669999+0000","last_fullsized":"2023-07-06T03:47:35.669999+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.674695+0000","last_change":"2023-07-06T03:47:35.674695+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.674695+0000","last_undegraded":"2023-07-06T03:47:35.674695+0000","last_fullsized":"2023-07-06T03:47:35.674695+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668319+0000","last_change":"2023-07-06T03:47:35.668319+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668319+0000","last_undegraded":"2023-07-06T03:47:35.668319+0000","last_fullsized":"2023-07-06T03:47:35.668319+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662011+0000","last_change":"2023-07-06T03:47:35.662011+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662011+0000","last_undegraded":"2023-07-06T03:47:35.662011+0000","last_fullsized":"2023-07-06T03:47:35.662011+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668114+0000","last_change":"2023-07-06T03:47:35.668114+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668114+0000","last_undegraded":"2023-07-06T03:47:35.668114+0000","last_fullsized":"2023-07-06T03:47:35.668114+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666321+0000","last_change":"2023-07-06T03:47:35.666321+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666321+0000","last_undegraded":"2023-07-06T03:47:35.666321+0000","last_fullsized":"2023-07-06T03:47:35.666321+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686079+0000","last_change":"2023-07-06T03:47:35.684675+0000","last_active":"2023-07-06T03:47:35.686079+0000","last_peered":"2023-07-06T03:47:35.686079+0000","last_clean":"2023-07-06T03:47:35.686079+0000","last_became_active":"2023-07-06T03:47:35.684398+0000","last_became_peered":"2023-07-06T03:47:35.684398+0000","last_unstale":"2023-07-06T03:47:35.686079+0000","last_undegraded":"2023-07-06T03:47:35.686079+0000","last_fullsized":"2023-07-06T03:47:35.686079+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:47.031506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666528+0000","last_change":"2023-07-06T03:47:35.666528+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666528+0000","last_undegraded":"2023-07-06T03:47:35.666528+0000","last_fullsized":"2023-07-06T03:47:35.666528+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682319+0000","last_change":"2023-07-06T03:47:35.680067+0000","last_active":"2023-07-06T03:47:35.682319+0000","last_peered":"2023-07-06T03:47:35.682319+0000","last_clean":"2023-07-06T03:47:35.682319+0000","last_became_active":"2023-07-06T03:47:35.679691+0000","last_became_peered":"2023-07-06T03:47:35.679691+0000","last_unstale":"2023-07-06T03:47:35.682319+0000","last_undegraded":"2023-07-06T03:47:35.682319+0000","last_fullsized":"2023-07-06T03:47:35.682319+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:14.005662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667029+0000","last_change":"2023-07-06T03:47:35.667029+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667029+0000","last_undegraded":"2023-07-06T03:47:35.667029+0000","last_fullsized":"2023-07-06T03:47:35.667029+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684178+0000","last_change":"2023-07-06T03:47:35.676405+0000","last_active":"2023-07-06T03:47:35.684178+0000","last_peered":"2023-07-06T03:47:35.684178+0000","last_clean":"2023-07-06T03:47:35.684178+0000","last_became_active":"2023-07-06T03:47:35.676089+0000","last_became_peered":"2023-07-06T03:47:35.676089+0000","last_unstale":"2023-07-06T03:47:35.684178+0000","last_undegraded":"2023-07-06T03:47:35.684178+0000","last_fullsized":"2023-07-06T03:47:35.684178+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:58:51.450312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664710+0000","last_change":"2023-07-06T03:47:35.664710+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664710+0000","last_undegraded":"2023-07-06T03:47:35.664710+0000","last_fullsized":"2023-07-06T03:47:35.664710+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682615+0000","last_change":"2023-07-06T03:47:35.678850+0000","last_active":"2023-07-06T03:47:35.682615+0000","last_peered":"2023-07-06T03:47:35.682615+0000","last_clean":"2023-07-06T03:47:35.682615+0000","last_became_active":"2023-07-06T03:47:35.678520+0000","last_became_peered":"2023-07-06T03:47:35.678520+0000","last_unstale":"2023-07-06T03:47:35.682615+0000","last_undegraded":"2023-07-06T03:47:35.682615+0000","last_fullsized":"2023-07-06T03:47:35.682615+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:08:14.846337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671834+0000","last_change":"2023-07-06T03:47:35.671834+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671834+0000","last_undegraded":"2023-07-06T03:47:35.671834+0000","last_fullsized":"2023-07-06T03:47:35.671834+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.683654+0000","last_change":"2023-07-06T03:47:35.679264+0000","last_active":"2023-07-06T03:47:35.683654+0000","last_peered":"2023-07-06T03:47:35.683654+0000","last_clean":"2023-07-06T03:47:35.683654+0000","last_became_active":"2023-07-06T03:47:35.678950+0000","last_became_peered":"2023-07-06T03:47:35.678950+0000","last_unstale":"2023-07-06T03:47:35.683654+0000","last_undegraded":"2023-07-06T03:47:35.683654+0000","last_fullsized":"2023-07-06T03:47:35.683654+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:11:09.395143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667838+0000","last_change":"2023-07-06T03:47:35.667838+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667838+0000","last_undegraded":"2023-07-06T03:47:35.667838+0000","last_fullsized":"2023-07-06T03:47:35.667838+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681981+0000","last_change":"2023-07-06T03:47:35.679463+0000","last_active":"2023-07-06T03:47:35.681981+0000","last_peered":"2023-07-06T03:47:35.681981+0000","last_clean":"2023-07-06T03:47:35.681981+0000","last_became_active":"2023-07-06T03:47:35.679249+0000","last_became_peered":"2023-07-06T03:47:35.679249+0000","last_unstale":"2023-07-06T03:47:35.681981+0000","last_undegraded":"2023-07-06T03:47:35.681981+0000","last_fullsized":"2023-07-06T03:47:35.681981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:48:41.016578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665066+0000","last_change":"2023-07-06T03:47:35.665066+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665066+0000","last_undegraded":"2023-07-06T03:47:35.665066+0000","last_fullsized":"2023-07-06T03:47:35.665066+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686346+0000","last_change":"2023-07-06T03:47:35.684332+0000","last_active":"2023-07-06T03:47:35.686346+0000","last_peered":"2023-07-06T03:47:35.686346+0000","last_clean":"2023-07-06T03:47:35.686346+0000","last_became_active":"2023-07-06T03:47:35.684064+0000","last_became_peered":"2023-07-06T03:47:35.684064+0000","last_unstale":"2023-07-06T03:47:35.686346+0000","last_undegraded":"2023-07-06T03:47:35.686346+0000","last_fullsized":"2023-07-06T03:47:35.686346+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:45:08.107109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664510+0000","last_change":"2023-07-06T03:47:35.664510+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664510+0000","last_undegraded":"2023-07-06T03:47:35.664510+0000","last_fullsized":"2023-07-06T03:47:35.664510+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684604+0000","last_change":"2023-07-06T03:47:35.679127+0000","last_active":"2023-07-06T03:47:35.684604+0000","last_peered":"2023-07-06T03:47:35.684604+0000","last_clean":"2023-07-06T03:47:35.684604+0000","last_became_active":"2023-07-06T03:47:35.678630+0000","last_became_peered":"2023-07-06T03:47:35.678630+0000","last_unstale":"2023-07-06T03:47:35.684604+0000","last_undegraded":"2023-07-06T03:47:35.684604+0000","last_fullsized":"2023-07-06T03:47:35.684604+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:26:20.114823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671405+0000","last_change":"2023-07-06T03:47:35.671405+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671405+0000","last_undegraded":"2023-07-06T03:47:35.671405+0000","last_fullsized":"2023-07-06T03:47:35.671405+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.680869+0000","last_change":"2023-07-06T03:47:35.678328+0000","last_active":"2023-07-06T03:47:35.680869+0000","last_peered":"2023-07-06T03:47:35.680869+0000","last_clean":"2023-07-06T03:47:35.680869+0000","last_became_active":"2023-07-06T03:47:35.678081+0000","last_became_peered":"2023-07-06T03:47:35.678081+0000","last_unstale":"2023-07-06T03:47:35.680869+0000","last_undegraded":"2023-07-06T03:47:35.680869+0000","last_fullsized":"2023-07-06T03:47:35.680869+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:24:18.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664273+0000","last_change":"2023-07-06T03:47:35.664273+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664273+0000","last_undegraded":"2023-07-06T03:47:35.664273+0000","last_fullsized":"2023-07-06T03:47:35.664273+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.679689+0000","last_change":"2023-07-06T03:47:35.673408+0000","last_active":"2023-07-06T03:47:35.679689+0000","last_peered":"2023-07-06T03:47:35.679689+0000","last_clean":"2023-07-06T03:47:35.679689+0000","last_became_active":"2023-07-06T03:47:35.672993+0000","last_became_peered":"2023-07-06T03:47:35.672993+0000","last_unstale":"2023-07-06T03:47:35.679689+0000","last_undegraded":"2023-07-06T03:47:35.679689+0000","last_fullsized":"2023-07-06T03:47:35.679689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:56:55.279062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670077+0000","last_change":"2023-07-06T03:47:35.670077+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670077+0000","last_undegraded":"2023-07-06T03:47:35.670077+0000","last_fullsized":"2023-07-06T03:47:35.670077+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681921+0000","last_change":"2023-07-06T03:47:35.679946+0000","last_active":"2023-07-06T03:47:35.681921+0000","last_peered":"2023-07-06T03:47:35.681921+0000","last_clean":"2023-07-06T03:47:35.681921+0000","last_became_active":"2023-07-06T03:47:35.679487+0000","last_became_peered":"2023-07-06T03:47:35.679487+0000","last_unstale":"2023-07-06T03:47:35.681921+0000","last_undegraded":"2023-07-06T03:47:35.681921+0000","last_fullsized":"2023-07-06T03:47:35.681921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:52.953019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669355+0000","last_change":"2023-07-06T03:47:35.669355+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669355+0000","last_undegraded":"2023-07-06T03:47:35.669355+0000","last_fullsized":"2023-07-06T03:47:35.669355+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686839+0000","last_change":"2023-07-06T03:47:35.679499+0000","last_active":"2023-07-06T03:47:35.686839+0000","last_peered":"2023-07-06T03:47:35.686839+0000","last_clean":"2023-07-06T03:47:35.686839+0000","last_became_active":"2023-07-06T03:47:35.679153+0000","last_became_peered":"2023-07-06T03:47:35.679153+0000","last_unstale":"2023-07-06T03:47:35.686839+0000","last_undegraded":"2023-07-06T03:47:35.686839+0000","last_fullsized":"2023-07-06T03:47:35.686839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:28:35.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667017+0000","last_change":"2023-07-06T03:47:35.667017+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667017+0000","last_undegraded":"2023-07-06T03:47:35.667017+0000","last_fullsized":"2023-07-06T03:47:35.667017+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.676626+0000","last_change":"2023-07-06T03:47:35.675074+0000","last_active":"2023-07-06T03:47:35.676626+0000","last_peered":"2023-07-06T03:47:35.676626+0000","last_clean":"2023-07-06T03:47:35.676626+0000","last_became_active":"2023-07-06T03:47:35.674815+0000","last_became_peered":"2023-07-06T03:47:35.674815+0000","last_unstale":"2023-07-06T03:47:35.676626+0000","last_undegraded":"2023-07-06T03:47:35.676626+0000","last_fullsized":"2023-07-06T03:47:35.676626+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:44:22.676309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669675+0000","last_change":"2023-07-06T03:47:35.669675+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669675+0000","last_undegraded":"2023-07-06T03:47:35.669675+0000","last_fullsized":"2023-07-06T03:47:35.669675+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681758+0000","last_change":"2023-07-06T03:47:35.680141+0000","last_active":"2023-07-06T03:47:35.681758+0000","last_peered":"2023-07-06T03:47:35.681758+0000","last_clean":"2023-07-06T03:47:35.681758+0000","last_became_active":"2023-07-06T03:47:35.679775+0000","last_became_peered":"2023-07-06T03:47:35.679775+0000","last_unstale":"2023-07-06T03:47:35.681758+0000","last_undegraded":"2023-07-06T03:47:35.681758+0000","last_fullsized":"2023-07-06T03:47:35.681758+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:08:52.125995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668183+0000","last_change":"2023-07-06T03:47:35.668183+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668183+0000","last_undegraded":"2023-07-06T03:47:35.668183+0000","last_fullsized":"2023-07-06T03:47:35.668183+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682381+0000","last_change":"2023-07-06T03:47:35.676310+0000","last_active":"2023-07-06T03:47:35.682381+0000","last_peered":"2023-07-06T03:47:35.682381+0000","last_clean":"2023-07-06T03:47:35.682381+0000","last_became_active":"2023-07-06T03:47:35.676038+0000","last_became_peered":"2023-07-06T03:47:35.676038+0000","last_unstale":"2023-07-06T03:47:35.682381+0000","last_undegraded":"2023-07-06T03:47:35.682381+0000","last_fullsized":"2023-07-06T03:47:35.682381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:02:33.045576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663051+0000","last_change":"2023-07-06T03:47:35.663051+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663051+0000","last_undegraded":"2023-07-06T03:47:35.663051+0000","last_fullsized":"2023-07-06T03:47:35.663051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686476+0000","last_change":"2023-07-06T03:47:35.683270+0000","last_active":"2023-07-06T03:47:35.686476+0000","last_peered":"2023-07-06T03:47:35.686476+0000","last_clean":"2023-07-06T03:47:35.686476+0000","last_became_active":"2023-07-06T03:47:35.682973+0000","last_became_peered":"2023-07-06T03:47:35.682973+0000","last_unstale":"2023-07-06T03:47:35.686476+0000","last_undegraded":"2023-07-06T03:47:35.686476+0000","last_fullsized":"2023-07-06T03:47:35.686476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:03:08.897884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662280+0000","last_change":"2023-07-06T03:47:35.662280+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662280+0000","last_undegraded":"2023-07-06T03:47:35.662280+0000","last_fullsized":"2023-07-06T03:47:35.662280+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682343+0000","last_change":"2023-07-06T03:47:35.680067+0000","last_active":"2023-07-06T03:47:35.682343+0000","last_peered":"2023-07-06T03:47:35.682343+0000","last_clean":"2023-07-06T03:47:35.682343+0000","last_became_active":"2023-07-06T03:47:35.679704+0000","last_became_peered":"2023-07-06T03:47:35.679704+0000","last_unstale":"2023-07-06T03:47:35.682343+0000","last_undegraded":"2023-07-06T03:47:35.682343+0000","last_fullsized":"2023-07-06T03:47:35.682343+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:31:03.309257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670261+0000","last_change":"2023-07-06T03:47:35.670261+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670261+0000","last_undegraded":"2023-07-06T03:47:35.670261+0000","last_fullsized":"2023-07-06T03:47:35.670261+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686133+0000","last_change":"2023-07-06T03:47:35.680931+0000","last_active":"2023-07-06T03:47:35.686133+0000","last_peered":"2023-07-06T03:47:35.686133+0000","last_clean":"2023-07-06T03:47:35.686133+0000","last_became_active":"2023-07-06T03:47:35.680659+0000","last_became_peered":"2023-07-06T03:47:35.680659+0000","last_unstale":"2023-07-06T03:47:35.686133+0000","last_undegraded":"2023-07-06T03:47:35.686133+0000","last_fullsized":"2023-07-06T03:47:35.686133+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:01:39.848129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667702+0000","last_change":"2023-07-06T03:47:35.667702+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667702+0000","last_undegraded":"2023-07-06T03:47:35.667702+0000","last_fullsized":"2023-07-06T03:47:35.667702+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681731+0000","last_change":"2023-07-06T03:47:35.679503+0000","last_active":"2023-07-06T03:47:35.681731+0000","last_peered":"2023-07-06T03:47:35.681731+0000","last_clean":"2023-07-06T03:47:35.681731+0000","last_became_active":"2023-07-06T03:47:35.679231+0000","last_became_peered":"2023-07-06T03:47:35.679231+0000","last_unstale":"2023-07-06T03:47:35.681731+0000","last_undegraded":"2023-07-06T03:47:35.681731+0000","last_fullsized":"2023-07-06T03:47:35.681731+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:08:16.691892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665577+0000","last_change":"2023-07-06T03:47:35.665577+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665577+0000","last_undegraded":"2023-07-06T03:47:35.665577+0000","last_fullsized":"2023-07-06T03:47:35.665577+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.687009+0000","last_change":"2023-07-06T03:47:35.685943+0000","last_active":"2023-07-06T03:47:35.687009+0000","last_peered":"2023-07-06T03:47:35.687009+0000","last_clean":"2023-07-06T03:47:35.687009+0000","last_became_active":"2023-07-06T03:47:35.685673+0000","last_became_peered":"2023-07-06T03:47:35.685673+0000","last_unstale":"2023-07-06T03:47:35.687009+0000","last_undegraded":"2023-07-06T03:47:35.687009+0000","last_fullsized":"2023-07-06T03:47:35.687009+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:14:16.989542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669618+0000","last_change":"2023-07-06T03:47:35.669618+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669618+0000","last_undegraded":"2023-07-06T03:47:35.669618+0000","last_fullsized":"2023-07-06T03:47:35.669618+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.689703+0000","last_change":"2023-07-06T03:47:35.684207+0000","last_active":"2023-07-06T03:47:35.689703+0000","last_peered":"2023-07-06T03:47:35.689703+0000","last_clean":"2023-07-06T03:47:35.689703+0000","last_became_active":"2023-07-06T03:47:35.683837+0000","last_became_peered":"2023-07-06T03:47:35.683837+0000","last_unstale":"2023-07-06T03:47:35.689703+0000","last_undegraded":"2023-07-06T03:47:35.689703+0000","last_fullsized":"2023-07-06T03:47:35.689703+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:11.065681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664327+0000","last_change":"2023-07-06T03:47:35.664327+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664327+0000","last_undegraded":"2023-07-06T03:47:35.664327+0000","last_fullsized":"2023-07-06T03:47:35.664327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684585+0000","last_change":"2023-07-06T03:47:35.680503+0000","last_active":"2023-07-06T03:47:35.684585+0000","last_peered":"2023-07-06T03:47:35.684585+0000","last_clean":"2023-07-06T03:47:35.684585+0000","last_became_active":"2023-07-06T03:47:35.680171+0000","last_became_peered":"2023-07-06T03:47:35.680171+0000","last_unstale":"2023-07-06T03:47:35.684585+0000","last_undegraded":"2023-07-06T03:47:35.684585+0000","last_fullsized":"2023-07-06T03:47:35.684585+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:07:12.686190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.675202+0000","last_change":"2023-07-06T03:47:35.675202+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.675202+0000","last_undegraded":"2023-07-06T03:47:35.675202+0000","last_fullsized":"2023-07-06T03:47:35.675202+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686872+0000","last_change":"2023-07-06T03:47:35.680421+0000","last_active":"2023-07-06T03:47:35.686872+0000","last_peered":"2023-07-06T03:47:35.686872+0000","last_clean":"2023-07-06T03:47:35.686872+0000","last_became_active":"2023-07-06T03:47:35.679465+0000","last_became_peered":"2023-07-06T03:47:35.679465+0000","last_unstale":"2023-07-06T03:47:35.686872+0000","last_undegraded":"2023-07-06T03:47:35.686872+0000","last_fullsized":"2023-07-06T03:47:35.686872+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:22.389725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667496+0000","last_change":"2023-07-06T03:47:35.667496+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667496+0000","last_undegraded":"2023-07-06T03:47:35.667496+0000","last_fullsized":"2023-07-06T03:47:35.667496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.683955+0000","last_change":"2023-07-06T03:47:35.673523+0000","last_active":"2023-07-06T03:47:35.683955+0000","last_peered":"2023-07-06T03:47:35.683955+0000","last_clean":"2023-07-06T03:47:35.683955+0000","last_became_active":"2023-07-06T03:47:35.672470+0000","last_became_peered":"2023-07-06T03:47:35.672470+0000","last_unstale":"2023-07-06T03:47:35.683955+0000","last_undegraded":"2023-07-06T03:47:35.683955+0000","last_fullsized":"2023-07-06T03:47:35.683955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:23:29.539377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662836+0000","last_change":"2023-07-06T03:47:35.662836+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662836+0000","last_undegraded":"2023-07-06T03:47:35.662836+0000","last_fullsized":"2023-07-06T03:47:35.662836+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682665+0000","last_change":"2023-07-06T03:47:35.681014+0000","last_active":"2023-07-06T03:47:35.682665+0000","last_peered":"2023-07-06T03:47:35.682665+0000","last_clean":"2023-07-06T03:47:35.682665+0000","last_became_active":"2023-07-06T03:47:35.680766+0000","last_became_peered":"2023-07-06T03:47:35.680766+0000","last_unstale":"2023-07-06T03:47:35.682665+0000","last_undegraded":"2023-07-06T03:47:35.682665+0000","last_fullsized":"2023-07-06T03:47:35.682665+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:11:00.458188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.665633+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:47:35.665633+0000","last_peered":"2023-07-06T03:47:35.665633+0000","last_clean":"2023-07-06T03:47:35.665633+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:47:35.665633+0000","last_undegraded":"2023-07-06T03:47:35.665633+0000","last_fullsized":"2023-07-06T03:47:35.665633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:25:11.511422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664761+0000","last_change":"2023-07-06T03:47:35.664761+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664761+0000","last_undegraded":"2023-07-06T03:47:35.664761+0000","last_fullsized":"2023-07-06T03:47:35.664761+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686329+0000","last_change":"2023-07-06T03:47:35.684268+0000","last_active":"2023-07-06T03:47:35.686329+0000","last_peered":"2023-07-06T03:47:35.686329+0000","last_clean":"2023-07-06T03:47:35.686329+0000","last_became_active":"2023-07-06T03:47:35.683961+0000","last_became_peered":"2023-07-06T03:47:35.683961+0000","last_unstale":"2023-07-06T03:47:35.686329+0000","last_undegraded":"2023-07-06T03:47:35.686329+0000","last_fullsized":"2023-07-06T03:47:35.686329+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:46:49.441174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.668531+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:47:35.668531+0000","last_peered":"2023-07-06T03:47:35.668531+0000","last_clean":"2023-07-06T03:47:35.668531+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:47:35.668531+0000","last_undegraded":"2023-07-06T03:47:35.668531+0000","last_fullsized":"2023-07-06T03:47:35.668531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:40:54.630791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665474+0000","last_change":"2023-07-06T03:47:35.665474+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665474+0000","last_undegraded":"2023-07-06T03:47:35.665474+0000","last_fullsized":"2023-07-06T03:47:35.665474+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.683642+0000","last_change":"2023-07-06T03:47:35.677576+0000","last_active":"2023-07-06T03:47:35.683642+0000","last_peered":"2023-07-06T03:47:35.683642+0000","last_clean":"2023-07-06T03:47:35.683642+0000","last_became_active":"2023-07-06T03:47:35.677247+0000","last_became_peered":"2023-07-06T03:47:35.677247+0000","last_unstale":"2023-07-06T03:47:35.683642+0000","last_undegraded":"2023-07-06T03:47:35.683642+0000","last_fullsized":"2023-07-06T03:47:35.683642+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:22:57.626952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.668542+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:47:35.668542+0000","last_peered":"2023-07-06T03:47:35.668542+0000","last_clean":"2023-07-06T03:47:35.668542+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:47:35.668542+0000","last_undegraded":"2023-07-06T03:47:35.668542+0000","last_fullsized":"2023-07-06T03:47:35.668542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:55:11.004301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671814+0000","last_change":"2023-07-06T03:47:35.671814+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671814+0000","last_undegraded":"2023-07-06T03:47:35.671814+0000","last_fullsized":"2023-07-06T03:47:35.671814+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684977+0000","last_change":"2023-07-06T03:47:35.677003+0000","last_active":"2023-07-06T03:47:35.684977+0000","last_peered":"2023-07-06T03:47:35.684977+0000","last_clean":"2023-07-06T03:47:35.684977+0000","last_became_active":"2023-07-06T03:47:35.676727+0000","last_became_peered":"2023-07-06T03:47:35.676727+0000","last_unstale":"2023-07-06T03:47:35.684977+0000","last_undegraded":"2023-07-06T03:47:35.684977+0000","last_fullsized":"2023-07-06T03:47:35.684977+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:35:50.061745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.667278+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:47:35.667278+0000","last_peered":"2023-07-06T03:47:35.667278+0000","last_clean":"2023-07-06T03:47:35.667278+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:47:35.667278+0000","last_undegraded":"2023-07-06T03:47:35.667278+0000","last_fullsized":"2023-07-06T03:47:35.667278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:18:53.047698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.666895+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:47:35.666895+0000","last_peered":"2023-07-06T03:47:35.666895+0000","last_clean":"2023-07-06T03:47:35.666895+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:47:35.666895+0000","last_undegraded":"2023-07-06T03:47:35.666895+0000","last_fullsized":"2023-07-06T03:47:35.666895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:22:28.543253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664261+0000","last_change":"2023-07-06T03:47:35.664261+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664261+0000","last_undegraded":"2023-07-06T03:47:35.664261+0000","last_fullsized":"2023-07-06T03:47:35.664261+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681903+0000","last_change":"2023-07-06T03:47:35.679794+0000","last_active":"2023-07-06T03:47:35.681903+0000","last_peered":"2023-07-06T03:47:35.681903+0000","last_clean":"2023-07-06T03:47:35.681903+0000","last_became_active":"2023-07-06T03:47:35.679116+0000","last_became_peered":"2023-07-06T03:47:35.679116+0000","last_unstale":"2023-07-06T03:47:35.681903+0000","last_undegraded":"2023-07-06T03:47:35.681903+0000","last_fullsized":"2023-07-06T03:47:35.681903+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:13:53.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.666703+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:47:35.666703+0000","last_peered":"2023-07-06T03:47:35.666703+0000","last_clean":"2023-07-06T03:47:35.666703+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:47:35.666703+0000","last_undegraded":"2023-07-06T03:47:35.666703+0000","last_fullsized":"2023-07-06T03:47:35.666703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:39:52.909931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667932+0000","last_change":"2023-07-06T03:47:35.667932+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667932+0000","last_undegraded":"2023-07-06T03:47:35.667932+0000","last_fullsized":"2023-07-06T03:47:35.667932+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686114+0000","last_change":"2023-07-06T03:47:35.684944+0000","last_active":"2023-07-06T03:47:35.686114+0000","last_peered":"2023-07-06T03:47:35.686114+0000","last_clean":"2023-07-06T03:47:35.686114+0000","last_became_active":"2023-07-06T03:47:35.684654+0000","last_became_peered":"2023-07-06T03:47:35.684654+0000","last_unstale":"2023-07-06T03:47:35.686114+0000","last_undegraded":"2023-07-06T03:47:35.686114+0000","last_fullsized":"2023-07-06T03:47:35.686114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:05.886555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.668075+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:47:35.668075+0000","last_peered":"2023-07-06T03:47:35.668075+0000","last_clean":"2023-07-06T03:47:35.668075+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:47:35.668075+0000","last_undegraded":"2023-07-06T03:47:35.668075+0000","last_fullsized":"2023-07-06T03:47:35.668075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:33:23.406225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663757+0000","last_change":"2023-07-06T03:47:35.663757+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663757+0000","last_undegraded":"2023-07-06T03:47:35.663757+0000","last_fullsized":"2023-07-06T03:47:35.663757+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684244+0000","last_change":"2023-07-06T03:47:35.677115+0000","last_active":"2023-07-06T03:47:35.684244+0000","last_peered":"2023-07-06T03:47:35.684244+0000","last_clean":"2023-07-06T03:47:35.684244+0000","last_became_active":"2023-07-06T03:47:35.676826+0000","last_became_peered":"2023-07-06T03:47:35.676826+0000","last_unstale":"2023-07-06T03:47:35.684244+0000","last_undegraded":"2023-07-06T03:47:35.684244+0000","last_fullsized":"2023-07-06T03:47:35.684244+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:26.484401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.664524+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:47:35.664524+0000","last_peered":"2023-07-06T03:47:35.664524+0000","last_clean":"2023-07-06T03:47:35.664524+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:47:35.664524+0000","last_undegraded":"2023-07-06T03:47:35.664524+0000","last_fullsized":"2023-07-06T03:47:35.664524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:55:15.702636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668249+0000","last_change":"2023-07-06T03:47:35.668249+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668249+0000","last_undegraded":"2023-07-06T03:47:35.668249+0000","last_fullsized":"2023-07-06T03:47:35.668249+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684607+0000","last_change":"2023-07-06T03:47:35.680550+0000","last_active":"2023-07-06T03:47:35.684607+0000","last_peered":"2023-07-06T03:47:35.684607+0000","last_clean":"2023-07-06T03:47:35.684607+0000","last_became_active":"2023-07-06T03:47:35.680201+0000","last_became_peered":"2023-07-06T03:47:35.680201+0000","last_unstale":"2023-07-06T03:47:35.684607+0000","last_undegraded":"2023-07-06T03:47:35.684607+0000","last_fullsized":"2023-07-06T03:47:35.684607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:54:15.807837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.667832+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:47:35.667832+0000","last_peered":"2023-07-06T03:47:35.667832+0000","last_clean":"2023-07-06T03:47:35.667832+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:47:35.667832+0000","last_undegraded":"2023-07-06T03:47:35.667832+0000","last_fullsized":"2023-07-06T03:47:35.667832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:11:43.805367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":528384,"data_stored":223271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86399999999999999,"5min":1.345,"15min":1.345},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.81499999999999995,"15min":0.83699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91100000000000003,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.748,"15min":0.76300000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85799999999999998,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.91800000000000004,"15min":0.95899999999999996},"min":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.876,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.90800000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.82399999999999995,"15min":0.84699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":3.77,"15min":3.77},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88400000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95699999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.94699999999999995,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96599999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.746,"15min":0.746},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98799999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92800000000000005,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.92800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.79400000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74199999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83899999999999997,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77800000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.44900000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.874,"5min":1.119,"15min":1.119},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.76500000000000001,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.73999999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78200000000000003,"5min":1.093,"15min":1.093},"last":0.621}]},{"osd":4,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93500000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.746,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.81000000000000005,"5min":1.085,"15min":1.085},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72599999999999998,"5min":1.218,"15min":1.218},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.80200000000000005,"15min":0.81899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0269999999999999,"5min":1.198,"15min":1.198},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.84299999999999997,"15min":0.86699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.83799999999999997,"5min":1.323,"15min":1.323},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.84899999999999998,"15min":0.877},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86799999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":1110016,"data_stored":807011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58399999999999996,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.58999999999999997,"15min":0.60999999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.628,"5min":1.96,"15min":1.96},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":1.0229999999999999,"15min":1.119},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54600000000000004,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.65800000000000003,"15min":0.68500000000000005},"min":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67900000000000005,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.63400000000000001,"15min":0.65600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.752,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.59499999999999997,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.439}]},{"osd":4,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":1.2789999999999999,"15min":1.375},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86199999999999999},"min":{"1min":0.48099999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94299999999999995,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.86199999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.78500000000000003,"15min":0.80100000000000005},"min":{"1min":0.54100000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92100000000000004,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.2929999999999999,"15min":1.393},"min":{"1min":0.55900000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.874,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.79200000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.60599999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.90600000000000003,"5min":1.365,"15min":1.365},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.335,"15min":1.4370000000000001},"min":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.88500000000000001,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.84899999999999998,"15min":0.875},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.86299999999999999,"5min":2.806,"15min":2.806},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82999999999999996,"15min":0.84599999999999997},"min":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.93000000000000005,"5min":1.337,"15min":1.337},"last":0.77200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41699999999999998,"15min":0.41799999999999998},"min":{"1min":0.316,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.443,"15min":0.441},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47599999999999998,"15min":0.47999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60199999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80700000000000005,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.438}]},{"osd":3,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.38800000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74199999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":0.76300000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":1.171,"15min":1.171},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.87,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91700000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.88500000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.546,"15min":1.546},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81000000000000005,"15min":0.81799999999999995},"min":{"1min":0.59199999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96799999999999997,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":1.155,"15min":1.2290000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.95599999999999996,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35648,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336192,"statfs":{"total":96636764160,"available":96600260608,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.57299999999999995,"5min":0.626,"15min":0.626},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.52600000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.66400000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66800000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.61099999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69299999999999995,"15min":0.69399999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":1.165,"15min":1.165},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.751},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.90500000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.746,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91400000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.90500000000000003,"5min":2.157,"15min":2.157},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.61299999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88900000000000001,"5min":1.966,"15min":1.966},"last":0.81799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.65700000000000003,"15min":0.69699999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.40699999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60599999999999998,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":1.196,"15min":1.3260000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63300000000000001,"5min":15.911,"15min":15.911},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.266,"15min":1.3959999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63700000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.55600000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":1.2709999999999999,"15min":1.401},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.59699999999999998,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.442}]},{"osd":4,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84899999999999998,"5min":1.179,"15min":1.179},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.97699999999999998,"15min":1.028},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89500000000000002,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.97999999999999998,"15min":1.0269999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92700000000000005,"5min":5.077,"15min":5.077},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.0409999999999999,"15min":1.0940000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89600000000000002,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.52500000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.90200000000000002,"5min":1.143,"15min":1.143},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.995,"15min":1.0449999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96799999999999997,"5min":5.157,"15min":5.157},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.85099999999999998,"15min":0.878},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96399999999999997,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.006,"15min":1.0549999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93000000000000005,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.73999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.91500000000000004,"15min":0.95899999999999996},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94399999999999995,"5min":6.258,"15min":6.258},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.93400000000000005,"15min":0.97299999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1419999999999999,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":1.0189999999999999,"15min":1.069},"min":{"1min":0.54800000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0089999999999999,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.073,"5min":1.413,"15min":1.413},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80800000000000005,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.499},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.873,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2330000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.028,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.76700000000000002,"15min":0.77700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.58699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1105920,"data_stored":805992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.90500000000000003,"15min":0.95499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94199999999999995,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81100000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.629},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95399999999999996,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69299999999999995,"15min":0.68500000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.749},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79600000000000004,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:47:41.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:47:41.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.288+0000 7f04ef1ff700 1 Processor -- start 2023-07-06T03:47:41.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.288+0000 7f04ef1ff700 1 -- start start 2023-07-06T03:47:41.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.289+0000 7f04ef1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04e815c520 0x7f04e80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.289+0000 7f04ef1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f04e8160180 0x7f04e81605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04ef1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e80cdae0 con 0x7f04e8160b30 2023-07-06T03:47:41.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04ef1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e80cd8d0 con 0x7f04e8160180 2023-07-06T03:47:41.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04ef1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e80b6640 con 0x7f04e815c520 2023-07-06T03:47:41.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04e7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f04e8160180 0x7f04e81605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04e7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f04e8160180 0x7f04e81605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33160/0 (socket says 172.21.15.97:33160) 2023-07-06T03:47:41.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04e7fff700 1 -- 172.21.15.97:0/2258458387 learned_addr learned my addr 172.21.15.97:0/2258458387 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04ecf9b700 1 --2- 172.21.15.97:0/2258458387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04e815c520 0x7f04e80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04e7fff700 1 -- 172.21.15.97:0/2258458387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04e815c520 msgr2=0x7f04e80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04e7fff700 1 --2- 172.21.15.97:0/2258458387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04e815c520 0x7f04e80cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04e7fff700 1 -- 172.21.15.97:0/2258458387 >> v1:172.21.15.146:6789/0 conn(0x7f04e8160b30 legacy=0x7f04e80cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04e7fff700 1 -- 172.21.15.97:0/2258458387 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e80b6920 con 0x7f04e8160180 2023-07-06T03:47:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.290+0000 7f04ecf9b700 1 --2- 172.21.15.97:0/2258458387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04e815c520 0x7f04e80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:41.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.291+0000 7f04e7fff700 1 --2- 172.21.15.97:0/2258458387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04e8160180 0x7f04e81605f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f04d40098f0 tx=0x7f04d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7109d9e7375bedf7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:41.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.291+0000 7f04e77fe700 1 -- 172.21.15.97:0/2258458387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d400b8a0 con 0x7f04e8160180 2023-07-06T03:47:41.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.291+0000 7f04e77fe700 1 -- 172.21.15.97:0/2258458387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d400ba00 con 0x7f04e8160180 2023-07-06T03:47:41.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.291+0000 7f04e77fe700 1 -- 172.21.15.97:0/2258458387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d400bd90 con 0x7f04e8160180 2023-07-06T03:47:41.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.291+0000 7f04ef1ff700 1 -- 172.21.15.97:0/2258458387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04e8160180 msgr2=0x7f04e81605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.291+0000 7f04ef1ff700 1 --2- 172.21.15.97:0/2258458387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04e8160180 0x7f04e81605f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f04d40098f0 tx=0x7f04d4009c00 comp rx=0 tx=0).stop 2023-07-06T03:47:41.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.292+0000 7f04ef1ff700 1 -- 172.21.15.97:0/2258458387 shutdown_connections 2023-07-06T03:47:41.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.292+0000 7f04ef1ff700 1 --2- 172.21.15.97:0/2258458387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04e815c520 0x7f04e80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.292+0000 7f04ef1ff700 1 --2- 172.21.15.97:0/2258458387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04e8160180 0x7f04e81605f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.292+0000 7f04ef1ff700 1 -- 172.21.15.97:0/2258458387 >> 172.21.15.97:0/2258458387 conn(0x7f04e8071790 msgr2=0x7f04e8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:41.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.292+0000 7f04ef1ff700 1 -- 172.21.15.97:0/2258458387 shutdown_connections 2023-07-06T03:47:41.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.292+0000 7f04ef1ff700 1 -- 172.21.15.97:0/2258458387 wait complete. 2023-07-06T03:47:41.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.294+0000 7f04ef1ff700 1 Processor -- start 2023-07-06T03:47:41.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.294+0000 7f04ef1ff700 1 -- start start 2023-07-06T03:47:41.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.295+0000 7f04ef1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04e815c520 0x7f04e80d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.295+0000 7f04ef1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04e8160180 0x7f04e8154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.295+0000 7f04ef1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8160b30 0x7f04e81e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.295+0000 7f04ef1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e8155cc0 con 0x7f04e815c520 2023-07-06T03:47:41.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.295+0000 7f04ef1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e815f200 con 0x7f04e8160b30 2023-07-06T03:47:41.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.295+0000 7f04ef1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e80d18d0 con 0x7f04e8160180 2023-07-06T03:47:41.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04ecf9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04e815c520 0x7f04e80d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04ecf9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04e815c520 0x7f04e80d36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33166/0 (socket says 172.21.15.97:33166) 2023-07-06T03:47:41.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04ecf9b700 1 -- 172.21.15.97:0/1802018415 learned_addr learned my addr 172.21.15.97:0/1802018415 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:41.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04e7fff700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04e8160180 0x7f04e8154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04ed79c700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8160b30 0x7f04e81e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04ecf9b700 1 -- 172.21.15.97:0/1802018415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04e8160180 msgr2=0x7f04e8154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04ecf9b700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04e8160180 0x7f04e8154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04ecf9b700 1 -- 172.21.15.97:0/1802018415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8160b30 msgr2=0x7f04e81e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04ecf9b700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8160b30 0x7f04e81e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04ecf9b700 1 -- 172.21.15.97:0/1802018415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d4009550 con 0x7f04e815c520 2023-07-06T03:47:41.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04ed79c700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8160b30 0x7f04e81e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:41.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.296+0000 7f04ecf9b700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04e815c520 0x7f04e80d36d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00e760 tx=0x7f04dc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:41.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.297+0000 7f04e57fa700 1 -- 172.21.15.97:0/1802018415 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc013610 con 0x7f04e815c520 2023-07-06T03:47:41.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.297+0000 7f04e57fa700 1 -- 172.21.15.97:0/1802018415 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04dc013770 con 0x7f04e815c520 2023-07-06T03:47:41.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.297+0000 7f04ef1ff700 1 -- 172.21.15.97:0/1802018415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e80d3c10 con 0x7f04e815c520 2023-07-06T03:47:41.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.297+0000 7f04e57fa700 1 -- 172.21.15.97:0/1802018415 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc013b20 con 0x7f04e815c520 2023-07-06T03:47:41.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.297+0000 7f04ef1ff700 1 -- 172.21.15.97:0/1802018415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e81559e0 con 0x7f04e815c520 2023-07-06T03:47:41.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.298+0000 7f04ef1ff700 1 -- 172.21.15.97:0/1802018415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e80d13c0 con 0x7f04e815c520 2023-07-06T03:47:41.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.299+0000 7f04e57fa700 1 -- 172.21.15.97:0/1802018415 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f04dc013c80 con 0x7f04e815c520 2023-07-06T03:47:41.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.300+0000 7f04e57fa700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f04cc06ace0 0x7f04cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.300+0000 7f04e57fa700 1 -- 172.21.15.97:0/1802018415 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f04dc089160 con 0x7f04e815c520 2023-07-06T03:47:41.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.300+0000 7f04e7fff700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f04cc06ace0 0x7f04cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.301+0000 7f04e7fff700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f04cc06ace0 0x7f04cc06d190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f04d4014040 tx=0x7f04d4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:41.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.304+0000 7f04e57fa700 1 -- 172.21.15.97:0/1802018415 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f04e80d13c0 con 0x7f04e815c520 2023-07-06T03:47:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.496+0000 7f04ef1ff700 1 -- 172.21.15.97:0/1802018415 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04e80b6ee0 con 0x7f04cc06ace0 2023-07-06T03:47:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.504+0000 7f04e57fa700 1 -- 172.21.15.97:0/1802018415 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+305841 (secure 0 0 0) 0x7f04d4006c60 con 0x7f04cc06ace0 2023-07-06T03:47:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:41.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.508+0000 7f04ef1ff700 1 -- 172.21.15.97:0/1802018415 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f04cc06ace0 msgr2=0x7f04cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.508+0000 7f04ef1ff700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f04cc06ace0 0x7f04cc06d190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f04d4014040 tx=0x7f04d4008030 comp rx=0 tx=0).stop 2023-07-06T03:47:41.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.508+0000 7f04ef1ff700 1 -- 172.21.15.97:0/1802018415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04e815c520 msgr2=0x7f04e80d36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.508+0000 7f04ef1ff700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04e815c520 0x7f04e80d36d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00e760 tx=0x7f04dc00eb20 comp rx=0 tx=0).stop 2023-07-06T03:47:41.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.509+0000 7f04ef1ff700 1 -- 172.21.15.97:0/1802018415 shutdown_connections 2023-07-06T03:47:41.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.509+0000 7f04ef1ff700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f04cc06ace0 0x7f04cc06d190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.509+0000 7f04ef1ff700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8160b30 0x7f04e81e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.509+0000 7f04ef1ff700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04e815c520 0x7f04e80d36d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.509+0000 7f04ef1ff700 1 --2- 172.21.15.97:0/1802018415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04e8160180 0x7f04e8154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.509+0000 7f04ef1ff700 1 -- 172.21.15.97:0/1802018415 >> 172.21.15.97:0/1802018415 conn(0x7f04e8071790 msgr2=0x7f04e80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:41.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.509+0000 7f04ef1ff700 1 -- 172.21.15.97:0/1802018415 shutdown_connections 2023-07-06T03:47:41.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.509+0000 7f04ef1ff700 1 -- 172.21.15.97:0/1802018415 wait complete. 2023-07-06T03:47:41.513 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:47:41.528 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-07-06T03:47:40.912045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249488,"kb_used_data":5200,"kb_used_omap":0,"kb_used_meta":244224,"kb_avail":754725232,"statfs":{"total":773094113280,"available":772838637568,"internally_reserved":0,"allocated":5324800,"data_stored":2919489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":250085376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001014"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671120+0000","last_change":"2023-07-06T03:47:35.671120+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671120+0000","last_undegraded":"2023-07-06T03:47:35.671120+0000","last_fullsized":"2023-07-06T03:47:35.671120+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668727+0000","last_change":"2023-07-06T03:47:35.668727+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668727+0000","last_undegraded":"2023-07-06T03:47:35.668727+0000","last_fullsized":"2023-07-06T03:47:35.668727+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664247+0000","last_change":"2023-07-06T03:47:35.664247+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664247+0000","last_undegraded":"2023-07-06T03:47:35.664247+0000","last_fullsized":"2023-07-06T03:47:35.664247+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664482+0000","last_change":"2023-07-06T03:47:35.664482+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664482+0000","last_undegraded":"2023-07-06T03:47:35.664482+0000","last_fullsized":"2023-07-06T03:47:35.664482+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668927+0000","last_change":"2023-07-06T03:47:35.668927+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668927+0000","last_undegraded":"2023-07-06T03:47:35.668927+0000","last_fullsized":"2023-07-06T03:47:35.668927+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665583+0000","last_change":"2023-07-06T03:47:35.665583+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665583+0000","last_undegraded":"2023-07-06T03:47:35.665583+0000","last_fullsized":"2023-07-06T03:47:35.665583+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664339+0000","last_change":"2023-07-06T03:47:35.664339+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664339+0000","last_undegraded":"2023-07-06T03:47:35.664339+0000","last_fullsized":"2023-07-06T03:47:35.664339+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665496+0000","last_change":"2023-07-06T03:47:35.665496+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665496+0000","last_undegraded":"2023-07-06T03:47:35.665496+0000","last_fullsized":"2023-07-06T03:47:35.665496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663507+0000","last_change":"2023-07-06T03:47:35.663507+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663507+0000","last_undegraded":"2023-07-06T03:47:35.663507+0000","last_fullsized":"2023-07-06T03:47:35.663507+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664563+0000","last_change":"2023-07-06T03:47:35.664563+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664563+0000","last_undegraded":"2023-07-06T03:47:35.664563+0000","last_fullsized":"2023-07-06T03:47:35.664563+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669176+0000","last_change":"2023-07-06T03:47:35.669176+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669176+0000","last_undegraded":"2023-07-06T03:47:35.669176+0000","last_fullsized":"2023-07-06T03:47:35.669176+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667342+0000","last_change":"2023-07-06T03:47:35.667342+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667342+0000","last_undegraded":"2023-07-06T03:47:35.667342+0000","last_fullsized":"2023-07-06T03:47:35.667342+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666058+0000","last_change":"2023-07-06T03:47:35.666058+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666058+0000","last_undegraded":"2023-07-06T03:47:35.666058+0000","last_fullsized":"2023-07-06T03:47:35.666058+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670675+0000","last_change":"2023-07-06T03:47:35.670675+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670675+0000","last_undegraded":"2023-07-06T03:47:35.670675+0000","last_fullsized":"2023-07-06T03:47:35.670675+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665278+0000","last_change":"2023-07-06T03:47:35.665278+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665278+0000","last_undegraded":"2023-07-06T03:47:35.665278+0000","last_fullsized":"2023-07-06T03:47:35.665278+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664921+0000","last_change":"2023-07-06T03:47:35.664921+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664921+0000","last_undegraded":"2023-07-06T03:47:35.664921+0000","last_fullsized":"2023-07-06T03:47:35.664921+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670602+0000","last_change":"2023-07-06T03:47:35.670602+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670602+0000","last_undegraded":"2023-07-06T03:47:35.670602+0000","last_fullsized":"2023-07-06T03:47:35.670602+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664731+0000","last_change":"2023-07-06T03:47:35.664731+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664731+0000","last_undegraded":"2023-07-06T03:47:35.664731+0000","last_fullsized":"2023-07-06T03:47:35.664731+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663759+0000","last_change":"2023-07-06T03:47:35.663759+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663759+0000","last_undegraded":"2023-07-06T03:47:35.663759+0000","last_fullsized":"2023-07-06T03:47:35.663759+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670330+0000","last_change":"2023-07-06T03:47:35.670330+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670330+0000","last_undegraded":"2023-07-06T03:47:35.670330+0000","last_fullsized":"2023-07-06T03:47:35.670330+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664418+0000","last_change":"2023-07-06T03:47:35.664418+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664418+0000","last_undegraded":"2023-07-06T03:47:35.664418+0000","last_fullsized":"2023-07-06T03:47:35.664418+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663209+0000","last_change":"2023-07-06T03:47:35.663209+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663209+0000","last_undegraded":"2023-07-06T03:47:35.663209+0000","last_fullsized":"2023-07-06T03:47:35.663209+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662599+0000","last_change":"2023-07-06T03:47:35.662599+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662599+0000","last_undegraded":"2023-07-06T03:47:35.662599+0000","last_fullsized":"2023-07-06T03:47:35.662599+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666150+0000","last_change":"2023-07-06T03:47:35.666150+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666150+0000","last_undegraded":"2023-07-06T03:47:35.666150+0000","last_fullsized":"2023-07-06T03:47:35.666150+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670081+0000","last_change":"2023-07-06T03:47:35.670081+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670081+0000","last_undegraded":"2023-07-06T03:47:35.670081+0000","last_fullsized":"2023-07-06T03:47:35.670081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664042+0000","last_change":"2023-07-06T03:47:35.664042+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664042+0000","last_undegraded":"2023-07-06T03:47:35.664042+0000","last_fullsized":"2023-07-06T03:47:35.664042+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667395+0000","last_change":"2023-07-06T03:47:35.667395+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667395+0000","last_undegraded":"2023-07-06T03:47:35.667395+0000","last_fullsized":"2023-07-06T03:47:35.667395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669999+0000","last_change":"2023-07-06T03:47:35.669999+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669999+0000","last_undegraded":"2023-07-06T03:47:35.669999+0000","last_fullsized":"2023-07-06T03:47:35.669999+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.674695+0000","last_change":"2023-07-06T03:47:35.674695+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.674695+0000","last_undegraded":"2023-07-06T03:47:35.674695+0000","last_fullsized":"2023-07-06T03:47:35.674695+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668319+0000","last_change":"2023-07-06T03:47:35.668319+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668319+0000","last_undegraded":"2023-07-06T03:47:35.668319+0000","last_fullsized":"2023-07-06T03:47:35.668319+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662011+0000","last_change":"2023-07-06T03:47:35.662011+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662011+0000","last_undegraded":"2023-07-06T03:47:35.662011+0000","last_fullsized":"2023-07-06T03:47:35.662011+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668114+0000","last_change":"2023-07-06T03:47:35.668114+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668114+0000","last_undegraded":"2023-07-06T03:47:35.668114+0000","last_fullsized":"2023-07-06T03:47:35.668114+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666321+0000","last_change":"2023-07-06T03:47:35.666321+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666321+0000","last_undegraded":"2023-07-06T03:47:35.666321+0000","last_fullsized":"2023-07-06T03:47:35.666321+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686079+0000","last_change":"2023-07-06T03:47:35.684675+0000","last_active":"2023-07-06T03:47:35.686079+0000","last_peered":"2023-07-06T03:47:35.686079+0000","last_clean":"2023-07-06T03:47:35.686079+0000","last_became_active":"2023-07-06T03:47:35.684398+0000","last_became_peered":"2023-07-06T03:47:35.684398+0000","last_unstale":"2023-07-06T03:47:35.686079+0000","last_undegraded":"2023-07-06T03:47:35.686079+0000","last_fullsized":"2023-07-06T03:47:35.686079+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:47.031506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666528+0000","last_change":"2023-07-06T03:47:35.666528+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666528+0000","last_undegraded":"2023-07-06T03:47:35.666528+0000","last_fullsized":"2023-07-06T03:47:35.666528+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682319+0000","last_change":"2023-07-06T03:47:35.680067+0000","last_active":"2023-07-06T03:47:35.682319+0000","last_peered":"2023-07-06T03:47:35.682319+0000","last_clean":"2023-07-06T03:47:35.682319+0000","last_became_active":"2023-07-06T03:47:35.679691+0000","last_became_peered":"2023-07-06T03:47:35.679691+0000","last_unstale":"2023-07-06T03:47:35.682319+0000","last_undegraded":"2023-07-06T03:47:35.682319+0000","last_fullsized":"2023-07-06T03:47:35.682319+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:14.005662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667029+0000","last_change":"2023-07-06T03:47:35.667029+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667029+0000","last_undegraded":"2023-07-06T03:47:35.667029+0000","last_fullsized":"2023-07-06T03:47:35.667029+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684178+0000","last_change":"2023-07-06T03:47:35.676405+0000","last_active":"2023-07-06T03:47:35.684178+0000","last_peered":"2023-07-06T03:47:35.684178+0000","last_clean":"2023-07-06T03:47:35.684178+0000","last_became_active":"2023-07-06T03:47:35.676089+0000","last_became_peered":"2023-07-06T03:47:35.676089+0000","last_unstale":"2023-07-06T03:47:35.684178+0000","last_undegraded":"2023-07-06T03:47:35.684178+0000","last_fullsized":"2023-07-06T03:47:35.684178+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:58:51.450312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664710+0000","last_change":"2023-07-06T03:47:35.664710+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664710+0000","last_undegraded":"2023-07-06T03:47:35.664710+0000","last_fullsized":"2023-07-06T03:47:35.664710+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682615+0000","last_change":"2023-07-06T03:47:35.678850+0000","last_active":"2023-07-06T03:47:35.682615+0000","last_peered":"2023-07-06T03:47:35.682615+0000","last_clean":"2023-07-06T03:47:35.682615+0000","last_became_active":"2023-07-06T03:47:35.678520+0000","last_became_peered":"2023-07-06T03:47:35.678520+0000","last_unstale":"2023-07-06T03:47:35.682615+0000","last_undegraded":"2023-07-06T03:47:35.682615+0000","last_fullsized":"2023-07-06T03:47:35.682615+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:08:14.846337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671834+0000","last_change":"2023-07-06T03:47:35.671834+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671834+0000","last_undegraded":"2023-07-06T03:47:35.671834+0000","last_fullsized":"2023-07-06T03:47:35.671834+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.683654+0000","last_change":"2023-07-06T03:47:35.679264+0000","last_active":"2023-07-06T03:47:35.683654+0000","last_peered":"2023-07-06T03:47:35.683654+0000","last_clean":"2023-07-06T03:47:35.683654+0000","last_became_active":"2023-07-06T03:47:35.678950+0000","last_became_peered":"2023-07-06T03:47:35.678950+0000","last_unstale":"2023-07-06T03:47:35.683654+0000","last_undegraded":"2023-07-06T03:47:35.683654+0000","last_fullsized":"2023-07-06T03:47:35.683654+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:11:09.395143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667838+0000","last_change":"2023-07-06T03:47:35.667838+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667838+0000","last_undegraded":"2023-07-06T03:47:35.667838+0000","last_fullsized":"2023-07-06T03:47:35.667838+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681981+0000","last_change":"2023-07-06T03:47:35.679463+0000","last_active":"2023-07-06T03:47:35.681981+0000","last_peered":"2023-07-06T03:47:35.681981+0000","last_clean":"2023-07-06T03:47:35.681981+0000","last_became_active":"2023-07-06T03:47:35.679249+0000","last_became_peered":"2023-07-06T03:47:35.679249+0000","last_unstale":"2023-07-06T03:47:35.681981+0000","last_undegraded":"2023-07-06T03:47:35.681981+0000","last_fullsized":"2023-07-06T03:47:35.681981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:48:41.016578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665066+0000","last_change":"2023-07-06T03:47:35.665066+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665066+0000","last_undegraded":"2023-07-06T03:47:35.665066+0000","last_fullsized":"2023-07-06T03:47:35.665066+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686346+0000","last_change":"2023-07-06T03:47:35.684332+0000","last_active":"2023-07-06T03:47:35.686346+0000","last_peered":"2023-07-06T03:47:35.686346+0000","last_clean":"2023-07-06T03:47:35.686346+0000","last_became_active":"2023-07-06T03:47:35.684064+0000","last_became_peered":"2023-07-06T03:47:35.684064+0000","last_unstale":"2023-07-06T03:47:35.686346+0000","last_undegraded":"2023-07-06T03:47:35.686346+0000","last_fullsized":"2023-07-06T03:47:35.686346+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:45:08.107109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664510+0000","last_change":"2023-07-06T03:47:35.664510+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664510+0000","last_undegraded":"2023-07-06T03:47:35.664510+0000","last_fullsized":"2023-07-06T03:47:35.664510+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684604+0000","last_change":"2023-07-06T03:47:35.679127+0000","last_active":"2023-07-06T03:47:35.684604+0000","last_peered":"2023-07-06T03:47:35.684604+0000","last_clean":"2023-07-06T03:47:35.684604+0000","last_became_active":"2023-07-06T03:47:35.678630+0000","last_became_peered":"2023-07-06T03:47:35.678630+0000","last_unstale":"2023-07-06T03:47:35.684604+0000","last_undegraded":"2023-07-06T03:47:35.684604+0000","last_fullsized":"2023-07-06T03:47:35.684604+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:26:20.114823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671405+0000","last_change":"2023-07-06T03:47:35.671405+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671405+0000","last_undegraded":"2023-07-06T03:47:35.671405+0000","last_fullsized":"2023-07-06T03:47:35.671405+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.680869+0000","last_change":"2023-07-06T03:47:35.678328+0000","last_active":"2023-07-06T03:47:35.680869+0000","last_peered":"2023-07-06T03:47:35.680869+0000","last_clean":"2023-07-06T03:47:35.680869+0000","last_became_active":"2023-07-06T03:47:35.678081+0000","last_became_peered":"2023-07-06T03:47:35.678081+0000","last_unstale":"2023-07-06T03:47:35.680869+0000","last_undegraded":"2023-07-06T03:47:35.680869+0000","last_fullsized":"2023-07-06T03:47:35.680869+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:24:18.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664273+0000","last_change":"2023-07-06T03:47:35.664273+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664273+0000","last_undegraded":"2023-07-06T03:47:35.664273+0000","last_fullsized":"2023-07-06T03:47:35.664273+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.679689+0000","last_change":"2023-07-06T03:47:35.673408+0000","last_active":"2023-07-06T03:47:35.679689+0000","last_peered":"2023-07-06T03:47:35.679689+0000","last_clean":"2023-07-06T03:47:35.679689+0000","last_became_active":"2023-07-06T03:47:35.672993+0000","last_became_peered":"2023-07-06T03:47:35.672993+0000","last_unstale":"2023-07-06T03:47:35.679689+0000","last_undegraded":"2023-07-06T03:47:35.679689+0000","last_fullsized":"2023-07-06T03:47:35.679689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:56:55.279062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670077+0000","last_change":"2023-07-06T03:47:35.670077+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670077+0000","last_undegraded":"2023-07-06T03:47:35.670077+0000","last_fullsized":"2023-07-06T03:47:35.670077+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681921+0000","last_change":"2023-07-06T03:47:35.679946+0000","last_active":"2023-07-06T03:47:35.681921+0000","last_peered":"2023-07-06T03:47:35.681921+0000","last_clean":"2023-07-06T03:47:35.681921+0000","last_became_active":"2023-07-06T03:47:35.679487+0000","last_became_peered":"2023-07-06T03:47:35.679487+0000","last_unstale":"2023-07-06T03:47:35.681921+0000","last_undegraded":"2023-07-06T03:47:35.681921+0000","last_fullsized":"2023-07-06T03:47:35.681921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:52.953019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669355+0000","last_change":"2023-07-06T03:47:35.669355+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669355+0000","last_undegraded":"2023-07-06T03:47:35.669355+0000","last_fullsized":"2023-07-06T03:47:35.669355+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686839+0000","last_change":"2023-07-06T03:47:35.679499+0000","last_active":"2023-07-06T03:47:35.686839+0000","last_peered":"2023-07-06T03:47:35.686839+0000","last_clean":"2023-07-06T03:47:35.686839+0000","last_became_active":"2023-07-06T03:47:35.679153+0000","last_became_peered":"2023-07-06T03:47:35.679153+0000","last_unstale":"2023-07-06T03:47:35.686839+0000","last_undegraded":"2023-07-06T03:47:35.686839+0000","last_fullsized":"2023-07-06T03:47:35.686839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:28:35.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667017+0000","last_change":"2023-07-06T03:47:35.667017+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667017+0000","last_undegraded":"2023-07-06T03:47:35.667017+0000","last_fullsized":"2023-07-06T03:47:35.667017+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.676626+0000","last_change":"2023-07-06T03:47:35.675074+0000","last_active":"2023-07-06T03:47:35.676626+0000","last_peered":"2023-07-06T03:47:35.676626+0000","last_clean":"2023-07-06T03:47:35.676626+0000","last_became_active":"2023-07-06T03:47:35.674815+0000","last_became_peered":"2023-07-06T03:47:35.674815+0000","last_unstale":"2023-07-06T03:47:35.676626+0000","last_undegraded":"2023-07-06T03:47:35.676626+0000","last_fullsized":"2023-07-06T03:47:35.676626+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:44:22.676309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669675+0000","last_change":"2023-07-06T03:47:35.669675+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669675+0000","last_undegraded":"2023-07-06T03:47:35.669675+0000","last_fullsized":"2023-07-06T03:47:35.669675+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681758+0000","last_change":"2023-07-06T03:47:35.680141+0000","last_active":"2023-07-06T03:47:35.681758+0000","last_peered":"2023-07-06T03:47:35.681758+0000","last_clean":"2023-07-06T03:47:35.681758+0000","last_became_active":"2023-07-06T03:47:35.679775+0000","last_became_peered":"2023-07-06T03:47:35.679775+0000","last_unstale":"2023-07-06T03:47:35.681758+0000","last_undegraded":"2023-07-06T03:47:35.681758+0000","last_fullsized":"2023-07-06T03:47:35.681758+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:08:52.125995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668183+0000","last_change":"2023-07-06T03:47:35.668183+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668183+0000","last_undegraded":"2023-07-06T03:47:35.668183+0000","last_fullsized":"2023-07-06T03:47:35.668183+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682381+0000","last_change":"2023-07-06T03:47:35.676310+0000","last_active":"2023-07-06T03:47:35.682381+0000","last_peered":"2023-07-06T03:47:35.682381+0000","last_clean":"2023-07-06T03:47:35.682381+0000","last_became_active":"2023-07-06T03:47:35.676038+0000","last_became_peered":"2023-07-06T03:47:35.676038+0000","last_unstale":"2023-07-06T03:47:35.682381+0000","last_undegraded":"2023-07-06T03:47:35.682381+0000","last_fullsized":"2023-07-06T03:47:35.682381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:02:33.045576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663051+0000","last_change":"2023-07-06T03:47:35.663051+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663051+0000","last_undegraded":"2023-07-06T03:47:35.663051+0000","last_fullsized":"2023-07-06T03:47:35.663051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686476+0000","last_change":"2023-07-06T03:47:35.683270+0000","last_active":"2023-07-06T03:47:35.686476+0000","last_peered":"2023-07-06T03:47:35.686476+0000","last_clean":"2023-07-06T03:47:35.686476+0000","last_became_active":"2023-07-06T03:47:35.682973+0000","last_became_peered":"2023-07-06T03:47:35.682973+0000","last_unstale":"2023-07-06T03:47:35.686476+0000","last_undegraded":"2023-07-06T03:47:35.686476+0000","last_fullsized":"2023-07-06T03:47:35.686476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:03:08.897884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662280+0000","last_change":"2023-07-06T03:47:35.662280+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662280+0000","last_undegraded":"2023-07-06T03:47:35.662280+0000","last_fullsized":"2023-07-06T03:47:35.662280+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682343+0000","last_change":"2023-07-06T03:47:35.680067+0000","last_active":"2023-07-06T03:47:35.682343+0000","last_peered":"2023-07-06T03:47:35.682343+0000","last_clean":"2023-07-06T03:47:35.682343+0000","last_became_active":"2023-07-06T03:47:35.679704+0000","last_became_peered":"2023-07-06T03:47:35.679704+0000","last_unstale":"2023-07-06T03:47:35.682343+0000","last_undegraded":"2023-07-06T03:47:35.682343+0000","last_fullsized":"2023-07-06T03:47:35.682343+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:31:03.309257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670261+0000","last_change":"2023-07-06T03:47:35.670261+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670261+0000","last_undegraded":"2023-07-06T03:47:35.670261+0000","last_fullsized":"2023-07-06T03:47:35.670261+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686133+0000","last_change":"2023-07-06T03:47:35.680931+0000","last_active":"2023-07-06T03:47:35.686133+0000","last_peered":"2023-07-06T03:47:35.686133+0000","last_clean":"2023-07-06T03:47:35.686133+0000","last_became_active":"2023-07-06T03:47:35.680659+0000","last_became_peered":"2023-07-06T03:47:35.680659+0000","last_unstale":"2023-07-06T03:47:35.686133+0000","last_undegraded":"2023-07-06T03:47:35.686133+0000","last_fullsized":"2023-07-06T03:47:35.686133+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:01:39.848129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667702+0000","last_change":"2023-07-06T03:47:35.667702+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667702+0000","last_undegraded":"2023-07-06T03:47:35.667702+0000","last_fullsized":"2023-07-06T03:47:35.667702+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681731+0000","last_change":"2023-07-06T03:47:35.679503+0000","last_active":"2023-07-06T03:47:35.681731+0000","last_peered":"2023-07-06T03:47:35.681731+0000","last_clean":"2023-07-06T03:47:35.681731+0000","last_became_active":"2023-07-06T03:47:35.679231+0000","last_became_peered":"2023-07-06T03:47:35.679231+0000","last_unstale":"2023-07-06T03:47:35.681731+0000","last_undegraded":"2023-07-06T03:47:35.681731+0000","last_fullsized":"2023-07-06T03:47:35.681731+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:08:16.691892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665577+0000","last_change":"2023-07-06T03:47:35.665577+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665577+0000","last_undegraded":"2023-07-06T03:47:35.665577+0000","last_fullsized":"2023-07-06T03:47:35.665577+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.687009+0000","last_change":"2023-07-06T03:47:35.685943+0000","last_active":"2023-07-06T03:47:35.687009+0000","last_peered":"2023-07-06T03:47:35.687009+0000","last_clean":"2023-07-06T03:47:35.687009+0000","last_became_active":"2023-07-06T03:47:35.685673+0000","last_became_peered":"2023-07-06T03:47:35.685673+0000","last_unstale":"2023-07-06T03:47:35.687009+0000","last_undegraded":"2023-07-06T03:47:35.687009+0000","last_fullsized":"2023-07-06T03:47:35.687009+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:14:16.989542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669618+0000","last_change":"2023-07-06T03:47:35.669618+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669618+0000","last_undegraded":"2023-07-06T03:47:35.669618+0000","last_fullsized":"2023-07-06T03:47:35.669618+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.689703+0000","last_change":"2023-07-06T03:47:35.684207+0000","last_active":"2023-07-06T03:47:35.689703+0000","last_peered":"2023-07-06T03:47:35.689703+0000","last_clean":"2023-07-06T03:47:35.689703+0000","last_became_active":"2023-07-06T03:47:35.683837+0000","last_became_peered":"2023-07-06T03:47:35.683837+0000","last_unstale":"2023-07-06T03:47:35.689703+0000","last_undegraded":"2023-07-06T03:47:35.689703+0000","last_fullsized":"2023-07-06T03:47:35.689703+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:11.065681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664327+0000","last_change":"2023-07-06T03:47:35.664327+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664327+0000","last_undegraded":"2023-07-06T03:47:35.664327+0000","last_fullsized":"2023-07-06T03:47:35.664327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684585+0000","last_change":"2023-07-06T03:47:35.680503+0000","last_active":"2023-07-06T03:47:35.684585+0000","last_peered":"2023-07-06T03:47:35.684585+0000","last_clean":"2023-07-06T03:47:35.684585+0000","last_became_active":"2023-07-06T03:47:35.680171+0000","last_became_peered":"2023-07-06T03:47:35.680171+0000","last_unstale":"2023-07-06T03:47:35.684585+0000","last_undegraded":"2023-07-06T03:47:35.684585+0000","last_fullsized":"2023-07-06T03:47:35.684585+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:07:12.686190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.675202+0000","last_change":"2023-07-06T03:47:35.675202+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.675202+0000","last_undegraded":"2023-07-06T03:47:35.675202+0000","last_fullsized":"2023-07-06T03:47:35.675202+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686872+0000","last_change":"2023-07-06T03:47:35.680421+0000","last_active":"2023-07-06T03:47:35.686872+0000","last_peered":"2023-07-06T03:47:35.686872+0000","last_clean":"2023-07-06T03:47:35.686872+0000","last_became_active":"2023-07-06T03:47:35.679465+0000","last_became_peered":"2023-07-06T03:47:35.679465+0000","last_unstale":"2023-07-06T03:47:35.686872+0000","last_undegraded":"2023-07-06T03:47:35.686872+0000","last_fullsized":"2023-07-06T03:47:35.686872+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:22.389725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667496+0000","last_change":"2023-07-06T03:47:35.667496+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667496+0000","last_undegraded":"2023-07-06T03:47:35.667496+0000","last_fullsized":"2023-07-06T03:47:35.667496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.683955+0000","last_change":"2023-07-06T03:47:35.673523+0000","last_active":"2023-07-06T03:47:35.683955+0000","last_peered":"2023-07-06T03:47:35.683955+0000","last_clean":"2023-07-06T03:47:35.683955+0000","last_became_active":"2023-07-06T03:47:35.672470+0000","last_became_peered":"2023-07-06T03:47:35.672470+0000","last_unstale":"2023-07-06T03:47:35.683955+0000","last_undegraded":"2023-07-06T03:47:35.683955+0000","last_fullsized":"2023-07-06T03:47:35.683955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:23:29.539377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662836+0000","last_change":"2023-07-06T03:47:35.662836+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662836+0000","last_undegraded":"2023-07-06T03:47:35.662836+0000","last_fullsized":"2023-07-06T03:47:35.662836+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682665+0000","last_change":"2023-07-06T03:47:35.681014+0000","last_active":"2023-07-06T03:47:35.682665+0000","last_peered":"2023-07-06T03:47:35.682665+0000","last_clean":"2023-07-06T03:47:35.682665+0000","last_became_active":"2023-07-06T03:47:35.680766+0000","last_became_peered":"2023-07-06T03:47:35.680766+0000","last_unstale":"2023-07-06T03:47:35.682665+0000","last_undegraded":"2023-07-06T03:47:35.682665+0000","last_fullsized":"2023-07-06T03:47:35.682665+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:11:00.458188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.665633+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:47:35.665633+0000","last_peered":"2023-07-06T03:47:35.665633+0000","last_clean":"2023-07-06T03:47:35.665633+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:47:35.665633+0000","last_undegraded":"2023-07-06T03:47:35.665633+0000","last_fullsized":"2023-07-06T03:47:35.665633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:25:11.511422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664761+0000","last_change":"2023-07-06T03:47:35.664761+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664761+0000","last_undegraded":"2023-07-06T03:47:35.664761+0000","last_fullsized":"2023-07-06T03:47:35.664761+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686329+0000","last_change":"2023-07-06T03:47:35.684268+0000","last_active":"2023-07-06T03:47:35.686329+0000","last_peered":"2023-07-06T03:47:35.686329+0000","last_clean":"2023-07-06T03:47:35.686329+0000","last_became_active":"2023-07-06T03:47:35.683961+0000","last_became_peered":"2023-07-06T03:47:35.683961+0000","last_unstale":"2023-07-06T03:47:35.686329+0000","last_undegraded":"2023-07-06T03:47:35.686329+0000","last_fullsized":"2023-07-06T03:47:35.686329+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:46:49.441174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.668531+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:47:35.668531+0000","last_peered":"2023-07-06T03:47:35.668531+0000","last_clean":"2023-07-06T03:47:35.668531+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:47:35.668531+0000","last_undegraded":"2023-07-06T03:47:35.668531+0000","last_fullsized":"2023-07-06T03:47:35.668531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:40:54.630791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665474+0000","last_change":"2023-07-06T03:47:35.665474+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665474+0000","last_undegraded":"2023-07-06T03:47:35.665474+0000","last_fullsized":"2023-07-06T03:47:35.665474+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.683642+0000","last_change":"2023-07-06T03:47:35.677576+0000","last_active":"2023-07-06T03:47:35.683642+0000","last_peered":"2023-07-06T03:47:35.683642+0000","last_clean":"2023-07-06T03:47:35.683642+0000","last_became_active":"2023-07-06T03:47:35.677247+0000","last_became_peered":"2023-07-06T03:47:35.677247+0000","last_unstale":"2023-07-06T03:47:35.683642+0000","last_undegraded":"2023-07-06T03:47:35.683642+0000","last_fullsized":"2023-07-06T03:47:35.683642+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:22:57.626952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.668542+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:47:35.668542+0000","last_peered":"2023-07-06T03:47:35.668542+0000","last_clean":"2023-07-06T03:47:35.668542+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:47:35.668542+0000","last_undegraded":"2023-07-06T03:47:35.668542+0000","last_fullsized":"2023-07-06T03:47:35.668542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:55:11.004301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671814+0000","last_change":"2023-07-06T03:47:35.671814+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671814+0000","last_undegraded":"2023-07-06T03:47:35.671814+0000","last_fullsized":"2023-07-06T03:47:35.671814+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684977+0000","last_change":"2023-07-06T03:47:35.677003+0000","last_active":"2023-07-06T03:47:35.684977+0000","last_peered":"2023-07-06T03:47:35.684977+0000","last_clean":"2023-07-06T03:47:35.684977+0000","last_became_active":"2023-07-06T03:47:35.676727+0000","last_became_peered":"2023-07-06T03:47:35.676727+0000","last_unstale":"2023-07-06T03:47:35.684977+0000","last_undegraded":"2023-07-06T03:47:35.684977+0000","last_fullsized":"2023-07-06T03:47:35.684977+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:35:50.061745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.667278+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:47:35.667278+0000","last_peered":"2023-07-06T03:47:35.667278+0000","last_clean":"2023-07-06T03:47:35.667278+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:47:35.667278+0000","last_undegraded":"2023-07-06T03:47:35.667278+0000","last_fullsized":"2023-07-06T03:47:35.667278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:18:53.047698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.666895+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:47:35.666895+0000","last_peered":"2023-07-06T03:47:35.666895+0000","last_clean":"2023-07-06T03:47:35.666895+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:47:35.666895+0000","last_undegraded":"2023-07-06T03:47:35.666895+0000","last_fullsized":"2023-07-06T03:47:35.666895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:22:28.543253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664261+0000","last_change":"2023-07-06T03:47:35.664261+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664261+0000","last_undegraded":"2023-07-06T03:47:35.664261+0000","last_fullsized":"2023-07-06T03:47:35.664261+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681903+0000","last_change":"2023-07-06T03:47:35.679794+0000","last_active":"2023-07-06T03:47:35.681903+0000","last_peered":"2023-07-06T03:47:35.681903+0000","last_clean":"2023-07-06T03:47:35.681903+0000","last_became_active":"2023-07-06T03:47:35.679116+0000","last_became_peered":"2023-07-06T03:47:35.679116+0000","last_unstale":"2023-07-06T03:47:35.681903+0000","last_undegraded":"2023-07-06T03:47:35.681903+0000","last_fullsized":"2023-07-06T03:47:35.681903+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:13:53.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.666703+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:47:35.666703+0000","last_peered":"2023-07-06T03:47:35.666703+0000","last_clean":"2023-07-06T03:47:35.666703+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:47:35.666703+0000","last_undegraded":"2023-07-06T03:47:35.666703+0000","last_fullsized":"2023-07-06T03:47:35.666703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:39:52.909931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667932+0000","last_change":"2023-07-06T03:47:35.667932+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667932+0000","last_undegraded":"2023-07-06T03:47:35.667932+0000","last_fullsized":"2023-07-06T03:47:35.667932+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686114+0000","last_change":"2023-07-06T03:47:35.684944+0000","last_active":"2023-07-06T03:47:35.686114+0000","last_peered":"2023-07-06T03:47:35.686114+0000","last_clean":"2023-07-06T03:47:35.686114+0000","last_became_active":"2023-07-06T03:47:35.684654+0000","last_became_peered":"2023-07-06T03:47:35.684654+0000","last_unstale":"2023-07-06T03:47:35.686114+0000","last_undegraded":"2023-07-06T03:47:35.686114+0000","last_fullsized":"2023-07-06T03:47:35.686114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:05.886555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.668075+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:47:35.668075+0000","last_peered":"2023-07-06T03:47:35.668075+0000","last_clean":"2023-07-06T03:47:35.668075+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:47:35.668075+0000","last_undegraded":"2023-07-06T03:47:35.668075+0000","last_fullsized":"2023-07-06T03:47:35.668075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:33:23.406225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663757+0000","last_change":"2023-07-06T03:47:35.663757+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663757+0000","last_undegraded":"2023-07-06T03:47:35.663757+0000","last_fullsized":"2023-07-06T03:47:35.663757+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684244+0000","last_change":"2023-07-06T03:47:35.677115+0000","last_active":"2023-07-06T03:47:35.684244+0000","last_peered":"2023-07-06T03:47:35.684244+0000","last_clean":"2023-07-06T03:47:35.684244+0000","last_became_active":"2023-07-06T03:47:35.676826+0000","last_became_peered":"2023-07-06T03:47:35.676826+0000","last_unstale":"2023-07-06T03:47:35.684244+0000","last_undegraded":"2023-07-06T03:47:35.684244+0000","last_fullsized":"2023-07-06T03:47:35.684244+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:26.484401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.664524+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:47:35.664524+0000","last_peered":"2023-07-06T03:47:35.664524+0000","last_clean":"2023-07-06T03:47:35.664524+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:47:35.664524+0000","last_undegraded":"2023-07-06T03:47:35.664524+0000","last_fullsized":"2023-07-06T03:47:35.664524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:55:15.702636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668249+0000","last_change":"2023-07-06T03:47:35.668249+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668249+0000","last_undegraded":"2023-07-06T03:47:35.668249+0000","last_fullsized":"2023-07-06T03:47:35.668249+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684607+0000","last_change":"2023-07-06T03:47:35.680550+0000","last_active":"2023-07-06T03:47:35.684607+0000","last_peered":"2023-07-06T03:47:35.684607+0000","last_clean":"2023-07-06T03:47:35.684607+0000","last_became_active":"2023-07-06T03:47:35.680201+0000","last_became_peered":"2023-07-06T03:47:35.680201+0000","last_unstale":"2023-07-06T03:47:35.684607+0000","last_undegraded":"2023-07-06T03:47:35.684607+0000","last_fullsized":"2023-07-06T03:47:35.684607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:54:15.807837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.667832+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:47:35.667832+0000","last_peered":"2023-07-06T03:47:35.667832+0000","last_clean":"2023-07-06T03:47:35.667832+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:47:35.667832+0000","last_undegraded":"2023-07-06T03:47:35.667832+0000","last_fullsized":"2023-07-06T03:47:35.667832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:11:43.805367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":528384,"data_stored":223271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86399999999999999,"5min":1.345,"15min":1.345},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.81499999999999995,"15min":0.83699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91100000000000003,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.748,"15min":0.76300000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85799999999999998,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.91800000000000004,"15min":0.95899999999999996},"min":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.876,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.90800000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.82399999999999995,"15min":0.84699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":3.77,"15min":3.77},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88400000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95699999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.94699999999999995,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96599999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.746,"15min":0.746},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98799999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92800000000000005,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.92800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.79400000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74199999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83899999999999997,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77800000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.44900000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.874,"5min":1.119,"15min":1.119},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.76500000000000001,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.73999999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78200000000000003,"5min":1.093,"15min":1.093},"last":0.621}]},{"osd":4,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93500000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.746,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.81000000000000005,"5min":1.085,"15min":1.085},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72599999999999998,"5min":1.218,"15min":1.218},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.80200000000000005,"15min":0.81899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0269999999999999,"5min":1.198,"15min":1.198},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.84299999999999997,"15min":0.86699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.83799999999999997,"5min":1.323,"15min":1.323},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.84899999999999998,"15min":0.877},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86799999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":1110016,"data_stored":807011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58399999999999996,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.58999999999999997,"15min":0.60999999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.628,"5min":1.96,"15min":1.96},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":1.0229999999999999,"15min":1.119},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54600000000000004,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.65800000000000003,"15min":0.68500000000000005},"min":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67900000000000005,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.63400000000000001,"15min":0.65600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.752,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.59499999999999997,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.439}]},{"osd":4,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":1.2789999999999999,"15min":1.375},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86199999999999999},"min":{"1min":0.48099999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94299999999999995,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.86199999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.78500000000000003,"15min":0.80100000000000005},"min":{"1min":0.54100000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92100000000000004,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.2929999999999999,"15min":1.393},"min":{"1min":0.55900000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.874,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.79200000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.60599999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.90600000000000003,"5min":1.365,"15min":1.365},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.335,"15min":1.4370000000000001},"min":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.88500000000000001,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.84899999999999998,"15min":0.875},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.86299999999999999,"5min":2.806,"15min":2.806},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82999999999999996,"15min":0.84599999999999997},"min":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.93000000000000005,"5min":1.337,"15min":1.337},"last":0.77200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41699999999999998,"15min":0.41799999999999998},"min":{"1min":0.316,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.443,"15min":0.441},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47599999999999998,"15min":0.47999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60199999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80700000000000005,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.438}]},{"osd":3,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.38800000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74199999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":0.76300000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":1.171,"15min":1.171},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.87,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91700000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.88500000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.546,"15min":1.546},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81000000000000005,"15min":0.81799999999999995},"min":{"1min":0.59199999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96799999999999997,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":1.155,"15min":1.2290000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.95599999999999996,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35648,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336192,"statfs":{"total":96636764160,"available":96600260608,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.57299999999999995,"5min":0.626,"15min":0.626},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.52600000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.66400000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66800000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.61099999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69299999999999995,"15min":0.69399999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":1.165,"15min":1.165},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.751},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.90500000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.746,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91400000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.90500000000000003,"5min":2.157,"15min":2.157},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.61299999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88900000000000001,"5min":1.966,"15min":1.966},"last":0.81799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.65700000000000003,"15min":0.69699999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.40699999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60599999999999998,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":1.196,"15min":1.3260000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63300000000000001,"5min":15.911,"15min":15.911},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.266,"15min":1.3959999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63700000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.55600000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":1.2709999999999999,"15min":1.401},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.59699999999999998,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.442}]},{"osd":4,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84899999999999998,"5min":1.179,"15min":1.179},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.97699999999999998,"15min":1.028},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89500000000000002,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.97999999999999998,"15min":1.0269999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92700000000000005,"5min":5.077,"15min":5.077},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.0409999999999999,"15min":1.0940000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89600000000000002,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.52500000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.90200000000000002,"5min":1.143,"15min":1.143},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.995,"15min":1.0449999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96799999999999997,"5min":5.157,"15min":5.157},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.85099999999999998,"15min":0.878},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96399999999999997,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.006,"15min":1.0549999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93000000000000005,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.73999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.91500000000000004,"15min":0.95899999999999996},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94399999999999995,"5min":6.258,"15min":6.258},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.93400000000000005,"15min":0.97299999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1419999999999999,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":1.0189999999999999,"15min":1.069},"min":{"1min":0.54800000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0089999999999999,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.073,"5min":1.413,"15min":1.413},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80800000000000005,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.499},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.873,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2330000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.028,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.76700000000000002,"15min":0.77700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.58699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1105920,"data_stored":805992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.90500000000000003,"15min":0.95499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94199999999999995,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81100000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.629},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95399999999999996,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69299999999999995,"15min":0.68500000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.749},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79600000000000004,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:47:41.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:47:41.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.762+0000 7f29bc5ff700 1 Processor -- start 2023-07-06T03:47:41.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.762+0000 7f29bc5ff700 1 -- start start 2023-07-06T03:47:41.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.763+0000 7f29bc5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f29b40d00d0 0x7f29b406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.763+0000 7f29bc5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f29b406a5a0 0x7f29b40ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.763+0000 7f29bc5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b40cdc80 con 0x7f29b415c740 2023-07-06T03:47:41.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.763+0000 7f29bc5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b4071f30 con 0x7f29b40d00d0 2023-07-06T03:47:41.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.763+0000 7f29bc5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b406aa20 con 0x7f29b406a5a0 2023-07-06T03:47:41.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.763+0000 7f29b9b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f29b40d00d0 0x7f29b406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.763+0000 7f29ba39b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f29b415c740 0x7f29b4071820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46580/0 (socket says 172.21.15.97:46580) 2023-07-06T03:47:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.763+0000 7f29ba39b700 1 -- 172.21.15.97:0/1992584755 learned_addr learned my addr 172.21.15.97:0/1992584755 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.764+0000 7f29bab9c700 1 --2- 172.21.15.97:0/1992584755 >> v2:172.21.15.146:3300/0 conn(0x7f29b406a5a0 0x7f29b40ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.764+0000 7f29b9b9a700 1 -- 172.21.15.97:0/1992584755 >> v2:172.21.15.146:3300/0 conn(0x7f29b406a5a0 msgr2=0x7f29b40ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.764+0000 7f29b9b9a700 1 --2- 172.21.15.97:0/1992584755 >> v2:172.21.15.146:3300/0 conn(0x7f29b406a5a0 0x7f29b40ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.764+0000 7f29b9b9a700 1 -- 172.21.15.97:0/1992584755 >> v1:172.21.15.97:6789/0 conn(0x7f29b415c740 legacy=0x7f29b4071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.764+0000 7f29b9b9a700 1 -- 172.21.15.97:0/1992584755 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b40ce380 con 0x7f29b40d00d0 2023-07-06T03:47:41.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.764+0000 7f29bab9c700 1 --2- 172.21.15.97:0/1992584755 >> v2:172.21.15.146:3300/0 conn(0x7f29b406a5a0 0x7f29b40ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:41.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.764+0000 7f29b9b9a700 1 --2- 172.21.15.97:0/1992584755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29b40d00d0 0x7f29b406a060 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f29b406f5b0 tx=0x7f29a4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98466ccb8e09fd10 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:41.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.764+0000 7f29b9399700 1 -- 172.21.15.97:0/1992584755 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4011830 con 0x7f29b40d00d0 2023-07-06T03:47:41.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.764+0000 7f29b9399700 1 -- 172.21.15.97:0/1992584755 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a4011990 con 0x7f29b40d00d0 2023-07-06T03:47:41.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.764+0000 7f29b9399700 1 -- 172.21.15.97:0/1992584755 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4011d20 con 0x7f29b40d00d0 2023-07-06T03:47:41.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.765+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1992584755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29b40d00d0 msgr2=0x7f29b406a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.765+0000 7f29bc5ff700 1 --2- 172.21.15.97:0/1992584755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29b40d00d0 0x7f29b406a060 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f29b406f5b0 tx=0x7f29a4009b20 comp rx=0 tx=0).stop 2023-07-06T03:47:41.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.765+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1992584755 shutdown_connections 2023-07-06T03:47:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.765+0000 7f29bc5ff700 1 --2- 172.21.15.97:0/1992584755 >> v2:172.21.15.146:3300/0 conn(0x7f29b406a5a0 0x7f29b40ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.765+0000 7f29bc5ff700 1 --2- 172.21.15.97:0/1992584755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29b40d00d0 0x7f29b406a060 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.765+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1992584755 >> 172.21.15.97:0/1992584755 conn(0x7f29b40b2280 msgr2=0x7f29b40cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.765+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1992584755 shutdown_connections 2023-07-06T03:47:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.765+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1992584755 wait complete. 2023-07-06T03:47:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.767+0000 7f29bc5ff700 1 Processor -- start 2023-07-06T03:47:41.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.768+0000 7f29bc5ff700 1 -- start start 2023-07-06T03:47:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29bc5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29b406a5a0 0x7f29b41f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29bc5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29b40d00d0 0x7f29b40b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29bc5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29b415c740 0x7f29b415d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29bc5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b40b39f0 con 0x7f29b406a5a0 2023-07-06T03:47:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29bc5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b41f6090 con 0x7f29b40d00d0 2023-07-06T03:47:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29bc5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b40b5570 con 0x7f29b415c740 2023-07-06T03:47:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29ba39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29b406a5a0 0x7f29b41f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29ba39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29b406a5a0 0x7f29b41f5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33184/0 (socket says 172.21.15.97:33184) 2023-07-06T03:47:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29ba39b700 1 -- 172.21.15.97:0/1054474183 learned_addr learned my addr 172.21.15.97:0/1054474183 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29b9b9a700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29b40d00d0 0x7f29b40b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29bab9c700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29b415c740 0x7f29b415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29ba39b700 1 -- 172.21.15.97:0/1054474183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29b415c740 msgr2=0x7f29b415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29ba39b700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29b415c740 0x7f29b415d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29ba39b700 1 -- 172.21.15.97:0/1054474183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29b40d00d0 msgr2=0x7f29b40b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29ba39b700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29b40d00d0 0x7f29b40b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29ba39b700 1 -- 172.21.15.97:0/1054474183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a40094a0 con 0x7f29b406a5a0 2023-07-06T03:47:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.769+0000 7f29b9b9a700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29b40d00d0 0x7f29b40b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.770+0000 7f29ba39b700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29b406a5a0 0x7f29b41f5b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f29b000c4d0 tx=0x7f29b000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:41.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.770+0000 7f29aaffd700 1 -- 172.21.15.97:0/1054474183 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00074e0 con 0x7f29b406a5a0 2023-07-06T03:47:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.770+0000 7f29aaffd700 1 -- 172.21.15.97:0/1054474183 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0007640 con 0x7f29b406a5a0 2023-07-06T03:47:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.770+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1054474183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b40b3c40 con 0x7f29b406a5a0 2023-07-06T03:47:41.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.770+0000 7f29aaffd700 1 -- 172.21.15.97:0/1054474183 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0005a30 con 0x7f29b406a5a0 2023-07-06T03:47:41.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.770+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1054474183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b40b4200 con 0x7f29b406a5a0 2023-07-06T03:47:41.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.771+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1054474183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b404e340 con 0x7f29b406a5a0 2023-07-06T03:47:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.772+0000 7f29aaffd700 1 -- 172.21.15.97:0/1054474183 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f29b0005b90 con 0x7f29b406a5a0 2023-07-06T03:47:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.773+0000 7f29aaffd700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f299806ace0 0x7f299806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.773+0000 7f29aaffd700 1 -- 172.21.15.97:0/1054474183 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29b0088740 con 0x7f29b406a5a0 2023-07-06T03:47:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.773+0000 7f29b9b9a700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f299806ace0 0x7f299806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:41.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.774+0000 7f29b9b9a700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f299806ace0 0x7f299806d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f29a4000c00 tx=0x7f29a4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:41.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.777+0000 7f29aaffd700 1 -- 172.21.15.97:0/1054474183 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f29b0053930 con 0x7f29b406a5a0 2023-07-06T03:47:41.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.970+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1054474183 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29b40cd910 con 0x7f299806ace0 2023-07-06T03:47:41.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.978+0000 7f29aaffd700 1 -- 172.21.15.97:0/1054474183 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+305841 (secure 0 0 0) 0x7f29a4011ce0 con 0x7f299806ace0 2023-07-06T03:47:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:41.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1054474183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f299806ace0 msgr2=0x7f299806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f299806ace0 0x7f299806d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f29a4000c00 tx=0x7f29a4008030 comp rx=0 tx=0).stop 2023-07-06T03:47:41.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1054474183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29b406a5a0 msgr2=0x7f29b41f5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:41.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29b406a5a0 0x7f29b41f5b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f29b000c4d0 tx=0x7f29b000c890 comp rx=0 tx=0).stop 2023-07-06T03:47:41.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1054474183 shutdown_connections 2023-07-06T03:47:41.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f299806ace0 0x7f299806d190 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f29b415c740 0x7f29b415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29b406a5a0 0x7f29b41f5b50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 --2- 172.21.15.97:0/1054474183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29b40d00d0 0x7f29b40b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:41.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1054474183 >> 172.21.15.97:0/1054474183 conn(0x7f29b40b2280 msgr2=0x7f29b4070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:41.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1054474183 shutdown_connections 2023-07-06T03:47:41.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:41.983+0000 7f29bc5ff700 1 -- 172.21.15.97:0/1054474183 wait complete. 2023-07-06T03:47:41.987 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:47:41.997 INFO:tasks.ceph.mds.d.smithi146.stderr:2023-07-06T03:47:41.995+0000 7fe2d4bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:47:42.003 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-07-06T03:47:40.912045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249488,"kb_used_data":5200,"kb_used_omap":0,"kb_used_meta":244224,"kb_avail":754725232,"statfs":{"total":773094113280,"available":772838637568,"internally_reserved":0,"allocated":5324800,"data_stored":2919489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":250085376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001014"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671120+0000","last_change":"2023-07-06T03:47:35.671120+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671120+0000","last_undegraded":"2023-07-06T03:47:35.671120+0000","last_fullsized":"2023-07-06T03:47:35.671120+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668727+0000","last_change":"2023-07-06T03:47:35.668727+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668727+0000","last_undegraded":"2023-07-06T03:47:35.668727+0000","last_fullsized":"2023-07-06T03:47:35.668727+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664247+0000","last_change":"2023-07-06T03:47:35.664247+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664247+0000","last_undegraded":"2023-07-06T03:47:35.664247+0000","last_fullsized":"2023-07-06T03:47:35.664247+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664482+0000","last_change":"2023-07-06T03:47:35.664482+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664482+0000","last_undegraded":"2023-07-06T03:47:35.664482+0000","last_fullsized":"2023-07-06T03:47:35.664482+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668927+0000","last_change":"2023-07-06T03:47:35.668927+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668927+0000","last_undegraded":"2023-07-06T03:47:35.668927+0000","last_fullsized":"2023-07-06T03:47:35.668927+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665583+0000","last_change":"2023-07-06T03:47:35.665583+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665583+0000","last_undegraded":"2023-07-06T03:47:35.665583+0000","last_fullsized":"2023-07-06T03:47:35.665583+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664339+0000","last_change":"2023-07-06T03:47:35.664339+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664339+0000","last_undegraded":"2023-07-06T03:47:35.664339+0000","last_fullsized":"2023-07-06T03:47:35.664339+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665496+0000","last_change":"2023-07-06T03:47:35.665496+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665496+0000","last_undegraded":"2023-07-06T03:47:35.665496+0000","last_fullsized":"2023-07-06T03:47:35.665496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663507+0000","last_change":"2023-07-06T03:47:35.663507+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663507+0000","last_undegraded":"2023-07-06T03:47:35.663507+0000","last_fullsized":"2023-07-06T03:47:35.663507+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664563+0000","last_change":"2023-07-06T03:47:35.664563+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664563+0000","last_undegraded":"2023-07-06T03:47:35.664563+0000","last_fullsized":"2023-07-06T03:47:35.664563+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669176+0000","last_change":"2023-07-06T03:47:35.669176+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669176+0000","last_undegraded":"2023-07-06T03:47:35.669176+0000","last_fullsized":"2023-07-06T03:47:35.669176+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667342+0000","last_change":"2023-07-06T03:47:35.667342+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667342+0000","last_undegraded":"2023-07-06T03:47:35.667342+0000","last_fullsized":"2023-07-06T03:47:35.667342+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666058+0000","last_change":"2023-07-06T03:47:35.666058+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666058+0000","last_undegraded":"2023-07-06T03:47:35.666058+0000","last_fullsized":"2023-07-06T03:47:35.666058+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670675+0000","last_change":"2023-07-06T03:47:35.670675+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670675+0000","last_undegraded":"2023-07-06T03:47:35.670675+0000","last_fullsized":"2023-07-06T03:47:35.670675+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665278+0000","last_change":"2023-07-06T03:47:35.665278+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665278+0000","last_undegraded":"2023-07-06T03:47:35.665278+0000","last_fullsized":"2023-07-06T03:47:35.665278+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664921+0000","last_change":"2023-07-06T03:47:35.664921+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664921+0000","last_undegraded":"2023-07-06T03:47:35.664921+0000","last_fullsized":"2023-07-06T03:47:35.664921+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670602+0000","last_change":"2023-07-06T03:47:35.670602+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670602+0000","last_undegraded":"2023-07-06T03:47:35.670602+0000","last_fullsized":"2023-07-06T03:47:35.670602+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664731+0000","last_change":"2023-07-06T03:47:35.664731+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664731+0000","last_undegraded":"2023-07-06T03:47:35.664731+0000","last_fullsized":"2023-07-06T03:47:35.664731+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663759+0000","last_change":"2023-07-06T03:47:35.663759+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663759+0000","last_undegraded":"2023-07-06T03:47:35.663759+0000","last_fullsized":"2023-07-06T03:47:35.663759+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670330+0000","last_change":"2023-07-06T03:47:35.670330+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670330+0000","last_undegraded":"2023-07-06T03:47:35.670330+0000","last_fullsized":"2023-07-06T03:47:35.670330+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664418+0000","last_change":"2023-07-06T03:47:35.664418+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664418+0000","last_undegraded":"2023-07-06T03:47:35.664418+0000","last_fullsized":"2023-07-06T03:47:35.664418+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663209+0000","last_change":"2023-07-06T03:47:35.663209+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663209+0000","last_undegraded":"2023-07-06T03:47:35.663209+0000","last_fullsized":"2023-07-06T03:47:35.663209+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662599+0000","last_change":"2023-07-06T03:47:35.662599+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662599+0000","last_undegraded":"2023-07-06T03:47:35.662599+0000","last_fullsized":"2023-07-06T03:47:35.662599+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666150+0000","last_change":"2023-07-06T03:47:35.666150+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666150+0000","last_undegraded":"2023-07-06T03:47:35.666150+0000","last_fullsized":"2023-07-06T03:47:35.666150+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670081+0000","last_change":"2023-07-06T03:47:35.670081+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670081+0000","last_undegraded":"2023-07-06T03:47:35.670081+0000","last_fullsized":"2023-07-06T03:47:35.670081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664042+0000","last_change":"2023-07-06T03:47:35.664042+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664042+0000","last_undegraded":"2023-07-06T03:47:35.664042+0000","last_fullsized":"2023-07-06T03:47:35.664042+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667395+0000","last_change":"2023-07-06T03:47:35.667395+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667395+0000","last_undegraded":"2023-07-06T03:47:35.667395+0000","last_fullsized":"2023-07-06T03:47:35.667395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669999+0000","last_change":"2023-07-06T03:47:35.669999+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669999+0000","last_undegraded":"2023-07-06T03:47:35.669999+0000","last_fullsized":"2023-07-06T03:47:35.669999+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.674695+0000","last_change":"2023-07-06T03:47:35.674695+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.674695+0000","last_undegraded":"2023-07-06T03:47:35.674695+0000","last_fullsized":"2023-07-06T03:47:35.674695+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668319+0000","last_change":"2023-07-06T03:47:35.668319+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668319+0000","last_undegraded":"2023-07-06T03:47:35.668319+0000","last_fullsized":"2023-07-06T03:47:35.668319+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662011+0000","last_change":"2023-07-06T03:47:35.662011+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662011+0000","last_undegraded":"2023-07-06T03:47:35.662011+0000","last_fullsized":"2023-07-06T03:47:35.662011+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668114+0000","last_change":"2023-07-06T03:47:35.668114+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668114+0000","last_undegraded":"2023-07-06T03:47:35.668114+0000","last_fullsized":"2023-07-06T03:47:35.668114+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666321+0000","last_change":"2023-07-06T03:47:35.666321+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666321+0000","last_undegraded":"2023-07-06T03:47:35.666321+0000","last_fullsized":"2023-07-06T03:47:35.666321+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686079+0000","last_change":"2023-07-06T03:47:35.684675+0000","last_active":"2023-07-06T03:47:35.686079+0000","last_peered":"2023-07-06T03:47:35.686079+0000","last_clean":"2023-07-06T03:47:35.686079+0000","last_became_active":"2023-07-06T03:47:35.684398+0000","last_became_peered":"2023-07-06T03:47:35.684398+0000","last_unstale":"2023-07-06T03:47:35.686079+0000","last_undegraded":"2023-07-06T03:47:35.686079+0000","last_fullsized":"2023-07-06T03:47:35.686079+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:47.031506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.666528+0000","last_change":"2023-07-06T03:47:35.666528+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.666528+0000","last_undegraded":"2023-07-06T03:47:35.666528+0000","last_fullsized":"2023-07-06T03:47:35.666528+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682319+0000","last_change":"2023-07-06T03:47:35.680067+0000","last_active":"2023-07-06T03:47:35.682319+0000","last_peered":"2023-07-06T03:47:35.682319+0000","last_clean":"2023-07-06T03:47:35.682319+0000","last_became_active":"2023-07-06T03:47:35.679691+0000","last_became_peered":"2023-07-06T03:47:35.679691+0000","last_unstale":"2023-07-06T03:47:35.682319+0000","last_undegraded":"2023-07-06T03:47:35.682319+0000","last_fullsized":"2023-07-06T03:47:35.682319+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:14.005662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667029+0000","last_change":"2023-07-06T03:47:35.667029+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667029+0000","last_undegraded":"2023-07-06T03:47:35.667029+0000","last_fullsized":"2023-07-06T03:47:35.667029+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684178+0000","last_change":"2023-07-06T03:47:35.676405+0000","last_active":"2023-07-06T03:47:35.684178+0000","last_peered":"2023-07-06T03:47:35.684178+0000","last_clean":"2023-07-06T03:47:35.684178+0000","last_became_active":"2023-07-06T03:47:35.676089+0000","last_became_peered":"2023-07-06T03:47:35.676089+0000","last_unstale":"2023-07-06T03:47:35.684178+0000","last_undegraded":"2023-07-06T03:47:35.684178+0000","last_fullsized":"2023-07-06T03:47:35.684178+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:58:51.450312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664710+0000","last_change":"2023-07-06T03:47:35.664710+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664710+0000","last_undegraded":"2023-07-06T03:47:35.664710+0000","last_fullsized":"2023-07-06T03:47:35.664710+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682615+0000","last_change":"2023-07-06T03:47:35.678850+0000","last_active":"2023-07-06T03:47:35.682615+0000","last_peered":"2023-07-06T03:47:35.682615+0000","last_clean":"2023-07-06T03:47:35.682615+0000","last_became_active":"2023-07-06T03:47:35.678520+0000","last_became_peered":"2023-07-06T03:47:35.678520+0000","last_unstale":"2023-07-06T03:47:35.682615+0000","last_undegraded":"2023-07-06T03:47:35.682615+0000","last_fullsized":"2023-07-06T03:47:35.682615+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:08:14.846337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671834+0000","last_change":"2023-07-06T03:47:35.671834+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671834+0000","last_undegraded":"2023-07-06T03:47:35.671834+0000","last_fullsized":"2023-07-06T03:47:35.671834+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.683654+0000","last_change":"2023-07-06T03:47:35.679264+0000","last_active":"2023-07-06T03:47:35.683654+0000","last_peered":"2023-07-06T03:47:35.683654+0000","last_clean":"2023-07-06T03:47:35.683654+0000","last_became_active":"2023-07-06T03:47:35.678950+0000","last_became_peered":"2023-07-06T03:47:35.678950+0000","last_unstale":"2023-07-06T03:47:35.683654+0000","last_undegraded":"2023-07-06T03:47:35.683654+0000","last_fullsized":"2023-07-06T03:47:35.683654+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:11:09.395143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667838+0000","last_change":"2023-07-06T03:47:35.667838+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667838+0000","last_undegraded":"2023-07-06T03:47:35.667838+0000","last_fullsized":"2023-07-06T03:47:35.667838+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681981+0000","last_change":"2023-07-06T03:47:35.679463+0000","last_active":"2023-07-06T03:47:35.681981+0000","last_peered":"2023-07-06T03:47:35.681981+0000","last_clean":"2023-07-06T03:47:35.681981+0000","last_became_active":"2023-07-06T03:47:35.679249+0000","last_became_peered":"2023-07-06T03:47:35.679249+0000","last_unstale":"2023-07-06T03:47:35.681981+0000","last_undegraded":"2023-07-06T03:47:35.681981+0000","last_fullsized":"2023-07-06T03:47:35.681981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:48:41.016578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665066+0000","last_change":"2023-07-06T03:47:35.665066+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665066+0000","last_undegraded":"2023-07-06T03:47:35.665066+0000","last_fullsized":"2023-07-06T03:47:35.665066+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686346+0000","last_change":"2023-07-06T03:47:35.684332+0000","last_active":"2023-07-06T03:47:35.686346+0000","last_peered":"2023-07-06T03:47:35.686346+0000","last_clean":"2023-07-06T03:47:35.686346+0000","last_became_active":"2023-07-06T03:47:35.684064+0000","last_became_peered":"2023-07-06T03:47:35.684064+0000","last_unstale":"2023-07-06T03:47:35.686346+0000","last_undegraded":"2023-07-06T03:47:35.686346+0000","last_fullsized":"2023-07-06T03:47:35.686346+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:45:08.107109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664510+0000","last_change":"2023-07-06T03:47:35.664510+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664510+0000","last_undegraded":"2023-07-06T03:47:35.664510+0000","last_fullsized":"2023-07-06T03:47:35.664510+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684604+0000","last_change":"2023-07-06T03:47:35.679127+0000","last_active":"2023-07-06T03:47:35.684604+0000","last_peered":"2023-07-06T03:47:35.684604+0000","last_clean":"2023-07-06T03:47:35.684604+0000","last_became_active":"2023-07-06T03:47:35.678630+0000","last_became_peered":"2023-07-06T03:47:35.678630+0000","last_unstale":"2023-07-06T03:47:35.684604+0000","last_undegraded":"2023-07-06T03:47:35.684604+0000","last_fullsized":"2023-07-06T03:47:35.684604+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:26:20.114823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671405+0000","last_change":"2023-07-06T03:47:35.671405+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671405+0000","last_undegraded":"2023-07-06T03:47:35.671405+0000","last_fullsized":"2023-07-06T03:47:35.671405+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.680869+0000","last_change":"2023-07-06T03:47:35.678328+0000","last_active":"2023-07-06T03:47:35.680869+0000","last_peered":"2023-07-06T03:47:35.680869+0000","last_clean":"2023-07-06T03:47:35.680869+0000","last_became_active":"2023-07-06T03:47:35.678081+0000","last_became_peered":"2023-07-06T03:47:35.678081+0000","last_unstale":"2023-07-06T03:47:35.680869+0000","last_undegraded":"2023-07-06T03:47:35.680869+0000","last_fullsized":"2023-07-06T03:47:35.680869+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:24:18.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664273+0000","last_change":"2023-07-06T03:47:35.664273+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664273+0000","last_undegraded":"2023-07-06T03:47:35.664273+0000","last_fullsized":"2023-07-06T03:47:35.664273+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.679689+0000","last_change":"2023-07-06T03:47:35.673408+0000","last_active":"2023-07-06T03:47:35.679689+0000","last_peered":"2023-07-06T03:47:35.679689+0000","last_clean":"2023-07-06T03:47:35.679689+0000","last_became_active":"2023-07-06T03:47:35.672993+0000","last_became_peered":"2023-07-06T03:47:35.672993+0000","last_unstale":"2023-07-06T03:47:35.679689+0000","last_undegraded":"2023-07-06T03:47:35.679689+0000","last_fullsized":"2023-07-06T03:47:35.679689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:56:55.279062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670077+0000","last_change":"2023-07-06T03:47:35.670077+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670077+0000","last_undegraded":"2023-07-06T03:47:35.670077+0000","last_fullsized":"2023-07-06T03:47:35.670077+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681921+0000","last_change":"2023-07-06T03:47:35.679946+0000","last_active":"2023-07-06T03:47:35.681921+0000","last_peered":"2023-07-06T03:47:35.681921+0000","last_clean":"2023-07-06T03:47:35.681921+0000","last_became_active":"2023-07-06T03:47:35.679487+0000","last_became_peered":"2023-07-06T03:47:35.679487+0000","last_unstale":"2023-07-06T03:47:35.681921+0000","last_undegraded":"2023-07-06T03:47:35.681921+0000","last_fullsized":"2023-07-06T03:47:35.681921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:52.953019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669355+0000","last_change":"2023-07-06T03:47:35.669355+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669355+0000","last_undegraded":"2023-07-06T03:47:35.669355+0000","last_fullsized":"2023-07-06T03:47:35.669355+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686839+0000","last_change":"2023-07-06T03:47:35.679499+0000","last_active":"2023-07-06T03:47:35.686839+0000","last_peered":"2023-07-06T03:47:35.686839+0000","last_clean":"2023-07-06T03:47:35.686839+0000","last_became_active":"2023-07-06T03:47:35.679153+0000","last_became_peered":"2023-07-06T03:47:35.679153+0000","last_unstale":"2023-07-06T03:47:35.686839+0000","last_undegraded":"2023-07-06T03:47:35.686839+0000","last_fullsized":"2023-07-06T03:47:35.686839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:28:35.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667017+0000","last_change":"2023-07-06T03:47:35.667017+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667017+0000","last_undegraded":"2023-07-06T03:47:35.667017+0000","last_fullsized":"2023-07-06T03:47:35.667017+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.676626+0000","last_change":"2023-07-06T03:47:35.675074+0000","last_active":"2023-07-06T03:47:35.676626+0000","last_peered":"2023-07-06T03:47:35.676626+0000","last_clean":"2023-07-06T03:47:35.676626+0000","last_became_active":"2023-07-06T03:47:35.674815+0000","last_became_peered":"2023-07-06T03:47:35.674815+0000","last_unstale":"2023-07-06T03:47:35.676626+0000","last_undegraded":"2023-07-06T03:47:35.676626+0000","last_fullsized":"2023-07-06T03:47:35.676626+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:44:22.676309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669675+0000","last_change":"2023-07-06T03:47:35.669675+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669675+0000","last_undegraded":"2023-07-06T03:47:35.669675+0000","last_fullsized":"2023-07-06T03:47:35.669675+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681758+0000","last_change":"2023-07-06T03:47:35.680141+0000","last_active":"2023-07-06T03:47:35.681758+0000","last_peered":"2023-07-06T03:47:35.681758+0000","last_clean":"2023-07-06T03:47:35.681758+0000","last_became_active":"2023-07-06T03:47:35.679775+0000","last_became_peered":"2023-07-06T03:47:35.679775+0000","last_unstale":"2023-07-06T03:47:35.681758+0000","last_undegraded":"2023-07-06T03:47:35.681758+0000","last_fullsized":"2023-07-06T03:47:35.681758+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:08:52.125995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668183+0000","last_change":"2023-07-06T03:47:35.668183+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668183+0000","last_undegraded":"2023-07-06T03:47:35.668183+0000","last_fullsized":"2023-07-06T03:47:35.668183+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682381+0000","last_change":"2023-07-06T03:47:35.676310+0000","last_active":"2023-07-06T03:47:35.682381+0000","last_peered":"2023-07-06T03:47:35.682381+0000","last_clean":"2023-07-06T03:47:35.682381+0000","last_became_active":"2023-07-06T03:47:35.676038+0000","last_became_peered":"2023-07-06T03:47:35.676038+0000","last_unstale":"2023-07-06T03:47:35.682381+0000","last_undegraded":"2023-07-06T03:47:35.682381+0000","last_fullsized":"2023-07-06T03:47:35.682381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:02:33.045576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663051+0000","last_change":"2023-07-06T03:47:35.663051+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663051+0000","last_undegraded":"2023-07-06T03:47:35.663051+0000","last_fullsized":"2023-07-06T03:47:35.663051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686476+0000","last_change":"2023-07-06T03:47:35.683270+0000","last_active":"2023-07-06T03:47:35.686476+0000","last_peered":"2023-07-06T03:47:35.686476+0000","last_clean":"2023-07-06T03:47:35.686476+0000","last_became_active":"2023-07-06T03:47:35.682973+0000","last_became_peered":"2023-07-06T03:47:35.682973+0000","last_unstale":"2023-07-06T03:47:35.686476+0000","last_undegraded":"2023-07-06T03:47:35.686476+0000","last_fullsized":"2023-07-06T03:47:35.686476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:03:08.897884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662280+0000","last_change":"2023-07-06T03:47:35.662280+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662280+0000","last_undegraded":"2023-07-06T03:47:35.662280+0000","last_fullsized":"2023-07-06T03:47:35.662280+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682343+0000","last_change":"2023-07-06T03:47:35.680067+0000","last_active":"2023-07-06T03:47:35.682343+0000","last_peered":"2023-07-06T03:47:35.682343+0000","last_clean":"2023-07-06T03:47:35.682343+0000","last_became_active":"2023-07-06T03:47:35.679704+0000","last_became_peered":"2023-07-06T03:47:35.679704+0000","last_unstale":"2023-07-06T03:47:35.682343+0000","last_undegraded":"2023-07-06T03:47:35.682343+0000","last_fullsized":"2023-07-06T03:47:35.682343+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:31:03.309257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.670261+0000","last_change":"2023-07-06T03:47:35.670261+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.670261+0000","last_undegraded":"2023-07-06T03:47:35.670261+0000","last_fullsized":"2023-07-06T03:47:35.670261+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686133+0000","last_change":"2023-07-06T03:47:35.680931+0000","last_active":"2023-07-06T03:47:35.686133+0000","last_peered":"2023-07-06T03:47:35.686133+0000","last_clean":"2023-07-06T03:47:35.686133+0000","last_became_active":"2023-07-06T03:47:35.680659+0000","last_became_peered":"2023-07-06T03:47:35.680659+0000","last_unstale":"2023-07-06T03:47:35.686133+0000","last_undegraded":"2023-07-06T03:47:35.686133+0000","last_fullsized":"2023-07-06T03:47:35.686133+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:01:39.848129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667702+0000","last_change":"2023-07-06T03:47:35.667702+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667702+0000","last_undegraded":"2023-07-06T03:47:35.667702+0000","last_fullsized":"2023-07-06T03:47:35.667702+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681731+0000","last_change":"2023-07-06T03:47:35.679503+0000","last_active":"2023-07-06T03:47:35.681731+0000","last_peered":"2023-07-06T03:47:35.681731+0000","last_clean":"2023-07-06T03:47:35.681731+0000","last_became_active":"2023-07-06T03:47:35.679231+0000","last_became_peered":"2023-07-06T03:47:35.679231+0000","last_unstale":"2023-07-06T03:47:35.681731+0000","last_undegraded":"2023-07-06T03:47:35.681731+0000","last_fullsized":"2023-07-06T03:47:35.681731+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:08:16.691892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665577+0000","last_change":"2023-07-06T03:47:35.665577+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665577+0000","last_undegraded":"2023-07-06T03:47:35.665577+0000","last_fullsized":"2023-07-06T03:47:35.665577+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.687009+0000","last_change":"2023-07-06T03:47:35.685943+0000","last_active":"2023-07-06T03:47:35.687009+0000","last_peered":"2023-07-06T03:47:35.687009+0000","last_clean":"2023-07-06T03:47:35.687009+0000","last_became_active":"2023-07-06T03:47:35.685673+0000","last_became_peered":"2023-07-06T03:47:35.685673+0000","last_unstale":"2023-07-06T03:47:35.687009+0000","last_undegraded":"2023-07-06T03:47:35.687009+0000","last_fullsized":"2023-07-06T03:47:35.687009+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:14:16.989542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.669618+0000","last_change":"2023-07-06T03:47:35.669618+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.669618+0000","last_undegraded":"2023-07-06T03:47:35.669618+0000","last_fullsized":"2023-07-06T03:47:35.669618+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.689703+0000","last_change":"2023-07-06T03:47:35.684207+0000","last_active":"2023-07-06T03:47:35.689703+0000","last_peered":"2023-07-06T03:47:35.689703+0000","last_clean":"2023-07-06T03:47:35.689703+0000","last_became_active":"2023-07-06T03:47:35.683837+0000","last_became_peered":"2023-07-06T03:47:35.683837+0000","last_unstale":"2023-07-06T03:47:35.689703+0000","last_undegraded":"2023-07-06T03:47:35.689703+0000","last_fullsized":"2023-07-06T03:47:35.689703+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:11.065681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664327+0000","last_change":"2023-07-06T03:47:35.664327+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664327+0000","last_undegraded":"2023-07-06T03:47:35.664327+0000","last_fullsized":"2023-07-06T03:47:35.664327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684585+0000","last_change":"2023-07-06T03:47:35.680503+0000","last_active":"2023-07-06T03:47:35.684585+0000","last_peered":"2023-07-06T03:47:35.684585+0000","last_clean":"2023-07-06T03:47:35.684585+0000","last_became_active":"2023-07-06T03:47:35.680171+0000","last_became_peered":"2023-07-06T03:47:35.680171+0000","last_unstale":"2023-07-06T03:47:35.684585+0000","last_undegraded":"2023-07-06T03:47:35.684585+0000","last_fullsized":"2023-07-06T03:47:35.684585+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:07:12.686190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.675202+0000","last_change":"2023-07-06T03:47:35.675202+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.675202+0000","last_undegraded":"2023-07-06T03:47:35.675202+0000","last_fullsized":"2023-07-06T03:47:35.675202+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686872+0000","last_change":"2023-07-06T03:47:35.680421+0000","last_active":"2023-07-06T03:47:35.686872+0000","last_peered":"2023-07-06T03:47:35.686872+0000","last_clean":"2023-07-06T03:47:35.686872+0000","last_became_active":"2023-07-06T03:47:35.679465+0000","last_became_peered":"2023-07-06T03:47:35.679465+0000","last_unstale":"2023-07-06T03:47:35.686872+0000","last_undegraded":"2023-07-06T03:47:35.686872+0000","last_fullsized":"2023-07-06T03:47:35.686872+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:22.389725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667496+0000","last_change":"2023-07-06T03:47:35.667496+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667496+0000","last_undegraded":"2023-07-06T03:47:35.667496+0000","last_fullsized":"2023-07-06T03:47:35.667496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.683955+0000","last_change":"2023-07-06T03:47:35.673523+0000","last_active":"2023-07-06T03:47:35.683955+0000","last_peered":"2023-07-06T03:47:35.683955+0000","last_clean":"2023-07-06T03:47:35.683955+0000","last_became_active":"2023-07-06T03:47:35.672470+0000","last_became_peered":"2023-07-06T03:47:35.672470+0000","last_unstale":"2023-07-06T03:47:35.683955+0000","last_undegraded":"2023-07-06T03:47:35.683955+0000","last_fullsized":"2023-07-06T03:47:35.683955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:23:29.539377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.662836+0000","last_change":"2023-07-06T03:47:35.662836+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.662836+0000","last_undegraded":"2023-07-06T03:47:35.662836+0000","last_fullsized":"2023-07-06T03:47:35.662836+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.682665+0000","last_change":"2023-07-06T03:47:35.681014+0000","last_active":"2023-07-06T03:47:35.682665+0000","last_peered":"2023-07-06T03:47:35.682665+0000","last_clean":"2023-07-06T03:47:35.682665+0000","last_became_active":"2023-07-06T03:47:35.680766+0000","last_became_peered":"2023-07-06T03:47:35.680766+0000","last_unstale":"2023-07-06T03:47:35.682665+0000","last_undegraded":"2023-07-06T03:47:35.682665+0000","last_fullsized":"2023-07-06T03:47:35.682665+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:11:00.458188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.665633+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:47:35.665633+0000","last_peered":"2023-07-06T03:47:35.665633+0000","last_clean":"2023-07-06T03:47:35.665633+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:47:35.665633+0000","last_undegraded":"2023-07-06T03:47:35.665633+0000","last_fullsized":"2023-07-06T03:47:35.665633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:25:11.511422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664761+0000","last_change":"2023-07-06T03:47:35.664761+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664761+0000","last_undegraded":"2023-07-06T03:47:35.664761+0000","last_fullsized":"2023-07-06T03:47:35.664761+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686329+0000","last_change":"2023-07-06T03:47:35.684268+0000","last_active":"2023-07-06T03:47:35.686329+0000","last_peered":"2023-07-06T03:47:35.686329+0000","last_clean":"2023-07-06T03:47:35.686329+0000","last_became_active":"2023-07-06T03:47:35.683961+0000","last_became_peered":"2023-07-06T03:47:35.683961+0000","last_unstale":"2023-07-06T03:47:35.686329+0000","last_undegraded":"2023-07-06T03:47:35.686329+0000","last_fullsized":"2023-07-06T03:47:35.686329+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:46:49.441174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.668531+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:47:35.668531+0000","last_peered":"2023-07-06T03:47:35.668531+0000","last_clean":"2023-07-06T03:47:35.668531+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:47:35.668531+0000","last_undegraded":"2023-07-06T03:47:35.668531+0000","last_fullsized":"2023-07-06T03:47:35.668531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:40:54.630791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.665474+0000","last_change":"2023-07-06T03:47:35.665474+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.665474+0000","last_undegraded":"2023-07-06T03:47:35.665474+0000","last_fullsized":"2023-07-06T03:47:35.665474+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.683642+0000","last_change":"2023-07-06T03:47:35.677576+0000","last_active":"2023-07-06T03:47:35.683642+0000","last_peered":"2023-07-06T03:47:35.683642+0000","last_clean":"2023-07-06T03:47:35.683642+0000","last_became_active":"2023-07-06T03:47:35.677247+0000","last_became_peered":"2023-07-06T03:47:35.677247+0000","last_unstale":"2023-07-06T03:47:35.683642+0000","last_undegraded":"2023-07-06T03:47:35.683642+0000","last_fullsized":"2023-07-06T03:47:35.683642+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:22:57.626952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.668542+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:47:35.668542+0000","last_peered":"2023-07-06T03:47:35.668542+0000","last_clean":"2023-07-06T03:47:35.668542+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:47:35.668542+0000","last_undegraded":"2023-07-06T03:47:35.668542+0000","last_fullsized":"2023-07-06T03:47:35.668542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:55:11.004301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.671814+0000","last_change":"2023-07-06T03:47:35.671814+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.671814+0000","last_undegraded":"2023-07-06T03:47:35.671814+0000","last_fullsized":"2023-07-06T03:47:35.671814+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684977+0000","last_change":"2023-07-06T03:47:35.677003+0000","last_active":"2023-07-06T03:47:35.684977+0000","last_peered":"2023-07-06T03:47:35.684977+0000","last_clean":"2023-07-06T03:47:35.684977+0000","last_became_active":"2023-07-06T03:47:35.676727+0000","last_became_peered":"2023-07-06T03:47:35.676727+0000","last_unstale":"2023-07-06T03:47:35.684977+0000","last_undegraded":"2023-07-06T03:47:35.684977+0000","last_fullsized":"2023-07-06T03:47:35.684977+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:35:50.061745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.667278+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:47:35.667278+0000","last_peered":"2023-07-06T03:47:35.667278+0000","last_clean":"2023-07-06T03:47:35.667278+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:47:35.667278+0000","last_undegraded":"2023-07-06T03:47:35.667278+0000","last_fullsized":"2023-07-06T03:47:35.667278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:18:53.047698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.666895+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:47:35.666895+0000","last_peered":"2023-07-06T03:47:35.666895+0000","last_clean":"2023-07-06T03:47:35.666895+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:47:35.666895+0000","last_undegraded":"2023-07-06T03:47:35.666895+0000","last_fullsized":"2023-07-06T03:47:35.666895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:22:28.543253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.664261+0000","last_change":"2023-07-06T03:47:35.664261+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.664261+0000","last_undegraded":"2023-07-06T03:47:35.664261+0000","last_fullsized":"2023-07-06T03:47:35.664261+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.681903+0000","last_change":"2023-07-06T03:47:35.679794+0000","last_active":"2023-07-06T03:47:35.681903+0000","last_peered":"2023-07-06T03:47:35.681903+0000","last_clean":"2023-07-06T03:47:35.681903+0000","last_became_active":"2023-07-06T03:47:35.679116+0000","last_became_peered":"2023-07-06T03:47:35.679116+0000","last_unstale":"2023-07-06T03:47:35.681903+0000","last_undegraded":"2023-07-06T03:47:35.681903+0000","last_fullsized":"2023-07-06T03:47:35.681903+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:13:53.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.666703+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:47:35.666703+0000","last_peered":"2023-07-06T03:47:35.666703+0000","last_clean":"2023-07-06T03:47:35.666703+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:47:35.666703+0000","last_undegraded":"2023-07-06T03:47:35.666703+0000","last_fullsized":"2023-07-06T03:47:35.666703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:39:52.909931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.667932+0000","last_change":"2023-07-06T03:47:35.667932+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.667932+0000","last_undegraded":"2023-07-06T03:47:35.667932+0000","last_fullsized":"2023-07-06T03:47:35.667932+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.686114+0000","last_change":"2023-07-06T03:47:35.684944+0000","last_active":"2023-07-06T03:47:35.686114+0000","last_peered":"2023-07-06T03:47:35.686114+0000","last_clean":"2023-07-06T03:47:35.686114+0000","last_became_active":"2023-07-06T03:47:35.684654+0000","last_became_peered":"2023-07-06T03:47:35.684654+0000","last_unstale":"2023-07-06T03:47:35.686114+0000","last_undegraded":"2023-07-06T03:47:35.686114+0000","last_fullsized":"2023-07-06T03:47:35.686114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:05.886555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.668075+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:47:35.668075+0000","last_peered":"2023-07-06T03:47:35.668075+0000","last_clean":"2023-07-06T03:47:35.668075+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:47:35.668075+0000","last_undegraded":"2023-07-06T03:47:35.668075+0000","last_fullsized":"2023-07-06T03:47:35.668075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:33:23.406225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.663757+0000","last_change":"2023-07-06T03:47:35.663757+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.663757+0000","last_undegraded":"2023-07-06T03:47:35.663757+0000","last_fullsized":"2023-07-06T03:47:35.663757+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684244+0000","last_change":"2023-07-06T03:47:35.677115+0000","last_active":"2023-07-06T03:47:35.684244+0000","last_peered":"2023-07-06T03:47:35.684244+0000","last_clean":"2023-07-06T03:47:35.684244+0000","last_became_active":"2023-07-06T03:47:35.676826+0000","last_became_peered":"2023-07-06T03:47:35.676826+0000","last_unstale":"2023-07-06T03:47:35.684244+0000","last_undegraded":"2023-07-06T03:47:35.684244+0000","last_fullsized":"2023-07-06T03:47:35.684244+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:26.484401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.664524+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:47:35.664524+0000","last_peered":"2023-07-06T03:47:35.664524+0000","last_clean":"2023-07-06T03:47:35.664524+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:47:35.664524+0000","last_undegraded":"2023-07-06T03:47:35.664524+0000","last_fullsized":"2023-07-06T03:47:35.664524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:55:15.702636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-07-06T03:47:35.668249+0000","last_change":"2023-07-06T03:47:35.668249+0000","last_active":"2023-07-06T03:47:35.650072+0000","last_peered":"2023-07-06T03:47:35.650072+0000","last_clean":"2023-07-06T03:47:35.650072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:47:35.668249+0000","last_undegraded":"2023-07-06T03:47:35.668249+0000","last_fullsized":"2023-07-06T03:47:35.668249+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.684607+0000","last_change":"2023-07-06T03:47:35.680550+0000","last_active":"2023-07-06T03:47:35.684607+0000","last_peered":"2023-07-06T03:47:35.684607+0000","last_clean":"2023-07-06T03:47:35.684607+0000","last_became_active":"2023-07-06T03:47:35.680201+0000","last_became_peered":"2023-07-06T03:47:35.680201+0000","last_unstale":"2023-07-06T03:47:35.684607+0000","last_undegraded":"2023-07-06T03:47:35.684607+0000","last_fullsized":"2023-07-06T03:47:35.684607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:54:15.807837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-07-06T03:47:35.667832+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:47:35.667832+0000","last_peered":"2023-07-06T03:47:35.667832+0000","last_clean":"2023-07-06T03:47:35.667832+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:47:35.667832+0000","last_undegraded":"2023-07-06T03:47:35.667832+0000","last_fullsized":"2023-07-06T03:47:35.667832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:11:43.805367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":528384,"data_stored":223271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86399999999999999,"5min":1.345,"15min":1.345},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.81499999999999995,"15min":0.83699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91100000000000003,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.748,"15min":0.76300000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85799999999999998,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.91800000000000004,"15min":0.95899999999999996},"min":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.876,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.90800000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.82399999999999995,"15min":0.84699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":3.77,"15min":3.77},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88400000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95699999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.94699999999999995,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96599999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.746,"15min":0.746},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98799999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92800000000000005,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.92800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.79400000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74199999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83899999999999997,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77800000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.44900000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.874,"5min":1.119,"15min":1.119},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.76500000000000001,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.73999999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78200000000000003,"5min":1.093,"15min":1.093},"last":0.621}]},{"osd":4,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93500000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.746,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.81000000000000005,"5min":1.085,"15min":1.085},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72599999999999998,"5min":1.218,"15min":1.218},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.80200000000000005,"15min":0.81899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0269999999999999,"5min":1.198,"15min":1.198},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.84299999999999997,"15min":0.86699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.83799999999999997,"5min":1.323,"15min":1.323},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.84899999999999998,"15min":0.877},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86799999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":1110016,"data_stored":807011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58399999999999996,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.58999999999999997,"15min":0.60999999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.628,"5min":1.96,"15min":1.96},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":1.0229999999999999,"15min":1.119},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54600000000000004,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.65800000000000003,"15min":0.68500000000000005},"min":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67900000000000005,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.63400000000000001,"15min":0.65600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.752,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.59499999999999997,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.439}]},{"osd":4,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":1.2789999999999999,"15min":1.375},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86199999999999999},"min":{"1min":0.48099999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94299999999999995,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.86199999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.78500000000000003,"15min":0.80100000000000005},"min":{"1min":0.54100000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92100000000000004,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.2929999999999999,"15min":1.393},"min":{"1min":0.55900000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.874,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.79200000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.60599999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.90600000000000003,"5min":1.365,"15min":1.365},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.335,"15min":1.4370000000000001},"min":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.88500000000000001,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.84899999999999998,"15min":0.875},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.86299999999999999,"5min":2.806,"15min":2.806},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82999999999999996,"15min":0.84599999999999997},"min":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.93000000000000005,"5min":1.337,"15min":1.337},"last":0.77200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41699999999999998,"15min":0.41799999999999998},"min":{"1min":0.316,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.443,"15min":0.441},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47599999999999998,"15min":0.47999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60199999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80700000000000005,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.438}]},{"osd":3,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.38800000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74199999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":0.76300000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":1.171,"15min":1.171},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.87,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91700000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.88500000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.546,"15min":1.546},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81000000000000005,"15min":0.81799999999999995},"min":{"1min":0.59199999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96799999999999997,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":1.155,"15min":1.2290000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.95599999999999996,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35648,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336192,"statfs":{"total":96636764160,"available":96600260608,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.57299999999999995,"5min":0.626,"15min":0.626},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.52600000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.66400000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66800000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.61099999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69299999999999995,"15min":0.69399999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":1.165,"15min":1.165},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.751},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.90500000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.746,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91400000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.90500000000000003,"5min":2.157,"15min":2.157},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.61299999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88900000000000001,"5min":1.966,"15min":1.966},"last":0.81799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.65700000000000003,"15min":0.69699999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.40699999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60599999999999998,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":1.196,"15min":1.3260000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63300000000000001,"5min":15.911,"15min":15.911},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.266,"15min":1.3959999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63700000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.55600000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":1.2709999999999999,"15min":1.401},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.59699999999999998,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.442}]},{"osd":4,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84899999999999998,"5min":1.179,"15min":1.179},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.97699999999999998,"15min":1.028},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89500000000000002,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.97999999999999998,"15min":1.0269999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92700000000000005,"5min":5.077,"15min":5.077},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.0409999999999999,"15min":1.0940000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89600000000000002,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.52500000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.90200000000000002,"5min":1.143,"15min":1.143},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.995,"15min":1.0449999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96799999999999997,"5min":5.157,"15min":5.157},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.85099999999999998,"15min":0.878},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96399999999999997,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.006,"15min":1.0549999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93000000000000005,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.73999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":516096,"data_stored":216643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.91500000000000004,"15min":0.95899999999999996},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94399999999999995,"5min":6.258,"15min":6.258},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.93400000000000005,"15min":0.97299999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1419999999999999,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":1.0189999999999999,"15min":1.069},"min":{"1min":0.54800000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0089999999999999,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.073,"5min":1.413,"15min":1.413},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80800000000000005,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.499},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.873,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2330000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.028,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.76700000000000002,"15min":0.77700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.58699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1105920,"data_stored":805992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.90500000000000003,"15min":0.95499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94199999999999995,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81100000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.629},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95399999999999996,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69299999999999995,"15min":0.68500000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.749},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79600000000000004,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:47:42.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2023-07-06T03:47:42.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.671120+0000', 'last_change': '2023-07-06T03:47:35.671120+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.671120+0000', 'last_undegraded': '2023-07-06T03:47:35.671120+0000', 'last_fullsized': '2023-07-06T03:47:35.671120+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:47:42.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2023-07-06T03:47:42.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.668727+0000', 'last_change': '2023-07-06T03:47:35.668727+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.668727+0000', 'last_undegraded': '2023-07-06T03:47:35.668727+0000', 'last_fullsized': '2023-07-06T03:47:35.668727+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:47:42.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2023-07-06T03:47:42.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664247+0000', 'last_change': '2023-07-06T03:47:35.664247+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664247+0000', 'last_undegraded': '2023-07-06T03:47:35.664247+0000', 'last_fullsized': '2023-07-06T03:47:35.664247+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:47:42.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2023-07-06T03:47:42.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664482+0000', 'last_change': '2023-07-06T03:47:35.664482+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664482+0000', 'last_undegraded': '2023-07-06T03:47:35.664482+0000', 'last_fullsized': '2023-07-06T03:47:35.664482+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:47:42.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2023-07-06T03:47:42.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.668927+0000', 'last_change': '2023-07-06T03:47:35.668927+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.668927+0000', 'last_undegraded': '2023-07-06T03:47:35.668927+0000', 'last_fullsized': '2023-07-06T03:47:35.668927+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:47:42.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2023-07-06T03:47:42.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.665583+0000', 'last_change': '2023-07-06T03:47:35.665583+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.665583+0000', 'last_undegraded': '2023-07-06T03:47:35.665583+0000', 'last_fullsized': '2023-07-06T03:47:35.665583+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:47:42.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2023-07-06T03:47:42.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664339+0000', 'last_change': '2023-07-06T03:47:35.664339+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664339+0000', 'last_undegraded': '2023-07-06T03:47:35.664339+0000', 'last_fullsized': '2023-07-06T03:47:35.664339+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:47:42.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2023-07-06T03:47:42.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.665496+0000', 'last_change': '2023-07-06T03:47:35.665496+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.665496+0000', 'last_undegraded': '2023-07-06T03:47:35.665496+0000', 'last_fullsized': '2023-07-06T03:47:35.665496+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:47:42.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2023-07-06T03:47:42.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.663507+0000', 'last_change': '2023-07-06T03:47:35.663507+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.663507+0000', 'last_undegraded': '2023-07-06T03:47:35.663507+0000', 'last_fullsized': '2023-07-06T03:47:35.663507+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:47:42.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2023-07-06T03:47:42.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664563+0000', 'last_change': '2023-07-06T03:47:35.664563+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664563+0000', 'last_undegraded': '2023-07-06T03:47:35.664563+0000', 'last_fullsized': '2023-07-06T03:47:35.664563+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:47:42.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2023-07-06T03:47:42.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.669176+0000', 'last_change': '2023-07-06T03:47:35.669176+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.669176+0000', 'last_undegraded': '2023-07-06T03:47:35.669176+0000', 'last_fullsized': '2023-07-06T03:47:35.669176+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:47:42.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2023-07-06T03:47:42.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.667342+0000', 'last_change': '2023-07-06T03:47:35.667342+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.667342+0000', 'last_undegraded': '2023-07-06T03:47:35.667342+0000', 'last_fullsized': '2023-07-06T03:47:35.667342+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:47:42.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2023-07-06T03:47:42.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.666058+0000', 'last_change': '2023-07-06T03:47:35.666058+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.666058+0000', 'last_undegraded': '2023-07-06T03:47:35.666058+0000', 'last_fullsized': '2023-07-06T03:47:35.666058+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:47:42.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2023-07-06T03:47:42.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.670675+0000', 'last_change': '2023-07-06T03:47:35.670675+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.670675+0000', 'last_undegraded': '2023-07-06T03:47:35.670675+0000', 'last_fullsized': '2023-07-06T03:47:35.670675+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:47:42.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2023-07-06T03:47:42.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.665278+0000', 'last_change': '2023-07-06T03:47:35.665278+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.665278+0000', 'last_undegraded': '2023-07-06T03:47:35.665278+0000', 'last_fullsized': '2023-07-06T03:47:35.665278+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:47:42.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2023-07-06T03:47:42.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664921+0000', 'last_change': '2023-07-06T03:47:35.664921+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664921+0000', 'last_undegraded': '2023-07-06T03:47:35.664921+0000', 'last_fullsized': '2023-07-06T03:47:35.664921+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:47:42.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2023-07-06T03:47:42.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.670602+0000', 'last_change': '2023-07-06T03:47:35.670602+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.670602+0000', 'last_undegraded': '2023-07-06T03:47:35.670602+0000', 'last_fullsized': '2023-07-06T03:47:35.670602+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:47:42.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2023-07-06T03:47:42.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664731+0000', 'last_change': '2023-07-06T03:47:35.664731+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664731+0000', 'last_undegraded': '2023-07-06T03:47:35.664731+0000', 'last_fullsized': '2023-07-06T03:47:35.664731+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:47:42.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2023-07-06T03:47:42.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.663759+0000', 'last_change': '2023-07-06T03:47:35.663759+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.663759+0000', 'last_undegraded': '2023-07-06T03:47:35.663759+0000', 'last_fullsized': '2023-07-06T03:47:35.663759+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:47:42.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2023-07-06T03:47:42.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.670330+0000', 'last_change': '2023-07-06T03:47:35.670330+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.670330+0000', 'last_undegraded': '2023-07-06T03:47:35.670330+0000', 'last_fullsized': '2023-07-06T03:47:35.670330+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:47:42.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2023-07-06T03:47:42.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664418+0000', 'last_change': '2023-07-06T03:47:35.664418+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664418+0000', 'last_undegraded': '2023-07-06T03:47:35.664418+0000', 'last_fullsized': '2023-07-06T03:47:35.664418+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:47:42.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2023-07-06T03:47:42.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.663209+0000', 'last_change': '2023-07-06T03:47:35.663209+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.663209+0000', 'last_undegraded': '2023-07-06T03:47:35.663209+0000', 'last_fullsized': '2023-07-06T03:47:35.663209+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:47:42.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2023-07-06T03:47:42.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.662599+0000', 'last_change': '2023-07-06T03:47:35.662599+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.662599+0000', 'last_undegraded': '2023-07-06T03:47:35.662599+0000', 'last_fullsized': '2023-07-06T03:47:35.662599+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:47:42.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2023-07-06T03:47:42.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.666150+0000', 'last_change': '2023-07-06T03:47:35.666150+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.666150+0000', 'last_undegraded': '2023-07-06T03:47:35.666150+0000', 'last_fullsized': '2023-07-06T03:47:35.666150+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:47:42.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2023-07-06T03:47:42.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.670081+0000', 'last_change': '2023-07-06T03:47:35.670081+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.670081+0000', 'last_undegraded': '2023-07-06T03:47:35.670081+0000', 'last_fullsized': '2023-07-06T03:47:35.670081+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:47:42.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2023-07-06T03:47:42.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664042+0000', 'last_change': '2023-07-06T03:47:35.664042+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664042+0000', 'last_undegraded': '2023-07-06T03:47:35.664042+0000', 'last_fullsized': '2023-07-06T03:47:35.664042+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:47:42.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2023-07-06T03:47:42.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.667395+0000', 'last_change': '2023-07-06T03:47:35.667395+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.667395+0000', 'last_undegraded': '2023-07-06T03:47:35.667395+0000', 'last_fullsized': '2023-07-06T03:47:35.667395+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:47:42.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2023-07-06T03:47:42.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.669999+0000', 'last_change': '2023-07-06T03:47:35.669999+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.669999+0000', 'last_undegraded': '2023-07-06T03:47:35.669999+0000', 'last_fullsized': '2023-07-06T03:47:35.669999+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:47:42.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2023-07-06T03:47:42.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.674695+0000', 'last_change': '2023-07-06T03:47:35.674695+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.674695+0000', 'last_undegraded': '2023-07-06T03:47:35.674695+0000', 'last_fullsized': '2023-07-06T03:47:35.674695+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:47:42.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2023-07-06T03:47:42.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.668319+0000', 'last_change': '2023-07-06T03:47:35.668319+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.668319+0000', 'last_undegraded': '2023-07-06T03:47:35.668319+0000', 'last_fullsized': '2023-07-06T03:47:35.668319+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T03:47:42.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2023-07-06T03:47:42.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.662011+0000', 'last_change': '2023-07-06T03:47:35.662011+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.662011+0000', 'last_undegraded': '2023-07-06T03:47:35.662011+0000', 'last_fullsized': '2023-07-06T03:47:35.662011+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:47:42.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2023-07-06T03:47:42.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.668114+0000', 'last_change': '2023-07-06T03:47:35.668114+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.668114+0000', 'last_undegraded': '2023-07-06T03:47:35.668114+0000', 'last_fullsized': '2023-07-06T03:47:35.668114+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:47:42.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-07-06T03:47:42.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.666321+0000', 'last_change': '2023-07-06T03:47:35.666321+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.666321+0000', 'last_undegraded': '2023-07-06T03:47:35.666321+0000', 'last_fullsized': '2023-07-06T03:47:35.666321+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T03:47:42.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-07-06T03:47:42.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.666528+0000', 'last_change': '2023-07-06T03:47:35.666528+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.666528+0000', 'last_undegraded': '2023-07-06T03:47:35.666528+0000', 'last_fullsized': '2023-07-06T03:47:35.666528+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T03:47:42.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-07-06T03:47:42.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.667029+0000', 'last_change': '2023-07-06T03:47:35.667029+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.667029+0000', 'last_undegraded': '2023-07-06T03:47:35.667029+0000', 'last_fullsized': '2023-07-06T03:47:35.667029+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T03:47:42.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-07-06T03:47:42.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664710+0000', 'last_change': '2023-07-06T03:47:35.664710+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664710+0000', 'last_undegraded': '2023-07-06T03:47:35.664710+0000', 'last_fullsized': '2023-07-06T03:47:35.664710+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:47:42.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-07-06T03:47:42.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.671834+0000', 'last_change': '2023-07-06T03:47:35.671834+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.671834+0000', 'last_undegraded': '2023-07-06T03:47:35.671834+0000', 'last_fullsized': '2023-07-06T03:47:35.671834+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T03:47:42.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-07-06T03:47:42.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.667838+0000', 'last_change': '2023-07-06T03:47:35.667838+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.667838+0000', 'last_undegraded': '2023-07-06T03:47:35.667838+0000', 'last_fullsized': '2023-07-06T03:47:35.667838+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T03:47:42.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-07-06T03:47:42.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.665066+0000', 'last_change': '2023-07-06T03:47:35.665066+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.665066+0000', 'last_undegraded': '2023-07-06T03:47:35.665066+0000', 'last_fullsized': '2023-07-06T03:47:35.665066+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:47:42.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-07-06T03:47:42.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664510+0000', 'last_change': '2023-07-06T03:47:35.664510+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664510+0000', 'last_undegraded': '2023-07-06T03:47:35.664510+0000', 'last_fullsized': '2023-07-06T03:47:35.664510+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:47:42.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-07-06T03:47:42.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.671405+0000', 'last_change': '2023-07-06T03:47:35.671405+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.671405+0000', 'last_undegraded': '2023-07-06T03:47:35.671405+0000', 'last_fullsized': '2023-07-06T03:47:35.671405+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T03:47:42.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-07-06T03:47:42.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664273+0000', 'last_change': '2023-07-06T03:47:35.664273+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664273+0000', 'last_undegraded': '2023-07-06T03:47:35.664273+0000', 'last_fullsized': '2023-07-06T03:47:35.664273+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:47:42.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-07-06T03:47:42.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.670077+0000', 'last_change': '2023-07-06T03:47:35.670077+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.670077+0000', 'last_undegraded': '2023-07-06T03:47:35.670077+0000', 'last_fullsized': '2023-07-06T03:47:35.670077+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:47:42.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-07-06T03:47:42.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.669355+0000', 'last_change': '2023-07-06T03:47:35.669355+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.669355+0000', 'last_undegraded': '2023-07-06T03:47:35.669355+0000', 'last_fullsized': '2023-07-06T03:47:35.669355+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:47:42.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-07-06T03:47:42.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.667017+0000', 'last_change': '2023-07-06T03:47:35.667017+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.667017+0000', 'last_undegraded': '2023-07-06T03:47:35.667017+0000', 'last_fullsized': '2023-07-06T03:47:35.667017+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T03:47:42.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-07-06T03:47:42.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.669675+0000', 'last_change': '2023-07-06T03:47:35.669675+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.669675+0000', 'last_undegraded': '2023-07-06T03:47:35.669675+0000', 'last_fullsized': '2023-07-06T03:47:35.669675+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:47:42.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-07-06T03:47:42.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.668183+0000', 'last_change': '2023-07-06T03:47:35.668183+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.668183+0000', 'last_undegraded': '2023-07-06T03:47:35.668183+0000', 'last_fullsized': '2023-07-06T03:47:35.668183+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:47:42.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-07-06T03:47:42.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.663051+0000', 'last_change': '2023-07-06T03:47:35.663051+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.663051+0000', 'last_undegraded': '2023-07-06T03:47:35.663051+0000', 'last_fullsized': '2023-07-06T03:47:35.663051+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:47:42.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-07-06T03:47:42.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.662280+0000', 'last_change': '2023-07-06T03:47:35.662280+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.662280+0000', 'last_undegraded': '2023-07-06T03:47:35.662280+0000', 'last_fullsized': '2023-07-06T03:47:35.662280+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:47:42.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2023-07-06T03:47:42.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.670261+0000', 'last_change': '2023-07-06T03:47:35.670261+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.670261+0000', 'last_undegraded': '2023-07-06T03:47:35.670261+0000', 'last_fullsized': '2023-07-06T03:47:35.670261+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:47:42.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-07-06T03:47:42.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.667702+0000', 'last_change': '2023-07-06T03:47:35.667702+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.667702+0000', 'last_undegraded': '2023-07-06T03:47:35.667702+0000', 'last_fullsized': '2023-07-06T03:47:35.667702+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:47:42.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2023-07-06T03:47:42.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.665577+0000', 'last_change': '2023-07-06T03:47:35.665577+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.665577+0000', 'last_undegraded': '2023-07-06T03:47:35.665577+0000', 'last_fullsized': '2023-07-06T03:47:35.665577+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:47:42.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-07-06T03:47:42.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.669618+0000', 'last_change': '2023-07-06T03:47:35.669618+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.669618+0000', 'last_undegraded': '2023-07-06T03:47:35.669618+0000', 'last_fullsized': '2023-07-06T03:47:35.669618+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:47:42.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-07-06T03:47:42.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664327+0000', 'last_change': '2023-07-06T03:47:35.664327+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664327+0000', 'last_undegraded': '2023-07-06T03:47:35.664327+0000', 'last_fullsized': '2023-07-06T03:47:35.664327+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:47:42.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-07-06T03:47:42.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.675202+0000', 'last_change': '2023-07-06T03:47:35.675202+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.675202+0000', 'last_undegraded': '2023-07-06T03:47:35.675202+0000', 'last_fullsized': '2023-07-06T03:47:35.675202+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:47:42.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2023-07-06T03:47:42.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.667496+0000', 'last_change': '2023-07-06T03:47:35.667496+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.667496+0000', 'last_undegraded': '2023-07-06T03:47:35.667496+0000', 'last_fullsized': '2023-07-06T03:47:35.667496+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:47:42.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-07-06T03:47:42.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.662836+0000', 'last_change': '2023-07-06T03:47:35.662836+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.662836+0000', 'last_undegraded': '2023-07-06T03:47:35.662836+0000', 'last_fullsized': '2023-07-06T03:47:35.662836+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:47:42.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-07-06T03:47:42.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664761+0000', 'last_change': '2023-07-06T03:47:35.664761+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664761+0000', 'last_undegraded': '2023-07-06T03:47:35.664761+0000', 'last_fullsized': '2023-07-06T03:47:35.664761+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:47:42.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-07-06T03:47:42.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.665474+0000', 'last_change': '2023-07-06T03:47:35.665474+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.665474+0000', 'last_undegraded': '2023-07-06T03:47:35.665474+0000', 'last_fullsized': '2023-07-06T03:47:35.665474+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:47:42.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-07-06T03:47:42.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.671814+0000', 'last_change': '2023-07-06T03:47:35.671814+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.671814+0000', 'last_undegraded': '2023-07-06T03:47:35.671814+0000', 'last_fullsized': '2023-07-06T03:47:35.671814+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T03:47:42.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2023-07-06T03:47:42.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.664261+0000', 'last_change': '2023-07-06T03:47:35.664261+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.664261+0000', 'last_undegraded': '2023-07-06T03:47:35.664261+0000', 'last_fullsized': '2023-07-06T03:47:35.664261+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:47:42.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-07-06T03:47:42.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.667932+0000', 'last_change': '2023-07-06T03:47:35.667932+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.667932+0000', 'last_undegraded': '2023-07-06T03:47:35.667932+0000', 'last_fullsized': '2023-07-06T03:47:35.667932+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:47:42.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-07-06T03:47:42.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.663757+0000', 'last_change': '2023-07-06T03:47:35.663757+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.663757+0000', 'last_undegraded': '2023-07-06T03:47:35.663757+0000', 'last_fullsized': '2023-07-06T03:47:35.663757+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:47:42.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-07-06T03:47:42.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:47:35.668249+0000', 'last_change': '2023-07-06T03:47:35.668249+0000', 'last_active': '2023-07-06T03:47:35.650072+0000', 'last_peered': '2023-07-06T03:47:35.650072+0000', 'last_clean': '2023-07-06T03:47:35.650072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:47:35.668249+0000', 'last_undegraded': '2023-07-06T03:47:35.668249+0000', 'last_fullsized': '2023-07-06T03:47:35.668249+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'last_clean_scrub_stamp': '2023-07-06T03:47:35.650072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:47:42.054 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-07-06T03:47:42.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.247+0000 7f3e3f7ff700 1 Processor -- start 2023-07-06T03:47:42.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.248+0000 7f3e3f7ff700 1 -- start start 2023-07-06T03:47:42.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.249+0000 7f3e3f7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e3815c520 0x7f3e380cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.249+0000 7f3e3f7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e38160d00 0x7f3e380b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.249+0000 7f3e3f7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e380cdae0 con 0x7f3e38160180 2023-07-06T03:47:42.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.249+0000 7f3e3f7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e380cd8d0 con 0x7f3e3815c520 2023-07-06T03:47:42.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.249+0000 7f3e3f7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e380b6640 con 0x7f3e38160d00 2023-07-06T03:47:42.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.249+0000 7f3e3d59b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e3815c520 0x7f3e380cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.249+0000 7f3e3d59b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e3815c520 0x7f3e380cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33192/0 (socket says 172.21.15.97:33192) 2023-07-06T03:47:42.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.249+0000 7f3e3d59b700 1 -- 172.21.15.97:0/2962292267 learned_addr learned my addr 172.21.15.97:0/2962292267 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:42.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.249+0000 7f3e3dd9c700 1 --2- 172.21.15.97:0/2962292267 >> v2:172.21.15.146:3300/0 conn(0x7f3e38160d00 0x7f3e380b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.250+0000 7f3e3d59b700 1 -- 172.21.15.97:0/2962292267 >> v2:172.21.15.146:3300/0 conn(0x7f3e38160d00 msgr2=0x7f3e380b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:42.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.250+0000 7f3e3d59b700 1 --2- 172.21.15.97:0/2962292267 >> v2:172.21.15.146:3300/0 conn(0x7f3e38160d00 0x7f3e380b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.250+0000 7f3e3d59b700 1 -- 172.21.15.97:0/2962292267 >> v1:172.21.15.97:6789/0 conn(0x7f3e38160180 legacy=0x7f3e381605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:42.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.250+0000 7f3e3d59b700 1 -- 172.21.15.97:0/2962292267 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e380b6920 con 0x7f3e3815c520 2023-07-06T03:47:42.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.250+0000 7f3e2ffff700 1 -- 172.21.15.97:0/2962292267 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550225050 0 0) 0x7f3e380cdae0 con 0x7f3e38160180 2023-07-06T03:47:42.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.250+0000 7f3e3d59b700 1 --2- 172.21.15.97:0/2962292267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e3815c520 0x7f3e380cd390 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3e380cd320 tx=0x7f3e34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5169d5aa30ef1b3f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:42.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.250+0000 7f3e2ffff700 1 -- 172.21.15.97:0/2962292267 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3400b870 con 0x7f3e3815c520 2023-07-06T03:47:42.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.250+0000 7f3e2ffff700 1 -- 172.21.15.97:0/2962292267 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3400b9d0 con 0x7f3e3815c520 2023-07-06T03:47:42.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.250+0000 7f3e2ffff700 1 -- 172.21.15.97:0/2962292267 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3400bd60 con 0x7f3e3815c520 2023-07-06T03:47:42.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.251+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/2962292267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e3815c520 msgr2=0x7f3e380cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:42.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.251+0000 7f3e3f7ff700 1 --2- 172.21.15.97:0/2962292267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e3815c520 0x7f3e380cd390 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3e380cd320 tx=0x7f3e34009bd0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.251+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/2962292267 shutdown_connections 2023-07-06T03:47:42.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.251+0000 7f3e3f7ff700 1 --2- 172.21.15.97:0/2962292267 >> v2:172.21.15.146:3300/0 conn(0x7f3e38160d00 0x7f3e380b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.251+0000 7f3e3f7ff700 1 --2- 172.21.15.97:0/2962292267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e3815c520 0x7f3e380cd390 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.251+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/2962292267 >> 172.21.15.97:0/2962292267 conn(0x7f3e38071790 msgr2=0x7f3e38071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:42.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.251+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/2962292267 shutdown_connections 2023-07-06T03:47:42.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.251+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/2962292267 wait complete. 2023-07-06T03:47:42.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.252+0000 7f3e3f7ff700 1 Processor -- start 2023-07-06T03:47:42.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.253+0000 7f3e3f7ff700 1 -- start start 2023-07-06T03:47:42.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3f7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e3815c520 0x7f3e3815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3f7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e38160180 0x7f3e380cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3f7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e38160d00 0x7f3e380cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3f7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380cb0d0 con 0x7f3e38160d00 2023-07-06T03:47:42.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3f7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380cadf0 con 0x7f3e3815c520 2023-07-06T03:47:42.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3f7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380cb4f0 con 0x7f3e38160180 2023-07-06T03:47:42.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3dd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e38160d00 0x7f3e380cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3dd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e38160d00 0x7f3e380cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33196/0 (socket says 172.21.15.97:33196) 2023-07-06T03:47:42.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3dd9c700 1 -- 172.21.15.97:0/3287221620 learned_addr learned my addr 172.21.15.97:0/3287221620 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:42.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3d59b700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e3815c520 0x7f3e3815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3cd9a700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e38160180 0x7f3e380cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3dd9c700 1 -- 172.21.15.97:0/3287221620 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e38160180 msgr2=0x7f3e380cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:42.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3dd9c700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e38160180 0x7f3e380cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3dd9c700 1 -- 172.21.15.97:0/3287221620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e3815c520 msgr2=0x7f3e3815d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:42.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3dd9c700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e3815c520 0x7f3e3815d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3dd9c700 1 -- 172.21.15.97:0/3287221620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e34009550 con 0x7f3e38160d00 2023-07-06T03:47:42.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.254+0000 7f3e3d59b700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e3815c520 0x7f3e3815d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:42.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.255+0000 7f3e3dd9c700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e38160d00 0x7f3e380cc510 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3e30007e40 tx=0x7f3e3000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:42.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.255+0000 7f3e2dffb700 1 -- 172.21.15.97:0/3287221620 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000eea0 con 0x7f3e38160d00 2023-07-06T03:47:42.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.255+0000 7f3e2dffb700 1 -- 172.21.15.97:0/3287221620 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e30014420 con 0x7f3e38160d00 2023-07-06T03:47:42.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.255+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/3287221620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e380cb7d0 con 0x7f3e38160d00 2023-07-06T03:47:42.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.255+0000 7f3e2dffb700 1 -- 172.21.15.97:0/3287221620 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300147b0 con 0x7f3e38160d00 2023-07-06T03:47:42.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.255+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/3287221620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e380caa30 con 0x7f3e38160d00 2023-07-06T03:47:42.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.256+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/3287221620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e380cbcf0 con 0x7f3e38160d00 2023-07-06T03:47:42.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.257+0000 7f3e2dffb700 1 -- 172.21.15.97:0/3287221620 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e30014910 con 0x7f3e38160d00 2023-07-06T03:47:42.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.258+0000 7f3e2dffb700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e1c06ace0 0x7f3e1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.258+0000 7f3e2dffb700 1 -- 172.21.15.97:0/3287221620 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e30089c60 con 0x7f3e38160d00 2023-07-06T03:47:42.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.258+0000 7f3e3d59b700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e1c06ace0 0x7f3e1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.259+0000 7f3e3d59b700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e1c06ace0 0x7f3e1c06d190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3e34014040 tx=0x7f3e34008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.262+0000 7f3e2dffb700 1 -- 172.21.15.97:0/3287221620 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3e380cbcf0 con 0x7f3e38160d00 2023-07-06T03:47:42.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.536+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/3287221620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3e380cabc0 con 0x7f3e38160d00 2023-07-06T03:47:42.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.536+0000 7f3e2dffb700 1 -- 172.21.15.97:0/3287221620 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1195 (secure 0 0 0) 0x7f3e30058790 con 0x7f3e38160d00 2023-07-06T03:47:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:42.538 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":178,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":47,"num_osds":8,"num_up_osds":8,"osd_up_since":1688615095,"num_in_osds":8,"osd_in_since":1688615087,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":255475712,"bytes_avail":772838637568,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":5838}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-07-06T03:47:30.910383+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-07-06T03:47:42.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.538+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/3287221620 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e1c06ace0 msgr2=0x7f3e1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:42.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.538+0000 7f3e3f7ff700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e1c06ace0 0x7f3e1c06d190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3e34014040 tx=0x7f3e34008030 comp rx=0 tx=0).stop 2023-07-06T03:47:42.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.538+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/3287221620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e38160d00 msgr2=0x7f3e380cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:42.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.538+0000 7f3e3f7ff700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e38160d00 0x7f3e380cc510 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3e30007e40 tx=0x7f3e3000e3c0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.539+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/3287221620 shutdown_connections 2023-07-06T03:47:42.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.539+0000 7f3e3f7ff700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e1c06ace0 0x7f3e1c06d190 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.539+0000 7f3e3f7ff700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e38160d00 0x7f3e380cc510 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.539+0000 7f3e3f7ff700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e3815c520 0x7f3e3815d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.539+0000 7f3e3f7ff700 1 --2- 172.21.15.97:0/3287221620 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e38160180 0x7f3e380cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.539+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/3287221620 >> 172.21.15.97:0/3287221620 conn(0x7f3e38071790 msgr2=0x7f3e38071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:42.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.539+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/3287221620 shutdown_connections 2023-07-06T03:47:42.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.539+0000 7f3e3f7ff700 1 -- 172.21.15.97:0/3287221620 wait complete. 2023-07-06T03:47:42.558 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-07-06T03:47:42.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:47:42.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.752+0000 7f7d2a3ff700 1 Processor -- start 2023-07-06T03:47:42.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.752+0000 7f7d2a3ff700 1 -- start start 2023-07-06T03:47:42.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d2a3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7d240cd390 0x7f7d24160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d2a3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d24160dd0 0x7f7d240b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d2a3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d240cdae0 con 0x7f7d2415c520 2023-07-06T03:47:42.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d2a3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d240b6640 con 0x7f7d240cd390 2023-07-06T03:47:42.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d2a3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d240b6920 con 0x7f7d24160dd0 2023-07-06T03:47:42.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d237fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7d240cd390 0x7f7d24160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d237fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7d240cd390 0x7f7d24160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33202/0 (socket says 172.21.15.97:33202) 2023-07-06T03:47:42.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d237fe700 1 -- 172.21.15.97:0/293541366 learned_addr learned my addr 172.21.15.97:0/293541366 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:42.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d2899c700 1 --2- 172.21.15.97:0/293541366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d24160dd0 0x7f7d240b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d237fe700 1 -- 172.21.15.97:0/293541366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d24160dd0 msgr2=0x7f7d240b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:42.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d237fe700 1 --2- 172.21.15.97:0/293541366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d24160dd0 0x7f7d240b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d237fe700 1 -- 172.21.15.97:0/293541366 >> v1:172.21.15.146:6789/0 conn(0x7f7d2415c520 legacy=0x7f7d24160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:42.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d237fe700 1 -- 172.21.15.97:0/293541366 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d240b6c00 con 0x7f7d240cd390 2023-07-06T03:47:42.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.754+0000 7f7d2899c700 1 --2- 172.21.15.97:0/293541366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d24160dd0 0x7f7d240b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:47:42.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.755+0000 7f7d237fe700 1 --2- 172.21.15.97:0/293541366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d240cd390 0x7f7d24160890 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7d180098f0 tx=0x7f7d18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830813cc8279b090 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:42.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.755+0000 7f7d22ffd700 1 -- 172.21.15.97:0/293541366 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18004230 con 0x7f7d240cd390 2023-07-06T03:47:42.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.755+0000 7f7d22ffd700 1 -- 172.21.15.97:0/293541366 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d18004390 con 0x7f7d240cd390 2023-07-06T03:47:42.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.755+0000 7f7d22ffd700 1 -- 172.21.15.97:0/293541366 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d180036a0 con 0x7f7d240cd390 2023-07-06T03:47:42.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.755+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/293541366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d240cd390 msgr2=0x7f7d24160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:42.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.755+0000 7f7d2a3ff700 1 --2- 172.21.15.97:0/293541366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d240cd390 0x7f7d24160890 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7d180098f0 tx=0x7f7d18009c00 comp rx=0 tx=0).stop 2023-07-06T03:47:42.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.755+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/293541366 shutdown_connections 2023-07-06T03:47:42.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.755+0000 7f7d2a3ff700 1 --2- 172.21.15.97:0/293541366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d24160dd0 0x7f7d240b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.755+0000 7f7d2a3ff700 1 --2- 172.21.15.97:0/293541366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d240cd390 0x7f7d24160890 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.756+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/293541366 >> 172.21.15.97:0/293541366 conn(0x7f7d24071790 msgr2=0x7f7d24071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:42.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.756+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/293541366 shutdown_connections 2023-07-06T03:47:42.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.756+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/293541366 wait complete. 2023-07-06T03:47:42.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.757+0000 7f7d2a3ff700 1 Processor -- start 2023-07-06T03:47:42.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.757+0000 7f7d2a3ff700 1 -- start start 2023-07-06T03:47:42.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.758+0000 7f7d2a3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d240cd390 0x7f7d2415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.758+0000 7f7d2a3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d2415c520 0x7f7d240cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.758+0000 7f7d2a3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24160dd0 0x7f7d2415d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.758+0000 7f7d2a3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2415f9f0 con 0x7f7d24160dd0 2023-07-06T03:47:42.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.758+0000 7f7d2a3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2415b6c0 con 0x7f7d240cd390 2023-07-06T03:47:42.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.758+0000 7f7d2a3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2415ad90 con 0x7f7d2415c520 2023-07-06T03:47:42.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.759+0000 7f7d2899c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24160dd0 0x7f7d2415d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.759+0000 7f7d2899c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24160dd0 0x7f7d2415d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33210/0 (socket says 172.21.15.97:33210) 2023-07-06T03:47:42.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.759+0000 7f7d2899c700 1 -- 172.21.15.97:0/4032860369 learned_addr learned my addr 172.21.15.97:0/4032860369 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:42.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.759+0000 7f7d237fe700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d2415c520 0x7f7d240cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.759+0000 7f7d23fff700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d240cd390 0x7f7d2415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.759+0000 7f7d2899c700 1 -- 172.21.15.97:0/4032860369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d2415c520 msgr2=0x7f7d240cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:42.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.759+0000 7f7d2899c700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d2415c520 0x7f7d240cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.759+0000 7f7d2899c700 1 -- 172.21.15.97:0/4032860369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d240cd390 msgr2=0x7f7d2415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.759+0000 7f7d2899c700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d240cd390 0x7f7d2415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.759+0000 7f7d2899c700 1 -- 172.21.15.97:0/4032860369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d18009550 con 0x7f7d24160dd0 2023-07-06T03:47:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.759+0000 7f7d2899c700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24160dd0 0x7f7d2415d0c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00cef0 tx=0x7f7d1c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.760+0000 7f7d20ff9700 1 -- 172.21.15.97:0/4032860369 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00ad00 con 0x7f7d24160dd0 2023-07-06T03:47:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.760+0000 7f7d20ff9700 1 -- 172.21.15.97:0/4032860369 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c00ae60 con 0x7f7d24160dd0 2023-07-06T03:47:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.760+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/4032860369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2415b070 con 0x7f7d24160dd0 2023-07-06T03:47:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.760+0000 7f7d20ff9700 1 -- 172.21.15.97:0/4032860369 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c015680 con 0x7f7d24160dd0 2023-07-06T03:47:42.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.760+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/4032860369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2415aa60 con 0x7f7d24160dd0 2023-07-06T03:47:42.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.761+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/4032860369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2404e340 con 0x7f7d24160dd0 2023-07-06T03:47:42.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.762+0000 7f7d20ff9700 1 -- 172.21.15.97:0/4032860369 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d1c00d430 con 0x7f7d24160dd0 2023-07-06T03:47:42.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.763+0000 7f7d20ff9700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d1006ad10 0x7f7d1006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:42.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.763+0000 7f7d20ff9700 1 -- 172.21.15.97:0/4032860369 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d1c089a10 con 0x7f7d24160dd0 2023-07-06T03:47:42.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.763+0000 7f7d23fff700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d1006ad10 0x7f7d1006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:42.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.763+0000 7f7d23fff700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d1006ad10 0x7f7d1006d1c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7d14007fd0 tx=0x7f7d14007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:42.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.767+0000 7f7d20ff9700 1 -- 172.21.15.97:0/4032860369 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7d1c054990 con 0x7f7d24160dd0 2023-07-06T03:47:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.959+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/4032860369 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d2415af00 con 0x7f7d1006ad10 2023-07-06T03:47:42.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.968+0000 7f7d20ff9700 1 -- 172.21.15.97:0/4032860369 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311104 (secure 0 0 0) 0x7f7d2415af00 con 0x7f7d1006ad10 2023-07-06T03:47:42.973 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:42.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.972+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/4032860369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d1006ad10 msgr2=0x7f7d1006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:42.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.972+0000 7f7d2a3ff700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d1006ad10 0x7f7d1006d1c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7d14007fd0 tx=0x7f7d14007ec0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.972+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/4032860369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24160dd0 msgr2=0x7f7d2415d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:42.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.972+0000 7f7d2a3ff700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24160dd0 0x7f7d2415d0c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00cef0 tx=0x7f7d1c00cf20 comp rx=0 tx=0).stop 2023-07-06T03:47:42.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.972+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/4032860369 shutdown_connections 2023-07-06T03:47:42.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.972+0000 7f7d2a3ff700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7d1006ad10 0x7f7d1006d1c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.972+0000 7f7d2a3ff700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24160dd0 0x7f7d2415d0c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.972+0000 7f7d2a3ff700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d240cd390 0x7f7d2415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.972+0000 7f7d2a3ff700 1 --2- 172.21.15.97:0/4032860369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7d2415c520 0x7f7d240cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:42.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.972+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/4032860369 >> 172.21.15.97:0/4032860369 conn(0x7f7d24071790 msgr2=0x7f7d240b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:42.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.972+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/4032860369 shutdown_connections 2023-07-06T03:47:42.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:42.973+0000 7f7d2a3ff700 1 -- 172.21.15.97:0/4032860369 wait complete. 2023-07-06T03:47:42.977 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:47:42.992 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2023-07-06T03:47:42.912396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249828,"kb_used_data":5540,"kb_used_omap":0,"kb_used_meta":244224,"kb_avail":754724892,"statfs":{"total":773094113280,"available":772838289408,"internally_reserved":0,"allocated":5672960,"data_stored":3080857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":250085376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.252371"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032632+0000","last_change":"2023-07-06T03:47:36.690034+0000","last_active":"2023-07-06T03:47:38.032632+0000","last_peered":"2023-07-06T03:47:38.032632+0000","last_clean":"2023-07-06T03:47:38.032632+0000","last_became_active":"2023-07-06T03:47:36.689699+0000","last_became_peered":"2023-07-06T03:47:36.689699+0000","last_unstale":"2023-07-06T03:47:38.032632+0000","last_undegraded":"2023-07-06T03:47:38.032632+0000","last_fullsized":"2023-07-06T03:47:38.032632+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:08:53.128913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679530+0000","last_change":"2023-07-06T03:47:36.684458+0000","last_active":"2023-07-06T03:47:37.679530+0000","last_peered":"2023-07-06T03:47:37.679530+0000","last_clean":"2023-07-06T03:47:37.679530+0000","last_became_active":"2023-07-06T03:47:36.684178+0000","last_became_peered":"2023-07-06T03:47:36.684178+0000","last_unstale":"2023-07-06T03:47:37.679530+0000","last_undegraded":"2023-07-06T03:47:37.679530+0000","last_fullsized":"2023-07-06T03:47:37.679530+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:54.714268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033517+0000","last_change":"2023-07-06T03:47:36.694324+0000","last_active":"2023-07-06T03:47:38.033517+0000","last_peered":"2023-07-06T03:47:38.033517+0000","last_clean":"2023-07-06T03:47:38.033517+0000","last_became_active":"2023-07-06T03:47:36.694042+0000","last_became_peered":"2023-07-06T03:47:36.694042+0000","last_unstale":"2023-07-06T03:47:38.033517+0000","last_undegraded":"2023-07-06T03:47:38.033517+0000","last_fullsized":"2023-07-06T03:47:38.033517+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:14:47.155587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031051+0000","last_change":"2023-07-06T03:47:36.679549+0000","last_active":"2023-07-06T03:47:38.031051+0000","last_peered":"2023-07-06T03:47:38.031051+0000","last_clean":"2023-07-06T03:47:38.031051+0000","last_became_active":"2023-07-06T03:47:36.679270+0000","last_became_peered":"2023-07-06T03:47:36.679270+0000","last_unstale":"2023-07-06T03:47:38.031051+0000","last_undegraded":"2023-07-06T03:47:38.031051+0000","last_fullsized":"2023-07-06T03:47:38.031051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:09.615630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679347+0000","last_change":"2023-07-06T03:47:36.684433+0000","last_active":"2023-07-06T03:47:37.679347+0000","last_peered":"2023-07-06T03:47:37.679347+0000","last_clean":"2023-07-06T03:47:37.679347+0000","last_became_active":"2023-07-06T03:47:36.684147+0000","last_became_peered":"2023-07-06T03:47:36.684147+0000","last_unstale":"2023-07-06T03:47:37.679347+0000","last_undegraded":"2023-07-06T03:47:37.679347+0000","last_fullsized":"2023-07-06T03:47:37.679347+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:31:19.105732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678751+0000","last_change":"2023-07-06T03:47:36.685535+0000","last_active":"2023-07-06T03:47:37.678751+0000","last_peered":"2023-07-06T03:47:37.678751+0000","last_clean":"2023-07-06T03:47:37.678751+0000","last_became_active":"2023-07-06T03:47:36.684607+0000","last_became_peered":"2023-07-06T03:47:36.684607+0000","last_unstale":"2023-07-06T03:47:37.678751+0000","last_undegraded":"2023-07-06T03:47:37.678751+0000","last_fullsized":"2023-07-06T03:47:37.678751+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:37:59.925688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032468+0000","last_change":"2023-07-06T03:47:36.692588+0000","last_active":"2023-07-06T03:47:38.032468+0000","last_peered":"2023-07-06T03:47:38.032468+0000","last_clean":"2023-07-06T03:47:38.032468+0000","last_became_active":"2023-07-06T03:47:36.692264+0000","last_became_peered":"2023-07-06T03:47:36.692264+0000","last_unstale":"2023-07-06T03:47:38.032468+0000","last_undegraded":"2023-07-06T03:47:38.032468+0000","last_fullsized":"2023-07-06T03:47:38.032468+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:44:23.679226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678405+0000","last_change":"2023-07-06T03:47:36.690548+0000","last_active":"2023-07-06T03:47:37.678405+0000","last_peered":"2023-07-06T03:47:37.678405+0000","last_clean":"2023-07-06T03:47:37.678405+0000","last_became_active":"2023-07-06T03:47:36.690269+0000","last_became_peered":"2023-07-06T03:47:36.690269+0000","last_unstale":"2023-07-06T03:47:37.678405+0000","last_undegraded":"2023-07-06T03:47:37.678405+0000","last_fullsized":"2023-07-06T03:47:37.678405+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:43.307999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032956+0000","last_change":"2023-07-06T03:47:36.696433+0000","last_active":"2023-07-06T03:47:38.032956+0000","last_peered":"2023-07-06T03:47:38.032956+0000","last_clean":"2023-07-06T03:47:38.032956+0000","last_became_active":"2023-07-06T03:47:36.696122+0000","last_became_peered":"2023-07-06T03:47:36.696122+0000","last_unstale":"2023-07-06T03:47:38.032956+0000","last_undegraded":"2023-07-06T03:47:38.032956+0000","last_fullsized":"2023-07-06T03:47:38.032956+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:07:25.581724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033354+0000","last_change":"2023-07-06T03:47:36.696522+0000","last_active":"2023-07-06T03:47:38.033354+0000","last_peered":"2023-07-06T03:47:38.033354+0000","last_clean":"2023-07-06T03:47:38.033354+0000","last_became_active":"2023-07-06T03:47:36.696078+0000","last_became_peered":"2023-07-06T03:47:36.696078+0000","last_unstale":"2023-07-06T03:47:38.033354+0000","last_undegraded":"2023-07-06T03:47:38.033354+0000","last_fullsized":"2023-07-06T03:47:38.033354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:44:02.670101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032125+0000","last_change":"2023-07-06T03:47:36.686755+0000","last_active":"2023-07-06T03:47:38.032125+0000","last_peered":"2023-07-06T03:47:38.032125+0000","last_clean":"2023-07-06T03:47:38.032125+0000","last_became_active":"2023-07-06T03:47:36.686499+0000","last_became_peered":"2023-07-06T03:47:36.686499+0000","last_unstale":"2023-07-06T03:47:38.032125+0000","last_undegraded":"2023-07-06T03:47:38.032125+0000","last_fullsized":"2023-07-06T03:47:38.032125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:40:35.757147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678770+0000","last_change":"2023-07-06T03:47:36.671270+0000","last_active":"2023-07-06T03:47:37.678770+0000","last_peered":"2023-07-06T03:47:37.678770+0000","last_clean":"2023-07-06T03:47:37.678770+0000","last_became_active":"2023-07-06T03:47:36.671009+0000","last_became_peered":"2023-07-06T03:47:36.671009+0000","last_unstale":"2023-07-06T03:47:37.678770+0000","last_undegraded":"2023-07-06T03:47:37.678770+0000","last_fullsized":"2023-07-06T03:47:37.678770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:20:42.997717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678752+0000","last_change":"2023-07-06T03:47:36.673999+0000","last_active":"2023-07-06T03:47:37.678752+0000","last_peered":"2023-07-06T03:47:37.678752+0000","last_clean":"2023-07-06T03:47:37.678752+0000","last_became_active":"2023-07-06T03:47:36.673684+0000","last_became_peered":"2023-07-06T03:47:36.673684+0000","last_unstale":"2023-07-06T03:47:37.678752+0000","last_undegraded":"2023-07-06T03:47:37.678752+0000","last_fullsized":"2023-07-06T03:47:37.678752+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:44:02.918959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031879+0000","last_change":"2023-07-06T03:47:36.691230+0000","last_active":"2023-07-06T03:47:38.031879+0000","last_peered":"2023-07-06T03:47:38.031879+0000","last_clean":"2023-07-06T03:47:38.031879+0000","last_became_active":"2023-07-06T03:47:36.690949+0000","last_became_peered":"2023-07-06T03:47:36.690949+0000","last_unstale":"2023-07-06T03:47:38.031879+0000","last_undegraded":"2023-07-06T03:47:38.031879+0000","last_fullsized":"2023-07-06T03:47:38.031879+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:44:02.670101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031076+0000","last_change":"2023-07-06T03:47:36.680280+0000","last_active":"2023-07-06T03:47:38.031076+0000","last_peered":"2023-07-06T03:47:38.031076+0000","last_clean":"2023-07-06T03:47:38.031076+0000","last_became_active":"2023-07-06T03:47:36.680012+0000","last_became_peered":"2023-07-06T03:47:36.680012+0000","last_unstale":"2023-07-06T03:47:38.031076+0000","last_undegraded":"2023-07-06T03:47:38.031076+0000","last_fullsized":"2023-07-06T03:47:38.031076+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:46:07.037740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678390+0000","last_change":"2023-07-06T03:47:36.673201+0000","last_active":"2023-07-06T03:47:37.678390+0000","last_peered":"2023-07-06T03:47:37.678390+0000","last_clean":"2023-07-06T03:47:37.678390+0000","last_became_active":"2023-07-06T03:47:36.672913+0000","last_became_peered":"2023-07-06T03:47:36.672913+0000","last_unstale":"2023-07-06T03:47:37.678390+0000","last_undegraded":"2023-07-06T03:47:37.678390+0000","last_fullsized":"2023-07-06T03:47:37.678390+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:45.025174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032476+0000","last_change":"2023-07-06T03:47:36.690227+0000","last_active":"2023-07-06T03:47:38.032476+0000","last_peered":"2023-07-06T03:47:38.032476+0000","last_clean":"2023-07-06T03:47:38.032476+0000","last_became_active":"2023-07-06T03:47:36.689751+0000","last_became_peered":"2023-07-06T03:47:36.689751+0000","last_unstale":"2023-07-06T03:47:38.032476+0000","last_undegraded":"2023-07-06T03:47:38.032476+0000","last_fullsized":"2023-07-06T03:47:38.032476+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:24:19.212787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031275+0000","last_change":"2023-07-06T03:47:36.679476+0000","last_active":"2023-07-06T03:47:38.031275+0000","last_peered":"2023-07-06T03:47:38.031275+0000","last_clean":"2023-07-06T03:47:38.031275+0000","last_became_active":"2023-07-06T03:47:36.679195+0000","last_became_peered":"2023-07-06T03:47:36.679195+0000","last_unstale":"2023-07-06T03:47:38.031275+0000","last_undegraded":"2023-07-06T03:47:38.031275+0000","last_fullsized":"2023-07-06T03:47:38.031275+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:52:04.996142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033580+0000","last_change":"2023-07-06T03:47:36.683417+0000","last_active":"2023-07-06T03:47:38.033580+0000","last_peered":"2023-07-06T03:47:38.033580+0000","last_clean":"2023-07-06T03:47:38.033580+0000","last_became_active":"2023-07-06T03:47:36.683107+0000","last_became_peered":"2023-07-06T03:47:36.683107+0000","last_unstale":"2023-07-06T03:47:38.033580+0000","last_undegraded":"2023-07-06T03:47:38.033580+0000","last_fullsized":"2023-07-06T03:47:38.033580+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:01:57.215705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679945+0000","last_change":"2023-07-06T03:47:36.685004+0000","last_active":"2023-07-06T03:47:37.679945+0000","last_peered":"2023-07-06T03:47:37.679945+0000","last_clean":"2023-07-06T03:47:37.679945+0000","last_became_active":"2023-07-06T03:47:36.684742+0000","last_became_peered":"2023-07-06T03:47:36.684742+0000","last_unstale":"2023-07-06T03:47:37.679945+0000","last_undegraded":"2023-07-06T03:47:37.679945+0000","last_fullsized":"2023-07-06T03:47:37.679945+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:12.068599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.030900+0000","last_change":"2023-07-06T03:47:36.680320+0000","last_active":"2023-07-06T03:47:38.030900+0000","last_peered":"2023-07-06T03:47:38.030900+0000","last_clean":"2023-07-06T03:47:38.030900+0000","last_became_active":"2023-07-06T03:47:36.680025+0000","last_became_peered":"2023-07-06T03:47:36.680025+0000","last_unstale":"2023-07-06T03:47:38.030900+0000","last_undegraded":"2023-07-06T03:47:38.030900+0000","last_fullsized":"2023-07-06T03:47:38.030900+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:07:55.161397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032634+0000","last_change":"2023-07-06T03:47:36.698971+0000","last_active":"2023-07-06T03:47:38.032634+0000","last_peered":"2023-07-06T03:47:38.032634+0000","last_clean":"2023-07-06T03:47:38.032634+0000","last_became_active":"2023-07-06T03:47:36.698714+0000","last_became_peered":"2023-07-06T03:47:36.698714+0000","last_unstale":"2023-07-06T03:47:38.032634+0000","last_undegraded":"2023-07-06T03:47:38.032634+0000","last_fullsized":"2023-07-06T03:47:38.032634+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:30:28.841133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031954+0000","last_change":"2023-07-06T03:47:36.679892+0000","last_active":"2023-07-06T03:47:38.031954+0000","last_peered":"2023-07-06T03:47:38.031954+0000","last_clean":"2023-07-06T03:47:38.031954+0000","last_became_active":"2023-07-06T03:47:36.679610+0000","last_became_peered":"2023-07-06T03:47:36.679610+0000","last_unstale":"2023-07-06T03:47:38.031954+0000","last_undegraded":"2023-07-06T03:47:38.031954+0000","last_fullsized":"2023-07-06T03:47:38.031954+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:20.150333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032597+0000","last_change":"2023-07-06T03:47:36.692786+0000","last_active":"2023-07-06T03:47:38.032597+0000","last_peered":"2023-07-06T03:47:38.032597+0000","last_clean":"2023-07-06T03:47:38.032597+0000","last_became_active":"2023-07-06T03:47:36.692480+0000","last_became_peered":"2023-07-06T03:47:36.692480+0000","last_unstale":"2023-07-06T03:47:38.032597+0000","last_undegraded":"2023-07-06T03:47:38.032597+0000","last_fullsized":"2023-07-06T03:47:38.032597+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:55:08.239557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032292+0000","last_change":"2023-07-06T03:47:36.697045+0000","last_active":"2023-07-06T03:47:38.032292+0000","last_peered":"2023-07-06T03:47:38.032292+0000","last_clean":"2023-07-06T03:47:38.032292+0000","last_became_active":"2023-07-06T03:47:36.696756+0000","last_became_peered":"2023-07-06T03:47:36.696756+0000","last_unstale":"2023-07-06T03:47:38.032292+0000","last_undegraded":"2023-07-06T03:47:38.032292+0000","last_fullsized":"2023-07-06T03:47:38.032292+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:56:37.217803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032542+0000","last_change":"2023-07-06T03:47:36.674863+0000","last_active":"2023-07-06T03:47:38.032542+0000","last_peered":"2023-07-06T03:47:38.032542+0000","last_clean":"2023-07-06T03:47:38.032542+0000","last_became_active":"2023-07-06T03:47:36.674576+0000","last_became_peered":"2023-07-06T03:47:36.674576+0000","last_unstale":"2023-07-06T03:47:38.032542+0000","last_undegraded":"2023-07-06T03:47:38.032542+0000","last_fullsized":"2023-07-06T03:47:38.032542+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:37:09.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679100+0000","last_change":"2023-07-06T03:47:36.691182+0000","last_active":"2023-07-06T03:47:37.679100+0000","last_peered":"2023-07-06T03:47:37.679100+0000","last_clean":"2023-07-06T03:47:37.679100+0000","last_became_active":"2023-07-06T03:47:36.690472+0000","last_became_peered":"2023-07-06T03:47:36.690472+0000","last_unstale":"2023-07-06T03:47:37.679100+0000","last_undegraded":"2023-07-06T03:47:37.679100+0000","last_fullsized":"2023-07-06T03:47:37.679100+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:00:22.356225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679806+0000","last_change":"2023-07-06T03:47:36.694072+0000","last_active":"2023-07-06T03:47:37.679806+0000","last_peered":"2023-07-06T03:47:37.679806+0000","last_clean":"2023-07-06T03:47:37.679806+0000","last_became_active":"2023-07-06T03:47:36.693790+0000","last_became_peered":"2023-07-06T03:47:36.693790+0000","last_unstale":"2023-07-06T03:47:37.679806+0000","last_undegraded":"2023-07-06T03:47:37.679806+0000","last_fullsized":"2023-07-06T03:47:37.679806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:03:49.636022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032568+0000","last_change":"2023-07-06T03:47:36.690908+0000","last_active":"2023-07-06T03:47:38.032568+0000","last_peered":"2023-07-06T03:47:38.032568+0000","last_clean":"2023-07-06T03:47:38.032568+0000","last_became_active":"2023-07-06T03:47:36.690513+0000","last_became_peered":"2023-07-06T03:47:36.690513+0000","last_unstale":"2023-07-06T03:47:38.032568+0000","last_undegraded":"2023-07-06T03:47:38.032568+0000","last_fullsized":"2023-07-06T03:47:38.032568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:56:56.281979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679889+0000","last_change":"2023-07-06T03:47:36.678674+0000","last_active":"2023-07-06T03:47:37.679889+0000","last_peered":"2023-07-06T03:47:37.679889+0000","last_clean":"2023-07-06T03:47:37.679889+0000","last_became_active":"2023-07-06T03:47:36.677990+0000","last_became_peered":"2023-07-06T03:47:36.677990+0000","last_unstale":"2023-07-06T03:47:37.679889+0000","last_undegraded":"2023-07-06T03:47:37.679889+0000","last_fullsized":"2023-07-06T03:47:37.679889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:49:42.465400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032108+0000","last_change":"2023-07-06T03:47:36.693968+0000","last_active":"2023-07-06T03:47:38.032108+0000","last_peered":"2023-07-06T03:47:38.032108+0000","last_clean":"2023-07-06T03:47:38.032108+0000","last_became_active":"2023-07-06T03:47:36.693687+0000","last_became_peered":"2023-07-06T03:47:36.693687+0000","last_unstale":"2023-07-06T03:47:38.032108+0000","last_undegraded":"2023-07-06T03:47:38.032108+0000","last_fullsized":"2023-07-06T03:47:38.032108+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:24:46.598193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678951+0000","last_change":"2023-07-06T03:47:36.690467+0000","last_active":"2023-07-06T03:47:37.678951+0000","last_peered":"2023-07-06T03:47:37.678951+0000","last_clean":"2023-07-06T03:47:37.678951+0000","last_became_active":"2023-07-06T03:47:36.689525+0000","last_became_peered":"2023-07-06T03:47:36.689525+0000","last_unstale":"2023-07-06T03:47:37.678951+0000","last_undegraded":"2023-07-06T03:47:37.678951+0000","last_fullsized":"2023-07-06T03:47:37.678951+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:13:56.051926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679425+0000","last_change":"2023-07-06T03:47:36.679202+0000","last_active":"2023-07-06T03:47:37.679425+0000","last_peered":"2023-07-06T03:47:37.679425+0000","last_clean":"2023-07-06T03:47:37.679425+0000","last_became_active":"2023-07-06T03:47:36.678019+0000","last_became_peered":"2023-07-06T03:47:36.678019+0000","last_unstale":"2023-07-06T03:47:37.679425+0000","last_undegraded":"2023-07-06T03:47:37.679425+0000","last_fullsized":"2023-07-06T03:47:37.679425+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:27:41.668800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031749+0000","last_change":"2023-07-06T03:47:35.684675+0000","last_active":"2023-07-06T03:47:38.031749+0000","last_peered":"2023-07-06T03:47:38.031749+0000","last_clean":"2023-07-06T03:47:38.031749+0000","last_became_active":"2023-07-06T03:47:35.684398+0000","last_became_peered":"2023-07-06T03:47:35.684398+0000","last_unstale":"2023-07-06T03:47:38.031749+0000","last_undegraded":"2023-07-06T03:47:38.031749+0000","last_fullsized":"2023-07-06T03:47:38.031749+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:03:08.897884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679576+0000","last_change":"2023-07-06T03:47:36.674737+0000","last_active":"2023-07-06T03:47:37.679576+0000","last_peered":"2023-07-06T03:47:37.679576+0000","last_clean":"2023-07-06T03:47:37.679576+0000","last_became_active":"2023-07-06T03:47:36.674426+0000","last_became_peered":"2023-07-06T03:47:36.674426+0000","last_unstale":"2023-07-06T03:47:37.679576+0000","last_undegraded":"2023-07-06T03:47:37.679576+0000","last_fullsized":"2023-07-06T03:47:37.679576+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:31.047623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"46'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679157+0000","last_change":"2023-07-06T03:47:35.680067+0000","last_active":"2023-07-06T03:47:37.679157+0000","last_peered":"2023-07-06T03:47:37.679157+0000","last_clean":"2023-07-06T03:47:37.679157+0000","last_became_active":"2023-07-06T03:47:35.679691+0000","last_became_peered":"2023-07-06T03:47:35.679691+0000","last_unstale":"2023-07-06T03:47:37.679157+0000","last_undegraded":"2023-07-06T03:47:37.679157+0000","last_fullsized":"2023-07-06T03:47:37.679157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:15:55.604018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680012+0000","last_change":"2023-07-06T03:47:36.678853+0000","last_active":"2023-07-06T03:47:37.680012+0000","last_peered":"2023-07-06T03:47:37.680012+0000","last_clean":"2023-07-06T03:47:37.680012+0000","last_became_active":"2023-07-06T03:47:36.678366+0000","last_became_peered":"2023-07-06T03:47:36.678366+0000","last_unstale":"2023-07-06T03:47:37.680012+0000","last_undegraded":"2023-07-06T03:47:37.680012+0000","last_fullsized":"2023-07-06T03:47:37.680012+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:50:56.264915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032772+0000","last_change":"2023-07-06T03:47:35.676405+0000","last_active":"2023-07-06T03:47:38.032772+0000","last_peered":"2023-07-06T03:47:38.032772+0000","last_clean":"2023-07-06T03:47:38.032772+0000","last_became_active":"2023-07-06T03:47:35.676089+0000","last_became_peered":"2023-07-06T03:47:35.676089+0000","last_unstale":"2023-07-06T03:47:38.032772+0000","last_undegraded":"2023-07-06T03:47:38.032772+0000","last_fullsized":"2023-07-06T03:47:38.032772+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:01:39.848129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033032+0000","last_change":"2023-07-06T03:47:36.688311+0000","last_active":"2023-07-06T03:47:38.033032+0000","last_peered":"2023-07-06T03:47:38.033032+0000","last_clean":"2023-07-06T03:47:38.033032+0000","last_became_active":"2023-07-06T03:47:36.687991+0000","last_became_peered":"2023-07-06T03:47:36.687991+0000","last_unstale":"2023-07-06T03:47:38.033032+0000","last_undegraded":"2023-07-06T03:47:38.033032+0000","last_fullsized":"2023-07-06T03:47:38.033032+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:03:09.900802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679842+0000","last_change":"2023-07-06T03:47:35.678850+0000","last_active":"2023-07-06T03:47:37.679842+0000","last_peered":"2023-07-06T03:47:37.679842+0000","last_clean":"2023-07-06T03:47:37.679842+0000","last_became_active":"2023-07-06T03:47:35.678520+0000","last_became_peered":"2023-07-06T03:47:35.678520+0000","last_unstale":"2023-07-06T03:47:37.679842+0000","last_undegraded":"2023-07-06T03:47:37.679842+0000","last_fullsized":"2023-07-06T03:47:37.679842+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:25:01.977539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031507+0000","last_change":"2023-07-06T03:47:36.684332+0000","last_active":"2023-07-06T03:47:38.031507+0000","last_peered":"2023-07-06T03:47:38.031507+0000","last_clean":"2023-07-06T03:47:38.031507+0000","last_became_active":"2023-07-06T03:47:36.684079+0000","last_became_peered":"2023-07-06T03:47:36.684079+0000","last_unstale":"2023-07-06T03:47:38.031507+0000","last_undegraded":"2023-07-06T03:47:38.031507+0000","last_fullsized":"2023-07-06T03:47:38.031507+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:31:04.312174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"46'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033224+0000","last_change":"2023-07-06T03:47:35.679264+0000","last_active":"2023-07-06T03:47:38.033224+0000","last_peered":"2023-07-06T03:47:38.033224+0000","last_clean":"2023-07-06T03:47:38.033224+0000","last_became_active":"2023-07-06T03:47:35.678950+0000","last_became_peered":"2023-07-06T03:47:35.678950+0000","last_unstale":"2023-07-06T03:47:38.033224+0000","last_undegraded":"2023-07-06T03:47:38.033224+0000","last_fullsized":"2023-07-06T03:47:38.033224+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:48:24.153481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679950+0000","last_change":"2023-07-06T03:47:36.682503+0000","last_active":"2023-07-06T03:47:37.679950+0000","last_peered":"2023-07-06T03:47:37.679950+0000","last_clean":"2023-07-06T03:47:37.679950+0000","last_became_active":"2023-07-06T03:47:36.682196+0000","last_became_peered":"2023-07-06T03:47:36.682196+0000","last_unstale":"2023-07-06T03:47:37.679950+0000","last_undegraded":"2023-07-06T03:47:37.679950+0000","last_fullsized":"2023-07-06T03:47:37.679950+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:01:29.045248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"46'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680034+0000","last_change":"2023-07-06T03:47:35.679463+0000","last_active":"2023-07-06T03:47:37.680034+0000","last_peered":"2023-07-06T03:47:37.680034+0000","last_clean":"2023-07-06T03:47:37.680034+0000","last_became_active":"2023-07-06T03:47:35.679249+0000","last_became_peered":"2023-07-06T03:47:35.679249+0000","last_unstale":"2023-07-06T03:47:37.680034+0000","last_undegraded":"2023-07-06T03:47:37.680034+0000","last_fullsized":"2023-07-06T03:47:37.680034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:45:31.979648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033187+0000","last_change":"2023-07-06T03:47:36.696351+0000","last_active":"2023-07-06T03:47:38.033187+0000","last_peered":"2023-07-06T03:47:38.033187+0000","last_clean":"2023-07-06T03:47:38.033187+0000","last_became_active":"2023-07-06T03:47:36.695783+0000","last_became_peered":"2023-07-06T03:47:36.695783+0000","last_unstale":"2023-07-06T03:47:38.033187+0000","last_undegraded":"2023-07-06T03:47:38.033187+0000","last_fullsized":"2023-07-06T03:47:38.033187+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:44:10.832855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679697+0000","last_change":"2023-07-06T03:47:35.684332+0000","last_active":"2023-07-06T03:47:37.679697+0000","last_peered":"2023-07-06T03:47:37.679697+0000","last_clean":"2023-07-06T03:47:37.679697+0000","last_became_active":"2023-07-06T03:47:35.684064+0000","last_became_peered":"2023-07-06T03:47:35.684064+0000","last_unstale":"2023-07-06T03:47:37.679697+0000","last_undegraded":"2023-07-06T03:47:37.679697+0000","last_fullsized":"2023-07-06T03:47:37.679697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:34:48.350570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031290+0000","last_change":"2023-07-06T03:47:36.674252+0000","last_active":"2023-07-06T03:47:38.031290+0000","last_peered":"2023-07-06T03:47:38.031290+0000","last_clean":"2023-07-06T03:47:38.031290+0000","last_became_active":"2023-07-06T03:47:36.673742+0000","last_became_peered":"2023-07-06T03:47:36.673742+0000","last_unstale":"2023-07-06T03:47:38.031290+0000","last_undegraded":"2023-07-06T03:47:38.031290+0000","last_fullsized":"2023-07-06T03:47:38.031290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:04:29.237378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031634+0000","last_change":"2023-07-06T03:47:35.679127+0000","last_active":"2023-07-06T03:47:38.031634+0000","last_peered":"2023-07-06T03:47:38.031634+0000","last_clean":"2023-07-06T03:47:38.031634+0000","last_became_active":"2023-07-06T03:47:35.678630+0000","last_became_peered":"2023-07-06T03:47:35.678630+0000","last_unstale":"2023-07-06T03:47:38.031634+0000","last_undegraded":"2023-07-06T03:47:38.031634+0000","last_fullsized":"2023-07-06T03:47:38.031634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:38:00.464893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.030782+0000","last_change":"2023-07-06T03:47:36.694845+0000","last_active":"2023-07-06T03:47:38.030782+0000","last_peered":"2023-07-06T03:47:38.030782+0000","last_clean":"2023-07-06T03:47:38.030782+0000","last_became_active":"2023-07-06T03:47:36.694409+0000","last_became_peered":"2023-07-06T03:47:36.694409+0000","last_unstale":"2023-07-06T03:47:38.030782+0000","last_undegraded":"2023-07-06T03:47:38.030782+0000","last_fullsized":"2023-07-06T03:47:38.030782+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:15.008580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031416+0000","last_change":"2023-07-06T03:47:35.678328+0000","last_active":"2023-07-06T03:47:38.031416+0000","last_peered":"2023-07-06T03:47:38.031416+0000","last_clean":"2023-07-06T03:47:38.031416+0000","last_became_active":"2023-07-06T03:47:35.678081+0000","last_became_peered":"2023-07-06T03:47:35.678081+0000","last_unstale":"2023-07-06T03:47:38.031416+0000","last_undegraded":"2023-07-06T03:47:38.031416+0000","last_fullsized":"2023-07-06T03:47:38.031416+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:06:42.075787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031093+0000","last_change":"2023-07-06T03:47:36.681930+0000","last_active":"2023-07-06T03:47:38.031093+0000","last_peered":"2023-07-06T03:47:38.031093+0000","last_clean":"2023-07-06T03:47:38.031093+0000","last_became_active":"2023-07-06T03:47:36.681625+0000","last_became_peered":"2023-07-06T03:47:36.681625+0000","last_unstale":"2023-07-06T03:47:38.031093+0000","last_undegraded":"2023-07-06T03:47:38.031093+0000","last_fullsized":"2023-07-06T03:47:38.031093+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:15:21.433427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031261+0000","last_change":"2023-07-06T03:47:35.673408+0000","last_active":"2023-07-06T03:47:38.031261+0000","last_peered":"2023-07-06T03:47:38.031261+0000","last_clean":"2023-07-06T03:47:38.031261+0000","last_became_active":"2023-07-06T03:47:35.672993+0000","last_became_peered":"2023-07-06T03:47:35.672993+0000","last_unstale":"2023-07-06T03:47:38.031261+0000","last_undegraded":"2023-07-06T03:47:38.031261+0000","last_fullsized":"2023-07-06T03:47:38.031261+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:24.596966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032055+0000","last_change":"2023-07-06T03:47:36.687284+0000","last_active":"2023-07-06T03:47:38.032055+0000","last_peered":"2023-07-06T03:47:38.032055+0000","last_clean":"2023-07-06T03:47:38.032055+0000","last_became_active":"2023-07-06T03:47:36.687017+0000","last_became_peered":"2023-07-06T03:47:36.687017+0000","last_unstale":"2023-07-06T03:47:38.032055+0000","last_undegraded":"2023-07-06T03:47:38.032055+0000","last_fullsized":"2023-07-06T03:47:38.032055+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:25:47.526435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680080+0000","last_change":"2023-07-06T03:47:35.679946+0000","last_active":"2023-07-06T03:47:37.680080+0000","last_peered":"2023-07-06T03:47:37.680080+0000","last_clean":"2023-07-06T03:47:37.680080+0000","last_became_active":"2023-07-06T03:47:35.679487+0000","last_became_peered":"2023-07-06T03:47:35.679487+0000","last_unstale":"2023-07-06T03:47:37.680080+0000","last_undegraded":"2023-07-06T03:47:37.680080+0000","last_fullsized":"2023-07-06T03:47:37.680080+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:20:35.484864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679291+0000","last_change":"2023-07-06T03:47:36.694018+0000","last_active":"2023-07-06T03:47:37.679291+0000","last_peered":"2023-07-06T03:47:37.679291+0000","last_clean":"2023-07-06T03:47:37.679291+0000","last_became_active":"2023-07-06T03:47:36.693840+0000","last_became_peered":"2023-07-06T03:47:36.693840+0000","last_unstale":"2023-07-06T03:47:37.679291+0000","last_undegraded":"2023-07-06T03:47:37.679291+0000","last_fullsized":"2023-07-06T03:47:37.679291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:15:40.828483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"46'6","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033239+0000","last_change":"2023-07-06T03:47:35.679499+0000","last_active":"2023-07-06T03:47:38.033239+0000","last_peered":"2023-07-06T03:47:38.033239+0000","last_clean":"2023-07-06T03:47:38.033239+0000","last_became_active":"2023-07-06T03:47:35.679153+0000","last_became_peered":"2023-07-06T03:47:35.679153+0000","last_unstale":"2023-07-06T03:47:38.033239+0000","last_undegraded":"2023-07-06T03:47:38.033239+0000","last_fullsized":"2023-07-06T03:47:38.033239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:52:44.393837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679691+0000","last_change":"2023-07-06T03:47:36.683371+0000","last_active":"2023-07-06T03:47:37.679691+0000","last_peered":"2023-07-06T03:47:37.679691+0000","last_clean":"2023-07-06T03:47:37.679691+0000","last_became_active":"2023-07-06T03:47:36.683062+0000","last_became_peered":"2023-07-06T03:47:36.683062+0000","last_unstale":"2023-07-06T03:47:37.679691+0000","last_undegraded":"2023-07-06T03:47:37.679691+0000","last_fullsized":"2023-07-06T03:47:37.679691+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:50:27.671381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679071+0000","last_change":"2023-07-06T03:47:35.675074+0000","last_active":"2023-07-06T03:47:37.679071+0000","last_peered":"2023-07-06T03:47:37.679071+0000","last_clean":"2023-07-06T03:47:37.679071+0000","last_became_active":"2023-07-06T03:47:35.674815+0000","last_became_peered":"2023-07-06T03:47:35.674815+0000","last_unstale":"2023-07-06T03:47:37.679071+0000","last_undegraded":"2023-07-06T03:47:37.679071+0000","last_fullsized":"2023-07-06T03:47:37.679071+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:24:18.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031965+0000","last_change":"2023-07-06T03:47:36.690771+0000","last_active":"2023-07-06T03:47:38.031965+0000","last_peered":"2023-07-06T03:47:38.031965+0000","last_clean":"2023-07-06T03:47:38.031965+0000","last_became_active":"2023-07-06T03:47:36.690400+0000","last_became_peered":"2023-07-06T03:47:36.690400+0000","last_unstale":"2023-07-06T03:47:38.031965+0000","last_undegraded":"2023-07-06T03:47:38.031965+0000","last_fullsized":"2023-07-06T03:47:38.031965+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:13.208271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031094+0000","last_change":"2023-07-06T03:47:35.680141+0000","last_active":"2023-07-06T03:47:38.031094+0000","last_peered":"2023-07-06T03:47:38.031094+0000","last_clean":"2023-07-06T03:47:38.031094+0000","last_became_active":"2023-07-06T03:47:35.679775+0000","last_became_peered":"2023-07-06T03:47:35.679775+0000","last_unstale":"2023-07-06T03:47:38.031094+0000","last_undegraded":"2023-07-06T03:47:38.031094+0000","last_fullsized":"2023-07-06T03:47:38.031094+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:55:07.602533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679006+0000","last_change":"2023-07-06T03:47:36.685315+0000","last_active":"2023-07-06T03:47:37.679006+0000","last_peered":"2023-07-06T03:47:37.679006+0000","last_clean":"2023-07-06T03:47:37.679006+0000","last_became_active":"2023-07-06T03:47:36.684988+0000","last_became_peered":"2023-07-06T03:47:36.684988+0000","last_unstale":"2023-07-06T03:47:37.679006+0000","last_undegraded":"2023-07-06T03:47:37.679006+0000","last_fullsized":"2023-07-06T03:47:37.679006+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:48:06.816404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679046+0000","last_change":"2023-07-06T03:47:35.676310+0000","last_active":"2023-07-06T03:47:37.679046+0000","last_peered":"2023-07-06T03:47:37.679046+0000","last_clean":"2023-07-06T03:47:37.679046+0000","last_became_active":"2023-07-06T03:47:35.676038+0000","last_became_peered":"2023-07-06T03:47:35.676038+0000","last_unstale":"2023-07-06T03:47:37.679046+0000","last_undegraded":"2023-07-06T03:47:37.679046+0000","last_fullsized":"2023-07-06T03:47:37.679046+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:25:18.433007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033115+0000","last_change":"2023-07-06T03:47:36.672749+0000","last_active":"2023-07-06T03:47:38.033115+0000","last_peered":"2023-07-06T03:47:38.033115+0000","last_clean":"2023-07-06T03:47:38.033115+0000","last_became_active":"2023-07-06T03:47:36.672454+0000","last_became_peered":"2023-07-06T03:47:36.672454+0000","last_unstale":"2023-07-06T03:47:38.033115+0000","last_undegraded":"2023-07-06T03:47:38.033115+0000","last_fullsized":"2023-07-06T03:47:38.033115+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:53:31.746241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"46'8","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678585+0000","last_change":"2023-07-06T03:47:35.683270+0000","last_active":"2023-07-06T03:47:37.678585+0000","last_peered":"2023-07-06T03:47:37.678585+0000","last_clean":"2023-07-06T03:47:37.678585+0000","last_became_active":"2023-07-06T03:47:35.682973+0000","last_became_peered":"2023-07-06T03:47:35.682973+0000","last_unstale":"2023-07-06T03:47:37.678585+0000","last_undegraded":"2023-07-06T03:47:37.678585+0000","last_fullsized":"2023-07-06T03:47:37.678585+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:49:53.695009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031953+0000","last_change":"2023-07-06T03:47:36.690955+0000","last_active":"2023-07-06T03:47:38.031953+0000","last_peered":"2023-07-06T03:47:38.031953+0000","last_clean":"2023-07-06T03:47:38.031953+0000","last_became_active":"2023-07-06T03:47:36.690689+0000","last_became_peered":"2023-07-06T03:47:36.690689+0000","last_unstale":"2023-07-06T03:47:38.031953+0000","last_undegraded":"2023-07-06T03:47:38.031953+0000","last_fullsized":"2023-07-06T03:47:38.031953+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:23.949883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"46'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679233+0000","last_change":"2023-07-06T03:47:35.680067+0000","last_active":"2023-07-06T03:47:37.679233+0000","last_peered":"2023-07-06T03:47:37.679233+0000","last_clean":"2023-07-06T03:47:37.679233+0000","last_became_active":"2023-07-06T03:47:35.679704+0000","last_became_peered":"2023-07-06T03:47:35.679704+0000","last_unstale":"2023-07-06T03:47:37.679233+0000","last_undegraded":"2023-07-06T03:47:37.679233+0000","last_fullsized":"2023-07-06T03:47:37.679233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:56:02.683970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032048+0000","last_change":"2023-07-06T03:47:36.696608+0000","last_active":"2023-07-06T03:47:38.032048+0000","last_peered":"2023-07-06T03:47:38.032048+0000","last_clean":"2023-07-06T03:47:38.032048+0000","last_became_active":"2023-07-06T03:47:36.696241+0000","last_became_peered":"2023-07-06T03:47:36.696241+0000","last_unstale":"2023-07-06T03:47:38.032048+0000","last_undegraded":"2023-07-06T03:47:38.032048+0000","last_fullsized":"2023-07-06T03:47:38.032048+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:21:46.455954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679314+0000","last_change":"2023-07-06T03:47:35.680931+0000","last_active":"2023-07-06T03:47:37.679314+0000","last_peered":"2023-07-06T03:47:37.679314+0000","last_clean":"2023-07-06T03:47:37.679314+0000","last_became_active":"2023-07-06T03:47:35.680659+0000","last_became_peered":"2023-07-06T03:47:35.680659+0000","last_unstale":"2023-07-06T03:47:37.679314+0000","last_undegraded":"2023-07-06T03:47:37.679314+0000","last_fullsized":"2023-07-06T03:47:37.679314+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:12:30.786821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679218+0000","last_change":"2023-07-06T03:47:36.687780+0000","last_active":"2023-07-06T03:47:37.679218+0000","last_peered":"2023-07-06T03:47:37.679218+0000","last_clean":"2023-07-06T03:47:37.679218+0000","last_became_active":"2023-07-06T03:47:36.687490+0000","last_became_peered":"2023-07-06T03:47:36.687490+0000","last_unstale":"2023-07-06T03:47:37.679218+0000","last_undegraded":"2023-07-06T03:47:37.679218+0000","last_fullsized":"2023-07-06T03:47:37.679218+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:18:25.341440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031552+0000","last_change":"2023-07-06T03:47:35.679503+0000","last_active":"2023-07-06T03:47:38.031552+0000","last_peered":"2023-07-06T03:47:38.031552+0000","last_clean":"2023-07-06T03:47:38.031552+0000","last_became_active":"2023-07-06T03:47:35.679231+0000","last_became_peered":"2023-07-06T03:47:35.679231+0000","last_unstale":"2023-07-06T03:47:38.031552+0000","last_undegraded":"2023-07-06T03:47:38.031552+0000","last_fullsized":"2023-07-06T03:47:38.031552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:04:59.473861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679107+0000","last_change":"2023-07-06T03:47:36.699756+0000","last_active":"2023-07-06T03:47:37.679107+0000","last_peered":"2023-07-06T03:47:37.679107+0000","last_clean":"2023-07-06T03:47:37.679107+0000","last_became_active":"2023-07-06T03:47:36.699459+0000","last_became_peered":"2023-07-06T03:47:36.699459+0000","last_unstale":"2023-07-06T03:47:37.679107+0000","last_undegraded":"2023-07-06T03:47:37.679107+0000","last_fullsized":"2023-07-06T03:47:37.679107+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:02:12.017358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679542+0000","last_change":"2023-07-06T03:47:35.685943+0000","last_active":"2023-07-06T03:47:37.679542+0000","last_peered":"2023-07-06T03:47:37.679542+0000","last_clean":"2023-07-06T03:47:37.679542+0000","last_became_active":"2023-07-06T03:47:35.685673+0000","last_became_peered":"2023-07-06T03:47:35.685673+0000","last_unstale":"2023-07-06T03:47:37.679542+0000","last_undegraded":"2023-07-06T03:47:37.679542+0000","last_fullsized":"2023-07-06T03:47:37.679542+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:06:36.141721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679682+0000","last_change":"2023-07-06T03:47:36.685295+0000","last_active":"2023-07-06T03:47:37.679682+0000","last_peered":"2023-07-06T03:47:37.679682+0000","last_clean":"2023-07-06T03:47:37.679682+0000","last_became_active":"2023-07-06T03:47:36.685009+0000","last_became_peered":"2023-07-06T03:47:36.685009+0000","last_unstale":"2023-07-06T03:47:37.679682+0000","last_undegraded":"2023-07-06T03:47:37.679682+0000","last_fullsized":"2023-07-06T03:47:37.679682+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:38:46.257565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032679+0000","last_change":"2023-07-06T03:47:35.684207+0000","last_active":"2023-07-06T03:47:38.032679+0000","last_peered":"2023-07-06T03:47:38.032679+0000","last_clean":"2023-07-06T03:47:38.032679+0000","last_became_active":"2023-07-06T03:47:35.683837+0000","last_became_peered":"2023-07-06T03:47:35.683837+0000","last_unstale":"2023-07-06T03:47:38.032679+0000","last_undegraded":"2023-07-06T03:47:38.032679+0000","last_fullsized":"2023-07-06T03:47:38.032679+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:08:16.691892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031458+0000","last_change":"2023-07-06T03:47:36.680438+0000","last_active":"2023-07-06T03:47:38.031458+0000","last_peered":"2023-07-06T03:47:38.031458+0000","last_clean":"2023-07-06T03:47:38.031458+0000","last_became_active":"2023-07-06T03:47:36.680224+0000","last_became_peered":"2023-07-06T03:47:36.680224+0000","last_unstale":"2023-07-06T03:47:38.031458+0000","last_undegraded":"2023-07-06T03:47:38.031458+0000","last_fullsized":"2023-07-06T03:47:38.031458+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:12:47.040859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032770+0000","last_change":"2023-07-06T03:47:35.680503+0000","last_active":"2023-07-06T03:47:38.032770+0000","last_peered":"2023-07-06T03:47:38.032770+0000","last_clean":"2023-07-06T03:47:38.032770+0000","last_became_active":"2023-07-06T03:47:35.680171+0000","last_became_peered":"2023-07-06T03:47:35.680171+0000","last_unstale":"2023-07-06T03:47:38.032770+0000","last_undegraded":"2023-07-06T03:47:38.032770+0000","last_fullsized":"2023-07-06T03:47:38.032770+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:14:45.275656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032362+0000","last_change":"2023-07-06T03:47:36.687514+0000","last_active":"2023-07-06T03:47:38.032362+0000","last_peered":"2023-07-06T03:47:38.032362+0000","last_clean":"2023-07-06T03:47:38.032362+0000","last_became_active":"2023-07-06T03:47:36.686771+0000","last_became_peered":"2023-07-06T03:47:36.686771+0000","last_unstale":"2023-07-06T03:47:38.032362+0000","last_undegraded":"2023-07-06T03:47:38.032362+0000","last_fullsized":"2023-07-06T03:47:38.032362+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:49:51.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032621+0000","last_change":"2023-07-06T03:47:35.680421+0000","last_active":"2023-07-06T03:47:38.032621+0000","last_peered":"2023-07-06T03:47:38.032621+0000","last_clean":"2023-07-06T03:47:38.032621+0000","last_became_active":"2023-07-06T03:47:35.679465+0000","last_became_peered":"2023-07-06T03:47:35.679465+0000","last_unstale":"2023-07-06T03:47:38.032621+0000","last_undegraded":"2023-07-06T03:47:38.032621+0000","last_fullsized":"2023-07-06T03:47:38.032621+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:14.005662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679191+0000","last_change":"2023-07-06T03:47:36.697621+0000","last_active":"2023-07-06T03:47:37.679191+0000","last_peered":"2023-07-06T03:47:37.679191+0000","last_clean":"2023-07-06T03:47:37.679191+0000","last_became_active":"2023-07-06T03:47:36.697357+0000","last_became_peered":"2023-07-06T03:47:36.697357+0000","last_unstale":"2023-07-06T03:47:37.679191+0000","last_undegraded":"2023-07-06T03:47:37.679191+0000","last_fullsized":"2023-07-06T03:47:37.679191+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:27:34.021470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032083+0000","last_change":"2023-07-06T03:47:35.673523+0000","last_active":"2023-07-06T03:47:38.032083+0000","last_peered":"2023-07-06T03:47:38.032083+0000","last_clean":"2023-07-06T03:47:38.032083+0000","last_became_active":"2023-07-06T03:47:35.672470+0000","last_became_peered":"2023-07-06T03:47:35.672470+0000","last_unstale":"2023-07-06T03:47:38.032083+0000","last_undegraded":"2023-07-06T03:47:38.032083+0000","last_fullsized":"2023-07-06T03:47:38.032083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:57:41.537850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033500+0000","last_change":"2023-07-06T03:47:36.696815+0000","last_active":"2023-07-06T03:47:38.033500+0000","last_peered":"2023-07-06T03:47:38.033500+0000","last_clean":"2023-07-06T03:47:38.033500+0000","last_became_active":"2023-07-06T03:47:36.695497+0000","last_became_peered":"2023-07-06T03:47:36.695497+0000","last_unstale":"2023-07-06T03:47:38.033500+0000","last_undegraded":"2023-07-06T03:47:38.033500+0000","last_fullsized":"2023-07-06T03:47:38.033500+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:13.208271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679107+0000","last_change":"2023-07-06T03:47:35.681014+0000","last_active":"2023-07-06T03:47:37.679107+0000","last_peered":"2023-07-06T03:47:37.679107+0000","last_clean":"2023-07-06T03:47:37.679107+0000","last_became_active":"2023-07-06T03:47:35.680766+0000","last_became_peered":"2023-07-06T03:47:35.680766+0000","last_unstale":"2023-07-06T03:47:37.679107+0000","last_undegraded":"2023-07-06T03:47:37.679107+0000","last_fullsized":"2023-07-06T03:47:37.679107+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:50:57.196031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033339+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:47:38.033339+0000","last_peered":"2023-07-06T03:47:38.033339+0000","last_clean":"2023-07-06T03:47:38.033339+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:47:38.033339+0000","last_undegraded":"2023-07-06T03:47:38.033339+0000","last_fullsized":"2023-07-06T03:47:38.033339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:56:55.588115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.030897+0000","last_change":"2023-07-06T03:47:36.690938+0000","last_active":"2023-07-06T03:47:38.030897+0000","last_peered":"2023-07-06T03:47:38.030897+0000","last_clean":"2023-07-06T03:47:38.030897+0000","last_became_active":"2023-07-06T03:47:36.690670+0000","last_became_peered":"2023-07-06T03:47:36.690670+0000","last_unstale":"2023-07-06T03:47:38.030897+0000","last_undegraded":"2023-07-06T03:47:38.030897+0000","last_fullsized":"2023-07-06T03:47:38.030897+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:32:43.768173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679494+0000","last_change":"2023-07-06T03:47:35.684268+0000","last_active":"2023-07-06T03:47:37.679494+0000","last_peered":"2023-07-06T03:47:37.679494+0000","last_clean":"2023-07-06T03:47:37.679494+0000","last_became_active":"2023-07-06T03:47:35.683961+0000","last_became_peered":"2023-07-06T03:47:35.683961+0000","last_unstale":"2023-07-06T03:47:37.679494+0000","last_undegraded":"2023-07-06T03:47:37.679494+0000","last_fullsized":"2023-07-06T03:47:37.679494+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:24:07.434670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680123+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:47:37.680123+0000","last_peered":"2023-07-06T03:47:37.680123+0000","last_clean":"2023-07-06T03:47:37.680123+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:47:37.680123+0000","last_undegraded":"2023-07-06T03:47:37.680123+0000","last_fullsized":"2023-07-06T03:47:37.680123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:52.349652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033210+0000","last_change":"2023-07-06T03:47:36.681576+0000","last_active":"2023-07-06T03:47:38.033210+0000","last_peered":"2023-07-06T03:47:38.033210+0000","last_clean":"2023-07-06T03:47:38.033210+0000","last_became_active":"2023-07-06T03:47:36.680747+0000","last_became_peered":"2023-07-06T03:47:36.680747+0000","last_unstale":"2023-07-06T03:47:38.033210+0000","last_undegraded":"2023-07-06T03:47:38.033210+0000","last_fullsized":"2023-07-06T03:47:38.033210+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:16:03.409941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033433+0000","last_change":"2023-07-06T03:47:35.677576+0000","last_active":"2023-07-06T03:47:38.033433+0000","last_peered":"2023-07-06T03:47:38.033433+0000","last_clean":"2023-07-06T03:47:38.033433+0000","last_became_active":"2023-07-06T03:47:35.677247+0000","last_became_peered":"2023-07-06T03:47:35.677247+0000","last_unstale":"2023-07-06T03:47:38.033433+0000","last_undegraded":"2023-07-06T03:47:38.033433+0000","last_fullsized":"2023-07-06T03:47:38.033433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:28:55.104375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678517+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:47:37.678517+0000","last_peered":"2023-07-06T03:47:37.678517+0000","last_clean":"2023-07-06T03:47:37.678517+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:47:37.678517+0000","last_undegraded":"2023-07-06T03:47:37.678517+0000","last_fullsized":"2023-07-06T03:47:37.678517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:47.962728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.030800+0000","last_change":"2023-07-06T03:47:36.680474+0000","last_active":"2023-07-06T03:47:38.030800+0000","last_peered":"2023-07-06T03:47:38.030800+0000","last_clean":"2023-07-06T03:47:38.030800+0000","last_became_active":"2023-07-06T03:47:36.680211+0000","last_became_peered":"2023-07-06T03:47:36.680211+0000","last_unstale":"2023-07-06T03:47:38.030800+0000","last_undegraded":"2023-07-06T03:47:38.030800+0000","last_fullsized":"2023-07-06T03:47:38.030800+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:01:40.851047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033583+0000","last_change":"2023-07-06T03:47:35.677003+0000","last_active":"2023-07-06T03:47:38.033583+0000","last_peered":"2023-07-06T03:47:38.033583+0000","last_clean":"2023-07-06T03:47:38.033583+0000","last_became_active":"2023-07-06T03:47:35.676727+0000","last_became_peered":"2023-07-06T03:47:35.676727+0000","last_unstale":"2023-07-06T03:47:38.033583+0000","last_undegraded":"2023-07-06T03:47:38.033583+0000","last_fullsized":"2023-07-06T03:47:38.033583+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:25:46.523518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679884+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:47:37.679884+0000","last_peered":"2023-07-06T03:47:37.679884+0000","last_clean":"2023-07-06T03:47:37.679884+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:47:37.679884+0000","last_undegraded":"2023-07-06T03:47:37.679884+0000","last_fullsized":"2023-07-06T03:47:37.679884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:05.435850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":210,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031054+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:47:38.031054+0000","last_peered":"2023-07-06T03:47:38.031054+0000","last_clean":"2023-07-06T03:47:38.031054+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:47:38.031054+0000","last_undegraded":"2023-07-06T03:47:38.031054+0000","last_fullsized":"2023-07-06T03:47:38.031054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:06:20.238158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032908+0000","last_change":"2023-07-06T03:47:36.688459+0000","last_active":"2023-07-06T03:47:38.032908+0000","last_peered":"2023-07-06T03:47:38.032908+0000","last_clean":"2023-07-06T03:47:38.032908+0000","last_became_active":"2023-07-06T03:47:36.688163+0000","last_became_peered":"2023-07-06T03:47:36.688163+0000","last_unstale":"2023-07-06T03:47:38.032908+0000","last_undegraded":"2023-07-06T03:47:38.032908+0000","last_fullsized":"2023-07-06T03:47:38.032908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:31:04.312174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680109+0000","last_change":"2023-07-06T03:47:35.679794+0000","last_active":"2023-07-06T03:47:37.680109+0000","last_peered":"2023-07-06T03:47:37.680109+0000","last_clean":"2023-07-06T03:47:37.680109+0000","last_became_active":"2023-07-06T03:47:35.679116+0000","last_became_peered":"2023-07-06T03:47:35.679116+0000","last_unstale":"2023-07-06T03:47:37.680109+0000","last_undegraded":"2023-07-06T03:47:37.680109+0000","last_fullsized":"2023-07-06T03:47:37.680109+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:25:58.524698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679414+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:47:37.679414+0000","last_peered":"2023-07-06T03:47:37.679414+0000","last_clean":"2023-07-06T03:47:37.679414+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:47:37.679414+0000","last_undegraded":"2023-07-06T03:47:37.679414+0000","last_fullsized":"2023-07-06T03:47:37.679414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:10.714832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678973+0000","last_change":"2023-07-06T03:47:36.691134+0000","last_active":"2023-07-06T03:47:37.678973+0000","last_peered":"2023-07-06T03:47:37.678973+0000","last_clean":"2023-07-06T03:47:37.678973+0000","last_became_active":"2023-07-06T03:47:36.690881+0000","last_became_peered":"2023-07-06T03:47:36.690881+0000","last_unstale":"2023-07-06T03:47:37.678973+0000","last_undegraded":"2023-07-06T03:47:37.678973+0000","last_fullsized":"2023-07-06T03:47:37.678973+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:40:50.948093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031070+0000","last_change":"2023-07-06T03:47:35.684944+0000","last_active":"2023-07-06T03:47:38.031070+0000","last_peered":"2023-07-06T03:47:38.031070+0000","last_clean":"2023-07-06T03:47:38.031070+0000","last_became_active":"2023-07-06T03:47:35.684654+0000","last_became_peered":"2023-07-06T03:47:35.684654+0000","last_unstale":"2023-07-06T03:47:38.031070+0000","last_undegraded":"2023-07-06T03:47:38.031070+0000","last_fullsized":"2023-07-06T03:47:38.031070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:22:20.323454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680206+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:47:37.680206+0000","last_peered":"2023-07-06T03:47:37.680206+0000","last_clean":"2023-07-06T03:47:37.680206+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:47:37.680206+0000","last_undegraded":"2023-07-06T03:47:37.680206+0000","last_fullsized":"2023-07-06T03:47:37.680206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:30:57.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033112+0000","last_change":"2023-07-06T03:47:36.697417+0000","last_active":"2023-07-06T03:47:38.033112+0000","last_peered":"2023-07-06T03:47:38.033112+0000","last_clean":"2023-07-06T03:47:38.033112+0000","last_became_active":"2023-07-06T03:47:36.693078+0000","last_became_peered":"2023-07-06T03:47:36.693078+0000","last_unstale":"2023-07-06T03:47:38.033112+0000","last_undegraded":"2023-07-06T03:47:38.033112+0000","last_fullsized":"2023-07-06T03:47:38.033112+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:12:57.654204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033166+0000","last_change":"2023-07-06T03:47:35.677115+0000","last_active":"2023-07-06T03:47:38.033166+0000","last_peered":"2023-07-06T03:47:38.033166+0000","last_clean":"2023-07-06T03:47:38.033166+0000","last_became_active":"2023-07-06T03:47:35.676826+0000","last_became_peered":"2023-07-06T03:47:35.676826+0000","last_unstale":"2023-07-06T03:47:38.033166+0000","last_undegraded":"2023-07-06T03:47:38.033166+0000","last_fullsized":"2023-07-06T03:47:38.033166+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:19:18.723847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032695+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:47:38.032695+0000","last_peered":"2023-07-06T03:47:38.032695+0000","last_clean":"2023-07-06T03:47:38.032695+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:47:38.032695+0000","last_undegraded":"2023-07-06T03:47:38.032695+0000","last_fullsized":"2023-07-06T03:47:38.032695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:19.297290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679641+0000","last_change":"2023-07-06T03:47:36.688377+0000","last_active":"2023-07-06T03:47:37.679641+0000","last_peered":"2023-07-06T03:47:37.679641+0000","last_clean":"2023-07-06T03:47:37.679641+0000","last_became_active":"2023-07-06T03:47:36.688094+0000","last_became_peered":"2023-07-06T03:47:36.688094+0000","last_unstale":"2023-07-06T03:47:37.679641+0000","last_undegraded":"2023-07-06T03:47:37.679641+0000","last_fullsized":"2023-07-06T03:47:37.679641+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:31:34.132094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032932+0000","last_change":"2023-07-06T03:47:35.680550+0000","last_active":"2023-07-06T03:47:38.032932+0000","last_peered":"2023-07-06T03:47:38.032932+0000","last_clean":"2023-07-06T03:47:38.032932+0000","last_became_active":"2023-07-06T03:47:35.680201+0000","last_became_peered":"2023-07-06T03:47:35.680201+0000","last_unstale":"2023-07-06T03:47:38.032932+0000","last_undegraded":"2023-07-06T03:47:38.032932+0000","last_fullsized":"2023-07-06T03:47:38.032932+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:40:39.427226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679619+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:47:37.679619+0000","last_peered":"2023-07-06T03:47:37.679619+0000","last_clean":"2023-07-06T03:47:37.679619+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:47:37.679619+0000","last_undegraded":"2023-07-06T03:47:37.679619+0000","last_fullsized":"2023-07-06T03:47:37.679619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:26:16.501757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":561152,"data_stored":237108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86399999999999999,"5min":1.345,"15min":1.345},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.81499999999999995,"15min":0.83699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91100000000000003,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.748,"15min":0.76300000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85799999999999998,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.91800000000000004,"15min":0.95899999999999996},"min":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.876,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.82399999999999995,"15min":0.84699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":3.77,"15min":3.77},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66300000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88400000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95699999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.626}]},{"osd":4,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.94699999999999995,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.77200000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96599999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.499},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.64100000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.746,"15min":0.746},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98799999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92800000000000005,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.75}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":557056,"data_stored":237634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.79400000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74199999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83899999999999997,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.78600000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77800000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.44900000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.874,"5min":1.119,"15min":1.119},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.76500000000000001,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.73999999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78200000000000003,"5min":1.093,"15min":1.093},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93500000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.746,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.81000000000000005,"5min":1.085,"15min":1.085},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72599999999999998,"5min":1.218,"15min":1.218},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.80200000000000005,"15min":0.81899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0269999999999999,"5min":1.198,"15min":1.198},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.84299999999999997,"15min":0.86699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.83799999999999997,"5min":1.323,"15min":1.323},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.84899999999999998,"15min":0.877},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86799999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1150976,"data_stored":827490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58399999999999996,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.31},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.58999999999999997,"15min":0.60999999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.628,"5min":1.96,"15min":1.96},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":1.0229999999999999,"15min":1.119},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54600000000000004,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.65800000000000003,"15min":0.68500000000000005},"min":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67900000000000005,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.38800000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.63400000000000001,"15min":0.65600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.752,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.437},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.59499999999999997,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":1.2789999999999999,"15min":1.375},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86199999999999999},"min":{"1min":0.48099999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94299999999999995,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.78500000000000003,"15min":0.80100000000000005},"min":{"1min":0.54100000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92100000000000004,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.2929999999999999,"15min":1.393},"min":{"1min":0.55900000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.874,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.60599999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.90600000000000003,"5min":1.365,"15min":1.365},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.335,"15min":1.4370000000000001},"min":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.88500000000000001,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.84899999999999998,"15min":0.875},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.86299999999999999,"5min":2.806,"15min":2.806},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82999999999999996,"15min":0.84599999999999997},"min":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.93000000000000005,"5min":1.337,"15min":1.337},"last":0.72699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":557056,"data_stored":237634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41699999999999998,"15min":0.41799999999999998},"min":{"1min":0.316,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.434},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.443,"15min":0.441},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47599999999999998,"15min":0.47999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60199999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80700000000000005,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.38800000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74199999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":0.76300000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":1.171,"15min":1.171},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.875}]},{"osd":5,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.87,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91700000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.83299999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.88500000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.546,"15min":1.546},"last":0.81999999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81000000000000005,"15min":0.81799999999999995},"min":{"1min":0.59199999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96799999999999997,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":1.155,"15min":1.2290000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.95599999999999996,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.70899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35688,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336152,"statfs":{"total":96636764160,"available":96600219648,"internally_reserved":0,"allocated":557056,"data_stored":237086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.57299999999999995,"5min":0.626,"15min":0.626},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.52600000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.66400000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66800000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.61099999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69299999999999995,"15min":0.69399999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":1.165,"15min":1.165},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.751},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.90500000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.746,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91400000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.90500000000000003,"5min":2.157,"15min":2.157},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.61299999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88900000000000001,"5min":1.966,"15min":1.966},"last":0.81799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":557056,"data_stored":237142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.65700000000000003,"15min":0.69699999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.40699999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60599999999999998,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":1.196,"15min":1.3260000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63300000000000001,"5min":15.911,"15min":15.911},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.266,"15min":1.3959999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63700000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.55600000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":1.2709999999999999,"15min":1.401},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.59699999999999998,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84899999999999998,"5min":1.179,"15min":1.179},"last":0.747},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.97699999999999998,"15min":1.028},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89500000000000002,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.97999999999999998,"15min":1.0269999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92700000000000005,"5min":5.077,"15min":5.077},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.0409999999999999,"15min":1.0940000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89600000000000002,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.52500000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.90200000000000002,"5min":1.143,"15min":1.143},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.995,"15min":1.0449999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96799999999999997,"5min":5.157,"15min":5.157},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.85099999999999998,"15min":0.878},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96399999999999997,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.006,"15min":1.0549999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93000000000000005,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.877}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":573440,"data_stored":238708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.628},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.91500000000000004,"15min":0.95899999999999996},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94399999999999995,"5min":6.258,"15min":6.258},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.93400000000000005,"15min":0.97299999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1419999999999999,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":1.0189999999999999,"15min":1.069},"min":{"1min":0.54800000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0089999999999999,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.073,"5min":1.413,"15min":1.413},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80800000000000005,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.873,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2330000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.028,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.76700000000000002,"15min":0.77700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1159168,"data_stored":828055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.90500000000000003,"15min":0.95499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94199999999999995,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81100000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.628},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95399999999999996,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.83699999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69299999999999995,"15min":0.68500000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.81999999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79600000000000004,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.75900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:47:46.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:47:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.203+0000 7f98b1dff700 1 Processor -- start 2023-07-06T03:47:46.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.203+0000 7f98b1dff700 1 -- start start 2023-07-06T03:47:46.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.204+0000 7f98b1dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98ac0cd390 0x7f98ac160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:46.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.204+0000 7f98b1dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac0cdae0 con 0x7f98ac15c520 2023-07-06T03:47:46.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.204+0000 7f98b1dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac0b6640 con 0x7f98ac160dd0 2023-07-06T03:47:46.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.204+0000 7f98b1dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0b6920 con 0x7f98ac0cd390 2023-07-06T03:47:46.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.205+0000 7f98ab7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f98ac15c520 0x7f98ac160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46596/0 (socket says 172.21.15.97:46596) 2023-07-06T03:47:46.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.205+0000 7f98ab7fe700 1 -- 172.21.15.97:0/1091728313 learned_addr learned my addr 172.21.15.97:0/1091728313 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:46.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.205+0000 7f98aaffd700 1 --2- 172.21.15.97:0/1091728313 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98ac0cd390 0x7f98ac160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.205+0000 7f98aa7fc700 1 -- 172.21.15.97:0/1091728313 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267540336 0 0) 0x7f98ac0cdae0 con 0x7f98ac15c520 2023-07-06T03:47:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.205+0000 7f98aa7fc700 1 -- 172.21.15.97:0/1091728313 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98940035b0 con 0x7f98ac15c520 2023-07-06T03:47:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.205+0000 7f98aa7fc700 1 -- 172.21.15.97:0/1091728313 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145397086 0 0) 0x7f98ac0b6640 con 0x7f98ac160dd0 2023-07-06T03:47:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.205+0000 7f98aa7fc700 1 -- 172.21.15.97:0/1091728313 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98ac0cdae0 con 0x7f98ac160dd0 2023-07-06T03:47:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.205+0000 7f98aaffd700 1 -- 172.21.15.97:0/1091728313 >> v1:172.21.15.146:6789/0 conn(0x7f98ac160dd0 legacy=0x7f98ac0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.205+0000 7f98aaffd700 1 -- 172.21.15.97:0/1091728313 >> v1:172.21.15.97:6789/0 conn(0x7f98ac15c520 legacy=0x7f98ac160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.205+0000 7f98aaffd700 1 -- 172.21.15.97:0/1091728313 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac0b6c00 con 0x7f98ac0cd390 2023-07-06T03:47:46.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.206+0000 7f98aaffd700 1 --2- 172.21.15.97:0/1091728313 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98ac0cd390 0x7f98ac160890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009870 tx=0x7f98a0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=739f20615992ff87 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:46.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.206+0000 7f98aa7fc700 1 -- 172.21.15.97:0/1091728313 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0011830 con 0x7f98ac0cd390 2023-07-06T03:47:46.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.206+0000 7f98aa7fc700 1 -- 172.21.15.97:0/1091728313 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a0011990 con 0x7f98ac0cd390 2023-07-06T03:47:46.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.206+0000 7f98aa7fc700 1 -- 172.21.15.97:0/1091728313 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0011d20 con 0x7f98ac0cd390 2023-07-06T03:47:46.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.206+0000 7f98b1dff700 1 -- 172.21.15.97:0/1091728313 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98ac0cd390 msgr2=0x7f98ac160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.206+0000 7f98b1dff700 1 --2- 172.21.15.97:0/1091728313 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98ac0cd390 0x7f98ac160890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009870 tx=0x7f98a0009b80 comp rx=0 tx=0).stop 2023-07-06T03:47:46.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.207+0000 7f98b1dff700 1 -- 172.21.15.97:0/1091728313 shutdown_connections 2023-07-06T03:47:46.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.207+0000 7f98b1dff700 1 --2- 172.21.15.97:0/1091728313 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98ac0cd390 0x7f98ac160890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.207+0000 7f98b1dff700 1 -- 172.21.15.97:0/1091728313 >> 172.21.15.97:0/1091728313 conn(0x7f98ac071790 msgr2=0x7f98ac071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:46.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.207+0000 7f98b1dff700 1 -- 172.21.15.97:0/1091728313 shutdown_connections 2023-07-06T03:47:46.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.207+0000 7f98b1dff700 1 -- 172.21.15.97:0/1091728313 wait complete. 2023-07-06T03:47:46.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.208+0000 7f98b1dff700 1 Processor -- start 2023-07-06T03:47:46.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.208+0000 7f98b1dff700 1 -- start start 2023-07-06T03:47:46.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.209+0000 7f98b1dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98ac0cd390 0x7f98ac15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:46.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.209+0000 7f98b1dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98ac15c520 0x7f98ac15d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:46.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.209+0000 7f98b1dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98ac160dd0 0x7f98ac159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:46.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.209+0000 7f98b1dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac15ef60 con 0x7f98ac0cd390 2023-07-06T03:47:46.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.209+0000 7f98b1dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac15a6f0 con 0x7f98ac15c520 2023-07-06T03:47:46.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.209+0000 7f98b1dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac15a9d0 con 0x7f98ac160dd0 2023-07-06T03:47:46.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.210+0000 7f98ab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98ac0cd390 0x7f98ac15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:46.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.210+0000 7f98ab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98ac0cd390 0x7f98ac15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33226/0 (socket says 172.21.15.97:33226) 2023-07-06T03:47:46.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.210+0000 7f98ab7fe700 1 -- 172.21.15.97:0/343969221 learned_addr learned my addr 172.21.15.97:0/343969221 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:46.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.210+0000 7f98aaffd700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98ac15c520 0x7f98ac15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:46.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.210+0000 7f98abfff700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98ac160dd0 0x7f98ac159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:46.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.210+0000 7f98ab7fe700 1 -- 172.21.15.97:0/343969221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98ac160dd0 msgr2=0x7f98ac159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.210+0000 7f98ab7fe700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98ac160dd0 0x7f98ac159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.210+0000 7f98ab7fe700 1 -- 172.21.15.97:0/343969221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98ac15c520 msgr2=0x7f98ac15d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.210+0000 7f98ab7fe700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98ac15c520 0x7f98ac15d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.210+0000 7f98ab7fe700 1 -- 172.21.15.97:0/343969221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00094a0 con 0x7f98ac0cd390 2023-07-06T03:47:46.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.210+0000 7f98ab7fe700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98ac0cd390 0x7f98ac15cbd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c700 tx=0x7f989c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:46.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.211+0000 7f9893fff700 1 -- 172.21.15.97:0/343969221 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c005d70 con 0x7f98ac0cd390 2023-07-06T03:47:46.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.211+0000 7f9893fff700 1 -- 172.21.15.97:0/343969221 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c0070d0 con 0x7f98ac0cd390 2023-07-06T03:47:46.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.211+0000 7f9893fff700 1 -- 172.21.15.97:0/343969221 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c007460 con 0x7f98ac0cd390 2023-07-06T03:47:46.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.211+0000 7f98b1dff700 1 -- 172.21.15.97:0/343969221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac15acb0 con 0x7f98ac0cd390 2023-07-06T03:47:46.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.211+0000 7f98b1dff700 1 -- 172.21.15.97:0/343969221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac15b270 con 0x7f98ac0cd390 2023-07-06T03:47:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.212+0000 7f98b1dff700 1 -- 172.21.15.97:0/343969221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac0b4940 con 0x7f98ac0cd390 2023-07-06T03:47:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.213+0000 7f9893fff700 1 -- 172.21.15.97:0/343969221 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f989c0075c0 con 0x7f98ac0cd390 2023-07-06T03:47:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.214+0000 7f9893fff700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f989806ace0 0x7f989806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.214+0000 7f9893fff700 1 -- 172.21.15.97:0/343969221 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f989c088980 con 0x7f98ac0cd390 2023-07-06T03:47:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.214+0000 7f98aaffd700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f989806ace0 0x7f989806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:46.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.214+0000 7f98aaffd700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f989806ace0 0x7f989806d190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f98a00090b0 tx=0x7f98a001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.218+0000 7f9893fff700 1 -- 172.21.15.97:0/343969221 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f989c053990 con 0x7f98ac0cd390 2023-07-06T03:47:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.410+0000 7f98b1dff700 1 -- 172.21.15.97:0/343969221 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98ac0cd800 con 0x7f989806ace0 2023-07-06T03:47:46.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.419+0000 7f9893fff700 1 -- 172.21.15.97:0/343969221 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311104 (secure 0 0 0) 0x7f98a0011b00 con 0x7f989806ace0 2023-07-06T03:47:46.425 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.423+0000 7f98b1dff700 1 -- 172.21.15.97:0/343969221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f989806ace0 msgr2=0x7f989806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.423+0000 7f98b1dff700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f989806ace0 0x7f989806d190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f98a00090b0 tx=0x7f98a001c040 comp rx=0 tx=0).stop 2023-07-06T03:47:46.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.423+0000 7f98b1dff700 1 -- 172.21.15.97:0/343969221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98ac0cd390 msgr2=0x7f98ac15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.423+0000 7f98b1dff700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98ac0cd390 0x7f98ac15cbd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c700 tx=0x7f989c00ca10 comp rx=0 tx=0).stop 2023-07-06T03:47:46.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.423+0000 7f98b1dff700 1 -- 172.21.15.97:0/343969221 shutdown_connections 2023-07-06T03:47:46.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.423+0000 7f98b1dff700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f989806ace0 0x7f989806d190 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.423+0000 7f98b1dff700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f98ac160dd0 0x7f98ac159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.423+0000 7f98b1dff700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98ac0cd390 0x7f98ac15cbd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.423+0000 7f98b1dff700 1 --2- 172.21.15.97:0/343969221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98ac15c520 0x7f98ac15d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.423+0000 7f98b1dff700 1 -- 172.21.15.97:0/343969221 >> 172.21.15.97:0/343969221 conn(0x7f98ac071790 msgr2=0x7f98ac0b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:46.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.424+0000 7f98b1dff700 1 -- 172.21.15.97:0/343969221 shutdown_connections 2023-07-06T03:47:46.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.424+0000 7f98b1dff700 1 -- 172.21.15.97:0/343969221 wait complete. 2023-07-06T03:47:46.432 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:47:46.443 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2023-07-06T03:47:44.912905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249828,"kb_used_data":5540,"kb_used_omap":0,"kb_used_meta":244224,"kb_avail":754724892,"statfs":{"total":773094113280,"available":772838289408,"internally_reserved":0,"allocated":5672960,"data_stored":3080857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":250085376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.252244"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032632+0000","last_change":"2023-07-06T03:47:36.690034+0000","last_active":"2023-07-06T03:47:38.032632+0000","last_peered":"2023-07-06T03:47:38.032632+0000","last_clean":"2023-07-06T03:47:38.032632+0000","last_became_active":"2023-07-06T03:47:36.689699+0000","last_became_peered":"2023-07-06T03:47:36.689699+0000","last_unstale":"2023-07-06T03:47:38.032632+0000","last_undegraded":"2023-07-06T03:47:38.032632+0000","last_fullsized":"2023-07-06T03:47:38.032632+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:08:53.128913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679530+0000","last_change":"2023-07-06T03:47:36.684458+0000","last_active":"2023-07-06T03:47:37.679530+0000","last_peered":"2023-07-06T03:47:37.679530+0000","last_clean":"2023-07-06T03:47:37.679530+0000","last_became_active":"2023-07-06T03:47:36.684178+0000","last_became_peered":"2023-07-06T03:47:36.684178+0000","last_unstale":"2023-07-06T03:47:37.679530+0000","last_undegraded":"2023-07-06T03:47:37.679530+0000","last_fullsized":"2023-07-06T03:47:37.679530+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:54.714268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033517+0000","last_change":"2023-07-06T03:47:36.694324+0000","last_active":"2023-07-06T03:47:38.033517+0000","last_peered":"2023-07-06T03:47:38.033517+0000","last_clean":"2023-07-06T03:47:38.033517+0000","last_became_active":"2023-07-06T03:47:36.694042+0000","last_became_peered":"2023-07-06T03:47:36.694042+0000","last_unstale":"2023-07-06T03:47:38.033517+0000","last_undegraded":"2023-07-06T03:47:38.033517+0000","last_fullsized":"2023-07-06T03:47:38.033517+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:14:47.155587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031051+0000","last_change":"2023-07-06T03:47:36.679549+0000","last_active":"2023-07-06T03:47:38.031051+0000","last_peered":"2023-07-06T03:47:38.031051+0000","last_clean":"2023-07-06T03:47:38.031051+0000","last_became_active":"2023-07-06T03:47:36.679270+0000","last_became_peered":"2023-07-06T03:47:36.679270+0000","last_unstale":"2023-07-06T03:47:38.031051+0000","last_undegraded":"2023-07-06T03:47:38.031051+0000","last_fullsized":"2023-07-06T03:47:38.031051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:09.615630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679347+0000","last_change":"2023-07-06T03:47:36.684433+0000","last_active":"2023-07-06T03:47:37.679347+0000","last_peered":"2023-07-06T03:47:37.679347+0000","last_clean":"2023-07-06T03:47:37.679347+0000","last_became_active":"2023-07-06T03:47:36.684147+0000","last_became_peered":"2023-07-06T03:47:36.684147+0000","last_unstale":"2023-07-06T03:47:37.679347+0000","last_undegraded":"2023-07-06T03:47:37.679347+0000","last_fullsized":"2023-07-06T03:47:37.679347+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:31:19.105732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678751+0000","last_change":"2023-07-06T03:47:36.685535+0000","last_active":"2023-07-06T03:47:37.678751+0000","last_peered":"2023-07-06T03:47:37.678751+0000","last_clean":"2023-07-06T03:47:37.678751+0000","last_became_active":"2023-07-06T03:47:36.684607+0000","last_became_peered":"2023-07-06T03:47:36.684607+0000","last_unstale":"2023-07-06T03:47:37.678751+0000","last_undegraded":"2023-07-06T03:47:37.678751+0000","last_fullsized":"2023-07-06T03:47:37.678751+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:37:59.925688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032468+0000","last_change":"2023-07-06T03:47:36.692588+0000","last_active":"2023-07-06T03:47:38.032468+0000","last_peered":"2023-07-06T03:47:38.032468+0000","last_clean":"2023-07-06T03:47:38.032468+0000","last_became_active":"2023-07-06T03:47:36.692264+0000","last_became_peered":"2023-07-06T03:47:36.692264+0000","last_unstale":"2023-07-06T03:47:38.032468+0000","last_undegraded":"2023-07-06T03:47:38.032468+0000","last_fullsized":"2023-07-06T03:47:38.032468+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:44:23.679226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678405+0000","last_change":"2023-07-06T03:47:36.690548+0000","last_active":"2023-07-06T03:47:37.678405+0000","last_peered":"2023-07-06T03:47:37.678405+0000","last_clean":"2023-07-06T03:47:37.678405+0000","last_became_active":"2023-07-06T03:47:36.690269+0000","last_became_peered":"2023-07-06T03:47:36.690269+0000","last_unstale":"2023-07-06T03:47:37.678405+0000","last_undegraded":"2023-07-06T03:47:37.678405+0000","last_fullsized":"2023-07-06T03:47:37.678405+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:43.307999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032956+0000","last_change":"2023-07-06T03:47:36.696433+0000","last_active":"2023-07-06T03:47:38.032956+0000","last_peered":"2023-07-06T03:47:38.032956+0000","last_clean":"2023-07-06T03:47:38.032956+0000","last_became_active":"2023-07-06T03:47:36.696122+0000","last_became_peered":"2023-07-06T03:47:36.696122+0000","last_unstale":"2023-07-06T03:47:38.032956+0000","last_undegraded":"2023-07-06T03:47:38.032956+0000","last_fullsized":"2023-07-06T03:47:38.032956+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:07:25.581724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033354+0000","last_change":"2023-07-06T03:47:36.696522+0000","last_active":"2023-07-06T03:47:38.033354+0000","last_peered":"2023-07-06T03:47:38.033354+0000","last_clean":"2023-07-06T03:47:38.033354+0000","last_became_active":"2023-07-06T03:47:36.696078+0000","last_became_peered":"2023-07-06T03:47:36.696078+0000","last_unstale":"2023-07-06T03:47:38.033354+0000","last_undegraded":"2023-07-06T03:47:38.033354+0000","last_fullsized":"2023-07-06T03:47:38.033354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:44:02.670101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032125+0000","last_change":"2023-07-06T03:47:36.686755+0000","last_active":"2023-07-06T03:47:38.032125+0000","last_peered":"2023-07-06T03:47:38.032125+0000","last_clean":"2023-07-06T03:47:38.032125+0000","last_became_active":"2023-07-06T03:47:36.686499+0000","last_became_peered":"2023-07-06T03:47:36.686499+0000","last_unstale":"2023-07-06T03:47:38.032125+0000","last_undegraded":"2023-07-06T03:47:38.032125+0000","last_fullsized":"2023-07-06T03:47:38.032125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:40:35.757147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678770+0000","last_change":"2023-07-06T03:47:36.671270+0000","last_active":"2023-07-06T03:47:37.678770+0000","last_peered":"2023-07-06T03:47:37.678770+0000","last_clean":"2023-07-06T03:47:37.678770+0000","last_became_active":"2023-07-06T03:47:36.671009+0000","last_became_peered":"2023-07-06T03:47:36.671009+0000","last_unstale":"2023-07-06T03:47:37.678770+0000","last_undegraded":"2023-07-06T03:47:37.678770+0000","last_fullsized":"2023-07-06T03:47:37.678770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:20:42.997717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678752+0000","last_change":"2023-07-06T03:47:36.673999+0000","last_active":"2023-07-06T03:47:37.678752+0000","last_peered":"2023-07-06T03:47:37.678752+0000","last_clean":"2023-07-06T03:47:37.678752+0000","last_became_active":"2023-07-06T03:47:36.673684+0000","last_became_peered":"2023-07-06T03:47:36.673684+0000","last_unstale":"2023-07-06T03:47:37.678752+0000","last_undegraded":"2023-07-06T03:47:37.678752+0000","last_fullsized":"2023-07-06T03:47:37.678752+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:44:02.918959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031879+0000","last_change":"2023-07-06T03:47:36.691230+0000","last_active":"2023-07-06T03:47:38.031879+0000","last_peered":"2023-07-06T03:47:38.031879+0000","last_clean":"2023-07-06T03:47:38.031879+0000","last_became_active":"2023-07-06T03:47:36.690949+0000","last_became_peered":"2023-07-06T03:47:36.690949+0000","last_unstale":"2023-07-06T03:47:38.031879+0000","last_undegraded":"2023-07-06T03:47:38.031879+0000","last_fullsized":"2023-07-06T03:47:38.031879+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:44:02.670101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031076+0000","last_change":"2023-07-06T03:47:36.680280+0000","last_active":"2023-07-06T03:47:38.031076+0000","last_peered":"2023-07-06T03:47:38.031076+0000","last_clean":"2023-07-06T03:47:38.031076+0000","last_became_active":"2023-07-06T03:47:36.680012+0000","last_became_peered":"2023-07-06T03:47:36.680012+0000","last_unstale":"2023-07-06T03:47:38.031076+0000","last_undegraded":"2023-07-06T03:47:38.031076+0000","last_fullsized":"2023-07-06T03:47:38.031076+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:46:07.037740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678390+0000","last_change":"2023-07-06T03:47:36.673201+0000","last_active":"2023-07-06T03:47:37.678390+0000","last_peered":"2023-07-06T03:47:37.678390+0000","last_clean":"2023-07-06T03:47:37.678390+0000","last_became_active":"2023-07-06T03:47:36.672913+0000","last_became_peered":"2023-07-06T03:47:36.672913+0000","last_unstale":"2023-07-06T03:47:37.678390+0000","last_undegraded":"2023-07-06T03:47:37.678390+0000","last_fullsized":"2023-07-06T03:47:37.678390+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:45.025174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032476+0000","last_change":"2023-07-06T03:47:36.690227+0000","last_active":"2023-07-06T03:47:38.032476+0000","last_peered":"2023-07-06T03:47:38.032476+0000","last_clean":"2023-07-06T03:47:38.032476+0000","last_became_active":"2023-07-06T03:47:36.689751+0000","last_became_peered":"2023-07-06T03:47:36.689751+0000","last_unstale":"2023-07-06T03:47:38.032476+0000","last_undegraded":"2023-07-06T03:47:38.032476+0000","last_fullsized":"2023-07-06T03:47:38.032476+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:24:19.212787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031275+0000","last_change":"2023-07-06T03:47:36.679476+0000","last_active":"2023-07-06T03:47:38.031275+0000","last_peered":"2023-07-06T03:47:38.031275+0000","last_clean":"2023-07-06T03:47:38.031275+0000","last_became_active":"2023-07-06T03:47:36.679195+0000","last_became_peered":"2023-07-06T03:47:36.679195+0000","last_unstale":"2023-07-06T03:47:38.031275+0000","last_undegraded":"2023-07-06T03:47:38.031275+0000","last_fullsized":"2023-07-06T03:47:38.031275+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:52:04.996142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033580+0000","last_change":"2023-07-06T03:47:36.683417+0000","last_active":"2023-07-06T03:47:38.033580+0000","last_peered":"2023-07-06T03:47:38.033580+0000","last_clean":"2023-07-06T03:47:38.033580+0000","last_became_active":"2023-07-06T03:47:36.683107+0000","last_became_peered":"2023-07-06T03:47:36.683107+0000","last_unstale":"2023-07-06T03:47:38.033580+0000","last_undegraded":"2023-07-06T03:47:38.033580+0000","last_fullsized":"2023-07-06T03:47:38.033580+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:01:57.215705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679945+0000","last_change":"2023-07-06T03:47:36.685004+0000","last_active":"2023-07-06T03:47:37.679945+0000","last_peered":"2023-07-06T03:47:37.679945+0000","last_clean":"2023-07-06T03:47:37.679945+0000","last_became_active":"2023-07-06T03:47:36.684742+0000","last_became_peered":"2023-07-06T03:47:36.684742+0000","last_unstale":"2023-07-06T03:47:37.679945+0000","last_undegraded":"2023-07-06T03:47:37.679945+0000","last_fullsized":"2023-07-06T03:47:37.679945+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:12.068599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.030900+0000","last_change":"2023-07-06T03:47:36.680320+0000","last_active":"2023-07-06T03:47:38.030900+0000","last_peered":"2023-07-06T03:47:38.030900+0000","last_clean":"2023-07-06T03:47:38.030900+0000","last_became_active":"2023-07-06T03:47:36.680025+0000","last_became_peered":"2023-07-06T03:47:36.680025+0000","last_unstale":"2023-07-06T03:47:38.030900+0000","last_undegraded":"2023-07-06T03:47:38.030900+0000","last_fullsized":"2023-07-06T03:47:38.030900+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:07:55.161397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032634+0000","last_change":"2023-07-06T03:47:36.698971+0000","last_active":"2023-07-06T03:47:38.032634+0000","last_peered":"2023-07-06T03:47:38.032634+0000","last_clean":"2023-07-06T03:47:38.032634+0000","last_became_active":"2023-07-06T03:47:36.698714+0000","last_became_peered":"2023-07-06T03:47:36.698714+0000","last_unstale":"2023-07-06T03:47:38.032634+0000","last_undegraded":"2023-07-06T03:47:38.032634+0000","last_fullsized":"2023-07-06T03:47:38.032634+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:30:28.841133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031954+0000","last_change":"2023-07-06T03:47:36.679892+0000","last_active":"2023-07-06T03:47:38.031954+0000","last_peered":"2023-07-06T03:47:38.031954+0000","last_clean":"2023-07-06T03:47:38.031954+0000","last_became_active":"2023-07-06T03:47:36.679610+0000","last_became_peered":"2023-07-06T03:47:36.679610+0000","last_unstale":"2023-07-06T03:47:38.031954+0000","last_undegraded":"2023-07-06T03:47:38.031954+0000","last_fullsized":"2023-07-06T03:47:38.031954+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:33:20.150333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032597+0000","last_change":"2023-07-06T03:47:36.692786+0000","last_active":"2023-07-06T03:47:38.032597+0000","last_peered":"2023-07-06T03:47:38.032597+0000","last_clean":"2023-07-06T03:47:38.032597+0000","last_became_active":"2023-07-06T03:47:36.692480+0000","last_became_peered":"2023-07-06T03:47:36.692480+0000","last_unstale":"2023-07-06T03:47:38.032597+0000","last_undegraded":"2023-07-06T03:47:38.032597+0000","last_fullsized":"2023-07-06T03:47:38.032597+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:55:08.239557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032292+0000","last_change":"2023-07-06T03:47:36.697045+0000","last_active":"2023-07-06T03:47:38.032292+0000","last_peered":"2023-07-06T03:47:38.032292+0000","last_clean":"2023-07-06T03:47:38.032292+0000","last_became_active":"2023-07-06T03:47:36.696756+0000","last_became_peered":"2023-07-06T03:47:36.696756+0000","last_unstale":"2023-07-06T03:47:38.032292+0000","last_undegraded":"2023-07-06T03:47:38.032292+0000","last_fullsized":"2023-07-06T03:47:38.032292+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:56:37.217803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032542+0000","last_change":"2023-07-06T03:47:36.674863+0000","last_active":"2023-07-06T03:47:38.032542+0000","last_peered":"2023-07-06T03:47:38.032542+0000","last_clean":"2023-07-06T03:47:38.032542+0000","last_became_active":"2023-07-06T03:47:36.674576+0000","last_became_peered":"2023-07-06T03:47:36.674576+0000","last_unstale":"2023-07-06T03:47:38.032542+0000","last_undegraded":"2023-07-06T03:47:38.032542+0000","last_fullsized":"2023-07-06T03:47:38.032542+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:37:09.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679100+0000","last_change":"2023-07-06T03:47:36.691182+0000","last_active":"2023-07-06T03:47:37.679100+0000","last_peered":"2023-07-06T03:47:37.679100+0000","last_clean":"2023-07-06T03:47:37.679100+0000","last_became_active":"2023-07-06T03:47:36.690472+0000","last_became_peered":"2023-07-06T03:47:36.690472+0000","last_unstale":"2023-07-06T03:47:37.679100+0000","last_undegraded":"2023-07-06T03:47:37.679100+0000","last_fullsized":"2023-07-06T03:47:37.679100+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:00:22.356225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679806+0000","last_change":"2023-07-06T03:47:36.694072+0000","last_active":"2023-07-06T03:47:37.679806+0000","last_peered":"2023-07-06T03:47:37.679806+0000","last_clean":"2023-07-06T03:47:37.679806+0000","last_became_active":"2023-07-06T03:47:36.693790+0000","last_became_peered":"2023-07-06T03:47:36.693790+0000","last_unstale":"2023-07-06T03:47:37.679806+0000","last_undegraded":"2023-07-06T03:47:37.679806+0000","last_fullsized":"2023-07-06T03:47:37.679806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:03:49.636022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032568+0000","last_change":"2023-07-06T03:47:36.690908+0000","last_active":"2023-07-06T03:47:38.032568+0000","last_peered":"2023-07-06T03:47:38.032568+0000","last_clean":"2023-07-06T03:47:38.032568+0000","last_became_active":"2023-07-06T03:47:36.690513+0000","last_became_peered":"2023-07-06T03:47:36.690513+0000","last_unstale":"2023-07-06T03:47:38.032568+0000","last_undegraded":"2023-07-06T03:47:38.032568+0000","last_fullsized":"2023-07-06T03:47:38.032568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:56:56.281979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679889+0000","last_change":"2023-07-06T03:47:36.678674+0000","last_active":"2023-07-06T03:47:37.679889+0000","last_peered":"2023-07-06T03:47:37.679889+0000","last_clean":"2023-07-06T03:47:37.679889+0000","last_became_active":"2023-07-06T03:47:36.677990+0000","last_became_peered":"2023-07-06T03:47:36.677990+0000","last_unstale":"2023-07-06T03:47:37.679889+0000","last_undegraded":"2023-07-06T03:47:37.679889+0000","last_fullsized":"2023-07-06T03:47:37.679889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:49:42.465400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032108+0000","last_change":"2023-07-06T03:47:36.693968+0000","last_active":"2023-07-06T03:47:38.032108+0000","last_peered":"2023-07-06T03:47:38.032108+0000","last_clean":"2023-07-06T03:47:38.032108+0000","last_became_active":"2023-07-06T03:47:36.693687+0000","last_became_peered":"2023-07-06T03:47:36.693687+0000","last_unstale":"2023-07-06T03:47:38.032108+0000","last_undegraded":"2023-07-06T03:47:38.032108+0000","last_fullsized":"2023-07-06T03:47:38.032108+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:24:46.598193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678951+0000","last_change":"2023-07-06T03:47:36.690467+0000","last_active":"2023-07-06T03:47:37.678951+0000","last_peered":"2023-07-06T03:47:37.678951+0000","last_clean":"2023-07-06T03:47:37.678951+0000","last_became_active":"2023-07-06T03:47:36.689525+0000","last_became_peered":"2023-07-06T03:47:36.689525+0000","last_unstale":"2023-07-06T03:47:37.678951+0000","last_undegraded":"2023-07-06T03:47:37.678951+0000","last_fullsized":"2023-07-06T03:47:37.678951+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:13:56.051926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679425+0000","last_change":"2023-07-06T03:47:36.679202+0000","last_active":"2023-07-06T03:47:37.679425+0000","last_peered":"2023-07-06T03:47:37.679425+0000","last_clean":"2023-07-06T03:47:37.679425+0000","last_became_active":"2023-07-06T03:47:36.678019+0000","last_became_peered":"2023-07-06T03:47:36.678019+0000","last_unstale":"2023-07-06T03:47:37.679425+0000","last_undegraded":"2023-07-06T03:47:37.679425+0000","last_fullsized":"2023-07-06T03:47:37.679425+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:27:41.668800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031749+0000","last_change":"2023-07-06T03:47:35.684675+0000","last_active":"2023-07-06T03:47:38.031749+0000","last_peered":"2023-07-06T03:47:38.031749+0000","last_clean":"2023-07-06T03:47:38.031749+0000","last_became_active":"2023-07-06T03:47:35.684398+0000","last_became_peered":"2023-07-06T03:47:35.684398+0000","last_unstale":"2023-07-06T03:47:38.031749+0000","last_undegraded":"2023-07-06T03:47:38.031749+0000","last_fullsized":"2023-07-06T03:47:38.031749+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:03:08.897884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679576+0000","last_change":"2023-07-06T03:47:36.674737+0000","last_active":"2023-07-06T03:47:37.679576+0000","last_peered":"2023-07-06T03:47:37.679576+0000","last_clean":"2023-07-06T03:47:37.679576+0000","last_became_active":"2023-07-06T03:47:36.674426+0000","last_became_peered":"2023-07-06T03:47:36.674426+0000","last_unstale":"2023-07-06T03:47:37.679576+0000","last_undegraded":"2023-07-06T03:47:37.679576+0000","last_fullsized":"2023-07-06T03:47:37.679576+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:31.047623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"46'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679157+0000","last_change":"2023-07-06T03:47:35.680067+0000","last_active":"2023-07-06T03:47:37.679157+0000","last_peered":"2023-07-06T03:47:37.679157+0000","last_clean":"2023-07-06T03:47:37.679157+0000","last_became_active":"2023-07-06T03:47:35.679691+0000","last_became_peered":"2023-07-06T03:47:35.679691+0000","last_unstale":"2023-07-06T03:47:37.679157+0000","last_undegraded":"2023-07-06T03:47:37.679157+0000","last_fullsized":"2023-07-06T03:47:37.679157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:15:55.604018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680012+0000","last_change":"2023-07-06T03:47:36.678853+0000","last_active":"2023-07-06T03:47:37.680012+0000","last_peered":"2023-07-06T03:47:37.680012+0000","last_clean":"2023-07-06T03:47:37.680012+0000","last_became_active":"2023-07-06T03:47:36.678366+0000","last_became_peered":"2023-07-06T03:47:36.678366+0000","last_unstale":"2023-07-06T03:47:37.680012+0000","last_undegraded":"2023-07-06T03:47:37.680012+0000","last_fullsized":"2023-07-06T03:47:37.680012+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:50:56.264915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032772+0000","last_change":"2023-07-06T03:47:35.676405+0000","last_active":"2023-07-06T03:47:38.032772+0000","last_peered":"2023-07-06T03:47:38.032772+0000","last_clean":"2023-07-06T03:47:38.032772+0000","last_became_active":"2023-07-06T03:47:35.676089+0000","last_became_peered":"2023-07-06T03:47:35.676089+0000","last_unstale":"2023-07-06T03:47:38.032772+0000","last_undegraded":"2023-07-06T03:47:38.032772+0000","last_fullsized":"2023-07-06T03:47:38.032772+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:01:39.848129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033032+0000","last_change":"2023-07-06T03:47:36.688311+0000","last_active":"2023-07-06T03:47:38.033032+0000","last_peered":"2023-07-06T03:47:38.033032+0000","last_clean":"2023-07-06T03:47:38.033032+0000","last_became_active":"2023-07-06T03:47:36.687991+0000","last_became_peered":"2023-07-06T03:47:36.687991+0000","last_unstale":"2023-07-06T03:47:38.033032+0000","last_undegraded":"2023-07-06T03:47:38.033032+0000","last_fullsized":"2023-07-06T03:47:38.033032+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:03:09.900802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679842+0000","last_change":"2023-07-06T03:47:35.678850+0000","last_active":"2023-07-06T03:47:37.679842+0000","last_peered":"2023-07-06T03:47:37.679842+0000","last_clean":"2023-07-06T03:47:37.679842+0000","last_became_active":"2023-07-06T03:47:35.678520+0000","last_became_peered":"2023-07-06T03:47:35.678520+0000","last_unstale":"2023-07-06T03:47:37.679842+0000","last_undegraded":"2023-07-06T03:47:37.679842+0000","last_fullsized":"2023-07-06T03:47:37.679842+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:25:01.977539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031507+0000","last_change":"2023-07-06T03:47:36.684332+0000","last_active":"2023-07-06T03:47:38.031507+0000","last_peered":"2023-07-06T03:47:38.031507+0000","last_clean":"2023-07-06T03:47:38.031507+0000","last_became_active":"2023-07-06T03:47:36.684079+0000","last_became_peered":"2023-07-06T03:47:36.684079+0000","last_unstale":"2023-07-06T03:47:38.031507+0000","last_undegraded":"2023-07-06T03:47:38.031507+0000","last_fullsized":"2023-07-06T03:47:38.031507+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:31:04.312174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"46'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033224+0000","last_change":"2023-07-06T03:47:35.679264+0000","last_active":"2023-07-06T03:47:38.033224+0000","last_peered":"2023-07-06T03:47:38.033224+0000","last_clean":"2023-07-06T03:47:38.033224+0000","last_became_active":"2023-07-06T03:47:35.678950+0000","last_became_peered":"2023-07-06T03:47:35.678950+0000","last_unstale":"2023-07-06T03:47:38.033224+0000","last_undegraded":"2023-07-06T03:47:38.033224+0000","last_fullsized":"2023-07-06T03:47:38.033224+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:48:24.153481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679950+0000","last_change":"2023-07-06T03:47:36.682503+0000","last_active":"2023-07-06T03:47:37.679950+0000","last_peered":"2023-07-06T03:47:37.679950+0000","last_clean":"2023-07-06T03:47:37.679950+0000","last_became_active":"2023-07-06T03:47:36.682196+0000","last_became_peered":"2023-07-06T03:47:36.682196+0000","last_unstale":"2023-07-06T03:47:37.679950+0000","last_undegraded":"2023-07-06T03:47:37.679950+0000","last_fullsized":"2023-07-06T03:47:37.679950+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:01:29.045248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"46'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680034+0000","last_change":"2023-07-06T03:47:35.679463+0000","last_active":"2023-07-06T03:47:37.680034+0000","last_peered":"2023-07-06T03:47:37.680034+0000","last_clean":"2023-07-06T03:47:37.680034+0000","last_became_active":"2023-07-06T03:47:35.679249+0000","last_became_peered":"2023-07-06T03:47:35.679249+0000","last_unstale":"2023-07-06T03:47:37.680034+0000","last_undegraded":"2023-07-06T03:47:37.680034+0000","last_fullsized":"2023-07-06T03:47:37.680034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:45:31.979648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033187+0000","last_change":"2023-07-06T03:47:36.696351+0000","last_active":"2023-07-06T03:47:38.033187+0000","last_peered":"2023-07-06T03:47:38.033187+0000","last_clean":"2023-07-06T03:47:38.033187+0000","last_became_active":"2023-07-06T03:47:36.695783+0000","last_became_peered":"2023-07-06T03:47:36.695783+0000","last_unstale":"2023-07-06T03:47:38.033187+0000","last_undegraded":"2023-07-06T03:47:38.033187+0000","last_fullsized":"2023-07-06T03:47:38.033187+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:44:10.832855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679697+0000","last_change":"2023-07-06T03:47:35.684332+0000","last_active":"2023-07-06T03:47:37.679697+0000","last_peered":"2023-07-06T03:47:37.679697+0000","last_clean":"2023-07-06T03:47:37.679697+0000","last_became_active":"2023-07-06T03:47:35.684064+0000","last_became_peered":"2023-07-06T03:47:35.684064+0000","last_unstale":"2023-07-06T03:47:37.679697+0000","last_undegraded":"2023-07-06T03:47:37.679697+0000","last_fullsized":"2023-07-06T03:47:37.679697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:34:48.350570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031290+0000","last_change":"2023-07-06T03:47:36.674252+0000","last_active":"2023-07-06T03:47:38.031290+0000","last_peered":"2023-07-06T03:47:38.031290+0000","last_clean":"2023-07-06T03:47:38.031290+0000","last_became_active":"2023-07-06T03:47:36.673742+0000","last_became_peered":"2023-07-06T03:47:36.673742+0000","last_unstale":"2023-07-06T03:47:38.031290+0000","last_undegraded":"2023-07-06T03:47:38.031290+0000","last_fullsized":"2023-07-06T03:47:38.031290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:04:29.237378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031634+0000","last_change":"2023-07-06T03:47:35.679127+0000","last_active":"2023-07-06T03:47:38.031634+0000","last_peered":"2023-07-06T03:47:38.031634+0000","last_clean":"2023-07-06T03:47:38.031634+0000","last_became_active":"2023-07-06T03:47:35.678630+0000","last_became_peered":"2023-07-06T03:47:35.678630+0000","last_unstale":"2023-07-06T03:47:38.031634+0000","last_undegraded":"2023-07-06T03:47:38.031634+0000","last_fullsized":"2023-07-06T03:47:38.031634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:38:00.464893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.030782+0000","last_change":"2023-07-06T03:47:36.694845+0000","last_active":"2023-07-06T03:47:38.030782+0000","last_peered":"2023-07-06T03:47:38.030782+0000","last_clean":"2023-07-06T03:47:38.030782+0000","last_became_active":"2023-07-06T03:47:36.694409+0000","last_became_peered":"2023-07-06T03:47:36.694409+0000","last_unstale":"2023-07-06T03:47:38.030782+0000","last_undegraded":"2023-07-06T03:47:38.030782+0000","last_fullsized":"2023-07-06T03:47:38.030782+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:15.008580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031416+0000","last_change":"2023-07-06T03:47:35.678328+0000","last_active":"2023-07-06T03:47:38.031416+0000","last_peered":"2023-07-06T03:47:38.031416+0000","last_clean":"2023-07-06T03:47:38.031416+0000","last_became_active":"2023-07-06T03:47:35.678081+0000","last_became_peered":"2023-07-06T03:47:35.678081+0000","last_unstale":"2023-07-06T03:47:38.031416+0000","last_undegraded":"2023-07-06T03:47:38.031416+0000","last_fullsized":"2023-07-06T03:47:38.031416+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:06:42.075787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031093+0000","last_change":"2023-07-06T03:47:36.681930+0000","last_active":"2023-07-06T03:47:38.031093+0000","last_peered":"2023-07-06T03:47:38.031093+0000","last_clean":"2023-07-06T03:47:38.031093+0000","last_became_active":"2023-07-06T03:47:36.681625+0000","last_became_peered":"2023-07-06T03:47:36.681625+0000","last_unstale":"2023-07-06T03:47:38.031093+0000","last_undegraded":"2023-07-06T03:47:38.031093+0000","last_fullsized":"2023-07-06T03:47:38.031093+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:15:21.433427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031261+0000","last_change":"2023-07-06T03:47:35.673408+0000","last_active":"2023-07-06T03:47:38.031261+0000","last_peered":"2023-07-06T03:47:38.031261+0000","last_clean":"2023-07-06T03:47:38.031261+0000","last_became_active":"2023-07-06T03:47:35.672993+0000","last_became_peered":"2023-07-06T03:47:35.672993+0000","last_unstale":"2023-07-06T03:47:38.031261+0000","last_undegraded":"2023-07-06T03:47:38.031261+0000","last_fullsized":"2023-07-06T03:47:38.031261+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:24.596966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032055+0000","last_change":"2023-07-06T03:47:36.687284+0000","last_active":"2023-07-06T03:47:38.032055+0000","last_peered":"2023-07-06T03:47:38.032055+0000","last_clean":"2023-07-06T03:47:38.032055+0000","last_became_active":"2023-07-06T03:47:36.687017+0000","last_became_peered":"2023-07-06T03:47:36.687017+0000","last_unstale":"2023-07-06T03:47:38.032055+0000","last_undegraded":"2023-07-06T03:47:38.032055+0000","last_fullsized":"2023-07-06T03:47:38.032055+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:25:47.526435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680080+0000","last_change":"2023-07-06T03:47:35.679946+0000","last_active":"2023-07-06T03:47:37.680080+0000","last_peered":"2023-07-06T03:47:37.680080+0000","last_clean":"2023-07-06T03:47:37.680080+0000","last_became_active":"2023-07-06T03:47:35.679487+0000","last_became_peered":"2023-07-06T03:47:35.679487+0000","last_unstale":"2023-07-06T03:47:37.680080+0000","last_undegraded":"2023-07-06T03:47:37.680080+0000","last_fullsized":"2023-07-06T03:47:37.680080+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:20:35.484864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679291+0000","last_change":"2023-07-06T03:47:36.694018+0000","last_active":"2023-07-06T03:47:37.679291+0000","last_peered":"2023-07-06T03:47:37.679291+0000","last_clean":"2023-07-06T03:47:37.679291+0000","last_became_active":"2023-07-06T03:47:36.693840+0000","last_became_peered":"2023-07-06T03:47:36.693840+0000","last_unstale":"2023-07-06T03:47:37.679291+0000","last_undegraded":"2023-07-06T03:47:37.679291+0000","last_fullsized":"2023-07-06T03:47:37.679291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:15:40.828483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"46'6","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033239+0000","last_change":"2023-07-06T03:47:35.679499+0000","last_active":"2023-07-06T03:47:38.033239+0000","last_peered":"2023-07-06T03:47:38.033239+0000","last_clean":"2023-07-06T03:47:38.033239+0000","last_became_active":"2023-07-06T03:47:35.679153+0000","last_became_peered":"2023-07-06T03:47:35.679153+0000","last_unstale":"2023-07-06T03:47:38.033239+0000","last_undegraded":"2023-07-06T03:47:38.033239+0000","last_fullsized":"2023-07-06T03:47:38.033239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:52:44.393837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679691+0000","last_change":"2023-07-06T03:47:36.683371+0000","last_active":"2023-07-06T03:47:37.679691+0000","last_peered":"2023-07-06T03:47:37.679691+0000","last_clean":"2023-07-06T03:47:37.679691+0000","last_became_active":"2023-07-06T03:47:36.683062+0000","last_became_peered":"2023-07-06T03:47:36.683062+0000","last_unstale":"2023-07-06T03:47:37.679691+0000","last_undegraded":"2023-07-06T03:47:37.679691+0000","last_fullsized":"2023-07-06T03:47:37.679691+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:50:27.671381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679071+0000","last_change":"2023-07-06T03:47:35.675074+0000","last_active":"2023-07-06T03:47:37.679071+0000","last_peered":"2023-07-06T03:47:37.679071+0000","last_clean":"2023-07-06T03:47:37.679071+0000","last_became_active":"2023-07-06T03:47:35.674815+0000","last_became_peered":"2023-07-06T03:47:35.674815+0000","last_unstale":"2023-07-06T03:47:37.679071+0000","last_undegraded":"2023-07-06T03:47:37.679071+0000","last_fullsized":"2023-07-06T03:47:37.679071+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:24:18.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031965+0000","last_change":"2023-07-06T03:47:36.690771+0000","last_active":"2023-07-06T03:47:38.031965+0000","last_peered":"2023-07-06T03:47:38.031965+0000","last_clean":"2023-07-06T03:47:38.031965+0000","last_became_active":"2023-07-06T03:47:36.690400+0000","last_became_peered":"2023-07-06T03:47:36.690400+0000","last_unstale":"2023-07-06T03:47:38.031965+0000","last_undegraded":"2023-07-06T03:47:38.031965+0000","last_fullsized":"2023-07-06T03:47:38.031965+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:13.208271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031094+0000","last_change":"2023-07-06T03:47:35.680141+0000","last_active":"2023-07-06T03:47:38.031094+0000","last_peered":"2023-07-06T03:47:38.031094+0000","last_clean":"2023-07-06T03:47:38.031094+0000","last_became_active":"2023-07-06T03:47:35.679775+0000","last_became_peered":"2023-07-06T03:47:35.679775+0000","last_unstale":"2023-07-06T03:47:38.031094+0000","last_undegraded":"2023-07-06T03:47:38.031094+0000","last_fullsized":"2023-07-06T03:47:38.031094+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:55:07.602533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679006+0000","last_change":"2023-07-06T03:47:36.685315+0000","last_active":"2023-07-06T03:47:37.679006+0000","last_peered":"2023-07-06T03:47:37.679006+0000","last_clean":"2023-07-06T03:47:37.679006+0000","last_became_active":"2023-07-06T03:47:36.684988+0000","last_became_peered":"2023-07-06T03:47:36.684988+0000","last_unstale":"2023-07-06T03:47:37.679006+0000","last_undegraded":"2023-07-06T03:47:37.679006+0000","last_fullsized":"2023-07-06T03:47:37.679006+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:48:06.816404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679046+0000","last_change":"2023-07-06T03:47:35.676310+0000","last_active":"2023-07-06T03:47:37.679046+0000","last_peered":"2023-07-06T03:47:37.679046+0000","last_clean":"2023-07-06T03:47:37.679046+0000","last_became_active":"2023-07-06T03:47:35.676038+0000","last_became_peered":"2023-07-06T03:47:35.676038+0000","last_unstale":"2023-07-06T03:47:37.679046+0000","last_undegraded":"2023-07-06T03:47:37.679046+0000","last_fullsized":"2023-07-06T03:47:37.679046+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:25:18.433007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033115+0000","last_change":"2023-07-06T03:47:36.672749+0000","last_active":"2023-07-06T03:47:38.033115+0000","last_peered":"2023-07-06T03:47:38.033115+0000","last_clean":"2023-07-06T03:47:38.033115+0000","last_became_active":"2023-07-06T03:47:36.672454+0000","last_became_peered":"2023-07-06T03:47:36.672454+0000","last_unstale":"2023-07-06T03:47:38.033115+0000","last_undegraded":"2023-07-06T03:47:38.033115+0000","last_fullsized":"2023-07-06T03:47:38.033115+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:53:31.746241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"46'8","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678585+0000","last_change":"2023-07-06T03:47:35.683270+0000","last_active":"2023-07-06T03:47:37.678585+0000","last_peered":"2023-07-06T03:47:37.678585+0000","last_clean":"2023-07-06T03:47:37.678585+0000","last_became_active":"2023-07-06T03:47:35.682973+0000","last_became_peered":"2023-07-06T03:47:35.682973+0000","last_unstale":"2023-07-06T03:47:37.678585+0000","last_undegraded":"2023-07-06T03:47:37.678585+0000","last_fullsized":"2023-07-06T03:47:37.678585+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:49:53.695009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031953+0000","last_change":"2023-07-06T03:47:36.690955+0000","last_active":"2023-07-06T03:47:38.031953+0000","last_peered":"2023-07-06T03:47:38.031953+0000","last_clean":"2023-07-06T03:47:38.031953+0000","last_became_active":"2023-07-06T03:47:36.690689+0000","last_became_peered":"2023-07-06T03:47:36.690689+0000","last_unstale":"2023-07-06T03:47:38.031953+0000","last_undegraded":"2023-07-06T03:47:38.031953+0000","last_fullsized":"2023-07-06T03:47:38.031953+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:18:23.949883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"46'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679233+0000","last_change":"2023-07-06T03:47:35.680067+0000","last_active":"2023-07-06T03:47:37.679233+0000","last_peered":"2023-07-06T03:47:37.679233+0000","last_clean":"2023-07-06T03:47:37.679233+0000","last_became_active":"2023-07-06T03:47:35.679704+0000","last_became_peered":"2023-07-06T03:47:35.679704+0000","last_unstale":"2023-07-06T03:47:37.679233+0000","last_undegraded":"2023-07-06T03:47:37.679233+0000","last_fullsized":"2023-07-06T03:47:37.679233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:56:02.683970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032048+0000","last_change":"2023-07-06T03:47:36.696608+0000","last_active":"2023-07-06T03:47:38.032048+0000","last_peered":"2023-07-06T03:47:38.032048+0000","last_clean":"2023-07-06T03:47:38.032048+0000","last_became_active":"2023-07-06T03:47:36.696241+0000","last_became_peered":"2023-07-06T03:47:36.696241+0000","last_unstale":"2023-07-06T03:47:38.032048+0000","last_undegraded":"2023-07-06T03:47:38.032048+0000","last_fullsized":"2023-07-06T03:47:38.032048+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:21:46.455954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679314+0000","last_change":"2023-07-06T03:47:35.680931+0000","last_active":"2023-07-06T03:47:37.679314+0000","last_peered":"2023-07-06T03:47:37.679314+0000","last_clean":"2023-07-06T03:47:37.679314+0000","last_became_active":"2023-07-06T03:47:35.680659+0000","last_became_peered":"2023-07-06T03:47:35.680659+0000","last_unstale":"2023-07-06T03:47:37.679314+0000","last_undegraded":"2023-07-06T03:47:37.679314+0000","last_fullsized":"2023-07-06T03:47:37.679314+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:12:30.786821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679218+0000","last_change":"2023-07-06T03:47:36.687780+0000","last_active":"2023-07-06T03:47:37.679218+0000","last_peered":"2023-07-06T03:47:37.679218+0000","last_clean":"2023-07-06T03:47:37.679218+0000","last_became_active":"2023-07-06T03:47:36.687490+0000","last_became_peered":"2023-07-06T03:47:36.687490+0000","last_unstale":"2023-07-06T03:47:37.679218+0000","last_undegraded":"2023-07-06T03:47:37.679218+0000","last_fullsized":"2023-07-06T03:47:37.679218+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:18:25.341440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031552+0000","last_change":"2023-07-06T03:47:35.679503+0000","last_active":"2023-07-06T03:47:38.031552+0000","last_peered":"2023-07-06T03:47:38.031552+0000","last_clean":"2023-07-06T03:47:38.031552+0000","last_became_active":"2023-07-06T03:47:35.679231+0000","last_became_peered":"2023-07-06T03:47:35.679231+0000","last_unstale":"2023-07-06T03:47:38.031552+0000","last_undegraded":"2023-07-06T03:47:38.031552+0000","last_fullsized":"2023-07-06T03:47:38.031552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:04:59.473861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679107+0000","last_change":"2023-07-06T03:47:36.699756+0000","last_active":"2023-07-06T03:47:37.679107+0000","last_peered":"2023-07-06T03:47:37.679107+0000","last_clean":"2023-07-06T03:47:37.679107+0000","last_became_active":"2023-07-06T03:47:36.699459+0000","last_became_peered":"2023-07-06T03:47:36.699459+0000","last_unstale":"2023-07-06T03:47:37.679107+0000","last_undegraded":"2023-07-06T03:47:37.679107+0000","last_fullsized":"2023-07-06T03:47:37.679107+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:02:12.017358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679542+0000","last_change":"2023-07-06T03:47:35.685943+0000","last_active":"2023-07-06T03:47:37.679542+0000","last_peered":"2023-07-06T03:47:37.679542+0000","last_clean":"2023-07-06T03:47:37.679542+0000","last_became_active":"2023-07-06T03:47:35.685673+0000","last_became_peered":"2023-07-06T03:47:35.685673+0000","last_unstale":"2023-07-06T03:47:37.679542+0000","last_undegraded":"2023-07-06T03:47:37.679542+0000","last_fullsized":"2023-07-06T03:47:37.679542+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:06:36.141721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679682+0000","last_change":"2023-07-06T03:47:36.685295+0000","last_active":"2023-07-06T03:47:37.679682+0000","last_peered":"2023-07-06T03:47:37.679682+0000","last_clean":"2023-07-06T03:47:37.679682+0000","last_became_active":"2023-07-06T03:47:36.685009+0000","last_became_peered":"2023-07-06T03:47:36.685009+0000","last_unstale":"2023-07-06T03:47:37.679682+0000","last_undegraded":"2023-07-06T03:47:37.679682+0000","last_fullsized":"2023-07-06T03:47:37.679682+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:38:46.257565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032679+0000","last_change":"2023-07-06T03:47:35.684207+0000","last_active":"2023-07-06T03:47:38.032679+0000","last_peered":"2023-07-06T03:47:38.032679+0000","last_clean":"2023-07-06T03:47:38.032679+0000","last_became_active":"2023-07-06T03:47:35.683837+0000","last_became_peered":"2023-07-06T03:47:35.683837+0000","last_unstale":"2023-07-06T03:47:38.032679+0000","last_undegraded":"2023-07-06T03:47:38.032679+0000","last_fullsized":"2023-07-06T03:47:38.032679+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:08:16.691892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031458+0000","last_change":"2023-07-06T03:47:36.680438+0000","last_active":"2023-07-06T03:47:38.031458+0000","last_peered":"2023-07-06T03:47:38.031458+0000","last_clean":"2023-07-06T03:47:38.031458+0000","last_became_active":"2023-07-06T03:47:36.680224+0000","last_became_peered":"2023-07-06T03:47:36.680224+0000","last_unstale":"2023-07-06T03:47:38.031458+0000","last_undegraded":"2023-07-06T03:47:38.031458+0000","last_fullsized":"2023-07-06T03:47:38.031458+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:12:47.040859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032770+0000","last_change":"2023-07-06T03:47:35.680503+0000","last_active":"2023-07-06T03:47:38.032770+0000","last_peered":"2023-07-06T03:47:38.032770+0000","last_clean":"2023-07-06T03:47:38.032770+0000","last_became_active":"2023-07-06T03:47:35.680171+0000","last_became_peered":"2023-07-06T03:47:35.680171+0000","last_unstale":"2023-07-06T03:47:38.032770+0000","last_undegraded":"2023-07-06T03:47:38.032770+0000","last_fullsized":"2023-07-06T03:47:38.032770+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:14:45.275656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032362+0000","last_change":"2023-07-06T03:47:36.687514+0000","last_active":"2023-07-06T03:47:38.032362+0000","last_peered":"2023-07-06T03:47:38.032362+0000","last_clean":"2023-07-06T03:47:38.032362+0000","last_became_active":"2023-07-06T03:47:36.686771+0000","last_became_peered":"2023-07-06T03:47:36.686771+0000","last_unstale":"2023-07-06T03:47:38.032362+0000","last_undegraded":"2023-07-06T03:47:38.032362+0000","last_fullsized":"2023-07-06T03:47:38.032362+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:49:51.634346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032621+0000","last_change":"2023-07-06T03:47:35.680421+0000","last_active":"2023-07-06T03:47:38.032621+0000","last_peered":"2023-07-06T03:47:38.032621+0000","last_clean":"2023-07-06T03:47:38.032621+0000","last_became_active":"2023-07-06T03:47:35.679465+0000","last_became_peered":"2023-07-06T03:47:35.679465+0000","last_unstale":"2023-07-06T03:47:38.032621+0000","last_undegraded":"2023-07-06T03:47:38.032621+0000","last_fullsized":"2023-07-06T03:47:38.032621+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:14.005662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679191+0000","last_change":"2023-07-06T03:47:36.697621+0000","last_active":"2023-07-06T03:47:37.679191+0000","last_peered":"2023-07-06T03:47:37.679191+0000","last_clean":"2023-07-06T03:47:37.679191+0000","last_became_active":"2023-07-06T03:47:36.697357+0000","last_became_peered":"2023-07-06T03:47:36.697357+0000","last_unstale":"2023-07-06T03:47:37.679191+0000","last_undegraded":"2023-07-06T03:47:37.679191+0000","last_fullsized":"2023-07-06T03:47:37.679191+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:27:34.021470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032083+0000","last_change":"2023-07-06T03:47:35.673523+0000","last_active":"2023-07-06T03:47:38.032083+0000","last_peered":"2023-07-06T03:47:38.032083+0000","last_clean":"2023-07-06T03:47:38.032083+0000","last_became_active":"2023-07-06T03:47:35.672470+0000","last_became_peered":"2023-07-06T03:47:35.672470+0000","last_unstale":"2023-07-06T03:47:38.032083+0000","last_undegraded":"2023-07-06T03:47:38.032083+0000","last_fullsized":"2023-07-06T03:47:38.032083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:57:41.537850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033500+0000","last_change":"2023-07-06T03:47:36.696815+0000","last_active":"2023-07-06T03:47:38.033500+0000","last_peered":"2023-07-06T03:47:38.033500+0000","last_clean":"2023-07-06T03:47:38.033500+0000","last_became_active":"2023-07-06T03:47:36.695497+0000","last_became_peered":"2023-07-06T03:47:36.695497+0000","last_unstale":"2023-07-06T03:47:38.033500+0000","last_undegraded":"2023-07-06T03:47:38.033500+0000","last_fullsized":"2023-07-06T03:47:38.033500+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:13.208271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679107+0000","last_change":"2023-07-06T03:47:35.681014+0000","last_active":"2023-07-06T03:47:37.679107+0000","last_peered":"2023-07-06T03:47:37.679107+0000","last_clean":"2023-07-06T03:47:37.679107+0000","last_became_active":"2023-07-06T03:47:35.680766+0000","last_became_peered":"2023-07-06T03:47:35.680766+0000","last_unstale":"2023-07-06T03:47:37.679107+0000","last_undegraded":"2023-07-06T03:47:37.679107+0000","last_fullsized":"2023-07-06T03:47:37.679107+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:50:57.196031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033339+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:47:38.033339+0000","last_peered":"2023-07-06T03:47:38.033339+0000","last_clean":"2023-07-06T03:47:38.033339+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:47:38.033339+0000","last_undegraded":"2023-07-06T03:47:38.033339+0000","last_fullsized":"2023-07-06T03:47:38.033339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:56:55.588115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.030897+0000","last_change":"2023-07-06T03:47:36.690938+0000","last_active":"2023-07-06T03:47:38.030897+0000","last_peered":"2023-07-06T03:47:38.030897+0000","last_clean":"2023-07-06T03:47:38.030897+0000","last_became_active":"2023-07-06T03:47:36.690670+0000","last_became_peered":"2023-07-06T03:47:36.690670+0000","last_unstale":"2023-07-06T03:47:38.030897+0000","last_undegraded":"2023-07-06T03:47:38.030897+0000","last_fullsized":"2023-07-06T03:47:38.030897+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:32:43.768173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679494+0000","last_change":"2023-07-06T03:47:35.684268+0000","last_active":"2023-07-06T03:47:37.679494+0000","last_peered":"2023-07-06T03:47:37.679494+0000","last_clean":"2023-07-06T03:47:37.679494+0000","last_became_active":"2023-07-06T03:47:35.683961+0000","last_became_peered":"2023-07-06T03:47:35.683961+0000","last_unstale":"2023-07-06T03:47:37.679494+0000","last_undegraded":"2023-07-06T03:47:37.679494+0000","last_fullsized":"2023-07-06T03:47:37.679494+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:24:07.434670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680123+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:47:37.680123+0000","last_peered":"2023-07-06T03:47:37.680123+0000","last_clean":"2023-07-06T03:47:37.680123+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:47:37.680123+0000","last_undegraded":"2023-07-06T03:47:37.680123+0000","last_fullsized":"2023-07-06T03:47:37.680123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:44:52.349652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033210+0000","last_change":"2023-07-06T03:47:36.681576+0000","last_active":"2023-07-06T03:47:38.033210+0000","last_peered":"2023-07-06T03:47:38.033210+0000","last_clean":"2023-07-06T03:47:38.033210+0000","last_became_active":"2023-07-06T03:47:36.680747+0000","last_became_peered":"2023-07-06T03:47:36.680747+0000","last_unstale":"2023-07-06T03:47:38.033210+0000","last_undegraded":"2023-07-06T03:47:38.033210+0000","last_fullsized":"2023-07-06T03:47:38.033210+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:16:03.409941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033433+0000","last_change":"2023-07-06T03:47:35.677576+0000","last_active":"2023-07-06T03:47:38.033433+0000","last_peered":"2023-07-06T03:47:38.033433+0000","last_clean":"2023-07-06T03:47:38.033433+0000","last_became_active":"2023-07-06T03:47:35.677247+0000","last_became_peered":"2023-07-06T03:47:35.677247+0000","last_unstale":"2023-07-06T03:47:38.033433+0000","last_undegraded":"2023-07-06T03:47:38.033433+0000","last_fullsized":"2023-07-06T03:47:38.033433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:28:55.104375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678517+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:47:37.678517+0000","last_peered":"2023-07-06T03:47:37.678517+0000","last_clean":"2023-07-06T03:47:37.678517+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:47:37.678517+0000","last_undegraded":"2023-07-06T03:47:37.678517+0000","last_fullsized":"2023-07-06T03:47:37.678517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:47.962728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.030800+0000","last_change":"2023-07-06T03:47:36.680474+0000","last_active":"2023-07-06T03:47:38.030800+0000","last_peered":"2023-07-06T03:47:38.030800+0000","last_clean":"2023-07-06T03:47:38.030800+0000","last_became_active":"2023-07-06T03:47:36.680211+0000","last_became_peered":"2023-07-06T03:47:36.680211+0000","last_unstale":"2023-07-06T03:47:38.030800+0000","last_undegraded":"2023-07-06T03:47:38.030800+0000","last_fullsized":"2023-07-06T03:47:38.030800+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:01:40.851047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033583+0000","last_change":"2023-07-06T03:47:35.677003+0000","last_active":"2023-07-06T03:47:38.033583+0000","last_peered":"2023-07-06T03:47:38.033583+0000","last_clean":"2023-07-06T03:47:38.033583+0000","last_became_active":"2023-07-06T03:47:35.676727+0000","last_became_peered":"2023-07-06T03:47:35.676727+0000","last_unstale":"2023-07-06T03:47:38.033583+0000","last_undegraded":"2023-07-06T03:47:38.033583+0000","last_fullsized":"2023-07-06T03:47:38.033583+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:25:46.523518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679884+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:47:37.679884+0000","last_peered":"2023-07-06T03:47:37.679884+0000","last_clean":"2023-07-06T03:47:37.679884+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:47:37.679884+0000","last_undegraded":"2023-07-06T03:47:37.679884+0000","last_fullsized":"2023-07-06T03:47:37.679884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:05.435850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":210,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031054+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:47:38.031054+0000","last_peered":"2023-07-06T03:47:38.031054+0000","last_clean":"2023-07-06T03:47:38.031054+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:47:38.031054+0000","last_undegraded":"2023-07-06T03:47:38.031054+0000","last_fullsized":"2023-07-06T03:47:38.031054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:06:20.238158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032908+0000","last_change":"2023-07-06T03:47:36.688459+0000","last_active":"2023-07-06T03:47:38.032908+0000","last_peered":"2023-07-06T03:47:38.032908+0000","last_clean":"2023-07-06T03:47:38.032908+0000","last_became_active":"2023-07-06T03:47:36.688163+0000","last_became_peered":"2023-07-06T03:47:36.688163+0000","last_unstale":"2023-07-06T03:47:38.032908+0000","last_undegraded":"2023-07-06T03:47:38.032908+0000","last_fullsized":"2023-07-06T03:47:38.032908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:31:04.312174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680109+0000","last_change":"2023-07-06T03:47:35.679794+0000","last_active":"2023-07-06T03:47:37.680109+0000","last_peered":"2023-07-06T03:47:37.680109+0000","last_clean":"2023-07-06T03:47:37.680109+0000","last_became_active":"2023-07-06T03:47:35.679116+0000","last_became_peered":"2023-07-06T03:47:35.679116+0000","last_unstale":"2023-07-06T03:47:37.680109+0000","last_undegraded":"2023-07-06T03:47:37.680109+0000","last_fullsized":"2023-07-06T03:47:37.680109+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:25:58.524698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679414+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:47:37.679414+0000","last_peered":"2023-07-06T03:47:37.679414+0000","last_clean":"2023-07-06T03:47:37.679414+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:47:37.679414+0000","last_undegraded":"2023-07-06T03:47:37.679414+0000","last_fullsized":"2023-07-06T03:47:37.679414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:10.714832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.678973+0000","last_change":"2023-07-06T03:47:36.691134+0000","last_active":"2023-07-06T03:47:37.678973+0000","last_peered":"2023-07-06T03:47:37.678973+0000","last_clean":"2023-07-06T03:47:37.678973+0000","last_became_active":"2023-07-06T03:47:36.690881+0000","last_became_peered":"2023-07-06T03:47:36.690881+0000","last_unstale":"2023-07-06T03:47:37.678973+0000","last_undegraded":"2023-07-06T03:47:37.678973+0000","last_fullsized":"2023-07-06T03:47:37.678973+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:40:50.948093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.031070+0000","last_change":"2023-07-06T03:47:35.684944+0000","last_active":"2023-07-06T03:47:38.031070+0000","last_peered":"2023-07-06T03:47:38.031070+0000","last_clean":"2023-07-06T03:47:38.031070+0000","last_became_active":"2023-07-06T03:47:35.684654+0000","last_became_peered":"2023-07-06T03:47:35.684654+0000","last_unstale":"2023-07-06T03:47:38.031070+0000","last_undegraded":"2023-07-06T03:47:38.031070+0000","last_fullsized":"2023-07-06T03:47:38.031070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:22:20.323454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.680206+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:47:37.680206+0000","last_peered":"2023-07-06T03:47:37.680206+0000","last_clean":"2023-07-06T03:47:37.680206+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:47:37.680206+0000","last_undegraded":"2023-07-06T03:47:37.680206+0000","last_fullsized":"2023-07-06T03:47:37.680206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:30:57.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033112+0000","last_change":"2023-07-06T03:47:36.697417+0000","last_active":"2023-07-06T03:47:38.033112+0000","last_peered":"2023-07-06T03:47:38.033112+0000","last_clean":"2023-07-06T03:47:38.033112+0000","last_became_active":"2023-07-06T03:47:36.693078+0000","last_became_peered":"2023-07-06T03:47:36.693078+0000","last_unstale":"2023-07-06T03:47:38.033112+0000","last_undegraded":"2023-07-06T03:47:38.033112+0000","last_fullsized":"2023-07-06T03:47:38.033112+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:12:57.654204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.033166+0000","last_change":"2023-07-06T03:47:35.677115+0000","last_active":"2023-07-06T03:47:38.033166+0000","last_peered":"2023-07-06T03:47:38.033166+0000","last_clean":"2023-07-06T03:47:38.033166+0000","last_became_active":"2023-07-06T03:47:35.676826+0000","last_became_peered":"2023-07-06T03:47:35.676826+0000","last_unstale":"2023-07-06T03:47:38.033166+0000","last_undegraded":"2023-07-06T03:47:38.033166+0000","last_fullsized":"2023-07-06T03:47:38.033166+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:19:18.723847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":58,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032695+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:47:38.032695+0000","last_peered":"2023-07-06T03:47:38.032695+0000","last_clean":"2023-07-06T03:47:38.032695+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:47:38.032695+0000","last_undegraded":"2023-07-06T03:47:38.032695+0000","last_fullsized":"2023-07-06T03:47:38.032695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:19.297290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679641+0000","last_change":"2023-07-06T03:47:36.688377+0000","last_active":"2023-07-06T03:47:37.679641+0000","last_peered":"2023-07-06T03:47:37.679641+0000","last_clean":"2023-07-06T03:47:37.679641+0000","last_became_active":"2023-07-06T03:47:36.688094+0000","last_became_peered":"2023-07-06T03:47:36.688094+0000","last_unstale":"2023-07-06T03:47:37.679641+0000","last_undegraded":"2023-07-06T03:47:37.679641+0000","last_fullsized":"2023-07-06T03:47:37.679641+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:35.650072+0000","last_clean_scrub_stamp":"2023-07-06T03:47:35.650072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:31:34.132094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:38.032932+0000","last_change":"2023-07-06T03:47:35.680550+0000","last_active":"2023-07-06T03:47:38.032932+0000","last_peered":"2023-07-06T03:47:38.032932+0000","last_clean":"2023-07-06T03:47:38.032932+0000","last_became_active":"2023-07-06T03:47:35.680201+0000","last_became_peered":"2023-07-06T03:47:35.680201+0000","last_unstale":"2023-07-06T03:47:38.032932+0000","last_undegraded":"2023-07-06T03:47:38.032932+0000","last_fullsized":"2023-07-06T03:47:38.032932+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:47:34.647154+0000","last_clean_scrub_stamp":"2023-07-06T03:47:34.647154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:40:39.427226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-07-06T03:47:37.679619+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:47:37.679619+0000","last_peered":"2023-07-06T03:47:37.679619+0000","last_clean":"2023-07-06T03:47:37.679619+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:47:37.679619+0000","last_undegraded":"2023-07-06T03:47:37.679619+0000","last_fullsized":"2023-07-06T03:47:37.679619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:26:16.501757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":561152,"data_stored":237108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86399999999999999,"5min":1.345,"15min":1.345},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.81499999999999995,"15min":0.83699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91100000000000003,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.748,"15min":0.76300000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85799999999999998,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.91800000000000004,"15min":0.95899999999999996},"min":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.876,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.82399999999999995,"15min":0.84699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93200000000000005,"5min":3.77,"15min":3.77},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91100000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66300000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88400000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95699999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.626}]},{"osd":4,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.94699999999999995,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.77200000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96599999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.499},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.64100000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.746,"15min":0.746},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98799999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.80500000000000005,"15min":0.81399999999999995},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92800000000000005,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.75}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":557056,"data_stored":237634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.79400000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74199999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83899999999999997,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.78600000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.65800000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77800000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.44900000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.874,"5min":1.119,"15min":1.119},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.76500000000000001,"15min":0.78600000000000003},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.73999999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78200000000000003,"5min":1.093,"15min":1.093},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93500000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.746,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.81000000000000005,"5min":1.085,"15min":1.085},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72599999999999998,"5min":1.218,"15min":1.218},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.80200000000000005,"15min":0.81899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0269999999999999,"5min":1.198,"15min":1.198},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.84299999999999997,"15min":0.86699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.83799999999999997,"5min":1.323,"15min":1.323},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.84899999999999998,"15min":0.877},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86799999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":1124,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1150976,"data_stored":827490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58399999999999996,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.31},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.58999999999999997,"15min":0.60999999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.628,"5min":1.96,"15min":1.96},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":1.0229999999999999,"15min":1.119},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54600000000000004,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.65800000000000003,"15min":0.68500000000000005},"min":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67900000000000005,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.38800000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.63400000000000001,"15min":0.65600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.752,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.437},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.59499999999999997,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":1.2789999999999999,"15min":1.375},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86199999999999999},"min":{"1min":0.48099999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94299999999999995,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.78500000000000003,"15min":0.80100000000000005},"min":{"1min":0.54100000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92100000000000004,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.2929999999999999,"15min":1.393},"min":{"1min":0.55900000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.874,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.60599999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.90600000000000003,"5min":1.365,"15min":1.365},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.335,"15min":1.4370000000000001},"min":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.88500000000000001,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.84899999999999998,"15min":0.875},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.86299999999999999,"5min":2.806,"15min":2.806},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.82999999999999996,"15min":0.84599999999999997},"min":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.93000000000000005,"5min":1.337,"15min":1.337},"last":0.72699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":557056,"data_stored":237634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41699999999999998,"15min":0.41799999999999998},"min":{"1min":0.316,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.434},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.443,"15min":0.441},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47599999999999998,"15min":0.47999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60199999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80700000000000005,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46999999999999997,"15min":0.46700000000000003},"min":{"1min":0.38800000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.74199999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":0.76300000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97899999999999998,"5min":1.171,"15min":1.171},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.875}]},{"osd":5,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.87,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91700000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.83299999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.88500000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.546,"15min":1.546},"last":0.81999999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81000000000000005,"15min":0.81799999999999995},"min":{"1min":0.59199999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96799999999999997,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":1.155,"15min":1.2290000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.95599999999999996,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.70899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35688,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336152,"statfs":{"total":96636764160,"available":96600219648,"internally_reserved":0,"allocated":557056,"data_stored":237086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.57299999999999995,"5min":0.626,"15min":0.626},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.52600000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.49399999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.66400000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66800000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.61099999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69299999999999995,"15min":0.69399999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":1.165,"15min":1.165},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.751},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.90500000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.746,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91400000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:47:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.90500000000000003,"5min":2.157,"15min":2.157},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.61299999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88900000000000001,"5min":1.966,"15min":1.966},"last":0.81799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":557056,"data_stored":237142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.65700000000000003,"15min":0.69699999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.40699999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60599999999999998,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":1.196,"15min":1.3260000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63300000000000001,"5min":15.911,"15min":15.911},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.266,"15min":1.3959999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63700000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.55600000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":1.2709999999999999,"15min":1.401},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.59699999999999998,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.76400000000000001,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84899999999999998,"5min":1.179,"15min":1.179},"last":0.747},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.97699999999999998,"15min":1.028},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89500000000000002,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.97999999999999998,"15min":1.0269999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92700000000000005,"5min":5.077,"15min":5.077},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.0409999999999999,"15min":1.0940000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89600000000000002,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.52500000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.90200000000000002,"5min":1.143,"15min":1.143},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.995,"15min":1.0449999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96799999999999997,"5min":5.157,"15min":5.157},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:47:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.85099999999999998,"15min":0.878},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96399999999999997,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.006,"15min":1.0549999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.93000000000000005,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.877}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31608,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":573440,"data_stored":238708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.628},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.91500000000000004,"15min":0.95899999999999996},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94399999999999995,"5min":6.258,"15min":6.258},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.93400000000000005,"15min":0.97299999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1419999999999999,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":1.0189999999999999,"15min":1.069},"min":{"1min":0.54800000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0089999999999999,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.073,"5min":1.413,"15min":1.413},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80800000000000005,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.873,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81899999999999995},"min":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2330000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.028,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.76700000000000002,"15min":0.77700000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1159168,"data_stored":828055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.90500000000000003,"15min":0.95499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94199999999999995,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81100000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68500000000000005,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.628},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90400000000000003,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95399999999999996,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.83699999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69299999999999995,"15min":0.68500000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.81999999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79600000000000004,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.75900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:47:46.450 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T03:47:46.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T03:47:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.683+0000 7fc8b09ff700 1 Processor -- start 2023-07-06T03:47:46.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.683+0000 7fc8b09ff700 1 -- start start 2023-07-06T03:47:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.684+0000 7fc8b09ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8ac0b4ea0 0x7fc8ac0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.684+0000 7fc8b09ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8ac160a50 0x7fc8ac0d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.684+0000 7fc8b09ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac160eb0 con 0x7fc8ac0b8590 2023-07-06T03:47:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.684+0000 7fc8b09ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ac0d0c10 con 0x7fc8ac0b4ea0 2023-07-06T03:47:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.684+0000 7fc8b09ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac0d08e0 con 0x7fc8ac160a50 2023-07-06T03:47:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.684+0000 7fc8a3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8ac0b4ea0 0x7fc8ac0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.684+0000 7fc8a3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8ac0b4ea0 0x7fc8ac0cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33232/0 (socket says 172.21.15.97:33232) 2023-07-06T03:47:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.684+0000 7fc8a3fff700 1 -- 172.21.15.97:0/1124024304 learned_addr learned my addr 172.21.15.97:0/1124024304 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.685+0000 7fc8aad9d700 1 --2- 172.21.15.97:0/1124024304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8ac160a50 0x7fc8ac0d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.685+0000 7fc8a3fff700 1 -- 172.21.15.97:0/1124024304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8ac160a50 msgr2=0x7fc8ac0d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.685+0000 7fc8a3fff700 1 --2- 172.21.15.97:0/1124024304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8ac160a50 0x7fc8ac0d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.685+0000 7fc8a3fff700 1 -- 172.21.15.97:0/1124024304 >> v1:172.21.15.146:6789/0 conn(0x7fc8ac0b8590 legacy=0x7fc8ac0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.685+0000 7fc8a3fff700 1 -- 172.21.15.97:0/1124024304 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac0b2560 con 0x7fc8ac0b4ea0 2023-07-06T03:47:46.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.685+0000 7fc8a3fff700 1 --2- 172.21.15.97:0/1124024304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8ac0b4ea0 0x7fc8ac0cef60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc8940098f0 tx=0x7fc894009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d34421e0f0d7e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:46.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.685+0000 7fc8a9d9b700 1 -- 172.21.15.97:0/1124024304 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400b8a0 con 0x7fc8ac0b4ea0 2023-07-06T03:47:46.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.685+0000 7fc8a9d9b700 1 -- 172.21.15.97:0/1124024304 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89400ba00 con 0x7fc8ac0b4ea0 2023-07-06T03:47:46.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.685+0000 7fc8a9d9b700 1 -- 172.21.15.97:0/1124024304 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400bd90 con 0x7fc8ac0b4ea0 2023-07-06T03:47:46.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.686+0000 7fc8b09ff700 1 -- 172.21.15.97:0/1124024304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8ac0b4ea0 msgr2=0x7fc8ac0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.686+0000 7fc8b09ff700 1 --2- 172.21.15.97:0/1124024304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8ac0b4ea0 0x7fc8ac0cef60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc8940098f0 tx=0x7fc894009c00 comp rx=0 tx=0).stop 2023-07-06T03:47:46.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.686+0000 7fc8b09ff700 1 -- 172.21.15.97:0/1124024304 shutdown_connections 2023-07-06T03:47:46.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.686+0000 7fc8b09ff700 1 --2- 172.21.15.97:0/1124024304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8ac160a50 0x7fc8ac0d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.686+0000 7fc8b09ff700 1 --2- 172.21.15.97:0/1124024304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8ac0b4ea0 0x7fc8ac0cef60 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.686+0000 7fc8b09ff700 1 -- 172.21.15.97:0/1124024304 >> 172.21.15.97:0/1124024304 conn(0x7fc8ac06b290 msgr2=0x7fc8ac0cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:46.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.686+0000 7fc8b09ff700 1 -- 172.21.15.97:0/1124024304 shutdown_connections 2023-07-06T03:47:46.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.686+0000 7fc8b09ff700 1 -- 172.21.15.97:0/1124024304 wait complete. 2023-07-06T03:47:46.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.688+0000 7fc8b09ff700 1 Processor -- start 2023-07-06T03:47:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.689+0000 7fc8b09ff700 1 -- start start 2023-07-06T03:47:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8b09ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8ac0b4ea0 0x7fc8ac1e6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8b09ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8ac0b8590 0x7fc8ac0cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8b09ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8ac160a50 0x7fc8ac0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8b09ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac0ccb60 con 0x7fc8ac160a50 2023-07-06T03:47:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8b09ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac0cac40 con 0x7fc8ac0b4ea0 2023-07-06T03:47:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8b09ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac156540 con 0x7fc8ac0b8590 2023-07-06T03:47:46.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8aad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8ac160a50 0x7fc8ac0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:46.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8aad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8ac160a50 0x7fc8ac0cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33244/0 (socket says 172.21.15.97:33244) 2023-07-06T03:47:46.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8aad9d700 1 -- 172.21.15.97:0/96396565 learned_addr learned my addr 172.21.15.97:0/96396565 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:46.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8aa59c700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8ac0b4ea0 0x7fc8ac1e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:46.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8a3fff700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8ac0b8590 0x7fc8ac0cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:46.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8aad9d700 1 -- 172.21.15.97:0/96396565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8ac0b8590 msgr2=0x7fc8ac0cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8aad9d700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8ac0b8590 0x7fc8ac0cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8aad9d700 1 -- 172.21.15.97:0/96396565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8ac0b4ea0 msgr2=0x7fc8ac1e6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8aad9d700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8ac0b4ea0 0x7fc8ac1e6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.690+0000 7fc8aad9d700 1 -- 172.21.15.97:0/96396565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc894009550 con 0x7fc8ac160a50 2023-07-06T03:47:46.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.691+0000 7fc8aad9d700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8ac160a50 0x7fc8ac0cc620 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4007e60 tx=0x7fc8a400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:46.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.691+0000 7fc8a2ffd700 1 -- 172.21.15.97:0/96396565 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a400ede0 con 0x7fc8ac160a50 2023-07-06T03:47:46.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.691+0000 7fc8a2ffd700 1 -- 172.21.15.97:0/96396565 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a4014420 con 0x7fc8ac160a50 2023-07-06T03:47:46.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.691+0000 7fc8b09ff700 1 -- 172.21.15.97:0/96396565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac0c9de0 con 0x7fc8ac160a50 2023-07-06T03:47:46.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.691+0000 7fc8a2ffd700 1 -- 172.21.15.97:0/96396565 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a40147d0 con 0x7fc8ac160a50 2023-07-06T03:47:46.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.691+0000 7fc8b09ff700 1 -- 172.21.15.97:0/96396565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac0ca3a0 con 0x7fc8ac160a50 2023-07-06T03:47:46.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.692+0000 7fc8b09ff700 1 -- 172.21.15.97:0/96396565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ac04e340 con 0x7fc8ac160a50 2023-07-06T03:47:46.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.693+0000 7fc8a2ffd700 1 -- 172.21.15.97:0/96396565 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc8a4014930 con 0x7fc8ac160a50 2023-07-06T03:47:46.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.694+0000 7fc8a2ffd700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc88c06ace0 0x7fc88c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:46.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.694+0000 7fc8a2ffd700 1 -- 172.21.15.97:0/96396565 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc8a4089b40 con 0x7fc8ac160a50 2023-07-06T03:47:46.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.694+0000 7fc8aa59c700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc88c06ace0 0x7fc88c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:46.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.695+0000 7fc8aa59c700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc88c06ace0 0x7fc88c06d190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc89c007fd0 tx=0x7fc89c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:46.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.698+0000 7fc8a2ffd700 1 -- 172.21.15.97:0/96396565 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc8a4054c00 con 0x7fc8ac160a50 2023-07-06T03:47:46.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.974+0000 7fc8b09ff700 1 -- 172.21.15.97:0/96396565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc8ac0d00c0 con 0x7fc8ac160a50 2023-07-06T03:47:46.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.978+0000 7fc8a2ffd700 1 -- 172.21.15.97:0/96396565 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc8a4058220 con 0x7fc8ac160a50 2023-07-06T03:47:46.980 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:47:46.980 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg== 2023-07-06T03:47:46.980 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-07-06T03:47:46.980 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-07-06T03:47:46.981 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T03:47:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.980+0000 7fc8b09ff700 1 -- 172.21.15.97:0/96396565 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc88c06ace0 msgr2=0x7fc88c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.980+0000 7fc8b09ff700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc88c06ace0 0x7fc88c06d190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc89c007fd0 tx=0x7fc89c009040 comp rx=0 tx=0).stop 2023-07-06T03:47:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.980+0000 7fc8b09ff700 1 -- 172.21.15.97:0/96396565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8ac160a50 msgr2=0x7fc8ac0cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.980+0000 7fc8b09ff700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8ac160a50 0x7fc8ac0cc620 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4007e60 tx=0x7fc8a400e3c0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.980+0000 7fc8b09ff700 1 -- 172.21.15.97:0/96396565 shutdown_connections 2023-07-06T03:47:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.980+0000 7fc8b09ff700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc88c06ace0 0x7fc88c06d190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.980+0000 7fc8b09ff700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8ac160a50 0x7fc8ac0cc620 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.980+0000 7fc8b09ff700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8ac0b4ea0 0x7fc8ac1e6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.980+0000 7fc8b09ff700 1 --2- 172.21.15.97:0/96396565 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc8ac0b8590 0x7fc8ac0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.980+0000 7fc8b09ff700 1 -- 172.21.15.97:0/96396565 >> 172.21.15.97:0/96396565 conn(0x7fc8ac06b290 msgr2=0x7fc8ac0b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.981+0000 7fc8b09ff700 1 -- 172.21.15.97:0/96396565 shutdown_connections 2023-07-06T03:47:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:46.981+0000 7fc8b09ff700 1 -- 172.21.15.97:0/96396565 wait complete. 2023-07-06T03:47:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-07-06T03:47:47.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:47:47.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.193+0000 7f84c71ff700 1 Processor -- start 2023-07-06T03:47:47.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.194+0000 7f84c71ff700 1 -- start start 2023-07-06T03:47:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.195+0000 7f84c71ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f84c00cd390 0x7f84c0160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.195+0000 7f84c71ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c00cdae0 con 0x7f84c015c520 2023-07-06T03:47:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.195+0000 7f84c71ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c00b6640 con 0x7f84c0160dd0 2023-07-06T03:47:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.195+0000 7f84c71ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c00b6920 con 0x7f84c00cd390 2023-07-06T03:47:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.195+0000 7f84bffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f84c00cd390 0x7f84c0160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:47.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.195+0000 7f84c4f9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f84c015c520 0x7f84c0160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46606/0 (socket says 172.21.15.97:46606) 2023-07-06T03:47:47.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.195+0000 7f84c4f9b700 1 -- 172.21.15.97:0/3068968439 learned_addr learned my addr 172.21.15.97:0/3068968439 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:47.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.195+0000 7f84bffff700 1 -- 172.21.15.97:0/3068968439 >> v1:172.21.15.146:6789/0 conn(0x7f84c0160dd0 legacy=0x7f84c00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:47.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.195+0000 7f84bf7fe700 1 -- 172.21.15.97:0/3068968439 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945881042 0 0) 0x7f84c00cdae0 con 0x7f84c015c520 2023-07-06T03:47:47.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.195+0000 7f84bffff700 1 -- 172.21.15.97:0/3068968439 >> v1:172.21.15.97:6789/0 conn(0x7f84c015c520 legacy=0x7f84c0160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:47.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.196+0000 7f84bffff700 1 -- 172.21.15.97:0/3068968439 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c00b6c00 con 0x7f84c00cd390 2023-07-06T03:47:47.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.196+0000 7f84bffff700 1 --2- 172.21.15.97:0/3068968439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c00cd390 0x7f84c0160890 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009840 tx=0x7f84ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16128383dadf118 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:47.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.196+0000 7f84bf7fe700 1 -- 172.21.15.97:0/3068968439 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac011830 con 0x7f84c00cd390 2023-07-06T03:47:47.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.196+0000 7f84bf7fe700 1 -- 172.21.15.97:0/3068968439 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac011990 con 0x7f84c00cd390 2023-07-06T03:47:47.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.196+0000 7f84bf7fe700 1 -- 172.21.15.97:0/3068968439 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac011d20 con 0x7f84c00cd390 2023-07-06T03:47:47.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.196+0000 7f84c71ff700 1 -- 172.21.15.97:0/3068968439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c00cd390 msgr2=0x7f84c0160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:47.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.196+0000 7f84c71ff700 1 --2- 172.21.15.97:0/3068968439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c00cd390 0x7f84c0160890 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009840 tx=0x7f84ac009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:47.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.197+0000 7f84c71ff700 1 -- 172.21.15.97:0/3068968439 shutdown_connections 2023-07-06T03:47:47.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.197+0000 7f84c71ff700 1 --2- 172.21.15.97:0/3068968439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c00cd390 0x7f84c0160890 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:47.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.197+0000 7f84c71ff700 1 -- 172.21.15.97:0/3068968439 >> 172.21.15.97:0/3068968439 conn(0x7f84c0071790 msgr2=0x7f84c0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:47.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.197+0000 7f84c71ff700 1 -- 172.21.15.97:0/3068968439 shutdown_connections 2023-07-06T03:47:47.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.197+0000 7f84c71ff700 1 -- 172.21.15.97:0/3068968439 wait complete. 2023-07-06T03:47:47.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.198+0000 7f84c71ff700 1 Processor -- start 2023-07-06T03:47:47.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.198+0000 7f84c71ff700 1 -- start start 2023-07-06T03:47:47.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.199+0000 7f84c71ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c00cd390 0x7f84c014cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:47.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.199+0000 7f84c71ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84c015c520 0x7f84c015eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:47.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.199+0000 7f84c71ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84c0160dd0 0x7f84c0154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:47.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.199+0000 7f84c71ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c01e74e0 con 0x7f84c00cd390 2023-07-06T03:47:47.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.199+0000 7f84c71ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c015db80 con 0x7f84c0160dd0 2023-07-06T03:47:47.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.199+0000 7f84c71ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c014d270 con 0x7f84c015c520 2023-07-06T03:47:47.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c4f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c00cd390 0x7f84c014cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:47.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c4f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c00cd390 0x7f84c014cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33268/0 (socket says 172.21.15.97:33268) 2023-07-06T03:47:47.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c4f9b700 1 -- 172.21.15.97:0/3437970906 learned_addr learned my addr 172.21.15.97:0/3437970906 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84bffff700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84c015c520 0x7f84c015eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c579c700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84c0160dd0 0x7f84c0154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c4f9b700 1 -- 172.21.15.97:0/3437970906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84c015c520 msgr2=0x7f84c015eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c4f9b700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84c015c520 0x7f84c015eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c4f9b700 1 -- 172.21.15.97:0/3437970906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84c0160dd0 msgr2=0x7f84c0154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c4f9b700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84c0160dd0 0x7f84c0154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c4f9b700 1 -- 172.21.15.97:0/3437970906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac0094a0 con 0x7f84c00cd390 2023-07-06T03:47:47.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c579c700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84c0160dd0 0x7f84c0154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:47.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c4f9b700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c00cd390 0x7f84c014cd30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f84b400c520 tx=0x7f84b400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:47.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84bd7fa700 1 -- 172.21.15.97:0/3437970906 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4004c80 con 0x7f84c00cd390 2023-07-06T03:47:47.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84bd7fa700 1 -- 172.21.15.97:0/3437970906 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b4004e00 con 0x7f84c00cd390 2023-07-06T03:47:47.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84c71ff700 1 -- 172.21.15.97:0/3437970906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c014d710 con 0x7f84c00cd390 2023-07-06T03:47:47.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.200+0000 7f84bd7fa700 1 -- 172.21.15.97:0/3437970906 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4007310 con 0x7f84c00cd390 2023-07-06T03:47:47.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.201+0000 7f84c71ff700 1 -- 172.21.15.97:0/3437970906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c015e070 con 0x7f84c00cd390 2023-07-06T03:47:47.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.201+0000 7f84c71ff700 1 -- 172.21.15.97:0/3437970906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c00b6200 con 0x7f84c00cd390 2023-07-06T03:47:47.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.202+0000 7f84bd7fa700 1 -- 172.21.15.97:0/3437970906 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f84b4007470 con 0x7f84c00cd390 2023-07-06T03:47:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.203+0000 7f84bd7fa700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84a406ace0 0x7f84a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.204+0000 7f84bd7fa700 1 -- 172.21.15.97:0/3437970906 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f84b4089100 con 0x7f84c00cd390 2023-07-06T03:47:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.204+0000 7f84bffff700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84a406ace0 0x7f84a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.204+0000 7f84bffff700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84a406ace0 0x7f84a406d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0090b0 tx=0x7f84ac01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.208+0000 7f84bd7fa700 1 -- 172.21.15.97:0/3437970906 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f84b40541b0 con 0x7f84c00cd390 2023-07-06T03:47:47.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.450+0000 7f84c71ff700 1 -- 172.21.15.97:0/3437970906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84c004e340 con 0x7f84c00cd390 2023-07-06T03:47:47.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.451+0000 7f84bd7fa700 1 -- 172.21.15.97:0/3437970906 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f84b40577d0 con 0x7f84c00cd390 2023-07-06T03:47:47.453 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:47:47.453 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1299982704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1299982704},{"type":"v1","addr":"172.21.15.146:6833","nonce":1299982704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5602,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/640394657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":640394657},{"type":"v1","addr":"172.21.15.97:6837","nonce":640394657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2363940464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2363940464},{"type":"v1","addr":"172.21.15.97:6836","nonce":2363940464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:47:36.149445+0000","modified":"2023-07-06T03:47:39.683749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5838},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5838":{"gid":5838,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/3623506860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3623506860},{"type":"v1","addr":"172.21.15.146:6835","nonce":3623506860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T03:47:47.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.453+0000 7f84c71ff700 1 -- 172.21.15.97:0/3437970906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84a406ace0 msgr2=0x7f84a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:47.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.453+0000 7f84c71ff700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84a406ace0 0x7f84a406d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0090b0 tx=0x7f84ac01c040 comp rx=0 tx=0).stop 2023-07-06T03:47:47.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.454+0000 7f84c71ff700 1 -- 172.21.15.97:0/3437970906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c00cd390 msgr2=0x7f84c014cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:47.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.454+0000 7f84c71ff700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c00cd390 0x7f84c014cd30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f84b400c520 tx=0x7f84b400c830 comp rx=0 tx=0).stop 2023-07-06T03:47:47.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.454+0000 7f84c71ff700 1 -- 172.21.15.97:0/3437970906 shutdown_connections 2023-07-06T03:47:47.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.454+0000 7f84c71ff700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84a406ace0 0x7f84a406d190 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:47.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.454+0000 7f84c71ff700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84c0160dd0 0x7f84c0154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:47.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.454+0000 7f84c71ff700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c00cd390 0x7f84c014cd30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:47.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.454+0000 7f84c71ff700 1 --2- 172.21.15.97:0/3437970906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84c015c520 0x7f84c015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:47.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.454+0000 7f84c71ff700 1 -- 172.21.15.97:0/3437970906 >> 172.21.15.97:0/3437970906 conn(0x7f84c0071790 msgr2=0x7f84c00c95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:47.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.454+0000 7f84c71ff700 1 -- 172.21.15.97:0/3437970906 shutdown_connections 2023-07-06T03:47:47.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:47.454+0000 7f84c71ff700 1 -- 172.21.15.97:0/3437970906 wait complete. 2023-07-06T03:47:47.458 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2023-07-06T03:47:47.875 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T03:47:36.149445+0000', 'modified': '2023-07-06T03:47:39.683749+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5838}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5838': {'gid': 5838, 'name': 'd', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6835/3623506860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 3623506860}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 3623506860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T03:47:47.876 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T03:47:47.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-07-06T03:47:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.071+0000 7fc4e77ff700 1 Processor -- start 2023-07-06T03:47:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.072+0000 7fc4e77ff700 1 -- start start 2023-07-06T03:47:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.073+0000 7fc4e77ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4e00d15a0 0x7fc4e00cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.073+0000 7fc4e77ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4e00cfed0 0x7fc4e00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.073+0000 7fc4e77ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e00d2d30 con 0x7fc4e00b4a40 2023-07-06T03:47:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.073+0000 7fc4e77ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e00b6520 con 0x7fc4e00d15a0 2023-07-06T03:47:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.073+0000 7fc4e77ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e00d1a10 con 0x7fc4e00cfed0 2023-07-06T03:47:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.073+0000 7fc4e559b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc4e00b4a40 0x7fc4e00b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46614/0 (socket says 172.21.15.97:46614) 2023-07-06T03:47:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.073+0000 7fc4e559b700 1 -- 172.21.15.97:0/3326338071 learned_addr learned my addr 172.21.15.97:0/3326338071 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.073+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/3326338071 >> v2:172.21.15.97:3300/0 conn(0x7fc4e00d15a0 0x7fc4e00cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.073+0000 7fc4e5d9c700 1 --2- 172.21.15.97:0/3326338071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4e00cfed0 0x7fc4e00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.073+0000 7fc4d7fff700 1 -- 172.21.15.97:0/3326338071 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 733383151 0 0) 0x7fc4e00d2d30 con 0x7fc4e00b4a40 2023-07-06T03:47:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.074+0000 7fc4d7fff700 1 -- 172.21.15.97:0/3326338071 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4cc0035b0 con 0x7fc4e00b4a40 2023-07-06T03:47:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.074+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/3326338071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4e00cfed0 msgr2=0x7fc4e00b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.074+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/3326338071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4e00cfed0 0x7fc4e00b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.074+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/3326338071 >> v1:172.21.15.97:6789/0 conn(0x7fc4e00b4a40 legacy=0x7fc4e00b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.074+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/3326338071 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e00cf210 con 0x7fc4e00d15a0 2023-07-06T03:47:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.074+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/3326338071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e00d15a0 0x7fc4e00cf990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009840 tx=0x7fc4d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873a9fcb16b967ef server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.074+0000 7fc4d7fff700 1 -- 172.21.15.97:0/3326338071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0011830 con 0x7fc4e00d15a0 2023-07-06T03:47:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.074+0000 7fc4d7fff700 1 -- 172.21.15.97:0/3326338071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d0011990 con 0x7fc4e00d15a0 2023-07-06T03:47:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.074+0000 7fc4d7fff700 1 -- 172.21.15.97:0/3326338071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0011d20 con 0x7fc4e00d15a0 2023-07-06T03:47:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.075+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3326338071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e00d15a0 msgr2=0x7fc4e00cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.075+0000 7fc4e77ff700 1 --2- 172.21.15.97:0/3326338071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e00d15a0 0x7fc4e00cf990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009840 tx=0x7fc4d0009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.075+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3326338071 shutdown_connections 2023-07-06T03:47:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.075+0000 7fc4e77ff700 1 --2- 172.21.15.97:0/3326338071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4e00cfed0 0x7fc4e00b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.075+0000 7fc4e77ff700 1 --2- 172.21.15.97:0/3326338071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e00d15a0 0x7fc4e00cf990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.075+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3326338071 >> 172.21.15.97:0/3326338071 conn(0x7fc4e006a900 msgr2=0x7fc4e006acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:48.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.075+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3326338071 shutdown_connections 2023-07-06T03:47:48.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.075+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3326338071 wait complete. 2023-07-06T03:47:48.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.076+0000 7fc4e77ff700 1 Processor -- start 2023-07-06T03:47:48.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.077+0000 7fc4e77ff700 1 -- start start 2023-07-06T03:47:48.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e77ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e00b4a40 0x7fc4e014d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e77ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4e00cfed0 0x7fc4e00c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e77ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e00d15a0 0x7fc4e015ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e77ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e015e840 con 0x7fc4e00d15a0 2023-07-06T03:47:48.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e77ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e014d9b0 con 0x7fc4e00b4a40 2023-07-06T03:47:48.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e77ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e015dbc0 con 0x7fc4e00cfed0 2023-07-06T03:47:48.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e5d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e00d15a0 0x7fc4e015ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e5d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e00d15a0 0x7fc4e015ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33288/0 (socket says 172.21.15.97:33288) 2023-07-06T03:47:48.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e5d9c700 1 -- 172.21.15.97:0/3293124256 learned_addr learned my addr 172.21.15.97:0/3293124256 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:48.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e559b700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e00b4a40 0x7fc4e014d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4e00cfed0 0x7fc4e00c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e5d9c700 1 -- 172.21.15.97:0/3293124256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4e00cfed0 msgr2=0x7fc4e00c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e5d9c700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4e00cfed0 0x7fc4e00c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e5d9c700 1 -- 172.21.15.97:0/3293124256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e00b4a40 msgr2=0x7fc4e014d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e5d9c700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e00b4a40 0x7fc4e014d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.078+0000 7fc4e5d9c700 1 -- 172.21.15.97:0/3293124256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d00094a0 con 0x7fc4e00d15a0 2023-07-06T03:47:48.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.079+0000 7fc4e5d9c700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e00d15a0 0x7fc4e015ec60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8007e60 tx=0x7fc4d800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:48.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.079+0000 7fc4d5ffb700 1 -- 172.21.15.97:0/3293124256 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800ed30 con 0x7fc4e00d15a0 2023-07-06T03:47:48.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.079+0000 7fc4d5ffb700 1 -- 172.21.15.97:0/3293124256 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d8014420 con 0x7fc4e00d15a0 2023-07-06T03:47:48.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.079+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3293124256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e015dea0 con 0x7fc4e00d15a0 2023-07-06T03:47:48.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.079+0000 7fc4d5ffb700 1 -- 172.21.15.97:0/3293124256 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80147d0 con 0x7fc4e00d15a0 2023-07-06T03:47:48.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.079+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3293124256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e015e460 con 0x7fc4e00d15a0 2023-07-06T03:47:48.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.081+0000 7fc4d5ffb700 1 -- 172.21.15.97:0/3293124256 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc4d8014930 con 0x7fc4e00d15a0 2023-07-06T03:47:48.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.082+0000 7fc4d5ffb700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4c406ace0 0x7fc4c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.082+0000 7fc4d5ffb700 1 -- 172.21.15.97:0/3293124256 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4d8089b10 con 0x7fc4e00d15a0 2023-07-06T03:47:48.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.082+0000 7fc4e559b700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4c406ace0 0x7fc4c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.083+0000 7fc4e559b700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4c406ace0 0x7fc4c406d190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc007fd0 tx=0x7fc4dc007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:48.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.084+0000 7fc4fb1adb80 1 Processor -- start 2023-07-06T03:47:48.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.084+0000 7fc4fb1adb80 1 -- start start 2023-07-06T03:47:48.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.085+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c193120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.085+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c1b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.085+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c16db50 0x55805c170110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.085+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805c0e4ea0 con 0x55805c192d50 2023-07-06T03:47:48.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.085+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805c058600 con 0x55805c198190 2023-07-06T03:47:48.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.085+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55805c0c7910 con 0x55805c16db50 2023-07-06T03:47:48.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.085+0000 7fc4e559b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c193120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e4d9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c1b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e5d9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c16db50 0x55805c170110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e4d9a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c1b6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:57342/0 (socket says 172.21.15.97:57342) 2023-07-06T03:47:48.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/2225610421 learned_addr learned my addr 172.21.15.97:0/2225610421 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:48.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e559b700 1 -- 172.21.15.97:0/2225610421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c16db50 msgr2=0x55805c170110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:48.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e559b700 1 --2- 172.21.15.97:0/2225610421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c16db50 0x55805c170110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e559b700 1 -- 172.21.15.97:0/2225610421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 msgr2=0x55805c1b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:48.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e559b700 1 --2- 172.21.15.97:0/2225610421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c1b6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e559b700 1 -- 172.21.15.97:0/2225610421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55805beb4790 con 0x55805c192d50 2023-07-06T03:47:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e5d9c700 1 --2- 172.21.15.97:0/2225610421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c16db50 0x55805c170110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:47:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/2225610421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c1b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:47:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4e559b700 1 --2- 172.21.15.97:0/2225610421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c193120 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc005b20 tx=0x7fc4dc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324bbdf14c183333 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.086+0000 7fc4caffd700 1 -- 172.21.15.97:0/2225610421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc006240 con 0x55805c192d50 2023-07-06T03:47:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.087+0000 7fc4caffd700 1 -- 172.21.15.97:0/2225610421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc0063a0 con 0x55805c192d50 2023-07-06T03:47:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.087+0000 7fc4caffd700 1 -- 172.21.15.97:0/2225610421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc006710 con 0x55805c192d50 2023-07-06T03:47:48.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.087+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2225610421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 msgr2=0x55805c193120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.087+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/2225610421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c193120 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc005b20 tx=0x7fc4dc012000 comp rx=0 tx=0).stop 2023-07-06T03:47:48.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.087+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2225610421 shutdown_connections 2023-07-06T03:47:48.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.087+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/2225610421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c16db50 0x55805c170110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.087+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/2225610421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c193120 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.087+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/2225610421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c1b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.087+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2225610421 >> 172.21.15.97:0/2225610421 conn(0x55805c165310 msgr2=0x55805c1b55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:48.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.087+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2225610421 shutdown_connections 2023-07-06T03:47:48.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.087+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2225610421 wait complete. 2023-07-06T03:47:48.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.088+0000 7fc4fb1adb80 1 -- start start 2023-07-06T03:47:48.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.088+0000 7fc4fb1adb80 1 Processor -- start 2023-07-06T03:47:48.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.089+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c16db50 0x55805c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.089+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c199310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.089+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c198190 0x55805c199850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.089+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805c0c0be0 con 0x55805c192d50 2023-07-06T03:47:48.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.089+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805c07e5b0 con 0x55805c16db50 2023-07-06T03:47:48.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.089+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55805c064910 con 0x55805c198190 2023-07-06T03:47:48.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.089+0000 7fc4fb1adb80 10 client.0 _ll_register_callbacks cb 0x55805c07cb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:47:48.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4e4d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4e4d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33300/0 (socket says 172.21.15.97:33300) 2023-07-06T03:47:48.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/2090602345 learned_addr learned my addr 172.21.15.97:0/2090602345 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:48.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4e559b700 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c16db50 0x55805c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4e5d9c700 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c198190 0x55805c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c198190 msgr2=0x55805c199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:48.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c198190 0x55805c199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c16db50 msgr2=0x55805c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:48.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c16db50 0x55805c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/2090602345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e00cf210 con 0x55805c192d50 2023-07-06T03:47:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c199310 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009ea0 tx=0x7fc4d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2ef07ce8a2d2f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.090+0000 7fc4bffff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:47:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4bffff700 1 -- 172.21.15.97:0/2090602345 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0020040 con 0x55805c192d50 2023-07-06T03:47:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2090602345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55805c162920 con 0x55805c192d50 2023-07-06T03:47:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4bffff700 1 -- 172.21.15.97:0/2090602345 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4d0004450 con 0x55805c192d50 2023-07-06T03:47:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4bffff700 1 -- 172.21.15.97:0/2090602345 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4d0011830 con 0x55805c192d50 2023-07-06T03:47:48.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4fb1adb80 10 client.6000 fetch_fsmap learned FSMap version 25 2023-07-06T03:47:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2090602345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x7fc4cc0035b0 con 0x55805c192d50 2023-07-06T03:47:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4bffff700 1 -- 172.21.15.97:0/2090602345 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55805c162920 con 0x55805c192d50 2023-07-06T03:47:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4bffff700 1 -- 172.21.15.97:0/2090602345 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc4d00036a0 con 0x55805c192d50 2023-07-06T03:47:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4fb1adb80 10 client.6000 fetch_fsmap finished waiting for FSMap version 25 2023-07-06T03:47:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4fb1adb80 10 client.6000 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T03:47:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4fb1adb80 20 client.6000 populate_metadata read hostname 'smithi097' 2023-07-06T03:47:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:6834/3623506860,v1:172.21.15.146:6835/3623506860] conn(0x55805c1889f0 0x55805c188dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4fb1adb80 4 client.6000 mds_command: new command op to 5838 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T03:47:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.091+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2090602345 --> [v2:172.21.15.146:6834/3623506860,v1:172.21.15.146:6835/3623506860] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55805c13f0d0 con 0x55805c1889f0 2023-07-06T03:47:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.092+0000 7fc4e559b700 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:6834/3623506860,v1:172.21.15.146:6835/3623506860] conn(0x55805c1889f0 0x55805c188dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.092+0000 7fc4e559b700 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:6834/3623506860,v1:172.21.15.146:6835/3623506860] conn(0x55805c1889f0 0x55805c188dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e56024a082db4c server_cookie=61acbcd42090bdb7 in_seq=0 out_seq=0 2023-07-06T03:47:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.092+0000 7fc4bffff700 10 client.6000 ms_handle_connect on v2:172.21.15.146:6834/3623506860 2023-07-06T03:47:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4bffff700 1 -- 172.21.15.97:0/2090602345 <== mds.0 v2:172.21.15.146:6834/3623506860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55805c13f0d0 con 0x55805c1889f0 2023-07-06T03:47:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4bffff700 10 client.6000 handle_command_reply: tid=0 2023-07-06T03:47:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4fb1adb80 1 client.6000 shutdown 2023-07-06T03:47:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2090602345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 msgr2=0x55805c199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c199310 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009ea0 tx=0x7fc4d00047c0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2090602345 shutdown_connections 2023-07-06T03:47:48.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:6834/3623506860,v1:172.21.15.146:6835/3623506860] conn(0x55805c1889f0 0x55805c188dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c198190 0x55805c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c16db50 0x55805c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4bffff700 0 client.6000 ms_handle_reset on v2:172.21.15.146:6834/3623506860 2023-07-06T03:47:48.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/2090602345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c199310 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.093+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2090602345 >> 172.21.15.97:0/2090602345 conn(0x55805c165310 msgr2=0x55805c170080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:48.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.094+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2090602345 shutdown_connections 2023-07-06T03:47:48.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.094+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/2090602345 wait complete. 2023-07-06T03:47:48.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.094+0000 7fc4fb1adb80 20 client.6000 trim_cache size 0 max 16384 2023-07-06T03:47:48.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.112+0000 7fc4fb1adb80 1 Processor -- start 2023-07-06T03:47:48.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.112+0000 7fc4fb1adb80 1 -- start start 2023-07-06T03:47:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c16db50 0x55805c16ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c192d50 0x55805c199810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c1b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805beb5390 con 0x55805c16db50 2023-07-06T03:47:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805c18acf0 con 0x55805c198190 2023-07-06T03:47:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55805c16bfb0 con 0x55805c192d50 2023-07-06T03:47:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4e559b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c16db50 0x55805c16ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4e559b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c16db50 0x55805c16ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33304/0 (socket says 172.21.15.97:33304) 2023-07-06T03:47:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4e559b700 1 -- 172.21.15.97:0/447967343 learned_addr learned my addr 172.21.15.97:0/447967343 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4e5d9c700 1 --2- 172.21.15.97:0/447967343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c1b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/447967343 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c192d50 0x55805c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4e559b700 1 -- 172.21.15.97:0/447967343 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c192d50 msgr2=0x55805c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4e559b700 1 --2- 172.21.15.97:0/447967343 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c192d50 0x55805c199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4e559b700 1 -- 172.21.15.97:0/447967343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 msgr2=0x55805c1b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:48.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4e559b700 1 --2- 172.21.15.97:0/447967343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c1b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4e559b700 1 -- 172.21.15.97:0/447967343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55805bfa10a0 con 0x55805c16db50 2023-07-06T03:47:48.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.113+0000 7fc4e559b700 1 --2- 172.21.15.97:0/447967343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c16db50 0x55805c16ba20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc005b20 tx=0x7fc4dc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06747cc379bdd49 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:48.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4cb7fe700 1 -- 172.21.15.97:0/447967343 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc006240 con 0x55805c16db50 2023-07-06T03:47:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4cb7fe700 1 -- 172.21.15.97:0/447967343 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc006730 con 0x55805c16db50 2023-07-06T03:47:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4cb7fe700 1 -- 172.21.15.97:0/447967343 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0344c0 con 0x55805c16db50 2023-07-06T03:47:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/447967343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c16db50 msgr2=0x55805c16ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/447967343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c16db50 0x55805c16ba20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc005b20 tx=0x7fc4dc01e000 comp rx=0 tx=0).stop 2023-07-06T03:47:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/447967343 shutdown_connections 2023-07-06T03:47:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/447967343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c1b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/447967343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c16db50 0x55805c16ba20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/447967343 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c192d50 0x55805c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/447967343 >> 172.21.15.97:0/447967343 conn(0x55805c165310 msgr2=0x55805c16a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/447967343 shutdown_connections 2023-07-06T03:47:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.114+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/447967343 wait complete. 2023-07-06T03:47:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.115+0000 7fc4fb1adb80 1 -- start start 2023-07-06T03:47:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.115+0000 7fc4fb1adb80 1 Processor -- start 2023-07-06T03:47:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c16db50 0x55805c21fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c21c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4fb1adb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c16adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805c189900 con 0x55805c192d50 2023-07-06T03:47:48.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55805c2204a0 con 0x55805c198190 2023-07-06T03:47:48.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4fb1adb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55805c1972e0 con 0x55805c16db50 2023-07-06T03:47:48.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4fb1adb80 10 client.0 _ll_register_callbacks cb 0x7fc4d00116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:47:48.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4e4d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c21c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4e4d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c21c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33306/0 (socket says 172.21.15.97:33306) 2023-07-06T03:47:48.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/3736449893 learned_addr learned my addr 172.21.15.97:0/3736449893 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:48.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4e5d9c700 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c16adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.116+0000 7fc4e559b700 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c16db50 0x55805c21fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.117+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c16db50 msgr2=0x55805c21fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:48.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.117+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c16db50 0x55805c21fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.117+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 msgr2=0x55805c16adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:48.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.117+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c16adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.117+0000 7fc4e4d9a700 1 -- 172.21.15.97:0/3736449893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55805beb4790 con 0x55805c192d50 2023-07-06T03:47:48.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.117+0000 7fc4e4d9a700 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c21c190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009840 tx=0x7fc4d0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc88b575b49d3d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:48.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.117+0000 7fc4bffff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:47:48.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.117+0000 7fc4bffff700 1 -- 172.21.15.97:0/3736449893 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0020040 con 0x55805c192d50 2023-07-06T03:47:48.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.117+0000 7fc4bffff700 1 -- 172.21.15.97:0/3736449893 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4d0000e40 con 0x55805c192d50 2023-07-06T03:47:48.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.117+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/3736449893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4dc0063a0 con 0x55805c192d50 2023-07-06T03:47:48.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4bffff700 1 -- 172.21.15.97:0/3736449893 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4d0003e90 con 0x55805c192d50 2023-07-06T03:47:48.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4fb1adb80 10 client.6006 fetch_fsmap learned FSMap version 25 2023-07-06T03:47:48.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/3736449893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55805c086b10 con 0x55805c192d50 2023-07-06T03:47:48.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4bffff700 1 -- 172.21.15.97:0/3736449893 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d0045020 con 0x55805c192d50 2023-07-06T03:47:48.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4bffff700 1 -- 172.21.15.97:0/3736449893 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc4d0004460 con 0x55805c192d50 2023-07-06T03:47:48.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4fb1adb80 10 client.6006 fetch_fsmap finished waiting for FSMap version 25 2023-07-06T03:47:48.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4fb1adb80 10 client.6006 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T03:47:48.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4fb1adb80 20 client.6006 populate_metadata read hostname 'smithi097' 2023-07-06T03:47:48.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:6834/3623506860,v1:172.21.15.146:6835/3623506860] conn(0x55805c16f8f0 0x55805c16fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4fb1adb80 4 client.6006 mds_command: new command op to 5838 tid=0[{"prefix": "status"}] 2023-07-06T03:47:48.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/3736449893 --> [v2:172.21.15.146:6834/3623506860,v1:172.21.15.146:6835/3623506860] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55805c0b9d20 con 0x55805c16f8f0 2023-07-06T03:47:48.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.118+0000 7fc4e559b700 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:6834/3623506860,v1:172.21.15.146:6835/3623506860] conn(0x55805c16f8f0 0x55805c16fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4bffff700 10 client.6006 ms_handle_connect on v2:172.21.15.146:6834/3623506860 2023-07-06T03:47:48.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4e559b700 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:6834/3623506860,v1:172.21.15.146:6835/3623506860] conn(0x55805c16f8f0 0x55805c16fcc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8de41ec3bfb14c19 server_cookie=d16f66490b8e536b in_seq=0 out_seq=0 2023-07-06T03:47:48.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4bffff700 1 -- 172.21.15.97:0/3736449893 <== mds.0 v2:172.21.15.146:6834/3623506860 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55805c0b9d20 con 0x55805c16f8f0 2023-07-06T03:47:48.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4bffff700 10 client.6006 handle_command_reply: tid=0 2023-07-06T03:47:48.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4fb1adb80 1 client.6006 shutdown 2023-07-06T03:47:48.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/3736449893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 msgr2=0x55805c21c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c21c190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009840 tx=0x7fc4d0011000 comp rx=0 tx=0).stop 2023-07-06T03:47:48.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/3736449893 shutdown_connections 2023-07-06T03:47:48.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:6834/3623506860,v1:172.21.15.146:6835/3623506860] conn(0x55805c16f8f0 0x55805c16fcc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55805c198190 0x55805c16adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55805c16db50 0x55805c21fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4bffff700 0 client.6006 ms_handle_reset on v2:172.21.15.146:6834/3623506860 2023-07-06T03:47:48.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4fb1adb80 1 --2- 172.21.15.97:0/3736449893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55805c192d50 0x55805c21c190 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/3736449893 >> 172.21.15.97:0/3736449893 conn(0x55805c165310 msgr2=0x55805c192060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:48.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/3736449893 shutdown_connections 2023-07-06T03:47:48.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.119+0000 7fc4fb1adb80 1 -- 172.21.15.97:0/3736449893 wait complete. 2023-07-06T03:47:48.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4fb1adb80 20 client.6006 trim_cache size 0 max 16384 2023-07-06T03:47:48.129 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:47:48.129 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36", 2023-07-06T03:47:48.130 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-07-06T03:47:48.130 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5838, 2023-07-06T03:47:48.130 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-07-06T03:47:48.130 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-07-06T03:47:48.130 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-07-06T03:47:48.130 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 11.458920728000001, 2023-07-06T03:47:48.130 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 25, 2023-07-06T03:47:48.130 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 47, 2023-07-06T03:47:48.131 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 47, 2023-07-06T03:47:48.131 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 17.544126754000001 2023-07-06T03:47:48.131 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:47:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3293124256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4c406ace0 msgr2=0x7fc4c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4c406ace0 0x7fc4c406d190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc007fd0 tx=0x7fc4dc007dc0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3293124256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e00d15a0 msgr2=0x7fc4e015ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e00d15a0 0x7fc4e015ec60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8007e60 tx=0x7fc4d800e3c0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3293124256 shutdown_connections 2023-07-06T03:47:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc4c406ace0 0x7fc4c406d190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4e00d15a0 0x7fc4e015ec60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e00b4a40 0x7fc4e014d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 --2- 172.21.15.97:0/3293124256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc4e00cfed0 0x7fc4e00c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3293124256 >> 172.21.15.97:0/3293124256 conn(0x7fc4e006a900 msgr2=0x7fc4e00b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:48.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3293124256 shutdown_connections 2023-07-06T03:47:48.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.120+0000 7fc4e77ff700 1 -- 172.21.15.97:0/3293124256 wait complete. 2023-07-06T03:47:48.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:47:48.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.331+0000 7f93b41ff700 1 Processor -- start 2023-07-06T03:47:48.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.332+0000 7f93b41ff700 1 -- start start 2023-07-06T03:47:48.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b41ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93ac0cd390 0x7f93ac160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b41ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac160dd0 0x7f93ac0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b41ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ac0cdae0 con 0x7f93ac15c520 2023-07-06T03:47:48.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b41ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ac0b6640 con 0x7f93ac0cd390 2023-07-06T03:47:48.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b41ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac0b6920 con 0x7f93ac160dd0 2023-07-06T03:47:48.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b179a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93ac0cd390 0x7f93ac160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b1f9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f93ac15c520 0x7f93ac160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46616/0 (socket says 172.21.15.97:46616) 2023-07-06T03:47:48.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b1f9b700 1 -- 172.21.15.97:0/1719729451 learned_addr learned my addr 172.21.15.97:0/1719729451 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:48.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b279c700 1 --2- 172.21.15.97:0/1719729451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac160dd0 0x7f93ac0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b179a700 1 -- 172.21.15.97:0/1719729451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac160dd0 msgr2=0x7f93ac0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:48.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b179a700 1 --2- 172.21.15.97:0/1719729451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac160dd0 0x7f93ac0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b179a700 1 -- 172.21.15.97:0/1719729451 >> v1:172.21.15.97:6789/0 conn(0x7f93ac15c520 legacy=0x7f93ac160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.334+0000 7f93b179a700 1 -- 172.21.15.97:0/1719729451 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac0b6c00 con 0x7f93ac0cd390 2023-07-06T03:47:48.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.333+0000 7f93b0f99700 1 -- 172.21.15.97:0/1719729451 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205552702 0 0) 0x7f93ac0cdae0 con 0x7f93ac15c520 2023-07-06T03:47:48.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.334+0000 7f93b179a700 1 --2- 172.21.15.97:0/1719729451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0cd390 0x7f93ac160890 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f939c009840 tx=0x7f939c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c03472786990dd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:48.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.334+0000 7f93b0f99700 1 -- 172.21.15.97:0/1719729451 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c011830 con 0x7f93ac0cd390 2023-07-06T03:47:48.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.334+0000 7f93b0f99700 1 -- 172.21.15.97:0/1719729451 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c011990 con 0x7f93ac0cd390 2023-07-06T03:47:48.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.334+0000 7f93b0f99700 1 -- 172.21.15.97:0/1719729451 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c011d20 con 0x7f93ac0cd390 2023-07-06T03:47:48.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.334+0000 7f93b41ff700 1 -- 172.21.15.97:0/1719729451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0cd390 msgr2=0x7f93ac160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.334+0000 7f93b41ff700 1 --2- 172.21.15.97:0/1719729451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0cd390 0x7f93ac160890 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f939c009840 tx=0x7f939c009b50 comp rx=0 tx=0).stop 2023-07-06T03:47:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.335+0000 7f93b41ff700 1 -- 172.21.15.97:0/1719729451 shutdown_connections 2023-07-06T03:47:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.335+0000 7f93b41ff700 1 --2- 172.21.15.97:0/1719729451 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac160dd0 0x7f93ac0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.335+0000 7f93b41ff700 1 --2- 172.21.15.97:0/1719729451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0cd390 0x7f93ac160890 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.335+0000 7f93b41ff700 1 -- 172.21.15.97:0/1719729451 >> 172.21.15.97:0/1719729451 conn(0x7f93ac071790 msgr2=0x7f93ac071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.335+0000 7f93b41ff700 1 -- 172.21.15.97:0/1719729451 shutdown_connections 2023-07-06T03:47:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.335+0000 7f93b41ff700 1 -- 172.21.15.97:0/1719729451 wait complete. 2023-07-06T03:47:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.336+0000 7f93b41ff700 1 Processor -- start 2023-07-06T03:47:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.337+0000 7f93b41ff700 1 -- start start 2023-07-06T03:47:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.337+0000 7f93b41ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0cd390 0x7f93ac0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b41ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac15c520 0x7f93ac0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b41ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93ac160dd0 0x7f93ac14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b41ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac15df70 con 0x7f93ac0cd390 2023-07-06T03:47:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b41ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac15d360 con 0x7f93ac160dd0 2023-07-06T03:47:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b41ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac15d640 con 0x7f93ac15c520 2023-07-06T03:47:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b1f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0cd390 0x7f93ac0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b1f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0cd390 0x7f93ac0d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33316/0 (socket says 172.21.15.97:33316) 2023-07-06T03:47:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b1f9b700 1 -- 172.21.15.97:0/858847350 learned_addr learned my addr 172.21.15.97:0/858847350 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b279c700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93ac160dd0 0x7f93ac14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b179a700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac15c520 0x7f93ac0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b1f9b700 1 -- 172.21.15.97:0/858847350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac15c520 msgr2=0x7f93ac0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b1f9b700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac15c520 0x7f93ac0d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b1f9b700 1 -- 172.21.15.97:0/858847350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93ac160dd0 msgr2=0x7f93ac14cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b1f9b700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93ac160dd0 0x7f93ac14cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b1f9b700 1 -- 172.21.15.97:0/858847350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c0094a0 con 0x7f93ac0cd390 2023-07-06T03:47:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.338+0000 7f93b1f9b700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0cd390 0x7f93ac0d2c70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f93a800c550 tx=0x7f93a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:48.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.339+0000 7f93a27fc700 1 -- 172.21.15.97:0/858847350 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8004c80 con 0x7f93ac0cd390 2023-07-06T03:47:48.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.339+0000 7f93a27fc700 1 -- 172.21.15.97:0/858847350 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a8004e00 con 0x7f93ac0cd390 2023-07-06T03:47:48.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.339+0000 7f93a27fc700 1 -- 172.21.15.97:0/858847350 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8007310 con 0x7f93ac0cd390 2023-07-06T03:47:48.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.339+0000 7f93b41ff700 1 -- 172.21.15.97:0/858847350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ac15d920 con 0x7f93ac0cd390 2023-07-06T03:47:48.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.339+0000 7f93b41ff700 1 -- 172.21.15.97:0/858847350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac0d31b0 con 0x7f93ac0cd390 2023-07-06T03:47:48.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.340+0000 7f93b41ff700 1 -- 172.21.15.97:0/858847350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ac04e340 con 0x7f93ac0cd390 2023-07-06T03:47:48.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.341+0000 7f93a27fc700 1 -- 172.21.15.97:0/858847350 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f93a8007470 con 0x7f93ac0cd390 2023-07-06T03:47:48.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.342+0000 7f93a27fc700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939006ace0 0x7f939006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.342+0000 7f93a27fc700 1 -- 172.21.15.97:0/858847350 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f93a8089200 con 0x7f93ac0cd390 2023-07-06T03:47:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.342+0000 7f93b179a700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939006ace0 0x7f939006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.343+0000 7f93b179a700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939006ace0 0x7f939006d190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f939c0090b0 tx=0x7f939c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.346+0000 7f93a27fc700 1 -- 172.21.15.97:0/858847350 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f93a80542a0 con 0x7f93ac0cd390 2023-07-06T03:47:48.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.585+0000 7f93b41ff700 1 -- 172.21.15.97:0/858847350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f93ac0b4f40 con 0x7f93ac0cd390 2023-07-06T03:47:48.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.586+0000 7f93a27fc700 1 -- 172.21.15.97:0/858847350 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f93a80578c0 con 0x7f93ac0cd390 2023-07-06T03:47:48.588 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:47:48.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 -- 172.21.15.97:0/858847350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939006ace0 msgr2=0x7f939006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939006ace0 0x7f939006d190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f939c0090b0 tx=0x7f939c01c040 comp rx=0 tx=0).stop 2023-07-06T03:47:48.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 -- 172.21.15.97:0/858847350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0cd390 msgr2=0x7f93ac0d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:48.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0cd390 0x7f93ac0d2c70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f93a800c550 tx=0x7f93a800c910 comp rx=0 tx=0).stop 2023-07-06T03:47:48.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 -- 172.21.15.97:0/858847350 shutdown_connections 2023-07-06T03:47:48.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939006ace0 0x7f939006d190 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93ac160dd0 0x7f93ac14cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0cd390 0x7f93ac0d2c70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 --2- 172.21.15.97:0/858847350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93ac15c520 0x7f93ac0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:48.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 -- 172.21.15.97:0/858847350 >> 172.21.15.97:0/858847350 conn(0x7f93ac071790 msgr2=0x7f93ac0cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:48.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 -- 172.21.15.97:0/858847350 shutdown_connections 2023-07-06T03:47:48.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:48.588+0000 7f93b41ff700 1 -- 172.21.15.97:0/858847350 wait complete. 2023-07-06T03:47:48.608 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:47:48.608 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:47:48.609 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:47:48.609 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:47:48.609 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:47:48.609 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:47:48.610 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:47:48.610 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:47:48.610 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:47:48.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:48.611 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:47:48.658 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:47:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:47:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:47:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:47:48.659 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:48.660 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:47:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:47:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:47:48.660 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:47:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:47:48.660 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:47:48.660 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:47:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:47:48.661 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:47:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:47:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:47:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42000sec preferred_lft 42000sec 2023-07-06T03:47:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:47:48.661 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:48.661 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:47:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:47:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:47:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:47:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:48.662 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:47:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:47:48.662 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:47:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:48.663 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:47:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:47:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:47:48.663 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:48.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:48.664 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:47:48.731 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:47:48.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:48.734 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:47:48.783 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:47:48.783 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:47:48.783 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:47:48.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:47:48.784 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:47:48.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:48.784 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:47:48.785 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:47:48.785 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:47:48.785 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:47:48.786 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:47:48.786 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:47:48.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:48.953 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:47:48.999 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:47:48.999 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:47:49.000 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:47:49.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:49.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:47:49.070 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:47:49.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:47:49.071 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:47:49.116 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:47:49.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:47:49.117 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:47:49.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:47:49.311 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:47:49.311 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:47:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:47:49.380 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:47:49.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:49.381 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:47:49.426 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:47:49.427 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:47:49.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:47:49.540 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:47:49.540 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:47:49.608 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:47:49.610 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:47:49.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:49.611 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:47:49.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:47:49.638+0000 7f3e79c685c0 -1 init, newargv = 0x555ea1381340 newargc=15 2023-07-06T03:47:49.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[134043]: starting ceph client 2023-07-06T03:47:49.647 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[134043]: starting fuse 2023-07-06T03:47:49.663 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:47:49.664 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:47:49.664 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:47:49.665 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:47:49.665 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:47:49.665 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:47:49.666 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:47:49.666 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:49.666 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:47:49.667 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:47:49.667 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:47:49.667 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:47:49.668 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:49.668 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:47:49.668 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:47:49.669 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:47:49.669 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:47:49.669 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:49.669 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:47:49.670 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:47:49.670 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:47:49.671 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:47:49.671 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:47:49.671 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:47:49.671 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:47:49.672 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:47:49.672 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:49.673 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:47:49.673 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:47:49.845 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.134043.asok 2023-07-06T03:47:50.023 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134043.asok 2023-07-06T03:47:50.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:50.024 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134043.asok status 2023-07-06T03:47:50.245 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:47:50.245 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:47:50.245 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:47:50.245 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:47:50.246 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:47:50.246 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:47:50.246 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:47:50.246 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "134043", 2023-07-06T03:47:50.246 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:47:50.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:50.247 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:47:50.247 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:47:50.247 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6021, 2023-07-06T03:47:50.247 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:47:50.247 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:47:50.247 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:47:50.247 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6021 2023-07-06T03:47:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:47:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:47:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:47:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 476559399 2023-07-06T03:47:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:47:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:47:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:47:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 476559399 2023-07-06T03:47:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6021 192.168.0.1:0/476559399", 2023-07-06T03:47:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/476559399", 2023-07-06T03:47:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:47:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 25, 2023-07-06T03:47:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 47, 2023-07-06T03:47:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:47:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:47:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:47:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:47:50.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:50.260 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:47:50.261 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:47:50.307 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:47:50.307 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:47:50.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:50.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:47:50.378 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:47:50.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.570+0000 7fde381ff700 1 Processor -- start 2023-07-06T03:47:50.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.570+0000 7fde381ff700 1 -- start start 2023-07-06T03:47:50.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.571+0000 7fde381ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde300b8410 0x7fde30167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:50.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.571+0000 7fde381ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde300cdae0 con 0x7fde300cd390 2023-07-06T03:47:50.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.571+0000 7fde381ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde300b6640 con 0x7fde3015c520 2023-07-06T03:47:50.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.571+0000 7fde381ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde300b6920 con 0x7fde300b8410 2023-07-06T03:47:50.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.571+0000 7fde3579a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fde300cd390 0x7fde30160890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56562/0 (socket says 172.21.15.97:56562) 2023-07-06T03:47:50.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.571+0000 7fde3579a700 1 -- 172.21.15.97:0/1210693281 learned_addr learned my addr 172.21.15.97:0/1210693281 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:47:50.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.571+0000 7fde3679c700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde300b8410 0x7fde30167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:50.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.571+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628945882 0 0) 0x7fde300cdae0 con 0x7fde300cd390 2023-07-06T03:47:50.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.571+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde1c0035b0 con 0x7fde300cd390 2023-07-06T03:47:50.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678293024 0 0) 0x7fde300b6640 con 0x7fde3015c520 2023-07-06T03:47:50.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde300cdae0 con 0x7fde3015c520 2023-07-06T03:47:50.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3157173244 0 0) 0x7fde1c0035b0 con 0x7fde300cd390 2023-07-06T03:47:50.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde300b8410 msgr2=0x7fde30167510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:50.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde300b8410 0x7fde30167510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:50.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 >> v1:172.21.15.146:6789/0 conn(0x7fde3015c520 legacy=0x7fde30160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:50.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde300b6c00 con 0x7fde300cd390 2023-07-06T03:47:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fde200015f0 con 0x7fde300cd390 2023-07-06T03:47:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 >> v1:172.21.15.97:6789/0 conn(0x7fde300cd390 legacy=0x7fde30160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde1c003bc0 0x7fde1c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde1c004850 0x7fde1c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde300b8410 0x7fde1c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300b6920 con 0x7fde1c003bc0 2023-07-06T03:47:50.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c00baf0 con 0x7fde1c004850 2023-07-06T03:47:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c00bcb0 con 0x7fde300b8410 2023-07-06T03:47:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde35f9b700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde1c003bc0 0x7fde1c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde3579a700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde1c004850 0x7fde1c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.572+0000 7fde3679c700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde300b8410 0x7fde1c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde35f9b700 1 -- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde300b8410 msgr2=0x7fde1c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:50.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde35f9b700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde300b8410 0x7fde1c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:50.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde35f9b700 1 -- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde1c004850 msgr2=0x7fde1c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:47:50.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde35f9b700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde1c004850 0x7fde1c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:50.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde35f9b700 1 -- 172.21.15.97:0/1210693281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0035b0 con 0x7fde1c003bc0 2023-07-06T03:47:50.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde3679c700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde300b8410 0x7fde1c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:47:50.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde35f9b700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde1c003bc0 0x7fde1c003fb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fde2c007fd0 tx=0x7fde1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd4ffad5e6979f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:50.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c001d80 con 0x7fde1c003bc0 2023-07-06T03:47:50.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c0052d0 con 0x7fde1c003bc0 2023-07-06T03:47:50.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde34f99700 1 -- 172.21.15.97:0/1210693281 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c001470 con 0x7fde1c003bc0 2023-07-06T03:47:50.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde381ff700 1 -- 172.21.15.97:0/1210693281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde1c003bc0 msgr2=0x7fde1c003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:50.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.573+0000 7fde381ff700 1 --2- 172.21.15.97:0/1210693281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde1c003bc0 0x7fde1c003fb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fde2c007fd0 tx=0x7fde1c005880 comp rx=0 tx=0).stop 2023-07-06T03:47:50.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.574+0000 7fde3679c700 1 -- 172.21.15.97:0/1210693281 reap_dead start 2023-07-06T03:47:50.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.574+0000 7fde381ff700 1 -- 172.21.15.97:0/1210693281 shutdown_connections 2023-07-06T03:47:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.574+0000 7fde381ff700 1 -- 172.21.15.97:0/1210693281 >> 172.21.15.97:0/1210693281 conn(0x7fde30071790 msgr2=0x7fde30071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.574+0000 7fde381ff700 1 -- 172.21.15.97:0/1210693281 shutdown_connections 2023-07-06T03:47:50.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.574+0000 7fde381ff700 1 -- 172.21.15.97:0/1210693281 wait complete. 2023-07-06T03:47:50.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.575+0000 7fde381ff700 1 Processor -- start 2023-07-06T03:47:50.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.576+0000 7fde381ff700 1 -- start start 2023-07-06T03:47:50.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.576+0000 7fde381ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde1c003bc0 0x7fde3014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:50.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.576+0000 7fde381ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3014cb00 0x7fde300d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.576+0000 7fde381ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde300d1180 0x7fde300d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde381ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300c6870 con 0x7fde3014cb00 2023-07-06T03:47:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde381ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300d1d70 con 0x7fde300d1180 2023-07-06T03:47:50.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde381ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde300d1b30 con 0x7fde1c003bc0 2023-07-06T03:47:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde3579a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3014cb00 0x7fde300d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde3579a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3014cb00 0x7fde300d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53110/0 (socket says 172.21.15.97:53110) 2023-07-06T03:47:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde3579a700 1 -- 172.21.15.97:0/720023508 learned_addr learned my addr 172.21.15.97:0/720023508 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:47:50.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde3679c700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde300d1180 0x7fde300d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:50.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde35f9b700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde1c003bc0 0x7fde3014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:50.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde3579a700 1 -- 172.21.15.97:0/720023508 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde1c003bc0 msgr2=0x7fde3014d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:50.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde3579a700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde1c003bc0 0x7fde3014d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:50.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde3579a700 1 -- 172.21.15.97:0/720023508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde300d1180 msgr2=0x7fde300d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:50.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde3579a700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde300d1180 0x7fde300d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:50.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde3579a700 1 -- 172.21.15.97:0/720023508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3015d940 con 0x7fde3014cb00 2023-07-06T03:47:50.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.577+0000 7fde3579a700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3014cb00 0x7fde300d0c40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fde20001700 tx=0x7fde20007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:50.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.578+0000 7fde267fc700 1 -- 172.21.15.97:0/720023508 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde200114c0 con 0x7fde3014cb00 2023-07-06T03:47:50.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.578+0000 7fde267fc700 1 -- 172.21.15.97:0/720023508 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde20011620 con 0x7fde3014cb00 2023-07-06T03:47:50.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.578+0000 7fde267fc700 1 -- 172.21.15.97:0/720023508 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20011990 con 0x7fde3014cb00 2023-07-06T03:47:50.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.578+0000 7fde381ff700 1 -- 172.21.15.97:0/720023508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde300ccc90 con 0x7fde3014cb00 2023-07-06T03:47:50.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.578+0000 7fde381ff700 1 -- 172.21.15.97:0/720023508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde300cc160 con 0x7fde3014cb00 2023-07-06T03:47:50.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.579+0000 7fde381ff700 1 -- 172.21.15.97:0/720023508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde300ce010 con 0x7fde3014cb00 2023-07-06T03:47:50.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.580+0000 7fde267fc700 1 -- 172.21.15.97:0/720023508 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde20011af0 con 0x7fde3014cb00 2023-07-06T03:47:50.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.581+0000 7fde267fc700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde1406ace0 0x7fde1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:47:50.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.581+0000 7fde267fc700 1 -- 172.21.15.97:0/720023508 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fde20088920 con 0x7fde3014cb00 2023-07-06T03:47:50.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.581+0000 7fde35f9b700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde1406ace0 0x7fde1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:47:50.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.582+0000 7fde35f9b700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde1406ace0 0x7fde1406d190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fde2c015040 tx=0x7fde2c00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:47:50.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.586+0000 7fde267fc700 1 -- 172.21.15.97:0/720023508 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fde20053a70 con 0x7fde3014cb00 2023-07-06T03:47:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.826+0000 7fde381ff700 1 -- 172.21.15.97:0/720023508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fde300cd800 con 0x7fde3014cb00 2023-07-06T03:47:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.826+0000 7fde267fc700 1 -- 172.21.15.97:0/720023508 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fde20019060 con 0x7fde3014cb00 2023-07-06T03:47:50.830 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:47:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 -- 172.21.15.97:0/720023508 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde1406ace0 msgr2=0x7fde1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde1406ace0 0x7fde1406d190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fde2c015040 tx=0x7fde2c00a000 comp rx=0 tx=0).stop 2023-07-06T03:47:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 -- 172.21.15.97:0/720023508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3014cb00 msgr2=0x7fde300d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:47:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3014cb00 0x7fde300d0c40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fde20001700 tx=0x7fde20007340 comp rx=0 tx=0).stop 2023-07-06T03:47:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 -- 172.21.15.97:0/720023508 shutdown_connections 2023-07-06T03:47:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde1406ace0 0x7fde1406d190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:50.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde300d1180 0x7fde300d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:50.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde1c003bc0 0x7fde3014d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:50.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 --2- 172.21.15.97:0/720023508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde3014cb00 0x7fde300d0c40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:47:50.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 -- 172.21.15.97:0/720023508 >> 172.21.15.97:0/720023508 conn(0x7fde30071790 msgr2=0x7fde3015fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:47:50.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 -- 172.21.15.97:0/720023508 shutdown_connections 2023-07-06T03:47:50.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:47:50.829+0000 7fde381ff700 1 -- 172.21.15.97:0/720023508 wait complete. 2023-07-06T03:47:50.849 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:47:50.849 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:47:50.850 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2023-07-06T03:47:50.850 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T03:47:50.850 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T03:47:50.850 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:47:50.851 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:47:50.851 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:47:50.851 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:47:50.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:50.852 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2023-07-06T03:47:50.903 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:47:50.904 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:47:50.904 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:47:50.904 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:50.904 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:47:50.904 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:50.904 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:47:50.905 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:47:50.905 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2023-07-06T03:47:50.905 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:47:50.905 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:47:50.905 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2023-07-06T03:47:50.905 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:47:50.905 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2023-07-06T03:47:50.905 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:47:50.906 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2023-07-06T03:47:50.906 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:47:50.906 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42014sec preferred_lft 42014sec 2023-07-06T03:47:50.906 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2023-07-06T03:47:50.906 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:50.906 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:47:50.906 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:47:50.906 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:47:50.907 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:50.907 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::58f0:6eff:fe04:3be0/64 scope link 2023-07-06T03:47:50.907 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:50.907 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:47:50.907 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:47:50.907 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::106d:9cff:fed5:9d0f/64 scope link 2023-07-06T03:47:50.907 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:50.908 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:47:50.908 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether a2:c5:04:8e:a2:a7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T03:47:50.908 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::a0c5:4ff:fe8e:a2a7/64 scope link 2023-07-06T03:47:50.908 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:47:50.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:50.909 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:47:50.980 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:47:50.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:50.984 DEBUG:teuthology.orchestra.run.smithi146:> route 2023-07-06T03:47:51.032 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2023-07-06T03:47:51.033 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:47:51.033 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:47:51.033 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:47:51.034 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:47:51.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:51.036 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:47:51.036 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:47:51.036 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:47:51.037 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:47:51.037 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:47:51.037 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:47:51.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:51.205 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:47:51.252 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T03:47:51.252 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T03:47:51.252 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:47:51.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:51.253 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2023-07-06T03:47:51.323 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-06T03:47:51.324 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:47:51.324 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T03:47:51.370 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:47:51.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:47:51.371 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:47:51.455 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:47:51.564 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:47:51.564 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:47:51.632 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:47:51.634 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:47:51.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:51.636 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:47:51.682 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:47:51.682 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-06T03:47:51.687 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:47:51.763 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:47:51.764 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:47:51.833 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:47:51.835 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:47:51.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:51.837 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:47:51.860 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2023-07-06T03:47:51.858+0000 7f5362d9c5c0 -1 init, newargv = 0x557eb8e5e3f0 newargc=15 2023-07-06T03:47:51.860 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[121319]: starting ceph client 2023-07-06T03:47:51.869 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[121319]: starting fuse 2023-07-06T03:47:51.887 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:47:51.887 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T03:47:51.888 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:47:51.888 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2023-07-06T03:47:51.888 DEBUG:teuthology.orchestra.run.smithi146:> import re 2023-07-06T03:47:51.889 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:47:51.889 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2023-07-06T03:47:51.889 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:47:51.889 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2023-07-06T03:47:51.890 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-06T03:47:51.890 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2023-07-06T03:47:51.890 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-06T03:47:51.891 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:47:51.891 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2023-07-06T03:47:51.891 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2023-07-06T03:47:51.891 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2023-07-06T03:47:51.892 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2023-07-06T03:47:51.892 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:47:51.893 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2023-07-06T03:47:51.893 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:47:51.893 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:47:51.894 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:47:51.894 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2023-07-06T03:47:51.894 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2023-07-06T03:47:51.895 DEBUG:teuthology.orchestra.run.smithi146:> return f 2023-07-06T03:47:51.895 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:47:51.895 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:47:51.895 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2023-07-06T03:47:51.896 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:47:52.072 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.121319.asok 2023-07-06T03:47:52.250 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.121319.asok 2023-07-06T03:47:52.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:52.251 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.121319.asok status 2023-07-06T03:47:52.475 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-07-06T03:47:52.476 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2023-07-06T03:47:52.476 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:47:52.476 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:47:52.476 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2023-07-06T03:47:52.476 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2023-07-06T03:47:52.476 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-06T03:47:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "121319", 2023-07-06T03:47:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2023-07-06T03:47:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:47:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2023-07-06T03:47:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2023-07-06T03:47:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 5776, 2023-07-06T03:47:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2023-07-06T03:47:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2023-07-06T03:47:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2023-07-06T03:47:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 5776 2023-07-06T03:47:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:47:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:47:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:47:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:47:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 644508316 2023-07-06T03:47:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-07-06T03:47:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:47:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:47:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:47:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:47:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 644508316 2023-07-06T03:47:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:47:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.5776 192.168.0.1:0/644508316", 2023-07-06T03:47:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/644508316", 2023-07-06T03:47:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2023-07-06T03:47:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 25, 2023-07-06T03:47:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 48, 2023-07-06T03:47:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:47:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2023-07-06T03:47:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2023-07-06T03:47:52.481 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-07-06T03:47:52.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:52.490 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2023-07-06T03:47:52.491 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-06T03:47:52.537 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2023-07-06T03:47:52.537 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-06T03:47:52.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:52.538 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:47:52.646 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:47:52.646 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:47:52.646 DEBUG:teuthology.orchestra.run.smithi097:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2023-07-06T03:47:52.647 DEBUG:teuthology.orchestra.run.smithi097:> for n in range(0, 1000): 2023-07-06T03:47:52.647 DEBUG:teuthology.orchestra.run.smithi097:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir/dir{0}".format(n)) 2023-07-06T03:47:52.647 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:47:54.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:54.761 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync) 2023-07-06T03:47:55.197 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:47:55.198 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:47:55.198 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:47:55.198 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:47:55.198 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:47:55.199 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:55.199 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:47:55.199 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:47:55.200 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:47:55.200 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:47:55.200 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:55.201 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:47:55.201 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:47:55.201 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:47:55.201 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:47:55.202 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:55.202 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:47:55.202 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:47:55.203 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:47:55.203 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:47:55.204 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:47:55.204 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:47:55.205 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:47:55.205 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:47:55.206 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:55.206 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:47:55.207 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:47:55.352 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.134043.asok 2023-07-06T03:47:55.529 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134043.asok 2023-07-06T03:47:55.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:55.530 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134043.asok status 2023-07-06T03:47:55.761 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:47:55.761 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:47:55.761 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:47:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:47:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:47:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:47:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:47:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "134043", 2023-07-06T03:47:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:47:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 1001, 2023-07-06T03:47:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 1001, 2023-07-06T03:47:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6021, 2023-07-06T03:47:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:47:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:47:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:47:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6021 2023-07-06T03:47:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:47:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:47:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:47:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 476559399 2023-07-06T03:47:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:47:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:47:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:47:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 476559399 2023-07-06T03:47:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6021 192.168.0.1:0/476559399", 2023-07-06T03:47:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/476559399", 2023-07-06T03:47:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1002, 2023-07-06T03:47:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 25, 2023-07-06T03:47:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 47, 2023-07-06T03:47:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 47, 2023-07-06T03:47:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:47:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:47:55.766 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:47:55.777 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:47:55.778 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:47:55.778 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:47:55.779 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:47:55.779 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:47:55.780 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:55.780 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:47:55.781 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:47:55.781 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:47:55.781 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:47:55.782 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:55.782 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:47:55.782 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:47:55.783 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:47:55.783 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:47:55.783 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:55.784 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:47:55.784 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:47:55.784 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:47:55.785 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:47:55.785 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:47:55.786 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:47:55.786 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:47:55.787 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:47:55.787 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:55.787 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:47:55.788 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:47:55.934 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.134043.asok 2023-07-06T03:47:56.112 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134043.asok 2023-07-06T03:47:56.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:56.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134043.asok config set client_cache_size 100 2023-07-06T03:47:56.334 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:47:56.335 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "client_cache_size = '100' " 2023-07-06T03:47:56.335 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:47:56.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:47:56.349 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:47:56.349 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:47:56.350 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:47:56.350 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:47:56.350 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:56.351 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:47:56.351 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:47:56.351 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:47:56.351 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:47:56.352 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:56.352 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:47:56.352 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:47:56.353 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:47:56.353 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:47:56.353 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:56.354 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:47:56.354 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:47:56.354 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:47:56.355 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:47:56.355 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:47:56.355 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:47:56.356 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:47:56.356 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:47:56.356 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:47:56.357 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:47:56.357 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:47:56.499 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.134043.asok 2023-07-06T03:47:56.676 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134043.asok 2023-07-06T03:47:56.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:47:56.677 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134043.asok status 2023-07-06T03:47:56.901 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:47:56.901 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:47:56.901 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:47:56.902 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:47:56.902 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:47:56.902 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:47:56.902 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:47:56.902 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "134043", 2023-07-06T03:47:56.902 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:47:56.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:56.903 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 1001, 2023-07-06T03:47:56.903 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 1, 2023-07-06T03:47:56.903 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6021, 2023-07-06T03:47:56.903 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:47:56.903 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:47:56.903 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:47:56.903 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6021 2023-07-06T03:47:56.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:56.904 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:47:56.904 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:47:56.904 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:47:56.904 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 476559399 2023-07-06T03:47:56.904 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:47:56.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:56.905 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:47:56.905 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:47:56.905 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:47:56.905 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 476559399 2023-07-06T03:47:56.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:47:56.905 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6021 192.168.0.1:0/476559399", 2023-07-06T03:47:56.905 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/476559399", 2023-07-06T03:47:56.906 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1002, 2023-07-06T03:47:56.906 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 25, 2023-07-06T03:47:56.906 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 47, 2023-07-06T03:47:56.906 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 47, 2023-07-06T03:47:56.906 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:47:56.906 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:47:56.906 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:47:56.915 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 1001, 1 2023-07-06T03:47:56.916 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-07-06T03:48:01.917 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:48:01.918 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:48:01.918 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:48:01.918 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:48:01.919 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:48:01.919 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:01.920 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:48:01.920 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:48:01.920 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:48:01.921 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:48:01.921 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:01.921 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:48:01.921 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:48:01.922 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:48:01.922 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:48:01.922 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:01.923 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:48:01.923 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:48:01.923 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:48:01.923 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:48:01.924 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:48:01.924 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:48:01.924 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:48:01.925 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:48:01.925 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:01.925 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:48:01.926 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:48:02.078 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.134043.asok 2023-07-06T03:48:02.256 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134043.asok 2023-07-06T03:48:02.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:02.256 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134043.asok status 2023-07-06T03:48:02.479 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:48:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:48:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:48:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:48:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:48:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:48:02.480 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "134043", 2023-07-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 100, 2023-07-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 1, 2023-07-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6021, 2023-07-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6021 2023-07-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 476559399 2023-07-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 476559399 2023-07-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6021 192.168.0.1:0/476559399", 2023-07-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/476559399", 2023-07-06T03:48:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 101, 2023-07-06T03:48:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 25, 2023-07-06T03:48:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 47, 2023-07-06T03:48:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 47, 2023-07-06T03:48:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:48:02.484 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:48:02.484 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:48:02.493 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 100, 1 2023-07-06T03:48:02.494 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-07-06T03:48:02.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:02.494 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-07-06T03:48:02.545 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:48:02.545 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:48:02.591 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:48:02.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.823+0000 7f639f9ff700 1 Processor -- start 2023-07-06T03:48:02.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.823+0000 7f639f9ff700 1 -- start start 2023-07-06T03:48:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639f9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6398158960 0x7f639806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639f9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6398071a00 0x7f6398158d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:02.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639f9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639806ddb0 con 0x7f639815ce90 2023-07-06T03:48:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639f9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639806bbf0 con 0x7f6398071a00 2023-07-06T03:48:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639f9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639806bed0 con 0x7f6398158960 2023-07-06T03:48:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639df9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6398071a00 0x7f6398158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639df9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6398071a00 0x7f6398158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56392/0 (socket says 172.21.15.97:56392) 2023-07-06T03:48:02.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639df9c700 1 -- 172.21.15.97:0/2523469412 learned_addr learned my addr 172.21.15.97:0/2523469412 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639d79b700 1 --2- 172.21.15.97:0/2523469412 >> v2:172.21.15.146:3300/0 conn(0x7f6398158960 0x7f639806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639df9c700 1 -- 172.21.15.97:0/2523469412 >> v2:172.21.15.146:3300/0 conn(0x7f6398158960 msgr2=0x7f639806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639df9c700 1 --2- 172.21.15.97:0/2523469412 >> v2:172.21.15.146:3300/0 conn(0x7f6398158960 0x7f639806d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639df9c700 1 -- 172.21.15.97:0/2523469412 >> v1:172.21.15.146:6789/0 conn(0x7f639815ce90 legacy=0x7f63980712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639df9c700 1 -- 172.21.15.97:0/2523469412 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639806c1b0 con 0x7f6398071a00 2023-07-06T03:48:02.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.825+0000 7f639d79b700 1 --2- 172.21.15.97:0/2523469412 >> v2:172.21.15.146:3300/0 conn(0x7f6398158960 0x7f639806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:02.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.826+0000 7f639df9c700 1 --2- 172.21.15.97:0/2523469412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398071a00 0x7f6398158d30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f639806f660 tx=0x7f639000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d21d6a1fcd17fe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:02.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.826+0000 7f638ffff700 1 -- 172.21.15.97:0/2523469412 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390003a20 con 0x7f6398071a00 2023-07-06T03:48:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.826+0000 7f638ffff700 1 -- 172.21.15.97:0/2523469412 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6390003b80 con 0x7f6398071a00 2023-07-06T03:48:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.826+0000 7f638ffff700 1 -- 172.21.15.97:0/2523469412 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639000b980 con 0x7f6398071a00 2023-07-06T03:48:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.826+0000 7f639f9ff700 1 -- 172.21.15.97:0/2523469412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398071a00 msgr2=0x7f6398158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.826+0000 7f639f9ff700 1 --2- 172.21.15.97:0/2523469412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398071a00 0x7f6398158d30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f639806f660 tx=0x7f639000b330 comp rx=0 tx=0).stop 2023-07-06T03:48:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.826+0000 7f639f9ff700 1 -- 172.21.15.97:0/2523469412 shutdown_connections 2023-07-06T03:48:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.826+0000 7f639f9ff700 1 --2- 172.21.15.97:0/2523469412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398071a00 0x7f6398158d30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.826+0000 7f639f9ff700 1 --2- 172.21.15.97:0/2523469412 >> v2:172.21.15.146:3300/0 conn(0x7f6398158960 0x7f639806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.826+0000 7f639f9ff700 1 -- 172.21.15.97:0/2523469412 >> 172.21.15.97:0/2523469412 conn(0x7f63980ce8c0 msgr2=0x7f63980cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:02.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.827+0000 7f639f9ff700 1 -- 172.21.15.97:0/2523469412 shutdown_connections 2023-07-06T03:48:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.827+0000 7f639f9ff700 1 -- 172.21.15.97:0/2523469412 wait complete. 2023-07-06T03:48:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.828+0000 7f639f9ff700 1 Processor -- start 2023-07-06T03:48:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.828+0000 7f639f9ff700 1 -- start start 2023-07-06T03:48:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.829+0000 7f639f9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6398071a00 0x7f63981f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.829+0000 7f639f9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6398158960 0x7f6398140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:02.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.829+0000 7f639f9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f639815ce90 0x7f6398164fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.829+0000 7f639f9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63981f6260 con 0x7f639815ce90 2023-07-06T03:48:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.829+0000 7f639f9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398149c60 con 0x7f6398071a00 2023-07-06T03:48:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.829+0000 7f639f9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6398165510 con 0x7f6398158960 2023-07-06T03:48:02.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.829+0000 7f639df9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f639815ce90 0x7f6398164fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.829+0000 7f639df9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f639815ce90 0x7f6398164fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56394/0 (socket says 172.21.15.97:56394) 2023-07-06T03:48:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.829+0000 7f639df9c700 1 -- 172.21.15.97:0/1127636236 learned_addr learned my addr 172.21.15.97:0/1127636236 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f639d79b700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6398071a00 0x7f63981f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f639cf9a700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6398158960 0x7f6398140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f639df9c700 1 -- 172.21.15.97:0/1127636236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6398158960 msgr2=0x7f6398140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:02.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f639df9c700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6398158960 0x7f6398140490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:02.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f639df9c700 1 -- 172.21.15.97:0/1127636236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6398071a00 msgr2=0x7f63981f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:02.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f639df9c700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6398071a00 0x7f63981f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:02.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f639df9c700 1 -- 172.21.15.97:0/1127636236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390009b30 con 0x7f639815ce90 2023-07-06T03:48:02.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f639df9c700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f639815ce90 0x7f6398164fd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6390009fd0 tx=0x7f6390004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:02.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f638dffb700 1 -- 172.21.15.97:0/1127636236 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900092e0 con 0x7f639815ce90 2023-07-06T03:48:02.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f638dffb700 1 -- 172.21.15.97:0/1127636236 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63900094d0 con 0x7f639815ce90 2023-07-06T03:48:02.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f638dffb700 1 -- 172.21.15.97:0/1127636236 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900236a0 con 0x7f639815ce90 2023-07-06T03:48:02.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f639f9ff700 1 -- 172.21.15.97:0/1127636236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63980ae210 con 0x7f639815ce90 2023-07-06T03:48:02.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.830+0000 7f639f9ff700 1 -- 172.21.15.97:0/1127636236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6398149880 con 0x7f639815ce90 2023-07-06T03:48:02.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.831+0000 7f639f9ff700 1 -- 172.21.15.97:0/1127636236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6398067830 con 0x7f639815ce90 2023-07-06T03:48:02.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.832+0000 7f638dffb700 1 -- 172.21.15.97:0/1127636236 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f639001a430 con 0x7f639815ce90 2023-07-06T03:48:02.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.833+0000 7f638dffb700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 0x7f637c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:02.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.833+0000 7f638dffb700 1 -- 172.21.15.97:0/1127636236 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f639008a8c0 con 0x7f639815ce90 2023-07-06T03:48:02.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.833+0000 7f639d79b700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 0x7f637c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:02.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.834+0000 7f639d79b700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 0x7f637c06d190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6394006fd0 tx=0x7f6394008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:02.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:02.838+0000 7f638dffb700 1 -- 172.21.15.97:0/1127636236 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f63900558e0 con 0x7f639815ce90 2023-07-06T03:48:03.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.036+0000 7f639f9ff700 1 -- 172.21.15.97:0/1127636236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f639804e340 con 0x7f639815ce90 2023-07-06T03:48:03.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.037+0000 7f638dffb700 1 -- 172.21.15.97:0/1127636236 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18524 (secure 0 0 0) 0x7f6390058f00 con 0x7f639815ce90 2023-07-06T03:48:03.040 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":49,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:47:52.725520+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:47:33.695348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-06T03:47:35.132699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":2,"to":5}]},{"pgid":"8.1a","mappings":[{"from":2,"to":1}]},{"pgid":"8.38","mappings":[{"from":6,"to":5}]},{"pgid":"8.3d","mappings":[{"from":6,"to":0},{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:48:03.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.039+0000 7f639f9ff700 1 -- 172.21.15.97:0/1127636236 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 msgr2=0x7f637c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.039+0000 7f639f9ff700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 0x7f637c06d190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6394006fd0 tx=0x7f6394008040 comp rx=0 tx=0).stop 2023-07-06T03:48:03.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.039+0000 7f639f9ff700 1 -- 172.21.15.97:0/1127636236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f639815ce90 msgr2=0x7f6398164fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.039+0000 7f639f9ff700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f639815ce90 0x7f6398164fd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6390009fd0 tx=0x7f6390004ad0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.040+0000 7f639f9ff700 1 -- 172.21.15.97:0/1127636236 shutdown_connections 2023-07-06T03:48:03.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.040+0000 7f639f9ff700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 0x7f637c06d190 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.040+0000 7f639f9ff700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f639815ce90 0x7f6398164fd0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.040+0000 7f639f9ff700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6398071a00 0x7f63981f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.040+0000 7f639f9ff700 1 --2- 172.21.15.97:0/1127636236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6398158960 0x7f6398140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.040+0000 7f639f9ff700 1 -- 172.21.15.97:0/1127636236 >> 172.21.15.97:0/1127636236 conn(0x7f63980ce8c0 msgr2=0x7f63980712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:03.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.040+0000 7f639f9ff700 1 -- 172.21.15.97:0/1127636236 shutdown_connections 2023-07-06T03:48:03.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.040+0000 7f639f9ff700 1 -- 172.21.15.97:0/1127636236 wait complete. 2023-07-06T03:48:03.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:47:33.695348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:47:35.132699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:48:03.061 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:48:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.255+0000 7fa8d17ff700 1 Processor -- start 2023-07-06T03:48:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.255+0000 7fa8d17ff700 1 -- start start 2023-07-06T03:48:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.256+0000 7fa8d17ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa8cc06b360 0x7fa8cc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.256+0000 7fa8d17ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc0d2210 0x7fa8cc0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8d17ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc0cff90 con 0x7fa8cc15c610 2023-07-06T03:48:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8d17ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8cc0d0780 con 0x7fa8cc06b360 2023-07-06T03:48:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8d17ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc15c9e0 con 0x7fa8cc0d2210 2023-07-06T03:48:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8ca7fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa8cc06b360 0x7fa8cc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8cb7fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc0d2210 0x7fa8cc0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8cb7fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc0d2210 0x7fa8cc0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:60222/0 (socket says 172.21.15.97:60222) 2023-07-06T03:48:03.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8ca7fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa8cc06b360 0x7fa8cc0b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:52442/0 (socket says 172.21.15.97:52442) 2023-07-06T03:48:03.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8cb7fe700 1 -- 172.21.15.97:0/3434874055 learned_addr learned my addr 172.21.15.97:0/3434874055 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:03.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8cb7fe700 1 -- 172.21.15.97:0/3434874055 >> v2:172.21.15.146:3300/0 conn(0x7fa8cc06b360 msgr2=0x7fa8cc0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:03.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8cb7fe700 1 --2- 172.21.15.97:0/3434874055 >> v2:172.21.15.146:3300/0 conn(0x7fa8cc06b360 0x7fa8cc0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8cb7fe700 1 -- 172.21.15.97:0/3434874055 >> v1:172.21.15.146:6789/0 conn(0x7fa8cc15c610 legacy=0x7fa8cc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.257+0000 7fa8cb7fe700 1 -- 172.21.15.97:0/3434874055 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc0b5650 con 0x7fa8cc0d2210 2023-07-06T03:48:03.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.258+0000 7fa8cb7fe700 1 --2- 172.21.15.97:0/3434874055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc0d2210 0x7fa8cc0d1310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009e50 tx=0x7fa8c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9da6f1c601e213 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:03.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.258+0000 7fa8c9ffb700 1 -- 172.21.15.97:0/3434874055 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400e070 con 0x7fa8cc0d2210 2023-07-06T03:48:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.258+0000 7fa8c9ffb700 1 -- 172.21.15.97:0/3434874055 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c400b910 con 0x7fa8cc0d2210 2023-07-06T03:48:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.258+0000 7fa8c9ffb700 1 -- 172.21.15.97:0/3434874055 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400bcc0 con 0x7fa8cc0d2210 2023-07-06T03:48:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.258+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3434874055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc0d2210 msgr2=0x7fa8cc0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.258+0000 7fa8d17ff700 1 --2- 172.21.15.97:0/3434874055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc0d2210 0x7fa8cc0d1310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009e50 tx=0x7fa8c400b330 comp rx=0 tx=0).stop 2023-07-06T03:48:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.259+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3434874055 shutdown_connections 2023-07-06T03:48:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.259+0000 7fa8d17ff700 1 --2- 172.21.15.97:0/3434874055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc0d2210 0x7fa8cc0d1310 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.259+0000 7fa8d17ff700 1 --2- 172.21.15.97:0/3434874055 >> v2:172.21.15.146:3300/0 conn(0x7fa8cc06b360 0x7fa8cc0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.259+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3434874055 >> 172.21.15.97:0/3434874055 conn(0x7fa8cc06a6c0 msgr2=0x7fa8cc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:03.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.259+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3434874055 shutdown_connections 2023-07-06T03:48:03.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.259+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3434874055 wait complete. 2023-07-06T03:48:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.260+0000 7fa8d17ff700 1 Processor -- start 2023-07-06T03:48:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.261+0000 7fa8d17ff700 1 -- start start 2023-07-06T03:48:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.261+0000 7fa8d17ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8cc06b360 0x7fa8cc15d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.261+0000 7fa8d17ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8cc0d2210 0x7fa8cc0c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8d17ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc15c610 0x7fa8cc15e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8d17ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc15f420 con 0x7fa8cc06b360 2023-07-06T03:48:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8d17ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc15d310 con 0x7fa8cc0d2210 2023-07-06T03:48:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8d17ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc15f140 con 0x7fa8cc15c610 2023-07-06T03:48:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8caffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8cc06b360 0x7fa8cc15d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8caffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8cc06b360 0x7fa8cc15d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56398/0 (socket says 172.21.15.97:56398) 2023-07-06T03:48:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8caffd700 1 -- 172.21.15.97:0/3256497122 learned_addr learned my addr 172.21.15.97:0/3256497122 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8cb7fe700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc15c610 0x7fa8cc15e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8ca7fc700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8cc0d2210 0x7fa8cc0c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8caffd700 1 -- 172.21.15.97:0/3256497122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc15c610 msgr2=0x7fa8cc15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8caffd700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc15c610 0x7fa8cc15e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8caffd700 1 -- 172.21.15.97:0/3256497122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8cc0d2210 msgr2=0x7fa8cc0c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8caffd700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8cc0d2210 0x7fa8cc0c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8caffd700 1 -- 172.21.15.97:0/3256497122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c4009b30 con 0x7fa8cc06b360 2023-07-06T03:48:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8ca7fc700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8cc0d2210 0x7fa8cc0c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.262+0000 7fa8caffd700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8cc06b360 0x7fa8cc15d7a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c450 tx=0x7fa8bc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.263+0000 7fa8b37fe700 1 -- 172.21.15.97:0/3256497122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0052d0 con 0x7fa8cc06b360 2023-07-06T03:48:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.263+0000 7fa8b37fe700 1 -- 172.21.15.97:0/3256497122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc005430 con 0x7fa8cc06b360 2023-07-06T03:48:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.263+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3256497122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc15ccc0 con 0x7fa8cc06b360 2023-07-06T03:48:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.263+0000 7fa8b37fe700 1 -- 172.21.15.97:0/3256497122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0038c0 con 0x7fa8cc06b360 2023-07-06T03:48:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.263+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3256497122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc1546b0 con 0x7fa8cc06b360 2023-07-06T03:48:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.264+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3256497122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8cc15cb20 con 0x7fa8cc06b360 2023-07-06T03:48:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.265+0000 7fa8b37fe700 1 -- 172.21.15.97:0/3256497122 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa8bc003a20 con 0x7fa8cc06b360 2023-07-06T03:48:03.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.266+0000 7fa8b37fe700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8b806ace0 0x7fa8b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.266+0000 7fa8b37fe700 1 -- 172.21.15.97:0/3256497122 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7fa8bc088940 con 0x7fa8cc06b360 2023-07-06T03:48:03.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.266+0000 7fa8ca7fc700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8b806ace0 0x7fa8b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.266+0000 7fa8ca7fc700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8b806ace0 0x7fa8b806d190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0b1b90 tx=0x7fa8c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:03.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.270+0000 7fa8b37fe700 1 -- 172.21.15.97:0/3256497122 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa8cc15cb20 con 0x7fa8cc06b360 2023-07-06T03:48:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.463+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3256497122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa8cc15cb20 con 0x7fa8cc06b360 2023-07-06T03:48:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.464+0000 7fa8b37fe700 1 -- 172.21.15.97:0/3256497122 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa8cc15cb20 con 0x7fa8cc06b360 2023-07-06T03:48:03.466 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:48:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3256497122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8b806ace0 msgr2=0x7fa8b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8b806ace0 0x7fa8b806d190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0b1b90 tx=0x7fa8c0008040 comp rx=0 tx=0).stop 2023-07-06T03:48:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3256497122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8cc06b360 msgr2=0x7fa8cc15d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8cc06b360 0x7fa8cc15d7a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c450 tx=0x7fa8bc00c810 comp rx=0 tx=0).stop 2023-07-06T03:48:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3256497122 shutdown_connections 2023-07-06T03:48:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa8b806ace0 0x7fa8b806d190 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa8cc15c610 0x7fa8cc15e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8cc06b360 0x7fa8cc15d7a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 --2- 172.21.15.97:0/3256497122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8cc0d2210 0x7fa8cc0c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3256497122 >> 172.21.15.97:0/3256497122 conn(0x7fa8cc06a6c0 msgr2=0x7fa8cc0b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3256497122 shutdown_connections 2023-07-06T03:48:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.466+0000 7fa8d17ff700 1 -- 172.21.15.97:0/3256497122 wait complete. 2023-07-06T03:48:03.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:48:03.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.682+0000 7f7c7abff700 1 Processor -- start 2023-07-06T03:48:03.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.682+0000 7f7c7abff700 1 -- start start 2023-07-06T03:48:03.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.683+0000 7f7c7abff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7c74159070 0x7f7c740c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.683+0000 7f7c7abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740ccbd0 0x7f7c740cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.683+0000 7f7c7abff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c740ca260 con 0x7f7c740c3cd0 2023-07-06T03:48:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.683+0000 7f7c7abff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c740c99c0 con 0x7f7c74159070 2023-07-06T03:48:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.683+0000 7f7c7abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740c9ca0 con 0x7f7c740ccbd0 2023-07-06T03:48:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.684+0000 7f7c7899b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7c74159070 0x7f7c740c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.684+0000 7f7c73fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7c740c3cd0 0x7f7c740cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58490/0 (socket says 172.21.15.97:58490) 2023-07-06T03:48:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.684+0000 7f7c73fff700 1 -- 172.21.15.97:0/1676512892 learned_addr learned my addr 172.21.15.97:0/1676512892 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:03.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.684+0000 7f7c7919c700 1 --2- 172.21.15.97:0/1676512892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740ccbd0 0x7f7c740cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.684+0000 7f7c737fe700 1 -- 172.21.15.97:0/1676512892 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255013154 0 0) 0x7f7c740ca260 con 0x7f7c740c3cd0 2023-07-06T03:48:03.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.684+0000 7f7c737fe700 1 -- 172.21.15.97:0/1676512892 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c640035b0 con 0x7f7c740c3cd0 2023-07-06T03:48:03.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.684+0000 7f7c7899b700 1 -- 172.21.15.97:0/1676512892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740ccbd0 msgr2=0x7f7c740cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:03.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.684+0000 7f7c7899b700 1 --2- 172.21.15.97:0/1676512892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740ccbd0 0x7f7c740cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.684+0000 7f7c7899b700 1 -- 172.21.15.97:0/1676512892 >> v1:172.21.15.97:6789/0 conn(0x7f7c740c3cd0 legacy=0x7f7c740cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.684+0000 7f7c7899b700 1 -- 172.21.15.97:0/1676512892 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740c9f80 con 0x7f7c74159070 2023-07-06T03:48:03.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.685+0000 7f7c7899b700 1 --2- 172.21.15.97:0/1676512892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c74159070 0x7f7c740c8f40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009840 tx=0x7f7c68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98d5f2ffa95df65 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:03.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.685+0000 7f7c737fe700 1 -- 172.21.15.97:0/1676512892 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68011830 con 0x7f7c74159070 2023-07-06T03:48:03.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.685+0000 7f7c737fe700 1 -- 172.21.15.97:0/1676512892 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c68011990 con 0x7f7c74159070 2023-07-06T03:48:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.685+0000 7f7c737fe700 1 -- 172.21.15.97:0/1676512892 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68011d20 con 0x7f7c74159070 2023-07-06T03:48:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.685+0000 7f7c7abff700 1 -- 172.21.15.97:0/1676512892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c74159070 msgr2=0x7f7c740c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.685+0000 7f7c7abff700 1 --2- 172.21.15.97:0/1676512892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c74159070 0x7f7c740c8f40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009840 tx=0x7f7c68009b50 comp rx=0 tx=0).stop 2023-07-06T03:48:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.685+0000 7f7c7abff700 1 -- 172.21.15.97:0/1676512892 shutdown_connections 2023-07-06T03:48:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.685+0000 7f7c7abff700 1 --2- 172.21.15.97:0/1676512892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c74159070 0x7f7c740c8f40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.685+0000 7f7c7abff700 1 --2- 172.21.15.97:0/1676512892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740ccbd0 0x7f7c740cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.685+0000 7f7c7abff700 1 -- 172.21.15.97:0/1676512892 >> 172.21.15.97:0/1676512892 conn(0x7f7c7406a060 msgr2=0x7f7c7406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.686+0000 7f7c7abff700 1 -- 172.21.15.97:0/1676512892 shutdown_connections 2023-07-06T03:48:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.686+0000 7f7c7abff700 1 -- 172.21.15.97:0/1676512892 wait complete. 2023-07-06T03:48:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.687+0000 7f7c7abff700 1 Processor -- start 2023-07-06T03:48:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.687+0000 7f7c7abff700 1 -- start start 2023-07-06T03:48:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.688+0000 7f7c7abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740c3cd0 0x7f7c740c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.688+0000 7f7c7abff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c740ccbd0 0x7f7c7414ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.688+0000 7f7c7abff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c74159070 0x7f7c740c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.688+0000 7f7c7abff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740af610 con 0x7f7c74159070 2023-07-06T03:48:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.688+0000 7f7c7abff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740c8840 con 0x7f7c740ccbd0 2023-07-06T03:48:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.688+0000 7f7c7abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740af000 con 0x7f7c740c3cd0 2023-07-06T03:48:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7899b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740c3cd0 0x7f7c740c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7919c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c74159070 0x7f7c740c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7899b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740c3cd0 0x7f7c740c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:60238/0 (socket says 172.21.15.97:60238) 2023-07-06T03:48:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7899b700 1 -- 172.21.15.97:0/1350883283 learned_addr learned my addr 172.21.15.97:0/1350883283 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c73fff700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c740ccbd0 0x7f7c7414ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7919c700 1 -- 172.21.15.97:0/1350883283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740c3cd0 msgr2=0x7f7c740c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7919c700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740c3cd0 0x7f7c740c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7919c700 1 -- 172.21.15.97:0/1350883283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c740ccbd0 msgr2=0x7f7c7414ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7919c700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c740ccbd0 0x7f7c7414ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7919c700 1 -- 172.21.15.97:0/1350883283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c680094a0 con 0x7f7c74159070 2023-07-06T03:48:03.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7899b700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740c3cd0 0x7f7c740c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:48:03.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7919c700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c74159070 0x7f7c740c7c50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7c74071f90 tx=0x7f7c6c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:03.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c717fa700 1 -- 172.21.15.97:0/1350883283 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00cc20 con 0x7f7c74159070 2023-07-06T03:48:03.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c717fa700 1 -- 172.21.15.97:0/1350883283 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c00f040 con 0x7f7c74159070 2023-07-06T03:48:03.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c717fa700 1 -- 172.21.15.97:0/1350883283 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c009e60 con 0x7f7c74159070 2023-07-06T03:48:03.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.689+0000 7f7c7abff700 1 -- 172.21.15.97:0/1350883283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c740c8560 con 0x7f7c74159070 2023-07-06T03:48:03.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.690+0000 7f7c7abff700 1 -- 172.21.15.97:0/1350883283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7414e980 con 0x7f7c74159070 2023-07-06T03:48:03.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.691+0000 7f7c7abff700 1 -- 172.21.15.97:0/1350883283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c740af7c0 con 0x7f7c74159070 2023-07-06T03:48:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.692+0000 7f7c717fa700 1 -- 172.21.15.97:0/1350883283 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7c6c014480 con 0x7f7c74159070 2023-07-06T03:48:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.693+0000 7f7c717fa700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7c5806ace0 0x7f7c5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.693+0000 7f7c717fa700 1 -- 172.21.15.97:0/1350883283 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f7c6c089f80 con 0x7f7c74159070 2023-07-06T03:48:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.693+0000 7f7c7899b700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7c5806ace0 0x7f7c5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.694+0000 7f7c7899b700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7c5806ace0 0x7f7c5806d190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009b90 tx=0x7f7c6801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:03.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.697+0000 7f7c717fa700 1 -- 172.21.15.97:0/1350883283 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7c740af7c0 con 0x7f7c74159070 2023-07-06T03:48:03.727 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:48:03.731 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:48:03.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.900+0000 7f7c7abff700 1 -- 172.21.15.97:0/1350883283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7c7404e340 con 0x7f7c74159070 2023-07-06T03:48:03.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.900+0000 7f7c717fa700 1 -- 172.21.15.97:0/1350883283 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7c6c0180a0 con 0x7f7c74159070 2023-07-06T03:48:03.904 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:48:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 -- 172.21.15.97:0/1350883283 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7c5806ace0 msgr2=0x7f7c5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7c5806ace0 0x7f7c5806d190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009b90 tx=0x7f7c6801c040 comp rx=0 tx=0).stop 2023-07-06T03:48:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 -- 172.21.15.97:0/1350883283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c74159070 msgr2=0x7f7c740c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c74159070 0x7f7c740c7c50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7c74071f90 tx=0x7f7c6c00ef90 comp rx=0 tx=0).stop 2023-07-06T03:48:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 -- 172.21.15.97:0/1350883283 shutdown_connections 2023-07-06T03:48:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7c5806ace0 0x7f7c5806d190 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c74159070 0x7f7c740c7c50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7c740c3cd0 0x7f7c740c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 --2- 172.21.15.97:0/1350883283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c740ccbd0 0x7f7c7414ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 -- 172.21.15.97:0/1350883283 >> 172.21.15.97:0/1350883283 conn(0x7f7c7406a060 msgr2=0x7f7c7406f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 -- 172.21.15.97:0/1350883283 shutdown_connections 2023-07-06T03:48:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:03.903+0000 7f7c7abff700 1 -- 172.21.15.97:0/1350883283 wait complete. 2023-07-06T03:48:03.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:48:04.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.118+0000 7f1188dff700 1 Processor -- start 2023-07-06T03:48:04.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.118+0000 7f1188dff700 1 -- start start 2023-07-06T03:48:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.120+0000 7f1188dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f118415cae0 0x7f118415d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.120+0000 7f1188dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118415de00 con 0x7f118406b170 2023-07-06T03:48:04.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.120+0000 7f1188dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11840ceb20 con 0x7f11840cfbc0 2023-07-06T03:48:04.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.120+0000 7f1188dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11840b25a0 con 0x7f118415cae0 2023-07-06T03:48:04.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.120+0000 7f118259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f118406b170 0x7f118406b540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58492/0 (socket says 172.21.15.97:58492) 2023-07-06T03:48:04.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.120+0000 7f118259c700 1 -- 172.21.15.97:0/652551246 learned_addr learned my addr 172.21.15.97:0/652551246 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:04.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.120+0000 7f117bfff700 1 --2- 172.21.15.97:0/652551246 >> v2:172.21.15.146:3300/0 conn(0x7f118415cae0 0x7f118415d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.120+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062082199 0 0) 0x7f1174001290 con 0x7f118406b170 2023-07-06T03:48:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.120+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11700035b0 con 0x7f118406b170 2023-07-06T03:48:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4282705658 0 0) 0x7f11700035b0 con 0x7f118406b170 2023-07-06T03:48:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 >> v2:172.21.15.146:3300/0 conn(0x7f118415cae0 msgr2=0x7f118415d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 --2- 172.21.15.97:0/652551246 >> v2:172.21.15.146:3300/0 conn(0x7f118415cae0 0x7f118415d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 >> v1:172.21.15.146:6789/0 conn(0x7f11840cfbc0 legacy=0x7f118415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11840cee00 con 0x7f118406b170 2023-07-06T03:48:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f1174001980 con 0x7f118406b170 2023-07-06T03:48:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 >> v1:172.21.15.97:6789/0 conn(0x7f118406b170 legacy=0x7f118406b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1170003a00 0x7f1170003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170004850 0x7f1170004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1170005200 0x7f117000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117000c0e0 con 0x7f1170003a00 2023-07-06T03:48:04.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117000c250 con 0x7f1170004850 2023-07-06T03:48:04.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117000c410 con 0x7f1170005200 2023-07-06T03:48:04.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f118259c700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1170003a00 0x7f1170003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f117bfff700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170004850 0x7f1170004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.121+0000 7f1182d9d700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1170005200 0x7f117000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f118259c700 1 -- 172.21.15.97:0/652551246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1170005200 msgr2=0x7f117000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:04.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f118259c700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1170005200 0x7f117000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f118259c700 1 -- 172.21.15.97:0/652551246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170004850 msgr2=0x7f1170004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:04.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f118259c700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170004850 0x7f1170004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f118259c700 1 -- 172.21.15.97:0/652551246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11700035b0 con 0x7f1170003a00 2023-07-06T03:48:04.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f117bfff700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170004850 0x7f1170004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:04.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f1182d9d700 1 -- 172.21.15.97:0/652551246 reap_dead start 2023-07-06T03:48:04.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f118259c700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1170003a00 0x7f1170003e10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f117400c210 tx=0x7f1170005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4515aa570e6f0f9c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:04.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400cdb0 con 0x7f1170003a00 2023-07-06T03:48:04.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1174007d40 con 0x7f1170003a00 2023-07-06T03:48:04.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f1181d9b700 1 -- 172.21.15.97:0/652551246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174007310 con 0x7f1170003a00 2023-07-06T03:48:04.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f1188dff700 1 -- 172.21.15.97:0/652551246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1170003a00 msgr2=0x7f1170003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.122+0000 7f1188dff700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1170003a00 0x7f1170003e10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f117400c210 tx=0x7f1170005880 comp rx=0 tx=0).stop 2023-07-06T03:48:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.123+0000 7f1188dff700 1 -- 172.21.15.97:0/652551246 shutdown_connections 2023-07-06T03:48:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.123+0000 7f1188dff700 1 --2- 172.21.15.97:0/652551246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1170003a00 0x7f1170003e10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.123+0000 7f1188dff700 1 -- 172.21.15.97:0/652551246 >> 172.21.15.97:0/652551246 conn(0x7f1184099f10 msgr2=0x7f11840d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.123+0000 7f1188dff700 1 -- 172.21.15.97:0/652551246 shutdown_connections 2023-07-06T03:48:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.123+0000 7f1188dff700 1 -- 172.21.15.97:0/652551246 wait complete. 2023-07-06T03:48:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.124+0000 7f1188dff700 1 Processor -- start 2023-07-06T03:48:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.125+0000 7f1188dff700 1 -- start start 2023-07-06T03:48:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.125+0000 7f1188dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170003a00 0x7f11841e4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.125+0000 7f1188dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11841e5410 0x7f11840c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.125+0000 7f1188dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11840c4c70 0x7f1184141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.125+0000 7f1188dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840c50f0 con 0x7f11840c4c70 2023-07-06T03:48:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.125+0000 7f1188dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11841e3e30 con 0x7f1170003a00 2023-07-06T03:48:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.125+0000 7f1188dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11841e4110 con 0x7f11841e5410 2023-07-06T03:48:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.126+0000 7f1182d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11840c4c70 0x7f1184141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.126+0000 7f1182d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11840c4c70 0x7f1184141ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56430/0 (socket says 172.21.15.97:56430) 2023-07-06T03:48:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.126+0000 7f1182d9d700 1 -- 172.21.15.97:0/2466429120 learned_addr learned my addr 172.21.15.97:0/2466429120 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.126+0000 7f118259c700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170003a00 0x7f11841e4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.126+0000 7f117bfff700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11841e5410 0x7f11840c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.126+0000 7f1182d9d700 1 -- 172.21.15.97:0/2466429120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11841e5410 msgr2=0x7f11840c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.126+0000 7f1182d9d700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11841e5410 0x7f11840c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.126+0000 7f1182d9d700 1 -- 172.21.15.97:0/2466429120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170003a00 msgr2=0x7f11841e4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.126+0000 7f1182d9d700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170003a00 0x7f11841e4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.126+0000 7f1182d9d700 1 -- 172.21.15.97:0/2466429120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11841e43f0 con 0x7f11840c4c70 2023-07-06T03:48:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.126+0000 7f1182d9d700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11840c4c70 0x7f1184141ff0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f11840ce600 tx=0x7f117c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.127+0000 7f117affd700 1 -- 172.21.15.97:0/2466429120 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c003c10 con 0x7f11840c4c70 2023-07-06T03:48:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.127+0000 7f117affd700 1 -- 172.21.15.97:0/2466429120 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c003d70 con 0x7f11840c4c70 2023-07-06T03:48:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.127+0000 7f117affd700 1 -- 172.21.15.97:0/2466429120 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c005310 con 0x7f11840c4c70 2023-07-06T03:48:04.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.127+0000 7f1188dff700 1 -- 172.21.15.97:0/2466429120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11841e46d0 con 0x7f11840c4c70 2023-07-06T03:48:04.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.127+0000 7f1188dff700 1 -- 172.21.15.97:0/2466429120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1184155860 con 0x7f11840c4c70 2023-07-06T03:48:04.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.128+0000 7f1188dff700 1 -- 172.21.15.97:0/2466429120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11841e4bf0 con 0x7f11840c4c70 2023-07-06T03:48:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.129+0000 7f117affd700 1 -- 172.21.15.97:0/2466429120 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f117c005470 con 0x7f11840c4c70 2023-07-06T03:48:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.130+0000 7f117affd700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f116406ace0 0x7f116406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.130+0000 7f117affd700 1 -- 172.21.15.97:0/2466429120 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f117c0895b0 con 0x7f11840c4c70 2023-07-06T03:48:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.130+0000 7f118259c700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f116406ace0 0x7f116406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.130+0000 7f118259c700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f116406ace0 0x7f116406d190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1174000c00 tx=0x7f1174007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.134+0000 7f117affd700 1 -- 172.21.15.97:0/2466429120 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f11841e4bf0 con 0x7f11840c4c70 2023-07-06T03:48:04.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.329+0000 7f1188dff700 1 -- 172.21.15.97:0/2466429120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11841e4bf0 con 0x7f11840c4c70 2023-07-06T03:48:04.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.329+0000 7f117affd700 1 -- 172.21.15.97:0/2466429120 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11841e4bf0 con 0x7f11840c4c70 2023-07-06T03:48:04.332 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:48:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.331+0000 7f1188dff700 1 -- 172.21.15.97:0/2466429120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f116406ace0 msgr2=0x7f116406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.331+0000 7f1188dff700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f116406ace0 0x7f116406d190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1174000c00 tx=0x7f1174007000 comp rx=0 tx=0).stop 2023-07-06T03:48:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.332+0000 7f1188dff700 1 -- 172.21.15.97:0/2466429120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11840c4c70 msgr2=0x7f1184141ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.332+0000 7f1188dff700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11840c4c70 0x7f1184141ff0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f11840ce600 tx=0x7f117c00ee80 comp rx=0 tx=0).stop 2023-07-06T03:48:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.332+0000 7f1188dff700 1 -- 172.21.15.97:0/2466429120 shutdown_connections 2023-07-06T03:48:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.332+0000 7f1188dff700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f116406ace0 0x7f116406d190 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.332+0000 7f1188dff700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11840c4c70 0x7f1184141ff0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.332+0000 7f1188dff700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170003a00 0x7f11841e4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.332+0000 7f1188dff700 1 --2- 172.21.15.97:0/2466429120 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11841e5410 0x7f11840c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.332+0000 7f1188dff700 1 -- 172.21.15.97:0/2466429120 >> 172.21.15.97:0/2466429120 conn(0x7f1184099f10 msgr2=0x7f118406ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:04.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.332+0000 7f1188dff700 1 -- 172.21.15.97:0/2466429120 shutdown_connections 2023-07-06T03:48:04.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.332+0000 7f1188dff700 1 -- 172.21.15.97:0/2466429120 wait complete. 2023-07-06T03:48:04.352 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:48:04.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.546+0000 7fb3b63ff700 1 Processor -- start 2023-07-06T03:48:04.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.547+0000 7fb3b63ff700 1 -- start start 2023-07-06T03:48:04.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.548+0000 7fb3b63ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3b006b360 0x7fb3b00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.548+0000 7fb3b63ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b00cff90 con 0x7fb3b00d2210 2023-07-06T03:48:04.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.548+0000 7fb3b63ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b00b5650 con 0x7fb3b015c610 2023-07-06T03:48:04.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.548+0000 7fb3b63ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b00d0780 con 0x7fb3b006b360 2023-07-06T03:48:04.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.548+0000 7fb3af7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3b006b360 0x7fb3b00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.548+0000 7fb3b499c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb3b00d2210 0x7fb3b00d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58496/0 (socket says 172.21.15.97:58496) 2023-07-06T03:48:04.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.548+0000 7fb3b499c700 1 -- 172.21.15.97:0/2047288893 learned_addr learned my addr 172.21.15.97:0/2047288893 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:04.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.548+0000 7fb3af7fe700 1 -- 172.21.15.97:0/2047288893 >> v1:172.21.15.146:6789/0 conn(0x7fb3b015c610 legacy=0x7fb3b00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.548+0000 7fb3af7fe700 1 -- 172.21.15.97:0/2047288893 >> v1:172.21.15.97:6789/0 conn(0x7fb3b00d2210 legacy=0x7fb3b00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.548+0000 7fb3af7fe700 1 -- 172.21.15.97:0/2047288893 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00b5400 con 0x7fb3b006b360 2023-07-06T03:48:04.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.549+0000 7fb3af7fe700 1 --2- 172.21.15.97:0/2047288893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b006b360 0x7fb3b00b4ec0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009840 tx=0x7fb3a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72b9beb836801c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:04.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.549+0000 7fb3aeffd700 1 -- 172.21.15.97:0/2047288893 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4011830 con 0x7fb3b006b360 2023-07-06T03:48:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.549+0000 7fb3aeffd700 1 -- 172.21.15.97:0/2047288893 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a4011990 con 0x7fb3b006b360 2023-07-06T03:48:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.549+0000 7fb3aeffd700 1 -- 172.21.15.97:0/2047288893 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4011d20 con 0x7fb3b006b360 2023-07-06T03:48:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.549+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2047288893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b006b360 msgr2=0x7fb3b00b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.549+0000 7fb3b63ff700 1 --2- 172.21.15.97:0/2047288893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b006b360 0x7fb3b00b4ec0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009840 tx=0x7fb3a4009b50 comp rx=0 tx=0).stop 2023-07-06T03:48:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.549+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2047288893 shutdown_connections 2023-07-06T03:48:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.549+0000 7fb3b63ff700 1 --2- 172.21.15.97:0/2047288893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b006b360 0x7fb3b00b4ec0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.549+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2047288893 >> 172.21.15.97:0/2047288893 conn(0x7fb3b006a6c0 msgr2=0x7fb3b006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:04.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.550+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2047288893 shutdown_connections 2023-07-06T03:48:04.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.550+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2047288893 wait complete. 2023-07-06T03:48:04.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.552+0000 7fb3b63ff700 1 Processor -- start 2023-07-06T03:48:04.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.552+0000 7fb3b63ff700 1 -- start start 2023-07-06T03:48:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.553+0000 7fb3b63ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b006b360 0x7fb3b00c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.553+0000 7fb3b63ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00d2210 0x7fb3b00cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.553+0000 7fb3b63ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b015c610 0x7fb3b00c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.553+0000 7fb3b63ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00cb550 con 0x7fb3b00d2210 2023-07-06T03:48:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.553+0000 7fb3b63ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00cb830 con 0x7fb3b006b360 2023-07-06T03:48:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.553+0000 7fb3b63ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00cbb10 con 0x7fb3b015c610 2023-07-06T03:48:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3af7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00d2210 0x7fb3b00cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3af7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00d2210 0x7fb3b00cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56450/0 (socket says 172.21.15.97:56450) 2023-07-06T03:48:04.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3af7fe700 1 -- 172.21.15.97:0/2683252445 learned_addr learned my addr 172.21.15.97:0/2683252445 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:04.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3affff700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b006b360 0x7fb3b00c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3b499c700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b015c610 0x7fb3b00c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3af7fe700 1 -- 172.21.15.97:0/2683252445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b015c610 msgr2=0x7fb3b00c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3af7fe700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b015c610 0x7fb3b00c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3af7fe700 1 -- 172.21.15.97:0/2683252445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b006b360 msgr2=0x7fb3b00c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3af7fe700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b006b360 0x7fb3b00c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3af7fe700 1 -- 172.21.15.97:0/2683252445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a40094a0 con 0x7fb3b00d2210 2023-07-06T03:48:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3af7fe700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00d2210 0x7fb3b00cae50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000c00 tx=0x7fb3a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3acff9700 1 -- 172.21.15.97:0/2683252445 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40036a0 con 0x7fb3b00d2210 2023-07-06T03:48:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.554+0000 7fb3acff9700 1 -- 172.21.15.97:0/2683252445 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a401e070 con 0x7fb3b00d2210 2023-07-06T03:48:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.555+0000 7fb3acff9700 1 -- 172.21.15.97:0/2683252445 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4003a40 con 0x7fb3b00d2210 2023-07-06T03:48:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.555+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2683252445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b00cbdf0 con 0x7fb3b00d2210 2023-07-06T03:48:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.555+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2683252445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b00cc5f0 con 0x7fb3b00d2210 2023-07-06T03:48:04.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.556+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2683252445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b00cc310 con 0x7fb3b00d2210 2023-07-06T03:48:04.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.556+0000 7fb3acff9700 1 -- 172.21.15.97:0/2683252445 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb3a4003ba0 con 0x7fb3b00d2210 2023-07-06T03:48:04.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.557+0000 7fb3acff9700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39c06ace0 0x7fb39c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.557+0000 7fb3acff9700 1 -- 172.21.15.97:0/2683252445 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7fb3a408a5b0 con 0x7fb3b00d2210 2023-07-06T03:48:04.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.558+0000 7fb3affff700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39c06ace0 0x7fb39c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.558+0000 7fb3affff700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39c06ace0 0x7fb39c06d190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0007fd0 tx=0x7fb3a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.562+0000 7fb3acff9700 1 -- 172.21.15.97:0/2683252445 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb3b00cc310 con 0x7fb3b00d2210 2023-07-06T03:48:04.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.754+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2683252445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3b00cc310 con 0x7fb3b00d2210 2023-07-06T03:48:04.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.755+0000 7fb3acff9700 1 -- 172.21.15.97:0/2683252445 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3b00cc310 con 0x7fb3b00d2210 2023-07-06T03:48:04.758 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:48:04.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2683252445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39c06ace0 msgr2=0x7fb39c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39c06ace0 0x7fb39c06d190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0007fd0 tx=0x7fb3a0009040 comp rx=0 tx=0).stop 2023-07-06T03:48:04.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2683252445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00d2210 msgr2=0x7fb3b00cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00d2210 0x7fb3b00cae50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000c00 tx=0x7fb3a4004750 comp rx=0 tx=0).stop 2023-07-06T03:48:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2683252445 shutdown_connections 2023-07-06T03:48:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39c06ace0 0x7fb39c06d190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b015c610 0x7fb3b00c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b006b360 0x7fb3b00c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 --2- 172.21.15.97:0/2683252445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00d2210 0x7fb3b00cae50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2683252445 >> 172.21.15.97:0/2683252445 conn(0x7fb3b006a6c0 msgr2=0x7fb3b00d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2683252445 shutdown_connections 2023-07-06T03:48:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.757+0000 7fb3b63ff700 1 -- 172.21.15.97:0/2683252445 wait complete. 2023-07-06T03:48:04.778 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:48:04.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.973+0000 7f5d699ff700 1 Processor -- start 2023-07-06T03:48:04.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.973+0000 7f5d699ff700 1 -- start start 2023-07-06T03:48:04.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.974+0000 7f5d699ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d6415c610 0x7f5d640b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.974+0000 7f5d699ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5d640d2210 0x7f5d640d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.974+0000 7f5d699ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d640cff90 con 0x7f5d6406b360 2023-07-06T03:48:04.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.974+0000 7f5d699ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d640d0780 con 0x7f5d640d2210 2023-07-06T03:48:04.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.974+0000 7f5d699ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6415c9e0 con 0x7f5d6415c610 2023-07-06T03:48:04.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.975+0000 7f5d627fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5d6406b360 0x7f5d640b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58498/0 (socket says 172.21.15.97:58498) 2023-07-06T03:48:04.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.975+0000 7f5d627fc700 1 -- 172.21.15.97:0/906067900 learned_addr learned my addr 172.21.15.97:0/906067900 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:04.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.975+0000 7f5d62ffd700 1 --2- 172.21.15.97:0/906067900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d6415c610 0x7f5d640b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.975+0000 7f5d637fe700 1 --2- 172.21.15.97:0/906067900 >> v2:172.21.15.146:3300/0 conn(0x7f5d640d2210 0x7f5d640d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.975+0000 7f5d61ffb700 1 -- 172.21.15.97:0/906067900 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631586079 0 0) 0x7f5d640cff90 con 0x7f5d6406b360 2023-07-06T03:48:04.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.975+0000 7f5d61ffb700 1 -- 172.21.15.97:0/906067900 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d4c0035b0 con 0x7f5d6406b360 2023-07-06T03:48:04.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.975+0000 7f5d62ffd700 1 -- 172.21.15.97:0/906067900 >> v2:172.21.15.146:3300/0 conn(0x7f5d640d2210 msgr2=0x7f5d640d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:04.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.975+0000 7f5d62ffd700 1 --2- 172.21.15.97:0/906067900 >> v2:172.21.15.146:3300/0 conn(0x7f5d640d2210 0x7f5d640d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.975+0000 7f5d62ffd700 1 -- 172.21.15.97:0/906067900 >> v1:172.21.15.97:6789/0 conn(0x7f5d6406b360 legacy=0x7f5d640b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.975+0000 7f5d61ffb700 1 -- 172.21.15.97:0/906067900 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1956119442 0 0) 0x7f5d4c0035b0 con 0x7f5d6406b360 2023-07-06T03:48:04.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.975+0000 7f5d62ffd700 1 -- 172.21.15.97:0/906067900 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d640b5650 con 0x7f5d6415c610 2023-07-06T03:48:04.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.976+0000 7f5d62ffd700 1 --2- 172.21.15.97:0/906067900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d6415c610 0x7f5d640b47b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5d54009840 tx=0x7f5d54009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f00be099776d0d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:04.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.976+0000 7f5d61ffb700 1 -- 172.21.15.97:0/906067900 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58001670 con 0x7f5d6415c610 2023-07-06T03:48:04.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.976+0000 7f5d61ffb700 1 -- 172.21.15.97:0/906067900 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d54004030 con 0x7f5d6415c610 2023-07-06T03:48:04.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.976+0000 7f5d61ffb700 1 -- 172.21.15.97:0/906067900 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d540043c0 con 0x7f5d6415c610 2023-07-06T03:48:04.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.976+0000 7f5d699ff700 1 -- 172.21.15.97:0/906067900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d6415c610 msgr2=0x7f5d640b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.976+0000 7f5d699ff700 1 --2- 172.21.15.97:0/906067900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d6415c610 0x7f5d640b47b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5d54009840 tx=0x7f5d54009b50 comp rx=0 tx=0).stop 2023-07-06T03:48:04.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.977+0000 7f5d699ff700 1 -- 172.21.15.97:0/906067900 shutdown_connections 2023-07-06T03:48:04.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.977+0000 7f5d699ff700 1 --2- 172.21.15.97:0/906067900 >> v2:172.21.15.146:3300/0 conn(0x7f5d640d2210 0x7f5d640d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.977+0000 7f5d699ff700 1 --2- 172.21.15.97:0/906067900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d6415c610 0x7f5d640b47b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.977+0000 7f5d699ff700 1 -- 172.21.15.97:0/906067900 >> 172.21.15.97:0/906067900 conn(0x7f5d6406a6c0 msgr2=0x7f5d6406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:04.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.977+0000 7f5d699ff700 1 -- 172.21.15.97:0/906067900 shutdown_connections 2023-07-06T03:48:04.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.977+0000 7f5d699ff700 1 -- 172.21.15.97:0/906067900 wait complete. 2023-07-06T03:48:04.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.978+0000 7f5d699ff700 1 Processor -- start 2023-07-06T03:48:04.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.978+0000 7f5d699ff700 1 -- start start 2023-07-06T03:48:04.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.979+0000 7f5d699ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d6406b360 0x7f5d6415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.979+0000 7f5d699ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d640d2210 0x7f5d6415e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.979+0000 7f5d699ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6415c610 0x7f5d640c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.979+0000 7f5d699ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6415d400 con 0x7f5d640d2210 2023-07-06T03:48:04.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.979+0000 7f5d699ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6415d6e0 con 0x7f5d6415c610 2023-07-06T03:48:04.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.979+0000 7f5d699ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6415d9c0 con 0x7f5d6406b360 2023-07-06T03:48:04.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d627fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d640d2210 0x7f5d6415e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d627fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d640d2210 0x7f5d6415e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56462/0 (socket says 172.21.15.97:56462) 2023-07-06T03:48:04.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d627fc700 1 -- 172.21.15.97:0/2048750779 learned_addr learned my addr 172.21.15.97:0/2048750779 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:04.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d62ffd700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d6406b360 0x7f5d6415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d637fe700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6415c610 0x7f5d640c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d627fc700 1 -- 172.21.15.97:0/2048750779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d6406b360 msgr2=0x7f5d6415cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d627fc700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d6406b360 0x7f5d6415cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d627fc700 1 -- 172.21.15.97:0/2048750779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6415c610 msgr2=0x7f5d640c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:04.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d627fc700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6415c610 0x7f5d640c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:04.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d627fc700 1 -- 172.21.15.97:0/2048750779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540094a0 con 0x7f5d640d2210 2023-07-06T03:48:04.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d627fc700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d640d2210 0x7f5d6415e760 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800c780 tx=0x7f5d5800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:04.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d4bfff700 1 -- 172.21.15.97:0/2048750779 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58004c80 con 0x7f5d640d2210 2023-07-06T03:48:04.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.980+0000 7f5d4bfff700 1 -- 172.21.15.97:0/2048750779 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58004de0 con 0x7f5d640d2210 2023-07-06T03:48:04.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.981+0000 7f5d4bfff700 1 -- 172.21.15.97:0/2048750779 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d580072d0 con 0x7f5d640d2210 2023-07-06T03:48:04.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.981+0000 7f5d699ff700 1 -- 172.21.15.97:0/2048750779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6415dca0 con 0x7f5d640d2210 2023-07-06T03:48:04.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.981+0000 7f5d699ff700 1 -- 172.21.15.97:0/2048750779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6415d060 con 0x7f5d640d2210 2023-07-06T03:48:04.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.982+0000 7f5d699ff700 1 -- 172.21.15.97:0/2048750779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6415e1c0 con 0x7f5d640d2210 2023-07-06T03:48:04.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.982+0000 7f5d4bfff700 1 -- 172.21.15.97:0/2048750779 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d58007430 con 0x7f5d640d2210 2023-07-06T03:48:04.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.983+0000 7f5d4bfff700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5006ace0 0x7f5d5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:04.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.983+0000 7f5d4bfff700 1 -- 172.21.15.97:0/2048750779 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f5d58089350 con 0x7f5d640d2210 2023-07-06T03:48:04.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.984+0000 7f5d62ffd700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5006ace0 0x7f5d5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:04.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.984+0000 7f5d62ffd700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5006ace0 0x7f5d5006d190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400a010 tx=0x7f5d54008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:04.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:04.988+0000 7f5d4bfff700 1 -- 172.21.15.97:0/2048750779 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d6415e1c0 con 0x7f5d640d2210 2023-07-06T03:48:05.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.198+0000 7f5d699ff700 1 -- 172.21.15.97:0/2048750779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5d6415e1c0 con 0x7f5d640d2210 2023-07-06T03:48:05.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.198+0000 7f5d4bfff700 1 -- 172.21.15.97:0/2048750779 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5d6415e1c0 con 0x7f5d640d2210 2023-07-06T03:48:05.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 -- 172.21.15.97:0/2048750779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5006ace0 msgr2=0x7f5d5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5006ace0 0x7f5d5006d190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400a010 tx=0x7f5d54008030 comp rx=0 tx=0).stop 2023-07-06T03:48:05.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 -- 172.21.15.97:0/2048750779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d640d2210 msgr2=0x7f5d6415e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d640d2210 0x7f5d6415e760 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800c780 tx=0x7f5d5800ca90 comp rx=0 tx=0).stop 2023-07-06T03:48:05.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 -- 172.21.15.97:0/2048750779 shutdown_connections 2023-07-06T03:48:05.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d5006ace0 0x7f5d5006d190 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6415c610 0x7f5d640c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d6406b360 0x7f5d6415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 --2- 172.21.15.97:0/2048750779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d640d2210 0x7f5d6415e760 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 -- 172.21.15.97:0/2048750779 >> 172.21.15.97:0/2048750779 conn(0x7f5d6406a6c0 msgr2=0x7f5d6415b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:05.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 -- 172.21.15.97:0/2048750779 shutdown_connections 2023-07-06T03:48:05.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.200+0000 7f5d699ff700 1 -- 172.21.15.97:0/2048750779 wait complete. 2023-07-06T03:48:05.205 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:48:05.221 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:48:05.222 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:48:05.269 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:48:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.500+0000 7fbb09dff700 1 Processor -- start 2023-07-06T03:48:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.500+0000 7fbb09dff700 1 -- start start 2023-07-06T03:48:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbb09dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbb040cd390 0x7fbb04160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbb09dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb04160dd0 0x7fbb040b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbb09dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb040cdae0 con 0x7fbb0415c520 2023-07-06T03:48:05.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbb09dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb040b6640 con 0x7fbb040cd390 2023-07-06T03:48:05.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbb09dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb040b6920 con 0x7fbb04160dd0 2023-07-06T03:48:05.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbafbfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbb040cd390 0x7fbb04160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbafbfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbb040cd390 0x7fbb04160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56478/0 (socket says 172.21.15.97:56478) 2023-07-06T03:48:05.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbafbfff700 1 -- 172.21.15.97:0/850177450 learned_addr learned my addr 172.21.15.97:0/850177450 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbb03fff700 1 --2- 172.21.15.97:0/850177450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb04160dd0 0x7fbb040b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbafbfff700 1 -- 172.21.15.97:0/850177450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb04160dd0 msgr2=0x7fbb040b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbafbfff700 1 --2- 172.21.15.97:0/850177450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb04160dd0 0x7fbb040b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbafbfff700 1 -- 172.21.15.97:0/850177450 >> v1:172.21.15.146:6789/0 conn(0x7fbb0415c520 legacy=0x7fbb04160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.502+0000 7fbafbfff700 1 -- 172.21.15.97:0/850177450 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb040b6c00 con 0x7fbb040cd390 2023-07-06T03:48:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.503+0000 7fbafbfff700 1 --2- 172.21.15.97:0/850177450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb040cd390 0x7fbb04160890 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0098f0 tx=0x7fbaec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d4891367950191 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.503+0000 7fbb02ffd700 1 -- 172.21.15.97:0/850177450 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec011830 con 0x7fbb040cd390 2023-07-06T03:48:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.503+0000 7fbb02ffd700 1 -- 172.21.15.97:0/850177450 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec011990 con 0x7fbb040cd390 2023-07-06T03:48:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.503+0000 7fbb02ffd700 1 -- 172.21.15.97:0/850177450 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec011d20 con 0x7fbb040cd390 2023-07-06T03:48:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.503+0000 7fbb09dff700 1 -- 172.21.15.97:0/850177450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb040cd390 msgr2=0x7fbb04160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.503+0000 7fbb09dff700 1 --2- 172.21.15.97:0/850177450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb040cd390 0x7fbb04160890 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0098f0 tx=0x7fbaec009c00 comp rx=0 tx=0).stop 2023-07-06T03:48:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.503+0000 7fbb09dff700 1 -- 172.21.15.97:0/850177450 shutdown_connections 2023-07-06T03:48:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.503+0000 7fbb09dff700 1 --2- 172.21.15.97:0/850177450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb04160dd0 0x7fbb040b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.503+0000 7fbb09dff700 1 --2- 172.21.15.97:0/850177450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb040cd390 0x7fbb04160890 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.503+0000 7fbb09dff700 1 -- 172.21.15.97:0/850177450 >> 172.21.15.97:0/850177450 conn(0x7fbb04071790 msgr2=0x7fbb04071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:05.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.504+0000 7fbb09dff700 1 -- 172.21.15.97:0/850177450 shutdown_connections 2023-07-06T03:48:05.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.504+0000 7fbb09dff700 1 -- 172.21.15.97:0/850177450 wait complete. 2023-07-06T03:48:05.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.505+0000 7fbb09dff700 1 Processor -- start 2023-07-06T03:48:05.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.505+0000 7fbb09dff700 1 -- start start 2023-07-06T03:48:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.506+0000 7fbb09dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb040cd390 0x7fbb040cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.506+0000 7fbb09dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415c520 0x7fbb040d3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.506+0000 7fbb09dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb04160dd0 0x7fbb0415d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.506+0000 7fbb09dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb040d3a70 con 0x7fbb04160dd0 2023-07-06T03:48:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.506+0000 7fbb09dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0415f1d0 con 0x7fbb040cd390 2023-07-06T03:48:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.506+0000 7fbb09dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0415fa30 con 0x7fbb0415c520 2023-07-06T03:48:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.506+0000 7fbb03fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb04160dd0 0x7fbb0415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb03fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb04160dd0 0x7fbb0415d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56488/0 (socket says 172.21.15.97:56488) 2023-07-06T03:48:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb03fff700 1 -- 172.21.15.97:0/1066375570 learned_addr learned my addr 172.21.15.97:0/1066375570 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb037fe700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb040cd390 0x7fbb040cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbafbfff700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415c520 0x7fbb040d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb03fff700 1 -- 172.21.15.97:0/1066375570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415c520 msgr2=0x7fbb040d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb03fff700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415c520 0x7fbb040d3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb03fff700 1 -- 172.21.15.97:0/1066375570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb040cd390 msgr2=0x7fbb040cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb03fff700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb040cd390 0x7fbb040cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb03fff700 1 -- 172.21.15.97:0/1066375570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec009550 con 0x7fbb04160dd0 2023-07-06T03:48:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbafbfff700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415c520 0x7fbb040d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb03fff700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb04160dd0 0x7fbb0415d790 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00efc0 tx=0x7fbafc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb00ff9700 1 -- 172.21.15.97:0/1066375570 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc009ba0 con 0x7fbb04160dd0 2023-07-06T03:48:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb00ff9700 1 -- 172.21.15.97:0/1066375570 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbafc00f040 con 0x7fbb04160dd0 2023-07-06T03:48:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb09dff700 1 -- 172.21.15.97:0/1066375570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0415f470 con 0x7fbb04160dd0 2023-07-06T03:48:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.507+0000 7fbb00ff9700 1 -- 172.21.15.97:0/1066375570 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc014480 con 0x7fbb04160dd0 2023-07-06T03:48:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.508+0000 7fbb09dff700 1 -- 172.21.15.97:0/1066375570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb040cab00 con 0x7fbb04160dd0 2023-07-06T03:48:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.508+0000 7fbb09dff700 1 -- 172.21.15.97:0/1066375570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0404e340 con 0x7fbb04160dd0 2023-07-06T03:48:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.509+0000 7fbb00ff9700 1 -- 172.21.15.97:0/1066375570 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbafc0145e0 con 0x7fbb04160dd0 2023-07-06T03:48:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.510+0000 7fbb00ff9700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbae406ad10 0x7fbae406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.510+0000 7fbb00ff9700 1 -- 172.21.15.97:0/1066375570 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7fbafc089ef0 con 0x7fbb04160dd0 2023-07-06T03:48:05.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.511+0000 7fbb037fe700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbae406ad10 0x7fbae406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.511+0000 7fbb037fe700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbae406ad10 0x7fbae406d1c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4000f60 tx=0x7fbaf4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:05.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.514+0000 7fbb00ff9700 1 -- 172.21.15.97:0/1066375570 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbafc054df0 con 0x7fbb04160dd0 2023-07-06T03:48:05.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.708+0000 7fbb09dff700 1 -- 172.21.15.97:0/1066375570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb040b4f40 con 0x7fbb04160dd0 2023-07-06T03:48:05.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.709+0000 7fbb00ff9700 1 -- 172.21.15.97:0/1066375570 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18524 (secure 0 0 0) 0x7fbafc018070 con 0x7fbb04160dd0 2023-07-06T03:48:05.712 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:05.712 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":49,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:47:52.725520+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:47:33.695348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-06T03:47:35.132699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":2,"to":5}]},{"pgid":"8.1a","mappings":[{"from":2,"to":1}]},{"pgid":"8.38","mappings":[{"from":6,"to":5}]},{"pgid":"8.3d","mappings":[{"from":6,"to":0},{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:48:05.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.711+0000 7fbb09dff700 1 -- 172.21.15.97:0/1066375570 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbae406ad10 msgr2=0x7fbae406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.711+0000 7fbb09dff700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbae406ad10 0x7fbae406d1c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4000f60 tx=0x7fbaf4009040 comp rx=0 tx=0).stop 2023-07-06T03:48:05.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.711+0000 7fbb09dff700 1 -- 172.21.15.97:0/1066375570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb04160dd0 msgr2=0x7fbb0415d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.711+0000 7fbb09dff700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb04160dd0 0x7fbb0415d790 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00efc0 tx=0x7fbafc00c3a0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.711+0000 7fbb09dff700 1 -- 172.21.15.97:0/1066375570 shutdown_connections 2023-07-06T03:48:05.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.711+0000 7fbb09dff700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbae406ad10 0x7fbae406d1c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.711+0000 7fbb09dff700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb04160dd0 0x7fbb0415d790 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.711+0000 7fbb09dff700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb040cd390 0x7fbb040cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.711+0000 7fbb09dff700 1 --2- 172.21.15.97:0/1066375570 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb0415c520 0x7fbb040d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.711+0000 7fbb09dff700 1 -- 172.21.15.97:0/1066375570 >> 172.21.15.97:0/1066375570 conn(0x7fbb04071790 msgr2=0x7fbb04071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:05.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.712+0000 7fbb09dff700 1 -- 172.21.15.97:0/1066375570 shutdown_connections 2023-07-06T03:48:05.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.712+0000 7fbb09dff700 1 -- 172.21.15.97:0/1066375570 wait complete. 2023-07-06T03:48:05.732 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:47:33.695348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:47:35.132699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:48:05.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:48:05.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.928+0000 7f2b669ff700 1 Processor -- start 2023-07-06T03:48:05.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.928+0000 7f2b669ff700 1 -- start start 2023-07-06T03:48:05.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.929+0000 7f2b669ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b6015c610 0x7f2b600b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.929+0000 7f2b669ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2b6006b360 0x7f2b600b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.929+0000 7f2b669ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b600cff90 con 0x7f2b600d2210 2023-07-06T03:48:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.929+0000 7f2b669ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b600d0780 con 0x7f2b6015c610 2023-07-06T03:48:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.929+0000 7f2b669ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6015c9e0 con 0x7f2b6006b360 2023-07-06T03:48:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.930+0000 7f2b64f9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2b600d2210 0x7f2b600d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58506/0 (socket says 172.21.15.97:58506) 2023-07-06T03:48:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.930+0000 7f2b64f9c700 1 -- 172.21.15.97:0/3701128578 learned_addr learned my addr 172.21.15.97:0/3701128578 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.930+0000 7f2b5ffff700 1 --2- 172.21.15.97:0/3701128578 >> v2:172.21.15.97:3300/0 conn(0x7f2b6015c610 0x7f2b600b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.930+0000 7f2b57fff700 1 --2- 172.21.15.97:0/3701128578 >> v2:172.21.15.146:3300/0 conn(0x7f2b6006b360 0x7f2b600b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.930+0000 7f2b5f7fe700 1 -- 172.21.15.97:0/3701128578 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013111298 0 0) 0x7f2b600cff90 con 0x7f2b600d2210 2023-07-06T03:48:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.930+0000 7f2b5f7fe700 1 -- 172.21.15.97:0/3701128578 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b4c0035b0 con 0x7f2b600d2210 2023-07-06T03:48:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.930+0000 7f2b5ffff700 1 -- 172.21.15.97:0/3701128578 >> v2:172.21.15.146:3300/0 conn(0x7f2b6006b360 msgr2=0x7f2b600b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.930+0000 7f2b5ffff700 1 --2- 172.21.15.97:0/3701128578 >> v2:172.21.15.146:3300/0 conn(0x7f2b6006b360 0x7f2b600b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.930+0000 7f2b5ffff700 1 -- 172.21.15.97:0/3701128578 >> v1:172.21.15.97:6789/0 conn(0x7f2b600d2210 legacy=0x7f2b600d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.930+0000 7f2b5ffff700 1 -- 172.21.15.97:0/3701128578 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b600b5650 con 0x7f2b6015c610 2023-07-06T03:48:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.930+0000 7f2b5ffff700 1 --2- 172.21.15.97:0/3701128578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015c610 0x7f2b600b47b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2b6006a590 tx=0x7f2b50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57da28c107ebeb80 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:05.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.931+0000 7f2b5f7fe700 1 -- 172.21.15.97:0/3701128578 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50011830 con 0x7f2b6015c610 2023-07-06T03:48:05.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.931+0000 7f2b5f7fe700 1 -- 172.21.15.97:0/3701128578 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b50011990 con 0x7f2b6015c610 2023-07-06T03:48:05.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.931+0000 7f2b5f7fe700 1 -- 172.21.15.97:0/3701128578 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50011d20 con 0x7f2b6015c610 2023-07-06T03:48:05.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.931+0000 7f2b669ff700 1 -- 172.21.15.97:0/3701128578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015c610 msgr2=0x7f2b600b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.931+0000 7f2b669ff700 1 --2- 172.21.15.97:0/3701128578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015c610 0x7f2b600b47b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2b6006a590 tx=0x7f2b50009b50 comp rx=0 tx=0).stop 2023-07-06T03:48:05.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.931+0000 7f2b669ff700 1 -- 172.21.15.97:0/3701128578 shutdown_connections 2023-07-06T03:48:05.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.931+0000 7f2b669ff700 1 --2- 172.21.15.97:0/3701128578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015c610 0x7f2b600b47b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.931+0000 7f2b669ff700 1 --2- 172.21.15.97:0/3701128578 >> v2:172.21.15.146:3300/0 conn(0x7f2b6006b360 0x7f2b600b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.931+0000 7f2b669ff700 1 -- 172.21.15.97:0/3701128578 >> 172.21.15.97:0/3701128578 conn(0x7f2b6006a6c0 msgr2=0x7f2b6006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:05.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.932+0000 7f2b669ff700 1 -- 172.21.15.97:0/3701128578 shutdown_connections 2023-07-06T03:48:05.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.932+0000 7f2b669ff700 1 -- 172.21.15.97:0/3701128578 wait complete. 2023-07-06T03:48:05.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.934+0000 7f2b669ff700 1 Processor -- start 2023-07-06T03:48:05.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.934+0000 7f2b669ff700 1 -- start start 2023-07-06T03:48:05.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.935+0000 7f2b669ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b6006b360 0x7f2b600cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.935+0000 7f2b669ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b600d2210 0x7f2b600ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.935+0000 7f2b669ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015c610 0x7f2b601f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b669ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60147eb0 con 0x7f2b6015c610 2023-07-06T03:48:05.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b669ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b600cad50 con 0x7f2b600d2210 2023-07-06T03:48:05.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b669ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b600cb870 con 0x7f2b6006b360 2023-07-06T03:48:05.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b64f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015c610 0x7f2b601f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b64f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015c610 0x7f2b601f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56498/0 (socket says 172.21.15.97:56498) 2023-07-06T03:48:05.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b64f9c700 1 -- 172.21.15.97:0/3002438122 learned_addr learned my addr 172.21.15.97:0/3002438122 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:05.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b5ffff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b6006b360 0x7f2b600cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b57fff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b600d2210 0x7f2b600ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b64f9c700 1 -- 172.21.15.97:0/3002438122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b6006b360 msgr2=0x7f2b600cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b64f9c700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b6006b360 0x7f2b600cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b64f9c700 1 -- 172.21.15.97:0/3002438122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b600d2210 msgr2=0x7f2b600ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:05.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b64f9c700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b600d2210 0x7f2b600ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:05.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b64f9c700 1 -- 172.21.15.97:0/3002438122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b500094a0 con 0x7f2b6015c610 2023-07-06T03:48:05.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b57fff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b600d2210 0x7f2b600ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:05.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b5ffff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b6006b360 0x7f2b600cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:05.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.936+0000 7f2b64f9c700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015c610 0x7f2b601f4620 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800cb70 tx=0x7f2b5800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:05.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.937+0000 7f2b5d7fa700 1 -- 172.21.15.97:0/3002438122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5800ab10 con 0x7f2b6015c610 2023-07-06T03:48:05.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.937+0000 7f2b5d7fa700 1 -- 172.21.15.97:0/3002438122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5800ac70 con 0x7f2b6015c610 2023-07-06T03:48:05.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.937+0000 7f2b669ff700 1 -- 172.21.15.97:0/3002438122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b600cafd0 con 0x7f2b6015c610 2023-07-06T03:48:05.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.937+0000 7f2b5d7fa700 1 -- 172.21.15.97:0/3002438122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58012400 con 0x7f2b6015c610 2023-07-06T03:48:05.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.937+0000 7f2b669ff700 1 -- 172.21.15.97:0/3002438122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b600cb590 con 0x7f2b6015c610 2023-07-06T03:48:05.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.938+0000 7f2b669ff700 1 -- 172.21.15.97:0/3002438122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b60148060 con 0x7f2b6015c610 2023-07-06T03:48:05.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.939+0000 7f2b5d7fa700 1 -- 172.21.15.97:0/3002438122 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b5801b3b0 con 0x7f2b6015c610 2023-07-06T03:48:05.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.940+0000 7f2b5d7fa700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2b4006ace0 0x7f2b4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:05.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.940+0000 7f2b5d7fa700 1 -- 172.21.15.97:0/3002438122 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f2b580892e0 con 0x7f2b6015c610 2023-07-06T03:48:05.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.940+0000 7f2b5ffff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2b4006ace0 0x7f2b4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:05.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.941+0000 7f2b5ffff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2b4006ace0 0x7f2b4006d190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2b50000c00 tx=0x7f2b5001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:05.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:05.944+0000 7f2b5d7fa700 1 -- 172.21.15.97:0/3002438122 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2b60148060 con 0x7f2b6015c610 2023-07-06T03:48:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.138+0000 7f2b669ff700 1 -- 172.21.15.97:0/3002438122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b60148060 con 0x7f2b6015c610 2023-07-06T03:48:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.139+0000 7f2b5d7fa700 1 -- 172.21.15.97:0/3002438122 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b60148060 con 0x7f2b6015c610 2023-07-06T03:48:06.142 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:48:06.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 -- 172.21.15.97:0/3002438122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2b4006ace0 msgr2=0x7f2b4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2b4006ace0 0x7f2b4006d190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2b50000c00 tx=0x7f2b5001c040 comp rx=0 tx=0).stop 2023-07-06T03:48:06.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 -- 172.21.15.97:0/3002438122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015c610 msgr2=0x7f2b601f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015c610 0x7f2b601f4620 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800cb70 tx=0x7f2b5800cf30 comp rx=0 tx=0).stop 2023-07-06T03:48:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 -- 172.21.15.97:0/3002438122 shutdown_connections 2023-07-06T03:48:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2b4006ace0 0x7f2b4006d190 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015c610 0x7f2b601f4620 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2b6006b360 0x7f2b600cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 --2- 172.21.15.97:0/3002438122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b600d2210 0x7f2b600ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 -- 172.21.15.97:0/3002438122 >> 172.21.15.97:0/3002438122 conn(0x7f2b6006a6c0 msgr2=0x7f2b6015aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 -- 172.21.15.97:0/3002438122 shutdown_connections 2023-07-06T03:48:06.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.141+0000 7f2b669ff700 1 -- 172.21.15.97:0/3002438122 wait complete. 2023-07-06T03:48:06.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:48:06.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.355+0000 7ff8cffff700 1 Processor -- start 2023-07-06T03:48:06.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.355+0000 7ff8cffff700 1 -- start start 2023-07-06T03:48:06.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.356+0000 7ff8cffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 0x7ff8c80b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:06.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.356+0000 7ff8cffff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c80cff90 con 0x7ff8c806b360 2023-07-06T03:48:06.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.356+0000 7ff8cffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c80b5650 con 0x7ff8c815c610 2023-07-06T03:48:06.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.356+0000 7ff8cffff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80d0780 con 0x7ff8c80d2210 2023-07-06T03:48:06.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.356+0000 7ff8cd59a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff8c806b360 0x7ff8c80d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58518/0 (socket says 172.21.15.97:58518) 2023-07-06T03:48:06.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.356+0000 7ff8cd59a700 1 -- 172.21.15.97:0/4037136063 learned_addr learned my addr 172.21.15.97:0/4037136063 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:06.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.356+0000 7ff8ce59c700 1 --2- 172.21.15.97:0/4037136063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 0x7ff8c80b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:06.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.357+0000 7ff8ccd99700 1 -- 172.21.15.97:0/4037136063 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995587793 0 0) 0x7ff8c80cff90 con 0x7ff8c806b360 2023-07-06T03:48:06.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.357+0000 7ff8ccd99700 1 -- 172.21.15.97:0/4037136063 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b40035b0 con 0x7ff8c806b360 2023-07-06T03:48:06.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.357+0000 7ff8ccd99700 1 -- 172.21.15.97:0/4037136063 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061639483 0 0) 0x7ff8c80b5650 con 0x7ff8c815c610 2023-07-06T03:48:06.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.357+0000 7ff8ccd99700 1 -- 172.21.15.97:0/4037136063 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c80cff90 con 0x7ff8c815c610 2023-07-06T03:48:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.357+0000 7ff8ce59c700 1 -- 172.21.15.97:0/4037136063 >> v1:172.21.15.146:6789/0 conn(0x7ff8c815c610 legacy=0x7ff8c80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.357+0000 7ff8ce59c700 1 -- 172.21.15.97:0/4037136063 >> v1:172.21.15.97:6789/0 conn(0x7ff8c806b360 legacy=0x7ff8c80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.357+0000 7ff8ce59c700 1 -- 172.21.15.97:0/4037136063 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c80b5400 con 0x7ff8c80d2210 2023-07-06T03:48:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.357+0000 7ff8ce59c700 1 --2- 172.21.15.97:0/4037136063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 0x7ff8c80b4ec0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009dc0 tx=0x7ff8c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7773e7e9572f48c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.358+0000 7ff8ccd99700 1 -- 172.21.15.97:0/4037136063 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000e070 con 0x7ff8c80d2210 2023-07-06T03:48:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.358+0000 7ff8ccd99700 1 -- 172.21.15.97:0/4037136063 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c000bb20 con 0x7ff8c80d2210 2023-07-06T03:48:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.358+0000 7ff8ccd99700 1 -- 172.21.15.97:0/4037136063 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0012430 con 0x7ff8c80d2210 2023-07-06T03:48:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.358+0000 7ff8cffff700 1 -- 172.21.15.97:0/4037136063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 msgr2=0x7ff8c80b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.358+0000 7ff8cffff700 1 --2- 172.21.15.97:0/4037136063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 0x7ff8c80b4ec0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009dc0 tx=0x7ff8c000b330 comp rx=0 tx=0).stop 2023-07-06T03:48:06.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.358+0000 7ff8cffff700 1 -- 172.21.15.97:0/4037136063 shutdown_connections 2023-07-06T03:48:06.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.358+0000 7ff8cffff700 1 --2- 172.21.15.97:0/4037136063 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 0x7ff8c80b4ec0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.358+0000 7ff8cffff700 1 -- 172.21.15.97:0/4037136063 >> 172.21.15.97:0/4037136063 conn(0x7ff8c806a6c0 msgr2=0x7ff8c806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:06.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.358+0000 7ff8cffff700 1 -- 172.21.15.97:0/4037136063 shutdown_connections 2023-07-06T03:48:06.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.358+0000 7ff8cffff700 1 -- 172.21.15.97:0/4037136063 wait complete. 2023-07-06T03:48:06.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.360+0000 7ff8cffff700 1 Processor -- start 2023-07-06T03:48:06.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.360+0000 7ff8cffff700 1 -- start start 2023-07-06T03:48:06.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.361+0000 7ff8cffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c806b360 0x7ff8c80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:06.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.361+0000 7ff8cffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 0x7ff8c814d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:06.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.361+0000 7ff8cffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8c815c610 0x7ff8c80cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:06.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.361+0000 7ff8cffff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80cc250 con 0x7ff8c815c610 2023-07-06T03:48:06.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.361+0000 7ff8cffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80ccc20 con 0x7ff8c806b360 2023-07-06T03:48:06.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.361+0000 7ff8cffff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80cb640 con 0x7ff8c80d2210 2023-07-06T03:48:06.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.361+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8c815c610 0x7ff8c80cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:06.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.361+0000 7ff8ce59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8c815c610 0x7ff8c80cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56502/0 (socket says 172.21.15.97:56502) 2023-07-06T03:48:06.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.361+0000 7ff8ce59c700 1 -- 172.21.15.97:0/29804470 learned_addr learned my addr 172.21.15.97:0/29804470 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:06.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8cdd9b700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c806b360 0x7ff8c80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8cd59a700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 0x7ff8c814d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8ce59c700 1 -- 172.21.15.97:0/29804470 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 msgr2=0x7ff8c814d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8ce59c700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 0x7ff8c814d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8ce59c700 1 -- 172.21.15.97:0/29804470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c806b360 msgr2=0x7ff8c80cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8ce59c700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c806b360 0x7ff8c80cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8ce59c700 1 -- 172.21.15.97:0/29804470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c0009a70 con 0x7ff8c815c610 2023-07-06T03:48:06.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8cdd9b700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c806b360 0x7ff8c80cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:06.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8cd59a700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 0x7ff8c814d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:06.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8ce59c700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8c815c610 0x7ff8c80cac60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff8c808b120 tx=0x7ff8c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:06.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8be7fc700 1 -- 172.21.15.97:0/29804470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0012430 con 0x7ff8c815c610 2023-07-06T03:48:06.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8be7fc700 1 -- 172.21.15.97:0/29804470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c0020070 con 0x7ff8c815c610 2023-07-06T03:48:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8cffff700 1 -- 172.21.15.97:0/29804470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c80cb920 con 0x7ff8c815c610 2023-07-06T03:48:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8be7fc700 1 -- 172.21.15.97:0/29804470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0004280 con 0x7ff8c815c610 2023-07-06T03:48:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.362+0000 7ff8cffff700 1 -- 172.21.15.97:0/29804470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c80cbee0 con 0x7ff8c815c610 2023-07-06T03:48:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.363+0000 7ff8cffff700 1 -- 172.21.15.97:0/29804470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c806ba30 con 0x7ff8c815c610 2023-07-06T03:48:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.364+0000 7ff8be7fc700 1 -- 172.21.15.97:0/29804470 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff8c00043e0 con 0x7ff8c815c610 2023-07-06T03:48:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.365+0000 7ff8be7fc700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8ac06ace0 0x7ff8ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.365+0000 7ff8be7fc700 1 -- 172.21.15.97:0/29804470 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7ff8c0022070 con 0x7ff8c815c610 2023-07-06T03:48:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.365+0000 7ff8cdd9b700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8ac06ace0 0x7ff8ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.366+0000 7ff8cdd9b700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8ac06ace0 0x7ff8ac06d190 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4001750 tx=0x7ff8c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.369+0000 7ff8be7fc700 1 -- 172.21.15.97:0/29804470 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff8c0056270 con 0x7ff8c815c610 2023-07-06T03:48:06.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.563+0000 7ff8cffff700 1 -- 172.21.15.97:0/29804470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff8c80cc3c0 con 0x7ff8c815c610 2023-07-06T03:48:06.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.564+0000 7ff8be7fc700 1 -- 172.21.15.97:0/29804470 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff8c0059890 con 0x7ff8c815c610 2023-07-06T03:48:06.567 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:48:06.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 -- 172.21.15.97:0/29804470 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8ac06ace0 msgr2=0x7ff8ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8ac06ace0 0x7ff8ac06d190 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4001750 tx=0x7ff8c4009040 comp rx=0 tx=0).stop 2023-07-06T03:48:06.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 -- 172.21.15.97:0/29804470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8c815c610 msgr2=0x7ff8c80cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8c815c610 0x7ff8c80cac60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff8c808b120 tx=0x7ff8c0004ad0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 -- 172.21.15.97:0/29804470 shutdown_connections 2023-07-06T03:48:06.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8ac06ace0 0x7ff8ac06d190 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8c815c610 0x7ff8c80cac60 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8c806b360 0x7ff8c80cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 --2- 172.21.15.97:0/29804470 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8c80d2210 0x7ff8c814d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 -- 172.21.15.97:0/29804470 >> 172.21.15.97:0/29804470 conn(0x7ff8c806a6c0 msgr2=0x7ff8c815b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:06.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 -- 172.21.15.97:0/29804470 shutdown_connections 2023-07-06T03:48:06.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.566+0000 7ff8cffff700 1 -- 172.21.15.97:0/29804470 wait complete. 2023-07-06T03:48:06.587 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:48:06.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.785+0000 7f58345ff700 1 Processor -- start 2023-07-06T03:48:06.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.785+0000 7f58345ff700 1 -- start start 2023-07-06T03:48:06.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.786+0000 7f58345ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c15c610 0x7f582c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:06.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.786+0000 7f58345ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f582c0d2210 0x7f582c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:06.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.786+0000 7f58345ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582c0cff90 con 0x7f582c06b360 2023-07-06T03:48:06.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.786+0000 7f58345ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582c0d0780 con 0x7f582c0d2210 2023-07-06T03:48:06.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.786+0000 7f58345ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c15c9e0 con 0x7f582c15c610 2023-07-06T03:48:06.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.786+0000 7f5831b9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f582c06b360 0x7f582c0b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58524/0 (socket says 172.21.15.97:58524) 2023-07-06T03:48:06.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.786+0000 7f5831b9a700 1 -- 172.21.15.97:0/3634648800 learned_addr learned my addr 172.21.15.97:0/3634648800 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.787+0000 7f5832b9c700 1 --2- 172.21.15.97:0/3634648800 >> v2:172.21.15.146:3300/0 conn(0x7f582c0d2210 0x7f582c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.787+0000 7f583239b700 1 --2- 172.21.15.97:0/3634648800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c15c610 0x7f582c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:06.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.787+0000 7f5831399700 1 -- 172.21.15.97:0/3634648800 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976165738 0 0) 0x7f582c0cff90 con 0x7f582c06b360 2023-07-06T03:48:06.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.787+0000 7f5831399700 1 -- 172.21.15.97:0/3634648800 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58180035b0 con 0x7f582c06b360 2023-07-06T03:48:06.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.787+0000 7f5832b9c700 1 -- 172.21.15.97:0/3634648800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c15c610 msgr2=0x7f582c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:06.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.787+0000 7f5832b9c700 1 --2- 172.21.15.97:0/3634648800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c15c610 0x7f582c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.787+0000 7f5832b9c700 1 -- 172.21.15.97:0/3634648800 >> v1:172.21.15.97:6789/0 conn(0x7f582c06b360 legacy=0x7f582c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.787+0000 7f5831399700 1 -- 172.21.15.97:0/3634648800 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2058747851 0 0) 0x7f58180035b0 con 0x7f582c06b360 2023-07-06T03:48:06.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.787+0000 7f5832b9c700 1 -- 172.21.15.97:0/3634648800 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c0b5650 con 0x7f582c0d2210 2023-07-06T03:48:06.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.787+0000 7f5832b9c700 1 --2- 172.21.15.97:0/3634648800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c0d2210 0x7f582c0d1310 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5824009dc0 tx=0x7f582400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a5ea4ceedb3ea50 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:06.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.788+0000 7f5831399700 1 -- 172.21.15.97:0/3634648800 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582400e070 con 0x7f582c0d2210 2023-07-06T03:48:06.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.788+0000 7f5831399700 1 -- 172.21.15.97:0/3634648800 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5824003a20 con 0x7f582c0d2210 2023-07-06T03:48:06.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.788+0000 7f5831399700 1 -- 172.21.15.97:0/3634648800 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c001670 con 0x7f582c0d2210 2023-07-06T03:48:06.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.788+0000 7f58345ff700 1 -- 172.21.15.97:0/3634648800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c0d2210 msgr2=0x7f582c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.788+0000 7f58345ff700 1 --2- 172.21.15.97:0/3634648800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c0d2210 0x7f582c0d1310 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5824009dc0 tx=0x7f582400b330 comp rx=0 tx=0).stop 2023-07-06T03:48:06.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.788+0000 7f58345ff700 1 -- 172.21.15.97:0/3634648800 shutdown_connections 2023-07-06T03:48:06.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.788+0000 7f58345ff700 1 --2- 172.21.15.97:0/3634648800 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c0d2210 0x7f582c0d1310 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.788+0000 7f58345ff700 1 --2- 172.21.15.97:0/3634648800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c15c610 0x7f582c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.788+0000 7f58345ff700 1 -- 172.21.15.97:0/3634648800 >> 172.21.15.97:0/3634648800 conn(0x7f582c06a6c0 msgr2=0x7f582c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:06.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.788+0000 7f58345ff700 1 -- 172.21.15.97:0/3634648800 shutdown_connections 2023-07-06T03:48:06.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.788+0000 7f58345ff700 1 -- 172.21.15.97:0/3634648800 wait complete. 2023-07-06T03:48:06.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.790+0000 7f58345ff700 1 Processor -- start 2023-07-06T03:48:06.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.790+0000 7f58345ff700 1 -- start start 2023-07-06T03:48:06.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.791+0000 7f58345ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c06b360 0x7f582c14fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:06.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.791+0000 7f58345ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c0d2210 0x7f582c0b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:06.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.791+0000 7f58345ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c15c610 0x7f582c0b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:06.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.791+0000 7f58345ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0cc390 con 0x7f582c15c610 2023-07-06T03:48:06.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.791+0000 7f58345ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0cc5e0 con 0x7f582c06b360 2023-07-06T03:48:06.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.791+0000 7f58345ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0cc8c0 con 0x7f582c0d2210 2023-07-06T03:48:06.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.791+0000 7f5832b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c15c610 0x7f582c0b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:06.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.791+0000 7f5832b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c15c610 0x7f582c0b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56510/0 (socket says 172.21.15.97:56510) 2023-07-06T03:48:06.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.791+0000 7f5832b9c700 1 -- 172.21.15.97:0/18010977 learned_addr learned my addr 172.21.15.97:0/18010977 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:06.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f583239b700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c06b360 0x7f582c14fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:06.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f5831b9a700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c0d2210 0x7f582c0b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:06.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f5832b9c700 1 -- 172.21.15.97:0/18010977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c0d2210 msgr2=0x7f582c0b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f5832b9c700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c0d2210 0x7f582c0b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f5832b9c700 1 -- 172.21.15.97:0/18010977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c06b360 msgr2=0x7f582c14fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:06.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f5832b9c700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c06b360 0x7f582c14fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:06.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f5832b9c700 1 -- 172.21.15.97:0/18010977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5824009a70 con 0x7f582c15c610 2023-07-06T03:48:06.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f5832b9c700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c15c610 0x7f582c0b68b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5824015040 tx=0x7f5824004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:06.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f5822ffd700 1 -- 172.21.15.97:0/18010977 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824004540 con 0x7f582c15c610 2023-07-06T03:48:06.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f5822ffd700 1 -- 172.21.15.97:0/18010977 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58240046e0 con 0x7f582c15c610 2023-07-06T03:48:06.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f58345ff700 1 -- 172.21.15.97:0/18010977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c0b2830 con 0x7f582c15c610 2023-07-06T03:48:06.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f5822ffd700 1 -- 172.21.15.97:0/18010977 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824009410 con 0x7f582c15c610 2023-07-06T03:48:06.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.792+0000 7f58345ff700 1 -- 172.21.15.97:0/18010977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c150060 con 0x7f582c15c610 2023-07-06T03:48:06.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.793+0000 7f58345ff700 1 -- 172.21.15.97:0/18010977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c04e340 con 0x7f582c15c610 2023-07-06T03:48:06.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.794+0000 7f5822ffd700 1 -- 172.21.15.97:0/18010977 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5824012450 con 0x7f582c15c610 2023-07-06T03:48:06.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.795+0000 7f5822ffd700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 0x7f581006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:06.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.795+0000 7f5822ffd700 1 -- 172.21.15.97:0/18010977 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f582408abe0 con 0x7f582c15c610 2023-07-06T03:48:06.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.796+0000 7f583239b700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 0x7f581006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:06.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.796+0000 7f583239b700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 0x7f581006d190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5828009b40 tx=0x7f5828009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:06.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.799+0000 7f5822ffd700 1 -- 172.21.15.97:0/18010977 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5824059610 con 0x7f582c15c610 2023-07-06T03:48:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.996+0000 7f58345ff700 1 -- 172.21.15.97:0/18010977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f582c06b9d0 con 0x7f582c15c610 2023-07-06T03:48:06.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.996+0000 7f5822ffd700 1 -- 172.21.15.97:0/18010977 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5824076e60 con 0x7f582c15c610 2023-07-06T03:48:06.999 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:48:07.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.998+0000 7f58345ff700 1 -- 172.21.15.97:0/18010977 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 msgr2=0x7f581006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.998+0000 7f58345ff700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 0x7f581006d190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5828009b40 tx=0x7f5828009210 comp rx=0 tx=0).stop 2023-07-06T03:48:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.998+0000 7f58345ff700 1 -- 172.21.15.97:0/18010977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c15c610 msgr2=0x7f582c0b68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.998+0000 7f58345ff700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c15c610 0x7f582c0b68b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5824015040 tx=0x7f5824004ad0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.999+0000 7f58345ff700 1 -- 172.21.15.97:0/18010977 shutdown_connections 2023-07-06T03:48:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.999+0000 7f58345ff700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 0x7f581006d190 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.999+0000 7f58345ff700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c15c610 0x7f582c0b68b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.999+0000 7f58345ff700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c06b360 0x7f582c14fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.999+0000 7f58345ff700 1 --2- 172.21.15.97:0/18010977 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c0d2210 0x7f582c0b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.999+0000 7f58345ff700 1 -- 172.21.15.97:0/18010977 >> 172.21.15.97:0/18010977 conn(0x7f582c06a6c0 msgr2=0x7f582c15aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:07.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.999+0000 7f58345ff700 1 -- 172.21.15.97:0/18010977 shutdown_connections 2023-07-06T03:48:07.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:06.999+0000 7f58345ff700 1 -- 172.21.15.97:0/18010977 wait complete. 2023-07-06T03:48:07.019 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:48:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.214+0000 7f97c87ff700 1 Processor -- start 2023-07-06T03:48:07.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.214+0000 7f97c87ff700 1 -- start start 2023-07-06T03:48:07.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.215+0000 7f97c87ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c015c610 0x7f97c00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:07.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.215+0000 7f97c87ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f97c00d2210 0x7f97c00d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:07.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.216+0000 7f97c87ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c00cff90 con 0x7f97c006b360 2023-07-06T03:48:07.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.216+0000 7f97c87ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c00d0780 con 0x7f97c00d2210 2023-07-06T03:48:07.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.216+0000 7f97c87ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c015c9e0 con 0x7f97c015c610 2023-07-06T03:48:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.216+0000 7f97c5d9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f97c006b360 0x7f97c00b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58526/0 (socket says 172.21.15.97:58526) 2023-07-06T03:48:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.216+0000 7f97c5d9a700 1 -- 172.21.15.97:0/3169085658 learned_addr learned my addr 172.21.15.97:0/3169085658 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.216+0000 7f97c6d9c700 1 --2- 172.21.15.97:0/3169085658 >> v2:172.21.15.97:3300/0 conn(0x7f97c00d2210 0x7f97c00d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.216+0000 7f97c659b700 1 --2- 172.21.15.97:0/3169085658 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c015c610 0x7f97c00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.216+0000 7f97c6d9c700 1 -- 172.21.15.97:0/3169085658 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c015c610 msgr2=0x7f97c00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.216+0000 7f97c6d9c700 1 --2- 172.21.15.97:0/3169085658 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c015c610 0x7f97c00b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.216+0000 7f97c6d9c700 1 -- 172.21.15.97:0/3169085658 >> v1:172.21.15.97:6789/0 conn(0x7f97c006b360 legacy=0x7f97c00b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.216+0000 7f97c6d9c700 1 -- 172.21.15.97:0/3169085658 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c00b5650 con 0x7f97c00d2210 2023-07-06T03:48:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.217+0000 7f97c6d9c700 1 --2- 172.21.15.97:0/3169085658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c00d2210 0x7f97c00d1310 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009dc0 tx=0x7f97b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10869e7daa2c5c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:07.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.217+0000 7f97c5599700 1 -- 172.21.15.97:0/3169085658 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b800e070 con 0x7f97c00d2210 2023-07-06T03:48:07.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.217+0000 7f97c5599700 1 -- 172.21.15.97:0/3169085658 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b800bb90 con 0x7f97c00d2210 2023-07-06T03:48:07.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.217+0000 7f97c5599700 1 -- 172.21.15.97:0/3169085658 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8012430 con 0x7f97c00d2210 2023-07-06T03:48:07.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.217+0000 7f97c87ff700 1 -- 172.21.15.97:0/3169085658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c00d2210 msgr2=0x7f97c00d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.217+0000 7f97c87ff700 1 --2- 172.21.15.97:0/3169085658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c00d2210 0x7f97c00d1310 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009dc0 tx=0x7f97b800b330 comp rx=0 tx=0).stop 2023-07-06T03:48:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.217+0000 7f97c87ff700 1 -- 172.21.15.97:0/3169085658 shutdown_connections 2023-07-06T03:48:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.217+0000 7f97c87ff700 1 --2- 172.21.15.97:0/3169085658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c00d2210 0x7f97c00d1310 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.217+0000 7f97c87ff700 1 --2- 172.21.15.97:0/3169085658 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c015c610 0x7f97c00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.217+0000 7f97c87ff700 1 -- 172.21.15.97:0/3169085658 >> 172.21.15.97:0/3169085658 conn(0x7f97c006a6c0 msgr2=0x7f97c006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:07.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.217+0000 7f97c87ff700 1 -- 172.21.15.97:0/3169085658 shutdown_connections 2023-07-06T03:48:07.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.218+0000 7f97c87ff700 1 -- 172.21.15.97:0/3169085658 wait complete. 2023-07-06T03:48:07.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.219+0000 7f97c87ff700 1 Processor -- start 2023-07-06T03:48:07.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.219+0000 7f97c87ff700 1 -- start start 2023-07-06T03:48:07.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.220+0000 7f97c87ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c006b360 0x7f97c00cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:07.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.220+0000 7f97c87ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c00d2210 0x7f97c00b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:07.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.220+0000 7f97c87ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c015c610 0x7f97c00b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:07.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.220+0000 7f97c87ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c015e040 con 0x7f97c015c610 2023-07-06T03:48:07.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.220+0000 7f97c87ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00cbdb0 con 0x7f97c006b360 2023-07-06T03:48:07.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.220+0000 7f97c87ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00cb7f0 con 0x7f97c00d2210 2023-07-06T03:48:07.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.220+0000 7f97c6d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c015c610 0x7f97c00b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:07.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.220+0000 7f97c6d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c015c610 0x7f97c00b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56526/0 (socket says 172.21.15.97:56526) 2023-07-06T03:48:07.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97c6d9c700 1 -- 172.21.15.97:0/2502974302 learned_addr learned my addr 172.21.15.97:0/2502974302 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:07.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97c5d9a700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c00d2210 0x7f97c00b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:07.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97c659b700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c006b360 0x7f97c00cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:07.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97c6d9c700 1 -- 172.21.15.97:0/2502974302 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c00d2210 msgr2=0x7f97c00b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97c6d9c700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c00d2210 0x7f97c00b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97c6d9c700 1 -- 172.21.15.97:0/2502974302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c006b360 msgr2=0x7f97c00cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97c6d9c700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c006b360 0x7f97c00cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97c6d9c700 1 -- 172.21.15.97:0/2502974302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8009a70 con 0x7f97c015c610 2023-07-06T03:48:07.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97c659b700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c006b360 0x7f97c00cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:07.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97c6d9c700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c015c610 0x7f97c00b4ce0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f97b800b370 tx=0x7f97b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:07.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97b6ffd700 1 -- 172.21.15.97:0/2502974302 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8012430 con 0x7f97c015c610 2023-07-06T03:48:07.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97b6ffd700 1 -- 172.21.15.97:0/2502974302 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b8020070 con 0x7f97c015c610 2023-07-06T03:48:07.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97b6ffd700 1 -- 172.21.15.97:0/2502974302 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8004280 con 0x7f97c015c610 2023-07-06T03:48:07.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.221+0000 7f97c87ff700 1 -- 172.21.15.97:0/2502974302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c00cbad0 con 0x7f97c015c610 2023-07-06T03:48:07.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.222+0000 7f97c87ff700 1 -- 172.21.15.97:0/2502974302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c015e550 con 0x7f97c015c610 2023-07-06T03:48:07.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.223+0000 7f97c87ff700 1 -- 172.21.15.97:0/2502974302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c004e340 con 0x7f97c015c610 2023-07-06T03:48:07.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.224+0000 7f97b6ffd700 1 -- 172.21.15.97:0/2502974302 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97b80043e0 con 0x7f97c015c610 2023-07-06T03:48:07.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.224+0000 7f97b6ffd700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97a406ace0 0x7f97a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:07.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.225+0000 7f97b6ffd700 1 -- 172.21.15.97:0/2502974302 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f97b8022070 con 0x7f97c015c610 2023-07-06T03:48:07.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.225+0000 7f97c659b700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97a406ace0 0x7f97a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:07.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.225+0000 7f97c659b700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97a406ace0 0x7f97a406d190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f97bc007a50 tx=0x7f97bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:07.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.229+0000 7f97b6ffd700 1 -- 172.21.15.97:0/2502974302 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f97b8056670 con 0x7f97c015c610 2023-07-06T03:48:07.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.423+0000 7f97c87ff700 1 -- 172.21.15.97:0/2502974302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97c015a500 con 0x7f97c015c610 2023-07-06T03:48:07.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.423+0000 7f97b6ffd700 1 -- 172.21.15.97:0/2502974302 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97b8059c90 con 0x7f97c015c610 2023-07-06T03:48:07.426 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:48:07.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.425+0000 7f97c87ff700 1 -- 172.21.15.97:0/2502974302 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97a406ace0 msgr2=0x7f97a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.425+0000 7f97c87ff700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97a406ace0 0x7f97a406d190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f97bc007a50 tx=0x7f97bc008040 comp rx=0 tx=0).stop 2023-07-06T03:48:07.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.425+0000 7f97c87ff700 1 -- 172.21.15.97:0/2502974302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c015c610 msgr2=0x7f97c00b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.425+0000 7f97c87ff700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c015c610 0x7f97c00b4ce0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f97b800b370 tx=0x7f97b8004ad0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.425+0000 7f97c87ff700 1 -- 172.21.15.97:0/2502974302 shutdown_connections 2023-07-06T03:48:07.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.425+0000 7f97c87ff700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97a406ace0 0x7f97a406d190 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.425+0000 7f97c87ff700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c015c610 0x7f97c00b4ce0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.425+0000 7f97c87ff700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97c006b360 0x7f97c00cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.425+0000 7f97c87ff700 1 --2- 172.21.15.97:0/2502974302 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97c00d2210 0x7f97c00b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.425+0000 7f97c87ff700 1 -- 172.21.15.97:0/2502974302 >> 172.21.15.97:0/2502974302 conn(0x7f97c006a6c0 msgr2=0x7f97c00b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:07.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.426+0000 7f97c87ff700 1 -- 172.21.15.97:0/2502974302 shutdown_connections 2023-07-06T03:48:07.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.426+0000 7f97c87ff700 1 -- 172.21.15.97:0/2502974302 wait complete. 2023-07-06T03:48:07.446 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:48:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.638+0000 7f5ad35ff700 1 Processor -- start 2023-07-06T03:48:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.638+0000 7f5ad35ff700 1 -- start start 2023-07-06T03:48:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.639+0000 7f5ad35ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5acc154890 0x7f5acc0cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.639+0000 7f5ad35ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc15b450 con 0x7f5acc15eb90 2023-07-06T03:48:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.639+0000 7f5ad35ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc15b6e0 con 0x7f5acc15b050 2023-07-06T03:48:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.639+0000 7f5ad35ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5acc15e0b0 con 0x7f5acc154890 2023-07-06T03:48:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.640+0000 7f5ad0b9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5acc15eb90 0x7f5acc15a940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58532/0 (socket says 172.21.15.97:58532) 2023-07-06T03:48:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.640+0000 7f5ad0b9a700 1 -- 172.21.15.97:0/3251787436 learned_addr learned my addr 172.21.15.97:0/3251787436 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.640+0000 7f5ad139b700 1 --2- 172.21.15.97:0/3251787436 >> v2:172.21.15.97:3300/0 conn(0x7f5acc154890 0x7f5acc0cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.640+0000 7f5ac3fff700 1 -- 172.21.15.97:0/3251787436 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368019352 0 0) 0x7f5acc15b450 con 0x7f5acc15eb90 2023-07-06T03:48:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.640+0000 7f5ac3fff700 1 -- 172.21.15.97:0/3251787436 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ab80035b0 con 0x7f5acc15eb90 2023-07-06T03:48:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.640+0000 7f5ad139b700 1 -- 172.21.15.97:0/3251787436 >> v1:172.21.15.146:6789/0 conn(0x7f5acc15b050 legacy=0x7f5acc0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.640+0000 7f5ad139b700 1 -- 172.21.15.97:0/3251787436 >> v1:172.21.15.97:6789/0 conn(0x7f5acc15eb90 legacy=0x7f5acc15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.640+0000 7f5ad139b700 1 -- 172.21.15.97:0/3251787436 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc15e390 con 0x7f5acc154890 2023-07-06T03:48:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.641+0000 7f5ad139b700 1 --2- 172.21.15.97:0/3251787436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5acc154890 0x7f5acc0cb390 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009840 tx=0x7f5ac8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb98e02214f67e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.641+0000 7f5ac3fff700 1 -- 172.21.15.97:0/3251787436 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8011830 con 0x7f5acc154890 2023-07-06T03:48:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.641+0000 7f5ac3fff700 1 -- 172.21.15.97:0/3251787436 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac8011990 con 0x7f5acc154890 2023-07-06T03:48:07.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.641+0000 7f5ac3fff700 1 -- 172.21.15.97:0/3251787436 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8011d20 con 0x7f5acc154890 2023-07-06T03:48:07.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.641+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3251787436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5acc154890 msgr2=0x7f5acc0cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.641+0000 7f5ad35ff700 1 --2- 172.21.15.97:0/3251787436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5acc154890 0x7f5acc0cb390 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009840 tx=0x7f5ac8009b50 comp rx=0 tx=0).stop 2023-07-06T03:48:07.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.641+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3251787436 shutdown_connections 2023-07-06T03:48:07.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.641+0000 7f5ad35ff700 1 --2- 172.21.15.97:0/3251787436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5acc154890 0x7f5acc0cb390 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.641+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3251787436 >> 172.21.15.97:0/3251787436 conn(0x7f5acc06a060 msgr2=0x7f5acc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:07.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.641+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3251787436 shutdown_connections 2023-07-06T03:48:07.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.642+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3251787436 wait complete. 2023-07-06T03:48:07.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.643+0000 7f5ad35ff700 1 Processor -- start 2023-07-06T03:48:07.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.643+0000 7f5ad35ff700 1 -- start start 2023-07-06T03:48:07.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.644+0000 7f5ad35ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5acc154890 0x7f5acc145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:07.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.644+0000 7f5ad35ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5acc15b050 0x7f5acc0c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:07.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.644+0000 7f5ad35ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5acc15eb90 0x7f5acc15c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:07.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.644+0000 7f5ad35ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0c6d40 con 0x7f5acc15b050 2023-07-06T03:48:07.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.644+0000 7f5ad35ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0c7020 con 0x7f5acc154890 2023-07-06T03:48:07.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.644+0000 7f5ad35ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0c7300 con 0x7f5acc15eb90 2023-07-06T03:48:07.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad0b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5acc15b050 0x7f5acc0c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:07.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad139b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5acc154890 0x7f5acc145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:07.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad0b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5acc15b050 0x7f5acc0c63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56538/0 (socket says 172.21.15.97:56538) 2023-07-06T03:48:07.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad0b9a700 1 -- 172.21.15.97:0/3768638516 learned_addr learned my addr 172.21.15.97:0/3768638516 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:07.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad139b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5acc154890 0x7f5acc145510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:52614/0 (socket says 172.21.15.97:52614) 2023-07-06T03:48:07.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad1b9c700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5acc15eb90 0x7f5acc15c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad0b9a700 1 -- 172.21.15.97:0/3768638516 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5acc15eb90 msgr2=0x7f5acc15c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad0b9a700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5acc15eb90 0x7f5acc15c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad0b9a700 1 -- 172.21.15.97:0/3768638516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5acc154890 msgr2=0x7f5acc145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad0b9a700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5acc154890 0x7f5acc145510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad0b9a700 1 -- 172.21.15.97:0/3768638516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac80094a0 con 0x7f5acc15b050 2023-07-06T03:48:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad0b9a700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5acc15b050 0x7f5acc0c63d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5abc001eb0 tx=0x7f5abc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ac1ffb700 1 -- 172.21.15.97:0/3768638516 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc005d70 con 0x7f5acc15b050 2023-07-06T03:48:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ac1ffb700 1 -- 172.21.15.97:0/3768638516 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5abc0070d0 con 0x7f5acc15b050 2023-07-06T03:48:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ac1ffb700 1 -- 172.21.15.97:0/3768638516 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc007460 con 0x7f5acc15b050 2023-07-06T03:48:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.645+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3768638516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc158350 con 0x7f5acc15b050 2023-07-06T03:48:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.646+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3768638516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc0c75e0 con 0x7f5acc15b050 2023-07-06T03:48:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.647+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3768638516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc04e340 con 0x7f5acc15b050 2023-07-06T03:48:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.648+0000 7f5ac1ffb700 1 -- 172.21.15.97:0/3768638516 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5abc0075c0 con 0x7f5acc15b050 2023-07-06T03:48:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.648+0000 7f5ac1ffb700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ab006ace0 0x7f5ab006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.649+0000 7f5ac1ffb700 1 -- 172.21.15.97:0/3768638516 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f5abc088a50 con 0x7f5acc15b050 2023-07-06T03:48:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.649+0000 7f5ad139b700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ab006ace0 0x7f5ab006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.649+0000 7f5ad139b700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ab006ace0 0x7f5ab006d190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80090b0 tx=0x7f5ac8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.653+0000 7f5ac1ffb700 1 -- 172.21.15.97:0/3768638516 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5abc053960 con 0x7f5acc15b050 2023-07-06T03:48:07.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.863+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3768638516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5acc06bef0 con 0x7f5acc15b050 2023-07-06T03:48:07.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.864+0000 7f5ac1ffb700 1 -- 172.21.15.97:0/3768638516 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5abc056f80 con 0x7f5acc15b050 2023-07-06T03:48:07.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3768638516 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ab006ace0 msgr2=0x7f5ab006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ab006ace0 0x7f5ab006d190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80090b0 tx=0x7f5ac8008030 comp rx=0 tx=0).stop 2023-07-06T03:48:07.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3768638516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5acc15b050 msgr2=0x7f5acc0c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5acc15b050 0x7f5acc0c63d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5abc001eb0 tx=0x7f5abc00c7b0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3768638516 shutdown_connections 2023-07-06T03:48:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ab006ace0 0x7f5ab006d190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5acc15eb90 0x7f5acc15c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5acc154890 0x7f5acc145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 --2- 172.21.15.97:0/3768638516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5acc15b050 0x7f5acc0c63d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3768638516 >> 172.21.15.97:0/3768638516 conn(0x7f5acc06a060 msgr2=0x7f5acc0ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:07.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3768638516 shutdown_connections 2023-07-06T03:48:07.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:07.866+0000 7f5ad35ff700 1 -- 172.21.15.97:0/3768638516 wait complete. 2023-07-06T03:48:07.871 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:48:07.887 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:48:07.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:07.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:07.955 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[134043]: fuse finished with error 0 and tester_r 0 2023-07-06T03:48:07.959 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:48:08.041 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:48:08.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:08.042 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:48:08.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:48:08.133 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:48:08.245 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2023-07-06T03:48:08.245 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2023-07-06T03:48:08.246 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:48:08.246 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2023-07-06T03:48:08.246 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.246 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2023-07-06T03:48:08.246 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.246 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2023-07-06T03:48:08.247 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.247 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2023-07-06T03:48:08.247 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2023-07-06T03:48:08.247 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.247 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2023-07-06T03:48:08.247 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2023-07-06T03:48:08.247 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.247 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:48:08.248 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2023-07-06T03:48:08.248 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.248 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:48:08.248 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:48:08.248 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.248 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2023-07-06T03:48:08.248 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.249 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2023-07-06T03:48:08.249 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2023-07-06T03:48:08.249 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:48:08.249 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:48:08.249 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2023-07-06T03:48:08.249 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:48:08.249 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2023-07-06T03:48:08.249 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:48:08.250 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:48:08.250 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2023-07-06T03:48:08.250 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.250 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:48:08.250 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.250 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2023-07-06T03:48:08.250 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.250 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2023-07-06T03:48:08.251 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.251 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2023-07-06T03:48:08.251 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2023-07-06T03:48:08.251 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2023-07-06T03:48:08.251 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:48:08.251 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:48:08.251 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.251 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2023-07-06T03:48:08.252 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2023-07-06T03:48:08.252 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.252 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2023-07-06T03:48:08.252 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2023-07-06T03:48:08.252 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:48:08.252 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.252 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2023-07-06T03:48:08.253 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2023-07-06T03:48:08.253 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.253 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2023-07-06T03:48:08.253 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2023-07-06T03:48:08.253 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2023-07-06T03:48:08.253 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.253 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2023-07-06T03:48:08.253 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2023-07-06T03:48:08.254 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.254 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:48:08.254 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.254 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2023-07-06T03:48:08.254 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.255 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2023-07-06T03:48:08.255 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.255 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:48:08.255 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2023-07-06T03:48:08.255 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.255 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2023-07-06T03:48:08.255 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2023-07-06T03:48:08.256 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2023-07-06T03:48:08.256 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2023-07-06T03:48:08.256 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2023-07-06T03:48:08.256 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2023-07-06T03:48:08.256 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2023-07-06T03:48:08.256 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2023-07-06T03:48:08.256 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2023-07-06T03:48:08.256 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:48:08.257 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2023-07-06T03:48:08.257 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2023-07-06T03:48:08.257 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:48:08.257 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2023-07-06T03:48:08.257 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:48:08.257 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:48:08.257 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2023-07-06T03:48:08.257 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:48:08.258 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.258 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2023-07-06T03:48:08.258 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:48:08.258 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2023-07-06T03:48:08.258 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:48:08.258 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:48:08.258 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:48:08.258 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:48:08.259 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2023-07-06T03:48:08.259 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.259 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2023-07-06T03:48:08.259 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:48:08.259 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:48:08.259 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2023-07-06T03:48:08.259 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:48:08.259 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2023-07-06T03:48:08.260 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2023-07-06T03:48:08.260 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:48:08.260 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2023-07-06T03:48:08.260 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2023-07-06T03:48:08.260 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:48:08.260 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.260 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:48:08.260 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:48:08.261 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:48:08.261 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.261 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:48:08.261 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:48:08.261 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:48:08.261 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.261 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2023-07-06T03:48:08.261 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:48:08.262 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.262 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:48:08.262 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2023-07-06T03:48:08.262 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:08.262 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2023-07-06T03:48:08.262 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2023-07-06T03:48:08.262 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2023-07-06T03:48:08.262 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2023-07-06T03:48:08.263 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:48:08.263 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:48:08.263 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2023-07-06T03:48:08.263 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2023-07-06T03:48:08.263 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:48:08.263 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2023-07-06T03:48:08.263 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:48:08.263 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:48:08.264 INFO:teuthology.orchestra.run.smithi097.stdout: client mountpoint = /subdir 2023-07-06T03:48:08.264 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2023-07-06T03:48:08.264 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2023-07-06T03:48:08.264 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2023-07-06T03:48:08.264 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-07-06T03:48:08.264 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-07-06T03:48:08.265 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-07-06T03:48:08.265 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-07-06T03:48:08.265 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2023-07-06T03:48:08.265 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2023-07-06T03:48:08.265 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2023-07-06T03:48:08.265 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:48:08.266 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2023-07-06T03:48:08.266 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2023-07-06T03:48:08.266 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2023-07-06T03:48:08.266 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2023-07-06T03:48:08.266 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:48:08.266 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:48:08.266 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:48:08.267 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2023-07-06T03:48:08.267 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2023-07-06T03:48:08.267 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:48:08.267 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2023-07-06T03:48:08.268 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.268 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2023-07-06T03:48:08.268 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.268 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2023-07-06T03:48:08.268 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.269 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2023-07-06T03:48:08.269 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2023-07-06T03:48:08.269 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.269 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2023-07-06T03:48:08.269 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2023-07-06T03:48:08.269 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.270 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:48:08.270 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2023-07-06T03:48:08.270 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.270 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:48:08.270 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:48:08.270 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.270 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2023-07-06T03:48:08.271 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.271 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2023-07-06T03:48:08.271 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2023-07-06T03:48:08.271 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:48:08.271 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:48:08.271 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2023-07-06T03:48:08.271 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:48:08.271 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2023-07-06T03:48:08.272 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:48:08.272 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:48:08.272 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2023-07-06T03:48:08.272 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.272 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:48:08.272 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.272 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2023-07-06T03:48:08.273 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.273 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2023-07-06T03:48:08.273 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.273 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2023-07-06T03:48:08.273 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2023-07-06T03:48:08.273 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2023-07-06T03:48:08.273 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:48:08.274 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:48:08.274 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.274 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2023-07-06T03:48:08.274 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2023-07-06T03:48:08.274 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.274 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2023-07-06T03:48:08.274 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2023-07-06T03:48:08.275 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:48:08.275 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.275 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2023-07-06T03:48:08.275 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2023-07-06T03:48:08.275 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.275 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2023-07-06T03:48:08.275 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2023-07-06T03:48:08.276 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2023-07-06T03:48:08.276 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.276 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2023-07-06T03:48:08.276 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2023-07-06T03:48:08.276 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.276 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:48:08.276 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.277 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2023-07-06T03:48:08.277 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.277 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2023-07-06T03:48:08.277 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.277 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:48:08.277 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2023-07-06T03:48:08.277 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.278 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2023-07-06T03:48:08.278 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2023-07-06T03:48:08.278 INFO:teuthology.orchestra.run.smithi146.stdout: bdev async discard = True 2023-07-06T03:48:08.278 INFO:teuthology.orchestra.run.smithi146.stdout: bdev enable discard = True 2023-07-06T03:48:08.278 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore allocator = bitmap 2023-07-06T03:48:08.278 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2023-07-06T03:48:08.278 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2023-07-06T03:48:08.278 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 1/20 2023-07-06T03:48:08.279 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 1/20 2023-07-06T03:48:08.279 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:48:08.279 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2023-07-06T03:48:08.279 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 4/10 2023-07-06T03:48:08.279 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:48:08.279 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2023-07-06T03:48:08.279 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:48:08.280 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:48:08.280 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2023-07-06T03:48:08.280 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:48:08.280 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.280 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2023-07-06T03:48:08.280 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:48:08.280 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2023-07-06T03:48:08.281 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:48:08.281 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:48:08.281 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:48:08.281 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:48:08.281 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2023-07-06T03:48:08.281 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.281 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2023-07-06T03:48:08.282 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:48:08.282 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:48:08.282 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2023-07-06T03:48:08.282 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:48:08.282 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2023-07-06T03:48:08.282 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2023-07-06T03:48:08.282 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:48:08.282 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2023-07-06T03:48:08.283 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2023-07-06T03:48:08.283 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:48:08.283 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.283 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:48:08.283 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:48:08.283 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:48:08.283 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.283 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:48:08.284 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:48:08.284 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:48:08.284 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.284 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2023-07-06T03:48:08.284 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:48:08.284 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.285 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:48:08.285 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2023-07-06T03:48:08.285 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:08.285 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2023-07-06T03:48:08.285 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2023-07-06T03:48:08.285 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2023-07-06T03:48:08.285 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2023-07-06T03:48:08.286 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:48:08.286 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:48:08.286 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2023-07-06T03:48:08.286 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2023-07-06T03:48:08.286 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:48:08.286 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2023-07-06T03:48:08.286 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:48:08.286 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:48:08.287 INFO:teuthology.orchestra.run.smithi146.stdout: client mountpoint = /subdir 2023-07-06T03:48:08.287 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2023-07-06T03:48:08.287 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2023-07-06T03:48:08.287 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2023-07-06T03:48:08.287 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2023-07-06T03:48:08.287 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2023-07-06T03:48:08.288 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2023-07-06T03:48:08.288 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2023-07-06T03:48:08.288 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2023-07-06T03:48:08.288 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2023-07-06T03:48:08.288 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2023-07-06T03:48:08.288 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:48:08.288 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2023-07-06T03:48:08.288 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2023-07-06T03:48:08.289 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2023-07-06T03:48:08.289 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2023-07-06T03:48:08.289 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:48:08.289 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:48:08.289 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:48:08.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:48:08.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.482+0000 7fbbbebff700 1 Processor -- start 2023-07-06T03:48:08.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.482+0000 7fbbbebff700 1 -- start start 2023-07-06T03:48:08.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.483+0000 7fbbbebff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbbb80cd440 0x7fbbb8160b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:08.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.483+0000 7fbbbebff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb8161060 con 0x7fbbb815c740 2023-07-06T03:48:08.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.483+0000 7fbbbebff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb80cdcc0 con 0x7fbbb806a680 2023-07-06T03:48:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.483+0000 7fbbbebff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb80cd9e0 con 0x7fbbb80cd440 2023-07-06T03:48:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.483+0000 7fbbb7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbbb80cd440 0x7fbbb8160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.484+0000 7fbbb7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbbb80cd440 0x7fbbb8160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56542/0 (socket says 172.21.15.97:56542) 2023-07-06T03:48:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.484+0000 7fbbb7fff700 1 -- 172.21.15.97:0/1151918108 learned_addr learned my addr 172.21.15.97:0/1151918108 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:08.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.484+0000 7fbbbc99b700 1 --1- 172.21.15.97:0/1151918108 >> v1:172.21.15.97:6789/0 conn(0x7fbbb815c740 0x7fbbb80b5cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58544/0 (socket says 172.21.15.97:58544) 2023-07-06T03:48:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.484+0000 7fbbb7fff700 1 -- 172.21.15.97:0/1151918108 >> v1:172.21.15.146:6789/0 conn(0x7fbbb806a680 legacy=0x7fbbb80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.484+0000 7fbbb7fff700 1 -- 172.21.15.97:0/1151918108 >> v1:172.21.15.97:6789/0 conn(0x7fbbb815c740 legacy=0x7fbbb80b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.484+0000 7fbbb7fff700 1 -- 172.21.15.97:0/1151918108 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb80d0b60 con 0x7fbbb80cd440 2023-07-06T03:48:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.484+0000 7fbbb7fff700 1 --2- 172.21.15.97:0/1151918108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb80cd440 0x7fbbb8160b20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbba40098f0 tx=0x7fbba4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8fd4ba256f4d63 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.484+0000 7fbbb77fe700 1 -- 172.21.15.97:0/1151918108 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4004230 con 0x7fbbb80cd440 2023-07-06T03:48:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.484+0000 7fbbb77fe700 1 -- 172.21.15.97:0/1151918108 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba4004390 con 0x7fbbb80cd440 2023-07-06T03:48:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.484+0000 7fbbb77fe700 1 -- 172.21.15.97:0/1151918108 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba40036a0 con 0x7fbbb80cd440 2023-07-06T03:48:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.485+0000 7fbbbebff700 1 -- 172.21.15.97:0/1151918108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb80cd440 msgr2=0x7fbbb8160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:08.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.485+0000 7fbbbebff700 1 --2- 172.21.15.97:0/1151918108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb80cd440 0x7fbbb8160b20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbba40098f0 tx=0x7fbba4009c00 comp rx=0 tx=0).stop 2023-07-06T03:48:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.485+0000 7fbbbebff700 1 -- 172.21.15.97:0/1151918108 shutdown_connections 2023-07-06T03:48:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.485+0000 7fbbbebff700 1 --2- 172.21.15.97:0/1151918108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb80cd440 0x7fbbb8160b20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.485+0000 7fbbbebff700 1 -- 172.21.15.97:0/1151918108 >> 172.21.15.97:0/1151918108 conn(0x7fbbb80d2e80 msgr2=0x7fbbb80d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.485+0000 7fbbbebff700 1 -- 172.21.15.97:0/1151918108 shutdown_connections 2023-07-06T03:48:08.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.485+0000 7fbbbebff700 1 -- 172.21.15.97:0/1151918108 wait complete. 2023-07-06T03:48:08.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.487+0000 7fbbbebff700 1 Processor -- start 2023-07-06T03:48:08.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.487+0000 7fbbbebff700 1 -- start start 2023-07-06T03:48:08.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.488+0000 7fbbbebff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbb806a680 0x7fbbb8154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:08.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.488+0000 7fbbbebff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbb80cd440 0x7fbbb8153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:08.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.488+0000 7fbbbebff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb815c740 0x7fbbb80c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:08.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.488+0000 7fbbbebff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81541f0 con 0x7fbbb815c740 2023-07-06T03:48:08.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.488+0000 7fbbbebff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb815e830 con 0x7fbbb806a680 2023-07-06T03:48:08.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.488+0000 7fbbbebff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80c9890 con 0x7fbbb80cd440 2023-07-06T03:48:08.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.488+0000 7fbbbd19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb815c740 0x7fbbb80c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:08.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.488+0000 7fbbbc99b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbb806a680 0x7fbbb8154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:08.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.488+0000 7fbbbd19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb815c740 0x7fbbb80c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56554/0 (socket says 172.21.15.97:56554) 2023-07-06T03:48:08.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.488+0000 7fbbbd19c700 1 -- 172.21.15.97:0/707182798 learned_addr learned my addr 172.21.15.97:0/707182798 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:08.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.488+0000 7fbbb7fff700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbb80cd440 0x7fbbb8153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:08.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.489+0000 7fbbbd19c700 1 -- 172.21.15.97:0/707182798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbb80cd440 msgr2=0x7fbbb8153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:08.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.489+0000 7fbbbd19c700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbb80cd440 0x7fbbb8153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:08.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.489+0000 7fbbbd19c700 1 -- 172.21.15.97:0/707182798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbb806a680 msgr2=0x7fbbb8154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:08.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.489+0000 7fbbbd19c700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbb806a680 0x7fbbb8154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:08.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.489+0000 7fbbbd19c700 1 -- 172.21.15.97:0/707182798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba4009550 con 0x7fbbb815c740 2023-07-06T03:48:08.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.489+0000 7fbbbd19c700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb815c740 0x7fbbb80c7330 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000b9c0 tx=0x7fbbb000bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:08.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.489+0000 7fbbb57fa700 1 -- 172.21.15.97:0/707182798 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000d8e0 con 0x7fbbb815c740 2023-07-06T03:48:08.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.489+0000 7fbbb57fa700 1 -- 172.21.15.97:0/707182798 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb000da40 con 0x7fbbb815c740 2023-07-06T03:48:08.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.489+0000 7fbbb57fa700 1 -- 172.21.15.97:0/707182798 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000ddf0 con 0x7fbbb815c740 2023-07-06T03:48:08.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.489+0000 7fbbbebff700 1 -- 172.21.15.97:0/707182798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb80c7870 con 0x7fbbb815c740 2023-07-06T03:48:08.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.489+0000 7fbbbebff700 1 -- 172.21.15.97:0/707182798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb815e1e0 con 0x7fbbb815c740 2023-07-06T03:48:08.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.490+0000 7fbbbebff700 1 -- 172.21.15.97:0/707182798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb815e9e0 con 0x7fbbb815c740 2023-07-06T03:48:08.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.491+0000 7fbbb57fa700 1 -- 172.21.15.97:0/707182798 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbbb0006d50 con 0x7fbbb815c740 2023-07-06T03:48:08.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.492+0000 7fbbb57fa700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb9c06ace0 0x7fbb9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:08.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.492+0000 7fbbb57fa700 1 -- 172.21.15.97:0/707182798 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7fbbb0011070 con 0x7fbbb815c740 2023-07-06T03:48:08.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.492+0000 7fbbbc99b700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb9c06ace0 0x7fbb9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:08.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.493+0000 7fbbbc99b700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb9c06ace0 0x7fbb9c06d190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8156970 tx=0x7fbbac007f10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:08.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.496+0000 7fbbb57fa700 1 -- 172.21.15.97:0/707182798 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbbb004e310 con 0x7fbbb815c740 2023-07-06T03:48:08.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.738+0000 7fbbbebff700 1 -- 172.21.15.97:0/707182798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbbb8067390 con 0x7fbbb815c740 2023-07-06T03:48:08.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.738+0000 7fbbb57fa700 1 -- 172.21.15.97:0/707182798 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbbb815e9e0 con 0x7fbbb815c740 2023-07-06T03:48:08.741 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:48:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 -- 172.21.15.97:0/707182798 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb9c06ace0 msgr2=0x7fbb9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb9c06ace0 0x7fbb9c06d190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8156970 tx=0x7fbbac007f10 comp rx=0 tx=0).stop 2023-07-06T03:48:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 -- 172.21.15.97:0/707182798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb815c740 msgr2=0x7fbbb80c7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb815c740 0x7fbbb80c7330 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000b9c0 tx=0x7fbbb000bcd0 comp rx=0 tx=0).stop 2023-07-06T03:48:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 -- 172.21.15.97:0/707182798 shutdown_connections 2023-07-06T03:48:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb9c06ace0 0x7fbb9c06d190 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:08.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb815c740 0x7fbbb80c7330 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:08.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbb806a680 0x7fbbb8154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:08.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 --2- 172.21.15.97:0/707182798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbbb80cd440 0x7fbbb8153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:08.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 -- 172.21.15.97:0/707182798 >> 172.21.15.97:0/707182798 conn(0x7fbbb80d2e80 msgr2=0x7fbbb80cf120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:08.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 -- 172.21.15.97:0/707182798 shutdown_connections 2023-07-06T03:48:08.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:08.741+0000 7fbbbebff700 1 -- 172.21.15.97:0/707182798 wait complete. 2023-07-06T03:48:08.761 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:48:08.762 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:48:08.762 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:48:08.762 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:48:08.763 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:08.763 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:48:08.763 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:48:08.763 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:48:08.764 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:48:08.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:08.764 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:48:08.811 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:48:08.811 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:48:08.811 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:48:08.812 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:08.812 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:48:08.812 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:08.813 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:48:08.813 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:08.813 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:48:08.814 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:48:08.814 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:08.814 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:48:08.814 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:48:08.815 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:08.815 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:48:08.815 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:08.815 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:48:08.816 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41980sec preferred_lft 41980sec 2023-07-06T03:48:08.816 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:48:08.816 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:08.816 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:48:08.817 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:08.817 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:48:08.817 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:08.817 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:48:08.818 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:08.818 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:48:08.818 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:48:08.818 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:48:08.819 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:08.819 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:48:08.819 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:48:08.819 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:48:08.819 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:08.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:08.820 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:48:08.887 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:48:08.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:08.890 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:48:08.938 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:48:08.939 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:48:08.939 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:48:08.939 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:48:08.939 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:48:08.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:08.941 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:48:08.941 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:48:08.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:48:08.942 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:48:08.942 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:48:08.942 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:48:09.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:09.109 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:48:09.155 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:48:09.156 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:48:09.157 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:48:09.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:09.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:48:09.229 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:48:09.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:48:09.230 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:09.275 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:48:09.275 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:48:09.275 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:09.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:48:09.469 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:48:09.469 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:48:09.536 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:48:09.538 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:48:09.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:09.539 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:48:09.585 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:48:09.585 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:48:09.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:48:09.699 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:48:09.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:48:09.768 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:48:09.771 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:48:09.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:09.772 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:48:09.795 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:48:09.793+0000 7f7c7b52e5c0 -1 init, newargv = 0x564f5a62f1b0 newargc=15 2023-07-06T03:48:09.796 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[135663]: starting ceph client 2023-07-06T03:48:09.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[135663]: starting fuse 2023-07-06T03:48:09.822 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:48:09.822 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:48:09.823 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:48:09.823 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:48:09.823 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:48:09.824 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:48:09.824 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:48:09.824 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:09.825 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:48:09.825 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:48:09.825 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:48:09.825 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:48:09.826 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:09.826 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:48:09.826 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:48:09.827 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:48:09.827 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:48:09.827 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:09.827 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:48:09.828 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:48:09.828 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:48:09.828 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:48:09.829 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:48:09.829 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:48:09.830 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:48:09.830 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:48:09.830 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:09.831 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:48:09.831 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:48:10.005 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.135663.asok 2023-07-06T03:48:10.183 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.135663.asok 2023-07-06T03:48:10.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:10.184 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.135663.asok status 2023-07-06T03:48:10.409 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:48:10.409 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:48:10.409 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:48:10.409 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:48:10.409 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:48:10.410 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:48:10.410 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:48:10.410 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "135663", 2023-07-06T03:48:10.410 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/subdir" 2023-07-06T03:48:10.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:10.410 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:48:10.410 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:48:10.410 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6132, 2023-07-06T03:48:10.411 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:48:10.411 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:48:10.411 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:48:10.411 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6132 2023-07-06T03:48:10.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:10.411 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:10.411 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:10.411 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:10.412 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2610031979 2023-07-06T03:48:10.412 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:10.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:10.412 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:10.412 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:10.412 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:10.412 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2610031979 2023-07-06T03:48:10.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:10.413 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6132 192.168.0.1:0/2610031979", 2023-07-06T03:48:10.413 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2610031979", 2023-07-06T03:48:10.413 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 2, 2023-07-06T03:48:10.413 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 25, 2023-07-06T03:48:10.413 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 49, 2023-07-06T03:48:10.413 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:48:10.413 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:48:10.414 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:48:10.414 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:48:10.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:10.423 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:48:10.423 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:10.469 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:48:10.470 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:10.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:10.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:10.580 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:48:10.580 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:48:10.580 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2023-07-06T03:48:10.581 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:10.581 DEBUG:teuthology.orchestra.run.smithi097:> print(os.stat("/home/ubuntu/cephtest/mnt.0/.").st_ino) 2023-07-06T03:48:10.581 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:48:10.694 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:48:10.879 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:48:10.879 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:48:10.879 DEBUG:teuthology.orchestra.run.smithi097:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2023-07-06T03:48:10.880 DEBUG:teuthology.orchestra.run.smithi097:> for n in range(0, 1000): 2023-07-06T03:48:10.880 DEBUG:teuthology.orchestra.run.smithi097:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir/dir{0}".format(n)) 2023-07-06T03:48:10.880 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:48:13.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:13.023 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync) 2023-07-06T03:48:13.449 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:48:13.450 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:48:13.450 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:48:13.450 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:48:13.451 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:48:13.451 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:13.451 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:48:13.452 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:48:13.452 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:48:13.452 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:48:13.453 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:13.453 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:48:13.453 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:48:13.453 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:48:13.454 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:48:13.454 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:13.454 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:48:13.455 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:48:13.455 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:48:13.455 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:48:13.455 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:48:13.456 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:48:13.456 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:48:13.456 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:48:13.457 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:13.457 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:48:13.457 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:48:13.602 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.135663.asok 2023-07-06T03:48:13.780 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.135663.asok 2023-07-06T03:48:13.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:13.781 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.135663.asok status 2023-07-06T03:48:14.004 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:48:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:48:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:48:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:48:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:48:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:48:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:48:14.005 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "135663", 2023-07-06T03:48:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/subdir" 2023-07-06T03:48:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 1001, 2023-07-06T03:48:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 1001, 2023-07-06T03:48:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6132, 2023-07-06T03:48:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:48:14.006 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:48:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:48:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6132 2023-07-06T03:48:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2610031979 2023-07-06T03:48:14.007 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2610031979 2023-07-06T03:48:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6132 192.168.0.1:0/2610031979", 2023-07-06T03:48:14.008 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2610031979", 2023-07-06T03:48:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1003, 2023-07-06T03:48:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 25, 2023-07-06T03:48:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 49, 2023-07-06T03:48:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 47, 2023-07-06T03:48:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:48:14.009 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:48:14.009 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:48:14.019 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:48:14.019 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:48:14.019 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:48:14.020 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:48:14.020 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:48:14.021 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:14.021 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:48:14.021 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:48:14.022 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:48:14.022 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:48:14.022 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:14.023 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:48:14.023 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:48:14.023 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:48:14.024 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:48:14.024 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:14.024 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:48:14.025 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:48:14.025 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:48:14.025 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:48:14.025 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:48:14.026 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:48:14.026 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:48:14.027 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:48:14.027 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:14.027 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:48:14.027 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:48:14.170 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.135663.asok 2023-07-06T03:48:14.348 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.135663.asok 2023-07-06T03:48:14.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:14.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.135663.asok config set client_cache_size 100 2023-07-06T03:48:14.570 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:48:14.570 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "client_cache_size = '100' " 2023-07-06T03:48:14.570 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:48:14.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:48:14.585 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:48:14.585 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:48:14.585 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:48:14.586 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:48:14.586 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:14.586 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:48:14.587 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:48:14.587 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:48:14.587 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:48:14.587 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:14.588 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:48:14.588 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:48:14.588 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:48:14.589 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:48:14.589 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:14.590 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:48:14.590 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:48:14.590 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:48:14.591 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:48:14.591 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:48:14.591 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:48:14.592 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:48:14.592 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:48:14.592 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:14.593 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:48:14.593 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:48:14.734 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.135663.asok 2023-07-06T03:48:14.913 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.135663.asok 2023-07-06T03:48:14.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:14.913 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.135663.asok status 2023-07-06T03:48:15.135 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:48:15.135 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:48:15.136 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:48:15.136 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:48:15.136 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:48:15.137 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:48:15.137 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:48:15.137 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "135663", 2023-07-06T03:48:15.137 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/subdir" 2023-07-06T03:48:15.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:15.137 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 1001, 2023-07-06T03:48:15.138 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 155, 2023-07-06T03:48:15.138 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6132, 2023-07-06T03:48:15.138 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:48:15.138 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:48:15.138 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:48:15.138 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6132 2023-07-06T03:48:15.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:15.138 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:15.139 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:15.139 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:15.139 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2610031979 2023-07-06T03:48:15.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:15.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:15.139 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:15.139 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:15.139 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:15.140 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2610031979 2023-07-06T03:48:15.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:15.140 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6132 192.168.0.1:0/2610031979", 2023-07-06T03:48:15.140 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2610031979", 2023-07-06T03:48:15.141 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1003, 2023-07-06T03:48:15.141 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 25, 2023-07-06T03:48:15.141 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 49, 2023-07-06T03:48:15.142 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 47, 2023-07-06T03:48:15.142 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:48:15.142 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:48:15.142 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:48:15.149 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 1001, 155 2023-07-06T03:48:15.150 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-07-06T03:48:20.150 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:48:20.151 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:48:20.151 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:48:20.152 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:48:20.152 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:48:20.152 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:20.153 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:48:20.153 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:48:20.153 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:48:20.153 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:48:20.154 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:20.154 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:48:20.154 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:48:20.155 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:48:20.155 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:48:20.155 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:20.155 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:48:20.156 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:48:20.156 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:48:20.156 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:48:20.157 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:48:20.157 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:48:20.157 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:48:20.158 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:48:20.158 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:20.158 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:48:20.158 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:48:20.311 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.135663.asok 2023-07-06T03:48:20.488 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.135663.asok 2023-07-06T03:48:20.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:20.489 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.135663.asok status 2023-07-06T03:48:20.713 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:48:20.713 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:48:20.713 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:48:20.713 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:48:20.714 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:48:20.714 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:48:20.714 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:48:20.714 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "135663", 2023-07-06T03:48:20.714 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/subdir" 2023-07-06T03:48:20.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:20.714 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 100, 2023-07-06T03:48:20.715 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 1, 2023-07-06T03:48:20.715 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6132, 2023-07-06T03:48:20.715 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:48:20.715 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:48:20.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:48:20.715 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6132 2023-07-06T03:48:20.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:20.715 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:20.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:20.716 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:20.716 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2610031979 2023-07-06T03:48:20.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:20.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:20.716 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:20.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:20.717 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:20.717 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2610031979 2023-07-06T03:48:20.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:20.717 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6132 192.168.0.1:0/2610031979", 2023-07-06T03:48:20.717 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2610031979", 2023-07-06T03:48:20.717 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 102, 2023-07-06T03:48:20.717 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 25, 2023-07-06T03:48:20.717 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 49, 2023-07-06T03:48:20.718 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 47, 2023-07-06T03:48:20.718 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:48:20.718 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:48:20.718 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:48:20.727 INFO:tasks.cephfs.test_client_limits:waiting, dentry_count, dentry_pinned_count: 100, 1 2023-07-06T03:48:20.754 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-07-06T03:48:20.755 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:48:20.755 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:48:20.855 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:48:20.961 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:48:20.962 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:48:20.962 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:48:21.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:48:21.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.241+0000 7f5e7c3ff700 1 Processor -- start 2023-07-06T03:48:21.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.241+0000 7f5e7c3ff700 1 -- start start 2023-07-06T03:48:21.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.243+0000 7f5e7c3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5e741593b0 0x7f5e740b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.243+0000 7f5e7c3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e740ce1f0 0x7f5e740b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.243+0000 7f5e7c3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e74159780 con 0x7f5e740b2b80 2023-07-06T03:48:21.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.243+0000 7f5e7c3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e740b3000 con 0x7f5e741593b0 2023-07-06T03:48:21.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.243+0000 7f5e7c3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74070860 con 0x7f5e740ce1f0 2023-07-06T03:48:21.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.243+0000 7f5e7a99c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5e740b2b80 0x7f5e7415cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54892/0 (socket says 172.21.15.97:54892) 2023-07-06T03:48:21.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.243+0000 7f5e7a99c700 1 -- 172.21.15.97:0/112907788 learned_addr learned my addr 172.21.15.97:0/112907788 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:21.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.243+0000 7f5e7a19b700 1 --2- 172.21.15.97:0/112907788 >> v2:172.21.15.146:3300/0 conn(0x7f5e741593b0 0x7f5e740b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.243+0000 7f5e7999a700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e740ce1f0 0x7f5e740b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.243+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873529198 0 0) 0x7f5e74159780 con 0x7f5e740b2b80 2023-07-06T03:48:21.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e600035b0 con 0x7f5e740b2b80 2023-07-06T03:48:21.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2101778888 0 0) 0x7f5e600035b0 con 0x7f5e740b2b80 2023-07-06T03:48:21.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e740ce1f0 msgr2=0x7f5e740b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:21.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e740ce1f0 0x7f5e740b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 >> v2:172.21.15.146:3300/0 conn(0x7f5e741593b0 msgr2=0x7f5e740b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:21.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 --2- 172.21.15.97:0/112907788 >> v2:172.21.15.146:3300/0 conn(0x7f5e741593b0 0x7f5e740b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74070b40 con 0x7f5e740b2b80 2023-07-06T03:48:21.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f5e6c0019f0 con 0x7f5e740b2b80 2023-07-06T03:48:21.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 >> v1:172.21.15.97:6789/0 conn(0x7f5e740b2b80 legacy=0x7f5e7415cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:21.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e60003a00 0x7f5e60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e740ce1f0 0x7f5e60004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e60004d90 0x7f5e60005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740b3000 con 0x7f5e60004d90 2023-07-06T03:48:21.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74070860 con 0x7f5e60003a00 2023-07-06T03:48:21.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6000dca0 con 0x7f5e740ce1f0 2023-07-06T03:48:21.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e7a99c700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e60004d90 0x7f5e60005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e7a19b700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e60003a00 0x7f5e60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.244+0000 7f5e7999a700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e740ce1f0 0x7f5e60004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.245+0000 7f5e7a99c700 1 -- 172.21.15.97:0/112907788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e740ce1f0 msgr2=0x7f5e60004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:21.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.245+0000 7f5e7a99c700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e740ce1f0 0x7f5e60004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.245+0000 7f5e7a99c700 1 -- 172.21.15.97:0/112907788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e60003a00 msgr2=0x7f5e60003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:21.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.245+0000 7f5e7a99c700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e60003a00 0x7f5e60003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.245+0000 7f5e7a99c700 1 -- 172.21.15.97:0/112907788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e600035b0 con 0x7f5e60004d90 2023-07-06T03:48:21.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.245+0000 7f5e7a99c700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e60004d90 0x7f5e60005200 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00a860 tx=0x7f5e60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f287b90c0ef06c61 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:21.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.245+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c012600 con 0x7f5e60004d90 2023-07-06T03:48:21.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.245+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c012780 con 0x7f5e60004d90 2023-07-06T03:48:21.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.245+0000 7f5e79199700 1 -- 172.21.15.97:0/112907788 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c012b30 con 0x7f5e60004d90 2023-07-06T03:48:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.246+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/112907788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e60004d90 msgr2=0x7f5e60005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.246+0000 7f5e7c3ff700 1 --2- 172.21.15.97:0/112907788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e60004d90 0x7f5e60005200 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00a860 tx=0x7f5e60005880 comp rx=0 tx=0).stop 2023-07-06T03:48:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.246+0000 7f5e7a99c700 1 -- 172.21.15.97:0/112907788 reap_dead start 2023-07-06T03:48:21.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.246+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/112907788 shutdown_connections 2023-07-06T03:48:21.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.246+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/112907788 >> 172.21.15.97:0/112907788 conn(0x7f5e74065c20 msgr2=0x7f5e74065ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:21.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.246+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/112907788 shutdown_connections 2023-07-06T03:48:21.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.246+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/112907788 wait complete. 2023-07-06T03:48:21.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.248+0000 7f5e7c3ff700 1 Processor -- start 2023-07-06T03:48:21.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.248+0000 7f5e7c3ff700 1 -- start start 2023-07-06T03:48:21.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7c3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e60004d90 0x7f5e7406d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7c3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e74144100 0x7f5e74140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7c3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e7406fcd0 0x7f5e74070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7c3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74070680 con 0x7f5e74144100 2023-07-06T03:48:21.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7c3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e741446d0 con 0x7f5e7406fcd0 2023-07-06T03:48:21.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7c3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74156ce0 con 0x7f5e60004d90 2023-07-06T03:48:21.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7999a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e74144100 0x7f5e74140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7999a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e74144100 0x7f5e74140d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57434/0 (socket says 172.21.15.97:57434) 2023-07-06T03:48:21.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7999a700 1 -- 172.21.15.97:0/1566848332 learned_addr learned my addr 172.21.15.97:0/1566848332 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:21.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7999a700 1 -- 172.21.15.97:0/1566848332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e60004d90 msgr2=0x7f5e7406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:21.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7999a700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e60004d90 0x7f5e7406d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7999a700 1 -- 172.21.15.97:0/1566848332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e7406fcd0 msgr2=0x7f5e74070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:21.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7999a700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e7406fcd0 0x7f5e74070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.249+0000 7f5e7999a700 1 -- 172.21.15.97:0/1566848332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74154f00 con 0x7f5e74144100 2023-07-06T03:48:21.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.250+0000 7f5e7999a700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e74144100 0x7f5e74140d70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5e64011ad0 tx=0x7f5e64011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:21.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.250+0000 7f5e6affd700 1 -- 172.21.15.97:0/1566848332 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6400f980 con 0x7f5e74144100 2023-07-06T03:48:21.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.250+0000 7f5e6affd700 1 -- 172.21.15.97:0/1566848332 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6400fae0 con 0x7f5e74144100 2023-07-06T03:48:21.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.250+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/1566848332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e741551e0 con 0x7f5e74144100 2023-07-06T03:48:21.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.250+0000 7f5e6affd700 1 -- 172.21.15.97:0/1566848332 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6400fe90 con 0x7f5e74144100 2023-07-06T03:48:21.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.250+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/1566848332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e741416c0 con 0x7f5e74144100 2023-07-06T03:48:21.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.251+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/1566848332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e74156ab0 con 0x7f5e74144100 2023-07-06T03:48:21.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.252+0000 7f5e6affd700 1 -- 172.21.15.97:0/1566848332 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5e64014440 con 0x7f5e74144100 2023-07-06T03:48:21.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.253+0000 7f5e6affd700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e5806ace0 0x7f5e5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.253+0000 7f5e6affd700 1 -- 172.21.15.97:0/1566848332 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f5e64089d10 con 0x7f5e74144100 2023-07-06T03:48:21.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.253+0000 7f5e7a19b700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e5806ace0 0x7f5e5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.254+0000 7f5e7a19b700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e5806ace0 0x7f5e5806d190 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5e70006ba0 tx=0x7f5e70006b30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:21.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.258+0000 7f5e6affd700 1 -- 172.21.15.97:0/1566848332 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5e74156ab0 con 0x7f5e74144100 2023-07-06T03:48:21.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.454+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/1566848332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e74156ab0 con 0x7f5e74144100 2023-07-06T03:48:21.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.456+0000 7f5e6affd700 1 -- 172.21.15.97:0/1566848332 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18524 (secure 0 0 0) 0x7f5e74156ab0 con 0x7f5e74144100 2023-07-06T03:48:21.458 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:21.458 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":49,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:47:52.725520+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:47:33.695348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-06T03:47:35.132699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":2,"to":5}]},{"pgid":"8.1a","mappings":[{"from":2,"to":1}]},{"pgid":"8.38","mappings":[{"from":6,"to":5}]},{"pgid":"8.3d","mappings":[{"from":6,"to":0},{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:48:21.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.458+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/1566848332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e5806ace0 msgr2=0x7f5e5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:21.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.458+0000 7f5e7c3ff700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e5806ace0 0x7f5e5806d190 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5e70006ba0 tx=0x7f5e70006b30 comp rx=0 tx=0).stop 2023-07-06T03:48:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.458+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/1566848332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e74144100 msgr2=0x7f5e74140d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.458+0000 7f5e7c3ff700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e74144100 0x7f5e74140d70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5e64011ad0 tx=0x7f5e64011e90 comp rx=0 tx=0).stop 2023-07-06T03:48:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.458+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/1566848332 shutdown_connections 2023-07-06T03:48:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.458+0000 7f5e7c3ff700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e5806ace0 0x7f5e5806d190 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.458+0000 7f5e7c3ff700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e7406fcd0 0x7f5e74070140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.458+0000 7f5e7c3ff700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e60004d90 0x7f5e7406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.458+0000 7f5e7c3ff700 1 --2- 172.21.15.97:0/1566848332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e74144100 0x7f5e74140d70 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.459+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/1566848332 >> 172.21.15.97:0/1566848332 conn(0x7f5e74065c20 msgr2=0x7f5e74158330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:21.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.459+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/1566848332 shutdown_connections 2023-07-06T03:48:21.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.459+0000 7f5e7c3ff700 1 -- 172.21.15.97:0/1566848332 wait complete. 2023-07-06T03:48:21.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:47:33.695348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:47:35.132699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:48:21.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:48:21.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.673+0000 7f6048bff700 1 Processor -- start 2023-07-06T03:48:21.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.673+0000 7f6048bff700 1 -- start start 2023-07-06T03:48:21.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.674+0000 7f6048bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f604415c540 0x7f6044160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.674+0000 7f6048bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60440b4870 0x7f60440b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f6048bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604406c090 0x7f604406c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f6048bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60440b5250 con 0x7f60440b4870 2023-07-06T03:48:21.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f6048bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604415c910 con 0x7f604415c540 2023-07-06T03:48:21.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f6048bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6044160fc0 con 0x7f604406c090 2023-07-06T03:48:21.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f603bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60440b4870 0x7f60440b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f603bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60440b4870 0x7f60440b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57448/0 (socket says 172.21.15.97:57448) 2023-07-06T03:48:21.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f603bfff700 1 -- 172.21.15.97:0/65411898 learned_addr learned my addr 172.21.15.97:0/65411898 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f6042d9d700 1 --2- 172.21.15.97:0/65411898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604406c090 0x7f604406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f603bfff700 1 -- 172.21.15.97:0/65411898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604406c090 msgr2=0x7f604406c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f603bfff700 1 --2- 172.21.15.97:0/65411898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604406c090 0x7f604406c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f603bfff700 1 -- 172.21.15.97:0/65411898 >> v2:172.21.15.146:3300/0 conn(0x7f604415c540 msgr2=0x7f6044160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f604259c700 1 --2- 172.21.15.97:0/65411898 >> v2:172.21.15.146:3300/0 conn(0x7f604415c540 0x7f6044160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f603bfff700 1 --2- 172.21.15.97:0/65411898 >> v2:172.21.15.146:3300/0 conn(0x7f604415c540 0x7f6044160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f603bfff700 1 -- 172.21.15.97:0/65411898 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440cd770 con 0x7f60440b4870 2023-07-06T03:48:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f6042d9d700 1 --2- 172.21.15.97:0/65411898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604406c090 0x7f604406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.675+0000 7f604259c700 1 --2- 172.21.15.97:0/65411898 >> v2:172.21.15.146:3300/0 conn(0x7f604415c540 0x7f6044160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f603bfff700 1 --2- 172.21.15.97:0/65411898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60440b4870 0x7f60440b4d10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f602c0098f0 tx=0x7f602c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7780d7ae799e66c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f6041d9b700 1 -- 172.21.15.97:0/65411898 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c00b860 con 0x7f60440b4870 2023-07-06T03:48:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f6041d9b700 1 -- 172.21.15.97:0/65411898 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c00b9c0 con 0x7f60440b4870 2023-07-06T03:48:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f6041d9b700 1 -- 172.21.15.97:0/65411898 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c00bd50 con 0x7f60440b4870 2023-07-06T03:48:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f6048bff700 1 -- 172.21.15.97:0/65411898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60440b4870 msgr2=0x7f60440b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f6048bff700 1 --2- 172.21.15.97:0/65411898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60440b4870 0x7f60440b4d10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f602c0098f0 tx=0x7f602c009c00 comp rx=0 tx=0).stop 2023-07-06T03:48:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f6048bff700 1 -- 172.21.15.97:0/65411898 shutdown_connections 2023-07-06T03:48:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f6048bff700 1 --2- 172.21.15.97:0/65411898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604406c090 0x7f604406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f6048bff700 1 --2- 172.21.15.97:0/65411898 >> v2:172.21.15.146:3300/0 conn(0x7f604415c540 0x7f6044160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f6048bff700 1 --2- 172.21.15.97:0/65411898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60440b4870 0x7f60440b4d10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f6048bff700 1 -- 172.21.15.97:0/65411898 >> 172.21.15.97:0/65411898 conn(0x7f604406b2f0 msgr2=0x7f60440cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.676+0000 7f6048bff700 1 -- 172.21.15.97:0/65411898 shutdown_connections 2023-07-06T03:48:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.677+0000 7f6048bff700 1 -- 172.21.15.97:0/65411898 wait complete. 2023-07-06T03:48:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.678+0000 7f6048bff700 1 Processor -- start 2023-07-06T03:48:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.678+0000 7f6048bff700 1 -- start start 2023-07-06T03:48:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.679+0000 7f6048bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406c090 0x7f604415b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.679+0000 7f6048bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60440b4870 0x7f60440c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.679+0000 7f6048bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604415c540 0x7f60441547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.679+0000 7f6048bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60440c6fa0 con 0x7f604406c090 2023-07-06T03:48:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.679+0000 7f6048bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60440c69c0 con 0x7f60440b4870 2023-07-06T03:48:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.679+0000 7f6048bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60441e7fd0 con 0x7f604415c540 2023-07-06T03:48:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.680+0000 7f604259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406c090 0x7f604415b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.680+0000 7f604259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406c090 0x7f604415b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57456/0 (socket says 172.21.15.97:57456) 2023-07-06T03:48:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.680+0000 7f604259c700 1 -- 172.21.15.97:0/1567763388 learned_addr learned my addr 172.21.15.97:0/1567763388 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.680+0000 7f603bfff700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60440b4870 0x7f60440c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.680+0000 7f6042d9d700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604415c540 0x7f60441547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.680+0000 7f604259c700 1 -- 172.21.15.97:0/1567763388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604415c540 msgr2=0x7f60441547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.680+0000 7f604259c700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604415c540 0x7f60441547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.680+0000 7f604259c700 1 -- 172.21.15.97:0/1567763388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60440b4870 msgr2=0x7f60440c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.680+0000 7f604259c700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60440b4870 0x7f60440c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.680+0000 7f604259c700 1 -- 172.21.15.97:0/1567763388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c009550 con 0x7f604406c090 2023-07-06T03:48:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.680+0000 7f604259c700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406c090 0x7f604415b760 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f603400c750 tx=0x7f603400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.681+0000 7f603affd700 1 -- 172.21.15.97:0/1567763388 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034011800 con 0x7f604406c090 2023-07-06T03:48:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.681+0000 7f603affd700 1 -- 172.21.15.97:0/1567763388 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6034011960 con 0x7f604406c090 2023-07-06T03:48:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.681+0000 7f6048bff700 1 -- 172.21.15.97:0/1567763388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60440c6c30 con 0x7f604406c090 2023-07-06T03:48:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.681+0000 7f603affd700 1 -- 172.21.15.97:0/1567763388 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034011cd0 con 0x7f604406c090 2023-07-06T03:48:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.681+0000 7f6048bff700 1 -- 172.21.15.97:0/1567763388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60441e82b0 con 0x7f604406c090 2023-07-06T03:48:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.682+0000 7f6048bff700 1 -- 172.21.15.97:0/1567763388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60440726a0 con 0x7f604406c090 2023-07-06T03:48:21.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.683+0000 7f603affd700 1 -- 172.21.15.97:0/1567763388 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6034021350 con 0x7f604406c090 2023-07-06T03:48:21.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.683+0000 7f603affd700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f602406ace0 0x7f602406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:21.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.684+0000 7f603affd700 1 -- 172.21.15.97:0/1567763388 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f60340892d0 con 0x7f604406c090 2023-07-06T03:48:21.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.684+0000 7f603bfff700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f602406ace0 0x7f602406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:21.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.684+0000 7f603bfff700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f602406ace0 0x7f602406d190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f602c000c00 tx=0x7f602c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:21.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.688+0000 7f603affd700 1 -- 172.21.15.97:0/1567763388 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6034057d80 con 0x7f604406c090 2023-07-06T03:48:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.885+0000 7f6048bff700 1 -- 172.21.15.97:0/1567763388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6044074390 con 0x7f604406c090 2023-07-06T03:48:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.886+0000 7f603affd700 1 -- 172.21.15.97:0/1567763388 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6034057910 con 0x7f604406c090 2023-07-06T03:48:21.888 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:48:21.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 -- 172.21.15.97:0/1567763388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f602406ace0 msgr2=0x7f602406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:21.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f602406ace0 0x7f602406d190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f602c000c00 tx=0x7f602c008030 comp rx=0 tx=0).stop 2023-07-06T03:48:21.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 -- 172.21.15.97:0/1567763388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406c090 msgr2=0x7f604415b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:21.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406c090 0x7f604415b760 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f603400c750 tx=0x7f603400ca60 comp rx=0 tx=0).stop 2023-07-06T03:48:21.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 -- 172.21.15.97:0/1567763388 shutdown_connections 2023-07-06T03:48:21.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f602406ace0 0x7f602406d190 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604415c540 0x7f60441547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406c090 0x7f604415b760 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 --2- 172.21.15.97:0/1567763388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60440b4870 0x7f60440c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 -- 172.21.15.97:0/1567763388 >> 172.21.15.97:0/1567763388 conn(0x7f604406b2f0 msgr2=0x7f6044160650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 -- 172.21.15.97:0/1567763388 shutdown_connections 2023-07-06T03:48:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:21.888+0000 7f6048bff700 1 -- 172.21.15.97:0/1567763388 wait complete. 2023-07-06T03:48:21.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:48:22.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.103+0000 7f1d65bff700 1 Processor -- start 2023-07-06T03:48:22.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.103+0000 7f1d65bff700 1 -- start start 2023-07-06T03:48:22.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.104+0000 7f1d65bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d6015c540 0x7f1d60160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d65bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d600d24b0 0x7f1d600b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d65bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6015c910 con 0x7f1d600b4870 2023-07-06T03:48:22.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d65bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d600b5250 con 0x7f1d6015c540 2023-07-06T03:48:22.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d65bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d60160fc0 con 0x7f1d600d24b0 2023-07-06T03:48:22.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d5f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d6015c540 0x7f1d60160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d5effd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1d600b4870 0x7f1d6006c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54894/0 (socket says 172.21.15.97:54894) 2023-07-06T03:48:22.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d5effd700 1 -- 172.21.15.97:0/1758454521 learned_addr learned my addr 172.21.15.97:0/1758454521 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d5e7fc700 1 -- 172.21.15.97:0/1758454521 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925790647 0 0) 0x7f1d6015c910 con 0x7f1d600b4870 2023-07-06T03:48:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d5f7fe700 1 -- 172.21.15.97:0/1758454521 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d600d24b0 msgr2=0x7f1d600b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d5f7fe700 1 --2- 172.21.15.97:0/1758454521 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d600d24b0 0x7f1d600b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d5f7fe700 1 -- 172.21.15.97:0/1758454521 >> v1:172.21.15.97:6789/0 conn(0x7f1d600b4870 legacy=0x7f1d6006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.105+0000 7f1d5f7fe700 1 -- 172.21.15.97:0/1758454521 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d600cd770 con 0x7f1d6015c540 2023-07-06T03:48:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d5f7fe700 1 --2- 172.21.15.97:0/1758454521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6015c540 0x7f1d60160a80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009870 tx=0x7f1d50009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c32a2a94fe45b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d5e7fc700 1 -- 172.21.15.97:0/1758454521 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50011830 con 0x7f1d6015c540 2023-07-06T03:48:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d5e7fc700 1 -- 172.21.15.97:0/1758454521 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d50011990 con 0x7f1d6015c540 2023-07-06T03:48:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d5e7fc700 1 -- 172.21.15.97:0/1758454521 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50011d20 con 0x7f1d6015c540 2023-07-06T03:48:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d65bff700 1 -- 172.21.15.97:0/1758454521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6015c540 msgr2=0x7f1d60160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d65bff700 1 --2- 172.21.15.97:0/1758454521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6015c540 0x7f1d60160a80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009870 tx=0x7f1d50009b80 comp rx=0 tx=0).stop 2023-07-06T03:48:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d65bff700 1 -- 172.21.15.97:0/1758454521 shutdown_connections 2023-07-06T03:48:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d65bff700 1 --2- 172.21.15.97:0/1758454521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6015c540 0x7f1d60160a80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d65bff700 1 --2- 172.21.15.97:0/1758454521 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d600d24b0 0x7f1d600b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d65bff700 1 -- 172.21.15.97:0/1758454521 >> 172.21.15.97:0/1758454521 conn(0x7f1d6006b2f0 msgr2=0x7f1d600cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d65bff700 1 -- 172.21.15.97:0/1758454521 shutdown_connections 2023-07-06T03:48:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.106+0000 7f1d65bff700 1 -- 172.21.15.97:0/1758454521 wait complete. 2023-07-06T03:48:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.108+0000 7f1d65bff700 1 Processor -- start 2023-07-06T03:48:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.108+0000 7f1d65bff700 1 -- start start 2023-07-06T03:48:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.109+0000 7f1d65bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d600b4870 0x7f1d60153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.109+0000 7f1d65bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600d24b0 0x7f1d6015a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.109+0000 7f1d65bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d6015c540 0x7f1d6015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.109+0000 7f1d65bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600cbb20 con 0x7f1d600d24b0 2023-07-06T03:48:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.109+0000 7f1d65bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600ccd80 con 0x7f1d600b4870 2023-07-06T03:48:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.109+0000 7f1d65bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600cbd00 con 0x7f1d6015c540 2023-07-06T03:48:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.109+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600d24b0 0x7f1d6015a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.109+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600d24b0 0x7f1d6015a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57474/0 (socket says 172.21.15.97:57474) 2023-07-06T03:48:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.109+0000 7f1d5effd700 1 -- 172.21.15.97:0/1275303099 learned_addr learned my addr 172.21.15.97:0/1275303099 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.109+0000 7f1d5f7fe700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d600b4870 0x7f1d60153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.109+0000 7f1d5ffff700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d6015c540 0x7f1d6015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.110+0000 7f1d5effd700 1 -- 172.21.15.97:0/1275303099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d6015c540 msgr2=0x7f1d6015a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.110+0000 7f1d5effd700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d6015c540 0x7f1d6015a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.110+0000 7f1d5effd700 1 -- 172.21.15.97:0/1275303099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d600b4870 msgr2=0x7f1d60153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.110+0000 7f1d5effd700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d600b4870 0x7f1d60153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.110+0000 7f1d5effd700 1 -- 172.21.15.97:0/1275303099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d500094a0 con 0x7f1d600d24b0 2023-07-06T03:48:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.110+0000 7f1d5effd700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600d24b0 0x7f1d6015a420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400c550 tx=0x7f1d5400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.110+0000 7f1d47fff700 1 -- 172.21.15.97:0/1275303099 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54004c80 con 0x7f1d600d24b0 2023-07-06T03:48:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.110+0000 7f1d47fff700 1 -- 172.21.15.97:0/1275303099 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d54004e00 con 0x7f1d600d24b0 2023-07-06T03:48:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.110+0000 7f1d47fff700 1 -- 172.21.15.97:0/1275303099 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54007310 con 0x7f1d600d24b0 2023-07-06T03:48:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.110+0000 7f1d65bff700 1 -- 172.21.15.97:0/1275303099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d600cbfe0 con 0x7f1d600d24b0 2023-07-06T03:48:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.110+0000 7f1d65bff700 1 -- 172.21.15.97:0/1275303099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d600cc5a0 con 0x7f1d600d24b0 2023-07-06T03:48:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.111+0000 7f1d65bff700 1 -- 172.21.15.97:0/1275303099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d600726a0 con 0x7f1d600d24b0 2023-07-06T03:48:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.112+0000 7f1d47fff700 1 -- 172.21.15.97:0/1275303099 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d54007470 con 0x7f1d600d24b0 2023-07-06T03:48:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.113+0000 7f1d47fff700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1d4c06ace0 0x7f1d4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.113+0000 7f1d47fff700 1 -- 172.21.15.97:0/1275303099 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f1d540891f0 con 0x7f1d600d24b0 2023-07-06T03:48:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.114+0000 7f1d5f7fe700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1d4c06ace0 0x7f1d4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.114+0000 7f1d5f7fe700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1d4c06ace0 0x7f1d4c06d190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1d50000c00 tx=0x7f1d50008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:22.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.118+0000 7f1d47fff700 1 -- 172.21.15.97:0/1275303099 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1d540540f0 con 0x7f1d600d24b0 2023-07-06T03:48:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.315+0000 7f1d65bff700 1 -- 172.21.15.97:0/1275303099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1d60074390 con 0x7f1d600d24b0 2023-07-06T03:48:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.316+0000 7f1d47fff700 1 -- 172.21.15.97:0/1275303099 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1d54057710 con 0x7f1d600d24b0 2023-07-06T03:48:22.318 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:48:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 -- 172.21.15.97:0/1275303099 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1d4c06ace0 msgr2=0x7f1d4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1d4c06ace0 0x7f1d4c06d190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1d50000c00 tx=0x7f1d50008030 comp rx=0 tx=0).stop 2023-07-06T03:48:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 -- 172.21.15.97:0/1275303099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600d24b0 msgr2=0x7f1d6015a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600d24b0 0x7f1d6015a420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400c550 tx=0x7f1d5400c860 comp rx=0 tx=0).stop 2023-07-06T03:48:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 -- 172.21.15.97:0/1275303099 shutdown_connections 2023-07-06T03:48:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1d4c06ace0 0x7f1d4c06d190 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d6015c540 0x7f1d6015a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d600b4870 0x7f1d60153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 --2- 172.21.15.97:0/1275303099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600d24b0 0x7f1d6015a420 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 -- 172.21.15.97:0/1275303099 >> 172.21.15.97:0/1275303099 conn(0x7f1d6006b2f0 msgr2=0x7f1d600d2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 -- 172.21.15.97:0/1275303099 shutdown_connections 2023-07-06T03:48:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.318+0000 7f1d65bff700 1 -- 172.21.15.97:0/1275303099 wait complete. 2023-07-06T03:48:22.338 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:48:22.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.534+0000 7f9bce9ff700 1 Processor -- start 2023-07-06T03:48:22.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.535+0000 7f9bce9ff700 1 -- start start 2023-07-06T03:48:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bce9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bc80cdcd0 0x7f9bc806fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bce9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bc80d03f0 0x7f9bc80b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bce9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9bc80b2b70 0x7f9bc8160440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bce9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc80b0310 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bce9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc80b2ff0 con 0x7f9bc80b2b70 2023-07-06T03:48:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bce9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80d0860 con 0x7f9bc80d03f0 2023-07-06T03:48:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bc7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bc80cdcd0 0x7f9bc806fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bc7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bc80cdcd0 0x7f9bc806fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57484/0 (socket says 172.21.15.97:57484) 2023-07-06T03:48:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bc7fff700 1 -- 172.21.15.97:0/2360098420 learned_addr learned my addr 172.21.15.97:0/2360098420 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bccf9c700 1 --2- 172.21.15.97:0/2360098420 >> v2:172.21.15.146:3300/0 conn(0x7f9bc80b2b70 0x7f9bc8160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bc77fe700 1 --2- 172.21.15.97:0/2360098420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bc80d03f0 0x7f9bc80b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bc7fff700 1 -- 172.21.15.97:0/2360098420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bc80d03f0 msgr2=0x7f9bc80b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:22.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bc7fff700 1 --2- 172.21.15.97:0/2360098420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bc80d03f0 0x7f9bc80b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bc7fff700 1 -- 172.21.15.97:0/2360098420 >> v2:172.21.15.146:3300/0 conn(0x7f9bc80b2b70 msgr2=0x7f9bc8160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:22.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bc7fff700 1 --2- 172.21.15.97:0/2360098420 >> v2:172.21.15.146:3300/0 conn(0x7f9bc80b2b70 0x7f9bc8160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.536+0000 7f9bc7fff700 1 -- 172.21.15.97:0/2360098420 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc806b290 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.537+0000 7f9bc7fff700 1 --2- 172.21.15.97:0/2360098420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc80cdcd0 0x7f9bc806fbe0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80c7b80 tx=0x7f9bb8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf835bb6f413743 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:22.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.537+0000 7f9bc6ffd700 1 -- 172.21.15.97:0/2360098420 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb800b870 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.537+0000 7f9bc6ffd700 1 -- 172.21.15.97:0/2360098420 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb800b9d0 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.537+0000 7f9bc6ffd700 1 -- 172.21.15.97:0/2360098420 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb800bd60 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.538+0000 7f9bce9ff700 1 -- 172.21.15.97:0/2360098420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc80cdcd0 msgr2=0x7f9bc806fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.538+0000 7f9bce9ff700 1 --2- 172.21.15.97:0/2360098420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc80cdcd0 0x7f9bc806fbe0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80c7b80 tx=0x7f9bb8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.538+0000 7f9bce9ff700 1 -- 172.21.15.97:0/2360098420 shutdown_connections 2023-07-06T03:48:22.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.538+0000 7f9bce9ff700 1 --2- 172.21.15.97:0/2360098420 >> v2:172.21.15.146:3300/0 conn(0x7f9bc80b2b70 0x7f9bc8160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.538+0000 7f9bce9ff700 1 --2- 172.21.15.97:0/2360098420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc80cdcd0 0x7f9bc806fbe0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.538+0000 7f9bce9ff700 1 --2- 172.21.15.97:0/2360098420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bc80d03f0 0x7f9bc80b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.538+0000 7f9bce9ff700 1 -- 172.21.15.97:0/2360098420 >> 172.21.15.97:0/2360098420 conn(0x7f9bc806dfe0 msgr2=0x7f9bc806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:22.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.538+0000 7f9bce9ff700 1 -- 172.21.15.97:0/2360098420 shutdown_connections 2023-07-06T03:48:22.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.538+0000 7f9bce9ff700 1 -- 172.21.15.97:0/2360098420 wait complete. 2023-07-06T03:48:22.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.540+0000 7f9bce9ff700 1 Processor -- start 2023-07-06T03:48:22.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.540+0000 7f9bce9ff700 1 -- start start 2023-07-06T03:48:22.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bce9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bc80b2b70 0x7f9bc80c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bce9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc80cdcd0 0x7f9bc8155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bce9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bc80d03f0 0x7f9bc80c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bce9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80c90e0 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bce9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80c84d0 con 0x7f9bc80d03f0 2023-07-06T03:48:22.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bce9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80c87b0 con 0x7f9bc80b2b70 2023-07-06T03:48:22.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bc77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc80cdcd0 0x7f9bc8155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bc77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc80cdcd0 0x7f9bc8155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57494/0 (socket says 172.21.15.97:57494) 2023-07-06T03:48:22.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bc77fe700 1 -- 172.21.15.97:0/3317600701 learned_addr learned my addr 172.21.15.97:0/3317600701 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:22.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bc7fff700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bc80b2b70 0x7f9bc80c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bccf9c700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bc80d03f0 0x7f9bc80c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bc77fe700 1 -- 172.21.15.97:0/3317600701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bc80b2b70 msgr2=0x7f9bc80c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bc77fe700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bc80b2b70 0x7f9bc80c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bc77fe700 1 -- 172.21.15.97:0/3317600701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bc80d03f0 msgr2=0x7f9bc80c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bc77fe700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bc80d03f0 0x7f9bc80c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bc77fe700 1 -- 172.21.15.97:0/3317600701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb8009550 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bc7fff700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bc80b2b70 0x7f9bc80c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.541+0000 7f9bccf9c700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bc80d03f0 0x7f9bc80c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.542+0000 7f9bc77fe700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc80cdcd0 0x7f9bc8155900 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00d800 tx=0x7f9bbc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:22.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.542+0000 7f9bc4ff9700 1 -- 172.21.15.97:0/3317600701 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc0125f0 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.542+0000 7f9bc4ff9700 1 -- 172.21.15.97:0/3317600701 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc012750 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.542+0000 7f9bce9ff700 1 -- 172.21.15.97:0/3317600701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc80c8a90 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.542+0000 7f9bc4ff9700 1 -- 172.21.15.97:0/3317600701 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc012b00 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.542+0000 7f9bce9ff700 1 -- 172.21.15.97:0/3317600701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc80c8130 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.543+0000 7f9bce9ff700 1 -- 172.21.15.97:0/3317600701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc8074390 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.544+0000 7f9bc4ff9700 1 -- 172.21.15.97:0/3317600701 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9bbc012c60 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.545+0000 7f9bc4ff9700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bb406ace0 0x7f9bb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.545+0000 7f9bc4ff9700 1 -- 172.21.15.97:0/3317600701 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f9bbc0898e0 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.545+0000 7f9bc7fff700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bb406ace0 0x7f9bb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.546+0000 7f9bc7fff700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bb406ace0 0x7f9bb406d190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8000c00 tx=0x7f9bb8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:22.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.549+0000 7f9bc4ff9700 1 -- 172.21.15.97:0/3317600701 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9bbc054870 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.742+0000 7f9bce9ff700 1 -- 172.21.15.97:0/3317600701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9bc80b6b30 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.742+0000 7f9bc4ff9700 1 -- 172.21.15.97:0/3317600701 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9bbc057e90 con 0x7f9bc80cdcd0 2023-07-06T03:48:22.744 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:48:22.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.744+0000 7f9bce9ff700 1 -- 172.21.15.97:0/3317600701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bb406ace0 msgr2=0x7f9bb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.744+0000 7f9bce9ff700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bb406ace0 0x7f9bb406d190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8000c00 tx=0x7f9bb8008030 comp rx=0 tx=0).stop 2023-07-06T03:48:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.744+0000 7f9bce9ff700 1 -- 172.21.15.97:0/3317600701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc80cdcd0 msgr2=0x7f9bc8155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.744+0000 7f9bce9ff700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc80cdcd0 0x7f9bc8155900 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00d800 tx=0x7f9bbc00dbc0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.745+0000 7f9bce9ff700 1 -- 172.21.15.97:0/3317600701 shutdown_connections 2023-07-06T03:48:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.745+0000 7f9bce9ff700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9bb406ace0 0x7f9bb406d190 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.745+0000 7f9bce9ff700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bc80d03f0 0x7f9bc80c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.745+0000 7f9bce9ff700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9bc80b2b70 0x7f9bc80c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.745+0000 7f9bce9ff700 1 --2- 172.21.15.97:0/3317600701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc80cdcd0 0x7f9bc8155900 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.745+0000 7f9bce9ff700 1 -- 172.21.15.97:0/3317600701 >> 172.21.15.97:0/3317600701 conn(0x7f9bc806dfe0 msgr2=0x7f9bc80cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:22.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.745+0000 7f9bce9ff700 1 -- 172.21.15.97:0/3317600701 shutdown_connections 2023-07-06T03:48:22.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.745+0000 7f9bce9ff700 1 -- 172.21.15.97:0/3317600701 wait complete. 2023-07-06T03:48:22.764 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:48:22.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.961+0000 7f89f1dff700 1 Processor -- start 2023-07-06T03:48:22.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.961+0000 7f89f1dff700 1 -- start start 2023-07-06T03:48:22.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.962+0000 7f89f1dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f89ec15c540 0x7f89ec160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89f1dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f89ec0d24b0 0x7f89ec0b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89f1dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ec15c910 con 0x7f89ec0b4870 2023-07-06T03:48:22.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89f1dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ec0b5250 con 0x7f89ec15c540 2023-07-06T03:48:22.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89f1dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ec160fc0 con 0x7f89ec0d24b0 2023-07-06T03:48:22.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89eb7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f89ec15c540 0x7f89ec160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89eb7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f89ec15c540 0x7f89ec160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57504/0 (socket says 172.21.15.97:57504) 2023-07-06T03:48:22.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89eb7fe700 1 -- 172.21.15.97:0/931763385 learned_addr learned my addr 172.21.15.97:0/931763385 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:22.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89eb7fe700 1 -- 172.21.15.97:0/931763385 >> v2:172.21.15.146:3300/0 conn(0x7f89ec0d24b0 msgr2=0x7f89ec0b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:22.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89eb7fe700 1 --2- 172.21.15.97:0/931763385 >> v2:172.21.15.146:3300/0 conn(0x7f89ec0d24b0 0x7f89ec0b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89eb7fe700 1 -- 172.21.15.97:0/931763385 >> v1:172.21.15.146:6789/0 conn(0x7f89ec0b4870 legacy=0x7f89ec06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:22.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89eb7fe700 1 -- 172.21.15.97:0/931763385 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec0cd770 con 0x7f89ec15c540 2023-07-06T03:48:22.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.963+0000 7f89eb7fe700 1 --2- 172.21.15.97:0/931763385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec15c540 0x7f89ec160a80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0098f0 tx=0x7f89dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1cf6b0289e0b2c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:22.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.964+0000 7f89eaffd700 1 -- 172.21.15.97:0/931763385 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc004230 con 0x7f89ec15c540 2023-07-06T03:48:22.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.964+0000 7f89eaffd700 1 -- 172.21.15.97:0/931763385 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc004390 con 0x7f89ec15c540 2023-07-06T03:48:22.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.964+0000 7f89eaffd700 1 -- 172.21.15.97:0/931763385 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0036a0 con 0x7f89ec15c540 2023-07-06T03:48:22.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.964+0000 7f89f1dff700 1 -- 172.21.15.97:0/931763385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec15c540 msgr2=0x7f89ec160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.964+0000 7f89f1dff700 1 --2- 172.21.15.97:0/931763385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec15c540 0x7f89ec160a80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0098f0 tx=0x7f89dc009c00 comp rx=0 tx=0).stop 2023-07-06T03:48:22.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.964+0000 7f89f1dff700 1 -- 172.21.15.97:0/931763385 shutdown_connections 2023-07-06T03:48:22.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.964+0000 7f89f1dff700 1 --2- 172.21.15.97:0/931763385 >> v2:172.21.15.146:3300/0 conn(0x7f89ec0d24b0 0x7f89ec0b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.964+0000 7f89f1dff700 1 --2- 172.21.15.97:0/931763385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec15c540 0x7f89ec160a80 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.964+0000 7f89f1dff700 1 -- 172.21.15.97:0/931763385 >> 172.21.15.97:0/931763385 conn(0x7f89ec06b2f0 msgr2=0x7f89ec0cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:22.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.964+0000 7f89f1dff700 1 -- 172.21.15.97:0/931763385 shutdown_connections 2023-07-06T03:48:22.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.964+0000 7f89f1dff700 1 -- 172.21.15.97:0/931763385 wait complete. 2023-07-06T03:48:22.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.966+0000 7f89f1dff700 1 Processor -- start 2023-07-06T03:48:22.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.966+0000 7f89f1dff700 1 -- start start 2023-07-06T03:48:22.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.967+0000 7f89f1dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89ec0b4870 0x7f89ec1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.967+0000 7f89f1dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89ec0d24b0 0x7f89ec0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.967+0000 7f89f1dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec15c540 0x7f89ec159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.967+0000 7f89f1dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec154550 con 0x7f89ec15c540 2023-07-06T03:48:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.967+0000 7f89f1dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec0ca230 con 0x7f89ec0b4870 2023-07-06T03:48:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.967+0000 7f89f1dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec0ca510 con 0x7f89ec0d24b0 2023-07-06T03:48:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.967+0000 7f89ebfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec15c540 0x7f89ec159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.967+0000 7f89ebfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec15c540 0x7f89ec159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57518/0 (socket says 172.21.15.97:57518) 2023-07-06T03:48:22.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89ebfff700 1 -- 172.21.15.97:0/3002998094 learned_addr learned my addr 172.21.15.97:0/3002998094 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:22.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89eb7fe700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89ec0b4870 0x7f89ec1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89e3fff700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89ec0d24b0 0x7f89ec0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89ebfff700 1 -- 172.21.15.97:0/3002998094 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89ec0d24b0 msgr2=0x7f89ec0cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89ebfff700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89ec0d24b0 0x7f89ec0cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89ebfff700 1 -- 172.21.15.97:0/3002998094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89ec0b4870 msgr2=0x7f89ec1539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:22.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89ebfff700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89ec0b4870 0x7f89ec1539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:22.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89ebfff700 1 -- 172.21.15.97:0/3002998094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc009550 con 0x7f89ec15c540 2023-07-06T03:48:22.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89ebfff700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec15c540 0x7f89ec159350 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f89e400b970 tx=0x7f89e400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:22.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89e8ff9700 1 -- 172.21.15.97:0/3002998094 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e400d860 con 0x7f89ec15c540 2023-07-06T03:48:22.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89e8ff9700 1 -- 172.21.15.97:0/3002998094 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e400d9e0 con 0x7f89ec15c540 2023-07-06T03:48:22.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89e8ff9700 1 -- 172.21.15.97:0/3002998094 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e400dd90 con 0x7f89ec15c540 2023-07-06T03:48:22.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.968+0000 7f89f1dff700 1 -- 172.21.15.97:0/3002998094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ec0ca7f0 con 0x7f89ec15c540 2023-07-06T03:48:22.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.969+0000 7f89f1dff700 1 -- 172.21.15.97:0/3002998094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ec0cadb0 con 0x7f89ec15c540 2023-07-06T03:48:22.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.970+0000 7f89f1dff700 1 -- 172.21.15.97:0/3002998094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ec0726a0 con 0x7f89ec15c540 2023-07-06T03:48:22.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.970+0000 7f89e8ff9700 1 -- 172.21.15.97:0/3002998094 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f89e400e430 con 0x7f89ec15c540 2023-07-06T03:48:22.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.971+0000 7f89e8ff9700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89cc06ace0 0x7f89cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:22.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.971+0000 7f89e8ff9700 1 -- 172.21.15.97:0/3002998094 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f89e40896c0 con 0x7f89ec15c540 2023-07-06T03:48:22.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.972+0000 7f89eb7fe700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89cc06ace0 0x7f89cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:22.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.972+0000 7f89eb7fe700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89cc06ace0 0x7f89cc06d190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009130 tx=0x7f89dc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:22.975+0000 7f89e8ff9700 1 -- 172.21.15.97:0/3002998094 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f89e40546e0 con 0x7f89ec15c540 2023-07-06T03:48:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.171+0000 7f89f1dff700 1 -- 172.21.15.97:0/3002998094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89ec074390 con 0x7f89ec15c540 2023-07-06T03:48:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.171+0000 7f89e8ff9700 1 -- 172.21.15.97:0/3002998094 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89e4057d00 con 0x7f89ec15c540 2023-07-06T03:48:23.173 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:48:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 -- 172.21.15.97:0/3002998094 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89cc06ace0 msgr2=0x7f89cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89cc06ace0 0x7f89cc06d190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009130 tx=0x7f89dc008030 comp rx=0 tx=0).stop 2023-07-06T03:48:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 -- 172.21.15.97:0/3002998094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec15c540 msgr2=0x7f89ec159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec15c540 0x7f89ec159350 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f89e400b970 tx=0x7f89e400bc80 comp rx=0 tx=0).stop 2023-07-06T03:48:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 -- 172.21.15.97:0/3002998094 shutdown_connections 2023-07-06T03:48:23.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89cc06ace0 0x7f89cc06d190 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ec15c540 0x7f89ec159350 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89ec0b4870 0x7f89ec1539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 --2- 172.21.15.97:0/3002998094 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89ec0d24b0 0x7f89ec0cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 -- 172.21.15.97:0/3002998094 >> 172.21.15.97:0/3002998094 conn(0x7f89ec06b2f0 msgr2=0x7f89ec160100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 -- 172.21.15.97:0/3002998094 shutdown_connections 2023-07-06T03:48:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.174+0000 7f89f1dff700 1 -- 172.21.15.97:0/3002998094 wait complete. 2023-07-06T03:48:23.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:48:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.386+0000 7f054f9ff700 1 Processor -- start 2023-07-06T03:48:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.386+0000 7f054f9ff700 1 -- start start 2023-07-06T03:48:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.387+0000 7f054f9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f05481593b0 0x7f05480b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.387+0000 7f054f9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f05480ce1f0 0x7f05480b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.387+0000 7f054f9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05480b2fe0 con 0x7f05480b2b80 2023-07-06T03:48:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.387+0000 7f054f9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0548159780 con 0x7f05480ce1f0 2023-07-06T03:48:23.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.387+0000 7f054f9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0548070860 con 0x7f05481593b0 2023-07-06T03:48:23.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.388+0000 7f054cf9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f05480ce1f0 0x7f05480b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:23.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.388+0000 7f054df9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f05480b2b80 0x7f054815cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54910/0 (socket says 172.21.15.97:54910) 2023-07-06T03:48:23.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.388+0000 7f054df9c700 1 -- 172.21.15.97:0/2186959349 learned_addr learned my addr 172.21.15.97:0/2186959349 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:23.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.388+0000 7f054d79b700 1 --2- 172.21.15.97:0/2186959349 >> v2:172.21.15.146:3300/0 conn(0x7f05481593b0 0x7f05480b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:23.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.388+0000 7f054cf9a700 1 -- 172.21.15.97:0/2186959349 >> v2:172.21.15.146:3300/0 conn(0x7f05481593b0 msgr2=0x7f05480b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:23.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.388+0000 7f054cf9a700 1 --2- 172.21.15.97:0/2186959349 >> v2:172.21.15.146:3300/0 conn(0x7f05481593b0 0x7f05480b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.388+0000 7f054cf9a700 1 -- 172.21.15.97:0/2186959349 >> v1:172.21.15.97:6789/0 conn(0x7f05480b2b80 legacy=0x7f054815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:23.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.388+0000 7f054cf9a700 1 -- 172.21.15.97:0/2186959349 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0548070b40 con 0x7f05480ce1f0 2023-07-06T03:48:23.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.388+0000 7f054cf9a700 1 --2- 172.21.15.97:0/2186959349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce1f0 0x7f05480b2640 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f054806db30 tx=0x7f0538009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1023b9b2f2944be server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:23.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.389+0000 7f053ffff700 1 -- 172.21.15.97:0/2186959349 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053800f8a0 con 0x7f05480ce1f0 2023-07-06T03:48:23.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.389+0000 7f053ffff700 1 -- 172.21.15.97:0/2186959349 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053800fa00 con 0x7f05480ce1f0 2023-07-06T03:48:23.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.389+0000 7f053ffff700 1 -- 172.21.15.97:0/2186959349 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053800fd90 con 0x7f05480ce1f0 2023-07-06T03:48:23.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.389+0000 7f054f9ff700 1 -- 172.21.15.97:0/2186959349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce1f0 msgr2=0x7f05480b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:23.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.389+0000 7f054f9ff700 1 --2- 172.21.15.97:0/2186959349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce1f0 0x7f05480b2640 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f054806db30 tx=0x7f0538009b20 comp rx=0 tx=0).stop 2023-07-06T03:48:23.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.389+0000 7f054f9ff700 1 -- 172.21.15.97:0/2186959349 shutdown_connections 2023-07-06T03:48:23.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.389+0000 7f054f9ff700 1 --2- 172.21.15.97:0/2186959349 >> v2:172.21.15.146:3300/0 conn(0x7f05481593b0 0x7f05480b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.389+0000 7f054f9ff700 1 --2- 172.21.15.97:0/2186959349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce1f0 0x7f05480b2640 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.389+0000 7f054f9ff700 1 -- 172.21.15.97:0/2186959349 >> 172.21.15.97:0/2186959349 conn(0x7f0548065c20 msgr2=0x7f0548065ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:23.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.389+0000 7f054f9ff700 1 -- 172.21.15.97:0/2186959349 shutdown_connections 2023-07-06T03:48:23.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.389+0000 7f054f9ff700 1 -- 172.21.15.97:0/2186959349 wait complete. 2023-07-06T03:48:23.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.391+0000 7f054f9ff700 1 Processor -- start 2023-07-06T03:48:23.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.391+0000 7f054f9ff700 1 -- start start 2023-07-06T03:48:23.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.392+0000 7f054f9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480b2b80 0x7f05481e8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:23.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.392+0000 7f054f9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480ce1f0 0x7f05481f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:23.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.392+0000 7f054f9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05481593b0 0x7f05481563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:23.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.392+0000 7f054f9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548152950 con 0x7f05480b2b80 2023-07-06T03:48:23.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.392+0000 7f054f9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05481552a0 con 0x7f05481593b0 2023-07-06T03:48:23.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.392+0000 7f054f9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0548154690 con 0x7f05480ce1f0 2023-07-06T03:48:23.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.392+0000 7f054d79b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480b2b80 0x7f05481e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:23.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.392+0000 7f054d79b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480b2b80 0x7f05481e8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57536/0 (socket says 172.21.15.97:57536) 2023-07-06T03:48:23.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.392+0000 7f054d79b700 1 -- 172.21.15.97:0/4125477389 learned_addr learned my addr 172.21.15.97:0/4125477389 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:23.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054df9c700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05481593b0 0x7f05481563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:23.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054cf9a700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480ce1f0 0x7f05481f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:23.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054d79b700 1 -- 172.21.15.97:0/4125477389 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480ce1f0 msgr2=0x7f05481f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:23.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054d79b700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480ce1f0 0x7f05481f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054d79b700 1 -- 172.21.15.97:0/4125477389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05481593b0 msgr2=0x7f05481563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:23.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054d79b700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05481593b0 0x7f05481563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054d79b700 1 -- 172.21.15.97:0/4125477389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05380094a0 con 0x7f05480b2b80 2023-07-06T03:48:23.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054cf9a700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480ce1f0 0x7f05481f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:23.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054df9c700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05481593b0 0x7f05481563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:23.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054d79b700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480b2b80 0x7f05481e8170 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f054400e870 tx=0x7f054400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:23.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f053dffb700 1 -- 172.21.15.97:0/4125477389 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544009590 con 0x7f05480b2b80 2023-07-06T03:48:23.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f053dffb700 1 -- 172.21.15.97:0/4125477389 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054400ee70 con 0x7f05480b2b80 2023-07-06T03:48:23.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f053dffb700 1 -- 172.21.15.97:0/4125477389 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054400ca50 con 0x7f05480b2b80 2023-07-06T03:48:23.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054f9ff700 1 -- 172.21.15.97:0/4125477389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0548154970 con 0x7f05480b2b80 2023-07-06T03:48:23.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.393+0000 7f054f9ff700 1 -- 172.21.15.97:0/4125477389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0548154f30 con 0x7f05480b2b80 2023-07-06T03:48:23.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.394+0000 7f054f9ff700 1 -- 172.21.15.97:0/4125477389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0548155580 con 0x7f05480b2b80 2023-07-06T03:48:23.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.395+0000 7f053dffb700 1 -- 172.21.15.97:0/4125477389 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f054400cbb0 con 0x7f05480b2b80 2023-07-06T03:48:23.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.396+0000 7f053dffb700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f052c06ace0 0x7f052c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:23.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.396+0000 7f053dffb700 1 -- 172.21.15.97:0/4125477389 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f0544089570 con 0x7f05480b2b80 2023-07-06T03:48:23.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.396+0000 7f054cf9a700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f052c06ace0 0x7f052c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:23.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.397+0000 7f054cf9a700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f052c06ace0 0x7f052c06d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0548153540 tx=0x7f0538008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:23.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.401+0000 7f053dffb700 1 -- 172.21.15.97:0/4125477389 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0544057f90 con 0x7f05480b2b80 2023-07-06T03:48:23.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.607+0000 7f054f9ff700 1 -- 172.21.15.97:0/4125477389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0548074390 con 0x7f05480b2b80 2023-07-06T03:48:23.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.608+0000 7f053dffb700 1 -- 172.21.15.97:0/4125477389 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7f054400ce60 con 0x7f05480b2b80 2023-07-06T03:48:23.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 -- 172.21.15.97:0/4125477389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f052c06ace0 msgr2=0x7f052c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:23.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f052c06ace0 0x7f052c06d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0548153540 tx=0x7f0538008030 comp rx=0 tx=0).stop 2023-07-06T03:48:23.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 -- 172.21.15.97:0/4125477389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480b2b80 msgr2=0x7f05481e8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:23.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480b2b80 0x7f05481e8170 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f054400e870 tx=0x7f054400ec30 comp rx=0 tx=0).stop 2023-07-06T03:48:23.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 -- 172.21.15.97:0/4125477389 shutdown_connections 2023-07-06T03:48:23.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f052c06ace0 0x7f052c06d190 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05481593b0 0x7f05481563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480b2b80 0x7f05481e8170 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 --2- 172.21.15.97:0/4125477389 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f05480ce1f0 0x7f05481f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:23.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 -- 172.21.15.97:0/4125477389 >> 172.21.15.97:0/4125477389 conn(0x7f0548065c20 msgr2=0x7f05480ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:23.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 -- 172.21.15.97:0/4125477389 shutdown_connections 2023-07-06T03:48:23.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:23.610+0000 7f054f9ff700 1 -- 172.21.15.97:0/4125477389 wait complete. 2023-07-06T03:48:23.614 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:48:23.629 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:48:23.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:23.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:23.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[135663]: fuse finished with error 0 and tester_r 0 2023-07-06T03:48:23.700 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:48:23.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:23.701 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:48:23.865 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:48:23.865 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:48:23.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:48:24.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.108+0000 7f4d613ff700 1 Processor -- start 2023-07-06T03:48:24.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.108+0000 7f4d613ff700 1 -- start start 2023-07-06T03:48:24.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.109+0000 7f4d613ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d5c0b4f80 0x7f4d5c0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.109+0000 7f4d613ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5c0d1160 0x7f4d5c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.109+0000 7f4d613ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5c15cc20 con 0x7f4d5c15c850 2023-07-06T03:48:24.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.109+0000 7f4d613ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5c0d1990 con 0x7f4d5c0b4f80 2023-07-06T03:48:24.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.109+0000 7f4d613ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c0d1c70 con 0x7f4d5c0d1160 2023-07-06T03:48:24.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.110+0000 7f4d5a7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d5c0b4f80 0x7f4d5c0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.110+0000 7f4d5affd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4d5c15c850 0x7f4d5c0b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54914/0 (socket says 172.21.15.97:54914) 2023-07-06T03:48:24.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.110+0000 7f4d5affd700 1 -- 172.21.15.97:0/3085795536 learned_addr learned my addr 172.21.15.97:0/3085795536 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:24.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.110+0000 7f4d5b7fe700 1 --2- 172.21.15.97:0/3085795536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5c0d1160 0x7f4d5c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.110+0000 7f4d5a7fc700 1 -- 172.21.15.97:0/3085795536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5c0d1160 msgr2=0x7f4d5c0b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:24.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.110+0000 7f4d5a7fc700 1 --2- 172.21.15.97:0/3085795536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5c0d1160 0x7f4d5c0b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.110+0000 7f4d5a7fc700 1 -- 172.21.15.97:0/3085795536 >> v1:172.21.15.97:6789/0 conn(0x7f4d5c15c850 legacy=0x7f4d5c0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.110+0000 7f4d5a7fc700 1 -- 172.21.15.97:0/3085795536 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c0d1f50 con 0x7f4d5c0b4f80 2023-07-06T03:48:24.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.110+0000 7f4d5a7fc700 1 --2- 172.21.15.97:0/3085795536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d5c0b4f80 0x7f4d5c0d0c20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4d50009840 tx=0x7f4d50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e856882bf0b5a12d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:24.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.110+0000 7f4d59ffb700 1 -- 172.21.15.97:0/3085795536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50011830 con 0x7f4d5c0b4f80 2023-07-06T03:48:24.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.111+0000 7f4d59ffb700 1 -- 172.21.15.97:0/3085795536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d50011990 con 0x7f4d5c0b4f80 2023-07-06T03:48:24.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.111+0000 7f4d59ffb700 1 -- 172.21.15.97:0/3085795536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50011d20 con 0x7f4d5c0b4f80 2023-07-06T03:48:24.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.111+0000 7f4d613ff700 1 -- 172.21.15.97:0/3085795536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d5c0b4f80 msgr2=0x7f4d5c0d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.111+0000 7f4d613ff700 1 --2- 172.21.15.97:0/3085795536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d5c0b4f80 0x7f4d5c0d0c20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4d50009840 tx=0x7f4d50009b50 comp rx=0 tx=0).stop 2023-07-06T03:48:24.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.111+0000 7f4d613ff700 1 -- 172.21.15.97:0/3085795536 shutdown_connections 2023-07-06T03:48:24.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.111+0000 7f4d613ff700 1 --2- 172.21.15.97:0/3085795536 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5c0d1160 0x7f4d5c0b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.111+0000 7f4d613ff700 1 --2- 172.21.15.97:0/3085795536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d5c0b4f80 0x7f4d5c0d0c20 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.111+0000 7f4d613ff700 1 -- 172.21.15.97:0/3085795536 >> 172.21.15.97:0/3085795536 conn(0x7f4d5c0d2df0 msgr2=0x7f4d5c0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:24.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.111+0000 7f4d613ff700 1 -- 172.21.15.97:0/3085795536 shutdown_connections 2023-07-06T03:48:24.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.111+0000 7f4d613ff700 1 -- 172.21.15.97:0/3085795536 wait complete. 2023-07-06T03:48:24.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.113+0000 7f4d613ff700 1 Processor -- start 2023-07-06T03:48:24.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.113+0000 7f4d613ff700 1 -- start start 2023-07-06T03:48:24.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.114+0000 7f4d613ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5c0b4f80 0x7f4d5c148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.114+0000 7f4d613ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d5c0d1160 0x7f4d5c157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.114+0000 7f4d613ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d5c15c850 0x7f4d5c157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.114+0000 7f4d613ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c1466f0 con 0x7f4d5c15c850 2023-07-06T03:48:24.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.114+0000 7f4d613ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c0c6630 con 0x7f4d5c0d1160 2023-07-06T03:48:24.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.114+0000 7f4d613ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c145a70 con 0x7f4d5c0b4f80 2023-07-06T03:48:24.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.114+0000 7f4d5b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d5c15c850 0x7f4d5c157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.114+0000 7f4d5b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d5c15c850 0x7f4d5c157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57556/0 (socket says 172.21.15.97:57556) 2023-07-06T03:48:24.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.114+0000 7f4d5b7fe700 1 -- 172.21.15.97:0/4167104595 learned_addr learned my addr 172.21.15.97:0/4167104595 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:24.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.114+0000 7f4d5affd700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5c0b4f80 0x7f4d5c148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d5a7fc700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d5c0d1160 0x7f4d5c157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d5b7fe700 1 -- 172.21.15.97:0/4167104595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5c0b4f80 msgr2=0x7f4d5c148480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d5b7fe700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5c0b4f80 0x7f4d5c148480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d5b7fe700 1 -- 172.21.15.97:0/4167104595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d5c0d1160 msgr2=0x7f4d5c157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d5b7fe700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d5c0d1160 0x7f4d5c157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d5b7fe700 1 -- 172.21.15.97:0/4167104595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d500094a0 con 0x7f4d5c15c850 2023-07-06T03:48:24.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d5affd700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5c0b4f80 0x7f4d5c148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:24.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d5a7fc700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d5c0d1160 0x7f4d5c157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d5b7fe700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d5c15c850 0x7f4d5c157aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4d54007e40 tx=0x7f4d5400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d437fe700 1 -- 172.21.15.97:0/4167104595 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400eea0 con 0x7f4d5c15c850 2023-07-06T03:48:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d437fe700 1 -- 172.21.15.97:0/4167104595 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54014420 con 0x7f4d5c15c850 2023-07-06T03:48:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d437fe700 1 -- 172.21.15.97:0/4167104595 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d540147b0 con 0x7f4d5c15c850 2023-07-06T03:48:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d613ff700 1 -- 172.21.15.97:0/4167104595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c145d50 con 0x7f4d5c15c850 2023-07-06T03:48:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.115+0000 7f4d613ff700 1 -- 172.21.15.97:0/4167104595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c146310 con 0x7f4d5c15c850 2023-07-06T03:48:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.116+0000 7f4d613ff700 1 -- 172.21.15.97:0/4167104595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5c0726a0 con 0x7f4d5c15c850 2023-07-06T03:48:24.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.117+0000 7f4d437fe700 1 -- 172.21.15.97:0/4167104595 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d54014910 con 0x7f4d5c15c850 2023-07-06T03:48:24.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.118+0000 7f4d437fe700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4806ace0 0x7f4d4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.118+0000 7f4d437fe700 1 -- 172.21.15.97:0/4167104595 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f4d54089de0 con 0x7f4d5c15c850 2023-07-06T03:48:24.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.118+0000 7f4d5affd700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4806ace0 0x7f4d4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.119+0000 7f4d5affd700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4806ace0 0x7f4d4806d190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c005910 tx=0x7f4d4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:24.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.123+0000 7f4d437fe700 1 -- 172.21.15.97:0/4167104595 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4d54054d70 con 0x7f4d5c15c850 2023-07-06T03:48:24.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.319+0000 7f4d613ff700 1 -- 172.21.15.97:0/4167104595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d5c146830 con 0x7f4d5c15c850 2023-07-06T03:48:24.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.320+0000 7f4d437fe700 1 -- 172.21.15.97:0/4167104595 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18524 (secure 0 0 0) 0x7f4d5c146830 con 0x7f4d5c15c850 2023-07-06T03:48:24.322 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:24.322 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":49,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:47:52.725520+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:47:33.695348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-06T03:47:35.132699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":2,"to":5}]},{"pgid":"8.1a","mappings":[{"from":2,"to":1}]},{"pgid":"8.38","mappings":[{"from":6,"to":5}]},{"pgid":"8.3d","mappings":[{"from":6,"to":0},{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:48:24.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.322+0000 7f4d613ff700 1 -- 172.21.15.97:0/4167104595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4806ace0 msgr2=0x7f4d4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.322+0000 7f4d613ff700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4806ace0 0x7f4d4806d190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c005910 tx=0x7f4d4c009040 comp rx=0 tx=0).stop 2023-07-06T03:48:24.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.322+0000 7f4d613ff700 1 -- 172.21.15.97:0/4167104595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d5c15c850 msgr2=0x7f4d5c157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.322+0000 7f4d613ff700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d5c15c850 0x7f4d5c157aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4d54007e40 tx=0x7f4d5400e3c0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.322+0000 7f4d613ff700 1 -- 172.21.15.97:0/4167104595 shutdown_connections 2023-07-06T03:48:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.322+0000 7f4d613ff700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d4806ace0 0x7f4d4806d190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.322+0000 7f4d613ff700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d5c15c850 0x7f4d5c157aa0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.322+0000 7f4d613ff700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d5c0b4f80 0x7f4d5c148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.322+0000 7f4d613ff700 1 --2- 172.21.15.97:0/4167104595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d5c0d1160 0x7f4d5c157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.322+0000 7f4d613ff700 1 -- 172.21.15.97:0/4167104595 >> 172.21.15.97:0/4167104595 conn(0x7f4d5c0d2df0 msgr2=0x7f4d5c15af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:24.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.323+0000 7f4d613ff700 1 -- 172.21.15.97:0/4167104595 shutdown_connections 2023-07-06T03:48:24.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.323+0000 7f4d613ff700 1 -- 172.21.15.97:0/4167104595 wait complete. 2023-07-06T03:48:24.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:47:33.695348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:47:35.132699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:48:24.343 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:48:24.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.539+0000 7f7dd6dff700 1 Processor -- start 2023-07-06T03:48:24.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.539+0000 7f7dd6dff700 1 -- start start 2023-07-06T03:48:24.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.540+0000 7f7dd6dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7dd00b4870 0x7f7dd00b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.540+0000 7f7dd6dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7dd00d14a0 0x7f7dd00d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.540+0000 7f7dd6dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd015c910 con 0x7f7dd015c540 2023-07-06T03:48:24.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.540+0000 7f7dd6dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd00b5250 con 0x7f7dd00b4870 2023-07-06T03:48:24.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.540+0000 7f7dd6dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd00cd770 con 0x7f7dd00d14a0 2023-07-06T03:48:24.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.540+0000 7f7dcffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7dd00b4870 0x7f7dd00b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.540+0000 7f7dcffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7dd00b4870 0x7f7dd00b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57566/0 (socket says 172.21.15.97:57566) 2023-07-06T03:48:24.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.540+0000 7f7dcffff700 1 -- 172.21.15.97:0/736673891 learned_addr learned my addr 172.21.15.97:0/736673891 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.541+0000 7f7dd539c700 1 --2- 172.21.15.97:0/736673891 >> v2:172.21.15.146:3300/0 conn(0x7f7dd00d14a0 0x7f7dd00d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.541+0000 7f7dcffff700 1 -- 172.21.15.97:0/736673891 >> v2:172.21.15.146:3300/0 conn(0x7f7dd00d14a0 msgr2=0x7f7dd00d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.541+0000 7f7dcffff700 1 --2- 172.21.15.97:0/736673891 >> v2:172.21.15.146:3300/0 conn(0x7f7dd00d14a0 0x7f7dd00d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.541+0000 7f7dcffff700 1 -- 172.21.15.97:0/736673891 >> v1:172.21.15.146:6789/0 conn(0x7f7dd015c540 legacy=0x7f7dd0160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.541+0000 7f7dcffff700 1 -- 172.21.15.97:0/736673891 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd00cda50 con 0x7f7dd00b4870 2023-07-06T03:48:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.541+0000 7f7dd539c700 1 --2- 172.21.15.97:0/736673891 >> v2:172.21.15.146:3300/0 conn(0x7f7dd00d14a0 0x7f7dd00d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:24.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.541+0000 7f7dcffff700 1 --2- 172.21.15.97:0/736673891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd00b4870 0x7f7dd00b4d10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0098f0 tx=0x7f7dbc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3d07199848d9c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:24.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.541+0000 7f7dcf7fe700 1 -- 172.21.15.97:0/736673891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc004230 con 0x7f7dd00b4870 2023-07-06T03:48:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.541+0000 7f7dcf7fe700 1 -- 172.21.15.97:0/736673891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc004390 con 0x7f7dd00b4870 2023-07-06T03:48:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.541+0000 7f7dcf7fe700 1 -- 172.21.15.97:0/736673891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0036a0 con 0x7f7dd00b4870 2023-07-06T03:48:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.542+0000 7f7dd6dff700 1 -- 172.21.15.97:0/736673891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd00b4870 msgr2=0x7f7dd00b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.542+0000 7f7dd6dff700 1 --2- 172.21.15.97:0/736673891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd00b4870 0x7f7dd00b4d10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0098f0 tx=0x7f7dbc009c00 comp rx=0 tx=0).stop 2023-07-06T03:48:24.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.542+0000 7f7dd6dff700 1 -- 172.21.15.97:0/736673891 shutdown_connections 2023-07-06T03:48:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.542+0000 7f7dd6dff700 1 --2- 172.21.15.97:0/736673891 >> v2:172.21.15.146:3300/0 conn(0x7f7dd00d14a0 0x7f7dd00d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.542+0000 7f7dd6dff700 1 --2- 172.21.15.97:0/736673891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd00b4870 0x7f7dd00b4d10 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.542+0000 7f7dd6dff700 1 -- 172.21.15.97:0/736673891 >> 172.21.15.97:0/736673891 conn(0x7f7dd006b2f0 msgr2=0x7f7dd00cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.542+0000 7f7dd6dff700 1 -- 172.21.15.97:0/736673891 shutdown_connections 2023-07-06T03:48:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.542+0000 7f7dd6dff700 1 -- 172.21.15.97:0/736673891 wait complete. 2023-07-06T03:48:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.544+0000 7f7dd6dff700 1 Processor -- start 2023-07-06T03:48:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.544+0000 7f7dd6dff700 1 -- start start 2023-07-06T03:48:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd6dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7dd00b4870 0x7f7dd00b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd6dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7dd00d14a0 0x7f7dd00b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd6dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd015c540 0x7f7dd01f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd6dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00b5f70 con 0x7f7dd015c540 2023-07-06T03:48:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd6dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00c9770 con 0x7f7dd00d14a0 2023-07-06T03:48:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd6dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00b7720 con 0x7f7dd00b4870 2023-07-06T03:48:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd539c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd015c540 0x7f7dd01f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd539c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd015c540 0x7f7dd01f24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57580/0 (socket says 172.21.15.97:57580) 2023-07-06T03:48:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd539c700 1 -- 172.21.15.97:0/2637307187 learned_addr learned my addr 172.21.15.97:0/2637307187 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd539c700 1 -- 172.21.15.97:0/2637307187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7dd00b4870 msgr2=0x7f7dd00b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd539c700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7dd00b4870 0x7f7dd00b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd539c700 1 -- 172.21.15.97:0/2637307187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7dd00d14a0 msgr2=0x7f7dd00b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd539c700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7dd00d14a0 0x7f7dd00b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.545+0000 7f7dd539c700 1 -- 172.21.15.97:0/2637307187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc009550 con 0x7f7dd015c540 2023-07-06T03:48:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.546+0000 7f7dd539c700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd015c540 0x7f7dd01f24a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800cef0 tx=0x7f7dc800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.546+0000 7f7dcd7fa700 1 -- 172.21.15.97:0/2637307187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc800ae50 con 0x7f7dd015c540 2023-07-06T03:48:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.546+0000 7f7dcd7fa700 1 -- 172.21.15.97:0/2637307187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc8015440 con 0x7f7dd015c540 2023-07-06T03:48:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.546+0000 7f7dd6dff700 1 -- 172.21.15.97:0/2637307187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd00b7a00 con 0x7f7dd015c540 2023-07-06T03:48:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.546+0000 7f7dcd7fa700 1 -- 172.21.15.97:0/2637307187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80157f0 con 0x7f7dd015c540 2023-07-06T03:48:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.546+0000 7f7dd6dff700 1 -- 172.21.15.97:0/2637307187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd01f3940 con 0x7f7dd015c540 2023-07-06T03:48:24.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.547+0000 7f7dd6dff700 1 -- 172.21.15.97:0/2637307187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd00726a0 con 0x7f7dd015c540 2023-07-06T03:48:24.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.548+0000 7f7dcd7fa700 1 -- 172.21.15.97:0/2637307187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7dc800d430 con 0x7f7dd015c540 2023-07-06T03:48:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.549+0000 7f7dcd7fa700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7db406ace0 0x7f7db406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.549+0000 7f7dcd7fa700 1 -- 172.21.15.97:0/2637307187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f7dc8089c00 con 0x7f7dd015c540 2023-07-06T03:48:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.549+0000 7f7dd4b9b700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7db406ace0 0x7f7db406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.550+0000 7f7dd4b9b700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7db406ace0 0x7f7db406d190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7dd01f35d0 tx=0x7f7dc4006c50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:24.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.553+0000 7f7dcd7fa700 1 -- 172.21.15.97:0/2637307187 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7dc8054b90 con 0x7f7dd015c540 2023-07-06T03:48:24.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.751+0000 7f7dd6dff700 1 -- 172.21.15.97:0/2637307187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7dd0074390 con 0x7f7dd015c540 2023-07-06T03:48:24.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.752+0000 7f7dcd7fa700 1 -- 172.21.15.97:0/2637307187 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7dc801a0a0 con 0x7f7dd015c540 2023-07-06T03:48:24.754 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:48:24.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.754+0000 7f7dd6dff700 1 -- 172.21.15.97:0/2637307187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7db406ace0 msgr2=0x7f7db406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.754+0000 7f7dd6dff700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7db406ace0 0x7f7db406d190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7dd01f35d0 tx=0x7f7dc4006c50 comp rx=0 tx=0).stop 2023-07-06T03:48:24.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.754+0000 7f7dd6dff700 1 -- 172.21.15.97:0/2637307187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd015c540 msgr2=0x7f7dd01f24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.754+0000 7f7dd6dff700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd015c540 0x7f7dd01f24a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800cef0 tx=0x7f7dc800a340 comp rx=0 tx=0).stop 2023-07-06T03:48:24.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.754+0000 7f7dd6dff700 1 -- 172.21.15.97:0/2637307187 shutdown_connections 2023-07-06T03:48:24.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.754+0000 7f7dd6dff700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7db406ace0 0x7f7db406d190 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.754+0000 7f7dd6dff700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dd015c540 0x7f7dd01f24a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.754+0000 7f7dd6dff700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7dd00b4870 0x7f7dd00b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.754+0000 7f7dd6dff700 1 --2- 172.21.15.97:0/2637307187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7dd00d14a0 0x7f7dd00b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.754+0000 7f7dd6dff700 1 -- 172.21.15.97:0/2637307187 >> 172.21.15.97:0/2637307187 conn(0x7f7dd006b2f0 msgr2=0x7f7dd015afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:24.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.754+0000 7f7dd6dff700 1 -- 172.21.15.97:0/2637307187 shutdown_connections 2023-07-06T03:48:24.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.755+0000 7f7dd6dff700 1 -- 172.21.15.97:0/2637307187 wait complete. 2023-07-06T03:48:24.774 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:48:24.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.971+0000 7f5f633ff700 1 Processor -- start 2023-07-06T03:48:24.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.971+0000 7f5f633ff700 1 -- start start 2023-07-06T03:48:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.973+0000 7f5f633ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5f5c0d0a30 0x7f5f5c0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.973+0000 7f5f633ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5c160770 con 0x7f5f5c0b48e0 2023-07-06T03:48:24.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.973+0000 7f5f633ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5c161050 con 0x7f5f5c160c00 2023-07-06T03:48:24.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.973+0000 7f5f633ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f5c0d0700 con 0x7f5f5c0d0a30 2023-07-06T03:48:24.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.973+0000 7f5f6119b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5f5c0b48e0 0x7f5f5c15d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54924/0 (socket says 172.21.15.97:54924) 2023-07-06T03:48:24.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.973+0000 7f5f6119b700 1 -- 172.21.15.97:0/3419564655 learned_addr learned my addr 172.21.15.97:0/3419564655 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:24.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.973+0000 7f5f6199c700 1 --2- 172.21.15.97:0/3419564655 >> v2:172.21.15.146:3300/0 conn(0x7f5f5c0d0a30 0x7f5f5c0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.973+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406596832 0 0) 0x7f5f5c160770 con 0x7f5f5c0b48e0 2023-07-06T03:48:24.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.973+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f480035b0 con 0x7f5f5c0b48e0 2023-07-06T03:48:24.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506624145 0 0) 0x7f5f5c161050 con 0x7f5f5c160c00 2023-07-06T03:48:24.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f5c160770 con 0x7f5f5c160c00 2023-07-06T03:48:24.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2057740734 0 0) 0x7f5f480035b0 con 0x7f5f5c0b48e0 2023-07-06T03:48:24.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 >> v2:172.21.15.146:3300/0 conn(0x7f5f5c0d0a30 msgr2=0x7f5f5c0d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:24.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 --2- 172.21.15.97:0/3419564655 >> v2:172.21.15.146:3300/0 conn(0x7f5f5c0d0a30 0x7f5f5c0d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 >> v1:172.21.15.146:6789/0 conn(0x7f5f5c160c00 legacy=0x7f5f5c0b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c0cd850 con 0x7f5f5c0b48e0 2023-07-06T03:48:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f5f58001670 con 0x7f5f5c0b48e0 2023-07-06T03:48:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 >> v1:172.21.15.97:6789/0 conn(0x7f5f5c0b48e0 legacy=0x7f5f5c15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 --2- 172.21.15.97:0/3419564655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 0x7f5f48003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 --2- 172.21.15.97:0/3419564655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f48004850 0x7f5f48004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 --2- 172.21.15.97:0/3419564655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f48005200 0x7f5f4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0d0700 con 0x7f5f48003be0 2023-07-06T03:48:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4800c0e0 con 0x7f5f48004850 2023-07-06T03:48:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4800c270 con 0x7f5f48005200 2023-07-06T03:48:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f6119b700 1 --2- 172.21.15.97:0/3419564655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 0x7f5f48003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f6099a700 1 --2- 172.21.15.97:0/3419564655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f48004850 0x7f5f48004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f6199c700 1 --2- 172.21.15.97:0/3419564655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f48005200 0x7f5f4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.974+0000 7f5f6119b700 1 -- 172.21.15.97:0/3419564655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f48005200 msgr2=0x7f5f4800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.975+0000 7f5f6119b700 1 --2- 172.21.15.97:0/3419564655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f48005200 0x7f5f4800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.975+0000 7f5f6119b700 1 -- 172.21.15.97:0/3419564655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f48004850 msgr2=0x7f5f48004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.975+0000 7f5f6119b700 1 --2- 172.21.15.97:0/3419564655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f48004850 0x7f5f48004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.975+0000 7f5f6119b700 1 -- 172.21.15.97:0/3419564655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480035b0 con 0x7f5f48003be0 2023-07-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.975+0000 7f5f6199c700 1 -- 172.21.15.97:0/3419564655 reap_dead start 2023-07-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.975+0000 7f5f6119b700 1 --2- 172.21.15.97:0/3419564655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 0x7f5f48003fd0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5f58001eb0 tx=0x7f5f48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3622257b9f0dc306 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.975+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800cc70 con 0x7f5f48003be0 2023-07-06T03:48:24.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.975+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5800cdf0 con 0x7f5f48003be0 2023-07-06T03:48:24.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.975+0000 7f5f53fff700 1 -- 172.21.15.97:0/3419564655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58005ba0 con 0x7f5f48003be0 2023-07-06T03:48:24.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.975+0000 7f5f633ff700 1 -- 172.21.15.97:0/3419564655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 msgr2=0x7f5f48003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.975+0000 7f5f633ff700 1 --2- 172.21.15.97:0/3419564655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 0x7f5f48003fd0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5f58001eb0 tx=0x7f5f48005880 comp rx=0 tx=0).stop 2023-07-06T03:48:24.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.976+0000 7f5f633ff700 1 -- 172.21.15.97:0/3419564655 shutdown_connections 2023-07-06T03:48:24.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.976+0000 7f5f633ff700 1 --2- 172.21.15.97:0/3419564655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 0x7f5f48003fd0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.976+0000 7f5f633ff700 1 -- 172.21.15.97:0/3419564655 >> 172.21.15.97:0/3419564655 conn(0x7f5f5c06cdd0 msgr2=0x7f5f5c0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:24.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.976+0000 7f5f633ff700 1 -- 172.21.15.97:0/3419564655 shutdown_connections 2023-07-06T03:48:24.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.976+0000 7f5f633ff700 1 -- 172.21.15.97:0/3419564655 wait complete. 2023-07-06T03:48:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.977+0000 7f5f633ff700 1 Processor -- start 2023-07-06T03:48:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.978+0000 7f5f633ff700 1 -- start start 2023-07-06T03:48:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.978+0000 7f5f633ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 0x7f5f5c0b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.978+0000 7f5f633ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f5c152160 0x7f5f5c1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f633ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f5c0b6e50 0x7f5f5c0b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f633ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0cab20 con 0x7f5f48003be0 2023-07-06T03:48:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f633ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0c9f30 con 0x7f5f5c0b6e50 2023-07-06T03:48:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f633ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c152b10 con 0x7f5f5c152160 2023-07-06T03:48:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6119b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 0x7f5f5c0b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6119b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 0x7f5f5c0b8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57608/0 (socket says 172.21.15.97:57608) 2023-07-06T03:48:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6119b700 1 -- 172.21.15.97:0/3524464504 learned_addr learned my addr 172.21.15.97:0/3524464504 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6099a700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f5c152160 0x7f5f5c1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6119b700 1 -- 172.21.15.97:0/3524464504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f5c152160 msgr2=0x7f5f5c1525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6199c700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f5c0b6e50 0x7f5f5c0b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6119b700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f5c152160 0x7f5f5c1525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6119b700 1 -- 172.21.15.97:0/3524464504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f5c0b6e50 msgr2=0x7f5f5c0b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6119b700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f5c0b6e50 0x7f5f5c0b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6119b700 1 -- 172.21.15.97:0/3524464504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c0ca180 con 0x7f5f48003be0 2023-07-06T03:48:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6099a700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f5c152160 0x7f5f5c1525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.979+0000 7f5f6119b700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 0x7f5f5c0b8070 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800c720 tx=0x7f5f58001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.980+0000 7f5f51ffb700 1 -- 172.21.15.97:0/3524464504 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f580070d0 con 0x7f5f48003be0 2023-07-06T03:48:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.980+0000 7f5f51ffb700 1 -- 172.21.15.97:0/3524464504 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f58007230 con 0x7f5f48003be0 2023-07-06T03:48:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.980+0000 7f5f633ff700 1 -- 172.21.15.97:0/3524464504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c0ca460 con 0x7f5f48003be0 2023-07-06T03:48:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.980+0000 7f5f51ffb700 1 -- 172.21.15.97:0/3524464504 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f580075c0 con 0x7f5f48003be0 2023-07-06T03:48:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.980+0000 7f5f633ff700 1 -- 172.21.15.97:0/3524464504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c0b7810 con 0x7f5f48003be0 2023-07-06T03:48:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.981+0000 7f5f633ff700 1 -- 172.21.15.97:0/3524464504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5c0726a0 con 0x7f5f48003be0 2023-07-06T03:48:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.981+0000 7f5f51ffb700 1 -- 172.21.15.97:0/3524464504 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f58027450 con 0x7f5f48003be0 2023-07-06T03:48:24.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.982+0000 7f5f51ffb700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f4006ace0 0x7f5f4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:24.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.983+0000 7f5f51ffb700 1 -- 172.21.15.97:0/3524464504 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f5f5808a960 con 0x7f5f48003be0 2023-07-06T03:48:24.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.983+0000 7f5f6099a700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f4006ace0 0x7f5f4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:24.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.983+0000 7f5f6099a700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f4006ace0 0x7f5f4006d190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c007b70 tx=0x7f5f4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:24.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:24.987+0000 7f5f51ffb700 1 -- 172.21.15.97:0/3524464504 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5f58055980 con 0x7f5f48003be0 2023-07-06T03:48:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.184+0000 7f5f633ff700 1 -- 172.21.15.97:0/3524464504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5f5c074390 con 0x7f5f48003be0 2023-07-06T03:48:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.184+0000 7f5f51ffb700 1 -- 172.21.15.97:0/3524464504 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5f58058fa0 con 0x7f5f48003be0 2023-07-06T03:48:25.187 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:48:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 -- 172.21.15.97:0/3524464504 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f4006ace0 msgr2=0x7f5f4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f4006ace0 0x7f5f4006d190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c007b70 tx=0x7f5f4c009040 comp rx=0 tx=0).stop 2023-07-06T03:48:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 -- 172.21.15.97:0/3524464504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 msgr2=0x7f5f5c0b8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 0x7f5f5c0b8070 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800c720 tx=0x7f5f58001930 comp rx=0 tx=0).stop 2023-07-06T03:48:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 -- 172.21.15.97:0/3524464504 shutdown_connections 2023-07-06T03:48:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f4006ace0 0x7f5f4006d190 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f5c0b6e50 0x7f5f5c0b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f48003be0 0x7f5f5c0b8070 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 --2- 172.21.15.97:0/3524464504 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f5c152160 0x7f5f5c1525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 -- 172.21.15.97:0/3524464504 >> 172.21.15.97:0/3524464504 conn(0x7f5f5c06cdd0 msgr2=0x7f5f5c06ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 -- 172.21.15.97:0/3524464504 shutdown_connections 2023-07-06T03:48:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.187+0000 7f5f633ff700 1 -- 172.21.15.97:0/3524464504 wait complete. 2023-07-06T03:48:25.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:48:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.405+0000 7f6d893ff700 1 Processor -- start 2023-07-06T03:48:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.405+0000 7f6d893ff700 1 -- start start 2023-07-06T03:48:25.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.406+0000 7f6d893ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d8415c540 0x7f6d84160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:25.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d893ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8415c910 con 0x7f6d840d24b0 2023-07-06T03:48:25.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d893ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d84160fc0 con 0x7f6d840b4870 2023-07-06T03:48:25.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d893ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8406ceb0 con 0x7f6d8415c540 2023-07-06T03:48:25.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d837fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6d840d24b0 0x7f6d8406c7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54932/0 (socket says 172.21.15.97:54932) 2023-07-06T03:48:25.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d837fe700 1 -- 172.21.15.97:0/1607989778 learned_addr learned my addr 172.21.15.97:0/1607989778 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:25.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d82ffd700 1 --2- 172.21.15.97:0/1607989778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d8415c540 0x7f6d84160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d827fc700 1 -- 172.21.15.97:0/1607989778 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83770181 0 0) 0x7f6d8415c910 con 0x7f6d840d24b0 2023-07-06T03:48:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d827fc700 1 -- 172.21.15.97:0/1607989778 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d640035b0 con 0x7f6d840d24b0 2023-07-06T03:48:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d82ffd700 1 -- 172.21.15.97:0/1607989778 >> v1:172.21.15.146:6789/0 conn(0x7f6d840b4870 legacy=0x7f6d8406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d82ffd700 1 -- 172.21.15.97:0/1607989778 >> v1:172.21.15.97:6789/0 conn(0x7f6d840d24b0 legacy=0x7f6d8406c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d827fc700 1 -- 172.21.15.97:0/1607989778 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529216391 0 0) 0x7f6d84160fc0 con 0x7f6d840b4870 2023-07-06T03:48:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.407+0000 7f6d82ffd700 1 -- 172.21.15.97:0/1607989778 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d840cd770 con 0x7f6d8415c540 2023-07-06T03:48:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.408+0000 7f6d82ffd700 1 --2- 172.21.15.97:0/1607989778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d8415c540 0x7f6d84160a80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009870 tx=0x7f6d74009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b555ca8351709bb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:25.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.408+0000 7f6d827fc700 1 -- 172.21.15.97:0/1607989778 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74011830 con 0x7f6d8415c540 2023-07-06T03:48:25.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.408+0000 7f6d827fc700 1 -- 172.21.15.97:0/1607989778 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d74011990 con 0x7f6d8415c540 2023-07-06T03:48:25.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.408+0000 7f6d827fc700 1 -- 172.21.15.97:0/1607989778 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74011d20 con 0x7f6d8415c540 2023-07-06T03:48:25.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.409+0000 7f6d893ff700 1 -- 172.21.15.97:0/1607989778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d8415c540 msgr2=0x7f6d84160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.409+0000 7f6d893ff700 1 --2- 172.21.15.97:0/1607989778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d8415c540 0x7f6d84160a80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009870 tx=0x7f6d74009b80 comp rx=0 tx=0).stop 2023-07-06T03:48:25.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.409+0000 7f6d893ff700 1 -- 172.21.15.97:0/1607989778 shutdown_connections 2023-07-06T03:48:25.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.409+0000 7f6d893ff700 1 --2- 172.21.15.97:0/1607989778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d8415c540 0x7f6d84160a80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.409+0000 7f6d893ff700 1 -- 172.21.15.97:0/1607989778 >> 172.21.15.97:0/1607989778 conn(0x7f6d8406b2f0 msgr2=0x7f6d840cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:25.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.409+0000 7f6d893ff700 1 -- 172.21.15.97:0/1607989778 shutdown_connections 2023-07-06T03:48:25.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.409+0000 7f6d893ff700 1 -- 172.21.15.97:0/1607989778 wait complete. 2023-07-06T03:48:25.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.410+0000 7f6d893ff700 1 Processor -- start 2023-07-06T03:48:25.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.410+0000 7f6d893ff700 1 -- start start 2023-07-06T03:48:25.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.411+0000 7f6d893ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d840b4870 0x7f6d840b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:25.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.411+0000 7f6d893ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d840d24b0 0x7f6d841f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:25.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.411+0000 7f6d893ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8415c540 0x7f6d84156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:25.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.411+0000 7f6d893ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8415d1c0 con 0x7f6d8415c540 2023-07-06T03:48:25.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.411+0000 7f6d893ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d840b6950 con 0x7f6d840b4870 2023-07-06T03:48:25.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.411+0000 7f6d893ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8415cb70 con 0x7f6d840d24b0 2023-07-06T03:48:25.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d837fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8415c540 0x7f6d84156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d837fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8415c540 0x7f6d84156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57612/0 (socket says 172.21.15.97:57612) 2023-07-06T03:48:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d837fe700 1 -- 172.21.15.97:0/1580455052 learned_addr learned my addr 172.21.15.97:0/1580455052 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d82ffd700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d840b4870 0x7f6d840b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d7a7fc700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d840d24b0 0x7f6d841f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d837fe700 1 -- 172.21.15.97:0/1580455052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d840d24b0 msgr2=0x7f6d841f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d837fe700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d840d24b0 0x7f6d841f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d837fe700 1 -- 172.21.15.97:0/1580455052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d840b4870 msgr2=0x7f6d840b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d837fe700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d840b4870 0x7f6d840b7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d837fe700 1 -- 172.21.15.97:0/1580455052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d740094a0 con 0x7f6d8415c540 2023-07-06T03:48:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d7a7fc700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d840d24b0 0x7f6d841f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d837fe700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8415c540 0x7f6d84156160 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00cb40 tx=0x7f6d7c00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.412+0000 7f6d7bfff700 1 -- 172.21.15.97:0/1580455052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00aaa0 con 0x7f6d8415c540 2023-07-06T03:48:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.413+0000 7f6d7bfff700 1 -- 172.21.15.97:0/1580455052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c00ac00 con 0x7f6d8415c540 2023-07-06T03:48:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.413+0000 7f6d893ff700 1 -- 172.21.15.97:0/1580455052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8415ce50 con 0x7f6d8415c540 2023-07-06T03:48:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.413+0000 7f6d7bfff700 1 -- 172.21.15.97:0/1580455052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c012400 con 0x7f6d8415c540 2023-07-06T03:48:25.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.413+0000 7f6d893ff700 1 -- 172.21.15.97:0/1580455052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d841f3120 con 0x7f6d8415c540 2023-07-06T03:48:25.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.414+0000 7f6d893ff700 1 -- 172.21.15.97:0/1580455052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d840726a0 con 0x7f6d8415c540 2023-07-06T03:48:25.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.415+0000 7f6d7bfff700 1 -- 172.21.15.97:0/1580455052 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6d7c01b3b0 con 0x7f6d8415c540 2023-07-06T03:48:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.415+0000 7f6d7bfff700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d6806ace0 0x7f6d6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.415+0000 7f6d7bfff700 1 -- 172.21.15.97:0/1580455052 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f6d7c089310 con 0x7f6d8415c540 2023-07-06T03:48:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.416+0000 7f6d82ffd700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d6806ace0 0x7f6d6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.416+0000 7f6d82ffd700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d6806ace0 0x7f6d6806d190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6d740090b0 tx=0x7f6d7401d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.420+0000 7f6d7bfff700 1 -- 172.21.15.97:0/1580455052 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6d7c0542a0 con 0x7f6d8415c540 2023-07-06T03:48:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.615+0000 7f6d893ff700 1 -- 172.21.15.97:0/1580455052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d84074390 con 0x7f6d8415c540 2023-07-06T03:48:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.616+0000 7f6d7bfff700 1 -- 172.21.15.97:0/1580455052 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d7c0578c0 con 0x7f6d8415c540 2023-07-06T03:48:25.618 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:48:25.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 -- 172.21.15.97:0/1580455052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d6806ace0 msgr2=0x7f6d6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d6806ace0 0x7f6d6806d190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6d740090b0 tx=0x7f6d7401d040 comp rx=0 tx=0).stop 2023-07-06T03:48:25.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 -- 172.21.15.97:0/1580455052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8415c540 msgr2=0x7f6d84156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8415c540 0x7f6d84156160 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00cb40 tx=0x7f6d7c00ce50 comp rx=0 tx=0).stop 2023-07-06T03:48:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 -- 172.21.15.97:0/1580455052 shutdown_connections 2023-07-06T03:48:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d6806ace0 0x7f6d6806d190 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8415c540 0x7f6d84156160 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d840b4870 0x7f6d840b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 --2- 172.21.15.97:0/1580455052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d840d24b0 0x7f6d841f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 -- 172.21.15.97:0/1580455052 >> 172.21.15.97:0/1580455052 conn(0x7f6d8406b2f0 msgr2=0x7f6d8415b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 -- 172.21.15.97:0/1580455052 shutdown_connections 2023-07-06T03:48:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.618+0000 7f6d893ff700 1 -- 172.21.15.97:0/1580455052 wait complete. 2023-07-06T03:48:25.638 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:48:25.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.834+0000 7f6316fff700 1 Processor -- start 2023-07-06T03:48:25.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.834+0000 7f6316fff700 1 -- start start 2023-07-06T03:48:25.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.835+0000 7f6316fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f63100b4870 0x7f63100b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:25.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.835+0000 7f6316fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f63100d14a0 0x7f63100d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:25.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.835+0000 7f6316fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631015c910 con 0x7f631015c540 2023-07-06T03:48:25.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.835+0000 7f6316fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63100b5250 con 0x7f63100d14a0 2023-07-06T03:48:25.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.835+0000 7f6316fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63100cd770 con 0x7f63100b4870 2023-07-06T03:48:25.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.835+0000 7f631559c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f63100d14a0 0x7f63100d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.835+0000 7f631559c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f63100d14a0 0x7f63100d18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57618/0 (socket says 172.21.15.97:57618) 2023-07-06T03:48:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.835+0000 7f631559c700 1 -- 172.21.15.97:0/2909025472 learned_addr learned my addr 172.21.15.97:0/2909025472 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.836+0000 7f630ffff700 1 --2- 172.21.15.97:0/2909025472 >> v2:172.21.15.146:3300/0 conn(0x7f63100b4870 0x7f63100b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.836+0000 7f631559c700 1 -- 172.21.15.97:0/2909025472 >> v2:172.21.15.146:3300/0 conn(0x7f63100b4870 msgr2=0x7f63100b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:25.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.836+0000 7f631559c700 1 --2- 172.21.15.97:0/2909025472 >> v2:172.21.15.146:3300/0 conn(0x7f63100b4870 0x7f63100b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.836+0000 7f631559c700 1 -- 172.21.15.97:0/2909025472 >> v1:172.21.15.146:6789/0 conn(0x7f631015c540 legacy=0x7f6310160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:25.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.836+0000 7f631559c700 1 -- 172.21.15.97:0/2909025472 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63100cda50 con 0x7f63100d14a0 2023-07-06T03:48:25.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.836+0000 7f631559c700 1 --2- 172.21.15.97:0/2909025472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63100d14a0 0x7f63100d18e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6308009e50 tx=0x7f630800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939fa4485d38f96d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:25.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.836+0000 7f630f7fe700 1 -- 172.21.15.97:0/2909025472 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630800e070 con 0x7f63100d14a0 2023-07-06T03:48:25.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.836+0000 7f630f7fe700 1 -- 172.21.15.97:0/2909025472 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6308003a20 con 0x7f63100d14a0 2023-07-06T03:48:25.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.836+0000 7f630f7fe700 1 -- 172.21.15.97:0/2909025472 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308003dd0 con 0x7f63100d14a0 2023-07-06T03:48:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.837+0000 7f6316fff700 1 -- 172.21.15.97:0/2909025472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63100d14a0 msgr2=0x7f63100d18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.837+0000 7f6316fff700 1 --2- 172.21.15.97:0/2909025472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63100d14a0 0x7f63100d18e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6308009e50 tx=0x7f630800b330 comp rx=0 tx=0).stop 2023-07-06T03:48:25.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.837+0000 7f6316fff700 1 -- 172.21.15.97:0/2909025472 shutdown_connections 2023-07-06T03:48:25.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.837+0000 7f6316fff700 1 --2- 172.21.15.97:0/2909025472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63100d14a0 0x7f63100d18e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.837+0000 7f6316fff700 1 --2- 172.21.15.97:0/2909025472 >> v2:172.21.15.146:3300/0 conn(0x7f63100b4870 0x7f63100b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.837+0000 7f6316fff700 1 -- 172.21.15.97:0/2909025472 >> 172.21.15.97:0/2909025472 conn(0x7f631006b2f0 msgr2=0x7f63100cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:25.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.837+0000 7f6316fff700 1 -- 172.21.15.97:0/2909025472 shutdown_connections 2023-07-06T03:48:25.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.837+0000 7f6316fff700 1 -- 172.21.15.97:0/2909025472 wait complete. 2023-07-06T03:48:25.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.839+0000 7f6316fff700 1 Processor -- start 2023-07-06T03:48:25.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.839+0000 7f6316fff700 1 -- start start 2023-07-06T03:48:25.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f6316fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63100b4870 0x7f631015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f6316fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63100d14a0 0x7f6310150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f6316fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f631015c540 0x7f63100b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f6316fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100c6a20 con 0x7f63100d14a0 2023-07-06T03:48:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f6316fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100b75c0 con 0x7f63100b4870 2023-07-06T03:48:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f6316fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6310154710 con 0x7f631015c540 2023-07-06T03:48:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f630ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63100d14a0 0x7f6310150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f630ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63100d14a0 0x7f6310150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57622/0 (socket says 172.21.15.97:57622) 2023-07-06T03:48:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f630ffff700 1 -- 172.21.15.97:0/458567549 learned_addr learned my addr 172.21.15.97:0/458567549 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f6314d9b700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63100b4870 0x7f631015f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f631559c700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f631015c540 0x7f63100b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f630ffff700 1 -- 172.21.15.97:0/458567549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f631015c540 msgr2=0x7f63100b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f630ffff700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f631015c540 0x7f63100b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f630ffff700 1 -- 172.21.15.97:0/458567549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63100b4870 msgr2=0x7f631015f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:25.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f630ffff700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63100b4870 0x7f631015f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:25.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f630ffff700 1 -- 172.21.15.97:0/458567549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6308009b30 con 0x7f63100d14a0 2023-07-06T03:48:25.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f6314d9b700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63100b4870 0x7f631015f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:25.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.840+0000 7f631559c700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f631015c540 0x7f63100b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.841+0000 7f630ffff700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63100d14a0 0x7f6310150580 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00e810 tx=0x7f62fc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.841+0000 7f630d7fa700 1 -- 172.21.15.97:0/458567549 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc013610 con 0x7f63100d14a0 2023-07-06T03:48:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.841+0000 7f630d7fa700 1 -- 172.21.15.97:0/458567549 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc013770 con 0x7f63100d14a0 2023-07-06T03:48:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.841+0000 7f630d7fa700 1 -- 172.21.15.97:0/458567549 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc013b20 con 0x7f63100d14a0 2023-07-06T03:48:25.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.841+0000 7f6316fff700 1 -- 172.21.15.97:0/458567549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63101549f0 con 0x7f63100d14a0 2023-07-06T03:48:25.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.841+0000 7f6316fff700 1 -- 172.21.15.97:0/458567549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6310150ac0 con 0x7f63100d14a0 2023-07-06T03:48:25.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.842+0000 7f6316fff700 1 -- 172.21.15.97:0/458567549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63100726a0 con 0x7f63100d14a0 2023-07-06T03:48:25.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.843+0000 7f630d7fa700 1 -- 172.21.15.97:0/458567549 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f62fc013c80 con 0x7f63100d14a0 2023-07-06T03:48:25.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.845+0000 7f630d7fa700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62f406ace0 0x7f62f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:25.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.845+0000 7f630d7fa700 1 -- 172.21.15.97:0/458567549 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f62fc0892c0 con 0x7f63100d14a0 2023-07-06T03:48:25.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.845+0000 7f6314d9b700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62f406ace0 0x7f62f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:25.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.846+0000 7f6314d9b700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62f406ace0 0x7f62f406d190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6304006fd0 tx=0x7f6304008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:25.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:25.849+0000 7f630d7fa700 1 -- 172.21.15.97:0/458567549 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f62fc054260 con 0x7f63100d14a0 2023-07-06T03:48:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.045+0000 7f6316fff700 1 -- 172.21.15.97:0/458567549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63100c6b60 con 0x7f63100d14a0 2023-07-06T03:48:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.046+0000 7f630d7fa700 1 -- 172.21.15.97:0/458567549 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f63100c6b60 con 0x7f63100d14a0 2023-07-06T03:48:26.048 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:48:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 -- 172.21.15.97:0/458567549 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62f406ace0 msgr2=0x7f62f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62f406ace0 0x7f62f406d190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6304006fd0 tx=0x7f6304008040 comp rx=0 tx=0).stop 2023-07-06T03:48:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 -- 172.21.15.97:0/458567549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63100d14a0 msgr2=0x7f6310150580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63100d14a0 0x7f6310150580 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00e810 tx=0x7f62fc00ebd0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 -- 172.21.15.97:0/458567549 shutdown_connections 2023-07-06T03:48:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62f406ace0 0x7f62f406d190 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f631015c540 0x7f63100b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63100b4870 0x7f631015f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 --2- 172.21.15.97:0/458567549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63100d14a0 0x7f6310150580 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 -- 172.21.15.97:0/458567549 >> 172.21.15.97:0/458567549 conn(0x7f631006b2f0 msgr2=0x7f63100b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 -- 172.21.15.97:0/458567549 shutdown_connections 2023-07-06T03:48:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.048+0000 7f6316fff700 1 -- 172.21.15.97:0/458567549 wait complete. 2023-07-06T03:48:26.068 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:48:26.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.262+0000 7f8efc3ff700 1 Processor -- start 2023-07-06T03:48:26.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.262+0000 7f8efc3ff700 1 -- start start 2023-07-06T03:48:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.263+0000 7f8efc3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ef415c850 0x7f8ef40b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:26.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.263+0000 7f8efc3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ef40d1330 0x7f8ef40b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:26.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.263+0000 7f8efc3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef415cc20 con 0x7f8ef40b4db0 2023-07-06T03:48:26.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.263+0000 7f8efc3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef40d17b0 con 0x7f8ef415c850 2023-07-06T03:48:26.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.263+0000 7f8efc3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef40b5220 con 0x7f8ef40d1330 2023-07-06T03:48:26.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.263+0000 7f8efa19b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ef415c850 0x7f8ef40b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:26.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.263+0000 7f8efa19b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ef415c850 0x7f8ef40b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57638/0 (socket says 172.21.15.97:57638) 2023-07-06T03:48:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.263+0000 7f8efa19b700 1 -- 172.21.15.97:0/2557284833 learned_addr learned my addr 172.21.15.97:0/2557284833 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.264+0000 7f8efa99c700 1 --2- 172.21.15.97:0/2557284833 >> v2:172.21.15.146:3300/0 conn(0x7f8ef40d1330 0x7f8ef40b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.264+0000 7f8efa19b700 1 -- 172.21.15.97:0/2557284833 >> v2:172.21.15.146:3300/0 conn(0x7f8ef40d1330 msgr2=0x7f8ef40b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:26.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.264+0000 7f8efa19b700 1 --2- 172.21.15.97:0/2557284833 >> v2:172.21.15.146:3300/0 conn(0x7f8ef40d1330 0x7f8ef40b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.264+0000 7f8efa19b700 1 -- 172.21.15.97:0/2557284833 >> v1:172.21.15.146:6789/0 conn(0x7f8ef40b4db0 legacy=0x7f8ef40d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:26.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.264+0000 7f8efa19b700 1 -- 172.21.15.97:0/2557284833 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40d1990 con 0x7f8ef415c850 2023-07-06T03:48:26.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.264+0000 7f8efa19b700 1 --2- 172.21.15.97:0/2557284833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef415c850 0x7f8ef40b4870 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8ef408c060 tx=0x7f8ef0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430d7066c2683c81 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.264+0000 7f8ef9199700 1 -- 172.21.15.97:0/2557284833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000b870 con 0x7f8ef415c850 2023-07-06T03:48:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.264+0000 7f8ef9199700 1 -- 172.21.15.97:0/2557284833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef000b9d0 con 0x7f8ef415c850 2023-07-06T03:48:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.264+0000 7f8ef9199700 1 -- 172.21.15.97:0/2557284833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000bd60 con 0x7f8ef415c850 2023-07-06T03:48:26.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.265+0000 7f8efc3ff700 1 -- 172.21.15.97:0/2557284833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef415c850 msgr2=0x7f8ef40b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:26.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.265+0000 7f8efc3ff700 1 --2- 172.21.15.97:0/2557284833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef415c850 0x7f8ef40b4870 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8ef408c060 tx=0x7f8ef0009bd0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.265+0000 7f8efc3ff700 1 -- 172.21.15.97:0/2557284833 shutdown_connections 2023-07-06T03:48:26.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.265+0000 7f8efc3ff700 1 --2- 172.21.15.97:0/2557284833 >> v2:172.21.15.146:3300/0 conn(0x7f8ef40d1330 0x7f8ef40b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.265+0000 7f8efc3ff700 1 --2- 172.21.15.97:0/2557284833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef415c850 0x7f8ef40b4870 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.265+0000 7f8efc3ff700 1 -- 172.21.15.97:0/2557284833 >> 172.21.15.97:0/2557284833 conn(0x7f8ef40d2df0 msgr2=0x7f8ef40d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:26.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.265+0000 7f8efc3ff700 1 -- 172.21.15.97:0/2557284833 shutdown_connections 2023-07-06T03:48:26.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.265+0000 7f8efc3ff700 1 -- 172.21.15.97:0/2557284833 wait complete. 2023-07-06T03:48:26.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.267+0000 7f8efc3ff700 1 Processor -- start 2023-07-06T03:48:26.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.267+0000 7f8efc3ff700 1 -- start start 2023-07-06T03:48:26.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.268+0000 7f8efc3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef40b4db0 0x7f8ef41526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:26.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.268+0000 7f8efc3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ef40d1330 0x7f8ef40ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:26.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.268+0000 7f8efc3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef415c850 0x7f8ef415ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:26.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.268+0000 7f8efc3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4152be0 con 0x7f8ef40b4db0 2023-07-06T03:48:26.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.268+0000 7f8efc3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef415cde0 con 0x7f8ef415c850 2023-07-06T03:48:26.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.268+0000 7f8efc3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40cb070 con 0x7f8ef40d1330 2023-07-06T03:48:26.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.268+0000 7f8efa19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef40b4db0 0x7f8ef41526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:26.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.268+0000 7f8efa19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef40b4db0 0x7f8ef41526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57644/0 (socket says 172.21.15.97:57644) 2023-07-06T03:48:26.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.268+0000 7f8efa19b700 1 -- 172.21.15.97:0/1299146500 learned_addr learned my addr 172.21.15.97:0/1299146500 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:26.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.268+0000 7f8efa99c700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef415c850 0x7f8ef415ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.268+0000 7f8ef999a700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ef40d1330 0x7f8ef40ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8efa19b700 1 -- 172.21.15.97:0/1299146500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ef40d1330 msgr2=0x7f8ef40ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8efa19b700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ef40d1330 0x7f8ef40ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8efa19b700 1 -- 172.21.15.97:0/1299146500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef415c850 msgr2=0x7f8ef415ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8efa19b700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef415c850 0x7f8ef415ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8efa19b700 1 -- 172.21.15.97:0/1299146500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef0009550 con 0x7f8ef40b4db0 2023-07-06T03:48:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8ef999a700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ef40d1330 0x7f8ef40ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8efa19b700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef40b4db0 0x7f8ef41526a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0000c00 tx=0x7f8ef0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8eeaffd700 1 -- 172.21.15.97:0/1299146500 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0004520 con 0x7f8ef40b4db0 2023-07-06T03:48:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8eeaffd700 1 -- 172.21.15.97:0/1299146500 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef001e070 con 0x7f8ef40b4db0 2023-07-06T03:48:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8eeaffd700 1 -- 172.21.15.97:0/1299146500 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef00038a0 con 0x7f8ef40b4db0 2023-07-06T03:48:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8efc3ff700 1 -- 172.21.15.97:0/1299146500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef40cb350 con 0x7f8ef40b4db0 2023-07-06T03:48:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.269+0000 7f8efc3ff700 1 -- 172.21.15.97:0/1299146500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef40cb910 con 0x7f8ef40b4db0 2023-07-06T03:48:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.270+0000 7f8efc3ff700 1 -- 172.21.15.97:0/1299146500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef40726a0 con 0x7f8ef40b4db0 2023-07-06T03:48:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.271+0000 7f8eeaffd700 1 -- 172.21.15.97:0/1299146500 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8ef0003a00 con 0x7f8ef40b4db0 2023-07-06T03:48:26.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.272+0000 7f8eeaffd700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ed806ace0 0x7f8ed806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:26.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.272+0000 7f8eeaffd700 1 -- 172.21.15.97:0/1299146500 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f8ef0020070 con 0x7f8ef40b4db0 2023-07-06T03:48:26.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.272+0000 7f8ef999a700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ed806ace0 0x7f8ed806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:26.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.273+0000 7f8ef999a700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ed806ace0 0x7f8ed806d190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40b3cf0 tx=0x7f8ee4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:26.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.276+0000 7f8eeaffd700 1 -- 172.21.15.97:0/1299146500 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8ef0055b60 con 0x7f8ef40b4db0 2023-07-06T03:48:26.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.485+0000 7f8efc3ff700 1 -- 172.21.15.97:0/1299146500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8ef4074390 con 0x7f8ef40b4db0 2023-07-06T03:48:26.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.485+0000 7f8eeaffd700 1 -- 172.21.15.97:0/1299146500 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8ef0015080 con 0x7f8ef40b4db0 2023-07-06T03:48:26.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.487+0000 7f8efc3ff700 1 -- 172.21.15.97:0/1299146500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ed806ace0 msgr2=0x7f8ed806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:26.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.487+0000 7f8efc3ff700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ed806ace0 0x7f8ed806d190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40b3cf0 tx=0x7f8ee4009040 comp rx=0 tx=0).stop 2023-07-06T03:48:26.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.487+0000 7f8efc3ff700 1 -- 172.21.15.97:0/1299146500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef40b4db0 msgr2=0x7f8ef41526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:26.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.487+0000 7f8efc3ff700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef40b4db0 0x7f8ef41526a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0000c00 tx=0x7f8ef0004750 comp rx=0 tx=0).stop 2023-07-06T03:48:26.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.487+0000 7f8efc3ff700 1 -- 172.21.15.97:0/1299146500 shutdown_connections 2023-07-06T03:48:26.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.487+0000 7f8efc3ff700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8ed806ace0 0x7f8ed806d190 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.487+0000 7f8efc3ff700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ef415c850 0x7f8ef415ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.487+0000 7f8efc3ff700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef40b4db0 0x7f8ef41526a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.487+0000 7f8efc3ff700 1 --2- 172.21.15.97:0/1299146500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8ef40d1330 0x7f8ef40ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:26.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.487+0000 7f8efc3ff700 1 -- 172.21.15.97:0/1299146500 >> 172.21.15.97:0/1299146500 conn(0x7f8ef40d2df0 msgr2=0x7f8ef415b340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:26.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.488+0000 7f8efc3ff700 1 -- 172.21.15.97:0/1299146500 shutdown_connections 2023-07-06T03:48:26.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:26.488+0000 7f8efc3ff700 1 -- 172.21.15.97:0/1299146500 wait complete. 2023-07-06T03:48:26.493 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:48:26.507 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2023-07-06T03:48:26.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:26.508 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-06T03:48:26.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[121319]: fuse finished with error 0 and tester_r 0 2023-07-06T03:48:26.585 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:48:26.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:26.586 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T03:48:26.693 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:48:26.694 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:48:26.741 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:48:26.741 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:48:26.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:26.742 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T03:48:26.826 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T03:48:26.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:48:26.828 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:48:26.828 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:48:26.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:48:26.875 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:48:26.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:26.876 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T03:48:26.960 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T03:48:26.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:48:26.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:48:27.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.155+0000 7fd8ee7ff700 1 Processor -- start 2023-07-06T03:48:27.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.156+0000 7fd8ee7ff700 1 -- start start 2023-07-06T03:48:27.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8ee7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8e815c850 0x7fd8e80b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8ee7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd8e80d1330 0x7fd8e80b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8ee7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e815cc20 con 0x7fd8e80b4db0 2023-07-06T03:48:27.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8ee7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e80d1780 con 0x7fd8e80d1330 2023-07-06T03:48:27.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8ee7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80b5220 con 0x7fd8e815c850 2023-07-06T03:48:27.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8ecd9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd8e80d1330 0x7fd8e80b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8dffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd8e80b4db0 0x7fd8e80d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54948/0 (socket says 172.21.15.97:54948) 2023-07-06T03:48:27.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8dffff700 1 -- 172.21.15.97:0/2690438371 learned_addr learned my addr 172.21.15.97:0/2690438371 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:27.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8e7fff700 1 --2- 172.21.15.97:0/2690438371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8e815c850 0x7fd8e80b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8ecd9c700 1 -- 172.21.15.97:0/2690438371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8e815c850 msgr2=0x7fd8e80b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:27.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8ecd9c700 1 --2- 172.21.15.97:0/2690438371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8e815c850 0x7fd8e80b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8ecd9c700 1 -- 172.21.15.97:0/2690438371 >> v1:172.21.15.97:6789/0 conn(0x7fd8e80b4db0 legacy=0x7fd8e80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.157+0000 7fd8ecd9c700 1 -- 172.21.15.97:0/2690438371 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e80d1990 con 0x7fd8e80d1330 2023-07-06T03:48:27.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.158+0000 7fd8ecd9c700 1 --2- 172.21.15.97:0/2690438371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e80d1330 0x7fd8e80b6300 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009d90 tx=0x7fd8e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e7b43ff03bf23 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:27.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.158+0000 7fd8e77fe700 1 -- 172.21.15.97:0/2690438371 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000e070 con 0x7fd8e80d1330 2023-07-06T03:48:27.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.158+0000 7fd8e77fe700 1 -- 172.21.15.97:0/2690438371 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e000bbd0 con 0x7fd8e80d1330 2023-07-06T03:48:27.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.158+0000 7fd8e77fe700 1 -- 172.21.15.97:0/2690438371 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0012430 con 0x7fd8e80d1330 2023-07-06T03:48:27.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.158+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/2690438371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e80d1330 msgr2=0x7fd8e80b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.158+0000 7fd8ee7ff700 1 --2- 172.21.15.97:0/2690438371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e80d1330 0x7fd8e80b6300 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009d90 tx=0x7fd8e000b330 comp rx=0 tx=0).stop 2023-07-06T03:48:27.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.158+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/2690438371 shutdown_connections 2023-07-06T03:48:27.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.158+0000 7fd8ee7ff700 1 --2- 172.21.15.97:0/2690438371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e80d1330 0x7fd8e80b6300 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.158+0000 7fd8ee7ff700 1 --2- 172.21.15.97:0/2690438371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8e815c850 0x7fd8e80b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.158+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/2690438371 >> 172.21.15.97:0/2690438371 conn(0x7fd8e80d2df0 msgr2=0x7fd8e80d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.159+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/2690438371 shutdown_connections 2023-07-06T03:48:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.159+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/2690438371 wait complete. 2023-07-06T03:48:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.160+0000 7fd8ee7ff700 1 Processor -- start 2023-07-06T03:48:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.160+0000 7fd8ee7ff700 1 -- start start 2023-07-06T03:48:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.161+0000 7fd8ee7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8e80b4db0 0x7fd8e80c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.161+0000 7fd8ee7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8e80d1330 0x7fd8e8156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.161+0000 7fd8ee7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e815c850 0x7fd8e81e5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.161+0000 7fd8ee7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80c6870 con 0x7fd8e815c850 2023-07-06T03:48:27.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.161+0000 7fd8ee7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80c6b50 con 0x7fd8e80b4db0 2023-07-06T03:48:27.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.161+0000 7fd8ee7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80c6e30 con 0x7fd8e80d1330 2023-07-06T03:48:27.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8ecd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e815c850 0x7fd8e81e5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8ecd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e815c850 0x7fd8e81e5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57672/0 (socket says 172.21.15.97:57672) 2023-07-06T03:48:27.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8ecd9c700 1 -- 172.21.15.97:0/3260595318 learned_addr learned my addr 172.21.15.97:0/3260595318 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:27.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8dffff700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8e80d1330 0x7fd8e8156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8e7fff700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8e80b4db0 0x7fd8e80c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8ecd9c700 1 -- 172.21.15.97:0/3260595318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8e80d1330 msgr2=0x7fd8e8156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8ecd9c700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8e80d1330 0x7fd8e8156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8ecd9c700 1 -- 172.21.15.97:0/3260595318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8e80b4db0 msgr2=0x7fd8e80c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8ecd9c700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8e80b4db0 0x7fd8e80c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8ecd9c700 1 -- 172.21.15.97:0/3260595318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e0009a70 con 0x7fd8e815c850 2023-07-06T03:48:27.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8e7fff700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8e80b4db0 0x7fd8e80c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:27.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8ecd9c700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e815c850 0x7fd8e81e5d30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0014040 tx=0x7fd8e0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:27.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.162+0000 7fd8e57fa700 1 -- 172.21.15.97:0/3260595318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0003ea0 con 0x7fd8e815c850 2023-07-06T03:48:27.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.163+0000 7fd8e57fa700 1 -- 172.21.15.97:0/3260595318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e0020070 con 0x7fd8e815c850 2023-07-06T03:48:27.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.163+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/3260595318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e80c7110 con 0x7fd8e815c850 2023-07-06T03:48:27.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.163+0000 7fd8e57fa700 1 -- 172.21.15.97:0/3260595318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0004280 con 0x7fd8e815c850 2023-07-06T03:48:27.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.163+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/3260595318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e8156a70 con 0x7fd8e815c850 2023-07-06T03:48:27.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.164+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/3260595318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e80726a0 con 0x7fd8e815c850 2023-07-06T03:48:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.165+0000 7fd8e57fa700 1 -- 172.21.15.97:0/3260595318 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd8e00043e0 con 0x7fd8e815c850 2023-07-06T03:48:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.165+0000 7fd8e57fa700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8c806ace0 0x7fd8c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.166+0000 7fd8e57fa700 1 -- 172.21.15.97:0/3260595318 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7fd8e0022070 con 0x7fd8e815c850 2023-07-06T03:48:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.166+0000 7fd8e7fff700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8c806ace0 0x7fd8c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.166+0000 7fd8e7fff700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8c806ace0 0x7fd8c806d190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8007a50 tx=0x7fd8d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.170+0000 7fd8e57fa700 1 -- 172.21.15.97:0/3260595318 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd8e00562f0 con 0x7fd8e815c850 2023-07-06T03:48:27.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.408+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/3260595318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8e80c7630 con 0x7fd8e815c850 2023-07-06T03:48:27.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.409+0000 7fd8e57fa700 1 -- 172.21.15.97:0/3260595318 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fd8e80c7630 con 0x7fd8e815c850 2023-07-06T03:48:27.411 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:27.411 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1299982704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1299982704},{"type":"v1","addr":"172.21.15.146:6833","nonce":1299982704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5602,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/640394657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":640394657},{"type":"v1","addr":"172.21.15.97:6837","nonce":640394657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2363940464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2363940464},{"type":"v1","addr":"172.21.15.97:6836","nonce":2363940464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:47:36.149445+0000","modified":"2023-07-06T03:47:39.683749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5838},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5838":{"gid":5838,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/3623506860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3623506860},{"type":"v1","addr":"172.21.15.146:6835","nonce":3623506860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T03:48:27.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/3260595318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8c806ace0 msgr2=0x7fd8c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8c806ace0 0x7fd8c806d190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8007a50 tx=0x7fd8d8008040 comp rx=0 tx=0).stop 2023-07-06T03:48:27.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/3260595318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e815c850 msgr2=0x7fd8e81e5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e815c850 0x7fd8e81e5d30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0014040 tx=0x7fd8e0004ad0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/3260595318 shutdown_connections 2023-07-06T03:48:27.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8c806ace0 0x7fd8c806d190 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e815c850 0x7fd8e81e5d30 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8e80b4db0 0x7fd8e80c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 --2- 172.21.15.97:0/3260595318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8e80d1330 0x7fd8e8156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/3260595318 >> 172.21.15.97:0/3260595318 conn(0x7fd8e80d2df0 msgr2=0x7fd8e80d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:27.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/3260595318 shutdown_connections 2023-07-06T03:48:27.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.411+0000 7fd8ee7ff700 1 -- 172.21.15.97:0/3260595318 wait complete. 2023-07-06T03:48:27.417 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2023-07-06T03:48:27.431 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:48:27.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.623+0000 7f36009ff700 1 Processor -- start 2023-07-06T03:48:27.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.623+0000 7f36009ff700 1 -- start start 2023-07-06T03:48:27.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.624+0000 7f36009ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cd440 0x7f35fc160b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.625+0000 7f36009ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc161060 con 0x7f35fc15c740 2023-07-06T03:48:27.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.625+0000 7f36009ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc0cdcc0 con 0x7f35fc06a680 2023-07-06T03:48:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.625+0000 7f36009ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0cd9e0 con 0x7f35fc0cd440 2023-07-06T03:48:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.625+0000 7f35fa59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f35fc15c740 0x7f35fc0b5cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54964/0 (socket says 172.21.15.97:54964) 2023-07-06T03:48:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.625+0000 7f35fa59c700 1 -- 172.21.15.97:0/78734364 learned_addr learned my addr 172.21.15.97:0/78734364 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:27.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.625+0000 7f35f1d9b700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cd440 0x7f35fc160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.625+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491858347 0 0) 0x7f35fc161060 con 0x7f35fc15c740 2023-07-06T03:48:27.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.625+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35e80035b0 con 0x7f35fc15c740 2023-07-06T03:48:27.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.625+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894237914 0 0) 0x7f35fc0cdcc0 con 0x7f35fc06a680 2023-07-06T03:48:27.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.625+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35fc161060 con 0x7f35fc06a680 2023-07-06T03:48:27.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1923016203 0 0) 0x7f35e80035b0 con 0x7f35fc15c740 2023-07-06T03:48:27.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cd440 msgr2=0x7f35fc160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:27.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cd440 0x7f35fc160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 >> v1:172.21.15.146:6789/0 conn(0x7f35fc06a680 legacy=0x7f35fc0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0d0b60 con 0x7f35fc15c740 2023-07-06T03:48:27.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f35ec0019f0 con 0x7f35fc15c740 2023-07-06T03:48:27.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 >> v1:172.21.15.97:6789/0 conn(0x7f35fc15c740 legacy=0x7f35fc0b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35e8003bc0 0x7f35e8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 0x7f35e8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cd440 0x7f35e8005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0cd9e0 con 0x7f35e8004850 2023-07-06T03:48:27.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e800bba0 con 0x7f35e8003bc0 2023-07-06T03:48:27.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e800bd60 con 0x7f35fc0cd440 2023-07-06T03:48:27.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f1d9b700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 0x7f35e8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35fa59c700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35e8003bc0 0x7f35e8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35fad9d700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cd440 0x7f35e8005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f1d9b700 1 -- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cd440 msgr2=0x7f35e8005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:27.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f1d9b700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cd440 0x7f35e8005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f1d9b700 1 -- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35e8003bc0 msgr2=0x7f35e8003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:27.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f1d9b700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35e8003bc0 0x7f35e8003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35f1d9b700 1 -- 172.21.15.97:0/78734364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e80035b0 con 0x7f35e8004850 2023-07-06T03:48:27.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.626+0000 7f35fad9d700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cd440 0x7f35e8005260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:27.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.627+0000 7f35f1d9b700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 0x7f35e8004cc0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009b30 tx=0x7f35e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a0e58d95fd2f05 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:27.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.627+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400e6c0 con 0x7f35e8004850 2023-07-06T03:48:27.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.627+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e400e820 con 0x7f35e8004850 2023-07-06T03:48:27.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.627+0000 7f35f9d9b700 1 -- 172.21.15.97:0/78734364 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400eb90 con 0x7f35e8004850 2023-07-06T03:48:27.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.627+0000 7f36009ff700 1 -- 172.21.15.97:0/78734364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 msgr2=0x7f35e8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.627+0000 7f36009ff700 1 --2- 172.21.15.97:0/78734364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 0x7f35e8004cc0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009b30 tx=0x7f35e8005880 comp rx=0 tx=0).stop 2023-07-06T03:48:27.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.627+0000 7f35fad9d700 1 -- 172.21.15.97:0/78734364 reap_dead start 2023-07-06T03:48:27.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.627+0000 7f36009ff700 1 -- 172.21.15.97:0/78734364 shutdown_connections 2023-07-06T03:48:27.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.627+0000 7f36009ff700 1 -- 172.21.15.97:0/78734364 >> 172.21.15.97:0/78734364 conn(0x7f35fc0d2e80 msgr2=0x7f35fc0d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:27.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.627+0000 7f36009ff700 1 -- 172.21.15.97:0/78734364 shutdown_connections 2023-07-06T03:48:27.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.627+0000 7f36009ff700 1 -- 172.21.15.97:0/78734364 wait complete. 2023-07-06T03:48:27.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.629+0000 7f36009ff700 1 Processor -- start 2023-07-06T03:48:27.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.629+0000 7f36009ff700 1 -- start start 2023-07-06T03:48:27.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.630+0000 7f36009ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 0x7f35fc0cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.630+0000 7f36009ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35fc0c70c0 0x7f35fc14a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.630+0000 7f36009ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cb630 0x7f35fc0b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.630+0000 7f36009ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1f5950 con 0x7f35e8004850 2023-07-06T03:48:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.630+0000 7f36009ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1f5c30 con 0x7f35fc0c70c0 2023-07-06T03:48:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.630+0000 7f36009ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1f5f10 con 0x7f35fc0cb630 2023-07-06T03:48:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.630+0000 7f35fa59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 0x7f35fc0cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.630+0000 7f35fa59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 0x7f35fc0cbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57686/0 (socket says 172.21.15.97:57686) 2023-07-06T03:48:27.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.630+0000 7f35fa59c700 1 -- 172.21.15.97:0/4223736091 learned_addr learned my addr 172.21.15.97:0/4223736091 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:27.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35f1d9b700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35fc0c70c0 0x7f35fc14a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35fa59c700 1 -- 172.21.15.97:0/4223736091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cb630 msgr2=0x7f35fc0b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35fad9d700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cb630 0x7f35fc0b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35fa59c700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cb630 0x7f35fc0b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35fa59c700 1 -- 172.21.15.97:0/4223736091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35fc0c70c0 msgr2=0x7f35fc14a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35fa59c700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35fc0c70c0 0x7f35fc14a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35fa59c700 1 -- 172.21.15.97:0/4223736091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc1f61f0 con 0x7f35e8004850 2023-07-06T03:48:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35f1d9b700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35fc0c70c0 0x7f35fc14a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35fad9d700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cb630 0x7f35fc0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35fa59c700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 0x7f35fc0cbb10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00a460 tx=0x7f35ec00a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35f37fe700 1 -- 172.21.15.97:0/4223736091 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0133f0 con 0x7f35e8004850 2023-07-06T03:48:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35f37fe700 1 -- 172.21.15.97:0/4223736091 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ec013570 con 0x7f35e8004850 2023-07-06T03:48:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f35f37fe700 1 -- 172.21.15.97:0/4223736091 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec013920 con 0x7f35e8004850 2023-07-06T03:48:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f36009ff700 1 -- 172.21.15.97:0/4223736091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc1f64d0 con 0x7f35e8004850 2023-07-06T03:48:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.631+0000 7f36009ff700 1 -- 172.21.15.97:0/4223736091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc14a940 con 0x7f35e8004850 2023-07-06T03:48:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.632+0000 7f36009ff700 1 -- 172.21.15.97:0/4223736091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc1f66c0 con 0x7f35e8004850 2023-07-06T03:48:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.633+0000 7f35f37fe700 1 -- 172.21.15.97:0/4223736091 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35ec013a80 con 0x7f35e8004850 2023-07-06T03:48:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.634+0000 7f35f37fe700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35dc06ace0 0x7f35dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.634+0000 7f35f37fe700 1 -- 172.21.15.97:0/4223736091 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f35ec08a450 con 0x7f35e8004850 2023-07-06T03:48:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.635+0000 7f35f1d9b700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35dc06ace0 0x7f35dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.635+0000 7f35f1d9b700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35dc06ace0 0x7f35dc06d190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009a80 tx=0x7f35e401e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.639+0000 7f35f37fe700 1 -- 172.21.15.97:0/4223736091 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f35ec055480 con 0x7f35e8004850 2023-07-06T03:48:27.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.838+0000 7f36009ff700 1 -- 172.21.15.97:0/4223736091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35fc074390 con 0x7f35e8004850 2023-07-06T03:48:27.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.840+0000 7f35f37fe700 1 -- 172.21.15.97:0/4223736091 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18524 (secure 0 0 0) 0x7f35ec017090 con 0x7f35e8004850 2023-07-06T03:48:27.842 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:27.842 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":49,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:47:52.725520+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:47:33.695348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-06T03:47:35.132699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":2,"to":5}]},{"pgid":"8.1a","mappings":[{"from":2,"to":1}]},{"pgid":"8.38","mappings":[{"from":6,"to":5}]},{"pgid":"8.3d","mappings":[{"from":6,"to":0},{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:48:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 -- 172.21.15.97:0/4223736091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35dc06ace0 msgr2=0x7f35dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35dc06ace0 0x7f35dc06d190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009a80 tx=0x7f35e401e040 comp rx=0 tx=0).stop 2023-07-06T03:48:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 -- 172.21.15.97:0/4223736091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 msgr2=0x7f35fc0cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 0x7f35fc0cbb10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00a460 tx=0x7f35ec00a820 comp rx=0 tx=0).stop 2023-07-06T03:48:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 -- 172.21.15.97:0/4223736091 shutdown_connections 2023-07-06T03:48:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35dc06ace0 0x7f35dc06d190 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35fc0cb630 0x7f35fc0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35e8004850 0x7f35fc0cbb10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 --2- 172.21.15.97:0/4223736091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35fc0c70c0 0x7f35fc14a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 -- 172.21.15.97:0/4223736091 >> 172.21.15.97:0/4223736091 conn(0x7f35fc0d2e80 msgr2=0x7f35fc0cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 -- 172.21.15.97:0/4223736091 shutdown_connections 2023-07-06T03:48:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:27.842+0000 7f36009ff700 1 -- 172.21.15.97:0/4223736091 wait complete. 2023-07-06T03:48:27.863 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:47:33.695348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:47:35.132699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:48:27.864 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:48:28.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.059+0000 7f73f2fff700 1 Processor -- start 2023-07-06T03:48:28.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.059+0000 7f73f2fff700 1 -- start start 2023-07-06T03:48:28.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.060+0000 7f73f2fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73ec0cb350 0x7f73ec0ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.060+0000 7f73f2fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ec0c9a60 con 0x7f73ec0cb720 2023-07-06T03:48:28.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.060+0000 7f73f2fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ec0cc5d0 con 0x7f73ec0c38a0 2023-07-06T03:48:28.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.060+0000 7f73f2fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec0cce70 con 0x7f73ec0cb350 2023-07-06T03:48:28.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.061+0000 7f73ebfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f73ec0cb720 0x7f73ec0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54968/0 (socket says 172.21.15.97:54968) 2023-07-06T03:48:28.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.061+0000 7f73ebfff700 1 -- 172.21.15.97:0/2397857225 learned_addr learned my addr 172.21.15.97:0/2397857225 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:28.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.061+0000 7f73f0d9b700 1 --2- 172.21.15.97:0/2397857225 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73ec0cb350 0x7f73ec0ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.061+0000 7f73eb7fe700 1 -- 172.21.15.97:0/2397857225 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991044430 0 0) 0x7f73ec0c9a60 con 0x7f73ec0cb720 2023-07-06T03:48:28.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.061+0000 7f73eb7fe700 1 -- 172.21.15.97:0/2397857225 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73dc0035b0 con 0x7f73ec0cb720 2023-07-06T03:48:28.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.061+0000 7f73eb7fe700 1 -- 172.21.15.97:0/2397857225 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075773915 0 0) 0x7f73ec0cc5d0 con 0x7f73ec0c38a0 2023-07-06T03:48:28.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.061+0000 7f73eb7fe700 1 -- 172.21.15.97:0/2397857225 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73ec0c9a60 con 0x7f73ec0c38a0 2023-07-06T03:48:28.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.061+0000 7f73f0d9b700 1 -- 172.21.15.97:0/2397857225 >> v1:172.21.15.146:6789/0 conn(0x7f73ec0c38a0 legacy=0x7f73ec0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.061+0000 7f73f0d9b700 1 -- 172.21.15.97:0/2397857225 >> v1:172.21.15.97:6789/0 conn(0x7f73ec0cb720 legacy=0x7f73ec0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.061+0000 7f73eb7fe700 1 -- 172.21.15.97:0/2397857225 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3155353033 0 0) 0x7f73dc0035b0 con 0x7f73ec0cb720 2023-07-06T03:48:28.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.061+0000 7f73f0d9b700 1 -- 172.21.15.97:0/2397857225 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec0cc8b0 con 0x7f73ec0cb350 2023-07-06T03:48:28.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.062+0000 7f73f0d9b700 1 --2- 172.21.15.97:0/2397857225 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73ec0cb350 0x7f73ec0ce4c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009870 tx=0x7f73e0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33fe18d90e2af25b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:28.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.062+0000 7f73eb7fe700 1 -- 172.21.15.97:0/2397857225 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8001670 con 0x7f73ec0cb350 2023-07-06T03:48:28.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.062+0000 7f73eb7fe700 1 -- 172.21.15.97:0/2397857225 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e0004030 con 0x7f73ec0cb350 2023-07-06T03:48:28.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.062+0000 7f73eb7fe700 1 -- 172.21.15.97:0/2397857225 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e00043c0 con 0x7f73ec0cb350 2023-07-06T03:48:28.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.063+0000 7f73f2fff700 1 -- 172.21.15.97:0/2397857225 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73ec0cb350 msgr2=0x7f73ec0ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.063+0000 7f73f2fff700 1 --2- 172.21.15.97:0/2397857225 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73ec0cb350 0x7f73ec0ce4c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009870 tx=0x7f73e0009b80 comp rx=0 tx=0).stop 2023-07-06T03:48:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.063+0000 7f73f2fff700 1 -- 172.21.15.97:0/2397857225 shutdown_connections 2023-07-06T03:48:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.063+0000 7f73f2fff700 1 --2- 172.21.15.97:0/2397857225 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73ec0cb350 0x7f73ec0ce4c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.063+0000 7f73f2fff700 1 -- 172.21.15.97:0/2397857225 >> 172.21.15.97:0/2397857225 conn(0x7f73ec069fe0 msgr2=0x7f73ec06a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.063+0000 7f73f2fff700 1 -- 172.21.15.97:0/2397857225 shutdown_connections 2023-07-06T03:48:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.063+0000 7f73f2fff700 1 -- 172.21.15.97:0/2397857225 wait complete. 2023-07-06T03:48:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.064+0000 7f73f2fff700 1 Processor -- start 2023-07-06T03:48:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.065+0000 7f73f2fff700 1 -- start start 2023-07-06T03:48:28.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.065+0000 7f73f2fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec0c38a0 0x7f73ec15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.065+0000 7f73f2fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73ec0cb350 0x7f73ec157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.065+0000 7f73f2fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73ec0cb720 0x7f73ec070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f2fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec0b1c30 con 0x7f73ec0c38a0 2023-07-06T03:48:28.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f2fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec15aa70 con 0x7f73ec0cb720 2023-07-06T03:48:28.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f2fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec15b380 con 0x7f73ec0cb350 2023-07-06T03:48:28.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f0d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec0c38a0 0x7f73ec15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f0d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec0c38a0 0x7f73ec15a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57696/0 (socket says 172.21.15.97:57696) 2023-07-06T03:48:28.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f0d9b700 1 -- 172.21.15.97:0/3832347840 learned_addr learned my addr 172.21.15.97:0/3832347840 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:28.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f159c700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73ec0cb720 0x7f73ec070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73ebfff700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73ec0cb350 0x7f73ec157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f0d9b700 1 -- 172.21.15.97:0/3832347840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73ec0cb350 msgr2=0x7f73ec157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f0d9b700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73ec0cb350 0x7f73ec157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f0d9b700 1 -- 172.21.15.97:0/3832347840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73ec0cb720 msgr2=0x7f73ec070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f0d9b700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73ec0cb720 0x7f73ec070490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f0d9b700 1 -- 172.21.15.97:0/3832347840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e00094a0 con 0x7f73ec0c38a0 2023-07-06T03:48:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f159c700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73ec0cb720 0x7f73ec070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.066+0000 7f73f0d9b700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec0c38a0 0x7f73ec15a530 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f73e0000c00 tx=0x7f73e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.067+0000 7f73e97fa700 1 -- 172.21.15.97:0/3832347840 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0011630 con 0x7f73ec0c38a0 2023-07-06T03:48:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.067+0000 7f73e97fa700 1 -- 172.21.15.97:0/3832347840 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e001f070 con 0x7f73ec0c38a0 2023-07-06T03:48:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.067+0000 7f73e97fa700 1 -- 172.21.15.97:0/3832347840 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e00119f0 con 0x7f73ec0c38a0 2023-07-06T03:48:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.067+0000 7f73f2fff700 1 -- 172.21.15.97:0/3832347840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ec15adc0 con 0x7f73ec0c38a0 2023-07-06T03:48:28.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.067+0000 7f73f2fff700 1 -- 172.21.15.97:0/3832347840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ec06fc90 con 0x7f73ec0c38a0 2023-07-06T03:48:28.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.068+0000 7f73f2fff700 1 -- 172.21.15.97:0/3832347840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ec074390 con 0x7f73ec0c38a0 2023-07-06T03:48:28.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.069+0000 7f73e97fa700 1 -- 172.21.15.97:0/3832347840 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f73e0011b50 con 0x7f73ec0c38a0 2023-07-06T03:48:28.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.070+0000 7f73e97fa700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73d006ace0 0x7f73d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.070+0000 7f73e97fa700 1 -- 172.21.15.97:0/3832347840 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f73e0021070 con 0x7f73ec0c38a0 2023-07-06T03:48:28.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.070+0000 7f73ebfff700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73d006ace0 0x7f73d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.071+0000 7f73ebfff700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73d006ace0 0x7f73d006d190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0af600 tx=0x7f73d8001490 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:28.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.074+0000 7f73e97fa700 1 -- 172.21.15.97:0/3832347840 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f73e0090020 con 0x7f73ec0c38a0 2023-07-06T03:48:28.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.266+0000 7f73f2fff700 1 -- 172.21.15.97:0/3832347840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73ec06ac40 con 0x7f73ec0c38a0 2023-07-06T03:48:28.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.266+0000 7f73e97fa700 1 -- 172.21.15.97:0/3832347840 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73e002b0a0 con 0x7f73ec0c38a0 2023-07-06T03:48:28.268 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:48:28.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 -- 172.21.15.97:0/3832347840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73d006ace0 msgr2=0x7f73d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73d006ace0 0x7f73d006d190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0af600 tx=0x7f73d8001490 comp rx=0 tx=0).stop 2023-07-06T03:48:28.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 -- 172.21.15.97:0/3832347840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec0c38a0 msgr2=0x7f73ec15a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec0c38a0 0x7f73ec15a530 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f73e0000c00 tx=0x7f73e0004750 comp rx=0 tx=0).stop 2023-07-06T03:48:28.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 -- 172.21.15.97:0/3832347840 shutdown_connections 2023-07-06T03:48:28.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f73d006ace0 0x7f73d006d190 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73ec0cb720 0x7f73ec070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec0c38a0 0x7f73ec15a530 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 --2- 172.21.15.97:0/3832347840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f73ec0cb350 0x7f73ec157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 -- 172.21.15.97:0/3832347840 >> 172.21.15.97:0/3832347840 conn(0x7f73ec069fe0 msgr2=0x7f73ec0c42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 -- 172.21.15.97:0/3832347840 shutdown_connections 2023-07-06T03:48:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.269+0000 7f73f2fff700 1 -- 172.21.15.97:0/3832347840 wait complete. 2023-07-06T03:48:28.289 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:48:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.484+0000 7f107a1ff700 1 Processor -- start 2023-07-06T03:48:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.484+0000 7f107a1ff700 1 -- start start 2023-07-06T03:48:28.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.485+0000 7f107a1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f107415c540 0x7f1074160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.485+0000 7f107a1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f10740b4870 0x7f10740b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.485+0000 7f107a1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107415c910 con 0x7f107406c090 2023-07-06T03:48:28.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.485+0000 7f107a1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10740b5250 con 0x7f107415c540 2023-07-06T03:48:28.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.485+0000 7f107a1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1074160fc0 con 0x7f10740b4870 2023-07-06T03:48:28.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.486+0000 7f10737fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f107415c540 0x7f1074160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.486+0000 7f10737fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f107415c540 0x7f1074160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57706/0 (socket says 172.21.15.97:57706) 2023-07-06T03:48:28.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.486+0000 7f1073fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f107406c090 0x7f107406c4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54980/0 (socket says 172.21.15.97:54980) 2023-07-06T03:48:28.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.486+0000 7f10737fe700 1 -- 172.21.15.97:0/3106298499 learned_addr learned my addr 172.21.15.97:0/3106298499 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:28.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.486+0000 7f1072ffd700 1 --2- 172.21.15.97:0/3106298499 >> v2:172.21.15.146:3300/0 conn(0x7f10740b4870 0x7f10740b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.486+0000 7f10737fe700 1 -- 172.21.15.97:0/3106298499 >> v2:172.21.15.146:3300/0 conn(0x7f10740b4870 msgr2=0x7f10740b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.486+0000 7f10737fe700 1 --2- 172.21.15.97:0/3106298499 >> v2:172.21.15.146:3300/0 conn(0x7f10740b4870 0x7f10740b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.486+0000 7f10737fe700 1 -- 172.21.15.97:0/3106298499 >> v1:172.21.15.97:6789/0 conn(0x7f107406c090 legacy=0x7f107406c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.486+0000 7f10737fe700 1 -- 172.21.15.97:0/3106298499 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10740cd770 con 0x7f107415c540 2023-07-06T03:48:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.486+0000 7f10727fc700 1 -- 172.21.15.97:0/3106298499 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724767697 0 0) 0x7f107415c910 con 0x7f107406c090 2023-07-06T03:48:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.486+0000 7f10737fe700 1 --2- 172.21.15.97:0/3106298499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f107415c540 0x7f1074160a80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f10640098f0 tx=0x7f1064009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20af4d0b27198b01 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.487+0000 7f10727fc700 1 -- 172.21.15.97:0/3106298499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064011830 con 0x7f107415c540 2023-07-06T03:48:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.487+0000 7f10727fc700 1 -- 172.21.15.97:0/3106298499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1064011990 con 0x7f107415c540 2023-07-06T03:48:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.487+0000 7f10727fc700 1 -- 172.21.15.97:0/3106298499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064011d20 con 0x7f107415c540 2023-07-06T03:48:28.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.487+0000 7f107a1ff700 1 -- 172.21.15.97:0/3106298499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f107415c540 msgr2=0x7f1074160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.487+0000 7f107a1ff700 1 --2- 172.21.15.97:0/3106298499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f107415c540 0x7f1074160a80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f10640098f0 tx=0x7f1064009c00 comp rx=0 tx=0).stop 2023-07-06T03:48:28.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.487+0000 7f107a1ff700 1 -- 172.21.15.97:0/3106298499 shutdown_connections 2023-07-06T03:48:28.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.487+0000 7f107a1ff700 1 --2- 172.21.15.97:0/3106298499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f107415c540 0x7f1074160a80 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.487+0000 7f107a1ff700 1 --2- 172.21.15.97:0/3106298499 >> v2:172.21.15.146:3300/0 conn(0x7f10740b4870 0x7f10740b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.487+0000 7f107a1ff700 1 -- 172.21.15.97:0/3106298499 >> 172.21.15.97:0/3106298499 conn(0x7f107406b2f0 msgr2=0x7f10740cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:28.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.487+0000 7f107a1ff700 1 -- 172.21.15.97:0/3106298499 shutdown_connections 2023-07-06T03:48:28.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.487+0000 7f107a1ff700 1 -- 172.21.15.97:0/3106298499 wait complete. 2023-07-06T03:48:28.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.489+0000 7f107a1ff700 1 Processor -- start 2023-07-06T03:48:28.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.489+0000 7f107a1ff700 1 -- start start 2023-07-06T03:48:28.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.490+0000 7f107a1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f107406c090 0x7f10741f1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.490+0000 7f107a1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10740b4870 0x7f107406c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.490+0000 7f107a1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f107415c540 0x7f107406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.490+0000 7f107a1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107415d000 con 0x7f107415c540 2023-07-06T03:48:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.490+0000 7f107a1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10740c6dc0 con 0x7f107406c090 2023-07-06T03:48:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.490+0000 7f107a1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107415ca40 con 0x7f10740b4870 2023-07-06T03:48:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.490+0000 7f1073fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f107415c540 0x7f107406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.490+0000 7f1073fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f107415c540 0x7f107406ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57720/0 (socket says 172.21.15.97:57720) 2023-07-06T03:48:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.490+0000 7f1073fff700 1 -- 172.21.15.97:0/3515884217 learned_addr learned my addr 172.21.15.97:0/3515884217 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:28.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f10737fe700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f107406c090 0x7f10741f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f1072ffd700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10740b4870 0x7f107406c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f1073fff700 1 -- 172.21.15.97:0/3515884217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10740b4870 msgr2=0x7f107406c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f1073fff700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10740b4870 0x7f107406c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f1073fff700 1 -- 172.21.15.97:0/3515884217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f107406c090 msgr2=0x7f10741f1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f1073fff700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f107406c090 0x7f10741f1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f1073fff700 1 -- 172.21.15.97:0/3515884217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064009550 con 0x7f107415c540 2023-07-06T03:48:28.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f10737fe700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f107406c090 0x7f10741f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:28.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f1072ffd700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10740b4870 0x7f107406c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:28.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f1073fff700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f107415c540 0x7f107406ca10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f106c00cbd0 tx=0x7f106c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:28.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f105bfff700 1 -- 172.21.15.97:0/3515884217 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00ab70 con 0x7f107415c540 2023-07-06T03:48:28.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f105bfff700 1 -- 172.21.15.97:0/3515884217 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c00acf0 con 0x7f107415c540 2023-07-06T03:48:28.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f107a1ff700 1 -- 172.21.15.97:0/3515884217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107415cd20 con 0x7f107415c540 2023-07-06T03:48:28.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f105bfff700 1 -- 172.21.15.97:0/3515884217 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c007310 con 0x7f107415c540 2023-07-06T03:48:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.491+0000 7f107a1ff700 1 -- 172.21.15.97:0/3515884217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10740b2e70 con 0x7f107415c540 2023-07-06T03:48:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.492+0000 7f107a1ff700 1 -- 172.21.15.97:0/3515884217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10740726a0 con 0x7f107415c540 2023-07-06T03:48:28.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.493+0000 7f105bfff700 1 -- 172.21.15.97:0/3515884217 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f106c007470 con 0x7f107415c540 2023-07-06T03:48:28.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.494+0000 7f105bfff700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106006ace0 0x7f106006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.494+0000 7f105bfff700 1 -- 172.21.15.97:0/3515884217 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f106c0893c0 con 0x7f107415c540 2023-07-06T03:48:28.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.494+0000 7f10737fe700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106006ace0 0x7f106006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.495+0000 7f10737fe700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106006ace0 0x7f106006d190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1064000c00 tx=0x7f1064008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:28.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.498+0000 7f105bfff700 1 -- 172.21.15.97:0/3515884217 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f106c0543e0 con 0x7f107415c540 2023-07-06T03:48:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.694+0000 7f107a1ff700 1 -- 172.21.15.97:0/3515884217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1074074390 con 0x7f107415c540 2023-07-06T03:48:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.695+0000 7f105bfff700 1 -- 172.21.15.97:0/3515884217 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f106c00ae60 con 0x7f107415c540 2023-07-06T03:48:28.697 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:48:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 -- 172.21.15.97:0/3515884217 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106006ace0 msgr2=0x7f106006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106006ace0 0x7f106006d190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1064000c00 tx=0x7f1064008030 comp rx=0 tx=0).stop 2023-07-06T03:48:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 -- 172.21.15.97:0/3515884217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f107415c540 msgr2=0x7f107406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f107415c540 0x7f107406ca10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f106c00cbd0 tx=0x7f106c00cf90 comp rx=0 tx=0).stop 2023-07-06T03:48:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 -- 172.21.15.97:0/3515884217 shutdown_connections 2023-07-06T03:48:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106006ace0 0x7f106006d190 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f107415c540 0x7f107406ca10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f107406c090 0x7f10741f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 --2- 172.21.15.97:0/3515884217 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f10740b4870 0x7f107406c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 -- 172.21.15.97:0/3515884217 >> 172.21.15.97:0/3515884217 conn(0x7f107406b2f0 msgr2=0x7f10740d13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 -- 172.21.15.97:0/3515884217 shutdown_connections 2023-07-06T03:48:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.697+0000 7f107a1ff700 1 -- 172.21.15.97:0/3515884217 wait complete. 2023-07-06T03:48:28.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:48:28.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.913+0000 7fdfa7dff700 1 Processor -- start 2023-07-06T03:48:28.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.914+0000 7fdfa7dff700 1 -- start start 2023-07-06T03:48:28.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.915+0000 7fdfa7dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdfa00ce4c0 0x7fdfa00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.915+0000 7fdfa7dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdfa00c38a0 0x7fdfa00ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.915+0000 7fdfa7dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa00c9a60 con 0x7fdfa00cb350 2023-07-06T03:48:28.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.915+0000 7fdfa7dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa00cce70 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.915+0000 7fdfa7dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa00cc5d0 con 0x7fdfa00c38a0 2023-07-06T03:48:28.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.915+0000 7fdfa539a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdfa00ce4c0 0x7fdfa00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.915+0000 7fdfa539a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdfa00ce4c0 0x7fdfa00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57732/0 (socket says 172.21.15.97:57732) 2023-07-06T03:48:28.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.915+0000 7fdfa5b9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdfa00cb350 0x7fdfa00cb720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54992/0 (socket says 172.21.15.97:54992) 2023-07-06T03:48:28.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.915+0000 7fdfa539a700 1 -- 172.21.15.97:0/2317932293 learned_addr learned my addr 172.21.15.97:0/2317932293 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:28.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.915+0000 7fdfa639c700 1 --2- 172.21.15.97:0/2317932293 >> v2:172.21.15.146:3300/0 conn(0x7fdfa00c38a0 0x7fdfa00ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.916+0000 7fdfa539a700 1 -- 172.21.15.97:0/2317932293 >> v2:172.21.15.146:3300/0 conn(0x7fdfa00c38a0 msgr2=0x7fdfa00ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:28.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.916+0000 7fdfa539a700 1 --2- 172.21.15.97:0/2317932293 >> v2:172.21.15.146:3300/0 conn(0x7fdfa00c38a0 0x7fdfa00ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.916+0000 7fdfa539a700 1 -- 172.21.15.97:0/2317932293 >> v1:172.21.15.97:6789/0 conn(0x7fdfa00cb350 legacy=0x7fdfa00cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.916+0000 7fdfa539a700 1 -- 172.21.15.97:0/2317932293 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa00cc8b0 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.916+0000 7fdfa4b99700 1 -- 172.21.15.97:0/2317932293 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541495801 0 0) 0x7fdfa00c9a60 con 0x7fdfa00cb350 2023-07-06T03:48:28.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.916+0000 7fdfa539a700 1 --2- 172.21.15.97:0/2317932293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa00ce4c0 0x7fdfa00cf290 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000bb90 tx=0x7fdf9000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725496d7d5a5dff5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:28.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.916+0000 7fdfa4b99700 1 -- 172.21.15.97:0/2317932293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000dd10 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.916+0000 7fdfa4b99700 1 -- 172.21.15.97:0/2317932293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9000de70 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.916+0000 7fdfa4b99700 1 -- 172.21.15.97:0/2317932293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90010670 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.916+0000 7fdfa7dff700 1 -- 172.21.15.97:0/2317932293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa00ce4c0 msgr2=0x7fdfa00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.916+0000 7fdfa7dff700 1 --2- 172.21.15.97:0/2317932293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa00ce4c0 0x7fdfa00cf290 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000bb90 tx=0x7fdf9000bea0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.917+0000 7fdfa7dff700 1 -- 172.21.15.97:0/2317932293 shutdown_connections 2023-07-06T03:48:28.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.917+0000 7fdfa7dff700 1 --2- 172.21.15.97:0/2317932293 >> v2:172.21.15.146:3300/0 conn(0x7fdfa00c38a0 0x7fdfa00ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.917+0000 7fdfa7dff700 1 --2- 172.21.15.97:0/2317932293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa00ce4c0 0x7fdfa00cf290 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.917+0000 7fdfa7dff700 1 -- 172.21.15.97:0/2317932293 >> 172.21.15.97:0/2317932293 conn(0x7fdfa0069fe0 msgr2=0x7fdfa006a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.917+0000 7fdfa7dff700 1 -- 172.21.15.97:0/2317932293 shutdown_connections 2023-07-06T03:48:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.917+0000 7fdfa7dff700 1 -- 172.21.15.97:0/2317932293 wait complete. 2023-07-06T03:48:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.918+0000 7fdfa7dff700 1 Processor -- start 2023-07-06T03:48:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.918+0000 7fdfa7dff700 1 -- start start 2023-07-06T03:48:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.919+0000 7fdfa7dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfa00c38a0 0x7fdfa01f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.919+0000 7fdfa7dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfa00cb350 0x7fdfa006fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.919+0000 7fdfa7dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa00ce4c0 0x7fdfa0070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.919+0000 7fdfa7dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00c3ce0 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.919+0000 7fdfa7dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00c4580 con 0x7fdfa00cb350 2023-07-06T03:48:28.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.919+0000 7fdfa7dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0159f70 con 0x7fdfa00c38a0 2023-07-06T03:48:28.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa639c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa00ce4c0 0x7fdfa0070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa639c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa00ce4c0 0x7fdfa0070380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57736/0 (socket says 172.21.15.97:57736) 2023-07-06T03:48:28.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa639c700 1 -- 172.21.15.97:0/1598709953 learned_addr learned my addr 172.21.15.97:0/1598709953 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:28.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa539a700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfa00cb350 0x7fdfa006fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa5b9b700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfa00c38a0 0x7fdfa01f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa639c700 1 -- 172.21.15.97:0/1598709953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfa00c38a0 msgr2=0x7fdfa01f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa639c700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfa00c38a0 0x7fdfa01f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa639c700 1 -- 172.21.15.97:0/1598709953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfa00cb350 msgr2=0x7fdfa006fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:28.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa639c700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfa00cb350 0x7fdfa006fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:28.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa639c700 1 -- 172.21.15.97:0/1598709953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9000b7f0 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa5b9b700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfa00c38a0 0x7fdfa01f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:28.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.920+0000 7fdfa639c700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa00ce4c0 0x7fdfa0070380 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdf98007e70 tx=0x7fdf9800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:28.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.921+0000 7fdf967fc700 1 -- 172.21.15.97:0/1598709953 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800ede0 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.921+0000 7fdf967fc700 1 -- 172.21.15.97:0/1598709953 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf98014420 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.921+0000 7fdfa7dff700 1 -- 172.21.15.97:0/1598709953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa00b41a0 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.921+0000 7fdf967fc700 1 -- 172.21.15.97:0/1598709953 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf980147d0 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.921+0000 7fdfa7dff700 1 -- 172.21.15.97:0/1598709953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa00c4210 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.922+0000 7fdfa7dff700 1 -- 172.21.15.97:0/1598709953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa00c4730 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.923+0000 7fdf967fc700 1 -- 172.21.15.97:0/1598709953 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf98014930 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.924+0000 7fdf967fc700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf8406ace0 0x7fdf8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:28.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.924+0000 7fdf967fc700 1 -- 172.21.15.97:0/1598709953 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7fdf98089c50 con 0x7fdfa00ce4c0 2023-07-06T03:48:28.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.924+0000 7fdfa5b9b700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf8406ace0 0x7fdf8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:28.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.925+0000 7fdfa5b9b700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf8406ace0 0x7fdf8406d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c000f60 tx=0x7fdf9c007cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:28.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:28.928+0000 7fdf967fc700 1 -- 172.21.15.97:0/1598709953 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf9808f020 con 0x7fdfa00ce4c0 2023-07-06T03:48:29.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.123+0000 7fdfa7dff700 1 -- 172.21.15.97:0/1598709953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfa0074390 con 0x7fdfa00ce4c0 2023-07-06T03:48:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.123+0000 7fdf967fc700 1 -- 172.21.15.97:0/1598709953 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf98054c80 con 0x7fdfa00ce4c0 2023-07-06T03:48:29.125 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:48:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 -- 172.21.15.97:0/1598709953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf8406ace0 msgr2=0x7fdf8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf8406ace0 0x7fdf8406d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c000f60 tx=0x7fdf9c007cc0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 -- 172.21.15.97:0/1598709953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa00ce4c0 msgr2=0x7fdfa0070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa00ce4c0 0x7fdfa0070380 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdf98007e70 tx=0x7fdf9800e3c0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 -- 172.21.15.97:0/1598709953 shutdown_connections 2023-07-06T03:48:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf8406ace0 0x7fdf8406d190 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa00ce4c0 0x7fdfa0070380 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdfa00c38a0 0x7fdfa01f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 --2- 172.21.15.97:0/1598709953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdfa00cb350 0x7fdfa006fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 -- 172.21.15.97:0/1598709953 >> 172.21.15.97:0/1598709953 conn(0x7fdfa0069fe0 msgr2=0x7fdfa00710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 -- 172.21.15.97:0/1598709953 shutdown_connections 2023-07-06T03:48:29.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.126+0000 7fdfa7dff700 1 -- 172.21.15.97:0/1598709953 wait complete. 2023-07-06T03:48:29.146 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:48:29.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.341+0000 7f4e281ff700 1 Processor -- start 2023-07-06T03:48:29.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.342+0000 7f4e281ff700 1 -- start start 2023-07-06T03:48:29.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.343+0000 7f4e281ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e2015c540 0x7f4e20160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:29.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.343+0000 7f4e281ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2015c910 con 0x7f4e200d24b0 2023-07-06T03:48:29.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.343+0000 7f4e281ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e20160fc0 con 0x7f4e200b4870 2023-07-06T03:48:29.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.343+0000 7f4e281ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2006ceb0 con 0x7f4e2015c540 2023-07-06T03:48:29.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.343+0000 7f4e2679c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4e200d24b0 0x7f4e2006c7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55004/0 (socket says 172.21.15.97:55004) 2023-07-06T03:48:29.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.343+0000 7f4e2679c700 1 -- 172.21.15.97:0/3843876759 learned_addr learned my addr 172.21.15.97:0/3843876759 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.343+0000 7f4e25f9b700 1 --2- 172.21.15.97:0/3843876759 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e2015c540 0x7f4e20160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.343+0000 7f4e24f99700 1 -- 172.21.15.97:0/3843876759 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350853427 0 0) 0x7f4e2015c910 con 0x7f4e200d24b0 2023-07-06T03:48:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.343+0000 7f4e24f99700 1 -- 172.21.15.97:0/3843876759 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e0c0035b0 con 0x7f4e200d24b0 2023-07-06T03:48:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.344+0000 7f4e24f99700 1 -- 172.21.15.97:0/3843876759 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193126682 0 0) 0x7f4e20160fc0 con 0x7f4e200b4870 2023-07-06T03:48:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.344+0000 7f4e24f99700 1 -- 172.21.15.97:0/3843876759 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e2015c910 con 0x7f4e200b4870 2023-07-06T03:48:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.344+0000 7f4e25f9b700 1 -- 172.21.15.97:0/3843876759 >> v1:172.21.15.146:6789/0 conn(0x7f4e200b4870 legacy=0x7f4e2006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.344+0000 7f4e25f9b700 1 -- 172.21.15.97:0/3843876759 >> v1:172.21.15.97:6789/0 conn(0x7f4e200d24b0 legacy=0x7f4e2006c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.344+0000 7f4e25f9b700 1 -- 172.21.15.97:0/3843876759 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e200cd770 con 0x7f4e2015c540 2023-07-06T03:48:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.344+0000 7f4e25f9b700 1 --2- 172.21.15.97:0/3843876759 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e2015c540 0x7f4e20160a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009870 tx=0x7f4e1c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c801d264c20600 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.344+0000 7f4e24f99700 1 -- 172.21.15.97:0/3843876759 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c011830 con 0x7f4e2015c540 2023-07-06T03:48:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.344+0000 7f4e24f99700 1 -- 172.21.15.97:0/3843876759 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c011990 con 0x7f4e2015c540 2023-07-06T03:48:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.344+0000 7f4e24f99700 1 -- 172.21.15.97:0/3843876759 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c011d00 con 0x7f4e2015c540 2023-07-06T03:48:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.345+0000 7f4e281ff700 1 -- 172.21.15.97:0/3843876759 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e2015c540 msgr2=0x7f4e20160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.345+0000 7f4e281ff700 1 --2- 172.21.15.97:0/3843876759 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e2015c540 0x7f4e20160a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009870 tx=0x7f4e1c009b80 comp rx=0 tx=0).stop 2023-07-06T03:48:29.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.345+0000 7f4e281ff700 1 -- 172.21.15.97:0/3843876759 shutdown_connections 2023-07-06T03:48:29.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.345+0000 7f4e281ff700 1 --2- 172.21.15.97:0/3843876759 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e2015c540 0x7f4e20160a80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.345+0000 7f4e281ff700 1 -- 172.21.15.97:0/3843876759 >> 172.21.15.97:0/3843876759 conn(0x7f4e2006b2f0 msgr2=0x7f4e200cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:29.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.345+0000 7f4e281ff700 1 -- 172.21.15.97:0/3843876759 shutdown_connections 2023-07-06T03:48:29.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.345+0000 7f4e281ff700 1 -- 172.21.15.97:0/3843876759 wait complete. 2023-07-06T03:48:29.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.347+0000 7f4e281ff700 1 Processor -- start 2023-07-06T03:48:29.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.347+0000 7f4e281ff700 1 -- start start 2023-07-06T03:48:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e281ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e200b4870 0x7f4e20156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e281ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e200d24b0 0x7f4e200cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e281ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2015c540 0x7f4e200cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e281ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200ccd90 con 0x7f4e200b4870 2023-07-06T03:48:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e281ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200cc180 con 0x7f4e2015c540 2023-07-06T03:48:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e281ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200cc460 con 0x7f4e200d24b0 2023-07-06T03:48:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e25f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e200b4870 0x7f4e20156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e25f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e200b4870 0x7f4e20156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57744/0 (socket says 172.21.15.97:57744) 2023-07-06T03:48:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e25f9b700 1 -- 172.21.15.97:0/322603420 learned_addr learned my addr 172.21.15.97:0/322603420 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e2679c700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2015c540 0x7f4e200cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e2579a700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e200d24b0 0x7f4e200cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e25f9b700 1 -- 172.21.15.97:0/322603420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e200d24b0 msgr2=0x7f4e200cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e25f9b700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e200d24b0 0x7f4e200cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e25f9b700 1 -- 172.21.15.97:0/322603420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2015c540 msgr2=0x7f4e200cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e25f9b700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2015c540 0x7f4e200cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.349+0000 7f4e25f9b700 1 -- 172.21.15.97:0/322603420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c0094a0 con 0x7f4e200b4870 2023-07-06T03:48:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.350+0000 7f4e25f9b700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e200b4870 0x7f4e20156ae0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0090b0 tx=0x7f4e1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:29.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.350+0000 7f4e167fc700 1 -- 172.21.15.97:0/322603420 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0036a0 con 0x7f4e200b4870 2023-07-06T03:48:29.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.350+0000 7f4e167fc700 1 -- 172.21.15.97:0/322603420 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c003820 con 0x7f4e200b4870 2023-07-06T03:48:29.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.350+0000 7f4e281ff700 1 -- 172.21.15.97:0/322603420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e200cc740 con 0x7f4e200b4870 2023-07-06T03:48:29.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.350+0000 7f4e167fc700 1 -- 172.21.15.97:0/322603420 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c003bd0 con 0x7f4e200b4870 2023-07-06T03:48:29.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.350+0000 7f4e281ff700 1 -- 172.21.15.97:0/322603420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e200cbe90 con 0x7f4e200b4870 2023-07-06T03:48:29.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.351+0000 7f4e281ff700 1 -- 172.21.15.97:0/322603420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e200726a0 con 0x7f4e200b4870 2023-07-06T03:48:29.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.352+0000 7f4e167fc700 1 -- 172.21.15.97:0/322603420 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e1c003d30 con 0x7f4e200b4870 2023-07-06T03:48:29.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.353+0000 7f4e167fc700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e0406ace0 0x7f4e0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:29.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.353+0000 7f4e167fc700 1 -- 172.21.15.97:0/322603420 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f4e1c08a8c0 con 0x7f4e200b4870 2023-07-06T03:48:29.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.353+0000 7f4e2579a700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e0406ace0 0x7f4e0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:29.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.354+0000 7f4e2579a700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e0406ace0 0x7f4e0406d190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4e20157c10 tx=0x7f4e10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:29.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.357+0000 7f4e167fc700 1 -- 172.21.15.97:0/322603420 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4e1c0558e0 con 0x7f4e200b4870 2023-07-06T03:48:29.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.551+0000 7f4e281ff700 1 -- 172.21.15.97:0/322603420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e200cda30 con 0x7f4e200b4870 2023-07-06T03:48:29.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.552+0000 7f4e167fc700 1 -- 172.21.15.97:0/322603420 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e200cda30 con 0x7f4e200b4870 2023-07-06T03:48:29.554 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:48:29.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 -- 172.21.15.97:0/322603420 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e0406ace0 msgr2=0x7f4e0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e0406ace0 0x7f4e0406d190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4e20157c10 tx=0x7f4e10009040 comp rx=0 tx=0).stop 2023-07-06T03:48:29.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 -- 172.21.15.97:0/322603420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e200b4870 msgr2=0x7f4e20156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e200b4870 0x7f4e20156ae0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0090b0 tx=0x7f4e1c004750 comp rx=0 tx=0).stop 2023-07-06T03:48:29.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 -- 172.21.15.97:0/322603420 shutdown_connections 2023-07-06T03:48:29.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4e0406ace0 0x7f4e0406d190 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2015c540 0x7f4e200cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e200b4870 0x7f4e20156ae0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 --2- 172.21.15.97:0/322603420 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4e200d24b0 0x7f4e200cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 -- 172.21.15.97:0/322603420 >> 172.21.15.97:0/322603420 conn(0x7f4e2006b2f0 msgr2=0x7f4e2015a200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:29.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 -- 172.21.15.97:0/322603420 shutdown_connections 2023-07-06T03:48:29.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.554+0000 7f4e281ff700 1 -- 172.21.15.97:0/322603420 wait complete. 2023-07-06T03:48:29.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:48:29.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.766+0000 7f202b9ff700 1 Processor -- start 2023-07-06T03:48:29.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.767+0000 7f202b9ff700 1 -- start start 2023-07-06T03:48:29.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.768+0000 7f202b9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f202415c850 0x7f20240b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:29.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.768+0000 7f202b9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202415cc20 con 0x7f20240b4db0 2023-07-06T03:48:29.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.768+0000 7f202b9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20240d1990 con 0x7f20240d1330 2023-07-06T03:48:29.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.768+0000 7f202b9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20240d17b0 con 0x7f202415c850 2023-07-06T03:48:29.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.768+0000 7f202979b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f202415c850 0x7f20240b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:29.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.768+0000 7f202979b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f202415c850 0x7f20240b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57746/0 (socket says 172.21.15.97:57746) 2023-07-06T03:48:29.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.768+0000 7f202979b700 1 -- 172.21.15.97:0/3715265272 learned_addr learned my addr 172.21.15.97:0/3715265272 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:29.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.768+0000 7f202979b700 1 -- 172.21.15.97:0/3715265272 >> v1:172.21.15.146:6789/0 conn(0x7f20240d1330 legacy=0x7f20240b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.768+0000 7f202979b700 1 -- 172.21.15.97:0/3715265272 >> v1:172.21.15.97:6789/0 conn(0x7f20240b4db0 legacy=0x7f20240d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.769+0000 7f202979b700 1 -- 172.21.15.97:0/3715265272 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20240d1c70 con 0x7f202415c850 2023-07-06T03:48:29.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.769+0000 7f201bfff700 1 -- 172.21.15.97:0/3715265272 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096249640 0 0) 0x7f202415cc20 con 0x7f20240b4db0 2023-07-06T03:48:29.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.769+0000 7f202979b700 1 --2- 172.21.15.97:0/3715265272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f202415c850 0x7f20240b4870 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f20200098f0 tx=0x7f2020009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7accd38c4f77d252 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:29.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.769+0000 7f201bfff700 1 -- 172.21.15.97:0/3715265272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000b8a0 con 0x7f202415c850 2023-07-06T03:48:29.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.769+0000 7f201bfff700 1 -- 172.21.15.97:0/3715265272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202000ba00 con 0x7f202415c850 2023-07-06T03:48:29.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.769+0000 7f201bfff700 1 -- 172.21.15.97:0/3715265272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000bd90 con 0x7f202415c850 2023-07-06T03:48:29.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.769+0000 7f202b9ff700 1 -- 172.21.15.97:0/3715265272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f202415c850 msgr2=0x7f20240b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.769+0000 7f202b9ff700 1 --2- 172.21.15.97:0/3715265272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f202415c850 0x7f20240b4870 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f20200098f0 tx=0x7f2020009c00 comp rx=0 tx=0).stop 2023-07-06T03:48:29.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.770+0000 7f202b9ff700 1 -- 172.21.15.97:0/3715265272 shutdown_connections 2023-07-06T03:48:29.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.770+0000 7f202b9ff700 1 --2- 172.21.15.97:0/3715265272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f202415c850 0x7f20240b4870 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.770+0000 7f202b9ff700 1 -- 172.21.15.97:0/3715265272 >> 172.21.15.97:0/3715265272 conn(0x7f20240d2df0 msgr2=0x7f20240d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:29.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.770+0000 7f202b9ff700 1 -- 172.21.15.97:0/3715265272 shutdown_connections 2023-07-06T03:48:29.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.770+0000 7f202b9ff700 1 -- 172.21.15.97:0/3715265272 wait complete. 2023-07-06T03:48:29.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.771+0000 7f202b9ff700 1 Processor -- start 2023-07-06T03:48:29.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.771+0000 7f202b9ff700 1 -- start start 2023-07-06T03:48:29.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.772+0000 7f202b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20240b4db0 0x7f20241f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:29.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.772+0000 7f202b9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20240d1330 0x7f2024146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:29.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.772+0000 7f202b9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f202415c850 0x7f20240cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:29.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.772+0000 7f202b9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202415e7b0 con 0x7f202415c850 2023-07-06T03:48:29.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.772+0000 7f202b9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202415dc70 con 0x7f20240d1330 2023-07-06T03:48:29.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.772+0000 7f202b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202415df10 con 0x7f20240b4db0 2023-07-06T03:48:29.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2029f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f202415c850 0x7f20240cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:29.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2029f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f202415c850 0x7f20240cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57750/0 (socket says 172.21.15.97:57750) 2023-07-06T03:48:29.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2029f9c700 1 -- 172.21.15.97:0/1717756290 learned_addr learned my addr 172.21.15.97:0/1717756290 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:29.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2028f9a700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20240d1330 0x7f2024146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:29.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f202979b700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20240b4db0 0x7f20241f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:29.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2029f9c700 1 -- 172.21.15.97:0/1717756290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20240b4db0 msgr2=0x7f20241f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2029f9c700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20240b4db0 0x7f20241f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2029f9c700 1 -- 172.21.15.97:0/1717756290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20240d1330 msgr2=0x7f2024146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2029f9c700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20240d1330 0x7f2024146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2029f9c700 1 -- 172.21.15.97:0/1717756290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2020009550 con 0x7f202415c850 2023-07-06T03:48:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f202979b700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20240b4db0 0x7f20241f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2029f9c700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f202415c850 0x7f20240cb350 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f201c00cc50 tx=0x7f201c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2019ffb700 1 -- 172.21.15.97:0/1717756290 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c003c00 con 0x7f202415c850 2023-07-06T03:48:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2019ffb700 1 -- 172.21.15.97:0/1717756290 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c003d60 con 0x7f202415c850 2023-07-06T03:48:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f202b9ff700 1 -- 172.21.15.97:0/1717756290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202415e1f0 con 0x7f202415c850 2023-07-06T03:48:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.773+0000 7f2019ffb700 1 -- 172.21.15.97:0/1717756290 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c00ab60 con 0x7f202415c850 2023-07-06T03:48:29.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.774+0000 7f202b9ff700 1 -- 172.21.15.97:0/1717756290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20241f4fd0 con 0x7f202415c850 2023-07-06T03:48:29.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.774+0000 7f202b9ff700 1 -- 172.21.15.97:0/1717756290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20240726a0 con 0x7f202415c850 2023-07-06T03:48:29.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.775+0000 7f2019ffb700 1 -- 172.21.15.97:0/1717756290 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f201c00acc0 con 0x7f202415c850 2023-07-06T03:48:29.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.776+0000 7f2019ffb700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f200806ace0 0x7f200806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:29.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.776+0000 7f2019ffb700 1 -- 172.21.15.97:0/1717756290 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f201c0895d0 con 0x7f202415c850 2023-07-06T03:48:29.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.777+0000 7f202979b700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f200806ace0 0x7f200806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:29.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.777+0000 7f202979b700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f200806ace0 0x7f200806d190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2020009130 tx=0x7f202001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:29.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:29.781+0000 7f2019ffb700 1 -- 172.21.15.97:0/1717756290 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f201c0545f0 con 0x7f202415c850 2023-07-06T03:48:30.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.020+0000 7f202b9ff700 1 -- 172.21.15.97:0/1717756290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20241e6f90 con 0x7f202415c850 2023-07-06T03:48:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.021+0000 7f2019ffb700 1 -- 172.21.15.97:0/1717756290 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f201c057c10 con 0x7f202415c850 2023-07-06T03:48:30.023 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:30.023 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1299982704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1299982704},{"type":"v1","addr":"172.21.15.146:6833","nonce":1299982704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5602,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/640394657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":640394657},{"type":"v1","addr":"172.21.15.97:6837","nonce":640394657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2363940464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2363940464},{"type":"v1","addr":"172.21.15.97:6836","nonce":2363940464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:47:36.149445+0000","modified":"2023-07-06T03:47:39.683749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5838},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5838":{"gid":5838,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/3623506860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3623506860},{"type":"v1","addr":"172.21.15.146:6835","nonce":3623506860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T03:48:30.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.023+0000 7f202b9ff700 1 -- 172.21.15.97:0/1717756290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f200806ace0 msgr2=0x7f200806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.023+0000 7f202b9ff700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f200806ace0 0x7f200806d190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2020009130 tx=0x7f202001c040 comp rx=0 tx=0).stop 2023-07-06T03:48:30.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.023+0000 7f202b9ff700 1 -- 172.21.15.97:0/1717756290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f202415c850 msgr2=0x7f20240cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.023+0000 7f202b9ff700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f202415c850 0x7f20240cb350 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f201c00cc50 tx=0x7f201c00cf60 comp rx=0 tx=0).stop 2023-07-06T03:48:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.023+0000 7f202b9ff700 1 -- 172.21.15.97:0/1717756290 shutdown_connections 2023-07-06T03:48:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.023+0000 7f202b9ff700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f200806ace0 0x7f200806d190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.023+0000 7f202b9ff700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f202415c850 0x7f20240cb350 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.023+0000 7f202b9ff700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20240b4db0 0x7f20241f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.023+0000 7f202b9ff700 1 --2- 172.21.15.97:0/1717756290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20240d1330 0x7f2024146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.023+0000 7f202b9ff700 1 -- 172.21.15.97:0/1717756290 >> 172.21.15.97:0/1717756290 conn(0x7f20240d2df0 msgr2=0x7f20240cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.024+0000 7f202b9ff700 1 -- 172.21.15.97:0/1717756290 shutdown_connections 2023-07-06T03:48:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.024+0000 7f202b9ff700 1 -- 172.21.15.97:0/1717756290 wait complete. 2023-07-06T03:48:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2023-07-06T03:48:30.043 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:48:30.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.234+0000 7f383e1ff700 1 Processor -- start 2023-07-06T03:48:30.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.234+0000 7f383e1ff700 1 -- start start 2023-07-06T03:48:30.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.235+0000 7f383e1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38380cd440 0x7f3838160b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:30.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.235+0000 7f383e1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f383806a680 0x7f383806aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:30.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f383e1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3838161060 con 0x7f383815c740 2023-07-06T03:48:30.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f383e1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f383806b030 con 0x7f383806a680 2023-07-06T03:48:30.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f383e1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38380cdcc0 con 0x7f38380cd440 2023-07-06T03:48:30.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f3837fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f383806a680 0x7f383806aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:30.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f3837fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f383806a680 0x7f383806aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39924/0 (socket says 172.21.15.97:39924) 2023-07-06T03:48:30.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f3837fff700 1 -- 172.21.15.97:0/670294284 learned_addr learned my addr 172.21.15.97:0/670294284 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:30.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f3836ffd700 1 --2- 172.21.15.97:0/670294284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38380cd440 0x7f3838160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:30.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f3837fff700 1 -- 172.21.15.97:0/670294284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38380cd440 msgr2=0x7f3838160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:30.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f3837fff700 1 --2- 172.21.15.97:0/670294284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38380cd440 0x7f3838160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f3837fff700 1 -- 172.21.15.97:0/670294284 >> v1:172.21.15.146:6789/0 conn(0x7f383815c740 legacy=0x7f38380b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:30.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f3837fff700 1 -- 172.21.15.97:0/670294284 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380cd9e0 con 0x7f383806a680 2023-07-06T03:48:30.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f3836ffd700 1 --2- 172.21.15.97:0/670294284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38380cd440 0x7f3838160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:30.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.236+0000 7f3837fff700 1 --2- 172.21.15.97:0/670294284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f383806a680 0x7f383806aaf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3830009e50 tx=0x7f383000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188db9849ee70a49 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:30.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.237+0000 7f38367fc700 1 -- 172.21.15.97:0/670294284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383000e070 con 0x7f383806a680 2023-07-06T03:48:30.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.237+0000 7f38367fc700 1 -- 172.21.15.97:0/670294284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3830003a20 con 0x7f383806a680 2023-07-06T03:48:30.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.237+0000 7f38367fc700 1 -- 172.21.15.97:0/670294284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830003dd0 con 0x7f383806a680 2023-07-06T03:48:30.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.237+0000 7f383e1ff700 1 -- 172.21.15.97:0/670294284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f383806a680 msgr2=0x7f383806aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.237+0000 7f383e1ff700 1 --2- 172.21.15.97:0/670294284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f383806a680 0x7f383806aaf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3830009e50 tx=0x7f383000b330 comp rx=0 tx=0).stop 2023-07-06T03:48:30.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.237+0000 7f383e1ff700 1 -- 172.21.15.97:0/670294284 shutdown_connections 2023-07-06T03:48:30.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.237+0000 7f383e1ff700 1 --2- 172.21.15.97:0/670294284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f383806a680 0x7f383806aaf0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.237+0000 7f383e1ff700 1 --2- 172.21.15.97:0/670294284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38380cd440 0x7f3838160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.237+0000 7f383e1ff700 1 -- 172.21.15.97:0/670294284 >> 172.21.15.97:0/670294284 conn(0x7f38380d2e80 msgr2=0x7f38380d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:30.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.237+0000 7f383e1ff700 1 -- 172.21.15.97:0/670294284 shutdown_connections 2023-07-06T03:48:30.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.237+0000 7f383e1ff700 1 -- 172.21.15.97:0/670294284 wait complete. 2023-07-06T03:48:30.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.239+0000 7f383e1ff700 1 Processor -- start 2023-07-06T03:48:30.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.239+0000 7f383e1ff700 1 -- start start 2023-07-06T03:48:30.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.240+0000 7f383e1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f383806a680 0x7f3838154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:30.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.240+0000 7f383e1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38380cd440 0x7f38381f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:30.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.240+0000 7f383e1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f383815c740 0x7f38381500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:30.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.240+0000 7f383e1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38380ca620 con 0x7f38380cd440 2023-07-06T03:48:30.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.240+0000 7f383e1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38380ca870 con 0x7f383815c740 2023-07-06T03:48:30.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.240+0000 7f383e1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38380cab50 con 0x7f383806a680 2023-07-06T03:48:30.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.240+0000 7f3836ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38380cd440 0x7f38381f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:30.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.240+0000 7f3836ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38380cd440 0x7f38381f5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39926/0 (socket says 172.21.15.97:39926) 2023-07-06T03:48:30.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.240+0000 7f3836ffd700 1 -- 172.21.15.97:0/1877573970 learned_addr learned my addr 172.21.15.97:0/1877573970 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:30.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f38377fe700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f383806a680 0x7f3838154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:30.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f3837fff700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f383815c740 0x7f38381500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:30.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f3836ffd700 1 -- 172.21.15.97:0/1877573970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f383806a680 msgr2=0x7f3838154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f3836ffd700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f383806a680 0x7f3838154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f3836ffd700 1 -- 172.21.15.97:0/1877573970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f383815c740 msgr2=0x7f38381500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f3836ffd700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f383815c740 0x7f38381500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f3836ffd700 1 -- 172.21.15.97:0/1877573970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3830009b30 con 0x7f38380cd440 2023-07-06T03:48:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f38377fe700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f383806a680 0x7f3838154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f3836ffd700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38380cd440 0x7f38381f5720 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c740 tx=0x7f382c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f381ffff700 1 -- 172.21.15.97:0/1877573970 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c004220 con 0x7f38380cd440 2023-07-06T03:48:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f381ffff700 1 -- 172.21.15.97:0/1877573970 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c004380 con 0x7f38380cd440 2023-07-06T03:48:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f381ffff700 1 -- 172.21.15.97:0/1877573970 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c0036a0 con 0x7f38380cd440 2023-07-06T03:48:30.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f383e1ff700 1 -- 172.21.15.97:0/1877573970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38380cae30 con 0x7f38380cd440 2023-07-06T03:48:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.241+0000 7f383e1ff700 1 -- 172.21.15.97:0/1877573970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3838154880 con 0x7f38380cd440 2023-07-06T03:48:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.242+0000 7f383e1ff700 1 -- 172.21.15.97:0/1877573970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3838074390 con 0x7f38380cd440 2023-07-06T03:48:30.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.244+0000 7f381ffff700 1 -- 172.21.15.97:0/1877573970 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f382c003800 con 0x7f38380cd440 2023-07-06T03:48:30.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.244+0000 7f381ffff700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382406ace0 0x7f382406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:30.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.244+0000 7f381ffff700 1 -- 172.21.15.97:0/1877573970 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f382c089890 con 0x7f38380cd440 2023-07-06T03:48:30.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.245+0000 7f38377fe700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382406ace0 0x7f382406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:30.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.245+0000 7f38377fe700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382406ace0 0x7f382406d190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3828003fc0 tx=0x7f3828009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:30.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.249+0000 7f381ffff700 1 -- 172.21.15.97:0/1877573970 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f382c0548d0 con 0x7f38380cd440 2023-07-06T03:48:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.444+0000 7f383e1ff700 1 -- 172.21.15.97:0/1877573970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38380d2760 con 0x7f38380cd440 2023-07-06T03:48:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.445+0000 7f381ffff700 1 -- 172.21.15.97:0/1877573970 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18524 (secure 0 0 0) 0x7f382c057ef0 con 0x7f38380cd440 2023-07-06T03:48:30.447 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:30.447 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":49,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:47:52.725520+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:47:33.695348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-06T03:47:35.132699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":2,"to":5}]},{"pgid":"8.1a","mappings":[{"from":2,"to":1}]},{"pgid":"8.38","mappings":[{"from":6,"to":5}]},{"pgid":"8.3d","mappings":[{"from":6,"to":0},{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:48:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.447+0000 7f383e1ff700 1 -- 172.21.15.97:0/1877573970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382406ace0 msgr2=0x7f382406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.447+0000 7f383e1ff700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382406ace0 0x7f382406d190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3828003fc0 tx=0x7f3828009040 comp rx=0 tx=0).stop 2023-07-06T03:48:30.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.447+0000 7f383e1ff700 1 -- 172.21.15.97:0/1877573970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38380cd440 msgr2=0x7f38381f5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.447+0000 7f383e1ff700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38380cd440 0x7f38381f5720 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c740 tx=0x7f382c00cb00 comp rx=0 tx=0).stop 2023-07-06T03:48:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.447+0000 7f383e1ff700 1 -- 172.21.15.97:0/1877573970 shutdown_connections 2023-07-06T03:48:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.447+0000 7f383e1ff700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382406ace0 0x7f382406d190 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.447+0000 7f383e1ff700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f383815c740 0x7f38381500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.447+0000 7f383e1ff700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f383806a680 0x7f3838154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.447+0000 7f383e1ff700 1 --2- 172.21.15.97:0/1877573970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38380cd440 0x7f38381f5720 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.447+0000 7f383e1ff700 1 -- 172.21.15.97:0/1877573970 >> 172.21.15.97:0/1877573970 conn(0x7f38380d2e80 msgr2=0x7f38380d1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.448+0000 7f383e1ff700 1 -- 172.21.15.97:0/1877573970 shutdown_connections 2023-07-06T03:48:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.448+0000 7f383e1ff700 1 -- 172.21.15.97:0/1877573970 wait complete. 2023-07-06T03:48:30.469 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T03:48:30.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:48:30.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.662+0000 7f832c7ff700 1 Processor -- start 2023-07-06T03:48:30.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.662+0000 7f832c7ff700 1 -- start start 2023-07-06T03:48:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.664+0000 7f832c7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83240b4f80 0x7f83240d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.664+0000 7f832c7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832415cc20 con 0x7f832415c850 2023-07-06T03:48:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.664+0000 7f832c7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83240d1990 con 0x7f83240d1160 2023-07-06T03:48:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.664+0000 7f832c7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83240d1c70 con 0x7f83240b4f80 2023-07-06T03:48:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.664+0000 7f832a59b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f832415c850 0x7f83240b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45186/0 (socket says 172.21.15.97:45186) 2023-07-06T03:48:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.664+0000 7f832a59b700 1 -- 172.21.15.97:0/3544244864 learned_addr learned my addr 172.21.15.97:0/3544244864 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.664+0000 7f8329d9a700 1 --2- 172.21.15.97:0/3544244864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83240b4f80 0x7f83240d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.664+0000 7f8329599700 1 -- 172.21.15.97:0/3544244864 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979910440 0 0) 0x7f832415cc20 con 0x7f832415c850 2023-07-06T03:48:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.664+0000 7f8329599700 1 -- 172.21.15.97:0/3544244864 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83100035b0 con 0x7f832415c850 2023-07-06T03:48:30.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.664+0000 7f8329599700 1 -- 172.21.15.97:0/3544244864 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685508035 0 0) 0x7f83240d1990 con 0x7f83240d1160 2023-07-06T03:48:30.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.664+0000 7f8329599700 1 -- 172.21.15.97:0/3544244864 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f832415cc20 con 0x7f83240d1160 2023-07-06T03:48:30.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.665+0000 7f8329d9a700 1 -- 172.21.15.97:0/3544244864 >> v1:172.21.15.146:6789/0 conn(0x7f83240d1160 legacy=0x7f83240b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.665+0000 7f8329d9a700 1 -- 172.21.15.97:0/3544244864 >> v1:172.21.15.97:6789/0 conn(0x7f832415c850 legacy=0x7f83240b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.665+0000 7f8329d9a700 1 -- 172.21.15.97:0/3544244864 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240d1f50 con 0x7f83240b4f80 2023-07-06T03:48:30.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.665+0000 7f8329d9a700 1 --2- 172.21.15.97:0/3544244864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83240b4f80 0x7f83240d0c20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8314009870 tx=0x7f8314009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca40518a6054d17c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:30.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.665+0000 7f8329599700 1 -- 172.21.15.97:0/3544244864 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314011830 con 0x7f83240b4f80 2023-07-06T03:48:30.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.665+0000 7f8329599700 1 -- 172.21.15.97:0/3544244864 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8314011990 con 0x7f83240b4f80 2023-07-06T03:48:30.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.665+0000 7f8329599700 1 -- 172.21.15.97:0/3544244864 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314011d00 con 0x7f83240b4f80 2023-07-06T03:48:30.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.666+0000 7f832c7ff700 1 -- 172.21.15.97:0/3544244864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83240b4f80 msgr2=0x7f83240d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.666+0000 7f832c7ff700 1 --2- 172.21.15.97:0/3544244864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83240b4f80 0x7f83240d0c20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8314009870 tx=0x7f8314009b80 comp rx=0 tx=0).stop 2023-07-06T03:48:30.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.666+0000 7f832c7ff700 1 -- 172.21.15.97:0/3544244864 shutdown_connections 2023-07-06T03:48:30.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.666+0000 7f832c7ff700 1 --2- 172.21.15.97:0/3544244864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83240b4f80 0x7f83240d0c20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.666+0000 7f832c7ff700 1 -- 172.21.15.97:0/3544244864 >> 172.21.15.97:0/3544244864 conn(0x7f83240d2df0 msgr2=0x7f83240d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:30.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.666+0000 7f832c7ff700 1 -- 172.21.15.97:0/3544244864 shutdown_connections 2023-07-06T03:48:30.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.666+0000 7f832c7ff700 1 -- 172.21.15.97:0/3544244864 wait complete. 2023-07-06T03:48:30.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.667+0000 7f832c7ff700 1 Processor -- start 2023-07-06T03:48:30.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.668+0000 7f832c7ff700 1 -- start start 2023-07-06T03:48:30.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.668+0000 7f832c7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83240b4f80 0x7f83240c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:30.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.668+0000 7f832c7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83240d1160 0x7f83240cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:30.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.668+0000 7f832c7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f832415c850 0x7f83240cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:30.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.668+0000 7f832c7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240cab60 con 0x7f83240d1160 2023-07-06T03:48:30.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f832c7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240cae40 con 0x7f83240b4f80 2023-07-06T03:48:30.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f832c7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83240cb120 con 0x7f832415c850 2023-07-06T03:48:30.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f8329d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83240d1160 0x7f83240cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:30.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f8329d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83240d1160 0x7f83240cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39934/0 (socket says 172.21.15.97:39934) 2023-07-06T03:48:30.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f8329d9a700 1 -- 172.21.15.97:0/3961588715 learned_addr learned my addr 172.21.15.97:0/3961588715 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:30.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f8329d9a700 1 -- 172.21.15.97:0/3961588715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f832415c850 msgr2=0x7f83240cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f832ad9c700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f832415c850 0x7f83240cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:30.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f832a59b700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83240b4f80 0x7f83240c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:30.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f8329d9a700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f832415c850 0x7f83240cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f8329d9a700 1 -- 172.21.15.97:0/3961588715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83240b4f80 msgr2=0x7f83240c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f8329d9a700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83240b4f80 0x7f83240c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f8329d9a700 1 -- 172.21.15.97:0/3961588715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83140094a0 con 0x7f83240d1160 2023-07-06T03:48:30.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f832ad9c700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f832415c850 0x7f83240cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:30.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f832a59b700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83240b4f80 0x7f83240c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:30.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.669+0000 7f8329d9a700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83240d1160 0x7f83240cc120 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f83140097c0 tx=0x7f8314004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:30.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.670+0000 7f831affd700 1 -- 172.21.15.97:0/3961588715 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83140045d0 con 0x7f83240d1160 2023-07-06T03:48:30.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.670+0000 7f831affd700 1 -- 172.21.15.97:0/3961588715 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831401f070 con 0x7f83240d1160 2023-07-06T03:48:30.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.670+0000 7f832c7ff700 1 -- 172.21.15.97:0/3961588715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83240cb400 con 0x7f83240d1160 2023-07-06T03:48:30.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.670+0000 7f831affd700 1 -- 172.21.15.97:0/3961588715 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83140038c0 con 0x7f83240d1160 2023-07-06T03:48:30.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.670+0000 7f832c7ff700 1 -- 172.21.15.97:0/3961588715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83240c8f20 con 0x7f83240d1160 2023-07-06T03:48:30.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.671+0000 7f832c7ff700 1 -- 172.21.15.97:0/3961588715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83240726a0 con 0x7f83240d1160 2023-07-06T03:48:30.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.672+0000 7f831affd700 1 -- 172.21.15.97:0/3961588715 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8314003a20 con 0x7f83240d1160 2023-07-06T03:48:30.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.673+0000 7f831affd700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f830806ace0 0x7f830806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:30.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.673+0000 7f831affd700 1 -- 172.21.15.97:0/3961588715 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f8314021070 con 0x7f83240d1160 2023-07-06T03:48:30.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.673+0000 7f832a59b700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f830806ace0 0x7f830806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:30.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.674+0000 7f832a59b700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f830806ace0 0x7f830806d190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8320000f60 tx=0x7f8320009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.677+0000 7f831affd700 1 -- 172.21.15.97:0/3961588715 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8314055e30 con 0x7f83240d1160 2023-07-06T03:48:30.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.916+0000 7f832c7ff700 1 -- 172.21.15.97:0/3961588715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83240caf80 con 0x7f83240d1160 2023-07-06T03:48:30.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.917+0000 7f831affd700 1 -- 172.21.15.97:0/3961588715 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f83240caf80 con 0x7f83240d1160 2023-07-06T03:48:30.919 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:30.919 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1299982704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1299982704},{"type":"v1","addr":"172.21.15.146:6833","nonce":1299982704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5602,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/640394657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":640394657},{"type":"v1","addr":"172.21.15.97:6837","nonce":640394657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2363940464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2363940464},{"type":"v1","addr":"172.21.15.97:6836","nonce":2363940464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:47:36.149445+0000","modified":"2023-07-06T03:47:39.683749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5838},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5838":{"gid":5838,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/3623506860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3623506860},{"type":"v1","addr":"172.21.15.146:6835","nonce":3623506860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T03:48:30.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 -- 172.21.15.97:0/3961588715 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f830806ace0 msgr2=0x7f830806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f830806ace0 0x7f830806d190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8320000f60 tx=0x7f8320009040 comp rx=0 tx=0).stop 2023-07-06T03:48:30.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 -- 172.21.15.97:0/3961588715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83240d1160 msgr2=0x7f83240cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:30.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83240d1160 0x7f83240cc120 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f83140097c0 tx=0x7f8314004750 comp rx=0 tx=0).stop 2023-07-06T03:48:30.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 -- 172.21.15.97:0/3961588715 shutdown_connections 2023-07-06T03:48:30.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f830806ace0 0x7f830806d190 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f832415c850 0x7f83240cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83240b4f80 0x7f83240c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 --2- 172.21.15.97:0/3961588715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83240d1160 0x7f83240cc120 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:30.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 -- 172.21.15.97:0/3961588715 >> 172.21.15.97:0/3961588715 conn(0x7f83240d2df0 msgr2=0x7f83240cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:30.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 -- 172.21.15.97:0/3961588715 shutdown_connections 2023-07-06T03:48:30.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:30.919+0000 7f832c7ff700 1 -- 172.21.15.97:0/3961588715 wait complete. 2023-07-06T03:48:30.923 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2023-07-06T03:48:30.940 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:48:31.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.134+0000 7f90471ff700 1 Processor -- start 2023-07-06T03:48:31.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.135+0000 7f90471ff700 1 -- start start 2023-07-06T03:48:31.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.136+0000 7f90471ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f904015c850 0x7f90400b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:31.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.136+0000 7f90471ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f90400b4db0 0x7f90400d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:31.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.136+0000 7f90471ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f904015cc20 con 0x7f90400d1160 2023-07-06T03:48:31.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.136+0000 7f90471ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90400b5220 con 0x7f90400b4db0 2023-07-06T03:48:31.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.136+0000 7f90471ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90400d1990 con 0x7f904015c850 2023-07-06T03:48:31.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.136+0000 7f903ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f90400b4db0 0x7f90400d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:31.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.136+0000 7f903ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f90400b4db0 0x7f90400d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39950/0 (socket says 172.21.15.97:39950) 2023-07-06T03:48:31.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.136+0000 7f903ffff700 1 -- 172.21.15.97:0/1191415263 learned_addr learned my addr 172.21.15.97:0/1191415263 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:31.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.136+0000 7f904579c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f90400d1160 0x7f904006a620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45196/0 (socket says 172.21.15.97:45196) 2023-07-06T03:48:31.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.136+0000 7f9044f9b700 1 --2- 172.21.15.97:0/1191415263 >> v2:172.21.15.146:3300/0 conn(0x7f904015c850 0x7f90400b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:31.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.137+0000 7f903ffff700 1 -- 172.21.15.97:0/1191415263 >> v2:172.21.15.146:3300/0 conn(0x7f904015c850 msgr2=0x7f90400b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:31.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.137+0000 7f903ffff700 1 --2- 172.21.15.97:0/1191415263 >> v2:172.21.15.146:3300/0 conn(0x7f904015c850 0x7f90400b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.137+0000 7f903ffff700 1 -- 172.21.15.97:0/1191415263 >> v1:172.21.15.97:6789/0 conn(0x7f90400d1160 legacy=0x7f904006a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.137+0000 7f903ffff700 1 -- 172.21.15.97:0/1191415263 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90400d1c70 con 0x7f90400b4db0 2023-07-06T03:48:31.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.137+0000 7f903f7fe700 1 -- 172.21.15.97:0/1191415263 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598081537 0 0) 0x7f904015cc20 con 0x7f90400d1160 2023-07-06T03:48:31.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.137+0000 7f903ffff700 1 --2- 172.21.15.97:0/1191415263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90400b4db0 0x7f90400d0c20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f90400728a0 tx=0x7f902c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded4713f9f5ff18f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:31.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.137+0000 7f903f7fe700 1 -- 172.21.15.97:0/1191415263 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c00b870 con 0x7f90400b4db0 2023-07-06T03:48:31.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.137+0000 7f903f7fe700 1 -- 172.21.15.97:0/1191415263 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c00b9d0 con 0x7f90400b4db0 2023-07-06T03:48:31.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.137+0000 7f903f7fe700 1 -- 172.21.15.97:0/1191415263 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c00bd60 con 0x7f90400b4db0 2023-07-06T03:48:31.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.137+0000 7f90471ff700 1 -- 172.21.15.97:0/1191415263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90400b4db0 msgr2=0x7f90400d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.137+0000 7f90471ff700 1 --2- 172.21.15.97:0/1191415263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90400b4db0 0x7f90400d0c20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f90400728a0 tx=0x7f902c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.138+0000 7f90471ff700 1 -- 172.21.15.97:0/1191415263 shutdown_connections 2023-07-06T03:48:31.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.138+0000 7f90471ff700 1 --2- 172.21.15.97:0/1191415263 >> v2:172.21.15.146:3300/0 conn(0x7f904015c850 0x7f90400b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.138+0000 7f90471ff700 1 --2- 172.21.15.97:0/1191415263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90400b4db0 0x7f90400d0c20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.138+0000 7f90471ff700 1 -- 172.21.15.97:0/1191415263 >> 172.21.15.97:0/1191415263 conn(0x7f90400d2df0 msgr2=0x7f90400d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:31.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.138+0000 7f90471ff700 1 -- 172.21.15.97:0/1191415263 shutdown_connections 2023-07-06T03:48:31.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.138+0000 7f90471ff700 1 -- 172.21.15.97:0/1191415263 wait complete. 2023-07-06T03:48:31.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.139+0000 7f90471ff700 1 Processor -- start 2023-07-06T03:48:31.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.139+0000 7f90471ff700 1 -- start start 2023-07-06T03:48:31.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.140+0000 7f90471ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90400b4db0 0x7f90400cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:31.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.140+0000 7f90471ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90400d1160 0x7f9040156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:31.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.140+0000 7f90471ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f904015c850 0x7f904015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:31.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.140+0000 7f90471ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90401f1930 con 0x7f90400d1160 2023-07-06T03:48:31.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.140+0000 7f90471ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90401f28f0 con 0x7f904015c850 2023-07-06T03:48:31.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.140+0000 7f90471ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904015bef0 con 0x7f90400b4db0 2023-07-06T03:48:31.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f903ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90400d1160 0x7f9040156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:31.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f903ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90400d1160 0x7f9040156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39960/0 (socket says 172.21.15.97:39960) 2023-07-06T03:48:31.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f903ffff700 1 -- 172.21.15.97:0/4008809410 learned_addr learned my addr 172.21.15.97:0/4008809410 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:31.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f9044f9b700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90400b4db0 0x7f90400cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f904579c700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f904015c850 0x7f904015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f903ffff700 1 -- 172.21.15.97:0/4008809410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90400b4db0 msgr2=0x7f90400cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f903ffff700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90400b4db0 0x7f90400cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f903ffff700 1 -- 172.21.15.97:0/4008809410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f904015c850 msgr2=0x7f904015f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f903ffff700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f904015c850 0x7f904015f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f903ffff700 1 -- 172.21.15.97:0/4008809410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c009550 con 0x7f90400d1160 2023-07-06T03:48:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f904579c700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f904015c850 0x7f904015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f903ffff700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90400d1160 0x7f9040156350 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f902c009c10 tx=0x7f902c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.141+0000 7f903d7fa700 1 -- 172.21.15.97:0/4008809410 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c004310 con 0x7f90400d1160 2023-07-06T03:48:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.142+0000 7f903d7fa700 1 -- 172.21.15.97:0/4008809410 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c01f070 con 0x7f90400d1160 2023-07-06T03:48:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.142+0000 7f903d7fa700 1 -- 172.21.15.97:0/4008809410 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c0036a0 con 0x7f90400d1160 2023-07-06T03:48:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.142+0000 7f90471ff700 1 -- 172.21.15.97:0/4008809410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90401f1b30 con 0x7f90400d1160 2023-07-06T03:48:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.142+0000 7f90471ff700 1 -- 172.21.15.97:0/4008809410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90401f20f0 con 0x7f90400d1160 2023-07-06T03:48:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.143+0000 7f90471ff700 1 -- 172.21.15.97:0/4008809410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90400726a0 con 0x7f90400d1160 2023-07-06T03:48:31.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.143+0000 7f903d7fa700 1 -- 172.21.15.97:0/4008809410 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f902c021030 con 0x7f90400d1160 2023-07-06T03:48:31.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.144+0000 7f903d7fa700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f902406ace0 0x7f902406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:31.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.145+0000 7f903d7fa700 1 -- 172.21.15.97:0/4008809410 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f902c08a8b0 con 0x7f90400d1160 2023-07-06T03:48:31.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.145+0000 7f9044f9b700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f902406ace0 0x7f902406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:31.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.145+0000 7f9044f9b700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f902406ace0 0x7f902406d190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f903400b9b0 tx=0x7f9034008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:31.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.148+0000 7f903d7fa700 1 -- 172.21.15.97:0/4008809410 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f902c0558d0 con 0x7f90400d1160 2023-07-06T03:48:31.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.384+0000 7f90471ff700 1 -- 172.21.15.97:0/4008809410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9040074390 con 0x7f90400d1160 2023-07-06T03:48:31.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.385+0000 7f903d7fa700 1 -- 172.21.15.97:0/4008809410 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f902c058ef0 con 0x7f90400d1160 2023-07-06T03:48:31.387 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:31.387 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1299982704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1299982704},{"type":"v1","addr":"172.21.15.146:6833","nonce":1299982704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5602,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/640394657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":640394657},{"type":"v1","addr":"172.21.15.97:6837","nonce":640394657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2363940464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2363940464},{"type":"v1","addr":"172.21.15.97:6836","nonce":2363940464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:47:36.149445+0000","modified":"2023-07-06T03:47:39.683749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5838},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5838":{"gid":5838,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6835/3623506860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3623506860},{"type":"v1","addr":"172.21.15.146:6835","nonce":3623506860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T03:48:31.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 -- 172.21.15.97:0/4008809410 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f902406ace0 msgr2=0x7f902406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f902406ace0 0x7f902406d190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f903400b9b0 tx=0x7f9034008040 comp rx=0 tx=0).stop 2023-07-06T03:48:31.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 -- 172.21.15.97:0/4008809410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90400d1160 msgr2=0x7f9040156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90400d1160 0x7f9040156350 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f902c009c10 tx=0x7f902c004750 comp rx=0 tx=0).stop 2023-07-06T03:48:31.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 -- 172.21.15.97:0/4008809410 shutdown_connections 2023-07-06T03:48:31.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f902406ace0 0x7f902406d190 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f904015c850 0x7f904015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f90400b4db0 0x7f90400cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 --2- 172.21.15.97:0/4008809410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90400d1160 0x7f9040156350 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 -- 172.21.15.97:0/4008809410 >> 172.21.15.97:0/4008809410 conn(0x7f90400d2df0 msgr2=0x7f90400d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:31.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 -- 172.21.15.97:0/4008809410 shutdown_connections 2023-07-06T03:48:31.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.387+0000 7f90471ff700 1 -- 172.21.15.97:0/4008809410 wait complete. 2023-07-06T03:48:31.391 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2023-07-06T03:48:31.407 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:48:31.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.600+0000 7f28499ff700 1 Processor -- start 2023-07-06T03:48:31.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.600+0000 7f28499ff700 1 -- start start 2023-07-06T03:48:31.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.601+0000 7f28499ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2844071ad0 0x7f2844141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:31.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.601+0000 7f28499ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2844071fb0 con 0x7f284406c6a0 2023-07-06T03:48:31.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.601+0000 7f28499ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2844159230 con 0x7f2844141ad0 2023-07-06T03:48:31.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.601+0000 7f28499ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2844159510 con 0x7f2844071ad0 2023-07-06T03:48:31.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.601+0000 7f28427fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f284406c6a0 0x7f28440ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45212/0 (socket says 172.21.15.97:45212) 2023-07-06T03:48:31.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.601+0000 7f28427fc700 1 -- 172.21.15.97:0/771753352 learned_addr learned my addr 172.21.15.97:0/771753352 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:31.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.602+0000 7f28437fe700 1 --2- 172.21.15.97:0/771753352 >> v2:172.21.15.97:3300/0 conn(0x7f2844071ad0 0x7f2844141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:31.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.602+0000 7f2841ffb700 1 -- 172.21.15.97:0/771753352 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057740686 0 0) 0x7f2844071fb0 con 0x7f284406c6a0 2023-07-06T03:48:31.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.602+0000 7f2841ffb700 1 -- 172.21.15.97:0/771753352 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f282c0035b0 con 0x7f284406c6a0 2023-07-06T03:48:31.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.602+0000 7f28437fe700 1 -- 172.21.15.97:0/771753352 >> v1:172.21.15.146:6789/0 conn(0x7f2844141ad0 legacy=0x7f284406bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.602+0000 7f28437fe700 1 -- 172.21.15.97:0/771753352 >> v1:172.21.15.97:6789/0 conn(0x7f284406c6a0 legacy=0x7f28440ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.602+0000 7f28437fe700 1 -- 172.21.15.97:0/771753352 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28441597f0 con 0x7f2844071ad0 2023-07-06T03:48:31.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.602+0000 7f28437fe700 1 --2- 172.21.15.97:0/771753352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844071ad0 0x7f2844141ea0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f283c00aea0 tx=0x7f283c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5215de38eb3e8f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:31.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.602+0000 7f2841ffb700 1 -- 172.21.15.97:0/771753352 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c00f070 con 0x7f2844071ad0 2023-07-06T03:48:31.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.603+0000 7f2841ffb700 1 -- 172.21.15.97:0/771753352 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c00c910 con 0x7f2844071ad0 2023-07-06T03:48:31.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.603+0000 7f2841ffb700 1 -- 172.21.15.97:0/771753352 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c00ccc0 con 0x7f2844071ad0 2023-07-06T03:48:31.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.603+0000 7f28499ff700 1 -- 172.21.15.97:0/771753352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844071ad0 msgr2=0x7f2844141ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.603+0000 7f28499ff700 1 --2- 172.21.15.97:0/771753352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844071ad0 0x7f2844141ea0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f283c00aea0 tx=0x7f283c00c330 comp rx=0 tx=0).stop 2023-07-06T03:48:31.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.603+0000 7f28499ff700 1 -- 172.21.15.97:0/771753352 shutdown_connections 2023-07-06T03:48:31.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.603+0000 7f28499ff700 1 --2- 172.21.15.97:0/771753352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844071ad0 0x7f2844141ea0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.603+0000 7f28499ff700 1 -- 172.21.15.97:0/771753352 >> 172.21.15.97:0/771753352 conn(0x7f2844065c20 msgr2=0x7f2844065ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:31.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.603+0000 7f28499ff700 1 -- 172.21.15.97:0/771753352 shutdown_connections 2023-07-06T03:48:31.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.603+0000 7f28499ff700 1 -- 172.21.15.97:0/771753352 wait complete. 2023-07-06T03:48:31.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.605+0000 7f28499ff700 1 Processor -- start 2023-07-06T03:48:31.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.605+0000 7f28499ff700 1 -- start start 2023-07-06T03:48:31.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.606+0000 7f28499ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f284406c6a0 0x7f2844139fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:31.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.606+0000 7f28499ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844071ad0 0x7f28440ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:31.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.606+0000 7f28499ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2844141ad0 0x7f28440ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:31.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.606+0000 7f28499ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284413d000 con 0x7f2844071ad0 2023-07-06T03:48:31.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.606+0000 7f28499ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284413d2e0 con 0x7f284406c6a0 2023-07-06T03:48:31.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.606+0000 7f28499ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284413d5c0 con 0x7f2844141ad0 2023-07-06T03:48:31.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.606+0000 7f28427fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844071ad0 0x7f28440ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:31.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.606+0000 7f28427fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844071ad0 0x7f28440ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39988/0 (socket says 172.21.15.97:39988) 2023-07-06T03:48:31.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.606+0000 7f28427fc700 1 -- 172.21.15.97:0/2942455692 learned_addr learned my addr 172.21.15.97:0/2942455692 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:31.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f28437fe700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2844141ad0 0x7f28440ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:31.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f2842ffd700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f284406c6a0 0x7f2844139fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:31.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f28427fc700 1 -- 172.21.15.97:0/2942455692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2844141ad0 msgr2=0x7f28440ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f28427fc700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2844141ad0 0x7f28440ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f28427fc700 1 -- 172.21.15.97:0/2942455692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f284406c6a0 msgr2=0x7f2844139fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f28427fc700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f284406c6a0 0x7f2844139fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f28427fc700 1 -- 172.21.15.97:0/2942455692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c00ab80 con 0x7f2844071ad0 2023-07-06T03:48:31.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f28437fe700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2844141ad0 0x7f28440ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:31.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f2842ffd700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f284406c6a0 0x7f2844139fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:31.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f28427fc700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844071ad0 0x7f28440ce970 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2838005910 tx=0x7f283800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:31.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f282bfff700 1 -- 172.21.15.97:0/2942455692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838005d70 con 0x7f2844071ad0 2023-07-06T03:48:31.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f282bfff700 1 -- 172.21.15.97:0/2942455692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28380070d0 con 0x7f2844071ad0 2023-07-06T03:48:31.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f28499ff700 1 -- 172.21.15.97:0/2942455692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284413d8a0 con 0x7f2844071ad0 2023-07-06T03:48:31.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f282bfff700 1 -- 172.21.15.97:0/2942455692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838007460 con 0x7f2844071ad0 2023-07-06T03:48:31.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.607+0000 7f28499ff700 1 -- 172.21.15.97:0/2942455692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284413e0a0 con 0x7f2844071ad0 2023-07-06T03:48:31.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.608+0000 7f28499ff700 1 -- 172.21.15.97:0/2942455692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284413ddc0 con 0x7f2844071ad0 2023-07-06T03:48:31.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.609+0000 7f282bfff700 1 -- 172.21.15.97:0/2942455692 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28380075c0 con 0x7f2844071ad0 2023-07-06T03:48:31.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.610+0000 7f282bfff700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283006ace0 0x7f283006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:31.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.610+0000 7f282bfff700 1 -- 172.21.15.97:0/2942455692 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f2838088b20 con 0x7f2844071ad0 2023-07-06T03:48:31.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.610+0000 7f2842ffd700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283006ace0 0x7f283006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:31.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.611+0000 7f2842ffd700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283006ace0 0x7f283006d190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2834000f60 tx=0x7f2834009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:31.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.614+0000 7f282bfff700 1 -- 172.21.15.97:0/2942455692 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2838053b50 con 0x7f2844071ad0 2023-07-06T03:48:31.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.813+0000 7f28499ff700 1 -- 172.21.15.97:0/2942455692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2844074390 con 0x7f2844071ad0 2023-07-06T03:48:31.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.814+0000 7f282bfff700 1 -- 172.21.15.97:0/2942455692 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18524 (secure 0 0 0) 0x7f2838057170 con 0x7f2844071ad0 2023-07-06T03:48:31.816 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:31.816 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":49,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:47:52.725520+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:47:33.695348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-07-06T03:47:35.132699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":2,"to":5}]},{"pgid":"8.1a","mappings":[{"from":2,"to":1}]},{"pgid":"8.38","mappings":[{"from":6,"to":5}]},{"pgid":"8.3d","mappings":[{"from":6,"to":0},{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:48:31.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 -- 172.21.15.97:0/2942455692 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283006ace0 msgr2=0x7f283006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283006ace0 0x7f283006d190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2834000f60 tx=0x7f2834009040 comp rx=0 tx=0).stop 2023-07-06T03:48:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 -- 172.21.15.97:0/2942455692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844071ad0 msgr2=0x7f28440ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844071ad0 0x7f28440ce970 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2838005910 tx=0x7f283800ca90 comp rx=0 tx=0).stop 2023-07-06T03:48:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 -- 172.21.15.97:0/2942455692 shutdown_connections 2023-07-06T03:48:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283006ace0 0x7f283006d190 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2844141ad0 0x7f28440ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f284406c6a0 0x7f2844139fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 --2- 172.21.15.97:0/2942455692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2844071ad0 0x7f28440ce970 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:31.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 -- 172.21.15.97:0/2942455692 >> 172.21.15.97:0/2942455692 conn(0x7f2844065c20 msgr2=0x7f284406be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:31.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 -- 172.21.15.97:0/2942455692 shutdown_connections 2023-07-06T03:48:31.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:31.817+0000 7f28499ff700 1 -- 172.21.15.97:0/2942455692 wait complete. 2023-07-06T03:48:31.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T03:48:32.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.031+0000 7f47a73ff700 1 Processor -- start 2023-07-06T03:48:32.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.031+0000 7f47a73ff700 1 -- start start 2023-07-06T03:48:32.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a73ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f47a01593b0 0x7f47a00b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a73ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47a00b2d50 0x7f47a006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a73ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a0159780 con 0x7f47a00ce1f0 2023-07-06T03:48:32.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a73ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a0070860 con 0x7f47a00b2d50 2023-07-06T03:48:32.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a73ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a0070b40 con 0x7f47a01593b0 2023-07-06T03:48:32.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a599c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47a00b2d50 0x7f47a006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a499a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f47a00ce1f0 0x7f47a00b2640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45218/0 (socket says 172.21.15.97:45218) 2023-07-06T03:48:32.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a599c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47a00b2d50 0x7f47a006c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39998/0 (socket says 172.21.15.97:39998) 2023-07-06T03:48:32.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a499a700 1 -- 172.21.15.97:0/2891794238 learned_addr learned my addr 172.21.15.97:0/2891794238 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a519b700 1 --2- 172.21.15.97:0/2891794238 >> v2:172.21.15.146:3300/0 conn(0x7f47a01593b0 0x7f47a00b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a599c700 1 -- 172.21.15.97:0/2891794238 >> v2:172.21.15.146:3300/0 conn(0x7f47a01593b0 msgr2=0x7f47a00b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a599c700 1 --2- 172.21.15.97:0/2891794238 >> v2:172.21.15.146:3300/0 conn(0x7f47a01593b0 0x7f47a00b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a599c700 1 -- 172.21.15.97:0/2891794238 >> v1:172.21.15.97:6789/0 conn(0x7f47a00ce1f0 legacy=0x7f47a00b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:32.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f47a599c700 1 -- 172.21.15.97:0/2891794238 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a006c0c0 con 0x7f47a00b2d50 2023-07-06T03:48:32.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.033+0000 7f4797fff700 1 -- 172.21.15.97:0/2891794238 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526743076 0 0) 0x7f47a0159780 con 0x7f47a00ce1f0 2023-07-06T03:48:32.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.034+0000 7f47a599c700 1 --2- 172.21.15.97:0/2891794238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a00b2d50 0x7f47a006c5e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f47a006db30 tx=0x7f479800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1586c4da2fb39ef server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:32.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.034+0000 7f4797fff700 1 -- 172.21.15.97:0/2891794238 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479800e070 con 0x7f47a00b2d50 2023-07-06T03:48:32.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.034+0000 7f4797fff700 1 -- 172.21.15.97:0/2891794238 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479800bbd0 con 0x7f47a00b2d50 2023-07-06T03:48:32.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.034+0000 7f4797fff700 1 -- 172.21.15.97:0/2891794238 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798012430 con 0x7f47a00b2d50 2023-07-06T03:48:32.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.034+0000 7f47a73ff700 1 -- 172.21.15.97:0/2891794238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a00b2d50 msgr2=0x7f47a006c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:32.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.034+0000 7f47a73ff700 1 --2- 172.21.15.97:0/2891794238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a00b2d50 0x7f47a006c5e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f47a006db30 tx=0x7f479800b330 comp rx=0 tx=0).stop 2023-07-06T03:48:32.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.035+0000 7f47a73ff700 1 -- 172.21.15.97:0/2891794238 shutdown_connections 2023-07-06T03:48:32.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.035+0000 7f47a73ff700 1 --2- 172.21.15.97:0/2891794238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a00b2d50 0x7f47a006c5e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.035+0000 7f47a73ff700 1 --2- 172.21.15.97:0/2891794238 >> v2:172.21.15.146:3300/0 conn(0x7f47a01593b0 0x7f47a00b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.035+0000 7f47a73ff700 1 -- 172.21.15.97:0/2891794238 >> 172.21.15.97:0/2891794238 conn(0x7f47a0065c20 msgr2=0x7f47a0065ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.035+0000 7f47a73ff700 1 -- 172.21.15.97:0/2891794238 shutdown_connections 2023-07-06T03:48:32.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.035+0000 7f47a73ff700 1 -- 172.21.15.97:0/2891794238 wait complete. 2023-07-06T03:48:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.036+0000 7f47a73ff700 1 Processor -- start 2023-07-06T03:48:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.036+0000 7f47a73ff700 1 -- start start 2023-07-06T03:48:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.037+0000 7f47a73ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a00b2d50 0x7f47a0165770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.037+0000 7f47a73ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a00ce1f0 0x7f47a0142620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.037+0000 7f47a73ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47a01593b0 0x7f47a0146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.037+0000 7f47a73ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a0142b60 con 0x7f47a00b2d50 2023-07-06T03:48:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.037+0000 7f47a73ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a0147150 con 0x7f47a00ce1f0 2023-07-06T03:48:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.037+0000 7f47a73ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a01484b0 con 0x7f47a01593b0 2023-07-06T03:48:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a519b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a00b2d50 0x7f47a0165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a519b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a00b2d50 0x7f47a0165770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40008/0 (socket says 172.21.15.97:40008) 2023-07-06T03:48:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a519b700 1 -- 172.21.15.97:0/3562492048 learned_addr learned my addr 172.21.15.97:0/3562492048 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a499a700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a00ce1f0 0x7f47a0142620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a519b700 1 -- 172.21.15.97:0/3562492048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47a01593b0 msgr2=0x7f47a0146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a599c700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47a01593b0 0x7f47a0146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a519b700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47a01593b0 0x7f47a0146c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a519b700 1 -- 172.21.15.97:0/3562492048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a00ce1f0 msgr2=0x7f47a0142620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a519b700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a00ce1f0 0x7f47a0142620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a519b700 1 -- 172.21.15.97:0/3562492048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4798009a70 con 0x7f47a00b2d50 2023-07-06T03:48:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a499a700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a00ce1f0 0x7f47a0142620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:32.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f47a519b700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a00b2d50 0x7f47a0165770 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f479c00d800 tx=0x7f479c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.038+0000 7f4795ffb700 1 -- 172.21.15.97:0/3562492048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c0125f0 con 0x7f47a00b2d50 2023-07-06T03:48:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.039+0000 7f4795ffb700 1 -- 172.21.15.97:0/3562492048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c012750 con 0x7f47a00b2d50 2023-07-06T03:48:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.039+0000 7f4795ffb700 1 -- 172.21.15.97:0/3562492048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c012b00 con 0x7f47a00b2d50 2023-07-06T03:48:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.039+0000 7f47a73ff700 1 -- 172.21.15.97:0/3562492048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a0147410 con 0x7f47a00b2d50 2023-07-06T03:48:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.039+0000 7f47a73ff700 1 -- 172.21.15.97:0/3562492048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a01479d0 con 0x7f47a00b2d50 2023-07-06T03:48:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.040+0000 7f47a73ff700 1 -- 172.21.15.97:0/3562492048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a00ce5e0 con 0x7f47a00b2d50 2023-07-06T03:48:32.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.040+0000 7f4795ffb700 1 -- 172.21.15.97:0/3562492048 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f479c012c60 con 0x7f47a00b2d50 2023-07-06T03:48:32.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.041+0000 7f4795ffb700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f478406ace0 0x7f478406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.041+0000 7f4795ffb700 1 -- 172.21.15.97:0/3562492048 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6186+0+0 (secure 0 0 0) 0x7f479c089980 con 0x7f47a00b2d50 2023-07-06T03:48:32.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.042+0000 7f47a499a700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f478406ace0 0x7f478406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.042+0000 7f47a499a700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f478406ace0 0x7f478406d190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f47a00ad890 tx=0x7f4790007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:32.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.045+0000 7f4795ffb700 1 -- 172.21.15.97:0/3562492048 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f479c054910 con 0x7f47a00b2d50 2023-07-06T03:48:32.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.283+0000 7f47a73ff700 1 -- 172.21.15.97:0/3562492048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f47a0074390 con 0x7f47a00b2d50 2023-07-06T03:48:32.285 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-07-06T03:48:32.284+0000 7fb463ff4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T03:48:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.290+0000 7f4795ffb700 1 -- 172.21.15.97:0/3562492048 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f479c057f30 con 0x7f47a00b2d50 2023-07-06T03:48:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.292+0000 7f47a73ff700 1 -- 172.21.15.97:0/3562492048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f478406ace0 msgr2=0x7f478406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.292+0000 7f47a73ff700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f478406ace0 0x7f478406d190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f47a00ad890 tx=0x7f4790007ca0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.292+0000 7f47a73ff700 1 -- 172.21.15.97:0/3562492048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a00b2d50 msgr2=0x7f47a0165770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.293+0000 7f47a73ff700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a00b2d50 0x7f47a0165770 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f479c00d800 tx=0x7f479c00dbc0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.293+0000 7f47a73ff700 1 -- 172.21.15.97:0/3562492048 shutdown_connections 2023-07-06T03:48:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.293+0000 7f47a73ff700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f478406ace0 0x7f478406d190 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.293+0000 7f47a73ff700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47a01593b0 0x7f47a0146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.293+0000 7f47a73ff700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a00b2d50 0x7f47a0165770 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.293+0000 7f47a73ff700 1 --2- 172.21.15.97:0/3562492048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a00ce1f0 0x7f47a0142620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.293+0000 7f47a73ff700 1 -- 172.21.15.97:0/3562492048 >> 172.21.15.97:0/3562492048 conn(0x7f47a0065c20 msgr2=0x7f47a00cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.293+0000 7f47a73ff700 1 -- 172.21.15.97:0/3562492048 shutdown_connections 2023-07-06T03:48:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.293+0000 7f47a73ff700 1 -- 172.21.15.97:0/3562492048 wait complete. 2023-07-06T03:48:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T03:48:32.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T03:48:32.361 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:48:32.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.507+0000 7f7059fff700 1 Processor -- start 2023-07-06T03:48:32.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.508+0000 7f7059fff700 1 -- start start 2023-07-06T03:48:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.509+0000 7f7059fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f705415c540 0x7f7054160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.509+0000 7f7059fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70540b4870 0x7f70540b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.509+0000 7f7059fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f705406c090 0x7f705406c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.509+0000 7f7059fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70540b5220 con 0x7f705415c540 2023-07-06T03:48:32.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.509+0000 7f7059fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705415c910 con 0x7f705406c090 2023-07-06T03:48:32.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.509+0000 7f7059fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7054160fc0 con 0x7f70540b4870 2023-07-06T03:48:32.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.509+0000 7f70537fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f705415c540 0x7f7054160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.509+0000 7f70537fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f705415c540 0x7f7054160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40014/0 (socket says 172.21.15.97:40014) 2023-07-06T03:48:32.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.509+0000 7f70537fe700 1 -- 172.21.15.97:0/1015224487 learned_addr learned my addr 172.21.15.97:0/1015224487 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.509+0000 7f704bfff700 1 --2- 172.21.15.97:0/1015224487 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70540b4870 0x7f70540b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.509+0000 7f7053fff700 1 --2- 172.21.15.97:0/1015224487 >> v2:172.21.15.146:3300/0 conn(0x7f705406c090 0x7f705406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.510+0000 7f70537fe700 1 -- 172.21.15.97:0/1015224487 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70540b4870 msgr2=0x7f70540b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.510+0000 7f70537fe700 1 --2- 172.21.15.97:0/1015224487 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70540b4870 0x7f70540b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.510+0000 7f70537fe700 1 -- 172.21.15.97:0/1015224487 >> v2:172.21.15.146:3300/0 conn(0x7f705406c090 msgr2=0x7f705406c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.510+0000 7f70537fe700 1 --2- 172.21.15.97:0/1015224487 >> v2:172.21.15.146:3300/0 conn(0x7f705406c090 0x7f705406c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.510+0000 7f70537fe700 1 -- 172.21.15.97:0/1015224487 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70540cd770 con 0x7f705415c540 2023-07-06T03:48:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.510+0000 7f7053fff700 1 --2- 172.21.15.97:0/1015224487 >> v2:172.21.15.146:3300/0 conn(0x7f705406c090 0x7f705406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.510+0000 7f70537fe700 1 --2- 172.21.15.97:0/1015224487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705415c540 0x7f7054160a80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f705408c0a0 tx=0x7f7044009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299ba4a0ac3c65df server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:32.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.510+0000 7f7052ffd700 1 -- 172.21.15.97:0/1015224487 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044011830 con 0x7f705415c540 2023-07-06T03:48:32.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.510+0000 7f7052ffd700 1 -- 172.21.15.97:0/1015224487 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7044011990 con 0x7f705415c540 2023-07-06T03:48:32.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.510+0000 7f7052ffd700 1 -- 172.21.15.97:0/1015224487 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044011d20 con 0x7f705415c540 2023-07-06T03:48:32.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.511+0000 7f7059fff700 1 -- 172.21.15.97:0/1015224487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705415c540 msgr2=0x7f7054160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:32.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.511+0000 7f7059fff700 1 --2- 172.21.15.97:0/1015224487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705415c540 0x7f7054160a80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f705408c0a0 tx=0x7f7044009bd0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.511+0000 7f7059fff700 1 -- 172.21.15.97:0/1015224487 shutdown_connections 2023-07-06T03:48:32.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.511+0000 7f7059fff700 1 --2- 172.21.15.97:0/1015224487 >> v2:172.21.15.146:3300/0 conn(0x7f705406c090 0x7f705406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.511+0000 7f7059fff700 1 --2- 172.21.15.97:0/1015224487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705415c540 0x7f7054160a80 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.511+0000 7f7059fff700 1 --2- 172.21.15.97:0/1015224487 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70540b4870 0x7f70540b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.511+0000 7f7059fff700 1 -- 172.21.15.97:0/1015224487 >> 172.21.15.97:0/1015224487 conn(0x7f705406b2f0 msgr2=0x7f70540cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:32.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.511+0000 7f7059fff700 1 -- 172.21.15.97:0/1015224487 shutdown_connections 2023-07-06T03:48:32.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.511+0000 7f7059fff700 1 -- 172.21.15.97:0/1015224487 wait complete. 2023-07-06T03:48:32.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.513+0000 7f7059fff700 1 Processor -- start 2023-07-06T03:48:32.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.513+0000 7f7059fff700 1 -- start start 2023-07-06T03:48:32.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7059fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f705406c090 0x7f70541573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7059fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70540b4870 0x7f70540b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7059fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705415c540 0x7f70540c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7059fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70541578f0 con 0x7f705415c540 2023-07-06T03:48:32.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7059fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70540c7460 con 0x7f705406c090 2023-07-06T03:48:32.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7059fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70540b6ed0 con 0x7f70540b4870 2023-07-06T03:48:32.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7053fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705415c540 0x7f70540c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7053fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705415c540 0x7f70540c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40024/0 (socket says 172.21.15.97:40024) 2023-07-06T03:48:32.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7053fff700 1 -- 172.21.15.97:0/142357539 learned_addr learned my addr 172.21.15.97:0/142357539 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:32.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f704bfff700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70540b4870 0x7f70540b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f70537fe700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f705406c090 0x7f70541573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7053fff700 1 -- 172.21.15.97:0/142357539 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70540b4870 msgr2=0x7f70540b79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:32.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7053fff700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70540b4870 0x7f70540b79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7053fff700 1 -- 172.21.15.97:0/142357539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f705406c090 msgr2=0x7f70541573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:32.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7053fff700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f705406c090 0x7f70541573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:32.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f7053fff700 1 -- 172.21.15.97:0/142357539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7044009550 con 0x7f705415c540 2023-07-06T03:48:32.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.514+0000 7f704bfff700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70540b4870 0x7f70540b79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:32.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.515+0000 7f7053fff700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705415c540 0x7f70540c68f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f704c007df0 tx=0x7f704c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:32.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.515+0000 7f7050ff9700 1 -- 172.21.15.97:0/142357539 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c00ddd0 con 0x7f705415c540 2023-07-06T03:48:32.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.515+0000 7f7050ff9700 1 -- 172.21.15.97:0/142357539 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c00f040 con 0x7f705415c540 2023-07-06T03:48:32.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.515+0000 7f7059fff700 1 -- 172.21.15.97:0/142357539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70540b71b0 con 0x7f705415c540 2023-07-06T03:48:32.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.515+0000 7f7050ff9700 1 -- 172.21.15.97:0/142357539 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c003c60 con 0x7f705415c540 2023-07-06T03:48:32.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.515+0000 7f7059fff700 1 -- 172.21.15.97:0/142357539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705415ef10 con 0x7f705415c540 2023-07-06T03:48:32.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.516+0000 7f7059fff700 1 -- 172.21.15.97:0/142357539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70540726a0 con 0x7f705415c540 2023-07-06T03:48:32.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.517+0000 7f7050ff9700 1 -- 172.21.15.97:0/142357539 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f704c004030 con 0x7f705415c540 2023-07-06T03:48:32.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.518+0000 7f7050ff9700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f703406ace0 0x7f703406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:32.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.518+0000 7f7050ff9700 1 -- 172.21.15.97:0/142357539 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6272+0+0 (secure 0 0 0) 0x7f704c089620 con 0x7f705415c540 2023-07-06T03:48:32.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.518+0000 7f70537fe700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f703406ace0 0x7f703406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:32.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.519+0000 7f70537fe700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f703406ace0 0x7f703406d190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7044009c10 tx=0x7f7044008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:32.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.522+0000 7f7050ff9700 1 -- 172.21.15.97:0/142357539 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f704c054640 con 0x7f705415c540 2023-07-06T03:48:32.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:32.761+0000 7f7059fff700 1 -- 172.21.15.97:0/142357539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f70540b76d0 con 0x7f705415c540 2023-07-06T03:48:33.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.290+0000 7f7050ff9700 1 -- 172.21.15.97:0/142357539 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f70540b76d0 con 0x7f705415c540 2023-07-06T03:48:33.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 -- 172.21.15.97:0/142357539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f703406ace0 msgr2=0x7f703406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:33.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f703406ace0 0x7f703406d190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7044009c10 tx=0x7f7044008030 comp rx=0 tx=0).stop 2023-07-06T03:48:33.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 -- 172.21.15.97:0/142357539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705415c540 msgr2=0x7f70540c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:33.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705415c540 0x7f70540c68f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f704c007df0 tx=0x7f704c00d3b0 comp rx=0 tx=0).stop 2023-07-06T03:48:33.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 -- 172.21.15.97:0/142357539 shutdown_connections 2023-07-06T03:48:33.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f703406ace0 0x7f703406d190 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:33.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705415c540 0x7f70540c68f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:33.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f705406c090 0x7f70541573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:33.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 --2- 172.21.15.97:0/142357539 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70540b4870 0x7f70540b79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:33.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 -- 172.21.15.97:0/142357539 >> 172.21.15.97:0/142357539 conn(0x7f705406b2f0 msgr2=0x7f7054160100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:33.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 -- 172.21.15.97:0/142357539 shutdown_connections 2023-07-06T03:48:33.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.293+0000 7f7059fff700 1 -- 172.21.15.97:0/142357539 wait complete. 2023-07-06T03:48:33.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T03:48:33.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.510+0000 7fb6ea3ff700 1 Processor -- start 2023-07-06T03:48:33.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.511+0000 7fb6ea3ff700 1 -- start start 2023-07-06T03:48:33.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.512+0000 7fb6ea3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6e40b4870 0x7fb6e40b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:33.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.512+0000 7fb6ea3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e415c910 con 0x7fb6e40d14a0 2023-07-06T03:48:33.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.512+0000 7fb6ea3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e40cd770 con 0x7fb6e415c540 2023-07-06T03:48:33.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.512+0000 7fb6ea3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e40b5250 con 0x7fb6e40b4870 2023-07-06T03:48:33.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.512+0000 7fb6dbfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6e40b4870 0x7fb6e40b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:33.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.512+0000 7fb6e899c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb6e40d14a0 0x7fb6e40d18e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45230/0 (socket says 172.21.15.97:45230) 2023-07-06T03:48:33.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.512+0000 7fb6dbfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6e40b4870 0x7fb6e40b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40040/0 (socket says 172.21.15.97:40040) 2023-07-06T03:48:33.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.512+0000 7fb6e899c700 1 -- 172.21.15.97:0/2436933312 learned_addr learned my addr 172.21.15.97:0/2436933312 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:33.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.512+0000 7fb6dbfff700 1 -- 172.21.15.97:0/2436933312 >> v1:172.21.15.146:6789/0 conn(0x7fb6e415c540 legacy=0x7fb6e4160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:33.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.512+0000 7fb6dbfff700 1 -- 172.21.15.97:0/2436933312 >> v1:172.21.15.97:6789/0 conn(0x7fb6e40d14a0 legacy=0x7fb6e40d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:33.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.513+0000 7fb6dbfff700 1 -- 172.21.15.97:0/2436933312 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e40cda50 con 0x7fb6e40b4870 2023-07-06T03:48:33.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.513+0000 7fb6e37fe700 1 -- 172.21.15.97:0/2436933312 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042318085 0 0) 0x7fb6e415c910 con 0x7fb6e40d14a0 2023-07-06T03:48:33.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.513+0000 7fb6dbfff700 1 --2- 172.21.15.97:0/2436933312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e40b4870 0x7fb6e40b4d10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009870 tx=0x7fb6cc009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9ca0a6577d072c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:33.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.513+0000 7fb6e37fe700 1 -- 172.21.15.97:0/2436933312 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc004230 con 0x7fb6e40b4870 2023-07-06T03:48:33.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.513+0000 7fb6e37fe700 1 -- 172.21.15.97:0/2436933312 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc004390 con 0x7fb6e40b4870 2023-07-06T03:48:33.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.513+0000 7fb6e37fe700 1 -- 172.21.15.97:0/2436933312 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0036a0 con 0x7fb6e40b4870 2023-07-06T03:48:33.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.513+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2436933312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e40b4870 msgr2=0x7fb6e40b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:33.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.514+0000 7fb6ea3ff700 1 --2- 172.21.15.97:0/2436933312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e40b4870 0x7fb6e40b4d10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009870 tx=0x7fb6cc009b80 comp rx=0 tx=0).stop 2023-07-06T03:48:33.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.514+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2436933312 shutdown_connections 2023-07-06T03:48:33.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.514+0000 7fb6ea3ff700 1 --2- 172.21.15.97:0/2436933312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e40b4870 0x7fb6e40b4d10 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:33.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.514+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2436933312 >> 172.21.15.97:0/2436933312 conn(0x7fb6e406b2f0 msgr2=0x7fb6e40cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:33.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.514+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2436933312 shutdown_connections 2023-07-06T03:48:33.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.514+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2436933312 wait complete. 2023-07-06T03:48:33.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.516+0000 7fb6ea3ff700 1 Processor -- start 2023-07-06T03:48:33.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.516+0000 7fb6ea3ff700 1 -- start start 2023-07-06T03:48:33.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.517+0000 7fb6ea3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e40b4870 0x7fb6e40c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:33.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.517+0000 7fb6ea3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6e40d14a0 0x7fb6e414d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:33.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.517+0000 7fb6ea3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e415c540 0x7fb6e415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:33.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.517+0000 7fb6ea3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e40c97f0 con 0x7fb6e415c540 2023-07-06T03:48:33.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.517+0000 7fb6ea3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e40c6fd0 con 0x7fb6e40d14a0 2023-07-06T03:48:33.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.517+0000 7fb6ea3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e40c6a50 con 0x7fb6e40b4870 2023-07-06T03:48:33.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.518+0000 7fb6e899c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e415c540 0x7fb6e415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:33.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.518+0000 7fb6e3fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e40b4870 0x7fb6e40c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:33.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.518+0000 7fb6dbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6e40d14a0 0x7fb6e414d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:33.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.518+0000 7fb6e899c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e415c540 0x7fb6e415ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40046/0 (socket says 172.21.15.97:40046) 2023-07-06T03:48:33.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.518+0000 7fb6e899c700 1 -- 172.21.15.97:0/2345765490 learned_addr learned my addr 172.21.15.97:0/2345765490 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:33.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.518+0000 7fb6e899c700 1 -- 172.21.15.97:0/2345765490 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e40b4870 msgr2=0x7fb6e40c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:33.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.518+0000 7fb6e899c700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e40b4870 0x7fb6e40c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:33.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.518+0000 7fb6e899c700 1 -- 172.21.15.97:0/2345765490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6e40d14a0 msgr2=0x7fb6e414d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:33.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.518+0000 7fb6e899c700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6e40d14a0 0x7fb6e414d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:33.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.518+0000 7fb6e899c700 1 -- 172.21.15.97:0/2345765490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc0094a0 con 0x7fb6e415c540 2023-07-06T03:48:33.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.519+0000 7fb6e899c700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e415c540 0x7fb6e415ab80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc002230 tx=0x7fb6dc00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:33.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.519+0000 7fb6e17fa700 1 -- 172.21.15.97:0/2345765490 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00aa60 con 0x7fb6e415c540 2023-07-06T03:48:33.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.519+0000 7fb6e17fa700 1 -- 172.21.15.97:0/2345765490 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc00abe0 con 0x7fb6e415c540 2023-07-06T03:48:33.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.519+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2345765490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e414f590 con 0x7fb6e415c540 2023-07-06T03:48:33.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.519+0000 7fb6e17fa700 1 -- 172.21.15.97:0/2345765490 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0070d0 con 0x7fb6e415c540 2023-07-06T03:48:33.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.519+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2345765490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e40b5dc0 con 0x7fb6e415c540 2023-07-06T03:48:33.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.520+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2345765490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e40726a0 con 0x7fb6e415c540 2023-07-06T03:48:33.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.521+0000 7fb6e17fa700 1 -- 172.21.15.97:0/2345765490 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb6dc007230 con 0x7fb6e415c540 2023-07-06T03:48:33.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.522+0000 7fb6e17fa700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb6c406ace0 0x7fb6c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:33.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.522+0000 7fb6e17fa700 1 -- 172.21.15.97:0/2345765490 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6272+0+0 (secure 0 0 0) 0x7fb6dc088f90 con 0x7fb6e415c540 2023-07-06T03:48:33.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.522+0000 7fb6e3fff700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb6c406ace0 0x7fb6c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:33.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.523+0000 7fb6e3fff700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb6c406ace0 0x7fb6c406d190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4007fd0 tx=0x7fb6d4007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:33.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.526+0000 7fb6e17fa700 1 -- 172.21.15.97:0/2345765490 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb6dc053ef0 con 0x7fb6e415c540 2023-07-06T03:48:33.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:33.721+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2345765490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb6e4074390 con 0x7fb6e415c540 2023-07-06T03:48:33.841 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:48:33.846 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:48:34.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.663+0000 7fb6e17fa700 1 -- 172.21.15.97:0/2345765490 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb6dc0180d0 con 0x7fb6e415c540 2023-07-06T03:48:34.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.667+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2345765490 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb6c406ace0 msgr2=0x7fb6c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:34.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.667+0000 7fb6ea3ff700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb6c406ace0 0x7fb6c406d190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4007fd0 tx=0x7fb6d4007ec0 comp rx=0 tx=0).stop 2023-07-06T03:48:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.667+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2345765490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e415c540 msgr2=0x7fb6e415ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.667+0000 7fb6ea3ff700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e415c540 0x7fb6e415ab80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc002230 tx=0x7fb6dc00cf50 comp rx=0 tx=0).stop 2023-07-06T03:48:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.667+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2345765490 shutdown_connections 2023-07-06T03:48:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.667+0000 7fb6ea3ff700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb6c406ace0 0x7fb6c406d190 secure :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4007fd0 tx=0x7fb6d4007ec0 comp rx=0 tx=0).stop 2023-07-06T03:48:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.667+0000 7fb6ea3ff700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6e415c540 0x7fb6e415ab80 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.667+0000 7fb6ea3ff700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6e40b4870 0x7fb6e40c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.667+0000 7fb6ea3ff700 1 --2- 172.21.15.97:0/2345765490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6e40d14a0 0x7fb6e414d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.667+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2345765490 >> 172.21.15.97:0/2345765490 conn(0x7fb6e406b2f0 msgr2=0x7fb6e41e6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.668+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2345765490 shutdown_connections 2023-07-06T03:48:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.668+0000 7fb6ea3ff700 1 -- 172.21.15.97:0/2345765490 wait complete. 2023-07-06T03:48:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-07-06T03:48:34.690 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T03:48:34.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.888+0000 7fb6439ff700 1 Processor -- start 2023-07-06T03:48:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.889+0000 7fb6439ff700 1 -- start start 2023-07-06T03:48:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.890+0000 7fb6439ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb63c0ce4c0 0x7fb63c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.890+0000 7fb6439ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c0c9a60 con 0x7fb63c0c38a0 2023-07-06T03:48:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.890+0000 7fb6439ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c0cc5d0 con 0x7fb63c0cb350 2023-07-06T03:48:34.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.890+0000 7fb6439ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c0cce70 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.890+0000 7fb640f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb63c0ce4c0 0x7fb63c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:34.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.890+0000 7fb640f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb63c0ce4c0 0x7fb63c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40054/0 (socket says 172.21.15.97:40054) 2023-07-06T03:48:34.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.890+0000 7fb641f9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb63c0c38a0 0x7fb63c0ca1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45242/0 (socket says 172.21.15.97:45242) 2023-07-06T03:48:34.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.890+0000 7fb640f9a700 1 -- 172.21.15.97:0/1704090737 learned_addr learned my addr 172.21.15.97:0/1704090737 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:34.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.891+0000 7fb640f9a700 1 -- 172.21.15.97:0/1704090737 >> v1:172.21.15.146:6789/0 conn(0x7fb63c0cb350 legacy=0x7fb63c0cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.891+0000 7fb640f9a700 1 -- 172.21.15.97:0/1704090737 >> v1:172.21.15.97:6789/0 conn(0x7fb63c0c38a0 legacy=0x7fb63c0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.891+0000 7fb640f9a700 1 -- 172.21.15.97:0/1704090737 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0cc8b0 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.891+0000 7fb633fff700 1 -- 172.21.15.97:0/1704090737 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497659862 0 0) 0x7fb63c0c9a60 con 0x7fb63c0c38a0 2023-07-06T03:48:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.891+0000 7fb640f9a700 1 --2- 172.21.15.97:0/1704090737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb63c0ce4c0 0x7fb63c0cf290 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00bbc0 tx=0x7fb62c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd968032113a4d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.891+0000 7fb633fff700 1 -- 172.21.15.97:0/1704090737 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c00dd10 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.891+0000 7fb633fff700 1 -- 172.21.15.97:0/1704090737 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c00de70 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.891+0000 7fb633fff700 1 -- 172.21.15.97:0/1704090737 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c004230 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.892+0000 7fb6439ff700 1 -- 172.21.15.97:0/1704090737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb63c0ce4c0 msgr2=0x7fb63c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.892+0000 7fb6439ff700 1 --2- 172.21.15.97:0/1704090737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb63c0ce4c0 0x7fb63c0cf290 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00bbc0 tx=0x7fb62c00bed0 comp rx=0 tx=0).stop 2023-07-06T03:48:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.892+0000 7fb6439ff700 1 -- 172.21.15.97:0/1704090737 shutdown_connections 2023-07-06T03:48:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.892+0000 7fb6439ff700 1 --2- 172.21.15.97:0/1704090737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb63c0ce4c0 0x7fb63c0cf290 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.892+0000 7fb6439ff700 1 -- 172.21.15.97:0/1704090737 >> 172.21.15.97:0/1704090737 conn(0x7fb63c069fe0 msgr2=0x7fb63c06a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.892+0000 7fb6439ff700 1 -- 172.21.15.97:0/1704090737 shutdown_connections 2023-07-06T03:48:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.892+0000 7fb6439ff700 1 -- 172.21.15.97:0/1704090737 wait complete. 2023-07-06T03:48:34.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.894+0000 7fb6439ff700 1 Processor -- start 2023-07-06T03:48:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.894+0000 7fb6439ff700 1 -- start start 2023-07-06T03:48:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.895+0000 7fb6439ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb63c0c38a0 0x7fb63c158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.895+0000 7fb6439ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb63c0cb350 0x7fb63c0acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb6439ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb63c0ce4c0 0x7fb63c0c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb6439ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0ae680 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb6439ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c1589a0 con 0x7fb63c0cb350 2023-07-06T03:48:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb6439ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0afde0 con 0x7fb63c0c38a0 2023-07-06T03:48:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb641f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb63c0ce4c0 0x7fb63c0c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb641f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb63c0ce4c0 0x7fb63c0c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40068/0 (socket says 172.21.15.97:40068) 2023-07-06T03:48:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb641f9c700 1 -- 172.21.15.97:0/3869727149 learned_addr learned my addr 172.21.15.97:0/3869727149 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb640f9a700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb63c0cb350 0x7fb63c0acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb64179b700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb63c0c38a0 0x7fb63c158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb641f9c700 1 -- 172.21.15.97:0/3869727149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb63c0c38a0 msgr2=0x7fb63c158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:34.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb641f9c700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb63c0c38a0 0x7fb63c158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb641f9c700 1 -- 172.21.15.97:0/3869727149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb63c0cb350 msgr2=0x7fb63c0acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb641f9c700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb63c0cb350 0x7fb63c0acc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb641f9c700 1 -- 172.21.15.97:0/3869727149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c00b7f0 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb64179b700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb63c0c38a0 0x7fb63c158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.896+0000 7fb641f9c700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb63c0ce4c0 0x7fb63c0c7ca0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb63400ccd0 tx=0x7fb63400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.897+0000 7fb631ffb700 1 -- 172.21.15.97:0/3869727149 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63400ada0 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.897+0000 7fb631ffb700 1 -- 172.21.15.97:0/3869727149 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6340070d0 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.897+0000 7fb6439ff700 1 -- 172.21.15.97:0/3869727149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c0b00c0 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.897+0000 7fb631ffb700 1 -- 172.21.15.97:0/3869727149 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634007480 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.897+0000 7fb6439ff700 1 -- 172.21.15.97:0/3869727149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c0b0680 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.898+0000 7fb6439ff700 1 -- 172.21.15.97:0/3869727149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c0ae830 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.899+0000 7fb631ffb700 1 -- 172.21.15.97:0/3869727149 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb6340075e0 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.900+0000 7fb631ffb700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb62006ace0 0x7fb62006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.900+0000 7fb631ffb700 1 -- 172.21.15.97:0/3869727149 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5834+0+0 (secure 0 0 0) 0x7fb634089120 con 0x7fb63c0ce4c0 2023-07-06T03:48:34.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.900+0000 7fb64179b700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb62006ace0 0x7fb62006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.900+0000 7fb64179b700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb62006ace0 0x7fb62006d190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb638003f40 tx=0x7fb638009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:34.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:34.904+0000 7fb631ffb700 1 -- 172.21.15.97:0/3869727149 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb63c0ae830 con 0x7fb63c0ce4c0 2023-07-06T03:48:35.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.108+0000 7fb6439ff700 1 -- 172.21.15.97:0/3869727149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb63c0ae830 con 0x7fb63c0ce4c0 2023-07-06T03:48:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.860+0000 7fb631ffb700 1 -- 172.21.15.97:0/3869727149 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb63c0ae830 con 0x7fb63c0ce4c0 2023-07-06T03:48:35.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.863+0000 7fb6439ff700 1 -- 172.21.15.97:0/3869727149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb62006ace0 msgr2=0x7fb62006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:35.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.863+0000 7fb6439ff700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb62006ace0 0x7fb62006d190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb638003f40 tx=0x7fb638009040 comp rx=0 tx=0).stop 2023-07-06T03:48:35.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.864+0000 7fb6439ff700 1 -- 172.21.15.97:0/3869727149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb63c0ce4c0 msgr2=0x7fb63c0c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:35.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.864+0000 7fb6439ff700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb63c0ce4c0 0x7fb63c0c7ca0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb63400ccd0 tx=0x7fb63400a340 comp rx=0 tx=0).stop 2023-07-06T03:48:35.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.864+0000 7fb6439ff700 1 -- 172.21.15.97:0/3869727149 shutdown_connections 2023-07-06T03:48:35.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.864+0000 7fb6439ff700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb62006ace0 0x7fb62006d190 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:35.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.864+0000 7fb6439ff700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb63c0ce4c0 0x7fb63c0c7ca0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:35.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.864+0000 7fb6439ff700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb63c0c38a0 0x7fb63c158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:35.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.864+0000 7fb6439ff700 1 --2- 172.21.15.97:0/3869727149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb63c0cb350 0x7fb63c0acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:35.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.864+0000 7fb6439ff700 1 -- 172.21.15.97:0/3869727149 >> 172.21.15.97:0/3869727149 conn(0x7fb63c069fe0 msgr2=0x7fb63c141320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:35.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.864+0000 7fb6439ff700 1 -- 172.21.15.97:0/3869727149 shutdown_connections 2023-07-06T03:48:35.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:35.864+0000 7fb6439ff700 1 -- 172.21.15.97:0/3869727149 wait complete. 2023-07-06T03:48:35.867 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-07-06T03:48:35.886 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:48:35.890 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:48:36.004 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2023-07-06T03:48:36.005 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2023-07-06T03:48:36.005 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:48:36.005 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2023-07-06T03:48:36.005 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.005 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2023-07-06T03:48:36.005 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.006 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2023-07-06T03:48:36.006 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.006 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2023-07-06T03:48:36.006 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2023-07-06T03:48:36.006 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.006 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2023-07-06T03:48:36.006 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2023-07-06T03:48:36.006 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.007 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:48:36.007 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2023-07-06T03:48:36.007 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.007 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:48:36.007 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:48:36.007 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.007 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2023-07-06T03:48:36.007 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.008 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2023-07-06T03:48:36.008 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2023-07-06T03:48:36.008 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:48:36.008 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:48:36.008 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2023-07-06T03:48:36.008 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:48:36.008 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2023-07-06T03:48:36.008 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:48:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:48:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2023-07-06T03:48:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:48:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2023-07-06T03:48:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2023-07-06T03:48:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2023-07-06T03:48:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2023-07-06T03:48:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2023-07-06T03:48:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:48:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:48:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.011 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2023-07-06T03:48:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2023-07-06T03:48:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2023-07-06T03:48:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2023-07-06T03:48:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:48:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2023-07-06T03:48:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2023-07-06T03:48:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2023-07-06T03:48:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2023-07-06T03:48:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2023-07-06T03:48:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2023-07-06T03:48:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2023-07-06T03:48:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:48:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2023-07-06T03:48:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2023-07-06T03:48:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:48:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2023-07-06T03:48:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2023-07-06T03:48:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2023-07-06T03:48:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2023-07-06T03:48:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2023-07-06T03:48:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2023-07-06T03:48:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2023-07-06T03:48:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2023-07-06T03:48:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2023-07-06T03:48:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2023-07-06T03:48:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:48:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2023-07-06T03:48:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2023-07-06T03:48:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:48:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2023-07-06T03:48:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:48:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:48:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2023-07-06T03:48:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:48:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.016 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2023-07-06T03:48:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:48:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2023-07-06T03:48:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:48:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:48:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:48:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:48:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2023-07-06T03:48:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.018 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2023-07-06T03:48:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:48:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:48:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2023-07-06T03:48:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:48:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2023-07-06T03:48:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2023-07-06T03:48:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:48:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2023-07-06T03:48:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2023-07-06T03:48:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:48:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:48:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:48:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:48:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:48:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:48:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:48:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2023-07-06T03:48:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:48:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:48:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2023-07-06T03:48:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:36.021 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2023-07-06T03:48:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2023-07-06T03:48:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2023-07-06T03:48:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2023-07-06T03:48:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:48:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:48:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2023-07-06T03:48:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2023-07-06T03:48:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:48:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2023-07-06T03:48:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:48:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:48:36.022 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2023-07-06T03:48:36.023 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2023-07-06T03:48:36.023 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2023-07-06T03:48:36.023 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-07-06T03:48:36.023 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-07-06T03:48:36.023 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-07-06T03:48:36.023 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-07-06T03:48:36.023 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2023-07-06T03:48:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2023-07-06T03:48:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2023-07-06T03:48:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:48:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2023-07-06T03:48:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2023-07-06T03:48:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2023-07-06T03:48:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2023-07-06T03:48:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:48:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:48:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:48:36.025 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2023-07-06T03:48:36.025 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2023-07-06T03:48:36.025 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:48:36.026 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2023-07-06T03:48:36.026 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.026 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2023-07-06T03:48:36.026 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.026 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2023-07-06T03:48:36.026 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.026 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2023-07-06T03:48:36.027 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2023-07-06T03:48:36.027 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.027 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2023-07-06T03:48:36.027 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2023-07-06T03:48:36.027 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.027 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:48:36.027 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2023-07-06T03:48:36.027 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.028 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:48:36.028 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:48:36.028 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.028 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2023-07-06T03:48:36.028 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.028 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2023-07-06T03:48:36.028 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2023-07-06T03:48:36.028 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:48:36.029 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:48:36.029 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2023-07-06T03:48:36.029 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:48:36.029 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2023-07-06T03:48:36.029 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:48:36.029 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:48:36.029 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2023-07-06T03:48:36.029 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.030 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:48:36.030 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.030 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2023-07-06T03:48:36.030 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.030 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2023-07-06T03:48:36.030 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.030 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2023-07-06T03:48:36.030 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2023-07-06T03:48:36.031 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2023-07-06T03:48:36.031 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:48:36.031 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:48:36.031 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.031 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2023-07-06T03:48:36.031 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2023-07-06T03:48:36.031 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.031 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2023-07-06T03:48:36.032 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2023-07-06T03:48:36.032 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:48:36.032 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.032 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2023-07-06T03:48:36.032 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2023-07-06T03:48:36.032 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.032 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2023-07-06T03:48:36.032 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2023-07-06T03:48:36.033 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2023-07-06T03:48:36.033 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.033 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2023-07-06T03:48:36.033 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2023-07-06T03:48:36.033 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.033 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:48:36.033 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.033 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2023-07-06T03:48:36.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.034 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2023-07-06T03:48:36.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.034 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:48:36.034 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2023-07-06T03:48:36.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.034 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2023-07-06T03:48:36.034 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2023-07-06T03:48:36.035 INFO:teuthology.orchestra.run.smithi146.stdout: bdev async discard = True 2023-07-06T03:48:36.035 INFO:teuthology.orchestra.run.smithi146.stdout: bdev enable discard = True 2023-07-06T03:48:36.035 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore allocator = bitmap 2023-07-06T03:48:36.035 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2023-07-06T03:48:36.035 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2023-07-06T03:48:36.035 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 1/20 2023-07-06T03:48:36.035 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 1/20 2023-07-06T03:48:36.035 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:48:36.036 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2023-07-06T03:48:36.036 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 4/10 2023-07-06T03:48:36.036 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:48:36.036 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2023-07-06T03:48:36.036 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:48:36.036 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:48:36.036 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2023-07-06T03:48:36.036 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:48:36.037 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.037 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2023-07-06T03:48:36.037 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:48:36.037 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2023-07-06T03:48:36.037 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:48:36.037 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:48:36.037 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:48:36.037 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:48:36.038 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2023-07-06T03:48:36.038 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.038 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2023-07-06T03:48:36.038 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:48:36.038 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:48:36.038 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2023-07-06T03:48:36.038 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:48:36.038 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2023-07-06T03:48:36.039 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2023-07-06T03:48:36.039 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:48:36.039 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2023-07-06T03:48:36.039 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2023-07-06T03:48:36.039 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:48:36.039 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.039 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:48:36.039 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:48:36.040 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:48:36.040 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.040 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:48:36.040 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:48:36.040 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:48:36.040 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.040 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2023-07-06T03:48:36.041 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:48:36.041 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.041 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:48:36.041 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2023-07-06T03:48:36.041 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:48:36.041 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2023-07-06T03:48:36.041 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2023-07-06T03:48:36.041 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2023-07-06T03:48:36.042 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2023-07-06T03:48:36.042 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:48:36.042 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:48:36.042 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2023-07-06T03:48:36.042 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2023-07-06T03:48:36.042 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:48:36.042 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2023-07-06T03:48:36.042 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:48:36.043 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:48:36.043 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2023-07-06T03:48:36.043 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2023-07-06T03:48:36.043 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2023-07-06T03:48:36.043 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2023-07-06T03:48:36.043 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2023-07-06T03:48:36.043 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2023-07-06T03:48:36.043 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2023-07-06T03:48:36.044 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2023-07-06T03:48:36.044 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2023-07-06T03:48:36.044 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2023-07-06T03:48:36.044 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:48:36.044 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2023-07-06T03:48:36.044 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2023-07-06T03:48:36.044 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2023-07-06T03:48:36.044 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2023-07-06T03:48:36.045 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:48:36.045 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:48:36.045 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:48:36.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T03:48:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.260+0000 7f5e323ff700 1 Processor -- start 2023-07-06T03:48:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.260+0000 7f5e323ff700 1 -- start start 2023-07-06T03:48:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.261+0000 7f5e323ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e2c15c610 0x7f5e2c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.261+0000 7f5e323ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06b360 0x7f5e2c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.261+0000 7f5e323ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c0cff90 con 0x7f5e2c0d2210 2023-07-06T03:48:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.261+0000 7f5e323ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2c0d0780 con 0x7f5e2c15c610 2023-07-06T03:48:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.261+0000 7f5e323ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c15c9e0 con 0x7f5e2c06b360 2023-07-06T03:48:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.262+0000 7f5e2bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e2c15c610 0x7f5e2c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.262+0000 7f5e2bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e2c15c610 0x7f5e2c0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40082/0 (socket says 172.21.15.97:40082) 2023-07-06T03:48:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.262+0000 7f5e2bfff700 1 -- 172.21.15.97:0/1904371507 learned_addr learned my addr 172.21.15.97:0/1904371507 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.262+0000 7f5e23fff700 1 --2- 172.21.15.97:0/1904371507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06b360 0x7f5e2c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.262+0000 7f5e2bfff700 1 -- 172.21.15.97:0/1904371507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06b360 msgr2=0x7f5e2c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.262+0000 7f5e2bfff700 1 --2- 172.21.15.97:0/1904371507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06b360 0x7f5e2c0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.262+0000 7f5e2bfff700 1 -- 172.21.15.97:0/1904371507 >> v1:172.21.15.97:6789/0 conn(0x7f5e2c0d2210 legacy=0x7f5e2c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.262+0000 7f5e2bfff700 1 -- 172.21.15.97:0/1904371507 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0b5650 con 0x7f5e2c15c610 2023-07-06T03:48:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.262+0000 7f5e2bfff700 1 --2- 172.21.15.97:0/1904371507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c15c610 0x7f5e2c0b47b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009920 tx=0x7f5e1c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e162fe654ecef185 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.263+0000 7f5e2b7fe700 1 -- 172.21.15.97:0/1904371507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00ba20 con 0x7f5e2c15c610 2023-07-06T03:48:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.263+0000 7f5e2b7fe700 1 -- 172.21.15.97:0/1904371507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c00bb80 con 0x7f5e2c15c610 2023-07-06T03:48:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.263+0000 7f5e2b7fe700 1 -- 172.21.15.97:0/1904371507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c009e70 con 0x7f5e2c15c610 2023-07-06T03:48:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.263+0000 7f5e323ff700 1 -- 172.21.15.97:0/1904371507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c15c610 msgr2=0x7f5e2c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.263+0000 7f5e323ff700 1 --2- 172.21.15.97:0/1904371507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c15c610 0x7f5e2c0b47b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009920 tx=0x7f5e1c009c30 comp rx=0 tx=0).stop 2023-07-06T03:48:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.263+0000 7f5e323ff700 1 -- 172.21.15.97:0/1904371507 shutdown_connections 2023-07-06T03:48:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.263+0000 7f5e323ff700 1 --2- 172.21.15.97:0/1904371507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c15c610 0x7f5e2c0b47b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.263+0000 7f5e323ff700 1 --2- 172.21.15.97:0/1904371507 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06b360 0x7f5e2c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.263+0000 7f5e323ff700 1 -- 172.21.15.97:0/1904371507 >> 172.21.15.97:0/1904371507 conn(0x7f5e2c06a6c0 msgr2=0x7f5e2c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.263+0000 7f5e323ff700 1 -- 172.21.15.97:0/1904371507 shutdown_connections 2023-07-06T03:48:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.263+0000 7f5e323ff700 1 -- 172.21.15.97:0/1904371507 wait complete. 2023-07-06T03:48:36.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.265+0000 7f5e323ff700 1 Processor -- start 2023-07-06T03:48:36.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.265+0000 7f5e323ff700 1 -- start start 2023-07-06T03:48:36.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.266+0000 7f5e323ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06b360 0x7f5e2c0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.266+0000 7f5e323ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c0d2210 0x7f5e2c0c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.266+0000 7f5e323ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c15c610 0x7f5e2c14f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.266+0000 7f5e323ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0ccc70 con 0x7f5e2c0d2210 2023-07-06T03:48:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.266+0000 7f5e323ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0cb690 con 0x7f5e2c15c610 2023-07-06T03:48:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.266+0000 7f5e323ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0cb970 con 0x7f5e2c06b360 2023-07-06T03:48:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.266+0000 7f5e23fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c0d2210 0x7f5e2c0c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.266+0000 7f5e23fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c0d2210 0x7f5e2c0c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40090/0 (socket says 172.21.15.97:40090) 2023-07-06T03:48:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.266+0000 7f5e23fff700 1 -- 172.21.15.97:0/1223470308 learned_addr learned my addr 172.21.15.97:0/1223470308 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.266+0000 7f5e2bfff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06b360 0x7f5e2c0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.266+0000 7f5e3099c700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c15c610 0x7f5e2c14f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.267+0000 7f5e23fff700 1 -- 172.21.15.97:0/1223470308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06b360 msgr2=0x7f5e2c0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.267+0000 7f5e23fff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06b360 0x7f5e2c0cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.267+0000 7f5e23fff700 1 -- 172.21.15.97:0/1223470308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c15c610 msgr2=0x7f5e2c14f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.267+0000 7f5e23fff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c15c610 0x7f5e2c14f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.267+0000 7f5e23fff700 1 -- 172.21.15.97:0/1223470308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c009550 con 0x7f5e2c0d2210 2023-07-06T03:48:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.267+0000 7f5e23fff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c0d2210 0x7f5e2c0c7a40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400e810 tx=0x7f5e1400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:36.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.267+0000 7f5e297fa700 1 -- 172.21.15.97:0/1223470308 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400c070 con 0x7f5e2c0d2210 2023-07-06T03:48:36.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.267+0000 7f5e297fa700 1 -- 172.21.15.97:0/1223470308 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e14013610 con 0x7f5e2c0d2210 2023-07-06T03:48:36.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.267+0000 7f5e297fa700 1 -- 172.21.15.97:0/1223470308 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e140139a0 con 0x7f5e2c0d2210 2023-07-06T03:48:36.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.267+0000 7f5e323ff700 1 -- 172.21.15.97:0/1223470308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c0cbc50 con 0x7f5e2c0d2210 2023-07-06T03:48:36.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.267+0000 7f5e323ff700 1 -- 172.21.15.97:0/1223470308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c0cc210 con 0x7f5e2c0d2210 2023-07-06T03:48:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.268+0000 7f5e323ff700 1 -- 172.21.15.97:0/1223470308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c04e340 con 0x7f5e2c0d2210 2023-07-06T03:48:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.269+0000 7f5e297fa700 1 -- 172.21.15.97:0/1223470308 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5e14013b00 con 0x7f5e2c0d2210 2023-07-06T03:48:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.270+0000 7f5e297fa700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e0c06ace0 0x7f5e0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.270+0000 7f5e297fa700 1 -- 172.21.15.97:0/1223470308 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5e14088c60 con 0x7f5e2c0d2210 2023-07-06T03:48:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.270+0000 7f5e2bfff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e0c06ace0 0x7f5e0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:36.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.271+0000 7f5e2bfff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e0c06ace0 0x7f5e0c06d190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c000c00 tx=0x7f5e1c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.274+0000 7f5e297fa700 1 -- 172.21.15.97:0/1223470308 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5e14057a20 con 0x7f5e2c0d2210 2023-07-06T03:48:36.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.471+0000 7f5e323ff700 1 -- 172.21.15.97:0/1223470308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5e2c15a400 con 0x7f5e2c0d2210 2023-07-06T03:48:36.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.475+0000 7f5e297fa700 1 -- 172.21.15.97:0/1223470308 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5e14013db0 con 0x7f5e2c0d2210 2023-07-06T03:48:36.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 -- 172.21.15.97:0/1223470308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e0c06ace0 msgr2=0x7f5e0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:36.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e0c06ace0 0x7f5e0c06d190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c000c00 tx=0x7f5e1c008030 comp rx=0 tx=0).stop 2023-07-06T03:48:36.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 -- 172.21.15.97:0/1223470308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c0d2210 msgr2=0x7f5e2c0c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:36.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c0d2210 0x7f5e2c0c7a40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400e810 tx=0x7f5e1400eb20 comp rx=0 tx=0).stop 2023-07-06T03:48:36.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 -- 172.21.15.97:0/1223470308 shutdown_connections 2023-07-06T03:48:36.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e0c06ace0 0x7f5e0c06d190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e2c15c610 0x7f5e2c14f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e2c06b360 0x7f5e2c0cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 --2- 172.21.15.97:0/1223470308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c0d2210 0x7f5e2c0c7a40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 -- 172.21.15.97:0/1223470308 >> 172.21.15.97:0/1223470308 conn(0x7f5e2c06a6c0 msgr2=0x7f5e2c0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:36.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 -- 172.21.15.97:0/1223470308 shutdown_connections 2023-07-06T03:48:36.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.478+0000 7f5e323ff700 1 -- 172.21.15.97:0/1223470308 wait complete. 2023-07-06T03:48:36.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2023-07-06T03:48:36.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.690+0000 7fdf8ffff700 1 Processor -- start 2023-07-06T03:48:36.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.690+0000 7fdf8ffff700 1 -- start start 2023-07-06T03:48:36.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.691+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8815ff00 0x7fdf880c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:36.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.691+0000 7fdf8ffff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf880b4740 con 0x7fdf880d10c0 2023-07-06T03:48:36.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.691+0000 7fdf8ffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf881602d0 con 0x7fdf880c8d70 2023-07-06T03:48:36.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.691+0000 7fdf8ffff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880b60b0 con 0x7fdf8815ff00 2023-07-06T03:48:36.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.692+0000 7fdf8e59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdf880d10c0 0x7fdf880b6540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45256/0 (socket says 172.21.15.97:45256) 2023-07-06T03:48:36.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.692+0000 7fdf8e59c700 1 -- 172.21.15.97:0/4289194762 learned_addr learned my addr 172.21.15.97:0/4289194762 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:36.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.692+0000 7fdf8dd9b700 1 --2- 172.21.15.97:0/4289194762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8815ff00 0x7fdf880c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:36.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.692+0000 7fdf8cd99700 1 -- 172.21.15.97:0/4289194762 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473511279 0 0) 0x7fdf880b4740 con 0x7fdf880d10c0 2023-07-06T03:48:36.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.692+0000 7fdf8cd99700 1 -- 172.21.15.97:0/4289194762 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf740035b0 con 0x7fdf880d10c0 2023-07-06T03:48:36.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.692+0000 7fdf8cd99700 1 -- 172.21.15.97:0/4289194762 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223867934 0 0) 0x7fdf78001110 con 0x7fdf880c8d70 2023-07-06T03:48:36.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.692+0000 7fdf8cd99700 1 -- 172.21.15.97:0/4289194762 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf880b4740 con 0x7fdf880c8d70 2023-07-06T03:48:36.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.692+0000 7fdf8dd9b700 1 -- 172.21.15.97:0/4289194762 >> v1:172.21.15.146:6789/0 conn(0x7fdf880c8d70 legacy=0x7fdf880d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:36.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.692+0000 7fdf8dd9b700 1 -- 172.21.15.97:0/4289194762 >> v1:172.21.15.97:6789/0 conn(0x7fdf880d10c0 legacy=0x7fdf880b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:36.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.692+0000 7fdf8dd9b700 1 -- 172.21.15.97:0/4289194762 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf880c9800 con 0x7fdf8815ff00 2023-07-06T03:48:36.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.693+0000 7fdf8dd9b700 1 --2- 172.21.15.97:0/4289194762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8815ff00 0x7fdf880c8830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009850 tx=0x7fdf84009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a74f2186db0a3d2f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:36.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.693+0000 7fdf8cd99700 1 -- 172.21.15.97:0/4289194762 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84005e30 con 0x7fdf8815ff00 2023-07-06T03:48:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.693+0000 7fdf8cd99700 1 -- 172.21.15.97:0/4289194762 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf84006030 con 0x7fdf8815ff00 2023-07-06T03:48:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.693+0000 7fdf8cd99700 1 -- 172.21.15.97:0/4289194762 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf840063e0 con 0x7fdf8815ff00 2023-07-06T03:48:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.693+0000 7fdf8ffff700 1 -- 172.21.15.97:0/4289194762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8815ff00 msgr2=0x7fdf880c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.693+0000 7fdf8ffff700 1 --2- 172.21.15.97:0/4289194762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8815ff00 0x7fdf880c8830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009850 tx=0x7fdf84009b60 comp rx=0 tx=0).stop 2023-07-06T03:48:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.694+0000 7fdf8ffff700 1 -- 172.21.15.97:0/4289194762 shutdown_connections 2023-07-06T03:48:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.694+0000 7fdf8ffff700 1 --2- 172.21.15.97:0/4289194762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8815ff00 0x7fdf880c8830 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.694+0000 7fdf8ffff700 1 -- 172.21.15.97:0/4289194762 >> 172.21.15.97:0/4289194762 conn(0x7fdf8809c130 msgr2=0x7fdf880d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.694+0000 7fdf8ffff700 1 -- 172.21.15.97:0/4289194762 shutdown_connections 2023-07-06T03:48:36.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.694+0000 7fdf8ffff700 1 -- 172.21.15.97:0/4289194762 wait complete. 2023-07-06T03:48:36.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.695+0000 7fdf8ffff700 1 Processor -- start 2023-07-06T03:48:36.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.695+0000 7fdf8ffff700 1 -- start start 2023-07-06T03:48:36.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.696+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf880c8d70 0x7fdf880b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:36.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.696+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf880d10c0 0x7fdf88154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:36.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.696+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8815ff00 0x7fdf8815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:36.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.696+0000 7fdf8ffff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf88159d50 con 0x7fdf880c8d70 2023-07-06T03:48:36.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.696+0000 7fdf8ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8809d8b0 con 0x7fdf880d10c0 2023-07-06T03:48:36.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.696+0000 7fdf8ffff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8809de70 con 0x7fdf8815ff00 2023-07-06T03:48:36.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf880c8d70 0x7fdf880b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:36.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf880c8d70 0x7fdf880b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40094/0 (socket says 172.21.15.97:40094) 2023-07-06T03:48:36.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8dd9b700 1 -- 172.21.15.97:0/3884844325 learned_addr learned my addr 172.21.15.97:0/3884844325 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:36.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8d59a700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf880d10c0 0x7fdf88154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:36.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8e59c700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8815ff00 0x7fdf8815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:36.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8dd9b700 1 -- 172.21.15.97:0/3884844325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8815ff00 msgr2=0x7fdf8815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:36.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8dd9b700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8815ff00 0x7fdf8815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8dd9b700 1 -- 172.21.15.97:0/3884844325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf880d10c0 msgr2=0x7fdf88154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:36.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8dd9b700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf880d10c0 0x7fdf88154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:36.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8dd9b700 1 -- 172.21.15.97:0/3884844325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf84009430 con 0x7fdf880c8d70 2023-07-06T03:48:36.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8d59a700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf880d10c0 0x7fdf88154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:36.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf8dd9b700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf880c8d70 0x7fdf880b3400 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009eb0 tx=0x7fdf84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:36.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.697+0000 7fdf7e7fc700 1 -- 172.21.15.97:0/3884844325 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84025430 con 0x7fdf880c8d70 2023-07-06T03:48:36.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.698+0000 7fdf7e7fc700 1 -- 172.21.15.97:0/3884844325 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf84020070 con 0x7fdf880c8d70 2023-07-06T03:48:36.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.698+0000 7fdf7e7fc700 1 -- 172.21.15.97:0/3884844325 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf840257d0 con 0x7fdf880c8d70 2023-07-06T03:48:36.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.698+0000 7fdf8ffff700 1 -- 172.21.15.97:0/3884844325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf880b3940 con 0x7fdf880c8d70 2023-07-06T03:48:36.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.698+0000 7fdf8ffff700 1 -- 172.21.15.97:0/3884844325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8815a280 con 0x7fdf880c8d70 2023-07-06T03:48:36.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.699+0000 7fdf8ffff700 1 -- 172.21.15.97:0/3884844325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8804e340 con 0x7fdf880c8d70 2023-07-06T03:48:36.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.700+0000 7fdf7e7fc700 1 -- 172.21.15.97:0/3884844325 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf84022030 con 0x7fdf880c8d70 2023-07-06T03:48:36.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.701+0000 7fdf7e7fc700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf6c06ace0 0x7fdf6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:36.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.701+0000 7fdf7e7fc700 1 -- 172.21.15.97:0/3884844325 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdf8408a6d0 con 0x7fdf880c8d70 2023-07-06T03:48:36.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.701+0000 7fdf8d59a700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf6c06ace0 0x7fdf6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:36.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.701+0000 7fdf8d59a700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf6c06ace0 0x7fdf6c06d190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdf881f5330 tx=0x7fdf78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:36.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.705+0000 7fdf7e7fc700 1 -- 172.21.15.97:0/3884844325 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf84055a10 con 0x7fdf880c8d70 2023-07-06T03:48:36.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:36.979+0000 7fdf8ffff700 1 -- 172.21.15.97:0/3884844325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) v1 -- 0x7fdf88155160 con 0x7fdf880c8d70 2023-07-06T03:48:37.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.862+0000 7fdf7e7fc700 1 -- 172.21.15.97:0/3884844325 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v203) v1 ==== 150+0+0 (secure 0 0 0) 0x7fdf84059030 con 0x7fdf880c8d70 2023-07-06T03:48:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 -- 172.21.15.97:0/3884844325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf6c06ace0 msgr2=0x7fdf6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf6c06ace0 0x7fdf6c06d190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdf881f5330 tx=0x7fdf78009040 comp rx=0 tx=0).stop 2023-07-06T03:48:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 -- 172.21.15.97:0/3884844325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf880c8d70 msgr2=0x7fdf880b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf880c8d70 0x7fdf880b3400 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009eb0 tx=0x7fdf84004750 comp rx=0 tx=0).stop 2023-07-06T03:48:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 -- 172.21.15.97:0/3884844325 shutdown_connections 2023-07-06T03:48:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf6c06ace0 0x7fdf6c06d190 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf8815ff00 0x7fdf8815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf880c8d70 0x7fdf880b3400 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 --2- 172.21.15.97:0/3884844325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf880d10c0 0x7fdf88154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 -- 172.21.15.97:0/3884844325 >> 172.21.15.97:0/3884844325 conn(0x7fdf8809c130 msgr2=0x7fdf880b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 -- 172.21.15.97:0/3884844325 shutdown_connections 2023-07-06T03:48:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:37.865+0000 7fdf8ffff700 1 -- 172.21.15.97:0/3884844325 wait complete. 2023-07-06T03:48:37.887 INFO:tasks.cephfs_test_runner:test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-06T03:48:37.887 INFO:tasks.cephfs_test_runner:Starting test: test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:48:37.887 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2023-07-06T03:48:38.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.082+0000 7fe9d0dff700 1 Processor -- start 2023-07-06T03:48:38.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.083+0000 7fe9d0dff700 1 -- start start 2023-07-06T03:48:38.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.084+0000 7fe9d0dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe9cc0d0e20 0x7fe9cc0b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:38.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.084+0000 7fe9d0dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9cc0b4740 con 0x7fe9cc0d09b0 2023-07-06T03:48:38.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.084+0000 7fe9d0dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9cc1602d0 con 0x7fe9cc15ff00 2023-07-06T03:48:38.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.084+0000 7fe9d0dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc0b60b0 con 0x7fe9cc0d0e20 2023-07-06T03:48:38.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.084+0000 7fe9c3fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe9cc0d09b0 0x7fe9cc0b6540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45264/0 (socket says 172.21.15.97:45264) 2023-07-06T03:48:38.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.084+0000 7fe9c3fff700 1 -- 172.21.15.97:0/2795136692 learned_addr learned my addr 172.21.15.97:0/2795136692 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:38.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.084+0000 7fe9cad9d700 1 --2- 172.21.15.97:0/2795136692 >> v2:172.21.15.146:3300/0 conn(0x7fe9cc0d0e20 0x7fe9cc0b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:38.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.084+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854484591 0 0) 0x7fe9cc0b4740 con 0x7fe9cc0d09b0 2023-07-06T03:48:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.084+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9b80035b0 con 0x7fe9cc0d09b0 2023-07-06T03:48:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1510766358 0 0) 0x7fe9b80035b0 con 0x7fe9cc0d09b0 2023-07-06T03:48:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 >> v2:172.21.15.146:3300/0 conn(0x7fe9cc0d0e20 msgr2=0x7fe9cc0b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 --2- 172.21.15.97:0/2795136692 >> v2:172.21.15.146:3300/0 conn(0x7fe9cc0d0e20 0x7fe9cc0b6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 >> v1:172.21.15.146:6789/0 conn(0x7fe9cc15ff00 legacy=0x7fe9cc0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc0c9800 con 0x7fe9cc0d09b0 2023-07-06T03:48:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fe9b4001670 con 0x7fe9cc0d09b0 2023-07-06T03:48:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 >> v1:172.21.15.97:6789/0 conn(0x7fe9cc0d09b0 legacy=0x7fe9cc0b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b8003a00 0x7fe9b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b80043f0 0x7fe9b80048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9b8005150 0x7fe9b800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc0b60b0 con 0x7fe9b8003a00 2023-07-06T03:48:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b800c080 con 0x7fe9b80043f0 2023-07-06T03:48:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b800c240 con 0x7fe9b8005150 2023-07-06T03:48:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9ca59c700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b8003a00 0x7fe9b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9c3fff700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b80043f0 0x7fe9b80048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.085+0000 7fe9cad9d700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9b8005150 0x7fe9b800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.086+0000 7fe9ca59c700 1 -- 172.21.15.97:0/2795136692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9b8005150 msgr2=0x7fe9b800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.086+0000 7fe9ca59c700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9b8005150 0x7fe9b800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.086+0000 7fe9ca59c700 1 -- 172.21.15.97:0/2795136692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b80043f0 msgr2=0x7fe9b80048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.086+0000 7fe9ca59c700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b80043f0 0x7fe9b80048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.086+0000 7fe9ca59c700 1 -- 172.21.15.97:0/2795136692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b80035b0 con 0x7fe9b8003a00 2023-07-06T03:48:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.086+0000 7fe9c3fff700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b80043f0 0x7fe9b80048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.086+0000 7fe9cad9d700 1 -- 172.21.15.97:0/2795136692 reap_dead start 2023-07-06T03:48:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.086+0000 7fe9ca59c700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b8003a00 0x7fe9b8003dd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0019d0 tx=0x7fe9b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1534f3c83a12000 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.086+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0052d0 con 0x7fe9b8003a00 2023-07-06T03:48:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.086+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc005430 con 0x7fe9b8003a00 2023-07-06T03:48:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.086+0000 7fe9c9d9b700 1 -- 172.21.15.97:0/2795136692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc003910 con 0x7fe9b8003a00 2023-07-06T03:48:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.087+0000 7fe9d0dff700 1 -- 172.21.15.97:0/2795136692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b8003a00 msgr2=0x7fe9b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.087+0000 7fe9d0dff700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b8003a00 0x7fe9b8003dd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0019d0 tx=0x7fe9b8005880 comp rx=0 tx=0).stop 2023-07-06T03:48:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.087+0000 7fe9d0dff700 1 -- 172.21.15.97:0/2795136692 shutdown_connections 2023-07-06T03:48:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.087+0000 7fe9d0dff700 1 --2- 172.21.15.97:0/2795136692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b8003a00 0x7fe9b8003dd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.087+0000 7fe9d0dff700 1 -- 172.21.15.97:0/2795136692 >> 172.21.15.97:0/2795136692 conn(0x7fe9cc09c130 msgr2=0x7fe9cc0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.087+0000 7fe9d0dff700 1 -- 172.21.15.97:0/2795136692 shutdown_connections 2023-07-06T03:48:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.087+0000 7fe9d0dff700 1 -- 172.21.15.97:0/2795136692 wait complete. 2023-07-06T03:48:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.089+0000 7fe9d0dff700 1 Processor -- start 2023-07-06T03:48:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.089+0000 7fe9d0dff700 1 -- start start 2023-07-06T03:48:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9d0dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b8003a00 0x7fe9cc15c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9d0dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9cc0b1580 0x7fe9cc14d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9d0dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc14db50 0x7fe9cc0b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9d0dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc0b19f0 con 0x7fe9cc14db50 2023-07-06T03:48:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9d0dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc0af9b0 con 0x7fe9b8003a00 2023-07-06T03:48:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9d0dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc150a50 con 0x7fe9cc0b1580 2023-07-06T03:48:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9cad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc14db50 0x7fe9cc0b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9cad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc14db50 0x7fe9cc0b6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40120/0 (socket says 172.21.15.97:40120) 2023-07-06T03:48:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9cad9d700 1 -- 172.21.15.97:0/915173379 learned_addr learned my addr 172.21.15.97:0/915173379 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9c3fff700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9cc0b1580 0x7fe9cc14d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9ca59c700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b8003a00 0x7fe9cc15c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9cad9d700 1 -- 172.21.15.97:0/915173379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9cc0b1580 msgr2=0x7fe9cc14d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9cad9d700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9cc0b1580 0x7fe9cc14d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9cad9d700 1 -- 172.21.15.97:0/915173379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b8003a00 msgr2=0x7fe9cc15c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9cad9d700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b8003a00 0x7fe9cc15c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.090+0000 7fe9cad9d700 1 -- 172.21.15.97:0/915173379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc0b1c10 con 0x7fe9cc14db50 2023-07-06T03:48:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.091+0000 7fe9cad9d700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc14db50 0x7fe9cc0b6540 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc08c7a0 tx=0x7fe9c40114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.091+0000 7fe9c2ffd700 1 -- 172.21.15.97:0/915173379 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c400e070 con 0x7fe9cc14db50 2023-07-06T03:48:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.091+0000 7fe9c2ffd700 1 -- 172.21.15.97:0/915173379 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c4003c30 con 0x7fe9cc14db50 2023-07-06T03:48:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.091+0000 7fe9d0dff700 1 -- 172.21.15.97:0/915173379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc0b1ef0 con 0x7fe9cc14db50 2023-07-06T03:48:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.091+0000 7fe9c2ffd700 1 -- 172.21.15.97:0/915173379 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4003e20 con 0x7fe9cc14db50 2023-07-06T03:48:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.091+0000 7fe9d0dff700 1 -- 172.21.15.97:0/915173379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc0afee0 con 0x7fe9cc14db50 2023-07-06T03:48:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.092+0000 7fe9d0dff700 1 -- 172.21.15.97:0/915173379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc0b0250 con 0x7fe9cc14db50 2023-07-06T03:48:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.093+0000 7fe9c2ffd700 1 -- 172.21.15.97:0/915173379 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe9c4011a10 con 0x7fe9cc14db50 2023-07-06T03:48:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.094+0000 7fe9c2ffd700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9ac06ace0 0x7fe9ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.094+0000 7fe9c2ffd700 1 -- 172.21.15.97:0/915173379 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe9c4089800 con 0x7fe9cc14db50 2023-07-06T03:48:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.094+0000 7fe9ca59c700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9ac06ace0 0x7fe9ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.095+0000 7fe9ca59c700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9ac06ace0 0x7fe9ac06d190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc015040 tx=0x7fe9bc00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.098+0000 7fe9c2ffd700 1 -- 172.21.15.97:0/915173379 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe9c4054b30 con 0x7fe9cc14db50 2023-07-06T03:48:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.369+0000 7fe9d0dff700 1 -- 172.21.15.97:0/915173379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) v1 -- 0x7fe9cc0b20e0 con 0x7fe9cc14db50 2023-07-06T03:48:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.864+0000 7fe9c2ffd700 1 -- 172.21.15.97:0/915173379 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v204) v1 ==== 151+0+0 (secure 0 0 0) 0x7fe9c4058150 con 0x7fe9cc14db50 2023-07-06T03:48:38.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.866+0000 7fe9d0dff700 1 -- 172.21.15.97:0/915173379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9ac06ace0 msgr2=0x7fe9ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:38.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.866+0000 7fe9d0dff700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9ac06ace0 0x7fe9ac06d190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc015040 tx=0x7fe9bc00a000 comp rx=0 tx=0).stop 2023-07-06T03:48:38.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.867+0000 7fe9d0dff700 1 -- 172.21.15.97:0/915173379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc14db50 msgr2=0x7fe9cc0b6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:38.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.867+0000 7fe9d0dff700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc14db50 0x7fe9cc0b6540 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc08c7a0 tx=0x7fe9c40114b0 comp rx=0 tx=0).stop 2023-07-06T03:48:38.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.867+0000 7fe9d0dff700 1 -- 172.21.15.97:0/915173379 shutdown_connections 2023-07-06T03:48:38.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.867+0000 7fe9d0dff700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9ac06ace0 0x7fe9ac06d190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.867+0000 7fe9d0dff700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc14db50 0x7fe9cc0b6540 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.867+0000 7fe9d0dff700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b8003a00 0x7fe9cc15c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.867+0000 7fe9d0dff700 1 --2- 172.21.15.97:0/915173379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9cc0b1580 0x7fe9cc14d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.867+0000 7fe9d0dff700 1 -- 172.21.15.97:0/915173379 >> 172.21.15.97:0/915173379 conn(0x7fe9cc09c130 msgr2=0x7fe9cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.867+0000 7fe9d0dff700 1 -- 172.21.15.97:0/915173379 shutdown_connections 2023-07-06T03:48:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:38.867+0000 7fe9d0dff700 1 -- 172.21.15.97:0/915173379 wait complete. 2023-07-06T03:48:38.887 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T03:48:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.090+0000 7f5a6b9ff700 1 Processor -- start 2023-07-06T03:48:39.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.090+0000 7f5a6b9ff700 1 -- start start 2023-07-06T03:48:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.091+0000 7f5a6b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6406b360 0x7f5a640b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.091+0000 7f5a6b9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5a640d2210 0x7f5a640d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.091+0000 7f5a6b9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a640cff90 con 0x7f5a6415c610 2023-07-06T03:48:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.091+0000 7f5a6b9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640d0780 con 0x7f5a640d2210 2023-07-06T03:48:39.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.091+0000 7f5a6b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6415c9e0 con 0x7f5a6406b360 2023-07-06T03:48:39.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a6979b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5a6415c610 0x7f5a640b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45274/0 (socket says 172.21.15.97:45274) 2023-07-06T03:48:39.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a6979b700 1 -- 172.21.15.97:0/3157874897 learned_addr learned my addr 172.21.15.97:0/3157874897 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:39.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a69f9c700 1 --2- 172.21.15.97:0/3157874897 >> v2:172.21.15.146:3300/0 conn(0x7f5a640d2210 0x7f5a640d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a68f9a700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6406b360 0x7f5a640b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112966792 0 0) 0x7f5a640cff90 con 0x7f5a6415c610 2023-07-06T03:48:39.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a500035b0 con 0x7f5a6415c610 2023-07-06T03:48:39.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 452645058 0 0) 0x7f5a500035b0 con 0x7f5a6415c610 2023-07-06T03:48:39.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6406b360 msgr2=0x7f5a640b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:39.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6406b360 0x7f5a640b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 >> v2:172.21.15.146:3300/0 conn(0x7f5a640d2210 msgr2=0x7f5a640d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:39.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 --2- 172.21.15.97:0/3157874897 >> v2:172.21.15.146:3300/0 conn(0x7f5a640d2210 0x7f5a640d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640b5650 con 0x7f5a6415c610 2023-07-06T03:48:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f5a60001670 con 0x7f5a6415c610 2023-07-06T03:48:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 >> v1:172.21.15.97:6789/0 conn(0x7f5a6415c610 legacy=0x7f5a640b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 0x7f5a50003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a50004850 0x7f5a50004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.092+0000 7f5a5bfff700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6406b360 0x7f5a50005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640d0780 con 0x7f5a50003a30 2023-07-06T03:48:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6415c9e0 con 0x7f5a50004850 2023-07-06T03:48:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5000ba10 con 0x7f5a6406b360 2023-07-06T03:48:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a6979b700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 0x7f5a50003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a68f9a700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a50004850 0x7f5a50004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a69f9c700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6406b360 0x7f5a50005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a6979b700 1 -- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6406b360 msgr2=0x7f5a50005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a6979b700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6406b360 0x7f5a50005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a6979b700 1 -- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a50004850 msgr2=0x7f5a50004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:39.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a6979b700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a50004850 0x7f5a50004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a6979b700 1 -- 172.21.15.97:0/3157874897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a500035b0 con 0x7f5a50003a30 2023-07-06T03:48:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a69f9c700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6406b360 0x7f5a50005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a6979b700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 0x7f5a50003e40 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000c240 tx=0x7f5a50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e56ce2b309a4909 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.093+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60007c10 con 0x7f5a50003a30 2023-07-06T03:48:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.094+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6000cde0 con 0x7f5a50003a30 2023-07-06T03:48:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.094+0000 7f5a5bfff700 1 -- 172.21.15.97:0/3157874897 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60005ba0 con 0x7f5a50003a30 2023-07-06T03:48:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.094+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/3157874897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 msgr2=0x7f5a50003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:39.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.094+0000 7f5a6b9ff700 1 --2- 172.21.15.97:0/3157874897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 0x7f5a50003e40 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000c240 tx=0x7f5a50005880 comp rx=0 tx=0).stop 2023-07-06T03:48:39.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.094+0000 7f5a69f9c700 1 -- 172.21.15.97:0/3157874897 reap_dead start 2023-07-06T03:48:39.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.094+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/3157874897 shutdown_connections 2023-07-06T03:48:39.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.094+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/3157874897 >> 172.21.15.97:0/3157874897 conn(0x7f5a6406a6c0 msgr2=0x7f5a6406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:39.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.094+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/3157874897 shutdown_connections 2023-07-06T03:48:39.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.094+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/3157874897 wait complete. 2023-07-06T03:48:39.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.096+0000 7f5a6b9ff700 1 Processor -- start 2023-07-06T03:48:39.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.096+0000 7f5a6b9ff700 1 -- start start 2023-07-06T03:48:39.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6b9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 0x7f5a641534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6415d180 0x7f5a6415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6b9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a640b31a0 0x7f5a640b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6b9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6415db30 con 0x7f5a50003a30 2023-07-06T03:48:39.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6b9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6415f680 con 0x7f5a640b31a0 2023-07-06T03:48:39.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640b3b50 con 0x7f5a6415d180 2023-07-06T03:48:39.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6979b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 0x7f5a641534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6979b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 0x7f5a641534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40138/0 (socket says 172.21.15.97:40138) 2023-07-06T03:48:39.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6979b700 1 -- 172.21.15.97:0/903699465 learned_addr learned my addr 172.21.15.97:0/903699465 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:39.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a69f9c700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a640b31a0 0x7f5a640b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a68f9a700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6415d180 0x7f5a6415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6979b700 1 -- 172.21.15.97:0/903699465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6415d180 msgr2=0x7f5a6415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:39.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6979b700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6415d180 0x7f5a6415d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6979b700 1 -- 172.21.15.97:0/903699465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a640b31a0 msgr2=0x7f5a640b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:39.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.097+0000 7f5a6979b700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a640b31a0 0x7f5a640b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.098+0000 7f5a6979b700 1 -- 172.21.15.97:0/903699465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64153a30 con 0x7f5a50003a30 2023-07-06T03:48:39.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.098+0000 7f5a6979b700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 0x7f5a641534f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000cf80 tx=0x7f5a60001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:39.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.098+0000 7f5a59ffb700 1 -- 172.21.15.97:0/903699465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60001640 con 0x7f5a50003a30 2023-07-06T03:48:39.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.098+0000 7f5a59ffb700 1 -- 172.21.15.97:0/903699465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a600070d0 con 0x7f5a50003a30 2023-07-06T03:48:39.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.098+0000 7f5a59ffb700 1 -- 172.21.15.97:0/903699465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a600074a0 con 0x7f5a50003a30 2023-07-06T03:48:39.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.098+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/903699465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a640b3d80 con 0x7f5a50003a30 2023-07-06T03:48:39.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.098+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/903699465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a640cab70 con 0x7f5a50003a30 2023-07-06T03:48:39.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.099+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/903699465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6404e340 con 0x7f5a50003a30 2023-07-06T03:48:39.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.100+0000 7f5a59ffb700 1 -- 172.21.15.97:0/903699465 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a60019030 con 0x7f5a50003a30 2023-07-06T03:48:39.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.101+0000 7f5a59ffb700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.101+0000 7f5a59ffb700 1 -- 172.21.15.97:0/903699465 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5a6008a220 con 0x7f5a50003a30 2023-07-06T03:48:39.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.101+0000 7f5a68f9a700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.102+0000 7f5a68f9a700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 0x7f5a4806d190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5a640ca7b0 tx=0x7f5a54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:39.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.106+0000 7f5a59ffb700 1 -- 172.21.15.97:0/903699465 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a60055560 con 0x7f5a50003a30 2023-07-06T03:48:39.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.299+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/903699465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5a640d1f90 con 0x7f5a50003a30 2023-07-06T03:48:39.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.303+0000 7f5a59ffb700 1 -- 172.21.15.97:0/903699465 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5a60058b80 con 0x7f5a50003a30 2023-07-06T03:48:39.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/903699465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 msgr2=0x7f5a4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:39.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 0x7f5a4806d190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5a640ca7b0 tx=0x7f5a54008040 comp rx=0 tx=0).stop 2023-07-06T03:48:39.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/903699465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 msgr2=0x7f5a641534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:39.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 0x7f5a641534f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000cf80 tx=0x7f5a60001a00 comp rx=0 tx=0).stop 2023-07-06T03:48:39.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/903699465 shutdown_connections 2023-07-06T03:48:39.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a640b31a0 0x7f5a640b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a50003a30 0x7f5a641534f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 --2- 172.21.15.97:0/903699465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a6415d180 0x7f5a6415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/903699465 >> 172.21.15.97:0/903699465 conn(0x7f5a6406a6c0 msgr2=0x7f5a640b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:39.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/903699465 shutdown_connections 2023-07-06T03:48:39.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.306+0000 7f5a6b9ff700 1 -- 172.21.15.97:0/903699465 wait complete. 2023-07-06T03:48:39.325 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:48:39.326 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:48:39.430 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:48:39.536 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:48:39.537 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:48:39.537 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:48:39.582 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:48:39.583 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:48:39.583 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:48:39.630 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:48:39.631 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:48:39.631 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:48:39.678 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:48:39.679 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:48:39.679 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:48:39.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:48:39.728 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T03:48:39.729 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:48:39.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.925+0000 7fda43bff700 1 Processor -- start 2023-07-06T03:48:39.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.925+0000 7fda43bff700 1 -- start start 2023-07-06T03:48:39.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.927+0000 7fda43bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda3c0b8410 0x7fda3c167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.927+0000 7fda43bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3c0cdae0 con 0x7fda3c15c520 2023-07-06T03:48:39.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.927+0000 7fda43bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3c0b6640 con 0x7fda3c0cd390 2023-07-06T03:48:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.927+0000 7fda43bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3c0b6920 con 0x7fda3c0b8410 2023-07-06T03:48:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.927+0000 7fda4199b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fda3c15c520 0x7fda3c160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54142/0 (socket says 172.21.15.97:54142) 2023-07-06T03:48:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.927+0000 7fda4199b700 1 -- 172.21.15.97:0/2056703861 learned_addr learned my addr 172.21.15.97:0/2056703861 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.927+0000 7fda4219c700 1 --2- 172.21.15.97:0/2056703861 >> v2:172.21.15.97:3300/0 conn(0x7fda3c0b8410 0x7fda3c167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.928+0000 7fda40999700 1 -- 172.21.15.97:0/2056703861 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486496114 0 0) 0x7fda3c0cdae0 con 0x7fda3c15c520 2023-07-06T03:48:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.928+0000 7fda40999700 1 -- 172.21.15.97:0/2056703861 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda280035b0 con 0x7fda3c15c520 2023-07-06T03:48:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.928+0000 7fda40999700 1 -- 172.21.15.97:0/2056703861 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486666837 0 0) 0x7fda3c0b6640 con 0x7fda3c0cd390 2023-07-06T03:48:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.928+0000 7fda40999700 1 -- 172.21.15.97:0/2056703861 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda3c0cdae0 con 0x7fda3c0cd390 2023-07-06T03:48:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.928+0000 7fda4219c700 1 -- 172.21.15.97:0/2056703861 >> v1:172.21.15.146:6789/0 conn(0x7fda3c0cd390 legacy=0x7fda3c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.928+0000 7fda4219c700 1 -- 172.21.15.97:0/2056703861 >> v1:172.21.15.97:6789/0 conn(0x7fda3c15c520 legacy=0x7fda3c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.928+0000 7fda4219c700 1 -- 172.21.15.97:0/2056703861 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c0b6c00 con 0x7fda3c0b8410 2023-07-06T03:48:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.928+0000 7fda4219c700 1 --2- 172.21.15.97:0/2056703861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0b8410 0x7fda3c167510 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fda34009d90 tx=0x7fda3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96ca509d1def058 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.928+0000 7fda40999700 1 -- 172.21.15.97:0/2056703861 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3400e070 con 0x7fda3c0b8410 2023-07-06T03:48:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.928+0000 7fda40999700 1 -- 172.21.15.97:0/2056703861 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3400bb20 con 0x7fda3c0b8410 2023-07-06T03:48:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.928+0000 7fda40999700 1 -- 172.21.15.97:0/2056703861 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34012430 con 0x7fda3c0b8410 2023-07-06T03:48:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.929+0000 7fda43bff700 1 -- 172.21.15.97:0/2056703861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0b8410 msgr2=0x7fda3c167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.929+0000 7fda43bff700 1 --2- 172.21.15.97:0/2056703861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0b8410 0x7fda3c167510 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fda34009d90 tx=0x7fda3400b330 comp rx=0 tx=0).stop 2023-07-06T03:48:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.929+0000 7fda43bff700 1 -- 172.21.15.97:0/2056703861 shutdown_connections 2023-07-06T03:48:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.929+0000 7fda43bff700 1 --2- 172.21.15.97:0/2056703861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0b8410 0x7fda3c167510 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.929+0000 7fda43bff700 1 -- 172.21.15.97:0/2056703861 >> 172.21.15.97:0/2056703861 conn(0x7fda3c071790 msgr2=0x7fda3c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.929+0000 7fda43bff700 1 -- 172.21.15.97:0/2056703861 shutdown_connections 2023-07-06T03:48:39.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.929+0000 7fda43bff700 1 -- 172.21.15.97:0/2056703861 wait complete. 2023-07-06T03:48:39.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.931+0000 7fda43bff700 1 Processor -- start 2023-07-06T03:48:39.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.931+0000 7fda43bff700 1 -- start start 2023-07-06T03:48:39.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda43bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0b8410 0x7fda3c14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda43bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda3c0cd390 0x7fda3c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda43bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c15c520 0x7fda3c0d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda43bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c14cdd0 con 0x7fda3c0b8410 2023-07-06T03:48:39.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda43bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c15e4f0 con 0x7fda3c0cd390 2023-07-06T03:48:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda43bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c15e7d0 con 0x7fda3c15c520 2023-07-06T03:48:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda4199b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0b8410 0x7fda3c14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda4199b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0b8410 0x7fda3c14c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50708/0 (socket says 172.21.15.97:50708) 2023-07-06T03:48:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda4199b700 1 -- 172.21.15.97:0/1591652103 learned_addr learned my addr 172.21.15.97:0/1591652103 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda4119a700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda3c0cd390 0x7fda3c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda4219c700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c15c520 0x7fda3c0d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda4199b700 1 -- 172.21.15.97:0/1591652103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c15c520 msgr2=0x7fda3c0d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda4199b700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c15c520 0x7fda3c0d30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda4199b700 1 -- 172.21.15.97:0/1591652103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda3c0cd390 msgr2=0x7fda3c1539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda4199b700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda3c0cd390 0x7fda3c1539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.932+0000 7fda4199b700 1 -- 172.21.15.97:0/1591652103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda34009a70 con 0x7fda3c0b8410 2023-07-06T03:48:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.933+0000 7fda4199b700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0b8410 0x7fda3c14c890 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fda38001eb0 tx=0x7fda3800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.933+0000 7fda327fc700 1 -- 172.21.15.97:0/1591652103 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38005d70 con 0x7fda3c0b8410 2023-07-06T03:48:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.933+0000 7fda327fc700 1 -- 172.21.15.97:0/1591652103 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda380070d0 con 0x7fda3c0b8410 2023-07-06T03:48:39.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.933+0000 7fda43bff700 1 -- 172.21.15.97:0/1591652103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3c15eab0 con 0x7fda3c0b8410 2023-07-06T03:48:39.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.933+0000 7fda327fc700 1 -- 172.21.15.97:0/1591652103 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38007460 con 0x7fda3c0b8410 2023-07-06T03:48:39.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.933+0000 7fda43bff700 1 -- 172.21.15.97:0/1591652103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3c15f070 con 0x7fda3c0b8410 2023-07-06T03:48:39.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.934+0000 7fda43bff700 1 -- 172.21.15.97:0/1591652103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3c04e340 con 0x7fda3c0b8410 2023-07-06T03:48:39.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.935+0000 7fda327fc700 1 -- 172.21.15.97:0/1591652103 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda380075c0 con 0x7fda3c0b8410 2023-07-06T03:48:39.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.937+0000 7fda327fc700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2006ace0 0x7fda2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:39.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.937+0000 7fda327fc700 1 -- 172.21.15.97:0/1591652103 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fda38088690 con 0x7fda3c0b8410 2023-07-06T03:48:39.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.937+0000 7fda4119a700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2006ace0 0x7fda2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:39.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.937+0000 7fda4119a700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2006ace0 0x7fda2006d190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fda2c007fd0 tx=0x7fda2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:39.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:39.941+0000 7fda327fc700 1 -- 172.21.15.97:0/1591652103 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda38053ad0 con 0x7fda3c0b8410 2023-07-06T03:48:40.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.176+0000 7fda43bff700 1 -- 172.21.15.97:0/1591652103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda3c14cf10 con 0x7fda3c0b8410 2023-07-06T03:48:40.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.177+0000 7fda327fc700 1 -- 172.21.15.97:0/1591652103 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fda38017de0 con 0x7fda3c0b8410 2023-07-06T03:48:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:40.179 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1299982704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1299982704},{"type":"v1","addr":"172.21.15.146:6833","nonce":1299982704}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5516,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2867767795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2867767795},{"type":"v1","addr":"172.21.15.146:6835","nonce":2867767795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5602,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/640394657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":640394657},{"type":"v1","addr":"172.21.15.97:6837","nonce":640394657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2363940464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2363940464},{"type":"v1","addr":"172.21.15.97:6836","nonce":2363940464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-07-06T03:48:40.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.179+0000 7fda43bff700 1 -- 172.21.15.97:0/1591652103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2006ace0 msgr2=0x7fda2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:40.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.179+0000 7fda43bff700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2006ace0 0x7fda2006d190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fda2c007fd0 tx=0x7fda2c009040 comp rx=0 tx=0).stop 2023-07-06T03:48:40.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.179+0000 7fda43bff700 1 -- 172.21.15.97:0/1591652103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0b8410 msgr2=0x7fda3c14c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:40.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.179+0000 7fda43bff700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0b8410 0x7fda3c14c890 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fda38001eb0 tx=0x7fda3800c860 comp rx=0 tx=0).stop 2023-07-06T03:48:40.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.179+0000 7fda43bff700 1 -- 172.21.15.97:0/1591652103 shutdown_connections 2023-07-06T03:48:40.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.179+0000 7fda43bff700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2006ace0 0x7fda2006d190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.179+0000 7fda43bff700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c15c520 0x7fda3c0d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.179+0000 7fda43bff700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0b8410 0x7fda3c14c890 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.179+0000 7fda43bff700 1 --2- 172.21.15.97:0/1591652103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda3c0cd390 0x7fda3c1539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.179+0000 7fda43bff700 1 -- 172.21.15.97:0/1591652103 >> 172.21.15.97:0/1591652103 conn(0x7fda3c071790 msgr2=0x7fda3c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:40.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.179+0000 7fda43bff700 1 -- 172.21.15.97:0/1591652103 shutdown_connections 2023-07-06T03:48:40.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.180+0000 7fda43bff700 1 -- 172.21.15.97:0/1591652103 wait complete. 2023-07-06T03:48:40.183 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2023-07-06T03:48:40.199 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T03:48:40.200 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T03:48:40.200 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T03:48:40.200 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:48:40.200 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T03:48:40.201 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T03:48:40.201 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T03:48:40.201 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:48:40.201 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T03:48:40.202 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T03:48:40.202 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T03:48:40.202 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:48:40.202 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T03:48:40.202 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T03:48:40.203 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T03:48:40.203 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:48:40.216 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T03:48:40.216 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T03:48:40.217 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T03:48:40.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T03:48:40.220 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T03:48:40.220 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T03:48:40.222 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T03:48:40.223 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T03:48:40.260 INFO:tasks.ceph.mds.a:Started 2023-07-06T03:48:40.260 DEBUG:teuthology.parallel:result is None 2023-07-06T03:48:40.262 INFO:tasks.ceph.mds.c:Started 2023-07-06T03:48:40.263 DEBUG:teuthology.parallel:result is None 2023-07-06T03:48:40.266 INFO:tasks.ceph.mds.b:Started 2023-07-06T03:48:40.266 DEBUG:teuthology.parallel:result is None 2023-07-06T03:48:40.268 INFO:tasks.ceph.mds.d:Started 2023-07-06T03:48:40.268 DEBUG:teuthology.parallel:result is None 2023-07-06T03:48:40.269 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T03:48:40.407 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:48:40.408 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:48:40.412 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-07-06T03:48:40.412 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:48:40.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.476+0000 7f97535ff700 1 Processor -- start 2023-07-06T03:48:40.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.476+0000 7f97535ff700 1 -- start start 2023-07-06T03:48:40.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.477+0000 7f97535ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f974c15c590 0x7f974c15d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:40.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.477+0000 7f97535ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f974c0719c0 0x7f974c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:40.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.477+0000 7f97535ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974c0ccc30 con 0x7f974c15c1c0 2023-07-06T03:48:40.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.477+0000 7f97535ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974c0ccf10 con 0x7f974c0719c0 2023-07-06T03:48:40.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.477+0000 7f97535ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c0cd1f0 con 0x7f974c15c590 2023-07-06T03:48:40.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f975139b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f974c15c1c0 0x7f974c0cc3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54172/0 (socket says 172.21.15.97:54172) 2023-07-06T03:48:40.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f975139b700 1 -- 172.21.15.97:0/3885873072 learned_addr learned my addr 172.21.15.97:0/3885873072 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:40.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f9751b9c700 1 --2- 172.21.15.97:0/3885873072 >> v2:172.21.15.97:3300/0 conn(0x7f974c0719c0 0x7f974c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:40.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f9750b9a700 1 --2- 172.21.15.97:0/3885873072 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f974c15c590 0x7f974c15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:40.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f9743fff700 1 -- 172.21.15.97:0/3885873072 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782899365 0 0) 0x7f974c0ccc30 con 0x7f974c15c1c0 2023-07-06T03:48:40.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f9743fff700 1 -- 172.21.15.97:0/3885873072 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97380035b0 con 0x7f974c15c1c0 2023-07-06T03:48:40.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f9751b9c700 1 -- 172.21.15.97:0/3885873072 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f974c15c590 msgr2=0x7f974c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:40.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f9751b9c700 1 --2- 172.21.15.97:0/3885873072 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f974c15c590 0x7f974c15d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f9751b9c700 1 -- 172.21.15.97:0/3885873072 >> v1:172.21.15.97:6789/0 conn(0x7f974c15c1c0 legacy=0x7f974c0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:40.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f9751b9c700 1 -- 172.21.15.97:0/3885873072 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c15cb00 con 0x7f974c0719c0 2023-07-06T03:48:40.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f9750b9a700 1 --2- 172.21.15.97:0/3885873072 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f974c15c590 0x7f974c15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:48:40.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.478+0000 7f9751b9c700 1 --2- 172.21.15.97:0/3885873072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f974c0719c0 0x7f974c06a060 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9744009d90 tx=0x7f974400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0abac930af2719d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:40.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.479+0000 7f9743fff700 1 -- 172.21.15.97:0/3885873072 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974400e070 con 0x7f974c0719c0 2023-07-06T03:48:40.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.479+0000 7f9743fff700 1 -- 172.21.15.97:0/3885873072 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974400bb20 con 0x7f974c0719c0 2023-07-06T03:48:40.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.479+0000 7f9743fff700 1 -- 172.21.15.97:0/3885873072 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744012430 con 0x7f974c0719c0 2023-07-06T03:48:40.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.479+0000 7f97535ff700 1 -- 172.21.15.97:0/3885873072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f974c0719c0 msgr2=0x7f974c06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:40.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.479+0000 7f97535ff700 1 --2- 172.21.15.97:0/3885873072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f974c0719c0 0x7f974c06a060 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9744009d90 tx=0x7f974400b330 comp rx=0 tx=0).stop 2023-07-06T03:48:40.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.479+0000 7f97535ff700 1 -- 172.21.15.97:0/3885873072 shutdown_connections 2023-07-06T03:48:40.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.479+0000 7f97535ff700 1 --2- 172.21.15.97:0/3885873072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f974c0719c0 0x7f974c06a060 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.479+0000 7f97535ff700 1 --2- 172.21.15.97:0/3885873072 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f974c15c590 0x7f974c15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.479+0000 7f97535ff700 1 -- 172.21.15.97:0/3885873072 >> 172.21.15.97:0/3885873072 conn(0x7f974c0b1650 msgr2=0x7f974c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:40.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.479+0000 7f97535ff700 1 -- 172.21.15.97:0/3885873072 shutdown_connections 2023-07-06T03:48:40.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.479+0000 7f97535ff700 1 -- 172.21.15.97:0/3885873072 wait complete. 2023-07-06T03:48:40.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.482+0000 7f97535ff700 1 Processor -- start 2023-07-06T03:48:40.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.482+0000 7f97535ff700 1 -- start start 2023-07-06T03:48:40.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.483+0000 7f97535ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f974c0719c0 0x7f974c160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:40.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.483+0000 7f97535ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f974c15c1c0 0x7f974c06eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:40.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.483+0000 7f97535ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f974c15c590 0x7f974c158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:40.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.483+0000 7f97535ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c1f5e90 con 0x7f974c15c590 2023-07-06T03:48:40.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.483+0000 7f97535ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c1609b0 con 0x7f974c0719c0 2023-07-06T03:48:40.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.483+0000 7f97535ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c06f2e0 con 0x7f974c15c1c0 2023-07-06T03:48:40.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.483+0000 7f9751b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f974c15c590 0x7f974c158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:40.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.483+0000 7f9751b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f974c15c590 0x7f974c158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50770/0 (socket says 172.21.15.97:50770) 2023-07-06T03:48:40.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.483+0000 7f9751b9c700 1 -- 172.21.15.97:0/1144009582 learned_addr learned my addr 172.21.15.97:0/1144009582 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:40.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f9750b9a700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f974c15c1c0 0x7f974c06eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:40.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f975139b700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f974c0719c0 0x7f974c160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:40.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f9751b9c700 1 -- 172.21.15.97:0/1144009582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f974c15c1c0 msgr2=0x7f974c06eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:40.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f9751b9c700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f974c15c1c0 0x7f974c06eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f9751b9c700 1 -- 172.21.15.97:0/1144009582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f974c0719c0 msgr2=0x7f974c160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:40.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f9751b9c700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f974c0719c0 0x7f974c160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f9751b9c700 1 -- 172.21.15.97:0/1144009582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0094a0 con 0x7f974c15c590 2023-07-06T03:48:40.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f9751b9c700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f974c15c590 0x7f974c158490 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9744014040 tx=0x7f9744004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:40.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f9741ffb700 1 -- 172.21.15.97:0/1144009582 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97440129f0 con 0x7f974c15c590 2023-07-06T03:48:40.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f9741ffb700 1 -- 172.21.15.97:0/1144009582 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9744020070 con 0x7f974c15c590 2023-07-06T03:48:40.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f97535ff700 1 -- 172.21.15.97:0/1144009582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9744009a70 con 0x7f974c15c590 2023-07-06T03:48:40.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.484+0000 7f9741ffb700 1 -- 172.21.15.97:0/1144009582 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744012db0 con 0x7f974c15c590 2023-07-06T03:48:40.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.485+0000 7f97535ff700 1 -- 172.21.15.97:0/1144009582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c1f5310 con 0x7f974c15c590 2023-07-06T03:48:40.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.485+0000 7f97535ff700 1 -- 172.21.15.97:0/1144009582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c04e340 con 0x7f974c15c590 2023-07-06T03:48:40.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.486+0000 7f9741ffb700 1 -- 172.21.15.97:0/1144009582 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9744004040 con 0x7f974c15c590 2023-07-06T03:48:40.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.487+0000 7f9741ffb700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f973006ace0 0x7f973006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:40.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.487+0000 7f9741ffb700 1 -- 172.21.15.97:0/1144009582 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9744022070 con 0x7f974c15c590 2023-07-06T03:48:40.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.488+0000 7f975139b700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f973006ace0 0x7f973006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:40.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.488+0000 7f975139b700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f973006ace0 0x7f973006d190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9748007fd0 tx=0x7f9748007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:40.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.492+0000 7f9741ffb700 1 -- 172.21.15.97:0/1144009582 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9744056490 con 0x7f974c15c590 2023-07-06T03:48:40.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.701+0000 7f97535ff700 1 -- 172.21.15.97:0/1144009582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f974c071290 con 0x7f974c15c590 2023-07-06T03:48:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.880+0000 7f9741ffb700 1 -- 172.21.15.97:0/1144009582 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9744059ab0 con 0x7f974c15c590 2023-07-06T03:48:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 -- 172.21.15.97:0/1144009582 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f973006ace0 msgr2=0x7f973006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f973006ace0 0x7f973006d190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9748007fd0 tx=0x7f9748007dc0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 -- 172.21.15.97:0/1144009582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f974c15c590 msgr2=0x7f974c158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f974c15c590 0x7f974c158490 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9744014040 tx=0x7f9744004ad0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 -- 172.21.15.97:0/1144009582 shutdown_connections 2023-07-06T03:48:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f973006ace0 0x7f973006d190 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f974c15c590 0x7f974c158490 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f974c0719c0 0x7f974c160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 --2- 172.21.15.97:0/1144009582 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f974c15c1c0 0x7f974c06eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 -- 172.21.15.97:0/1144009582 >> 172.21.15.97:0/1144009582 conn(0x7f974c0b1650 msgr2=0x7f974c15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 -- 172.21.15.97:0/1144009582 shutdown_connections 2023-07-06T03:48:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:40.883+0000 7f97535ff700 1 -- 172.21.15.97:0/1144009582 wait complete. 2023-07-06T03:48:40.887 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-07-06T03:48:40.904 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T03:48:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.097+0000 7f9b53fff700 1 Processor -- start 2023-07-06T03:48:41.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.097+0000 7f9b53fff700 1 -- start start 2023-07-06T03:48:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.098+0000 7f9b53fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9b4c15c520 0x7f9b4c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.098+0000 7f9b53fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c160d00 0x7f9b4c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.098+0000 7f9b53fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b4c0cdae0 con 0x7f9b4c160180 2023-07-06T03:48:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.098+0000 7f9b53fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4c0cd8d0 con 0x7f9b4c15c520 2023-07-06T03:48:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.098+0000 7f9b53fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0b6640 con 0x7f9b4c160d00 2023-07-06T03:48:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.098+0000 7f9b5159a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9b4c160180 0x7f9b4c1605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54184/0 (socket says 172.21.15.97:54184) 2023-07-06T03:48:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b5159a700 1 -- 172.21.15.97:0/1271505062 learned_addr learned my addr 172.21.15.97:0/1271505062 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b5259c700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c160d00 0x7f9b4c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b51d9b700 1 --2- 172.21.15.97:0/1271505062 >> v2:172.21.15.146:3300/0 conn(0x7f9b4c15c520 0x7f9b4c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958305203 0 0) 0x7f9b4c0cdae0 con 0x7f9b4c160180 2023-07-06T03:48:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b380035b0 con 0x7f9b4c160180 2023-07-06T03:48:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1274264378 0 0) 0x7f9b380035b0 con 0x7f9b4c160180 2023-07-06T03:48:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c160d00 msgr2=0x7f9b4c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c160d00 0x7f9b4c0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 >> v2:172.21.15.146:3300/0 conn(0x7f9b4c15c520 msgr2=0x7f9b4c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 --2- 172.21.15.97:0/1271505062 >> v2:172.21.15.146:3300/0 conn(0x7f9b4c15c520 0x7f9b4c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c0b6920 con 0x7f9b4c160180 2023-07-06T03:48:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f9b3c001670 con 0x7f9b4c160180 2023-07-06T03:48:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 >> v1:172.21.15.97:6789/0 conn(0x7f9b4c160180 legacy=0x7f9b4c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b38003a00 0x7f9b38003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c160d00 0x7f9b38004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38004d90 0x7f9b38005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0cd8d0 con 0x7f9b38004d90 2023-07-06T03:48:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0b6640 con 0x7f9b38003a00 2023-07-06T03:48:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.099+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3800ba10 con 0x7f9b4c160d00 2023-07-06T03:48:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.100+0000 7f9b5259c700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38004d90 0x7f9b38005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.100+0000 7f9b51d9b700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b38003a00 0x7f9b38003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.100+0000 7f9b5159a700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c160d00 0x7f9b38004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.100+0000 7f9b51d9b700 1 -- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c160d00 msgr2=0x7f9b38004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.100+0000 7f9b51d9b700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c160d00 0x7f9b38004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.100+0000 7f9b51d9b700 1 -- 172.21.15.97:0/1271505062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38004d90 msgr2=0x7f9b38005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.100+0000 7f9b51d9b700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38004d90 0x7f9b38005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.100+0000 7f9b51d9b700 1 -- 172.21.15.97:0/1271505062 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b380035b0 con 0x7f9b38003a00 2023-07-06T03:48:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.100+0000 7f9b5259c700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38004d90 0x7f9b38005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:48:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.100+0000 7f9b51d9b700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b38003a00 0x7f9b38003e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009b60 tx=0x7f9b38005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edb546caf9a41da5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.101+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4800e6c0 con 0x7f9b38003a00 2023-07-06T03:48:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.101+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4800e820 con 0x7f9b38003a00 2023-07-06T03:48:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.101+0000 7f9b50d99700 1 -- 172.21.15.97:0/1271505062 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4800eb90 con 0x7f9b38003a00 2023-07-06T03:48:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.101+0000 7f9b53fff700 1 -- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b38003a00 msgr2=0x7f9b38003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.101+0000 7f9b53fff700 1 --2- 172.21.15.97:0/1271505062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b38003a00 0x7f9b38003e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009b60 tx=0x7f9b38005880 comp rx=0 tx=0).stop 2023-07-06T03:48:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.101+0000 7f9b5259c700 1 -- 172.21.15.97:0/1271505062 reap_dead start 2023-07-06T03:48:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.101+0000 7f9b53fff700 1 -- 172.21.15.97:0/1271505062 shutdown_connections 2023-07-06T03:48:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.101+0000 7f9b53fff700 1 -- 172.21.15.97:0/1271505062 >> 172.21.15.97:0/1271505062 conn(0x7f9b4c071790 msgr2=0x7f9b4c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.101+0000 7f9b53fff700 1 -- 172.21.15.97:0/1271505062 shutdown_connections 2023-07-06T03:48:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.102+0000 7f9b53fff700 1 -- 172.21.15.97:0/1271505062 wait complete. 2023-07-06T03:48:41.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.103+0000 7f9b53fff700 1 Processor -- start 2023-07-06T03:48:41.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.103+0000 7f9b53fff700 1 -- start start 2023-07-06T03:48:41.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.104+0000 7f9b53fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38003a00 0x7f9b4c14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.104+0000 7f9b53fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c0c9540 0x7f9b4c0c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.104+0000 7f9b53fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b4c158330 0x7f9b4c0c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.104+0000 7f9b53fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0c6db0 con 0x7f9b38003a00 2023-07-06T03:48:41.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.104+0000 7f9b53fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0d1d70 con 0x7f9b4c158330 2023-07-06T03:48:41.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.104+0000 7f9b53fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c0d2050 con 0x7f9b4c0c9540 2023-07-06T03:48:41.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b51d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38003a00 0x7f9b4c14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b5159a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c0c9540 0x7f9b4c0c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b51d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38003a00 0x7f9b4c14d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50794/0 (socket says 172.21.15.97:50794) 2023-07-06T03:48:41.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b51d9b700 1 -- 172.21.15.97:0/3722592372 learned_addr learned my addr 172.21.15.97:0/3722592372 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:41.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b5259c700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b4c158330 0x7f9b4c0c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b51d9b700 1 -- 172.21.15.97:0/3722592372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c0c9540 msgr2=0x7f9b4c0c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b51d9b700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c0c9540 0x7f9b4c0c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b51d9b700 1 -- 172.21.15.97:0/3722592372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b4c158330 msgr2=0x7f9b4c0c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b51d9b700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b4c158330 0x7f9b4c0c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b51d9b700 1 -- 172.21.15.97:0/3722592372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c0d2330 con 0x7f9b38003a00 2023-07-06T03:48:41.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b5259c700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b4c158330 0x7f9b4c0c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:41.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b51d9b700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38003a00 0x7f9b4c14d210 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9b38004780 tx=0x7f9b480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:41.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.105+0000 7f9b427fc700 1 -- 172.21.15.97:0/3722592372 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48021070 con 0x7f9b38003a00 2023-07-06T03:48:41.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.106+0000 7f9b427fc700 1 -- 172.21.15.97:0/3722592372 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b48004520 con 0x7f9b38003a00 2023-07-06T03:48:41.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.106+0000 7f9b53fff700 1 -- 172.21.15.97:0/3722592372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4c0d2610 con 0x7f9b38003a00 2023-07-06T03:48:41.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.106+0000 7f9b427fc700 1 -- 172.21.15.97:0/3722592372 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4800e6c0 con 0x7f9b38003a00 2023-07-06T03:48:41.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.106+0000 7f9b53fff700 1 -- 172.21.15.97:0/3722592372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4c0c77c0 con 0x7f9b38003a00 2023-07-06T03:48:41.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.107+0000 7f9b53fff700 1 -- 172.21.15.97:0/3722592372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4c0d2980 con 0x7f9b38003a00 2023-07-06T03:48:41.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.108+0000 7f9b427fc700 1 -- 172.21.15.97:0/3722592372 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9b48025070 con 0x7f9b38003a00 2023-07-06T03:48:41.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.109+0000 7f9b427fc700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b3006ace0 0x7f9b3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.109+0000 7f9b427fc700 1 -- 172.21.15.97:0/3722592372 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9b4808a250 con 0x7f9b38003a00 2023-07-06T03:48:41.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.109+0000 7f9b5159a700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b3006ace0 0x7f9b3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.110+0000 7f9b5159a700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b3006ace0 0x7f9b3006d190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c001cc0 tx=0x7f9b3c007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:41.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.113+0000 7f9b427fc700 1 -- 172.21.15.97:0/3722592372 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9b48055640 con 0x7f9b38003a00 2023-07-06T03:48:41.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.387+0000 7f9b53fff700 1 -- 172.21.15.97:0/3722592372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9b4c0c9910 con 0x7f9b38003a00 2023-07-06T03:48:41.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.388+0000 7f9b427fc700 1 -- 172.21.15.97:0/3722592372 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9b48058c60 con 0x7f9b38003a00 2023-07-06T03:48:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:41.390 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:48:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-07-06T03:48:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-07-06T03:48:41.390 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkczuvGRAAUKjfmMyKOTCYzOQmtf3n3A==", 2023-07-06T03:48:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:48:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:48:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:48:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.392 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-07-06T03:48:41.392 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkFHvUMBAA2WgMcUUf+Qi8T2ZjhfWaHA==", 2023-07-06T03:48:41.392 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.392 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:48:41.392 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:48:41.393 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:48:41.393 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.393 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.393 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-07-06T03:48:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZk0N4+JRAA3tjqoIBPpZ6C7TMmOOg7lw==", 2023-07-06T03:48:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:48:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:48:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:48:41.394 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-07-06T03:48:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkJnHUABAAwIMH2DmoHaQN3frJCgRiQw==", 2023-07-06T03:48:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.395 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:48:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:48:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:48:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-07-06T03:48:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCLOKZkShOIABAAzIBHDcNkmnDOLSY7Qdb+FA==", 2023-07-06T03:48:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.397 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:48:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:48:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-07-06T03:48:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCOOKZkq1t7AxAAwihuhimbjitZDeki1QuMug==", 2023-07-06T03:48:41.399 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:48:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:48:41.400 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-07-06T03:48:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCROKZkdYv4BhAApTHg/pmlC0lO7hVTkimEgw==", 2023-07-06T03:48:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:48:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:48:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.402 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-07-06T03:48:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCUOKZkqhuMChAAsV9xG6g1dofrpWt3F8tw5w==", 2023-07-06T03:48:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:48:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:48:41.403 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-07-06T03:48:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCaOKZk1Q5aIBAABU1+afwIihm+g4xylryz/A==", 2023-07-06T03:48:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.404 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:48:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:48:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-07-06T03:48:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCdOKZkRskIJBAA+821aoG98aEV7h4FTINQFA==", 2023-07-06T03:48:41.405 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:48:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:48:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.406 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-07-06T03:48:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCgOKZk9qnMJxAATcWG+a/9Hl1AuIL/TSrdEA==", 2023-07-06T03:48:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:48:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:48:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-07-06T03:48:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCjOKZk6wJ/KxAAWnZxt7h7OJhQ+bqdel4+iQ==", 2023-07-06T03:48:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:48:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:48:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-07-06T03:48:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg==", 2023-07-06T03:48:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.409 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:48:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-07-06T03:48:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-06T03:48:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-07-06T03:48:41.410 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkq4/pGRAAucHmR1eBA8MNMy3/l7tl0g==", 2023-07-06T03:48:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:48:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:48:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:48:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:48:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-07-06T03:48:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkhL5fEhAAWhw4IbCPqTO2sXPIcpTVHA==", 2023-07-06T03:48:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:48:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:48:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:48:41.412 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:48:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-07-06T03:48:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkO81dIRAAffjVyvsb/ygBtx8UrYQK0w==", 2023-07-06T03:48:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:48:41.413 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:48:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:48:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:48:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-07-06T03:48:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCFOKZk6uVlHRAA92RDhdDKW/q65SEOQMWgiA==", 2023-07-06T03:48:41.414 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:48:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:48:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:48:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.415 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-07-06T03:48:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkT4ALEhAAXwwkcodQU9meJA0tfseJCg==", 2023-07-06T03:48:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:48:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:48:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-07-06T03:48:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkfnoEChAAuRxqXqPI33dgWkBJoUMh3w==", 2023-07-06T03:48:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:48:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:48:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:48:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:48:41.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.418 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:41.418 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:48:41.418 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:48:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 -- 172.21.15.97:0/3722592372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b3006ace0 msgr2=0x7f9b3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b3006ace0 0x7f9b3006d190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c001cc0 tx=0x7f9b3c007980 comp rx=0 tx=0).stop 2023-07-06T03:48:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 -- 172.21.15.97:0/3722592372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38003a00 msgr2=0x7f9b4c14d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38003a00 0x7f9b4c14d210 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9b38004780 tx=0x7f9b480047c0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 -- 172.21.15.97:0/3722592372 shutdown_connections 2023-07-06T03:48:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b3006ace0 0x7f9b3006d190 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b4c158330 0x7f9b4c0c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b38003a00 0x7f9b4c14d210 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 --2- 172.21.15.97:0/3722592372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b4c0c9540 0x7f9b4c0c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 -- 172.21.15.97:0/3722592372 >> 172.21.15.97:0/3722592372 conn(0x7f9b4c071790 msgr2=0x7f9b4c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 -- 172.21.15.97:0/3722592372 shutdown_connections 2023-07-06T03:48:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.390+0000 7f9b53fff700 1 -- 172.21.15.97:0/3722592372 wait complete. 2023-07-06T03:48:41.422 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:48:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.614+0000 7f7e93dff700 1 Processor -- start 2023-07-06T03:48:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.614+0000 7f7e93dff700 1 -- start start 2023-07-06T03:48:41.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.615+0000 7f7e93dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0cd390 0x7f7e8c160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.615+0000 7f7e93dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7e8c160dd0 0x7f7e8c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.615+0000 7f7e93dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e8c0cdae0 con 0x7f7e8c15c520 2023-07-06T03:48:41.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.615+0000 7f7e93dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e8c0b6640 con 0x7f7e8c160dd0 2023-07-06T03:48:41.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.615+0000 7f7e93dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c0b6920 con 0x7f7e8c0cd390 2023-07-06T03:48:41.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.615+0000 7f7e91b9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7e8c15c520 0x7f7e8c160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54190/0 (socket says 172.21.15.97:54190) 2023-07-06T03:48:41.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.615+0000 7f7e91b9b700 1 -- 172.21.15.97:0/3928053479 learned_addr learned my addr 172.21.15.97:0/3928053479 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:41.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.615+0000 7f7e9239c700 1 --2- 172.21.15.97:0/3928053479 >> v2:172.21.15.146:3300/0 conn(0x7f7e8c160dd0 0x7f7e8c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.616+0000 7f7e9139a700 1 --2- 172.21.15.97:0/3928053479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0cd390 0x7f7e8c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.616+0000 7f7e90b99700 1 -- 172.21.15.97:0/3928053479 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777476255 0 0) 0x7f7e8c0cdae0 con 0x7f7e8c15c520 2023-07-06T03:48:41.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.616+0000 7f7e90b99700 1 -- 172.21.15.97:0/3928053479 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e780035b0 con 0x7f7e8c15c520 2023-07-06T03:48:41.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.616+0000 7f7e9239c700 1 -- 172.21.15.97:0/3928053479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0cd390 msgr2=0x7f7e8c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:41.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.616+0000 7f7e9239c700 1 --2- 172.21.15.97:0/3928053479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0cd390 0x7f7e8c160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.616+0000 7f7e9239c700 1 -- 172.21.15.97:0/3928053479 >> v1:172.21.15.97:6789/0 conn(0x7f7e8c15c520 legacy=0x7f7e8c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.616+0000 7f7e9239c700 1 -- 172.21.15.97:0/3928053479 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c0b6c00 con 0x7f7e8c160dd0 2023-07-06T03:48:41.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.616+0000 7f7e9239c700 1 --2- 172.21.15.97:0/3928053479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c160dd0 0x7f7e8c0b7400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009d90 tx=0x7f7e8400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b540a2e4780ccaa9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.616+0000 7f7e90b99700 1 -- 172.21.15.97:0/3928053479 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8400e070 con 0x7f7e8c160dd0 2023-07-06T03:48:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.617+0000 7f7e90b99700 1 -- 172.21.15.97:0/3928053479 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8400bb20 con 0x7f7e8c160dd0 2023-07-06T03:48:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.617+0000 7f7e90b99700 1 -- 172.21.15.97:0/3928053479 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84012430 con 0x7f7e8c160dd0 2023-07-06T03:48:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.617+0000 7f7e93dff700 1 -- 172.21.15.97:0/3928053479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c160dd0 msgr2=0x7f7e8c0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.617+0000 7f7e93dff700 1 --2- 172.21.15.97:0/3928053479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c160dd0 0x7f7e8c0b7400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009d90 tx=0x7f7e8400b330 comp rx=0 tx=0).stop 2023-07-06T03:48:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.617+0000 7f7e93dff700 1 -- 172.21.15.97:0/3928053479 shutdown_connections 2023-07-06T03:48:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.617+0000 7f7e93dff700 1 --2- 172.21.15.97:0/3928053479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c160dd0 0x7f7e8c0b7400 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.617+0000 7f7e93dff700 1 --2- 172.21.15.97:0/3928053479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0cd390 0x7f7e8c160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.617+0000 7f7e93dff700 1 -- 172.21.15.97:0/3928053479 >> 172.21.15.97:0/3928053479 conn(0x7f7e8c071790 msgr2=0x7f7e8c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.617+0000 7f7e93dff700 1 -- 172.21.15.97:0/3928053479 shutdown_connections 2023-07-06T03:48:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.617+0000 7f7e93dff700 1 -- 172.21.15.97:0/3928053479 wait complete. 2023-07-06T03:48:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.619+0000 7f7e93dff700 1 Processor -- start 2023-07-06T03:48:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.619+0000 7f7e93dff700 1 -- start start 2023-07-06T03:48:41.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.620+0000 7f7e93dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0cd390 0x7f7e8c14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.620+0000 7f7e93dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e8c15c520 0x7f7e8c15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.620+0000 7f7e93dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c160dd0 0x7f7e8c1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.620+0000 7f7e93dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c15b240 con 0x7f7e8c15c520 2023-07-06T03:48:41.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.620+0000 7f7e93dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c15a910 con 0x7f7e8c160dd0 2023-07-06T03:48:41.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.620+0000 7f7e93dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c15abf0 con 0x7f7e8c0cd390 2023-07-06T03:48:41.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.620+0000 7f7e9139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e8c15c520 0x7f7e8c15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.620+0000 7f7e9139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e8c15c520 0x7f7e8c15b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50798/0 (socket says 172.21.15.97:50798) 2023-07-06T03:48:41.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.620+0000 7f7e9139a700 1 -- 172.21.15.97:0/3617907805 learned_addr learned my addr 172.21.15.97:0/3617907805 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:41.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e9139a700 1 -- 172.21.15.97:0/3617907805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0cd390 msgr2=0x7f7e8c14c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e91b9b700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0cd390 0x7f7e8c14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e9239c700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c160dd0 0x7f7e8c1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e9139a700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0cd390 0x7f7e8c14c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e9139a700 1 -- 172.21.15.97:0/3617907805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c160dd0 msgr2=0x7f7e8c1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e9139a700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c160dd0 0x7f7e8c1586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e9139a700 1 -- 172.21.15.97:0/3617907805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e84009a70 con 0x7f7e8c15c520 2023-07-06T03:48:41.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e9239c700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c160dd0 0x7f7e8c1586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:41.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e91b9b700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0cd390 0x7f7e8c14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:41.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e9139a700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e8c15c520 0x7f7e8c15b6d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00e870 tx=0x7f7e7c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:41.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e827fc700 1 -- 172.21.15.97:0/3617907805 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c009590 con 0x7f7e8c15c520 2023-07-06T03:48:41.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e827fc700 1 -- 172.21.15.97:0/3617907805 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7c00ee70 con 0x7f7e8c15c520 2023-07-06T03:48:41.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e93dff700 1 -- 172.21.15.97:0/3617907805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e8c15aed0 con 0x7f7e8c15c520 2023-07-06T03:48:41.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e827fc700 1 -- 172.21.15.97:0/3617907805 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c00ca50 con 0x7f7e8c15c520 2023-07-06T03:48:41.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.621+0000 7f7e93dff700 1 -- 172.21.15.97:0/3617907805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8c1f2920 con 0x7f7e8c15c520 2023-07-06T03:48:41.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.622+0000 7f7e93dff700 1 -- 172.21.15.97:0/3617907805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8c15b060 con 0x7f7e8c15c520 2023-07-06T03:48:41.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.623+0000 7f7e827fc700 1 -- 172.21.15.97:0/3617907805 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e7c00cbb0 con 0x7f7e8c15c520 2023-07-06T03:48:41.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.625+0000 7f7e827fc700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7006ace0 0x7f7e7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:41.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.625+0000 7f7e827fc700 1 -- 172.21.15.97:0/3617907805 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7e7c0891a0 con 0x7f7e8c15c520 2023-07-06T03:48:41.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.625+0000 7f7e91b9b700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7006ace0 0x7f7e7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.626+0000 7f7e91b9b700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7006ace0 0x7f7e7006d190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7e880011e0 tx=0x7f7e88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:41.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.629+0000 7f7e827fc700 1 -- 172.21.15.97:0/3617907805 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7e7c057f90 con 0x7f7e8c15c520 2023-07-06T03:48:41.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.826+0000 7f7e93dff700 1 -- 172.21.15.97:0/3617907805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e8c04e340 con 0x7f7e8c15c520 2023-07-06T03:48:41.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.827+0000 7f7e827fc700 1 -- 172.21.15.97:0/3617907805 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+14665 (secure 0 0 0) 0x7f7e7c00ce60 con 0x7f7e8c15c520 2023-07-06T03:48:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:41.829 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":56,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:48:40.897680+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:48:41.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 -- 172.21.15.97:0/3617907805 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7006ace0 msgr2=0x7f7e7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7006ace0 0x7f7e7006d190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7e880011e0 tx=0x7f7e88009040 comp rx=0 tx=0).stop 2023-07-06T03:48:41.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 -- 172.21.15.97:0/3617907805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e8c15c520 msgr2=0x7f7e8c15b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:41.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e8c15c520 0x7f7e8c15b6d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00e870 tx=0x7f7e7c00ec30 comp rx=0 tx=0).stop 2023-07-06T03:48:41.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 -- 172.21.15.97:0/3617907805 shutdown_connections 2023-07-06T03:48:41.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7e7006ace0 0x7f7e7006d190 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c160dd0 0x7f7e8c1586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7e8c0cd390 0x7f7e8c14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 --2- 172.21.15.97:0/3617907805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e8c15c520 0x7f7e8c15b6d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:41.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 -- 172.21.15.97:0/3617907805 >> 172.21.15.97:0/3617907805 conn(0x7f7e8c071790 msgr2=0x7f7e8c15eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:41.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 -- 172.21.15.97:0/3617907805 shutdown_connections 2023-07-06T03:48:41.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:41.829+0000 7f7e93dff700 1 -- 172.21.15.97:0/3617907805 wait complete. 2023-07-06T03:48:41.849 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:48:41.850 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:48:42.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.056+0000 7f8c34fff700 1 Processor -- start 2023-07-06T03:48:42.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.056+0000 7f8c34fff700 1 -- start start 2023-07-06T03:48:42.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.057+0000 7f8c34fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8c300b4a40 0x7f8c300d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.057+0000 7f8c34fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c300b5e10 0x7f8c300cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.057+0000 7f8c34fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c300b6280 0x7f8c300cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.057+0000 7f8c34fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c300d0410 con 0x7f8c300b6280 2023-07-06T03:48:42.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.057+0000 7f8c34fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c300d2d30 con 0x7f8c300b4a40 2023-07-06T03:48:42.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.057+0000 7f8c34fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300cf210 con 0x7f8c300b5e10 2023-07-06T03:48:42.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.057+0000 7f8c2ed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c300b6280 0x7f8c300cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.057+0000 7f8c2ed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c300b6280 0x7f8c300cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50802/0 (socket says 172.21.15.97:50802) 2023-07-06T03:48:42.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.057+0000 7f8c2ed9d700 1 -- 172.21.15.97:0/991766123 learned_addr learned my addr 172.21.15.97:0/991766123 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:42.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.058+0000 7f8c2e59c700 1 --2- 172.21.15.97:0/991766123 >> v2:172.21.15.146:3300/0 conn(0x7f8c300b4a40 0x7f8c300d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.058+0000 7f8c27fff700 1 --2- 172.21.15.97:0/991766123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c300b5e10 0x7f8c300cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.058+0000 7f8c2ed9d700 1 -- 172.21.15.97:0/991766123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c300b5e10 msgr2=0x7f8c300cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:42.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.058+0000 7f8c2ed9d700 1 --2- 172.21.15.97:0/991766123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c300b5e10 0x7f8c300cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.058+0000 7f8c2ed9d700 1 -- 172.21.15.97:0/991766123 >> v2:172.21.15.146:3300/0 conn(0x7f8c300b4a40 msgr2=0x7f8c300d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:42.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.058+0000 7f8c2ed9d700 1 --2- 172.21.15.97:0/991766123 >> v2:172.21.15.146:3300/0 conn(0x7f8c300b4a40 0x7f8c300d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.058+0000 7f8c2ed9d700 1 -- 172.21.15.97:0/991766123 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c300cf4b0 con 0x7f8c300b6280 2023-07-06T03:48:42.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.058+0000 7f8c2ed9d700 1 --2- 172.21.15.97:0/991766123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c300b6280 0x7f8c300cfed0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8c3006b960 tx=0x7f8c2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff57242f66e120c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:42.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.058+0000 7f8c2dd9b700 1 -- 172.21.15.97:0/991766123 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2800e070 con 0x7f8c300b6280 2023-07-06T03:48:42.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.058+0000 7f8c2dd9b700 1 -- 172.21.15.97:0/991766123 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2800b9c0 con 0x7f8c300b6280 2023-07-06T03:48:42.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.058+0000 7f8c2dd9b700 1 -- 172.21.15.97:0/991766123 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2800bd70 con 0x7f8c300b6280 2023-07-06T03:48:42.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.059+0000 7f8c34fff700 1 -- 172.21.15.97:0/991766123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c300b6280 msgr2=0x7f8c300cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.059+0000 7f8c34fff700 1 --2- 172.21.15.97:0/991766123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c300b6280 0x7f8c300cfed0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8c3006b960 tx=0x7f8c2800b330 comp rx=0 tx=0).stop 2023-07-06T03:48:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.059+0000 7f8c34fff700 1 -- 172.21.15.97:0/991766123 shutdown_connections 2023-07-06T03:48:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.059+0000 7f8c34fff700 1 --2- 172.21.15.97:0/991766123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c300b6280 0x7f8c300cfed0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.059+0000 7f8c34fff700 1 --2- 172.21.15.97:0/991766123 >> v2:172.21.15.146:3300/0 conn(0x7f8c300b4a40 0x7f8c300d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.059+0000 7f8c34fff700 1 --2- 172.21.15.97:0/991766123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c300b5e10 0x7f8c300cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.059+0000 7f8c34fff700 1 -- 172.21.15.97:0/991766123 >> 172.21.15.97:0/991766123 conn(0x7f8c3006a900 msgr2=0x7f8c3006acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.059+0000 7f8c34fff700 1 -- 172.21.15.97:0/991766123 shutdown_connections 2023-07-06T03:48:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.059+0000 7f8c34fff700 1 -- 172.21.15.97:0/991766123 wait complete. 2023-07-06T03:48:42.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.061+0000 7f8c34fff700 1 Processor -- start 2023-07-06T03:48:42.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.061+0000 7f8c34fff700 1 -- start start 2023-07-06T03:48:42.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c34fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c300b4a40 0x7f8c3014cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c34fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c300b5e10 0x7f8c300cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c34fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c300b6280 0x7f8c300cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c34fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3015d9c0 con 0x7f8c300b6280 2023-07-06T03:48:42.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c34fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3015cad0 con 0x7f8c300b4a40 2023-07-06T03:48:42.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c34fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3015cdb0 con 0x7f8c300b5e10 2023-07-06T03:48:42.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c2ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c300b6280 0x7f8c300cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c2ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c300b6280 0x7f8c300cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50808/0 (socket says 172.21.15.97:50808) 2023-07-06T03:48:42.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c2ed9d700 1 -- 172.21.15.97:0/763389064 learned_addr learned my addr 172.21.15.97:0/763389064 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:42.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c2e59c700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c300b4a40 0x7f8c3014cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c27fff700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c300b5e10 0x7f8c300cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c2ed9d700 1 -- 172.21.15.97:0/763389064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c300b5e10 msgr2=0x7f8c300cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c2ed9d700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c300b5e10 0x7f8c300cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c2ed9d700 1 -- 172.21.15.97:0/763389064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c300b4a40 msgr2=0x7f8c3014cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c2ed9d700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c300b4a40 0x7f8c3014cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.062+0000 7f8c2ed9d700 1 -- 172.21.15.97:0/763389064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c28009b30 con 0x7f8c300b6280 2023-07-06T03:48:42.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.063+0000 7f8c2ed9d700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c300b6280 0x7f8c300cc760 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8c28012000 tx=0x7f8c28004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:42.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.063+0000 7f8c26ffd700 1 -- 172.21.15.97:0/763389064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28004450 con 0x7f8c300b6280 2023-07-06T03:48:42.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.063+0000 7f8c26ffd700 1 -- 172.21.15.97:0/763389064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c280045d0 con 0x7f8c300b6280 2023-07-06T03:48:42.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.063+0000 7f8c34fff700 1 -- 172.21.15.97:0/763389064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3015d090 con 0x7f8c300b6280 2023-07-06T03:48:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.063+0000 7f8c26ffd700 1 -- 172.21.15.97:0/763389064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c280099b0 con 0x7f8c300b6280 2023-07-06T03:48:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.063+0000 7f8c34fff700 1 -- 172.21.15.97:0/763389064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3015d650 con 0x7f8c300b6280 2023-07-06T03:48:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.064+0000 7f8c34fff700 1 -- 172.21.15.97:0/763389064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3015db70 con 0x7f8c300b6280 2023-07-06T03:48:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.065+0000 7f8c26ffd700 1 -- 172.21.15.97:0/763389064 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c28025460 con 0x7f8c300b6280 2023-07-06T03:48:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.066+0000 7f8c26ffd700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8c1006ace0 0x7f8c1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.066+0000 7f8c26ffd700 1 -- 172.21.15.97:0/763389064 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8c2808ac70 con 0x7f8c300b6280 2023-07-06T03:48:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.067+0000 7f8c2e59c700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8c1006ace0 0x7f8c1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.067+0000 7f8c2e59c700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8c1006ace0 0x7f8c1006d190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8c200045f0 tx=0x7f8c20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.071+0000 7f8c26ffd700 1 -- 172.21.15.97:0/763389064 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8c3015db70 con 0x7f8c300b6280 2023-07-06T03:48:42.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.265+0000 7f8c34fff700 1 -- 172.21.15.97:0/763389064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c3004e340 con 0x7f8c300b6280 2023-07-06T03:48:42.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.266+0000 7f8c26ffd700 1 -- 172.21.15.97:0/763389064 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c28056220 con 0x7f8c300b6280 2023-07-06T03:48:42.268 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:48:42.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 -- 172.21.15.97:0/763389064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8c1006ace0 msgr2=0x7f8c1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8c1006ace0 0x7f8c1006d190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8c200045f0 tx=0x7f8c20008040 comp rx=0 tx=0).stop 2023-07-06T03:48:42.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 -- 172.21.15.97:0/763389064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c300b6280 msgr2=0x7f8c300cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c300b6280 0x7f8c300cc760 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8c28012000 tx=0x7f8c28004ad0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 -- 172.21.15.97:0/763389064 shutdown_connections 2023-07-06T03:48:42.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8c1006ace0 0x7f8c1006d190 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c300b6280 0x7f8c300cc760 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c300b4a40 0x7f8c3014cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 --2- 172.21.15.97:0/763389064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8c300b5e10 0x7f8c300cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 -- 172.21.15.97:0/763389064 >> 172.21.15.97:0/763389064 conn(0x7f8c3006a900 msgr2=0x7f8c300b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:42.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 -- 172.21.15.97:0/763389064 shutdown_connections 2023-07-06T03:48:42.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.268+0000 7f8c34fff700 1 -- 172.21.15.97:0/763389064 wait complete. 2023-07-06T03:48:42.288 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:48:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.482+0000 7fd56cdff700 1 Processor -- start 2023-07-06T03:48:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.482+0000 7fd56cdff700 1 -- start start 2023-07-06T03:48:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.483+0000 7fd56cdff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd5680b84e0 0x7fd5680cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.483+0000 7fd56cdff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5680d03d0 0x7fd5680b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.483+0000 7fd56cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5680b4cb0 0x7fd56806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.483+0000 7fd56cdff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5680b5130 con 0x7fd5680d03d0 2023-07-06T03:48:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd56cdff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56815cad0 con 0x7fd5680b84e0 2023-07-06T03:48:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd56cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd568160b80 con 0x7fd5680b4cb0 2023-07-06T03:48:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd55ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5680d03d0 0x7fd5680b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd55ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5680d03d0 0x7fd5680b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50812/0 (socket says 172.21.15.97:50812) 2023-07-06T03:48:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd55ffff700 1 -- 172.21.15.97:0/3891245375 learned_addr learned my addr 172.21.15.97:0/3891245375 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd566d9d700 1 --2- 172.21.15.97:0/3891245375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5680b4cb0 0x7fd56806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd56659c700 1 --2- 172.21.15.97:0/3891245375 >> v2:172.21.15.146:3300/0 conn(0x7fd5680b84e0 0x7fd5680cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd55ffff700 1 -- 172.21.15.97:0/3891245375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5680b4cb0 msgr2=0x7fd56806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd55ffff700 1 --2- 172.21.15.97:0/3891245375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5680b4cb0 0x7fd56806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd55ffff700 1 -- 172.21.15.97:0/3891245375 >> v2:172.21.15.146:3300/0 conn(0x7fd5680b84e0 msgr2=0x7fd5680cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd55ffff700 1 --2- 172.21.15.97:0/3891245375 >> v2:172.21.15.146:3300/0 conn(0x7fd5680b84e0 0x7fd5680cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd55ffff700 1 -- 172.21.15.97:0/3891245375 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5681610b0 con 0x7fd5680d03d0 2023-07-06T03:48:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.484+0000 7fd55ffff700 1 --2- 172.21.15.97:0/3891245375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5680d03d0 0x7fd5680b4770 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd5680677e0 tx=0x7fd550009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdd313d26258646 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd565d9b700 1 -- 172.21.15.97:0/3891245375 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000b870 con 0x7fd5680d03d0 2023-07-06T03:48:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd565d9b700 1 -- 172.21.15.97:0/3891245375 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55000b9d0 con 0x7fd5680d03d0 2023-07-06T03:48:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd565d9b700 1 -- 172.21.15.97:0/3891245375 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000bd60 con 0x7fd5680d03d0 2023-07-06T03:48:42.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd56cdff700 1 -- 172.21.15.97:0/3891245375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5680d03d0 msgr2=0x7fd5680b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd56cdff700 1 --2- 172.21.15.97:0/3891245375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5680d03d0 0x7fd5680b4770 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd5680677e0 tx=0x7fd550009bd0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd56cdff700 1 -- 172.21.15.97:0/3891245375 shutdown_connections 2023-07-06T03:48:42.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd56cdff700 1 --2- 172.21.15.97:0/3891245375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5680b4cb0 0x7fd56806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd56cdff700 1 --2- 172.21.15.97:0/3891245375 >> v2:172.21.15.146:3300/0 conn(0x7fd5680b84e0 0x7fd5680cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd56cdff700 1 --2- 172.21.15.97:0/3891245375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5680d03d0 0x7fd5680b4770 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd56cdff700 1 -- 172.21.15.97:0/3891245375 >> 172.21.15.97:0/3891245375 conn(0x7fd56806b2b0 msgr2=0x7fd5680cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:42.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd56cdff700 1 -- 172.21.15.97:0/3891245375 shutdown_connections 2023-07-06T03:48:42.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.485+0000 7fd56cdff700 1 -- 172.21.15.97:0/3891245375 wait complete. 2023-07-06T03:48:42.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.487+0000 7fd56cdff700 1 Processor -- start 2023-07-06T03:48:42.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.487+0000 7fd56cdff700 1 -- start start 2023-07-06T03:48:42.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.488+0000 7fd56cdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5680b4cb0 0x7fd568154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.488+0000 7fd56cdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5680b84e0 0x7fd5680cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.488+0000 7fd56cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5680d03d0 0x7fd5681591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.488+0000 7fd56cdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5681f5a50 con 0x7fd5680b4cb0 2023-07-06T03:48:42.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.488+0000 7fd56cdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56815e460 con 0x7fd5680b84e0 2023-07-06T03:48:42.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.488+0000 7fd56cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd568154750 con 0x7fd5680d03d0 2023-07-06T03:48:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.488+0000 7fd56659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5680b4cb0 0x7fd568154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.488+0000 7fd56659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5680b4cb0 0x7fd568154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50828/0 (socket says 172.21.15.97:50828) 2023-07-06T03:48:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.488+0000 7fd56659c700 1 -- 172.21.15.97:0/3733042515 learned_addr learned my addr 172.21.15.97:0/3733042515 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd55ffff700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5680b84e0 0x7fd5680cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd566d9d700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5680d03d0 0x7fd5681591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd56659c700 1 -- 172.21.15.97:0/3733042515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5680d03d0 msgr2=0x7fd5681591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd56659c700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5680d03d0 0x7fd5681591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd56659c700 1 -- 172.21.15.97:0/3733042515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5680b84e0 msgr2=0x7fd5680cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd56659c700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5680b84e0 0x7fd5680cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd56659c700 1 -- 172.21.15.97:0/3733042515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd550009550 con 0x7fd5680b4cb0 2023-07-06T03:48:42.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd55ffff700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5680b84e0 0x7fd5680cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:42.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd566d9d700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5680d03d0 0x7fd5681591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:42.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd56659c700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5680b4cb0 0x7fd568154210 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd55800e810 tx=0x7fd55800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:42.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd55effd700 1 -- 172.21.15.97:0/3733042515 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558013610 con 0x7fd5680b4cb0 2023-07-06T03:48:42.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd55effd700 1 -- 172.21.15.97:0/3733042515 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd558013770 con 0x7fd5680b4cb0 2023-07-06T03:48:42.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd55effd700 1 -- 172.21.15.97:0/3733042515 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558013b20 con 0x7fd5680b4cb0 2023-07-06T03:48:42.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd56cdff700 1 -- 172.21.15.97:0/3733042515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5680ccd00 con 0x7fd5680b4cb0 2023-07-06T03:48:42.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.489+0000 7fd56cdff700 1 -- 172.21.15.97:0/3733042515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5681f5390 con 0x7fd5680b4cb0 2023-07-06T03:48:42.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.490+0000 7fd56cdff700 1 -- 172.21.15.97:0/3733042515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5681f5cc0 con 0x7fd5680b4cb0 2023-07-06T03:48:42.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.491+0000 7fd55effd700 1 -- 172.21.15.97:0/3733042515 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd558013c80 con 0x7fd5680b4cb0 2023-07-06T03:48:42.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.492+0000 7fd55effd700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd54806ace0 0x7fd54806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.492+0000 7fd55effd700 1 -- 172.21.15.97:0/3733042515 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd558088e60 con 0x7fd5680b4cb0 2023-07-06T03:48:42.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.492+0000 7fd55ffff700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd54806ace0 0x7fd54806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.493+0000 7fd55ffff700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd54806ace0 0x7fd54806d190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5680c9ee0 tx=0x7fd550008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:42.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.496+0000 7fd55effd700 1 -- 172.21.15.97:0/3733042515 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd5580541d0 con 0x7fd5680b4cb0 2023-07-06T03:48:42.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.691+0000 7fd56cdff700 1 -- 172.21.15.97:0/3733042515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd56804e340 con 0x7fd5680b4cb0 2023-07-06T03:48:42.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.692+0000 7fd55effd700 1 -- 172.21.15.97:0/3733042515 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd5580577f0 con 0x7fd5680b4cb0 2023-07-06T03:48:42.694 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:48:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 -- 172.21.15.97:0/3733042515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd54806ace0 msgr2=0x7fd54806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd54806ace0 0x7fd54806d190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5680c9ee0 tx=0x7fd550008030 comp rx=0 tx=0).stop 2023-07-06T03:48:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 -- 172.21.15.97:0/3733042515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5680b4cb0 msgr2=0x7fd568154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5680b4cb0 0x7fd568154210 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd55800e810 tx=0x7fd55800ebd0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 -- 172.21.15.97:0/3733042515 shutdown_connections 2023-07-06T03:48:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd54806ace0 0x7fd54806d190 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd5680d03d0 0x7fd5681591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5680b4cb0 0x7fd568154210 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 --2- 172.21.15.97:0/3733042515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5680b84e0 0x7fd5680cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 -- 172.21.15.97:0/3733042515 >> 172.21.15.97:0/3733042515 conn(0x7fd56806b2b0 msgr2=0x7fd5680b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 -- 172.21.15.97:0/3733042515 shutdown_connections 2023-07-06T03:48:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.694+0000 7fd56cdff700 1 -- 172.21.15.97:0/3733042515 wait complete. 2023-07-06T03:48:42.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T03:48:42.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.910+0000 7fd021bff700 1 Processor -- start 2023-07-06T03:48:42.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.910+0000 7fd021bff700 1 -- start start 2023-07-06T03:48:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.911+0000 7fd021bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd01c0d2930 0x7fd01c0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.911+0000 7fd021bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd01c0b1990 0x7fd01c0b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.911+0000 7fd021bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01c0d0c70 con 0x7fd01c15c980 2023-07-06T03:48:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.911+0000 7fd021bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01c0b2310 con 0x7fd01c0b1990 2023-07-06T03:48:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.911+0000 7fd021bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01c0d0ec0 con 0x7fd01c0d2930 2023-07-06T03:48:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.911+0000 7fd01bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd01c0b1990 0x7fd01c0b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.911+0000 7fd01b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd01c15c980 0x7fd01c0cd6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54204/0 (socket says 172.21.15.97:54204) 2023-07-06T03:48:42.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.911+0000 7fd01b7fe700 1 -- 172.21.15.97:0/1736518654 learned_addr learned my addr 172.21.15.97:0/1736518654 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.912+0000 7fd01affd700 1 --2- 172.21.15.97:0/1736518654 >> v2:172.21.15.146:3300/0 conn(0x7fd01c0d2930 0x7fd01c0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.912+0000 7fd01a7fc700 1 -- 172.21.15.97:0/1736518654 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897088357 0 0) 0x7fd01c0d0c70 con 0x7fd01c15c980 2023-07-06T03:48:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.912+0000 7fd01bfff700 1 -- 172.21.15.97:0/1736518654 >> v2:172.21.15.146:3300/0 conn(0x7fd01c0d2930 msgr2=0x7fd01c0d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.912+0000 7fd01bfff700 1 --2- 172.21.15.97:0/1736518654 >> v2:172.21.15.146:3300/0 conn(0x7fd01c0d2930 0x7fd01c0d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.912+0000 7fd01bfff700 1 -- 172.21.15.97:0/1736518654 >> v1:172.21.15.97:6789/0 conn(0x7fd01c15c980 legacy=0x7fd01c0cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.912+0000 7fd01bfff700 1 -- 172.21.15.97:0/1736518654 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c0d11a0 con 0x7fd01c0b1990 2023-07-06T03:48:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.912+0000 7fd01bfff700 1 --2- 172.21.15.97:0/1736518654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd01c0b1990 0x7fd01c0b1dd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0cd350 tx=0x7fd01400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1906b4592a3a49b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.912+0000 7fd01a7fc700 1 -- 172.21.15.97:0/1736518654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01400e070 con 0x7fd01c0b1990 2023-07-06T03:48:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.912+0000 7fd01a7fc700 1 -- 172.21.15.97:0/1736518654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01400bbd0 con 0x7fd01c0b1990 2023-07-06T03:48:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.912+0000 7fd01a7fc700 1 -- 172.21.15.97:0/1736518654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014012430 con 0x7fd01c0b1990 2023-07-06T03:48:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.913+0000 7fd021bff700 1 -- 172.21.15.97:0/1736518654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd01c0b1990 msgr2=0x7fd01c0b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.913+0000 7fd021bff700 1 --2- 172.21.15.97:0/1736518654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd01c0b1990 0x7fd01c0b1dd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0cd350 tx=0x7fd01400b330 comp rx=0 tx=0).stop 2023-07-06T03:48:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.913+0000 7fd021bff700 1 -- 172.21.15.97:0/1736518654 shutdown_connections 2023-07-06T03:48:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.913+0000 7fd021bff700 1 --2- 172.21.15.97:0/1736518654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd01c0b1990 0x7fd01c0b1dd0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.913+0000 7fd021bff700 1 --2- 172.21.15.97:0/1736518654 >> v2:172.21.15.146:3300/0 conn(0x7fd01c0d2930 0x7fd01c0d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.913+0000 7fd021bff700 1 -- 172.21.15.97:0/1736518654 >> 172.21.15.97:0/1736518654 conn(0x7fd01c06b2f0 msgr2=0x7fd01c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.913+0000 7fd021bff700 1 -- 172.21.15.97:0/1736518654 shutdown_connections 2023-07-06T03:48:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.913+0000 7fd021bff700 1 -- 172.21.15.97:0/1736518654 wait complete. 2023-07-06T03:48:42.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.915+0000 7fd021bff700 1 Processor -- start 2023-07-06T03:48:42.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.915+0000 7fd021bff700 1 -- start start 2023-07-06T03:48:42.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd021bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd01c0b1990 0x7fd01c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd021bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c0d2930 0x7fd01c154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd021bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01c15c980 0x7fd01c154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd021bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0b32a0 con 0x7fd01c0b1990 2023-07-06T03:48:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd021bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c1551d0 con 0x7fd01c15c980 2023-07-06T03:48:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd021bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c1554b0 con 0x7fd01c0d2930 2023-07-06T03:48:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd01c0b1990 0x7fd01c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd01c0b1990 0x7fd01c0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50854/0 (socket says 172.21.15.97:50854) 2023-07-06T03:48:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01b7fe700 1 -- 172.21.15.97:0/1019799009 learned_addr learned my addr 172.21.15.97:0/1019799009 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01affd700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c0d2930 0x7fd01c154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01bfff700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01c15c980 0x7fd01c154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01b7fe700 1 -- 172.21.15.97:0/1019799009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c0d2930 msgr2=0x7fd01c154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01b7fe700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c0d2930 0x7fd01c154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01b7fe700 1 -- 172.21.15.97:0/1019799009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01c15c980 msgr2=0x7fd01c154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01b7fe700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01c15c980 0x7fd01c154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01b7fe700 1 -- 172.21.15.97:0/1019799009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd014009a70 con 0x7fd01c0b1990 2023-07-06T03:48:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01bfff700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01c15c980 0x7fd01c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.916+0000 7fd01b7fe700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd01c0b1990 0x7fd01c0b3900 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00c550 tx=0x7fd00c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.917+0000 7fd003fff700 1 -- 172.21.15.97:0/1019799009 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c004c80 con 0x7fd01c0b1990 2023-07-06T03:48:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.917+0000 7fd003fff700 1 -- 172.21.15.97:0/1019799009 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c004e00 con 0x7fd01c0b1990 2023-07-06T03:48:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.917+0000 7fd021bff700 1 -- 172.21.15.97:0/1019799009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01c155790 con 0x7fd01c0b1990 2023-07-06T03:48:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.917+0000 7fd003fff700 1 -- 172.21.15.97:0/1019799009 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c007310 con 0x7fd01c0b1990 2023-07-06T03:48:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.917+0000 7fd021bff700 1 -- 172.21.15.97:0/1019799009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01c0b3e40 con 0x7fd01c0b1990 2023-07-06T03:48:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.918+0000 7fd021bff700 1 -- 172.21.15.97:0/1019799009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd01c0b7050 con 0x7fd01c0b1990 2023-07-06T03:48:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.919+0000 7fd003fff700 1 -- 172.21.15.97:0/1019799009 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd00c007470 con 0x7fd01c0b1990 2023-07-06T03:48:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.920+0000 7fd003fff700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd00806ace0 0x7fd00806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.920+0000 7fd003fff700 1 -- 172.21.15.97:0/1019799009 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd00c088ea0 con 0x7fd01c0b1990 2023-07-06T03:48:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.920+0000 7fd01affd700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd00806ace0 0x7fd00806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.920+0000 7fd01affd700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd00806ace0 0x7fd00806d190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd01c1f5600 tx=0x7fd010008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:42.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:42.924+0000 7fd003fff700 1 -- 172.21.15.97:0/1019799009 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd01c0b7050 con 0x7fd01c0b1990 2023-07-06T03:48:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.126+0000 7fd021bff700 1 -- 172.21.15.97:0/1019799009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd01c0b7050 con 0x7fd01c0b1990 2023-07-06T03:48:43.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.127+0000 7fd003fff700 1 -- 172.21.15.97:0/1019799009 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) v1 ==== 84+0+126 (secure 0 0 0) 0x7fd01c0b7050 con 0x7fd01c0b1990 2023-07-06T03:48:43.129 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:43.129 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-07-06T03:48:43.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:43.129 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-07-06T03:48:43.129 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-07-06T03:48:43.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:43.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:48:43.130 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 2, 2023-07-06T03:48:43.130 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": "rbd" 2023-07-06T03:48:43.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:43.130 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-07-06T03:48:43.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 -- 172.21.15.97:0/1019799009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd00806ace0 msgr2=0x7fd00806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:43.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd00806ace0 0x7fd00806d190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd01c1f5600 tx=0x7fd010008040 comp rx=0 tx=0).stop 2023-07-06T03:48:43.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 -- 172.21.15.97:0/1019799009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd01c0b1990 msgr2=0x7fd01c0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:43.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd01c0b1990 0x7fd01c0b3900 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00c550 tx=0x7fd00c00c910 comp rx=0 tx=0).stop 2023-07-06T03:48:43.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 -- 172.21.15.97:0/1019799009 shutdown_connections 2023-07-06T03:48:43.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd00806ace0 0x7fd00806d190 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd01c15c980 0x7fd01c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd01c0b1990 0x7fd01c0b3900 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 --2- 172.21.15.97:0/1019799009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd01c0d2930 0x7fd01c154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 -- 172.21.15.97:0/1019799009 >> 172.21.15.97:0/1019799009 conn(0x7fd01c06b2f0 msgr2=0x7fd01c15a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:43.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 -- 172.21.15.97:0/1019799009 shutdown_connections 2023-07-06T03:48:43.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.129+0000 7fd021bff700 1 -- 172.21.15.97:0/1019799009 wait complete. 2023-07-06T03:48:43.149 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T03:48:43.150 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T03:48:43.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.346+0000 7fa8715ff700 1 Processor -- start 2023-07-06T03:48:43.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.346+0000 7fa8715ff700 1 -- start start 2023-07-06T03:48:43.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.347+0000 7fa8715ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa86c0cfde0 0x7fa86c0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:43.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.347+0000 7fa8715ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c0b5710 con 0x7fa86c15c640 2023-07-06T03:48:43.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.347+0000 7fa8715ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c15ca10 con 0x7fa86c0b5270 2023-07-06T03:48:43.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.347+0000 7fa8715ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86c0cda40 con 0x7fa86c0cfde0 2023-07-06T03:48:43.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.347+0000 7fa86a7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa86c0cfde0 0x7fa86c0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:43.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.348+0000 7fa86a7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa86c0cfde0 0x7fa86c0cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50864/0 (socket says 172.21.15.97:50864) 2023-07-06T03:48:43.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.348+0000 7fa86a7fc700 1 -- 172.21.15.97:0/1933908971 learned_addr learned my addr 172.21.15.97:0/1933908971 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:43.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.348+0000 7fa86affd700 1 --1- 172.21.15.97:0/1933908971 >> v1:172.21.15.97:6789/0 conn(0x7fa86c15c640 0x7fa86c0cf6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54208/0 (socket says 172.21.15.97:54208) 2023-07-06T03:48:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.348+0000 7fa86a7fc700 1 -- 172.21.15.97:0/1933908971 >> v1:172.21.15.146:6789/0 conn(0x7fa86c0b5270 legacy=0x7fa86c0b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.348+0000 7fa86a7fc700 1 -- 172.21.15.97:0/1933908971 >> v1:172.21.15.97:6789/0 conn(0x7fa86c15c640 legacy=0x7fa86c0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:43.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.348+0000 7fa86a7fc700 1 -- 172.21.15.97:0/1933908971 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0b65f0 con 0x7fa86c0cfde0 2023-07-06T03:48:43.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.348+0000 7fa869ffb700 1 -- 172.21.15.97:0/1933908971 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679959908 0 0) 0x7fa86c0b5710 con 0x7fa86c15c640 2023-07-06T03:48:43.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.348+0000 7fa86a7fc700 1 --2- 172.21.15.97:0/1933908971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86c0cfde0 0x7fa86c0cd340 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa860006fd0 tx=0x7fa860009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cf3a4ba6be3ee0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:43.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.348+0000 7fa869ffb700 1 -- 172.21.15.97:0/1933908971 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860005e30 con 0x7fa86c0cfde0 2023-07-06T03:48:43.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.349+0000 7fa869ffb700 1 -- 172.21.15.97:0/1933908971 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860004030 con 0x7fa86c0cfde0 2023-07-06T03:48:43.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.349+0000 7fa869ffb700 1 -- 172.21.15.97:0/1933908971 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8600043e0 con 0x7fa86c0cfde0 2023-07-06T03:48:43.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.349+0000 7fa8715ff700 1 -- 172.21.15.97:0/1933908971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86c0cfde0 msgr2=0x7fa86c0cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:43.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.349+0000 7fa8715ff700 1 --2- 172.21.15.97:0/1933908971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86c0cfde0 0x7fa86c0cd340 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa860006fd0 tx=0x7fa860009bd0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.349+0000 7fa8715ff700 1 -- 172.21.15.97:0/1933908971 shutdown_connections 2023-07-06T03:48:43.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.349+0000 7fa8715ff700 1 --2- 172.21.15.97:0/1933908971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86c0cfde0 0x7fa86c0cd340 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.349+0000 7fa8715ff700 1 -- 172.21.15.97:0/1933908971 >> 172.21.15.97:0/1933908971 conn(0x7fa86c071790 msgr2=0x7fa86c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:43.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.349+0000 7fa8715ff700 1 -- 172.21.15.97:0/1933908971 shutdown_connections 2023-07-06T03:48:43.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.349+0000 7fa8715ff700 1 -- 172.21.15.97:0/1933908971 wait complete. 2023-07-06T03:48:43.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.351+0000 7fa8715ff700 1 Processor -- start 2023-07-06T03:48:43.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.351+0000 7fa8715ff700 1 -- start start 2023-07-06T03:48:43.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.352+0000 7fa8715ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86c0b5270 0x7fa86c14d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:43.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.352+0000 7fa8715ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa86c0cfde0 0x7fa86c155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:43.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.352+0000 7fa8715ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86c15c640 0x7fa86c155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:43.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.352+0000 7fa8715ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0d3b50 con 0x7fa86c15c640 2023-07-06T03:48:43.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.352+0000 7fa8715ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0d36c0 con 0x7fa86c0cfde0 2023-07-06T03:48:43.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.352+0000 7fa8715ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0d2ab0 con 0x7fa86c0b5270 2023-07-06T03:48:43.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.352+0000 7fa86b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86c15c640 0x7fa86c155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:43.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.352+0000 7fa86b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86c15c640 0x7fa86c155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50868/0 (socket says 172.21.15.97:50868) 2023-07-06T03:48:43.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.352+0000 7fa86b7fe700 1 -- 172.21.15.97:0/3376417574 learned_addr learned my addr 172.21.15.97:0/3376417574 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:43.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.352+0000 7fa86affd700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86c0b5270 0x7fa86c14d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:43.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.352+0000 7fa86b7fe700 1 -- 172.21.15.97:0/3376417574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86c0b5270 msgr2=0x7fa86c14d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:43.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa86b7fe700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86c0b5270 0x7fa86c14d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa86b7fe700 1 -- 172.21.15.97:0/3376417574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa86c0cfde0 msgr2=0x7fa86c155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:43.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa86a7fc700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa86c0cfde0 0x7fa86c155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:43.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa86b7fe700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa86c0cfde0 0x7fa86c155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa86b7fe700 1 -- 172.21.15.97:0/3376417574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860009500 con 0x7fa86c15c640 2023-07-06T03:48:43.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa86a7fc700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa86c0cfde0 0x7fa86c155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:43.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa86b7fe700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86c15c640 0x7fa86c155fa0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa86400cb50 tx=0x7fa86400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:43.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa8537fe700 1 -- 172.21.15.97:0/3376417574 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864003c00 con 0x7fa86c15c640 2023-07-06T03:48:43.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa8537fe700 1 -- 172.21.15.97:0/3376417574 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa864003d60 con 0x7fa86c15c640 2023-07-06T03:48:43.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa8715ff700 1 -- 172.21.15.97:0/3376417574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c0d2d90 con 0x7fa86c15c640 2023-07-06T03:48:43.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa8537fe700 1 -- 172.21.15.97:0/3376417574 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86400ab70 con 0x7fa86c15c640 2023-07-06T03:48:43.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.353+0000 7fa8715ff700 1 -- 172.21.15.97:0/3376417574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c0d3350 con 0x7fa86c15c640 2023-07-06T03:48:43.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.354+0000 7fa8715ff700 1 -- 172.21.15.97:0/3376417574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c0d3830 con 0x7fa86c15c640 2023-07-06T03:48:43.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.355+0000 7fa8537fe700 1 -- 172.21.15.97:0/3376417574 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa86400acd0 con 0x7fa86c15c640 2023-07-06T03:48:43.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.356+0000 7fa8537fe700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa85806ace0 0x7fa85806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:43.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.356+0000 7fa8537fe700 1 -- 172.21.15.97:0/3376417574 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa864089210 con 0x7fa86c15c640 2023-07-06T03:48:43.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.356+0000 7fa86affd700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa85806ace0 0x7fa85806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:43.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.357+0000 7fa86affd700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa85806ace0 0x7fa85806d190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa85c007fd0 tx=0x7fa85c007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:43.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.360+0000 7fa8537fe700 1 -- 172.21.15.97:0/3376417574 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa864054570 con 0x7fa86c15c640 2023-07-06T03:48:43.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.555+0000 7fa8715ff700 1 -- 172.21.15.97:0/3376417574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa86c0d34e0 con 0x7fa86c15c640 2023-07-06T03:48:43.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.896+0000 7fa8537fe700 1 -- 172.21.15.97:0/3376417574 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa864057b90 con 0x7fa86c15c640 2023-07-06T03:48:43.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.900+0000 7fa8715ff700 1 -- 172.21.15.97:0/3376417574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa85806ace0 msgr2=0x7fa85806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:43.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.900+0000 7fa8715ff700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa85806ace0 0x7fa85806d190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa85c007fd0 tx=0x7fa85c007c20 comp rx=0 tx=0).stop 2023-07-06T03:48:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.900+0000 7fa8715ff700 1 -- 172.21.15.97:0/3376417574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86c15c640 msgr2=0x7fa86c155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.900+0000 7fa8715ff700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86c15c640 0x7fa86c155fa0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa86400cb50 tx=0x7fa86400cf10 comp rx=0 tx=0).stop 2023-07-06T03:48:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.900+0000 7fa8715ff700 1 -- 172.21.15.97:0/3376417574 shutdown_connections 2023-07-06T03:48:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.900+0000 7fa8715ff700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa85806ace0 0x7fa85806d190 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.901+0000 7fa8715ff700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86c15c640 0x7fa86c155fa0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.901+0000 7fa8715ff700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa86c0b5270 0x7fa86c14d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.901+0000 7fa8715ff700 1 --2- 172.21.15.97:0/3376417574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa86c0cfde0 0x7fa86c155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.901+0000 7fa8715ff700 1 -- 172.21.15.97:0/3376417574 >> 172.21.15.97:0/3376417574 conn(0x7fa86c071790 msgr2=0x7fa86c0cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.901+0000 7fa8715ff700 1 -- 172.21.15.97:0/3376417574 shutdown_connections 2023-07-06T03:48:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:43.901+0000 7fa8715ff700 1 -- 172.21.15.97:0/3376417574 wait complete. 2023-07-06T03:48:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-07-06T03:48:43.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T03:48:44.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.121+0000 7fb3e5bff700 1 Processor -- start 2023-07-06T03:48:44.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.121+0000 7fb3e5bff700 1 -- start start 2023-07-06T03:48:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.122+0000 7fb3e5bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3e00b8590 0x7fb3e00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.122+0000 7fb3e5bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb3e00b4ea0 0x7fb3e00d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.122+0000 7fb3e5bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e00d0c10 con 0x7fb3e0160a50 2023-07-06T03:48:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.122+0000 7fb3e5bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e0160ef0 con 0x7fb3e00b8590 2023-07-06T03:48:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.122+0000 7fb3e5bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e00d08e0 con 0x7fb3e00b4ea0 2023-07-06T03:48:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.123+0000 7fb3df7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3e00b8590 0x7fb3e00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.123+0000 7fb3df7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3e00b8590 0x7fb3e00cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50880/0 (socket says 172.21.15.97:50880) 2023-07-06T03:48:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.123+0000 7fb3deffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb3e0160a50 0x7fb3e00b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54220/0 (socket says 172.21.15.97:54220) 2023-07-06T03:48:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.123+0000 7fb3df7fe700 1 -- 172.21.15.97:0/1181165992 learned_addr learned my addr 172.21.15.97:0/1181165992 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:44.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.123+0000 7fb3dffff700 1 --2- 172.21.15.97:0/1181165992 >> v2:172.21.15.146:3300/0 conn(0x7fb3e00b4ea0 0x7fb3e00d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:44.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.123+0000 7fb3df7fe700 1 -- 172.21.15.97:0/1181165992 >> v2:172.21.15.146:3300/0 conn(0x7fb3e00b4ea0 msgr2=0x7fb3e00d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:44.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.123+0000 7fb3df7fe700 1 --2- 172.21.15.97:0/1181165992 >> v2:172.21.15.146:3300/0 conn(0x7fb3e00b4ea0 0x7fb3e00d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:44.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.123+0000 7fb3df7fe700 1 -- 172.21.15.97:0/1181165992 >> v1:172.21.15.97:6789/0 conn(0x7fb3e0160a50 legacy=0x7fb3e00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:44.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.123+0000 7fb3df7fe700 1 -- 172.21.15.97:0/1181165992 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00b2560 con 0x7fb3e00b8590 2023-07-06T03:48:44.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.123+0000 7fb3dffff700 1 --2- 172.21.15.97:0/1181165992 >> v2:172.21.15.146:3300/0 conn(0x7fb3e00b4ea0 0x7fb3e00d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:44.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.123+0000 7fb3df7fe700 1 --2- 172.21.15.97:0/1181165992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e00b8590 0x7fb3e00cef60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb3e01610e0 tx=0x7fb3d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289e4e6cbfe4ddd3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:44.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.124+0000 7fb3de7fc700 1 -- 172.21.15.97:0/1181165992 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000b870 con 0x7fb3e00b8590 2023-07-06T03:48:44.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.124+0000 7fb3de7fc700 1 -- 172.21.15.97:0/1181165992 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d000b9d0 con 0x7fb3e00b8590 2023-07-06T03:48:44.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.124+0000 7fb3de7fc700 1 -- 172.21.15.97:0/1181165992 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000bd60 con 0x7fb3e00b8590 2023-07-06T03:48:44.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.124+0000 7fb3e5bff700 1 -- 172.21.15.97:0/1181165992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e00b8590 msgr2=0x7fb3e00cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:44.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.124+0000 7fb3e5bff700 1 --2- 172.21.15.97:0/1181165992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e00b8590 0x7fb3e00cef60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb3e01610e0 tx=0x7fb3d0009bd0 comp rx=0 tx=0).stop 2023-07-06T03:48:44.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.124+0000 7fb3e5bff700 1 -- 172.21.15.97:0/1181165992 shutdown_connections 2023-07-06T03:48:44.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.124+0000 7fb3e5bff700 1 --2- 172.21.15.97:0/1181165992 >> v2:172.21.15.146:3300/0 conn(0x7fb3e00b4ea0 0x7fb3e00d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:44.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.124+0000 7fb3e5bff700 1 --2- 172.21.15.97:0/1181165992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e00b8590 0x7fb3e00cef60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:44.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.124+0000 7fb3e5bff700 1 -- 172.21.15.97:0/1181165992 >> 172.21.15.97:0/1181165992 conn(0x7fb3e006b290 msgr2=0x7fb3e00cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:44.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.124+0000 7fb3e5bff700 1 -- 172.21.15.97:0/1181165992 shutdown_connections 2023-07-06T03:48:44.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.124+0000 7fb3e5bff700 1 -- 172.21.15.97:0/1181165992 wait complete. 2023-07-06T03:48:44.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.126+0000 7fb3e5bff700 1 Processor -- start 2023-07-06T03:48:44.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.126+0000 7fb3e5bff700 1 -- start start 2023-07-06T03:48:44.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.127+0000 7fb3e5bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3e00b4ea0 0x7fb3e0160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:44.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.127+0000 7fb3e5bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e00b8590 0x7fb3e01f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:44.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.127+0000 7fb3e5bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3e0160a50 0x7fb3e0155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:44.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.127+0000 7fb3e5bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e01608c0 con 0x7fb3e00b8590 2023-07-06T03:48:44.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.127+0000 7fb3e5bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00cab70 con 0x7fb3e0160a50 2023-07-06T03:48:44.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.127+0000 7fb3e5bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e014dc50 con 0x7fb3e00b4ea0 2023-07-06T03:48:44.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.127+0000 7fb3deffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e00b8590 0x7fb3e01f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:44.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.127+0000 7fb3deffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e00b8590 0x7fb3e01f2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50892/0 (socket says 172.21.15.97:50892) 2023-07-06T03:48:44.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3deffd700 1 -- 172.21.15.97:0/2995573950 learned_addr learned my addr 172.21.15.97:0/2995573950 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:44.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3dffff700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3e0160a50 0x7fb3e0155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3df7fe700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3e00b4ea0 0x7fb3e0160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3deffd700 1 -- 172.21.15.97:0/2995573950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3e00b4ea0 msgr2=0x7fb3e0160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3deffd700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3e00b4ea0 0x7fb3e0160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3deffd700 1 -- 172.21.15.97:0/2995573950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3e0160a50 msgr2=0x7fb3e0155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3deffd700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3e0160a50 0x7fb3e0155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3deffd700 1 -- 172.21.15.97:0/2995573950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d0009550 con 0x7fb3e00b8590 2023-07-06T03:48:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3df7fe700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3e00b4ea0 0x7fb3e0160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:44.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3deffd700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e00b8590 0x7fb3e01f2070 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400c520 tx=0x7fb3d400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:44.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3c7fff700 1 -- 172.21.15.97:0/2995573950 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4005cb0 con 0x7fb3e00b8590 2023-07-06T03:48:44.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3c7fff700 1 -- 172.21.15.97:0/2995573950 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d4005e10 con 0x7fb3e00b8590 2023-07-06T03:48:44.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3c7fff700 1 -- 172.21.15.97:0/2995573950 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40072d0 con 0x7fb3e00b8590 2023-07-06T03:48:44.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.128+0000 7fb3e5bff700 1 -- 172.21.15.97:0/2995573950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e00c9f60 con 0x7fb3e00b8590 2023-07-06T03:48:44.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.129+0000 7fb3e5bff700 1 -- 172.21.15.97:0/2995573950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e00ca520 con 0x7fb3e00b8590 2023-07-06T03:48:44.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.130+0000 7fb3e5bff700 1 -- 172.21.15.97:0/2995573950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e004e340 con 0x7fb3e00b8590 2023-07-06T03:48:44.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.131+0000 7fb3c7fff700 1 -- 172.21.15.97:0/2995573950 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb3d4007430 con 0x7fb3e00b8590 2023-07-06T03:48:44.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.132+0000 7fb3c7fff700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3cc06ace0 0x7fb3cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:44.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.132+0000 7fb3c7fff700 1 -- 172.21.15.97:0/2995573950 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fb3d4088990 con 0x7fb3e00b8590 2023-07-06T03:48:44.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.132+0000 7fb3df7fe700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3cc06ace0 0x7fb3cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:44.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.133+0000 7fb3df7fe700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3cc06ace0 0x7fb3cc06d190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0014040 tx=0x7fb3d0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:44.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.136+0000 7fb3c7fff700 1 -- 172.21.15.97:0/2995573950 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb3d4053e40 con 0x7fb3e00b8590 2023-07-06T03:48:44.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.332+0000 7fb3e5bff700 1 -- 172.21.15.97:0/2995573950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb3e00d00c0 con 0x7fb3e00b8590 2023-07-06T03:48:44.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.906+0000 7fb3c7fff700 1 -- 172.21.15.97:0/2995573950 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb3d40210d0 con 0x7fb3e00b8590 2023-07-06T03:48:44.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.915+0000 7fb3e5bff700 1 -- 172.21.15.97:0/2995573950 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3cc06ace0 msgr2=0x7fb3cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:44.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.915+0000 7fb3e5bff700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3cc06ace0 0x7fb3cc06d190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0014040 tx=0x7fb3d0008030 comp rx=0 tx=0).stop 2023-07-06T03:48:44.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.915+0000 7fb3e5bff700 1 -- 172.21.15.97:0/2995573950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e00b8590 msgr2=0x7fb3e01f2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:44.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.915+0000 7fb3e5bff700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e00b8590 0x7fb3e01f2070 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400c520 tx=0x7fb3d400c830 comp rx=0 tx=0).stop 2023-07-06T03:48:44.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.915+0000 7fb3e5bff700 1 -- 172.21.15.97:0/2995573950 shutdown_connections 2023-07-06T03:48:44.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.915+0000 7fb3e5bff700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3cc06ace0 0x7fb3cc06d190 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:44.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.915+0000 7fb3e5bff700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3e0160a50 0x7fb3e0155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:44.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.915+0000 7fb3e5bff700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3e00b4ea0 0x7fb3e0160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:44.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.915+0000 7fb3e5bff700 1 --2- 172.21.15.97:0/2995573950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e00b8590 0x7fb3e01f2070 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:44.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.915+0000 7fb3e5bff700 1 -- 172.21.15.97:0/2995573950 >> 172.21.15.97:0/2995573950 conn(0x7fb3e006b290 msgr2=0x7fb3e006c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:44.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.920+0000 7fb3e5bff700 1 -- 172.21.15.97:0/2995573950 shutdown_connections 2023-07-06T03:48:44.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:44.921+0000 7fb3e5bff700 1 -- 172.21.15.97:0/2995573950 wait complete. 2023-07-06T03:48:44.930 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-07-06T03:48:44.949 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T03:48:45.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.145+0000 7fdc0cdff700 1 Processor -- start 2023-07-06T03:48:45.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.146+0000 7fdc0cdff700 1 -- start start 2023-07-06T03:48:45.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.147+0000 7fdc0cdff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdc08160770 0x7fdc080b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:45.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.147+0000 7fdc0cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc080adb80 0x7fdc080ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:45.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.147+0000 7fdc0cdff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdc080ce590 0x7fdc080c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:45.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.147+0000 7fdc0cdff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc080b1480 con 0x7fdc080ce590 2023-07-06T03:48:45.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.147+0000 7fdc0cdff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0806c5c0 con 0x7fdc08160770 2023-07-06T03:48:45.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.147+0000 7fdc0cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0806c8e0 con 0x7fdc080adb80 2023-07-06T03:48:45.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.147+0000 7fdc06d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdc080ce590 0x7fdc080c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:45.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.147+0000 7fdc06d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdc080ce590 0x7fdc080c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50908/0 (socket says 172.21.15.97:50908) 2023-07-06T03:48:45.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.147+0000 7fdc06d9d700 1 -- 172.21.15.97:0/2764168544 learned_addr learned my addr 172.21.15.97:0/2764168544 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:45.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.147+0000 7fdc05d9b700 1 --2- 172.21.15.97:0/2764168544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc080adb80 0x7fdc080ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:45.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.147+0000 7fdc0659c700 1 --2- 172.21.15.97:0/2764168544 >> v2:172.21.15.146:3300/0 conn(0x7fdc08160770 0x7fdc080b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:45.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.148+0000 7fdc06d9d700 1 -- 172.21.15.97:0/2764168544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc080adb80 msgr2=0x7fdc080ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:45.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.148+0000 7fdc06d9d700 1 --2- 172.21.15.97:0/2764168544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc080adb80 0x7fdc080ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.148+0000 7fdc06d9d700 1 -- 172.21.15.97:0/2764168544 >> v2:172.21.15.146:3300/0 conn(0x7fdc08160770 msgr2=0x7fdc080b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:45.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.148+0000 7fdc06d9d700 1 --2- 172.21.15.97:0/2764168544 >> v2:172.21.15.146:3300/0 conn(0x7fdc08160770 0x7fdc080b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.148+0000 7fdc06d9d700 1 -- 172.21.15.97:0/2764168544 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0806cbc0 con 0x7fdc080ce590 2023-07-06T03:48:45.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.148+0000 7fdc0659c700 1 --2- 172.21.15.97:0/2764168544 >> v2:172.21.15.146:3300/0 conn(0x7fdc08160770 0x7fdc080b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:45.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.148+0000 7fdc05d9b700 1 --2- 172.21.15.97:0/2764168544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc080adb80 0x7fdc080ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:45.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.148+0000 7fdc06d9d700 1 --2- 172.21.15.97:0/2764168544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc080ce590 0x7fdc080c4a60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdc080cf9f0 tx=0x7fdc0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73944e854d0ec1e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:45.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.148+0000 7fdc0559a700 1 -- 172.21.15.97:0/2764168544 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000e070 con 0x7fdc080ce590 2023-07-06T03:48:45.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.148+0000 7fdc0559a700 1 -- 172.21.15.97:0/2764168544 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0000b9c0 con 0x7fdc080ce590 2023-07-06T03:48:45.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.148+0000 7fdc0559a700 1 -- 172.21.15.97:0/2764168544 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000bd70 con 0x7fdc080ce590 2023-07-06T03:48:45.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.149+0000 7fdc0cdff700 1 -- 172.21.15.97:0/2764168544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc080ce590 msgr2=0x7fdc080c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:45.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.149+0000 7fdc0cdff700 1 --2- 172.21.15.97:0/2764168544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc080ce590 0x7fdc080c4a60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdc080cf9f0 tx=0x7fdc0000b330 comp rx=0 tx=0).stop 2023-07-06T03:48:45.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.149+0000 7fdc0cdff700 1 -- 172.21.15.97:0/2764168544 shutdown_connections 2023-07-06T03:48:45.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.149+0000 7fdc0cdff700 1 --2- 172.21.15.97:0/2764168544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc080ce590 0x7fdc080c4a60 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.149+0000 7fdc0cdff700 1 --2- 172.21.15.97:0/2764168544 >> v2:172.21.15.146:3300/0 conn(0x7fdc08160770 0x7fdc080b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.149+0000 7fdc0cdff700 1 --2- 172.21.15.97:0/2764168544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc080adb80 0x7fdc080ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.149+0000 7fdc0cdff700 1 -- 172.21.15.97:0/2764168544 >> 172.21.15.97:0/2764168544 conn(0x7fdc0806a060 msgr2=0x7fdc0806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.149+0000 7fdc0cdff700 1 -- 172.21.15.97:0/2764168544 shutdown_connections 2023-07-06T03:48:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.149+0000 7fdc0cdff700 1 -- 172.21.15.97:0/2764168544 wait complete. 2023-07-06T03:48:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.150+0000 7fdc0cdff700 1 Processor -- start 2023-07-06T03:48:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.151+0000 7fdc0cdff700 1 -- start start 2023-07-06T03:48:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.151+0000 7fdc0cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc080adb80 0x7fdc08155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:45.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.151+0000 7fdc0cdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc080ce590 0x7fdc080b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:45.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.151+0000 7fdc0cdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08160770 0x7fdc080c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:45.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.151+0000 7fdc0cdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080b1970 con 0x7fdc08160770 2023-07-06T03:48:45.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.151+0000 7fdc0cdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080b1c50 con 0x7fdc080ce590 2023-07-06T03:48:45.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.151+0000 7fdc0cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080b1f30 con 0x7fdc080adb80 2023-07-06T03:48:45.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.152+0000 7fdc06d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08160770 0x7fdc080c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:45.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.152+0000 7fdc06d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08160770 0x7fdc080c38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50916/0 (socket says 172.21.15.97:50916) 2023-07-06T03:48:45.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.152+0000 7fdc06d9d700 1 -- 172.21.15.97:0/4201371483 learned_addr learned my addr 172.21.15.97:0/4201371483 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:45.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.152+0000 7fdc05d9b700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc080ce590 0x7fdc080b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:45.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.152+0000 7fdc0659c700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc080adb80 0x7fdc08155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.152+0000 7fdc06d9d700 1 -- 172.21.15.97:0/4201371483 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc080adb80 msgr2=0x7fdc08155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.152+0000 7fdc06d9d700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc080adb80 0x7fdc08155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.152+0000 7fdc06d9d700 1 -- 172.21.15.97:0/4201371483 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc080ce590 msgr2=0x7fdc080b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.152+0000 7fdc06d9d700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc080ce590 0x7fdc080b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.152+0000 7fdc06d9d700 1 -- 172.21.15.97:0/4201371483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc00009b30 con 0x7fdc08160770 2023-07-06T03:48:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.152+0000 7fdc06d9d700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08160770 0x7fdc080c38d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdc00012000 tx=0x7fdc00004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.153+0000 7fdbf6ffd700 1 -- 172.21.15.97:0/4201371483 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000099b0 con 0x7fdc08160770 2023-07-06T03:48:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.153+0000 7fdbf6ffd700 1 -- 172.21.15.97:0/4201371483 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc00004250 con 0x7fdc08160770 2023-07-06T03:48:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.153+0000 7fdbf6ffd700 1 -- 172.21.15.97:0/4201371483 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00004600 con 0x7fdc08160770 2023-07-06T03:48:45.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.153+0000 7fdc0cdff700 1 -- 172.21.15.97:0/4201371483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc080b2210 con 0x7fdc08160770 2023-07-06T03:48:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.153+0000 7fdc0cdff700 1 -- 172.21.15.97:0/4201371483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc080b4900 con 0x7fdc08160770 2023-07-06T03:48:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.154+0000 7fdc0cdff700 1 -- 172.21.15.97:0/4201371483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc080ce360 con 0x7fdc08160770 2023-07-06T03:48:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.155+0000 7fdbf6ffd700 1 -- 172.21.15.97:0/4201371483 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc00004760 con 0x7fdc08160770 2023-07-06T03:48:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.156+0000 7fdbf6ffd700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbe806ace0 0x7fdbe806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.156+0000 7fdbf6ffd700 1 -- 172.21.15.97:0/4201371483 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fdc0008a880 con 0x7fdc08160770 2023-07-06T03:48:45.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.157+0000 7fdc0659c700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbe806ace0 0x7fdbe806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:45.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.157+0000 7fdc0659c700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbe806ace0 0x7fdbe806d190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80099d0 tx=0x7fdbf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:45.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.161+0000 7fdbf6ffd700 1 -- 172.21.15.97:0/4201371483 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdc00055980 con 0x7fdc08160770 2023-07-06T03:48:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.399+0000 7fdc0cdff700 1 -- 172.21.15.97:0/4201371483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdc080b1ab0 con 0x7fdc08160770 2023-07-06T03:48:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.907+0000 7fdbf6ffd700 1 -- 172.21.15.97:0/4201371483 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v30) v1 ==== 173+0+0 (secure 0 0 0) 0x7fdc080b1ab0 con 0x7fdc08160770 2023-07-06T03:48:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.913+0000 7fdc0cdff700 1 -- 172.21.15.97:0/4201371483 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbe806ace0 msgr2=0x7fdbe806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.913+0000 7fdc0cdff700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbe806ace0 0x7fdbe806d190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80099d0 tx=0x7fdbf8008040 comp rx=0 tx=0).stop 2023-07-06T03:48:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.913+0000 7fdc0cdff700 1 -- 172.21.15.97:0/4201371483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08160770 msgr2=0x7fdc080c38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.913+0000 7fdc0cdff700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08160770 0x7fdc080c38d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdc00012000 tx=0x7fdc00004ad0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.913+0000 7fdc0cdff700 1 -- 172.21.15.97:0/4201371483 shutdown_connections 2023-07-06T03:48:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.913+0000 7fdc0cdff700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbe806ace0 0x7fdbe806d190 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.913+0000 7fdc0cdff700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08160770 0x7fdc080c38d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.913+0000 7fdc0cdff700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc080adb80 0x7fdc08155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.913+0000 7fdc0cdff700 1 --2- 172.21.15.97:0/4201371483 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc080ce590 0x7fdc080b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.913+0000 7fdc0cdff700 1 -- 172.21.15.97:0/4201371483 >> 172.21.15.97:0/4201371483 conn(0x7fdc0806a060 msgr2=0x7fdc080c4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.917+0000 7fdc0cdff700 1 -- 172.21.15.97:0/4201371483 shutdown_connections 2023-07-06T03:48:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:45.917+0000 7fdc0cdff700 1 -- 172.21.15.97:0/4201371483 wait complete. 2023-07-06T03:48:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 9 and data pool 10 2023-07-06T03:48:45.954 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:48:46.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.147+0000 7fe972bff700 1 Processor -- start 2023-07-06T03:48:46.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.147+0000 7fe972bff700 1 -- start start 2023-07-06T03:48:46.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.149+0000 7fe972bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe96c0cd390 0x7fe96c160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.149+0000 7fe972bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe96c160dd0 0x7fe96c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.149+0000 7fe972bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c0cdae0 con 0x7fe96c15c520 2023-07-06T03:48:46.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.149+0000 7fe972bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96c0b6640 con 0x7fe96c0cd390 2023-07-06T03:48:46.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.149+0000 7fe972bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0b6920 con 0x7fe96c160dd0 2023-07-06T03:48:46.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.149+0000 7fe97119c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe96c160dd0 0x7fe96c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.149+0000 7fe96bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe96c0cd390 0x7fe96c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.149+0000 7fe97099b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fe96c15c520 0x7fe96c160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:48970/0 (socket says 172.21.15.97:48970) 2023-07-06T03:48:46.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.149+0000 7fe97099b700 1 -- 172.21.15.97:0/3262195969 learned_addr learned my addr 172.21.15.97:0/3262195969 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.149+0000 7fe96b7fe700 1 -- 172.21.15.97:0/3262195969 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994023628 0 0) 0x7fe96c0cdae0 con 0x7fe96c15c520 2023-07-06T03:48:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.150+0000 7fe96b7fe700 1 -- 172.21.15.97:0/3262195969 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe95c0035b0 con 0x7fe96c15c520 2023-07-06T03:48:46.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.150+0000 7fe97119c700 1 -- 172.21.15.97:0/3262195969 >> v2:172.21.15.146:3300/0 conn(0x7fe96c0cd390 msgr2=0x7fe96c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:46.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.150+0000 7fe97119c700 1 --2- 172.21.15.97:0/3262195969 >> v2:172.21.15.146:3300/0 conn(0x7fe96c0cd390 0x7fe96c160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.150+0000 7fe97119c700 1 -- 172.21.15.97:0/3262195969 >> v1:172.21.15.146:6789/0 conn(0x7fe96c15c520 legacy=0x7fe96c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.150+0000 7fe97119c700 1 -- 172.21.15.97:0/3262195969 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0b6c00 con 0x7fe96c160dd0 2023-07-06T03:48:46.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.150+0000 7fe96bfff700 1 --2- 172.21.15.97:0/3262195969 >> v2:172.21.15.146:3300/0 conn(0x7fe96c0cd390 0x7fe96c160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:48:46.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.150+0000 7fe97119c700 1 --2- 172.21.15.97:0/3262195969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe96c160dd0 0x7fe96c0b7400 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe964009d90 tx=0x7fe96400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f86dd23002526b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:46.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.150+0000 7fe96b7fe700 1 -- 172.21.15.97:0/3262195969 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96400e070 con 0x7fe96c160dd0 2023-07-06T03:48:46.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.150+0000 7fe96b7fe700 1 -- 172.21.15.97:0/3262195969 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96400bb20 con 0x7fe96c160dd0 2023-07-06T03:48:46.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.150+0000 7fe96b7fe700 1 -- 172.21.15.97:0/3262195969 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964012430 con 0x7fe96c160dd0 2023-07-06T03:48:46.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.151+0000 7fe972bff700 1 -- 172.21.15.97:0/3262195969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe96c160dd0 msgr2=0x7fe96c0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.151+0000 7fe972bff700 1 --2- 172.21.15.97:0/3262195969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe96c160dd0 0x7fe96c0b7400 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe964009d90 tx=0x7fe96400b330 comp rx=0 tx=0).stop 2023-07-06T03:48:46.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.151+0000 7fe972bff700 1 -- 172.21.15.97:0/3262195969 shutdown_connections 2023-07-06T03:48:46.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.151+0000 7fe972bff700 1 --2- 172.21.15.97:0/3262195969 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe96c160dd0 0x7fe96c0b7400 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.151+0000 7fe972bff700 1 --2- 172.21.15.97:0/3262195969 >> v2:172.21.15.146:3300/0 conn(0x7fe96c0cd390 0x7fe96c160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.151+0000 7fe972bff700 1 -- 172.21.15.97:0/3262195969 >> 172.21.15.97:0/3262195969 conn(0x7fe96c071790 msgr2=0x7fe96c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:46.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.151+0000 7fe972bff700 1 -- 172.21.15.97:0/3262195969 shutdown_connections 2023-07-06T03:48:46.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.151+0000 7fe972bff700 1 -- 172.21.15.97:0/3262195969 wait complete. 2023-07-06T03:48:46.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.152+0000 7fe972bff700 1 Processor -- start 2023-07-06T03:48:46.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.153+0000 7fe972bff700 1 -- start start 2023-07-06T03:48:46.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.153+0000 7fe972bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe96c0cd390 0x7fe96c14a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.153+0000 7fe972bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe96c15c520 0x7fe96c0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe972bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe96c160dd0 0x7fe96c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe972bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c14ab30 con 0x7fe96c160dd0 2023-07-06T03:48:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe972bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0ca860 con 0x7fe96c0cd390 2023-07-06T03:48:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe972bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0cab40 con 0x7fe96c15c520 2023-07-06T03:48:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe97119c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe96c160dd0 0x7fe96c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe97119c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe96c160dd0 0x7fe96c0c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50932/0 (socket says 172.21.15.97:50932) 2023-07-06T03:48:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe97119c700 1 -- 172.21.15.97:0/4097479496 learned_addr learned my addr 172.21.15.97:0/4097479496 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:46.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe97099b700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe96c0cd390 0x7fe96c14a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe96bfff700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe96c15c520 0x7fe96c0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe97119c700 1 -- 172.21.15.97:0/4097479496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe96c15c520 msgr2=0x7fe96c0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe97119c700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe96c15c520 0x7fe96c0cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe97119c700 1 -- 172.21.15.97:0/4097479496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe96c0cd390 msgr2=0x7fe96c14a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe97119c700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe96c0cd390 0x7fe96c14a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.154+0000 7fe97119c700 1 -- 172.21.15.97:0/4097479496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580094a0 con 0x7fe96c160dd0 2023-07-06T03:48:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.155+0000 7fe97119c700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe96c160dd0 0x7fe96c0c9e10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe964014040 tx=0x7fe964004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:46.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.155+0000 7fe9697fa700 1 -- 172.21.15.97:0/4097479496 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9640129f0 con 0x7fe96c160dd0 2023-07-06T03:48:46.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.155+0000 7fe9697fa700 1 -- 172.21.15.97:0/4097479496 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe964020070 con 0x7fe96c160dd0 2023-07-06T03:48:46.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.155+0000 7fe972bff700 1 -- 172.21.15.97:0/4097479496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe964009a70 con 0x7fe96c160dd0 2023-07-06T03:48:46.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.155+0000 7fe9697fa700 1 -- 172.21.15.97:0/4097479496 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964012db0 con 0x7fe96c160dd0 2023-07-06T03:48:46.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.155+0000 7fe972bff700 1 -- 172.21.15.97:0/4097479496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c0cb100 con 0x7fe96c160dd0 2023-07-06T03:48:46.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.156+0000 7fe972bff700 1 -- 172.21.15.97:0/4097479496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96c04e340 con 0x7fe96c160dd0 2023-07-06T03:48:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.157+0000 7fe9697fa700 1 -- 172.21.15.97:0/4097479496 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe964004040 con 0x7fe96c160dd0 2023-07-06T03:48:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.158+0000 7fe9697fa700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe95006ad10 0x7fe95006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.158+0000 7fe9697fa700 1 -- 172.21.15.97:0/4097479496 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe964022070 con 0x7fe96c160dd0 2023-07-06T03:48:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.158+0000 7fe97099b700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe95006ad10 0x7fe95006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.159+0000 7fe97099b700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe95006ad10 0x7fe95006d1c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe9600011e0 tx=0x7fe960009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.162+0000 7fe9697fa700 1 -- 172.21.15.97:0/4097479496 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe964056220 con 0x7fe96c160dd0 2023-07-06T03:48:46.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.357+0000 7fe972bff700 1 -- 172.21.15.97:0/4097479496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe96c0b4f40 con 0x7fe96c160dd0 2023-07-06T03:48:46.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.359+0000 7fe9697fa700 1 -- 172.21.15.97:0/4097479496 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18380 (secure 0 0 0) 0x7fe96402c090 con 0x7fe96c160dd0 2023-07-06T03:48:46.361 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:46.361 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":59,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:48:45.400704+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:48:43.594325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-06T03:48:44.371987+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:48:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 -- 172.21.15.97:0/4097479496 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe95006ad10 msgr2=0x7fe95006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe95006ad10 0x7fe95006d1c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe9600011e0 tx=0x7fe960009040 comp rx=0 tx=0).stop 2023-07-06T03:48:46.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 -- 172.21.15.97:0/4097479496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe96c160dd0 msgr2=0x7fe96c0c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe96c160dd0 0x7fe96c0c9e10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe964014040 tx=0x7fe964004ad0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 -- 172.21.15.97:0/4097479496 shutdown_connections 2023-07-06T03:48:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe95006ad10 0x7fe95006d1c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe96c160dd0 0x7fe96c0c9e10 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe96c0cd390 0x7fe96c14a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 --2- 172.21.15.97:0/4097479496 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe96c15c520 0x7fe96c0cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 -- 172.21.15.97:0/4097479496 >> 172.21.15.97:0/4097479496 conn(0x7fe96c071790 msgr2=0x7fe96c0b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:46.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 -- 172.21.15.97:0/4097479496 shutdown_connections 2023-07-06T03:48:46.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.361+0000 7fe972bff700 1 -- 172.21.15.97:0/4097479496 wait complete. 2023-07-06T03:48:46.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:48:46.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.574+0000 7f7ef47ff700 1 Processor -- start 2023-07-06T03:48:46.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.574+0000 7f7ef47ff700 1 -- start start 2023-07-06T03:48:46.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef47ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7eec0cd390 0x7f7eec160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef47ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7eec160dd0 0x7f7eec0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef47ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eec0cdae0 con 0x7f7eec15c520 2023-07-06T03:48:46.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef47ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec0b6640 con 0x7f7eec160dd0 2023-07-06T03:48:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef47ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec0b6920 con 0x7f7eec0cd390 2023-07-06T03:48:46.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef2d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7eec160dd0 0x7f7eec0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef2d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7eec160dd0 0x7f7eec0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50934/0 (socket says 172.21.15.97:50934) 2023-07-06T03:48:46.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef2d9c700 1 -- 172.21.15.97:0/3746941735 learned_addr learned my addr 172.21.15.97:0/3746941735 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:46.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef1d9a700 1 --2- 172.21.15.97:0/3746941735 >> v2:172.21.15.146:3300/0 conn(0x7f7eec0cd390 0x7f7eec160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef2d9c700 1 -- 172.21.15.97:0/3746941735 >> v2:172.21.15.146:3300/0 conn(0x7f7eec0cd390 msgr2=0x7f7eec160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef2d9c700 1 --2- 172.21.15.97:0/3746941735 >> v2:172.21.15.146:3300/0 conn(0x7f7eec0cd390 0x7f7eec160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef2d9c700 1 -- 172.21.15.97:0/3746941735 >> v1:172.21.15.146:6789/0 conn(0x7f7eec15c520 legacy=0x7f7eec160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.576+0000 7f7ef2d9c700 1 -- 172.21.15.97:0/3746941735 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec0b6c00 con 0x7f7eec160dd0 2023-07-06T03:48:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef2d9c700 1 --2- 172.21.15.97:0/3746941735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7eec160dd0 0x7f7eec0b7400 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0b5930 tx=0x7f7ee400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c821274fd0b66940 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef1599700 1 -- 172.21.15.97:0/3746941735 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400e070 con 0x7f7eec160dd0 2023-07-06T03:48:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef1599700 1 -- 172.21.15.97:0/3746941735 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee400b9c0 con 0x7f7eec160dd0 2023-07-06T03:48:46.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef1599700 1 -- 172.21.15.97:0/3746941735 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400bd70 con 0x7f7eec160dd0 2023-07-06T03:48:46.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3746941735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7eec160dd0 msgr2=0x7f7eec0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef47ff700 1 --2- 172.21.15.97:0/3746941735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7eec160dd0 0x7f7eec0b7400 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0b5930 tx=0x7f7ee400b330 comp rx=0 tx=0).stop 2023-07-06T03:48:46.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3746941735 shutdown_connections 2023-07-06T03:48:46.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef47ff700 1 --2- 172.21.15.97:0/3746941735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7eec160dd0 0x7f7eec0b7400 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef47ff700 1 --2- 172.21.15.97:0/3746941735 >> v2:172.21.15.146:3300/0 conn(0x7f7eec0cd390 0x7f7eec160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3746941735 >> 172.21.15.97:0/3746941735 conn(0x7f7eec071790 msgr2=0x7f7eec071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:46.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3746941735 shutdown_connections 2023-07-06T03:48:46.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.577+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3746941735 wait complete. 2023-07-06T03:48:46.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.579+0000 7f7ef47ff700 1 Processor -- start 2023-07-06T03:48:46.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.579+0000 7f7ef47ff700 1 -- start start 2023-07-06T03:48:46.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.580+0000 7f7ef47ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eec0cd390 0x7f7eec0cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.580+0000 7f7ef47ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7eec15c520 0x7f7eec158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.580+0000 7f7ef47ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7eec160dd0 0x7f7eec155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.580+0000 7f7ef47ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec0cb7f0 con 0x7f7eec15c520 2023-07-06T03:48:46.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.580+0000 7f7ef47ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec0cb590 con 0x7f7eec160dd0 2023-07-06T03:48:46.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.580+0000 7f7ef47ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec0d2350 con 0x7f7eec0cd390 2023-07-06T03:48:46.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.580+0000 7f7ef1d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7eec15c520 0x7f7eec158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.580+0000 7f7ef1d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7eec15c520 0x7f7eec158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50936/0 (socket says 172.21.15.97:50936) 2023-07-06T03:48:46.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.580+0000 7f7ef1d9a700 1 -- 172.21.15.97:0/3551671341 learned_addr learned my addr 172.21.15.97:0/3551671341 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:46.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ef2d9c700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7eec160dd0 0x7f7eec155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ef259b700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eec0cd390 0x7f7eec0cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ef1d9a700 1 -- 172.21.15.97:0/3551671341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eec0cd390 msgr2=0x7f7eec0cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ef1d9a700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eec0cd390 0x7f7eec0cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ef1d9a700 1 -- 172.21.15.97:0/3551671341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7eec160dd0 msgr2=0x7f7eec155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ef1d9a700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7eec160dd0 0x7f7eec155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ef1d9a700 1 -- 172.21.15.97:0/3551671341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee4009b30 con 0x7f7eec15c520 2023-07-06T03:48:46.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ef1d9a700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7eec15c520 0x7f7eec158190 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00d800 tx=0x7f7edc00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:46.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ee2ffd700 1 -- 172.21.15.97:0/3551671341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc0125f0 con 0x7f7eec15c520 2023-07-06T03:48:46.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ee2ffd700 1 -- 172.21.15.97:0/3551671341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc012750 con 0x7f7eec15c520 2023-07-06T03:48:46.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3551671341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec0d2630 con 0x7f7eec15c520 2023-07-06T03:48:46.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ee2ffd700 1 -- 172.21.15.97:0/3551671341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc012b00 con 0x7f7eec15c520 2023-07-06T03:48:46.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.581+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3551671341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec0d2bf0 con 0x7f7eec15c520 2023-07-06T03:48:46.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.582+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3551671341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eec0d2db0 con 0x7f7eec15c520 2023-07-06T03:48:46.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.583+0000 7f7ee2ffd700 1 -- 172.21.15.97:0/3551671341 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7edc012c60 con 0x7f7eec15c520 2023-07-06T03:48:46.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.584+0000 7f7ee2ffd700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ed006ace0 0x7f7ed006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:46.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.584+0000 7f7ee2ffd700 1 -- 172.21.15.97:0/3551671341 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7edc0897f0 con 0x7f7eec15c520 2023-07-06T03:48:46.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.585+0000 7f7ef259b700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ed006ace0 0x7f7ed006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:46.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.585+0000 7f7ef259b700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ed006ace0 0x7f7ed006d190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8000f60 tx=0x7f7ee8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:46.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.589+0000 7f7ee2ffd700 1 -- 172.21.15.97:0/3551671341 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7edc054890 con 0x7f7eec15c520 2023-07-06T03:48:46.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.784+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3551671341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7eec0d2f60 con 0x7f7eec15c520 2023-07-06T03:48:46.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.785+0000 7f7ee2ffd700 1 -- 172.21.15.97:0/3551671341 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18380 (secure 0 0 0) 0x7f7edc057eb0 con 0x7f7eec15c520 2023-07-06T03:48:46.787 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:46.787 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":59,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:48:45.400704+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:48:43.594325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-06T03:48:44.371987+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:48:46.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.787+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3551671341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ed006ace0 msgr2=0x7f7ed006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.787+0000 7f7ef47ff700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ed006ace0 0x7f7ed006d190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8000f60 tx=0x7f7ee8009040 comp rx=0 tx=0).stop 2023-07-06T03:48:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.787+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3551671341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7eec15c520 msgr2=0x7f7eec158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.787+0000 7f7ef47ff700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7eec15c520 0x7f7eec158190 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00d800 tx=0x7f7edc00db10 comp rx=0 tx=0).stop 2023-07-06T03:48:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.788+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3551671341 shutdown_connections 2023-07-06T03:48:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.788+0000 7f7ef47ff700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ed006ace0 0x7f7ed006d190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.788+0000 7f7ef47ff700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7eec160dd0 0x7f7eec155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.788+0000 7f7ef47ff700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7eec0cd390 0x7f7eec0cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.788+0000 7f7ef47ff700 1 --2- 172.21.15.97:0/3551671341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7eec15c520 0x7f7eec158190 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.788+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3551671341 >> 172.21.15.97:0/3551671341 conn(0x7f7eec071790 msgr2=0x7f7eec1605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:46.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.788+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3551671341 shutdown_connections 2023-07-06T03:48:46.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:46.788+0000 7f7ef47ff700 1 -- 172.21.15.97:0/3551671341 wait complete. 2023-07-06T03:48:46.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T03:48:47.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.008+0000 7fe4589ff700 1 Processor -- start 2023-07-06T03:48:47.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.008+0000 7fe4589ff700 1 -- start start 2023-07-06T03:48:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.010+0000 7fe4589ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b0560 0x7fe4540b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.010+0000 7fe4589ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4540adcf0 con 0x7fe454158b20 2023-07-06T03:48:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.010+0000 7fe4589ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4540adfd0 con 0x7fe45406cab0 2023-07-06T03:48:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.010+0000 7fe4589ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540ce820 con 0x7fe4540b0560 2023-07-06T03:48:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.010+0000 7fe45259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe454158b20 0x7fe4540cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54232/0 (socket says 172.21.15.97:54232) 2023-07-06T03:48:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.010+0000 7fe45259c700 1 -- 172.21.15.97:0/4069354055 learned_addr learned my addr 172.21.15.97:0/4069354055 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:47.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.010+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069354350 0 0) 0x7fe4540adcf0 con 0x7fe454158b20 2023-07-06T03:48:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.010+0000 7fe452d9d700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b0560 0x7fe4540b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.010+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe43c0034d0 con 0x7fe454158b20 2023-07-06T03:48:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 979916581 0 0) 0x7fe43c0034d0 con 0x7fe454158b20 2023-07-06T03:48:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b0560 msgr2=0x7fe4540b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b0560 0x7fe4540b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 >> v1:172.21.15.146:6789/0 conn(0x7fe45406cab0 legacy=0x7fe4540ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4540ae2b0 con 0x7fe454158b20 2023-07-06T03:48:47.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fe444001670 con 0x7fe454158b20 2023-07-06T03:48:47.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 >> v1:172.21.15.97:6789/0 conn(0x7fe454158b20 legacy=0x7fe4540cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:47.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b0560 0x7fe43c003b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:47.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe43c004060 0x7fe43c00a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:47.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c00afc0 0x7fe43c00d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:47.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540ce820 con 0x7fe43c00afc0 2023-07-06T03:48:47.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c00dc60 con 0x7fe43c004060 2023-07-06T03:48:47.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c00de20 con 0x7fe4540b0560 2023-07-06T03:48:47.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe452d9d700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c00afc0 0x7fe43c00d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:47.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe451d9b700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe43c004060 0x7fe43c00a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:47.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45259c700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b0560 0x7fe43c003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:47.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe452d9d700 1 -- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b0560 msgr2=0x7fe43c003b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:47.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe452d9d700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b0560 0x7fe43c003b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:47.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe452d9d700 1 -- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe43c004060 msgr2=0x7fe43c00a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:47.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe452d9d700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe43c004060 0x7fe43c00a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:47.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe452d9d700 1 -- 172.21.15.97:0/4069354055 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c0034d0 con 0x7fe43c00afc0 2023-07-06T03:48:47.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.011+0000 7fe45259c700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b0560 0x7fe43c003b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:47.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.012+0000 7fe452d9d700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c00afc0 0x7fe43c00d420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009fc0 tx=0x7fe43c003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edd210913d3ca8f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:47.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.012+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00ec30 con 0x7fe43c00afc0 2023-07-06T03:48:47.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.012+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c00f040 con 0x7fe43c00afc0 2023-07-06T03:48:47.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.012+0000 7fe45159a700 1 -- 172.21.15.97:0/4069354055 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c014480 con 0x7fe43c00afc0 2023-07-06T03:48:47.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.012+0000 7fe4589ff700 1 -- 172.21.15.97:0/4069354055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c00afc0 msgr2=0x7fe43c00d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:47.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.012+0000 7fe4589ff700 1 --2- 172.21.15.97:0/4069354055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c00afc0 0x7fe43c00d420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009fc0 tx=0x7fe43c003780 comp rx=0 tx=0).stop 2023-07-06T03:48:47.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.012+0000 7fe452d9d700 1 -- 172.21.15.97:0/4069354055 reap_dead start 2023-07-06T03:48:47.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.013+0000 7fe4589ff700 1 -- 172.21.15.97:0/4069354055 shutdown_connections 2023-07-06T03:48:47.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.013+0000 7fe4589ff700 1 -- 172.21.15.97:0/4069354055 >> 172.21.15.97:0/4069354055 conn(0x7fe45406a060 msgr2=0x7fe45406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:47.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.013+0000 7fe4589ff700 1 -- 172.21.15.97:0/4069354055 shutdown_connections 2023-07-06T03:48:47.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.013+0000 7fe4589ff700 1 -- 172.21.15.97:0/4069354055 wait complete. 2023-07-06T03:48:47.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.014+0000 7fe4589ff700 1 Processor -- start 2023-07-06T03:48:47.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.014+0000 7fe4589ff700 1 -- start start 2023-07-06T03:48:47.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.015+0000 7fe4589ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe43c00afc0 0x7fe45415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.015+0000 7fe4589ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b1a20 0x7fe4540b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.015+0000 7fe4589ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45415b850 0x7fe4540b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.015+0000 7fe4589ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45415bdc0 con 0x7fe45415b850 2023-07-06T03:48:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.015+0000 7fe4589ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540b3f80 con 0x7fe43c00afc0 2023-07-06T03:48:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.015+0000 7fe4589ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540b4260 con 0x7fe4540b1a20 2023-07-06T03:48:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe452d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45415b850 0x7fe4540b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe452d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45415b850 0x7fe4540b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50952/0 (socket says 172.21.15.97:50952) 2023-07-06T03:48:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe452d9d700 1 -- 172.21.15.97:0/3312024140 learned_addr learned my addr 172.21.15.97:0/3312024140 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe45259c700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe43c00afc0 0x7fe45415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe451d9b700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b1a20 0x7fe4540b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe452d9d700 1 -- 172.21.15.97:0/3312024140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b1a20 msgr2=0x7fe4540b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe452d9d700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b1a20 0x7fe4540b3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe452d9d700 1 -- 172.21.15.97:0/3312024140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe43c00afc0 msgr2=0x7fe45415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe452d9d700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe43c00afc0 0x7fe45415afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe452d9d700 1 -- 172.21.15.97:0/3312024140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45415b500 con 0x7fe45415b850 2023-07-06T03:48:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe45259c700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe43c00afc0 0x7fe45415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe452d9d700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45415b850 0x7fe4540b35b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009f60 tx=0x7fe44c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.016+0000 7fe442ffd700 1 -- 172.21.15.97:0/3312024140 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c007000 con 0x7fe45415b850 2023-07-06T03:48:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.017+0000 7fe442ffd700 1 -- 172.21.15.97:0/3312024140 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c014cf0 con 0x7fe45415b850 2023-07-06T03:48:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.017+0000 7fe442ffd700 1 -- 172.21.15.97:0/3312024140 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c014ea0 con 0x7fe45415b850 2023-07-06T03:48:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.017+0000 7fe4589ff700 1 -- 172.21.15.97:0/3312024140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45415b690 con 0x7fe45415b850 2023-07-06T03:48:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.017+0000 7fe4589ff700 1 -- 172.21.15.97:0/3312024140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4540b3cd0 con 0x7fe45415b850 2023-07-06T03:48:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.018+0000 7fe4589ff700 1 -- 172.21.15.97:0/3312024140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4540b4420 con 0x7fe45415b850 2023-07-06T03:48:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.019+0000 7fe442ffd700 1 -- 172.21.15.97:0/3312024140 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe44c023020 con 0x7fe45415b850 2023-07-06T03:48:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.019+0000 7fe442ffd700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe43406ace0 0x7fe43406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.019+0000 7fe442ffd700 1 -- 172.21.15.97:0/3312024140 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe44c08ab50 con 0x7fe45415b850 2023-07-06T03:48:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.020+0000 7fe45259c700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe43406ace0 0x7fe43406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:47.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.020+0000 7fe45259c700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe43406ace0 0x7fe43406d190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe444000ff0 tx=0x7fe444009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:47.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.024+0000 7fe442ffd700 1 -- 172.21.15.97:0/3312024140 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe44c055d10 con 0x7fe45415b850 2023-07-06T03:48:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.265+0000 7fe4589ff700 1 -- 172.21.15.97:0/3312024140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe454067320 con 0x7fe45415b850 2023-07-06T03:48:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.931+0000 7fe442ffd700 1 -- 172.21.15.97:0/3312024140 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe44c059330 con 0x7fe45415b850 2023-07-06T03:48:47.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.934+0000 7fe4589ff700 1 -- 172.21.15.97:0/3312024140 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe43406ace0 msgr2=0x7fe43406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:47.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.934+0000 7fe4589ff700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe43406ace0 0x7fe43406d190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe444000ff0 tx=0x7fe444009040 comp rx=0 tx=0).stop 2023-07-06T03:48:47.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.934+0000 7fe4589ff700 1 -- 172.21.15.97:0/3312024140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45415b850 msgr2=0x7fe4540b35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:47.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.934+0000 7fe4589ff700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45415b850 0x7fe4540b35b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009f60 tx=0x7fe44c009750 comp rx=0 tx=0).stop 2023-07-06T03:48:47.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.934+0000 7fe4589ff700 1 -- 172.21.15.97:0/3312024140 shutdown_connections 2023-07-06T03:48:47.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.935+0000 7fe4589ff700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe43406ace0 0x7fe43406d190 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:47.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.935+0000 7fe4589ff700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45415b850 0x7fe4540b35b0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:47.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.935+0000 7fe4589ff700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe43c00afc0 0x7fe45415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:47.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.935+0000 7fe4589ff700 1 --2- 172.21.15.97:0/3312024140 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe4540b1a20 0x7fe4540b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.935+0000 7fe4589ff700 1 -- 172.21.15.97:0/3312024140 >> 172.21.15.97:0/3312024140 conn(0x7fe45406a060 msgr2=0x7fe45406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.935+0000 7fe4589ff700 1 -- 172.21.15.97:0/3312024140 shutdown_connections 2023-07-06T03:48:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:47.935+0000 7fe4589ff700 1 -- 172.21.15.97:0/3312024140 wait complete. 2023-07-06T03:48:47.957 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T03:48:47.958 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T03:48:48.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.156+0000 7f67043ff700 1 Processor -- start 2023-07-06T03:48:48.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.156+0000 7f67043ff700 1 -- start start 2023-07-06T03:48:48.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.157+0000 7f67043ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f66fc06b360 0x7f66fc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:48.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.157+0000 7f67043ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f66fc0d2210 0x7f66fc0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:48.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f67043ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66fc0cff90 con 0x7f66fc15c610 2023-07-06T03:48:48.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f67043ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66fc0d0780 con 0x7f66fc0d2210 2023-07-06T03:48:48.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f67043ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66fc15c9e0 con 0x7f66fc06b360 2023-07-06T03:48:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f670299c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f66fc0d2210 0x7f66fc0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f670299c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f66fc0d2210 0x7f66fc0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50962/0 (socket says 172.21.15.97:50962) 2023-07-06T03:48:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f670299c700 1 -- 172.21.15.97:0/2445644599 learned_addr learned my addr 172.21.15.97:0/2445644599 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f670199a700 1 --2- 172.21.15.97:0/2445644599 >> v2:172.21.15.146:3300/0 conn(0x7f66fc06b360 0x7f66fc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f670299c700 1 -- 172.21.15.97:0/2445644599 >> v2:172.21.15.146:3300/0 conn(0x7f66fc06b360 msgr2=0x7f66fc0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f670299c700 1 --2- 172.21.15.97:0/2445644599 >> v2:172.21.15.146:3300/0 conn(0x7f66fc06b360 0x7f66fc0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f670299c700 1 -- 172.21.15.97:0/2445644599 >> v1:172.21.15.146:6789/0 conn(0x7f66fc15c610 legacy=0x7f66fc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:48.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f670299c700 1 -- 172.21.15.97:0/2445644599 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc0b5650 con 0x7f66fc0d2210 2023-07-06T03:48:48.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.158+0000 7f670299c700 1 --2- 172.21.15.97:0/2445644599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66fc0d2210 0x7f66fc0d1310 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f66fc06a590 tx=0x7f66f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02a5d5cfa0803b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:48.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.159+0000 7f6701199700 1 -- 172.21.15.97:0/2445644599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f400e070 con 0x7f66fc0d2210 2023-07-06T03:48:48.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.159+0000 7f6701199700 1 -- 172.21.15.97:0/2445644599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f4003a20 con 0x7f66fc0d2210 2023-07-06T03:48:48.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.159+0000 7f6701199700 1 -- 172.21.15.97:0/2445644599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4003dd0 con 0x7f66fc0d2210 2023-07-06T03:48:48.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.159+0000 7f67043ff700 1 -- 172.21.15.97:0/2445644599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66fc0d2210 msgr2=0x7f66fc0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:48.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.159+0000 7f67043ff700 1 --2- 172.21.15.97:0/2445644599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66fc0d2210 0x7f66fc0d1310 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f66fc06a590 tx=0x7f66f400b330 comp rx=0 tx=0).stop 2023-07-06T03:48:48.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.159+0000 7f67043ff700 1 -- 172.21.15.97:0/2445644599 shutdown_connections 2023-07-06T03:48:48.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.159+0000 7f67043ff700 1 --2- 172.21.15.97:0/2445644599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66fc0d2210 0x7f66fc0d1310 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:48.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.159+0000 7f67043ff700 1 --2- 172.21.15.97:0/2445644599 >> v2:172.21.15.146:3300/0 conn(0x7f66fc06b360 0x7f66fc0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:48.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.159+0000 7f67043ff700 1 -- 172.21.15.97:0/2445644599 >> 172.21.15.97:0/2445644599 conn(0x7f66fc06a6c0 msgr2=0x7f66fc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:48.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.159+0000 7f67043ff700 1 -- 172.21.15.97:0/2445644599 shutdown_connections 2023-07-06T03:48:48.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.159+0000 7f67043ff700 1 -- 172.21.15.97:0/2445644599 wait complete. 2023-07-06T03:48:48.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.161+0000 7f67043ff700 1 Processor -- start 2023-07-06T03:48:48.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.161+0000 7f67043ff700 1 -- start start 2023-07-06T03:48:48.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.162+0000 7f67043ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66fc06b360 0x7f66fc148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:48.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.162+0000 7f67043ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66fc0d2210 0x7f66fc0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:48.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.162+0000 7f67043ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66fc15c610 0x7f66fc1f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:48.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.162+0000 7f67043ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0cb390 con 0x7f66fc15c610 2023-07-06T03:48:48.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.162+0000 7f67043ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0cb580 con 0x7f66fc0d2210 2023-07-06T03:48:48.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.162+0000 7f67043ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0cb860 con 0x7f66fc06b360 2023-07-06T03:48:48.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.162+0000 7f670299c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66fc15c610 0x7f66fc1f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:48.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.162+0000 7f670299c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66fc15c610 0x7f66fc1f1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50968/0 (socket says 172.21.15.97:50968) 2023-07-06T03:48:48.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.162+0000 7f670299c700 1 -- 172.21.15.97:0/3671582501 learned_addr learned my addr 172.21.15.97:0/3671582501 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:48.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f670199a700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66fc0d2210 0x7f66fc0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:48.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f670219b700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66fc06b360 0x7f66fc148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:48.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f670299c700 1 -- 172.21.15.97:0/3671582501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66fc06b360 msgr2=0x7f66fc148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f670299c700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66fc06b360 0x7f66fc148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f670299c700 1 -- 172.21.15.97:0/3671582501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66fc0d2210 msgr2=0x7f66fc0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:48.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f670299c700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66fc0d2210 0x7f66fc0cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:48.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f670299c700 1 -- 172.21.15.97:0/3671582501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4009b30 con 0x7f66fc15c610 2023-07-06T03:48:48.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f670219b700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66fc06b360 0x7f66fc148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:48.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f670299c700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66fc15c610 0x7f66fc1f1a40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f66fc08b120 tx=0x7f66f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:48.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f66f2ffd700 1 -- 172.21.15.97:0/3671582501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4004440 con 0x7f66fc15c610 2023-07-06T03:48:48.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f66f2ffd700 1 -- 172.21.15.97:0/3671582501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f40045c0 con 0x7f66fc15c610 2023-07-06T03:48:48.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f66f2ffd700 1 -- 172.21.15.97:0/3671582501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4004970 con 0x7f66fc15c610 2023-07-06T03:48:48.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.163+0000 7f67043ff700 1 -- 172.21.15.97:0/3671582501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc0cbb40 con 0x7f66fc15c610 2023-07-06T03:48:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.164+0000 7f67043ff700 1 -- 172.21.15.97:0/3671582501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc0cc100 con 0x7f66fc15c610 2023-07-06T03:48:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.164+0000 7f67043ff700 1 -- 172.21.15.97:0/3671582501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc066fe0 con 0x7f66fc15c610 2023-07-06T03:48:48.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.165+0000 7f66f2ffd700 1 -- 172.21.15.97:0/3671582501 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f66f4012430 con 0x7f66fc15c610 2023-07-06T03:48:48.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.166+0000 7f66f2ffd700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f66e006ace0 0x7f66e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:48.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.166+0000 7f66f2ffd700 1 -- 172.21.15.97:0/3671582501 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f66f408a9d0 con 0x7f66fc15c610 2023-07-06T03:48:48.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.167+0000 7f670219b700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f66e006ace0 0x7f66e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:48.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.167+0000 7f670219b700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f66e006ace0 0x7f66e006d190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009930 tx=0x7f66f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:48.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.171+0000 7f66f2ffd700 1 -- 172.21.15.97:0/3671582501 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f66f4059480 con 0x7f66fc15c610 2023-07-06T03:48:48.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.410+0000 7f67043ff700 1 -- 172.21.15.97:0/3671582501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f66fc0cc2f0 con 0x7f66fc15c610 2023-07-06T03:48:48.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.933+0000 7f66f2ffd700 1 -- 172.21.15.97:0/3671582501 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f66f4059010 con 0x7f66fc15c610 2023-07-06T03:48:48.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.936+0000 7f67043ff700 1 -- 172.21.15.97:0/3671582501 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f66e006ace0 msgr2=0x7f66e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:48.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.936+0000 7f67043ff700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f66e006ace0 0x7f66e006d190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009930 tx=0x7f66f8008040 comp rx=0 tx=0).stop 2023-07-06T03:48:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.936+0000 7f67043ff700 1 -- 172.21.15.97:0/3671582501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66fc15c610 msgr2=0x7f66fc1f1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.936+0000 7f67043ff700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66fc15c610 0x7f66fc1f1a40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f66fc08b120 tx=0x7f66f4004ad0 comp rx=0 tx=0).stop 2023-07-06T03:48:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.936+0000 7f67043ff700 1 -- 172.21.15.97:0/3671582501 shutdown_connections 2023-07-06T03:48:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.936+0000 7f67043ff700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f66e006ace0 0x7f66e006d190 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.936+0000 7f67043ff700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66fc15c610 0x7f66fc1f1a40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.936+0000 7f67043ff700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f66fc06b360 0x7f66fc148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.936+0000 7f67043ff700 1 --2- 172.21.15.97:0/3671582501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66fc0d2210 0x7f66fc0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.936+0000 7f67043ff700 1 -- 172.21.15.97:0/3671582501 >> 172.21.15.97:0/3671582501 conn(0x7f66fc06a6c0 msgr2=0x7f66fc15b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:48.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.937+0000 7f67043ff700 1 -- 172.21.15.97:0/3671582501 shutdown_connections 2023-07-06T03:48:48.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:48.937+0000 7f67043ff700 1 -- 172.21.15.97:0/3671582501 wait complete. 2023-07-06T03:48:48.957 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T03:48:48.958 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:48:49.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.152+0000 7f41e6dff700 1 Processor -- start 2023-07-06T03:48:49.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.152+0000 7f41e6dff700 1 -- start start 2023-07-06T03:48:49.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.153+0000 7f41e6dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41e015c520 0x7f41e00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.153+0000 7f41e6dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f41e0160180 0x7f41e01605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.153+0000 7f41e6dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e00cdae0 con 0x7f41e0160b30 2023-07-06T03:48:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.153+0000 7f41e6dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e00cd8d0 con 0x7f41e0160180 2023-07-06T03:48:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.153+0000 7f41e6dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00b6640 con 0x7f41e015c520 2023-07-06T03:48:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.154+0000 7f41dffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f41e0160180 0x7f41e01605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.154+0000 7f41e4b9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41e015c520 0x7f41e00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.154+0000 7f41dffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f41e0160180 0x7f41e01605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:41550/0 (socket says 172.21.15.97:41550) 2023-07-06T03:48:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.154+0000 7f41dffff700 1 -- 172.21.15.97:0/1894410782 learned_addr learned my addr 172.21.15.97:0/1894410782 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.154+0000 7f41dffff700 1 -- 172.21.15.97:0/1894410782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41e015c520 msgr2=0x7f41e00cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.154+0000 7f41dffff700 1 --2- 172.21.15.97:0/1894410782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41e015c520 0x7f41e00cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.154+0000 7f41dffff700 1 -- 172.21.15.97:0/1894410782 >> v1:172.21.15.146:6789/0 conn(0x7f41e0160b30 legacy=0x7f41e00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.154+0000 7f41dffff700 1 -- 172.21.15.97:0/1894410782 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00b6920 con 0x7f41e0160180 2023-07-06T03:48:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.154+0000 7f41e4b9b700 1 --2- 172.21.15.97:0/1894410782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41e015c520 0x7f41e00cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:48:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.154+0000 7f41df7fe700 1 -- 172.21.15.97:0/1894410782 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596477951 0 0) 0x7f41e00cdae0 con 0x7f41e0160b30 2023-07-06T03:48:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.154+0000 7f41dffff700 1 --2- 172.21.15.97:0/1894410782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e0160180 0x7f41e01605f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0098f0 tx=0x7f41cc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa00dd66358f45b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.155+0000 7f41df7fe700 1 -- 172.21.15.97:0/1894410782 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc011830 con 0x7f41e0160180 2023-07-06T03:48:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.155+0000 7f41df7fe700 1 -- 172.21.15.97:0/1894410782 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc011990 con 0x7f41e0160180 2023-07-06T03:48:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.155+0000 7f41df7fe700 1 -- 172.21.15.97:0/1894410782 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc011d20 con 0x7f41e0160180 2023-07-06T03:48:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.155+0000 7f41e6dff700 1 -- 172.21.15.97:0/1894410782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e0160180 msgr2=0x7f41e01605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.155+0000 7f41e6dff700 1 --2- 172.21.15.97:0/1894410782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e0160180 0x7f41e01605f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0098f0 tx=0x7f41cc009c00 comp rx=0 tx=0).stop 2023-07-06T03:48:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.155+0000 7f41e6dff700 1 -- 172.21.15.97:0/1894410782 shutdown_connections 2023-07-06T03:48:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.155+0000 7f41e6dff700 1 --2- 172.21.15.97:0/1894410782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41e015c520 0x7f41e00cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.155+0000 7f41e6dff700 1 --2- 172.21.15.97:0/1894410782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e0160180 0x7f41e01605f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.155+0000 7f41e6dff700 1 -- 172.21.15.97:0/1894410782 >> 172.21.15.97:0/1894410782 conn(0x7f41e0071790 msgr2=0x7f41e0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.155+0000 7f41e6dff700 1 -- 172.21.15.97:0/1894410782 shutdown_connections 2023-07-06T03:48:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.155+0000 7f41e6dff700 1 -- 172.21.15.97:0/1894410782 wait complete. 2023-07-06T03:48:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.157+0000 7f41e6dff700 1 Processor -- start 2023-07-06T03:48:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.157+0000 7f41e6dff700 1 -- start start 2023-07-06T03:48:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.158+0000 7f41e6dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41e015c520 0x7f41e01f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.158+0000 7f41e6dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41e0160180 0x7f41e00d35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.158+0000 7f41e6dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e0160b30 0x7f41e00d1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.158+0000 7f41e6dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e01f2f20 con 0x7f41e015c520 2023-07-06T03:48:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.158+0000 7f41e6dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e01f2c50 con 0x7f41e0160b30 2023-07-06T03:48:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.158+0000 7f41e6dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00d2810 con 0x7f41e0160180 2023-07-06T03:48:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.158+0000 7f41e4b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41e015c520 0x7f41e01f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.158+0000 7f41e4b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41e015c520 0x7f41e01f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50974/0 (socket says 172.21.15.97:50974) 2023-07-06T03:48:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.158+0000 7f41e4b9b700 1 -- 172.21.15.97:0/77392920 learned_addr learned my addr 172.21.15.97:0/77392920 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41dffff700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41e0160180 0x7f41e00d35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41e539c700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e0160b30 0x7f41e00d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41e4b9b700 1 -- 172.21.15.97:0/77392920 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41e0160180 msgr2=0x7f41e00d35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41e4b9b700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41e0160180 0x7f41e00d35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41e4b9b700 1 -- 172.21.15.97:0/77392920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e0160b30 msgr2=0x7f41e00d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41e4b9b700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e0160b30 0x7f41e00d1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41e4b9b700 1 -- 172.21.15.97:0/77392920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40094a0 con 0x7f41e015c520 2023-07-06T03:48:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41e4b9b700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41e015c520 0x7f41e01f2710 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f41d400ebe0 tx=0x7f41d400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41dd7fa700 1 -- 172.21.15.97:0/77392920 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4009760 con 0x7f41e015c520 2023-07-06T03:48:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41dd7fa700 1 -- 172.21.15.97:0/77392920 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d400c850 con 0x7f41e015c520 2023-07-06T03:48:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41e6dff700 1 -- 172.21.15.97:0/77392920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc009550 con 0x7f41e015c520 2023-07-06T03:48:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.159+0000 7f41dd7fa700 1 -- 172.21.15.97:0/77392920 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d400cbe0 con 0x7f41e015c520 2023-07-06T03:48:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.160+0000 7f41e6dff700 1 -- 172.21.15.97:0/77392920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e00d2dd0 con 0x7f41e015c520 2023-07-06T03:48:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.161+0000 7f41e6dff700 1 -- 172.21.15.97:0/77392920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e004e340 con 0x7f41e015c520 2023-07-06T03:48:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.161+0000 7f41dd7fa700 1 -- 172.21.15.97:0/77392920 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41d400cd40 con 0x7f41e015c520 2023-07-06T03:48:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.163+0000 7f41dd7fa700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41c406ace0 0x7f41c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.163+0000 7f41dd7fa700 1 -- 172.21.15.97:0/77392920 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f41d4089560 con 0x7f41e015c520 2023-07-06T03:48:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.164+0000 7f41dffff700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41c406ace0 0x7f41c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.164+0000 7f41dffff700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41c406ace0 0x7f41c406d190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f41cc014040 tx=0x7f41cc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:49.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.168+0000 7f41dd7fa700 1 -- 172.21.15.97:0/77392920 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f41d4054570 con 0x7f41e015c520 2023-07-06T03:48:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.405+0000 7f41e6dff700 1 -- 172.21.15.97:0/77392920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41e00b4f40 con 0x7f41e015c520 2023-07-06T03:48:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.406+0000 7f41dd7fa700 1 -- 172.21.15.97:0/77392920 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4678 (secure 0 0 0) 0x7f41d4057b90 con 0x7f41e015c520 2023-07-06T03:48:49.408 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:49.408 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1108873506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1108873506},{"type":"v1","addr":"172.21.15.146:6835","nonce":1108873506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2471899168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2471899168},{"type":"v1","addr":"172.21.15.97:6837","nonce":2471899168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2970792733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2970792733},{"type":"v1","addr":"172.21.15.97:6836","nonce":2970792733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:48:45.401033+0000","modified":"2023-07-06T03:48:48.930905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6399":{"gid":6399,"name":"d","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/2151424592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2151424592},{"type":"v1","addr":"172.21.15.146:6834","nonce":2151424592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-06T03:48:49.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 -- 172.21.15.97:0/77392920 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41c406ace0 msgr2=0x7f41c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41c406ace0 0x7f41c406d190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f41cc014040 tx=0x7f41cc008030 comp rx=0 tx=0).stop 2023-07-06T03:48:49.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 -- 172.21.15.97:0/77392920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41e015c520 msgr2=0x7f41e01f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41e015c520 0x7f41e01f2710 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f41d400ebe0 tx=0x7f41d400eef0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 -- 172.21.15.97:0/77392920 shutdown_connections 2023-07-06T03:48:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41c406ace0 0x7f41c406d190 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41e0160b30 0x7f41e00d1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41e015c520 0x7f41e01f2710 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 --2- 172.21.15.97:0/77392920 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41e0160180 0x7f41e00d35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 -- 172.21.15.97:0/77392920 >> 172.21.15.97:0/77392920 conn(0x7f41e0071790 msgr2=0x7f41e0158250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 -- 172.21.15.97:0/77392920 shutdown_connections 2023-07-06T03:48:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.408+0000 7f41e6dff700 1 -- 172.21.15.97:0/77392920 wait complete. 2023-07-06T03:48:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 34 2023-07-06T03:48:49.428 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:48:49.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.620+0000 7f432c7ff700 1 Processor -- start 2023-07-06T03:48:49.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.620+0000 7f432c7ff700 1 -- start start 2023-07-06T03:48:49.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.621+0000 7f432c7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240b8410 0x7f4324167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:49.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.621+0000 7f432c7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43240cdae0 con 0x7f432415c520 2023-07-06T03:48:49.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f432c7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43240b6640 con 0x7f43240cd390 2023-07-06T03:48:49.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f432c7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43240b6920 con 0x7f43240b8410 2023-07-06T03:48:49.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f432a59b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f432415c520 0x7f4324160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54236/0 (socket says 172.21.15.97:54236) 2023-07-06T03:48:49.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f432a59b700 1 -- 172.21.15.97:0/1869936110 learned_addr learned my addr 172.21.15.97:0/1869936110 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:49.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f432ad9c700 1 --2- 172.21.15.97:0/1869936110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240b8410 0x7f4324167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:49.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f4329599700 1 -- 172.21.15.97:0/1869936110 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048476978 0 0) 0x7f43240cdae0 con 0x7f432415c520 2023-07-06T03:48:49.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f4329599700 1 -- 172.21.15.97:0/1869936110 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43100035b0 con 0x7f432415c520 2023-07-06T03:48:49.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f4329599700 1 -- 172.21.15.97:0/1869936110 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623607127 0 0) 0x7f43240b6640 con 0x7f43240cd390 2023-07-06T03:48:49.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f4329599700 1 -- 172.21.15.97:0/1869936110 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43240cdae0 con 0x7f43240cd390 2023-07-06T03:48:49.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f432ad9c700 1 -- 172.21.15.97:0/1869936110 >> v1:172.21.15.146:6789/0 conn(0x7f43240cd390 legacy=0x7f4324160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f432ad9c700 1 -- 172.21.15.97:0/1869936110 >> v1:172.21.15.97:6789/0 conn(0x7f432415c520 legacy=0x7f4324160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.622+0000 7f432ad9c700 1 -- 172.21.15.97:0/1869936110 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43240b6c00 con 0x7f43240b8410 2023-07-06T03:48:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.623+0000 7f432ad9c700 1 --2- 172.21.15.97:0/1869936110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240b8410 0x7f4324167510 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f431c009dc0 tx=0x7f431c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997360b2e5d7f5b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.623+0000 7f4329599700 1 -- 172.21.15.97:0/1869936110 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c00e070 con 0x7f43240b8410 2023-07-06T03:48:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.623+0000 7f4329599700 1 -- 172.21.15.97:0/1869936110 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c00bb20 con 0x7f43240b8410 2023-07-06T03:48:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.623+0000 7f4329599700 1 -- 172.21.15.97:0/1869936110 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c012430 con 0x7f43240b8410 2023-07-06T03:48:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.623+0000 7f432c7ff700 1 -- 172.21.15.97:0/1869936110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240b8410 msgr2=0x7f4324167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.623+0000 7f432c7ff700 1 --2- 172.21.15.97:0/1869936110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240b8410 0x7f4324167510 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f431c009dc0 tx=0x7f431c00b330 comp rx=0 tx=0).stop 2023-07-06T03:48:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.624+0000 7f432c7ff700 1 -- 172.21.15.97:0/1869936110 shutdown_connections 2023-07-06T03:48:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.624+0000 7f432c7ff700 1 --2- 172.21.15.97:0/1869936110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240b8410 0x7f4324167510 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.624+0000 7f432c7ff700 1 -- 172.21.15.97:0/1869936110 >> 172.21.15.97:0/1869936110 conn(0x7f4324071790 msgr2=0x7f4324071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.624+0000 7f432c7ff700 1 -- 172.21.15.97:0/1869936110 shutdown_connections 2023-07-06T03:48:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.624+0000 7f432c7ff700 1 -- 172.21.15.97:0/1869936110 wait complete. 2023-07-06T03:48:49.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.625+0000 7f432c7ff700 1 Processor -- start 2023-07-06T03:48:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.625+0000 7f432c7ff700 1 -- start start 2023-07-06T03:48:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.626+0000 7f432c7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43240b8410 0x7f432415ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.626+0000 7f432c7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240cd390 0x7f43240d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.626+0000 7f432c7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432415c520 0x7f432413cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.626+0000 7f432c7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240d16e0 con 0x7f432415c520 2023-07-06T03:48:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.626+0000 7f432c7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4324159c40 con 0x7f43240b8410 2023-07-06T03:48:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.626+0000 7f432c7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4324159f20 con 0x7f43240cd390 2023-07-06T03:48:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f432ad9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432415c520 0x7f432413cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f432ad9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432415c520 0x7f432413cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50986/0 (socket says 172.21.15.97:50986) 2023-07-06T03:48:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f432ad9c700 1 -- 172.21.15.97:0/2892757462 learned_addr learned my addr 172.21.15.97:0/2892757462 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f432a59b700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43240b8410 0x7f432415ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f4329d9a700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240cd390 0x7f43240d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f432ad9c700 1 -- 172.21.15.97:0/2892757462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240cd390 msgr2=0x7f43240d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f432ad9c700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240cd390 0x7f43240d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f432ad9c700 1 -- 172.21.15.97:0/2892757462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43240b8410 msgr2=0x7f432415ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f432ad9c700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43240b8410 0x7f432415ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f432ad9c700 1 -- 172.21.15.97:0/2892757462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c009a70 con 0x7f432415c520 2023-07-06T03:48:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f4329d9a700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240cd390 0x7f43240d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f432ad9c700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432415c520 0x7f432413cac0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f431c00b370 tx=0x7f431c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f431affd700 1 -- 172.21.15.97:0/2892757462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c01b810 con 0x7f432415c520 2023-07-06T03:48:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f431affd700 1 -- 172.21.15.97:0/2892757462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c020070 con 0x7f432415c520 2023-07-06T03:48:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f432c7ff700 1 -- 172.21.15.97:0/2892757462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432415a200 con 0x7f432415c520 2023-07-06T03:48:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.627+0000 7f431affd700 1 -- 172.21.15.97:0/2892757462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c01bbf0 con 0x7f432415c520 2023-07-06T03:48:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.628+0000 7f432c7ff700 1 -- 172.21.15.97:0/2892757462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432415a7c0 con 0x7f432415c520 2023-07-06T03:48:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.629+0000 7f432c7ff700 1 -- 172.21.15.97:0/2892757462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4324160f20 con 0x7f432415c520 2023-07-06T03:48:49.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.630+0000 7f431affd700 1 -- 172.21.15.97:0/2892757462 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f431c01bd50 con 0x7f432415c520 2023-07-06T03:48:49.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.631+0000 7f431affd700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430806ace0 0x7f430806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:49.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.631+0000 7f431affd700 1 -- 172.21.15.97:0/2892757462 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f431c022070 con 0x7f432415c520 2023-07-06T03:48:49.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.631+0000 7f432a59b700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430806ace0 0x7f430806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:49.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.632+0000 7f432a59b700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430806ace0 0x7f430806d190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4320000f60 tx=0x7f4320009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:49.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.636+0000 7f431affd700 1 -- 172.21.15.97:0/2892757462 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f431c056df0 con 0x7f432415c520 2023-07-06T03:48:49.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.831+0000 7f432c7ff700 1 -- 172.21.15.97:0/2892757462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f432404e340 con 0x7f432415c520 2023-07-06T03:48:49.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.833+0000 7f431affd700 1 -- 172.21.15.97:0/2892757462 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+18367 (secure 0 0 0) 0x7f431c05a410 con 0x7f432415c520 2023-07-06T03:48:49.835 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:49.835 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:48:46.913849+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:48:43.594325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-06T03:48:44.371987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:48:49.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.835+0000 7f432c7ff700 1 -- 172.21.15.97:0/2892757462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430806ace0 msgr2=0x7f430806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.835+0000 7f432c7ff700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430806ace0 0x7f430806d190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4320000f60 tx=0x7f4320009040 comp rx=0 tx=0).stop 2023-07-06T03:48:49.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.835+0000 7f432c7ff700 1 -- 172.21.15.97:0/2892757462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432415c520 msgr2=0x7f432413cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.835+0000 7f432c7ff700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432415c520 0x7f432413cac0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f431c00b370 tx=0x7f431c004ad0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.835+0000 7f432c7ff700 1 -- 172.21.15.97:0/2892757462 shutdown_connections 2023-07-06T03:48:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.835+0000 7f432c7ff700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430806ace0 0x7f430806d190 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.835+0000 7f432c7ff700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432415c520 0x7f432413cac0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.835+0000 7f432c7ff700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43240b8410 0x7f432415ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.835+0000 7f432c7ff700 1 --2- 172.21.15.97:0/2892757462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43240cd390 0x7f43240d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.835+0000 7f432c7ff700 1 -- 172.21.15.97:0/2892757462 >> 172.21.15.97:0/2892757462 conn(0x7f4324071790 msgr2=0x7f43240cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:49.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.836+0000 7f432c7ff700 1 -- 172.21.15.97:0/2892757462 shutdown_connections 2023-07-06T03:48:49.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:49.836+0000 7f432c7ff700 1 -- 172.21.15.97:0/2892757462 wait complete. 2023-07-06T03:48:49.855 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T03:48:49.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:48:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.049+0000 7fdbd75ff700 1 Processor -- start 2023-07-06T03:48:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.049+0000 7fdbd75ff700 1 -- start start 2023-07-06T03:48:50.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd75ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdbd015c520 0x7fdbd00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd75ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdbd0160180 0x7fdbd01605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd75ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbd0160b30 0x7fdbd00cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd75ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd00cd8d0 con 0x7fdbd0160180 2023-07-06T03:48:50.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd75ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd00cdae0 con 0x7fdbd015c520 2023-07-06T03:48:50.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd75ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00b6640 con 0x7fdbd0160b30 2023-07-06T03:48:50.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd4b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdbd0160180 0x7fdbd01605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd4b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdbd0160180 0x7fdbd01605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49522/0 (socket says 172.21.15.97:49522) 2023-07-06T03:48:50.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd4b9a700 1 -- 172.21.15.97:0/2318166212 learned_addr learned my addr 172.21.15.97:0/2318166212 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd5b9c700 1 --2- 172.21.15.97:0/2318166212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbd0160b30 0x7fdbd00cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd539b700 1 --2- 172.21.15.97:0/2318166212 >> v2:172.21.15.146:3300/0 conn(0x7fdbd015c520 0x7fdbd00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd4b9a700 1 -- 172.21.15.97:0/2318166212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbd0160b30 msgr2=0x7fdbd00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd4b9a700 1 --2- 172.21.15.97:0/2318166212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbd0160b30 0x7fdbd00cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd4b9a700 1 -- 172.21.15.97:0/2318166212 >> v2:172.21.15.146:3300/0 conn(0x7fdbd015c520 msgr2=0x7fdbd00cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:50.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd4b9a700 1 --2- 172.21.15.97:0/2318166212 >> v2:172.21.15.146:3300/0 conn(0x7fdbd015c520 0x7fdbd00cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd4b9a700 1 -- 172.21.15.97:0/2318166212 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00b6920 con 0x7fdbd0160180 2023-07-06T03:48:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd539b700 1 --2- 172.21.15.97:0/2318166212 >> v2:172.21.15.146:3300/0 conn(0x7fdbd015c520 0x7fdbd00cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.051+0000 7fdbd5b9c700 1 --2- 172.21.15.97:0/2318166212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbd0160b30 0x7fdbd00cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.052+0000 7fdbd4b9a700 1 --2- 172.21.15.97:0/2318166212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd0160180 0x7fdbd01605f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00b4c70 tx=0x7fdbc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a262af2371730c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.052+0000 7fdbc7fff700 1 -- 172.21.15.97:0/2318166212 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000b870 con 0x7fdbd0160180 2023-07-06T03:48:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.052+0000 7fdbc7fff700 1 -- 172.21.15.97:0/2318166212 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc000b9d0 con 0x7fdbd0160180 2023-07-06T03:48:50.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.052+0000 7fdbc7fff700 1 -- 172.21.15.97:0/2318166212 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000bd60 con 0x7fdbd0160180 2023-07-06T03:48:50.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.052+0000 7fdbd75ff700 1 -- 172.21.15.97:0/2318166212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd0160180 msgr2=0x7fdbd01605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:50.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.052+0000 7fdbd75ff700 1 --2- 172.21.15.97:0/2318166212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd0160180 0x7fdbd01605f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00b4c70 tx=0x7fdbc0009bd0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.052+0000 7fdbd75ff700 1 -- 172.21.15.97:0/2318166212 shutdown_connections 2023-07-06T03:48:50.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.052+0000 7fdbd75ff700 1 --2- 172.21.15.97:0/2318166212 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbd0160b30 0x7fdbd00cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.052+0000 7fdbd75ff700 1 --2- 172.21.15.97:0/2318166212 >> v2:172.21.15.146:3300/0 conn(0x7fdbd015c520 0x7fdbd00cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.052+0000 7fdbd75ff700 1 --2- 172.21.15.97:0/2318166212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd0160180 0x7fdbd01605f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.052+0000 7fdbd75ff700 1 -- 172.21.15.97:0/2318166212 >> 172.21.15.97:0/2318166212 conn(0x7fdbd0071790 msgr2=0x7fdbd0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:50.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.053+0000 7fdbd75ff700 1 -- 172.21.15.97:0/2318166212 shutdown_connections 2023-07-06T03:48:50.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.053+0000 7fdbd75ff700 1 -- 172.21.15.97:0/2318166212 wait complete. 2023-07-06T03:48:50.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.054+0000 7fdbd75ff700 1 Processor -- start 2023-07-06T03:48:50.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.054+0000 7fdbd75ff700 1 -- start start 2023-07-06T03:48:50.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.055+0000 7fdbd75ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdbd015c520 0x7fdbd015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.055+0000 7fdbd75ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbd0160180 0x7fdbd01f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.055+0000 7fdbd75ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd0160b30 0x7fdbd01f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.055+0000 7fdbd75ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd015ed40 con 0x7fdbd0160b30 2023-07-06T03:48:50.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.055+0000 7fdbd75ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00d1e60 con 0x7fdbd015c520 2023-07-06T03:48:50.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.055+0000 7fdbd75ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00d1b80 con 0x7fdbd0160180 2023-07-06T03:48:50.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.055+0000 7fdbd5b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd0160b30 0x7fdbd01f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbd5b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd0160b30 0x7fdbd01f5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49524/0 (socket says 172.21.15.97:49524) 2023-07-06T03:48:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbd5b9c700 1 -- 172.21.15.97:0/4169178857 learned_addr learned my addr 172.21.15.97:0/4169178857 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbd4b9a700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbd0160180 0x7fdbd01f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbd539b700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdbd015c520 0x7fdbd015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbd5b9c700 1 -- 172.21.15.97:0/4169178857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbd0160180 msgr2=0x7fdbd01f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbd5b9c700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbd0160180 0x7fdbd01f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbd5b9c700 1 -- 172.21.15.97:0/4169178857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdbd015c520 msgr2=0x7fdbd015b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbd5b9c700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdbd015c520 0x7fdbd015b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbd5b9c700 1 -- 172.21.15.97:0/4169178857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc0009550 con 0x7fdbd0160b30 2023-07-06T03:48:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbd5b9c700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd0160b30 0x7fdbd01f5850 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800cd80 tx=0x7fdbc800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbc5ffb700 1 -- 172.21.15.97:0/4169178857 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800f480 con 0x7fdbd0160b30 2023-07-06T03:48:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbc5ffb700 1 -- 172.21.15.97:0/4169178857 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc800f600 con 0x7fdbd0160b30 2023-07-06T03:48:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbc5ffb700 1 -- 172.21.15.97:0/4169178857 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800f9b0 con 0x7fdbd0160b30 2023-07-06T03:48:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.056+0000 7fdbd75ff700 1 -- 172.21.15.97:0/4169178857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd015e6f0 con 0x7fdbd0160b30 2023-07-06T03:48:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.057+0000 7fdbd75ff700 1 -- 172.21.15.97:0/4169178857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd015bcd0 con 0x7fdbd0160b30 2023-07-06T03:48:50.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.058+0000 7fdbd75ff700 1 -- 172.21.15.97:0/4169178857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd015eef0 con 0x7fdbd0160b30 2023-07-06T03:48:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.059+0000 7fdbc5ffb700 1 -- 172.21.15.97:0/4169178857 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdbc800b040 con 0x7fdbd0160b30 2023-07-06T03:48:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.060+0000 7fdbc5ffb700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbb406ace0 0x7fdbb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.060+0000 7fdbc5ffb700 1 -- 172.21.15.97:0/4169178857 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdbc80899c0 con 0x7fdbd0160b30 2023-07-06T03:48:50.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.060+0000 7fdbd539b700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbb406ace0 0x7fdbb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.060+0000 7fdbd539b700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbb406ace0 0x7fdbb406d190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc006fd0 tx=0x7fdbcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:50.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.064+0000 7fdbc5ffb700 1 -- 172.21.15.97:0/4169178857 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdbd015eef0 con 0x7fdbd0160b30 2023-07-06T03:48:50.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.253+0000 7fdbd75ff700 1 -- 172.21.15.97:0/4169178857 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdbd00b6ee0 con 0x7fdbb406ace0 2023-07-06T03:48:50.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.263+0000 7fdbc5ffb700 1 -- 172.21.15.97:0/4169178857 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311119 (secure 0 0 0) 0x7fdbcc00c040 con 0x7fdbb406ace0 2023-07-06T03:48:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:50.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.267+0000 7fdbd75ff700 1 -- 172.21.15.97:0/4169178857 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbb406ace0 msgr2=0x7fdbb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.267+0000 7fdbd75ff700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbb406ace0 0x7fdbb406d190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc006fd0 tx=0x7fdbcc008040 comp rx=0 tx=0).stop 2023-07-06T03:48:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.267+0000 7fdbd75ff700 1 -- 172.21.15.97:0/4169178857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd0160b30 msgr2=0x7fdbd01f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.267+0000 7fdbd75ff700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd0160b30 0x7fdbd01f5850 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800cd80 tx=0x7fdbc800a560 comp rx=0 tx=0).stop 2023-07-06T03:48:50.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.267+0000 7fdbd75ff700 1 -- 172.21.15.97:0/4169178857 shutdown_connections 2023-07-06T03:48:50.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.267+0000 7fdbd75ff700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbb406ace0 0x7fdbb406d190 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.267+0000 7fdbd75ff700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd0160b30 0x7fdbd01f5850 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.267+0000 7fdbd75ff700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdbd015c520 0x7fdbd015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.267+0000 7fdbd75ff700 1 --2- 172.21.15.97:0/4169178857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbd0160180 0x7fdbd01f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.267+0000 7fdbd75ff700 1 -- 172.21.15.97:0/4169178857 >> 172.21.15.97:0/4169178857 conn(0x7fdbd0071790 msgr2=0x7fdbd0071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:50.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.268+0000 7fdbd75ff700 1 -- 172.21.15.97:0/4169178857 shutdown_connections 2023-07-06T03:48:50.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.268+0000 7fdbd75ff700 1 -- 172.21.15.97:0/4169178857 wait complete. 2023-07-06T03:48:50.275 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:48:50.288 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2023-07-06T03:48:48.926100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263184,"kb_used_data":6608,"kb_used_omap":0,"kb_used_meta":256512,"kb_avail":754711536,"statfs":{"total":773094113280,"available":772824612864,"internally_reserved":0,"allocated":6766592,"data_stored":3618541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":262668288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.034241"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938459+0000","last_change":"2023-07-06T03:48:45.936421+0000","last_active":"2023-07-06T03:48:45.938459+0000","last_peered":"2023-07-06T03:48:45.938459+0000","last_clean":"2023-07-06T03:48:45.938459+0000","last_became_active":"2023-07-06T03:48:45.936158+0000","last_became_peered":"2023-07-06T03:48:45.936158+0000","last_unstale":"2023-07-06T03:48:45.938459+0000","last_undegraded":"2023-07-06T03:48:45.938459+0000","last_fullsized":"2023-07-06T03:48:45.938459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:41:12.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.934244+0000","last_change":"2023-07-06T03:48:45.927570+0000","last_active":"2023-07-06T03:48:45.934244+0000","last_peered":"2023-07-06T03:48:45.934244+0000","last_clean":"2023-07-06T03:48:45.934244+0000","last_became_active":"2023-07-06T03:48:45.927050+0000","last_became_peered":"2023-07-06T03:48:45.927050+0000","last_unstale":"2023-07-06T03:48:45.934244+0000","last_undegraded":"2023-07-06T03:48:45.934244+0000","last_fullsized":"2023-07-06T03:48:45.934244+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:45:54.389534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.941953+0000","last_change":"2023-07-06T03:48:45.937436+0000","last_active":"2023-07-06T03:48:45.941953+0000","last_peered":"2023-07-06T03:48:45.941953+0000","last_clean":"2023-07-06T03:48:45.941953+0000","last_became_active":"2023-07-06T03:48:45.937098+0000","last_became_peered":"2023-07-06T03:48:45.937098+0000","last_unstale":"2023-07-06T03:48:45.941953+0000","last_undegraded":"2023-07-06T03:48:45.941953+0000","last_fullsized":"2023-07-06T03:48:45.941953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:19:23.201389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945625+0000","last_change":"2023-07-06T03:48:45.940318+0000","last_active":"2023-07-06T03:48:45.945625+0000","last_peered":"2023-07-06T03:48:45.945625+0000","last_clean":"2023-07-06T03:48:45.945625+0000","last_became_active":"2023-07-06T03:48:45.939910+0000","last_became_peered":"2023-07-06T03:48:45.939910+0000","last_unstale":"2023-07-06T03:48:45.945625+0000","last_undegraded":"2023-07-06T03:48:45.945625+0000","last_fullsized":"2023-07-06T03:48:45.945625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:21:27.456563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.927467+0000","last_change":"2023-07-06T03:48:45.922937+0000","last_active":"2023-07-06T03:48:45.927467+0000","last_peered":"2023-07-06T03:48:45.927467+0000","last_clean":"2023-07-06T03:48:45.927467+0000","last_became_active":"2023-07-06T03:48:45.922680+0000","last_became_peered":"2023-07-06T03:48:45.922680+0000","last_unstale":"2023-07-06T03:48:45.927467+0000","last_undegraded":"2023-07-06T03:48:45.927467+0000","last_fullsized":"2023-07-06T03:48:45.927467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:56:11.613853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937946+0000","last_change":"2023-07-06T03:48:45.936324+0000","last_active":"2023-07-06T03:48:45.937946+0000","last_peered":"2023-07-06T03:48:45.937946+0000","last_clean":"2023-07-06T03:48:45.937946+0000","last_became_active":"2023-07-06T03:48:45.935704+0000","last_became_peered":"2023-07-06T03:48:45.935704+0000","last_unstale":"2023-07-06T03:48:45.937946+0000","last_undegraded":"2023-07-06T03:48:45.937946+0000","last_fullsized":"2023-07-06T03:48:45.937946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:08:52.349909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937737+0000","last_change":"2023-07-06T03:48:45.929921+0000","last_active":"2023-07-06T03:48:45.937737+0000","last_peered":"2023-07-06T03:48:45.937737+0000","last_clean":"2023-07-06T03:48:45.937737+0000","last_became_active":"2023-07-06T03:48:45.929599+0000","last_became_peered":"2023-07-06T03:48:45.929599+0000","last_unstale":"2023-07-06T03:48:45.937737+0000","last_undegraded":"2023-07-06T03:48:45.937737+0000","last_fullsized":"2023-07-06T03:48:45.937737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:28.861995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.933369+0000","last_change":"2023-07-06T03:48:45.924753+0000","last_active":"2023-07-06T03:48:45.933369+0000","last_peered":"2023-07-06T03:48:45.933369+0000","last_clean":"2023-07-06T03:48:45.933369+0000","last_became_active":"2023-07-06T03:48:45.924441+0000","last_became_peered":"2023-07-06T03:48:45.924441+0000","last_unstale":"2023-07-06T03:48:45.933369+0000","last_undegraded":"2023-07-06T03:48:45.933369+0000","last_fullsized":"2023-07-06T03:48:45.933369+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:54:08.183436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.926363+0000","last_change":"2023-07-06T03:48:45.921212+0000","last_active":"2023-07-06T03:48:45.926363+0000","last_peered":"2023-07-06T03:48:45.926363+0000","last_clean":"2023-07-06T03:48:45.926363+0000","last_became_active":"2023-07-06T03:48:45.920785+0000","last_became_peered":"2023-07-06T03:48:45.920785+0000","last_unstale":"2023-07-06T03:48:45.926363+0000","last_undegraded":"2023-07-06T03:48:45.926363+0000","last_fullsized":"2023-07-06T03:48:45.926363+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:13:14.506309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.930082+0000","last_change":"2023-07-06T03:48:45.927488+0000","last_active":"2023-07-06T03:48:45.930082+0000","last_peered":"2023-07-06T03:48:45.930082+0000","last_clean":"2023-07-06T03:48:45.930082+0000","last_became_active":"2023-07-06T03:48:45.927208+0000","last_became_peered":"2023-07-06T03:48:45.927208+0000","last_unstale":"2023-07-06T03:48:45.930082+0000","last_undegraded":"2023-07-06T03:48:45.930082+0000","last_fullsized":"2023-07-06T03:48:45.930082+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:15:37.713996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.940995+0000","last_change":"2023-07-06T03:48:45.936161+0000","last_active":"2023-07-06T03:48:45.940995+0000","last_peered":"2023-07-06T03:48:45.940995+0000","last_clean":"2023-07-06T03:48:45.940995+0000","last_became_active":"2023-07-06T03:48:45.934879+0000","last_became_peered":"2023-07-06T03:48:45.934879+0000","last_unstale":"2023-07-06T03:48:45.940995+0000","last_undegraded":"2023-07-06T03:48:45.940995+0000","last_fullsized":"2023-07-06T03:48:45.940995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:17:06.277791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945947+0000","last_change":"2023-07-06T03:48:45.942831+0000","last_active":"2023-07-06T03:48:45.945947+0000","last_peered":"2023-07-06T03:48:45.945947+0000","last_clean":"2023-07-06T03:48:45.945947+0000","last_became_active":"2023-07-06T03:48:45.942539+0000","last_became_peered":"2023-07-06T03:48:45.942539+0000","last_unstale":"2023-07-06T03:48:45.945947+0000","last_undegraded":"2023-07-06T03:48:45.945947+0000","last_fullsized":"2023-07-06T03:48:45.945947+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:14.401412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.947453+0000","last_change":"2023-07-06T03:48:45.945006+0000","last_active":"2023-07-06T03:48:45.947453+0000","last_peered":"2023-07-06T03:48:45.947453+0000","last_clean":"2023-07-06T03:48:45.947453+0000","last_became_active":"2023-07-06T03:48:45.944558+0000","last_became_peered":"2023-07-06T03:48:45.944558+0000","last_unstale":"2023-07-06T03:48:45.947453+0000","last_undegraded":"2023-07-06T03:48:45.947453+0000","last_fullsized":"2023-07-06T03:48:45.947453+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:53:29.567555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.946966+0000","last_change":"2023-07-06T03:48:45.940073+0000","last_active":"2023-07-06T03:48:45.946966+0000","last_peered":"2023-07-06T03:48:45.946966+0000","last_clean":"2023-07-06T03:48:45.946966+0000","last_became_active":"2023-07-06T03:48:45.939838+0000","last_became_peered":"2023-07-06T03:48:45.939838+0000","last_unstale":"2023-07-06T03:48:45.946966+0000","last_undegraded":"2023-07-06T03:48:45.946966+0000","last_fullsized":"2023-07-06T03:48:45.946966+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:17:12.651762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.935951+0000","last_change":"2023-07-06T03:48:45.933692+0000","last_active":"2023-07-06T03:48:45.935951+0000","last_peered":"2023-07-06T03:48:45.935951+0000","last_clean":"2023-07-06T03:48:45.935951+0000","last_became_active":"2023-07-06T03:48:45.933384+0000","last_became_peered":"2023-07-06T03:48:45.933384+0000","last_unstale":"2023-07-06T03:48:45.935951+0000","last_undegraded":"2023-07-06T03:48:45.935951+0000","last_fullsized":"2023-07-06T03:48:45.935951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:09:26.159497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.936725+0000","last_change":"2023-07-06T03:48:45.935476+0000","last_active":"2023-07-06T03:48:45.936725+0000","last_peered":"2023-07-06T03:48:45.936725+0000","last_clean":"2023-07-06T03:48:45.936725+0000","last_became_active":"2023-07-06T03:48:45.935106+0000","last_became_peered":"2023-07-06T03:48:45.935106+0000","last_unstale":"2023-07-06T03:48:45.936725+0000","last_undegraded":"2023-07-06T03:48:45.936725+0000","last_fullsized":"2023-07-06T03:48:45.936725+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:46:22.384643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938446+0000","last_change":"2023-07-06T03:48:45.934922+0000","last_active":"2023-07-06T03:48:45.938446+0000","last_peered":"2023-07-06T03:48:45.938446+0000","last_clean":"2023-07-06T03:48:45.938446+0000","last_became_active":"2023-07-06T03:48:45.934639+0000","last_became_peered":"2023-07-06T03:48:45.934639+0000","last_unstale":"2023-07-06T03:48:45.938446+0000","last_undegraded":"2023-07-06T03:48:45.938446+0000","last_fullsized":"2023-07-06T03:48:45.938446+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:09:27.067517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.936067+0000","last_change":"2023-07-06T03:48:45.931455+0000","last_active":"2023-07-06T03:48:45.936067+0000","last_peered":"2023-07-06T03:48:45.936067+0000","last_clean":"2023-07-06T03:48:45.936067+0000","last_became_active":"2023-07-06T03:48:45.930983+0000","last_became_peered":"2023-07-06T03:48:45.930983+0000","last_unstale":"2023-07-06T03:48:45.936067+0000","last_undegraded":"2023-07-06T03:48:45.936067+0000","last_fullsized":"2023-07-06T03:48:45.936067+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:54:27.935910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938025+0000","last_change":"2023-07-06T03:48:45.935164+0000","last_active":"2023-07-06T03:48:45.938025+0000","last_peered":"2023-07-06T03:48:45.938025+0000","last_clean":"2023-07-06T03:48:45.938025+0000","last_became_active":"2023-07-06T03:48:45.934539+0000","last_became_peered":"2023-07-06T03:48:45.934539+0000","last_unstale":"2023-07-06T03:48:45.938025+0000","last_undegraded":"2023-07-06T03:48:45.938025+0000","last_fullsized":"2023-07-06T03:48:45.938025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:20:09.326471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.931690+0000","last_change":"2023-07-06T03:48:45.924527+0000","last_active":"2023-07-06T03:48:45.931690+0000","last_peered":"2023-07-06T03:48:45.931690+0000","last_clean":"2023-07-06T03:48:45.931690+0000","last_became_active":"2023-07-06T03:48:45.924232+0000","last_became_peered":"2023-07-06T03:48:45.924232+0000","last_unstale":"2023-07-06T03:48:45.931690+0000","last_undegraded":"2023-07-06T03:48:45.931690+0000","last_fullsized":"2023-07-06T03:48:45.931690+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:33:35.631454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.944613+0000","last_change":"2023-07-06T03:48:45.937588+0000","last_active":"2023-07-06T03:48:45.944613+0000","last_peered":"2023-07-06T03:48:45.944613+0000","last_clean":"2023-07-06T03:48:45.944613+0000","last_became_active":"2023-07-06T03:48:45.937248+0000","last_became_peered":"2023-07-06T03:48:45.937248+0000","last_unstale":"2023-07-06T03:48:45.944613+0000","last_undegraded":"2023-07-06T03:48:45.944613+0000","last_fullsized":"2023-07-06T03:48:45.944613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:40:19.956777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937963+0000","last_change":"2023-07-06T03:48:45.936412+0000","last_active":"2023-07-06T03:48:45.937963+0000","last_peered":"2023-07-06T03:48:45.937963+0000","last_clean":"2023-07-06T03:48:45.937963+0000","last_became_active":"2023-07-06T03:48:45.936056+0000","last_became_peered":"2023-07-06T03:48:45.936056+0000","last_unstale":"2023-07-06T03:48:45.937963+0000","last_undegraded":"2023-07-06T03:48:45.937963+0000","last_fullsized":"2023-07-06T03:48:45.937963+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:07:02.951830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.924178+0000","last_change":"2023-07-06T03:48:45.920952+0000","last_active":"2023-07-06T03:48:45.924178+0000","last_peered":"2023-07-06T03:48:45.924178+0000","last_clean":"2023-07-06T03:48:45.924178+0000","last_became_active":"2023-07-06T03:48:45.920629+0000","last_became_peered":"2023-07-06T03:48:45.920629+0000","last_unstale":"2023-07-06T03:48:45.924178+0000","last_undegraded":"2023-07-06T03:48:45.924178+0000","last_fullsized":"2023-07-06T03:48:45.924178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:15:37.713996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.924626+0000","last_change":"2023-07-06T03:48:45.915892+0000","last_active":"2023-07-06T03:48:45.924626+0000","last_peered":"2023-07-06T03:48:45.924626+0000","last_clean":"2023-07-06T03:48:45.924626+0000","last_became_active":"2023-07-06T03:48:45.915629+0000","last_became_peered":"2023-07-06T03:48:45.915629+0000","last_unstale":"2023-07-06T03:48:45.924626+0000","last_undegraded":"2023-07-06T03:48:45.924626+0000","last_fullsized":"2023-07-06T03:48:45.924626+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:04:14.349930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.940408+0000","last_change":"2023-07-06T03:48:45.932289+0000","last_active":"2023-07-06T03:48:45.940408+0000","last_peered":"2023-07-06T03:48:45.940408+0000","last_clean":"2023-07-06T03:48:45.940408+0000","last_became_active":"2023-07-06T03:48:45.932024+0000","last_became_peered":"2023-07-06T03:48:45.932024+0000","last_unstale":"2023-07-06T03:48:45.940408+0000","last_undegraded":"2023-07-06T03:48:45.940408+0000","last_fullsized":"2023-07-06T03:48:45.940408+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:15:56.397408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.935761+0000","last_change":"2023-07-06T03:48:45.926955+0000","last_active":"2023-07-06T03:48:45.935761+0000","last_peered":"2023-07-06T03:48:45.935761+0000","last_clean":"2023-07-06T03:48:45.935761+0000","last_became_active":"2023-07-06T03:48:45.926640+0000","last_became_peered":"2023-07-06T03:48:45.926640+0000","last_unstale":"2023-07-06T03:48:45.935761+0000","last_undegraded":"2023-07-06T03:48:45.935761+0000","last_fullsized":"2023-07-06T03:48:45.935761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:08.579590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.939111+0000","last_change":"2023-07-06T03:48:45.933396+0000","last_active":"2023-07-06T03:48:45.939111+0000","last_peered":"2023-07-06T03:48:45.939111+0000","last_clean":"2023-07-06T03:48:45.939111+0000","last_became_active":"2023-07-06T03:48:45.933120+0000","last_became_peered":"2023-07-06T03:48:45.933120+0000","last_unstale":"2023-07-06T03:48:45.939111+0000","last_undegraded":"2023-07-06T03:48:45.939111+0000","last_fullsized":"2023-07-06T03:48:45.939111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.457526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938485+0000","last_change":"2023-07-06T03:48:45.933258+0000","last_active":"2023-07-06T03:48:45.938485+0000","last_peered":"2023-07-06T03:48:45.938485+0000","last_clean":"2023-07-06T03:48:45.938485+0000","last_became_active":"2023-07-06T03:48:45.932796+0000","last_became_peered":"2023-07-06T03:48:45.932796+0000","last_unstale":"2023-07-06T03:48:45.938485+0000","last_undegraded":"2023-07-06T03:48:45.938485+0000","last_fullsized":"2023-07-06T03:48:45.938485+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:42:29.955680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.946085+0000","last_change":"2023-07-06T03:48:45.942965+0000","last_active":"2023-07-06T03:48:45.946085+0000","last_peered":"2023-07-06T03:48:45.946085+0000","last_clean":"2023-07-06T03:48:45.946085+0000","last_became_active":"2023-07-06T03:48:45.942694+0000","last_became_peered":"2023-07-06T03:48:45.942694+0000","last_unstale":"2023-07-06T03:48:45.946085+0000","last_undegraded":"2023-07-06T03:48:45.946085+0000","last_fullsized":"2023-07-06T03:48:45.946085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:17:30.665732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.929914+0000","last_change":"2023-07-06T03:48:45.922389+0000","last_active":"2023-07-06T03:48:45.929914+0000","last_peered":"2023-07-06T03:48:45.929914+0000","last_clean":"2023-07-06T03:48:45.929914+0000","last_became_active":"2023-07-06T03:48:45.921930+0000","last_became_peered":"2023-07-06T03:48:45.921930+0000","last_unstale":"2023-07-06T03:48:45.929914+0000","last_undegraded":"2023-07-06T03:48:45.929914+0000","last_fullsized":"2023-07-06T03:48:45.929914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:38:48.869271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.916843+0000","last_change":"2023-07-06T03:48:45.914819+0000","last_active":"2023-07-06T03:48:45.916843+0000","last_peered":"2023-07-06T03:48:45.916843+0000","last_clean":"2023-07-06T03:48:45.916843+0000","last_became_active":"2023-07-06T03:48:45.914495+0000","last_became_peered":"2023-07-06T03:48:45.914495+0000","last_unstale":"2023-07-06T03:48:45.916843+0000","last_undegraded":"2023-07-06T03:48:45.916843+0000","last_fullsized":"2023-07-06T03:48:45.916843+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:34:55.838514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.933719+0000","last_change":"2023-07-06T03:48:45.928736+0000","last_active":"2023-07-06T03:48:45.933719+0000","last_peered":"2023-07-06T03:48:45.933719+0000","last_clean":"2023-07-06T03:48:45.933719+0000","last_became_active":"2023-07-06T03:48:45.928423+0000","last_became_peered":"2023-07-06T03:48:45.928423+0000","last_unstale":"2023-07-06T03:48:45.933719+0000","last_undegraded":"2023-07-06T03:48:45.933719+0000","last_fullsized":"2023-07-06T03:48:45.933719+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:03:15.250420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945590+0000","last_change":"2023-07-06T03:48:45.940164+0000","last_active":"2023-07-06T03:48:45.945590+0000","last_peered":"2023-07-06T03:48:45.945590+0000","last_clean":"2023-07-06T03:48:45.945590+0000","last_became_active":"2023-07-06T03:48:45.938126+0000","last_became_peered":"2023-07-06T03:48:45.938126+0000","last_unstale":"2023-07-06T03:48:45.945590+0000","last_undegraded":"2023-07-06T03:48:45.945590+0000","last_fullsized":"2023-07-06T03:48:45.945590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:32:39.098957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.930075+0000","last_change":"2023-07-06T03:48:44.913633+0000","last_active":"2023-07-06T03:48:45.930075+0000","last_peered":"2023-07-06T03:48:45.930075+0000","last_clean":"2023-07-06T03:48:45.930075+0000","last_became_active":"2023-07-06T03:48:44.913361+0000","last_became_peered":"2023-07-06T03:48:44.913361+0000","last_unstale":"2023-07-06T03:48:45.930075+0000","last_undegraded":"2023-07-06T03:48:45.930075+0000","last_fullsized":"2023-07-06T03:48:45.930075+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:11:41.522108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.932292+0000","last_change":"2023-07-06T03:48:45.923424+0000","last_active":"2023-07-06T03:48:45.932292+0000","last_peered":"2023-07-06T03:48:45.932292+0000","last_clean":"2023-07-06T03:48:45.932292+0000","last_became_active":"2023-07-06T03:48:45.923146+0000","last_became_peered":"2023-07-06T03:48:45.923146+0000","last_unstale":"2023-07-06T03:48:45.932292+0000","last_undegraded":"2023-07-06T03:48:45.932292+0000","last_fullsized":"2023-07-06T03:48:45.932292+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:53:29.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.936798+0000","last_change":"2023-07-06T03:48:44.921283+0000","last_active":"2023-07-06T03:48:45.936798+0000","last_peered":"2023-07-06T03:48:45.936798+0000","last_clean":"2023-07-06T03:48:45.936798+0000","last_became_active":"2023-07-06T03:48:44.920943+0000","last_became_peered":"2023-07-06T03:48:44.920943+0000","last_unstale":"2023-07-06T03:48:45.936798+0000","last_undegraded":"2023-07-06T03:48:45.936798+0000","last_fullsized":"2023-07-06T03:48:45.936798+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:54.735005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.934223+0000","last_change":"2023-07-06T03:48:45.927374+0000","last_active":"2023-07-06T03:48:45.934223+0000","last_peered":"2023-07-06T03:48:45.934223+0000","last_clean":"2023-07-06T03:48:45.934223+0000","last_became_active":"2023-07-06T03:48:45.926739+0000","last_became_peered":"2023-07-06T03:48:45.926739+0000","last_unstale":"2023-07-06T03:48:45.934223+0000","last_undegraded":"2023-07-06T03:48:45.934223+0000","last_fullsized":"2023-07-06T03:48:45.934223+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:53:12.064750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937563+0000","last_change":"2023-07-06T03:48:44.917394+0000","last_active":"2023-07-06T03:48:45.937563+0000","last_peered":"2023-07-06T03:48:45.937563+0000","last_clean":"2023-07-06T03:48:45.937563+0000","last_became_active":"2023-07-06T03:48:44.916937+0000","last_became_peered":"2023-07-06T03:48:44.916937+0000","last_unstale":"2023-07-06T03:48:45.937563+0000","last_undegraded":"2023-07-06T03:48:45.937563+0000","last_fullsized":"2023-07-06T03:48:45.937563+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:26:39.450856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.936280+0000","last_change":"2023-07-06T03:48:45.925290+0000","last_active":"2023-07-06T03:48:45.936280+0000","last_peered":"2023-07-06T03:48:45.936280+0000","last_clean":"2023-07-06T03:48:45.936280+0000","last_became_active":"2023-07-06T03:48:45.925034+0000","last_became_peered":"2023-07-06T03:48:45.925034+0000","last_unstale":"2023-07-06T03:48:45.936280+0000","last_undegraded":"2023-07-06T03:48:45.936280+0000","last_fullsized":"2023-07-06T03:48:45.936280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:27:27.148134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938931+0000","last_change":"2023-07-06T03:48:44.923930+0000","last_active":"2023-07-06T03:48:45.938931+0000","last_peered":"2023-07-06T03:48:45.938931+0000","last_clean":"2023-07-06T03:48:45.938931+0000","last_became_active":"2023-07-06T03:48:44.923591+0000","last_became_peered":"2023-07-06T03:48:44.923591+0000","last_unstale":"2023-07-06T03:48:45.938931+0000","last_undegraded":"2023-07-06T03:48:45.938931+0000","last_fullsized":"2023-07-06T03:48:45.938931+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:43:32.679358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.927704+0000","last_change":"2023-07-06T03:48:45.925264+0000","last_active":"2023-07-06T03:48:45.927704+0000","last_peered":"2023-07-06T03:48:45.927704+0000","last_clean":"2023-07-06T03:48:45.927704+0000","last_became_active":"2023-07-06T03:48:45.924719+0000","last_became_peered":"2023-07-06T03:48:45.924719+0000","last_unstale":"2023-07-06T03:48:45.927704+0000","last_undegraded":"2023-07-06T03:48:45.927704+0000","last_fullsized":"2023-07-06T03:48:45.927704+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:23:57.868352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.929347+0000","last_change":"2023-07-06T03:48:44.914772+0000","last_active":"2023-07-06T03:48:45.929347+0000","last_peered":"2023-07-06T03:48:45.929347+0000","last_clean":"2023-07-06T03:48:45.929347+0000","last_became_active":"2023-07-06T03:48:44.913233+0000","last_became_peered":"2023-07-06T03:48:44.913233+0000","last_unstale":"2023-07-06T03:48:45.929347+0000","last_undegraded":"2023-07-06T03:48:45.929347+0000","last_fullsized":"2023-07-06T03:48:45.929347+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:18:39.820151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938332+0000","last_change":"2023-07-06T03:48:45.932622+0000","last_active":"2023-07-06T03:48:45.938332+0000","last_peered":"2023-07-06T03:48:45.938332+0000","last_clean":"2023-07-06T03:48:45.938332+0000","last_became_active":"2023-07-06T03:48:45.932282+0000","last_became_peered":"2023-07-06T03:48:45.932282+0000","last_unstale":"2023-07-06T03:48:45.938332+0000","last_undegraded":"2023-07-06T03:48:45.938332+0000","last_fullsized":"2023-07-06T03:48:45.938332+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:15:19.314836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.939422+0000","last_change":"2023-07-06T03:48:44.925189+0000","last_active":"2023-07-06T03:48:45.939422+0000","last_peered":"2023-07-06T03:48:45.939422+0000","last_clean":"2023-07-06T03:48:45.939422+0000","last_became_active":"2023-07-06T03:48:44.924922+0000","last_became_peered":"2023-07-06T03:48:44.924922+0000","last_unstale":"2023-07-06T03:48:45.939422+0000","last_undegraded":"2023-07-06T03:48:45.939422+0000","last_fullsized":"2023-07-06T03:48:45.939422+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:46:17.342988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.928656+0000","last_change":"2023-07-06T03:48:45.920827+0000","last_active":"2023-07-06T03:48:45.928656+0000","last_peered":"2023-07-06T03:48:45.928656+0000","last_clean":"2023-07-06T03:48:45.928656+0000","last_became_active":"2023-07-06T03:48:45.920546+0000","last_became_peered":"2023-07-06T03:48:45.920546+0000","last_unstale":"2023-07-06T03:48:45.928656+0000","last_undegraded":"2023-07-06T03:48:45.928656+0000","last_fullsized":"2023-07-06T03:48:45.928656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:30.784151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.918957+0000","last_change":"2023-07-06T03:48:44.923265+0000","last_active":"2023-07-06T03:48:45.918957+0000","last_peered":"2023-07-06T03:48:45.918957+0000","last_clean":"2023-07-06T03:48:45.918957+0000","last_became_active":"2023-07-06T03:48:44.923008+0000","last_became_peered":"2023-07-06T03:48:44.923008+0000","last_unstale":"2023-07-06T03:48:45.918957+0000","last_undegraded":"2023-07-06T03:48:45.918957+0000","last_fullsized":"2023-07-06T03:48:45.918957+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:55:49.403296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.927877+0000","last_change":"2023-07-06T03:48:45.924929+0000","last_active":"2023-07-06T03:48:45.927877+0000","last_peered":"2023-07-06T03:48:45.927877+0000","last_clean":"2023-07-06T03:48:45.927877+0000","last_became_active":"2023-07-06T03:48:45.924530+0000","last_became_peered":"2023-07-06T03:48:45.924530+0000","last_unstale":"2023-07-06T03:48:45.927877+0000","last_undegraded":"2023-07-06T03:48:45.927877+0000","last_fullsized":"2023-07-06T03:48:45.927877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:47:09.763176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.916718+0000","last_change":"2023-07-06T03:48:44.926871+0000","last_active":"2023-07-06T03:48:45.916718+0000","last_peered":"2023-07-06T03:48:45.916718+0000","last_clean":"2023-07-06T03:48:45.916718+0000","last_became_active":"2023-07-06T03:48:44.926605+0000","last_became_peered":"2023-07-06T03:48:44.926605+0000","last_unstale":"2023-07-06T03:48:45.916718+0000","last_undegraded":"2023-07-06T03:48:45.916718+0000","last_fullsized":"2023-07-06T03:48:45.916718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:29:27.653969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937761+0000","last_change":"2023-07-06T03:48:45.930632+0000","last_active":"2023-07-06T03:48:45.937761+0000","last_peered":"2023-07-06T03:48:45.937761+0000","last_clean":"2023-07-06T03:48:45.937761+0000","last_became_active":"2023-07-06T03:48:45.930285+0000","last_became_peered":"2023-07-06T03:48:45.930285+0000","last_unstale":"2023-07-06T03:48:45.937761+0000","last_undegraded":"2023-07-06T03:48:45.937761+0000","last_fullsized":"2023-07-06T03:48:45.937761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:54:45.278281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.916227+0000","last_change":"2023-07-06T03:48:44.928020+0000","last_active":"2023-07-06T03:48:45.916227+0000","last_peered":"2023-07-06T03:48:45.916227+0000","last_clean":"2023-07-06T03:48:45.916227+0000","last_became_active":"2023-07-06T03:48:44.927772+0000","last_became_peered":"2023-07-06T03:48:44.927772+0000","last_unstale":"2023-07-06T03:48:45.916227+0000","last_undegraded":"2023-07-06T03:48:45.916227+0000","last_fullsized":"2023-07-06T03:48:45.916227+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:00.554009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937204+0000","last_change":"2023-07-06T03:48:45.929325+0000","last_active":"2023-07-06T03:48:45.937204+0000","last_peered":"2023-07-06T03:48:45.937204+0000","last_clean":"2023-07-06T03:48:45.937204+0000","last_became_active":"2023-07-06T03:48:45.929031+0000","last_became_peered":"2023-07-06T03:48:45.929031+0000","last_unstale":"2023-07-06T03:48:45.937204+0000","last_undegraded":"2023-07-06T03:48:45.937204+0000","last_fullsized":"2023-07-06T03:48:45.937204+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:46:11.766517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.925011+0000","last_change":"2023-07-06T03:48:44.927012+0000","last_active":"2023-07-06T03:48:45.925011+0000","last_peered":"2023-07-06T03:48:45.925011+0000","last_clean":"2023-07-06T03:48:45.925011+0000","last_became_active":"2023-07-06T03:48:44.926719+0000","last_became_peered":"2023-07-06T03:48:44.926719+0000","last_unstale":"2023-07-06T03:48:45.925011+0000","last_undegraded":"2023-07-06T03:48:45.925011+0000","last_fullsized":"2023-07-06T03:48:45.925011+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:31:10.982994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.941899+0000","last_change":"2023-07-06T03:48:45.936935+0000","last_active":"2023-07-06T03:48:45.941899+0000","last_peered":"2023-07-06T03:48:45.941899+0000","last_clean":"2023-07-06T03:48:45.941899+0000","last_became_active":"2023-07-06T03:48:45.936327+0000","last_became_peered":"2023-07-06T03:48:45.936327+0000","last_unstale":"2023-07-06T03:48:45.941899+0000","last_undegraded":"2023-07-06T03:48:45.941899+0000","last_fullsized":"2023-07-06T03:48:45.941899+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:39:32.023179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.939316+0000","last_change":"2023-07-06T03:48:44.924386+0000","last_active":"2023-07-06T03:48:45.939316+0000","last_peered":"2023-07-06T03:48:45.939316+0000","last_clean":"2023-07-06T03:48:45.939316+0000","last_became_active":"2023-07-06T03:48:44.924089+0000","last_became_peered":"2023-07-06T03:48:44.924089+0000","last_unstale":"2023-07-06T03:48:45.939316+0000","last_undegraded":"2023-07-06T03:48:45.939316+0000","last_fullsized":"2023-07-06T03:48:45.939316+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:14:37.992686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.923624+0000","last_change":"2023-07-06T03:48:45.914921+0000","last_active":"2023-07-06T03:48:45.923624+0000","last_peered":"2023-07-06T03:48:45.923624+0000","last_clean":"2023-07-06T03:48:45.923624+0000","last_became_active":"2023-07-06T03:48:45.914606+0000","last_became_peered":"2023-07-06T03:48:45.914606+0000","last_unstale":"2023-07-06T03:48:45.923624+0000","last_undegraded":"2023-07-06T03:48:45.923624+0000","last_fullsized":"2023-07-06T03:48:45.923624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:25.409027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"59'6","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945452+0000","last_change":"2023-07-06T03:48:44.923668+0000","last_active":"2023-07-06T03:48:45.945452+0000","last_peered":"2023-07-06T03:48:45.945452+0000","last_clean":"2023-07-06T03:48:45.945452+0000","last_became_active":"2023-07-06T03:48:44.923268+0000","last_became_peered":"2023-07-06T03:48:44.923268+0000","last_unstale":"2023-07-06T03:48:45.945452+0000","last_undegraded":"2023-07-06T03:48:45.945452+0000","last_fullsized":"2023-07-06T03:48:45.945452+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:49:50.252456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.928690+0000","last_change":"2023-07-06T03:48:45.920070+0000","last_active":"2023-07-06T03:48:45.928690+0000","last_peered":"2023-07-06T03:48:45.928690+0000","last_clean":"2023-07-06T03:48:45.928690+0000","last_became_active":"2023-07-06T03:48:45.919318+0000","last_became_peered":"2023-07-06T03:48:45.919318+0000","last_unstale":"2023-07-06T03:48:45.928690+0000","last_undegraded":"2023-07-06T03:48:45.928690+0000","last_fullsized":"2023-07-06T03:48:45.928690+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:46:22.384643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.946288+0000","last_change":"2023-07-06T03:48:44.927623+0000","last_active":"2023-07-06T03:48:45.946288+0000","last_peered":"2023-07-06T03:48:45.946288+0000","last_clean":"2023-07-06T03:48:45.946288+0000","last_became_active":"2023-07-06T03:48:44.927354+0000","last_became_peered":"2023-07-06T03:48:44.927354+0000","last_unstale":"2023-07-06T03:48:45.946288+0000","last_undegraded":"2023-07-06T03:48:45.946288+0000","last_fullsized":"2023-07-06T03:48:45.946288+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:21:39.443238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.928473+0000","last_change":"2023-07-06T03:48:45.926163+0000","last_active":"2023-07-06T03:48:45.928473+0000","last_peered":"2023-07-06T03:48:45.928473+0000","last_clean":"2023-07-06T03:48:45.928473+0000","last_became_active":"2023-07-06T03:48:45.925903+0000","last_became_peered":"2023-07-06T03:48:45.925903+0000","last_unstale":"2023-07-06T03:48:45.928473+0000","last_undegraded":"2023-07-06T03:48:45.928473+0000","last_fullsized":"2023-07-06T03:48:45.928473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:03:21.355033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.911639+0000","last_change":"2023-07-06T03:48:44.915434+0000","last_active":"2023-07-06T03:48:45.911639+0000","last_peered":"2023-07-06T03:48:45.911639+0000","last_clean":"2023-07-06T03:48:45.911639+0000","last_became_active":"2023-07-06T03:48:44.915151+0000","last_became_peered":"2023-07-06T03:48:44.915151+0000","last_unstale":"2023-07-06T03:48:45.911639+0000","last_undegraded":"2023-07-06T03:48:45.911639+0000","last_fullsized":"2023-07-06T03:48:45.911639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:22:43.026243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.934946+0000","last_change":"2023-07-06T03:48:45.929492+0000","last_active":"2023-07-06T03:48:45.934946+0000","last_peered":"2023-07-06T03:48:45.934946+0000","last_clean":"2023-07-06T03:48:45.934946+0000","last_became_active":"2023-07-06T03:48:45.929237+0000","last_became_peered":"2023-07-06T03:48:45.929237+0000","last_unstale":"2023-07-06T03:48:45.934946+0000","last_undegraded":"2023-07-06T03:48:45.934946+0000","last_fullsized":"2023-07-06T03:48:45.934946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:48.452424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.909611+0000","last_change":"2023-07-06T03:48:44.920425+0000","last_active":"2023-07-06T03:48:45.909611+0000","last_peered":"2023-07-06T03:48:45.909611+0000","last_clean":"2023-07-06T03:48:45.909611+0000","last_became_active":"2023-07-06T03:48:44.920158+0000","last_became_peered":"2023-07-06T03:48:44.920158+0000","last_unstale":"2023-07-06T03:48:45.909611+0000","last_undegraded":"2023-07-06T03:48:45.909611+0000","last_fullsized":"2023-07-06T03:48:45.909611+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:54:31.066629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938042+0000","last_change":"2023-07-06T03:48:45.929982+0000","last_active":"2023-07-06T03:48:45.938042+0000","last_peered":"2023-07-06T03:48:45.938042+0000","last_clean":"2023-07-06T03:48:45.938042+0000","last_became_active":"2023-07-06T03:48:45.929688+0000","last_became_peered":"2023-07-06T03:48:45.929688+0000","last_unstale":"2023-07-06T03:48:45.938042+0000","last_undegraded":"2023-07-06T03:48:45.938042+0000","last_fullsized":"2023-07-06T03:48:45.938042+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:20.615059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.943073+0000","last_change":"2023-07-06T03:48:44.925926+0000","last_active":"2023-07-06T03:48:45.943073+0000","last_peered":"2023-07-06T03:48:45.943073+0000","last_clean":"2023-07-06T03:48:45.943073+0000","last_became_active":"2023-07-06T03:48:44.925613+0000","last_became_peered":"2023-07-06T03:48:44.925613+0000","last_unstale":"2023-07-06T03:48:45.943073+0000","last_undegraded":"2023-07-06T03:48:45.943073+0000","last_fullsized":"2023-07-06T03:48:45.943073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:42:50.484259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937213+0000","last_change":"2023-07-06T03:48:45.931421+0000","last_active":"2023-07-06T03:48:45.937213+0000","last_peered":"2023-07-06T03:48:45.937213+0000","last_clean":"2023-07-06T03:48:45.937213+0000","last_became_active":"2023-07-06T03:48:45.931136+0000","last_became_peered":"2023-07-06T03:48:45.931136+0000","last_unstale":"2023-07-06T03:48:45.937213+0000","last_undegraded":"2023-07-06T03:48:45.937213+0000","last_fullsized":"2023-07-06T03:48:45.937213+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:06.097637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945684+0000","last_change":"2023-07-06T03:48:44.926876+0000","last_active":"2023-07-06T03:48:45.945684+0000","last_peered":"2023-07-06T03:48:45.945684+0000","last_clean":"2023-07-06T03:48:45.945684+0000","last_became_active":"2023-07-06T03:48:44.926612+0000","last_became_peered":"2023-07-06T03:48:44.926612+0000","last_unstale":"2023-07-06T03:48:45.945684+0000","last_undegraded":"2023-07-06T03:48:45.945684+0000","last_fullsized":"2023-07-06T03:48:45.945684+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:59.283071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937226+0000","last_change":"2023-07-06T03:48:45.931154+0000","last_active":"2023-07-06T03:48:45.937226+0000","last_peered":"2023-07-06T03:48:45.937226+0000","last_clean":"2023-07-06T03:48:45.937226+0000","last_became_active":"2023-07-06T03:48:45.930865+0000","last_became_peered":"2023-07-06T03:48:45.930865+0000","last_unstale":"2023-07-06T03:48:45.937226+0000","last_undegraded":"2023-07-06T03:48:45.937226+0000","last_fullsized":"2023-07-06T03:48:45.937226+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:05.078262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.915713+0000","last_change":"2023-07-06T03:48:44.920013+0000","last_active":"2023-07-06T03:48:45.915713+0000","last_peered":"2023-07-06T03:48:45.915713+0000","last_clean":"2023-07-06T03:48:45.915713+0000","last_became_active":"2023-07-06T03:48:44.919761+0000","last_became_peered":"2023-07-06T03:48:44.919761+0000","last_unstale":"2023-07-06T03:48:45.915713+0000","last_undegraded":"2023-07-06T03:48:45.915713+0000","last_fullsized":"2023-07-06T03:48:45.915713+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:41:47.704928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.924449+0000","last_change":"2023-07-06T03:48:45.920246+0000","last_active":"2023-07-06T03:48:45.924449+0000","last_peered":"2023-07-06T03:48:45.924449+0000","last_clean":"2023-07-06T03:48:45.924449+0000","last_became_active":"2023-07-06T03:48:45.919918+0000","last_became_peered":"2023-07-06T03:48:45.919918+0000","last_unstale":"2023-07-06T03:48:45.924449+0000","last_undegraded":"2023-07-06T03:48:45.924449+0000","last_fullsized":"2023-07-06T03:48:45.924449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:44:18.117762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.919398+0000","last_change":"2023-07-06T03:48:44.926131+0000","last_active":"2023-07-06T03:48:45.919398+0000","last_peered":"2023-07-06T03:48:45.919398+0000","last_clean":"2023-07-06T03:48:45.919398+0000","last_became_active":"2023-07-06T03:48:44.925790+0000","last_became_peered":"2023-07-06T03:48:44.925790+0000","last_unstale":"2023-07-06T03:48:45.919398+0000","last_undegraded":"2023-07-06T03:48:45.919398+0000","last_fullsized":"2023-07-06T03:48:45.919398+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:09:06.481520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.935833+0000","last_change":"2023-07-06T03:48:45.926258+0000","last_active":"2023-07-06T03:48:45.935833+0000","last_peered":"2023-07-06T03:48:45.935833+0000","last_clean":"2023-07-06T03:48:45.935833+0000","last_became_active":"2023-07-06T03:48:45.925754+0000","last_became_peered":"2023-07-06T03:48:45.925754+0000","last_unstale":"2023-07-06T03:48:45.935833+0000","last_undegraded":"2023-07-06T03:48:45.935833+0000","last_fullsized":"2023-07-06T03:48:45.935833+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:07:36.302648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.908693+0000","last_change":"2023-07-06T03:48:44.923656+0000","last_active":"2023-07-06T03:48:45.908693+0000","last_peered":"2023-07-06T03:48:45.908693+0000","last_clean":"2023-07-06T03:48:45.908693+0000","last_became_active":"2023-07-06T03:48:44.923372+0000","last_became_peered":"2023-07-06T03:48:44.923372+0000","last_unstale":"2023-07-06T03:48:45.908693+0000","last_undegraded":"2023-07-06T03:48:45.908693+0000","last_fullsized":"2023-07-06T03:48:45.908693+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:29.775291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.935461+0000","last_change":"2023-07-06T03:48:45.930046+0000","last_active":"2023-07-06T03:48:45.935461+0000","last_peered":"2023-07-06T03:48:45.935461+0000","last_clean":"2023-07-06T03:48:45.935461+0000","last_became_active":"2023-07-06T03:48:45.929799+0000","last_became_peered":"2023-07-06T03:48:45.929799+0000","last_unstale":"2023-07-06T03:48:45.935461+0000","last_undegraded":"2023-07-06T03:48:45.935461+0000","last_fullsized":"2023-07-06T03:48:45.935461+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:26:00.176042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.924647+0000","last_change":"2023-07-06T03:48:44.924367+0000","last_active":"2023-07-06T03:48:45.924647+0000","last_peered":"2023-07-06T03:48:45.924647+0000","last_clean":"2023-07-06T03:48:45.924647+0000","last_became_active":"2023-07-06T03:48:44.924124+0000","last_became_peered":"2023-07-06T03:48:44.924124+0000","last_unstale":"2023-07-06T03:48:45.924647+0000","last_undegraded":"2023-07-06T03:48:45.924647+0000","last_fullsized":"2023-07-06T03:48:45.924647+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:29.668635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.947104+0000","last_change":"2023-07-06T03:48:45.944422+0000","last_active":"2023-07-06T03:48:45.947104+0000","last_peered":"2023-07-06T03:48:45.947104+0000","last_clean":"2023-07-06T03:48:45.947104+0000","last_became_active":"2023-07-06T03:48:45.944153+0000","last_became_peered":"2023-07-06T03:48:45.944153+0000","last_unstale":"2023-07-06T03:48:45.947104+0000","last_undegraded":"2023-07-06T03:48:45.947104+0000","last_fullsized":"2023-07-06T03:48:45.947104+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:49:34.398220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.918231+0000","last_change":"2023-07-06T03:48:44.913655+0000","last_active":"2023-07-06T03:48:45.918231+0000","last_peered":"2023-07-06T03:48:45.918231+0000","last_clean":"2023-07-06T03:48:45.918231+0000","last_became_active":"2023-07-06T03:48:44.913413+0000","last_became_peered":"2023-07-06T03:48:44.913413+0000","last_unstale":"2023-07-06T03:48:45.918231+0000","last_undegraded":"2023-07-06T03:48:45.918231+0000","last_fullsized":"2023-07-06T03:48:45.918231+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:57:15.487713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.946252+0000","last_change":"2023-07-06T03:48:45.943008+0000","last_active":"2023-07-06T03:48:45.946252+0000","last_peered":"2023-07-06T03:48:45.946252+0000","last_clean":"2023-07-06T03:48:45.946252+0000","last_became_active":"2023-07-06T03:48:45.942719+0000","last_became_peered":"2023-07-06T03:48:45.942719+0000","last_unstale":"2023-07-06T03:48:45.946252+0000","last_undegraded":"2023-07-06T03:48:45.946252+0000","last_fullsized":"2023-07-06T03:48:45.946252+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:50.819405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.908476+0000","last_change":"2023-07-06T03:48:44.916132+0000","last_active":"2023-07-06T03:48:45.908476+0000","last_peered":"2023-07-06T03:48:45.908476+0000","last_clean":"2023-07-06T03:48:45.908476+0000","last_became_active":"2023-07-06T03:48:44.915846+0000","last_became_peered":"2023-07-06T03:48:44.915846+0000","last_unstale":"2023-07-06T03:48:45.908476+0000","last_undegraded":"2023-07-06T03:48:45.908476+0000","last_fullsized":"2023-07-06T03:48:45.908476+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:09:25.150637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.926572+0000","last_change":"2023-07-06T03:48:45.922015+0000","last_active":"2023-07-06T03:48:45.926572+0000","last_peered":"2023-07-06T03:48:45.926572+0000","last_clean":"2023-07-06T03:48:45.926572+0000","last_became_active":"2023-07-06T03:48:45.921635+0000","last_became_peered":"2023-07-06T03:48:45.921635+0000","last_unstale":"2023-07-06T03:48:45.926572+0000","last_undegraded":"2023-07-06T03:48:45.926572+0000","last_fullsized":"2023-07-06T03:48:45.926572+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:39:12.633544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.917807+0000","last_change":"2023-07-06T03:48:44.926204+0000","last_active":"2023-07-06T03:48:45.917807+0000","last_peered":"2023-07-06T03:48:45.917807+0000","last_clean":"2023-07-06T03:48:45.917807+0000","last_became_active":"2023-07-06T03:48:44.925899+0000","last_became_peered":"2023-07-06T03:48:44.925899+0000","last_unstale":"2023-07-06T03:48:45.917807+0000","last_undegraded":"2023-07-06T03:48:45.917807+0000","last_fullsized":"2023-07-06T03:48:45.917807+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:17:17.815231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945926+0000","last_change":"2023-07-06T03:48:45.942472+0000","last_active":"2023-07-06T03:48:45.945926+0000","last_peered":"2023-07-06T03:48:45.945926+0000","last_clean":"2023-07-06T03:48:45.945926+0000","last_became_active":"2023-07-06T03:48:45.942184+0000","last_became_peered":"2023-07-06T03:48:45.942184+0000","last_unstale":"2023-07-06T03:48:45.945926+0000","last_undegraded":"2023-07-06T03:48:45.945926+0000","last_fullsized":"2023-07-06T03:48:45.945926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:16.887206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.941690+0000","last_change":"2023-07-06T03:48:44.926031+0000","last_active":"2023-07-06T03:48:45.941690+0000","last_peered":"2023-07-06T03:48:45.941690+0000","last_clean":"2023-07-06T03:48:45.941690+0000","last_became_active":"2023-07-06T03:48:44.925752+0000","last_became_peered":"2023-07-06T03:48:44.925752+0000","last_unstale":"2023-07-06T03:48:45.941690+0000","last_undegraded":"2023-07-06T03:48:45.941690+0000","last_fullsized":"2023-07-06T03:48:45.941690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:23:25.221363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.925902+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:48:45.925902+0000","last_peered":"2023-07-06T03:48:45.925902+0000","last_clean":"2023-07-06T03:48:45.925902+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:48:45.925902+0000","last_undegraded":"2023-07-06T03:48:45.925902+0000","last_fullsized":"2023-07-06T03:48:45.925902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:11:23.347400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.941111+0000","last_change":"2023-07-06T03:48:45.935497+0000","last_active":"2023-07-06T03:48:45.941111+0000","last_peered":"2023-07-06T03:48:45.941111+0000","last_clean":"2023-07-06T03:48:45.941111+0000","last_became_active":"2023-07-06T03:48:45.935181+0000","last_became_peered":"2023-07-06T03:48:45.935181+0000","last_unstale":"2023-07-06T03:48:45.941111+0000","last_undegraded":"2023-07-06T03:48:45.941111+0000","last_fullsized":"2023-07-06T03:48:45.941111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:26:37.641808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.917660+0000","last_change":"2023-07-06T03:48:44.926111+0000","last_active":"2023-07-06T03:48:45.917660+0000","last_peered":"2023-07-06T03:48:45.917660+0000","last_clean":"2023-07-06T03:48:45.917660+0000","last_became_active":"2023-07-06T03:48:44.925856+0000","last_became_peered":"2023-07-06T03:48:44.925856+0000","last_unstale":"2023-07-06T03:48:45.917660+0000","last_undegraded":"2023-07-06T03:48:45.917660+0000","last_fullsized":"2023-07-06T03:48:45.917660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:31:53.312131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.908581+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:48:45.908581+0000","last_peered":"2023-07-06T03:48:45.908581+0000","last_clean":"2023-07-06T03:48:45.908581+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:48:45.908581+0000","last_undegraded":"2023-07-06T03:48:45.908581+0000","last_fullsized":"2023-07-06T03:48:45.908581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:51:33.596463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.939068+0000","last_change":"2023-07-06T03:48:45.934398+0000","last_active":"2023-07-06T03:48:45.939068+0000","last_peered":"2023-07-06T03:48:45.939068+0000","last_clean":"2023-07-06T03:48:45.939068+0000","last_became_active":"2023-07-06T03:48:45.934139+0000","last_became_peered":"2023-07-06T03:48:45.934139+0000","last_unstale":"2023-07-06T03:48:45.939068+0000","last_undegraded":"2023-07-06T03:48:45.939068+0000","last_fullsized":"2023-07-06T03:48:45.939068+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:39:35.277083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.911718+0000","last_change":"2023-07-06T03:48:44.922169+0000","last_active":"2023-07-06T03:48:45.911718+0000","last_peered":"2023-07-06T03:48:45.911718+0000","last_clean":"2023-07-06T03:48:45.911718+0000","last_became_active":"2023-07-06T03:48:44.920785+0000","last_became_peered":"2023-07-06T03:48:44.920785+0000","last_unstale":"2023-07-06T03:48:45.911718+0000","last_undegraded":"2023-07-06T03:48:45.911718+0000","last_fullsized":"2023-07-06T03:48:45.911718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:08:27.303274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":90,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.908605+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:48:45.908605+0000","last_peered":"2023-07-06T03:48:45.908605+0000","last_clean":"2023-07-06T03:48:45.908605+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:48:45.908605+0000","last_undegraded":"2023-07-06T03:48:45.908605+0000","last_fullsized":"2023-07-06T03:48:45.908605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:19:03.256816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.930655+0000","last_change":"2023-07-06T03:48:45.926396+0000","last_active":"2023-07-06T03:48:45.930655+0000","last_peered":"2023-07-06T03:48:45.930655+0000","last_clean":"2023-07-06T03:48:45.930655+0000","last_became_active":"2023-07-06T03:48:45.926108+0000","last_became_peered":"2023-07-06T03:48:45.926108+0000","last_unstale":"2023-07-06T03:48:45.930655+0000","last_undegraded":"2023-07-06T03:48:45.930655+0000","last_fullsized":"2023-07-06T03:48:45.930655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:27:44.219454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.918099+0000","last_change":"2023-07-06T03:48:44.924030+0000","last_active":"2023-07-06T03:48:45.918099+0000","last_peered":"2023-07-06T03:48:45.918099+0000","last_clean":"2023-07-06T03:48:45.918099+0000","last_became_active":"2023-07-06T03:48:44.923723+0000","last_became_peered":"2023-07-06T03:48:44.923723+0000","last_unstale":"2023-07-06T03:48:45.918099+0000","last_undegraded":"2023-07-06T03:48:45.918099+0000","last_fullsized":"2023-07-06T03:48:45.918099+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:24:57.134005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":84,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.909174+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:48:45.909174+0000","last_peered":"2023-07-06T03:48:45.909174+0000","last_clean":"2023-07-06T03:48:45.909174+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:48:45.909174+0000","last_undegraded":"2023-07-06T03:48:45.909174+0000","last_fullsized":"2023-07-06T03:48:45.909174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:57:05.254201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":228,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.912188+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:48:45.912188+0000","last_peered":"2023-07-06T03:48:45.912188+0000","last_clean":"2023-07-06T03:48:45.912188+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:48:45.912188+0000","last_undegraded":"2023-07-06T03:48:45.912188+0000","last_fullsized":"2023-07-06T03:48:45.912188+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:25:03.097211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937918+0000","last_change":"2023-07-06T03:48:45.936192+0000","last_active":"2023-07-06T03:48:45.937918+0000","last_peered":"2023-07-06T03:48:45.937918+0000","last_clean":"2023-07-06T03:48:45.937918+0000","last_became_active":"2023-07-06T03:48:45.935893+0000","last_became_peered":"2023-07-06T03:48:45.935893+0000","last_unstale":"2023-07-06T03:48:45.937918+0000","last_undegraded":"2023-07-06T03:48:45.937918+0000","last_fullsized":"2023-07-06T03:48:45.937918+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:09:38.787882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.943942+0000","last_change":"2023-07-06T03:48:44.928006+0000","last_active":"2023-07-06T03:48:45.943942+0000","last_peered":"2023-07-06T03:48:45.943942+0000","last_clean":"2023-07-06T03:48:45.943942+0000","last_became_active":"2023-07-06T03:48:44.927709+0000","last_became_peered":"2023-07-06T03:48:44.927709+0000","last_unstale":"2023-07-06T03:48:45.943942+0000","last_undegraded":"2023-07-06T03:48:45.943942+0000","last_fullsized":"2023-07-06T03:48:45.943942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:42:32.328554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":84,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.909574+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:48:45.909574+0000","last_peered":"2023-07-06T03:48:45.909574+0000","last_clean":"2023-07-06T03:48:45.909574+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:48:45.909574+0000","last_undegraded":"2023-07-06T03:48:45.909574+0000","last_fullsized":"2023-07-06T03:48:45.909574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:35.550657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.944921+0000","last_change":"2023-07-06T03:48:45.939012+0000","last_active":"2023-07-06T03:48:45.944921+0000","last_peered":"2023-07-06T03:48:45.944921+0000","last_clean":"2023-07-06T03:48:45.944921+0000","last_became_active":"2023-07-06T03:48:45.938715+0000","last_became_peered":"2023-07-06T03:48:45.938715+0000","last_unstale":"2023-07-06T03:48:45.944921+0000","last_undegraded":"2023-07-06T03:48:45.944921+0000","last_fullsized":"2023-07-06T03:48:45.944921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:52.452649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.941118+0000","last_change":"2023-07-06T03:48:44.926366+0000","last_active":"2023-07-06T03:48:45.941118+0000","last_peered":"2023-07-06T03:48:45.941118+0000","last_clean":"2023-07-06T03:48:45.941118+0000","last_became_active":"2023-07-06T03:48:44.926115+0000","last_became_peered":"2023-07-06T03:48:44.926115+0000","last_unstale":"2023-07-06T03:48:45.941118+0000","last_undegraded":"2023-07-06T03:48:45.941118+0000","last_fullsized":"2023-07-06T03:48:45.941118+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:21:38.432585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":84,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.910049+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:48:45.910049+0000","last_peered":"2023-07-06T03:48:45.910049+0000","last_clean":"2023-07-06T03:48:45.910049+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:48:45.910049+0000","last_undegraded":"2023-07-06T03:48:45.910049+0000","last_fullsized":"2023-07-06T03:48:45.910049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:46:14.068727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.940703+0000","last_change":"2023-07-06T03:48:45.936727+0000","last_active":"2023-07-06T03:48:45.940703+0000","last_peered":"2023-07-06T03:48:45.940703+0000","last_clean":"2023-07-06T03:48:45.940703+0000","last_became_active":"2023-07-06T03:48:45.935957+0000","last_became_peered":"2023-07-06T03:48:45.935957+0000","last_unstale":"2023-07-06T03:48:45.940703+0000","last_undegraded":"2023-07-06T03:48:45.940703+0000","last_fullsized":"2023-07-06T03:48:45.940703+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:30:05.349113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.942193+0000","last_change":"2023-07-06T03:48:44.923503+0000","last_active":"2023-07-06T03:48:45.942193+0000","last_peered":"2023-07-06T03:48:45.942193+0000","last_clean":"2023-07-06T03:48:45.942193+0000","last_became_active":"2023-07-06T03:48:44.923200+0000","last_became_peered":"2023-07-06T03:48:44.923200+0000","last_unstale":"2023-07-06T03:48:45.942193+0000","last_undegraded":"2023-07-06T03:48:45.942193+0000","last_fullsized":"2023-07-06T03:48:45.942193+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:13.093100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.912739+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:48:45.912739+0000","last_peered":"2023-07-06T03:48:45.912739+0000","last_clean":"2023-07-06T03:48:45.912739+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:48:45.912739+0000","last_undegraded":"2023-07-06T03:48:45.912739+0000","last_fullsized":"2023-07-06T03:48:45.912739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:49:25.633769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.935853+0000","last_change":"2023-07-06T03:48:45.926381+0000","last_active":"2023-07-06T03:48:45.935853+0000","last_peered":"2023-07-06T03:48:45.935853+0000","last_clean":"2023-07-06T03:48:45.935853+0000","last_became_active":"2023-07-06T03:48:45.926115+0000","last_became_peered":"2023-07-06T03:48:45.926115+0000","last_unstale":"2023-07-06T03:48:45.935853+0000","last_undegraded":"2023-07-06T03:48:45.935853+0000","last_fullsized":"2023-07-06T03:48:45.935853+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:49.290541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.908121+0000","last_change":"2023-07-06T03:48:44.926109+0000","last_active":"2023-07-06T03:48:45.908121+0000","last_peered":"2023-07-06T03:48:45.908121+0000","last_clean":"2023-07-06T03:48:45.908121+0000","last_became_active":"2023-07-06T03:48:44.925836+0000","last_became_peered":"2023-07-06T03:48:44.925836+0000","last_unstale":"2023-07-06T03:48:45.908121+0000","last_undegraded":"2023-07-06T03:48:45.908121+0000","last_fullsized":"2023-07-06T03:48:45.908121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:47.443564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":84,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.909947+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:48:45.909947+0000","last_peered":"2023-07-06T03:48:45.909947+0000","last_clean":"2023-07-06T03:48:45.909947+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:48:45.909947+0000","last_undegraded":"2023-07-06T03:48:45.909947+0000","last_fullsized":"2023-07-06T03:48:45.909947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:43:49.173497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":696320,"data_stored":303962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.69299999999999995,"15min":0.72199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.874,"5min":1.345,"15min":1.345},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.77100000000000002,"15min":0.82299999999999995},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.68899999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.72299999999999998,"15min":0.755},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85599999999999998,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.85399999999999998,"15min":0.93799999999999994},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90800000000000003,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78400000000000003,"15min":0.83399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88300000000000001,"5min":3.77,"15min":3.77},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72999999999999998,"15min":0.74099999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84399999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.501}]},{"osd":3,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83399999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.024},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94399999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.375,"5min":0.374,"15min":0.374},"max":{"1min":0.93000000000000005,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46700000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.96699999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.68500000000000005,"15min":0.67400000000000004},"min":{"1min":0.54200000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75,"15min":0.747},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.026,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79500000000000004,"15min":0.81100000000000005},"min":{"1min":0.44800000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98899999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.64700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":696320,"data_stored":303962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65100000000000002,"15min":0.65400000000000003},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.89100000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86899999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66200000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65900000000000003,"15min":0.67100000000000004},"min":{"1min":0.44500000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89200000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65700000000000003,"15min":0.67900000000000005},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79100000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.624}]},{"osd":2,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63900000000000001,"15min":0.65500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.872,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69899999999999995,"15min":0.72099999999999997},"min":{"1min":0.46500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.93999999999999995,"5min":1.119,"15min":1.119},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73999999999999999,"15min":0.77800000000000002},"min":{"1min":0.48599999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.84999999999999998,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71299999999999997,"15min":0.746},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.81899999999999995,"5min":1.093,"15min":1.093},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96999999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.75800000000000001},"min":{"1min":0.46700000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88300000000000001,"5min":1.085,"15min":1.085},"last":0.748}]},{"osd":5,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70199999999999996,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84399999999999997,"5min":1.218,"15min":1.218},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.52600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89000000000000001,"5min":1.198,"15min":1.198},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81999999999999995,"15min":0.85899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94799999999999995,"5min":1.323,"15min":1.323},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.82199999999999995,"15min":0.86799999999999999},"min":{"1min":0.51100000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.028,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1290240,"data_stored":894330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.52100000000000002,"15min":0.58399999999999996},"min":{"1min":0.27800000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60099999999999998,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.55600000000000005,"15min":0.59799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67900000000000005,"5min":1.96,"15min":1.96},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.873,"15min":1.069},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.626,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.61899999999999999,"15min":0.67200000000000004},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77700000000000002,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.59799999999999998,"15min":0.64400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79300000000000004,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53800000000000003,"15min":0.56899999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82599999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.34599999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":1.139,"15min":1.329},"min":{"1min":0.51600000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89900000000000002,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.82099999999999995,"15min":0.85399999999999998},"min":{"1min":0.54100000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.371,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.76600000000000001,"15min":0.79400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97199999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":1.1439999999999999,"15min":1.3440000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89800000000000002,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.80600000000000005,"15min":0.83999999999999997},"min":{"1min":0.50600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93200000000000005,"5min":1.365,"15min":1.365},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.745,"5min":1.1859999999999999,"15min":1.387},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.254,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.81699999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.86299999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.4690000000000001,"5min":2.806,"15min":2.806},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.80600000000000005,"15min":0.83799999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.006,"5min":1.337,"15min":1.337},"last":0.72799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":708608,"data_stored":311090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.436,"15min":0.438},"min":{"1min":0.28199999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.57299999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.434}]},{"osd":2,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46800000000000003,"15min":0.47699999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79400000000000004,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.34100000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71199999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.378},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70599999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.73299999999999998,"15min":0.755},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96299999999999997,"5min":1.171,"15min":1.171},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.60199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.875,"5min":1.034,"15min":1.034},"last":0.70799999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.879,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94499999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.96999999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85999999999999999,"5min":1.546,"15min":1.546},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.79600000000000004,"15min":0.81399999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.93200000000000005,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":1.042,"15min":1.1910000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.88100000000000001,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.80000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35824,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336016,"statfs":{"total":96636764160,"available":96600080384,"internally_reserved":0,"allocated":696320,"data_stored":303962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.32100000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.623,"5min":0.626,"15min":0.626},"last":0.627},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.47199999999999998},"min":{"1min":0.31,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.40000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.58699999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.505,"15min":0.499},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82699999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66600000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.76300000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0469999999999999,"5min":1.165,"15min":1.165},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.878}]},{"osd":5,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.753,"15min":0.754},"min":{"1min":0.60699999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0249999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.57799999999999996,"5min":0.5,"15min":0.5},"max":{"1min":1.002,"5min":1.046,"15min":1.046},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.75,"15min":0.75},"min":{"1min":0.58899999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.034,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.99199999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.81299999999999994,"15min":0.83599999999999997},"min":{"1min":0.59099999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0680000000000001,"5min":2.157,"15min":2.157},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.57299999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0369999999999999,"5min":1.966,"15min":1.966},"last":0.84399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":696320,"data_stored":303962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.58899999999999997,"15min":0.67500000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51300000000000001,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.502,"15min":0.51900000000000002},"min":{"1min":0.37,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.54500000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.999,"15min":1.2609999999999999},"min":{"1min":0.32200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.54700000000000004,"5min":15.911,"15min":15.911},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":1.0680000000000001,"15min":1.3300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.65200000000000002,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.48799999999999999,"15min":0.51600000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.627,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.38},{"interface":"front","average":{"1min":0.48799999999999999,"5min":1.075,"15min":1.3360000000000001},"min":{"1min":0.36899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70299999999999996,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.77300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94499999999999995,"5min":1.179,"15min":1.179},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.90600000000000003,"15min":1.004},"min":{"1min":0.504,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.875,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.86799999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.90800000000000003,"15min":1.0029999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92600000000000005,"5min":5.077,"15min":5.077},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.96099999999999997,"15min":1.0669999999999999},"min":{"1min":0.53500000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90100000000000002,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.748}]},{"osd":6,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.78600000000000003,"15min":0.82399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.91000000000000003,"5min":1.143,"15min":1.143},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.92200000000000004,"15min":1.0209999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84799999999999998,"5min":5.157,"15min":5.157},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.81499999999999995,"15min":0.86599999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96399999999999997,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.93500000000000005,"15min":1.0309999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91100000000000003,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.83599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35824,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336016,"statfs":{"total":96636764160,"available":96600080384,"internally_reserved":0,"allocated":696320,"data_stored":303962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93400000000000005,"5min":1.087,"15min":1.087},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63300000000000001,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.83899999999999997,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.84199999999999997,"15min":0.93500000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.95399999999999996,"5min":6.258,"15min":6.258},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65400000000000003,"15min":0.65500000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91000000000000003,"5min":1.171,"15min":1.171},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.85899999999999999,"15min":0.94799999999999995},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.998,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.93700000000000006,"15min":1.0409999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.88900000000000001,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.76500000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97499999999999998,"5min":1.25,"15min":1.25},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89700000000000002,"5min":1.413,"15min":1.413},"last":0.81100000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65600000000000003,"15min":0.68100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81499999999999995,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.44},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.65200000000000002,"15min":0.68500000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77500000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.67900000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.95099999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93100000000000005,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.747,"15min":0.77100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89400000000000002,"5min":1.21,"15min":1.21},"last":0.57999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1286144,"data_stored":893311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.83599999999999997,"15min":0.93200000000000005},"min":{"1min":0.41799999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84299999999999997,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.75700000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.39800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.89800000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82999999999999996,"5min":1.546,"15min":1.546},"last":0.63400000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.68400000000000005,"15min":0.70999999999999996},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77300000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69099999999999995,"15min":0.69899999999999995},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.873,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81399999999999995,"5min":1.03,"15min":1.03},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73199999999999998,"15min":0.75800000000000001},"min":{"1min":0.496,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88200000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.625,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79200000000000004,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.505},{"interface":"front","average":{"1min":0.629,"5min":0.70499999999999996,"15min":0.71299999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95499999999999996,"5min":2.718,"15min":2.718},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93200000000000005,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.71999999999999997,"15min":0.73099999999999998},"min":{"1min":0.52100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85199999999999998,"5min":1.669,"15min":1.669},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90500000000000003,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78300000000000003,"15min":0.79500000000000004},"min":{"1min":0.52500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97699999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.70699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:48:50.295 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:48:50.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.519+0000 7f7fce5ff700 1 Processor -- start 2023-07-06T03:48:50.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.519+0000 7f7fce5ff700 1 -- start start 2023-07-06T03:48:50.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.520+0000 7f7fce5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7fc80d10b0 0x7f7fc806c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.520+0000 7f7fce5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7fc815d940 0x7f7fc806c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.520+0000 7f7fce5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc8160f70 con 0x7f7fc8160200 2023-07-06T03:48:50.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.520+0000 7f7fce5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc80d1480 con 0x7f7fc80d10b0 2023-07-06T03:48:50.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.520+0000 7f7fce5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc8160d50 con 0x7f7fc815d940 2023-07-06T03:48:50.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7fc80d10b0 0x7f7fc806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7fc80d10b0 0x7f7fc806c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49536/0 (socket says 172.21.15.97:49536) 2023-07-06T03:48:50.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fccb9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7fc8160200 0x7f7fc8160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59112/0 (socket says 172.21.15.97:59112) 2023-07-06T03:48:50.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc7fff700 1 -- 172.21.15.97:0/211486132 learned_addr learned my addr 172.21.15.97:0/211486132 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:50.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc77fe700 1 --2- 172.21.15.97:0/211486132 >> v2:172.21.15.146:3300/0 conn(0x7f7fc815d940 0x7f7fc806c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc7fff700 1 -- 172.21.15.97:0/211486132 >> v2:172.21.15.146:3300/0 conn(0x7f7fc815d940 msgr2=0x7f7fc806c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:50.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc7fff700 1 --2- 172.21.15.97:0/211486132 >> v2:172.21.15.146:3300/0 conn(0x7f7fc815d940 0x7f7fc806c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc7fff700 1 -- 172.21.15.97:0/211486132 >> v1:172.21.15.97:6789/0 conn(0x7f7fc8160200 legacy=0x7f7fc8160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:50.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc7fff700 1 -- 172.21.15.97:0/211486132 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc80d0590 con 0x7f7fc80d10b0 2023-07-06T03:48:50.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc6ffd700 1 -- 172.21.15.97:0/211486132 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936482585 0 0) 0x7f7fc8160f70 con 0x7f7fc8160200 2023-07-06T03:48:50.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc77fe700 1 --2- 172.21.15.97:0/211486132 >> v2:172.21.15.146:3300/0 conn(0x7f7fc815d940 0x7f7fc806c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:50.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc7fff700 1 --2- 172.21.15.97:0/211486132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fc80d10b0 0x7f7fc806c290 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80cf0c0 tx=0x7f7fb8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a337a1a025cc4cdc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:50.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.521+0000 7f7fc6ffd700 1 -- 172.21.15.97:0/211486132 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8005e30 con 0x7f7fc80d10b0 2023-07-06T03:48:50.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.522+0000 7f7fc6ffd700 1 -- 172.21.15.97:0/211486132 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb8004030 con 0x7f7fc80d10b0 2023-07-06T03:48:50.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.522+0000 7f7fc6ffd700 1 -- 172.21.15.97:0/211486132 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80043e0 con 0x7f7fc80d10b0 2023-07-06T03:48:50.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.522+0000 7f7fce5ff700 1 -- 172.21.15.97:0/211486132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fc80d10b0 msgr2=0x7f7fc806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:50.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.522+0000 7f7fce5ff700 1 --2- 172.21.15.97:0/211486132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fc80d10b0 0x7f7fc806c290 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80cf0c0 tx=0x7f7fb8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.522+0000 7f7fce5ff700 1 -- 172.21.15.97:0/211486132 shutdown_connections 2023-07-06T03:48:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.522+0000 7f7fce5ff700 1 --2- 172.21.15.97:0/211486132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fc80d10b0 0x7f7fc806c290 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.522+0000 7f7fce5ff700 1 --2- 172.21.15.97:0/211486132 >> v2:172.21.15.146:3300/0 conn(0x7f7fc815d940 0x7f7fc806c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.522+0000 7f7fce5ff700 1 -- 172.21.15.97:0/211486132 >> 172.21.15.97:0/211486132 conn(0x7f7fc806a060 msgr2=0x7f7fc806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.522+0000 7f7fce5ff700 1 -- 172.21.15.97:0/211486132 shutdown_connections 2023-07-06T03:48:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.522+0000 7f7fce5ff700 1 -- 172.21.15.97:0/211486132 wait complete. 2023-07-06T03:48:50.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.524+0000 7f7fce5ff700 1 Processor -- start 2023-07-06T03:48:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.524+0000 7f7fce5ff700 1 -- start start 2023-07-06T03:48:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.525+0000 7f7fce5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fc80d10b0 0x7f7fc80b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.525+0000 7f7fce5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fc815d940 0x7f7fc80c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.525+0000 7f7fce5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fc8160200 0x7f7fc80b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.525+0000 7f7fce5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc8151370 con 0x7f7fc815d940 2023-07-06T03:48:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.525+0000 7f7fce5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc81f5860 con 0x7f7fc80d10b0 2023-07-06T03:48:50.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.525+0000 7f7fce5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc81f5b40 con 0x7f7fc8160200 2023-07-06T03:48:50.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.525+0000 7f7fc77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fc815d940 0x7f7fc80c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.525+0000 7f7fc77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fc815d940 0x7f7fc80c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49544/0 (socket says 172.21.15.97:49544) 2023-07-06T03:48:50.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.525+0000 7f7fc77fe700 1 -- 172.21.15.97:0/2906808324 learned_addr learned my addr 172.21.15.97:0/2906808324 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:50.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.525+0000 7f7fc7fff700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fc80d10b0 0x7f7fc80b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.525+0000 7f7fccb9c700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fc8160200 0x7f7fc80b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fc77fe700 1 -- 172.21.15.97:0/2906808324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fc8160200 msgr2=0x7f7fc80b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:50.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fc77fe700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fc8160200 0x7f7fc80b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fc77fe700 1 -- 172.21.15.97:0/2906808324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fc80d10b0 msgr2=0x7f7fc80b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:50.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fc77fe700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fc80d10b0 0x7f7fc80b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fc77fe700 1 -- 172.21.15.97:0/2906808324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb8009500 con 0x7f7fc815d940 2023-07-06T03:48:50.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fc7fff700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fc80d10b0 0x7f7fc80b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:50.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fc77fe700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fc815d940 0x7f7fc80c7ae0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00c750 tx=0x7f7fbc00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:50.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fc4ff9700 1 -- 172.21.15.97:0/2906808324 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc011800 con 0x7f7fc815d940 2023-07-06T03:48:50.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fc4ff9700 1 -- 172.21.15.97:0/2906808324 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc011960 con 0x7f7fc815d940 2023-07-06T03:48:50.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fc4ff9700 1 -- 172.21.15.97:0/2906808324 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc011cd0 con 0x7f7fc815d940 2023-07-06T03:48:50.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fce5ff700 1 -- 172.21.15.97:0/2906808324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc81f5e20 con 0x7f7fc815d940 2023-07-06T03:48:50.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.526+0000 7f7fce5ff700 1 -- 172.21.15.97:0/2906808324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc81f63e0 con 0x7f7fc815d940 2023-07-06T03:48:50.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.527+0000 7f7fce5ff700 1 -- 172.21.15.97:0/2906808324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc80b38d0 con 0x7f7fc815d940 2023-07-06T03:48:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.528+0000 7f7fc4ff9700 1 -- 172.21.15.97:0/2906808324 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7fbc021350 con 0x7f7fc815d940 2023-07-06T03:48:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.529+0000 7f7fc4ff9700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7fb406ace0 0x7f7fb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.529+0000 7f7fc4ff9700 1 -- 172.21.15.97:0/2906808324 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7fbc089500 con 0x7f7fc815d940 2023-07-06T03:48:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.529+0000 7f7fc7fff700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7fb406ace0 0x7f7fb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:50.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.530+0000 7f7fc7fff700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7fb406ace0 0x7f7fb406d190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8000c00 tx=0x7f7fb8008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:50.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.533+0000 7f7fc4ff9700 1 -- 172.21.15.97:0/2906808324 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7fc80b38d0 con 0x7f7fc815d940 2023-07-06T03:48:50.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.726+0000 7f7fce5ff700 1 -- 172.21.15.97:0/2906808324 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7fc815d140 con 0x7f7fb406ace0 2023-07-06T03:48:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.735+0000 7f7fc4ff9700 1 -- 172.21.15.97:0/2906808324 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311119 (secure 0 0 0) 0x7f7fb8004300 con 0x7f7fb406ace0 2023-07-06T03:48:50.741 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:50.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 -- 172.21.15.97:0/2906808324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7fb406ace0 msgr2=0x7f7fb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:50.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7fb406ace0 0x7f7fb406d190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8000c00 tx=0x7f7fb8008070 comp rx=0 tx=0).stop 2023-07-06T03:48:50.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 -- 172.21.15.97:0/2906808324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fc815d940 msgr2=0x7f7fc80c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fc815d940 0x7f7fc80c7ae0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00c750 tx=0x7f7fbc00cb10 comp rx=0 tx=0).stop 2023-07-06T03:48:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 -- 172.21.15.97:0/2906808324 shutdown_connections 2023-07-06T03:48:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7fb406ace0 0x7f7fb406d190 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7fc8160200 0x7f7fc80b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7fc80d10b0 0x7f7fc80b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 --2- 172.21.15.97:0/2906808324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fc815d940 0x7f7fc80c7ae0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 -- 172.21.15.97:0/2906808324 >> 172.21.15.97:0/2906808324 conn(0x7f7fc806a060 msgr2=0x7f7fc815f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 -- 172.21.15.97:0/2906808324 shutdown_connections 2023-07-06T03:48:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:50.740+0000 7f7fce5ff700 1 -- 172.21.15.97:0/2906808324 wait complete. 2023-07-06T03:48:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:48:50.761 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2023-07-06T03:48:48.926100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263184,"kb_used_data":6608,"kb_used_omap":0,"kb_used_meta":256512,"kb_avail":754711536,"statfs":{"total":773094113280,"available":772824612864,"internally_reserved":0,"allocated":6766592,"data_stored":3618541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":262668288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.034241"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938459+0000","last_change":"2023-07-06T03:48:45.936421+0000","last_active":"2023-07-06T03:48:45.938459+0000","last_peered":"2023-07-06T03:48:45.938459+0000","last_clean":"2023-07-06T03:48:45.938459+0000","last_became_active":"2023-07-06T03:48:45.936158+0000","last_became_peered":"2023-07-06T03:48:45.936158+0000","last_unstale":"2023-07-06T03:48:45.938459+0000","last_undegraded":"2023-07-06T03:48:45.938459+0000","last_fullsized":"2023-07-06T03:48:45.938459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:41:12.642685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.934244+0000","last_change":"2023-07-06T03:48:45.927570+0000","last_active":"2023-07-06T03:48:45.934244+0000","last_peered":"2023-07-06T03:48:45.934244+0000","last_clean":"2023-07-06T03:48:45.934244+0000","last_became_active":"2023-07-06T03:48:45.927050+0000","last_became_peered":"2023-07-06T03:48:45.927050+0000","last_unstale":"2023-07-06T03:48:45.934244+0000","last_undegraded":"2023-07-06T03:48:45.934244+0000","last_fullsized":"2023-07-06T03:48:45.934244+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:45:54.389534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.941953+0000","last_change":"2023-07-06T03:48:45.937436+0000","last_active":"2023-07-06T03:48:45.941953+0000","last_peered":"2023-07-06T03:48:45.941953+0000","last_clean":"2023-07-06T03:48:45.941953+0000","last_became_active":"2023-07-06T03:48:45.937098+0000","last_became_peered":"2023-07-06T03:48:45.937098+0000","last_unstale":"2023-07-06T03:48:45.941953+0000","last_undegraded":"2023-07-06T03:48:45.941953+0000","last_fullsized":"2023-07-06T03:48:45.941953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:19:23.201389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945625+0000","last_change":"2023-07-06T03:48:45.940318+0000","last_active":"2023-07-06T03:48:45.945625+0000","last_peered":"2023-07-06T03:48:45.945625+0000","last_clean":"2023-07-06T03:48:45.945625+0000","last_became_active":"2023-07-06T03:48:45.939910+0000","last_became_peered":"2023-07-06T03:48:45.939910+0000","last_unstale":"2023-07-06T03:48:45.945625+0000","last_undegraded":"2023-07-06T03:48:45.945625+0000","last_fullsized":"2023-07-06T03:48:45.945625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:21:27.456563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.927467+0000","last_change":"2023-07-06T03:48:45.922937+0000","last_active":"2023-07-06T03:48:45.927467+0000","last_peered":"2023-07-06T03:48:45.927467+0000","last_clean":"2023-07-06T03:48:45.927467+0000","last_became_active":"2023-07-06T03:48:45.922680+0000","last_became_peered":"2023-07-06T03:48:45.922680+0000","last_unstale":"2023-07-06T03:48:45.927467+0000","last_undegraded":"2023-07-06T03:48:45.927467+0000","last_fullsized":"2023-07-06T03:48:45.927467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:56:11.613853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937946+0000","last_change":"2023-07-06T03:48:45.936324+0000","last_active":"2023-07-06T03:48:45.937946+0000","last_peered":"2023-07-06T03:48:45.937946+0000","last_clean":"2023-07-06T03:48:45.937946+0000","last_became_active":"2023-07-06T03:48:45.935704+0000","last_became_peered":"2023-07-06T03:48:45.935704+0000","last_unstale":"2023-07-06T03:48:45.937946+0000","last_undegraded":"2023-07-06T03:48:45.937946+0000","last_fullsized":"2023-07-06T03:48:45.937946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:08:52.349909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937737+0000","last_change":"2023-07-06T03:48:45.929921+0000","last_active":"2023-07-06T03:48:45.937737+0000","last_peered":"2023-07-06T03:48:45.937737+0000","last_clean":"2023-07-06T03:48:45.937737+0000","last_became_active":"2023-07-06T03:48:45.929599+0000","last_became_peered":"2023-07-06T03:48:45.929599+0000","last_unstale":"2023-07-06T03:48:45.937737+0000","last_undegraded":"2023-07-06T03:48:45.937737+0000","last_fullsized":"2023-07-06T03:48:45.937737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:35:28.861995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.933369+0000","last_change":"2023-07-06T03:48:45.924753+0000","last_active":"2023-07-06T03:48:45.933369+0000","last_peered":"2023-07-06T03:48:45.933369+0000","last_clean":"2023-07-06T03:48:45.933369+0000","last_became_active":"2023-07-06T03:48:45.924441+0000","last_became_peered":"2023-07-06T03:48:45.924441+0000","last_unstale":"2023-07-06T03:48:45.933369+0000","last_undegraded":"2023-07-06T03:48:45.933369+0000","last_fullsized":"2023-07-06T03:48:45.933369+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:54:08.183436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.926363+0000","last_change":"2023-07-06T03:48:45.921212+0000","last_active":"2023-07-06T03:48:45.926363+0000","last_peered":"2023-07-06T03:48:45.926363+0000","last_clean":"2023-07-06T03:48:45.926363+0000","last_became_active":"2023-07-06T03:48:45.920785+0000","last_became_peered":"2023-07-06T03:48:45.920785+0000","last_unstale":"2023-07-06T03:48:45.926363+0000","last_undegraded":"2023-07-06T03:48:45.926363+0000","last_fullsized":"2023-07-06T03:48:45.926363+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:13:14.506309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.930082+0000","last_change":"2023-07-06T03:48:45.927488+0000","last_active":"2023-07-06T03:48:45.930082+0000","last_peered":"2023-07-06T03:48:45.930082+0000","last_clean":"2023-07-06T03:48:45.930082+0000","last_became_active":"2023-07-06T03:48:45.927208+0000","last_became_peered":"2023-07-06T03:48:45.927208+0000","last_unstale":"2023-07-06T03:48:45.930082+0000","last_undegraded":"2023-07-06T03:48:45.930082+0000","last_fullsized":"2023-07-06T03:48:45.930082+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:15:37.713996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.940995+0000","last_change":"2023-07-06T03:48:45.936161+0000","last_active":"2023-07-06T03:48:45.940995+0000","last_peered":"2023-07-06T03:48:45.940995+0000","last_clean":"2023-07-06T03:48:45.940995+0000","last_became_active":"2023-07-06T03:48:45.934879+0000","last_became_peered":"2023-07-06T03:48:45.934879+0000","last_unstale":"2023-07-06T03:48:45.940995+0000","last_undegraded":"2023-07-06T03:48:45.940995+0000","last_fullsized":"2023-07-06T03:48:45.940995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:17:06.277791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945947+0000","last_change":"2023-07-06T03:48:45.942831+0000","last_active":"2023-07-06T03:48:45.945947+0000","last_peered":"2023-07-06T03:48:45.945947+0000","last_clean":"2023-07-06T03:48:45.945947+0000","last_became_active":"2023-07-06T03:48:45.942539+0000","last_became_peered":"2023-07-06T03:48:45.942539+0000","last_unstale":"2023-07-06T03:48:45.945947+0000","last_undegraded":"2023-07-06T03:48:45.945947+0000","last_fullsized":"2023-07-06T03:48:45.945947+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:14.401412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.947453+0000","last_change":"2023-07-06T03:48:45.945006+0000","last_active":"2023-07-06T03:48:45.947453+0000","last_peered":"2023-07-06T03:48:45.947453+0000","last_clean":"2023-07-06T03:48:45.947453+0000","last_became_active":"2023-07-06T03:48:45.944558+0000","last_became_peered":"2023-07-06T03:48:45.944558+0000","last_unstale":"2023-07-06T03:48:45.947453+0000","last_undegraded":"2023-07-06T03:48:45.947453+0000","last_fullsized":"2023-07-06T03:48:45.947453+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:53:29.567555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.946966+0000","last_change":"2023-07-06T03:48:45.940073+0000","last_active":"2023-07-06T03:48:45.946966+0000","last_peered":"2023-07-06T03:48:45.946966+0000","last_clean":"2023-07-06T03:48:45.946966+0000","last_became_active":"2023-07-06T03:48:45.939838+0000","last_became_peered":"2023-07-06T03:48:45.939838+0000","last_unstale":"2023-07-06T03:48:45.946966+0000","last_undegraded":"2023-07-06T03:48:45.946966+0000","last_fullsized":"2023-07-06T03:48:45.946966+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:17:12.651762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.935951+0000","last_change":"2023-07-06T03:48:45.933692+0000","last_active":"2023-07-06T03:48:45.935951+0000","last_peered":"2023-07-06T03:48:45.935951+0000","last_clean":"2023-07-06T03:48:45.935951+0000","last_became_active":"2023-07-06T03:48:45.933384+0000","last_became_peered":"2023-07-06T03:48:45.933384+0000","last_unstale":"2023-07-06T03:48:45.935951+0000","last_undegraded":"2023-07-06T03:48:45.935951+0000","last_fullsized":"2023-07-06T03:48:45.935951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:09:26.159497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.936725+0000","last_change":"2023-07-06T03:48:45.935476+0000","last_active":"2023-07-06T03:48:45.936725+0000","last_peered":"2023-07-06T03:48:45.936725+0000","last_clean":"2023-07-06T03:48:45.936725+0000","last_became_active":"2023-07-06T03:48:45.935106+0000","last_became_peered":"2023-07-06T03:48:45.935106+0000","last_unstale":"2023-07-06T03:48:45.936725+0000","last_undegraded":"2023-07-06T03:48:45.936725+0000","last_fullsized":"2023-07-06T03:48:45.936725+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:46:22.384643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938446+0000","last_change":"2023-07-06T03:48:45.934922+0000","last_active":"2023-07-06T03:48:45.938446+0000","last_peered":"2023-07-06T03:48:45.938446+0000","last_clean":"2023-07-06T03:48:45.938446+0000","last_became_active":"2023-07-06T03:48:45.934639+0000","last_became_peered":"2023-07-06T03:48:45.934639+0000","last_unstale":"2023-07-06T03:48:45.938446+0000","last_undegraded":"2023-07-06T03:48:45.938446+0000","last_fullsized":"2023-07-06T03:48:45.938446+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:09:27.067517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.936067+0000","last_change":"2023-07-06T03:48:45.931455+0000","last_active":"2023-07-06T03:48:45.936067+0000","last_peered":"2023-07-06T03:48:45.936067+0000","last_clean":"2023-07-06T03:48:45.936067+0000","last_became_active":"2023-07-06T03:48:45.930983+0000","last_became_peered":"2023-07-06T03:48:45.930983+0000","last_unstale":"2023-07-06T03:48:45.936067+0000","last_undegraded":"2023-07-06T03:48:45.936067+0000","last_fullsized":"2023-07-06T03:48:45.936067+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:54:27.935910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938025+0000","last_change":"2023-07-06T03:48:45.935164+0000","last_active":"2023-07-06T03:48:45.938025+0000","last_peered":"2023-07-06T03:48:45.938025+0000","last_clean":"2023-07-06T03:48:45.938025+0000","last_became_active":"2023-07-06T03:48:45.934539+0000","last_became_peered":"2023-07-06T03:48:45.934539+0000","last_unstale":"2023-07-06T03:48:45.938025+0000","last_undegraded":"2023-07-06T03:48:45.938025+0000","last_fullsized":"2023-07-06T03:48:45.938025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:20:09.326471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.931690+0000","last_change":"2023-07-06T03:48:45.924527+0000","last_active":"2023-07-06T03:48:45.931690+0000","last_peered":"2023-07-06T03:48:45.931690+0000","last_clean":"2023-07-06T03:48:45.931690+0000","last_became_active":"2023-07-06T03:48:45.924232+0000","last_became_peered":"2023-07-06T03:48:45.924232+0000","last_unstale":"2023-07-06T03:48:45.931690+0000","last_undegraded":"2023-07-06T03:48:45.931690+0000","last_fullsized":"2023-07-06T03:48:45.931690+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:33:35.631454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.944613+0000","last_change":"2023-07-06T03:48:45.937588+0000","last_active":"2023-07-06T03:48:45.944613+0000","last_peered":"2023-07-06T03:48:45.944613+0000","last_clean":"2023-07-06T03:48:45.944613+0000","last_became_active":"2023-07-06T03:48:45.937248+0000","last_became_peered":"2023-07-06T03:48:45.937248+0000","last_unstale":"2023-07-06T03:48:45.944613+0000","last_undegraded":"2023-07-06T03:48:45.944613+0000","last_fullsized":"2023-07-06T03:48:45.944613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:40:19.956777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937963+0000","last_change":"2023-07-06T03:48:45.936412+0000","last_active":"2023-07-06T03:48:45.937963+0000","last_peered":"2023-07-06T03:48:45.937963+0000","last_clean":"2023-07-06T03:48:45.937963+0000","last_became_active":"2023-07-06T03:48:45.936056+0000","last_became_peered":"2023-07-06T03:48:45.936056+0000","last_unstale":"2023-07-06T03:48:45.937963+0000","last_undegraded":"2023-07-06T03:48:45.937963+0000","last_fullsized":"2023-07-06T03:48:45.937963+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:07:02.951830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.924178+0000","last_change":"2023-07-06T03:48:45.920952+0000","last_active":"2023-07-06T03:48:45.924178+0000","last_peered":"2023-07-06T03:48:45.924178+0000","last_clean":"2023-07-06T03:48:45.924178+0000","last_became_active":"2023-07-06T03:48:45.920629+0000","last_became_peered":"2023-07-06T03:48:45.920629+0000","last_unstale":"2023-07-06T03:48:45.924178+0000","last_undegraded":"2023-07-06T03:48:45.924178+0000","last_fullsized":"2023-07-06T03:48:45.924178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:15:37.713996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.924626+0000","last_change":"2023-07-06T03:48:45.915892+0000","last_active":"2023-07-06T03:48:45.924626+0000","last_peered":"2023-07-06T03:48:45.924626+0000","last_clean":"2023-07-06T03:48:45.924626+0000","last_became_active":"2023-07-06T03:48:45.915629+0000","last_became_peered":"2023-07-06T03:48:45.915629+0000","last_unstale":"2023-07-06T03:48:45.924626+0000","last_undegraded":"2023-07-06T03:48:45.924626+0000","last_fullsized":"2023-07-06T03:48:45.924626+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:04:14.349930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.940408+0000","last_change":"2023-07-06T03:48:45.932289+0000","last_active":"2023-07-06T03:48:45.940408+0000","last_peered":"2023-07-06T03:48:45.940408+0000","last_clean":"2023-07-06T03:48:45.940408+0000","last_became_active":"2023-07-06T03:48:45.932024+0000","last_became_peered":"2023-07-06T03:48:45.932024+0000","last_unstale":"2023-07-06T03:48:45.940408+0000","last_undegraded":"2023-07-06T03:48:45.940408+0000","last_fullsized":"2023-07-06T03:48:45.940408+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:15:56.397408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.935761+0000","last_change":"2023-07-06T03:48:45.926955+0000","last_active":"2023-07-06T03:48:45.935761+0000","last_peered":"2023-07-06T03:48:45.935761+0000","last_clean":"2023-07-06T03:48:45.935761+0000","last_became_active":"2023-07-06T03:48:45.926640+0000","last_became_peered":"2023-07-06T03:48:45.926640+0000","last_unstale":"2023-07-06T03:48:45.935761+0000","last_undegraded":"2023-07-06T03:48:45.935761+0000","last_fullsized":"2023-07-06T03:48:45.935761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:08.579590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.939111+0000","last_change":"2023-07-06T03:48:45.933396+0000","last_active":"2023-07-06T03:48:45.939111+0000","last_peered":"2023-07-06T03:48:45.939111+0000","last_clean":"2023-07-06T03:48:45.939111+0000","last_became_active":"2023-07-06T03:48:45.933120+0000","last_became_peered":"2023-07-06T03:48:45.933120+0000","last_unstale":"2023-07-06T03:48:45.939111+0000","last_undegraded":"2023-07-06T03:48:45.939111+0000","last_fullsized":"2023-07-06T03:48:45.939111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:06.457526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938485+0000","last_change":"2023-07-06T03:48:45.933258+0000","last_active":"2023-07-06T03:48:45.938485+0000","last_peered":"2023-07-06T03:48:45.938485+0000","last_clean":"2023-07-06T03:48:45.938485+0000","last_became_active":"2023-07-06T03:48:45.932796+0000","last_became_peered":"2023-07-06T03:48:45.932796+0000","last_unstale":"2023-07-06T03:48:45.938485+0000","last_undegraded":"2023-07-06T03:48:45.938485+0000","last_fullsized":"2023-07-06T03:48:45.938485+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:42:29.955680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.946085+0000","last_change":"2023-07-06T03:48:45.942965+0000","last_active":"2023-07-06T03:48:45.946085+0000","last_peered":"2023-07-06T03:48:45.946085+0000","last_clean":"2023-07-06T03:48:45.946085+0000","last_became_active":"2023-07-06T03:48:45.942694+0000","last_became_peered":"2023-07-06T03:48:45.942694+0000","last_unstale":"2023-07-06T03:48:45.946085+0000","last_undegraded":"2023-07-06T03:48:45.946085+0000","last_fullsized":"2023-07-06T03:48:45.946085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:17:30.665732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.929914+0000","last_change":"2023-07-06T03:48:45.922389+0000","last_active":"2023-07-06T03:48:45.929914+0000","last_peered":"2023-07-06T03:48:45.929914+0000","last_clean":"2023-07-06T03:48:45.929914+0000","last_became_active":"2023-07-06T03:48:45.921930+0000","last_became_peered":"2023-07-06T03:48:45.921930+0000","last_unstale":"2023-07-06T03:48:45.929914+0000","last_undegraded":"2023-07-06T03:48:45.929914+0000","last_fullsized":"2023-07-06T03:48:45.929914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:38:48.869271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.916843+0000","last_change":"2023-07-06T03:48:45.914819+0000","last_active":"2023-07-06T03:48:45.916843+0000","last_peered":"2023-07-06T03:48:45.916843+0000","last_clean":"2023-07-06T03:48:45.916843+0000","last_became_active":"2023-07-06T03:48:45.914495+0000","last_became_peered":"2023-07-06T03:48:45.914495+0000","last_unstale":"2023-07-06T03:48:45.916843+0000","last_undegraded":"2023-07-06T03:48:45.916843+0000","last_fullsized":"2023-07-06T03:48:45.916843+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:34:55.838514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.933719+0000","last_change":"2023-07-06T03:48:45.928736+0000","last_active":"2023-07-06T03:48:45.933719+0000","last_peered":"2023-07-06T03:48:45.933719+0000","last_clean":"2023-07-06T03:48:45.933719+0000","last_became_active":"2023-07-06T03:48:45.928423+0000","last_became_peered":"2023-07-06T03:48:45.928423+0000","last_unstale":"2023-07-06T03:48:45.933719+0000","last_undegraded":"2023-07-06T03:48:45.933719+0000","last_fullsized":"2023-07-06T03:48:45.933719+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:03:15.250420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945590+0000","last_change":"2023-07-06T03:48:45.940164+0000","last_active":"2023-07-06T03:48:45.945590+0000","last_peered":"2023-07-06T03:48:45.945590+0000","last_clean":"2023-07-06T03:48:45.945590+0000","last_became_active":"2023-07-06T03:48:45.938126+0000","last_became_peered":"2023-07-06T03:48:45.938126+0000","last_unstale":"2023-07-06T03:48:45.945590+0000","last_undegraded":"2023-07-06T03:48:45.945590+0000","last_fullsized":"2023-07-06T03:48:45.945590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:32:39.098957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.930075+0000","last_change":"2023-07-06T03:48:44.913633+0000","last_active":"2023-07-06T03:48:45.930075+0000","last_peered":"2023-07-06T03:48:45.930075+0000","last_clean":"2023-07-06T03:48:45.930075+0000","last_became_active":"2023-07-06T03:48:44.913361+0000","last_became_peered":"2023-07-06T03:48:44.913361+0000","last_unstale":"2023-07-06T03:48:45.930075+0000","last_undegraded":"2023-07-06T03:48:45.930075+0000","last_fullsized":"2023-07-06T03:48:45.930075+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:11:41.522108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.932292+0000","last_change":"2023-07-06T03:48:45.923424+0000","last_active":"2023-07-06T03:48:45.932292+0000","last_peered":"2023-07-06T03:48:45.932292+0000","last_clean":"2023-07-06T03:48:45.932292+0000","last_became_active":"2023-07-06T03:48:45.923146+0000","last_became_peered":"2023-07-06T03:48:45.923146+0000","last_unstale":"2023-07-06T03:48:45.932292+0000","last_undegraded":"2023-07-06T03:48:45.932292+0000","last_fullsized":"2023-07-06T03:48:45.932292+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:53:29.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.936798+0000","last_change":"2023-07-06T03:48:44.921283+0000","last_active":"2023-07-06T03:48:45.936798+0000","last_peered":"2023-07-06T03:48:45.936798+0000","last_clean":"2023-07-06T03:48:45.936798+0000","last_became_active":"2023-07-06T03:48:44.920943+0000","last_became_peered":"2023-07-06T03:48:44.920943+0000","last_unstale":"2023-07-06T03:48:45.936798+0000","last_undegraded":"2023-07-06T03:48:45.936798+0000","last_fullsized":"2023-07-06T03:48:45.936798+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:54.735005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.934223+0000","last_change":"2023-07-06T03:48:45.927374+0000","last_active":"2023-07-06T03:48:45.934223+0000","last_peered":"2023-07-06T03:48:45.934223+0000","last_clean":"2023-07-06T03:48:45.934223+0000","last_became_active":"2023-07-06T03:48:45.926739+0000","last_became_peered":"2023-07-06T03:48:45.926739+0000","last_unstale":"2023-07-06T03:48:45.934223+0000","last_undegraded":"2023-07-06T03:48:45.934223+0000","last_fullsized":"2023-07-06T03:48:45.934223+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:53:12.064750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937563+0000","last_change":"2023-07-06T03:48:44.917394+0000","last_active":"2023-07-06T03:48:45.937563+0000","last_peered":"2023-07-06T03:48:45.937563+0000","last_clean":"2023-07-06T03:48:45.937563+0000","last_became_active":"2023-07-06T03:48:44.916937+0000","last_became_peered":"2023-07-06T03:48:44.916937+0000","last_unstale":"2023-07-06T03:48:45.937563+0000","last_undegraded":"2023-07-06T03:48:45.937563+0000","last_fullsized":"2023-07-06T03:48:45.937563+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:26:39.450856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.936280+0000","last_change":"2023-07-06T03:48:45.925290+0000","last_active":"2023-07-06T03:48:45.936280+0000","last_peered":"2023-07-06T03:48:45.936280+0000","last_clean":"2023-07-06T03:48:45.936280+0000","last_became_active":"2023-07-06T03:48:45.925034+0000","last_became_peered":"2023-07-06T03:48:45.925034+0000","last_unstale":"2023-07-06T03:48:45.936280+0000","last_undegraded":"2023-07-06T03:48:45.936280+0000","last_fullsized":"2023-07-06T03:48:45.936280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:27:27.148134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938931+0000","last_change":"2023-07-06T03:48:44.923930+0000","last_active":"2023-07-06T03:48:45.938931+0000","last_peered":"2023-07-06T03:48:45.938931+0000","last_clean":"2023-07-06T03:48:45.938931+0000","last_became_active":"2023-07-06T03:48:44.923591+0000","last_became_peered":"2023-07-06T03:48:44.923591+0000","last_unstale":"2023-07-06T03:48:45.938931+0000","last_undegraded":"2023-07-06T03:48:45.938931+0000","last_fullsized":"2023-07-06T03:48:45.938931+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:43:32.679358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.927704+0000","last_change":"2023-07-06T03:48:45.925264+0000","last_active":"2023-07-06T03:48:45.927704+0000","last_peered":"2023-07-06T03:48:45.927704+0000","last_clean":"2023-07-06T03:48:45.927704+0000","last_became_active":"2023-07-06T03:48:45.924719+0000","last_became_peered":"2023-07-06T03:48:45.924719+0000","last_unstale":"2023-07-06T03:48:45.927704+0000","last_undegraded":"2023-07-06T03:48:45.927704+0000","last_fullsized":"2023-07-06T03:48:45.927704+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:23:57.868352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.929347+0000","last_change":"2023-07-06T03:48:44.914772+0000","last_active":"2023-07-06T03:48:45.929347+0000","last_peered":"2023-07-06T03:48:45.929347+0000","last_clean":"2023-07-06T03:48:45.929347+0000","last_became_active":"2023-07-06T03:48:44.913233+0000","last_became_peered":"2023-07-06T03:48:44.913233+0000","last_unstale":"2023-07-06T03:48:45.929347+0000","last_undegraded":"2023-07-06T03:48:45.929347+0000","last_fullsized":"2023-07-06T03:48:45.929347+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:18:39.820151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938332+0000","last_change":"2023-07-06T03:48:45.932622+0000","last_active":"2023-07-06T03:48:45.938332+0000","last_peered":"2023-07-06T03:48:45.938332+0000","last_clean":"2023-07-06T03:48:45.938332+0000","last_became_active":"2023-07-06T03:48:45.932282+0000","last_became_peered":"2023-07-06T03:48:45.932282+0000","last_unstale":"2023-07-06T03:48:45.938332+0000","last_undegraded":"2023-07-06T03:48:45.938332+0000","last_fullsized":"2023-07-06T03:48:45.938332+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:15:19.314836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.939422+0000","last_change":"2023-07-06T03:48:44.925189+0000","last_active":"2023-07-06T03:48:45.939422+0000","last_peered":"2023-07-06T03:48:45.939422+0000","last_clean":"2023-07-06T03:48:45.939422+0000","last_became_active":"2023-07-06T03:48:44.924922+0000","last_became_peered":"2023-07-06T03:48:44.924922+0000","last_unstale":"2023-07-06T03:48:45.939422+0000","last_undegraded":"2023-07-06T03:48:45.939422+0000","last_fullsized":"2023-07-06T03:48:45.939422+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:46:17.342988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.928656+0000","last_change":"2023-07-06T03:48:45.920827+0000","last_active":"2023-07-06T03:48:45.928656+0000","last_peered":"2023-07-06T03:48:45.928656+0000","last_clean":"2023-07-06T03:48:45.928656+0000","last_became_active":"2023-07-06T03:48:45.920546+0000","last_became_peered":"2023-07-06T03:48:45.920546+0000","last_unstale":"2023-07-06T03:48:45.928656+0000","last_undegraded":"2023-07-06T03:48:45.928656+0000","last_fullsized":"2023-07-06T03:48:45.928656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:30.784151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.918957+0000","last_change":"2023-07-06T03:48:44.923265+0000","last_active":"2023-07-06T03:48:45.918957+0000","last_peered":"2023-07-06T03:48:45.918957+0000","last_clean":"2023-07-06T03:48:45.918957+0000","last_became_active":"2023-07-06T03:48:44.923008+0000","last_became_peered":"2023-07-06T03:48:44.923008+0000","last_unstale":"2023-07-06T03:48:45.918957+0000","last_undegraded":"2023-07-06T03:48:45.918957+0000","last_fullsized":"2023-07-06T03:48:45.918957+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:55:49.403296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.927877+0000","last_change":"2023-07-06T03:48:45.924929+0000","last_active":"2023-07-06T03:48:45.927877+0000","last_peered":"2023-07-06T03:48:45.927877+0000","last_clean":"2023-07-06T03:48:45.927877+0000","last_became_active":"2023-07-06T03:48:45.924530+0000","last_became_peered":"2023-07-06T03:48:45.924530+0000","last_unstale":"2023-07-06T03:48:45.927877+0000","last_undegraded":"2023-07-06T03:48:45.927877+0000","last_fullsized":"2023-07-06T03:48:45.927877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:47:09.763176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.916718+0000","last_change":"2023-07-06T03:48:44.926871+0000","last_active":"2023-07-06T03:48:45.916718+0000","last_peered":"2023-07-06T03:48:45.916718+0000","last_clean":"2023-07-06T03:48:45.916718+0000","last_became_active":"2023-07-06T03:48:44.926605+0000","last_became_peered":"2023-07-06T03:48:44.926605+0000","last_unstale":"2023-07-06T03:48:45.916718+0000","last_undegraded":"2023-07-06T03:48:45.916718+0000","last_fullsized":"2023-07-06T03:48:45.916718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:29:27.653969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937761+0000","last_change":"2023-07-06T03:48:45.930632+0000","last_active":"2023-07-06T03:48:45.937761+0000","last_peered":"2023-07-06T03:48:45.937761+0000","last_clean":"2023-07-06T03:48:45.937761+0000","last_became_active":"2023-07-06T03:48:45.930285+0000","last_became_peered":"2023-07-06T03:48:45.930285+0000","last_unstale":"2023-07-06T03:48:45.937761+0000","last_undegraded":"2023-07-06T03:48:45.937761+0000","last_fullsized":"2023-07-06T03:48:45.937761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:54:45.278281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.916227+0000","last_change":"2023-07-06T03:48:44.928020+0000","last_active":"2023-07-06T03:48:45.916227+0000","last_peered":"2023-07-06T03:48:45.916227+0000","last_clean":"2023-07-06T03:48:45.916227+0000","last_became_active":"2023-07-06T03:48:44.927772+0000","last_became_peered":"2023-07-06T03:48:44.927772+0000","last_unstale":"2023-07-06T03:48:45.916227+0000","last_undegraded":"2023-07-06T03:48:45.916227+0000","last_fullsized":"2023-07-06T03:48:45.916227+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:00.554009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937204+0000","last_change":"2023-07-06T03:48:45.929325+0000","last_active":"2023-07-06T03:48:45.937204+0000","last_peered":"2023-07-06T03:48:45.937204+0000","last_clean":"2023-07-06T03:48:45.937204+0000","last_became_active":"2023-07-06T03:48:45.929031+0000","last_became_peered":"2023-07-06T03:48:45.929031+0000","last_unstale":"2023-07-06T03:48:45.937204+0000","last_undegraded":"2023-07-06T03:48:45.937204+0000","last_fullsized":"2023-07-06T03:48:45.937204+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:46:11.766517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.925011+0000","last_change":"2023-07-06T03:48:44.927012+0000","last_active":"2023-07-06T03:48:45.925011+0000","last_peered":"2023-07-06T03:48:45.925011+0000","last_clean":"2023-07-06T03:48:45.925011+0000","last_became_active":"2023-07-06T03:48:44.926719+0000","last_became_peered":"2023-07-06T03:48:44.926719+0000","last_unstale":"2023-07-06T03:48:45.925011+0000","last_undegraded":"2023-07-06T03:48:45.925011+0000","last_fullsized":"2023-07-06T03:48:45.925011+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:31:10.982994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.941899+0000","last_change":"2023-07-06T03:48:45.936935+0000","last_active":"2023-07-06T03:48:45.941899+0000","last_peered":"2023-07-06T03:48:45.941899+0000","last_clean":"2023-07-06T03:48:45.941899+0000","last_became_active":"2023-07-06T03:48:45.936327+0000","last_became_peered":"2023-07-06T03:48:45.936327+0000","last_unstale":"2023-07-06T03:48:45.941899+0000","last_undegraded":"2023-07-06T03:48:45.941899+0000","last_fullsized":"2023-07-06T03:48:45.941899+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:39:32.023179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.939316+0000","last_change":"2023-07-06T03:48:44.924386+0000","last_active":"2023-07-06T03:48:45.939316+0000","last_peered":"2023-07-06T03:48:45.939316+0000","last_clean":"2023-07-06T03:48:45.939316+0000","last_became_active":"2023-07-06T03:48:44.924089+0000","last_became_peered":"2023-07-06T03:48:44.924089+0000","last_unstale":"2023-07-06T03:48:45.939316+0000","last_undegraded":"2023-07-06T03:48:45.939316+0000","last_fullsized":"2023-07-06T03:48:45.939316+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:14:37.992686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.923624+0000","last_change":"2023-07-06T03:48:45.914921+0000","last_active":"2023-07-06T03:48:45.923624+0000","last_peered":"2023-07-06T03:48:45.923624+0000","last_clean":"2023-07-06T03:48:45.923624+0000","last_became_active":"2023-07-06T03:48:45.914606+0000","last_became_peered":"2023-07-06T03:48:45.914606+0000","last_unstale":"2023-07-06T03:48:45.923624+0000","last_undegraded":"2023-07-06T03:48:45.923624+0000","last_fullsized":"2023-07-06T03:48:45.923624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:25.409027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"59'6","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945452+0000","last_change":"2023-07-06T03:48:44.923668+0000","last_active":"2023-07-06T03:48:45.945452+0000","last_peered":"2023-07-06T03:48:45.945452+0000","last_clean":"2023-07-06T03:48:45.945452+0000","last_became_active":"2023-07-06T03:48:44.923268+0000","last_became_peered":"2023-07-06T03:48:44.923268+0000","last_unstale":"2023-07-06T03:48:45.945452+0000","last_undegraded":"2023-07-06T03:48:45.945452+0000","last_fullsized":"2023-07-06T03:48:45.945452+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:49:50.252456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.928690+0000","last_change":"2023-07-06T03:48:45.920070+0000","last_active":"2023-07-06T03:48:45.928690+0000","last_peered":"2023-07-06T03:48:45.928690+0000","last_clean":"2023-07-06T03:48:45.928690+0000","last_became_active":"2023-07-06T03:48:45.919318+0000","last_became_peered":"2023-07-06T03:48:45.919318+0000","last_unstale":"2023-07-06T03:48:45.928690+0000","last_undegraded":"2023-07-06T03:48:45.928690+0000","last_fullsized":"2023-07-06T03:48:45.928690+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:46:22.384643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.946288+0000","last_change":"2023-07-06T03:48:44.927623+0000","last_active":"2023-07-06T03:48:45.946288+0000","last_peered":"2023-07-06T03:48:45.946288+0000","last_clean":"2023-07-06T03:48:45.946288+0000","last_became_active":"2023-07-06T03:48:44.927354+0000","last_became_peered":"2023-07-06T03:48:44.927354+0000","last_unstale":"2023-07-06T03:48:45.946288+0000","last_undegraded":"2023-07-06T03:48:45.946288+0000","last_fullsized":"2023-07-06T03:48:45.946288+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:21:39.443238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.928473+0000","last_change":"2023-07-06T03:48:45.926163+0000","last_active":"2023-07-06T03:48:45.928473+0000","last_peered":"2023-07-06T03:48:45.928473+0000","last_clean":"2023-07-06T03:48:45.928473+0000","last_became_active":"2023-07-06T03:48:45.925903+0000","last_became_peered":"2023-07-06T03:48:45.925903+0000","last_unstale":"2023-07-06T03:48:45.928473+0000","last_undegraded":"2023-07-06T03:48:45.928473+0000","last_fullsized":"2023-07-06T03:48:45.928473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:03:21.355033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.911639+0000","last_change":"2023-07-06T03:48:44.915434+0000","last_active":"2023-07-06T03:48:45.911639+0000","last_peered":"2023-07-06T03:48:45.911639+0000","last_clean":"2023-07-06T03:48:45.911639+0000","last_became_active":"2023-07-06T03:48:44.915151+0000","last_became_peered":"2023-07-06T03:48:44.915151+0000","last_unstale":"2023-07-06T03:48:45.911639+0000","last_undegraded":"2023-07-06T03:48:45.911639+0000","last_fullsized":"2023-07-06T03:48:45.911639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:22:43.026243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.934946+0000","last_change":"2023-07-06T03:48:45.929492+0000","last_active":"2023-07-06T03:48:45.934946+0000","last_peered":"2023-07-06T03:48:45.934946+0000","last_clean":"2023-07-06T03:48:45.934946+0000","last_became_active":"2023-07-06T03:48:45.929237+0000","last_became_peered":"2023-07-06T03:48:45.929237+0000","last_unstale":"2023-07-06T03:48:45.934946+0000","last_undegraded":"2023-07-06T03:48:45.934946+0000","last_fullsized":"2023-07-06T03:48:45.934946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:48.452424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.909611+0000","last_change":"2023-07-06T03:48:44.920425+0000","last_active":"2023-07-06T03:48:45.909611+0000","last_peered":"2023-07-06T03:48:45.909611+0000","last_clean":"2023-07-06T03:48:45.909611+0000","last_became_active":"2023-07-06T03:48:44.920158+0000","last_became_peered":"2023-07-06T03:48:44.920158+0000","last_unstale":"2023-07-06T03:48:45.909611+0000","last_undegraded":"2023-07-06T03:48:45.909611+0000","last_fullsized":"2023-07-06T03:48:45.909611+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:54:31.066629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.938042+0000","last_change":"2023-07-06T03:48:45.929982+0000","last_active":"2023-07-06T03:48:45.938042+0000","last_peered":"2023-07-06T03:48:45.938042+0000","last_clean":"2023-07-06T03:48:45.938042+0000","last_became_active":"2023-07-06T03:48:45.929688+0000","last_became_peered":"2023-07-06T03:48:45.929688+0000","last_unstale":"2023-07-06T03:48:45.938042+0000","last_undegraded":"2023-07-06T03:48:45.938042+0000","last_fullsized":"2023-07-06T03:48:45.938042+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:20.615059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.943073+0000","last_change":"2023-07-06T03:48:44.925926+0000","last_active":"2023-07-06T03:48:45.943073+0000","last_peered":"2023-07-06T03:48:45.943073+0000","last_clean":"2023-07-06T03:48:45.943073+0000","last_became_active":"2023-07-06T03:48:44.925613+0000","last_became_peered":"2023-07-06T03:48:44.925613+0000","last_unstale":"2023-07-06T03:48:45.943073+0000","last_undegraded":"2023-07-06T03:48:45.943073+0000","last_fullsized":"2023-07-06T03:48:45.943073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:42:50.484259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937213+0000","last_change":"2023-07-06T03:48:45.931421+0000","last_active":"2023-07-06T03:48:45.937213+0000","last_peered":"2023-07-06T03:48:45.937213+0000","last_clean":"2023-07-06T03:48:45.937213+0000","last_became_active":"2023-07-06T03:48:45.931136+0000","last_became_peered":"2023-07-06T03:48:45.931136+0000","last_unstale":"2023-07-06T03:48:45.937213+0000","last_undegraded":"2023-07-06T03:48:45.937213+0000","last_fullsized":"2023-07-06T03:48:45.937213+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:06.097637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945684+0000","last_change":"2023-07-06T03:48:44.926876+0000","last_active":"2023-07-06T03:48:45.945684+0000","last_peered":"2023-07-06T03:48:45.945684+0000","last_clean":"2023-07-06T03:48:45.945684+0000","last_became_active":"2023-07-06T03:48:44.926612+0000","last_became_peered":"2023-07-06T03:48:44.926612+0000","last_unstale":"2023-07-06T03:48:45.945684+0000","last_undegraded":"2023-07-06T03:48:45.945684+0000","last_fullsized":"2023-07-06T03:48:45.945684+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:59.283071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937226+0000","last_change":"2023-07-06T03:48:45.931154+0000","last_active":"2023-07-06T03:48:45.937226+0000","last_peered":"2023-07-06T03:48:45.937226+0000","last_clean":"2023-07-06T03:48:45.937226+0000","last_became_active":"2023-07-06T03:48:45.930865+0000","last_became_peered":"2023-07-06T03:48:45.930865+0000","last_unstale":"2023-07-06T03:48:45.937226+0000","last_undegraded":"2023-07-06T03:48:45.937226+0000","last_fullsized":"2023-07-06T03:48:45.937226+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:05.078262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.915713+0000","last_change":"2023-07-06T03:48:44.920013+0000","last_active":"2023-07-06T03:48:45.915713+0000","last_peered":"2023-07-06T03:48:45.915713+0000","last_clean":"2023-07-06T03:48:45.915713+0000","last_became_active":"2023-07-06T03:48:44.919761+0000","last_became_peered":"2023-07-06T03:48:44.919761+0000","last_unstale":"2023-07-06T03:48:45.915713+0000","last_undegraded":"2023-07-06T03:48:45.915713+0000","last_fullsized":"2023-07-06T03:48:45.915713+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:41:47.704928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.924449+0000","last_change":"2023-07-06T03:48:45.920246+0000","last_active":"2023-07-06T03:48:45.924449+0000","last_peered":"2023-07-06T03:48:45.924449+0000","last_clean":"2023-07-06T03:48:45.924449+0000","last_became_active":"2023-07-06T03:48:45.919918+0000","last_became_peered":"2023-07-06T03:48:45.919918+0000","last_unstale":"2023-07-06T03:48:45.924449+0000","last_undegraded":"2023-07-06T03:48:45.924449+0000","last_fullsized":"2023-07-06T03:48:45.924449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:44:18.117762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.919398+0000","last_change":"2023-07-06T03:48:44.926131+0000","last_active":"2023-07-06T03:48:45.919398+0000","last_peered":"2023-07-06T03:48:45.919398+0000","last_clean":"2023-07-06T03:48:45.919398+0000","last_became_active":"2023-07-06T03:48:44.925790+0000","last_became_peered":"2023-07-06T03:48:44.925790+0000","last_unstale":"2023-07-06T03:48:45.919398+0000","last_undegraded":"2023-07-06T03:48:45.919398+0000","last_fullsized":"2023-07-06T03:48:45.919398+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:09:06.481520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.935833+0000","last_change":"2023-07-06T03:48:45.926258+0000","last_active":"2023-07-06T03:48:45.935833+0000","last_peered":"2023-07-06T03:48:45.935833+0000","last_clean":"2023-07-06T03:48:45.935833+0000","last_became_active":"2023-07-06T03:48:45.925754+0000","last_became_peered":"2023-07-06T03:48:45.925754+0000","last_unstale":"2023-07-06T03:48:45.935833+0000","last_undegraded":"2023-07-06T03:48:45.935833+0000","last_fullsized":"2023-07-06T03:48:45.935833+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:07:36.302648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.908693+0000","last_change":"2023-07-06T03:48:44.923656+0000","last_active":"2023-07-06T03:48:45.908693+0000","last_peered":"2023-07-06T03:48:45.908693+0000","last_clean":"2023-07-06T03:48:45.908693+0000","last_became_active":"2023-07-06T03:48:44.923372+0000","last_became_peered":"2023-07-06T03:48:44.923372+0000","last_unstale":"2023-07-06T03:48:45.908693+0000","last_undegraded":"2023-07-06T03:48:45.908693+0000","last_fullsized":"2023-07-06T03:48:45.908693+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:29.775291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.935461+0000","last_change":"2023-07-06T03:48:45.930046+0000","last_active":"2023-07-06T03:48:45.935461+0000","last_peered":"2023-07-06T03:48:45.935461+0000","last_clean":"2023-07-06T03:48:45.935461+0000","last_became_active":"2023-07-06T03:48:45.929799+0000","last_became_peered":"2023-07-06T03:48:45.929799+0000","last_unstale":"2023-07-06T03:48:45.935461+0000","last_undegraded":"2023-07-06T03:48:45.935461+0000","last_fullsized":"2023-07-06T03:48:45.935461+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:26:00.176042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.924647+0000","last_change":"2023-07-06T03:48:44.924367+0000","last_active":"2023-07-06T03:48:45.924647+0000","last_peered":"2023-07-06T03:48:45.924647+0000","last_clean":"2023-07-06T03:48:45.924647+0000","last_became_active":"2023-07-06T03:48:44.924124+0000","last_became_peered":"2023-07-06T03:48:44.924124+0000","last_unstale":"2023-07-06T03:48:45.924647+0000","last_undegraded":"2023-07-06T03:48:45.924647+0000","last_fullsized":"2023-07-06T03:48:45.924647+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:29.668635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.947104+0000","last_change":"2023-07-06T03:48:45.944422+0000","last_active":"2023-07-06T03:48:45.947104+0000","last_peered":"2023-07-06T03:48:45.947104+0000","last_clean":"2023-07-06T03:48:45.947104+0000","last_became_active":"2023-07-06T03:48:45.944153+0000","last_became_peered":"2023-07-06T03:48:45.944153+0000","last_unstale":"2023-07-06T03:48:45.947104+0000","last_undegraded":"2023-07-06T03:48:45.947104+0000","last_fullsized":"2023-07-06T03:48:45.947104+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:49:34.398220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.918231+0000","last_change":"2023-07-06T03:48:44.913655+0000","last_active":"2023-07-06T03:48:45.918231+0000","last_peered":"2023-07-06T03:48:45.918231+0000","last_clean":"2023-07-06T03:48:45.918231+0000","last_became_active":"2023-07-06T03:48:44.913413+0000","last_became_peered":"2023-07-06T03:48:44.913413+0000","last_unstale":"2023-07-06T03:48:45.918231+0000","last_undegraded":"2023-07-06T03:48:45.918231+0000","last_fullsized":"2023-07-06T03:48:45.918231+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:57:15.487713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.946252+0000","last_change":"2023-07-06T03:48:45.943008+0000","last_active":"2023-07-06T03:48:45.946252+0000","last_peered":"2023-07-06T03:48:45.946252+0000","last_clean":"2023-07-06T03:48:45.946252+0000","last_became_active":"2023-07-06T03:48:45.942719+0000","last_became_peered":"2023-07-06T03:48:45.942719+0000","last_unstale":"2023-07-06T03:48:45.946252+0000","last_undegraded":"2023-07-06T03:48:45.946252+0000","last_fullsized":"2023-07-06T03:48:45.946252+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:50.819405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.908476+0000","last_change":"2023-07-06T03:48:44.916132+0000","last_active":"2023-07-06T03:48:45.908476+0000","last_peered":"2023-07-06T03:48:45.908476+0000","last_clean":"2023-07-06T03:48:45.908476+0000","last_became_active":"2023-07-06T03:48:44.915846+0000","last_became_peered":"2023-07-06T03:48:44.915846+0000","last_unstale":"2023-07-06T03:48:45.908476+0000","last_undegraded":"2023-07-06T03:48:45.908476+0000","last_fullsized":"2023-07-06T03:48:45.908476+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:09:25.150637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.926572+0000","last_change":"2023-07-06T03:48:45.922015+0000","last_active":"2023-07-06T03:48:45.926572+0000","last_peered":"2023-07-06T03:48:45.926572+0000","last_clean":"2023-07-06T03:48:45.926572+0000","last_became_active":"2023-07-06T03:48:45.921635+0000","last_became_peered":"2023-07-06T03:48:45.921635+0000","last_unstale":"2023-07-06T03:48:45.926572+0000","last_undegraded":"2023-07-06T03:48:45.926572+0000","last_fullsized":"2023-07-06T03:48:45.926572+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:39:12.633544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.917807+0000","last_change":"2023-07-06T03:48:44.926204+0000","last_active":"2023-07-06T03:48:45.917807+0000","last_peered":"2023-07-06T03:48:45.917807+0000","last_clean":"2023-07-06T03:48:45.917807+0000","last_became_active":"2023-07-06T03:48:44.925899+0000","last_became_peered":"2023-07-06T03:48:44.925899+0000","last_unstale":"2023-07-06T03:48:45.917807+0000","last_undegraded":"2023-07-06T03:48:45.917807+0000","last_fullsized":"2023-07-06T03:48:45.917807+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:17:17.815231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.945926+0000","last_change":"2023-07-06T03:48:45.942472+0000","last_active":"2023-07-06T03:48:45.945926+0000","last_peered":"2023-07-06T03:48:45.945926+0000","last_clean":"2023-07-06T03:48:45.945926+0000","last_became_active":"2023-07-06T03:48:45.942184+0000","last_became_peered":"2023-07-06T03:48:45.942184+0000","last_unstale":"2023-07-06T03:48:45.945926+0000","last_undegraded":"2023-07-06T03:48:45.945926+0000","last_fullsized":"2023-07-06T03:48:45.945926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:42:16.887206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.941690+0000","last_change":"2023-07-06T03:48:44.926031+0000","last_active":"2023-07-06T03:48:45.941690+0000","last_peered":"2023-07-06T03:48:45.941690+0000","last_clean":"2023-07-06T03:48:45.941690+0000","last_became_active":"2023-07-06T03:48:44.925752+0000","last_became_peered":"2023-07-06T03:48:44.925752+0000","last_unstale":"2023-07-06T03:48:45.941690+0000","last_undegraded":"2023-07-06T03:48:45.941690+0000","last_fullsized":"2023-07-06T03:48:45.941690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:23:25.221363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.925902+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:48:45.925902+0000","last_peered":"2023-07-06T03:48:45.925902+0000","last_clean":"2023-07-06T03:48:45.925902+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:48:45.925902+0000","last_undegraded":"2023-07-06T03:48:45.925902+0000","last_fullsized":"2023-07-06T03:48:45.925902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:11:23.347400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.941111+0000","last_change":"2023-07-06T03:48:45.935497+0000","last_active":"2023-07-06T03:48:45.941111+0000","last_peered":"2023-07-06T03:48:45.941111+0000","last_clean":"2023-07-06T03:48:45.941111+0000","last_became_active":"2023-07-06T03:48:45.935181+0000","last_became_peered":"2023-07-06T03:48:45.935181+0000","last_unstale":"2023-07-06T03:48:45.941111+0000","last_undegraded":"2023-07-06T03:48:45.941111+0000","last_fullsized":"2023-07-06T03:48:45.941111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:26:37.641808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.917660+0000","last_change":"2023-07-06T03:48:44.926111+0000","last_active":"2023-07-06T03:48:45.917660+0000","last_peered":"2023-07-06T03:48:45.917660+0000","last_clean":"2023-07-06T03:48:45.917660+0000","last_became_active":"2023-07-06T03:48:44.925856+0000","last_became_peered":"2023-07-06T03:48:44.925856+0000","last_unstale":"2023-07-06T03:48:45.917660+0000","last_undegraded":"2023-07-06T03:48:45.917660+0000","last_fullsized":"2023-07-06T03:48:45.917660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:31:53.312131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.908581+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:48:45.908581+0000","last_peered":"2023-07-06T03:48:45.908581+0000","last_clean":"2023-07-06T03:48:45.908581+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:48:45.908581+0000","last_undegraded":"2023-07-06T03:48:45.908581+0000","last_fullsized":"2023-07-06T03:48:45.908581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:51:33.596463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.939068+0000","last_change":"2023-07-06T03:48:45.934398+0000","last_active":"2023-07-06T03:48:45.939068+0000","last_peered":"2023-07-06T03:48:45.939068+0000","last_clean":"2023-07-06T03:48:45.939068+0000","last_became_active":"2023-07-06T03:48:45.934139+0000","last_became_peered":"2023-07-06T03:48:45.934139+0000","last_unstale":"2023-07-06T03:48:45.939068+0000","last_undegraded":"2023-07-06T03:48:45.939068+0000","last_fullsized":"2023-07-06T03:48:45.939068+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:39:35.277083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.911718+0000","last_change":"2023-07-06T03:48:44.922169+0000","last_active":"2023-07-06T03:48:45.911718+0000","last_peered":"2023-07-06T03:48:45.911718+0000","last_clean":"2023-07-06T03:48:45.911718+0000","last_became_active":"2023-07-06T03:48:44.920785+0000","last_became_peered":"2023-07-06T03:48:44.920785+0000","last_unstale":"2023-07-06T03:48:45.911718+0000","last_undegraded":"2023-07-06T03:48:45.911718+0000","last_fullsized":"2023-07-06T03:48:45.911718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:08:27.303274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":90,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.908605+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:48:45.908605+0000","last_peered":"2023-07-06T03:48:45.908605+0000","last_clean":"2023-07-06T03:48:45.908605+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:48:45.908605+0000","last_undegraded":"2023-07-06T03:48:45.908605+0000","last_fullsized":"2023-07-06T03:48:45.908605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:19:03.256816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.930655+0000","last_change":"2023-07-06T03:48:45.926396+0000","last_active":"2023-07-06T03:48:45.930655+0000","last_peered":"2023-07-06T03:48:45.930655+0000","last_clean":"2023-07-06T03:48:45.930655+0000","last_became_active":"2023-07-06T03:48:45.926108+0000","last_became_peered":"2023-07-06T03:48:45.926108+0000","last_unstale":"2023-07-06T03:48:45.930655+0000","last_undegraded":"2023-07-06T03:48:45.930655+0000","last_fullsized":"2023-07-06T03:48:45.930655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:27:44.219454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.918099+0000","last_change":"2023-07-06T03:48:44.924030+0000","last_active":"2023-07-06T03:48:45.918099+0000","last_peered":"2023-07-06T03:48:45.918099+0000","last_clean":"2023-07-06T03:48:45.918099+0000","last_became_active":"2023-07-06T03:48:44.923723+0000","last_became_peered":"2023-07-06T03:48:44.923723+0000","last_unstale":"2023-07-06T03:48:45.918099+0000","last_undegraded":"2023-07-06T03:48:45.918099+0000","last_fullsized":"2023-07-06T03:48:45.918099+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:24:57.134005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":84,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.909174+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:48:45.909174+0000","last_peered":"2023-07-06T03:48:45.909174+0000","last_clean":"2023-07-06T03:48:45.909174+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:48:45.909174+0000","last_undegraded":"2023-07-06T03:48:45.909174+0000","last_fullsized":"2023-07-06T03:48:45.909174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:57:05.254201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":228,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.912188+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:48:45.912188+0000","last_peered":"2023-07-06T03:48:45.912188+0000","last_clean":"2023-07-06T03:48:45.912188+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:48:45.912188+0000","last_undegraded":"2023-07-06T03:48:45.912188+0000","last_fullsized":"2023-07-06T03:48:45.912188+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:25:03.097211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.937918+0000","last_change":"2023-07-06T03:48:45.936192+0000","last_active":"2023-07-06T03:48:45.937918+0000","last_peered":"2023-07-06T03:48:45.937918+0000","last_clean":"2023-07-06T03:48:45.937918+0000","last_became_active":"2023-07-06T03:48:45.935893+0000","last_became_peered":"2023-07-06T03:48:45.935893+0000","last_unstale":"2023-07-06T03:48:45.937918+0000","last_undegraded":"2023-07-06T03:48:45.937918+0000","last_fullsized":"2023-07-06T03:48:45.937918+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:09:38.787882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.943942+0000","last_change":"2023-07-06T03:48:44.928006+0000","last_active":"2023-07-06T03:48:45.943942+0000","last_peered":"2023-07-06T03:48:45.943942+0000","last_clean":"2023-07-06T03:48:45.943942+0000","last_became_active":"2023-07-06T03:48:44.927709+0000","last_became_peered":"2023-07-06T03:48:44.927709+0000","last_unstale":"2023-07-06T03:48:45.943942+0000","last_undegraded":"2023-07-06T03:48:45.943942+0000","last_fullsized":"2023-07-06T03:48:45.943942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:42:32.328554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":84,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.909574+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:48:45.909574+0000","last_peered":"2023-07-06T03:48:45.909574+0000","last_clean":"2023-07-06T03:48:45.909574+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:48:45.909574+0000","last_undegraded":"2023-07-06T03:48:45.909574+0000","last_fullsized":"2023-07-06T03:48:45.909574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:35.550657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.944921+0000","last_change":"2023-07-06T03:48:45.939012+0000","last_active":"2023-07-06T03:48:45.944921+0000","last_peered":"2023-07-06T03:48:45.944921+0000","last_clean":"2023-07-06T03:48:45.944921+0000","last_became_active":"2023-07-06T03:48:45.938715+0000","last_became_peered":"2023-07-06T03:48:45.938715+0000","last_unstale":"2023-07-06T03:48:45.944921+0000","last_undegraded":"2023-07-06T03:48:45.944921+0000","last_fullsized":"2023-07-06T03:48:45.944921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:52.452649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.941118+0000","last_change":"2023-07-06T03:48:44.926366+0000","last_active":"2023-07-06T03:48:45.941118+0000","last_peered":"2023-07-06T03:48:45.941118+0000","last_clean":"2023-07-06T03:48:45.941118+0000","last_became_active":"2023-07-06T03:48:44.926115+0000","last_became_peered":"2023-07-06T03:48:44.926115+0000","last_unstale":"2023-07-06T03:48:45.941118+0000","last_undegraded":"2023-07-06T03:48:45.941118+0000","last_fullsized":"2023-07-06T03:48:45.941118+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:21:38.432585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":84,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.910049+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:48:45.910049+0000","last_peered":"2023-07-06T03:48:45.910049+0000","last_clean":"2023-07-06T03:48:45.910049+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:48:45.910049+0000","last_undegraded":"2023-07-06T03:48:45.910049+0000","last_fullsized":"2023-07-06T03:48:45.910049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:46:14.068727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.940703+0000","last_change":"2023-07-06T03:48:45.936727+0000","last_active":"2023-07-06T03:48:45.940703+0000","last_peered":"2023-07-06T03:48:45.940703+0000","last_clean":"2023-07-06T03:48:45.940703+0000","last_became_active":"2023-07-06T03:48:45.935957+0000","last_became_peered":"2023-07-06T03:48:45.935957+0000","last_unstale":"2023-07-06T03:48:45.940703+0000","last_undegraded":"2023-07-06T03:48:45.940703+0000","last_fullsized":"2023-07-06T03:48:45.940703+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:30:05.349113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.942193+0000","last_change":"2023-07-06T03:48:44.923503+0000","last_active":"2023-07-06T03:48:45.942193+0000","last_peered":"2023-07-06T03:48:45.942193+0000","last_clean":"2023-07-06T03:48:45.942193+0000","last_became_active":"2023-07-06T03:48:44.923200+0000","last_became_peered":"2023-07-06T03:48:44.923200+0000","last_unstale":"2023-07-06T03:48:45.942193+0000","last_undegraded":"2023-07-06T03:48:45.942193+0000","last_fullsized":"2023-07-06T03:48:45.942193+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:01:13.093100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.912739+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:48:45.912739+0000","last_peered":"2023-07-06T03:48:45.912739+0000","last_clean":"2023-07-06T03:48:45.912739+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:48:45.912739+0000","last_undegraded":"2023-07-06T03:48:45.912739+0000","last_fullsized":"2023-07-06T03:48:45.912739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:49:25.633769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.935853+0000","last_change":"2023-07-06T03:48:45.926381+0000","last_active":"2023-07-06T03:48:45.935853+0000","last_peered":"2023-07-06T03:48:45.935853+0000","last_clean":"2023-07-06T03:48:45.935853+0000","last_became_active":"2023-07-06T03:48:45.926115+0000","last_became_peered":"2023-07-06T03:48:45.926115+0000","last_unstale":"2023-07-06T03:48:45.935853+0000","last_undegraded":"2023-07-06T03:48:45.935853+0000","last_fullsized":"2023-07-06T03:48:45.935853+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:44.891893+0000","last_clean_scrub_stamp":"2023-07-06T03:48:44.891893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:49.290541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.908121+0000","last_change":"2023-07-06T03:48:44.926109+0000","last_active":"2023-07-06T03:48:45.908121+0000","last_peered":"2023-07-06T03:48:45.908121+0000","last_clean":"2023-07-06T03:48:45.908121+0000","last_became_active":"2023-07-06T03:48:44.925836+0000","last_became_peered":"2023-07-06T03:48:44.925836+0000","last_unstale":"2023-07-06T03:48:45.908121+0000","last_undegraded":"2023-07-06T03:48:45.908121+0000","last_fullsized":"2023-07-06T03:48:45.908121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:48:43.883033+0000","last_clean_scrub_stamp":"2023-07-06T03:48:43.883033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:47.443564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":84,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-07-06T03:48:45.909947+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:48:45.909947+0000","last_peered":"2023-07-06T03:48:45.909947+0000","last_clean":"2023-07-06T03:48:45.909947+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:48:45.909947+0000","last_undegraded":"2023-07-06T03:48:45.909947+0000","last_fullsized":"2023-07-06T03:48:45.909947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:43:49.173497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":696320,"data_stored":303962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.69299999999999995,"15min":0.72199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.874,"5min":1.345,"15min":1.345},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.77100000000000002,"15min":0.82299999999999995},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.68899999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.72299999999999998,"15min":0.755},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85599999999999998,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.85399999999999998,"15min":0.93799999999999994},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90800000000000003,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78400000000000003,"15min":0.83399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88300000000000001,"5min":3.77,"15min":3.77},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72999999999999998,"15min":0.74099999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84399999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.501}]},{"osd":3,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.70099999999999996,"15min":0.70899999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83399999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.024},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94399999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.375,"5min":0.374,"15min":0.374},"max":{"1min":0.93000000000000005,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46700000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.96699999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.68500000000000005,"15min":0.67400000000000004},"min":{"1min":0.54200000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75,"15min":0.747},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.026,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79500000000000004,"15min":0.81100000000000005},"min":{"1min":0.44800000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98899999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.64700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":696320,"data_stored":303962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65100000000000002,"15min":0.65400000000000003},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.89100000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.86899999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66200000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65900000000000003,"15min":0.67100000000000004},"min":{"1min":0.44500000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89200000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65700000000000003,"15min":0.67900000000000005},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79100000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.624}]},{"osd":2,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63900000000000001,"15min":0.65500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.872,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69899999999999995,"15min":0.72099999999999997},"min":{"1min":0.46500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.93999999999999995,"5min":1.119,"15min":1.119},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73999999999999999,"15min":0.77800000000000002},"min":{"1min":0.48599999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.84999999999999998,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71299999999999997,"15min":0.746},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.81899999999999995,"5min":1.093,"15min":1.093},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96999999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.75800000000000001},"min":{"1min":0.46700000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88300000000000001,"5min":1.085,"15min":1.085},"last":0.748}]},{"osd":5,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70199999999999996,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84399999999999997,"5min":1.218,"15min":1.218},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.52600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89000000000000001,"5min":1.198,"15min":1.198},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:48:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81999999999999995,"15min":0.85899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94799999999999995,"5min":1.323,"15min":1.323},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.82199999999999995,"15min":0.86799999999999999},"min":{"1min":0.51100000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.028,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1290240,"data_stored":894330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.52100000000000002,"15min":0.58399999999999996},"min":{"1min":0.27800000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60099999999999998,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.55600000000000005,"15min":0.59799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67900000000000005,"5min":1.96,"15min":1.96},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.873,"15min":1.069},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.626,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.61899999999999999,"15min":0.67200000000000004},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77700000000000002,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.59799999999999998,"15min":0.64400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79300000000000004,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53800000000000003,"15min":0.56899999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82599999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.34599999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":1.139,"15min":1.329},"min":{"1min":0.51600000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89900000000000002,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.82099999999999995,"15min":0.85399999999999998},"min":{"1min":0.54100000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.371,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.76600000000000001,"15min":0.79400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97199999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":1.1439999999999999,"15min":1.3440000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89800000000000002,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.80600000000000005,"15min":0.83999999999999997},"min":{"1min":0.50600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93200000000000005,"5min":1.365,"15min":1.365},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.745,"5min":1.1859999999999999,"15min":1.387},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.254,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.81699999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:48:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.86299999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.4690000000000001,"5min":2.806,"15min":2.806},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.80600000000000005,"15min":0.83799999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.006,"5min":1.337,"15min":1.337},"last":0.72799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":708608,"data_stored":311090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.436,"15min":0.438},"min":{"1min":0.28199999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.57299999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.434}]},{"osd":2,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46800000000000003,"15min":0.47699999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79400000000000004,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.34100000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71199999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.378},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70599999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.73299999999999998,"15min":0.755},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96299999999999997,"5min":1.171,"15min":1.171},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71399999999999997,"15min":0.72199999999999998},"min":{"1min":0.60199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.875,"5min":1.034,"15min":1.034},"last":0.70799999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.879,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94499999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.96999999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85999999999999999,"5min":1.546,"15min":1.546},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.79600000000000004,"15min":0.81399999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.93200000000000005,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":1.042,"15min":1.1910000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.88100000000000001,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.80000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35824,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336016,"statfs":{"total":96636764160,"available":96600080384,"internally_reserved":0,"allocated":696320,"data_stored":303962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.32100000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.623,"5min":0.626,"15min":0.626},"last":0.627},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.47199999999999998},"min":{"1min":0.31,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.40000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.58699999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.505,"15min":0.499},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82699999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66600000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.76300000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0469999999999999,"5min":1.165,"15min":1.165},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.878}]},{"osd":5,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.753,"15min":0.754},"min":{"1min":0.60699999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0249999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.57799999999999996,"5min":0.5,"15min":0.5},"max":{"1min":1.002,"5min":1.046,"15min":1.046},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.75,"15min":0.75},"min":{"1min":0.58899999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.034,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.99199999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:48:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.81299999999999994,"15min":0.83599999999999997},"min":{"1min":0.59099999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0680000000000001,"5min":2.157,"15min":2.157},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.57299999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0369999999999999,"5min":1.966,"15min":1.966},"last":0.84399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":696320,"data_stored":303962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.58899999999999997,"15min":0.67500000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51300000000000001,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.502,"15min":0.51900000000000002},"min":{"1min":0.37,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.54500000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.999,"15min":1.2609999999999999},"min":{"1min":0.32200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.54700000000000004,"5min":15.911,"15min":15.911},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":1.0680000000000001,"15min":1.3300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.65200000000000002,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.48799999999999999,"15min":0.51600000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.627,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.38},{"interface":"front","average":{"1min":0.48799999999999999,"5min":1.075,"15min":1.3360000000000001},"min":{"1min":0.36899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70299999999999996,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.77300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94499999999999995,"5min":1.179,"15min":1.179},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.90600000000000003,"15min":1.004},"min":{"1min":0.504,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.875,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.86799999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.90800000000000003,"15min":1.0029999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92600000000000005,"5min":5.077,"15min":5.077},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.96099999999999997,"15min":1.0669999999999999},"min":{"1min":0.53500000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90100000000000002,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.748}]},{"osd":6,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.78600000000000003,"15min":0.82399999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.91000000000000003,"5min":1.143,"15min":1.143},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.92200000000000004,"15min":1.0209999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84799999999999998,"5min":5.157,"15min":5.157},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jul 6 03:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.81499999999999995,"15min":0.86599999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96399999999999997,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.93500000000000005,"15min":1.0309999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91100000000000003,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.83599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35824,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336016,"statfs":{"total":96636764160,"available":96600080384,"internally_reserved":0,"allocated":696320,"data_stored":303962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93400000000000005,"5min":1.087,"15min":1.087},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63300000000000001,"15min":0.64200000000000002},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.83899999999999997,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.84199999999999997,"15min":0.93500000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.95399999999999996,"5min":6.258,"15min":6.258},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65400000000000003,"15min":0.65500000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91000000000000003,"5min":1.171,"15min":1.171},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.85899999999999999,"15min":0.94799999999999995},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.998,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.93700000000000006,"15min":1.0409999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.88900000000000001,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.76500000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97499999999999998,"5min":1.25,"15min":1.25},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89700000000000002,"5min":1.413,"15min":1.413},"last":0.81100000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65600000000000003,"15min":0.68100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81499999999999995,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.44},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.65200000000000002,"15min":0.68500000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77500000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.67900000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.95099999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93100000000000005,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.747,"15min":0.77100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89400000000000002,"5min":1.21,"15min":1.21},"last":0.57999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1286144,"data_stored":893311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.83599999999999997,"15min":0.93200000000000005},"min":{"1min":0.41799999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84299999999999997,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.75700000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.39800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.89800000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82999999999999996,"5min":1.546,"15min":1.546},"last":0.63400000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.68400000000000005,"15min":0.70999999999999996},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77300000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69099999999999995,"15min":0.69899999999999995},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.873,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81399999999999995,"5min":1.03,"15min":1.03},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73199999999999998,"15min":0.75800000000000001},"min":{"1min":0.496,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88200000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.625,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79200000000000004,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.505},{"interface":"front","average":{"1min":0.629,"5min":0.70499999999999996,"15min":0.71299999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95499999999999996,"5min":2.718,"15min":2.718},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93200000000000005,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.71999999999999997,"15min":0.73099999999999998},"min":{"1min":0.52100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85199999999999998,"5min":1.669,"15min":1.669},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:48:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.90500000000000003,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78300000000000003,"15min":0.79500000000000004},"min":{"1min":0.52500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97699999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.70699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:48:50.768 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T03:48:50.769 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T03:48:51.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.005+0000 7f0665bff700 1 Processor -- start 2023-07-06T03:48:51.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.005+0000 7f0665bff700 1 -- start start 2023-07-06T03:48:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.006+0000 7f0665bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f06600b8590 0x7f06600cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.006+0000 7f0665bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0660160a50 0x7f06600b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.006+0000 7f0665bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06600d0c10 con 0x7f06600b4cd0 2023-07-06T03:48:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.006+0000 7f0665bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06600b5150 con 0x7f06600b8590 2023-07-06T03:48:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.006+0000 7f0665bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0660160ec0 con 0x7f0660160a50 2023-07-06T03:48:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.006+0000 7f065f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f06600b8590 0x7f06600cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.006+0000 7f065f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f06600b8590 0x7f06600cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49552/0 (socket says 172.21.15.97:49552) 2023-07-06T03:48:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.006+0000 7f065f7fe700 1 -- 172.21.15.97:0/2555467401 learned_addr learned my addr 172.21.15.97:0/2555467401 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:51.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.007+0000 7f065f7fe700 1 -- 172.21.15.97:0/2555467401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0660160a50 msgr2=0x7f06600b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:51.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.007+0000 7f065effd700 1 --2- 172.21.15.97:0/2555467401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0660160a50 0x7f06600b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:51.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.007+0000 7f065f7fe700 1 --2- 172.21.15.97:0/2555467401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0660160a50 0x7f06600b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.007+0000 7f065f7fe700 1 -- 172.21.15.97:0/2555467401 >> v1:172.21.15.146:6789/0 conn(0x7f06600b4cd0 legacy=0x7f066006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.007+0000 7f065f7fe700 1 -- 172.21.15.97:0/2555467401 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06600d08e0 con 0x7f06600b8590 2023-07-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.007+0000 7f065effd700 1 --2- 172.21.15.97:0/2555467401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0660160a50 0x7f06600b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.007+0000 7f065f7fe700 1 --2- 172.21.15.97:0/2555467401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06600b8590 0x7f06600cef60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f06500098f0 tx=0x7f0650009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266a507701e8d21a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.007+0000 7f065e7fc700 1 -- 172.21.15.97:0/2555467401 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650004230 con 0x7f06600b8590 2023-07-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.007+0000 7f065e7fc700 1 -- 172.21.15.97:0/2555467401 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0650004390 con 0x7f06600b8590 2023-07-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.007+0000 7f065e7fc700 1 -- 172.21.15.97:0/2555467401 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06500036a0 con 0x7f06600b8590 2023-07-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.008+0000 7f0665bff700 1 -- 172.21.15.97:0/2555467401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06600b8590 msgr2=0x7f06600cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.008+0000 7f0665bff700 1 --2- 172.21.15.97:0/2555467401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06600b8590 0x7f06600cef60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f06500098f0 tx=0x7f0650009c00 comp rx=0 tx=0).stop 2023-07-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.008+0000 7f0665bff700 1 -- 172.21.15.97:0/2555467401 shutdown_connections 2023-07-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.008+0000 7f0665bff700 1 --2- 172.21.15.97:0/2555467401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06600b8590 0x7f06600cef60 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.008+0000 7f0665bff700 1 --2- 172.21.15.97:0/2555467401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0660160a50 0x7f06600b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.008+0000 7f0665bff700 1 -- 172.21.15.97:0/2555467401 >> 172.21.15.97:0/2555467401 conn(0x7f066006b290 msgr2=0x7f06600cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.008+0000 7f0665bff700 1 -- 172.21.15.97:0/2555467401 shutdown_connections 2023-07-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.008+0000 7f0665bff700 1 -- 172.21.15.97:0/2555467401 wait complete. 2023-07-06T03:48:51.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.010+0000 7f0665bff700 1 Processor -- start 2023-07-06T03:48:51.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.010+0000 7f0665bff700 1 -- start start 2023-07-06T03:48:51.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f0665bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06600b4cd0 0x7f066015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:51.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f0665bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06600b8590 0x7f066015e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:51.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f0665bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0660160a50 0x7f06600aa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f0665bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066015fb20 con 0x7f06600b8590 2023-07-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f0665bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066015a790 con 0x7f06600b4cd0 2023-07-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f0665bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066015eea0 con 0x7f0660160a50 2023-07-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f065effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06600b8590 0x7f066015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f065effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06600b8590 0x7f066015e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49562/0 (socket says 172.21.15.97:49562) 2023-07-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f065effd700 1 -- 172.21.15.97:0/2819720479 learned_addr learned my addr 172.21.15.97:0/2819720479 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f065effd700 1 -- 172.21.15.97:0/2819720479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0660160a50 msgr2=0x7f06600aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f065effd700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0660160a50 0x7f06600aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f065effd700 1 -- 172.21.15.97:0/2819720479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06600b4cd0 msgr2=0x7f066015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.011+0000 7f065f7fe700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06600b4cd0 0x7f066015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.012+0000 7f065effd700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06600b4cd0 0x7f066015a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.012+0000 7f065effd700 1 -- 172.21.15.97:0/2819720479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0650009550 con 0x7f06600b8590 2023-07-06T03:48:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.012+0000 7f065f7fe700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06600b4cd0 0x7f066015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.012+0000 7f065effd700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06600b8590 0x7f066015e400 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f065400c7f0 tx=0x7f065400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.012+0000 7f0647fff700 1 -- 172.21.15.97:0/2819720479 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654011800 con 0x7f06600b8590 2023-07-06T03:48:51.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.012+0000 7f0647fff700 1 -- 172.21.15.97:0/2819720479 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0654011960 con 0x7f06600b8590 2023-07-06T03:48:51.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.012+0000 7f0647fff700 1 -- 172.21.15.97:0/2819720479 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654011d10 con 0x7f06600b8590 2023-07-06T03:48:51.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.012+0000 7f0665bff700 1 -- 172.21.15.97:0/2819720479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066015f180 con 0x7f06600b8590 2023-07-06T03:48:51.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.012+0000 7f0665bff700 1 -- 172.21.15.97:0/2819720479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066015f740 con 0x7f06600b8590 2023-07-06T03:48:51.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.013+0000 7f0665bff700 1 -- 172.21.15.97:0/2819720479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066004e340 con 0x7f06600b8590 2023-07-06T03:48:51.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.014+0000 7f0647fff700 1 -- 172.21.15.97:0/2819720479 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0654018360 con 0x7f06600b8590 2023-07-06T03:48:51.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.015+0000 7f0647fff700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f064c06ace0 0x7f064c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:51.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.015+0000 7f0647fff700 1 -- 172.21.15.97:0/2819720479 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f06540893d0 con 0x7f06600b8590 2023-07-06T03:48:51.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.015+0000 7f065f7fe700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f064c06ace0 0x7f064c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:51.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.016+0000 7f065f7fe700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f064c06ace0 0x7f064c06d190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0650009c40 tx=0x7f0650008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.019+0000 7f0647fff700 1 -- 172.21.15.97:0/2819720479 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0654054510 con 0x7f06600b8590 2023-07-06T03:48:51.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.291+0000 7f0665bff700 1 -- 172.21.15.97:0/2819720479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f066015cb20 con 0x7f06600b8590 2023-07-06T03:48:51.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.296+0000 7f0647fff700 1 -- 172.21.15.97:0/2819720479 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f066015cb20 con 0x7f06600b8590 2023-07-06T03:48:51.299 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:48:51.299 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg== 2023-07-06T03:48:51.299 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-07-06T03:48:51.299 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-07-06T03:48:51.299 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T03:48:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.298+0000 7f0665bff700 1 -- 172.21.15.97:0/2819720479 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f064c06ace0 msgr2=0x7f064c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.298+0000 7f0665bff700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f064c06ace0 0x7f064c06d190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0650009c40 tx=0x7f0650008030 comp rx=0 tx=0).stop 2023-07-06T03:48:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.298+0000 7f0665bff700 1 -- 172.21.15.97:0/2819720479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06600b8590 msgr2=0x7f066015e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.298+0000 7f0665bff700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06600b8590 0x7f066015e400 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f065400c7f0 tx=0x7f065400cbb0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.298+0000 7f0665bff700 1 -- 172.21.15.97:0/2819720479 shutdown_connections 2023-07-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.298+0000 7f0665bff700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f064c06ace0 0x7f064c06d190 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.298+0000 7f0665bff700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0660160a50 0x7f06600aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.298+0000 7f0665bff700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06600b4cd0 0x7f066015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.298+0000 7f0665bff700 1 --2- 172.21.15.97:0/2819720479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06600b8590 0x7f066015e400 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.299+0000 7f0665bff700 1 -- 172.21.15.97:0/2819720479 >> 172.21.15.97:0/2819720479 conn(0x7f066006b290 msgr2=0x7f06600b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:51.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.299+0000 7f0665bff700 1 -- 172.21.15.97:0/2819720479 shutdown_connections 2023-07-06T03:48:51.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.299+0000 7f0665bff700 1 -- 172.21.15.97:0/2819720479 wait complete. 2023-07-06T03:48:51.303 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-07-06T03:48:51.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:48:51.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.509+0000 7f0daadff700 1 Processor -- start 2023-07-06T03:48:51.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.510+0000 7f0daadff700 1 -- start start 2023-07-06T03:48:51.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.511+0000 7f0daadff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da40cd390 0x7f0da4160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:51.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.511+0000 7f0daadff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da40cdae0 con 0x7f0da4160dd0 2023-07-06T03:48:51.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.511+0000 7f0daadff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da40b6640 con 0x7f0da415c520 2023-07-06T03:48:51.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.511+0000 7f0daadff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da40b6920 con 0x7f0da40cd390 2023-07-06T03:48:51.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.511+0000 7f0da939c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0da4160dd0 0x7f0da40b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59128/0 (socket says 172.21.15.97:59128) 2023-07-06T03:48:51.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.511+0000 7f0da939c700 1 -- 172.21.15.97:0/4155354637 learned_addr learned my addr 172.21.15.97:0/4155354637 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:51.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.511+0000 7f0da3fff700 1 --2- 172.21.15.97:0/4155354637 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da40cd390 0x7f0da4160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:51.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.512+0000 7f0da37fe700 1 -- 172.21.15.97:0/4155354637 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250209889 0 0) 0x7f0da40cdae0 con 0x7f0da4160dd0 2023-07-06T03:48:51.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.512+0000 7f0da37fe700 1 -- 172.21.15.97:0/4155354637 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d940035b0 con 0x7f0da4160dd0 2023-07-06T03:48:51.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.512+0000 7f0da37fe700 1 -- 172.21.15.97:0/4155354637 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804056974 0 0) 0x7f0da40b6640 con 0x7f0da415c520 2023-07-06T03:48:51.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.512+0000 7f0da37fe700 1 -- 172.21.15.97:0/4155354637 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0da40cdae0 con 0x7f0da415c520 2023-07-06T03:48:51.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.512+0000 7f0da37fe700 1 -- 172.21.15.97:0/4155354637 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 190300926 0 0) 0x7f0d940035b0 con 0x7f0da4160dd0 2023-07-06T03:48:51.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.512+0000 7f0da3fff700 1 -- 172.21.15.97:0/4155354637 >> v1:172.21.15.146:6789/0 conn(0x7f0da415c520 legacy=0x7f0da4160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.512+0000 7f0da3fff700 1 -- 172.21.15.97:0/4155354637 >> v1:172.21.15.97:6789/0 conn(0x7f0da4160dd0 legacy=0x7f0da40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.512+0000 7f0da3fff700 1 -- 172.21.15.97:0/4155354637 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da40b6c00 con 0x7f0da40cd390 2023-07-06T03:48:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.512+0000 7f0da3fff700 1 --2- 172.21.15.97:0/4155354637 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da40cd390 0x7f0da4160890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009870 tx=0x7f0d90009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d27335b449cc4c09 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.512+0000 7f0da37fe700 1 -- 172.21.15.97:0/4155354637 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0019f0 con 0x7f0da40cd390 2023-07-06T03:48:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.512+0000 7f0da37fe700 1 -- 172.21.15.97:0/4155354637 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d90004030 con 0x7f0da40cd390 2023-07-06T03:48:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.513+0000 7f0da37fe700 1 -- 172.21.15.97:0/4155354637 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d900043c0 con 0x7f0da40cd390 2023-07-06T03:48:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.513+0000 7f0daadff700 1 -- 172.21.15.97:0/4155354637 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da40cd390 msgr2=0x7f0da4160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.513+0000 7f0daadff700 1 --2- 172.21.15.97:0/4155354637 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da40cd390 0x7f0da4160890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009870 tx=0x7f0d90009b80 comp rx=0 tx=0).stop 2023-07-06T03:48:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.513+0000 7f0daadff700 1 -- 172.21.15.97:0/4155354637 shutdown_connections 2023-07-06T03:48:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.513+0000 7f0daadff700 1 --2- 172.21.15.97:0/4155354637 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da40cd390 0x7f0da4160890 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.513+0000 7f0daadff700 1 -- 172.21.15.97:0/4155354637 >> 172.21.15.97:0/4155354637 conn(0x7f0da4071790 msgr2=0x7f0da4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.513+0000 7f0daadff700 1 -- 172.21.15.97:0/4155354637 shutdown_connections 2023-07-06T03:48:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.513+0000 7f0daadff700 1 -- 172.21.15.97:0/4155354637 wait complete. 2023-07-06T03:48:51.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.515+0000 7f0daadff700 1 Processor -- start 2023-07-06T03:48:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.515+0000 7f0daadff700 1 -- start start 2023-07-06T03:48:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.516+0000 7f0daadff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40cd390 0x7f0da40c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.516+0000 7f0daadff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da415c520 0x7f0da414d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.516+0000 7f0daadff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0da4160dd0 0x7f0da40d3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.516+0000 7f0daadff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da415dc10 con 0x7f0da40cd390 2023-07-06T03:48:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.516+0000 7f0daadff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da40d1650 con 0x7f0da4160dd0 2023-07-06T03:48:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.516+0000 7f0daadff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da40ca350 con 0x7f0da415c520 2023-07-06T03:48:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.516+0000 7f0da8b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40cd390 0x7f0da40c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.516+0000 7f0da8b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40cd390 0x7f0da40c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49574/0 (socket says 172.21.15.97:49574) 2023-07-06T03:48:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.516+0000 7f0da8b9b700 1 -- 172.21.15.97:0/1928572290 learned_addr learned my addr 172.21.15.97:0/1928572290 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.516+0000 7f0da939c700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0da4160dd0 0x7f0da40d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0da3fff700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da415c520 0x7f0da414d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0da8b9b700 1 -- 172.21.15.97:0/1928572290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da415c520 msgr2=0x7f0da414d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0da8b9b700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da415c520 0x7f0da414d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0da8b9b700 1 -- 172.21.15.97:0/1928572290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0da4160dd0 msgr2=0x7f0da40d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0da8b9b700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0da4160dd0 0x7f0da40d3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0da8b9b700 1 -- 172.21.15.97:0/1928572290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900094a0 con 0x7f0da40cd390 2023-07-06T03:48:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0da3fff700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da415c520 0x7f0da414d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0da8b9b700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40cd390 0x7f0da40c9e10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800c550 tx=0x7f0d9800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0da17fa700 1 -- 172.21.15.97:0/1928572290 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d980052d0 con 0x7f0da40cd390 2023-07-06T03:48:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0da17fa700 1 -- 172.21.15.97:0/1928572290 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98005430 con 0x7f0da40cd390 2023-07-06T03:48:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0daadff700 1 -- 172.21.15.97:0/1928572290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da414d7c0 con 0x7f0da40cd390 2023-07-06T03:48:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0da17fa700 1 -- 172.21.15.97:0/1928572290 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d980038c0 con 0x7f0da40cd390 2023-07-06T03:48:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.517+0000 7f0daadff700 1 -- 172.21.15.97:0/1928572290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da40d1b80 con 0x7f0da40cd390 2023-07-06T03:48:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.518+0000 7f0daadff700 1 -- 172.21.15.97:0/1928572290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da404e340 con 0x7f0da40cd390 2023-07-06T03:48:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.519+0000 7f0da17fa700 1 -- 172.21.15.97:0/1928572290 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d98003a20 con 0x7f0da40cd390 2023-07-06T03:48:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.520+0000 7f0da17fa700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d8806ace0 0x7f0d8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.520+0000 7f0da17fa700 1 -- 172.21.15.97:0/1928572290 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d98088a50 con 0x7f0da40cd390 2023-07-06T03:48:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.520+0000 7f0da3fff700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d8806ace0 0x7f0d8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.521+0000 7f0da3fff700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d8806ace0 0x7f0d8806d190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0da4072660 tx=0x7f0d90008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:51.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.524+0000 7f0da17fa700 1 -- 172.21.15.97:0/1928572290 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0d98053af0 con 0x7f0da40cd390 2023-07-06T03:48:51.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.762+0000 7f0daadff700 1 -- 172.21.15.97:0/1928572290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0da40b4f40 con 0x7f0da40cd390 2023-07-06T03:48:51.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.762+0000 7f0da17fa700 1 -- 172.21.15.97:0/1928572290 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4678 (secure 0 0 0) 0x7f0d98057110 con 0x7f0da40cd390 2023-07-06T03:48:51.765 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:48:51.766 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1108873506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1108873506},{"type":"v1","addr":"172.21.15.146:6835","nonce":1108873506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2471899168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2471899168},{"type":"v1","addr":"172.21.15.97:6837","nonce":2471899168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2970792733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2970792733},{"type":"v1","addr":"172.21.15.97:6836","nonce":2970792733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:48:45.401033+0000","modified":"2023-07-06T03:48:48.930905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6399":{"gid":6399,"name":"d","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/2151424592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2151424592},{"type":"v1","addr":"172.21.15.146:6834","nonce":2151424592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-06T03:48:51.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.764+0000 7f0daadff700 1 -- 172.21.15.97:0/1928572290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d8806ace0 msgr2=0x7f0d8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.764+0000 7f0daadff700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d8806ace0 0x7f0d8806d190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0da4072660 tx=0x7f0d90008030 comp rx=0 tx=0).stop 2023-07-06T03:48:51.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.764+0000 7f0daadff700 1 -- 172.21.15.97:0/1928572290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40cd390 msgr2=0x7f0da40c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:51.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.764+0000 7f0daadff700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40cd390 0x7f0da40c9e10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800c550 tx=0x7f0d9800c910 comp rx=0 tx=0).stop 2023-07-06T03:48:51.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.764+0000 7f0daadff700 1 -- 172.21.15.97:0/1928572290 shutdown_connections 2023-07-06T03:48:51.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.764+0000 7f0daadff700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d8806ace0 0x7f0d8806d190 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.764+0000 7f0daadff700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0da4160dd0 0x7f0da40d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.764+0000 7f0daadff700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da40cd390 0x7f0da40c9e10 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.764+0000 7f0daadff700 1 --2- 172.21.15.97:0/1928572290 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0da415c520 0x7f0da414d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:51.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.764+0000 7f0daadff700 1 -- 172.21.15.97:0/1928572290 >> 172.21.15.97:0/1928572290 conn(0x7f0da4071790 msgr2=0x7f0da415ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:51.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.765+0000 7f0daadff700 1 -- 172.21.15.97:0/1928572290 shutdown_connections 2023-07-06T03:48:51.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:51.765+0000 7f0daadff700 1 -- 172.21.15.97:0/1928572290 wait complete. 2023-07-06T03:48:51.769 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 34 2023-07-06T03:48:51.889 INFO:tasks.ceph.mds.d.smithi146.stderr:2023-07-06T03:48:51.887+0000 7f9185bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:48:52.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T03:48:45.401033+0000', 'modified': '2023-07-06T03:48:48.930905+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6399}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6399': {'gid': 6399, 'name': 'd', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6834/2151424592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 2151424592}, {'type': 'v1', 'addr': '172.21.15.146:6834', 'nonce': 2151424592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T03:48:52.188 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T03:48:52.188 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-07-06T03:48:52.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.390+0000 7f5f36bff700 1 Processor -- start 2023-07-06T03:48:52.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.390+0000 7f5f36bff700 1 -- start start 2023-07-06T03:48:52.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.391+0000 7f5f36bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5f3015c610 0x7f5f300b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.391+0000 7f5f36bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f300d2210 0x7f5f300d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.391+0000 7f5f36bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f300cff90 con 0x7f5f3006b360 2023-07-06T03:48:52.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.391+0000 7f5f36bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f300d0780 con 0x7f5f3015c610 2023-07-06T03:48:52.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.391+0000 7f5f36bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3015c9e0 con 0x7f5f300d2210 2023-07-06T03:48:52.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.391+0000 7f5f3499b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5f3015c610 0x7f5f300b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.391+0000 7f5f2ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5f3006b360 0x7f5f300b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59130/0 (socket says 172.21.15.97:59130) 2023-07-06T03:48:52.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.391+0000 7f5f2ffff700 1 -- 172.21.15.97:0/2157029975 learned_addr learned my addr 172.21.15.97:0/2157029975 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:52.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.392+0000 7f5f3519c700 1 --2- 172.21.15.97:0/2157029975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f300d2210 0x7f5f300d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.392+0000 7f5f3499b700 1 -- 172.21.15.97:0/2157029975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f300d2210 msgr2=0x7f5f300d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:52.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.392+0000 7f5f3499b700 1 --2- 172.21.15.97:0/2157029975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f300d2210 0x7f5f300d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.392+0000 7f5f3499b700 1 -- 172.21.15.97:0/2157029975 >> v1:172.21.15.97:6789/0 conn(0x7f5f3006b360 legacy=0x7f5f300b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.392+0000 7f5f3499b700 1 -- 172.21.15.97:0/2157029975 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f300b5650 con 0x7f5f3015c610 2023-07-06T03:48:52.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.392+0000 7f5f2f7fe700 1 -- 172.21.15.97:0/2157029975 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421325588 0 0) 0x7f5f300cff90 con 0x7f5f3006b360 2023-07-06T03:48:52.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.392+0000 7f5f3499b700 1 --2- 172.21.15.97:0/2157029975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f3015c610 0x7f5f300b47b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009840 tx=0x7f5f24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7728fb646a7edb06 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:52.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.392+0000 7f5f2f7fe700 1 -- 172.21.15.97:0/2157029975 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24011830 con 0x7f5f3015c610 2023-07-06T03:48:52.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.392+0000 7f5f2f7fe700 1 -- 172.21.15.97:0/2157029975 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f24011990 con 0x7f5f3015c610 2023-07-06T03:48:52.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.392+0000 7f5f2f7fe700 1 -- 172.21.15.97:0/2157029975 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24011d20 con 0x7f5f3015c610 2023-07-06T03:48:52.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.393+0000 7f5f36bff700 1 -- 172.21.15.97:0/2157029975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f3015c610 msgr2=0x7f5f300b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.393+0000 7f5f36bff700 1 --2- 172.21.15.97:0/2157029975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f3015c610 0x7f5f300b47b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009840 tx=0x7f5f24009b50 comp rx=0 tx=0).stop 2023-07-06T03:48:52.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.393+0000 7f5f36bff700 1 -- 172.21.15.97:0/2157029975 shutdown_connections 2023-07-06T03:48:52.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.393+0000 7f5f36bff700 1 --2- 172.21.15.97:0/2157029975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f3015c610 0x7f5f300b47b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.393+0000 7f5f36bff700 1 --2- 172.21.15.97:0/2157029975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f300d2210 0x7f5f300d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.393+0000 7f5f36bff700 1 -- 172.21.15.97:0/2157029975 >> 172.21.15.97:0/2157029975 conn(0x7f5f3006a6c0 msgr2=0x7f5f3006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:52.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.393+0000 7f5f36bff700 1 -- 172.21.15.97:0/2157029975 shutdown_connections 2023-07-06T03:48:52.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.393+0000 7f5f36bff700 1 -- 172.21.15.97:0/2157029975 wait complete. 2023-07-06T03:48:52.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.395+0000 7f5f36bff700 1 Processor -- start 2023-07-06T03:48:52.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.395+0000 7f5f36bff700 1 -- start start 2023-07-06T03:48:52.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.396+0000 7f5f36bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f3006b360 0x7f5f300b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.396+0000 7f5f36bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f300d2210 0x7f5f30152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.396+0000 7f5f36bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f3015c610 0x7f5f30153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.396+0000 7f5f36bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3015dee0 con 0x7f5f3015c610 2023-07-06T03:48:52.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.396+0000 7f5f36bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300ca1e0 con 0x7f5f3006b360 2023-07-06T03:48:52.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.396+0000 7f5f36bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300ca4c0 con 0x7f5f300d2210 2023-07-06T03:48:52.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.396+0000 7f5f3519c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f3015c610 0x7f5f30153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.396+0000 7f5f3519c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f3015c610 0x7f5f30153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49580/0 (socket says 172.21.15.97:49580) 2023-07-06T03:48:52.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.396+0000 7f5f3519c700 1 -- 172.21.15.97:0/4101993636 learned_addr learned my addr 172.21.15.97:0/4101993636 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:52.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.396+0000 7f5f2ffff700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f300d2210 0x7f5f30152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.396+0000 7f5f3499b700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f3006b360 0x7f5f300b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.397+0000 7f5f3519c700 1 -- 172.21.15.97:0/4101993636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f300d2210 msgr2=0x7f5f30152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.397+0000 7f5f3519c700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f300d2210 0x7f5f30152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.397+0000 7f5f3519c700 1 -- 172.21.15.97:0/4101993636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f3006b360 msgr2=0x7f5f300b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.397+0000 7f5f3519c700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f3006b360 0x7f5f300b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.397+0000 7f5f3519c700 1 -- 172.21.15.97:0/4101993636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240094a0 con 0x7f5f3015c610 2023-07-06T03:48:52.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.397+0000 7f5f3519c700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f3015c610 0x7f5f30153350 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5f28007e40 tx=0x7f5f2800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:52.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.397+0000 7f5f2d7fa700 1 -- 172.21.15.97:0/4101993636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2800ed30 con 0x7f5f3015c610 2023-07-06T03:48:52.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.397+0000 7f5f2d7fa700 1 -- 172.21.15.97:0/4101993636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f28014420 con 0x7f5f3015c610 2023-07-06T03:48:52.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.397+0000 7f5f36bff700 1 -- 172.21.15.97:0/4101993636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f300ca7a0 con 0x7f5f3015c610 2023-07-06T03:48:52.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.397+0000 7f5f2d7fa700 1 -- 172.21.15.97:0/4101993636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280147d0 con 0x7f5f3015c610 2023-07-06T03:48:52.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.397+0000 7f5f36bff700 1 -- 172.21.15.97:0/4101993636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f300cad60 con 0x7f5f3015c610 2023-07-06T03:48:52.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.399+0000 7f5f2d7fa700 1 -- 172.21.15.97:0/4101993636 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f28014930 con 0x7f5f3015c610 2023-07-06T03:48:52.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.400+0000 7f5f2d7fa700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f1406ace0 0x7f5f1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.400+0000 7f5f2d7fa700 1 -- 172.21.15.97:0/4101993636 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f5f28089b60 con 0x7f5f3015c610 2023-07-06T03:48:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.400+0000 7f5f3499b700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f1406ace0 0x7f5f1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.401+0000 7f5f3499b700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f1406ace0 0x7f5f1406d190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5f240090b0 tx=0x7f5f2401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.402+0000 7f5f4a4e2b80 1 Processor -- start 2023-07-06T03:48:52.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.403+0000 7f5f4a4e2b80 1 -- start start 2023-07-06T03:48:52.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.403+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426228d50 0x55c426229120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.403+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c42624ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.403+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426203b50 0x55c426206110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.403+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42617aea0 con 0x55c42622e190 2023-07-06T03:48:52.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.403+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4260ee600 con 0x55c426228d50 2023-07-06T03:48:52.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.403+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c42615d910 con 0x55c426203b50 2023-07-06T03:48:52.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f2ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c42624ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f3499b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426228d50 0x55c426229120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f2ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c42624ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49582/0 (socket says 172.21.15.97:49582) 2023-07-06T03:48:52.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f2ffff700 1 -- 172.21.15.97:0/1258106918 learned_addr learned my addr 172.21.15.97:0/1258106918 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:52.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f3519c700 1 --2- 172.21.15.97:0/1258106918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426203b50 0x55c426206110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f2ffff700 1 -- 172.21.15.97:0/1258106918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426203b50 msgr2=0x55c426206110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:52.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f2ffff700 1 --2- 172.21.15.97:0/1258106918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426203b50 0x55c426206110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f2ffff700 1 -- 172.21.15.97:0/1258106918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426228d50 msgr2=0x55c426229120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:52.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f2ffff700 1 --2- 172.21.15.97:0/1258106918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426228d50 0x55c426229120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f2ffff700 1 -- 172.21.15.97:0/1258106918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c425f4a790 con 0x55c42622e190 2023-07-06T03:48:52.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f2ffff700 1 --2- 172.21.15.97:0/1258106918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c42624ca30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c004f60 tx=0x7f5f1c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb3117eadb7cd46 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.404+0000 7f5f1a7fc700 1 -- 172.21.15.97:0/1258106918 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c017070 con 0x55c42622e190 2023-07-06T03:48:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.405+0000 7f5f1a7fc700 1 -- 172.21.15.97:0/1258106918 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c001640 con 0x55c42622e190 2023-07-06T03:48:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.405+0000 7f5f1a7fc700 1 -- 172.21.15.97:0/1258106918 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c004a00 con 0x55c42622e190 2023-07-06T03:48:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.405+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/1258106918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 msgr2=0x55c42624ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.405+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/1258106918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c42624ca30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c004f60 tx=0x7f5f1c004f90 comp rx=0 tx=0).stop 2023-07-06T03:48:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.405+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/1258106918 shutdown_connections 2023-07-06T03:48:52.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.405+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/1258106918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426203b50 0x55c426206110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.405+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/1258106918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426228d50 0x55c426229120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.405+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/1258106918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c42624ca30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.405+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/1258106918 >> 172.21.15.97:0/1258106918 conn(0x55c4261fb310 msgr2=0x55c42624b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:52.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.405+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/1258106918 shutdown_connections 2023-07-06T03:48:52.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.405+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/1258106918 wait complete. 2023-07-06T03:48:52.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.406+0000 7f5f4a4e2b80 1 -- start start 2023-07-06T03:48:52.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.406+0000 7f5f4a4e2b80 1 Processor -- start 2023-07-06T03:48:52.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.407+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426203b50 0x55c42622f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.407+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426228d50 0x55c426231850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.407+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c426225dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.407+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c426156be0 con 0x55c42622e190 2023-07-06T03:48:52.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.407+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4261145b0 con 0x55c426228d50 2023-07-06T03:48:52.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.407+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4260fa910 con 0x55c426203b50 2023-07-06T03:48:52.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.407+0000 7f5f4a4e2b80 10 client.0 _ll_register_callbacks cb 0x55c426112b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:48:52.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f3519c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c426225dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f3519c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c426225dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49594/0 (socket says 172.21.15.97:49594) 2023-07-06T03:48:52.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f3519c700 1 -- 172.21.15.97:0/4003713294 learned_addr learned my addr 172.21.15.97:0/4003713294 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:52.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f3499b700 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426203b50 0x55c42622f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f2ffff700 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426228d50 0x55c426231850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f3519c700 1 -- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426203b50 msgr2=0x55c42622f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:52.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f3519c700 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426203b50 0x55c42622f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f3519c700 1 -- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426228d50 msgr2=0x55c426231850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:52.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f3519c700 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426228d50 0x55c426231850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f3519c700 1 -- 172.21.15.97:0/4003713294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c004b60 con 0x55c42622e190 2023-07-06T03:48:52.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f2ffff700 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426228d50 0x55c426231850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:48:52.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f3519c700 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c426225dd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e710 tx=0x7f5f28060be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c9fc8bb4a11805 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:52.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:48:52.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/4003713294 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28085070 con 0x55c42622e190 2023-07-06T03:48:52.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/4003713294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4261f8920 con 0x55c42622e190 2023-07-06T03:48:52.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.408+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/4003713294 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f5f28060960 con 0x55c42622e190 2023-07-06T03:48:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/4003713294 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f28054680 con 0x55c42622e190 2023-07-06T03:48:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f4a4e2b80 10 client.6540 fetch_fsmap learned FSMap version 34 2023-07-06T03:48:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/4003713294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55c4260370a0 con 0x55c42622e190 2023-07-06T03:48:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/4003713294 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c4261f8920 con 0x55c42622e190 2023-07-06T03:48:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/4003713294 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5f280547e0 con 0x55c42622e190 2023-07-06T03:48:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f4a4e2b80 10 client.6540 fetch_fsmap finished waiting for FSMap version 34 2023-07-06T03:48:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f4a4e2b80 10 client.6540 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T03:48:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f4a4e2b80 20 client.6540 populate_metadata read hostname 'smithi097' 2023-07-06T03:48:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:6832/2151424592,v1:172.21.15.146:6834/2151424592] conn(0x55c42624cee0 0x55c42624d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f4a4e2b80 4 client.6540 mds_command: new command op to 6399 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T03:48:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/4003713294 --> [v2:172.21.15.146:6832/2151424592,v1:172.21.15.146:6834/2151424592] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4261d50d0 con 0x55c42624cee0 2023-07-06T03:48:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.409+0000 7f5f3499b700 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:6832/2151424592,v1:172.21.15.146:6834/2151424592] conn(0x55c42624cee0 0x55c42624d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.410+0000 7f5f0f7fe700 10 client.6540 ms_handle_connect on v2:172.21.15.146:6832/2151424592 2023-07-06T03:48:52.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.410+0000 7f5f3499b700 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:6832/2151424592,v1:172.21.15.146:6834/2151424592] conn(0x55c42624cee0 0x55c42624d2b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d1ad08682b271d server_cookie=df57e0ba0769daff in_seq=0 out_seq=0 2023-07-06T03:48:52.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/4003713294 <== mds.0 v2:172.21.15.146:6832/2151424592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c4261d50d0 con 0x55c42624cee0 2023-07-06T03:48:52.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f0f7fe700 10 client.6540 handle_command_reply: tid=0 2023-07-06T03:48:52.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f4a4e2b80 1 client.6540 shutdown 2023-07-06T03:48:52.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/4003713294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 msgr2=0x55c426225dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c426225dd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e710 tx=0x7f5f28060be0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/4003713294 shutdown_connections 2023-07-06T03:48:52.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:6832/2151424592,v1:172.21.15.146:6834/2151424592] conn(0x55c42624cee0 0x55c42624d2b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42622e190 0x55c426225dd0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426203b50 0x55c42622f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/4003713294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426228d50 0x55c426231850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f0f7fe700 0 client.6540 ms_handle_reset on v2:172.21.15.146:6832/2151424592 2023-07-06T03:48:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.411+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/4003713294 >> 172.21.15.97:0/4003713294 conn(0x55c4261fb310 msgr2=0x55c42622e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.412+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/4003713294 shutdown_connections 2023-07-06T03:48:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.412+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/4003713294 wait complete. 2023-07-06T03:48:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.412+0000 7f5f4a4e2b80 20 client.6540 trim_cache size 0 max 16384 2023-07-06T03:48:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.430+0000 7f5f4a4e2b80 1 Processor -- start 2023-07-06T03:48:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.430+0000 7f5f4a4e2b80 1 -- start start 2023-07-06T03:48:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426203b50 0x55c426244860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426228d50 0x55c4261ff5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c42622e190 0x55c4262302b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c425f4b390 con 0x55c426203b50 2023-07-06T03:48:52.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4261f8460 con 0x55c42622e190 2023-07-06T03:48:52.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c426232c80 con 0x55c426228d50 2023-07-06T03:48:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f3499b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426203b50 0x55c426244860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f3499b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426203b50 0x55c426244860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49610/0 (socket says 172.21.15.97:49610) 2023-07-06T03:48:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f3499b700 1 -- 172.21.15.97:0/3054030831 learned_addr learned my addr 172.21.15.97:0/3054030831 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f2ffff700 1 --2- 172.21.15.97:0/3054030831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426228d50 0x55c4261ff5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f3519c700 1 --2- 172.21.15.97:0/3054030831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c42622e190 0x55c4262302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f3499b700 1 -- 172.21.15.97:0/3054030831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426228d50 msgr2=0x55c4261ff5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f3499b700 1 --2- 172.21.15.97:0/3054030831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426228d50 0x55c4261ff5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f3499b700 1 -- 172.21.15.97:0/3054030831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c42622e190 msgr2=0x55c4262302b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:52.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f3499b700 1 --2- 172.21.15.97:0/3054030831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c42622e190 0x55c4262302b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.431+0000 7f5f3499b700 1 -- 172.21.15.97:0/3054030831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c42611cb10 con 0x55c426203b50 2023-07-06T03:48:52.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.432+0000 7f5f3499b700 1 --2- 172.21.15.97:0/3054030831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426203b50 0x55c426244860 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5f3008b120 tx=0x7f5f24010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3be23b8e1498c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:52.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.432+0000 7f5f1affd700 1 -- 172.21.15.97:0/3054030831 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240038a0 con 0x55c426203b50 2023-07-06T03:48:52.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.432+0000 7f5f1affd700 1 -- 172.21.15.97:0/3054030831 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2402a040 con 0x55c426203b50 2023-07-06T03:48:52.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.432+0000 7f5f1affd700 1 -- 172.21.15.97:0/3054030831 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24003c00 con 0x55c426203b50 2023-07-06T03:48:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.432+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3054030831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426203b50 msgr2=0x55c426244860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.432+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/3054030831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426203b50 0x55c426244860 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5f3008b120 tx=0x7f5f24010000 comp rx=0 tx=0).stop 2023-07-06T03:48:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.432+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3054030831 shutdown_connections 2023-07-06T03:48:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.432+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/3054030831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c42622e190 0x55c4262302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.432+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/3054030831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426203b50 0x55c426244860 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.432+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/3054030831 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c426228d50 0x55c4261ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.432+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3054030831 >> 172.21.15.97:0/3054030831 conn(0x55c4261fb310 msgr2=0x55c42626d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.433+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3054030831 shutdown_connections 2023-07-06T03:48:52.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.433+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3054030831 wait complete. 2023-07-06T03:48:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.433+0000 7f5f4a4e2b80 1 -- start start 2023-07-06T03:48:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.433+0000 7f5f4a4e2b80 1 Processor -- start 2023-07-06T03:48:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.434+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426203b50 0x55c426297d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.434+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426228d50 0x55c426229120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.434+0000 7f5f4a4e2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c42622e190 0x55c426229660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.434+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4262044a0 con 0x55c426228d50 2023-07-06T03:48:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.434+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4262abf20 con 0x55c426203b50 2023-07-06T03:48:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.434+0000 7f5f4a4e2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4261ffac0 con 0x55c42622e190 2023-07-06T03:48:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f4a4e2b80 10 client.0 _ll_register_callbacks cb 0x55c426112b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:48:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f2ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426228d50 0x55c426229120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f2ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426228d50 0x55c426229120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49614/0 (socket says 172.21.15.97:49614) 2023-07-06T03:48:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f2ffff700 1 -- 172.21.15.97:0/3259210461 learned_addr learned my addr 172.21.15.97:0/3259210461 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f3499b700 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426203b50 0x55c426297d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f3519c700 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c42622e190 0x55c426229660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f2ffff700 1 -- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c42622e190 msgr2=0x55c426229660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f2ffff700 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c42622e190 0x55c426229660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f2ffff700 1 -- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426203b50 msgr2=0x55c426297d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f2ffff700 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426203b50 0x55c426297d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f2ffff700 1 -- 172.21.15.97:0/3259210461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f300b5650 con 0x55c426228d50 2023-07-06T03:48:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f3499b700 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426203b50 0x55c426297d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:48:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f2ffff700 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426228d50 0x55c426229120 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c015040 tx=0x7f5f1c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cfb69cd67e3005 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.435+0000 7f5f0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:48:52.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/3259210461 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c017050 con 0x55c426228d50 2023-07-06T03:48:52.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/3259210461 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f5f1c007a80 con 0x55c426228d50 2023-07-06T03:48:52.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3259210461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f1c001640 con 0x55c426228d50 2023-07-06T03:48:52.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/3259210461 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f1c022070 con 0x55c426228d50 2023-07-06T03:48:52.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f4a4e2b80 10 client.6546 fetch_fsmap learned FSMap version 34 2023-07-06T03:48:52.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3259210461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55c425fab1d0 con 0x55c426228d50 2023-07-06T03:48:52.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/3259210461 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c001640 con 0x55c426228d50 2023-07-06T03:48:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/3259210461 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5f1c012070 con 0x55c426228d50 2023-07-06T03:48:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f4a4e2b80 10 client.6546 fetch_fsmap finished waiting for FSMap version 34 2023-07-06T03:48:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f4a4e2b80 10 client.6546 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T03:48:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f4a4e2b80 20 client.6546 populate_metadata read hostname 'smithi097' 2023-07-06T03:48:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:6832/2151424592,v1:172.21.15.146:6834/2151424592] conn(0x55c426204960 0x55c426204d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f4a4e2b80 4 client.6546 mds_command: new command op to 6399 tid=0[{"prefix": "status"}] 2023-07-06T03:48:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.436+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3259210461 --> [v2:172.21.15.146:6832/2151424592,v1:172.21.15.146:6834/2151424592] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c42614fd20 con 0x55c426204960 2023-07-06T03:48:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.437+0000 7f5f3499b700 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:6832/2151424592,v1:172.21.15.146:6834/2151424592] conn(0x55c426204960 0x55c426204d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.437+0000 7f5f0f7fe700 10 client.6546 ms_handle_connect on v2:172.21.15.146:6832/2151424592 2023-07-06T03:48:52.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.437+0000 7f5f3499b700 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:6832/2151424592,v1:172.21.15.146:6834/2151424592] conn(0x55c426204960 0x55c426204d30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67fa31e51612ed4 server_cookie=be25121aeefddd09 in_seq=0 out_seq=0 2023-07-06T03:48:52.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.437+0000 7f5f0f7fe700 1 -- 172.21.15.97:0/3259210461 <== mds.0 v2:172.21.15.146:6832/2151424592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c42614fd20 con 0x55c426204960 2023-07-06T03:48:52.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.437+0000 7f5f0f7fe700 10 client.6546 handle_command_reply: tid=0 2023-07-06T03:48:52.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.437+0000 7f5f4a4e2b80 1 client.6546 shutdown 2023-07-06T03:48:52.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3259210461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426228d50 msgr2=0x55c426229120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426228d50 0x55c426229120 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c015040 tx=0x7f5f1c000ed0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3259210461 shutdown_connections 2023-07-06T03:48:52.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:6832/2151424592,v1:172.21.15.146:6834/2151424592] conn(0x55c426204960 0x55c426204d30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55c42622e190 0x55c426229660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c426203b50 0x55c426297d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f0f7fe700 0 client.6546 ms_handle_reset on v2:172.21.15.146:6832/2151424592 2023-07-06T03:48:52.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f4a4e2b80 1 --2- 172.21.15.97:0/3259210461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c426228d50 0x55c426229120 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3259210461 >> 172.21.15.97:0/3259210461 conn(0x55c4261fb310 msgr2=0x55c4262003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:52.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3259210461 shutdown_connections 2023-07-06T03:48:52.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f4a4e2b80 1 -- 172.21.15.97:0/3259210461 wait complete. 2023-07-06T03:48:52.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f4a4e2b80 20 client.6546 trim_cache size 0 max 16384 2023-07-06T03:48:52.449 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:48:52.449 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36", 2023-07-06T03:48:52.449 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-07-06T03:48:52.450 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6399, 2023-07-06T03:48:52.450 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-07-06T03:48:52.450 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-07-06T03:48:52.450 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-07-06T03:48:52.450 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.5335982939999999, 2023-07-06T03:48:52.450 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 34, 2023-07-06T03:48:52.450 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 61, 2023-07-06T03:48:52.450 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 60, 2023-07-06T03:48:52.451 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 12.023808453999999 2023-07-06T03:48:52.451 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:48:52.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.438+0000 7f5f36bff700 1 -- 172.21.15.97:0/4101993636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f1406ace0 msgr2=0x7f5f1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.439+0000 7f5f36bff700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f1406ace0 0x7f5f1406d190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5f240090b0 tx=0x7f5f2401c040 comp rx=0 tx=0).stop 2023-07-06T03:48:52.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.439+0000 7f5f36bff700 1 -- 172.21.15.97:0/4101993636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f3015c610 msgr2=0x7f5f30153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.439+0000 7f5f36bff700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f3015c610 0x7f5f30153350 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5f28007e40 tx=0x7f5f2800e3c0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.439+0000 7f5f36bff700 1 -- 172.21.15.97:0/4101993636 shutdown_connections 2023-07-06T03:48:52.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.439+0000 7f5f36bff700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5f1406ace0 0x7f5f1406d190 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.439+0000 7f5f36bff700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f3015c610 0x7f5f30153350 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.439+0000 7f5f36bff700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f3006b360 0x7f5f300b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.439+0000 7f5f36bff700 1 --2- 172.21.15.97:0/4101993636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5f300d2210 0x7f5f30152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.439+0000 7f5f36bff700 1 -- 172.21.15.97:0/4101993636 >> 172.21.15.97:0/4101993636 conn(0x7f5f3006a6c0 msgr2=0x7f5f300b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:52.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.439+0000 7f5f36bff700 1 -- 172.21.15.97:0/4101993636 shutdown_connections 2023-07-06T03:48:52.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.439+0000 7f5f36bff700 1 -- 172.21.15.97:0/4101993636 wait complete. 2023-07-06T03:48:52.458 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:48:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.650+0000 7f9ec37ff700 1 Processor -- start 2023-07-06T03:48:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.650+0000 7f9ec37ff700 1 -- start start 2023-07-06T03:48:52.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.652+0000 7f9ec37ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ebc0b8410 0x7f9ebc167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.652+0000 7f9ec37ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc0cdae0 con 0x7f9ebc15c520 2023-07-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.652+0000 7f9ec37ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc0b6640 con 0x7f9ebc0cd390 2023-07-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.652+0000 7f9ec37ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ebc0b6920 con 0x7f9ebc0b8410 2023-07-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.652+0000 7f9ec1d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ebc0b8410 0x7f9ebc167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.652+0000 7f9ec1d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ebc0b8410 0x7f9ebc167510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49616/0 (socket says 172.21.15.97:49616) 2023-07-06T03:48:52.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.652+0000 7f9ec1d9c700 1 -- 172.21.15.97:0/442701726 learned_addr learned my addr 172.21.15.97:0/442701726 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:52.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.652+0000 7f9ec1d9c700 1 -- 172.21.15.97:0/442701726 >> v1:172.21.15.146:6789/0 conn(0x7f9ebc0cd390 legacy=0x7f9ebc160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.652+0000 7f9ec1d9c700 1 -- 172.21.15.97:0/442701726 >> v1:172.21.15.97:6789/0 conn(0x7f9ebc15c520 legacy=0x7f9ebc160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.652+0000 7f9ec1d9c700 1 -- 172.21.15.97:0/442701726 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc0b6c00 con 0x7f9ebc0b8410 2023-07-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.653+0000 7f9ec1d9c700 1 --2- 172.21.15.97:0/442701726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc0b8410 0x7f9ebc167510 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009e50 tx=0x7f9eb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cc4f6cb7b51e31 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.653+0000 7f9eb3fff700 1 -- 172.21.15.97:0/442701726 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400e070 con 0x7f9ebc0b8410 2023-07-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.653+0000 7f9eb3fff700 1 -- 172.21.15.97:0/442701726 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb400b9c0 con 0x7f9ebc0b8410 2023-07-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.653+0000 7f9eb3fff700 1 -- 172.21.15.97:0/442701726 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400bd70 con 0x7f9ebc0b8410 2023-07-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.653+0000 7f9ec37ff700 1 -- 172.21.15.97:0/442701726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc0b8410 msgr2=0x7f9ebc167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.653+0000 7f9ec37ff700 1 --2- 172.21.15.97:0/442701726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc0b8410 0x7f9ebc167510 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009e50 tx=0x7f9eb400b330 comp rx=0 tx=0).stop 2023-07-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.653+0000 7f9ec37ff700 1 -- 172.21.15.97:0/442701726 shutdown_connections 2023-07-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.653+0000 7f9ec37ff700 1 --2- 172.21.15.97:0/442701726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc0b8410 0x7f9ebc167510 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.653+0000 7f9ec37ff700 1 -- 172.21.15.97:0/442701726 >> 172.21.15.97:0/442701726 conn(0x7f9ebc071790 msgr2=0x7f9ebc071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.653+0000 7f9ec37ff700 1 -- 172.21.15.97:0/442701726 shutdown_connections 2023-07-06T03:48:52.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.653+0000 7f9ec37ff700 1 -- 172.21.15.97:0/442701726 wait complete. 2023-07-06T03:48:52.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.655+0000 7f9ec37ff700 1 Processor -- start 2023-07-06T03:48:52.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.655+0000 7f9ec37ff700 1 -- start start 2023-07-06T03:48:52.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.656+0000 7f9ec37ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ebc0b8410 0x7f9ebc14b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.656+0000 7f9ec37ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc0cd390 0x7f9ebc1567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.656+0000 7f9ec37ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ebc15c520 0x7f9ebc156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.656+0000 7f9ec37ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0cbd30 con 0x7f9ebc0cd390 2023-07-06T03:48:52.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.656+0000 7f9ec37ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0cac90 con 0x7f9ebc15c520 2023-07-06T03:48:52.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.656+0000 7f9ec37ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0caf70 con 0x7f9ebc0b8410 2023-07-06T03:48:52.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.656+0000 7f9ec0d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc0cd390 0x7f9ebc1567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.656+0000 7f9ec0d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc0cd390 0x7f9ebc1567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49626/0 (socket says 172.21.15.97:49626) 2023-07-06T03:48:52.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.656+0000 7f9ec0d9a700 1 -- 172.21.15.97:0/1620220799 learned_addr learned my addr 172.21.15.97:0/1620220799 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:52.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9ec1d9c700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ebc15c520 0x7f9ebc156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9ec159b700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ebc0b8410 0x7f9ebc14b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9ec0d9a700 1 -- 172.21.15.97:0/1620220799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ebc0b8410 msgr2=0x7f9ebc14b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9ec0d9a700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ebc0b8410 0x7f9ebc14b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9ec0d9a700 1 -- 172.21.15.97:0/1620220799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ebc15c520 msgr2=0x7f9ebc156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9ec0d9a700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ebc15c520 0x7f9ebc156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9ec0d9a700 1 -- 172.21.15.97:0/1620220799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4009b30 con 0x7f9ebc0cd390 2023-07-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9ec0d9a700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc0cd390 0x7f9ebc1567c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00c4e0 tx=0x7f9eac00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:52.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9eb1ffb700 1 -- 172.21.15.97:0/1620220799 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0052d0 con 0x7f9ebc0cd390 2023-07-06T03:48:52.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9eb1ffb700 1 -- 172.21.15.97:0/1620220799 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac005430 con 0x7f9ebc0cd390 2023-07-06T03:48:52.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9ec37ff700 1 -- 172.21.15.97:0/1620220799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ebc0cb250 con 0x7f9ebc0cd390 2023-07-06T03:48:52.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.657+0000 7f9eb1ffb700 1 -- 172.21.15.97:0/1620220799 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0038c0 con 0x7f9ebc0cd390 2023-07-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.658+0000 7f9ec37ff700 1 -- 172.21.15.97:0/1620220799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc0cb810 con 0x7f9ebc0cd390 2023-07-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.658+0000 7f9ec37ff700 1 -- 172.21.15.97:0/1620220799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ebc04e340 con 0x7f9ebc0cd390 2023-07-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.659+0000 7f9eb1ffb700 1 -- 172.21.15.97:0/1620220799 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9eac003a20 con 0x7f9ebc0cd390 2023-07-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.660+0000 7f9eb1ffb700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ea006ace0 0x7f9ea006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.660+0000 7f9eb1ffb700 1 -- 172.21.15.97:0/1620220799 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f9eac088950 con 0x7f9ebc0cd390 2023-07-06T03:48:52.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.660+0000 7f9ec159b700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ea006ace0 0x7f9ea006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:52.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.661+0000 7f9ec159b700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ea006ace0 0x7f9ea006d190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8007fd0 tx=0x7f9eb8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:52.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.664+0000 7f9eb1ffb700 1 -- 172.21.15.97:0/1620220799 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9eac053890 con 0x7f9ebc0cd390 2023-07-06T03:48:52.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.904+0000 7f9ec37ff700 1 -- 172.21.15.97:0/1620220799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ebc0b4f40 con 0x7f9ebc0cd390 2023-07-06T03:48:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.904+0000 7f9eb1ffb700 1 -- 172.21.15.97:0/1620220799 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9eac056eb0 con 0x7f9ebc0cd390 2023-07-06T03:48:52.907 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.906+0000 7f9ec37ff700 1 -- 172.21.15.97:0/1620220799 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ea006ace0 msgr2=0x7f9ea006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.906+0000 7f9ec37ff700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ea006ace0 0x7f9ea006d190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8007fd0 tx=0x7f9eb8009040 comp rx=0 tx=0).stop 2023-07-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.906+0000 7f9ec37ff700 1 -- 172.21.15.97:0/1620220799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc0cd390 msgr2=0x7f9ebc1567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:52.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.906+0000 7f9ec37ff700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc0cd390 0x7f9ebc1567c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00c4e0 tx=0x7f9eac00c7f0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.906+0000 7f9ec37ff700 1 -- 172.21.15.97:0/1620220799 shutdown_connections 2023-07-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.906+0000 7f9ec37ff700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ea006ace0 0x7f9ea006d190 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.906+0000 7f9ec37ff700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ebc15c520 0x7f9ebc156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.906+0000 7f9ec37ff700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ebc0b8410 0x7f9ebc14b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.906+0000 7f9ec37ff700 1 --2- 172.21.15.97:0/1620220799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc0cd390 0x7f9ebc1567c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.906+0000 7f9ec37ff700 1 -- 172.21.15.97:0/1620220799 >> 172.21.15.97:0/1620220799 conn(0x7f9ebc071790 msgr2=0x7f9ebc0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.907+0000 7f9ec37ff700 1 -- 172.21.15.97:0/1620220799 shutdown_connections 2023-07-06T03:48:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:52.907+0000 7f9ec37ff700 1 -- 172.21.15.97:0/1620220799 wait complete. 2023-07-06T03:48:52.927 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:48:52.928 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:48:52.928 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:48:52.928 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:48:52.929 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:52.929 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:48:52.929 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:48:52.929 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:48:52.930 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:48:52.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:52.930 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:48:52.976 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:48:52.977 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:52.978 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41936sec preferred_lft 41936sec 2023-07-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:52.979 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:48:52.980 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:48:52.981 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:52.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:52.982 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:48:53.048 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:48:53.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:53.052 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:48:53.101 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:48:53.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:53.103 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:48:53.103 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:48:53.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:48:53.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:48:53.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:48:53.104 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:48:53.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:53.271 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:48:53.318 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:48:53.318 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:48:53.319 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:48:53.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:53.320 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:48:53.393 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:48:53.394 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:48:53.394 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:53.439 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:48:53.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:48:53.440 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:53.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:48:53.632 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:48:53.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:48:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:48:53.702 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:48:53.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:53.703 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:48:53.748 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:48:53.749 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:48:53.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:48:53.864 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:48:53.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:48:53.936 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:48:53.938 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:48:53.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:53.940 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:48:53.963 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:48:53.960+0000 7f71ec4df5c0 -1 init, newargv = 0x55d301adad50 newargc=15 2023-07-06T03:48:53.963 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[139442]: starting ceph client 2023-07-06T03:48:53.971 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[139442]: starting fuse 2023-07-06T03:48:53.989 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:48:53.989 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:48:53.990 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:48:53.990 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:48:53.991 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:48:53.991 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:48:53.991 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:48:53.992 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:53.992 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:48:53.993 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:48:53.993 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:48:53.993 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:48:53.993 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:53.994 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:48:53.994 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:48:53.994 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:48:53.995 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:48:53.995 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:53.995 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:48:53.996 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:48:53.996 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:48:53.996 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:48:53.997 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:48:53.997 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:48:53.997 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:48:53.997 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:48:53.998 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:53.998 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:48:53.998 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:48:54.172 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.139442.asok 2023-07-06T03:48:54.350 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.139442.asok 2023-07-06T03:48:54.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:54.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.139442.asok status 2023-07-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:48:54.576 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "139442", 2023-07-06T03:48:54.577 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:48:54.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:54.582 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:48:54.583 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:48:54.583 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6564, 2023-07-06T03:48:54.583 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6564 2023-07-06T03:48:54.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:54.585 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 837549164 2023-07-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:48:54.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:48:54.587 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:54.587 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 837549164 2023-07-06T03:48:54.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6564 192.168.0.1:0/837549164", 2023-07-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/837549164", 2023-07-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:48:54.588 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 34, 2023-07-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 62, 2023-07-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:48:54.589 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:48:54.590 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:48:54.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:54.592 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:48:54.592 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:54.639 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:48:54.639 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:54.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:54.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:48:54.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:48:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.941+0000 7f9fcabff700 1 Processor -- start 2023-07-06T03:48:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.941+0000 7f9fcabff700 1 -- start start 2023-07-06T03:48:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.942+0000 7f9fcabff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9fc415c520 0x7f9fc40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.942+0000 7f9fcabff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc40cdae0 con 0x7f9fc4160180 2023-07-06T03:48:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.942+0000 7f9fcabff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc40b6640 con 0x7f9fc4160d00 2023-07-06T03:48:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.942+0000 7f9fcabff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc40cd8d0 con 0x7f9fc415c520 2023-07-06T03:48:54.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.942+0000 7f9fc3fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9fc4160180 0x7f9fc41605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59136/0 (socket says 172.21.15.97:59136) 2023-07-06T03:48:54.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.942+0000 7f9fc3fff700 1 -- 172.21.15.97:0/2271279060 learned_addr learned my addr 172.21.15.97:0/2271279060 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:54.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc899b700 1 --2- 172.21.15.97:0/2271279060 >> v2:172.21.15.146:3300/0 conn(0x7f9fc415c520 0x7f9fc40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720013780 0 0) 0x7f9fc40cdae0 con 0x7f9fc4160180 2023-07-06T03:48:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fb40035b0 con 0x7f9fc4160180 2023-07-06T03:48:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3250632835 0 0) 0x7f9fb40035b0 con 0x7f9fc4160180 2023-07-06T03:48:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 >> v2:172.21.15.146:3300/0 conn(0x7f9fc415c520 msgr2=0x7f9fc40cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 --2- 172.21.15.97:0/2271279060 >> v2:172.21.15.146:3300/0 conn(0x7f9fc415c520 0x7f9fc40cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 >> v1:172.21.15.146:6789/0 conn(0x7f9fc4160d00 legacy=0x7f9fc40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40b6920 con 0x7f9fc4160180 2023-07-06T03:48:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f9fb0001670 con 0x7f9fc4160180 2023-07-06T03:48:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc899b700 1 --2- 172.21.15.97:0/2271279060 >> v2:172.21.15.146:3300/0 conn(0x7f9fc415c520 0x7f9fc40cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:48:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 >> v1:172.21.15.97:6789/0 conn(0x7f9fc4160180 legacy=0x7f9fc41605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:54.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 --2- 172.21.15.97:0/2271279060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb4003a00 0x7f9fb4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:54.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 --2- 172.21.15.97:0/2271279060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fb4004850 0x7f9fb4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:54.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 --2- 172.21.15.97:0/2271279060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fb4005200 0x7f9fb400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:54.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40cd8d0 con 0x7f9fb4003a00 2023-07-06T03:48:54.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb400c080 con 0x7f9fb4005200 2023-07-06T03:48:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb400c240 con 0x7f9fb4004850 2023-07-06T03:48:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.943+0000 7f9fc899b700 1 --2- 172.21.15.97:0/2271279060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb4003a00 0x7f9fb4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc3fff700 1 --2- 172.21.15.97:0/2271279060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fb4004850 0x7f9fb4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc919c700 1 --2- 172.21.15.97:0/2271279060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fb4005200 0x7f9fb400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc899b700 1 -- 172.21.15.97:0/2271279060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fb4004850 msgr2=0x7f9fb4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc899b700 1 --2- 172.21.15.97:0/2271279060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fb4004850 0x7f9fb4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc899b700 1 -- 172.21.15.97:0/2271279060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fb4005200 msgr2=0x7f9fb400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc899b700 1 --2- 172.21.15.97:0/2271279060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fb4005200 0x7f9fb400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc899b700 1 -- 172.21.15.97:0/2271279060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40035b0 con 0x7f9fb4003a00 2023-07-06T03:48:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc919c700 1 -- 172.21.15.97:0/2271279060 reap_dead start 2023-07-06T03:48:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc899b700 1 --2- 172.21.15.97:0/2271279060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb4003a00 0x7f9fb4003e10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009d30 tx=0x7f9fb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163b9c3b6e2df428 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8009e80 con 0x7f9fb4003a00 2023-07-06T03:48:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb800e9a0 con 0x7f9fb4003a00 2023-07-06T03:48:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.944+0000 7f9fc37fe700 1 -- 172.21.15.97:0/2271279060 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb800ed10 con 0x7f9fb4003a00 2023-07-06T03:48:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.945+0000 7f9fcabff700 1 -- 172.21.15.97:0/2271279060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb4003a00 msgr2=0x7f9fb4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.945+0000 7f9fcabff700 1 --2- 172.21.15.97:0/2271279060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb4003a00 0x7f9fb4003e10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009d30 tx=0x7f9fb4005880 comp rx=0 tx=0).stop 2023-07-06T03:48:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.945+0000 7f9fcabff700 1 -- 172.21.15.97:0/2271279060 shutdown_connections 2023-07-06T03:48:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.945+0000 7f9fcabff700 1 --2- 172.21.15.97:0/2271279060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fb4003a00 0x7f9fb4003e10 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.945+0000 7f9fcabff700 1 -- 172.21.15.97:0/2271279060 >> 172.21.15.97:0/2271279060 conn(0x7f9fc4071790 msgr2=0x7f9fc4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.945+0000 7f9fcabff700 1 -- 172.21.15.97:0/2271279060 shutdown_connections 2023-07-06T03:48:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.945+0000 7f9fcabff700 1 -- 172.21.15.97:0/2271279060 wait complete. 2023-07-06T03:48:54.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.947+0000 7f9fcabff700 1 Processor -- start 2023-07-06T03:48:54.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.947+0000 7f9fcabff700 1 -- start start 2023-07-06T03:48:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fcabff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fb4003a00 0x7f9fc415ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fcabff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc40d3980 0x7f9fc40c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:54.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fcabff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fc40ca350 0x7f9fc414cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:54.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fcabff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40ca7f0 con 0x7f9fc40d3980 2023-07-06T03:48:54.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fcabff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40caad0 con 0x7f9fc40ca350 2023-07-06T03:48:54.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fcabff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40cadb0 con 0x7f9fb4003a00 2023-07-06T03:48:54.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc40d3980 0x7f9fc40c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:54.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc40d3980 0x7f9fc40c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49652/0 (socket says 172.21.15.97:49652) 2023-07-06T03:48:54.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc3fff700 1 -- 172.21.15.97:0/746589219 learned_addr learned my addr 172.21.15.97:0/746589219 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:54.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc919c700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fc40ca350 0x7f9fc414cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:54.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc899b700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fb4003a00 0x7f9fc415ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:54.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc3fff700 1 -- 172.21.15.97:0/746589219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fb4003a00 msgr2=0x7f9fc415ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:54.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc3fff700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fb4003a00 0x7f9fc415ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:54.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc3fff700 1 -- 172.21.15.97:0/746589219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fc40ca350 msgr2=0x7f9fc414cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:54.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc3fff700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fc40ca350 0x7f9fc414cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:54.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc3fff700 1 -- 172.21.15.97:0/746589219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40cb090 con 0x7f9fc40d3980 2023-07-06T03:48:54.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc919c700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fc40ca350 0x7f9fc414cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:54.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.948+0000 7f9fc899b700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fb4003a00 0x7f9fc415ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:54.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.949+0000 7f9fc3fff700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc40d3980 0x7f9fc40c9e10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0007290 tx=0x7f9fb0007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:54.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.949+0000 7f9fc17fa700 1 -- 172.21.15.97:0/746589219 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0007830 con 0x7f9fc40d3980 2023-07-06T03:48:54.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.949+0000 7f9fc17fa700 1 -- 172.21.15.97:0/746589219 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb000f8d0 con 0x7f9fc40d3980 2023-07-06T03:48:54.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.949+0000 7f9fcabff700 1 -- 172.21.15.97:0/746589219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc40cb370 con 0x7f9fc40d3980 2023-07-06T03:48:54.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.949+0000 7f9fc17fa700 1 -- 172.21.15.97:0/746589219 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb000fc80 con 0x7f9fc40d3980 2023-07-06T03:48:54.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.949+0000 7f9fcabff700 1 -- 172.21.15.97:0/746589219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc415f500 con 0x7f9fc40d3980 2023-07-06T03:48:54.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.950+0000 7f9fcabff700 1 -- 172.21.15.97:0/746589219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc40cb5b0 con 0x7f9fc40d3980 2023-07-06T03:48:54.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.951+0000 7f9fc17fa700 1 -- 172.21.15.97:0/746589219 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9fb001d460 con 0x7f9fc40d3980 2023-07-06T03:48:54.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.953+0000 7f9fc17fa700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fa806ace0 0x7f9fa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:54.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.953+0000 7f9fc17fa700 1 -- 172.21.15.97:0/746589219 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f9fb0089500 con 0x7f9fc40d3980 2023-07-06T03:48:54.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.953+0000 7f9fc899b700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fa806ace0 0x7f9fa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:54.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.954+0000 7f9fc899b700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fa806ace0 0x7f9fa806d190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80090b0 tx=0x7f9fb8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:54.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:54.957+0000 7f9fc17fa700 1 -- 172.21.15.97:0/746589219 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9fc40cb5b0 con 0x7f9fc40d3980 2023-07-06T03:48:55.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.195+0000 7f9fcabff700 1 -- 172.21.15.97:0/746589219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9fc415f500 con 0x7f9fc40d3980 2023-07-06T03:48:55.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.195+0000 7f9fc17fa700 1 -- 172.21.15.97:0/746589219 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9fc415f500 con 0x7f9fc40d3980 2023-07-06T03:48:55.198 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:48:55.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.197+0000 7f9fcabff700 1 -- 172.21.15.97:0/746589219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fa806ace0 msgr2=0x7f9fa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.197+0000 7f9fcabff700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fa806ace0 0x7f9fa806d190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80090b0 tx=0x7f9fb8008030 comp rx=0 tx=0).stop 2023-07-06T03:48:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.197+0000 7f9fcabff700 1 -- 172.21.15.97:0/746589219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc40d3980 msgr2=0x7f9fc40c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.197+0000 7f9fcabff700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc40d3980 0x7f9fc40c9e10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0007290 tx=0x7f9fb0007650 comp rx=0 tx=0).stop 2023-07-06T03:48:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.198+0000 7f9fcabff700 1 -- 172.21.15.97:0/746589219 shutdown_connections 2023-07-06T03:48:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.198+0000 7f9fcabff700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fa806ace0 0x7f9fa806d190 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.198+0000 7f9fcabff700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fc40ca350 0x7f9fc414cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.198+0000 7f9fcabff700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fb4003a00 0x7f9fc415ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:55.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.198+0000 7f9fcabff700 1 --2- 172.21.15.97:0/746589219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc40d3980 0x7f9fc40c9e10 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:55.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.198+0000 7f9fcabff700 1 -- 172.21.15.97:0/746589219 >> 172.21.15.97:0/746589219 conn(0x7f9fc4071790 msgr2=0x7f9fc40b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:55.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.198+0000 7f9fcabff700 1 -- 172.21.15.97:0/746589219 shutdown_connections 2023-07-06T03:48:55.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:55.198+0000 7f9fcabff700 1 -- 172.21.15.97:0/746589219 wait complete. 2023-07-06T03:48:55.218 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:48:55.218 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:48:55.219 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2023-07-06T03:48:55.219 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T03:48:55.219 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T03:48:55.220 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:48:55.220 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:48:55.220 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:48:55.220 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:48:55.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:55.221 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2023-07-06T03:48:55.273 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:48:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:48:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:48:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:48:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:55.274 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:48:55.274 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2023-07-06T03:48:55.275 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:48:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2023-07-06T03:48:55.275 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:48:55.275 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:55.276 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:48:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:48:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 41949sec preferred_lft 41949sec 2023-07-06T03:48:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2023-07-06T03:48:55.276 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:55.276 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:48:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:48:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:48:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::58f0:6eff:fe04:3be0/64 scope link 2023-07-06T03:48:55.277 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:55.277 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:48:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:48:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::106d:9cff:fed5:9d0f/64 scope link 2023-07-06T03:48:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:55.278 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:48:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether a2:c5:04:8e:a2:a7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T03:48:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::a0c5:4ff:fe8e:a2a7/64 scope link 2023-07-06T03:48:55.278 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:48:55.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:55.280 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:48:55.350 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:48:55.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:55.354 DEBUG:teuthology.orchestra.run.smithi146:> route 2023-07-06T03:48:55.402 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2023-07-06T03:48:55.402 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:48:55.402 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:48:55.402 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:48:55.402 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:48:55.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:55.403 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:48:55.403 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:48:55.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:48:55.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:48:55.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:48:55.405 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:48:55.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:55.573 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:48:55.620 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T03:48:55.620 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T03:48:55.620 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:48:55.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:55.621 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2023-07-06T03:48:55.691 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-06T03:48:55.692 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:48:55.692 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T03:48:55.738 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:48:55.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:48:55.739 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:48:55.823 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:48:55.933 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:48:55.934 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:48:56.001 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:48:56.003 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:48:56.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:56.005 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:48:56.051 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:48:56.052 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-06T03:48:56.096 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:48:56.167 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:48:56.168 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:48:56.237 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:48:56.239 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:48:56.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:56.241 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:48:56.263 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2023-07-06T03:48:56.261+0000 7f54763755c0 -1 init, newargv = 0x5557e71d70f0 newargc=15 2023-07-06T03:48:56.264 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[122283]: starting ceph client 2023-07-06T03:48:56.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[122283]: starting fuse 2023-07-06T03:48:56.290 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:48:56.291 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T03:48:56.291 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:48:56.292 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2023-07-06T03:48:56.292 DEBUG:teuthology.orchestra.run.smithi146:> import re 2023-07-06T03:48:56.292 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:48:56.293 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2023-07-06T03:48:56.293 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:48:56.293 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2023-07-06T03:48:56.293 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-06T03:48:56.294 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2023-07-06T03:48:56.294 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-06T03:48:56.294 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:48:56.295 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2023-07-06T03:48:56.295 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2023-07-06T03:48:56.295 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2023-07-06T03:48:56.296 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2023-07-06T03:48:56.296 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:48:56.296 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2023-07-06T03:48:56.296 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:48:56.297 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:48:56.297 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:48:56.297 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2023-07-06T03:48:56.298 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2023-07-06T03:48:56.298 DEBUG:teuthology.orchestra.run.smithi146:> return f 2023-07-06T03:48:56.298 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:48:56.299 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:48:56.299 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2023-07-06T03:48:56.299 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:48:56.474 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.122283.asok 2023-07-06T03:48:56.652 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.122283.asok 2023-07-06T03:48:56.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:56.653 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.122283.asok status 2023-07-06T03:48:56.877 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-07-06T03:48:56.877 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2023-07-06T03:48:56.877 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:48:56.877 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:48:56.877 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2023-07-06T03:48:56.878 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2023-07-06T03:48:56.878 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-06T03:48:56.878 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "122283", 2023-07-06T03:48:56.878 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2023-07-06T03:48:56.878 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:48:56.878 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2023-07-06T03:48:56.878 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2023-07-06T03:48:56.879 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 6142, 2023-07-06T03:48:56.879 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2023-07-06T03:48:56.879 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2023-07-06T03:48:56.879 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2023-07-06T03:48:56.879 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 6142 2023-07-06T03:48:56.879 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:48:56.879 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:48:56.879 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:48:56.880 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:56.880 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 488095976 2023-07-06T03:48:56.880 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-07-06T03:48:56.880 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:48:56.880 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:48:56.880 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:48:56.880 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:48:56.880 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 488095976 2023-07-06T03:48:56.881 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:48:56.881 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.6142 192.168.0.1:0/488095976", 2023-07-06T03:48:56.881 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/488095976", 2023-07-06T03:48:56.881 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2023-07-06T03:48:56.881 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 34, 2023-07-06T03:48:56.881 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 62, 2023-07-06T03:48:56.881 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:48:56.882 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2023-07-06T03:48:56.882 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2023-07-06T03:48:56.882 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-07-06T03:48:56.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:56.891 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2023-07-06T03:48:56.892 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-06T03:48:56.938 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2023-07-06T03:48:56.938 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-06T03:48:56.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:48:56.939 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:48:57.008 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-07-06T03:48:57.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.213+0000 7f71827ff700 1 Processor -- start 2023-07-06T03:48:57.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.213+0000 7f71827ff700 1 -- start start 2023-07-06T03:48:57.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.214+0000 7f71827ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c06b360 0x7f717c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.214+0000 7f71827ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f717c0d2210 0x7f717c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.214+0000 7f71827ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717c0cff90 con 0x7f717c15c610 2023-07-06T03:48:57.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.214+0000 7f71827ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717c0d0780 con 0x7f717c0d2210 2023-07-06T03:48:57.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.214+0000 7f71827ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c15c9e0 con 0x7f717c06b360 2023-07-06T03:48:57.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f7180d9c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f717c0d2210 0x7f717c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f717b7fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c06b360 0x7f717c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f717bfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f717c15c610 0x7f717c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:53750/0 (socket says 172.21.15.97:53750) 2023-07-06T03:48:57.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f717bfff700 1 -- 172.21.15.97:0/2500666350 learned_addr learned my addr 172.21.15.97:0/2500666350 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:57.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f717affd700 1 -- 172.21.15.97:0/2500666350 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432140792 0 0) 0x7f717c0cff90 con 0x7f717c15c610 2023-07-06T03:48:57.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f717affd700 1 -- 172.21.15.97:0/2500666350 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71640035b0 con 0x7f717c15c610 2023-07-06T03:48:57.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f717b7fe700 1 -- 172.21.15.97:0/2500666350 >> v2:172.21.15.146:3300/0 conn(0x7f717c0d2210 msgr2=0x7f717c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:57.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f717b7fe700 1 --2- 172.21.15.97:0/2500666350 >> v2:172.21.15.146:3300/0 conn(0x7f717c0d2210 0x7f717c0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f717b7fe700 1 -- 172.21.15.97:0/2500666350 >> v1:172.21.15.146:6789/0 conn(0x7f717c15c610 legacy=0x7f717c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f717b7fe700 1 -- 172.21.15.97:0/2500666350 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0b5650 con 0x7f717c06b360 2023-07-06T03:48:57.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f7180d9c700 1 --2- 172.21.15.97:0/2500666350 >> v2:172.21.15.146:3300/0 conn(0x7f717c0d2210 0x7f717c0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:48:57.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.215+0000 7f717b7fe700 1 --2- 172.21.15.97:0/2500666350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c06b360 0x7f717c0b4ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7170009840 tx=0x7f7170009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=542a12f05bdbe7f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:57.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.216+0000 7f717affd700 1 -- 172.21.15.97:0/2500666350 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170011830 con 0x7f717c06b360 2023-07-06T03:48:57.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.216+0000 7f717affd700 1 -- 172.21.15.97:0/2500666350 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7170011990 con 0x7f717c06b360 2023-07-06T03:48:57.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.216+0000 7f717affd700 1 -- 172.21.15.97:0/2500666350 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170011d20 con 0x7f717c06b360 2023-07-06T03:48:57.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.216+0000 7f71827ff700 1 -- 172.21.15.97:0/2500666350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c06b360 msgr2=0x7f717c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.216+0000 7f71827ff700 1 --2- 172.21.15.97:0/2500666350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c06b360 0x7f717c0b4ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7170009840 tx=0x7f7170009b50 comp rx=0 tx=0).stop 2023-07-06T03:48:57.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.216+0000 7f71827ff700 1 -- 172.21.15.97:0/2500666350 shutdown_connections 2023-07-06T03:48:57.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.216+0000 7f71827ff700 1 --2- 172.21.15.97:0/2500666350 >> v2:172.21.15.146:3300/0 conn(0x7f717c0d2210 0x7f717c0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.216+0000 7f71827ff700 1 --2- 172.21.15.97:0/2500666350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c06b360 0x7f717c0b4ec0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.216+0000 7f71827ff700 1 -- 172.21.15.97:0/2500666350 >> 172.21.15.97:0/2500666350 conn(0x7f717c06a6c0 msgr2=0x7f717c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:57.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.216+0000 7f71827ff700 1 -- 172.21.15.97:0/2500666350 shutdown_connections 2023-07-06T03:48:57.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.216+0000 7f71827ff700 1 -- 172.21.15.97:0/2500666350 wait complete. 2023-07-06T03:48:57.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.218+0000 7f71827ff700 1 Processor -- start 2023-07-06T03:48:57.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.218+0000 7f71827ff700 1 -- start start 2023-07-06T03:48:57.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.219+0000 7f71827ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c06b360 0x7f717c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.219+0000 7f71827ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f717c0d2210 0x7f717c14b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f71827ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c15c610 0x7f717c15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f71827ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0ca350 con 0x7f717c15c610 2023-07-06T03:48:57.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f71827ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0ca630 con 0x7f717c0d2210 2023-07-06T03:48:57.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f71827ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0ca910 con 0x7f717c06b360 2023-07-06T03:48:57.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f7180d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c15c610 0x7f717c15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f7180d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c15c610 0x7f717c15efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49658/0 (socket says 172.21.15.97:49658) 2023-07-06T03:48:57.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f7180d9c700 1 -- 172.21.15.97:0/3350924465 learned_addr learned my addr 172.21.15.97:0/3350924465 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:57.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f717b7fe700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f717c0d2210 0x7f717c14b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f717bfff700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c06b360 0x7f717c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f7180d9c700 1 -- 172.21.15.97:0/3350924465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c06b360 msgr2=0x7f717c158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f7180d9c700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c06b360 0x7f717c158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f7180d9c700 1 -- 172.21.15.97:0/3350924465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f717c0d2210 msgr2=0x7f717c14b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f7180d9c700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f717c0d2210 0x7f717c14b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f7180d9c700 1 -- 172.21.15.97:0/3350924465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7174009a70 con 0x7f717c15c610 2023-07-06T03:48:57.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.220+0000 7f7180d9c700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c15c610 0x7f717c15efd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f717400efc0 tx=0x7f717400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:57.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.221+0000 7f7178ff9700 1 -- 172.21.15.97:0/3350924465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717400ca90 con 0x7f717c15c610 2023-07-06T03:48:57.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.221+0000 7f7178ff9700 1 -- 172.21.15.97:0/3350924465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717400f040 con 0x7f717c15c610 2023-07-06T03:48:57.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.221+0000 7f7178ff9700 1 -- 172.21.15.97:0/3350924465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717400ce10 con 0x7f717c15c610 2023-07-06T03:48:57.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.221+0000 7f71827ff700 1 -- 172.21.15.97:0/3350924465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71700094a0 con 0x7f717c15c610 2023-07-06T03:48:57.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.221+0000 7f71827ff700 1 -- 172.21.15.97:0/3350924465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717c0caed0 con 0x7f717c15c610 2023-07-06T03:48:57.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.222+0000 7f71827ff700 1 -- 172.21.15.97:0/3350924465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717c0b19c0 con 0x7f717c15c610 2023-07-06T03:48:57.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.223+0000 7f7178ff9700 1 -- 172.21.15.97:0/3350924465 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7174014480 con 0x7f717c15c610 2023-07-06T03:48:57.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.224+0000 7f7178ff9700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716806ace0 0x7f716806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.224+0000 7f7178ff9700 1 -- 172.21.15.97:0/3350924465 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f7174089c30 con 0x7f717c15c610 2023-07-06T03:48:57.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.224+0000 7f717bfff700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716806ace0 0x7f716806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.224+0000 7f717bfff700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716806ace0 0x7f716806d190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f716c007fd0 tx=0x7f716c007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:57.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.228+0000 7f7178ff9700 1 -- 172.21.15.97:0/3350924465 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7174058600 con 0x7f717c15c610 2023-07-06T03:48:57.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.426+0000 7f71827ff700 1 -- 172.21.15.97:0/3350924465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7f717c0cb0c0 con 0x7f717c15c610 2023-07-06T03:48:57.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.426+0000 7f7178ff9700 1 -- 172.21.15.97:0/3350924465 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v18) v1 ==== 106+0+4 (secure 0 0 0) 0x7f7174009d70 con 0x7f717c15c610 2023-07-06T03:48:57.429 INFO:teuthology.orchestra.run.smithi097.stdout:100 2023-07-06T03:48:57.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.428+0000 7f71827ff700 1 -- 172.21.15.97:0/3350924465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716806ace0 msgr2=0x7f716806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.428+0000 7f71827ff700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716806ace0 0x7f716806d190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f716c007fd0 tx=0x7f716c007dc0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.428+0000 7f71827ff700 1 -- 172.21.15.97:0/3350924465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c15c610 msgr2=0x7f717c15efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.428+0000 7f71827ff700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c15c610 0x7f717c15efd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f717400efc0 tx=0x7f717400c3a0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.428+0000 7f71827ff700 1 -- 172.21.15.97:0/3350924465 shutdown_connections 2023-07-06T03:48:57.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.428+0000 7f71827ff700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f716806ace0 0x7f716806d190 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.428+0000 7f71827ff700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f717c15c610 0x7f717c15efd0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.428+0000 7f71827ff700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f717c06b360 0x7f717c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.428+0000 7f71827ff700 1 --2- 172.21.15.97:0/3350924465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f717c0d2210 0x7f717c14b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.428+0000 7f71827ff700 1 -- 172.21.15.97:0/3350924465 >> 172.21.15.97:0/3350924465 conn(0x7f717c06a6c0 msgr2=0x7f717c159250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:57.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.429+0000 7f71827ff700 1 -- 172.21.15.97:0/3350924465 shutdown_connections 2023-07-06T03:48:57.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.429+0000 7f71827ff700 1 -- 172.21.15.97:0/3350924465 wait complete. 2023-07-06T03:48:57.449 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 200 2023-07-06T03:48:57.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.642+0000 7fb87e1ff700 1 Processor -- start 2023-07-06T03:48:57.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.642+0000 7fb87e1ff700 1 -- start start 2023-07-06T03:48:57.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.643+0000 7fb87e1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb878154890 0x7fb8780cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.643+0000 7fb87e1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb87815eb90 0x7fb8780cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.643+0000 7fb87e1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87815b450 con 0x7fb87815a940 2023-07-06T03:48:57.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.643+0000 7fb87e1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87815b6e0 con 0x7fb878154890 2023-07-06T03:48:57.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.643+0000 7fb87e1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87815e0b0 con 0x7fb87815eb90 2023-07-06T03:48:57.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.644+0000 7fb877fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb87815a940 0x7fb87815ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59148/0 (socket says 172.21.15.97:59148) 2023-07-06T03:48:57.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.644+0000 7fb877fff700 1 -- 172.21.15.97:0/308299108 learned_addr learned my addr 172.21.15.97:0/308299108 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:48:57.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.644+0000 7fb8777fe700 1 --2- 172.21.15.97:0/308299108 >> v2:172.21.15.97:3300/0 conn(0x7fb878154890 0x7fb8780cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.644+0000 7fb876ffd700 1 --2- 172.21.15.97:0/308299108 >> v2:172.21.15.146:3300/0 conn(0x7fb87815eb90 0x7fb8780cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.644+0000 7fb8767fc700 1 -- 172.21.15.97:0/308299108 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209220351 0 0) 0x7fb87815b450 con 0x7fb87815a940 2023-07-06T03:48:57.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.644+0000 7fb8767fc700 1 -- 172.21.15.97:0/308299108 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8600035b0 con 0x7fb87815a940 2023-07-06T03:48:57.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.644+0000 7fb8777fe700 1 -- 172.21.15.97:0/308299108 >> v2:172.21.15.146:3300/0 conn(0x7fb87815eb90 msgr2=0x7fb8780cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:48:57.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.644+0000 7fb8777fe700 1 --2- 172.21.15.97:0/308299108 >> v2:172.21.15.146:3300/0 conn(0x7fb87815eb90 0x7fb8780cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.644+0000 7fb8777fe700 1 -- 172.21.15.97:0/308299108 >> v1:172.21.15.97:6789/0 conn(0x7fb87815a940 legacy=0x7fb87815ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.644+0000 7fb8777fe700 1 -- 172.21.15.97:0/308299108 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87815e390 con 0x7fb878154890 2023-07-06T03:48:57.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.644+0000 7fb8777fe700 1 --2- 172.21.15.97:0/308299108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb878154890 0x7fb8780cb390 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb8780cf9f0 tx=0x7fb868009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef722864a1e332a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:57.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.645+0000 7fb8767fc700 1 -- 172.21.15.97:0/308299108 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868011830 con 0x7fb878154890 2023-07-06T03:48:57.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.645+0000 7fb8767fc700 1 -- 172.21.15.97:0/308299108 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb868011990 con 0x7fb878154890 2023-07-06T03:48:57.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.645+0000 7fb8767fc700 1 -- 172.21.15.97:0/308299108 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868011d20 con 0x7fb878154890 2023-07-06T03:48:57.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.645+0000 7fb87e1ff700 1 -- 172.21.15.97:0/308299108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb878154890 msgr2=0x7fb8780cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.645+0000 7fb87e1ff700 1 --2- 172.21.15.97:0/308299108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb878154890 0x7fb8780cb390 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb8780cf9f0 tx=0x7fb868009b20 comp rx=0 tx=0).stop 2023-07-06T03:48:57.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.645+0000 7fb87e1ff700 1 -- 172.21.15.97:0/308299108 shutdown_connections 2023-07-06T03:48:57.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.645+0000 7fb87e1ff700 1 --2- 172.21.15.97:0/308299108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb878154890 0x7fb8780cb390 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.645+0000 7fb87e1ff700 1 --2- 172.21.15.97:0/308299108 >> v2:172.21.15.146:3300/0 conn(0x7fb87815eb90 0x7fb8780cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.645+0000 7fb87e1ff700 1 -- 172.21.15.97:0/308299108 >> 172.21.15.97:0/308299108 conn(0x7fb87806a060 msgr2=0x7fb87806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:57.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.645+0000 7fb87e1ff700 1 -- 172.21.15.97:0/308299108 shutdown_connections 2023-07-06T03:48:57.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.645+0000 7fb87e1ff700 1 -- 172.21.15.97:0/308299108 wait complete. 2023-07-06T03:48:57.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.647+0000 7fb87e1ff700 1 Processor -- start 2023-07-06T03:48:57.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.647+0000 7fb87e1ff700 1 -- start start 2023-07-06T03:48:57.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.648+0000 7fb87e1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb878154890 0x7fb8781f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.648+0000 7fb87e1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87815a940 0x7fb8780ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.648+0000 7fb87e1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87815eb90 0x7fb8780acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.648+0000 7fb87e1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780cc8f0 con 0x7fb87815a940 2023-07-06T03:48:57.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.648+0000 7fb87e1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781f2720 con 0x7fb878154890 2023-07-06T03:48:57.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.648+0000 7fb87e1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781f2a00 con 0x7fb87815eb90 2023-07-06T03:48:57.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.648+0000 7fb876ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87815a940 0x7fb8780ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.648+0000 7fb876ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87815a940 0x7fb8780ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49674/0 (socket says 172.21.15.97:49674) 2023-07-06T03:48:57.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.648+0000 7fb876ffd700 1 -- 172.21.15.97:0/2168769858 learned_addr learned my addr 172.21.15.97:0/2168769858 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:48:57.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb8777fe700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb878154890 0x7fb8781f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb877fff700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87815eb90 0x7fb8780acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb876ffd700 1 -- 172.21.15.97:0/2168769858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87815eb90 msgr2=0x7fb8780acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb876ffd700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87815eb90 0x7fb8780acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb876ffd700 1 -- 172.21.15.97:0/2168769858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb878154890 msgr2=0x7fb8781f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb876ffd700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb878154890 0x7fb8781f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb876ffd700 1 -- 172.21.15.97:0/2168769858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8680094a0 con 0x7fb87815a940 2023-07-06T03:48:57.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb877fff700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87815eb90 0x7fb8780acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:48:57.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb876ffd700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87815a940 0x7fb8780ac870 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00e870 tx=0x7fb86c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:57.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb85ffff700 1 -- 172.21.15.97:0/2168769858 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c009590 con 0x7fb87815a940 2023-07-06T03:48:57.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb85ffff700 1 -- 172.21.15.97:0/2168769858 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c00ee70 con 0x7fb87815a940 2023-07-06T03:48:57.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb87e1ff700 1 -- 172.21.15.97:0/2168769858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8781f2ce0 con 0x7fb87815a940 2023-07-06T03:48:57.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb85ffff700 1 -- 172.21.15.97:0/2168769858 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00ca50 con 0x7fb87815a940 2023-07-06T03:48:57.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.649+0000 7fb87e1ff700 1 -- 172.21.15.97:0/2168769858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8781ef930 con 0x7fb87815a940 2023-07-06T03:48:57.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.650+0000 7fb87e1ff700 1 -- 172.21.15.97:0/2168769858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8780cca60 con 0x7fb87815a940 2023-07-06T03:48:57.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.651+0000 7fb85ffff700 1 -- 172.21.15.97:0/2168769858 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb86c00cbb0 con 0x7fb87815a940 2023-07-06T03:48:57.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.652+0000 7fb85ffff700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86406ace0 0x7fb86406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:48:57.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.652+0000 7fb85ffff700 1 -- 172.21.15.97:0/2168769858 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fb86c089530 con 0x7fb87815a940 2023-07-06T03:48:57.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.653+0000 7fb8777fe700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86406ace0 0x7fb86406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:48:57.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.653+0000 7fb8777fe700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86406ace0 0x7fb86406d190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb8680090b0 tx=0x7fb868008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:48:57.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.657+0000 7fb85ffff700 1 -- 172.21.15.97:0/2168769858 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb86c057f90 con 0x7fb87815a940 2023-07-06T03:48:57.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.853+0000 7fb87e1ff700 1 -- 172.21.15.97:0/2168769858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) v1 -- 0x7fb87804e340 con 0x7fb87815a940 2023-07-06T03:48:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.857+0000 7fb85ffff700 1 -- 172.21.15.97:0/2168769858 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v19)=0 v19) v1 ==== 123+0+0 (secure 0 0 0) 0x7fb86c00ce60 con 0x7fb87815a940 2023-07-06T03:48:57.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 -- 172.21.15.97:0/2168769858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86406ace0 msgr2=0x7fb86406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86406ace0 0x7fb86406d190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb8680090b0 tx=0x7fb868008030 comp rx=0 tx=0).stop 2023-07-06T03:48:57.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 -- 172.21.15.97:0/2168769858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87815a940 msgr2=0x7fb8780ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:48:57.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87815a940 0x7fb8780ac870 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00e870 tx=0x7fb86c00ec30 comp rx=0 tx=0).stop 2023-07-06T03:48:57.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 -- 172.21.15.97:0/2168769858 shutdown_connections 2023-07-06T03:48:57.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb86406ace0 0x7fb86406d190 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb87815eb90 0x7fb8780acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb878154890 0x7fb8781f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 --2- 172.21.15.97:0/2168769858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb87815a940 0x7fb8780ac870 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:48:57.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 -- 172.21.15.97:0/2168769858 >> 172.21.15.97:0/2168769858 conn(0x7fb87806a060 msgr2=0x7fb87806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:48:57.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 -- 172.21.15.97:0/2168769858 shutdown_connections 2023-07-06T03:48:57.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:48:57.860+0000 7fb87e1ff700 1 -- 172.21.15.97:0/2168769858 wait complete. 2023-07-06T03:48:57.880 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:48:57.881 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:48:57.926 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:48:57.927 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:48:57.927 DEBUG:teuthology.orchestra.run.smithi097:> import uuid 2023-07-06T03:48:57.927 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:57.928 DEBUG:teuthology.orchestra.run.smithi097:> n = 600 2023-07-06T03:48:57.928 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = False 2023-07-06T03:48:57.928 DEBUG:teuthology.orchestra.run.smithi097:> if 0 > 0: 2023-07-06T03:48:57.928 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = True 2023-07-06T03:48:57.929 DEBUG:teuthology.orchestra.run.smithi097:> path = "/home/ubuntu/cephtest/mnt.0/foo/" 2023-07-06T03:48:57.929 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:57.929 DEBUG:teuthology.orchestra.run.smithi097:> dpath = os.path.dirname(path) 2023-07-06T03:48:57.930 DEBUG:teuthology.orchestra.run.smithi097:> fnameprefix = os.path.basename(path) 2023-07-06T03:48:57.930 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(dpath, exist_ok=True) 2023-07-06T03:48:57.930 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:57.930 DEBUG:teuthology.orchestra.run.smithi097:> try: 2023-07-06T03:48:57.931 DEBUG:teuthology.orchestra.run.smithi097:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-06T03:48:57.931 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:48:57.932 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(n): 2023-07-06T03:48:57.932 DEBUG:teuthology.orchestra.run.smithi097:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-06T03:48:57.932 DEBUG:teuthology.orchestra.run.smithi097:> with open(fpath, '"'"'w'"'"') as f: 2023-07-06T03:48:57.932 DEBUG:teuthology.orchestra.run.smithi097:> f.write(f"{i}") 2023-07-06T03:48:57.933 DEBUG:teuthology.orchestra.run.smithi097:> if True: 2023-07-06T03:48:57.933 DEBUG:teuthology.orchestra.run.smithi097:> f.flush() 2023-07-06T03:48:57.933 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(f.fileno()) 2023-07-06T03:48:57.934 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:48:57.934 DEBUG:teuthology.orchestra.run.smithi097:> os.unlink(fpath) 2023-07-06T03:48:57.934 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:48:57.935 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:48:57.935 DEBUG:teuthology.orchestra.run.smithi097:> if create_hard_links: 2023-07-06T03:48:57.935 DEBUG:teuthology.orchestra.run.smithi097:> for j in range(0): 2023-07-06T03:48:57.935 DEBUG:teuthology.orchestra.run.smithi097:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-06T03:48:57.936 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:48:57.936 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:48:57.936 DEBUG:teuthology.orchestra.run.smithi097:> finally: 2023-07-06T03:48:57.937 DEBUG:teuthology.orchestra.run.smithi097:> os.close(dirfd) 2023-07-06T03:48:57.937 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:49:01.866 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:49:01.867 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:49:01.868 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:49:01.868 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:49:01.868 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:49:01.869 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:01.869 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:49:01.869 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:49:01.870 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:49:01.870 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:49:01.870 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:01.870 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:49:01.871 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:49:01.871 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:49:01.871 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:49:01.871 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:01.872 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:49:01.872 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:49:01.872 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:49:01.873 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:49:01.873 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:49:01.873 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:49:01.873 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:49:01.874 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:49:01.874 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:01.874 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:49:01.874 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:49:02.051 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.139442.asok 2023-07-06T03:49:02.229 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.139442.asok 2023-07-06T03:49:02.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:02.230 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.139442.asok mds_sessions 2023-07-06T03:49:02.451 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:49:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6564, 2023-07-06T03:49:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:49:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:49:02.451 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:49:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6564 2023-07-06T03:49:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:49:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:49:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:49:02.452 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 837549164 2023-07-06T03:49:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6564 v1:192.168.0.1:0/837549164", 2023-07-06T03:49:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "v1:192.168.0.1:0/837549164", 2023-07-06T03:49:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions": [ 2023-07-06T03:49:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:02.453 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": 0, 2023-07-06T03:49:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "addrs": { 2023-07-06T03:49:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-07-06T03:49:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-07-06T03:49:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.146:6832", 2023-07-06T03:49:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2151424592 2023-07-06T03:49:02.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:49:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.146:6834", 2023-07-06T03:49:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2151424592 2023-07-06T03:49:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:49:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:02.455 INFO:teuthology.orchestra.run.smithi097.stdout: "seq": 1201, 2023-07-06T03:49:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_gen": 0, 2023-07-06T03:49:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_ttl": "2023-07-06T03:49:54.965172+0000", 2023-07-06T03:49:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "last_cap_renew_request": "2023-07-06T03:48:54.965172+0000", 2023-07-06T03:49:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_renew_seq": 2, 2023-07-06T03:49:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "num_caps": 100, 2023-07-06T03:49:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "open" 2023-07-06T03:49:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:02.456 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-07-06T03:49:02.457 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 34 2023-07-06T03:49:02.457 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:49:02.465 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:49:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.655+0000 7f21fe5ff700 1 Processor -- start 2023-07-06T03:49:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.655+0000 7f21fe5ff700 1 -- start start 2023-07-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.656+0000 7f21fe5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21f815c520 0x7f21f80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.656+0000 7f21fe5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f21f8160d00 0x7f21f80b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.656+0000 7f21fe5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f80cdae0 con 0x7f21f8160180 2023-07-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.656+0000 7f21fe5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f80cd8d0 con 0x7f21f815c520 2023-07-06T03:49:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.656+0000 7f21fe5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f80b6640 con 0x7f21f8160d00 2023-07-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.657+0000 7f21f7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21f815c520 0x7f21f80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.657+0000 7f21f7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21f815c520 0x7f21f80cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52400/0 (socket says 172.21.15.97:52400) 2023-07-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.657+0000 7f21f7fff700 1 -- 172.21.15.97:0/4252041397 learned_addr learned my addr 172.21.15.97:0/4252041397 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.657+0000 7f21fcb9c700 1 --2- 172.21.15.97:0/4252041397 >> v2:172.21.15.146:3300/0 conn(0x7f21f8160d00 0x7f21f80b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.657+0000 7f21f7fff700 1 -- 172.21.15.97:0/4252041397 >> v2:172.21.15.146:3300/0 conn(0x7f21f8160d00 msgr2=0x7f21f80b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.657+0000 7f21f7fff700 1 --2- 172.21.15.97:0/4252041397 >> v2:172.21.15.146:3300/0 conn(0x7f21f8160d00 0x7f21f80b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.657+0000 7f21f7fff700 1 -- 172.21.15.97:0/4252041397 >> v1:172.21.15.146:6789/0 conn(0x7f21f8160180 legacy=0x7f21f81605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.657+0000 7f21f7fff700 1 -- 172.21.15.97:0/4252041397 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f80b6920 con 0x7f21f815c520 2023-07-06T03:49:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.657+0000 7f21f7fff700 1 --2- 172.21.15.97:0/4252041397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f815c520 0x7f21f80cd390 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f21f80cd320 tx=0x7f21e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4a8bab69088317 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.658+0000 7f21f6ffd700 1 -- 172.21.15.97:0/4252041397 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8004230 con 0x7f21f815c520 2023-07-06T03:49:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.658+0000 7f21f6ffd700 1 -- 172.21.15.97:0/4252041397 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e8004390 con 0x7f21f815c520 2023-07-06T03:49:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.658+0000 7f21f6ffd700 1 -- 172.21.15.97:0/4252041397 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e80036a0 con 0x7f21f815c520 2023-07-06T03:49:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.658+0000 7f21fe5ff700 1 -- 172.21.15.97:0/4252041397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f815c520 msgr2=0x7f21f80cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.658+0000 7f21fe5ff700 1 --2- 172.21.15.97:0/4252041397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f815c520 0x7f21f80cd390 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f21f80cd320 tx=0x7f21e8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:49:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.658+0000 7f21fe5ff700 1 -- 172.21.15.97:0/4252041397 shutdown_connections 2023-07-06T03:49:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.658+0000 7f21fe5ff700 1 --2- 172.21.15.97:0/4252041397 >> v2:172.21.15.146:3300/0 conn(0x7f21f8160d00 0x7f21f80b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.658+0000 7f21fe5ff700 1 --2- 172.21.15.97:0/4252041397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f815c520 0x7f21f80cd390 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.658+0000 7f21fe5ff700 1 -- 172.21.15.97:0/4252041397 >> 172.21.15.97:0/4252041397 conn(0x7f21f8071790 msgr2=0x7f21f8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.658+0000 7f21fe5ff700 1 -- 172.21.15.97:0/4252041397 shutdown_connections 2023-07-06T03:49:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.658+0000 7f21fe5ff700 1 -- 172.21.15.97:0/4252041397 wait complete. 2023-07-06T03:49:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.660+0000 7f21fe5ff700 1 Processor -- start 2023-07-06T03:49:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.660+0000 7f21fe5ff700 1 -- start start 2023-07-06T03:49:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.661+0000 7f21fe5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21f815c520 0x7f21f80d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.661+0000 7f21fe5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21f8160180 0x7f21f81ef450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.661+0000 7f21fe5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f8160d00 0x7f21f81f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.661+0000 7f21fe5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80cb9c0 con 0x7f21f8160d00 2023-07-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.661+0000 7f21fe5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80cb0d0 con 0x7f21f815c520 2023-07-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.661+0000 7f21fe5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f8155810 con 0x7f21f8160180 2023-07-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.661+0000 7f21fcb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f8160d00 0x7f21f81f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.661+0000 7f21fcb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f8160d00 0x7f21f81f5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52412/0 (socket says 172.21.15.97:52412) 2023-07-06T03:49:02.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.661+0000 7f21fcb9c700 1 -- 172.21.15.97:0/3677565790 learned_addr learned my addr 172.21.15.97:0/3677565790 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.661+0000 7f21f7fff700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21f815c520 0x7f21f80d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.661+0000 7f21f77fe700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21f8160180 0x7f21f81ef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.662+0000 7f21fcb9c700 1 -- 172.21.15.97:0/3677565790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21f8160180 msgr2=0x7f21f81ef450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.662+0000 7f21fcb9c700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21f8160180 0x7f21f81ef450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.662+0000 7f21fcb9c700 1 -- 172.21.15.97:0/3677565790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21f815c520 msgr2=0x7f21f80d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.662+0000 7f21fcb9c700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21f815c520 0x7f21f80d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:02.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.662+0000 7f21fcb9c700 1 -- 172.21.15.97:0/3677565790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8009550 con 0x7f21f8160d00 2023-07-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.662+0000 7f21fcb9c700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f8160d00 0x7f21f81f5630 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f21f0007e40 tx=0x7f21f000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.662+0000 7f21f4ff9700 1 -- 172.21.15.97:0/3677565790 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f000ed30 con 0x7f21f8160d00 2023-07-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.662+0000 7f21f4ff9700 1 -- 172.21.15.97:0/3677565790 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f0014420 con 0x7f21f8160d00 2023-07-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.662+0000 7f21f4ff9700 1 -- 172.21.15.97:0/3677565790 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f00147d0 con 0x7f21f8160d00 2023-07-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.662+0000 7f21fe5ff700 1 -- 172.21.15.97:0/3677565790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f80cb370 con 0x7f21f8160d00 2023-07-06T03:49:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.662+0000 7f21fe5ff700 1 -- 172.21.15.97:0/3677565790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f80d2080 con 0x7f21f8160d00 2023-07-06T03:49:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.663+0000 7f21fe5ff700 1 -- 172.21.15.97:0/3677565790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f80cbb70 con 0x7f21f8160d00 2023-07-06T03:49:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.664+0000 7f21f4ff9700 1 -- 172.21.15.97:0/3677565790 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21f0014930 con 0x7f21f8160d00 2023-07-06T03:49:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.665+0000 7f21f4ff9700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f21e406ace0 0x7f21e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.665+0000 7f21f4ff9700 1 -- 172.21.15.97:0/3677565790 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f21f0089bf0 con 0x7f21f8160d00 2023-07-06T03:49:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.665+0000 7f21f7fff700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f21e406ace0 0x7f21e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.666+0000 7f21f7fff700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f21e406ace0 0x7f21e406d190 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009130 tx=0x7f21e8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.670+0000 7f21f4ff9700 1 -- 172.21.15.97:0/3677565790 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f21f80cbb70 con 0x7f21f8160d00 2023-07-06T03:49:02.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.908+0000 7f21fe5ff700 1 -- 172.21.15.97:0/3677565790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21f804e340 con 0x7f21f8160d00 2023-07-06T03:49:02.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.908+0000 7f21f4ff9700 1 -- 172.21.15.97:0/3677565790 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4678 (secure 0 0 0) 0x7f21f0058660 con 0x7f21f8160d00 2023-07-06T03:49:02.911 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:02.911 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1108873506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1108873506},{"type":"v1","addr":"172.21.15.146:6835","nonce":1108873506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2471899168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2471899168},{"type":"v1","addr":"172.21.15.97:6837","nonce":2471899168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2970792733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2970792733},{"type":"v1","addr":"172.21.15.97:6836","nonce":2970792733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:48:45.401033+0000","modified":"2023-07-06T03:48:48.930905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6399":{"gid":6399,"name":"d","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/2151424592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2151424592},{"type":"v1","addr":"172.21.15.146:6834","nonce":2151424592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-06T03:49:02.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.910+0000 7f21fe5ff700 1 -- 172.21.15.97:0/3677565790 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f21e406ace0 msgr2=0x7f21e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:02.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.910+0000 7f21fe5ff700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f21e406ace0 0x7f21e406d190 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009130 tx=0x7f21e8008030 comp rx=0 tx=0).stop 2023-07-06T03:49:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.910+0000 7f21fe5ff700 1 -- 172.21.15.97:0/3677565790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f8160d00 msgr2=0x7f21f81f5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.910+0000 7f21fe5ff700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f8160d00 0x7f21f81f5630 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f21f0007e40 tx=0x7f21f000e3c0 comp rx=0 tx=0).stop 2023-07-06T03:49:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.910+0000 7f21fe5ff700 1 -- 172.21.15.97:0/3677565790 shutdown_connections 2023-07-06T03:49:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.910+0000 7f21fe5ff700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f21e406ace0 0x7f21e406d190 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.910+0000 7f21fe5ff700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f8160d00 0x7f21f81f5630 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.910+0000 7f21fe5ff700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21f815c520 0x7f21f80d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:02.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.910+0000 7f21fe5ff700 1 --2- 172.21.15.97:0/3677565790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21f8160180 0x7f21f81ef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:02.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.910+0000 7f21fe5ff700 1 -- 172.21.15.97:0/3677565790 >> 172.21.15.97:0/3677565790 conn(0x7f21f8071790 msgr2=0x7f21f815eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:02.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.911+0000 7f21fe5ff700 1 -- 172.21.15.97:0/3677565790 shutdown_connections 2023-07-06T03:49:02.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:02.911+0000 7f21fe5ff700 1 -- 172.21.15.97:0/3677565790 wait complete. 2023-07-06T03:49:02.915 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 34 2023-07-06T03:49:02.931 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-07-06T03:49:03.157 INFO:teuthology.orchestra.run.smithi146.stdout:[{"id":6142,"entity":{"name":{"type":"client","num":6142},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":488095976}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":6.8748970299999996,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9610890390310116,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6142 192.168.0.1:0/488095976","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"122283","root":"/"}},{"id":6564,"entity":{"name":{"type":"client","num":6564},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":837549164}},"state":"open","num_leases":1,"num_caps":100,"request_load_avg":1100,"uptime":9.1758625669999994,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":487.48348366329367,"halflife":60},"release_caps":{"value":493.9630230394306,"halflife":60},"recall_caps_throttle":{"value":444.22872501421068,"halflife":1.5},"recall_caps_throttle2o":{"value":150.41224371564223,"halflife":0.5},"session_cache_liveness":{"value":1196.778028266383,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1136,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6564 192.168.0.1:0/837549164","completed_requests":[{"tid":1137,"created_ino":"0x1000000044c"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"139442","root":"/"}}] 2023-07-06T03:49:03.158 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "122283", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 488095976, "type": "any" }, "name": { "num": 6142, "type": "client" } }, "id": 6142, "inst": "client.6142 192.168.0.1:0/488095976", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9610890390310116 }, "state": "open", "uptime": 6.87489703 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "139442", "root": "/" }, "completed_requests": [ { "created_ino": "0x1000000044c", "tid": 1137 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 837549164, "type": "any" }, "name": { "num": 6564, "type": "client" } }, "id": 6564, "inst": "client.6564 192.168.0.1:0/837549164", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 1136, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 487.48348366329367 }, "recall_caps_throttle": { "halflife": 1.5, "value": 444.2287250142107 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 150.41224371564223 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 493.9630230394306 }, "request_load_avg": 1100, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1196.778028266383 }, "state": "open", "uptime": 9.175862567 } ] 2023-07-06T03:49:03.158 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-06T03:49:03.159 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:49:03.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:49:03.258 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:49:03.358 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:49:03.359 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:49:03.359 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:49:03.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:49:03.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.637+0000 7f8127bff700 1 Processor -- start 2023-07-06T03:49:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.638+0000 7f8127bff700 1 -- start start 2023-07-06T03:49:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f8127bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f81200cd390 0x7f8120160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f8127bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8120160dd0 0x7f81200b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f8127bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81200cdae0 con 0x7f812015c520 2023-07-06T03:49:03.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f8127bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81200b6640 con 0x7f8120160dd0 2023-07-06T03:49:03.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f8127bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81200b6920 con 0x7f81200cd390 2023-07-06T03:49:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f812619c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8120160dd0 0x7f81200b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f812619c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8120160dd0 0x7f81200b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52416/0 (socket says 172.21.15.97:52416) 2023-07-06T03:49:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f812619c700 1 -- 172.21.15.97:0/4187616354 learned_addr learned my addr 172.21.15.97:0/4187616354 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f812519a700 1 --2- 172.21.15.97:0/4187616354 >> v2:172.21.15.146:3300/0 conn(0x7f81200cd390 0x7f8120160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f812619c700 1 -- 172.21.15.97:0/4187616354 >> v2:172.21.15.146:3300/0 conn(0x7f81200cd390 msgr2=0x7f8120160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f812619c700 1 --2- 172.21.15.97:0/4187616354 >> v2:172.21.15.146:3300/0 conn(0x7f81200cd390 0x7f8120160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.639+0000 7f812619c700 1 -- 172.21.15.97:0/4187616354 >> v1:172.21.15.146:6789/0 conn(0x7f812015c520 legacy=0x7f8120160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.640+0000 7f812619c700 1 -- 172.21.15.97:0/4187616354 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81200b6c00 con 0x7f8120160dd0 2023-07-06T03:49:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.640+0000 7f812519a700 1 --2- 172.21.15.97:0/4187616354 >> v2:172.21.15.146:3300/0 conn(0x7f81200cd390 0x7f8120160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.640+0000 7f812619c700 1 --2- 172.21.15.97:0/4187616354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8120160dd0 0x7f81200b7400 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f81200b5930 tx=0x7f811800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc96a2f6191c1bc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.640+0000 7f8124999700 1 -- 172.21.15.97:0/4187616354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811800e070 con 0x7f8120160dd0 2023-07-06T03:49:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.640+0000 7f8124999700 1 -- 172.21.15.97:0/4187616354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8118003a20 con 0x7f8120160dd0 2023-07-06T03:49:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.640+0000 7f8124999700 1 -- 172.21.15.97:0/4187616354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118003dd0 con 0x7f8120160dd0 2023-07-06T03:49:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.640+0000 7f8127bff700 1 -- 172.21.15.97:0/4187616354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8120160dd0 msgr2=0x7f81200b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:03.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.640+0000 7f8127bff700 1 --2- 172.21.15.97:0/4187616354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8120160dd0 0x7f81200b7400 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f81200b5930 tx=0x7f811800b330 comp rx=0 tx=0).stop 2023-07-06T03:49:03.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.641+0000 7f8127bff700 1 -- 172.21.15.97:0/4187616354 shutdown_connections 2023-07-06T03:49:03.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.641+0000 7f8127bff700 1 --2- 172.21.15.97:0/4187616354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8120160dd0 0x7f81200b7400 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:03.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.641+0000 7f8127bff700 1 --2- 172.21.15.97:0/4187616354 >> v2:172.21.15.146:3300/0 conn(0x7f81200cd390 0x7f8120160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:03.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.641+0000 7f8127bff700 1 -- 172.21.15.97:0/4187616354 >> 172.21.15.97:0/4187616354 conn(0x7f8120071790 msgr2=0x7f8120071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:03.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.641+0000 7f8127bff700 1 -- 172.21.15.97:0/4187616354 shutdown_connections 2023-07-06T03:49:03.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.641+0000 7f8127bff700 1 -- 172.21.15.97:0/4187616354 wait complete. 2023-07-06T03:49:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.642+0000 7f8127bff700 1 Processor -- start 2023-07-06T03:49:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.643+0000 7f8127bff700 1 -- start start 2023-07-06T03:49:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.643+0000 7f8127bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81200cd390 0x7f8120155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f8127bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f812015c520 0x7f8120155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f8127bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8120160dd0 0x7f81200d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f8127bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8120154a00 con 0x7f81200cd390 2023-07-06T03:49:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f8127bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812015d5d0 con 0x7f8120160dd0 2023-07-06T03:49:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f8127bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812015d8b0 con 0x7f812015c520 2023-07-06T03:49:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f812599b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81200cd390 0x7f8120155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f812599b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81200cd390 0x7f8120155570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52426/0 (socket says 172.21.15.97:52426) 2023-07-06T03:49:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f812599b700 1 -- 172.21.15.97:0/2322936889 learned_addr learned my addr 172.21.15.97:0/2322936889 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f812519a700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f812015c520 0x7f8120155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f812619c700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8120160dd0 0x7f81200d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f812599b700 1 -- 172.21.15.97:0/2322936889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f812015c520 msgr2=0x7f8120155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f812599b700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f812015c520 0x7f8120155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f812599b700 1 -- 172.21.15.97:0/2322936889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8120160dd0 msgr2=0x7f81200d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f812599b700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8120160dd0 0x7f81200d2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f812599b700 1 -- 172.21.15.97:0/2322936889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8118009b30 con 0x7f81200cd390 2023-07-06T03:49:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.644+0000 7f812599b700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81200cd390 0x7f8120155570 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f811c00c4a0 tx=0x7f811c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.645+0000 7f81167fc700 1 -- 172.21.15.97:0/2322936889 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c0054e0 con 0x7f81200cd390 2023-07-06T03:49:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.645+0000 7f81167fc700 1 -- 172.21.15.97:0/2322936889 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c003680 con 0x7f81200cd390 2023-07-06T03:49:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.645+0000 7f81167fc700 1 -- 172.21.15.97:0/2322936889 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c003a30 con 0x7f81200cd390 2023-07-06T03:49:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.645+0000 7f8127bff700 1 -- 172.21.15.97:0/2322936889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812015db90 con 0x7f81200cd390 2023-07-06T03:49:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.645+0000 7f8127bff700 1 -- 172.21.15.97:0/2322936889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812015e150 con 0x7f81200cd390 2023-07-06T03:49:03.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.646+0000 7f8127bff700 1 -- 172.21.15.97:0/2322936889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812015ff40 con 0x7f81200cd390 2023-07-06T03:49:03.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.647+0000 7f81167fc700 1 -- 172.21.15.97:0/2322936889 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f811c003b90 con 0x7f81200cd390 2023-07-06T03:49:03.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.648+0000 7f81167fc700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f810406ace0 0x7f810406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:03.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.648+0000 7f81167fc700 1 -- 172.21.15.97:0/2322936889 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f811c088690 con 0x7f81200cd390 2023-07-06T03:49:03.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.648+0000 7f812519a700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f810406ace0 0x7f810406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:03.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.648+0000 7f812519a700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f810406ace0 0x7f810406d190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8110006fd0 tx=0x7f8110008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:03.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.652+0000 7f81167fc700 1 -- 172.21.15.97:0/2322936889 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f811c053660 con 0x7f81200cd390 2023-07-06T03:49:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.850+0000 7f8127bff700 1 -- 172.21.15.97:0/2322936889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8120155ff0 con 0x7f81200cd390 2023-07-06T03:49:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.851+0000 7f81167fc700 1 -- 172.21.15.97:0/2322936889 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18703 (secure 0 0 0) 0x7f811c056c80 con 0x7f81200cd390 2023-07-06T03:49:03.854 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:03.855 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":62,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:48:53.298133+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:48:43.594325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-06T03:48:44.371987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.3","mappings":[{"from":2,"to":5}]},{"pgid":"10.7","mappings":[{"from":4,"to":1}]},{"pgid":"10.16","mappings":[{"from":2,"to":1}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.1f","mappings":[{"from":2,"to":5}]},{"pgid":"10.22","mappings":[{"from":2,"to":7}]},{"pgid":"10.2d","mappings":[{"from":2,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:49:03.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 -- 172.21.15.97:0/2322936889 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f810406ace0 msgr2=0x7f810406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:03.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f810406ace0 0x7f810406d190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8110006fd0 tx=0x7f8110008040 comp rx=0 tx=0).stop 2023-07-06T03:49:03.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 -- 172.21.15.97:0/2322936889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81200cd390 msgr2=0x7f8120155570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:03.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81200cd390 0x7f8120155570 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f811c00c4a0 tx=0x7f811c00c860 comp rx=0 tx=0).stop 2023-07-06T03:49:03.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 -- 172.21.15.97:0/2322936889 shutdown_connections 2023-07-06T03:49:03.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f810406ace0 0x7f810406d190 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:03.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8120160dd0 0x7f81200d2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:03.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81200cd390 0x7f8120155570 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:03.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 --2- 172.21.15.97:0/2322936889 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f812015c520 0x7f8120155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:03.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 -- 172.21.15.97:0/2322936889 >> 172.21.15.97:0/2322936889 conn(0x7f8120071790 msgr2=0x7f81200c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:03.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 -- 172.21.15.97:0/2322936889 shutdown_connections 2023-07-06T03:49:03.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:03.854+0000 7f8127bff700 1 -- 172.21.15.97:0/2322936889 wait complete. 2023-07-06T03:49:03.874 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:48:43.594325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:48:44.371987+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:49:03.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:49:04.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:49:04.035 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:49:04.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.072+0000 7fa21e5ff700 1 Processor -- start 2023-07-06T03:49:04.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.073+0000 7fa21e5ff700 1 -- start start 2023-07-06T03:49:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.074+0000 7fa21e5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180d2210 0x7fa2180b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.074+0000 7fa21e5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2180cff90 con 0x7fa21815c610 2023-07-06T03:49:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.074+0000 7fa21e5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2180b5650 con 0x7fa21806b360 2023-07-06T03:49:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.074+0000 7fa21e5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2180d0780 con 0x7fa2180d2210 2023-07-06T03:49:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa217fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa21815c610 0x7fa2180b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41802/0 (socket says 172.21.15.97:41802) 2023-07-06T03:49:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa217fff700 1 -- 172.21.15.97:0/2632099876 learned_addr learned my addr 172.21.15.97:0/2632099876 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa21cb9c700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180d2210 0x7fa2180b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282934162 0 0) 0x7fa2180cff90 con 0x7fa21815c610 2023-07-06T03:49:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2000035b0 con 0x7fa21815c610 2023-07-06T03:49:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030047851 0 0) 0x7fa2180b5650 con 0x7fa21806b360 2023-07-06T03:49:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2180cff90 con 0x7fa21806b360 2023-07-06T03:49:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2708059490 0 0) 0x7fa2000035b0 con 0x7fa21815c610 2023-07-06T03:49:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180d2210 msgr2=0x7fa2180b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180d2210 0x7fa2180b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 >> v1:172.21.15.146:6789/0 conn(0x7fa21806b360 legacy=0x7fa2180d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2180b5400 con 0x7fa21815c610 2023-07-06T03:49:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fa208001670 con 0x7fa21815c610 2023-07-06T03:49:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa21cb9c700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180d2210 0x7fa2180b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 >> v1:172.21.15.97:6789/0 conn(0x7fa21815c610 legacy=0x7fa2180b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180d2210 0x7fa200003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.075+0000 7fa216ffd700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa200004150 0x7fa200004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa216ffd700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 0x7fa200005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2180d0780 con 0x7fa200004d90 2023-07-06T03:49:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20000baf0 con 0x7fa200004150 2023-07-06T03:49:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20000bcb0 con 0x7fa2180d2210 2023-07-06T03:49:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa21cb9c700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 0x7fa200005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa2177fe700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa200004150 0x7fa200004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa21cb9c700 1 -- 172.21.15.97:0/2632099876 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180d2210 msgr2=0x7fa200003bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T03:49:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa21cb9c700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180d2210 0x7fa200003bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa21cb9c700 1 -- 172.21.15.97:0/2632099876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa200004150 msgr2=0x7fa200004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa21cb9c700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa200004150 0x7fa200004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa21cb9c700 1 -- 172.21.15.97:0/2632099876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa210009a70 con 0x7fa200004d90 2023-07-06T03:49:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.076+0000 7fa21cb9c700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 0x7fa200005200 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e350 tx=0x7fa200005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f4854f2ce9b5e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.077+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21000ee10 con 0x7fa200004d90 2023-07-06T03:49:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.077+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21000f040 con 0x7fa200004d90 2023-07-06T03:49:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.077+0000 7fa216ffd700 1 -- 172.21.15.97:0/2632099876 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210003c40 con 0x7fa200004d90 2023-07-06T03:49:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.077+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2632099876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 msgr2=0x7fa200005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.077+0000 7fa21e5ff700 1 --2- 172.21.15.97:0/2632099876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 0x7fa200005200 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e350 tx=0x7fa200005880 comp rx=0 tx=0).stop 2023-07-06T03:49:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.077+0000 7fa21cb9c700 1 -- 172.21.15.97:0/2632099876 reap_dead start 2023-07-06T03:49:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.077+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2632099876 shutdown_connections 2023-07-06T03:49:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.077+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2632099876 >> 172.21.15.97:0/2632099876 conn(0x7fa21806a6c0 msgr2=0x7fa21806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.077+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2632099876 shutdown_connections 2023-07-06T03:49:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.078+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2632099876 wait complete. 2023-07-06T03:49:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.079+0000 7fa21e5ff700 1 Processor -- start 2023-07-06T03:49:04.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.079+0000 7fa21e5ff700 1 -- start start 2023-07-06T03:49:04.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.080+0000 7fa21e5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 0x7fa21815dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.080+0000 7fa21e5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa21815f230 0x7fa2180c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.080+0000 7fa21e5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180c6da0 0x7fa2180c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.080+0000 7fa21e5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2180c7220 con 0x7fa200004d90 2023-07-06T03:49:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.080+0000 7fa21e5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21815eda0 con 0x7fa21815f230 2023-07-06T03:49:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.080+0000 7fa21e5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2181f1da0 con 0x7fa2180c6da0 2023-07-06T03:49:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.081+0000 7fa217fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 0x7fa21815dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.081+0000 7fa217fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 0x7fa21815dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52448/0 (socket says 172.21.15.97:52448) 2023-07-06T03:49:04.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.081+0000 7fa217fff700 1 -- 172.21.15.97:0/2788365331 learned_addr learned my addr 172.21.15.97:0/2788365331 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:04.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.081+0000 7fa2177fe700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa21815f230 0x7fa2180c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.081+0000 7fa21cb9c700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180c6da0 0x7fa2180c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.081+0000 7fa217fff700 1 -- 172.21.15.97:0/2788365331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180c6da0 msgr2=0x7fa2180c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.081+0000 7fa217fff700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180c6da0 0x7fa2180c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.081+0000 7fa217fff700 1 -- 172.21.15.97:0/2788365331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa21815f230 msgr2=0x7fa2180c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.081+0000 7fa217fff700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa21815f230 0x7fa2180c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.081+0000 7fa217fff700 1 -- 172.21.15.97:0/2788365331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21815e190 con 0x7fa200004d90 2023-07-06T03:49:04.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.081+0000 7fa217fff700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 0x7fa21815dbb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa20800c760 tx=0x7fa20800ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:04.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.082+0000 7fa214ff9700 1 -- 172.21.15.97:0/2788365331 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208004c80 con 0x7fa200004d90 2023-07-06T03:49:04.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.082+0000 7fa214ff9700 1 -- 172.21.15.97:0/2788365331 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa208004de0 con 0x7fa200004d90 2023-07-06T03:49:04.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.082+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2788365331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21815e470 con 0x7fa200004d90 2023-07-06T03:49:04.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.082+0000 7fa214ff9700 1 -- 172.21.15.97:0/2788365331 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2080072f0 con 0x7fa200004d90 2023-07-06T03:49:04.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.082+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2788365331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21815ea30 con 0x7fa200004d90 2023-07-06T03:49:04.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.083+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2788365331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2180ce040 con 0x7fa200004d90 2023-07-06T03:49:04.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.084+0000 7fa214ff9700 1 -- 172.21.15.97:0/2788365331 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa208007450 con 0x7fa200004d90 2023-07-06T03:49:04.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.085+0000 7fa214ff9700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa20406ad10 0x7fa20406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.085+0000 7fa214ff9700 1 -- 172.21.15.97:0/2788365331 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fa2080892f0 con 0x7fa200004d90 2023-07-06T03:49:04.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.085+0000 7fa2177fe700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa20406ad10 0x7fa20406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.086+0000 7fa2177fe700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa20406ad10 0x7fa20406d1c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa20c000c00 tx=0x7fa20c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:04.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.090+0000 7fa214ff9700 1 -- 172.21.15.97:0/2788365331 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa2080542d0 con 0x7fa200004d90 2023-07-06T03:49:04.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.286+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2788365331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa21804e340 con 0x7fa200004d90 2023-07-06T03:49:04.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.287+0000 7fa214ff9700 1 -- 172.21.15.97:0/2788365331 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa208007700 con 0x7fa200004d90 2023-07-06T03:49:04.290 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:49:04.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2788365331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa20406ad10 msgr2=0x7fa20406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa20406ad10 0x7fa20406d1c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa20c000c00 tx=0x7fa20c009040 comp rx=0 tx=0).stop 2023-07-06T03:49:04.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2788365331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 msgr2=0x7fa21815dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 0x7fa21815dbb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa20800c760 tx=0x7fa20800ca70 comp rx=0 tx=0).stop 2023-07-06T03:49:04.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2788365331 shutdown_connections 2023-07-06T03:49:04.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa20406ad10 0x7fa20406d1c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa2180c6da0 0x7fa2180c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200004d90 0x7fa21815dbb0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 --2- 172.21.15.97:0/2788365331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa21815f230 0x7fa2180c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2788365331 >> 172.21.15.97:0/2788365331 conn(0x7fa21806a6c0 msgr2=0x7fa2180d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:04.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2788365331 shutdown_connections 2023-07-06T03:49:04.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.289+0000 7fa21e5ff700 1 -- 172.21.15.97:0/2788365331 wait complete. 2023-07-06T03:49:04.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:49:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.504+0000 7f201cbff700 1 Processor -- start 2023-07-06T03:49:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.504+0000 7f201cbff700 1 -- start start 2023-07-06T03:49:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.505+0000 7f201cbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20180cbaa0 0x7f201815a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.505+0000 7f201cbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f201815eb90 0x7f201815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.505+0000 7f201cbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f201815b6e0 con 0x7f2018154890 2023-07-06T03:49:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.505+0000 7f201cbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f201815e0b0 con 0x7f201815eb90 2023-07-06T03:49:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.505+0000 7f201cbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201815e390 con 0x7f20180cbaa0 2023-07-06T03:49:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.505+0000 7f201659c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2018154890 0x7f20180cb390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41814/0 (socket says 172.21.15.97:41814) 2023-07-06T03:49:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.505+0000 7f201659c700 1 -- 172.21.15.97:0/192360413 learned_addr learned my addr 172.21.15.97:0/192360413 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.505+0000 7f2016d9d700 1 --2- 172.21.15.97:0/192360413 >> v2:172.21.15.97:3300/0 conn(0x7f201815eb90 0x7f201815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.505+0000 7f200ffff700 1 --2- 172.21.15.97:0/192360413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20180cbaa0 0x7f201815a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.506+0000 7f2016d9d700 1 -- 172.21.15.97:0/192360413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20180cbaa0 msgr2=0x7f201815a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.506+0000 7f2016d9d700 1 --2- 172.21.15.97:0/192360413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20180cbaa0 0x7f201815a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.506+0000 7f2016d9d700 1 -- 172.21.15.97:0/192360413 >> v1:172.21.15.97:6789/0 conn(0x7f2018154890 legacy=0x7f20180cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.506+0000 7f2016d9d700 1 -- 172.21.15.97:0/192360413 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201815e670 con 0x7f201815eb90 2023-07-06T03:49:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.506+0000 7f2016d9d700 1 --2- 172.21.15.97:0/192360413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201815eb90 0x7f201815ae80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2010009d90 tx=0x7f201000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802da9076ab3c00b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.506+0000 7f2015d9b700 1 -- 172.21.15.97:0/192360413 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201000e070 con 0x7f201815eb90 2023-07-06T03:49:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.506+0000 7f2015d9b700 1 -- 172.21.15.97:0/192360413 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201000bb20 con 0x7f201815eb90 2023-07-06T03:49:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.506+0000 7f2015d9b700 1 -- 172.21.15.97:0/192360413 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010012430 con 0x7f201815eb90 2023-07-06T03:49:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.507+0000 7f201cbff700 1 -- 172.21.15.97:0/192360413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201815eb90 msgr2=0x7f201815ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.507+0000 7f201cbff700 1 --2- 172.21.15.97:0/192360413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201815eb90 0x7f201815ae80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2010009d90 tx=0x7f201000b330 comp rx=0 tx=0).stop 2023-07-06T03:49:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.507+0000 7f201cbff700 1 -- 172.21.15.97:0/192360413 shutdown_connections 2023-07-06T03:49:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.507+0000 7f201cbff700 1 --2- 172.21.15.97:0/192360413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201815eb90 0x7f201815ae80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.507+0000 7f201cbff700 1 --2- 172.21.15.97:0/192360413 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20180cbaa0 0x7f201815a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.507+0000 7f201cbff700 1 -- 172.21.15.97:0/192360413 >> 172.21.15.97:0/192360413 conn(0x7f201806a060 msgr2=0x7f201806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.507+0000 7f201cbff700 1 -- 172.21.15.97:0/192360413 shutdown_connections 2023-07-06T03:49:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.507+0000 7f201cbff700 1 -- 172.21.15.97:0/192360413 wait complete. 2023-07-06T03:49:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.509+0000 7f201cbff700 1 Processor -- start 2023-07-06T03:49:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.509+0000 7f201cbff700 1 -- start start 2023-07-06T03:49:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.510+0000 7f201cbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20180cbaa0 0x7f2018157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.510+0000 7f201cbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2018154890 0x7f20180b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.510+0000 7f201cbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201815eb90 0x7f201806a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.510+0000 7f201cbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20181efec0 con 0x7f201815eb90 2023-07-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.510+0000 7f201cbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20181f01a0 con 0x7f20180cbaa0 2023-07-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.510+0000 7f201cbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20181f0480 con 0x7f2018154890 2023-07-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.510+0000 7f2016d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201815eb90 0x7f201806a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.510+0000 7f2016d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201815eb90 0x7f201806a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52460/0 (socket says 172.21.15.97:52460) 2023-07-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.510+0000 7f2016d9d700 1 -- 172.21.15.97:0/614999452 learned_addr learned my addr 172.21.15.97:0/614999452 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.510+0000 7f201659c700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20180cbaa0 0x7f2018157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.510+0000 7f200ffff700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2018154890 0x7f20180b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.511+0000 7f2016d9d700 1 -- 172.21.15.97:0/614999452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2018154890 msgr2=0x7f20180b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.511+0000 7f2016d9d700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2018154890 0x7f20180b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.511+0000 7f2016d9d700 1 -- 172.21.15.97:0/614999452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20180cbaa0 msgr2=0x7f2018157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.511+0000 7f2016d9d700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20180cbaa0 0x7f2018157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.511+0000 7f2016d9d700 1 -- 172.21.15.97:0/614999452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2010009a70 con 0x7f201815eb90 2023-07-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.511+0000 7f2016d9d700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201815eb90 0x7f201806a430 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2010014040 tx=0x7f2010004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.511+0000 7f200effd700 1 -- 172.21.15.97:0/614999452 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010012430 con 0x7f201815eb90 2023-07-06T03:49:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.511+0000 7f200effd700 1 -- 172.21.15.97:0/614999452 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2010020070 con 0x7f201815eb90 2023-07-06T03:49:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.511+0000 7f200effd700 1 -- 172.21.15.97:0/614999452 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010003ea0 con 0x7f201815eb90 2023-07-06T03:49:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.511+0000 7f201cbff700 1 -- 172.21.15.97:0/614999452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20181f0760 con 0x7f201815eb90 2023-07-06T03:49:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.511+0000 7f201cbff700 1 -- 172.21.15.97:0/614999452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20180b3220 con 0x7f201815eb90 2023-07-06T03:49:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.512+0000 7f200cff9700 1 -- 172.21.15.97:0/614999452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20181f0980 con 0x7f201815eb90 2023-07-06T03:49:04.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.513+0000 7f200effd700 1 -- 172.21.15.97:0/614999452 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2010004040 con 0x7f201815eb90 2023-07-06T03:49:04.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.514+0000 7f200effd700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1ff806ace0 0x7f1ff806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.514+0000 7f200effd700 1 -- 172.21.15.97:0/614999452 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f2010022070 con 0x7f201815eb90 2023-07-06T03:49:04.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.514+0000 7f201659c700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1ff806ace0 0x7f1ff806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.515+0000 7f201659c700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1ff806ace0 0x7f1ff806d190 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2008005910 tx=0x7f2008009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:04.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.518+0000 7f200effd700 1 -- 172.21.15.97:0/614999452 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2010056190 con 0x7f201815eb90 2023-07-06T03:49:04.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.715+0000 7f200cff9700 1 -- 172.21.15.97:0/614999452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f20181f0000 con 0x7f201815eb90 2023-07-06T03:49:04.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.716+0000 7f200effd700 1 -- 172.21.15.97:0/614999452 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7f20181f0000 con 0x7f201815eb90 2023-07-06T03:49:04.719 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:49:04.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 -- 172.21.15.97:0/614999452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1ff806ace0 msgr2=0x7f1ff806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1ff806ace0 0x7f1ff806d190 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2008005910 tx=0x7f2008009040 comp rx=0 tx=0).stop 2023-07-06T03:49:04.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 -- 172.21.15.97:0/614999452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201815eb90 msgr2=0x7f201806a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201815eb90 0x7f201806a430 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2010014040 tx=0x7f2010004ad0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 -- 172.21.15.97:0/614999452 shutdown_connections 2023-07-06T03:49:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1ff806ace0 0x7f1ff806d190 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201815eb90 0x7f201806a430 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20180cbaa0 0x7f2018157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 --2- 172.21.15.97:0/614999452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2018154890 0x7f20180b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 -- 172.21.15.97:0/614999452 >> 172.21.15.97:0/614999452 conn(0x7f201806a060 msgr2=0x7f20180c3840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 -- 172.21.15.97:0/614999452 shutdown_connections 2023-07-06T03:49:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.718+0000 7f200cff9700 1 -- 172.21.15.97:0/614999452 wait complete. 2023-07-06T03:49:04.739 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:49:04.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.935+0000 7f356b3ff700 1 Processor -- start 2023-07-06T03:49:04.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.935+0000 7f356b3ff700 1 -- start start 2023-07-06T03:49:04.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.936+0000 7f356b3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f356406b360 0x7f35640b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.936+0000 7f356b3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35640d2210 0x7f35640d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.936+0000 7f356b3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35640cff90 con 0x7f356415c610 2023-07-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.936+0000 7f356b3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35640d0780 con 0x7f35640d2210 2023-07-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.936+0000 7f356b3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356415c9e0 con 0x7f356406b360 2023-07-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.936+0000 7f356999c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35640d2210 0x7f35640d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.936+0000 7f356999c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35640d2210 0x7f35640d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52464/0 (socket says 172.21.15.97:52464) 2023-07-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.936+0000 7f356899a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f356406b360 0x7f35640b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.936+0000 7f356999c700 1 -- 172.21.15.97:0/2294263828 learned_addr learned my addr 172.21.15.97:0/2294263828 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:04.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.937+0000 7f356999c700 1 -- 172.21.15.97:0/2294263828 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f356406b360 msgr2=0x7f35640b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:04.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.937+0000 7f356999c700 1 --2- 172.21.15.97:0/2294263828 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f356406b360 0x7f35640b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.937+0000 7f356999c700 1 -- 172.21.15.97:0/2294263828 >> v1:172.21.15.146:6789/0 conn(0x7f356415c610 legacy=0x7f35640b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.937+0000 7f355bfff700 1 -- 172.21.15.97:0/2294263828 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188871571 0 0) 0x7f35640cff90 con 0x7f356415c610 2023-07-06T03:49:04.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.937+0000 7f356999c700 1 -- 172.21.15.97:0/2294263828 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35640b5650 con 0x7f35640d2210 2023-07-06T03:49:04.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.937+0000 7f356999c700 1 --2- 172.21.15.97:0/2294263828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35640d2210 0x7f35640d1310 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f355c009e80 tx=0x7f355c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f49e1aa3a0d31 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.937+0000 7f355bfff700 1 -- 172.21.15.97:0/2294263828 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c00e070 con 0x7f35640d2210 2023-07-06T03:49:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.937+0000 7f355bfff700 1 -- 172.21.15.97:0/2294263828 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c00b8d0 con 0x7f35640d2210 2023-07-06T03:49:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.937+0000 7f355bfff700 1 -- 172.21.15.97:0/2294263828 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c00bc80 con 0x7f35640d2210 2023-07-06T03:49:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.938+0000 7f356b3ff700 1 -- 172.21.15.97:0/2294263828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35640d2210 msgr2=0x7f35640d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.938+0000 7f356b3ff700 1 --2- 172.21.15.97:0/2294263828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35640d2210 0x7f35640d1310 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f355c009e80 tx=0x7f355c00b330 comp rx=0 tx=0).stop 2023-07-06T03:49:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.938+0000 7f356b3ff700 1 -- 172.21.15.97:0/2294263828 shutdown_connections 2023-07-06T03:49:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.938+0000 7f356b3ff700 1 --2- 172.21.15.97:0/2294263828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35640d2210 0x7f35640d1310 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.938+0000 7f356b3ff700 1 --2- 172.21.15.97:0/2294263828 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f356406b360 0x7f35640b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.938+0000 7f356b3ff700 1 -- 172.21.15.97:0/2294263828 >> 172.21.15.97:0/2294263828 conn(0x7f356406a6c0 msgr2=0x7f356406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:04.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.938+0000 7f356b3ff700 1 -- 172.21.15.97:0/2294263828 shutdown_connections 2023-07-06T03:49:04.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.938+0000 7f356b3ff700 1 -- 172.21.15.97:0/2294263828 wait complete. 2023-07-06T03:49:04.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.940+0000 7f356b3ff700 1 Processor -- start 2023-07-06T03:49:04.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.941+0000 7f356b3ff700 1 -- start start 2023-07-06T03:49:04.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356b3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f356406b360 0x7f3564157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356b3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35640d2210 0x7f356415ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356b3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356415c610 0x7f356415f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356b3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356415e1f0 con 0x7f356415c610 2023-07-06T03:49:04.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356b3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356415e4d0 con 0x7f356406b360 2023-07-06T03:49:04.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356b3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35640c7410 con 0x7f35640d2210 2023-07-06T03:49:04.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356999c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356415c610 0x7f356415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356999c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356415c610 0x7f356415f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52472/0 (socket says 172.21.15.97:52472) 2023-07-06T03:49:04.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356999c700 1 -- 172.21.15.97:0/3103413776 learned_addr learned my addr 172.21.15.97:0/3103413776 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:04.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356999c700 1 -- 172.21.15.97:0/3103413776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35640d2210 msgr2=0x7f356415ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356899a700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35640d2210 0x7f356415ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356999c700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35640d2210 0x7f356415ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356919b700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f356406b360 0x7f3564157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356999c700 1 -- 172.21.15.97:0/3103413776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f356406b360 msgr2=0x7f3564157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:04.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.942+0000 7f356999c700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f356406b360 0x7f3564157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:04.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.943+0000 7f356999c700 1 -- 172.21.15.97:0/3103413776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c009b30 con 0x7f356415c610 2023-07-06T03:49:04.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.943+0000 7f356899a700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35640d2210 0x7f356415ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:04.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.943+0000 7f356919b700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f356406b360 0x7f3564157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:04.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.943+0000 7f356999c700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356415c610 0x7f356415f100 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f355c00b680 tx=0x7f355c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:04.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.943+0000 7f3559ffb700 1 -- 172.21.15.97:0/3103413776 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c0099b0 con 0x7f356415c610 2023-07-06T03:49:04.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.943+0000 7f3559ffb700 1 -- 172.21.15.97:0/3103413776 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c0092e0 con 0x7f356415c610 2023-07-06T03:49:04.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.943+0000 7f356b3ff700 1 -- 172.21.15.97:0/3103413776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35640c76f0 con 0x7f356415c610 2023-07-06T03:49:04.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.943+0000 7f3559ffb700 1 -- 172.21.15.97:0/3103413776 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c01c640 con 0x7f356415c610 2023-07-06T03:49:04.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.943+0000 7f356b3ff700 1 -- 172.21.15.97:0/3103413776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3564158450 con 0x7f356415c610 2023-07-06T03:49:04.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.944+0000 7f356b3ff700 1 -- 172.21.15.97:0/3103413776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35640c7270 con 0x7f356415c610 2023-07-06T03:49:04.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.945+0000 7f3559ffb700 1 -- 172.21.15.97:0/3103413776 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f355c01c7a0 con 0x7f356415c610 2023-07-06T03:49:04.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.946+0000 7f3559ffb700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f354806ace0 0x7f354806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:04.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.946+0000 7f3559ffb700 1 -- 172.21.15.97:0/3103413776 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f355c08afc0 con 0x7f356415c610 2023-07-06T03:49:04.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.946+0000 7f356919b700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f354806ace0 0x7f354806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:04.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.947+0000 7f356919b700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f354806ace0 0x7f354806d190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3560000f60 tx=0x7f3560009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:04.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:04.950+0000 7f3559ffb700 1 -- 172.21.15.97:0/3103413776 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f35640c7270 con 0x7f356415c610 2023-07-06T03:49:05.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.144+0000 7f356b3ff700 1 -- 172.21.15.97:0/3103413776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35640c7270 con 0x7f356415c610 2023-07-06T03:49:05.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.145+0000 7f3559ffb700 1 -- 172.21.15.97:0/3103413776 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f35640c7270 con 0x7f356415c610 2023-07-06T03:49:05.148 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:49:05.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.147+0000 7f356b3ff700 1 -- 172.21.15.97:0/3103413776 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f354806ace0 msgr2=0x7f354806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:05.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.147+0000 7f356b3ff700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f354806ace0 0x7f354806d190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3560000f60 tx=0x7f3560009040 comp rx=0 tx=0).stop 2023-07-06T03:49:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.147+0000 7f356b3ff700 1 -- 172.21.15.97:0/3103413776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356415c610 msgr2=0x7f356415f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.147+0000 7f356b3ff700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356415c610 0x7f356415f100 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f355c00b680 tx=0x7f355c004ad0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.148+0000 7f356b3ff700 1 -- 172.21.15.97:0/3103413776 shutdown_connections 2023-07-06T03:49:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.148+0000 7f356b3ff700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f354806ace0 0x7f354806d190 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.148+0000 7f356b3ff700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356415c610 0x7f356415f100 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.148+0000 7f356b3ff700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f356406b360 0x7f3564157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.148+0000 7f356b3ff700 1 --2- 172.21.15.97:0/3103413776 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35640d2210 0x7f356415ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.148+0000 7f356b3ff700 1 -- 172.21.15.97:0/3103413776 >> 172.21.15.97:0/3103413776 conn(0x7f356406a6c0 msgr2=0x7f35640b6870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:05.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.148+0000 7f356b3ff700 1 -- 172.21.15.97:0/3103413776 shutdown_connections 2023-07-06T03:49:05.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.148+0000 7f356b3ff700 1 -- 172.21.15.97:0/3103413776 wait complete. 2023-07-06T03:49:05.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:49:05.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.362+0000 7f6d61bff700 1 Processor -- start 2023-07-06T03:49:05.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.363+0000 7f6d61bff700 1 -- start start 2023-07-06T03:49:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.364+0000 7f6d61bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d5c159070 0x7f6d5c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.364+0000 7f6d61bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6d5c0c3cd0 0x7f6d5c0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.364+0000 7f6d61bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d5c159440 0x7f6d5c0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.364+0000 7f6d61bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5c0ca260 con 0x7f6d5c159440 2023-07-06T03:49:05.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.364+0000 7f6d61bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5c0c99c0 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.364+0000 7f6d61bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0c9ca0 con 0x7f6d5c159070 2023-07-06T03:49:05.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.364+0000 7f6d5bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d5c159440 0x7f6d5c0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.364+0000 7f6d5bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d5c159440 0x7f6d5c0cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52488/0 (socket says 172.21.15.97:52488) 2023-07-06T03:49:05.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.364+0000 7f6d5bfff700 1 -- 172.21.15.97:0/2174221417 learned_addr learned my addr 172.21.15.97:0/2174221417 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:05.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.364+0000 7f6d53fff700 1 --2- 172.21.15.97:0/2174221417 >> v2:172.21.15.146:3300/0 conn(0x7f6d5c0c3cd0 0x7f6d5c0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.364+0000 7f6d5b7fe700 1 --2- 172.21.15.97:0/2174221417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d5c159070 0x7f6d5c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.365+0000 7f6d5bfff700 1 -- 172.21.15.97:0/2174221417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d5c159070 msgr2=0x7f6d5c0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:05.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.365+0000 7f6d5bfff700 1 --2- 172.21.15.97:0/2174221417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d5c159070 0x7f6d5c0c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.365+0000 7f6d5bfff700 1 -- 172.21.15.97:0/2174221417 >> v2:172.21.15.146:3300/0 conn(0x7f6d5c0c3cd0 msgr2=0x7f6d5c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:05.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.365+0000 7f6d5bfff700 1 --2- 172.21.15.97:0/2174221417 >> v2:172.21.15.146:3300/0 conn(0x7f6d5c0c3cd0 0x7f6d5c0cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.365+0000 7f6d5bfff700 1 -- 172.21.15.97:0/2174221417 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c0c9f80 con 0x7f6d5c159440 2023-07-06T03:49:05.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.365+0000 7f6d5bfff700 1 --2- 172.21.15.97:0/2174221417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d5c159440 0x7f6d5c0cca00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0b15e0 tx=0x7f6d5400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d1c092122da94a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:05.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.365+0000 7f6d5affd700 1 -- 172.21.15.97:0/2174221417 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54010070 con 0x7f6d5c159440 2023-07-06T03:49:05.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.365+0000 7f6d5affd700 1 -- 172.21.15.97:0/2174221417 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5400dce0 con 0x7f6d5c159440 2023-07-06T03:49:05.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.365+0000 7f6d5affd700 1 -- 172.21.15.97:0/2174221417 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400b310 con 0x7f6d5c159440 2023-07-06T03:49:05.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.366+0000 7f6d61bff700 1 -- 172.21.15.97:0/2174221417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d5c159440 msgr2=0x7f6d5c0cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:05.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.366+0000 7f6d61bff700 1 --2- 172.21.15.97:0/2174221417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d5c159440 0x7f6d5c0cca00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0b15e0 tx=0x7f6d5400d430 comp rx=0 tx=0).stop 2023-07-06T03:49:05.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.366+0000 7f6d61bff700 1 -- 172.21.15.97:0/2174221417 shutdown_connections 2023-07-06T03:49:05.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.366+0000 7f6d61bff700 1 --2- 172.21.15.97:0/2174221417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d5c159440 0x7f6d5c0cca00 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.366+0000 7f6d61bff700 1 --2- 172.21.15.97:0/2174221417 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d5c159070 0x7f6d5c0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.366+0000 7f6d61bff700 1 --2- 172.21.15.97:0/2174221417 >> v2:172.21.15.146:3300/0 conn(0x7f6d5c0c3cd0 0x7f6d5c0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.366+0000 7f6d61bff700 1 -- 172.21.15.97:0/2174221417 >> 172.21.15.97:0/2174221417 conn(0x7f6d5c06a060 msgr2=0x7f6d5c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:05.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.366+0000 7f6d61bff700 1 -- 172.21.15.97:0/2174221417 shutdown_connections 2023-07-06T03:49:05.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.366+0000 7f6d61bff700 1 -- 172.21.15.97:0/2174221417 wait complete. 2023-07-06T03:49:05.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.368+0000 7f6d61bff700 1 Processor -- start 2023-07-06T03:49:05.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.368+0000 7f6d61bff700 1 -- start start 2023-07-06T03:49:05.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d61bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d5c0c3cd0 0x7f6d5c0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d61bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d5c159070 0x7f6d5c1e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d61bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d5c159440 0x7f6d5c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d61bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0b1460 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d61bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c14c640 con 0x7f6d5c159440 2023-07-06T03:49:05.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d61bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c1e44c0 con 0x7f6d5c159070 2023-07-06T03:49:05.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d5b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d5c0c3cd0 0x7f6d5c0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d5b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d5c0c3cd0 0x7f6d5c0ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52494/0 (socket says 172.21.15.97:52494) 2023-07-06T03:49:05.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d5b7fe700 1 -- 172.21.15.97:0/2107855834 learned_addr learned my addr 172.21.15.97:0/2107855834 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:05.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d5bfff700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d5c159440 0x7f6d5c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d53fff700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d5c159070 0x7f6d5c1e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d5b7fe700 1 -- 172.21.15.97:0/2107855834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d5c159070 msgr2=0x7f6d5c1e3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:05.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d5b7fe700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d5c159070 0x7f6d5c1e3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d5b7fe700 1 -- 172.21.15.97:0/2107855834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d5c159440 msgr2=0x7f6d5c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:05.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d5b7fe700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d5c159440 0x7f6d5c0b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d5b7fe700 1 -- 172.21.15.97:0/2107855834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5400bd40 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.369+0000 7f6d5bfff700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d5c159440 0x7f6d5c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:05.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.370+0000 7f6d5b7fe700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d5c0c3cd0 0x7f6d5c0ae540 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00e9b0 tx=0x7f6d4c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.370+0000 7f6d58ff9700 1 -- 172.21.15.97:0/2107855834 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c009710 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.370+0000 7f6d58ff9700 1 -- 172.21.15.97:0/2107855834 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c00c850 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.370+0000 7f6d61bff700 1 -- 172.21.15.97:0/2107855834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c14c310 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.370+0000 7f6d58ff9700 1 -- 172.21.15.97:0/2107855834 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c00cbc0 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.370+0000 7f6d61bff700 1 -- 172.21.15.97:0/2107855834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c1e6840 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.371+0000 7f6d61bff700 1 -- 172.21.15.97:0/2107855834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c14c8b0 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.372+0000 7f6d58ff9700 1 -- 172.21.15.97:0/2107855834 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6d4c00cd20 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.373+0000 7f6d58ff9700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d3c06ace0 0x7f6d3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.373+0000 7f6d58ff9700 1 -- 172.21.15.97:0/2107855834 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f6d4c0895f0 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.373+0000 7f6d53fff700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d3c06ace0 0x7f6d3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.373+0000 7f6d53fff700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d3c06ace0 0x7f6d3c06d190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c1f53e0 tx=0x7f6d44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:05.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.377+0000 7f6d58ff9700 1 -- 172.21.15.97:0/2107855834 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6d4c058050 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.572+0000 7f6d61bff700 1 -- 172.21.15.97:0/2107855834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d5c04e340 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.572+0000 7f6d58ff9700 1 -- 172.21.15.97:0/2107855834 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d4c057be0 con 0x7f6d5c0c3cd0 2023-07-06T03:49:05.575 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:49:05.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.574+0000 7f6d61bff700 1 -- 172.21.15.97:0/2107855834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d3c06ace0 msgr2=0x7f6d3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:05.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.574+0000 7f6d61bff700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d3c06ace0 0x7f6d3c06d190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c1f53e0 tx=0x7f6d44008040 comp rx=0 tx=0).stop 2023-07-06T03:49:05.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.574+0000 7f6d61bff700 1 -- 172.21.15.97:0/2107855834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d5c0c3cd0 msgr2=0x7f6d5c0ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:05.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.574+0000 7f6d61bff700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d5c0c3cd0 0x7f6d5c0ae540 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00e9b0 tx=0x7f6d4c00ed70 comp rx=0 tx=0).stop 2023-07-06T03:49:05.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.574+0000 7f6d61bff700 1 -- 172.21.15.97:0/2107855834 shutdown_connections 2023-07-06T03:49:05.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.574+0000 7f6d61bff700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d3c06ace0 0x7f6d3c06d190 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.574+0000 7f6d61bff700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d5c159440 0x7f6d5c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.574+0000 7f6d61bff700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d5c0c3cd0 0x7f6d5c0ae540 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.574+0000 7f6d61bff700 1 --2- 172.21.15.97:0/2107855834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d5c159070 0x7f6d5c1e3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.574+0000 7f6d61bff700 1 -- 172.21.15.97:0/2107855834 >> 172.21.15.97:0/2107855834 conn(0x7f6d5c06a060 msgr2=0x7f6d5c0cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:05.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.575+0000 7f6d61bff700 1 -- 172.21.15.97:0/2107855834 shutdown_connections 2023-07-06T03:49:05.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.575+0000 7f6d61bff700 1 -- 172.21.15.97:0/2107855834 wait complete. 2023-07-06T03:49:05.595 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:49:05.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.792+0000 7f9623dff700 1 Processor -- start 2023-07-06T03:49:05.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.792+0000 7f9623dff700 1 -- start start 2023-07-06T03:49:05.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.793+0000 7f9623dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961c06b170 0x7f961c15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.793+0000 7f9623dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f961c0cfbc0 0x7f961c15e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.793+0000 7f9623dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c15de00 con 0x7f961c15cae0 2023-07-06T03:49:05.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.793+0000 7f9623dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961c0b25a0 con 0x7f961c0cfbc0 2023-07-06T03:49:05.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.793+0000 7f9623dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961c0ceb20 con 0x7f961c06b170 2023-07-06T03:49:05.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.793+0000 7f962239c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f961c0cfbc0 0x7f961c15e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.794+0000 7f962239c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f961c0cfbc0 0x7f961c15e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52498/0 (socket says 172.21.15.97:52498) 2023-07-06T03:49:05.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.794+0000 7f962239c700 1 -- 172.21.15.97:0/1671801114 learned_addr learned my addr 172.21.15.97:0/1671801114 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:05.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.794+0000 7f9621b9b700 1 --2- 172.21.15.97:0/1671801114 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961c06b170 0x7f961c15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.794+0000 7f962239c700 1 -- 172.21.15.97:0/1671801114 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961c06b170 msgr2=0x7f961c15d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:05.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.794+0000 7f962239c700 1 --2- 172.21.15.97:0/1671801114 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961c06b170 0x7f961c15d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.794+0000 7f962239c700 1 -- 172.21.15.97:0/1671801114 >> v1:172.21.15.146:6789/0 conn(0x7f961c15cae0 legacy=0x7f961c06b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:05.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.794+0000 7f962239c700 1 -- 172.21.15.97:0/1671801114 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c0cee00 con 0x7f961c0cfbc0 2023-07-06T03:49:05.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.794+0000 7f962239c700 1 --2- 172.21.15.97:0/1671801114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961c0cfbc0 0x7f961c15e1b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9614009e50 tx=0x7f961400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84c55e7b60b542 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:05.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.794+0000 7f9620b99700 1 -- 172.21.15.97:0/1671801114 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961400e070 con 0x7f961c0cfbc0 2023-07-06T03:49:05.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.794+0000 7f9620b99700 1 -- 172.21.15.97:0/1671801114 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961400b910 con 0x7f961c0cfbc0 2023-07-06T03:49:05.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.794+0000 7f9620b99700 1 -- 172.21.15.97:0/1671801114 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961400bcc0 con 0x7f961c0cfbc0 2023-07-06T03:49:05.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.795+0000 7f9623dff700 1 -- 172.21.15.97:0/1671801114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961c0cfbc0 msgr2=0x7f961c15e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:05.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.795+0000 7f9623dff700 1 --2- 172.21.15.97:0/1671801114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961c0cfbc0 0x7f961c15e1b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9614009e50 tx=0x7f961400b330 comp rx=0 tx=0).stop 2023-07-06T03:49:05.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.795+0000 7f9623dff700 1 -- 172.21.15.97:0/1671801114 shutdown_connections 2023-07-06T03:49:05.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.795+0000 7f9623dff700 1 --2- 172.21.15.97:0/1671801114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961c0cfbc0 0x7f961c15e1b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.795+0000 7f9623dff700 1 --2- 172.21.15.97:0/1671801114 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961c06b170 0x7f961c15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.795+0000 7f9623dff700 1 -- 172.21.15.97:0/1671801114 >> 172.21.15.97:0/1671801114 conn(0x7f961c099f10 msgr2=0x7f961c0d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:05.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.795+0000 7f9623dff700 1 -- 172.21.15.97:0/1671801114 shutdown_connections 2023-07-06T03:49:05.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.795+0000 7f9623dff700 1 -- 172.21.15.97:0/1671801114 wait complete. 2023-07-06T03:49:05.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.797+0000 7f9623dff700 1 Processor -- start 2023-07-06T03:49:05.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.797+0000 7f9623dff700 1 -- start start 2023-07-06T03:49:05.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.798+0000 7f9623dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c06b170 0x7f961c154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.798+0000 7f9623dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961c0cfbc0 0x7f961c153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.798+0000 7f9623dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961c15cae0 0x7f961c1e8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.798+0000 7f9623dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c157ee0 con 0x7f961c0cfbc0 2023-07-06T03:49:05.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.798+0000 7f9623dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c1e77c0 con 0x7f961c06b170 2023-07-06T03:49:05.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.798+0000 7f9623dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961c1542e0 con 0x7f961c15cae0 2023-07-06T03:49:05.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.798+0000 7f962139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961c0cfbc0 0x7f961c153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.798+0000 7f962139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961c0cfbc0 0x7f961c153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52500/0 (socket says 172.21.15.97:52500) 2023-07-06T03:49:05.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.798+0000 7f962139a700 1 -- 172.21.15.97:0/120056461 learned_addr learned my addr 172.21.15.97:0/120056461 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:05.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.798+0000 7f962239c700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961c15cae0 0x7f961c1e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.798+0000 7f9621b9b700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c06b170 0x7f961c154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f962139a700 1 -- 172.21.15.97:0/120056461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961c15cae0 msgr2=0x7f961c1e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:05.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f962139a700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961c15cae0 0x7f961c1e8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f962139a700 1 -- 172.21.15.97:0/120056461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c06b170 msgr2=0x7f961c154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:05.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f962139a700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c06b170 0x7f961c154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:05.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f962139a700 1 -- 172.21.15.97:0/120056461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9614009b30 con 0x7f961c0cfbc0 2023-07-06T03:49:05.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f962239c700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961c15cae0 0x7f961c1e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:05.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f962139a700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961c0cfbc0 0x7f961c153da0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c450 tx=0x7f960c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:05.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f96127fc700 1 -- 172.21.15.97:0/120056461 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0052d0 con 0x7f961c0cfbc0 2023-07-06T03:49:05.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f96127fc700 1 -- 172.21.15.97:0/120056461 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c005430 con 0x7f961c0cfbc0 2023-07-06T03:49:05.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f96127fc700 1 -- 172.21.15.97:0/120056461 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0038c0 con 0x7f961c0cfbc0 2023-07-06T03:49:05.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f9623dff700 1 -- 172.21.15.97:0/120056461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961c1e6e20 con 0x7f961c0cfbc0 2023-07-06T03:49:05.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.799+0000 7f9623dff700 1 -- 172.21.15.97:0/120056461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961c1e73e0 con 0x7f961c0cfbc0 2023-07-06T03:49:05.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.800+0000 7f9623dff700 1 -- 172.21.15.97:0/120056461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961c1e7900 con 0x7f961c0cfbc0 2023-07-06T03:49:05.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.801+0000 7f96127fc700 1 -- 172.21.15.97:0/120056461 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f960c003a20 con 0x7f961c0cfbc0 2023-07-06T03:49:05.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.802+0000 7f96127fc700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f960006ace0 0x7f960006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:05.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.802+0000 7f96127fc700 1 -- 172.21.15.97:0/120056461 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f960c0887d0 con 0x7f961c0cfbc0 2023-07-06T03:49:05.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.802+0000 7f9621b9b700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f960006ace0 0x7f960006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:05.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.803+0000 7f9621b9b700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f960006ace0 0x7f960006d190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9618009b50 tx=0x7f9618008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:05.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:05.807+0000 7f96127fc700 1 -- 172.21.15.97:0/120056461 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f960c0537a0 con 0x7f961c0cfbc0 2023-07-06T03:49:06.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.019+0000 7f9623dff700 1 -- 172.21.15.97:0/120056461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f961c15de00 con 0x7f961c0cfbc0 2023-07-06T03:49:06.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.019+0000 7f96127fc700 1 -- 172.21.15.97:0/120056461 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7f961c15de00 con 0x7f961c0cfbc0 2023-07-06T03:49:06.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 -- 172.21.15.97:0/120056461 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f960006ace0 msgr2=0x7f960006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f960006ace0 0x7f960006d190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9618009b50 tx=0x7f9618008040 comp rx=0 tx=0).stop 2023-07-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 -- 172.21.15.97:0/120056461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961c0cfbc0 msgr2=0x7f961c153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961c0cfbc0 0x7f961c153da0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c450 tx=0x7f960c00c810 comp rx=0 tx=0).stop 2023-07-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 -- 172.21.15.97:0/120056461 shutdown_connections 2023-07-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f960006ace0 0x7f960006d190 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961c15cae0 0x7f961c1e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f961c06b170 0x7f961c154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 --2- 172.21.15.97:0/120056461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961c0cfbc0 0x7f961c153da0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 -- 172.21.15.97:0/120056461 >> 172.21.15.97:0/120056461 conn(0x7f961c099f10 msgr2=0x7f961c0b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 -- 172.21.15.97:0/120056461 shutdown_connections 2023-07-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.022+0000 7f9623dff700 1 -- 172.21.15.97:0/120056461 wait complete. 2023-07-06T03:49:06.026 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:49:06.042 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:49:06.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:06.043 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:49:06.110 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[139442]: fuse finished with error 0 and tester_r 0 2023-07-06T03:49:06.113 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:49:06.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:06.114 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:49:06.300 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:49:06.301 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:49:06.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:49:06.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.539+0000 7f407e7ff700 1 Processor -- start 2023-07-06T03:49:06.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.540+0000 7f407e7ff700 1 -- start start 2023-07-06T03:49:06.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f407e7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f407815c520 0x7f40780cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f407e7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4078160d00 0x7f40780b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f407e7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40780cdae0 con 0x7f4078160180 2023-07-06T03:49:06.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f407e7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40780cd8d0 con 0x7f407815c520 2023-07-06T03:49:06.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f407e7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40780b6640 con 0x7f4078160d00 2023-07-06T03:49:06.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f4077fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f407815c520 0x7f40780cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f4077fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f407815c520 0x7f40780cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52516/0 (socket says 172.21.15.97:52516) 2023-07-06T03:49:06.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f4077fff700 1 -- 172.21.15.97:0/2075496076 learned_addr learned my addr 172.21.15.97:0/2075496076 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:06.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f407cd9c700 1 --2- 172.21.15.97:0/2075496076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4078160d00 0x7f40780b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f4077fff700 1 -- 172.21.15.97:0/2075496076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4078160d00 msgr2=0x7f40780b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f4077fff700 1 --2- 172.21.15.97:0/2075496076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4078160d00 0x7f40780b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f4077fff700 1 -- 172.21.15.97:0/2075496076 >> v1:172.21.15.97:6789/0 conn(0x7f4078160180 legacy=0x7f40781605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.541+0000 7f4077fff700 1 -- 172.21.15.97:0/2075496076 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780b6920 con 0x7f407815c520 2023-07-06T03:49:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.542+0000 7f4077fff700 1 --2- 172.21.15.97:0/2075496076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f407815c520 0x7f40780cd390 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f40680098f0 tx=0x7f4068009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745153263d107c30 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.542+0000 7f4076ffd700 1 -- 172.21.15.97:0/2075496076 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406800b8a0 con 0x7f407815c520 2023-07-06T03:49:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.542+0000 7f4076ffd700 1 -- 172.21.15.97:0/2075496076 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406800ba00 con 0x7f407815c520 2023-07-06T03:49:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.542+0000 7f4076ffd700 1 -- 172.21.15.97:0/2075496076 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406800bd90 con 0x7f407815c520 2023-07-06T03:49:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.542+0000 7f407e7ff700 1 -- 172.21.15.97:0/2075496076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f407815c520 msgr2=0x7f40780cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.542+0000 7f407e7ff700 1 --2- 172.21.15.97:0/2075496076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f407815c520 0x7f40780cd390 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f40680098f0 tx=0x7f4068009c00 comp rx=0 tx=0).stop 2023-07-06T03:49:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.542+0000 7f407e7ff700 1 -- 172.21.15.97:0/2075496076 shutdown_connections 2023-07-06T03:49:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.542+0000 7f407e7ff700 1 --2- 172.21.15.97:0/2075496076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f407815c520 0x7f40780cd390 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.542+0000 7f407e7ff700 1 --2- 172.21.15.97:0/2075496076 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4078160d00 0x7f40780b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.542+0000 7f407e7ff700 1 -- 172.21.15.97:0/2075496076 >> 172.21.15.97:0/2075496076 conn(0x7f4078071790 msgr2=0x7f4078071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:06.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.543+0000 7f407e7ff700 1 -- 172.21.15.97:0/2075496076 shutdown_connections 2023-07-06T03:49:06.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.543+0000 7f407e7ff700 1 -- 172.21.15.97:0/2075496076 wait complete. 2023-07-06T03:49:06.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.544+0000 7f407e7ff700 1 Processor -- start 2023-07-06T03:49:06.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.544+0000 7f407e7ff700 1 -- start start 2023-07-06T03:49:06.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.545+0000 7f407e7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f407815c520 0x7f40780cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.545+0000 7f407e7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4078160180 0x7f40781e6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.545+0000 7f407e7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4078160d00 0x7f40781e6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.545+0000 7f407e7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780d1130 con 0x7f4078160d00 2023-07-06T03:49:06.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.545+0000 7f407e7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780cc7c0 con 0x7f407815c520 2023-07-06T03:49:06.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.545+0000 7f407e7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40780ccaa0 con 0x7f4078160180 2023-07-06T03:49:06.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f407cd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4078160d00 0x7f40781e6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f407cd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4078160d00 0x7f40781e6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52526/0 (socket says 172.21.15.97:52526) 2023-07-06T03:49:06.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f407cd9c700 1 -- 172.21.15.97:0/3001404720 learned_addr learned my addr 172.21.15.97:0/3001404720 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:06.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f4077fff700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f407815c520 0x7f40780cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f40777fe700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4078160180 0x7f40781e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f407cd9c700 1 -- 172.21.15.97:0/3001404720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4078160180 msgr2=0x7f40781e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f407cd9c700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4078160180 0x7f40781e6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f407cd9c700 1 -- 172.21.15.97:0/3001404720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f407815c520 msgr2=0x7f40780cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f407cd9c700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f407815c520 0x7f40780cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f407cd9c700 1 -- 172.21.15.97:0/3001404720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4068009550 con 0x7f4078160d00 2023-07-06T03:49:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f4077fff700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f407815c520 0x7f40780cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f407cd9c700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4078160d00 0x7f40781e6a80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4070007e60 tx=0x7f407000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f4074ff9700 1 -- 172.21.15.97:0/3001404720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407000ede0 con 0x7f4078160d00 2023-07-06T03:49:06.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.546+0000 7f4074ff9700 1 -- 172.21.15.97:0/3001404720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4070014420 con 0x7f4078160d00 2023-07-06T03:49:06.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.547+0000 7f407e7ff700 1 -- 172.21.15.97:0/3001404720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40780d0dc0 con 0x7f4078160d00 2023-07-06T03:49:06.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.547+0000 7f4074ff9700 1 -- 172.21.15.97:0/3001404720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40700147d0 con 0x7f4078160d00 2023-07-06T03:49:06.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.547+0000 7f407e7ff700 1 -- 172.21.15.97:0/3001404720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40781e7ee0 con 0x7f4078160d00 2023-07-06T03:49:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.548+0000 7f407e7ff700 1 -- 172.21.15.97:0/3001404720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40780d12e0 con 0x7f4078160d00 2023-07-06T03:49:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.548+0000 7f4074ff9700 1 -- 172.21.15.97:0/3001404720 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4070014930 con 0x7f4078160d00 2023-07-06T03:49:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.550+0000 7f4074ff9700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f406406ace0 0x7f406406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.550+0000 7f4074ff9700 1 -- 172.21.15.97:0/3001404720 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f4070089dd0 con 0x7f4078160d00 2023-07-06T03:49:06.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.550+0000 7f4077fff700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f406406ace0 0x7f406406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.551+0000 7f4077fff700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f406406ace0 0x7f406406d190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4068009c40 tx=0x7f4068008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:06.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.554+0000 7f4074ff9700 1 -- 172.21.15.97:0/3001404720 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f40780d12e0 con 0x7f4078160d00 2023-07-06T03:49:06.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.752+0000 7f407e7ff700 1 -- 172.21.15.97:0/3001404720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f407804e340 con 0x7f4078160d00 2023-07-06T03:49:06.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.753+0000 7f4074ff9700 1 -- 172.21.15.97:0/3001404720 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18703 (secure 0 0 0) 0x7f4070058790 con 0x7f4078160d00 2023-07-06T03:49:06.756 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:06.757 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":62,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:48:53.298133+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:48:43.594325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-06T03:48:44.371987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.3","mappings":[{"from":2,"to":5}]},{"pgid":"10.7","mappings":[{"from":4,"to":1}]},{"pgid":"10.16","mappings":[{"from":2,"to":1}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.1f","mappings":[{"from":2,"to":5}]},{"pgid":"10.22","mappings":[{"from":2,"to":7}]},{"pgid":"10.2d","mappings":[{"from":2,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:49:06.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 -- 172.21.15.97:0/3001404720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f406406ace0 msgr2=0x7f406406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f406406ace0 0x7f406406d190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4068009c40 tx=0x7f4068008030 comp rx=0 tx=0).stop 2023-07-06T03:49:06.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 -- 172.21.15.97:0/3001404720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4078160d00 msgr2=0x7f40781e6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4078160d00 0x7f40781e6a80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4070007e60 tx=0x7f407000e3c0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 -- 172.21.15.97:0/3001404720 shutdown_connections 2023-07-06T03:49:06.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f406406ace0 0x7f406406d190 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4078160d00 0x7f40781e6a80 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f407815c520 0x7f40780cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 --2- 172.21.15.97:0/3001404720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4078160180 0x7f40781e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 -- 172.21.15.97:0/3001404720 >> 172.21.15.97:0/3001404720 conn(0x7f4078071790 msgr2=0x7f4078071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:06.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 -- 172.21.15.97:0/3001404720 shutdown_connections 2023-07-06T03:49:06.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.756+0000 7f407e7ff700 1 -- 172.21.15.97:0/3001404720 wait complete. 2023-07-06T03:49:06.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:48:43.594325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:48:44.371987+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:49:06.778 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:49:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.974+0000 7f17d3fff700 1 Processor -- start 2023-07-06T03:49:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.974+0000 7f17d3fff700 1 -- start start 2023-07-06T03:49:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.976+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17cc06cab0 0x7f17cc0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.976+0000 7f17d3fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f17cc0b0560 0x7f17cc0b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.976+0000 7f17d3fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17cc0ce560 con 0x7f17cc158b20 2023-07-06T03:49:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.976+0000 7f17d3fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc0ce820 con 0x7f17cc0b0560 2023-07-06T03:49:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.976+0000 7f17d3fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0b0f20 con 0x7f17cc06cab0 2023-07-06T03:49:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.976+0000 7f17d1d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f17cc158b20 0x7f17cc0cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41828/0 (socket says 172.21.15.97:41828) 2023-07-06T03:49:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.976+0000 7f17d1d9b700 1 -- 172.21.15.97:0/2135901074 learned_addr learned my addr 172.21.15.97:0/2135901074 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:06.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.976+0000 7f17d259c700 1 --2- 172.21.15.97:0/2135901074 >> v2:172.21.15.146:3300/0 conn(0x7f17cc0b0560 0x7f17cc0b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.976+0000 7f17d159a700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17cc06cab0 0x7f17cc0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.976+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069760638 0 0) 0x7f17cc0ce560 con 0x7f17cc158b20 2023-07-06T03:49:06.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.976+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17b80035b0 con 0x7f17cc158b20 2023-07-06T03:49:06.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2450974678 0 0) 0x7f17b80035b0 con 0x7f17cc158b20 2023-07-06T03:49:06.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17cc06cab0 msgr2=0x7f17cc0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:06.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17cc06cab0 0x7f17cc0ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 >> v2:172.21.15.146:3300/0 conn(0x7f17cc0b0560 msgr2=0x7f17cc0b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:06.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 --2- 172.21.15.97:0/2135901074 >> v2:172.21.15.146:3300/0 conn(0x7f17cc0b0560 0x7f17cc0b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc0adcf0 con 0x7f17cc158b20 2023-07-06T03:49:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f17c8001670 con 0x7f17cc158b20 2023-07-06T03:49:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d159a700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17cc06cab0 0x7f17cc0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 >> v1:172.21.15.97:6789/0 conn(0x7f17cc158b20 legacy=0x7f17cc0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17cc06cab0 0x7f17b8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17b8003f40 0x7f17b8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17b8004d90 0x7f17b8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0ce820 con 0x7f17b8003f40 2023-07-06T03:49:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0b0f20 con 0x7f17b8004d90 2023-07-06T03:49:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b800ba10 con 0x7f17cc06cab0 2023-07-06T03:49:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d259c700 1 --2- 172.21.15.97:0/2135901074 >> v2:172.21.15.146:3300/0 conn(0x7f17cc0b0560 0x7f17cc0b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d159a700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17b8003f40 0x7f17b8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d1d9b700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17cc06cab0 0x7f17b8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d259c700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17b8004d90 0x7f17b8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d159a700 1 -- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17cc06cab0 msgr2=0x7f17b8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d159a700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17cc06cab0 0x7f17b8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d159a700 1 -- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17b8004d90 msgr2=0x7f17b8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d159a700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17b8004d90 0x7f17b8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.977+0000 7f17d159a700 1 -- 172.21.15.97:0/2135901074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc0094a0 con 0x7f17b8003f40 2023-07-06T03:49:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.978+0000 7f17d159a700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17b8003f40 0x7f17b8004850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009d30 tx=0x7f17b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f2a91478abadd3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.978+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc009e80 con 0x7f17b8003f40 2023-07-06T03:49:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.978+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc00e9a0 con 0x7f17b8003f40 2023-07-06T03:49:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.978+0000 7f17d0d99700 1 -- 172.21.15.97:0/2135901074 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00ed10 con 0x7f17b8003f40 2023-07-06T03:49:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.978+0000 7f17d3fff700 1 -- 172.21.15.97:0/2135901074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17b8003f40 msgr2=0x7f17b8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.978+0000 7f17d3fff700 1 --2- 172.21.15.97:0/2135901074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17b8003f40 0x7f17b8004850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009d30 tx=0x7f17b8005880 comp rx=0 tx=0).stop 2023-07-06T03:49:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.978+0000 7f17d259c700 1 -- 172.21.15.97:0/2135901074 reap_dead start 2023-07-06T03:49:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.978+0000 7f17d3fff700 1 -- 172.21.15.97:0/2135901074 shutdown_connections 2023-07-06T03:49:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.978+0000 7f17d3fff700 1 -- 172.21.15.97:0/2135901074 >> 172.21.15.97:0/2135901074 conn(0x7f17cc06a060 msgr2=0x7f17cc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.979+0000 7f17d3fff700 1 -- 172.21.15.97:0/2135901074 shutdown_connections 2023-07-06T03:49:06.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.979+0000 7f17d3fff700 1 -- 172.21.15.97:0/2135901074 wait complete. 2023-07-06T03:49:06.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.981+0000 7f17d3fff700 1 Processor -- start 2023-07-06T03:49:06.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.981+0000 7f17d3fff700 1 -- start start 2023-07-06T03:49:06.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.982+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17b8003f40 0x7f17cc13f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.982+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc15b860 0x7f17cc1e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.982+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17cc1e1d90 0x7f17cc1e2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.982+0000 7f17d3fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc1e2740 con 0x7f17cc15b860 2023-07-06T03:49:06.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.982+0000 7f17d3fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc15b3d0 con 0x7f17cc1e1d90 2023-07-06T03:49:06.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.982+0000 7f17d3fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc14fa70 con 0x7f17b8003f40 2023-07-06T03:49:06.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.983+0000 7f17d159a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc15b860 0x7f17cc1e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.983+0000 7f17d159a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc15b860 0x7f17cc1e0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52544/0 (socket says 172.21.15.97:52544) 2023-07-06T03:49:06.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.983+0000 7f17d159a700 1 -- 172.21.15.97:0/1106234851 learned_addr learned my addr 172.21.15.97:0/1106234851 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:06.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.983+0000 7f17d1d9b700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17b8003f40 0x7f17cc13f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.983+0000 7f17d259c700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17cc1e1d90 0x7f17cc1e2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.983+0000 7f17d159a700 1 -- 172.21.15.97:0/1106234851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17b8003f40 msgr2=0x7f17cc13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.983+0000 7f17d159a700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17b8003f40 0x7f17cc13f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.983+0000 7f17d159a700 1 -- 172.21.15.97:0/1106234851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17cc1e1d90 msgr2=0x7f17cc1e2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.983+0000 7f17d159a700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17cc1e1d90 0x7f17cc1e2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.983+0000 7f17d159a700 1 -- 172.21.15.97:0/1106234851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc15a4e0 con 0x7f17cc15b860 2023-07-06T03:49:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.983+0000 7f17d159a700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc15b860 0x7f17cc1e0fc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f17b8004720 tx=0x7f17bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.984+0000 7f17c27fc700 1 -- 172.21.15.97:0/1106234851 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc003ab0 con 0x7f17cc15b860 2023-07-06T03:49:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.984+0000 7f17c27fc700 1 -- 172.21.15.97:0/1106234851 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc003c10 con 0x7f17cc15b860 2023-07-06T03:49:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.984+0000 7f17d3fff700 1 -- 172.21.15.97:0/1106234851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc15a7c0 con 0x7f17cc15b860 2023-07-06T03:49:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.984+0000 7f17c27fc700 1 -- 172.21.15.97:0/1106234851 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc009e80 con 0x7f17cc15b860 2023-07-06T03:49:06.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.984+0000 7f17d3fff700 1 -- 172.21.15.97:0/1106234851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc15ad80 con 0x7f17cc15b860 2023-07-06T03:49:06.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.985+0000 7f17d3fff700 1 -- 172.21.15.97:0/1106234851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc15b540 con 0x7f17cc15b860 2023-07-06T03:49:06.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.986+0000 7f17c27fc700 1 -- 172.21.15.97:0/1106234851 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f17bc013470 con 0x7f17cc15b860 2023-07-06T03:49:06.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.987+0000 7f17c27fc700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f17b006ace0 0x7f17b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:06.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.988+0000 7f17c27fc700 1 -- 172.21.15.97:0/1106234851 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f17bc08ad70 con 0x7f17cc15b860 2023-07-06T03:49:06.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.988+0000 7f17d1d9b700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f17b006ace0 0x7f17b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:06.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.988+0000 7f17d1d9b700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f17b006ace0 0x7f17b006d190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f17c8000c00 tx=0x7f17c8007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:06.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:06.992+0000 7f17c27fc700 1 -- 172.21.15.97:0/1106234851 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f17bc0bd310 con 0x7f17cc15b860 2023-07-06T03:49:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.187+0000 7f17d3fff700 1 -- 172.21.15.97:0/1106234851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17cc04e340 con 0x7f17cc15b860 2023-07-06T03:49:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.187+0000 7f17c27fc700 1 -- 172.21.15.97:0/1106234851 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17bc055e60 con 0x7f17cc15b860 2023-07-06T03:49:07.190 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:49:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.189+0000 7f17d3fff700 1 -- 172.21.15.97:0/1106234851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f17b006ace0 msgr2=0x7f17b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.189+0000 7f17d3fff700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f17b006ace0 0x7f17b006d190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f17c8000c00 tx=0x7f17c8007980 comp rx=0 tx=0).stop 2023-07-06T03:49:07.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.189+0000 7f17d3fff700 1 -- 172.21.15.97:0/1106234851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc15b860 msgr2=0x7f17cc1e0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.189+0000 7f17d3fff700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc15b860 0x7f17cc1e0fc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f17b8004720 tx=0x7f17bc0047c0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.190+0000 7f17d3fff700 1 -- 172.21.15.97:0/1106234851 shutdown_connections 2023-07-06T03:49:07.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.190+0000 7f17d3fff700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f17b006ace0 0x7f17b006d190 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.190+0000 7f17d3fff700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17cc1e1d90 0x7f17cc1e2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.190+0000 7f17d3fff700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f17b8003f40 0x7f17cc13f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.190+0000 7f17d3fff700 1 --2- 172.21.15.97:0/1106234851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17cc15b860 0x7f17cc1e0fc0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.190+0000 7f17d3fff700 1 -- 172.21.15.97:0/1106234851 >> 172.21.15.97:0/1106234851 conn(0x7f17cc06a060 msgr2=0x7f17cc0cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:07.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.190+0000 7f17d3fff700 1 -- 172.21.15.97:0/1106234851 shutdown_connections 2023-07-06T03:49:07.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.190+0000 7f17d3fff700 1 -- 172.21.15.97:0/1106234851 wait complete. 2023-07-06T03:49:07.210 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:49:07.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.405+0000 7f4dfcbff700 1 Processor -- start 2023-07-06T03:49:07.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.405+0000 7f4dfcbff700 1 -- start start 2023-07-06T03:49:07.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.406+0000 7f4dfcbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df80d2210 0x7f4df80b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:07.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.406+0000 7f4dfcbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df80cff90 con 0x7f4df806b360 2023-07-06T03:49:07.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.406+0000 7f4dfcbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df80b5650 con 0x7f4df815c610 2023-07-06T03:49:07.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.406+0000 7f4dfcbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80d0780 con 0x7f4df80d2210 2023-07-06T03:49:07.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.407+0000 7f4df5d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4df806b360 0x7f4df80d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41842/0 (socket says 172.21.15.97:41842) 2023-07-06T03:49:07.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.407+0000 7f4df5d9b700 1 -- 172.21.15.97:0/3011457307 learned_addr learned my addr 172.21.15.97:0/3011457307 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.407+0000 7f4df6d9d700 1 --2- 172.21.15.97:0/3011457307 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df80d2210 0x7f4df80b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.407+0000 7f4df559a700 1 -- 172.21.15.97:0/3011457307 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067642245 0 0) 0x7f4df80cff90 con 0x7f4df806b360 2023-07-06T03:49:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.407+0000 7f4df559a700 1 -- 172.21.15.97:0/3011457307 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4de00035b0 con 0x7f4df806b360 2023-07-06T03:49:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.407+0000 7f4df559a700 1 -- 172.21.15.97:0/3011457307 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146623160 0 0) 0x7f4df80b5650 con 0x7f4df815c610 2023-07-06T03:49:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.407+0000 7f4df559a700 1 -- 172.21.15.97:0/3011457307 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4df80cff90 con 0x7f4df815c610 2023-07-06T03:49:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.407+0000 7f4df6d9d700 1 -- 172.21.15.97:0/3011457307 >> v1:172.21.15.146:6789/0 conn(0x7f4df815c610 legacy=0x7f4df80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.407+0000 7f4df6d9d700 1 -- 172.21.15.97:0/3011457307 >> v1:172.21.15.97:6789/0 conn(0x7f4df806b360 legacy=0x7f4df80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.407+0000 7f4df6d9d700 1 -- 172.21.15.97:0/3011457307 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80b5400 con 0x7f4df80d2210 2023-07-06T03:49:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.408+0000 7f4df6d9d700 1 --2- 172.21.15.97:0/3011457307 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df80d2210 0x7f4df80b4ec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009dc0 tx=0x7f4df000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a4ab047f1cc7b7e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:07.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.408+0000 7f4df559a700 1 -- 172.21.15.97:0/3011457307 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df000e070 con 0x7f4df80d2210 2023-07-06T03:49:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.408+0000 7f4df559a700 1 -- 172.21.15.97:0/3011457307 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df000bb20 con 0x7f4df80d2210 2023-07-06T03:49:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.408+0000 7f4df559a700 1 -- 172.21.15.97:0/3011457307 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df000be90 con 0x7f4df80d2210 2023-07-06T03:49:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.408+0000 7f4dfcbff700 1 -- 172.21.15.97:0/3011457307 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df80d2210 msgr2=0x7f4df80b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.408+0000 7f4dfcbff700 1 --2- 172.21.15.97:0/3011457307 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df80d2210 0x7f4df80b4ec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009dc0 tx=0x7f4df000b330 comp rx=0 tx=0).stop 2023-07-06T03:49:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.409+0000 7f4dfcbff700 1 -- 172.21.15.97:0/3011457307 shutdown_connections 2023-07-06T03:49:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.409+0000 7f4dfcbff700 1 --2- 172.21.15.97:0/3011457307 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df80d2210 0x7f4df80b4ec0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.409+0000 7f4dfcbff700 1 -- 172.21.15.97:0/3011457307 >> 172.21.15.97:0/3011457307 conn(0x7f4df806a6c0 msgr2=0x7f4df806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.409+0000 7f4dfcbff700 1 -- 172.21.15.97:0/3011457307 shutdown_connections 2023-07-06T03:49:07.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.409+0000 7f4dfcbff700 1 -- 172.21.15.97:0/3011457307 wait complete. 2023-07-06T03:49:07.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.410+0000 7f4dfcbff700 1 Processor -- start 2023-07-06T03:49:07.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.411+0000 7f4dfcbff700 1 -- start start 2023-07-06T03:49:07.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.411+0000 7f4dfcbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df806b360 0x7f4df80c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:07.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.411+0000 7f4dfcbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df80d2210 0x7f4df814b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:07.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.411+0000 7f4dfcbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4df815c610 0x7f4df815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:07.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4dfcbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80c7360 con 0x7f4df806b360 2023-07-06T03:49:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4dfcbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80c9e00 con 0x7f4df815c610 2023-07-06T03:49:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4dfcbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80ca050 con 0x7f4df80d2210 2023-07-06T03:49:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4df659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df806b360 0x7f4df80c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4df659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df806b360 0x7f4df80c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52546/0 (socket says 172.21.15.97:52546) 2023-07-06T03:49:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4df659c700 1 -- 172.21.15.97:0/4229568472 learned_addr learned my addr 172.21.15.97:0/4229568472 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4df659c700 1 -- 172.21.15.97:0/4229568472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df80d2210 msgr2=0x7f4df814b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4df659c700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df80d2210 0x7f4df814b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4df659c700 1 -- 172.21.15.97:0/4229568472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4df815c610 msgr2=0x7f4df815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4df659c700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4df815c610 0x7f4df815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4df659c700 1 -- 172.21.15.97:0/4229568472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df0009a70 con 0x7f4df806b360 2023-07-06T03:49:07.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4df659c700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df806b360 0x7f4df80c6e20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c520 tx=0x7f4de800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:07.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.412+0000 7f4de6ffd700 1 -- 172.21.15.97:0/4229568472 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80052d0 con 0x7f4df806b360 2023-07-06T03:49:07.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.413+0000 7f4de6ffd700 1 -- 172.21.15.97:0/4229568472 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de8005430 con 0x7f4df806b360 2023-07-06T03:49:07.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.413+0000 7f4dfcbff700 1 -- 172.21.15.97:0/4229568472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df80ca330 con 0x7f4df806b360 2023-07-06T03:49:07.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.413+0000 7f4de6ffd700 1 -- 172.21.15.97:0/4229568472 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80038c0 con 0x7f4df806b360 2023-07-06T03:49:07.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.413+0000 7f4dfcbff700 1 -- 172.21.15.97:0/4229568472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df80ca8f0 con 0x7f4df806b360 2023-07-06T03:49:07.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.414+0000 7f4dfcbff700 1 -- 172.21.15.97:0/4229568472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df80ce600 con 0x7f4df806b360 2023-07-06T03:49:07.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.415+0000 7f4de6ffd700 1 -- 172.21.15.97:0/4229568472 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4de8003a20 con 0x7f4df806b360 2023-07-06T03:49:07.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.415+0000 7f4de6ffd700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd806ace0 0x7f4dd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:07.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.416+0000 7f4de6ffd700 1 -- 172.21.15.97:0/4229568472 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f4de8088b10 con 0x7f4df806b360 2023-07-06T03:49:07.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.416+0000 7f4df5d9b700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd806ace0 0x7f4dd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:07.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.416+0000 7f4df5d9b700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd806ace0 0x7f4dd806d190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4dec001610 tx=0x7f4dec0079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:07.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.420+0000 7f4de6ffd700 1 -- 172.21.15.97:0/4229568472 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4de8053ae0 con 0x7f4df806b360 2023-07-06T03:49:07.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.614+0000 7f4dfcbff700 1 -- 172.21.15.97:0/4229568472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4df804e340 con 0x7f4df806b360 2023-07-06T03:49:07.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.614+0000 7f4de6ffd700 1 -- 172.21.15.97:0/4229568472 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4de8017020 con 0x7f4df806b360 2023-07-06T03:49:07.617 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:49:07.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 -- 172.21.15.97:0/4229568472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd806ace0 msgr2=0x7f4dd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd806ace0 0x7f4dd806d190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4dec001610 tx=0x7f4dec0079c0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 -- 172.21.15.97:0/4229568472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df806b360 msgr2=0x7f4df80c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df806b360 0x7f4df80c6e20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c520 tx=0x7f4de800c8e0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 -- 172.21.15.97:0/4229568472 shutdown_connections 2023-07-06T03:49:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd806ace0 0x7f4dd806d190 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4df815c610 0x7f4df815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df806b360 0x7f4df80c6e20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 --2- 172.21.15.97:0/4229568472 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df80d2210 0x7f4df814b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 -- 172.21.15.97:0/4229568472 >> 172.21.15.97:0/4229568472 conn(0x7f4df806a6c0 msgr2=0x7f4df815a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 -- 172.21.15.97:0/4229568472 shutdown_connections 2023-07-06T03:49:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.617+0000 7f4dfcbff700 1 -- 172.21.15.97:0/4229568472 wait complete. 2023-07-06T03:49:07.637 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:49:07.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.833+0000 7f838d5ff700 1 Processor -- start 2023-07-06T03:49:07.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.834+0000 7f838d5ff700 1 -- start start 2023-07-06T03:49:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.835+0000 7f838d5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f838815c610 0x7f83880b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:07.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.835+0000 7f838d5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f838806b360 0x7f83880b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:07.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.835+0000 7f838d5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83880cff90 con 0x7f83880d2210 2023-07-06T03:49:07.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.835+0000 7f838d5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83880d0780 con 0x7f838806b360 2023-07-06T03:49:07.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.835+0000 7f838d5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838815c9e0 con 0x7f838815c610 2023-07-06T03:49:07.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.835+0000 7f83877fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f83880d2210 0x7f83880d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41852/0 (socket says 172.21.15.97:41852) 2023-07-06T03:49:07.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.835+0000 7f83877fe700 1 -- 172.21.15.97:0/3985627680 learned_addr learned my addr 172.21.15.97:0/3985627680 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:07.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.835+0000 7f8386ffd700 1 --2- 172.21.15.97:0/3985627680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f838815c610 0x7f83880b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:07.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.835+0000 7f837ffff700 1 --2- 172.21.15.97:0/3985627680 >> v2:172.21.15.146:3300/0 conn(0x7f838806b360 0x7f83880b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:07.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.836+0000 7f83867fc700 1 -- 172.21.15.97:0/3985627680 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840101077 0 0) 0x7f83880cff90 con 0x7f83880d2210 2023-07-06T03:49:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.836+0000 7f83867fc700 1 -- 172.21.15.97:0/3985627680 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83740035b0 con 0x7f83880d2210 2023-07-06T03:49:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.836+0000 7f837ffff700 1 -- 172.21.15.97:0/3985627680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f838815c610 msgr2=0x7f83880b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.836+0000 7f837ffff700 1 --2- 172.21.15.97:0/3985627680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f838815c610 0x7f83880b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.836+0000 7f837ffff700 1 -- 172.21.15.97:0/3985627680 >> v1:172.21.15.97:6789/0 conn(0x7f83880d2210 legacy=0x7f83880d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.836+0000 7f837ffff700 1 -- 172.21.15.97:0/3985627680 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83880b5650 con 0x7f838806b360 2023-07-06T03:49:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.836+0000 7f8386ffd700 1 --2- 172.21.15.97:0/3985627680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f838815c610 0x7f83880b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.836+0000 7f837ffff700 1 --2- 172.21.15.97:0/3985627680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f838806b360 0x7f83880b4cf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8370009870 tx=0x7f8370009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cc5dea79224c0eb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.836+0000 7f83867fc700 1 -- 172.21.15.97:0/3985627680 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370011830 con 0x7f838806b360 2023-07-06T03:49:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.836+0000 7f83867fc700 1 -- 172.21.15.97:0/3985627680 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8370011990 con 0x7f838806b360 2023-07-06T03:49:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.836+0000 7f83867fc700 1 -- 172.21.15.97:0/3985627680 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370011d00 con 0x7f838806b360 2023-07-06T03:49:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.837+0000 7f838d5ff700 1 -- 172.21.15.97:0/3985627680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f838806b360 msgr2=0x7f83880b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.837+0000 7f838d5ff700 1 --2- 172.21.15.97:0/3985627680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f838806b360 0x7f83880b4cf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8370009870 tx=0x7f8370009b80 comp rx=0 tx=0).stop 2023-07-06T03:49:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.837+0000 7f838d5ff700 1 -- 172.21.15.97:0/3985627680 shutdown_connections 2023-07-06T03:49:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.837+0000 7f838d5ff700 1 --2- 172.21.15.97:0/3985627680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f838815c610 0x7f83880b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.837+0000 7f838d5ff700 1 --2- 172.21.15.97:0/3985627680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f838806b360 0x7f83880b4cf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.837+0000 7f838d5ff700 1 -- 172.21.15.97:0/3985627680 >> 172.21.15.97:0/3985627680 conn(0x7f838806a6c0 msgr2=0x7f838806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:07.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.837+0000 7f838d5ff700 1 -- 172.21.15.97:0/3985627680 shutdown_connections 2023-07-06T03:49:07.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.837+0000 7f838d5ff700 1 -- 172.21.15.97:0/3985627680 wait complete. 2023-07-06T03:49:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.839+0000 7f838d5ff700 1 Processor -- start 2023-07-06T03:49:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.839+0000 7f838d5ff700 1 -- start start 2023-07-06T03:49:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f838d5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f838806b360 0x7f83880c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f838d5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83880d2210 0x7f838815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f838d5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f838815c610 0x7f838814efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f838d5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838815fa00 con 0x7f838815c610 2023-07-06T03:49:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f838d5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838814f510 con 0x7f838806b360 2023-07-06T03:49:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f838d5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880c9110 con 0x7f83880d2210 2023-07-06T03:49:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f83877fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f838815c610 0x7f838814efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f83877fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f838815c610 0x7f838814efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52556/0 (socket says 172.21.15.97:52556) 2023-07-06T03:49:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f83877fe700 1 -- 172.21.15.97:0/2734651854 learned_addr learned my addr 172.21.15.97:0/2734651854 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f8386ffd700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f838806b360 0x7f83880c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f837ffff700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83880d2210 0x7f838815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f83877fe700 1 -- 172.21.15.97:0/2734651854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83880d2210 msgr2=0x7f838815f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.840+0000 7f83877fe700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83880d2210 0x7f838815f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.841+0000 7f83877fe700 1 -- 172.21.15.97:0/2734651854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f838806b360 msgr2=0x7f83880c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:07.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.841+0000 7f83877fe700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f838806b360 0x7f83880c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:07.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.841+0000 7f83877fe700 1 -- 172.21.15.97:0/2734651854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83780094a0 con 0x7f838815c610 2023-07-06T03:49:07.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.841+0000 7f8386ffd700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f838806b360 0x7f83880c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:07.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.841+0000 7f83877fe700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f838815c610 0x7f838814efd0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f838000cb40 tx=0x7f838000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:07.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.841+0000 7f837f7fe700 1 -- 172.21.15.97:0/2734651854 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838000ab50 con 0x7f838815c610 2023-07-06T03:49:07.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.841+0000 7f837f7fe700 1 -- 172.21.15.97:0/2734651854 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838000acb0 con 0x7f838815c610 2023-07-06T03:49:07.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.841+0000 7f838d5ff700 1 -- 172.21.15.97:0/2734651854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83700094a0 con 0x7f838815c610 2023-07-06T03:49:07.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.841+0000 7f837f7fe700 1 -- 172.21.15.97:0/2734651854 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380012620 con 0x7f838815c610 2023-07-06T03:49:07.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.841+0000 7f838d5ff700 1 -- 172.21.15.97:0/2734651854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83880c96d0 con 0x7f838815c610 2023-07-06T03:49:07.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.842+0000 7f838d5ff700 1 -- 172.21.15.97:0/2734651854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838804e340 con 0x7f838815c610 2023-07-06T03:49:07.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.843+0000 7f837f7fe700 1 -- 172.21.15.97:0/2734651854 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f838001b3b0 con 0x7f838815c610 2023-07-06T03:49:07.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.844+0000 7f837f7fe700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836806ace0 0x7f836806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:07.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.845+0000 7f837f7fe700 1 -- 172.21.15.97:0/2734651854 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f83800894a0 con 0x7f838815c610 2023-07-06T03:49:07.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.845+0000 7f8386ffd700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836806ace0 0x7f836806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:07.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.845+0000 7f8386ffd700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836806ace0 0x7f836806d190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8378009d20 tx=0x7f8378008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:07.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:07.849+0000 7f837f7fe700 1 -- 172.21.15.97:0/2734651854 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8380054350 con 0x7f838815c610 2023-07-06T03:49:08.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.046+0000 7f838d5ff700 1 -- 172.21.15.97:0/2734651854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8388073830 con 0x7f838815c610 2023-07-06T03:49:08.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.047+0000 7f837f7fe700 1 -- 172.21.15.97:0/2734651854 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8380057970 con 0x7f838815c610 2023-07-06T03:49:08.050 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:49:08.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.049+0000 7f838d5ff700 1 -- 172.21.15.97:0/2734651854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836806ace0 msgr2=0x7f836806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.049+0000 7f838d5ff700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836806ace0 0x7f836806d190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8378009d20 tx=0x7f8378008040 comp rx=0 tx=0).stop 2023-07-06T03:49:08.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.049+0000 7f838d5ff700 1 -- 172.21.15.97:0/2734651854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f838815c610 msgr2=0x7f838814efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.049+0000 7f838d5ff700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f838815c610 0x7f838814efd0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f838000cb40 tx=0x7f838000cf00 comp rx=0 tx=0).stop 2023-07-06T03:49:08.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.050+0000 7f838d5ff700 1 -- 172.21.15.97:0/2734651854 shutdown_connections 2023-07-06T03:49:08.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.050+0000 7f838d5ff700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f836806ace0 0x7f836806d190 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.050+0000 7f838d5ff700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f838815c610 0x7f838814efd0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.050+0000 7f838d5ff700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f838806b360 0x7f83880c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.050+0000 7f838d5ff700 1 --2- 172.21.15.97:0/2734651854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83880d2210 0x7f838815f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.050+0000 7f838d5ff700 1 -- 172.21.15.97:0/2734651854 >> 172.21.15.97:0/2734651854 conn(0x7f838806a6c0 msgr2=0x7f83880b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.050+0000 7f838d5ff700 1 -- 172.21.15.97:0/2734651854 shutdown_connections 2023-07-06T03:49:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.050+0000 7f838d5ff700 1 -- 172.21.15.97:0/2734651854 wait complete. 2023-07-06T03:49:08.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:49:08.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.264+0000 7f30ea1ff700 1 Processor -- start 2023-07-06T03:49:08.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.264+0000 7f30ea1ff700 1 -- start start 2023-07-06T03:49:08.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30ea1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30e415c610 0x7f30e40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30ea1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30e406b360 0x7f30e40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30ea1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e40d2210 0x7f30e40d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30ea1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e40d0780 con 0x7f30e406b360 2023-07-06T03:49:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30ea1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e415c9e0 con 0x7f30e415c610 2023-07-06T03:49:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30ea1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40b5650 con 0x7f30e40d2210 2023-07-06T03:49:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30e2ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30e406b360 0x7f30e40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30e2ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30e406b360 0x7f30e40b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52564/0 (socket says 172.21.15.97:52564) 2023-07-06T03:49:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30e2ffd700 1 -- 172.21.15.97:0/819242391 learned_addr learned my addr 172.21.15.97:0/819242391 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30e2ffd700 1 -- 172.21.15.97:0/819242391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e40d2210 msgr2=0x7f30e40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30e2ffd700 1 --2- 172.21.15.97:0/819242391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e40d2210 0x7f30e40d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30e2ffd700 1 -- 172.21.15.97:0/819242391 >> v2:172.21.15.146:3300/0 conn(0x7f30e415c610 msgr2=0x7f30e40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30e2ffd700 1 --2- 172.21.15.97:0/819242391 >> v2:172.21.15.146:3300/0 conn(0x7f30e415c610 0x7f30e40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30e2ffd700 1 -- 172.21.15.97:0/819242391 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e40d01c0 con 0x7f30e406b360 2023-07-06T03:49:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.266+0000 7f30e2ffd700 1 --2- 172.21.15.97:0/819242391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e406b360 0x7f30e40b4cf0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f30e408b000 tx=0x7f30d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf0eb78fc26c03 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.267+0000 7f30e27fc700 1 -- 172.21.15.97:0/819242391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d800b860 con 0x7f30e406b360 2023-07-06T03:49:08.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.267+0000 7f30e27fc700 1 -- 172.21.15.97:0/819242391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d800b9c0 con 0x7f30e406b360 2023-07-06T03:49:08.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.267+0000 7f30e27fc700 1 -- 172.21.15.97:0/819242391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d800bd50 con 0x7f30e406b360 2023-07-06T03:49:08.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.267+0000 7f30ea1ff700 1 -- 172.21.15.97:0/819242391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e406b360 msgr2=0x7f30e40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.267+0000 7f30ea1ff700 1 --2- 172.21.15.97:0/819242391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e406b360 0x7f30e40b4cf0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f30e408b000 tx=0x7f30d8009c00 comp rx=0 tx=0).stop 2023-07-06T03:49:08.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.267+0000 7f30ea1ff700 1 -- 172.21.15.97:0/819242391 shutdown_connections 2023-07-06T03:49:08.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.267+0000 7f30ea1ff700 1 --2- 172.21.15.97:0/819242391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e40d2210 0x7f30e40d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.267+0000 7f30ea1ff700 1 --2- 172.21.15.97:0/819242391 >> v2:172.21.15.146:3300/0 conn(0x7f30e415c610 0x7f30e40b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.267+0000 7f30ea1ff700 1 --2- 172.21.15.97:0/819242391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e406b360 0x7f30e40b4cf0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.267+0000 7f30ea1ff700 1 -- 172.21.15.97:0/819242391 >> 172.21.15.97:0/819242391 conn(0x7f30e406a6c0 msgr2=0x7f30e406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:08.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.267+0000 7f30ea1ff700 1 -- 172.21.15.97:0/819242391 shutdown_connections 2023-07-06T03:49:08.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.268+0000 7f30ea1ff700 1 -- 172.21.15.97:0/819242391 wait complete. 2023-07-06T03:49:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.269+0000 7f30ea1ff700 1 Processor -- start 2023-07-06T03:49:08.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.269+0000 7f30ea1ff700 1 -- start start 2023-07-06T03:49:08.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.270+0000 7f30ea1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e406b360 0x7f30e40cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.270+0000 7f30ea1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e40d2210 0x7f30e4148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.270+0000 7f30ea1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e415c610 0x7f30e41f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.270+0000 7f30ea1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4159520 con 0x7f30e40d2210 2023-07-06T03:49:08.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.270+0000 7f30ea1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40cb350 con 0x7f30e415c610 2023-07-06T03:49:08.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.270+0000 7f30ea1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40cb630 con 0x7f30e406b360 2023-07-06T03:49:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.270+0000 7f30e2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e40d2210 0x7f30e4148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e40d2210 0x7f30e4148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52572/0 (socket says 172.21.15.97:52572) 2023-07-06T03:49:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e2ffd700 1 -- 172.21.15.97:0/1160861854 learned_addr learned my addr 172.21.15.97:0/1160861854 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e37fe700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e406b360 0x7f30e40cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e3fff700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e415c610 0x7f30e41f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e2ffd700 1 -- 172.21.15.97:0/1160861854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e406b360 msgr2=0x7f30e40cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e2ffd700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e406b360 0x7f30e40cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e2ffd700 1 -- 172.21.15.97:0/1160861854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e415c610 msgr2=0x7f30e41f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e2ffd700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e415c610 0x7f30e41f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e2ffd700 1 -- 172.21.15.97:0/1160861854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d8009550 con 0x7f30e40d2210 2023-07-06T03:49:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e37fe700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e406b360 0x7f30e40cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e3fff700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e415c610 0x7f30e41f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30e2ffd700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e40d2210 0x7f30e4148560 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f30d8000c00 tx=0x7f30d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30cbfff700 1 -- 172.21.15.97:0/1160861854 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8004480 con 0x7f30e40d2210 2023-07-06T03:49:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30cbfff700 1 -- 172.21.15.97:0/1160861854 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d801f070 con 0x7f30e40d2210 2023-07-06T03:49:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30cbfff700 1 -- 172.21.15.97:0/1160861854 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d80038e0 con 0x7f30e40d2210 2023-07-06T03:49:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.271+0000 7f30ea1ff700 1 -- 172.21.15.97:0/1160861854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e40cb910 con 0x7f30e40d2210 2023-07-06T03:49:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.272+0000 7f30ea1ff700 1 -- 172.21.15.97:0/1160861854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e40cbed0 con 0x7f30e40d2210 2023-07-06T03:49:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.273+0000 7f30ea1ff700 1 -- 172.21.15.97:0/1160861854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e404e340 con 0x7f30e40d2210 2023-07-06T03:49:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.273+0000 7f30cbfff700 1 -- 172.21.15.97:0/1160861854 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30d8021030 con 0x7f30e40d2210 2023-07-06T03:49:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.274+0000 7f30cbfff700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30d006ad10 0x7f30d006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.274+0000 7f30cbfff700 1 -- 172.21.15.97:0/1160861854 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f30d808ab10 con 0x7f30e40d2210 2023-07-06T03:49:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.275+0000 7f30e37fe700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30d006ad10 0x7f30d006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.275+0000 7f30e37fe700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30d006ad10 0x7f30d006d1c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f30d400a8f0 tx=0x7f30d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.279+0000 7f30cbfff700 1 -- 172.21.15.97:0/1160861854 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30d8055b00 con 0x7f30e40d2210 2023-07-06T03:49:08.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.476+0000 7f30ea1ff700 1 -- 172.21.15.97:0/1160861854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f30e40b8a40 con 0x7f30e40d2210 2023-07-06T03:49:08.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.476+0000 7f30cbfff700 1 -- 172.21.15.97:0/1160861854 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7f30d8059120 con 0x7f30e40d2210 2023-07-06T03:49:08.479 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:49:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.478+0000 7f30ea1ff700 1 -- 172.21.15.97:0/1160861854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30d006ad10 msgr2=0x7f30d006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.478+0000 7f30ea1ff700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30d006ad10 0x7f30d006d1c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f30d400a8f0 tx=0x7f30d4008040 comp rx=0 tx=0).stop 2023-07-06T03:49:08.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.478+0000 7f30ea1ff700 1 -- 172.21.15.97:0/1160861854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e40d2210 msgr2=0x7f30e4148560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.478+0000 7f30ea1ff700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e40d2210 0x7f30e4148560 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f30d8000c00 tx=0x7f30d8004750 comp rx=0 tx=0).stop 2023-07-06T03:49:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.478+0000 7f30ea1ff700 1 -- 172.21.15.97:0/1160861854 shutdown_connections 2023-07-06T03:49:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.478+0000 7f30ea1ff700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30d006ad10 0x7f30d006d1c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.478+0000 7f30ea1ff700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e415c610 0x7f30e41f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.478+0000 7f30ea1ff700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e406b360 0x7f30e40cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.478+0000 7f30ea1ff700 1 --2- 172.21.15.97:0/1160861854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e40d2210 0x7f30e4148560 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.478+0000 7f30ea1ff700 1 -- 172.21.15.97:0/1160861854 >> 172.21.15.97:0/1160861854 conn(0x7f30e406a6c0 msgr2=0x7f30e415b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:08.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.479+0000 7f30ea1ff700 1 -- 172.21.15.97:0/1160861854 shutdown_connections 2023-07-06T03:49:08.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.479+0000 7f30ea1ff700 1 -- 172.21.15.97:0/1160861854 wait complete. 2023-07-06T03:49:08.499 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:49:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.693+0000 7f41187ff700 1 Processor -- start 2023-07-06T03:49:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.693+0000 7f41187ff700 1 -- start start 2023-07-06T03:49:08.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.694+0000 7f41187ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41100ccbd0 0x7f41101640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.694+0000 7f41187ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41100ca260 con 0x7f41100c3cd0 2023-07-06T03:49:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.694+0000 7f41187ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41100c99c0 con 0x7f4110159070 2023-07-06T03:49:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.694+0000 7f41187ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41100c9ca0 con 0x7f41100ccbd0 2023-07-06T03:49:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.694+0000 7f4115d9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f41100c3cd0 0x7f41100cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41860/0 (socket says 172.21.15.97:41860) 2023-07-06T03:49:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.694+0000 7f4115d9a700 1 -- 172.21.15.97:0/538092464 learned_addr learned my addr 172.21.15.97:0/538092464 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.695+0000 7f4116d9c700 1 --2- 172.21.15.97:0/538092464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41100ccbd0 0x7f41101640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.695+0000 7f4115599700 1 -- 172.21.15.97:0/538092464 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795909246 0 0) 0x7f41100ca260 con 0x7f41100c3cd0 2023-07-06T03:49:08.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.695+0000 7f4115599700 1 -- 172.21.15.97:0/538092464 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40fc0035b0 con 0x7f41100c3cd0 2023-07-06T03:49:08.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.695+0000 7f4116d9c700 1 -- 172.21.15.97:0/538092464 >> v1:172.21.15.146:6789/0 conn(0x7f4110159070 legacy=0x7f41100c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.695+0000 7f4116d9c700 1 -- 172.21.15.97:0/538092464 >> v1:172.21.15.97:6789/0 conn(0x7f41100c3cd0 legacy=0x7f41100cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.695+0000 7f4116d9c700 1 -- 172.21.15.97:0/538092464 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41100c9f80 con 0x7f41100ccbd0 2023-07-06T03:49:08.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.696+0000 7f4116d9c700 1 --2- 172.21.15.97:0/538092464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41100ccbd0 0x7f41101640b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4108009dc0 tx=0x7f410800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f203742bb990d03 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.698+0000 7f4115599700 1 -- 172.21.15.97:0/538092464 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410800e070 con 0x7f41100ccbd0 2023-07-06T03:49:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.698+0000 7f4115599700 1 -- 172.21.15.97:0/538092464 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410800bb20 con 0x7f41100ccbd0 2023-07-06T03:49:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.698+0000 7f4115599700 1 -- 172.21.15.97:0/538092464 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108012430 con 0x7f41100ccbd0 2023-07-06T03:49:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.699+0000 7f41187ff700 1 -- 172.21.15.97:0/538092464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41100ccbd0 msgr2=0x7f41101640b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.699+0000 7f41187ff700 1 --2- 172.21.15.97:0/538092464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41100ccbd0 0x7f41101640b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4108009dc0 tx=0x7f410800b330 comp rx=0 tx=0).stop 2023-07-06T03:49:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.699+0000 7f41187ff700 1 -- 172.21.15.97:0/538092464 shutdown_connections 2023-07-06T03:49:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.699+0000 7f41187ff700 1 --2- 172.21.15.97:0/538092464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41100ccbd0 0x7f41101640b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.699+0000 7f41187ff700 1 -- 172.21.15.97:0/538092464 >> 172.21.15.97:0/538092464 conn(0x7f411006a060 msgr2=0x7f411006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.699+0000 7f41187ff700 1 -- 172.21.15.97:0/538092464 shutdown_connections 2023-07-06T03:49:08.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.699+0000 7f41187ff700 1 -- 172.21.15.97:0/538092464 wait complete. 2023-07-06T03:49:08.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.701+0000 7f41187ff700 1 Processor -- start 2023-07-06T03:49:08.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.701+0000 7f41187ff700 1 -- start start 2023-07-06T03:49:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f41187ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41100c3cd0 0x7f4110154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f41187ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41100ccbd0 0x7f41101519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f41187ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110159070 0x7f41100b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f41187ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4110151ef0 con 0x7f4110159070 2023-07-06T03:49:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f41187ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4110153b90 con 0x7f41100ccbd0 2023-07-06T03:49:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f41187ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4110153e70 con 0x7f41100c3cd0 2023-07-06T03:49:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f4116d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110159070 0x7f41100b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f4116d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110159070 0x7f41100b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52580/0 (socket says 172.21.15.97:52580) 2023-07-06T03:49:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f4116d9c700 1 -- 172.21.15.97:0/638574641 learned_addr learned my addr 172.21.15.97:0/638574641 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f4115d9a700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41100ccbd0 0x7f41101519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f411659b700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41100c3cd0 0x7f4110154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f4116d9c700 1 -- 172.21.15.97:0/638574641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41100c3cd0 msgr2=0x7f4110154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f4116d9c700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41100c3cd0 0x7f4110154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f4116d9c700 1 -- 172.21.15.97:0/638574641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41100ccbd0 msgr2=0x7f41101519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.702+0000 7f4116d9c700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41100ccbd0 0x7f41101519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.703+0000 7f4116d9c700 1 -- 172.21.15.97:0/638574641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4108009a70 con 0x7f4110159070 2023-07-06T03:49:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.703+0000 7f4116d9c700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110159070 0x7f41100b00e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4108014040 tx=0x7f4108004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.703+0000 7f4106ffd700 1 -- 172.21.15.97:0/638574641 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108012430 con 0x7f4110159070 2023-07-06T03:49:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.703+0000 7f4106ffd700 1 -- 172.21.15.97:0/638574641 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4108020070 con 0x7f4110159070 2023-07-06T03:49:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.703+0000 7f4106ffd700 1 -- 172.21.15.97:0/638574641 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108004280 con 0x7f4110159070 2023-07-06T03:49:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.703+0000 7f41187ff700 1 -- 172.21.15.97:0/638574641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4110154150 con 0x7f4110159070 2023-07-06T03:49:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.703+0000 7f41187ff700 1 -- 172.21.15.97:0/638574641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4110154710 con 0x7f4110159070 2023-07-06T03:49:08.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.704+0000 7f41187ff700 1 -- 172.21.15.97:0/638574641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41100cd1a0 con 0x7f4110159070 2023-07-06T03:49:08.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.705+0000 7f4106ffd700 1 -- 172.21.15.97:0/638574641 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41080043e0 con 0x7f4110159070 2023-07-06T03:49:08.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.706+0000 7f4106ffd700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40f406ace0 0x7f40f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:08.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.706+0000 7f4106ffd700 1 -- 172.21.15.97:0/638574641 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f4108022070 con 0x7f4110159070 2023-07-06T03:49:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.707+0000 7f411659b700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40f406ace0 0x7f40f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.707+0000 7f411659b700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40f406ace0 0x7f40f406d190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f410c007fd0 tx=0x7f410c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.711+0000 7f4106ffd700 1 -- 172.21.15.97:0/638574641 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f41080564e0 con 0x7f4110159070 2023-07-06T03:49:08.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.921+0000 7f41187ff700 1 -- 172.21.15.97:0/638574641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f411004e340 con 0x7f4110159070 2023-07-06T03:49:08.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.921+0000 7f4106ffd700 1 -- 172.21.15.97:0/638574641 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4108012e60 con 0x7f4110159070 2023-07-06T03:49:08.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 -- 172.21.15.97:0/638574641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40f406ace0 msgr2=0x7f40f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40f406ace0 0x7f40f406d190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f410c007fd0 tx=0x7f410c009040 comp rx=0 tx=0).stop 2023-07-06T03:49:08.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 -- 172.21.15.97:0/638574641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110159070 msgr2=0x7f41100b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:08.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110159070 0x7f41100b00e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4108014040 tx=0x7f4108004ad0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 -- 172.21.15.97:0/638574641 shutdown_connections 2023-07-06T03:49:08.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40f406ace0 0x7f40f406d190 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4110159070 0x7f41100b00e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41100c3cd0 0x7f4110154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 --2- 172.21.15.97:0/638574641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41100ccbd0 0x7f41101519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 -- 172.21.15.97:0/638574641 >> 172.21.15.97:0/638574641 conn(0x7f411006a060 msgr2=0x7f411006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 -- 172.21.15.97:0/638574641 shutdown_connections 2023-07-06T03:49:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:08.924+0000 7f41187ff700 1 -- 172.21.15.97:0/638574641 wait complete. 2023-07-06T03:49:08.929 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:49:08.945 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2023-07-06T03:49:08.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:08.946 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-06T03:49:09.019 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[122283]: fuse finished with error 0 and tester_r 0 2023-07-06T03:49:09.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:49:09.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:09.024 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T03:49:09.075 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:49:09.076 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:49:09.121 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:49:09.122 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:49:09.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:09.122 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T03:49:09.206 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T03:49:09.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:49:09.207 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:49:09.208 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:49:09.254 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:49:09.254 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:49:09.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:09.255 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T03:49:09.340 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T03:49:09.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:49:09.342 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:49:09.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.533+0000 7f9d18bff700 1 Processor -- start 2023-07-06T03:49:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.534+0000 7f9d18bff700 1 -- start start 2023-07-06T03:49:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.535+0000 7f9d18bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d14158f20 0x7f9d140caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.535+0000 7f9d18bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9d141592f0 0x7f9d140b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.535+0000 7f9d18bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d1406d860 0x7f9d140b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.535+0000 7f9d18bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1415cc40 con 0x7f9d1406d860 2023-07-06T03:49:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.535+0000 7f9d18bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1415cf20 con 0x7f9d141592f0 2023-07-06T03:49:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.535+0000 7f9d18bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1415d200 con 0x7f9d14158f20 2023-07-06T03:49:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.535+0000 7f9d12d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d1406d860 0x7f9d140b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.535+0000 7f9d12d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d1406d860 0x7f9d140b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52590/0 (socket says 172.21.15.97:52590) 2023-07-06T03:49:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.535+0000 7f9d12d9d700 1 -- 172.21.15.97:0/1245080927 learned_addr learned my addr 172.21.15.97:0/1245080927 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.535+0000 7f9d0bfff700 1 --2- 172.21.15.97:0/1245080927 >> v2:172.21.15.146:3300/0 conn(0x7f9d141592f0 0x7f9d140b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.535+0000 7f9d1259c700 1 --2- 172.21.15.97:0/1245080927 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d14158f20 0x7f9d140caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:09.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.536+0000 7f9d12d9d700 1 -- 172.21.15.97:0/1245080927 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d14158f20 msgr2=0x7f9d140caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:09.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.536+0000 7f9d12d9d700 1 --2- 172.21.15.97:0/1245080927 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d14158f20 0x7f9d140caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:09.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.536+0000 7f9d12d9d700 1 -- 172.21.15.97:0/1245080927 >> v2:172.21.15.146:3300/0 conn(0x7f9d141592f0 msgr2=0x7f9d140b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:09.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.536+0000 7f9d12d9d700 1 --2- 172.21.15.97:0/1245080927 >> v2:172.21.15.146:3300/0 conn(0x7f9d141592f0 0x7f9d140b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:09.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.536+0000 7f9d12d9d700 1 -- 172.21.15.97:0/1245080927 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1415d4e0 con 0x7f9d1406d860 2023-07-06T03:49:09.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.536+0000 7f9d12d9d700 1 --2- 172.21.15.97:0/1245080927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1406d860 0x7f9d140b0aa0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9d1406f660 tx=0x7f9d0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd893cc603c8b96f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:09.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.536+0000 7f9d11d9b700 1 -- 172.21.15.97:0/1245080927 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00e070 con 0x7f9d1406d860 2023-07-06T03:49:09.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.536+0000 7f9d11d9b700 1 -- 172.21.15.97:0/1245080927 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c00b910 con 0x7f9d1406d860 2023-07-06T03:49:09.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.536+0000 7f9d11d9b700 1 -- 172.21.15.97:0/1245080927 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00bcc0 con 0x7f9d1406d860 2023-07-06T03:49:09.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.537+0000 7f9d18bff700 1 -- 172.21.15.97:0/1245080927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1406d860 msgr2=0x7f9d140b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:09.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.537+0000 7f9d18bff700 1 --2- 172.21.15.97:0/1245080927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1406d860 0x7f9d140b0aa0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9d1406f660 tx=0x7f9d0c00b330 comp rx=0 tx=0).stop 2023-07-06T03:49:09.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.537+0000 7f9d18bff700 1 -- 172.21.15.97:0/1245080927 shutdown_connections 2023-07-06T03:49:09.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.537+0000 7f9d18bff700 1 --2- 172.21.15.97:0/1245080927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1406d860 0x7f9d140b0aa0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:09.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.537+0000 7f9d18bff700 1 --2- 172.21.15.97:0/1245080927 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d14158f20 0x7f9d140caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:09.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.537+0000 7f9d18bff700 1 --2- 172.21.15.97:0/1245080927 >> v2:172.21.15.146:3300/0 conn(0x7f9d141592f0 0x7f9d140b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:09.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.537+0000 7f9d18bff700 1 -- 172.21.15.97:0/1245080927 >> 172.21.15.97:0/1245080927 conn(0x7f9d140ce8c0 msgr2=0x7f9d140cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:09.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.537+0000 7f9d18bff700 1 -- 172.21.15.97:0/1245080927 shutdown_connections 2023-07-06T03:49:09.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.537+0000 7f9d18bff700 1 -- 172.21.15.97:0/1245080927 wait complete. 2023-07-06T03:49:09.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.539+0000 7f9d18bff700 1 Processor -- start 2023-07-06T03:49:09.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.539+0000 7f9d18bff700 1 -- start start 2023-07-06T03:49:09.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d18bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1406d860 0x7f9d1413add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:09.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d18bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14158f20 0x7f9d1414afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:09.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d18bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d141592f0 0x7f9d14144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:09.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d18bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d14149f10 con 0x7f9d1406d860 2023-07-06T03:49:09.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d18bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1414a1f0 con 0x7f9d14158f20 2023-07-06T03:49:09.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d18bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1414a4d0 con 0x7f9d141592f0 2023-07-06T03:49:09.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d1259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1406d860 0x7f9d1413add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:09.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d1259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1406d860 0x7f9d1413add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52598/0 (socket says 172.21.15.97:52598) 2023-07-06T03:49:09.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d1259c700 1 -- 172.21.15.97:0/3207299056 learned_addr learned my addr 172.21.15.97:0/3207299056 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:09.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d0bfff700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14158f20 0x7f9d1414afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:09.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d12d9d700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d141592f0 0x7f9d14144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:09.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d1259c700 1 -- 172.21.15.97:0/3207299056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d141592f0 msgr2=0x7f9d14144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:09.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d1259c700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d141592f0 0x7f9d14144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:09.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d1259c700 1 -- 172.21.15.97:0/3207299056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14158f20 msgr2=0x7f9d1414afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:09.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d1259c700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14158f20 0x7f9d1414afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:09.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.540+0000 7f9d1259c700 1 -- 172.21.15.97:0/3207299056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c009b30 con 0x7f9d1406d860 2023-07-06T03:49:09.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.541+0000 7f9d1259c700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1406d860 0x7f9d1413add0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400e9b0 tx=0x7f9d0400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:09.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.541+0000 7f9d0affd700 1 -- 172.21.15.97:0/3207299056 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04009710 con 0x7f9d1406d860 2023-07-06T03:49:09.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.541+0000 7f9d0affd700 1 -- 172.21.15.97:0/3207299056 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0400c850 con 0x7f9d1406d860 2023-07-06T03:49:09.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.541+0000 7f9d0affd700 1 -- 172.21.15.97:0/3207299056 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0400cbc0 con 0x7f9d1406d860 2023-07-06T03:49:09.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.541+0000 7f9d18bff700 1 -- 172.21.15.97:0/3207299056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1414a7b0 con 0x7f9d1406d860 2023-07-06T03:49:09.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.541+0000 7f9d18bff700 1 -- 172.21.15.97:0/3207299056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d14149ac0 con 0x7f9d1406d860 2023-07-06T03:49:09.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.542+0000 7f9d18bff700 1 -- 172.21.15.97:0/3207299056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1414b4f0 con 0x7f9d1406d860 2023-07-06T03:49:09.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.543+0000 7f9d0affd700 1 -- 172.21.15.97:0/3207299056 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d0400cd20 con 0x7f9d1406d860 2023-07-06T03:49:09.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.544+0000 7f9d0affd700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cf406ace0 0x7f9cf406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:09.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.544+0000 7f9d0affd700 1 -- 172.21.15.97:0/3207299056 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f9d04089760 con 0x7f9d1406d860 2023-07-06T03:49:09.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.544+0000 7f9d0bfff700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cf406ace0 0x7f9cf406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:09.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.545+0000 7f9d0bfff700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cf406ace0 0x7f9cf406d190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9d141497e0 tx=0x7f9cfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:09.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.548+0000 7f9d0affd700 1 -- 172.21.15.97:0/3207299056 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9d04058130 con 0x7f9d1406d860 2023-07-06T03:49:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.788+0000 7f9d18bff700 1 -- 172.21.15.97:0/3207299056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d1414acd0 con 0x7f9d1406d860 2023-07-06T03:49:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.789+0000 7f9d0affd700 1 -- 172.21.15.97:0/3207299056 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4678 (secure 0 0 0) 0x7f9d04057cc0 con 0x7f9d1406d860 2023-07-06T03:49:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:09.792 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1108873506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1108873506},{"type":"v1","addr":"172.21.15.146:6835","nonce":1108873506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2471899168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2471899168},{"type":"v1","addr":"172.21.15.97:6837","nonce":2471899168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2970792733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2970792733},{"type":"v1","addr":"172.21.15.97:6836","nonce":2970792733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:48:45.401033+0000","modified":"2023-07-06T03:48:48.930905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6399":{"gid":6399,"name":"d","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/2151424592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2151424592},{"type":"v1","addr":"172.21.15.146:6834","nonce":2151424592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-06T03:49:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 -- 172.21.15.97:0/3207299056 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cf406ace0 msgr2=0x7f9cf406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cf406ace0 0x7f9cf406d190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9d141497e0 tx=0x7f9cfc008040 comp rx=0 tx=0).stop 2023-07-06T03:49:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 -- 172.21.15.97:0/3207299056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1406d860 msgr2=0x7f9d1413add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1406d860 0x7f9d1413add0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400e9b0 tx=0x7f9d0400ed70 comp rx=0 tx=0).stop 2023-07-06T03:49:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 -- 172.21.15.97:0/3207299056 shutdown_connections 2023-07-06T03:49:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cf406ace0 0x7f9cf406d190 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d141592f0 0x7f9d14144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:09.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1406d860 0x7f9d1413add0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:09.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 --2- 172.21.15.97:0/3207299056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14158f20 0x7f9d1414afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:09.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 -- 172.21.15.97:0/3207299056 >> 172.21.15.97:0/3207299056 conn(0x7f9d140ce8c0 msgr2=0x7f9d140cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:09.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 -- 172.21.15.97:0/3207299056 shutdown_connections 2023-07-06T03:49:09.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:09.791+0000 7f9d18bff700 1 -- 172.21.15.97:0/3207299056 wait complete. 2023-07-06T03:49:09.796 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 34 2023-07-06T03:49:09.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:49:10.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.009+0000 7ff3927ff700 1 Processor -- start 2023-07-06T03:49:10.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.010+0000 7ff3927ff700 1 -- start start 2023-07-06T03:49:10.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.011+0000 7ff3927ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff38c15c520 0x7ff38c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.011+0000 7ff3927ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c0cdae0 con 0x7ff38c160180 2023-07-06T03:49:10.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.011+0000 7ff3927ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c0b6640 con 0x7ff38c160d00 2023-07-06T03:49:10.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.011+0000 7ff3927ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38c0cd8d0 con 0x7ff38c15c520 2023-07-06T03:49:10.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.011+0000 7ff38b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff38c160180 0x7ff38c1605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39868/0 (socket says 172.21.15.97:39868) 2023-07-06T03:49:10.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.011+0000 7ff38b7fe700 1 -- 172.21.15.97:0/1021527819 learned_addr learned my addr 172.21.15.97:0/1021527819 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.011+0000 7ff38bfff700 1 --2- 172.21.15.97:0/1021527819 >> v2:172.21.15.146:3300/0 conn(0x7ff38c15c520 0x7ff38c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.011+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042450521 0 0) 0x7ff38c0cdae0 con 0x7ff38c160180 2023-07-06T03:49:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.011+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3740035b0 con 0x7ff38c160180 2023-07-06T03:49:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2818799476 0 0) 0x7ff3740035b0 con 0x7ff38c160180 2023-07-06T03:49:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 >> v2:172.21.15.146:3300/0 conn(0x7ff38c15c520 msgr2=0x7ff38c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 --2- 172.21.15.97:0/1021527819 >> v2:172.21.15.146:3300/0 conn(0x7ff38c15c520 0x7ff38c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 >> v1:172.21.15.146:6789/0 conn(0x7ff38c160d00 legacy=0x7ff38c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0b6920 con 0x7ff38c160180 2023-07-06T03:49:10.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7ff380001670 con 0x7ff38c160180 2023-07-06T03:49:10.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38bfff700 1 --2- 172.21.15.97:0/1021527819 >> v2:172.21.15.146:3300/0 conn(0x7ff38c15c520 0x7ff38c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:10.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 >> v1:172.21.15.97:6789/0 conn(0x7ff38c160180 legacy=0x7ff38c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 0x7ff374003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3740043f0 0x7ff3740048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740050b0 0x7ff37400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0cd8d0 con 0x7ff374003a00 2023-07-06T03:49:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37400c000 con 0x7ff3740050b0 2023-07-06T03:49:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37400c1c0 con 0x7ff3740043f0 2023-07-06T03:49:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38bfff700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 0x7ff374003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff390d9c700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740050b0 0x7ff37400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38b7fe700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3740043f0 0x7ff3740048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.012+0000 7ff38bfff700 1 -- 172.21.15.97:0/1021527819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3740043f0 msgr2=0x7ff3740048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff38bfff700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3740043f0 0x7ff3740048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff38bfff700 1 -- 172.21.15.97:0/1021527819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740050b0 msgr2=0x7ff37400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff38bfff700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740050b0 0x7ff37400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff38bfff700 1 -- 172.21.15.97:0/1021527819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3740035b0 con 0x7ff374003a00 2023-07-06T03:49:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff38b7fe700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3740043f0 0x7ff3740048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff390d9c700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff3740050b0 0x7ff37400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff390d9c700 1 -- 172.21.15.97:0/1021527819 reap_dead start 2023-07-06T03:49:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff38bfff700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 0x7ff374003dd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009d30 tx=0x7ff374005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5fffd4a7b09bd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c009e80 con 0x7ff374003a00 2023-07-06T03:49:10.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c00e9a0 con 0x7ff374003a00 2023-07-06T03:49:10.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff38affd700 1 -- 172.21.15.97:0/1021527819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c00ed10 con 0x7ff374003a00 2023-07-06T03:49:10.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff3927ff700 1 -- 172.21.15.97:0/1021527819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 msgr2=0x7ff374003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.013+0000 7ff3927ff700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 0x7ff374003dd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009d30 tx=0x7ff374005880 comp rx=0 tx=0).stop 2023-07-06T03:49:10.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.014+0000 7ff3927ff700 1 -- 172.21.15.97:0/1021527819 shutdown_connections 2023-07-06T03:49:10.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.014+0000 7ff3927ff700 1 --2- 172.21.15.97:0/1021527819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 0x7ff374003dd0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.014+0000 7ff3927ff700 1 -- 172.21.15.97:0/1021527819 >> 172.21.15.97:0/1021527819 conn(0x7ff38c071790 msgr2=0x7ff38c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:10.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.014+0000 7ff3927ff700 1 -- 172.21.15.97:0/1021527819 shutdown_connections 2023-07-06T03:49:10.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.014+0000 7ff3927ff700 1 -- 172.21.15.97:0/1021527819 wait complete. 2023-07-06T03:49:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.015+0000 7ff3927ff700 1 Processor -- start 2023-07-06T03:49:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.016+0000 7ff3927ff700 1 -- start start 2023-07-06T03:49:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.016+0000 7ff3927ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 0x7ff38c1f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff3927ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff38c14d250 0x7ff38c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff3927ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff38c1585b0 0x7ff38c15dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff3927ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c1f5d00 con 0x7ff374003a00 2023-07-06T03:49:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff3927ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c1f56f0 con 0x7ff38c1585b0 2023-07-06T03:49:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff3927ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c14d6c0 con 0x7ff38c14d250 2023-07-06T03:49:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff38bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 0x7ff38c1f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff38bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 0x7ff38c1f6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33780/0 (socket says 172.21.15.97:33780) 2023-07-06T03:49:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff38bfff700 1 -- 172.21.15.97:0/3162394626 learned_addr learned my addr 172.21.15.97:0/3162394626 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff38b7fe700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff38c14d250 0x7ff38c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff390d9c700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff38c1585b0 0x7ff38c15dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff38bfff700 1 -- 172.21.15.97:0/3162394626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff38c14d250 msgr2=0x7ff38c158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff38bfff700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff38c14d250 0x7ff38c158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff38bfff700 1 -- 172.21.15.97:0/3162394626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff38c1585b0 msgr2=0x7ff38c15dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff38bfff700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff38c1585b0 0x7ff38c15dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff38bfff700 1 -- 172.21.15.97:0/3162394626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c158a50 con 0x7ff374003a00 2023-07-06T03:49:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff38b7fe700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff38c14d250 0x7ff38c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff390d9c700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff38c1585b0 0x7ff38c15dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.017+0000 7ff38bfff700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 0x7ff38c1f6190 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0090b0 tx=0x7ff37c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.018+0000 7ff388ff9700 1 -- 172.21.15.97:0/3162394626 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c009e80 con 0x7ff374003a00 2023-07-06T03:49:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.018+0000 7ff388ff9700 1 -- 172.21.15.97:0/3162394626 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c01c450 con 0x7ff374003a00 2023-07-06T03:49:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.018+0000 7ff3927ff700 1 -- 172.21.15.97:0/3162394626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c1f5990 con 0x7ff374003a00 2023-07-06T03:49:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.018+0000 7ff388ff9700 1 -- 172.21.15.97:0/3162394626 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c01c7e0 con 0x7ff374003a00 2023-07-06T03:49:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.018+0000 7ff3927ff700 1 -- 172.21.15.97:0/3162394626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c15bb80 con 0x7ff374003a00 2023-07-06T03:49:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.019+0000 7ff3927ff700 1 -- 172.21.15.97:0/3162394626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c1f5fe0 con 0x7ff374003a00 2023-07-06T03:49:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.020+0000 7ff388ff9700 1 -- 172.21.15.97:0/3162394626 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff37c01c940 con 0x7ff374003a00 2023-07-06T03:49:10.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.020+0000 7ff388ff9700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff37806ace0 0x7ff37806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.021+0000 7ff388ff9700 1 -- 172.21.15.97:0/3162394626 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7ff37c08ae40 con 0x7ff374003a00 2023-07-06T03:49:10.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.021+0000 7ff38b7fe700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff37806ace0 0x7ff37806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.021+0000 7ff38b7fe700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff37806ace0 0x7ff37806d190 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff380001cb0 tx=0x7ff380009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:10.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.025+0000 7ff388ff9700 1 -- 172.21.15.97:0/3162394626 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff37c055f30 con 0x7ff374003a00 2023-07-06T03:49:10.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.220+0000 7ff3927ff700 1 -- 172.21.15.97:0/3162394626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff38c04e340 con 0x7ff374003a00 2023-07-06T03:49:10.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.222+0000 7ff388ff9700 1 -- 172.21.15.97:0/3162394626 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18703 (secure 0 0 0) 0x7ff37c059550 con 0x7ff374003a00 2023-07-06T03:49:10.225 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:10.225 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":62,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:48:53.298133+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:48:43.594325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-06T03:48:44.371987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.3","mappings":[{"from":2,"to":5}]},{"pgid":"10.7","mappings":[{"from":4,"to":1}]},{"pgid":"10.16","mappings":[{"from":2,"to":1}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.1f","mappings":[{"from":2,"to":5}]},{"pgid":"10.22","mappings":[{"from":2,"to":7}]},{"pgid":"10.2d","mappings":[{"from":2,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:49:10.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.224+0000 7ff3927ff700 1 -- 172.21.15.97:0/3162394626 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff37806ace0 msgr2=0x7ff37806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.224+0000 7ff3927ff700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff37806ace0 0x7ff37806d190 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff380001cb0 tx=0x7ff380009040 comp rx=0 tx=0).stop 2023-07-06T03:49:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.224+0000 7ff3927ff700 1 -- 172.21.15.97:0/3162394626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 msgr2=0x7ff38c1f6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.224+0000 7ff3927ff700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 0x7ff38c1f6190 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0090b0 tx=0x7ff37c004750 comp rx=0 tx=0).stop 2023-07-06T03:49:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.224+0000 7ff3927ff700 1 -- 172.21.15.97:0/3162394626 shutdown_connections 2023-07-06T03:49:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.224+0000 7ff3927ff700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff37806ace0 0x7ff37806d190 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.225+0000 7ff3927ff700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff38c1585b0 0x7ff38c15dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.225+0000 7ff3927ff700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff374003a00 0x7ff38c1f6190 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.225+0000 7ff3927ff700 1 --2- 172.21.15.97:0/3162394626 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff38c14d250 0x7ff38c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.225+0000 7ff3927ff700 1 -- 172.21.15.97:0/3162394626 >> 172.21.15.97:0/3162394626 conn(0x7ff38c071790 msgr2=0x7ff38c0b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:10.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.225+0000 7ff3927ff700 1 -- 172.21.15.97:0/3162394626 shutdown_connections 2023-07-06T03:49:10.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.225+0000 7ff3927ff700 1 -- 172.21.15.97:0/3162394626 wait complete. 2023-07-06T03:49:10.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:48:43.594325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:48:44.371987+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:49:10.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:49:10.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.440+0000 7f5613bff700 1 Processor -- start 2023-07-06T03:49:10.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.440+0000 7f5613bff700 1 -- start start 2023-07-06T03:49:10.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.441+0000 7f5613bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f560c15c610 0x7f560c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.442+0000 7f5613bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f560c0d2210 0x7f560c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.442+0000 7f5613bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560c0cff90 con 0x7f560c06b360 2023-07-06T03:49:10.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.442+0000 7f5613bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c0d0780 con 0x7f560c15c610 2023-07-06T03:49:10.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.442+0000 7f5613bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15c9e0 con 0x7f560c0d2210 2023-07-06T03:49:10.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.442+0000 7f561119a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f560c06b360 0x7f560c0b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39882/0 (socket says 172.21.15.97:39882) 2023-07-06T03:49:10.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.442+0000 7f561119a700 1 -- 172.21.15.97:0/1174864754 learned_addr learned my addr 172.21.15.97:0/1174864754 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:10.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.442+0000 7f561219c700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f560c0d2210 0x7f560c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.442+0000 7f561199b700 1 --2- 172.21.15.97:0/1174864754 >> v2:172.21.15.146:3300/0 conn(0x7f560c15c610 0x7f560c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.442+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25219712 0 0) 0x7f560c0cff90 con 0x7f560c06b360 2023-07-06T03:49:10.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.442+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55f80035b0 con 0x7f560c06b360 2023-07-06T03:49:10.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 833259280 0 0) 0x7f55f80035b0 con 0x7f560c06b360 2023-07-06T03:49:10.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f560c0d2210 msgr2=0x7f560c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:10.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f560c0d2210 0x7f560c0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 >> v2:172.21.15.146:3300/0 conn(0x7f560c15c610 msgr2=0x7f560c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:10.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 --2- 172.21.15.97:0/1174864754 >> v2:172.21.15.146:3300/0 conn(0x7f560c15c610 0x7f560c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c0b5650 con 0x7f560c06b360 2023-07-06T03:49:10.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f55fc001670 con 0x7f560c06b360 2023-07-06T03:49:10.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 >> v1:172.21.15.97:6789/0 conn(0x7f560c06b360 legacy=0x7f560c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8003a00 0x7f55f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f8004850 0x7f55f8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f560c0d2210 0x7f55f8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0d0780 con 0x7f55f8004850 2023-07-06T03:49:10.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15c9e0 con 0x7f55f8003a00 2023-07-06T03:49:10.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f800ba10 con 0x7f560c0d2210 2023-07-06T03:49:10.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f561119a700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f8004850 0x7f55f8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f561199b700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8003a00 0x7f55f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f561219c700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f560c0d2210 0x7f55f8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f561119a700 1 -- 172.21.15.97:0/1174864754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f560c0d2210 msgr2=0x7f55f8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:10.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f561119a700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f560c0d2210 0x7f55f8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f561119a700 1 -- 172.21.15.97:0/1174864754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8003a00 msgr2=0x7f55f8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:10.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f561119a700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8003a00 0x7f55f8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.443+0000 7f561119a700 1 -- 172.21.15.97:0/1174864754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f80035b0 con 0x7f55f8004850 2023-07-06T03:49:10.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.444+0000 7f561119a700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f8004850 0x7f55f8004cc0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00c1c0 tx=0x7f55f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96462e0f59dda4ab server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:10.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.444+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc007b70 con 0x7f55f8004850 2023-07-06T03:49:10.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.444+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc00cd20 con 0x7f55f8004850 2023-07-06T03:49:10.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.444+0000 7f5610999700 1 -- 172.21.15.97:0/1174864754 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc005ba0 con 0x7f55f8004850 2023-07-06T03:49:10.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.444+0000 7f5613bff700 1 -- 172.21.15.97:0/1174864754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f8004850 msgr2=0x7f55f8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.444+0000 7f5613bff700 1 --2- 172.21.15.97:0/1174864754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f8004850 0x7f55f8004cc0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00c1c0 tx=0x7f55f8005880 comp rx=0 tx=0).stop 2023-07-06T03:49:10.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.444+0000 7f561219c700 1 -- 172.21.15.97:0/1174864754 reap_dead start 2023-07-06T03:49:10.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.444+0000 7f5613bff700 1 -- 172.21.15.97:0/1174864754 shutdown_connections 2023-07-06T03:49:10.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.444+0000 7f5613bff700 1 -- 172.21.15.97:0/1174864754 >> 172.21.15.97:0/1174864754 conn(0x7f560c06a6c0 msgr2=0x7f560c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:10.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.445+0000 7f5613bff700 1 -- 172.21.15.97:0/1174864754 shutdown_connections 2023-07-06T03:49:10.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.445+0000 7f5613bff700 1 -- 172.21.15.97:0/1174864754 wait complete. 2023-07-06T03:49:10.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.446+0000 7f5613bff700 1 Processor -- start 2023-07-06T03:49:10.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.446+0000 7f5613bff700 1 -- start start 2023-07-06T03:49:10.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.447+0000 7f5613bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f8004850 0x7f560c14b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.447+0000 7f5613bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f560c157f90 0x7f560c0b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.447+0000 7f5613bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f560c0b8490 0x7f560c0c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.447+0000 7f5613bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0b7400 con 0x7f560c0b8490 2023-07-06T03:49:10.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.447+0000 7f5613bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0b8900 con 0x7f560c157f90 2023-07-06T03:49:10.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.447+0000 7f5613bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c14b560 con 0x7f55f8004850 2023-07-06T03:49:10.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561219c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f560c0b8490 0x7f560c0c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561219c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f560c0b8490 0x7f560c0c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33794/0 (socket says 172.21.15.97:33794) 2023-07-06T03:49:10.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561219c700 1 -- 172.21.15.97:0/2488695758 learned_addr learned my addr 172.21.15.97:0/2488695758 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:10.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561199b700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f8004850 0x7f560c14b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561119a700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f560c157f90 0x7f560c0b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561219c700 1 -- 172.21.15.97:0/2488695758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f8004850 msgr2=0x7f560c14b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561219c700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f8004850 0x7f560c14b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561219c700 1 -- 172.21.15.97:0/2488695758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f560c157f90 msgr2=0x7f560c0b7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561219c700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f560c157f90 0x7f560c0b7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561219c700 1 -- 172.21.15.97:0/2488695758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c15e470 con 0x7f560c0b8490 2023-07-06T03:49:10.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561119a700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f560c157f90 0x7f560c0b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:10.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f561219c700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f560c0b8490 0x7f560c0c7c80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5604011bf0 tx=0x7f560400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:10.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f56027fc700 1 -- 172.21.15.97:0/2488695758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400e040 con 0x7f560c0b8490 2023-07-06T03:49:10.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f56027fc700 1 -- 172.21.15.97:0/2488695758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560400f040 con 0x7f560c0b8490 2023-07-06T03:49:10.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.448+0000 7f5613bff700 1 -- 172.21.15.97:0/2488695758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c15d6b0 con 0x7f560c0b8490 2023-07-06T03:49:10.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.449+0000 7f56027fc700 1 -- 172.21.15.97:0/2488695758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400fd30 con 0x7f560c0b8490 2023-07-06T03:49:10.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.449+0000 7f5613bff700 1 -- 172.21.15.97:0/2488695758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c15dc70 con 0x7f560c0b8490 2023-07-06T03:49:10.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.450+0000 7f5613bff700 1 -- 172.21.15.97:0/2488695758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c0ce600 con 0x7f560c0b8490 2023-07-06T03:49:10.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.451+0000 7f56027fc700 1 -- 172.21.15.97:0/2488695758 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5604022460 con 0x7f560c0b8490 2023-07-06T03:49:10.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.452+0000 7f56027fc700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f55f006ad10 0x7f55f006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.452+0000 7f56027fc700 1 -- 172.21.15.97:0/2488695758 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f56040899a0 con 0x7f560c0b8490 2023-07-06T03:49:10.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.452+0000 7f561199b700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f55f006ad10 0x7f55f006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.452+0000 7f561199b700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f55f006ad10 0x7f55f006d1c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5608009c60 tx=0x7f5608009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:10.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.456+0000 7f56027fc700 1 -- 172.21.15.97:0/2488695758 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f56040582e0 con 0x7f560c0b8490 2023-07-06T03:49:10.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.650+0000 7f5613bff700 1 -- 172.21.15.97:0/2488695758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f560c0d2030 con 0x7f560c0b8490 2023-07-06T03:49:10.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.650+0000 7f56027fc700 1 -- 172.21.15.97:0/2488695758 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5604057e70 con 0x7f560c0b8490 2023-07-06T03:49:10.653 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:49:10.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.652+0000 7f5613bff700 1 -- 172.21.15.97:0/2488695758 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f55f006ad10 msgr2=0x7f55f006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.652+0000 7f5613bff700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f55f006ad10 0x7f55f006d1c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5608009c60 tx=0x7f5608009210 comp rx=0 tx=0).stop 2023-07-06T03:49:10.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.652+0000 7f5613bff700 1 -- 172.21.15.97:0/2488695758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f560c0b8490 msgr2=0x7f560c0c7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.652+0000 7f5613bff700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f560c0b8490 0x7f560c0c7c80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5604011bf0 tx=0x7f560400f1b0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.653+0000 7f5613bff700 1 -- 172.21.15.97:0/2488695758 shutdown_connections 2023-07-06T03:49:10.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.653+0000 7f5613bff700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f55f006ad10 0x7f55f006d1c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.653+0000 7f5613bff700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f560c0b8490 0x7f560c0c7c80 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.653+0000 7f5613bff700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f55f8004850 0x7f560c14b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.653+0000 7f5613bff700 1 --2- 172.21.15.97:0/2488695758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f560c157f90 0x7f560c0b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.653+0000 7f5613bff700 1 -- 172.21.15.97:0/2488695758 >> 172.21.15.97:0/2488695758 conn(0x7f560c06a6c0 msgr2=0x7f560c15a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:10.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.653+0000 7f5613bff700 1 -- 172.21.15.97:0/2488695758 shutdown_connections 2023-07-06T03:49:10.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.653+0000 7f5613bff700 1 -- 172.21.15.97:0/2488695758 wait complete. 2023-07-06T03:49:10.673 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:49:10.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.868+0000 7fbd31bff700 1 Processor -- start 2023-07-06T03:49:10.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.869+0000 7fbd31bff700 1 -- start start 2023-07-06T03:49:10.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.870+0000 7fbd31bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbd2c158b20 0x7fbd2c0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.870+0000 7fbd31bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd2c06cab0 0x7fbd2c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.870+0000 7fbd31bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbd2c0b0560 0x7fbd2c0b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.870+0000 7fbd31bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd2c0ce820 con 0x7fbd2c0b0560 2023-07-06T03:49:10.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.870+0000 7fbd31bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd2c0b0f20 con 0x7fbd2c158b20 2023-07-06T03:49:10.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.870+0000 7fbd31bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c158ef0 con 0x7fbd2c06cab0 2023-07-06T03:49:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.870+0000 7fbd2bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbd2c0b0560 0x7fbd2c0b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.870+0000 7fbd2bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbd2c0b0560 0x7fbd2c0b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33798/0 (socket says 172.21.15.97:33798) 2023-07-06T03:49:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.870+0000 7fbd2bfff700 1 -- 172.21.15.97:0/1968519238 learned_addr learned my addr 172.21.15.97:0/1968519238 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.870+0000 7fbd23fff700 1 --2- 172.21.15.97:0/1968519238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd2c06cab0 0x7fbd2c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.870+0000 7fbd2b7fe700 1 --2- 172.21.15.97:0/1968519238 >> v2:172.21.15.146:3300/0 conn(0x7fbd2c158b20 0x7fbd2c0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.871+0000 7fbd2bfff700 1 -- 172.21.15.97:0/1968519238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd2c06cab0 msgr2=0x7fbd2c0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.871+0000 7fbd2bfff700 1 --2- 172.21.15.97:0/1968519238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd2c06cab0 0x7fbd2c0ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.871+0000 7fbd2bfff700 1 -- 172.21.15.97:0/1968519238 >> v2:172.21.15.146:3300/0 conn(0x7fbd2c158b20 msgr2=0x7fbd2c0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.871+0000 7fbd2bfff700 1 --2- 172.21.15.97:0/1968519238 >> v2:172.21.15.146:3300/0 conn(0x7fbd2c158b20 0x7fbd2c0cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.871+0000 7fbd2bfff700 1 -- 172.21.15.97:0/1968519238 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c0ccbc0 con 0x7fbd2c0b0560 2023-07-06T03:49:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.871+0000 7fbd2b7fe700 1 --2- 172.21.15.97:0/1968519238 >> v2:172.21.15.146:3300/0 conn(0x7fbd2c158b20 0x7fbd2c0cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.871+0000 7fbd2bfff700 1 --2- 172.21.15.97:0/1968519238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd2c0b0560 0x7fbd2c0b09e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c06f8e0 tx=0x7fbd2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9e474a7a3f2ccc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.871+0000 7fbd2affd700 1 -- 172.21.15.97:0/1968519238 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2400e070 con 0x7fbd2c0b0560 2023-07-06T03:49:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.871+0000 7fbd2affd700 1 -- 172.21.15.97:0/1968519238 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2400b8d0 con 0x7fbd2c0b0560 2023-07-06T03:49:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.871+0000 7fbd2affd700 1 -- 172.21.15.97:0/1968519238 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2400bc80 con 0x7fbd2c0b0560 2023-07-06T03:49:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.872+0000 7fbd31bff700 1 -- 172.21.15.97:0/1968519238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd2c0b0560 msgr2=0x7fbd2c0b09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.872+0000 7fbd31bff700 1 --2- 172.21.15.97:0/1968519238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd2c0b0560 0x7fbd2c0b09e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c06f8e0 tx=0x7fbd2400b330 comp rx=0 tx=0).stop 2023-07-06T03:49:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.872+0000 7fbd31bff700 1 -- 172.21.15.97:0/1968519238 shutdown_connections 2023-07-06T03:49:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.872+0000 7fbd31bff700 1 --2- 172.21.15.97:0/1968519238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd2c0b0560 0x7fbd2c0b09e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.872+0000 7fbd31bff700 1 --2- 172.21.15.97:0/1968519238 >> v2:172.21.15.146:3300/0 conn(0x7fbd2c158b20 0x7fbd2c0cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.872+0000 7fbd31bff700 1 --2- 172.21.15.97:0/1968519238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd2c06cab0 0x7fbd2c0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.872+0000 7fbd31bff700 1 -- 172.21.15.97:0/1968519238 >> 172.21.15.97:0/1968519238 conn(0x7fbd2c06a060 msgr2=0x7fbd2c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.872+0000 7fbd31bff700 1 -- 172.21.15.97:0/1968519238 shutdown_connections 2023-07-06T03:49:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.872+0000 7fbd31bff700 1 -- 172.21.15.97:0/1968519238 wait complete. 2023-07-06T03:49:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.874+0000 7fbd31bff700 1 Processor -- start 2023-07-06T03:49:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.874+0000 7fbd31bff700 1 -- start start 2023-07-06T03:49:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd31bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd2c06cab0 0x7fbd2c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd31bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd2c0b0560 0x7fbd2c0af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd31bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd2c158b20 0x7fbd2c0afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd31bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c1f1ce0 con 0x7fbd2c06cab0 2023-07-06T03:49:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd31bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c1f2d40 con 0x7fbd2c158b20 2023-07-06T03:49:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd31bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c1f1f80 con 0x7fbd2c0b0560 2023-07-06T03:49:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd2b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd2c06cab0 0x7fbd2c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd2b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd2c06cab0 0x7fbd2c15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33806/0 (socket says 172.21.15.97:33806) 2023-07-06T03:49:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd2b7fe700 1 -- 172.21.15.97:0/1737948492 learned_addr learned my addr 172.21.15.97:0/1737948492 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd2b7fe700 1 -- 172.21.15.97:0/1737948492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd2c0b0560 msgr2=0x7fbd2c0af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd2b7fe700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd2c0b0560 0x7fbd2c0af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd2b7fe700 1 -- 172.21.15.97:0/1737948492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd2c158b20 msgr2=0x7fbd2c0afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd2b7fe700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd2c158b20 0x7fbd2c0afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.875+0000 7fbd2b7fe700 1 -- 172.21.15.97:0/1737948492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd24009b30 con 0x7fbd2c06cab0 2023-07-06T03:49:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.876+0000 7fbd2b7fe700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd2c06cab0 0x7fbd2c15a080 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00e810 tx=0x7fbd1c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:10.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.876+0000 7fbd28ff9700 1 -- 172.21.15.97:0/1737948492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c004230 con 0x7fbd2c06cab0 2023-07-06T03:49:10.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.876+0000 7fbd28ff9700 1 -- 172.21.15.97:0/1737948492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c004390 con 0x7fbd2c06cab0 2023-07-06T03:49:10.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.876+0000 7fbd31bff700 1 -- 172.21.15.97:0/1737948492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2c1f2260 con 0x7fbd2c06cab0 2023-07-06T03:49:10.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.876+0000 7fbd28ff9700 1 -- 172.21.15.97:0/1737948492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0036a0 con 0x7fbd2c06cab0 2023-07-06T03:49:10.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.876+0000 7fbd31bff700 1 -- 172.21.15.97:0/1737948492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2c1f2820 con 0x7fbd2c06cab0 2023-07-06T03:49:10.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.877+0000 7fbd31bff700 1 -- 172.21.15.97:0/1737948492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd2c04e340 con 0x7fbd2c06cab0 2023-07-06T03:49:10.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.878+0000 7fbd28ff9700 1 -- 172.21.15.97:0/1737948492 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbd1c003800 con 0x7fbd2c06cab0 2023-07-06T03:49:10.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.879+0000 7fbd28ff9700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbd0c06ace0 0x7fbd0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:10.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.879+0000 7fbd28ff9700 1 -- 172.21.15.97:0/1737948492 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fbd1c089150 con 0x7fbd2c06cab0 2023-07-06T03:49:10.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.879+0000 7fbd23fff700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbd0c06ace0 0x7fbd0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.879+0000 7fbd23fff700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbd0c06ace0 0x7fbd0c06d190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c15c070 tx=0x7fbd1400b0e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:10.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:10.883+0000 7fbd28ff9700 1 -- 172.21.15.97:0/1737948492 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbd1c0540d0 con 0x7fbd2c06cab0 2023-07-06T03:49:11.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.080+0000 7fbd31bff700 1 -- 172.21.15.97:0/1737948492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd2c0715e0 con 0x7fbd2c06cab0 2023-07-06T03:49:11.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.081+0000 7fbd28ff9700 1 -- 172.21.15.97:0/1737948492 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbd1c0576f0 con 0x7fbd2c06cab0 2023-07-06T03:49:11.084 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:49:11.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 -- 172.21.15.97:0/1737948492 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbd0c06ace0 msgr2=0x7fbd0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbd0c06ace0 0x7fbd0c06d190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c15c070 tx=0x7fbd1400b0e0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 -- 172.21.15.97:0/1737948492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd2c06cab0 msgr2=0x7fbd2c15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd2c06cab0 0x7fbd2c15a080 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00e810 tx=0x7fbd1c00ebd0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 -- 172.21.15.97:0/1737948492 shutdown_connections 2023-07-06T03:49:11.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbd0c06ace0 0x7fbd0c06d190 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd2c158b20 0x7fbd2c0afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd2c06cab0 0x7fbd2c15a080 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 --2- 172.21.15.97:0/1737948492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbd2c0b0560 0x7fbd2c0af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 -- 172.21.15.97:0/1737948492 >> 172.21.15.97:0/1737948492 conn(0x7fbd2c06a060 msgr2=0x7fbd2c157120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 -- 172.21.15.97:0/1737948492 shutdown_connections 2023-07-06T03:49:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.083+0000 7fbd31bff700 1 -- 172.21.15.97:0/1737948492 wait complete. 2023-07-06T03:49:11.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:49:11.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.298+0000 7f79223ff700 1 Processor -- start 2023-07-06T03:49:11.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.298+0000 7f79223ff700 1 -- start start 2023-07-06T03:49:11.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.299+0000 7f79223ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f791c15c610 0x7f791c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f79223ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f791c06b360 0x7f791c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f79223ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791c0cff90 con 0x7f791c0d2210 2023-07-06T03:49:11.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f79223ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f791c0d0780 con 0x7f791c15c610 2023-07-06T03:49:11.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f79223ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c15c9e0 con 0x7f791c06b360 2023-07-06T03:49:11.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f791bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f791c15c610 0x7f791c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f791bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f791c15c610 0x7f791c0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33822/0 (socket says 172.21.15.97:33822) 2023-07-06T03:49:11.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f791bfff700 1 -- 172.21.15.97:0/2708752236 learned_addr learned my addr 172.21.15.97:0/2708752236 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:11.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f791b7fe700 1 --2- 172.21.15.97:0/2708752236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f791c06b360 0x7f791c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f791bfff700 1 -- 172.21.15.97:0/2708752236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f791c06b360 msgr2=0x7f791c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:11.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f791bfff700 1 --2- 172.21.15.97:0/2708752236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f791c06b360 0x7f791c0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f791bfff700 1 -- 172.21.15.97:0/2708752236 >> v1:172.21.15.146:6789/0 conn(0x7f791c0d2210 legacy=0x7f791c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f791bfff700 1 -- 172.21.15.97:0/2708752236 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c0b5650 con 0x7f791c15c610 2023-07-06T03:49:11.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.300+0000 7f791affd700 1 -- 172.21.15.97:0/2708752236 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491077766 0 0) 0x7f791c0cff90 con 0x7f791c0d2210 2023-07-06T03:49:11.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.301+0000 7f791bfff700 1 --2- 172.21.15.97:0/2708752236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791c15c610 0x7f791c0b47b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f790c009920 tx=0x7f790c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f564b24976ce5a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:11.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.301+0000 7f791affd700 1 -- 172.21.15.97:0/2708752236 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c011830 con 0x7f791c15c610 2023-07-06T03:49:11.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.301+0000 7f791affd700 1 -- 172.21.15.97:0/2708752236 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c011990 con 0x7f791c15c610 2023-07-06T03:49:11.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.301+0000 7f791affd700 1 -- 172.21.15.97:0/2708752236 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c011d20 con 0x7f791c15c610 2023-07-06T03:49:11.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.301+0000 7f79223ff700 1 -- 172.21.15.97:0/2708752236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791c15c610 msgr2=0x7f791c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.301+0000 7f79223ff700 1 --2- 172.21.15.97:0/2708752236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791c15c610 0x7f791c0b47b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f790c009920 tx=0x7f790c009c30 comp rx=0 tx=0).stop 2023-07-06T03:49:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.301+0000 7f79223ff700 1 -- 172.21.15.97:0/2708752236 shutdown_connections 2023-07-06T03:49:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.301+0000 7f79223ff700 1 --2- 172.21.15.97:0/2708752236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791c15c610 0x7f791c0b47b0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.301+0000 7f79223ff700 1 --2- 172.21.15.97:0/2708752236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f791c06b360 0x7f791c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.301+0000 7f79223ff700 1 -- 172.21.15.97:0/2708752236 >> 172.21.15.97:0/2708752236 conn(0x7f791c06a6c0 msgr2=0x7f791c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.302+0000 7f79223ff700 1 -- 172.21.15.97:0/2708752236 shutdown_connections 2023-07-06T03:49:11.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.302+0000 7f79223ff700 1 -- 172.21.15.97:0/2708752236 wait complete. 2023-07-06T03:49:11.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.303+0000 7f79223ff700 1 Processor -- start 2023-07-06T03:49:11.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.303+0000 7f79223ff700 1 -- start start 2023-07-06T03:49:11.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.304+0000 7f79223ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f791c06b360 0x7f791c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.304+0000 7f79223ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f791c0d2210 0x7f791c0cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.304+0000 7f79223ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791c15c610 0x7f791c149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.304+0000 7f79223ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0ccc50 con 0x7f791c15c610 2023-07-06T03:49:11.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.304+0000 7f79223ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0cb670 con 0x7f791c0d2210 2023-07-06T03:49:11.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.304+0000 7f79223ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0cb950 con 0x7f791c06b360 2023-07-06T03:49:11.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f792099c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791c15c610 0x7f791c149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f792099c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791c15c610 0x7f791c149ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33824/0 (socket says 172.21.15.97:33824) 2023-07-06T03:49:11.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f792099c700 1 -- 172.21.15.97:0/3558556306 learned_addr learned my addr 172.21.15.97:0/3558556306 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:11.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f791bfff700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f791c06b360 0x7f791c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f791b7fe700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f791c0d2210 0x7f791c0cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f792099c700 1 -- 172.21.15.97:0/3558556306 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f791c06b360 msgr2=0x7f791c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f792099c700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f791c06b360 0x7f791c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f792099c700 1 -- 172.21.15.97:0/3558556306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f791c0d2210 msgr2=0x7f791c0cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f792099c700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f791c0d2210 0x7f791c0cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f792099c700 1 -- 172.21.15.97:0/3558556306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c009550 con 0x7f791c15c610 2023-07-06T03:49:11.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f792099c700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791c15c610 0x7f791c149ff0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7914001560 tx=0x7f791400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:11.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.305+0000 7f7918ff9700 1 -- 172.21.15.97:0/3558556306 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914003e30 con 0x7f791c15c610 2023-07-06T03:49:11.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.306+0000 7f7918ff9700 1 -- 172.21.15.97:0/3558556306 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7914005070 con 0x7f791c15c610 2023-07-06T03:49:11.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.306+0000 7f7918ff9700 1 -- 172.21.15.97:0/3558556306 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914005420 con 0x7f791c15c610 2023-07-06T03:49:11.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.306+0000 7f79223ff700 1 -- 172.21.15.97:0/3558556306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791c0cbc30 con 0x7f791c15c610 2023-07-06T03:49:11.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.306+0000 7f79223ff700 1 -- 172.21.15.97:0/3558556306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791c0cc1f0 con 0x7f791c15c610 2023-07-06T03:49:11.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.307+0000 7f79223ff700 1 -- 172.21.15.97:0/3558556306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791c066fe0 con 0x7f791c15c610 2023-07-06T03:49:11.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.308+0000 7f7918ff9700 1 -- 172.21.15.97:0/3558556306 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7914005580 con 0x7f791c15c610 2023-07-06T03:49:11.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.308+0000 7f7918ff9700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f790806ace0 0x7f790806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.309+0000 7f7918ff9700 1 -- 172.21.15.97:0/3558556306 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f7914088d60 con 0x7f791c15c610 2023-07-06T03:49:11.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.309+0000 7f791bfff700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f790806ace0 0x7f790806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.309+0000 7f791bfff700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f790806ace0 0x7f790806d190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f790c000c00 tx=0x7f790c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:11.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.313+0000 7f7918ff9700 1 -- 172.21.15.97:0/3558556306 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7914053d30 con 0x7f791c15c610 2023-07-06T03:49:11.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.504+0000 7f79223ff700 1 -- 172.21.15.97:0/3558556306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f791c04e340 con 0x7f791c15c610 2023-07-06T03:49:11.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.505+0000 7f7918ff9700 1 -- 172.21.15.97:0/3558556306 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7914017020 con 0x7f791c15c610 2023-07-06T03:49:11.508 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:49:11.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 -- 172.21.15.97:0/3558556306 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f790806ace0 msgr2=0x7f790806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f790806ace0 0x7f790806d190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f790c000c00 tx=0x7f790c008030 comp rx=0 tx=0).stop 2023-07-06T03:49:11.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 -- 172.21.15.97:0/3558556306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791c15c610 msgr2=0x7f791c149ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791c15c610 0x7f791c149ff0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7914001560 tx=0x7f791400cc00 comp rx=0 tx=0).stop 2023-07-06T03:49:11.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 -- 172.21.15.97:0/3558556306 shutdown_connections 2023-07-06T03:49:11.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f790806ace0 0x7f790806d190 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791c15c610 0x7f791c149ff0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f791c06b360 0x7f791c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 --2- 172.21.15.97:0/3558556306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f791c0d2210 0x7f791c0cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 -- 172.21.15.97:0/3558556306 >> 172.21.15.97:0/3558556306 conn(0x7f791c06a6c0 msgr2=0x7f791c0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:11.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 -- 172.21.15.97:0/3558556306 shutdown_connections 2023-07-06T03:49:11.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.507+0000 7f79223ff700 1 -- 172.21.15.97:0/3558556306 wait complete. 2023-07-06T03:49:11.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:49:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.723+0000 7fb8c8fff700 1 Processor -- start 2023-07-06T03:49:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.723+0000 7fb8c8fff700 1 -- start start 2023-07-06T03:49:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.724+0000 7fb8c8fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c406b360 0x7fb8c40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.724+0000 7fb8c8fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8c40d2210 0x7fb8c40d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.724+0000 7fb8c8fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c40cff90 con 0x7fb8c415c610 2023-07-06T03:49:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.724+0000 7fb8c8fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c40d0780 con 0x7fb8c40d2210 2023-07-06T03:49:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.724+0000 7fb8c8fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c415c9e0 con 0x7fb8c406b360 2023-07-06T03:49:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.725+0000 7fb8c2d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8c40d2210 0x7fb8c40d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.725+0000 7fb8c2d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8c40d2210 0x7fb8c40d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33828/0 (socket says 172.21.15.97:33828) 2023-07-06T03:49:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.725+0000 7fb8c2d9d700 1 -- 172.21.15.97:0/3277341304 learned_addr learned my addr 172.21.15.97:0/3277341304 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.725+0000 7fb8c1d9b700 1 --2- 172.21.15.97:0/3277341304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c406b360 0x7fb8c40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.725+0000 7fb8c2d9d700 1 -- 172.21.15.97:0/3277341304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c406b360 msgr2=0x7fb8c40b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.725+0000 7fb8c2d9d700 1 --2- 172.21.15.97:0/3277341304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c406b360 0x7fb8c40b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.725+0000 7fb8c2d9d700 1 -- 172.21.15.97:0/3277341304 >> v1:172.21.15.146:6789/0 conn(0x7fb8c415c610 legacy=0x7fb8c40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.725+0000 7fb8c2d9d700 1 -- 172.21.15.97:0/3277341304 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c40b5650 con 0x7fb8c40d2210 2023-07-06T03:49:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.725+0000 7fb8c2d9d700 1 --2- 172.21.15.97:0/3277341304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c40d2210 0x7fb8c40d1310 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009e80 tx=0x7fb8bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69183f77c9b38a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:11.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.725+0000 7fb8c159a700 1 -- 172.21.15.97:0/3277341304 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00e070 con 0x7fb8c40d2210 2023-07-06T03:49:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.726+0000 7fb8c159a700 1 -- 172.21.15.97:0/3277341304 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc00b980 con 0x7fb8c40d2210 2023-07-06T03:49:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.726+0000 7fb8c159a700 1 -- 172.21.15.97:0/3277341304 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00bd30 con 0x7fb8c40d2210 2023-07-06T03:49:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.726+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3277341304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c40d2210 msgr2=0x7fb8c40d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.726+0000 7fb8c8fff700 1 --2- 172.21.15.97:0/3277341304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c40d2210 0x7fb8c40d1310 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009e80 tx=0x7fb8bc00b330 comp rx=0 tx=0).stop 2023-07-06T03:49:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.726+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3277341304 shutdown_connections 2023-07-06T03:49:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.726+0000 7fb8c8fff700 1 --2- 172.21.15.97:0/3277341304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c40d2210 0x7fb8c40d1310 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.726+0000 7fb8c8fff700 1 --2- 172.21.15.97:0/3277341304 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c406b360 0x7fb8c40b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.726+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3277341304 >> 172.21.15.97:0/3277341304 conn(0x7fb8c406a6c0 msgr2=0x7fb8c406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.726+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3277341304 shutdown_connections 2023-07-06T03:49:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.726+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3277341304 wait complete. 2023-07-06T03:49:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.729+0000 7fb8c8fff700 1 Processor -- start 2023-07-06T03:49:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.729+0000 7fb8c8fff700 1 -- start start 2023-07-06T03:49:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.730+0000 7fb8c8fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c406b360 0x7fb8c41e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.730+0000 7fb8c8fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c40d2210 0x7fb8c40b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.730+0000 7fb8c8fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c415c610 0x7fb8c4153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.730+0000 7fb8c8fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4154700 con 0x7fb8c415c610 2023-07-06T03:49:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.730+0000 7fb8c8fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41e7250 con 0x7fb8c40d2210 2023-07-06T03:49:11.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.730+0000 7fb8c8fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41e7530 con 0x7fb8c406b360 2023-07-06T03:49:11.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.730+0000 7fb8c2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c415c610 0x7fb8c4153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.730+0000 7fb8c2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c415c610 0x7fb8c4153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33840/0 (socket says 172.21.15.97:33840) 2023-07-06T03:49:11.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.730+0000 7fb8c2d9d700 1 -- 172.21.15.97:0/3618927368 learned_addr learned my addr 172.21.15.97:0/3618927368 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.730+0000 7fb8c1d9b700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c40d2210 0x7fb8c40b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8c259c700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c406b360 0x7fb8c41e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8c2d9d700 1 -- 172.21.15.97:0/3618927368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c406b360 msgr2=0x7fb8c41e82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8c2d9d700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c406b360 0x7fb8c41e82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8c2d9d700 1 -- 172.21.15.97:0/3618927368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c40d2210 msgr2=0x7fb8c40b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8c2d9d700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c40d2210 0x7fb8c40b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8c2d9d700 1 -- 172.21.15.97:0/3618927368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc009b30 con 0x7fb8c415c610 2023-07-06T03:49:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8c1d9b700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c40d2210 0x7fb8c40b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8c259c700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c406b360 0x7fb8c41e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:11.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8c2d9d700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c415c610 0x7fb8c4153b60 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb8c408b120 tx=0x7fb8bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8b2ffd700 1 -- 172.21.15.97:0/3618927368 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc004960 con 0x7fb8c415c610 2023-07-06T03:49:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8b2ffd700 1 -- 172.21.15.97:0/3618927368 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc0099b0 con 0x7fb8c415c610 2023-07-06T03:49:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8b2ffd700 1 -- 172.21.15.97:0/3618927368 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc01c680 con 0x7fb8c415c610 2023-07-06T03:49:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3618927368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c41e7810 con 0x7fb8c415c610 2023-07-06T03:49:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.731+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3618927368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c41e7dd0 con 0x7fb8c415c610 2023-07-06T03:49:11.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.732+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3618927368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c404e340 con 0x7fb8c415c610 2023-07-06T03:49:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.733+0000 7fb8b2ffd700 1 -- 172.21.15.97:0/3618927368 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb8bc01c7e0 con 0x7fb8c415c610 2023-07-06T03:49:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.734+0000 7fb8b2ffd700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8a406ace0 0x7fb8a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.734+0000 7fb8b2ffd700 1 -- 172.21.15.97:0/3618927368 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fb8bc08af80 con 0x7fb8c415c610 2023-07-06T03:49:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.734+0000 7fb8c259c700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8a406ace0 0x7fb8a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:11.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.735+0000 7fb8c259c700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8a406ace0 0x7fb8a406d190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4000f60 tx=0x7fb8b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:11.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.738+0000 7fb8b2ffd700 1 -- 172.21.15.97:0/3618927368 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb8bc055f50 con 0x7fb8c415c610 2023-07-06T03:49:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.932+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3618927368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8c40b5b60 con 0x7fb8c415c610 2023-07-06T03:49:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.932+0000 7fb8b2ffd700 1 -- 172.21.15.97:0/3618927368 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8bc059570 con 0x7fb8c415c610 2023-07-06T03:49:11.938 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:49:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3618927368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8a406ace0 msgr2=0x7fb8a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8a406ace0 0x7fb8a406d190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4000f60 tx=0x7fb8b4009040 comp rx=0 tx=0).stop 2023-07-06T03:49:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3618927368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c415c610 msgr2=0x7fb8c4153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c415c610 0x7fb8c4153b60 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb8c408b120 tx=0x7fb8bc004ad0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3618927368 shutdown_connections 2023-07-06T03:49:11.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8a406ace0 0x7fb8a406d190 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8c415c610 0x7fb8c4153b60 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8c406b360 0x7fb8c41e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 --2- 172.21.15.97:0/3618927368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8c40d2210 0x7fb8c40b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3618927368 >> 172.21.15.97:0/3618927368 conn(0x7fb8c406a6c0 msgr2=0x7fb8c40d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3618927368 shutdown_connections 2023-07-06T03:49:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:11.935+0000 7fb8c8fff700 1 -- 172.21.15.97:0/3618927368 wait complete. 2023-07-06T03:49:11.956 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:49:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.146+0000 7f2d523ff700 1 Processor -- start 2023-07-06T03:49:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.146+0000 7f2d523ff700 1 -- start start 2023-07-06T03:49:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.148+0000 7f2d523ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 0x7f2d4c167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.148+0000 7f2d523ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4c0cdae0 con 0x7f2d4c15c520 2023-07-06T03:49:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.148+0000 7f2d523ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4c0b6640 con 0x7f2d4c0cd390 2023-07-06T03:49:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.148+0000 7f2d523ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0b6920 con 0x7f2d4c0b8410 2023-07-06T03:49:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.148+0000 7f2d4bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2d4c15c520 0x7f2d4c160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39890/0 (socket says 172.21.15.97:39890) 2023-07-06T03:49:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.148+0000 7f2d4bfff700 1 -- 172.21.15.97:0/2059720870 learned_addr learned my addr 172.21.15.97:0/2059720870 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.148+0000 7f2d5099c700 1 --2- 172.21.15.97:0/2059720870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 0x7f2d4c167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.148+0000 7f2d4b7fe700 1 -- 172.21.15.97:0/2059720870 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3180141884 0 0) 0x7f2d4c0cdae0 con 0x7f2d4c15c520 2023-07-06T03:49:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.148+0000 7f2d4b7fe700 1 -- 172.21.15.97:0/2059720870 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d380035b0 con 0x7f2d4c15c520 2023-07-06T03:49:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.148+0000 7f2d4b7fe700 1 -- 172.21.15.97:0/2059720870 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946301738 0 0) 0x7f2d4c0b6640 con 0x7f2d4c0cd390 2023-07-06T03:49:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.148+0000 7f2d4b7fe700 1 -- 172.21.15.97:0/2059720870 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d4c0cdae0 con 0x7f2d4c0cd390 2023-07-06T03:49:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.149+0000 7f2d5099c700 1 -- 172.21.15.97:0/2059720870 >> v1:172.21.15.146:6789/0 conn(0x7f2d4c0cd390 legacy=0x7f2d4c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.149+0000 7f2d5099c700 1 -- 172.21.15.97:0/2059720870 >> v1:172.21.15.97:6789/0 conn(0x7f2d4c15c520 legacy=0x7f2d4c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.149+0000 7f2d5099c700 1 -- 172.21.15.97:0/2059720870 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c0b6c00 con 0x7f2d4c0b8410 2023-07-06T03:49:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.149+0000 7f2d5099c700 1 --2- 172.21.15.97:0/2059720870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 0x7f2d4c167510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009dc0 tx=0x7f2d4400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=513704d9a1640d46 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.149+0000 7f2d4b7fe700 1 -- 172.21.15.97:0/2059720870 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4400e070 con 0x7f2d4c0b8410 2023-07-06T03:49:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.149+0000 7f2d4b7fe700 1 -- 172.21.15.97:0/2059720870 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4400bb20 con 0x7f2d4c0b8410 2023-07-06T03:49:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.149+0000 7f2d4b7fe700 1 -- 172.21.15.97:0/2059720870 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4400be90 con 0x7f2d4c0b8410 2023-07-06T03:49:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.150+0000 7f2d523ff700 1 -- 172.21.15.97:0/2059720870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 msgr2=0x7f2d4c167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.150+0000 7f2d523ff700 1 --2- 172.21.15.97:0/2059720870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 0x7f2d4c167510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009dc0 tx=0x7f2d4400b330 comp rx=0 tx=0).stop 2023-07-06T03:49:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.150+0000 7f2d523ff700 1 -- 172.21.15.97:0/2059720870 shutdown_connections 2023-07-06T03:49:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.150+0000 7f2d523ff700 1 --2- 172.21.15.97:0/2059720870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 0x7f2d4c167510 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.150+0000 7f2d523ff700 1 -- 172.21.15.97:0/2059720870 >> 172.21.15.97:0/2059720870 conn(0x7f2d4c071790 msgr2=0x7f2d4c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.150+0000 7f2d523ff700 1 -- 172.21.15.97:0/2059720870 shutdown_connections 2023-07-06T03:49:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.150+0000 7f2d523ff700 1 -- 172.21.15.97:0/2059720870 wait complete. 2023-07-06T03:49:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.151+0000 7f2d523ff700 1 Processor -- start 2023-07-06T03:49:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.152+0000 7f2d523ff700 1 -- start start 2023-07-06T03:49:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.152+0000 7f2d523ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 0x7f2d4c0d3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d523ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d4c0cd390 0x7f2d4c15f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d523ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4c15c520 0x7f2d4c15c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d523ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0d3ba0 con 0x7f2d4c15c520 2023-07-06T03:49:12.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d523ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c15a060 con 0x7f2d4c0cd390 2023-07-06T03:49:12.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d523ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c15a340 con 0x7f2d4c0b8410 2023-07-06T03:49:12.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d5099c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4c15c520 0x7f2d4c15c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d5099c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4c15c520 0x7f2d4c15c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33844/0 (socket says 172.21.15.97:33844) 2023-07-06T03:49:12.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d5099c700 1 -- 172.21.15.97:0/160099668 learned_addr learned my addr 172.21.15.97:0/160099668 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:12.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d4bfff700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 0x7f2d4c0d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d43fff700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d4c0cd390 0x7f2d4c15f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d5099c700 1 -- 172.21.15.97:0/160099668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 msgr2=0x7f2d4c0d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d5099c700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 0x7f2d4c0d3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d5099c700 1 -- 172.21.15.97:0/160099668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d4c0cd390 msgr2=0x7f2d4c15f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d5099c700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d4c0cd390 0x7f2d4c15f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d5099c700 1 -- 172.21.15.97:0/160099668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d44009a70 con 0x7f2d4c15c520 2023-07-06T03:49:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.153+0000 7f2d4bfff700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 0x7f2d4c0d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.154+0000 7f2d5099c700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4c15c520 0x7f2d4c15c8f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2d44014040 tx=0x7f2d44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.154+0000 7f2d497fa700 1 -- 172.21.15.97:0/160099668 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d440098f0 con 0x7f2d4c15c520 2023-07-06T03:49:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.154+0000 7f2d497fa700 1 -- 172.21.15.97:0/160099668 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d44020070 con 0x7f2d4c15c520 2023-07-06T03:49:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.154+0000 7f2d523ff700 1 -- 172.21.15.97:0/160099668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c15a620 con 0x7f2d4c15c520 2023-07-06T03:49:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.154+0000 7f2d497fa700 1 -- 172.21.15.97:0/160099668 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d44027680 con 0x7f2d4c15c520 2023-07-06T03:49:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.154+0000 7f2d523ff700 1 -- 172.21.15.97:0/160099668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c15abe0 con 0x7f2d4c15c520 2023-07-06T03:49:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.155+0000 7f2d523ff700 1 -- 172.21.15.97:0/160099668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4c04e340 con 0x7f2d4c15c520 2023-07-06T03:49:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.156+0000 7f2d497fa700 1 -- 172.21.15.97:0/160099668 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2d44012430 con 0x7f2d4c15c520 2023-07-06T03:49:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.157+0000 7f2d497fa700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d2c06ace0 0x7f2d2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.157+0000 7f2d497fa700 1 -- 172.21.15.97:0/160099668 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f2d44022070 con 0x7f2d4c15c520 2023-07-06T03:49:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.157+0000 7f2d4bfff700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d2c06ace0 0x7f2d2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.158+0000 7f2d4bfff700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d2c06ace0 0x7f2d2c06d190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c000f60 tx=0x7f2d3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.161+0000 7f2d497fa700 1 -- 172.21.15.97:0/160099668 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2d44056290 con 0x7f2d4c15c520 2023-07-06T03:49:12.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.393+0000 7f2d523ff700 1 -- 172.21.15.97:0/160099668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d4c0b4f40 con 0x7f2d4c15c520 2023-07-06T03:49:12.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.393+0000 7f2d497fa700 1 -- 172.21.15.97:0/160099668 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4678 (secure 0 0 0) 0x7f2d440598b0 con 0x7f2d4c15c520 2023-07-06T03:49:12.396 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:12.396 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1108873506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1108873506},{"type":"v1","addr":"172.21.15.146:6835","nonce":1108873506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2471899168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2471899168},{"type":"v1","addr":"172.21.15.97:6837","nonce":2471899168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2970792733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2970792733},{"type":"v1","addr":"172.21.15.97:6836","nonce":2970792733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:48:45.401033+0000","modified":"2023-07-06T03:48:48.930905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6399":{"gid":6399,"name":"d","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/2151424592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2151424592},{"type":"v1","addr":"172.21.15.146:6834","nonce":2151424592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-06T03:49:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 -- 172.21.15.97:0/160099668 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d2c06ace0 msgr2=0x7f2d2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d2c06ace0 0x7f2d2c06d190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c000f60 tx=0x7f2d3c009040 comp rx=0 tx=0).stop 2023-07-06T03:49:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 -- 172.21.15.97:0/160099668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4c15c520 msgr2=0x7f2d4c15c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4c15c520 0x7f2d4c15c8f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2d44014040 tx=0x7f2d44004ad0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 -- 172.21.15.97:0/160099668 shutdown_connections 2023-07-06T03:49:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d2c06ace0 0x7f2d2c06d190 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4c15c520 0x7f2d4c15c8f0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d4c0b8410 0x7f2d4c0d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 --2- 172.21.15.97:0/160099668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d4c0cd390 0x7f2d4c15f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 -- 172.21.15.97:0/160099668 >> 172.21.15.97:0/160099668 conn(0x7f2d4c071790 msgr2=0x7f2d4c1606b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 -- 172.21.15.97:0/160099668 shutdown_connections 2023-07-06T03:49:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.396+0000 7f2d523ff700 1 -- 172.21.15.97:0/160099668 wait complete. 2023-07-06T03:49:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 34 2023-07-06T03:49:12.417 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:49:12.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.608+0000 7f84341ff700 1 Processor -- start 2023-07-06T03:49:12.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.608+0000 7f84341ff700 1 -- start start 2023-07-06T03:49:12.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.609+0000 7f84341ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f842c15c520 0x7f842c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.609+0000 7f84341ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f842c160180 0x7f842c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.609+0000 7f84341ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f842c160b30 0x7f842c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.609+0000 7f84341ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c0cd8d0 con 0x7f842c160180 2023-07-06T03:49:12.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.609+0000 7f84341ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c0cdae0 con 0x7f842c15c520 2023-07-06T03:49:12.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.609+0000 7f84341ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0b6640 con 0x7f842c160b30 2023-07-06T03:49:12.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.609+0000 7f843179a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f842c160180 0x7f842c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.609+0000 7f843179a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f842c160180 0x7f842c1605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33848/0 (socket says 172.21.15.97:33848) 2023-07-06T03:49:12.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.609+0000 7f843179a700 1 -- 172.21.15.97:0/2622230334 learned_addr learned my addr 172.21.15.97:0/2622230334 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:12.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.610+0000 7f8431f9b700 1 --2- 172.21.15.97:0/2622230334 >> v2:172.21.15.146:3300/0 conn(0x7f842c15c520 0x7f842c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.610+0000 7f843279c700 1 --2- 172.21.15.97:0/2622230334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f842c160b30 0x7f842c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.610+0000 7f843179a700 1 -- 172.21.15.97:0/2622230334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f842c160b30 msgr2=0x7f842c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.610+0000 7f843179a700 1 --2- 172.21.15.97:0/2622230334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f842c160b30 0x7f842c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.610+0000 7f843179a700 1 -- 172.21.15.97:0/2622230334 >> v2:172.21.15.146:3300/0 conn(0x7f842c15c520 msgr2=0x7f842c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.610+0000 7f843179a700 1 --2- 172.21.15.97:0/2622230334 >> v2:172.21.15.146:3300/0 conn(0x7f842c15c520 0x7f842c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.610+0000 7f843179a700 1 -- 172.21.15.97:0/2622230334 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0b6920 con 0x7f842c160180 2023-07-06T03:49:12.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.610+0000 7f843179a700 1 --2- 172.21.15.97:0/2622230334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842c160180 0x7f842c1605f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f842c0b4c70 tx=0x7f841c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a71da8367bbd7e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.610+0000 7f8430f99700 1 -- 172.21.15.97:0/2622230334 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c00b870 con 0x7f842c160180 2023-07-06T03:49:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.610+0000 7f8430f99700 1 -- 172.21.15.97:0/2622230334 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841c00b9d0 con 0x7f842c160180 2023-07-06T03:49:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.610+0000 7f8430f99700 1 -- 172.21.15.97:0/2622230334 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c00bd60 con 0x7f842c160180 2023-07-06T03:49:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.611+0000 7f84341ff700 1 -- 172.21.15.97:0/2622230334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842c160180 msgr2=0x7f842c1605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.611+0000 7f84341ff700 1 --2- 172.21.15.97:0/2622230334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842c160180 0x7f842c1605f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f842c0b4c70 tx=0x7f841c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.611+0000 7f84341ff700 1 -- 172.21.15.97:0/2622230334 shutdown_connections 2023-07-06T03:49:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.611+0000 7f84341ff700 1 --2- 172.21.15.97:0/2622230334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f842c160b30 0x7f842c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.611+0000 7f84341ff700 1 --2- 172.21.15.97:0/2622230334 >> v2:172.21.15.146:3300/0 conn(0x7f842c15c520 0x7f842c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.611+0000 7f84341ff700 1 --2- 172.21.15.97:0/2622230334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842c160180 0x7f842c1605f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.611+0000 7f84341ff700 1 -- 172.21.15.97:0/2622230334 >> 172.21.15.97:0/2622230334 conn(0x7f842c071790 msgr2=0x7f842c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:12.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.611+0000 7f84341ff700 1 -- 172.21.15.97:0/2622230334 shutdown_connections 2023-07-06T03:49:12.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.611+0000 7f84341ff700 1 -- 172.21.15.97:0/2622230334 wait complete. 2023-07-06T03:49:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.613+0000 7f84341ff700 1 Processor -- start 2023-07-06T03:49:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.613+0000 7f84341ff700 1 -- start start 2023-07-06T03:49:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.614+0000 7f84341ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f842c15c520 0x7f842c15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.614+0000 7f84341ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842c160180 0x7f842c0d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.614+0000 7f84341ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f842c160b30 0x7f842c158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.614+0000 7f84341ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c15b3c0 con 0x7f842c160180 2023-07-06T03:49:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.614+0000 7f84341ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0d2a30 con 0x7f842c160b30 2023-07-06T03:49:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.614+0000 7f84341ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0d2d10 con 0x7f842c15c520 2023-07-06T03:49:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.614+0000 7f843179a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842c160180 0x7f842c0d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.614+0000 7f843179a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842c160180 0x7f842c0d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33854/0 (socket says 172.21.15.97:33854) 2023-07-06T03:49:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.614+0000 7f843179a700 1 -- 172.21.15.97:0/2271135242 learned_addr learned my addr 172.21.15.97:0/2271135242 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.614+0000 7f843279c700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f842c160b30 0x7f842c158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.614+0000 7f8431f9b700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f842c15c520 0x7f842c15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.615+0000 7f843179a700 1 -- 172.21.15.97:0/2271135242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f842c15c520 msgr2=0x7f842c15ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.615+0000 7f843179a700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f842c15c520 0x7f842c15ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.615+0000 7f843179a700 1 -- 172.21.15.97:0/2271135242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f842c160b30 msgr2=0x7f842c158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.615+0000 7f843179a700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f842c160b30 0x7f842c158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.615+0000 7f843179a700 1 -- 172.21.15.97:0/2271135242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c009550 con 0x7f842c160180 2023-07-06T03:49:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.615+0000 7f843179a700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842c160180 0x7f842c0d1f70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.615+0000 7f84227fc700 1 -- 172.21.15.97:0/2271135242 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c004480 con 0x7f842c160180 2023-07-06T03:49:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.615+0000 7f84227fc700 1 -- 172.21.15.97:0/2271135242 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841c0045e0 con 0x7f842c160180 2023-07-06T03:49:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.615+0000 7f84227fc700 1 -- 172.21.15.97:0/2271135242 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c0038e0 con 0x7f842c160180 2023-07-06T03:49:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.615+0000 7f84341ff700 1 -- 172.21.15.97:0/2271135242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c0d2ff0 con 0x7f842c160180 2023-07-06T03:49:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.616+0000 7f84341ff700 1 -- 172.21.15.97:0/2271135242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c0d35b0 con 0x7f842c160180 2023-07-06T03:49:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.617+0000 7f84341ff700 1 -- 172.21.15.97:0/2271135242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c04e340 con 0x7f842c160180 2023-07-06T03:49:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.617+0000 7f84227fc700 1 -- 172.21.15.97:0/2271135242 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f841c021070 con 0x7f842c160180 2023-07-06T03:49:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.619+0000 7f84227fc700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f841006ace0 0x7f841006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.619+0000 7f84227fc700 1 -- 172.21.15.97:0/2271135242 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f841c08ab90 con 0x7f842c160180 2023-07-06T03:49:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.619+0000 7f8431f9b700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f841006ace0 0x7f841006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:12.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.620+0000 7f8431f9b700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f841006ace0 0x7f841006d190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8428006b20 tx=0x7f8428008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:12.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.623+0000 7f84227fc700 1 -- 172.21.15.97:0/2271135242 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f841c055b60 con 0x7f842c160180 2023-07-06T03:49:12.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.820+0000 7f84341ff700 1 -- 172.21.15.97:0/2271135242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f842c0d0480 con 0x7f842c160180 2023-07-06T03:49:12.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.821+0000 7f84227fc700 1 -- 172.21.15.97:0/2271135242 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18703 (secure 0 0 0) 0x7f841c015090 con 0x7f842c160180 2023-07-06T03:49:12.824 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:12.825 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":62,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:48:53.298133+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:48:43.594325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-06T03:48:44.371987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.3","mappings":[{"from":2,"to":5}]},{"pgid":"10.7","mappings":[{"from":4,"to":1}]},{"pgid":"10.16","mappings":[{"from":2,"to":1}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.1f","mappings":[{"from":2,"to":5}]},{"pgid":"10.22","mappings":[{"from":2,"to":7}]},{"pgid":"10.2d","mappings":[{"from":2,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:49:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.823+0000 7f84341ff700 1 -- 172.21.15.97:0/2271135242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f841006ace0 msgr2=0x7f841006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.823+0000 7f84341ff700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f841006ace0 0x7f841006d190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8428006b20 tx=0x7f8428008040 comp rx=0 tx=0).stop 2023-07-06T03:49:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.823+0000 7f84341ff700 1 -- 172.21.15.97:0/2271135242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842c160180 msgr2=0x7f842c0d1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.823+0000 7f84341ff700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842c160180 0x7f842c0d1f70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c004750 comp rx=0 tx=0).stop 2023-07-06T03:49:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.824+0000 7f84341ff700 1 -- 172.21.15.97:0/2271135242 shutdown_connections 2023-07-06T03:49:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.824+0000 7f84341ff700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f841006ace0 0x7f841006d190 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.824+0000 7f84341ff700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f842c160b30 0x7f842c158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.824+0000 7f84341ff700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f842c15c520 0x7f842c15ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.824+0000 7f84341ff700 1 --2- 172.21.15.97:0/2271135242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842c160180 0x7f842c0d1f70 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:12.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.824+0000 7f84341ff700 1 -- 172.21.15.97:0/2271135242 >> 172.21.15.97:0/2271135242 conn(0x7f842c071790 msgr2=0x7f842c0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:12.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.824+0000 7f84341ff700 1 -- 172.21.15.97:0/2271135242 shutdown_connections 2023-07-06T03:49:12.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:12.824+0000 7f84341ff700 1 -- 172.21.15.97:0/2271135242 wait complete. 2023-07-06T03:49:12.844 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T03:49:12.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:49:13.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.036+0000 7ff7921ff700 1 Processor -- start 2023-07-06T03:49:13.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.036+0000 7ff7921ff700 1 -- start start 2023-07-06T03:49:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.037+0000 7ff7921ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff78c15c920 0x7ff78c06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff7921ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff78c15ccf0 0x7ff78c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff7921ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff78c0b2d70 0x7ff78c06b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff7921ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78c0cfd00 con 0x7ff78c15ccf0 2023-07-06T03:49:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff7921ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78c0b28f0 con 0x7ff78c15c920 2023-07-06T03:49:13.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff7921ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c1610b0 con 0x7ff78c0b2d70 2023-07-06T03:49:13.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff783fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff78c15ccf0 0x7ff78c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff783fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff78c15ccf0 0x7ff78c06c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33856/0 (socket says 172.21.15.97:33856) 2023-07-06T03:49:13.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff783fff700 1 -- 172.21.15.97:0/1333067957 learned_addr learned my addr 172.21.15.97:0/1333067957 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:13.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff78bfff700 1 --2- 172.21.15.97:0/1333067957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff78c0b2d70 0x7ff78c06b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff78b7fe700 1 --2- 172.21.15.97:0/1333067957 >> v2:172.21.15.146:3300/0 conn(0x7ff78c15c920 0x7ff78c06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff783fff700 1 -- 172.21.15.97:0/1333067957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff78c0b2d70 msgr2=0x7ff78c06b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff783fff700 1 --2- 172.21.15.97:0/1333067957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff78c0b2d70 0x7ff78c06b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff783fff700 1 -- 172.21.15.97:0/1333067957 >> v2:172.21.15.146:3300/0 conn(0x7ff78c15c920 msgr2=0x7ff78c06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff783fff700 1 --2- 172.21.15.97:0/1333067957 >> v2:172.21.15.146:3300/0 conn(0x7ff78c15c920 0x7ff78c06c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.038+0000 7ff783fff700 1 -- 172.21.15.97:0/1333067957 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c161390 con 0x7ff78c15ccf0 2023-07-06T03:49:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.039+0000 7ff783fff700 1 --2- 172.21.15.97:0/1333067957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff78c15ccf0 0x7ff78c06c8d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff78c06f280 tx=0x7ff774009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b31ac26e7756e79 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.039+0000 7ff78affd700 1 -- 172.21.15.97:0/1333067957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77400b870 con 0x7ff78c15ccf0 2023-07-06T03:49:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.039+0000 7ff78affd700 1 -- 172.21.15.97:0/1333067957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77400b9d0 con 0x7ff78c15ccf0 2023-07-06T03:49:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.039+0000 7ff78affd700 1 -- 172.21.15.97:0/1333067957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77400bd60 con 0x7ff78c15ccf0 2023-07-06T03:49:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.039+0000 7ff7921ff700 1 -- 172.21.15.97:0/1333067957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff78c15ccf0 msgr2=0x7ff78c06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.039+0000 7ff7921ff700 1 --2- 172.21.15.97:0/1333067957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff78c15ccf0 0x7ff78c06c8d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff78c06f280 tx=0x7ff774009bd0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.040+0000 7ff7921ff700 1 -- 172.21.15.97:0/1333067957 shutdown_connections 2023-07-06T03:49:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.040+0000 7ff7921ff700 1 --2- 172.21.15.97:0/1333067957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff78c0b2d70 0x7ff78c06b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.040+0000 7ff7921ff700 1 --2- 172.21.15.97:0/1333067957 >> v2:172.21.15.146:3300/0 conn(0x7ff78c15c920 0x7ff78c06c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.040+0000 7ff7921ff700 1 --2- 172.21.15.97:0/1333067957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff78c15ccf0 0x7ff78c06c8d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.040+0000 7ff7921ff700 1 -- 172.21.15.97:0/1333067957 >> 172.21.15.97:0/1333067957 conn(0x7ff78c06ace0 msgr2=0x7ff78c06dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.040+0000 7ff7921ff700 1 -- 172.21.15.97:0/1333067957 shutdown_connections 2023-07-06T03:49:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.040+0000 7ff7921ff700 1 -- 172.21.15.97:0/1333067957 wait complete. 2023-07-06T03:49:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.041+0000 7ff7921ff700 1 Processor -- start 2023-07-06T03:49:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.041+0000 7ff7921ff700 1 -- start start 2023-07-06T03:49:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.042+0000 7ff7921ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff78c15c920 0x7ff78c0b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.042+0000 7ff7921ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff78c15ccf0 0x7ff78c1f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.042+0000 7ff7921ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff78c157fe0 0x7ff78c1f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.042+0000 7ff7921ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c1f5e00 con 0x7ff78c15ccf0 2023-07-06T03:49:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.042+0000 7ff7921ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c1f5af0 con 0x7ff78c157fe0 2023-07-06T03:49:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.042+0000 7ff7921ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c1f2a60 con 0x7ff78c15c920 2023-07-06T03:49:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.043+0000 7ff783fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff78c15ccf0 0x7ff78c1f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.043+0000 7ff783fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff78c15ccf0 0x7ff78c1f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33868/0 (socket says 172.21.15.97:33868) 2023-07-06T03:49:13.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.043+0000 7ff783fff700 1 -- 172.21.15.97:0/9027438 learned_addr learned my addr 172.21.15.97:0/9027438 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:13.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.043+0000 7ff78b7fe700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff78c15c920 0x7ff78c0b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.043+0000 7ff78bfff700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff78c157fe0 0x7ff78c1f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.043+0000 7ff783fff700 1 -- 172.21.15.97:0/9027438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff78c15c920 msgr2=0x7ff78c0b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.043+0000 7ff783fff700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff78c15c920 0x7ff78c0b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.043+0000 7ff783fff700 1 -- 172.21.15.97:0/9027438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff78c157fe0 msgr2=0x7ff78c1f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.043+0000 7ff783fff700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff78c157fe0 0x7ff78c1f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.043+0000 7ff783fff700 1 -- 172.21.15.97:0/9027438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff774009550 con 0x7ff78c15ccf0 2023-07-06T03:49:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.043+0000 7ff783fff700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff78c15ccf0 0x7ff78c1f2520 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff774000c00 tx=0x7ff774004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.044+0000 7ff788ff9700 1 -- 172.21.15.97:0/9027438 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774004310 con 0x7ff78c15ccf0 2023-07-06T03:49:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.044+0000 7ff788ff9700 1 -- 172.21.15.97:0/9027438 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77401e070 con 0x7ff78c15ccf0 2023-07-06T03:49:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.044+0000 7ff7921ff700 1 -- 172.21.15.97:0/9027438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c1f2d30 con 0x7ff78c15ccf0 2023-07-06T03:49:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.044+0000 7ff788ff9700 1 -- 172.21.15.97:0/9027438 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7740036a0 con 0x7ff78c15ccf0 2023-07-06T03:49:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.044+0000 7ff7921ff700 1 -- 172.21.15.97:0/9027438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c1f32f0 con 0x7ff78c15ccf0 2023-07-06T03:49:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.045+0000 7ff7921ff700 1 -- 172.21.15.97:0/9027438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c066a40 con 0x7ff78c15ccf0 2023-07-06T03:49:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.046+0000 7ff788ff9700 1 -- 172.21.15.97:0/9027438 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff774020030 con 0x7ff78c15ccf0 2023-07-06T03:49:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.047+0000 7ff788ff9700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff76c06ad30 0x7ff76c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.047+0000 7ff788ff9700 1 -- 172.21.15.97:0/9027438 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7ff77408a450 con 0x7ff78c15ccf0 2023-07-06T03:49:13.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.047+0000 7ff78b7fe700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff76c06ad30 0x7ff76c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.048+0000 7ff78b7fe700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff76c06ad30 0x7ff76c06d1e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff77c006b20 tx=0x7ff77c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:13.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.052+0000 7ff788ff9700 1 -- 172.21.15.97:0/9027438 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff7740554a0 con 0x7ff78c15ccf0 2023-07-06T03:49:13.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.294+0000 7ff7921ff700 1 -- 172.21.15.97:0/9027438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff78c1f5f40 con 0x7ff78c15ccf0 2023-07-06T03:49:13.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.295+0000 7ff788ff9700 1 -- 172.21.15.97:0/9027438 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4678 (secure 0 0 0) 0x7ff78c1f5f40 con 0x7ff78c15ccf0 2023-07-06T03:49:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:13.298 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1108873506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1108873506},{"type":"v1","addr":"172.21.15.146:6835","nonce":1108873506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2471899168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2471899168},{"type":"v1","addr":"172.21.15.97:6837","nonce":2471899168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2970792733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2970792733},{"type":"v1","addr":"172.21.15.97:6836","nonce":2970792733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:48:45.401033+0000","modified":"2023-07-06T03:48:48.930905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6399":{"gid":6399,"name":"d","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/2151424592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2151424592},{"type":"v1","addr":"172.21.15.146:6834","nonce":2151424592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-06T03:49:13.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 -- 172.21.15.97:0/9027438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff76c06ad30 msgr2=0x7ff76c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff76c06ad30 0x7ff76c06d1e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff77c006b20 tx=0x7ff77c008040 comp rx=0 tx=0).stop 2023-07-06T03:49:13.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 -- 172.21.15.97:0/9027438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff78c15ccf0 msgr2=0x7ff78c1f2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff78c15ccf0 0x7ff78c1f2520 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff774000c00 tx=0x7ff774004750 comp rx=0 tx=0).stop 2023-07-06T03:49:13.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 -- 172.21.15.97:0/9027438 shutdown_connections 2023-07-06T03:49:13.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff76c06ad30 0x7ff76c06d1e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff78c157fe0 0x7ff78c1f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff78c15c920 0x7ff78c0b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 --2- 172.21.15.97:0/9027438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff78c15ccf0 0x7ff78c1f2520 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 -- 172.21.15.97:0/9027438 >> 172.21.15.97:0/9027438 conn(0x7ff78c0b2d70 msgr2=0x7ff78c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:13.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 -- 172.21.15.97:0/9027438 shutdown_connections 2023-07-06T03:49:13.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.297+0000 7ff7921ff700 1 -- 172.21.15.97:0/9027438 wait complete. 2023-07-06T03:49:13.302 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 34 2023-07-06T03:49:13.318 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:49:13.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.509+0000 7fddfadff700 1 Processor -- start 2023-07-06T03:49:13.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.509+0000 7fddfadff700 1 -- start start 2023-07-06T03:49:13.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddfadff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fddf40cd390 0x7fddf4160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddfadff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fddf4160dd0 0x7fddf40b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddfadff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf40cdae0 con 0x7fddf415c520 2023-07-06T03:49:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddfadff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf40b6640 con 0x7fddf4160dd0 2023-07-06T03:49:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddfadff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf40b6920 con 0x7fddf40cd390 2023-07-06T03:49:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddf939c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fddf4160dd0 0x7fddf40b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddf939c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fddf4160dd0 0x7fddf40b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33884/0 (socket says 172.21.15.97:33884) 2023-07-06T03:49:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddf939c700 1 -- 172.21.15.97:0/558598980 learned_addr learned my addr 172.21.15.97:0/558598980 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:13.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddf3fff700 1 --2- 172.21.15.97:0/558598980 >> v2:172.21.15.146:3300/0 conn(0x7fddf40cd390 0x7fddf4160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddf939c700 1 -- 172.21.15.97:0/558598980 >> v2:172.21.15.146:3300/0 conn(0x7fddf40cd390 msgr2=0x7fddf4160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:13.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddf939c700 1 --2- 172.21.15.97:0/558598980 >> v2:172.21.15.146:3300/0 conn(0x7fddf40cd390 0x7fddf4160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddf939c700 1 -- 172.21.15.97:0/558598980 >> v1:172.21.15.146:6789/0 conn(0x7fddf415c520 legacy=0x7fddf4160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:13.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddf939c700 1 -- 172.21.15.97:0/558598980 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40b6c00 con 0x7fddf4160dd0 2023-07-06T03:49:13.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.511+0000 7fddf3fff700 1 --2- 172.21.15.97:0/558598980 >> v2:172.21.15.146:3300/0 conn(0x7fddf40cd390 0x7fddf4160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.512+0000 7fddf939c700 1 --2- 172.21.15.97:0/558598980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf4160dd0 0x7fddf40b7400 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fddf40b5930 tx=0x7fddec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f448ef51f00836 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.512+0000 7fddf37fe700 1 -- 172.21.15.97:0/558598980 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec00e070 con 0x7fddf4160dd0 2023-07-06T03:49:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.512+0000 7fddf37fe700 1 -- 172.21.15.97:0/558598980 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec003a20 con 0x7fddf4160dd0 2023-07-06T03:49:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.512+0000 7fddf37fe700 1 -- 172.21.15.97:0/558598980 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec003dd0 con 0x7fddf4160dd0 2023-07-06T03:49:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.512+0000 7fddfadff700 1 -- 172.21.15.97:0/558598980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf4160dd0 msgr2=0x7fddf40b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.512+0000 7fddfadff700 1 --2- 172.21.15.97:0/558598980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf4160dd0 0x7fddf40b7400 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fddf40b5930 tx=0x7fddec00b330 comp rx=0 tx=0).stop 2023-07-06T03:49:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.512+0000 7fddfadff700 1 -- 172.21.15.97:0/558598980 shutdown_connections 2023-07-06T03:49:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.512+0000 7fddfadff700 1 --2- 172.21.15.97:0/558598980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf4160dd0 0x7fddf40b7400 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.512+0000 7fddfadff700 1 --2- 172.21.15.97:0/558598980 >> v2:172.21.15.146:3300/0 conn(0x7fddf40cd390 0x7fddf4160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.512+0000 7fddfadff700 1 -- 172.21.15.97:0/558598980 >> 172.21.15.97:0/558598980 conn(0x7fddf4071790 msgr2=0x7fddf4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.513+0000 7fddfadff700 1 -- 172.21.15.97:0/558598980 shutdown_connections 2023-07-06T03:49:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.513+0000 7fddfadff700 1 -- 172.21.15.97:0/558598980 wait complete. 2023-07-06T03:49:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.514+0000 7fddfadff700 1 Processor -- start 2023-07-06T03:49:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.514+0000 7fddfadff700 1 -- start start 2023-07-06T03:49:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.515+0000 7fddfadff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddf40cd390 0x7fddf414d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.515+0000 7fddfadff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddf415c520 0x7fddf40d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.515+0000 7fddfadff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf4160dd0 0x7fddf40d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.515+0000 7fddfadff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40d1800 con 0x7fddf4160dd0 2023-07-06T03:49:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.515+0000 7fddfadff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf415e290 con 0x7fddf415c520 2023-07-06T03:49:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.515+0000 7fddfadff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf414d7c0 con 0x7fddf40cd390 2023-07-06T03:49:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.516+0000 7fddf939c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf4160dd0 0x7fddf40d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.516+0000 7fddf939c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf4160dd0 0x7fddf40d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33888/0 (socket says 172.21.15.97:33888) 2023-07-06T03:49:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.516+0000 7fddf939c700 1 -- 172.21.15.97:0/3757036123 learned_addr learned my addr 172.21.15.97:0/3757036123 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.516+0000 7fddf8b9b700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddf40cd390 0x7fddf414d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.516+0000 7fddf3fff700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddf415c520 0x7fddf40d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.516+0000 7fddf939c700 1 -- 172.21.15.97:0/3757036123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddf40cd390 msgr2=0x7fddf414d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.516+0000 7fddf939c700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddf40cd390 0x7fddf414d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.516+0000 7fddf939c700 1 -- 172.21.15.97:0/3757036123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddf415c520 msgr2=0x7fddf40d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.516+0000 7fddf939c700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddf415c520 0x7fddf40d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.516+0000 7fddf939c700 1 -- 172.21.15.97:0/3757036123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec009b30 con 0x7fddf4160dd0 2023-07-06T03:49:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.516+0000 7fddf939c700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf4160dd0 0x7fddf40d1180 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fddec015040 tx=0x7fddec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.517+0000 7fddf17fa700 1 -- 172.21.15.97:0/3757036123 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec0092e0 con 0x7fddf4160dd0 2023-07-06T03:49:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.517+0000 7fddf17fa700 1 -- 172.21.15.97:0/3757036123 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec004240 con 0x7fddf4160dd0 2023-07-06T03:49:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.517+0000 7fddf17fa700 1 -- 172.21.15.97:0/3757036123 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec0045f0 con 0x7fddf4160dd0 2023-07-06T03:49:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.517+0000 7fddfadff700 1 -- 172.21.15.97:0/3757036123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf415dbf0 con 0x7fddf4160dd0 2023-07-06T03:49:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.517+0000 7fddfadff700 1 -- 172.21.15.97:0/3757036123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf415deb0 con 0x7fddf4160dd0 2023-07-06T03:49:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.518+0000 7fddfadff700 1 -- 172.21.15.97:0/3757036123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf415e3d0 con 0x7fddf4160dd0 2023-07-06T03:49:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.519+0000 7fddf17fa700 1 -- 172.21.15.97:0/3757036123 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fddec019070 con 0x7fddf4160dd0 2023-07-06T03:49:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.520+0000 7fddf17fa700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fddd806ace0 0x7fddd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.521+0000 7fddf17fa700 1 -- 172.21.15.97:0/3757036123 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fddec08a3c0 con 0x7fddf4160dd0 2023-07-06T03:49:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.521+0000 7fddf8b9b700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fddd806ace0 0x7fddd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.521+0000 7fddf8b9b700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fddd806ace0 0x7fddd806d190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdde8007fd0 tx=0x7fdde8007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:13.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.525+0000 7fddf17fa700 1 -- 172.21.15.97:0/3757036123 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fddf415e3d0 con 0x7fddf4160dd0 2023-07-06T03:49:13.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.766+0000 7fddfadff700 1 -- 172.21.15.97:0/3757036123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddf415e3d0 con 0x7fddf4160dd0 2023-07-06T03:49:13.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.767+0000 7fddf17fa700 1 -- 172.21.15.97:0/3757036123 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4678 (secure 0 0 0) 0x7fddf415e3d0 con 0x7fddf4160dd0 2023-07-06T03:49:13.770 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:13.770 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1108873506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1108873506},{"type":"v1","addr":"172.21.15.146:6835","nonce":1108873506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2471899168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2471899168},{"type":"v1","addr":"172.21.15.97:6837","nonce":2471899168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2970792733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2970792733},{"type":"v1","addr":"172.21.15.97:6836","nonce":2970792733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:48:45.401033+0000","modified":"2023-07-06T03:48:48.930905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6399":{"gid":6399,"name":"d","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/2151424592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2151424592},{"type":"v1","addr":"172.21.15.146:6834","nonce":2151424592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-07-06T03:49:13.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 -- 172.21.15.97:0/3757036123 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fddd806ace0 msgr2=0x7fddd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fddd806ace0 0x7fddd806d190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdde8007fd0 tx=0x7fdde8007ec0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 -- 172.21.15.97:0/3757036123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf4160dd0 msgr2=0x7fddf40d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf4160dd0 0x7fddf40d1180 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fddec015040 tx=0x7fddec004ad0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 -- 172.21.15.97:0/3757036123 shutdown_connections 2023-07-06T03:49:13.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fddd806ace0 0x7fddd806d190 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf4160dd0 0x7fddf40d1180 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fddf40cd390 0x7fddf414d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 --2- 172.21.15.97:0/3757036123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fddf415c520 0x7fddf40d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 -- 172.21.15.97:0/3757036123 >> 172.21.15.97:0/3757036123 conn(0x7fddf4071790 msgr2=0x7fddf40c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:13.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 -- 172.21.15.97:0/3757036123 shutdown_connections 2023-07-06T03:49:13.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.769+0000 7fddfadff700 1 -- 172.21.15.97:0/3757036123 wait complete. 2023-07-06T03:49:13.774 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 34 2023-07-06T03:49:13.790 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:49:13.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.981+0000 7f64931ff700 1 Processor -- start 2023-07-06T03:49:13.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.981+0000 7f64931ff700 1 -- start start 2023-07-06T03:49:13.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.982+0000 7f64931ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 0x7f648c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.982+0000 7f64931ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648c06ddb0 con 0x7f648c15ce90 2023-07-06T03:49:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.982+0000 7f64931ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648c06bbf0 con 0x7f648c158960 2023-07-06T03:49:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.982+0000 7f64931ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648c06bed0 con 0x7f648c071a00 2023-07-06T03:49:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.983+0000 7f649179c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f648c15ce90 0x7f648c158d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39896/0 (socket says 172.21.15.97:39896) 2023-07-06T03:49:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.983+0000 7f649179c700 1 -- 172.21.15.97:0/2311519395 learned_addr learned my addr 172.21.15.97:0/2311519395 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.983+0000 7f648bfff700 1 --2- 172.21.15.97:0/2311519395 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 0x7f648c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.983+0000 7f648b7fe700 1 -- 172.21.15.97:0/2311519395 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020378393 0 0) 0x7f648c06ddb0 con 0x7f648c15ce90 2023-07-06T03:49:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.983+0000 7f648b7fe700 1 -- 172.21.15.97:0/2311519395 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f647c0035b0 con 0x7f648c15ce90 2023-07-06T03:49:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.983+0000 7f648bfff700 1 -- 172.21.15.97:0/2311519395 >> v1:172.21.15.146:6789/0 conn(0x7f648c158960 legacy=0x7f648c0712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.983+0000 7f648b7fe700 1 -- 172.21.15.97:0/2311519395 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771178720 0 0) 0x7f648c06bbf0 con 0x7f648c158960 2023-07-06T03:49:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.983+0000 7f648bfff700 1 -- 172.21.15.97:0/2311519395 >> v1:172.21.15.97:6789/0 conn(0x7f648c15ce90 legacy=0x7f648c158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.983+0000 7f648bfff700 1 -- 172.21.15.97:0/2311519395 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c06c1b0 con 0x7f648c071a00 2023-07-06T03:49:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.984+0000 7f648bfff700 1 --2- 172.21.15.97:0/2311519395 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 0x7f648c06d6f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6478009870 tx=0x7f6478009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27bd7c21f34bf3e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.984+0000 7f648b7fe700 1 -- 172.21.15.97:0/2311519395 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840019f0 con 0x7f648c071a00 2023-07-06T03:49:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.984+0000 7f648b7fe700 1 -- 172.21.15.97:0/2311519395 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6478004030 con 0x7f648c071a00 2023-07-06T03:49:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.984+0000 7f648b7fe700 1 -- 172.21.15.97:0/2311519395 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64780043a0 con 0x7f648c071a00 2023-07-06T03:49:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.984+0000 7f64931ff700 1 -- 172.21.15.97:0/2311519395 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 msgr2=0x7f648c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.984+0000 7f64931ff700 1 --2- 172.21.15.97:0/2311519395 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 0x7f648c06d6f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6478009870 tx=0x7f6478009b80 comp rx=0 tx=0).stop 2023-07-06T03:49:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.985+0000 7f64931ff700 1 -- 172.21.15.97:0/2311519395 shutdown_connections 2023-07-06T03:49:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.985+0000 7f64931ff700 1 --2- 172.21.15.97:0/2311519395 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 0x7f648c06d6f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.985+0000 7f64931ff700 1 -- 172.21.15.97:0/2311519395 >> 172.21.15.97:0/2311519395 conn(0x7f648c0ce8c0 msgr2=0x7f648c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:13.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.985+0000 7f64931ff700 1 -- 172.21.15.97:0/2311519395 shutdown_connections 2023-07-06T03:49:13.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.985+0000 7f64931ff700 1 -- 172.21.15.97:0/2311519395 wait complete. 2023-07-06T03:49:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.986+0000 7f64931ff700 1 Processor -- start 2023-07-06T03:49:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.987+0000 7f64931ff700 1 -- start start 2023-07-06T03:49:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.987+0000 7f64931ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 0x7f648c147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.987+0000 7f64931ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c158960 0x7f648c1e5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.987+0000 7f64931ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f648c15ce90 0x7f648c1e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.987+0000 7f64931ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648c0a8ab0 con 0x7f648c158960 2023-07-06T03:49:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.987+0000 7f64931ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648c147840 con 0x7f648c15ce90 2023-07-06T03:49:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.987+0000 7f64931ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648c1e4d40 con 0x7f648c071a00 2023-07-06T03:49:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f648bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c158960 0x7f648c1e5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f648bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c158960 0x7f648c1e5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33892/0 (socket says 172.21.15.97:33892) 2023-07-06T03:49:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f648bfff700 1 -- 172.21.15.97:0/3667542654 learned_addr learned my addr 172.21.15.97:0/3667542654 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f649179c700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f648c15ce90 0x7f648c1e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f6490f9b700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 0x7f648c147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f648bfff700 1 -- 172.21.15.97:0/3667542654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 msgr2=0x7f648c147300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f648bfff700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 0x7f648c147300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f648bfff700 1 -- 172.21.15.97:0/3667542654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f648c15ce90 msgr2=0x7f648c1e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f648bfff700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f648c15ce90 0x7f648c1e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f648bfff700 1 -- 172.21.15.97:0/3667542654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64780094a0 con 0x7f648c158960 2023-07-06T03:49:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f6490f9b700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 0x7f648c147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.988+0000 7f648bfff700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c158960 0x7f648c1e5520 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6478000c00 tx=0x7f6478004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.989+0000 7f64897fa700 1 -- 172.21.15.97:0/3667542654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478010590 con 0x7f648c158960 2023-07-06T03:49:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.989+0000 7f64897fa700 1 -- 172.21.15.97:0/3667542654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64780106f0 con 0x7f648c158960 2023-07-06T03:49:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.989+0000 7f64931ff700 1 -- 172.21.15.97:0/3667542654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648c1e3ca0 con 0x7f648c158960 2023-07-06T03:49:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.989+0000 7f64897fa700 1 -- 172.21.15.97:0/3667542654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478010a80 con 0x7f648c158960 2023-07-06T03:49:13.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.989+0000 7f64931ff700 1 -- 172.21.15.97:0/3667542654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648c1e4260 con 0x7f648c158960 2023-07-06T03:49:13.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.990+0000 7f64931ff700 1 -- 172.21.15.97:0/3667542654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648c067830 con 0x7f648c158960 2023-07-06T03:49:13.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.991+0000 7f64897fa700 1 -- 172.21.15.97:0/3667542654 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6478010be0 con 0x7f648c158960 2023-07-06T03:49:13.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.992+0000 7f64897fa700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647006ace0 0x7f647006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:13.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.993+0000 7f64897fa700 1 -- 172.21.15.97:0/3667542654 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f647808af70 con 0x7f648c158960 2023-07-06T03:49:13.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.993+0000 7f6490f9b700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647006ace0 0x7f647006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:13.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.993+0000 7f6490f9b700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647006ace0 0x7f647006d190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6480000f60 tx=0x7f6480009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:13.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:13.997+0000 7f64897fa700 1 -- 172.21.15.97:0/3667542654 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6478056070 con 0x7f648c158960 2023-07-06T03:49:14.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.196+0000 7f64931ff700 1 -- 172.21.15.97:0/3667542654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f648c04e340 con 0x7f648c158960 2023-07-06T03:49:14.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.197+0000 7f64897fa700 1 -- 172.21.15.97:0/3667542654 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18703 (secure 0 0 0) 0x7f6478059690 con 0x7f648c158960 2023-07-06T03:49:14.200 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:14.200 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":62,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:48:53.298133+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:48:43.594325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-07-06T03:48:44.371987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.3","mappings":[{"from":2,"to":5}]},{"pgid":"10.7","mappings":[{"from":4,"to":1}]},{"pgid":"10.16","mappings":[{"from":2,"to":1}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.1f","mappings":[{"from":2,"to":5}]},{"pgid":"10.22","mappings":[{"from":2,"to":7}]},{"pgid":"10.2d","mappings":[{"from":2,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:49:14.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 -- 172.21.15.97:0/3667542654 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647006ace0 msgr2=0x7f647006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647006ace0 0x7f647006d190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6480000f60 tx=0x7f6480009040 comp rx=0 tx=0).stop 2023-07-06T03:49:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 -- 172.21.15.97:0/3667542654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c158960 msgr2=0x7f648c1e5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c158960 0x7f648c1e5520 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6478000c00 tx=0x7f6478004750 comp rx=0 tx=0).stop 2023-07-06T03:49:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 -- 172.21.15.97:0/3667542654 shutdown_connections 2023-07-06T03:49:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647006ace0 0x7f647006d190 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f648c15ce90 0x7f648c1e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c071a00 0x7f648c147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 --2- 172.21.15.97:0/3667542654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c158960 0x7f648c1e5520 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 -- 172.21.15.97:0/3667542654 >> 172.21.15.97:0/3667542654 conn(0x7f648c0ce8c0 msgr2=0x7f648c0cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:14.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 -- 172.21.15.97:0/3667542654 shutdown_connections 2023-07-06T03:49:14.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.200+0000 7f64931ff700 1 -- 172.21.15.97:0/3667542654 wait complete. 2023-07-06T03:49:14.221 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.413+0000 7f54971ff700 1 Processor -- start 2023-07-06T03:49:14.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.414+0000 7f54971ff700 1 -- start start 2023-07-06T03:49:14.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f54971ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f549006b360 0x7f54900b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:14.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f54971ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54900d2210 0x7f54900d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:14.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f54971ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54900cff90 con 0x7f549015c610 2023-07-06T03:49:14.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f54971ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54900d0780 con 0x7f54900d2210 2023-07-06T03:49:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f54971ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549015c9e0 con 0x7f549006b360 2023-07-06T03:49:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f549579c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54900d2210 0x7f54900d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f549579c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54900d2210 0x7f54900d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33908/0 (socket says 172.21.15.97:33908) 2023-07-06T03:49:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f5494f9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f549015c610 0x7f54900b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39904/0 (socket says 172.21.15.97:39904) 2023-07-06T03:49:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f549579c700 1 -- 172.21.15.97:0/3701109222 learned_addr learned my addr 172.21.15.97:0/3701109222 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f548ffff700 1 --2- 172.21.15.97:0/3701109222 >> v2:172.21.15.146:3300/0 conn(0x7f549006b360 0x7f54900b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:14.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f549579c700 1 -- 172.21.15.97:0/3701109222 >> v2:172.21.15.146:3300/0 conn(0x7f549006b360 msgr2=0x7f54900b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:14.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f549579c700 1 --2- 172.21.15.97:0/3701109222 >> v2:172.21.15.146:3300/0 conn(0x7f549006b360 0x7f54900b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f549579c700 1 -- 172.21.15.97:0/3701109222 >> v1:172.21.15.97:6789/0 conn(0x7f549015c610 legacy=0x7f54900b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f549579c700 1 -- 172.21.15.97:0/3701109222 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54900b5650 con 0x7f54900d2210 2023-07-06T03:49:14.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.415+0000 7f548ffff700 1 --2- 172.21.15.97:0/3701109222 >> v2:172.21.15.146:3300/0 conn(0x7f549006b360 0x7f54900b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:14.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.416+0000 7f549579c700 1 --2- 172.21.15.97:0/3701109222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54900d2210 0x7f54900d1310 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f549006a590 tx=0x7f5488009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560a52d82764ed2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:14.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.416+0000 7f548f7fe700 1 -- 172.21.15.97:0/3701109222 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800b860 con 0x7f54900d2210 2023-07-06T03:49:14.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.416+0000 7f548f7fe700 1 -- 172.21.15.97:0/3701109222 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548800b9c0 con 0x7f54900d2210 2023-07-06T03:49:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.416+0000 7f548f7fe700 1 -- 172.21.15.97:0/3701109222 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800bd50 con 0x7f54900d2210 2023-07-06T03:49:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.416+0000 7f54971ff700 1 -- 172.21.15.97:0/3701109222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54900d2210 msgr2=0x7f54900d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.416+0000 7f54971ff700 1 --2- 172.21.15.97:0/3701109222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54900d2210 0x7f54900d1310 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f549006a590 tx=0x7f5488009c00 comp rx=0 tx=0).stop 2023-07-06T03:49:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.417+0000 7f54971ff700 1 -- 172.21.15.97:0/3701109222 shutdown_connections 2023-07-06T03:49:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.417+0000 7f54971ff700 1 --2- 172.21.15.97:0/3701109222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54900d2210 0x7f54900d1310 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.417+0000 7f54971ff700 1 --2- 172.21.15.97:0/3701109222 >> v2:172.21.15.146:3300/0 conn(0x7f549006b360 0x7f54900b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.417+0000 7f54971ff700 1 -- 172.21.15.97:0/3701109222 >> 172.21.15.97:0/3701109222 conn(0x7f549006a6c0 msgr2=0x7f549006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.417+0000 7f54971ff700 1 -- 172.21.15.97:0/3701109222 shutdown_connections 2023-07-06T03:49:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.417+0000 7f54971ff700 1 -- 172.21.15.97:0/3701109222 wait complete. 2023-07-06T03:49:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.418+0000 7f54971ff700 1 Processor -- start 2023-07-06T03:49:14.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.419+0000 7f54971ff700 1 -- start start 2023-07-06T03:49:14.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.419+0000 7f54971ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f549006b360 0x7f549015e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:14.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f54971ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54900d2210 0x7f549015b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:14.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f54971ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549015c610 0x7f549015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:14.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f54971ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54901f51a0 con 0x7f549015c610 2023-07-06T03:49:14.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f54971ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54901f5a40 con 0x7f54900d2210 2023-07-06T03:49:14.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f54971ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549015edb0 con 0x7f549006b360 2023-07-06T03:49:14.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f549579c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549015c610 0x7f549015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f549579c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549015c610 0x7f549015b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33912/0 (socket says 172.21.15.97:33912) 2023-07-06T03:49:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f549579c700 1 -- 172.21.15.97:0/3943364655 learned_addr learned my addr 172.21.15.97:0/3943364655 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f548ffff700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54900d2210 0x7f549015b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f5494f9b700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f549006b360 0x7f549015e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f549579c700 1 -- 172.21.15.97:0/3943364655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f549006b360 msgr2=0x7f549015e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f549579c700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f549006b360 0x7f549015e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f549579c700 1 -- 172.21.15.97:0/3943364655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54900d2210 msgr2=0x7f549015b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f549579c700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54900d2210 0x7f549015b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f549579c700 1 -- 172.21.15.97:0/3943364655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488009550 con 0x7f549015c610 2023-07-06T03:49:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.420+0000 7f549579c700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549015c610 0x7f549015b570 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5488009870 tx=0x7f5488004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.421+0000 7f548d7fa700 1 -- 172.21.15.97:0/3943364655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880042c0 con 0x7f549015c610 2023-07-06T03:49:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.421+0000 7f548d7fa700 1 -- 172.21.15.97:0/3943364655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548801f070 con 0x7f549015c610 2023-07-06T03:49:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.421+0000 7f548d7fa700 1 -- 172.21.15.97:0/3943364655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880036a0 con 0x7f549015c610 2023-07-06T03:49:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.421+0000 7f54971ff700 1 -- 172.21.15.97:0/3943364655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54901f53f0 con 0x7f549015c610 2023-07-06T03:49:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.421+0000 7f54971ff700 1 -- 172.21.15.97:0/3943364655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54901f5ed0 con 0x7f549015c610 2023-07-06T03:49:14.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.422+0000 7f54971ff700 1 -- 172.21.15.97:0/3943364655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54901f5bb0 con 0x7f549015c610 2023-07-06T03:49:14.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.423+0000 7f548d7fa700 1 -- 172.21.15.97:0/3943364655 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5488021030 con 0x7f549015c610 2023-07-06T03:49:14.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.424+0000 7f548d7fa700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f547406ace0 0x7f547406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:14.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.424+0000 7f548d7fa700 1 -- 172.21.15.97:0/3943364655 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f548808a920 con 0x7f549015c610 2023-07-06T03:49:14.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.424+0000 7f5494f9b700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f547406ace0 0x7f547406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:14.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.424+0000 7f5494f9b700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f547406ace0 0x7f547406d190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f54840012e0 tx=0x7f54840018a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:14.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.428+0000 7f548d7fa700 1 -- 172.21.15.97:0/3943364655 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5488055910 con 0x7f549015c610 2023-07-06T03:49:14.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.666+0000 7f54971ff700 1 -- 172.21.15.97:0/3943364655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f549004e340 con 0x7f549015c610 2023-07-06T03:49:14.670 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-07-06T03:49:14.667+0000 7fb463ff4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T03:49:14.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.672+0000 7f548d7fa700 1 -- 172.21.15.97:0/3943364655 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5488058f30 con 0x7f549015c610 2023-07-06T03:49:14.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 -- 172.21.15.97:0/3943364655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f547406ace0 msgr2=0x7f547406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f547406ace0 0x7f547406d190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f54840012e0 tx=0x7f54840018a0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 -- 172.21.15.97:0/3943364655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549015c610 msgr2=0x7f549015b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549015c610 0x7f549015b570 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5488009870 tx=0x7f5488004750 comp rx=0 tx=0).stop 2023-07-06T03:49:14.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 -- 172.21.15.97:0/3943364655 shutdown_connections 2023-07-06T03:49:14.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f547406ace0 0x7f547406d190 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f549015c610 0x7f549015b570 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f549006b360 0x7f549015e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 --2- 172.21.15.97:0/3943364655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54900d2210 0x7f549015b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 -- 172.21.15.97:0/3943364655 >> 172.21.15.97:0/3943364655 conn(0x7f549006a6c0 msgr2=0x7f54900d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:14.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 -- 172.21.15.97:0/3943364655 shutdown_connections 2023-07-06T03:49:14.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.675+0000 7f54971ff700 1 -- 172.21.15.97:0/3943364655 wait complete. 2023-07-06T03:49:14.680 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T03:49:14.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T03:49:14.746 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:49:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.891+0000 7fafb59ff700 1 Processor -- start 2023-07-06T03:49:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.891+0000 7fafb59ff700 1 -- start start 2023-07-06T03:49:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.891+0000 7fafb59ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafb00d2210 0x7fafb00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.891+0000 7fafb59ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb00cff90 con 0x7fafb015c610 2023-07-06T03:49:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.891+0000 7fafb59ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb00b5650 con 0x7fafb006b360 2023-07-06T03:49:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.891+0000 7fafb59ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb00d0780 con 0x7fafb00d2210 2023-07-06T03:49:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.891+0000 7fafaf7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafb00d2210 0x7fafb00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.891+0000 7fafaf7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafb00d2210 0x7fafb00b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33926/0 (socket says 172.21.15.97:33926) 2023-07-06T03:49:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.891+0000 7fafaf7fe700 1 -- 172.21.15.97:0/376666429 learned_addr learned my addr 172.21.15.97:0/376666429 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:14.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.892+0000 7fafaf7fe700 1 -- 172.21.15.97:0/376666429 >> v1:172.21.15.146:6789/0 conn(0x7fafb006b360 legacy=0x7fafb00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.892+0000 7fafaf7fe700 1 -- 172.21.15.97:0/376666429 >> v1:172.21.15.97:6789/0 conn(0x7fafb015c610 legacy=0x7fafb00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.892+0000 7fafaf7fe700 1 -- 172.21.15.97:0/376666429 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb00b5400 con 0x7fafb00d2210 2023-07-06T03:49:14.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.892+0000 7fafadffb700 1 -- 172.21.15.97:0/376666429 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994800926 0 0) 0x7fafb00b5650 con 0x7fafb006b360 2023-07-06T03:49:14.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.892+0000 7fafaf7fe700 1 --2- 172.21.15.97:0/376666429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb00d2210 0x7fafb00b4ec0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009e50 tx=0x7fafa800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac97563954e11759 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:14.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.892+0000 7fafadffb700 1 -- 172.21.15.97:0/376666429 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa800e070 con 0x7fafb00d2210 2023-07-06T03:49:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.892+0000 7fafadffb700 1 -- 172.21.15.97:0/376666429 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa800b9c0 con 0x7fafb00d2210 2023-07-06T03:49:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.892+0000 7fafadffb700 1 -- 172.21.15.97:0/376666429 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa800bd70 con 0x7fafb00d2210 2023-07-06T03:49:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.893+0000 7fafb59ff700 1 -- 172.21.15.97:0/376666429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb00d2210 msgr2=0x7fafb00b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.893+0000 7fafb59ff700 1 --2- 172.21.15.97:0/376666429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb00d2210 0x7fafb00b4ec0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009e50 tx=0x7fafa800b330 comp rx=0 tx=0).stop 2023-07-06T03:49:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.893+0000 7fafb59ff700 1 -- 172.21.15.97:0/376666429 shutdown_connections 2023-07-06T03:49:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.893+0000 7fafb59ff700 1 --2- 172.21.15.97:0/376666429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb00d2210 0x7fafb00b4ec0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.893+0000 7fafb59ff700 1 -- 172.21.15.97:0/376666429 >> 172.21.15.97:0/376666429 conn(0x7fafb006a6c0 msgr2=0x7fafb006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.893+0000 7fafb59ff700 1 -- 172.21.15.97:0/376666429 shutdown_connections 2023-07-06T03:49:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.893+0000 7fafb59ff700 1 -- 172.21.15.97:0/376666429 wait complete. 2023-07-06T03:49:14.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.895+0000 7fafb59ff700 1 Processor -- start 2023-07-06T03:49:14.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.895+0000 7fafb59ff700 1 -- start start 2023-07-06T03:49:14.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafb59ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafb006b360 0x7fafb00cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:14.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafb59ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafb00d2210 0x7fafb01f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:14.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafb59ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb015c610 0x7fafb00c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:14.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafb59ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00cbf60 con 0x7fafb015c610 2023-07-06T03:49:14.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafb59ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00cb350 con 0x7fafb006b360 2023-07-06T03:49:14.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafb59ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00cb630 con 0x7fafb00d2210 2023-07-06T03:49:14.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafaf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb015c610 0x7fafb00c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:14.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafae7fc700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafb00d2210 0x7fafb01f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:14.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafaf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb015c610 0x7fafb00c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33930/0 (socket says 172.21.15.97:33930) 2023-07-06T03:49:14.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafaf7fe700 1 -- 172.21.15.97:0/1383300966 learned_addr learned my addr 172.21.15.97:0/1383300966 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:14.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafaeffd700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafb006b360 0x7fafb00cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:14.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafaf7fe700 1 -- 172.21.15.97:0/1383300966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafb00d2210 msgr2=0x7fafb01f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafaf7fe700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafb00d2210 0x7fafb01f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafaf7fe700 1 -- 172.21.15.97:0/1383300966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafb006b360 msgr2=0x7fafb00cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:14.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafaf7fe700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafb006b360 0x7fafb00cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:14.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.896+0000 7fafaf7fe700 1 -- 172.21.15.97:0/1383300966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa8009b30 con 0x7fafb015c610 2023-07-06T03:49:14.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.897+0000 7fafaf7fe700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb015c610 0x7fafb00c6860 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fafa800b730 tx=0x7fafa8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:14.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.897+0000 7faf97fff700 1 -- 172.21.15.97:0/1383300966 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa801c640 con 0x7fafb015c610 2023-07-06T03:49:14.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.897+0000 7faf97fff700 1 -- 172.21.15.97:0/1383300966 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa801c7c0 con 0x7fafb015c610 2023-07-06T03:49:14.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.897+0000 7fafb59ff700 1 -- 172.21.15.97:0/1383300966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb00cb910 con 0x7fafb015c610 2023-07-06T03:49:14.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.897+0000 7faf97fff700 1 -- 172.21.15.97:0/1383300966 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa801cb70 con 0x7fafb015c610 2023-07-06T03:49:14.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.897+0000 7fafb59ff700 1 -- 172.21.15.97:0/1383300966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb01f5040 con 0x7fafb015c610 2023-07-06T03:49:14.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.899+0000 7fafb59ff700 1 -- 172.21.15.97:0/1383300966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb00cc0d0 con 0x7fafb015c610 2023-07-06T03:49:14.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.899+0000 7faf97fff700 1 -- 172.21.15.97:0/1383300966 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fafa801ccd0 con 0x7fafb015c610 2023-07-06T03:49:14.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.900+0000 7faf97fff700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf9c06ace0 0x7faf9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:14.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.900+0000 7faf97fff700 1 -- 172.21.15.97:0/1383300966 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6351+0+0 (secure 0 0 0) 0x7fafa808af00 con 0x7fafb015c610 2023-07-06T03:49:14.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.900+0000 7fafaeffd700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf9c06ace0 0x7faf9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:14.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.901+0000 7fafaeffd700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf9c06ace0 0x7faf9c06d190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fafa0007fd0 tx=0x7fafa0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:14.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:14.905+0000 7faf97fff700 1 -- 172.21.15.97:0/1383300966 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fafa8055e80 con 0x7fafb015c610 2023-07-06T03:49:15.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.146+0000 7fafb59ff700 1 -- 172.21.15.97:0/1383300966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fafb004e340 con 0x7fafb015c610 2023-07-06T03:49:15.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.727+0000 7faf97fff700 1 -- 172.21.15.97:0/1383300966 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7fafa80594a0 con 0x7fafb015c610 2023-07-06T03:49:15.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 -- 172.21.15.97:0/1383300966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf9c06ace0 msgr2=0x7faf9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:15.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf9c06ace0 0x7faf9c06d190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fafa0007fd0 tx=0x7fafa0009040 comp rx=0 tx=0).stop 2023-07-06T03:49:15.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 -- 172.21.15.97:0/1383300966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb015c610 msgr2=0x7fafb00c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:15.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb015c610 0x7fafb00c6860 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fafa800b730 tx=0x7fafa8004ad0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 -- 172.21.15.97:0/1383300966 shutdown_connections 2023-07-06T03:49:15.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf9c06ace0 0x7faf9c06d190 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb015c610 0x7fafb00c6860 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafb006b360 0x7fafb00cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 --2- 172.21.15.97:0/1383300966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fafb00d2210 0x7fafb01f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 -- 172.21.15.97:0/1383300966 >> 172.21.15.97:0/1383300966 conn(0x7fafb006a6c0 msgr2=0x7fafb015b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:15.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 -- 172.21.15.97:0/1383300966 shutdown_connections 2023-07-06T03:49:15.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.729+0000 7fafb59ff700 1 -- 172.21.15.97:0/1383300966 wait complete. 2023-07-06T03:49:15.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T03:49:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.948+0000 7f3ed99ff700 1 Processor -- start 2023-07-06T03:49:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.948+0000 7f3ed99ff700 1 -- start start 2023-07-06T03:49:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.949+0000 7f3ed99ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3ed415c670 0x7f3ed40cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.949+0000 7f3ed99ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ed40cd4c0 0x7f3ed406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.949+0000 7f3ed99ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ed40716a0 0x7f3ed406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.949+0000 7f3ed99ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed40cdc40 con 0x7f3ed40cd4c0 2023-07-06T03:49:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.949+0000 7f3ed99ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed40cd960 con 0x7f3ed415c670 2023-07-06T03:49:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.949+0000 7f3ed99ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed4070bc0 con 0x7f3ed40716a0 2023-07-06T03:49:15.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.949+0000 7f3ed27fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ed40cd4c0 0x7f3ed406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:15.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.949+0000 7f3ed27fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ed40cd4c0 0x7f3ed406a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33944/0 (socket says 172.21.15.97:33944) 2023-07-06T03:49:15.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.949+0000 7f3ed27fc700 1 -- 172.21.15.97:0/1241472236 learned_addr learned my addr 172.21.15.97:0/1241472236 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:15.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed2ffd700 1 --2- 172.21.15.97:0/1241472236 >> v2:172.21.15.146:3300/0 conn(0x7f3ed415c670 0x7f3ed40cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:15.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed37fe700 1 --2- 172.21.15.97:0/1241472236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ed40716a0 0x7f3ed406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:15.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed27fc700 1 -- 172.21.15.97:0/1241472236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ed40716a0 msgr2=0x7f3ed406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:15.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed27fc700 1 --2- 172.21.15.97:0/1241472236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ed40716a0 0x7f3ed406a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed27fc700 1 -- 172.21.15.97:0/1241472236 >> v2:172.21.15.146:3300/0 conn(0x7f3ed415c670 msgr2=0x7f3ed40cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:15.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed27fc700 1 --2- 172.21.15.97:0/1241472236 >> v2:172.21.15.146:3300/0 conn(0x7f3ed415c670 0x7f3ed40cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed27fc700 1 -- 172.21.15.97:0/1241472236 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed4070ea0 con 0x7f3ed40cd4c0 2023-07-06T03:49:15.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed2ffd700 1 --2- 172.21.15.97:0/1241472236 >> v2:172.21.15.146:3300/0 conn(0x7f3ed415c670 0x7f3ed40cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:15.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed27fc700 1 --2- 172.21.15.97:0/1241472236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed40cd4c0 0x7f3ed406a060 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4068060 tx=0x7f3ec8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57170c73780a13d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:15.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed1ffb700 1 -- 172.21.15.97:0/1241472236 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800b870 con 0x7f3ed40cd4c0 2023-07-06T03:49:15.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed1ffb700 1 -- 172.21.15.97:0/1241472236 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec800b9d0 con 0x7f3ed40cd4c0 2023-07-06T03:49:15.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.950+0000 7f3ed1ffb700 1 -- 172.21.15.97:0/1241472236 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800bd60 con 0x7f3ed40cd4c0 2023-07-06T03:49:15.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.951+0000 7f3ed99ff700 1 -- 172.21.15.97:0/1241472236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed40cd4c0 msgr2=0x7f3ed406a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:15.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.951+0000 7f3ed99ff700 1 --2- 172.21.15.97:0/1241472236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed40cd4c0 0x7f3ed406a060 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4068060 tx=0x7f3ec8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.951+0000 7f3ed99ff700 1 -- 172.21.15.97:0/1241472236 shutdown_connections 2023-07-06T03:49:15.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.951+0000 7f3ed99ff700 1 --2- 172.21.15.97:0/1241472236 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ed40716a0 0x7f3ed406a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.951+0000 7f3ed99ff700 1 --2- 172.21.15.97:0/1241472236 >> v2:172.21.15.146:3300/0 conn(0x7f3ed415c670 0x7f3ed40cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.951+0000 7f3ed99ff700 1 --2- 172.21.15.97:0/1241472236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed40cd4c0 0x7f3ed406a060 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.951+0000 7f3ed99ff700 1 -- 172.21.15.97:0/1241472236 >> 172.21.15.97:0/1241472236 conn(0x7f3ed40d05b0 msgr2=0x7f3ed40cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:15.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.951+0000 7f3ed99ff700 1 -- 172.21.15.97:0/1241472236 shutdown_connections 2023-07-06T03:49:15.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.951+0000 7f3ed99ff700 1 -- 172.21.15.97:0/1241472236 wait complete. 2023-07-06T03:49:15.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.953+0000 7f3ed99ff700 1 Processor -- start 2023-07-06T03:49:15.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.953+0000 7f3ed99ff700 1 -- start start 2023-07-06T03:49:15.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.954+0000 7f3ed99ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed40716a0 0x7f3ed4157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:15.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.954+0000 7f3ed99ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ed40cd4c0 0x7f3ed41f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:15.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.954+0000 7f3ed99ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ed415c670 0x7f3ed40ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:15.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.954+0000 7f3ed99ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed4158020 con 0x7f3ed40716a0 2023-07-06T03:49:15.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.954+0000 7f3ed99ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40b3470 con 0x7f3ed40cd4c0 2023-07-06T03:49:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.954+0000 7f3ed99ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed41e2530 con 0x7f3ed415c670 2023-07-06T03:49:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.954+0000 7f3ed2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed40716a0 0x7f3ed4157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.954+0000 7f3ed2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed40716a0 0x7f3ed4157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33956/0 (socket says 172.21.15.97:33956) 2023-07-06T03:49:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.954+0000 7f3ed2ffd700 1 -- 172.21.15.97:0/3413921491 learned_addr learned my addr 172.21.15.97:0/3413921491 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.954+0000 7f3ed37fe700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ed415c670 0x7f3ed40ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.954+0000 7f3ed27fc700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ed40cd4c0 0x7f3ed41f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.955+0000 7f3ed2ffd700 1 -- 172.21.15.97:0/3413921491 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ed415c670 msgr2=0x7f3ed40ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.955+0000 7f3ed2ffd700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ed415c670 0x7f3ed40ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.955+0000 7f3ed2ffd700 1 -- 172.21.15.97:0/3413921491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ed40cd4c0 msgr2=0x7f3ed41f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.955+0000 7f3ed2ffd700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ed40cd4c0 0x7f3ed41f2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.955+0000 7f3ed2ffd700 1 -- 172.21.15.97:0/3413921491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec8009550 con 0x7f3ed40716a0 2023-07-06T03:49:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.955+0000 7f3ed2ffd700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed40716a0 0x7f3ed4157ae0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e810 tx=0x7f3ec400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.955+0000 7f3ebbfff700 1 -- 172.21.15.97:0/3413921491 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec400c020 con 0x7f3ed40716a0 2023-07-06T03:49:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.955+0000 7f3ebbfff700 1 -- 172.21.15.97:0/3413921491 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4004250 con 0x7f3ed40716a0 2023-07-06T03:49:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.955+0000 7f3ebbfff700 1 -- 172.21.15.97:0/3413921491 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40036a0 con 0x7f3ed40716a0 2023-07-06T03:49:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.955+0000 7f3ed99ff700 1 -- 172.21.15.97:0/3413921491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed41f28d0 con 0x7f3ed40716a0 2023-07-06T03:49:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.956+0000 7f3ed99ff700 1 -- 172.21.15.97:0/3413921491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed41f2e90 con 0x7f3ed40716a0 2023-07-06T03:49:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.956+0000 7f3ed99ff700 1 -- 172.21.15.97:0/3413921491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed404e340 con 0x7f3ed40716a0 2023-07-06T03:49:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.957+0000 7f3ebbfff700 1 -- 172.21.15.97:0/3413921491 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ec4003800 con 0x7f3ed40716a0 2023-07-06T03:49:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.958+0000 7f3ebbfff700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3ec006ace0 0x7f3ec006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.958+0000 7f3ebbfff700 1 -- 172.21.15.97:0/3413921491 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6351+0+0 (secure 0 0 0) 0x7f3ec40890b0 con 0x7f3ed40716a0 2023-07-06T03:49:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.959+0000 7f3ed27fc700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3ec006ace0 0x7f3ec006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.959+0000 7f3ed27fc700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3ec006ace0 0x7f3ec006d190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8000c00 tx=0x7f3ec8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:15.962+0000 7f3ebbfff700 1 -- 172.21.15.97:0/3413921491 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3ec4054030 con 0x7f3ed40716a0 2023-07-06T03:49:16.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.160+0000 7f3ed99ff700 1 -- 172.21.15.97:0/3413921491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3ed406af10 con 0x7f3ed40716a0 2023-07-06T03:49:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.731+0000 7f3ebbfff700 1 -- 172.21.15.97:0/3413921491 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v64) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3ec4057650 con 0x7f3ed40716a0 2023-07-06T03:49:16.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.735+0000 7f3ed99ff700 1 -- 172.21.15.97:0/3413921491 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3ec006ace0 msgr2=0x7f3ec006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:16.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.735+0000 7f3ed99ff700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3ec006ace0 0x7f3ec006d190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8000c00 tx=0x7f3ec8008030 comp rx=0 tx=0).stop 2023-07-06T03:49:16.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.735+0000 7f3ed99ff700 1 -- 172.21.15.97:0/3413921491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed40716a0 msgr2=0x7f3ed4157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:16.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.735+0000 7f3ed99ff700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed40716a0 0x7f3ed4157ae0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e810 tx=0x7f3ec400ebd0 comp rx=0 tx=0).stop 2023-07-06T03:49:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.736+0000 7f3ed99ff700 1 -- 172.21.15.97:0/3413921491 shutdown_connections 2023-07-06T03:49:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.736+0000 7f3ed99ff700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3ec006ace0 0x7f3ec006d190 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.736+0000 7f3ed99ff700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ed415c670 0x7f3ed40ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.736+0000 7f3ed99ff700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed40716a0 0x7f3ed4157ae0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.736+0000 7f3ed99ff700 1 --2- 172.21.15.97:0/3413921491 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ed40cd4c0 0x7f3ed41f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.736+0000 7f3ed99ff700 1 -- 172.21.15.97:0/3413921491 >> 172.21.15.97:0/3413921491 conn(0x7f3ed40d05b0 msgr2=0x7f3ed40ca220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.736+0000 7f3ed99ff700 1 -- 172.21.15.97:0/3413921491 shutdown_connections 2023-07-06T03:49:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.736+0000 7f3ed99ff700 1 -- 172.21.15.97:0/3413921491 wait complete. 2023-07-06T03:49:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-07-06T03:49:16.760 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T03:49:16.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.961+0000 7f9a1c1ff700 1 Processor -- start 2023-07-06T03:49:16.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.961+0000 7f9a1c1ff700 1 -- start start 2023-07-06T03:49:16.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.962+0000 7f9a1c1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9a1415c640 0x7f9a140cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:16.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.962+0000 7f9a1c1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a140cfdb0 0x7f9a140b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:16.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.962+0000 7f9a1c1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1415ca10 con 0x7f9a140b5270 2023-07-06T03:49:16.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.962+0000 7f9a1c1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a140b56e0 con 0x7f9a1415c640 2023-07-06T03:49:16.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.962+0000 7f9a1c1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140cda40 con 0x7f9a140cfdb0 2023-07-06T03:49:16.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.963+0000 7f9a19f9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9a1415c640 0x7f9a140cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:16.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.963+0000 7f9a1a79c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a140cfdb0 0x7f9a140b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:16.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.963+0000 7f9a1979a700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9a140b5270 0x7f9a140cf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:51756/0 (socket says 172.21.15.97:51756) 2023-07-06T03:49:16.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.963+0000 7f9a1979a700 1 -- 172.21.15.97:0/37360672 learned_addr learned my addr 172.21.15.97:0/37360672 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:16.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.963+0000 7f9a1a79c700 1 -- 172.21.15.97:0/37360672 >> v2:172.21.15.146:3300/0 conn(0x7f9a1415c640 msgr2=0x7f9a140cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:16.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.963+0000 7f9a1a79c700 1 --2- 172.21.15.97:0/37360672 >> v2:172.21.15.146:3300/0 conn(0x7f9a1415c640 0x7f9a140cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:16.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.963+0000 7f9a1a79c700 1 -- 172.21.15.97:0/37360672 >> v1:172.21.15.146:6789/0 conn(0x7f9a140b5270 legacy=0x7f9a140cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:16.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.963+0000 7f9a1a79c700 1 -- 172.21.15.97:0/37360672 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140b65f0 con 0x7f9a140cfdb0 2023-07-06T03:49:16.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.963+0000 7f9a19f9b700 1 --2- 172.21.15.97:0/37360672 >> v2:172.21.15.146:3300/0 conn(0x7f9a1415c640 0x7f9a140cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:16.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.963+0000 7f9a1a79c700 1 --2- 172.21.15.97:0/37360672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a140cfdb0 0x7f9a140b73b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009d90 tx=0x7f9a0c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a73b246e1abf8a86 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:16.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.964+0000 7f9a18f99700 1 -- 172.21.15.97:0/37360672 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00e070 con 0x7f9a140cfdb0 2023-07-06T03:49:16.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.964+0000 7f9a18f99700 1 -- 172.21.15.97:0/37360672 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c00bb20 con 0x7f9a140cfdb0 2023-07-06T03:49:16.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.964+0000 7f9a18f99700 1 -- 172.21.15.97:0/37360672 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c012430 con 0x7f9a140cfdb0 2023-07-06T03:49:16.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.964+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/37360672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a140cfdb0 msgr2=0x7f9a140b73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:16.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.964+0000 7f9a1c1ff700 1 --2- 172.21.15.97:0/37360672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a140cfdb0 0x7f9a140b73b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009d90 tx=0x7f9a0c00b330 comp rx=0 tx=0).stop 2023-07-06T03:49:16.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.964+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/37360672 shutdown_connections 2023-07-06T03:49:16.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.964+0000 7f9a1c1ff700 1 --2- 172.21.15.97:0/37360672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a140cfdb0 0x7f9a140b73b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:16.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.964+0000 7f9a1c1ff700 1 --2- 172.21.15.97:0/37360672 >> v2:172.21.15.146:3300/0 conn(0x7f9a1415c640 0x7f9a140cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:16.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.964+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/37360672 >> 172.21.15.97:0/37360672 conn(0x7f9a14071790 msgr2=0x7f9a14071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:16.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.964+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/37360672 shutdown_connections 2023-07-06T03:49:16.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.964+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/37360672 wait complete. 2023-07-06T03:49:16.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.966+0000 7f9a1c1ff700 1 Processor -- start 2023-07-06T03:49:16.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.966+0000 7f9a1c1ff700 1 -- start start 2023-07-06T03:49:16.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.967+0000 7f9a1c1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a140b5270 0x7f9a140ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:16.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.967+0000 7f9a1c1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a140cfdb0 0x7f9a141f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.967+0000 7f9a1c1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a1415c640 0x7f9a140b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.967+0000 7f9a1c1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140d2410 con 0x7f9a140cfdb0 2023-07-06T03:49:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.967+0000 7f9a1c1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140d1ae0 con 0x7f9a1415c640 2023-07-06T03:49:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.967+0000 7f9a1c1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140d0bf0 con 0x7f9a140b5270 2023-07-06T03:49:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a1979a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a140cfdb0 0x7f9a141f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a1979a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a140cfdb0 0x7f9a141f1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33970/0 (socket says 172.21.15.97:33970) 2023-07-06T03:49:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a1979a700 1 -- 172.21.15.97:0/2346710685 learned_addr learned my addr 172.21.15.97:0/2346710685 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a19f9b700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a140b5270 0x7f9a140ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a1a79c700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a1415c640 0x7f9a140b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a1979a700 1 -- 172.21.15.97:0/2346710685 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a140b5270 msgr2=0x7f9a140ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a1979a700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a140b5270 0x7f9a140ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a1979a700 1 -- 172.21.15.97:0/2346710685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a1415c640 msgr2=0x7f9a140b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a1979a700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a1415c640 0x7f9a140b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a1979a700 1 -- 172.21.15.97:0/2346710685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a100094a0 con 0x7f9a140cfdb0 2023-07-06T03:49:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a1979a700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a140cfdb0 0x7f9a141f1740 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c4f0 tx=0x7f9a0400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a0a7fc700 1 -- 172.21.15.97:0/2346710685 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04005dc0 con 0x7f9a140cfdb0 2023-07-06T03:49:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.968+0000 7f9a0a7fc700 1 -- 172.21.15.97:0/2346710685 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a040070d0 con 0x7f9a140cfdb0 2023-07-06T03:49:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.969+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/2346710685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c009a70 con 0x7f9a140cfdb0 2023-07-06T03:49:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.969+0000 7f9a0a7fc700 1 -- 172.21.15.97:0/2346710685 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04007440 con 0x7f9a140cfdb0 2023-07-06T03:49:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.969+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/2346710685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a140d0f80 con 0x7f9a140cfdb0 2023-07-06T03:49:16.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.970+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/2346710685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1404e340 con 0x7f9a140cfdb0 2023-07-06T03:49:16.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.970+0000 7f9a0a7fc700 1 -- 172.21.15.97:0/2346710685 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a040075a0 con 0x7f9a140cfdb0 2023-07-06T03:49:16.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.971+0000 7f9a0a7fc700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99f806ace0 0x7f99f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:16.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.972+0000 7f9a0a7fc700 1 -- 172.21.15.97:0/2346710685 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5913+0+0 (secure 0 0 0) 0x7f9a04088710 con 0x7f9a140cfdb0 2023-07-06T03:49:16.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.972+0000 7f9a19f9b700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99f806ace0 0x7f99f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:16.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.972+0000 7f9a19f9b700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99f806ace0 0x7f99f806d190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009cd0 tx=0x7f9a10009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:16.976+0000 7f9a0a7fc700 1 -- 172.21.15.97:0/2346710685 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9a04053860 con 0x7f9a140cfdb0 2023-07-06T03:49:17.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.171+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/2346710685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9a140d1c20 con 0x7f9a140cfdb0 2023-07-06T03:49:17.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.750+0000 7f9a0a7fc700 1 -- 172.21.15.97:0/2346710685 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v65) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9a04056e80 con 0x7f9a140cfdb0 2023-07-06T03:49:17.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.755+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/2346710685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99f806ace0 msgr2=0x7f99f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:17.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.755+0000 7f9a1c1ff700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99f806ace0 0x7f99f806d190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009cd0 tx=0x7f9a10009210 comp rx=0 tx=0).stop 2023-07-06T03:49:17.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.755+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/2346710685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a140cfdb0 msgr2=0x7f9a141f1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.755+0000 7f9a1c1ff700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a140cfdb0 0x7f9a141f1740 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c4f0 tx=0x7f9a0400c800 comp rx=0 tx=0).stop 2023-07-06T03:49:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.756+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/2346710685 shutdown_connections 2023-07-06T03:49:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.756+0000 7f9a1c1ff700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99f806ace0 0x7f99f806d190 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:17.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.756+0000 7f9a1c1ff700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a1415c640 0x7f9a140b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:17.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.756+0000 7f9a1c1ff700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a140b5270 0x7f9a140ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:17.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.756+0000 7f9a1c1ff700 1 --2- 172.21.15.97:0/2346710685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a140cfdb0 0x7f9a141f1740 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:17.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.756+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/2346710685 >> 172.21.15.97:0/2346710685 conn(0x7f9a14071790 msgr2=0x7f9a1415b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:17.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.756+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/2346710685 shutdown_connections 2023-07-06T03:49:17.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.756+0000 7f9a1c1ff700 1 -- 172.21.15.97:0/2346710685 wait complete. 2023-07-06T03:49:17.760 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-07-06T03:49:17.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2023-07-06T03:49:17.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.976+0000 7fdc9d9ff700 1 Processor -- start 2023-07-06T03:49:17.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.976+0000 7fdc9d9ff700 1 -- start start 2023-07-06T03:49:17.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.978+0000 7fdc9d9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdc9815c610 0x7fdc980b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:17.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.978+0000 7fdc9d9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdc9806b360 0x7fdc980b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:17.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.978+0000 7fdc9d9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc980cff90 con 0x7fdc980d2210 2023-07-06T03:49:17.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.978+0000 7fdc9d9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc980d0780 con 0x7fdc9806b360 2023-07-06T03:49:17.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.978+0000 7fdc9d9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9815c9e0 con 0x7fdc9815c610 2023-07-06T03:49:17.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.978+0000 7fdc967fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdc9806b360 0x7fdc980b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:17.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.978+0000 7fdc967fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdc9806b360 0x7fdc980b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33984/0 (socket says 172.21.15.97:33984) 2023-07-06T03:49:17.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.978+0000 7fdc977fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdc980d2210 0x7fdc980d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39920/0 (socket says 172.21.15.97:39920) 2023-07-06T03:49:17.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.978+0000 7fdc967fc700 1 -- 172.21.15.97:0/3931160399 learned_addr learned my addr 172.21.15.97:0/3931160399 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:17.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.978+0000 7fdc96ffd700 1 --2- 172.21.15.97:0/3931160399 >> v2:172.21.15.146:3300/0 conn(0x7fdc9815c610 0x7fdc980b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:17.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.979+0000 7fdc967fc700 1 -- 172.21.15.97:0/3931160399 >> v2:172.21.15.146:3300/0 conn(0x7fdc9815c610 msgr2=0x7fdc980b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:17.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.979+0000 7fdc967fc700 1 --2- 172.21.15.97:0/3931160399 >> v2:172.21.15.146:3300/0 conn(0x7fdc9815c610 0x7fdc980b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:17.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.979+0000 7fdc967fc700 1 -- 172.21.15.97:0/3931160399 >> v1:172.21.15.97:6789/0 conn(0x7fdc980d2210 legacy=0x7fdc980d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:17.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.979+0000 7fdc967fc700 1 -- 172.21.15.97:0/3931160399 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc980b5650 con 0x7fdc9806b360 2023-07-06T03:49:17.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.979+0000 7fdc96ffd700 1 --2- 172.21.15.97:0/3931160399 >> v2:172.21.15.146:3300/0 conn(0x7fdc9815c610 0x7fdc980b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:17.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.979+0000 7fdc95ffb700 1 -- 172.21.15.97:0/3931160399 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19951911 0 0) 0x7fdc980cff90 con 0x7fdc980d2210 2023-07-06T03:49:17.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.979+0000 7fdc967fc700 1 --2- 172.21.15.97:0/3931160399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9806b360 0x7fdc980b4cf0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fdc9808b000 tx=0x7fdc8c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5868c7532c72fcc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:17.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.979+0000 7fdc95ffb700 1 -- 172.21.15.97:0/3931160399 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00b860 con 0x7fdc9806b360 2023-07-06T03:49:17.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.979+0000 7fdc95ffb700 1 -- 172.21.15.97:0/3931160399 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c00b9c0 con 0x7fdc9806b360 2023-07-06T03:49:17.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.979+0000 7fdc95ffb700 1 -- 172.21.15.97:0/3931160399 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00bd50 con 0x7fdc9806b360 2023-07-06T03:49:17.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.980+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3931160399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9806b360 msgr2=0x7fdc980b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:17.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.980+0000 7fdc9d9ff700 1 --2- 172.21.15.97:0/3931160399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9806b360 0x7fdc980b4cf0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fdc9808b000 tx=0x7fdc8c009c00 comp rx=0 tx=0).stop 2023-07-06T03:49:17.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.980+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3931160399 shutdown_connections 2023-07-06T03:49:17.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.980+0000 7fdc9d9ff700 1 --2- 172.21.15.97:0/3931160399 >> v2:172.21.15.146:3300/0 conn(0x7fdc9815c610 0x7fdc980b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:17.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.980+0000 7fdc9d9ff700 1 --2- 172.21.15.97:0/3931160399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9806b360 0x7fdc980b4cf0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:17.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.980+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3931160399 >> 172.21.15.97:0/3931160399 conn(0x7fdc9806a6c0 msgr2=0x7fdc9806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:17.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.980+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3931160399 shutdown_connections 2023-07-06T03:49:17.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.980+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3931160399 wait complete. 2023-07-06T03:49:17.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.981+0000 7fdc9d9ff700 1 Processor -- start 2023-07-06T03:49:17.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.982+0000 7fdc9d9ff700 1 -- start start 2023-07-06T03:49:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.982+0000 7fdc9d9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9806b360 0x7fdc9815cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.982+0000 7fdc9d9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc980d2210 0x7fdc980b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc9d9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc9815c610 0x7fdc980b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc9d9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981f5540 con 0x7fdc9806b360 2023-07-06T03:49:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc9d9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981f5790 con 0x7fdc9815c610 2023-07-06T03:49:17.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc9d9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981f5a70 con 0x7fdc980d2210 2023-07-06T03:49:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc96ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9806b360 0x7fdc9815cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc96ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9806b360 0x7fdc9815cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33998/0 (socket says 172.21.15.97:33998) 2023-07-06T03:49:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc96ffd700 1 -- 172.21.15.97:0/3430633312 learned_addr learned my addr 172.21.15.97:0/3430633312 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc977fe700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc9815c610 0x7fdc980b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc967fc700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc980d2210 0x7fdc980b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc96ffd700 1 -- 172.21.15.97:0/3430633312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc980d2210 msgr2=0x7fdc980b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc96ffd700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc980d2210 0x7fdc980b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc96ffd700 1 -- 172.21.15.97:0/3430633312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc9815c610 msgr2=0x7fdc980b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc96ffd700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc9815c610 0x7fdc980b7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc96ffd700 1 -- 172.21.15.97:0/3430633312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c009550 con 0x7fdc9806b360 2023-07-06T03:49:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.983+0000 7fdc96ffd700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9806b360 0x7fdc9815cb20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800e810 tx=0x7fdc8800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.984+0000 7fdc7ffff700 1 -- 172.21.15.97:0/3430633312 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8800c020 con 0x7fdc9806b360 2023-07-06T03:49:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.984+0000 7fdc7ffff700 1 -- 172.21.15.97:0/3430633312 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc88004250 con 0x7fdc9806b360 2023-07-06T03:49:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.984+0000 7fdc7ffff700 1 -- 172.21.15.97:0/3430633312 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc880036a0 con 0x7fdc9806b360 2023-07-06T03:49:17.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.984+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3430633312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc981f5d50 con 0x7fdc9806b360 2023-07-06T03:49:17.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.984+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3430633312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc980cb320 con 0x7fdc9806b360 2023-07-06T03:49:17.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.985+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3430633312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9804e340 con 0x7fdc9806b360 2023-07-06T03:49:17.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.986+0000 7fdc7ffff700 1 -- 172.21.15.97:0/3430633312 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc88003800 con 0x7fdc9806b360 2023-07-06T03:49:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.986+0000 7fdc7ffff700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8406ace0 0x7fdc8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.987+0000 7fdc7ffff700 1 -- 172.21.15.97:0/3430633312 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdc88088f20 con 0x7fdc9806b360 2023-07-06T03:49:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.987+0000 7fdc967fc700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8406ace0 0x7fdc8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.987+0000 7fdc967fc700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8406ace0 0x7fdc8406d190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009870 tx=0x7fdc8c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:17.990+0000 7fdc7ffff700 1 -- 172.21.15.97:0/3430633312 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdc880542e0 con 0x7fdc9806b360 2023-07-06T03:49:18.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.188+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3430633312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) v1 -- 0x7fdc9806b9d0 con 0x7fdc9806b360 2023-07-06T03:49:18.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.193+0000 7fdc7ffff700 1 -- 172.21.15.97:0/3430633312 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v20) v1 ==== 106+0+0 (secure 0 0 0) 0x7fdc88057900 con 0x7fdc9806b360 2023-07-06T03:49:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3430633312 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8406ace0 msgr2=0x7fdc8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8406ace0 0x7fdc8406d190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009870 tx=0x7fdc8c008030 comp rx=0 tx=0).stop 2023-07-06T03:49:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3430633312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9806b360 msgr2=0x7fdc9815cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9806b360 0x7fdc9815cb20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800e810 tx=0x7fdc8800ebd0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3430633312 shutdown_connections 2023-07-06T03:49:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8406ace0 0x7fdc8406d190 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc9815c610 0x7fdc980b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9806b360 0x7fdc9815cb20 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 --2- 172.21.15.97:0/3430633312 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc980d2210 0x7fdc980b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3430633312 >> 172.21.15.97:0/3430633312 conn(0x7fdc9806a6c0 msgr2=0x7fdc9815b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:18.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3430633312 shutdown_connections 2023-07-06T03:49:18.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.195+0000 7fdc9d9ff700 1 -- 172.21.15.97:0/3430633312 wait complete. 2023-07-06T03:49:18.215 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T03:49:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.408+0000 7fe581fff700 1 Processor -- start 2023-07-06T03:49:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.409+0000 7fe581fff700 1 -- start start 2023-07-06T03:49:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.410+0000 7fe581fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe57c06b360 0x7fe57c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.410+0000 7fe581fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57c0cff90 con 0x7fe57c0d2210 2023-07-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.410+0000 7fe581fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57c0b5650 con 0x7fe57c15c610 2023-07-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.410+0000 7fe581fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57c0d0780 con 0x7fe57c06b360 2023-07-06T03:49:18.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.410+0000 7fe57bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe57c0d2210 0x7fe57c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39930/0 (socket says 172.21.15.97:39930) 2023-07-06T03:49:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.410+0000 7fe57bfff700 1 -- 172.21.15.97:0/3179164669 learned_addr learned my addr 172.21.15.97:0/3179164669 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.410+0000 7fe57affd700 1 --2- 172.21.15.97:0/3179164669 >> v2:172.21.15.146:3300/0 conn(0x7fe57c06b360 0x7fe57c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.410+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3179164669 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677083703 0 0) 0x7fe57c0cff90 con 0x7fe57c0d2210 2023-07-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.411+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3179164669 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5640035b0 con 0x7fe57c0d2210 2023-07-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.411+0000 7fe57affd700 1 -- 172.21.15.97:0/3179164669 >> v1:172.21.15.146:6789/0 conn(0x7fe57c15c610 legacy=0x7fe57c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.411+0000 7fe57affd700 1 -- 172.21.15.97:0/3179164669 >> v1:172.21.15.97:6789/0 conn(0x7fe57c0d2210 legacy=0x7fe57c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.411+0000 7fe57affd700 1 -- 172.21.15.97:0/3179164669 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c0b5400 con 0x7fe57c06b360 2023-07-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.411+0000 7fe57affd700 1 --2- 172.21.15.97:0/3179164669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe57c06b360 0x7fe57c0b4ec0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe570009840 tx=0x7fe570009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91f2003f8616f21a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.411+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3179164669 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570011830 con 0x7fe57c06b360 2023-07-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.411+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3179164669 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe570011990 con 0x7fe57c06b360 2023-07-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.411+0000 7fe57a7fc700 1 -- 172.21.15.97:0/3179164669 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570011d20 con 0x7fe57c06b360 2023-07-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.412+0000 7fe581fff700 1 -- 172.21.15.97:0/3179164669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe57c06b360 msgr2=0x7fe57c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.412+0000 7fe581fff700 1 --2- 172.21.15.97:0/3179164669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe57c06b360 0x7fe57c0b4ec0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe570009840 tx=0x7fe570009b50 comp rx=0 tx=0).stop 2023-07-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.412+0000 7fe581fff700 1 -- 172.21.15.97:0/3179164669 shutdown_connections 2023-07-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.412+0000 7fe581fff700 1 --2- 172.21.15.97:0/3179164669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe57c06b360 0x7fe57c0b4ec0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.412+0000 7fe581fff700 1 -- 172.21.15.97:0/3179164669 >> 172.21.15.97:0/3179164669 conn(0x7fe57c06a6c0 msgr2=0x7fe57c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.412+0000 7fe581fff700 1 -- 172.21.15.97:0/3179164669 shutdown_connections 2023-07-06T03:49:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.412+0000 7fe581fff700 1 -- 172.21.15.97:0/3179164669 wait complete. 2023-07-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.414+0000 7fe581fff700 1 Processor -- start 2023-07-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.414+0000 7fe581fff700 1 -- start start 2023-07-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe581fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe57c06b360 0x7fe57c15d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe581fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe57c0d2210 0x7fe57c0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe581fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15c610 0x7fe57c156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe581fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c1548c0 con 0x7fe57c15c610 2023-07-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe581fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c156880 con 0x7fe57c06b360 2023-07-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe581fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c155f50 con 0x7fe57c0d2210 2023-07-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe57bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15c610 0x7fe57c156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe57bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15c610 0x7fe57c156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34006/0 (socket says 172.21.15.97:34006) 2023-07-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe57bfff700 1 -- 172.21.15.97:0/2407466105 learned_addr learned my addr 172.21.15.97:0/2407466105 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe57b7fe700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe57c06b360 0x7fe57c15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe57affd700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe57c0d2210 0x7fe57c0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe57bfff700 1 -- 172.21.15.97:0/2407466105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe57c0d2210 msgr2=0x7fe57c0c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe57bfff700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe57c0d2210 0x7fe57c0c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe57bfff700 1 -- 172.21.15.97:0/2407466105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe57c06b360 msgr2=0x7fe57c15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe57bfff700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe57c06b360 0x7fe57c15d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.415+0000 7fe57bfff700 1 -- 172.21.15.97:0/2407466105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5700094a0 con 0x7fe57c15c610 2023-07-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.416+0000 7fe57bfff700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15c610 0x7fe57c156d10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe57400cb70 tx=0x7fe57400ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.416+0000 7fe563fff700 1 -- 172.21.15.97:0/2407466105 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57400aa60 con 0x7fe57c15c610 2023-07-06T03:49:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.416+0000 7fe563fff700 1 -- 172.21.15.97:0/2407466105 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57400abc0 con 0x7fe57c15c610 2023-07-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.416+0000 7fe581fff700 1 -- 172.21.15.97:0/2407466105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c156230 con 0x7fe57c15c610 2023-07-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.416+0000 7fe563fff700 1 -- 172.21.15.97:0/2407466105 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574012400 con 0x7fe57c15c610 2023-07-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.416+0000 7fe581fff700 1 -- 172.21.15.97:0/2407466105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57c15dab0 con 0x7fe57c15c610 2023-07-06T03:49:18.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.417+0000 7fe581fff700 1 -- 172.21.15.97:0/2407466105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57c0d0480 con 0x7fe57c15c610 2023-07-06T03:49:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.418+0000 7fe563fff700 1 -- 172.21.15.97:0/2407466105 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe57401b3b0 con 0x7fe57c15c610 2023-07-06T03:49:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.420+0000 7fe563fff700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 0x7fe56806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.420+0000 7fe563fff700 1 -- 172.21.15.97:0/2407466105 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe574088ed0 con 0x7fe57c15c610 2023-07-06T03:49:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.420+0000 7fe57b7fe700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 0x7fe56806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.421+0000 7fe57b7fe700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 0x7fe56806d190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe56c007fd0 tx=0x7fe56c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:18.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.424+0000 7fe563fff700 1 -- 172.21.15.97:0/2407466105 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe57c0d0480 con 0x7fe57c15c610 2023-07-06T03:49:18.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.619+0000 7fe581fff700 1 -- 172.21.15.97:0/2407466105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe57c0d0480 con 0x7fe57c15c610 2023-07-06T03:49:18.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.625+0000 7fe563fff700 1 -- 172.21.15.97:0/2407466105 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe57c0d0480 con 0x7fe57c15c610 2023-07-06T03:49:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 -- 172.21.15.97:0/2407466105 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 msgr2=0x7fe56806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 0x7fe56806d190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe56c007fd0 tx=0x7fe56c009040 comp rx=0 tx=0).stop 2023-07-06T03:49:18.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 -- 172.21.15.97:0/2407466105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15c610 msgr2=0x7fe57c156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15c610 0x7fe57c156d10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe57400cb70 tx=0x7fe57400ce80 comp rx=0 tx=0).stop 2023-07-06T03:49:18.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 -- 172.21.15.97:0/2407466105 shutdown_connections 2023-07-06T03:49:18.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe56806ace0 0x7fe56806d190 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15c610 0x7fe57c156d10 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe57c06b360 0x7fe57c15d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 --2- 172.21.15.97:0/2407466105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe57c0d2210 0x7fe57c0c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 -- 172.21.15.97:0/2407466105 >> 172.21.15.97:0/2407466105 conn(0x7fe57c06a6c0 msgr2=0x7fe57c15a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 -- 172.21.15.97:0/2407466105 shutdown_connections 2023-07-06T03:49:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.628+0000 7fe581fff700 1 -- 172.21.15.97:0/2407466105 wait complete. 2023-07-06T03:49:18.647 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2023-07-06T03:49:18.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.846+0000 7f7eeedff700 1 Processor -- start 2023-07-06T03:49:18.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.846+0000 7f7eeedff700 1 -- start start 2023-07-06T03:49:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.847+0000 7f7eeedff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7ee8160190 0x7f7ee80ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.847+0000 7f7eeedff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ee8071bf0 0x7f7ee806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.847+0000 7f7eeedff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee806a9f0 con 0x7f7ee806a060 2023-07-06T03:49:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.847+0000 7f7eeedff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee80cf710 con 0x7f7ee8071bf0 2023-07-06T03:49:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.847+0000 7f7eeedff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee80ceb40 con 0x7f7ee8160190 2023-07-06T03:49:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.848+0000 7f7eed39c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ee8071bf0 0x7f7ee806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.848+0000 7f7eed39c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ee8071bf0 0x7f7ee806a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34014/0 (socket says 172.21.15.97:34014) 2023-07-06T03:49:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.848+0000 7f7eed39c700 1 -- 172.21.15.97:0/1401732835 learned_addr learned my addr 172.21.15.97:0/1401732835 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.848+0000 7f7ee7fff700 1 --1- 172.21.15.97:0/1401732835 >> v1:172.21.15.97:6789/0 conn(0x7f7ee806a060 0x7f7ee80714e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39932/0 (socket says 172.21.15.97:39932) 2023-07-06T03:49:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.848+0000 7f7eecb9b700 1 --2- 172.21.15.97:0/1401732835 >> v2:172.21.15.146:3300/0 conn(0x7f7ee8160190 0x7f7ee80ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.848+0000 7f7eed39c700 1 -- 172.21.15.97:0/1401732835 >> v2:172.21.15.146:3300/0 conn(0x7f7ee8160190 msgr2=0x7f7ee80ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:18.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.848+0000 7f7eed39c700 1 --2- 172.21.15.97:0/1401732835 >> v2:172.21.15.146:3300/0 conn(0x7f7ee8160190 0x7f7ee80ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.848+0000 7f7eed39c700 1 -- 172.21.15.97:0/1401732835 >> v1:172.21.15.97:6789/0 conn(0x7f7ee806a060 legacy=0x7f7ee80714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.848+0000 7f7eed39c700 1 -- 172.21.15.97:0/1401732835 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee80cede0 con 0x7f7ee8071bf0 2023-07-06T03:49:18.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.848+0000 7f7ee77fe700 1 -- 172.21.15.97:0/1401732835 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322938668 0 0) 0x7f7ed4001110 con 0x7f7ee806a060 2023-07-06T03:49:18.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.848+0000 7f7eed39c700 1 --2- 172.21.15.97:0/1401732835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8071bf0 0x7f7ee806a4b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8070700 tx=0x7f7ee000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5012d912cac0b8a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:18.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.849+0000 7f7ee77fe700 1 -- 172.21.15.97:0/1401732835 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee000e070 con 0x7f7ee8071bf0 2023-07-06T03:49:18.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.849+0000 7f7ee77fe700 1 -- 172.21.15.97:0/1401732835 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee000b9c0 con 0x7f7ee8071bf0 2023-07-06T03:49:18.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.849+0000 7f7ee77fe700 1 -- 172.21.15.97:0/1401732835 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee000bd70 con 0x7f7ee8071bf0 2023-07-06T03:49:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.849+0000 7f7eeedff700 1 -- 172.21.15.97:0/1401732835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8071bf0 msgr2=0x7f7ee806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.849+0000 7f7eeedff700 1 --2- 172.21.15.97:0/1401732835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8071bf0 0x7f7ee806a4b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8070700 tx=0x7f7ee000b330 comp rx=0 tx=0).stop 2023-07-06T03:49:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.849+0000 7f7eeedff700 1 -- 172.21.15.97:0/1401732835 shutdown_connections 2023-07-06T03:49:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.849+0000 7f7eeedff700 1 --2- 172.21.15.97:0/1401732835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8071bf0 0x7f7ee806a4b0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.849+0000 7f7eeedff700 1 --2- 172.21.15.97:0/1401732835 >> v2:172.21.15.146:3300/0 conn(0x7f7ee8160190 0x7f7ee80ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.849+0000 7f7eeedff700 1 -- 172.21.15.97:0/1401732835 >> 172.21.15.97:0/1401732835 conn(0x7f7ee80b22a0 msgr2=0x7f7ee80cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.849+0000 7f7eeedff700 1 -- 172.21.15.97:0/1401732835 shutdown_connections 2023-07-06T03:49:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.849+0000 7f7eeedff700 1 -- 172.21.15.97:0/1401732835 wait complete. 2023-07-06T03:49:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.851+0000 7f7eeedff700 1 Processor -- start 2023-07-06T03:49:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.851+0000 7f7eeedff700 1 -- start start 2023-07-06T03:49:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.852+0000 7f7eeedff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ee806a060 0x7f7ee80c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.852+0000 7f7eeedff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8071bf0 0x7f7ee8154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.852+0000 7f7eeedff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee8160190 0x7f7ee80c3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.852+0000 7f7eeedff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee80c9720 con 0x7f7ee8071bf0 2023-07-06T03:49:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.852+0000 7f7eeedff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee80c8700 con 0x7f7ee8160190 2023-07-06T03:49:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.852+0000 7f7eeedff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee81546a0 con 0x7f7ee806a060 2023-07-06T03:49:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.852+0000 7f7ee7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8071bf0 0x7f7ee8154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.852+0000 7f7ee7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8071bf0 0x7f7ee8154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34016/0 (socket says 172.21.15.97:34016) 2023-07-06T03:49:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.852+0000 7f7ee7fff700 1 -- 172.21.15.97:0/3509281846 learned_addr learned my addr 172.21.15.97:0/3509281846 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.852+0000 7f7eecb9b700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ee806a060 0x7f7ee80c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:18.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.852+0000 7f7eed39c700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee8160190 0x7f7ee80c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:18.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.853+0000 7f7ee7fff700 1 -- 172.21.15.97:0/3509281846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ee806a060 msgr2=0x7f7ee80c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.853+0000 7f7ee7fff700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ee806a060 0x7f7ee80c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.853+0000 7f7ee7fff700 1 -- 172.21.15.97:0/3509281846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee8160190 msgr2=0x7f7ee80c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:18.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.853+0000 7f7ee7fff700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee8160190 0x7f7ee80c3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:18.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.853+0000 7f7ee7fff700 1 -- 172.21.15.97:0/3509281846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee0009b30 con 0x7f7ee8071bf0 2023-07-06T03:49:18.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.853+0000 7f7ee7fff700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8071bf0 0x7f7ee8154160 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400c730 tx=0x7f7ed400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:18.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.853+0000 7f7ee57fa700 1 -- 172.21.15.97:0/3509281846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4004c80 con 0x7f7ee8071bf0 2023-07-06T03:49:18.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.853+0000 7f7ee57fa700 1 -- 172.21.15.97:0/3509281846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed4004e00 con 0x7f7ee8071bf0 2023-07-06T03:49:18.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.853+0000 7f7eeedff700 1 -- 172.21.15.97:0/3509281846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee80c9440 con 0x7f7ee8071bf0 2023-07-06T03:49:18.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.853+0000 7f7ee57fa700 1 -- 172.21.15.97:0/3509281846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4007310 con 0x7f7ee8071bf0 2023-07-06T03:49:18.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.853+0000 7f7eeedff700 1 -- 172.21.15.97:0/3509281846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee80c79c0 con 0x7f7ee8071bf0 2023-07-06T03:49:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.854+0000 7f7eeedff700 1 -- 172.21.15.97:0/3509281846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee804e340 con 0x7f7ee8071bf0 2023-07-06T03:49:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.856+0000 7f7ee57fa700 1 -- 172.21.15.97:0/3509281846 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ed4007470 con 0x7f7ee8071bf0 2023-07-06T03:49:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.857+0000 7f7ee57fa700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ecc06ace0 0x7f7ecc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.857+0000 7f7ee57fa700 1 -- 172.21.15.97:0/3509281846 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7ed4089420 con 0x7f7ee8071bf0 2023-07-06T03:49:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.857+0000 7f7eecb9b700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ecc06ace0 0x7f7ecc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:18.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.858+0000 7f7eecb9b700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ecc06ace0 0x7f7ecc06d190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00b9b0 tx=0x7f7edc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:18.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:18.860+0000 7f7ee57fa700 1 -- 172.21.15.97:0/3509281846 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7ed40547d0 con 0x7f7ee8071bf0 2023-07-06T03:49:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.131+0000 7f7eeedff700 1 -- 172.21.15.97:0/3509281846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) v1 -- 0x7f7ee806a9f0 con 0x7f7ee8071bf0 2023-07-06T03:49:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.754+0000 7f7ee57fa700 1 -- 172.21.15.97:0/3509281846 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v241) v1 ==== 148+0+0 (secure 0 0 0) 0x7f7ee806a9f0 con 0x7f7ee8071bf0 2023-07-06T03:49:19.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.756+0000 7f7eeedff700 1 -- 172.21.15.97:0/3509281846 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ecc06ace0 msgr2=0x7f7ecc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:19.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.756+0000 7f7eeedff700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ecc06ace0 0x7f7ecc06d190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00b9b0 tx=0x7f7edc008040 comp rx=0 tx=0).stop 2023-07-06T03:49:19.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.756+0000 7f7eeedff700 1 -- 172.21.15.97:0/3509281846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8071bf0 msgr2=0x7f7ee8154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:19.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.756+0000 7f7eeedff700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8071bf0 0x7f7ee8154160 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400c730 tx=0x7f7ed400caf0 comp rx=0 tx=0).stop 2023-07-06T03:49:19.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.757+0000 7f7eeedff700 1 -- 172.21.15.97:0/3509281846 shutdown_connections 2023-07-06T03:49:19.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.757+0000 7f7eeedff700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7ecc06ace0 0x7f7ecc06d190 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:19.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.757+0000 7f7eeedff700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ee8160190 0x7f7ee80c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:19.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.757+0000 7f7eeedff700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ee806a060 0x7f7ee80c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:19.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.757+0000 7f7eeedff700 1 --2- 172.21.15.97:0/3509281846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8071bf0 0x7f7ee8154160 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:19.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.757+0000 7f7eeedff700 1 -- 172.21.15.97:0/3509281846 >> 172.21.15.97:0/3509281846 conn(0x7f7ee80b22a0 msgr2=0x7f7ee806eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:19.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.757+0000 7f7eeedff700 1 -- 172.21.15.97:0/3509281846 shutdown_connections 2023-07-06T03:49:19.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.757+0000 7f7eeedff700 1 -- 172.21.15.97:0/3509281846 wait complete. 2023-07-06T03:49:19.777 INFO:tasks.cephfs_test_runner:test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-06T03:49:19.778 INFO:tasks.cephfs_test_runner:Starting test: test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:49:19.778 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2023-07-06T03:49:19.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.972+0000 7f47b45ff700 1 Processor -- start 2023-07-06T03:49:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.973+0000 7f47b45ff700 1 -- start start 2023-07-06T03:49:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.974+0000 7f47b45ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f47ac15ff00 0x7f47ac0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.974+0000 7f47b45ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac0b4740 con 0x7f47ac0c8d70 2023-07-06T03:49:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.974+0000 7f47b45ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac1602d0 con 0x7f47ac0d10c0 2023-07-06T03:49:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.974+0000 7f47b45ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ac0b60b0 con 0x7f47ac15ff00 2023-07-06T03:49:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.974+0000 7f47b1b9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f47ac0c8d70 0x7f47ac0d09b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56550/0 (socket says 172.21.15.97:56550) 2023-07-06T03:49:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.974+0000 7f47b1b9a700 1 -- 172.21.15.97:0/1913250069 learned_addr learned my addr 172.21.15.97:0/1913250069 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.974+0000 7f47b239b700 1 --2- 172.21.15.97:0/1913250069 >> v2:172.21.15.146:3300/0 conn(0x7f47ac15ff00 0x7f47ac0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.975+0000 7f47b1399700 1 -- 172.21.15.97:0/1913250069 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990678844 0 0) 0x7f47ac0b4740 con 0x7f47ac0c8d70 2023-07-06T03:49:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.975+0000 7f47b1399700 1 -- 172.21.15.97:0/1913250069 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47980035b0 con 0x7f47ac0c8d70 2023-07-06T03:49:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.975+0000 7f47b239b700 1 -- 172.21.15.97:0/1913250069 >> v1:172.21.15.146:6789/0 conn(0x7f47ac0d10c0 legacy=0x7f47ac0b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.975+0000 7f47b239b700 1 -- 172.21.15.97:0/1913250069 >> v1:172.21.15.97:6789/0 conn(0x7f47ac0c8d70 legacy=0x7f47ac0d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.975+0000 7f47b239b700 1 -- 172.21.15.97:0/1913250069 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0c9800 con 0x7f47ac15ff00 2023-07-06T03:49:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.975+0000 7f47b239b700 1 --2- 172.21.15.97:0/1913250069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac15ff00 0x7f47ac0c8830 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009820 tx=0x7f47a8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77ea34eb02c9502d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.975+0000 7f47b1399700 1 -- 172.21.15.97:0/1913250069 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8005e30 con 0x7f47ac15ff00 2023-07-06T03:49:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.975+0000 7f47b1399700 1 -- 172.21.15.97:0/1913250069 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a8006030 con 0x7f47ac15ff00 2023-07-06T03:49:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.975+0000 7f47b1399700 1 -- 172.21.15.97:0/1913250069 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a80063c0 con 0x7f47ac15ff00 2023-07-06T03:49:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.976+0000 7f47b45ff700 1 -- 172.21.15.97:0/1913250069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac15ff00 msgr2=0x7f47ac0c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.976+0000 7f47b45ff700 1 --2- 172.21.15.97:0/1913250069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac15ff00 0x7f47ac0c8830 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009820 tx=0x7f47a8009b30 comp rx=0 tx=0).stop 2023-07-06T03:49:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.976+0000 7f47b45ff700 1 -- 172.21.15.97:0/1913250069 shutdown_connections 2023-07-06T03:49:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.976+0000 7f47b45ff700 1 --2- 172.21.15.97:0/1913250069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac15ff00 0x7f47ac0c8830 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.976+0000 7f47b45ff700 1 -- 172.21.15.97:0/1913250069 >> 172.21.15.97:0/1913250069 conn(0x7f47ac09c130 msgr2=0x7f47ac0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.976+0000 7f47b45ff700 1 -- 172.21.15.97:0/1913250069 shutdown_connections 2023-07-06T03:49:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.976+0000 7f47b45ff700 1 -- 172.21.15.97:0/1913250069 wait complete. 2023-07-06T03:49:19.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.978+0000 7f47b45ff700 1 Processor -- start 2023-07-06T03:49:19.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.978+0000 7f47b45ff700 1 -- start start 2023-07-06T03:49:19.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b45ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47ac0c8d70 0x7f47ac1f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:19.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b45ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47ac0d10c0 0x7f47ac0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:19.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b45ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac15ff00 0x7f47ac0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:19.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b45ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac09ded0 con 0x7f47ac0d10c0 2023-07-06T03:49:19.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b45ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac09e1b0 con 0x7f47ac15ff00 2023-07-06T03:49:19.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b45ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac09e490 con 0x7f47ac0c8d70 2023-07-06T03:49:19.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b1b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47ac0d10c0 0x7f47ac0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:19.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b1b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47ac0d10c0 0x7f47ac0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33686/0 (socket says 172.21.15.97:33686) 2023-07-06T03:49:19.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b1b9a700 1 -- 172.21.15.97:0/3313929599 learned_addr learned my addr 172.21.15.97:0/3313929599 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:19.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b2b9c700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac15ff00 0x7f47ac0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:19.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b239b700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47ac0c8d70 0x7f47ac1f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:19.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b1b9a700 1 -- 172.21.15.97:0/3313929599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47ac0c8d70 msgr2=0x7f47ac1f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:19.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b1b9a700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47ac0c8d70 0x7f47ac1f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b1b9a700 1 -- 172.21.15.97:0/3313929599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac15ff00 msgr2=0x7f47ac0b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.979+0000 7f47b1b9a700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac15ff00 0x7f47ac0b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.980+0000 7f47b1b9a700 1 -- 172.21.15.97:0/3313929599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8009430 con 0x7f47ac0d10c0 2023-07-06T03:49:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.980+0000 7f47b2b9c700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac15ff00 0x7f47ac0b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.980+0000 7f47b1b9a700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47ac0d10c0 0x7f47ac0b2160 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f479c00c700 tx=0x7f479c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.980+0000 7f47a2ffd700 1 -- 172.21.15.97:0/3313929599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c005d70 con 0x7f47ac0d10c0 2023-07-06T03:49:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.980+0000 7f47a2ffd700 1 -- 172.21.15.97:0/3313929599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c0070d0 con 0x7f47ac0d10c0 2023-07-06T03:49:19.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.980+0000 7f47b45ff700 1 -- 172.21.15.97:0/3313929599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac09e770 con 0x7f47ac0d10c0 2023-07-06T03:49:19.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.980+0000 7f47a2ffd700 1 -- 172.21.15.97:0/3313929599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c007460 con 0x7f47ac0d10c0 2023-07-06T03:49:19.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.980+0000 7f47b45ff700 1 -- 172.21.15.97:0/3313929599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac09daf0 con 0x7f47ac0d10c0 2023-07-06T03:49:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.981+0000 7f47b45ff700 1 -- 172.21.15.97:0/3313929599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ac04e340 con 0x7f47ac0d10c0 2023-07-06T03:49:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.982+0000 7f47a2ffd700 1 -- 172.21.15.97:0/3313929599 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f479c0075c0 con 0x7f47ac0d10c0 2023-07-06T03:49:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.983+0000 7f47a2ffd700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f479006ace0 0x7f479006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.983+0000 7f47a2ffd700 1 -- 172.21.15.97:0/3313929599 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f479c088750 con 0x7f47ac0d10c0 2023-07-06T03:49:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.983+0000 7f47b239b700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f479006ace0 0x7f479006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.984+0000 7f47b239b700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f479006ace0 0x7f479006d190 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f47a8013040 tx=0x7f47a8008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:19.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:19.987+0000 7f47a2ffd700 1 -- 172.21.15.97:0/3313929599 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f479c053b10 con 0x7f47ac0d10c0 2023-07-06T03:49:20.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.261+0000 7f47b45ff700 1 -- 172.21.15.97:0/3313929599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) v1 -- 0x7f47ac09dc80 con 0x7f47ac0d10c0 2023-07-06T03:49:20.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.756+0000 7f47a2ffd700 1 -- 172.21.15.97:0/3313929599 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v242) v1 ==== 163+0+0 (secure 0 0 0) 0x7f479c057130 con 0x7f47ac0d10c0 2023-07-06T03:49:20.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.759+0000 7f47b45ff700 1 -- 172.21.15.97:0/3313929599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f479006ace0 msgr2=0x7f479006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:20.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.759+0000 7f47b45ff700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f479006ace0 0x7f479006d190 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f47a8013040 tx=0x7f47a8008070 comp rx=0 tx=0).stop 2023-07-06T03:49:20.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.759+0000 7f47b45ff700 1 -- 172.21.15.97:0/3313929599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47ac0d10c0 msgr2=0x7f47ac0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.759+0000 7f47b45ff700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47ac0d10c0 0x7f47ac0b2160 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f479c00c700 tx=0x7f479c00cac0 comp rx=0 tx=0).stop 2023-07-06T03:49:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.759+0000 7f47b45ff700 1 -- 172.21.15.97:0/3313929599 shutdown_connections 2023-07-06T03:49:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.759+0000 7f47b45ff700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f479006ace0 0x7f479006d190 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.759+0000 7f47b45ff700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47ac15ff00 0x7f47ac0b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.759+0000 7f47b45ff700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47ac0c8d70 0x7f47ac1f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.759+0000 7f47b45ff700 1 --2- 172.21.15.97:0/3313929599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47ac0d10c0 0x7f47ac0b2160 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.759+0000 7f47b45ff700 1 -- 172.21.15.97:0/3313929599 >> 172.21.15.97:0/3313929599 conn(0x7f47ac09c130 msgr2=0x7f47ac0b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.760+0000 7f47b45ff700 1 -- 172.21.15.97:0/3313929599 shutdown_connections 2023-07-06T03:49:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.760+0000 7f47b45ff700 1 -- 172.21.15.97:0/3313929599 wait complete. 2023-07-06T03:49:20.780 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T03:49:20.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.978+0000 7f828dfff700 1 Processor -- start 2023-07-06T03:49:20.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.978+0000 7f828dfff700 1 -- start start 2023-07-06T03:49:20.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.979+0000 7f828dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828815c610 0x7f82880b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:20.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.980+0000 7f828dfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f828806b360 0x7f82880b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:20.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.980+0000 7f828dfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82880cff90 con 0x7f82880d2210 2023-07-06T03:49:20.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.980+0000 7f828dfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82880d0780 con 0x7f828806b360 2023-07-06T03:49:20.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.980+0000 7f828dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828815c9e0 con 0x7f828815c610 2023-07-06T03:49:20.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.980+0000 7f8287fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f82880d2210 0x7f82880d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56554/0 (socket says 172.21.15.97:56554) 2023-07-06T03:49:20.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.980+0000 7f8287fff700 1 -- 172.21.15.97:0/602741821 learned_addr learned my addr 172.21.15.97:0/602741821 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:20.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.980+0000 7f82877fe700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828815c610 0x7f82880b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:20.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.980+0000 7f827ffff700 1 --2- 172.21.15.97:0/602741821 >> v2:172.21.15.146:3300/0 conn(0x7f828806b360 0x7f82880b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:20.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.980+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573871344 0 0) 0x7f82880cff90 con 0x7f82880d2210 2023-07-06T03:49:20.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.980+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82740035b0 con 0x7f82880d2210 2023-07-06T03:49:20.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2396547952 0 0) 0x7f82740035b0 con 0x7f82880d2210 2023-07-06T03:49:20.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828815c610 msgr2=0x7f82880b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:20.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828815c610 0x7f82880b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:20.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 >> v2:172.21.15.146:3300/0 conn(0x7f828806b360 msgr2=0x7f82880b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:20.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 --2- 172.21.15.97:0/602741821 >> v2:172.21.15.146:3300/0 conn(0x7f828806b360 0x7f82880b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:20.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82880b5650 con 0x7f82880d2210 2023-07-06T03:49:20.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f82800019f0 con 0x7f82880d2210 2023-07-06T03:49:20.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 >> v1:172.21.15.97:6789/0 conn(0x7f82880d2210 legacy=0x7f82880d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:20.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828815c610 0x7f8274003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:20.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8274003fb0 0x7f8274004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:20.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 0x7f8274005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:20.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82880d0780 con 0x7f8274004d90 2023-07-06T03:49:20.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828815c9e0 con 0x7f8274003fb0 2023-07-06T03:49:20.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827400ba10 con 0x7f828815c610 2023-07-06T03:49:20.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f827ffff700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8274003fb0 0x7f8274004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:20.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f8287fff700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 0x7f8274005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:20.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.981+0000 7f82877fe700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828815c610 0x7f8274003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.982+0000 7f8287fff700 1 -- 172.21.15.97:0/602741821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828815c610 msgr2=0x7f8274003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.982+0000 7f8287fff700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f828815c610 0x7f8274003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.982+0000 7f8287fff700 1 -- 172.21.15.97:0/602741821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8274003fb0 msgr2=0x7f8274004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.982+0000 7f8287fff700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8274003fb0 0x7f8274004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.982+0000 7f8287fff700 1 -- 172.21.15.97:0/602741821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82740035b0 con 0x7f8274004d90 2023-07-06T03:49:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.982+0000 7f8287fff700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 0x7f8274005200 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f828000a830 tx=0x7f8274005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63240a2c19c8e23 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.982+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828000a2f0 con 0x7f8274004d90 2023-07-06T03:49:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.982+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8280012620 con 0x7f8274004d90 2023-07-06T03:49:20.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.982+0000 7f8286ffd700 1 -- 172.21.15.97:0/602741821 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82800129d0 con 0x7f8274004d90 2023-07-06T03:49:20.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.983+0000 7f828dfff700 1 -- 172.21.15.97:0/602741821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 msgr2=0x7f8274005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:20.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.983+0000 7f828dfff700 1 --2- 172.21.15.97:0/602741821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 0x7f8274005200 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f828000a830 tx=0x7f8274005880 comp rx=0 tx=0).stop 2023-07-06T03:49:20.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.983+0000 7f8287fff700 1 -- 172.21.15.97:0/602741821 reap_dead start 2023-07-06T03:49:20.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.983+0000 7f828dfff700 1 -- 172.21.15.97:0/602741821 shutdown_connections 2023-07-06T03:49:20.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.983+0000 7f828dfff700 1 -- 172.21.15.97:0/602741821 >> 172.21.15.97:0/602741821 conn(0x7f828806a6c0 msgr2=0x7f828806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:20.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.983+0000 7f828dfff700 1 -- 172.21.15.97:0/602741821 shutdown_connections 2023-07-06T03:49:20.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.983+0000 7f828dfff700 1 -- 172.21.15.97:0/602741821 wait complete. 2023-07-06T03:49:20.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.985+0000 7f828dfff700 1 Processor -- start 2023-07-06T03:49:20.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.985+0000 7f828dfff700 1 -- start start 2023-07-06T03:49:20.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f828dfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 0x7f82880c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:20.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f828dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82880b6140 0x7f82880cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:20.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f828dfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8288150130 0x7f82881505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:20.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f828dfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8288150b10 con 0x7f8274004d90 2023-07-06T03:49:20.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f828dfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82881f6a20 con 0x7f8288150130 2023-07-06T03:49:20.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f828dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82880c8aa0 con 0x7f82880b6140 2023-07-06T03:49:20.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f82877fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 0x7f82880c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:20.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f82877fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 0x7f82880c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33700/0 (socket says 172.21.15.97:33700) 2023-07-06T03:49:20.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f82877fe700 1 -- 172.21.15.97:0/2842181963 learned_addr learned my addr 172.21.15.97:0/2842181963 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:20.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f827ffff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82880b6140 0x7f82880cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:20.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f8287fff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8288150130 0x7f82881505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:20.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f82877fe700 1 -- 172.21.15.97:0/2842181963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82880b6140 msgr2=0x7f82880cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:20.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f82877fe700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82880b6140 0x7f82880cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:20.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f82877fe700 1 -- 172.21.15.97:0/2842181963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8288150130 msgr2=0x7f82881505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:20.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f82877fe700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8288150130 0x7f82881505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:20.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f82877fe700 1 -- 172.21.15.97:0/2842181963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82881f5ea0 con 0x7f8274004d90 2023-07-06T03:49:20.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f827ffff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82880b6140 0x7f82880cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:20.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f8287fff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8288150130 0x7f82881505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:20.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.987+0000 7f82877fe700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 0x7f82880c8560 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8278011aa0 tx=0x7f8278011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:20.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.988+0000 7f8284ff9700 1 -- 172.21.15.97:0/2842181963 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827800f980 con 0x7f8274004d90 2023-07-06T03:49:20.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.988+0000 7f8284ff9700 1 -- 172.21.15.97:0/2842181963 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827800fae0 con 0x7f8274004d90 2023-07-06T03:49:20.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.988+0000 7f828dfff700 1 -- 172.21.15.97:0/2842181963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82881f6180 con 0x7f8274004d90 2023-07-06T03:49:20.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.988+0000 7f8284ff9700 1 -- 172.21.15.97:0/2842181963 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827800fe90 con 0x7f8274004d90 2023-07-06T03:49:20.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.988+0000 7f828dfff700 1 -- 172.21.15.97:0/2842181963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82881f6740 con 0x7f8274004d90 2023-07-06T03:49:20.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.989+0000 7f828dfff700 1 -- 172.21.15.97:0/2842181963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828804e340 con 0x7f8274004d90 2023-07-06T03:49:20.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.990+0000 7f8284ff9700 1 -- 172.21.15.97:0/2842181963 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8278014440 con 0x7f8274004d90 2023-07-06T03:49:20.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.991+0000 7f8284ff9700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 0x7f826806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:20.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.991+0000 7f8284ff9700 1 -- 172.21.15.97:0/2842181963 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f82780898e0 con 0x7f8274004d90 2023-07-06T03:49:20.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.991+0000 7f827ffff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 0x7f826806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:20.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.992+0000 7f827ffff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 0x7f826806d190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f828815a380 tx=0x7f8270008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:20.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:20.995+0000 7f8284ff9700 1 -- 172.21.15.97:0/2842181963 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8278058870 con 0x7f8274004d90 2023-07-06T03:49:21.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.195+0000 7f828dfff700 1 -- 172.21.15.97:0/2842181963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f82880b17b0 con 0x7f8274004d90 2023-07-06T03:49:21.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.199+0000 7f8284ff9700 1 -- 172.21.15.97:0/2842181963 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f82780220c0 con 0x7f8274004d90 2023-07-06T03:49:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.201+0000 7f828dfff700 1 -- 172.21.15.97:0/2842181963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 msgr2=0x7f826806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.201+0000 7f828dfff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 0x7f826806d190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f828815a380 tx=0x7f8270008040 comp rx=0 tx=0).stop 2023-07-06T03:49:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.202+0000 7f828dfff700 1 -- 172.21.15.97:0/2842181963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 msgr2=0x7f82880c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.202+0000 7f828dfff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 0x7f82880c8560 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8278011aa0 tx=0x7f8278011e60 comp rx=0 tx=0).stop 2023-07-06T03:49:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.202+0000 7f828dfff700 1 -- 172.21.15.97:0/2842181963 shutdown_connections 2023-07-06T03:49:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.202+0000 7f828dfff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 0x7f826806d190 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.202+0000 7f828dfff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8288150130 0x7f82881505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.202+0000 7f828dfff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8274004d90 0x7f82880c8560 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.202+0000 7f828dfff700 1 --2- 172.21.15.97:0/2842181963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f82880b6140 0x7f82880cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.202+0000 7f828dfff700 1 -- 172.21.15.97:0/2842181963 >> 172.21.15.97:0/2842181963 conn(0x7f828806a6c0 msgr2=0x7f82880d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.202+0000 7f828dfff700 1 -- 172.21.15.97:0/2842181963 shutdown_connections 2023-07-06T03:49:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.202+0000 7f828dfff700 1 -- 172.21.15.97:0/2842181963 wait complete. 2023-07-06T03:49:21.221 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:49:21.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:49:21.315 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:49:21.424 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:49:21.425 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:49:21.425 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:49:21.470 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:49:21.471 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:49:21.471 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:49:21.518 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:49:21.519 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:49:21.519 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:49:21.564 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:49:21.564 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:49:21.565 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:49:21.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:49:21.611 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T03:49:21.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:49:21.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.805+0000 7f0277bff700 1 Processor -- start 2023-07-06T03:49:21.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.805+0000 7f0277bff700 1 -- start start 2023-07-06T03:49:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.806+0000 7f0277bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f027015c520 0x7f02700cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.806+0000 7f0277bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02700cdae0 con 0x7f0270160d00 2023-07-06T03:49:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.806+0000 7f0277bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02700b6640 con 0x7f0270160180 2023-07-06T03:49:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.806+0000 7f0277bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02700cd8d0 con 0x7f027015c520 2023-07-06T03:49:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.806+0000 7f027619c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0270160d00 0x7f02700b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56560/0 (socket says 172.21.15.97:56560) 2023-07-06T03:49:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.806+0000 7f027619c700 1 -- 172.21.15.97:0/2637831025 learned_addr learned my addr 172.21.15.97:0/2637831025 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.806+0000 7f027599b700 1 --2- 172.21.15.97:0/2637831025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f027015c520 0x7f02700cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.807+0000 7f0274999700 1 -- 172.21.15.97:0/2637831025 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049534751 0 0) 0x7f02700cdae0 con 0x7f0270160d00 2023-07-06T03:49:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.807+0000 7f0274999700 1 -- 172.21.15.97:0/2637831025 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f025c0035b0 con 0x7f0270160d00 2023-07-06T03:49:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.807+0000 7f0274999700 1 -- 172.21.15.97:0/2637831025 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318660805 0 0) 0x7f02700b6640 con 0x7f0270160180 2023-07-06T03:49:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.807+0000 7f027599b700 1 -- 172.21.15.97:0/2637831025 >> v1:172.21.15.146:6789/0 conn(0x7f0270160180 legacy=0x7f02701605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.807+0000 7f027599b700 1 -- 172.21.15.97:0/2637831025 >> v1:172.21.15.97:6789/0 conn(0x7f0270160d00 legacy=0x7f02700b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.807+0000 7f027599b700 1 -- 172.21.15.97:0/2637831025 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02700b6920 con 0x7f027015c520 2023-07-06T03:49:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.807+0000 7f027599b700 1 --2- 172.21.15.97:0/2637831025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f027015c520 0x7f02700cd390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f026c009870 tx=0x7f026c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=505e7aaa1415cc2d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.807+0000 7f0274999700 1 -- 172.21.15.97:0/2637831025 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c011830 con 0x7f027015c520 2023-07-06T03:49:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.808+0000 7f0274999700 1 -- 172.21.15.97:0/2637831025 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c011990 con 0x7f027015c520 2023-07-06T03:49:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.808+0000 7f0274999700 1 -- 172.21.15.97:0/2637831025 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c011d20 con 0x7f027015c520 2023-07-06T03:49:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.808+0000 7f0277bff700 1 -- 172.21.15.97:0/2637831025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f027015c520 msgr2=0x7f02700cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.808+0000 7f0277bff700 1 --2- 172.21.15.97:0/2637831025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f027015c520 0x7f02700cd390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f026c009870 tx=0x7f026c009b80 comp rx=0 tx=0).stop 2023-07-06T03:49:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.808+0000 7f0277bff700 1 -- 172.21.15.97:0/2637831025 shutdown_connections 2023-07-06T03:49:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.808+0000 7f0277bff700 1 --2- 172.21.15.97:0/2637831025 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f027015c520 0x7f02700cd390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.808+0000 7f0277bff700 1 -- 172.21.15.97:0/2637831025 >> 172.21.15.97:0/2637831025 conn(0x7f0270071790 msgr2=0x7f0270071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.808+0000 7f0277bff700 1 -- 172.21.15.97:0/2637831025 shutdown_connections 2023-07-06T03:49:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.808+0000 7f0277bff700 1 -- 172.21.15.97:0/2637831025 wait complete. 2023-07-06T03:49:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.810+0000 7f0277bff700 1 Processor -- start 2023-07-06T03:49:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.810+0000 7f0277bff700 1 -- start start 2023-07-06T03:49:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f0277bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f027015c520 0x7f0270150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f0277bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0270160180 0x7f02700cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f0277bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0270160d00 0x7f027015dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f0277bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02701f55d0 con 0x7f027015c520 2023-07-06T03:49:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f0277bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02701f58b0 con 0x7f0270160180 2023-07-06T03:49:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f0277bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02701f5b90 con 0x7f0270160d00 2023-07-06T03:49:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f027599b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f027015c520 0x7f0270150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f027599b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f027015c520 0x7f0270150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33712/0 (socket says 172.21.15.97:33712) 2023-07-06T03:49:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f027599b700 1 -- 172.21.15.97:0/4214877412 learned_addr learned my addr 172.21.15.97:0/4214877412 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f027599b700 1 -- 172.21.15.97:0/4214877412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0270160d00 msgr2=0x7f027015dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f027599b700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0270160d00 0x7f027015dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f027599b700 1 -- 172.21.15.97:0/4214877412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0270160180 msgr2=0x7f02700cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.811+0000 7f027519a700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0270160180 0x7f02700cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:21.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.812+0000 7f027599b700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0270160180 0x7f02700cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:21.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.812+0000 7f027599b700 1 -- 172.21.15.97:0/4214877412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c0094a0 con 0x7f027015c520 2023-07-06T03:49:21.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.812+0000 7f027519a700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0270160180 0x7f02700cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.812+0000 7f027599b700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f027015c520 0x7f0270150190 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f026c0090b0 tx=0x7f026c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.812+0000 7f02667fc700 1 -- 172.21.15.97:0/4214877412 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c01a910 con 0x7f027015c520 2023-07-06T03:49:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.812+0000 7f02667fc700 1 -- 172.21.15.97:0/4214877412 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c01aa70 con 0x7f027015c520 2023-07-06T03:49:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.812+0000 7f02667fc700 1 -- 172.21.15.97:0/4214877412 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c01ade0 con 0x7f027015c520 2023-07-06T03:49:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.812+0000 7f0277bff700 1 -- 172.21.15.97:0/4214877412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02701f5e70 con 0x7f027015c520 2023-07-06T03:49:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.812+0000 7f0277bff700 1 -- 172.21.15.97:0/4214877412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02700d1f00 con 0x7f027015c520 2023-07-06T03:49:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.813+0000 7f0277bff700 1 -- 172.21.15.97:0/4214877412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02701f6390 con 0x7f027015c520 2023-07-06T03:49:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.814+0000 7f02667fc700 1 -- 172.21.15.97:0/4214877412 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f026c022070 con 0x7f027015c520 2023-07-06T03:49:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.815+0000 7f02667fc700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f025406ace0 0x7f025406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.815+0000 7f02667fc700 1 -- 172.21.15.97:0/4214877412 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f026c089f00 con 0x7f027015c520 2023-07-06T03:49:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.815+0000 7f027519a700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f025406ace0 0x7f025406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:21.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.816+0000 7f027519a700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f025406ace0 0x7f025406d190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f02700b1880 tx=0x7f0260009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:21.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:21.820+0000 7f02667fc700 1 -- 172.21.15.97:0/4214877412 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f026c0553f0 con 0x7f027015c520 2023-07-06T03:49:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.057+0000 7f0277bff700 1 -- 172.21.15.97:0/4214877412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f027004e340 con 0x7f027015c520 2023-07-06T03:49:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.057+0000 7f02667fc700 1 -- 172.21.15.97:0/4214877412 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f026c029020 con 0x7f027015c520 2023-07-06T03:49:22.059 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:22.060 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1108873506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1108873506},{"type":"v1","addr":"172.21.15.146:6835","nonce":1108873506}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5801,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/487899884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":487899884},{"type":"v1","addr":"172.21.15.146:6834","nonce":487899884}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6010,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2471899168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2471899168},{"type":"v1","addr":"172.21.15.97:6837","nonce":2471899168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2970792733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2970792733},{"type":"v1","addr":"172.21.15.97:6836","nonce":2970792733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[]} 2023-07-06T03:49:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 -- 172.21.15.97:0/4214877412 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f025406ace0 msgr2=0x7f025406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f025406ace0 0x7f025406d190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f02700b1880 tx=0x7f0260009040 comp rx=0 tx=0).stop 2023-07-06T03:49:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 -- 172.21.15.97:0/4214877412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f027015c520 msgr2=0x7f0270150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f027015c520 0x7f0270150190 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f026c0090b0 tx=0x7f026c004750 comp rx=0 tx=0).stop 2023-07-06T03:49:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 -- 172.21.15.97:0/4214877412 shutdown_connections 2023-07-06T03:49:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f025406ace0 0x7f025406d190 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0270160d00 0x7f027015dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f027015c520 0x7f0270150190 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 --2- 172.21.15.97:0/4214877412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0270160180 0x7f02700cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 -- 172.21.15.97:0/4214877412 >> 172.21.15.97:0/4214877412 conn(0x7f0270071790 msgr2=0x7f02700b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 -- 172.21.15.97:0/4214877412 shutdown_connections 2023-07-06T03:49:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:22.060+0000 7f0277bff700 1 -- 172.21.15.97:0/4214877412 wait complete. 2023-07-06T03:49:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 36 2023-07-06T03:49:22.080 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T03:49:22.080 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T03:49:22.081 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T03:49:22.081 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:49:22.081 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T03:49:22.081 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T03:49:22.082 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T03:49:22.082 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:49:22.082 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T03:49:22.082 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T03:49:22.082 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T03:49:22.083 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:49:22.083 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T03:49:22.083 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T03:49:22.083 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T03:49:22.083 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:49:22.094 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T03:49:22.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T03:49:22.095 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T03:49:22.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T03:49:22.099 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T03:49:22.099 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T03:49:22.139 INFO:tasks.ceph.mds.a:Started 2023-07-06T03:49:22.140 DEBUG:teuthology.parallel:result is None 2023-07-06T03:49:22.142 INFO:tasks.ceph.mds.c:Started 2023-07-06T03:49:22.143 DEBUG:teuthology.parallel:result is None 2023-07-06T03:49:22.144 INFO:tasks.ceph.mds.d:Started 2023-07-06T03:49:22.144 DEBUG:teuthology.parallel:result is None 2023-07-06T03:49:22.284 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:49:22.284 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:49:22.288 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:49:24.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-07-06T03:49:28.099 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T03:49:28.100 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T03:49:28.105 INFO:tasks.ceph.mds.b:Started 2023-07-06T03:49:28.105 DEBUG:teuthology.parallel:result is None 2023-07-06T03:49:28.106 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T03:49:28.257 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-07-06T03:49:28.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.309+0000 7ff2ec9ff700 1 Processor -- start 2023-07-06T03:49:28.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.310+0000 7ff2ec9ff700 1 -- start start 2023-07-06T03:49:28.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.311+0000 7ff2ec9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff2e815c1c0 0x7ff2e80cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:28.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.311+0000 7ff2ec9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff2e80719c0 0x7ff2e806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:28.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.311+0000 7ff2ec9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e815da10 con 0x7ff2e815c590 2023-07-06T03:49:28.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.311+0000 7ff2ec9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e80ccc30 con 0x7ff2e815c1c0 2023-07-06T03:49:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.311+0000 7ff2ec9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e80ccf10 con 0x7ff2e80719c0 2023-07-06T03:49:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.311+0000 7ff2e659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff2e815c1c0 0x7ff2e80cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.311+0000 7ff2e5d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff2e815c590 0x7ff2e815d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56574/0 (socket says 172.21.15.97:56574) 2023-07-06T03:49:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.311+0000 7ff2e5d9b700 1 -- 172.21.15.97:0/3648768481 learned_addr learned my addr 172.21.15.97:0/3648768481 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.311+0000 7ff2e6d9d700 1 --2- 172.21.15.97:0/3648768481 >> v2:172.21.15.146:3300/0 conn(0x7ff2e80719c0 0x7ff2e806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:28.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.312+0000 7ff2e659c700 1 -- 172.21.15.97:0/3648768481 >> v2:172.21.15.146:3300/0 conn(0x7ff2e80719c0 msgr2=0x7ff2e806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.312+0000 7ff2e659c700 1 --2- 172.21.15.97:0/3648768481 >> v2:172.21.15.146:3300/0 conn(0x7ff2e80719c0 0x7ff2e806a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.312+0000 7ff2e659c700 1 -- 172.21.15.97:0/3648768481 >> v1:172.21.15.97:6789/0 conn(0x7ff2e815c590 legacy=0x7ff2e815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.312+0000 7ff2e659c700 1 -- 172.21.15.97:0/3648768481 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e80cd1f0 con 0x7ff2e815c1c0 2023-07-06T03:49:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.312+0000 7ff2e559a700 1 -- 172.21.15.97:0/3648768481 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393241010 0 0) 0x7ff2dc001110 con 0x7ff2e815c590 2023-07-06T03:49:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.312+0000 7ff2e659c700 1 --2- 172.21.15.97:0/3648768481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e815c1c0 0x7ff2e80cc3e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80b26c0 tx=0x7ff2d8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5346c765ca803181 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.312+0000 7ff2e559a700 1 -- 172.21.15.97:0/3648768481 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8011830 con 0x7ff2e815c1c0 2023-07-06T03:49:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.312+0000 7ff2e559a700 1 -- 172.21.15.97:0/3648768481 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d8011990 con 0x7ff2e815c1c0 2023-07-06T03:49:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.312+0000 7ff2e559a700 1 -- 172.21.15.97:0/3648768481 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8011d20 con 0x7ff2e815c1c0 2023-07-06T03:49:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.313+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3648768481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e815c1c0 msgr2=0x7ff2e80cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.313+0000 7ff2ec9ff700 1 --2- 172.21.15.97:0/3648768481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e815c1c0 0x7ff2e80cc3e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80b26c0 tx=0x7ff2d8009b20 comp rx=0 tx=0).stop 2023-07-06T03:49:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.313+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3648768481 shutdown_connections 2023-07-06T03:49:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.313+0000 7ff2ec9ff700 1 --2- 172.21.15.97:0/3648768481 >> v2:172.21.15.146:3300/0 conn(0x7ff2e80719c0 0x7ff2e806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.313+0000 7ff2ec9ff700 1 --2- 172.21.15.97:0/3648768481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e815c1c0 0x7ff2e80cc3e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.313+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3648768481 >> 172.21.15.97:0/3648768481 conn(0x7ff2e80b1650 msgr2=0x7ff2e80b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.313+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3648768481 shutdown_connections 2023-07-06T03:49:28.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.313+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3648768481 wait complete. 2023-07-06T03:49:28.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.315+0000 7ff2ec9ff700 1 Processor -- start 2023-07-06T03:49:28.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.315+0000 7ff2ec9ff700 1 -- start start 2023-07-06T03:49:28.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2ec9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2e80719c0 0x7ff2e81f1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:28.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2ec9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e815c1c0 0x7ff2e80b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:28.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2ec9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2e815c590 0x7ff2e81591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:28.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2ec9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e81f1d50 con 0x7ff2e815c1c0 2023-07-06T03:49:28.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2ec9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e80b3230 con 0x7ff2e815c590 2023-07-06T03:49:28.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2ec9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e80b3510 con 0x7ff2e80719c0 2023-07-06T03:49:28.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2e5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e815c1c0 0x7ff2e80b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:28.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2e5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e815c1c0 0x7ff2e80b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33758/0 (socket says 172.21.15.97:33758) 2023-07-06T03:49:28.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2e5d9b700 1 -- 172.21.15.97:0/3460459066 learned_addr learned my addr 172.21.15.97:0/3460459066 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:28.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2e659c700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2e80719c0 0x7ff2e81f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:28.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2e6d9d700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2e815c590 0x7ff2e81591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:28.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2e5d9b700 1 -- 172.21.15.97:0/3460459066 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2e80719c0 msgr2=0x7ff2e81f1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:28.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2e5d9b700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2e80719c0 0x7ff2e81f1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:28.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2e5d9b700 1 -- 172.21.15.97:0/3460459066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2e815c590 msgr2=0x7ff2e81591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:28.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2e5d9b700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2e815c590 0x7ff2e81591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:28.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.316+0000 7ff2e5d9b700 1 -- 172.21.15.97:0/3460459066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d80094a0 con 0x7ff2e815c1c0 2023-07-06T03:49:28.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.317+0000 7ff2e5d9b700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e815c1c0 0x7ff2e80b2950 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00c520 tx=0x7ff2dc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:28.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.317+0000 7ff2d6ffd700 1 -- 172.21.15.97:0/3460459066 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc004c80 con 0x7ff2e815c1c0 2023-07-06T03:49:28.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.317+0000 7ff2d6ffd700 1 -- 172.21.15.97:0/3460459066 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2dc004de0 con 0x7ff2e815c1c0 2023-07-06T03:49:28.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.317+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3460459066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e80b37f0 con 0x7ff2e815c1c0 2023-07-06T03:49:28.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.317+0000 7ff2d6ffd700 1 -- 172.21.15.97:0/3460459066 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0072d0 con 0x7ff2e815c1c0 2023-07-06T03:49:28.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.317+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3460459066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e815e6c0 con 0x7ff2e815c1c0 2023-07-06T03:49:28.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.318+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3460459066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e815ec30 con 0x7ff2e815c1c0 2023-07-06T03:49:28.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.319+0000 7ff2d6ffd700 1 -- 172.21.15.97:0/3460459066 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff2dc007430 con 0x7ff2e815c1c0 2023-07-06T03:49:28.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.320+0000 7ff2d6ffd700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff2c806ace0 0x7ff2c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:28.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.320+0000 7ff2d6ffd700 1 -- 172.21.15.97:0/3460459066 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff2dc088cd0 con 0x7ff2e815c1c0 2023-07-06T03:49:28.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.320+0000 7ff2e659c700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff2c806ace0 0x7ff2c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:28.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.321+0000 7ff2e659c700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff2c806ace0 0x7ff2c806d190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80090b0 tx=0x7ff2d8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:28.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.324+0000 7ff2d6ffd700 1 -- 172.21.15.97:0/3460459066 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff2e815ec30 con 0x7ff2e815c1c0 2023-07-06T03:49:28.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:28.534+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3460459066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff2e815da10 con 0x7ff2e815c1c0 2023-07-06T03:49:29.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.676+0000 7ff2d6ffd700 1 -- 172.21.15.97:0/3460459066 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v70) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff2e815da10 con 0x7ff2e815c1c0 2023-07-06T03:49:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.679+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3460459066 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff2c806ace0 msgr2=0x7ff2c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.679+0000 7ff2ec9ff700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff2c806ace0 0x7ff2c806d190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80090b0 tx=0x7ff2d8008030 comp rx=0 tx=0).stop 2023-07-06T03:49:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.679+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3460459066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e815c1c0 msgr2=0x7ff2e80b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.679+0000 7ff2ec9ff700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e815c1c0 0x7ff2e80b2950 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00c520 tx=0x7ff2dc00c8e0 comp rx=0 tx=0).stop 2023-07-06T03:49:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.679+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3460459066 shutdown_connections 2023-07-06T03:49:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.679+0000 7ff2ec9ff700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff2c806ace0 0x7ff2c806d190 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.679+0000 7ff2ec9ff700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2e815c590 0x7ff2e81591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.679+0000 7ff2ec9ff700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2e80719c0 0x7ff2e81f1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.679+0000 7ff2ec9ff700 1 --2- 172.21.15.97:0/3460459066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e815c1c0 0x7ff2e80b2950 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.679+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3460459066 >> 172.21.15.97:0/3460459066 conn(0x7ff2e80b1650 msgr2=0x7ff2e815b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.680+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3460459066 shutdown_connections 2023-07-06T03:49:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.680+0000 7ff2ec9ff700 1 -- 172.21.15.97:0/3460459066 wait complete. 2023-07-06T03:49:29.683 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-07-06T03:49:29.703 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T03:49:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.902+0000 7f514c5ff700 1 Processor -- start 2023-07-06T03:49:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.903+0000 7f514c5ff700 1 -- start start 2023-07-06T03:49:29.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.904+0000 7f514c5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f51440d2930 0x7f51440d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:29.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.904+0000 7f514c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440b1990 0x7f51440b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:29.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.904+0000 7f514c5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51440d0c70 con 0x7f514415c980 2023-07-06T03:49:29.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.904+0000 7f514c5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51440b2310 con 0x7f51440d2930 2023-07-06T03:49:29.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.904+0000 7f514c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51440d0ec0 con 0x7f51440b1990 2023-07-06T03:49:29.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.904+0000 7f5149b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f51440d2930 0x7f51440d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:29.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.904+0000 7f5149b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f51440d2930 0x7f51440d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45956/0 (socket says 172.21.15.97:45956) 2023-07-06T03:49:29.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.904+0000 7f5149b9a700 1 -- 172.21.15.97:0/3570483464 learned_addr learned my addr 172.21.15.97:0/3570483464 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:29.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.904+0000 7f514ab9c700 1 --2- 172.21.15.97:0/3570483464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440b1990 0x7f51440b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.904+0000 7f5149b9a700 1 -- 172.21.15.97:0/3570483464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440b1990 msgr2=0x7f51440b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.904+0000 7f5149b9a700 1 --2- 172.21.15.97:0/3570483464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440b1990 0x7f51440b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.905+0000 7f5149b9a700 1 -- 172.21.15.97:0/3570483464 >> v1:172.21.15.146:6789/0 conn(0x7f514415c980 legacy=0x7f51440cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.905+0000 7f5149b9a700 1 -- 172.21.15.97:0/3570483464 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440d11a0 con 0x7f51440d2930 2023-07-06T03:49:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.905+0000 7f514ab9c700 1 --2- 172.21.15.97:0/3570483464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440b1990 0x7f51440b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.905+0000 7f5149b9a700 1 --2- 172.21.15.97:0/3570483464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51440d2930 0x7f51440d1c80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f51340098f0 tx=0x7f5134009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca31ae25dea6b3f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.905+0000 7f5149399700 1 -- 172.21.15.97:0/3570483464 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134004230 con 0x7f51440d2930 2023-07-06T03:49:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.905+0000 7f5149399700 1 -- 172.21.15.97:0/3570483464 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5134004390 con 0x7f51440d2930 2023-07-06T03:49:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.905+0000 7f5149399700 1 -- 172.21.15.97:0/3570483464 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51340036a0 con 0x7f51440d2930 2023-07-06T03:49:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.905+0000 7f514c5ff700 1 -- 172.21.15.97:0/3570483464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51440d2930 msgr2=0x7f51440d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.905+0000 7f514c5ff700 1 --2- 172.21.15.97:0/3570483464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51440d2930 0x7f51440d1c80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f51340098f0 tx=0x7f5134009c00 comp rx=0 tx=0).stop 2023-07-06T03:49:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.906+0000 7f514c5ff700 1 -- 172.21.15.97:0/3570483464 shutdown_connections 2023-07-06T03:49:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.906+0000 7f514c5ff700 1 --2- 172.21.15.97:0/3570483464 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440b1990 0x7f51440b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.906+0000 7f514c5ff700 1 --2- 172.21.15.97:0/3570483464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51440d2930 0x7f51440d1c80 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.906+0000 7f514c5ff700 1 -- 172.21.15.97:0/3570483464 >> 172.21.15.97:0/3570483464 conn(0x7f514406b2f0 msgr2=0x7f514406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.906+0000 7f514c5ff700 1 -- 172.21.15.97:0/3570483464 shutdown_connections 2023-07-06T03:49:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.906+0000 7f514c5ff700 1 -- 172.21.15.97:0/3570483464 wait complete. 2023-07-06T03:49:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.907+0000 7f514c5ff700 1 Processor -- start 2023-07-06T03:49:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.908+0000 7f514c5ff700 1 -- start start 2023-07-06T03:49:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.908+0000 7f514c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440b1990 0x7f51440ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.908+0000 7f514c5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51440d2930 0x7f514415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:29.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.908+0000 7f514c5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514415c980 0x7f514415eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:29.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.908+0000 7f514c5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440cab90 con 0x7f51440d2930 2023-07-06T03:49:29.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.908+0000 7f514c5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514415cfb0 con 0x7f514415c980 2023-07-06T03:49:29.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.908+0000 7f514c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51440ccd20 con 0x7f51440b1990 2023-07-06T03:49:29.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f5149b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51440d2930 0x7f514415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:29.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f5149b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51440d2930 0x7f514415e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45960/0 (socket says 172.21.15.97:45960) 2023-07-06T03:49:29.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f5149b9a700 1 -- 172.21.15.97:0/4071843266 learned_addr learned my addr 172.21.15.97:0/4071843266 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:29.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f514a39b700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440b1990 0x7f51440ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:29.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f514ab9c700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514415c980 0x7f514415eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:29.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f5149b9a700 1 -- 172.21.15.97:0/4071843266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440b1990 msgr2=0x7f51440ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:29.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f5149b9a700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440b1990 0x7f51440ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:29.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f5149b9a700 1 -- 172.21.15.97:0/4071843266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514415c980 msgr2=0x7f514415eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:29.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f5149b9a700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514415c980 0x7f514415eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:29.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f5149b9a700 1 -- 172.21.15.97:0/4071843266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5134009550 con 0x7f51440d2930 2023-07-06T03:49:29.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f514ab9c700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514415c980 0x7f514415eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:29.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.909+0000 7f5149b9a700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51440d2930 0x7f514415e5b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5134009130 tx=0x7f5134004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:29.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.910+0000 7f513affd700 1 -- 172.21.15.97:0/4071843266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134003d80 con 0x7f51440d2930 2023-07-06T03:49:29.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.910+0000 7f513affd700 1 -- 172.21.15.97:0/4071843266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513401f070 con 0x7f51440d2930 2023-07-06T03:49:29.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.910+0000 7f513affd700 1 -- 172.21.15.97:0/4071843266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400b8a0 con 0x7f51440d2930 2023-07-06T03:49:29.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.910+0000 7f514c5ff700 1 -- 172.21.15.97:0/4071843266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51440cd000 con 0x7f51440d2930 2023-07-06T03:49:29.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.910+0000 7f514c5ff700 1 -- 172.21.15.97:0/4071843266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514415d4e0 con 0x7f51440d2930 2023-07-06T03:49:29.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.911+0000 7f514c5ff700 1 -- 172.21.15.97:0/4071843266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514404e340 con 0x7f51440d2930 2023-07-06T03:49:29.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.912+0000 7f513affd700 1 -- 172.21.15.97:0/4071843266 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5134021030 con 0x7f51440d2930 2023-07-06T03:49:29.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.913+0000 7f513affd700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806acb0 0x7f512806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:29.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.913+0000 7f513affd700 1 -- 172.21.15.97:0/4071843266 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f513408a7c0 con 0x7f51440d2930 2023-07-06T03:49:29.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.913+0000 7f514a39b700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806acb0 0x7f512806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:29.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.914+0000 7f514a39b700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806acb0 0x7f512806d160 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5140009960 tx=0x7f51400090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:29.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:29.917+0000 7f513affd700 1 -- 172.21.15.97:0/4071843266 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5134055cd0 con 0x7f51440d2930 2023-07-06T03:49:30.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.186+0000 7f514c5ff700 1 -- 172.21.15.97:0/4071843266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f514404e340 con 0x7f51440d2930 2023-07-06T03:49:30.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.187+0000 7f513affd700 1 -- 172.21.15.97:0/4071843266 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f51340592f0 con 0x7f51440d2930 2023-07-06T03:49:30.189 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:30.190 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:49:30.190 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-07-06T03:49:30.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.190 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-07-06T03:49:30.191 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkczuvGRAAUKjfmMyKOTCYzOQmtf3n3A==", 2023-07-06T03:49:30.191 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.191 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:49:30.191 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:49:30.191 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:49:30.191 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.192 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-07-06T03:49:30.192 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkFHvUMBAA2WgMcUUf+Qi8T2ZjhfWaHA==", 2023-07-06T03:49:30.192 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.192 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:49:30.192 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:49:30.193 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:49:30.193 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.193 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.193 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-07-06T03:49:30.193 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZk0N4+JRAA3tjqoIBPpZ6C7TMmOOg7lw==", 2023-07-06T03:49:30.194 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.194 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:49:30.194 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:49:30.194 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:49:30.194 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.195 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.195 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-07-06T03:49:30.195 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkJnHUABAAwIMH2DmoHaQN3frJCgRiQw==", 2023-07-06T03:49:30.195 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.195 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:49:30.196 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:49:30.196 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:49:30.196 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.198 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-07-06T03:49:30.198 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCLOKZkShOIABAAzIBHDcNkmnDOLSY7Qdb+FA==", 2023-07-06T03:49:30.198 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.198 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:49:30.199 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:49:30.199 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.199 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.199 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-07-06T03:49:30.200 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCOOKZkq1t7AxAAwihuhimbjitZDeki1QuMug==", 2023-07-06T03:49:30.200 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.200 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:49:30.200 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:49:30.200 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.200 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.201 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-07-06T03:49:30.201 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCROKZkdYv4BhAApTHg/pmlC0lO7hVTkimEgw==", 2023-07-06T03:49:30.201 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.201 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:49:30.201 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:49:30.201 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.202 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.202 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-07-06T03:49:30.202 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCUOKZkqhuMChAAsV9xG6g1dofrpWt3F8tw5w==", 2023-07-06T03:49:30.202 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.202 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:49:30.203 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:49:30.203 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.203 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.203 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-07-06T03:49:30.203 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCaOKZk1Q5aIBAABU1+afwIihm+g4xylryz/A==", 2023-07-06T03:49:30.204 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.204 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:49:30.204 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:49:30.204 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.204 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.205 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-07-06T03:49:30.205 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCdOKZkRskIJBAA+821aoG98aEV7h4FTINQFA==", 2023-07-06T03:49:30.205 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.205 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:49:30.205 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:49:30.206 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.206 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.206 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-07-06T03:49:30.206 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCgOKZk9qnMJxAATcWG+a/9Hl1AuIL/TSrdEA==", 2023-07-06T03:49:30.206 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:49:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:49:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.207 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-07-06T03:49:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCjOKZk6wJ/KxAAWnZxt7h7OJhQ+bqdel4+iQ==", 2023-07-06T03:49:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:49:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:49:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-07-06T03:49:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg==", 2023-07-06T03:49:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:49:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-07-06T03:49:30.209 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-06T03:49:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-07-06T03:49:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkq4/pGRAAucHmR1eBA8MNMy3/l7tl0g==", 2023-07-06T03:49:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.210 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:49:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:49:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:49:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:49:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-07-06T03:49:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkhL5fEhAAWhw4IbCPqTO2sXPIcpTVHA==", 2023-07-06T03:49:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:49:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:49:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:49:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:49:30.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-07-06T03:49:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkO81dIRAAffjVyvsb/ygBtx8UrYQK0w==", 2023-07-06T03:49:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:49:30.213 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:49:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:49:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:49:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-07-06T03:49:30.214 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCFOKZk6uVlHRAA92RDhdDKW/q65SEOQMWgiA==", 2023-07-06T03:49:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:49:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:49:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:49:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-07-06T03:49:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkT4ALEhAAXwwkcodQU9meJA0tfseJCg==", 2023-07-06T03:49:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:49:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:49:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.216 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-07-06T03:49:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkfnoEChAAuRxqXqPI33dgWkBJoUMh3w==", 2023-07-06T03:49:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:49:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:49:30.217 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:49:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:49:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:30.218 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:49:30.218 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:49:30.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.189+0000 7f514c5ff700 1 -- 172.21.15.97:0/4071843266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806acb0 msgr2=0x7f512806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.189+0000 7f514c5ff700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806acb0 0x7f512806d160 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5140009960 tx=0x7f51400090e0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.189+0000 7f514c5ff700 1 -- 172.21.15.97:0/4071843266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51440d2930 msgr2=0x7f514415e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.189+0000 7f514c5ff700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51440d2930 0x7f514415e5b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5134009130 tx=0x7f5134004750 comp rx=0 tx=0).stop 2023-07-06T03:49:30.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.189+0000 7f514c5ff700 1 -- 172.21.15.97:0/4071843266 shutdown_connections 2023-07-06T03:49:30.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.189+0000 7f514c5ff700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806acb0 0x7f512806d160 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.189+0000 7f514c5ff700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f514415c980 0x7f514415eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.189+0000 7f514c5ff700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440b1990 0x7f51440ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.189+0000 7f514c5ff700 1 --2- 172.21.15.97:0/4071843266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51440d2930 0x7f514415e5b0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.189+0000 7f514c5ff700 1 -- 172.21.15.97:0/4071843266 >> 172.21.15.97:0/4071843266 conn(0x7f514406b2f0 msgr2=0x7f514415b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:30.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.189+0000 7f514c5ff700 1 -- 172.21.15.97:0/4071843266 shutdown_connections 2023-07-06T03:49:30.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.190+0000 7f514c5ff700 1 -- 172.21.15.97:0/4071843266 wait complete. 2023-07-06T03:49:30.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:49:30.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.414+0000 7f9f653ff700 1 Processor -- start 2023-07-06T03:49:30.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.414+0000 7f9f653ff700 1 -- start start 2023-07-06T03:49:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.415+0000 7f9f653ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9f60158960 0x7f9f6006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.415+0000 7f9f653ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f60071a00 0x7f9f60158d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.415+0000 7f9f653ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6006ddb0 con 0x7f9f6015ce90 2023-07-06T03:49:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.415+0000 7f9f653ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f6006bbf0 con 0x7f9f60158960 2023-07-06T03:49:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.415+0000 7f9f653ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6006bed0 con 0x7f9f60071a00 2023-07-06T03:49:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.416+0000 7f9f5effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9f60158960 0x7f9f6006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.416+0000 7f9f5effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9f60158960 0x7f9f6006d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:41802/0 (socket says 172.21.15.97:41802) 2023-07-06T03:49:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.416+0000 7f9f5effd700 1 -- 172.21.15.97:0/815516046 learned_addr learned my addr 172.21.15.97:0/815516046 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:30.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.416+0000 7f9f5f7fe700 1 --2- 172.21.15.97:0/815516046 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f60071a00 0x7f9f60158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.416+0000 7f9f5effd700 1 -- 172.21.15.97:0/815516046 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f60071a00 msgr2=0x7f9f60158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:30.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.416+0000 7f9f5effd700 1 --2- 172.21.15.97:0/815516046 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f60071a00 0x7f9f60158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.416+0000 7f9f5effd700 1 -- 172.21.15.97:0/815516046 >> v1:172.21.15.146:6789/0 conn(0x7f9f6015ce90 legacy=0x7f9f600712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.416+0000 7f9f5effd700 1 -- 172.21.15.97:0/815516046 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6006c1b0 con 0x7f9f60158960 2023-07-06T03:49:30.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.416+0000 7f9f5effd700 1 --2- 172.21.15.97:0/815516046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f60158960 0x7f9f6006d6f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000bb90 tx=0x7f9f5000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae205f82663e2306 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:30.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.417+0000 7f9f5dffb700 1 -- 172.21.15.97:0/815516046 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000dd30 con 0x7f9f60158960 2023-07-06T03:49:30.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.417+0000 7f9f5dffb700 1 -- 172.21.15.97:0/815516046 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5000de90 con 0x7f9f60158960 2023-07-06T03:49:30.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.417+0000 7f9f5dffb700 1 -- 172.21.15.97:0/815516046 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50010690 con 0x7f9f60158960 2023-07-06T03:49:30.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.417+0000 7f9f653ff700 1 -- 172.21.15.97:0/815516046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f60158960 msgr2=0x7f9f6006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.417+0000 7f9f653ff700 1 --2- 172.21.15.97:0/815516046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f60158960 0x7f9f6006d6f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000bb90 tx=0x7f9f5000bea0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.417+0000 7f9f653ff700 1 -- 172.21.15.97:0/815516046 shutdown_connections 2023-07-06T03:49:30.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.417+0000 7f9f653ff700 1 --2- 172.21.15.97:0/815516046 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f60071a00 0x7f9f60158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.417+0000 7f9f653ff700 1 --2- 172.21.15.97:0/815516046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f60158960 0x7f9f6006d6f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.417+0000 7f9f653ff700 1 -- 172.21.15.97:0/815516046 >> 172.21.15.97:0/815516046 conn(0x7f9f600ce8c0 msgr2=0x7f9f600cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:30.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.417+0000 7f9f653ff700 1 -- 172.21.15.97:0/815516046 shutdown_connections 2023-07-06T03:49:30.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.417+0000 7f9f653ff700 1 -- 172.21.15.97:0/815516046 wait complete. 2023-07-06T03:49:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.419+0000 7f9f653ff700 1 Processor -- start 2023-07-06T03:49:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.419+0000 7f9f653ff700 1 -- start start 2023-07-06T03:49:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.420+0000 7f9f653ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60071a00 0x7f9f6014a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.420+0000 7f9f653ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f60158960 0x7f9f600adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.420+0000 7f9f653ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f6015ce90 0x7f9f601442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.420+0000 7f9f653ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6014b0e0 con 0x7f9f60071a00 2023-07-06T03:49:30.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.420+0000 7f9f653ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600ae2c0 con 0x7f9f6015ce90 2023-07-06T03:49:30.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.420+0000 7f9f653ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600ad740 con 0x7f9f60158960 2023-07-06T03:49:30.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.420+0000 7f9f5effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60071a00 0x7f9f6014a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f5effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60071a00 0x7f9f6014a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45968/0 (socket says 172.21.15.97:45968) 2023-07-06T03:49:30.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f5effd700 1 -- 172.21.15.97:0/3741347273 learned_addr learned my addr 172.21.15.97:0/3741347273 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:30.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f5e7fc700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f60158960 0x7f9f600adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f5f7fe700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f6015ce90 0x7f9f601442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f5effd700 1 -- 172.21.15.97:0/3741347273 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f60158960 msgr2=0x7f9f600adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f5effd700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f60158960 0x7f9f600adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f5effd700 1 -- 172.21.15.97:0/3741347273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f6015ce90 msgr2=0x7f9f601442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f5effd700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f6015ce90 0x7f9f601442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f5effd700 1 -- 172.21.15.97:0/3741347273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5000b7f0 con 0x7f9f60071a00 2023-07-06T03:49:30.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f5effd700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60071a00 0x7f9f6014a760 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000bee0 tx=0x7f9f50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:30.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f477fe700 1 -- 172.21.15.97:0/3741347273 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50003b70 con 0x7f9f60071a00 2023-07-06T03:49:30.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f477fe700 1 -- 172.21.15.97:0/3741347273 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f50003cd0 con 0x7f9f60071a00 2023-07-06T03:49:30.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f477fe700 1 -- 172.21.15.97:0/3741347273 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500106b0 con 0x7f9f60071a00 2023-07-06T03:49:30.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.421+0000 7f9f653ff700 1 -- 172.21.15.97:0/3741347273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f600ac980 con 0x7f9f60071a00 2023-07-06T03:49:30.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.422+0000 7f9f653ff700 1 -- 172.21.15.97:0/3741347273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f600acf40 con 0x7f9f60071a00 2023-07-06T03:49:30.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.422+0000 7f9f653ff700 1 -- 172.21.15.97:0/3741347273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f60067830 con 0x7f9f60071a00 2023-07-06T03:49:30.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.423+0000 7f9f477fe700 1 -- 172.21.15.97:0/3741347273 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f50019400 con 0x7f9f60071a00 2023-07-06T03:49:30.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.424+0000 7f9f477fe700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f4c06ace0 0x7f9f4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.424+0000 7f9f477fe700 1 -- 172.21.15.97:0/3741347273 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9f5008a420 con 0x7f9f60071a00 2023-07-06T03:49:30.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.425+0000 7f9f5e7fc700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f4c06ace0 0x7f9f4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.425+0000 7f9f5e7fc700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f4c06ace0 0x7f9f4c06d190 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9f60154a10 tx=0x7f9f54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.428+0000 7f9f477fe700 1 -- 172.21.15.97:0/3741347273 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9f50055780 con 0x7f9f60071a00 2023-07-06T03:49:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.623+0000 7f9f653ff700 1 -- 172.21.15.97:0/3741347273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f6004e340 con 0x7f9f60071a00 2023-07-06T03:49:30.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.624+0000 7f9f477fe700 1 -- 172.21.15.97:0/3741347273 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+14666 (secure 0 0 0) 0x7f9f50058da0 con 0x7f9f60071a00 2023-07-06T03:49:30.626 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:30.627 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":70,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:49:29.672757+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:49:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 -- 172.21.15.97:0/3741347273 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f4c06ace0 msgr2=0x7f9f4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f4c06ace0 0x7f9f4c06d190 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9f60154a10 tx=0x7f9f54009040 comp rx=0 tx=0).stop 2023-07-06T03:49:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 -- 172.21.15.97:0/3741347273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60071a00 msgr2=0x7f9f6014a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60071a00 0x7f9f6014a760 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000bee0 tx=0x7f9f50004750 comp rx=0 tx=0).stop 2023-07-06T03:49:30.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 -- 172.21.15.97:0/3741347273 shutdown_connections 2023-07-06T03:49:30.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9f4c06ace0 0x7f9f4c06d190 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f6015ce90 0x7f9f601442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60071a00 0x7f9f6014a760 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 --2- 172.21.15.97:0/3741347273 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9f60158960 0x7f9f600adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 -- 172.21.15.97:0/3741347273 >> 172.21.15.97:0/3741347273 conn(0x7f9f600ce8c0 msgr2=0x7f9f60151bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:30.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 -- 172.21.15.97:0/3741347273 shutdown_connections 2023-07-06T03:49:30.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.627+0000 7f9f653ff700 1 -- 172.21.15.97:0/3741347273 wait complete. 2023-07-06T03:49:30.647 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:49:30.647 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:49:30.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.843+0000 7f47175ff700 1 Processor -- start 2023-07-06T03:49:30.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.844+0000 7f47175ff700 1 -- start start 2023-07-06T03:49:30.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f47175ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f471015c610 0x7f47100b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f47175ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f47100d2210 0x7f47100d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f47175ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47100cff90 con 0x7f471006b360 2023-07-06T03:49:30.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f47175ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47100d0780 con 0x7f471015c610 2023-07-06T03:49:30.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f47175ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471015c9e0 con 0x7f47100d2210 2023-07-06T03:49:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f4714b9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f471006b360 0x7f47100b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35514/0 (socket says 172.21.15.97:35514) 2023-07-06T03:49:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f4714b9a700 1 -- 172.21.15.97:0/566104699 learned_addr learned my addr 172.21.15.97:0/566104699 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:30.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f471539b700 1 --2- 172.21.15.97:0/566104699 >> v2:172.21.15.97:3300/0 conn(0x7f471015c610 0x7f47100b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f4715b9c700 1 --2- 172.21.15.97:0/566104699 >> v2:172.21.15.146:3300/0 conn(0x7f47100d2210 0x7f47100d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f4707fff700 1 -- 172.21.15.97:0/566104699 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402035045 0 0) 0x7f47100cff90 con 0x7f471006b360 2023-07-06T03:49:30.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f4707fff700 1 -- 172.21.15.97:0/566104699 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46fc0035b0 con 0x7f471006b360 2023-07-06T03:49:30.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.845+0000 7f471539b700 1 -- 172.21.15.97:0/566104699 >> v2:172.21.15.146:3300/0 conn(0x7f47100d2210 msgr2=0x7f47100d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:30.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.846+0000 7f471539b700 1 --2- 172.21.15.97:0/566104699 >> v2:172.21.15.146:3300/0 conn(0x7f47100d2210 0x7f47100d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.846+0000 7f471539b700 1 -- 172.21.15.97:0/566104699 >> v1:172.21.15.97:6789/0 conn(0x7f471006b360 legacy=0x7f47100b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.846+0000 7f471539b700 1 -- 172.21.15.97:0/566104699 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47100b5650 con 0x7f471015c610 2023-07-06T03:49:30.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.846+0000 7f4715b9c700 1 --2- 172.21.15.97:0/566104699 >> v2:172.21.15.146:3300/0 conn(0x7f47100d2210 0x7f47100d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:30.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.846+0000 7f471539b700 1 --2- 172.21.15.97:0/566104699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471015c610 0x7f47100b47b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f47100d2bc0 tx=0x7f470c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0c337b63e1e34e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.846+0000 7f4707fff700 1 -- 172.21.15.97:0/566104699 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c011830 con 0x7f471015c610 2023-07-06T03:49:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.846+0000 7f4707fff700 1 -- 172.21.15.97:0/566104699 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c011990 con 0x7f471015c610 2023-07-06T03:49:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.846+0000 7f4707fff700 1 -- 172.21.15.97:0/566104699 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c011d20 con 0x7f471015c610 2023-07-06T03:49:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.846+0000 7f47175ff700 1 -- 172.21.15.97:0/566104699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471015c610 msgr2=0x7f47100b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.846+0000 7f47175ff700 1 --2- 172.21.15.97:0/566104699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471015c610 0x7f47100b47b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f47100d2bc0 tx=0x7f470c009b50 comp rx=0 tx=0).stop 2023-07-06T03:49:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.847+0000 7f47175ff700 1 -- 172.21.15.97:0/566104699 shutdown_connections 2023-07-06T03:49:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.847+0000 7f47175ff700 1 --2- 172.21.15.97:0/566104699 >> v2:172.21.15.146:3300/0 conn(0x7f47100d2210 0x7f47100d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.847+0000 7f47175ff700 1 --2- 172.21.15.97:0/566104699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471015c610 0x7f47100b47b0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.847+0000 7f47175ff700 1 -- 172.21.15.97:0/566104699 >> 172.21.15.97:0/566104699 conn(0x7f471006a6c0 msgr2=0x7f471006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.847+0000 7f47175ff700 1 -- 172.21.15.97:0/566104699 shutdown_connections 2023-07-06T03:49:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.847+0000 7f47175ff700 1 -- 172.21.15.97:0/566104699 wait complete. 2023-07-06T03:49:30.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.848+0000 7f47175ff700 1 Processor -- start 2023-07-06T03:49:30.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.849+0000 7f47175ff700 1 -- start start 2023-07-06T03:49:30.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.849+0000 7f47175ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f471006b360 0x7f471014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f47175ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47100d2210 0x7f47100cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f47175ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471015c610 0x7f47100cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f47175ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47100ca1c0 con 0x7f471015c610 2023-07-06T03:49:30.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f47175ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47100ca4a0 con 0x7f47100d2210 2023-07-06T03:49:30.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f47175ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47100ca780 con 0x7f471006b360 2023-07-06T03:49:30.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f4715b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471015c610 0x7f47100cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f4715b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471015c610 0x7f47100cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45988/0 (socket says 172.21.15.97:45988) 2023-07-06T03:49:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f4715b9c700 1 -- 172.21.15.97:0/4236570048 learned_addr learned my addr 172.21.15.97:0/4236570048 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f4714b9a700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47100d2210 0x7f47100cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f471539b700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f471006b360 0x7f471014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f4715b9c700 1 -- 172.21.15.97:0/4236570048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f471006b360 msgr2=0x7f471014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f4715b9c700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f471006b360 0x7f471014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f4715b9c700 1 -- 172.21.15.97:0/4236570048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47100d2210 msgr2=0x7f47100cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f4715b9c700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47100d2210 0x7f47100cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.850+0000 7f4715b9c700 1 -- 172.21.15.97:0/4236570048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c0094a0 con 0x7f471015c610 2023-07-06T03:49:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.851+0000 7f4715b9c700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471015c610 0x7f47100cc1b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4708007ef0 tx=0x7f470800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.851+0000 7f4705ffb700 1 -- 172.21.15.97:0/4236570048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800de60 con 0x7f471015c610 2023-07-06T03:49:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.851+0000 7f4705ffb700 1 -- 172.21.15.97:0/4236570048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470800f040 con 0x7f471015c610 2023-07-06T03:49:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.851+0000 7f47175ff700 1 -- 172.21.15.97:0/4236570048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47100caa60 con 0x7f471015c610 2023-07-06T03:49:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.851+0000 7f4705ffb700 1 -- 172.21.15.97:0/4236570048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708003c60 con 0x7f471015c610 2023-07-06T03:49:30.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.851+0000 7f47175ff700 1 -- 172.21.15.97:0/4236570048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471014b690 con 0x7f471015c610 2023-07-06T03:49:30.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.852+0000 7f47175ff700 1 -- 172.21.15.97:0/4236570048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47100b19c0 con 0x7f471015c610 2023-07-06T03:49:30.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.853+0000 7f4705ffb700 1 -- 172.21.15.97:0/4236570048 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4708004030 con 0x7f471015c610 2023-07-06T03:49:30.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.854+0000 7f4705ffb700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f46f406ace0 0x7f46f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:30.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.854+0000 7f4705ffb700 1 -- 172.21.15.97:0/4236570048 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f47080893f0 con 0x7f471015c610 2023-07-06T03:49:30.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.854+0000 7f471539b700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f46f406ace0 0x7f46f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:30.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.855+0000 7f471539b700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f46f406ace0 0x7f46f406d190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f470c000c00 tx=0x7f470c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:30.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:30.858+0000 7f4705ffb700 1 -- 172.21.15.97:0/4236570048 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4708054750 con 0x7f471015c610 2023-07-06T03:49:31.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.052+0000 7f47175ff700 1 -- 172.21.15.97:0/4236570048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47100ce630 con 0x7f471015c610 2023-07-06T03:49:31.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.052+0000 7f4705ffb700 1 -- 172.21.15.97:0/4236570048 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4708057d70 con 0x7f471015c610 2023-07-06T03:49:31.054 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:49:31.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 -- 172.21.15.97:0/4236570048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f46f406ace0 msgr2=0x7f46f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f46f406ace0 0x7f46f406d190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f470c000c00 tx=0x7f470c01c040 comp rx=0 tx=0).stop 2023-07-06T03:49:31.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 -- 172.21.15.97:0/4236570048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471015c610 msgr2=0x7f47100cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471015c610 0x7f47100cc1b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4708007ef0 tx=0x7f470800d3b0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 -- 172.21.15.97:0/4236570048 shutdown_connections 2023-07-06T03:49:31.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f46f406ace0 0x7f46f406d190 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f471015c610 0x7f47100cc1b0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f471006b360 0x7f471014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 --2- 172.21.15.97:0/4236570048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47100d2210 0x7f47100cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 -- 172.21.15.97:0/4236570048 >> 172.21.15.97:0/4236570048 conn(0x7f471006a6c0 msgr2=0x7f47101590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 -- 172.21.15.97:0/4236570048 shutdown_connections 2023-07-06T03:49:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.055+0000 7f47175ff700 1 -- 172.21.15.97:0/4236570048 wait complete. 2023-07-06T03:49:31.075 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:49:31.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.270+0000 7fea505ff700 1 Processor -- start 2023-07-06T03:49:31.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.270+0000 7fea505ff700 1 -- start start 2023-07-06T03:49:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.271+0000 7fea505ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea4815c610 0x7fea480b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.271+0000 7fea505ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fea480d2210 0x7fea480d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.271+0000 7fea505ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea480cff90 con 0x7fea4806b360 2023-07-06T03:49:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.271+0000 7fea505ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea480d0780 con 0x7fea4815c610 2023-07-06T03:49:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.271+0000 7fea505ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4815c9e0 con 0x7fea480d2210 2023-07-06T03:49:31.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.272+0000 7fea4e39b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea4815c610 0x7fea480b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.272+0000 7fea4db9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fea4806b360 0x7fea480b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35524/0 (socket says 172.21.15.97:35524) 2023-07-06T03:49:31.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.272+0000 7fea4db9a700 1 -- 172.21.15.97:0/2919311997 learned_addr learned my addr 172.21.15.97:0/2919311997 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.272+0000 7fea4eb9c700 1 --2- 172.21.15.97:0/2919311997 >> v2:172.21.15.146:3300/0 conn(0x7fea480d2210 0x7fea480d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.272+0000 7fea4e39b700 1 -- 172.21.15.97:0/2919311997 >> v2:172.21.15.146:3300/0 conn(0x7fea480d2210 msgr2=0x7fea480d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:31.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.272+0000 7fea4e39b700 1 --2- 172.21.15.97:0/2919311997 >> v2:172.21.15.146:3300/0 conn(0x7fea480d2210 0x7fea480d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.272+0000 7fea4e39b700 1 -- 172.21.15.97:0/2919311997 >> v1:172.21.15.97:6789/0 conn(0x7fea4806b360 legacy=0x7fea480b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.272+0000 7fea4e39b700 1 -- 172.21.15.97:0/2919311997 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea480b5650 con 0x7fea4815c610 2023-07-06T03:49:31.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.272+0000 7fea4d399700 1 -- 172.21.15.97:0/2919311997 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572257942 0 0) 0x7fea480cff90 con 0x7fea4806b360 2023-07-06T03:49:31.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.272+0000 7fea4e39b700 1 --2- 172.21.15.97:0/2919311997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4815c610 0x7fea480b47b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fea480d2bc0 tx=0x7fea44009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8fdc3d094b3d62 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:31.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.273+0000 7fea4d399700 1 -- 172.21.15.97:0/2919311997 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44011830 con 0x7fea4815c610 2023-07-06T03:49:31.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.273+0000 7fea4d399700 1 -- 172.21.15.97:0/2919311997 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44011990 con 0x7fea4815c610 2023-07-06T03:49:31.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.273+0000 7fea4d399700 1 -- 172.21.15.97:0/2919311997 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44011d20 con 0x7fea4815c610 2023-07-06T03:49:31.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.273+0000 7fea505ff700 1 -- 172.21.15.97:0/2919311997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4815c610 msgr2=0x7fea480b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.273+0000 7fea505ff700 1 --2- 172.21.15.97:0/2919311997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4815c610 0x7fea480b47b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fea480d2bc0 tx=0x7fea44009b50 comp rx=0 tx=0).stop 2023-07-06T03:49:31.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.273+0000 7fea505ff700 1 -- 172.21.15.97:0/2919311997 shutdown_connections 2023-07-06T03:49:31.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.273+0000 7fea505ff700 1 --2- 172.21.15.97:0/2919311997 >> v2:172.21.15.146:3300/0 conn(0x7fea480d2210 0x7fea480d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.273+0000 7fea505ff700 1 --2- 172.21.15.97:0/2919311997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4815c610 0x7fea480b47b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.273+0000 7fea505ff700 1 -- 172.21.15.97:0/2919311997 >> 172.21.15.97:0/2919311997 conn(0x7fea4806a6c0 msgr2=0x7fea4806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:31.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.273+0000 7fea505ff700 1 -- 172.21.15.97:0/2919311997 shutdown_connections 2023-07-06T03:49:31.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.273+0000 7fea505ff700 1 -- 172.21.15.97:0/2919311997 wait complete. 2023-07-06T03:49:31.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.275+0000 7fea505ff700 1 Processor -- start 2023-07-06T03:49:31.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.275+0000 7fea505ff700 1 -- start start 2023-07-06T03:49:31.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.276+0000 7fea505ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4806b360 0x7fea481511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.276+0000 7fea505ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea480d2210 0x7fea480cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.276+0000 7fea505ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea4815c610 0x7fea480cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.276+0000 7fea505ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4814dab0 con 0x7fea480d2210 2023-07-06T03:49:31.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.276+0000 7fea505ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea480ca560 con 0x7fea4815c610 2023-07-06T03:49:31.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.276+0000 7fea505ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea480ca840 con 0x7fea4806b360 2023-07-06T03:49:31.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.276+0000 7fea4db9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea480d2210 0x7fea480cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.276+0000 7fea4db9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea480d2210 0x7fea480cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46010/0 (socket says 172.21.15.97:46010) 2023-07-06T03:49:31.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.276+0000 7fea4db9a700 1 -- 172.21.15.97:0/3274821627 learned_addr learned my addr 172.21.15.97:0/3274821627 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:31.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea4eb9c700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea4815c610 0x7fea480cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea4e39b700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4806b360 0x7fea481511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea4db9a700 1 -- 172.21.15.97:0/3274821627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4806b360 msgr2=0x7fea481511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea4db9a700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4806b360 0x7fea481511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea4db9a700 1 -- 172.21.15.97:0/3274821627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea4815c610 msgr2=0x7fea480cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea4db9a700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea4815c610 0x7fea480cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea4db9a700 1 -- 172.21.15.97:0/3274821627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea440094a0 con 0x7fea480d2210 2023-07-06T03:49:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea4e39b700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4806b360 0x7fea481511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea4db9a700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea480d2210 0x7fea480cb600 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fea380011e0 tx=0x7fea3800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea3effd700 1 -- 172.21.15.97:0/3274821627 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38004c80 con 0x7fea480d2210 2023-07-06T03:49:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea3effd700 1 -- 172.21.15.97:0/3274821627 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea38004e00 con 0x7fea480d2210 2023-07-06T03:49:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea505ff700 1 -- 172.21.15.97:0/3274821627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea480cab20 con 0x7fea480d2210 2023-07-06T03:49:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea3effd700 1 -- 172.21.15.97:0/3274821627 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38007310 con 0x7fea480d2210 2023-07-06T03:49:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.277+0000 7fea505ff700 1 -- 172.21.15.97:0/3274821627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea480cb0e0 con 0x7fea480d2210 2023-07-06T03:49:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.278+0000 7fea505ff700 1 -- 172.21.15.97:0/3274821627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea480cb300 con 0x7fea480d2210 2023-07-06T03:49:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.279+0000 7fea3effd700 1 -- 172.21.15.97:0/3274821627 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea38007470 con 0x7fea480d2210 2023-07-06T03:49:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.280+0000 7fea3effd700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2c06ace0 0x7fea2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.280+0000 7fea3effd700 1 -- 172.21.15.97:0/3274821627 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fea38088f00 con 0x7fea480d2210 2023-07-06T03:49:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.281+0000 7fea4e39b700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2c06ace0 0x7fea2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.281+0000 7fea4e39b700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2c06ace0 0x7fea2c06d190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fea44009b90 tx=0x7fea44008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.284+0000 7fea3effd700 1 -- 172.21.15.97:0/3274821627 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fea38054300 con 0x7fea480d2210 2023-07-06T03:49:31.328 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-07-06T03:49:31.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.479+0000 7fea505ff700 1 -- 172.21.15.97:0/3274821627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fea480b19c0 con 0x7fea480d2210 2023-07-06T03:49:31.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.479+0000 7fea3effd700 1 -- 172.21.15.97:0/3274821627 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v70) v1 ==== 92+0+10 (secure 0 0 0) 0x7fea38057920 con 0x7fea480d2210 2023-07-06T03:49:31.482 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:49:31.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 -- 172.21.15.97:0/3274821627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2c06ace0 msgr2=0x7fea2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2c06ace0 0x7fea2c06d190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fea44009b90 tx=0x7fea44008030 comp rx=0 tx=0).stop 2023-07-06T03:49:31.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 -- 172.21.15.97:0/3274821627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea480d2210 msgr2=0x7fea480cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea480d2210 0x7fea480cb600 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fea380011e0 tx=0x7fea3800cae0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 -- 172.21.15.97:0/3274821627 shutdown_connections 2023-07-06T03:49:31.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2c06ace0 0x7fea2c06d190 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea4815c610 0x7fea480cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4806b360 0x7fea481511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 --2- 172.21.15.97:0/3274821627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea480d2210 0x7fea480cb600 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 -- 172.21.15.97:0/3274821627 >> 172.21.15.97:0/3274821627 conn(0x7fea4806a6c0 msgr2=0x7fea480b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 -- 172.21.15.97:0/3274821627 shutdown_connections 2023-07-06T03:49:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.482+0000 7fea505ff700 1 -- 172.21.15.97:0/3274821627 wait complete. 2023-07-06T03:49:31.502 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.694+0000 7fa12b7ff700 1 Processor -- start 2023-07-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.694+0000 7fa12b7ff700 1 -- start start 2023-07-06T03:49:31.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.696+0000 7fa12b7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa124071a00 0x7fa12406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.696+0000 7fa12b7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa12415ce90 0x7fa124158d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.696+0000 7fa12b7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12406ddb0 con 0x7fa124158960 2023-07-06T03:49:31.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.696+0000 7fa12b7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12406bbf0 con 0x7fa12415ce90 2023-07-06T03:49:31.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.696+0000 7fa12b7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12406bed0 con 0x7fa124071a00 2023-07-06T03:49:31.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.696+0000 7fa129d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa12415ce90 0x7fa124158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.696+0000 7fa129d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa12415ce90 0x7fa124158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46012/0 (socket says 172.21.15.97:46012) 2023-07-06T03:49:31.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.696+0000 7fa129d9c700 1 -- 172.21.15.97:0/1116378593 learned_addr learned my addr 172.21.15.97:0/1116378593 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:31.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.696+0000 7fa128d9a700 1 --2- 172.21.15.97:0/1116378593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa124071a00 0x7fa12406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.697+0000 7fa129d9c700 1 -- 172.21.15.97:0/1116378593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa124071a00 msgr2=0x7fa12406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.697+0000 7fa129d9c700 1 --2- 172.21.15.97:0/1116378593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa124071a00 0x7fa12406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.697+0000 7fa129d9c700 1 -- 172.21.15.97:0/1116378593 >> v1:172.21.15.146:6789/0 conn(0x7fa124158960 legacy=0x7fa1240712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.697+0000 7fa129d9c700 1 -- 172.21.15.97:0/1116378593 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12406c1b0 con 0x7fa12415ce90 2023-07-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.697+0000 7fa11bfff700 1 -- 172.21.15.97:0/1116378593 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846130101 0 0) 0x7fa12406ddb0 con 0x7fa124158960 2023-07-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.697+0000 7fa129d9c700 1 --2- 172.21.15.97:0/1116378593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa12415ce90 0x7fa124158d30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00d150 tx=0x7fa11c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3344c79704aadb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.697+0000 7fa11bfff700 1 -- 172.21.15.97:0/1116378593 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c010070 con 0x7fa12415ce90 2023-07-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.697+0000 7fa11bfff700 1 -- 172.21.15.97:0/1116378593 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c00dd10 con 0x7fa12415ce90 2023-07-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.697+0000 7fa11bfff700 1 -- 172.21.15.97:0/1116378593 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00b310 con 0x7fa12415ce90 2023-07-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.698+0000 7fa12b7ff700 1 -- 172.21.15.97:0/1116378593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa12415ce90 msgr2=0x7fa124158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.698+0000 7fa12b7ff700 1 --2- 172.21.15.97:0/1116378593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa12415ce90 0x7fa124158d30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00d150 tx=0x7fa11c00d460 comp rx=0 tx=0).stop 2023-07-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.698+0000 7fa12b7ff700 1 -- 172.21.15.97:0/1116378593 shutdown_connections 2023-07-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.698+0000 7fa12b7ff700 1 --2- 172.21.15.97:0/1116378593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa12415ce90 0x7fa124158d30 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.698+0000 7fa12b7ff700 1 --2- 172.21.15.97:0/1116378593 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa124071a00 0x7fa12406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.698+0000 7fa12b7ff700 1 -- 172.21.15.97:0/1116378593 >> 172.21.15.97:0/1116378593 conn(0x7fa1240ce8c0 msgr2=0x7fa1240cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.698+0000 7fa12b7ff700 1 -- 172.21.15.97:0/1116378593 shutdown_connections 2023-07-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.698+0000 7fa12b7ff700 1 -- 172.21.15.97:0/1116378593 wait complete. 2023-07-06T03:49:31.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.699+0000 7fa12b7ff700 1 Processor -- start 2023-07-06T03:49:31.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.700+0000 7fa12b7ff700 1 -- start start 2023-07-06T03:49:31.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.700+0000 7fa12b7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa124071a00 0x7fa1241e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.700+0000 7fa12b7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa124158960 0x7fa1241e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.700+0000 7fa12b7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa12415ce90 0x7fa1241ed540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa12b7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1241516e0 con 0x7fa12415ce90 2023-07-06T03:49:31.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa12b7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12414a980 con 0x7fa124071a00 2023-07-06T03:49:31.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa12b7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1241e3750 con 0x7fa124158960 2023-07-06T03:49:31.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa129d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa12415ce90 0x7fa1241ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa129d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa12415ce90 0x7fa1241ed540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46028/0 (socket says 172.21.15.97:46028) 2023-07-06T03:49:31.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa129d9c700 1 -- 172.21.15.97:0/4175121018 learned_addr learned my addr 172.21.15.97:0/4175121018 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa12959b700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa124071a00 0x7fa1241e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa128d9a700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa124158960 0x7fa1241e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa129d9c700 1 -- 172.21.15.97:0/4175121018 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa124158960 msgr2=0x7fa1241e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa129d9c700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa124158960 0x7fa1241e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa129d9c700 1 -- 172.21.15.97:0/4175121018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa124071a00 msgr2=0x7fa1241e29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa129d9c700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa124071a00 0x7fa1241e29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.701+0000 7fa129d9c700 1 -- 172.21.15.97:0/4175121018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c00bd40 con 0x7fa12415ce90 2023-07-06T03:49:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.702+0000 7fa129d9c700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa12415ce90 0x7fa1241ed540 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00d7b0 tx=0x7fa11c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.702+0000 7fa119ffb700 1 -- 172.21.15.97:0/4175121018 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00ac30 con 0x7fa12415ce90 2023-07-06T03:49:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.702+0000 7fa119ffb700 1 -- 172.21.15.97:0/4175121018 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c00ad90 con 0x7fa12415ce90 2023-07-06T03:49:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.702+0000 7fa119ffb700 1 -- 172.21.15.97:0/4175121018 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c004260 con 0x7fa12415ce90 2023-07-06T03:49:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.702+0000 7fa12b7ff700 1 -- 172.21.15.97:0/4175121018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1241e3a30 con 0x7fa12415ce90 2023-07-06T03:49:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.702+0000 7fa12b7ff700 1 -- 172.21.15.97:0/4175121018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1241e3ff0 con 0x7fa12415ce90 2023-07-06T03:49:31.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.703+0000 7fa12b7ff700 1 -- 172.21.15.97:0/4175121018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa124067830 con 0x7fa12415ce90 2023-07-06T03:49:31.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.704+0000 7fa119ffb700 1 -- 172.21.15.97:0/4175121018 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa11c009040 con 0x7fa12415ce90 2023-07-06T03:49:31.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.705+0000 7fa119ffb700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa10806ace0 0x7fa10806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:31.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.705+0000 7fa119ffb700 1 -- 172.21.15.97:0/4175121018 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa11c08a370 con 0x7fa12415ce90 2023-07-06T03:49:31.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.705+0000 7fa12959b700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa10806ace0 0x7fa10806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:31.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.705+0000 7fa12959b700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa10806ace0 0x7fa10806d190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa120000f60 tx=0x7fa120007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:31.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.709+0000 7fa119ffb700 1 -- 172.21.15.97:0/4175121018 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa11c055760 con 0x7fa12415ce90 2023-07-06T03:49:31.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.908+0000 7fa12b7ff700 1 -- 172.21.15.97:0/4175121018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa12404e340 con 0x7fa12415ce90 2023-07-06T03:49:31.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.908+0000 7fa119ffb700 1 -- 172.21.15.97:0/4175121018 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa11c058d80 con 0x7fa12415ce90 2023-07-06T03:49:31.910 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:31.910 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-07-06T03:49:31.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:31.910 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-07-06T03:49:31.911 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-07-06T03:49:31.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:31.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:31.911 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 2, 2023-07-06T03:49:31.911 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": "rbd" 2023-07-06T03:49:31.911 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:31.912 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-07-06T03:49:31.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.910+0000 7fa12b7ff700 1 -- 172.21.15.97:0/4175121018 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa10806ace0 msgr2=0x7fa10806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.910+0000 7fa12b7ff700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa10806ace0 0x7fa10806d190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa120000f60 tx=0x7fa120007dc0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.910+0000 7fa12b7ff700 1 -- 172.21.15.97:0/4175121018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa12415ce90 msgr2=0x7fa1241ed540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:31.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.910+0000 7fa12b7ff700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa12415ce90 0x7fa1241ed540 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00d7b0 tx=0x7fa11c004ad0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.910+0000 7fa12b7ff700 1 -- 172.21.15.97:0/4175121018 shutdown_connections 2023-07-06T03:49:31.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.910+0000 7fa12b7ff700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa10806ace0 0x7fa10806d190 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.911+0000 7fa12b7ff700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa12415ce90 0x7fa1241ed540 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.911+0000 7fa12b7ff700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa124071a00 0x7fa1241e29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.911+0000 7fa12b7ff700 1 --2- 172.21.15.97:0/4175121018 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa124158960 0x7fa1241e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.911+0000 7fa12b7ff700 1 -- 172.21.15.97:0/4175121018 >> 172.21.15.97:0/4175121018 conn(0x7fa1240ce8c0 msgr2=0x7fa124158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.911+0000 7fa12b7ff700 1 -- 172.21.15.97:0/4175121018 shutdown_connections 2023-07-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:31.911+0000 7fa12b7ff700 1 -- 172.21.15.97:0/4175121018 wait complete. 2023-07-06T03:49:31.930 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T03:49:31.930 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T03:49:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.125+0000 7fec4b3ff700 1 Processor -- start 2023-07-06T03:49:32.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.125+0000 7fec4b3ff700 1 -- start start 2023-07-06T03:49:32.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.126+0000 7fec4b3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fec440d0940 0x7fec440d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:32.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4b3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fec440b84f0 0x7fec4415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:32.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4b3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec440b8060 con 0x7fec440b5ca0 2023-07-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4b3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec440b71c0 con 0x7fec440d0940 2023-07-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4b3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec440d1250 con 0x7fec440b84f0 2023-07-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4919b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fec440d0940 0x7fec440d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4899a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fec440b5ca0 0x7fec440b6ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35534/0 (socket says 172.21.15.97:35534) 2023-07-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4899a700 1 -- 172.21.15.97:0/4074536360 learned_addr learned my addr 172.21.15.97:0/4074536360 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4999c700 1 --2- 172.21.15.97:0/4074536360 >> v2:172.21.15.146:3300/0 conn(0x7fec440b84f0 0x7fec4415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4919b700 1 -- 172.21.15.97:0/4074536360 >> v2:172.21.15.146:3300/0 conn(0x7fec440b84f0 msgr2=0x7fec4415cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4919b700 1 --2- 172.21.15.97:0/4074536360 >> v2:172.21.15.146:3300/0 conn(0x7fec440b84f0 0x7fec4415cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4919b700 1 -- 172.21.15.97:0/4074536360 >> v1:172.21.15.97:6789/0 conn(0x7fec440b5ca0 legacy=0x7fec440b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec4919b700 1 -- 172.21.15.97:0/4074536360 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440b7450 con 0x7fec440d0940 2023-07-06T03:49:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.127+0000 7fec3bfff700 1 -- 172.21.15.97:0/4074536360 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497898711 0 0) 0x7fec440b8060 con 0x7fec440b5ca0 2023-07-06T03:49:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.128+0000 7fec4919b700 1 --2- 172.21.15.97:0/4074536360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec440d0940 0x7fec440d0d10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fec4408cfc0 tx=0x7fec40009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665c082eaecbda2e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:32.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.128+0000 7fec3bfff700 1 -- 172.21.15.97:0/4074536360 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40011830 con 0x7fec440d0940 2023-07-06T03:49:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.128+0000 7fec3bfff700 1 -- 172.21.15.97:0/4074536360 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec40011990 con 0x7fec440d0940 2023-07-06T03:49:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.128+0000 7fec3bfff700 1 -- 172.21.15.97:0/4074536360 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40011d20 con 0x7fec440d0940 2023-07-06T03:49:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.128+0000 7fec4b3ff700 1 -- 172.21.15.97:0/4074536360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec440d0940 msgr2=0x7fec440d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.128+0000 7fec4b3ff700 1 --2- 172.21.15.97:0/4074536360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec440d0940 0x7fec440d0d10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fec4408cfc0 tx=0x7fec40009b20 comp rx=0 tx=0).stop 2023-07-06T03:49:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.128+0000 7fec4b3ff700 1 -- 172.21.15.97:0/4074536360 shutdown_connections 2023-07-06T03:49:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.128+0000 7fec4b3ff700 1 --2- 172.21.15.97:0/4074536360 >> v2:172.21.15.146:3300/0 conn(0x7fec440b84f0 0x7fec4415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:32.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.128+0000 7fec4b3ff700 1 --2- 172.21.15.97:0/4074536360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec440d0940 0x7fec440d0d10 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:32.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.128+0000 7fec4b3ff700 1 -- 172.21.15.97:0/4074536360 >> 172.21.15.97:0/4074536360 conn(0x7fec4409c040 msgr2=0x7fec440d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:32.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.128+0000 7fec4b3ff700 1 -- 172.21.15.97:0/4074536360 shutdown_connections 2023-07-06T03:49:32.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.129+0000 7fec4b3ff700 1 -- 172.21.15.97:0/4074536360 wait complete. 2023-07-06T03:49:32.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.130+0000 7fec4b3ff700 1 Processor -- start 2023-07-06T03:49:32.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.130+0000 7fec4b3ff700 1 -- start start 2023-07-06T03:49:32.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.131+0000 7fec4b3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec440b5ca0 0x7fec4409e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:32.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.131+0000 7fec4b3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec440b84f0 0x7fec44158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:32.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.131+0000 7fec4b3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec440d0940 0x7fec44157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:32.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.131+0000 7fec4b3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec441575c0 con 0x7fec440b84f0 2023-07-06T03:49:32.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.131+0000 7fec4b3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec441577f0 con 0x7fec440b5ca0 2023-07-06T03:49:32.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.131+0000 7fec4b3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec44157ad0 con 0x7fec440d0940 2023-07-06T03:49:32.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.131+0000 7fec4899a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec440b84f0 0x7fec44158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:32.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.131+0000 7fec4899a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec440b84f0 0x7fec44158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46046/0 (socket says 172.21.15.97:46046) 2023-07-06T03:49:32.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.131+0000 7fec4899a700 1 -- 172.21.15.97:0/3711624842 learned_addr learned my addr 172.21.15.97:0/3711624842 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:32.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4999c700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec440d0940 0x7fec44157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4919b700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec440b5ca0 0x7fec4409e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4899a700 1 -- 172.21.15.97:0/3711624842 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec440d0940 msgr2=0x7fec44157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4899a700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec440d0940 0x7fec44157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4899a700 1 -- 172.21.15.97:0/3711624842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec440b5ca0 msgr2=0x7fec4409e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4899a700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec440b5ca0 0x7fec4409e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4899a700 1 -- 172.21.15.97:0/3711624842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400094a0 con 0x7fec440b84f0 2023-07-06T03:49:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4999c700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec440d0940 0x7fec44157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:32.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4919b700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec440b5ca0 0x7fec4409e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:32.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4899a700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec440b84f0 0x7fec44158890 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fec3400c540 tx=0x7fec3400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:32.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec39ffb700 1 -- 172.21.15.97:0/3711624842 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34004c80 con 0x7fec440b84f0 2023-07-06T03:49:32.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec39ffb700 1 -- 172.21.15.97:0/3711624842 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec34004de0 con 0x7fec440b84f0 2023-07-06T03:49:32.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4b3ff700 1 -- 172.21.15.97:0/3711624842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec44157db0 con 0x7fec440b84f0 2023-07-06T03:49:32.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec39ffb700 1 -- 172.21.15.97:0/3711624842 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34007310 con 0x7fec440b84f0 2023-07-06T03:49:32.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.132+0000 7fec4b3ff700 1 -- 172.21.15.97:0/3711624842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec44158370 con 0x7fec440b84f0 2023-07-06T03:49:32.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.133+0000 7fec4b3ff700 1 -- 172.21.15.97:0/3711624842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4404e340 con 0x7fec440b84f0 2023-07-06T03:49:32.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.134+0000 7fec39ffb700 1 -- 172.21.15.97:0/3711624842 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec34007470 con 0x7fec440b84f0 2023-07-06T03:49:32.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.135+0000 7fec39ffb700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec2806ace0 0x7fec2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:32.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.135+0000 7fec39ffb700 1 -- 172.21.15.97:0/3711624842 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fec34088f10 con 0x7fec440b84f0 2023-07-06T03:49:32.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.135+0000 7fec4919b700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec2806ace0 0x7fec2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:32.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.136+0000 7fec4919b700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec2806ace0 0x7fec2806d190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fec400090b0 tx=0x7fec40008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:32.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.139+0000 7fec39ffb700 1 -- 172.21.15.97:0/3711624842 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fec34054300 con 0x7fec440b84f0 2023-07-06T03:49:32.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.333+0000 7fec4b3ff700 1 -- 172.21.15.97:0/3711624842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fec4406b410 con 0x7fec440b84f0 2023-07-06T03:49:32.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.809+0000 7fec39ffb700 1 -- 172.21.15.97:0/3711624842 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7fec34057920 con 0x7fec440b84f0 2023-07-06T03:49:32.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.813+0000 7fec4b3ff700 1 -- 172.21.15.97:0/3711624842 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec2806ace0 msgr2=0x7fec2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:32.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.813+0000 7fec4b3ff700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec2806ace0 0x7fec2806d190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fec400090b0 tx=0x7fec40008030 comp rx=0 tx=0).stop 2023-07-06T03:49:32.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.813+0000 7fec4b3ff700 1 -- 172.21.15.97:0/3711624842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec440b84f0 msgr2=0x7fec44158890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:32.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.813+0000 7fec4b3ff700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec440b84f0 0x7fec44158890 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fec3400c540 tx=0x7fec3400c900 comp rx=0 tx=0).stop 2023-07-06T03:49:32.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.813+0000 7fec4b3ff700 1 -- 172.21.15.97:0/3711624842 shutdown_connections 2023-07-06T03:49:32.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.813+0000 7fec4b3ff700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec2806ace0 0x7fec2806d190 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:32.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.813+0000 7fec4b3ff700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec440d0940 0x7fec44157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:32.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.813+0000 7fec4b3ff700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec440b5ca0 0x7fec4409e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:32.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.813+0000 7fec4b3ff700 1 --2- 172.21.15.97:0/3711624842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec440b84f0 0x7fec44158890 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:32.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.813+0000 7fec4b3ff700 1 -- 172.21.15.97:0/3711624842 >> 172.21.15.97:0/3711624842 conn(0x7fec4409c040 msgr2=0x7fec4406a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:32.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.814+0000 7fec4b3ff700 1 -- 172.21.15.97:0/3711624842 shutdown_connections 2023-07-06T03:49:32.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:32.814+0000 7fec4b3ff700 1 -- 172.21.15.97:0/3711624842 wait complete. 2023-07-06T03:49:32.817 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-07-06T03:49:32.834 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T03:49:33.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.034+0000 7f92bffff700 1 Processor -- start 2023-07-06T03:49:33.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.034+0000 7f92bffff700 1 -- start start 2023-07-06T03:49:33.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.035+0000 7f92bffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f92b80b8590 0x7f92b80cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:33.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.035+0000 7f92bffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92b8160a50 0x7f92b80b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:33.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.035+0000 7f92bffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92b80b4cd0 0x7f92b806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:33.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.035+0000 7f92bffff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b8160ef0 con 0x7f92b8160a50 2023-07-06T03:49:33.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.035+0000 7f92bffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b80b5150 con 0x7f92b80b8590 2023-07-06T03:49:33.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.035+0000 7f92bffff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b80d0c10 con 0x7f92b80b4cd0 2023-07-06T03:49:33.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bd59a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92b8160a50 0x7f92b80b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:33.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bd59a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92b8160a50 0x7f92b80b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46048/0 (socket says 172.21.15.97:46048) 2023-07-06T03:49:33.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bd59a700 1 -- 172.21.15.97:0/3961977878 learned_addr learned my addr 172.21.15.97:0/3961977878 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:33.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92be59c700 1 --2- 172.21.15.97:0/3961977878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92b80b4cd0 0x7f92b806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:33.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bdd9b700 1 --2- 172.21.15.97:0/3961977878 >> v2:172.21.15.146:3300/0 conn(0x7f92b80b8590 0x7f92b80cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:33.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bd59a700 1 -- 172.21.15.97:0/3961977878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92b80b4cd0 msgr2=0x7f92b806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:33.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bd59a700 1 --2- 172.21.15.97:0/3961977878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92b80b4cd0 0x7f92b806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bd59a700 1 -- 172.21.15.97:0/3961977878 >> v2:172.21.15.146:3300/0 conn(0x7f92b80b8590 msgr2=0x7f92b80cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:33.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bd59a700 1 --2- 172.21.15.97:0/3961977878 >> v2:172.21.15.146:3300/0 conn(0x7f92b80b8590 0x7f92b80cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bd59a700 1 -- 172.21.15.97:0/3961977878 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b80d08e0 con 0x7f92b8160a50 2023-07-06T03:49:33.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bdd9b700 1 --2- 172.21.15.97:0/3961977878 >> v2:172.21.15.146:3300/0 conn(0x7f92b80b8590 0x7f92b80cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:33.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bd59a700 1 --2- 172.21.15.97:0/3961977878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8160a50 0x7f92b80b4790 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f92b806a1e0 tx=0x7f92a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82baa2ef4b55b7b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:33.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bcd99700 1 -- 172.21.15.97:0/3961977878 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a800b870 con 0x7f92b8160a50 2023-07-06T03:49:33.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bcd99700 1 -- 172.21.15.97:0/3961977878 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a800b9d0 con 0x7f92b8160a50 2023-07-06T03:49:33.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.036+0000 7f92bcd99700 1 -- 172.21.15.97:0/3961977878 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a800bd60 con 0x7f92b8160a50 2023-07-06T03:49:33.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.037+0000 7f92bffff700 1 -- 172.21.15.97:0/3961977878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8160a50 msgr2=0x7f92b80b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:33.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.037+0000 7f92bffff700 1 --2- 172.21.15.97:0/3961977878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8160a50 0x7f92b80b4790 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f92b806a1e0 tx=0x7f92a8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.037+0000 7f92bffff700 1 -- 172.21.15.97:0/3961977878 shutdown_connections 2023-07-06T03:49:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.037+0000 7f92bffff700 1 --2- 172.21.15.97:0/3961977878 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92b80b4cd0 0x7f92b806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.037+0000 7f92bffff700 1 --2- 172.21.15.97:0/3961977878 >> v2:172.21.15.146:3300/0 conn(0x7f92b80b8590 0x7f92b80cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.037+0000 7f92bffff700 1 --2- 172.21.15.97:0/3961977878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8160a50 0x7f92b80b4790 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.037+0000 7f92bffff700 1 -- 172.21.15.97:0/3961977878 >> 172.21.15.97:0/3961977878 conn(0x7f92b806b290 msgr2=0x7f92b80cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.037+0000 7f92bffff700 1 -- 172.21.15.97:0/3961977878 shutdown_connections 2023-07-06T03:49:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.037+0000 7f92bffff700 1 -- 172.21.15.97:0/3961977878 wait complete. 2023-07-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.039+0000 7f92bffff700 1 Processor -- start 2023-07-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.039+0000 7f92bffff700 1 -- start start 2023-07-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.040+0000 7f92bffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b80b4cd0 0x7f92b815e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.040+0000 7f92bffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92b80b8590 0x7f92b8153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.040+0000 7f92bffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92b8160a50 0x7f92b806c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.040+0000 7f92bffff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b80ca710 con 0x7f92b80b4cd0 2023-07-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.040+0000 7f92bffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8153bb0 con 0x7f92b80b8590 2023-07-06T03:49:33.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.040+0000 7f92bffff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b815ef90 con 0x7f92b8160a50 2023-07-06T03:49:33.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.040+0000 7f92bdd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b80b4cd0 0x7f92b815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:33.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.040+0000 7f92bdd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b80b4cd0 0x7f92b815e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46050/0 (socket says 172.21.15.97:46050) 2023-07-06T03:49:33.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.040+0000 7f92bdd9b700 1 -- 172.21.15.97:0/2079881258 learned_addr learned my addr 172.21.15.97:0/2079881258 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:33.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92be59c700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92b8160a50 0x7f92b806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:33.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92bd59a700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92b80b8590 0x7f92b8153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:33.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92bdd9b700 1 -- 172.21.15.97:0/2079881258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92b8160a50 msgr2=0x7f92b806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:33.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92bdd9b700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92b8160a50 0x7f92b806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92bdd9b700 1 -- 172.21.15.97:0/2079881258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92b80b8590 msgr2=0x7f92b8153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:33.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92bdd9b700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92b80b8590 0x7f92b8153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92bdd9b700 1 -- 172.21.15.97:0/2079881258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a8009550 con 0x7f92b80b4cd0 2023-07-06T03:49:33.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92bd59a700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92b80b8590 0x7f92b8153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:33.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92bdd9b700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b80b4cd0 0x7f92b815e900 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f92b400c7f0 tx=0x7f92b400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:33.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92ae7fc700 1 -- 172.21.15.97:0/2079881258 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4011800 con 0x7f92b80b4cd0 2023-07-06T03:49:33.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92ae7fc700 1 -- 172.21.15.97:0/2079881258 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b4011960 con 0x7f92b80b4cd0 2023-07-06T03:49:33.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92bffff700 1 -- 172.21.15.97:0/2079881258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b80c9de0 con 0x7f92b80b4cd0 2023-07-06T03:49:33.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92ae7fc700 1 -- 172.21.15.97:0/2079881258 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4011cf0 con 0x7f92b80b4cd0 2023-07-06T03:49:33.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.041+0000 7f92bffff700 1 -- 172.21.15.97:0/2079881258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b80ca3a0 con 0x7f92b80b4cd0 2023-07-06T03:49:33.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.042+0000 7f92bffff700 1 -- 172.21.15.97:0/2079881258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b804e340 con 0x7f92b80b4cd0 2023-07-06T03:49:33.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.045+0000 7f92ae7fc700 1 -- 172.21.15.97:0/2079881258 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f92b4024480 con 0x7f92b80b4cd0 2023-07-06T03:49:33.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.045+0000 7f92ae7fc700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f929c06ace0 0x7f929c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:33.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.046+0000 7f92ae7fc700 1 -- 172.21.15.97:0/2079881258 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f92b40897a0 con 0x7f92b80b4cd0 2023-07-06T03:49:33.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.046+0000 7f92bd59a700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f929c06ace0 0x7f929c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:33.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.046+0000 7f92bd59a700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f929c06ace0 0x7f929c06d190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f92a8014040 tx=0x7f92a8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:33.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.049+0000 7f92ae7fc700 1 -- 172.21.15.97:0/2079881258 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f92b4054990 con 0x7f92b80b4cd0 2023-07-06T03:49:33.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.242+0000 7f92bffff700 1 -- 172.21.15.97:0/2079881258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f92b80d00c0 con 0x7f92b80b4cd0 2023-07-06T03:49:33.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.815+0000 7f92ae7fc700 1 -- 172.21.15.97:0/2079881258 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) v1 ==== 170+0+0 (secure 0 0 0) 0x7f92b4057fb0 con 0x7f92b80b4cd0 2023-07-06T03:49:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.821+0000 7f92bffff700 1 -- 172.21.15.97:0/2079881258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f929c06ace0 msgr2=0x7f929c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.821+0000 7f92bffff700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f929c06ace0 0x7f929c06d190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f92a8014040 tx=0x7f92a8008030 comp rx=0 tx=0).stop 2023-07-06T03:49:33.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.822+0000 7f92bffff700 1 -- 172.21.15.97:0/2079881258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b80b4cd0 msgr2=0x7f92b815e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:33.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.822+0000 7f92bffff700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b80b4cd0 0x7f92b815e900 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f92b400c7f0 tx=0x7f92b400cb00 comp rx=0 tx=0).stop 2023-07-06T03:49:33.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.822+0000 7f92bffff700 1 -- 172.21.15.97:0/2079881258 shutdown_connections 2023-07-06T03:49:33.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.822+0000 7f92bffff700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f929c06ace0 0x7f929c06d190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.822+0000 7f92bffff700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f92b8160a50 0x7f92b806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.822+0000 7f92bffff700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b80b4cd0 0x7f92b815e900 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.822+0000 7f92bffff700 1 --2- 172.21.15.97:0/2079881258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92b80b8590 0x7f92b8153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:33.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.822+0000 7f92bffff700 1 -- 172.21.15.97:0/2079881258 >> 172.21.15.97:0/2079881258 conn(0x7f92b806b290 msgr2=0x7f92b815c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.824+0000 7f92bffff700 1 -- 172.21.15.97:0/2079881258 shutdown_connections 2023-07-06T03:49:33.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:33.825+0000 7f92bffff700 1 -- 172.21.15.97:0/2079881258 wait complete. 2023-07-06T03:49:33.830 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-07-06T03:49:33.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T03:49:34.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.047+0000 7fec5b1ff700 1 Processor -- start 2023-07-06T03:49:34.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.048+0000 7fec5b1ff700 1 -- start start 2023-07-06T03:49:34.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.049+0000 7fec5b1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fec540d2210 0x7fec540b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:34.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.049+0000 7fec5b1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec540cff90 con 0x7fec5415c610 2023-07-06T03:49:34.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.049+0000 7fec5b1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec540b5650 con 0x7fec5406b360 2023-07-06T03:49:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.049+0000 7fec5b1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec540d0780 con 0x7fec540d2210 2023-07-06T03:49:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.049+0000 7fec58f9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fec5415c610 0x7fec540b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35548/0 (socket says 172.21.15.97:35548) 2023-07-06T03:49:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.049+0000 7fec58f9b700 1 -- 172.21.15.97:0/3068366896 learned_addr learned my addr 172.21.15.97:0/3068366896 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.049+0000 7fec5979c700 1 --2- 172.21.15.97:0/3068366896 >> v2:172.21.15.146:3300/0 conn(0x7fec540d2210 0x7fec540b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.049+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822526075 0 0) 0x7fec540cff90 con 0x7fec5415c610 2023-07-06T03:49:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec440035b0 con 0x7fec5415c610 2023-07-06T03:49:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 496252846 0 0) 0x7fec440035b0 con 0x7fec5415c610 2023-07-06T03:49:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 >> v2:172.21.15.146:3300/0 conn(0x7fec540d2210 msgr2=0x7fec540b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 --2- 172.21.15.97:0/3068366896 >> v2:172.21.15.146:3300/0 conn(0x7fec540d2210 0x7fec540b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 >> v1:172.21.15.146:6789/0 conn(0x7fec5406b360 legacy=0x7fec540d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec540b5400 con 0x7fec5415c610 2023-07-06T03:49:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fec48001670 con 0x7fec5415c610 2023-07-06T03:49:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec5979c700 1 --2- 172.21.15.97:0/3068366896 >> v2:172.21.15.146:3300/0 conn(0x7fec540d2210 0x7fec540b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 >> v1:172.21.15.97:6789/0 conn(0x7fec5415c610 legacy=0x7fec540b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 --2- 172.21.15.97:0/3068366896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec44003a00 0x7fec44003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 --2- 172.21.15.97:0/3068366896 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec44004850 0x7fec44004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 --2- 172.21.15.97:0/3068366896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec44005200 0x7fec4400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec540d0780 con 0x7fec44003a00 2023-07-06T03:49:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4400c080 con 0x7fec44005200 2023-07-06T03:49:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4400c240 con 0x7fec44004850 2023-07-06T03:49:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec58f9b700 1 --2- 172.21.15.97:0/3068366896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec44003a00 0x7fec44003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec53fff700 1 --2- 172.21.15.97:0/3068366896 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec44004850 0x7fec44004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:34.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.050+0000 7fec5979c700 1 --2- 172.21.15.97:0/3068366896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec44005200 0x7fec4400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.051+0000 7fec58f9b700 1 -- 172.21.15.97:0/3068366896 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec44004850 msgr2=0x7fec44004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.051+0000 7fec58f9b700 1 --2- 172.21.15.97:0/3068366896 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec44004850 0x7fec44004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.051+0000 7fec58f9b700 1 -- 172.21.15.97:0/3068366896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec44005200 msgr2=0x7fec4400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.051+0000 7fec58f9b700 1 --2- 172.21.15.97:0/3068366896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec44005200 0x7fec4400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.051+0000 7fec58f9b700 1 -- 172.21.15.97:0/3068366896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440035b0 con 0x7fec44003a00 2023-07-06T03:49:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.051+0000 7fec5979c700 1 -- 172.21.15.97:0/3068366896 reap_dead start 2023-07-06T03:49:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.051+0000 7fec58f9b700 1 --2- 172.21.15.97:0/3068366896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec44003a00 0x7fec44003e10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fec4800c240 tx=0x7fec44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ad3e6fb307c8c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:34.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.051+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48007c10 con 0x7fec44003a00 2023-07-06T03:49:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.051+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4800cd80 con 0x7fec44003a00 2023-07-06T03:49:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.051+0000 7fec537fe700 1 -- 172.21.15.97:0/3068366896 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48005ba0 con 0x7fec44003a00 2023-07-06T03:49:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.052+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3068366896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec44003a00 msgr2=0x7fec44003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.052+0000 7fec5b1ff700 1 --2- 172.21.15.97:0/3068366896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec44003a00 0x7fec44003e10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fec4800c240 tx=0x7fec44005880 comp rx=0 tx=0).stop 2023-07-06T03:49:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.052+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3068366896 shutdown_connections 2023-07-06T03:49:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.052+0000 7fec5b1ff700 1 --2- 172.21.15.97:0/3068366896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec44003a00 0x7fec44003e10 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.052+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3068366896 >> 172.21.15.97:0/3068366896 conn(0x7fec5406a6c0 msgr2=0x7fec5406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:34.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.052+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3068366896 shutdown_connections 2023-07-06T03:49:34.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.052+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3068366896 wait complete. 2023-07-06T03:49:34.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.054+0000 7fec5b1ff700 1 Processor -- start 2023-07-06T03:49:34.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.054+0000 7fec5b1ff700 1 -- start start 2023-07-06T03:49:34.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec5b1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec44003a00 0x7fec541534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:34.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec5b1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec5415cb60 0x7fec540ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:34.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec5b1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5415cfd0 0x7fec540cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:34.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec5b1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec540cb100 con 0x7fec5415cb60 2023-07-06T03:49:34.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec5b1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5415d540 con 0x7fec5415cfd0 2023-07-06T03:49:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec5b1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5415d820 con 0x7fec44003a00 2023-07-06T03:49:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec53fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec5415cb60 0x7fec540ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec53fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec5415cb60 0x7fec540ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46064/0 (socket says 172.21.15.97:46064) 2023-07-06T03:49:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec53fff700 1 -- 172.21.15.97:0/3764529107 learned_addr learned my addr 172.21.15.97:0/3764529107 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec58f9b700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec44003a00 0x7fec541534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec5979c700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5415cfd0 0x7fec540cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:34.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec53fff700 1 -- 172.21.15.97:0/3764529107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec44003a00 msgr2=0x7fec541534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:34.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec53fff700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec44003a00 0x7fec541534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:34.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec53fff700 1 -- 172.21.15.97:0/3764529107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5415cfd0 msgr2=0x7fec540cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:34.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.055+0000 7fec53fff700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5415cfd0 0x7fec540cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:34.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.056+0000 7fec53fff700 1 -- 172.21.15.97:0/3764529107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5415db00 con 0x7fec5415cb60 2023-07-06T03:49:34.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.056+0000 7fec53fff700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec5415cb60 0x7fec540ca680 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fec4000f820 tx=0x7fec4000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:34.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.056+0000 7fec517fa700 1 -- 172.21.15.97:0/3764529107 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec400052e0 con 0x7fec5415cb60 2023-07-06T03:49:34.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.056+0000 7fec517fa700 1 -- 172.21.15.97:0/3764529107 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec40005460 con 0x7fec5415cb60 2023-07-06T03:49:34.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.056+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3764529107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5415dde0 con 0x7fec5415cb60 2023-07-06T03:49:34.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.056+0000 7fec517fa700 1 -- 172.21.15.97:0/3764529107 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec400038c0 con 0x7fec5415cb60 2023-07-06T03:49:34.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.056+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3764529107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec540cc720 con 0x7fec5415cb60 2023-07-06T03:49:34.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.057+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3764529107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5415e430 con 0x7fec5415cb60 2023-07-06T03:49:34.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.059+0000 7fec517fa700 1 -- 172.21.15.97:0/3764529107 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec40003a20 con 0x7fec5415cb60 2023-07-06T03:49:34.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.060+0000 7fec517fa700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec3806ace0 0x7fec3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:34.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.060+0000 7fec517fa700 1 -- 172.21.15.97:0/3764529107 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fec40089c20 con 0x7fec5415cb60 2023-07-06T03:49:34.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.060+0000 7fec58f9b700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec3806ace0 0x7fec3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:34.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.061+0000 7fec58f9b700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec3806ace0 0x7fec3806d190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fec48000c00 tx=0x7fec48007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:34.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.063+0000 7fec517fa700 1 -- 172.21.15.97:0/3764529107 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fec40089e60 con 0x7fec5415cb60 2023-07-06T03:49:34.189 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:49:34.193 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:49:34.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.308+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3764529107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fec54153a10 con 0x7fec5415cb60 2023-07-06T03:49:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.842+0000 7fec517fa700 1 -- 172.21.15.97:0/3764529107 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7fec40054da0 con 0x7fec5415cb60 2023-07-06T03:49:34.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.855+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3764529107 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec3806ace0 msgr2=0x7fec3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:34.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.855+0000 7fec5b1ff700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec3806ace0 0x7fec3806d190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fec48000c00 tx=0x7fec48007000 comp rx=0 tx=0).stop 2023-07-06T03:49:34.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.855+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3764529107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec5415cb60 msgr2=0x7fec540ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:34.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.855+0000 7fec5b1ff700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec5415cb60 0x7fec540ca680 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fec4000f820 tx=0x7fec4000fbe0 comp rx=0 tx=0).stop 2023-07-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.863+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3764529107 shutdown_connections 2023-07-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.863+0000 7fec5b1ff700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec3806ace0 0x7fec3806d190 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.863+0000 7fec5b1ff700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec5415cfd0 0x7fec540cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.863+0000 7fec5b1ff700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec44003a00 0x7fec541534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.863+0000 7fec5b1ff700 1 --2- 172.21.15.97:0/3764529107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec5415cb60 0x7fec540ca680 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.863+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3764529107 >> 172.21.15.97:0/3764529107 conn(0x7fec5406a6c0 msgr2=0x7fec5415b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:34.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.866+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3764529107 shutdown_connections 2023-07-06T03:49:34.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:34.867+0000 7fec5b1ff700 1 -- 172.21.15.97:0/3764529107 wait complete. 2023-07-06T03:49:34.874 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 11 and data pool 12 2023-07-06T03:49:34.893 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:49:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.087+0000 7f6c64bff700 1 Processor -- start 2023-07-06T03:49:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.087+0000 7f6c64bff700 1 -- start start 2023-07-06T03:49:35.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.088+0000 7f6c64bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c6015c520 0x7f6c600cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.088+0000 7f6c64bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6c60160180 0x7f6c601605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.088+0000 7f6c64bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c60160b30 0x7f6c600cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.088+0000 7f6c64bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c600cd8d0 con 0x7f6c60160b30 2023-07-06T03:49:35.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.088+0000 7f6c64bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c600cdae0 con 0x7f6c60160180 2023-07-06T03:49:35.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.088+0000 7f6c64bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600b6640 con 0x7f6c6015c520 2023-07-06T03:49:35.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.088+0000 7f6c5ed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c60160b30 0x7f6c600cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.088+0000 7f6c5ed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c60160b30 0x7f6c600cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46070/0 (socket says 172.21.15.97:46070) 2023-07-06T03:49:35.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.088+0000 7f6c5ed9d700 1 -- 172.21.15.97:0/3128559336 learned_addr learned my addr 172.21.15.97:0/3128559336 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:35.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.089+0000 7f6c57fff700 1 --2- 172.21.15.97:0/3128559336 >> v2:172.21.15.146:3300/0 conn(0x7f6c60160180 0x7f6c601605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.089+0000 7f6c5e59c700 1 --2- 172.21.15.97:0/3128559336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c6015c520 0x7f6c600cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.089+0000 7f6c5ed9d700 1 -- 172.21.15.97:0/3128559336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c6015c520 msgr2=0x7f6c600cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:35.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.089+0000 7f6c5ed9d700 1 --2- 172.21.15.97:0/3128559336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c6015c520 0x7f6c600cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.089+0000 7f6c5ed9d700 1 -- 172.21.15.97:0/3128559336 >> v2:172.21.15.146:3300/0 conn(0x7f6c60160180 msgr2=0x7f6c601605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:35.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.089+0000 7f6c5ed9d700 1 --2- 172.21.15.97:0/3128559336 >> v2:172.21.15.146:3300/0 conn(0x7f6c60160180 0x7f6c601605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.089+0000 7f6c5ed9d700 1 -- 172.21.15.97:0/3128559336 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c600b6920 con 0x7f6c60160b30 2023-07-06T03:49:35.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.089+0000 7f6c5ed9d700 1 --2- 172.21.15.97:0/3128559336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c60160b30 0x7f6c600cf050 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6c60072770 tx=0x7f6c5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd22dd452351a7da server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:35.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.089+0000 7f6c5dd9b700 1 -- 172.21.15.97:0/3128559336 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800e070 con 0x7f6c60160b30 2023-07-06T03:49:35.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.089+0000 7f6c5dd9b700 1 -- 172.21.15.97:0/3128559336 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5800b910 con 0x7f6c60160b30 2023-07-06T03:49:35.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.089+0000 7f6c5dd9b700 1 -- 172.21.15.97:0/3128559336 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800bcc0 con 0x7f6c60160b30 2023-07-06T03:49:35.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.090+0000 7f6c64bff700 1 -- 172.21.15.97:0/3128559336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c60160b30 msgr2=0x7f6c600cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.090+0000 7f6c64bff700 1 --2- 172.21.15.97:0/3128559336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c60160b30 0x7f6c600cf050 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6c60072770 tx=0x7f6c5800b330 comp rx=0 tx=0).stop 2023-07-06T03:49:35.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.090+0000 7f6c64bff700 1 -- 172.21.15.97:0/3128559336 shutdown_connections 2023-07-06T03:49:35.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.090+0000 7f6c64bff700 1 --2- 172.21.15.97:0/3128559336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c60160b30 0x7f6c600cf050 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.090+0000 7f6c64bff700 1 --2- 172.21.15.97:0/3128559336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c6015c520 0x7f6c600cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.090+0000 7f6c64bff700 1 --2- 172.21.15.97:0/3128559336 >> v2:172.21.15.146:3300/0 conn(0x7f6c60160180 0x7f6c601605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.090+0000 7f6c64bff700 1 -- 172.21.15.97:0/3128559336 >> 172.21.15.97:0/3128559336 conn(0x7f6c60071790 msgr2=0x7f6c60071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:35.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.090+0000 7f6c64bff700 1 -- 172.21.15.97:0/3128559336 shutdown_connections 2023-07-06T03:49:35.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.090+0000 7f6c64bff700 1 -- 172.21.15.97:0/3128559336 wait complete. 2023-07-06T03:49:35.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.092+0000 7f6c64bff700 1 Processor -- start 2023-07-06T03:49:35.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.092+0000 7f6c64bff700 1 -- start start 2023-07-06T03:49:35.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.093+0000 7f6c64bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c6015c520 0x7f6c600caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.093+0000 7f6c64bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c60160180 0x7f6c600d2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.093+0000 7f6c64bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c60160b30 0x7f6c601f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.093+0000 7f6c64bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600d2ac0 con 0x7f6c60160180 2023-07-06T03:49:35.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.093+0000 7f6c64bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600d2da0 con 0x7f6c60160b30 2023-07-06T03:49:35.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.093+0000 7f6c64bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600d3080 con 0x7f6c6015c520 2023-07-06T03:49:35.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.093+0000 7f6c57fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c60160180 0x7f6c600d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.093+0000 7f6c57fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c60160180 0x7f6c600d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46078/0 (socket says 172.21.15.97:46078) 2023-07-06T03:49:35.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.093+0000 7f6c57fff700 1 -- 172.21.15.97:0/1134957985 learned_addr learned my addr 172.21.15.97:0/1134957985 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:35.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c5ed9d700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c60160b30 0x7f6c601f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c5e59c700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c6015c520 0x7f6c600caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c57fff700 1 -- 172.21.15.97:0/1134957985 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c6015c520 msgr2=0x7f6c600caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c57fff700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c6015c520 0x7f6c600caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c57fff700 1 -- 172.21.15.97:0/1134957985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c60160b30 msgr2=0x7f6c601f1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c57fff700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c60160b30 0x7f6c601f1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c57fff700 1 -- 172.21.15.97:0/1134957985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c58009b30 con 0x7f6c60160180 2023-07-06T03:49:35.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c5e59c700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c6015c520 0x7f6c600caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:35.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c57fff700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c60160180 0x7f6c600d2020 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e9b0 tx=0x7f6c4800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:35.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c56ffd700 1 -- 172.21.15.97:0/1134957985 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48009710 con 0x7f6c60160180 2023-07-06T03:49:35.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c56ffd700 1 -- 172.21.15.97:0/1134957985 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4800c850 con 0x7f6c60160180 2023-07-06T03:49:35.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c64bff700 1 -- 172.21.15.97:0/1134957985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c600d3360 con 0x7f6c60160180 2023-07-06T03:49:35.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c56ffd700 1 -- 172.21.15.97:0/1134957985 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800cbc0 con 0x7f6c60160180 2023-07-06T03:49:35.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.094+0000 7f6c64bff700 1 -- 172.21.15.97:0/1134957985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c600ca480 con 0x7f6c60160180 2023-07-06T03:49:35.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.095+0000 7f6c64bff700 1 -- 172.21.15.97:0/1134957985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c600c9880 con 0x7f6c60160180 2023-07-06T03:49:35.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.096+0000 7f6c56ffd700 1 -- 172.21.15.97:0/1134957985 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c4800cd20 con 0x7f6c60160180 2023-07-06T03:49:35.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.097+0000 7f6c56ffd700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6c4006ace0 0x7f6c4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.097+0000 7f6c56ffd700 1 -- 172.21.15.97:0/1134957985 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c48089580 con 0x7f6c60160180 2023-07-06T03:49:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.097+0000 7f6c5e59c700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6c4006ace0 0x7f6c4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.098+0000 7f6c5e59c700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6c4006ace0 0x7f6c4006d190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009c30 tx=0x7f6c50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:35.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.101+0000 7f6c56ffd700 1 -- 172.21.15.97:0/1134957985 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6c48054590 con 0x7f6c60160180 2023-07-06T03:49:35.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.299+0000 7f6c64bff700 1 -- 172.21.15.97:0/1134957985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c6004e340 con 0x7f6c60160180 2023-07-06T03:49:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.301+0000 7f6c56ffd700 1 -- 172.21.15.97:0/1134957985 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18333 (secure 0 0 0) 0x7f6c48057bb0 con 0x7f6c60160180 2023-07-06T03:49:35.303 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:35.303 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":73,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:49:34.309731+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:49:32.371992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-06T03:49:33.281260+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:49:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 -- 172.21.15.97:0/1134957985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6c4006ace0 msgr2=0x7f6c4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6c4006ace0 0x7f6c4006d190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009c30 tx=0x7f6c50008040 comp rx=0 tx=0).stop 2023-07-06T03:49:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 -- 172.21.15.97:0/1134957985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c60160180 msgr2=0x7f6c600d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c60160180 0x7f6c600d2020 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e9b0 tx=0x7f6c4800ecc0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 -- 172.21.15.97:0/1134957985 shutdown_connections 2023-07-06T03:49:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6c4006ace0 0x7f6c4006d190 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c60160b30 0x7f6c601f1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6c6015c520 0x7f6c600caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 --2- 172.21.15.97:0/1134957985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c60160180 0x7f6c600d2020 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 -- 172.21.15.97:0/1134957985 >> 172.21.15.97:0/1134957985 conn(0x7f6c60071790 msgr2=0x7f6c600b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 -- 172.21.15.97:0/1134957985 shutdown_connections 2023-07-06T03:49:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.303+0000 7f6c64bff700 1 -- 172.21.15.97:0/1134957985 wait complete. 2023-07-06T03:49:35.323 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:49:35.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.515+0000 7fed82dff700 1 Processor -- start 2023-07-06T03:49:35.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.515+0000 7fed82dff700 1 -- start start 2023-07-06T03:49:35.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.516+0000 7fed82dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7c15b440 0x7fed7c15e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.517+0000 7fed82dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c071070 con 0x7fed7c0cd8e0 2023-07-06T03:49:35.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.517+0000 7fed82dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c071330 con 0x7fed7c070c30 2023-07-06T03:49:35.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.517+0000 7fed82dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c071610 con 0x7fed7c15b440 2023-07-06T03:49:35.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.517+0000 7fed80b9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fed7c0cd8e0 0x7fed7c0cdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35564/0 (socket says 172.21.15.97:35564) 2023-07-06T03:49:35.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.517+0000 7fed80b9b700 1 -- 172.21.15.97:0/4243296326 learned_addr learned my addr 172.21.15.97:0/4243296326 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:35.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.517+0000 7fed7bfff700 1 --2- 172.21.15.97:0/4243296326 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7c15b440 0x7fed7c15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.517+0000 7fed7b7fe700 1 -- 172.21.15.97:0/4243296326 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613458801 0 0) 0x7fed7c071070 con 0x7fed7c0cd8e0 2023-07-06T03:49:35.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.517+0000 7fed7b7fe700 1 -- 172.21.15.97:0/4243296326 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed6c0035b0 con 0x7fed7c0cd8e0 2023-07-06T03:49:35.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.517+0000 7fed7b7fe700 1 -- 172.21.15.97:0/4243296326 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336863914 0 0) 0x7fed7c071330 con 0x7fed7c070c30 2023-07-06T03:49:35.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.517+0000 7fed7b7fe700 1 -- 172.21.15.97:0/4243296326 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed7c071070 con 0x7fed7c070c30 2023-07-06T03:49:35.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.517+0000 7fed7bfff700 1 -- 172.21.15.97:0/4243296326 >> v1:172.21.15.146:6789/0 conn(0x7fed7c070c30 legacy=0x7fed7c06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.518+0000 7fed7bfff700 1 -- 172.21.15.97:0/4243296326 >> v1:172.21.15.97:6789/0 conn(0x7fed7c0cd8e0 legacy=0x7fed7c0cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.518+0000 7fed7bfff700 1 -- 172.21.15.97:0/4243296326 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c15dd80 con 0x7fed7c15b440 2023-07-06T03:49:35.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.518+0000 7fed7bfff700 1 --2- 172.21.15.97:0/4243296326 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7c15b440 0x7fed7c15e860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fed68009870 tx=0x7fed68009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=479536090bc989a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:35.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.518+0000 7fed7b7fe700 1 -- 172.21.15.97:0/4243296326 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68011830 con 0x7fed7c15b440 2023-07-06T03:49:35.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.518+0000 7fed7b7fe700 1 -- 172.21.15.97:0/4243296326 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed68011990 con 0x7fed7c15b440 2023-07-06T03:49:35.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.518+0000 7fed7b7fe700 1 -- 172.21.15.97:0/4243296326 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68011d20 con 0x7fed7c15b440 2023-07-06T03:49:35.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.519+0000 7fed82dff700 1 -- 172.21.15.97:0/4243296326 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7c15b440 msgr2=0x7fed7c15e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.519+0000 7fed82dff700 1 --2- 172.21.15.97:0/4243296326 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7c15b440 0x7fed7c15e860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fed68009870 tx=0x7fed68009b80 comp rx=0 tx=0).stop 2023-07-06T03:49:35.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.519+0000 7fed82dff700 1 -- 172.21.15.97:0/4243296326 shutdown_connections 2023-07-06T03:49:35.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.519+0000 7fed82dff700 1 --2- 172.21.15.97:0/4243296326 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7c15b440 0x7fed7c15e860 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.519+0000 7fed82dff700 1 -- 172.21.15.97:0/4243296326 >> 172.21.15.97:0/4243296326 conn(0x7fed7c0ce8c0 msgr2=0x7fed7c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:35.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.519+0000 7fed82dff700 1 -- 172.21.15.97:0/4243296326 shutdown_connections 2023-07-06T03:49:35.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.519+0000 7fed82dff700 1 -- 172.21.15.97:0/4243296326 wait complete. 2023-07-06T03:49:35.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.520+0000 7fed82dff700 1 Processor -- start 2023-07-06T03:49:35.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.521+0000 7fed82dff700 1 -- start start 2023-07-06T03:49:35.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed82dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed7c070c30 0x7fed7c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed82dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7c0cd8e0 0x7fed7c149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed82dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c15b440 0x7fed7c0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed82dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c06c390 con 0x7fed7c15b440 2023-07-06T03:49:35.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed82dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c06cc30 con 0x7fed7c070c30 2023-07-06T03:49:35.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed82dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c06d600 con 0x7fed7c0cd8e0 2023-07-06T03:49:35.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed8139c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c15b440 0x7fed7c0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed8139c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c15b440 0x7fed7c0ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46082/0 (socket says 172.21.15.97:46082) 2023-07-06T03:49:35.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed8139c700 1 -- 172.21.15.97:0/664463194 learned_addr learned my addr 172.21.15.97:0/664463194 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:35.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed7bfff700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7c0cd8e0 0x7fed7c149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed80b9b700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed7c070c30 0x7fed7c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed8139c700 1 -- 172.21.15.97:0/664463194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7c0cd8e0 msgr2=0x7fed7c149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed8139c700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7c0cd8e0 0x7fed7c149260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed8139c700 1 -- 172.21.15.97:0/664463194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed7c070c30 msgr2=0x7fed7c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed8139c700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed7c070c30 0x7fed7c06d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.522+0000 7fed8139c700 1 -- 172.21.15.97:0/664463194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed680094a0 con 0x7fed7c15b440 2023-07-06T03:49:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.523+0000 7fed8139c700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c15b440 0x7fed7c0ac870 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fed7400cc50 tx=0x7fed7400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.523+0000 7fed797fa700 1 -- 172.21.15.97:0/664463194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74003c00 con 0x7fed7c15b440 2023-07-06T03:49:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.523+0000 7fed82dff700 1 -- 172.21.15.97:0/664463194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c0aefc0 con 0x7fed7c15b440 2023-07-06T03:49:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.523+0000 7fed797fa700 1 -- 172.21.15.97:0/664463194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed74003d60 con 0x7fed7c15b440 2023-07-06T03:49:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.523+0000 7fed797fa700 1 -- 172.21.15.97:0/664463194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7400aad0 con 0x7fed7c15b440 2023-07-06T03:49:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.523+0000 7fed82dff700 1 -- 172.21.15.97:0/664463194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c06c8c0 con 0x7fed7c15b440 2023-07-06T03:49:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.524+0000 7fed82dff700 1 -- 172.21.15.97:0/664463194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c070aa0 con 0x7fed7c15b440 2023-07-06T03:49:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.525+0000 7fed797fa700 1 -- 172.21.15.97:0/664463194 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed7400ac30 con 0x7fed7c15b440 2023-07-06T03:49:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.526+0000 7fed797fa700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed6006ace0 0x7fed6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.526+0000 7fed797fa700 1 -- 172.21.15.97:0/664463194 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fed74089690 con 0x7fed7c15b440 2023-07-06T03:49:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.526+0000 7fed80b9b700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed6006ace0 0x7fed6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.527+0000 7fed80b9b700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed6006ace0 0x7fed6006d190 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fed70007fd0 tx=0x7fed70007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.530+0000 7fed797fa700 1 -- 172.21.15.97:0/664463194 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fed74054590 con 0x7fed7c15b440 2023-07-06T03:49:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.724+0000 7fed82dff700 1 -- 172.21.15.97:0/664463194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed7c04e340 con 0x7fed7c15b440 2023-07-06T03:49:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.725+0000 7fed797fa700 1 -- 172.21.15.97:0/664463194 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18333 (secure 0 0 0) 0x7fed74057bb0 con 0x7fed7c15b440 2023-07-06T03:49:35.728 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:35.728 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":73,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:49:34.309731+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:49:32.371992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-06T03:49:33.281260+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:49:35.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 -- 172.21.15.97:0/664463194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed6006ace0 msgr2=0x7fed6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed6006ace0 0x7fed6006d190 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fed70007fd0 tx=0x7fed70007dc0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 -- 172.21.15.97:0/664463194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c15b440 msgr2=0x7fed7c0ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c15b440 0x7fed7c0ac870 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fed7400cc50 tx=0x7fed7400cf60 comp rx=0 tx=0).stop 2023-07-06T03:49:35.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 -- 172.21.15.97:0/664463194 shutdown_connections 2023-07-06T03:49:35.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed6006ace0 0x7fed6006d190 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7c15b440 0x7fed7c0ac870 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed7c070c30 0x7fed7c06d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 --2- 172.21.15.97:0/664463194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed7c0cd8e0 0x7fed7c149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 -- 172.21.15.97:0/664463194 >> 172.21.15.97:0/664463194 conn(0x7fed7c0ce8c0 msgr2=0x7fed7c15b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:35.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 -- 172.21.15.97:0/664463194 shutdown_connections 2023-07-06T03:49:35.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.728+0000 7fed82dff700 1 -- 172.21.15.97:0/664463194 wait complete. 2023-07-06T03:49:35.748 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T03:49:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.950+0000 7fbffc3ff700 1 Processor -- start 2023-07-06T03:49:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.950+0000 7fbffc3ff700 1 -- start start 2023-07-06T03:49:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.951+0000 7fbffc3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbff4159070 0x7fbff40c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.951+0000 7fbffc3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbff40c3cd0 0x7fbff40cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.951+0000 7fbffc3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff40ca260 con 0x7fbff4159440 2023-07-06T03:49:35.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.951+0000 7fbffc3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff40c99c0 con 0x7fbff4159070 2023-07-06T03:49:35.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.951+0000 7fbffc3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff40c9ca0 con 0x7fbff40c3cd0 2023-07-06T03:49:35.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.951+0000 7fbffa19b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbff4159070 0x7fbff40c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.951+0000 7fbffa99c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbff4159440 0x7fbff40b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35574/0 (socket says 172.21.15.97:35574) 2023-07-06T03:49:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.951+0000 7fbffa99c700 1 -- 172.21.15.97:0/3563932356 learned_addr learned my addr 172.21.15.97:0/3563932356 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:35.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.951+0000 7fbff999a700 1 --2- 172.21.15.97:0/3563932356 >> v2:172.21.15.146:3300/0 conn(0x7fbff40c3cd0 0x7fbff40cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.952+0000 7fbff9199700 1 -- 172.21.15.97:0/3563932356 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760084723 0 0) 0x7fbff40ca260 con 0x7fbff4159440 2023-07-06T03:49:35.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.952+0000 7fbffa19b700 1 -- 172.21.15.97:0/3563932356 >> v2:172.21.15.146:3300/0 conn(0x7fbff40c3cd0 msgr2=0x7fbff40cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:35.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.952+0000 7fbffa19b700 1 --2- 172.21.15.97:0/3563932356 >> v2:172.21.15.146:3300/0 conn(0x7fbff40c3cd0 0x7fbff40cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.952+0000 7fbffa19b700 1 -- 172.21.15.97:0/3563932356 >> v1:172.21.15.97:6789/0 conn(0x7fbff4159440 legacy=0x7fbff40b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.952+0000 7fbffa19b700 1 -- 172.21.15.97:0/3563932356 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40c9f80 con 0x7fbff4159070 2023-07-06T03:49:35.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.952+0000 7fbffa19b700 1 --2- 172.21.15.97:0/3563932356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbff4159070 0x7fbff40c8f40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbff40cf9f0 tx=0x7fbff0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f0a29ca8c7c903 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:35.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.952+0000 7fbff9199700 1 -- 172.21.15.97:0/3563932356 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0011830 con 0x7fbff4159070 2023-07-06T03:49:35.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.952+0000 7fbff9199700 1 -- 172.21.15.97:0/3563932356 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0011990 con 0x7fbff4159070 2023-07-06T03:49:35.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.952+0000 7fbff9199700 1 -- 172.21.15.97:0/3563932356 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0011d20 con 0x7fbff4159070 2023-07-06T03:49:35.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.953+0000 7fbffc3ff700 1 -- 172.21.15.97:0/3563932356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbff4159070 msgr2=0x7fbff40c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.953+0000 7fbffc3ff700 1 --2- 172.21.15.97:0/3563932356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbff4159070 0x7fbff40c8f40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbff40cf9f0 tx=0x7fbff0009b20 comp rx=0 tx=0).stop 2023-07-06T03:49:35.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.953+0000 7fbffc3ff700 1 -- 172.21.15.97:0/3563932356 shutdown_connections 2023-07-06T03:49:35.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.953+0000 7fbffc3ff700 1 --2- 172.21.15.97:0/3563932356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbff4159070 0x7fbff40c8f40 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.953+0000 7fbffc3ff700 1 --2- 172.21.15.97:0/3563932356 >> v2:172.21.15.146:3300/0 conn(0x7fbff40c3cd0 0x7fbff40cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.953+0000 7fbffc3ff700 1 -- 172.21.15.97:0/3563932356 >> 172.21.15.97:0/3563932356 conn(0x7fbff406a060 msgr2=0x7fbff406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:35.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.953+0000 7fbffc3ff700 1 -- 172.21.15.97:0/3563932356 shutdown_connections 2023-07-06T03:49:35.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.953+0000 7fbffc3ff700 1 -- 172.21.15.97:0/3563932356 wait complete. 2023-07-06T03:49:35.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.955+0000 7fbffc3ff700 1 Processor -- start 2023-07-06T03:49:35.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.955+0000 7fbffc3ff700 1 -- start start 2023-07-06T03:49:35.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.956+0000 7fbffc3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff40c3cd0 0x7fbff414e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.956+0000 7fbffc3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff4159070 0x7fbff40b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.956+0000 7fbffc3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbff4159440 0x7fbff41430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.956+0000 7fbffc3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff414e480 con 0x7fbff4159440 2023-07-06T03:49:35.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.956+0000 7fbffc3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff414ee50 con 0x7fbff4159070 2023-07-06T03:49:35.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.956+0000 7fbffc3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40b45e0 con 0x7fbff40c3cd0 2023-07-06T03:49:35.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.956+0000 7fbffa99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbff4159440 0x7fbff41430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.956+0000 7fbffa99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbff4159440 0x7fbff41430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46108/0 (socket says 172.21.15.97:46108) 2023-07-06T03:49:35.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.956+0000 7fbffa99c700 1 -- 172.21.15.97:0/1198745454 learned_addr learned my addr 172.21.15.97:0/1198745454 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:35.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.956+0000 7fbffa19b700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff40c3cd0 0x7fbff414e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.956+0000 7fbff999a700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff4159070 0x7fbff40b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.957+0000 7fbffa99c700 1 -- 172.21.15.97:0/1198745454 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff40c3cd0 msgr2=0x7fbff414e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.957+0000 7fbffa99c700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff40c3cd0 0x7fbff414e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.957+0000 7fbffa99c700 1 -- 172.21.15.97:0/1198745454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff4159070 msgr2=0x7fbff40b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.957+0000 7fbffa99c700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff4159070 0x7fbff40b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.957+0000 7fbffa99c700 1 -- 172.21.15.97:0/1198745454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00094a0 con 0x7fbff4159440 2023-07-06T03:49:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.957+0000 7fbffa99c700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbff4159440 0x7fbff41430c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00cba0 tx=0x7fbfec00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.957+0000 7fbfeaffd700 1 -- 172.21.15.97:0/1198745454 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec00aaa0 con 0x7fbff4159440 2023-07-06T03:49:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.957+0000 7fbfeaffd700 1 -- 172.21.15.97:0/1198745454 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec00ac20 con 0x7fbff4159440 2023-07-06T03:49:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.957+0000 7fbffc3ff700 1 -- 172.21.15.97:0/1198745454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff414db50 con 0x7fbff4159440 2023-07-06T03:49:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.957+0000 7fbfeaffd700 1 -- 172.21.15.97:0/1198745454 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0070d0 con 0x7fbff4159440 2023-07-06T03:49:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.957+0000 7fbffc3ff700 1 -- 172.21.15.97:0/1198745454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff414e110 con 0x7fbff4159440 2023-07-06T03:49:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.958+0000 7fbffc3ff700 1 -- 172.21.15.97:0/1198745454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff404e340 con 0x7fbff4159440 2023-07-06T03:49:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.959+0000 7fbfeaffd700 1 -- 172.21.15.97:0/1198745454 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbfec007230 con 0x7fbff4159440 2023-07-06T03:49:35.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.960+0000 7fbfeaffd700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfd806ace0 0x7fbfd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:35.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.960+0000 7fbfeaffd700 1 -- 172.21.15.97:0/1198745454 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbfec088fe0 con 0x7fbff4159440 2023-07-06T03:49:35.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.960+0000 7fbffa19b700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfd806ace0 0x7fbfd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:35.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.961+0000 7fbffa19b700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfd806ace0 0x7fbfd806d190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbff00090b0 tx=0x7fbff0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:35.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:35.964+0000 7fbfeaffd700 1 -- 172.21.15.97:0/1198745454 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbfec054030 con 0x7fbff4159440 2023-07-06T03:49:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.202+0000 7fbffc3ff700 1 -- 172.21.15.97:0/1198745454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbff406d220 con 0x7fbff4159440 2023-07-06T03:49:36.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.865+0000 7fbfeaffd700 1 -- 172.21.15.97:0/1198745454 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbfec057650 con 0x7fbff4159440 2023-07-06T03:49:36.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.867+0000 7fbffc3ff700 1 -- 172.21.15.97:0/1198745454 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfd806ace0 msgr2=0x7fbfd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:36.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.867+0000 7fbffc3ff700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfd806ace0 0x7fbfd806d190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbff00090b0 tx=0x7fbff0008030 comp rx=0 tx=0).stop 2023-07-06T03:49:36.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.867+0000 7fbffc3ff700 1 -- 172.21.15.97:0/1198745454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbff4159440 msgr2=0x7fbff41430c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:36.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.867+0000 7fbffc3ff700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbff4159440 0x7fbff41430c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00cba0 tx=0x7fbfec00ceb0 comp rx=0 tx=0).stop 2023-07-06T03:49:36.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.868+0000 7fbffc3ff700 1 -- 172.21.15.97:0/1198745454 shutdown_connections 2023-07-06T03:49:36.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.868+0000 7fbffc3ff700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfd806ace0 0x7fbfd806d190 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:36.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.868+0000 7fbffc3ff700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbff4159440 0x7fbff41430c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:36.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.868+0000 7fbffc3ff700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbff40c3cd0 0x7fbff414e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:36.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.868+0000 7fbffc3ff700 1 --2- 172.21.15.97:0/1198745454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbff4159070 0x7fbff40b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:36.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.868+0000 7fbffc3ff700 1 -- 172.21.15.97:0/1198745454 >> 172.21.15.97:0/1198745454 conn(0x7fbff406a060 msgr2=0x7fbff40ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:36.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.868+0000 7fbffc3ff700 1 -- 172.21.15.97:0/1198745454 shutdown_connections 2023-07-06T03:49:36.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:36.868+0000 7fbffc3ff700 1 -- 172.21.15.97:0/1198745454 wait complete. 2023-07-06T03:49:36.893 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T03:49:36.893 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T03:49:37.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.091+0000 7f88ea1ff700 1 Processor -- start 2023-07-06T03:49:37.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.091+0000 7f88ea1ff700 1 -- start start 2023-07-06T03:49:37.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.092+0000 7f88ea1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f88e415c610 0x7f88e40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:37.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.092+0000 7f88ea1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e40d2210 0x7f88e40d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:37.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.092+0000 7f88ea1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e40cff90 con 0x7f88e406b360 2023-07-06T03:49:37.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.092+0000 7f88ea1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e40d0780 con 0x7f88e415c610 2023-07-06T03:49:37.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.092+0000 7f88ea1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e415c9e0 con 0x7f88e40d2210 2023-07-06T03:49:37.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.092+0000 7f88e37fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f88e415c610 0x7f88e40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:37.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e37fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f88e415c610 0x7f88e40b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46112/0 (socket says 172.21.15.97:46112) 2023-07-06T03:49:37.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e37fe700 1 -- 172.21.15.97:0/4231213422 learned_addr learned my addr 172.21.15.97:0/4231213422 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:37.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e3fff700 1 --2- 172.21.15.97:0/4231213422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e40d2210 0x7f88e40d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:37.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e37fe700 1 -- 172.21.15.97:0/4231213422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e40d2210 msgr2=0x7f88e40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:37.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e37fe700 1 --2- 172.21.15.97:0/4231213422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e40d2210 0x7f88e40d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:37.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e37fe700 1 -- 172.21.15.97:0/4231213422 >> v1:172.21.15.146:6789/0 conn(0x7f88e406b360 legacy=0x7f88e40b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:37.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e37fe700 1 -- 172.21.15.97:0/4231213422 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e40b5650 con 0x7f88e415c610 2023-07-06T03:49:37.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e3fff700 1 --2- 172.21.15.97:0/4231213422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e40d2210 0x7f88e40d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:37.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e37fe700 1 --2- 172.21.15.97:0/4231213422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88e415c610 0x7f88e40b47b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f88d40098f0 tx=0x7f88d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f49f1da4262a2d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:37.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e27fc700 1 -- 172.21.15.97:0/4231213422 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d400b8a0 con 0x7f88e415c610 2023-07-06T03:49:37.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e27fc700 1 -- 172.21.15.97:0/4231213422 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d400ba00 con 0x7f88e415c610 2023-07-06T03:49:37.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.093+0000 7f88e27fc700 1 -- 172.21.15.97:0/4231213422 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d400bd90 con 0x7f88e415c610 2023-07-06T03:49:37.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.094+0000 7f88ea1ff700 1 -- 172.21.15.97:0/4231213422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88e415c610 msgr2=0x7f88e40b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:37.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.094+0000 7f88ea1ff700 1 --2- 172.21.15.97:0/4231213422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88e415c610 0x7f88e40b47b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f88d40098f0 tx=0x7f88d4009c00 comp rx=0 tx=0).stop 2023-07-06T03:49:37.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.094+0000 7f88ea1ff700 1 -- 172.21.15.97:0/4231213422 shutdown_connections 2023-07-06T03:49:37.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.094+0000 7f88ea1ff700 1 --2- 172.21.15.97:0/4231213422 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e40d2210 0x7f88e40d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:37.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.094+0000 7f88ea1ff700 1 --2- 172.21.15.97:0/4231213422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88e415c610 0x7f88e40b47b0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:37.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.094+0000 7f88ea1ff700 1 -- 172.21.15.97:0/4231213422 >> 172.21.15.97:0/4231213422 conn(0x7f88e406a6c0 msgr2=0x7f88e406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:37.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.094+0000 7f88ea1ff700 1 -- 172.21.15.97:0/4231213422 shutdown_connections 2023-07-06T03:49:37.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.094+0000 7f88ea1ff700 1 -- 172.21.15.97:0/4231213422 wait complete. 2023-07-06T03:49:37.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.096+0000 7f88ea1ff700 1 Processor -- start 2023-07-06T03:49:37.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.096+0000 7f88ea1ff700 1 -- start start 2023-07-06T03:49:37.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88ea1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88e406b360 0x7f88e40d1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:37.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88ea1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e40d2210 0x7f88e415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:37.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88ea1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e415c610 0x7f88e415ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:37.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88ea1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e40d1660 con 0x7f88e406b360 2023-07-06T03:49:37.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88ea1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e415db00 con 0x7f88e40d2210 2023-07-06T03:49:37.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88ea1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e41f1e60 con 0x7f88e415c610 2023-07-06T03:49:37.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88e37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88e406b360 0x7f88e40d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:37.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88e37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88e406b360 0x7f88e40d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46116/0 (socket says 172.21.15.97:46116) 2023-07-06T03:49:37.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88e37fe700 1 -- 172.21.15.97:0/1587942028 learned_addr learned my addr 172.21.15.97:0/1587942028 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:37.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88e37fe700 1 -- 172.21.15.97:0/1587942028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e415c610 msgr2=0x7f88e415ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:37.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88e3fff700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e415c610 0x7f88e415ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:37.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88e2ffd700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e40d2210 0x7f88e415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:37.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88e37fe700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e415c610 0x7f88e415ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:37.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.097+0000 7f88e37fe700 1 -- 172.21.15.97:0/1587942028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e40d2210 msgr2=0x7f88e415d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:37.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.098+0000 7f88e37fe700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e40d2210 0x7f88e415d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:37.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.098+0000 7f88e37fe700 1 -- 172.21.15.97:0/1587942028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d4009550 con 0x7f88e406b360 2023-07-06T03:49:37.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.098+0000 7f88e2ffd700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e40d2210 0x7f88e415d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:37.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.098+0000 7f88e37fe700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88e406b360 0x7f88e40d1120 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009c40 tx=0x7f88d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:37.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.098+0000 7f88cbfff700 1 -- 172.21.15.97:0/1587942028 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d40036a0 con 0x7f88e406b360 2023-07-06T03:49:37.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.098+0000 7f88cbfff700 1 -- 172.21.15.97:0/1587942028 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d401e070 con 0x7f88e406b360 2023-07-06T03:49:37.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.098+0000 7f88ea1ff700 1 -- 172.21.15.97:0/1587942028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e41f2140 con 0x7f88e406b360 2023-07-06T03:49:37.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.098+0000 7f88cbfff700 1 -- 172.21.15.97:0/1587942028 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4003a60 con 0x7f88e406b360 2023-07-06T03:49:37.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.098+0000 7f88ea1ff700 1 -- 172.21.15.97:0/1587942028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e41f2700 con 0x7f88e406b360 2023-07-06T03:49:37.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.099+0000 7f88ea1ff700 1 -- 172.21.15.97:0/1587942028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e404e340 con 0x7f88e406b360 2023-07-06T03:49:37.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.100+0000 7f88cbfff700 1 -- 172.21.15.97:0/1587942028 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f88d4003bc0 con 0x7f88e406b360 2023-07-06T03:49:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.101+0000 7f88cbfff700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88d006ace0 0x7f88d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.102+0000 7f88cbfff700 1 -- 172.21.15.97:0/1587942028 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f88d4020070 con 0x7f88e406b360 2023-07-06T03:49:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.102+0000 7f88e2ffd700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88d006ace0 0x7f88d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.102+0000 7f88e2ffd700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88d006ace0 0x7f88d006d190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000f60 tx=0x7f88d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.106+0000 7f88cbfff700 1 -- 172.21.15.97:0/1587942028 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f88d4055ad0 con 0x7f88e406b360 2023-07-06T03:49:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.347+0000 7f88ea1ff700 1 -- 172.21.15.97:0/1587942028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f88e415aa60 con 0x7f88e406b360 2023-07-06T03:49:37.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.871+0000 7f88cbfff700 1 -- 172.21.15.97:0/1587942028 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f88d40590f0 con 0x7f88e406b360 2023-07-06T03:49:37.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.873+0000 7f88ea1ff700 1 -- 172.21.15.97:0/1587942028 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88d006ace0 msgr2=0x7f88d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:37.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.873+0000 7f88ea1ff700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88d006ace0 0x7f88d006d190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000f60 tx=0x7f88d8009040 comp rx=0 tx=0).stop 2023-07-06T03:49:37.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.873+0000 7f88ea1ff700 1 -- 172.21.15.97:0/1587942028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88e406b360 msgr2=0x7f88e40d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:37.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.873+0000 7f88ea1ff700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88e406b360 0x7f88e40d1120 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009c40 tx=0x7f88d4004750 comp rx=0 tx=0).stop 2023-07-06T03:49:37.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.873+0000 7f88ea1ff700 1 -- 172.21.15.97:0/1587942028 shutdown_connections 2023-07-06T03:49:37.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.873+0000 7f88ea1ff700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88d006ace0 0x7f88d006d190 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:37.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.873+0000 7f88ea1ff700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88e415c610 0x7f88e415ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:37.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.873+0000 7f88ea1ff700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88e406b360 0x7f88e40d1120 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:37.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.873+0000 7f88ea1ff700 1 --2- 172.21.15.97:0/1587942028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88e40d2210 0x7f88e415d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:37.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.873+0000 7f88ea1ff700 1 -- 172.21.15.97:0/1587942028 >> 172.21.15.97:0/1587942028 conn(0x7f88e406a6c0 msgr2=0x7f88e4159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:37.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.873+0000 7f88ea1ff700 1 -- 172.21.15.97:0/1587942028 shutdown_connections 2023-07-06T03:49:37.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:37.874+0000 7f88ea1ff700 1 -- 172.21.15.97:0/1587942028 wait complete. 2023-07-06T03:49:37.894 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T03:49:37.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:49:38.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.086+0000 7fc72a3ff700 1 Processor -- start 2023-07-06T03:49:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.087+0000 7fc72a3ff700 1 -- start start 2023-07-06T03:49:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.088+0000 7fc72a3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7240cd390 0x7fc724160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.088+0000 7fc72a3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc724160dd0 0x7fc7240b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.088+0000 7fc72a3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7240cdae0 con 0x7fc72415c520 2023-07-06T03:49:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.088+0000 7fc72a3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7240b6640 con 0x7fc7240cd390 2023-07-06T03:49:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.088+0000 7fc72a3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7240b6920 con 0x7fc724160dd0 2023-07-06T03:49:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.088+0000 7fc723fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc72415c520 0x7fc724160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35590/0 (socket says 172.21.15.97:35590) 2023-07-06T03:49:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.088+0000 7fc723fff700 1 -- 172.21.15.97:0/1253468254 learned_addr learned my addr 172.21.15.97:0/1253468254 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.088+0000 7fc7237fe700 1 --2- 172.21.15.97:0/1253468254 >> v2:172.21.15.97:3300/0 conn(0x7fc7240cd390 0x7fc724160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.088+0000 7fc72899c700 1 --2- 172.21.15.97:0/1253468254 >> v2:172.21.15.146:3300/0 conn(0x7fc724160dd0 0x7fc7240b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.089+0000 7fc722ffd700 1 -- 172.21.15.97:0/1253468254 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268296236 0 0) 0x7fc7240cdae0 con 0x7fc72415c520 2023-07-06T03:49:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.089+0000 7fc722ffd700 1 -- 172.21.15.97:0/1253468254 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc70c0035b0 con 0x7fc72415c520 2023-07-06T03:49:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.089+0000 7fc7237fe700 1 -- 172.21.15.97:0/1253468254 >> v2:172.21.15.146:3300/0 conn(0x7fc724160dd0 msgr2=0x7fc7240b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.089+0000 7fc7237fe700 1 --2- 172.21.15.97:0/1253468254 >> v2:172.21.15.146:3300/0 conn(0x7fc724160dd0 0x7fc7240b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.089+0000 7fc7237fe700 1 -- 172.21.15.97:0/1253468254 >> v1:172.21.15.97:6789/0 conn(0x7fc72415c520 legacy=0x7fc724160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.089+0000 7fc7237fe700 1 -- 172.21.15.97:0/1253468254 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7240b6c00 con 0x7fc7240cd390 2023-07-06T03:49:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.089+0000 7fc7237fe700 1 --2- 172.21.15.97:0/1253468254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cd390 0x7fc724160890 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc7240cd320 tx=0x7fc718009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29fbe9e60bc0053 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.089+0000 7fc722ffd700 1 -- 172.21.15.97:0/1253468254 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718011830 con 0x7fc7240cd390 2023-07-06T03:49:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.089+0000 7fc722ffd700 1 -- 172.21.15.97:0/1253468254 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc718011990 con 0x7fc7240cd390 2023-07-06T03:49:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.089+0000 7fc722ffd700 1 -- 172.21.15.97:0/1253468254 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718011d20 con 0x7fc7240cd390 2023-07-06T03:49:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.090+0000 7fc72a3ff700 1 -- 172.21.15.97:0/1253468254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cd390 msgr2=0x7fc724160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.090+0000 7fc72a3ff700 1 --2- 172.21.15.97:0/1253468254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cd390 0x7fc724160890 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc7240cd320 tx=0x7fc718009b20 comp rx=0 tx=0).stop 2023-07-06T03:49:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.090+0000 7fc72a3ff700 1 -- 172.21.15.97:0/1253468254 shutdown_connections 2023-07-06T03:49:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.090+0000 7fc72a3ff700 1 --2- 172.21.15.97:0/1253468254 >> v2:172.21.15.146:3300/0 conn(0x7fc724160dd0 0x7fc7240b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.090+0000 7fc72a3ff700 1 --2- 172.21.15.97:0/1253468254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cd390 0x7fc724160890 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.090+0000 7fc72a3ff700 1 -- 172.21.15.97:0/1253468254 >> 172.21.15.97:0/1253468254 conn(0x7fc724071790 msgr2=0x7fc724071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.090+0000 7fc72a3ff700 1 -- 172.21.15.97:0/1253468254 shutdown_connections 2023-07-06T03:49:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.090+0000 7fc72a3ff700 1 -- 172.21.15.97:0/1253468254 wait complete. 2023-07-06T03:49:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.091+0000 7fc72a3ff700 1 Processor -- start 2023-07-06T03:49:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.092+0000 7fc72a3ff700 1 -- start start 2023-07-06T03:49:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc72a3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cd390 0x7fc7240cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc72a3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc72415c520 0x7fc72414fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc72a3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc724160dd0 0x7fc7240d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc72a3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240d2090 con 0x7fc7240cd390 2023-07-06T03:49:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc72a3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240d1540 con 0x7fc72415c520 2023-07-06T03:49:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc72a3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240d1820 con 0x7fc724160dd0 2023-07-06T03:49:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc723fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cd390 0x7fc7240cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc723fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cd390 0x7fc7240cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46138/0 (socket says 172.21.15.97:46138) 2023-07-06T03:49:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc723fff700 1 -- 172.21.15.97:0/792126249 learned_addr learned my addr 172.21.15.97:0/792126249 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc72899c700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc724160dd0 0x7fc7240d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc7237fe700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc72415c520 0x7fc72414fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc723fff700 1 -- 172.21.15.97:0/792126249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc724160dd0 msgr2=0x7fc7240d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc723fff700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc724160dd0 0x7fc7240d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc723fff700 1 -- 172.21.15.97:0/792126249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc72415c520 msgr2=0x7fc72414fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc723fff700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc72415c520 0x7fc72414fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc723fff700 1 -- 172.21.15.97:0/792126249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7180094a0 con 0x7fc7240cd390 2023-07-06T03:49:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc7237fe700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc72415c520 0x7fc72414fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc72899c700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc724160dd0 0x7fc7240d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.093+0000 7fc723fff700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cd390 0x7fc7240cb2c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c700 tx=0x7fc71400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.094+0000 7fc720ff9700 1 -- 172.21.15.97:0/792126249 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714005d70 con 0x7fc7240cd390 2023-07-06T03:49:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.094+0000 7fc720ff9700 1 -- 172.21.15.97:0/792126249 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7140070d0 con 0x7fc7240cd390 2023-07-06T03:49:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.094+0000 7fc720ff9700 1 -- 172.21.15.97:0/792126249 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714007460 con 0x7fc7240cd390 2023-07-06T03:49:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.094+0000 7fc72a3ff700 1 -- 172.21.15.97:0/792126249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7240d1b00 con 0x7fc7240cd390 2023-07-06T03:49:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.094+0000 7fc72a3ff700 1 -- 172.21.15.97:0/792126249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7241502a0 con 0x7fc7240cd390 2023-07-06T03:49:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.095+0000 7fc72a3ff700 1 -- 172.21.15.97:0/792126249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7240d2520 con 0x7fc7240cd390 2023-07-06T03:49:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.096+0000 7fc720ff9700 1 -- 172.21.15.97:0/792126249 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc7140075c0 con 0x7fc7240cd390 2023-07-06T03:49:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.097+0000 7fc720ff9700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc71006ace0 0x7fc71006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.097+0000 7fc720ff9700 1 -- 172.21.15.97:0/792126249 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc714088b90 con 0x7fc7240cd390 2023-07-06T03:49:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.097+0000 7fc7237fe700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc71006ace0 0x7fc71006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.098+0000 7fc7237fe700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc71006ace0 0x7fc71006d190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc724156ff0 tx=0x7fc718008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.101+0000 7fc720ff9700 1 -- 172.21.15.97:0/792126249 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc714053bb0 con 0x7fc7240cd390 2023-07-06T03:49:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.338+0000 7fc72a3ff700 1 -- 172.21.15.97:0/792126249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc72404e340 con 0x7fc7240cd390 2023-07-06T03:49:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.339+0000 7fc720ff9700 1 -- 172.21.15.97:0/792126249 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4679 (secure 0 0 0) 0x7fc7140571d0 con 0x7fc7240cd390 2023-07-06T03:49:38.341 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:38.342 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2344840493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2344840493},{"type":"v1","addr":"172.21.15.97:6836","nonce":2344840493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1930131645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1930131645},{"type":"v1","addr":"172.21.15.146:6835","nonce":1930131645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3936349837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3936349837},{"type":"v1","addr":"172.21.15.146:6833","nonce":3936349837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:49:34.310076+0000","modified":"2023-07-06T03:49:37.864717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6837":{"gid":6837,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6837/1957695290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1957695290},{"type":"v1","addr":"172.21.15.97:6837","nonce":1957695290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-06T03:49:38.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 -- 172.21.15.97:0/792126249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc71006ace0 msgr2=0x7fc71006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc71006ace0 0x7fc71006d190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc724156ff0 tx=0x7fc718008030 comp rx=0 tx=0).stop 2023-07-06T03:49:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 -- 172.21.15.97:0/792126249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cd390 msgr2=0x7fc7240cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cd390 0x7fc7240cb2c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c700 tx=0x7fc71400cac0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 -- 172.21.15.97:0/792126249 shutdown_connections 2023-07-06T03:49:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc71006ace0 0x7fc71006d190 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc724160dd0 0x7fc7240d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7240cd390 0x7fc7240cb2c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 --2- 172.21.15.97:0/792126249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc72415c520 0x7fc72414fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 -- 172.21.15.97:0/792126249 >> 172.21.15.97:0/792126249 conn(0x7fc724071790 msgr2=0x7fc7240b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:38.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 -- 172.21.15.97:0/792126249 shutdown_connections 2023-07-06T03:49:38.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.342+0000 7fc72a3ff700 1 -- 172.21.15.97:0/792126249 wait complete. 2023-07-06T03:49:38.345 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 44 2023-07-06T03:49:38.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:49:38.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.553+0000 7fb8dc5ff700 1 Processor -- start 2023-07-06T03:49:38.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.553+0000 7fb8dc5ff700 1 -- start start 2023-07-06T03:49:38.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.554+0000 7fb8dc5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8d415c520 0x7fb8d40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.554+0000 7fb8dc5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 0x7fb8d40b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.554+0000 7fb8dc5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d40cdae0 con 0x7fb8d4160180 2023-07-06T03:49:38.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.554+0000 7fb8dc5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d40cd8d0 con 0x7fb8d415c520 2023-07-06T03:49:38.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.554+0000 7fb8dc5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40b6640 con 0x7fb8d4160d00 2023-07-06T03:49:38.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.554+0000 7fb8da39b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8d415c520 0x7fb8d40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.554+0000 7fb8da39b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8d415c520 0x7fb8d40cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46142/0 (socket says 172.21.15.97:46142) 2023-07-06T03:49:38.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.554+0000 7fb8da39b700 1 -- 172.21.15.97:0/2460485904 learned_addr learned my addr 172.21.15.97:0/2460485904 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.555+0000 7fb8dab9c700 1 --2- 172.21.15.97:0/2460485904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 0x7fb8d40b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.555+0000 7fb8da39b700 1 -- 172.21.15.97:0/2460485904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 msgr2=0x7fb8d40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.555+0000 7fb8da39b700 1 --2- 172.21.15.97:0/2460485904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 0x7fb8d40b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.555+0000 7fb8da39b700 1 -- 172.21.15.97:0/2460485904 >> v1:172.21.15.146:6789/0 conn(0x7fb8d4160180 legacy=0x7fb8d41605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.555+0000 7fb8da39b700 1 -- 172.21.15.97:0/2460485904 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40b6920 con 0x7fb8d415c520 2023-07-06T03:49:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.555+0000 7fb8dab9c700 1 --2- 172.21.15.97:0/2460485904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 0x7fb8d40b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.555+0000 7fb8da39b700 1 --2- 172.21.15.97:0/2460485904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d415c520 0x7fb8d40cd390 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00098f0 tx=0x7fb8d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d035c1bab1afd578 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:38.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.555+0000 7fb8d9399700 1 -- 172.21.15.97:0/2460485904 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d000b8a0 con 0x7fb8d415c520 2023-07-06T03:49:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.555+0000 7fb8d9399700 1 -- 172.21.15.97:0/2460485904 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d000ba00 con 0x7fb8d415c520 2023-07-06T03:49:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.555+0000 7fb8d9399700 1 -- 172.21.15.97:0/2460485904 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d000bd90 con 0x7fb8d415c520 2023-07-06T03:49:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.556+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/2460485904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d415c520 msgr2=0x7fb8d40cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.556+0000 7fb8dc5ff700 1 --2- 172.21.15.97:0/2460485904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d415c520 0x7fb8d40cd390 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00098f0 tx=0x7fb8d0009c00 comp rx=0 tx=0).stop 2023-07-06T03:49:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.556+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/2460485904 shutdown_connections 2023-07-06T03:49:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.556+0000 7fb8dc5ff700 1 --2- 172.21.15.97:0/2460485904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 0x7fb8d40b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.556+0000 7fb8dc5ff700 1 --2- 172.21.15.97:0/2460485904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d415c520 0x7fb8d40cd390 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.556+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/2460485904 >> 172.21.15.97:0/2460485904 conn(0x7fb8d4071790 msgr2=0x7fb8d4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:38.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.556+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/2460485904 shutdown_connections 2023-07-06T03:49:38.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.556+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/2460485904 wait complete. 2023-07-06T03:49:38.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.558+0000 7fb8dc5ff700 1 Processor -- start 2023-07-06T03:49:38.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.558+0000 7fb8dc5ff700 1 -- start start 2023-07-06T03:49:38.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8dc5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d415c520 0x7fb8d414d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8dc5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8d4160180 0x7fb8d40d3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8dc5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 0x7fb8d40d3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8dc5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d414d850 con 0x7fb8d415c520 2023-07-06T03:49:38.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8dc5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40d3ba0 con 0x7fb8d4160180 2023-07-06T03:49:38.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8dc5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d415f330 con 0x7fb8d4160d00 2023-07-06T03:49:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8da39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d415c520 0x7fb8d414d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8da39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d415c520 0x7fb8d414d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46146/0 (socket says 172.21.15.97:46146) 2023-07-06T03:49:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8da39b700 1 -- 172.21.15.97:0/3682130575 learned_addr learned my addr 172.21.15.97:0/3682130575 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8da39b700 1 -- 172.21.15.97:0/3682130575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 msgr2=0x7fb8d40d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8d9b9a700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8d4160180 0x7fb8d40d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8dab9c700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 0x7fb8d40d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8da39b700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 0x7fb8d40d3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8da39b700 1 -- 172.21.15.97:0/3682130575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8d4160180 msgr2=0x7fb8d40d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8da39b700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8d4160180 0x7fb8d40d3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8da39b700 1 -- 172.21.15.97:0/3682130575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0009550 con 0x7fb8d415c520 2023-07-06T03:49:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8d9b9a700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8d4160180 0x7fb8d40d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.559+0000 7fb8dab9c700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 0x7fb8d40d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.560+0000 7fb8da39b700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d415c520 0x7fb8d414d310 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009c40 tx=0x7fb8d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.560+0000 7fb8caffd700 1 -- 172.21.15.97:0/3682130575 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0004530 con 0x7fb8d415c520 2023-07-06T03:49:38.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.560+0000 7fb8caffd700 1 -- 172.21.15.97:0/3682130575 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d001f070 con 0x7fb8d415c520 2023-07-06T03:49:38.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.560+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/3682130575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d40cab90 con 0x7fb8d415c520 2023-07-06T03:49:38.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.560+0000 7fb8caffd700 1 -- 172.21.15.97:0/3682130575 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00038e0 con 0x7fb8d415c520 2023-07-06T03:49:38.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.560+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/3682130575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d40cb150 con 0x7fb8d415c520 2023-07-06T03:49:38.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.561+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/3682130575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d404e340 con 0x7fb8d415c520 2023-07-06T03:49:38.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.562+0000 7fb8caffd700 1 -- 172.21.15.97:0/3682130575 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb8d0021030 con 0x7fb8d415c520 2023-07-06T03:49:38.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.563+0000 7fb8caffd700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8b806ace0 0x7fb8b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.563+0000 7fb8caffd700 1 -- 172.21.15.97:0/3682130575 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb8d008aa90 con 0x7fb8d415c520 2023-07-06T03:49:38.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.563+0000 7fb8d9b9a700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8b806ace0 0x7fb8b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.564+0000 7fb8d9b9a700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8b806ace0 0x7fb8b806d190 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb8d415f830 tx=0x7fb8c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:38.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.567+0000 7fb8caffd700 1 -- 172.21.15.97:0/3682130575 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb8d0055bc0 con 0x7fb8d415c520 2023-07-06T03:49:38.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.766+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/3682130575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8d40b4f40 con 0x7fb8d415c520 2023-07-06T03:49:38.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.767+0000 7fb8caffd700 1 -- 172.21.15.97:0/3682130575 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18320 (secure 0 0 0) 0x7fb8d0015090 con 0x7fb8d415c520 2023-07-06T03:49:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:38.770 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":74,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:49:35.835648+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:49:32.371992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-06T03:49:33.281260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:49:38.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/3682130575 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8b806ace0 msgr2=0x7fb8b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8b806ace0 0x7fb8b806d190 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb8d415f830 tx=0x7fb8c4009040 comp rx=0 tx=0).stop 2023-07-06T03:49:38.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/3682130575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d415c520 msgr2=0x7fb8d414d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d415c520 0x7fb8d414d310 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009c40 tx=0x7fb8d0004750 comp rx=0 tx=0).stop 2023-07-06T03:49:38.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/3682130575 shutdown_connections 2023-07-06T03:49:38.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb8b806ace0 0x7fb8b806d190 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8d4160d00 0x7fb8d40d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d415c520 0x7fb8d414d310 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 --2- 172.21.15.97:0/3682130575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8d4160180 0x7fb8d40d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/3682130575 >> 172.21.15.97:0/3682130575 conn(0x7fb8d4071790 msgr2=0x7fb8d40b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:38.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/3682130575 shutdown_connections 2023-07-06T03:49:38.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.770+0000 7fb8dc5ff700 1 -- 172.21.15.97:0/3682130575 wait complete. 2023-07-06T03:49:38.780 INFO:tasks.ceph.mds.a.smithi097.stderr:2023-07-06T03:49:38.779+0000 7fa8f6ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:49:38.790 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T03:49:38.790 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:49:38.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.985+0000 7f34b3fff700 1 Processor -- start 2023-07-06T03:49:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.986+0000 7f34b3fff700 1 -- start start 2023-07-06T03:49:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.987+0000 7f34b3fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34ac0cd390 0x7f34ac160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.987+0000 7f34b3fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f34ac160dd0 0x7f34ac0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.987+0000 7f34b3fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ac0cdae0 con 0x7f34ac15c520 2023-07-06T03:49:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.987+0000 7f34b3fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ac0b6640 con 0x7f34ac160dd0 2023-07-06T03:49:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.987+0000 7f34b3fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0b6920 con 0x7f34ac0cd390 2023-07-06T03:49:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.987+0000 7f34b259c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f34ac160dd0 0x7f34ac0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.987+0000 7f34b159a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34ac0cd390 0x7f34ac160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.987+0000 7f34b259c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f34ac160dd0 0x7f34ac0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:41984/0 (socket says 172.21.15.97:41984) 2023-07-06T03:49:38.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.987+0000 7f34b259c700 1 -- 172.21.15.97:0/1924515876 learned_addr learned my addr 172.21.15.97:0/1924515876 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:38.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.987+0000 7f34b1d9b700 1 --1- 172.21.15.97:0/1924515876 >> v1:172.21.15.146:6789/0 conn(0x7f34ac15c520 0x7f34ac160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:52930/0 (socket says 172.21.15.97:52930) 2023-07-06T03:49:38.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.988+0000 7f34b0d99700 1 -- 172.21.15.97:0/1924515876 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161063563 0 0) 0x7f34ac0cdae0 con 0x7f34ac15c520 2023-07-06T03:49:38.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.988+0000 7f34b0d99700 1 -- 172.21.15.97:0/1924515876 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34980035b0 con 0x7f34ac15c520 2023-07-06T03:49:38.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.988+0000 7f34b259c700 1 -- 172.21.15.97:0/1924515876 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34ac0cd390 msgr2=0x7f34ac160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:38.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.988+0000 7f34b259c700 1 --2- 172.21.15.97:0/1924515876 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34ac0cd390 0x7f34ac160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.988+0000 7f34b259c700 1 -- 172.21.15.97:0/1924515876 >> v1:172.21.15.146:6789/0 conn(0x7f34ac15c520 legacy=0x7f34ac160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.988+0000 7f34b259c700 1 -- 172.21.15.97:0/1924515876 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac0b6c00 con 0x7f34ac160dd0 2023-07-06T03:49:38.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.988+0000 7f34b159a700 1 --2- 172.21.15.97:0/1924515876 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34ac0cd390 0x7f34ac160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:38.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.988+0000 7f34b259c700 1 --2- 172.21.15.97:0/1924515876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34ac160dd0 0x7f34ac0b7400 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009e50 tx=0x7f34a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4537093b2bdc17a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:38.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.988+0000 7f34b0d99700 1 -- 172.21.15.97:0/1924515876 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400e070 con 0x7f34ac160dd0 2023-07-06T03:49:38.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.988+0000 7f34b0d99700 1 -- 172.21.15.97:0/1924515876 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a400b910 con 0x7f34ac160dd0 2023-07-06T03:49:38.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.988+0000 7f34b0d99700 1 -- 172.21.15.97:0/1924515876 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400bcc0 con 0x7f34ac160dd0 2023-07-06T03:49:38.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.989+0000 7f34b3fff700 1 -- 172.21.15.97:0/1924515876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34ac160dd0 msgr2=0x7f34ac0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.989+0000 7f34b3fff700 1 --2- 172.21.15.97:0/1924515876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34ac160dd0 0x7f34ac0b7400 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009e50 tx=0x7f34a400b330 comp rx=0 tx=0).stop 2023-07-06T03:49:38.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.989+0000 7f34b3fff700 1 -- 172.21.15.97:0/1924515876 shutdown_connections 2023-07-06T03:49:38.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.989+0000 7f34b3fff700 1 --2- 172.21.15.97:0/1924515876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34ac160dd0 0x7f34ac0b7400 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.989+0000 7f34b3fff700 1 --2- 172.21.15.97:0/1924515876 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34ac0cd390 0x7f34ac160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.989+0000 7f34b3fff700 1 -- 172.21.15.97:0/1924515876 >> 172.21.15.97:0/1924515876 conn(0x7f34ac071790 msgr2=0x7f34ac071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:38.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.989+0000 7f34b3fff700 1 -- 172.21.15.97:0/1924515876 shutdown_connections 2023-07-06T03:49:38.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.989+0000 7f34b3fff700 1 -- 172.21.15.97:0/1924515876 wait complete. 2023-07-06T03:49:38.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.991+0000 7f34b3fff700 1 Processor -- start 2023-07-06T03:49:38.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.991+0000 7f34b3fff700 1 -- start start 2023-07-06T03:49:38.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.992+0000 7f34b3fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34ac0cd390 0x7f34ac14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.992+0000 7f34b3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34ac15c520 0x7f34ac15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.992+0000 7f34b3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac160dd0 0x7f34ac1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.992+0000 7f34b3fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac15b240 con 0x7f34ac160dd0 2023-07-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.992+0000 7f34b3fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac15a910 con 0x7f34ac15c520 2023-07-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.992+0000 7f34b3fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac15abf0 con 0x7f34ac0cd390 2023-07-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.992+0000 7f34b259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac160dd0 0x7f34ac1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.992+0000 7f34b259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac160dd0 0x7f34ac1586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46154/0 (socket says 172.21.15.97:46154) 2023-07-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.992+0000 7f34b259c700 1 -- 172.21.15.97:0/4000768024 learned_addr learned my addr 172.21.15.97:0/4000768024 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.992+0000 7f34b1d9b700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34ac0cd390 0x7f34ac14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.992+0000 7f34b159a700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34ac15c520 0x7f34ac15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:38.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.993+0000 7f34b259c700 1 -- 172.21.15.97:0/4000768024 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34ac0cd390 msgr2=0x7f34ac14c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.993+0000 7f34b259c700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34ac0cd390 0x7f34ac14c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.993+0000 7f34b259c700 1 -- 172.21.15.97:0/4000768024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34ac15c520 msgr2=0x7f34ac15b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:38.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.993+0000 7f34b259c700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34ac15c520 0x7f34ac15b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:38.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.993+0000 7f34b259c700 1 -- 172.21.15.97:0/4000768024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c0094a0 con 0x7f34ac160dd0 2023-07-06T03:49:38.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.993+0000 7f34b259c700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac160dd0 0x7f34ac1586f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f34ac06c240 tx=0x7f34a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:38.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.993+0000 7f34a27fc700 1 -- 172.21.15.97:0/4000768024 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4004800 con 0x7f34ac160dd0 2023-07-06T03:49:38.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.993+0000 7f34a27fc700 1 -- 172.21.15.97:0/4000768024 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a4004960 con 0x7f34ac160dd0 2023-07-06T03:49:38.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.993+0000 7f34a27fc700 1 -- 172.21.15.97:0/4000768024 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a401c650 con 0x7f34ac160dd0 2023-07-06T03:49:38.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.993+0000 7f34b3fff700 1 -- 172.21.15.97:0/4000768024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a4009b30 con 0x7f34ac160dd0 2023-07-06T03:49:38.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.993+0000 7f34b3fff700 1 -- 172.21.15.97:0/4000768024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac15bc10 con 0x7f34ac160dd0 2023-07-06T03:49:38.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.994+0000 7f34b3fff700 1 -- 172.21.15.97:0/4000768024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ac15b3b0 con 0x7f34ac160dd0 2023-07-06T03:49:38.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.995+0000 7f34a27fc700 1 -- 172.21.15.97:0/4000768024 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34a401c7b0 con 0x7f34ac160dd0 2023-07-06T03:49:38.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.996+0000 7f34a27fc700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f349006ace0 0x7f349006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.996+0000 7f34a27fc700 1 -- 172.21.15.97:0/4000768024 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f34a408abc0 con 0x7f34ac160dd0 2023-07-06T03:49:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.996+0000 7f34b1d9b700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f349006ace0 0x7f349006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:38.997+0000 7f34b1d9b700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f349006ace0 0x7f349006d190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f34a8007fd0 tx=0x7f34a8007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.001+0000 7f34a27fc700 1 -- 172.21.15.97:0/4000768024 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f34a4055c60 con 0x7f34ac160dd0 2023-07-06T03:49:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.190+0000 7f34b3fff700 1 -- 172.21.15.97:0/4000768024 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f34ac0cd800 con 0x7f349006ace0 2023-07-06T03:49:39.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.200+0000 7f34a27fc700 1 -- 172.21.15.97:0/4000768024 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311197 (secure 0 0 0) 0x7f34a8001370 con 0x7f349006ace0 2023-07-06T03:49:39.205 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.204+0000 7f34b3fff700 1 -- 172.21.15.97:0/4000768024 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f349006ace0 msgr2=0x7f349006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.204+0000 7f34b3fff700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f349006ace0 0x7f349006d190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f34a8007fd0 tx=0x7f34a8007dc0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.204+0000 7f34b3fff700 1 -- 172.21.15.97:0/4000768024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac160dd0 msgr2=0x7f34ac1586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.204+0000 7f34b3fff700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac160dd0 0x7f34ac1586f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f34ac06c240 tx=0x7f34a4004ad0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.204+0000 7f34b3fff700 1 -- 172.21.15.97:0/4000768024 shutdown_connections 2023-07-06T03:49:39.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.204+0000 7f34b3fff700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f349006ace0 0x7f349006d190 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.204+0000 7f34b3fff700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac160dd0 0x7f34ac1586f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.204+0000 7f34b3fff700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34ac0cd390 0x7f34ac14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.204+0000 7f34b3fff700 1 --2- 172.21.15.97:0/4000768024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34ac15c520 0x7f34ac15b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.204+0000 7f34b3fff700 1 -- 172.21.15.97:0/4000768024 >> 172.21.15.97:0/4000768024 conn(0x7f34ac071790 msgr2=0x7f34ac15eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.205+0000 7f34b3fff700 1 -- 172.21.15.97:0/4000768024 shutdown_connections 2023-07-06T03:49:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.205+0000 7f34b3fff700 1 -- 172.21.15.97:0/4000768024 wait complete. 2023-07-06T03:49:39.209 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:49:39.224 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2023-07-06T03:49:38.936423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264764,"kb_used_data":8188,"kb_used_omap":0,"kb_used_meta":256512,"kb_avail":754709956,"statfs":{"total":773094113280,"available":772822994944,"internally_reserved":0,"allocated":8384512,"data_stored":4382931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":262668288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001304"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852821+0000","last_change":"2023-07-06T03:49:34.857392+0000","last_active":"2023-07-06T03:49:35.852821+0000","last_peered":"2023-07-06T03:49:35.852821+0000","last_clean":"2023-07-06T03:49:35.852821+0000","last_became_active":"2023-07-06T03:49:34.856886+0000","last_became_peered":"2023-07-06T03:49:34.856886+0000","last_unstale":"2023-07-06T03:49:35.852821+0000","last_undegraded":"2023-07-06T03:49:35.852821+0000","last_fullsized":"2023-07-06T03:49:35.852821+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:12:38.263762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853120+0000","last_change":"2023-07-06T03:49:34.861439+0000","last_active":"2023-07-06T03:49:35.853120+0000","last_peered":"2023-07-06T03:49:35.853120+0000","last_clean":"2023-07-06T03:49:35.853120+0000","last_became_active":"2023-07-06T03:49:34.861097+0000","last_became_peered":"2023-07-06T03:49:34.861097+0000","last_unstale":"2023-07-06T03:49:35.853120+0000","last_undegraded":"2023-07-06T03:49:35.853120+0000","last_fullsized":"2023-07-06T03:49:35.853120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:09:16.549661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852989+0000","last_change":"2023-07-06T03:49:34.859353+0000","last_active":"2023-07-06T03:49:35.852989+0000","last_peered":"2023-07-06T03:49:35.852989+0000","last_clean":"2023-07-06T03:49:35.852989+0000","last_became_active":"2023-07-06T03:49:34.858960+0000","last_became_peered":"2023-07-06T03:49:34.858960+0000","last_unstale":"2023-07-06T03:49:35.852989+0000","last_undegraded":"2023-07-06T03:49:35.852989+0000","last_fullsized":"2023-07-06T03:49:35.852989+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:10.128139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853190+0000","last_change":"2023-07-06T03:49:34.856387+0000","last_active":"2023-07-06T03:49:35.853190+0000","last_peered":"2023-07-06T03:49:35.853190+0000","last_clean":"2023-07-06T03:49:35.853190+0000","last_became_active":"2023-07-06T03:49:34.856126+0000","last_became_peered":"2023-07-06T03:49:34.856126+0000","last_unstale":"2023-07-06T03:49:35.853190+0000","last_undegraded":"2023-07-06T03:49:35.853190+0000","last_fullsized":"2023-07-06T03:49:35.853190+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:17.448184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842500+0000","last_change":"2023-07-06T03:49:34.861369+0000","last_active":"2023-07-06T03:49:35.842500+0000","last_peered":"2023-07-06T03:49:35.842500+0000","last_clean":"2023-07-06T03:49:35.842500+0000","last_became_active":"2023-07-06T03:49:34.861097+0000","last_became_peered":"2023-07-06T03:49:34.861097+0000","last_unstale":"2023-07-06T03:49:35.842500+0000","last_undegraded":"2023-07-06T03:49:35.842500+0000","last_fullsized":"2023-07-06T03:49:35.842500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:32:50.648377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844955+0000","last_change":"2023-07-06T03:49:34.863174+0000","last_active":"2023-07-06T03:49:35.844955+0000","last_peered":"2023-07-06T03:49:35.844955+0000","last_clean":"2023-07-06T03:49:35.844955+0000","last_became_active":"2023-07-06T03:49:34.862827+0000","last_became_peered":"2023-07-06T03:49:34.862827+0000","last_unstale":"2023-07-06T03:49:35.844955+0000","last_undegraded":"2023-07-06T03:49:35.844955+0000","last_fullsized":"2023-07-06T03:49:35.844955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:40:19.464216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852502+0000","last_change":"2023-07-06T03:49:34.863940+0000","last_active":"2023-07-06T03:49:35.852502+0000","last_peered":"2023-07-06T03:49:35.852502+0000","last_clean":"2023-07-06T03:49:35.852502+0000","last_became_active":"2023-07-06T03:49:34.863668+0000","last_became_peered":"2023-07-06T03:49:34.863668+0000","last_unstale":"2023-07-06T03:49:35.852502+0000","last_undegraded":"2023-07-06T03:49:35.852502+0000","last_fullsized":"2023-07-06T03:49:35.852502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:19:29.742076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843458+0000","last_change":"2023-07-06T03:49:34.864528+0000","last_active":"2023-07-06T03:49:35.843458+0000","last_peered":"2023-07-06T03:49:35.843458+0000","last_clean":"2023-07-06T03:49:35.843458+0000","last_became_active":"2023-07-06T03:49:34.864259+0000","last_became_peered":"2023-07-06T03:49:34.864259+0000","last_unstale":"2023-07-06T03:49:35.843458+0000","last_undegraded":"2023-07-06T03:49:35.843458+0000","last_fullsized":"2023-07-06T03:49:35.843458+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:53:57.372988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853745+0000","last_change":"2023-07-06T03:49:34.859360+0000","last_active":"2023-07-06T03:49:35.853745+0000","last_peered":"2023-07-06T03:49:35.853745+0000","last_clean":"2023-07-06T03:49:35.853745+0000","last_became_active":"2023-07-06T03:49:34.859001+0000","last_became_peered":"2023-07-06T03:49:34.859001+0000","last_unstale":"2023-07-06T03:49:35.853745+0000","last_undegraded":"2023-07-06T03:49:35.853745+0000","last_fullsized":"2023-07-06T03:49:35.853745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:40:59.286805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852523+0000","last_change":"2023-07-06T03:49:34.855165+0000","last_active":"2023-07-06T03:49:35.852523+0000","last_peered":"2023-07-06T03:49:35.852523+0000","last_clean":"2023-07-06T03:49:35.852523+0000","last_became_active":"2023-07-06T03:49:34.854887+0000","last_became_peered":"2023-07-06T03:49:34.854887+0000","last_unstale":"2023-07-06T03:49:35.852523+0000","last_undegraded":"2023-07-06T03:49:35.852523+0000","last_fullsized":"2023-07-06T03:49:35.852523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:32:24.890494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853043+0000","last_change":"2023-07-06T03:49:34.860985+0000","last_active":"2023-07-06T03:49:35.853043+0000","last_peered":"2023-07-06T03:49:35.853043+0000","last_clean":"2023-07-06T03:49:35.853043+0000","last_became_active":"2023-07-06T03:49:34.860690+0000","last_became_peered":"2023-07-06T03:49:34.860690+0000","last_unstale":"2023-07-06T03:49:35.853043+0000","last_undegraded":"2023-07-06T03:49:35.853043+0000","last_fullsized":"2023-07-06T03:49:35.853043+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:02.931578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847357+0000","last_change":"2023-07-06T03:49:34.873436+0000","last_active":"2023-07-06T03:49:35.847357+0000","last_peered":"2023-07-06T03:49:35.847357+0000","last_clean":"2023-07-06T03:49:35.847357+0000","last_became_active":"2023-07-06T03:49:34.873184+0000","last_became_peered":"2023-07-06T03:49:34.873184+0000","last_unstale":"2023-07-06T03:49:35.847357+0000","last_undegraded":"2023-07-06T03:49:35.847357+0000","last_fullsized":"2023-07-06T03:49:35.847357+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:02.931578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852890+0000","last_change":"2023-07-06T03:49:34.861034+0000","last_active":"2023-07-06T03:49:35.852890+0000","last_peered":"2023-07-06T03:49:35.852890+0000","last_clean":"2023-07-06T03:49:35.852890+0000","last_became_active":"2023-07-06T03:49:34.860723+0000","last_became_peered":"2023-07-06T03:49:34.860723+0000","last_unstale":"2023-07-06T03:49:35.852890+0000","last_undegraded":"2023-07-06T03:49:35.852890+0000","last_fullsized":"2023-07-06T03:49:35.852890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:36:40.499610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843975+0000","last_change":"2023-07-06T03:49:34.861781+0000","last_active":"2023-07-06T03:49:35.843975+0000","last_peered":"2023-07-06T03:49:35.843975+0000","last_clean":"2023-07-06T03:49:35.843975+0000","last_became_active":"2023-07-06T03:49:34.861467+0000","last_became_peered":"2023-07-06T03:49:34.861467+0000","last_unstale":"2023-07-06T03:49:35.843975+0000","last_undegraded":"2023-07-06T03:49:35.843975+0000","last_fullsized":"2023-07-06T03:49:35.843975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:57:00.526918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842573+0000","last_change":"2023-07-06T03:49:34.854894+0000","last_active":"2023-07-06T03:49:35.842573+0000","last_peered":"2023-07-06T03:49:35.842573+0000","last_clean":"2023-07-06T03:49:35.842573+0000","last_became_active":"2023-07-06T03:49:34.854588+0000","last_became_peered":"2023-07-06T03:49:34.854588+0000","last_unstale":"2023-07-06T03:49:35.842573+0000","last_undegraded":"2023-07-06T03:49:35.842573+0000","last_fullsized":"2023-07-06T03:49:35.842573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:19:11.597750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852420+0000","last_change":"2023-07-06T03:49:34.867737+0000","last_active":"2023-07-06T03:49:35.852420+0000","last_peered":"2023-07-06T03:49:35.852420+0000","last_clean":"2023-07-06T03:49:35.852420+0000","last_became_active":"2023-07-06T03:49:34.867245+0000","last_became_peered":"2023-07-06T03:49:34.867245+0000","last_unstale":"2023-07-06T03:49:35.852420+0000","last_undegraded":"2023-07-06T03:49:35.852420+0000","last_fullsized":"2023-07-06T03:49:35.852420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:53.991327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844408+0000","last_change":"2023-07-06T03:49:34.866813+0000","last_active":"2023-07-06T03:49:35.844408+0000","last_peered":"2023-07-06T03:49:35.844408+0000","last_clean":"2023-07-06T03:49:35.844408+0000","last_became_active":"2023-07-06T03:49:34.866553+0000","last_became_peered":"2023-07-06T03:49:34.866553+0000","last_unstale":"2023-07-06T03:49:35.844408+0000","last_undegraded":"2023-07-06T03:49:35.844408+0000","last_fullsized":"2023-07-06T03:49:35.844408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:54:57.096501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853248+0000","last_change":"2023-07-06T03:49:34.860176+0000","last_active":"2023-07-06T03:49:35.853248+0000","last_peered":"2023-07-06T03:49:35.853248+0000","last_clean":"2023-07-06T03:49:35.853248+0000","last_became_active":"2023-07-06T03:49:34.859795+0000","last_became_peered":"2023-07-06T03:49:34.859795+0000","last_unstale":"2023-07-06T03:49:35.853248+0000","last_undegraded":"2023-07-06T03:49:35.853248+0000","last_fullsized":"2023-07-06T03:49:35.853248+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:35.947643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844462+0000","last_change":"2023-07-06T03:49:34.871124+0000","last_active":"2023-07-06T03:49:35.844462+0000","last_peered":"2023-07-06T03:49:35.844462+0000","last_clean":"2023-07-06T03:49:35.844462+0000","last_became_active":"2023-07-06T03:49:34.870767+0000","last_became_peered":"2023-07-06T03:49:34.870767+0000","last_unstale":"2023-07-06T03:49:35.844462+0000","last_undegraded":"2023-07-06T03:49:35.844462+0000","last_fullsized":"2023-07-06T03:49:35.844462+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:08:25.215713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844256+0000","last_change":"2023-07-06T03:49:34.866285+0000","last_active":"2023-07-06T03:49:35.844256+0000","last_peered":"2023-07-06T03:49:35.844256+0000","last_clean":"2023-07-06T03:49:35.844256+0000","last_became_active":"2023-07-06T03:49:34.865816+0000","last_became_peered":"2023-07-06T03:49:34.865816+0000","last_unstale":"2023-07-06T03:49:35.844256+0000","last_undegraded":"2023-07-06T03:49:35.844256+0000","last_fullsized":"2023-07-06T03:49:35.844256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:51.920331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847358+0000","last_change":"2023-07-06T03:49:34.863645+0000","last_active":"2023-07-06T03:49:35.847358+0000","last_peered":"2023-07-06T03:49:35.847358+0000","last_clean":"2023-07-06T03:49:35.847358+0000","last_became_active":"2023-07-06T03:49:34.863084+0000","last_became_peered":"2023-07-06T03:49:34.863084+0000","last_unstale":"2023-07-06T03:49:35.847358+0000","last_undegraded":"2023-07-06T03:49:35.847358+0000","last_fullsized":"2023-07-06T03:49:35.847358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:10.579687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852676+0000","last_change":"2023-07-06T03:49:34.866780+0000","last_active":"2023-07-06T03:49:35.852676+0000","last_peered":"2023-07-06T03:49:35.852676+0000","last_clean":"2023-07-06T03:49:35.852676+0000","last_became_active":"2023-07-06T03:49:34.866518+0000","last_became_peered":"2023-07-06T03:49:34.866518+0000","last_unstale":"2023-07-06T03:49:35.852676+0000","last_undegraded":"2023-07-06T03:49:35.852676+0000","last_fullsized":"2023-07-06T03:49:35.852676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:56:22.166639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852688+0000","last_change":"2023-07-06T03:49:34.861289+0000","last_active":"2023-07-06T03:49:35.852688+0000","last_peered":"2023-07-06T03:49:35.852688+0000","last_clean":"2023-07-06T03:49:35.852688+0000","last_became_active":"2023-07-06T03:49:34.860976+0000","last_became_peered":"2023-07-06T03:49:34.860976+0000","last_unstale":"2023-07-06T03:49:35.852688+0000","last_undegraded":"2023-07-06T03:49:35.852688+0000","last_fullsized":"2023-07-06T03:49:35.852688+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:38:04.661296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852494+0000","last_change":"2023-07-06T03:49:34.863001+0000","last_active":"2023-07-06T03:49:35.852494+0000","last_peered":"2023-07-06T03:49:35.852494+0000","last_clean":"2023-07-06T03:49:35.852494+0000","last_became_active":"2023-07-06T03:49:34.862732+0000","last_became_peered":"2023-07-06T03:49:34.862732+0000","last_unstale":"2023-07-06T03:49:35.852494+0000","last_undegraded":"2023-07-06T03:49:35.852494+0000","last_fullsized":"2023-07-06T03:49:35.852494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:12:33.006465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844155+0000","last_change":"2023-07-06T03:49:34.859976+0000","last_active":"2023-07-06T03:49:35.844155+0000","last_peered":"2023-07-06T03:49:35.844155+0000","last_clean":"2023-07-06T03:49:35.844155+0000","last_became_active":"2023-07-06T03:49:34.859719+0000","last_became_peered":"2023-07-06T03:49:34.859719+0000","last_unstale":"2023-07-06T03:49:35.844155+0000","last_undegraded":"2023-07-06T03:49:35.844155+0000","last_fullsized":"2023-07-06T03:49:35.844155+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:00.819925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853236+0000","last_change":"2023-07-06T03:49:34.855924+0000","last_active":"2023-07-06T03:49:35.853236+0000","last_peered":"2023-07-06T03:49:35.853236+0000","last_clean":"2023-07-06T03:49:35.853236+0000","last_became_active":"2023-07-06T03:49:34.855644+0000","last_became_peered":"2023-07-06T03:49:34.855644+0000","last_unstale":"2023-07-06T03:49:35.853236+0000","last_undegraded":"2023-07-06T03:49:35.853236+0000","last_fullsized":"2023-07-06T03:49:35.853236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:17:24.091237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853073+0000","last_change":"2023-07-06T03:49:34.866832+0000","last_active":"2023-07-06T03:49:35.853073+0000","last_peered":"2023-07-06T03:49:35.853073+0000","last_clean":"2023-07-06T03:49:35.853073+0000","last_became_active":"2023-07-06T03:49:34.866537+0000","last_became_peered":"2023-07-06T03:49:34.866537+0000","last_unstale":"2023-07-06T03:49:35.853073+0000","last_undegraded":"2023-07-06T03:49:35.853073+0000","last_fullsized":"2023-07-06T03:49:35.853073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:58:09.047067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842680+0000","last_change":"2023-07-06T03:49:34.861645+0000","last_active":"2023-07-06T03:49:35.842680+0000","last_peered":"2023-07-06T03:49:35.842680+0000","last_clean":"2023-07-06T03:49:35.842680+0000","last_became_active":"2023-07-06T03:49:34.861348+0000","last_became_peered":"2023-07-06T03:49:34.861348+0000","last_unstale":"2023-07-06T03:49:35.842680+0000","last_undegraded":"2023-07-06T03:49:35.842680+0000","last_fullsized":"2023-07-06T03:49:35.842680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:56:44.253681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844259+0000","last_change":"2023-07-06T03:49:34.861537+0000","last_active":"2023-07-06T03:49:35.844259+0000","last_peered":"2023-07-06T03:49:35.844259+0000","last_clean":"2023-07-06T03:49:35.844259+0000","last_became_active":"2023-07-06T03:49:34.861268+0000","last_became_peered":"2023-07-06T03:49:34.861268+0000","last_unstale":"2023-07-06T03:49:35.844259+0000","last_undegraded":"2023-07-06T03:49:35.844259+0000","last_fullsized":"2023-07-06T03:49:35.844259+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:35.842450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852175+0000","last_change":"2023-07-06T03:49:34.864092+0000","last_active":"2023-07-06T03:49:35.852175+0000","last_peered":"2023-07-06T03:49:35.852175+0000","last_clean":"2023-07-06T03:49:35.852175+0000","last_became_active":"2023-07-06T03:49:34.863833+0000","last_became_peered":"2023-07-06T03:49:34.863833+0000","last_unstale":"2023-07-06T03:49:35.852175+0000","last_undegraded":"2023-07-06T03:49:35.852175+0000","last_fullsized":"2023-07-06T03:49:35.852175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:44:27.220590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852594+0000","last_change":"2023-07-06T03:49:34.865589+0000","last_active":"2023-07-06T03:49:35.852594+0000","last_peered":"2023-07-06T03:49:35.852594+0000","last_clean":"2023-07-06T03:49:35.852594+0000","last_became_active":"2023-07-06T03:49:34.865282+0000","last_became_peered":"2023-07-06T03:49:34.865282+0000","last_unstale":"2023-07-06T03:49:35.852594+0000","last_undegraded":"2023-07-06T03:49:35.852594+0000","last_fullsized":"2023-07-06T03:49:35.852594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:30:51.254414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852549+0000","last_change":"2023-07-06T03:49:34.865615+0000","last_active":"2023-07-06T03:49:35.852549+0000","last_peered":"2023-07-06T03:49:35.852549+0000","last_clean":"2023-07-06T03:49:35.852549+0000","last_became_active":"2023-07-06T03:49:34.865316+0000","last_became_peered":"2023-07-06T03:49:34.865316+0000","last_unstale":"2023-07-06T03:49:35.852549+0000","last_undegraded":"2023-07-06T03:49:35.852549+0000","last_fullsized":"2023-07-06T03:49:35.852549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:00.679582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842347+0000","last_change":"2023-07-06T03:49:34.858416+0000","last_active":"2023-07-06T03:49:35.842347+0000","last_peered":"2023-07-06T03:49:35.842347+0000","last_clean":"2023-07-06T03:49:35.842347+0000","last_became_active":"2023-07-06T03:49:34.858136+0000","last_became_peered":"2023-07-06T03:49:34.858136+0000","last_unstale":"2023-07-06T03:49:35.842347+0000","last_undegraded":"2023-07-06T03:49:35.842347+0000","last_fullsized":"2023-07-06T03:49:35.842347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:55:39.894783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843042+0000","last_change":"2023-07-06T03:49:33.824414+0000","last_active":"2023-07-06T03:49:35.843042+0000","last_peered":"2023-07-06T03:49:35.843042+0000","last_clean":"2023-07-06T03:49:35.843042+0000","last_became_active":"2023-07-06T03:49:33.824113+0000","last_became_peered":"2023-07-06T03:49:33.824113+0000","last_unstale":"2023-07-06T03:49:35.843042+0000","last_undegraded":"2023-07-06T03:49:35.843042+0000","last_fullsized":"2023-07-06T03:49:35.843042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:46:42.367746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843022+0000","last_change":"2023-07-06T03:49:34.847528+0000","last_active":"2023-07-06T03:49:35.843022+0000","last_peered":"2023-07-06T03:49:35.843022+0000","last_clean":"2023-07-06T03:49:35.843022+0000","last_became_active":"2023-07-06T03:49:34.847231+0000","last_became_peered":"2023-07-06T03:49:34.847231+0000","last_unstale":"2023-07-06T03:49:35.843022+0000","last_undegraded":"2023-07-06T03:49:35.843022+0000","last_fullsized":"2023-07-06T03:49:35.843022+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:51.920331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.854060+0000","last_change":"2023-07-06T03:49:33.818932+0000","last_active":"2023-07-06T03:49:35.854060+0000","last_peered":"2023-07-06T03:49:35.854060+0000","last_clean":"2023-07-06T03:49:35.854060+0000","last_became_active":"2023-07-06T03:49:33.818659+0000","last_became_peered":"2023-07-06T03:49:33.818659+0000","last_unstale":"2023-07-06T03:49:35.854060+0000","last_undegraded":"2023-07-06T03:49:35.854060+0000","last_fullsized":"2023-07-06T03:49:35.854060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:45:15.172488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847206+0000","last_change":"2023-07-06T03:49:34.869130+0000","last_active":"2023-07-06T03:49:35.847206+0000","last_peered":"2023-07-06T03:49:35.847206+0000","last_clean":"2023-07-06T03:49:35.847206+0000","last_became_active":"2023-07-06T03:49:34.868890+0000","last_became_peered":"2023-07-06T03:49:34.868890+0000","last_unstale":"2023-07-06T03:49:35.847206+0000","last_undegraded":"2023-07-06T03:49:35.847206+0000","last_fullsized":"2023-07-06T03:49:35.847206+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:40:14.144373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852478+0000","last_change":"2023-07-06T03:49:33.830805+0000","last_active":"2023-07-06T03:49:35.852478+0000","last_peered":"2023-07-06T03:49:35.852478+0000","last_clean":"2023-07-06T03:49:35.852478+0000","last_became_active":"2023-07-06T03:49:33.830514+0000","last_became_peered":"2023-07-06T03:49:33.830514+0000","last_unstale":"2023-07-06T03:49:35.852478+0000","last_undegraded":"2023-07-06T03:49:35.852478+0000","last_fullsized":"2023-07-06T03:49:35.852478+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:56.794849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852934+0000","last_change":"2023-07-06T03:49:34.856932+0000","last_active":"2023-07-06T03:49:35.852934+0000","last_peered":"2023-07-06T03:49:35.852934+0000","last_clean":"2023-07-06T03:49:35.852934+0000","last_became_active":"2023-07-06T03:49:34.856672+0000","last_became_peered":"2023-07-06T03:49:34.856672+0000","last_unstale":"2023-07-06T03:49:35.852934+0000","last_undegraded":"2023-07-06T03:49:35.852934+0000","last_fullsized":"2023-07-06T03:49:35.852934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:32.212195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847587+0000","last_change":"2023-07-06T03:49:33.829363+0000","last_active":"2023-07-06T03:49:35.847587+0000","last_peered":"2023-07-06T03:49:35.847587+0000","last_clean":"2023-07-06T03:49:35.847587+0000","last_became_active":"2023-07-06T03:49:33.829049+0000","last_became_peered":"2023-07-06T03:49:33.829049+0000","last_unstale":"2023-07-06T03:49:35.847587+0000","last_undegraded":"2023-07-06T03:49:35.847587+0000","last_fullsized":"2023-07-06T03:49:35.847587+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:55:12.573057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844282+0000","last_change":"2023-07-06T03:49:34.866080+0000","last_active":"2023-07-06T03:49:35.844282+0000","last_peered":"2023-07-06T03:49:35.844282+0000","last_clean":"2023-07-06T03:49:35.844282+0000","last_became_active":"2023-07-06T03:49:34.865477+0000","last_became_peered":"2023-07-06T03:49:34.865477+0000","last_unstale":"2023-07-06T03:49:35.844282+0000","last_undegraded":"2023-07-06T03:49:35.844282+0000","last_fullsized":"2023-07-06T03:49:35.844282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:43.751249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847380+0000","last_change":"2023-07-06T03:49:33.831571+0000","last_active":"2023-07-06T03:49:35.847380+0000","last_peered":"2023-07-06T03:49:35.847380+0000","last_clean":"2023-07-06T03:49:35.847380+0000","last_became_active":"2023-07-06T03:49:33.831291+0000","last_became_peered":"2023-07-06T03:49:33.831291+0000","last_unstale":"2023-07-06T03:49:35.847380+0000","last_undegraded":"2023-07-06T03:49:35.847380+0000","last_fullsized":"2023-07-06T03:49:35.847380+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:09:15.546939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853139+0000","last_change":"2023-07-06T03:49:34.859640+0000","last_active":"2023-07-06T03:49:35.853139+0000","last_peered":"2023-07-06T03:49:35.853139+0000","last_clean":"2023-07-06T03:49:35.853139+0000","last_became_active":"2023-07-06T03:49:34.859348+0000","last_became_peered":"2023-07-06T03:49:34.859348+0000","last_unstale":"2023-07-06T03:49:35.853139+0000","last_undegraded":"2023-07-06T03:49:35.853139+0000","last_fullsized":"2023-07-06T03:49:35.853139+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:37:31.474862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852552+0000","last_change":"2023-07-06T03:49:33.830053+0000","last_active":"2023-07-06T03:49:35.852552+0000","last_peered":"2023-07-06T03:49:35.852552+0000","last_clean":"2023-07-06T03:49:35.852552+0000","last_became_active":"2023-07-06T03:49:33.829779+0000","last_became_peered":"2023-07-06T03:49:33.829779+0000","last_unstale":"2023-07-06T03:49:35.852552+0000","last_undegraded":"2023-07-06T03:49:35.852552+0000","last_fullsized":"2023-07-06T03:49:35.852552+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:11:13.359833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847539+0000","last_change":"2023-07-06T03:49:34.871128+0000","last_active":"2023-07-06T03:49:35.847539+0000","last_peered":"2023-07-06T03:49:35.847539+0000","last_clean":"2023-07-06T03:49:35.847539+0000","last_became_active":"2023-07-06T03:49:34.870830+0000","last_became_peered":"2023-07-06T03:49:34.870830+0000","last_unstale":"2023-07-06T03:49:35.847539+0000","last_undegraded":"2023-07-06T03:49:35.847539+0000","last_fullsized":"2023-07-06T03:49:35.847539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:35.947643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844572+0000","last_change":"2023-07-06T03:49:33.830130+0000","last_active":"2023-07-06T03:49:35.844572+0000","last_peered":"2023-07-06T03:49:35.844572+0000","last_clean":"2023-07-06T03:49:35.844572+0000","last_became_active":"2023-07-06T03:49:33.829812+0000","last_became_peered":"2023-07-06T03:49:33.829812+0000","last_unstale":"2023-07-06T03:49:35.844572+0000","last_undegraded":"2023-07-06T03:49:35.844572+0000","last_fullsized":"2023-07-06T03:49:35.844572+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:20:11.111731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.854077+0000","last_change":"2023-07-06T03:49:34.871872+0000","last_active":"2023-07-06T03:49:35.854077+0000","last_peered":"2023-07-06T03:49:35.854077+0000","last_clean":"2023-07-06T03:49:35.854077+0000","last_became_active":"2023-07-06T03:49:34.871572+0000","last_became_peered":"2023-07-06T03:49:34.871572+0000","last_unstale":"2023-07-06T03:49:35.854077+0000","last_undegraded":"2023-07-06T03:49:35.854077+0000","last_fullsized":"2023-07-06T03:49:35.854077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:24:48.045223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852703+0000","last_change":"2023-07-06T03:49:33.824719+0000","last_active":"2023-07-06T03:49:35.852703+0000","last_peered":"2023-07-06T03:49:35.852703+0000","last_clean":"2023-07-06T03:49:35.852703+0000","last_became_active":"2023-07-06T03:49:33.824457+0000","last_became_peered":"2023-07-06T03:49:33.824457+0000","last_unstale":"2023-07-06T03:49:35.852703+0000","last_undegraded":"2023-07-06T03:49:35.852703+0000","last_fullsized":"2023-07-06T03:49:35.852703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:52:38.585500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852504+0000","last_change":"2023-07-06T03:49:34.862528+0000","last_active":"2023-07-06T03:49:35.852504+0000","last_peered":"2023-07-06T03:49:35.852504+0000","last_clean":"2023-07-06T03:49:35.852504+0000","last_became_active":"2023-07-06T03:49:34.862088+0000","last_became_peered":"2023-07-06T03:49:34.862088+0000","last_unstale":"2023-07-06T03:49:35.852504+0000","last_undegraded":"2023-07-06T03:49:35.852504+0000","last_fullsized":"2023-07-06T03:49:35.852504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:26:37.844936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853622+0000","last_change":"2023-07-06T03:49:33.821887+0000","last_active":"2023-07-06T03:49:35.853622+0000","last_peered":"2023-07-06T03:49:35.853622+0000","last_clean":"2023-07-06T03:49:35.853622+0000","last_became_active":"2023-07-06T03:49:33.821606+0000","last_became_peered":"2023-07-06T03:49:33.821606+0000","last_unstale":"2023-07-06T03:49:35.853622+0000","last_undegraded":"2023-07-06T03:49:35.853622+0000","last_fullsized":"2023-07-06T03:49:35.853622+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:34.545522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847222+0000","last_change":"2023-07-06T03:49:34.862631+0000","last_active":"2023-07-06T03:49:35.847222+0000","last_peered":"2023-07-06T03:49:35.847222+0000","last_clean":"2023-07-06T03:49:35.847222+0000","last_became_active":"2023-07-06T03:49:34.862358+0000","last_became_peered":"2023-07-06T03:49:34.862358+0000","last_unstale":"2023-07-06T03:49:35.847222+0000","last_undegraded":"2023-07-06T03:49:35.847222+0000","last_fullsized":"2023-07-06T03:49:35.847222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:10.128139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852661+0000","last_change":"2023-07-06T03:49:33.823032+0000","last_active":"2023-07-06T03:49:35.852661+0000","last_peered":"2023-07-06T03:49:35.852661+0000","last_clean":"2023-07-06T03:49:35.852661+0000","last_became_active":"2023-07-06T03:49:33.822768+0000","last_became_peered":"2023-07-06T03:49:33.822768+0000","last_unstale":"2023-07-06T03:49:35.852661+0000","last_undegraded":"2023-07-06T03:49:35.852661+0000","last_fullsized":"2023-07-06T03:49:35.852661+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:14:20.388073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843813+0000","last_change":"2023-07-06T03:49:34.865928+0000","last_active":"2023-07-06T03:49:35.843813+0000","last_peered":"2023-07-06T03:49:35.843813+0000","last_clean":"2023-07-06T03:49:35.843813+0000","last_became_active":"2023-07-06T03:49:34.865408+0000","last_became_peered":"2023-07-06T03:49:34.865408+0000","last_unstale":"2023-07-06T03:49:35.843813+0000","last_undegraded":"2023-07-06T03:49:35.843813+0000","last_fullsized":"2023-07-06T03:49:35.843813+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:34:27.019155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844716+0000","last_change":"2023-07-06T03:49:33.827783+0000","last_active":"2023-07-06T03:49:35.844716+0000","last_peered":"2023-07-06T03:49:35.844716+0000","last_clean":"2023-07-06T03:49:35.844716+0000","last_became_active":"2023-07-06T03:49:33.827520+0000","last_became_peered":"2023-07-06T03:49:33.827520+0000","last_unstale":"2023-07-06T03:49:35.844716+0000","last_undegraded":"2023-07-06T03:49:35.844716+0000","last_fullsized":"2023-07-06T03:49:35.844716+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:20:35.716810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844497+0000","last_change":"2023-07-06T03:49:34.864553+0000","last_active":"2023-07-06T03:49:35.844497+0000","last_peered":"2023-07-06T03:49:35.844497+0000","last_clean":"2023-07-06T03:49:35.844497+0000","last_became_active":"2023-07-06T03:49:34.864200+0000","last_became_peered":"2023-07-06T03:49:34.864200+0000","last_unstale":"2023-07-06T03:49:35.844497+0000","last_undegraded":"2023-07-06T03:49:35.844497+0000","last_fullsized":"2023-07-06T03:49:35.844497+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:14:16.181864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853973+0000","last_change":"2023-07-06T03:49:33.821142+0000","last_active":"2023-07-06T03:49:35.853973+0000","last_peered":"2023-07-06T03:49:35.853973+0000","last_clean":"2023-07-06T03:49:35.853973+0000","last_became_active":"2023-07-06T03:49:33.820618+0000","last_became_peered":"2023-07-06T03:49:33.820618+0000","last_unstale":"2023-07-06T03:49:35.853973+0000","last_undegraded":"2023-07-06T03:49:35.853973+0000","last_fullsized":"2023-07-06T03:49:35.853973+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:34:48.683822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853265+0000","last_change":"2023-07-06T03:49:34.865809+0000","last_active":"2023-07-06T03:49:35.853265+0000","last_peered":"2023-07-06T03:49:35.853265+0000","last_clean":"2023-07-06T03:49:35.853265+0000","last_became_active":"2023-07-06T03:49:34.865529+0000","last_became_peered":"2023-07-06T03:49:34.865529+0000","last_unstale":"2023-07-06T03:49:35.853265+0000","last_undegraded":"2023-07-06T03:49:35.853265+0000","last_fullsized":"2023-07-06T03:49:35.853265+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:44:40.247032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852919+0000","last_change":"2023-07-06T03:49:33.818687+0000","last_active":"2023-07-06T03:49:35.852919+0000","last_peered":"2023-07-06T03:49:35.852919+0000","last_clean":"2023-07-06T03:49:35.852919+0000","last_became_active":"2023-07-06T03:49:33.818371+0000","last_became_peered":"2023-07-06T03:49:33.818371+0000","last_unstale":"2023-07-06T03:49:35.852919+0000","last_undegraded":"2023-07-06T03:49:35.852919+0000","last_fullsized":"2023-07-06T03:49:35.852919+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:17:57.883769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852349+0000","last_change":"2023-07-06T03:49:34.872706+0000","last_active":"2023-07-06T03:49:35.852349+0000","last_peered":"2023-07-06T03:49:35.852349+0000","last_clean":"2023-07-06T03:49:35.852349+0000","last_became_active":"2023-07-06T03:49:34.872373+0000","last_became_peered":"2023-07-06T03:49:34.872373+0000","last_unstale":"2023-07-06T03:49:35.852349+0000","last_undegraded":"2023-07-06T03:49:35.852349+0000","last_fullsized":"2023-07-06T03:49:35.852349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:40:19.464216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847378+0000","last_change":"2023-07-06T03:49:33.834564+0000","last_active":"2023-07-06T03:49:35.847378+0000","last_peered":"2023-07-06T03:49:35.847378+0000","last_clean":"2023-07-06T03:49:35.847378+0000","last_became_active":"2023-07-06T03:49:33.834288+0000","last_became_peered":"2023-07-06T03:49:33.834288+0000","last_unstale":"2023-07-06T03:49:35.847378+0000","last_undegraded":"2023-07-06T03:49:35.847378+0000","last_fullsized":"2023-07-06T03:49:35.847378+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:37:30.472139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847555+0000","last_change":"2023-07-06T03:49:34.869775+0000","last_active":"2023-07-06T03:49:35.847555+0000","last_peered":"2023-07-06T03:49:35.847555+0000","last_clean":"2023-07-06T03:49:35.847555+0000","last_became_active":"2023-07-06T03:49:34.869513+0000","last_became_peered":"2023-07-06T03:49:34.869513+0000","last_unstale":"2023-07-06T03:49:35.847555+0000","last_undegraded":"2023-07-06T03:49:35.847555+0000","last_fullsized":"2023-07-06T03:49:35.847555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:09:57.398183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842918+0000","last_change":"2023-07-06T03:49:33.824608+0000","last_active":"2023-07-06T03:49:35.842918+0000","last_peered":"2023-07-06T03:49:35.842918+0000","last_clean":"2023-07-06T03:49:35.842918+0000","last_became_active":"2023-07-06T03:49:33.824272+0000","last_became_peered":"2023-07-06T03:49:33.824272+0000","last_unstale":"2023-07-06T03:49:35.842918+0000","last_undegraded":"2023-07-06T03:49:35.842918+0000","last_fullsized":"2023-07-06T03:49:35.842918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:27:59.817202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852852+0000","last_change":"2023-07-06T03:49:34.858352+0000","last_active":"2023-07-06T03:49:35.852852+0000","last_peered":"2023-07-06T03:49:35.852852+0000","last_clean":"2023-07-06T03:49:35.852852+0000","last_became_active":"2023-07-06T03:49:34.858116+0000","last_became_peered":"2023-07-06T03:49:34.858116+0000","last_unstale":"2023-07-06T03:49:35.852852+0000","last_undegraded":"2023-07-06T03:49:35.852852+0000","last_fullsized":"2023-07-06T03:49:35.852852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:46:17.272577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844538+0000","last_change":"2023-07-06T03:49:33.833266+0000","last_active":"2023-07-06T03:49:35.844538+0000","last_peered":"2023-07-06T03:49:35.844538+0000","last_clean":"2023-07-06T03:49:35.844538+0000","last_became_active":"2023-07-06T03:49:33.832703+0000","last_became_peered":"2023-07-06T03:49:33.832703+0000","last_unstale":"2023-07-06T03:49:35.844538+0000","last_undegraded":"2023-07-06T03:49:35.844538+0000","last_fullsized":"2023-07-06T03:49:35.844538+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:47:49.443158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852687+0000","last_change":"2023-07-06T03:49:34.863482+0000","last_active":"2023-07-06T03:49:35.852687+0000","last_peered":"2023-07-06T03:49:35.852687+0000","last_clean":"2023-07-06T03:49:35.852687+0000","last_became_active":"2023-07-06T03:49:34.863127+0000","last_became_peered":"2023-07-06T03:49:34.863127+0000","last_unstale":"2023-07-06T03:49:35.852687+0000","last_undegraded":"2023-07-06T03:49:35.852687+0000","last_fullsized":"2023-07-06T03:49:35.852687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:16.291086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852973+0000","last_change":"2023-07-06T03:49:33.820353+0000","last_active":"2023-07-06T03:49:35.852973+0000","last_peered":"2023-07-06T03:49:35.852973+0000","last_clean":"2023-07-06T03:49:35.852973+0000","last_became_active":"2023-07-06T03:49:33.820031+0000","last_became_peered":"2023-07-06T03:49:33.820031+0000","last_unstale":"2023-07-06T03:49:35.852973+0000","last_undegraded":"2023-07-06T03:49:35.852973+0000","last_fullsized":"2023-07-06T03:49:35.852973+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:18:43.350865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852758+0000","last_change":"2023-07-06T03:49:34.869378+0000","last_active":"2023-07-06T03:49:35.852758+0000","last_peered":"2023-07-06T03:49:35.852758+0000","last_clean":"2023-07-06T03:49:35.852758+0000","last_became_active":"2023-07-06T03:49:34.869133+0000","last_became_peered":"2023-07-06T03:49:34.869133+0000","last_unstale":"2023-07-06T03:49:35.852758+0000","last_undegraded":"2023-07-06T03:49:35.852758+0000","last_fullsized":"2023-07-06T03:49:35.852758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:49:08.018890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844045+0000","last_change":"2023-07-06T03:49:33.828309+0000","last_active":"2023-07-06T03:49:35.844045+0000","last_peered":"2023-07-06T03:49:35.844045+0000","last_clean":"2023-07-06T03:49:35.844045+0000","last_became_active":"2023-07-06T03:49:33.828026+0000","last_became_peered":"2023-07-06T03:49:33.828026+0000","last_unstale":"2023-07-06T03:49:35.844045+0000","last_undegraded":"2023-07-06T03:49:35.844045+0000","last_fullsized":"2023-07-06T03:49:35.844045+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:29.056482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847226+0000","last_change":"2023-07-06T03:49:34.865081+0000","last_active":"2023-07-06T03:49:35.847226+0000","last_peered":"2023-07-06T03:49:35.847226+0000","last_clean":"2023-07-06T03:49:35.847226+0000","last_became_active":"2023-07-06T03:49:34.864678+0000","last_became_peered":"2023-07-06T03:49:34.864678+0000","last_unstale":"2023-07-06T03:49:35.847226+0000","last_undegraded":"2023-07-06T03:49:35.847226+0000","last_fullsized":"2023-07-06T03:49:35.847226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:36:40.499610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852613+0000","last_change":"2023-07-06T03:49:33.829210+0000","last_active":"2023-07-06T03:49:35.852613+0000","last_peered":"2023-07-06T03:49:35.852613+0000","last_clean":"2023-07-06T03:49:35.852613+0000","last_became_active":"2023-07-06T03:49:33.828914+0000","last_became_peered":"2023-07-06T03:49:33.828914+0000","last_unstale":"2023-07-06T03:49:35.852613+0000","last_undegraded":"2023-07-06T03:49:35.852613+0000","last_fullsized":"2023-07-06T03:49:35.852613+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:30:20.604178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843921+0000","last_change":"2023-07-06T03:49:34.862597+0000","last_active":"2023-07-06T03:49:35.843921+0000","last_peered":"2023-07-06T03:49:35.843921+0000","last_clean":"2023-07-06T03:49:35.843921+0000","last_became_active":"2023-07-06T03:49:34.862332+0000","last_became_peered":"2023-07-06T03:49:34.862332+0000","last_unstale":"2023-07-06T03:49:35.843921+0000","last_undegraded":"2023-07-06T03:49:35.843921+0000","last_fullsized":"2023-07-06T03:49:35.843921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:02:06.863468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853311+0000","last_change":"2023-07-06T03:49:33.832386+0000","last_active":"2023-07-06T03:49:35.853311+0000","last_peered":"2023-07-06T03:49:35.853311+0000","last_clean":"2023-07-06T03:49:35.853311+0000","last_became_active":"2023-07-06T03:49:33.832095+0000","last_became_peered":"2023-07-06T03:49:33.832095+0000","last_unstale":"2023-07-06T03:49:35.853311+0000","last_undegraded":"2023-07-06T03:49:35.853311+0000","last_fullsized":"2023-07-06T03:49:35.853311+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:39:55.076647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847367+0000","last_change":"2023-07-06T03:49:34.871135+0000","last_active":"2023-07-06T03:49:35.847367+0000","last_peered":"2023-07-06T03:49:35.847367+0000","last_clean":"2023-07-06T03:49:35.847367+0000","last_became_active":"2023-07-06T03:49:34.870875+0000","last_became_peered":"2023-07-06T03:49:34.870875+0000","last_unstale":"2023-07-06T03:49:35.847367+0000","last_undegraded":"2023-07-06T03:49:35.847367+0000","last_fullsized":"2023-07-06T03:49:35.847367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:52:46.294045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852524+0000","last_change":"2023-07-06T03:49:33.823501+0000","last_active":"2023-07-06T03:49:35.852524+0000","last_peered":"2023-07-06T03:49:35.852524+0000","last_clean":"2023-07-06T03:49:35.852524+0000","last_became_active":"2023-07-06T03:49:33.823222+0000","last_became_peered":"2023-07-06T03:49:33.823222+0000","last_unstale":"2023-07-06T03:49:35.852524+0000","last_undegraded":"2023-07-06T03:49:35.852524+0000","last_fullsized":"2023-07-06T03:49:35.852524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:13:13.043537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853114+0000","last_change":"2023-07-06T03:49:34.859911+0000","last_active":"2023-07-06T03:49:35.853114+0000","last_peered":"2023-07-06T03:49:35.853114+0000","last_clean":"2023-07-06T03:49:35.853114+0000","last_became_active":"2023-07-06T03:49:34.859664+0000","last_became_peered":"2023-07-06T03:49:34.859664+0000","last_unstale":"2023-07-06T03:49:35.853114+0000","last_undegraded":"2023-07-06T03:49:35.853114+0000","last_fullsized":"2023-07-06T03:49:35.853114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:54.861404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844892+0000","last_change":"2023-07-06T03:49:33.830049+0000","last_active":"2023-07-06T03:49:35.844892+0000","last_peered":"2023-07-06T03:49:35.844892+0000","last_clean":"2023-07-06T03:49:35.844892+0000","last_became_active":"2023-07-06T03:49:33.829724+0000","last_became_peered":"2023-07-06T03:49:33.829724+0000","last_unstale":"2023-07-06T03:49:35.844892+0000","last_undegraded":"2023-07-06T03:49:35.844892+0000","last_fullsized":"2023-07-06T03:49:35.844892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:44:26.217867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853133+0000","last_change":"2023-07-06T03:49:34.860503+0000","last_active":"2023-07-06T03:49:35.853133+0000","last_peered":"2023-07-06T03:49:35.853133+0000","last_clean":"2023-07-06T03:49:35.853133+0000","last_became_active":"2023-07-06T03:49:34.860214+0000","last_became_peered":"2023-07-06T03:49:34.860214+0000","last_unstale":"2023-07-06T03:49:35.853133+0000","last_undegraded":"2023-07-06T03:49:35.853133+0000","last_fullsized":"2023-07-06T03:49:35.853133+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:52:46.294045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852643+0000","last_change":"2023-07-06T03:49:33.822068+0000","last_active":"2023-07-06T03:49:35.852643+0000","last_peered":"2023-07-06T03:49:35.852643+0000","last_clean":"2023-07-06T03:49:35.852643+0000","last_became_active":"2023-07-06T03:49:33.821647+0000","last_became_peered":"2023-07-06T03:49:33.821647+0000","last_unstale":"2023-07-06T03:49:35.852643+0000","last_undegraded":"2023-07-06T03:49:35.852643+0000","last_fullsized":"2023-07-06T03:49:35.852643+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:53:54.322399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842723+0000","last_change":"2023-07-06T03:49:34.861916+0000","last_active":"2023-07-06T03:49:35.842723+0000","last_peered":"2023-07-06T03:49:35.842723+0000","last_clean":"2023-07-06T03:49:35.842723+0000","last_became_active":"2023-07-06T03:49:34.861637+0000","last_became_peered":"2023-07-06T03:49:34.861637+0000","last_unstale":"2023-07-06T03:49:35.842723+0000","last_undegraded":"2023-07-06T03:49:35.842723+0000","last_fullsized":"2023-07-06T03:49:35.842723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:47:04.357163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853138+0000","last_change":"2023-07-06T03:49:33.827584+0000","last_active":"2023-07-06T03:49:35.853138+0000","last_peered":"2023-07-06T03:49:35.853138+0000","last_clean":"2023-07-06T03:49:35.853138+0000","last_became_active":"2023-07-06T03:49:33.827258+0000","last_became_peered":"2023-07-06T03:49:33.827258+0000","last_unstale":"2023-07-06T03:49:35.853138+0000","last_undegraded":"2023-07-06T03:49:35.853138+0000","last_fullsized":"2023-07-06T03:49:35.853138+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:09.576964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844122+0000","last_change":"2023-07-06T03:49:34.867876+0000","last_active":"2023-07-06T03:49:35.844122+0000","last_peered":"2023-07-06T03:49:35.844122+0000","last_clean":"2023-07-06T03:49:35.844122+0000","last_became_active":"2023-07-06T03:49:34.867596+0000","last_became_peered":"2023-07-06T03:49:34.867596+0000","last_unstale":"2023-07-06T03:49:35.844122+0000","last_undegraded":"2023-07-06T03:49:35.844122+0000","last_fullsized":"2023-07-06T03:49:35.844122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:03.287815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842714+0000","last_change":"2023-07-06T03:49:33.823149+0000","last_active":"2023-07-06T03:49:35.842714+0000","last_peered":"2023-07-06T03:49:35.842714+0000","last_clean":"2023-07-06T03:49:35.842714+0000","last_became_active":"2023-07-06T03:49:33.821412+0000","last_became_peered":"2023-07-06T03:49:33.821412+0000","last_unstale":"2023-07-06T03:49:35.842714+0000","last_undegraded":"2023-07-06T03:49:35.842714+0000","last_fullsized":"2023-07-06T03:49:35.842714+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:55:50.863561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844361+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:49:35.844361+0000","last_peered":"2023-07-06T03:49:35.844361+0000","last_clean":"2023-07-06T03:49:35.844361+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:49:35.844361+0000","last_undegraded":"2023-07-06T03:49:35.844361+0000","last_fullsized":"2023-07-06T03:49:35.844361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:51:30.705000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842826+0000","last_change":"2023-07-06T03:49:34.870611+0000","last_active":"2023-07-06T03:49:35.842826+0000","last_peered":"2023-07-06T03:49:35.842826+0000","last_clean":"2023-07-06T03:49:35.842826+0000","last_became_active":"2023-07-06T03:49:34.870083+0000","last_became_peered":"2023-07-06T03:49:34.870083+0000","last_unstale":"2023-07-06T03:49:35.842826+0000","last_undegraded":"2023-07-06T03:49:35.842826+0000","last_fullsized":"2023-07-06T03:49:35.842826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:26.019247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852890+0000","last_change":"2023-07-06T03:49:33.825662+0000","last_active":"2023-07-06T03:49:35.852890+0000","last_peered":"2023-07-06T03:49:35.852890+0000","last_clean":"2023-07-06T03:49:35.852890+0000","last_became_active":"2023-07-06T03:49:33.825400+0000","last_became_peered":"2023-07-06T03:49:33.825400+0000","last_unstale":"2023-07-06T03:49:35.852890+0000","last_undegraded":"2023-07-06T03:49:35.852890+0000","last_fullsized":"2023-07-06T03:49:35.852890+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:22:15.733559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852642+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:49:35.852642+0000","last_peered":"2023-07-06T03:49:35.852642+0000","last_clean":"2023-07-06T03:49:35.852642+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:49:35.852642+0000","last_undegraded":"2023-07-06T03:49:35.852642+0000","last_fullsized":"2023-07-06T03:49:35.852642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:01:24.805927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.854116+0000","last_change":"2023-07-06T03:49:34.870290+0000","last_active":"2023-07-06T03:49:35.854116+0000","last_peered":"2023-07-06T03:49:35.854116+0000","last_clean":"2023-07-06T03:49:35.854116+0000","last_became_active":"2023-07-06T03:49:34.870012+0000","last_became_peered":"2023-07-06T03:49:34.870012+0000","last_unstale":"2023-07-06T03:49:35.854116+0000","last_undegraded":"2023-07-06T03:49:35.854116+0000","last_fullsized":"2023-07-06T03:49:35.854116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:35:58.635737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852836+0000","last_change":"2023-07-06T03:49:33.830269+0000","last_active":"2023-07-06T03:49:35.852836+0000","last_peered":"2023-07-06T03:49:35.852836+0000","last_clean":"2023-07-06T03:49:35.852836+0000","last_became_active":"2023-07-06T03:49:33.829954+0000","last_became_peered":"2023-07-06T03:49:33.829954+0000","last_unstale":"2023-07-06T03:49:35.852836+0000","last_undegraded":"2023-07-06T03:49:35.852836+0000","last_fullsized":"2023-07-06T03:49:35.852836+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:28:41.278256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843005+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:49:35.843005+0000","last_peered":"2023-07-06T03:49:35.843005+0000","last_clean":"2023-07-06T03:49:35.843005+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:49:35.843005+0000","last_undegraded":"2023-07-06T03:49:35.843005+0000","last_fullsized":"2023-07-06T03:49:35.843005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:56:14.059897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853691+0000","last_change":"2023-07-06T03:49:34.871495+0000","last_active":"2023-07-06T03:49:35.853691+0000","last_peered":"2023-07-06T03:49:35.853691+0000","last_clean":"2023-07-06T03:49:35.853691+0000","last_became_active":"2023-07-06T03:49:34.871202+0000","last_became_peered":"2023-07-06T03:49:34.871202+0000","last_unstale":"2023-07-06T03:49:35.853691+0000","last_undegraded":"2023-07-06T03:49:35.853691+0000","last_fullsized":"2023-07-06T03:49:35.853691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:18:04.906859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844868+0000","last_change":"2023-07-06T03:49:33.828898+0000","last_active":"2023-07-06T03:49:35.844868+0000","last_peered":"2023-07-06T03:49:35.844868+0000","last_clean":"2023-07-06T03:49:35.844868+0000","last_became_active":"2023-07-06T03:49:33.828646+0000","last_became_peered":"2023-07-06T03:49:33.828646+0000","last_unstale":"2023-07-06T03:49:35.844868+0000","last_undegraded":"2023-07-06T03:49:35.844868+0000","last_fullsized":"2023-07-06T03:49:35.844868+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:50:44.305762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853466+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:49:35.853466+0000","last_peered":"2023-07-06T03:49:35.853466+0000","last_clean":"2023-07-06T03:49:35.853466+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:49:35.853466+0000","last_undegraded":"2023-07-06T03:49:35.853466+0000","last_fullsized":"2023-07-06T03:49:35.853466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:36:08.075483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.851969+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:49:35.851969+0000","last_peered":"2023-07-06T03:49:35.851969+0000","last_clean":"2023-07-06T03:49:35.851969+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:49:35.851969+0000","last_undegraded":"2023-07-06T03:49:35.851969+0000","last_fullsized":"2023-07-06T03:49:35.851969+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:46:08.582009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843084+0000","last_change":"2023-07-06T03:49:34.870804+0000","last_active":"2023-07-06T03:49:35.843084+0000","last_peered":"2023-07-06T03:49:35.843084+0000","last_clean":"2023-07-06T03:49:35.843084+0000","last_became_active":"2023-07-06T03:49:34.870475+0000","last_became_peered":"2023-07-06T03:49:34.870475+0000","last_unstale":"2023-07-06T03:49:35.843084+0000","last_undegraded":"2023-07-06T03:49:35.843084+0000","last_fullsized":"2023-07-06T03:49:35.843084+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:35.842450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844670+0000","last_change":"2023-07-06T03:49:33.833602+0000","last_active":"2023-07-06T03:49:35.844670+0000","last_peered":"2023-07-06T03:49:35.844670+0000","last_clean":"2023-07-06T03:49:35.844670+0000","last_became_active":"2023-07-06T03:49:33.833331+0000","last_became_peered":"2023-07-06T03:49:33.833331+0000","last_unstale":"2023-07-06T03:49:35.844670+0000","last_undegraded":"2023-07-06T03:49:35.844670+0000","last_fullsized":"2023-07-06T03:49:35.844670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:35:43.748856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853042+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:49:35.853042+0000","last_peered":"2023-07-06T03:49:35.853042+0000","last_clean":"2023-07-06T03:49:35.853042+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:49:35.853042+0000","last_undegraded":"2023-07-06T03:49:35.853042+0000","last_fullsized":"2023-07-06T03:49:35.853042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:00:27.596531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842884+0000","last_change":"2023-07-06T03:49:34.856943+0000","last_active":"2023-07-06T03:49:35.842884+0000","last_peered":"2023-07-06T03:49:35.842884+0000","last_clean":"2023-07-06T03:49:35.842884+0000","last_became_active":"2023-07-06T03:49:34.856631+0000","last_became_peered":"2023-07-06T03:49:34.856631+0000","last_unstale":"2023-07-06T03:49:35.842884+0000","last_undegraded":"2023-07-06T03:49:35.842884+0000","last_fullsized":"2023-07-06T03:49:35.842884+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:10:03.558670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842893+0000","last_change":"2023-07-06T03:49:33.833448+0000","last_active":"2023-07-06T03:49:35.842893+0000","last_peered":"2023-07-06T03:49:35.842893+0000","last_clean":"2023-07-06T03:49:35.842893+0000","last_became_active":"2023-07-06T03:49:33.833163+0000","last_became_peered":"2023-07-06T03:49:33.833163+0000","last_unstale":"2023-07-06T03:49:35.842893+0000","last_undegraded":"2023-07-06T03:49:35.842893+0000","last_fullsized":"2023-07-06T03:49:35.842893+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:29.056482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853850+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:49:35.853850+0000","last_peered":"2023-07-06T03:49:35.853850+0000","last_clean":"2023-07-06T03:49:35.853850+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:49:35.853850+0000","last_undegraded":"2023-07-06T03:49:35.853850+0000","last_fullsized":"2023-07-06T03:49:35.853850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:43.221211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842376+0000","last_change":"2023-07-06T03:49:34.851758+0000","last_active":"2023-07-06T03:49:35.842376+0000","last_peered":"2023-07-06T03:49:35.842376+0000","last_clean":"2023-07-06T03:49:35.842376+0000","last_became_active":"2023-07-06T03:49:34.851454+0000","last_became_peered":"2023-07-06T03:49:34.851454+0000","last_unstale":"2023-07-06T03:49:35.842376+0000","last_undegraded":"2023-07-06T03:49:35.842376+0000","last_fullsized":"2023-07-06T03:49:35.842376+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:57:46.131361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853495+0000","last_change":"2023-07-06T03:49:33.830097+0000","last_active":"2023-07-06T03:49:35.853495+0000","last_peered":"2023-07-06T03:49:35.853495+0000","last_clean":"2023-07-06T03:49:35.853495+0000","last_became_active":"2023-07-06T03:49:33.829804+0000","last_became_peered":"2023-07-06T03:49:33.829804+0000","last_unstale":"2023-07-06T03:49:35.853495+0000","last_undegraded":"2023-07-06T03:49:35.853495+0000","last_fullsized":"2023-07-06T03:49:35.853495+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:33:21.962429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844202+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:49:35.844202+0000","last_peered":"2023-07-06T03:49:35.844202+0000","last_clean":"2023-07-06T03:49:35.844202+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:49:35.844202+0000","last_undegraded":"2023-07-06T03:49:35.844202+0000","last_fullsized":"2023-07-06T03:49:35.844202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:34:27.801302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852957+0000","last_change":"2023-07-06T03:49:34.853054+0000","last_active":"2023-07-06T03:49:35.852957+0000","last_peered":"2023-07-06T03:49:35.852957+0000","last_clean":"2023-07-06T03:49:35.852957+0000","last_became_active":"2023-07-06T03:49:34.852779+0000","last_became_peered":"2023-07-06T03:49:34.852779+0000","last_unstale":"2023-07-06T03:49:35.852957+0000","last_undegraded":"2023-07-06T03:49:35.852957+0000","last_fullsized":"2023-07-06T03:49:35.852957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:40:14.144373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853361+0000","last_change":"2023-07-06T03:49:33.833515+0000","last_active":"2023-07-06T03:49:35.853361+0000","last_peered":"2023-07-06T03:49:35.853361+0000","last_clean":"2023-07-06T03:49:35.853361+0000","last_became_active":"2023-07-06T03:49:33.833183+0000","last_became_peered":"2023-07-06T03:49:33.833183+0000","last_unstale":"2023-07-06T03:49:35.853361+0000","last_undegraded":"2023-07-06T03:49:35.853361+0000","last_fullsized":"2023-07-06T03:49:35.853361+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:58.747527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.854147+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:49:35.854147+0000","last_peered":"2023-07-06T03:49:35.854147+0000","last_clean":"2023-07-06T03:49:35.854147+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:49:35.854147+0000","last_undegraded":"2023-07-06T03:49:35.854147+0000","last_fullsized":"2023-07-06T03:49:35.854147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:11:38.112030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":897024,"data_stored":398800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69199999999999995,"15min":0.72199999999999998},"min":{"1min":0.32500000000000001,"5min":0.307,"15min":0.307},"max":{"1min":4.0570000000000004,"5min":4.0570000000000004,"15min":4.0570000000000004},"last":0.626},{"interface":"front","average":{"1min":0.63,"5min":0.72799999999999998,"15min":0.80800000000000005},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.95499999999999996,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.378}]},{"osd":1,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69699999999999995,"15min":0.746},"min":{"1min":0.45500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91300000000000003,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.92300000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.0389999999999997,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.505}]},{"osd":2,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.74099999999999999,"15min":0.81899999999999995},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.878,"5min":3.77,"15min":3.77},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92000000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.72799999999999998},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.95999999999999996,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.439,"5min":0.433,"15min":0.433},"max":{"1min":0.94199999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69099999999999995,"15min":0.67600000000000005},"min":{"1min":0.47799999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.748,"15min":0.746},"min":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.03,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.006,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.68400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":897024,"data_stored":398820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63800000000000001,"15min":0.65000000000000002},"min":{"1min":0.30599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94099999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.88},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.94899999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.64400000000000002,"15min":0.66600000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64300000000000002,"15min":0.67400000000000004},"min":{"1min":0.46400000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96399999999999997,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.90800000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63200000000000001,"15min":0.65200000000000002},"min":{"1min":0.45000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88800000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.67600000000000005,"15min":0.71299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":1.119,"15min":1.119},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.70499999999999996,"15min":0.76600000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85499999999999998,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69499999999999995,"15min":0.73999999999999999},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.91100000000000003,"5min":1.093,"15min":1.093},"last":0.89500000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69599999999999995,"15min":0.72599999999999998},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98299999999999998,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70499999999999996,"15min":0.751},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86799999999999999,"5min":1.085,"15min":1.085},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.67800000000000005,"15min":0.72999999999999998},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93700000000000006,"5min":1.218,"15min":1.218},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.75700000000000001,"15min":0.80400000000000005},"min":{"1min":0.46899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95099999999999996,"5min":1.198,"15min":1.198},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.78300000000000003,"15min":0.84699999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.054,"5min":1.323,"15min":1.323},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78400000000000003,"15min":0.85499999999999998},"min":{"1min":0.504,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0009999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1511424,"data_stored":996767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.47999999999999998,"15min":0.56999999999999995},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.002,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.52200000000000002,"15min":0.58699999999999997},"min":{"1min":0.31900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.65000000000000002,"5min":1.96,"15min":1.96},"last":0.50900000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.72899999999999998,"15min":1.0209999999999999},"min":{"1min":0.29399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70799999999999996,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.57899999999999996,"15min":0.65900000000000003},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.92400000000000004,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.56399999999999995,"15min":0.63300000000000001},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.96799999999999997,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.51000000000000001,"15min":0.56000000000000005},"min":{"1min":0.34599999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60199999999999998,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.99099999999999999,"15min":1.2789999999999999},"min":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0640000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.78900000000000003,"15min":0.84299999999999997},"min":{"1min":0.54800000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0800000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73699999999999999,"15min":0.78500000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86099999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.99099999999999999,"15min":1.2929999999999999},"min":{"1min":0.44700000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1020000000000001,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77600000000000002,"15min":0.82999999999999996},"min":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.83899999999999997,"5min":1.365,"15min":1.365},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.0329999999999999,"15min":1.3360000000000001},"min":{"1min":0.48699999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91600000000000004,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77700000000000002,"15min":0.85099999999999998},"min":{"1min":0.51200000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0409999999999999,"5min":2.806,"15min":2.806},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.77700000000000002,"15min":0.82799999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.88700000000000001,"5min":1.337,"15min":1.337},"last":0.84999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31932,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339908,"statfs":{"total":96636764160,"available":96604065792,"internally_reserved":0,"allocated":905216,"data_stored":404823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40200000000000002,"15min":0.41299999999999998},"min":{"1min":0.26400000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54600000000000004,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42899999999999999,"15min":0.436},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.60399999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.377}]},{"osd":2,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.47599999999999998},"min":{"1min":0.33000000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.66700000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.432},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48699999999999999,"15min":0.505},"min":{"1min":0.32700000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.60399999999999998,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.30099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71999999999999997,"15min":0.751},"min":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86899999999999999,"5min":1.171,"15min":1.171},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.99099999999999999,"5min":1.034,"15min":1.034},"last":0.67100000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.47799999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.02,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.72499999999999998,"15min":0.754},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92100000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72799999999999998,"15min":0.752},"min":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.86399999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.006,"5min":1.546,"15min":1.546},"last":0.74099999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77900000000000003,"15min":0.80800000000000005},"min":{"1min":0.56899999999999995,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.93300000000000005,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.93600000000000005,"15min":1.1559999999999999},"min":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.94999999999999996,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36012,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335828,"statfs":{"total":96636764160,"available":96599887872,"internally_reserved":0,"allocated":888832,"data_stored":398148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.371}]},{"osd":1,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48299999999999998,"15min":0.49199999999999999},"min":{"1min":0.33500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.68100000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.5},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91300000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77400000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.46400000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71699999999999997,"15min":0.70199999999999996},"min":{"1min":0.50800000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.075,"5min":1.165,"15min":1.165},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73199999999999998,"15min":0.72799999999999998},"min":{"1min":0.56799999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96499999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.754,"15min":0.754},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0549999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71999999999999997,"15min":0.70799999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94799999999999995,"5min":1.046,"15min":1.046},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.55000000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.012,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.79600000000000004,"15min":0.82999999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.087,"5min":2.157,"15min":2.157},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.77900000000000003,"15min":0.79600000000000004},"min":{"1min":0.56000000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0600000000000001,"5min":1.966,"15min":1.966},"last":0.73699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":909312,"data_stored":399816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.52600000000000002,"15min":0.65400000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48999999999999999,"15min":0.51500000000000001},"min":{"1min":0.33600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63800000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.38}]},{"osd":1,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.80500000000000005,"15min":1.196},"min":{"1min":0.29299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":15.911,"15min":15.911},"last":0.311},{"interface":"front","average":{"1min":0.501,"5min":0.876,"15min":1.266},"min":{"1min":0.36299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69099999999999995,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.49199999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.46700000000000003,"15min":0.50900000000000001},"min":{"1min":0.32800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57699999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.878,"15min":1.27},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.55700000000000005,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.071,"5min":1.179,"15min":1.179},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.83999999999999997,"15min":0.98199999999999998},"min":{"1min":0.46100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.085,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.84299999999999997,"15min":0.98099999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1080000000000001,"5min":5.077,"15min":5.077},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.88500000000000001,"15min":1.042},"min":{"1min":0.499,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95199999999999996,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.75700000000000001,"15min":0.81499999999999995},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94399999999999995,"5min":1.143,"15min":1.143},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.84899999999999998,"15min":0.997},"min":{"1min":0.48799999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.032,"5min":5.157,"15min":5.157},"last":0.621}]},{"osd":7,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78300000000000003,"15min":0.85499999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96599999999999997,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.749},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.86699999999999999,"15min":1.0089999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1399999999999999,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.68400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36016,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335824,"statfs":{"total":96636764160,"available":96599883776,"internally_reserved":0,"allocated":892928,"data_stored":398170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.63600000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88700000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.77000000000000002,"15min":0.91100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84899999999999998,"5min":6.258,"15min":6.258},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65700000000000003},"min":{"1min":0.46800000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.78700000000000003,"15min":0.92400000000000004},"min":{"1min":0.38300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.85299999999999998,"15min":1.0129999999999999},"min":{"1min":0.54700000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95099999999999996,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79800000000000004,"5min":1.25,"15min":1.25},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93700000000000006,"5min":1.413,"15min":1.413},"last":0.745}]},{"osd":5,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.67400000000000004},"min":{"1min":0.42399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63300000000000001,"15min":0.67900000000000005},"min":{"1min":0.40200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82499999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64500000000000002,"15min":0.67400000000000004},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80900000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76800000000000002,"15min":0.80500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.70699999999999996,"15min":0.74099999999999999},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80900000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.76500000000000001},"min":{"1min":0.46500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.079,"5min":1.21,"15min":1.21},"last":0.63}]}]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1482752,"data_stored":987587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.66900000000000004,"15min":0.65300000000000002},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":4.5010000000000003,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.77000000000000002,"15min":0.91000000000000003},"min":{"1min":0.38100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.873,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64400000000000002,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.65800000000000003,"15min":0.67900000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":1.546,"15min":1.546},"last":1.1970000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.70499999999999996},"min":{"1min":0.42899999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.91100000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.879,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.93000000000000005}]},{"osd":3,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91500000000000004,"5min":1.03,"15min":1.03},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.747,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.484,"5min":4.484,"15min":4.484},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.621,"15min":0.629},"min":{"1min":0.45300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90600000000000003,"5min":2.718,"15min":2.718},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92400000000000004,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89600000000000002,"5min":1.669,"15min":1.669},"last":1.246}]},{"osd":7,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.77800000000000002},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93600000000000005,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.3360000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54100000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97999999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.97699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:49:39.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:49:39.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.455+0000 7f86481ff700 1 Processor -- start 2023-07-06T03:49:39.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.455+0000 7f86481ff700 1 -- start start 2023-07-06T03:49:39.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.456+0000 7f86481ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f864015c520 0x7f86400cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.457+0000 7f86481ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86400cdae0 con 0x7f8640160d00 2023-07-06T03:49:39.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.457+0000 7f86481ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86400b6640 con 0x7f8640160180 2023-07-06T03:49:39.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.457+0000 7f86481ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86400cd8d0 con 0x7f864015c520 2023-07-06T03:49:39.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.457+0000 7f864679c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8640160d00 0x7f86400b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35604/0 (socket says 172.21.15.97:35604) 2023-07-06T03:49:39.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.457+0000 7f864679c700 1 -- 172.21.15.97:0/591539102 learned_addr learned my addr 172.21.15.97:0/591539102 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:39.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.457+0000 7f8645f9b700 1 --2- 172.21.15.97:0/591539102 >> v2:172.21.15.146:3300/0 conn(0x7f864015c520 0x7f86400cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.457+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329748234 0 0) 0x7f86400cdae0 con 0x7f8640160d00 2023-07-06T03:49:39.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.457+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f862c0035b0 con 0x7f8640160d00 2023-07-06T03:49:39.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 103855141 0 0) 0x7f862c0035b0 con 0x7f8640160d00 2023-07-06T03:49:39.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 >> v2:172.21.15.146:3300/0 conn(0x7f864015c520 msgr2=0x7f86400cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:39.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 --2- 172.21.15.97:0/591539102 >> v2:172.21.15.146:3300/0 conn(0x7f864015c520 0x7f86400cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 >> v1:172.21.15.146:6789/0 conn(0x7f8640160180 legacy=0x7f86401605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86400b6920 con 0x7f8640160d00 2023-07-06T03:49:39.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f86380019f0 con 0x7f8640160d00 2023-07-06T03:49:39.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 >> v1:172.21.15.97:6789/0 conn(0x7f8640160d00 legacy=0x7f86400b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 --2- 172.21.15.97:0/591539102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c003a00 0x7f862c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 --2- 172.21.15.97:0/591539102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f862c004850 0x7f862c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 --2- 172.21.15.97:0/591539102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f862c005200 0x7f862c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86400cd8d0 con 0x7f862c003a00 2023-07-06T03:49:39.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c00c080 con 0x7f862c005200 2023-07-06T03:49:39.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c00c240 con 0x7f862c004850 2023-07-06T03:49:39.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8645f9b700 1 --2- 172.21.15.97:0/591539102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c003a00 0x7f862c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f864679c700 1 --2- 172.21.15.97:0/591539102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f862c005200 0x7f862c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f864579a700 1 --2- 172.21.15.97:0/591539102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f862c004850 0x7f862c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8645f9b700 1 -- 172.21.15.97:0/591539102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f862c004850 msgr2=0x7f862c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:39.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8645f9b700 1 --2- 172.21.15.97:0/591539102 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f862c004850 0x7f862c004cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8645f9b700 1 -- 172.21.15.97:0/591539102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f862c005200 msgr2=0x7f862c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:39.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8645f9b700 1 --2- 172.21.15.97:0/591539102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f862c005200 0x7f862c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f8645f9b700 1 -- 172.21.15.97:0/591539102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0035b0 con 0x7f862c003a00 2023-07-06T03:49:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.458+0000 7f864679c700 1 -- 172.21.15.97:0/591539102 reap_dead start 2023-07-06T03:49:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.459+0000 7f8645f9b700 1 --2- 172.21.15.97:0/591539102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c003a00 0x7f862c003e10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f863c009b60 tx=0x7f862c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4da6a6f6b9c475f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.459+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c00e6c0 con 0x7f862c003a00 2023-07-06T03:49:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.459+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c00e820 con 0x7f862c003a00 2023-07-06T03:49:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.459+0000 7f8644f99700 1 -- 172.21.15.97:0/591539102 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c00eb90 con 0x7f862c003a00 2023-07-06T03:49:39.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.459+0000 7f86481ff700 1 -- 172.21.15.97:0/591539102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c003a00 msgr2=0x7f862c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.459+0000 7f86481ff700 1 --2- 172.21.15.97:0/591539102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c003a00 0x7f862c003e10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f863c009b60 tx=0x7f862c005880 comp rx=0 tx=0).stop 2023-07-06T03:49:39.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.459+0000 7f86481ff700 1 -- 172.21.15.97:0/591539102 shutdown_connections 2023-07-06T03:49:39.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.459+0000 7f86481ff700 1 --2- 172.21.15.97:0/591539102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f862c003a00 0x7f862c003e10 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.459+0000 7f86481ff700 1 -- 172.21.15.97:0/591539102 >> 172.21.15.97:0/591539102 conn(0x7f8640071790 msgr2=0x7f8640071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:39.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.459+0000 7f86481ff700 1 -- 172.21.15.97:0/591539102 shutdown_connections 2023-07-06T03:49:39.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.460+0000 7f86481ff700 1 -- 172.21.15.97:0/591539102 wait complete. 2023-07-06T03:49:39.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.461+0000 7f86481ff700 1 Processor -- start 2023-07-06T03:49:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.461+0000 7f86481ff700 1 -- start start 2023-07-06T03:49:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.462+0000 7f86481ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f862c003a00 0x7f86401f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.462+0000 7f86481ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86401f3350 0x7f86401f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.462+0000 7f86481ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86400c6a30 0x7f86400c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.462+0000 7f86481ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864014dac0 con 0x7f86401f3350 2023-07-06T03:49:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.462+0000 7f86481ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86400c77a0 con 0x7f862c003a00 2023-07-06T03:49:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.462+0000 7f86481ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86400c7a80 con 0x7f86400c6a30 2023-07-06T03:49:39.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f864579a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86401f3350 0x7f86401f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f8645f9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f862c003a00 0x7f86401f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f8645f9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f862c003a00 0x7f86401f5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:42014/0 (socket says 172.21.15.97:42014) 2023-07-06T03:49:39.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f864579a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86401f3350 0x7f86401f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46176/0 (socket says 172.21.15.97:46176) 2023-07-06T03:49:39.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f8645f9b700 1 -- 172.21.15.97:0/2863402676 learned_addr learned my addr 172.21.15.97:0/2863402676 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:39.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f864679c700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86400c6a30 0x7f86400c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f864579a700 1 -- 172.21.15.97:0/2863402676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86400c6a30 msgr2=0x7f86400c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f864579a700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86400c6a30 0x7f86400c6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f864579a700 1 -- 172.21.15.97:0/2863402676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f862c003a00 msgr2=0x7f86401f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f864579a700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f862c003a00 0x7f86401f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f864579a700 1 -- 172.21.15.97:0/2863402676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86400cae60 con 0x7f86401f3350 2023-07-06T03:49:39.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f8645f9b700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f862c003a00 0x7f86401f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:39.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f864579a700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86401f3350 0x7f86401f5cc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8640068f70 tx=0x7f863000d770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.463+0000 7f86367fc700 1 -- 172.21.15.97:0/2863402676 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000b070 con 0x7f86401f3350 2023-07-06T03:49:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.464+0000 7f86367fc700 1 -- 172.21.15.97:0/2863402676 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86300054e0 con 0x7f86401f3350 2023-07-06T03:49:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.464+0000 7f86367fc700 1 -- 172.21.15.97:0/2863402676 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300038a0 con 0x7f86401f3350 2023-07-06T03:49:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.464+0000 7f86481ff700 1 -- 172.21.15.97:0/2863402676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86400cb140 con 0x7f86401f3350 2023-07-06T03:49:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.464+0000 7f86481ff700 1 -- 172.21.15.97:0/2863402676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86400c7410 con 0x7f86401f3350 2023-07-06T03:49:39.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.465+0000 7f86481ff700 1 -- 172.21.15.97:0/2863402676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86400c7c40 con 0x7f86401f3350 2023-07-06T03:49:39.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.466+0000 7f86367fc700 1 -- 172.21.15.97:0/2863402676 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8630003a00 con 0x7f86401f3350 2023-07-06T03:49:39.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.466+0000 7f86367fc700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f862406ace0 0x7f862406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.467+0000 7f86367fc700 1 -- 172.21.15.97:0/2863402676 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f86300889c0 con 0x7f86401f3350 2023-07-06T03:49:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.467+0000 7f8645f9b700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f862406ace0 0x7f862406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.467+0000 7f8645f9b700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f862406ace0 0x7f862406d190 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f863c00a010 tx=0x7f863c01f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.471+0000 7f86367fc700 1 -- 172.21.15.97:0/2863402676 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8630053b10 con 0x7f86401f3350 2023-07-06T03:49:39.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.659+0000 7f86481ff700 1 -- 172.21.15.97:0/2863402676 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f86400b6ee0 con 0x7f862406ace0 2023-07-06T03:49:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.668+0000 7f86367fc700 1 -- 172.21.15.97:0/2863402676 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311197 (secure 0 0 0) 0x7f863c00ee40 con 0x7f862406ace0 2023-07-06T03:49:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:39.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.672+0000 7f86481ff700 1 -- 172.21.15.97:0/2863402676 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f862406ace0 msgr2=0x7f862406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.672+0000 7f86481ff700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f862406ace0 0x7f862406d190 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f863c00a010 tx=0x7f863c01f040 comp rx=0 tx=0).stop 2023-07-06T03:49:39.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.672+0000 7f86481ff700 1 -- 172.21.15.97:0/2863402676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86401f3350 msgr2=0x7f86401f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.672+0000 7f86481ff700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86401f3350 0x7f86401f5cc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8640068f70 tx=0x7f863000d770 comp rx=0 tx=0).stop 2023-07-06T03:49:39.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.672+0000 7f86481ff700 1 -- 172.21.15.97:0/2863402676 shutdown_connections 2023-07-06T03:49:39.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.672+0000 7f86481ff700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f862406ace0 0x7f862406d190 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.672+0000 7f86481ff700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86400c6a30 0x7f86400c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.672+0000 7f86481ff700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f862c003a00 0x7f86401f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.672+0000 7f86481ff700 1 --2- 172.21.15.97:0/2863402676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86401f3350 0x7f86401f5cc0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.672+0000 7f86481ff700 1 -- 172.21.15.97:0/2863402676 >> 172.21.15.97:0/2863402676 conn(0x7f8640071790 msgr2=0x7f86400b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:39.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.673+0000 7f86481ff700 1 -- 172.21.15.97:0/2863402676 shutdown_connections 2023-07-06T03:49:39.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.673+0000 7f86481ff700 1 -- 172.21.15.97:0/2863402676 wait complete. 2023-07-06T03:49:39.677 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:49:39.692 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2023-07-06T03:49:38.936423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264764,"kb_used_data":8188,"kb_used_omap":0,"kb_used_meta":256512,"kb_avail":754709956,"statfs":{"total":773094113280,"available":772822994944,"internally_reserved":0,"allocated":8384512,"data_stored":4382931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":262668288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001304"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852821+0000","last_change":"2023-07-06T03:49:34.857392+0000","last_active":"2023-07-06T03:49:35.852821+0000","last_peered":"2023-07-06T03:49:35.852821+0000","last_clean":"2023-07-06T03:49:35.852821+0000","last_became_active":"2023-07-06T03:49:34.856886+0000","last_became_peered":"2023-07-06T03:49:34.856886+0000","last_unstale":"2023-07-06T03:49:35.852821+0000","last_undegraded":"2023-07-06T03:49:35.852821+0000","last_fullsized":"2023-07-06T03:49:35.852821+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:12:38.263762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853120+0000","last_change":"2023-07-06T03:49:34.861439+0000","last_active":"2023-07-06T03:49:35.853120+0000","last_peered":"2023-07-06T03:49:35.853120+0000","last_clean":"2023-07-06T03:49:35.853120+0000","last_became_active":"2023-07-06T03:49:34.861097+0000","last_became_peered":"2023-07-06T03:49:34.861097+0000","last_unstale":"2023-07-06T03:49:35.853120+0000","last_undegraded":"2023-07-06T03:49:35.853120+0000","last_fullsized":"2023-07-06T03:49:35.853120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:09:16.549661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852989+0000","last_change":"2023-07-06T03:49:34.859353+0000","last_active":"2023-07-06T03:49:35.852989+0000","last_peered":"2023-07-06T03:49:35.852989+0000","last_clean":"2023-07-06T03:49:35.852989+0000","last_became_active":"2023-07-06T03:49:34.858960+0000","last_became_peered":"2023-07-06T03:49:34.858960+0000","last_unstale":"2023-07-06T03:49:35.852989+0000","last_undegraded":"2023-07-06T03:49:35.852989+0000","last_fullsized":"2023-07-06T03:49:35.852989+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:10.128139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853190+0000","last_change":"2023-07-06T03:49:34.856387+0000","last_active":"2023-07-06T03:49:35.853190+0000","last_peered":"2023-07-06T03:49:35.853190+0000","last_clean":"2023-07-06T03:49:35.853190+0000","last_became_active":"2023-07-06T03:49:34.856126+0000","last_became_peered":"2023-07-06T03:49:34.856126+0000","last_unstale":"2023-07-06T03:49:35.853190+0000","last_undegraded":"2023-07-06T03:49:35.853190+0000","last_fullsized":"2023-07-06T03:49:35.853190+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:17.448184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842500+0000","last_change":"2023-07-06T03:49:34.861369+0000","last_active":"2023-07-06T03:49:35.842500+0000","last_peered":"2023-07-06T03:49:35.842500+0000","last_clean":"2023-07-06T03:49:35.842500+0000","last_became_active":"2023-07-06T03:49:34.861097+0000","last_became_peered":"2023-07-06T03:49:34.861097+0000","last_unstale":"2023-07-06T03:49:35.842500+0000","last_undegraded":"2023-07-06T03:49:35.842500+0000","last_fullsized":"2023-07-06T03:49:35.842500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:32:50.648377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844955+0000","last_change":"2023-07-06T03:49:34.863174+0000","last_active":"2023-07-06T03:49:35.844955+0000","last_peered":"2023-07-06T03:49:35.844955+0000","last_clean":"2023-07-06T03:49:35.844955+0000","last_became_active":"2023-07-06T03:49:34.862827+0000","last_became_peered":"2023-07-06T03:49:34.862827+0000","last_unstale":"2023-07-06T03:49:35.844955+0000","last_undegraded":"2023-07-06T03:49:35.844955+0000","last_fullsized":"2023-07-06T03:49:35.844955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:40:19.464216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852502+0000","last_change":"2023-07-06T03:49:34.863940+0000","last_active":"2023-07-06T03:49:35.852502+0000","last_peered":"2023-07-06T03:49:35.852502+0000","last_clean":"2023-07-06T03:49:35.852502+0000","last_became_active":"2023-07-06T03:49:34.863668+0000","last_became_peered":"2023-07-06T03:49:34.863668+0000","last_unstale":"2023-07-06T03:49:35.852502+0000","last_undegraded":"2023-07-06T03:49:35.852502+0000","last_fullsized":"2023-07-06T03:49:35.852502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:19:29.742076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843458+0000","last_change":"2023-07-06T03:49:34.864528+0000","last_active":"2023-07-06T03:49:35.843458+0000","last_peered":"2023-07-06T03:49:35.843458+0000","last_clean":"2023-07-06T03:49:35.843458+0000","last_became_active":"2023-07-06T03:49:34.864259+0000","last_became_peered":"2023-07-06T03:49:34.864259+0000","last_unstale":"2023-07-06T03:49:35.843458+0000","last_undegraded":"2023-07-06T03:49:35.843458+0000","last_fullsized":"2023-07-06T03:49:35.843458+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:53:57.372988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853745+0000","last_change":"2023-07-06T03:49:34.859360+0000","last_active":"2023-07-06T03:49:35.853745+0000","last_peered":"2023-07-06T03:49:35.853745+0000","last_clean":"2023-07-06T03:49:35.853745+0000","last_became_active":"2023-07-06T03:49:34.859001+0000","last_became_peered":"2023-07-06T03:49:34.859001+0000","last_unstale":"2023-07-06T03:49:35.853745+0000","last_undegraded":"2023-07-06T03:49:35.853745+0000","last_fullsized":"2023-07-06T03:49:35.853745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:40:59.286805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852523+0000","last_change":"2023-07-06T03:49:34.855165+0000","last_active":"2023-07-06T03:49:35.852523+0000","last_peered":"2023-07-06T03:49:35.852523+0000","last_clean":"2023-07-06T03:49:35.852523+0000","last_became_active":"2023-07-06T03:49:34.854887+0000","last_became_peered":"2023-07-06T03:49:34.854887+0000","last_unstale":"2023-07-06T03:49:35.852523+0000","last_undegraded":"2023-07-06T03:49:35.852523+0000","last_fullsized":"2023-07-06T03:49:35.852523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:32:24.890494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853043+0000","last_change":"2023-07-06T03:49:34.860985+0000","last_active":"2023-07-06T03:49:35.853043+0000","last_peered":"2023-07-06T03:49:35.853043+0000","last_clean":"2023-07-06T03:49:35.853043+0000","last_became_active":"2023-07-06T03:49:34.860690+0000","last_became_peered":"2023-07-06T03:49:34.860690+0000","last_unstale":"2023-07-06T03:49:35.853043+0000","last_undegraded":"2023-07-06T03:49:35.853043+0000","last_fullsized":"2023-07-06T03:49:35.853043+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:02.931578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847357+0000","last_change":"2023-07-06T03:49:34.873436+0000","last_active":"2023-07-06T03:49:35.847357+0000","last_peered":"2023-07-06T03:49:35.847357+0000","last_clean":"2023-07-06T03:49:35.847357+0000","last_became_active":"2023-07-06T03:49:34.873184+0000","last_became_peered":"2023-07-06T03:49:34.873184+0000","last_unstale":"2023-07-06T03:49:35.847357+0000","last_undegraded":"2023-07-06T03:49:35.847357+0000","last_fullsized":"2023-07-06T03:49:35.847357+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:02.931578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852890+0000","last_change":"2023-07-06T03:49:34.861034+0000","last_active":"2023-07-06T03:49:35.852890+0000","last_peered":"2023-07-06T03:49:35.852890+0000","last_clean":"2023-07-06T03:49:35.852890+0000","last_became_active":"2023-07-06T03:49:34.860723+0000","last_became_peered":"2023-07-06T03:49:34.860723+0000","last_unstale":"2023-07-06T03:49:35.852890+0000","last_undegraded":"2023-07-06T03:49:35.852890+0000","last_fullsized":"2023-07-06T03:49:35.852890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:36:40.499610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843975+0000","last_change":"2023-07-06T03:49:34.861781+0000","last_active":"2023-07-06T03:49:35.843975+0000","last_peered":"2023-07-06T03:49:35.843975+0000","last_clean":"2023-07-06T03:49:35.843975+0000","last_became_active":"2023-07-06T03:49:34.861467+0000","last_became_peered":"2023-07-06T03:49:34.861467+0000","last_unstale":"2023-07-06T03:49:35.843975+0000","last_undegraded":"2023-07-06T03:49:35.843975+0000","last_fullsized":"2023-07-06T03:49:35.843975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:57:00.526918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842573+0000","last_change":"2023-07-06T03:49:34.854894+0000","last_active":"2023-07-06T03:49:35.842573+0000","last_peered":"2023-07-06T03:49:35.842573+0000","last_clean":"2023-07-06T03:49:35.842573+0000","last_became_active":"2023-07-06T03:49:34.854588+0000","last_became_peered":"2023-07-06T03:49:34.854588+0000","last_unstale":"2023-07-06T03:49:35.842573+0000","last_undegraded":"2023-07-06T03:49:35.842573+0000","last_fullsized":"2023-07-06T03:49:35.842573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:19:11.597750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852420+0000","last_change":"2023-07-06T03:49:34.867737+0000","last_active":"2023-07-06T03:49:35.852420+0000","last_peered":"2023-07-06T03:49:35.852420+0000","last_clean":"2023-07-06T03:49:35.852420+0000","last_became_active":"2023-07-06T03:49:34.867245+0000","last_became_peered":"2023-07-06T03:49:34.867245+0000","last_unstale":"2023-07-06T03:49:35.852420+0000","last_undegraded":"2023-07-06T03:49:35.852420+0000","last_fullsized":"2023-07-06T03:49:35.852420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:53.991327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844408+0000","last_change":"2023-07-06T03:49:34.866813+0000","last_active":"2023-07-06T03:49:35.844408+0000","last_peered":"2023-07-06T03:49:35.844408+0000","last_clean":"2023-07-06T03:49:35.844408+0000","last_became_active":"2023-07-06T03:49:34.866553+0000","last_became_peered":"2023-07-06T03:49:34.866553+0000","last_unstale":"2023-07-06T03:49:35.844408+0000","last_undegraded":"2023-07-06T03:49:35.844408+0000","last_fullsized":"2023-07-06T03:49:35.844408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:54:57.096501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853248+0000","last_change":"2023-07-06T03:49:34.860176+0000","last_active":"2023-07-06T03:49:35.853248+0000","last_peered":"2023-07-06T03:49:35.853248+0000","last_clean":"2023-07-06T03:49:35.853248+0000","last_became_active":"2023-07-06T03:49:34.859795+0000","last_became_peered":"2023-07-06T03:49:34.859795+0000","last_unstale":"2023-07-06T03:49:35.853248+0000","last_undegraded":"2023-07-06T03:49:35.853248+0000","last_fullsized":"2023-07-06T03:49:35.853248+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:35.947643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844462+0000","last_change":"2023-07-06T03:49:34.871124+0000","last_active":"2023-07-06T03:49:35.844462+0000","last_peered":"2023-07-06T03:49:35.844462+0000","last_clean":"2023-07-06T03:49:35.844462+0000","last_became_active":"2023-07-06T03:49:34.870767+0000","last_became_peered":"2023-07-06T03:49:34.870767+0000","last_unstale":"2023-07-06T03:49:35.844462+0000","last_undegraded":"2023-07-06T03:49:35.844462+0000","last_fullsized":"2023-07-06T03:49:35.844462+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:08:25.215713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844256+0000","last_change":"2023-07-06T03:49:34.866285+0000","last_active":"2023-07-06T03:49:35.844256+0000","last_peered":"2023-07-06T03:49:35.844256+0000","last_clean":"2023-07-06T03:49:35.844256+0000","last_became_active":"2023-07-06T03:49:34.865816+0000","last_became_peered":"2023-07-06T03:49:34.865816+0000","last_unstale":"2023-07-06T03:49:35.844256+0000","last_undegraded":"2023-07-06T03:49:35.844256+0000","last_fullsized":"2023-07-06T03:49:35.844256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:51.920331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847358+0000","last_change":"2023-07-06T03:49:34.863645+0000","last_active":"2023-07-06T03:49:35.847358+0000","last_peered":"2023-07-06T03:49:35.847358+0000","last_clean":"2023-07-06T03:49:35.847358+0000","last_became_active":"2023-07-06T03:49:34.863084+0000","last_became_peered":"2023-07-06T03:49:34.863084+0000","last_unstale":"2023-07-06T03:49:35.847358+0000","last_undegraded":"2023-07-06T03:49:35.847358+0000","last_fullsized":"2023-07-06T03:49:35.847358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:10.579687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852676+0000","last_change":"2023-07-06T03:49:34.866780+0000","last_active":"2023-07-06T03:49:35.852676+0000","last_peered":"2023-07-06T03:49:35.852676+0000","last_clean":"2023-07-06T03:49:35.852676+0000","last_became_active":"2023-07-06T03:49:34.866518+0000","last_became_peered":"2023-07-06T03:49:34.866518+0000","last_unstale":"2023-07-06T03:49:35.852676+0000","last_undegraded":"2023-07-06T03:49:35.852676+0000","last_fullsized":"2023-07-06T03:49:35.852676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:56:22.166639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852688+0000","last_change":"2023-07-06T03:49:34.861289+0000","last_active":"2023-07-06T03:49:35.852688+0000","last_peered":"2023-07-06T03:49:35.852688+0000","last_clean":"2023-07-06T03:49:35.852688+0000","last_became_active":"2023-07-06T03:49:34.860976+0000","last_became_peered":"2023-07-06T03:49:34.860976+0000","last_unstale":"2023-07-06T03:49:35.852688+0000","last_undegraded":"2023-07-06T03:49:35.852688+0000","last_fullsized":"2023-07-06T03:49:35.852688+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:38:04.661296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852494+0000","last_change":"2023-07-06T03:49:34.863001+0000","last_active":"2023-07-06T03:49:35.852494+0000","last_peered":"2023-07-06T03:49:35.852494+0000","last_clean":"2023-07-06T03:49:35.852494+0000","last_became_active":"2023-07-06T03:49:34.862732+0000","last_became_peered":"2023-07-06T03:49:34.862732+0000","last_unstale":"2023-07-06T03:49:35.852494+0000","last_undegraded":"2023-07-06T03:49:35.852494+0000","last_fullsized":"2023-07-06T03:49:35.852494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:12:33.006465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844155+0000","last_change":"2023-07-06T03:49:34.859976+0000","last_active":"2023-07-06T03:49:35.844155+0000","last_peered":"2023-07-06T03:49:35.844155+0000","last_clean":"2023-07-06T03:49:35.844155+0000","last_became_active":"2023-07-06T03:49:34.859719+0000","last_became_peered":"2023-07-06T03:49:34.859719+0000","last_unstale":"2023-07-06T03:49:35.844155+0000","last_undegraded":"2023-07-06T03:49:35.844155+0000","last_fullsized":"2023-07-06T03:49:35.844155+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:00.819925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853236+0000","last_change":"2023-07-06T03:49:34.855924+0000","last_active":"2023-07-06T03:49:35.853236+0000","last_peered":"2023-07-06T03:49:35.853236+0000","last_clean":"2023-07-06T03:49:35.853236+0000","last_became_active":"2023-07-06T03:49:34.855644+0000","last_became_peered":"2023-07-06T03:49:34.855644+0000","last_unstale":"2023-07-06T03:49:35.853236+0000","last_undegraded":"2023-07-06T03:49:35.853236+0000","last_fullsized":"2023-07-06T03:49:35.853236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:17:24.091237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853073+0000","last_change":"2023-07-06T03:49:34.866832+0000","last_active":"2023-07-06T03:49:35.853073+0000","last_peered":"2023-07-06T03:49:35.853073+0000","last_clean":"2023-07-06T03:49:35.853073+0000","last_became_active":"2023-07-06T03:49:34.866537+0000","last_became_peered":"2023-07-06T03:49:34.866537+0000","last_unstale":"2023-07-06T03:49:35.853073+0000","last_undegraded":"2023-07-06T03:49:35.853073+0000","last_fullsized":"2023-07-06T03:49:35.853073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:58:09.047067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842680+0000","last_change":"2023-07-06T03:49:34.861645+0000","last_active":"2023-07-06T03:49:35.842680+0000","last_peered":"2023-07-06T03:49:35.842680+0000","last_clean":"2023-07-06T03:49:35.842680+0000","last_became_active":"2023-07-06T03:49:34.861348+0000","last_became_peered":"2023-07-06T03:49:34.861348+0000","last_unstale":"2023-07-06T03:49:35.842680+0000","last_undegraded":"2023-07-06T03:49:35.842680+0000","last_fullsized":"2023-07-06T03:49:35.842680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:56:44.253681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844259+0000","last_change":"2023-07-06T03:49:34.861537+0000","last_active":"2023-07-06T03:49:35.844259+0000","last_peered":"2023-07-06T03:49:35.844259+0000","last_clean":"2023-07-06T03:49:35.844259+0000","last_became_active":"2023-07-06T03:49:34.861268+0000","last_became_peered":"2023-07-06T03:49:34.861268+0000","last_unstale":"2023-07-06T03:49:35.844259+0000","last_undegraded":"2023-07-06T03:49:35.844259+0000","last_fullsized":"2023-07-06T03:49:35.844259+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:35.842450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852175+0000","last_change":"2023-07-06T03:49:34.864092+0000","last_active":"2023-07-06T03:49:35.852175+0000","last_peered":"2023-07-06T03:49:35.852175+0000","last_clean":"2023-07-06T03:49:35.852175+0000","last_became_active":"2023-07-06T03:49:34.863833+0000","last_became_peered":"2023-07-06T03:49:34.863833+0000","last_unstale":"2023-07-06T03:49:35.852175+0000","last_undegraded":"2023-07-06T03:49:35.852175+0000","last_fullsized":"2023-07-06T03:49:35.852175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:44:27.220590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852594+0000","last_change":"2023-07-06T03:49:34.865589+0000","last_active":"2023-07-06T03:49:35.852594+0000","last_peered":"2023-07-06T03:49:35.852594+0000","last_clean":"2023-07-06T03:49:35.852594+0000","last_became_active":"2023-07-06T03:49:34.865282+0000","last_became_peered":"2023-07-06T03:49:34.865282+0000","last_unstale":"2023-07-06T03:49:35.852594+0000","last_undegraded":"2023-07-06T03:49:35.852594+0000","last_fullsized":"2023-07-06T03:49:35.852594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:30:51.254414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852549+0000","last_change":"2023-07-06T03:49:34.865615+0000","last_active":"2023-07-06T03:49:35.852549+0000","last_peered":"2023-07-06T03:49:35.852549+0000","last_clean":"2023-07-06T03:49:35.852549+0000","last_became_active":"2023-07-06T03:49:34.865316+0000","last_became_peered":"2023-07-06T03:49:34.865316+0000","last_unstale":"2023-07-06T03:49:35.852549+0000","last_undegraded":"2023-07-06T03:49:35.852549+0000","last_fullsized":"2023-07-06T03:49:35.852549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:00.679582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842347+0000","last_change":"2023-07-06T03:49:34.858416+0000","last_active":"2023-07-06T03:49:35.842347+0000","last_peered":"2023-07-06T03:49:35.842347+0000","last_clean":"2023-07-06T03:49:35.842347+0000","last_became_active":"2023-07-06T03:49:34.858136+0000","last_became_peered":"2023-07-06T03:49:34.858136+0000","last_unstale":"2023-07-06T03:49:35.842347+0000","last_undegraded":"2023-07-06T03:49:35.842347+0000","last_fullsized":"2023-07-06T03:49:35.842347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:55:39.894783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843042+0000","last_change":"2023-07-06T03:49:33.824414+0000","last_active":"2023-07-06T03:49:35.843042+0000","last_peered":"2023-07-06T03:49:35.843042+0000","last_clean":"2023-07-06T03:49:35.843042+0000","last_became_active":"2023-07-06T03:49:33.824113+0000","last_became_peered":"2023-07-06T03:49:33.824113+0000","last_unstale":"2023-07-06T03:49:35.843042+0000","last_undegraded":"2023-07-06T03:49:35.843042+0000","last_fullsized":"2023-07-06T03:49:35.843042+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:46:42.367746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843022+0000","last_change":"2023-07-06T03:49:34.847528+0000","last_active":"2023-07-06T03:49:35.843022+0000","last_peered":"2023-07-06T03:49:35.843022+0000","last_clean":"2023-07-06T03:49:35.843022+0000","last_became_active":"2023-07-06T03:49:34.847231+0000","last_became_peered":"2023-07-06T03:49:34.847231+0000","last_unstale":"2023-07-06T03:49:35.843022+0000","last_undegraded":"2023-07-06T03:49:35.843022+0000","last_fullsized":"2023-07-06T03:49:35.843022+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:51.920331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.854060+0000","last_change":"2023-07-06T03:49:33.818932+0000","last_active":"2023-07-06T03:49:35.854060+0000","last_peered":"2023-07-06T03:49:35.854060+0000","last_clean":"2023-07-06T03:49:35.854060+0000","last_became_active":"2023-07-06T03:49:33.818659+0000","last_became_peered":"2023-07-06T03:49:33.818659+0000","last_unstale":"2023-07-06T03:49:35.854060+0000","last_undegraded":"2023-07-06T03:49:35.854060+0000","last_fullsized":"2023-07-06T03:49:35.854060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:45:15.172488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847206+0000","last_change":"2023-07-06T03:49:34.869130+0000","last_active":"2023-07-06T03:49:35.847206+0000","last_peered":"2023-07-06T03:49:35.847206+0000","last_clean":"2023-07-06T03:49:35.847206+0000","last_became_active":"2023-07-06T03:49:34.868890+0000","last_became_peered":"2023-07-06T03:49:34.868890+0000","last_unstale":"2023-07-06T03:49:35.847206+0000","last_undegraded":"2023-07-06T03:49:35.847206+0000","last_fullsized":"2023-07-06T03:49:35.847206+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:40:14.144373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852478+0000","last_change":"2023-07-06T03:49:33.830805+0000","last_active":"2023-07-06T03:49:35.852478+0000","last_peered":"2023-07-06T03:49:35.852478+0000","last_clean":"2023-07-06T03:49:35.852478+0000","last_became_active":"2023-07-06T03:49:33.830514+0000","last_became_peered":"2023-07-06T03:49:33.830514+0000","last_unstale":"2023-07-06T03:49:35.852478+0000","last_undegraded":"2023-07-06T03:49:35.852478+0000","last_fullsized":"2023-07-06T03:49:35.852478+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:56.794849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852934+0000","last_change":"2023-07-06T03:49:34.856932+0000","last_active":"2023-07-06T03:49:35.852934+0000","last_peered":"2023-07-06T03:49:35.852934+0000","last_clean":"2023-07-06T03:49:35.852934+0000","last_became_active":"2023-07-06T03:49:34.856672+0000","last_became_peered":"2023-07-06T03:49:34.856672+0000","last_unstale":"2023-07-06T03:49:35.852934+0000","last_undegraded":"2023-07-06T03:49:35.852934+0000","last_fullsized":"2023-07-06T03:49:35.852934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:32.212195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847587+0000","last_change":"2023-07-06T03:49:33.829363+0000","last_active":"2023-07-06T03:49:35.847587+0000","last_peered":"2023-07-06T03:49:35.847587+0000","last_clean":"2023-07-06T03:49:35.847587+0000","last_became_active":"2023-07-06T03:49:33.829049+0000","last_became_peered":"2023-07-06T03:49:33.829049+0000","last_unstale":"2023-07-06T03:49:35.847587+0000","last_undegraded":"2023-07-06T03:49:35.847587+0000","last_fullsized":"2023-07-06T03:49:35.847587+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:55:12.573057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844282+0000","last_change":"2023-07-06T03:49:34.866080+0000","last_active":"2023-07-06T03:49:35.844282+0000","last_peered":"2023-07-06T03:49:35.844282+0000","last_clean":"2023-07-06T03:49:35.844282+0000","last_became_active":"2023-07-06T03:49:34.865477+0000","last_became_peered":"2023-07-06T03:49:34.865477+0000","last_unstale":"2023-07-06T03:49:35.844282+0000","last_undegraded":"2023-07-06T03:49:35.844282+0000","last_fullsized":"2023-07-06T03:49:35.844282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:00:43.751249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847380+0000","last_change":"2023-07-06T03:49:33.831571+0000","last_active":"2023-07-06T03:49:35.847380+0000","last_peered":"2023-07-06T03:49:35.847380+0000","last_clean":"2023-07-06T03:49:35.847380+0000","last_became_active":"2023-07-06T03:49:33.831291+0000","last_became_peered":"2023-07-06T03:49:33.831291+0000","last_unstale":"2023-07-06T03:49:35.847380+0000","last_undegraded":"2023-07-06T03:49:35.847380+0000","last_fullsized":"2023-07-06T03:49:35.847380+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:09:15.546939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853139+0000","last_change":"2023-07-06T03:49:34.859640+0000","last_active":"2023-07-06T03:49:35.853139+0000","last_peered":"2023-07-06T03:49:35.853139+0000","last_clean":"2023-07-06T03:49:35.853139+0000","last_became_active":"2023-07-06T03:49:34.859348+0000","last_became_peered":"2023-07-06T03:49:34.859348+0000","last_unstale":"2023-07-06T03:49:35.853139+0000","last_undegraded":"2023-07-06T03:49:35.853139+0000","last_fullsized":"2023-07-06T03:49:35.853139+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:37:31.474862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852552+0000","last_change":"2023-07-06T03:49:33.830053+0000","last_active":"2023-07-06T03:49:35.852552+0000","last_peered":"2023-07-06T03:49:35.852552+0000","last_clean":"2023-07-06T03:49:35.852552+0000","last_became_active":"2023-07-06T03:49:33.829779+0000","last_became_peered":"2023-07-06T03:49:33.829779+0000","last_unstale":"2023-07-06T03:49:35.852552+0000","last_undegraded":"2023-07-06T03:49:35.852552+0000","last_fullsized":"2023-07-06T03:49:35.852552+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:11:13.359833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847539+0000","last_change":"2023-07-06T03:49:34.871128+0000","last_active":"2023-07-06T03:49:35.847539+0000","last_peered":"2023-07-06T03:49:35.847539+0000","last_clean":"2023-07-06T03:49:35.847539+0000","last_became_active":"2023-07-06T03:49:34.870830+0000","last_became_peered":"2023-07-06T03:49:34.870830+0000","last_unstale":"2023-07-06T03:49:35.847539+0000","last_undegraded":"2023-07-06T03:49:35.847539+0000","last_fullsized":"2023-07-06T03:49:35.847539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:35.947643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844572+0000","last_change":"2023-07-06T03:49:33.830130+0000","last_active":"2023-07-06T03:49:35.844572+0000","last_peered":"2023-07-06T03:49:35.844572+0000","last_clean":"2023-07-06T03:49:35.844572+0000","last_became_active":"2023-07-06T03:49:33.829812+0000","last_became_peered":"2023-07-06T03:49:33.829812+0000","last_unstale":"2023-07-06T03:49:35.844572+0000","last_undegraded":"2023-07-06T03:49:35.844572+0000","last_fullsized":"2023-07-06T03:49:35.844572+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:20:11.111731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.854077+0000","last_change":"2023-07-06T03:49:34.871872+0000","last_active":"2023-07-06T03:49:35.854077+0000","last_peered":"2023-07-06T03:49:35.854077+0000","last_clean":"2023-07-06T03:49:35.854077+0000","last_became_active":"2023-07-06T03:49:34.871572+0000","last_became_peered":"2023-07-06T03:49:34.871572+0000","last_unstale":"2023-07-06T03:49:35.854077+0000","last_undegraded":"2023-07-06T03:49:35.854077+0000","last_fullsized":"2023-07-06T03:49:35.854077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:24:48.045223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852703+0000","last_change":"2023-07-06T03:49:33.824719+0000","last_active":"2023-07-06T03:49:35.852703+0000","last_peered":"2023-07-06T03:49:35.852703+0000","last_clean":"2023-07-06T03:49:35.852703+0000","last_became_active":"2023-07-06T03:49:33.824457+0000","last_became_peered":"2023-07-06T03:49:33.824457+0000","last_unstale":"2023-07-06T03:49:35.852703+0000","last_undegraded":"2023-07-06T03:49:35.852703+0000","last_fullsized":"2023-07-06T03:49:35.852703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:52:38.585500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852504+0000","last_change":"2023-07-06T03:49:34.862528+0000","last_active":"2023-07-06T03:49:35.852504+0000","last_peered":"2023-07-06T03:49:35.852504+0000","last_clean":"2023-07-06T03:49:35.852504+0000","last_became_active":"2023-07-06T03:49:34.862088+0000","last_became_peered":"2023-07-06T03:49:34.862088+0000","last_unstale":"2023-07-06T03:49:35.852504+0000","last_undegraded":"2023-07-06T03:49:35.852504+0000","last_fullsized":"2023-07-06T03:49:35.852504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:26:37.844936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853622+0000","last_change":"2023-07-06T03:49:33.821887+0000","last_active":"2023-07-06T03:49:35.853622+0000","last_peered":"2023-07-06T03:49:35.853622+0000","last_clean":"2023-07-06T03:49:35.853622+0000","last_became_active":"2023-07-06T03:49:33.821606+0000","last_became_peered":"2023-07-06T03:49:33.821606+0000","last_unstale":"2023-07-06T03:49:35.853622+0000","last_undegraded":"2023-07-06T03:49:35.853622+0000","last_fullsized":"2023-07-06T03:49:35.853622+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:34.545522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847222+0000","last_change":"2023-07-06T03:49:34.862631+0000","last_active":"2023-07-06T03:49:35.847222+0000","last_peered":"2023-07-06T03:49:35.847222+0000","last_clean":"2023-07-06T03:49:35.847222+0000","last_became_active":"2023-07-06T03:49:34.862358+0000","last_became_peered":"2023-07-06T03:49:34.862358+0000","last_unstale":"2023-07-06T03:49:35.847222+0000","last_undegraded":"2023-07-06T03:49:35.847222+0000","last_fullsized":"2023-07-06T03:49:35.847222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:10.128139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852661+0000","last_change":"2023-07-06T03:49:33.823032+0000","last_active":"2023-07-06T03:49:35.852661+0000","last_peered":"2023-07-06T03:49:35.852661+0000","last_clean":"2023-07-06T03:49:35.852661+0000","last_became_active":"2023-07-06T03:49:33.822768+0000","last_became_peered":"2023-07-06T03:49:33.822768+0000","last_unstale":"2023-07-06T03:49:35.852661+0000","last_undegraded":"2023-07-06T03:49:35.852661+0000","last_fullsized":"2023-07-06T03:49:35.852661+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:14:20.388073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843813+0000","last_change":"2023-07-06T03:49:34.865928+0000","last_active":"2023-07-06T03:49:35.843813+0000","last_peered":"2023-07-06T03:49:35.843813+0000","last_clean":"2023-07-06T03:49:35.843813+0000","last_became_active":"2023-07-06T03:49:34.865408+0000","last_became_peered":"2023-07-06T03:49:34.865408+0000","last_unstale":"2023-07-06T03:49:35.843813+0000","last_undegraded":"2023-07-06T03:49:35.843813+0000","last_fullsized":"2023-07-06T03:49:35.843813+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:34:27.019155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844716+0000","last_change":"2023-07-06T03:49:33.827783+0000","last_active":"2023-07-06T03:49:35.844716+0000","last_peered":"2023-07-06T03:49:35.844716+0000","last_clean":"2023-07-06T03:49:35.844716+0000","last_became_active":"2023-07-06T03:49:33.827520+0000","last_became_peered":"2023-07-06T03:49:33.827520+0000","last_unstale":"2023-07-06T03:49:35.844716+0000","last_undegraded":"2023-07-06T03:49:35.844716+0000","last_fullsized":"2023-07-06T03:49:35.844716+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:20:35.716810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844497+0000","last_change":"2023-07-06T03:49:34.864553+0000","last_active":"2023-07-06T03:49:35.844497+0000","last_peered":"2023-07-06T03:49:35.844497+0000","last_clean":"2023-07-06T03:49:35.844497+0000","last_became_active":"2023-07-06T03:49:34.864200+0000","last_became_peered":"2023-07-06T03:49:34.864200+0000","last_unstale":"2023-07-06T03:49:35.844497+0000","last_undegraded":"2023-07-06T03:49:35.844497+0000","last_fullsized":"2023-07-06T03:49:35.844497+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:14:16.181864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853973+0000","last_change":"2023-07-06T03:49:33.821142+0000","last_active":"2023-07-06T03:49:35.853973+0000","last_peered":"2023-07-06T03:49:35.853973+0000","last_clean":"2023-07-06T03:49:35.853973+0000","last_became_active":"2023-07-06T03:49:33.820618+0000","last_became_peered":"2023-07-06T03:49:33.820618+0000","last_unstale":"2023-07-06T03:49:35.853973+0000","last_undegraded":"2023-07-06T03:49:35.853973+0000","last_fullsized":"2023-07-06T03:49:35.853973+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:34:48.683822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853265+0000","last_change":"2023-07-06T03:49:34.865809+0000","last_active":"2023-07-06T03:49:35.853265+0000","last_peered":"2023-07-06T03:49:35.853265+0000","last_clean":"2023-07-06T03:49:35.853265+0000","last_became_active":"2023-07-06T03:49:34.865529+0000","last_became_peered":"2023-07-06T03:49:34.865529+0000","last_unstale":"2023-07-06T03:49:35.853265+0000","last_undegraded":"2023-07-06T03:49:35.853265+0000","last_fullsized":"2023-07-06T03:49:35.853265+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:44:40.247032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852919+0000","last_change":"2023-07-06T03:49:33.818687+0000","last_active":"2023-07-06T03:49:35.852919+0000","last_peered":"2023-07-06T03:49:35.852919+0000","last_clean":"2023-07-06T03:49:35.852919+0000","last_became_active":"2023-07-06T03:49:33.818371+0000","last_became_peered":"2023-07-06T03:49:33.818371+0000","last_unstale":"2023-07-06T03:49:35.852919+0000","last_undegraded":"2023-07-06T03:49:35.852919+0000","last_fullsized":"2023-07-06T03:49:35.852919+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:17:57.883769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852349+0000","last_change":"2023-07-06T03:49:34.872706+0000","last_active":"2023-07-06T03:49:35.852349+0000","last_peered":"2023-07-06T03:49:35.852349+0000","last_clean":"2023-07-06T03:49:35.852349+0000","last_became_active":"2023-07-06T03:49:34.872373+0000","last_became_peered":"2023-07-06T03:49:34.872373+0000","last_unstale":"2023-07-06T03:49:35.852349+0000","last_undegraded":"2023-07-06T03:49:35.852349+0000","last_fullsized":"2023-07-06T03:49:35.852349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:40:19.464216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847378+0000","last_change":"2023-07-06T03:49:33.834564+0000","last_active":"2023-07-06T03:49:35.847378+0000","last_peered":"2023-07-06T03:49:35.847378+0000","last_clean":"2023-07-06T03:49:35.847378+0000","last_became_active":"2023-07-06T03:49:33.834288+0000","last_became_peered":"2023-07-06T03:49:33.834288+0000","last_unstale":"2023-07-06T03:49:35.847378+0000","last_undegraded":"2023-07-06T03:49:35.847378+0000","last_fullsized":"2023-07-06T03:49:35.847378+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:37:30.472139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847555+0000","last_change":"2023-07-06T03:49:34.869775+0000","last_active":"2023-07-06T03:49:35.847555+0000","last_peered":"2023-07-06T03:49:35.847555+0000","last_clean":"2023-07-06T03:49:35.847555+0000","last_became_active":"2023-07-06T03:49:34.869513+0000","last_became_peered":"2023-07-06T03:49:34.869513+0000","last_unstale":"2023-07-06T03:49:35.847555+0000","last_undegraded":"2023-07-06T03:49:35.847555+0000","last_fullsized":"2023-07-06T03:49:35.847555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:09:57.398183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842918+0000","last_change":"2023-07-06T03:49:33.824608+0000","last_active":"2023-07-06T03:49:35.842918+0000","last_peered":"2023-07-06T03:49:35.842918+0000","last_clean":"2023-07-06T03:49:35.842918+0000","last_became_active":"2023-07-06T03:49:33.824272+0000","last_became_peered":"2023-07-06T03:49:33.824272+0000","last_unstale":"2023-07-06T03:49:35.842918+0000","last_undegraded":"2023-07-06T03:49:35.842918+0000","last_fullsized":"2023-07-06T03:49:35.842918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:27:59.817202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852852+0000","last_change":"2023-07-06T03:49:34.858352+0000","last_active":"2023-07-06T03:49:35.852852+0000","last_peered":"2023-07-06T03:49:35.852852+0000","last_clean":"2023-07-06T03:49:35.852852+0000","last_became_active":"2023-07-06T03:49:34.858116+0000","last_became_peered":"2023-07-06T03:49:34.858116+0000","last_unstale":"2023-07-06T03:49:35.852852+0000","last_undegraded":"2023-07-06T03:49:35.852852+0000","last_fullsized":"2023-07-06T03:49:35.852852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:46:17.272577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844538+0000","last_change":"2023-07-06T03:49:33.833266+0000","last_active":"2023-07-06T03:49:35.844538+0000","last_peered":"2023-07-06T03:49:35.844538+0000","last_clean":"2023-07-06T03:49:35.844538+0000","last_became_active":"2023-07-06T03:49:33.832703+0000","last_became_peered":"2023-07-06T03:49:33.832703+0000","last_unstale":"2023-07-06T03:49:35.844538+0000","last_undegraded":"2023-07-06T03:49:35.844538+0000","last_fullsized":"2023-07-06T03:49:35.844538+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:47:49.443158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852687+0000","last_change":"2023-07-06T03:49:34.863482+0000","last_active":"2023-07-06T03:49:35.852687+0000","last_peered":"2023-07-06T03:49:35.852687+0000","last_clean":"2023-07-06T03:49:35.852687+0000","last_became_active":"2023-07-06T03:49:34.863127+0000","last_became_peered":"2023-07-06T03:49:34.863127+0000","last_unstale":"2023-07-06T03:49:35.852687+0000","last_undegraded":"2023-07-06T03:49:35.852687+0000","last_fullsized":"2023-07-06T03:49:35.852687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:16.291086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852973+0000","last_change":"2023-07-06T03:49:33.820353+0000","last_active":"2023-07-06T03:49:35.852973+0000","last_peered":"2023-07-06T03:49:35.852973+0000","last_clean":"2023-07-06T03:49:35.852973+0000","last_became_active":"2023-07-06T03:49:33.820031+0000","last_became_peered":"2023-07-06T03:49:33.820031+0000","last_unstale":"2023-07-06T03:49:35.852973+0000","last_undegraded":"2023-07-06T03:49:35.852973+0000","last_fullsized":"2023-07-06T03:49:35.852973+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:18:43.350865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852758+0000","last_change":"2023-07-06T03:49:34.869378+0000","last_active":"2023-07-06T03:49:35.852758+0000","last_peered":"2023-07-06T03:49:35.852758+0000","last_clean":"2023-07-06T03:49:35.852758+0000","last_became_active":"2023-07-06T03:49:34.869133+0000","last_became_peered":"2023-07-06T03:49:34.869133+0000","last_unstale":"2023-07-06T03:49:35.852758+0000","last_undegraded":"2023-07-06T03:49:35.852758+0000","last_fullsized":"2023-07-06T03:49:35.852758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:49:08.018890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844045+0000","last_change":"2023-07-06T03:49:33.828309+0000","last_active":"2023-07-06T03:49:35.844045+0000","last_peered":"2023-07-06T03:49:35.844045+0000","last_clean":"2023-07-06T03:49:35.844045+0000","last_became_active":"2023-07-06T03:49:33.828026+0000","last_became_peered":"2023-07-06T03:49:33.828026+0000","last_unstale":"2023-07-06T03:49:35.844045+0000","last_undegraded":"2023-07-06T03:49:35.844045+0000","last_fullsized":"2023-07-06T03:49:35.844045+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:29.056482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847226+0000","last_change":"2023-07-06T03:49:34.865081+0000","last_active":"2023-07-06T03:49:35.847226+0000","last_peered":"2023-07-06T03:49:35.847226+0000","last_clean":"2023-07-06T03:49:35.847226+0000","last_became_active":"2023-07-06T03:49:34.864678+0000","last_became_peered":"2023-07-06T03:49:34.864678+0000","last_unstale":"2023-07-06T03:49:35.847226+0000","last_undegraded":"2023-07-06T03:49:35.847226+0000","last_fullsized":"2023-07-06T03:49:35.847226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:36:40.499610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852613+0000","last_change":"2023-07-06T03:49:33.829210+0000","last_active":"2023-07-06T03:49:35.852613+0000","last_peered":"2023-07-06T03:49:35.852613+0000","last_clean":"2023-07-06T03:49:35.852613+0000","last_became_active":"2023-07-06T03:49:33.828914+0000","last_became_peered":"2023-07-06T03:49:33.828914+0000","last_unstale":"2023-07-06T03:49:35.852613+0000","last_undegraded":"2023-07-06T03:49:35.852613+0000","last_fullsized":"2023-07-06T03:49:35.852613+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:30:20.604178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843921+0000","last_change":"2023-07-06T03:49:34.862597+0000","last_active":"2023-07-06T03:49:35.843921+0000","last_peered":"2023-07-06T03:49:35.843921+0000","last_clean":"2023-07-06T03:49:35.843921+0000","last_became_active":"2023-07-06T03:49:34.862332+0000","last_became_peered":"2023-07-06T03:49:34.862332+0000","last_unstale":"2023-07-06T03:49:35.843921+0000","last_undegraded":"2023-07-06T03:49:35.843921+0000","last_fullsized":"2023-07-06T03:49:35.843921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:02:06.863468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853311+0000","last_change":"2023-07-06T03:49:33.832386+0000","last_active":"2023-07-06T03:49:35.853311+0000","last_peered":"2023-07-06T03:49:35.853311+0000","last_clean":"2023-07-06T03:49:35.853311+0000","last_became_active":"2023-07-06T03:49:33.832095+0000","last_became_peered":"2023-07-06T03:49:33.832095+0000","last_unstale":"2023-07-06T03:49:35.853311+0000","last_undegraded":"2023-07-06T03:49:35.853311+0000","last_fullsized":"2023-07-06T03:49:35.853311+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:39:55.076647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.847367+0000","last_change":"2023-07-06T03:49:34.871135+0000","last_active":"2023-07-06T03:49:35.847367+0000","last_peered":"2023-07-06T03:49:35.847367+0000","last_clean":"2023-07-06T03:49:35.847367+0000","last_became_active":"2023-07-06T03:49:34.870875+0000","last_became_peered":"2023-07-06T03:49:34.870875+0000","last_unstale":"2023-07-06T03:49:35.847367+0000","last_undegraded":"2023-07-06T03:49:35.847367+0000","last_fullsized":"2023-07-06T03:49:35.847367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:52:46.294045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852524+0000","last_change":"2023-07-06T03:49:33.823501+0000","last_active":"2023-07-06T03:49:35.852524+0000","last_peered":"2023-07-06T03:49:35.852524+0000","last_clean":"2023-07-06T03:49:35.852524+0000","last_became_active":"2023-07-06T03:49:33.823222+0000","last_became_peered":"2023-07-06T03:49:33.823222+0000","last_unstale":"2023-07-06T03:49:35.852524+0000","last_undegraded":"2023-07-06T03:49:35.852524+0000","last_fullsized":"2023-07-06T03:49:35.852524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:13:13.043537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853114+0000","last_change":"2023-07-06T03:49:34.859911+0000","last_active":"2023-07-06T03:49:35.853114+0000","last_peered":"2023-07-06T03:49:35.853114+0000","last_clean":"2023-07-06T03:49:35.853114+0000","last_became_active":"2023-07-06T03:49:34.859664+0000","last_became_peered":"2023-07-06T03:49:34.859664+0000","last_unstale":"2023-07-06T03:49:35.853114+0000","last_undegraded":"2023-07-06T03:49:35.853114+0000","last_fullsized":"2023-07-06T03:49:35.853114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:54.861404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844892+0000","last_change":"2023-07-06T03:49:33.830049+0000","last_active":"2023-07-06T03:49:35.844892+0000","last_peered":"2023-07-06T03:49:35.844892+0000","last_clean":"2023-07-06T03:49:35.844892+0000","last_became_active":"2023-07-06T03:49:33.829724+0000","last_became_peered":"2023-07-06T03:49:33.829724+0000","last_unstale":"2023-07-06T03:49:35.844892+0000","last_undegraded":"2023-07-06T03:49:35.844892+0000","last_fullsized":"2023-07-06T03:49:35.844892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:44:26.217867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853133+0000","last_change":"2023-07-06T03:49:34.860503+0000","last_active":"2023-07-06T03:49:35.853133+0000","last_peered":"2023-07-06T03:49:35.853133+0000","last_clean":"2023-07-06T03:49:35.853133+0000","last_became_active":"2023-07-06T03:49:34.860214+0000","last_became_peered":"2023-07-06T03:49:34.860214+0000","last_unstale":"2023-07-06T03:49:35.853133+0000","last_undegraded":"2023-07-06T03:49:35.853133+0000","last_fullsized":"2023-07-06T03:49:35.853133+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:52:46.294045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852643+0000","last_change":"2023-07-06T03:49:33.822068+0000","last_active":"2023-07-06T03:49:35.852643+0000","last_peered":"2023-07-06T03:49:35.852643+0000","last_clean":"2023-07-06T03:49:35.852643+0000","last_became_active":"2023-07-06T03:49:33.821647+0000","last_became_peered":"2023-07-06T03:49:33.821647+0000","last_unstale":"2023-07-06T03:49:35.852643+0000","last_undegraded":"2023-07-06T03:49:35.852643+0000","last_fullsized":"2023-07-06T03:49:35.852643+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:53:54.322399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842723+0000","last_change":"2023-07-06T03:49:34.861916+0000","last_active":"2023-07-06T03:49:35.842723+0000","last_peered":"2023-07-06T03:49:35.842723+0000","last_clean":"2023-07-06T03:49:35.842723+0000","last_became_active":"2023-07-06T03:49:34.861637+0000","last_became_peered":"2023-07-06T03:49:34.861637+0000","last_unstale":"2023-07-06T03:49:35.842723+0000","last_undegraded":"2023-07-06T03:49:35.842723+0000","last_fullsized":"2023-07-06T03:49:35.842723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:47:04.357163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853138+0000","last_change":"2023-07-06T03:49:33.827584+0000","last_active":"2023-07-06T03:49:35.853138+0000","last_peered":"2023-07-06T03:49:35.853138+0000","last_clean":"2023-07-06T03:49:35.853138+0000","last_became_active":"2023-07-06T03:49:33.827258+0000","last_became_peered":"2023-07-06T03:49:33.827258+0000","last_unstale":"2023-07-06T03:49:35.853138+0000","last_undegraded":"2023-07-06T03:49:35.853138+0000","last_fullsized":"2023-07-06T03:49:35.853138+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:09.576964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844122+0000","last_change":"2023-07-06T03:49:34.867876+0000","last_active":"2023-07-06T03:49:35.844122+0000","last_peered":"2023-07-06T03:49:35.844122+0000","last_clean":"2023-07-06T03:49:35.844122+0000","last_became_active":"2023-07-06T03:49:34.867596+0000","last_became_peered":"2023-07-06T03:49:34.867596+0000","last_unstale":"2023-07-06T03:49:35.844122+0000","last_undegraded":"2023-07-06T03:49:35.844122+0000","last_fullsized":"2023-07-06T03:49:35.844122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:56:03.287815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842714+0000","last_change":"2023-07-06T03:49:33.823149+0000","last_active":"2023-07-06T03:49:35.842714+0000","last_peered":"2023-07-06T03:49:35.842714+0000","last_clean":"2023-07-06T03:49:35.842714+0000","last_became_active":"2023-07-06T03:49:33.821412+0000","last_became_peered":"2023-07-06T03:49:33.821412+0000","last_unstale":"2023-07-06T03:49:35.842714+0000","last_undegraded":"2023-07-06T03:49:35.842714+0000","last_fullsized":"2023-07-06T03:49:35.842714+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:55:50.863561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844361+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:49:35.844361+0000","last_peered":"2023-07-06T03:49:35.844361+0000","last_clean":"2023-07-06T03:49:35.844361+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:49:35.844361+0000","last_undegraded":"2023-07-06T03:49:35.844361+0000","last_fullsized":"2023-07-06T03:49:35.844361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:51:30.705000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842826+0000","last_change":"2023-07-06T03:49:34.870611+0000","last_active":"2023-07-06T03:49:35.842826+0000","last_peered":"2023-07-06T03:49:35.842826+0000","last_clean":"2023-07-06T03:49:35.842826+0000","last_became_active":"2023-07-06T03:49:34.870083+0000","last_became_peered":"2023-07-06T03:49:34.870083+0000","last_unstale":"2023-07-06T03:49:35.842826+0000","last_undegraded":"2023-07-06T03:49:35.842826+0000","last_fullsized":"2023-07-06T03:49:35.842826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:26.019247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852890+0000","last_change":"2023-07-06T03:49:33.825662+0000","last_active":"2023-07-06T03:49:35.852890+0000","last_peered":"2023-07-06T03:49:35.852890+0000","last_clean":"2023-07-06T03:49:35.852890+0000","last_became_active":"2023-07-06T03:49:33.825400+0000","last_became_peered":"2023-07-06T03:49:33.825400+0000","last_unstale":"2023-07-06T03:49:35.852890+0000","last_undegraded":"2023-07-06T03:49:35.852890+0000","last_fullsized":"2023-07-06T03:49:35.852890+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:22:15.733559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852642+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:49:35.852642+0000","last_peered":"2023-07-06T03:49:35.852642+0000","last_clean":"2023-07-06T03:49:35.852642+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:49:35.852642+0000","last_undegraded":"2023-07-06T03:49:35.852642+0000","last_fullsized":"2023-07-06T03:49:35.852642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:01:24.805927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.854116+0000","last_change":"2023-07-06T03:49:34.870290+0000","last_active":"2023-07-06T03:49:35.854116+0000","last_peered":"2023-07-06T03:49:35.854116+0000","last_clean":"2023-07-06T03:49:35.854116+0000","last_became_active":"2023-07-06T03:49:34.870012+0000","last_became_peered":"2023-07-06T03:49:34.870012+0000","last_unstale":"2023-07-06T03:49:35.854116+0000","last_undegraded":"2023-07-06T03:49:35.854116+0000","last_fullsized":"2023-07-06T03:49:35.854116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:35:58.635737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852836+0000","last_change":"2023-07-06T03:49:33.830269+0000","last_active":"2023-07-06T03:49:35.852836+0000","last_peered":"2023-07-06T03:49:35.852836+0000","last_clean":"2023-07-06T03:49:35.852836+0000","last_became_active":"2023-07-06T03:49:33.829954+0000","last_became_peered":"2023-07-06T03:49:33.829954+0000","last_unstale":"2023-07-06T03:49:35.852836+0000","last_undegraded":"2023-07-06T03:49:35.852836+0000","last_fullsized":"2023-07-06T03:49:35.852836+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:28:41.278256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843005+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:49:35.843005+0000","last_peered":"2023-07-06T03:49:35.843005+0000","last_clean":"2023-07-06T03:49:35.843005+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:49:35.843005+0000","last_undegraded":"2023-07-06T03:49:35.843005+0000","last_fullsized":"2023-07-06T03:49:35.843005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:56:14.059897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853691+0000","last_change":"2023-07-06T03:49:34.871495+0000","last_active":"2023-07-06T03:49:35.853691+0000","last_peered":"2023-07-06T03:49:35.853691+0000","last_clean":"2023-07-06T03:49:35.853691+0000","last_became_active":"2023-07-06T03:49:34.871202+0000","last_became_peered":"2023-07-06T03:49:34.871202+0000","last_unstale":"2023-07-06T03:49:35.853691+0000","last_undegraded":"2023-07-06T03:49:35.853691+0000","last_fullsized":"2023-07-06T03:49:35.853691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:18:04.906859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844868+0000","last_change":"2023-07-06T03:49:33.828898+0000","last_active":"2023-07-06T03:49:35.844868+0000","last_peered":"2023-07-06T03:49:35.844868+0000","last_clean":"2023-07-06T03:49:35.844868+0000","last_became_active":"2023-07-06T03:49:33.828646+0000","last_became_peered":"2023-07-06T03:49:33.828646+0000","last_unstale":"2023-07-06T03:49:35.844868+0000","last_undegraded":"2023-07-06T03:49:35.844868+0000","last_fullsized":"2023-07-06T03:49:35.844868+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:50:44.305762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853466+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:49:35.853466+0000","last_peered":"2023-07-06T03:49:35.853466+0000","last_clean":"2023-07-06T03:49:35.853466+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:49:35.853466+0000","last_undegraded":"2023-07-06T03:49:35.853466+0000","last_fullsized":"2023-07-06T03:49:35.853466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:36:08.075483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.851969+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:49:35.851969+0000","last_peered":"2023-07-06T03:49:35.851969+0000","last_clean":"2023-07-06T03:49:35.851969+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:49:35.851969+0000","last_undegraded":"2023-07-06T03:49:35.851969+0000","last_fullsized":"2023-07-06T03:49:35.851969+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:46:08.582009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.843084+0000","last_change":"2023-07-06T03:49:34.870804+0000","last_active":"2023-07-06T03:49:35.843084+0000","last_peered":"2023-07-06T03:49:35.843084+0000","last_clean":"2023-07-06T03:49:35.843084+0000","last_became_active":"2023-07-06T03:49:34.870475+0000","last_became_peered":"2023-07-06T03:49:34.870475+0000","last_unstale":"2023-07-06T03:49:35.843084+0000","last_undegraded":"2023-07-06T03:49:35.843084+0000","last_fullsized":"2023-07-06T03:49:35.843084+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:35.842450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844670+0000","last_change":"2023-07-06T03:49:33.833602+0000","last_active":"2023-07-06T03:49:35.844670+0000","last_peered":"2023-07-06T03:49:35.844670+0000","last_clean":"2023-07-06T03:49:35.844670+0000","last_became_active":"2023-07-06T03:49:33.833331+0000","last_became_peered":"2023-07-06T03:49:33.833331+0000","last_unstale":"2023-07-06T03:49:35.844670+0000","last_undegraded":"2023-07-06T03:49:35.844670+0000","last_fullsized":"2023-07-06T03:49:35.844670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:35:43.748856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853042+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:49:35.853042+0000","last_peered":"2023-07-06T03:49:35.853042+0000","last_clean":"2023-07-06T03:49:35.853042+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:49:35.853042+0000","last_undegraded":"2023-07-06T03:49:35.853042+0000","last_fullsized":"2023-07-06T03:49:35.853042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:00:27.596531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842884+0000","last_change":"2023-07-06T03:49:34.856943+0000","last_active":"2023-07-06T03:49:35.842884+0000","last_peered":"2023-07-06T03:49:35.842884+0000","last_clean":"2023-07-06T03:49:35.842884+0000","last_became_active":"2023-07-06T03:49:34.856631+0000","last_became_peered":"2023-07-06T03:49:34.856631+0000","last_unstale":"2023-07-06T03:49:35.842884+0000","last_undegraded":"2023-07-06T03:49:35.842884+0000","last_fullsized":"2023-07-06T03:49:35.842884+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:10:03.558670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842893+0000","last_change":"2023-07-06T03:49:33.833448+0000","last_active":"2023-07-06T03:49:35.842893+0000","last_peered":"2023-07-06T03:49:35.842893+0000","last_clean":"2023-07-06T03:49:35.842893+0000","last_became_active":"2023-07-06T03:49:33.833163+0000","last_became_peered":"2023-07-06T03:49:33.833163+0000","last_unstale":"2023-07-06T03:49:35.842893+0000","last_undegraded":"2023-07-06T03:49:35.842893+0000","last_fullsized":"2023-07-06T03:49:35.842893+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:29.056482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853850+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:49:35.853850+0000","last_peered":"2023-07-06T03:49:35.853850+0000","last_clean":"2023-07-06T03:49:35.853850+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:49:35.853850+0000","last_undegraded":"2023-07-06T03:49:35.853850+0000","last_fullsized":"2023-07-06T03:49:35.853850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:43.221211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.842376+0000","last_change":"2023-07-06T03:49:34.851758+0000","last_active":"2023-07-06T03:49:35.842376+0000","last_peered":"2023-07-06T03:49:35.842376+0000","last_clean":"2023-07-06T03:49:35.842376+0000","last_became_active":"2023-07-06T03:49:34.851454+0000","last_became_peered":"2023-07-06T03:49:34.851454+0000","last_unstale":"2023-07-06T03:49:35.842376+0000","last_undegraded":"2023-07-06T03:49:35.842376+0000","last_fullsized":"2023-07-06T03:49:35.842376+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:57:46.131361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853495+0000","last_change":"2023-07-06T03:49:33.830097+0000","last_active":"2023-07-06T03:49:35.853495+0000","last_peered":"2023-07-06T03:49:35.853495+0000","last_clean":"2023-07-06T03:49:35.853495+0000","last_became_active":"2023-07-06T03:49:33.829804+0000","last_became_peered":"2023-07-06T03:49:33.829804+0000","last_unstale":"2023-07-06T03:49:35.853495+0000","last_undegraded":"2023-07-06T03:49:35.853495+0000","last_fullsized":"2023-07-06T03:49:35.853495+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:33:21.962429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.844202+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:49:35.844202+0000","last_peered":"2023-07-06T03:49:35.844202+0000","last_clean":"2023-07-06T03:49:35.844202+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:49:35.844202+0000","last_undegraded":"2023-07-06T03:49:35.844202+0000","last_fullsized":"2023-07-06T03:49:35.844202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:34:27.801302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.852957+0000","last_change":"2023-07-06T03:49:34.853054+0000","last_active":"2023-07-06T03:49:35.852957+0000","last_peered":"2023-07-06T03:49:35.852957+0000","last_clean":"2023-07-06T03:49:35.852957+0000","last_became_active":"2023-07-06T03:49:34.852779+0000","last_became_peered":"2023-07-06T03:49:34.852779+0000","last_unstale":"2023-07-06T03:49:35.852957+0000","last_undegraded":"2023-07-06T03:49:35.852957+0000","last_fullsized":"2023-07-06T03:49:35.852957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:33.804958+0000","last_clean_scrub_stamp":"2023-07-06T03:49:33.804958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:40:14.144373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.853361+0000","last_change":"2023-07-06T03:49:33.833515+0000","last_active":"2023-07-06T03:49:35.853361+0000","last_peered":"2023-07-06T03:49:35.853361+0000","last_clean":"2023-07-06T03:49:35.853361+0000","last_became_active":"2023-07-06T03:49:33.833183+0000","last_became_peered":"2023-07-06T03:49:33.833183+0000","last_unstale":"2023-07-06T03:49:35.853361+0000","last_undegraded":"2023-07-06T03:49:35.853361+0000","last_fullsized":"2023-07-06T03:49:35.853361+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:49:32.802235+0000","last_clean_scrub_stamp":"2023-07-06T03:49:32.802235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:58.747527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-07-06T03:49:35.854147+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:49:35.854147+0000","last_peered":"2023-07-06T03:49:35.854147+0000","last_clean":"2023-07-06T03:49:35.854147+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:49:35.854147+0000","last_undegraded":"2023-07-06T03:49:35.854147+0000","last_fullsized":"2023-07-06T03:49:35.854147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:11:38.112030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":897024,"data_stored":398800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69199999999999995,"15min":0.72199999999999998},"min":{"1min":0.32500000000000001,"5min":0.307,"15min":0.307},"max":{"1min":4.0570000000000004,"5min":4.0570000000000004,"15min":4.0570000000000004},"last":0.626},{"interface":"front","average":{"1min":0.63,"5min":0.72799999999999998,"15min":0.80800000000000005},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.95499999999999996,"5min":3.8660000000000001,"15min":3.8660000000000001},"last":0.378}]},{"osd":1,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69699999999999995,"15min":0.746},"min":{"1min":0.45500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91300000000000003,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.92300000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.0389999999999997,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.505}]},{"osd":2,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.74099999999999999,"15min":0.81899999999999995},"min":{"1min":0.495,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.878,"5min":3.77,"15min":3.77},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.70699999999999996},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92000000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.72799999999999998},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.95999999999999996,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.439,"5min":0.433,"15min":0.433},"max":{"1min":0.94199999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69099999999999995,"15min":0.67600000000000005},"min":{"1min":0.47799999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:49:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.748,"15min":0.746},"min":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.03,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.006,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.68400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":897024,"data_stored":398820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63800000000000001,"15min":0.65000000000000002},"min":{"1min":0.30599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94099999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.88},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.94899999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.64400000000000002,"15min":0.66600000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64300000000000002,"15min":0.67400000000000004},"min":{"1min":0.46400000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96399999999999997,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.90800000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63200000000000001,"15min":0.65200000000000002},"min":{"1min":0.45000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88800000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.67600000000000005,"15min":0.71299999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":1.119,"15min":1.119},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.70499999999999996,"15min":0.76600000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85499999999999998,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69499999999999995,"15min":0.73999999999999999},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.91100000000000003,"5min":1.093,"15min":1.093},"last":0.89500000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69599999999999995,"15min":0.72599999999999998},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98299999999999998,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70499999999999996,"15min":0.751},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86799999999999999,"5min":1.085,"15min":1.085},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.67800000000000005,"15min":0.72999999999999998},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93700000000000006,"5min":1.218,"15min":1.218},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.75700000000000001,"15min":0.80400000000000005},"min":{"1min":0.46899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95099999999999996,"5min":1.198,"15min":1.198},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.78300000000000003,"15min":0.84699999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.054,"5min":1.323,"15min":1.323},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78400000000000003,"15min":0.85499999999999998},"min":{"1min":0.504,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0009999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1511424,"data_stored":996767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.47999999999999998,"15min":0.56999999999999995},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.002,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.52200000000000002,"15min":0.58699999999999997},"min":{"1min":0.31900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.65000000000000002,"5min":1.96,"15min":1.96},"last":0.50900000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.72899999999999998,"15min":1.0209999999999999},"min":{"1min":0.29399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70799999999999996,"5min":13.055999999999999,"15min":13.055999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.57899999999999996,"15min":0.65900000000000003},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.92400000000000004,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.56399999999999995,"15min":0.63300000000000001},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.96799999999999997,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.51000000000000001,"15min":0.56000000000000005},"min":{"1min":0.34599999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60199999999999998,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.99099999999999999,"15min":1.2789999999999999},"min":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0640000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.78900000000000003,"15min":0.84299999999999997},"min":{"1min":0.54800000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0800000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73699999999999999,"15min":0.78500000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86099999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.99099999999999999,"15min":1.2929999999999999},"min":{"1min":0.44700000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1020000000000001,"5min":13.023999999999999,"15min":13.023999999999999},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77600000000000002,"15min":0.82999999999999996},"min":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.83899999999999997,"5min":1.365,"15min":1.365},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.0329999999999999,"15min":1.3360000000000001},"min":{"1min":0.48699999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91600000000000004,"5min":12.941000000000001,"15min":12.941000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:49:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77700000000000002,"15min":0.85099999999999998},"min":{"1min":0.51200000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0409999999999999,"5min":2.806,"15min":2.806},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.77700000000000002,"15min":0.82799999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.88700000000000001,"5min":1.337,"15min":1.337},"last":0.84999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31932,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339908,"statfs":{"total":96636764160,"available":96604065792,"internally_reserved":0,"allocated":905216,"data_stored":404823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40200000000000002,"15min":0.41299999999999998},"min":{"1min":0.26400000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54600000000000004,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42899999999999999,"15min":0.436},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.60399999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.377}]},{"osd":2,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.47599999999999998},"min":{"1min":0.33000000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.66700000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.432},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48699999999999999,"15min":0.505},"min":{"1min":0.32700000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.60399999999999998,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.30099999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71999999999999997,"15min":0.751},"min":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86899999999999999,"5min":1.171,"15min":1.171},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.99099999999999999,"5min":1.034,"15min":1.034},"last":0.67100000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.47799999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.02,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.72499999999999998,"15min":0.754},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92100000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72799999999999998,"15min":0.752},"min":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.86399999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.006,"5min":1.546,"15min":1.546},"last":0.74099999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77900000000000003,"15min":0.80800000000000005},"min":{"1min":0.56899999999999995,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.93300000000000005,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.93600000000000005,"15min":1.1559999999999999},"min":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.94999999999999996,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36012,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335828,"statfs":{"total":96636764160,"available":96599887872,"internally_reserved":0,"allocated":888832,"data_stored":398148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.371}]},{"osd":1,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48299999999999998,"15min":0.49199999999999999},"min":{"1min":0.33500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.68100000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.5},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91300000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77400000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.46400000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71699999999999997,"15min":0.70199999999999996},"min":{"1min":0.50800000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.075,"5min":1.165,"15min":1.165},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73199999999999998,"15min":0.72799999999999998},"min":{"1min":0.56799999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96499999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.754,"15min":0.754},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0549999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71999999999999997,"15min":0.70799999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94799999999999995,"5min":1.046,"15min":1.046},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.55000000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.012,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:49:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.79600000000000004,"15min":0.82999999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.087,"5min":2.157,"15min":2.157},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.77900000000000003,"15min":0.79600000000000004},"min":{"1min":0.56000000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0600000000000001,"5min":1.966,"15min":1.966},"last":0.73699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":909312,"data_stored":399816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.52600000000000002,"15min":0.65400000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.55500000000000005,"5min":5.6550000000000002,"15min":5.6550000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48999999999999999,"15min":0.51500000000000001},"min":{"1min":0.33600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63800000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.38}]},{"osd":1,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.80500000000000005,"15min":1.196},"min":{"1min":0.29299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.745,"5min":15.911,"15min":15.911},"last":0.311},{"interface":"front","average":{"1min":0.501,"5min":0.876,"15min":1.266},"min":{"1min":0.36299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69099999999999995,"5min":16.024000000000001,"15min":16.024000000000001},"last":0.49199999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.46700000000000003,"15min":0.50900000000000001},"min":{"1min":0.32800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57699999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.878,"15min":1.27},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.55700000000000005,"5min":16.068999999999999,"15min":16.068999999999999},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.76800000000000002},"min":{"1min":0.51000000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.071,"5min":1.179,"15min":1.179},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.83999999999999997,"15min":0.98199999999999998},"min":{"1min":0.46100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.085,"5min":5.7300000000000004,"15min":5.7300000000000004},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.84299999999999997,"15min":0.98099999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1080000000000001,"5min":5.077,"15min":5.077},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.88500000000000001,"15min":1.042},"min":{"1min":0.499,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95199999999999996,"5min":5.6980000000000004,"15min":5.6980000000000004},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.75700000000000001,"15min":0.81499999999999995},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94399999999999995,"5min":1.143,"15min":1.143},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.84899999999999998,"15min":0.997},"min":{"1min":0.48799999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.032,"5min":5.157,"15min":5.157},"last":0.621}]},{"osd":7,"last update":"Thu Jul 6 03:49:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78300000000000003,"15min":0.85499999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96599999999999997,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.749},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.86699999999999999,"15min":1.0089999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1399999999999999,"5min":5.1189999999999998,"15min":5.1189999999999998},"last":0.68400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36016,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335824,"statfs":{"total":96636764160,"available":96599883776,"internally_reserved":0,"allocated":892928,"data_stored":398170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.63600000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88700000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.77000000000000002,"15min":0.91100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84899999999999998,"5min":6.258,"15min":6.258},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65700000000000003},"min":{"1min":0.46800000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.78700000000000003,"15min":0.92400000000000004},"min":{"1min":0.38300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.85299999999999998,"15min":1.0129999999999999},"min":{"1min":0.54700000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95099999999999996,"5min":6.4859999999999998,"15min":6.4859999999999998},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79800000000000004,"5min":1.25,"15min":1.25},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93700000000000006,"5min":1.413,"15min":1.413},"last":0.745}]},{"osd":5,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.67400000000000004},"min":{"1min":0.42399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63300000000000001,"15min":0.67900000000000005},"min":{"1min":0.40200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82499999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64500000000000002,"15min":0.67400000000000004},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80900000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76800000000000002,"15min":0.80500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jul 6 03:49:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.70699999999999996,"15min":0.74099999999999999},"min":{"1min":0.44700000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80900000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.76500000000000001},"min":{"1min":0.46500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.079,"5min":1.21,"15min":1.21},"last":0.63}]}]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1482752,"data_stored":987587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.66900000000000004,"15min":0.65300000000000002},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":4.5010000000000003,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.77000000000000002,"15min":0.91000000000000003},"min":{"1min":0.38100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.873,"5min":6.4059999999999997,"15min":6.4059999999999997},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64400000000000002,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.65800000000000003,"15min":0.67900000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":1.546,"15min":1.546},"last":1.1970000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.70499999999999996},"min":{"1min":0.42899999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.91100000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.879,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.93000000000000005}]},{"osd":3,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91500000000000004,"5min":1.03,"15min":1.03},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.747,"15min":0.76300000000000001},"min":{"1min":0.53000000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.484,"5min":4.484,"15min":4.484},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.621,"15min":0.629},"min":{"1min":0.45300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90600000000000003,"5min":2.718,"15min":2.718},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69799999999999995,"15min":0.68600000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92400000000000004,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89600000000000002,"5min":1.669,"15min":1.669},"last":1.246}]},{"osd":7,"last update":"Thu Jul 6 03:49:01 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.77800000000000002},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93600000000000005,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.3360000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54100000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97999999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.97699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:49:39.700 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T03:49:39.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T03:49:39.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.934+0000 7fbc389ff700 1 Processor -- start 2023-07-06T03:49:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.934+0000 7fbc389ff700 1 -- start start 2023-07-06T03:49:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.935+0000 7fbc389ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbc340cc530 0x7fbc340cab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.935+0000 7fbc389ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340adb70 0x7fbc3406fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.935+0000 7fbc389ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc340c9610 con 0x7fbc340cd620 2023-07-06T03:49:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.935+0000 7fbc389ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc340c9050 con 0x7fbc340cc530 2023-07-06T03:49:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.935+0000 7fbc389ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340c9330 con 0x7fbc340adb70 2023-07-06T03:49:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.935+0000 7fbc31d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbc340cd620 0x7fbc340cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49364/0 (socket says 172.21.15.97:49364) 2023-07-06T03:49:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.935+0000 7fbc31d9b700 1 -- 172.21.15.97:0/2827127469 learned_addr learned my addr 172.21.15.97:0/2827127469 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.936+0000 7fbc3259c700 1 --2- 172.21.15.97:0/2827127469 >> v2:172.21.15.146:3300/0 conn(0x7fbc340cc530 0x7fbc340cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.936+0000 7fbc32d9d700 1 --2- 172.21.15.97:0/2827127469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340adb70 0x7fbc3406fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.936+0000 7fbc3159a700 1 -- 172.21.15.97:0/2827127469 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713964127 0 0) 0x7fbc340c9610 con 0x7fbc340cd620 2023-07-06T03:49:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.936+0000 7fbc3159a700 1 -- 172.21.15.97:0/2827127469 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc1c0035b0 con 0x7fbc340cd620 2023-07-06T03:49:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.936+0000 7fbc32d9d700 1 -- 172.21.15.97:0/2827127469 >> v2:172.21.15.146:3300/0 conn(0x7fbc340cc530 msgr2=0x7fbc340cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.936+0000 7fbc32d9d700 1 --2- 172.21.15.97:0/2827127469 >> v2:172.21.15.146:3300/0 conn(0x7fbc340cc530 0x7fbc340cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.936+0000 7fbc32d9d700 1 -- 172.21.15.97:0/2827127469 >> v1:172.21.15.97:6789/0 conn(0x7fbc340cd620 legacy=0x7fbc340cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.936+0000 7fbc3159a700 1 -- 172.21.15.97:0/2827127469 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1054095822 0 0) 0x7fbc1c0035b0 con 0x7fbc340cd620 2023-07-06T03:49:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.936+0000 7fbc32d9d700 1 -- 172.21.15.97:0/2827127469 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc340cd100 con 0x7fbc340adb70 2023-07-06T03:49:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.937+0000 7fbc32d9d700 1 --2- 172.21.15.97:0/2827127469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340adb70 0x7fbc3406fae0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009d90 tx=0x7fbc2c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9848438d7f775726 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.937+0000 7fbc3159a700 1 -- 172.21.15.97:0/2827127469 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00e070 con 0x7fbc340adb70 2023-07-06T03:49:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.937+0000 7fbc3159a700 1 -- 172.21.15.97:0/2827127469 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c003a20 con 0x7fbc340adb70 2023-07-06T03:49:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.937+0000 7fbc3159a700 1 -- 172.21.15.97:0/2827127469 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc28001670 con 0x7fbc340adb70 2023-07-06T03:49:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.937+0000 7fbc389ff700 1 -- 172.21.15.97:0/2827127469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340adb70 msgr2=0x7fbc3406fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.937+0000 7fbc389ff700 1 --2- 172.21.15.97:0/2827127469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340adb70 0x7fbc3406fae0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009d90 tx=0x7fbc2c00b330 comp rx=0 tx=0).stop 2023-07-06T03:49:39.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.937+0000 7fbc389ff700 1 -- 172.21.15.97:0/2827127469 shutdown_connections 2023-07-06T03:49:39.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.937+0000 7fbc389ff700 1 --2- 172.21.15.97:0/2827127469 >> v2:172.21.15.146:3300/0 conn(0x7fbc340cc530 0x7fbc340cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.937+0000 7fbc389ff700 1 --2- 172.21.15.97:0/2827127469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340adb70 0x7fbc3406fae0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.937+0000 7fbc389ff700 1 -- 172.21.15.97:0/2827127469 >> 172.21.15.97:0/2827127469 conn(0x7fbc340b05d0 msgr2=0x7fbc340b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:39.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.937+0000 7fbc389ff700 1 -- 172.21.15.97:0/2827127469 shutdown_connections 2023-07-06T03:49:39.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.938+0000 7fbc389ff700 1 -- 172.21.15.97:0/2827127469 wait complete. 2023-07-06T03:49:39.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.939+0000 7fbc389ff700 1 Processor -- start 2023-07-06T03:49:39.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.939+0000 7fbc389ff700 1 -- start start 2023-07-06T03:49:39.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.940+0000 7fbc389ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc340adb70 0x7fbc340b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.940+0000 7fbc389ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340cc530 0x7fbc340af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.940+0000 7fbc389ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc340cd620 0x7fbc34153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.940+0000 7fbc389ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340b2f40 con 0x7fbc340adb70 2023-07-06T03:49:39.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.940+0000 7fbc389ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340b3220 con 0x7fbc340cd620 2023-07-06T03:49:39.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.940+0000 7fbc389ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340b3500 con 0x7fbc340cc530 2023-07-06T03:49:39.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.940+0000 7fbc3259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc340adb70 0x7fbc340b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.940+0000 7fbc3259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc340adb70 0x7fbc340b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44668/0 (socket says 172.21.15.97:44668) 2023-07-06T03:49:39.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.940+0000 7fbc3259c700 1 -- 172.21.15.97:0/3792657954 learned_addr learned my addr 172.21.15.97:0/3792657954 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:39.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc31d9b700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340cc530 0x7fbc340af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc32d9d700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc340cd620 0x7fbc34153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc3259c700 1 -- 172.21.15.97:0/3792657954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340cc530 msgr2=0x7fbc340af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc3259c700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340cc530 0x7fbc340af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc3259c700 1 -- 172.21.15.97:0/3792657954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc340cd620 msgr2=0x7fbc34153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:39.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc3259c700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc340cd620 0x7fbc34153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:39.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc3259c700 1 -- 172.21.15.97:0/3792657954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c009a70 con 0x7fbc340adb70 2023-07-06T03:49:39.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc31d9b700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340cc530 0x7fbc340af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:39.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc3259c700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc340adb70 0x7fbc340b3fe0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400e9b0 tx=0x7fbc2400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:39.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc22ffd700 1 -- 172.21.15.97:0/3792657954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24009710 con 0x7fbc340adb70 2023-07-06T03:49:39.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc22ffd700 1 -- 172.21.15.97:0/3792657954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2400c850 con 0x7fbc340adb70 2023-07-06T03:49:39.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc389ff700 1 -- 172.21.15.97:0/3792657954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc340b37e0 con 0x7fbc340adb70 2023-07-06T03:49:39.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.941+0000 7fbc22ffd700 1 -- 172.21.15.97:0/3792657954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400cbc0 con 0x7fbc340adb70 2023-07-06T03:49:39.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.942+0000 7fbc389ff700 1 -- 172.21.15.97:0/3792657954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc340af7d0 con 0x7fbc340adb70 2023-07-06T03:49:39.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.942+0000 7fbc389ff700 1 -- 172.21.15.97:0/3792657954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3404e340 con 0x7fbc340adb70 2023-07-06T03:49:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.943+0000 7fbc22ffd700 1 -- 172.21.15.97:0/3792657954 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbc2400cd20 con 0x7fbc340adb70 2023-07-06T03:49:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.944+0000 7fbc22ffd700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc1406ad10 0x7fbc1406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.944+0000 7fbc22ffd700 1 -- 172.21.15.97:0/3792657954 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbc240894f0 con 0x7fbc340adb70 2023-07-06T03:49:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.945+0000 7fbc31d9b700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc1406ad10 0x7fbc1406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:39.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.945+0000 7fbc31d9b700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc1406ad10 0x7fbc1406d1c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbc28007fd0 tx=0x7fbc28007b20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:39.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:39.949+0000 7fbc22ffd700 1 -- 172.21.15.97:0/3792657954 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbc24054500 con 0x7fbc340adb70 2023-07-06T03:49:40.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.217+0000 7fbc389ff700 1 -- 172.21.15.97:0/3792657954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbc340c8d20 con 0x7fbc340adb70 2023-07-06T03:49:40.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.221+0000 7fbc22ffd700 1 -- 172.21.15.97:0/3792657954 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbc24057b20 con 0x7fbc340adb70 2023-07-06T03:49:40.223 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:49:40.223 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg== 2023-07-06T03:49:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-07-06T03:49:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-07-06T03:49:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T03:49:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 -- 172.21.15.97:0/3792657954 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc1406ad10 msgr2=0x7fbc1406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc1406ad10 0x7fbc1406d1c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbc28007fd0 tx=0x7fbc28007b20 comp rx=0 tx=0).stop 2023-07-06T03:49:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 -- 172.21.15.97:0/3792657954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc340adb70 msgr2=0x7fbc340b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc340adb70 0x7fbc340b3fe0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400e9b0 tx=0x7fbc2400ecc0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 -- 172.21.15.97:0/3792657954 shutdown_connections 2023-07-06T03:49:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc1406ad10 0x7fbc1406d1c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc340cd620 0x7fbc34153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc340adb70 0x7fbc340b3fe0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 --2- 172.21.15.97:0/3792657954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc340cc530 0x7fbc340af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 -- 172.21.15.97:0/3792657954 >> 172.21.15.97:0/3792657954 conn(0x7fbc340b05d0 msgr2=0x7fbc340c3840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 -- 172.21.15.97:0/3792657954 shutdown_connections 2023-07-06T03:49:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.224+0000 7fbc389ff700 1 -- 172.21.15.97:0/3792657954 wait complete. 2023-07-06T03:49:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-07-06T03:49:40.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:49:40.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.437+0000 7fcb9abff700 1 Processor -- start 2023-07-06T03:49:40.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.437+0000 7fcb9abff700 1 -- start start 2023-07-06T03:49:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9abff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcb940cd390 0x7fcb94160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9abff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcb94160dd0 0x7fcb940b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9abff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb940cdae0 con 0x7fcb9415c520 2023-07-06T03:49:40.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9abff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb940b6640 con 0x7fcb94160dd0 2023-07-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9abff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb940b6920 con 0x7fcb940cd390 2023-07-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9919c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcb94160dd0 0x7fcb940b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9919c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcb94160dd0 0x7fcb940b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44676/0 (socket says 172.21.15.97:44676) 2023-07-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9919c700 1 -- 172.21.15.97:0/3742228856 learned_addr learned my addr 172.21.15.97:0/3742228856 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb93fff700 1 --2- 172.21.15.97:0/3742228856 >> v2:172.21.15.146:3300/0 conn(0x7fcb940cd390 0x7fcb94160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9919c700 1 -- 172.21.15.97:0/3742228856 >> v2:172.21.15.146:3300/0 conn(0x7fcb940cd390 msgr2=0x7fcb94160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9919c700 1 --2- 172.21.15.97:0/3742228856 >> v2:172.21.15.146:3300/0 conn(0x7fcb940cd390 0x7fcb94160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9919c700 1 -- 172.21.15.97:0/3742228856 >> v1:172.21.15.146:6789/0 conn(0x7fcb9415c520 legacy=0x7fcb94160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.439+0000 7fcb9919c700 1 -- 172.21.15.97:0/3742228856 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb940b6c00 con 0x7fcb94160dd0 2023-07-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.440+0000 7fcb9919c700 1 --2- 172.21.15.97:0/3742228856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb94160dd0 0x7fcb940b7400 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fcb940b5930 tx=0x7fcb8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d453e81f500ca1e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.440+0000 7fcb937fe700 1 -- 172.21.15.97:0/3742228856 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00e070 con 0x7fcb94160dd0 2023-07-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.440+0000 7fcb937fe700 1 -- 172.21.15.97:0/3742228856 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c00b910 con 0x7fcb94160dd0 2023-07-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.440+0000 7fcb937fe700 1 -- 172.21.15.97:0/3742228856 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00bcc0 con 0x7fcb94160dd0 2023-07-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.440+0000 7fcb9abff700 1 -- 172.21.15.97:0/3742228856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb94160dd0 msgr2=0x7fcb940b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.440+0000 7fcb9abff700 1 --2- 172.21.15.97:0/3742228856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb94160dd0 0x7fcb940b7400 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fcb940b5930 tx=0x7fcb8c00b330 comp rx=0 tx=0).stop 2023-07-06T03:49:40.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.440+0000 7fcb9abff700 1 -- 172.21.15.97:0/3742228856 shutdown_connections 2023-07-06T03:49:40.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.441+0000 7fcb9abff700 1 --2- 172.21.15.97:0/3742228856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb94160dd0 0x7fcb940b7400 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.441+0000 7fcb9abff700 1 --2- 172.21.15.97:0/3742228856 >> v2:172.21.15.146:3300/0 conn(0x7fcb940cd390 0x7fcb94160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.441+0000 7fcb9abff700 1 -- 172.21.15.97:0/3742228856 >> 172.21.15.97:0/3742228856 conn(0x7fcb94071790 msgr2=0x7fcb94071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:40.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.441+0000 7fcb9abff700 1 -- 172.21.15.97:0/3742228856 shutdown_connections 2023-07-06T03:49:40.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.441+0000 7fcb9abff700 1 -- 172.21.15.97:0/3742228856 wait complete. 2023-07-06T03:49:40.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.442+0000 7fcb9abff700 1 Processor -- start 2023-07-06T03:49:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.442+0000 7fcb9abff700 1 -- start start 2023-07-06T03:49:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.443+0000 7fcb9abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb940cd390 0x7fcb940cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.443+0000 7fcb9abff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9415c520 0x7fcb94158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.443+0000 7fcb9abff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb94160dd0 0x7fcb94155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.443+0000 7fcb9abff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940cb7f0 con 0x7fcb9415c520 2023-07-06T03:49:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.443+0000 7fcb9abff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940cb590 con 0x7fcb94160dd0 2023-07-06T03:49:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.443+0000 7fcb9abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940d2350 con 0x7fcb940cd390 2023-07-06T03:49:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb93fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9415c520 0x7fcb94158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb93fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9415c520 0x7fcb94158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44684/0 (socket says 172.21.15.97:44684) 2023-07-06T03:49:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb93fff700 1 -- 172.21.15.97:0/1717567494 learned_addr learned my addr 172.21.15.97:0/1717567494 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb9919c700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb94160dd0 0x7fcb94155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb9899b700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb940cd390 0x7fcb940cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb93fff700 1 -- 172.21.15.97:0/1717567494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb940cd390 msgr2=0x7fcb940cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb93fff700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb940cd390 0x7fcb940cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb93fff700 1 -- 172.21.15.97:0/1717567494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb94160dd0 msgr2=0x7fcb94155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:40.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb93fff700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb94160dd0 0x7fcb94155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb93fff700 1 -- 172.21.15.97:0/1717567494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c009b30 con 0x7fcb9415c520 2023-07-06T03:49:40.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb9919c700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb94160dd0 0x7fcb94155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:40.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb9899b700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb940cd390 0x7fcb940cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:40.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.444+0000 7fcb93fff700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9415c520 0x7fcb94158190 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000e870 tx=0x7fcb8000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:40.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.445+0000 7fcb917fa700 1 -- 172.21.15.97:0/1717567494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb80009590 con 0x7fcb9415c520 2023-07-06T03:49:40.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.445+0000 7fcb917fa700 1 -- 172.21.15.97:0/1717567494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8000ee70 con 0x7fcb9415c520 2023-07-06T03:49:40.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.445+0000 7fcb9abff700 1 -- 172.21.15.97:0/1717567494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb940d2630 con 0x7fcb9415c520 2023-07-06T03:49:40.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.445+0000 7fcb917fa700 1 -- 172.21.15.97:0/1717567494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000ca50 con 0x7fcb9415c520 2023-07-06T03:49:40.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.445+0000 7fcb9abff700 1 -- 172.21.15.97:0/1717567494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb940d2bf0 con 0x7fcb9415c520 2023-07-06T03:49:40.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.446+0000 7fcb9abff700 1 -- 172.21.15.97:0/1717567494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb940d2db0 con 0x7fcb9415c520 2023-07-06T03:49:40.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.447+0000 7fcb917fa700 1 -- 172.21.15.97:0/1717567494 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb8000cbb0 con 0x7fcb9415c520 2023-07-06T03:49:40.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.448+0000 7fcb917fa700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb7806ace0 0x7fcb7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.448+0000 7fcb917fa700 1 -- 172.21.15.97:0/1717567494 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb800894a0 con 0x7fcb9415c520 2023-07-06T03:49:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.448+0000 7fcb9899b700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb7806ace0 0x7fcb7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.449+0000 7fcb9899b700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb7806ace0 0x7fcb7806d190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000f60 tx=0x7fcb88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.452+0000 7fcb917fa700 1 -- 172.21.15.97:0/1717567494 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcb80057fd0 con 0x7fcb9415c520 2023-07-06T03:49:40.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.686+0000 7fcb9abff700 1 -- 172.21.15.97:0/1717567494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb940d2f60 con 0x7fcb9415c520 2023-07-06T03:49:40.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.687+0000 7fcb917fa700 1 -- 172.21.15.97:0/1717567494 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4679 (secure 0 0 0) 0x7fcb8000ce60 con 0x7fcb9415c520 2023-07-06T03:49:40.689 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:40.690 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2344840493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2344840493},{"type":"v1","addr":"172.21.15.97:6836","nonce":2344840493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1930131645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1930131645},{"type":"v1","addr":"172.21.15.146:6835","nonce":1930131645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3936349837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3936349837},{"type":"v1","addr":"172.21.15.146:6833","nonce":3936349837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:49:34.310076+0000","modified":"2023-07-06T03:49:37.864717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6837":{"gid":6837,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6837/1957695290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1957695290},{"type":"v1","addr":"172.21.15.97:6837","nonce":1957695290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-06T03:49:40.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.689+0000 7fcb9abff700 1 -- 172.21.15.97:0/1717567494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb7806ace0 msgr2=0x7fcb7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:40.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.689+0000 7fcb9abff700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb7806ace0 0x7fcb7806d190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000f60 tx=0x7fcb88009040 comp rx=0 tx=0).stop 2023-07-06T03:49:40.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.689+0000 7fcb9abff700 1 -- 172.21.15.97:0/1717567494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9415c520 msgr2=0x7fcb94158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:40.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.689+0000 7fcb9abff700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9415c520 0x7fcb94158190 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000e870 tx=0x7fcb8000ec30 comp rx=0 tx=0).stop 2023-07-06T03:49:40.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.689+0000 7fcb9abff700 1 -- 172.21.15.97:0/1717567494 shutdown_connections 2023-07-06T03:49:40.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.689+0000 7fcb9abff700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb7806ace0 0x7fcb7806d190 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.689+0000 7fcb9abff700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb94160dd0 0x7fcb94155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.689+0000 7fcb9abff700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb940cd390 0x7fcb940cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.689+0000 7fcb9abff700 1 --2- 172.21.15.97:0/1717567494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9415c520 0x7fcb94158190 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:40.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.689+0000 7fcb9abff700 1 -- 172.21.15.97:0/1717567494 >> 172.21.15.97:0/1717567494 conn(0x7fcb94071790 msgr2=0x7fcb941605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:40.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.689+0000 7fcb9abff700 1 -- 172.21.15.97:0/1717567494 shutdown_connections 2023-07-06T03:49:40.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:40.690+0000 7fcb9abff700 1 -- 172.21.15.97:0/1717567494 wait complete. 2023-07-06T03:49:40.695 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 44 2023-07-06T03:49:41.111 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T03:49:34.310076+0000', 'modified': '2023-07-06T03:49:37.864717+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6837}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6837': {'gid': 6837, 'name': 'a', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.97:6837/1957695290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6835', 'nonce': 1957695290}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 1957695290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T03:49:41.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T03:49:41.112 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-06T03:49:41.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.306+0000 7f41d33ff700 1 Processor -- start 2023-07-06T03:49:41.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.306+0000 7f41d33ff700 1 -- start start 2023-07-06T03:49:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.307+0000 7f41d33ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f41cc15c1c0 0x7f41cc0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.307+0000 7f41d33ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41cc15da10 con 0x7f41cc15c590 2023-07-06T03:49:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.307+0000 7f41d33ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41cc0ccc30 con 0x7f41cc0719c0 2023-07-06T03:49:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.308+0000 7f41d33ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41cc0ccf10 con 0x7f41cc15c1c0 2023-07-06T03:49:41.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.308+0000 7f41d119b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f41cc15c1c0 0x7f41cc0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.308+0000 7f41d099a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f41cc15c590 0x7f41cc15d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49372/0 (socket says 172.21.15.97:49372) 2023-07-06T03:49:41.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.308+0000 7f41d099a700 1 -- 172.21.15.97:0/1450770644 learned_addr learned my addr 172.21.15.97:0/1450770644 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:41.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.308+0000 7f41d119b700 1 -- 172.21.15.97:0/1450770644 >> v1:172.21.15.146:6789/0 conn(0x7f41cc0719c0 legacy=0x7f41cc06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.308+0000 7f41d119b700 1 -- 172.21.15.97:0/1450770644 >> v1:172.21.15.97:6789/0 conn(0x7f41cc15c590 legacy=0x7f41cc15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.308+0000 7f41d119b700 1 -- 172.21.15.97:0/1450770644 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc0cd1f0 con 0x7f41cc15c1c0 2023-07-06T03:49:41.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.308+0000 7f41c3fff700 1 -- 172.21.15.97:0/1450770644 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605202889 0 0) 0x7f41bc001110 con 0x7f41cc15c590 2023-07-06T03:49:41.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.309+0000 7f41d119b700 1 --2- 172.21.15.97:0/1450770644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41cc15c1c0 0x7f41cc0cc3e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009840 tx=0x7f41c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9802e4d7ebf315e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:41.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.309+0000 7f41c3fff700 1 -- 172.21.15.97:0/1450770644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8011830 con 0x7f41cc15c1c0 2023-07-06T03:49:41.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.309+0000 7f41c3fff700 1 -- 172.21.15.97:0/1450770644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c8011990 con 0x7f41cc15c1c0 2023-07-06T03:49:41.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.309+0000 7f41c3fff700 1 -- 172.21.15.97:0/1450770644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8011d20 con 0x7f41cc15c1c0 2023-07-06T03:49:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.309+0000 7f41d33ff700 1 -- 172.21.15.97:0/1450770644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41cc15c1c0 msgr2=0x7f41cc0cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.309+0000 7f41d33ff700 1 --2- 172.21.15.97:0/1450770644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41cc15c1c0 0x7f41cc0cc3e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009840 tx=0x7f41c8009b50 comp rx=0 tx=0).stop 2023-07-06T03:49:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.309+0000 7f41d33ff700 1 -- 172.21.15.97:0/1450770644 shutdown_connections 2023-07-06T03:49:41.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.309+0000 7f41d33ff700 1 --2- 172.21.15.97:0/1450770644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41cc15c1c0 0x7f41cc0cc3e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.309+0000 7f41d33ff700 1 -- 172.21.15.97:0/1450770644 >> 172.21.15.97:0/1450770644 conn(0x7f41cc0b1650 msgr2=0x7f41cc0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:41.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.309+0000 7f41d33ff700 1 -- 172.21.15.97:0/1450770644 shutdown_connections 2023-07-06T03:49:41.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.310+0000 7f41d33ff700 1 -- 172.21.15.97:0/1450770644 wait complete. 2023-07-06T03:49:41.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.311+0000 7f41d33ff700 1 Processor -- start 2023-07-06T03:49:41.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.311+0000 7f41d33ff700 1 -- start start 2023-07-06T03:49:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.312+0000 7f41d33ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41cc0719c0 0x7f41cc150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.312+0000 7f41d33ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41cc15c1c0 0x7f41cc0b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.312+0000 7f41d33ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41cc15c590 0x7f41cc06ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.312+0000 7f41d33ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc160940 con 0x7f41cc0719c0 2023-07-06T03:49:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.312+0000 7f41d33ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc15f810 con 0x7f41cc15c590 2023-07-06T03:49:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.312+0000 7f41d33ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc15fca0 con 0x7f41cc15c1c0 2023-07-06T03:49:41.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.312+0000 7f41d119b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41cc0719c0 0x7f41cc150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.312+0000 7f41d119b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41cc0719c0 0x7f41cc150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44700/0 (socket says 172.21.15.97:44700) 2023-07-06T03:49:41.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.312+0000 7f41d119b700 1 -- 172.21.15.97:0/718682527 learned_addr learned my addr 172.21.15.97:0/718682527 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:41.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d199c700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41cc15c590 0x7f41cc06ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d099a700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41cc15c1c0 0x7f41cc0b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d119b700 1 -- 172.21.15.97:0/718682527 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41cc15c1c0 msgr2=0x7f41cc0b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d119b700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41cc15c1c0 0x7f41cc0b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d119b700 1 -- 172.21.15.97:0/718682527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41cc15c590 msgr2=0x7f41cc06ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d119b700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41cc15c590 0x7f41cc06ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d119b700 1 -- 172.21.15.97:0/718682527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c80094a0 con 0x7f41cc0719c0 2023-07-06T03:49:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d199c700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41cc15c590 0x7f41cc06ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d099a700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41cc15c1c0 0x7f41cc0b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d119b700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41cc0719c0 0x7f41cc150970 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f41c80090b0 tx=0x7f41c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:41.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41c1ffb700 1 -- 172.21.15.97:0/718682527 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c801a980 con 0x7f41cc0719c0 2023-07-06T03:49:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41c1ffb700 1 -- 172.21.15.97:0/718682527 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c801ab00 con 0x7f41cc0719c0 2023-07-06T03:49:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d33ff700 1 -- 172.21.15.97:0/718682527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc15ec00 con 0x7f41cc0719c0 2023-07-06T03:49:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41c1ffb700 1 -- 172.21.15.97:0/718682527 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8024420 con 0x7f41cc0719c0 2023-07-06T03:49:41.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.313+0000 7f41d33ff700 1 -- 172.21.15.97:0/718682527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc15f1c0 con 0x7f41cc0719c0 2023-07-06T03:49:41.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.315+0000 7f41c1ffb700 1 -- 172.21.15.97:0/718682527 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41c8022070 con 0x7f41cc0719c0 2023-07-06T03:49:41.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.316+0000 7f41c1ffb700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41b006ace0 0x7f41b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.316+0000 7f41c1ffb700 1 -- 172.21.15.97:0/718682527 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f41c808a8a0 con 0x7f41cc0719c0 2023-07-06T03:49:41.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.317+0000 7f41d099a700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41b006ace0 0x7f41b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.317+0000 7f41d099a700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41b006ace0 0x7f41b006d190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f41cc15bda0 tx=0x7f41bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:41.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.318+0000 7f41e6e1cb80 1 Processor -- start 2023-07-06T03:49:41.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.318+0000 7f41e6e1cb80 1 -- start start 2023-07-06T03:49:41.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.319+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5160d50 0x5618b5161120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.319+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5166190 0x5618b5184a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.319+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b513e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.319+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b50b2ea0 con 0x5618b513bb50 2023-07-06T03:49:41.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.319+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b5026600 con 0x5618b5160d50 2023-07-06T03:49:41.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.319+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618b5095910 con 0x5618b5166190 2023-07-06T03:49:41.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.319+0000 7f41d199c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b513e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41d199c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b513e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44706/0 (socket says 172.21.15.97:44706) 2023-07-06T03:49:41.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41d199c700 1 -- 172.21.15.97:0/724740387 learned_addr learned my addr 172.21.15.97:0/724740387 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:41.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41d119b700 1 --2- 172.21.15.97:0/724740387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5160d50 0x5618b5161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41d099a700 1 --2- 172.21.15.97:0/724740387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5166190 0x5618b5184a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41d199c700 1 -- 172.21.15.97:0/724740387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5166190 msgr2=0x5618b5184a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:41.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41d199c700 1 --2- 172.21.15.97:0/724740387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5166190 0x5618b5184a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41d199c700 1 -- 172.21.15.97:0/724740387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5160d50 msgr2=0x5618b5161120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:41.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41d199c700 1 --2- 172.21.15.97:0/724740387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5160d50 0x5618b5161120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41d199c700 1 -- 172.21.15.97:0/724740387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618b4e82790 con 0x5618b513bb50 2023-07-06T03:49:41.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41d199c700 1 --2- 172.21.15.97:0/724740387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b513e110 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f41c4002180 tx=0x7f41c400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c7ab2b5051a18 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:41.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41b6ffd700 1 -- 172.21.15.97:0/724740387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c400b070 con 0x5618b513bb50 2023-07-06T03:49:41.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41b6ffd700 1 -- 172.21.15.97:0/724740387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c4003e30 con 0x5618b513bb50 2023-07-06T03:49:41.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.320+0000 7f41b6ffd700 1 -- 172.21.15.97:0/724740387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c400dc50 con 0x5618b513bb50 2023-07-06T03:49:41.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.321+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/724740387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 msgr2=0x5618b513e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.321+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/724740387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b513e110 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f41c4002180 tx=0x7f41c400d430 comp rx=0 tx=0).stop 2023-07-06T03:49:41.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.321+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/724740387 shutdown_connections 2023-07-06T03:49:41.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.321+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/724740387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b513e110 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.321+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/724740387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5160d50 0x5618b5161120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.321+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/724740387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5166190 0x5618b5184a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.321+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/724740387 >> 172.21.15.97:0/724740387 conn(0x5618b5133310 msgr2=0x5618b51835b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:41.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.321+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/724740387 shutdown_connections 2023-07-06T03:49:41.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.321+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/724740387 wait complete. 2023-07-06T03:49:41.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.322+0000 7f41e6e1cb80 1 -- start start 2023-07-06T03:49:41.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.322+0000 7f41e6e1cb80 1 Processor -- start 2023-07-06T03:49:41.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.323+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b5155f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.323+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5160d50 0x5618b515f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.323+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5166190 0x5618b515f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.323+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b508ebe0 con 0x5618b513bb50 2023-07-06T03:49:41.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.323+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b504c5b0 con 0x5618b5166190 2023-07-06T03:49:41.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.323+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618b5032910 con 0x5618b5160d50 2023-07-06T03:49:41.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.323+0000 7f41e6e1cb80 10 client.0 _ll_register_callbacks cb 0x5618b504ab70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:49:41.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.323+0000 7f41d119b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b5155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.323+0000 7f41d119b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b5155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44708/0 (socket says 172.21.15.97:44708) 2023-07-06T03:49:41.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.323+0000 7f41d119b700 1 -- 172.21.15.97:0/773248962 learned_addr learned my addr 172.21.15.97:0/773248962 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:41.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.323+0000 7f41d099a700 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5160d50 0x5618b515f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41d199c700 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5166190 0x5618b515f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41d119b700 1 -- 172.21.15.97:0/773248962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5160d50 msgr2=0x5618b515f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:41.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41d119b700 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5160d50 0x5618b515f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41d119b700 1 -- 172.21.15.97:0/773248962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5166190 msgr2=0x5618b515f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:41.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41d119b700 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5166190 0x5618b515f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41d119b700 1 -- 172.21.15.97:0/773248962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc0ccc30 con 0x5618b513bb50 2023-07-06T03:49:41.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41d199c700 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5166190 0x5618b515f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:41.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41d119b700 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b5155f30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f41c8088c00 tx=0x7f41c805d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e0775219f42c11 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:41.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41abfff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:49:41.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41abfff700 1 -- 172.21.15.97:0/773248962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c805d370 con 0x5618b513bb50 2023-07-06T03:49:41.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41abfff700 1 -- 172.21.15.97:0/773248962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f41c80614c0 con 0x5618b513bb50 2023-07-06T03:49:41.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/773248962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618b5130920 con 0x5618b513bb50 2023-07-06T03:49:41.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41abfff700 1 -- 172.21.15.97:0/773248962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41c8061680 con 0x5618b513bb50 2023-07-06T03:49:41.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41e6e1cb80 10 client.6987 fetch_fsmap learned FSMap version 44 2023-07-06T03:49:41.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/773248962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x5618b4f6f0a0 con 0x5618b513bb50 2023-07-06T03:49:41.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.324+0000 7f41abfff700 1 -- 172.21.15.97:0/773248962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5618b5130920 con 0x5618b513bb50 2023-07-06T03:49:41.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.325+0000 7f41abfff700 1 -- 172.21.15.97:0/773248962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 44) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f41c80553f0 con 0x5618b513bb50 2023-07-06T03:49:41.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.325+0000 7f41e6e1cb80 10 client.6987 fetch_fsmap finished waiting for FSMap version 44 2023-07-06T03:49:41.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.325+0000 7f41e6e1cb80 10 client.6987 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-06T03:49:41.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.325+0000 7f41e6e1cb80 20 client.6987 populate_metadata read hostname 'smithi097' 2023-07-06T03:49:41.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.325+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.97:6835/1957695290,v1:172.21.15.97:6837/1957695290] conn(0x5618b517af50 0x5618b517b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.325+0000 7f41e6e1cb80 4 client.6987 mds_command: new command op to 6837 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T03:49:41.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.325+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/773248962 --> [v2:172.21.15.97:6835/1957695290,v1:172.21.15.97:6837/1957695290] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618b510d0d0 con 0x5618b517af50 2023-07-06T03:49:41.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.325+0000 7f41d099a700 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.97:6835/1957695290,v1:172.21.15.97:6837/1957695290] conn(0x5618b517af50 0x5618b517b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.325+0000 7f41abfff700 10 client.6987 ms_handle_connect on v2:172.21.15.97:6835/1957695290 2023-07-06T03:49:41.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.325+0000 7f41d099a700 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.97:6835/1957695290,v1:172.21.15.97:6837/1957695290] conn(0x5618b517af50 0x5618b517b320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6738da0a6e0418a9 server_cookie=5f59ea0b92c333e0 in_seq=0 out_seq=0 2023-07-06T03:49:41.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.326+0000 7f41abfff700 1 -- 172.21.15.97:0/773248962 <== mds.0 v2:172.21.15.97:6835/1957695290 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5618b510d0d0 con 0x5618b517af50 2023-07-06T03:49:41.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.326+0000 7f41abfff700 10 client.6987 handle_command_reply: tid=0 2023-07-06T03:49:41.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.326+0000 7f41e6e1cb80 1 client.6987 shutdown 2023-07-06T03:49:41.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.326+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/773248962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 msgr2=0x5618b5155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.326+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b5155f30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f41c8088c00 tx=0x7f41c805d740 comp rx=0 tx=0).stop 2023-07-06T03:49:41.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.326+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/773248962 shutdown_connections 2023-07-06T03:49:41.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.326+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.97:6835/1957695290,v1:172.21.15.97:6837/1957695290] conn(0x5618b517af50 0x5618b517b320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.327+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5166190 0x5618b515f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.327+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b513bb50 0x5618b5155f30 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.327+0000 7f41abfff700 0 client.6987 ms_handle_reset on v2:172.21.15.97:6835/1957695290 2023-07-06T03:49:41.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.327+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/773248962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5160d50 0x5618b515f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.327+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/773248962 >> 172.21.15.97:0/773248962 conn(0x5618b5133310 msgr2=0x5618b5137ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:41.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.327+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/773248962 shutdown_connections 2023-07-06T03:49:41.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.327+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/773248962 wait complete. 2023-07-06T03:49:41.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.327+0000 7f41e6e1cb80 20 client.6987 trim_cache size 0 max 16384 2023-07-06T03:49:41.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.345+0000 7f41e6e1cb80 1 Processor -- start 2023-07-06T03:49:41.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.345+0000 7f41e6e1cb80 1 -- start start 2023-07-06T03:49:41.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.346+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b513bb50 0x5618b515f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.346+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51e3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.346+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5166190 0x5618b51e3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.346+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b4e83390 con 0x5618b5160d50 2023-07-06T03:49:41.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.346+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b51e44b0 con 0x5618b5166190 2023-07-06T03:49:41.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.346+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618b517cb50 con 0x5618b513bb50 2023-07-06T03:49:41.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.346+0000 7f41d099a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51e3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.346+0000 7f41d099a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51e3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44716/0 (socket says 172.21.15.97:44716) 2023-07-06T03:49:41.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.346+0000 7f41d099a700 1 -- 172.21.15.97:0/2848051769 learned_addr learned my addr 172.21.15.97:0/2848051769 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:41.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.347+0000 7f41d099a700 1 -- 172.21.15.97:0/2848051769 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b513bb50 msgr2=0x5618b515f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:41.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.347+0000 7f41d099a700 1 --2- 172.21.15.97:0/2848051769 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b513bb50 0x5618b515f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.347+0000 7f41d099a700 1 -- 172.21.15.97:0/2848051769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5166190 msgr2=0x5618b51e3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.347+0000 7f41d099a700 1 --2- 172.21.15.97:0/2848051769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5166190 0x5618b51e3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.347+0000 7f41d099a700 1 -- 172.21.15.97:0/2848051769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618b5054b10 con 0x5618b5160d50 2023-07-06T03:49:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.347+0000 7f41d099a700 1 --2- 172.21.15.97:0/2848051769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51e3a30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f41cc15ca30 tx=0x7f41bc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b50e8eeac96b44b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.347+0000 7f41b77fe700 1 -- 172.21.15.97:0/2848051769 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc005a50 con 0x5618b5160d50 2023-07-06T03:49:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.347+0000 7f41b77fe700 1 -- 172.21.15.97:0/2848051769 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc0070d0 con 0x5618b5160d50 2023-07-06T03:49:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.347+0000 7f41b77fe700 1 -- 172.21.15.97:0/2848051769 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc007440 con 0x5618b5160d50 2023-07-06T03:49:41.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.348+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2848051769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 msgr2=0x5618b51e3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.348+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/2848051769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51e3a30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f41cc15ca30 tx=0x7f41bc011000 comp rx=0 tx=0).stop 2023-07-06T03:49:41.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.348+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2848051769 shutdown_connections 2023-07-06T03:49:41.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.348+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/2848051769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b5166190 0x5618b51e3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.348+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/2848051769 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b513bb50 0x5618b515f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.348+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/2848051769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51e3a30 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.348+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2848051769 >> 172.21.15.97:0/2848051769 conn(0x5618b5133310 msgr2=0x5618b51a4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:41.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.348+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2848051769 shutdown_connections 2023-07-06T03:49:41.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.348+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2848051769 wait complete. 2023-07-06T03:49:41.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.349+0000 7f41e6e1cb80 1 -- start start 2023-07-06T03:49:41.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.349+0000 7f41e6e1cb80 1 Processor -- start 2023-07-06T03:49:41.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b513bb50 0x5618b51e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41e6e1cb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5166190 0x5618b5179e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b51e2f80 con 0x5618b5160d50 2023-07-06T03:49:41.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b5163360 con 0x5618b513bb50 2023-07-06T03:49:41.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41e6e1cb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618b51637c0 con 0x5618b5166190 2023-07-06T03:49:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41e6e1cb80 10 client.0 _ll_register_callbacks cb 0x5618b504ab70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:49:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41d099a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41d099a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51798e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44720/0 (socket says 172.21.15.97:44720) 2023-07-06T03:49:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41d099a700 1 -- 172.21.15.97:0/2216565421 learned_addr learned my addr 172.21.15.97:0/2216565421 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41d199c700 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5166190 0x5618b5179e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41d119b700 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b513bb50 0x5618b51e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41d099a700 1 -- 172.21.15.97:0/2216565421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5166190 msgr2=0x5618b5179e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:41.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41d099a700 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5166190 0x5618b5179e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41d099a700 1 -- 172.21.15.97:0/2216565421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b513bb50 msgr2=0x5618b51e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:41.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41d099a700 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b513bb50 0x5618b51e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.350+0000 7f41d099a700 1 -- 172.21.15.97:0/2216565421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc01c040 con 0x5618b5160d50 2023-07-06T03:49:41.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.351+0000 7f41d199c700 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5166190 0x5618b5179e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.351+0000 7f41d119b700 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b513bb50 0x5618b51e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.351+0000 7f41d099a700 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51798e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f41bc001230 tx=0x7f41bc0079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbb343d97db53b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.351+0000 7f41abfff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:49:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.351+0000 7f41abfff700 1 -- 172.21.15.97:0/2216565421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc033070 con 0x5618b5160d50 2023-07-06T03:49:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.351+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2216565421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41c4003e30 con 0x5618b5160d50 2023-07-06T03:49:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.351+0000 7f41abfff700 1 -- 172.21.15.97:0/2216565421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f41bc007450 con 0x5618b5160d50 2023-07-06T03:49:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.351+0000 7f41abfff700 1 -- 172.21.15.97:0/2216565421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41bc007610 con 0x5618b5160d50 2023-07-06T03:49:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.351+0000 7f41e6e1cb80 10 client.6993 fetch_fsmap learned FSMap version 44 2023-07-06T03:49:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.351+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2216565421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x5618b4ee31d0 con 0x5618b5160d50 2023-07-06T03:49:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.351+0000 7f41abfff700 1 -- 172.21.15.97:0/2216565421 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c4003e30 con 0x5618b5160d50 2023-07-06T03:49:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.352+0000 7f41abfff700 1 -- 172.21.15.97:0/2216565421 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 44) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f41bc02c030 con 0x5618b5160d50 2023-07-06T03:49:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.352+0000 7f41e6e1cb80 10 client.6993 fetch_fsmap finished waiting for FSMap version 44 2023-07-06T03:49:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.352+0000 7f41e6e1cb80 10 client.6993 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-06T03:49:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.352+0000 7f41e6e1cb80 20 client.6993 populate_metadata read hostname 'smithi097' 2023-07-06T03:49:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.352+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.97:6835/1957695290,v1:172.21.15.97:6837/1957695290] conn(0x5618b51edfc0 0x5618b51ee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.352+0000 7f41e6e1cb80 4 client.6993 mds_command: new command op to 6837 tid=0[{"prefix": "status"}] 2023-07-06T03:49:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.352+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2216565421 --> [v2:172.21.15.97:6835/1957695290,v1:172.21.15.97:6837/1957695290] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5618b5087d20 con 0x5618b51edfc0 2023-07-06T03:49:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.352+0000 7f41d119b700 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.97:6835/1957695290,v1:172.21.15.97:6837/1957695290] conn(0x5618b51edfc0 0x5618b51ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.352+0000 7f41abfff700 10 client.6993 ms_handle_connect on v2:172.21.15.97:6835/1957695290 2023-07-06T03:49:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.352+0000 7f41d119b700 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.97:6835/1957695290,v1:172.21.15.97:6837/1957695290] conn(0x5618b51edfc0 0x5618b51ee390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d104bfa3d973b84c server_cookie=e973aa8babb8706c in_seq=0 out_seq=0 2023-07-06T03:49:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41abfff700 1 -- 172.21.15.97:0/2216565421 <== mds.0 v2:172.21.15.97:6835/1957695290 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5618b5087d20 con 0x5618b51edfc0 2023-07-06T03:49:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41abfff700 10 client.6993 handle_command_reply: tid=0 2023-07-06T03:49:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 1 client.6993 shutdown 2023-07-06T03:49:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2216565421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 msgr2=0x5618b51798e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51798e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f41bc001230 tx=0x7f41bc0079d0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2216565421 shutdown_connections 2023-07-06T03:49:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x5618b5166190 0x5618b5179e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.97:6835/1957695290,v1:172.21.15.97:6837/1957695290] conn(0x5618b51edfc0 0x5618b51ee390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618b513bb50 0x5618b51e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 1 --2- 172.21.15.97:0/2216565421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5618b5160d50 0x5618b51798e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2216565421 >> 172.21.15.97:0/2216565421 conn(0x5618b5133310 msgr2=0x5618b515eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:41.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41abfff700 0 client.6993 ms_handle_reset on v2:172.21.15.97:6835/1957695290 2023-07-06T03:49:41.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2216565421 shutdown_connections 2023-07-06T03:49:41.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 1 -- 172.21.15.97:0/2216565421 wait complete. 2023-07-06T03:49:41.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.353+0000 7f41e6e1cb80 20 client.6993 trim_cache size 0 max 16384 2023-07-06T03:49:41.363 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:49:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36", 2023-07-06T03:49:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-07-06T03:49:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6837, 2023-07-06T03:49:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-07-06T03:49:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-07-06T03:49:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-07-06T03:49:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.517090241, 2023-07-06T03:49:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 44, 2023-07-06T03:49:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 74, 2023-07-06T03:49:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 74, 2023-07-06T03:49:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 19.066293127000002 2023-07-06T03:49:41.365 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:49:41.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 -- 172.21.15.97:0/718682527 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41b006ace0 msgr2=0x7f41b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41b006ace0 0x7f41b006d190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f41cc15bda0 tx=0x7f41bc009040 comp rx=0 tx=0).stop 2023-07-06T03:49:41.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 -- 172.21.15.97:0/718682527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41cc0719c0 msgr2=0x7f41cc150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41cc0719c0 0x7f41cc150970 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f41c80090b0 tx=0x7f41c8004750 comp rx=0 tx=0).stop 2023-07-06T03:49:41.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 -- 172.21.15.97:0/718682527 shutdown_connections 2023-07-06T03:49:41.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41b006ace0 0x7f41b006d190 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41cc15c590 0x7f41cc06ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41cc0719c0 0x7f41cc150970 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 --2- 172.21.15.97:0/718682527 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41cc15c1c0 0x7f41cc0b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 -- 172.21.15.97:0/718682527 >> 172.21.15.97:0/718682527 conn(0x7f41cc0b1650 msgr2=0x7f41cc0c54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 -- 172.21.15.97:0/718682527 shutdown_connections 2023-07-06T03:49:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.354+0000 7f41d33ff700 1 -- 172.21.15.97:0/718682527 wait complete. 2023-07-06T03:49:41.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:49:41.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.564+0000 7f54a63ff700 1 Processor -- start 2023-07-06T03:49:41.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.564+0000 7f54a63ff700 1 -- start start 2023-07-06T03:49:41.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.565+0000 7f54a63ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54a015c520 0x7f54a00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.565+0000 7f54a63ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f54a0160180 0x7f54a01605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.565+0000 7f54a63ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a0160b30 0x7f54a00cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.565+0000 7f54a63ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a00cd8d0 con 0x7f54a015c520 2023-07-06T03:49:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.565+0000 7f54a63ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a00cdae0 con 0x7f54a0160180 2023-07-06T03:49:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.565+0000 7f54a63ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a00b6640 con 0x7f54a0160b30 2023-07-06T03:49:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.565+0000 7f549ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54a015c520 0x7f54a00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.565+0000 7f549ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54a015c520 0x7f54a00cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44722/0 (socket says 172.21.15.97:44722) 2023-07-06T03:49:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.565+0000 7f549ffff700 1 -- 172.21.15.97:0/1917019082 learned_addr learned my addr 172.21.15.97:0/1917019082 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:41.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.565+0000 7f54a499c700 1 --2- 172.21.15.97:0/1917019082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a0160b30 0x7f54a00cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.565+0000 7f549f7fe700 1 --2- 172.21.15.97:0/1917019082 >> v2:172.21.15.146:3300/0 conn(0x7f54a0160180 0x7f54a01605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.566+0000 7f549ffff700 1 -- 172.21.15.97:0/1917019082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a0160b30 msgr2=0x7f54a00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.566+0000 7f549ffff700 1 --2- 172.21.15.97:0/1917019082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a0160b30 0x7f54a00cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.566+0000 7f549ffff700 1 -- 172.21.15.97:0/1917019082 >> v2:172.21.15.146:3300/0 conn(0x7f54a0160180 msgr2=0x7f54a01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.566+0000 7f549ffff700 1 --2- 172.21.15.97:0/1917019082 >> v2:172.21.15.146:3300/0 conn(0x7f54a0160180 0x7f54a01605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.566+0000 7f549ffff700 1 -- 172.21.15.97:0/1917019082 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a00b6920 con 0x7f54a015c520 2023-07-06T03:49:41.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.566+0000 7f54a499c700 1 --2- 172.21.15.97:0/1917019082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a0160b30 0x7f54a00cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:41.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.566+0000 7f549f7fe700 1 --2- 172.21.15.97:0/1917019082 >> v2:172.21.15.146:3300/0 conn(0x7f54a0160180 0x7f54a01605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:41.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.566+0000 7f549ffff700 1 --2- 172.21.15.97:0/1917019082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54a015c520 0x7f54a00cd390 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f54a00b5930 tx=0x7f5490009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385c224b702f496f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:41.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.566+0000 7f549effd700 1 -- 172.21.15.97:0/1917019082 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549000b870 con 0x7f54a015c520 2023-07-06T03:49:41.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.566+0000 7f549effd700 1 -- 172.21.15.97:0/1917019082 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549000b9d0 con 0x7f54a015c520 2023-07-06T03:49:41.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.566+0000 7f549effd700 1 -- 172.21.15.97:0/1917019082 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549000bd60 con 0x7f54a015c520 2023-07-06T03:49:41.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.567+0000 7f54a63ff700 1 -- 172.21.15.97:0/1917019082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54a015c520 msgr2=0x7f54a00cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.567+0000 7f54a63ff700 1 --2- 172.21.15.97:0/1917019082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54a015c520 0x7f54a00cd390 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f54a00b5930 tx=0x7f5490009bd0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.567+0000 7f54a63ff700 1 -- 172.21.15.97:0/1917019082 shutdown_connections 2023-07-06T03:49:41.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.567+0000 7f54a63ff700 1 --2- 172.21.15.97:0/1917019082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a0160b30 0x7f54a00cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.567+0000 7f54a63ff700 1 --2- 172.21.15.97:0/1917019082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54a015c520 0x7f54a00cd390 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.567+0000 7f54a63ff700 1 --2- 172.21.15.97:0/1917019082 >> v2:172.21.15.146:3300/0 conn(0x7f54a0160180 0x7f54a01605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.567+0000 7f54a63ff700 1 -- 172.21.15.97:0/1917019082 >> 172.21.15.97:0/1917019082 conn(0x7f54a0071790 msgr2=0x7f54a0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:41.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.567+0000 7f54a63ff700 1 -- 172.21.15.97:0/1917019082 shutdown_connections 2023-07-06T03:49:41.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.567+0000 7f54a63ff700 1 -- 172.21.15.97:0/1917019082 wait complete. 2023-07-06T03:49:41.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.569+0000 7f54a63ff700 1 Processor -- start 2023-07-06T03:49:41.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.569+0000 7f54a63ff700 1 -- start start 2023-07-06T03:49:41.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f54a63ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a015c520 0x7f54a00d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f54a63ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54a0160180 0x7f54a00cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f54a63ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a0160b30 0x7f54a014e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f54a63ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a00d1180 con 0x7f54a0160180 2023-07-06T03:49:41.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f54a63ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a00cc8b0 con 0x7f54a0160b30 2023-07-06T03:49:41.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f54a63ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a00d1420 con 0x7f54a015c520 2023-07-06T03:49:41.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f549f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54a0160180 0x7f54a00cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f549f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54a0160180 0x7f54a00cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44728/0 (socket says 172.21.15.97:44728) 2023-07-06T03:49:41.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f549f7fe700 1 -- 172.21.15.97:0/2855964020 learned_addr learned my addr 172.21.15.97:0/2855964020 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:41.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f549ffff700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a015c520 0x7f54a00d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f54a499c700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a0160b30 0x7f54a014e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f549f7fe700 1 -- 172.21.15.97:0/2855964020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a015c520 msgr2=0x7f54a00d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f549f7fe700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a015c520 0x7f54a00d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f549f7fe700 1 -- 172.21.15.97:0/2855964020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a0160b30 msgr2=0x7f54a014e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f549f7fe700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a0160b30 0x7f54a014e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f549f7fe700 1 -- 172.21.15.97:0/2855964020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5490009550 con 0x7f54a0160180 2023-07-06T03:49:41.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.570+0000 7f54a499c700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a0160b30 0x7f54a014e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:41.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.571+0000 7f549f7fe700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54a0160180 0x7f54a00cc370 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f549400d800 tx=0x7f549400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:41.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.571+0000 7f549cff9700 1 -- 172.21.15.97:0/2855964020 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494004030 con 0x7f54a0160180 2023-07-06T03:49:41.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.571+0000 7f549cff9700 1 -- 172.21.15.97:0/2855964020 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5494004190 con 0x7f54a0160180 2023-07-06T03:49:41.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.571+0000 7f54a63ff700 1 -- 172.21.15.97:0/2855964020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a00d1700 con 0x7f54a0160180 2023-07-06T03:49:41.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.571+0000 7f549cff9700 1 -- 172.21.15.97:0/2855964020 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494004520 con 0x7f54a0160180 2023-07-06T03:49:41.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.571+0000 7f54a63ff700 1 -- 172.21.15.97:0/2855964020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a00d1cc0 con 0x7f54a0160180 2023-07-06T03:49:41.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.572+0000 7f54a63ff700 1 -- 172.21.15.97:0/2855964020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a004e340 con 0x7f54a0160180 2023-07-06T03:49:41.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.573+0000 7f549cff9700 1 -- 172.21.15.97:0/2855964020 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f54940036a0 con 0x7f54a0160180 2023-07-06T03:49:41.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.574+0000 7f549cff9700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f548c06ace0 0x7f548c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:41.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.574+0000 7f549cff9700 1 -- 172.21.15.97:0/2855964020 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5494089a00 con 0x7f54a0160180 2023-07-06T03:49:41.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.574+0000 7f549ffff700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f548c06ace0 0x7f548c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:41.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.574+0000 7f549ffff700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f548c06ace0 0x7f548c06d190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5490000c00 tx=0x7f5490008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:41.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.578+0000 7f549cff9700 1 -- 172.21.15.97:0/2855964020 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f54940584a0 con 0x7f54a0160180 2023-07-06T03:49:41.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.815+0000 7f54a63ff700 1 -- 172.21.15.97:0/2855964020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f54a00b4f40 con 0x7f54a0160180 2023-07-06T03:49:41.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.816+0000 7f549cff9700 1 -- 172.21.15.97:0/2855964020 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5494058030 con 0x7f54a0160180 2023-07-06T03:49:41.817 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:49:41.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 -- 172.21.15.97:0/2855964020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f548c06ace0 msgr2=0x7f548c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f548c06ace0 0x7f548c06d190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5490000c00 tx=0x7f5490008030 comp rx=0 tx=0).stop 2023-07-06T03:49:41.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 -- 172.21.15.97:0/2855964020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54a0160180 msgr2=0x7f54a00cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:41.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54a0160180 0x7f54a00cc370 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f549400d800 tx=0x7f549400db10 comp rx=0 tx=0).stop 2023-07-06T03:49:41.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 -- 172.21.15.97:0/2855964020 shutdown_connections 2023-07-06T03:49:41.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f548c06ace0 0x7f548c06d190 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a0160b30 0x7f54a014e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54a015c520 0x7f54a00d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 --2- 172.21.15.97:0/2855964020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54a0160180 0x7f54a00cc370 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 -- 172.21.15.97:0/2855964020 >> 172.21.15.97:0/2855964020 conn(0x7f54a0071790 msgr2=0x7f54a0071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 -- 172.21.15.97:0/2855964020 shutdown_connections 2023-07-06T03:49:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:41.818+0000 7f54a63ff700 1 -- 172.21.15.97:0/2855964020 wait complete. 2023-07-06T03:49:41.839 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:49:41.840 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:49:41.840 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:49:41.840 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:49:41.841 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:49:41.841 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:49:41.841 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:49:41.841 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:49:41.842 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:49:41.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:41.843 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:49:41.889 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:49:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:49:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:49:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:49:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:41.890 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:49:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:49:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:49:41.891 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:49:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:49:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:49:41.891 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:49:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:49:41.891 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:49:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:49:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:49:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41887sec preferred_lft 41887sec 2023-07-06T03:49:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:49:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:41.892 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:49:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:49:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:49:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:49:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:41.893 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:49:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:49:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:49:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:41.893 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:49:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:49:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:49:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:41.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:41.895 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:49:41.962 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:49:41.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:41.965 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:49:42.013 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:49:42.013 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:49:42.013 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:49:42.013 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:49:42.014 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:49:42.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:42.014 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:49:42.015 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:49:42.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:49:42.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:49:42.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:49:42.016 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:49:42.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:42.182 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:49:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:49:42.229 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:49:42.230 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:49:42.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:42.230 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:49:42.338 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:49:42.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:49:42.339 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:49:42.385 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:49:42.385 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:49:42.385 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:49:42.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:49:42.580 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:49:42.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:49:42.648 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:49:42.712 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:49:42.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:42.713 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:49:42.758 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:49:42.759 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:49:42.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:49:42.876 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:49:42.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:49:42.944 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:49:42.947 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:49:42.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:42.949 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:49:42.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:49:42.973+0000 7feb72f945c0 -1 init, newargv = 0x562253142cd0 newargc=15 2023-07-06T03:49:42.975 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[143292]: starting ceph client 2023-07-06T03:49:42.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[143292]: starting fuse 2023-07-06T03:49:43.000 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:49:43.000 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:49:43.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:49:43.001 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:49:43.002 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:49:43.002 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:49:43.002 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:49:43.003 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:43.003 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:49:43.003 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:49:43.003 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:49:43.004 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:49:43.004 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:43.004 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:49:43.005 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:49:43.005 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:49:43.005 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:49:43.006 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:43.006 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:49:43.006 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:49:43.007 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:49:43.007 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:49:43.007 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:49:43.007 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:49:43.008 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:49:43.008 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:49:43.008 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:43.009 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:49:43.009 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:49:43.183 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.143292.asok 2023-07-06T03:49:43.361 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.143292.asok 2023-07-06T03:49:43.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:43.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.143292.asok status 2023-07-06T03:49:43.586 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:49:43.586 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:49:43.586 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:49:43.587 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:49:43.587 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:49:43.587 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:49:43.587 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:49:43.587 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "143292", 2023-07-06T03:49:43.588 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:49:43.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:43.588 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:49:43.588 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:49:43.588 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7005, 2023-07-06T03:49:43.589 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:49:43.589 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:49:43.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:49:43.589 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 7005 2023-07-06T03:49:43.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:43.589 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:49:43.590 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:49:43.590 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2023-07-06T03:49:43.591 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2398905682 2023-07-06T03:49:43.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:43.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:43.591 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:49:43.591 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:49:43.592 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2023-07-06T03:49:43.592 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2398905682 2023-07-06T03:49:43.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:43.593 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.7005 172.21.15.97:0/2398905682", 2023-07-06T03:49:43.593 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "172.21.15.97:0/2398905682", 2023-07-06T03:49:43.593 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:49:43.593 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 44, 2023-07-06T03:49:43.593 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 74, 2023-07-06T03:49:43.593 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:49:43.593 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:49:43.594 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:49:43.594 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:49:43.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:43.601 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:49:43.601 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:49:43.647 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:49:43.647 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:49:43.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:43.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:49:43.717 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:49:43.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.910+0000 7f2da95ff700 1 Processor -- start 2023-07-06T03:49:43.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.911+0000 7f2da95ff700 1 -- start start 2023-07-06T03:49:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.912+0000 7f2da95ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2da40b8410 0x7f2da4167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.912+0000 7f2da95ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da40cdae0 con 0x7f2da415c520 2023-07-06T03:49:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.912+0000 7f2da95ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da40b6640 con 0x7f2da40cd390 2023-07-06T03:49:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.912+0000 7f2da95ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da40b6920 con 0x7f2da40b8410 2023-07-06T03:49:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da2ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2da415c520 0x7f2da4160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49376/0 (socket says 172.21.15.97:49376) 2023-07-06T03:49:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da2ffd700 1 -- 172.21.15.97:0/2336322805 learned_addr learned my addr 172.21.15.97:0/2336322805 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da37fe700 1 --2- 172.21.15.97:0/2336322805 >> v2:172.21.15.146:3300/0 conn(0x7f2da40b8410 0x7f2da4167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575014668 0 0) 0x7f2da40cdae0 con 0x7f2da415c520 2023-07-06T03:49:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d900035b0 con 0x7f2da415c520 2023-07-06T03:49:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191773507 0 0) 0x7f2da40b6640 con 0x7f2da40cd390 2023-07-06T03:49:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2da40cdae0 con 0x7f2da40cd390 2023-07-06T03:49:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1442763656 0 0) 0x7f2d900035b0 con 0x7f2da415c520 2023-07-06T03:49:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 >> v2:172.21.15.146:3300/0 conn(0x7f2da40b8410 msgr2=0x7f2da4167510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da27fc700 1 --2- 172.21.15.97:0/2336322805 >> v2:172.21.15.146:3300/0 conn(0x7f2da40b8410 0x7f2da4167510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 >> v1:172.21.15.146:6789/0 conn(0x7f2da40cd390 legacy=0x7f2da4160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da40b6c00 con 0x7f2da415c520 2023-07-06T03:49:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f2d94001670 con 0x7f2da415c520 2023-07-06T03:49:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da37fe700 1 --2- 172.21.15.97:0/2336322805 >> v2:172.21.15.146:3300/0 conn(0x7f2da40b8410 0x7f2da4167510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.913+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 >> v1:172.21.15.97:6789/0 conn(0x7f2da415c520 legacy=0x7f2da4160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da27fc700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d90003be0 0x7f2d90003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da27fc700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d90004850 0x7f2d90004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da27fc700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d90005200 0x7f2d9000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da40b6920 con 0x7f2d90003be0 2023-07-06T03:49:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9000c0e0 con 0x7f2d90005200 2023-07-06T03:49:43.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9000c270 con 0x7f2d90004850 2023-07-06T03:49:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da2ffd700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d90003be0 0x7f2d90003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da37fe700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d90005200 0x7f2d9000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2d9bfff700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d90004850 0x7f2d90004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da2ffd700 1 -- 172.21.15.97:0/2336322805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d90004850 msgr2=0x7f2d90004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da2ffd700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d90004850 0x7f2d90004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da2ffd700 1 -- 172.21.15.97:0/2336322805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d90005200 msgr2=0x7f2d9000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da2ffd700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d90005200 0x7f2d9000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da2ffd700 1 -- 172.21.15.97:0/2336322805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d900035b0 con 0x7f2d90003be0 2023-07-06T03:49:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da37fe700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d90005200 0x7f2d9000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da37fe700 1 -- 172.21.15.97:0/2336322805 reap_dead start 2023-07-06T03:49:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.914+0000 7f2da2ffd700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d90003be0 0x7f2d90003fd0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2d94001eb0 tx=0x7f2d90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b033c1436bb6db server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.915+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400cc70 con 0x7f2d90003be0 2023-07-06T03:49:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.915+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9400cdf0 con 0x7f2d90003be0 2023-07-06T03:49:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.915+0000 7f2da27fc700 1 -- 172.21.15.97:0/2336322805 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94005ba0 con 0x7f2d90003be0 2023-07-06T03:49:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.915+0000 7f2da95ff700 1 -- 172.21.15.97:0/2336322805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d90003be0 msgr2=0x7f2d90003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.915+0000 7f2da95ff700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d90003be0 0x7f2d90003fd0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2d94001eb0 tx=0x7f2d90005880 comp rx=0 tx=0).stop 2023-07-06T03:49:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.915+0000 7f2da95ff700 1 -- 172.21.15.97:0/2336322805 shutdown_connections 2023-07-06T03:49:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.915+0000 7f2da95ff700 1 --2- 172.21.15.97:0/2336322805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d90003be0 0x7f2d90003fd0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.915+0000 7f2da95ff700 1 -- 172.21.15.97:0/2336322805 >> 172.21.15.97:0/2336322805 conn(0x7f2da4071790 msgr2=0x7f2da4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.915+0000 7f2da95ff700 1 -- 172.21.15.97:0/2336322805 shutdown_connections 2023-07-06T03:49:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.915+0000 7f2da95ff700 1 -- 172.21.15.97:0/2336322805 wait complete. 2023-07-06T03:49:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.917+0000 7f2da95ff700 1 Processor -- start 2023-07-06T03:49:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.917+0000 7f2da95ff700 1 -- start start 2023-07-06T03:49:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.918+0000 7f2da95ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d90003be0 0x7f2da4156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.918+0000 7f2da95ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da40c86e0 0x7f2da40c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.918+0000 7f2da95ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da40d0cd0 0x7f2da40d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.918+0000 7f2da95ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da40c9090 con 0x7f2da40c86e0 2023-07-06T03:49:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.918+0000 7f2da95ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da40d1a50 con 0x7f2da40d0cd0 2023-07-06T03:49:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.918+0000 7f2da95ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da40d1d30 con 0x7f2d90003be0 2023-07-06T03:49:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.918+0000 7f2d9bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da40c86e0 0x7f2da40c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.918+0000 7f2d9bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da40c86e0 0x7f2da40c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44738/0 (socket says 172.21.15.97:44738) 2023-07-06T03:49:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.918+0000 7f2d9bfff700 1 -- 172.21.15.97:0/1130002131 learned_addr learned my addr 172.21.15.97:0/1130002131 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.918+0000 7f2da37fe700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da40d0cd0 0x7f2da40d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.918+0000 7f2da2ffd700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d90003be0 0x7f2da4156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2d9bfff700 1 -- 172.21.15.97:0/1130002131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d90003be0 msgr2=0x7f2da4156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2d9bfff700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d90003be0 0x7f2da4156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2d9bfff700 1 -- 172.21.15.97:0/1130002131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da40d0cd0 msgr2=0x7f2da40d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2d9bfff700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da40d0cd0 0x7f2da40d1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2d9bfff700 1 -- 172.21.15.97:0/1130002131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da40d2010 con 0x7f2da40c86e0 2023-07-06T03:49:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2da2ffd700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d90003be0 0x7f2da4156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2da37fe700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da40d0cd0 0x7f2da40d1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2d9bfff700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da40c86e0 0x7f2da40c8b50 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00e7b0 tx=0x7f2d8c00eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2d9b7fe700 1 -- 172.21.15.97:0/1130002131 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c0053a0 con 0x7f2da40c86e0 2023-07-06T03:49:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2d9b7fe700 1 -- 172.21.15.97:0/1130002131 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c005500 con 0x7f2da40c86e0 2023-07-06T03:49:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2da95ff700 1 -- 172.21.15.97:0/1130002131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da40d22f0 con 0x7f2da40c86e0 2023-07-06T03:49:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2d9b7fe700 1 -- 172.21.15.97:0/1130002131 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c0038c0 con 0x7f2da40c86e0 2023-07-06T03:49:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.919+0000 7f2da95ff700 1 -- 172.21.15.97:0/1130002131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da4157260 con 0x7f2da40c86e0 2023-07-06T03:49:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.920+0000 7f2da95ff700 1 -- 172.21.15.97:0/1130002131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da404e340 con 0x7f2da40c86e0 2023-07-06T03:49:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.921+0000 7f2d9b7fe700 1 -- 172.21.15.97:0/1130002131 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2d8c003a20 con 0x7f2da40c86e0 2023-07-06T03:49:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.922+0000 7f2d9b7fe700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d8406ad10 0x7f2d8406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.922+0000 7f2d9b7fe700 1 -- 172.21.15.97:0/1130002131 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2d8c088e80 con 0x7f2da40c86e0 2023-07-06T03:49:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.922+0000 7f2da2ffd700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d8406ad10 0x7f2d8406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.923+0000 7f2da2ffd700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d8406ad10 0x7f2d8406d1c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d94007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:43.927+0000 7f2d9b7fe700 1 -- 172.21.15.97:0/1130002131 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2d8c053e00 con 0x7f2da40c86e0 2023-07-06T03:49:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.161+0000 7f2da95ff700 1 -- 172.21.15.97:0/1130002131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2da415fa10 con 0x7f2da40c86e0 2023-07-06T03:49:44.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.162+0000 7f2d9b7fe700 1 -- 172.21.15.97:0/1130002131 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2d8c019030 con 0x7f2da40c86e0 2023-07-06T03:49:44.164 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:49:44.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 -- 172.21.15.97:0/1130002131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d8406ad10 msgr2=0x7f2d8406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:44.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d8406ad10 0x7f2d8406d1c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d94007000 comp rx=0 tx=0).stop 2023-07-06T03:49:44.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 -- 172.21.15.97:0/1130002131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da40c86e0 msgr2=0x7f2da40c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:44.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da40c86e0 0x7f2da40c8b50 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00e7b0 tx=0x7f2d8c00eb70 comp rx=0 tx=0).stop 2023-07-06T03:49:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 -- 172.21.15.97:0/1130002131 shutdown_connections 2023-07-06T03:49:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2d8406ad10 0x7f2d8406d1c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2da40d0cd0 0x7f2da40d1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d90003be0 0x7f2da4156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 --2- 172.21.15.97:0/1130002131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2da40c86e0 0x7f2da40c8b50 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 -- 172.21.15.97:0/1130002131 >> 172.21.15.97:0/1130002131 conn(0x7f2da4071790 msgr2=0x7f2da40b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 -- 172.21.15.97:0/1130002131 shutdown_connections 2023-07-06T03:49:44.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:44.164+0000 7f2da95ff700 1 -- 172.21.15.97:0/1130002131 wait complete. 2023-07-06T03:49:44.183 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:49:44.184 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:49:44.184 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2023-07-06T03:49:44.185 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T03:49:44.185 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T03:49:44.185 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:49:44.186 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:49:44.186 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:49:44.186 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:49:44.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:44.187 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2023-07-06T03:49:44.241 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:49:44.241 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:49:44.241 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:49:44.241 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:44.242 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:49:44.242 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:44.242 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:49:44.242 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:49:44.242 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2023-07-06T03:49:44.242 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:49:44.243 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:49:44.243 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2023-07-06T03:49:44.243 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:49:44.243 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2023-07-06T03:49:44.244 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:49:44.244 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2023-07-06T03:49:44.244 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:49:44.244 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 41900sec preferred_lft 41900sec 2023-07-06T03:49:44.244 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2023-07-06T03:49:44.245 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:44.245 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:49:44.245 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:49:44.245 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:49:44.245 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:44.245 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::58f0:6eff:fe04:3be0/64 scope link 2023-07-06T03:49:44.245 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:44.245 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:49:44.246 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:49:44.246 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::106d:9cff:fed5:9d0f/64 scope link 2023-07-06T03:49:44.246 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:44.246 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:49:44.246 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether a2:c5:04:8e:a2:a7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T03:49:44.246 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::a0c5:4ff:fe8e:a2a7/64 scope link 2023-07-06T03:49:44.246 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:49:44.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:44.247 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:49:44.317 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:49:44.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:44.321 DEBUG:teuthology.orchestra.run.smithi146:> route 2023-07-06T03:49:44.371 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2023-07-06T03:49:44.371 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:49:44.372 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:49:44.372 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:49:44.372 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:49:44.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:44.373 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:49:44.373 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:49:44.374 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:49:44.374 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:49:44.374 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:49:44.375 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:49:44.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:44.542 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:49:44.587 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T03:49:44.587 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T03:49:44.588 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:49:44.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:44.589 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2023-07-06T03:49:44.698 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-06T03:49:44.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:49:44.699 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T03:49:44.745 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:49:44.746 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:49:44.746 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:49:44.830 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:49:44.939 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:49:44.940 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:49:45.009 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:49:45.011 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:49:45.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:45.013 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:49:45.058 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:49:45.059 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-06T03:49:45.103 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:49:45.175 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:49:45.176 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:49:45.245 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:49:45.247 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:49:45.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:45.248 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:49:45.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2023-07-06T03:49:45.268+0000 7ffb431ad5c0 -1 init, newargv = 0x557b7d13ef10 newargc=15 2023-07-06T03:49:45.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[123196]: starting ceph client 2023-07-06T03:49:45.279 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[123196]: starting fuse 2023-07-06T03:49:45.297 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:49:45.298 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T03:49:45.298 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:49:45.299 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2023-07-06T03:49:45.299 DEBUG:teuthology.orchestra.run.smithi146:> import re 2023-07-06T03:49:45.300 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:49:45.300 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2023-07-06T03:49:45.300 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:49:45.301 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2023-07-06T03:49:45.301 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-06T03:49:45.302 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2023-07-06T03:49:45.302 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-06T03:49:45.302 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:49:45.303 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2023-07-06T03:49:45.303 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2023-07-06T03:49:45.303 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2023-07-06T03:49:45.304 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2023-07-06T03:49:45.304 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:49:45.304 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2023-07-06T03:49:45.305 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:49:45.305 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:49:45.305 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:49:45.306 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2023-07-06T03:49:45.306 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2023-07-06T03:49:45.306 DEBUG:teuthology.orchestra.run.smithi146:> return f 2023-07-06T03:49:45.306 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:49:45.307 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:49:45.308 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2023-07-06T03:49:45.308 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:49:45.482 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.123196.asok 2023-07-06T03:49:45.659 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.123196.asok 2023-07-06T03:49:45.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:45.660 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.123196.asok status 2023-07-06T03:49:45.884 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-07-06T03:49:45.884 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2023-07-06T03:49:45.884 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:49:45.885 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:49:45.885 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2023-07-06T03:49:45.885 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2023-07-06T03:49:45.886 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-06T03:49:45.886 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "123196", 2023-07-06T03:49:45.886 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2023-07-06T03:49:45.886 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:49:45.887 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2023-07-06T03:49:45.887 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2023-07-06T03:49:45.887 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 5933, 2023-07-06T03:49:45.888 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2023-07-06T03:49:45.888 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2023-07-06T03:49:45.888 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2023-07-06T03:49:45.889 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 5933 2023-07-06T03:49:45.889 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:49:45.889 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:49:45.889 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:49:45.889 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:49:45.889 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 671928949 2023-07-06T03:49:45.889 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-07-06T03:49:45.890 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:49:45.890 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:49:45.890 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:49:45.890 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:49:45.890 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 671928949 2023-07-06T03:49:45.890 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:49:45.890 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.5933 192.168.0.1:0/671928949", 2023-07-06T03:49:45.891 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/671928949", 2023-07-06T03:49:45.891 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2023-07-06T03:49:45.891 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 44, 2023-07-06T03:49:45.891 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 74, 2023-07-06T03:49:45.891 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:49:45.891 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2023-07-06T03:49:45.891 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2023-07-06T03:49:45.891 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-07-06T03:49:45.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:45.898 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2023-07-06T03:49:45.898 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-06T03:49:45.945 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2023-07-06T03:49:45.946 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-06T03:49:45.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:45.947 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:49:46.055 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-07-06T03:49:46.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.260+0000 7f27a67ff700 1 Processor -- start 2023-07-06T03:49:46.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.260+0000 7f27a67ff700 1 -- start start 2023-07-06T03:49:46.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.261+0000 7f27a67ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a006b360 0x7f27a00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.261+0000 7f27a67ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a00cff90 con 0x7f27a00d2210 2023-07-06T03:49:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.261+0000 7f27a67ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a00b5650 con 0x7f27a015c610 2023-07-06T03:49:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.261+0000 7f27a67ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00d0780 con 0x7f27a006b360 2023-07-06T03:49:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.261+0000 7f27a4d9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f27a00d2210 0x7f27a00d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49390/0 (socket says 172.21.15.97:49390) 2023-07-06T03:49:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.261+0000 7f27a4d9c700 1 -- 172.21.15.97:0/4166453798 learned_addr learned my addr 172.21.15.97:0/4166453798 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.261+0000 7f279f7fe700 1 --2- 172.21.15.97:0/4166453798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a006b360 0x7f27a00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.262+0000 7f279effd700 1 -- 172.21.15.97:0/4166453798 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071467807 0 0) 0x7f27a00cff90 con 0x7f27a00d2210 2023-07-06T03:49:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.262+0000 7f279effd700 1 -- 172.21.15.97:0/4166453798 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27880035b0 con 0x7f27a00d2210 2023-07-06T03:49:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.262+0000 7f279effd700 1 -- 172.21.15.97:0/4166453798 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812503763 0 0) 0x7f27a00b5650 con 0x7f27a015c610 2023-07-06T03:49:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.262+0000 7f279effd700 1 -- 172.21.15.97:0/4166453798 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27a00cff90 con 0x7f27a015c610 2023-07-06T03:49:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.262+0000 7f279f7fe700 1 -- 172.21.15.97:0/4166453798 >> v1:172.21.15.146:6789/0 conn(0x7f27a015c610 legacy=0x7f27a00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.262+0000 7f279effd700 1 -- 172.21.15.97:0/4166453798 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1729960671 0 0) 0x7f27880035b0 con 0x7f27a00d2210 2023-07-06T03:49:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.262+0000 7f279f7fe700 1 -- 172.21.15.97:0/4166453798 >> v1:172.21.15.97:6789/0 conn(0x7f27a00d2210 legacy=0x7f27a00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.262+0000 7f279f7fe700 1 -- 172.21.15.97:0/4166453798 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a00b5400 con 0x7f27a006b360 2023-07-06T03:49:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.262+0000 7f279f7fe700 1 --2- 172.21.15.97:0/4166453798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a006b360 0x7f27a00b4ec0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2794009870 tx=0x7f2794009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eee26284e602fc6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.262+0000 7f279effd700 1 -- 172.21.15.97:0/4166453798 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27980019f0 con 0x7f27a006b360 2023-07-06T03:49:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.263+0000 7f279effd700 1 -- 172.21.15.97:0/4166453798 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2794004030 con 0x7f27a006b360 2023-07-06T03:49:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.263+0000 7f279effd700 1 -- 172.21.15.97:0/4166453798 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27940043a0 con 0x7f27a006b360 2023-07-06T03:49:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.263+0000 7f27a67ff700 1 -- 172.21.15.97:0/4166453798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a006b360 msgr2=0x7f27a00b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.263+0000 7f27a67ff700 1 --2- 172.21.15.97:0/4166453798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a006b360 0x7f27a00b4ec0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2794009870 tx=0x7f2794009b80 comp rx=0 tx=0).stop 2023-07-06T03:49:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.263+0000 7f27a67ff700 1 -- 172.21.15.97:0/4166453798 shutdown_connections 2023-07-06T03:49:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.263+0000 7f27a67ff700 1 --2- 172.21.15.97:0/4166453798 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a006b360 0x7f27a00b4ec0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.263+0000 7f27a67ff700 1 -- 172.21.15.97:0/4166453798 >> 172.21.15.97:0/4166453798 conn(0x7f27a006a6c0 msgr2=0x7f27a006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.263+0000 7f27a67ff700 1 -- 172.21.15.97:0/4166453798 shutdown_connections 2023-07-06T03:49:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.263+0000 7f27a67ff700 1 -- 172.21.15.97:0/4166453798 wait complete. 2023-07-06T03:49:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.265+0000 7f27a67ff700 1 Processor -- start 2023-07-06T03:49:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.265+0000 7f27a67ff700 1 -- start start 2023-07-06T03:49:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.266+0000 7f27a67ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a006b360 0x7f27a014af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.266+0000 7f27a67ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a00d2210 0x7f27a015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.266+0000 7f27a67ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27a015c610 0x7f27a00cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.266+0000 7f27a67ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a015e0c0 con 0x7f27a00d2210 2023-07-06T03:49:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.266+0000 7f27a67ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a015e4e0 con 0x7f27a015c610 2023-07-06T03:49:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.266+0000 7f27a67ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a015d440 con 0x7f27a006b360 2023-07-06T03:49:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.266+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a00d2210 0x7f27a015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.266+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a00d2210 0x7f27a015cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44750/0 (socket says 172.21.15.97:44750) 2023-07-06T03:49:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.266+0000 7f279f7fe700 1 -- 172.21.15.97:0/866868957 learned_addr learned my addr 172.21.15.97:0/866868957 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.266+0000 7f279ffff700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a006b360 0x7f27a014af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.266+0000 7f27a4d9c700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27a015c610 0x7f27a00cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f279f7fe700 1 -- 172.21.15.97:0/866868957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a006b360 msgr2=0x7f27a014af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f279f7fe700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a006b360 0x7f27a014af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f279f7fe700 1 -- 172.21.15.97:0/866868957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27a015c610 msgr2=0x7f27a00cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f279f7fe700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27a015c610 0x7f27a00cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f279f7fe700 1 -- 172.21.15.97:0/866868957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940094a0 con 0x7f27a00d2210 2023-07-06T03:49:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f27a4d9c700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27a015c610 0x7f27a00cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f279f7fe700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a00d2210 0x7f27a015cb60 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f27940090b0 tx=0x7f2794004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f279cff9700 1 -- 172.21.15.97:0/866868957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2794003d10 con 0x7f27a00d2210 2023-07-06T03:49:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f279cff9700 1 -- 172.21.15.97:0/866868957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279401f070 con 0x7f27a00d2210 2023-07-06T03:49:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f27a67ff700 1 -- 172.21.15.97:0/866868957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a015d720 con 0x7f27a00d2210 2023-07-06T03:49:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f279cff9700 1 -- 172.21.15.97:0/866868957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2794011870 con 0x7f27a00d2210 2023-07-06T03:49:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.267+0000 7f27a67ff700 1 -- 172.21.15.97:0/866868957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a015dce0 con 0x7f27a00d2210 2023-07-06T03:49:46.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.268+0000 7f27a67ff700 1 -- 172.21.15.97:0/866868957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a015e200 con 0x7f27a00d2210 2023-07-06T03:49:46.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.269+0000 7f279cff9700 1 -- 172.21.15.97:0/866868957 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f27940119d0 con 0x7f27a00d2210 2023-07-06T03:49:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.270+0000 7f279cff9700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f278c06acb0 0x7f278c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.270+0000 7f279cff9700 1 -- 172.21.15.97:0/866868957 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2794021070 con 0x7f27a00d2210 2023-07-06T03:49:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.270+0000 7f279ffff700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f278c06acb0 0x7f278c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.271+0000 7f279ffff700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f278c06acb0 0x7f278c06d160 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2790001220 tx=0x7f2790009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.274+0000 7f279cff9700 1 -- 172.21.15.97:0/866868957 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f27a015e200 con 0x7f27a00d2210 2023-07-06T03:49:46.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.468+0000 7f27a67ff700 1 -- 172.21.15.97:0/866868957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7f27a015e200 con 0x7f27a00d2210 2023-07-06T03:49:46.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.472+0000 7f279cff9700 1 -- 172.21.15.97:0/866868957 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v23)=0 v23) v1 ==== 121+0+0 (secure 0 0 0) 0x7f27a015e200 con 0x7f27a00d2210 2023-07-06T03:49:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 -- 172.21.15.97:0/866868957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f278c06acb0 msgr2=0x7f278c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f278c06acb0 0x7f278c06d160 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2790001220 tx=0x7f2790009040 comp rx=0 tx=0).stop 2023-07-06T03:49:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 -- 172.21.15.97:0/866868957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a00d2210 msgr2=0x7f27a015cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a00d2210 0x7f27a015cb60 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f27940090b0 tx=0x7f2794004750 comp rx=0 tx=0).stop 2023-07-06T03:49:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 -- 172.21.15.97:0/866868957 shutdown_connections 2023-07-06T03:49:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f278c06acb0 0x7f278c06d160 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27a015c610 0x7f27a00cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27a006b360 0x7f27a014af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 --2- 172.21.15.97:0/866868957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a00d2210 0x7f27a015cb60 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 -- 172.21.15.97:0/866868957 >> 172.21.15.97:0/866868957 conn(0x7f27a006a6c0 msgr2=0x7f27a00d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 -- 172.21.15.97:0/866868957 shutdown_connections 2023-07-06T03:49:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.475+0000 7f27a67ff700 1 -- 172.21.15.97:0/866868957 wait complete. 2023-07-06T03:49:46.494 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2023-07-06T03:49:46.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.691+0000 7fc3a8bff700 1 Processor -- start 2023-07-06T03:49:46.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.691+0000 7fc3a8bff700 1 -- start start 2023-07-06T03:49:46.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.692+0000 7fc3a8bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3a40d2210 0x7fc3a40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:46.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.692+0000 7fc3a8bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40cff90 con 0x7fc3a406b360 2023-07-06T03:49:46.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.692+0000 7fc3a8bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40b5650 con 0x7fc3a415c610 2023-07-06T03:49:46.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.692+0000 7fc3a8bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a40d0780 con 0x7fc3a40d2210 2023-07-06T03:49:46.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.692+0000 7fc3a2d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3a40d2210 0x7fc3a40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:46.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.692+0000 7fc3a2d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3a40d2210 0x7fc3a40b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44760/0 (socket says 172.21.15.97:44760) 2023-07-06T03:49:46.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.692+0000 7fc3a2d9d700 1 -- 172.21.15.97:0/3966146395 learned_addr learned my addr 172.21.15.97:0/3966146395 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:46.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.693+0000 7fc3a2d9d700 1 -- 172.21.15.97:0/3966146395 >> v1:172.21.15.146:6789/0 conn(0x7fc3a415c610 legacy=0x7fc3a40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.693+0000 7fc3a2d9d700 1 -- 172.21.15.97:0/3966146395 >> v1:172.21.15.97:6789/0 conn(0x7fc3a406b360 legacy=0x7fc3a40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.693+0000 7fc3a2d9d700 1 -- 172.21.15.97:0/3966146395 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a40b5400 con 0x7fc3a40d2210 2023-07-06T03:49:46.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.693+0000 7fc3a2d9d700 1 --2- 172.21.15.97:0/3966146395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a40d2210 0x7fc3a40b4ec0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009e50 tx=0x7fc39c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90db11f27984a7ba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:46.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.693+0000 7fc3a159a700 1 -- 172.21.15.97:0/3966146395 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c00e070 con 0x7fc3a40d2210 2023-07-06T03:49:46.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.693+0000 7fc3a159a700 1 -- 172.21.15.97:0/3966146395 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c00b9c0 con 0x7fc3a40d2210 2023-07-06T03:49:46.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.693+0000 7fc3a159a700 1 -- 172.21.15.97:0/3966146395 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c00bd70 con 0x7fc3a40d2210 2023-07-06T03:49:46.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.694+0000 7fc3a8bff700 1 -- 172.21.15.97:0/3966146395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a40d2210 msgr2=0x7fc3a40b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.694+0000 7fc3a8bff700 1 --2- 172.21.15.97:0/3966146395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a40d2210 0x7fc3a40b4ec0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009e50 tx=0x7fc39c00b330 comp rx=0 tx=0).stop 2023-07-06T03:49:46.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.694+0000 7fc3a8bff700 1 -- 172.21.15.97:0/3966146395 shutdown_connections 2023-07-06T03:49:46.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.694+0000 7fc3a8bff700 1 --2- 172.21.15.97:0/3966146395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a40d2210 0x7fc3a40b4ec0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.694+0000 7fc3a8bff700 1 -- 172.21.15.97:0/3966146395 >> 172.21.15.97:0/3966146395 conn(0x7fc3a406a6c0 msgr2=0x7fc3a406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:46.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.694+0000 7fc3a8bff700 1 -- 172.21.15.97:0/3966146395 shutdown_connections 2023-07-06T03:49:46.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.694+0000 7fc3a8bff700 1 -- 172.21.15.97:0/3966146395 wait complete. 2023-07-06T03:49:46.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.696+0000 7fc3a8bff700 1 Processor -- start 2023-07-06T03:49:46.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.696+0000 7fc3a8bff700 1 -- start start 2023-07-06T03:49:46.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a8bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3a406b360 0x7fc3a41f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:46.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a8bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3a40d2210 0x7fc3a414d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:46.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a8bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a415c610 0x7fc3a415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:46.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a8bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a41f5ca0 con 0x7fc3a415c610 2023-07-06T03:49:46.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a8bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a415f580 con 0x7fc3a40d2210 2023-07-06T03:49:46.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a8bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a415f860 con 0x7fc3a406b360 2023-07-06T03:49:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a415c610 0x7fc3a415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a415c610 0x7fc3a415eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44774/0 (socket says 172.21.15.97:44774) 2023-07-06T03:49:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a2d9d700 1 -- 172.21.15.97:0/2525321538 learned_addr learned my addr 172.21.15.97:0/2525321538 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a2d9d700 1 -- 172.21.15.97:0/2525321538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3a406b360 msgr2=0x7fc3a41f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a2d9d700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3a406b360 0x7fc3a41f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a2d9d700 1 -- 172.21.15.97:0/2525321538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3a40d2210 msgr2=0x7fc3a414d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a1d9b700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3a40d2210 0x7fc3a414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a2d9d700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3a40d2210 0x7fc3a414d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a2d9d700 1 -- 172.21.15.97:0/2525321538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c009b30 con 0x7fc3a415c610 2023-07-06T03:49:46.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.697+0000 7fc3a1d9b700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3a40d2210 0x7fc3a414d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:46.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.698+0000 7fc3a2d9d700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a415c610 0x7fc3a415eb00 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc3a408b120 tx=0x7fc39c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:46.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.698+0000 7fc392ffd700 1 -- 172.21.15.97:0/2525321538 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c004960 con 0x7fc3a415c610 2023-07-06T03:49:46.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.698+0000 7fc392ffd700 1 -- 172.21.15.97:0/2525321538 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c0099b0 con 0x7fc3a415c610 2023-07-06T03:49:46.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.698+0000 7fc3a8bff700 1 -- 172.21.15.97:0/2525321538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a41f55e0 con 0x7fc3a415c610 2023-07-06T03:49:46.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.698+0000 7fc392ffd700 1 -- 172.21.15.97:0/2525321538 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c01c680 con 0x7fc3a415c610 2023-07-06T03:49:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.698+0000 7fc3a8bff700 1 -- 172.21.15.97:0/2525321538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a414d580 con 0x7fc3a415c610 2023-07-06T03:49:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.699+0000 7fc3a8bff700 1 -- 172.21.15.97:0/2525321538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a404e340 con 0x7fc3a415c610 2023-07-06T03:49:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.700+0000 7fc392ffd700 1 -- 172.21.15.97:0/2525321538 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc39c01c7e0 con 0x7fc3a415c610 2023-07-06T03:49:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.701+0000 7fc392ffd700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc38406ace0 0x7fc38406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.701+0000 7fc392ffd700 1 -- 172.21.15.97:0/2525321538 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc39c08b320 con 0x7fc3a415c610 2023-07-06T03:49:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.701+0000 7fc3a259c700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc38406ace0 0x7fc38406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.702+0000 7fc3a259c700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc38406ace0 0x7fc38406d190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc394000f60 tx=0x7fc394007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.706+0000 7fc392ffd700 1 -- 172.21.15.97:0/2525321538 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc39c059e50 con 0x7fc3a415c610 2023-07-06T03:49:46.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.902+0000 7fc3a8bff700 1 -- 172.21.15.97:0/2525321538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7fc3a40b7af0 con 0x7fc3a415c610 2023-07-06T03:49:46.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.906+0000 7fc392ffd700 1 -- 172.21.15.97:0/2525321538 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v24)=0 v24) v1 ==== 119+0+0 (secure 0 0 0) 0x7fc39c0599e0 con 0x7fc3a415c610 2023-07-06T03:49:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.908+0000 7fc3a8bff700 1 -- 172.21.15.97:0/2525321538 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc38406ace0 msgr2=0x7fc38406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.908+0000 7fc3a8bff700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc38406ace0 0x7fc38406d190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc394000f60 tx=0x7fc394007a40 comp rx=0 tx=0).stop 2023-07-06T03:49:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.908+0000 7fc3a8bff700 1 -- 172.21.15.97:0/2525321538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a415c610 msgr2=0x7fc3a415eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.908+0000 7fc3a8bff700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a415c610 0x7fc3a415eb00 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc3a408b120 tx=0x7fc39c004ad0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.908+0000 7fc3a8bff700 1 -- 172.21.15.97:0/2525321538 shutdown_connections 2023-07-06T03:49:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.908+0000 7fc3a8bff700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc38406ace0 0x7fc38406d190 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.908+0000 7fc3a8bff700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a415c610 0x7fc3a415eb00 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.908+0000 7fc3a8bff700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3a406b360 0x7fc3a41f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.908+0000 7fc3a8bff700 1 --2- 172.21.15.97:0/2525321538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3a40d2210 0x7fc3a414d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:46.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.909+0000 7fc3a8bff700 1 -- 172.21.15.97:0/2525321538 >> 172.21.15.97:0/2525321538 conn(0x7fc3a406a6c0 msgr2=0x7fc3a40d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:46.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.909+0000 7fc3a8bff700 1 -- 172.21.15.97:0/2525321538 shutdown_connections 2023-07-06T03:49:46.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:46.909+0000 7fc3a8bff700 1 -- 172.21.15.97:0/2525321538 wait complete. 2023-07-06T03:49:46.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2023-07-06T03:49:47.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.124+0000 7f3c67dff700 1 Processor -- start 2023-07-06T03:49:47.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.124+0000 7f3c67dff700 1 -- start start 2023-07-06T03:49:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.125+0000 7f3c67dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c6006b360 0x7f3c600b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.125+0000 7f3c67dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c600d2210 0x7f3c600d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.125+0000 7f3c67dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c600cff90 con 0x7f3c6015c610 2023-07-06T03:49:47.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.125+0000 7f3c67dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c600d0780 con 0x7f3c600d2210 2023-07-06T03:49:47.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.125+0000 7f3c67dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6015c9e0 con 0x7f3c6006b360 2023-07-06T03:49:47.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c65b9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3c6015c610 0x7f3c600b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49420/0 (socket says 172.21.15.97:49420) 2023-07-06T03:49:47.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c65b9b700 1 -- 172.21.15.97:0/175837327 learned_addr learned my addr 172.21.15.97:0/175837327 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:47.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c6639c700 1 --2- 172.21.15.97:0/175837327 >> v2:172.21.15.146:3300/0 conn(0x7f3c600d2210 0x7f3c600d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c6539a700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c6006b360 0x7f3c600b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594581006 0 0) 0x7f3c600cff90 con 0x7f3c6015c610 2023-07-06T03:49:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c4c0035b0 con 0x7f3c6015c610 2023-07-06T03:49:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2111559082 0 0) 0x7f3c4c0035b0 con 0x7f3c6015c610 2023-07-06T03:49:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c6006b360 msgr2=0x7f3c600b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c64b99700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c6006b360 0x7f3c600b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 >> v2:172.21.15.146:3300/0 conn(0x7f3c600d2210 msgr2=0x7f3c600d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c64b99700 1 --2- 172.21.15.97:0/175837327 >> v2:172.21.15.146:3300/0 conn(0x7f3c600d2210 0x7f3c600d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600b5650 con 0x7f3c6015c610 2023-07-06T03:49:47.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.126+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f3c5c001670 con 0x7f3c6015c610 2023-07-06T03:49:47.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 >> v1:172.21.15.97:6789/0 conn(0x7f3c6015c610 legacy=0x7f3c600b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c64b99700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 0x7f3c4c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c64b99700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c4c004850 0x7f3c4c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c64b99700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c6006b360 0x7f3c4c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600d0780 con 0x7f3c4c003a30 2023-07-06T03:49:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6015c9e0 con 0x7f3c4c004850 2023-07-06T03:49:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c00ba10 con 0x7f3c6006b360 2023-07-06T03:49:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c65b9b700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 0x7f3c4c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c6639c700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c6006b360 0x7f3c4c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c6539a700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c4c004850 0x7f3c4c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c65b9b700 1 -- 172.21.15.97:0/175837327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c6006b360 msgr2=0x7f3c4c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c65b9b700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c6006b360 0x7f3c4c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c65b9b700 1 -- 172.21.15.97:0/175837327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c4c004850 msgr2=0x7f3c4c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c65b9b700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c4c004850 0x7f3c4c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.127+0000 7f3c65b9b700 1 -- 172.21.15.97:0/175837327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c0035b0 con 0x7f3c4c003a30 2023-07-06T03:49:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.128+0000 7f3c65b9b700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 0x7f3c4c003e40 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c160 tx=0x7f3c4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c486b5cfbe1a95 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.128+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c007b70 con 0x7f3c4c003a30 2023-07-06T03:49:47.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.128+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c00ccc0 con 0x7f3c4c003a30 2023-07-06T03:49:47.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.128+0000 7f3c64b99700 1 -- 172.21.15.97:0/175837327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c005ba0 con 0x7f3c4c003a30 2023-07-06T03:49:47.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.128+0000 7f3c67dff700 1 -- 172.21.15.97:0/175837327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 msgr2=0x7f3c4c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.128+0000 7f3c67dff700 1 --2- 172.21.15.97:0/175837327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 0x7f3c4c003e40 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c160 tx=0x7f3c4c005880 comp rx=0 tx=0).stop 2023-07-06T03:49:47.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.128+0000 7f3c6639c700 1 -- 172.21.15.97:0/175837327 reap_dead start 2023-07-06T03:49:47.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.128+0000 7f3c67dff700 1 -- 172.21.15.97:0/175837327 shutdown_connections 2023-07-06T03:49:47.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.128+0000 7f3c67dff700 1 -- 172.21.15.97:0/175837327 >> 172.21.15.97:0/175837327 conn(0x7f3c6006a6c0 msgr2=0x7f3c6006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:47.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.128+0000 7f3c67dff700 1 -- 172.21.15.97:0/175837327 shutdown_connections 2023-07-06T03:49:47.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.128+0000 7f3c67dff700 1 -- 172.21.15.97:0/175837327 wait complete. 2023-07-06T03:49:47.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.131+0000 7f3c67dff700 1 Processor -- start 2023-07-06T03:49:47.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.131+0000 7f3c67dff700 1 -- start start 2023-07-06T03:49:47.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.132+0000 7f3c67dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 0x7f3c6015def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.132+0000 7f3c67dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c60153170 0x7f3c6014a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.132+0000 7f3c67dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c6015adb0 0x7f3c6015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.132+0000 7f3c67dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6015b790 con 0x7f3c4c003a30 2023-07-06T03:49:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.132+0000 7f3c67dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6014a550 con 0x7f3c6015adb0 2023-07-06T03:49:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.132+0000 7f3c67dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600cabb0 con 0x7f3c60153170 2023-07-06T03:49:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.132+0000 7f3c65b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 0x7f3c6015def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.132+0000 7f3c65b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 0x7f3c6015def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44782/0 (socket says 172.21.15.97:44782) 2023-07-06T03:49:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.132+0000 7f3c65b9b700 1 -- 172.21.15.97:0/2885620612 learned_addr learned my addr 172.21.15.97:0/2885620612 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c6639c700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c6015adb0 0x7f3c6015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c6539a700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c60153170 0x7f3c6014a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c65b9b700 1 -- 172.21.15.97:0/2885620612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c60153170 msgr2=0x7f3c6014a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c65b9b700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c60153170 0x7f3c6014a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c65b9b700 1 -- 172.21.15.97:0/2885620612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c6015adb0 msgr2=0x7f3c6015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c65b9b700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c6015adb0 0x7f3c6015b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c65b9b700 1 -- 172.21.15.97:0/2885620612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c60153970 con 0x7f3c4c003a30 2023-07-06T03:49:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c6539a700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c60153170 0x7f3c6014a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c65b9b700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 0x7f3c6015def0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c750 tx=0x7f3c5c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c567fc700 1 -- 172.21.15.97:0/2885620612 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c007b70 con 0x7f3c4c003a30 2023-07-06T03:49:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c567fc700 1 -- 172.21.15.97:0/2885620612 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c004c80 con 0x7f3c4c003a30 2023-07-06T03:49:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c67dff700 1 -- 172.21.15.97:0/2885620612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c600c9fa0 con 0x7f3c4c003a30 2023-07-06T03:49:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.133+0000 7f3c567fc700 1 -- 172.21.15.97:0/2885620612 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0070d0 con 0x7f3c4c003a30 2023-07-06T03:49:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.134+0000 7f3c67dff700 1 -- 172.21.15.97:0/2885620612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c600ca560 con 0x7f3c4c003a30 2023-07-06T03:49:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.135+0000 7f3c67dff700 1 -- 172.21.15.97:0/2885620612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c600b7d70 con 0x7f3c4c003a30 2023-07-06T03:49:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.135+0000 7f3c567fc700 1 -- 172.21.15.97:0/2885620612 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3c5c019030 con 0x7f3c4c003a30 2023-07-06T03:49:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.136+0000 7f3c567fc700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c4406ace0 0x7f3c4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.136+0000 7f3c567fc700 1 -- 172.21.15.97:0/2885620612 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3c5c08a4e0 con 0x7f3c4c003a30 2023-07-06T03:49:47.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.137+0000 7f3c6539a700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c4406ace0 0x7f3c4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.137+0000 7f3c6539a700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c4406ace0 0x7f3c4406d190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3c6015d140 tx=0x7f3c50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:47.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.141+0000 7f3c567fc700 1 -- 172.21.15.97:0/2885620612 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3c5c0590c0 con 0x7f3c4c003a30 2023-07-06T03:49:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.338+0000 7f3c67dff700 1 -- 172.21.15.97:0/2885620612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7f3c6004e340 con 0x7f3c4c003a30 2023-07-06T03:49:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.344+0000 7f3c567fc700 1 -- 172.21.15.97:0/2885620612 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v25)=0 v25) v1 ==== 128+0+0 (secure 0 0 0) 0x7f3c5c058c50 con 0x7f3c4c003a30 2023-07-06T03:49:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 -- 172.21.15.97:0/2885620612 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c4406ace0 msgr2=0x7f3c4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c4406ace0 0x7f3c4406d190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3c6015d140 tx=0x7f3c50008040 comp rx=0 tx=0).stop 2023-07-06T03:49:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 -- 172.21.15.97:0/2885620612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 msgr2=0x7f3c6015def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 0x7f3c6015def0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c750 tx=0x7f3c5c001930 comp rx=0 tx=0).stop 2023-07-06T03:49:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 -- 172.21.15.97:0/2885620612 shutdown_connections 2023-07-06T03:49:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c4406ace0 0x7f3c4406d190 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c6015adb0 0x7f3c6015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c003a30 0x7f3c6015def0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 --2- 172.21.15.97:0/2885620612 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c60153170 0x7f3c6014a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 -- 172.21.15.97:0/2885620612 >> 172.21.15.97:0/2885620612 conn(0x7f3c6006a6c0 msgr2=0x7f3c600d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 -- 172.21.15.97:0/2885620612 shutdown_connections 2023-07-06T03:49:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.347+0000 7f3c67dff700 1 -- 172.21.15.97:0/2885620612 wait complete. 2023-07-06T03:49:47.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 800 2023-07-06T03:49:47.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.563+0000 7fcc33dff700 1 Processor -- start 2023-07-06T03:49:47.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.563+0000 7fcc33dff700 1 -- start start 2023-07-06T03:49:47.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.564+0000 7fcc33dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcc2c06b360 0x7fcc2c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.564+0000 7fcc33dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc2c0d2210 0x7fcc2c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.564+0000 7fcc33dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2c0cff90 con 0x7fcc2c15c610 2023-07-06T03:49:47.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.564+0000 7fcc33dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2c0d0780 con 0x7fcc2c0d2210 2023-07-06T03:49:47.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc33dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2c15c9e0 con 0x7fcc2c06b360 2023-07-06T03:49:47.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc3239c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc2c0d2210 0x7fcc2c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc3239c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc2c0d2210 0x7fcc2c0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44798/0 (socket says 172.21.15.97:44798) 2023-07-06T03:49:47.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc31b9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcc2c15c610 0x7fcc2c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49434/0 (socket says 172.21.15.97:49434) 2023-07-06T03:49:47.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc3239c700 1 -- 172.21.15.97:0/3396611301 learned_addr learned my addr 172.21.15.97:0/3396611301 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc3139a700 1 --2- 172.21.15.97:0/3396611301 >> v2:172.21.15.146:3300/0 conn(0x7fcc2c06b360 0x7fcc2c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc3239c700 1 -- 172.21.15.97:0/3396611301 >> v2:172.21.15.146:3300/0 conn(0x7fcc2c06b360 msgr2=0x7fcc2c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc3239c700 1 --2- 172.21.15.97:0/3396611301 >> v2:172.21.15.146:3300/0 conn(0x7fcc2c06b360 0x7fcc2c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc3239c700 1 -- 172.21.15.97:0/3396611301 >> v1:172.21.15.97:6789/0 conn(0x7fcc2c15c610 legacy=0x7fcc2c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc3239c700 1 -- 172.21.15.97:0/3396611301 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c0b5650 con 0x7fcc2c0d2210 2023-07-06T03:49:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc30b99700 1 -- 172.21.15.97:0/3396611301 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023405981 0 0) 0x7fcc2c0cff90 con 0x7fcc2c15c610 2023-07-06T03:49:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.565+0000 7fcc3239c700 1 --2- 172.21.15.97:0/3396611301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c0d1310 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c06a590 tx=0x7fcc2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871d064de8a012af server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.566+0000 7fcc30b99700 1 -- 172.21.15.97:0/3396611301 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400e070 con 0x7fcc2c0d2210 2023-07-06T03:49:47.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.566+0000 7fcc30b99700 1 -- 172.21.15.97:0/3396611301 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2400b980 con 0x7fcc2c0d2210 2023-07-06T03:49:47.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.566+0000 7fcc30b99700 1 -- 172.21.15.97:0/3396611301 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400bd30 con 0x7fcc2c0d2210 2023-07-06T03:49:47.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.566+0000 7fcc33dff700 1 -- 172.21.15.97:0/3396611301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 msgr2=0x7fcc2c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.566+0000 7fcc33dff700 1 --2- 172.21.15.97:0/3396611301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c0d1310 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c06a590 tx=0x7fcc2400b330 comp rx=0 tx=0).stop 2023-07-06T03:49:47.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.566+0000 7fcc33dff700 1 -- 172.21.15.97:0/3396611301 shutdown_connections 2023-07-06T03:49:47.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.566+0000 7fcc33dff700 1 --2- 172.21.15.97:0/3396611301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c0d1310 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.566+0000 7fcc33dff700 1 --2- 172.21.15.97:0/3396611301 >> v2:172.21.15.146:3300/0 conn(0x7fcc2c06b360 0x7fcc2c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.566+0000 7fcc33dff700 1 -- 172.21.15.97:0/3396611301 >> 172.21.15.97:0/3396611301 conn(0x7fcc2c06a6c0 msgr2=0x7fcc2c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.566+0000 7fcc33dff700 1 -- 172.21.15.97:0/3396611301 shutdown_connections 2023-07-06T03:49:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.566+0000 7fcc33dff700 1 -- 172.21.15.97:0/3396611301 wait complete. 2023-07-06T03:49:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.568+0000 7fcc33dff700 1 Processor -- start 2023-07-06T03:49:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.568+0000 7fcc33dff700 1 -- start start 2023-07-06T03:49:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.569+0000 7fcc33dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c06b360 0x7fcc2c157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.569+0000 7fcc33dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.569+0000 7fcc33dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c15c610 0x7fcc2c15e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.569+0000 7fcc33dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0c9e00 con 0x7fcc2c0d2210 2023-07-06T03:49:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.569+0000 7fcc33dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c15db30 con 0x7fcc2c06b360 2023-07-06T03:49:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.569+0000 7fcc33dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c15de10 con 0x7fcc2c15c610 2023-07-06T03:49:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.569+0000 7fcc3139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.569+0000 7fcc3139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44806/0 (socket says 172.21.15.97:44806) 2023-07-06T03:49:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.569+0000 7fcc3139a700 1 -- 172.21.15.97:0/1151661891 learned_addr learned my addr 172.21.15.97:0/1151661891 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc3239c700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c15c610 0x7fcc2c15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc31b9b700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c06b360 0x7fcc2c157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc3139a700 1 -- 172.21.15.97:0/1151661891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c15c610 msgr2=0x7fcc2c15e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc3139a700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c15c610 0x7fcc2c15e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc3139a700 1 -- 172.21.15.97:0/1151661891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c06b360 msgr2=0x7fcc2c157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc3139a700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c06b360 0x7fcc2c157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc3139a700 1 -- 172.21.15.97:0/1151661891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc24009b30 con 0x7fcc2c0d2210 2023-07-06T03:49:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc31b9b700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c06b360 0x7fcc2c157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc3239c700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c15c610 0x7fcc2c15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:49:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc3139a700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15e340 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e810 tx=0x7fcc1c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc227fc700 1 -- 172.21.15.97:0/1151661891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c013610 con 0x7fcc2c0d2210 2023-07-06T03:49:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc227fc700 1 -- 172.21.15.97:0/1151661891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c013770 con 0x7fcc2c0d2210 2023-07-06T03:49:47.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc33dff700 1 -- 172.21.15.97:0/1151661891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2c0ca050 con 0x7fcc2c0d2210 2023-07-06T03:49:47.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc227fc700 1 -- 172.21.15.97:0/1151661891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c013b20 con 0x7fcc2c0d2210 2023-07-06T03:49:47.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.570+0000 7fcc33dff700 1 -- 172.21.15.97:0/1151661891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c158420 con 0x7fcc2c0d2210 2023-07-06T03:49:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.572+0000 7fcc33dff700 1 -- 172.21.15.97:0/1151661891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2c04e340 con 0x7fcc2c0d2210 2023-07-06T03:49:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.572+0000 7fcc227fc700 1 -- 172.21.15.97:0/1151661891 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc1c013c80 con 0x7fcc2c0d2210 2023-07-06T03:49:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.573+0000 7fcc227fc700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 0x7fcc1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.573+0000 7fcc227fc700 1 -- 172.21.15.97:0/1151661891 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc1c089150 con 0x7fcc2c0d2210 2023-07-06T03:49:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.574+0000 7fcc31b9b700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 0x7fcc1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.574+0000 7fcc31b9b700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 0x7fcc1006d190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc28000f60 tx=0x7fcc28007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.578+0000 7fcc227fc700 1 -- 172.21.15.97:0/1151661891 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcc1c054210 con 0x7fcc2c0d2210 2023-07-06T03:49:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.772+0000 7fcc33dff700 1 -- 172.21.15.97:0/1151661891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7fcc2c1587b0 con 0x7fcc2c0d2210 2023-07-06T03:49:47.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.778+0000 7fcc227fc700 1 -- 172.21.15.97:0/1151661891 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v26)=0 v26) v1 ==== 124+0+0 (secure 0 0 0) 0x7fcc1c057830 con 0x7fcc2c0d2210 2023-07-06T03:49:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 -- 172.21.15.97:0/1151661891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 msgr2=0x7fcc1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 0x7fcc1006d190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc28000f60 tx=0x7fcc28007d30 comp rx=0 tx=0).stop 2023-07-06T03:49:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 -- 172.21.15.97:0/1151661891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 msgr2=0x7fcc2c15e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15e340 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e810 tx=0x7fcc1c00ebd0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 -- 172.21.15.97:0/1151661891 shutdown_connections 2023-07-06T03:49:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc1006ace0 0x7fcc1006d190 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc2c15c610 0x7fcc2c15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c06b360 0x7fcc2c157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 --2- 172.21.15.97:0/1151661891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc2c0d2210 0x7fcc2c15e340 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 -- 172.21.15.97:0/1151661891 >> 172.21.15.97:0/1151661891 conn(0x7fcc2c06a6c0 msgr2=0x7fcc2c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 -- 172.21.15.97:0/1151661891 shutdown_connections 2023-07-06T03:49:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.780+0000 7fcc33dff700 1 -- 172.21.15.97:0/1151661891 wait complete. 2023-07-06T03:49:47.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-07-06T03:49:47.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.997+0000 7fb0c3bff700 1 Processor -- start 2023-07-06T03:49:47.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.997+0000 7fb0c3bff700 1 -- start start 2023-07-06T03:49:48.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.998+0000 7fb0c3bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0bc06b360 0x7fb0bc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.998+0000 7fb0c3bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0bc0d2210 0x7fb0bc0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.998+0000 7fb0c3bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc0cff90 con 0x7fb0bc15c610 2023-07-06T03:49:48.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.998+0000 7fb0c3bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc0d0780 con 0x7fb0bc0d2210 2023-07-06T03:49:48.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.998+0000 7fb0c3bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc15c9e0 con 0x7fb0bc06b360 2023-07-06T03:49:48.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c119a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0bc06b360 0x7fb0bc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c219c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0bc0d2210 0x7fb0bc0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c199b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb0bc15c610 0x7fb0bc0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:55700/0 (socket says 172.21.15.97:55700) 2023-07-06T03:49:48.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c219c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0bc0d2210 0x7fb0bc0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:44376/0 (socket says 172.21.15.97:44376) 2023-07-06T03:49:48.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c199b700 1 -- 172.21.15.97:0/4115758049 learned_addr learned my addr 172.21.15.97:0/4115758049 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:48.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c0999700 1 -- 172.21.15.97:0/4115758049 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686051619 0 0) 0x7fb0bc0cff90 con 0x7fb0bc15c610 2023-07-06T03:49:48.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c0999700 1 -- 172.21.15.97:0/4115758049 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a80035b0 con 0x7fb0bc15c610 2023-07-06T03:49:48.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c119a700 1 -- 172.21.15.97:0/4115758049 >> v2:172.21.15.146:3300/0 conn(0x7fb0bc0d2210 msgr2=0x7fb0bc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:48.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c119a700 1 --2- 172.21.15.97:0/4115758049 >> v2:172.21.15.146:3300/0 conn(0x7fb0bc0d2210 0x7fb0bc0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c119a700 1 -- 172.21.15.97:0/4115758049 >> v1:172.21.15.146:6789/0 conn(0x7fb0bc15c610 legacy=0x7fb0bc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c119a700 1 -- 172.21.15.97:0/4115758049 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0b5650 con 0x7fb0bc06b360 2023-07-06T03:49:48.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:47.999+0000 7fb0c219c700 1 --2- 172.21.15.97:0/4115758049 >> v2:172.21.15.146:3300/0 conn(0x7fb0bc0d2210 0x7fb0bc0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:48.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.000+0000 7fb0c119a700 1 --2- 172.21.15.97:0/4115758049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0bc06b360 0x7fb0bc0b4ec0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009840 tx=0x7fb0ac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0a9e1f0ffbc6511 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:48.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.000+0000 7fb0c0999700 1 -- 172.21.15.97:0/4115758049 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac011830 con 0x7fb0bc06b360 2023-07-06T03:49:48.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.000+0000 7fb0c0999700 1 -- 172.21.15.97:0/4115758049 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac011990 con 0x7fb0bc06b360 2023-07-06T03:49:48.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.000+0000 7fb0c0999700 1 -- 172.21.15.97:0/4115758049 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac011d20 con 0x7fb0bc06b360 2023-07-06T03:49:48.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.000+0000 7fb0c3bff700 1 -- 172.21.15.97:0/4115758049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0bc06b360 msgr2=0x7fb0bc0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.000+0000 7fb0c3bff700 1 --2- 172.21.15.97:0/4115758049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0bc06b360 0x7fb0bc0b4ec0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009840 tx=0x7fb0ac009b50 comp rx=0 tx=0).stop 2023-07-06T03:49:48.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.000+0000 7fb0c3bff700 1 -- 172.21.15.97:0/4115758049 shutdown_connections 2023-07-06T03:49:48.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.000+0000 7fb0c3bff700 1 --2- 172.21.15.97:0/4115758049 >> v2:172.21.15.146:3300/0 conn(0x7fb0bc0d2210 0x7fb0bc0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.000+0000 7fb0c3bff700 1 --2- 172.21.15.97:0/4115758049 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0bc06b360 0x7fb0bc0b4ec0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.000+0000 7fb0c3bff700 1 -- 172.21.15.97:0/4115758049 >> 172.21.15.97:0/4115758049 conn(0x7fb0bc06a6c0 msgr2=0x7fb0bc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:48.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.000+0000 7fb0c3bff700 1 -- 172.21.15.97:0/4115758049 shutdown_connections 2023-07-06T03:49:48.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.001+0000 7fb0c3bff700 1 -- 172.21.15.97:0/4115758049 wait complete. 2023-07-06T03:49:48.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.002+0000 7fb0c3bff700 1 Processor -- start 2023-07-06T03:49:48.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.002+0000 7fb0c3bff700 1 -- start start 2023-07-06T03:49:48.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.003+0000 7fb0c3bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0bc06b360 0x7fb0bc0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.003+0000 7fb0c3bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0bc0d2210 0x7fb0bc15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.003+0000 7fb0c3bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0bc15c610 0x7fb0bc15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.003+0000 7fb0c3bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0cba40 con 0x7fb0bc06b360 2023-07-06T03:49:48.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.003+0000 7fb0c3bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0c8be0 con 0x7fb0bc15c610 2023-07-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.003+0000 7fb0c3bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc15e0d0 con 0x7fb0bc0d2210 2023-07-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.003+0000 7fb0c199b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0bc06b360 0x7fb0bc0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.003+0000 7fb0c199b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0bc06b360 0x7fb0bc0c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44808/0 (socket says 172.21.15.97:44808) 2023-07-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.003+0000 7fb0c199b700 1 -- 172.21.15.97:0/3536054691 learned_addr learned my addr 172.21.15.97:0/3536054691 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0c219c700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0bc15c610 0x7fb0bc15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0c119a700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0bc0d2210 0x7fb0bc15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0c199b700 1 -- 172.21.15.97:0/3536054691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0bc0d2210 msgr2=0x7fb0bc15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0c199b700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0bc0d2210 0x7fb0bc15cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0c199b700 1 -- 172.21.15.97:0/3536054691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0bc15c610 msgr2=0x7fb0bc15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0c199b700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0bc15c610 0x7fb0bc15d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0c199b700 1 -- 172.21.15.97:0/3536054691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b4009a70 con 0x7fb0bc06b360 2023-07-06T03:49:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0c199b700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0bc06b360 0x7fb0bc0c6860 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c4d0 tx=0x7fb0b800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0b27fc700 1 -- 172.21.15.97:0/3536054691 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8005d70 con 0x7fb0bc06b360 2023-07-06T03:49:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0b27fc700 1 -- 172.21.15.97:0/3536054691 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b800ce90 con 0x7fb0bc06b360 2023-07-06T03:49:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0c3bff700 1 -- 172.21.15.97:0/3536054691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ac0094a0 con 0x7fb0bc06b360 2023-07-06T03:49:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.004+0000 7fb0b27fc700 1 -- 172.21.15.97:0/3536054691 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80072f0 con 0x7fb0bc06b360 2023-07-06T03:49:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.005+0000 7fb0c3bff700 1 -- 172.21.15.97:0/3536054691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc0cb660 con 0x7fb0bc06b360 2023-07-06T03:49:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.005+0000 7fb0c3bff700 1 -- 172.21.15.97:0/3536054691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc0d1f90 con 0x7fb0bc06b360 2023-07-06T03:49:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.006+0000 7fb0b27fc700 1 -- 172.21.15.97:0/3536054691 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb0b8007450 con 0x7fb0bc06b360 2023-07-06T03:49:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.007+0000 7fb0b27fc700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a006ace0 0x7fb0a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.007+0000 7fb0b27fc700 1 -- 172.21.15.97:0/3536054691 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0b8088870 con 0x7fb0bc06b360 2023-07-06T03:49:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.008+0000 7fb0c119a700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a006ace0 0x7fb0a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.008+0000 7fb0c119a700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a006ace0 0x7fb0a006d190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0090b0 tx=0x7fb0ac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.011+0000 7fb0b27fc700 1 -- 172.21.15.97:0/3536054691 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb0b8053910 con 0x7fb0bc06b360 2023-07-06T03:49:48.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.207+0000 7fb0c3bff700 1 -- 172.21.15.97:0/3536054691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7fb0bc066fe0 con 0x7fb0bc06b360 2023-07-06T03:49:48.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.207+0000 7fb0b27fc700 1 -- 172.21.15.97:0/3536054691 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v26) v1 ==== 106+0+4 (secure 0 0 0) 0x7fb0b8056f30 con 0x7fb0bc06b360 2023-07-06T03:49:48.209 INFO:teuthology.orchestra.run.smithi097.stdout:100 2023-07-06T03:49:48.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 -- 172.21.15.97:0/3536054691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a006ace0 msgr2=0x7fb0a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a006ace0 0x7fb0a006d190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0090b0 tx=0x7fb0ac008030 comp rx=0 tx=0).stop 2023-07-06T03:49:48.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 -- 172.21.15.97:0/3536054691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0bc06b360 msgr2=0x7fb0bc0c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0bc06b360 0x7fb0bc0c6860 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c4d0 tx=0x7fb0b800c7e0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 -- 172.21.15.97:0/3536054691 shutdown_connections 2023-07-06T03:49:48.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a006ace0 0x7fb0a006d190 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0bc15c610 0x7fb0bc15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0bc06b360 0x7fb0bc0c6860 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 --2- 172.21.15.97:0/3536054691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0bc0d2210 0x7fb0bc15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 -- 172.21.15.97:0/3536054691 >> 172.21.15.97:0/3536054691 conn(0x7fb0bc06a6c0 msgr2=0x7fb0bc0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 -- 172.21.15.97:0/3536054691 shutdown_connections 2023-07-06T03:49:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.210+0000 7fb0c3bff700 1 -- 172.21.15.97:0/3536054691 wait complete. 2023-07-06T03:49:48.229 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-07-06T03:49:48.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.424+0000 7f11703ff700 1 Processor -- start 2023-07-06T03:49:48.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.424+0000 7f11703ff700 1 -- start start 2023-07-06T03:49:48.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.426+0000 7f11703ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f116806b360 0x7f11680b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.426+0000 7f11703ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11680d2210 0x7f11680d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.426+0000 7f11703ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11680cff90 con 0x7f116815c610 2023-07-06T03:49:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.426+0000 7f11703ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11680d0780 con 0x7f116806b360 2023-07-06T03:49:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.426+0000 7f11703ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116815c9e0 con 0x7f11680d2210 2023-07-06T03:49:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.426+0000 7f116e19b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f116815c610 0x7f11680b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49448/0 (socket says 172.21.15.97:49448) 2023-07-06T03:49:48.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.426+0000 7f116e19b700 1 -- 172.21.15.97:0/3270646150 learned_addr learned my addr 172.21.15.97:0/3270646150 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:48.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.426+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571309240 0 0) 0x7f11680cff90 con 0x7f116815c610 2023-07-06T03:49:48.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.426+0000 7f116e99c700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11680d2210 0x7f11680d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.426+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11540035b0 con 0x7f116815c610 2023-07-06T03:49:48.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.426+0000 7f116d99a700 1 --2- 172.21.15.97:0/3270646150 >> v2:172.21.15.146:3300/0 conn(0x7f116806b360 0x7f11680b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3463384084 0 0) 0x7f11540035b0 con 0x7f116815c610 2023-07-06T03:49:48.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11680d2210 msgr2=0x7f11680d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:48.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11680d2210 0x7f11680d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 >> v2:172.21.15.146:3300/0 conn(0x7f116806b360 msgr2=0x7f11680b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:48.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 --2- 172.21.15.97:0/3270646150 >> v2:172.21.15.146:3300/0 conn(0x7f116806b360 0x7f11680b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11680b5650 con 0x7f116815c610 2023-07-06T03:49:48.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f1164001670 con 0x7f116815c610 2023-07-06T03:49:48.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 >> v1:172.21.15.97:6789/0 conn(0x7f116815c610 legacy=0x7f11680b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11680d2210 0x7f1154003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1154003fd0 0x7f1154004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 0x7f1154005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11680d0780 con 0x7f1154004d90 2023-07-06T03:49:48.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116815c9e0 con 0x7f1154003fd0 2023-07-06T03:49:48.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115400ba10 con 0x7f11680d2210 2023-07-06T03:49:48.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116e99c700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 0x7f1154005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116e19b700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11680d2210 0x7f1154003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116d99a700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1154003fd0 0x7f1154004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116e99c700 1 -- 172.21.15.97:0/3270646150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11680d2210 msgr2=0x7f1154003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:48.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116e99c700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11680d2210 0x7f1154003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116e99c700 1 -- 172.21.15.97:0/3270646150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1154003fd0 msgr2=0x7f1154004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:48.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116e99c700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1154003fd0 0x7f1154004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.427+0000 7f116e99c700 1 -- 172.21.15.97:0/3270646150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11540035b0 con 0x7f1154004d90 2023-07-06T03:49:48.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.428+0000 7f116e99c700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 0x7f1154005200 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1160009fc0 tx=0x7f1154005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ae636df94bb43d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:48.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.428+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116000ec30 con 0x7f1154004d90 2023-07-06T03:49:48.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.428+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116000f040 con 0x7f1154004d90 2023-07-06T03:49:48.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.428+0000 7f116d199700 1 -- 172.21.15.97:0/3270646150 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160014480 con 0x7f1154004d90 2023-07-06T03:49:48.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.428+0000 7f11703ff700 1 -- 172.21.15.97:0/3270646150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 msgr2=0x7f1154005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.428+0000 7f11703ff700 1 --2- 172.21.15.97:0/3270646150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 0x7f1154005200 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1160009fc0 tx=0x7f1154005880 comp rx=0 tx=0).stop 2023-07-06T03:49:48.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.428+0000 7f116e99c700 1 -- 172.21.15.97:0/3270646150 reap_dead start 2023-07-06T03:49:48.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.428+0000 7f11703ff700 1 -- 172.21.15.97:0/3270646150 shutdown_connections 2023-07-06T03:49:48.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.428+0000 7f11703ff700 1 -- 172.21.15.97:0/3270646150 >> 172.21.15.97:0/3270646150 conn(0x7f116806a6c0 msgr2=0x7f116806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.429+0000 7f11703ff700 1 -- 172.21.15.97:0/3270646150 shutdown_connections 2023-07-06T03:49:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.429+0000 7f11703ff700 1 -- 172.21.15.97:0/3270646150 wait complete. 2023-07-06T03:49:48.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.430+0000 7f11703ff700 1 Processor -- start 2023-07-06T03:49:48.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.430+0000 7f11703ff700 1 -- start start 2023-07-06T03:49:48.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.431+0000 7f11703ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 0x7f1168153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.431+0000 7f11703ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f116815f4a0 0x7f116815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.431+0000 7f11703ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11681f1ab0 0x7f11680d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.431+0000 7f11703ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11680ca130 con 0x7f1154004d90 2023-07-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.431+0000 7f11703ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116815e1d0 con 0x7f116815f4a0 2023-07-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.431+0000 7f11703ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116815e400 con 0x7f11681f1ab0 2023-07-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f116e19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 0x7f1168153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f116e19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 0x7f1168153390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44826/0 (socket says 172.21.15.97:44826) 2023-07-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f116e19b700 1 -- 172.21.15.97:0/3628987543 learned_addr learned my addr 172.21.15.97:0/3628987543 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f116d99a700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f116815f4a0 0x7f116815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f116e99c700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11681f1ab0 0x7f11680d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f116e19b700 1 -- 172.21.15.97:0/3628987543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11681f1ab0 msgr2=0x7f11680d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f116e19b700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11681f1ab0 0x7f11680d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f116e19b700 1 -- 172.21.15.97:0/3628987543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f116815f4a0 msgr2=0x7f116815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f116e19b700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f116815f4a0 0x7f116815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f116e19b700 1 -- 172.21.15.97:0/3628987543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116815e6e0 con 0x7f1154004d90 2023-07-06T03:49:48.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f116e19b700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 0x7f1168153390 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1164007310 tx=0x7f11640076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:48.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f115effd700 1 -- 172.21.15.97:0/3628987543 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11640077f0 con 0x7f1154004d90 2023-07-06T03:49:48.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f115effd700 1 -- 172.21.15.97:0/3628987543 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116400f8d0 con 0x7f1154004d90 2023-07-06T03:49:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f115effd700 1 -- 172.21.15.97:0/3628987543 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116400fc80 con 0x7f1154004d90 2023-07-06T03:49:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.432+0000 7f11703ff700 1 -- 172.21.15.97:0/3628987543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116815e9c0 con 0x7f1154004d90 2023-07-06T03:49:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.433+0000 7f11703ff700 1 -- 172.21.15.97:0/3628987543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116815ef80 con 0x7f1154004d90 2023-07-06T03:49:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.434+0000 7f11703ff700 1 -- 172.21.15.97:0/3628987543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116804e340 con 0x7f1154004d90 2023-07-06T03:49:48.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.435+0000 7f115effd700 1 -- 172.21.15.97:0/3628987543 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f116401d460 con 0x7f1154004d90 2023-07-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.436+0000 7f115effd700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f114c06ace0 0x7f114c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.436+0000 7f115effd700 1 -- 172.21.15.97:0/3628987543 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1164089690 con 0x7f1154004d90 2023-07-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.436+0000 7f116d99a700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f114c06ace0 0x7f114c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.436+0000 7f116d99a700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f114c06ace0 0x7f114c06d190 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f11680c9b40 tx=0x7f1158008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:48.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.440+0000 7f115effd700 1 -- 172.21.15.97:0/3628987543 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f11640546a0 con 0x7f1154004d90 2023-07-06T03:49:48.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.635+0000 7f11703ff700 1 -- 172.21.15.97:0/3628987543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7f11680b5f50 con 0x7f1154004d90 2023-07-06T03:49:48.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.635+0000 7f115effd700 1 -- 172.21.15.97:0/3628987543 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v26) v1 ==== 112+0+10 (secure 0 0 0) 0x7f1164057cc0 con 0x7f1154004d90 2023-07-06T03:49:48.638 INFO:teuthology.orchestra.run.smithi097.stdout:60.000000 2023-07-06T03:49:48.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 -- 172.21.15.97:0/3628987543 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f114c06ace0 msgr2=0x7f114c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f114c06ace0 0x7f114c06d190 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f11680c9b40 tx=0x7f1158008040 comp rx=0 tx=0).stop 2023-07-06T03:49:48.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 -- 172.21.15.97:0/3628987543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 msgr2=0x7f1168153390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:48.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 0x7f1168153390 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1164007310 tx=0x7f11640076d0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 -- 172.21.15.97:0/3628987543 shutdown_connections 2023-07-06T03:49:48.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f114c06ace0 0x7f114c06d190 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11681f1ab0 0x7f11680d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1154004d90 0x7f1168153390 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 --2- 172.21.15.97:0/3628987543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f116815f4a0 0x7f116815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:48.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 -- 172.21.15.97:0/3628987543 >> 172.21.15.97:0/3628987543 conn(0x7f116806a6c0 msgr2=0x7f1168159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:48.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 -- 172.21.15.97:0/3628987543 shutdown_connections 2023-07-06T03:49:48.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:48.638+0000 7f11703ff700 1 -- 172.21.15.97:0/3628987543 wait complete. 2023-07-06T03:49:48.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:49:48.658 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:49:48.659 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:49:48.659 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:49:48.660 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:49:48.660 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:48.661 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:49:48.661 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:49:48.662 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:49:48.662 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:49:48.662 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:48.662 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:49:48.663 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:49:48.663 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:49:48.663 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:49:48.664 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:48.664 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:49:48.664 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:49:48.664 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:49:48.665 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:49:48.665 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:49:48.665 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:49:48.666 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:49:48.666 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:49:48.666 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:48.666 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:49:48.667 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:49:48.808 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.143292.asok 2023-07-06T03:49:48.986 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.143292.asok 2023-07-06T03:49:48.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:49:48.987 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.143292.asok mds_sessions 2023-07-06T03:49:49.207 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:49:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7005, 2023-07-06T03:49:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:49:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:49:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:49:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 7005 2023-07-06T03:49:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:49:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:49:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2023-07-06T03:49:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2398905682 2023-07-06T03:49:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.7005 v1:172.21.15.97:0/2398905682", 2023-07-06T03:49:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "v1:172.21.15.97:0/2398905682", 2023-07-06T03:49:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions": [ 2023-07-06T03:49:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": 0, 2023-07-06T03:49:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: "addrs": { 2023-07-06T03:49:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-07-06T03:49:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-07-06T03:49:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6835", 2023-07-06T03:49:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1957695290 2023-07-06T03:49:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:49:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:49:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6837", 2023-07-06T03:49:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1957695290 2023-07-06T03:49:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:49:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:49:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: "seq": 0, 2023-07-06T03:49:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_gen": 0, 2023-07-06T03:49:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_ttl": "2023-07-06T03:50:43.976824+0000", 2023-07-06T03:49:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: "last_cap_renew_request": "2023-07-06T03:49:43.976824+0000", 2023-07-06T03:49:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_renew_seq": 2, 2023-07-06T03:49:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: "num_caps": 1, 2023-07-06T03:49:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "open" 2023-07-06T03:49:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:49:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-07-06T03:49:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 44 2023-07-06T03:49:49.215 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:49:49.222 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:49:49.222 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:49:49.267 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:49:49.268 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2023-07-06T03:49:49.268 DEBUG:teuthology.orchestra.run.smithi097:> import time 2023-07-06T03:49:49.268 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:49:49.268 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:49.269 DEBUG:teuthology.orchestra.run.smithi097:> n = 400 2023-07-06T03:49:49.269 DEBUG:teuthology.orchestra.run.smithi097:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2023-07-06T03:49:49.269 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:49.270 DEBUG:teuthology.orchestra.run.smithi097:> if not os.path.exists(abs_path): 2023-07-06T03:49:49.270 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(abs_path) 2023-07-06T03:49:49.270 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:49.270 DEBUG:teuthology.orchestra.run.smithi097:> handles = [] 2023-07-06T03:49:49.271 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(0, n): 2023-07-06T03:49:49.271 DEBUG:teuthology.orchestra.run.smithi097:> fname = "file_"+str(i) 2023-07-06T03:49:49.271 DEBUG:teuthology.orchestra.run.smithi097:> path = os.path.join(abs_path, fname) 2023-07-06T03:49:49.272 DEBUG:teuthology.orchestra.run.smithi097:> handles.append(open(path, '"'"'w'"'"')) 2023-07-06T03:49:49.273 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:49:49.273 DEBUG:teuthology.orchestra.run.smithi097:> while True: 2023-07-06T03:49:49.273 DEBUG:teuthology.orchestra.run.smithi097:> time.sleep(1) 2023-07-06T03:49:49.274 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:49:49.312 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:49:49.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.518+0000 7f1b47dff700 1 Processor -- start 2023-07-06T03:49:49.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.518+0000 7f1b47dff700 1 -- start start 2023-07-06T03:49:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.519+0000 7f1b47dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1b4015c520 0x7f1b400cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.519+0000 7f1b47dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b40160d00 0x7f1b400b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.519+0000 7f1b47dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b400cdae0 con 0x7f1b40160180 2023-07-06T03:49:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.519+0000 7f1b47dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b400cd8d0 con 0x7f1b4015c520 2023-07-06T03:49:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.519+0000 7f1b47dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b400b6640 con 0x7f1b40160d00 2023-07-06T03:49:49.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.520+0000 7f1b45b9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1b4015c520 0x7f1b400cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.520+0000 7f1b45b9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1b4015c520 0x7f1b400cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:44412/0 (socket says 172.21.15.97:44412) 2023-07-06T03:49:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.520+0000 7f1b45b9b700 1 -- 172.21.15.97:0/399748940 learned_addr learned my addr 172.21.15.97:0/399748940 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.520+0000 7f1b4639c700 1 --2- 172.21.15.97:0/399748940 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b40160d00 0x7f1b400b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.520+0000 7f1b4539a700 1 --1- 172.21.15.97:0/399748940 >> v1:172.21.15.146:6789/0 conn(0x7f1b40160180 0x7f1b401605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:55706/0 (socket says 172.21.15.97:55706) 2023-07-06T03:49:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.520+0000 7f1b45b9b700 1 -- 172.21.15.97:0/399748940 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b40160d00 msgr2=0x7f1b400b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.520+0000 7f1b45b9b700 1 --2- 172.21.15.97:0/399748940 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b40160d00 0x7f1b400b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.520+0000 7f1b45b9b700 1 -- 172.21.15.97:0/399748940 >> v1:172.21.15.146:6789/0 conn(0x7f1b40160180 legacy=0x7f1b401605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.520+0000 7f1b45b9b700 1 -- 172.21.15.97:0/399748940 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b400b6920 con 0x7f1b4015c520 2023-07-06T03:49:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.520+0000 7f1b44b99700 1 -- 172.21.15.97:0/399748940 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085488158 0 0) 0x7f1b400cdae0 con 0x7f1b40160180 2023-07-06T03:49:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.521+0000 7f1b45b9b700 1 --2- 172.21.15.97:0/399748940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b4015c520 0x7f1b400cd390 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0098f0 tx=0x7f1b3c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2531fce5d4dd344a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.521+0000 7f1b44b99700 1 -- 172.21.15.97:0/399748940 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c011830 con 0x7f1b4015c520 2023-07-06T03:49:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.521+0000 7f1b44b99700 1 -- 172.21.15.97:0/399748940 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c011990 con 0x7f1b4015c520 2023-07-06T03:49:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.521+0000 7f1b44b99700 1 -- 172.21.15.97:0/399748940 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c011d20 con 0x7f1b4015c520 2023-07-06T03:49:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.522+0000 7f1b47dff700 1 -- 172.21.15.97:0/399748940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b4015c520 msgr2=0x7f1b400cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.522+0000 7f1b47dff700 1 --2- 172.21.15.97:0/399748940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b4015c520 0x7f1b400cd390 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0098f0 tx=0x7f1b3c009c00 comp rx=0 tx=0).stop 2023-07-06T03:49:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.522+0000 7f1b47dff700 1 -- 172.21.15.97:0/399748940 shutdown_connections 2023-07-06T03:49:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.522+0000 7f1b47dff700 1 --2- 172.21.15.97:0/399748940 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b40160d00 0x7f1b400b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:49.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.522+0000 7f1b47dff700 1 --2- 172.21.15.97:0/399748940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b4015c520 0x7f1b400cd390 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:49.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.522+0000 7f1b47dff700 1 -- 172.21.15.97:0/399748940 >> 172.21.15.97:0/399748940 conn(0x7f1b40071790 msgr2=0x7f1b40071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:49.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.522+0000 7f1b47dff700 1 -- 172.21.15.97:0/399748940 shutdown_connections 2023-07-06T03:49:49.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.522+0000 7f1b47dff700 1 -- 172.21.15.97:0/399748940 wait complete. 2023-07-06T03:49:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.524+0000 7f1b47dff700 1 Processor -- start 2023-07-06T03:49:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.524+0000 7f1b47dff700 1 -- start start 2023-07-06T03:49:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.525+0000 7f1b47dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4015c520 0x7f1b400cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.525+0000 7f1b47dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b40160180 0x7f1b4014cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.525+0000 7f1b47dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b40160d00 0x7f1b400d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.525+0000 7f1b47dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4015a7e0 con 0x7f1b4015c520 2023-07-06T03:49:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.525+0000 7f1b47dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4014d270 con 0x7f1b40160d00 2023-07-06T03:49:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.525+0000 7f1b47dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b400d1170 con 0x7f1b40160180 2023-07-06T03:49:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.525+0000 7f1b45b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4015c520 0x7f1b400cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.526+0000 7f1b45b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4015c520 0x7f1b400cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44838/0 (socket says 172.21.15.97:44838) 2023-07-06T03:49:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.526+0000 7f1b45b9b700 1 -- 172.21.15.97:0/1777679227 learned_addr learned my addr 172.21.15.97:0/1777679227 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.526+0000 7f1b4539a700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b40160180 0x7f1b4014cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.526+0000 7f1b4639c700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b40160d00 0x7f1b400d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.526+0000 7f1b45b9b700 1 -- 172.21.15.97:0/1777679227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b40160180 msgr2=0x7f1b4014cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.526+0000 7f1b45b9b700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b40160180 0x7f1b4014cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.526+0000 7f1b45b9b700 1 -- 172.21.15.97:0/1777679227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b40160d00 msgr2=0x7f1b400d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.526+0000 7f1b45b9b700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b40160d00 0x7f1b400d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.526+0000 7f1b45b9b700 1 -- 172.21.15.97:0/1777679227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c009550 con 0x7f1b4015c520 2023-07-06T03:49:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.526+0000 7f1b45b9b700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4015c520 0x7f1b400cccb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009c40 tx=0x7f1b3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.526+0000 7f1b367fc700 1 -- 172.21.15.97:0/1777679227 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c004300 con 0x7f1b4015c520 2023-07-06T03:49:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.527+0000 7f1b367fc700 1 -- 172.21.15.97:0/1777679227 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c01f070 con 0x7f1b4015c520 2023-07-06T03:49:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.527+0000 7f1b47dff700 1 -- 172.21.15.97:0/1777679227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4015a9c0 con 0x7f1b4015c520 2023-07-06T03:49:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.527+0000 7f1b367fc700 1 -- 172.21.15.97:0/1777679227 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c0036a0 con 0x7f1b4015c520 2023-07-06T03:49:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.527+0000 7f1b47dff700 1 -- 172.21.15.97:0/1777679227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b400d1b10 con 0x7f1b4015c520 2023-07-06T03:49:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.528+0000 7f1b47dff700 1 -- 172.21.15.97:0/1777679227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b400ccb00 con 0x7f1b4015c520 2023-07-06T03:49:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.530+0000 7f1b367fc700 1 -- 172.21.15.97:0/1777679227 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b3c021030 con 0x7f1b4015c520 2023-07-06T03:49:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.531+0000 7f1b367fc700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2406ace0 0x7f1b2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.531+0000 7f1b367fc700 1 -- 172.21.15.97:0/1777679227 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b3c08a980 con 0x7f1b4015c520 2023-07-06T03:49:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.532+0000 7f1b4539a700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2406ace0 0x7f1b2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.532+0000 7f1b4539a700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2406ace0 0x7f1b2406d190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1b40151010 tx=0x7f1b30007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.536+0000 7f1b367fc700 1 -- 172.21.15.97:0/1777679227 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1b3c055b60 con 0x7f1b4015c520 2023-07-06T03:49:49.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.811+0000 7f1b47dff700 1 -- 172.21.15.97:0/1777679227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b4004e340 con 0x7f1b4015c520 2023-07-06T03:49:49.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.811+0000 7f1b367fc700 1 -- 172.21.15.97:0/1777679227 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f1b3c015090 con 0x7f1b4015c520 2023-07-06T03:49:49.813 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:49.813 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2344840493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2344840493},{"type":"v1","addr":"172.21.15.97:6836","nonce":2344840493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1930131645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1930131645},{"type":"v1","addr":"172.21.15.146:6835","nonce":1930131645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3936349837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3936349837},{"type":"v1","addr":"172.21.15.146:6833","nonce":3936349837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:49:34.310076+0000","modified":"2023-07-06T03:49:37.864717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6837":{"gid":6837,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6837/1957695290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1957695290},{"type":"v1","addr":"172.21.15.97:6837","nonce":1957695290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-06T03:49:49.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 -- 172.21.15.97:0/1777679227 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2406ace0 msgr2=0x7f1b2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:49.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2406ace0 0x7f1b2406d190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1b40151010 tx=0x7f1b30007de0 comp rx=0 tx=0).stop 2023-07-06T03:49:49.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 -- 172.21.15.97:0/1777679227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4015c520 msgr2=0x7f1b400cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:49.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4015c520 0x7f1b400cccb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009c40 tx=0x7f1b3c004750 comp rx=0 tx=0).stop 2023-07-06T03:49:49.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 -- 172.21.15.97:0/1777679227 shutdown_connections 2023-07-06T03:49:49.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b2406ace0 0x7f1b2406d190 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:49.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b40160d00 0x7f1b400d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b4015c520 0x7f1b400cccb0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 --2- 172.21.15.97:0/1777679227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b40160180 0x7f1b4014cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 -- 172.21.15.97:0/1777679227 >> 172.21.15.97:0/1777679227 conn(0x7f1b40071790 msgr2=0x7f1b400c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 -- 172.21.15.97:0/1777679227 shutdown_connections 2023-07-06T03:49:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:49.814+0000 7f1b47dff700 1 -- 172.21.15.97:0/1777679227 wait complete. 2023-07-06T03:49:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 44 2023-07-06T03:49:49.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-07-06T03:49:50.100 INFO:teuthology.orchestra.run.smithi097.stdout:[{"id":5933,"entity":{"name":{"type":"client","num":5933},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":671928949}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":4.8069203959999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9859237129855307,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5933 192.168.0.1:0/671928949","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"123196","root":"/"}},{"id":7005,"entity":{"name":{"type":"client","num":7005},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":2398905682}},"state":"open","num_leases":1,"num_caps":368,"request_load_avg":369,"uptime":7.1038823799999999,"requests_in_flight":366,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":146.60654915466912,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":132.05620126829641,"halflife":1.5},"recall_caps_throttle2o":{"value":106.57166012645652,"halflife":0.5},"session_cache_liveness":{"value":737.40324756080611,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7005 172.21.15.97:0/2398905682","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"143292","root":"/"}}] 2023-07-06T03:49:50.101 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "123196", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 671928949, "type": "any" }, "name": { "num": 5933, "type": "client" } }, "id": 5933, "inst": "client.5933 192.168.0.1:0/671928949", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9859237129855307 }, "state": "open", "uptime": 4.806920396 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "143292", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 2398905682, "type": "any" }, "name": { "num": 7005, "type": "client" } }, "id": 7005, "inst": "client.7005 172.21.15.97:0/2398905682", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 368, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 146.60654915466912 }, "recall_caps_throttle": { "halflife": 1.5, "value": 132.0562012682964 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 106.57166012645652 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 369, "requests_in_flight": 366, "session_cache_liveness": { "halflife": 300, "value": 737.4032475608061 }, "state": "open", "uptime": 7.10388238 } ] 2023-07-06T03:49:50.101 DEBUG:tasks.ceph_test_case:wait_until_equal: 368 != 402, waiting (timeout=600)... 2023-07-06T03:49:55.102 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:49:55.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.304+0000 7f2f713ff700 1 Processor -- start 2023-07-06T03:49:55.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.304+0000 7f2f713ff700 1 -- start start 2023-07-06T03:49:55.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.305+0000 7f2f713ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2f6c0d2930 0x7f2f6c0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:55.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.305+0000 7f2f713ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 0x7f2f6c0b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:55.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.305+0000 7f2f713ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6c0d0c70 con 0x7f2f6c15c980 2023-07-06T03:49:55.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.305+0000 7f2f713ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c0b2310 con 0x7f2f6c0d2930 2023-07-06T03:49:55.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.305+0000 7f2f713ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0d0ec0 con 0x7f2f6c0b1990 2023-07-06T03:49:55.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.306+0000 7f2f63fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2f6c0d2930 0x7f2f6c0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.306+0000 7f2f6b7fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 0x7f2f6c0b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.306+0000 7f2f63fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2f6c0d2930 0x7f2f6c0d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:56336/0 (socket says 172.21.15.97:56336) 2023-07-06T03:49:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.306+0000 7f2f63fff700 1 -- 172.21.15.97:0/685295657 learned_addr learned my addr 172.21.15.97:0/685295657 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.306+0000 7f2f6affd700 1 --1- 172.21.15.97:0/685295657 >> v1:172.21.15.146:6789/0 conn(0x7f2f6c15c980 0x7f2f6c0cd6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:51866/0 (socket says 172.21.15.97:51866) 2023-07-06T03:49:55.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.306+0000 7f2f6b7fe700 1 -- 172.21.15.97:0/685295657 >> v2:172.21.15.146:3300/0 conn(0x7f2f6c0d2930 msgr2=0x7f2f6c0d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:55.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.306+0000 7f2f6b7fe700 1 --2- 172.21.15.97:0/685295657 >> v2:172.21.15.146:3300/0 conn(0x7f2f6c0d2930 0x7f2f6c0d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:55.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.306+0000 7f2f6b7fe700 1 -- 172.21.15.97:0/685295657 >> v1:172.21.15.146:6789/0 conn(0x7f2f6c15c980 legacy=0x7f2f6c0cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:55.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.306+0000 7f2f6b7fe700 1 -- 172.21.15.97:0/685295657 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c0d11a0 con 0x7f2f6c0b1990 2023-07-06T03:49:55.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.306+0000 7f2f6b7fe700 1 --2- 172.21.15.97:0/685295657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 0x7f2f6c0b1dd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009d90 tx=0x7f2f6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb1661c62883f60a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:55.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.307+0000 7f2f6a7fc700 1 -- 172.21.15.97:0/685295657 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6400e070 con 0x7f2f6c0b1990 2023-07-06T03:49:55.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.307+0000 7f2f6a7fc700 1 -- 172.21.15.97:0/685295657 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6400bb20 con 0x7f2f6c0b1990 2023-07-06T03:49:55.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.307+0000 7f2f6a7fc700 1 -- 172.21.15.97:0/685295657 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64012430 con 0x7f2f6c0b1990 2023-07-06T03:49:55.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.307+0000 7f2f713ff700 1 -- 172.21.15.97:0/685295657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 msgr2=0x7f2f6c0b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:55.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.307+0000 7f2f713ff700 1 --2- 172.21.15.97:0/685295657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 0x7f2f6c0b1dd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009d90 tx=0x7f2f6400b330 comp rx=0 tx=0).stop 2023-07-06T03:49:55.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.307+0000 7f2f713ff700 1 -- 172.21.15.97:0/685295657 shutdown_connections 2023-07-06T03:49:55.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.307+0000 7f2f713ff700 1 --2- 172.21.15.97:0/685295657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 0x7f2f6c0b1dd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:55.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.307+0000 7f2f713ff700 1 --2- 172.21.15.97:0/685295657 >> v2:172.21.15.146:3300/0 conn(0x7f2f6c0d2930 0x7f2f6c0d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:55.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.307+0000 7f2f713ff700 1 -- 172.21.15.97:0/685295657 >> 172.21.15.97:0/685295657 conn(0x7f2f6c06b2f0 msgr2=0x7f2f6c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.307+0000 7f2f713ff700 1 -- 172.21.15.97:0/685295657 shutdown_connections 2023-07-06T03:49:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.307+0000 7f2f713ff700 1 -- 172.21.15.97:0/685295657 wait complete. 2023-07-06T03:49:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.309+0000 7f2f713ff700 1 Processor -- start 2023-07-06T03:49:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.309+0000 7f2f713ff700 1 -- start start 2023-07-06T03:49:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.310+0000 7f2f713ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 0x7f2f6c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:55.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.310+0000 7f2f713ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f6c0d2930 0x7f2f6c15ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:55.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.310+0000 7f2f713ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15c980 0x7f2f6c0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:55.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.310+0000 7f2f713ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0ca390 con 0x7f2f6c15c980 2023-07-06T03:49:55.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.310+0000 7f2f713ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c15ff80 con 0x7f2f6c0d2930 2023-07-06T03:49:55.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.310+0000 7f2f713ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0cc5c0 con 0x7f2f6c0b1990 2023-07-06T03:49:55.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.310+0000 7f2f6b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15c980 0x7f2f6c0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:55.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.310+0000 7f2f6b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15c980 0x7f2f6c0b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34422/0 (socket says 172.21.15.97:34422) 2023-07-06T03:49:55.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.310+0000 7f2f6b7fe700 1 -- 172.21.15.97:0/370891997 learned_addr learned my addr 172.21.15.97:0/370891997 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:55.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f63fff700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f6c0d2930 0x7f2f6c15ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:55.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f6affd700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 0x7f2f6c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f6b7fe700 1 -- 172.21.15.97:0/370891997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 msgr2=0x7f2f6c158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f6b7fe700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 0x7f2f6c158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f6b7fe700 1 -- 172.21.15.97:0/370891997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f6c0d2930 msgr2=0x7f2f6c15ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f6b7fe700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f6c0d2930 0x7f2f6c15ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f6b7fe700 1 -- 172.21.15.97:0/370891997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64009a70 con 0x7f2f6c15c980 2023-07-06T03:49:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f6affd700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 0x7f2f6c158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:55.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f6b7fe700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15c980 0x7f2f6c0b3c10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2f64014040 tx=0x7f2f64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f637fe700 1 -- 172.21.15.97:0/370891997 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64012430 con 0x7f2f6c15c980 2023-07-06T03:49:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f637fe700 1 -- 172.21.15.97:0/370891997 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f64020070 con 0x7f2f6c15c980 2023-07-06T03:49:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f713ff700 1 -- 172.21.15.97:0/370891997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6c0cc8a0 con 0x7f2f6c15c980 2023-07-06T03:49:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f637fe700 1 -- 172.21.15.97:0/370891997 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64003ea0 con 0x7f2f6c15c980 2023-07-06T03:49:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.311+0000 7f2f713ff700 1 -- 172.21.15.97:0/370891997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c158780 con 0x7f2f6c15c980 2023-07-06T03:49:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.312+0000 7f2f713ff700 1 -- 172.21.15.97:0/370891997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6c0ccdc0 con 0x7f2f6c15c980 2023-07-06T03:49:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.313+0000 7f2f637fe700 1 -- 172.21.15.97:0/370891997 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f64004280 con 0x7f2f6c15c980 2023-07-06T03:49:55.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.314+0000 7f2f637fe700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4c06ace0 0x7f2f4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:55.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.314+0000 7f2f637fe700 1 -- 172.21.15.97:0/370891997 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2f64022070 con 0x7f2f6c15c980 2023-07-06T03:49:55.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.314+0000 7f2f6affd700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4c06ace0 0x7f2f4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:55.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.315+0000 7f2f6affd700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4c06ace0 0x7f2f4c06d190 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c000f60 tx=0x7f2f5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:55.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.318+0000 7f2f637fe700 1 -- 172.21.15.97:0/370891997 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2f6c0ccdc0 con 0x7f2f6c15c980 2023-07-06T03:49:55.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.556+0000 7f2f713ff700 1 -- 172.21.15.97:0/370891997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f6c0ccdc0 con 0x7f2f6c15c980 2023-07-06T03:49:55.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.557+0000 7f2f637fe700 1 -- 172.21.15.97:0/370891997 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f2f6c0ccdc0 con 0x7f2f6c15c980 2023-07-06T03:49:55.560 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:55.560 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2344840493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2344840493},{"type":"v1","addr":"172.21.15.97:6836","nonce":2344840493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1930131645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1930131645},{"type":"v1","addr":"172.21.15.146:6835","nonce":1930131645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3936349837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3936349837},{"type":"v1","addr":"172.21.15.146:6833","nonce":3936349837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:49:34.310076+0000","modified":"2023-07-06T03:49:50.919249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6837":{"gid":6837,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/1957695290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1957695290},{"type":"v1","addr":"172.21.15.97:6837","nonce":1957695290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-06T03:49:55.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 -- 172.21.15.97:0/370891997 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4c06ace0 msgr2=0x7f2f4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:55.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4c06ace0 0x7f2f4c06d190 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c000f60 tx=0x7f2f5c009040 comp rx=0 tx=0).stop 2023-07-06T03:49:55.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 -- 172.21.15.97:0/370891997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15c980 msgr2=0x7f2f6c0b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:55.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15c980 0x7f2f6c0b3c10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2f64014040 tx=0x7f2f64004ad0 comp rx=0 tx=0).stop 2023-07-06T03:49:55.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 -- 172.21.15.97:0/370891997 shutdown_connections 2023-07-06T03:49:55.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4c06ace0 0x7f2f4c06d190 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:55.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f6c15c980 0x7f2f6c0b3c10 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:55.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f6c0b1990 0x7f2f6c158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:55.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 --2- 172.21.15.97:0/370891997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f6c0d2930 0x7f2f6c15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:55.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 -- 172.21.15.97:0/370891997 >> 172.21.15.97:0/370891997 conn(0x7f2f6c06b2f0 msgr2=0x7f2f6c073030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:55.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 -- 172.21.15.97:0/370891997 shutdown_connections 2023-07-06T03:49:55.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:55.559+0000 7f2f713ff700 1 -- 172.21.15.97:0/370891997 wait complete. 2023-07-06T03:49:55.564 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 45 2023-07-06T03:49:55.579 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-07-06T03:49:55.811 INFO:teuthology.orchestra.run.smithi097.stdout:[{"id":5933,"entity":{"name":{"type":"client","num":5933},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":671928949}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":10.518825764000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9467764582936149,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5933 192.168.0.1:0/671928949","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"123196","root":"/"}},{"id":7005,"entity":{"name":{"type":"client","num":7005},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":2398905682}},"state":"open","num_leases":1,"num_caps":402,"request_load_avg":377,"uptime":12.815787748,"requests_in_flight":0,"num_completed_requests":400,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1103.9740709844307,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":324.84866957991272,"halflife":1.5},"recall_caps_throttle2o":{"value":72.595084323338654,"halflife":0.5},"session_cache_liveness":{"value":794.84840073517171,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7005 172.21.15.97:0/2398905682","completed_requests":[{"tid":5,"created_ino":"0x10000000001"},{"tid":6,"created_ino":"0x100000001f6"},{"tid":7,"created_ino":"0x100000001f7"},{"tid":8,"created_ino":"0x100000001f8"},{"tid":9,"created_ino":"0x100000001f9"},{"tid":10,"created_ino":"0x100000001fa"},{"tid":11,"created_ino":"0x100000001fb"},{"tid":12,"created_ino":"0x100000001fc"},{"tid":13,"created_ino":"0x100000001fd"},{"tid":14,"created_ino":"0x100000001fe"},{"tid":15,"created_ino":"0x100000001ff"},{"tid":16,"created_ino":"0x10000000200"},{"tid":17,"created_ino":"0x10000000201"},{"tid":18,"created_ino":"0x10000000202"},{"tid":19,"created_ino":"0x10000000203"},{"tid":20,"created_ino":"0x10000000204"},{"tid":21,"created_ino":"0x10000000205"},{"tid":22,"created_ino":"0x10000000206"},{"tid":23,"created_ino":"0x10000000207"},{"tid":24,"created_ino":"0x10000000208"},{"tid":25,"created_ino":"0x10000000209"},{"tid":26,"created_ino":"0x1000000020a"},{"tid":27,"created_ino":"0x1000000020b"},{"tid":28,"created_ino":"0x1000000020c"},{"tid":29,"created_ino":"0x1000000020d"},{"tid":30,"created_ino":"0x1000000020e"},{"tid":31,"created_ino":"0x1000000020f"},{"tid":32,"created_ino":"0x10000000210"},{"tid":33,"created_ino":"0x10000000211"},{"tid":34,"created_ino":"0x10000000212"},{"tid":35,"created_ino":"0x10000000213"},{"tid":36,"created_ino":"0x10000000214"},{"tid":37,"created_ino":"0x10000000215"},{"tid":38,"created_ino":"0x10000000216"},{"tid":39,"created_ino":"0x10000000217"},{"tid":40,"created_ino":"0x10000000218"},{"tid":41,"created_ino":"0x10000000219"},{"tid":42,"created_ino":"0x1000000021a"},{"tid":43,"created_ino":"0x1000000021b"},{"tid":44,"created_ino":"0x1000000021c"},{"tid":45,"created_ino":"0x1000000021d"},{"tid":46,"created_ino":"0x1000000021e"},{"tid":47,"created_ino":"0x1000000021f"},{"tid":48,"created_ino":"0x10000000220"},{"tid":49,"created_ino":"0x10000000221"},{"tid":50,"created_ino":"0x10000000222"},{"tid":51,"created_ino":"0x10000000223"},{"tid":52,"created_ino":"0x10000000224"},{"tid":53,"created_ino":"0x10000000225"},{"tid":54,"created_ino":"0x10000000226"},{"tid":55,"created_ino":"0x10000000227"},{"tid":56,"created_ino":"0x10000000228"},{"tid":57,"created_ino":"0x10000000229"},{"tid":58,"created_ino":"0x1000000022a"},{"tid":59,"created_ino":"0x1000000022b"},{"tid":60,"created_ino":"0x1000000022c"},{"tid":61,"created_ino":"0x1000000022d"},{"tid":62,"created_ino":"0x1000000022e"},{"tid":63,"created_ino":"0x1000000022f"},{"tid":64,"created_ino":"0x10000000230"},{"tid":65,"created_ino":"0x10000000231"},{"tid":66,"created_ino":"0x10000000232"},{"tid":67,"created_ino":"0x10000000233"},{"tid":68,"created_ino":"0x10000000234"},{"tid":69,"created_ino":"0x10000000235"},{"tid":70,"created_ino":"0x10000000236"},{"tid":71,"created_ino":"0x10000000237"},{"tid":72,"created_ino":"0x10000000238"},{"tid":73,"created_ino":"0x10000000239"},{"tid":74,"created_ino":"0x1000000023a"},{"tid":75,"created_ino":"0x1000000023b"},{"tid":76,"created_ino":"0x1000000023c"},{"tid":77,"created_ino":"0x1000000023d"},{"tid":78,"created_ino":"0x1000000023e"},{"tid":79,"created_ino":"0x1000000023f"},{"tid":80,"created_ino":"0x10000000240"},{"tid":81,"created_ino":"0x10000000241"},{"tid":82,"created_ino":"0x10000000242"},{"tid":83,"created_ino":"0x10000000243"},{"tid":84,"created_ino":"0x10000000244"},{"tid":85,"created_ino":"0x10000000245"},{"tid":86,"created_ino":"0x10000000246"},{"tid":87,"created_ino":"0x10000000247"},{"tid":88,"created_ino":"0x10000000248"},{"tid":89,"created_ino":"0x10000000249"},{"tid":90,"created_ino":"0x1000000024a"},{"tid":91,"created_ino":"0x1000000024b"},{"tid":92,"created_ino":"0x1000000024c"},{"tid":93,"created_ino":"0x1000000024d"},{"tid":94,"created_ino":"0x1000000024e"},{"tid":95,"created_ino":"0x1000000024f"},{"tid":96,"created_ino":"0x10000000250"},{"tid":97,"created_ino":"0x10000000251"},{"tid":98,"created_ino":"0x10000000252"},{"tid":99,"created_ino":"0x10000000253"},{"tid":100,"created_ino":"0x10000000254"},{"tid":101,"created_ino":"0x10000000255"},{"tid":102,"created_ino":"0x10000000256"},{"tid":103,"created_ino":"0x10000000257"},{"tid":104,"created_ino":"0x10000000258"},{"tid":105,"created_ino":"0x10000000259"},{"tid":106,"created_ino":"0x1000000025a"},{"tid":107,"created_ino":"0x1000000025b"},{"tid":108,"created_ino":"0x1000000025c"},{"tid":109,"created_ino":"0x1000000025d"},{"tid":110,"created_ino":"0x1000000025e"},{"tid":111,"created_ino":"0x1000000025f"},{"tid":112,"created_ino":"0x10000000260"},{"tid":113,"created_ino":"0x10000000261"},{"tid":114,"created_ino":"0x10000000262"},{"tid":115,"created_ino":"0x10000000263"},{"tid":116,"created_ino":"0x10000000264"},{"tid":117,"created_ino":"0x10000000265"},{"tid":118,"created_ino":"0x10000000266"},{"tid":119,"created_ino":"0x10000000267"},{"tid":120,"created_ino":"0x10000000268"},{"tid":121,"created_ino":"0x10000000269"},{"tid":122,"created_ino":"0x1000000026a"},{"tid":123,"created_ino":"0x1000000026b"},{"tid":124,"created_ino":"0x1000000026c"},{"tid":125,"created_ino":"0x1000000026d"},{"tid":126,"created_ino":"0x1000000026e"},{"tid":127,"created_ino":"0x1000000026f"},{"tid":128,"created_ino":"0x10000000270"},{"tid":129,"created_ino":"0x10000000271"},{"tid":130,"created_ino":"0x10000000272"},{"tid":131,"created_ino":"0x10000000273"},{"tid":132,"created_ino":"0x10000000274"},{"tid":133,"created_ino":"0x10000000275"},{"tid":134,"created_ino":"0x10000000276"},{"tid":135,"created_ino":"0x10000000277"},{"tid":136,"created_ino":"0x10000000278"},{"tid":137,"created_ino":"0x10000000279"},{"tid":138,"created_ino":"0x1000000027a"},{"tid":139,"created_ino":"0x1000000027b"},{"tid":140,"created_ino":"0x1000000027c"},{"tid":141,"created_ino":"0x1000000027d"},{"tid":142,"created_ino":"0x1000000027e"},{"tid":143,"created_ino":"0x1000000027f"},{"tid":144,"created_ino":"0x10000000280"},{"tid":145,"created_ino":"0x10000000281"},{"tid":146,"created_ino":"0x10000000282"},{"tid":147,"created_ino":"0x10000000283"},{"tid":148,"created_ino":"0x10000000284"},{"tid":149,"created_ino":"0x10000000285"},{"tid":150,"created_ino":"0x10000000286"},{"tid":151,"created_ino":"0x10000000287"},{"tid":152,"created_ino":"0x10000000288"},{"tid":153,"created_ino":"0x10000000289"},{"tid":154,"created_ino":"0x1000000028a"},{"tid":155,"created_ino":"0x1000000028b"},{"tid":156,"created_ino":"0x1000000028c"},{"tid":157,"created_ino":"0x1000000028d"},{"tid":158,"created_ino":"0x1000000028e"},{"tid":159,"created_ino":"0x1000000028f"},{"tid":160,"created_ino":"0x10000000290"},{"tid":161,"created_ino":"0x10000000291"},{"tid":162,"created_ino":"0x10000000292"},{"tid":163,"created_ino":"0x10000000293"},{"tid":164,"created_ino":"0x10000000294"},{"tid":165,"created_ino":"0x10000000295"},{"tid":166,"created_ino":"0x10000000296"},{"tid":167,"created_ino":"0x10000000297"},{"tid":168,"created_ino":"0x10000000298"},{"tid":169,"created_ino":"0x10000000299"},{"tid":170,"created_ino":"0x1000000029a"},{"tid":171,"created_ino":"0x1000000029b"},{"tid":172,"created_ino":"0x1000000029c"},{"tid":173,"created_ino":"0x1000000029d"},{"tid":174,"created_ino":"0x1000000029e"},{"tid":175,"created_ino":"0x1000000029f"},{"tid":176,"created_ino":"0x100000002a0"},{"tid":177,"created_ino":"0x100000002a1"},{"tid":178,"created_ino":"0x100000002a2"},{"tid":179,"created_ino":"0x100000002a3"},{"tid":180,"created_ino":"0x100000002a4"},{"tid":181,"created_ino":"0x100000002a5"},{"tid":182,"created_ino":"0x100000002a6"},{"tid":183,"created_ino":"0x100000002a7"},{"tid":184,"created_ino":"0x100000002a8"},{"tid":185,"created_ino":"0x100000002a9"},{"tid":186,"created_ino":"0x100000002aa"},{"tid":187,"created_ino":"0x100000002ab"},{"tid":188,"created_ino":"0x100000002ac"},{"tid":189,"created_ino":"0x100000002ad"},{"tid":190,"created_ino":"0x100000002ae"},{"tid":191,"created_ino":"0x100000002af"},{"tid":192,"created_ino":"0x100000002b0"},{"tid":193,"created_ino":"0x100000002b1"},{"tid":194,"created_ino":"0x100000002b2"},{"tid":195,"created_ino":"0x100000002b3"},{"tid":196,"created_ino":"0x100000002b4"},{"tid":197,"created_ino":"0x100000002b5"},{"tid":198,"created_ino":"0x100000002b6"},{"tid":199,"created_ino":"0x100000002b7"},{"tid":200,"created_ino":"0x100000002b8"},{"tid":201,"created_ino":"0x100000002b9"},{"tid":202,"created_ino":"0x100000002ba"},{"tid":203,"created_ino":"0x100000002bb"},{"tid":204,"created_ino":"0x100000002bc"},{"tid":205,"created_ino":"0x100000002bd"},{"tid":206,"created_ino":"0x100000002be"},{"tid":207,"created_ino":"0x100000002bf"},{"tid":208,"created_ino":"0x100000002c0"},{"tid":209,"created_ino":"0x100000002c1"},{"tid":210,"created_ino":"0x100000002c2"},{"tid":211,"created_ino":"0x100000002c3"},{"tid":212,"created_ino":"0x100000002c4"},{"tid":213,"created_ino":"0x100000002c5"},{"tid":214,"created_ino":"0x100000002c6"},{"tid":215,"created_ino":"0x100000002c7"},{"tid":216,"created_ino":"0x100000002c8"},{"tid":217,"created_ino":"0x100000002c9"},{"tid":218,"created_ino":"0x100000002ca"},{"tid":219,"created_ino":"0x100000002cb"},{"tid":220,"created_ino":"0x100000002cc"},{"tid":221,"created_ino":"0x100000002cd"},{"tid":222,"created_ino":"0x100000002ce"},{"tid":223,"created_ino":"0x100000002cf"},{"tid":224,"created_ino":"0x100000002d0"},{"tid":225,"created_ino":"0x100000002d1"},{"tid":226,"created_ino":"0x100000002d2"},{"tid":227,"created_ino":"0x100000002d3"},{"tid":228,"created_ino":"0x100000002d4"},{"tid":229,"created_ino":"0x100000002d5"},{"tid":230,"created_ino":"0x100000002d6"},{"tid":231,"created_ino":"0x100000002d7"},{"tid":232,"created_ino":"0x100000002d8"},{"tid":233,"created_ino":"0x100000002d9"},{"tid":234,"created_ino":"0x100000002da"},{"tid":235,"created_ino":"0x100000002db"},{"tid":236,"created_ino":"0x100000002dc"},{"tid":237,"created_ino":"0x100000002dd"},{"tid":238,"created_ino":"0x100000002de"},{"tid":239,"created_ino":"0x100000002df"},{"tid":240,"created_ino":"0x100000002e0"},{"tid":241,"created_ino":"0x100000002e1"},{"tid":242,"created_ino":"0x100000002e2"},{"tid":243,"created_ino":"0x100000002e3"},{"tid":244,"created_ino":"0x100000002e4"},{"tid":245,"created_ino":"0x100000002e5"},{"tid":246,"created_ino":"0x100000002e6"},{"tid":247,"created_ino":"0x100000002e7"},{"tid":248,"created_ino":"0x100000002e8"},{"tid":249,"created_ino":"0x100000002e9"},{"tid":250,"created_ino":"0x100000002ea"},{"tid":251,"created_ino":"0x100000002eb"},{"tid":252,"created_ino":"0x100000002ec"},{"tid":254,"created_ino":"0x100000002ed"},{"tid":255,"created_ino":"0x100000002ee"},{"tid":256,"created_ino":"0x100000002ef"},{"tid":257,"created_ino":"0x100000002f0"},{"tid":258,"created_ino":"0x100000002f1"},{"tid":259,"created_ino":"0x100000002f2"},{"tid":260,"created_ino":"0x100000002f3"},{"tid":261,"created_ino":"0x100000002f4"},{"tid":262,"created_ino":"0x100000002f5"},{"tid":263,"created_ino":"0x100000002f6"},{"tid":264,"created_ino":"0x100000002f7"},{"tid":265,"created_ino":"0x100000002f8"},{"tid":266,"created_ino":"0x100000002f9"},{"tid":267,"created_ino":"0x100000002fa"},{"tid":268,"created_ino":"0x100000002fb"},{"tid":269,"created_ino":"0x100000002fc"},{"tid":270,"created_ino":"0x100000002fd"},{"tid":271,"created_ino":"0x100000002fe"},{"tid":272,"created_ino":"0x100000002ff"},{"tid":273,"created_ino":"0x10000000300"},{"tid":274,"created_ino":"0x10000000301"},{"tid":275,"created_ino":"0x10000000302"},{"tid":276,"created_ino":"0x10000000303"},{"tid":277,"created_ino":"0x10000000304"},{"tid":278,"created_ino":"0x10000000305"},{"tid":279,"created_ino":"0x10000000306"},{"tid":280,"created_ino":"0x10000000307"},{"tid":281,"created_ino":"0x10000000308"},{"tid":282,"created_ino":"0x10000000309"},{"tid":283,"created_ino":"0x1000000030a"},{"tid":284,"created_ino":"0x1000000030b"},{"tid":285,"created_ino":"0x1000000030c"},{"tid":286,"created_ino":"0x1000000030d"},{"tid":287,"created_ino":"0x1000000030e"},{"tid":288,"created_ino":"0x1000000030f"},{"tid":289,"created_ino":"0x10000000310"},{"tid":290,"created_ino":"0x10000000311"},{"tid":291,"created_ino":"0x10000000312"},{"tid":292,"created_ino":"0x10000000313"},{"tid":293,"created_ino":"0x10000000314"},{"tid":294,"created_ino":"0x10000000315"},{"tid":295,"created_ino":"0x10000000316"},{"tid":296,"created_ino":"0x10000000317"},{"tid":297,"created_ino":"0x10000000318"},{"tid":298,"created_ino":"0x10000000319"},{"tid":299,"created_ino":"0x1000000031a"},{"tid":300,"created_ino":"0x1000000031b"},{"tid":301,"created_ino":"0x1000000031c"},{"tid":302,"created_ino":"0x1000000031d"},{"tid":303,"created_ino":"0x1000000031e"},{"tid":304,"created_ino":"0x1000000031f"},{"tid":305,"created_ino":"0x10000000320"},{"tid":306,"created_ino":"0x10000000321"},{"tid":307,"created_ino":"0x10000000322"},{"tid":308,"created_ino":"0x10000000323"},{"tid":309,"created_ino":"0x10000000324"},{"tid":310,"created_ino":"0x10000000325"},{"tid":311,"created_ino":"0x10000000326"},{"tid":312,"created_ino":"0x10000000327"},{"tid":313,"created_ino":"0x10000000328"},{"tid":314,"created_ino":"0x10000000329"},{"tid":315,"created_ino":"0x1000000032a"},{"tid":316,"created_ino":"0x1000000032b"},{"tid":317,"created_ino":"0x1000000032c"},{"tid":318,"created_ino":"0x1000000032d"},{"tid":319,"created_ino":"0x1000000032e"},{"tid":320,"created_ino":"0x1000000032f"},{"tid":321,"created_ino":"0x10000000330"},{"tid":322,"created_ino":"0x10000000331"},{"tid":323,"created_ino":"0x10000000332"},{"tid":324,"created_ino":"0x10000000333"},{"tid":325,"created_ino":"0x10000000334"},{"tid":326,"created_ino":"0x10000000335"},{"tid":327,"created_ino":"0x10000000336"},{"tid":328,"created_ino":"0x10000000337"},{"tid":329,"created_ino":"0x10000000338"},{"tid":330,"created_ino":"0x10000000339"},{"tid":331,"created_ino":"0x1000000033a"},{"tid":332,"created_ino":"0x1000000033b"},{"tid":333,"created_ino":"0x1000000033c"},{"tid":334,"created_ino":"0x1000000033d"},{"tid":335,"created_ino":"0x1000000033e"},{"tid":336,"created_ino":"0x1000000033f"},{"tid":337,"created_ino":"0x10000000340"},{"tid":338,"created_ino":"0x10000000341"},{"tid":339,"created_ino":"0x10000000342"},{"tid":340,"created_ino":"0x10000000343"},{"tid":341,"created_ino":"0x10000000344"},{"tid":342,"created_ino":"0x10000000345"},{"tid":343,"created_ino":"0x10000000346"},{"tid":344,"created_ino":"0x10000000347"},{"tid":345,"created_ino":"0x10000000348"},{"tid":346,"created_ino":"0x10000000349"},{"tid":347,"created_ino":"0x1000000034a"},{"tid":348,"created_ino":"0x1000000034b"},{"tid":349,"created_ino":"0x1000000034c"},{"tid":350,"created_ino":"0x1000000034d"},{"tid":351,"created_ino":"0x1000000034e"},{"tid":352,"created_ino":"0x1000000034f"},{"tid":353,"created_ino":"0x10000000350"},{"tid":354,"created_ino":"0x10000000351"},{"tid":355,"created_ino":"0x10000000352"},{"tid":356,"created_ino":"0x10000000353"},{"tid":357,"created_ino":"0x10000000354"},{"tid":358,"created_ino":"0x10000000355"},{"tid":359,"created_ino":"0x10000000356"},{"tid":360,"created_ino":"0x10000000357"},{"tid":361,"created_ino":"0x10000000358"},{"tid":362,"created_ino":"0x10000000359"},{"tid":363,"created_ino":"0x1000000035a"},{"tid":364,"created_ino":"0x1000000035b"},{"tid":365,"created_ino":"0x1000000035c"},{"tid":366,"created_ino":"0x1000000035d"},{"tid":367,"created_ino":"0x1000000035e"},{"tid":368,"created_ino":"0x1000000035f"},{"tid":369,"created_ino":"0x10000000360"},{"tid":370,"created_ino":"0x10000000361"},{"tid":371,"created_ino":"0x10000000362"},{"tid":372,"created_ino":"0x10000000363"},{"tid":373,"created_ino":"0x10000000364"},{"tid":374,"created_ino":"0x10000000365"},{"tid":375,"created_ino":"0x10000000366"},{"tid":376,"created_ino":"0x10000000367"},{"tid":377,"created_ino":"0x10000000368"},{"tid":378,"created_ino":"0x10000000369"},{"tid":379,"created_ino":"0x1000000036a"},{"tid":380,"created_ino":"0x1000000036b"},{"tid":381,"created_ino":"0x1000000036c"},{"tid":382,"created_ino":"0x1000000036d"},{"tid":383,"created_ino":"0x1000000036e"},{"tid":384,"created_ino":"0x1000000036f"},{"tid":385,"created_ino":"0x10000000370"},{"tid":386,"created_ino":"0x10000000371"},{"tid":387,"created_ino":"0x10000000372"},{"tid":388,"created_ino":"0x10000000373"},{"tid":389,"created_ino":"0x10000000374"},{"tid":390,"created_ino":"0x10000000375"},{"tid":391,"created_ino":"0x10000000376"},{"tid":392,"created_ino":"0x10000000377"},{"tid":393,"created_ino":"0x10000000378"},{"tid":394,"created_ino":"0x10000000379"},{"tid":395,"created_ino":"0x1000000037a"},{"tid":396,"created_ino":"0x1000000037b"},{"tid":397,"created_ino":"0x1000000037c"},{"tid":398,"created_ino":"0x1000000037d"},{"tid":399,"created_ino":"0x1000000037e"},{"tid":400,"created_ino":"0x1000000037f"},{"tid":401,"created_ino":"0x10000000380"},{"tid":402,"created_ino":"0x10000000381"},{"tid":403,"created_ino":"0x10000000382"},{"tid":404,"created_ino":"0x10000000383"},{"tid":405,"created_ino":"0x10000000384"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000385","length":602}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"143292","root":"/"}}] 2023-07-06T03:49:55.814 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "123196", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 671928949, "type": "any" }, "name": { "num": 5933, "type": "client" } }, "id": 5933, "inst": "client.5933 192.168.0.1:0/671928949", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.946776458293615 }, "state": "open", "uptime": 10.518825764 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "143292", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 5 }, { "created_ino": "0x100000001f6", "tid": 6 }, { "created_ino": "0x100000001f7", "tid": 7 }, { "created_ino": "0x100000001f8", "tid": 8 }, { "created_ino": "0x100000001f9", "tid": 9 }, { "created_ino": "0x100000001fa", "tid": 10 }, { "created_ino": "0x100000001fb", "tid": 11 }, { "created_ino": "0x100000001fc", "tid": 12 }, { "created_ino": "0x100000001fd", "tid": 13 }, { "created_ino": "0x100000001fe", "tid": 14 }, { "created_ino": "0x100000001ff", "tid": 15 }, { "created_ino": "0x10000000200", "tid": 16 }, { "created_ino": "0x10000000201", "tid": 17 }, { "created_ino": "0x10000000202", "tid": 18 }, { "created_ino": "0x10000000203", "tid": 19 }, { "created_ino": "0x10000000204", "tid": 20 }, { "created_ino": "0x10000000205", "tid": 21 }, { "created_ino": "0x10000000206", "tid": 22 }, { "created_ino": "0x10000000207", "tid": 23 }, { "created_ino": "0x10000000208", "tid": 24 }, { "created_ino": "0x10000000209", "tid": 25 }, { "created_ino": "0x1000000020a", "tid": 26 }, { "created_ino": "0x1000000020b", "tid": 27 }, { "created_ino": "0x1000000020c", "tid": 28 }, { "created_ino": "0x1000000020d", "tid": 29 }, { "created_ino": "0x1000000020e", "tid": 30 }, { "created_ino": "0x1000000020f", "tid": 31 }, { "created_ino": "0x10000000210", "tid": 32 }, { "created_ino": "0x10000000211", "tid": 33 }, { "created_ino": "0x10000000212", "tid": 34 }, { "created_ino": "0x10000000213", "tid": 35 }, { "created_ino": "0x10000000214", "tid": 36 }, { "created_ino": "0x10000000215", "tid": 37 }, { "created_ino": "0x10000000216", "tid": 38 }, { "created_ino": "0x10000000217", "tid": 39 }, { "created_ino": "0x10000000218", "tid": 40 }, { "created_ino": "0x10000000219", "tid": 41 }, { "created_ino": "0x1000000021a", "tid": 42 }, { "created_ino": "0x1000000021b", "tid": 43 }, { "created_ino": "0x1000000021c", "tid": 44 }, { "created_ino": "0x1000000021d", "tid": 45 }, { "created_ino": "0x1000000021e", "tid": 46 }, { "created_ino": "0x1000000021f", "tid": 47 }, { "created_ino": "0x10000000220", "tid": 48 }, { "created_ino": "0x10000000221", "tid": 49 }, { "created_ino": "0x10000000222", "tid": 50 }, { "created_ino": "0x10000000223", "tid": 51 }, { "created_ino": "0x10000000224", "tid": 52 }, { "created_ino": "0x10000000225", "tid": 53 }, { "created_ino": "0x10000000226", "tid": 54 }, { "created_ino": "0x10000000227", "tid": 55 }, { "created_ino": "0x10000000228", "tid": 56 }, { "created_ino": "0x10000000229", "tid": 57 }, { "created_ino": "0x1000000022a", "tid": 58 }, { "created_ino": "0x1000000022b", "tid": 59 }, { "created_ino": "0x1000000022c", "tid": 60 }, { "created_ino": "0x1000000022d", "tid": 61 }, { "created_ino": "0x1000000022e", "tid": 62 }, { "created_ino": "0x1000000022f", "tid": 63 }, { "created_ino": "0x10000000230", "tid": 64 }, { "created_ino": "0x10000000231", "tid": 65 }, { "created_ino": "0x10000000232", "tid": 66 }, { "created_ino": "0x10000000233", "tid": 67 }, { "created_ino": "0x10000000234", "tid": 68 }, { "created_ino": "0x10000000235", "tid": 69 }, { "created_ino": "0x10000000236", "tid": 70 }, { "created_ino": "0x10000000237", "tid": 71 }, { "created_ino": "0x10000000238", "tid": 72 }, { "created_ino": "0x10000000239", "tid": 73 }, { "created_ino": "0x1000000023a", "tid": 74 }, { "created_ino": "0x1000000023b", "tid": 75 }, { "created_ino": "0x1000000023c", "tid": 76 }, { "created_ino": "0x1000000023d", "tid": 77 }, { "created_ino": "0x1000000023e", "tid": 78 }, { "created_ino": "0x1000000023f", "tid": 79 }, { "created_ino": "0x10000000240", "tid": 80 }, { "created_ino": "0x10000000241", "tid": 81 }, { "created_ino": "0x10000000242", "tid": 82 }, { "created_ino": "0x10000000243", "tid": 83 }, { "created_ino": "0x10000000244", "tid": 84 }, { "created_ino": "0x10000000245", "tid": 85 }, { "created_ino": "0x10000000246", "tid": 86 }, { "created_ino": "0x10000000247", "tid": 87 }, { "created_ino": "0x10000000248", "tid": 88 }, { "created_ino": "0x10000000249", "tid": 89 }, { "created_ino": "0x1000000024a", "tid": 90 }, { "created_ino": "0x1000000024b", "tid": 91 }, { "created_ino": "0x1000000024c", "tid": 92 }, { "created_ino": "0x1000000024d", "tid": 93 }, { "created_ino": "0x1000000024e", "tid": 94 }, { "created_ino": "0x1000000024f", "tid": 95 }, { "created_ino": "0x10000000250", "tid": 96 }, { "created_ino": "0x10000000251", "tid": 97 }, { "created_ino": "0x10000000252", "tid": 98 }, { "created_ino": "0x10000000253", "tid": 99 }, { "created_ino": "0x10000000254", "tid": 100 }, { "created_ino": "0x10000000255", "tid": 101 }, { "created_ino": "0x10000000256", "tid": 102 }, { "created_ino": "0x10000000257", "tid": 103 }, { "created_ino": "0x10000000258", "tid": 104 }, { "created_ino": "0x10000000259", "tid": 105 }, { "created_ino": "0x1000000025a", "tid": 106 }, { "created_ino": "0x1000000025b", "tid": 107 }, { "created_ino": "0x1000000025c", "tid": 108 }, { "created_ino": "0x1000000025d", "tid": 109 }, { "created_ino": "0x1000000025e", "tid": 110 }, { "created_ino": "0x1000000025f", "tid": 111 }, { "created_ino": "0x10000000260", "tid": 112 }, { "created_ino": "0x10000000261", "tid": 113 }, { "created_ino": "0x10000000262", "tid": 114 }, { "created_ino": "0x10000000263", "tid": 115 }, { "created_ino": "0x10000000264", "tid": 116 }, { "created_ino": "0x10000000265", "tid": 117 }, { "created_ino": "0x10000000266", "tid": 118 }, { "created_ino": "0x10000000267", "tid": 119 }, { "created_ino": "0x10000000268", "tid": 120 }, { "created_ino": "0x10000000269", "tid": 121 }, { "created_ino": "0x1000000026a", "tid": 122 }, { "created_ino": "0x1000000026b", "tid": 123 }, { "created_ino": "0x1000000026c", "tid": 124 }, { "created_ino": "0x1000000026d", "tid": 125 }, { "created_ino": "0x1000000026e", "tid": 126 }, { "created_ino": "0x1000000026f", "tid": 127 }, { "created_ino": "0x10000000270", "tid": 128 }, { "created_ino": "0x10000000271", "tid": 129 }, { "created_ino": "0x10000000272", "tid": 130 }, { "created_ino": "0x10000000273", "tid": 131 }, { "created_ino": "0x10000000274", "tid": 132 }, { "created_ino": "0x10000000275", "tid": 133 }, { "created_ino": "0x10000000276", "tid": 134 }, { "created_ino": "0x10000000277", "tid": 135 }, { "created_ino": "0x10000000278", "tid": 136 }, { "created_ino": "0x10000000279", "tid": 137 }, { "created_ino": "0x1000000027a", "tid": 138 }, { "created_ino": "0x1000000027b", "tid": 139 }, { "created_ino": "0x1000000027c", "tid": 140 }, { "created_ino": "0x1000000027d", "tid": 141 }, { "created_ino": "0x1000000027e", "tid": 142 }, { "created_ino": "0x1000000027f", "tid": 143 }, { "created_ino": "0x10000000280", "tid": 144 }, { "created_ino": "0x10000000281", "tid": 145 }, { "created_ino": "0x10000000282", "tid": 146 }, { "created_ino": "0x10000000283", "tid": 147 }, { "created_ino": "0x10000000284", "tid": 148 }, { "created_ino": "0x10000000285", "tid": 149 }, { "created_ino": "0x10000000286", "tid": 150 }, { "created_ino": "0x10000000287", "tid": 151 }, { "created_ino": "0x10000000288", "tid": 152 }, { "created_ino": "0x10000000289", "tid": 153 }, { "created_ino": "0x1000000028a", "tid": 154 }, { "created_ino": "0x1000000028b", "tid": 155 }, { "created_ino": "0x1000000028c", "tid": 156 }, { "created_ino": "0x1000000028d", "tid": 157 }, { "created_ino": "0x1000000028e", "tid": 158 }, { "created_ino": "0x1000000028f", "tid": 159 }, { "created_ino": "0x10000000290", "tid": 160 }, { "created_ino": "0x10000000291", "tid": 161 }, { "created_ino": "0x10000000292", "tid": 162 }, { "created_ino": "0x10000000293", "tid": 163 }, { "created_ino": "0x10000000294", "tid": 164 }, { "created_ino": "0x10000000295", "tid": 165 }, { "created_ino": "0x10000000296", "tid": 166 }, { "created_ino": "0x10000000297", "tid": 167 }, { "created_ino": "0x10000000298", "tid": 168 }, { "created_ino": "0x10000000299", "tid": 169 }, { "created_ino": "0x1000000029a", "tid": 170 }, { "created_ino": "0x1000000029b", "tid": 171 }, { "created_ino": "0x1000000029c", "tid": 172 }, { "created_ino": "0x1000000029d", "tid": 173 }, { "created_ino": "0x1000000029e", "tid": 174 }, { "created_ino": "0x1000000029f", "tid": 175 }, { "created_ino": "0x100000002a0", "tid": 176 }, { "created_ino": "0x100000002a1", "tid": 177 }, { "created_ino": "0x100000002a2", "tid": 178 }, { "created_ino": "0x100000002a3", "tid": 179 }, { "created_ino": "0x100000002a4", "tid": 180 }, { "created_ino": "0x100000002a5", "tid": 181 }, { "created_ino": "0x100000002a6", "tid": 182 }, { "created_ino": "0x100000002a7", "tid": 183 }, { "created_ino": "0x100000002a8", "tid": 184 }, { "created_ino": "0x100000002a9", "tid": 185 }, { "created_ino": "0x100000002aa", "tid": 186 }, { "created_ino": "0x100000002ab", "tid": 187 }, { "created_ino": "0x100000002ac", "tid": 188 }, { "created_ino": "0x100000002ad", "tid": 189 }, { "created_ino": "0x100000002ae", "tid": 190 }, { "created_ino": "0x100000002af", "tid": 191 }, { "created_ino": "0x100000002b0", "tid": 192 }, { "created_ino": "0x100000002b1", "tid": 193 }, { "created_ino": "0x100000002b2", "tid": 194 }, { "created_ino": "0x100000002b3", "tid": 195 }, { "created_ino": "0x100000002b4", "tid": 196 }, { "created_ino": "0x100000002b5", "tid": 197 }, { "created_ino": "0x100000002b6", "tid": 198 }, { "created_ino": "0x100000002b7", "tid": 199 }, { "created_ino": "0x100000002b8", "tid": 200 }, { "created_ino": "0x100000002b9", "tid": 201 }, { "created_ino": "0x100000002ba", "tid": 202 }, { "created_ino": "0x100000002bb", "tid": 203 }, { "created_ino": "0x100000002bc", "tid": 204 }, { "created_ino": "0x100000002bd", "tid": 205 }, { "created_ino": "0x100000002be", "tid": 206 }, { "created_ino": "0x100000002bf", "tid": 207 }, { "created_ino": "0x100000002c0", "tid": 208 }, { "created_ino": "0x100000002c1", "tid": 209 }, { "created_ino": "0x100000002c2", "tid": 210 }, { "created_ino": "0x100000002c3", "tid": 211 }, { "created_ino": "0x100000002c4", "tid": 212 }, { "created_ino": "0x100000002c5", "tid": 213 }, { "created_ino": "0x100000002c6", "tid": 214 }, { "created_ino": "0x100000002c7", "tid": 215 }, { "created_ino": "0x100000002c8", "tid": 216 }, { "created_ino": "0x100000002c9", "tid": 217 }, { "created_ino": "0x100000002ca", "tid": 218 }, { "created_ino": "0x100000002cb", "tid": 219 }, { "created_ino": "0x100000002cc", "tid": 220 }, { "created_ino": "0x100000002cd", "tid": 221 }, { "created_ino": "0x100000002ce", "tid": 222 }, { "created_ino": "0x100000002cf", "tid": 223 }, { "created_ino": "0x100000002d0", "tid": 224 }, { "created_ino": "0x100000002d1", "tid": 225 }, { "created_ino": "0x100000002d2", "tid": 226 }, { "created_ino": "0x100000002d3", "tid": 227 }, { "created_ino": "0x100000002d4", "tid": 228 }, { "created_ino": "0x100000002d5", "tid": 229 }, { "created_ino": "0x100000002d6", "tid": 230 }, { "created_ino": "0x100000002d7", "tid": 231 }, { "created_ino": "0x100000002d8", "tid": 232 }, { "created_ino": "0x100000002d9", "tid": 233 }, { "created_ino": "0x100000002da", "tid": 234 }, { "created_ino": "0x100000002db", "tid": 235 }, { "created_ino": "0x100000002dc", "tid": 236 }, { "created_ino": "0x100000002dd", "tid": 237 }, { "created_ino": "0x100000002de", "tid": 238 }, { "created_ino": "0x100000002df", "tid": 239 }, { "created_ino": "0x100000002e0", "tid": 240 }, { "created_ino": "0x100000002e1", "tid": 241 }, { "created_ino": "0x100000002e2", "tid": 242 }, { "created_ino": "0x100000002e3", "tid": 243 }, { "created_ino": "0x100000002e4", "tid": 244 }, { "created_ino": "0x100000002e5", "tid": 245 }, { "created_ino": "0x100000002e6", "tid": 246 }, { "created_ino": "0x100000002e7", "tid": 247 }, { "created_ino": "0x100000002e8", "tid": 248 }, { "created_ino": "0x100000002e9", "tid": 249 }, { "created_ino": "0x100000002ea", "tid": 250 }, { "created_ino": "0x100000002eb", "tid": 251 }, { "created_ino": "0x100000002ec", "tid": 252 }, { "created_ino": "0x100000002ed", "tid": 254 }, { "created_ino": "0x100000002ee", "tid": 255 }, { "created_ino": "0x100000002ef", "tid": 256 }, { "created_ino": "0x100000002f0", "tid": 257 }, { "created_ino": "0x100000002f1", "tid": 258 }, { "created_ino": "0x100000002f2", "tid": 259 }, { "created_ino": "0x100000002f3", "tid": 260 }, { "created_ino": "0x100000002f4", "tid": 261 }, { "created_ino": "0x100000002f5", "tid": 262 }, { "created_ino": "0x100000002f6", "tid": 263 }, { "created_ino": "0x100000002f7", "tid": 264 }, { "created_ino": "0x100000002f8", "tid": 265 }, { "created_ino": "0x100000002f9", "tid": 266 }, { "created_ino": "0x100000002fa", "tid": 267 }, { "created_ino": "0x100000002fb", "tid": 268 }, { "created_ino": "0x100000002fc", "tid": 269 }, { "created_ino": "0x100000002fd", "tid": 270 }, { "created_ino": "0x100000002fe", "tid": 271 }, { "created_ino": "0x100000002ff", "tid": 272 }, { "created_ino": "0x10000000300", "tid": 273 }, { "created_ino": "0x10000000301", "tid": 274 }, { "created_ino": "0x10000000302", "tid": 275 }, { "created_ino": "0x10000000303", "tid": 276 }, { "created_ino": "0x10000000304", "tid": 277 }, { "created_ino": "0x10000000305", "tid": 278 }, { "created_ino": "0x10000000306", "tid": 279 }, { "created_ino": "0x10000000307", "tid": 280 }, { "created_ino": "0x10000000308", "tid": 281 }, { "created_ino": "0x10000000309", "tid": 282 }, { "created_ino": "0x1000000030a", "tid": 283 }, { "created_ino": "0x1000000030b", "tid": 284 }, { "created_ino": "0x1000000030c", "tid": 285 }, { "created_ino": "0x1000000030d", "tid": 286 }, { "created_ino": "0x1000000030e", "tid": 287 }, { "created_ino": "0x1000000030f", "tid": 288 }, { "created_ino": "0x10000000310", "tid": 289 }, { "created_ino": "0x10000000311", "tid": 290 }, { "created_ino": "0x10000000312", "tid": 291 }, { "created_ino": "0x10000000313", "tid": 292 }, { "created_ino": "0x10000000314", "tid": 293 }, { "created_ino": "0x10000000315", "tid": 294 }, { "created_ino": "0x10000000316", "tid": 295 }, { "created_ino": "0x10000000317", "tid": 296 }, { "created_ino": "0x10000000318", "tid": 297 }, { "created_ino": "0x10000000319", "tid": 298 }, { "created_ino": "0x1000000031a", "tid": 299 }, { "created_ino": "0x1000000031b", "tid": 300 }, { "created_ino": "0x1000000031c", "tid": 301 }, { "created_ino": "0x1000000031d", "tid": 302 }, { "created_ino": "0x1000000031e", "tid": 303 }, { "created_ino": "0x1000000031f", "tid": 304 }, { "created_ino": "0x10000000320", "tid": 305 }, { "created_ino": "0x10000000321", "tid": 306 }, { "created_ino": "0x10000000322", "tid": 307 }, { "created_ino": "0x10000000323", "tid": 308 }, { "created_ino": "0x10000000324", "tid": 309 }, { "created_ino": "0x10000000325", "tid": 310 }, { "created_ino": "0x10000000326", "tid": 311 }, { "created_ino": "0x10000000327", "tid": 312 }, { "created_ino": "0x10000000328", "tid": 313 }, { "created_ino": "0x10000000329", "tid": 314 }, { "created_ino": "0x1000000032a", "tid": 315 }, { "created_ino": "0x1000000032b", "tid": 316 }, { "created_ino": "0x1000000032c", "tid": 317 }, { "created_ino": "0x1000000032d", "tid": 318 }, { "created_ino": "0x1000000032e", "tid": 319 }, { "created_ino": "0x1000000032f", "tid": 320 }, { "created_ino": "0x10000000330", "tid": 321 }, { "created_ino": "0x10000000331", "tid": 322 }, { "created_ino": "0x10000000332", "tid": 323 }, { "created_ino": "0x10000000333", "tid": 324 }, { "created_ino": "0x10000000334", "tid": 325 }, { "created_ino": "0x10000000335", "tid": 326 }, { "created_ino": "0x10000000336", "tid": 327 }, { "created_ino": "0x10000000337", "tid": 328 }, { "created_ino": "0x10000000338", "tid": 329 }, { "created_ino": "0x10000000339", "tid": 330 }, { "created_ino": "0x1000000033a", "tid": 331 }, { "created_ino": "0x1000000033b", "tid": 332 }, { "created_ino": "0x1000000033c", "tid": 333 }, { "created_ino": "0x1000000033d", "tid": 334 }, { "created_ino": "0x1000000033e", "tid": 335 }, { "created_ino": "0x1000000033f", "tid": 336 }, { "created_ino": "0x10000000340", "tid": 337 }, { "created_ino": "0x10000000341", "tid": 338 }, { "created_ino": "0x10000000342", "tid": 339 }, { "created_ino": "0x10000000343", "tid": 340 }, { "created_ino": "0x10000000344", "tid": 341 }, { "created_ino": "0x10000000345", "tid": 342 }, { "created_ino": "0x10000000346", "tid": 343 }, { "created_ino": "0x10000000347", "tid": 344 }, { "created_ino": "0x10000000348", "tid": 345 }, { "created_ino": "0x10000000349", "tid": 346 }, { "created_ino": "0x1000000034a", "tid": 347 }, { "created_ino": "0x1000000034b", "tid": 348 }, { "created_ino": "0x1000000034c", "tid": 349 }, { "created_ino": "0x1000000034d", "tid": 350 }, { "created_ino": "0x1000000034e", "tid": 351 }, { "created_ino": "0x1000000034f", "tid": 352 }, { "created_ino": "0x10000000350", "tid": 353 }, { "created_ino": "0x10000000351", "tid": 354 }, { "created_ino": "0x10000000352", "tid": 355 }, { "created_ino": "0x10000000353", "tid": 356 }, { "created_ino": "0x10000000354", "tid": 357 }, { "created_ino": "0x10000000355", "tid": 358 }, { "created_ino": "0x10000000356", "tid": 359 }, { "created_ino": "0x10000000357", "tid": 360 }, { "created_ino": "0x10000000358", "tid": 361 }, { "created_ino": "0x10000000359", "tid": 362 }, { "created_ino": "0x1000000035a", "tid": 363 }, { "created_ino": "0x1000000035b", "tid": 364 }, { "created_ino": "0x1000000035c", "tid": 365 }, { "created_ino": "0x1000000035d", "tid": 366 }, { "created_ino": "0x1000000035e", "tid": 367 }, { "created_ino": "0x1000000035f", "tid": 368 }, { "created_ino": "0x10000000360", "tid": 369 }, { "created_ino": "0x10000000361", "tid": 370 }, { "created_ino": "0x10000000362", "tid": 371 }, { "created_ino": "0x10000000363", "tid": 372 }, { "created_ino": "0x10000000364", "tid": 373 }, { "created_ino": "0x10000000365", "tid": 374 }, { "created_ino": "0x10000000366", "tid": 375 }, { "created_ino": "0x10000000367", "tid": 376 }, { "created_ino": "0x10000000368", "tid": 377 }, { "created_ino": "0x10000000369", "tid": 378 }, { "created_ino": "0x1000000036a", "tid": 379 }, { "created_ino": "0x1000000036b", "tid": 380 }, { "created_ino": "0x1000000036c", "tid": 381 }, { "created_ino": "0x1000000036d", "tid": 382 }, { "created_ino": "0x1000000036e", "tid": 383 }, { "created_ino": "0x1000000036f", "tid": 384 }, { "created_ino": "0x10000000370", "tid": 385 }, { "created_ino": "0x10000000371", "tid": 386 }, { "created_ino": "0x10000000372", "tid": 387 }, { "created_ino": "0x10000000373", "tid": 388 }, { "created_ino": "0x10000000374", "tid": 389 }, { "created_ino": "0x10000000375", "tid": 390 }, { "created_ino": "0x10000000376", "tid": 391 }, { "created_ino": "0x10000000377", "tid": 392 }, { "created_ino": "0x10000000378", "tid": 393 }, { "created_ino": "0x10000000379", "tid": 394 }, { "created_ino": "0x1000000037a", "tid": 395 }, { "created_ino": "0x1000000037b", "tid": 396 }, { "created_ino": "0x1000000037c", "tid": 397 }, { "created_ino": "0x1000000037d", "tid": 398 }, { "created_ino": "0x1000000037e", "tid": 399 }, { "created_ino": "0x1000000037f", "tid": 400 }, { "created_ino": "0x10000000380", "tid": 401 }, { "created_ino": "0x10000000381", "tid": 402 }, { "created_ino": "0x10000000382", "tid": 403 }, { "created_ino": "0x10000000383", "tid": 404 }, { "created_ino": "0x10000000384", "tid": 405 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 2398905682, "type": "any" }, "name": { "num": 7005, "type": "client" } }, "id": 7005, "inst": "client.7005 172.21.15.97:0/2398905682", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 402, "num_completed_flushes": 0, "num_completed_requests": 400, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 602, "start": "0x10000000385" } ], "recall_caps": { "halflife": 60, "value": 1103.9740709844307 }, "recall_caps_throttle": { "halflife": 1.5, "value": 324.8486695799127 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 72.59508432333865 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 377, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 794.8484007351717 }, "state": "open", "uptime": 12.815787748 } ] 2023-07-06T03:49:55.815 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-07-06T03:49:55.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:49:56.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.009+0000 7fae455ff700 1 Processor -- start 2023-07-06T03:49:56.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.009+0000 7fae455ff700 1 -- start start 2023-07-06T03:49:56.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.010+0000 7fae455ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae4015ca10 0x7fae40160e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.010+0000 7fae455ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fae400cf3c0 0x7fae400b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.010+0000 7fae455ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae400b4ee0 0x7fae400cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.010+0000 7fae455ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae400b5360 con 0x7fae400b4ee0 2023-07-06T03:49:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.010+0000 7fae455ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae400d1190 con 0x7fae400cf3c0 2023-07-06T03:49:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.010+0000 7fae455ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae40161340 con 0x7fae4015ca10 2023-07-06T03:49:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae400b4ee0 0x7fae400cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae400b4ee0 0x7fae400cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34438/0 (socket says 172.21.15.97:34438) 2023-07-06T03:49:56.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3f7fe700 1 -- 172.21.15.97:0/2475477546 learned_addr learned my addr 172.21.15.97:0/2475477546 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:56.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3effd700 1 --2- 172.21.15.97:0/2475477546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae4015ca10 0x7fae40160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3f7fe700 1 -- 172.21.15.97:0/2475477546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae4015ca10 msgr2=0x7fae40160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:56.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3e7fc700 1 --2- 172.21.15.97:0/2475477546 >> v2:172.21.15.146:3300/0 conn(0x7fae400cf3c0 0x7fae400b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3f7fe700 1 --2- 172.21.15.97:0/2475477546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae4015ca10 0x7fae40160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3f7fe700 1 -- 172.21.15.97:0/2475477546 >> v2:172.21.15.146:3300/0 conn(0x7fae400cf3c0 msgr2=0x7fae400b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:56.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3f7fe700 1 --2- 172.21.15.97:0/2475477546 >> v2:172.21.15.146:3300/0 conn(0x7fae400cf3c0 0x7fae400b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3f7fe700 1 -- 172.21.15.97:0/2475477546 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae400cd680 con 0x7fae400b4ee0 2023-07-06T03:49:56.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3e7fc700 1 --2- 172.21.15.97:0/2475477546 >> v2:172.21.15.146:3300/0 conn(0x7fae400cf3c0 0x7fae400b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:56.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3effd700 1 --2- 172.21.15.97:0/2475477546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae4015ca10 0x7fae40160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:49:56.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.011+0000 7fae3f7fe700 1 --2- 172.21.15.97:0/2475477546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae400b4ee0 0x7fae400cf930 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fae4008c100 tx=0x7fae3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaaa1879dfb43d84 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae3dffb700 1 -- 172.21.15.97:0/2475477546 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3800e070 con 0x7fae400b4ee0 2023-07-06T03:49:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae3dffb700 1 -- 172.21.15.97:0/2475477546 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3800b9c0 con 0x7fae400b4ee0 2023-07-06T03:49:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae3dffb700 1 -- 172.21.15.97:0/2475477546 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3800bd70 con 0x7fae400b4ee0 2023-07-06T03:49:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae455ff700 1 -- 172.21.15.97:0/2475477546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae400b4ee0 msgr2=0x7fae400cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae455ff700 1 --2- 172.21.15.97:0/2475477546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae400b4ee0 0x7fae400cf930 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fae4008c100 tx=0x7fae3800b330 comp rx=0 tx=0).stop 2023-07-06T03:49:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae455ff700 1 -- 172.21.15.97:0/2475477546 shutdown_connections 2023-07-06T03:49:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae455ff700 1 --2- 172.21.15.97:0/2475477546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae400b4ee0 0x7fae400cf930 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae455ff700 1 --2- 172.21.15.97:0/2475477546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae4015ca10 0x7fae40160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae455ff700 1 --2- 172.21.15.97:0/2475477546 >> v2:172.21.15.146:3300/0 conn(0x7fae400cf3c0 0x7fae400b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae455ff700 1 -- 172.21.15.97:0/2475477546 >> 172.21.15.97:0/2475477546 conn(0x7fae4006a920 msgr2=0x7fae4006ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae455ff700 1 -- 172.21.15.97:0/2475477546 shutdown_connections 2023-07-06T03:49:56.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.012+0000 7fae455ff700 1 -- 172.21.15.97:0/2475477546 wait complete. 2023-07-06T03:49:56.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.014+0000 7fae455ff700 1 Processor -- start 2023-07-06T03:49:56.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.014+0000 7fae455ff700 1 -- start start 2023-07-06T03:49:56.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.015+0000 7fae455ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae400b4ee0 0x7fae400b8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.015+0000 7fae455ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae400cf3c0 0x7fae4014a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.015+0000 7fae455ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae4015ca10 0x7fae400cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.015+0000 7fae455ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae400c6a50 con 0x7fae400b4ee0 2023-07-06T03:49:56.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.015+0000 7fae455ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae400c72f0 con 0x7fae400cf3c0 2023-07-06T03:49:56.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.015+0000 7fae455ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae400b8e40 con 0x7fae4015ca10 2023-07-06T03:49:56.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.015+0000 7fae3effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae400b4ee0 0x7fae400b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.015+0000 7fae3effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae400b4ee0 0x7fae400b8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34442/0 (socket says 172.21.15.97:34442) 2023-07-06T03:49:56.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.015+0000 7fae3effd700 1 -- 172.21.15.97:0/2621410841 learned_addr learned my addr 172.21.15.97:0/2621410841 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:56.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.015+0000 7fae3f7fe700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae4015ca10 0x7fae400cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.015+0000 7fae3e7fc700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae400cf3c0 0x7fae4014a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.016+0000 7fae3effd700 1 -- 172.21.15.97:0/2621410841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae4015ca10 msgr2=0x7fae400cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.016+0000 7fae3effd700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae4015ca10 0x7fae400cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.016+0000 7fae3effd700 1 -- 172.21.15.97:0/2621410841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae400cf3c0 msgr2=0x7fae4014a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.016+0000 7fae3effd700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae400cf3c0 0x7fae4014a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.016+0000 7fae3effd700 1 -- 172.21.15.97:0/2621410841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae38009b30 con 0x7fae400b4ee0 2023-07-06T03:49:56.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.016+0000 7fae3effd700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae400b4ee0 0x7fae400b8900 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c7f0 tx=0x7fae3000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:56.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.016+0000 7fae277fe700 1 -- 172.21.15.97:0/2621410841 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30011800 con 0x7fae400b4ee0 2023-07-06T03:49:56.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.016+0000 7fae277fe700 1 -- 172.21.15.97:0/2621410841 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae30011960 con 0x7fae400b4ee0 2023-07-06T03:49:56.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.016+0000 7fae455ff700 1 -- 172.21.15.97:0/2621410841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae400b6110 con 0x7fae400b4ee0 2023-07-06T03:49:56.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.016+0000 7fae277fe700 1 -- 172.21.15.97:0/2621410841 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30011d10 con 0x7fae400b4ee0 2023-07-06T03:49:56.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.016+0000 7fae455ff700 1 -- 172.21.15.97:0/2621410841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae400c6f80 con 0x7fae400b4ee0 2023-07-06T03:49:56.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.017+0000 7fae455ff700 1 -- 172.21.15.97:0/2621410841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4006a790 con 0x7fae400b4ee0 2023-07-06T03:49:56.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.018+0000 7fae277fe700 1 -- 172.21.15.97:0/2621410841 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fae30018360 con 0x7fae400b4ee0 2023-07-06T03:49:56.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.019+0000 7fae277fe700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fae2c06ace0 0x7fae2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.019+0000 7fae277fe700 1 -- 172.21.15.97:0/2621410841 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fae300893f0 con 0x7fae400b4ee0 2023-07-06T03:49:56.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.019+0000 7fae3e7fc700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fae2c06ace0 0x7fae2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.020+0000 7fae3e7fc700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fae2c06ace0 0x7fae2c06d190 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fae34006fd0 tx=0x7fae34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:56.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.024+0000 7fae277fe700 1 -- 172.21.15.97:0/2621410841 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fae30054490 con 0x7fae400b4ee0 2023-07-06T03:49:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.264+0000 7fae455ff700 1 -- 172.21.15.97:0/2621410841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fae4004e340 con 0x7fae400b4ee0 2023-07-06T03:49:56.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.264+0000 7fae277fe700 1 -- 172.21.15.97:0/2621410841 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fae30057ab0 con 0x7fae400b4ee0 2023-07-06T03:49:56.267 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:56.267 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:49:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.266+0000 7fae455ff700 1 -- 172.21.15.97:0/2621410841 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fae2c06ace0 msgr2=0x7fae2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.266+0000 7fae455ff700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fae2c06ace0 0x7fae2c06d190 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fae34006fd0 tx=0x7fae34008040 comp rx=0 tx=0).stop 2023-07-06T03:49:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.266+0000 7fae455ff700 1 -- 172.21.15.97:0/2621410841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae400b4ee0 msgr2=0x7fae400b8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.266+0000 7fae455ff700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae400b4ee0 0x7fae400b8900 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c7f0 tx=0x7fae3000cbb0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.266+0000 7fae455ff700 1 -- 172.21.15.97:0/2621410841 shutdown_connections 2023-07-06T03:49:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.266+0000 7fae455ff700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fae2c06ace0 0x7fae2c06d190 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.266+0000 7fae455ff700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae4015ca10 0x7fae400cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.266+0000 7fae455ff700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae400b4ee0 0x7fae400b8900 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.266+0000 7fae455ff700 1 --2- 172.21.15.97:0/2621410841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae400cf3c0 0x7fae4014a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.266+0000 7fae455ff700 1 -- 172.21.15.97:0/2621410841 >> 172.21.15.97:0/2621410841 conn(0x7fae4006a920 msgr2=0x7fae4015f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:56.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.266+0000 7fae455ff700 1 -- 172.21.15.97:0/2621410841 shutdown_connections 2023-07-06T03:49:56.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.267+0000 7fae455ff700 1 -- 172.21.15.97:0/2621410841 wait complete. 2023-07-06T03:49:56.286 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-06T03:49:56.287 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-07-06T03:49:56.287 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:49:56.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.478+0000 7f742b9ff700 1 Processor -- start 2023-07-06T03:49:56.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.478+0000 7f742b9ff700 1 -- start start 2023-07-06T03:49:56.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.479+0000 7f742b9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f742415c520 0x7f74240cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.479+0000 7f742b9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74240cdae0 con 0x7f7424160d00 2023-07-06T03:49:56.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.479+0000 7f742b9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74240b6640 con 0x7f7424160180 2023-07-06T03:49:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.479+0000 7f742b9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74240cd8d0 con 0x7f742415c520 2023-07-06T03:49:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.480+0000 7f7429f9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7424160d00 0x7f74240b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45460/0 (socket says 172.21.15.97:45460) 2023-07-06T03:49:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.480+0000 7f7429f9c700 1 -- 172.21.15.97:0/1727677052 learned_addr learned my addr 172.21.15.97:0/1727677052 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:49:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.480+0000 7f742979b700 1 --2- 172.21.15.97:0/1727677052 >> v2:172.21.15.146:3300/0 conn(0x7f742415c520 0x7f74240cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.480+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867045719 0 0) 0x7f74240cdae0 con 0x7f7424160d00 2023-07-06T03:49:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.480+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74100035b0 con 0x7f7424160d00 2023-07-06T03:49:56.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.480+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2145818279 0 0) 0x7f74100035b0 con 0x7f7424160d00 2023-07-06T03:49:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.480+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 >> v2:172.21.15.146:3300/0 conn(0x7f742415c520 msgr2=0x7f74240cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.480+0000 7f741bfff700 1 --2- 172.21.15.97:0/1727677052 >> v2:172.21.15.146:3300/0 conn(0x7f742415c520 0x7f74240cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.480+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 >> v1:172.21.15.146:6789/0 conn(0x7f7424160180 legacy=0x7f74241605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.480+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74240b6920 con 0x7f7424160d00 2023-07-06T03:49:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.480+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f741c0019f0 con 0x7f7424160d00 2023-07-06T03:49:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 >> v1:172.21.15.97:6789/0 conn(0x7f7424160d00 legacy=0x7f74240b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f741bfff700 1 --2- 172.21.15.97:0/1727677052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7410003a00 0x7f7410003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f741bfff700 1 --2- 172.21.15.97:0/1727677052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7410004850 0x7f7410004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f741bfff700 1 --2- 172.21.15.97:0/1727677052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7410005200 0x7f741000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74240cd8d0 con 0x7f7410005200 2023-07-06T03:49:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741000c080 con 0x7f7410004850 2023-07-06T03:49:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741000c240 con 0x7f7410003a00 2023-07-06T03:49:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f742979b700 1 --2- 172.21.15.97:0/1727677052 >> v2:172.21.15.146:3300/0 conn(0x7f742415c520 0x7f74240cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:49:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f7429f9c700 1 --2- 172.21.15.97:0/1727677052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7410005200 0x7f741000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f7428f9a700 1 --2- 172.21.15.97:0/1727677052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7410004850 0x7f7410004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f742979b700 1 --2- 172.21.15.97:0/1727677052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7410003a00 0x7f7410003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f7429f9c700 1 -- 172.21.15.97:0/1727677052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7410003a00 msgr2=0x7f7410003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f7429f9c700 1 --2- 172.21.15.97:0/1727677052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7410003a00 0x7f7410003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f7429f9c700 1 -- 172.21.15.97:0/1727677052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7410004850 msgr2=0x7f7410004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:49:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f7429f9c700 1 --2- 172.21.15.97:0/1727677052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7410004850 0x7f7410004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f7429f9c700 1 -- 172.21.15.97:0/1727677052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74100035b0 con 0x7f7410005200 2023-07-06T03:49:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.481+0000 7f7429f9c700 1 -- 172.21.15.97:0/1727677052 reap_dead start 2023-07-06T03:49:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.482+0000 7f7429f9c700 1 --2- 172.21.15.97:0/1727677052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7410005200 0x7f741000ba10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f741c00a830 tx=0x7f7410005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddcc4985a6ba056 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.482+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c00a2f0 con 0x7f7410005200 2023-07-06T03:49:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.482+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c012620 con 0x7f7410005200 2023-07-06T03:49:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.482+0000 7f741bfff700 1 -- 172.21.15.97:0/1727677052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c0129d0 con 0x7f7410005200 2023-07-06T03:49:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.482+0000 7f742b9ff700 1 -- 172.21.15.97:0/1727677052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7410005200 msgr2=0x7f741000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.482+0000 7f742b9ff700 1 --2- 172.21.15.97:0/1727677052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7410005200 0x7f741000ba10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f741c00a830 tx=0x7f7410005880 comp rx=0 tx=0).stop 2023-07-06T03:49:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.482+0000 7f742b9ff700 1 -- 172.21.15.97:0/1727677052 shutdown_connections 2023-07-06T03:49:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.482+0000 7f742b9ff700 1 --2- 172.21.15.97:0/1727677052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7410005200 0x7f741000ba10 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.482+0000 7f742b9ff700 1 -- 172.21.15.97:0/1727677052 >> 172.21.15.97:0/1727677052 conn(0x7f7424071790 msgr2=0x7f7424071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.482+0000 7f742b9ff700 1 -- 172.21.15.97:0/1727677052 shutdown_connections 2023-07-06T03:49:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.483+0000 7f742b9ff700 1 -- 172.21.15.97:0/1727677052 wait complete. 2023-07-06T03:49:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.484+0000 7f742b9ff700 1 Processor -- start 2023-07-06T03:49:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.484+0000 7f742b9ff700 1 -- start start 2023-07-06T03:49:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.485+0000 7f742b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7410005200 0x7f7424150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.485+0000 7f742b9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742414d210 0x7f74241535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.485+0000 7f742b9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f742414e570 0x7f74240b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.485+0000 7f742b9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74241540e0 con 0x7f742414d210 2023-07-06T03:49:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.485+0000 7f742b9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7424152860 con 0x7f742414e570 2023-07-06T03:49:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.485+0000 7f742b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7424152b00 con 0x7f7410005200 2023-07-06T03:49:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.485+0000 7f7428f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742414d210 0x7f74241535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.485+0000 7f7428f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742414d210 0x7f74241535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34468/0 (socket says 172.21.15.97:34468) 2023-07-06T03:49:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.485+0000 7f7428f9a700 1 -- 172.21.15.97:0/2129859778 learned_addr learned my addr 172.21.15.97:0/2129859778 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:49:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f742979b700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7410005200 0x7f7424150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f7429f9c700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f742414e570 0x7f74240b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f7428f9a700 1 -- 172.21.15.97:0/2129859778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7410005200 msgr2=0x7f7424150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f7428f9a700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7410005200 0x7f7424150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f7428f9a700 1 -- 172.21.15.97:0/2129859778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f742414e570 msgr2=0x7f74240b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f7428f9a700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f742414e570 0x7f74240b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f7428f9a700 1 -- 172.21.15.97:0/2129859778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7424152de0 con 0x7f742414d210 2023-07-06T03:49:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f7428f9a700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742414d210 0x7f74241535e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f741400f820 tx=0x7f741400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f7419ffb700 1 -- 172.21.15.97:0/2129859778 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74140052e0 con 0x7f742414d210 2023-07-06T03:49:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f7419ffb700 1 -- 172.21.15.97:0/2129859778 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7414005440 con 0x7f742414d210 2023-07-06T03:49:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f742b9ff700 1 -- 172.21.15.97:0/2129859778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74241530c0 con 0x7f742414d210 2023-07-06T03:49:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.486+0000 7f7419ffb700 1 -- 172.21.15.97:0/2129859778 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74140038a0 con 0x7f742414d210 2023-07-06T03:49:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.487+0000 7f742b9ff700 1 -- 172.21.15.97:0/2129859778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742414ece0 con 0x7f742414d210 2023-07-06T03:49:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.488+0000 7f742b9ff700 1 -- 172.21.15.97:0/2129859778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742404e340 con 0x7f742414d210 2023-07-06T03:49:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.489+0000 7f7419ffb700 1 -- 172.21.15.97:0/2129859778 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7414003a00 con 0x7f742414d210 2023-07-06T03:49:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.490+0000 7f7419ffb700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 0x7f740806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:49:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.490+0000 7f7419ffb700 1 -- 172.21.15.97:0/2129859778 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74140890c0 con 0x7f742414d210 2023-07-06T03:49:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.490+0000 7f742979b700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 0x7f740806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:49:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.490+0000 7f742979b700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 0x7f740806d190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7420009470 tx=0x7f7420008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:49:56.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.494+0000 7f7419ffb700 1 -- 172.21.15.97:0/2129859778 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7414054170 con 0x7f742414d210 2023-07-06T03:49:56.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.733+0000 7f742b9ff700 1 -- 172.21.15.97:0/2129859778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7424153430 con 0x7f742414d210 2023-07-06T03:49:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.734+0000 7f7419ffb700 1 -- 172.21.15.97:0/2129859778 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f7414003cb0 con 0x7f742414d210 2023-07-06T03:49:56.737 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:49:56.737 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:49:56.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.736+0000 7f742b9ff700 1 -- 172.21.15.97:0/2129859778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 msgr2=0x7f740806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.736+0000 7f742b9ff700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 0x7f740806d190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7420009470 tx=0x7f7420008040 comp rx=0 tx=0).stop 2023-07-06T03:49:56.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.736+0000 7f742b9ff700 1 -- 172.21.15.97:0/2129859778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742414d210 msgr2=0x7f74241535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:49:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.736+0000 7f742b9ff700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742414d210 0x7f74241535e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f741400f820 tx=0x7f741400fb30 comp rx=0 tx=0).stop 2023-07-06T03:49:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.736+0000 7f742b9ff700 1 -- 172.21.15.97:0/2129859778 shutdown_connections 2023-07-06T03:49:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.736+0000 7f742b9ff700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f740806ace0 0x7f740806d190 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.736+0000 7f742b9ff700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f742414e570 0x7f74240b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.736+0000 7f742b9ff700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7410005200 0x7f7424150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.736+0000 7f742b9ff700 1 --2- 172.21.15.97:0/2129859778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742414d210 0x7f74241535e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:49:56.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.736+0000 7f742b9ff700 1 -- 172.21.15.97:0/2129859778 >> 172.21.15.97:0/2129859778 conn(0x7f7424071790 msgr2=0x7f74240b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:49:56.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.737+0000 7f742b9ff700 1 -- 172.21.15.97:0/2129859778 shutdown_connections 2023-07-06T03:49:56.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:49:56.737+0000 7f742b9ff700 1 -- 172.21.15.97:0/2129859778 wait complete. 2023-07-06T03:49:56.757 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:49:56.757 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:50:01.759 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:50:01.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.963+0000 7f9b30dff700 1 Processor -- start 2023-07-06T03:50:01.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.964+0000 7f9b30dff700 1 -- start start 2023-07-06T03:50:01.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.965+0000 7f9b30dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9b2c15c520 0x7f9b2c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:01.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.965+0000 7f9b30dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9b2c160180 0x7f9b2c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:01.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.965+0000 7f9b30dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2c0cdae0 con 0x7f9b2c160b30 2023-07-06T03:50:01.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.965+0000 7f9b30dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c0cd8d0 con 0x7f9b2c15c520 2023-07-06T03:50:01.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.965+0000 7f9b30dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c0b6640 con 0x7f9b2c160180 2023-07-06T03:50:01.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.965+0000 7f9b2a59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9b2c15c520 0x7f9b2c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:01.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.965+0000 7f9b2a59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9b2c15c520 0x7f9b2c0cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58848/0 (socket says 172.21.15.97:58848) 2023-07-06T03:50:01.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.965+0000 7f9b2a59c700 1 -- 172.21.15.97:0/1720668342 learned_addr learned my addr 172.21.15.97:0/1720668342 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.965+0000 7f9b23fff700 1 --2- 172.21.15.97:0/1720668342 >> v2:172.21.15.146:3300/0 conn(0x7f9b2c160180 0x7f9b2c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.965+0000 7f9b2a59c700 1 -- 172.21.15.97:0/1720668342 >> v2:172.21.15.146:3300/0 conn(0x7f9b2c160180 msgr2=0x7f9b2c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.965+0000 7f9b2a59c700 1 --2- 172.21.15.97:0/1720668342 >> v2:172.21.15.146:3300/0 conn(0x7f9b2c160180 0x7f9b2c1605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:01.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.966+0000 7f9b2a59c700 1 -- 172.21.15.97:0/1720668342 >> v1:172.21.15.146:6789/0 conn(0x7f9b2c160b30 legacy=0x7f9b2c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:01.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.966+0000 7f9b2a59c700 1 -- 172.21.15.97:0/1720668342 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c0b6920 con 0x7f9b2c15c520 2023-07-06T03:50:01.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.966+0000 7f9b23fff700 1 --2- 172.21.15.97:0/1720668342 >> v2:172.21.15.146:3300/0 conn(0x7f9b2c160180 0x7f9b2c1605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:50:01.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.966+0000 7f9b2a59c700 1 --2- 172.21.15.97:0/1720668342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c15c520 0x7f9b2c0cd390 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0b5930 tx=0x7f9b1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20eb48ddea4d0d97 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:01.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.966+0000 7f9b29d9b700 1 -- 172.21.15.97:0/1720668342 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c004230 con 0x7f9b2c15c520 2023-07-06T03:50:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.966+0000 7f9b29d9b700 1 -- 172.21.15.97:0/1720668342 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c004390 con 0x7f9b2c15c520 2023-07-06T03:50:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.966+0000 7f9b29d9b700 1 -- 172.21.15.97:0/1720668342 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0036a0 con 0x7f9b2c15c520 2023-07-06T03:50:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.966+0000 7f9b30dff700 1 -- 172.21.15.97:0/1720668342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c15c520 msgr2=0x7f9b2c0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.967+0000 7f9b30dff700 1 --2- 172.21.15.97:0/1720668342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c15c520 0x7f9b2c0cd390 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0b5930 tx=0x7f9b1c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:50:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.967+0000 7f9b30dff700 1 -- 172.21.15.97:0/1720668342 shutdown_connections 2023-07-06T03:50:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.967+0000 7f9b30dff700 1 --2- 172.21.15.97:0/1720668342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c15c520 0x7f9b2c0cd390 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:01.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.967+0000 7f9b30dff700 1 --2- 172.21.15.97:0/1720668342 >> v2:172.21.15.146:3300/0 conn(0x7f9b2c160180 0x7f9b2c1605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:01.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.967+0000 7f9b30dff700 1 -- 172.21.15.97:0/1720668342 >> 172.21.15.97:0/1720668342 conn(0x7f9b2c071790 msgr2=0x7f9b2c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:01.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.967+0000 7f9b30dff700 1 -- 172.21.15.97:0/1720668342 shutdown_connections 2023-07-06T03:50:01.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.967+0000 7f9b30dff700 1 -- 172.21.15.97:0/1720668342 wait complete. 2023-07-06T03:50:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.969+0000 7f9b30dff700 1 Processor -- start 2023-07-06T03:50:01.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.969+0000 7f9b30dff700 1 -- start start 2023-07-06T03:50:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.970+0000 7f9b30dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c15c520 0x7f9b2c15da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.970+0000 7f9b30dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b2c160180 0x7f9b2c0d3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.970+0000 7f9b30dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c160b30 0x7f9b2c0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b30dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0d24a0 con 0x7f9b2c160b30 2023-07-06T03:50:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b30dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0d26d0 con 0x7f9b2c160180 2023-07-06T03:50:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b30dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0d29b0 con 0x7f9b2c15c520 2023-07-06T03:50:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b2ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c160b30 0x7f9b2c0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:01.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b2a59c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c15c520 0x7f9b2c15da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:01.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b2ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c160b30 0x7f9b2c0d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58850/0 (socket says 172.21.15.97:58850) 2023-07-06T03:50:01.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b2ad9d700 1 -- 172.21.15.97:0/889212198 learned_addr learned my addr 172.21.15.97:0/889212198 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:01.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b23fff700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b2c160180 0x7f9b2c0d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:01.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b2ad9d700 1 -- 172.21.15.97:0/889212198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c15c520 msgr2=0x7f9b2c15da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:01.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b2ad9d700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c15c520 0x7f9b2c15da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:01.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b2ad9d700 1 -- 172.21.15.97:0/889212198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b2c160180 msgr2=0x7f9b2c0d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:01.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b2ad9d700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b2c160180 0x7f9b2c0d3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:01.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b2ad9d700 1 -- 172.21.15.97:0/889212198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c009550 con 0x7f9b2c160b30 2023-07-06T03:50:01.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.971+0000 7f9b2ad9d700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c160b30 0x7f9b2c0d1f60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400cc10 tx=0x7f9b2400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:01.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.972+0000 7f9b22ffd700 1 -- 172.21.15.97:0/889212198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400aa60 con 0x7f9b2c160b30 2023-07-06T03:50:01.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.972+0000 7f9b22ffd700 1 -- 172.21.15.97:0/889212198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2400abc0 con 0x7f9b2c160b30 2023-07-06T03:50:01.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.972+0000 7f9b30dff700 1 -- 172.21.15.97:0/889212198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c0d2c90 con 0x7f9b2c160b30 2023-07-06T03:50:01.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.972+0000 7f9b22ffd700 1 -- 172.21.15.97:0/889212198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24012420 con 0x7f9b2c160b30 2023-07-06T03:50:01.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.972+0000 7f9b30dff700 1 -- 172.21.15.97:0/889212198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c0d3250 con 0x7f9b2c160b30 2023-07-06T03:50:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.973+0000 7f9b30dff700 1 -- 172.21.15.97:0/889212198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c0d3470 con 0x7f9b2c160b30 2023-07-06T03:50:01.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.974+0000 7f9b22ffd700 1 -- 172.21.15.97:0/889212198 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9b24012580 con 0x7f9b2c160b30 2023-07-06T03:50:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.975+0000 7f9b22ffd700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b0c06ace0 0x7f9b0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.975+0000 7f9b22ffd700 1 -- 172.21.15.97:0/889212198 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9b24089690 con 0x7f9b2c160b30 2023-07-06T03:50:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.975+0000 7f9b2a59c700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b0c06ace0 0x7f9b0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:01.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.976+0000 7f9b2a59c700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b0c06ace0 0x7f9b0c06d190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009130 tx=0x7f9b1c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:01.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:01.980+0000 7f9b22ffd700 1 -- 172.21.15.97:0/889212198 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9b240547d0 con 0x7f9b2c160b30 2023-07-06T03:50:02.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.213+0000 7f9b30dff700 1 -- 172.21.15.97:0/889212198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9b2c04e340 con 0x7f9b2c160b30 2023-07-06T03:50:02.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.214+0000 7f9b22ffd700 1 -- 172.21.15.97:0/889212198 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f9b24057df0 con 0x7f9b2c160b30 2023-07-06T03:50:02.217 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:50:02.217 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:50:02.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.216+0000 7f9b30dff700 1 -- 172.21.15.97:0/889212198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b0c06ace0 msgr2=0x7f9b0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:02.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.216+0000 7f9b30dff700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b0c06ace0 0x7f9b0c06d190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009130 tx=0x7f9b1c008030 comp rx=0 tx=0).stop 2023-07-06T03:50:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.216+0000 7f9b30dff700 1 -- 172.21.15.97:0/889212198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c160b30 msgr2=0x7f9b2c0d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.216+0000 7f9b30dff700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c160b30 0x7f9b2c0d1f60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400cc10 tx=0x7f9b2400cf20 comp rx=0 tx=0).stop 2023-07-06T03:50:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.216+0000 7f9b30dff700 1 -- 172.21.15.97:0/889212198 shutdown_connections 2023-07-06T03:50:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.216+0000 7f9b30dff700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9b0c06ace0 0x7f9b0c06d190 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.216+0000 7f9b30dff700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c160b30 0x7f9b2c0d1f60 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.216+0000 7f9b30dff700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b2c15c520 0x7f9b2c15da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.216+0000 7f9b30dff700 1 --2- 172.21.15.97:0/889212198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b2c160180 0x7f9b2c0d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:02.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.217+0000 7f9b30dff700 1 -- 172.21.15.97:0/889212198 >> 172.21.15.97:0/889212198 conn(0x7f9b2c071790 msgr2=0x7f9b2c072150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:02.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.217+0000 7f9b30dff700 1 -- 172.21.15.97:0/889212198 shutdown_connections 2023-07-06T03:50:02.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:02.217+0000 7f9b30dff700 1 -- 172.21.15.97:0/889212198 wait complete. 2023-07-06T03:50:02.237 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:50:02.237 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:50:04.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:50:04.366 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:50:07.239 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:50:07.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.440+0000 7fa8313ff700 1 Processor -- start 2023-07-06T03:50:07.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.441+0000 7fa8313ff700 1 -- start start 2023-07-06T03:50:07.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa8313ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa82c158f20 0x7fa82c0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:07.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa8313ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa82c0b0c70 0x7fa82c06bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:07.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa8313ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c15cc40 con 0x7fa82c1592f0 2023-07-06T03:50:07.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa8313ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c15cf20 con 0x7fa82c0b0c70 2023-07-06T03:50:07.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa8313ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c15d200 con 0x7fa82c158f20 2023-07-06T03:50:07.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa82b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa82c0b0c70 0x7fa82c06bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:07.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa82b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa82c0b0c70 0x7fa82c06bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58862/0 (socket says 172.21.15.97:58862) 2023-07-06T03:50:07.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa82b7fe700 1 -- 172.21.15.97:0/3310040910 learned_addr learned my addr 172.21.15.97:0/3310040910 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:07.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa82affd700 1 --2- 172.21.15.97:0/3310040910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa82c158f20 0x7fa82c0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:07.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa82b7fe700 1 -- 172.21.15.97:0/3310040910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa82c158f20 msgr2=0x7fa82c0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:07.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa82b7fe700 1 --2- 172.21.15.97:0/3310040910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa82c158f20 0x7fa82c0caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:07.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.442+0000 7fa82b7fe700 1 -- 172.21.15.97:0/3310040910 >> v1:172.21.15.146:6789/0 conn(0x7fa82c1592f0 legacy=0x7fa82c0b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:07.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.443+0000 7fa82b7fe700 1 -- 172.21.15.97:0/3310040910 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c15d4e0 con 0x7fa82c0b0c70 2023-07-06T03:50:07.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.443+0000 7fa82b7fe700 1 --2- 172.21.15.97:0/3310040910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c0b0c70 0x7fa82c06bd60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa824009e50 tx=0x7fa82400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5334f6b7279051d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:07.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.443+0000 7fa829ffb700 1 -- 172.21.15.97:0/3310040910 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400bbc0 con 0x7fa82c0b0c70 2023-07-06T03:50:07.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.443+0000 7fa829ffb700 1 -- 172.21.15.97:0/3310040910 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82400bd20 con 0x7fa82c0b0c70 2023-07-06T03:50:07.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.443+0000 7fa829ffb700 1 -- 172.21.15.97:0/3310040910 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8240116a0 con 0x7fa82c0b0c70 2023-07-06T03:50:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.443+0000 7fa8313ff700 1 -- 172.21.15.97:0/3310040910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c0b0c70 msgr2=0x7fa82c06bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.443+0000 7fa8313ff700 1 --2- 172.21.15.97:0/3310040910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c0b0c70 0x7fa82c06bd60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa824009e50 tx=0x7fa82400b330 comp rx=0 tx=0).stop 2023-07-06T03:50:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.443+0000 7fa8313ff700 1 -- 172.21.15.97:0/3310040910 shutdown_connections 2023-07-06T03:50:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.443+0000 7fa8313ff700 1 --2- 172.21.15.97:0/3310040910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c0b0c70 0x7fa82c06bd60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:07.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.443+0000 7fa8313ff700 1 --2- 172.21.15.97:0/3310040910 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa82c158f20 0x7fa82c0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:07.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.443+0000 7fa8313ff700 1 -- 172.21.15.97:0/3310040910 >> 172.21.15.97:0/3310040910 conn(0x7fa82c0ce8c0 msgr2=0x7fa82c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:07.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.444+0000 7fa8313ff700 1 -- 172.21.15.97:0/3310040910 shutdown_connections 2023-07-06T03:50:07.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.444+0000 7fa8313ff700 1 -- 172.21.15.97:0/3310040910 wait complete. 2023-07-06T03:50:07.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.445+0000 7fa8313ff700 1 Processor -- start 2023-07-06T03:50:07.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.445+0000 7fa8313ff700 1 -- start start 2023-07-06T03:50:07.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.446+0000 7fa8313ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa82c0b0c70 0x7fa82c147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:07.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.446+0000 7fa8313ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa82c158f20 0x7fa82c154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:07.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.446+0000 7fa8313ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c1592f0 0x7fa82c154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:07.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.446+0000 7fa8313ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0b4950 con 0x7fa82c1592f0 2023-07-06T03:50:07.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.446+0000 7fa8313ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c15c590 con 0x7fa82c158f20 2023-07-06T03:50:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.446+0000 7fa8313ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c148240 con 0x7fa82c0b0c70 2023-07-06T03:50:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.447+0000 7fa82b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c1592f0 0x7fa82c154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.447+0000 7fa82b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c1592f0 0x7fa82c154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58872/0 (socket says 172.21.15.97:58872) 2023-07-06T03:50:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.447+0000 7fa82b7fe700 1 -- 172.21.15.97:0/2699107113 learned_addr learned my addr 172.21.15.97:0/2699107113 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:07.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.447+0000 7fa82affd700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa82c0b0c70 0x7fa82c147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:07.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.447+0000 7fa82a7fc700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa82c158f20 0x7fa82c154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:07.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.447+0000 7fa82b7fe700 1 -- 172.21.15.97:0/2699107113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa82c0b0c70 msgr2=0x7fa82c147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:07.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.447+0000 7fa82b7fe700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa82c0b0c70 0x7fa82c147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:07.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.447+0000 7fa82b7fe700 1 -- 172.21.15.97:0/2699107113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa82c158f20 msgr2=0x7fa82c154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:07.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.447+0000 7fa82b7fe700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa82c158f20 0x7fa82c154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:07.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.447+0000 7fa82b7fe700 1 -- 172.21.15.97:0/2699107113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824009b30 con 0x7fa82c1592f0 2023-07-06T03:50:07.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.447+0000 7fa82b7fe700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c1592f0 0x7fa82c154840 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa824014000 tx=0x7fa824004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.448+0000 7fa8137fe700 1 -- 172.21.15.97:0/2699107113 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824020070 con 0x7fa82c1592f0 2023-07-06T03:50:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.448+0000 7fa8137fe700 1 -- 172.21.15.97:0/2699107113 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8240099b0 con 0x7fa82c1592f0 2023-07-06T03:50:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.448+0000 7fa8137fe700 1 -- 172.21.15.97:0/2699107113 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824011a90 con 0x7fa82c1592f0 2023-07-06T03:50:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.448+0000 7fa8313ff700 1 -- 172.21.15.97:0/2699107113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c154e70 con 0x7fa82c1592f0 2023-07-06T03:50:07.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.448+0000 7fa8313ff700 1 -- 172.21.15.97:0/2699107113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c155430 con 0x7fa82c1592f0 2023-07-06T03:50:07.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.449+0000 7fa8313ff700 1 -- 172.21.15.97:0/2699107113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c04e340 con 0x7fa82c1592f0 2023-07-06T03:50:07.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.450+0000 7fa8137fe700 1 -- 172.21.15.97:0/2699107113 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa824011bf0 con 0x7fa82c1592f0 2023-07-06T03:50:07.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.451+0000 7fa8137fe700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa81806ace0 0x7fa81806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:07.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.451+0000 7fa8137fe700 1 -- 172.21.15.97:0/2699107113 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa82408cc10 con 0x7fa82c1592f0 2023-07-06T03:50:07.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.452+0000 7fa82affd700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa81806ace0 0x7fa81806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:07.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.452+0000 7fa82affd700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa81806ace0 0x7fa81806d190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa82c148600 tx=0x7fa81c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:07.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.456+0000 7fa8137fe700 1 -- 172.21.15.97:0/2699107113 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa824057dd0 con 0x7fa82c1592f0 2023-07-06T03:50:07.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.690+0000 7fa8313ff700 1 -- 172.21.15.97:0/2699107113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa82c073360 con 0x7fa82c1592f0 2023-07-06T03:50:07.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.691+0000 7fa8137fe700 1 -- 172.21.15.97:0/2699107113 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fa8240150c0 con 0x7fa82c1592f0 2023-07-06T03:50:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:50:07.694 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:50:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.693+0000 7fa8313ff700 1 -- 172.21.15.97:0/2699107113 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa81806ace0 msgr2=0x7fa81806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.693+0000 7fa8313ff700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa81806ace0 0x7fa81806d190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa82c148600 tx=0x7fa81c008040 comp rx=0 tx=0).stop 2023-07-06T03:50:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.693+0000 7fa8313ff700 1 -- 172.21.15.97:0/2699107113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c1592f0 msgr2=0x7fa82c154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.693+0000 7fa8313ff700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c1592f0 0x7fa82c154840 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa824014000 tx=0x7fa824004ad0 comp rx=0 tx=0).stop 2023-07-06T03:50:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.693+0000 7fa8313ff700 1 -- 172.21.15.97:0/2699107113 shutdown_connections 2023-07-06T03:50:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.693+0000 7fa8313ff700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa81806ace0 0x7fa81806d190 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.693+0000 7fa8313ff700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c1592f0 0x7fa82c154840 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.693+0000 7fa8313ff700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa82c0b0c70 0x7fa82c147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.693+0000 7fa8313ff700 1 --2- 172.21.15.97:0/2699107113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa82c158f20 0x7fa82c154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.693+0000 7fa8313ff700 1 -- 172.21.15.97:0/2699107113 >> 172.21.15.97:0/2699107113 conn(0x7fa82c0ce8c0 msgr2=0x7fa82c1576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:07.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.694+0000 7fa8313ff700 1 -- 172.21.15.97:0/2699107113 shutdown_connections 2023-07-06T03:50:07.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:07.694+0000 7fa8313ff700 1 -- 172.21.15.97:0/2699107113 wait complete. 2023-07-06T03:50:07.713 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:50:07.714 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:50:12.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:50:12.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.918+0000 7f64a7fff700 1 Processor -- start 2023-07-06T03:50:12.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.918+0000 7f64a7fff700 1 -- start start 2023-07-06T03:50:12.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.919+0000 7f64a7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f64a00cd390 0x7f64a0160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:12.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.919+0000 7f64a7fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a0160dd0 0x7f64a00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.919+0000 7f64a7fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a00cdae0 con 0x7f64a015c520 2023-07-06T03:50:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.919+0000 7f64a7fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a00b6640 con 0x7f64a00cd390 2023-07-06T03:50:12.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.919+0000 7f64a7fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00b6920 con 0x7f64a0160dd0 2023-07-06T03:50:12.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.919+0000 7f64a5d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f64a015c520 0x7f64a0160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57810/0 (socket says 172.21.15.97:57810) 2023-07-06T03:50:12.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.919+0000 7f64a5d9b700 1 -- 172.21.15.97:0/1639747990 learned_addr learned my addr 172.21.15.97:0/1639747990 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:50:12.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.919+0000 7f64a559a700 1 --2- 172.21.15.97:0/1639747990 >> v2:172.21.15.146:3300/0 conn(0x7f64a00cd390 0x7f64a0160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:12.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.919+0000 7f64a659c700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a0160dd0 0x7f64a00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:12.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.919+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541227096 0 0) 0x7f64a00cdae0 con 0x7f64a015c520 2023-07-06T03:50:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.919+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f648c0035b0 con 0x7f64a015c520 2023-07-06T03:50:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1508848139 0 0) 0x7f648c0035b0 con 0x7f64a015c520 2023-07-06T03:50:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a0160dd0 msgr2=0x7f64a00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a0160dd0 0x7f64a00b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 >> v2:172.21.15.146:3300/0 conn(0x7f64a00cd390 msgr2=0x7f64a0160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 --2- 172.21.15.97:0/1639747990 >> v2:172.21.15.146:3300/0 conn(0x7f64a00cd390 0x7f64a0160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a00b6c00 con 0x7f64a015c520 2023-07-06T03:50:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f649c001670 con 0x7f64a015c520 2023-07-06T03:50:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 >> v1:172.21.15.97:6789/0 conn(0x7f64a015c520 legacy=0x7f64a0160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c003a00 0x7f648c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a0160dd0 0x7f648c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f648c004d90 0x7f648c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00b6640 con 0x7f648c003a00 2023-07-06T03:50:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00b6920 con 0x7f648c004d90 2023-07-06T03:50:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648c00ba10 con 0x7f64a0160dd0 2023-07-06T03:50:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a559a700 1 --2- 172.21.15.97:0/1639747990 >> v2:172.21.15.146:3300/0 conn(0x7f64a00cd390 0x7f64a0160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:50:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a5d9b700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c003a00 0x7f648c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a659c700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f648c004d90 0x7f648c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a559a700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a0160dd0 0x7f648c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.920+0000 7f64a5d9b700 1 -- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a0160dd0 msgr2=0x7f648c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.921+0000 7f64a5d9b700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a0160dd0 0x7f648c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.921+0000 7f64a5d9b700 1 -- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f648c004d90 msgr2=0x7f648c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.921+0000 7f64a5d9b700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f648c004d90 0x7f648c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.921+0000 7f64a5d9b700 1 -- 172.21.15.97:0/1639747990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c0035b0 con 0x7f648c003a00 2023-07-06T03:50:12.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.921+0000 7f64a659c700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f648c004d90 0x7f648c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:50:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.921+0000 7f64a5d9b700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c003a00 0x7f648c003e10 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f649c00c370 tx=0x7f648c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04642338832d31a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.921+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c007c60 con 0x7f648c003a00 2023-07-06T03:50:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.921+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c005960 con 0x7f648c003a00 2023-07-06T03:50:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.921+0000 7f64a4d99700 1 -- 172.21.15.97:0/1639747990 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c005d10 con 0x7f648c003a00 2023-07-06T03:50:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.922+0000 7f64a7fff700 1 -- 172.21.15.97:0/1639747990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c003a00 msgr2=0x7f648c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.922+0000 7f64a7fff700 1 --2- 172.21.15.97:0/1639747990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c003a00 0x7f648c003e10 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f649c00c370 tx=0x7f648c005880 comp rx=0 tx=0).stop 2023-07-06T03:50:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.922+0000 7f64a659c700 1 -- 172.21.15.97:0/1639747990 reap_dead start 2023-07-06T03:50:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.922+0000 7f64a7fff700 1 -- 172.21.15.97:0/1639747990 shutdown_connections 2023-07-06T03:50:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.922+0000 7f64a7fff700 1 -- 172.21.15.97:0/1639747990 >> 172.21.15.97:0/1639747990 conn(0x7f64a0071790 msgr2=0x7f64a0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.922+0000 7f64a7fff700 1 -- 172.21.15.97:0/1639747990 shutdown_connections 2023-07-06T03:50:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.922+0000 7f64a7fff700 1 -- 172.21.15.97:0/1639747990 wait complete. 2023-07-06T03:50:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.923+0000 7f64a7fff700 1 Processor -- start 2023-07-06T03:50:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.924+0000 7f64a7fff700 1 -- start start 2023-07-06T03:50:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.924+0000 7f64a7fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c003a00 0x7f64a014c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.924+0000 7f64a7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64a014cfe0 0x7f64a015dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.924+0000 7f64a7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a014ca20 0x7f64a015e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.924+0000 7f64a7fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00d2ff0 con 0x7f64a014ca20 2023-07-06T03:50:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.924+0000 7f64a7fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00d32d0 con 0x7f64a014cfe0 2023-07-06T03:50:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.924+0000 7f64a7fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00d35b0 con 0x7f648c003a00 2023-07-06T03:50:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a014ca20 0x7f64a015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a014ca20 0x7f64a015e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38986/0 (socket says 172.21.15.97:38986) 2023-07-06T03:50:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a659c700 1 -- 172.21.15.97:0/76863071 learned_addr learned my addr 172.21.15.97:0/76863071 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a5d9b700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c003a00 0x7f64a014c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a559a700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64a014cfe0 0x7f64a015dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a659c700 1 -- 172.21.15.97:0/76863071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c003a00 msgr2=0x7f64a014c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a659c700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c003a00 0x7f64a014c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a659c700 1 -- 172.21.15.97:0/76863071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64a014cfe0 msgr2=0x7f64a015dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a659c700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64a014cfe0 0x7f64a015dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a659c700 1 -- 172.21.15.97:0/76863071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a00d3890 con 0x7f64a014ca20 2023-07-06T03:50:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a5d9b700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c003a00 0x7f64a014c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:50:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.925+0000 7f64a659c700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a014ca20 0x7f64a015e150 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f64a00cee40 tx=0x7f64980104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.926+0000 7f64967fc700 1 -- 172.21.15.97:0/76863071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649800e050 con 0x7f64a014ca20 2023-07-06T03:50:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.926+0000 7f64967fc700 1 -- 172.21.15.97:0/76863071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649800f040 con 0x7f64a014ca20 2023-07-06T03:50:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.926+0000 7f64a7fff700 1 -- 172.21.15.97:0/76863071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a00d2c80 con 0x7f64a014ca20 2023-07-06T03:50:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.926+0000 7f64967fc700 1 -- 172.21.15.97:0/76863071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498010db0 con 0x7f64a014ca20 2023-07-06T03:50:12.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.926+0000 7f64a7fff700 1 -- 172.21.15.97:0/76863071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a00d0e20 con 0x7f64a014ca20 2023-07-06T03:50:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.927+0000 7f64a7fff700 1 -- 172.21.15.97:0/76863071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a00c9ea0 con 0x7f64a014ca20 2023-07-06T03:50:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.928+0000 7f64967fc700 1 -- 172.21.15.97:0/76863071 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6498015440 con 0x7f64a014ca20 2023-07-06T03:50:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.929+0000 7f64967fc700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648406ace0 0x7f648406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:12.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.929+0000 7f64967fc700 1 -- 172.21.15.97:0/76863071 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6498089a00 con 0x7f64a014ca20 2023-07-06T03:50:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.929+0000 7f64a5d9b700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648406ace0 0x7f648406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.929+0000 7f64a5d9b700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648406ace0 0x7f648406d190 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f649c015040 tx=0x7f649c00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:12.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:12.933+0000 7f64967fc700 1 -- 172.21.15.97:0/76863071 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f64a00c9ea0 con 0x7f64a014ca20 2023-07-06T03:50:13.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.171+0000 7f64a7fff700 1 -- 172.21.15.97:0/76863071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f64a00c9ea0 con 0x7f64a014ca20 2023-07-06T03:50:13.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.172+0000 7f64967fc700 1 -- 172.21.15.97:0/76863071 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f64a00c9ea0 con 0x7f64a014ca20 2023-07-06T03:50:13.175 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:50:13.175 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:50:13.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.174+0000 7f64a7fff700 1 -- 172.21.15.97:0/76863071 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648406ace0 msgr2=0x7f648406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:13.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.174+0000 7f64a7fff700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648406ace0 0x7f648406d190 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f649c015040 tx=0x7f649c00a000 comp rx=0 tx=0).stop 2023-07-06T03:50:13.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.174+0000 7f64a7fff700 1 -- 172.21.15.97:0/76863071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a014ca20 msgr2=0x7f64a015e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:13.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.174+0000 7f64a7fff700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a014ca20 0x7f64a015e150 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f64a00cee40 tx=0x7f64980104a0 comp rx=0 tx=0).stop 2023-07-06T03:50:13.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.174+0000 7f64a7fff700 1 -- 172.21.15.97:0/76863071 shutdown_connections 2023-07-06T03:50:13.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.174+0000 7f64a7fff700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648406ace0 0x7f648406d190 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:13.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.174+0000 7f64a7fff700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a014ca20 0x7f64a015e150 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:13.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.174+0000 7f64a7fff700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f648c003a00 0x7f64a014c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:13.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.174+0000 7f64a7fff700 1 --2- 172.21.15.97:0/76863071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64a014cfe0 0x7f64a015dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:13.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.174+0000 7f64a7fff700 1 -- 172.21.15.97:0/76863071 >> 172.21.15.97:0/76863071 conn(0x7f64a0071790 msgr2=0x7f64a00b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:13.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.174+0000 7f64a7fff700 1 -- 172.21.15.97:0/76863071 shutdown_connections 2023-07-06T03:50:13.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:13.175+0000 7f64a7fff700 1 -- 172.21.15.97:0/76863071 wait complete. 2023-07-06T03:50:13.195 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:50:13.195 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:50:18.197 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:50:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.399+0000 7f00137ff700 1 Processor -- start 2023-07-06T03:50:18.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.399+0000 7f00137ff700 1 -- start start 2023-07-06T03:50:18.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.400+0000 7f00137ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f000c15b440 0x7f000c15e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:18.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.400+0000 7f00137ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f000c070c30 0x7f000c0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:18.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.400+0000 7f00137ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000c071070 con 0x7f000c0cd8e0 2023-07-06T03:50:18.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.400+0000 7f00137ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000c0ae3e0 con 0x7f000c070c30 2023-07-06T03:50:18.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.400+0000 7f00137ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c071330 con 0x7f000c15b440 2023-07-06T03:50:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.401+0000 7f0011d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f000c070c30 0x7f000c0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.401+0000 7f0011d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f000c070c30 0x7f000c0adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39002/0 (socket says 172.21.15.97:39002) 2023-07-06T03:50:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.401+0000 7f0011d9c700 1 -- 172.21.15.97:0/62280817 learned_addr learned my addr 172.21.15.97:0/62280817 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.401+0000 7f0010d9a700 1 --2- 172.21.15.97:0/62280817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f000c15b440 0x7f000c15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.401+0000 7f0011d9c700 1 -- 172.21.15.97:0/62280817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f000c15b440 msgr2=0x7f000c15e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:18.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.401+0000 7f0011d9c700 1 --2- 172.21.15.97:0/62280817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f000c15b440 0x7f000c15e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.401+0000 7f0011d9c700 1 -- 172.21.15.97:0/62280817 >> v1:172.21.15.146:6789/0 conn(0x7f000c0cd8e0 legacy=0x7f000c0cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.401+0000 7f0011d9c700 1 -- 172.21.15.97:0/62280817 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c071610 con 0x7f000c070c30 2023-07-06T03:50:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.401+0000 7f0011d9c700 1 --2- 172.21.15.97:0/62280817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f000c070c30 0x7f000c0adea0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0004009e50 tx=0x7f000400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5d8f79a77a5bd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.402+0000 7f0003fff700 1 -- 172.21.15.97:0/62280817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000400e070 con 0x7f000c070c30 2023-07-06T03:50:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.402+0000 7f0003fff700 1 -- 172.21.15.97:0/62280817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000400b9c0 con 0x7f000c070c30 2023-07-06T03:50:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.402+0000 7f0003fff700 1 -- 172.21.15.97:0/62280817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000400bd70 con 0x7f000c070c30 2023-07-06T03:50:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.402+0000 7f00137ff700 1 -- 172.21.15.97:0/62280817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f000c070c30 msgr2=0x7f000c0adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:18.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.402+0000 7f00137ff700 1 --2- 172.21.15.97:0/62280817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f000c070c30 0x7f000c0adea0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0004009e50 tx=0x7f000400b330 comp rx=0 tx=0).stop 2023-07-06T03:50:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.402+0000 7f00137ff700 1 -- 172.21.15.97:0/62280817 shutdown_connections 2023-07-06T03:50:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.402+0000 7f00137ff700 1 --2- 172.21.15.97:0/62280817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f000c070c30 0x7f000c0adea0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.402+0000 7f00137ff700 1 --2- 172.21.15.97:0/62280817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f000c15b440 0x7f000c15e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.402+0000 7f00137ff700 1 -- 172.21.15.97:0/62280817 >> 172.21.15.97:0/62280817 conn(0x7f000c0ce8c0 msgr2=0x7f000c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.402+0000 7f00137ff700 1 -- 172.21.15.97:0/62280817 shutdown_connections 2023-07-06T03:50:18.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.402+0000 7f00137ff700 1 -- 172.21.15.97:0/62280817 wait complete. 2023-07-06T03:50:18.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.405+0000 7f00137ff700 1 Processor -- start 2023-07-06T03:50:18.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.405+0000 7f00137ff700 1 -- start start 2023-07-06T03:50:18.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.406+0000 7f00137ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f000c070c30 0x7f000c0ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:18.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.406+0000 7f00137ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f000c0cd8e0 0x7f000c158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:18.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.406+0000 7f00137ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f000c15b440 0x7f000c0c4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.406+0000 7f00137ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0a9ca0 con 0x7f000c070c30 2023-07-06T03:50:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.406+0000 7f00137ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0c4bc0 con 0x7f000c0cd8e0 2023-07-06T03:50:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.406+0000 7f00137ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0a9650 con 0x7f000c15b440 2023-07-06T03:50:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.406+0000 7f001159b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f000c070c30 0x7f000c0ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.406+0000 7f001159b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f000c070c30 0x7f000c0ab750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39008/0 (socket says 172.21.15.97:39008) 2023-07-06T03:50:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.406+0000 7f001159b700 1 -- 172.21.15.97:0/47153864 learned_addr learned my addr 172.21.15.97:0/47153864 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f0010d9a700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f000c0cd8e0 0x7f000c158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:18.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f0011d9c700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f000c15b440 0x7f000c0c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f001159b700 1 -- 172.21.15.97:0/47153864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f000c15b440 msgr2=0x7f000c0c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f001159b700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f000c15b440 0x7f000c0c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f001159b700 1 -- 172.21.15.97:0/47153864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f000c0cd8e0 msgr2=0x7f000c158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f001159b700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f000c0cd8e0 0x7f000c158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f001159b700 1 -- 172.21.15.97:0/47153864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0004009b30 con 0x7f000c070c30 2023-07-06T03:50:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f001159b700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f000c070c30 0x7f000c0ab750 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f000800c450 tx=0x7f000800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f0001ffb700 1 -- 172.21.15.97:0/47153864 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00080052d0 con 0x7f000c070c30 2023-07-06T03:50:18.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f0001ffb700 1 -- 172.21.15.97:0/47153864 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0008005430 con 0x7f000c070c30 2023-07-06T03:50:18.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f00137ff700 1 -- 172.21.15.97:0/47153864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000c0a9930 con 0x7f000c070c30 2023-07-06T03:50:18.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f0001ffb700 1 -- 172.21.15.97:0/47153864 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00080038c0 con 0x7f000c070c30 2023-07-06T03:50:18.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.407+0000 7f00137ff700 1 -- 172.21.15.97:0/47153864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c0c4970 con 0x7f000c070c30 2023-07-06T03:50:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.408+0000 7f00137ff700 1 -- 172.21.15.97:0/47153864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000c070aa0 con 0x7f000c070c30 2023-07-06T03:50:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.409+0000 7f0001ffb700 1 -- 172.21.15.97:0/47153864 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0008003a20 con 0x7f000c070c30 2023-07-06T03:50:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.410+0000 7f0001ffb700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efff006ace0 0x7efff006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.410+0000 7f0001ffb700 1 -- 172.21.15.97:0/47153864 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00080886f0 con 0x7f000c070c30 2023-07-06T03:50:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.411+0000 7f0010d9a700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efff006ace0 0x7efff006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.411+0000 7f0010d9a700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efff006ace0 0x7efff006d190 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f000c0ce740 tx=0x7efffc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.415+0000 7f0001ffb700 1 -- 172.21.15.97:0/47153864 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0008053830 con 0x7f000c070c30 2023-07-06T03:50:18.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.654+0000 7f00137ff700 1 -- 172.21.15.97:0/47153864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f000c0a94b0 con 0x7f000c070c30 2023-07-06T03:50:18.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.654+0000 7f0001ffb700 1 -- 172.21.15.97:0/47153864 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f000c0a94b0 con 0x7f000c070c30 2023-07-06T03:50:18.656 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:50:18.657 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:50:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 -- 172.21.15.97:0/47153864 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efff006ace0 msgr2=0x7efff006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efff006ace0 0x7efff006d190 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f000c0ce740 tx=0x7efffc008040 comp rx=0 tx=0).stop 2023-07-06T03:50:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 -- 172.21.15.97:0/47153864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f000c070c30 msgr2=0x7f000c0ab750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f000c070c30 0x7f000c0ab750 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f000800c450 tx=0x7f000800c810 comp rx=0 tx=0).stop 2023-07-06T03:50:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 -- 172.21.15.97:0/47153864 shutdown_connections 2023-07-06T03:50:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efff006ace0 0x7efff006d190 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f000c15b440 0x7f000c0c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f000c070c30 0x7f000c0ab750 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 --2- 172.21.15.97:0/47153864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f000c0cd8e0 0x7f000c158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 -- 172.21.15.97:0/47153864 >> 172.21.15.97:0/47153864 conn(0x7f000c0ce8c0 msgr2=0x7f000c15b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 -- 172.21.15.97:0/47153864 shutdown_connections 2023-07-06T03:50:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:18.657+0000 7f00137ff700 1 -- 172.21.15.97:0/47153864 wait complete. 2023-07-06T03:50:18.676 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:50:18.677 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:50:23.678 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:50:23.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.881+0000 7f5cca1ff700 1 Processor -- start 2023-07-06T03:50:23.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.881+0000 7f5cca1ff700 1 -- start start 2023-07-06T03:50:23.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.883+0000 7f5cca1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5cc4158960 0x7f5cc406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:23.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.883+0000 7f5cca1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc415ce90 0x7f5cc40712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:23.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.883+0000 7f5cca1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc4159d40 con 0x7f5cc4071830 2023-07-06T03:50:23.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.883+0000 7f5cca1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc406ddb0 con 0x7f5cc4158960 2023-07-06T03:50:23.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.883+0000 7f5cca1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc406bbf0 con 0x7f5cc415ce90 2023-07-06T03:50:23.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.883+0000 7f5cc3fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5cc4071830 0x7f5cc40cc400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54282/0 (socket says 172.21.15.97:54282) 2023-07-06T03:50:23.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.883+0000 7f5cc3fff700 1 -- 172.21.15.97:0/3986185050 learned_addr learned my addr 172.21.15.97:0/3986185050 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:50:23.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.883+0000 7f5cc37fe700 1 --2- 172.21.15.97:0/3986185050 >> v2:172.21.15.146:3300/0 conn(0x7f5cc4158960 0x7f5cc406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:23.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.883+0000 7f5cc2ffd700 1 --2- 172.21.15.97:0/3986185050 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc415ce90 0x7f5cc40712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.883+0000 7f5cc27fc700 1 -- 172.21.15.97:0/3986185050 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331338578 0 0) 0x7f5cbc001490 con 0x7f5cc4071830 2023-07-06T03:50:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.884+0000 7f5cc27fc700 1 -- 172.21.15.97:0/3986185050 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cac0035b0 con 0x7f5cc4071830 2023-07-06T03:50:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.884+0000 7f5cc2ffd700 1 -- 172.21.15.97:0/3986185050 >> v2:172.21.15.146:3300/0 conn(0x7f5cc4158960 msgr2=0x7f5cc406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:23.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.884+0000 7f5cc2ffd700 1 --2- 172.21.15.97:0/3986185050 >> v2:172.21.15.146:3300/0 conn(0x7f5cc4158960 0x7f5cc406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:23.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.884+0000 7f5cc2ffd700 1 -- 172.21.15.97:0/3986185050 >> v1:172.21.15.97:6789/0 conn(0x7f5cc4071830 legacy=0x7f5cc40cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:23.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.884+0000 7f5cc2ffd700 1 -- 172.21.15.97:0/3986185050 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc406bed0 con 0x7f5cc415ce90 2023-07-06T03:50:23.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.884+0000 7f5cc37fe700 1 --2- 172.21.15.97:0/3986185050 >> v2:172.21.15.146:3300/0 conn(0x7f5cc4158960 0x7f5cc406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:50:23.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.884+0000 7f5cc2ffd700 1 --2- 172.21.15.97:0/3986185050 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc415ce90 0x7f5cc40712f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009840 tx=0x7f5cb8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bc353f03a6f71b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:23.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.884+0000 7f5cc27fc700 1 -- 172.21.15.97:0/3986185050 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8011830 con 0x7f5cc415ce90 2023-07-06T03:50:23.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.884+0000 7f5cc27fc700 1 -- 172.21.15.97:0/3986185050 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb8011990 con 0x7f5cc415ce90 2023-07-06T03:50:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.884+0000 7f5cc27fc700 1 -- 172.21.15.97:0/3986185050 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8011d00 con 0x7f5cc415ce90 2023-07-06T03:50:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.885+0000 7f5cca1ff700 1 -- 172.21.15.97:0/3986185050 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc415ce90 msgr2=0x7f5cc40712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.885+0000 7f5cca1ff700 1 --2- 172.21.15.97:0/3986185050 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc415ce90 0x7f5cc40712f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009840 tx=0x7f5cb8009b50 comp rx=0 tx=0).stop 2023-07-06T03:50:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.885+0000 7f5cca1ff700 1 -- 172.21.15.97:0/3986185050 shutdown_connections 2023-07-06T03:50:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.885+0000 7f5cca1ff700 1 --2- 172.21.15.97:0/3986185050 >> v2:172.21.15.146:3300/0 conn(0x7f5cc4158960 0x7f5cc406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.885+0000 7f5cca1ff700 1 --2- 172.21.15.97:0/3986185050 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc415ce90 0x7f5cc40712f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.885+0000 7f5cca1ff700 1 -- 172.21.15.97:0/3986185050 >> 172.21.15.97:0/3986185050 conn(0x7f5cc40ce8c0 msgr2=0x7f5cc40cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.885+0000 7f5cca1ff700 1 -- 172.21.15.97:0/3986185050 shutdown_connections 2023-07-06T03:50:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.885+0000 7f5cca1ff700 1 -- 172.21.15.97:0/3986185050 wait complete. 2023-07-06T03:50:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.887+0000 7f5cca1ff700 1 Processor -- start 2023-07-06T03:50:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.887+0000 7f5cca1ff700 1 -- start start 2023-07-06T03:50:23.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.889+0000 7f5cca1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cc4071830 0x7f5cc414dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:23.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.889+0000 7f5cca1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc4158960 0x7f5cc40b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:23.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.889+0000 7f5cca1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc415ce90 0x7f5cc4155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:23.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.889+0000 7f5cca1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40aecc0 con 0x7f5cc415ce90 2023-07-06T03:50:23.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.889+0000 7f5cca1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc41569d0 con 0x7f5cc4071830 2023-07-06T03:50:23.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.889+0000 7f5cca1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40ae670 con 0x7f5cc4158960 2023-07-06T03:50:23.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.889+0000 7f5cc3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc415ce90 0x7f5cc4155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:23.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.889+0000 7f5cc3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc415ce90 0x7f5cc4155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58726/0 (socket says 172.21.15.97:58726) 2023-07-06T03:50:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.889+0000 7f5cc3fff700 1 -- 172.21.15.97:0/686510616 learned_addr learned my addr 172.21.15.97:0/686510616 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.889+0000 7f5cc2ffd700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc4158960 0x7f5cc40b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.889+0000 7f5cc37fe700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cc4071830 0x7f5cc414dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.890+0000 7f5cc3fff700 1 -- 172.21.15.97:0/686510616 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc4158960 msgr2=0x7f5cc40b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.890+0000 7f5cc3fff700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc4158960 0x7f5cc40b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.890+0000 7f5cc3fff700 1 -- 172.21.15.97:0/686510616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cc4071830 msgr2=0x7f5cc414dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.890+0000 7f5cc3fff700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cc4071830 0x7f5cc414dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:23.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.890+0000 7f5cc3fff700 1 -- 172.21.15.97:0/686510616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb400a640 con 0x7f5cc415ce90 2023-07-06T03:50:23.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.890+0000 7f5cc3fff700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc415ce90 0x7f5cc4155f50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00cb40 tx=0x7f5cbc00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:23.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.890+0000 7f5cabfff700 1 -- 172.21.15.97:0/686510616 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc00a9e0 con 0x7f5cc415ce90 2023-07-06T03:50:23.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.890+0000 7f5cabfff700 1 -- 172.21.15.97:0/686510616 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc00ab40 con 0x7f5cc415ce90 2023-07-06T03:50:23.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.890+0000 7f5cca1ff700 1 -- 172.21.15.97:0/686510616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb80094a0 con 0x7f5cc415ce90 2023-07-06T03:50:23.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.890+0000 7f5cabfff700 1 -- 172.21.15.97:0/686510616 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc012400 con 0x7f5cc415ce90 2023-07-06T03:50:23.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.890+0000 7f5cca1ff700 1 -- 172.21.15.97:0/686510616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc4156cb0 con 0x7f5cc415ce90 2023-07-06T03:50:23.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.891+0000 7f5cca1ff700 1 -- 172.21.15.97:0/686510616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc4067830 con 0x7f5cc415ce90 2023-07-06T03:50:23.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.892+0000 7f5cabfff700 1 -- 172.21.15.97:0/686510616 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5cbc01b3b0 con 0x7f5cc415ce90 2023-07-06T03:50:23.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.894+0000 7f5cabfff700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cb006ace0 0x7f5cb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:23.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.894+0000 7f5cabfff700 1 -- 172.21.15.97:0/686510616 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5cbc088e40 con 0x7f5cc415ce90 2023-07-06T03:50:23.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.894+0000 7f5cc37fe700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cb006ace0 0x7f5cb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:23.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.894+0000 7f5cc37fe700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cb006ace0 0x7f5cb006d190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400a610 tx=0x7f5cb400a320 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:23.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:23.899+0000 7f5cabfff700 1 -- 172.21.15.97:0/686510616 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5cbc054030 con 0x7f5cc415ce90 2023-07-06T03:50:24.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.137+0000 7f5cca1ff700 1 -- 172.21.15.97:0/686510616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5cc4159d40 con 0x7f5cc415ce90 2023-07-06T03:50:24.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.137+0000 7f5cabfff700 1 -- 172.21.15.97:0/686510616 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f5cc4159d40 con 0x7f5cc415ce90 2023-07-06T03:50:24.140 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:50:24.140 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:50:24.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.139+0000 7f5cca1ff700 1 -- 172.21.15.97:0/686510616 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cb006ace0 msgr2=0x7f5cb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:24.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.139+0000 7f5cca1ff700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cb006ace0 0x7f5cb006d190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400a610 tx=0x7f5cb400a320 comp rx=0 tx=0).stop 2023-07-06T03:50:24.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.139+0000 7f5cca1ff700 1 -- 172.21.15.97:0/686510616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc415ce90 msgr2=0x7f5cc4155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:24.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.139+0000 7f5cca1ff700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc415ce90 0x7f5cc4155f50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00cb40 tx=0x7f5cbc00ce50 comp rx=0 tx=0).stop 2023-07-06T03:50:24.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.140+0000 7f5cca1ff700 1 -- 172.21.15.97:0/686510616 shutdown_connections 2023-07-06T03:50:24.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.140+0000 7f5cca1ff700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cb006ace0 0x7f5cb006d190 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:24.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.140+0000 7f5cca1ff700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cc415ce90 0x7f5cc4155f50 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:24.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.140+0000 7f5cca1ff700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cc4071830 0x7f5cc414dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:24.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.140+0000 7f5cca1ff700 1 --2- 172.21.15.97:0/686510616 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cc4158960 0x7f5cc40b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:24.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.140+0000 7f5cca1ff700 1 -- 172.21.15.97:0/686510616 >> 172.21.15.97:0/686510616 conn(0x7f5cc40ce8c0 msgr2=0x7f5cc41533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:24.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.140+0000 7f5cca1ff700 1 -- 172.21.15.97:0/686510616 shutdown_connections 2023-07-06T03:50:24.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:24.140+0000 7f5cca1ff700 1 -- 172.21.15.97:0/686510616 wait complete. 2023-07-06T03:50:24.159 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:50:24.159 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:50:29.160 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:50:29.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.362+0000 7ff21dbff700 1 Processor -- start 2023-07-06T03:50:29.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.363+0000 7ff21dbff700 1 -- start start 2023-07-06T03:50:29.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.364+0000 7ff21dbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff2180b8410 0x7ff218167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:29.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.364+0000 7ff21dbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2180cdae0 con 0x7ff2180cd390 2023-07-06T03:50:29.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.364+0000 7ff21dbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2180b6640 con 0x7ff21815c520 2023-07-06T03:50:29.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.364+0000 7ff21dbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2180b6920 con 0x7ff2180b8410 2023-07-06T03:50:29.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.364+0000 7ff20ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff2180cd390 0x7ff218160890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54294/0 (socket says 172.21.15.97:54294) 2023-07-06T03:50:29.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.364+0000 7ff20ffff700 1 -- 172.21.15.97:0/484507613 learned_addr learned my addr 172.21.15.97:0/484507613 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:50:29.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.364+0000 7ff217fff700 1 --2- 172.21.15.97:0/484507613 >> v2:172.21.15.97:3300/0 conn(0x7ff2180b8410 0x7ff218167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:29.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.364+0000 7ff216ffd700 1 -- 172.21.15.97:0/484507613 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211627831 0 0) 0x7ff2180cdae0 con 0x7ff2180cd390 2023-07-06T03:50:29.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.365+0000 7ff216ffd700 1 -- 172.21.15.97:0/484507613 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2040035b0 con 0x7ff2180cd390 2023-07-06T03:50:29.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.365+0000 7ff217fff700 1 -- 172.21.15.97:0/484507613 >> v1:172.21.15.146:6789/0 conn(0x7ff21815c520 legacy=0x7ff218160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:29.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.365+0000 7ff217fff700 1 -- 172.21.15.97:0/484507613 >> v1:172.21.15.97:6789/0 conn(0x7ff2180cd390 legacy=0x7ff218160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:29.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.365+0000 7ff217fff700 1 -- 172.21.15.97:0/484507613 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2180b6c00 con 0x7ff2180b8410 2023-07-06T03:50:29.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.365+0000 7ff217fff700 1 --2- 172.21.15.97:0/484507613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180b8410 0x7ff218167510 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff210009d90 tx=0x7ff21000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5433ae95494854af server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:29.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.365+0000 7ff216ffd700 1 -- 172.21.15.97:0/484507613 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21000e070 con 0x7ff2180b8410 2023-07-06T03:50:29.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.365+0000 7ff216ffd700 1 -- 172.21.15.97:0/484507613 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21000bb20 con 0x7ff2180b8410 2023-07-06T03:50:29.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.365+0000 7ff216ffd700 1 -- 172.21.15.97:0/484507613 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210012430 con 0x7ff2180b8410 2023-07-06T03:50:29.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.366+0000 7ff21dbff700 1 -- 172.21.15.97:0/484507613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180b8410 msgr2=0x7ff218167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:29.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.366+0000 7ff21dbff700 1 --2- 172.21.15.97:0/484507613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180b8410 0x7ff218167510 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff210009d90 tx=0x7ff21000b330 comp rx=0 tx=0).stop 2023-07-06T03:50:29.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.366+0000 7ff21dbff700 1 -- 172.21.15.97:0/484507613 shutdown_connections 2023-07-06T03:50:29.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.366+0000 7ff21dbff700 1 --2- 172.21.15.97:0/484507613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180b8410 0x7ff218167510 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:29.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.366+0000 7ff21dbff700 1 -- 172.21.15.97:0/484507613 >> 172.21.15.97:0/484507613 conn(0x7ff218071790 msgr2=0x7ff218071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:29.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.366+0000 7ff21dbff700 1 -- 172.21.15.97:0/484507613 shutdown_connections 2023-07-06T03:50:29.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.366+0000 7ff21dbff700 1 -- 172.21.15.97:0/484507613 wait complete. 2023-07-06T03:50:29.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.368+0000 7ff21dbff700 1 Processor -- start 2023-07-06T03:50:29.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.368+0000 7ff21dbff700 1 -- start start 2023-07-06T03:50:29.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff21dbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2180b8410 0x7ff2180cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:29.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff21dbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180cd390 0x7ff2180c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:29.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff21dbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff21815c520 0x7ff2180cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:29.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff21dbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180cab30 con 0x7ff2180cd390 2023-07-06T03:50:29.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff21dbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180cb170 con 0x7ff21815c520 2023-07-06T03:50:29.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff21dbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180cba10 con 0x7ff2180b8410 2023-07-06T03:50:29.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff20ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180cd390 0x7ff2180c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:29.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff20ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180cd390 0x7ff2180c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58740/0 (socket says 172.21.15.97:58740) 2023-07-06T03:50:29.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff20ffff700 1 -- 172.21.15.97:0/3206578638 learned_addr learned my addr 172.21.15.97:0/3206578638 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:29.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff217fff700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff21815c520 0x7ff2180cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:29.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff2177fe700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2180b8410 0x7ff2180cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:29.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff20ffff700 1 -- 172.21.15.97:0/3206578638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2180b8410 msgr2=0x7ff2180cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:29.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff20ffff700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2180b8410 0x7ff2180cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:29.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff20ffff700 1 -- 172.21.15.97:0/3206578638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff21815c520 msgr2=0x7ff2180cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:29.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff20ffff700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff21815c520 0x7ff2180cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:29.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.369+0000 7ff20ffff700 1 -- 172.21.15.97:0/3206578638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff210009a70 con 0x7ff2180cd390 2023-07-06T03:50:29.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.370+0000 7ff20ffff700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180cd390 0x7ff2180c8bd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff20000c700 tx=0x7ff20000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:29.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.370+0000 7ff214ff9700 1 -- 172.21.15.97:0/3206578638 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff200005d70 con 0x7ff2180cd390 2023-07-06T03:50:29.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.370+0000 7ff214ff9700 1 -- 172.21.15.97:0/3206578638 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2000070d0 con 0x7ff2180cd390 2023-07-06T03:50:29.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.370+0000 7ff214ff9700 1 -- 172.21.15.97:0/3206578638 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff200007460 con 0x7ff2180cd390 2023-07-06T03:50:29.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.370+0000 7ff21dbff700 1 -- 172.21.15.97:0/3206578638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2180d1820 con 0x7ff2180cd390 2023-07-06T03:50:29.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.370+0000 7ff21dbff700 1 -- 172.21.15.97:0/3206578638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2180cb6a0 con 0x7ff2180cd390 2023-07-06T03:50:29.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.371+0000 7ff21dbff700 1 -- 172.21.15.97:0/3206578638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2180cbb80 con 0x7ff2180cd390 2023-07-06T03:50:29.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.372+0000 7ff214ff9700 1 -- 172.21.15.97:0/3206578638 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff2000075c0 con 0x7ff2180cd390 2023-07-06T03:50:29.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.373+0000 7ff214ff9700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1f806ace0 0x7ff1f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:29.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.373+0000 7ff214ff9700 1 -- 172.21.15.97:0/3206578638 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff200088b10 con 0x7ff2180cd390 2023-07-06T03:50:29.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.373+0000 7ff2177fe700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1f806ace0 0x7ff1f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:29.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.373+0000 7ff2177fe700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1f806ace0 0x7ff1f806d190 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff21815ed10 tx=0x7ff208009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:29.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.377+0000 7ff214ff9700 1 -- 172.21.15.97:0/3206578638 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff200053b30 con 0x7ff2180cd390 2023-07-06T03:50:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.611+0000 7ff21dbff700 1 -- 172.21.15.97:0/3206578638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff218160f20 con 0x7ff2180cd390 2023-07-06T03:50:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.611+0000 7ff214ff9700 1 -- 172.21.15.97:0/3206578638 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7ff200057150 con 0x7ff2180cd390 2023-07-06T03:50:29.614 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:50:29.615 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:50:29.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 -- 172.21.15.97:0/3206578638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1f806ace0 msgr2=0x7ff1f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:29.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1f806ace0 0x7ff1f806d190 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff21815ed10 tx=0x7ff208009040 comp rx=0 tx=0).stop 2023-07-06T03:50:29.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 -- 172.21.15.97:0/3206578638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180cd390 msgr2=0x7ff2180c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180cd390 0x7ff2180c8bd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff20000c700 tx=0x7ff20000cac0 comp rx=0 tx=0).stop 2023-07-06T03:50:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 -- 172.21.15.97:0/3206578638 shutdown_connections 2023-07-06T03:50:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1f806ace0 0x7ff1f806d190 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff21815c520 0x7ff2180cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2180b8410 0x7ff2180cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 --2- 172.21.15.97:0/3206578638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180cd390 0x7ff2180c8bd0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 -- 172.21.15.97:0/3206578638 >> 172.21.15.97:0/3206578638 conn(0x7ff218071790 msgr2=0x7ff2180d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:29.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 -- 172.21.15.97:0/3206578638 shutdown_connections 2023-07-06T03:50:29.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:29.614+0000 7ff21dbff700 1 -- 172.21.15.97:0/3206578638 wait complete. 2023-07-06T03:50:29.633 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:50:29.634 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:50:34.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:50:34.554 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:50:34.634 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:50:34.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.828+0000 7faa6d9ff700 1 Processor -- start 2023-07-06T03:50:34.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.828+0000 7faa6d9ff700 1 -- start start 2023-07-06T03:50:34.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.829+0000 7faa6d9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa6815c520 0x7faa680cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:34.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.829+0000 7faa6d9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa680cdae0 con 0x7faa68160180 2023-07-06T03:50:34.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.829+0000 7faa6d9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa680b6640 con 0x7faa68160d00 2023-07-06T03:50:34.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.829+0000 7faa6d9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa680cd8d0 con 0x7faa6815c520 2023-07-06T03:50:34.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.829+0000 7faa66ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa6815c520 0x7faa680cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:34.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.829+0000 7faa66ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa6815c520 0x7faa680cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49040/0 (socket says 172.21.15.97:49040) 2023-07-06T03:50:34.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.829+0000 7faa66ffd700 1 -- 172.21.15.97:0/1329825807 learned_addr learned my addr 172.21.15.97:0/1329825807 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:34.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.829+0000 7faa667fc700 1 --1- 172.21.15.97:0/1329825807 >> v1:172.21.15.97:6789/0 conn(0x7faa68160180 0x7faa681605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33460/0 (socket says 172.21.15.97:33460) 2023-07-06T03:50:34.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.830+0000 7faa66ffd700 1 -- 172.21.15.97:0/1329825807 >> v1:172.21.15.146:6789/0 conn(0x7faa68160d00 legacy=0x7faa680b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:34.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.830+0000 7faa66ffd700 1 -- 172.21.15.97:0/1329825807 >> v1:172.21.15.97:6789/0 conn(0x7faa68160180 legacy=0x7faa681605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:34.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.830+0000 7faa66ffd700 1 -- 172.21.15.97:0/1329825807 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa680b6920 con 0x7faa6815c520 2023-07-06T03:50:34.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.830+0000 7faa65ffb700 1 -- 172.21.15.97:0/1329825807 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789389350 0 0) 0x7faa680cdae0 con 0x7faa68160180 2023-07-06T03:50:34.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.830+0000 7faa66ffd700 1 --2- 172.21.15.97:0/1329825807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6815c520 0x7faa680cd390 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7faa580098f0 tx=0x7faa58009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42940512fda18a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:34.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.830+0000 7faa65ffb700 1 -- 172.21.15.97:0/1329825807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58004230 con 0x7faa6815c520 2023-07-06T03:50:34.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.830+0000 7faa65ffb700 1 -- 172.21.15.97:0/1329825807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58004390 con 0x7faa6815c520 2023-07-06T03:50:34.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.830+0000 7faa65ffb700 1 -- 172.21.15.97:0/1329825807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa580036a0 con 0x7faa6815c520 2023-07-06T03:50:34.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.831+0000 7faa6d9ff700 1 -- 172.21.15.97:0/1329825807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6815c520 msgr2=0x7faa680cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:34.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.831+0000 7faa6d9ff700 1 --2- 172.21.15.97:0/1329825807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6815c520 0x7faa680cd390 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7faa580098f0 tx=0x7faa58009c00 comp rx=0 tx=0).stop 2023-07-06T03:50:34.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.831+0000 7faa6d9ff700 1 -- 172.21.15.97:0/1329825807 shutdown_connections 2023-07-06T03:50:34.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.831+0000 7faa6d9ff700 1 --2- 172.21.15.97:0/1329825807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6815c520 0x7faa680cd390 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:34.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.831+0000 7faa6d9ff700 1 -- 172.21.15.97:0/1329825807 >> 172.21.15.97:0/1329825807 conn(0x7faa68071790 msgr2=0x7faa68071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:34.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.831+0000 7faa6d9ff700 1 -- 172.21.15.97:0/1329825807 shutdown_connections 2023-07-06T03:50:34.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.831+0000 7faa6d9ff700 1 -- 172.21.15.97:0/1329825807 wait complete. 2023-07-06T03:50:34.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.833+0000 7faa6d9ff700 1 Processor -- start 2023-07-06T03:50:34.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.833+0000 7faa6d9ff700 1 -- start start 2023-07-06T03:50:34.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.834+0000 7faa6d9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6815c520 0x7faa680d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.834+0000 7faa6d9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa68160180 0x7faa68151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.834+0000 7faa6d9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa68160d00 0x7faa68153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.834+0000 7faa6d9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa68154030 con 0x7faa68160d00 2023-07-06T03:50:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.834+0000 7faa6d9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa68153760 con 0x7faa68160180 2023-07-06T03:50:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.834+0000 7faa6d9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa680d1b20 con 0x7faa6815c520 2023-07-06T03:50:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.834+0000 7faa677fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa68160d00 0x7faa68153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.834+0000 7faa677fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa68160d00 0x7faa68153af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49056/0 (socket says 172.21.15.97:49056) 2023-07-06T03:50:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.834+0000 7faa677fe700 1 -- 172.21.15.97:0/3829028978 learned_addr learned my addr 172.21.15.97:0/3829028978 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.834+0000 7faa667fc700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa68160180 0x7faa68151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.834+0000 7faa66ffd700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6815c520 0x7faa680d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.835+0000 7faa677fe700 1 -- 172.21.15.97:0/3829028978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6815c520 msgr2=0x7faa680d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.835+0000 7faa677fe700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6815c520 0x7faa680d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.835+0000 7faa677fe700 1 -- 172.21.15.97:0/3829028978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa68160180 msgr2=0x7faa68151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.835+0000 7faa677fe700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa68160180 0x7faa68151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.835+0000 7faa677fe700 1 -- 172.21.15.97:0/3829028978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa58009550 con 0x7faa68160d00 2023-07-06T03:50:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.835+0000 7faa677fe700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa68160d00 0x7faa68153af0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7faa6000cc40 tx=0x7faa6000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.835+0000 7faa4ffff700 1 -- 172.21.15.97:0/3829028978 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6000aa80 con 0x7faa68160d00 2023-07-06T03:50:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.835+0000 7faa4ffff700 1 -- 172.21.15.97:0/3829028978 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6000abe0 con 0x7faa68160d00 2023-07-06T03:50:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.835+0000 7faa6d9ff700 1 -- 172.21.15.97:0/3829028978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa680d1e00 con 0x7faa68160d00 2023-07-06T03:50:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.835+0000 7faa4ffff700 1 -- 172.21.15.97:0/3829028978 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60012420 con 0x7faa68160d00 2023-07-06T03:50:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.835+0000 7faa6d9ff700 1 -- 172.21.15.97:0/3829028978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa680d23c0 con 0x7faa68160d00 2023-07-06T03:50:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.836+0000 7faa6d9ff700 1 -- 172.21.15.97:0/3829028978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6804e340 con 0x7faa68160d00 2023-07-06T03:50:34.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.837+0000 7faa4ffff700 1 -- 172.21.15.97:0/3829028978 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faa60012580 con 0x7faa68160d00 2023-07-06T03:50:34.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.838+0000 7faa4ffff700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa5406ace0 0x7faa5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:34.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.838+0000 7faa4ffff700 1 -- 172.21.15.97:0/3829028978 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faa60089c10 con 0x7faa68160d00 2023-07-06T03:50:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.838+0000 7faa66ffd700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa5406ace0 0x7faa5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.839+0000 7faa66ffd700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa5406ace0 0x7faa5406d190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faa58009130 tx=0x7faa58008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:34.843+0000 7faa4ffff700 1 -- 172.21.15.97:0/3829028978 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faa60054b90 con 0x7faa68160d00 2023-07-06T03:50:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.083+0000 7faa6d9ff700 1 -- 172.21.15.97:0/3829028978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7faa680b4f40 con 0x7faa68160d00 2023-07-06T03:50:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.084+0000 7faa4ffff700 1 -- 172.21.15.97:0/3829028978 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7faa60016070 con 0x7faa68160d00 2023-07-06T03:50:35.086 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:50:35.086 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:50:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 -- 172.21.15.97:0/3829028978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa5406ace0 msgr2=0x7faa5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa5406ace0 0x7faa5406d190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faa58009130 tx=0x7faa58008030 comp rx=0 tx=0).stop 2023-07-06T03:50:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 -- 172.21.15.97:0/3829028978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa68160d00 msgr2=0x7faa68153af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa68160d00 0x7faa68153af0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7faa6000cc40 tx=0x7faa6000cf50 comp rx=0 tx=0).stop 2023-07-06T03:50:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 -- 172.21.15.97:0/3829028978 shutdown_connections 2023-07-06T03:50:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa5406ace0 0x7faa5406d190 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa68160d00 0x7faa68153af0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6815c520 0x7faa680d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 --2- 172.21.15.97:0/3829028978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa68160180 0x7faa68151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 -- 172.21.15.97:0/3829028978 >> 172.21.15.97:0/3829028978 conn(0x7faa68071790 msgr2=0x7faa681605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 -- 172.21.15.97:0/3829028978 shutdown_connections 2023-07-06T03:50:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:35.086+0000 7faa6d9ff700 1 -- 172.21.15.97:0/3829028978 wait complete. 2023-07-06T03:50:35.105 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:50:35.106 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:50:40.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:50:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.308+0000 7f755e3ff700 1 Processor -- start 2023-07-06T03:50:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.308+0000 7f755e3ff700 1 -- start start 2023-07-06T03:50:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.309+0000 7f755e3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f755815c740 0x7f75580d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.309+0000 7f755e3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7558071820 0x7f755806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f755e3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7558071c90 0x7f755806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:40.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f755e3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75580d0610 con 0x7f7558071820 2023-07-06T03:50:40.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f755e3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75580cdc80 con 0x7f7558071c90 2023-07-06T03:50:40.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f755e3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75580ce380 con 0x7f755815c740 2023-07-06T03:50:40.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f75577fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7558071820 0x7f755806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:40.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f75577fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7558071820 0x7f755806a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45976/0 (socket says 172.21.15.97:45976) 2023-07-06T03:50:40.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f75577fe700 1 -- 172.21.15.97:0/2712157092 learned_addr learned my addr 172.21.15.97:0/2712157092 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f7557fff700 1 --2- 172.21.15.97:0/2712157092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f755815c740 0x7f75580d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f755c99c700 1 --2- 172.21.15.97:0/2712157092 >> v2:172.21.15.146:3300/0 conn(0x7f7558071c90 0x7f755806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f75577fe700 1 -- 172.21.15.97:0/2712157092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f755815c740 msgr2=0x7f75580d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f75577fe700 1 --2- 172.21.15.97:0/2712157092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f755815c740 0x7f75580d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f75577fe700 1 -- 172.21.15.97:0/2712157092 >> v2:172.21.15.146:3300/0 conn(0x7f7558071c90 msgr2=0x7f755806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f75577fe700 1 --2- 172.21.15.97:0/2712157092 >> v2:172.21.15.146:3300/0 conn(0x7f7558071c90 0x7f755806a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f75577fe700 1 -- 172.21.15.97:0/2712157092 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580cd630 con 0x7f7558071820 2023-07-06T03:50:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.310+0000 7f7557fff700 1 --2- 172.21.15.97:0/2712157092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f755815c740 0x7f75580d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:50:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.311+0000 7f75577fe700 1 --2- 172.21.15.97:0/2712157092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7558071820 0x7f755806a060 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f755806f5b0 tx=0x7f754c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ed9fedfb89ba23 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.311+0000 7f7556ffd700 1 -- 172.21.15.97:0/2712157092 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c00b870 con 0x7f7558071820 2023-07-06T03:50:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.311+0000 7f7556ffd700 1 -- 172.21.15.97:0/2712157092 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c00b9d0 con 0x7f7558071820 2023-07-06T03:50:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.311+0000 7f7556ffd700 1 -- 172.21.15.97:0/2712157092 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c00bd60 con 0x7f7558071820 2023-07-06T03:50:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.311+0000 7f755e3ff700 1 -- 172.21.15.97:0/2712157092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7558071820 msgr2=0x7f755806a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.311+0000 7f755e3ff700 1 --2- 172.21.15.97:0/2712157092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7558071820 0x7f755806a060 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f755806f5b0 tx=0x7f754c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.311+0000 7f755e3ff700 1 -- 172.21.15.97:0/2712157092 shutdown_connections 2023-07-06T03:50:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.311+0000 7f755e3ff700 1 --2- 172.21.15.97:0/2712157092 >> v2:172.21.15.146:3300/0 conn(0x7f7558071c90 0x7f755806a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.311+0000 7f755e3ff700 1 --2- 172.21.15.97:0/2712157092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f755815c740 0x7f75580d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.311+0000 7f755e3ff700 1 --2- 172.21.15.97:0/2712157092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7558071820 0x7f755806a060 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.311+0000 7f755e3ff700 1 -- 172.21.15.97:0/2712157092 >> 172.21.15.97:0/2712157092 conn(0x7f75580b2280 msgr2=0x7f75580cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.312+0000 7f755e3ff700 1 -- 172.21.15.97:0/2712157092 shutdown_connections 2023-07-06T03:50:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.312+0000 7f755e3ff700 1 -- 172.21.15.97:0/2712157092 wait complete. 2023-07-06T03:50:40.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.313+0000 7f755e3ff700 1 Processor -- start 2023-07-06T03:50:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.313+0000 7f755e3ff700 1 -- start start 2023-07-06T03:50:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.314+0000 7f755e3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7558071820 0x7f755813fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.314+0000 7f755e3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f755815c740 0x7f7558159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.314+0000 7f755e3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75580b35d0 0x7f7558159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.314+0000 7f755e3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75581436d0 con 0x7f75580b35d0 2023-07-06T03:50:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.314+0000 7f755e3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755815e6e0 con 0x7f7558071820 2023-07-06T03:50:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.314+0000 7f755e3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75581433f0 con 0x7f755815c740 2023-07-06T03:50:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.315+0000 7f755c99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75580b35d0 0x7f7558159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.315+0000 7f7557fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7558071820 0x7f755813fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.315+0000 7f755c99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75580b35d0 0x7f7558159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45992/0 (socket says 172.21.15.97:45992) 2023-07-06T03:50:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.315+0000 7f755c99c700 1 -- 172.21.15.97:0/69891640 learned_addr learned my addr 172.21.15.97:0/69891640 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.315+0000 7f75577fe700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f755815c740 0x7f7558159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.315+0000 7f755c99c700 1 -- 172.21.15.97:0/69891640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f755815c740 msgr2=0x7f7558159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.315+0000 7f755c99c700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f755815c740 0x7f7558159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.315+0000 7f755c99c700 1 -- 172.21.15.97:0/69891640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7558071820 msgr2=0x7f755813fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.315+0000 7f755c99c700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7558071820 0x7f755813fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.315+0000 7f755c99c700 1 -- 172.21.15.97:0/69891640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c009550 con 0x7f75580b35d0 2023-07-06T03:50:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.315+0000 7f755c99c700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75580b35d0 0x7f7558159c90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f755000e810 tx=0x7f755000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.316+0000 7f7554ff9700 1 -- 172.21.15.97:0/69891640 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755000c070 con 0x7f75580b35d0 2023-07-06T03:50:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.316+0000 7f7554ff9700 1 -- 172.21.15.97:0/69891640 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7550013610 con 0x7f75580b35d0 2023-07-06T03:50:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.316+0000 7f7554ff9700 1 -- 172.21.15.97:0/69891640 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75500139c0 con 0x7f75580b35d0 2023-07-06T03:50:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.316+0000 7f755e3ff700 1 -- 172.21.15.97:0/69891640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75580b2af0 con 0x7f75580b35d0 2023-07-06T03:50:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.316+0000 7f755e3ff700 1 -- 172.21.15.97:0/69891640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75580b30b0 con 0x7f75580b35d0 2023-07-06T03:50:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.317+0000 7f755e3ff700 1 -- 172.21.15.97:0/69891640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75580b2950 con 0x7f75580b35d0 2023-07-06T03:50:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.318+0000 7f7554ff9700 1 -- 172.21.15.97:0/69891640 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7550013b20 con 0x7f75580b35d0 2023-07-06T03:50:40.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.319+0000 7f7554ff9700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f754406ace0 0x7f754406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:40.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.319+0000 7f7554ff9700 1 -- 172.21.15.97:0/69891640 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7550089030 con 0x7f75580b35d0 2023-07-06T03:50:40.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.319+0000 7f7557fff700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f754406ace0 0x7f754406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:40.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.320+0000 7f7557fff700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f754406ace0 0x7f754406d190 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7548003c10 tx=0x7f75480098e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:40.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.323+0000 7f7554ff9700 1 -- 172.21.15.97:0/69891640 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f75580b2950 con 0x7f75580b35d0 2023-07-06T03:50:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.559+0000 7f755e3ff700 1 -- 172.21.15.97:0/69891640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f75580b2950 con 0x7f75580b35d0 2023-07-06T03:50:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.559+0000 7f7554ff9700 1 -- 172.21.15.97:0/69891640 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f75580b2950 con 0x7f75580b35d0 2023-07-06T03:50:40.561 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:50:40.561 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:50:40.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 -- 172.21.15.97:0/69891640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f754406ace0 msgr2=0x7f754406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:40.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f754406ace0 0x7f754406d190 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7548003c10 tx=0x7f75480098e0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 -- 172.21.15.97:0/69891640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75580b35d0 msgr2=0x7f7558159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75580b35d0 0x7f7558159c90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f755000e810 tx=0x7f755000eb20 comp rx=0 tx=0).stop 2023-07-06T03:50:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 -- 172.21.15.97:0/69891640 shutdown_connections 2023-07-06T03:50:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f754406ace0 0x7f754406d190 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75580b35d0 0x7f7558159c90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7558071820 0x7f755813fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 --2- 172.21.15.97:0/69891640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f755815c740 0x7f7558159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 -- 172.21.15.97:0/69891640 >> 172.21.15.97:0/69891640 conn(0x7f75580b2280 msgr2=0x7f75580cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 -- 172.21.15.97:0/69891640 shutdown_connections 2023-07-06T03:50:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:40.561+0000 7f755e3ff700 1 -- 172.21.15.97:0/69891640 wait complete. 2023-07-06T03:50:40.580 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:50:40.581 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:50:45.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:50:45.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.783+0000 7fe77f3ff700 1 Processor -- start 2023-07-06T03:50:45.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.783+0000 7fe77f3ff700 1 -- start start 2023-07-06T03:50:45.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.784+0000 7fe77f3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe77815c980 0x7fe7780d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:45.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.784+0000 7fe77f3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe7780d2930 0x7fe7780cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:45.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.784+0000 7fe77f3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7780b1990 0x7fe7780b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:45.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.784+0000 7fe77f3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780d0c70 con 0x7fe7780b1990 2023-07-06T03:50:45.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77f3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780b2310 con 0x7fe7780d2930 2023-07-06T03:50:45.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77f3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780d0ec0 con 0x7fe77815c980 2023-07-06T03:50:45.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77d99c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7780b1990 0x7fe7780b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:45.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77d99c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7780b1990 0x7fe7780b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46002/0 (socket says 172.21.15.97:46002) 2023-07-06T03:50:45.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77d99c700 1 -- 172.21.15.97:0/1432816258 learned_addr learned my addr 172.21.15.97:0/1432816258 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:45.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77d19b700 1 --2- 172.21.15.97:0/1432816258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe77815c980 0x7fe7780d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:45.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77c99a700 1 --2- 172.21.15.97:0/1432816258 >> v2:172.21.15.146:3300/0 conn(0x7fe7780d2930 0x7fe7780cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:45.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77d99c700 1 -- 172.21.15.97:0/1432816258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe77815c980 msgr2=0x7fe7780d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:45.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77d99c700 1 --2- 172.21.15.97:0/1432816258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe77815c980 0x7fe7780d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:45.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77d99c700 1 -- 172.21.15.97:0/1432816258 >> v2:172.21.15.146:3300/0 conn(0x7fe7780d2930 msgr2=0x7fe7780cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:45.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77d99c700 1 --2- 172.21.15.97:0/1432816258 >> v2:172.21.15.146:3300/0 conn(0x7fe7780d2930 0x7fe7780cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:45.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77d99c700 1 -- 172.21.15.97:0/1432816258 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7780d11a0 con 0x7fe7780b1990 2023-07-06T03:50:45.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.785+0000 7fe77d99c700 1 --2- 172.21.15.97:0/1432816258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7780b1990 0x7fe7780b1dd0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe7780b5930 tx=0x7fe77000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5425c66baa4b32 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:45.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe76ffff700 1 -- 172.21.15.97:0/1432816258 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77000e070 con 0x7fe7780b1990 2023-07-06T03:50:45.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe76ffff700 1 -- 172.21.15.97:0/1432816258 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77000b9c0 con 0x7fe7780b1990 2023-07-06T03:50:45.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe76ffff700 1 -- 172.21.15.97:0/1432816258 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77000bd70 con 0x7fe7780b1990 2023-07-06T03:50:45.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1432816258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7780b1990 msgr2=0x7fe7780b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:45.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe77f3ff700 1 --2- 172.21.15.97:0/1432816258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7780b1990 0x7fe7780b1dd0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe7780b5930 tx=0x7fe77000b330 comp rx=0 tx=0).stop 2023-07-06T03:50:45.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1432816258 shutdown_connections 2023-07-06T03:50:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe77f3ff700 1 --2- 172.21.15.97:0/1432816258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7780b1990 0x7fe7780b1dd0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe77f3ff700 1 --2- 172.21.15.97:0/1432816258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe77815c980 0x7fe7780d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe77f3ff700 1 --2- 172.21.15.97:0/1432816258 >> v2:172.21.15.146:3300/0 conn(0x7fe7780d2930 0x7fe7780cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1432816258 >> 172.21.15.97:0/1432816258 conn(0x7fe77806b2f0 msgr2=0x7fe77806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1432816258 shutdown_connections 2023-07-06T03:50:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.786+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1432816258 wait complete. 2023-07-06T03:50:45.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.788+0000 7fe77f3ff700 1 Processor -- start 2023-07-06T03:50:45.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.788+0000 7fe77f3ff700 1 -- start start 2023-07-06T03:50:45.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.789+0000 7fe77f3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7780b1990 0x7fe7780ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:45.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.789+0000 7fe77f3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7780d2930 0x7fe778153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:45.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.789+0000 7fe77f3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe77815c980 0x7fe7781e6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:45.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.789+0000 7fe77f3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780c7810 con 0x7fe7780d2930 2023-07-06T03:50:45.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.789+0000 7fe77f3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780ca000 con 0x7fe7780b1990 2023-07-06T03:50:45.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.789+0000 7fe77f3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7781543a0 con 0x7fe77815c980 2023-07-06T03:50:45.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.789+0000 7fe77c99a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7780d2930 0x7fe778153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:45.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77c99a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7780d2930 0x7fe778153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46004/0 (socket says 172.21.15.97:46004) 2023-07-06T03:50:45.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77c99a700 1 -- 172.21.15.97:0/1586431906 learned_addr learned my addr 172.21.15.97:0/1586431906 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:45.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77d19b700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7780b1990 0x7fe7780ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:45.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77d99c700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe77815c980 0x7fe7781e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:45.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77c99a700 1 -- 172.21.15.97:0/1586431906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe77815c980 msgr2=0x7fe7781e6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:45.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77c99a700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe77815c980 0x7fe7781e6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:45.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77c99a700 1 -- 172.21.15.97:0/1586431906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7780b1990 msgr2=0x7fe7780ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:45.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77c99a700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7780b1990 0x7fe7780ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:45.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77c99a700 1 -- 172.21.15.97:0/1586431906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe770009b30 con 0x7fe7780d2930 2023-07-06T03:50:45.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77d19b700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7780b1990 0x7fe7780ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:50:45.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77c99a700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7780d2930 0x7fe778153e60 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe76800e810 tx=0x7fe76800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:45.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe76dffb700 1 -- 172.21.15.97:0/1586431906 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768013610 con 0x7fe7780d2930 2023-07-06T03:50:45.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe76dffb700 1 -- 172.21.15.97:0/1586431906 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe768013770 con 0x7fe7780d2930 2023-07-06T03:50:45.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe76dffb700 1 -- 172.21.15.97:0/1586431906 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768013b00 con 0x7fe7780d2930 2023-07-06T03:50:45.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.790+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1586431906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7780c6da0 con 0x7fe7780d2930 2023-07-06T03:50:45.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.791+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1586431906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7780c7530 con 0x7fe7780d2930 2023-07-06T03:50:45.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.791+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1586431906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7780ca190 con 0x7fe7780d2930 2023-07-06T03:50:45.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.792+0000 7fe76dffb700 1 -- 172.21.15.97:0/1586431906 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe768013c60 con 0x7fe7780d2930 2023-07-06T03:50:45.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.793+0000 7fe76dffb700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe75c06ace0 0x7fe75c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:45.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.793+0000 7fe76dffb700 1 -- 172.21.15.97:0/1586431906 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe768088fa0 con 0x7fe7780d2930 2023-07-06T03:50:45.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.793+0000 7fe77d19b700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe75c06ace0 0x7fe75c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:45.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.794+0000 7fe77d19b700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe75c06ace0 0x7fe75c06d190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe774007a50 tx=0x7fe774008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:45.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:45.798+0000 7fe76dffb700 1 -- 172.21.15.97:0/1586431906 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe768057b70 con 0x7fe7780d2930 2023-07-06T03:50:46.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.044+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1586431906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe77804e340 con 0x7fe7780d2930 2023-07-06T03:50:46.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.044+0000 7fe76dffb700 1 -- 172.21.15.97:0/1586431906 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fe768057700 con 0x7fe7780d2930 2023-07-06T03:50:46.046 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:50:46.047 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:50:46.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1586431906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe75c06ace0 msgr2=0x7fe75c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:46.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe75c06ace0 0x7fe75c06d190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe774007a50 tx=0x7fe774008040 comp rx=0 tx=0).stop 2023-07-06T03:50:46.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1586431906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7780d2930 msgr2=0x7fe778153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:46.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7780d2930 0x7fe778153e60 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe76800e810 tx=0x7fe76800ebd0 comp rx=0 tx=0).stop 2023-07-06T03:50:46.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1586431906 shutdown_connections 2023-07-06T03:50:46.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe75c06ace0 0x7fe75c06d190 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:46.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe77815c980 0x7fe7781e6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:46.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7780b1990 0x7fe7780ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:46.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 --2- 172.21.15.97:0/1586431906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7780d2930 0x7fe778153e60 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:46.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1586431906 >> 172.21.15.97:0/1586431906 conn(0x7fe77806b2f0 msgr2=0x7fe77815a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:46.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1586431906 shutdown_connections 2023-07-06T03:50:46.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:46.047+0000 7fe77f3ff700 1 -- 172.21.15.97:0/1586431906 wait complete. 2023-07-06T03:50:46.066 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:50:46.066 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:50:51.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:50:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.270+0000 7fcbf61ff700 1 Processor -- start 2023-07-06T03:50:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.270+0000 7fcbf61ff700 1 -- start start 2023-07-06T03:50:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.271+0000 7fcbf61ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcbf015b440 0x7fcbf015e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.271+0000 7fcbf61ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcbf0070c30 0x7fcbf00adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.271+0000 7fcbf61ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf0071330 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.271+0000 7fcbf61ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf00ae3e0 con 0x7fcbf0070c30 2023-07-06T03:50:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.271+0000 7fcbf61ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf0071610 con 0x7fcbf015b440 2023-07-06T03:50:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.271+0000 7fcbeffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcbf0070c30 0x7fcbf00adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbeffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcbf0070c30 0x7fcbf00adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39858/0 (socket says 172.21.15.97:39858) 2023-07-06T03:50:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbeffff700 1 -- 172.21.15.97:0/992429951 learned_addr learned my addr 172.21.15.97:0/992429951 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbef7fe700 1 --1- 172.21.15.97:0/992429951 >> v1:172.21.15.97:6789/0 conn(0x7fcbf00cd8e0 0x7fcbf00cdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51212/0 (socket says 172.21.15.97:51212) 2023-07-06T03:50:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbeeffd700 1 --2- 172.21.15.97:0/992429951 >> v2:172.21.15.146:3300/0 conn(0x7fcbf015b440 0x7fcbf015e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:51.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbeffff700 1 -- 172.21.15.97:0/992429951 >> v2:172.21.15.146:3300/0 conn(0x7fcbf015b440 msgr2=0x7fcbf015e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:51.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbeffff700 1 --2- 172.21.15.97:0/992429951 >> v2:172.21.15.146:3300/0 conn(0x7fcbf015b440 0x7fcbf015e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:51.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbeffff700 1 -- 172.21.15.97:0/992429951 >> v1:172.21.15.97:6789/0 conn(0x7fcbf00cd8e0 legacy=0x7fcbf00cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:51.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbeffff700 1 -- 172.21.15.97:0/992429951 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf015dd80 con 0x7fcbf0070c30 2023-07-06T03:50:51.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbee7fc700 1 -- 172.21.15.97:0/992429951 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778397730 0 0) 0x7fcbf0071330 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbeffff700 1 --2- 172.21.15.97:0/992429951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf0070c30 0x7fcbf00adea0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00b4970 tx=0x7fcbe800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca0f04cc17f59e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:51.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbee7fc700 1 -- 172.21.15.97:0/992429951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe800e070 con 0x7fcbf0070c30 2023-07-06T03:50:51.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbee7fc700 1 -- 172.21.15.97:0/992429951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe800b910 con 0x7fcbf0070c30 2023-07-06T03:50:51.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.272+0000 7fcbee7fc700 1 -- 172.21.15.97:0/992429951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe800bcc0 con 0x7fcbf0070c30 2023-07-06T03:50:51.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.273+0000 7fcbf61ff700 1 -- 172.21.15.97:0/992429951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf0070c30 msgr2=0x7fcbf00adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:51.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.273+0000 7fcbf61ff700 1 --2- 172.21.15.97:0/992429951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf0070c30 0x7fcbf00adea0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00b4970 tx=0x7fcbe800b330 comp rx=0 tx=0).stop 2023-07-06T03:50:51.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.273+0000 7fcbf61ff700 1 -- 172.21.15.97:0/992429951 shutdown_connections 2023-07-06T03:50:51.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.273+0000 7fcbf61ff700 1 --2- 172.21.15.97:0/992429951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf0070c30 0x7fcbf00adea0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:51.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.273+0000 7fcbf61ff700 1 --2- 172.21.15.97:0/992429951 >> v2:172.21.15.146:3300/0 conn(0x7fcbf015b440 0x7fcbf015e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:51.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.273+0000 7fcbf61ff700 1 -- 172.21.15.97:0/992429951 >> 172.21.15.97:0/992429951 conn(0x7fcbf00ce8c0 msgr2=0x7fcbf00cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:51.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.273+0000 7fcbf61ff700 1 -- 172.21.15.97:0/992429951 shutdown_connections 2023-07-06T03:50:51.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.273+0000 7fcbf61ff700 1 -- 172.21.15.97:0/992429951 wait complete. 2023-07-06T03:50:51.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.275+0000 7fcbf61ff700 1 Processor -- start 2023-07-06T03:50:51.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.275+0000 7fcbf61ff700 1 -- start start 2023-07-06T03:50:51.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.276+0000 7fcbf61ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbf0070c30 0x7fcbf006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:51.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.276+0000 7fcbf61ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00cd8e0 0x7fcbf00acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:51.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.276+0000 7fcbf61ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbf015b440 0x7fcbf01451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:51.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.276+0000 7fcbf61ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf006d1a0 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.276+0000 7fcbf61ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf006c520 con 0x7fcbf015b440 2023-07-06T03:50:51.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.276+0000 7fcbf61ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf006c800 con 0x7fcbf0070c30 2023-07-06T03:50:51.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.276+0000 7fcbeeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00cd8e0 0x7fcbf00acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:51.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.276+0000 7fcbeeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00cd8e0 0x7fcbf00acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39864/0 (socket says 172.21.15.97:39864) 2023-07-06T03:50:51.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.276+0000 7fcbeeffd700 1 -- 172.21.15.97:0/263627331 learned_addr learned my addr 172.21.15.97:0/263627331 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:51.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.276+0000 7fcbef7fe700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbf0070c30 0x7fcbf006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:51.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.276+0000 7fcbeffff700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbf015b440 0x7fcbf01451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:51.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbeeffd700 1 -- 172.21.15.97:0/263627331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbf0070c30 msgr2=0x7fcbf006d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:51.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbeeffd700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbf0070c30 0x7fcbf006d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:51.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbeeffd700 1 -- 172.21.15.97:0/263627331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbf015b440 msgr2=0x7fcbf01451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:51.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbeeffd700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbf015b440 0x7fcbf01451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:51.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbeeffd700 1 -- 172.21.15.97:0/263627331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe8009b30 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbef7fe700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbf0070c30 0x7fcbf006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:50:51.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbeeffd700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00cd8e0 0x7fcbf00acae0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400e980 tx=0x7fcbe400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:51.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbd7fff700 1 -- 172.21.15.97:0/263627331 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4009710 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbd7fff700 1 -- 172.21.15.97:0/263627331 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe400eed0 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbf61ff700 1 -- 172.21.15.97:0/263627331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf006cae0 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbd7fff700 1 -- 172.21.15.97:0/263627331 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400ca70 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.277+0000 7fcbf61ff700 1 -- 172.21.15.97:0/263627331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf006db00 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.278+0000 7fcbf61ff700 1 -- 172.21.15.97:0/263627331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf0070aa0 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.279+0000 7fcbd7fff700 1 -- 172.21.15.97:0/263627331 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcbe400cbd0 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.280+0000 7fcbd7fff700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbdc06ace0 0x7fcbdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:51.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.280+0000 7fcbd7fff700 1 -- 172.21.15.97:0/263627331 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbe4089540 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.280+0000 7fcbef7fe700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbdc06ace0 0x7fcbdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:51.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.281+0000 7fcbef7fe700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbdc06ace0 0x7fcbdc06d190 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0000f60 tx=0x7fcbe0007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:51.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.285+0000 7fcbd7fff700 1 -- 172.21.15.97:0/263627331 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcbe40545f0 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.521+0000 7fcbf61ff700 1 -- 172.21.15.97:0/263627331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcbf006d2e0 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.521+0000 7fcbd7fff700 1 -- 172.21.15.97:0/263627331 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fcbf006d2e0 con 0x7fcbf00cd8e0 2023-07-06T03:50:51.524 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:50:51.525 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:50:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.523+0000 7fcbf61ff700 1 -- 172.21.15.97:0/263627331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbdc06ace0 msgr2=0x7fcbdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.523+0000 7fcbf61ff700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbdc06ace0 0x7fcbdc06d190 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0000f60 tx=0x7fcbe0007ca0 comp rx=0 tx=0).stop 2023-07-06T03:50:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.523+0000 7fcbf61ff700 1 -- 172.21.15.97:0/263627331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00cd8e0 msgr2=0x7fcbf00acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.523+0000 7fcbf61ff700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00cd8e0 0x7fcbf00acae0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400e980 tx=0x7fcbe400ec90 comp rx=0 tx=0).stop 2023-07-06T03:50:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.524+0000 7fcbf61ff700 1 -- 172.21.15.97:0/263627331 shutdown_connections 2023-07-06T03:50:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.524+0000 7fcbf61ff700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbdc06ace0 0x7fcbdc06d190 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.524+0000 7fcbf61ff700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbf015b440 0x7fcbf01451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.524+0000 7fcbf61ff700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbf0070c30 0x7fcbf006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.524+0000 7fcbf61ff700 1 --2- 172.21.15.97:0/263627331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00cd8e0 0x7fcbf00acae0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.524+0000 7fcbf61ff700 1 -- 172.21.15.97:0/263627331 >> 172.21.15.97:0/263627331 conn(0x7fcbf00ce8c0 msgr2=0x7fcbf015b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.524+0000 7fcbf61ff700 1 -- 172.21.15.97:0/263627331 shutdown_connections 2023-07-06T03:50:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:51.524+0000 7fcbf61ff700 1 -- 172.21.15.97:0/263627331 wait complete. 2023-07-06T03:50:51.544 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:50:51.545 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:50:56.546 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:50:56.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.746+0000 7f2fc5dff700 1 Processor -- start 2023-07-06T03:50:56.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.746+0000 7f2fc5dff700 1 -- start start 2023-07-06T03:50:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.748+0000 7f2fc5dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2fc00cd390 0x7f2fc0160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.748+0000 7f2fc5dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fc0160dd0 0x7f2fc00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.748+0000 7f2fc5dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc00cdae0 con 0x7f2fc015c520 2023-07-06T03:50:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.748+0000 7f2fc5dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00b6640 con 0x7f2fc00cd390 2023-07-06T03:50:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.748+0000 7f2fc5dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00b6920 con 0x7f2fc0160dd0 2023-07-06T03:50:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.748+0000 7f2fbf7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2fc015c520 0x7f2fc0160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51218/0 (socket says 172.21.15.97:51218) 2023-07-06T03:50:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.748+0000 7f2fbf7fe700 1 -- 172.21.15.97:0/436084403 learned_addr learned my addr 172.21.15.97:0/436084403 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:50:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.748+0000 7f2fbffff700 1 --2- 172.21.15.97:0/436084403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fc0160dd0 0x7f2fc00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.748+0000 7f2fbeffd700 1 --2- 172.21.15.97:0/436084403 >> v2:172.21.15.146:3300/0 conn(0x7f2fc00cd390 0x7f2fc0160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.748+0000 7f2fbe7fc700 1 -- 172.21.15.97:0/436084403 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478723420 0 0) 0x7f2fc00cdae0 con 0x7f2fc015c520 2023-07-06T03:50:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.748+0000 7f2fbe7fc700 1 -- 172.21.15.97:0/436084403 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa80035b0 con 0x7f2fc015c520 2023-07-06T03:50:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.749+0000 7f2fbeffd700 1 -- 172.21.15.97:0/436084403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fc0160dd0 msgr2=0x7f2fc00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:50:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.749+0000 7f2fbeffd700 1 --2- 172.21.15.97:0/436084403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fc0160dd0 0x7f2fc00b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:56.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.749+0000 7f2fbeffd700 1 -- 172.21.15.97:0/436084403 >> v1:172.21.15.97:6789/0 conn(0x7f2fc015c520 legacy=0x7f2fc0160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:56.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.749+0000 7f2fbeffd700 1 -- 172.21.15.97:0/436084403 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00b6c00 con 0x7f2fc00cd390 2023-07-06T03:50:56.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.749+0000 7f2fbeffd700 1 --2- 172.21.15.97:0/436084403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fc00cd390 0x7f2fc0160890 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009840 tx=0x7f2fb4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49258814d97d250d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:56.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.749+0000 7f2fbe7fc700 1 -- 172.21.15.97:0/436084403 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4011830 con 0x7f2fc00cd390 2023-07-06T03:50:56.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.749+0000 7f2fbe7fc700 1 -- 172.21.15.97:0/436084403 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb4011990 con 0x7f2fc00cd390 2023-07-06T03:50:56.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.749+0000 7f2fbe7fc700 1 -- 172.21.15.97:0/436084403 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4011d00 con 0x7f2fc00cd390 2023-07-06T03:50:56.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.750+0000 7f2fc5dff700 1 -- 172.21.15.97:0/436084403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fc00cd390 msgr2=0x7f2fc0160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:56.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.750+0000 7f2fc5dff700 1 --2- 172.21.15.97:0/436084403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fc00cd390 0x7f2fc0160890 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009840 tx=0x7f2fb4009b50 comp rx=0 tx=0).stop 2023-07-06T03:50:56.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.750+0000 7f2fc5dff700 1 -- 172.21.15.97:0/436084403 shutdown_connections 2023-07-06T03:50:56.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.750+0000 7f2fc5dff700 1 --2- 172.21.15.97:0/436084403 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fc0160dd0 0x7f2fc00b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.750+0000 7f2fc5dff700 1 --2- 172.21.15.97:0/436084403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fc00cd390 0x7f2fc0160890 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.750+0000 7f2fc5dff700 1 -- 172.21.15.97:0/436084403 >> 172.21.15.97:0/436084403 conn(0x7f2fc0071790 msgr2=0x7f2fc0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.750+0000 7f2fc5dff700 1 -- 172.21.15.97:0/436084403 shutdown_connections 2023-07-06T03:50:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.750+0000 7f2fc5dff700 1 -- 172.21.15.97:0/436084403 wait complete. 2023-07-06T03:50:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.752+0000 7f2fc5dff700 1 Processor -- start 2023-07-06T03:50:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.752+0000 7f2fc5dff700 1 -- start start 2023-07-06T03:50:56.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fc5dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc00cd390 0x7f2fc015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:56.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fc5dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fc015c520 0x7f2fc00d2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:56.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fc5dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fc0160dd0 0x7f2fc00d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:56.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fc5dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc015a9e0 con 0x7f2fc00cd390 2023-07-06T03:50:56.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fc5dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc015acc0 con 0x7f2fc0160dd0 2023-07-06T03:50:56.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fc5dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc015afa0 con 0x7f2fc015c520 2023-07-06T03:50:56.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fbf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc00cd390 0x7f2fc015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:56.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fbf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc00cd390 0x7f2fc015f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39880/0 (socket says 172.21.15.97:39880) 2023-07-06T03:50:56.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fbf7fe700 1 -- 172.21.15.97:0/546683203 learned_addr learned my addr 172.21.15.97:0/546683203 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:50:56.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fbffff700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fc0160dd0 0x7f2fc00d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:56.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fbeffd700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fc015c520 0x7f2fc00d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:56.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fbf7fe700 1 -- 172.21.15.97:0/546683203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fc015c520 msgr2=0x7f2fc00d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:56.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fbf7fe700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fc015c520 0x7f2fc00d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:56.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fbf7fe700 1 -- 172.21.15.97:0/546683203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fc0160dd0 msgr2=0x7f2fc00d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:56.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fbf7fe700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fc0160dd0 0x7f2fc00d2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:56.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.753+0000 7f2fbf7fe700 1 -- 172.21.15.97:0/546683203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb40094a0 con 0x7f2fc00cd390 2023-07-06T03:50:56.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.754+0000 7f2fbf7fe700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc00cd390 0x7f2fc015f3c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000c420 tx=0x7f2fb000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:56.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.754+0000 7f2fa7fff700 1 -- 172.21.15.97:0/546683203 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0005d70 con 0x7f2fc00cd390 2023-07-06T03:50:56.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.754+0000 7f2fa7fff700 1 -- 172.21.15.97:0/546683203 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb00070d0 con 0x7f2fc00cd390 2023-07-06T03:50:56.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.754+0000 7f2fa7fff700 1 -- 172.21.15.97:0/546683203 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0007440 con 0x7f2fc00cd390 2023-07-06T03:50:56.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.754+0000 7f2fc5dff700 1 -- 172.21.15.97:0/546683203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc015b280 con 0x7f2fc00cd390 2023-07-06T03:50:56.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.754+0000 7f2fc5dff700 1 -- 172.21.15.97:0/546683203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc0150810 con 0x7f2fc00cd390 2023-07-06T03:50:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.755+0000 7f2fc5dff700 1 -- 172.21.15.97:0/546683203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc015cdb0 con 0x7f2fc00cd390 2023-07-06T03:50:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.756+0000 7f2fa7fff700 1 -- 172.21.15.97:0/546683203 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2fb00075a0 con 0x7f2fc00cd390 2023-07-06T03:50:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.757+0000 7f2fa7fff700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fac06ace0 0x7f2fac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:50:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.757+0000 7f2fa7fff700 1 -- 172.21.15.97:0/546683203 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2fb0088890 con 0x7f2fc00cd390 2023-07-06T03:50:56.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.757+0000 7f2fbeffd700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fac06ace0 0x7f2fac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:50:56.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.758+0000 7f2fbeffd700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fac06ace0 0x7f2fac06d190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4000c00 tx=0x7f2fb401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:50:56.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:56.761+0000 7f2fa7fff700 1 -- 172.21.15.97:0/546683203 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2fc015cdb0 con 0x7f2fc00cd390 2023-07-06T03:50:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.005+0000 7f2fc5dff700 1 -- 172.21.15.97:0/546683203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2fc015cdb0 con 0x7f2fc00cd390 2023-07-06T03:50:57.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.005+0000 7f2fa7fff700 1 -- 172.21.15.97:0/546683203 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f2fb0053950 con 0x7f2fc00cd390 2023-07-06T03:50:57.008 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:50:57.008 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:50:57.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.007+0000 7f2fc5dff700 1 -- 172.21.15.97:0/546683203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fac06ace0 msgr2=0x7f2fac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:57.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.007+0000 7f2fc5dff700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fac06ace0 0x7f2fac06d190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4000c00 tx=0x7f2fb401c040 comp rx=0 tx=0).stop 2023-07-06T03:50:57.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.007+0000 7f2fc5dff700 1 -- 172.21.15.97:0/546683203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc00cd390 msgr2=0x7f2fc015f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:50:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.007+0000 7f2fc5dff700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc00cd390 0x7f2fc015f3c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000c420 tx=0x7f2fb000c7e0 comp rx=0 tx=0).stop 2023-07-06T03:50:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.008+0000 7f2fc5dff700 1 -- 172.21.15.97:0/546683203 shutdown_connections 2023-07-06T03:50:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.008+0000 7f2fc5dff700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fac06ace0 0x7f2fac06d190 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.008+0000 7f2fc5dff700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fc0160dd0 0x7f2fc00d2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.008+0000 7f2fc5dff700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fc00cd390 0x7f2fc015f3c0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.008+0000 7f2fc5dff700 1 --2- 172.21.15.97:0/546683203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2fc015c520 0x7f2fc00d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:50:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.008+0000 7f2fc5dff700 1 -- 172.21.15.97:0/546683203 >> 172.21.15.97:0/546683203 conn(0x7f2fc0071790 msgr2=0x7f2fc00b8260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:50:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.008+0000 7f2fc5dff700 1 -- 172.21.15.97:0/546683203 shutdown_connections 2023-07-06T03:50:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:50:57.008+0000 7f2fc5dff700 1 -- 172.21.15.97:0/546683203 wait complete. 2023-07-06T03:50:57.028 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:50:57.028 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:51:02.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:51:02.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.230+0000 7f32911ff700 1 Processor -- start 2023-07-06T03:51:02.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.230+0000 7f32911ff700 1 -- start start 2023-07-06T03:51:02.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.231+0000 7f32911ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328c15c920 0x7f328c06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:02.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.232+0000 7f32911ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328c0cfd00 con 0x7f328c15ccf0 2023-07-06T03:51:02.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.232+0000 7f32911ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328c06bf90 con 0x7f328c0d11a0 2023-07-06T03:51:02.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.232+0000 7f32911ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0b28f0 con 0x7f328c15c920 2023-07-06T03:51:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.232+0000 7f3283fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f328c15ccf0 0x7f328c06b170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53342/0 (socket says 172.21.15.97:53342) 2023-07-06T03:51:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.232+0000 7f3283fff700 1 -- 172.21.15.97:0/3478094230 learned_addr learned my addr 172.21.15.97:0/3478094230 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:51:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.232+0000 7f328ad9d700 1 --2- 172.21.15.97:0/3478094230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328c15c920 0x7f328c06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.232+0000 7f328a59c700 1 -- 172.21.15.97:0/3478094230 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884267640 0 0) 0x7f328c0cfd00 con 0x7f328c15ccf0 2023-07-06T03:51:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.232+0000 7f328a59c700 1 -- 172.21.15.97:0/3478094230 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32780035b0 con 0x7f328c15ccf0 2023-07-06T03:51:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.233+0000 7f328a59c700 1 -- 172.21.15.97:0/3478094230 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508099082 0 0) 0x7f328c06bf90 con 0x7f328c0d11a0 2023-07-06T03:51:02.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.233+0000 7f328a59c700 1 -- 172.21.15.97:0/3478094230 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f328c0cfd00 con 0x7f328c0d11a0 2023-07-06T03:51:02.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.233+0000 7f328ad9d700 1 -- 172.21.15.97:0/3478094230 >> v1:172.21.15.146:6789/0 conn(0x7f328c0d11a0 legacy=0x7f328c06b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:02.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.233+0000 7f328a59c700 1 -- 172.21.15.97:0/3478094230 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3397328648 0 0) 0x7f32780035b0 con 0x7f328c15ccf0 2023-07-06T03:51:02.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.233+0000 7f328ad9d700 1 -- 172.21.15.97:0/3478094230 >> v1:172.21.15.97:6789/0 conn(0x7f328c15ccf0 legacy=0x7f328c06b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:02.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.233+0000 7f328ad9d700 1 -- 172.21.15.97:0/3478094230 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c1610b0 con 0x7f328c15c920 2023-07-06T03:51:02.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.233+0000 7f328ad9d700 1 --2- 172.21.15.97:0/3478094230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328c15c920 0x7f328c06c390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f327c009870 tx=0x7f327c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92450c8d9d646afb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:02.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.233+0000 7f328a59c700 1 -- 172.21.15.97:0/3478094230 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274001670 con 0x7f328c15c920 2023-07-06T03:51:02.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.233+0000 7f328a59c700 1 -- 172.21.15.97:0/3478094230 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c004030 con 0x7f328c15c920 2023-07-06T03:51:02.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.233+0000 7f328a59c700 1 -- 172.21.15.97:0/3478094230 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0043c0 con 0x7f328c15c920 2023-07-06T03:51:02.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.234+0000 7f32911ff700 1 -- 172.21.15.97:0/3478094230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328c15c920 msgr2=0x7f328c06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:02.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.234+0000 7f32911ff700 1 --2- 172.21.15.97:0/3478094230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328c15c920 0x7f328c06c390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f327c009870 tx=0x7f327c009b80 comp rx=0 tx=0).stop 2023-07-06T03:51:02.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.234+0000 7f32911ff700 1 -- 172.21.15.97:0/3478094230 shutdown_connections 2023-07-06T03:51:02.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.234+0000 7f32911ff700 1 --2- 172.21.15.97:0/3478094230 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328c15c920 0x7f328c06c390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:02.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.234+0000 7f32911ff700 1 -- 172.21.15.97:0/3478094230 >> 172.21.15.97:0/3478094230 conn(0x7f328c06ace0 msgr2=0x7f328c06dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:02.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.234+0000 7f32911ff700 1 -- 172.21.15.97:0/3478094230 shutdown_connections 2023-07-06T03:51:02.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.234+0000 7f32911ff700 1 -- 172.21.15.97:0/3478094230 wait complete. 2023-07-06T03:51:02.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.236+0000 7f32911ff700 1 Processor -- start 2023-07-06T03:51:02.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.236+0000 7f32911ff700 1 -- start start 2023-07-06T03:51:02.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f32911ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328c15c920 0x7f328c1f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:02.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f32911ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328c15ccf0 0x7f328c0c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:02.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f32911ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0c5a50 0x7f328c0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:02.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f32911ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0c5ef0 con 0x7f328c0c5a50 2023-07-06T03:51:02.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f32911ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c1f5930 con 0x7f328c15c920 2023-07-06T03:51:02.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f32911ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c1f21f0 con 0x7f328c15ccf0 2023-07-06T03:51:02.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f328b59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0c5a50 0x7f328c0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:02.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f328b59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0c5a50 0x7f328c0c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57162/0 (socket says 172.21.15.97:57162) 2023-07-06T03:51:02.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f328b59e700 1 -- 172.21.15.97:0/465085122 learned_addr learned my addr 172.21.15.97:0/465085122 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:02.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f3283fff700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328c15ccf0 0x7f328c0c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:02.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f328ad9d700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328c15c920 0x7f328c1f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:02.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f328b59e700 1 -- 172.21.15.97:0/465085122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328c15ccf0 msgr2=0x7f328c0c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:02.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f328b59e700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328c15ccf0 0x7f328c0c5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:02.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f328b59e700 1 -- 172.21.15.97:0/465085122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328c15c920 msgr2=0x7f328c1f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:02.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f328b59e700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328c15c920 0x7f328c1f1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:02.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.237+0000 7f328b59e700 1 -- 172.21.15.97:0/465085122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c0094a0 con 0x7f328c0c5a50 2023-07-06T03:51:02.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.238+0000 7f328b59e700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0c5a50 0x7f328c0c7f40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f328400cc20 tx=0x7f328400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:02.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.238+0000 7f32837fe700 1 -- 172.21.15.97:0/465085122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284003c00 con 0x7f328c0c5a50 2023-07-06T03:51:02.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.238+0000 7f32837fe700 1 -- 172.21.15.97:0/465085122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3284003d60 con 0x7f328c0c5a50 2023-07-06T03:51:02.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.238+0000 7f32911ff700 1 -- 172.21.15.97:0/465085122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c1f5b80 con 0x7f328c0c5a50 2023-07-06T03:51:02.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.238+0000 7f32837fe700 1 -- 172.21.15.97:0/465085122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328400abe0 con 0x7f328c0c5a50 2023-07-06T03:51:02.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.238+0000 7f32911ff700 1 -- 172.21.15.97:0/465085122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c1f6140 con 0x7f328c0c5a50 2023-07-06T03:51:02.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.239+0000 7f32911ff700 1 -- 172.21.15.97:0/465085122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328c066a40 con 0x7f328c0c5a50 2023-07-06T03:51:02.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.240+0000 7f32837fe700 1 -- 172.21.15.97:0/465085122 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f328400ad40 con 0x7f328c0c5a50 2023-07-06T03:51:02.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.240+0000 7f32837fe700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f326c06ace0 0x7f326c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:02.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.241+0000 7f32837fe700 1 -- 172.21.15.97:0/465085122 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3284089670 con 0x7f328c0c5a50 2023-07-06T03:51:02.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.241+0000 7f328ad9d700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f326c06ace0 0x7f326c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:02.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.241+0000 7f328ad9d700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f326c06ace0 0x7f326c06d190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f327c0097c0 tx=0x7f327c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:02.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.245+0000 7f32837fe700 1 -- 172.21.15.97:0/465085122 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3284054680 con 0x7f328c0c5a50 2023-07-06T03:51:02.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.483+0000 7f32911ff700 1 -- 172.21.15.97:0/465085122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f328c0d1930 con 0x7f328c0c5a50 2023-07-06T03:51:02.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.483+0000 7f32837fe700 1 -- 172.21.15.97:0/465085122 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f3284057ca0 con 0x7f328c0c5a50 2023-07-06T03:51:02.487 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:51:02.487 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:51:02.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 -- 172.21.15.97:0/465085122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f326c06ace0 msgr2=0x7f326c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:02.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f326c06ace0 0x7f326c06d190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f327c0097c0 tx=0x7f327c008030 comp rx=0 tx=0).stop 2023-07-06T03:51:02.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 -- 172.21.15.97:0/465085122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0c5a50 msgr2=0x7f328c0c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:02.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0c5a50 0x7f328c0c7f40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f328400cc20 tx=0x7f328400a060 comp rx=0 tx=0).stop 2023-07-06T03:51:02.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 -- 172.21.15.97:0/465085122 shutdown_connections 2023-07-06T03:51:02.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f326c06ace0 0x7f326c06d190 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0c5a50 0x7f328c0c7f40 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328c15c920 0x7f328c1f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 --2- 172.21.15.97:0/465085122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f328c15ccf0 0x7f328c0c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 -- 172.21.15.97:0/465085122 >> 172.21.15.97:0/465085122 conn(0x7f328c0d11a0 msgr2=0x7f328c160590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:02.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 -- 172.21.15.97:0/465085122 shutdown_connections 2023-07-06T03:51:02.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:02.486+0000 7f32911ff700 1 -- 172.21.15.97:0/465085122 wait complete. 2023-07-06T03:51:02.506 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:51:02.507 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:51:04.730 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:51:04.735 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:51:07.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:51:07.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.708+0000 7f4c717ff700 1 Processor -- start 2023-07-06T03:51:07.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.708+0000 7f4c717ff700 1 -- start start 2023-07-06T03:51:07.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.709+0000 7f4c717ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4c6c0b21e0 0x7f4c6c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:07.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.709+0000 7f4c717ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6c0d1e80 con 0x7f4c6c160b10 2023-07-06T03:51:07.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.709+0000 7f4c717ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6c0d09b0 con 0x7f4c6c0b47c0 2023-07-06T03:51:07.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.709+0000 7f4c717ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6c0d0c90 con 0x7f4c6c0b21e0 2023-07-06T03:51:07.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.709+0000 7f4c6a7fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4c6c160b10 0x7f4c6c0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53346/0 (socket says 172.21.15.97:53346) 2023-07-06T03:51:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.709+0000 7f4c6a7fc700 1 -- 172.21.15.97:0/3783072197 learned_addr learned my addr 172.21.15.97:0/3783072197 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:51:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895069958 0 0) 0x7f4c6c0d1e80 con 0x7f4c6c160b10 2023-07-06T03:51:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c6affd700 1 --2- 172.21.15.97:0/3783072197 >> v2:172.21.15.146:3300/0 conn(0x7f4c6c0b21e0 0x7f4c6c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c540035b0 con 0x7f4c6c160b10 2023-07-06T03:51:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3442186084 0 0) 0x7f4c540035b0 con 0x7f4c6c160b10 2023-07-06T03:51:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 >> v2:172.21.15.146:3300/0 conn(0x7f4c6c0b21e0 msgr2=0x7f4c6c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 --2- 172.21.15.97:0/3783072197 >> v2:172.21.15.146:3300/0 conn(0x7f4c6c0b21e0 0x7f4c6c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 >> v1:172.21.15.146:6789/0 conn(0x7f4c6c0b47c0 legacy=0x7f4c6c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c0d0f70 con 0x7f4c6c160b10 2023-07-06T03:51:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f4c60001670 con 0x7f4c6c160b10 2023-07-06T03:51:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 >> v1:172.21.15.97:6789/0 conn(0x7f4c6c160b10 legacy=0x7f4c6c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 --2- 172.21.15.97:0/3783072197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c54003a00 0x7f4c54003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 --2- 172.21.15.97:0/3783072197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54004850 0x7f4c54004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 --2- 172.21.15.97:0/3783072197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c54005200 0x7f4c5400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0d0c90 con 0x7f4c54004850 2023-07-06T03:51:07.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5400c060 con 0x7f4c54003a00 2023-07-06T03:51:07.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5400c220 con 0x7f4c54005200 2023-07-06T03:51:07.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.710+0000 7f4c6a7fc700 1 --2- 172.21.15.97:0/3783072197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54004850 0x7f4c54004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:07.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c6affd700 1 --2- 172.21.15.97:0/3783072197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c54003a00 0x7f4c54003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:07.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c6b7fe700 1 --2- 172.21.15.97:0/3783072197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c54005200 0x7f4c5400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:07.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c6a7fc700 1 -- 172.21.15.97:0/3783072197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c54005200 msgr2=0x7f4c5400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:07.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c6a7fc700 1 --2- 172.21.15.97:0/3783072197 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c54005200 0x7f4c5400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:07.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c6a7fc700 1 -- 172.21.15.97:0/3783072197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c54003a00 msgr2=0x7f4c54003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:07.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c6a7fc700 1 --2- 172.21.15.97:0/3783072197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c54003a00 0x7f4c54003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:07.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c6a7fc700 1 -- 172.21.15.97:0/3783072197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540035b0 con 0x7f4c54004850 2023-07-06T03:51:07.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c6b7fe700 1 -- 172.21.15.97:0/3783072197 reap_dead start 2023-07-06T03:51:07.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c6a7fc700 1 --2- 172.21.15.97:0/3783072197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54004850 0x7f4c54004cc0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c2a0 tx=0x7f4c54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6862c9d139d5fcbb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:07.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60007c10 con 0x7f4c54004850 2023-07-06T03:51:07.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6000cde0 con 0x7f4c54004850 2023-07-06T03:51:07.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.711+0000 7f4c69ffb700 1 -- 172.21.15.97:0/3783072197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60005ba0 con 0x7f4c54004850 2023-07-06T03:51:07.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.712+0000 7f4c717ff700 1 -- 172.21.15.97:0/3783072197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54004850 msgr2=0x7f4c54004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:07.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.712+0000 7f4c717ff700 1 --2- 172.21.15.97:0/3783072197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54004850 0x7f4c54004cc0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c2a0 tx=0x7f4c54005880 comp rx=0 tx=0).stop 2023-07-06T03:51:07.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.712+0000 7f4c717ff700 1 -- 172.21.15.97:0/3783072197 shutdown_connections 2023-07-06T03:51:07.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.712+0000 7f4c717ff700 1 --2- 172.21.15.97:0/3783072197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54004850 0x7f4c54004cc0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:07.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.712+0000 7f4c717ff700 1 -- 172.21.15.97:0/3783072197 >> 172.21.15.97:0/3783072197 conn(0x7f4c6c06b260 msgr2=0x7f4c6c0cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:07.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.712+0000 7f4c717ff700 1 -- 172.21.15.97:0/3783072197 shutdown_connections 2023-07-06T03:51:07.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.712+0000 7f4c717ff700 1 -- 172.21.15.97:0/3783072197 wait complete. 2023-07-06T03:51:07.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.714+0000 7f4c717ff700 1 Processor -- start 2023-07-06T03:51:07.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.714+0000 7f4c717ff700 1 -- start start 2023-07-06T03:51:07.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c717ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c54004850 0x7f4c6c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:07.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c717ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c6c0b6e70 0x7f4c6c1e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c717ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c6c0cc9a0 0x7f4c6c1e8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c717ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0cd130 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c717ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0b5ca0 con 0x7f4c6c0cc9a0 2023-07-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c717ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0cce50 con 0x7f4c54004850 2023-07-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c6a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c6c0b6e70 0x7f4c6c1e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c6a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c6c0b6e70 0x7f4c6c1e7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57170/0 (socket says 172.21.15.97:57170) 2023-07-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c6a7fc700 1 -- 172.21.15.97:0/25812665 learned_addr learned my addr 172.21.15.97:0/25812665 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c6affd700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c54004850 0x7f4c6c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c6b7fe700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c6c0cc9a0 0x7f4c6c1e8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c6a7fc700 1 -- 172.21.15.97:0/25812665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c54004850 msgr2=0x7f4c6c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c6a7fc700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c54004850 0x7f4c6c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c6a7fc700 1 -- 172.21.15.97:0/25812665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c6c0cc9a0 msgr2=0x7f4c6c1e8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c6a7fc700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c6c0cc9a0 0x7f4c6c1e8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c6a7fc700 1 -- 172.21.15.97:0/25812665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c0b5f40 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.715+0000 7f4c6affd700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c54004850 0x7f4c6c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:51:07.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.716+0000 7f4c6a7fc700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c6c0b6e70 0x7f4c6c1e7ff0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c030 tx=0x7f4c6c0cef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:07.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.716+0000 7f4c537fe700 1 -- 172.21.15.97:0/25812665 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60001670 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.716+0000 7f4c537fe700 1 -- 172.21.15.97:0/25812665 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c600070d0 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.716+0000 7f4c537fe700 1 -- 172.21.15.97:0/25812665 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60007460 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.716+0000 7f4c717ff700 1 -- 172.21.15.97:0/25812665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c0b6220 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.716+0000 7f4c717ff700 1 -- 172.21.15.97:0/25812665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6c0c9e10 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.717+0000 7f4c717ff700 1 -- 172.21.15.97:0/25812665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6c0b6590 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.718+0000 7f4c537fe700 1 -- 172.21.15.97:0/25812665 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c600075c0 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.719+0000 7f4c537fe700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c5806ace0 0x7f4c5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:07.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.720+0000 7f4c537fe700 1 -- 172.21.15.97:0/25812665 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c6008a760 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.720+0000 7f4c6affd700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c5806ace0 0x7f4c5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:07.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.720+0000 7f4c6affd700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c5806ace0 0x7f4c5806d190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0b1930 tx=0x7f4c5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:07.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.724+0000 7f4c537fe700 1 -- 172.21.15.97:0/25812665 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4c600558a0 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.963+0000 7f4c717ff700 1 -- 172.21.15.97:0/25812665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4c6c04e340 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.964+0000 7f4c537fe700 1 -- 172.21.15.97:0/25812665 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f4c60058ec0 con 0x7f4c6c0b6e70 2023-07-06T03:51:07.972 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:51:07.972 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:51:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 -- 172.21.15.97:0/25812665 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c5806ace0 msgr2=0x7f4c5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c5806ace0 0x7f4c5806d190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0b1930 tx=0x7f4c5c008040 comp rx=0 tx=0).stop 2023-07-06T03:51:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 -- 172.21.15.97:0/25812665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c6c0b6e70 msgr2=0x7f4c6c1e7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c6c0b6e70 0x7f4c6c1e7ff0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c030 tx=0x7f4c6c0cef80 comp rx=0 tx=0).stop 2023-07-06T03:51:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 -- 172.21.15.97:0/25812665 shutdown_connections 2023-07-06T03:51:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c5806ace0 0x7f4c5806d190 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c6c0cc9a0 0x7f4c6c1e8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c54004850 0x7f4c6c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 --2- 172.21.15.97:0/25812665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c6c0b6e70 0x7f4c6c1e7ff0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 -- 172.21.15.97:0/25812665 >> 172.21.15.97:0/25812665 conn(0x7f4c6c06b260 msgr2=0x7f4c6c0b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 -- 172.21.15.97:0/25812665 shutdown_connections 2023-07-06T03:51:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:07.966+0000 7f4c717ff700 1 -- 172.21.15.97:0/25812665 wait complete. 2023-07-06T03:51:07.987 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:51:07.987 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:51:12.988 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:51:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.190+0000 7f5d115ff700 1 Processor -- start 2023-07-06T03:51:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.190+0000 7f5d115ff700 1 -- start start 2023-07-06T03:51:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.192+0000 7f5d115ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c15c520 0x7f5d0c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.192+0000 7f5d115ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5d0c160d00 0x7f5d0c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.192+0000 7f5d115ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0c0cdae0 con 0x7f5d0c160180 2023-07-06T03:51:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.192+0000 7f5d115ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c0cd8d0 con 0x7f5d0c160d00 2023-07-06T03:51:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.192+0000 7f5d115ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0b6640 con 0x7f5d0c15c520 2023-07-06T03:51:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.192+0000 7f5d0b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5d0c160d00 0x7f5d0c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.192+0000 7f5d0b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5d0c160d00 0x7f5d0c0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54226/0 (socket says 172.21.15.97:54226) 2023-07-06T03:51:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.192+0000 7f5d0b7fe700 1 -- 172.21.15.97:0/383019713 learned_addr learned my addr 172.21.15.97:0/383019713 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:13.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.192+0000 7f5d0affd700 1 --2- 172.21.15.97:0/383019713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c15c520 0x7f5d0c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:13.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.193+0000 7f5d0b7fe700 1 -- 172.21.15.97:0/383019713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c15c520 msgr2=0x7f5d0c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:13.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.193+0000 7f5d0b7fe700 1 --2- 172.21.15.97:0/383019713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c15c520 0x7f5d0c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:13.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.193+0000 7f5d0b7fe700 1 -- 172.21.15.97:0/383019713 >> v1:172.21.15.146:6789/0 conn(0x7f5d0c160180 legacy=0x7f5d0c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.193+0000 7f5d0b7fe700 1 -- 172.21.15.97:0/383019713 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c0b6920 con 0x7f5d0c160d00 2023-07-06T03:51:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.193+0000 7f5d0b7fe700 1 --2- 172.21.15.97:0/383019713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c160d00 0x7f5d0c0b7400 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009e50 tx=0x7f5d0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fee287c19b8506 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.193+0000 7f5d09ffb700 1 -- 172.21.15.97:0/383019713 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400e070 con 0x7f5d0c160d00 2023-07-06T03:51:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.193+0000 7f5d09ffb700 1 -- 172.21.15.97:0/383019713 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0400b910 con 0x7f5d0c160d00 2023-07-06T03:51:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.193+0000 7f5d09ffb700 1 -- 172.21.15.97:0/383019713 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400bcc0 con 0x7f5d0c160d00 2023-07-06T03:51:13.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.194+0000 7f5d115ff700 1 -- 172.21.15.97:0/383019713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c160d00 msgr2=0x7f5d0c0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:13.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.194+0000 7f5d115ff700 1 --2- 172.21.15.97:0/383019713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c160d00 0x7f5d0c0b7400 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009e50 tx=0x7f5d0400b330 comp rx=0 tx=0).stop 2023-07-06T03:51:13.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.194+0000 7f5d115ff700 1 -- 172.21.15.97:0/383019713 shutdown_connections 2023-07-06T03:51:13.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.194+0000 7f5d115ff700 1 --2- 172.21.15.97:0/383019713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c160d00 0x7f5d0c0b7400 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:13.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.194+0000 7f5d115ff700 1 --2- 172.21.15.97:0/383019713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c15c520 0x7f5d0c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:13.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.194+0000 7f5d115ff700 1 -- 172.21.15.97:0/383019713 >> 172.21.15.97:0/383019713 conn(0x7f5d0c071790 msgr2=0x7f5d0c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:13.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.194+0000 7f5d115ff700 1 -- 172.21.15.97:0/383019713 shutdown_connections 2023-07-06T03:51:13.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.194+0000 7f5d115ff700 1 -- 172.21.15.97:0/383019713 wait complete. 2023-07-06T03:51:13.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.196+0000 7f5d115ff700 1 Processor -- start 2023-07-06T03:51:13.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.197+0000 7f5d115ff700 1 -- start start 2023-07-06T03:51:13.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d115ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c15c520 0x7f5d0c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:13.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d115ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c160180 0x7f5d0c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:13.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d115ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c160d00 0x7f5d0c1f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:13.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d115ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c157a60 con 0x7f5d0c160180 2023-07-06T03:51:13.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d115ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0ca710 con 0x7f5d0c15c520 2023-07-06T03:51:13.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d115ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0ca9f0 con 0x7f5d0c160d00 2023-07-06T03:51:13.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d0a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c160180 0x7f5d0c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:13.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d0a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c160180 0x7f5d0c0c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54234/0 (socket says 172.21.15.97:54234) 2023-07-06T03:51:13.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d0a7fc700 1 -- 172.21.15.97:0/2800055500 learned_addr learned my addr 172.21.15.97:0/2800055500 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:13.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d0affd700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c15c520 0x7f5d0c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:13.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d0b7fe700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c160d00 0x7f5d0c1f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:13.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d0a7fc700 1 -- 172.21.15.97:0/2800055500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c160d00 msgr2=0x7f5d0c1f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:13.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d0a7fc700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c160d00 0x7f5d0c1f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:13.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d0a7fc700 1 -- 172.21.15.97:0/2800055500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c15c520 msgr2=0x7f5d0c157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:13.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d0a7fc700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c15c520 0x7f5d0c157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:13.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d0a7fc700 1 -- 172.21.15.97:0/2800055500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d04009b30 con 0x7f5d0c160180 2023-07-06T03:51:13.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.198+0000 7f5d0b7fe700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c160d00 0x7f5d0c1f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:51:13.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.199+0000 7f5d0a7fc700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c160180 0x7f5d0c0c9e10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c4e0 tx=0x7f5d0000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:13.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.199+0000 7f5cf37fe700 1 -- 172.21.15.97:0/2800055500 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d000052d0 con 0x7f5d0c160180 2023-07-06T03:51:13.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.199+0000 7f5cf37fe700 1 -- 172.21.15.97:0/2800055500 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d00005430 con 0x7f5d0c160180 2023-07-06T03:51:13.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.199+0000 7f5d115ff700 1 -- 172.21.15.97:0/2800055500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c0cacd0 con 0x7f5d0c160180 2023-07-06T03:51:13.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.199+0000 7f5cf37fe700 1 -- 172.21.15.97:0/2800055500 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d000038c0 con 0x7f5d0c160180 2023-07-06T03:51:13.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.199+0000 7f5d115ff700 1 -- 172.21.15.97:0/2800055500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c0cb290 con 0x7f5d0c160180 2023-07-06T03:51:13.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.200+0000 7f5d115ff700 1 -- 172.21.15.97:0/2800055500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c0cb480 con 0x7f5d0c160180 2023-07-06T03:51:13.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.201+0000 7f5cf37fe700 1 -- 172.21.15.97:0/2800055500 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d00003a20 con 0x7f5d0c160180 2023-07-06T03:51:13.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.202+0000 7f5cf37fe700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 0x7f5cf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:13.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.202+0000 7f5cf37fe700 1 -- 172.21.15.97:0/2800055500 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d00088930 con 0x7f5d0c160180 2023-07-06T03:51:13.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.202+0000 7f5d0affd700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 0x7f5cf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:13.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.203+0000 7f5d0affd700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 0x7f5cf806d190 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0cd1b0 tx=0x7f5cfc009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:13.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.206+0000 7f5cf37fe700 1 -- 172.21.15.97:0/2800055500 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d000539d0 con 0x7f5d0c160180 2023-07-06T03:51:13.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.442+0000 7f5d115ff700 1 -- 172.21.15.97:0/2800055500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5d0c04e340 con 0x7f5d0c160180 2023-07-06T03:51:13.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.443+0000 7f5cf37fe700 1 -- 172.21.15.97:0/2800055500 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f5d00056ff0 con 0x7f5d0c160180 2023-07-06T03:51:13.446 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:51:13.446 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:51:13.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.445+0000 7f5d115ff700 1 -- 172.21.15.97:0/2800055500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 msgr2=0x7f5cf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:13.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.445+0000 7f5d115ff700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 0x7f5cf806d190 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0cd1b0 tx=0x7f5cfc009210 comp rx=0 tx=0).stop 2023-07-06T03:51:13.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.445+0000 7f5d115ff700 1 -- 172.21.15.97:0/2800055500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c160180 msgr2=0x7f5d0c0c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:13.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.445+0000 7f5d115ff700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c160180 0x7f5d0c0c9e10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c4e0 tx=0x7f5d0000c7f0 comp rx=0 tx=0).stop 2023-07-06T03:51:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.445+0000 7f5d115ff700 1 -- 172.21.15.97:0/2800055500 shutdown_connections 2023-07-06T03:51:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.445+0000 7f5d115ff700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 0x7f5cf806d190 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.445+0000 7f5d115ff700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c160d00 0x7f5d0c1f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.445+0000 7f5d115ff700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c15c520 0x7f5d0c157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.445+0000 7f5d115ff700 1 --2- 172.21.15.97:0/2800055500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c160180 0x7f5d0c0c9e10 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.445+0000 7f5d115ff700 1 -- 172.21.15.97:0/2800055500 >> 172.21.15.97:0/2800055500 conn(0x7f5d0c071790 msgr2=0x7f5d0c0b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.446+0000 7f5d115ff700 1 -- 172.21.15.97:0/2800055500 shutdown_connections 2023-07-06T03:51:13.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:13.446+0000 7f5d115ff700 1 -- 172.21.15.97:0/2800055500 wait complete. 2023-07-06T03:51:13.465 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:51:13.466 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:51:18.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:51:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.668+0000 7f3ea91ff700 1 Processor -- start 2023-07-06T03:51:18.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.668+0000 7f3ea91ff700 1 -- start start 2023-07-06T03:51:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.669+0000 7f3ea91ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ea40b0c70 0x7f3ea4163fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:18.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.669+0000 7f3ea91ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea415cc40 con 0x7f3ea41592f0 2023-07-06T03:51:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.669+0000 7f3ea91ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea415cf20 con 0x7f3ea4158f20 2023-07-06T03:51:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.669+0000 7f3ea91ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea415d200 con 0x7f3ea40b0c70 2023-07-06T03:51:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.670+0000 7f3ea259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3ea41592f0 0x7f3ea40b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55076/0 (socket says 172.21.15.97:55076) 2023-07-06T03:51:18.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.670+0000 7f3ea259c700 1 -- 172.21.15.97:0/3661405334 learned_addr learned my addr 172.21.15.97:0/3661405334 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:51:18.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.670+0000 7f3ea359e700 1 --2- 172.21.15.97:0/3661405334 >> v2:172.21.15.97:3300/0 conn(0x7f3ea40b0c70 0x7f3ea4163fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:18.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.670+0000 7f3ea359e700 1 -- 172.21.15.97:0/3661405334 >> v1:172.21.15.146:6789/0 conn(0x7f3ea4158f20 legacy=0x7f3ea40caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.670+0000 7f3ea1d9b700 1 -- 172.21.15.97:0/3661405334 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491160222 0 0) 0x7f3ea415cc40 con 0x7f3ea41592f0 2023-07-06T03:51:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.670+0000 7f3ea359e700 1 -- 172.21.15.97:0/3661405334 >> v1:172.21.15.97:6789/0 conn(0x7f3ea41592f0 legacy=0x7f3ea40b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.670+0000 7f3ea359e700 1 -- 172.21.15.97:0/3661405334 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea415d4e0 con 0x7f3ea40b0c70 2023-07-06T03:51:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.670+0000 7f3ea359e700 1 --2- 172.21.15.97:0/3661405334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea40b0c70 0x7f3ea4163fd0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009d90 tx=0x7f3e9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b49c0ba40f29a0f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.671+0000 7f3ea1d9b700 1 -- 172.21.15.97:0/3661405334 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00e070 con 0x7f3ea40b0c70 2023-07-06T03:51:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.671+0000 7f3ea1d9b700 1 -- 172.21.15.97:0/3661405334 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c00bbd0 con 0x7f3ea40b0c70 2023-07-06T03:51:18.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.671+0000 7f3ea1d9b700 1 -- 172.21.15.97:0/3661405334 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c012430 con 0x7f3ea40b0c70 2023-07-06T03:51:18.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.671+0000 7f3ea91ff700 1 -- 172.21.15.97:0/3661405334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea40b0c70 msgr2=0x7f3ea4163fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:18.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.671+0000 7f3ea91ff700 1 --2- 172.21.15.97:0/3661405334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea40b0c70 0x7f3ea4163fd0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009d90 tx=0x7f3e9c00b330 comp rx=0 tx=0).stop 2023-07-06T03:51:18.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.671+0000 7f3ea91ff700 1 -- 172.21.15.97:0/3661405334 shutdown_connections 2023-07-06T03:51:18.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.671+0000 7f3ea91ff700 1 --2- 172.21.15.97:0/3661405334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea40b0c70 0x7f3ea4163fd0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:18.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.671+0000 7f3ea91ff700 1 -- 172.21.15.97:0/3661405334 >> 172.21.15.97:0/3661405334 conn(0x7f3ea40ce8c0 msgr2=0x7f3ea40cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:18.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.671+0000 7f3ea91ff700 1 -- 172.21.15.97:0/3661405334 shutdown_connections 2023-07-06T03:51:18.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.671+0000 7f3ea91ff700 1 -- 172.21.15.97:0/3661405334 wait complete. 2023-07-06T03:51:18.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.673+0000 7f3ea91ff700 1 Processor -- start 2023-07-06T03:51:18.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.673+0000 7f3ea91ff700 1 -- start start 2023-07-06T03:51:18.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.674+0000 7f3ea91ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea40b0c70 0x7f3ea40b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:18.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.674+0000 7f3ea91ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea4158f20 0x7f3ea40ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:18.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.674+0000 7f3ea91ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ea41592f0 0x7f3ea40acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:18.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.674+0000 7f3ea91ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea40b4a20 con 0x7f3ea40b0c70 2023-07-06T03:51:18.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.674+0000 7f3ea91ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea41596e0 con 0x7f3ea4158f20 2023-07-06T03:51:18.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.674+0000 7f3ea91ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea413d790 con 0x7f3ea41592f0 2023-07-06T03:51:18.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.674+0000 7f3ea2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea40b0c70 0x7f3ea40b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:18.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.674+0000 7f3ea2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea40b0c70 0x7f3ea40b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54250/0 (socket says 172.21.15.97:54250) 2023-07-06T03:51:18.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.674+0000 7f3ea2d9d700 1 -- 172.21.15.97:0/35260364 learned_addr learned my addr 172.21.15.97:0/35260364 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:18.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.674+0000 7f3ea259c700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea4158f20 0x7f3ea40ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:18.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.674+0000 7f3ea359e700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ea41592f0 0x7f3ea40acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:18.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.675+0000 7f3ea2d9d700 1 -- 172.21.15.97:0/35260364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ea41592f0 msgr2=0x7f3ea40acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:18.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.675+0000 7f3ea2d9d700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ea41592f0 0x7f3ea40acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:18.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.675+0000 7f3ea2d9d700 1 -- 172.21.15.97:0/35260364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea4158f20 msgr2=0x7f3ea40ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:18.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.675+0000 7f3ea2d9d700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea4158f20 0x7f3ea40ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:18.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.675+0000 7f3ea2d9d700 1 -- 172.21.15.97:0/35260364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c009a70 con 0x7f3ea40b0c70 2023-07-06T03:51:18.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.675+0000 7f3ea2d9d700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea40b0c70 0x7f3ea40b44e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c4e0 tx=0x7f3e9400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:18.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.675+0000 7f3e8b7fe700 1 -- 172.21.15.97:0/35260364 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940052d0 con 0x7f3ea40b0c70 2023-07-06T03:51:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.675+0000 7f3e8b7fe700 1 -- 172.21.15.97:0/35260364 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e94005430 con 0x7f3ea40b0c70 2023-07-06T03:51:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.675+0000 7f3ea91ff700 1 -- 172.21.15.97:0/35260364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea413da70 con 0x7f3ea40b0c70 2023-07-06T03:51:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.675+0000 7f3e8b7fe700 1 -- 172.21.15.97:0/35260364 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940038c0 con 0x7f3ea40b0c70 2023-07-06T03:51:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.675+0000 7f3ea91ff700 1 -- 172.21.15.97:0/35260364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea4159c10 con 0x7f3ea40b0c70 2023-07-06T03:51:18.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.676+0000 7f3ea91ff700 1 -- 172.21.15.97:0/35260364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea404e340 con 0x7f3ea40b0c70 2023-07-06T03:51:18.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.677+0000 7f3e8b7fe700 1 -- 172.21.15.97:0/35260364 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e94003a20 con 0x7f3ea40b0c70 2023-07-06T03:51:18.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.679+0000 7f3e8b7fe700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e9006ace0 0x7f3e9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:18.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.679+0000 7f3e8b7fe700 1 -- 172.21.15.97:0/35260364 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e940889b0 con 0x7f3ea40b0c70 2023-07-06T03:51:18.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.679+0000 7f3ea259c700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e9006ace0 0x7f3e9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:18.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.679+0000 7f3ea259c700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e9006ace0 0x7f3e9006d190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3e98000f60 tx=0x7f3e98007d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:18.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.683+0000 7f3e8b7fe700 1 -- 172.21.15.97:0/35260364 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3e94053a50 con 0x7f3ea40b0c70 2023-07-06T03:51:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.927+0000 7f3ea91ff700 1 -- 172.21.15.97:0/35260364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3ea4159dd0 con 0x7f3ea40b0c70 2023-07-06T03:51:18.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.927+0000 7f3e8b7fe700 1 -- 172.21.15.97:0/35260364 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f3e94057070 con 0x7f3ea40b0c70 2023-07-06T03:51:18.929 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:51:18.929 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:51:18.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 -- 172.21.15.97:0/35260364 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e9006ace0 msgr2=0x7f3e9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:18.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e9006ace0 0x7f3e9006d190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3e98000f60 tx=0x7f3e98007d90 comp rx=0 tx=0).stop 2023-07-06T03:51:18.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 -- 172.21.15.97:0/35260364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea40b0c70 msgr2=0x7f3ea40b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:18.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea40b0c70 0x7f3ea40b44e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c4e0 tx=0x7f3e9400c8a0 comp rx=0 tx=0).stop 2023-07-06T03:51:18.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 -- 172.21.15.97:0/35260364 shutdown_connections 2023-07-06T03:51:18.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e9006ace0 0x7f3e9006d190 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:18.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ea41592f0 0x7f3ea40acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:18.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ea40b0c70 0x7f3ea40b44e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:18.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 --2- 172.21.15.97:0/35260364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ea4158f20 0x7f3ea40ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:18.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 -- 172.21.15.97:0/35260364 >> 172.21.15.97:0/35260364 conn(0x7f3ea40ce8c0 msgr2=0x7f3ea41547c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:18.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 -- 172.21.15.97:0/35260364 shutdown_connections 2023-07-06T03:51:18.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:18.929+0000 7f3ea91ff700 1 -- 172.21.15.97:0/35260364 wait complete. 2023-07-06T03:51:18.948 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:51:18.948 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:51:23.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:51:24.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.151+0000 7fa92abff700 1 Processor -- start 2023-07-06T03:51:24.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.151+0000 7fa92abff700 1 -- start start 2023-07-06T03:51:24.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.152+0000 7fa92abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92406d9d0 0x7fa92406ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:24.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.152+0000 7fa92abff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa924160fa0 con 0x7fa9240ae390 2023-07-06T03:51:24.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.152+0000 7fa92abff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92406e330 con 0x7fa92406e940 2023-07-06T03:51:24.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.152+0000 7fa92abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9241607f0 con 0x7fa92406d9d0 2023-07-06T03:51:24.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.152+0000 7fa92899b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa9240ae390 0x7fa92415d490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41044/0 (socket says 172.21.15.97:41044) 2023-07-06T03:51:24.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.152+0000 7fa92899b700 1 -- 172.21.15.97:0/3323392634 learned_addr learned my addr 172.21.15.97:0/3323392634 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:51:24.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.152+0000 7fa92919c700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92406d9d0 0x7fa92406ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:24.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433371634 0 0) 0x7fa924160fa0 con 0x7fa9240ae390 2023-07-06T03:51:24.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9140035b0 con 0x7fa9240ae390 2023-07-06T03:51:24.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488634701 0 0) 0x7fa92406e330 con 0x7fa92406e940 2023-07-06T03:51:24.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa924160fa0 con 0x7fa92406e940 2023-07-06T03:51:24.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 658566487 0 0) 0x7fa9140035b0 con 0x7fa9240ae390 2023-07-06T03:51:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92406d9d0 msgr2=0x7fa92406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92406d9d0 0x7fa92406ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 >> v1:172.21.15.146:6789/0 conn(0x7fa92406e940 legacy=0x7fa9241600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9240ae760 con 0x7fa9240ae390 2023-07-06T03:51:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fa918001670 con 0x7fa9240ae390 2023-07-06T03:51:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 >> v1:172.21.15.97:6789/0 conn(0x7fa9240ae390 legacy=0x7fa92415d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa914003bc0 0x7fa914003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:24.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa914004850 0x7fa914004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:24.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92406d9d0 0x7fa914005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:24.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91400bac0 con 0x7fa914003bc0 2023-07-06T03:51:24.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91400bc00 con 0x7fa914004850 2023-07-06T03:51:24.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91400bd70 con 0x7fa92406d9d0 2023-07-06T03:51:24.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa92899b700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa914003bc0 0x7fa914003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:24.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.153+0000 7fa923fff700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa914004850 0x7fa914004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:24.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.154+0000 7fa92919c700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92406d9d0 0x7fa914005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:24.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.154+0000 7fa92899b700 1 -- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92406d9d0 msgr2=0x7fa914005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:24.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.154+0000 7fa92899b700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa92406d9d0 0x7fa914005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:24.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.154+0000 7fa92899b700 1 -- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa914004850 msgr2=0x7fa914004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:24.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.154+0000 7fa92899b700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa914004850 0x7fa914004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:24.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.154+0000 7fa92899b700 1 -- 172.21.15.97:0/3323392634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140035b0 con 0x7fa914003bc0 2023-07-06T03:51:24.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.154+0000 7fa923fff700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa914004850 0x7fa914004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:51:24.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.154+0000 7fa92899b700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa914003bc0 0x7fa914003fb0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa91800c210 tx=0x7fa914005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9e82cc6cc44ae6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:24.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.154+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91800cdb0 con 0x7fa914003bc0 2023-07-06T03:51:24.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.154+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa918001af0 con 0x7fa914003bc0 2023-07-06T03:51:24.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.154+0000 7fa9237fe700 1 -- 172.21.15.97:0/3323392634 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918005ba0 con 0x7fa914003bc0 2023-07-06T03:51:24.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.155+0000 7fa92abff700 1 -- 172.21.15.97:0/3323392634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa914003bc0 msgr2=0x7fa914003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:24.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.155+0000 7fa92abff700 1 --2- 172.21.15.97:0/3323392634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa914003bc0 0x7fa914003fb0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa91800c210 tx=0x7fa914005880 comp rx=0 tx=0).stop 2023-07-06T03:51:24.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.155+0000 7fa92919c700 1 -- 172.21.15.97:0/3323392634 reap_dead start 2023-07-06T03:51:24.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.155+0000 7fa92abff700 1 -- 172.21.15.97:0/3323392634 shutdown_connections 2023-07-06T03:51:24.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.155+0000 7fa92abff700 1 -- 172.21.15.97:0/3323392634 >> 172.21.15.97:0/3323392634 conn(0x7fa9240cbc70 msgr2=0x7fa9240cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:24.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.155+0000 7fa92abff700 1 -- 172.21.15.97:0/3323392634 shutdown_connections 2023-07-06T03:51:24.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.155+0000 7fa92abff700 1 -- 172.21.15.97:0/3323392634 wait complete. 2023-07-06T03:51:24.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.157+0000 7fa92abff700 1 Processor -- start 2023-07-06T03:51:24.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.157+0000 7fa92abff700 1 -- start start 2023-07-06T03:51:24.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa92abff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa914003bc0 0x7fa924155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:24.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa92abff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9240b2160 0x7fa9240b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:24.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa92abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9240b4db0 0x7fa9240c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:24.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa92abff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9241f1f00 con 0x7fa9240b2160 2023-07-06T03:51:24.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa92abff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9241f21e0 con 0x7fa914003bc0 2023-07-06T03:51:24.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa92abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9241f24c0 con 0x7fa9240b4db0 2023-07-06T03:51:24.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa923fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9240b2160 0x7fa9240b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:24.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa923fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9240b2160 0x7fa9240b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58542/0 (socket says 172.21.15.97:58542) 2023-07-06T03:51:24.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa923fff700 1 -- 172.21.15.97:0/1552119045 learned_addr learned my addr 172.21.15.97:0/1552119045 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:24.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa92919c700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9240b4db0 0x7fa9240c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:24.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa92899b700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa914003bc0 0x7fa924155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:24.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa923fff700 1 -- 172.21.15.97:0/1552119045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9240b4db0 msgr2=0x7fa9240c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:24.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa923fff700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9240b4db0 0x7fa9240c5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:24.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa923fff700 1 -- 172.21.15.97:0/1552119045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa914003bc0 msgr2=0x7fa924155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:24.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa923fff700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa914003bc0 0x7fa924155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:24.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.158+0000 7fa923fff700 1 -- 172.21.15.97:0/1552119045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9241f27a0 con 0x7fa9240b2160 2023-07-06T03:51:24.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.159+0000 7fa923fff700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9240b2160 0x7fa9240b4870 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa91000e580 tx=0x7fa91000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:24.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.159+0000 7fa9217fa700 1 -- 172.21.15.97:0/1552119045 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100052e0 con 0x7fa9240b2160 2023-07-06T03:51:24.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.159+0000 7fa9217fa700 1 -- 172.21.15.97:0/1552119045 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910005440 con 0x7fa9240b2160 2023-07-06T03:51:24.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.159+0000 7fa92abff700 1 -- 172.21.15.97:0/1552119045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9241f2a80 con 0x7fa9240b2160 2023-07-06T03:51:24.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.159+0000 7fa9217fa700 1 -- 172.21.15.97:0/1552119045 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910006a60 con 0x7fa9240b2160 2023-07-06T03:51:24.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.159+0000 7fa92abff700 1 -- 172.21.15.97:0/1552119045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9241561c0 con 0x7fa9240b2160 2023-07-06T03:51:24.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.160+0000 7fa92abff700 1 -- 172.21.15.97:0/1552119045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9241f2cc0 con 0x7fa9240b2160 2023-07-06T03:51:24.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.161+0000 7fa9217fa700 1 -- 172.21.15.97:0/1552119045 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa910006bc0 con 0x7fa9240b2160 2023-07-06T03:51:24.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.162+0000 7fa9217fa700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa90806ace0 0x7fa90806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:24.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.162+0000 7fa9217fa700 1 -- 172.21.15.97:0/1552119045 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa910088770 con 0x7fa9240b2160 2023-07-06T03:51:24.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.162+0000 7fa92899b700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa90806ace0 0x7fa90806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:24.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.163+0000 7fa92899b700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa90806ace0 0x7fa90806d190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa9240ad3b0 tx=0x7fa91800a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:24.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.166+0000 7fa9217fa700 1 -- 172.21.15.97:0/1552119045 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa9241f2cc0 con 0x7fa9240b2160 2023-07-06T03:51:24.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.402+0000 7fa92abff700 1 -- 172.21.15.97:0/1552119045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa9241f2cc0 con 0x7fa9240b2160 2023-07-06T03:51:24.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.402+0000 7fa9217fa700 1 -- 172.21.15.97:0/1552119045 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7fa9241f2cc0 con 0x7fa9240b2160 2023-07-06T03:51:24.404 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:51:24.405 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:51:24.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.404+0000 7fa92abff700 1 -- 172.21.15.97:0/1552119045 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa90806ace0 msgr2=0x7fa90806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:24.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.404+0000 7fa92abff700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa90806ace0 0x7fa90806d190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa9240ad3b0 tx=0x7fa91800a000 comp rx=0 tx=0).stop 2023-07-06T03:51:24.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.404+0000 7fa92abff700 1 -- 172.21.15.97:0/1552119045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9240b2160 msgr2=0x7fa9240b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:24.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.404+0000 7fa92abff700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9240b2160 0x7fa9240b4870 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa91000e580 tx=0x7fa91000e940 comp rx=0 tx=0).stop 2023-07-06T03:51:24.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.405+0000 7fa92abff700 1 -- 172.21.15.97:0/1552119045 shutdown_connections 2023-07-06T03:51:24.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.405+0000 7fa92abff700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa90806ace0 0x7fa90806d190 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:24.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.405+0000 7fa92abff700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9240b4db0 0x7fa9240c5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:24.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.405+0000 7fa92abff700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa914003bc0 0x7fa924155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:24.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.405+0000 7fa92abff700 1 --2- 172.21.15.97:0/1552119045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9240b2160 0x7fa9240b4870 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:24.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.405+0000 7fa92abff700 1 -- 172.21.15.97:0/1552119045 >> 172.21.15.97:0/1552119045 conn(0x7fa9240cbc70 msgr2=0x7fa92415d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:24.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.405+0000 7fa92abff700 1 -- 172.21.15.97:0/1552119045 shutdown_connections 2023-07-06T03:51:24.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:24.405+0000 7fa92abff700 1 -- 172.21.15.97:0/1552119045 wait complete. 2023-07-06T03:51:24.424 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:51:24.424 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:51:29.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:51:29.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.626+0000 7f49e45ff700 1 Processor -- start 2023-07-06T03:51:29.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.626+0000 7f49e45ff700 1 -- start start 2023-07-06T03:51:29.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.627+0000 7f49e45ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f49dc0af9f0 0x7f49dc0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:29.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.627+0000 7f49e45ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49dc0d1950 con 0x7f49dc0afdc0 2023-07-06T03:51:29.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e45ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49dc0d10b0 con 0x7f49dc15e5e0 2023-07-06T03:51:29.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e45ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc0cc490 con 0x7f49dc0af9f0 2023-07-06T03:51:29.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e1b9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f49dc0afdc0 0x7f49dc0d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41050/0 (socket says 172.21.15.97:41050) 2023-07-06T03:51:29.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e1b9a700 1 -- 172.21.15.97:0/344680544 learned_addr learned my addr 172.21.15.97:0/344680544 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:51:29.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e239b700 1 --2- 172.21.15.97:0/344680544 >> v2:172.21.15.146:3300/0 conn(0x7f49dc0af9f0 0x7f49dc0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:29.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686417523 0 0) 0x7f49dc0d1950 con 0x7f49dc0afdc0 2023-07-06T03:51:29.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49c80035b0 con 0x7f49dc0afdc0 2023-07-06T03:51:29.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3901071631 0 0) 0x7f49c80035b0 con 0x7f49dc0afdc0 2023-07-06T03:51:29.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 >> v2:172.21.15.146:3300/0 conn(0x7f49dc0af9f0 msgr2=0x7f49dc0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:29.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e1399700 1 --2- 172.21.15.97:0/344680544 >> v2:172.21.15.146:3300/0 conn(0x7f49dc0af9f0 0x7f49dc0b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:29.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 >> v1:172.21.15.146:6789/0 conn(0x7f49dc15e5e0 legacy=0x7f49dc15ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc0d1390 con 0x7f49dc0afdc0 2023-07-06T03:51:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.628+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f49cc001670 con 0x7f49dc0afdc0 2023-07-06T03:51:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 >> v1:172.21.15.97:6789/0 conn(0x7f49dc0afdc0 legacy=0x7f49dc0d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e1399700 1 --2- 172.21.15.97:0/344680544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49c8003a00 0x7f49c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e1399700 1 --2- 172.21.15.97:0/344680544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49c8004850 0x7f49c8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e1399700 1 --2- 172.21.15.97:0/344680544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c8005200 0x7f49c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0cc490 con 0x7f49c8005200 2023-07-06T03:51:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c800c110 con 0x7f49c8003a00 2023-07-06T03:51:29.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c800c2d0 con 0x7f49c8004850 2023-07-06T03:51:29.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e2b9c700 1 --2- 172.21.15.97:0/344680544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c8005200 0x7f49c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:29.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e239b700 1 --2- 172.21.15.97:0/344680544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49c8003a00 0x7f49c8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:29.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e1b9a700 1 --2- 172.21.15.97:0/344680544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49c8004850 0x7f49c8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:29.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e2b9c700 1 -- 172.21.15.97:0/344680544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49c8004850 msgr2=0x7f49c8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:29.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e2b9c700 1 --2- 172.21.15.97:0/344680544 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49c8004850 0x7f49c8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:29.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e2b9c700 1 -- 172.21.15.97:0/344680544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49c8003a00 msgr2=0x7f49c8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e2b9c700 1 --2- 172.21.15.97:0/344680544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49c8003a00 0x7f49c8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.629+0000 7f49e2b9c700 1 -- 172.21.15.97:0/344680544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c80035b0 con 0x7f49c8005200 2023-07-06T03:51:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.630+0000 7f49e2b9c700 1 -- 172.21.15.97:0/344680544 reap_dead start 2023-07-06T03:51:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.630+0000 7f49e2b9c700 1 --2- 172.21.15.97:0/344680544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c8005200 0x7f49c800ba10 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c190 tx=0x7f49c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752fa1683f5ee228 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.630+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4001db0 con 0x7f49c8005200 2023-07-06T03:51:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.630+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d400ccd0 con 0x7f49c8005200 2023-07-06T03:51:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.630+0000 7f49e1399700 1 -- 172.21.15.97:0/344680544 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4003c20 con 0x7f49c8005200 2023-07-06T03:51:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.630+0000 7f49e45ff700 1 -- 172.21.15.97:0/344680544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c8005200 msgr2=0x7f49c800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.630+0000 7f49e45ff700 1 --2- 172.21.15.97:0/344680544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c8005200 0x7f49c800ba10 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c190 tx=0x7f49c8005880 comp rx=0 tx=0).stop 2023-07-06T03:51:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.630+0000 7f49e45ff700 1 -- 172.21.15.97:0/344680544 shutdown_connections 2023-07-06T03:51:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.630+0000 7f49e45ff700 1 --2- 172.21.15.97:0/344680544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c8005200 0x7f49c800ba10 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.630+0000 7f49e45ff700 1 -- 172.21.15.97:0/344680544 >> 172.21.15.97:0/344680544 conn(0x7f49dc06ace0 msgr2=0x7f49dc06dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.630+0000 7f49e45ff700 1 -- 172.21.15.97:0/344680544 shutdown_connections 2023-07-06T03:51:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.631+0000 7f49e45ff700 1 -- 172.21.15.97:0/344680544 wait complete. 2023-07-06T03:51:29.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.632+0000 7f49e45ff700 1 Processor -- start 2023-07-06T03:51:29.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.632+0000 7f49e45ff700 1 -- start start 2023-07-06T03:51:29.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.633+0000 7f49e45ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc14bd80 0x7f49dc0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.633+0000 7f49e45ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49dc0b1ab0 0x7f49dc0c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.633+0000 7f49e45ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49dc0c5b80 0x7f49dc1f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.633+0000 7f49e45ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0b1f20 con 0x7f49dc14bd80 2023-07-06T03:51:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.633+0000 7f49e45ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc14c150 con 0x7f49dc0c5b80 2023-07-06T03:51:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.633+0000 7f49e45ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0cc950 con 0x7f49dc0b1ab0 2023-07-06T03:51:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49e239b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc14bd80 0x7f49dc0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49e239b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc14bd80 0x7f49dc0b1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58562/0 (socket says 172.21.15.97:58562) 2023-07-06T03:51:29.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49e239b700 1 -- 172.21.15.97:0/4230215077 learned_addr learned my addr 172.21.15.97:0/4230215077 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:29.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49e239b700 1 -- 172.21.15.97:0/4230215077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49dc0b1ab0 msgr2=0x7f49dc0c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:29.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49e239b700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49dc0b1ab0 0x7f49dc0c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:29.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49e239b700 1 -- 172.21.15.97:0/4230215077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49dc0c5b80 msgr2=0x7f49dc1f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:29.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49e2b9c700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49dc0c5b80 0x7f49dc1f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:29.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49e239b700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49dc0c5b80 0x7f49dc1f3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:29.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49e239b700 1 -- 172.21.15.97:0/4230215077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc0ccc30 con 0x7f49dc14bd80 2023-07-06T03:51:29.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49e2b9c700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49dc0c5b80 0x7f49dc1f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:51:29.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49e239b700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc14bd80 0x7f49dc0b1570 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f49d80119a0 tx=0x7f49d8011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:29.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.634+0000 7f49d2ffd700 1 -- 172.21.15.97:0/4230215077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d800f940 con 0x7f49dc14bd80 2023-07-06T03:51:29.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.635+0000 7f49d2ffd700 1 -- 172.21.15.97:0/4230215077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d800faa0 con 0x7f49dc14bd80 2023-07-06T03:51:29.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.635+0000 7f49d2ffd700 1 -- 172.21.15.97:0/4230215077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d800fe30 con 0x7f49dc14bd80 2023-07-06T03:51:29.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.635+0000 7f49e45ff700 1 -- 172.21.15.97:0/4230215077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc0ccf10 con 0x7f49dc14bd80 2023-07-06T03:51:29.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.635+0000 7f49e45ff700 1 -- 172.21.15.97:0/4230215077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc0cd4d0 con 0x7f49dc14bd80 2023-07-06T03:51:29.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.636+0000 7f49e45ff700 1 -- 172.21.15.97:0/4230215077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc066a40 con 0x7f49dc14bd80 2023-07-06T03:51:29.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.637+0000 7f49d2ffd700 1 -- 172.21.15.97:0/4230215077 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f49d8014440 con 0x7f49dc14bd80 2023-07-06T03:51:29.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.638+0000 7f49d2ffd700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49c006ace0 0x7f49c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:29.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.638+0000 7f49d2ffd700 1 -- 172.21.15.97:0/4230215077 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f49d8089c30 con 0x7f49dc14bd80 2023-07-06T03:51:29.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.638+0000 7f49e1b9a700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49c006ace0 0x7f49c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:29.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.638+0000 7f49e1b9a700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49c006ace0 0x7f49c006d190 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f49cc000f60 tx=0x7f49cc000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:29.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.642+0000 7f49d2ffd700 1 -- 172.21.15.97:0/4230215077 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f49d80586d0 con 0x7f49dc14bd80 2023-07-06T03:51:29.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.879+0000 7f49e45ff700 1 -- 172.21.15.97:0/4230215077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f49dc04e340 con 0x7f49dc14bd80 2023-07-06T03:51:29.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.880+0000 7f49d2ffd700 1 -- 172.21.15.97:0/4230215077 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f49d8022020 con 0x7f49dc14bd80 2023-07-06T03:51:29.882 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:51:29.882 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:51:29.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 -- 172.21.15.97:0/4230215077 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49c006ace0 msgr2=0x7f49c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:29.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49c006ace0 0x7f49c006d190 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f49cc000f60 tx=0x7f49cc000ed0 comp rx=0 tx=0).stop 2023-07-06T03:51:29.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 -- 172.21.15.97:0/4230215077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc14bd80 msgr2=0x7f49dc0b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:29.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc14bd80 0x7f49dc0b1570 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f49d80119a0 tx=0x7f49d8011d60 comp rx=0 tx=0).stop 2023-07-06T03:51:29.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 -- 172.21.15.97:0/4230215077 shutdown_connections 2023-07-06T03:51:29.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f49c006ace0 0x7f49c006d190 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:29.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49dc0c5b80 0x7f49dc1f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:29.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc14bd80 0x7f49dc0b1570 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:29.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 --2- 172.21.15.97:0/4230215077 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f49dc0b1ab0 0x7f49dc0c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:29.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 -- 172.21.15.97:0/4230215077 >> 172.21.15.97:0/4230215077 conn(0x7f49c8005200 msgr2=0x7f49dc0b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:29.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 -- 172.21.15.97:0/4230215077 shutdown_connections 2023-07-06T03:51:29.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:29.882+0000 7f49e45ff700 1 -- 172.21.15.97:0/4230215077 wait complete. 2023-07-06T03:51:29.901 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:51:29.902 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:51:34.902 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:51:34.927 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:51:34.953 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:51:35.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.109+0000 7efef53ff700 1 Processor -- start 2023-07-06T03:51:35.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.110+0000 7efef53ff700 1 -- start start 2023-07-06T03:51:35.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.111+0000 7efef53ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef015c520 0x7efef00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:35.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.111+0000 7efef53ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efef0160d00 0x7efef00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:35.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.111+0000 7efef53ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef00cdae0 con 0x7efef0160180 2023-07-06T03:51:35.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.111+0000 7efef53ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef00cd8d0 con 0x7efef0160d00 2023-07-06T03:51:35.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.111+0000 7efef53ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef00b6640 con 0x7efef015c520 2023-07-06T03:51:35.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.111+0000 7efeeeffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef015c520 0x7efef00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:35.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.111+0000 7efeef7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efef0160d00 0x7efef00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:35.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.111+0000 7efeee7fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7efef0160180 0x7efef01605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:45654/0 (socket says 172.21.15.97:45654) 2023-07-06T03:51:35.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.111+0000 7efeee7fc700 1 -- 172.21.15.97:0/1134032225 learned_addr learned my addr 172.21.15.97:0/1134032225 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:51:35.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.111+0000 7efeeeffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef015c520 0x7efef00cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:55736/0 (socket says 172.21.15.97:55736) 2023-07-06T03:51:35.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.112+0000 7efeedffb700 1 -- 172.21.15.97:0/1134032225 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535257948 0 0) 0x7efef00cdae0 con 0x7efef0160180 2023-07-06T03:51:35.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.112+0000 7efeedffb700 1 -- 172.21.15.97:0/1134032225 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efed80035b0 con 0x7efef0160180 2023-07-06T03:51:35.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.112+0000 7efeef7fe700 1 -- 172.21.15.97:0/1134032225 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef015c520 msgr2=0x7efef00cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:35.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.112+0000 7efeef7fe700 1 --2- 172.21.15.97:0/1134032225 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef015c520 0x7efef00cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:35.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.112+0000 7efeef7fe700 1 -- 172.21.15.97:0/1134032225 >> v1:172.21.15.146:6789/0 conn(0x7efef0160180 legacy=0x7efef01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:35.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.112+0000 7efeef7fe700 1 -- 172.21.15.97:0/1134032225 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef00b6920 con 0x7efef0160d00 2023-07-06T03:51:35.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.112+0000 7efeeeffd700 1 --2- 172.21.15.97:0/1134032225 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef015c520 0x7efef00cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:51:35.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.112+0000 7efeef7fe700 1 --2- 172.21.15.97:0/1134032225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efef0160d00 0x7efef00b7400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efee8009d90 tx=0x7efee800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbfc9e3f5c03aeff server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:35.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.112+0000 7efeedffb700 1 -- 172.21.15.97:0/1134032225 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee800e070 con 0x7efef0160d00 2023-07-06T03:51:35.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.113+0000 7efeedffb700 1 -- 172.21.15.97:0/1134032225 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee800bb20 con 0x7efef0160d00 2023-07-06T03:51:35.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.113+0000 7efeedffb700 1 -- 172.21.15.97:0/1134032225 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8012430 con 0x7efef0160d00 2023-07-06T03:51:35.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.113+0000 7efef53ff700 1 -- 172.21.15.97:0/1134032225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efef0160d00 msgr2=0x7efef00b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:35.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.113+0000 7efef53ff700 1 --2- 172.21.15.97:0/1134032225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efef0160d00 0x7efef00b7400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efee8009d90 tx=0x7efee800b330 comp rx=0 tx=0).stop 2023-07-06T03:51:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.113+0000 7efef53ff700 1 -- 172.21.15.97:0/1134032225 shutdown_connections 2023-07-06T03:51:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.113+0000 7efef53ff700 1 --2- 172.21.15.97:0/1134032225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efef0160d00 0x7efef00b7400 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.113+0000 7efef53ff700 1 --2- 172.21.15.97:0/1134032225 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef015c520 0x7efef00cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.113+0000 7efef53ff700 1 -- 172.21.15.97:0/1134032225 >> 172.21.15.97:0/1134032225 conn(0x7efef0071790 msgr2=0x7efef0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.113+0000 7efef53ff700 1 -- 172.21.15.97:0/1134032225 shutdown_connections 2023-07-06T03:51:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.113+0000 7efef53ff700 1 -- 172.21.15.97:0/1134032225 wait complete. 2023-07-06T03:51:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.116+0000 7efef53ff700 1 Processor -- start 2023-07-06T03:51:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.116+0000 7efef53ff700 1 -- start start 2023-07-06T03:51:35.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.117+0000 7efef53ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efef015c520 0x7efef014ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:35.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.117+0000 7efef53ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efef0160180 0x7efef015f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:35.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.117+0000 7efef53ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef0160d00 0x7efef015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:35.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.117+0000 7efef53ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef015fa90 con 0x7efef0160180 2023-07-06T03:51:35.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.117+0000 7efef53ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef015e6f0 con 0x7efef015c520 2023-07-06T03:51:35.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.117+0000 7efef53ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef015dae0 con 0x7efef0160d00 2023-07-06T03:51:35.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.117+0000 7efeee7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efef0160180 0x7efef015f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:35.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.117+0000 7efeee7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efef0160180 0x7efef015f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46542/0 (socket says 172.21.15.97:46542) 2023-07-06T03:51:35.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.117+0000 7efeee7fc700 1 -- 172.21.15.97:0/2136896753 learned_addr learned my addr 172.21.15.97:0/2136896753 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:35.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.117+0000 7efeeeffd700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efef015c520 0x7efef014ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:35.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.117+0000 7efeef7fe700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef0160d00 0x7efef015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.118+0000 7efeee7fc700 1 -- 172.21.15.97:0/2136896753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef0160d00 msgr2=0x7efef015eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.118+0000 7efeee7fc700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef0160d00 0x7efef015eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.118+0000 7efeee7fc700 1 -- 172.21.15.97:0/2136896753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efef015c520 msgr2=0x7efef014ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.118+0000 7efeee7fc700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efef015c520 0x7efef014ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.118+0000 7efeee7fc700 1 -- 172.21.15.97:0/2136896753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee00094a0 con 0x7efef0160180 2023-07-06T03:51:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.118+0000 7efeee7fc700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efef0160180 0x7efef015f550 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7efee400c700 tx=0x7efee400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.118+0000 7efed77fe700 1 -- 172.21.15.97:0/2136896753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4005d70 con 0x7efef0160180 2023-07-06T03:51:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.118+0000 7efed77fe700 1 -- 172.21.15.97:0/2136896753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee40070d0 con 0x7efef0160180 2023-07-06T03:51:35.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.118+0000 7efef53ff700 1 -- 172.21.15.97:0/2136896753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee8009a70 con 0x7efef0160180 2023-07-06T03:51:35.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.118+0000 7efed77fe700 1 -- 172.21.15.97:0/2136896753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4007460 con 0x7efef0160180 2023-07-06T03:51:35.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.118+0000 7efef53ff700 1 -- 172.21.15.97:0/2136896753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef015e0a0 con 0x7efef0160180 2023-07-06T03:51:35.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.119+0000 7efef53ff700 1 -- 172.21.15.97:0/2136896753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef00cde30 con 0x7efef0160180 2023-07-06T03:51:35.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.120+0000 7efed77fe700 1 -- 172.21.15.97:0/2136896753 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efee40075c0 con 0x7efef0160180 2023-07-06T03:51:35.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.122+0000 7efed77fe700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efedc06ace0 0x7efedc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:35.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.122+0000 7efed77fe700 1 -- 172.21.15.97:0/2136896753 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efee4088970 con 0x7efef0160180 2023-07-06T03:51:35.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.122+0000 7efeeeffd700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efedc06ace0 0x7efedc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:35.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.122+0000 7efeeeffd700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efedc06ace0 0x7efedc06d190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efee0009cd0 tx=0x7efee0009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:35.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.126+0000 7efed77fe700 1 -- 172.21.15.97:0/2136896753 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efee4053980 con 0x7efef0160180 2023-07-06T03:51:35.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.364+0000 7efef53ff700 1 -- 172.21.15.97:0/2136896753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7efef015feb0 con 0x7efef0160180 2023-07-06T03:51:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.365+0000 7efed77fe700 1 -- 172.21.15.97:0/2136896753 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7efee40170c0 con 0x7efef0160180 2023-07-06T03:51:35.367 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:51:35.367 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:51:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.367+0000 7efef53ff700 1 -- 172.21.15.97:0/2136896753 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efedc06ace0 msgr2=0x7efedc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.367+0000 7efef53ff700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efedc06ace0 0x7efedc06d190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efee0009cd0 tx=0x7efee0009210 comp rx=0 tx=0).stop 2023-07-06T03:51:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.367+0000 7efef53ff700 1 -- 172.21.15.97:0/2136896753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efef0160180 msgr2=0x7efef015f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.367+0000 7efef53ff700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efef0160180 0x7efef015f550 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7efee400c700 tx=0x7efee400ca10 comp rx=0 tx=0).stop 2023-07-06T03:51:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.367+0000 7efef53ff700 1 -- 172.21.15.97:0/2136896753 shutdown_connections 2023-07-06T03:51:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.367+0000 7efef53ff700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efedc06ace0 0x7efedc06d190 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.368+0000 7efef53ff700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efef0160d00 0x7efef015eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.368+0000 7efef53ff700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efef015c520 0x7efef014ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.368+0000 7efef53ff700 1 --2- 172.21.15.97:0/2136896753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efef0160180 0x7efef015f550 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.368+0000 7efef53ff700 1 -- 172.21.15.97:0/2136896753 >> 172.21.15.97:0/2136896753 conn(0x7efef0071790 msgr2=0x7efef00cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.368+0000 7efef53ff700 1 -- 172.21.15.97:0/2136896753 shutdown_connections 2023-07-06T03:51:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:35.368+0000 7efef53ff700 1 -- 172.21.15.97:0/2136896753 wait complete. 2023-07-06T03:51:35.387 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:51:35.388 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:51:40.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:51:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.589+0000 7f20525ff700 1 Processor -- start 2023-07-06T03:51:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.589+0000 7f20525ff700 1 -- start start 2023-07-06T03:51:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.590+0000 7f20525ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f204c15c520 0x7f204c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.590+0000 7f20525ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f204c160180 0x7f204c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.590+0000 7f20525ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c0cdae0 con 0x7f204c160b30 2023-07-06T03:51:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.591+0000 7f20525ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c0cd8d0 con 0x7f204c15c520 2023-07-06T03:51:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.591+0000 7f20525ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c0b6640 con 0x7f204c160180 2023-07-06T03:51:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.591+0000 7f204bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f204c15c520 0x7f204c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.591+0000 7f2050b9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f204c160b30 0x7f204c0cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46960/0 (socket says 172.21.15.97:46960) 2023-07-06T03:51:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.591+0000 7f2050b9c700 1 -- 172.21.15.97:0/2715505128 learned_addr learned my addr 172.21.15.97:0/2715505128 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:51:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.591+0000 7f204b7fe700 1 --2- 172.21.15.97:0/2715505128 >> v2:172.21.15.146:3300/0 conn(0x7f204c160180 0x7f204c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.591+0000 7f204bfff700 1 -- 172.21.15.97:0/2715505128 >> v2:172.21.15.146:3300/0 conn(0x7f204c160180 msgr2=0x7f204c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.591+0000 7f204bfff700 1 --2- 172.21.15.97:0/2715505128 >> v2:172.21.15.146:3300/0 conn(0x7f204c160180 0x7f204c1605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.591+0000 7f204bfff700 1 -- 172.21.15.97:0/2715505128 >> v1:172.21.15.97:6789/0 conn(0x7f204c160b30 legacy=0x7f204c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.591+0000 7f204bfff700 1 -- 172.21.15.97:0/2715505128 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c0b6920 con 0x7f204c15c520 2023-07-06T03:51:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.591+0000 7f204bfff700 1 --2- 172.21.15.97:0/2715505128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f204c15c520 0x7f204c0cd390 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f204c0b5930 tx=0x7f203c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e581a04daca40d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.592+0000 7f204affd700 1 -- 172.21.15.97:0/2715505128 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c011830 con 0x7f204c15c520 2023-07-06T03:51:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.592+0000 7f204affd700 1 -- 172.21.15.97:0/2715505128 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c011990 con 0x7f204c15c520 2023-07-06T03:51:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.592+0000 7f204affd700 1 -- 172.21.15.97:0/2715505128 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c011d20 con 0x7f204c15c520 2023-07-06T03:51:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.592+0000 7f20525ff700 1 -- 172.21.15.97:0/2715505128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f204c15c520 msgr2=0x7f204c0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.592+0000 7f20525ff700 1 --2- 172.21.15.97:0/2715505128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f204c15c520 0x7f204c0cd390 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f204c0b5930 tx=0x7f203c009b20 comp rx=0 tx=0).stop 2023-07-06T03:51:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.592+0000 7f20525ff700 1 -- 172.21.15.97:0/2715505128 shutdown_connections 2023-07-06T03:51:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.592+0000 7f20525ff700 1 --2- 172.21.15.97:0/2715505128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f204c15c520 0x7f204c0cd390 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.592+0000 7f20525ff700 1 --2- 172.21.15.97:0/2715505128 >> v2:172.21.15.146:3300/0 conn(0x7f204c160180 0x7f204c1605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.592+0000 7f20525ff700 1 -- 172.21.15.97:0/2715505128 >> 172.21.15.97:0/2715505128 conn(0x7f204c071790 msgr2=0x7f204c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.592+0000 7f20525ff700 1 -- 172.21.15.97:0/2715505128 shutdown_connections 2023-07-06T03:51:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.592+0000 7f20525ff700 1 -- 172.21.15.97:0/2715505128 wait complete. 2023-07-06T03:51:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.594+0000 7f20525ff700 1 Processor -- start 2023-07-06T03:51:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.594+0000 7f20525ff700 1 -- start start 2023-07-06T03:51:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.595+0000 7f20525ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f204c15c520 0x7f204c15b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:40.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.595+0000 7f20525ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f204c160180 0x7f204c1f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.595+0000 7f20525ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f204c160b30 0x7f204c1f1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.595+0000 7f20525ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c15bd20 con 0x7f204c160180 2023-07-06T03:51:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.595+0000 7f20525ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c1f2fb0 con 0x7f204c160b30 2023-07-06T03:51:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.595+0000 7f20525ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0d1710 con 0x7f204c15c520 2023-07-06T03:51:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.595+0000 7f204b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f204c160180 0x7f204c1f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.595+0000 7f204b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f204c160180 0x7f204c1f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49838/0 (socket says 172.21.15.97:49838) 2023-07-06T03:51:40.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.595+0000 7f204b7fe700 1 -- 172.21.15.97:0/1060359369 learned_addr learned my addr 172.21.15.97:0/1060359369 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f204bfff700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f204c15c520 0x7f204c15b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f2050b9c700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f204c160b30 0x7f204c1f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f204b7fe700 1 -- 172.21.15.97:0/1060359369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f204c15c520 msgr2=0x7f204c15b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f204b7fe700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f204c15c520 0x7f204c15b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f204b7fe700 1 -- 172.21.15.97:0/1060359369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f204c160b30 msgr2=0x7f204c1f1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f204b7fe700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f204c160b30 0x7f204c1f1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f204b7fe700 1 -- 172.21.15.97:0/1060359369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0094a0 con 0x7f204c160180 2023-07-06T03:51:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f204b7fe700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f204c160180 0x7f204c1f2a70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f204000e810 tx=0x7f204000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:40.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f2048ff9700 1 -- 172.21.15.97:0/1060359369 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204000c070 con 0x7f204c160180 2023-07-06T03:51:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f2048ff9700 1 -- 172.21.15.97:0/1060359369 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2040013610 con 0x7f204c160180 2023-07-06T03:51:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f20525ff700 1 -- 172.21.15.97:0/1060359369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c0d0de0 con 0x7f204c160180 2023-07-06T03:51:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f2048ff9700 1 -- 172.21.15.97:0/1060359369 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20400139c0 con 0x7f204c160180 2023-07-06T03:51:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.596+0000 7f20525ff700 1 -- 172.21.15.97:0/1060359369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c0d13a0 con 0x7f204c160180 2023-07-06T03:51:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.598+0000 7f20525ff700 1 -- 172.21.15.97:0/1060359369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c0d18c0 con 0x7f204c160180 2023-07-06T03:51:40.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.598+0000 7f2048ff9700 1 -- 172.21.15.97:0/1060359369 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2040013b20 con 0x7f204c160180 2023-07-06T03:51:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.599+0000 7f2048ff9700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f203806ace0 0x7f203806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.599+0000 7f2048ff9700 1 -- 172.21.15.97:0/1060359369 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2040089200 con 0x7f204c160180 2023-07-06T03:51:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.599+0000 7f204bfff700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f203806ace0 0x7f203806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:40.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.600+0000 7f204bfff700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f203806ace0 0x7f203806d190 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f203c0090b0 tx=0x7f203c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:40.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.603+0000 7f2048ff9700 1 -- 172.21.15.97:0/1060359369 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f204c0d18c0 con 0x7f204c160180 2023-07-06T03:51:40.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.840+0000 7f20525ff700 1 -- 172.21.15.97:0/1060359369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f204c0d1880 con 0x7f204c160180 2023-07-06T03:51:40.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.840+0000 7f2048ff9700 1 -- 172.21.15.97:0/1060359369 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f2040057cb0 con 0x7f204c160180 2023-07-06T03:51:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:51:40.843 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:51:40.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 -- 172.21.15.97:0/1060359369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f203806ace0 msgr2=0x7f203806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f203806ace0 0x7f203806d190 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f203c0090b0 tx=0x7f203c01c040 comp rx=0 tx=0).stop 2023-07-06T03:51:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 -- 172.21.15.97:0/1060359369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f204c160180 msgr2=0x7f204c1f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f204c160180 0x7f204c1f2a70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f204000e810 tx=0x7f204000eb20 comp rx=0 tx=0).stop 2023-07-06T03:51:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 -- 172.21.15.97:0/1060359369 shutdown_connections 2023-07-06T03:51:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f203806ace0 0x7f203806d190 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f204c160b30 0x7f204c1f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f204c15c520 0x7f204c15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:40.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 --2- 172.21.15.97:0/1060359369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f204c160180 0x7f204c1f2a70 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:40.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 -- 172.21.15.97:0/1060359369 >> 172.21.15.97:0/1060359369 conn(0x7f204c071790 msgr2=0x7f204c0b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:40.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 -- 172.21.15.97:0/1060359369 shutdown_connections 2023-07-06T03:51:40.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:40.843+0000 7f20525ff700 1 -- 172.21.15.97:0/1060359369 wait complete. 2023-07-06T03:51:40.862 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:51:40.862 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:51:45.863 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:51:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.063+0000 7f925a7ff700 1 Processor -- start 2023-07-06T03:51:46.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.063+0000 7f925a7ff700 1 -- start start 2023-07-06T03:51:46.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.065+0000 7f925a7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9254158960 0x7f925406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:46.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.065+0000 7f925a7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f925415ce90 0x7f92540712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:46.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.065+0000 7f925a7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9254071830 0x7f92540cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:46.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.065+0000 7f925a7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925406ddb0 con 0x7f9254071830 2023-07-06T03:51:46.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.065+0000 7f925a7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925406bbf0 con 0x7f925415ce90 2023-07-06T03:51:46.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.065+0000 7f925a7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925406bed0 con 0x7f9254158960 2023-07-06T03:51:46.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.065+0000 7f9258d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9254071830 0x7f92540cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:46.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.065+0000 7f9258d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9254071830 0x7f92540cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49842/0 (socket says 172.21.15.97:49842) 2023-07-06T03:51:46.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.065+0000 7f9258d9c700 1 -- 172.21.15.97:0/2496110532 learned_addr learned my addr 172.21.15.97:0/2496110532 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:46.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.065+0000 7f92537fe700 1 --2- 172.21.15.97:0/2496110532 >> v2:172.21.15.146:3300/0 conn(0x7f925415ce90 0x7f92540712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:46.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.065+0000 7f9253fff700 1 --2- 172.21.15.97:0/2496110532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9254158960 0x7f925406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:46.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.066+0000 7f9258d9c700 1 -- 172.21.15.97:0/2496110532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9254158960 msgr2=0x7f925406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:46.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.066+0000 7f9258d9c700 1 --2- 172.21.15.97:0/2496110532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9254158960 0x7f925406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:46.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.066+0000 7f9258d9c700 1 -- 172.21.15.97:0/2496110532 >> v2:172.21.15.146:3300/0 conn(0x7f925415ce90 msgr2=0x7f92540712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:46.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.066+0000 7f9258d9c700 1 --2- 172.21.15.97:0/2496110532 >> v2:172.21.15.146:3300/0 conn(0x7f925415ce90 0x7f92540712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:46.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.066+0000 7f9258d9c700 1 -- 172.21.15.97:0/2496110532 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925406c1b0 con 0x7f9254071830 2023-07-06T03:51:46.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.066+0000 7f9258d9c700 1 --2- 172.21.15.97:0/2496110532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9254071830 0x7f92540cc400 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f92540b1690 tx=0x7f924c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a0b26503be9251 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:46.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.066+0000 7f9252ffd700 1 -- 172.21.15.97:0/2496110532 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c00bbc0 con 0x7f9254071830 2023-07-06T03:51:46.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.066+0000 7f9252ffd700 1 -- 172.21.15.97:0/2496110532 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c00bd20 con 0x7f9254071830 2023-07-06T03:51:46.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.066+0000 7f9252ffd700 1 -- 172.21.15.97:0/2496110532 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c0116a0 con 0x7f9254071830 2023-07-06T03:51:46.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.067+0000 7f925a7ff700 1 -- 172.21.15.97:0/2496110532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9254071830 msgr2=0x7f92540cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:46.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.067+0000 7f925a7ff700 1 --2- 172.21.15.97:0/2496110532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9254071830 0x7f92540cc400 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f92540b1690 tx=0x7f924c00b330 comp rx=0 tx=0).stop 2023-07-06T03:51:46.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.067+0000 7f925a7ff700 1 -- 172.21.15.97:0/2496110532 shutdown_connections 2023-07-06T03:51:46.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.067+0000 7f925a7ff700 1 --2- 172.21.15.97:0/2496110532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9254071830 0x7f92540cc400 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:46.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.067+0000 7f925a7ff700 1 --2- 172.21.15.97:0/2496110532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9254158960 0x7f925406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:46.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.067+0000 7f925a7ff700 1 --2- 172.21.15.97:0/2496110532 >> v2:172.21.15.146:3300/0 conn(0x7f925415ce90 0x7f92540712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:46.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.067+0000 7f925a7ff700 1 -- 172.21.15.97:0/2496110532 >> 172.21.15.97:0/2496110532 conn(0x7f92540ce8c0 msgr2=0x7f92540cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:46.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.067+0000 7f925a7ff700 1 -- 172.21.15.97:0/2496110532 shutdown_connections 2023-07-06T03:51:46.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.067+0000 7f925a7ff700 1 -- 172.21.15.97:0/2496110532 wait complete. 2023-07-06T03:51:46.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.068+0000 7f925a7ff700 1 Processor -- start 2023-07-06T03:51:46.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.069+0000 7f925a7ff700 1 -- start start 2023-07-06T03:51:46.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.069+0000 7f925a7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9254071830 0x7f92540b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:46.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f925a7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9254158960 0x7f92541482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:46.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f925a7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f925415ce90 0x7f9254155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:46.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f925a7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92540b06a0 con 0x7f9254158960 2023-07-06T03:51:46.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f925a7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9254148830 con 0x7f925415ce90 2023-07-06T03:51:46.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f925a7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9254157fe0 con 0x7f9254071830 2023-07-06T03:51:46.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f92537fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9254158960 0x7f92541482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:46.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f92537fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9254158960 0x7f92541482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49846/0 (socket says 172.21.15.97:49846) 2023-07-06T03:51:46.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f92537fe700 1 -- 172.21.15.97:0/1024432468 learned_addr learned my addr 172.21.15.97:0/1024432468 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:46.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f9258d9c700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f925415ce90 0x7f9254155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:46.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f9253fff700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9254071830 0x7f92540b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:46.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f92537fe700 1 -- 172.21.15.97:0/1024432468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9254071830 msgr2=0x7f92540b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:46.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f92537fe700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9254071830 0x7f92540b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:46.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f92537fe700 1 -- 172.21.15.97:0/1024432468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f925415ce90 msgr2=0x7f9254155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:46.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f92537fe700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f925415ce90 0x7f9254155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:46.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f92537fe700 1 -- 172.21.15.97:0/1024432468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c009b30 con 0x7f9254158960 2023-07-06T03:51:46.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.070+0000 7f9253fff700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9254071830 0x7f92540b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:51:46.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.071+0000 7f92537fe700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9254158960 0x7f92541482f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f924800d830 tx=0x7f924800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:46.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.071+0000 7f9250ff9700 1 -- 172.21.15.97:0/1024432468 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92480125f0 con 0x7f9254158960 2023-07-06T03:51:46.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.071+0000 7f9250ff9700 1 -- 172.21.15.97:0/1024432468 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9248012750 con 0x7f9254158960 2023-07-06T03:51:46.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.071+0000 7f9250ff9700 1 -- 172.21.15.97:0/1024432468 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248012b00 con 0x7f9254158960 2023-07-06T03:51:46.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.071+0000 7f925a7ff700 1 -- 172.21.15.97:0/1024432468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92540b0120 con 0x7f9254158960 2023-07-06T03:51:46.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.071+0000 7f925a7ff700 1 -- 172.21.15.97:0/1024432468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92540b03c0 con 0x7f9254158960 2023-07-06T03:51:46.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.072+0000 7f925a7ff700 1 -- 172.21.15.97:0/1024432468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9254067830 con 0x7f9254158960 2023-07-06T03:51:46.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.073+0000 7f9250ff9700 1 -- 172.21.15.97:0/1024432468 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9248012c60 con 0x7f9254158960 2023-07-06T03:51:46.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.074+0000 7f9250ff9700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f924006ace0 0x7f924006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:46.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.074+0000 7f9250ff9700 1 -- 172.21.15.97:0/1024432468 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9248089860 con 0x7f9254158960 2023-07-06T03:51:46.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.074+0000 7f9253fff700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f924006ace0 0x7f924006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:46.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.075+0000 7f9253fff700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f924006ace0 0x7f924006d190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9244009970 tx=0x7f9244008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:46.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.079+0000 7f9250ff9700 1 -- 172.21.15.97:0/1024432468 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9248054890 con 0x7f9254158960 2023-07-06T03:51:46.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.316+0000 7f925a7ff700 1 -- 172.21.15.97:0/1024432468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f925404e340 con 0x7f9254158960 2023-07-06T03:51:46.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.316+0000 7f9250ff9700 1 -- 172.21.15.97:0/1024432468 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f9248057eb0 con 0x7f9254158960 2023-07-06T03:51:46.318 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:51:46.318 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:51:46.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 -- 172.21.15.97:0/1024432468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f924006ace0 msgr2=0x7f924006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:46.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f924006ace0 0x7f924006d190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9244009970 tx=0x7f9244008040 comp rx=0 tx=0).stop 2023-07-06T03:51:46.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 -- 172.21.15.97:0/1024432468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9254158960 msgr2=0x7f92541482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:46.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9254158960 0x7f92541482f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f924800d830 tx=0x7f924800db40 comp rx=0 tx=0).stop 2023-07-06T03:51:46.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 -- 172.21.15.97:0/1024432468 shutdown_connections 2023-07-06T03:51:46.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f924006ace0 0x7f924006d190 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:46.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f925415ce90 0x7f9254155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:46.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9254071830 0x7f92540b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:46.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 --2- 172.21.15.97:0/1024432468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9254158960 0x7f92541482f0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:46.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 -- 172.21.15.97:0/1024432468 >> 172.21.15.97:0/1024432468 conn(0x7f92540ce8c0 msgr2=0x7f92540adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:46.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 -- 172.21.15.97:0/1024432468 shutdown_connections 2023-07-06T03:51:46.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:46.319+0000 7f925a7ff700 1 -- 172.21.15.97:0/1024432468 wait complete. 2023-07-06T03:51:46.344 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:51:46.344 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:51:51.345 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:51:51.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.546+0000 7f6ac17ff700 1 Processor -- start 2023-07-06T03:51:51.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.546+0000 7f6ac17ff700 1 -- start start 2023-07-06T03:51:51.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.547+0000 7f6ac17ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6abc06ee80 0x7f6abc071730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:51.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.547+0000 7f6ac17ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc06a060 0x7f6abc06a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:51.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.547+0000 7f6ac17ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6abc06aa00 con 0x7f6abc0cc010 2023-07-06T03:51:51.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.547+0000 7f6ac17ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6abc15c840 con 0x7f6abc06ee80 2023-07-06T03:51:51.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.547+0000 7f6ac17ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc0ceab0 con 0x7f6abc06a060 2023-07-06T03:51:51.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.547+0000 7f6abaffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6abc0cc010 0x7f6abc0b14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57744/0 (socket says 172.21.15.97:57744) 2023-07-06T03:51:51.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.547+0000 7f6abaffd700 1 -- 172.21.15.97:0/3728760476 learned_addr learned my addr 172.21.15.97:0/3728760476 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:51:51.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.548+0000 7f6abb7fe700 1 --2- 172.21.15.97:0/3728760476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc06a060 0x7f6abc06a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:51.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.548+0000 7f6ab27fc700 1 --2- 172.21.15.97:0/3728760476 >> v2:172.21.15.146:3300/0 conn(0x7f6abc06ee80 0x7f6abc071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:51.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.548+0000 7f6aba7fc700 1 -- 172.21.15.97:0/3728760476 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550395256 0 0) 0x7f6abc06aa00 con 0x7f6abc0cc010 2023-07-06T03:51:51.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.548+0000 7f6aba7fc700 1 -- 172.21.15.97:0/3728760476 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aa80035b0 con 0x7f6abc0cc010 2023-07-06T03:51:51.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.548+0000 7f6abb7fe700 1 -- 172.21.15.97:0/3728760476 >> v2:172.21.15.146:3300/0 conn(0x7f6abc06ee80 msgr2=0x7f6abc071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:51.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.548+0000 7f6abb7fe700 1 --2- 172.21.15.97:0/3728760476 >> v2:172.21.15.146:3300/0 conn(0x7f6abc06ee80 0x7f6abc071730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:51.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.548+0000 7f6abb7fe700 1 -- 172.21.15.97:0/3728760476 >> v1:172.21.15.97:6789/0 conn(0x7f6abc0cc010 legacy=0x7f6abc0b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:51.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.548+0000 7f6abb7fe700 1 -- 172.21.15.97:0/3728760476 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc0ced90 con 0x7f6abc06a060 2023-07-06T03:51:51.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.548+0000 7f6ab27fc700 1 --2- 172.21.15.97:0/3728760476 >> v2:172.21.15.146:3300/0 conn(0x7f6abc06ee80 0x7f6abc071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:51:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.548+0000 7f6abb7fe700 1 --2- 172.21.15.97:0/3728760476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc06a060 0x7f6abc06a4c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009840 tx=0x7f6ab4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e86336df71c5fb9d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.549+0000 7f6aba7fc700 1 -- 172.21.15.97:0/3728760476 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4011830 con 0x7f6abc06a060 2023-07-06T03:51:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.549+0000 7f6aba7fc700 1 -- 172.21.15.97:0/3728760476 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab4011990 con 0x7f6abc06a060 2023-07-06T03:51:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.549+0000 7f6aba7fc700 1 -- 172.21.15.97:0/3728760476 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4011d00 con 0x7f6abc06a060 2023-07-06T03:51:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.549+0000 7f6ac17ff700 1 -- 172.21.15.97:0/3728760476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc06a060 msgr2=0x7f6abc06a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.549+0000 7f6ac17ff700 1 --2- 172.21.15.97:0/3728760476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc06a060 0x7f6abc06a4c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009840 tx=0x7f6ab4009b50 comp rx=0 tx=0).stop 2023-07-06T03:51:51.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.549+0000 7f6ac17ff700 1 -- 172.21.15.97:0/3728760476 shutdown_connections 2023-07-06T03:51:51.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.549+0000 7f6ac17ff700 1 --2- 172.21.15.97:0/3728760476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc06a060 0x7f6abc06a4c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:51.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.549+0000 7f6ac17ff700 1 --2- 172.21.15.97:0/3728760476 >> v2:172.21.15.146:3300/0 conn(0x7f6abc06ee80 0x7f6abc071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:51.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.549+0000 7f6ac17ff700 1 -- 172.21.15.97:0/3728760476 >> 172.21.15.97:0/3728760476 conn(0x7f6abc0cc4b0 msgr2=0x7f6abc0cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.549+0000 7f6ac17ff700 1 -- 172.21.15.97:0/3728760476 shutdown_connections 2023-07-06T03:51:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.549+0000 7f6ac17ff700 1 -- 172.21.15.97:0/3728760476 wait complete. 2023-07-06T03:51:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.551+0000 7f6ac17ff700 1 Processor -- start 2023-07-06T03:51:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.551+0000 7f6ac17ff700 1 -- start start 2023-07-06T03:51:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.552+0000 7f6ac17ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc06a060 0x7f6abc159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.552+0000 7f6ac17ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc06ee80 0x7f6abc157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.552+0000 7f6ac17ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc0cc010 0x7f6abc157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.552+0000 7f6ac17ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc1580c0 con 0x7f6abc0cc010 2023-07-06T03:51:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.552+0000 7f6ac17ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc15d9b0 con 0x7f6abc06a060 2023-07-06T03:51:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.552+0000 7f6ac17ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc15dc90 con 0x7f6abc06ee80 2023-07-06T03:51:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.552+0000 7f6abb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc0cc010 0x7f6abc157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.552+0000 7f6abb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc0cc010 0x7f6abc157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33276/0 (socket says 172.21.15.97:33276) 2023-07-06T03:51:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.552+0000 7f6abb7fe700 1 -- 172.21.15.97:0/2888096686 learned_addr learned my addr 172.21.15.97:0/2888096686 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6abaffd700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc06a060 0x7f6abc159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6ab27fc700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc06ee80 0x7f6abc157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6abb7fe700 1 -- 172.21.15.97:0/2888096686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc06ee80 msgr2=0x7f6abc157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6abb7fe700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc06ee80 0x7f6abc157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6abb7fe700 1 -- 172.21.15.97:0/2888096686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc06a060 msgr2=0x7f6abc159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6abb7fe700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc06a060 0x7f6abc159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6abb7fe700 1 -- 172.21.15.97:0/2888096686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa40094a0 con 0x7f6abc0cc010 2023-07-06T03:51:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6abb7fe700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc0cc010 0x7f6abc157b80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009b90 tx=0x7f6ab4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6ab3fff700 1 -- 172.21.15.97:0/2888096686 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40045d0 con 0x7f6abc0cc010 2023-07-06T03:51:51.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6ab3fff700 1 -- 172.21.15.97:0/2888096686 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab401f070 con 0x7f6abc0cc010 2023-07-06T03:51:51.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6ac17ff700 1 -- 172.21.15.97:0/2888096686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab40094a0 con 0x7f6abc0cc010 2023-07-06T03:51:51.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6ab3fff700 1 -- 172.21.15.97:0/2888096686 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40038c0 con 0x7f6abc0cc010 2023-07-06T03:51:51.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.553+0000 7f6ac17ff700 1 -- 172.21.15.97:0/2888096686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc15e250 con 0x7f6abc0cc010 2023-07-06T03:51:51.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.554+0000 7f6ac17ff700 1 -- 172.21.15.97:0/2888096686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc15e950 con 0x7f6abc0cc010 2023-07-06T03:51:51.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.555+0000 7f6ab3fff700 1 -- 172.21.15.97:0/2888096686 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ab4003a20 con 0x7f6abc0cc010 2023-07-06T03:51:51.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.556+0000 7f6ab3fff700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6a9c06ace0 0x7f6a9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:51.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.556+0000 7f6ab3fff700 1 -- 172.21.15.97:0/2888096686 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6ab4021070 con 0x7f6abc0cc010 2023-07-06T03:51:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.556+0000 7f6abaffd700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6a9c06ace0 0x7f6a9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.557+0000 7f6abaffd700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6a9c06ace0 0x7f6a9c06d190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6aac005c90 tx=0x7f6aac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.561+0000 7f6ab3fff700 1 -- 172.21.15.97:0/2888096686 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6ab4055e00 con 0x7f6abc0cc010 2023-07-06T03:51:51.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.797+0000 7f6ac17ff700 1 -- 172.21.15.97:0/2888096686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6abc15e770 con 0x7f6abc0cc010 2023-07-06T03:51:51.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.798+0000 7f6ab3fff700 1 -- 172.21.15.97:0/2888096686 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f6ab40150c0 con 0x7f6abc0cc010 2023-07-06T03:51:51.800 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:51:51.801 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:51:51.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 -- 172.21.15.97:0/2888096686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6a9c06ace0 msgr2=0x7f6a9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:51.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6a9c06ace0 0x7f6a9c06d190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6aac005c90 tx=0x7f6aac009040 comp rx=0 tx=0).stop 2023-07-06T03:51:51.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 -- 172.21.15.97:0/2888096686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc0cc010 msgr2=0x7f6abc157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:51.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc0cc010 0x7f6abc157b80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009b90 tx=0x7f6ab4004750 comp rx=0 tx=0).stop 2023-07-06T03:51:51.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 -- 172.21.15.97:0/2888096686 shutdown_connections 2023-07-06T03:51:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6a9c06ace0 0x7f6a9c06d190 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc0cc010 0x7f6abc157b80 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc06a060 0x7f6abc159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 --2- 172.21.15.97:0/2888096686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc06ee80 0x7f6abc157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 -- 172.21.15.97:0/2888096686 >> 172.21.15.97:0/2888096686 conn(0x7f6abc0cc4b0 msgr2=0x7f6abc0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 -- 172.21.15.97:0/2888096686 shutdown_connections 2023-07-06T03:51:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:51.800+0000 7f6ac17ff700 1 -- 172.21.15.97:0/2888096686 wait complete. 2023-07-06T03:51:51.821 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:51:51.821 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:51:56.822 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:51:57.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.026+0000 7f69491ff700 1 Processor -- start 2023-07-06T03:51:57.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.026+0000 7f69491ff700 1 -- start start 2023-07-06T03:51:57.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.027+0000 7f69491ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69440b8410 0x7f6944167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:57.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.027+0000 7f69491ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69440cdae0 con 0x7f694415c520 2023-07-06T03:51:57.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.027+0000 7f69491ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69440b6640 con 0x7f69440cd390 2023-07-06T03:51:57.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.027+0000 7f69491ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69440b6920 con 0x7f69440b8410 2023-07-06T03:51:57.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.027+0000 7f6942d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f694415c520 0x7f6944160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57746/0 (socket says 172.21.15.97:57746) 2023-07-06T03:51:57.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.027+0000 7f6942d9d700 1 -- 172.21.15.97:0/3752671173 learned_addr learned my addr 172.21.15.97:0/3752671173 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:51:57.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f694359e700 1 --2- 172.21.15.97:0/3752671173 >> v2:172.21.15.146:3300/0 conn(0x7f69440b8410 0x7f6944167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:57.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883873036 0 0) 0x7f69440cdae0 con 0x7f694415c520 2023-07-06T03:51:57.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f692c0035b0 con 0x7f694415c520 2023-07-06T03:51:57.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1017405873 0 0) 0x7f692c0035b0 con 0x7f694415c520 2023-07-06T03:51:57.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 >> v2:172.21.15.146:3300/0 conn(0x7f69440b8410 msgr2=0x7f6944167510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:57.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 --2- 172.21.15.97:0/3752671173 >> v2:172.21.15.146:3300/0 conn(0x7f69440b8410 0x7f6944167510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:57.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 >> v1:172.21.15.146:6789/0 conn(0x7f69440cd390 legacy=0x7f6944160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:57.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440b6c00 con 0x7f694415c520 2023-07-06T03:51:57.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f6934001670 con 0x7f694415c520 2023-07-06T03:51:57.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 >> v1:172.21.15.97:6789/0 conn(0x7f694415c520 legacy=0x7f6944160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:57.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c003a00 0x7f692c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:57.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f692c004850 0x7f692c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:57.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f692c005200 0x7f692c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:57.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440b6920 con 0x7f692c003a00 2023-07-06T03:51:57.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c00c080 con 0x7f692c004850 2023-07-06T03:51:57.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.028+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c00c240 con 0x7f692c005200 2023-07-06T03:51:57.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f6942d9d700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c003a00 0x7f692c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:57.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f694259c700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f692c004850 0x7f692c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:57.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f694359e700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f692c005200 0x7f692c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:57.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f6942d9d700 1 -- 172.21.15.97:0/3752671173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f692c005200 msgr2=0x7f692c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:57.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f6942d9d700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f692c005200 0x7f692c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:57.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f6942d9d700 1 -- 172.21.15.97:0/3752671173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f692c004850 msgr2=0x7f692c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:51:57.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f6942d9d700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f692c004850 0x7f692c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:57.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f6942d9d700 1 -- 172.21.15.97:0/3752671173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c0035b0 con 0x7f692c003a00 2023-07-06T03:51:57.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f694259c700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f692c004850 0x7f692c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:51:57.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f694359e700 1 -- 172.21.15.97:0/3752671173 reap_dead start 2023-07-06T03:51:57.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f6942d9d700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c003a00 0x7f692c003e10 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f693400c240 tx=0x7f692c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8be32bcbafef19 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:57.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934007c10 con 0x7f692c003a00 2023-07-06T03:51:57.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693400ce00 con 0x7f692c003a00 2023-07-06T03:51:57.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.029+0000 7f6941d9b700 1 -- 172.21.15.97:0/3752671173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934005ba0 con 0x7f692c003a00 2023-07-06T03:51:57.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.030+0000 7f69491ff700 1 -- 172.21.15.97:0/3752671173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c003a00 msgr2=0x7f692c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:57.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.030+0000 7f69491ff700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c003a00 0x7f692c003e10 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f693400c240 tx=0x7f692c005880 comp rx=0 tx=0).stop 2023-07-06T03:51:57.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.030+0000 7f69491ff700 1 -- 172.21.15.97:0/3752671173 shutdown_connections 2023-07-06T03:51:57.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.030+0000 7f69491ff700 1 --2- 172.21.15.97:0/3752671173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692c003a00 0x7f692c003e10 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:57.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.030+0000 7f69491ff700 1 -- 172.21.15.97:0/3752671173 >> 172.21.15.97:0/3752671173 conn(0x7f6944071790 msgr2=0x7f6944071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:57.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.030+0000 7f69491ff700 1 -- 172.21.15.97:0/3752671173 shutdown_connections 2023-07-06T03:51:57.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.030+0000 7f69491ff700 1 -- 172.21.15.97:0/3752671173 wait complete. 2023-07-06T03:51:57.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.032+0000 7f69491ff700 1 Processor -- start 2023-07-06T03:51:57.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.032+0000 7f69491ff700 1 -- start start 2023-07-06T03:51:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f69491ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f692c003a00 0x7f6944158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f69491ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6944155580 0x7f6944143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f69491ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6944144380 0x7f69441547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f69491ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6944144820 con 0x7f6944144380 2023-07-06T03:51:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f69491ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440cb980 con 0x7f692c003a00 2023-07-06T03:51:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f69491ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69440cbc60 con 0x7f6944155580 2023-07-06T03:51:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f694359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6944144380 0x7f69441547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f694359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6944144380 0x7f69441547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33294/0 (socket says 172.21.15.97:33294) 2023-07-06T03:51:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f694359e700 1 -- 172.21.15.97:0/3124631087 learned_addr learned my addr 172.21.15.97:0/3124631087 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:51:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f694359e700 1 -- 172.21.15.97:0/3124631087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6944155580 msgr2=0x7f6944143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f694359e700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6944155580 0x7f6944143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f694359e700 1 -- 172.21.15.97:0/3124631087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f692c003a00 msgr2=0x7f6944158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f694359e700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f692c003a00 0x7f6944158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.033+0000 7f694359e700 1 -- 172.21.15.97:0/3124631087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440cbf40 con 0x7f6944144380 2023-07-06T03:51:57.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.034+0000 7f694359e700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6944144380 0x7f69441547f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f69440c9dc0 tx=0x7f693c0114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:57.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.034+0000 7f692b7fe700 1 -- 172.21.15.97:0/3124631087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c00e070 con 0x7f6944144380 2023-07-06T03:51:57.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.034+0000 7f692b7fe700 1 -- 172.21.15.97:0/3124631087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c003c30 con 0x7f6944144380 2023-07-06T03:51:57.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.034+0000 7f69491ff700 1 -- 172.21.15.97:0/3124631087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69440cc220 con 0x7f6944144380 2023-07-06T03:51:57.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.034+0000 7f692b7fe700 1 -- 172.21.15.97:0/3124631087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c003de0 con 0x7f6944144380 2023-07-06T03:51:57.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.034+0000 7f69491ff700 1 -- 172.21.15.97:0/3124631087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69441589b0 con 0x7f6944144380 2023-07-06T03:51:57.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.035+0000 7f69491ff700 1 -- 172.21.15.97:0/3124631087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69440cc740 con 0x7f6944144380 2023-07-06T03:51:57.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.036+0000 7f692b7fe700 1 -- 172.21.15.97:0/3124631087 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f693c011a10 con 0x7f6944144380 2023-07-06T03:51:57.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.037+0000 7f692b7fe700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f693006ace0 0x7f693006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:51:57.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.037+0000 7f692b7fe700 1 -- 172.21.15.97:0/3124631087 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f693c089990 con 0x7f6944144380 2023-07-06T03:51:57.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.037+0000 7f6942d9d700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f693006ace0 0x7f693006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:51:57.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.038+0000 7f6942d9d700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f693006ace0 0x7f693006d190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f693400c240 tx=0x7f693400a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:51:57.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.042+0000 7f692b7fe700 1 -- 172.21.15.97:0/3124631087 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f69440cc740 con 0x7f6944144380 2023-07-06T03:51:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.277+0000 7f69491ff700 1 -- 172.21.15.97:0/3124631087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f69440cc740 con 0x7f6944144380 2023-07-06T03:51:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.278+0000 7f692b7fe700 1 -- 172.21.15.97:0/3124631087 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f69440cc740 con 0x7f6944144380 2023-07-06T03:51:57.281 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:51:57.281 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:51:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 -- 172.21.15.97:0/3124631087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f693006ace0 msgr2=0x7f693006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f693006ace0 0x7f693006d190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f693400c240 tx=0x7f693400a000 comp rx=0 tx=0).stop 2023-07-06T03:51:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 -- 172.21.15.97:0/3124631087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6944144380 msgr2=0x7f69441547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:51:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6944144380 0x7f69441547f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f69440c9dc0 tx=0x7f693c0114b0 comp rx=0 tx=0).stop 2023-07-06T03:51:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 -- 172.21.15.97:0/3124631087 shutdown_connections 2023-07-06T03:51:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f693006ace0 0x7f693006d190 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6944144380 0x7f69441547f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f692c003a00 0x7f6944158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 --2- 172.21.15.97:0/3124631087 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6944155580 0x7f6944143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:51:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 -- 172.21.15.97:0/3124631087 >> 172.21.15.97:0/3124631087 conn(0x7f6944071790 msgr2=0x7f694415f410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:51:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 -- 172.21.15.97:0/3124631087 shutdown_connections 2023-07-06T03:51:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:51:57.280+0000 7f69491ff700 1 -- 172.21.15.97:0/3124631087 wait complete. 2023-07-06T03:51:57.301 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:51:57.301 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:52:02.302 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:52:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.504+0000 7f589d3ff700 1 Processor -- start 2023-07-06T03:52:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.504+0000 7f589d3ff700 1 -- start start 2023-07-06T03:52:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.505+0000 7f589d3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f589815c740 0x7f58980d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.505+0000 7f589d3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58980cdc80 con 0x7f5898071c90 2023-07-06T03:52:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.505+0000 7f589d3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58980ce380 con 0x7f5898071820 2023-07-06T03:52:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.505+0000 7f589d3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58980d0610 con 0x7f589815c740 2023-07-06T03:52:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.506+0000 7f5896ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f589815c740 0x7f58980d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:02.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.506+0000 7f58977fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5898071c90 0x7f58980ca2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55656/0 (socket says 172.21.15.97:55656) 2023-07-06T03:52:02.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.506+0000 7f58977fe700 1 -- 172.21.15.97:0/1646444673 learned_addr learned my addr 172.21.15.97:0/1646444673 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:02.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.506+0000 7f5896ffd700 1 -- 172.21.15.97:0/1646444673 >> v1:172.21.15.146:6789/0 conn(0x7f5898071820 legacy=0x7f589806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:02.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.506+0000 7f5896ffd700 1 -- 172.21.15.97:0/1646444673 >> v1:172.21.15.97:6789/0 conn(0x7f5898071c90 legacy=0x7f58980ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.506+0000 7f5896ffd700 1 -- 172.21.15.97:0/1646444673 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58980cd630 con 0x7f589815c740 2023-07-06T03:52:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.506+0000 7f5895ffb700 1 -- 172.21.15.97:0/1646444673 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599452696 0 0) 0x7f58980cdc80 con 0x7f5898071c90 2023-07-06T03:52:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.506+0000 7f5896ffd700 1 --2- 172.21.15.97:0/1646444673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815c740 0x7f58980d00d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f5888009860 tx=0x7f5888009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b83d22148a2f498 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.507+0000 7f5895ffb700 1 -- 172.21.15.97:0/1646444673 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5888004230 con 0x7f589815c740 2023-07-06T03:52:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.507+0000 7f5895ffb700 1 -- 172.21.15.97:0/1646444673 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5888004390 con 0x7f589815c740 2023-07-06T03:52:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.507+0000 7f5895ffb700 1 -- 172.21.15.97:0/1646444673 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58880036a0 con 0x7f589815c740 2023-07-06T03:52:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.507+0000 7f589d3ff700 1 -- 172.21.15.97:0/1646444673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815c740 msgr2=0x7f58980d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:02.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.507+0000 7f589d3ff700 1 --2- 172.21.15.97:0/1646444673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815c740 0x7f58980d00d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f5888009860 tx=0x7f5888009b70 comp rx=0 tx=0).stop 2023-07-06T03:52:02.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.507+0000 7f589d3ff700 1 -- 172.21.15.97:0/1646444673 shutdown_connections 2023-07-06T03:52:02.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.507+0000 7f589d3ff700 1 --2- 172.21.15.97:0/1646444673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815c740 0x7f58980d00d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:02.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.507+0000 7f589d3ff700 1 -- 172.21.15.97:0/1646444673 >> 172.21.15.97:0/1646444673 conn(0x7f58980b2280 msgr2=0x7f58980cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:02.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.507+0000 7f589d3ff700 1 -- 172.21.15.97:0/1646444673 shutdown_connections 2023-07-06T03:52:02.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.507+0000 7f589d3ff700 1 -- 172.21.15.97:0/1646444673 wait complete. 2023-07-06T03:52:02.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.509+0000 7f589d3ff700 1 Processor -- start 2023-07-06T03:52:02.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.509+0000 7f589d3ff700 1 -- start start 2023-07-06T03:52:02.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.510+0000 7f589d3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5898071820 0x7f58981f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:02.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.510+0000 7f589d3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589815c740 0x7f5898147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:02.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.510+0000 7f589d3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815e0d0 0x7f58980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:02.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.510+0000 7f589d3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58981f2dc0 con 0x7f589815e0d0 2023-07-06T03:52:02.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.510+0000 7f589d3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58981479a0 con 0x7f589815c740 2023-07-06T03:52:02.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.510+0000 7f589d3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589815d030 con 0x7f5898071820 2023-07-06T03:52:02.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.510+0000 7f58977fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815e0d0 0x7f58980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:02.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.510+0000 7f58977fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815e0d0 0x7f58980b4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41926/0 (socket says 172.21.15.97:41926) 2023-07-06T03:52:02.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.510+0000 7f58977fe700 1 -- 172.21.15.97:0/832988735 learned_addr learned my addr 172.21.15.97:0/832988735 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:02.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.510+0000 7f58967fc700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589815c740 0x7f5898147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:02.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.510+0000 7f5896ffd700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5898071820 0x7f58981f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:02.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f58977fe700 1 -- 172.21.15.97:0/832988735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5898071820 msgr2=0x7f58981f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:02.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f58977fe700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5898071820 0x7f58981f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:02.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f58977fe700 1 -- 172.21.15.97:0/832988735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589815c740 msgr2=0x7f5898147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:02.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f58977fe700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589815c740 0x7f5898147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:02.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f58977fe700 1 -- 172.21.15.97:0/832988735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5888009440 con 0x7f589815e0d0 2023-07-06T03:52:02.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f58967fc700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589815c740 0x7f5898147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:02.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f5896ffd700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5898071820 0x7f58981f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:02.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f58977fe700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815e0d0 0x7f58980b4d40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f589000cba0 tx=0x7f589000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:02.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f587f7fe700 1 -- 172.21.15.97:0/832988735 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589000ab80 con 0x7f589815e0d0 2023-07-06T03:52:02.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f587f7fe700 1 -- 172.21.15.97:0/832988735 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589000ace0 con 0x7f589815e0d0 2023-07-06T03:52:02.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f589d3ff700 1 -- 172.21.15.97:0/832988735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589815d310 con 0x7f589815e0d0 2023-07-06T03:52:02.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f587f7fe700 1 -- 172.21.15.97:0/832988735 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890007310 con 0x7f589815e0d0 2023-07-06T03:52:02.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.511+0000 7f589d3ff700 1 -- 172.21.15.97:0/832988735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589815d8d0 con 0x7f589815e0d0 2023-07-06T03:52:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.512+0000 7f589d3ff700 1 -- 172.21.15.97:0/832988735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5898067870 con 0x7f589815e0d0 2023-07-06T03:52:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.513+0000 7f587f7fe700 1 -- 172.21.15.97:0/832988735 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5890007470 con 0x7f589815e0d0 2023-07-06T03:52:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.514+0000 7f587f7fe700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f588406ace0 0x7f588406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.514+0000 7f587f7fe700 1 -- 172.21.15.97:0/832988735 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5890089110 con 0x7f589815e0d0 2023-07-06T03:52:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.514+0000 7f5896ffd700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f588406ace0 0x7f588406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.515+0000 7f5896ffd700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f588406ace0 0x7f588406d190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5888009860 tx=0x7f5888008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.519+0000 7f587f7fe700 1 -- 172.21.15.97:0/832988735 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5890054120 con 0x7f589815e0d0 2023-07-06T03:52:02.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.755+0000 7f589d3ff700 1 -- 172.21.15.97:0/832988735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f589804e340 con 0x7f589815e0d0 2023-07-06T03:52:02.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.755+0000 7f587f7fe700 1 -- 172.21.15.97:0/832988735 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f5890057740 con 0x7f589815e0d0 2023-07-06T03:52:02.758 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:02.758 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:52:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.757+0000 7f589d3ff700 1 -- 172.21.15.97:0/832988735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f588406ace0 msgr2=0x7f588406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:02.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.757+0000 7f589d3ff700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f588406ace0 0x7f588406d190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5888009860 tx=0x7f5888008030 comp rx=0 tx=0).stop 2023-07-06T03:52:02.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.757+0000 7f589d3ff700 1 -- 172.21.15.97:0/832988735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815e0d0 msgr2=0x7f58980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:02.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.757+0000 7f589d3ff700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815e0d0 0x7f58980b4d40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f589000cba0 tx=0x7f589000cf60 comp rx=0 tx=0).stop 2023-07-06T03:52:02.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.757+0000 7f589d3ff700 1 -- 172.21.15.97:0/832988735 shutdown_connections 2023-07-06T03:52:02.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.757+0000 7f589d3ff700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f588406ace0 0x7f588406d190 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:02.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.757+0000 7f589d3ff700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589815e0d0 0x7f58980b4d40 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:02.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.757+0000 7f589d3ff700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5898071820 0x7f58981f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:02.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.757+0000 7f589d3ff700 1 --2- 172.21.15.97:0/832988735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589815c740 0x7f5898147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.757+0000 7f589d3ff700 1 -- 172.21.15.97:0/832988735 >> 172.21.15.97:0/832988735 conn(0x7f58980b2280 msgr2=0x7f58980b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.758+0000 7f589d3ff700 1 -- 172.21.15.97:0/832988735 shutdown_connections 2023-07-06T03:52:02.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:02.758+0000 7f589d3ff700 1 -- 172.21.15.97:0/832988735 wait complete. 2023-07-06T03:52:02.778 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:52:02.778 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:52:05.112 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:52:05.117 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:52:07.780 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:52:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.982+0000 7f78583ff700 1 Processor -- start 2023-07-06T03:52:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.983+0000 7f78583ff700 1 -- start start 2023-07-06T03:52:07.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.984+0000 7f78583ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7850158f20 0x7f78500caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:07.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.984+0000 7f78583ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f78501592f0 0x7f78500b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:07.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.984+0000 7f78583ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f785015cc40 con 0x7f785006d860 2023-07-06T03:52:07.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.984+0000 7f78583ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f785015cf20 con 0x7f78501592f0 2023-07-06T03:52:07.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.984+0000 7f78583ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785015d200 con 0x7f7850158f20 2023-07-06T03:52:07.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.984+0000 7f785599a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f78501592f0 0x7f78500b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:07.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.984+0000 7f785699c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f785006d860 0x7f78500cd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55666/0 (socket says 172.21.15.97:55666) 2023-07-06T03:52:07.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.984+0000 7f785699c700 1 -- 172.21.15.97:0/3037682723 learned_addr learned my addr 172.21.15.97:0/3037682723 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.984+0000 7f785619b700 1 --2- 172.21.15.97:0/3037682723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7850158f20 0x7f78500caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.985+0000 7f785599a700 1 -- 172.21.15.97:0/3037682723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7850158f20 msgr2=0x7f78500caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.985+0000 7f785599a700 1 --2- 172.21.15.97:0/3037682723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7850158f20 0x7f78500caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.985+0000 7f785599a700 1 -- 172.21.15.97:0/3037682723 >> v1:172.21.15.97:6789/0 conn(0x7f785006d860 legacy=0x7f78500cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:07.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.985+0000 7f785599a700 1 -- 172.21.15.97:0/3037682723 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785015d4e0 con 0x7f78501592f0 2023-07-06T03:52:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.985+0000 7f7855199700 1 -- 172.21.15.97:0/3037682723 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918269466 0 0) 0x7f785015cc40 con 0x7f785006d860 2023-07-06T03:52:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.985+0000 7f785619b700 1 --2- 172.21.15.97:0/3037682723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7850158f20 0x7f78500caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.985+0000 7f785599a700 1 --2- 172.21.15.97:0/3037682723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78501592f0 0x7f78500b0560 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7840009840 tx=0x7f7840009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809a2dac11f2c8e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.985+0000 7f7855199700 1 -- 172.21.15.97:0/3037682723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840011830 con 0x7f78501592f0 2023-07-06T03:52:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.985+0000 7f7855199700 1 -- 172.21.15.97:0/3037682723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7840011990 con 0x7f78501592f0 2023-07-06T03:52:07.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.985+0000 7f7855199700 1 -- 172.21.15.97:0/3037682723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840011d20 con 0x7f78501592f0 2023-07-06T03:52:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.986+0000 7f78583ff700 1 -- 172.21.15.97:0/3037682723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78501592f0 msgr2=0x7f78500b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.986+0000 7f78583ff700 1 --2- 172.21.15.97:0/3037682723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78501592f0 0x7f78500b0560 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7840009840 tx=0x7f7840009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.986+0000 7f78583ff700 1 -- 172.21.15.97:0/3037682723 shutdown_connections 2023-07-06T03:52:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.986+0000 7f78583ff700 1 --2- 172.21.15.97:0/3037682723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7850158f20 0x7f78500caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.986+0000 7f78583ff700 1 --2- 172.21.15.97:0/3037682723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78501592f0 0x7f78500b0560 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.986+0000 7f78583ff700 1 -- 172.21.15.97:0/3037682723 >> 172.21.15.97:0/3037682723 conn(0x7f78500ce8c0 msgr2=0x7f78500cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.986+0000 7f78583ff700 1 -- 172.21.15.97:0/3037682723 shutdown_connections 2023-07-06T03:52:07.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.986+0000 7f78583ff700 1 -- 172.21.15.97:0/3037682723 wait complete. 2023-07-06T03:52:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.987+0000 7f78583ff700 1 Processor -- start 2023-07-06T03:52:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.988+0000 7f78583ff700 1 -- start start 2023-07-06T03:52:07.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.988+0000 7f78583ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f785006d860 0x7f78500b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:07.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.988+0000 7f78583ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7850158f20 0x7f78500b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:07.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f78583ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78501592f0 0x7f7850150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:07.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f78583ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78500b4730 con 0x7f7850158f20 2023-07-06T03:52:07.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f78583ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785015c3d0 con 0x7f78501592f0 2023-07-06T03:52:07.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f78583ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78500b1a70 con 0x7f785006d860 2023-07-06T03:52:07.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785599a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7850158f20 0x7f78500b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:07.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785599a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7850158f20 0x7f78500b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41932/0 (socket says 172.21.15.97:41932) 2023-07-06T03:52:07.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785599a700 1 -- 172.21.15.97:0/3422784631 learned_addr learned my addr 172.21.15.97:0/3422784631 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:07.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785619b700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f785006d860 0x7f78500b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:07.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785699c700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78501592f0 0x7f7850150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:07.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785599a700 1 -- 172.21.15.97:0/3422784631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f785006d860 msgr2=0x7f78500b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:07.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785599a700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f785006d860 0x7f78500b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:07.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785599a700 1 -- 172.21.15.97:0/3422784631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78501592f0 msgr2=0x7f7850150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:07.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785599a700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78501592f0 0x7f7850150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:07.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785599a700 1 -- 172.21.15.97:0/3422784631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400094a0 con 0x7f7850158f20 2023-07-06T03:52:07.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785699c700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78501592f0 0x7f7850150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:07.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.989+0000 7f785599a700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7850158f20 0x7f78500b2550 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f7840014040 tx=0x7f7840004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:07.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.990+0000 7f7846ffd700 1 -- 172.21.15.97:0/3422784631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400045f0 con 0x7f7850158f20 2023-07-06T03:52:07.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.990+0000 7f7846ffd700 1 -- 172.21.15.97:0/3422784631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784001e070 con 0x7f7850158f20 2023-07-06T03:52:07.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.990+0000 7f78583ff700 1 -- 172.21.15.97:0/3422784631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78500b1d50 con 0x7f7850158f20 2023-07-06T03:52:07.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.990+0000 7f7846ffd700 1 -- 172.21.15.97:0/3422784631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840003ae0 con 0x7f7850158f20 2023-07-06T03:52:07.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.990+0000 7f78583ff700 1 -- 172.21.15.97:0/3422784631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785015c0c0 con 0x7f7850158f20 2023-07-06T03:52:07.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.991+0000 7f78583ff700 1 -- 172.21.15.97:0/3422784631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78500b18d0 con 0x7f7850158f20 2023-07-06T03:52:07.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.992+0000 7f7846ffd700 1 -- 172.21.15.97:0/3422784631 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7840003c40 con 0x7f7850158f20 2023-07-06T03:52:07.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.993+0000 7f7846ffd700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f783406ace0 0x7f783406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.993+0000 7f7846ffd700 1 -- 172.21.15.97:0/3422784631 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f784008a680 con 0x7f7850158f20 2023-07-06T03:52:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.993+0000 7f785619b700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f783406ace0 0x7f783406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:07.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.993+0000 7f785619b700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f783406ace0 0x7f783406d190 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f784c006fd0 tx=0x7f784c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:07.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:07.997+0000 7f7846ffd700 1 -- 172.21.15.97:0/3422784631 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f78500b18d0 con 0x7f7850158f20 2023-07-06T03:52:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.232+0000 7f78583ff700 1 -- 172.21.15.97:0/3422784631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f78500b2270 con 0x7f7850158f20 2023-07-06T03:52:08.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.233+0000 7f7846ffd700 1 -- 172.21.15.97:0/3422784631 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f78500b2270 con 0x7f7850158f20 2023-07-06T03:52:08.236 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:08.236 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:52:08.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 -- 172.21.15.97:0/3422784631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f783406ace0 msgr2=0x7f783406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:08.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f783406ace0 0x7f783406d190 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f784c006fd0 tx=0x7f784c008040 comp rx=0 tx=0).stop 2023-07-06T03:52:08.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 -- 172.21.15.97:0/3422784631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7850158f20 msgr2=0x7f78500b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:08.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7850158f20 0x7f78500b2550 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f7840014040 tx=0x7f7840004750 comp rx=0 tx=0).stop 2023-07-06T03:52:08.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 -- 172.21.15.97:0/3422784631 shutdown_connections 2023-07-06T03:52:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f783406ace0 0x7f783406d190 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78501592f0 0x7f7850150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f785006d860 0x7f78500b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 --2- 172.21.15.97:0/3422784631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7850158f20 0x7f78500b2550 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 -- 172.21.15.97:0/3422784631 >> 172.21.15.97:0/3422784631 conn(0x7f78500ce8c0 msgr2=0x7f78501527f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 -- 172.21.15.97:0/3422784631 shutdown_connections 2023-07-06T03:52:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.235+0000 7f78583ff700 1 -- 172.21.15.97:0/3422784631 wait complete. 2023-07-06T03:52:08.255 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:52:08.256 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:52:08.256 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:52:08.355 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:52:08.466 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:52:08.466 INFO:tasks.cephfs.mount:Terminating background process 2023-07-06T03:52:08.478 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:52:08.479 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:52:08.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:52:08.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.799+0000 7fe29dfff700 1 Processor -- start 2023-07-06T03:52:08.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.799+0000 7fe29dfff700 1 -- start start 2023-07-06T03:52:08.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.800+0000 7fe29dfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe2980cd390 0x7fe298160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:08.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.800+0000 7fe29dfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2980cdae0 con 0x7fe29815c520 2023-07-06T03:52:08.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.800+0000 7fe29dfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2980b6640 con 0x7fe298160dd0 2023-07-06T03:52:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.800+0000 7fe29dfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2980b6920 con 0x7fe2980cd390 2023-07-06T03:52:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.800+0000 7fe2977fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe29815c520 0x7fe298160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55678/0 (socket says 172.21.15.97:55678) 2023-07-06T03:52:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.800+0000 7fe2977fe700 1 -- 172.21.15.97:0/4091878791 learned_addr learned my addr 172.21.15.97:0/4091878791 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe296ffd700 1 --2- 172.21.15.97:0/4091878791 >> v2:172.21.15.146:3300/0 conn(0x7fe2980cd390 0x7fe298160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144392714 0 0) 0x7fe2980cdae0 con 0x7fe29815c520 2023-07-06T03:52:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2800035b0 con 0x7fe29815c520 2023-07-06T03:52:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3548992288 0 0) 0x7fe2800035b0 con 0x7fe29815c520 2023-07-06T03:52:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 >> v2:172.21.15.146:3300/0 conn(0x7fe2980cd390 msgr2=0x7fe298160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 --2- 172.21.15.97:0/4091878791 >> v2:172.21.15.146:3300/0 conn(0x7fe2980cd390 0x7fe298160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 >> v1:172.21.15.146:6789/0 conn(0x7fe298160dd0 legacy=0x7fe2980b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2980b6c00 con 0x7fe29815c520 2023-07-06T03:52:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fe288001670 con 0x7fe29815c520 2023-07-06T03:52:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 >> v1:172.21.15.97:6789/0 conn(0x7fe29815c520 legacy=0x7fe298160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe280003a50 0x7fe280003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 0x7fe280004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe280005200 0x7fe28000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2980b6920 con 0x7fe280004850 2023-07-06T03:52:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28000c0e0 con 0x7fe280005200 2023-07-06T03:52:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.801+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28000c2a0 con 0x7fe280003a50 2023-07-06T03:52:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe296ffd700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 0x7fe280004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe2977fe700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe280003a50 0x7fe280003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe297fff700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe280005200 0x7fe28000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe296ffd700 1 -- 172.21.15.97:0/4091878791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe280003a50 msgr2=0x7fe280003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe296ffd700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe280003a50 0x7fe280003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe296ffd700 1 -- 172.21.15.97:0/4091878791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe280005200 msgr2=0x7fe28000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe296ffd700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe280005200 0x7fe28000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe296ffd700 1 -- 172.21.15.97:0/4091878791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2800035b0 con 0x7fe280004850 2023-07-06T03:52:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe297fff700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe280005200 0x7fe28000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe297fff700 1 -- 172.21.15.97:0/4091878791 reap_dead start 2023-07-06T03:52:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe296ffd700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 0x7fe280004cc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009b30 tx=0x7fe280005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a408bb5fe3e26d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00e6c0 con 0x7fe280004850 2023-07-06T03:52:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c00e820 con 0x7fe280004850 2023-07-06T03:52:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.802+0000 7fe2967fc700 1 -- 172.21.15.97:0/4091878791 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00eb90 con 0x7fe280004850 2023-07-06T03:52:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.803+0000 7fe29dfff700 1 -- 172.21.15.97:0/4091878791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 msgr2=0x7fe280004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.803+0000 7fe29dfff700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 0x7fe280004cc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009b30 tx=0x7fe280005880 comp rx=0 tx=0).stop 2023-07-06T03:52:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.803+0000 7fe29dfff700 1 -- 172.21.15.97:0/4091878791 shutdown_connections 2023-07-06T03:52:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.803+0000 7fe29dfff700 1 --2- 172.21.15.97:0/4091878791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 0x7fe280004cc0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.803+0000 7fe29dfff700 1 -- 172.21.15.97:0/4091878791 >> 172.21.15.97:0/4091878791 conn(0x7fe298071790 msgr2=0x7fe298071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.803+0000 7fe29dfff700 1 -- 172.21.15.97:0/4091878791 shutdown_connections 2023-07-06T03:52:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.803+0000 7fe29dfff700 1 -- 172.21.15.97:0/4091878791 wait complete. 2023-07-06T03:52:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.805+0000 7fe29dfff700 1 Processor -- start 2023-07-06T03:52:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.805+0000 7fe29dfff700 1 -- start start 2023-07-06T03:52:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe29dfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 0x7fe29815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe29dfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe29815f6d0 0x7fe29815aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe29dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2981f5170 0x7fe2981e5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe29dfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29815f2b0 con 0x7fe280004850 2023-07-06T03:52:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe29dfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29815b420 con 0x7fe29815f6d0 2023-07-06T03:52:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe29dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29815e630 con 0x7fe2981f5170 2023-07-06T03:52:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe2977fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 0x7fe29815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe2977fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 0x7fe29815dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41960/0 (socket says 172.21.15.97:41960) 2023-07-06T03:52:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe2977fe700 1 -- 172.21.15.97:0/2602182070 learned_addr learned my addr 172.21.15.97:0/2602182070 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:08.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe296ffd700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe29815f6d0 0x7fe29815aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:08.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe297fff700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2981f5170 0x7fe2981e5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:08.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe2977fe700 1 -- 172.21.15.97:0/2602182070 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2981f5170 msgr2=0x7fe2981e5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:08.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe2977fe700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2981f5170 0x7fe2981e5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:08.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe2977fe700 1 -- 172.21.15.97:0/2602182070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe29815f6d0 msgr2=0x7fe29815aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:08.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe2977fe700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe29815f6d0 0x7fe29815aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:08.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe2977fe700 1 -- 172.21.15.97:0/2602182070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29815e910 con 0x7fe280004850 2023-07-06T03:52:08.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe296ffd700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe29815f6d0 0x7fe29815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.806+0000 7fe297fff700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2981f5170 0x7fe2981e5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.807+0000 7fe2977fe700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 0x7fe29815dfa0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fe288007fd0 tx=0x7fe288007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.807+0000 7fe27ffff700 1 -- 172.21.15.97:0/2602182070 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880014e0 con 0x7fe280004850 2023-07-06T03:52:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.807+0000 7fe27ffff700 1 -- 172.21.15.97:0/2602182070 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800f6c0 con 0x7fe280004850 2023-07-06T03:52:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.807+0000 7fe27ffff700 1 -- 172.21.15.97:0/2602182070 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800fa90 con 0x7fe280004850 2023-07-06T03:52:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.807+0000 7fe29dfff700 1 -- 172.21.15.97:0/2602182070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29815ebf0 con 0x7fe280004850 2023-07-06T03:52:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.807+0000 7fe29dfff700 1 -- 172.21.15.97:0/2602182070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2981f5610 con 0x7fe280004850 2023-07-06T03:52:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.808+0000 7fe29dfff700 1 -- 172.21.15.97:0/2602182070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29804e340 con 0x7fe280004850 2023-07-06T03:52:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.809+0000 7fe27ffff700 1 -- 172.21.15.97:0/2602182070 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe28800fbf0 con 0x7fe280004850 2023-07-06T03:52:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.810+0000 7fe27ffff700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe28406ace0 0x7fe28406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:08.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.810+0000 7fe27ffff700 1 -- 172.21.15.97:0/2602182070 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe288089570 con 0x7fe280004850 2023-07-06T03:52:08.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.810+0000 7fe296ffd700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe28406ace0 0x7fe28406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:08.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.811+0000 7fe296ffd700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe28406ace0 0x7fe28406d190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0090b0 tx=0x7fe28c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:08.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:08.814+0000 7fe27ffff700 1 -- 172.21.15.97:0/2602182070 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe2880580a0 con 0x7fe280004850 2023-07-06T03:52:09.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.017+0000 7fe29dfff700 1 -- 172.21.15.97:0/2602182070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2980b4f40 con 0x7fe280004850 2023-07-06T03:52:09.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.018+0000 7fe27ffff700 1 -- 172.21.15.97:0/2602182070 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18320 (secure 0 0 0) 0x7fe28800fea0 con 0x7fe280004850 2023-07-06T03:52:09.021 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:09.021 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":74,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:49:35.835648+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:49:32.371992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-06T03:49:33.281260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:52:09.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 -- 172.21.15.97:0/2602182070 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe28406ace0 msgr2=0x7fe28406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe28406ace0 0x7fe28406d190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0090b0 tx=0x7fe28c008030 comp rx=0 tx=0).stop 2023-07-06T03:52:09.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 -- 172.21.15.97:0/2602182070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 msgr2=0x7fe29815dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 0x7fe29815dfa0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fe288007fd0 tx=0x7fe288007620 comp rx=0 tx=0).stop 2023-07-06T03:52:09.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 -- 172.21.15.97:0/2602182070 shutdown_connections 2023-07-06T03:52:09.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe28406ace0 0x7fe28406d190 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2981f5170 0x7fe2981e5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe280004850 0x7fe29815dfa0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 --2- 172.21.15.97:0/2602182070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe29815f6d0 0x7fe29815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 -- 172.21.15.97:0/2602182070 >> 172.21.15.97:0/2602182070 conn(0x7fe298071790 msgr2=0x7fe298160180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:09.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 -- 172.21.15.97:0/2602182070 shutdown_connections 2023-07-06T03:52:09.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.021+0000 7fe29dfff700 1 -- 172.21.15.97:0/2602182070 wait complete. 2023-07-06T03:52:09.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:49:32.371992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:49:33.281260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:52:09.042 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:52:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.242+0000 7f44261ff700 1 Processor -- start 2023-07-06T03:52:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.242+0000 7f44261ff700 1 -- start start 2023-07-06T03:52:09.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.243+0000 7f44261ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f442006b360 0x7f44200b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.243+0000 7f44261ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44200d2210 0x7f44200d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.243+0000 7f44261ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44200cff90 con 0x7f442015c610 2023-07-06T03:52:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.243+0000 7f44261ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44200d0780 con 0x7f442006b360 2023-07-06T03:52:09.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.243+0000 7f44261ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442015c9e0 con 0x7f44200d2210 2023-07-06T03:52:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.244+0000 7f441effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f442006b360 0x7f44200b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.244+0000 7f441ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44200d2210 0x7f44200d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.244+0000 7f441effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f442006b360 0x7f44200b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:51880/0 (socket says 172.21.15.97:51880) 2023-07-06T03:52:09.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.244+0000 7f441effd700 1 -- 172.21.15.97:0/296423529 learned_addr learned my addr 172.21.15.97:0/296423529 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:09.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.244+0000 7f441effd700 1 -- 172.21.15.97:0/296423529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44200d2210 msgr2=0x7f44200d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:09.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.244+0000 7f441effd700 1 --2- 172.21.15.97:0/296423529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44200d2210 0x7f44200d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.244+0000 7f441effd700 1 -- 172.21.15.97:0/296423529 >> v1:172.21.15.146:6789/0 conn(0x7f442015c610 legacy=0x7f44200b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.244+0000 7f441effd700 1 -- 172.21.15.97:0/296423529 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200b5650 con 0x7f442006b360 2023-07-06T03:52:09.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.244+0000 7f441effd700 1 --2- 172.21.15.97:0/296423529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f442006b360 0x7f44200b4ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f44140098f0 tx=0x7f4414009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b2ac1521b4a2b67 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:09.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.245+0000 7f441e7fc700 1 -- 172.21.15.97:0/296423529 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414011830 con 0x7f442006b360 2023-07-06T03:52:09.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.245+0000 7f441e7fc700 1 -- 172.21.15.97:0/296423529 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4414011990 con 0x7f442006b360 2023-07-06T03:52:09.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.245+0000 7f441e7fc700 1 -- 172.21.15.97:0/296423529 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414011d20 con 0x7f442006b360 2023-07-06T03:52:09.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.245+0000 7f44261ff700 1 -- 172.21.15.97:0/296423529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f442006b360 msgr2=0x7f44200b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.245+0000 7f44261ff700 1 --2- 172.21.15.97:0/296423529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f442006b360 0x7f44200b4ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f44140098f0 tx=0x7f4414009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:09.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.245+0000 7f44261ff700 1 -- 172.21.15.97:0/296423529 shutdown_connections 2023-07-06T03:52:09.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.245+0000 7f44261ff700 1 --2- 172.21.15.97:0/296423529 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f44200d2210 0x7f44200d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.245+0000 7f44261ff700 1 --2- 172.21.15.97:0/296423529 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f442006b360 0x7f44200b4ec0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.245+0000 7f44261ff700 1 -- 172.21.15.97:0/296423529 >> 172.21.15.97:0/296423529 conn(0x7f442006a6c0 msgr2=0x7f442006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.245+0000 7f44261ff700 1 -- 172.21.15.97:0/296423529 shutdown_connections 2023-07-06T03:52:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.245+0000 7f44261ff700 1 -- 172.21.15.97:0/296423529 wait complete. 2023-07-06T03:52:09.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.247+0000 7f44261ff700 1 Processor -- start 2023-07-06T03:52:09.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.247+0000 7f44261ff700 1 -- start start 2023-07-06T03:52:09.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.248+0000 7f44261ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f442006b360 0x7f442015f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.248+0000 7f44261ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44200d2210 0x7f442014fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.248+0000 7f44261ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442015c610 0x7f442015b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.248+0000 7f44261ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442015b0f0 con 0x7f442015c610 2023-07-06T03:52:09.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.248+0000 7f44261ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442015ab20 con 0x7f44200d2210 2023-07-06T03:52:09.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.248+0000 7f44261ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442015bac0 con 0x7f442006b360 2023-07-06T03:52:09.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.248+0000 7f441ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442015c610 0x7f442015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.248+0000 7f441ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442015c610 0x7f442015b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41964/0 (socket says 172.21.15.97:41964) 2023-07-06T03:52:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.248+0000 7f441ffff700 1 -- 172.21.15.97:0/2916727689 learned_addr learned my addr 172.21.15.97:0/2916727689 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.248+0000 7f441effd700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44200d2210 0x7f442014fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.248+0000 7f441f7fe700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f442006b360 0x7f442015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.249+0000 7f441effd700 1 -- 172.21.15.97:0/2916727689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f442006b360 msgr2=0x7f442015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.249+0000 7f441effd700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f442006b360 0x7f442015f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.249+0000 7f441effd700 1 -- 172.21.15.97:0/2916727689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442015c610 msgr2=0x7f442015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.249+0000 7f441effd700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442015c610 0x7f442015b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.249+0000 7f441effd700 1 -- 172.21.15.97:0/2916727689 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414009550 con 0x7f44200d2210 2023-07-06T03:52:09.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.249+0000 7f441effd700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44200d2210 0x7f442014fa70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4414016000 tx=0x7f4414004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.249+0000 7f4407fff700 1 -- 172.21.15.97:0/2916727689 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414004400 con 0x7f44200d2210 2023-07-06T03:52:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.250+0000 7f4407fff700 1 -- 172.21.15.97:0/2916727689 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441401f070 con 0x7f44200d2210 2023-07-06T03:52:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.250+0000 7f44261ff700 1 -- 172.21.15.97:0/2916727689 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442015cbc0 con 0x7f44200d2210 2023-07-06T03:52:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.250+0000 7f4407fff700 1 -- 172.21.15.97:0/2916727689 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140038e0 con 0x7f44200d2210 2023-07-06T03:52:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.250+0000 7f44261ff700 1 -- 172.21.15.97:0/2916727689 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442015ad80 con 0x7f44200d2210 2023-07-06T03:52:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.251+0000 7f44261ff700 1 -- 172.21.15.97:0/2916727689 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442004e340 con 0x7f44200d2210 2023-07-06T03:52:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.252+0000 7f4407fff700 1 -- 172.21.15.97:0/2916727689 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4414021030 con 0x7f44200d2210 2023-07-06T03:52:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.253+0000 7f4407fff700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f440c06ace0 0x7f440c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.253+0000 7f4407fff700 1 -- 172.21.15.97:0/2916727689 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f441408ac30 con 0x7f44200d2210 2023-07-06T03:52:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.253+0000 7f441f7fe700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f440c06ace0 0x7f440c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.254+0000 7f441f7fe700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f440c06ace0 0x7f440c06d190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4410007fd0 tx=0x7f4410009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.257+0000 7f4407fff700 1 -- 172.21.15.97:0/2916727689 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4414055e80 con 0x7f44200d2210 2023-07-06T03:52:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.453+0000 7f44261ff700 1 -- 172.21.15.97:0/2916727689 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f44200d3b30 con 0x7f44200d2210 2023-07-06T03:52:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.455+0000 7f4407fff700 1 -- 172.21.15.97:0/2916727689 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44140594a0 con 0x7f44200d2210 2023-07-06T03:52:09.457 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:52:09.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 -- 172.21.15.97:0/2916727689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f440c06ace0 msgr2=0x7f440c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f440c06ace0 0x7f440c06d190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4410007fd0 tx=0x7f4410009040 comp rx=0 tx=0).stop 2023-07-06T03:52:09.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 -- 172.21.15.97:0/2916727689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44200d2210 msgr2=0x7f442014fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44200d2210 0x7f442014fa70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4414016000 tx=0x7f4414004750 comp rx=0 tx=0).stop 2023-07-06T03:52:09.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 -- 172.21.15.97:0/2916727689 shutdown_connections 2023-07-06T03:52:09.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f440c06ace0 0x7f440c06d190 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442015c610 0x7f442015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f442006b360 0x7f442015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 --2- 172.21.15.97:0/2916727689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44200d2210 0x7f442014fa70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 -- 172.21.15.97:0/2916727689 >> 172.21.15.97:0/2916727689 conn(0x7f442006a6c0 msgr2=0x7f44200d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 -- 172.21.15.97:0/2916727689 shutdown_connections 2023-07-06T03:52:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.456+0000 7f44261ff700 1 -- 172.21.15.97:0/2916727689 wait complete. 2023-07-06T03:52:09.476 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:52:09.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.671+0000 7f87665ff700 1 Processor -- start 2023-07-06T03:52:09.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.671+0000 7f87665ff700 1 -- start start 2023-07-06T03:52:09.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.672+0000 7f87665ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f876006b360 0x7f87600b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.673+0000 7f87665ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87600cff90 con 0x7f87600d2210 2023-07-06T03:52:09.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.673+0000 7f87665ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87600b5650 con 0x7f876015c610 2023-07-06T03:52:09.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.673+0000 7f87665ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87600d0780 con 0x7f876006b360 2023-07-06T03:52:09.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.673+0000 7f8764b9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f87600d2210 0x7f87600d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55682/0 (socket says 172.21.15.97:55682) 2023-07-06T03:52:09.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.673+0000 7f8764b9c700 1 -- 172.21.15.97:0/1623940095 learned_addr learned my addr 172.21.15.97:0/1623940095 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:09.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.673+0000 7f8757fff700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f876006b360 0x7f87600b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.673+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 135433555 0 0) 0x7f87600cff90 con 0x7f87600d2210 2023-07-06T03:52:09.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.673+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f874c0035b0 con 0x7f87600d2210 2023-07-06T03:52:09.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.673+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534239426 0 0) 0x7f87600b5650 con 0x7f876015c610 2023-07-06T03:52:09.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.673+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87600cff90 con 0x7f876015c610 2023-07-06T03:52:09.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.673+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2220998098 0 0) 0x7f874c0035b0 con 0x7f87600d2210 2023-07-06T03:52:09.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f876006b360 msgr2=0x7f87600b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:09.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f876006b360 0x7f87600b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 >> v1:172.21.15.146:6789/0 conn(0x7f876015c610 legacy=0x7f87600b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87600b5400 con 0x7f87600d2210 2023-07-06T03:52:09.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f87580019f0 con 0x7f87600d2210 2023-07-06T03:52:09.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 >> v1:172.21.15.97:6789/0 conn(0x7f87600d2210 legacy=0x7f87600d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f874c003bc0 0x7f874c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f874c004850 0x7f874c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874c005200 0x7f874c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87600d0780 con 0x7f874c005200 2023-07-06T03:52:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c00bfd0 con 0x7f874c004850 2023-07-06T03:52:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c00c190 con 0x7f874c003bc0 2023-07-06T03:52:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f8757fff700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f876006b360 0x7f87600b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f8764b9c700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874c005200 0x7f874c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f875ffff700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f874c003bc0 0x7f874c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f8757fff700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f874c004850 0x7f874c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f8764b9c700 1 -- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f874c003bc0 msgr2=0x7f874c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f8764b9c700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f874c003bc0 0x7f874c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f8764b9c700 1 -- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f874c004850 msgr2=0x7f874c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f8764b9c700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f874c004850 0x7f874c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f8764b9c700 1 -- 172.21.15.97:0/1623940095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c0035b0 con 0x7f874c005200 2023-07-06T03:52:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.674+0000 7f8757fff700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f874c004850 0x7f874c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.675+0000 7f8764b9c700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874c005200 0x7f874c00ba10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f875800a7c0 tx=0x7f874c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6cedc7952d96d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:09.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.675+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758012600 con 0x7f874c005200 2023-07-06T03:52:09.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.675+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758012760 con 0x7f874c005200 2023-07-06T03:52:09.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.675+0000 7f875f7fe700 1 -- 172.21.15.97:0/1623940095 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758012ad0 con 0x7f874c005200 2023-07-06T03:52:09.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.675+0000 7f87665ff700 1 -- 172.21.15.97:0/1623940095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874c005200 msgr2=0x7f874c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.675+0000 7f87665ff700 1 --2- 172.21.15.97:0/1623940095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874c005200 0x7f874c00ba10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f875800a7c0 tx=0x7f874c005880 comp rx=0 tx=0).stop 2023-07-06T03:52:09.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.675+0000 7f8764b9c700 1 -- 172.21.15.97:0/1623940095 reap_dead start 2023-07-06T03:52:09.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.675+0000 7f87665ff700 1 -- 172.21.15.97:0/1623940095 shutdown_connections 2023-07-06T03:52:09.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.675+0000 7f87665ff700 1 -- 172.21.15.97:0/1623940095 >> 172.21.15.97:0/1623940095 conn(0x7f876006a6c0 msgr2=0x7f876006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:09.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.676+0000 7f87665ff700 1 -- 172.21.15.97:0/1623940095 shutdown_connections 2023-07-06T03:52:09.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.676+0000 7f87665ff700 1 -- 172.21.15.97:0/1623940095 wait complete. 2023-07-06T03:52:09.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.677+0000 7f87665ff700 1 Processor -- start 2023-07-06T03:52:09.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.677+0000 7f87665ff700 1 -- start start 2023-07-06T03:52:09.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.678+0000 7f87665ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f874c005200 0x7f87600cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.678+0000 7f87665ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87600cc430 0x7f87600cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.678+0000 7f87665ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87600c9e00 0x7f87600ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.678+0000 7f87665ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87600cc010 con 0x7f87600c9e00 2023-07-06T03:52:09.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.678+0000 7f87665ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876015efb0 con 0x7f874c005200 2023-07-06T03:52:09.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.678+0000 7f87665ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87600ca7b0 con 0x7f87600cc430 2023-07-06T03:52:09.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f8764b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87600c9e00 0x7f87600ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f8764b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87600c9e00 0x7f87600ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41986/0 (socket says 172.21.15.97:41986) 2023-07-06T03:52:09.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f8764b9c700 1 -- 172.21.15.97:0/2569887731 learned_addr learned my addr 172.21.15.97:0/2569887731 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:09.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f875ffff700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f874c005200 0x7f87600cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f8757fff700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87600cc430 0x7f87600cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f8764b9c700 1 -- 172.21.15.97:0/2569887731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87600cc430 msgr2=0x7f87600cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f8764b9c700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87600cc430 0x7f87600cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f8764b9c700 1 -- 172.21.15.97:0/2569887731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f874c005200 msgr2=0x7f87600cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f8764b9c700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f874c005200 0x7f87600cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f8764b9c700 1 -- 172.21.15.97:0/2569887731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87601f2a80 con 0x7f87600c9e00 2023-07-06T03:52:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f8757fff700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87600cc430 0x7f87600cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f8764b9c700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87600c9e00 0x7f87600ca270 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8758001c30 tx=0x7f8758001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f875d7fa700 1 -- 172.21.15.97:0/2569887731 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875801e600 con 0x7f87600c9e00 2023-07-06T03:52:09.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f875d7fa700 1 -- 172.21.15.97:0/2569887731 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875801e7a0 con 0x7f87600c9e00 2023-07-06T03:52:09.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.679+0000 7f87665ff700 1 -- 172.21.15.97:0/2569887731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87600cb670 con 0x7f87600c9e00 2023-07-06T03:52:09.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.680+0000 7f875d7fa700 1 -- 172.21.15.97:0/2569887731 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875801eb50 con 0x7f87600c9e00 2023-07-06T03:52:09.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.680+0000 7f87665ff700 1 -- 172.21.15.97:0/2569887731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87600cbc30 con 0x7f87600c9e00 2023-07-06T03:52:09.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.680+0000 7f87665ff700 1 -- 172.21.15.97:0/2569887731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87600cc150 con 0x7f87600c9e00 2023-07-06T03:52:09.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.681+0000 7f875d7fa700 1 -- 172.21.15.97:0/2569887731 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8758019030 con 0x7f87600c9e00 2023-07-06T03:52:09.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.682+0000 7f875d7fa700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f874006ace0 0x7f874006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:09.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.682+0000 7f875d7fa700 1 -- 172.21.15.97:0/2569887731 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f875808b200 con 0x7f87600c9e00 2023-07-06T03:52:09.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.683+0000 7f875ffff700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f874006ace0 0x7f874006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:09.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.683+0000 7f875ffff700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f874006ace0 0x7f874006d190 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8750007fd0 tx=0x7f8750009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:09.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.686+0000 7f875d7fa700 1 -- 172.21.15.97:0/2569887731 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f87600cc150 con 0x7f87600c9e00 2023-07-06T03:52:09.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.882+0000 7f87665ff700 1 -- 172.21.15.97:0/2569887731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f87600cc150 con 0x7f87600c9e00 2023-07-06T03:52:09.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.883+0000 7f875d7fa700 1 -- 172.21.15.97:0/2569887731 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7f87600cc150 con 0x7f87600c9e00 2023-07-06T03:52:09.886 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:52:09.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.885+0000 7f87665ff700 1 -- 172.21.15.97:0/2569887731 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f874006ace0 msgr2=0x7f874006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.885+0000 7f87665ff700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f874006ace0 0x7f874006d190 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8750007fd0 tx=0x7f8750009040 comp rx=0 tx=0).stop 2023-07-06T03:52:09.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.885+0000 7f87665ff700 1 -- 172.21.15.97:0/2569887731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87600c9e00 msgr2=0x7f87600ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:09.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.885+0000 7f87665ff700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87600c9e00 0x7f87600ca270 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8758001c30 tx=0x7f8758001d00 comp rx=0 tx=0).stop 2023-07-06T03:52:09.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.885+0000 7f87665ff700 1 -- 172.21.15.97:0/2569887731 shutdown_connections 2023-07-06T03:52:09.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.885+0000 7f87665ff700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f874006ace0 0x7f874006d190 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.885+0000 7f87665ff700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87600c9e00 0x7f87600ca270 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.885+0000 7f87665ff700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f874c005200 0x7f87600cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.885+0000 7f87665ff700 1 --2- 172.21.15.97:0/2569887731 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f87600cc430 0x7f87600cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:09.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.886+0000 7f87665ff700 1 -- 172.21.15.97:0/2569887731 >> 172.21.15.97:0/2569887731 conn(0x7f876006a6c0 msgr2=0x7f87600b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:09.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.886+0000 7f87665ff700 1 -- 172.21.15.97:0/2569887731 shutdown_connections 2023-07-06T03:52:09.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:09.886+0000 7f87665ff700 1 -- 172.21.15.97:0/2569887731 wait complete. 2023-07-06T03:52:09.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:52:10.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.101+0000 7f5e909ff700 1 Processor -- start 2023-07-06T03:52:10.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.101+0000 7f5e909ff700 1 -- start start 2023-07-06T03:52:10.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.102+0000 7f5e909ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e8c06b360 0x7f5e8c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.102+0000 7f5e909ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8c0d2210 0x7f5e8c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.102+0000 7f5e909ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8c0cff90 con 0x7f5e8c15c610 2023-07-06T03:52:10.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.102+0000 7f5e909ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8c0d0780 con 0x7f5e8c06b360 2023-07-06T03:52:10.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.102+0000 7f5e909ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c15c9e0 con 0x7f5e8c0d2210 2023-07-06T03:52:10.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.103+0000 7f5e83fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e8c06b360 0x7f5e8c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.103+0000 7f5e8a59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5e8c15c610 0x7f5e8c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46530/0 (socket says 172.21.15.97:46530) 2023-07-06T03:52:10.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.103+0000 7f5e8a59c700 1 -- 172.21.15.97:0/711973469 learned_addr learned my addr 172.21.15.97:0/711973469 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:10.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.103+0000 7f5e8ad9d700 1 --2- 172.21.15.97:0/711973469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8c0d2210 0x7f5e8c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.103+0000 7f5e83fff700 1 -- 172.21.15.97:0/711973469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8c0d2210 msgr2=0x7f5e8c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:10.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.103+0000 7f5e83fff700 1 --2- 172.21.15.97:0/711973469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8c0d2210 0x7f5e8c0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.103+0000 7f5e83fff700 1 -- 172.21.15.97:0/711973469 >> v1:172.21.15.97:6789/0 conn(0x7f5e8c15c610 legacy=0x7f5e8c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.103+0000 7f5e83fff700 1 -- 172.21.15.97:0/711973469 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c0b5650 con 0x7f5e8c06b360 2023-07-06T03:52:10.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.103+0000 7f5e89d9b700 1 -- 172.21.15.97:0/711973469 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284577691 0 0) 0x7f5e8c0cff90 con 0x7f5e8c15c610 2023-07-06T03:52:10.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.103+0000 7f5e83fff700 1 --2- 172.21.15.97:0/711973469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e8c06b360 0x7f5e8c0b4ec0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009840 tx=0x7f5e74009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4046f722e69152 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:10.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.104+0000 7f5e89d9b700 1 -- 172.21.15.97:0/711973469 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74011830 con 0x7f5e8c06b360 2023-07-06T03:52:10.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.104+0000 7f5e89d9b700 1 -- 172.21.15.97:0/711973469 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e74011990 con 0x7f5e8c06b360 2023-07-06T03:52:10.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.104+0000 7f5e89d9b700 1 -- 172.21.15.97:0/711973469 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74011d20 con 0x7f5e8c06b360 2023-07-06T03:52:10.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.104+0000 7f5e909ff700 1 -- 172.21.15.97:0/711973469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e8c06b360 msgr2=0x7f5e8c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.104+0000 7f5e909ff700 1 --2- 172.21.15.97:0/711973469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e8c06b360 0x7f5e8c0b4ec0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009840 tx=0x7f5e74009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:10.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.104+0000 7f5e909ff700 1 -- 172.21.15.97:0/711973469 shutdown_connections 2023-07-06T03:52:10.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.104+0000 7f5e909ff700 1 --2- 172.21.15.97:0/711973469 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8c0d2210 0x7f5e8c0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.104+0000 7f5e909ff700 1 --2- 172.21.15.97:0/711973469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e8c06b360 0x7f5e8c0b4ec0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.104+0000 7f5e909ff700 1 -- 172.21.15.97:0/711973469 >> 172.21.15.97:0/711973469 conn(0x7f5e8c06a6c0 msgr2=0x7f5e8c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:10.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.104+0000 7f5e909ff700 1 -- 172.21.15.97:0/711973469 shutdown_connections 2023-07-06T03:52:10.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.104+0000 7f5e909ff700 1 -- 172.21.15.97:0/711973469 wait complete. 2023-07-06T03:52:10.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.106+0000 7f5e909ff700 1 Processor -- start 2023-07-06T03:52:10.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.106+0000 7f5e909ff700 1 -- start start 2023-07-06T03:52:10.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.107+0000 7f5e909ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8c06b360 0x7f5e8c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.107+0000 7f5e909ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e8c0d2210 0x7f5e8c1f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.107+0000 7f5e909ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c15c610 0x7f5e8c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.107+0000 7f5e909ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c15d390 con 0x7f5e8c0d2210 2023-07-06T03:52:10.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.107+0000 7f5e909ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c15d670 con 0x7f5e8c15c610 2023-07-06T03:52:10.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.107+0000 7f5e909ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c15d950 con 0x7f5e8c06b360 2023-07-06T03:52:10.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.108+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e8c0d2210 0x7f5e8c1f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.108+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e8c0d2210 0x7f5e8c1f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60642/0 (socket says 172.21.15.97:60642) 2023-07-06T03:52:10.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.108+0000 7f5e83fff700 1 -- 172.21.15.97:0/4041579863 learned_addr learned my addr 172.21.15.97:0/4041579863 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:10.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.108+0000 7f5e8a59c700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8c06b360 0x7f5e8c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.108+0000 7f5e8ad9d700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c15c610 0x7f5e8c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.108+0000 7f5e83fff700 1 -- 172.21.15.97:0/4041579863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8c06b360 msgr2=0x7f5e8c15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.108+0000 7f5e83fff700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8c06b360 0x7f5e8c15cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.108+0000 7f5e83fff700 1 -- 172.21.15.97:0/4041579863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c15c610 msgr2=0x7f5e8c15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.108+0000 7f5e83fff700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c15c610 0x7f5e8c15ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.108+0000 7f5e83fff700 1 -- 172.21.15.97:0/4041579863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e740094a0 con 0x7f5e8c0d2210 2023-07-06T03:52:10.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.108+0000 7f5e83fff700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e8c0d2210 0x7f5e8c1f1b80 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009b90 tx=0x7f5e74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:10.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.109+0000 7f5e82ffd700 1 -- 172.21.15.97:0/4041579863 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74004550 con 0x7f5e8c0d2210 2023-07-06T03:52:10.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.109+0000 7f5e82ffd700 1 -- 172.21.15.97:0/4041579863 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7401f070 con 0x7f5e8c0d2210 2023-07-06T03:52:10.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.109+0000 7f5e909ff700 1 -- 172.21.15.97:0/4041579863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c15dc30 con 0x7f5e8c0d2210 2023-07-06T03:52:10.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.109+0000 7f5e82ffd700 1 -- 172.21.15.97:0/4041579863 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e740038e0 con 0x7f5e8c0d2210 2023-07-06T03:52:10.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.109+0000 7f5e909ff700 1 -- 172.21.15.97:0/4041579863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c15d060 con 0x7f5e8c0d2210 2023-07-06T03:52:10.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.110+0000 7f5e909ff700 1 -- 172.21.15.97:0/4041579863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8c15e150 con 0x7f5e8c0d2210 2023-07-06T03:52:10.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.111+0000 7f5e82ffd700 1 -- 172.21.15.97:0/4041579863 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5e74003a40 con 0x7f5e8c0d2210 2023-07-06T03:52:10.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.112+0000 7f5e82ffd700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e6c06ace0 0x7f5e6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.112+0000 7f5e82ffd700 1 -- 172.21.15.97:0/4041579863 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e74021070 con 0x7f5e8c0d2210 2023-07-06T03:52:10.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.112+0000 7f5e8a59c700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e6c06ace0 0x7f5e6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.112+0000 7f5e8a59c700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e6c06ace0 0x7f5e6c06d190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000f60 tx=0x7f5e7c007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:10.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.116+0000 7f5e82ffd700 1 -- 172.21.15.97:0/4041579863 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5e8c15e150 con 0x7f5e8c0d2210 2023-07-06T03:52:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.310+0000 7f5e909ff700 1 -- 172.21.15.97:0/4041579863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e8c15e150 con 0x7f5e8c0d2210 2023-07-06T03:52:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.310+0000 7f5e82ffd700 1 -- 172.21.15.97:0/4041579863 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e8c15e150 con 0x7f5e8c0d2210 2023-07-06T03:52:10.313 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:52:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.312+0000 7f5e909ff700 1 -- 172.21.15.97:0/4041579863 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e6c06ace0 msgr2=0x7f5e6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.312+0000 7f5e909ff700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e6c06ace0 0x7f5e6c06d190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000f60 tx=0x7f5e7c007e70 comp rx=0 tx=0).stop 2023-07-06T03:52:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.312+0000 7f5e909ff700 1 -- 172.21.15.97:0/4041579863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e8c0d2210 msgr2=0x7f5e8c1f1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.312+0000 7f5e909ff700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e8c0d2210 0x7f5e8c1f1b80 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009b90 tx=0x7f5e74004750 comp rx=0 tx=0).stop 2023-07-06T03:52:10.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.312+0000 7f5e909ff700 1 -- 172.21.15.97:0/4041579863 shutdown_connections 2023-07-06T03:52:10.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.313+0000 7f5e909ff700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e6c06ace0 0x7f5e6c06d190 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.313+0000 7f5e909ff700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c15c610 0x7f5e8c15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.313+0000 7f5e909ff700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8c06b360 0x7f5e8c15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.313+0000 7f5e909ff700 1 --2- 172.21.15.97:0/4041579863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e8c0d2210 0x7f5e8c1f1b80 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.313+0000 7f5e909ff700 1 -- 172.21.15.97:0/4041579863 >> 172.21.15.97:0/4041579863 conn(0x7f5e8c06a6c0 msgr2=0x7f5e8c0b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.313+0000 7f5e909ff700 1 -- 172.21.15.97:0/4041579863 shutdown_connections 2023-07-06T03:52:10.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.313+0000 7f5e909ff700 1 -- 172.21.15.97:0/4041579863 wait complete. 2023-07-06T03:52:10.333 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:52:10.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.529+0000 7f27c45ff700 1 Processor -- start 2023-07-06T03:52:10.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.529+0000 7f27c45ff700 1 -- start start 2023-07-06T03:52:10.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.530+0000 7f27c45ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc06b360 0x7f27bc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.530+0000 7f27c45ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27bc0d2210 0x7f27bc0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.530+0000 7f27c45ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27bc0cff90 con 0x7f27bc15c610 2023-07-06T03:52:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.530+0000 7f27c45ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27bc0d0780 con 0x7f27bc0d2210 2023-07-06T03:52:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.530+0000 7f27c45ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc15c9e0 con 0x7f27bc06b360 2023-07-06T03:52:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.531+0000 7f27c1b9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc06b360 0x7f27bc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.531+0000 7f27c2b9c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27bc0d2210 0x7f27bc0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.531+0000 7f27c1b9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc06b360 0x7f27bc0b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:38678/0 (socket says 172.21.15.97:38678) 2023-07-06T03:52:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.531+0000 7f27c1b9a700 1 -- 172.21.15.97:0/3687748885 learned_addr learned my addr 172.21.15.97:0/3687748885 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:10.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.531+0000 7f27c2b9c700 1 -- 172.21.15.97:0/3687748885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc06b360 msgr2=0x7f27bc0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:10.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.531+0000 7f27c2b9c700 1 --2- 172.21.15.97:0/3687748885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc06b360 0x7f27bc0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.531+0000 7f27c2b9c700 1 -- 172.21.15.97:0/3687748885 >> v1:172.21.15.146:6789/0 conn(0x7f27bc15c610 legacy=0x7f27bc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.531+0000 7f27c2b9c700 1 -- 172.21.15.97:0/3687748885 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc0b5650 con 0x7f27bc0d2210 2023-07-06T03:52:10.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.531+0000 7f27c1399700 1 -- 172.21.15.97:0/3687748885 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306516376 0 0) 0x7f27bc0cff90 con 0x7f27bc15c610 2023-07-06T03:52:10.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.531+0000 7f27c1b9a700 1 --2- 172.21.15.97:0/3687748885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc06b360 0x7f27bc0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:10.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.532+0000 7f27c2b9c700 1 --2- 172.21.15.97:0/3687748885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc0d2210 0x7f27bc0d1310 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009dc0 tx=0x7f27b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61b1c684b5fadc41 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:10.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.532+0000 7f27c1399700 1 -- 172.21.15.97:0/3687748885 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b400e070 con 0x7f27bc0d2210 2023-07-06T03:52:10.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.532+0000 7f27c1399700 1 -- 172.21.15.97:0/3687748885 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b400bae0 con 0x7f27bc0d2210 2023-07-06T03:52:10.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.532+0000 7f27c1399700 1 -- 172.21.15.97:0/3687748885 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b400be90 con 0x7f27bc0d2210 2023-07-06T03:52:10.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.532+0000 7f27c45ff700 1 -- 172.21.15.97:0/3687748885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc0d2210 msgr2=0x7f27bc0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.532+0000 7f27c45ff700 1 --2- 172.21.15.97:0/3687748885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc0d2210 0x7f27bc0d1310 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009dc0 tx=0x7f27b400b330 comp rx=0 tx=0).stop 2023-07-06T03:52:10.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.532+0000 7f27c45ff700 1 -- 172.21.15.97:0/3687748885 shutdown_connections 2023-07-06T03:52:10.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.532+0000 7f27c45ff700 1 --2- 172.21.15.97:0/3687748885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc0d2210 0x7f27bc0d1310 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.532+0000 7f27c45ff700 1 --2- 172.21.15.97:0/3687748885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc06b360 0x7f27bc0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.532+0000 7f27c45ff700 1 -- 172.21.15.97:0/3687748885 >> 172.21.15.97:0/3687748885 conn(0x7f27bc06a6c0 msgr2=0x7f27bc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:10.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.533+0000 7f27c45ff700 1 -- 172.21.15.97:0/3687748885 shutdown_connections 2023-07-06T03:52:10.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.533+0000 7f27c45ff700 1 -- 172.21.15.97:0/3687748885 wait complete. 2023-07-06T03:52:10.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.534+0000 7f27c45ff700 1 Processor -- start 2023-07-06T03:52:10.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.534+0000 7f27c45ff700 1 -- start start 2023-07-06T03:52:10.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.535+0000 7f27c45ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc06b360 0x7f27bc158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.535+0000 7f27c45ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc0d2210 0x7f27bc0b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.535+0000 7f27c45ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc15c610 0x7f27bc0b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.535+0000 7f27c45ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc15e2f0 con 0x7f27bc06b360 2023-07-06T03:52:10.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.535+0000 7f27c45ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc1588d0 con 0x7f27bc15c610 2023-07-06T03:52:10.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.535+0000 7f27c45ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc0c7c80 con 0x7f27bc0d2210 2023-07-06T03:52:10.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.536+0000 7f27c239b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc06b360 0x7f27bc158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.536+0000 7f27c239b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc06b360 0x7f27bc158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60656/0 (socket says 172.21.15.97:60656) 2023-07-06T03:52:10.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.536+0000 7f27c239b700 1 -- 172.21.15.97:0/3011584368 learned_addr learned my addr 172.21.15.97:0/3011584368 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:10.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.536+0000 7f27c1b9a700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc0d2210 0x7f27bc0b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.536+0000 7f27c2b9c700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc15c610 0x7f27bc0b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.536+0000 7f27c239b700 1 -- 172.21.15.97:0/3011584368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc0d2210 msgr2=0x7f27bc0b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.536+0000 7f27c239b700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc0d2210 0x7f27bc0b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.536+0000 7f27c239b700 1 -- 172.21.15.97:0/3011584368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc15c610 msgr2=0x7f27bc0b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.536+0000 7f27c239b700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc15c610 0x7f27bc0b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.536+0000 7f27c239b700 1 -- 172.21.15.97:0/3011584368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac009550 con 0x7f27bc06b360 2023-07-06T03:52:10.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.536+0000 7f27c239b700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc06b360 0x7f27bc158390 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f27b80011e0 tx=0x7f27b800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:10.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.537+0000 7f27b2ffd700 1 -- 172.21.15.97:0/3011584368 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b80054e0 con 0x7f27bc06b360 2023-07-06T03:52:10.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.537+0000 7f27b2ffd700 1 -- 172.21.15.97:0/3011584368 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b8003680 con 0x7f27bc06b360 2023-07-06T03:52:10.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.537+0000 7f27c45ff700 1 -- 172.21.15.97:0/3011584368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b4009a70 con 0x7f27bc06b360 2023-07-06T03:52:10.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.537+0000 7f27b2ffd700 1 -- 172.21.15.97:0/3011584368 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8003a30 con 0x7f27bc06b360 2023-07-06T03:52:10.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.537+0000 7f27c45ff700 1 -- 172.21.15.97:0/3011584368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27bc15dd30 con 0x7f27bc06b360 2023-07-06T03:52:10.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.538+0000 7f27c45ff700 1 -- 172.21.15.97:0/3011584368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27bc04e340 con 0x7f27bc06b360 2023-07-06T03:52:10.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.539+0000 7f27b2ffd700 1 -- 172.21.15.97:0/3011584368 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f27b8003b90 con 0x7f27bc06b360 2023-07-06T03:52:10.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.540+0000 7f27b2ffd700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 0x7f27a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.540+0000 7f27b2ffd700 1 -- 172.21.15.97:0/3011584368 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f27b8088930 con 0x7f27bc06b360 2023-07-06T03:52:10.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.540+0000 7f27c1b9a700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 0x7f27a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.541+0000 7f27c1b9a700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 0x7f27a006d190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f27bc15fb00 tx=0x7f27ac0092c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:10.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.545+0000 7f27b2ffd700 1 -- 172.21.15.97:0/3011584368 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f27b8053940 con 0x7f27bc06b360 2023-07-06T03:52:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.739+0000 7f27c45ff700 1 -- 172.21.15.97:0/3011584368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27bc0b6530 con 0x7f27bc06b360 2023-07-06T03:52:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.739+0000 7f27b2ffd700 1 -- 172.21.15.97:0/3011584368 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27b80170a0 con 0x7f27bc06b360 2023-07-06T03:52:10.742 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:52:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.741+0000 7f27c45ff700 1 -- 172.21.15.97:0/3011584368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 msgr2=0x7f27a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.741+0000 7f27c45ff700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 0x7f27a006d190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f27bc15fb00 tx=0x7f27ac0092c0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.741+0000 7f27c45ff700 1 -- 172.21.15.97:0/3011584368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc06b360 msgr2=0x7f27bc158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.741+0000 7f27c45ff700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc06b360 0x7f27bc158390 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f27b80011e0 tx=0x7f27b800ca30 comp rx=0 tx=0).stop 2023-07-06T03:52:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.741+0000 7f27c45ff700 1 -- 172.21.15.97:0/3011584368 shutdown_connections 2023-07-06T03:52:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.741+0000 7f27c45ff700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 0x7f27a006d190 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.741+0000 7f27c45ff700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc15c610 0x7f27bc0b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.741+0000 7f27c45ff700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc06b360 0x7f27bc158390 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.741+0000 7f27c45ff700 1 --2- 172.21.15.97:0/3011584368 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc0d2210 0x7f27bc0b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.741+0000 7f27c45ff700 1 -- 172.21.15.97:0/3011584368 >> 172.21.15.97:0/3011584368 conn(0x7f27bc06a6c0 msgr2=0x7f27bc0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.742+0000 7f27c45ff700 1 -- 172.21.15.97:0/3011584368 shutdown_connections 2023-07-06T03:52:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.742+0000 7f27c45ff700 1 -- 172.21.15.97:0/3011584368 wait complete. 2023-07-06T03:52:10.761 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:52:10.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.957+0000 7f33ae5ff700 1 Processor -- start 2023-07-06T03:52:10.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.957+0000 7f33ae5ff700 1 -- start start 2023-07-06T03:52:10.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.958+0000 7f33ae5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a815c610 0x7f33a80b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.958+0000 7f33ae5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a80cff90 con 0x7f33a806b360 2023-07-06T03:52:10.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.958+0000 7f33ae5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a80b5650 con 0x7f33a80d2210 2023-07-06T03:52:10.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.958+0000 7f33ae5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a80d0780 con 0x7f33a815c610 2023-07-06T03:52:10.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.958+0000 7f33a77fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f33a806b360 0x7f33a80b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46540/0 (socket says 172.21.15.97:46540) 2023-07-06T03:52:10.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.958+0000 7f33a77fe700 1 -- 172.21.15.97:0/1312699585 learned_addr learned my addr 172.21.15.97:0/1312699585 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:10.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.959+0000 7f33a7fff700 1 --2- 172.21.15.97:0/1312699585 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a815c610 0x7f33a80b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.959+0000 7f33a6ffd700 1 -- 172.21.15.97:0/1312699585 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751745508 0 0) 0x7f33a80cff90 con 0x7f33a806b360 2023-07-06T03:52:10.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.959+0000 7f33a6ffd700 1 -- 172.21.15.97:0/1312699585 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33900035b0 con 0x7f33a806b360 2023-07-06T03:52:10.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.959+0000 7f33a6ffd700 1 -- 172.21.15.97:0/1312699585 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520035513 0 0) 0x7f33a80b5650 con 0x7f33a80d2210 2023-07-06T03:52:10.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.959+0000 7f33a6ffd700 1 -- 172.21.15.97:0/1312699585 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a80cff90 con 0x7f33a80d2210 2023-07-06T03:52:10.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.959+0000 7f33a7fff700 1 -- 172.21.15.97:0/1312699585 >> v1:172.21.15.146:6789/0 conn(0x7f33a80d2210 legacy=0x7f33a80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.959+0000 7f33a7fff700 1 -- 172.21.15.97:0/1312699585 >> v1:172.21.15.97:6789/0 conn(0x7f33a806b360 legacy=0x7f33a80b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.959+0000 7f33a6ffd700 1 -- 172.21.15.97:0/1312699585 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 30855143 0 0) 0x7f33900035b0 con 0x7f33a806b360 2023-07-06T03:52:10.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.959+0000 7f33a7fff700 1 -- 172.21.15.97:0/1312699585 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a80b5400 con 0x7f33a815c610 2023-07-06T03:52:10.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.960+0000 7f33a7fff700 1 --2- 172.21.15.97:0/1312699585 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a815c610 0x7f33a80b47b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3398009870 tx=0x7f3398009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb0da2bcac70795 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:10.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.960+0000 7f33a6ffd700 1 -- 172.21.15.97:0/1312699585 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c001670 con 0x7f33a815c610 2023-07-06T03:52:10.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.960+0000 7f33a6ffd700 1 -- 172.21.15.97:0/1312699585 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3398004030 con 0x7f33a815c610 2023-07-06T03:52:10.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.960+0000 7f33a6ffd700 1 -- 172.21.15.97:0/1312699585 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980043c0 con 0x7f33a815c610 2023-07-06T03:52:10.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.960+0000 7f33ae5ff700 1 -- 172.21.15.97:0/1312699585 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a815c610 msgr2=0x7f33a80b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.960+0000 7f33ae5ff700 1 --2- 172.21.15.97:0/1312699585 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a815c610 0x7f33a80b47b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3398009870 tx=0x7f3398009b80 comp rx=0 tx=0).stop 2023-07-06T03:52:10.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.961+0000 7f33ae5ff700 1 -- 172.21.15.97:0/1312699585 shutdown_connections 2023-07-06T03:52:10.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.961+0000 7f33ae5ff700 1 --2- 172.21.15.97:0/1312699585 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a815c610 0x7f33a80b47b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.961+0000 7f33ae5ff700 1 -- 172.21.15.97:0/1312699585 >> 172.21.15.97:0/1312699585 conn(0x7f33a806a6c0 msgr2=0x7f33a806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:10.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.961+0000 7f33ae5ff700 1 -- 172.21.15.97:0/1312699585 shutdown_connections 2023-07-06T03:52:10.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.961+0000 7f33ae5ff700 1 -- 172.21.15.97:0/1312699585 wait complete. 2023-07-06T03:52:10.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.962+0000 7f33ae5ff700 1 Processor -- start 2023-07-06T03:52:10.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.963+0000 7f33ae5ff700 1 -- start start 2023-07-06T03:52:10.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.963+0000 7f33ae5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a806b360 0x7f33a814b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.963+0000 7f33ae5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33a80d2210 0x7f33a8159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.963+0000 7f33ae5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a815c610 0x7f33a815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.963+0000 7f33ae5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a80c9ea0 con 0x7f33a806b360 2023-07-06T03:52:10.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.963+0000 7f33ae5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a80ca180 con 0x7f33a80d2210 2023-07-06T03:52:10.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.963+0000 7f33ae5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a80ca460 con 0x7f33a815c610 2023-07-06T03:52:10.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33a7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a806b360 0x7f33a814b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33a7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a806b360 0x7f33a814b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60664/0 (socket says 172.21.15.97:60664) 2023-07-06T03:52:10.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33a7fff700 1 -- 172.21.15.97:0/99519164 learned_addr learned my addr 172.21.15.97:0/99519164 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:10.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33a77fe700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33a80d2210 0x7f33a8159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33acb9c700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a815c610 0x7f33a815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33a7fff700 1 -- 172.21.15.97:0/99519164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a815c610 msgr2=0x7f33a815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33a7fff700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a815c610 0x7f33a815ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33a7fff700 1 -- 172.21.15.97:0/99519164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33a80d2210 msgr2=0x7f33a8159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:10.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33a7fff700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33a80d2210 0x7f33a8159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:10.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33a7fff700 1 -- 172.21.15.97:0/99519164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33980094a0 con 0x7f33a806b360 2023-07-06T03:52:10.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33a77fe700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33a80d2210 0x7f33a8159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:10.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.964+0000 7f33a7fff700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a806b360 0x7f33a814b020 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3398000c00 tx=0x7f3398004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:10.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.965+0000 7f33a4ff9700 1 -- 172.21.15.97:0/99519164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398003e30 con 0x7f33a806b360 2023-07-06T03:52:10.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.965+0000 7f33a4ff9700 1 -- 172.21.15.97:0/99519164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339801f070 con 0x7f33a806b360 2023-07-06T03:52:10.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.965+0000 7f33ae5ff700 1 -- 172.21.15.97:0/99519164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a80ca740 con 0x7f33a806b360 2023-07-06T03:52:10.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.965+0000 7f33a4ff9700 1 -- 172.21.15.97:0/99519164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398011870 con 0x7f33a806b360 2023-07-06T03:52:10.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.965+0000 7f33ae5ff700 1 -- 172.21.15.97:0/99519164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a814b560 con 0x7f33a806b360 2023-07-06T03:52:10.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.966+0000 7f33ae5ff700 1 -- 172.21.15.97:0/99519164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a80cac60 con 0x7f33a806b360 2023-07-06T03:52:10.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.968+0000 7f33a4ff9700 1 -- 172.21.15.97:0/99519164 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3398021030 con 0x7f33a806b360 2023-07-06T03:52:10.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.969+0000 7f33a4ff9700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f339406ace0 0x7f339406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:10.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.969+0000 7f33a4ff9700 1 -- 172.21.15.97:0/99519164 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f339808a8b0 con 0x7f33a806b360 2023-07-06T03:52:10.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.969+0000 7f33a77fe700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f339406ace0 0x7f339406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:10.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.970+0000 7f33a77fe700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f339406ace0 0x7f339406d190 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f339c001860 tx=0x7f339c001690 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:10.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:10.973+0000 7f33a4ff9700 1 -- 172.21.15.97:0/99519164 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3398055950 con 0x7f33a806b360 2023-07-06T03:52:11.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.181+0000 7f33ae5ff700 1 -- 172.21.15.97:0/99519164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f33a804e340 con 0x7f33a806b360 2023-07-06T03:52:11.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.182+0000 7f33a4ff9700 1 -- 172.21.15.97:0/99519164 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3398058f70 con 0x7f33a806b360 2023-07-06T03:52:11.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 -- 172.21.15.97:0/99519164 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f339406ace0 msgr2=0x7f339406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f339406ace0 0x7f339406d190 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f339c001860 tx=0x7f339c001690 comp rx=0 tx=0).stop 2023-07-06T03:52:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 -- 172.21.15.97:0/99519164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a806b360 msgr2=0x7f33a814b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a806b360 0x7f33a814b020 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3398000c00 tx=0x7f3398004750 comp rx=0 tx=0).stop 2023-07-06T03:52:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 -- 172.21.15.97:0/99519164 shutdown_connections 2023-07-06T03:52:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f339406ace0 0x7f339406d190 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:11.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33a815c610 0x7f33a815ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a806b360 0x7f33a814b020 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 --2- 172.21.15.97:0/99519164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33a80d2210 0x7f33a8159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 -- 172.21.15.97:0/99519164 >> 172.21.15.97:0/99519164 conn(0x7f33a806a6c0 msgr2=0x7f33a80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 -- 172.21.15.97:0/99519164 shutdown_connections 2023-07-06T03:52:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.184+0000 7f33ae5ff700 1 -- 172.21.15.97:0/99519164 wait complete. 2023-07-06T03:52:11.189 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:52:11.205 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:52:11.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:11.206 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:11.272 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[143292]: fuse finished with error 0 and tester_r 0 2023-07-06T03:52:11.276 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:52:11.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:11.277 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:52:11.470 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:52:11.471 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:52:11.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:52:11.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.713+0000 7fac4bbff700 1 Processor -- start 2023-07-06T03:52:11.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.713+0000 7fac4bbff700 1 -- start start 2023-07-06T03:52:11.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.715+0000 7fac4bbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fac4415c520 0x7fac440cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.715+0000 7fac4bbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac440cdae0 con 0x7fac44160180 2023-07-06T03:52:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.715+0000 7fac4bbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac440b6640 con 0x7fac44160d00 2023-07-06T03:52:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.715+0000 7fac4bbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac440cd8d0 con 0x7fac4415c520 2023-07-06T03:52:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.715+0000 7fac4919a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fac44160180 0x7fac441605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46550/0 (socket says 172.21.15.97:46550) 2023-07-06T03:52:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.715+0000 7fac4919a700 1 -- 172.21.15.97:0/2798679589 learned_addr learned my addr 172.21.15.97:0/2798679589 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.715+0000 7fac4999b700 1 --2- 172.21.15.97:0/2798679589 >> v2:172.21.15.97:3300/0 conn(0x7fac4415c520 0x7fac440cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:11.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.716+0000 7fac48999700 1 -- 172.21.15.97:0/2798679589 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908715245 0 0) 0x7fac440cdae0 con 0x7fac44160180 2023-07-06T03:52:11.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.716+0000 7fac48999700 1 -- 172.21.15.97:0/2798679589 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac300035b0 con 0x7fac44160180 2023-07-06T03:52:11.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.716+0000 7fac4999b700 1 -- 172.21.15.97:0/2798679589 >> v1:172.21.15.146:6789/0 conn(0x7fac44160d00 legacy=0x7fac440b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:11.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.716+0000 7fac4999b700 1 -- 172.21.15.97:0/2798679589 >> v1:172.21.15.97:6789/0 conn(0x7fac44160180 legacy=0x7fac441605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:11.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.716+0000 7fac4999b700 1 -- 172.21.15.97:0/2798679589 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac440b6920 con 0x7fac4415c520 2023-07-06T03:52:11.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.716+0000 7fac4999b700 1 --2- 172.21.15.97:0/2798679589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac4415c520 0x7fac440cd390 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fac40009840 tx=0x7fac40009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf8b9d410fff2ec server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:11.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.716+0000 7fac48999700 1 -- 172.21.15.97:0/2798679589 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40011830 con 0x7fac4415c520 2023-07-06T03:52:11.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.716+0000 7fac48999700 1 -- 172.21.15.97:0/2798679589 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac40011990 con 0x7fac4415c520 2023-07-06T03:52:11.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.716+0000 7fac48999700 1 -- 172.21.15.97:0/2798679589 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40011d20 con 0x7fac4415c520 2023-07-06T03:52:11.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.717+0000 7fac4bbff700 1 -- 172.21.15.97:0/2798679589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac4415c520 msgr2=0x7fac440cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:11.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.717+0000 7fac4bbff700 1 --2- 172.21.15.97:0/2798679589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac4415c520 0x7fac440cd390 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fac40009840 tx=0x7fac40009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:11.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.717+0000 7fac4bbff700 1 -- 172.21.15.97:0/2798679589 shutdown_connections 2023-07-06T03:52:11.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.717+0000 7fac4bbff700 1 --2- 172.21.15.97:0/2798679589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac4415c520 0x7fac440cd390 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:11.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.717+0000 7fac4bbff700 1 -- 172.21.15.97:0/2798679589 >> 172.21.15.97:0/2798679589 conn(0x7fac44071790 msgr2=0x7fac44071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:11.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.717+0000 7fac4bbff700 1 -- 172.21.15.97:0/2798679589 shutdown_connections 2023-07-06T03:52:11.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.717+0000 7fac4bbff700 1 -- 172.21.15.97:0/2798679589 wait complete. 2023-07-06T03:52:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.718+0000 7fac4bbff700 1 Processor -- start 2023-07-06T03:52:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.719+0000 7fac4bbff700 1 -- start start 2023-07-06T03:52:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.719+0000 7fac4bbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac4415c520 0x7fac4414dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4bbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac44160180 0x7fac4414e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:11.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4bbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac44160d00 0x7fac4415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4bbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac44150540 con 0x7fac44160d00 2023-07-06T03:52:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4bbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4415de20 con 0x7fac4415c520 2023-07-06T03:52:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4bbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac44150770 con 0x7fac44160180 2023-07-06T03:52:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4a19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac44160d00 0x7fac4415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4a19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac44160d00 0x7fac4415e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60674/0 (socket says 172.21.15.97:60674) 2023-07-06T03:52:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4a19c700 1 -- 172.21.15.97:0/23525578 learned_addr learned my addr 172.21.15.97:0/23525578 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4999b700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac4415c520 0x7fac4414dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4919a700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac44160180 0x7fac4414e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4a19c700 1 -- 172.21.15.97:0/23525578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac44160180 msgr2=0x7fac4414e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4a19c700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac44160180 0x7fac4414e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4a19c700 1 -- 172.21.15.97:0/23525578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac4415c520 msgr2=0x7fac4414dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4a19c700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac4415c520 0x7fac4414dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.720+0000 7fac4a19c700 1 -- 172.21.15.97:0/23525578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac400094a0 con 0x7fac44160d00 2023-07-06T03:52:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.721+0000 7fac4a19c700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac44160d00 0x7fac4415e8a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00cc50 tx=0x7fac3c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.721+0000 7fac3a7fc700 1 -- 172.21.15.97:0/23525578 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c003c00 con 0x7fac44160d00 2023-07-06T03:52:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.721+0000 7fac3a7fc700 1 -- 172.21.15.97:0/23525578 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c003d60 con 0x7fac44160d00 2023-07-06T03:52:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.721+0000 7fac4bbff700 1 -- 172.21.15.97:0/23525578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac44150a50 con 0x7fac44160d00 2023-07-06T03:52:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.721+0000 7fac3a7fc700 1 -- 172.21.15.97:0/23525578 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c00ab60 con 0x7fac44160d00 2023-07-06T03:52:11.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.721+0000 7fac4bbff700 1 -- 172.21.15.97:0/23525578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4415e380 con 0x7fac44160d00 2023-07-06T03:52:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.722+0000 7fac4bbff700 1 -- 172.21.15.97:0/23525578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac44150c40 con 0x7fac44160d00 2023-07-06T03:52:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.723+0000 7fac3a7fc700 1 -- 172.21.15.97:0/23525578 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac3c00acc0 con 0x7fac44160d00 2023-07-06T03:52:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.724+0000 7fac3a7fc700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac2806ace0 0x7fac2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.724+0000 7fac3a7fc700 1 -- 172.21.15.97:0/23525578 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fac3c0895e0 con 0x7fac44160d00 2023-07-06T03:52:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.724+0000 7fac4999b700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac2806ace0 0x7fac2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.725+0000 7fac4999b700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac2806ace0 0x7fac2806d190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fac400090b0 tx=0x7fac4001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:11.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.728+0000 7fac3a7fc700 1 -- 172.21.15.97:0/23525578 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fac3c0545f0 con 0x7fac44160d00 2023-07-06T03:52:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.928+0000 7fac4bbff700 1 -- 172.21.15.97:0/23525578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac4415e570 con 0x7fac44160d00 2023-07-06T03:52:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.929+0000 7fac3a7fc700 1 -- 172.21.15.97:0/23525578 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18320 (secure 0 0 0) 0x7fac3c057c10 con 0x7fac44160d00 2023-07-06T03:52:11.932 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:11.932 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":74,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:49:35.835648+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:49:32.371992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-06T03:49:33.281260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:52:11.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 -- 172.21.15.97:0/23525578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac2806ace0 msgr2=0x7fac2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:11.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac2806ace0 0x7fac2806d190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fac400090b0 tx=0x7fac4001c040 comp rx=0 tx=0).stop 2023-07-06T03:52:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 -- 172.21.15.97:0/23525578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac44160d00 msgr2=0x7fac4415e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac44160d00 0x7fac4415e8a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00cc50 tx=0x7fac3c00cf60 comp rx=0 tx=0).stop 2023-07-06T03:52:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 -- 172.21.15.97:0/23525578 shutdown_connections 2023-07-06T03:52:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fac2806ace0 0x7fac2806d190 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac44160d00 0x7fac4415e8a0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac4415c520 0x7fac4414dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 --2- 172.21.15.97:0/23525578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fac44160180 0x7fac4414e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 -- 172.21.15.97:0/23525578 >> 172.21.15.97:0/23525578 conn(0x7fac44071790 msgr2=0x7fac441605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 -- 172.21.15.97:0/23525578 shutdown_connections 2023-07-06T03:52:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:11.931+0000 7fac4bbff700 1 -- 172.21.15.97:0/23525578 wait complete. 2023-07-06T03:52:11.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:49:32.371992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:49:33.281260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:52:11.953 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:52:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.147+0000 7f62d79ff700 1 Processor -- start 2023-07-06T03:52:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.147+0000 7f62d79ff700 1 -- start start 2023-07-06T03:52:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.149+0000 7f62d79ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f62d015c610 0x7f62d00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.149+0000 7f62d79ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d006b360 0x7f62d00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.149+0000 7f62d79ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d00cff90 con 0x7f62d00d2210 2023-07-06T03:52:12.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.149+0000 7f62d79ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d00d0780 con 0x7f62d015c610 2023-07-06T03:52:12.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.149+0000 7f62d79ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d015c9e0 con 0x7f62d006b360 2023-07-06T03:52:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.149+0000 7f62d4f9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d006b360 0x7f62d00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.149+0000 7f62d579b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f62d015c610 0x7f62d00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.149+0000 7f62d5f9c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f62d00d2210 0x7f62d00d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:54822/0 (socket says 172.21.15.97:54822) 2023-07-06T03:52:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.149+0000 7f62d5f9c700 1 -- 172.21.15.97:0/4000739355 learned_addr learned my addr 172.21.15.97:0/4000739355 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.149+0000 7f62d579b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f62d015c610 0x7f62d00b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:55582/0 (socket says 172.21.15.97:55582) 2023-07-06T03:52:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.149+0000 7f62d4f9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d006b360 0x7f62d00b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:38716/0 (socket says 172.21.15.97:38716) 2023-07-06T03:52:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.150+0000 7f62c7fff700 1 -- 172.21.15.97:0/4000739355 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045332677 0 0) 0x7f62d00cff90 con 0x7f62d00d2210 2023-07-06T03:52:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.150+0000 7f62c7fff700 1 -- 172.21.15.97:0/4000739355 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62bc0035b0 con 0x7f62d00d2210 2023-07-06T03:52:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.150+0000 7f62d4f9a700 1 -- 172.21.15.97:0/4000739355 >> v2:172.21.15.146:3300/0 conn(0x7f62d015c610 msgr2=0x7f62d00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.150+0000 7f62d4f9a700 1 --2- 172.21.15.97:0/4000739355 >> v2:172.21.15.146:3300/0 conn(0x7f62d015c610 0x7f62d00b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.150+0000 7f62d4f9a700 1 -- 172.21.15.97:0/4000739355 >> v1:172.21.15.146:6789/0 conn(0x7f62d00d2210 legacy=0x7f62d00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.150+0000 7f62d4f9a700 1 -- 172.21.15.97:0/4000739355 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00b5650 con 0x7f62d006b360 2023-07-06T03:52:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.150+0000 7f62d579b700 1 --2- 172.21.15.97:0/4000739355 >> v2:172.21.15.146:3300/0 conn(0x7f62d015c610 0x7f62d00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.150+0000 7f62d4f9a700 1 --2- 172.21.15.97:0/4000739355 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d006b360 0x7f62d00b4cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009840 tx=0x7f62c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=295bc01c67d4cb76 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.151+0000 7f62c7fff700 1 -- 172.21.15.97:0/4000739355 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0011830 con 0x7f62d006b360 2023-07-06T03:52:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.151+0000 7f62c7fff700 1 -- 172.21.15.97:0/4000739355 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c0011990 con 0x7f62d006b360 2023-07-06T03:52:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.151+0000 7f62c7fff700 1 -- 172.21.15.97:0/4000739355 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0011d20 con 0x7f62d006b360 2023-07-06T03:52:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.151+0000 7f62d79ff700 1 -- 172.21.15.97:0/4000739355 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d006b360 msgr2=0x7f62d00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.151+0000 7f62d79ff700 1 --2- 172.21.15.97:0/4000739355 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d006b360 0x7f62d00b4cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009840 tx=0x7f62c0009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.151+0000 7f62d79ff700 1 -- 172.21.15.97:0/4000739355 shutdown_connections 2023-07-06T03:52:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.151+0000 7f62d79ff700 1 --2- 172.21.15.97:0/4000739355 >> v2:172.21.15.146:3300/0 conn(0x7f62d015c610 0x7f62d00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.151+0000 7f62d79ff700 1 --2- 172.21.15.97:0/4000739355 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d006b360 0x7f62d00b4cf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.151+0000 7f62d79ff700 1 -- 172.21.15.97:0/4000739355 >> 172.21.15.97:0/4000739355 conn(0x7f62d006a6c0 msgr2=0x7f62d006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.151+0000 7f62d79ff700 1 -- 172.21.15.97:0/4000739355 shutdown_connections 2023-07-06T03:52:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.151+0000 7f62d79ff700 1 -- 172.21.15.97:0/4000739355 wait complete. 2023-07-06T03:52:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.153+0000 7f62d79ff700 1 Processor -- start 2023-07-06T03:52:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.153+0000 7f62d79ff700 1 -- start start 2023-07-06T03:52:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.154+0000 7f62d79ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d006b360 0x7f62d0153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.154+0000 7f62d79ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d00d2210 0x7f62d00c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.154+0000 7f62d79ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62d015c610 0x7f62d00ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.154+0000 7f62d79ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00cb5f0 con 0x7f62d006b360 2023-07-06T03:52:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.154+0000 7f62d79ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0154750 con 0x7f62d015c610 2023-07-06T03:52:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.154+0000 7f62d79ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00ca970 con 0x7f62d00d2210 2023-07-06T03:52:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.154+0000 7f62d579b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d006b360 0x7f62d0153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.154+0000 7f62d579b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d006b360 0x7f62d0153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60680/0 (socket says 172.21.15.97:60680) 2023-07-06T03:52:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.154+0000 7f62d579b700 1 -- 172.21.15.97:0/2332829640 learned_addr learned my addr 172.21.15.97:0/2332829640 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d5f9c700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62d015c610 0x7f62d00ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d4f9a700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d00d2210 0x7f62d00c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d579b700 1 -- 172.21.15.97:0/2332829640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d00d2210 msgr2=0x7f62d00c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d579b700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d00d2210 0x7f62d00c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d579b700 1 -- 172.21.15.97:0/2332829640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62d015c610 msgr2=0x7f62d00ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d579b700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62d015c610 0x7f62d00ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d579b700 1 -- 172.21.15.97:0/2332829640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0094a0 con 0x7f62d006b360 2023-07-06T03:52:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d5f9c700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62d015c610 0x7f62d00ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d4f9a700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d00d2210 0x7f62d00c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d579b700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d006b360 0x7f62d0153c00 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00ebe0 tx=0x7f62cc009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62c5ffb700 1 -- 172.21.15.97:0/2332829640 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00c8b0 con 0x7f62d006b360 2023-07-06T03:52:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62c5ffb700 1 -- 172.21.15.97:0/2332829640 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc00ca10 con 0x7f62d006b360 2023-07-06T03:52:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62c5ffb700 1 -- 172.21.15.97:0/2332829640 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00cda0 con 0x7f62d006b360 2023-07-06T03:52:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d79ff700 1 -- 172.21.15.97:0/2332829640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c00094a0 con 0x7f62d006b360 2023-07-06T03:52:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.155+0000 7f62d79ff700 1 -- 172.21.15.97:0/2332829640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d00caf30 con 0x7f62d006b360 2023-07-06T03:52:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.156+0000 7f62d79ff700 1 -- 172.21.15.97:0/2332829640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d00cb860 con 0x7f62d006b360 2023-07-06T03:52:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.157+0000 7f62c5ffb700 1 -- 172.21.15.97:0/2332829640 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f62cc013470 con 0x7f62d006b360 2023-07-06T03:52:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.158+0000 7f62c5ffb700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62b406ace0 0x7f62b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.158+0000 7f62c5ffb700 1 -- 172.21.15.97:0/2332829640 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f62cc0895a0 con 0x7f62d006b360 2023-07-06T03:52:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.158+0000 7f62d4f9a700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62b406ace0 0x7f62b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.159+0000 7f62d4f9a700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62b406ace0 0x7f62b406d190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f62d00b1b40 tx=0x7f62c0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:12.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.162+0000 7f62c5ffb700 1 -- 172.21.15.97:0/2332829640 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f62cc054680 con 0x7f62d006b360 2023-07-06T03:52:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.354+0000 7f62d79ff700 1 -- 172.21.15.97:0/2332829640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62d004e340 con 0x7f62d006b360 2023-07-06T03:52:12.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.355+0000 7f62c5ffb700 1 -- 172.21.15.97:0/2332829640 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62cc057ca0 con 0x7f62d006b360 2023-07-06T03:52:12.358 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:52:12.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 -- 172.21.15.97:0/2332829640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62b406ace0 msgr2=0x7f62b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62b406ace0 0x7f62b406d190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f62d00b1b40 tx=0x7f62c0008030 comp rx=0 tx=0).stop 2023-07-06T03:52:12.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 -- 172.21.15.97:0/2332829640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d006b360 msgr2=0x7f62d0153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d006b360 0x7f62d0153c00 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00ebe0 tx=0x7f62cc009e20 comp rx=0 tx=0).stop 2023-07-06T03:52:12.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 -- 172.21.15.97:0/2332829640 shutdown_connections 2023-07-06T03:52:12.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f62b406ace0 0x7f62b406d190 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f62d015c610 0x7f62d00ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62d006b360 0x7f62d0153c00 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 --2- 172.21.15.97:0/2332829640 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f62d00d2210 0x7f62d00c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 -- 172.21.15.97:0/2332829640 >> 172.21.15.97:0/2332829640 conn(0x7f62d006a6c0 msgr2=0x7f62d015ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 -- 172.21.15.97:0/2332829640 shutdown_connections 2023-07-06T03:52:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.357+0000 7f62d79ff700 1 -- 172.21.15.97:0/2332829640 wait complete. 2023-07-06T03:52:12.378 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:52:12.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.572+0000 7fe094dff700 1 Processor -- start 2023-07-06T03:52:12.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.572+0000 7fe094dff700 1 -- start start 2023-07-06T03:52:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.574+0000 7fe094dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe09015c610 0x7fe0900b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.574+0000 7fe094dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe09006b360 0x7fe0900b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.574+0000 7fe094dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0900cff90 con 0x7fe0900d2210 2023-07-06T03:52:12.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.574+0000 7fe094dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0900d0780 con 0x7fe09006b360 2023-07-06T03:52:12.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.574+0000 7fe094dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09015c9e0 con 0x7fe09015c610 2023-07-06T03:52:12.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.574+0000 7fe08dd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe09006b360 0x7fe0900b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:12.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.574+0000 7fe08dd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe09006b360 0x7fe0900b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60696/0 (socket says 172.21.15.97:60696) 2023-07-06T03:52:12.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.574+0000 7fe08dd9b700 1 -- 172.21.15.97:0/2890065065 learned_addr learned my addr 172.21.15.97:0/2890065065 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:12.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.574+0000 7fe08e59c700 1 --2- 172.21.15.97:0/2890065065 >> v2:172.21.15.146:3300/0 conn(0x7fe09015c610 0x7fe0900b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.575+0000 7fe08dd9b700 1 -- 172.21.15.97:0/2890065065 >> v2:172.21.15.146:3300/0 conn(0x7fe09015c610 msgr2=0x7fe0900b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.575+0000 7fe08dd9b700 1 --2- 172.21.15.97:0/2890065065 >> v2:172.21.15.146:3300/0 conn(0x7fe09015c610 0x7fe0900b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.575+0000 7fe08dd9b700 1 -- 172.21.15.97:0/2890065065 >> v1:172.21.15.146:6789/0 conn(0x7fe0900d2210 legacy=0x7fe0900d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.575+0000 7fe08dd9b700 1 -- 172.21.15.97:0/2890065065 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0900b5650 con 0x7fe09006b360 2023-07-06T03:52:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.575+0000 7fe08dd9b700 1 --2- 172.21.15.97:0/2890065065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe09006b360 0x7fe0900b4cf0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe09008b000 tx=0x7fe084009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58a64880b7612d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:12.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.575+0000 7fe08d59a700 1 -- 172.21.15.97:0/2890065065 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08400b860 con 0x7fe09006b360 2023-07-06T03:52:12.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.575+0000 7fe08d59a700 1 -- 172.21.15.97:0/2890065065 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08400b9c0 con 0x7fe09006b360 2023-07-06T03:52:12.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.575+0000 7fe08d59a700 1 -- 172.21.15.97:0/2890065065 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08400bd50 con 0x7fe09006b360 2023-07-06T03:52:12.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.576+0000 7fe094dff700 1 -- 172.21.15.97:0/2890065065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe09006b360 msgr2=0x7fe0900b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.576+0000 7fe094dff700 1 --2- 172.21.15.97:0/2890065065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe09006b360 0x7fe0900b4cf0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe09008b000 tx=0x7fe084009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:12.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.576+0000 7fe094dff700 1 -- 172.21.15.97:0/2890065065 shutdown_connections 2023-07-06T03:52:12.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.576+0000 7fe094dff700 1 --2- 172.21.15.97:0/2890065065 >> v2:172.21.15.146:3300/0 conn(0x7fe09015c610 0x7fe0900b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.576+0000 7fe094dff700 1 --2- 172.21.15.97:0/2890065065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe09006b360 0x7fe0900b4cf0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.576+0000 7fe094dff700 1 -- 172.21.15.97:0/2890065065 >> 172.21.15.97:0/2890065065 conn(0x7fe09006a6c0 msgr2=0x7fe09006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:12.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.576+0000 7fe094dff700 1 -- 172.21.15.97:0/2890065065 shutdown_connections 2023-07-06T03:52:12.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.576+0000 7fe094dff700 1 -- 172.21.15.97:0/2890065065 wait complete. 2023-07-06T03:52:12.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.577+0000 7fe094dff700 1 Processor -- start 2023-07-06T03:52:12.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.578+0000 7fe094dff700 1 -- start start 2023-07-06T03:52:12.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.578+0000 7fe094dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe09006b360 0x7fe090157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe094dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0900d2210 0x7fe0900c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe094dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09015c610 0x7fe0900c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe094dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09015de60 con 0x7fe09006b360 2023-07-06T03:52:12.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe094dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900c98f0 con 0x7fe09015c610 2023-07-06T03:52:12.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe094dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901594e0 con 0x7fe0900d2210 2023-07-06T03:52:12.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe08e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe09006b360 0x7fe090157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:12.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe08e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe09006b360 0x7fe090157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60706/0 (socket says 172.21.15.97:60706) 2023-07-06T03:52:12.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe08e59c700 1 -- 172.21.15.97:0/2775141808 learned_addr learned my addr 172.21.15.97:0/2775141808 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:12.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe08e59c700 1 -- 172.21.15.97:0/2775141808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0900d2210 msgr2=0x7fe0900c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe08e59c700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0900d2210 0x7fe0900c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe08e59c700 1 -- 172.21.15.97:0/2775141808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09015c610 msgr2=0x7fe0900c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe08e59c700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09015c610 0x7fe0900c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe08e59c700 1 -- 172.21.15.97:0/2775141808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe084009550 con 0x7fe09006b360 2023-07-06T03:52:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.579+0000 7fe08e59c700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe09006b360 0x7fe090157f80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe08000d800 tx=0x7fe08000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.580+0000 7fe07effd700 1 -- 172.21.15.97:0/2775141808 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0800125f0 con 0x7fe09006b360 2023-07-06T03:52:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.580+0000 7fe07effd700 1 -- 172.21.15.97:0/2775141808 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe080012750 con 0x7fe09006b360 2023-07-06T03:52:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.580+0000 7fe07effd700 1 -- 172.21.15.97:0/2775141808 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe080012b00 con 0x7fe09006b360 2023-07-06T03:52:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.580+0000 7fe094dff700 1 -- 172.21.15.97:0/2775141808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0900c7380 con 0x7fe09006b360 2023-07-06T03:52:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.580+0000 7fe094dff700 1 -- 172.21.15.97:0/2775141808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09015da80 con 0x7fe09006b360 2023-07-06T03:52:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.581+0000 7fe094dff700 1 -- 172.21.15.97:0/2775141808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09015e0d0 con 0x7fe09006b360 2023-07-06T03:52:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.582+0000 7fe07effd700 1 -- 172.21.15.97:0/2775141808 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe080012c60 con 0x7fe09006b360 2023-07-06T03:52:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.582+0000 7fe07effd700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe07006ace0 0x7fe07006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.583+0000 7fe07effd700 1 -- 172.21.15.97:0/2775141808 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe080089870 con 0x7fe09006b360 2023-07-06T03:52:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.583+0000 7fe08dd9b700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe07006ace0 0x7fe07006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.583+0000 7fe08dd9b700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe07006ace0 0x7fe07006d190 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe09015d6d0 tx=0x7fe084010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.587+0000 7fe07effd700 1 -- 172.21.15.97:0/2775141808 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe080054910 con 0x7fe09006b360 2023-07-06T03:52:12.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.782+0000 7fe094dff700 1 -- 172.21.15.97:0/2775141808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe09004e340 con 0x7fe09006b360 2023-07-06T03:52:12.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.783+0000 7fe07effd700 1 -- 172.21.15.97:0/2775141808 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe080057f30 con 0x7fe09006b360 2023-07-06T03:52:12.786 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:52:12.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 -- 172.21.15.97:0/2775141808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe07006ace0 msgr2=0x7fe07006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe07006ace0 0x7fe07006d190 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe09015d6d0 tx=0x7fe084010000 comp rx=0 tx=0).stop 2023-07-06T03:52:12.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 -- 172.21.15.97:0/2775141808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe09006b360 msgr2=0x7fe090157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:12.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe09006b360 0x7fe090157f80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe08000d800 tx=0x7fe08000dbc0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 -- 172.21.15.97:0/2775141808 shutdown_connections 2023-07-06T03:52:12.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe07006ace0 0x7fe07006d190 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe09015c610 0x7fe0900c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe09006b360 0x7fe090157f80 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 --2- 172.21.15.97:0/2775141808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0900d2210 0x7fe0900c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:12.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 -- 172.21.15.97:0/2775141808 >> 172.21.15.97:0/2775141808 conn(0x7fe09006a6c0 msgr2=0x7fe0900d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:12.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 -- 172.21.15.97:0/2775141808 shutdown_connections 2023-07-06T03:52:12.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:12.785+0000 7fe094dff700 1 -- 172.21.15.97:0/2775141808 wait complete. 2023-07-06T03:52:12.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:52:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.003+0000 7f4cb57ff700 1 Processor -- start 2023-07-06T03:52:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.003+0000 7f4cb57ff700 1 -- start start 2023-07-06T03:52:13.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.004+0000 7f4cb57ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4cb015c610 0x7f4cb00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.004+0000 7f4cb57ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4cb006b360 0x7f4cb00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.004+0000 7f4cb57ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb00cff90 con 0x7f4cb00d2210 2023-07-06T03:52:13.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.004+0000 7f4cb57ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb00d0780 con 0x7f4cb006b360 2023-07-06T03:52:13.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.004+0000 7f4cb57ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb015c9e0 con 0x7f4cb015c610 2023-07-06T03:52:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4caf7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4cb00d2210 0x7f4cb00d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46552/0 (socket says 172.21.15.97:46552) 2023-07-06T03:52:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4cae7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4cb006b360 0x7f4cb00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4caf7fe700 1 -- 172.21.15.97:0/2864336894 learned_addr learned my addr 172.21.15.97:0/2864336894 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4caeffd700 1 --2- 172.21.15.97:0/2864336894 >> v2:172.21.15.146:3300/0 conn(0x7f4cb015c610 0x7f4cb00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4cae7fc700 1 -- 172.21.15.97:0/2864336894 >> v2:172.21.15.146:3300/0 conn(0x7f4cb015c610 msgr2=0x7f4cb00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4cae7fc700 1 --2- 172.21.15.97:0/2864336894 >> v2:172.21.15.146:3300/0 conn(0x7f4cb015c610 0x7f4cb00b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4cae7fc700 1 -- 172.21.15.97:0/2864336894 >> v1:172.21.15.97:6789/0 conn(0x7f4cb00d2210 legacy=0x7f4cb00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4cae7fc700 1 -- 172.21.15.97:0/2864336894 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb00b5650 con 0x7f4cb006b360 2023-07-06T03:52:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4caeffd700 1 --2- 172.21.15.97:0/2864336894 >> v2:172.21.15.146:3300/0 conn(0x7f4cb015c610 0x7f4cb00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4cadffb700 1 -- 172.21.15.97:0/2864336894 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436271444 0 0) 0x7f4cb00cff90 con 0x7f4cb00d2210 2023-07-06T03:52:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4cae7fc700 1 --2- 172.21.15.97:0/2864336894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb006b360 0x7f4cb00b4cf0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4cb008b000 tx=0x7f4ca4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea283c5846863bc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.005+0000 7f4cadffb700 1 -- 172.21.15.97:0/2864336894 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4011830 con 0x7f4cb006b360 2023-07-06T03:52:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.006+0000 7f4cadffb700 1 -- 172.21.15.97:0/2864336894 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca4011990 con 0x7f4cb006b360 2023-07-06T03:52:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.006+0000 7f4cadffb700 1 -- 172.21.15.97:0/2864336894 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4011d20 con 0x7f4cb006b360 2023-07-06T03:52:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.006+0000 7f4cb57ff700 1 -- 172.21.15.97:0/2864336894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb006b360 msgr2=0x7f4cb00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.006+0000 7f4cb57ff700 1 --2- 172.21.15.97:0/2864336894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb006b360 0x7f4cb00b4cf0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4cb008b000 tx=0x7f4ca4009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.006+0000 7f4cb57ff700 1 -- 172.21.15.97:0/2864336894 shutdown_connections 2023-07-06T03:52:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.006+0000 7f4cb57ff700 1 --2- 172.21.15.97:0/2864336894 >> v2:172.21.15.146:3300/0 conn(0x7f4cb015c610 0x7f4cb00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.006+0000 7f4cb57ff700 1 --2- 172.21.15.97:0/2864336894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb006b360 0x7f4cb00b4cf0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.006+0000 7f4cb57ff700 1 -- 172.21.15.97:0/2864336894 >> 172.21.15.97:0/2864336894 conn(0x7f4cb006a6c0 msgr2=0x7f4cb006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:13.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.006+0000 7f4cb57ff700 1 -- 172.21.15.97:0/2864336894 shutdown_connections 2023-07-06T03:52:13.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.006+0000 7f4cb57ff700 1 -- 172.21.15.97:0/2864336894 wait complete. 2023-07-06T03:52:13.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.008+0000 7f4cb57ff700 1 Processor -- start 2023-07-06T03:52:13.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.008+0000 7f4cb57ff700 1 -- start start 2023-07-06T03:52:13.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.009+0000 7f4cb57ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cb006b360 0x7f4cb0153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.009+0000 7f4cb57ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cb00d2210 0x7f4cb015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.009+0000 7f4cb57ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb015c610 0x7f4cb015d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.009+0000 7f4cb57ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb00ca810 con 0x7f4cb015c610 2023-07-06T03:52:13.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.009+0000 7f4cb57ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb00cb390 con 0x7f4cb00d2210 2023-07-06T03:52:13.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.009+0000 7f4cb57ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb00caa60 con 0x7f4cb006b360 2023-07-06T03:52:13.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.009+0000 7f4caf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb015c610 0x7f4cb015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.009+0000 7f4caf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb015c610 0x7f4cb015d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60722/0 (socket says 172.21.15.97:60722) 2023-07-06T03:52:13.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.009+0000 7f4caf7fe700 1 -- 172.21.15.97:0/187939673 learned_addr learned my addr 172.21.15.97:0/187939673 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:13.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4cae7fc700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cb00d2210 0x7f4cb015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4caeffd700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cb006b360 0x7f4cb0153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4caf7fe700 1 -- 172.21.15.97:0/187939673 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cb006b360 msgr2=0x7f4cb0153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4caf7fe700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cb006b360 0x7f4cb0153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4caf7fe700 1 -- 172.21.15.97:0/187939673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cb00d2210 msgr2=0x7f4cb015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4caf7fe700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cb00d2210 0x7f4cb015cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4caf7fe700 1 -- 172.21.15.97:0/187939673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca40094a0 con 0x7f4cb015c610 2023-07-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4caf7fe700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb015c610 0x7f4cb015d060 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800cc70 tx=0x7f4ca800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4c977fe700 1 -- 172.21.15.97:0/187939673 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca800ada0 con 0x7f4cb015c610 2023-07-06T03:52:13.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4c977fe700 1 -- 172.21.15.97:0/187939673 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca800ce70 con 0x7f4cb015c610 2023-07-06T03:52:13.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4cb57ff700 1 -- 172.21.15.97:0/187939673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb00cad40 con 0x7f4cb015c610 2023-07-06T03:52:13.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4c977fe700 1 -- 172.21.15.97:0/187939673 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca80072f0 con 0x7f4cb015c610 2023-07-06T03:52:13.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.010+0000 7f4cb57ff700 1 -- 172.21.15.97:0/187939673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb01544d0 con 0x7f4cb015c610 2023-07-06T03:52:13.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.012+0000 7f4cb57ff700 1 -- 172.21.15.97:0/187939673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb004e340 con 0x7f4cb015c610 2023-07-06T03:52:13.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.012+0000 7f4c977fe700 1 -- 172.21.15.97:0/187939673 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ca8007450 con 0x7f4cb015c610 2023-07-06T03:52:13.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.013+0000 7f4c977fe700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c9c06ace0 0x7f4c9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.013+0000 7f4c977fe700 1 -- 172.21.15.97:0/187939673 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ca8089020 con 0x7f4cb015c610 2023-07-06T03:52:13.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.014+0000 7f4caeffd700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c9c06ace0 0x7f4c9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.014+0000 7f4caeffd700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c9c06ace0 0x7f4c9c06d190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0006fd0 tx=0x7f4ca0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:13.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.018+0000 7f4c977fe700 1 -- 172.21.15.97:0/187939673 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4ca80540d0 con 0x7f4cb015c610 2023-07-06T03:52:13.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.213+0000 7f4cb57ff700 1 -- 172.21.15.97:0/187939673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4cb00cb4d0 con 0x7f4cb015c610 2023-07-06T03:52:13.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.214+0000 7f4c977fe700 1 -- 172.21.15.97:0/187939673 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ca80576f0 con 0x7f4cb015c610 2023-07-06T03:52:13.217 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:52:13.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 -- 172.21.15.97:0/187939673 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c9c06ace0 msgr2=0x7f4c9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c9c06ace0 0x7f4c9c06d190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0006fd0 tx=0x7f4ca0008040 comp rx=0 tx=0).stop 2023-07-06T03:52:13.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 -- 172.21.15.97:0/187939673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb015c610 msgr2=0x7f4cb015d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb015c610 0x7f4cb015d060 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800cc70 tx=0x7f4ca800a340 comp rx=0 tx=0).stop 2023-07-06T03:52:13.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 -- 172.21.15.97:0/187939673 shutdown_connections 2023-07-06T03:52:13.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c9c06ace0 0x7f4c9c06d190 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb015c610 0x7f4cb015d060 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cb006b360 0x7f4cb0153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 --2- 172.21.15.97:0/187939673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cb00d2210 0x7f4cb015cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 -- 172.21.15.97:0/187939673 >> 172.21.15.97:0/187939673 conn(0x7f4cb006a6c0 msgr2=0x7f4cb015a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:13.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 -- 172.21.15.97:0/187939673 shutdown_connections 2023-07-06T03:52:13.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.216+0000 7f4cb57ff700 1 -- 172.21.15.97:0/187939673 wait complete. 2023-07-06T03:52:13.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:52:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.432+0000 7f6d6b9ff700 1 Processor -- start 2023-07-06T03:52:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.432+0000 7f6d6b9ff700 1 -- start start 2023-07-06T03:52:13.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d6b9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6d6406b360 0x7f6d640b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d6b9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d640d2210 0x7f6d640d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d6b9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d640cff90 con 0x7f6d6415c610 2023-07-06T03:52:13.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d6b9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d640d0780 con 0x7f6d640d2210 2023-07-06T03:52:13.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d6b9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6415c9e0 con 0x7f6d6406b360 2023-07-06T03:52:13.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d69f9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d640d2210 0x7f6d640d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d6979b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6d6415c610 0x7f6d640b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46566/0 (socket says 172.21.15.97:46566) 2023-07-06T03:52:13.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d6979b700 1 -- 172.21.15.97:0/1790649704 learned_addr learned my addr 172.21.15.97:0/1790649704 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:13.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d68f9a700 1 --2- 172.21.15.97:0/1790649704 >> v2:172.21.15.146:3300/0 conn(0x7f6d6406b360 0x7f6d640b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d69f9c700 1 -- 172.21.15.97:0/1790649704 >> v2:172.21.15.146:3300/0 conn(0x7f6d6406b360 msgr2=0x7f6d640b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d69f9c700 1 --2- 172.21.15.97:0/1790649704 >> v2:172.21.15.146:3300/0 conn(0x7f6d6406b360 0x7f6d640b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d69f9c700 1 -- 172.21.15.97:0/1790649704 >> v1:172.21.15.97:6789/0 conn(0x7f6d6415c610 legacy=0x7f6d640b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d69f9c700 1 -- 172.21.15.97:0/1790649704 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640b5650 con 0x7f6d640d2210 2023-07-06T03:52:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.434+0000 7f6d5bfff700 1 -- 172.21.15.97:0/1790649704 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090136137 0 0) 0x7f6d640cff90 con 0x7f6d6415c610 2023-07-06T03:52:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.435+0000 7f6d69f9c700 1 --2- 172.21.15.97:0/1790649704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d640d2210 0x7f6d640d1310 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6d6406a590 tx=0x7f6d5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f256956a5b96058 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.435+0000 7f6d5bfff700 1 -- 172.21.15.97:0/1790649704 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00e070 con 0x7f6d640d2210 2023-07-06T03:52:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.435+0000 7f6d5bfff700 1 -- 172.21.15.97:0/1790649704 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c00bae0 con 0x7f6d640d2210 2023-07-06T03:52:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.435+0000 7f6d5bfff700 1 -- 172.21.15.97:0/1790649704 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00be90 con 0x7f6d640d2210 2023-07-06T03:52:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.435+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/1790649704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d640d2210 msgr2=0x7f6d640d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.435+0000 7f6d6b9ff700 1 --2- 172.21.15.97:0/1790649704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d640d2210 0x7f6d640d1310 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6d6406a590 tx=0x7f6d5c00b330 comp rx=0 tx=0).stop 2023-07-06T03:52:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.435+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/1790649704 shutdown_connections 2023-07-06T03:52:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.435+0000 7f6d6b9ff700 1 --2- 172.21.15.97:0/1790649704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d640d2210 0x7f6d640d1310 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.435+0000 7f6d6b9ff700 1 --2- 172.21.15.97:0/1790649704 >> v2:172.21.15.146:3300/0 conn(0x7f6d6406b360 0x7f6d640b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.435+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/1790649704 >> 172.21.15.97:0/1790649704 conn(0x7f6d6406a6c0 msgr2=0x7f6d6406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.436+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/1790649704 shutdown_connections 2023-07-06T03:52:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.436+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/1790649704 wait complete. 2023-07-06T03:52:13.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.437+0000 7f6d6b9ff700 1 Processor -- start 2023-07-06T03:52:13.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.437+0000 7f6d6b9ff700 1 -- start start 2023-07-06T03:52:13.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.438+0000 7f6d6b9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d6406b360 0x7f6d640c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.438+0000 7f6d6b9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d640d2210 0x7f6d6415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.438+0000 7f6d6b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d6415c610 0x7f6d641f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.438+0000 7f6d6b9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6415f270 con 0x7f6d640d2210 2023-07-06T03:52:13.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.438+0000 7f6d6b9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d64154470 con 0x7f6d6406b360 2023-07-06T03:52:13.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.438+0000 7f6d6b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640cb0a0 con 0x7f6d6415c610 2023-07-06T03:52:13.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.439+0000 7f6d68f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d640d2210 0x7f6d6415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.439+0000 7f6d68f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d640d2210 0x7f6d6415ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60730/0 (socket says 172.21.15.97:60730) 2023-07-06T03:52:13.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.439+0000 7f6d68f9a700 1 -- 172.21.15.97:0/2042012931 learned_addr learned my addr 172.21.15.97:0/2042012931 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:13.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.439+0000 7f6d6979b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d6406b360 0x7f6d640c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.439+0000 7f6d69f9c700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d6415c610 0x7f6d641f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.439+0000 7f6d68f9a700 1 -- 172.21.15.97:0/2042012931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d6415c610 msgr2=0x7f6d641f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.439+0000 7f6d68f9a700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d6415c610 0x7f6d641f17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.439+0000 7f6d68f9a700 1 -- 172.21.15.97:0/2042012931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d6406b360 msgr2=0x7f6d640c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.439+0000 7f6d68f9a700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d6406b360 0x7f6d640c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.439+0000 7f6d68f9a700 1 -- 172.21.15.97:0/2042012931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c009a70 con 0x7f6d640d2210 2023-07-06T03:52:13.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.439+0000 7f6d68f9a700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d640d2210 0x7f6d6415ed30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400e9b0 tx=0x7f6d5400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:13.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.440+0000 7f6d59ffb700 1 -- 172.21.15.97:0/2042012931 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54009710 con 0x7f6d640d2210 2023-07-06T03:52:13.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.440+0000 7f6d59ffb700 1 -- 172.21.15.97:0/2042012931 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5400ee50 con 0x7f6d640d2210 2023-07-06T03:52:13.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.440+0000 7f6d59ffb700 1 -- 172.21.15.97:0/2042012931 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400ca50 con 0x7f6d640d2210 2023-07-06T03:52:13.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.440+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/2042012931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d640ca490 con 0x7f6d640d2210 2023-07-06T03:52:13.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.440+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/2042012931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d640caa50 con 0x7f6d640d2210 2023-07-06T03:52:13.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.441+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/2042012931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6404e340 con 0x7f6d640d2210 2023-07-06T03:52:13.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.442+0000 7f6d59ffb700 1 -- 172.21.15.97:0/2042012931 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6d5400cbb0 con 0x7f6d640d2210 2023-07-06T03:52:13.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.443+0000 7f6d59ffb700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d4806ace0 0x7f6d4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.443+0000 7f6d59ffb700 1 -- 172.21.15.97:0/2042012931 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6d540894c0 con 0x7f6d640d2210 2023-07-06T03:52:13.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.443+0000 7f6d6979b700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d4806ace0 0x7f6d4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.443+0000 7f6d6979b700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d4806ace0 0x7f6d4806d190 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6d60000f60 tx=0x7f6d60007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.447+0000 7f6d59ffb700 1 -- 172.21.15.97:0/2042012931 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6d54057f60 con 0x7f6d640d2210 2023-07-06T03:52:13.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.641+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/2042012931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d640cd4f0 con 0x7f6d640d2210 2023-07-06T03:52:13.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.641+0000 7f6d59ffb700 1 -- 172.21.15.97:0/2042012931 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d5400ce60 con 0x7f6d640d2210 2023-07-06T03:52:13.645 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:52:13.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/2042012931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d4806ace0 msgr2=0x7f6d4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d4806ace0 0x7f6d4806d190 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6d60000f60 tx=0x7f6d60007e70 comp rx=0 tx=0).stop 2023-07-06T03:52:13.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/2042012931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d640d2210 msgr2=0x7f6d6415ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d640d2210 0x7f6d6415ed30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400e9b0 tx=0x7f6d5400ecc0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/2042012931 shutdown_connections 2023-07-06T03:52:13.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d4806ace0 0x7f6d4806d190 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d6415c610 0x7f6d641f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d6406b360 0x7f6d640c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 --2- 172.21.15.97:0/2042012931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d640d2210 0x7f6d6415ed30 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/2042012931 >> 172.21.15.97:0/2042012931 conn(0x7f6d6406a6c0 msgr2=0x7f6d6406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:13.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/2042012931 shutdown_connections 2023-07-06T03:52:13.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.644+0000 7f6d6b9ff700 1 -- 172.21.15.97:0/2042012931 wait complete. 2023-07-06T03:52:13.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:52:13.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.860+0000 7fd3119ff700 1 Processor -- start 2023-07-06T03:52:13.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.860+0000 7fd3119ff700 1 -- start start 2023-07-06T03:52:13.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.861+0000 7fd3119ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd30c15c610 0x7fd30c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.861+0000 7fd3119ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30c0cff90 con 0x7fd30c0d2210 2023-07-06T03:52:13.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.861+0000 7fd3119ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30c0b5650 con 0x7fd30c06b360 2023-07-06T03:52:13.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.861+0000 7fd3119ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30c0d0780 con 0x7fd30c15c610 2023-07-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.862+0000 7fd30affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd30c15c610 0x7fd30c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.862+0000 7fd30b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd30c0d2210 0x7fd30c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46580/0 (socket says 172.21.15.97:46580) 2023-07-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.862+0000 7fd30b7fe700 1 -- 172.21.15.97:0/2737427052 learned_addr learned my addr 172.21.15.97:0/2737427052 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:13.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.862+0000 7fd309ffb700 1 -- 172.21.15.97:0/2737427052 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634673828 0 0) 0x7fd30c0cff90 con 0x7fd30c0d2210 2023-07-06T03:52:13.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.862+0000 7fd309ffb700 1 -- 172.21.15.97:0/2737427052 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f40035b0 con 0x7fd30c0d2210 2023-07-06T03:52:13.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.862+0000 7fd30affd700 1 -- 172.21.15.97:0/2737427052 >> v1:172.21.15.146:6789/0 conn(0x7fd30c06b360 legacy=0x7fd30c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:13.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.862+0000 7fd30affd700 1 -- 172.21.15.97:0/2737427052 >> v1:172.21.15.97:6789/0 conn(0x7fd30c0d2210 legacy=0x7fd30c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.862+0000 7fd30affd700 1 -- 172.21.15.97:0/2737427052 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c0b5400 con 0x7fd30c15c610 2023-07-06T03:52:13.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.862+0000 7fd30affd700 1 --2- 172.21.15.97:0/2737427052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c15c610 0x7fd30c0b47b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009840 tx=0x7fd2fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569470f6fcb86bce server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:13.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.863+0000 7fd309ffb700 1 -- 172.21.15.97:0/2737427052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc004230 con 0x7fd30c15c610 2023-07-06T03:52:13.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.863+0000 7fd309ffb700 1 -- 172.21.15.97:0/2737427052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc004390 con 0x7fd30c15c610 2023-07-06T03:52:13.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.863+0000 7fd309ffb700 1 -- 172.21.15.97:0/2737427052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0036c0 con 0x7fd30c15c610 2023-07-06T03:52:13.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.863+0000 7fd3119ff700 1 -- 172.21.15.97:0/2737427052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c15c610 msgr2=0x7fd30c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.863+0000 7fd3119ff700 1 --2- 172.21.15.97:0/2737427052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c15c610 0x7fd30c0b47b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009840 tx=0x7fd2fc009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:13.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.863+0000 7fd3119ff700 1 -- 172.21.15.97:0/2737427052 shutdown_connections 2023-07-06T03:52:13.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.863+0000 7fd3119ff700 1 --2- 172.21.15.97:0/2737427052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c15c610 0x7fd30c0b47b0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.863+0000 7fd3119ff700 1 -- 172.21.15.97:0/2737427052 >> 172.21.15.97:0/2737427052 conn(0x7fd30c06a6c0 msgr2=0x7fd30c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:13.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.863+0000 7fd3119ff700 1 -- 172.21.15.97:0/2737427052 shutdown_connections 2023-07-06T03:52:13.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.863+0000 7fd3119ff700 1 -- 172.21.15.97:0/2737427052 wait complete. 2023-07-06T03:52:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.865+0000 7fd3119ff700 1 Processor -- start 2023-07-06T03:52:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.865+0000 7fd3119ff700 1 -- start start 2023-07-06T03:52:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.866+0000 7fd3119ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c06b360 0x7fd30c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.866+0000 7fd3119ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c0d2210 0x7fd30c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.866+0000 7fd3119ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c15c610 0x7fd30c1579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.866+0000 7fd3119ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0c9c40 con 0x7fd30c15c610 2023-07-06T03:52:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.866+0000 7fd3119ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0c90c0 con 0x7fd30c0d2210 2023-07-06T03:52:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.866+0000 7fd3119ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0c93a0 con 0x7fd30c06b360 2023-07-06T03:52:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.866+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c15c610 0x7fd30c1579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c15c610 0x7fd30c1579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60740/0 (socket says 172.21.15.97:60740) 2023-07-06T03:52:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd30b7fe700 1 -- 172.21.15.97:0/692827020 learned_addr learned my addr 172.21.15.97:0/692827020 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd30affd700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c06b360 0x7fd30c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd30a7fc700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c0d2210 0x7fd30c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd30b7fe700 1 -- 172.21.15.97:0/692827020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c06b360 msgr2=0x7fd30c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd30b7fe700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c06b360 0x7fd30c0cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd30b7fe700 1 -- 172.21.15.97:0/692827020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c0d2210 msgr2=0x7fd30c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd30b7fe700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c0d2210 0x7fd30c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd30b7fe700 1 -- 172.21.15.97:0/692827020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc0094a0 con 0x7fd30c15c610 2023-07-06T03:52:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd30a7fc700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c0d2210 0x7fd30c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd30b7fe700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c15c610 0x7fd30c1579d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd30400caa0 tx=0x7fd30400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd2f3fff700 1 -- 172.21.15.97:0/692827020 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400a9e0 con 0x7fd30c15c610 2023-07-06T03:52:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd2f3fff700 1 -- 172.21.15.97:0/692827020 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30400ab40 con 0x7fd30c15c610 2023-07-06T03:52:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd3119ff700 1 -- 172.21.15.97:0/692827020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c0c9680 con 0x7fd30c15c610 2023-07-06T03:52:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.867+0000 7fd2f3fff700 1 -- 172.21.15.97:0/692827020 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304012400 con 0x7fd30c15c610 2023-07-06T03:52:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.868+0000 7fd3119ff700 1 -- 172.21.15.97:0/692827020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c0ccb80 con 0x7fd30c15c610 2023-07-06T03:52:13.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.868+0000 7fd3119ff700 1 -- 172.21.15.97:0/692827020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c0cda20 con 0x7fd30c15c610 2023-07-06T03:52:13.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.869+0000 7fd2f3fff700 1 -- 172.21.15.97:0/692827020 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd30401b3b0 con 0x7fd30c15c610 2023-07-06T03:52:13.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.870+0000 7fd2f3fff700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2f806ace0 0x7fd2f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:13.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.870+0000 7fd2f3fff700 1 -- 172.21.15.97:0/692827020 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd304089300 con 0x7fd30c15c610 2023-07-06T03:52:13.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.871+0000 7fd30affd700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2f806ace0 0x7fd2f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.871+0000 7fd30affd700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2f806ace0 0x7fd2f806d190 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0090b0 tx=0x7fd2fc010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:13.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:13.875+0000 7fd2f3fff700 1 -- 172.21.15.97:0/692827020 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd304054310 con 0x7fd30c15c610 2023-07-06T03:52:14.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.088+0000 7fd3119ff700 1 -- 172.21.15.97:0/692827020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd30c04e340 con 0x7fd30c15c610 2023-07-06T03:52:14.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.088+0000 7fd2f3fff700 1 -- 172.21.15.97:0/692827020 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd304057930 con 0x7fd30c15c610 2023-07-06T03:52:14.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 -- 172.21.15.97:0/692827020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2f806ace0 msgr2=0x7fd2f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:14.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2f806ace0 0x7fd2f806d190 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0090b0 tx=0x7fd2fc010000 comp rx=0 tx=0).stop 2023-07-06T03:52:14.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 -- 172.21.15.97:0/692827020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c15c610 msgr2=0x7fd30c1579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:14.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c15c610 0x7fd30c1579d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd30400caa0 tx=0x7fd30400cdb0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 -- 172.21.15.97:0/692827020 shutdown_connections 2023-07-06T03:52:14.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2f806ace0 0x7fd2f806d190 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c15c610 0x7fd30c1579d0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c06b360 0x7fd30c0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 --2- 172.21.15.97:0/692827020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c0d2210 0x7fd30c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 -- 172.21.15.97:0/692827020 >> 172.21.15.97:0/692827020 conn(0x7fd30c06a6c0 msgr2=0x7fd30c15aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:14.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 -- 172.21.15.97:0/692827020 shutdown_connections 2023-07-06T03:52:14.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.091+0000 7fd3119ff700 1 -- 172.21.15.97:0/692827020 wait complete. 2023-07-06T03:52:14.095 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:52:14.112 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2023-07-06T03:52:14.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:14.112 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-06T03:52:14.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[123196]: fuse finished with error 0 and tester_r 0 2023-07-06T03:52:14.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:52:14.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:14.190 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T03:52:14.276 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:52:14.276 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:52:14.322 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:52:14.323 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:52:14.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:14.323 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T03:52:14.407 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T03:52:14.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:52:14.409 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:52:14.409 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:52:14.455 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:52:14.456 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:52:14.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:14.457 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T03:52:14.541 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T03:52:14.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:52:14.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:52:14.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.734+0000 7f80f8dff700 1 Processor -- start 2023-07-06T03:52:14.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.734+0000 7f80f8dff700 1 -- start start 2023-07-06T03:52:14.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.735+0000 7f80f8dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80f406d6f0 0x7f80f415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:14.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.735+0000 7f80f8dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80f40b05d0 0x7f80f40712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:14.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.735+0000 7f80f8dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f406ddb0 con 0x7f80f40b0a40 2023-07-06T03:52:14.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.735+0000 7f80f8dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f415d3d0 con 0x7f80f40b05d0 2023-07-06T03:52:14.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.735+0000 7f80f8dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f406bbf0 con 0x7f80f406d6f0 2023-07-06T03:52:14.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.736+0000 7f80f1d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80f40b05d0 0x7f80f40712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:14.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.736+0000 7f80f1d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80f40b05d0 0x7f80f40712f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60746/0 (socket says 172.21.15.97:60746) 2023-07-06T03:52:14.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.736+0000 7f80f1d9b700 1 -- 172.21.15.97:0/3316266851 learned_addr learned my addr 172.21.15.97:0/3316266851 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.736+0000 7f80f259c700 1 --2- 172.21.15.97:0/3316266851 >> v2:172.21.15.146:3300/0 conn(0x7f80f406d6f0 0x7f80f415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.736+0000 7f80f1d9b700 1 -- 172.21.15.97:0/3316266851 >> v2:172.21.15.146:3300/0 conn(0x7f80f406d6f0 msgr2=0x7f80f415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.736+0000 7f80f1d9b700 1 --2- 172.21.15.97:0/3316266851 >> v2:172.21.15.146:3300/0 conn(0x7f80f406d6f0 0x7f80f415ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.736+0000 7f80f1d9b700 1 -- 172.21.15.97:0/3316266851 >> v1:172.21.15.146:6789/0 conn(0x7f80f40b0a40 legacy=0x7f80f4158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.736+0000 7f80f1d9b700 1 -- 172.21.15.97:0/3316266851 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f406bed0 con 0x7f80f40b05d0 2023-07-06T03:52:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.736+0000 7f80f1d9b700 1 --2- 172.21.15.97:0/3316266851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f40b05d0 0x7f80f40712f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f80f406f660 tx=0x7f80e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105e45e3f9043259 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.737+0000 7f80f159a700 1 -- 172.21.15.97:0/3316266851 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e800b870 con 0x7f80f40b05d0 2023-07-06T03:52:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.737+0000 7f80f159a700 1 -- 172.21.15.97:0/3316266851 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e800b9d0 con 0x7f80f40b05d0 2023-07-06T03:52:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.737+0000 7f80f159a700 1 -- 172.21.15.97:0/3316266851 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e800bd60 con 0x7f80f40b05d0 2023-07-06T03:52:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.737+0000 7f80f8dff700 1 -- 172.21.15.97:0/3316266851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f40b05d0 msgr2=0x7f80f40712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.737+0000 7f80f8dff700 1 --2- 172.21.15.97:0/3316266851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f40b05d0 0x7f80f40712f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f80f406f660 tx=0x7f80e8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.737+0000 7f80f8dff700 1 -- 172.21.15.97:0/3316266851 shutdown_connections 2023-07-06T03:52:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.737+0000 7f80f8dff700 1 --2- 172.21.15.97:0/3316266851 >> v2:172.21.15.146:3300/0 conn(0x7f80f406d6f0 0x7f80f415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.737+0000 7f80f8dff700 1 --2- 172.21.15.97:0/3316266851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f40b05d0 0x7f80f40712f0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.737+0000 7f80f8dff700 1 -- 172.21.15.97:0/3316266851 >> 172.21.15.97:0/3316266851 conn(0x7f80f40ce8c0 msgr2=0x7f80f40cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.737+0000 7f80f8dff700 1 -- 172.21.15.97:0/3316266851 shutdown_connections 2023-07-06T03:52:14.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.737+0000 7f80f8dff700 1 -- 172.21.15.97:0/3316266851 wait complete. 2023-07-06T03:52:14.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.739+0000 7f80f8dff700 1 Processor -- start 2023-07-06T03:52:14.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.739+0000 7f80f8dff700 1 -- start start 2023-07-06T03:52:14.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.740+0000 7f80f8dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80f406d6f0 0x7f80f4156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.740+0000 7f80f8dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80f40b05d0 0x7f80f40ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.740+0000 7f80f8dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f40b0a40 0x7f80f40cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.740+0000 7f80f8dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4156f40 con 0x7f80f40b0a40 2023-07-06T03:52:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.740+0000 7f80f8dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40ade90 con 0x7f80f40b05d0 2023-07-06T03:52:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.740+0000 7f80f8dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41f5720 con 0x7f80f406d6f0 2023-07-06T03:52:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.740+0000 7f80f2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f40b0a40 0x7f80f40cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.740+0000 7f80f1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80f40b05d0 0x7f80f40ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.740+0000 7f80f2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f40b0a40 0x7f80f40cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60762/0 (socket says 172.21.15.97:60762) 2023-07-06T03:52:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.740+0000 7f80f2d9d700 1 -- 172.21.15.97:0/215786493 learned_addr learned my addr 172.21.15.97:0/215786493 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.740+0000 7f80f259c700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80f406d6f0 0x7f80f4156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.741+0000 7f80f2d9d700 1 -- 172.21.15.97:0/215786493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80f406d6f0 msgr2=0x7f80f4156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.741+0000 7f80f2d9d700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80f406d6f0 0x7f80f4156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.741+0000 7f80f2d9d700 1 -- 172.21.15.97:0/215786493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80f40b05d0 msgr2=0x7f80f40ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.741+0000 7f80f2d9d700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80f40b05d0 0x7f80f40ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.741+0000 7f80f2d9d700 1 -- 172.21.15.97:0/215786493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e8009550 con 0x7f80f40b0a40 2023-07-06T03:52:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.741+0000 7f80f2d9d700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f40b0a40 0x7f80f40cc670 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00caf0 tx=0x7f80ec00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.741+0000 7f80e2ffd700 1 -- 172.21.15.97:0/215786493 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec003c00 con 0x7f80f40b0a40 2023-07-06T03:52:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.741+0000 7f80e2ffd700 1 -- 172.21.15.97:0/215786493 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ec003d60 con 0x7f80f40b0a40 2023-07-06T03:52:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.741+0000 7f80f8dff700 1 -- 172.21.15.97:0/215786493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f4157910 con 0x7f80f40b0a40 2023-07-06T03:52:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.741+0000 7f80e2ffd700 1 -- 172.21.15.97:0/215786493 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec00aa20 con 0x7f80f40b0a40 2023-07-06T03:52:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.741+0000 7f80f8dff700 1 -- 172.21.15.97:0/215786493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f4157ed0 con 0x7f80f40b0a40 2023-07-06T03:52:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.743+0000 7f80f8dff700 1 -- 172.21.15.97:0/215786493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f41f5920 con 0x7f80f40b0a40 2023-07-06T03:52:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.743+0000 7f80e2ffd700 1 -- 172.21.15.97:0/215786493 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f80ec00ab80 con 0x7f80f40b0a40 2023-07-06T03:52:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.744+0000 7f80e2ffd700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f80d406ace0 0x7f80d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.744+0000 7f80e2ffd700 1 -- 172.21.15.97:0/215786493 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f80ec0893a0 con 0x7f80f40b0a40 2023-07-06T03:52:14.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.744+0000 7f80f259c700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f80d406ace0 0x7f80d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:14.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.745+0000 7f80f259c700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f80d406ace0 0x7f80d406d190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f80f40cd720 tx=0x7f80e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.748+0000 7f80e2ffd700 1 -- 172.21.15.97:0/215786493 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f80ec0543b0 con 0x7f80f40b0a40 2023-07-06T03:52:14.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.989+0000 7f80f8dff700 1 -- 172.21.15.97:0/215786493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80f4067830 con 0x7f80f40b0a40 2023-07-06T03:52:14.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.989+0000 7f80e2ffd700 1 -- 172.21.15.97:0/215786493 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4680 (secure 0 0 0) 0x7f80ec0579d0 con 0x7f80f40b0a40 2023-07-06T03:52:14.992 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:14.992 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2344840493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2344840493},{"type":"v1","addr":"172.21.15.97:6836","nonce":2344840493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1930131645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1930131645},{"type":"v1","addr":"172.21.15.146:6835","nonce":1930131645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3936349837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3936349837},{"type":"v1","addr":"172.21.15.146:6833","nonce":3936349837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:49:34.310076+0000","modified":"2023-07-06T03:49:55.680062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6837":{"gid":6837,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":10,"addr":"172.21.15.97:6837/1957695290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1957695290},{"type":"v1","addr":"172.21.15.97:6837","nonce":1957695290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-06T03:52:14.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 -- 172.21.15.97:0/215786493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f80d406ace0 msgr2=0x7f80d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:14.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f80d406ace0 0x7f80d406d190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f80f40cd720 tx=0x7f80e4008040 comp rx=0 tx=0).stop 2023-07-06T03:52:14.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 -- 172.21.15.97:0/215786493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f40b0a40 msgr2=0x7f80f40cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:14.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f40b0a40 0x7f80f40cc670 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00caf0 tx=0x7f80ec00ce00 comp rx=0 tx=0).stop 2023-07-06T03:52:14.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 -- 172.21.15.97:0/215786493 shutdown_connections 2023-07-06T03:52:14.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f80d406ace0 0x7f80d406d190 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80f40b0a40 0x7f80f40cc670 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80f406d6f0 0x7f80f4156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 --2- 172.21.15.97:0/215786493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80f40b05d0 0x7f80f40ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:14.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 -- 172.21.15.97:0/215786493 >> 172.21.15.97:0/215786493 conn(0x7f80f40ce8c0 msgr2=0x7f80f40ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:14.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 -- 172.21.15.97:0/215786493 shutdown_connections 2023-07-06T03:52:14.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:14.992+0000 7f80f8dff700 1 -- 172.21.15.97:0/215786493 wait complete. 2023-07-06T03:52:14.997 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 46 2023-07-06T03:52:15.013 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:52:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.205+0000 7fdcabdff700 1 Processor -- start 2023-07-06T03:52:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.205+0000 7fdcabdff700 1 -- start start 2023-07-06T03:52:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.205+0000 7fdcabdff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdca40cf3c0 0x7fdca40b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.205+0000 7fdcabdff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca40d1190 con 0x7fdca40b4ee0 2023-07-06T03:52:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.205+0000 7fdcabdff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca40cd680 con 0x7fdca415ca10 2023-07-06T03:52:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.205+0000 7fdcabdff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca40b5360 con 0x7fdca40cf3c0 2023-07-06T03:52:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.206+0000 7fdcaa39c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdca40b4ee0 0x7fdca40d1620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46594/0 (socket says 172.21.15.97:46594) 2023-07-06T03:52:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.206+0000 7fdcaa39c700 1 -- 172.21.15.97:0/3922734739 learned_addr learned my addr 172.21.15.97:0/3922734739 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.206+0000 7fdca939a700 1 --2- 172.21.15.97:0/3922734739 >> v2:172.21.15.146:3300/0 conn(0x7fdca40cf3c0 0x7fdca40b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.206+0000 7fdca8b99700 1 -- 172.21.15.97:0/3922734739 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961790362 0 0) 0x7fdca40d1190 con 0x7fdca40b4ee0 2023-07-06T03:52:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.206+0000 7fdca8b99700 1 -- 172.21.15.97:0/3922734739 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc900035b0 con 0x7fdca40b4ee0 2023-07-06T03:52:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.206+0000 7fdca8b99700 1 -- 172.21.15.97:0/3922734739 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843539786 0 0) 0x7fdca40cd680 con 0x7fdca415ca10 2023-07-06T03:52:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.206+0000 7fdca8b99700 1 -- 172.21.15.97:0/3922734739 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdca40d1190 con 0x7fdca415ca10 2023-07-06T03:52:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.206+0000 7fdca939a700 1 -- 172.21.15.97:0/3922734739 >> v1:172.21.15.146:6789/0 conn(0x7fdca415ca10 legacy=0x7fdca4160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.206+0000 7fdca939a700 1 -- 172.21.15.97:0/3922734739 >> v1:172.21.15.97:6789/0 conn(0x7fdca40b4ee0 legacy=0x7fdca40d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.206+0000 7fdca939a700 1 -- 172.21.15.97:0/3922734739 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca40cd960 con 0x7fdca40cf3c0 2023-07-06T03:52:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.206+0000 7fdca939a700 1 --2- 172.21.15.97:0/3922734739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdca40cf3c0 0x7fdca40b49a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009840 tx=0x7fdc94009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f9f31155a22f546 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.207+0000 7fdca8b99700 1 -- 172.21.15.97:0/3922734739 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94011830 con 0x7fdca40cf3c0 2023-07-06T03:52:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.207+0000 7fdca8b99700 1 -- 172.21.15.97:0/3922734739 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc94011990 con 0x7fdca40cf3c0 2023-07-06T03:52:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.207+0000 7fdca8b99700 1 -- 172.21.15.97:0/3922734739 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94011d20 con 0x7fdca40cf3c0 2023-07-06T03:52:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.207+0000 7fdcabdff700 1 -- 172.21.15.97:0/3922734739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdca40cf3c0 msgr2=0x7fdca40b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.207+0000 7fdcabdff700 1 --2- 172.21.15.97:0/3922734739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdca40cf3c0 0x7fdca40b49a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009840 tx=0x7fdc94009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.207+0000 7fdcabdff700 1 -- 172.21.15.97:0/3922734739 shutdown_connections 2023-07-06T03:52:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.207+0000 7fdcabdff700 1 --2- 172.21.15.97:0/3922734739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdca40cf3c0 0x7fdca40b49a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.207+0000 7fdcabdff700 1 -- 172.21.15.97:0/3922734739 >> 172.21.15.97:0/3922734739 conn(0x7fdca406a920 msgr2=0x7fdca406ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:15.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.207+0000 7fdcabdff700 1 -- 172.21.15.97:0/3922734739 shutdown_connections 2023-07-06T03:52:15.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.207+0000 7fdcabdff700 1 -- 172.21.15.97:0/3922734739 wait complete. 2023-07-06T03:52:15.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.209+0000 7fdcabdff700 1 Processor -- start 2023-07-06T03:52:15.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.209+0000 7fdcabdff700 1 -- start start 2023-07-06T03:52:15.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.210+0000 7fdcabdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdca40b4ee0 0x7fdca41f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:15.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.210+0000 7fdcabdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdca40cf3c0 0x7fdca40c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:15.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.210+0000 7fdcabdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdca415ca10 0x7fdca415b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:15.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.210+0000 7fdcabdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40c9590 con 0x7fdca415ca10 2023-07-06T03:52:15.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.210+0000 7fdcabdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40c7fb0 con 0x7fdca40cf3c0 2023-07-06T03:52:15.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.210+0000 7fdcabdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40c8290 con 0x7fdca40b4ee0 2023-07-06T03:52:15.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.210+0000 7fdcaa39c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdca415ca10 0x7fdca415b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:15.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.210+0000 7fdcaa39c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdca415ca10 0x7fdca415b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60774/0 (socket says 172.21.15.97:60774) 2023-07-06T03:52:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.210+0000 7fdcaa39c700 1 -- 172.21.15.97:0/2797960737 learned_addr learned my addr 172.21.15.97:0/2797960737 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdca939a700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdca40cf3c0 0x7fdca40c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdca9b9b700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdca40b4ee0 0x7fdca41f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdcaa39c700 1 -- 172.21.15.97:0/2797960737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdca40b4ee0 msgr2=0x7fdca41f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdcaa39c700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdca40b4ee0 0x7fdca41f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdcaa39c700 1 -- 172.21.15.97:0/2797960737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdca40cf3c0 msgr2=0x7fdca40c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdcaa39c700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdca40cf3c0 0x7fdca40c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdcaa39c700 1 -- 172.21.15.97:0/2797960737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc940094a0 con 0x7fdca415ca10 2023-07-06T03:52:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdca939a700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdca40cf3c0 0x7fdca40c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdcaa39c700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdca415ca10 0x7fdca415b8f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00cba0 tx=0x7fdc9c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdc9a7fc700 1 -- 172.21.15.97:0/2797960737 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00ab20 con 0x7fdca415ca10 2023-07-06T03:52:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdc9a7fc700 1 -- 172.21.15.97:0/2797960737 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c00ac80 con 0x7fdca415ca10 2023-07-06T03:52:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdc9a7fc700 1 -- 172.21.15.97:0/2797960737 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c012400 con 0x7fdca415ca10 2023-07-06T03:52:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdcabdff700 1 -- 172.21.15.97:0/2797960737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca40c8570 con 0x7fdca415ca10 2023-07-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.211+0000 7fdcabdff700 1 -- 172.21.15.97:0/2797960737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca40c8b30 con 0x7fdca415ca10 2023-07-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.212+0000 7fdcabdff700 1 -- 172.21.15.97:0/2797960737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca406a790 con 0x7fdca415ca10 2023-07-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.213+0000 7fdc9a7fc700 1 -- 172.21.15.97:0/2797960737 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc9c01b3b0 con 0x7fdca415ca10 2023-07-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.214+0000 7fdc9a7fc700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ad10 0x7fdc8806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.214+0000 7fdc9a7fc700 1 -- 172.21.15.97:0/2797960737 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc9c089320 con 0x7fdca415ca10 2023-07-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.215+0000 7fdca9b9b700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ad10 0x7fdc8806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.215+0000 7fdca9b9b700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ad10 0x7fdc8806d1c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdca0007fd0 tx=0x7fdca0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.218+0000 7fdc9a7fc700 1 -- 172.21.15.97:0/2797960737 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdc9c054330 con 0x7fdca415ca10 2023-07-06T03:52:15.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.415+0000 7fdcabdff700 1 -- 172.21.15.97:0/2797960737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdca404e340 con 0x7fdca415ca10 2023-07-06T03:52:15.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.417+0000 7fdc9a7fc700 1 -- 172.21.15.97:0/2797960737 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18320 (secure 0 0 0) 0x7fdc9c057950 con 0x7fdca415ca10 2023-07-06T03:52:15.420 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:15.420 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":74,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:49:35.835648+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:49:32.371992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-06T03:49:33.281260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:52:15.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.419+0000 7fdcabdff700 1 -- 172.21.15.97:0/2797960737 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ad10 msgr2=0x7fdc8806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.419+0000 7fdcabdff700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ad10 0x7fdc8806d1c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdca0007fd0 tx=0x7fdca0009040 comp rx=0 tx=0).stop 2023-07-06T03:52:15.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.419+0000 7fdcabdff700 1 -- 172.21.15.97:0/2797960737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdca415ca10 msgr2=0x7fdca415b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.419+0000 7fdcabdff700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdca415ca10 0x7fdca415b8f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00cba0 tx=0x7fdc9c00cf60 comp rx=0 tx=0).stop 2023-07-06T03:52:15.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.419+0000 7fdcabdff700 1 -- 172.21.15.97:0/2797960737 shutdown_connections 2023-07-06T03:52:15.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.419+0000 7fdcabdff700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc8806ad10 0x7fdc8806d1c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.419+0000 7fdcabdff700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdca415ca10 0x7fdca415b8f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.419+0000 7fdcabdff700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdca40b4ee0 0x7fdca41f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.419+0000 7fdcabdff700 1 --2- 172.21.15.97:0/2797960737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdca40cf3c0 0x7fdca40c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.419+0000 7fdcabdff700 1 -- 172.21.15.97:0/2797960737 >> 172.21.15.97:0/2797960737 conn(0x7fdca406a920 msgr2=0x7fdca40d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:15.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.419+0000 7fdcabdff700 1 -- 172.21.15.97:0/2797960737 shutdown_connections 2023-07-06T03:52:15.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.420+0000 7fdcabdff700 1 -- 172.21.15.97:0/2797960737 wait complete. 2023-07-06T03:52:15.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:49:32.371992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:49:33.281260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:52:15.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:52:15.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.634+0000 7f0e6cfff700 1 Processor -- start 2023-07-06T03:52:15.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.634+0000 7f0e6cfff700 1 -- start start 2023-07-06T03:52:15.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.635+0000 7f0e6cfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0e6815c610 0x7f0e680b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:15.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.635+0000 7f0e6cfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e680cff90 con 0x7f0e6806b360 2023-07-06T03:52:15.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.635+0000 7f0e6cfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e680b5650 con 0x7f0e680d2210 2023-07-06T03:52:15.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.635+0000 7f0e6cfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e680d0780 con 0x7f0e6815c610 2023-07-06T03:52:15.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.636+0000 7f0e5ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0e6806b360 0x7f0e680b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46610/0 (socket says 172.21.15.97:46610) 2023-07-06T03:52:15.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.636+0000 7f0e5ffff700 1 -- 172.21.15.97:0/3058923470 learned_addr learned my addr 172.21.15.97:0/3058923470 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:15.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.636+0000 7f0e6659c700 1 --2- 172.21.15.97:0/3058923470 >> v2:172.21.15.97:3300/0 conn(0x7f0e6815c610 0x7f0e680b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:15.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.636+0000 7f0e6659c700 1 -- 172.21.15.97:0/3058923470 >> v1:172.21.15.146:6789/0 conn(0x7f0e680d2210 legacy=0x7f0e680d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.636+0000 7f0e6659c700 1 -- 172.21.15.97:0/3058923470 >> v1:172.21.15.97:6789/0 conn(0x7f0e6806b360 legacy=0x7f0e680b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.636+0000 7f0e6659c700 1 -- 172.21.15.97:0/3058923470 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e680b5400 con 0x7f0e6815c610 2023-07-06T03:52:15.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.636+0000 7f0e6659c700 1 --2- 172.21.15.97:0/3058923470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e6815c610 0x7f0e680b47b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009840 tx=0x7f0e58009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8305c64e54278e7c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:15.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.637+0000 7f0e65d9b700 1 -- 172.21.15.97:0/3058923470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58011830 con 0x7f0e6815c610 2023-07-06T03:52:15.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.637+0000 7f0e65d9b700 1 -- 172.21.15.97:0/3058923470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e58011990 con 0x7f0e6815c610 2023-07-06T03:52:15.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.637+0000 7f0e65d9b700 1 -- 172.21.15.97:0/3058923470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58011d20 con 0x7f0e6815c610 2023-07-06T03:52:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.637+0000 7f0e6cfff700 1 -- 172.21.15.97:0/3058923470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e6815c610 msgr2=0x7f0e680b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.637+0000 7f0e6cfff700 1 --2- 172.21.15.97:0/3058923470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e6815c610 0x7f0e680b47b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009840 tx=0x7f0e58009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.637+0000 7f0e6cfff700 1 -- 172.21.15.97:0/3058923470 shutdown_connections 2023-07-06T03:52:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.637+0000 7f0e6cfff700 1 --2- 172.21.15.97:0/3058923470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e6815c610 0x7f0e680b47b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.637+0000 7f0e6cfff700 1 -- 172.21.15.97:0/3058923470 >> 172.21.15.97:0/3058923470 conn(0x7f0e6806a6c0 msgr2=0x7f0e6806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.637+0000 7f0e6cfff700 1 -- 172.21.15.97:0/3058923470 shutdown_connections 2023-07-06T03:52:15.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.637+0000 7f0e6cfff700 1 -- 172.21.15.97:0/3058923470 wait complete. 2023-07-06T03:52:15.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.640+0000 7f0e6cfff700 1 Processor -- start 2023-07-06T03:52:15.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.640+0000 7f0e6cfff700 1 -- start start 2023-07-06T03:52:15.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.641+0000 7f0e6cfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e6806b360 0x7f0e6815e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:15.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.641+0000 7f0e6cfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e680d2210 0x7f0e68155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:15.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.641+0000 7f0e6cfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e6815c610 0x7f0e680b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:15.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.641+0000 7f0e6cfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e681f5b90 con 0x7f0e6815c610 2023-07-06T03:52:15.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.641+0000 7f0e6cfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e681f5de0 con 0x7f0e680d2210 2023-07-06T03:52:15.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.641+0000 7f0e6cfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e681f60c0 con 0x7f0e6806b360 2023-07-06T03:52:15.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.641+0000 7f0e66d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e6815c610 0x7f0e680b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:15.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e66d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e6815c610 0x7f0e680b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60780/0 (socket says 172.21.15.97:60780) 2023-07-06T03:52:15.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e66d9d700 1 -- 172.21.15.97:0/440637893 learned_addr learned my addr 172.21.15.97:0/440637893 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:15.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e6659c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e6806b360 0x7f0e6815e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:15.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e5ffff700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e680d2210 0x7f0e68155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:15.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e66d9d700 1 -- 172.21.15.97:0/440637893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e6806b360 msgr2=0x7f0e6815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e66d9d700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e6806b360 0x7f0e6815e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e66d9d700 1 -- 172.21.15.97:0/440637893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e680d2210 msgr2=0x7f0e68155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e66d9d700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e680d2210 0x7f0e68155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e66d9d700 1 -- 172.21.15.97:0/440637893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580094a0 con 0x7f0e6815c610 2023-07-06T03:52:15.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e66d9d700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e6815c610 0x7f0e680b34c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000cc50 tx=0x7f0e6000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:15.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e5effd700 1 -- 172.21.15.97:0/440637893 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60003c00 con 0x7f0e6815c610 2023-07-06T03:52:15.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e5effd700 1 -- 172.21.15.97:0/440637893 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e60003d60 con 0x7f0e6815c610 2023-07-06T03:52:15.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e6cfff700 1 -- 172.21.15.97:0/440637893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e681f63a0 con 0x7f0e6815c610 2023-07-06T03:52:15.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.642+0000 7f0e5effd700 1 -- 172.21.15.97:0/440637893 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6000aab0 con 0x7f0e6815c610 2023-07-06T03:52:15.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.643+0000 7f0e6cfff700 1 -- 172.21.15.97:0/440637893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e681f6960 con 0x7f0e6815c610 2023-07-06T03:52:15.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.644+0000 7f0e6cfff700 1 -- 172.21.15.97:0/440637893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e680cdfe0 con 0x7f0e6815c610 2023-07-06T03:52:15.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.644+0000 7f0e5effd700 1 -- 172.21.15.97:0/440637893 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e6000ac10 con 0x7f0e6815c610 2023-07-06T03:52:15.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.645+0000 7f0e5effd700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0e4806ace0 0x7f0e4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:15.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.645+0000 7f0e5effd700 1 -- 172.21.15.97:0/440637893 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e600893d0 con 0x7f0e6815c610 2023-07-06T03:52:15.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.645+0000 7f0e6659c700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0e4806ace0 0x7f0e4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:15.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.646+0000 7f0e6659c700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0e4806ace0 0x7f0e4806d190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0e580090b0 tx=0x7f0e58008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:15.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.649+0000 7f0e5effd700 1 -- 172.21.15.97:0/440637893 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0e60054470 con 0x7f0e6815c610 2023-07-06T03:52:15.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.841+0000 7f0e6cfff700 1 -- 172.21.15.97:0/440637893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e680c9e00 con 0x7f0e6815c610 2023-07-06T03:52:15.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.842+0000 7f0e5effd700 1 -- 172.21.15.97:0/440637893 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e60057a90 con 0x7f0e6815c610 2023-07-06T03:52:15.844 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:52:15.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 -- 172.21.15.97:0/440637893 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0e4806ace0 msgr2=0x7f0e4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0e4806ace0 0x7f0e4806d190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0e580090b0 tx=0x7f0e58008030 comp rx=0 tx=0).stop 2023-07-06T03:52:15.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 -- 172.21.15.97:0/440637893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e6815c610 msgr2=0x7f0e680b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:15.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e6815c610 0x7f0e680b34c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000cc50 tx=0x7f0e6000cf60 comp rx=0 tx=0).stop 2023-07-06T03:52:15.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 -- 172.21.15.97:0/440637893 shutdown_connections 2023-07-06T03:52:15.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0e4806ace0 0x7f0e4806d190 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e6815c610 0x7f0e680b34c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0e6806b360 0x7f0e6815e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 --2- 172.21.15.97:0/440637893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e680d2210 0x7f0e68155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:15.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 -- 172.21.15.97:0/440637893 >> 172.21.15.97:0/440637893 conn(0x7f0e6806a6c0 msgr2=0x7f0e680b7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:15.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 -- 172.21.15.97:0/440637893 shutdown_connections 2023-07-06T03:52:15.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:15.844+0000 7f0e6cfff700 1 -- 172.21.15.97:0/440637893 wait complete. 2023-07-06T03:52:15.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:52:16.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.061+0000 7f7de9bff700 1 Processor -- start 2023-07-06T03:52:16.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.062+0000 7f7de9bff700 1 -- start start 2023-07-06T03:52:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de9bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7de415c610 0x7f7de40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de9bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7de406b360 0x7f7de40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de9bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d2210 0x7f7de40d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de9bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de40d0780 con 0x7f7de406b360 2023-07-06T03:52:16.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de9bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de415c9e0 con 0x7f7de415c610 2023-07-06T03:52:16.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de9bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de40b5650 con 0x7f7de40d2210 2023-07-06T03:52:16.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de2ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7de406b360 0x7f7de40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de2ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7de406b360 0x7f7de40b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60784/0 (socket says 172.21.15.97:60784) 2023-07-06T03:52:16.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de2ffd700 1 -- 172.21.15.97:0/2363781499 learned_addr learned my addr 172.21.15.97:0/2363781499 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:16.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de37fe700 1 --2- 172.21.15.97:0/2363781499 >> v2:172.21.15.146:3300/0 conn(0x7f7de415c610 0x7f7de40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de3fff700 1 --2- 172.21.15.97:0/2363781499 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d2210 0x7f7de40d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.063+0000 7f7de2ffd700 1 -- 172.21.15.97:0/2363781499 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d2210 msgr2=0x7f7de40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:16.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.064+0000 7f7de2ffd700 1 --2- 172.21.15.97:0/2363781499 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d2210 0x7f7de40d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.064+0000 7f7de2ffd700 1 -- 172.21.15.97:0/2363781499 >> v2:172.21.15.146:3300/0 conn(0x7f7de415c610 msgr2=0x7f7de40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:16.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.064+0000 7f7de2ffd700 1 --2- 172.21.15.97:0/2363781499 >> v2:172.21.15.146:3300/0 conn(0x7f7de415c610 0x7f7de40b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.064+0000 7f7de2ffd700 1 -- 172.21.15.97:0/2363781499 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de40d01c0 con 0x7f7de406b360 2023-07-06T03:52:16.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.064+0000 7f7de3fff700 1 --2- 172.21.15.97:0/2363781499 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d2210 0x7f7de40d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:16.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.064+0000 7f7de37fe700 1 --2- 172.21.15.97:0/2363781499 >> v2:172.21.15.146:3300/0 conn(0x7f7de415c610 0x7f7de40b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:16.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.064+0000 7f7de2ffd700 1 --2- 172.21.15.97:0/2363781499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de406b360 0x7f7de40b4cf0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7de408b000 tx=0x7f7dd8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e908ab74cc8bca server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:16.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.064+0000 7f7de27fc700 1 -- 172.21.15.97:0/2363781499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800b860 con 0x7f7de406b360 2023-07-06T03:52:16.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.064+0000 7f7de27fc700 1 -- 172.21.15.97:0/2363781499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd800b9c0 con 0x7f7de406b360 2023-07-06T03:52:16.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.064+0000 7f7de27fc700 1 -- 172.21.15.97:0/2363781499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800bd50 con 0x7f7de406b360 2023-07-06T03:52:16.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.065+0000 7f7de9bff700 1 -- 172.21.15.97:0/2363781499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de406b360 msgr2=0x7f7de40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.065+0000 7f7de9bff700 1 --2- 172.21.15.97:0/2363781499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de406b360 0x7f7de40b4cf0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7de408b000 tx=0x7f7dd8009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:16.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.065+0000 7f7de9bff700 1 -- 172.21.15.97:0/2363781499 shutdown_connections 2023-07-06T03:52:16.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.065+0000 7f7de9bff700 1 --2- 172.21.15.97:0/2363781499 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d2210 0x7f7de40d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.065+0000 7f7de9bff700 1 --2- 172.21.15.97:0/2363781499 >> v2:172.21.15.146:3300/0 conn(0x7f7de415c610 0x7f7de40b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.065+0000 7f7de9bff700 1 --2- 172.21.15.97:0/2363781499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de406b360 0x7f7de40b4cf0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.065+0000 7f7de9bff700 1 -- 172.21.15.97:0/2363781499 >> 172.21.15.97:0/2363781499 conn(0x7f7de406a6c0 msgr2=0x7f7de406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:16.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.065+0000 7f7de9bff700 1 -- 172.21.15.97:0/2363781499 shutdown_connections 2023-07-06T03:52:16.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.065+0000 7f7de9bff700 1 -- 172.21.15.97:0/2363781499 wait complete. 2023-07-06T03:52:16.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.066+0000 7f7de9bff700 1 Processor -- start 2023-07-06T03:52:16.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.067+0000 7f7de9bff700 1 -- start start 2023-07-06T03:52:16.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.067+0000 7f7de9bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de406b360 0x7f7de4154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.067+0000 7f7de9bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d2210 0x7f7de40b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.067+0000 7f7de9bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7de415c610 0x7f7de40cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.067+0000 7f7de9bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de415dd10 con 0x7f7de406b360 2023-07-06T03:52:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.067+0000 7f7de9bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de415d090 con 0x7f7de415c610 2023-07-06T03:52:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.067+0000 7f7de9bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de415d370 con 0x7f7de40d2210 2023-07-06T03:52:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.068+0000 7f7de37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de406b360 0x7f7de4154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.068+0000 7f7de37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de406b360 0x7f7de4154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60800/0 (socket says 172.21.15.97:60800) 2023-07-06T03:52:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.068+0000 7f7de37fe700 1 -- 172.21.15.97:0/3900847744 learned_addr learned my addr 172.21.15.97:0/3900847744 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:16.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.068+0000 7f7de2ffd700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d2210 0x7f7de40b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.068+0000 7f7de3fff700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7de415c610 0x7f7de40cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.068+0000 7f7de37fe700 1 -- 172.21.15.97:0/3900847744 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d2210 msgr2=0x7f7de40b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.068+0000 7f7de37fe700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d2210 0x7f7de40b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.068+0000 7f7de37fe700 1 -- 172.21.15.97:0/3900847744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7de415c610 msgr2=0x7f7de40cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.068+0000 7f7de37fe700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7de415c610 0x7f7de40cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.068+0000 7f7de37fe700 1 -- 172.21.15.97:0/3900847744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8009550 con 0x7f7de406b360 2023-07-06T03:52:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.068+0000 7f7de37fe700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de406b360 0x7f7de4154490 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400c7f0 tx=0x7f7dd400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:16.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.069+0000 7f7dcbfff700 1 -- 172.21.15.97:0/3900847744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4011800 con 0x7f7de406b360 2023-07-06T03:52:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.069+0000 7f7dcbfff700 1 -- 172.21.15.97:0/3900847744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd4011960 con 0x7f7de406b360 2023-07-06T03:52:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.069+0000 7f7de9bff700 1 -- 172.21.15.97:0/3900847744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de415d650 con 0x7f7de406b360 2023-07-06T03:52:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.069+0000 7f7dcbfff700 1 -- 172.21.15.97:0/3900847744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4011cf0 con 0x7f7de406b360 2023-07-06T03:52:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.069+0000 7f7de9bff700 1 -- 172.21.15.97:0/3900847744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de40b6dc0 con 0x7f7de406b360 2023-07-06T03:52:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.070+0000 7f7de9bff700 1 -- 172.21.15.97:0/3900847744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de404e340 con 0x7f7de406b360 2023-07-06T03:52:16.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.071+0000 7f7dcbfff700 1 -- 172.21.15.97:0/3900847744 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7dd4018360 con 0x7f7de406b360 2023-07-06T03:52:16.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.071+0000 7f7dcbfff700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7dd006ace0 0x7f7dd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.072+0000 7f7dcbfff700 1 -- 172.21.15.97:0/3900847744 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7dd4089410 con 0x7f7de406b360 2023-07-06T03:52:16.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.072+0000 7f7de2ffd700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7dd006ace0 0x7f7dd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.072+0000 7f7de2ffd700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7dd006ace0 0x7f7dd006d190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8000c00 tx=0x7f7dd8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:16.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.076+0000 7f7dcbfff700 1 -- 172.21.15.97:0/3900847744 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7dd4054420 con 0x7f7de406b360 2023-07-06T03:52:16.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.271+0000 7f7de9bff700 1 -- 172.21.15.97:0/3900847744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7de40b88c0 con 0x7f7de406b360 2023-07-06T03:52:16.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.271+0000 7f7dcbfff700 1 -- 172.21.15.97:0/3900847744 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7dd4057a40 con 0x7f7de406b360 2023-07-06T03:52:16.273 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:52:16.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 -- 172.21.15.97:0/3900847744 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7dd006ace0 msgr2=0x7f7dd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7dd006ace0 0x7f7dd006d190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8000c00 tx=0x7f7dd8008030 comp rx=0 tx=0).stop 2023-07-06T03:52:16.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 -- 172.21.15.97:0/3900847744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de406b360 msgr2=0x7f7de4154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de406b360 0x7f7de4154490 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400c7f0 tx=0x7f7dd400cb00 comp rx=0 tx=0).stop 2023-07-06T03:52:16.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 -- 172.21.15.97:0/3900847744 shutdown_connections 2023-07-06T03:52:16.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7dd006ace0 0x7f7dd006d190 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7de415c610 0x7f7de40cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de406b360 0x7f7de4154490 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 --2- 172.21.15.97:0/3900847744 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7de40d2210 0x7f7de40b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 -- 172.21.15.97:0/3900847744 >> 172.21.15.97:0/3900847744 conn(0x7f7de406a6c0 msgr2=0x7f7de40cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:16.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 -- 172.21.15.97:0/3900847744 shutdown_connections 2023-07-06T03:52:16.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.274+0000 7f7de9bff700 1 -- 172.21.15.97:0/3900847744 wait complete. 2023-07-06T03:52:16.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:52:16.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.489+0000 7f9a7f7ff700 1 Processor -- start 2023-07-06T03:52:16.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.490+0000 7f9a7f7ff700 1 -- start start 2023-07-06T03:52:16.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.491+0000 7f9a7f7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78154890 0x7f9a780cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.491+0000 7f9a7f7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7815b490 con 0x7f9a7815eb90 2023-07-06T03:52:16.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.491+0000 7f9a7f7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7815b6e0 con 0x7f9a7815b050 2023-07-06T03:52:16.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.491+0000 7f9a7f7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7815e0b0 con 0x7f9a78154890 2023-07-06T03:52:16.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.491+0000 7f9a7cd9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9a7815eb90 0x7f9a7815a940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46614/0 (socket says 172.21.15.97:46614) 2023-07-06T03:52:16.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.491+0000 7f9a7cd9a700 1 -- 172.21.15.97:0/800393981 learned_addr learned my addr 172.21.15.97:0/800393981 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.491+0000 7f9a7d59b700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78154890 0x7f9a780cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.491+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882095286 0 0) 0x7f9a7815b490 con 0x7f9a7815eb90 2023-07-06T03:52:16.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a640035b0 con 0x7f9a7815eb90 2023-07-06T03:52:16.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835643021 0 0) 0x7f9a7815b6e0 con 0x7f9a7815b050 2023-07-06T03:52:16.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a7815b490 con 0x7f9a7815b050 2023-07-06T03:52:16.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737619748 0 0) 0x7f9a640035b0 con 0x7f9a7815eb90 2023-07-06T03:52:16.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78154890 msgr2=0x7f9a780cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:16.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78154890 0x7f9a780cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 >> v1:172.21.15.146:6789/0 conn(0x7f9a7815b050 legacy=0x7f9a780c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7815e390 con 0x7f9a7815eb90 2023-07-06T03:52:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f9a68001670 con 0x7f9a7815eb90 2023-07-06T03:52:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 >> v1:172.21.15.97:6789/0 conn(0x7f9a7815eb90 legacy=0x7f9a7815a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 0x7f9a64003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78154890 0x7f9a64004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a64004d90 0x7f9a64005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7815e0b0 con 0x7f9a64003bc0 2023-07-06T03:52:16.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6400bac0 con 0x7f9a64004d90 2023-07-06T03:52:16.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6400bc30 con 0x7f9a78154890 2023-07-06T03:52:16.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a7d59b700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 0x7f9a64003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a7cd9a700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78154890 0x7f9a64004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.492+0000 7f9a7dd9c700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a64004d90 0x7f9a64005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.493+0000 7f9a7d59b700 1 -- 172.21.15.97:0/800393981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78154890 msgr2=0x7f9a64004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:16.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.493+0000 7f9a7d59b700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a78154890 0x7f9a64004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.493+0000 7f9a7d59b700 1 -- 172.21.15.97:0/800393981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a64004d90 msgr2=0x7f9a64005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.493+0000 7f9a7d59b700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a64004d90 0x7f9a64005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.493+0000 7f9a7d59b700 1 -- 172.21.15.97:0/800393981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a640035b0 con 0x7f9a64003bc0 2023-07-06T03:52:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.493+0000 7f9a7d59b700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 0x7f9a64003fb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009b60 tx=0x7f9a64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d33b0efe497c56 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.493+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7400e660 con 0x7f9a64003bc0 2023-07-06T03:52:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.493+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7400e7c0 con 0x7f9a64003bc0 2023-07-06T03:52:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.493+0000 7f9a6ffff700 1 -- 172.21.15.97:0/800393981 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7400eb30 con 0x7f9a64003bc0 2023-07-06T03:52:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.494+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/800393981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 msgr2=0x7f9a64003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.494+0000 7f9a7f7ff700 1 --2- 172.21.15.97:0/800393981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 0x7f9a64003fb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009b60 tx=0x7f9a64005880 comp rx=0 tx=0).stop 2023-07-06T03:52:16.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.494+0000 7f9a7dd9c700 1 -- 172.21.15.97:0/800393981 reap_dead start 2023-07-06T03:52:16.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.494+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/800393981 shutdown_connections 2023-07-06T03:52:16.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.494+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/800393981 >> 172.21.15.97:0/800393981 conn(0x7f9a7806a060 msgr2=0x7f9a7806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:16.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.494+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/800393981 shutdown_connections 2023-07-06T03:52:16.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.494+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/800393981 wait complete. 2023-07-06T03:52:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.496+0000 7f9a7f7ff700 1 Processor -- start 2023-07-06T03:52:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.496+0000 7f9a7f7ff700 1 -- start start 2023-07-06T03:52:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7f7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 0x7f9a780acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7f7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a780ccc20 0x7f9a780b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7f7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a780b1e10 0x7f9a7814dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7f7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780b4780 con 0x7f9a64003bc0 2023-07-06T03:52:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7f7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a781f5710 con 0x7f9a780ccc20 2023-07-06T03:52:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7f7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a781f59b0 con 0x7f9a780b1e10 2023-07-06T03:52:16.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7d59b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 0x7f9a780acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7d59b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 0x7f9a780acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60812/0 (socket says 172.21.15.97:60812) 2023-07-06T03:52:16.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7d59b700 1 -- 172.21.15.97:0/2878523768 learned_addr learned my addr 172.21.15.97:0/2878523768 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:16.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7d59b700 1 -- 172.21.15.97:0/2878523768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a780b1e10 msgr2=0x7f9a7814dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7d59b700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a780b1e10 0x7f9a7814dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7d59b700 1 -- 172.21.15.97:0/2878523768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a780ccc20 msgr2=0x7f9a780b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7d59b700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a780ccc20 0x7f9a780b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.498+0000 7f9a7d59b700 1 -- 172.21.15.97:0/2878523768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a781f5c90 con 0x7f9a64003bc0 2023-07-06T03:52:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.499+0000 7f9a7d59b700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 0x7f9a780acae0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e4b0 tx=0x7f9a740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.499+0000 7f9a6dffb700 1 -- 172.21.15.97:0/2878523768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7400e660 con 0x7f9a64003bc0 2023-07-06T03:52:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.499+0000 7f9a6dffb700 1 -- 172.21.15.97:0/2878523768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a74003ab0 con 0x7f9a64003bc0 2023-07-06T03:52:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.499+0000 7f9a6dffb700 1 -- 172.21.15.97:0/2878523768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a74003e60 con 0x7f9a64003bc0 2023-07-06T03:52:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.499+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/2878523768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a781f5f70 con 0x7f9a64003bc0 2023-07-06T03:52:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.499+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/2878523768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a781f6530 con 0x7f9a64003bc0 2023-07-06T03:52:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.500+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/2878523768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7804e340 con 0x7f9a64003bc0 2023-07-06T03:52:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.501+0000 7f9a6dffb700 1 -- 172.21.15.97:0/2878523768 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a74013470 con 0x7f9a64003bc0 2023-07-06T03:52:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.502+0000 7f9a6dffb700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 0x7f9a5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.502+0000 7f9a6dffb700 1 -- 172.21.15.97:0/2878523768 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a7408ad70 con 0x7f9a64003bc0 2023-07-06T03:52:16.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.502+0000 7f9a7cd9a700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 0x7f9a5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.503+0000 7f9a7cd9a700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 0x7f9a5c06d190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9a68000c00 tx=0x7f9a68001170 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.506+0000 7f9a6dffb700 1 -- 172.21.15.97:0/2878523768 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9a74055f30 con 0x7f9a64003bc0 2023-07-06T03:52:16.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.703+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/2878523768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a7806bef0 con 0x7f9a64003bc0 2023-07-06T03:52:16.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.704+0000 7f9a6dffb700 1 -- 172.21.15.97:0/2878523768 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a74059550 con 0x7f9a64003bc0 2023-07-06T03:52:16.706 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:52:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/2878523768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 msgr2=0x7f9a5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 0x7f9a5c06d190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9a68000c00 tx=0x7f9a68001170 comp rx=0 tx=0).stop 2023-07-06T03:52:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/2878523768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 msgr2=0x7f9a780acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 0x7f9a780acae0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400e4b0 tx=0x7f9a740047c0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/2878523768 shutdown_connections 2023-07-06T03:52:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a5c06ace0 0x7f9a5c06d190 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a780b1e10 0x7f9a7814dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a64003bc0 0x7f9a780acae0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 --2- 172.21.15.97:0/2878523768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a780ccc20 0x7f9a780b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/2878523768 >> 172.21.15.97:0/2878523768 conn(0x7f9a7806a060 msgr2=0x7f9a780c4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/2878523768 shutdown_connections 2023-07-06T03:52:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.706+0000 7f9a7f7ff700 1 -- 172.21.15.97:0/2878523768 wait complete. 2023-07-06T03:52:16.726 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:52:16.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.923+0000 7fabc35ff700 1 Processor -- start 2023-07-06T03:52:16.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.923+0000 7fabc35ff700 1 -- start start 2023-07-06T03:52:16.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.924+0000 7fabc35ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fabbc15c610 0x7fabbc0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.925+0000 7fabc35ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabbc0cff90 con 0x7fabbc06b360 2023-07-06T03:52:16.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.925+0000 7fabc35ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabbc0b5650 con 0x7fabbc0d2210 2023-07-06T03:52:16.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.925+0000 7fabc35ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabbc0d0780 con 0x7fabbc15c610 2023-07-06T03:52:16.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.925+0000 7fabc0b9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fabbc06b360 0x7fabbc0b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46622/0 (socket says 172.21.15.97:46622) 2023-07-06T03:52:16.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.925+0000 7fabc0b9a700 1 -- 172.21.15.97:0/1268790452 learned_addr learned my addr 172.21.15.97:0/1268790452 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.925+0000 7fabc139b700 1 --2- 172.21.15.97:0/1268790452 >> v2:172.21.15.146:3300/0 conn(0x7fabbc15c610 0x7fabbc0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.925+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361649746 0 0) 0x7fabbc0cff90 con 0x7fabbc06b360 2023-07-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.925+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba80035b0 con 0x7fabbc06b360 2023-07-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.925+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3490577845 0 0) 0x7faba80035b0 con 0x7fabbc06b360 2023-07-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.925+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 >> v2:172.21.15.146:3300/0 conn(0x7fabbc15c610 msgr2=0x7fabbc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.925+0000 7fabb3fff700 1 --2- 172.21.15.97:0/1268790452 >> v2:172.21.15.146:3300/0 conn(0x7fabbc15c610 0x7fabbc0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 >> v1:172.21.15.146:6789/0 conn(0x7fabbc0d2210 legacy=0x7fabbc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc0b5400 con 0x7fabbc06b360 2023-07-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fabac001670 con 0x7fabbc06b360 2023-07-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 >> v1:172.21.15.97:6789/0 conn(0x7fabbc06b360 legacy=0x7fabbc0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabb3fff700 1 --2- 172.21.15.97:0/1268790452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba8003a00 0x7faba8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabb3fff700 1 --2- 172.21.15.97:0/1268790452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faba80043f0 0x7faba80048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabb3fff700 1 --2- 172.21.15.97:0/1268790452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 0x7faba800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0d0780 con 0x7faba80050b0 2023-07-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba800c000 con 0x7faba80043f0 2023-07-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba800c1c0 con 0x7faba8003a00 2023-07-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabc139b700 1 --2- 172.21.15.97:0/1268790452 >> v2:172.21.15.146:3300/0 conn(0x7fabbc15c610 0x7fabbc0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabc1b9c700 1 --2- 172.21.15.97:0/1268790452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 0x7faba800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabc0b9a700 1 --2- 172.21.15.97:0/1268790452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faba80043f0 0x7faba80048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabc139b700 1 --2- 172.21.15.97:0/1268790452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba8003a00 0x7faba8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabc1b9c700 1 -- 172.21.15.97:0/1268790452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba8003a00 msgr2=0x7faba8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:16.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabc1b9c700 1 --2- 172.21.15.97:0/1268790452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faba8003a00 0x7faba8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabc1b9c700 1 -- 172.21.15.97:0/1268790452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faba80043f0 msgr2=0x7faba80048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:16.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabc1b9c700 1 --2- 172.21.15.97:0/1268790452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faba80043f0 0x7faba80048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.926+0000 7fabc1b9c700 1 -- 172.21.15.97:0/1268790452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80035b0 con 0x7faba80050b0 2023-07-06T03:52:16.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.927+0000 7fabc1b9c700 1 -- 172.21.15.97:0/1268790452 reap_dead start 2023-07-06T03:52:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.927+0000 7fabc1b9c700 1 --2- 172.21.15.97:0/1268790452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 0x7faba800ba10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c290 tx=0x7faba8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa07c9652266a90 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.927+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4001e70 con 0x7faba80050b0 2023-07-06T03:52:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.927+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb400ce70 con 0x7faba80050b0 2023-07-06T03:52:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.927+0000 7fabb3fff700 1 -- 172.21.15.97:0/1268790452 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4003c20 con 0x7faba80050b0 2023-07-06T03:52:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.927+0000 7fabc35ff700 1 -- 172.21.15.97:0/1268790452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 msgr2=0x7faba800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.927+0000 7fabc35ff700 1 --2- 172.21.15.97:0/1268790452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 0x7faba800ba10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c290 tx=0x7faba8005880 comp rx=0 tx=0).stop 2023-07-06T03:52:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.927+0000 7fabc35ff700 1 -- 172.21.15.97:0/1268790452 shutdown_connections 2023-07-06T03:52:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.927+0000 7fabc35ff700 1 --2- 172.21.15.97:0/1268790452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 0x7faba800ba10 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.927+0000 7fabc35ff700 1 -- 172.21.15.97:0/1268790452 >> 172.21.15.97:0/1268790452 conn(0x7fabbc06a6c0 msgr2=0x7fabbc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.928+0000 7fabc35ff700 1 -- 172.21.15.97:0/1268790452 shutdown_connections 2023-07-06T03:52:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.928+0000 7fabc35ff700 1 -- 172.21.15.97:0/1268790452 wait complete. 2023-07-06T03:52:16.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.929+0000 7fabc35ff700 1 Processor -- start 2023-07-06T03:52:16.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.929+0000 7fabc35ff700 1 -- start start 2023-07-06T03:52:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.930+0000 7fabc35ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 0x7fabbc0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.930+0000 7fabc35ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabbc0ca340 0x7fabbc1f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.930+0000 7fabc35ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabbc1f5640 0x7fabbc14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.930+0000 7fabc35ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc1f5ab0 con 0x7faba80050b0 2023-07-06T03:52:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.930+0000 7fabc35ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0c8bf0 con 0x7fabbc1f5640 2023-07-06T03:52:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.930+0000 7fabc35ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0b6d90 con 0x7fabbc0ca340 2023-07-06T03:52:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 0x7fabbc0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 0x7fabbc0c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60820/0 (socket says 172.21.15.97:60820) 2023-07-06T03:52:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc139b700 1 -- 172.21.15.97:0/3276708978 learned_addr learned my addr 172.21.15.97:0/3276708978 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc0b9a700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabbc0ca340 0x7fabbc1f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc1b9c700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabbc1f5640 0x7fabbc14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc139b700 1 -- 172.21.15.97:0/3276708978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabbc0ca340 msgr2=0x7fabbc1f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc139b700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabbc0ca340 0x7fabbc1f5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc139b700 1 -- 172.21.15.97:0/3276708978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabbc1f5640 msgr2=0x7fabbc14e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:16.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc139b700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabbc1f5640 0x7fabbc14e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:16.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc139b700 1 -- 172.21.15.97:0/3276708978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc0b7070 con 0x7faba80050b0 2023-07-06T03:52:16.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc0b9a700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabbc0ca340 0x7fabbc1f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:16.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabc139b700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 0x7fabbc0c9e00 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fabb8010a30 tx=0x7fabb8010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.931+0000 7fabb1ffb700 1 -- 172.21.15.97:0/3276708978 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb800e930 con 0x7faba80050b0 2023-07-06T03:52:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.932+0000 7fabb1ffb700 1 -- 172.21.15.97:0/3276708978 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb800ea90 con 0x7faba80050b0 2023-07-06T03:52:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.932+0000 7fabb1ffb700 1 -- 172.21.15.97:0/3276708978 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb800ee00 con 0x7faba80050b0 2023-07-06T03:52:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.932+0000 7fabc35ff700 1 -- 172.21.15.97:0/3276708978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabbc0b7350 con 0x7faba80050b0 2023-07-06T03:52:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.932+0000 7fabc35ff700 1 -- 172.21.15.97:0/3276708978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabbc0b7910 con 0x7faba80050b0 2023-07-06T03:52:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.933+0000 7fabc35ff700 1 -- 172.21.15.97:0/3276708978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabbc0cfe10 con 0x7faba80050b0 2023-07-06T03:52:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.934+0000 7fabb1ffb700 1 -- 172.21.15.97:0/3276708978 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fabb8016460 con 0x7faba80050b0 2023-07-06T03:52:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.934+0000 7fabb1ffb700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faba006ace0 0x7faba006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.935+0000 7fabb1ffb700 1 -- 172.21.15.97:0/3276708978 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fabb80899b0 con 0x7faba80050b0 2023-07-06T03:52:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.935+0000 7fabc0b9a700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faba006ace0 0x7faba006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.935+0000 7fabc0b9a700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faba006ace0 0x7faba006d190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0c9720 tx=0x7fabac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:16.939+0000 7fabb1ffb700 1 -- 172.21.15.97:0/3276708978 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fabb8054af0 con 0x7faba80050b0 2023-07-06T03:52:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.133+0000 7fabc35ff700 1 -- 172.21.15.97:0/3276708978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fabbc0c6860 con 0x7faba80050b0 2023-07-06T03:52:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.133+0000 7fabb1ffb700 1 -- 172.21.15.97:0/3276708978 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabb8009300 con 0x7faba80050b0 2023-07-06T03:52:17.136 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:52:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 -- 172.21.15.97:0/3276708978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faba006ace0 msgr2=0x7faba006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faba006ace0 0x7faba006d190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0c9720 tx=0x7fabac009040 comp rx=0 tx=0).stop 2023-07-06T03:52:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 -- 172.21.15.97:0/3276708978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 msgr2=0x7fabbc0c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 0x7fabbc0c9e00 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fabb8010a30 tx=0x7fabb8010df0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 -- 172.21.15.97:0/3276708978 shutdown_connections 2023-07-06T03:52:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faba006ace0 0x7faba006d190 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabbc1f5640 0x7fabbc14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba80050b0 0x7fabbc0c9e00 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 --2- 172.21.15.97:0/3276708978 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fabbc0ca340 0x7fabbc1f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 -- 172.21.15.97:0/3276708978 >> 172.21.15.97:0/3276708978 conn(0x7fabbc06a6c0 msgr2=0x7fabbc0b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 -- 172.21.15.97:0/3276708978 shutdown_connections 2023-07-06T03:52:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.136+0000 7fabc35ff700 1 -- 172.21.15.97:0/3276708978 wait complete. 2023-07-06T03:52:17.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:52:17.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.349+0000 7f6da47ff700 1 Processor -- start 2023-07-06T03:52:17.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.349+0000 7f6da47ff700 1 -- start start 2023-07-06T03:52:17.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.350+0000 7f6da47ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d9c15c520 0x7f6d9c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:17.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.350+0000 7f6da47ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9c0cdae0 con 0x7f6d9c160d00 2023-07-06T03:52:17.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.350+0000 7f6da47ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9c0b6640 con 0x7f6d9c160180 2023-07-06T03:52:17.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.350+0000 7f6da47ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9c0cd8d0 con 0x7f6d9c15c520 2023-07-06T03:52:17.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.351+0000 7f6da2d9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6d9c160d00 0x7f6d9c0b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46626/0 (socket says 172.21.15.97:46626) 2023-07-06T03:52:17.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.351+0000 7f6da259b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d9c15c520 0x7f6d9c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:17.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.351+0000 7f6da2d9c700 1 -- 172.21.15.97:0/1531471720 learned_addr learned my addr 172.21.15.97:0/1531471720 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:17.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.351+0000 7f6da1599700 1 -- 172.21.15.97:0/1531471720 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208342649 0 0) 0x7f6d9c0cdae0 con 0x7f6d9c160d00 2023-07-06T03:52:17.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.351+0000 7f6da259b700 1 -- 172.21.15.97:0/1531471720 >> v1:172.21.15.146:6789/0 conn(0x7f6d9c160180 legacy=0x7f6d9c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.351+0000 7f6da259b700 1 -- 172.21.15.97:0/1531471720 >> v1:172.21.15.97:6789/0 conn(0x7f6d9c160d00 legacy=0x7f6d9c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.351+0000 7f6da259b700 1 -- 172.21.15.97:0/1531471720 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0b6920 con 0x7f6d9c15c520 2023-07-06T03:52:17.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.351+0000 7f6da259b700 1 --2- 172.21.15.97:0/1531471720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c15c520 0x7f6d9c0cd390 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009840 tx=0x7f6d98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87351cc54341501 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:17.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.352+0000 7f6da1599700 1 -- 172.21.15.97:0/1531471720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98011830 con 0x7f6d9c15c520 2023-07-06T03:52:17.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.352+0000 7f6da1599700 1 -- 172.21.15.97:0/1531471720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d98011990 con 0x7f6d9c15c520 2023-07-06T03:52:17.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.352+0000 7f6da1599700 1 -- 172.21.15.97:0/1531471720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98011d20 con 0x7f6d9c15c520 2023-07-06T03:52:17.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.352+0000 7f6da47ff700 1 -- 172.21.15.97:0/1531471720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c15c520 msgr2=0x7f6d9c0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.352+0000 7f6da47ff700 1 --2- 172.21.15.97:0/1531471720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c15c520 0x7f6d9c0cd390 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009840 tx=0x7f6d98009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:17.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.352+0000 7f6da47ff700 1 -- 172.21.15.97:0/1531471720 shutdown_connections 2023-07-06T03:52:17.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.352+0000 7f6da47ff700 1 --2- 172.21.15.97:0/1531471720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c15c520 0x7f6d9c0cd390 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.352+0000 7f6da47ff700 1 -- 172.21.15.97:0/1531471720 >> 172.21.15.97:0/1531471720 conn(0x7f6d9c071790 msgr2=0x7f6d9c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.352+0000 7f6da47ff700 1 -- 172.21.15.97:0/1531471720 shutdown_connections 2023-07-06T03:52:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.352+0000 7f6da47ff700 1 -- 172.21.15.97:0/1531471720 wait complete. 2023-07-06T03:52:17.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.354+0000 7f6da47ff700 1 Processor -- start 2023-07-06T03:52:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.354+0000 7f6da47ff700 1 -- start start 2023-07-06T03:52:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.355+0000 7f6da47ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d9c15c520 0x7f6d9c0d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.355+0000 7f6da47ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c160180 0x7f6d9c0c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.355+0000 7f6da47ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d9c160d00 0x7f6d9c0ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.355+0000 7f6da47ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0ccdb0 con 0x7f6d9c160180 2023-07-06T03:52:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.355+0000 7f6da47ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0d3090 con 0x7f6d9c15c520 2023-07-06T03:52:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.355+0000 7f6da47ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0cc130 con 0x7f6d9c160d00 2023-07-06T03:52:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.355+0000 7f6da1d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c160180 0x7f6d9c0c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.355+0000 7f6da1d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c160180 0x7f6d9c0c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60844/0 (socket says 172.21.15.97:60844) 2023-07-06T03:52:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.355+0000 7f6da1d9a700 1 -- 172.21.15.97:0/2386977828 learned_addr learned my addr 172.21.15.97:0/2386977828 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.355+0000 7f6da2d9c700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d9c160d00 0x7f6d9c0ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.355+0000 7f6da259b700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d9c15c520 0x7f6d9c0d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6da1d9a700 1 -- 172.21.15.97:0/2386977828 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d9c160d00 msgr2=0x7f6d9c0ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6da1d9a700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d9c160d00 0x7f6d9c0ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6da1d9a700 1 -- 172.21.15.97:0/2386977828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d9c15c520 msgr2=0x7f6d9c0d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6da1d9a700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d9c15c520 0x7f6d9c0d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6da1d9a700 1 -- 172.21.15.97:0/2386977828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d980094a0 con 0x7f6d9c160180 2023-07-06T03:52:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6da2d9c700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d9c160d00 0x7f6d9c0ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6da259b700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d9c15c520 0x7f6d9c0d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6da1d9a700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c160180 0x7f6d9c0c9f30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c4e0 tx=0x7f6d8c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6d92ffd700 1 -- 172.21.15.97:0/2386977828 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c0052d0 con 0x7f6d9c160180 2023-07-06T03:52:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6d92ffd700 1 -- 172.21.15.97:0/2386977828 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c005430 con 0x7f6d9c160180 2023-07-06T03:52:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6d92ffd700 1 -- 172.21.15.97:0/2386977828 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c0038c0 con 0x7f6d9c160180 2023-07-06T03:52:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6da47ff700 1 -- 172.21.15.97:0/2386977828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9c0cc410 con 0x7f6d9c160180 2023-07-06T03:52:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.356+0000 7f6da47ff700 1 -- 172.21.15.97:0/2386977828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9c0cc9d0 con 0x7f6d9c160180 2023-07-06T03:52:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.357+0000 7f6da47ff700 1 -- 172.21.15.97:0/2386977828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9c0d0c40 con 0x7f6d9c160180 2023-07-06T03:52:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.358+0000 7f6d92ffd700 1 -- 172.21.15.97:0/2386977828 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6d8c003a20 con 0x7f6d9c160180 2023-07-06T03:52:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.359+0000 7f6d92ffd700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8006ace0 0x7f6d8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.359+0000 7f6d92ffd700 1 -- 172.21.15.97:0/2386977828 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6d8c0889b0 con 0x7f6d9c160180 2023-07-06T03:52:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.359+0000 7f6da259b700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8006ace0 0x7f6d8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.360+0000 7f6da259b700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8006ace0 0x7f6d8006d190 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6d980090b0 tx=0x7f6d9801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.364+0000 7f6d92ffd700 1 -- 172.21.15.97:0/2386977828 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6d9c0d0c40 con 0x7f6d9c160180 2023-07-06T03:52:17.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.599+0000 7f6da47ff700 1 -- 172.21.15.97:0/2386977828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d9c0d0c40 con 0x7f6d9c160180 2023-07-06T03:52:17.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.600+0000 7f6d92ffd700 1 -- 172.21.15.97:0/2386977828 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4680 (secure 0 0 0) 0x7f6d9c0d0c40 con 0x7f6d9c160180 2023-07-06T03:52:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:17.602 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2344840493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2344840493},{"type":"v1","addr":"172.21.15.97:6836","nonce":2344840493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1930131645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1930131645},{"type":"v1","addr":"172.21.15.146:6835","nonce":1930131645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3936349837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3936349837},{"type":"v1","addr":"172.21.15.146:6833","nonce":3936349837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:49:34.310076+0000","modified":"2023-07-06T03:52:15.843340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6837":{"gid":6837,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":45,"addr":"172.21.15.97:6837/1957695290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1957695290},{"type":"v1","addr":"172.21.15.97:6837","nonce":1957695290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-06T03:52:17.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 -- 172.21.15.97:0/2386977828 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8006ace0 msgr2=0x7f6d8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8006ace0 0x7f6d8006d190 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6d980090b0 tx=0x7f6d9801c040 comp rx=0 tx=0).stop 2023-07-06T03:52:17.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 -- 172.21.15.97:0/2386977828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c160180 msgr2=0x7f6d9c0c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c160180 0x7f6d9c0c9f30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c4e0 tx=0x7f6d8c00c8a0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 -- 172.21.15.97:0/2386977828 shutdown_connections 2023-07-06T03:52:17.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8006ace0 0x7f6d8006d190 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d9c160d00 0x7f6d9c0ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d9c15c520 0x7f6d9c0d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 --2- 172.21.15.97:0/2386977828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d9c160180 0x7f6d9c0c9f30 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 -- 172.21.15.97:0/2386977828 >> 172.21.15.97:0/2386977828 conn(0x7f6d9c071790 msgr2=0x7f6d9c0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:17.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 -- 172.21.15.97:0/2386977828 shutdown_connections 2023-07-06T03:52:17.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.602+0000 7f6da47ff700 1 -- 172.21.15.97:0/2386977828 wait complete. 2023-07-06T03:52:17.606 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 47 2023-07-06T03:52:17.622 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:52:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.814+0000 7f16ed1ff700 1 Processor -- start 2023-07-06T03:52:17.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.815+0000 7f16ed1ff700 1 -- start start 2023-07-06T03:52:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.816+0000 7f16ed1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e815c520 0x7f16e80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.816+0000 7f16ed1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e80cdae0 con 0x7f16e8160180 2023-07-06T03:52:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.816+0000 7f16ed1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e80b6640 con 0x7f16e8160d00 2023-07-06T03:52:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.816+0000 7f16ed1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80cd8d0 con 0x7f16e815c520 2023-07-06T03:52:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.816+0000 7f16e659c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f16e8160180 0x7f16e81605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46642/0 (socket says 172.21.15.97:46642) 2023-07-06T03:52:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.816+0000 7f16e659c700 1 -- 172.21.15.97:0/297100852 learned_addr learned my addr 172.21.15.97:0/297100852 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:17.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.816+0000 7f16e6d9d700 1 --2- 172.21.15.97:0/297100852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e815c520 0x7f16e80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.816+0000 7f16e5d9b700 1 -- 172.21.15.97:0/297100852 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213470272 0 0) 0x7f16e80cdae0 con 0x7f16e8160180 2023-07-06T03:52:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.817+0000 7f16e5d9b700 1 -- 172.21.15.97:0/297100852 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16d00035b0 con 0x7f16e8160180 2023-07-06T03:52:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.817+0000 7f16e5d9b700 1 -- 172.21.15.97:0/297100852 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588383178 0 0) 0x7f16e80b6640 con 0x7f16e8160d00 2023-07-06T03:52:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.817+0000 7f16e5d9b700 1 -- 172.21.15.97:0/297100852 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16e80cdae0 con 0x7f16e8160d00 2023-07-06T03:52:17.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.817+0000 7f16e6d9d700 1 -- 172.21.15.97:0/297100852 >> v1:172.21.15.146:6789/0 conn(0x7f16e8160d00 legacy=0x7f16e80b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.817+0000 7f16e6d9d700 1 -- 172.21.15.97:0/297100852 >> v1:172.21.15.97:6789/0 conn(0x7f16e8160180 legacy=0x7f16e81605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.817+0000 7f16e6d9d700 1 -- 172.21.15.97:0/297100852 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e80b6920 con 0x7f16e815c520 2023-07-06T03:52:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.817+0000 7f16e6d9d700 1 --2- 172.21.15.97:0/297100852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e815c520 0x7f16e80cd390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009870 tx=0x7f16d8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a37b4bd0d97250a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.817+0000 7f16e5d9b700 1 -- 172.21.15.97:0/297100852 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8011830 con 0x7f16e815c520 2023-07-06T03:52:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.817+0000 7f16e5d9b700 1 -- 172.21.15.97:0/297100852 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8011990 con 0x7f16e815c520 2023-07-06T03:52:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.817+0000 7f16e5d9b700 1 -- 172.21.15.97:0/297100852 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8011d20 con 0x7f16e815c520 2023-07-06T03:52:17.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.818+0000 7f16ed1ff700 1 -- 172.21.15.97:0/297100852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e815c520 msgr2=0x7f16e80cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.818+0000 7f16ed1ff700 1 --2- 172.21.15.97:0/297100852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e815c520 0x7f16e80cd390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009870 tx=0x7f16d8009b80 comp rx=0 tx=0).stop 2023-07-06T03:52:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.818+0000 7f16ed1ff700 1 -- 172.21.15.97:0/297100852 shutdown_connections 2023-07-06T03:52:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.818+0000 7f16ed1ff700 1 --2- 172.21.15.97:0/297100852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e815c520 0x7f16e80cd390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.818+0000 7f16ed1ff700 1 -- 172.21.15.97:0/297100852 >> 172.21.15.97:0/297100852 conn(0x7f16e8071790 msgr2=0x7f16e8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.818+0000 7f16ed1ff700 1 -- 172.21.15.97:0/297100852 shutdown_connections 2023-07-06T03:52:17.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.818+0000 7f16ed1ff700 1 -- 172.21.15.97:0/297100852 wait complete. 2023-07-06T03:52:17.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.820+0000 7f16ed1ff700 1 Processor -- start 2023-07-06T03:52:17.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.820+0000 7f16ed1ff700 1 -- start start 2023-07-06T03:52:17.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16ed1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e815c520 0x7f16e80d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:17.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16ed1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e8160180 0x7f16e80c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:17.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16ed1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e8160d00 0x7f16e814cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:17.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16ed1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80d3bf0 con 0x7f16e8160180 2023-07-06T03:52:17.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16ed1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80cc4a0 con 0x7f16e8160d00 2023-07-06T03:52:17.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16ed1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80c6db0 con 0x7f16e815c520 2023-07-06T03:52:17.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16e659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e8160180 0x7f16e80c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:17.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16e659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e8160180 0x7f16e80c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60860/0 (socket says 172.21.15.97:60860) 2023-07-06T03:52:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16e659c700 1 -- 172.21.15.97:0/2053506183 learned_addr learned my addr 172.21.15.97:0/2053506183 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16e6d9d700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e815c520 0x7f16e80d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16e759e700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e8160d00 0x7f16e814cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16e659c700 1 -- 172.21.15.97:0/2053506183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e815c520 msgr2=0x7f16e80d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16e659c700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e815c520 0x7f16e80d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16e659c700 1 -- 172.21.15.97:0/2053506183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e8160d00 msgr2=0x7f16e814cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16e659c700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e8160d00 0x7f16e814cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16e659c700 1 -- 172.21.15.97:0/2053506183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80094a0 con 0x7f16e8160180 2023-07-06T03:52:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.821+0000 7f16e759e700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e8160d00 0x7f16e814cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.822+0000 7f16e659c700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e8160180 0x7f16e80c6870 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c700 tx=0x7f16dc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.822+0000 7f16cf7fe700 1 -- 172.21.15.97:0/2053506183 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc005d70 con 0x7f16e8160180 2023-07-06T03:52:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.822+0000 7f16cf7fe700 1 -- 172.21.15.97:0/2053506183 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16dc0070d0 con 0x7f16e8160180 2023-07-06T03:52:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.822+0000 7f16ed1ff700 1 -- 172.21.15.97:0/2053506183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e80d1810 con 0x7f16e8160180 2023-07-06T03:52:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.822+0000 7f16cf7fe700 1 -- 172.21.15.97:0/2053506183 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc007460 con 0x7f16e8160180 2023-07-06T03:52:17.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.822+0000 7f16ed1ff700 1 -- 172.21.15.97:0/2053506183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e80d1dd0 con 0x7f16e8160180 2023-07-06T03:52:17.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.823+0000 7f16ed1ff700 1 -- 172.21.15.97:0/2053506183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e8066fe0 con 0x7f16e8160180 2023-07-06T03:52:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.824+0000 7f16cf7fe700 1 -- 172.21.15.97:0/2053506183 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f16dc0075c0 con 0x7f16e8160180 2023-07-06T03:52:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.825+0000 7f16cf7fe700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f16d406ace0 0x7f16d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.826+0000 7f16cf7fe700 1 -- 172.21.15.97:0/2053506183 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f16dc088b30 con 0x7f16e8160180 2023-07-06T03:52:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.826+0000 7f16e6d9d700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f16d406ace0 0x7f16d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.826+0000 7f16e6d9d700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f16d406ace0 0x7f16d406d190 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f16d80090b0 tx=0x7f16d801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:17.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:17.830+0000 7f16cf7fe700 1 -- 172.21.15.97:0/2053506183 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f16dc053ac0 con 0x7f16e8160180 2023-07-06T03:52:18.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.029+0000 7f16ed1ff700 1 -- 172.21.15.97:0/2053506183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16e804e340 con 0x7f16e8160180 2023-07-06T03:52:18.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.030+0000 7f16cf7fe700 1 -- 172.21.15.97:0/2053506183 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18320 (secure 0 0 0) 0x7f16dc0570e0 con 0x7f16e8160180 2023-07-06T03:52:18.032 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:18.032 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":74,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:49:35.835648+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:49:32.371992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-06T03:49:33.281260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:52:18.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 -- 172.21.15.97:0/2053506183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f16d406ace0 msgr2=0x7f16d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f16d406ace0 0x7f16d406d190 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f16d80090b0 tx=0x7f16d801c040 comp rx=0 tx=0).stop 2023-07-06T03:52:18.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 -- 172.21.15.97:0/2053506183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e8160180 msgr2=0x7f16e80c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e8160180 0x7f16e80c6870 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c700 tx=0x7f16dc00ca10 comp rx=0 tx=0).stop 2023-07-06T03:52:18.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 -- 172.21.15.97:0/2053506183 shutdown_connections 2023-07-06T03:52:18.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f16d406ace0 0x7f16d406d190 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e8160d00 0x7f16e814cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f16e815c520 0x7f16e80d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 --2- 172.21.15.97:0/2053506183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16e8160180 0x7f16e80c6870 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 -- 172.21.15.97:0/2053506183 >> 172.21.15.97:0/2053506183 conn(0x7f16e8071790 msgr2=0x7f16e80b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:18.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 -- 172.21.15.97:0/2053506183 shutdown_connections 2023-07-06T03:52:18.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.033+0000 7f16ed1ff700 1 -- 172.21.15.97:0/2053506183 wait complete. 2023-07-06T03:52:18.052 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T03:52:18.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:52:18.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.246+0000 7f6db5fff700 1 Processor -- start 2023-07-06T03:52:18.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.246+0000 7f6db5fff700 1 -- start start 2023-07-06T03:52:18.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.247+0000 7f6db5fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6db015ca10 0x7f6db0160e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.247+0000 7f6db5fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6db00b50b0 0x7f6db00d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.247+0000 7f6db5fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db00d1190 con 0x7f6db00cf3c0 2023-07-06T03:52:18.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.247+0000 7f6db5fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db0161340 con 0x7f6db015ca10 2023-07-06T03:52:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.247+0000 7f6db5fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db00cd680 con 0x7f6db00b50b0 2023-07-06T03:52:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.247+0000 7f6daf7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6db015ca10 0x7f6db0160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.248+0000 7f6daf7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6db015ca10 0x7f6db0160e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60866/0 (socket says 172.21.15.97:60866) 2023-07-06T03:52:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.248+0000 7f6daf7fe700 1 -- 172.21.15.97:0/2480352563 learned_addr learned my addr 172.21.15.97:0/2480352563 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:18.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.248+0000 7f6daffff700 1 --2- 172.21.15.97:0/2480352563 >> v2:172.21.15.146:3300/0 conn(0x7f6db00b50b0 0x7f6db00d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.248+0000 7f6daf7fe700 1 -- 172.21.15.97:0/2480352563 >> v2:172.21.15.146:3300/0 conn(0x7f6db00b50b0 msgr2=0x7f6db00d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:18.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.248+0000 7f6daf7fe700 1 --2- 172.21.15.97:0/2480352563 >> v2:172.21.15.146:3300/0 conn(0x7f6db00b50b0 0x7f6db00d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.248+0000 7f6daf7fe700 1 -- 172.21.15.97:0/2480352563 >> v1:172.21.15.146:6789/0 conn(0x7f6db00cf3c0 legacy=0x7f6db00b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:18.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.248+0000 7f6daf7fe700 1 -- 172.21.15.97:0/2480352563 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db00cd960 con 0x7f6db015ca10 2023-07-06T03:52:18.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.248+0000 7f6daf7fe700 1 --2- 172.21.15.97:0/2480352563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db015ca10 0x7f6db0160e00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6db00b2910 tx=0x7f6da0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce154682f53ecebf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:18.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.248+0000 7f6daeffd700 1 -- 172.21.15.97:0/2480352563 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0004230 con 0x7f6db015ca10 2023-07-06T03:52:18.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.248+0000 7f6daeffd700 1 -- 172.21.15.97:0/2480352563 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da0004390 con 0x7f6db015ca10 2023-07-06T03:52:18.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.248+0000 7f6daeffd700 1 -- 172.21.15.97:0/2480352563 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da00036a0 con 0x7f6db015ca10 2023-07-06T03:52:18.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.249+0000 7f6db5fff700 1 -- 172.21.15.97:0/2480352563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db015ca10 msgr2=0x7f6db0160e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.249+0000 7f6db5fff700 1 --2- 172.21.15.97:0/2480352563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db015ca10 0x7f6db0160e00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6db00b2910 tx=0x7f6da0009bd0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.249+0000 7f6db5fff700 1 -- 172.21.15.97:0/2480352563 shutdown_connections 2023-07-06T03:52:18.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.249+0000 7f6db5fff700 1 --2- 172.21.15.97:0/2480352563 >> v2:172.21.15.146:3300/0 conn(0x7f6db00b50b0 0x7f6db00d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.249+0000 7f6db5fff700 1 --2- 172.21.15.97:0/2480352563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db015ca10 0x7f6db0160e00 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.249+0000 7f6db5fff700 1 -- 172.21.15.97:0/2480352563 >> 172.21.15.97:0/2480352563 conn(0x7f6db006a920 msgr2=0x7f6db006ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:18.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.249+0000 7f6db5fff700 1 -- 172.21.15.97:0/2480352563 shutdown_connections 2023-07-06T03:52:18.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.249+0000 7f6db5fff700 1 -- 172.21.15.97:0/2480352563 wait complete. 2023-07-06T03:52:18.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.251+0000 7f6db5fff700 1 Processor -- start 2023-07-06T03:52:18.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.252+0000 7f6db5fff700 1 -- start start 2023-07-06T03:52:18.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.253+0000 7f6db5fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6db00b50b0 0x7f6db0157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.253+0000 7f6db5fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6db00cf3c0 0x7f6db0157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.253+0000 7f6db5fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db015ca10 0x7f6db00c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.253+0000 7f6db5fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db00b66d0 con 0x7f6db015ca10 2023-07-06T03:52:18.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.253+0000 7f6db5fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db0158ee0 con 0x7f6db00cf3c0 2023-07-06T03:52:18.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.253+0000 7f6db5fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db015bcf0 con 0x7f6db00b50b0 2023-07-06T03:52:18.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.253+0000 7f6daffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db015ca10 0x7f6db00c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.253+0000 7f6daffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db015ca10 0x7f6db00c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60870/0 (socket says 172.21.15.97:60870) 2023-07-06T03:52:18.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.253+0000 7f6daffff700 1 -- 172.21.15.97:0/405469875 learned_addr learned my addr 172.21.15.97:0/405469875 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:18.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.253+0000 7f6daf7fe700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6db00b50b0 0x7f6db0157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.253+0000 7f6da7fff700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6db00cf3c0 0x7f6db0157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6daffff700 1 -- 172.21.15.97:0/405469875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6db00b50b0 msgr2=0x7f6db0157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6daffff700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6db00b50b0 0x7f6db0157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6daffff700 1 -- 172.21.15.97:0/405469875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6db00cf3c0 msgr2=0x7f6db0157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6daffff700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6db00cf3c0 0x7f6db0157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6daffff700 1 -- 172.21.15.97:0/405469875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da0009550 con 0x7f6db015ca10 2023-07-06T03:52:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6daf7fe700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6db00b50b0 0x7f6db0157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6daffff700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db015ca10 0x7f6db00c8c20 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6da8007e60 tx=0x7f6da800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6dacff9700 1 -- 172.21.15.97:0/405469875 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800eea0 con 0x7f6db015ca10 2023-07-06T03:52:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6dacff9700 1 -- 172.21.15.97:0/405469875 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da8014420 con 0x7f6db015ca10 2023-07-06T03:52:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6db5fff700 1 -- 172.21.15.97:0/405469875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db015bfd0 con 0x7f6db015ca10 2023-07-06T03:52:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6dacff9700 1 -- 172.21.15.97:0/405469875 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80147d0 con 0x7f6db015ca10 2023-07-06T03:52:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.254+0000 7f6db5fff700 1 -- 172.21.15.97:0/405469875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db00b6360 con 0x7f6db015ca10 2023-07-06T03:52:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.255+0000 7f6db5fff700 1 -- 172.21.15.97:0/405469875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db006a790 con 0x7f6db015ca10 2023-07-06T03:52:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.256+0000 7f6dacff9700 1 -- 172.21.15.97:0/405469875 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6da8014930 con 0x7f6db015ca10 2023-07-06T03:52:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.257+0000 7f6dacff9700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d9006ace0 0x7f6d9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.257+0000 7f6dacff9700 1 -- 172.21.15.97:0/405469875 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6da8089d30 con 0x7f6db015ca10 2023-07-06T03:52:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.257+0000 7f6daf7fe700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d9006ace0 0x7f6d9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.258+0000 7f6daf7fe700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d9006ace0 0x7f6d9006d190 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009130 tx=0x7f6da0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.261+0000 7f6dacff9700 1 -- 172.21.15.97:0/405469875 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6da8054d40 con 0x7f6db015ca10 2023-07-06T03:52:18.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.500+0000 7f6db5fff700 1 -- 172.21.15.97:0/405469875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6db00ce920 con 0x7f6db015ca10 2023-07-06T03:52:18.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.500+0000 7f6dacff9700 1 -- 172.21.15.97:0/405469875 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4680 (secure 0 0 0) 0x7f6da8058360 con 0x7f6db015ca10 2023-07-06T03:52:18.502 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:18.503 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2344840493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2344840493},{"type":"v1","addr":"172.21.15.97:6836","nonce":2344840493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1930131645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1930131645},{"type":"v1","addr":"172.21.15.146:6835","nonce":1930131645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3936349837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3936349837},{"type":"v1","addr":"172.21.15.146:6833","nonce":3936349837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:49:34.310076+0000","modified":"2023-07-06T03:52:15.843340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6837":{"gid":6837,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":45,"addr":"172.21.15.97:6837/1957695290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1957695290},{"type":"v1","addr":"172.21.15.97:6837","nonce":1957695290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-06T03:52:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 -- 172.21.15.97:0/405469875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d9006ace0 msgr2=0x7f6d9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d9006ace0 0x7f6d9006d190 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009130 tx=0x7f6da0008030 comp rx=0 tx=0).stop 2023-07-06T03:52:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 -- 172.21.15.97:0/405469875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db015ca10 msgr2=0x7f6db00c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db015ca10 0x7f6db00c8c20 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6da8007e60 tx=0x7f6da800e3c0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 -- 172.21.15.97:0/405469875 shutdown_connections 2023-07-06T03:52:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d9006ace0 0x7f6d9006d190 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db015ca10 0x7f6db00c8c20 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6db00b50b0 0x7f6db0157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 --2- 172.21.15.97:0/405469875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6db00cf3c0 0x7f6db0157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 -- 172.21.15.97:0/405469875 >> 172.21.15.97:0/405469875 conn(0x7f6db006a920 msgr2=0x7f6db00d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 -- 172.21.15.97:0/405469875 shutdown_connections 2023-07-06T03:52:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.503+0000 7f6db5fff700 1 -- 172.21.15.97:0/405469875 wait complete. 2023-07-06T03:52:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 47 2023-07-06T03:52:18.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:52:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.714+0000 7feef0bff700 1 Processor -- start 2023-07-06T03:52:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.715+0000 7feef0bff700 1 -- start start 2023-07-06T03:52:18.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.716+0000 7feef0bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec15c520 0x7feeec0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.716+0000 7feef0bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7feeec160d00 0x7feeec0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.716+0000 7feef0bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec0cdae0 con 0x7feeec160180 2023-07-06T03:52:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.716+0000 7feef0bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec0cd8d0 con 0x7feeec160d00 2023-07-06T03:52:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.716+0000 7feef0bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0b6640 con 0x7feeec15c520 2023-07-06T03:52:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.716+0000 7feeead9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7feeec160d00 0x7feeec0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.716+0000 7feeea59c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec15c520 0x7feeec0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.716+0000 7feeea59c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec15c520 0x7feeec0cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:38854/0 (socket says 172.21.15.97:38854) 2023-07-06T03:52:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.716+0000 7feeead9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7feeec160d00 0x7feeec0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:55810/0 (socket says 172.21.15.97:55810) 2023-07-06T03:52:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.716+0000 7feeea59c700 1 -- 172.21.15.97:0/2388346270 learned_addr learned my addr 172.21.15.97:0/2388346270 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.717+0000 7feeea59c700 1 -- 172.21.15.97:0/2388346270 >> v2:172.21.15.146:3300/0 conn(0x7feeec160d00 msgr2=0x7feeec0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.717+0000 7feeea59c700 1 --2- 172.21.15.97:0/2388346270 >> v2:172.21.15.146:3300/0 conn(0x7feeec160d00 0x7feeec0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.717+0000 7feee959a700 1 -- 172.21.15.97:0/2388346270 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417270070 0 0) 0x7feeec0cdae0 con 0x7feeec160180 2023-07-06T03:52:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.717+0000 7feeea59c700 1 -- 172.21.15.97:0/2388346270 >> v1:172.21.15.146:6789/0 conn(0x7feeec160180 legacy=0x7feeec1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.717+0000 7feeea59c700 1 -- 172.21.15.97:0/2388346270 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec0b6920 con 0x7feeec15c520 2023-07-06T03:52:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.717+0000 7feeead9d700 1 --2- 172.21.15.97:0/2388346270 >> v2:172.21.15.146:3300/0 conn(0x7feeec160d00 0x7feeec0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:18.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.717+0000 7feeea59c700 1 --2- 172.21.15.97:0/2388346270 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec15c520 0x7feeec0cd390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feedc0098f0 tx=0x7feedc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9982d5d45376537 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:18.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.717+0000 7feee959a700 1 -- 172.21.15.97:0/2388346270 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc011830 con 0x7feeec15c520 2023-07-06T03:52:18.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.717+0000 7feee959a700 1 -- 172.21.15.97:0/2388346270 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc011990 con 0x7feeec15c520 2023-07-06T03:52:18.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.717+0000 7feee959a700 1 -- 172.21.15.97:0/2388346270 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc011d20 con 0x7feeec15c520 2023-07-06T03:52:18.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.718+0000 7feef0bff700 1 -- 172.21.15.97:0/2388346270 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec15c520 msgr2=0x7feeec0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.718+0000 7feef0bff700 1 --2- 172.21.15.97:0/2388346270 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec15c520 0x7feeec0cd390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feedc0098f0 tx=0x7feedc009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:18.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.718+0000 7feef0bff700 1 -- 172.21.15.97:0/2388346270 shutdown_connections 2023-07-06T03:52:18.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.718+0000 7feef0bff700 1 --2- 172.21.15.97:0/2388346270 >> v2:172.21.15.146:3300/0 conn(0x7feeec160d00 0x7feeec0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.718+0000 7feef0bff700 1 --2- 172.21.15.97:0/2388346270 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec15c520 0x7feeec0cd390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.718+0000 7feef0bff700 1 -- 172.21.15.97:0/2388346270 >> 172.21.15.97:0/2388346270 conn(0x7feeec071790 msgr2=0x7feeec071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:18.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.718+0000 7feef0bff700 1 -- 172.21.15.97:0/2388346270 shutdown_connections 2023-07-06T03:52:18.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.718+0000 7feef0bff700 1 -- 172.21.15.97:0/2388346270 wait complete. 2023-07-06T03:52:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.720+0000 7feef0bff700 1 Processor -- start 2023-07-06T03:52:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.720+0000 7feef0bff700 1 -- start start 2023-07-06T03:52:18.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feef0bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feeec15c520 0x7feeec0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feef0bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeec160180 0x7feeec15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feef0bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec160d00 0x7feeec15e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feef0bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec14dba0 con 0x7feeec160180 2023-07-06T03:52:18.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feef0bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0cbe40 con 0x7feeec15c520 2023-07-06T03:52:18.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feef0bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0cb230 con 0x7feeec160d00 2023-07-06T03:52:18.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feee9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeec160180 0x7feeec15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feee9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeec160180 0x7feeec15dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60882/0 (socket says 172.21.15.97:60882) 2023-07-06T03:52:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feee9d9b700 1 -- 172.21.15.97:0/2744873473 learned_addr learned my addr 172.21.15.97:0/2744873473 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feeea59c700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feeec15c520 0x7feeec0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feeead9d700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec160d00 0x7feeec15e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feee9d9b700 1 -- 172.21.15.97:0/2744873473 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec160d00 msgr2=0x7feeec15e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feee9d9b700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec160d00 0x7feeec15e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feee9d9b700 1 -- 172.21.15.97:0/2744873473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feeec15c520 msgr2=0x7feeec0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feee9d9b700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feeec15c520 0x7feeec0d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.721+0000 7feee9d9b700 1 -- 172.21.15.97:0/2744873473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee4009a70 con 0x7feeec160180 2023-07-06T03:52:18.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.722+0000 7feee9d9b700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeec160180 0x7feeec15dbe0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7feee00011e0 tx=0x7feee000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:18.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.722+0000 7feedaffd700 1 -- 172.21.15.97:0/2744873473 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee00054e0 con 0x7feeec160180 2023-07-06T03:52:18.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.722+0000 7feedaffd700 1 -- 172.21.15.97:0/2744873473 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee0003680 con 0x7feeec160180 2023-07-06T03:52:18.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.722+0000 7feef0bff700 1 -- 172.21.15.97:0/2744873473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc009550 con 0x7feeec160180 2023-07-06T03:52:18.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.722+0000 7feedaffd700 1 -- 172.21.15.97:0/2744873473 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0003a30 con 0x7feeec160180 2023-07-06T03:52:18.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.722+0000 7feef0bff700 1 -- 172.21.15.97:0/2744873473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec0cb7f0 con 0x7feeec160180 2023-07-06T03:52:18.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.723+0000 7feef0bff700 1 -- 172.21.15.97:0/2744873473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeec04e340 con 0x7feeec160180 2023-07-06T03:52:18.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.724+0000 7feedaffd700 1 -- 172.21.15.97:0/2744873473 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feee0003b90 con 0x7feeec160180 2023-07-06T03:52:18.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.725+0000 7feedaffd700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feecc06ace0 0x7feecc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:18.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.725+0000 7feedaffd700 1 -- 172.21.15.97:0/2744873473 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feee0088a90 con 0x7feeec160180 2023-07-06T03:52:18.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.725+0000 7feeea59c700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feecc06ace0 0x7feecc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:18.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.726+0000 7feeea59c700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feecc06ace0 0x7feecc06d190 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7feedc014040 tx=0x7feedc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:18.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.729+0000 7feedaffd700 1 -- 172.21.15.97:0/2744873473 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feee0053aa0 con 0x7feeec160180 2023-07-06T03:52:18.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.970+0000 7feef0bff700 1 -- 172.21.15.97:0/2744873473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feeec0b4f40 con 0x7feeec160180 2023-07-06T03:52:18.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.971+0000 7feedaffd700 1 -- 172.21.15.97:0/2744873473 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4680 (secure 0 0 0) 0x7feee0017020 con 0x7feeec160180 2023-07-06T03:52:18.973 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:18.974 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2344840493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2344840493},{"type":"v1","addr":"172.21.15.97:6836","nonce":2344840493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1930131645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1930131645},{"type":"v1","addr":"172.21.15.146:6835","nonce":1930131645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3936349837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3936349837},{"type":"v1","addr":"172.21.15.146:6833","nonce":3936349837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:49:34.310076+0000","modified":"2023-07-06T03:52:15.843340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6837":{"gid":6837,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":45,"addr":"172.21.15.97:6837/1957695290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1957695290},{"type":"v1","addr":"172.21.15.97:6837","nonce":1957695290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-07-06T03:52:18.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.973+0000 7feef0bff700 1 -- 172.21.15.97:0/2744873473 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feecc06ace0 msgr2=0x7feecc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.973+0000 7feef0bff700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feecc06ace0 0x7feecc06d190 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7feedc014040 tx=0x7feedc008030 comp rx=0 tx=0).stop 2023-07-06T03:52:18.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.973+0000 7feef0bff700 1 -- 172.21.15.97:0/2744873473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeec160180 msgr2=0x7feeec15dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:18.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.973+0000 7feef0bff700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeec160180 0x7feeec15dbe0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7feee00011e0 tx=0x7feee000cae0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.973+0000 7feef0bff700 1 -- 172.21.15.97:0/2744873473 shutdown_connections 2023-07-06T03:52:18.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.973+0000 7feef0bff700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feecc06ace0 0x7feecc06d190 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.973+0000 7feef0bff700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feeec160d00 0x7feeec15e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.973+0000 7feef0bff700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feeec15c520 0x7feeec0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.973+0000 7feef0bff700 1 --2- 172.21.15.97:0/2744873473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeec160180 0x7feeec15dbe0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:18.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.973+0000 7feef0bff700 1 -- 172.21.15.97:0/2744873473 >> 172.21.15.97:0/2744873473 conn(0x7feeec071790 msgr2=0x7feeec0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:18.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.974+0000 7feef0bff700 1 -- 172.21.15.97:0/2744873473 shutdown_connections 2023-07-06T03:52:18.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:18.974+0000 7feef0bff700 1 -- 172.21.15.97:0/2744873473 wait complete. 2023-07-06T03:52:18.977 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 47 2023-07-06T03:52:18.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:52:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.185+0000 7f89c85ff700 1 Processor -- start 2023-07-06T03:52:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.185+0000 7f89c85ff700 1 -- start start 2023-07-06T03:52:19.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.186+0000 7f89c85ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f89c015ca10 0x7f89c0160e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.186+0000 7f89c85ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 0x7f89c00d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.186+0000 7f89c85ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c00d1190 con 0x7f89c00cf3c0 2023-07-06T03:52:19.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.186+0000 7f89c85ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c0161340 con 0x7f89c015ca10 2023-07-06T03:52:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.186+0000 7f89c85ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00cd680 con 0x7f89c00b50b0 2023-07-06T03:52:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.187+0000 7f89c6b9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 0x7f89c00d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.187+0000 7f89c639b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f89c015ca10 0x7f89c0160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.187+0000 7f89c6b9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 0x7f89c00d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:38870/0 (socket says 172.21.15.97:38870) 2023-07-06T03:52:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.187+0000 7f89c6b9c700 1 -- 172.21.15.97:0/4039089995 learned_addr learned my addr 172.21.15.97:0/4039089995 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.187+0000 7f89c5399700 1 -- 172.21.15.97:0/4039089995 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720303350 0 0) 0x7f89c00d1190 con 0x7f89c00cf3c0 2023-07-06T03:52:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.187+0000 7f89c5399700 1 -- 172.21.15.97:0/4039089995 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89ac0035b0 con 0x7f89c00cf3c0 2023-07-06T03:52:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.187+0000 7f89c6b9c700 1 -- 172.21.15.97:0/4039089995 >> v2:172.21.15.146:3300/0 conn(0x7f89c015ca10 msgr2=0x7f89c0160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.187+0000 7f89c6b9c700 1 --2- 172.21.15.97:0/4039089995 >> v2:172.21.15.146:3300/0 conn(0x7f89c015ca10 0x7f89c0160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.188+0000 7f89c6b9c700 1 -- 172.21.15.97:0/4039089995 >> v1:172.21.15.146:6789/0 conn(0x7f89c00cf3c0 legacy=0x7f89c00b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.188+0000 7f89c6b9c700 1 -- 172.21.15.97:0/4039089995 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c00cd960 con 0x7f89c00b50b0 2023-07-06T03:52:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.188+0000 7f89c639b700 1 --2- 172.21.15.97:0/4039089995 >> v2:172.21.15.146:3300/0 conn(0x7f89c015ca10 0x7f89c0160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.188+0000 7f89c6b9c700 1 --2- 172.21.15.97:0/4039089995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 0x7f89c00d1620 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009e50 tx=0x7f89b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=422b3e61c16d3727 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.188+0000 7f89c5399700 1 -- 172.21.15.97:0/4039089995 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b800e070 con 0x7f89c00b50b0 2023-07-06T03:52:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.188+0000 7f89c5399700 1 -- 172.21.15.97:0/4039089995 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b800b910 con 0x7f89c00b50b0 2023-07-06T03:52:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.188+0000 7f89c5399700 1 -- 172.21.15.97:0/4039089995 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b800bc80 con 0x7f89c00b50b0 2023-07-06T03:52:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.189+0000 7f89c85ff700 1 -- 172.21.15.97:0/4039089995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 msgr2=0x7f89c00d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.189+0000 7f89c85ff700 1 --2- 172.21.15.97:0/4039089995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 0x7f89c00d1620 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009e50 tx=0x7f89b800b330 comp rx=0 tx=0).stop 2023-07-06T03:52:19.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.189+0000 7f89c85ff700 1 -- 172.21.15.97:0/4039089995 shutdown_connections 2023-07-06T03:52:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.189+0000 7f89c85ff700 1 --2- 172.21.15.97:0/4039089995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 0x7f89c00d1620 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.189+0000 7f89c85ff700 1 --2- 172.21.15.97:0/4039089995 >> v2:172.21.15.146:3300/0 conn(0x7f89c015ca10 0x7f89c0160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.189+0000 7f89c85ff700 1 -- 172.21.15.97:0/4039089995 >> 172.21.15.97:0/4039089995 conn(0x7f89c006a920 msgr2=0x7f89c006ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.189+0000 7f89c85ff700 1 -- 172.21.15.97:0/4039089995 shutdown_connections 2023-07-06T03:52:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.189+0000 7f89c85ff700 1 -- 172.21.15.97:0/4039089995 wait complete. 2023-07-06T03:52:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.190+0000 7f89c85ff700 1 Processor -- start 2023-07-06T03:52:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.191+0000 7f89c85ff700 1 -- start start 2023-07-06T03:52:19.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.191+0000 7f89c85ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 0x7f89c0150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.191+0000 7f89c85ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c00cf3c0 0x7f89c014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.191+0000 7f89c85ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89c015ca10 0x7f89c015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c85ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c0150550 con 0x7f89c00cf3c0 2023-07-06T03:52:19.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c85ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c0150eb0 con 0x7f89c015ca10 2023-07-06T03:52:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c85ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c014f8d0 con 0x7f89c00b50b0 2023-07-06T03:52:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c5b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c00cf3c0 0x7f89c014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c5b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c00cf3c0 0x7f89c014e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60890/0 (socket says 172.21.15.97:60890) 2023-07-06T03:52:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c5b9a700 1 -- 172.21.15.97:0/1674810001 learned_addr learned my addr 172.21.15.97:0/1674810001 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c639b700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 0x7f89c0150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c6b9c700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89c015ca10 0x7f89c015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c5b9a700 1 -- 172.21.15.97:0/1674810001 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 msgr2=0x7f89c0150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c5b9a700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 0x7f89c0150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c5b9a700 1 -- 172.21.15.97:0/1674810001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89c015ca10 msgr2=0x7f89c015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c5b9a700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89c015ca10 0x7f89c015bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c5b9a700 1 -- 172.21.15.97:0/1674810001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc0094a0 con 0x7f89c00cf3c0 2023-07-06T03:52:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.192+0000 7f89c5b9a700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c00cf3c0 0x7f89c014e9e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c500 tx=0x7f89b000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.193+0000 7f89b6ffd700 1 -- 172.21.15.97:0/1674810001 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b00052d0 con 0x7f89c00cf3c0 2023-07-06T03:52:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.193+0000 7f89b6ffd700 1 -- 172.21.15.97:0/1674810001 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b0005430 con 0x7f89c00cf3c0 2023-07-06T03:52:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.193+0000 7f89c85ff700 1 -- 172.21.15.97:0/1674810001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b8009b30 con 0x7f89c00cf3c0 2023-07-06T03:52:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.193+0000 7f89b6ffd700 1 -- 172.21.15.97:0/1674810001 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b00038c0 con 0x7f89c00cf3c0 2023-07-06T03:52:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.193+0000 7f89c85ff700 1 -- 172.21.15.97:0/1674810001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c014fe90 con 0x7f89c00cf3c0 2023-07-06T03:52:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.194+0000 7f89c85ff700 1 -- 172.21.15.97:0/1674810001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c006a790 con 0x7f89c00cf3c0 2023-07-06T03:52:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.195+0000 7f89b6ffd700 1 -- 172.21.15.97:0/1674810001 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f89b0003a20 con 0x7f89c00cf3c0 2023-07-06T03:52:19.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.196+0000 7f89b6ffd700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89a406ace0 0x7f89a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.196+0000 7f89b6ffd700 1 -- 172.21.15.97:0/1674810001 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f89b0088940 con 0x7f89c00cf3c0 2023-07-06T03:52:19.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.197+0000 7f89c639b700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89a406ace0 0x7f89a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.197+0000 7f89c639b700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89a406ace0 0x7f89a406d190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009cd0 tx=0x7f89bc009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:19.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.201+0000 7f89b6ffd700 1 -- 172.21.15.97:0/1674810001 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f89b0053950 con 0x7f89c00cf3c0 2023-07-06T03:52:19.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.398+0000 7f89c85ff700 1 -- 172.21.15.97:0/1674810001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89c0150690 con 0x7f89c00cf3c0 2023-07-06T03:52:19.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.399+0000 7f89b6ffd700 1 -- 172.21.15.97:0/1674810001 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18320 (secure 0 0 0) 0x7f89c0150690 con 0x7f89c00cf3c0 2023-07-06T03:52:19.401 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:19.401 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":74,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:49:35.835648+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:49:32.371992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-07-06T03:49:33.281260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:52:19.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 -- 172.21.15.97:0/1674810001 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89a406ace0 msgr2=0x7f89a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89a406ace0 0x7f89a406d190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009cd0 tx=0x7f89bc009210 comp rx=0 tx=0).stop 2023-07-06T03:52:19.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 -- 172.21.15.97:0/1674810001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c00cf3c0 msgr2=0x7f89c014e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c00cf3c0 0x7f89c014e9e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c500 tx=0x7f89b000c810 comp rx=0 tx=0).stop 2023-07-06T03:52:19.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 -- 172.21.15.97:0/1674810001 shutdown_connections 2023-07-06T03:52:19.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f89a406ace0 0x7f89a406d190 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89c015ca10 0x7f89c015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f89c00b50b0 0x7f89c0150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 --2- 172.21.15.97:0/1674810001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c00cf3c0 0x7f89c014e9e0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 -- 172.21.15.97:0/1674810001 >> 172.21.15.97:0/1674810001 conn(0x7f89c006a920 msgr2=0x7f89c0156f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:19.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 -- 172.21.15.97:0/1674810001 shutdown_connections 2023-07-06T03:52:19.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.402+0000 7f89c85ff700 1 -- 172.21.15.97:0/1674810001 wait complete. 2023-07-06T03:52:19.422 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T03:52:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.615+0000 7ffacdbff700 1 Processor -- start 2023-07-06T03:52:19.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.615+0000 7ffacdbff700 1 -- start start 2023-07-06T03:52:19.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.616+0000 7ffacdbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac815c610 0x7ffac80b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.616+0000 7ffacdbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffac80d2210 0x7ffac80d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.616+0000 7ffacdbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac80cff90 con 0x7ffac806b360 2023-07-06T03:52:19.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.616+0000 7ffacdbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac80d0780 con 0x7ffac80d2210 2023-07-06T03:52:19.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.616+0000 7ffacdbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac815c9e0 con 0x7ffac815c610 2023-07-06T03:52:19.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.616+0000 7ffabffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ffac806b360 0x7ffac80b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46646/0 (socket says 172.21.15.97:46646) 2023-07-06T03:52:19.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.616+0000 7ffabffff700 1 -- 172.21.15.97:0/2382880393 learned_addr learned my addr 172.21.15.97:0/2382880393 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:19.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.616+0000 7ffac77fe700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac815c610 0x7ffac80b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.616+0000 7ffac7fff700 1 --2- 172.21.15.97:0/2382880393 >> v2:172.21.15.146:3300/0 conn(0x7ffac80d2210 0x7ffac80d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417479452 0 0) 0x7ffac80cff90 con 0x7ffac806b360 2023-07-06T03:52:19.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffab40035b0 con 0x7ffac806b360 2023-07-06T03:52:19.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3246013817 0 0) 0x7ffab40035b0 con 0x7ffac806b360 2023-07-06T03:52:19.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac815c610 msgr2=0x7ffac80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:19.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac815c610 0x7ffac80b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 >> v2:172.21.15.146:3300/0 conn(0x7ffac80d2210 msgr2=0x7ffac80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:19.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 --2- 172.21.15.97:0/2382880393 >> v2:172.21.15.146:3300/0 conn(0x7ffac80d2210 0x7ffac80d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac80b5650 con 0x7ffac806b360 2023-07-06T03:52:19.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7ffab0001670 con 0x7ffac806b360 2023-07-06T03:52:19.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac77fe700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac815c610 0x7ffac80b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:19.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 >> v1:172.21.15.97:6789/0 conn(0x7ffac806b360 legacy=0x7ffac80b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab4003a30 0x7ffab4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab4004850 0x7ffab4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac815c610 0x7ffab4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac80d0780 con 0x7ffab4004850 2023-07-06T03:52:19.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac815c9e0 con 0x7ffab4003a30 2023-07-06T03:52:19.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab400ba10 con 0x7ffac815c610 2023-07-06T03:52:19.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffabffff700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab4004850 0x7ffab4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac77fe700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab4003a30 0x7ffab4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.617+0000 7ffac7fff700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac815c610 0x7ffab4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.618+0000 7ffabffff700 1 -- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac815c610 msgr2=0x7ffab4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:19.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.618+0000 7ffabffff700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac815c610 0x7ffab4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.618+0000 7ffabffff700 1 -- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab4003a30 msgr2=0x7ffab4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:19.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.618+0000 7ffabffff700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab4003a30 0x7ffab4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.618+0000 7ffabffff700 1 -- 172.21.15.97:0/2382880393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab80094a0 con 0x7ffab4004850 2023-07-06T03:52:19.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.618+0000 7ffabffff700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab4004850 0x7ffab4004cc0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ffac806b790 tx=0x7ffab4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2602e38279eba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:19.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.618+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0007b70 con 0x7ffab4004850 2023-07-06T03:52:19.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.618+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab000ccf0 con 0x7ffab4004850 2023-07-06T03:52:19.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.618+0000 7ffac6ffd700 1 -- 172.21.15.97:0/2382880393 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0005ba0 con 0x7ffab4004850 2023-07-06T03:52:19.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.619+0000 7ffacdbff700 1 -- 172.21.15.97:0/2382880393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab4004850 msgr2=0x7ffab4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.619+0000 7ffacdbff700 1 --2- 172.21.15.97:0/2382880393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab4004850 0x7ffab4004cc0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ffac806b790 tx=0x7ffab4005880 comp rx=0 tx=0).stop 2023-07-06T03:52:19.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.619+0000 7ffac7fff700 1 -- 172.21.15.97:0/2382880393 reap_dead start 2023-07-06T03:52:19.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.619+0000 7ffacdbff700 1 -- 172.21.15.97:0/2382880393 shutdown_connections 2023-07-06T03:52:19.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.619+0000 7ffacdbff700 1 -- 172.21.15.97:0/2382880393 >> 172.21.15.97:0/2382880393 conn(0x7ffac806a6c0 msgr2=0x7ffac806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:19.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.619+0000 7ffacdbff700 1 -- 172.21.15.97:0/2382880393 shutdown_connections 2023-07-06T03:52:19.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.619+0000 7ffacdbff700 1 -- 172.21.15.97:0/2382880393 wait complete. 2023-07-06T03:52:19.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.621+0000 7ffacdbff700 1 Processor -- start 2023-07-06T03:52:19.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.621+0000 7ffacdbff700 1 -- start start 2023-07-06T03:52:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffacdbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab4004850 0x7ffac80cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffacdbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac8152d80 0x7ffac8159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffacdbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac8159b60 0x7ffac80b6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffacdbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac815a000 con 0x7ffac8152d80 2023-07-06T03:52:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffacdbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac80b7310 con 0x7ffab4004850 2023-07-06T03:52:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffacdbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac80b75f0 con 0x7ffac8159b60 2023-07-06T03:52:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffabffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac8152d80 0x7ffac8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffabffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac8152d80 0x7ffac8159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60916/0 (socket says 172.21.15.97:60916) 2023-07-06T03:52:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffabffff700 1 -- 172.21.15.97:0/2180355710 learned_addr learned my addr 172.21.15.97:0/2180355710 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffac77fe700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab4004850 0x7ffac80cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffac7fff700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac8159b60 0x7ffac80b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffabffff700 1 -- 172.21.15.97:0/2180355710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac8159b60 msgr2=0x7ffac80b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffabffff700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac8159b60 0x7ffac80b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffabffff700 1 -- 172.21.15.97:0/2180355710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab4004850 msgr2=0x7ffac80cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffabffff700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab4004850 0x7ffac80cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffabffff700 1 -- 172.21.15.97:0/2180355710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac80b5b60 con 0x7ffac8152d80 2023-07-06T03:52:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.622+0000 7ffabffff700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac8152d80 0x7ffac8159620 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ffab000c720 tx=0x7ffab0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.623+0000 7ffac4ff9700 1 -- 172.21.15.97:0/2180355710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0004c80 con 0x7ffac8152d80 2023-07-06T03:52:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.623+0000 7ffac4ff9700 1 -- 172.21.15.97:0/2180355710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab0004de0 con 0x7ffac8152d80 2023-07-06T03:52:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.623+0000 7ffac4ff9700 1 -- 172.21.15.97:0/2180355710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0007310 con 0x7ffac8152d80 2023-07-06T03:52:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.623+0000 7ffacdbff700 1 -- 172.21.15.97:0/2180355710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac80b5e40 con 0x7ffac8152d80 2023-07-06T03:52:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.623+0000 7ffacdbff700 1 -- 172.21.15.97:0/2180355710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac81531f0 con 0x7ffac8152d80 2023-07-06T03:52:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.624+0000 7ffacdbff700 1 -- 172.21.15.97:0/2180355710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac8153380 con 0x7ffac8152d80 2023-07-06T03:52:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.625+0000 7ffac4ff9700 1 -- 172.21.15.97:0/2180355710 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffab0019070 con 0x7ffac8152d80 2023-07-06T03:52:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.626+0000 7ffac4ff9700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa806ace0 0x7ffaa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.626+0000 7ffac4ff9700 1 -- 172.21.15.97:0/2180355710 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffab008a560 con 0x7ffac8152d80 2023-07-06T03:52:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.626+0000 7ffac77fe700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa806ace0 0x7ffaa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.626+0000 7ffac77fe700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa806ace0 0x7ffaa806d190 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009bd0 tx=0x7ffab8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:19.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.630+0000 7ffac4ff9700 1 -- 172.21.15.97:0/2180355710 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffab0055620 con 0x7ffac8152d80 2023-07-06T03:52:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.867+0000 7ffacdbff700 1 -- 172.21.15.97:0/2180355710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ffac804e340 con 0x7ffac8152d80 2023-07-06T03:52:19.869 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-07-06T03:52:19.868+0000 7fb463ff4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T03:52:19.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.872+0000 7ffac4ff9700 1 -- 172.21.15.97:0/2180355710 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) v1 ==== 161+0+0 (secure 0 0 0) 0x7ffab0058c40 con 0x7ffac8152d80 2023-07-06T03:52:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 -- 172.21.15.97:0/2180355710 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa806ace0 msgr2=0x7ffaa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa806ace0 0x7ffaa806d190 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009bd0 tx=0x7ffab8008040 comp rx=0 tx=0).stop 2023-07-06T03:52:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 -- 172.21.15.97:0/2180355710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac8152d80 msgr2=0x7ffac8159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac8152d80 0x7ffac8159620 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ffab000c720 tx=0x7ffab0001930 comp rx=0 tx=0).stop 2023-07-06T03:52:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 -- 172.21.15.97:0/2180355710 shutdown_connections 2023-07-06T03:52:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa806ace0 0x7ffaa806d190 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac8159b60 0x7ffac80b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab4004850 0x7ffac80cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 --2- 172.21.15.97:0/2180355710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac8152d80 0x7ffac8159620 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:19.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 -- 172.21.15.97:0/2180355710 >> 172.21.15.97:0/2180355710 conn(0x7ffac806a6c0 msgr2=0x7ffac80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:19.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 -- 172.21.15.97:0/2180355710 shutdown_connections 2023-07-06T03:52:19.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:19.875+0000 7ffacdbff700 1 -- 172.21.15.97:0/2180355710 wait complete. 2023-07-06T03:52:19.878 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T03:52:19.895 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T03:52:19.946 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:52:20.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.093+0000 7f61771ff700 1 Processor -- start 2023-07-06T03:52:20.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.094+0000 7f61771ff700 1 -- start start 2023-07-06T03:52:20.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.095+0000 7f61771ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f617006b360 0x7f61700b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:20.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.095+0000 7f61771ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f61700d2210 0x7f61700d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:20.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.095+0000 7f61771ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61700cff90 con 0x7f617015c610 2023-07-06T03:52:20.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.095+0000 7f61771ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61700d0780 con 0x7f61700d2210 2023-07-06T03:52:20.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.095+0000 7f61771ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617015c9e0 con 0x7f617006b360 2023-07-06T03:52:20.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.095+0000 7f617579c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f61700d2210 0x7f61700d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:20.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.095+0000 7f616ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f617006b360 0x7f61700b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:20.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.095+0000 7f617579c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f61700d2210 0x7f61700d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:55670/0 (socket says 172.21.15.97:55670) 2023-07-06T03:52:20.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.095+0000 7f617579c700 1 -- 172.21.15.97:0/274590966 learned_addr learned my addr 172.21.15.97:0/274590966 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:20.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.095+0000 7f6174f9b700 1 --1- 172.21.15.97:0/274590966 >> v1:172.21.15.146:6789/0 conn(0x7f617015c610 0x7f61700b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:50134/0 (socket says 172.21.15.97:50134) 2023-07-06T03:52:20.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.096+0000 7f617579c700 1 -- 172.21.15.97:0/274590966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f617006b360 msgr2=0x7f61700b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:20.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.096+0000 7f617579c700 1 --2- 172.21.15.97:0/274590966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f617006b360 0x7f61700b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:20.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.096+0000 7f617579c700 1 -- 172.21.15.97:0/274590966 >> v1:172.21.15.146:6789/0 conn(0x7f617015c610 legacy=0x7f61700b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:20.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.096+0000 7f617579c700 1 -- 172.21.15.97:0/274590966 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700b5650 con 0x7f61700d2210 2023-07-06T03:52:20.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.096+0000 7f616f7fe700 1 -- 172.21.15.97:0/274590966 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635643800 0 0) 0x7f61700cff90 con 0x7f617015c610 2023-07-06T03:52:20.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.096+0000 7f617579c700 1 --2- 172.21.15.97:0/274590966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61700d2210 0x7f61700d1310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6168009e80 tx=0x7f616800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26b8572985e41983 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:20.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.096+0000 7f616f7fe700 1 -- 172.21.15.97:0/274590966 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800e070 con 0x7f61700d2210 2023-07-06T03:52:20.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.096+0000 7f616f7fe700 1 -- 172.21.15.97:0/274590966 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616800b8d0 con 0x7f61700d2210 2023-07-06T03:52:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.096+0000 7f616f7fe700 1 -- 172.21.15.97:0/274590966 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800bc80 con 0x7f61700d2210 2023-07-06T03:52:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.097+0000 7f61771ff700 1 -- 172.21.15.97:0/274590966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61700d2210 msgr2=0x7f61700d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.097+0000 7f61771ff700 1 --2- 172.21.15.97:0/274590966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61700d2210 0x7f61700d1310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6168009e80 tx=0x7f616800b330 comp rx=0 tx=0).stop 2023-07-06T03:52:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.097+0000 7f61771ff700 1 -- 172.21.15.97:0/274590966 shutdown_connections 2023-07-06T03:52:20.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.097+0000 7f61771ff700 1 --2- 172.21.15.97:0/274590966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f61700d2210 0x7f61700d1310 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:20.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.097+0000 7f61771ff700 1 --2- 172.21.15.97:0/274590966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f617006b360 0x7f61700b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:20.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.097+0000 7f61771ff700 1 -- 172.21.15.97:0/274590966 >> 172.21.15.97:0/274590966 conn(0x7f617006a6c0 msgr2=0x7f617006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:20.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.097+0000 7f61771ff700 1 -- 172.21.15.97:0/274590966 shutdown_connections 2023-07-06T03:52:20.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.097+0000 7f61771ff700 1 -- 172.21.15.97:0/274590966 wait complete. 2023-07-06T03:52:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.099+0000 7f61771ff700 1 Processor -- start 2023-07-06T03:52:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.099+0000 7f61771ff700 1 -- start start 2023-07-06T03:52:20.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f61771ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f617006b360 0x7f61700c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:20.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f61771ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61700d2210 0x7f61700c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:20.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f61771ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f617015c610 0x7f617014b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:20.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f61771ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700c6bf0 con 0x7f61700d2210 2023-07-06T03:52:20.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f61771ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700c8b30 con 0x7f617006b360 2023-07-06T03:52:20.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f61771ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61700c8e00 con 0x7f617015c610 2023-07-06T03:52:20.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f616ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61700d2210 0x7f61700c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:20.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f616ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61700d2210 0x7f61700c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54646/0 (socket says 172.21.15.97:54646) 2023-07-06T03:52:20.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f616ffff700 1 -- 172.21.15.97:0/3298164484 learned_addr learned my addr 172.21.15.97:0/3298164484 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:20.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f6174f9b700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f617006b360 0x7f61700c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:20.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f617579c700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f617015c610 0x7f617014b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:20.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f616ffff700 1 -- 172.21.15.97:0/3298164484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f617015c610 msgr2=0x7f617014b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:20.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f616ffff700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f617015c610 0x7f617014b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:20.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f616ffff700 1 -- 172.21.15.97:0/3298164484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f617006b360 msgr2=0x7f61700c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:20.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f616ffff700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f617006b360 0x7f61700c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.100+0000 7f616ffff700 1 -- 172.21.15.97:0/3298164484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168009b30 con 0x7f61700d2210 2023-07-06T03:52:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.101+0000 7f616ffff700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61700d2210 0x7f61700c85f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f615c00e980 tx=0x7f615c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.101+0000 7f616d7fa700 1 -- 172.21.15.97:0/3298164484 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c009710 con 0x7f61700d2210 2023-07-06T03:52:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.101+0000 7f616d7fa700 1 -- 172.21.15.97:0/3298164484 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c00c690 con 0x7f61700d2210 2023-07-06T03:52:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.101+0000 7f61771ff700 1 -- 172.21.15.97:0/3298164484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61700c90e0 con 0x7f61700d2210 2023-07-06T03:52:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.101+0000 7f616d7fa700 1 -- 172.21.15.97:0/3298164484 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00ca40 con 0x7f61700d2210 2023-07-06T03:52:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.101+0000 7f61771ff700 1 -- 172.21.15.97:0/3298164484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61700c96a0 con 0x7f61700d2210 2023-07-06T03:52:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.102+0000 7f61771ff700 1 -- 172.21.15.97:0/3298164484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617004e340 con 0x7f61700d2210 2023-07-06T03:52:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.103+0000 7f616d7fa700 1 -- 172.21.15.97:0/3298164484 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f615c00cba0 con 0x7f61700d2210 2023-07-06T03:52:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.104+0000 7f616d7fa700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f615406ace0 0x7f615406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.105+0000 7f616d7fa700 1 -- 172.21.15.97:0/3298164484 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f615c089cd0 con 0x7f61700d2210 2023-07-06T03:52:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.105+0000 7f6174f9b700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f615406ace0 0x7f615406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.105+0000 7f6174f9b700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f615406ace0 0x7f615406d190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6164000f60 tx=0x7f6164007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:20.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.107+0000 7f616d7fa700 1 -- 172.21.15.97:0/3298164484 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f615c054d00 con 0x7f61700d2210 2023-07-06T03:52:20.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.344+0000 7f61771ff700 1 -- 172.21.15.97:0/3298164484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f61700b8530 con 0x7f61700d2210 2023-07-06T03:52:20.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.872+0000 7f616d7fa700 1 -- 172.21.15.97:0/3298164484 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7f615c058320 con 0x7f61700d2210 2023-07-06T03:52:20.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 -- 172.21.15.97:0/3298164484 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f615406ace0 msgr2=0x7f615406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f615406ace0 0x7f615406d190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6164000f60 tx=0x7f6164007e70 comp rx=0 tx=0).stop 2023-07-06T03:52:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 -- 172.21.15.97:0/3298164484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61700d2210 msgr2=0x7f61700c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61700d2210 0x7f61700c85f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f615c00e980 tx=0x7f615c00ec90 comp rx=0 tx=0).stop 2023-07-06T03:52:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 -- 172.21.15.97:0/3298164484 shutdown_connections 2023-07-06T03:52:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f615406ace0 0x7f615406d190 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f617015c610 0x7f617014b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f617006b360 0x7f61700c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:20.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 --2- 172.21.15.97:0/3298164484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61700d2210 0x7f61700c85f0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:20.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 -- 172.21.15.97:0/3298164484 >> 172.21.15.97:0/3298164484 conn(0x7f617006a6c0 msgr2=0x7f617015b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:20.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 -- 172.21.15.97:0/3298164484 shutdown_connections 2023-07-06T03:52:20.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:20.876+0000 7f61771ff700 1 -- 172.21.15.97:0/3298164484 wait complete. 2023-07-06T03:52:20.897 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T03:52:21.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.096+0000 7fe0c65ff700 1 Processor -- start 2023-07-06T03:52:21.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.096+0000 7fe0c65ff700 1 -- start start 2023-07-06T03:52:21.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.098+0000 7fe0c65ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c015a4a0 0x7fe0c00cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:21.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.098+0000 7fe0c65ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c00cd730 con 0x7fe0c0071030 2023-07-06T03:52:21.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.098+0000 7fe0c65ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c00ae140 con 0x7fe0c006c3f0 2023-07-06T03:52:21.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.098+0000 7fe0c65ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00ce360 con 0x7fe0c015a4a0 2023-07-06T03:52:21.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.098+0000 7fe0bf7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe0c0071030 0x7fe0c00714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53510/0 (socket says 172.21.15.97:53510) 2023-07-06T03:52:21.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.098+0000 7fe0bf7fe700 1 -- 172.21.15.97:0/2065709489 learned_addr learned my addr 172.21.15.97:0/2065709489 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:21.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.098+0000 7fe0bffff700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c015a4a0 0x7fe0c00cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:21.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.098+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722563995 0 0) 0x7fe0c00cd730 con 0x7fe0c0071030 2023-07-06T03:52:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.098+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a80035b0 con 0x7fe0c0071030 2023-07-06T03:52:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186996758 0 0) 0x7fe0c00ae140 con 0x7fe0c006c3f0 2023-07-06T03:52:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0c00cd730 con 0x7fe0c006c3f0 2023-07-06T03:52:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 652069725 0 0) 0x7fe0a80035b0 con 0x7fe0c0071030 2023-07-06T03:52:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c015a4a0 msgr2=0x7fe0c00cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c015a4a0 0x7fe0c00cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 >> v1:172.21.15.146:6789/0 conn(0x7fe0c006c3f0 legacy=0x7fe0c006c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00ce640 con 0x7fe0c0071030 2023-07-06T03:52:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fe0b40015f0 con 0x7fe0c0071030 2023-07-06T03:52:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0bffff700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c015a4a0 0x7fe0c00cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 >> v1:172.21.15.97:6789/0 conn(0x7fe0c0071030 legacy=0x7fe0c00714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 0x7fe0a8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0a8004850 0x7fe0a8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c015a4a0 0x7fe0a8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00ce360 con 0x7fe0a8003bc0 2023-07-06T03:52:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a800bb60 con 0x7fe0a8004850 2023-07-06T03:52:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a800bd20 con 0x7fe0c015a4a0 2023-07-06T03:52:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0bffff700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 0x7fe0a8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0bf7fe700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0a8004850 0x7fe0a8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0c4b9c700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c015a4a0 0x7fe0a8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0bffff700 1 -- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c015a4a0 msgr2=0x7fe0a8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0bffff700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c015a4a0 0x7fe0a8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0bffff700 1 -- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0a8004850 msgr2=0x7fe0a8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0bffff700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0a8004850 0x7fe0a8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0bffff700 1 -- 172.21.15.97:0/2065709489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b00094a0 con 0x7fe0a8003bc0 2023-07-06T03:52:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.099+0000 7fe0c4b9c700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c015a4a0 0x7fe0a8005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.100+0000 7fe0bffff700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 0x7fe0a8003fb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009d30 tx=0x7fe0a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8e9e14f01f2420 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.100+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0009e80 con 0x7fe0a8003bc0 2023-07-06T03:52:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.100+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b000e9a0 con 0x7fe0a8003bc0 2023-07-06T03:52:21.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.100+0000 7fe0beffd700 1 -- 172.21.15.97:0/2065709489 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000ed10 con 0x7fe0a8003bc0 2023-07-06T03:52:21.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.100+0000 7fe0c65ff700 1 -- 172.21.15.97:0/2065709489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 msgr2=0x7fe0a8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:21.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.100+0000 7fe0c65ff700 1 --2- 172.21.15.97:0/2065709489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 0x7fe0a8003fb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009d30 tx=0x7fe0a8005880 comp rx=0 tx=0).stop 2023-07-06T03:52:21.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.100+0000 7fe0c4b9c700 1 -- 172.21.15.97:0/2065709489 reap_dead start 2023-07-06T03:52:21.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.100+0000 7fe0c65ff700 1 -- 172.21.15.97:0/2065709489 shutdown_connections 2023-07-06T03:52:21.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.100+0000 7fe0c65ff700 1 -- 172.21.15.97:0/2065709489 >> 172.21.15.97:0/2065709489 conn(0x7fe0c00b0510 msgr2=0x7fe0c00b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:21.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.101+0000 7fe0c65ff700 1 -- 172.21.15.97:0/2065709489 shutdown_connections 2023-07-06T03:52:21.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.101+0000 7fe0c65ff700 1 -- 172.21.15.97:0/2065709489 wait complete. 2023-07-06T03:52:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.103+0000 7fe0c65ff700 1 Processor -- start 2023-07-06T03:52:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.103+0000 7fe0c65ff700 1 -- start start 2023-07-06T03:52:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.104+0000 7fe0c65ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 0x7fe0c01f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.104+0000 7fe0c65ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c01f2860 0x7fe0c0159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.104+0000 7fe0c65ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c01efac0 0x7fe0c0159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.104+0000 7fe0c65ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c01f3360 con 0x7fe0a8003bc0 2023-07-06T03:52:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.104+0000 7fe0c65ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c014ecb0 con 0x7fe0c01f2860 2023-07-06T03:52:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.104+0000 7fe0c65ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00c6f10 con 0x7fe0c01efac0 2023-07-06T03:52:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0bffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 0x7fe0c01f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0bffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 0x7fe0c01f2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54660/0 (socket says 172.21.15.97:54660) 2023-07-06T03:52:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0bffff700 1 -- 172.21.15.97:0/1568450082 learned_addr learned my addr 172.21.15.97:0/1568450082 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0bf7fe700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c01f2860 0x7fe0c0159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0c4b9c700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c01efac0 0x7fe0c0159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0bffff700 1 -- 172.21.15.97:0/1568450082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c01efac0 msgr2=0x7fe0c0159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0bffff700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c01efac0 0x7fe0c0159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0bffff700 1 -- 172.21.15.97:0/1568450082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c01f2860 msgr2=0x7fe0c0159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0bffff700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c01f2860 0x7fe0c0159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:21.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0bffff700 1 -- 172.21.15.97:0/1568450082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c01f2da0 con 0x7fe0a8003bc0 2023-07-06T03:52:21.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0bf7fe700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c01f2860 0x7fe0c0159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:21.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.105+0000 7fe0bffff700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 0x7fe0c01f2320 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0004540 tx=0x7fe0b00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:21.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.106+0000 7fe0bcff9700 1 -- 172.21.15.97:0/1568450082 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0003ab0 con 0x7fe0a8003bc0 2023-07-06T03:52:21.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.106+0000 7fe0bcff9700 1 -- 172.21.15.97:0/1568450082 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b0003c30 con 0x7fe0a8003bc0 2023-07-06T03:52:21.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.106+0000 7fe0bcff9700 1 -- 172.21.15.97:0/1568450082 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0009e80 con 0x7fe0a8003bc0 2023-07-06T03:52:21.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.106+0000 7fe0c65ff700 1 -- 172.21.15.97:0/1568450082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c01f3080 con 0x7fe0a8003bc0 2023-07-06T03:52:21.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.106+0000 7fe0c65ff700 1 -- 172.21.15.97:0/1568450082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c00c3d00 con 0x7fe0a8003bc0 2023-07-06T03:52:21.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.107+0000 7fe0c65ff700 1 -- 172.21.15.97:0/1568450082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c00c6a00 con 0x7fe0a8003bc0 2023-07-06T03:52:21.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.107+0000 7fe0bcff9700 1 -- 172.21.15.97:0/1568450082 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe0b0013470 con 0x7fe0a8003bc0 2023-07-06T03:52:21.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.108+0000 7fe0bcff9700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe0ac06ace0 0x7fe0ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:21.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.108+0000 7fe0bcff9700 1 -- 172.21.15.97:0/1568450082 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fe0b008aed0 con 0x7fe0a8003bc0 2023-07-06T03:52:21.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.109+0000 7fe0bf7fe700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe0ac06ace0 0x7fe0ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:21.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.109+0000 7fe0bf7fe700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe0ac06ace0 0x7fe0ac06d190 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4001eb0 tx=0x7fe0b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:21.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.113+0000 7fe0bcff9700 1 -- 172.21.15.97:0/1568450082 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe0c00c6a00 con 0x7fe0a8003bc0 2023-07-06T03:52:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.305+0000 7fe0c65ff700 1 -- 172.21.15.97:0/1568450082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe0c00c6a00 con 0x7fe0a8003bc0 2023-07-06T03:52:21.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.879+0000 7fe0bcff9700 1 -- 172.21.15.97:0/1568450082 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe0c00c6a00 con 0x7fe0a8003bc0 2023-07-06T03:52:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.885+0000 7fe0c65ff700 1 -- 172.21.15.97:0/1568450082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe0ac06ace0 msgr2=0x7fe0ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.885+0000 7fe0c65ff700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe0ac06ace0 0x7fe0ac06d190 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4001eb0 tx=0x7fe0b4009040 comp rx=0 tx=0).stop 2023-07-06T03:52:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.885+0000 7fe0c65ff700 1 -- 172.21.15.97:0/1568450082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 msgr2=0x7fe0c01f2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.885+0000 7fe0c65ff700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 0x7fe0c01f2320 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0004540 tx=0x7fe0b00047c0 comp rx=0 tx=0).stop 2023-07-06T03:52:21.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.886+0000 7fe0c65ff700 1 -- 172.21.15.97:0/1568450082 shutdown_connections 2023-07-06T03:52:21.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.886+0000 7fe0c65ff700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe0ac06ace0 0x7fe0ac06d190 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:21.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.886+0000 7fe0c65ff700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe0c01efac0 0x7fe0c0159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:21.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.886+0000 7fe0c65ff700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0a8003bc0 0x7fe0c01f2320 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:21.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.886+0000 7fe0c65ff700 1 --2- 172.21.15.97:0/1568450082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c01f2860 0x7fe0c0159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:21.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.886+0000 7fe0c65ff700 1 -- 172.21.15.97:0/1568450082 >> 172.21.15.97:0/1568450082 conn(0x7fe0c00b0510 msgr2=0x7fe0c006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:21.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.887+0000 7fe0c65ff700 1 -- 172.21.15.97:0/1568450082 shutdown_connections 2023-07-06T03:52:21.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:21.887+0000 7fe0c65ff700 1 -- 172.21.15.97:0/1568450082 wait complete. 2023-07-06T03:52:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-07-06T03:52:21.909 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T03:52:22.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.105+0000 7f0dde3ff700 1 Processor -- start 2023-07-06T03:52:22.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.105+0000 7f0dde3ff700 1 -- start start 2023-07-06T03:52:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.106+0000 7f0dde3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0dd806b0c0 0x7f0dd80b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.106+0000 7f0dde3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd80cd330 con 0x7f0dd80b85f0 2023-07-06T03:52:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.106+0000 7f0dde3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd80d02a0 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.106+0000 7f0dde3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd80b89c0 con 0x7f0dd806b0c0 2023-07-06T03:52:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.106+0000 7f0dd77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0dd806b0c0 0x7f0dd80b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.107+0000 7f0dd77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0dd806b0c0 0x7f0dd80b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54672/0 (socket says 172.21.15.97:54672) 2023-07-06T03:52:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.107+0000 7f0dd77fe700 1 -- 172.21.15.97:0/2548881770 learned_addr learned my addr 172.21.15.97:0/2548881770 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.107+0000 7f0dd77fe700 1 -- 172.21.15.97:0/2548881770 >> v1:172.21.15.146:6789/0 conn(0x7f0dd80b4ca0 legacy=0x7f0dd80d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:22.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.107+0000 7f0dd77fe700 1 -- 172.21.15.97:0/2548881770 >> v1:172.21.15.97:6789/0 conn(0x7f0dd80b85f0 legacy=0x7f0dd80cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.107+0000 7f0dd77fe700 1 -- 172.21.15.97:0/2548881770 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd80d0540 con 0x7f0dd806b0c0 2023-07-06T03:52:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.107+0000 7f0dd6ffd700 1 -- 172.21.15.97:0/2548881770 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773718530 0 0) 0x7f0dd80cd330 con 0x7f0dd80b85f0 2023-07-06T03:52:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.107+0000 7f0dd77fe700 1 --2- 172.21.15.97:0/2548881770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd806b0c0 0x7f0dd80b4760 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0098f0 tx=0x7f0dcc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c499ce818941c71 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.107+0000 7f0dd6ffd700 1 -- 172.21.15.97:0/2548881770 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc004230 con 0x7f0dd806b0c0 2023-07-06T03:52:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.108+0000 7f0dd6ffd700 1 -- 172.21.15.97:0/2548881770 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc004390 con 0x7f0dd806b0c0 2023-07-06T03:52:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.108+0000 7f0dd6ffd700 1 -- 172.21.15.97:0/2548881770 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0036a0 con 0x7f0dd806b0c0 2023-07-06T03:52:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.108+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2548881770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd806b0c0 msgr2=0x7f0dd80b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.108+0000 7f0dde3ff700 1 --2- 172.21.15.97:0/2548881770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd806b0c0 0x7f0dd80b4760 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0098f0 tx=0x7f0dcc009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.108+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2548881770 shutdown_connections 2023-07-06T03:52:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.108+0000 7f0dde3ff700 1 --2- 172.21.15.97:0/2548881770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd806b0c0 0x7f0dd80b4760 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.108+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2548881770 >> 172.21.15.97:0/2548881770 conn(0x7f0dd806a5b0 msgr2=0x7f0dd806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.108+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2548881770 shutdown_connections 2023-07-06T03:52:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.108+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2548881770 wait complete. 2023-07-06T03:52:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.110+0000 7f0dde3ff700 1 Processor -- start 2023-07-06T03:52:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.110+0000 7f0dde3ff700 1 -- start start 2023-07-06T03:52:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.111+0000 7f0dde3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dd806b0c0 0x7f0dd8149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.111+0000 7f0dde3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd80b4ca0 0x7f0dd815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.111+0000 7f0dde3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0dd80b85f0 0x7f0dd8147e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.111+0000 7f0dde3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd806b510 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.111+0000 7f0dde3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd815b2d0 con 0x7f0dd80b85f0 2023-07-06T03:52:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.111+0000 7f0dde3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd81487f0 con 0x7f0dd806b0c0 2023-07-06T03:52:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.111+0000 7f0dd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd80b4ca0 0x7f0dd815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.111+0000 7f0dd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd80b4ca0 0x7f0dd815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54674/0 (socket says 172.21.15.97:54674) 2023-07-06T03:52:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.111+0000 7f0dd77fe700 1 -- 172.21.15.97:0/2365735727 learned_addr learned my addr 172.21.15.97:0/2365735727 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.111+0000 7f0dd7fff700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dd806b0c0 0x7f0dd8149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.111+0000 7f0ddc99c700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0dd80b85f0 0x7f0dd8147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.112+0000 7f0dd77fe700 1 -- 172.21.15.97:0/2365735727 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dd806b0c0 msgr2=0x7f0dd8149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.112+0000 7f0dd77fe700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dd806b0c0 0x7f0dd8149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.112+0000 7f0dd77fe700 1 -- 172.21.15.97:0/2365735727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0dd80b85f0 msgr2=0x7f0dd8147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.112+0000 7f0dd77fe700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0dd80b85f0 0x7f0dd8147e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.112+0000 7f0dd77fe700 1 -- 172.21.15.97:0/2365735727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc009550 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.112+0000 7f0dd77fe700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd80b4ca0 0x7f0dd815ad90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80b58d0 tx=0x7f0dcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.112+0000 7f0dd4ff9700 1 -- 172.21.15.97:0/2365735727 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc003dc0 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.112+0000 7f0dd4ff9700 1 -- 172.21.15.97:0/2365735727 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc01f070 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.112+0000 7f0dd4ff9700 1 -- 172.21.15.97:0/2365735727 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc00b8a0 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.112+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2365735727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd8148ad0 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.112+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2365735727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd8149090 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.113+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2365735727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd81496e0 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.114+0000 7f0dd4ff9700 1 -- 172.21.15.97:0/2365735727 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0dcc021030 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.115+0000 7f0dd4ff9700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0dc406ace0 0x7f0dc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:22.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.115+0000 7f0dd4ff9700 1 -- 172.21.15.97:0/2365735727 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f0dcc08aac0 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.115+0000 7f0dd7fff700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0dc406ace0 0x7f0dc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:22.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.116+0000 7f0dd7fff700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0dc406ace0 0x7f0dc406d190 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8000f60 tx=0x7f0dc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:22.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.119+0000 7f0dd4ff9700 1 -- 172.21.15.97:0/2365735727 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0dcc055cc0 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.313+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2365735727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0dd81496e0 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.886+0000 7f0dd4ff9700 1 -- 172.21.15.97:0/2365735727 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0dcc0592e0 con 0x7f0dd80b4ca0 2023-07-06T03:52:22.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.892+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2365735727 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0dc406ace0 msgr2=0x7f0dc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:22.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.892+0000 7f0dde3ff700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0dc406ace0 0x7f0dc406d190 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8000f60 tx=0x7f0dc8009040 comp rx=0 tx=0).stop 2023-07-06T03:52:22.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.892+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2365735727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd80b4ca0 msgr2=0x7f0dd815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:22.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.892+0000 7f0dde3ff700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd80b4ca0 0x7f0dd815ad90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80b58d0 tx=0x7f0dcc004750 comp rx=0 tx=0).stop 2023-07-06T03:52:22.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.892+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2365735727 shutdown_connections 2023-07-06T03:52:22.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.892+0000 7f0dde3ff700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0dc406ace0 0x7f0dc406d190 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:22.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.892+0000 7f0dde3ff700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0dd80b85f0 0x7f0dd8147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:22.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.892+0000 7f0dde3ff700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0dd806b0c0 0x7f0dd8149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:22.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.892+0000 7f0dde3ff700 1 --2- 172.21.15.97:0/2365735727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd80b4ca0 0x7f0dd815ad90 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:22.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.892+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2365735727 >> 172.21.15.97:0/2365735727 conn(0x7f0dd806a5b0 msgr2=0x7f0dd80b6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:22.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.893+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2365735727 shutdown_connections 2023-07-06T03:52:22.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:22.893+0000 7f0dde3ff700 1 -- 172.21.15.97:0/2365735727 wait complete. 2023-07-06T03:52:22.897 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-07-06T03:52:22.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-07-06T03:52:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.112+0000 7f72b53ff700 1 Processor -- start 2023-07-06T03:52:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.112+0000 7f72b53ff700 1 -- start start 2023-07-06T03:52:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.113+0000 7f72b53ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b006b360 0x7f72b00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.113+0000 7f72b53ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72b00d2210 0x7f72b00d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.113+0000 7f72b53ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b00cff90 con 0x7f72b015c610 2023-07-06T03:52:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.113+0000 7f72b53ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b00d0780 con 0x7f72b00d2210 2023-07-06T03:52:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.113+0000 7f72b53ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b015c9e0 con 0x7f72b006b360 2023-07-06T03:52:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.114+0000 7f72af7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72b00d2210 0x7f72b00d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.114+0000 7f72af7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72b00d2210 0x7f72b00d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54690/0 (socket says 172.21.15.97:54690) 2023-07-06T03:52:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.114+0000 7f72af7fe700 1 -- 172.21.15.97:0/1888499151 learned_addr learned my addr 172.21.15.97:0/1888499151 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.114+0000 7f72ae7fc700 1 --2- 172.21.15.97:0/1888499151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b006b360 0x7f72b00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.114+0000 7f72af7fe700 1 -- 172.21.15.97:0/1888499151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b006b360 msgr2=0x7f72b00b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.114+0000 7f72af7fe700 1 --2- 172.21.15.97:0/1888499151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b006b360 0x7f72b00b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.114+0000 7f72af7fe700 1 -- 172.21.15.97:0/1888499151 >> v1:172.21.15.146:6789/0 conn(0x7f72b015c610 legacy=0x7f72b00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.114+0000 7f72af7fe700 1 -- 172.21.15.97:0/1888499151 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b00b5650 con 0x7f72b00d2210 2023-07-06T03:52:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.114+0000 7f72ae7fc700 1 --2- 172.21.15.97:0/1888499151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b006b360 0x7f72b00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.114+0000 7f72af7fe700 1 --2- 172.21.15.97:0/1888499151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b00d2210 0x7f72b00d1310 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009e80 tx=0x7f72a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66520fd791d58bfe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:23.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.114+0000 7f72adffb700 1 -- 172.21.15.97:0/1888499151 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a800e070 con 0x7f72b00d2210 2023-07-06T03:52:23.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.115+0000 7f72adffb700 1 -- 172.21.15.97:0/1888499151 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a800b980 con 0x7f72b00d2210 2023-07-06T03:52:23.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.115+0000 7f72adffb700 1 -- 172.21.15.97:0/1888499151 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a800bd30 con 0x7f72b00d2210 2023-07-06T03:52:23.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.115+0000 7f72b53ff700 1 -- 172.21.15.97:0/1888499151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b00d2210 msgr2=0x7f72b00d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.115+0000 7f72b53ff700 1 --2- 172.21.15.97:0/1888499151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b00d2210 0x7f72b00d1310 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009e80 tx=0x7f72a800b330 comp rx=0 tx=0).stop 2023-07-06T03:52:23.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.115+0000 7f72b53ff700 1 -- 172.21.15.97:0/1888499151 shutdown_connections 2023-07-06T03:52:23.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.115+0000 7f72b53ff700 1 --2- 172.21.15.97:0/1888499151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b00d2210 0x7f72b00d1310 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.115+0000 7f72b53ff700 1 --2- 172.21.15.97:0/1888499151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b006b360 0x7f72b00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.115+0000 7f72b53ff700 1 -- 172.21.15.97:0/1888499151 >> 172.21.15.97:0/1888499151 conn(0x7f72b006a6c0 msgr2=0x7f72b006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:23.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.115+0000 7f72b53ff700 1 -- 172.21.15.97:0/1888499151 shutdown_connections 2023-07-06T03:52:23.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.115+0000 7f72b53ff700 1 -- 172.21.15.97:0/1888499151 wait complete. 2023-07-06T03:52:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.117+0000 7f72b53ff700 1 Processor -- start 2023-07-06T03:52:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.117+0000 7f72b53ff700 1 -- start start 2023-07-06T03:52:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.118+0000 7f72b53ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b006b360 0x7f72b00cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.118+0000 7f72b53ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72b00d2210 0x7f72b0154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.118+0000 7f72b53ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b015c610 0x7f72b00c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.118+0000 7f72b53ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b00c6d50 con 0x7f72b006b360 2023-07-06T03:52:23.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.118+0000 7f72b53ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b00c8630 con 0x7f72b00d2210 2023-07-06T03:52:23.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.118+0000 7f72b53ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b00c8890 con 0x7f72b015c610 2023-07-06T03:52:23.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.118+0000 7f72aeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b006b360 0x7f72b00cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.118+0000 7f72aeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b006b360 0x7f72b00cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54700/0 (socket says 172.21.15.97:54700) 2023-07-06T03:52:23.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.118+0000 7f72aeffd700 1 -- 172.21.15.97:0/358982926 learned_addr learned my addr 172.21.15.97:0/358982926 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:23.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72ae7fc700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72b00d2210 0x7f72b0154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72af7fe700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b015c610 0x7f72b00c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72aeffd700 1 -- 172.21.15.97:0/358982926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b015c610 msgr2=0x7f72b00c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72aeffd700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b015c610 0x7f72b00c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72aeffd700 1 -- 172.21.15.97:0/358982926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72b00d2210 msgr2=0x7f72b0154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72aeffd700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72b00d2210 0x7f72b0154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72aeffd700 1 -- 172.21.15.97:0/358982926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a8009b30 con 0x7f72b006b360 2023-07-06T03:52:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72ae7fc700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72b00d2210 0x7f72b0154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72aeffd700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b006b360 0x7f72b00cc280 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f72a000c450 tx=0x7f72a000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72977fe700 1 -- 172.21.15.97:0/358982926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a00052d0 con 0x7f72b006b360 2023-07-06T03:52:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72977fe700 1 -- 172.21.15.97:0/358982926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a0005430 con 0x7f72b006b360 2023-07-06T03:52:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72977fe700 1 -- 172.21.15.97:0/358982926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a00038c0 con 0x7f72b006b360 2023-07-06T03:52:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72b53ff700 1 -- 172.21.15.97:0/358982926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b00c8b70 con 0x7f72b006b360 2023-07-06T03:52:23.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.119+0000 7f72b53ff700 1 -- 172.21.15.97:0/358982926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b00c8e50 con 0x7f72b006b360 2023-07-06T03:52:23.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.120+0000 7f72b53ff700 1 -- 172.21.15.97:0/358982926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b00c6e90 con 0x7f72b006b360 2023-07-06T03:52:23.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.121+0000 7f72977fe700 1 -- 172.21.15.97:0/358982926 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72a0003a20 con 0x7f72b006b360 2023-07-06T03:52:23.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.122+0000 7f72977fe700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f729c06ace0 0x7f729c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.122+0000 7f72977fe700 1 -- 172.21.15.97:0/358982926 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f72a0088510 con 0x7f72b006b360 2023-07-06T03:52:23.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.122+0000 7f72ae7fc700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f729c06ace0 0x7f729c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.123+0000 7f72ae7fc700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f729c06ace0 0x7f729c06d190 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f72b01598e0 tx=0x7f72a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.126+0000 7f72977fe700 1 -- 172.21.15.97:0/358982926 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f72b00c6e90 con 0x7f72b006b360 2023-07-06T03:52:23.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.320+0000 7f72b53ff700 1 -- 172.21.15.97:0/358982926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7f72b00c6e90 con 0x7f72b006b360 2023-07-06T03:52:23.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.324+0000 7f72977fe700 1 -- 172.21.15.97:0/358982926 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v27) v1 ==== 111+0+0 (secure 0 0 0) 0x7f72b00c6e90 con 0x7f72b006b360 2023-07-06T03:52:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 -- 172.21.15.97:0/358982926 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f729c06ace0 msgr2=0x7f729c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f729c06ace0 0x7f729c06d190 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f72b01598e0 tx=0x7f72a4008040 comp rx=0 tx=0).stop 2023-07-06T03:52:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 -- 172.21.15.97:0/358982926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b006b360 msgr2=0x7f72b00cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b006b360 0x7f72b00cc280 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f72a000c450 tx=0x7f72a000c810 comp rx=0 tx=0).stop 2023-07-06T03:52:23.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 -- 172.21.15.97:0/358982926 shutdown_connections 2023-07-06T03:52:23.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f729c06ace0 0x7f729c06d190 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f72b015c610 0x7f72b00c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b006b360 0x7f72b00cc280 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 --2- 172.21.15.97:0/358982926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72b00d2210 0x7f72b0154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 -- 172.21.15.97:0/358982926 >> 172.21.15.97:0/358982926 conn(0x7f72b006a6c0 msgr2=0x7f72b01599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:23.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 -- 172.21.15.97:0/358982926 shutdown_connections 2023-07-06T03:52:23.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.327+0000 7f72b53ff700 1 -- 172.21.15.97:0/358982926 wait complete. 2023-07-06T03:52:23.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-07-06T03:52:23.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.541+0000 7f0c87bff700 1 Processor -- start 2023-07-06T03:52:23.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.542+0000 7f0c87bff700 1 -- start start 2023-07-06T03:52:23.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.543+0000 7f0c87bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c8015eb90 0x7f0c8015b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.543+0000 7f0c87bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8015b6e0 con 0x7f0c800cbaa0 2023-07-06T03:52:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.543+0000 7f0c87bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8015e0b0 con 0x7f0c80154890 2023-07-06T03:52:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.543+0000 7f0c87bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8015e390 con 0x7f0c8015eb90 2023-07-06T03:52:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.543+0000 7f0c8519a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0c800cbaa0 0x7f0c8015a940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53528/0 (socket says 172.21.15.97:53528) 2023-07-06T03:52:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.543+0000 7f0c8519a700 1 -- 172.21.15.97:0/1258900514 learned_addr learned my addr 172.21.15.97:0/1258900514 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.543+0000 7f0c8619c700 1 --2- 172.21.15.97:0/1258900514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c8015eb90 0x7f0c8015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.544+0000 7f0c84999700 1 -- 172.21.15.97:0/1258900514 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60152611 0 0) 0x7f0c8015b6e0 con 0x7f0c800cbaa0 2023-07-06T03:52:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.544+0000 7f0c84999700 1 -- 172.21.15.97:0/1258900514 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c6c0035b0 con 0x7f0c800cbaa0 2023-07-06T03:52:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.544+0000 7f0c84999700 1 -- 172.21.15.97:0/1258900514 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878607473 0 0) 0x7f0c8015e0b0 con 0x7f0c80154890 2023-07-06T03:52:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.544+0000 7f0c84999700 1 -- 172.21.15.97:0/1258900514 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c8015b6e0 con 0x7f0c80154890 2023-07-06T03:52:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.544+0000 7f0c84999700 1 -- 172.21.15.97:0/1258900514 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 659839660 0 0) 0x7f0c6c0035b0 con 0x7f0c800cbaa0 2023-07-06T03:52:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.544+0000 7f0c8619c700 1 -- 172.21.15.97:0/1258900514 >> v1:172.21.15.146:6789/0 conn(0x7f0c80154890 legacy=0x7f0c800cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.544+0000 7f0c8619c700 1 -- 172.21.15.97:0/1258900514 >> v1:172.21.15.97:6789/0 conn(0x7f0c800cbaa0 legacy=0x7f0c8015a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.544+0000 7f0c8619c700 1 -- 172.21.15.97:0/1258900514 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8015e670 con 0x7f0c8015eb90 2023-07-06T03:52:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.544+0000 7f0c8619c700 1 --2- 172.21.15.97:0/1258900514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c8015eb90 0x7f0c8015b050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009dc0 tx=0x7f0c7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ff9d167d4608b94 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.545+0000 7f0c84999700 1 -- 172.21.15.97:0/1258900514 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7800e070 con 0x7f0c8015eb90 2023-07-06T03:52:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.545+0000 7f0c84999700 1 -- 172.21.15.97:0/1258900514 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c78003a20 con 0x7f0c8015eb90 2023-07-06T03:52:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.545+0000 7f0c84999700 1 -- 172.21.15.97:0/1258900514 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70001670 con 0x7f0c8015eb90 2023-07-06T03:52:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.545+0000 7f0c87bff700 1 -- 172.21.15.97:0/1258900514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c8015eb90 msgr2=0x7f0c8015b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.545+0000 7f0c87bff700 1 --2- 172.21.15.97:0/1258900514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c8015eb90 0x7f0c8015b050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009dc0 tx=0x7f0c7800b330 comp rx=0 tx=0).stop 2023-07-06T03:52:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.545+0000 7f0c87bff700 1 -- 172.21.15.97:0/1258900514 shutdown_connections 2023-07-06T03:52:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.545+0000 7f0c87bff700 1 --2- 172.21.15.97:0/1258900514 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c8015eb90 0x7f0c8015b050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.545+0000 7f0c87bff700 1 -- 172.21.15.97:0/1258900514 >> 172.21.15.97:0/1258900514 conn(0x7f0c8006a060 msgr2=0x7f0c8006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.545+0000 7f0c87bff700 1 -- 172.21.15.97:0/1258900514 shutdown_connections 2023-07-06T03:52:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.545+0000 7f0c87bff700 1 -- 172.21.15.97:0/1258900514 wait complete. 2023-07-06T03:52:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.547+0000 7f0c87bff700 1 Processor -- start 2023-07-06T03:52:23.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.547+0000 7f0c87bff700 1 -- start start 2023-07-06T03:52:23.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.548+0000 7f0c87bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c800cbaa0 0x7f0c8014b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.548+0000 7f0c87bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c80154890 0x7f0c80148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.548+0000 7f0c87bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c8015eb90 0x7f0c800acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.548+0000 7f0c87bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80152f50 con 0x7f0c8015eb90 2023-07-06T03:52:23.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.548+0000 7f0c87bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8014bef0 con 0x7f0c800cbaa0 2023-07-06T03:52:23.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.548+0000 7f0c87bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80152790 con 0x7f0c80154890 2023-07-06T03:52:23.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.548+0000 7f0c8619c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c8015eb90 0x7f0c800acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.548+0000 7f0c8619c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c8015eb90 0x7f0c800acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54708/0 (socket says 172.21.15.97:54708) 2023-07-06T03:52:23.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.548+0000 7f0c8619c700 1 -- 172.21.15.97:0/4110258851 learned_addr learned my addr 172.21.15.97:0/4110258851 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.548+0000 7f0c8519a700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c80154890 0x7f0c80148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.548+0000 7f0c8599b700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c800cbaa0 0x7f0c8014b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.549+0000 7f0c8619c700 1 -- 172.21.15.97:0/4110258851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c80154890 msgr2=0x7f0c80148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.549+0000 7f0c8619c700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c80154890 0x7f0c80148c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.549+0000 7f0c8619c700 1 -- 172.21.15.97:0/4110258851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c800cbaa0 msgr2=0x7f0c8014b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.549+0000 7f0c8619c700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c800cbaa0 0x7f0c8014b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.549+0000 7f0c8619c700 1 -- 172.21.15.97:0/4110258851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c78009a70 con 0x7f0c8015eb90 2023-07-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.549+0000 7f0c8619c700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c8015eb90 0x7f0c800acae0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009d90 tx=0x7f0c78004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.549+0000 7f0c767fc700 1 -- 172.21.15.97:0/4110258851 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c780043c0 con 0x7f0c8015eb90 2023-07-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.549+0000 7f0c767fc700 1 -- 172.21.15.97:0/4110258851 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c78004540 con 0x7f0c8015eb90 2023-07-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.549+0000 7f0c87bff700 1 -- 172.21.15.97:0/4110258851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c80152020 con 0x7f0c8015eb90 2023-07-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.549+0000 7f0c767fc700 1 -- 172.21.15.97:0/4110258851 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c780048f0 con 0x7f0c8015eb90 2023-07-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.549+0000 7f0c87bff700 1 -- 172.21.15.97:0/4110258851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c80152a30 con 0x7f0c8015eb90 2023-07-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.550+0000 7f0c87bff700 1 -- 172.21.15.97:0/4110258851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c80152c20 con 0x7f0c8015eb90 2023-07-06T03:52:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.551+0000 7f0c767fc700 1 -- 172.21.15.97:0/4110258851 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0c7801c460 con 0x7f0c8015eb90 2023-07-06T03:52:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.552+0000 7f0c767fc700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c6406ace0 0x7f0c6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.552+0000 7f0c767fc700 1 -- 172.21.15.97:0/4110258851 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0c7808a8a0 con 0x7f0c8015eb90 2023-07-06T03:52:23.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.553+0000 7f0c8599b700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c6406ace0 0x7f0c6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.553+0000 7f0c8599b700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c6406ace0 0x7f0c6406d190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c007fd0 tx=0x7f0c7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:23.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.556+0000 7f0c767fc700 1 -- 172.21.15.97:0/4110258851 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0c780596f0 con 0x7f0c8015eb90 2023-07-06T03:52:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.758+0000 7f0c87bff700 1 -- 172.21.15.97:0/4110258851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7f0c80165270 con 0x7f0c8015eb90 2023-07-06T03:52:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.762+0000 7f0c767fc700 1 -- 172.21.15.97:0/4110258851 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v28) v1 ==== 102+0+0 (secure 0 0 0) 0x7f0c78059280 con 0x7f0c8015eb90 2023-07-06T03:52:23.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.764+0000 7f0c87bff700 1 -- 172.21.15.97:0/4110258851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c6406ace0 msgr2=0x7f0c6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.764+0000 7f0c87bff700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c6406ace0 0x7f0c6406d190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c007fd0 tx=0x7f0c7c009040 comp rx=0 tx=0).stop 2023-07-06T03:52:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.764+0000 7f0c87bff700 1 -- 172.21.15.97:0/4110258851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c8015eb90 msgr2=0x7f0c800acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.764+0000 7f0c87bff700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c8015eb90 0x7f0c800acae0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009d90 tx=0x7f0c78004ad0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.765+0000 7f0c87bff700 1 -- 172.21.15.97:0/4110258851 shutdown_connections 2023-07-06T03:52:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.765+0000 7f0c87bff700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c6406ace0 0x7f0c6406d190 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.765+0000 7f0c87bff700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c8015eb90 0x7f0c800acae0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.765+0000 7f0c87bff700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c800cbaa0 0x7f0c8014b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.765+0000 7f0c87bff700 1 --2- 172.21.15.97:0/4110258851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c80154890 0x7f0c80148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.765+0000 7f0c87bff700 1 -- 172.21.15.97:0/4110258851 >> 172.21.15.97:0/4110258851 conn(0x7f0c8006a060 msgr2=0x7f0c801514f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:23.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.765+0000 7f0c87bff700 1 -- 172.21.15.97:0/4110258851 shutdown_connections 2023-07-06T03:52:23.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.765+0000 7f0c87bff700 1 -- 172.21.15.97:0/4110258851 wait complete. 2023-07-06T03:52:23.784 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-07-06T03:52:23.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.983+0000 7f447edff700 1 Processor -- start 2023-07-06T03:52:23.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.984+0000 7f447edff700 1 -- start start 2023-07-06T03:52:23.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447edff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f447815c610 0x7f44780b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447edff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447806b360 0x7f44780b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447edff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f44780d2210 0x7f44780d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447edff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44780d0780 con 0x7f447815c610 2023-07-06T03:52:23.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447edff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447815c9e0 con 0x7f44780d2210 2023-07-06T03:52:23.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447edff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780b5650 con 0x7f447806b360 2023-07-06T03:52:23.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447cb9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f447815c610 0x7f44780b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447cb9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f447815c610 0x7f44780b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54714/0 (socket says 172.21.15.97:54714) 2023-07-06T03:52:23.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447cb9b700 1 -- 172.21.15.97:0/3263534305 learned_addr learned my addr 172.21.15.97:0/3263534305 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:23.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f4477fff700 1 --2- 172.21.15.97:0/3263534305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447806b360 0x7f44780b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447d39c700 1 --2- 172.21.15.97:0/3263534305 >> v2:172.21.15.146:3300/0 conn(0x7f44780d2210 0x7f44780d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447cb9b700 1 -- 172.21.15.97:0/3263534305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447806b360 msgr2=0x7f44780b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:23.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447cb9b700 1 --2- 172.21.15.97:0/3263534305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447806b360 0x7f44780b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447cb9b700 1 -- 172.21.15.97:0/3263534305 >> v2:172.21.15.146:3300/0 conn(0x7f44780d2210 msgr2=0x7f44780d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:23.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447cb9b700 1 --2- 172.21.15.97:0/3263534305 >> v2:172.21.15.146:3300/0 conn(0x7f44780d2210 0x7f44780d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.985+0000 7f447cb9b700 1 -- 172.21.15.97:0/3263534305 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780d01c0 con 0x7f447815c610 2023-07-06T03:52:23.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.986+0000 7f447cb9b700 1 --2- 172.21.15.97:0/3263534305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f447815c610 0x7f44780b47b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f44780b1b70 tx=0x7f446c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbeb94971f4fb91 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:23.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.986+0000 7f44777fe700 1 -- 172.21.15.97:0/3263534305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00e070 con 0x7f447815c610 2023-07-06T03:52:23.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.986+0000 7f44777fe700 1 -- 172.21.15.97:0/3263534305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c00b980 con 0x7f447815c610 2023-07-06T03:52:23.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.986+0000 7f44777fe700 1 -- 172.21.15.97:0/3263534305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00bd30 con 0x7f447815c610 2023-07-06T03:52:23.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.986+0000 7f447edff700 1 -- 172.21.15.97:0/3263534305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f447815c610 msgr2=0x7f44780b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.986+0000 7f447edff700 1 --2- 172.21.15.97:0/3263534305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f447815c610 0x7f44780b47b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f44780b1b70 tx=0x7f446c00b330 comp rx=0 tx=0).stop 2023-07-06T03:52:23.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.987+0000 7f447edff700 1 -- 172.21.15.97:0/3263534305 shutdown_connections 2023-07-06T03:52:23.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.987+0000 7f447edff700 1 --2- 172.21.15.97:0/3263534305 >> v2:172.21.15.146:3300/0 conn(0x7f44780d2210 0x7f44780d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.987+0000 7f447edff700 1 --2- 172.21.15.97:0/3263534305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f447815c610 0x7f44780b47b0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.987+0000 7f447edff700 1 --2- 172.21.15.97:0/3263534305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447806b360 0x7f44780b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.987+0000 7f447edff700 1 -- 172.21.15.97:0/3263534305 >> 172.21.15.97:0/3263534305 conn(0x7f447806a6c0 msgr2=0x7f447806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:23.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.987+0000 7f447edff700 1 -- 172.21.15.97:0/3263534305 shutdown_connections 2023-07-06T03:52:23.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.987+0000 7f447edff700 1 -- 172.21.15.97:0/3263534305 wait complete. 2023-07-06T03:52:23.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.988+0000 7f447edff700 1 Processor -- start 2023-07-06T03:52:23.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.989+0000 7f447edff700 1 -- start start 2023-07-06T03:52:23.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.989+0000 7f447edff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447806b360 0x7f4478156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.989+0000 7f447edff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44780d2210 0x7f447815d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.989+0000 7f447edff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447815c610 0x7f447815e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.989+0000 7f447edff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4478157470 con 0x7f44780d2210 2023-07-06T03:52:23.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.989+0000 7f447edff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447815cb20 con 0x7f447806b360 2023-07-06T03:52:23.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.989+0000 7f447edff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447815d0e0 con 0x7f447815c610 2023-07-06T03:52:23.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.990+0000 7f4477fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44780d2210 0x7f447815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.990+0000 7f4477fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44780d2210 0x7f447815d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54722/0 (socket says 172.21.15.97:54722) 2023-07-06T03:52:23.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.990+0000 7f4477fff700 1 -- 172.21.15.97:0/1896625602 learned_addr learned my addr 172.21.15.97:0/1896625602 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:23.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.990+0000 7f447d39c700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447815c610 0x7f447815e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.990+0000 7f447cb9b700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447806b360 0x7f4478156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.990+0000 7f4477fff700 1 -- 172.21.15.97:0/1896625602 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447815c610 msgr2=0x7f447815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.990+0000 7f4477fff700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447815c610 0x7f447815e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.990+0000 7f4477fff700 1 -- 172.21.15.97:0/1896625602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447806b360 msgr2=0x7f4478156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:23.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.990+0000 7f4477fff700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447806b360 0x7f4478156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:23.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.990+0000 7f4477fff700 1 -- 172.21.15.97:0/1896625602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c009b30 con 0x7f44780d2210 2023-07-06T03:52:23.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.990+0000 7f4477fff700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44780d2210 0x7f447815d570 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f446400e810 tx=0x7f446400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:23.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.991+0000 7f44757fa700 1 -- 172.21.15.97:0/1896625602 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464013610 con 0x7f44780d2210 2023-07-06T03:52:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.991+0000 7f44757fa700 1 -- 172.21.15.97:0/1896625602 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4464013770 con 0x7f44780d2210 2023-07-06T03:52:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.991+0000 7f447edff700 1 -- 172.21.15.97:0/1896625602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4478157890 con 0x7f44780d2210 2023-07-06T03:52:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.991+0000 7f44757fa700 1 -- 172.21.15.97:0/1896625602 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464013b00 con 0x7f44780d2210 2023-07-06T03:52:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.991+0000 7f447edff700 1 -- 172.21.15.97:0/1896625602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447815f410 con 0x7f44780d2210 2023-07-06T03:52:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.992+0000 7f447edff700 1 -- 172.21.15.97:0/1896625602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447804e340 con 0x7f44780d2210 2023-07-06T03:52:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.993+0000 7f44757fa700 1 -- 172.21.15.97:0/1896625602 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4464013c60 con 0x7f44780d2210 2023-07-06T03:52:23.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.994+0000 7f44757fa700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f445c06ace0 0x7f445c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:23.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.994+0000 7f44757fa700 1 -- 172.21.15.97:0/1896625602 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4464088dc0 con 0x7f44780d2210 2023-07-06T03:52:23.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.994+0000 7f447cb9b700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f445c06ace0 0x7f445c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:23.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.994+0000 7f447cb9b700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f445c06ace0 0x7f445c06d190 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f446c00b370 tx=0x7f446c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:23.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:23.998+0000 7f44757fa700 1 -- 172.21.15.97:0/1896625602 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4464057c10 con 0x7f44780d2210 2023-07-06T03:52:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.195+0000 7f447edff700 1 -- 172.21.15.97:0/1896625602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7f447815d220 con 0x7f44780d2210 2023-07-06T03:52:24.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.199+0000 7f44757fa700 1 -- 172.21.15.97:0/1896625602 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v29) v1 ==== 107+0+0 (secure 0 0 0) 0x7f44640210b0 con 0x7f44780d2210 2023-07-06T03:52:24.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.201+0000 7f447edff700 1 -- 172.21.15.97:0/1896625602 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f445c06ace0 msgr2=0x7f445c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.201+0000 7f447edff700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f445c06ace0 0x7f445c06d190 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f446c00b370 tx=0x7f446c009000 comp rx=0 tx=0).stop 2023-07-06T03:52:24.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.201+0000 7f447edff700 1 -- 172.21.15.97:0/1896625602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44780d2210 msgr2=0x7f447815d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.201+0000 7f447edff700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44780d2210 0x7f447815d570 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f446400e810 tx=0x7f446400ebd0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.202+0000 7f447edff700 1 -- 172.21.15.97:0/1896625602 shutdown_connections 2023-07-06T03:52:24.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.202+0000 7f447edff700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f445c06ace0 0x7f445c06d190 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.202+0000 7f447edff700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f447815c610 0x7f447815e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.202+0000 7f447edff700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447806b360 0x7f4478156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.202+0000 7f447edff700 1 --2- 172.21.15.97:0/1896625602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44780d2210 0x7f447815d570 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.202+0000 7f447edff700 1 -- 172.21.15.97:0/1896625602 >> 172.21.15.97:0/1896625602 conn(0x7f447806a6c0 msgr2=0x7f44780b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:24.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.202+0000 7f447edff700 1 -- 172.21.15.97:0/1896625602 shutdown_connections 2023-07-06T03:52:24.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.202+0000 7f447edff700 1 -- 172.21.15.97:0/1896625602 wait complete. 2023-07-06T03:52:24.221 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T03:52:24.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.417+0000 7fbb62fff700 1 Processor -- start 2023-07-06T03:52:24.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.417+0000 7fbb62fff700 1 -- start start 2023-07-06T03:52:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.419+0000 7fbb62fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbb5c0cbaa0 0x7fbb5c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.419+0000 7fbb62fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c15eb90 0x7fbb5c15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.419+0000 7fbb62fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5c15b6e0 con 0x7fbb5c154890 2023-07-06T03:52:24.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.419+0000 7fbb62fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5c15e0b0 con 0x7fbb5c0cbaa0 2023-07-06T03:52:24.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.419+0000 7fbb62fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c15e390 con 0x7fbb5c15eb90 2023-07-06T03:52:24.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.419+0000 7fbb5bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbb5c0cbaa0 0x7fbb5c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.419+0000 7fbb6159c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c15eb90 0x7fbb5c15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.419+0000 7fbb5bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbb5c0cbaa0 0x7fbb5c15a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:55744/0 (socket says 172.21.15.97:55744) 2023-07-06T03:52:24.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.419+0000 7fbb5bfff700 1 -- 172.21.15.97:0/3748249713 learned_addr learned my addr 172.21.15.97:0/3748249713 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:24.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.420+0000 7fbb6159c700 1 -- 172.21.15.97:0/3748249713 >> v2:172.21.15.146:3300/0 conn(0x7fbb5c0cbaa0 msgr2=0x7fbb5c15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:24.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.420+0000 7fbb6159c700 1 --2- 172.21.15.97:0/3748249713 >> v2:172.21.15.146:3300/0 conn(0x7fbb5c0cbaa0 0x7fbb5c15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.420+0000 7fbb6159c700 1 -- 172.21.15.97:0/3748249713 >> v1:172.21.15.146:6789/0 conn(0x7fbb5c154890 legacy=0x7fbb5c0cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.420+0000 7fbb6159c700 1 -- 172.21.15.97:0/3748249713 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c15e670 con 0x7fbb5c15eb90 2023-07-06T03:52:24.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.420+0000 7fbb5bfff700 1 --2- 172.21.15.97:0/3748249713 >> v2:172.21.15.146:3300/0 conn(0x7fbb5c0cbaa0 0x7fbb5c15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:24.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.420+0000 7fbb6159c700 1 --2- 172.21.15.97:0/3748249713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c15eb90 0x7fbb5c15ae80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009d90 tx=0x7fbb5400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31aaf170d4f753e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:24.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.420+0000 7fbb5b7fe700 1 -- 172.21.15.97:0/3748249713 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5400e070 con 0x7fbb5c15eb90 2023-07-06T03:52:24.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.420+0000 7fbb5b7fe700 1 -- 172.21.15.97:0/3748249713 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5400bb20 con 0x7fbb5c15eb90 2023-07-06T03:52:24.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.420+0000 7fbb5b7fe700 1 -- 172.21.15.97:0/3748249713 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54012430 con 0x7fbb5c15eb90 2023-07-06T03:52:24.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.421+0000 7fbb62fff700 1 -- 172.21.15.97:0/3748249713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c15eb90 msgr2=0x7fbb5c15ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.421+0000 7fbb62fff700 1 --2- 172.21.15.97:0/3748249713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c15eb90 0x7fbb5c15ae80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009d90 tx=0x7fbb5400b330 comp rx=0 tx=0).stop 2023-07-06T03:52:24.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.421+0000 7fbb62fff700 1 -- 172.21.15.97:0/3748249713 shutdown_connections 2023-07-06T03:52:24.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.421+0000 7fbb62fff700 1 --2- 172.21.15.97:0/3748249713 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c15eb90 0x7fbb5c15ae80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.421+0000 7fbb62fff700 1 --2- 172.21.15.97:0/3748249713 >> v2:172.21.15.146:3300/0 conn(0x7fbb5c0cbaa0 0x7fbb5c15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.421+0000 7fbb62fff700 1 -- 172.21.15.97:0/3748249713 >> 172.21.15.97:0/3748249713 conn(0x7fbb5c06a060 msgr2=0x7fbb5c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:24.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.421+0000 7fbb62fff700 1 -- 172.21.15.97:0/3748249713 shutdown_connections 2023-07-06T03:52:24.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.421+0000 7fbb62fff700 1 -- 172.21.15.97:0/3748249713 wait complete. 2023-07-06T03:52:24.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.423+0000 7fbb62fff700 1 Processor -- start 2023-07-06T03:52:24.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.423+0000 7fbb62fff700 1 -- start start 2023-07-06T03:52:24.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb62fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0cbaa0 0x7fbb5c0703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb62fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c154890 0x7fbb5c0b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb62fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb5c15eb90 0x7fbb5c14f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb62fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c0c69b0 con 0x7fbb5c154890 2023-07-06T03:52:24.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb62fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c070930 con 0x7fbb5c15eb90 2023-07-06T03:52:24.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb62fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c0b2410 con 0x7fbb5c0cbaa0 2023-07-06T03:52:24.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb5bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c154890 0x7fbb5c0b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb5bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c154890 0x7fbb5c0b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54726/0 (socket says 172.21.15.97:54726) 2023-07-06T03:52:24.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb5bfff700 1 -- 172.21.15.97:0/3751970215 learned_addr learned my addr 172.21.15.97:0/3751970215 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:24.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb60d9b700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0cbaa0 0x7fbb5c0703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb6159c700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb5c15eb90 0x7fbb5c14f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb5bfff700 1 -- 172.21.15.97:0/3751970215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0cbaa0 msgr2=0x7fbb5c0703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb5bfff700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0cbaa0 0x7fbb5c0703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.424+0000 7fbb5bfff700 1 -- 172.21.15.97:0/3751970215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb5c15eb90 msgr2=0x7fbb5c14f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.425+0000 7fbb5bfff700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb5c15eb90 0x7fbb5c14f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.425+0000 7fbb5bfff700 1 -- 172.21.15.97:0/3751970215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb48009550 con 0x7fbb5c154890 2023-07-06T03:52:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.425+0000 7fbb5bfff700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c154890 0x7fbb5c0b1ed0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800ebb0 tx=0x7fbb4800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.425+0000 7fbb597fa700 1 -- 172.21.15.97:0/3751970215 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4800c850 con 0x7fbb5c154890 2023-07-06T03:52:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.425+0000 7fbb597fa700 1 -- 172.21.15.97:0/3751970215 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4800c9b0 con 0x7fbb5c154890 2023-07-06T03:52:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.425+0000 7fbb62fff700 1 -- 172.21.15.97:0/3751970215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb54009a70 con 0x7fbb5c154890 2023-07-06T03:52:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.425+0000 7fbb597fa700 1 -- 172.21.15.97:0/3751970215 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4800cd40 con 0x7fbb5c154890 2023-07-06T03:52:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.425+0000 7fbb62fff700 1 -- 172.21.15.97:0/3751970215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5c070e80 con 0x7fbb5c154890 2023-07-06T03:52:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.426+0000 7fbb62fff700 1 -- 172.21.15.97:0/3751970215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5c04e340 con 0x7fbb5c154890 2023-07-06T03:52:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.427+0000 7fbb597fa700 1 -- 172.21.15.97:0/3751970215 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb48013470 con 0x7fbb5c154890 2023-07-06T03:52:24.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.428+0000 7fbb597fa700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb4006ace0 0x7fbb4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.428+0000 7fbb597fa700 1 -- 172.21.15.97:0/3751970215 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbb48089290 con 0x7fbb5c154890 2023-07-06T03:52:24.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.429+0000 7fbb60d9b700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb4006ace0 0x7fbb4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.429+0000 7fbb60d9b700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb4006ace0 0x7fbb4006d190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb50007fd0 tx=0x7fbb50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:24.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.432+0000 7fbb597fa700 1 -- 172.21.15.97:0/3751970215 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbb48054650 con 0x7fbb5c154890 2023-07-06T03:52:24.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.627+0000 7fbb62fff700 1 -- 172.21.15.97:0/3751970215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbb5c06bef0 con 0x7fbb5c154890 2023-07-06T03:52:24.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.632+0000 7fbb597fa700 1 -- 172.21.15.97:0/3751970215 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbb48057c70 con 0x7fbb5c154890 2023-07-06T03:52:24.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 -- 172.21.15.97:0/3751970215 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb4006ace0 msgr2=0x7fbb4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb4006ace0 0x7fbb4006d190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb50007fd0 tx=0x7fbb50009040 comp rx=0 tx=0).stop 2023-07-06T03:52:24.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 -- 172.21.15.97:0/3751970215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c154890 msgr2=0x7fbb5c0b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c154890 0x7fbb5c0b1ed0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800ebb0 tx=0x7fbb4800eec0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 -- 172.21.15.97:0/3751970215 shutdown_connections 2023-07-06T03:52:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbb4006ace0 0x7fbb4006d190 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb5c15eb90 0x7fbb5c14f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbb5c0cbaa0 0x7fbb5c0703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 --2- 172.21.15.97:0/3751970215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c154890 0x7fbb5c0b1ed0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 -- 172.21.15.97:0/3751970215 >> 172.21.15.97:0/3751970215 conn(0x7fbb5c06a060 msgr2=0x7fbb5c0c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 -- 172.21.15.97:0/3751970215 shutdown_connections 2023-07-06T03:52:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.634+0000 7fbb62fff700 1 -- 172.21.15.97:0/3751970215 wait complete. 2023-07-06T03:52:24.654 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-07-06T03:52:24.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.849+0000 7fc00179d700 1 Processor -- start 2023-07-06T03:52:24.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.849+0000 7fc00179d700 1 -- start start 2023-07-06T03:52:24.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.850+0000 7fc00179d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b8470 0x7fbffc0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.850+0000 7fc00179d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbffc0cd660 0x7fbffc0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.850+0000 7fc00179d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc15cb70 con 0x7fbffc0b1b00 2023-07-06T03:52:24.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.850+0000 7fc00179d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc0cfe10 con 0x7fbffc0cd660 2023-07-06T03:52:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.850+0000 7fc00179d700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0cd1d0 con 0x7fbffc0b8470 2023-07-06T03:52:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.850+0000 7fbffb7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbffc0cd660 0x7fbffc0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.850+0000 7fbffbfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b8470 0x7fbffc0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.850+0000 7fc000f9c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fbffc0b1b00 0x7fbffc0b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:50190/0 (socket says 172.21.15.97:50190) 2023-07-06T03:52:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.851+0000 7fc000f9c700 1 -- 172.21.15.97:0/466247107 learned_addr learned my addr 172.21.15.97:0/466247107 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.851+0000 7fbffaffd700 1 -- 172.21.15.97:0/466247107 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954929560 0 0) 0x7fbff4001490 con 0x7fbffc0b1b00 2023-07-06T03:52:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.851+0000 7fbffaffd700 1 -- 172.21.15.97:0/466247107 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfe40035b0 con 0x7fbffc0b1b00 2023-07-06T03:52:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.851+0000 7fbffbfff700 1 -- 172.21.15.97:0/466247107 >> v2:172.21.15.146:3300/0 conn(0x7fbffc0cd660 msgr2=0x7fbffc0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.851+0000 7fbffbfff700 1 --2- 172.21.15.97:0/466247107 >> v2:172.21.15.146:3300/0 conn(0x7fbffc0cd660 0x7fbffc0d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.851+0000 7fbffbfff700 1 -- 172.21.15.97:0/466247107 >> v1:172.21.15.146:6789/0 conn(0x7fbffc0b1b00 legacy=0x7fbffc0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.851+0000 7fbffbfff700 1 -- 172.21.15.97:0/466247107 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc0d2690 con 0x7fbffc0b8470 2023-07-06T03:52:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.851+0000 7fbffb7fe700 1 --2- 172.21.15.97:0/466247107 >> v2:172.21.15.146:3300/0 conn(0x7fbffc0cd660 0x7fbffc0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.851+0000 7fbffbfff700 1 --2- 172.21.15.97:0/466247107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b8470 0x7fbffc0b2050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009840 tx=0x7fbfec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=579272ef41272828 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.852+0000 7fbffaffd700 1 -- 172.21.15.97:0/466247107 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011830 con 0x7fbffc0b8470 2023-07-06T03:52:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.852+0000 7fbffaffd700 1 -- 172.21.15.97:0/466247107 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec011990 con 0x7fbffc0b8470 2023-07-06T03:52:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.852+0000 7fbffaffd700 1 -- 172.21.15.97:0/466247107 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011d00 con 0x7fbffc0b8470 2023-07-06T03:52:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.852+0000 7fc00179d700 1 -- 172.21.15.97:0/466247107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b8470 msgr2=0x7fbffc0b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.852+0000 7fc00179d700 1 --2- 172.21.15.97:0/466247107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b8470 0x7fbffc0b2050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009840 tx=0x7fbfec009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.852+0000 7fc00179d700 1 -- 172.21.15.97:0/466247107 shutdown_connections 2023-07-06T03:52:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.852+0000 7fc00179d700 1 --2- 172.21.15.97:0/466247107 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b8470 0x7fbffc0b2050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.852+0000 7fc00179d700 1 --2- 172.21.15.97:0/466247107 >> v2:172.21.15.146:3300/0 conn(0x7fbffc0cd660 0x7fbffc0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.852+0000 7fc00179d700 1 -- 172.21.15.97:0/466247107 >> 172.21.15.97:0/466247107 conn(0x7fbffc06b160 msgr2=0x7fbffc071790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.852+0000 7fc00179d700 1 -- 172.21.15.97:0/466247107 shutdown_connections 2023-07-06T03:52:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.852+0000 7fc00179d700 1 -- 172.21.15.97:0/466247107 wait complete. 2023-07-06T03:52:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.855+0000 7fc00179d700 1 Processor -- start 2023-07-06T03:52:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.855+0000 7fc00179d700 1 -- start start 2023-07-06T03:52:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc00179d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b1b00 0x7fbffc0a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc00179d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbffc0b8470 0x7fbffc0a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc00179d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbffc0cd660 0x7fbffc15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc00179d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc14e510 con 0x7fbffc0cd660 2023-07-06T03:52:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc00179d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0aa410 con 0x7fbffc0b8470 2023-07-06T03:52:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc00179d700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc14e760 con 0x7fbffc0b1b00 2023-07-06T03:52:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc000f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbffc0cd660 0x7fbffc15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc000f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbffc0cd660 0x7fbffc15cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54728/0 (socket says 172.21.15.97:54728) 2023-07-06T03:52:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc000f9c700 1 -- 172.21.15.97:0/3761600677 learned_addr learned my addr 172.21.15.97:0/3761600677 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fbffb7fe700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbffc0b8470 0x7fbffc0a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fbffbfff700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b1b00 0x7fbffc0a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc000f9c700 1 -- 172.21.15.97:0/3761600677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b1b00 msgr2=0x7fbffc0a9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc000f9c700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b1b00 0x7fbffc0a9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc000f9c700 1 -- 172.21.15.97:0/3761600677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbffc0b8470 msgr2=0x7fbffc0a94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.856+0000 7fc000f9c700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbffc0b8470 0x7fbffc0a94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.857+0000 7fc000f9c700 1 -- 172.21.15.97:0/3761600677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00094a0 con 0x7fbffc0cd660 2023-07-06T03:52:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.857+0000 7fbffbfff700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b1b00 0x7fbffc0a9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.857+0000 7fc000f9c700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbffc0cd660 0x7fbffc15cd10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fbff400cb40 tx=0x7fbff400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.857+0000 7fbff8ff9700 1 -- 172.21.15.97:0/3761600677 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff400ab50 con 0x7fbffc0cd660 2023-07-06T03:52:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.857+0000 7fbff8ff9700 1 -- 172.21.15.97:0/3761600677 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff400acb0 con 0x7fbffc0cd660 2023-07-06T03:52:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.857+0000 7fc00179d700 1 -- 172.21.15.97:0/3761600677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec0094a0 con 0x7fbffc0cd660 2023-07-06T03:52:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.857+0000 7fbff8ff9700 1 -- 172.21.15.97:0/3761600677 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4012620 con 0x7fbffc0cd660 2023-07-06T03:52:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.857+0000 7fc00179d700 1 -- 172.21.15.97:0/3761600677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc14ed20 con 0x7fbffc0cd660 2023-07-06T03:52:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.858+0000 7fc00179d700 1 -- 172.21.15.97:0/3761600677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc14ef40 con 0x7fbffc0cd660 2023-07-06T03:52:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.859+0000 7fbff8ff9700 1 -- 172.21.15.97:0/3761600677 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbff401b3b0 con 0x7fbffc0cd660 2023-07-06T03:52:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.860+0000 7fbff8ff9700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfe806ace0 0x7fbfe806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.860+0000 7fbff8ff9700 1 -- 172.21.15.97:0/3761600677 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbff4088dc0 con 0x7fbffc0cd660 2023-07-06T03:52:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.861+0000 7fbffbfff700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfe806ace0 0x7fbfe806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.861+0000 7fbffbfff700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfe806ace0 0x7fbfe806d190 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0090b0 tx=0x7fbfec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:24.865+0000 7fbff8ff9700 1 -- 172.21.15.97:0/3761600677 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbff4054180 con 0x7fbffc0cd660 2023-07-06T03:52:25.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.062+0000 7fc00179d700 1 -- 172.21.15.97:0/3761600677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7fbffc15cb70 con 0x7fbffc0cd660 2023-07-06T03:52:25.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.066+0000 7fbff8ff9700 1 -- 172.21.15.97:0/3761600677 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v31) v1 ==== 105+0+0 (secure 0 0 0) 0x7fbffc15cb70 con 0x7fbffc0cd660 2023-07-06T03:52:25.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.068+0000 7fc00179d700 1 -- 172.21.15.97:0/3761600677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfe806ace0 msgr2=0x7fbfe806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.068+0000 7fc00179d700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfe806ace0 0x7fbfe806d190 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0090b0 tx=0x7fbfec008030 comp rx=0 tx=0).stop 2023-07-06T03:52:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.068+0000 7fc00179d700 1 -- 172.21.15.97:0/3761600677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbffc0cd660 msgr2=0x7fbffc15cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.068+0000 7fc00179d700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbffc0cd660 0x7fbffc15cd10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fbff400cb40 tx=0x7fbff400cf00 comp rx=0 tx=0).stop 2023-07-06T03:52:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.069+0000 7fc00179d700 1 -- 172.21.15.97:0/3761600677 shutdown_connections 2023-07-06T03:52:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.069+0000 7fc00179d700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfe806ace0 0x7fbfe806d190 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.069+0000 7fc00179d700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbffc0cd660 0x7fbffc15cd10 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.069+0000 7fc00179d700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbffc0b1b00 0x7fbffc0a9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.069+0000 7fc00179d700 1 --2- 172.21.15.97:0/3761600677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbffc0b8470 0x7fbffc0a94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.069+0000 7fc00179d700 1 -- 172.21.15.97:0/3761600677 >> 172.21.15.97:0/3761600677 conn(0x7fbffc06b160 msgr2=0x7fbffc0b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:25.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.069+0000 7fc00179d700 1 -- 172.21.15.97:0/3761600677 shutdown_connections 2023-07-06T03:52:25.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.069+0000 7fc00179d700 1 -- 172.21.15.97:0/3761600677 wait complete. 2023-07-06T03:52:25.088 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2023-07-06T03:52:25.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.282+0000 7fa5b63ff700 1 Processor -- start 2023-07-06T03:52:25.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.283+0000 7fa5b63ff700 1 -- start start 2023-07-06T03:52:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.284+0000 7fa5b63ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa5b0160190 0x7fa5b00ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.284+0000 7fa5b63ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5b006a060 0x7fa5b006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.284+0000 7fa5b63ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b006a9f0 con 0x7fa5b00714e0 2023-07-06T03:52:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.284+0000 7fa5b63ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b00cf710 con 0x7fa5b0160190 2023-07-06T03:52:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.284+0000 7fa5b63ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00ceb40 con 0x7fa5b006a060 2023-07-06T03:52:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.285+0000 7fa5affff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa5b0160190 0x7fa5b00ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.285+0000 7fa5affff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa5b0160190 0x7fa5b00ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54734/0 (socket says 172.21.15.97:54734) 2023-07-06T03:52:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.285+0000 7fa5affff700 1 -- 172.21.15.97:0/1797460 learned_addr learned my addr 172.21.15.97:0/1797460 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:25.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.285+0000 7fa5af7fe700 1 --2- 172.21.15.97:0/1797460 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5b006a060 0x7fa5b006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:25.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.285+0000 7fa5affff700 1 -- 172.21.15.97:0/1797460 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5b006a060 msgr2=0x7fa5b006a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.285+0000 7fa5affff700 1 --2- 172.21.15.97:0/1797460 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5b006a060 0x7fa5b006a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.285+0000 7fa5affff700 1 -- 172.21.15.97:0/1797460 >> v1:172.21.15.146:6789/0 conn(0x7fa5b00714e0 legacy=0x7fa5b0071960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.285+0000 7fa5affff700 1 -- 172.21.15.97:0/1797460 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b00cede0 con 0x7fa5b0160190 2023-07-06T03:52:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.285+0000 7fa5affff700 1 --2- 172.21.15.97:0/1797460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5b0160190 0x7fa5b00ac880 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00098f0 tx=0x7fa5a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be15a7cb072231d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.286+0000 7fa5aeffd700 1 -- 172.21.15.97:0/1797460 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0011830 con 0x7fa5b0160190 2023-07-06T03:52:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.286+0000 7fa5aeffd700 1 -- 172.21.15.97:0/1797460 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a0011990 con 0x7fa5b0160190 2023-07-06T03:52:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.286+0000 7fa5aeffd700 1 -- 172.21.15.97:0/1797460 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0011d20 con 0x7fa5b0160190 2023-07-06T03:52:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.286+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1797460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5b0160190 msgr2=0x7fa5b00ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.286+0000 7fa5b63ff700 1 --2- 172.21.15.97:0/1797460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5b0160190 0x7fa5b00ac880 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00098f0 tx=0x7fa5a0009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.286+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1797460 shutdown_connections 2023-07-06T03:52:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.286+0000 7fa5b63ff700 1 --2- 172.21.15.97:0/1797460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5b0160190 0x7fa5b00ac880 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.286+0000 7fa5b63ff700 1 --2- 172.21.15.97:0/1797460 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5b006a060 0x7fa5b006a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.286+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1797460 >> 172.21.15.97:0/1797460 conn(0x7fa5b00b22a0 msgr2=0x7fa5b00cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.286+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1797460 shutdown_connections 2023-07-06T03:52:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.286+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1797460 wait complete. 2023-07-06T03:52:25.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.288+0000 7fa5b63ff700 1 Processor -- start 2023-07-06T03:52:25.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.288+0000 7fa5b63ff700 1 -- start start 2023-07-06T03:52:25.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.289+0000 7fa5b63ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5b006a060 0x7fa5b00c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:25.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.289+0000 7fa5b63ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5b00714e0 0x7fa5b01f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:25.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.289+0000 7fa5b63ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5b0160190 0x7fa5b01f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:25.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.289+0000 7fa5b63ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00b4c70 con 0x7fa5b00714e0 2023-07-06T03:52:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.289+0000 7fa5b63ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00b4f50 con 0x7fa5b0160190 2023-07-06T03:52:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.289+0000 7fa5b63ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00b5230 con 0x7fa5b006a060 2023-07-06T03:52:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.289+0000 7fa5af7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5b00714e0 0x7fa5b01f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.289+0000 7fa5af7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5b00714e0 0x7fa5b01f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54744/0 (socket says 172.21.15.97:54744) 2023-07-06T03:52:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.289+0000 7fa5af7fe700 1 -- 172.21.15.97:0/1163903468 learned_addr learned my addr 172.21.15.97:0/1163903468 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.289+0000 7fa5affff700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5b006a060 0x7fa5b00c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5b499c700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5b0160190 0x7fa5b01f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5af7fe700 1 -- 172.21.15.97:0/1163903468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5b006a060 msgr2=0x7fa5b00c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5af7fe700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5b006a060 0x7fa5b00c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5af7fe700 1 -- 172.21.15.97:0/1163903468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5b0160190 msgr2=0x7fa5b01f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5af7fe700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5b0160190 0x7fa5b01f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5af7fe700 1 -- 172.21.15.97:0/1163903468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a0009550 con 0x7fa5b00714e0 2023-07-06T03:52:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5b499c700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5b0160190 0x7fa5b01f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5af7fe700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5b00714e0 0x7fa5b01f1ed0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9b0 tx=0x7fa5a400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5acff9700 1 -- 172.21.15.97:0/1163903468 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4009710 con 0x7fa5b00714e0 2023-07-06T03:52:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5acff9700 1 -- 172.21.15.97:0/1163903468 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a400c850 con 0x7fa5b00714e0 2023-07-06T03:52:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1163903468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b00b5510 con 0x7fa5b00714e0 2023-07-06T03:52:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5acff9700 1 -- 172.21.15.97:0/1163903468 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a400cbc0 con 0x7fa5b00714e0 2023-07-06T03:52:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.290+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1163903468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b00c97d0 con 0x7fa5b00714e0 2023-07-06T03:52:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.291+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1163903468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b004e340 con 0x7fa5b00714e0 2023-07-06T03:52:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.292+0000 7fa5acff9700 1 -- 172.21.15.97:0/1163903468 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5a400cd20 con 0x7fa5b00714e0 2023-07-06T03:52:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.293+0000 7fa5acff9700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa59c06ace0 0x7fa59c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.293+0000 7fa5acff9700 1 -- 172.21.15.97:0/1163903468 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa5a4089310 con 0x7fa5b00714e0 2023-07-06T03:52:25.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.293+0000 7fa5affff700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa59c06ace0 0x7fa59c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:25.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.294+0000 7fa5affff700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa59c06ace0 0x7fa59c06d190 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0014040 tx=0x7fa5a0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:25.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.297+0000 7fa5acff9700 1 -- 172.21.15.97:0/1163903468 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa5a40580d0 con 0x7fa5b00714e0 2023-07-06T03:52:25.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.572+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1163903468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) v1 -- 0x7fa5b015a710 con 0x7fa5b00714e0 2023-07-06T03:52:25.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.906+0000 7fa5acff9700 1 -- 172.21.15.97:0/1163903468 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v366) v1 ==== 160+0+0 (secure 0 0 0) 0x7fa5a4057c60 con 0x7fa5b00714e0 2023-07-06T03:52:25.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1163903468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa59c06ace0 msgr2=0x7fa59c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:25.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa59c06ace0 0x7fa59c06d190 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0014040 tx=0x7fa5a0008030 comp rx=0 tx=0).stop 2023-07-06T03:52:25.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1163903468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5b00714e0 msgr2=0x7fa5b01f1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5b00714e0 0x7fa5b01f1ed0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9b0 tx=0x7fa5a400ed70 comp rx=0 tx=0).stop 2023-07-06T03:52:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1163903468 shutdown_connections 2023-07-06T03:52:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa59c06ace0 0x7fa59c06d190 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5b0160190 0x7fa5b01f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa5b006a060 0x7fa5b00c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 --2- 172.21.15.97:0/1163903468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5b00714e0 0x7fa5b01f1ed0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1163903468 >> 172.21.15.97:0/1163903468 conn(0x7fa5b00b22a0 msgr2=0x7fa5b00ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1163903468 shutdown_connections 2023-07-06T03:52:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:25.909+0000 7fa5b63ff700 1 -- 172.21.15.97:0/1163903468 wait complete. 2023-07-06T03:52:25.928 INFO:tasks.cephfs_test_runner:test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-06T03:52:25.929 INFO:tasks.cephfs_test_runner:Starting test: test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:52:25.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2023-07-06T03:52:26.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.122+0000 7f4df9bff700 1 Processor -- start 2023-07-06T03:52:26.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.123+0000 7f4df9bff700 1 -- start start 2023-07-06T03:52:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.124+0000 7f4df9bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4df40d0e20 0x7f4df40b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.124+0000 7f4df9bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df40b4740 con 0x7f4df415ff00 2023-07-06T03:52:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.124+0000 7f4df9bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df41602d0 con 0x7f4df40d09b0 2023-07-06T03:52:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.124+0000 7f4df9bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df40b60b0 con 0x7f4df40d0e20 2023-07-06T03:52:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.124+0000 7f4df37fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4df415ff00 0x7f4df40c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53536/0 (socket says 172.21.15.97:53536) 2023-07-06T03:52:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.124+0000 7f4df37fe700 1 -- 172.21.15.97:0/699569101 learned_addr learned my addr 172.21.15.97:0/699569101 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.124+0000 7f4df3fff700 1 --2- 172.21.15.97:0/699569101 >> v2:172.21.15.97:3300/0 conn(0x7f4df40d0e20 0x7f4df40b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.125+0000 7f4df3fff700 1 -- 172.21.15.97:0/699569101 >> v1:172.21.15.146:6789/0 conn(0x7f4df40d09b0 legacy=0x7f4df40b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.125+0000 7f4df3fff700 1 -- 172.21.15.97:0/699569101 >> v1:172.21.15.97:6789/0 conn(0x7f4df415ff00 legacy=0x7f4df40c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.125+0000 7f4df3fff700 1 -- 172.21.15.97:0/699569101 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40c9800 con 0x7f4df40d0e20 2023-07-06T03:52:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.125+0000 7f4df3fff700 1 --2- 172.21.15.97:0/699569101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df40d0e20 0x7f4df40b6c50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009d90 tx=0x7f4dec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8ad10ffd1f6572 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.125+0000 7f4df2ffd700 1 -- 172.21.15.97:0/699569101 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00e070 con 0x7f4df40d0e20 2023-07-06T03:52:26.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.125+0000 7f4df2ffd700 1 -- 172.21.15.97:0/699569101 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec00bb90 con 0x7f4df40d0e20 2023-07-06T03:52:26.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.125+0000 7f4df2ffd700 1 -- 172.21.15.97:0/699569101 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec003a20 con 0x7f4df40d0e20 2023-07-06T03:52:26.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.125+0000 7f4df9bff700 1 -- 172.21.15.97:0/699569101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df40d0e20 msgr2=0x7f4df40b6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:26.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.125+0000 7f4df9bff700 1 --2- 172.21.15.97:0/699569101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df40d0e20 0x7f4df40b6c50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009d90 tx=0x7f4dec00b330 comp rx=0 tx=0).stop 2023-07-06T03:52:26.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.126+0000 7f4df9bff700 1 -- 172.21.15.97:0/699569101 shutdown_connections 2023-07-06T03:52:26.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.126+0000 7f4df9bff700 1 --2- 172.21.15.97:0/699569101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df40d0e20 0x7f4df40b6c50 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:26.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.126+0000 7f4df9bff700 1 -- 172.21.15.97:0/699569101 >> 172.21.15.97:0/699569101 conn(0x7f4df409c130 msgr2=0x7f4df40d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:26.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.126+0000 7f4df9bff700 1 -- 172.21.15.97:0/699569101 shutdown_connections 2023-07-06T03:52:26.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.126+0000 7f4df9bff700 1 -- 172.21.15.97:0/699569101 wait complete. 2023-07-06T03:52:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.127+0000 7f4df9bff700 1 Processor -- start 2023-07-06T03:52:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.128+0000 7f4df9bff700 1 -- start start 2023-07-06T03:52:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.128+0000 7f4df9bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df40d09b0 0x7f4df4155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df9bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df40d0e20 0x7f4df415b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df9bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4df415ff00 0x7f4df415bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df9bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df41f31e0 con 0x7f4df40d09b0 2023-07-06T03:52:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df9bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4157020 con 0x7f4df415ff00 2023-07-06T03:52:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df9bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4157300 con 0x7f4df40d0e20 2023-07-06T03:52:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df40d09b0 0x7f4df4155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df40d09b0 0x7f4df4155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54766/0 (socket says 172.21.15.97:54766) 2023-07-06T03:52:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df37fe700 1 -- 172.21.15.97:0/864880840 learned_addr learned my addr 172.21.15.97:0/864880840 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df37fe700 1 -- 172.21.15.97:0/864880840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df40d0e20 msgr2=0x7f4df415b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df37fe700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df40d0e20 0x7f4df415b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df37fe700 1 -- 172.21.15.97:0/864880840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4df415ff00 msgr2=0x7f4df415bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df37fe700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4df415ff00 0x7f4df415bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df37fe700 1 -- 172.21.15.97:0/864880840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec009a70 con 0x7f4df40d09b0 2023-07-06T03:52:26.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.129+0000 7f4df37fe700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df40d09b0 0x7f4df4155280 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f4de4007fd0 tx=0x7f4de400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:26.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.130+0000 7f4df0ff9700 1 -- 172.21.15.97:0/864880840 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de40074e0 con 0x7f4df40d09b0 2023-07-06T03:52:26.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.130+0000 7f4df0ff9700 1 -- 172.21.15.97:0/864880840 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de4007640 con 0x7f4df40d09b0 2023-07-06T03:52:26.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.130+0000 7f4df9bff700 1 -- 172.21.15.97:0/864880840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df41575e0 con 0x7f4df40d09b0 2023-07-06T03:52:26.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.130+0000 7f4df0ff9700 1 -- 172.21.15.97:0/864880840 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4005a50 con 0x7f4df40d09b0 2023-07-06T03:52:26.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.130+0000 7f4df9bff700 1 -- 172.21.15.97:0/864880840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df41557c0 con 0x7f4df40d09b0 2023-07-06T03:52:26.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.131+0000 7f4df9bff700 1 -- 172.21.15.97:0/864880840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df4157c30 con 0x7f4df40d09b0 2023-07-06T03:52:26.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.132+0000 7f4df0ff9700 1 -- 172.21.15.97:0/864880840 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4de4005bb0 con 0x7f4df40d09b0 2023-07-06T03:52:26.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.133+0000 7f4df0ff9700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd406ace0 0x7f4dd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:26.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.133+0000 7f4df0ff9700 1 -- 172.21.15.97:0/864880840 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4de4088540 con 0x7f4df40d09b0 2023-07-06T03:52:26.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.133+0000 7f4debfff700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd406ace0 0x7f4dd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:26.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.133+0000 7f4debfff700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd406ace0 0x7f4dd406d190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4df415b730 tx=0x7f4ddc007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:26.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.137+0000 7f4df0ff9700 1 -- 172.21.15.97:0/864880840 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4de4053a40 con 0x7f4df40d09b0 2023-07-06T03:52:26.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.409+0000 7f4df9bff700 1 -- 172.21.15.97:0/864880840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) v1 -- 0x7f4df404e340 con 0x7f4df40d09b0 2023-07-06T03:52:26.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.909+0000 7f4df0ff9700 1 -- 172.21.15.97:0/864880840 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v367) v1 ==== 153+0+0 (secure 0 0 0) 0x7f4de4005e60 con 0x7f4df40d09b0 2023-07-06T03:52:26.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.911+0000 7f4df9bff700 1 -- 172.21.15.97:0/864880840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd406ace0 msgr2=0x7f4dd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:26.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.911+0000 7f4df9bff700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd406ace0 0x7f4dd406d190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4df415b730 tx=0x7f4ddc007a40 comp rx=0 tx=0).stop 2023-07-06T03:52:26.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.911+0000 7f4df9bff700 1 -- 172.21.15.97:0/864880840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df40d09b0 msgr2=0x7f4df4155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:26.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.911+0000 7f4df9bff700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df40d09b0 0x7f4df4155280 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f4de4007fd0 tx=0x7f4de400c790 comp rx=0 tx=0).stop 2023-07-06T03:52:26.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.912+0000 7f4df9bff700 1 -- 172.21.15.97:0/864880840 shutdown_connections 2023-07-06T03:52:26.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.912+0000 7f4df9bff700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4dd406ace0 0x7f4dd406d190 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:26.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.912+0000 7f4df9bff700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4df415ff00 0x7f4df415bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:26.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.912+0000 7f4df9bff700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df40d09b0 0x7f4df4155280 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:26.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.912+0000 7f4df9bff700 1 --2- 172.21.15.97:0/864880840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4df40d0e20 0x7f4df415b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:26.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.912+0000 7f4df9bff700 1 -- 172.21.15.97:0/864880840 >> 172.21.15.97:0/864880840 conn(0x7f4df409c130 msgr2=0x7f4df415f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:26.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.912+0000 7f4df9bff700 1 -- 172.21.15.97:0/864880840 shutdown_connections 2023-07-06T03:52:26.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:26.912+0000 7f4df9bff700 1 -- 172.21.15.97:0/864880840 wait complete. 2023-07-06T03:52:26.932 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T03:52:27.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.128+0000 7f8b359ff700 1 Processor -- start 2023-07-06T03:52:27.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.128+0000 7f8b359ff700 1 -- start start 2023-07-06T03:52:27.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b359ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3015c610 0x7f8b300b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b359ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8b300d2210 0x7f8b300d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b359ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b300cff90 con 0x7f8b3006b360 2023-07-06T03:52:27.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b359ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b300d0780 con 0x7f8b300d2210 2023-07-06T03:52:27.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b359ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3015c9e0 con 0x7f8b3015c610 2023-07-06T03:52:27.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b27fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8b3006b360 0x7f8b300b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53550/0 (socket says 172.21.15.97:53550) 2023-07-06T03:52:27.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b27fff700 1 -- 172.21.15.97:0/1424518717 learned_addr learned my addr 172.21.15.97:0/1424518717 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:27.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b2f7fe700 1 --2- 172.21.15.97:0/1424518717 >> v2:172.21.15.97:3300/0 conn(0x7f8b300d2210 0x7f8b300d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b2effd700 1 --2- 172.21.15.97:0/1424518717 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3015c610 0x7f8b300b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b2e7fc700 1 -- 172.21.15.97:0/1424518717 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570523821 0 0) 0x7f8b300cff90 con 0x7f8b3006b360 2023-07-06T03:52:27.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b2e7fc700 1 -- 172.21.15.97:0/1424518717 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b1c0035b0 con 0x7f8b3006b360 2023-07-06T03:52:27.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b2f7fe700 1 -- 172.21.15.97:0/1424518717 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3015c610 msgr2=0x7f8b300b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:27.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b2f7fe700 1 --2- 172.21.15.97:0/1424518717 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3015c610 0x7f8b300b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b2f7fe700 1 -- 172.21.15.97:0/1424518717 >> v1:172.21.15.97:6789/0 conn(0x7f8b3006b360 legacy=0x7f8b300b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:27.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b2f7fe700 1 -- 172.21.15.97:0/1424518717 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b300b5650 con 0x7f8b300d2210 2023-07-06T03:52:27.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.130+0000 7f8b2effd700 1 --2- 172.21.15.97:0/1424518717 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3015c610 0x7f8b300b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:27.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.131+0000 7f8b2f7fe700 1 --2- 172.21.15.97:0/1424518717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b300d2210 0x7f8b300d1310 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009dc0 tx=0x7f8b2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a1fdbc6fad340b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:27.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.131+0000 7f8b2e7fc700 1 -- 172.21.15.97:0/1424518717 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800e070 con 0x7f8b300d2210 2023-07-06T03:52:27.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.131+0000 7f8b2e7fc700 1 -- 172.21.15.97:0/1424518717 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2800bae0 con 0x7f8b300d2210 2023-07-06T03:52:27.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.131+0000 7f8b2e7fc700 1 -- 172.21.15.97:0/1424518717 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800be90 con 0x7f8b300d2210 2023-07-06T03:52:27.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.131+0000 7f8b359ff700 1 -- 172.21.15.97:0/1424518717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b300d2210 msgr2=0x7f8b300d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.131+0000 7f8b359ff700 1 --2- 172.21.15.97:0/1424518717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b300d2210 0x7f8b300d1310 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009dc0 tx=0x7f8b2800b330 comp rx=0 tx=0).stop 2023-07-06T03:52:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.132+0000 7f8b359ff700 1 -- 172.21.15.97:0/1424518717 shutdown_connections 2023-07-06T03:52:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.132+0000 7f8b359ff700 1 --2- 172.21.15.97:0/1424518717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b300d2210 0x7f8b300d1310 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.132+0000 7f8b359ff700 1 --2- 172.21.15.97:0/1424518717 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3015c610 0x7f8b300b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.132+0000 7f8b359ff700 1 -- 172.21.15.97:0/1424518717 >> 172.21.15.97:0/1424518717 conn(0x7f8b3006a6c0 msgr2=0x7f8b3006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:27.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.132+0000 7f8b359ff700 1 -- 172.21.15.97:0/1424518717 shutdown_connections 2023-07-06T03:52:27.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.132+0000 7f8b359ff700 1 -- 172.21.15.97:0/1424518717 wait complete. 2023-07-06T03:52:27.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.133+0000 7f8b359ff700 1 Processor -- start 2023-07-06T03:52:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.134+0000 7f8b359ff700 1 -- start start 2023-07-06T03:52:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b359ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3006b360 0x7f8b30153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b359ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b300d2210 0x7f8b300ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b359ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3015c610 0x7f8b30153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b359ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b300ca310 con 0x7f8b3015c610 2023-07-06T03:52:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b359ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b300cac70 con 0x7f8b300d2210 2023-07-06T03:52:27.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b359ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3015ee50 con 0x7f8b3006b360 2023-07-06T03:52:27.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b2f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3015c610 0x7f8b30153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b2f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3015c610 0x7f8b30153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54788/0 (socket says 172.21.15.97:54788) 2023-07-06T03:52:27.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b2f7fe700 1 -- 172.21.15.97:0/1284213313 learned_addr learned my addr 172.21.15.97:0/1284213313 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:27.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b27fff700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b300d2210 0x7f8b300ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b2effd700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3006b360 0x7f8b30153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b2f7fe700 1 -- 172.21.15.97:0/1284213313 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3006b360 msgr2=0x7f8b30153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:27.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b2f7fe700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3006b360 0x7f8b30153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b2f7fe700 1 -- 172.21.15.97:0/1284213313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b300d2210 msgr2=0x7f8b300ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:27.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b2f7fe700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b300d2210 0x7f8b300ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b2f7fe700 1 -- 172.21.15.97:0/1284213313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28009a70 con 0x7f8b3015c610 2023-07-06T03:52:27.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b2effd700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3006b360 0x7f8b30153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:27.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.135+0000 7f8b27fff700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b300d2210 0x7f8b300ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:27.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.136+0000 7f8b2f7fe700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3015c610 0x7f8b30153560 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800b370 tx=0x7f8b28004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:27.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.136+0000 7f8b2cff9700 1 -- 172.21.15.97:0/1284213313 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28004730 con 0x7f8b3015c610 2023-07-06T03:52:27.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.136+0000 7f8b2cff9700 1 -- 172.21.15.97:0/1284213313 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b28020070 con 0x7f8b3015c610 2023-07-06T03:52:27.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.136+0000 7f8b2cff9700 1 -- 172.21.15.97:0/1284213313 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28027480 con 0x7f8b3015c610 2023-07-06T03:52:27.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.136+0000 7f8b359ff700 1 -- 172.21.15.97:0/1284213313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b30153aa0 con 0x7f8b3015c610 2023-07-06T03:52:27.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.136+0000 7f8b359ff700 1 -- 172.21.15.97:0/1284213313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3015eb70 con 0x7f8b3015c610 2023-07-06T03:52:27.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.137+0000 7f8b359ff700 1 -- 172.21.15.97:0/1284213313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b300ca450 con 0x7f8b3015c610 2023-07-06T03:52:27.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.138+0000 7f8b2cff9700 1 -- 172.21.15.97:0/1284213313 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8b28012430 con 0x7f8b3015c610 2023-07-06T03:52:27.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.139+0000 7f8b2cff9700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1006ace0 0x7f8b1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.139+0000 7f8b2cff9700 1 -- 172.21.15.97:0/1284213313 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8b28022070 con 0x7f8b3015c610 2023-07-06T03:52:27.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.139+0000 7f8b2effd700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1006ace0 0x7f8b1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.140+0000 7f8b2effd700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1006ace0 0x7f8b1006d190 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8b301f51a0 tx=0x7f8b20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:27.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.143+0000 7f8b2cff9700 1 -- 172.21.15.97:0/1284213313 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8b300ca450 con 0x7f8b3015c610 2023-07-06T03:52:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.340+0000 7f8b359ff700 1 -- 172.21.15.97:0/1284213313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8b300ca450 con 0x7f8b3015c610 2023-07-06T03:52:27.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.344+0000 7f8b2cff9700 1 -- 172.21.15.97:0/1284213313 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8b300ca450 con 0x7f8b3015c610 2023-07-06T03:52:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 -- 172.21.15.97:0/1284213313 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1006ace0 msgr2=0x7f8b1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1006ace0 0x7f8b1006d190 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8b301f51a0 tx=0x7f8b20008040 comp rx=0 tx=0).stop 2023-07-06T03:52:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 -- 172.21.15.97:0/1284213313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3015c610 msgr2=0x7f8b30153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3015c610 0x7f8b30153560 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800b370 tx=0x7f8b28004ad0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 -- 172.21.15.97:0/1284213313 shutdown_connections 2023-07-06T03:52:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b1006ace0 0x7f8b1006d190 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3015c610 0x7f8b30153560 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3006b360 0x7f8b30153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 --2- 172.21.15.97:0/1284213313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b300d2210 0x7f8b300ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 -- 172.21.15.97:0/1284213313 >> 172.21.15.97:0/1284213313 conn(0x7f8b3006a6c0 msgr2=0x7f8b3015b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 -- 172.21.15.97:0/1284213313 shutdown_connections 2023-07-06T03:52:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.346+0000 7f8b359ff700 1 -- 172.21.15.97:0/1284213313 wait complete. 2023-07-06T03:52:27.366 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:52:27.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:52:27.471 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:52:27.573 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:52:27.574 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:52:27.574 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:52:27.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:52:27.619 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:52:27.620 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:52:27.667 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:52:27.668 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:52:27.668 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:52:27.714 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:52:27.715 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:52:27.715 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:52:27.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:52:27.761 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T03:52:27.762 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:52:27.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.955+0000 7fa0031ff700 1 Processor -- start 2023-07-06T03:52:27.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.955+0000 7fa0031ff700 1 -- start start 2023-07-06T03:52:27.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.956+0000 7fa0031ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ffc15c980 0x7f9ffc0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.956+0000 7fa0031ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9ffc0d2930 0x7f9ffc0cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.956+0000 7fa0031ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ffc0d0c70 con 0x7f9ffc0b1990 2023-07-06T03:52:27.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.956+0000 7fa0031ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ffc0d0ec0 con 0x7f9ffc15c980 2023-07-06T03:52:27.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.956+0000 7fa0031ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ffc0d11a0 con 0x7f9ffc0d2930 2023-07-06T03:52:27.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.956+0000 7fa000f9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ffc15c980 0x7f9ffc0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7fa000f9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ffc15c980 0x7f9ffc0d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54800/0 (socket says 172.21.15.97:54800) 2023-07-06T03:52:27.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7fa00179c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9ffc0b1990 0x7f9ffc0b5ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53560/0 (socket says 172.21.15.97:53560) 2023-07-06T03:52:27.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7fa000f9b700 1 -- 172.21.15.97:0/3444365524 learned_addr learned my addr 172.21.15.97:0/3444365524 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:27.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7f9ffbfff700 1 --2- 172.21.15.97:0/3444365524 >> v2:172.21.15.146:3300/0 conn(0x7f9ffc0d2930 0x7f9ffc0cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7fa000f9b700 1 -- 172.21.15.97:0/3444365524 >> v2:172.21.15.146:3300/0 conn(0x7f9ffc0d2930 msgr2=0x7f9ffc0cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7fa000f9b700 1 --2- 172.21.15.97:0/3444365524 >> v2:172.21.15.146:3300/0 conn(0x7f9ffc0d2930 0x7f9ffc0cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7fa000f9b700 1 -- 172.21.15.97:0/3444365524 >> v1:172.21.15.97:6789/0 conn(0x7f9ffc0b1990 legacy=0x7f9ffc0b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7fa000f9b700 1 -- 172.21.15.97:0/3444365524 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc0d1480 con 0x7f9ffc15c980 2023-07-06T03:52:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7f9ffb7fe700 1 -- 172.21.15.97:0/3444365524 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899961680 0 0) 0x7f9ffc0d0c70 con 0x7f9ffc0b1990 2023-07-06T03:52:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7fa000f9b700 1 --2- 172.21.15.97:0/3444365524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ffc15c980 0x7f9ffc0d1c80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0cd350 tx=0x7f9ff0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b794c004308b8f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7f9ffb7fe700 1 -- 172.21.15.97:0/3444365524 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff000b870 con 0x7f9ffc15c980 2023-07-06T03:52:27.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7f9ffb7fe700 1 -- 172.21.15.97:0/3444365524 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff000b9d0 con 0x7f9ffc15c980 2023-07-06T03:52:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.957+0000 7f9ffb7fe700 1 -- 172.21.15.97:0/3444365524 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff000bd60 con 0x7f9ffc15c980 2023-07-06T03:52:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.958+0000 7fa0031ff700 1 -- 172.21.15.97:0/3444365524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ffc15c980 msgr2=0x7f9ffc0d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.958+0000 7fa0031ff700 1 --2- 172.21.15.97:0/3444365524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ffc15c980 0x7f9ffc0d1c80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0cd350 tx=0x7f9ff0009bd0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.958+0000 7fa0031ff700 1 -- 172.21.15.97:0/3444365524 shutdown_connections 2023-07-06T03:52:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.958+0000 7fa0031ff700 1 --2- 172.21.15.97:0/3444365524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ffc15c980 0x7f9ffc0d1c80 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.958+0000 7fa0031ff700 1 --2- 172.21.15.97:0/3444365524 >> v2:172.21.15.146:3300/0 conn(0x7f9ffc0d2930 0x7f9ffc0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.958+0000 7fa0031ff700 1 -- 172.21.15.97:0/3444365524 >> 172.21.15.97:0/3444365524 conn(0x7f9ffc06b2f0 msgr2=0x7f9ffc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:27.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.958+0000 7fa0031ff700 1 -- 172.21.15.97:0/3444365524 shutdown_connections 2023-07-06T03:52:27.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.958+0000 7fa0031ff700 1 -- 172.21.15.97:0/3444365524 wait complete. 2023-07-06T03:52:27.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.960+0000 7fa0031ff700 1 Processor -- start 2023-07-06T03:52:27.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.960+0000 7fa0031ff700 1 -- start start 2023-07-06T03:52:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7fa0031ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ffc0b1990 0x7f9ffc0c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7fa0031ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ffc0d2930 0x7f9ffc1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7fa0031ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ffc15c980 0x7f9ffc0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7fa0031ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc156910 con 0x7f9ffc0b1990 2023-07-06T03:52:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7fa0031ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc0b3620 con 0x7f9ffc15c980 2023-07-06T03:52:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7fa0031ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc0b3850 con 0x7f9ffc0d2930 2023-07-06T03:52:27.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7fa000f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ffc0b1990 0x7f9ffc0c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7fa000f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ffc0b1990 0x7f9ffc0c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54806/0 (socket says 172.21.15.97:54806) 2023-07-06T03:52:27.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7fa000f9b700 1 -- 172.21.15.97:0/2545637040 learned_addr learned my addr 172.21.15.97:0/2545637040 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:27.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7fa00179c700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ffc15c980 0x7f9ffc0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7f9ffbfff700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ffc0d2930 0x7f9ffc1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.961+0000 7fa000f9b700 1 -- 172.21.15.97:0/2545637040 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ffc0d2930 msgr2=0x7f9ffc1563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:27.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.962+0000 7fa000f9b700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ffc0d2930 0x7f9ffc1563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.962+0000 7fa000f9b700 1 -- 172.21.15.97:0/2545637040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ffc15c980 msgr2=0x7f9ffc0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:27.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.962+0000 7fa000f9b700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ffc15c980 0x7f9ffc0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.962+0000 7fa000f9b700 1 -- 172.21.15.97:0/2545637040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0009550 con 0x7f9ffc0b1990 2023-07-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.962+0000 7fa000f9b700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ffc0b1990 0x7f9ffc0c7e00 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0000c00 tx=0x7f9ff0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.962+0000 7f9ff97fa700 1 -- 172.21.15.97:0/2545637040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0004520 con 0x7f9ffc0b1990 2023-07-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.962+0000 7f9ff97fa700 1 -- 172.21.15.97:0/2545637040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff001e070 con 0x7f9ffc0b1990 2023-07-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.962+0000 7fa0031ff700 1 -- 172.21.15.97:0/2545637040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc0b3b30 con 0x7f9ffc0b1990 2023-07-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.962+0000 7f9ff97fa700 1 -- 172.21.15.97:0/2545637040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00038a0 con 0x7f9ffc0b1990 2023-07-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.962+0000 7fa0031ff700 1 -- 172.21.15.97:0/2545637040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ffc0b40f0 con 0x7f9ffc0b1990 2023-07-06T03:52:27.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.963+0000 7fa0031ff700 1 -- 172.21.15.97:0/2545637040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ffc04e340 con 0x7f9ffc0b1990 2023-07-06T03:52:27.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.964+0000 7f9ff97fa700 1 -- 172.21.15.97:0/2545637040 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ff0003a00 con 0x7f9ffc0b1990 2023-07-06T03:52:27.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.965+0000 7f9ff97fa700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fe006ace0 0x7f9fe006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:27.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.965+0000 7f9ff97fa700 1 -- 172.21.15.97:0/2545637040 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9ff0020070 con 0x7f9ffc0b1990 2023-07-06T03:52:27.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.966+0000 7f9ffbfff700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fe006ace0 0x7f9fe006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:27.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.966+0000 7f9ffbfff700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fe006ace0 0x7f9fe006d190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0cdb80 tx=0x7f9fe8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:27.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:27.969+0000 7f9ff97fa700 1 -- 172.21.15.97:0/2545637040 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9ff0055b00 con 0x7f9ffc0b1990 2023-07-06T03:52:28.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.207+0000 7fa0031ff700 1 -- 172.21.15.97:0/2545637040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ffc0d1f00 con 0x7f9ffc0b1990 2023-07-06T03:52:28.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.208+0000 7f9ff97fa700 1 -- 172.21.15.97:0/2545637040 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f9ff0015de0 con 0x7f9ffc0b1990 2023-07-06T03:52:28.209 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:28.210 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2344840493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2344840493},{"type":"v1","addr":"172.21.15.97:6836","nonce":2344840493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1930131645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1930131645},{"type":"v1","addr":"172.21.15.146:6835","nonce":1930131645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3936349837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3936349837},{"type":"v1","addr":"172.21.15.146:6833","nonce":3936349837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":7395,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/870744723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":870744723},{"type":"v1","addr":"172.21.15.97:6837","nonce":870744723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[]} 2023-07-06T03:52:28.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 -- 172.21.15.97:0/2545637040 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fe006ace0 msgr2=0x7f9fe006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:28.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fe006ace0 0x7f9fe006d190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0cdb80 tx=0x7f9fe8008040 comp rx=0 tx=0).stop 2023-07-06T03:52:28.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 -- 172.21.15.97:0/2545637040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ffc0b1990 msgr2=0x7f9ffc0c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:28.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ffc0b1990 0x7f9ffc0c7e00 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0000c00 tx=0x7f9ff0004750 comp rx=0 tx=0).stop 2023-07-06T03:52:28.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 -- 172.21.15.97:0/2545637040 shutdown_connections 2023-07-06T03:52:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fe006ace0 0x7f9fe006d190 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ffc15c980 0x7f9ffc0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ffc0b1990 0x7f9ffc0c7e00 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 --2- 172.21.15.97:0/2545637040 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ffc0d2930 0x7f9ffc1563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 -- 172.21.15.97:0/2545637040 >> 172.21.15.97:0/2545637040 conn(0x7f9ffc06b2f0 msgr2=0x7f9ffc1581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 -- 172.21.15.97:0/2545637040 shutdown_connections 2023-07-06T03:52:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.210+0000 7fa0031ff700 1 -- 172.21.15.97:0/2545637040 wait complete. 2023-07-06T03:52:28.214 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 50 2023-07-06T03:52:28.230 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T03:52:28.230 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T03:52:28.230 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T03:52:28.231 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:52:28.231 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T03:52:28.231 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T03:52:28.231 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T03:52:28.232 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:52:28.232 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T03:52:28.232 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T03:52:28.232 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T03:52:28.233 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:52:28.233 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T03:52:28.233 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T03:52:28.233 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T03:52:28.233 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:52:28.246 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T03:52:28.247 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T03:52:28.247 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T03:52:28.248 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T03:52:28.249 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T03:52:28.249 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T03:52:28.250 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T03:52:28.250 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T03:52:28.292 INFO:tasks.ceph.mds.a:Started 2023-07-06T03:52:28.292 DEBUG:teuthology.parallel:result is None 2023-07-06T03:52:28.293 INFO:tasks.ceph.mds.d:Started 2023-07-06T03:52:28.294 DEBUG:teuthology.parallel:result is None 2023-07-06T03:52:28.295 INFO:tasks.ceph.mds.c:Started 2023-07-06T03:52:28.295 DEBUG:teuthology.parallel:result is None 2023-07-06T03:52:28.296 INFO:tasks.ceph.mds.b:Started 2023-07-06T03:52:28.296 DEBUG:teuthology.parallel:result is None 2023-07-06T03:52:28.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T03:52:28.439 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-07-06T03:52:28.440 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:52:28.440 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:52:28.441 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:52:28.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.500+0000 7f1895bff700 1 Processor -- start 2023-07-06T03:52:28.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.500+0000 7f1895bff700 1 -- start start 2023-07-06T03:52:28.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.501+0000 7f1895bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f189006b170 0x7f189015d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:28.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.501+0000 7f1895bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f189015cae0 0x7f189006b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:28.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.501+0000 7f1895bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18900cfbc0 0x7f189015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:28.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.501+0000 7f1895bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18900b25a0 con 0x7f189015cae0 2023-07-06T03:52:28.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.501+0000 7f1895bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189015de00 con 0x7f189006b170 2023-07-06T03:52:28.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.501+0000 7f1895bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189006ba80 con 0x7f18900cfbc0 2023-07-06T03:52:28.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.501+0000 7f188effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f189015cae0 0x7f189006b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:28.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f189015cae0 0x7f189006b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54864/0 (socket says 172.21.15.97:54864) 2023-07-06T03:52:28.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188effd700 1 -- 172.21.15.97:0/2842857954 learned_addr learned my addr 172.21.15.97:0/2842857954 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:28.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188f7fe700 1 --2- 172.21.15.97:0/2842857954 >> v2:172.21.15.146:3300/0 conn(0x7f189006b170 0x7f189015d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:28.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188ffff700 1 --2- 172.21.15.97:0/2842857954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18900cfbc0 0x7f189015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:28.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188effd700 1 -- 172.21.15.97:0/2842857954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18900cfbc0 msgr2=0x7f189015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:28.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188effd700 1 --2- 172.21.15.97:0/2842857954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18900cfbc0 0x7f189015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188effd700 1 -- 172.21.15.97:0/2842857954 >> v2:172.21.15.146:3300/0 conn(0x7f189006b170 msgr2=0x7f189015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:28.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188effd700 1 --2- 172.21.15.97:0/2842857954 >> v2:172.21.15.146:3300/0 conn(0x7f189006b170 0x7f189015d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188effd700 1 -- 172.21.15.97:0/2842857954 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18900ceb20 con 0x7f189015cae0 2023-07-06T03:52:28.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188effd700 1 --2- 172.21.15.97:0/2842857954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f189015cae0 0x7f189006b540 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f18900ce700 tx=0x7f1884009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c48391fe21ca462 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188e7fc700 1 -- 172.21.15.97:0/2842857954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884011830 con 0x7f189015cae0 2023-07-06T03:52:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188e7fc700 1 -- 172.21.15.97:0/2842857954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1884011990 con 0x7f189015cae0 2023-07-06T03:52:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.502+0000 7f188e7fc700 1 -- 172.21.15.97:0/2842857954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884011d20 con 0x7f189015cae0 2023-07-06T03:52:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.503+0000 7f1895bff700 1 -- 172.21.15.97:0/2842857954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f189015cae0 msgr2=0x7f189006b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.503+0000 7f1895bff700 1 --2- 172.21.15.97:0/2842857954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f189015cae0 0x7f189006b540 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f18900ce700 tx=0x7f1884009bd0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.503+0000 7f1895bff700 1 -- 172.21.15.97:0/2842857954 shutdown_connections 2023-07-06T03:52:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.503+0000 7f1895bff700 1 --2- 172.21.15.97:0/2842857954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18900cfbc0 0x7f189015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.503+0000 7f1895bff700 1 --2- 172.21.15.97:0/2842857954 >> v2:172.21.15.146:3300/0 conn(0x7f189006b170 0x7f189015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.503+0000 7f1895bff700 1 --2- 172.21.15.97:0/2842857954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f189015cae0 0x7f189006b540 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.503+0000 7f1895bff700 1 -- 172.21.15.97:0/2842857954 >> 172.21.15.97:0/2842857954 conn(0x7f1890099f10 msgr2=0x7f18900d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:28.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.503+0000 7f1895bff700 1 -- 172.21.15.97:0/2842857954 shutdown_connections 2023-07-06T03:52:28.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.503+0000 7f1895bff700 1 -- 172.21.15.97:0/2842857954 wait complete. 2023-07-06T03:52:28.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.505+0000 7f1895bff700 1 Processor -- start 2023-07-06T03:52:28.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.505+0000 7f1895bff700 1 -- start start 2023-07-06T03:52:28.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f1895bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f189006b170 0x7f189009b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:28.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f1895bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18900cfbc0 0x7f18901f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:28.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f1895bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f189015cae0 0x7f189009da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:28.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f1895bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189014ae40 con 0x7f189006b170 2023-07-06T03:52:28.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f1895bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189009bae0 con 0x7f18900cfbc0 2023-07-06T03:52:28.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f1895bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189009bdc0 con 0x7f189015cae0 2023-07-06T03:52:28.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f188f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f189006b170 0x7f189009b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:28.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f188f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f189006b170 0x7f189009b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54872/0 (socket says 172.21.15.97:54872) 2023-07-06T03:52:28.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f188f7fe700 1 -- 172.21.15.97:0/1569444890 learned_addr learned my addr 172.21.15.97:0/1569444890 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:28.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f188f7fe700 1 -- 172.21.15.97:0/1569444890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f189015cae0 msgr2=0x7f189009da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:28.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f188f7fe700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f189015cae0 0x7f189009da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f188f7fe700 1 -- 172.21.15.97:0/1569444890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18900cfbc0 msgr2=0x7f18901f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:28.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f188f7fe700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18900cfbc0 0x7f18901f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:28.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.506+0000 7f188f7fe700 1 -- 172.21.15.97:0/1569444890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1884009550 con 0x7f189006b170 2023-07-06T03:52:28.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.507+0000 7f188f7fe700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f189006b170 0x7f189009b290 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f188000e9b0 tx=0x7f188000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:28.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.507+0000 7f1877fff700 1 -- 172.21.15.97:0/1569444890 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1880009670 con 0x7f189006b170 2023-07-06T03:52:28.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.507+0000 7f1877fff700 1 -- 172.21.15.97:0/1569444890 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1880005a20 con 0x7f189006b170 2023-07-06T03:52:28.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.507+0000 7f1895bff700 1 -- 172.21.15.97:0/1569444890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189009c0a0 con 0x7f189006b170 2023-07-06T03:52:28.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.507+0000 7f1877fff700 1 -- 172.21.15.97:0/1569444890 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1880005db0 con 0x7f189006b170 2023-07-06T03:52:28.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.507+0000 7f1895bff700 1 -- 172.21.15.97:0/1569444890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189009b7d0 con 0x7f189006b170 2023-07-06T03:52:28.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.508+0000 7f1895bff700 1 -- 172.21.15.97:0/1569444890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189006c630 con 0x7f189006b170 2023-07-06T03:52:28.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.509+0000 7f1877fff700 1 -- 172.21.15.97:0/1569444890 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f188000c7d0 con 0x7f189006b170 2023-07-06T03:52:28.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.510+0000 7f1877fff700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f187c06ace0 0x7f187c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:28.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.510+0000 7f1877fff700 1 -- 172.21.15.97:0/1569444890 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1880089330 con 0x7f189006b170 2023-07-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.510+0000 7f188effd700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f187c06ace0 0x7f187c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.511+0000 7f188effd700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f187c06ace0 0x7f187c06d190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f189014d3e0 tx=0x7f1884010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.514+0000 7f1877fff700 1 -- 172.21.15.97:0/1569444890 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1880058200 con 0x7f189006b170 2023-07-06T03:52:28.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:28.727+0000 7f1895bff700 1 -- 172.21.15.97:0/1569444890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f189004e340 con 0x7f189006b170 2023-07-06T03:52:29.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.738+0000 7f1877fff700 1 -- 172.21.15.97:0/1569444890 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v82) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1880057d90 con 0x7f189006b170 2023-07-06T03:52:29.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 -- 172.21.15.97:0/1569444890 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f187c06ace0 msgr2=0x7f187c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:29.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f187c06ace0 0x7f187c06d190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f189014d3e0 tx=0x7f1884010000 comp rx=0 tx=0).stop 2023-07-06T03:52:29.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 -- 172.21.15.97:0/1569444890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f189006b170 msgr2=0x7f189009b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:29.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f189006b170 0x7f189009b290 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f188000e9b0 tx=0x7f188000ed70 comp rx=0 tx=0).stop 2023-07-06T03:52:29.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 -- 172.21.15.97:0/1569444890 shutdown_connections 2023-07-06T03:52:29.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f187c06ace0 0x7f187c06d190 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:29.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f189015cae0 0x7f189009da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:29.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f189006b170 0x7f189009b290 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:29.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 --2- 172.21.15.97:0/1569444890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18900cfbc0 0x7f18901f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:29.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 -- 172.21.15.97:0/1569444890 >> 172.21.15.97:0/1569444890 conn(0x7f1890099f10 msgr2=0x7f18900d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:29.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 -- 172.21.15.97:0/1569444890 shutdown_connections 2023-07-06T03:52:29.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.741+0000 7f1895bff700 1 -- 172.21.15.97:0/1569444890 wait complete. 2023-07-06T03:52:29.745 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-07-06T03:52:29.764 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T03:52:29.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.963+0000 7f197edff700 1 Processor -- start 2023-07-06T03:52:29.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.963+0000 7f197edff700 1 -- start start 2023-07-06T03:52:29.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.964+0000 7f197edff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780b8410 0x7f1978167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:29.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.964+0000 7f197edff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19780cdae0 con 0x7f19780cd390 2023-07-06T03:52:29.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.964+0000 7f197edff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19780b6640 con 0x7f197815c520 2023-07-06T03:52:29.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.964+0000 7f197edff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19780b6920 con 0x7f19780b8410 2023-07-06T03:52:29.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.964+0000 7f1977fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f19780cd390 0x7f1978160890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34598/0 (socket says 172.21.15.97:34598) 2023-07-06T03:52:29.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.964+0000 7f1977fff700 1 -- 172.21.15.97:0/1508397690 learned_addr learned my addr 172.21.15.97:0/1508397690 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.964+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163779678 0 0) 0x7f19780cdae0 con 0x7f19780cd390 2023-07-06T03:52:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19680035b0 con 0x7f19780cd390 2023-07-06T03:52:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f197d39c700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780b8410 0x7f1978167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 248542999 0 0) 0x7f19680035b0 con 0x7f19780cd390 2023-07-06T03:52:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780b8410 msgr2=0x7f1978167510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780b8410 0x7f1978167510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 >> v1:172.21.15.146:6789/0 conn(0x7f197815c520 legacy=0x7f1978160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19780b6c00 con 0x7f19780cd390 2023-07-06T03:52:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f1964001670 con 0x7f19780cd390 2023-07-06T03:52:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 >> v1:172.21.15.97:6789/0 conn(0x7f19780cd390 legacy=0x7f1978160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968003a30 0x7f1968003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1968004850 0x7f1968004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780b8410 0x7f1968005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780b6920 con 0x7f1968003a30 2023-07-06T03:52:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196800bac0 con 0x7f1968004850 2023-07-06T03:52:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196800bc80 con 0x7f19780b8410 2023-07-06T03:52:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f197cb9b700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968003a30 0x7f1968003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.965+0000 7f1977fff700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1968004850 0x7f1968004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.966+0000 7f197d39c700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780b8410 0x7f1968005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.966+0000 7f197cb9b700 1 -- 172.21.15.97:0/1508397690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780b8410 msgr2=0x7f1968005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.966+0000 7f197cb9b700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19780b8410 0x7f1968005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.966+0000 7f197cb9b700 1 -- 172.21.15.97:0/1508397690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1968004850 msgr2=0x7f1968004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.966+0000 7f197cb9b700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1968004850 0x7f1968004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.966+0000 7f197cb9b700 1 -- 172.21.15.97:0/1508397690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680035b0 con 0x7f1968003a30 2023-07-06T03:52:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.966+0000 7f197cb9b700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968003a30 0x7f1968003e40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f196c001800 tx=0x7f1968005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a351d48e956f1168 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.966+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c0012f0 con 0x7f1968003a30 2023-07-06T03:52:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.966+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c0052f0 con 0x7f1968003a30 2023-07-06T03:52:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.966+0000 7f19777fe700 1 -- 172.21.15.97:0/1508397690 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c006860 con 0x7f1968003a30 2023-07-06T03:52:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.967+0000 7f197edff700 1 -- 172.21.15.97:0/1508397690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968003a30 msgr2=0x7f1968003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.967+0000 7f197edff700 1 --2- 172.21.15.97:0/1508397690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968003a30 0x7f1968003e40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f196c001800 tx=0x7f1968005880 comp rx=0 tx=0).stop 2023-07-06T03:52:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.967+0000 7f197d39c700 1 -- 172.21.15.97:0/1508397690 reap_dead start 2023-07-06T03:52:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.967+0000 7f197edff700 1 -- 172.21.15.97:0/1508397690 shutdown_connections 2023-07-06T03:52:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.967+0000 7f197edff700 1 -- 172.21.15.97:0/1508397690 >> 172.21.15.97:0/1508397690 conn(0x7f1978071790 msgr2=0x7f1978071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.967+0000 7f197edff700 1 -- 172.21.15.97:0/1508397690 shutdown_connections 2023-07-06T03:52:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.967+0000 7f197edff700 1 -- 172.21.15.97:0/1508397690 wait complete. 2023-07-06T03:52:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.969+0000 7f197edff700 1 Processor -- start 2023-07-06T03:52:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.969+0000 7f197edff700 1 -- start start 2023-07-06T03:52:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197edff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1968003a30 0x7f19781f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197edff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19781f3300 0x7f19781f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197edff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19781f5d20 0x7f19780cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197edff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781f61c0 con 0x7f19781f5d20 2023-07-06T03:52:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197edff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197815d9e0 con 0x7f1968003a30 2023-07-06T03:52:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197edff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19781f5590 con 0x7f19781f3300 2023-07-06T03:52:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197d39c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19781f5d20 0x7f19780cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197d39c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19781f5d20 0x7f19780cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54774/0 (socket says 172.21.15.97:54774) 2023-07-06T03:52:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197d39c700 1 -- 172.21.15.97:0/36640187 learned_addr learned my addr 172.21.15.97:0/36640187 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f1977fff700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19781f3300 0x7f19781f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197cb9b700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1968003a30 0x7f19781f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197d39c700 1 -- 172.21.15.97:0/36640187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19781f3300 msgr2=0x7f19781f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197d39c700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19781f3300 0x7f19781f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:29.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197d39c700 1 -- 172.21.15.97:0/36640187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1968003a30 msgr2=0x7f19781f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197d39c700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1968003a30 0x7f19781f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f197d39c700 1 -- 172.21.15.97:0/36640187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197815dc50 con 0x7f19781f5d20 2023-07-06T03:52:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.970+0000 7f1977fff700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19781f3300 0x7f19781f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.971+0000 7f197d39c700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19781f5d20 0x7f19780cbdf0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f197808bd00 tx=0x7f197000f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.971+0000 7f19757fa700 1 -- 172.21.15.97:0/36640187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000e040 con 0x7f19781f5d20 2023-07-06T03:52:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.971+0000 7f19757fa700 1 -- 172.21.15.97:0/36640187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197000f040 con 0x7f19781f5d20 2023-07-06T03:52:29.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.971+0000 7f197edff700 1 -- 172.21.15.97:0/36640187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197815df30 con 0x7f19781f5d20 2023-07-06T03:52:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.971+0000 7f19757fa700 1 -- 172.21.15.97:0/36640187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970015430 con 0x7f19781f5d20 2023-07-06T03:52:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.971+0000 7f197edff700 1 -- 172.21.15.97:0/36640187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19780ccb90 con 0x7f19781f5d20 2023-07-06T03:52:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.972+0000 7f197edff700 1 -- 172.21.15.97:0/36640187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197804e340 con 0x7f19781f5d20 2023-07-06T03:52:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.973+0000 7f19757fa700 1 -- 172.21.15.97:0/36640187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1970015590 con 0x7f19781f5d20 2023-07-06T03:52:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.974+0000 7f19757fa700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195c06ace0 0x7f195c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.974+0000 7f19757fa700 1 -- 172.21.15.97:0/36640187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1970089730 con 0x7f19781f5d20 2023-07-06T03:52:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.974+0000 7f197cb9b700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195c06ace0 0x7f195c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.975+0000 7f197cb9b700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195c06ace0 0x7f195c06d190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f19780b3c70 tx=0x7f196c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:29.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:29.978+0000 7f19757fa700 1 -- 172.21.15.97:0/36640187 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1970054a90 con 0x7f19781f5d20 2023-07-06T03:52:30.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.249+0000 7f197edff700 1 -- 172.21.15.97:0/36640187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f19780b4f40 con 0x7f19781f5d20 2023-07-06T03:52:30.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.250+0000 7f19757fa700 1 -- 172.21.15.97:0/36640187 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f19700580b0 con 0x7f19781f5d20 2023-07-06T03:52:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:30.252 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:52:30.252 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-07-06T03:52:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-07-06T03:52:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkczuvGRAAUKjfmMyKOTCYzOQmtf3n3A==", 2023-07-06T03:52:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:52:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:52:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:52:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-07-06T03:52:30.254 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkFHvUMBAA2WgMcUUf+Qi8T2ZjhfWaHA==", 2023-07-06T03:52:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:52:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:52:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:52:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-07-06T03:52:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZk0N4+JRAA3tjqoIBPpZ6C7TMmOOg7lw==", 2023-07-06T03:52:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:52:30.256 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:52:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:52:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-07-06T03:52:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkJnHUABAAwIMH2DmoHaQN3frJCgRiQw==", 2023-07-06T03:52:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:52:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:52:30.258 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:52:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-07-06T03:52:30.259 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCLOKZkShOIABAAzIBHDcNkmnDOLSY7Qdb+FA==", 2023-07-06T03:52:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.260 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:52:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:52:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-07-06T03:52:30.261 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCOOKZkq1t7AxAAwihuhimbjitZDeki1QuMug==", 2023-07-06T03:52:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:52:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:52:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-07-06T03:52:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCROKZkdYv4BhAApTHg/pmlC0lO7hVTkimEgw==", 2023-07-06T03:52:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:52:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:52:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.263 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-07-06T03:52:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCUOKZkqhuMChAAsV9xG6g1dofrpWt3F8tw5w==", 2023-07-06T03:52:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:52:30.264 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:52:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-07-06T03:52:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCaOKZk1Q5aIBAABU1+afwIihm+g4xylryz/A==", 2023-07-06T03:52:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.265 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:52:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:52:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-07-06T03:52:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCdOKZkRskIJBAA+821aoG98aEV7h4FTINQFA==", 2023-07-06T03:52:30.266 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:52:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:52:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.267 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-07-06T03:52:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCgOKZk9qnMJxAATcWG+a/9Hl1AuIL/TSrdEA==", 2023-07-06T03:52:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:52:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:52:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-07-06T03:52:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCjOKZk6wJ/KxAAWnZxt7h7OJhQ+bqdel4+iQ==", 2023-07-06T03:52:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:52:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:52:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-07-06T03:52:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg==", 2023-07-06T03:52:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:52:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-07-06T03:52:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-06T03:52:30.270 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-07-06T03:52:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkq4/pGRAAucHmR1eBA8MNMy3/l7tl0g==", 2023-07-06T03:52:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:52:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:52:30.271 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:52:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:52:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-07-06T03:52:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkhL5fEhAAWhw4IbCPqTO2sXPIcpTVHA==", 2023-07-06T03:52:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:52:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:52:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:52:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:52:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.273 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-07-06T03:52:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkO81dIRAAffjVyvsb/ygBtx8UrYQK0w==", 2023-07-06T03:52:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:52:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:52:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:52:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:52:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-07-06T03:52:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCFOKZk6uVlHRAA92RDhdDKW/q65SEOQMWgiA==", 2023-07-06T03:52:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:52:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:52:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:52:30.275 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-07-06T03:52:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkT4ALEhAAXwwkcodQU9meJA0tfseJCg==", 2023-07-06T03:52:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.276 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:52:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:52:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-07-06T03:52:30.277 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkfnoEChAAuRxqXqPI33dgWkBJoUMh3w==", 2023-07-06T03:52:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:52:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:52:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:52:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:52:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:30.278 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:52:30.278 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:52:30.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 -- 172.21.15.97:0/36640187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195c06ace0 msgr2=0x7f195c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195c06ace0 0x7f195c06d190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f19780b3c70 tx=0x7f196c009030 comp rx=0 tx=0).stop 2023-07-06T03:52:30.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 -- 172.21.15.97:0/36640187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19781f5d20 msgr2=0x7f19780cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19781f5d20 0x7f19780cbdf0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f197808bd00 tx=0x7f197000f490 comp rx=0 tx=0).stop 2023-07-06T03:52:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 -- 172.21.15.97:0/36640187 shutdown_connections 2023-07-06T03:52:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195c06ace0 0x7f195c06d190 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19781f5d20 0x7f19780cbdf0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1968003a30 0x7f19781f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 --2- 172.21.15.97:0/36640187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19781f3300 0x7f19781f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 -- 172.21.15.97:0/36640187 >> 172.21.15.97:0/36640187 conn(0x7f1978071790 msgr2=0x7f19781573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 -- 172.21.15.97:0/36640187 shutdown_connections 2023-07-06T03:52:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.252+0000 7f197edff700 1 -- 172.21.15.97:0/36640187 wait complete. 2023-07-06T03:52:30.282 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:52:30.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.475+0000 7fb6ae9ff700 1 Processor -- start 2023-07-06T03:52:30.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.475+0000 7fb6ae9ff700 1 -- start start 2023-07-06T03:52:30.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.476+0000 7fb6ae9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6a8158f20 0x7fb6a80caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.476+0000 7fb6ae9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb6a81592f0 0x7fb6a80b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.476+0000 7fb6ae9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6a806d860 0x7fb6a80b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6ae9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a815cc40 con 0x7fb6a806d860 2023-07-06T03:52:30.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6ae9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a815cf20 con 0x7fb6a81592f0 2023-07-06T03:52:30.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6ae9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a815d200 con 0x7fb6a8158f20 2023-07-06T03:52:30.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6acf9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6a806d860 0x7fb6a80b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6acf9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6a806d860 0x7fb6a80b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54790/0 (socket says 172.21.15.97:54790) 2023-07-06T03:52:30.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6acf9c700 1 -- 172.21.15.97:0/3271739248 learned_addr learned my addr 172.21.15.97:0/3271739248 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:30.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6a77fe700 1 --2- 172.21.15.97:0/3271739248 >> v2:172.21.15.146:3300/0 conn(0x7fb6a81592f0 0x7fb6a80b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6a7fff700 1 --2- 172.21.15.97:0/3271739248 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6a8158f20 0x7fb6a80caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6acf9c700 1 -- 172.21.15.97:0/3271739248 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6a8158f20 msgr2=0x7fb6a80caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:30.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6acf9c700 1 --2- 172.21.15.97:0/3271739248 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6a8158f20 0x7fb6a80caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6acf9c700 1 -- 172.21.15.97:0/3271739248 >> v2:172.21.15.146:3300/0 conn(0x7fb6a81592f0 msgr2=0x7fb6a80b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:30.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6acf9c700 1 --2- 172.21.15.97:0/3271739248 >> v2:172.21.15.146:3300/0 conn(0x7fb6a81592f0 0x7fb6a80b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6acf9c700 1 -- 172.21.15.97:0/3271739248 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a815d4e0 con 0x7fb6a806d860 2023-07-06T03:52:30.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.477+0000 7fb6acf9c700 1 --2- 172.21.15.97:0/3271739248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6a806d860 0x7fb6a80b0aa0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb6a806f660 tx=0x7fb6a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c821eacf3511c8e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:30.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6a6ffd700 1 -- 172.21.15.97:0/3271739248 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000e070 con 0x7fb6a806d860 2023-07-06T03:52:30.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6a6ffd700 1 -- 172.21.15.97:0/3271739248 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a000b9c0 con 0x7fb6a806d860 2023-07-06T03:52:30.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6a6ffd700 1 -- 172.21.15.97:0/3271739248 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000bd70 con 0x7fb6a806d860 2023-07-06T03:52:30.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/3271739248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6a806d860 msgr2=0x7fb6a80b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6ae9ff700 1 --2- 172.21.15.97:0/3271739248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6a806d860 0x7fb6a80b0aa0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb6a806f660 tx=0x7fb6a000b330 comp rx=0 tx=0).stop 2023-07-06T03:52:30.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/3271739248 shutdown_connections 2023-07-06T03:52:30.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6ae9ff700 1 --2- 172.21.15.97:0/3271739248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6a806d860 0x7fb6a80b0aa0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6ae9ff700 1 --2- 172.21.15.97:0/3271739248 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6a8158f20 0x7fb6a80caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6ae9ff700 1 --2- 172.21.15.97:0/3271739248 >> v2:172.21.15.146:3300/0 conn(0x7fb6a81592f0 0x7fb6a80b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/3271739248 >> 172.21.15.97:0/3271739248 conn(0x7fb6a80ce8c0 msgr2=0x7fb6a80cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:30.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/3271739248 shutdown_connections 2023-07-06T03:52:30.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.478+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/3271739248 wait complete. 2023-07-06T03:52:30.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.480+0000 7fb6ae9ff700 1 Processor -- start 2023-07-06T03:52:30.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.480+0000 7fb6ae9ff700 1 -- start start 2023-07-06T03:52:30.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.481+0000 7fb6ae9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6a806d860 0x7fb6a81463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.481+0000 7fb6ae9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6a8158f20 0x7fb6a814b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.481+0000 7fb6ae9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6a81592f0 0x7fb6a81438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.481+0000 7fb6ae9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a814b6b0 con 0x7fb6a81592f0 2023-07-06T03:52:30.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.481+0000 7fb6ae9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a814b8c0 con 0x7fb6a806d860 2023-07-06T03:52:30.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.481+0000 7fb6ae9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a814bba0 con 0x7fb6a8158f20 2023-07-06T03:52:30.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.481+0000 7fb6acf9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6a81592f0 0x7fb6a81438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6acf9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6a81592f0 0x7fb6a81438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54798/0 (socket says 172.21.15.97:54798) 2023-07-06T03:52:30.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6acf9c700 1 -- 172.21.15.97:0/2816461704 learned_addr learned my addr 172.21.15.97:0/2816461704 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:30.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6a77fe700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6a8158f20 0x7fb6a814b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6a7fff700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6a806d860 0x7fb6a81463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6acf9c700 1 -- 172.21.15.97:0/2816461704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6a8158f20 msgr2=0x7fb6a814b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6acf9c700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6a8158f20 0x7fb6a814b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6acf9c700 1 -- 172.21.15.97:0/2816461704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6a806d860 msgr2=0x7fb6a81463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6acf9c700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6a806d860 0x7fb6a81463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6acf9c700 1 -- 172.21.15.97:0/2816461704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a0009b30 con 0x7fb6a81592f0 2023-07-06T03:52:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6acf9c700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6a81592f0 0x7fb6a81438a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb6a806b980 tx=0x7fb6a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6a4ff9700 1 -- 172.21.15.97:0/2816461704 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00099b0 con 0x7fb6a81592f0 2023-07-06T03:52:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6a4ff9700 1 -- 172.21.15.97:0/2816461704 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0004250 con 0x7fb6a81592f0 2023-07-06T03:52:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/2816461704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a813d520 con 0x7fb6a81592f0 2023-07-06T03:52:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.482+0000 7fb6a4ff9700 1 -- 172.21.15.97:0/2816461704 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0004600 con 0x7fb6a81592f0 2023-07-06T03:52:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.483+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/2816461704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a813dae0 con 0x7fb6a81592f0 2023-07-06T03:52:30.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.484+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/2816461704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a813dd00 con 0x7fb6a81592f0 2023-07-06T03:52:30.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.485+0000 7fb6a4ff9700 1 -- 172.21.15.97:0/2816461704 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb6a0004760 con 0x7fb6a81592f0 2023-07-06T03:52:30.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.485+0000 7fb6a4ff9700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69406ace0 0x7fb69406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.486+0000 7fb6a4ff9700 1 -- 172.21.15.97:0/2816461704 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb6a008a590 con 0x7fb6a81592f0 2023-07-06T03:52:30.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.486+0000 7fb6a7fff700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69406ace0 0x7fb69406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.486+0000 7fb6a7fff700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69406ace0 0x7fb69406d190 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb698006b20 tx=0x7fb698008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:30.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.490+0000 7fb6a4ff9700 1 -- 172.21.15.97:0/2816461704 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb6a0055980 con 0x7fb6a81592f0 2023-07-06T03:52:30.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.683+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/2816461704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6a804e340 con 0x7fb6a81592f0 2023-07-06T03:52:30.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.684+0000 7fb6a4ff9700 1 -- 172.21.15.97:0/2816461704 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+14666 (secure 0 0 0) 0x7fb6a0058fa0 con 0x7fb6a81592f0 2023-07-06T03:52:30.686 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:30.686 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:52:29.733827+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:52:30.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.686+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/2816461704 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69406ace0 msgr2=0x7fb69406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.686+0000 7fb6ae9ff700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69406ace0 0x7fb69406d190 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb698006b20 tx=0x7fb698008040 comp rx=0 tx=0).stop 2023-07-06T03:52:30.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.686+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/2816461704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6a81592f0 msgr2=0x7fb6a81438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.686+0000 7fb6ae9ff700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6a81592f0 0x7fb6a81438a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb6a806b980 tx=0x7fb6a0004ad0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.686+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/2816461704 shutdown_connections 2023-07-06T03:52:30.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.686+0000 7fb6ae9ff700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb69406ace0 0x7fb69406d190 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.686+0000 7fb6ae9ff700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6a81592f0 0x7fb6a81438a0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.686+0000 7fb6ae9ff700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb6a806d860 0x7fb6a81463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.686+0000 7fb6ae9ff700 1 --2- 172.21.15.97:0/2816461704 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb6a8158f20 0x7fb6a814b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.687+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/2816461704 >> 172.21.15.97:0/2816461704 conn(0x7fb6a80ce8c0 msgr2=0x7fb6a80cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.687+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/2816461704 shutdown_connections 2023-07-06T03:52:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.687+0000 7fb6ae9ff700 1 -- 172.21.15.97:0/2816461704 wait complete. 2023-07-06T03:52:30.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:52:30.708 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:52:30.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.905+0000 7fe6be5ff700 1 Processor -- start 2023-07-06T03:52:30.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.905+0000 7fe6be5ff700 1 -- start start 2023-07-06T03:52:30.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.906+0000 7fe6be5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6b806b360 0x7fe6b80b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.906+0000 7fe6be5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b80cff90 con 0x7fe6b80d2210 2023-07-06T03:52:30.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.906+0000 7fe6be5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b80b5650 con 0x7fe6b815c610 2023-07-06T03:52:30.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.906+0000 7fe6be5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80d0780 con 0x7fe6b806b360 2023-07-06T03:52:30.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.906+0000 7fe6bcb9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe6b80d2210 0x7fe6b80d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34608/0 (socket says 172.21.15.97:34608) 2023-07-06T03:52:30.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.906+0000 7fe6bcb9c700 1 -- 172.21.15.97:0/919292819 learned_addr learned my addr 172.21.15.97:0/919292819 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:30.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b77fe700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6b806b360 0x7fe6b80b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247831189 0 0) 0x7fe6b80cff90 con 0x7fe6b80d2210 2023-07-06T03:52:30.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a00035b0 con 0x7fe6b80d2210 2023-07-06T03:52:30.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 496953221 0 0) 0x7fe6a00035b0 con 0x7fe6b80d2210 2023-07-06T03:52:30.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6b806b360 msgr2=0x7fe6b80b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:30.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6b806b360 0x7fe6b80b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 >> v1:172.21.15.146:6789/0 conn(0x7fe6b815c610 legacy=0x7fe6b80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b80b5400 con 0x7fe6b80d2210 2023-07-06T03:52:30.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fe6b00019f0 con 0x7fe6b80d2210 2023-07-06T03:52:30.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 >> v1:172.21.15.97:6789/0 conn(0x7fe6b80d2210 legacy=0x7fe6b80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6a0003a30 0x7fe6a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6a0004380 0x7fe6a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 0x7fe6a000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80d0780 con 0x7fe6a0005050 2023-07-06T03:52:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a000bf50 con 0x7fe6a0004380 2023-07-06T03:52:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a000c110 con 0x7fe6a0003a30 2023-07-06T03:52:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.907+0000 7fe6b77fe700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6b806b360 0x7fe6b80b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6bcb9c700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 0x7fe6a000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6b7fff700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6a0003a30 0x7fe6a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6b77fe700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6a0004380 0x7fe6a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6bcb9c700 1 -- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6a0003a30 msgr2=0x7fe6a0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6bcb9c700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6a0003a30 0x7fe6a0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6bcb9c700 1 -- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6a0004380 msgr2=0x7fe6a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6bcb9c700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6a0004380 0x7fe6a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6bcb9c700 1 -- 172.21.15.97:0/919292819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a00035b0 con 0x7fe6a0005050 2023-07-06T03:52:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6bcb9c700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 0x7fe6a000ba10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000a7d0 tx=0x7fe6a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e23d53f4c7cec7d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:30.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0012600 con 0x7fe6a0005050 2023-07-06T03:52:30.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0012760 con 0x7fe6a0005050 2023-07-06T03:52:30.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.908+0000 7fe6b6ffd700 1 -- 172.21.15.97:0/919292819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0012b10 con 0x7fe6a0005050 2023-07-06T03:52:30.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.909+0000 7fe6be5ff700 1 -- 172.21.15.97:0/919292819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 msgr2=0x7fe6a000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.909+0000 7fe6be5ff700 1 --2- 172.21.15.97:0/919292819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 0x7fe6a000ba10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000a7d0 tx=0x7fe6a0005880 comp rx=0 tx=0).stop 2023-07-06T03:52:30.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.909+0000 7fe6bcb9c700 1 -- 172.21.15.97:0/919292819 reap_dead start 2023-07-06T03:52:30.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.909+0000 7fe6be5ff700 1 -- 172.21.15.97:0/919292819 shutdown_connections 2023-07-06T03:52:30.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.909+0000 7fe6be5ff700 1 -- 172.21.15.97:0/919292819 >> 172.21.15.97:0/919292819 conn(0x7fe6b806a6c0 msgr2=0x7fe6b806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:30.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.909+0000 7fe6be5ff700 1 -- 172.21.15.97:0/919292819 shutdown_connections 2023-07-06T03:52:30.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.909+0000 7fe6be5ff700 1 -- 172.21.15.97:0/919292819 wait complete. 2023-07-06T03:52:30.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.911+0000 7fe6be5ff700 1 Processor -- start 2023-07-06T03:52:30.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.911+0000 7fe6be5ff700 1 -- start start 2023-07-06T03:52:30.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.912+0000 7fe6be5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 0x7fe6b80c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.912+0000 7fe6be5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b80c6860 0x7fe6b815d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.912+0000 7fe6be5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6b815db60 0x7fe6b80d1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.912+0000 7fe6be5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80cb080 con 0x7fe6a0005050 2023-07-06T03:52:30.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.912+0000 7fe6be5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80cb360 con 0x7fe6b80c6860 2023-07-06T03:52:30.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.912+0000 7fe6be5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80cb640 con 0x7fe6b815db60 2023-07-06T03:52:30.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.912+0000 7fe6b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 0x7fe6b80c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.912+0000 7fe6b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 0x7fe6b80c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54816/0 (socket says 172.21.15.97:54816) 2023-07-06T03:52:30.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.912+0000 7fe6b7fff700 1 -- 172.21.15.97:0/340254684 learned_addr learned my addr 172.21.15.97:0/340254684 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:30.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.912+0000 7fe6bcb9c700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6b815db60 0x7fe6b80d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.912+0000 7fe6b77fe700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b80c6860 0x7fe6b815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6b7fff700 1 -- 172.21.15.97:0/340254684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6b815db60 msgr2=0x7fe6b80d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6b7fff700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6b815db60 0x7fe6b80d1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6b7fff700 1 -- 172.21.15.97:0/340254684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b80c6860 msgr2=0x7fe6b815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:30.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6b7fff700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b80c6860 0x7fe6b815d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:30.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6b7fff700 1 -- 172.21.15.97:0/340254684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b80cb920 con 0x7fe6a0005050 2023-07-06T03:52:30.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6b77fe700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b80c6860 0x7fe6b815d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:30.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6b7fff700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 0x7fe6b80c9650 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800f820 tx=0x7fe6a800fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:30.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6b4ff9700 1 -- 172.21.15.97:0/340254684 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80052e0 con 0x7fe6a0005050 2023-07-06T03:52:30.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6b4ff9700 1 -- 172.21.15.97:0/340254684 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a8005460 con 0x7fe6a0005050 2023-07-06T03:52:30.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6be5ff700 1 -- 172.21.15.97:0/340254684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b815e0f0 con 0x7fe6a0005050 2023-07-06T03:52:30.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6b4ff9700 1 -- 172.21.15.97:0/340254684 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80038c0 con 0x7fe6a0005050 2023-07-06T03:52:30.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.913+0000 7fe6be5ff700 1 -- 172.21.15.97:0/340254684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b81f5ea0 con 0x7fe6a0005050 2023-07-06T03:52:30.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.914+0000 7fe6be5ff700 1 -- 172.21.15.97:0/340254684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b804e340 con 0x7fe6a0005050 2023-07-06T03:52:30.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.915+0000 7fe6b4ff9700 1 -- 172.21.15.97:0/340254684 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe6a8003a20 con 0x7fe6a0005050 2023-07-06T03:52:30.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.916+0000 7fe6b4ff9700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6a406ace0 0x7fe6a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:30.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.916+0000 7fe6b4ff9700 1 -- 172.21.15.97:0/340254684 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe6a80890f0 con 0x7fe6a0005050 2023-07-06T03:52:30.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.916+0000 7fe6b77fe700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6a406ace0 0x7fe6a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:30.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.917+0000 7fe6b77fe700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6a406ace0 0x7fe6a406d190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80b1b70 tx=0x7fe6ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:30.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:30.920+0000 7fe6b4ff9700 1 -- 172.21.15.97:0/340254684 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe6a8054450 con 0x7fe6a0005050 2023-07-06T03:52:31.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.118+0000 7fe6be5ff700 1 -- 172.21.15.97:0/340254684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6b80cb1c0 con 0x7fe6a0005050 2023-07-06T03:52:31.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.118+0000 7fe6b4ff9700 1 -- 172.21.15.97:0/340254684 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6b80cb1c0 con 0x7fe6a0005050 2023-07-06T03:52:31.120 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:52:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 -- 172.21.15.97:0/340254684 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6a406ace0 msgr2=0x7fe6a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6a406ace0 0x7fe6a406d190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80b1b70 tx=0x7fe6ac008040 comp rx=0 tx=0).stop 2023-07-06T03:52:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 -- 172.21.15.97:0/340254684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 msgr2=0x7fe6b80c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 0x7fe6b80c9650 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800f820 tx=0x7fe6a800fbe0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 -- 172.21.15.97:0/340254684 shutdown_connections 2023-07-06T03:52:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6a406ace0 0x7fe6a406d190 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe6b815db60 0x7fe6b80d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6a0005050 0x7fe6b80c9650 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 --2- 172.21.15.97:0/340254684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6b80c6860 0x7fe6b815d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 -- 172.21.15.97:0/340254684 >> 172.21.15.97:0/340254684 conn(0x7fe6b806a6c0 msgr2=0x7fe6b806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 -- 172.21.15.97:0/340254684 shutdown_connections 2023-07-06T03:52:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.121+0000 7fe6be5ff700 1 -- 172.21.15.97:0/340254684 wait complete. 2023-07-06T03:52:31.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:52:31.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.336+0000 7ff55fbff700 1 Processor -- start 2023-07-06T03:52:31.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.337+0000 7ff55fbff700 1 -- start start 2023-07-06T03:52:31.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.338+0000 7ff55fbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff55815c610 0x7ff5580b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.338+0000 7ff55fbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5580d2210 0x7ff5580d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.338+0000 7ff55fbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5580cff90 con 0x7ff55806b360 2023-07-06T03:52:31.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.338+0000 7ff55fbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5580d0780 con 0x7ff5580d2210 2023-07-06T03:52:31.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.338+0000 7ff55fbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff55815c9e0 con 0x7ff55815c610 2023-07-06T03:52:31.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.338+0000 7ff55e19c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5580d2210 0x7ff5580d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.338+0000 7ff55e19c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5580d2210 0x7ff5580d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54830/0 (socket says 172.21.15.97:54830) 2023-07-06T03:52:31.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.338+0000 7ff55e19c700 1 -- 172.21.15.97:0/2899256903 learned_addr learned my addr 172.21.15.97:0/2899256903 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:31.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.338+0000 7ff55d99b700 1 --2- 172.21.15.97:0/2899256903 >> v2:172.21.15.146:3300/0 conn(0x7ff55815c610 0x7ff5580b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.339+0000 7ff55e19c700 1 -- 172.21.15.97:0/2899256903 >> v2:172.21.15.146:3300/0 conn(0x7ff55815c610 msgr2=0x7ff5580b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:31.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.339+0000 7ff55e19c700 1 --2- 172.21.15.97:0/2899256903 >> v2:172.21.15.146:3300/0 conn(0x7ff55815c610 0x7ff5580b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.339+0000 7ff55e19c700 1 -- 172.21.15.97:0/2899256903 >> v1:172.21.15.146:6789/0 conn(0x7ff55806b360 legacy=0x7ff5580b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.339+0000 7ff55e19c700 1 -- 172.21.15.97:0/2899256903 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580b5650 con 0x7ff5580d2210 2023-07-06T03:52:31.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.339+0000 7ff55e19c700 1 --2- 172.21.15.97:0/2899256903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580d2210 0x7ff5580d1310 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff55808b000 tx=0x7ff55000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7d52657d6472c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:31.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.339+0000 7ff55c999700 1 -- 172.21.15.97:0/2899256903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55000e070 con 0x7ff5580d2210 2023-07-06T03:52:31.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.339+0000 7ff55c999700 1 -- 172.21.15.97:0/2899256903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55000b8d0 con 0x7ff5580d2210 2023-07-06T03:52:31.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.339+0000 7ff55c999700 1 -- 172.21.15.97:0/2899256903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55000bc80 con 0x7ff5580d2210 2023-07-06T03:52:31.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.340+0000 7ff55fbff700 1 -- 172.21.15.97:0/2899256903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580d2210 msgr2=0x7ff5580d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.340+0000 7ff55fbff700 1 --2- 172.21.15.97:0/2899256903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580d2210 0x7ff5580d1310 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff55808b000 tx=0x7ff55000b330 comp rx=0 tx=0).stop 2023-07-06T03:52:31.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.340+0000 7ff55fbff700 1 -- 172.21.15.97:0/2899256903 shutdown_connections 2023-07-06T03:52:31.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.340+0000 7ff55fbff700 1 --2- 172.21.15.97:0/2899256903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580d2210 0x7ff5580d1310 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.340+0000 7ff55fbff700 1 --2- 172.21.15.97:0/2899256903 >> v2:172.21.15.146:3300/0 conn(0x7ff55815c610 0x7ff5580b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.340+0000 7ff55fbff700 1 -- 172.21.15.97:0/2899256903 >> 172.21.15.97:0/2899256903 conn(0x7ff55806a6c0 msgr2=0x7ff55806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.340+0000 7ff55fbff700 1 -- 172.21.15.97:0/2899256903 shutdown_connections 2023-07-06T03:52:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.340+0000 7ff55fbff700 1 -- 172.21.15.97:0/2899256903 wait complete. 2023-07-06T03:52:31.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.341+0000 7ff55fbff700 1 Processor -- start 2023-07-06T03:52:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.342+0000 7ff55fbff700 1 -- start start 2023-07-06T03:52:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55fbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff55806b360 0x7ff5581f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55fbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5580d2210 0x7ff5581f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55fbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55815c610 0x7ff5580c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55fbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5581f2e10 con 0x7ff55815c610 2023-07-06T03:52:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55fbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5581f6320 con 0x7ff55806b360 2023-07-06T03:52:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55fbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55815f1b0 con 0x7ff5580d2210 2023-07-06T03:52:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55e19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55815c610 0x7ff5580c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55e19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55815c610 0x7ff5580c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54838/0 (socket says 172.21.15.97:54838) 2023-07-06T03:52:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55e19c700 1 -- 172.21.15.97:0/3782654962 learned_addr learned my addr 172.21.15.97:0/3782654962 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55d99b700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff55806b360 0x7ff5581f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55d19a700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5580d2210 0x7ff5581f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55e19c700 1 -- 172.21.15.97:0/3782654962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5580d2210 msgr2=0x7ff5581f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55e19c700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5580d2210 0x7ff5581f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55e19c700 1 -- 172.21.15.97:0/3782654962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff55806b360 msgr2=0x7ff5581f28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55e19c700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff55806b360 0x7ff5581f28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.343+0000 7ff55e19c700 1 -- 172.21.15.97:0/3782654962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff550009b30 con 0x7ff55815c610 2023-07-06T03:52:31.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.344+0000 7ff55e19c700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55815c610 0x7ff5580c9590 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7ff55000b680 tx=0x7ff550004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:31.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.344+0000 7ff54e7fc700 1 -- 172.21.15.97:0/3782654962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5500099b0 con 0x7ff55815c610 2023-07-06T03:52:31.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.344+0000 7ff54e7fc700 1 -- 172.21.15.97:0/3782654962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5500092e0 con 0x7ff55815c610 2023-07-06T03:52:31.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.344+0000 7ff55fbff700 1 -- 172.21.15.97:0/3782654962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55815f470 con 0x7ff55815c610 2023-07-06T03:52:31.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.344+0000 7ff54e7fc700 1 -- 172.21.15.97:0/3782654962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55001c640 con 0x7ff55815c610 2023-07-06T03:52:31.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.344+0000 7ff55fbff700 1 -- 172.21.15.97:0/3782654962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5581f55e0 con 0x7ff55815c610 2023-07-06T03:52:31.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.345+0000 7ff55fbff700 1 -- 172.21.15.97:0/3782654962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5581f5440 con 0x7ff55815c610 2023-07-06T03:52:31.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.346+0000 7ff54e7fc700 1 -- 172.21.15.97:0/3782654962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff55001c7a0 con 0x7ff55815c610 2023-07-06T03:52:31.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.347+0000 7ff54e7fc700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53c06ace0 0x7ff53c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.347+0000 7ff54e7fc700 1 -- 172.21.15.97:0/3782654962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff55008aba0 con 0x7ff55815c610 2023-07-06T03:52:31.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.347+0000 7ff55d99b700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53c06ace0 0x7ff53c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.348+0000 7ff55d99b700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53c06ace0 0x7ff53c06d190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff554009ad0 tx=0x7ff554008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:31.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.351+0000 7ff54e7fc700 1 -- 172.21.15.97:0/3782654962 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff5581f5440 con 0x7ff55815c610 2023-07-06T03:52:31.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.543+0000 7ff55fbff700 1 -- 172.21.15.97:0/3782654962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff5581f5440 con 0x7ff55815c610 2023-07-06T03:52:31.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.544+0000 7ff54e7fc700 1 -- 172.21.15.97:0/3782654962 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v82) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff5581f5440 con 0x7ff55815c610 2023-07-06T03:52:31.546 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:52:31.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 -- 172.21.15.97:0/3782654962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53c06ace0 msgr2=0x7ff53c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53c06ace0 0x7ff53c06d190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff554009ad0 tx=0x7ff554008040 comp rx=0 tx=0).stop 2023-07-06T03:52:31.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 -- 172.21.15.97:0/3782654962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55815c610 msgr2=0x7ff5580c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55815c610 0x7ff5580c9590 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7ff55000b680 tx=0x7ff550004ad0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 -- 172.21.15.97:0/3782654962 shutdown_connections 2023-07-06T03:52:31.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff53c06ace0 0x7ff53c06d190 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55815c610 0x7ff5580c9590 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff55806b360 0x7ff5581f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 --2- 172.21.15.97:0/3782654962 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff5580d2210 0x7ff5581f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 -- 172.21.15.97:0/3782654962 >> 172.21.15.97:0/3782654962 conn(0x7ff55806a6c0 msgr2=0x7ff55815b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:31.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 -- 172.21.15.97:0/3782654962 shutdown_connections 2023-07-06T03:52:31.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.546+0000 7ff55fbff700 1 -- 172.21.15.97:0/3782654962 wait complete. 2023-07-06T03:52:31.566 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T03:52:31.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.759+0000 7f99e5bff700 1 Processor -- start 2023-07-06T03:52:31.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.759+0000 7f99e5bff700 1 -- start start 2023-07-06T03:52:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.760+0000 7f99e5bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f99e015c520 0x7f99e00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.760+0000 7f99e5bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e00cdae0 con 0x7f99e0160180 2023-07-06T03:52:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.760+0000 7f99e5bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e00b6640 con 0x7f99e0160d00 2023-07-06T03:52:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.760+0000 7f99e5bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e00cd8d0 con 0x7f99e015c520 2023-07-06T03:52:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.760+0000 7f99d7fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f99e0160180 0x7f99e01605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34616/0 (socket says 172.21.15.97:34616) 2023-07-06T03:52:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.760+0000 7f99d7fff700 1 -- 172.21.15.97:0/2935519059 learned_addr learned my addr 172.21.15.97:0/2935519059 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.760+0000 7f99df7fe700 1 --2- 172.21.15.97:0/2935519059 >> v2:172.21.15.146:3300/0 conn(0x7f99e015c520 0x7f99e00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874801700 0 0) 0x7f99e00cdae0 con 0x7f99e0160180 2023-07-06T03:52:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99cc0035b0 con 0x7f99e0160180 2023-07-06T03:52:31.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706860490 0 0) 0x7f99e00b6640 con 0x7f99e0160d00 2023-07-06T03:52:31.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99e00cdae0 con 0x7f99e0160d00 2023-07-06T03:52:31.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 600506709 0 0) 0x7f99cc0035b0 con 0x7f99e0160180 2023-07-06T03:52:31.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 >> v2:172.21.15.146:3300/0 conn(0x7f99e015c520 msgr2=0x7f99e00cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:31.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 --2- 172.21.15.97:0/2935519059 >> v2:172.21.15.146:3300/0 conn(0x7f99e015c520 0x7f99e00cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 >> v1:172.21.15.146:6789/0 conn(0x7f99e0160d00 legacy=0x7f99e00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00b6920 con 0x7f99e0160180 2023-07-06T03:52:31.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f99c8001670 con 0x7f99e0160180 2023-07-06T03:52:31.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 >> v1:172.21.15.97:6789/0 conn(0x7f99e0160180 legacy=0x7f99e01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 --2- 172.21.15.97:0/2935519059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc003b90 0x7f99cc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 --2- 172.21.15.97:0/2935519059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99cc004850 0x7f99cc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 --2- 172.21.15.97:0/2935519059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99cc005200 0x7f99cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e00cd8d0 con 0x7f99cc003b90 2023-07-06T03:52:31.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc00c060 con 0x7f99cc004850 2023-07-06T03:52:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc00c220 con 0x7f99cc005200 2023-07-06T03:52:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.761+0000 7f99df7fe700 1 --2- 172.21.15.97:0/2935519059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc003b90 0x7f99cc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99d7fff700 1 --2- 172.21.15.97:0/2935519059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99cc004850 0x7f99cc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99dffff700 1 --2- 172.21.15.97:0/2935519059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99cc005200 0x7f99cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99df7fe700 1 -- 172.21.15.97:0/2935519059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99cc005200 msgr2=0x7f99cc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99df7fe700 1 --2- 172.21.15.97:0/2935519059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99cc005200 0x7f99cc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99df7fe700 1 -- 172.21.15.97:0/2935519059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99cc004850 msgr2=0x7f99cc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:31.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99df7fe700 1 --2- 172.21.15.97:0/2935519059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99cc004850 0x7f99cc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99df7fe700 1 -- 172.21.15.97:0/2935519059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc0035b0 con 0x7f99cc003b90 2023-07-06T03:52:31.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99dffff700 1 -- 172.21.15.97:0/2935519059 reap_dead start 2023-07-06T03:52:31.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99df7fe700 1 --2- 172.21.15.97:0/2935519059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc003b90 0x7f99cc003f80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009b60 tx=0x7f99cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9f0e11a82fc6e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:31.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d000e6c0 con 0x7f99cc003b90 2023-07-06T03:52:31.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d000e820 con 0x7f99cc003b90 2023-07-06T03:52:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.762+0000 7f99deffd700 1 -- 172.21.15.97:0/2935519059 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d000eb90 con 0x7f99cc003b90 2023-07-06T03:52:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.763+0000 7f99e5bff700 1 -- 172.21.15.97:0/2935519059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc003b90 msgr2=0x7f99cc003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.763+0000 7f99e5bff700 1 --2- 172.21.15.97:0/2935519059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc003b90 0x7f99cc003f80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009b60 tx=0x7f99cc005880 comp rx=0 tx=0).stop 2023-07-06T03:52:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.763+0000 7f99e5bff700 1 -- 172.21.15.97:0/2935519059 shutdown_connections 2023-07-06T03:52:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.763+0000 7f99e5bff700 1 --2- 172.21.15.97:0/2935519059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99cc003b90 0x7f99cc003f80 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.763+0000 7f99e5bff700 1 -- 172.21.15.97:0/2935519059 >> 172.21.15.97:0/2935519059 conn(0x7f99e0071790 msgr2=0x7f99e0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:31.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.763+0000 7f99e5bff700 1 -- 172.21.15.97:0/2935519059 shutdown_connections 2023-07-06T03:52:31.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.763+0000 7f99e5bff700 1 -- 172.21.15.97:0/2935519059 wait complete. 2023-07-06T03:52:31.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.765+0000 7f99e5bff700 1 Processor -- start 2023-07-06T03:52:31.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.765+0000 7f99e5bff700 1 -- start start 2023-07-06T03:52:31.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99e5bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99cc003b90 0x7f99e015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99e5bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e014d1a0 0x7f99e015f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99e5bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99e014c890 0x7f99e014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99e5bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e015fa60 con 0x7f99e014d1a0 2023-07-06T03:52:31.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99e5bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e015f090 con 0x7f99cc003b90 2023-07-06T03:52:31.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99e5bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e00b3440 con 0x7f99e014c890 2023-07-06T03:52:31.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99d7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e014d1a0 0x7f99e015f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99d7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e014d1a0 0x7f99e015f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54848/0 (socket says 172.21.15.97:54848) 2023-07-06T03:52:31.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99d7fff700 1 -- 172.21.15.97:0/3928397284 learned_addr learned my addr 172.21.15.97:0/3928397284 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:31.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99df7fe700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99cc003b90 0x7f99e015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99dffff700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99e014c890 0x7f99e014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99d7fff700 1 -- 172.21.15.97:0/3928397284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99e014c890 msgr2=0x7f99e014fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99d7fff700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99e014c890 0x7f99e014fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99d7fff700 1 -- 172.21.15.97:0/3928397284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99cc003b90 msgr2=0x7f99e015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99d7fff700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99cc003b90 0x7f99e015ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.767+0000 7f99d7fff700 1 -- 172.21.15.97:0/3928397284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e015e480 con 0x7f99e014d1a0 2023-07-06T03:52:31.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.768+0000 7f99d7fff700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e014d1a0 0x7f99e015f520 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f99c8000fb0 tx=0x7f99c800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:31.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.768+0000 7f99dcff9700 1 -- 172.21.15.97:0/3928397284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8005e00 con 0x7f99e014d1a0 2023-07-06T03:52:31.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.768+0000 7f99dcff9700 1 -- 172.21.15.97:0/3928397284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c800c7b0 con 0x7f99e014d1a0 2023-07-06T03:52:31.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.768+0000 7f99dcff9700 1 -- 172.21.15.97:0/3928397284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c800cb40 con 0x7f99e014d1a0 2023-07-06T03:52:31.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.768+0000 7f99e5bff700 1 -- 172.21.15.97:0/3928397284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e015e760 con 0x7f99e014d1a0 2023-07-06T03:52:31.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.768+0000 7f99e5bff700 1 -- 172.21.15.97:0/3928397284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e015ed20 con 0x7f99e014d1a0 2023-07-06T03:52:31.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.769+0000 7f99e5bff700 1 -- 172.21.15.97:0/3928397284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e004e340 con 0x7f99e014d1a0 2023-07-06T03:52:31.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.770+0000 7f99dcff9700 1 -- 172.21.15.97:0/3928397284 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f99c800cca0 con 0x7f99e014d1a0 2023-07-06T03:52:31.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.771+0000 7f99dcff9700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99c006ace0 0x7f99c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:31.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.771+0000 7f99dcff9700 1 -- 172.21.15.97:0/3928397284 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f99c8088e30 con 0x7f99e014d1a0 2023-07-06T03:52:31.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.771+0000 7f99df7fe700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99c006ace0 0x7f99c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:31.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.771+0000 7f99df7fe700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99c006ace0 0x7f99c006d190 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f99d00090b0 tx=0x7f99d001f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:31.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.775+0000 7f99dcff9700 1 -- 172.21.15.97:0/3928397284 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f99c8054230 con 0x7f99e014d1a0 2023-07-06T03:52:31.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.979+0000 7f99e5bff700 1 -- 172.21.15.97:0/3928397284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f99e00b4f40 con 0x7f99e014d1a0 2023-07-06T03:52:31.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.980+0000 7f99dcff9700 1 -- 172.21.15.97:0/3928397284 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+126 (secure 0 0 0) 0x7f99c8057850 con 0x7f99e014d1a0 2023-07-06T03:52:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:31.982 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-07-06T03:52:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-07-06T03:52:31.982 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-07-06T03:52:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:52:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 2, 2023-07-06T03:52:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": "rbd" 2023-07-06T03:52:31.983 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:31.983 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-07-06T03:52:31.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.982+0000 7f99e5bff700 1 -- 172.21.15.97:0/3928397284 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99c006ace0 msgr2=0x7f99c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.982+0000 7f99e5bff700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99c006ace0 0x7f99c006d190 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f99d00090b0 tx=0x7f99d001f040 comp rx=0 tx=0).stop 2023-07-06T03:52:31.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.982+0000 7f99e5bff700 1 -- 172.21.15.97:0/3928397284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e014d1a0 msgr2=0x7f99e015f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:31.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.982+0000 7f99e5bff700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e014d1a0 0x7f99e015f520 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f99c8000fb0 tx=0x7f99c800edd0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.982+0000 7f99e5bff700 1 -- 172.21.15.97:0/3928397284 shutdown_connections 2023-07-06T03:52:31.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.982+0000 7f99e5bff700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99c006ace0 0x7f99c006d190 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.982+0000 7f99e5bff700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99e014c890 0x7f99e014fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.982+0000 7f99e5bff700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99cc003b90 0x7f99e015ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.982+0000 7f99e5bff700 1 --2- 172.21.15.97:0/3928397284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e014d1a0 0x7f99e015f520 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:31.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.982+0000 7f99e5bff700 1 -- 172.21.15.97:0/3928397284 >> 172.21.15.97:0/3928397284 conn(0x7f99e0071790 msgr2=0x7f99e00b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.983+0000 7f99e5bff700 1 -- 172.21.15.97:0/3928397284 shutdown_connections 2023-07-06T03:52:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:31.983+0000 7f99e5bff700 1 -- 172.21.15.97:0/3928397284 wait complete. 2023-07-06T03:52:32.002 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T03:52:32.003 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T03:52:32.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.200+0000 7ffac85ff700 1 Processor -- start 2023-07-06T03:52:32.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.200+0000 7ffac85ff700 1 -- start start 2023-07-06T03:52:32.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.201+0000 7ffac85ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffac00b4e70 0x7ffac01670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:32.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.201+0000 7ffac85ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac00cd330 con 0x7ffac00b85f0 2023-07-06T03:52:32.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.201+0000 7ffac85ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac00d02a0 con 0x7ffac006b0c0 2023-07-06T03:52:32.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.201+0000 7ffac85ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac00b89c0 con 0x7ffac00b4e70 2023-07-06T03:52:32.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.202+0000 7ffac6b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffac00b4e70 0x7ffac01670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:32.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.202+0000 7ffac639b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ffac00b85f0 0x7ffac00cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34626/0 (socket says 172.21.15.97:34626) 2023-07-06T03:52:32.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.202+0000 7ffac639b700 1 -- 172.21.15.97:0/4002846286 learned_addr learned my addr 172.21.15.97:0/4002846286 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:32.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.202+0000 7ffac6b9c700 1 -- 172.21.15.97:0/4002846286 >> v1:172.21.15.146:6789/0 conn(0x7ffac006b0c0 legacy=0x7ffac00b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:32.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.202+0000 7ffac6b9c700 1 -- 172.21.15.97:0/4002846286 >> v1:172.21.15.97:6789/0 conn(0x7ffac00b85f0 legacy=0x7ffac00cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:32.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.202+0000 7ffac6b9c700 1 -- 172.21.15.97:0/4002846286 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00d0540 con 0x7ffac00b4e70 2023-07-06T03:52:32.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.202+0000 7ffac5399700 1 -- 172.21.15.97:0/4002846286 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286707193 0 0) 0x7ffac00cd330 con 0x7ffac00b85f0 2023-07-06T03:52:32.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.202+0000 7ffac6b9c700 1 --2- 172.21.15.97:0/4002846286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac00b4e70 0x7ffac01670b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009d90 tx=0x7ffab800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73728ef8fe487da4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:32.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.203+0000 7ffac5399700 1 -- 172.21.15.97:0/4002846286 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab800e070 con 0x7ffac00b4e70 2023-07-06T03:52:32.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.203+0000 7ffac5399700 1 -- 172.21.15.97:0/4002846286 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab800bbd0 con 0x7ffac00b4e70 2023-07-06T03:52:32.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.203+0000 7ffac5399700 1 -- 172.21.15.97:0/4002846286 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8012430 con 0x7ffac00b4e70 2023-07-06T03:52:32.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.203+0000 7ffac85ff700 1 -- 172.21.15.97:0/4002846286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac00b4e70 msgr2=0x7ffac01670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:32.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.203+0000 7ffac85ff700 1 --2- 172.21.15.97:0/4002846286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac00b4e70 0x7ffac01670b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009d90 tx=0x7ffab800b330 comp rx=0 tx=0).stop 2023-07-06T03:52:32.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.203+0000 7ffac85ff700 1 -- 172.21.15.97:0/4002846286 shutdown_connections 2023-07-06T03:52:32.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.203+0000 7ffac85ff700 1 --2- 172.21.15.97:0/4002846286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac00b4e70 0x7ffac01670b0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.203+0000 7ffac85ff700 1 -- 172.21.15.97:0/4002846286 >> 172.21.15.97:0/4002846286 conn(0x7ffac006a5b0 msgr2=0x7ffac006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.203+0000 7ffac85ff700 1 -- 172.21.15.97:0/4002846286 shutdown_connections 2023-07-06T03:52:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.203+0000 7ffac85ff700 1 -- 172.21.15.97:0/4002846286 wait complete. 2023-07-06T03:52:32.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.205+0000 7ffac85ff700 1 Processor -- start 2023-07-06T03:52:32.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.205+0000 7ffac85ff700 1 -- start start 2023-07-06T03:52:32.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.206+0000 7ffac85ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac006b0c0 0x7ffac0155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:32.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.206+0000 7ffac85ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac00b4e70 0x7ffac01f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:32.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.206+0000 7ffac85ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac00b85f0 0x7ffac0157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:32.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.206+0000 7ffac85ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac006b510 con 0x7ffac006b0c0 2023-07-06T03:52:32.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.206+0000 7ffac85ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac0159940 con 0x7ffac00b4e70 2023-07-06T03:52:32.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.206+0000 7ffac85ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac015e7c0 con 0x7ffac00b85f0 2023-07-06T03:52:32.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.206+0000 7ffac639b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac006b0c0 0x7ffac0155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:32.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.206+0000 7ffac639b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac006b0c0 0x7ffac0155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54856/0 (socket says 172.21.15.97:54856) 2023-07-06T03:52:32.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.206+0000 7ffac639b700 1 -- 172.21.15.97:0/780408788 learned_addr learned my addr 172.21.15.97:0/780408788 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:32.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.206+0000 7ffac6b9c700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac00b85f0 0x7ffac0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:32.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.206+0000 7ffac5b9a700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac00b4e70 0x7ffac01f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:32.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffac639b700 1 -- 172.21.15.97:0/780408788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac00b85f0 msgr2=0x7ffac0157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:32.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffac639b700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac00b85f0 0x7ffac0157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:32.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffac639b700 1 -- 172.21.15.97:0/780408788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac00b4e70 msgr2=0x7ffac01f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:32.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffac639b700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac00b4e70 0x7ffac01f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:32.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffac639b700 1 -- 172.21.15.97:0/780408788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab8009a70 con 0x7ffac006b0c0 2023-07-06T03:52:32.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffac6b9c700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac00b85f0 0x7ffac0157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:32.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffac5b9a700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac00b4e70 0x7ffac01f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:32.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffac639b700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac006b0c0 0x7ffac0155780 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c520 tx=0x7ffabc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:32.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffab6ffd700 1 -- 172.21.15.97:0/780408788 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc004c80 con 0x7ffac006b0c0 2023-07-06T03:52:32.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffab6ffd700 1 -- 172.21.15.97:0/780408788 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc004e20 con 0x7ffac006b0c0 2023-07-06T03:52:32.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffab6ffd700 1 -- 172.21.15.97:0/780408788 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc007310 con 0x7ffac006b0c0 2023-07-06T03:52:32.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffac85ff700 1 -- 172.21.15.97:0/780408788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac00b77d0 con 0x7ffac006b0c0 2023-07-06T03:52:32.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.207+0000 7ffac85ff700 1 -- 172.21.15.97:0/780408788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac0159e10 con 0x7ffac006b0c0 2023-07-06T03:52:32.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.208+0000 7ffac85ff700 1 -- 172.21.15.97:0/780408788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac004e340 con 0x7ffac006b0c0 2023-07-06T03:52:32.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.209+0000 7ffab6ffd700 1 -- 172.21.15.97:0/780408788 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffabc007470 con 0x7ffac006b0c0 2023-07-06T03:52:32.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.210+0000 7ffab6ffd700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa406ace0 0x7ffaa406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:32.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.210+0000 7ffab6ffd700 1 -- 172.21.15.97:0/780408788 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ffabc088ef0 con 0x7ffac006b0c0 2023-07-06T03:52:32.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.210+0000 7ffac5b9a700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa406ace0 0x7ffaa406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:32.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.211+0000 7ffac5b9a700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa406ace0 0x7ffaa406d190 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffac0156ba0 tx=0x7ffab0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:32.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.214+0000 7ffab6ffd700 1 -- 172.21.15.97:0/780408788 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffabc0541c0 con 0x7ffac006b0c0 2023-07-06T03:52:32.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.409+0000 7ffac85ff700 1 -- 172.21.15.97:0/780408788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ffac015eaa0 con 0x7ffac006b0c0 2023-07-06T03:52:32.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.945+0000 7ffab6ffd700 1 -- 172.21.15.97:0/780408788 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7ffabc0577e0 con 0x7ffac006b0c0 2023-07-06T03:52:32.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 -- 172.21.15.97:0/780408788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa406ace0 msgr2=0x7ffaa406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:32.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa406ace0 0x7ffaa406d190 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffac0156ba0 tx=0x7ffab0009040 comp rx=0 tx=0).stop 2023-07-06T03:52:32.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 -- 172.21.15.97:0/780408788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac006b0c0 msgr2=0x7ffac0155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:32.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac006b0c0 0x7ffac0155780 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c520 tx=0x7ffabc00c8e0 comp rx=0 tx=0).stop 2023-07-06T03:52:32.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 -- 172.21.15.97:0/780408788 shutdown_connections 2023-07-06T03:52:32.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffaa406ace0 0x7ffaa406d190 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:32.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffac00b85f0 0x7ffac0157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:32.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffac006b0c0 0x7ffac0155780 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:32.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 --2- 172.21.15.97:0/780408788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac00b4e70 0x7ffac01f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:32.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 -- 172.21.15.97:0/780408788 >> 172.21.15.97:0/780408788 conn(0x7ffac006a5b0 msgr2=0x7ffac006a980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:32.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 -- 172.21.15.97:0/780408788 shutdown_connections 2023-07-06T03:52:32.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:32.951+0000 7ffac85ff700 1 -- 172.21.15.97:0/780408788 wait complete. 2023-07-06T03:52:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-07-06T03:52:32.974 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T03:52:33.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.176+0000 7f7cd0bff700 1 Processor -- start 2023-07-06T03:52:33.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.176+0000 7f7cd0bff700 1 -- start start 2023-07-06T03:52:33.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.177+0000 7f7cd0bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ccc0b4ea0 0x7f7ccc0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:33.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.177+0000 7f7cd0bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc160a50 0x7f7ccc0d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:33.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.177+0000 7f7cd0bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ccc160eb0 con 0x7f7ccc0b8590 2023-07-06T03:52:33.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.177+0000 7f7cd0bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc0d0c10 con 0x7f7ccc0b4ea0 2023-07-06T03:52:33.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.177+0000 7f7cd0bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0d08e0 con 0x7f7ccc160a50 2023-07-06T03:52:33.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.177+0000 7f7cc3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ccc0b4ea0 0x7f7ccc0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:33.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.177+0000 7f7cc3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ccc0b4ea0 0x7f7ccc0cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54874/0 (socket says 172.21.15.97:54874) 2023-07-06T03:52:33.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.177+0000 7f7cc3fff700 1 -- 172.21.15.97:0/3496839712 learned_addr learned my addr 172.21.15.97:0/3496839712 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:33.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.177+0000 7f7ccad9d700 1 --2- 172.21.15.97:0/3496839712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc160a50 0x7f7ccc0d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:33.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.178+0000 7f7cc3fff700 1 -- 172.21.15.97:0/3496839712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc160a50 msgr2=0x7f7ccc0d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:33.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.178+0000 7f7cc3fff700 1 --2- 172.21.15.97:0/3496839712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc160a50 0x7f7ccc0d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:33.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.178+0000 7f7cc3fff700 1 -- 172.21.15.97:0/3496839712 >> v1:172.21.15.146:6789/0 conn(0x7f7ccc0b8590 legacy=0x7f7ccc0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:33.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.178+0000 7f7cc3fff700 1 -- 172.21.15.97:0/3496839712 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc0b2560 con 0x7f7ccc0b4ea0 2023-07-06T03:52:33.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.178+0000 7f7cc3fff700 1 --2- 172.21.15.97:0/3496839712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc0b4ea0 0x7f7ccc0cef60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40098f0 tx=0x7f7cb4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e7859c0c4cc479 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:33.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.178+0000 7f7cc9d9b700 1 -- 172.21.15.97:0/3496839712 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4011830 con 0x7f7ccc0b4ea0 2023-07-06T03:52:33.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.178+0000 7f7cc9d9b700 1 -- 172.21.15.97:0/3496839712 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb4011990 con 0x7f7ccc0b4ea0 2023-07-06T03:52:33.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.178+0000 7f7cc9d9b700 1 -- 172.21.15.97:0/3496839712 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4011d20 con 0x7f7ccc0b4ea0 2023-07-06T03:52:33.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.179+0000 7f7cd0bff700 1 -- 172.21.15.97:0/3496839712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc0b4ea0 msgr2=0x7f7ccc0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:33.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.179+0000 7f7cd0bff700 1 --2- 172.21.15.97:0/3496839712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc0b4ea0 0x7f7ccc0cef60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40098f0 tx=0x7f7cb4009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:33.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.179+0000 7f7cd0bff700 1 -- 172.21.15.97:0/3496839712 shutdown_connections 2023-07-06T03:52:33.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.179+0000 7f7cd0bff700 1 --2- 172.21.15.97:0/3496839712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc160a50 0x7f7ccc0d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:33.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.179+0000 7f7cd0bff700 1 --2- 172.21.15.97:0/3496839712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc0b4ea0 0x7f7ccc0cef60 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:33.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.179+0000 7f7cd0bff700 1 -- 172.21.15.97:0/3496839712 >> 172.21.15.97:0/3496839712 conn(0x7f7ccc06b290 msgr2=0x7f7ccc0cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:33.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.179+0000 7f7cd0bff700 1 -- 172.21.15.97:0/3496839712 shutdown_connections 2023-07-06T03:52:33.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.179+0000 7f7cd0bff700 1 -- 172.21.15.97:0/3496839712 wait complete. 2023-07-06T03:52:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.181+0000 7f7cd0bff700 1 Processor -- start 2023-07-06T03:52:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.181+0000 7f7cd0bff700 1 -- start start 2023-07-06T03:52:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cd0bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc0b4ea0 0x7f7ccc1e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cd0bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc0b8590 0x7f7ccc1590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cd0bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ccc160a50 0x7f7ccc0cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cd0bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0c97b0 con 0x7f7ccc0b8590 2023-07-06T03:52:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cd0bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0cac10 con 0x7f7ccc160a50 2023-07-06T03:52:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cd0bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0c9a00 con 0x7f7ccc0b4ea0 2023-07-06T03:52:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cc3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc0b8590 0x7f7ccc1590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cc3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc0b8590 0x7f7ccc1590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54878/0 (socket says 172.21.15.97:54878) 2023-07-06T03:52:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cc3fff700 1 -- 172.21.15.97:0/4114723341 learned_addr learned my addr 172.21.15.97:0/4114723341 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cc3fff700 1 -- 172.21.15.97:0/4114723341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc0b4ea0 msgr2=0x7f7ccc1e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cc3fff700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc0b4ea0 0x7f7ccc1e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cc3fff700 1 -- 172.21.15.97:0/4114723341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ccc160a50 msgr2=0x7f7ccc0cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cc3fff700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ccc160a50 0x7f7ccc0cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:33.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cc3fff700 1 -- 172.21.15.97:0/4114723341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb4009550 con 0x7f7ccc0b8590 2023-07-06T03:52:33.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.182+0000 7f7cc3fff700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc0b8590 0x7f7ccc1590c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4014040 tx=0x7f7cb4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:33.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.183+0000 7f7cc2ffd700 1 -- 172.21.15.97:0/4114723341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb40036a0 con 0x7f7ccc0b8590 2023-07-06T03:52:33.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.183+0000 7f7cc2ffd700 1 -- 172.21.15.97:0/4114723341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb401e070 con 0x7f7ccc0b8590 2023-07-06T03:52:33.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.183+0000 7f7cc2ffd700 1 -- 172.21.15.97:0/4114723341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4003a60 con 0x7f7ccc0b8590 2023-07-06T03:52:33.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.183+0000 7f7cd0bff700 1 -- 172.21.15.97:0/4114723341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc0cae60 con 0x7f7ccc0b8590 2023-07-06T03:52:33.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.183+0000 7f7cd0bff700 1 -- 172.21.15.97:0/4114723341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc0cb420 con 0x7f7ccc0b8590 2023-07-06T03:52:33.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.184+0000 7f7cd0bff700 1 -- 172.21.15.97:0/4114723341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc04e340 con 0x7f7ccc0b8590 2023-07-06T03:52:33.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.185+0000 7f7cc2ffd700 1 -- 172.21.15.97:0/4114723341 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7cb4003bc0 con 0x7f7ccc0b8590 2023-07-06T03:52:33.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.186+0000 7f7cc2ffd700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cac06ace0 0x7f7cac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:33.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.186+0000 7f7cc2ffd700 1 -- 172.21.15.97:0/4114723341 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f7cb408a260 con 0x7f7ccc0b8590 2023-07-06T03:52:33.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.186+0000 7f7cca59c700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cac06ace0 0x7f7cac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:33.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.187+0000 7f7cca59c700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cac06ace0 0x7f7cac06d190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc000f60 tx=0x7f7cbc007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:33.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.191+0000 7f7cc2ffd700 1 -- 172.21.15.97:0/4114723341 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7cb40554e0 con 0x7f7ccc0b8590 2023-07-06T03:52:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.388+0000 7f7cd0bff700 1 -- 172.21.15.97:0/4114723341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7ccc0b2340 con 0x7f7ccc0b8590 2023-07-06T03:52:33.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.955+0000 7f7cc2ffd700 1 -- 172.21.15.97:0/4114723341 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7cb4058b00 con 0x7f7ccc0b8590 2023-07-06T03:52:33.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.963+0000 7f7cd0bff700 1 -- 172.21.15.97:0/4114723341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cac06ace0 msgr2=0x7f7cac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:33.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.963+0000 7f7cd0bff700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cac06ace0 0x7f7cac06d190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc000f60 tx=0x7f7cbc007a40 comp rx=0 tx=0).stop 2023-07-06T03:52:33.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.963+0000 7f7cd0bff700 1 -- 172.21.15.97:0/4114723341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc0b8590 msgr2=0x7f7ccc1590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:33.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.963+0000 7f7cd0bff700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc0b8590 0x7f7ccc1590c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4014040 tx=0x7f7cb4004750 comp rx=0 tx=0).stop 2023-07-06T03:52:33.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.965+0000 7f7cd0bff700 1 -- 172.21.15.97:0/4114723341 shutdown_connections 2023-07-06T03:52:33.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.965+0000 7f7cd0bff700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cac06ace0 0x7f7cac06d190 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:33.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.965+0000 7f7cd0bff700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ccc160a50 0x7f7ccc0cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:33.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.965+0000 7f7cd0bff700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ccc0b4ea0 0x7f7ccc1e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:33.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.965+0000 7f7cd0bff700 1 --2- 172.21.15.97:0/4114723341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ccc0b8590 0x7f7ccc1590c0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:33.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.965+0000 7f7cd0bff700 1 -- 172.21.15.97:0/4114723341 >> 172.21.15.97:0/4114723341 conn(0x7f7ccc06b290 msgr2=0x7f7ccc0b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:33.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.965+0000 7f7cd0bff700 1 -- 172.21.15.97:0/4114723341 shutdown_connections 2023-07-06T03:52:33.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:33.966+0000 7f7cd0bff700 1 -- 172.21.15.97:0/4114723341 wait complete. 2023-07-06T03:52:33.975 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-07-06T03:52:33.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T03:52:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.189+0000 7f74c9bff700 1 Processor -- start 2023-07-06T03:52:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.189+0000 7f74c9bff700 1 -- start start 2023-07-06T03:52:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.190+0000 7f74c9bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f74c415c590 0x7f74c415d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.190+0000 7f74c9bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c40ccc30 con 0x7f74c40719c0 2023-07-06T03:52:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.190+0000 7f74c9bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c40ccf10 con 0x7f74c415c1c0 2023-07-06T03:52:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.190+0000 7f74c9bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c40cd1f0 con 0x7f74c415c590 2023-07-06T03:52:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.191+0000 7f74c3fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f74c40719c0 0x7f74c406a060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34634/0 (socket says 172.21.15.97:34634) 2023-07-06T03:52:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.191+0000 7f74c3fff700 1 -- 172.21.15.97:0/2488487690 learned_addr learned my addr 172.21.15.97:0/2488487690 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.191+0000 7f74c2ffd700 1 --2- 172.21.15.97:0/2488487690 >> v2:172.21.15.146:3300/0 conn(0x7f74c415c590 0x7f74c415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.191+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434332526 0 0) 0x7f74c40ccc30 con 0x7f74c40719c0 2023-07-06T03:52:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.191+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ac0035b0 con 0x7f74c40719c0 2023-07-06T03:52:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.191+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3403418404 0 0) 0x7f74ac0035b0 con 0x7f74c40719c0 2023-07-06T03:52:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.191+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 >> v2:172.21.15.146:3300/0 conn(0x7f74c415c590 msgr2=0x7f74c415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.191+0000 7f74c27fc700 1 --2- 172.21.15.97:0/2488487690 >> v2:172.21.15.146:3300/0 conn(0x7f74c415c590 0x7f74c415d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.191+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 >> v1:172.21.15.146:6789/0 conn(0x7f74c415c1c0 legacy=0x7f74c40cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.191+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c415cb00 con 0x7f74c40719c0 2023-07-06T03:52:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.191+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f74bc0019f0 con 0x7f74c40719c0 2023-07-06T03:52:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 >> v1:172.21.15.97:6789/0 conn(0x7f74c40719c0 legacy=0x7f74c406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:34.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c27fc700 1 --2- 172.21.15.97:0/2488487690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74ac003a00 0x7f74ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:34.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c27fc700 1 --2- 172.21.15.97:0/2488487690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74ac004850 0x7f74ac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:34.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c27fc700 1 --2- 172.21.15.97:0/2488487690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ac005200 0x7f74ac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:34.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40cd1f0 con 0x7f74ac004850 2023-07-06T03:52:34.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac00c080 con 0x7f74ac005200 2023-07-06T03:52:34.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac00c240 con 0x7f74ac003a00 2023-07-06T03:52:34.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c2ffd700 1 --2- 172.21.15.97:0/2488487690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74ac004850 0x7f74ac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:34.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c37fe700 1 --2- 172.21.15.97:0/2488487690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74ac003a00 0x7f74ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:34.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c3fff700 1 --2- 172.21.15.97:0/2488487690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ac005200 0x7f74ac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:34.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c2ffd700 1 -- 172.21.15.97:0/2488487690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74ac003a00 msgr2=0x7f74ac003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:34.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c2ffd700 1 --2- 172.21.15.97:0/2488487690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74ac003a00 0x7f74ac003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:34.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c2ffd700 1 -- 172.21.15.97:0/2488487690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ac005200 msgr2=0x7f74ac00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:34.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c2ffd700 1 --2- 172.21.15.97:0/2488487690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ac005200 0x7f74ac00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:34.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c2ffd700 1 -- 172.21.15.97:0/2488487690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac0035b0 con 0x7f74ac004850 2023-07-06T03:52:34.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.192+0000 7f74c3fff700 1 -- 172.21.15.97:0/2488487690 reap_dead start 2023-07-06T03:52:34.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.193+0000 7f74c2ffd700 1 --2- 172.21.15.97:0/2488487690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74ac004850 0x7f74ac004cc0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009b30 tx=0x7f74ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5454b361ba1f30 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.193+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b800e6c0 con 0x7f74ac004850 2023-07-06T03:52:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.193+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b800e820 con 0x7f74ac004850 2023-07-06T03:52:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.193+0000 7f74c27fc700 1 -- 172.21.15.97:0/2488487690 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b800eb90 con 0x7f74ac004850 2023-07-06T03:52:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.193+0000 7f74c9bff700 1 -- 172.21.15.97:0/2488487690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74ac004850 msgr2=0x7f74ac004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.193+0000 7f74c9bff700 1 --2- 172.21.15.97:0/2488487690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74ac004850 0x7f74ac004cc0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009b30 tx=0x7f74ac005880 comp rx=0 tx=0).stop 2023-07-06T03:52:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.193+0000 7f74c9bff700 1 -- 172.21.15.97:0/2488487690 shutdown_connections 2023-07-06T03:52:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.193+0000 7f74c9bff700 1 --2- 172.21.15.97:0/2488487690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74ac004850 0x7f74ac004cc0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:34.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.193+0000 7f74c9bff700 1 -- 172.21.15.97:0/2488487690 >> 172.21.15.97:0/2488487690 conn(0x7f74c40b1650 msgr2=0x7f74c40b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:34.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.193+0000 7f74c9bff700 1 -- 172.21.15.97:0/2488487690 shutdown_connections 2023-07-06T03:52:34.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.194+0000 7f74c9bff700 1 -- 172.21.15.97:0/2488487690 wait complete. 2023-07-06T03:52:34.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.195+0000 7f74c9bff700 1 Processor -- start 2023-07-06T03:52:34.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.195+0000 7f74c9bff700 1 -- start start 2023-07-06T03:52:34.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.196+0000 7f74c9bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ac004850 0x7f74c41552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:34.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.196+0000 7f74c9bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c40b52b0 0x7f74c415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:34.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.196+0000 7f74c9bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74c415e800 0x7f74c415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:34.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.196+0000 7f74c9bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40b3f30 con 0x7f74c40b52b0 2023-07-06T03:52:34.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.196+0000 7f74c9bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40b4210 con 0x7f74ac004850 2023-07-06T03:52:34.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.196+0000 7f74c9bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40b44f0 con 0x7f74c415e800 2023-07-06T03:52:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.196+0000 7f74c2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c40b52b0 0x7f74c415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.196+0000 7f74c2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c40b52b0 0x7f74c415e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54892/0 (socket says 172.21.15.97:54892) 2023-07-06T03:52:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74c2ffd700 1 -- 172.21.15.97:0/1586748059 learned_addr learned my addr 172.21.15.97:0/1586748059 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74c37fe700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ac004850 0x7f74c41552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74c3fff700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74c415e800 0x7f74c415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74c2ffd700 1 -- 172.21.15.97:0/1586748059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74c415e800 msgr2=0x7f74c415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:34.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74c2ffd700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74c415e800 0x7f74c415a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:34.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74c2ffd700 1 -- 172.21.15.97:0/1586748059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ac004850 msgr2=0x7f74c41552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:34.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74c2ffd700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ac004850 0x7f74c41552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:34.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74c2ffd700 1 -- 172.21.15.97:0/1586748059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c40b47d0 con 0x7f74c40b52b0 2023-07-06T03:52:34.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74c2ffd700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c40b52b0 0x7f74c415e2c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00c050 tx=0x7f74b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:34.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74abfff700 1 -- 172.21.15.97:0/1586748059 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b800e6c0 con 0x7f74c40b52b0 2023-07-06T03:52:34.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74abfff700 1 -- 172.21.15.97:0/1586748059 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8013aa0 con 0x7f74c40b52b0 2023-07-06T03:52:34.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74abfff700 1 -- 172.21.15.97:0/1586748059 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8013e50 con 0x7f74c40b52b0 2023-07-06T03:52:34.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.197+0000 7f74c9bff700 1 -- 172.21.15.97:0/1586748059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c40b4ab0 con 0x7f74c40b52b0 2023-07-06T03:52:34.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.198+0000 7f74c9bff700 1 -- 172.21.15.97:0/1586748059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c40b5720 con 0x7f74c40b52b0 2023-07-06T03:52:34.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.199+0000 7f74c9bff700 1 -- 172.21.15.97:0/1586748059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c40b5100 con 0x7f74c40b52b0 2023-07-06T03:52:34.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.200+0000 7f74abfff700 1 -- 172.21.15.97:0/1586748059 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f74b8024070 con 0x7f74c40b52b0 2023-07-06T03:52:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.200+0000 7f74abfff700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74b006ace0 0x7f74b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.201+0000 7f74abfff700 1 -- 172.21.15.97:0/1586748059 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f74b808a830 con 0x7f74c40b52b0 2023-07-06T03:52:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.201+0000 7f74c37fe700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74b006ace0 0x7f74b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.201+0000 7f74c37fe700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74b006ace0 0x7f74b006d190 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f74c415b760 tx=0x7f74b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.206+0000 7f74abfff700 1 -- 172.21.15.97:0/1586748059 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f74b8055af0 con 0x7f74c40b52b0 2023-07-06T03:52:34.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.446+0000 7f74c9bff700 1 -- 172.21.15.97:0/1586748059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f74c415ebd0 con 0x7f74c40b52b0 2023-07-06T03:52:34.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.961+0000 7f74abfff700 1 -- 172.21.15.97:0/1586748059 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v52) v1 ==== 174+0+0 (secure 0 0 0) 0x7f74b8059110 con 0x7f74c40b52b0 2023-07-06T03:52:34.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.972+0000 7f74c9bff700 1 -- 172.21.15.97:0/1586748059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74b006ace0 msgr2=0x7f74b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:34.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.972+0000 7f74c9bff700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74b006ace0 0x7f74b006d190 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f74c415b760 tx=0x7f74b4009040 comp rx=0 tx=0).stop 2023-07-06T03:52:34.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.972+0000 7f74c9bff700 1 -- 172.21.15.97:0/1586748059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c40b52b0 msgr2=0x7f74c415e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:34.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.972+0000 7f74c9bff700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c40b52b0 0x7f74c415e2c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00c050 tx=0x7f74b80047c0 comp rx=0 tx=0).stop 2023-07-06T03:52:34.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.973+0000 7f74c9bff700 1 -- 172.21.15.97:0/1586748059 shutdown_connections 2023-07-06T03:52:34.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.973+0000 7f74c9bff700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74b006ace0 0x7f74b006d190 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:34.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.973+0000 7f74c9bff700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f74c415e800 0x7f74c415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:34.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.973+0000 7f74c9bff700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ac004850 0x7f74c41552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:34.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.973+0000 7f74c9bff700 1 --2- 172.21.15.97:0/1586748059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c40b52b0 0x7f74c415e2c0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:34.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.973+0000 7f74c9bff700 1 -- 172.21.15.97:0/1586748059 >> 172.21.15.97:0/1586748059 conn(0x7f74c40b1650 msgr2=0x7f74c40c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.977+0000 7f74c9bff700 1 -- 172.21.15.97:0/1586748059 shutdown_connections 2023-07-06T03:52:34.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:34.977+0000 7f74c9bff700 1 -- 172.21.15.97:0/1586748059 wait complete. 2023-07-06T03:52:34.985 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 13 and data pool 14 2023-07-06T03:52:35.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:52:35.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.200+0000 7fe704dff700 1 Processor -- start 2023-07-06T03:52:35.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.200+0000 7fe704dff700 1 -- start start 2023-07-06T03:52:35.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.201+0000 7fe704dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe700158f20 0x7fe7000caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:35.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.201+0000 7fe704dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70015cc40 con 0x7fe7001592f0 2023-07-06T03:52:35.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.201+0000 7fe704dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70015cf20 con 0x7fe7000b0c70 2023-07-06T03:52:35.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.201+0000 7fe704dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70015d200 con 0x7fe700158f20 2023-07-06T03:52:35.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.202+0000 7fe6fe59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe700158f20 0x7fe7000caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:35.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.202+0000 7fe6fe59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe700158f20 0x7fe7000caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54908/0 (socket says 172.21.15.97:54908) 2023-07-06T03:52:35.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.202+0000 7fe6fe59c700 1 -- 172.21.15.97:0/1551640562 learned_addr learned my addr 172.21.15.97:0/1551640562 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:35.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.202+0000 7fe6fdd9b700 1 --1- 172.21.15.97:0/1551640562 >> v1:172.21.15.97:6789/0 conn(0x7fe7001592f0 0x7fe7000b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34640/0 (socket says 172.21.15.97:34640) 2023-07-06T03:52:35.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.202+0000 7fe6fe59c700 1 -- 172.21.15.97:0/1551640562 >> v1:172.21.15.146:6789/0 conn(0x7fe7000b0c70 legacy=0x7fe70006bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.202+0000 7fe6fe59c700 1 -- 172.21.15.97:0/1551640562 >> v1:172.21.15.97:6789/0 conn(0x7fe7001592f0 legacy=0x7fe7000b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.202+0000 7fe6fe59c700 1 -- 172.21.15.97:0/1551640562 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70015d4e0 con 0x7fe700158f20 2023-07-06T03:52:35.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.202+0000 7fe6fd59a700 1 -- 172.21.15.97:0/1551640562 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146607855 0 0) 0x7fe70015cc40 con 0x7fe7001592f0 2023-07-06T03:52:35.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.202+0000 7fe6fe59c700 1 --2- 172.21.15.97:0/1551640562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe700158f20 0x7fe7000caac0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000bb90 tx=0x7fe6f000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5295cb8886bc97 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:35.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.203+0000 7fe6fd59a700 1 -- 172.21.15.97:0/1551640562 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000dd50 con 0x7fe700158f20 2023-07-06T03:52:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.203+0000 7fe6fd59a700 1 -- 172.21.15.97:0/1551640562 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f0010470 con 0x7fe700158f20 2023-07-06T03:52:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.203+0000 7fe6fd59a700 1 -- 172.21.15.97:0/1551640562 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0010800 con 0x7fe700158f20 2023-07-06T03:52:35.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.203+0000 7fe704dff700 1 -- 172.21.15.97:0/1551640562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe700158f20 msgr2=0x7fe7000caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.203+0000 7fe704dff700 1 --2- 172.21.15.97:0/1551640562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe700158f20 0x7fe7000caac0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000bb90 tx=0x7fe6f000bea0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.203+0000 7fe704dff700 1 -- 172.21.15.97:0/1551640562 shutdown_connections 2023-07-06T03:52:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.203+0000 7fe704dff700 1 --2- 172.21.15.97:0/1551640562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe700158f20 0x7fe7000caac0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.203+0000 7fe704dff700 1 -- 172.21.15.97:0/1551640562 >> 172.21.15.97:0/1551640562 conn(0x7fe7000ce8c0 msgr2=0x7fe7000cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.203+0000 7fe704dff700 1 -- 172.21.15.97:0/1551640562 shutdown_connections 2023-07-06T03:52:35.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.203+0000 7fe704dff700 1 -- 172.21.15.97:0/1551640562 wait complete. 2023-07-06T03:52:35.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.206+0000 7fe704dff700 1 Processor -- start 2023-07-06T03:52:35.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.206+0000 7fe704dff700 1 -- start start 2023-07-06T03:52:35.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.207+0000 7fe704dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7000b0c70 0x7fe7000b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:35.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.207+0000 7fe704dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe700158f20 0x7fe7000b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:35.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.207+0000 7fe704dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7001592f0 0x7fe70015be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:35.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.207+0000 7fe704dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000b36c0 con 0x7fe700158f20 2023-07-06T03:52:35.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.207+0000 7fe704dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000b2050 con 0x7fe7001592f0 2023-07-06T03:52:35.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.207+0000 7fe704dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000b3110 con 0x7fe7000b0c70 2023-07-06T03:52:35.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.207+0000 7fe6fdd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe700158f20 0x7fe7000b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:35.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.207+0000 7fe6fdd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe700158f20 0x7fe7000b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54922/0 (socket says 172.21.15.97:54922) 2023-07-06T03:52:35.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.207+0000 7fe6fdd9b700 1 -- 172.21.15.97:0/2057339256 learned_addr learned my addr 172.21.15.97:0/2057339256 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:35.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6fe59c700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7000b0c70 0x7fe7000b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:35.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6fed9d700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7001592f0 0x7fe70015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:35.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6fdd9b700 1 -- 172.21.15.97:0/2057339256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7000b0c70 msgr2=0x7fe7000b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6fdd9b700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7000b0c70 0x7fe7000b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6fdd9b700 1 -- 172.21.15.97:0/2057339256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7001592f0 msgr2=0x7fe70015be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6fdd9b700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7001592f0 0x7fe70015be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6fdd9b700 1 -- 172.21.15.97:0/2057339256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f000b7f0 con 0x7fe700158f20 2023-07-06T03:52:35.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6fe59c700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7000b0c70 0x7fe7000b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:35.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6fed9d700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7001592f0 0x7fe70015be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:35.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6fdd9b700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe700158f20 0x7fe7000b2bd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c550 tx=0x7fe6f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:35.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6eeffd700 1 -- 172.21.15.97:0/2057339256 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4004c80 con 0x7fe700158f20 2023-07-06T03:52:35.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6eeffd700 1 -- 172.21.15.97:0/2057339256 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f4004de0 con 0x7fe700158f20 2023-07-06T03:52:35.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe6eeffd700 1 -- 172.21.15.97:0/2057339256 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4007310 con 0x7fe700158f20 2023-07-06T03:52:35.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe704dff700 1 -- 172.21.15.97:0/2057339256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7000b33e0 con 0x7fe700158f20 2023-07-06T03:52:35.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.208+0000 7fe704dff700 1 -- 172.21.15.97:0/2057339256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7000b1ce0 con 0x7fe700158f20 2023-07-06T03:52:35.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.209+0000 7fe704dff700 1 -- 172.21.15.97:0/2057339256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7000b2a20 con 0x7fe700158f20 2023-07-06T03:52:35.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.210+0000 7fe6eeffd700 1 -- 172.21.15.97:0/2057339256 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe6f4007470 con 0x7fe700158f20 2023-07-06T03:52:35.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.211+0000 7fe6eeffd700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6e006ace0 0x7fe6e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:35.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.211+0000 7fe6eeffd700 1 -- 172.21.15.97:0/2057339256 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe6f4089160 con 0x7fe700158f20 2023-07-06T03:52:35.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.211+0000 7fe6fe59c700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6e006ace0 0x7fe6e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:35.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.212+0000 7fe6fe59c700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6e006ace0 0x7fe6e006d190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe70014ac10 tx=0x7fe6f0013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:35.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.215+0000 7fe6eeffd700 1 -- 172.21.15.97:0/2057339256 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe6f4054200 con 0x7fe700158f20 2023-07-06T03:52:35.312 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:52:35.318 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:52:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.423+0000 7fe704dff700 1 -- 172.21.15.97:0/2057339256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe70004e340 con 0x7fe700158f20 2023-07-06T03:52:35.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.425+0000 7fe6eeffd700 1 -- 172.21.15.97:0/2057339256 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+18381 (secure 0 0 0) 0x7fe6f4057820 con 0x7fe700158f20 2023-07-06T03:52:35.426 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:35.427 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:52:34.447016+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:52:32.448828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-06T03:52:33.428266+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:52:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 -- 172.21.15.97:0/2057339256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6e006ace0 msgr2=0x7fe6e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6e006ace0 0x7fe6e006d190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe70014ac10 tx=0x7fe6f0013000 comp rx=0 tx=0).stop 2023-07-06T03:52:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 -- 172.21.15.97:0/2057339256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe700158f20 msgr2=0x7fe7000b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe700158f20 0x7fe7000b2bd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c550 tx=0x7fe6f400c910 comp rx=0 tx=0).stop 2023-07-06T03:52:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 -- 172.21.15.97:0/2057339256 shutdown_connections 2023-07-06T03:52:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe6e006ace0 0x7fe6e006d190 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7001592f0 0x7fe70015be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7000b0c70 0x7fe7000b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 --2- 172.21.15.97:0/2057339256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe700158f20 0x7fe7000b2bd0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 -- 172.21.15.97:0/2057339256 >> 172.21.15.97:0/2057339256 conn(0x7fe7000ce8c0 msgr2=0x7fe70006f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:35.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 -- 172.21.15.97:0/2057339256 shutdown_connections 2023-07-06T03:52:35.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.427+0000 7fe704dff700 1 -- 172.21.15.97:0/2057339256 wait complete. 2023-07-06T03:52:35.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:52:35.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.681+0000 7f18b0fff700 1 Processor -- start 2023-07-06T03:52:35.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.681+0000 7f18b0fff700 1 -- start start 2023-07-06T03:52:35.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.682+0000 7f18b0fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18ac15c920 0x7f18ac06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:35.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.682+0000 7f18b0fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ac0cfd00 con 0x7f18ac15ccf0 2023-07-06T03:52:35.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.682+0000 7f18b0fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ac06bf90 con 0x7f18ac0d11a0 2023-07-06T03:52:35.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.682+0000 7f18b0fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ac0b28f0 con 0x7f18ac15c920 2023-07-06T03:52:35.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.682+0000 7f18aa59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18ac15c920 0x7f18ac06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:35.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.682+0000 7f18aa59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18ac15c920 0x7f18ac06c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54938/0 (socket says 172.21.15.97:54938) 2023-07-06T03:52:35.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.682+0000 7f18a9d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f18ac15ccf0 0x7f18ac06b170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34648/0 (socket says 172.21.15.97:34648) 2023-07-06T03:52:35.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.682+0000 7f18aa59c700 1 -- 172.21.15.97:0/1069016217 learned_addr learned my addr 172.21.15.97:0/1069016217 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:35.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.683+0000 7f18aa59c700 1 -- 172.21.15.97:0/1069016217 >> v1:172.21.15.146:6789/0 conn(0x7f18ac0d11a0 legacy=0x7f18ac06b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:35.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.683+0000 7f18aa59c700 1 -- 172.21.15.97:0/1069016217 >> v1:172.21.15.97:6789/0 conn(0x7f18ac15ccf0 legacy=0x7f18ac06b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.683+0000 7f18aa59c700 1 -- 172.21.15.97:0/1069016217 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac1610b0 con 0x7f18ac15c920 2023-07-06T03:52:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.683+0000 7f18aa59c700 1 --2- 172.21.15.97:0/1069016217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ac15c920 0x7f18ac06c390 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f189c0098f0 tx=0x7f189c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbba8d37fa33ec10 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.683+0000 7f18a959a700 1 -- 172.21.15.97:0/1069016217 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c004230 con 0x7f18ac15c920 2023-07-06T03:52:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.683+0000 7f18a959a700 1 -- 172.21.15.97:0/1069016217 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189c004390 con 0x7f18ac15c920 2023-07-06T03:52:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.683+0000 7f18a959a700 1 -- 172.21.15.97:0/1069016217 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c0036a0 con 0x7f18ac15c920 2023-07-06T03:52:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.684+0000 7f18b0fff700 1 -- 172.21.15.97:0/1069016217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ac15c920 msgr2=0x7f18ac06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.684+0000 7f18b0fff700 1 --2- 172.21.15.97:0/1069016217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ac15c920 0x7f18ac06c390 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f189c0098f0 tx=0x7f189c009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.684+0000 7f18b0fff700 1 -- 172.21.15.97:0/1069016217 shutdown_connections 2023-07-06T03:52:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.684+0000 7f18b0fff700 1 --2- 172.21.15.97:0/1069016217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ac15c920 0x7f18ac06c390 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.684+0000 7f18b0fff700 1 -- 172.21.15.97:0/1069016217 >> 172.21.15.97:0/1069016217 conn(0x7f18ac06ace0 msgr2=0x7f18ac06dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:35.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.684+0000 7f18b0fff700 1 -- 172.21.15.97:0/1069016217 shutdown_connections 2023-07-06T03:52:35.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.684+0000 7f18b0fff700 1 -- 172.21.15.97:0/1069016217 wait complete. 2023-07-06T03:52:35.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.686+0000 7f18b0fff700 1 Processor -- start 2023-07-06T03:52:35.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.686+0000 7f18b0fff700 1 -- start start 2023-07-06T03:52:35.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18b0fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18ac15c920 0x7f18ac0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:35.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18b0fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac15ccf0 0x7f18ac0cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:35.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18b0fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ac0b3990 0x7f18ac1f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:35.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18b0fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac154620 con 0x7f18ac0b3990 2023-07-06T03:52:35.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18b0fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac0c5840 con 0x7f18ac15ccf0 2023-07-06T03:52:35.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18b0fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac0c5b20 con 0x7f18ac15c920 2023-07-06T03:52:35.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18aad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ac0b3990 0x7f18ac1f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:35.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18aad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ac0b3990 0x7f18ac1f1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54942/0 (socket says 172.21.15.97:54942) 2023-07-06T03:52:35.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18aad9d700 1 -- 172.21.15.97:0/3316506374 learned_addr learned my addr 172.21.15.97:0/3316506374 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:35.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18aa59c700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18ac15c920 0x7f18ac0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:35.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18a9d9b700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac15ccf0 0x7f18ac0cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:35.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18aad9d700 1 -- 172.21.15.97:0/3316506374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18ac15c920 msgr2=0x7f18ac0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18aad9d700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18ac15c920 0x7f18ac0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18aad9d700 1 -- 172.21.15.97:0/3316506374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac15ccf0 msgr2=0x7f18ac0cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18aad9d700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac15ccf0 0x7f18ac0cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18aad9d700 1 -- 172.21.15.97:0/3316506374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c009550 con 0x7f18ac0b3990 2023-07-06T03:52:35.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.687+0000 7f18aad9d700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ac0b3990 0x7f18ac1f1b70 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f18a400b970 tx=0x7f18a400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:35.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.688+0000 7f189affd700 1 -- 172.21.15.97:0/3316506374 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a400c9b0 con 0x7f18ac0b3990 2023-07-06T03:52:35.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.688+0000 7f189affd700 1 -- 172.21.15.97:0/3316506374 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a400cb10 con 0x7f18ac0b3990 2023-07-06T03:52:35.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.688+0000 7f18b0fff700 1 -- 172.21.15.97:0/3316506374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac0c5e00 con 0x7f18ac0b3990 2023-07-06T03:52:35.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.688+0000 7f189affd700 1 -- 172.21.15.97:0/3316506374 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4014440 con 0x7f18ac0b3990 2023-07-06T03:52:35.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.688+0000 7f18b0fff700 1 -- 172.21.15.97:0/3316506374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac0c63c0 con 0x7f18ac0b3990 2023-07-06T03:52:35.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.689+0000 7f18b0fff700 1 -- 172.21.15.97:0/3316506374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ac066a40 con 0x7f18ac0b3990 2023-07-06T03:52:35.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.690+0000 7f189affd700 1 -- 172.21.15.97:0/3316506374 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f18a401d3b0 con 0x7f18ac0b3990 2023-07-06T03:52:35.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.691+0000 7f189affd700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f188c06ace0 0x7f188c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:35.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.691+0000 7f189affd700 1 -- 172.21.15.97:0/3316506374 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18a40891a0 con 0x7f18ac0b3990 2023-07-06T03:52:35.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.691+0000 7f18aa59c700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f188c06ace0 0x7f188c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:35.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.691+0000 7f18aa59c700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f188c06ace0 0x7f188c06d190 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f189c009130 tx=0x7f189c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:35.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.695+0000 7f189affd700 1 -- 172.21.15.97:0/3316506374 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f18a40541b0 con 0x7f18ac0b3990 2023-07-06T03:52:35.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.893+0000 7f18b0fff700 1 -- 172.21.15.97:0/3316506374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18ac0c65e0 con 0x7f18ac0b3990 2023-07-06T03:52:35.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.894+0000 7f189affd700 1 -- 172.21.15.97:0/3316506374 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+18381 (secure 0 0 0) 0x7f18a40577d0 con 0x7f18ac0b3990 2023-07-06T03:52:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:35.897 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:52:34.447016+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:52:32.448828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-06T03:52:33.428266+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:52:35.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 -- 172.21.15.97:0/3316506374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f188c06ace0 msgr2=0x7f188c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f188c06ace0 0x7f188c06d190 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f189c009130 tx=0x7f189c008030 comp rx=0 tx=0).stop 2023-07-06T03:52:35.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 -- 172.21.15.97:0/3316506374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ac0b3990 msgr2=0x7f18ac1f1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:35.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ac0b3990 0x7f18ac1f1b70 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f18a400b970 tx=0x7f18a400bd30 comp rx=0 tx=0).stop 2023-07-06T03:52:35.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 -- 172.21.15.97:0/3316506374 shutdown_connections 2023-07-06T03:52:35.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f188c06ace0 0x7f188c06d190 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ac0b3990 0x7f18ac1f1b70 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f18ac15c920 0x7f18ac0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 --2- 172.21.15.97:0/3316506374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18ac15ccf0 0x7f18ac0cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:35.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 -- 172.21.15.97:0/3316506374 >> 172.21.15.97:0/3316506374 conn(0x7f18ac0d11a0 msgr2=0x7f18ac160590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:35.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 -- 172.21.15.97:0/3316506374 shutdown_connections 2023-07-06T03:52:35.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:35.897+0000 7f18b0fff700 1 -- 172.21.15.97:0/3316506374 wait complete. 2023-07-06T03:52:35.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T03:52:36.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.117+0000 7f3c53bff700 1 Processor -- start 2023-07-06T03:52:36.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.118+0000 7f3c53bff700 1 -- start start 2023-07-06T03:52:36.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.119+0000 7f3c53bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c4c06b360 0x7f3c4c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:36.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.119+0000 7f3c53bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c4c0d2210 0x7f3c4c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:36.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.119+0000 7f3c53bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c4c0cff90 con 0x7f3c4c15c610 2023-07-06T03:52:36.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.119+0000 7f3c53bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c4c0d0780 con 0x7f3c4c0d2210 2023-07-06T03:52:36.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.119+0000 7f3c53bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c4c15c9e0 con 0x7f3c4c06b360 2023-07-06T03:52:36.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.119+0000 7f3c5219c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c4c0d2210 0x7f3c4c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:36.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.119+0000 7f3c5219c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c4c0d2210 0x7f3c4c0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54948/0 (socket says 172.21.15.97:54948) 2023-07-06T03:52:36.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.119+0000 7f3c5219c700 1 -- 172.21.15.97:0/3317077119 learned_addr learned my addr 172.21.15.97:0/3317077119 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.119+0000 7f3c5119a700 1 --2- 172.21.15.97:0/3317077119 >> v2:172.21.15.146:3300/0 conn(0x7f3c4c06b360 0x7f3c4c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.119+0000 7f3c5219c700 1 -- 172.21.15.97:0/3317077119 >> v2:172.21.15.146:3300/0 conn(0x7f3c4c06b360 msgr2=0x7f3c4c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.120+0000 7f3c5219c700 1 --2- 172.21.15.97:0/3317077119 >> v2:172.21.15.146:3300/0 conn(0x7f3c4c06b360 0x7f3c4c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.120+0000 7f3c5219c700 1 -- 172.21.15.97:0/3317077119 >> v1:172.21.15.146:6789/0 conn(0x7f3c4c15c610 legacy=0x7f3c4c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.120+0000 7f3c5219c700 1 -- 172.21.15.97:0/3317077119 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c0b5650 con 0x7f3c4c0d2210 2023-07-06T03:52:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.120+0000 7f3c5119a700 1 --2- 172.21.15.97:0/3317077119 >> v2:172.21.15.146:3300/0 conn(0x7f3c4c06b360 0x7f3c4c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.120+0000 7f3c5219c700 1 --2- 172.21.15.97:0/3317077119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c0d2210 0x7f3c4c0d1310 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c06a590 tx=0x7f3c4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63041177341e9f35 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.120+0000 7f3c50999700 1 -- 172.21.15.97:0/3317077119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4400e070 con 0x7f3c4c0d2210 2023-07-06T03:52:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.120+0000 7f3c50999700 1 -- 172.21.15.97:0/3317077119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c44003a20 con 0x7f3c4c0d2210 2023-07-06T03:52:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.120+0000 7f3c50999700 1 -- 172.21.15.97:0/3317077119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44003dd0 con 0x7f3c4c0d2210 2023-07-06T03:52:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.120+0000 7f3c53bff700 1 -- 172.21.15.97:0/3317077119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c0d2210 msgr2=0x7f3c4c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.120+0000 7f3c53bff700 1 --2- 172.21.15.97:0/3317077119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c0d2210 0x7f3c4c0d1310 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c06a590 tx=0x7f3c4400b330 comp rx=0 tx=0).stop 2023-07-06T03:52:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.121+0000 7f3c53bff700 1 -- 172.21.15.97:0/3317077119 shutdown_connections 2023-07-06T03:52:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.121+0000 7f3c53bff700 1 --2- 172.21.15.97:0/3317077119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c0d2210 0x7f3c4c0d1310 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.121+0000 7f3c53bff700 1 --2- 172.21.15.97:0/3317077119 >> v2:172.21.15.146:3300/0 conn(0x7f3c4c06b360 0x7f3c4c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.121+0000 7f3c53bff700 1 -- 172.21.15.97:0/3317077119 >> 172.21.15.97:0/3317077119 conn(0x7f3c4c06a6c0 msgr2=0x7f3c4c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.121+0000 7f3c53bff700 1 -- 172.21.15.97:0/3317077119 shutdown_connections 2023-07-06T03:52:36.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.121+0000 7f3c53bff700 1 -- 172.21.15.97:0/3317077119 wait complete. 2023-07-06T03:52:36.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.123+0000 7f3c53bff700 1 Processor -- start 2023-07-06T03:52:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.123+0000 7f3c53bff700 1 -- start start 2023-07-06T03:52:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.124+0000 7f3c53bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c4c06b360 0x7f3c4c0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.124+0000 7f3c53bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c4c0d2210 0x7f3c4c153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.124+0000 7f3c53bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c15c610 0x7f3c4c15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.124+0000 7f3c53bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c0cc910 con 0x7f3c4c15c610 2023-07-06T03:52:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.124+0000 7f3c53bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c0cc670 con 0x7f3c4c06b360 2023-07-06T03:52:36.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.124+0000 7f3c53bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c0ca9b0 con 0x7f3c4c0d2210 2023-07-06T03:52:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5219c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c15c610 0x7f3c4c15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5219c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c15c610 0x7f3c4c15e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54960/0 (socket says 172.21.15.97:54960) 2023-07-06T03:52:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5219c700 1 -- 172.21.15.97:0/3954974138 learned_addr learned my addr 172.21.15.97:0/3954974138 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5199b700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c4c06b360 0x7f3c4c0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5119a700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c4c0d2210 0x7f3c4c153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5219c700 1 -- 172.21.15.97:0/3954974138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c4c0d2210 msgr2=0x7f3c4c153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5219c700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c4c0d2210 0x7f3c4c153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5219c700 1 -- 172.21.15.97:0/3954974138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c4c06b360 msgr2=0x7f3c4c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5219c700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c4c06b360 0x7f3c4c0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5219c700 1 -- 172.21.15.97:0/3954974138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c44009b30 con 0x7f3c4c15c610 2023-07-06T03:52:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5199b700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c4c06b360 0x7f3c4c0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.125+0000 7f3c5219c700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c15c610 0x7f3c4c15e6e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c08b120 tx=0x7f3c44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.126+0000 7f3c427fc700 1 -- 172.21.15.97:0/3954974138 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44004440 con 0x7f3c4c15c610 2023-07-06T03:52:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.126+0000 7f3c427fc700 1 -- 172.21.15.97:0/3954974138 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c440092e0 con 0x7f3c4c15c610 2023-07-06T03:52:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.126+0000 7f3c427fc700 1 -- 172.21.15.97:0/3954974138 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44004820 con 0x7f3c4c15c610 2023-07-06T03:52:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.126+0000 7f3c53bff700 1 -- 172.21.15.97:0/3954974138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4c0cac90 con 0x7f3c4c15c610 2023-07-06T03:52:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.126+0000 7f3c53bff700 1 -- 172.21.15.97:0/3954974138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4c0cb250 con 0x7f3c4c15c610 2023-07-06T03:52:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.127+0000 7f3c53bff700 1 -- 172.21.15.97:0/3954974138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4c066fe0 con 0x7f3c4c15c610 2023-07-06T03:52:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.128+0000 7f3c427fc700 1 -- 172.21.15.97:0/3954974138 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3c44012430 con 0x7f3c4c15c610 2023-07-06T03:52:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.129+0000 7f3c427fc700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c3006ace0 0x7f3c3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.129+0000 7f3c427fc700 1 -- 172.21.15.97:0/3954974138 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3c4408a980 con 0x7f3c4c15c610 2023-07-06T03:52:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.129+0000 7f3c5199b700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c3006ace0 0x7f3c3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:36.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.129+0000 7f3c5199b700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c3006ace0 0x7f3c3006d190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009930 tx=0x7f3c48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:36.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.133+0000 7f3c427fc700 1 -- 172.21.15.97:0/3954974138 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3c44055ac0 con 0x7f3c4c15c610 2023-07-06T03:52:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.373+0000 7f3c53bff700 1 -- 172.21.15.97:0/3954974138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3c4c04e340 con 0x7f3c4c15c610 2023-07-06T03:52:36.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.985+0000 7f3c427fc700 1 -- 172.21.15.97:0/3954974138 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3c4408a030 con 0x7f3c4c15c610 2023-07-06T03:52:36.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 -- 172.21.15.97:0/3954974138 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c3006ace0 msgr2=0x7f3c3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:36.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c3006ace0 0x7f3c3006d190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009930 tx=0x7f3c48008040 comp rx=0 tx=0).stop 2023-07-06T03:52:36.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 -- 172.21.15.97:0/3954974138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c15c610 msgr2=0x7f3c4c15e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:36.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c15c610 0x7f3c4c15e6e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c08b120 tx=0x7f3c44004ad0 comp rx=0 tx=0).stop 2023-07-06T03:52:36.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 -- 172.21.15.97:0/3954974138 shutdown_connections 2023-07-06T03:52:36.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3c3006ace0 0x7f3c3006d190 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:36.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c4c15c610 0x7f3c4c15e6e0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:36.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c4c06b360 0x7f3c4c0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:36.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 --2- 172.21.15.97:0/3954974138 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c4c0d2210 0x7f3c4c153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:36.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 -- 172.21.15.97:0/3954974138 >> 172.21.15.97:0/3954974138 conn(0x7f3c4c06a6c0 msgr2=0x7f3c4c0d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:36.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 -- 172.21.15.97:0/3954974138 shutdown_connections 2023-07-06T03:52:36.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:36.988+0000 7f3c53bff700 1 -- 172.21.15.97:0/3954974138 wait complete. 2023-07-06T03:52:37.009 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T03:52:37.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T03:52:37.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.204+0000 7fde84dff700 1 Processor -- start 2023-07-06T03:52:37.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.205+0000 7fde84dff700 1 -- start start 2023-07-06T03:52:37.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.206+0000 7fde84dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde800d2210 0x7fde800b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:37.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.206+0000 7fde84dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde800cff90 con 0x7fde8015c610 2023-07-06T03:52:37.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.206+0000 7fde84dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde800b5650 con 0x7fde8006b360 2023-07-06T03:52:37.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.206+0000 7fde84dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde800d0780 con 0x7fde800d2210 2023-07-06T03:52:37.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.206+0000 7fde7e59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fde8015c610 0x7fde800b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34658/0 (socket says 172.21.15.97:34658) 2023-07-06T03:52:37.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.206+0000 7fde7e59c700 1 -- 172.21.15.97:0/3298499567 learned_addr learned my addr 172.21.15.97:0/3298499567 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:37.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.206+0000 7fde7ed9d700 1 --2- 172.21.15.97:0/3298499567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde800d2210 0x7fde800b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:37.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.207+0000 7fde7dd9b700 1 -- 172.21.15.97:0/3298499567 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787802819 0 0) 0x7fde800cff90 con 0x7fde8015c610 2023-07-06T03:52:37.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.207+0000 7fde7dd9b700 1 -- 172.21.15.97:0/3298499567 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde6c0035b0 con 0x7fde8015c610 2023-07-06T03:52:37.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.207+0000 7fde7dd9b700 1 -- 172.21.15.97:0/3298499567 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888849703 0 0) 0x7fde800b5650 con 0x7fde8006b360 2023-07-06T03:52:37.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.207+0000 7fde7ed9d700 1 -- 172.21.15.97:0/3298499567 >> v1:172.21.15.146:6789/0 conn(0x7fde8006b360 legacy=0x7fde800d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:37.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.207+0000 7fde7ed9d700 1 -- 172.21.15.97:0/3298499567 >> v1:172.21.15.97:6789/0 conn(0x7fde8015c610 legacy=0x7fde800b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:37.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.207+0000 7fde7ed9d700 1 -- 172.21.15.97:0/3298499567 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde800b5400 con 0x7fde800d2210 2023-07-06T03:52:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.207+0000 7fde7ed9d700 1 --2- 172.21.15.97:0/3298499567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde800d2210 0x7fde800b4ec0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fde78009dc0 tx=0x7fde7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26c53da223c22ef server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.207+0000 7fde7dd9b700 1 -- 172.21.15.97:0/3298499567 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800e070 con 0x7fde800d2210 2023-07-06T03:52:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.207+0000 7fde7dd9b700 1 -- 172.21.15.97:0/3298499567 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde78003a20 con 0x7fde800d2210 2023-07-06T03:52:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.208+0000 7fde7dd9b700 1 -- 172.21.15.97:0/3298499567 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70001670 con 0x7fde800d2210 2023-07-06T03:52:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.208+0000 7fde84dff700 1 -- 172.21.15.97:0/3298499567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde800d2210 msgr2=0x7fde800b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.208+0000 7fde84dff700 1 --2- 172.21.15.97:0/3298499567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde800d2210 0x7fde800b4ec0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fde78009dc0 tx=0x7fde7800b330 comp rx=0 tx=0).stop 2023-07-06T03:52:37.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.208+0000 7fde84dff700 1 -- 172.21.15.97:0/3298499567 shutdown_connections 2023-07-06T03:52:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.208+0000 7fde84dff700 1 --2- 172.21.15.97:0/3298499567 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde800d2210 0x7fde800b4ec0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.208+0000 7fde84dff700 1 -- 172.21.15.97:0/3298499567 >> 172.21.15.97:0/3298499567 conn(0x7fde8006a6c0 msgr2=0x7fde8006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.208+0000 7fde84dff700 1 -- 172.21.15.97:0/3298499567 shutdown_connections 2023-07-06T03:52:37.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.208+0000 7fde84dff700 1 -- 172.21.15.97:0/3298499567 wait complete. 2023-07-06T03:52:37.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.210+0000 7fde84dff700 1 Processor -- start 2023-07-06T03:52:37.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.210+0000 7fde84dff700 1 -- start start 2023-07-06T03:52:37.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde84dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde8006b360 0x7fde801551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:37.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde84dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde800d2210 0x7fde80153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:37.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde84dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8015c610 0x7fde801599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:37.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde84dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde80154930 con 0x7fde800d2210 2023-07-06T03:52:37.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde84dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800cbbc0 con 0x7fde8015c610 2023-07-06T03:52:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde84dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8015d000 con 0x7fde8006b360 2023-07-06T03:52:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde77fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde800d2210 0x7fde80153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde77fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde800d2210 0x7fde80153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54968/0 (socket says 172.21.15.97:54968) 2023-07-06T03:52:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde77fff700 1 -- 172.21.15.97:0/3900682051 learned_addr learned my addr 172.21.15.97:0/3900682051 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde7ed9d700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8015c610 0x7fde801599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde7e59c700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde8006b360 0x7fde801551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.211+0000 7fde77fff700 1 -- 172.21.15.97:0/3900682051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde8006b360 msgr2=0x7fde801551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.212+0000 7fde77fff700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde8006b360 0x7fde801551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.212+0000 7fde77fff700 1 -- 172.21.15.97:0/3900682051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8015c610 msgr2=0x7fde801599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.212+0000 7fde77fff700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8015c610 0x7fde801599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.212+0000 7fde77fff700 1 -- 172.21.15.97:0/3900682051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde78009a70 con 0x7fde800d2210 2023-07-06T03:52:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.212+0000 7fde77fff700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde800d2210 0x7fde80153340 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fde6800c550 tx=0x7fde6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.212+0000 7fde76ffd700 1 -- 172.21.15.97:0/3900682051 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde680054e0 con 0x7fde800d2210 2023-07-06T03:52:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.212+0000 7fde76ffd700 1 -- 172.21.15.97:0/3900682051 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde68003680 con 0x7fde800d2210 2023-07-06T03:52:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.212+0000 7fde76ffd700 1 -- 172.21.15.97:0/3900682051 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68003a30 con 0x7fde800d2210 2023-07-06T03:52:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.212+0000 7fde84dff700 1 -- 172.21.15.97:0/3900682051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde800cc3d0 con 0x7fde800d2210 2023-07-06T03:52:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.212+0000 7fde84dff700 1 -- 172.21.15.97:0/3900682051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde800cc0f0 con 0x7fde800d2210 2023-07-06T03:52:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.213+0000 7fde84dff700 1 -- 172.21.15.97:0/3900682051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8015d170 con 0x7fde800d2210 2023-07-06T03:52:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.214+0000 7fde76ffd700 1 -- 172.21.15.97:0/3900682051 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde68003b90 con 0x7fde800d2210 2023-07-06T03:52:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.215+0000 7fde76ffd700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde6006ace0 0x7fde6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.215+0000 7fde76ffd700 1 -- 172.21.15.97:0/3900682051 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fde68088950 con 0x7fde800d2210 2023-07-06T03:52:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.215+0000 7fde7e59c700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde6006ace0 0x7fde6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.216+0000 7fde7e59c700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde6006ace0 0x7fde6006d190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fde700017a0 tx=0x7fde70001650 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.219+0000 7fde76ffd700 1 -- 172.21.15.97:0/3900682051 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fde680539f0 con 0x7fde800d2210 2023-07-06T03:52:37.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.461+0000 7fde84dff700 1 -- 172.21.15.97:0/3900682051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fde8004e340 con 0x7fde800d2210 2023-07-06T03:52:37.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.992+0000 7fde76ffd700 1 -- 172.21.15.97:0/3900682051 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7fde68017020 con 0x7fde800d2210 2023-07-06T03:52:37.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 -- 172.21.15.97:0/3900682051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde6006ace0 msgr2=0x7fde6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:37.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde6006ace0 0x7fde6006d190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fde700017a0 tx=0x7fde70001650 comp rx=0 tx=0).stop 2023-07-06T03:52:37.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 -- 172.21.15.97:0/3900682051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde800d2210 msgr2=0x7fde80153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:37.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde800d2210 0x7fde80153340 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fde6800c550 tx=0x7fde6800c910 comp rx=0 tx=0).stop 2023-07-06T03:52:37.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 -- 172.21.15.97:0/3900682051 shutdown_connections 2023-07-06T03:52:37.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fde6006ace0 0x7fde6006d190 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:37.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde8015c610 0x7fde801599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:37.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fde8006b360 0x7fde801551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:37.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 --2- 172.21.15.97:0/3900682051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde800d2210 0x7fde80153340 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:37.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 -- 172.21.15.97:0/3900682051 >> 172.21.15.97:0/3900682051 conn(0x7fde8006a6c0 msgr2=0x7fde800b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:37.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 -- 172.21.15.97:0/3900682051 shutdown_connections 2023-07-06T03:52:37.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:37.994+0000 7fde84dff700 1 -- 172.21.15.97:0/3900682051 wait complete. 2023-07-06T03:52:38.015 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T03:52:38.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:52:38.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.208+0000 7f67b73ff700 1 Processor -- start 2023-07-06T03:52:38.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.208+0000 7f67b73ff700 1 -- start start 2023-07-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.210+0000 7f67b73ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67b015ca10 0x7f67b0160e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.210+0000 7f67b73ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f67b00cf3c0 0x7f67b00b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.210+0000 7f67b73ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b00d1190 con 0x7f67b00b4ee0 2023-07-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.210+0000 7f67b73ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b00b5360 con 0x7f67b00cf3c0 2023-07-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.210+0000 7f67b73ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0161340 con 0x7f67b015ca10 2023-07-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.210+0000 7f67b499a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f67b00cf3c0 0x7f67b00b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.210+0000 7f67b499a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f67b00cf3c0 0x7f67b00b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54972/0 (socket says 172.21.15.97:54972) 2023-07-06T03:52:38.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.210+0000 7f67b499a700 1 -- 172.21.15.97:0/439656920 learned_addr learned my addr 172.21.15.97:0/439656920 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.210+0000 7f67b519b700 1 --2- 172.21.15.97:0/439656920 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67b015ca10 0x7f67b0160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.211+0000 7f67b499a700 1 -- 172.21.15.97:0/439656920 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67b015ca10 msgr2=0x7f67b0160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.211+0000 7f67b499a700 1 --2- 172.21.15.97:0/439656920 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67b015ca10 0x7f67b0160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.211+0000 7f67b499a700 1 -- 172.21.15.97:0/439656920 >> v1:172.21.15.146:6789/0 conn(0x7f67b00b4ee0 legacy=0x7f67b00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.211+0000 7f67b499a700 1 -- 172.21.15.97:0/439656920 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b00cd680 con 0x7f67b00cf3c0 2023-07-06T03:52:38.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.211+0000 7f67b519b700 1 --2- 172.21.15.97:0/439656920 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67b015ca10 0x7f67b0160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:38.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.211+0000 7f67b499a700 1 --2- 172.21.15.97:0/439656920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67b00cf3c0 0x7f67b00b49a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f67a00098f0 tx=0x7f67a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc7c1a3a7d5aa21 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:38.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.211+0000 7f67a7fff700 1 -- 172.21.15.97:0/439656920 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a000b8a0 con 0x7f67b00cf3c0 2023-07-06T03:52:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.211+0000 7f67a7fff700 1 -- 172.21.15.97:0/439656920 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a000ba00 con 0x7f67b00cf3c0 2023-07-06T03:52:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.211+0000 7f67a7fff700 1 -- 172.21.15.97:0/439656920 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a000bd90 con 0x7f67b00cf3c0 2023-07-06T03:52:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.212+0000 7f67b73ff700 1 -- 172.21.15.97:0/439656920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67b00cf3c0 msgr2=0x7f67b00b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.212+0000 7f67b73ff700 1 --2- 172.21.15.97:0/439656920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67b00cf3c0 0x7f67b00b49a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f67a00098f0 tx=0x7f67a0009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.212+0000 7f67b73ff700 1 -- 172.21.15.97:0/439656920 shutdown_connections 2023-07-06T03:52:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.212+0000 7f67b73ff700 1 --2- 172.21.15.97:0/439656920 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67b015ca10 0x7f67b0160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.212+0000 7f67b73ff700 1 --2- 172.21.15.97:0/439656920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67b00cf3c0 0x7f67b00b49a0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.212+0000 7f67b73ff700 1 -- 172.21.15.97:0/439656920 >> 172.21.15.97:0/439656920 conn(0x7f67b006a920 msgr2=0x7f67b006ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:38.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.212+0000 7f67b73ff700 1 -- 172.21.15.97:0/439656920 shutdown_connections 2023-07-06T03:52:38.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.212+0000 7f67b73ff700 1 -- 172.21.15.97:0/439656920 wait complete. 2023-07-06T03:52:38.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.214+0000 7f67b73ff700 1 Processor -- start 2023-07-06T03:52:38.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.214+0000 7f67b73ff700 1 -- start start 2023-07-06T03:52:38.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b73ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67b00b4ee0 0x7f67b0157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b73ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67b00cf3c0 0x7f67b00cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b73ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67b015ca10 0x7f67b01e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b73ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00cb720 con 0x7f67b00cf3c0 2023-07-06T03:52:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b73ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0157b60 con 0x7f67b00b4ee0 2023-07-06T03:52:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b73ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00caaa0 con 0x7f67b015ca10 2023-07-06T03:52:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b499a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67b00cf3c0 0x7f67b00cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b499a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67b00cf3c0 0x7f67b00cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54974/0 (socket says 172.21.15.97:54974) 2023-07-06T03:52:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b499a700 1 -- 172.21.15.97:0/339826447 learned_addr learned my addr 172.21.15.97:0/339826447 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b519b700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67b00b4ee0 0x7f67b0157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b599c700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67b015ca10 0x7f67b01e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b499a700 1 -- 172.21.15.97:0/339826447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67b015ca10 msgr2=0x7f67b01e7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b499a700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67b015ca10 0x7f67b01e7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b499a700 1 -- 172.21.15.97:0/339826447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67b00b4ee0 msgr2=0x7f67b0157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b499a700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67b00b4ee0 0x7f67b0157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.215+0000 7f67b499a700 1 -- 172.21.15.97:0/339826447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a0009550 con 0x7f67b00cf3c0 2023-07-06T03:52:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.216+0000 7f67b499a700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67b00cf3c0 0x7f67b00cbb40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f67a0000c00 tx=0x7f67a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.216+0000 7f67a5ffb700 1 -- 172.21.15.97:0/339826447 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0004590 con 0x7f67b00cf3c0 2023-07-06T03:52:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.216+0000 7f67a5ffb700 1 -- 172.21.15.97:0/339826447 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a001e070 con 0x7f67b00cf3c0 2023-07-06T03:52:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.216+0000 7f67a5ffb700 1 -- 172.21.15.97:0/339826447 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a00038e0 con 0x7f67b00cf3c0 2023-07-06T03:52:38.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.216+0000 7f67b73ff700 1 -- 172.21.15.97:0/339826447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b00cad80 con 0x7f67b00cf3c0 2023-07-06T03:52:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.216+0000 7f67b73ff700 1 -- 172.21.15.97:0/339826447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b00cb340 con 0x7f67b00cf3c0 2023-07-06T03:52:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.217+0000 7f67b73ff700 1 -- 172.21.15.97:0/339826447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b006a790 con 0x7f67b00cf3c0 2023-07-06T03:52:38.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.218+0000 7f67a5ffb700 1 -- 172.21.15.97:0/339826447 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f67a0003a40 con 0x7f67b00cf3c0 2023-07-06T03:52:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.219+0000 7f67a5ffb700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f679406ace0 0x7f679406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.219+0000 7f67a5ffb700 1 -- 172.21.15.97:0/339826447 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67a0020070 con 0x7f67b00cf3c0 2023-07-06T03:52:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.219+0000 7f67b519b700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f679406ace0 0x7f679406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.219+0000 7f67b519b700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f679406ace0 0x7f679406d190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f67ac006f10 tx=0x7f67ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:38.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.223+0000 7f67a5ffb700 1 -- 172.21.15.97:0/339826447 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f67a00558d0 con 0x7f67b00cf3c0 2023-07-06T03:52:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.460+0000 7f67b73ff700 1 -- 172.21.15.97:0/339826447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67b004e340 con 0x7f67b00cf3c0 2023-07-06T03:52:38.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.461+0000 7f67a5ffb700 1 -- 172.21.15.97:0/339826447 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f67a0058ef0 con 0x7f67b00cf3c0 2023-07-06T03:52:38.463 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:38.463 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3945452207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3945452207},{"type":"v1","addr":"172.21.15.146:6835","nonce":3945452207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2428591387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2428591387},{"type":"v1","addr":"172.21.15.97:6837","nonce":2428591387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/2089126027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2089126027},{"type":"v1","addr":"172.21.15.146:6834","nonce":2089126027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:52:34.447330+0000","modified":"2023-07-06T03:52:37.985332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7497":{"gid":7497,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/2046478602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2046478602},{"type":"v1","addr":"172.21.15.97:6836","nonce":2046478602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-06T03:52:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.463+0000 7f67b73ff700 1 -- 172.21.15.97:0/339826447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f679406ace0 msgr2=0x7f679406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.463+0000 7f67b73ff700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f679406ace0 0x7f679406d190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f67ac006f10 tx=0x7f67ac008040 comp rx=0 tx=0).stop 2023-07-06T03:52:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.463+0000 7f67b73ff700 1 -- 172.21.15.97:0/339826447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67b00cf3c0 msgr2=0x7f67b00cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.463+0000 7f67b73ff700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67b00cf3c0 0x7f67b00cbb40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f67a0000c00 tx=0x7f67a0004750 comp rx=0 tx=0).stop 2023-07-06T03:52:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.464+0000 7f67b73ff700 1 -- 172.21.15.97:0/339826447 shutdown_connections 2023-07-06T03:52:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.464+0000 7f67b73ff700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f679406ace0 0x7f679406d190 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.464+0000 7f67b73ff700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f67b015ca10 0x7f67b01e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.464+0000 7f67b73ff700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67b00b4ee0 0x7f67b0157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.464+0000 7f67b73ff700 1 --2- 172.21.15.97:0/339826447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67b00cf3c0 0x7f67b00cbb40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.464+0000 7f67b73ff700 1 -- 172.21.15.97:0/339826447 >> 172.21.15.97:0/339826447 conn(0x7f67b006a920 msgr2=0x7f67b015f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:38.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.464+0000 7f67b73ff700 1 -- 172.21.15.97:0/339826447 shutdown_connections 2023-07-06T03:52:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.464+0000 7f67b73ff700 1 -- 172.21.15.97:0/339826447 wait complete. 2023-07-06T03:52:38.468 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 56 2023-07-06T03:52:38.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:52:38.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.676+0000 7f28cc7ff700 1 Processor -- start 2023-07-06T03:52:38.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.676+0000 7f28cc7ff700 1 -- start start 2023-07-06T03:52:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.677+0000 7f28cc7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28c4158f20 0x7f28c40caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.677+0000 7f28cc7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f28c40b0c70 0x7f28c406bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.677+0000 7f28cc7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c415cc40 con 0x7f28c41592f0 2023-07-06T03:52:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.677+0000 7f28cc7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c415cf20 con 0x7f28c40b0c70 2023-07-06T03:52:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.677+0000 7f28cc7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c415d200 con 0x7f28c4158f20 2023-07-06T03:52:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.677+0000 7f28cad9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f28c40b0c70 0x7f28c406bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.677+0000 7f28c9d9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f28c41592f0 0x7f28c40b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34672/0 (socket says 172.21.15.97:34672) 2023-07-06T03:52:38.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.677+0000 7f28cad9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f28c40b0c70 0x7f28c406bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54982/0 (socket says 172.21.15.97:54982) 2023-07-06T03:52:38.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28c9d9a700 1 -- 172.21.15.97:0/1222616994 learned_addr learned my addr 172.21.15.97:0/1222616994 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:38.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28cad9c700 1 -- 172.21.15.97:0/1222616994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28c4158f20 msgr2=0x7f28c40caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:38.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28ca59b700 1 --2- 172.21.15.97:0/1222616994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28c4158f20 0x7f28c40caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28cad9c700 1 --2- 172.21.15.97:0/1222616994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28c4158f20 0x7f28c40caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28cad9c700 1 -- 172.21.15.97:0/1222616994 >> v1:172.21.15.97:6789/0 conn(0x7f28c41592f0 legacy=0x7f28c40b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28cad9c700 1 -- 172.21.15.97:0/1222616994 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c415d4e0 con 0x7f28c40b0c70 2023-07-06T03:52:38.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28ca59b700 1 --2- 172.21.15.97:0/1222616994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28c4158f20 0x7f28c40caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:38.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28c9599700 1 -- 172.21.15.97:0/1222616994 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442869543 0 0) 0x7f28c415cc40 con 0x7f28c41592f0 2023-07-06T03:52:38.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28cad9c700 1 --2- 172.21.15.97:0/1222616994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c40b0c70 0x7f28c406bd60 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00bae0 tx=0x7f28bc00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7964b8a7219cdbaf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:38.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28c9599700 1 -- 172.21.15.97:0/1222616994 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc00dd30 con 0x7f28c40b0c70 2023-07-06T03:52:38.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28c9599700 1 -- 172.21.15.97:0/1222616994 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc00de90 con 0x7f28c40b0c70 2023-07-06T03:52:38.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.678+0000 7f28c9599700 1 -- 172.21.15.97:0/1222616994 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc010690 con 0x7f28c40b0c70 2023-07-06T03:52:38.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.679+0000 7f28cc7ff700 1 -- 172.21.15.97:0/1222616994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c40b0c70 msgr2=0x7f28c406bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.679+0000 7f28cc7ff700 1 --2- 172.21.15.97:0/1222616994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c40b0c70 0x7f28c406bd60 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00bae0 tx=0x7f28bc00bdf0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.679+0000 7f28cc7ff700 1 -- 172.21.15.97:0/1222616994 shutdown_connections 2023-07-06T03:52:38.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.679+0000 7f28cc7ff700 1 --2- 172.21.15.97:0/1222616994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c40b0c70 0x7f28c406bd60 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.679+0000 7f28cc7ff700 1 --2- 172.21.15.97:0/1222616994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28c4158f20 0x7f28c40caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.679+0000 7f28cc7ff700 1 -- 172.21.15.97:0/1222616994 >> 172.21.15.97:0/1222616994 conn(0x7f28c40ce8c0 msgr2=0x7f28c40cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:38.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.679+0000 7f28cc7ff700 1 -- 172.21.15.97:0/1222616994 shutdown_connections 2023-07-06T03:52:38.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.679+0000 7f28cc7ff700 1 -- 172.21.15.97:0/1222616994 wait complete. 2023-07-06T03:52:38.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.681+0000 7f28cc7ff700 1 Processor -- start 2023-07-06T03:52:38.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.681+0000 7f28cc7ff700 1 -- start start 2023-07-06T03:52:38.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.682+0000 7f28cc7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c40b0c70 0x7f28c4149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.682+0000 7f28cc7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28c4158f20 0x7f28c4149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.682+0000 7f28cc7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c41592f0 0x7f28c4150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.682+0000 7f28cc7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c414ba60 con 0x7f28c40b0c70 2023-07-06T03:52:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.682+0000 7f28cc7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c40b1a00 con 0x7f28c41592f0 2023-07-06T03:52:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.682+0000 7f28cc7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c40b1ce0 con 0x7f28c4158f20 2023-07-06T03:52:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.682+0000 7f28ca59b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c40b0c70 0x7f28c4149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.682+0000 7f28ca59b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c40b0c70 0x7f28c4149c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54994/0 (socket says 172.21.15.97:54994) 2023-07-06T03:52:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.682+0000 7f28ca59b700 1 -- 172.21.15.97:0/298201885 learned_addr learned my addr 172.21.15.97:0/298201885 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.682+0000 7f28c9d9a700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28c4158f20 0x7f28c4149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.682+0000 7f28cad9c700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c41592f0 0x7f28c4150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.683+0000 7f28ca59b700 1 -- 172.21.15.97:0/298201885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28c4158f20 msgr2=0x7f28c4149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.683+0000 7f28ca59b700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28c4158f20 0x7f28c4149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.683+0000 7f28ca59b700 1 -- 172.21.15.97:0/298201885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c41592f0 msgr2=0x7f28c4150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.683+0000 7f28ca59b700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c41592f0 0x7f28c4150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.683+0000 7f28ca59b700 1 -- 172.21.15.97:0/298201885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc00b740 con 0x7f28c40b0c70 2023-07-06T03:52:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.683+0000 7f28ca59b700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c40b0c70 0x7f28c4149c80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f28c000cf60 tx=0x7f28c000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.683+0000 7f28baffd700 1 -- 172.21.15.97:0/298201885 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0015440 con 0x7f28c40b0c70 2023-07-06T03:52:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.683+0000 7f28baffd700 1 -- 172.21.15.97:0/298201885 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c00155a0 con 0x7f28c40b0c70 2023-07-06T03:52:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.683+0000 7f28cc7ff700 1 -- 172.21.15.97:0/298201885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c40b2410 con 0x7f28c40b0c70 2023-07-06T03:52:38.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.683+0000 7f28baffd700 1 -- 172.21.15.97:0/298201885 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0015930 con 0x7f28c40b0c70 2023-07-06T03:52:38.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.683+0000 7f28cc7ff700 1 -- 172.21.15.97:0/298201885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c4151310 con 0x7f28c40b0c70 2023-07-06T03:52:38.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.684+0000 7f28cc7ff700 1 -- 172.21.15.97:0/298201885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c40b2250 con 0x7f28c40b0c70 2023-07-06T03:52:38.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.685+0000 7f28baffd700 1 -- 172.21.15.97:0/298201885 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28c0015a90 con 0x7f28c40b0c70 2023-07-06T03:52:38.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.686+0000 7f28baffd700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f28a806ace0 0x7f28a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:38.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.686+0000 7f28baffd700 1 -- 172.21.15.97:0/298201885 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f28c00899c0 con 0x7f28c40b0c70 2023-07-06T03:52:38.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.686+0000 7f28c9d9a700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f28a806ace0 0x7f28a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:38.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.687+0000 7f28c9d9a700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f28a806ace0 0x7f28a806d190 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f28b4000f60 tx=0x7f28b4007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:38.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.690+0000 7f28baffd700 1 -- 172.21.15.97:0/298201885 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f28c0054b10 con 0x7f28c40b0c70 2023-07-06T03:52:38.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.888+0000 7f28cc7ff700 1 -- 172.21.15.97:0/298201885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28c404e340 con 0x7f28c40b0c70 2023-07-06T03:52:38.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.889+0000 7f28baffd700 1 -- 172.21.15.97:0/298201885 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+18368 (secure 0 0 0) 0x7f28c0058130 con 0x7f28c40b0c70 2023-07-06T03:52:38.891 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:38.891 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:52:35.953109+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:52:32.448828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-06T03:52:33.428266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:52:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 -- 172.21.15.97:0/298201885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f28a806ace0 msgr2=0x7f28a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f28a806ace0 0x7f28a806d190 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f28b4000f60 tx=0x7f28b4007c90 comp rx=0 tx=0).stop 2023-07-06T03:52:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 -- 172.21.15.97:0/298201885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c40b0c70 msgr2=0x7f28c4149c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c40b0c70 0x7f28c4149c80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f28c000cf60 tx=0x7f28c000cf90 comp rx=0 tx=0).stop 2023-07-06T03:52:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 -- 172.21.15.97:0/298201885 shutdown_connections 2023-07-06T03:52:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f28a806ace0 0x7f28a806d190 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28c41592f0 0x7f28c4150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c40b0c70 0x7f28c4149c80 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 --2- 172.21.15.97:0/298201885 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28c4158f20 0x7f28c4149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 -- 172.21.15.97:0/298201885 >> 172.21.15.97:0/298201885 conn(0x7f28c40ce8c0 msgr2=0x7f28c4152f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 -- 172.21.15.97:0/298201885 shutdown_connections 2023-07-06T03:52:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:38.891+0000 7f28cc7ff700 1 -- 172.21.15.97:0/298201885 wait complete. 2023-07-06T03:52:38.911 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T03:52:38.912 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:52:38.923 INFO:tasks.ceph.mds.a.smithi097.stderr:2023-07-06T03:52:38.922+0000 7f2d4bff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:52:39.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.106+0000 7fe3d07ff700 1 Processor -- start 2023-07-06T03:52:39.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.106+0000 7fe3d07ff700 1 -- start start 2023-07-06T03:52:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3d07ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe3c815c520 0x7fe3c80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3d07ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3c8160180 0x7fe3c81605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3d07ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c80cdae0 con 0x7fe3c8160b30 2023-07-06T03:52:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3d07ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c80cd8d0 con 0x7fe3c8160180 2023-07-06T03:52:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3d07ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c80b6640 con 0x7fe3c815c520 2023-07-06T03:52:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3cdd9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3c8160180 0x7fe3c81605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3cdd9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3c8160180 0x7fe3c81605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55004/0 (socket says 172.21.15.97:55004) 2023-07-06T03:52:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3cdd9a700 1 -- 172.21.15.97:0/4217116458 learned_addr learned my addr 172.21.15.97:0/4217116458 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3ce59b700 1 --2- 172.21.15.97:0/4217116458 >> v2:172.21.15.146:3300/0 conn(0x7fe3c815c520 0x7fe3c80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3cdd9a700 1 -- 172.21.15.97:0/4217116458 >> v2:172.21.15.146:3300/0 conn(0x7fe3c815c520 msgr2=0x7fe3c80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:39.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3cdd9a700 1 --2- 172.21.15.97:0/4217116458 >> v2:172.21.15.146:3300/0 conn(0x7fe3c815c520 0x7fe3c80cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3cdd9a700 1 -- 172.21.15.97:0/4217116458 >> v1:172.21.15.146:6789/0 conn(0x7fe3c8160b30 legacy=0x7fe3c80cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:39.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.108+0000 7fe3cdd9a700 1 -- 172.21.15.97:0/4217116458 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c80b6920 con 0x7fe3c8160180 2023-07-06T03:52:39.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.109+0000 7fe3cdd9a700 1 --2- 172.21.15.97:0/4217116458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c8160180 0x7fe3c81605f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80b4c70 tx=0x7fe3b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad44f5265f7891d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:39.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.109+0000 7fe3cd599700 1 -- 172.21.15.97:0/4217116458 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800b870 con 0x7fe3c8160180 2023-07-06T03:52:39.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.109+0000 7fe3cd599700 1 -- 172.21.15.97:0/4217116458 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b800b9d0 con 0x7fe3c8160180 2023-07-06T03:52:39.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.109+0000 7fe3cd599700 1 -- 172.21.15.97:0/4217116458 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800bd60 con 0x7fe3c8160180 2023-07-06T03:52:39.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.109+0000 7fe3d07ff700 1 -- 172.21.15.97:0/4217116458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c8160180 msgr2=0x7fe3c81605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:39.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.109+0000 7fe3d07ff700 1 --2- 172.21.15.97:0/4217116458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c8160180 0x7fe3c81605f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80b4c70 tx=0x7fe3b8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.110+0000 7fe3d07ff700 1 -- 172.21.15.97:0/4217116458 shutdown_connections 2023-07-06T03:52:39.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.110+0000 7fe3d07ff700 1 --2- 172.21.15.97:0/4217116458 >> v2:172.21.15.146:3300/0 conn(0x7fe3c815c520 0x7fe3c80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.110+0000 7fe3d07ff700 1 --2- 172.21.15.97:0/4217116458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c8160180 0x7fe3c81605f0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.110+0000 7fe3d07ff700 1 -- 172.21.15.97:0/4217116458 >> 172.21.15.97:0/4217116458 conn(0x7fe3c8071790 msgr2=0x7fe3c8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:39.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.110+0000 7fe3d07ff700 1 -- 172.21.15.97:0/4217116458 shutdown_connections 2023-07-06T03:52:39.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.110+0000 7fe3d07ff700 1 -- 172.21.15.97:0/4217116458 wait complete. 2023-07-06T03:52:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.111+0000 7fe3d07ff700 1 Processor -- start 2023-07-06T03:52:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.111+0000 7fe3d07ff700 1 -- start start 2023-07-06T03:52:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.112+0000 7fe3d07ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c815c520 0x7fe3c8155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.112+0000 7fe3d07ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3c8160180 0x7fe3c815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.112+0000 7fe3d07ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3c8160b30 0x7fe3c815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.112+0000 7fe3d07ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c815f700 con 0x7fe3c815c520 2023-07-06T03:52:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.112+0000 7fe3d07ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80ca120 con 0x7fe3c8160b30 2023-07-06T03:52:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.112+0000 7fe3d07ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8156520 con 0x7fe3c8160180 2023-07-06T03:52:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.112+0000 7fe3ce59b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c815c520 0x7fe3c8155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.113+0000 7fe3ce59b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c815c520 0x7fe3c8155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55020/0 (socket says 172.21.15.97:55020) 2023-07-06T03:52:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.113+0000 7fe3ce59b700 1 -- 172.21.15.97:0/2539493032 learned_addr learned my addr 172.21.15.97:0/2539493032 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.113+0000 7fe3cdd9a700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3c8160180 0x7fe3c815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.113+0000 7fe3ced9c700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3c8160b30 0x7fe3c815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.113+0000 7fe3ce59b700 1 -- 172.21.15.97:0/2539493032 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3c8160180 msgr2=0x7fe3c815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.113+0000 7fe3ce59b700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3c8160180 0x7fe3c815ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.113+0000 7fe3ce59b700 1 -- 172.21.15.97:0/2539493032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3c8160b30 msgr2=0x7fe3c815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.113+0000 7fe3ce59b700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3c8160b30 0x7fe3c815cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.113+0000 7fe3ce59b700 1 -- 172.21.15.97:0/2539493032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b8009550 con 0x7fe3c815c520 2023-07-06T03:52:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.113+0000 7fe3ce59b700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c815c520 0x7fe3c8155fe0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400d800 tx=0x7fe3c400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.113+0000 7fe3beffd700 1 -- 172.21.15.97:0/2539493032 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c40126b0 con 0x7fe3c815c520 2023-07-06T03:52:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.114+0000 7fe3beffd700 1 -- 172.21.15.97:0/2539493032 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c4012810 con 0x7fe3c815c520 2023-07-06T03:52:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.114+0000 7fe3beffd700 1 -- 172.21.15.97:0/2539493032 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4012bc0 con 0x7fe3c815c520 2023-07-06T03:52:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.114+0000 7fe3d07ff700 1 -- 172.21.15.97:0/2539493032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c815eaf0 con 0x7fe3c815c520 2023-07-06T03:52:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.114+0000 7fe3d07ff700 1 -- 172.21.15.97:0/2539493032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c815f040 con 0x7fe3c815c520 2023-07-06T03:52:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.115+0000 7fe3d07ff700 1 -- 172.21.15.97:0/2539493032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c804e340 con 0x7fe3c815c520 2023-07-06T03:52:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.116+0000 7fe3beffd700 1 -- 172.21.15.97:0/2539493032 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3c4012d20 con 0x7fe3c815c520 2023-07-06T03:52:39.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.117+0000 7fe3beffd700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3ac06ace0 0x7fe3ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.117+0000 7fe3beffd700 1 -- 172.21.15.97:0/2539493032 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3c4089470 con 0x7fe3c815c520 2023-07-06T03:52:39.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.117+0000 7fe3cdd9a700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3ac06ace0 0x7fe3ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.118+0000 7fe3cdd9a700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3ac06ace0 0x7fe3ac06d190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000c00 tx=0x7fe3b8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.122+0000 7fe3beffd700 1 -- 172.21.15.97:0/2539493032 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe3c40545b0 con 0x7fe3c815c520 2023-07-06T03:52:39.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.311+0000 7fe3d07ff700 1 -- 172.21.15.97:0/2539493032 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3c80b6ee0 con 0x7fe3ac06ace0 2023-07-06T03:52:39.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.321+0000 7fe3beffd700 1 -- 172.21.15.97:0/2539493032 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311471 (secure 0 0 0) 0x7fe3b800bb40 con 0x7fe3ac06ace0 2023-07-06T03:52:39.326 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:39.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 -- 172.21.15.97:0/2539493032 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3ac06ace0 msgr2=0x7fe3ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:39.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3ac06ace0 0x7fe3ac06d190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000c00 tx=0x7fe3b8008030 comp rx=0 tx=0).stop 2023-07-06T03:52:39.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 -- 172.21.15.97:0/2539493032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c815c520 msgr2=0x7fe3c8155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:39.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c815c520 0x7fe3c8155fe0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400d800 tx=0x7fe3c400db10 comp rx=0 tx=0).stop 2023-07-06T03:52:39.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 -- 172.21.15.97:0/2539493032 shutdown_connections 2023-07-06T03:52:39.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3ac06ace0 0x7fe3ac06d190 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3c8160b30 0x7fe3c815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c815c520 0x7fe3c8155fe0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 --2- 172.21.15.97:0/2539493032 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3c8160180 0x7fe3c815ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 -- 172.21.15.97:0/2539493032 >> 172.21.15.97:0/2539493032 conn(0x7fe3c8071790 msgr2=0x7fe3c80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:39.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 -- 172.21.15.97:0/2539493032 shutdown_connections 2023-07-06T03:52:39.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.326+0000 7fe3d07ff700 1 -- 172.21.15.97:0/2539493032 wait complete. 2023-07-06T03:52:39.331 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:52:39.346 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2023-07-06T03:52:38.981769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265956,"kb_used_data":9380,"kb_used_omap":0,"kb_used_meta":256512,"kb_avail":754708764,"statfs":{"total":773094113280,"available":772821774336,"internally_reserved":0,"allocated":9605120,"data_stored":4951633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":262668288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001500"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.987252+0000","last_change":"2023-07-06T03:52:34.977882+0000","last_active":"2023-07-06T03:52:34.987252+0000","last_peered":"2023-07-06T03:52:34.987252+0000","last_clean":"2023-07-06T03:52:34.987252+0000","last_became_active":"2023-07-06T03:52:34.977618+0000","last_became_peered":"2023-07-06T03:52:34.977618+0000","last_unstale":"2023-07-06T03:52:34.987252+0000","last_undegraded":"2023-07-06T03:52:34.987252+0000","last_fullsized":"2023-07-06T03:52:34.987252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:11:06.616526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988311+0000","last_change":"2023-07-06T03:52:34.980835+0000","last_active":"2023-07-06T03:52:34.988311+0000","last_peered":"2023-07-06T03:52:34.988311+0000","last_clean":"2023-07-06T03:52:34.988311+0000","last_became_active":"2023-07-06T03:52:34.980386+0000","last_became_peered":"2023-07-06T03:52:34.980386+0000","last_unstale":"2023-07-06T03:52:34.988311+0000","last_undegraded":"2023-07-06T03:52:34.988311+0000","last_fullsized":"2023-07-06T03:52:34.988311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:34:25.363307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958334+0000","last_change":"2023-07-06T03:52:34.975433+0000","last_active":"2023-07-06T03:52:35.958334+0000","last_peered":"2023-07-06T03:52:35.958334+0000","last_clean":"2023-07-06T03:52:35.958334+0000","last_became_active":"2023-07-06T03:52:34.975049+0000","last_became_peered":"2023-07-06T03:52:34.975049+0000","last_unstale":"2023-07-06T03:52:35.958334+0000","last_undegraded":"2023-07-06T03:52:35.958334+0000","last_fullsized":"2023-07-06T03:52:35.958334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:19:02.780761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.957919+0000","last_change":"2023-07-06T03:52:34.986371+0000","last_active":"2023-07-06T03:52:35.957919+0000","last_peered":"2023-07-06T03:52:35.957919+0000","last_clean":"2023-07-06T03:52:35.957919+0000","last_became_active":"2023-07-06T03:52:34.986031+0000","last_became_peered":"2023-07-06T03:52:34.986031+0000","last_unstale":"2023-07-06T03:52:35.957919+0000","last_undegraded":"2023-07-06T03:52:35.957919+0000","last_fullsized":"2023-07-06T03:52:35.957919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:28:50.974864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983120+0000","last_change":"2023-07-06T03:52:34.969896+0000","last_active":"2023-07-06T03:52:34.983120+0000","last_peered":"2023-07-06T03:52:34.983120+0000","last_clean":"2023-07-06T03:52:34.983120+0000","last_became_active":"2023-07-06T03:52:34.969634+0000","last_became_peered":"2023-07-06T03:52:34.969634+0000","last_unstale":"2023-07-06T03:52:34.983120+0000","last_undegraded":"2023-07-06T03:52:34.983120+0000","last_fullsized":"2023-07-06T03:52:34.983120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:37:49.824908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959456+0000","last_change":"2023-07-06T03:52:34.980127+0000","last_active":"2023-07-06T03:52:35.959456+0000","last_peered":"2023-07-06T03:52:35.959456+0000","last_clean":"2023-07-06T03:52:35.959456+0000","last_became_active":"2023-07-06T03:52:34.979408+0000","last_became_peered":"2023-07-06T03:52:34.979408+0000","last_unstale":"2023-07-06T03:52:35.959456+0000","last_undegraded":"2023-07-06T03:52:35.959456+0000","last_fullsized":"2023-07-06T03:52:35.959456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:23:42.598710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958331+0000","last_change":"2023-07-06T03:52:34.980083+0000","last_active":"2023-07-06T03:52:35.958331+0000","last_peered":"2023-07-06T03:52:35.958331+0000","last_clean":"2023-07-06T03:52:35.958331+0000","last_became_active":"2023-07-06T03:52:34.979449+0000","last_became_peered":"2023-07-06T03:52:34.979449+0000","last_unstale":"2023-07-06T03:52:35.958331+0000","last_undegraded":"2023-07-06T03:52:35.958331+0000","last_fullsized":"2023-07-06T03:52:35.958331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:49:21.615646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959214+0000","last_change":"2023-07-06T03:52:34.975517+0000","last_active":"2023-07-06T03:52:35.959214+0000","last_peered":"2023-07-06T03:52:35.959214+0000","last_clean":"2023-07-06T03:52:35.959214+0000","last_became_active":"2023-07-06T03:52:34.975100+0000","last_became_peered":"2023-07-06T03:52:34.975100+0000","last_unstale":"2023-07-06T03:52:35.959214+0000","last_undegraded":"2023-07-06T03:52:35.959214+0000","last_fullsized":"2023-07-06T03:52:35.959214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:19.868969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958610+0000","last_change":"2023-07-06T03:52:34.977802+0000","last_active":"2023-07-06T03:52:35.958610+0000","last_peered":"2023-07-06T03:52:35.958610+0000","last_clean":"2023-07-06T03:52:35.958610+0000","last_became_active":"2023-07-06T03:52:34.977503+0000","last_became_peered":"2023-07-06T03:52:34.977503+0000","last_unstale":"2023-07-06T03:52:35.958610+0000","last_undegraded":"2023-07-06T03:52:35.958610+0000","last_fullsized":"2023-07-06T03:52:35.958610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:05:54.626714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958036+0000","last_change":"2023-07-06T03:52:34.976935+0000","last_active":"2023-07-06T03:52:35.958036+0000","last_peered":"2023-07-06T03:52:35.958036+0000","last_clean":"2023-07-06T03:52:35.958036+0000","last_became_active":"2023-07-06T03:52:34.976572+0000","last_became_peered":"2023-07-06T03:52:34.976572+0000","last_unstale":"2023-07-06T03:52:35.958036+0000","last_undegraded":"2023-07-06T03:52:35.958036+0000","last_fullsized":"2023-07-06T03:52:35.958036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:14:49.245209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958395+0000","last_change":"2023-07-06T03:52:34.980831+0000","last_active":"2023-07-06T03:52:35.958395+0000","last_peered":"2023-07-06T03:52:35.958395+0000","last_clean":"2023-07-06T03:52:35.958395+0000","last_became_active":"2023-07-06T03:52:34.980544+0000","last_became_peered":"2023-07-06T03:52:34.980544+0000","last_unstale":"2023-07-06T03:52:35.958395+0000","last_undegraded":"2023-07-06T03:52:35.958395+0000","last_fullsized":"2023-07-06T03:52:35.958395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:48:52.526357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958064+0000","last_change":"2023-07-06T03:52:34.976587+0000","last_active":"2023-07-06T03:52:35.958064+0000","last_peered":"2023-07-06T03:52:35.958064+0000","last_clean":"2023-07-06T03:52:35.958064+0000","last_became_active":"2023-07-06T03:52:34.976328+0000","last_became_peered":"2023-07-06T03:52:34.976328+0000","last_unstale":"2023-07-06T03:52:35.958064+0000","last_undegraded":"2023-07-06T03:52:35.958064+0000","last_fullsized":"2023-07-06T03:52:35.958064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:10:45.427796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988801+0000","last_change":"2023-07-06T03:52:34.984814+0000","last_active":"2023-07-06T03:52:34.988801+0000","last_peered":"2023-07-06T03:52:34.988801+0000","last_clean":"2023-07-06T03:52:34.988801+0000","last_became_active":"2023-07-06T03:52:34.984567+0000","last_became_peered":"2023-07-06T03:52:34.984567+0000","last_unstale":"2023-07-06T03:52:34.988801+0000","last_undegraded":"2023-07-06T03:52:34.988801+0000","last_fullsized":"2023-07-06T03:52:34.988801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:44:45.115650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.978279+0000","last_change":"2023-07-06T03:52:34.975049+0000","last_active":"2023-07-06T03:52:34.978279+0000","last_peered":"2023-07-06T03:52:34.978279+0000","last_clean":"2023-07-06T03:52:34.978279+0000","last_became_active":"2023-07-06T03:52:34.974774+0000","last_became_peered":"2023-07-06T03:52:34.974774+0000","last_unstale":"2023-07-06T03:52:34.978279+0000","last_undegraded":"2023-07-06T03:52:34.978279+0000","last_fullsized":"2023-07-06T03:52:34.978279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:40:10.226780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958596+0000","last_change":"2023-07-06T03:52:34.978342+0000","last_active":"2023-07-06T03:52:35.958596+0000","last_peered":"2023-07-06T03:52:35.958596+0000","last_clean":"2023-07-06T03:52:35.958596+0000","last_became_active":"2023-07-06T03:52:34.977879+0000","last_became_peered":"2023-07-06T03:52:34.977879+0000","last_unstale":"2023-07-06T03:52:35.958596+0000","last_undegraded":"2023-07-06T03:52:35.958596+0000","last_fullsized":"2023-07-06T03:52:35.958596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:08:54.635642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959232+0000","last_change":"2023-07-06T03:52:34.991737+0000","last_active":"2023-07-06T03:52:35.959232+0000","last_peered":"2023-07-06T03:52:35.959232+0000","last_clean":"2023-07-06T03:52:35.959232+0000","last_became_active":"2023-07-06T03:52:34.991402+0000","last_became_peered":"2023-07-06T03:52:34.991402+0000","last_unstale":"2023-07-06T03:52:35.959232+0000","last_undegraded":"2023-07-06T03:52:35.959232+0000","last_fullsized":"2023-07-06T03:52:35.959232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:50.814912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959701+0000","last_change":"2023-07-06T03:52:34.981157+0000","last_active":"2023-07-06T03:52:35.959701+0000","last_peered":"2023-07-06T03:52:35.959701+0000","last_clean":"2023-07-06T03:52:35.959701+0000","last_became_active":"2023-07-06T03:52:34.980640+0000","last_became_peered":"2023-07-06T03:52:34.980640+0000","last_unstale":"2023-07-06T03:52:35.959701+0000","last_undegraded":"2023-07-06T03:52:35.959701+0000","last_fullsized":"2023-07-06T03:52:35.959701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:21:32.792141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958039+0000","last_change":"2023-07-06T03:52:34.976346+0000","last_active":"2023-07-06T03:52:35.958039+0000","last_peered":"2023-07-06T03:52:35.958039+0000","last_clean":"2023-07-06T03:52:35.958039+0000","last_became_active":"2023-07-06T03:52:34.975892+0000","last_became_peered":"2023-07-06T03:52:34.975892+0000","last_unstale":"2023-07-06T03:52:35.958039+0000","last_undegraded":"2023-07-06T03:52:35.958039+0000","last_fullsized":"2023-07-06T03:52:35.958039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:49:11.015333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986115+0000","last_change":"2023-07-06T03:52:34.982531+0000","last_active":"2023-07-06T03:52:34.986115+0000","last_peered":"2023-07-06T03:52:34.986115+0000","last_clean":"2023-07-06T03:52:34.986115+0000","last_became_active":"2023-07-06T03:52:34.981923+0000","last_became_peered":"2023-07-06T03:52:34.981923+0000","last_unstale":"2023-07-06T03:52:34.986115+0000","last_undegraded":"2023-07-06T03:52:34.986115+0000","last_fullsized":"2023-07-06T03:52:34.986115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:25:53.451281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959670+0000","last_change":"2023-07-06T03:52:34.988638+0000","last_active":"2023-07-06T03:52:35.959670+0000","last_peered":"2023-07-06T03:52:35.959670+0000","last_clean":"2023-07-06T03:52:35.959670+0000","last_became_active":"2023-07-06T03:52:34.988337+0000","last_became_peered":"2023-07-06T03:52:34.988337+0000","last_unstale":"2023-07-06T03:52:35.959670+0000","last_undegraded":"2023-07-06T03:52:35.959670+0000","last_fullsized":"2023-07-06T03:52:35.959670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:08:20.630298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983068+0000","last_change":"2023-07-06T03:52:34.969454+0000","last_active":"2023-07-06T03:52:34.983068+0000","last_peered":"2023-07-06T03:52:34.983068+0000","last_clean":"2023-07-06T03:52:34.983068+0000","last_became_active":"2023-07-06T03:52:34.969192+0000","last_became_peered":"2023-07-06T03:52:34.969192+0000","last_unstale":"2023-07-06T03:52:34.983068+0000","last_undegraded":"2023-07-06T03:52:34.983068+0000","last_fullsized":"2023-07-06T03:52:34.983068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:49.769569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.981333+0000","last_change":"2023-07-06T03:52:34.978581+0000","last_active":"2023-07-06T03:52:34.981333+0000","last_peered":"2023-07-06T03:52:34.981333+0000","last_clean":"2023-07-06T03:52:34.981333+0000","last_became_active":"2023-07-06T03:52:34.978267+0000","last_became_peered":"2023-07-06T03:52:34.978267+0000","last_unstale":"2023-07-06T03:52:34.981333+0000","last_undegraded":"2023-07-06T03:52:34.981333+0000","last_fullsized":"2023-07-06T03:52:34.981333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:29:46.175292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983627+0000","last_change":"2023-07-06T03:52:34.971696+0000","last_active":"2023-07-06T03:52:34.983627+0000","last_peered":"2023-07-06T03:52:34.983627+0000","last_clean":"2023-07-06T03:52:34.983627+0000","last_became_active":"2023-07-06T03:52:34.971389+0000","last_became_peered":"2023-07-06T03:52:34.971389+0000","last_unstale":"2023-07-06T03:52:34.983627+0000","last_undegraded":"2023-07-06T03:52:34.983627+0000","last_fullsized":"2023-07-06T03:52:34.983627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:19:02.780761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.989096+0000","last_change":"2023-07-06T03:52:34.983511+0000","last_active":"2023-07-06T03:52:34.989096+0000","last_peered":"2023-07-06T03:52:34.989096+0000","last_clean":"2023-07-06T03:52:34.989096+0000","last_became_active":"2023-07-06T03:52:34.983028+0000","last_became_peered":"2023-07-06T03:52:34.983028+0000","last_unstale":"2023-07-06T03:52:34.989096+0000","last_undegraded":"2023-07-06T03:52:34.989096+0000","last_fullsized":"2023-07-06T03:52:34.989096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:43:54.846787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983020+0000","last_change":"2023-07-06T03:52:34.968903+0000","last_active":"2023-07-06T03:52:34.983020+0000","last_peered":"2023-07-06T03:52:34.983020+0000","last_clean":"2023-07-06T03:52:34.983020+0000","last_became_active":"2023-07-06T03:52:34.968628+0000","last_became_peered":"2023-07-06T03:52:34.968628+0000","last_unstale":"2023-07-06T03:52:34.983020+0000","last_undegraded":"2023-07-06T03:52:34.983020+0000","last_fullsized":"2023-07-06T03:52:34.983020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:16:14.184623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988288+0000","last_change":"2023-07-06T03:52:34.980658+0000","last_active":"2023-07-06T03:52:34.988288+0000","last_peered":"2023-07-06T03:52:34.988288+0000","last_clean":"2023-07-06T03:52:34.988288+0000","last_became_active":"2023-07-06T03:52:34.980055+0000","last_became_peered":"2023-07-06T03:52:34.980055+0000","last_unstale":"2023-07-06T03:52:34.988288+0000","last_undegraded":"2023-07-06T03:52:34.988288+0000","last_fullsized":"2023-07-06T03:52:34.988288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:48:16.313574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958313+0000","last_change":"2023-07-06T03:52:34.972777+0000","last_active":"2023-07-06T03:52:35.958313+0000","last_peered":"2023-07-06T03:52:35.958313+0000","last_clean":"2023-07-06T03:52:35.958313+0000","last_became_active":"2023-07-06T03:52:34.972514+0000","last_became_peered":"2023-07-06T03:52:34.972514+0000","last_unstale":"2023-07-06T03:52:35.958313+0000","last_undegraded":"2023-07-06T03:52:35.958313+0000","last_fullsized":"2023-07-06T03:52:35.958313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:22:03.328610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986593+0000","last_change":"2023-07-06T03:52:34.980403+0000","last_active":"2023-07-06T03:52:34.986593+0000","last_peered":"2023-07-06T03:52:34.986593+0000","last_clean":"2023-07-06T03:52:34.986593+0000","last_became_active":"2023-07-06T03:52:34.980104+0000","last_became_peered":"2023-07-06T03:52:34.980104+0000","last_unstale":"2023-07-06T03:52:34.986593+0000","last_undegraded":"2023-07-06T03:52:34.986593+0000","last_fullsized":"2023-07-06T03:52:34.986593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:21:57.347621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988798+0000","last_change":"2023-07-06T03:52:34.979952+0000","last_active":"2023-07-06T03:52:34.988798+0000","last_peered":"2023-07-06T03:52:34.988798+0000","last_clean":"2023-07-06T03:52:34.988798+0000","last_became_active":"2023-07-06T03:52:34.979642+0000","last_became_peered":"2023-07-06T03:52:34.979642+0000","last_unstale":"2023-07-06T03:52:34.988798+0000","last_undegraded":"2023-07-06T03:52:34.988798+0000","last_fullsized":"2023-07-06T03:52:34.988798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:19:07.941745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986552+0000","last_change":"2023-07-06T03:52:34.975620+0000","last_active":"2023-07-06T03:52:34.986552+0000","last_peered":"2023-07-06T03:52:34.986552+0000","last_clean":"2023-07-06T03:52:34.986552+0000","last_became_active":"2023-07-06T03:52:34.975282+0000","last_became_peered":"2023-07-06T03:52:34.975282+0000","last_unstale":"2023-07-06T03:52:34.986552+0000","last_undegraded":"2023-07-06T03:52:34.986552+0000","last_fullsized":"2023-07-06T03:52:34.986552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:10:45.427796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986077+0000","last_change":"2023-07-06T03:52:34.984673+0000","last_active":"2023-07-06T03:52:34.986077+0000","last_peered":"2023-07-06T03:52:34.986077+0000","last_clean":"2023-07-06T03:52:34.986077+0000","last_became_active":"2023-07-06T03:52:34.984337+0000","last_became_peered":"2023-07-06T03:52:34.984337+0000","last_unstale":"2023-07-06T03:52:34.986077+0000","last_undegraded":"2023-07-06T03:52:34.986077+0000","last_fullsized":"2023-07-06T03:52:34.986077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:49.655068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959431+0000","last_change":"2023-07-06T03:52:34.989786+0000","last_active":"2023-07-06T03:52:35.959431+0000","last_peered":"2023-07-06T03:52:35.959431+0000","last_clean":"2023-07-06T03:52:35.959431+0000","last_became_active":"2023-07-06T03:52:34.989494+0000","last_became_peered":"2023-07-06T03:52:34.989494+0000","last_unstale":"2023-07-06T03:52:35.959431+0000","last_undegraded":"2023-07-06T03:52:35.959431+0000","last_fullsized":"2023-07-06T03:52:35.959431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:10:14.471097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.980828+0000","last_change":"2023-07-06T03:52:34.975490+0000","last_active":"2023-07-06T03:52:34.980828+0000","last_peered":"2023-07-06T03:52:34.980828+0000","last_clean":"2023-07-06T03:52:34.980828+0000","last_became_active":"2023-07-06T03:52:34.975153+0000","last_became_peered":"2023-07-06T03:52:34.975153+0000","last_unstale":"2023-07-06T03:52:34.980828+0000","last_undegraded":"2023-07-06T03:52:34.980828+0000","last_fullsized":"2023-07-06T03:52:34.980828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:48:20.899074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"85'4","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.990324+0000","last_change":"2023-07-06T03:52:33.972008+0000","last_active":"2023-07-06T03:52:34.990324+0000","last_peered":"2023-07-06T03:52:34.990324+0000","last_clean":"2023-07-06T03:52:34.990324+0000","last_became_active":"2023-07-06T03:52:33.971546+0000","last_became_peered":"2023-07-06T03:52:33.971546+0000","last_unstale":"2023-07-06T03:52:34.990324+0000","last_undegraded":"2023-07-06T03:52:34.990324+0000","last_fullsized":"2023-07-06T03:52:34.990324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:38:47.581991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959954+0000","last_change":"2023-07-06T03:52:34.988755+0000","last_active":"2023-07-06T03:52:35.959954+0000","last_peered":"2023-07-06T03:52:35.959954+0000","last_clean":"2023-07-06T03:52:35.959954+0000","last_became_active":"2023-07-06T03:52:34.988423+0000","last_became_peered":"2023-07-06T03:52:34.988423+0000","last_unstale":"2023-07-06T03:52:35.959954+0000","last_undegraded":"2023-07-06T03:52:35.959954+0000","last_fullsized":"2023-07-06T03:52:35.959954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:42.357746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988566+0000","last_change":"2023-07-06T03:52:33.971055+0000","last_active":"2023-07-06T03:52:34.988566+0000","last_peered":"2023-07-06T03:52:34.988566+0000","last_clean":"2023-07-06T03:52:34.988566+0000","last_became_active":"2023-07-06T03:52:33.970732+0000","last_became_peered":"2023-07-06T03:52:33.970732+0000","last_unstale":"2023-07-06T03:52:34.988566+0000","last_undegraded":"2023-07-06T03:52:34.988566+0000","last_fullsized":"2023-07-06T03:52:34.988566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:34:41.448578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958526+0000","last_change":"2023-07-06T03:52:34.982511+0000","last_active":"2023-07-06T03:52:35.958526+0000","last_peered":"2023-07-06T03:52:35.958526+0000","last_clean":"2023-07-06T03:52:35.958526+0000","last_became_active":"2023-07-06T03:52:34.982247+0000","last_became_peered":"2023-07-06T03:52:34.982247+0000","last_unstale":"2023-07-06T03:52:35.958526+0000","last_undegraded":"2023-07-06T03:52:35.958526+0000","last_fullsized":"2023-07-06T03:52:35.958526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:14.879632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.991210+0000","last_change":"2023-07-06T03:52:33.969562+0000","last_active":"2023-07-06T03:52:34.991210+0000","last_peered":"2023-07-06T03:52:34.991210+0000","last_clean":"2023-07-06T03:52:34.991210+0000","last_became_active":"2023-07-06T03:52:33.969319+0000","last_became_peered":"2023-07-06T03:52:33.969319+0000","last_unstale":"2023-07-06T03:52:34.991210+0000","last_undegraded":"2023-07-06T03:52:34.991210+0000","last_fullsized":"2023-07-06T03:52:34.991210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:27:47.174750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.957986+0000","last_change":"2023-07-06T03:52:34.974988+0000","last_active":"2023-07-06T03:52:35.957986+0000","last_peered":"2023-07-06T03:52:35.957986+0000","last_clean":"2023-07-06T03:52:35.957986+0000","last_became_active":"2023-07-06T03:52:34.974707+0000","last_became_peered":"2023-07-06T03:52:34.974707+0000","last_unstale":"2023-07-06T03:52:35.957986+0000","last_undegraded":"2023-07-06T03:52:35.957986+0000","last_fullsized":"2023-07-06T03:52:35.957986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:17:34.625196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.985530+0000","last_change":"2023-07-06T03:52:33.975234+0000","last_active":"2023-07-06T03:52:34.985530+0000","last_peered":"2023-07-06T03:52:34.985530+0000","last_clean":"2023-07-06T03:52:34.985530+0000","last_became_active":"2023-07-06T03:52:33.974995+0000","last_became_peered":"2023-07-06T03:52:33.974995+0000","last_unstale":"2023-07-06T03:52:34.985530+0000","last_undegraded":"2023-07-06T03:52:34.985530+0000","last_fullsized":"2023-07-06T03:52:34.985530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:13:49.605835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.990296+0000","last_change":"2023-07-06T03:52:34.985412+0000","last_active":"2023-07-06T03:52:34.990296+0000","last_peered":"2023-07-06T03:52:34.990296+0000","last_clean":"2023-07-06T03:52:34.990296+0000","last_became_active":"2023-07-06T03:52:34.985068+0000","last_became_peered":"2023-07-06T03:52:34.985068+0000","last_unstale":"2023-07-06T03:52:34.990296+0000","last_undegraded":"2023-07-06T03:52:34.990296+0000","last_fullsized":"2023-07-06T03:52:34.990296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:00:51.296286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.981543+0000","last_change":"2023-07-06T03:52:33.962873+0000","last_active":"2023-07-06T03:52:34.981543+0000","last_peered":"2023-07-06T03:52:34.981543+0000","last_clean":"2023-07-06T03:52:34.981543+0000","last_became_active":"2023-07-06T03:52:33.962628+0000","last_became_peered":"2023-07-06T03:52:33.962628+0000","last_unstale":"2023-07-06T03:52:34.981543+0000","last_undegraded":"2023-07-06T03:52:34.981543+0000","last_fullsized":"2023-07-06T03:52:34.981543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:56:13.294284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958256+0000","last_change":"2023-07-06T03:52:34.980277+0000","last_active":"2023-07-06T03:52:35.958256+0000","last_peered":"2023-07-06T03:52:35.958256+0000","last_clean":"2023-07-06T03:52:35.958256+0000","last_became_active":"2023-07-06T03:52:34.979833+0000","last_became_peered":"2023-07-06T03:52:34.979833+0000","last_unstale":"2023-07-06T03:52:35.958256+0000","last_undegraded":"2023-07-06T03:52:35.958256+0000","last_fullsized":"2023-07-06T03:52:35.958256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:34:50.108422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.991350+0000","last_change":"2023-07-06T03:52:33.971390+0000","last_active":"2023-07-06T03:52:34.991350+0000","last_peered":"2023-07-06T03:52:34.991350+0000","last_clean":"2023-07-06T03:52:34.991350+0000","last_became_active":"2023-07-06T03:52:33.971107+0000","last_became_peered":"2023-07-06T03:52:33.971107+0000","last_unstale":"2023-07-06T03:52:34.991350+0000","last_undegraded":"2023-07-06T03:52:34.991350+0000","last_fullsized":"2023-07-06T03:52:34.991350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:43.344086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959883+0000","last_change":"2023-07-06T03:52:34.980322+0000","last_active":"2023-07-06T03:52:35.959883+0000","last_peered":"2023-07-06T03:52:35.959883+0000","last_clean":"2023-07-06T03:52:35.959883+0000","last_became_active":"2023-07-06T03:52:34.979784+0000","last_became_peered":"2023-07-06T03:52:34.979784+0000","last_unstale":"2023-07-06T03:52:35.959883+0000","last_undegraded":"2023-07-06T03:52:35.959883+0000","last_fullsized":"2023-07-06T03:52:35.959883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:56:11.702031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958491+0000","last_change":"2023-07-06T03:52:33.962319+0000","last_active":"2023-07-06T03:52:35.958491+0000","last_peered":"2023-07-06T03:52:35.958491+0000","last_clean":"2023-07-06T03:52:35.958491+0000","last_became_active":"2023-07-06T03:52:33.961990+0000","last_became_peered":"2023-07-06T03:52:33.961990+0000","last_unstale":"2023-07-06T03:52:35.958491+0000","last_undegraded":"2023-07-06T03:52:35.958491+0000","last_fullsized":"2023-07-06T03:52:35.958491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:14:13.492083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988499+0000","last_change":"2023-07-06T03:52:34.980267+0000","last_active":"2023-07-06T03:52:34.988499+0000","last_peered":"2023-07-06T03:52:34.988499+0000","last_clean":"2023-07-06T03:52:34.988499+0000","last_became_active":"2023-07-06T03:52:34.979997+0000","last_became_peered":"2023-07-06T03:52:34.979997+0000","last_unstale":"2023-07-06T03:52:34.988499+0000","last_undegraded":"2023-07-06T03:52:34.988499+0000","last_fullsized":"2023-07-06T03:52:34.988499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:14:10.352742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.958261+0000","last_change":"2023-07-06T03:52:33.971835+0000","last_active":"2023-07-06T03:52:34.958261+0000","last_peered":"2023-07-06T03:52:34.958261+0000","last_clean":"2023-07-06T03:52:34.958261+0000","last_became_active":"2023-07-06T03:52:33.971183+0000","last_became_peered":"2023-07-06T03:52:33.971183+0000","last_unstale":"2023-07-06T03:52:34.958261+0000","last_undegraded":"2023-07-06T03:52:34.958261+0000","last_fullsized":"2023-07-06T03:52:34.958261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:07:56.417410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.987781+0000","last_change":"2023-07-06T03:52:34.979613+0000","last_active":"2023-07-06T03:52:34.987781+0000","last_peered":"2023-07-06T03:52:34.987781+0000","last_clean":"2023-07-06T03:52:34.987781+0000","last_became_active":"2023-07-06T03:52:34.979316+0000","last_became_peered":"2023-07-06T03:52:34.979316+0000","last_unstale":"2023-07-06T03:52:34.987781+0000","last_undegraded":"2023-07-06T03:52:34.987781+0000","last_fullsized":"2023-07-06T03:52:34.987781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:22:03.328610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.960568+0000","last_change":"2023-07-06T03:52:33.971837+0000","last_active":"2023-07-06T03:52:34.960568+0000","last_peered":"2023-07-06T03:52:34.960568+0000","last_clean":"2023-07-06T03:52:34.960568+0000","last_became_active":"2023-07-06T03:52:33.971201+0000","last_became_peered":"2023-07-06T03:52:33.971201+0000","last_unstale":"2023-07-06T03:52:34.960568+0000","last_undegraded":"2023-07-06T03:52:34.960568+0000","last_fullsized":"2023-07-06T03:52:34.960568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:08:31.234265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958561+0000","last_change":"2023-07-06T03:52:34.978612+0000","last_active":"2023-07-06T03:52:35.958561+0000","last_peered":"2023-07-06T03:52:35.958561+0000","last_clean":"2023-07-06T03:52:35.958561+0000","last_became_active":"2023-07-06T03:52:34.978347+0000","last_became_peered":"2023-07-06T03:52:34.978347+0000","last_unstale":"2023-07-06T03:52:35.958561+0000","last_undegraded":"2023-07-06T03:52:35.958561+0000","last_fullsized":"2023-07-06T03:52:35.958561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:35.539652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958294+0000","last_change":"2023-07-06T03:52:33.962485+0000","last_active":"2023-07-06T03:52:35.958294+0000","last_peered":"2023-07-06T03:52:35.958294+0000","last_clean":"2023-07-06T03:52:35.958294+0000","last_became_active":"2023-07-06T03:52:33.962168+0000","last_became_peered":"2023-07-06T03:52:33.962168+0000","last_unstale":"2023-07-06T03:52:35.958294+0000","last_undegraded":"2023-07-06T03:52:35.958294+0000","last_fullsized":"2023-07-06T03:52:35.958294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:45.645931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.987779+0000","last_change":"2023-07-06T03:52:34.975836+0000","last_active":"2023-07-06T03:52:34.987779+0000","last_peered":"2023-07-06T03:52:34.987779+0000","last_clean":"2023-07-06T03:52:34.987779+0000","last_became_active":"2023-07-06T03:52:34.975547+0000","last_became_peered":"2023-07-06T03:52:34.975547+0000","last_unstale":"2023-07-06T03:52:34.987779+0000","last_undegraded":"2023-07-06T03:52:34.987779+0000","last_fullsized":"2023-07-06T03:52:34.987779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:41:42.215703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"85'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958711+0000","last_change":"2023-07-06T03:52:33.971177+0000","last_active":"2023-07-06T03:52:35.958711+0000","last_peered":"2023-07-06T03:52:35.958711+0000","last_clean":"2023-07-06T03:52:35.958711+0000","last_became_active":"2023-07-06T03:52:33.970653+0000","last_became_peered":"2023-07-06T03:52:33.970653+0000","last_unstale":"2023-07-06T03:52:35.958711+0000","last_undegraded":"2023-07-06T03:52:35.958711+0000","last_fullsized":"2023-07-06T03:52:35.958711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:57:50.263331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959845+0000","last_change":"2023-07-06T03:52:34.982461+0000","last_active":"2023-07-06T03:52:35.959845+0000","last_peered":"2023-07-06T03:52:35.959845+0000","last_clean":"2023-07-06T03:52:35.959845+0000","last_became_active":"2023-07-06T03:52:34.982192+0000","last_became_peered":"2023-07-06T03:52:34.982192+0000","last_unstale":"2023-07-06T03:52:35.959845+0000","last_undegraded":"2023-07-06T03:52:35.959845+0000","last_fullsized":"2023-07-06T03:52:35.959845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:02.793520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.994073+0000","last_change":"2023-07-06T03:52:33.969663+0000","last_active":"2023-07-06T03:52:34.994073+0000","last_peered":"2023-07-06T03:52:34.994073+0000","last_clean":"2023-07-06T03:52:34.994073+0000","last_became_active":"2023-07-06T03:52:33.969291+0000","last_became_peered":"2023-07-06T03:52:33.969291+0000","last_unstale":"2023-07-06T03:52:34.994073+0000","last_undegraded":"2023-07-06T03:52:34.994073+0000","last_fullsized":"2023-07-06T03:52:34.994073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:58:06.580256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.989076+0000","last_change":"2023-07-06T03:52:34.983292+0000","last_active":"2023-07-06T03:52:34.989076+0000","last_peered":"2023-07-06T03:52:34.989076+0000","last_clean":"2023-07-06T03:52:34.989076+0000","last_became_active":"2023-07-06T03:52:34.982689+0000","last_became_peered":"2023-07-06T03:52:34.982689+0000","last_unstale":"2023-07-06T03:52:34.989076+0000","last_undegraded":"2023-07-06T03:52:34.989076+0000","last_fullsized":"2023-07-06T03:52:34.989076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:18:55.170333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.991708+0000","last_change":"2023-07-06T03:52:33.969641+0000","last_active":"2023-07-06T03:52:34.991708+0000","last_peered":"2023-07-06T03:52:34.991708+0000","last_clean":"2023-07-06T03:52:34.991708+0000","last_became_active":"2023-07-06T03:52:33.969334+0000","last_became_peered":"2023-07-06T03:52:33.969334+0000","last_unstale":"2023-07-06T03:52:34.991708+0000","last_undegraded":"2023-07-06T03:52:34.991708+0000","last_fullsized":"2023-07-06T03:52:34.991708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:31:26.756116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983648+0000","last_change":"2023-07-06T03:52:34.975282+0000","last_active":"2023-07-06T03:52:34.983648+0000","last_peered":"2023-07-06T03:52:34.983648+0000","last_clean":"2023-07-06T03:52:34.983648+0000","last_became_active":"2023-07-06T03:52:34.975010+0000","last_became_peered":"2023-07-06T03:52:34.975010+0000","last_unstale":"2023-07-06T03:52:34.983648+0000","last_undegraded":"2023-07-06T03:52:34.983648+0000","last_fullsized":"2023-07-06T03:52:34.983648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:08:54.635642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.965449+0000","last_change":"2023-07-06T03:52:33.970052+0000","last_active":"2023-07-06T03:52:34.965449+0000","last_peered":"2023-07-06T03:52:34.965449+0000","last_clean":"2023-07-06T03:52:34.965449+0000","last_became_active":"2023-07-06T03:52:33.969777+0000","last_became_peered":"2023-07-06T03:52:33.969777+0000","last_unstale":"2023-07-06T03:52:34.965449+0000","last_undegraded":"2023-07-06T03:52:34.965449+0000","last_fullsized":"2023-07-06T03:52:34.965449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:07:30.791888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986968+0000","last_change":"2023-07-06T03:52:34.975902+0000","last_active":"2023-07-06T03:52:34.986968+0000","last_peered":"2023-07-06T03:52:34.986968+0000","last_clean":"2023-07-06T03:52:34.986968+0000","last_became_active":"2023-07-06T03:52:34.975592+0000","last_became_peered":"2023-07-06T03:52:34.975592+0000","last_unstale":"2023-07-06T03:52:34.986968+0000","last_undegraded":"2023-07-06T03:52:34.986968+0000","last_fullsized":"2023-07-06T03:52:34.986968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:13:36.660096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.961307+0000","last_change":"2023-07-06T03:52:33.974022+0000","last_active":"2023-07-06T03:52:34.961307+0000","last_peered":"2023-07-06T03:52:34.961307+0000","last_clean":"2023-07-06T03:52:34.961307+0000","last_became_active":"2023-07-06T03:52:33.973660+0000","last_became_peered":"2023-07-06T03:52:33.973660+0000","last_unstale":"2023-07-06T03:52:34.961307+0000","last_undegraded":"2023-07-06T03:52:34.961307+0000","last_fullsized":"2023-07-06T03:52:34.961307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:51:13.802000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988783+0000","last_change":"2023-07-06T03:52:34.981246+0000","last_active":"2023-07-06T03:52:34.988783+0000","last_peered":"2023-07-06T03:52:34.988783+0000","last_clean":"2023-07-06T03:52:34.988783+0000","last_became_active":"2023-07-06T03:52:34.980967+0000","last_became_peered":"2023-07-06T03:52:34.980967+0000","last_unstale":"2023-07-06T03:52:34.988783+0000","last_undegraded":"2023-07-06T03:52:34.988783+0000","last_fullsized":"2023-07-06T03:52:34.988783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:01.473197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.957695+0000","last_change":"2023-07-06T03:52:33.970438+0000","last_active":"2023-07-06T03:52:35.957695+0000","last_peered":"2023-07-06T03:52:35.957695+0000","last_clean":"2023-07-06T03:52:35.957695+0000","last_became_active":"2023-07-06T03:52:33.970151+0000","last_became_peered":"2023-07-06T03:52:33.970151+0000","last_unstale":"2023-07-06T03:52:35.957695+0000","last_undegraded":"2023-07-06T03:52:35.957695+0000","last_fullsized":"2023-07-06T03:52:35.957695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:28:25.343977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.981436+0000","last_change":"2023-07-06T03:52:34.978791+0000","last_active":"2023-07-06T03:52:34.981436+0000","last_peered":"2023-07-06T03:52:34.981436+0000","last_clean":"2023-07-06T03:52:34.981436+0000","last_became_active":"2023-07-06T03:52:34.978512+0000","last_became_peered":"2023-07-06T03:52:34.978512+0000","last_unstale":"2023-07-06T03:52:34.981436+0000","last_undegraded":"2023-07-06T03:52:34.981436+0000","last_fullsized":"2023-07-06T03:52:34.981436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:14:52.498021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958739+0000","last_change":"2023-07-06T03:52:33.970072+0000","last_active":"2023-07-06T03:52:35.958739+0000","last_peered":"2023-07-06T03:52:35.958739+0000","last_clean":"2023-07-06T03:52:35.958739+0000","last_became_active":"2023-07-06T03:52:33.969827+0000","last_became_peered":"2023-07-06T03:52:33.969827+0000","last_unstale":"2023-07-06T03:52:35.958739+0000","last_undegraded":"2023-07-06T03:52:35.958739+0000","last_fullsized":"2023-07-06T03:52:35.958739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:36:38.665055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988749+0000","last_change":"2023-07-06T03:52:34.981416+0000","last_active":"2023-07-06T03:52:34.988749+0000","last_peered":"2023-07-06T03:52:34.988749+0000","last_clean":"2023-07-06T03:52:34.988749+0000","last_became_active":"2023-07-06T03:52:34.981122+0000","last_became_peered":"2023-07-06T03:52:34.981122+0000","last_unstale":"2023-07-06T03:52:34.988749+0000","last_undegraded":"2023-07-06T03:52:34.988749+0000","last_fullsized":"2023-07-06T03:52:34.988749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:42:25.798222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958574+0000","last_change":"2023-07-06T03:52:33.967889+0000","last_active":"2023-07-06T03:52:35.958574+0000","last_peered":"2023-07-06T03:52:35.958574+0000","last_clean":"2023-07-06T03:52:35.958574+0000","last_became_active":"2023-07-06T03:52:33.967625+0000","last_became_peered":"2023-07-06T03:52:33.967625+0000","last_unstale":"2023-07-06T03:52:35.958574+0000","last_undegraded":"2023-07-06T03:52:35.958574+0000","last_fullsized":"2023-07-06T03:52:35.958574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:04.279525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.985073+0000","last_change":"2023-07-06T03:52:34.972471+0000","last_active":"2023-07-06T03:52:34.985073+0000","last_peered":"2023-07-06T03:52:34.985073+0000","last_clean":"2023-07-06T03:52:34.985073+0000","last_became_active":"2023-07-06T03:52:34.972167+0000","last_became_peered":"2023-07-06T03:52:34.972167+0000","last_unstale":"2023-07-06T03:52:34.985073+0000","last_undegraded":"2023-07-06T03:52:34.985073+0000","last_fullsized":"2023-07-06T03:52:34.985073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:25:26.256618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.969740+0000","last_change":"2023-07-06T03:52:33.962367+0000","last_active":"2023-07-06T03:52:34.969740+0000","last_peered":"2023-07-06T03:52:34.969740+0000","last_clean":"2023-07-06T03:52:34.969740+0000","last_became_active":"2023-07-06T03:52:33.962012+0000","last_became_peered":"2023-07-06T03:52:33.962012+0000","last_unstale":"2023-07-06T03:52:34.969740+0000","last_undegraded":"2023-07-06T03:52:34.969740+0000","last_fullsized":"2023-07-06T03:52:34.969740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:59:29.963987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.991020+0000","last_change":"2023-07-06T03:52:34.990137+0000","last_active":"2023-07-06T03:52:34.991020+0000","last_peered":"2023-07-06T03:52:34.991020+0000","last_clean":"2023-07-06T03:52:34.991020+0000","last_became_active":"2023-07-06T03:52:34.989665+0000","last_became_peered":"2023-07-06T03:52:34.989665+0000","last_unstale":"2023-07-06T03:52:34.991020+0000","last_undegraded":"2023-07-06T03:52:34.991020+0000","last_fullsized":"2023-07-06T03:52:34.991020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:43:37.905199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958656+0000","last_change":"2023-07-06T03:52:33.972579+0000","last_active":"2023-07-06T03:52:35.958656+0000","last_peered":"2023-07-06T03:52:35.958656+0000","last_clean":"2023-07-06T03:52:35.958656+0000","last_became_active":"2023-07-06T03:52:33.972337+0000","last_became_peered":"2023-07-06T03:52:33.972337+0000","last_unstale":"2023-07-06T03:52:35.958656+0000","last_undegraded":"2023-07-06T03:52:35.958656+0000","last_fullsized":"2023-07-06T03:52:35.958656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:39:11.615424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959613+0000","last_change":"2023-07-06T03:52:34.971572+0000","last_active":"2023-07-06T03:52:35.959613+0000","last_peered":"2023-07-06T03:52:35.959613+0000","last_clean":"2023-07-06T03:52:35.959613+0000","last_became_active":"2023-07-06T03:52:34.971273+0000","last_became_peered":"2023-07-06T03:52:34.971273+0000","last_unstale":"2023-07-06T03:52:35.959613+0000","last_undegraded":"2023-07-06T03:52:35.959613+0000","last_fullsized":"2023-07-06T03:52:35.959613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:34:32.890102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.962685+0000","last_change":"2023-07-06T03:52:33.965584+0000","last_active":"2023-07-06T03:52:34.962685+0000","last_peered":"2023-07-06T03:52:34.962685+0000","last_clean":"2023-07-06T03:52:34.962685+0000","last_became_active":"2023-07-06T03:52:33.964659+0000","last_became_peered":"2023-07-06T03:52:33.964659+0000","last_unstale":"2023-07-06T03:52:34.962685+0000","last_undegraded":"2023-07-06T03:52:34.962685+0000","last_fullsized":"2023-07-06T03:52:34.962685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:40.458505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958453+0000","last_change":"2023-07-06T03:52:34.986293+0000","last_active":"2023-07-06T03:52:35.958453+0000","last_peered":"2023-07-06T03:52:35.958453+0000","last_clean":"2023-07-06T03:52:35.958453+0000","last_became_active":"2023-07-06T03:52:34.985619+0000","last_became_peered":"2023-07-06T03:52:34.985619+0000","last_unstale":"2023-07-06T03:52:35.958453+0000","last_undegraded":"2023-07-06T03:52:35.958453+0000","last_fullsized":"2023-07-06T03:52:35.958453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:16:14.184623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.957526+0000","last_change":"2023-07-06T03:52:33.970691+0000","last_active":"2023-07-06T03:52:34.957526+0000","last_peered":"2023-07-06T03:52:34.957526+0000","last_clean":"2023-07-06T03:52:34.957526+0000","last_became_active":"2023-07-06T03:52:33.970396+0000","last_became_peered":"2023-07-06T03:52:33.970396+0000","last_unstale":"2023-07-06T03:52:34.957526+0000","last_undegraded":"2023-07-06T03:52:34.957526+0000","last_fullsized":"2023-07-06T03:52:34.957526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:45.645931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986733+0000","last_change":"2023-07-06T03:52:34.985020+0000","last_active":"2023-07-06T03:52:34.986733+0000","last_peered":"2023-07-06T03:52:34.986733+0000","last_clean":"2023-07-06T03:52:34.986733+0000","last_became_active":"2023-07-06T03:52:34.984742+0000","last_became_peered":"2023-07-06T03:52:34.984742+0000","last_unstale":"2023-07-06T03:52:34.986733+0000","last_undegraded":"2023-07-06T03:52:34.986733+0000","last_fullsized":"2023-07-06T03:52:34.986733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:17:22.458751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.958230+0000","last_change":"2023-07-06T03:52:33.972861+0000","last_active":"2023-07-06T03:52:34.958230+0000","last_peered":"2023-07-06T03:52:34.958230+0000","last_clean":"2023-07-06T03:52:34.958230+0000","last_became_active":"2023-07-06T03:52:33.972618+0000","last_became_peered":"2023-07-06T03:52:33.972618+0000","last_unstale":"2023-07-06T03:52:34.958230+0000","last_undegraded":"2023-07-06T03:52:34.958230+0000","last_fullsized":"2023-07-06T03:52:34.958230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:01:12.042090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959576+0000","last_change":"2023-07-06T03:52:34.989323+0000","last_active":"2023-07-06T03:52:35.959576+0000","last_peered":"2023-07-06T03:52:35.959576+0000","last_clean":"2023-07-06T03:52:35.959576+0000","last_became_active":"2023-07-06T03:52:34.989002+0000","last_became_peered":"2023-07-06T03:52:34.989002+0000","last_unstale":"2023-07-06T03:52:35.959576+0000","last_undegraded":"2023-07-06T03:52:35.959576+0000","last_fullsized":"2023-07-06T03:52:35.959576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:16:23.267541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.957018+0000","last_change":"2023-07-06T03:52:33.971920+0000","last_active":"2023-07-06T03:52:34.957018+0000","last_peered":"2023-07-06T03:52:34.957018+0000","last_clean":"2023-07-06T03:52:34.957018+0000","last_became_active":"2023-07-06T03:52:33.971535+0000","last_became_peered":"2023-07-06T03:52:33.971535+0000","last_unstale":"2023-07-06T03:52:34.957018+0000","last_undegraded":"2023-07-06T03:52:34.957018+0000","last_fullsized":"2023-07-06T03:52:34.957018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:49:10.006497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959935+0000","last_change":"2023-07-06T03:52:34.971829+0000","last_active":"2023-07-06T03:52:35.959935+0000","last_peered":"2023-07-06T03:52:35.959935+0000","last_clean":"2023-07-06T03:52:35.959935+0000","last_became_active":"2023-07-06T03:52:34.971447+0000","last_became_peered":"2023-07-06T03:52:34.971447+0000","last_unstale":"2023-07-06T03:52:35.959935+0000","last_undegraded":"2023-07-06T03:52:35.959935+0000","last_fullsized":"2023-07-06T03:52:35.959935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:26:01.534419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958348+0000","last_change":"2023-07-06T03:52:33.970798+0000","last_active":"2023-07-06T03:52:35.958348+0000","last_peered":"2023-07-06T03:52:35.958348+0000","last_clean":"2023-07-06T03:52:35.958348+0000","last_became_active":"2023-07-06T03:52:33.970470+0000","last_became_peered":"2023-07-06T03:52:33.970470+0000","last_unstale":"2023-07-06T03:52:35.958348+0000","last_undegraded":"2023-07-06T03:52:35.958348+0000","last_fullsized":"2023-07-06T03:52:35.958348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:55:47.828003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":130,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.969121+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:52:34.969121+0000","last_peered":"2023-07-06T03:52:34.969121+0000","last_clean":"2023-07-06T03:52:34.969121+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:52:34.969121+0000","last_undegraded":"2023-07-06T03:52:34.969121+0000","last_fullsized":"2023-07-06T03:52:34.969121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:19.738306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983689+0000","last_change":"2023-07-06T03:52:34.970874+0000","last_active":"2023-07-06T03:52:34.983689+0000","last_peered":"2023-07-06T03:52:34.983689+0000","last_clean":"2023-07-06T03:52:34.983689+0000","last_became_active":"2023-07-06T03:52:34.970612+0000","last_became_peered":"2023-07-06T03:52:34.970612+0000","last_unstale":"2023-07-06T03:52:34.983689+0000","last_undegraded":"2023-07-06T03:52:34.983689+0000","last_fullsized":"2023-07-06T03:52:34.983689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:38:58.774100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.989268+0000","last_change":"2023-07-06T03:52:33.974733+0000","last_active":"2023-07-06T03:52:34.989268+0000","last_peered":"2023-07-06T03:52:34.989268+0000","last_clean":"2023-07-06T03:52:34.989268+0000","last_became_active":"2023-07-06T03:52:33.974396+0000","last_became_peered":"2023-07-06T03:52:33.974396+0000","last_unstale":"2023-07-06T03:52:34.989268+0000","last_undegraded":"2023-07-06T03:52:34.989268+0000","last_fullsized":"2023-07-06T03:52:34.989268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:24:20.059347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958184+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:52:35.958184+0000","last_peered":"2023-07-06T03:52:35.958184+0000","last_clean":"2023-07-06T03:52:35.958184+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:52:35.958184+0000","last_undegraded":"2023-07-06T03:52:35.958184+0000","last_fullsized":"2023-07-06T03:52:35.958184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:39:32.576440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958681+0000","last_change":"2023-07-06T03:52:34.981298+0000","last_active":"2023-07-06T03:52:35.958681+0000","last_peered":"2023-07-06T03:52:35.958681+0000","last_clean":"2023-07-06T03:52:35.958681+0000","last_became_active":"2023-07-06T03:52:34.981044+0000","last_became_peered":"2023-07-06T03:52:34.981044+0000","last_unstale":"2023-07-06T03:52:35.958681+0000","last_undegraded":"2023-07-06T03:52:35.958681+0000","last_fullsized":"2023-07-06T03:52:35.958681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:20:07.915721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.987001+0000","last_change":"2023-07-06T03:52:33.974721+0000","last_active":"2023-07-06T03:52:34.987001+0000","last_peered":"2023-07-06T03:52:34.987001+0000","last_clean":"2023-07-06T03:52:34.987001+0000","last_became_active":"2023-07-06T03:52:33.974466+0000","last_became_peered":"2023-07-06T03:52:33.974466+0000","last_unstale":"2023-07-06T03:52:34.987001+0000","last_undegraded":"2023-07-06T03:52:34.987001+0000","last_fullsized":"2023-07-06T03:52:34.987001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:41:12.435532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.957803+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:52:35.957803+0000","last_peered":"2023-07-06T03:52:35.957803+0000","last_clean":"2023-07-06T03:52:35.957803+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:52:35.957803+0000","last_undegraded":"2023-07-06T03:52:35.957803+0000","last_fullsized":"2023-07-06T03:52:35.957803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:45:51.126881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.989727+0000","last_change":"2023-07-06T03:52:34.976773+0000","last_active":"2023-07-06T03:52:34.989727+0000","last_peered":"2023-07-06T03:52:34.989727+0000","last_clean":"2023-07-06T03:52:34.989727+0000","last_became_active":"2023-07-06T03:52:34.976393+0000","last_became_peered":"2023-07-06T03:52:34.976393+0000","last_unstale":"2023-07-06T03:52:34.989727+0000","last_undegraded":"2023-07-06T03:52:34.989727+0000","last_fullsized":"2023-07-06T03:52:34.989727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:21:05.045222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.962472+0000","last_change":"2023-07-06T03:52:33.970885+0000","last_active":"2023-07-06T03:52:34.962472+0000","last_peered":"2023-07-06T03:52:34.962472+0000","last_clean":"2023-07-06T03:52:34.962472+0000","last_became_active":"2023-07-06T03:52:33.970603+0000","last_became_peered":"2023-07-06T03:52:33.970603+0000","last_unstale":"2023-07-06T03:52:34.962472+0000","last_undegraded":"2023-07-06T03:52:34.962472+0000","last_fullsized":"2023-07-06T03:52:34.962472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:54:26.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958482+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:52:35.958482+0000","last_peered":"2023-07-06T03:52:35.958482+0000","last_clean":"2023-07-06T03:52:35.958482+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:52:35.958482+0000","last_undegraded":"2023-07-06T03:52:35.958482+0000","last_fullsized":"2023-07-06T03:52:35.958482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:45.067333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":278,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.957848+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:52:34.957848+0000","last_peered":"2023-07-06T03:52:34.957848+0000","last_clean":"2023-07-06T03:52:34.957848+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:52:34.957848+0000","last_undegraded":"2023-07-06T03:52:34.957848+0000","last_fullsized":"2023-07-06T03:52:34.957848+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:06:37.636080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986572+0000","last_change":"2023-07-06T03:52:34.980363+0000","last_active":"2023-07-06T03:52:34.986572+0000","last_peered":"2023-07-06T03:52:34.986572+0000","last_clean":"2023-07-06T03:52:34.986572+0000","last_became_active":"2023-07-06T03:52:34.980037+0000","last_became_peered":"2023-07-06T03:52:34.980037+0000","last_unstale":"2023-07-06T03:52:34.986572+0000","last_undegraded":"2023-07-06T03:52:34.986572+0000","last_fullsized":"2023-07-06T03:52:34.986572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:44:17.184944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.979511+0000","last_change":"2023-07-06T03:52:33.966585+0000","last_active":"2023-07-06T03:52:34.979511+0000","last_peered":"2023-07-06T03:52:34.979511+0000","last_clean":"2023-07-06T03:52:34.979511+0000","last_became_active":"2023-07-06T03:52:33.966320+0000","last_became_peered":"2023-07-06T03:52:33.966320+0000","last_unstale":"2023-07-06T03:52:34.979511+0000","last_undegraded":"2023-07-06T03:52:34.979511+0000","last_fullsized":"2023-07-06T03:52:34.979511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:07:33.793257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958173+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:52:35.958173+0000","last_peered":"2023-07-06T03:52:35.958173+0000","last_clean":"2023-07-06T03:52:35.958173+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:52:35.958173+0000","last_undegraded":"2023-07-06T03:52:35.958173+0000","last_fullsized":"2023-07-06T03:52:35.958173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:17:04.875422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.985660+0000","last_change":"2023-07-06T03:52:34.975165+0000","last_active":"2023-07-06T03:52:34.985660+0000","last_peered":"2023-07-06T03:52:34.985660+0000","last_clean":"2023-07-06T03:52:34.985660+0000","last_became_active":"2023-07-06T03:52:34.974821+0000","last_became_peered":"2023-07-06T03:52:34.974821+0000","last_unstale":"2023-07-06T03:52:34.985660+0000","last_undegraded":"2023-07-06T03:52:34.985660+0000","last_fullsized":"2023-07-06T03:52:34.985660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:33:17.996105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959871+0000","last_change":"2023-07-06T03:52:33.972725+0000","last_active":"2023-07-06T03:52:35.959871+0000","last_peered":"2023-07-06T03:52:35.959871+0000","last_clean":"2023-07-06T03:52:35.959871+0000","last_became_active":"2023-07-06T03:52:33.972458+0000","last_became_peered":"2023-07-06T03:52:33.972458+0000","last_unstale":"2023-07-06T03:52:35.959871+0000","last_undegraded":"2023-07-06T03:52:35.959871+0000","last_fullsized":"2023-07-06T03:52:35.959871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:05:10.364515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958761+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:52:35.958761+0000","last_peered":"2023-07-06T03:52:35.958761+0000","last_clean":"2023-07-06T03:52:35.958761+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:52:35.958761+0000","last_undegraded":"2023-07-06T03:52:35.958761+0000","last_fullsized":"2023-07-06T03:52:35.958761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:49:54.804514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.980981+0000","last_change":"2023-07-06T03:52:34.974983+0000","last_active":"2023-07-06T03:52:34.980981+0000","last_peered":"2023-07-06T03:52:34.980981+0000","last_clean":"2023-07-06T03:52:34.980981+0000","last_became_active":"2023-07-06T03:52:34.974730+0000","last_became_peered":"2023-07-06T03:52:34.974730+0000","last_unstale":"2023-07-06T03:52:34.980981+0000","last_undegraded":"2023-07-06T03:52:34.980981+0000","last_fullsized":"2023-07-06T03:52:34.980981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:54:06.326418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.959329+0000","last_change":"2023-07-06T03:52:33.974879+0000","last_active":"2023-07-06T03:52:34.959329+0000","last_peered":"2023-07-06T03:52:34.959329+0000","last_clean":"2023-07-06T03:52:34.959329+0000","last_became_active":"2023-07-06T03:52:33.974574+0000","last_became_peered":"2023-07-06T03:52:33.974574+0000","last_unstale":"2023-07-06T03:52:34.959329+0000","last_undegraded":"2023-07-06T03:52:34.959329+0000","last_fullsized":"2023-07-06T03:52:34.959329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:45:15.238557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959981+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:52:35.959981+0000","last_peered":"2023-07-06T03:52:35.959981+0000","last_clean":"2023-07-06T03:52:35.959981+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:52:35.959981+0000","last_undegraded":"2023-07-06T03:52:35.959981+0000","last_fullsized":"2023-07-06T03:52:35.959981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:01.339783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.976650+0000","last_change":"2023-07-06T03:52:34.973908+0000","last_active":"2023-07-06T03:52:34.976650+0000","last_peered":"2023-07-06T03:52:34.976650+0000","last_clean":"2023-07-06T03:52:34.976650+0000","last_became_active":"2023-07-06T03:52:34.973618+0000","last_became_peered":"2023-07-06T03:52:34.973618+0000","last_unstale":"2023-07-06T03:52:34.976650+0000","last_undegraded":"2023-07-06T03:52:34.976650+0000","last_fullsized":"2023-07-06T03:52:34.976650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:47:00.396664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.969269+0000","last_change":"2023-07-06T03:52:33.970361+0000","last_active":"2023-07-06T03:52:34.969269+0000","last_peered":"2023-07-06T03:52:34.969269+0000","last_clean":"2023-07-06T03:52:34.969269+0000","last_became_active":"2023-07-06T03:52:33.970075+0000","last_became_peered":"2023-07-06T03:52:33.970075+0000","last_unstale":"2023-07-06T03:52:34.969269+0000","last_undegraded":"2023-07-06T03:52:34.969269+0000","last_fullsized":"2023-07-06T03:52:34.969269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:46:44.854195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958741+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:52:35.958741+0000","last_peered":"2023-07-06T03:52:35.958741+0000","last_clean":"2023-07-06T03:52:35.958741+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:52:35.958741+0000","last_undegraded":"2023-07-06T03:52:35.958741+0000","last_fullsized":"2023-07-06T03:52:35.958741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:30.642773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":470899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68799999999999994,"15min":0.71299999999999997},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.91200000000000003,"5min":4.0570000000000004,"15min":4.0570000000000004},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.67600000000000005,"15min":0.78000000000000003},"min":{"1min":0.47799999999999998,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67000000000000004,"15min":0.72999999999999998},"min":{"1min":0.46200000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86499999999999999,"5min":0.95799999999999996,"15min":2.6219999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70399999999999996,"15min":0.86699999999999999},"min":{"1min":0.501,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89900000000000002,"5min":4.0389999999999997,"15min":6.6310000000000002},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.67600000000000005,"15min":0.78600000000000003},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":3.77},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.70899999999999996,"15min":0.73299999999999998},"min":{"1min":0.56399999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94399999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70099999999999996,"15min":0.70799999999999996},"min":{"1min":0.46700000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":1.024,"15min":1.024},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.747,"15min":0.755},"min":{"1min":0.58899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.85699999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.622},"min":{"1min":0.437,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.65100000000000002,"15min":0.70699999999999996},"min":{"1min":0.51500000000000001,"5min":0.375,"15min":0.374},"max":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":1.4590000000000001},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66700000000000004,"15min":0.69599999999999995},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.88500000000000001,"5min":0.96699999999999997,"15min":1.0820000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69099999999999995,"15min":0.67600000000000005},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.871,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.61199999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.90400000000000003,"5min":1.03,"15min":1.5349999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73399999999999999,"15min":0.78600000000000003},"min":{"1min":0.52100000000000002,"5min":0.44800000000000001,"15min":0.42299999999999999},"max":{"1min":0.91200000000000003,"5min":1.006,"15min":1.3779999999999999},"last":0.753}]}]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":1064960,"data_stored":472587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70199999999999996,"15min":0.67000000000000004},"min":{"1min":0.55500000000000005,"5min":0.30599999999999999,"15min":0.29399999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.62},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77000000000000002,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.92600000000000005,"5min":3.3399999999999999,"15min":3.3399999999999999},"last":0.79900000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68400000000000005,"15min":0.67400000000000004},"min":{"1min":0.57399999999999995,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.40100000000000002},"max":{"1min":0.94699999999999995,"5min":0.96399999999999997,"15min":1.1759999999999999},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71599999999999997,"15min":0.67600000000000005},"min":{"1min":0.51100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90700000000000003,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94799999999999995,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.74099999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69599999999999995,"15min":0.752},"min":{"1min":0.58999999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88500000000000001,"5min":0.93200000000000005,"15min":2.1850000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.70699999999999996,"15min":0.73399999999999999},"min":{"1min":0.51000000000000001,"5min":0.40400000000000003,"15min":0.373},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.093},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66700000000000004,"15min":0.71199999999999997},"min":{"1min":0.42799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96899999999999997,"5min":1.0309999999999999,"15min":1.1339999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.72999999999999998},"min":{"1min":0.48699999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.085},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64600000000000002,"15min":0.70799999999999996},"min":{"1min":0.48199999999999998,"5min":0.443,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":1.218},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.78300000000000003},"min":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.36299999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.198},"last":0.81799999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72399999999999998,"15min":0.81499999999999995},"min":{"1min":0.51100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0269999999999999,"5min":1.054,"15min":1.323},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73599999999999999,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.504,"15min":0.41499999999999998},"max":{"1min":0.94599999999999995,"5min":1.028,"15min":1.4139999999999999},"last":0.77900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1650688,"data_stored":1061953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42999999999999999,"15min":0.53700000000000003},"min":{"1min":0.28999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.67000000000000004,"5min":1.002,"15min":2.8879999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.498,"15min":0.56899999999999995},"min":{"1min":0.371,"5min":0.313,"15min":0.313},"max":{"1min":0.69099999999999995,"5min":0.78300000000000003,"15min":1.96},"last":0.435}]},{"osd":2,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45500000000000002,"15min":0.88200000000000001},"min":{"1min":0.312,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59799999999999998,"5min":0.70799999999999996,"15min":13.055999999999999},"last":0.317},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54200000000000004,"15min":0.63300000000000001},"min":{"1min":0.39400000000000002,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.81499999999999995,"5min":0.92400000000000004,"15min":2.8479999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.60899999999999999},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66000000000000003,"5min":0.96799999999999997,"15min":2.9249999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48399999999999999,"15min":0.54300000000000004},"min":{"1min":0.38100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70599999999999996,"5min":0.82599999999999996,"15min":1.2709999999999999},"last":0.496}]},{"osd":4,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.751,"15min":1.151},"min":{"1min":0.63700000000000001,"5min":0.47599999999999998,"15min":0.39200000000000002},"max":{"1min":0.94399999999999995,"5min":1.0640000000000001,"15min":12.989000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.77600000000000002,"15min":0.82999999999999996},"min":{"1min":0.70799999999999996,"5min":0.53000000000000003,"15min":0.46000000000000002},"max":{"1min":1.004,"5min":1.371,"15min":2.7629999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73799999999999999,"15min":0.77700000000000002},"min":{"1min":0.60199999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.3100000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.72699999999999998,"15min":1.155},"min":{"1min":0.56999999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0529999999999999,"5min":1.1020000000000001,"15min":13.023999999999999},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.75900000000000001,"15min":0.81499999999999995},"min":{"1min":0.58099999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.044,"5min":1.119,"15min":1.365},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77400000000000002,"15min":1.1990000000000001},"min":{"1min":0.56299999999999994,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96999999999999997,"5min":1.254,"15min":12.941000000000001},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.75700000000000001,"15min":0.83099999999999996},"min":{"1min":0.64600000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.042,"5min":1.4690000000000001,"15min":2.806},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76500000000000001,"15min":0.81599999999999995},"min":{"1min":0.61099999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97699999999999998,"5min":1.1319999999999999,"15min":1.337},"last":0.57099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":1048576,"data_stored":470969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.29299999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.442,"15min":0.441},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.248},"max":{"1min":0.65000000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.377}]},{"osd":2,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.38800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69199999999999995,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.48999999999999999,"15min":0.503},"min":{"1min":0.40500000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.65700000000000003,"5min":0.79400000000000004,"15min":1.2030000000000001},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.41899999999999998,"5min":0.30099999999999999,"15min":0.29899999999999999},"max":{"1min":0.72699999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79000000000000004,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.442}]},{"osd":4,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.72499999999999998,"15min":0.749},"min":{"1min":0.51700000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0189999999999999,"5min":1.1699999999999999,"15min":1.171},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.56000000000000005,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.034},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.58399999999999996,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":1.0820000000000001,"5min":1.1080000000000001,"15min":1.1419999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73099999999999998,"15min":0.752},"min":{"1min":0.63400000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.99199999999999999,"5min":1.1040000000000001,"15min":1.1220000000000001},"last":0.56399999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.74099999999999999,"15min":0.752},"min":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0329999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74299999999999999,"15min":0.75800000000000001},"min":{"1min":0.58199999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0649999999999999,"15min":1.546},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.78300000000000003,"15min":0.80500000000000005},"min":{"1min":0.54600000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0529999999999999,"5min":1.0820000000000001,"15min":1.6830000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77100000000000002,"15min":1.0640000000000001},"min":{"1min":0.60599999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":9.5960000000000001},"last":0.58699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36172,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335668,"statfs":{"total":96636764160,"available":96599724032,"internally_reserved":0,"allocated":1052672,"data_stored":471503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.432,"15min":0.42699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.50900000000000001,"15min":0.48199999999999998},"min":{"1min":0.29599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.77800000000000002,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.093}]},{"osd":1,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.60899999999999999,"5min":0.72799999999999998,"15min":0.81799999999999995},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54600000000000004,"15min":0.51100000000000001},"min":{"1min":0.42499999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72799999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.5529999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.54400000000000004},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.752,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":2.1240000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59299999999999997,"15min":0.57299999999999995},"min":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80600000000000005,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":2.1629999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78600000000000003,"15min":0.72399999999999998},"min":{"1min":0.54600000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98199999999999998,"5min":2.863,"15min":2.863},"last":5.5380000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79800000000000004,"15min":0.72199999999999998},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.034,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.748,"15min":0.73299999999999998},"min":{"1min":0.50900000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93999999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":1.746},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80500000000000005,"15min":0.76800000000000002},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.004,"5min":2.4990000000000001,"15min":2.4990000000000001},"last":6.8440000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78500000000000003,"15min":0.72999999999999998},"min":{"1min":0.57999999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":2.964,"15min":2.964},"last":11.696999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79100000000000004,"15min":0.76200000000000001},"min":{"1min":0.63900000000000001,"5min":0.55000000000000004,"15min":0.50700000000000001},"max":{"1min":0.97899999999999998,"5min":2.085,"15min":2.085},"last":1.6040000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81100000000000005,"15min":0.82699999999999996},"min":{"1min":0.57999999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.2},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81799999999999995,"15min":0.80400000000000005},"min":{"1min":0.624,"5min":0.56000000000000005,"15min":0.47499999999999998},"max":{"1min":0.92000000000000004,"5min":2.702,"15min":2.702},"last":4.484}]}]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":1064960,"data_stored":472575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42599999999999999,"15min":0.59999999999999998},"min":{"1min":0.28499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":5.6550000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.497,"15min":0.51400000000000001},"min":{"1min":0.43099999999999999,"5min":0.33600000000000002,"15min":0.26700000000000002},"max":{"1min":0.68700000000000006,"5min":0.73799999999999999,"15min":1.0620000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44600000000000001,"15min":1.0109999999999999},"min":{"1min":0.32700000000000001,"5min":0.29299999999999998,"15min":0.29099999999999998},"max":{"1min":0.60399999999999998,"5min":0.745,"15min":15.911},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52200000000000002,"15min":1.083},"min":{"1min":0.34599999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69099999999999995,"5min":0.78800000000000003,"15min":16.024000000000001},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.501},"min":{"1min":0.34699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61699999999999999,"5min":0.70199999999999996,"15min":1.0309999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51500000000000001,"15min":1.0840000000000001},"min":{"1min":0.40699999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.73799999999999999,"5min":0.75700000000000001,"15min":16.068999999999999},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73299999999999998,"15min":0.76200000000000001},"min":{"1min":0.58099999999999996,"5min":0.44,"15min":0.44},"max":{"1min":0.94699999999999995,"5min":1.171,"15min":1.179},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.73699999999999999,"15min":0.92200000000000004},"min":{"1min":0.54300000000000004,"5min":0.46100000000000002,"15min":0.36699999999999999},"max":{"1min":0.95899999999999996,"5min":1.0940000000000001,"15min":5.7300000000000004},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.748,"15min":0.92600000000000005},"min":{"1min":0.628,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":1.1080000000000001,"15min":5.077},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77400000000000002,"15min":0.97899999999999998},"min":{"1min":0.64100000000000001,"5min":0.499,"15min":0.44800000000000001},"max":{"1min":0.998,"5min":1.196,"15min":5.6980000000000004},"last":0.82599999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.748,"15min":0.80200000000000005},"min":{"1min":0.627,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.028,"5min":1.0629999999999999,"15min":1.143},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.74399999999999999,"15min":0.93600000000000005},"min":{"1min":0.59799999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96799999999999997,"5min":1.127,"15min":5.157},"last":0.81100000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.751,"15min":0.83099999999999996},"min":{"1min":0.62,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93999999999999995,"5min":0.997,"15min":1.6679999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.94799999999999995},"min":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97399999999999998,"5min":1.1399999999999999,"15min":5.1189999999999998},"last":0.72199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36164,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335676,"statfs":{"total":96636764160,"available":96599732224,"internally_reserved":0,"allocated":1044480,"data_stored":470899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.54100000000000004,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88300000000000001,"5min":1.784,"15min":1.784},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.38},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.72799999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65600000000000003,"15min":0.85099999999999998},"min":{"1min":0.58199999999999996,"5min":0.38,"15min":0.38},"max":{"1min":0.81399999999999995,"5min":0.95399999999999996,"15min":6.258},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68200000000000005,"15min":0.66400000000000003},"min":{"1min":0.60199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79400000000000004,"5min":1.806,"15min":1.806},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67600000000000005,"15min":0.86799999999999999},"min":{"1min":0.436,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89100000000000001,"5min":0.998,"15min":6.4740000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73199999999999998,"15min":0.94799999999999995},"min":{"1min":0.67200000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.88100000000000001,"5min":0.95099999999999996,"15min":6.4859999999999998},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.58999999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80000000000000004,"5min":0.97799999999999998,"15min":1.25},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85999999999999999,"5min":0.99099999999999999,"15min":1.413},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59699999999999998,"15min":0.65600000000000003},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78800000000000003,"5min":0.92100000000000004,"15min":1.3280000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.66400000000000003},"min":{"1min":0.46400000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82699999999999996,"5min":0.94299999999999995,"15min":0.99299999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63200000000000001,"15min":0.66500000000000004},"min":{"1min":0.45500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85999999999999999,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73599999999999999,"15min":0.79000000000000004},"min":{"1min":0.65500000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.86699999999999999,"5min":1.008,"15min":1.6839999999999999},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67900000000000005,"15min":0.72799999999999998},"min":{"1min":0.52700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80000000000000004,"5min":0.95699999999999996,"15min":1.0660000000000001},"last":0.749},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69799999999999995,"15min":0.749},"min":{"1min":0.50800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83899999999999997,"5min":1.079,"15min":1.21},"last":0.85299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1596,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1634304,"data_stored":1060248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.66200000000000003},"min":{"1min":0.55600000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82099999999999995,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67100000000000004,"15min":0.85199999999999998},"min":{"1min":0.59499999999999997,"5min":0.38100000000000001,"15min":0.29999999999999999},"max":{"1min":0.82799999999999996,"5min":0.93000000000000005,"15min":6.4059999999999997},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.60999999999999999,"5min":0.39700000000000002,"15min":0.307},"max":{"1min":0.90000000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.54900000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81999999999999995,"5min":1.1970000000000001,"15min":1.546},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68899999999999995,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":1.071,"15min":1.071},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.47099999999999997,"5min":0.439,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":1.0029999999999999},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.55200000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92900000000000005,"5min":1.0009999999999999,"15min":1.03},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73499999999999999,"15min":0.752},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.39600000000000002},"max":{"1min":0.86599999999999999,"5min":4.484,"15min":4.484},"last":1.3140000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.34699999999999998},"max":{"1min":0.77400000000000002,"5min":1.1359999999999999,"15min":1.6240000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79300000000000004,"5min":1.0569999999999999,"15min":2.718},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.49099999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68799999999999994,"15min":0.71899999999999997},"min":{"1min":0.51100000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88200000000000001,"5min":1.246,"15min":1.669},"last":0.753}]},{"osd":7,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72899999999999998,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89800000000000002,"5min":1.3360000000000001,"15min":2.0459999999999998},"last":0.622},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.749,"15min":0.78200000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.47699999999999998},"max":{"1min":0.94499999999999995,"5min":1.0609999999999999,"15min":2.2919999999999998},"last":0.58199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:52:39.354 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:52:39.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.578+0000 7f38505ff700 1 Processor -- start 2023-07-06T03:52:39.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.578+0000 7f38505ff700 1 -- start start 2023-07-06T03:52:39.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.579+0000 7f38505ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f384815c520 0x7f38480cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.579+0000 7f38505ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3848160180 0x7f38481605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.579+0000 7f38505ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38480cdae0 con 0x7f3848160b30 2023-07-06T03:52:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.579+0000 7f38505ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38480cd8d0 con 0x7f3848160180 2023-07-06T03:52:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.579+0000 7f38505ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38480b6640 con 0x7f384815c520 2023-07-06T03:52:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.580+0000 7f384db9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3848160180 0x7f38481605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.580+0000 7f384db9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3848160180 0x7f38481605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55032/0 (socket says 172.21.15.97:55032) 2023-07-06T03:52:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.580+0000 7f384db9a700 1 -- 172.21.15.97:0/536149238 learned_addr learned my addr 172.21.15.97:0/536149238 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.580+0000 7f384e39b700 1 --2- 172.21.15.97:0/536149238 >> v2:172.21.15.146:3300/0 conn(0x7f384815c520 0x7f38480cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.580+0000 7f384db9a700 1 -- 172.21.15.97:0/536149238 >> v2:172.21.15.146:3300/0 conn(0x7f384815c520 msgr2=0x7f38480cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.580+0000 7f384db9a700 1 --2- 172.21.15.97:0/536149238 >> v2:172.21.15.146:3300/0 conn(0x7f384815c520 0x7f38480cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.580+0000 7f384db9a700 1 -- 172.21.15.97:0/536149238 >> v1:172.21.15.146:6789/0 conn(0x7f3848160b30 legacy=0x7f38480cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:39.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.580+0000 7f384db9a700 1 -- 172.21.15.97:0/536149238 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480b6920 con 0x7f3848160180 2023-07-06T03:52:39.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.580+0000 7f384e39b700 1 --2- 172.21.15.97:0/536149238 >> v2:172.21.15.146:3300/0 conn(0x7f384815c520 0x7f38480cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:39.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.580+0000 7f384db9a700 1 --2- 172.21.15.97:0/536149238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3848160180 0x7f38481605f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f38480b4c70 tx=0x7f3838009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74543fe2bd4505f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:39.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.581+0000 7f384d399700 1 -- 172.21.15.97:0/536149238 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383800b870 con 0x7f3848160180 2023-07-06T03:52:39.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.581+0000 7f384d399700 1 -- 172.21.15.97:0/536149238 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383800b9d0 con 0x7f3848160180 2023-07-06T03:52:39.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.581+0000 7f384d399700 1 -- 172.21.15.97:0/536149238 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383800bd60 con 0x7f3848160180 2023-07-06T03:52:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.581+0000 7f38505ff700 1 -- 172.21.15.97:0/536149238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3848160180 msgr2=0x7f38481605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.581+0000 7f38505ff700 1 --2- 172.21.15.97:0/536149238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3848160180 0x7f38481605f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f38480b4c70 tx=0x7f3838009bd0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.581+0000 7f38505ff700 1 -- 172.21.15.97:0/536149238 shutdown_connections 2023-07-06T03:52:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.581+0000 7f38505ff700 1 --2- 172.21.15.97:0/536149238 >> v2:172.21.15.146:3300/0 conn(0x7f384815c520 0x7f38480cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.581+0000 7f38505ff700 1 --2- 172.21.15.97:0/536149238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3848160180 0x7f38481605f0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.581+0000 7f38505ff700 1 -- 172.21.15.97:0/536149238 >> 172.21.15.97:0/536149238 conn(0x7f3848071790 msgr2=0x7f3848071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.581+0000 7f38505ff700 1 -- 172.21.15.97:0/536149238 shutdown_connections 2023-07-06T03:52:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.581+0000 7f38505ff700 1 -- 172.21.15.97:0/536149238 wait complete. 2023-07-06T03:52:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.583+0000 7f38505ff700 1 Processor -- start 2023-07-06T03:52:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.583+0000 7f38505ff700 1 -- start start 2023-07-06T03:52:39.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.584+0000 7f38505ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384815c520 0x7f3848155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.584+0000 7f38505ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3848160180 0x7f384815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.584+0000 7f38505ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3848160b30 0x7f38481e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.584+0000 7f38505ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3848156300 con 0x7f384815c520 2023-07-06T03:52:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.584+0000 7f38505ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38481f2b60 con 0x7f3848160b30 2023-07-06T03:52:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.584+0000 7f38505ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38481e74e0 con 0x7f3848160180 2023-07-06T03:52:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.584+0000 7f384e39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384815c520 0x7f3848155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.584+0000 7f384e39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384815c520 0x7f3848155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55048/0 (socket says 172.21.15.97:55048) 2023-07-06T03:52:39.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.584+0000 7f384e39b700 1 -- 172.21.15.97:0/4081813154 learned_addr learned my addr 172.21.15.97:0/4081813154 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:39.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.584+0000 7f384db9a700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3848160180 0x7f384815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.584+0000 7f384eb9c700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3848160b30 0x7f38481e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.585+0000 7f384e39b700 1 -- 172.21.15.97:0/4081813154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3848160180 msgr2=0x7f384815cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:39.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.585+0000 7f384e39b700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3848160180 0x7f384815cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.585+0000 7f384e39b700 1 -- 172.21.15.97:0/4081813154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3848160b30 msgr2=0x7f38481e6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:39.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.585+0000 7f384e39b700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3848160b30 0x7f38481e6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.585+0000 7f384e39b700 1 -- 172.21.15.97:0/4081813154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3838009550 con 0x7f384815c520 2023-07-06T03:52:39.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.585+0000 7f384e39b700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384815c520 0x7f3848155dc0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f384400e760 tx=0x7f384400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:39.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.585+0000 7f383effd700 1 -- 172.21.15.97:0/4081813154 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844013610 con 0x7f384815c520 2023-07-06T03:52:39.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.585+0000 7f383effd700 1 -- 172.21.15.97:0/4081813154 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3844013770 con 0x7f384815c520 2023-07-06T03:52:39.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.585+0000 7f38505ff700 1 -- 172.21.15.97:0/4081813154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38480cb730 con 0x7f384815c520 2023-07-06T03:52:39.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.585+0000 7f383effd700 1 -- 172.21.15.97:0/4081813154 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844013b20 con 0x7f384815c520 2023-07-06T03:52:39.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.585+0000 7f38505ff700 1 -- 172.21.15.97:0/4081813154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38480cbcf0 con 0x7f384815c520 2023-07-06T03:52:39.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.586+0000 7f38505ff700 1 -- 172.21.15.97:0/4081813154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384804e340 con 0x7f384815c520 2023-07-06T03:52:39.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.587+0000 7f383effd700 1 -- 172.21.15.97:0/4081813154 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3844013c80 con 0x7f384815c520 2023-07-06T03:52:39.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.588+0000 7f383effd700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382c06ace0 0x7f382c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:39.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.588+0000 7f383effd700 1 -- 172.21.15.97:0/4081813154 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3844089030 con 0x7f384815c520 2023-07-06T03:52:39.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.589+0000 7f384db9a700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382c06ace0 0x7f382c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:39.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.589+0000 7f384db9a700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382c06ace0 0x7f382c06d190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3838000c00 tx=0x7f3838008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:39.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.592+0000 7f383effd700 1 -- 172.21.15.97:0/4081813154 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3844057c00 con 0x7f384815c520 2023-07-06T03:52:39.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.786+0000 7f38505ff700 1 -- 172.21.15.97:0/4081813154 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38480b6ee0 con 0x7f382c06ace0 2023-07-06T03:52:39.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.794+0000 7f383effd700 1 -- 172.21.15.97:0/4081813154 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311471 (secure 0 0 0) 0x7f383800bb40 con 0x7f382c06ace0 2023-07-06T03:52:39.799 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 -- 172.21.15.97:0/4081813154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382c06ace0 msgr2=0x7f382c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382c06ace0 0x7f382c06d190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3838000c00 tx=0x7f3838008030 comp rx=0 tx=0).stop 2023-07-06T03:52:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 -- 172.21.15.97:0/4081813154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384815c520 msgr2=0x7f3848155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384815c520 0x7f3848155dc0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f384400e760 tx=0x7f384400eb20 comp rx=0 tx=0).stop 2023-07-06T03:52:39.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 -- 172.21.15.97:0/4081813154 shutdown_connections 2023-07-06T03:52:39.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f382c06ace0 0x7f382c06d190 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3848160b30 0x7f38481e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384815c520 0x7f3848155dc0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 --2- 172.21.15.97:0/4081813154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3848160180 0x7f384815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:39.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 -- 172.21.15.97:0/4081813154 >> 172.21.15.97:0/4081813154 conn(0x7f3848071790 msgr2=0x7f38480b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:39.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 -- 172.21.15.97:0/4081813154 shutdown_connections 2023-07-06T03:52:39.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:39.799+0000 7f38505ff700 1 -- 172.21.15.97:0/4081813154 wait complete. 2023-07-06T03:52:39.803 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:52:39.819 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2023-07-06T03:52:38.981769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265956,"kb_used_data":9380,"kb_used_omap":0,"kb_used_meta":256512,"kb_avail":754708764,"statfs":{"total":773094113280,"available":772821774336,"internally_reserved":0,"allocated":9605120,"data_stored":4951633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":262668288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001500"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.987252+0000","last_change":"2023-07-06T03:52:34.977882+0000","last_active":"2023-07-06T03:52:34.987252+0000","last_peered":"2023-07-06T03:52:34.987252+0000","last_clean":"2023-07-06T03:52:34.987252+0000","last_became_active":"2023-07-06T03:52:34.977618+0000","last_became_peered":"2023-07-06T03:52:34.977618+0000","last_unstale":"2023-07-06T03:52:34.987252+0000","last_undegraded":"2023-07-06T03:52:34.987252+0000","last_fullsized":"2023-07-06T03:52:34.987252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:11:06.616526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988311+0000","last_change":"2023-07-06T03:52:34.980835+0000","last_active":"2023-07-06T03:52:34.988311+0000","last_peered":"2023-07-06T03:52:34.988311+0000","last_clean":"2023-07-06T03:52:34.988311+0000","last_became_active":"2023-07-06T03:52:34.980386+0000","last_became_peered":"2023-07-06T03:52:34.980386+0000","last_unstale":"2023-07-06T03:52:34.988311+0000","last_undegraded":"2023-07-06T03:52:34.988311+0000","last_fullsized":"2023-07-06T03:52:34.988311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:34:25.363307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958334+0000","last_change":"2023-07-06T03:52:34.975433+0000","last_active":"2023-07-06T03:52:35.958334+0000","last_peered":"2023-07-06T03:52:35.958334+0000","last_clean":"2023-07-06T03:52:35.958334+0000","last_became_active":"2023-07-06T03:52:34.975049+0000","last_became_peered":"2023-07-06T03:52:34.975049+0000","last_unstale":"2023-07-06T03:52:35.958334+0000","last_undegraded":"2023-07-06T03:52:35.958334+0000","last_fullsized":"2023-07-06T03:52:35.958334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:19:02.780761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.957919+0000","last_change":"2023-07-06T03:52:34.986371+0000","last_active":"2023-07-06T03:52:35.957919+0000","last_peered":"2023-07-06T03:52:35.957919+0000","last_clean":"2023-07-06T03:52:35.957919+0000","last_became_active":"2023-07-06T03:52:34.986031+0000","last_became_peered":"2023-07-06T03:52:34.986031+0000","last_unstale":"2023-07-06T03:52:35.957919+0000","last_undegraded":"2023-07-06T03:52:35.957919+0000","last_fullsized":"2023-07-06T03:52:35.957919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:28:50.974864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983120+0000","last_change":"2023-07-06T03:52:34.969896+0000","last_active":"2023-07-06T03:52:34.983120+0000","last_peered":"2023-07-06T03:52:34.983120+0000","last_clean":"2023-07-06T03:52:34.983120+0000","last_became_active":"2023-07-06T03:52:34.969634+0000","last_became_peered":"2023-07-06T03:52:34.969634+0000","last_unstale":"2023-07-06T03:52:34.983120+0000","last_undegraded":"2023-07-06T03:52:34.983120+0000","last_fullsized":"2023-07-06T03:52:34.983120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:37:49.824908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959456+0000","last_change":"2023-07-06T03:52:34.980127+0000","last_active":"2023-07-06T03:52:35.959456+0000","last_peered":"2023-07-06T03:52:35.959456+0000","last_clean":"2023-07-06T03:52:35.959456+0000","last_became_active":"2023-07-06T03:52:34.979408+0000","last_became_peered":"2023-07-06T03:52:34.979408+0000","last_unstale":"2023-07-06T03:52:35.959456+0000","last_undegraded":"2023-07-06T03:52:35.959456+0000","last_fullsized":"2023-07-06T03:52:35.959456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:23:42.598710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958331+0000","last_change":"2023-07-06T03:52:34.980083+0000","last_active":"2023-07-06T03:52:35.958331+0000","last_peered":"2023-07-06T03:52:35.958331+0000","last_clean":"2023-07-06T03:52:35.958331+0000","last_became_active":"2023-07-06T03:52:34.979449+0000","last_became_peered":"2023-07-06T03:52:34.979449+0000","last_unstale":"2023-07-06T03:52:35.958331+0000","last_undegraded":"2023-07-06T03:52:35.958331+0000","last_fullsized":"2023-07-06T03:52:35.958331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:49:21.615646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959214+0000","last_change":"2023-07-06T03:52:34.975517+0000","last_active":"2023-07-06T03:52:35.959214+0000","last_peered":"2023-07-06T03:52:35.959214+0000","last_clean":"2023-07-06T03:52:35.959214+0000","last_became_active":"2023-07-06T03:52:34.975100+0000","last_became_peered":"2023-07-06T03:52:34.975100+0000","last_unstale":"2023-07-06T03:52:35.959214+0000","last_undegraded":"2023-07-06T03:52:35.959214+0000","last_fullsized":"2023-07-06T03:52:35.959214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:19.868969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958610+0000","last_change":"2023-07-06T03:52:34.977802+0000","last_active":"2023-07-06T03:52:35.958610+0000","last_peered":"2023-07-06T03:52:35.958610+0000","last_clean":"2023-07-06T03:52:35.958610+0000","last_became_active":"2023-07-06T03:52:34.977503+0000","last_became_peered":"2023-07-06T03:52:34.977503+0000","last_unstale":"2023-07-06T03:52:35.958610+0000","last_undegraded":"2023-07-06T03:52:35.958610+0000","last_fullsized":"2023-07-06T03:52:35.958610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:05:54.626714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958036+0000","last_change":"2023-07-06T03:52:34.976935+0000","last_active":"2023-07-06T03:52:35.958036+0000","last_peered":"2023-07-06T03:52:35.958036+0000","last_clean":"2023-07-06T03:52:35.958036+0000","last_became_active":"2023-07-06T03:52:34.976572+0000","last_became_peered":"2023-07-06T03:52:34.976572+0000","last_unstale":"2023-07-06T03:52:35.958036+0000","last_undegraded":"2023-07-06T03:52:35.958036+0000","last_fullsized":"2023-07-06T03:52:35.958036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:14:49.245209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958395+0000","last_change":"2023-07-06T03:52:34.980831+0000","last_active":"2023-07-06T03:52:35.958395+0000","last_peered":"2023-07-06T03:52:35.958395+0000","last_clean":"2023-07-06T03:52:35.958395+0000","last_became_active":"2023-07-06T03:52:34.980544+0000","last_became_peered":"2023-07-06T03:52:34.980544+0000","last_unstale":"2023-07-06T03:52:35.958395+0000","last_undegraded":"2023-07-06T03:52:35.958395+0000","last_fullsized":"2023-07-06T03:52:35.958395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:48:52.526357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958064+0000","last_change":"2023-07-06T03:52:34.976587+0000","last_active":"2023-07-06T03:52:35.958064+0000","last_peered":"2023-07-06T03:52:35.958064+0000","last_clean":"2023-07-06T03:52:35.958064+0000","last_became_active":"2023-07-06T03:52:34.976328+0000","last_became_peered":"2023-07-06T03:52:34.976328+0000","last_unstale":"2023-07-06T03:52:35.958064+0000","last_undegraded":"2023-07-06T03:52:35.958064+0000","last_fullsized":"2023-07-06T03:52:35.958064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:10:45.427796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988801+0000","last_change":"2023-07-06T03:52:34.984814+0000","last_active":"2023-07-06T03:52:34.988801+0000","last_peered":"2023-07-06T03:52:34.988801+0000","last_clean":"2023-07-06T03:52:34.988801+0000","last_became_active":"2023-07-06T03:52:34.984567+0000","last_became_peered":"2023-07-06T03:52:34.984567+0000","last_unstale":"2023-07-06T03:52:34.988801+0000","last_undegraded":"2023-07-06T03:52:34.988801+0000","last_fullsized":"2023-07-06T03:52:34.988801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:44:45.115650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.978279+0000","last_change":"2023-07-06T03:52:34.975049+0000","last_active":"2023-07-06T03:52:34.978279+0000","last_peered":"2023-07-06T03:52:34.978279+0000","last_clean":"2023-07-06T03:52:34.978279+0000","last_became_active":"2023-07-06T03:52:34.974774+0000","last_became_peered":"2023-07-06T03:52:34.974774+0000","last_unstale":"2023-07-06T03:52:34.978279+0000","last_undegraded":"2023-07-06T03:52:34.978279+0000","last_fullsized":"2023-07-06T03:52:34.978279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:40:10.226780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958596+0000","last_change":"2023-07-06T03:52:34.978342+0000","last_active":"2023-07-06T03:52:35.958596+0000","last_peered":"2023-07-06T03:52:35.958596+0000","last_clean":"2023-07-06T03:52:35.958596+0000","last_became_active":"2023-07-06T03:52:34.977879+0000","last_became_peered":"2023-07-06T03:52:34.977879+0000","last_unstale":"2023-07-06T03:52:35.958596+0000","last_undegraded":"2023-07-06T03:52:35.958596+0000","last_fullsized":"2023-07-06T03:52:35.958596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:08:54.635642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959232+0000","last_change":"2023-07-06T03:52:34.991737+0000","last_active":"2023-07-06T03:52:35.959232+0000","last_peered":"2023-07-06T03:52:35.959232+0000","last_clean":"2023-07-06T03:52:35.959232+0000","last_became_active":"2023-07-06T03:52:34.991402+0000","last_became_peered":"2023-07-06T03:52:34.991402+0000","last_unstale":"2023-07-06T03:52:35.959232+0000","last_undegraded":"2023-07-06T03:52:35.959232+0000","last_fullsized":"2023-07-06T03:52:35.959232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:50.814912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959701+0000","last_change":"2023-07-06T03:52:34.981157+0000","last_active":"2023-07-06T03:52:35.959701+0000","last_peered":"2023-07-06T03:52:35.959701+0000","last_clean":"2023-07-06T03:52:35.959701+0000","last_became_active":"2023-07-06T03:52:34.980640+0000","last_became_peered":"2023-07-06T03:52:34.980640+0000","last_unstale":"2023-07-06T03:52:35.959701+0000","last_undegraded":"2023-07-06T03:52:35.959701+0000","last_fullsized":"2023-07-06T03:52:35.959701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:21:32.792141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958039+0000","last_change":"2023-07-06T03:52:34.976346+0000","last_active":"2023-07-06T03:52:35.958039+0000","last_peered":"2023-07-06T03:52:35.958039+0000","last_clean":"2023-07-06T03:52:35.958039+0000","last_became_active":"2023-07-06T03:52:34.975892+0000","last_became_peered":"2023-07-06T03:52:34.975892+0000","last_unstale":"2023-07-06T03:52:35.958039+0000","last_undegraded":"2023-07-06T03:52:35.958039+0000","last_fullsized":"2023-07-06T03:52:35.958039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:49:11.015333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986115+0000","last_change":"2023-07-06T03:52:34.982531+0000","last_active":"2023-07-06T03:52:34.986115+0000","last_peered":"2023-07-06T03:52:34.986115+0000","last_clean":"2023-07-06T03:52:34.986115+0000","last_became_active":"2023-07-06T03:52:34.981923+0000","last_became_peered":"2023-07-06T03:52:34.981923+0000","last_unstale":"2023-07-06T03:52:34.986115+0000","last_undegraded":"2023-07-06T03:52:34.986115+0000","last_fullsized":"2023-07-06T03:52:34.986115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:25:53.451281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959670+0000","last_change":"2023-07-06T03:52:34.988638+0000","last_active":"2023-07-06T03:52:35.959670+0000","last_peered":"2023-07-06T03:52:35.959670+0000","last_clean":"2023-07-06T03:52:35.959670+0000","last_became_active":"2023-07-06T03:52:34.988337+0000","last_became_peered":"2023-07-06T03:52:34.988337+0000","last_unstale":"2023-07-06T03:52:35.959670+0000","last_undegraded":"2023-07-06T03:52:35.959670+0000","last_fullsized":"2023-07-06T03:52:35.959670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:08:20.630298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983068+0000","last_change":"2023-07-06T03:52:34.969454+0000","last_active":"2023-07-06T03:52:34.983068+0000","last_peered":"2023-07-06T03:52:34.983068+0000","last_clean":"2023-07-06T03:52:34.983068+0000","last_became_active":"2023-07-06T03:52:34.969192+0000","last_became_peered":"2023-07-06T03:52:34.969192+0000","last_unstale":"2023-07-06T03:52:34.983068+0000","last_undegraded":"2023-07-06T03:52:34.983068+0000","last_fullsized":"2023-07-06T03:52:34.983068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:48:49.769569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.981333+0000","last_change":"2023-07-06T03:52:34.978581+0000","last_active":"2023-07-06T03:52:34.981333+0000","last_peered":"2023-07-06T03:52:34.981333+0000","last_clean":"2023-07-06T03:52:34.981333+0000","last_became_active":"2023-07-06T03:52:34.978267+0000","last_became_peered":"2023-07-06T03:52:34.978267+0000","last_unstale":"2023-07-06T03:52:34.981333+0000","last_undegraded":"2023-07-06T03:52:34.981333+0000","last_fullsized":"2023-07-06T03:52:34.981333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:29:46.175292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983627+0000","last_change":"2023-07-06T03:52:34.971696+0000","last_active":"2023-07-06T03:52:34.983627+0000","last_peered":"2023-07-06T03:52:34.983627+0000","last_clean":"2023-07-06T03:52:34.983627+0000","last_became_active":"2023-07-06T03:52:34.971389+0000","last_became_peered":"2023-07-06T03:52:34.971389+0000","last_unstale":"2023-07-06T03:52:34.983627+0000","last_undegraded":"2023-07-06T03:52:34.983627+0000","last_fullsized":"2023-07-06T03:52:34.983627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:19:02.780761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.989096+0000","last_change":"2023-07-06T03:52:34.983511+0000","last_active":"2023-07-06T03:52:34.989096+0000","last_peered":"2023-07-06T03:52:34.989096+0000","last_clean":"2023-07-06T03:52:34.989096+0000","last_became_active":"2023-07-06T03:52:34.983028+0000","last_became_peered":"2023-07-06T03:52:34.983028+0000","last_unstale":"2023-07-06T03:52:34.989096+0000","last_undegraded":"2023-07-06T03:52:34.989096+0000","last_fullsized":"2023-07-06T03:52:34.989096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:43:54.846787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983020+0000","last_change":"2023-07-06T03:52:34.968903+0000","last_active":"2023-07-06T03:52:34.983020+0000","last_peered":"2023-07-06T03:52:34.983020+0000","last_clean":"2023-07-06T03:52:34.983020+0000","last_became_active":"2023-07-06T03:52:34.968628+0000","last_became_peered":"2023-07-06T03:52:34.968628+0000","last_unstale":"2023-07-06T03:52:34.983020+0000","last_undegraded":"2023-07-06T03:52:34.983020+0000","last_fullsized":"2023-07-06T03:52:34.983020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:16:14.184623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988288+0000","last_change":"2023-07-06T03:52:34.980658+0000","last_active":"2023-07-06T03:52:34.988288+0000","last_peered":"2023-07-06T03:52:34.988288+0000","last_clean":"2023-07-06T03:52:34.988288+0000","last_became_active":"2023-07-06T03:52:34.980055+0000","last_became_peered":"2023-07-06T03:52:34.980055+0000","last_unstale":"2023-07-06T03:52:34.988288+0000","last_undegraded":"2023-07-06T03:52:34.988288+0000","last_fullsized":"2023-07-06T03:52:34.988288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:48:16.313574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958313+0000","last_change":"2023-07-06T03:52:34.972777+0000","last_active":"2023-07-06T03:52:35.958313+0000","last_peered":"2023-07-06T03:52:35.958313+0000","last_clean":"2023-07-06T03:52:35.958313+0000","last_became_active":"2023-07-06T03:52:34.972514+0000","last_became_peered":"2023-07-06T03:52:34.972514+0000","last_unstale":"2023-07-06T03:52:35.958313+0000","last_undegraded":"2023-07-06T03:52:35.958313+0000","last_fullsized":"2023-07-06T03:52:35.958313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:22:03.328610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986593+0000","last_change":"2023-07-06T03:52:34.980403+0000","last_active":"2023-07-06T03:52:34.986593+0000","last_peered":"2023-07-06T03:52:34.986593+0000","last_clean":"2023-07-06T03:52:34.986593+0000","last_became_active":"2023-07-06T03:52:34.980104+0000","last_became_peered":"2023-07-06T03:52:34.980104+0000","last_unstale":"2023-07-06T03:52:34.986593+0000","last_undegraded":"2023-07-06T03:52:34.986593+0000","last_fullsized":"2023-07-06T03:52:34.986593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:21:57.347621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988798+0000","last_change":"2023-07-06T03:52:34.979952+0000","last_active":"2023-07-06T03:52:34.988798+0000","last_peered":"2023-07-06T03:52:34.988798+0000","last_clean":"2023-07-06T03:52:34.988798+0000","last_became_active":"2023-07-06T03:52:34.979642+0000","last_became_peered":"2023-07-06T03:52:34.979642+0000","last_unstale":"2023-07-06T03:52:34.988798+0000","last_undegraded":"2023-07-06T03:52:34.988798+0000","last_fullsized":"2023-07-06T03:52:34.988798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:19:07.941745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986552+0000","last_change":"2023-07-06T03:52:34.975620+0000","last_active":"2023-07-06T03:52:34.986552+0000","last_peered":"2023-07-06T03:52:34.986552+0000","last_clean":"2023-07-06T03:52:34.986552+0000","last_became_active":"2023-07-06T03:52:34.975282+0000","last_became_peered":"2023-07-06T03:52:34.975282+0000","last_unstale":"2023-07-06T03:52:34.986552+0000","last_undegraded":"2023-07-06T03:52:34.986552+0000","last_fullsized":"2023-07-06T03:52:34.986552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:10:45.427796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986077+0000","last_change":"2023-07-06T03:52:34.984673+0000","last_active":"2023-07-06T03:52:34.986077+0000","last_peered":"2023-07-06T03:52:34.986077+0000","last_clean":"2023-07-06T03:52:34.986077+0000","last_became_active":"2023-07-06T03:52:34.984337+0000","last_became_peered":"2023-07-06T03:52:34.984337+0000","last_unstale":"2023-07-06T03:52:34.986077+0000","last_undegraded":"2023-07-06T03:52:34.986077+0000","last_fullsized":"2023-07-06T03:52:34.986077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:49.655068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959431+0000","last_change":"2023-07-06T03:52:34.989786+0000","last_active":"2023-07-06T03:52:35.959431+0000","last_peered":"2023-07-06T03:52:35.959431+0000","last_clean":"2023-07-06T03:52:35.959431+0000","last_became_active":"2023-07-06T03:52:34.989494+0000","last_became_peered":"2023-07-06T03:52:34.989494+0000","last_unstale":"2023-07-06T03:52:35.959431+0000","last_undegraded":"2023-07-06T03:52:35.959431+0000","last_fullsized":"2023-07-06T03:52:35.959431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:10:14.471097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.980828+0000","last_change":"2023-07-06T03:52:34.975490+0000","last_active":"2023-07-06T03:52:34.980828+0000","last_peered":"2023-07-06T03:52:34.980828+0000","last_clean":"2023-07-06T03:52:34.980828+0000","last_became_active":"2023-07-06T03:52:34.975153+0000","last_became_peered":"2023-07-06T03:52:34.975153+0000","last_unstale":"2023-07-06T03:52:34.980828+0000","last_undegraded":"2023-07-06T03:52:34.980828+0000","last_fullsized":"2023-07-06T03:52:34.980828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:48:20.899074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"85'4","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.990324+0000","last_change":"2023-07-06T03:52:33.972008+0000","last_active":"2023-07-06T03:52:34.990324+0000","last_peered":"2023-07-06T03:52:34.990324+0000","last_clean":"2023-07-06T03:52:34.990324+0000","last_became_active":"2023-07-06T03:52:33.971546+0000","last_became_peered":"2023-07-06T03:52:33.971546+0000","last_unstale":"2023-07-06T03:52:34.990324+0000","last_undegraded":"2023-07-06T03:52:34.990324+0000","last_fullsized":"2023-07-06T03:52:34.990324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:38:47.581991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959954+0000","last_change":"2023-07-06T03:52:34.988755+0000","last_active":"2023-07-06T03:52:35.959954+0000","last_peered":"2023-07-06T03:52:35.959954+0000","last_clean":"2023-07-06T03:52:35.959954+0000","last_became_active":"2023-07-06T03:52:34.988423+0000","last_became_peered":"2023-07-06T03:52:34.988423+0000","last_unstale":"2023-07-06T03:52:35.959954+0000","last_undegraded":"2023-07-06T03:52:35.959954+0000","last_fullsized":"2023-07-06T03:52:35.959954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:42.357746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988566+0000","last_change":"2023-07-06T03:52:33.971055+0000","last_active":"2023-07-06T03:52:34.988566+0000","last_peered":"2023-07-06T03:52:34.988566+0000","last_clean":"2023-07-06T03:52:34.988566+0000","last_became_active":"2023-07-06T03:52:33.970732+0000","last_became_peered":"2023-07-06T03:52:33.970732+0000","last_unstale":"2023-07-06T03:52:34.988566+0000","last_undegraded":"2023-07-06T03:52:34.988566+0000","last_fullsized":"2023-07-06T03:52:34.988566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:34:41.448578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958526+0000","last_change":"2023-07-06T03:52:34.982511+0000","last_active":"2023-07-06T03:52:35.958526+0000","last_peered":"2023-07-06T03:52:35.958526+0000","last_clean":"2023-07-06T03:52:35.958526+0000","last_became_active":"2023-07-06T03:52:34.982247+0000","last_became_peered":"2023-07-06T03:52:34.982247+0000","last_unstale":"2023-07-06T03:52:35.958526+0000","last_undegraded":"2023-07-06T03:52:35.958526+0000","last_fullsized":"2023-07-06T03:52:35.958526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:14.879632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.991210+0000","last_change":"2023-07-06T03:52:33.969562+0000","last_active":"2023-07-06T03:52:34.991210+0000","last_peered":"2023-07-06T03:52:34.991210+0000","last_clean":"2023-07-06T03:52:34.991210+0000","last_became_active":"2023-07-06T03:52:33.969319+0000","last_became_peered":"2023-07-06T03:52:33.969319+0000","last_unstale":"2023-07-06T03:52:34.991210+0000","last_undegraded":"2023-07-06T03:52:34.991210+0000","last_fullsized":"2023-07-06T03:52:34.991210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:27:47.174750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.957986+0000","last_change":"2023-07-06T03:52:34.974988+0000","last_active":"2023-07-06T03:52:35.957986+0000","last_peered":"2023-07-06T03:52:35.957986+0000","last_clean":"2023-07-06T03:52:35.957986+0000","last_became_active":"2023-07-06T03:52:34.974707+0000","last_became_peered":"2023-07-06T03:52:34.974707+0000","last_unstale":"2023-07-06T03:52:35.957986+0000","last_undegraded":"2023-07-06T03:52:35.957986+0000","last_fullsized":"2023-07-06T03:52:35.957986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:17:34.625196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.985530+0000","last_change":"2023-07-06T03:52:33.975234+0000","last_active":"2023-07-06T03:52:34.985530+0000","last_peered":"2023-07-06T03:52:34.985530+0000","last_clean":"2023-07-06T03:52:34.985530+0000","last_became_active":"2023-07-06T03:52:33.974995+0000","last_became_peered":"2023-07-06T03:52:33.974995+0000","last_unstale":"2023-07-06T03:52:34.985530+0000","last_undegraded":"2023-07-06T03:52:34.985530+0000","last_fullsized":"2023-07-06T03:52:34.985530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:13:49.605835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.990296+0000","last_change":"2023-07-06T03:52:34.985412+0000","last_active":"2023-07-06T03:52:34.990296+0000","last_peered":"2023-07-06T03:52:34.990296+0000","last_clean":"2023-07-06T03:52:34.990296+0000","last_became_active":"2023-07-06T03:52:34.985068+0000","last_became_peered":"2023-07-06T03:52:34.985068+0000","last_unstale":"2023-07-06T03:52:34.990296+0000","last_undegraded":"2023-07-06T03:52:34.990296+0000","last_fullsized":"2023-07-06T03:52:34.990296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:00:51.296286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.981543+0000","last_change":"2023-07-06T03:52:33.962873+0000","last_active":"2023-07-06T03:52:34.981543+0000","last_peered":"2023-07-06T03:52:34.981543+0000","last_clean":"2023-07-06T03:52:34.981543+0000","last_became_active":"2023-07-06T03:52:33.962628+0000","last_became_peered":"2023-07-06T03:52:33.962628+0000","last_unstale":"2023-07-06T03:52:34.981543+0000","last_undegraded":"2023-07-06T03:52:34.981543+0000","last_fullsized":"2023-07-06T03:52:34.981543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:56:13.294284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958256+0000","last_change":"2023-07-06T03:52:34.980277+0000","last_active":"2023-07-06T03:52:35.958256+0000","last_peered":"2023-07-06T03:52:35.958256+0000","last_clean":"2023-07-06T03:52:35.958256+0000","last_became_active":"2023-07-06T03:52:34.979833+0000","last_became_peered":"2023-07-06T03:52:34.979833+0000","last_unstale":"2023-07-06T03:52:35.958256+0000","last_undegraded":"2023-07-06T03:52:35.958256+0000","last_fullsized":"2023-07-06T03:52:35.958256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:34:50.108422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.991350+0000","last_change":"2023-07-06T03:52:33.971390+0000","last_active":"2023-07-06T03:52:34.991350+0000","last_peered":"2023-07-06T03:52:34.991350+0000","last_clean":"2023-07-06T03:52:34.991350+0000","last_became_active":"2023-07-06T03:52:33.971107+0000","last_became_peered":"2023-07-06T03:52:33.971107+0000","last_unstale":"2023-07-06T03:52:34.991350+0000","last_undegraded":"2023-07-06T03:52:34.991350+0000","last_fullsized":"2023-07-06T03:52:34.991350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:43.344086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959883+0000","last_change":"2023-07-06T03:52:34.980322+0000","last_active":"2023-07-06T03:52:35.959883+0000","last_peered":"2023-07-06T03:52:35.959883+0000","last_clean":"2023-07-06T03:52:35.959883+0000","last_became_active":"2023-07-06T03:52:34.979784+0000","last_became_peered":"2023-07-06T03:52:34.979784+0000","last_unstale":"2023-07-06T03:52:35.959883+0000","last_undegraded":"2023-07-06T03:52:35.959883+0000","last_fullsized":"2023-07-06T03:52:35.959883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:56:11.702031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958491+0000","last_change":"2023-07-06T03:52:33.962319+0000","last_active":"2023-07-06T03:52:35.958491+0000","last_peered":"2023-07-06T03:52:35.958491+0000","last_clean":"2023-07-06T03:52:35.958491+0000","last_became_active":"2023-07-06T03:52:33.961990+0000","last_became_peered":"2023-07-06T03:52:33.961990+0000","last_unstale":"2023-07-06T03:52:35.958491+0000","last_undegraded":"2023-07-06T03:52:35.958491+0000","last_fullsized":"2023-07-06T03:52:35.958491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:14:13.492083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988499+0000","last_change":"2023-07-06T03:52:34.980267+0000","last_active":"2023-07-06T03:52:34.988499+0000","last_peered":"2023-07-06T03:52:34.988499+0000","last_clean":"2023-07-06T03:52:34.988499+0000","last_became_active":"2023-07-06T03:52:34.979997+0000","last_became_peered":"2023-07-06T03:52:34.979997+0000","last_unstale":"2023-07-06T03:52:34.988499+0000","last_undegraded":"2023-07-06T03:52:34.988499+0000","last_fullsized":"2023-07-06T03:52:34.988499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:14:10.352742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.958261+0000","last_change":"2023-07-06T03:52:33.971835+0000","last_active":"2023-07-06T03:52:34.958261+0000","last_peered":"2023-07-06T03:52:34.958261+0000","last_clean":"2023-07-06T03:52:34.958261+0000","last_became_active":"2023-07-06T03:52:33.971183+0000","last_became_peered":"2023-07-06T03:52:33.971183+0000","last_unstale":"2023-07-06T03:52:34.958261+0000","last_undegraded":"2023-07-06T03:52:34.958261+0000","last_fullsized":"2023-07-06T03:52:34.958261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:07:56.417410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.987781+0000","last_change":"2023-07-06T03:52:34.979613+0000","last_active":"2023-07-06T03:52:34.987781+0000","last_peered":"2023-07-06T03:52:34.987781+0000","last_clean":"2023-07-06T03:52:34.987781+0000","last_became_active":"2023-07-06T03:52:34.979316+0000","last_became_peered":"2023-07-06T03:52:34.979316+0000","last_unstale":"2023-07-06T03:52:34.987781+0000","last_undegraded":"2023-07-06T03:52:34.987781+0000","last_fullsized":"2023-07-06T03:52:34.987781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:22:03.328610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.960568+0000","last_change":"2023-07-06T03:52:33.971837+0000","last_active":"2023-07-06T03:52:34.960568+0000","last_peered":"2023-07-06T03:52:34.960568+0000","last_clean":"2023-07-06T03:52:34.960568+0000","last_became_active":"2023-07-06T03:52:33.971201+0000","last_became_peered":"2023-07-06T03:52:33.971201+0000","last_unstale":"2023-07-06T03:52:34.960568+0000","last_undegraded":"2023-07-06T03:52:34.960568+0000","last_fullsized":"2023-07-06T03:52:34.960568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:08:31.234265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958561+0000","last_change":"2023-07-06T03:52:34.978612+0000","last_active":"2023-07-06T03:52:35.958561+0000","last_peered":"2023-07-06T03:52:35.958561+0000","last_clean":"2023-07-06T03:52:35.958561+0000","last_became_active":"2023-07-06T03:52:34.978347+0000","last_became_peered":"2023-07-06T03:52:34.978347+0000","last_unstale":"2023-07-06T03:52:35.958561+0000","last_undegraded":"2023-07-06T03:52:35.958561+0000","last_fullsized":"2023-07-06T03:52:35.958561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:35.539652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958294+0000","last_change":"2023-07-06T03:52:33.962485+0000","last_active":"2023-07-06T03:52:35.958294+0000","last_peered":"2023-07-06T03:52:35.958294+0000","last_clean":"2023-07-06T03:52:35.958294+0000","last_became_active":"2023-07-06T03:52:33.962168+0000","last_became_peered":"2023-07-06T03:52:33.962168+0000","last_unstale":"2023-07-06T03:52:35.958294+0000","last_undegraded":"2023-07-06T03:52:35.958294+0000","last_fullsized":"2023-07-06T03:52:35.958294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:45.645931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.987779+0000","last_change":"2023-07-06T03:52:34.975836+0000","last_active":"2023-07-06T03:52:34.987779+0000","last_peered":"2023-07-06T03:52:34.987779+0000","last_clean":"2023-07-06T03:52:34.987779+0000","last_became_active":"2023-07-06T03:52:34.975547+0000","last_became_peered":"2023-07-06T03:52:34.975547+0000","last_unstale":"2023-07-06T03:52:34.987779+0000","last_undegraded":"2023-07-06T03:52:34.987779+0000","last_fullsized":"2023-07-06T03:52:34.987779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:41:42.215703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"85'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958711+0000","last_change":"2023-07-06T03:52:33.971177+0000","last_active":"2023-07-06T03:52:35.958711+0000","last_peered":"2023-07-06T03:52:35.958711+0000","last_clean":"2023-07-06T03:52:35.958711+0000","last_became_active":"2023-07-06T03:52:33.970653+0000","last_became_peered":"2023-07-06T03:52:33.970653+0000","last_unstale":"2023-07-06T03:52:35.958711+0000","last_undegraded":"2023-07-06T03:52:35.958711+0000","last_fullsized":"2023-07-06T03:52:35.958711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:57:50.263331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959845+0000","last_change":"2023-07-06T03:52:34.982461+0000","last_active":"2023-07-06T03:52:35.959845+0000","last_peered":"2023-07-06T03:52:35.959845+0000","last_clean":"2023-07-06T03:52:35.959845+0000","last_became_active":"2023-07-06T03:52:34.982192+0000","last_became_peered":"2023-07-06T03:52:34.982192+0000","last_unstale":"2023-07-06T03:52:35.959845+0000","last_undegraded":"2023-07-06T03:52:35.959845+0000","last_fullsized":"2023-07-06T03:52:35.959845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:02.793520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.994073+0000","last_change":"2023-07-06T03:52:33.969663+0000","last_active":"2023-07-06T03:52:34.994073+0000","last_peered":"2023-07-06T03:52:34.994073+0000","last_clean":"2023-07-06T03:52:34.994073+0000","last_became_active":"2023-07-06T03:52:33.969291+0000","last_became_peered":"2023-07-06T03:52:33.969291+0000","last_unstale":"2023-07-06T03:52:34.994073+0000","last_undegraded":"2023-07-06T03:52:34.994073+0000","last_fullsized":"2023-07-06T03:52:34.994073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:58:06.580256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.989076+0000","last_change":"2023-07-06T03:52:34.983292+0000","last_active":"2023-07-06T03:52:34.989076+0000","last_peered":"2023-07-06T03:52:34.989076+0000","last_clean":"2023-07-06T03:52:34.989076+0000","last_became_active":"2023-07-06T03:52:34.982689+0000","last_became_peered":"2023-07-06T03:52:34.982689+0000","last_unstale":"2023-07-06T03:52:34.989076+0000","last_undegraded":"2023-07-06T03:52:34.989076+0000","last_fullsized":"2023-07-06T03:52:34.989076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:18:55.170333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.991708+0000","last_change":"2023-07-06T03:52:33.969641+0000","last_active":"2023-07-06T03:52:34.991708+0000","last_peered":"2023-07-06T03:52:34.991708+0000","last_clean":"2023-07-06T03:52:34.991708+0000","last_became_active":"2023-07-06T03:52:33.969334+0000","last_became_peered":"2023-07-06T03:52:33.969334+0000","last_unstale":"2023-07-06T03:52:34.991708+0000","last_undegraded":"2023-07-06T03:52:34.991708+0000","last_fullsized":"2023-07-06T03:52:34.991708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:31:26.756116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983648+0000","last_change":"2023-07-06T03:52:34.975282+0000","last_active":"2023-07-06T03:52:34.983648+0000","last_peered":"2023-07-06T03:52:34.983648+0000","last_clean":"2023-07-06T03:52:34.983648+0000","last_became_active":"2023-07-06T03:52:34.975010+0000","last_became_peered":"2023-07-06T03:52:34.975010+0000","last_unstale":"2023-07-06T03:52:34.983648+0000","last_undegraded":"2023-07-06T03:52:34.983648+0000","last_fullsized":"2023-07-06T03:52:34.983648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:08:54.635642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.965449+0000","last_change":"2023-07-06T03:52:33.970052+0000","last_active":"2023-07-06T03:52:34.965449+0000","last_peered":"2023-07-06T03:52:34.965449+0000","last_clean":"2023-07-06T03:52:34.965449+0000","last_became_active":"2023-07-06T03:52:33.969777+0000","last_became_peered":"2023-07-06T03:52:33.969777+0000","last_unstale":"2023-07-06T03:52:34.965449+0000","last_undegraded":"2023-07-06T03:52:34.965449+0000","last_fullsized":"2023-07-06T03:52:34.965449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:07:30.791888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986968+0000","last_change":"2023-07-06T03:52:34.975902+0000","last_active":"2023-07-06T03:52:34.986968+0000","last_peered":"2023-07-06T03:52:34.986968+0000","last_clean":"2023-07-06T03:52:34.986968+0000","last_became_active":"2023-07-06T03:52:34.975592+0000","last_became_peered":"2023-07-06T03:52:34.975592+0000","last_unstale":"2023-07-06T03:52:34.986968+0000","last_undegraded":"2023-07-06T03:52:34.986968+0000","last_fullsized":"2023-07-06T03:52:34.986968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:13:36.660096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.961307+0000","last_change":"2023-07-06T03:52:33.974022+0000","last_active":"2023-07-06T03:52:34.961307+0000","last_peered":"2023-07-06T03:52:34.961307+0000","last_clean":"2023-07-06T03:52:34.961307+0000","last_became_active":"2023-07-06T03:52:33.973660+0000","last_became_peered":"2023-07-06T03:52:33.973660+0000","last_unstale":"2023-07-06T03:52:34.961307+0000","last_undegraded":"2023-07-06T03:52:34.961307+0000","last_fullsized":"2023-07-06T03:52:34.961307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:51:13.802000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988783+0000","last_change":"2023-07-06T03:52:34.981246+0000","last_active":"2023-07-06T03:52:34.988783+0000","last_peered":"2023-07-06T03:52:34.988783+0000","last_clean":"2023-07-06T03:52:34.988783+0000","last_became_active":"2023-07-06T03:52:34.980967+0000","last_became_peered":"2023-07-06T03:52:34.980967+0000","last_unstale":"2023-07-06T03:52:34.988783+0000","last_undegraded":"2023-07-06T03:52:34.988783+0000","last_fullsized":"2023-07-06T03:52:34.988783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:01.473197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.957695+0000","last_change":"2023-07-06T03:52:33.970438+0000","last_active":"2023-07-06T03:52:35.957695+0000","last_peered":"2023-07-06T03:52:35.957695+0000","last_clean":"2023-07-06T03:52:35.957695+0000","last_became_active":"2023-07-06T03:52:33.970151+0000","last_became_peered":"2023-07-06T03:52:33.970151+0000","last_unstale":"2023-07-06T03:52:35.957695+0000","last_undegraded":"2023-07-06T03:52:35.957695+0000","last_fullsized":"2023-07-06T03:52:35.957695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:28:25.343977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.981436+0000","last_change":"2023-07-06T03:52:34.978791+0000","last_active":"2023-07-06T03:52:34.981436+0000","last_peered":"2023-07-06T03:52:34.981436+0000","last_clean":"2023-07-06T03:52:34.981436+0000","last_became_active":"2023-07-06T03:52:34.978512+0000","last_became_peered":"2023-07-06T03:52:34.978512+0000","last_unstale":"2023-07-06T03:52:34.981436+0000","last_undegraded":"2023-07-06T03:52:34.981436+0000","last_fullsized":"2023-07-06T03:52:34.981436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:14:52.498021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958739+0000","last_change":"2023-07-06T03:52:33.970072+0000","last_active":"2023-07-06T03:52:35.958739+0000","last_peered":"2023-07-06T03:52:35.958739+0000","last_clean":"2023-07-06T03:52:35.958739+0000","last_became_active":"2023-07-06T03:52:33.969827+0000","last_became_peered":"2023-07-06T03:52:33.969827+0000","last_unstale":"2023-07-06T03:52:35.958739+0000","last_undegraded":"2023-07-06T03:52:35.958739+0000","last_fullsized":"2023-07-06T03:52:35.958739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:36:38.665055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.988749+0000","last_change":"2023-07-06T03:52:34.981416+0000","last_active":"2023-07-06T03:52:34.988749+0000","last_peered":"2023-07-06T03:52:34.988749+0000","last_clean":"2023-07-06T03:52:34.988749+0000","last_became_active":"2023-07-06T03:52:34.981122+0000","last_became_peered":"2023-07-06T03:52:34.981122+0000","last_unstale":"2023-07-06T03:52:34.988749+0000","last_undegraded":"2023-07-06T03:52:34.988749+0000","last_fullsized":"2023-07-06T03:52:34.988749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:42:25.798222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958574+0000","last_change":"2023-07-06T03:52:33.967889+0000","last_active":"2023-07-06T03:52:35.958574+0000","last_peered":"2023-07-06T03:52:35.958574+0000","last_clean":"2023-07-06T03:52:35.958574+0000","last_became_active":"2023-07-06T03:52:33.967625+0000","last_became_peered":"2023-07-06T03:52:33.967625+0000","last_unstale":"2023-07-06T03:52:35.958574+0000","last_undegraded":"2023-07-06T03:52:35.958574+0000","last_fullsized":"2023-07-06T03:52:35.958574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:04.279525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.985073+0000","last_change":"2023-07-06T03:52:34.972471+0000","last_active":"2023-07-06T03:52:34.985073+0000","last_peered":"2023-07-06T03:52:34.985073+0000","last_clean":"2023-07-06T03:52:34.985073+0000","last_became_active":"2023-07-06T03:52:34.972167+0000","last_became_peered":"2023-07-06T03:52:34.972167+0000","last_unstale":"2023-07-06T03:52:34.985073+0000","last_undegraded":"2023-07-06T03:52:34.985073+0000","last_fullsized":"2023-07-06T03:52:34.985073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:25:26.256618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.969740+0000","last_change":"2023-07-06T03:52:33.962367+0000","last_active":"2023-07-06T03:52:34.969740+0000","last_peered":"2023-07-06T03:52:34.969740+0000","last_clean":"2023-07-06T03:52:34.969740+0000","last_became_active":"2023-07-06T03:52:33.962012+0000","last_became_peered":"2023-07-06T03:52:33.962012+0000","last_unstale":"2023-07-06T03:52:34.969740+0000","last_undegraded":"2023-07-06T03:52:34.969740+0000","last_fullsized":"2023-07-06T03:52:34.969740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:59:29.963987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.991020+0000","last_change":"2023-07-06T03:52:34.990137+0000","last_active":"2023-07-06T03:52:34.991020+0000","last_peered":"2023-07-06T03:52:34.991020+0000","last_clean":"2023-07-06T03:52:34.991020+0000","last_became_active":"2023-07-06T03:52:34.989665+0000","last_became_peered":"2023-07-06T03:52:34.989665+0000","last_unstale":"2023-07-06T03:52:34.991020+0000","last_undegraded":"2023-07-06T03:52:34.991020+0000","last_fullsized":"2023-07-06T03:52:34.991020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:43:37.905199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958656+0000","last_change":"2023-07-06T03:52:33.972579+0000","last_active":"2023-07-06T03:52:35.958656+0000","last_peered":"2023-07-06T03:52:35.958656+0000","last_clean":"2023-07-06T03:52:35.958656+0000","last_became_active":"2023-07-06T03:52:33.972337+0000","last_became_peered":"2023-07-06T03:52:33.972337+0000","last_unstale":"2023-07-06T03:52:35.958656+0000","last_undegraded":"2023-07-06T03:52:35.958656+0000","last_fullsized":"2023-07-06T03:52:35.958656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:39:11.615424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959613+0000","last_change":"2023-07-06T03:52:34.971572+0000","last_active":"2023-07-06T03:52:35.959613+0000","last_peered":"2023-07-06T03:52:35.959613+0000","last_clean":"2023-07-06T03:52:35.959613+0000","last_became_active":"2023-07-06T03:52:34.971273+0000","last_became_peered":"2023-07-06T03:52:34.971273+0000","last_unstale":"2023-07-06T03:52:35.959613+0000","last_undegraded":"2023-07-06T03:52:35.959613+0000","last_fullsized":"2023-07-06T03:52:35.959613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:34:32.890102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.962685+0000","last_change":"2023-07-06T03:52:33.965584+0000","last_active":"2023-07-06T03:52:34.962685+0000","last_peered":"2023-07-06T03:52:34.962685+0000","last_clean":"2023-07-06T03:52:34.962685+0000","last_became_active":"2023-07-06T03:52:33.964659+0000","last_became_peered":"2023-07-06T03:52:33.964659+0000","last_unstale":"2023-07-06T03:52:34.962685+0000","last_undegraded":"2023-07-06T03:52:34.962685+0000","last_fullsized":"2023-07-06T03:52:34.962685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:40.458505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958453+0000","last_change":"2023-07-06T03:52:34.986293+0000","last_active":"2023-07-06T03:52:35.958453+0000","last_peered":"2023-07-06T03:52:35.958453+0000","last_clean":"2023-07-06T03:52:35.958453+0000","last_became_active":"2023-07-06T03:52:34.985619+0000","last_became_peered":"2023-07-06T03:52:34.985619+0000","last_unstale":"2023-07-06T03:52:35.958453+0000","last_undegraded":"2023-07-06T03:52:35.958453+0000","last_fullsized":"2023-07-06T03:52:35.958453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:16:14.184623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.957526+0000","last_change":"2023-07-06T03:52:33.970691+0000","last_active":"2023-07-06T03:52:34.957526+0000","last_peered":"2023-07-06T03:52:34.957526+0000","last_clean":"2023-07-06T03:52:34.957526+0000","last_became_active":"2023-07-06T03:52:33.970396+0000","last_became_peered":"2023-07-06T03:52:33.970396+0000","last_unstale":"2023-07-06T03:52:34.957526+0000","last_undegraded":"2023-07-06T03:52:34.957526+0000","last_fullsized":"2023-07-06T03:52:34.957526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:45.645931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986733+0000","last_change":"2023-07-06T03:52:34.985020+0000","last_active":"2023-07-06T03:52:34.986733+0000","last_peered":"2023-07-06T03:52:34.986733+0000","last_clean":"2023-07-06T03:52:34.986733+0000","last_became_active":"2023-07-06T03:52:34.984742+0000","last_became_peered":"2023-07-06T03:52:34.984742+0000","last_unstale":"2023-07-06T03:52:34.986733+0000","last_undegraded":"2023-07-06T03:52:34.986733+0000","last_fullsized":"2023-07-06T03:52:34.986733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:17:22.458751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.958230+0000","last_change":"2023-07-06T03:52:33.972861+0000","last_active":"2023-07-06T03:52:34.958230+0000","last_peered":"2023-07-06T03:52:34.958230+0000","last_clean":"2023-07-06T03:52:34.958230+0000","last_became_active":"2023-07-06T03:52:33.972618+0000","last_became_peered":"2023-07-06T03:52:33.972618+0000","last_unstale":"2023-07-06T03:52:34.958230+0000","last_undegraded":"2023-07-06T03:52:34.958230+0000","last_fullsized":"2023-07-06T03:52:34.958230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:01:12.042090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959576+0000","last_change":"2023-07-06T03:52:34.989323+0000","last_active":"2023-07-06T03:52:35.959576+0000","last_peered":"2023-07-06T03:52:35.959576+0000","last_clean":"2023-07-06T03:52:35.959576+0000","last_became_active":"2023-07-06T03:52:34.989002+0000","last_became_peered":"2023-07-06T03:52:34.989002+0000","last_unstale":"2023-07-06T03:52:35.959576+0000","last_undegraded":"2023-07-06T03:52:35.959576+0000","last_fullsized":"2023-07-06T03:52:35.959576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:16:23.267541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.957018+0000","last_change":"2023-07-06T03:52:33.971920+0000","last_active":"2023-07-06T03:52:34.957018+0000","last_peered":"2023-07-06T03:52:34.957018+0000","last_clean":"2023-07-06T03:52:34.957018+0000","last_became_active":"2023-07-06T03:52:33.971535+0000","last_became_peered":"2023-07-06T03:52:33.971535+0000","last_unstale":"2023-07-06T03:52:34.957018+0000","last_undegraded":"2023-07-06T03:52:34.957018+0000","last_fullsized":"2023-07-06T03:52:34.957018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:49:10.006497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959935+0000","last_change":"2023-07-06T03:52:34.971829+0000","last_active":"2023-07-06T03:52:35.959935+0000","last_peered":"2023-07-06T03:52:35.959935+0000","last_clean":"2023-07-06T03:52:35.959935+0000","last_became_active":"2023-07-06T03:52:34.971447+0000","last_became_peered":"2023-07-06T03:52:34.971447+0000","last_unstale":"2023-07-06T03:52:35.959935+0000","last_undegraded":"2023-07-06T03:52:35.959935+0000","last_fullsized":"2023-07-06T03:52:35.959935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:26:01.534419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958348+0000","last_change":"2023-07-06T03:52:33.970798+0000","last_active":"2023-07-06T03:52:35.958348+0000","last_peered":"2023-07-06T03:52:35.958348+0000","last_clean":"2023-07-06T03:52:35.958348+0000","last_became_active":"2023-07-06T03:52:33.970470+0000","last_became_peered":"2023-07-06T03:52:33.970470+0000","last_unstale":"2023-07-06T03:52:35.958348+0000","last_undegraded":"2023-07-06T03:52:35.958348+0000","last_fullsized":"2023-07-06T03:52:35.958348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:55:47.828003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":130,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.969121+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:52:34.969121+0000","last_peered":"2023-07-06T03:52:34.969121+0000","last_clean":"2023-07-06T03:52:34.969121+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:52:34.969121+0000","last_undegraded":"2023-07-06T03:52:34.969121+0000","last_fullsized":"2023-07-06T03:52:34.969121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:38:19.738306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.983689+0000","last_change":"2023-07-06T03:52:34.970874+0000","last_active":"2023-07-06T03:52:34.983689+0000","last_peered":"2023-07-06T03:52:34.983689+0000","last_clean":"2023-07-06T03:52:34.983689+0000","last_became_active":"2023-07-06T03:52:34.970612+0000","last_became_peered":"2023-07-06T03:52:34.970612+0000","last_unstale":"2023-07-06T03:52:34.983689+0000","last_undegraded":"2023-07-06T03:52:34.983689+0000","last_fullsized":"2023-07-06T03:52:34.983689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:38:58.774100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.989268+0000","last_change":"2023-07-06T03:52:33.974733+0000","last_active":"2023-07-06T03:52:34.989268+0000","last_peered":"2023-07-06T03:52:34.989268+0000","last_clean":"2023-07-06T03:52:34.989268+0000","last_became_active":"2023-07-06T03:52:33.974396+0000","last_became_peered":"2023-07-06T03:52:33.974396+0000","last_unstale":"2023-07-06T03:52:34.989268+0000","last_undegraded":"2023-07-06T03:52:34.989268+0000","last_fullsized":"2023-07-06T03:52:34.989268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:24:20.059347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958184+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:52:35.958184+0000","last_peered":"2023-07-06T03:52:35.958184+0000","last_clean":"2023-07-06T03:52:35.958184+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:52:35.958184+0000","last_undegraded":"2023-07-06T03:52:35.958184+0000","last_fullsized":"2023-07-06T03:52:35.958184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:39:32.576440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958681+0000","last_change":"2023-07-06T03:52:34.981298+0000","last_active":"2023-07-06T03:52:35.958681+0000","last_peered":"2023-07-06T03:52:35.958681+0000","last_clean":"2023-07-06T03:52:35.958681+0000","last_became_active":"2023-07-06T03:52:34.981044+0000","last_became_peered":"2023-07-06T03:52:34.981044+0000","last_unstale":"2023-07-06T03:52:35.958681+0000","last_undegraded":"2023-07-06T03:52:35.958681+0000","last_fullsized":"2023-07-06T03:52:35.958681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:20:07.915721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.987001+0000","last_change":"2023-07-06T03:52:33.974721+0000","last_active":"2023-07-06T03:52:34.987001+0000","last_peered":"2023-07-06T03:52:34.987001+0000","last_clean":"2023-07-06T03:52:34.987001+0000","last_became_active":"2023-07-06T03:52:33.974466+0000","last_became_peered":"2023-07-06T03:52:33.974466+0000","last_unstale":"2023-07-06T03:52:34.987001+0000","last_undegraded":"2023-07-06T03:52:34.987001+0000","last_fullsized":"2023-07-06T03:52:34.987001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:41:12.435532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.957803+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:52:35.957803+0000","last_peered":"2023-07-06T03:52:35.957803+0000","last_clean":"2023-07-06T03:52:35.957803+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:52:35.957803+0000","last_undegraded":"2023-07-06T03:52:35.957803+0000","last_fullsized":"2023-07-06T03:52:35.957803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:45:51.126881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.989727+0000","last_change":"2023-07-06T03:52:34.976773+0000","last_active":"2023-07-06T03:52:34.989727+0000","last_peered":"2023-07-06T03:52:34.989727+0000","last_clean":"2023-07-06T03:52:34.989727+0000","last_became_active":"2023-07-06T03:52:34.976393+0000","last_became_peered":"2023-07-06T03:52:34.976393+0000","last_unstale":"2023-07-06T03:52:34.989727+0000","last_undegraded":"2023-07-06T03:52:34.989727+0000","last_fullsized":"2023-07-06T03:52:34.989727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:21:05.045222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.962472+0000","last_change":"2023-07-06T03:52:33.970885+0000","last_active":"2023-07-06T03:52:34.962472+0000","last_peered":"2023-07-06T03:52:34.962472+0000","last_clean":"2023-07-06T03:52:34.962472+0000","last_became_active":"2023-07-06T03:52:33.970603+0000","last_became_peered":"2023-07-06T03:52:33.970603+0000","last_unstale":"2023-07-06T03:52:34.962472+0000","last_undegraded":"2023-07-06T03:52:34.962472+0000","last_fullsized":"2023-07-06T03:52:34.962472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:54:26.445108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958482+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:52:35.958482+0000","last_peered":"2023-07-06T03:52:35.958482+0000","last_clean":"2023-07-06T03:52:35.958482+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:52:35.958482+0000","last_undegraded":"2023-07-06T03:52:35.958482+0000","last_fullsized":"2023-07-06T03:52:35.958482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:45.067333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":278,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.957848+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:52:34.957848+0000","last_peered":"2023-07-06T03:52:34.957848+0000","last_clean":"2023-07-06T03:52:34.957848+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:52:34.957848+0000","last_undegraded":"2023-07-06T03:52:34.957848+0000","last_fullsized":"2023-07-06T03:52:34.957848+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:06:37.636080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.986572+0000","last_change":"2023-07-06T03:52:34.980363+0000","last_active":"2023-07-06T03:52:34.986572+0000","last_peered":"2023-07-06T03:52:34.986572+0000","last_clean":"2023-07-06T03:52:34.986572+0000","last_became_active":"2023-07-06T03:52:34.980037+0000","last_became_peered":"2023-07-06T03:52:34.980037+0000","last_unstale":"2023-07-06T03:52:34.986572+0000","last_undegraded":"2023-07-06T03:52:34.986572+0000","last_fullsized":"2023-07-06T03:52:34.986572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:44:17.184944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.979511+0000","last_change":"2023-07-06T03:52:33.966585+0000","last_active":"2023-07-06T03:52:34.979511+0000","last_peered":"2023-07-06T03:52:34.979511+0000","last_clean":"2023-07-06T03:52:34.979511+0000","last_became_active":"2023-07-06T03:52:33.966320+0000","last_became_peered":"2023-07-06T03:52:33.966320+0000","last_unstale":"2023-07-06T03:52:34.979511+0000","last_undegraded":"2023-07-06T03:52:34.979511+0000","last_fullsized":"2023-07-06T03:52:34.979511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:07:33.793257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958173+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:52:35.958173+0000","last_peered":"2023-07-06T03:52:35.958173+0000","last_clean":"2023-07-06T03:52:35.958173+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:52:35.958173+0000","last_undegraded":"2023-07-06T03:52:35.958173+0000","last_fullsized":"2023-07-06T03:52:35.958173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:17:04.875422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.985660+0000","last_change":"2023-07-06T03:52:34.975165+0000","last_active":"2023-07-06T03:52:34.985660+0000","last_peered":"2023-07-06T03:52:34.985660+0000","last_clean":"2023-07-06T03:52:34.985660+0000","last_became_active":"2023-07-06T03:52:34.974821+0000","last_became_peered":"2023-07-06T03:52:34.974821+0000","last_unstale":"2023-07-06T03:52:34.985660+0000","last_undegraded":"2023-07-06T03:52:34.985660+0000","last_fullsized":"2023-07-06T03:52:34.985660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:33:17.996105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959871+0000","last_change":"2023-07-06T03:52:33.972725+0000","last_active":"2023-07-06T03:52:35.959871+0000","last_peered":"2023-07-06T03:52:35.959871+0000","last_clean":"2023-07-06T03:52:35.959871+0000","last_became_active":"2023-07-06T03:52:33.972458+0000","last_became_peered":"2023-07-06T03:52:33.972458+0000","last_unstale":"2023-07-06T03:52:35.959871+0000","last_undegraded":"2023-07-06T03:52:35.959871+0000","last_fullsized":"2023-07-06T03:52:35.959871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:05:10.364515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958761+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:52:35.958761+0000","last_peered":"2023-07-06T03:52:35.958761+0000","last_clean":"2023-07-06T03:52:35.958761+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:52:35.958761+0000","last_undegraded":"2023-07-06T03:52:35.958761+0000","last_fullsized":"2023-07-06T03:52:35.958761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:49:54.804514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.980981+0000","last_change":"2023-07-06T03:52:34.974983+0000","last_active":"2023-07-06T03:52:34.980981+0000","last_peered":"2023-07-06T03:52:34.980981+0000","last_clean":"2023-07-06T03:52:34.980981+0000","last_became_active":"2023-07-06T03:52:34.974730+0000","last_became_peered":"2023-07-06T03:52:34.974730+0000","last_unstale":"2023-07-06T03:52:34.980981+0000","last_undegraded":"2023-07-06T03:52:34.980981+0000","last_fullsized":"2023-07-06T03:52:34.980981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:54:06.326418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.959329+0000","last_change":"2023-07-06T03:52:33.974879+0000","last_active":"2023-07-06T03:52:34.959329+0000","last_peered":"2023-07-06T03:52:34.959329+0000","last_clean":"2023-07-06T03:52:34.959329+0000","last_became_active":"2023-07-06T03:52:33.974574+0000","last_became_peered":"2023-07-06T03:52:33.974574+0000","last_unstale":"2023-07-06T03:52:34.959329+0000","last_undegraded":"2023-07-06T03:52:34.959329+0000","last_fullsized":"2023-07-06T03:52:34.959329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:45:15.238557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.959981+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:52:35.959981+0000","last_peered":"2023-07-06T03:52:35.959981+0000","last_clean":"2023-07-06T03:52:35.959981+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:52:35.959981+0000","last_undegraded":"2023-07-06T03:52:35.959981+0000","last_fullsized":"2023-07-06T03:52:35.959981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:01.339783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.976650+0000","last_change":"2023-07-06T03:52:34.973908+0000","last_active":"2023-07-06T03:52:34.976650+0000","last_peered":"2023-07-06T03:52:34.976650+0000","last_clean":"2023-07-06T03:52:34.976650+0000","last_became_active":"2023-07-06T03:52:34.973618+0000","last_became_peered":"2023-07-06T03:52:34.973618+0000","last_unstale":"2023-07-06T03:52:34.976650+0000","last_undegraded":"2023-07-06T03:52:34.976650+0000","last_fullsized":"2023-07-06T03:52:34.976650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:33.943321+0000","last_clean_scrub_stamp":"2023-07-06T03:52:33.943321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:47:00.396664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-06T03:52:34.969269+0000","last_change":"2023-07-06T03:52:33.970361+0000","last_active":"2023-07-06T03:52:34.969269+0000","last_peered":"2023-07-06T03:52:34.969269+0000","last_clean":"2023-07-06T03:52:34.969269+0000","last_became_active":"2023-07-06T03:52:33.970075+0000","last_became_peered":"2023-07-06T03:52:33.970075+0000","last_unstale":"2023-07-06T03:52:34.969269+0000","last_undegraded":"2023-07-06T03:52:34.969269+0000","last_fullsized":"2023-07-06T03:52:34.969269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:52:32.934485+0000","last_clean_scrub_stamp":"2023-07-06T03:52:32.934485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:46:44.854195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-06T03:52:35.958741+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:52:35.958741+0000","last_peered":"2023-07-06T03:52:35.958741+0000","last_clean":"2023-07-06T03:52:35.958741+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:52:35.958741+0000","last_undegraded":"2023-07-06T03:52:35.958741+0000","last_fullsized":"2023-07-06T03:52:35.958741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:41:30.642773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":470899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68799999999999994,"15min":0.71299999999999997},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.91200000000000003,"5min":4.0570000000000004,"15min":4.0570000000000004},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.67600000000000005,"15min":0.78000000000000003},"min":{"1min":0.47799999999999998,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67000000000000004,"15min":0.72999999999999998},"min":{"1min":0.46200000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86499999999999999,"5min":0.95799999999999996,"15min":2.6219999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70399999999999996,"15min":0.86699999999999999},"min":{"1min":0.501,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89900000000000002,"5min":4.0389999999999997,"15min":6.6310000000000002},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.67600000000000005,"15min":0.78600000000000003},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":3.77},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.70899999999999996,"15min":0.73299999999999998},"min":{"1min":0.56399999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94399999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70099999999999996,"15min":0.70799999999999996},"min":{"1min":0.46700000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":1.024,"15min":1.024},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.747,"15min":0.755},"min":{"1min":0.58899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.85699999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.622},"min":{"1min":0.437,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80300000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.65100000000000002,"15min":0.70699999999999996},"min":{"1min":0.51500000000000001,"5min":0.375,"15min":0.374},"max":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":1.4590000000000001},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66700000000000004,"15min":0.69599999999999995},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.88500000000000001,"5min":0.96699999999999997,"15min":1.0820000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69099999999999995,"15min":0.67600000000000005},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.871,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.61199999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.90400000000000003,"5min":1.03,"15min":1.5349999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73399999999999999,"15min":0.78600000000000003},"min":{"1min":0.52100000000000002,"5min":0.44800000000000001,"15min":0.42299999999999999},"max":{"1min":0.91200000000000003,"5min":1.006,"15min":1.3779999999999999},"last":0.753}]}]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":1064960,"data_stored":472587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70199999999999996,"15min":0.67000000000000004},"min":{"1min":0.55500000000000005,"5min":0.30599999999999999,"15min":0.29399999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.62},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77000000000000002,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.92600000000000005,"5min":3.3399999999999999,"15min":3.3399999999999999},"last":0.79900000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68400000000000005,"15min":0.67400000000000004},"min":{"1min":0.57399999999999995,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.53700000000000003,"5min":0.441,"15min":0.40100000000000002},"max":{"1min":0.94699999999999995,"5min":0.96399999999999997,"15min":1.1759999999999999},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71599999999999997,"15min":0.67600000000000005},"min":{"1min":0.51100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90700000000000003,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94799999999999995,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.74099999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69599999999999995,"15min":0.752},"min":{"1min":0.58999999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88500000000000001,"5min":0.93200000000000005,"15min":2.1850000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.70699999999999996,"15min":0.73399999999999999},"min":{"1min":0.51000000000000001,"5min":0.40400000000000003,"15min":0.373},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.093},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66700000000000004,"15min":0.71199999999999997},"min":{"1min":0.42799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96899999999999997,"5min":1.0309999999999999,"15min":1.1339999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.72999999999999998},"min":{"1min":0.48699999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.085},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64600000000000002,"15min":0.70799999999999996},"min":{"1min":0.48199999999999998,"5min":0.443,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":0.94799999999999995,"15min":1.218},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.78300000000000003},"min":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.36299999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.198},"last":0.81799999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:52:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72399999999999998,"15min":0.81499999999999995},"min":{"1min":0.51100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0269999999999999,"5min":1.054,"15min":1.323},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73599999999999999,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.504,"15min":0.41499999999999998},"max":{"1min":0.94599999999999995,"5min":1.028,"15min":1.4139999999999999},"last":0.77900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1650688,"data_stored":1061953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42999999999999999,"15min":0.53700000000000003},"min":{"1min":0.28999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.67000000000000004,"5min":1.002,"15min":2.8879999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.498,"15min":0.56899999999999995},"min":{"1min":0.371,"5min":0.313,"15min":0.313},"max":{"1min":0.69099999999999995,"5min":0.78300000000000003,"15min":1.96},"last":0.435}]},{"osd":2,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45500000000000002,"15min":0.88200000000000001},"min":{"1min":0.312,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59799999999999998,"5min":0.70799999999999996,"15min":13.055999999999999},"last":0.317},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54200000000000004,"15min":0.63300000000000001},"min":{"1min":0.39400000000000002,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.81499999999999995,"5min":0.92400000000000004,"15min":2.8479999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.60899999999999999},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66000000000000003,"5min":0.96799999999999997,"15min":2.9249999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48399999999999999,"15min":0.54300000000000004},"min":{"1min":0.38100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70599999999999996,"5min":0.82599999999999996,"15min":1.2709999999999999},"last":0.496}]},{"osd":4,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.751,"15min":1.151},"min":{"1min":0.63700000000000001,"5min":0.47599999999999998,"15min":0.39200000000000002},"max":{"1min":0.94399999999999995,"5min":1.0640000000000001,"15min":12.989000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.77600000000000002,"15min":0.82999999999999996},"min":{"1min":0.70799999999999996,"5min":0.53000000000000003,"15min":0.46000000000000002},"max":{"1min":1.004,"5min":1.371,"15min":2.7629999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73799999999999999,"15min":0.77700000000000002},"min":{"1min":0.60199999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.3100000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.72699999999999998,"15min":1.155},"min":{"1min":0.56999999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0529999999999999,"5min":1.1020000000000001,"15min":13.023999999999999},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.75900000000000001,"15min":0.81499999999999995},"min":{"1min":0.58099999999999996,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.044,"5min":1.119,"15min":1.365},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77400000000000002,"15min":1.1990000000000001},"min":{"1min":0.56299999999999994,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96999999999999997,"5min":1.254,"15min":12.941000000000001},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.75700000000000001,"15min":0.83099999999999996},"min":{"1min":0.64600000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.042,"5min":1.4690000000000001,"15min":2.806},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76500000000000001,"15min":0.81599999999999995},"min":{"1min":0.61099999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97699999999999998,"5min":1.1319999999999999,"15min":1.337},"last":0.57099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":1048576,"data_stored":470969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.29299999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.442,"15min":0.441},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.248},"max":{"1min":0.65000000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.377}]},{"osd":2,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.38800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69199999999999995,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.48999999999999999,"15min":0.503},"min":{"1min":0.40500000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.65700000000000003,"5min":0.79400000000000004,"15min":1.2030000000000001},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.41899999999999998,"5min":0.30099999999999999,"15min":0.29899999999999999},"max":{"1min":0.72699999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79000000000000004,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.442}]},{"osd":4,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.72499999999999998,"15min":0.749},"min":{"1min":0.51700000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0189999999999999,"5min":1.1699999999999999,"15min":1.171},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.56000000000000005,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.034},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.58399999999999996,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":1.0820000000000001,"5min":1.1080000000000001,"15min":1.1419999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73099999999999998,"15min":0.752},"min":{"1min":0.63400000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.99199999999999999,"5min":1.1040000000000001,"15min":1.1220000000000001},"last":0.56399999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.74099999999999999,"15min":0.752},"min":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0329999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74299999999999999,"15min":0.75800000000000001},"min":{"1min":0.58199999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0649999999999999,"15min":1.546},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.78300000000000003,"15min":0.80500000000000005},"min":{"1min":0.54600000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0529999999999999,"5min":1.0820000000000001,"15min":1.6830000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77100000000000002,"15min":1.0640000000000001},"min":{"1min":0.60599999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":9.5960000000000001},"last":0.58699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36172,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335668,"statfs":{"total":96636764160,"available":96599724032,"internally_reserved":0,"allocated":1052672,"data_stored":471503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.432,"15min":0.42699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.50900000000000001,"15min":0.48199999999999998},"min":{"1min":0.29599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.77800000000000002,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.093}]},{"osd":1,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.60899999999999999,"5min":0.72799999999999998,"15min":0.81799999999999995},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54600000000000004,"15min":0.51100000000000001},"min":{"1min":0.42499999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72799999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.5529999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.54400000000000004},"min":{"1min":0.44900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.752,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":2.1240000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59299999999999997,"15min":0.57299999999999995},"min":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80600000000000005,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":2.1629999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78600000000000003,"15min":0.72399999999999998},"min":{"1min":0.54600000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98199999999999998,"5min":2.863,"15min":2.863},"last":5.5380000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79800000000000004,"15min":0.72199999999999998},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.034,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.748,"15min":0.73299999999999998},"min":{"1min":0.50900000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93999999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":1.746},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80500000000000005,"15min":0.76800000000000002},"min":{"1min":0.55300000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.004,"5min":2.4990000000000001,"15min":2.4990000000000001},"last":6.8440000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78500000000000003,"15min":0.72999999999999998},"min":{"1min":0.57999999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":2.964,"15min":2.964},"last":11.696999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.79100000000000004,"15min":0.76200000000000001},"min":{"1min":0.63900000000000001,"5min":0.55000000000000004,"15min":0.50700000000000001},"max":{"1min":0.97899999999999998,"5min":2.085,"15min":2.085},"last":1.6040000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81100000000000005,"15min":0.82699999999999996},"min":{"1min":0.57999999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.2},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81799999999999995,"15min":0.80400000000000005},"min":{"1min":0.624,"5min":0.56000000000000005,"15min":0.47499999999999998},"max":{"1min":0.92000000000000004,"5min":2.702,"15min":2.702},"last":4.484}]}]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":1064960,"data_stored":472575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42599999999999999,"15min":0.59999999999999998},"min":{"1min":0.28499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":5.6550000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.497,"15min":0.51400000000000001},"min":{"1min":0.43099999999999999,"5min":0.33600000000000002,"15min":0.26700000000000002},"max":{"1min":0.68700000000000006,"5min":0.73799999999999999,"15min":1.0620000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44600000000000001,"15min":1.0109999999999999},"min":{"1min":0.32700000000000001,"5min":0.29299999999999998,"15min":0.29099999999999998},"max":{"1min":0.60399999999999998,"5min":0.745,"15min":15.911},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52200000000000002,"15min":1.083},"min":{"1min":0.34599999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69099999999999995,"5min":0.78800000000000003,"15min":16.024000000000001},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.501},"min":{"1min":0.34699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61699999999999999,"5min":0.70199999999999996,"15min":1.0309999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51500000000000001,"15min":1.0840000000000001},"min":{"1min":0.40699999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.73799999999999999,"5min":0.75700000000000001,"15min":16.068999999999999},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73299999999999998,"15min":0.76200000000000001},"min":{"1min":0.58099999999999996,"5min":0.44,"15min":0.44},"max":{"1min":0.94699999999999995,"5min":1.171,"15min":1.179},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.73699999999999999,"15min":0.92200000000000004},"min":{"1min":0.54300000000000004,"5min":0.46100000000000002,"15min":0.36699999999999999},"max":{"1min":0.95899999999999996,"5min":1.0940000000000001,"15min":5.7300000000000004},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.748,"15min":0.92600000000000005},"min":{"1min":0.628,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":1.1080000000000001,"15min":5.077},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77400000000000002,"15min":0.97899999999999998},"min":{"1min":0.64100000000000001,"5min":0.499,"15min":0.44800000000000001},"max":{"1min":0.998,"5min":1.196,"15min":5.6980000000000004},"last":0.82599999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.748,"15min":0.80200000000000005},"min":{"1min":0.627,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.028,"5min":1.0629999999999999,"15min":1.143},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.74399999999999999,"15min":0.93600000000000005},"min":{"1min":0.59799999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96799999999999997,"5min":1.127,"15min":5.157},"last":0.81100000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.751,"15min":0.83099999999999996},"min":{"1min":0.62,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93999999999999995,"5min":0.997,"15min":1.6679999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.94799999999999995},"min":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97399999999999998,"5min":1.1399999999999999,"15min":5.1189999999999998},"last":0.72199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36164,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335676,"statfs":{"total":96636764160,"available":96599732224,"internally_reserved":0,"allocated":1044480,"data_stored":470899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.54100000000000004,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88300000000000001,"5min":1.784,"15min":1.784},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.38},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.72799999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65600000000000003,"15min":0.85099999999999998},"min":{"1min":0.58199999999999996,"5min":0.38,"15min":0.38},"max":{"1min":0.81399999999999995,"5min":0.95399999999999996,"15min":6.258},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68200000000000005,"15min":0.66400000000000003},"min":{"1min":0.60199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79400000000000004,"5min":1.806,"15min":1.806},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67600000000000005,"15min":0.86799999999999999},"min":{"1min":0.436,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89100000000000001,"5min":0.998,"15min":6.4740000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73199999999999998,"15min":0.94799999999999995},"min":{"1min":0.67200000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.88100000000000001,"5min":0.95099999999999996,"15min":6.4859999999999998},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.58999999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80000000000000004,"5min":0.97799999999999998,"15min":1.25},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85999999999999999,"5min":0.99099999999999999,"15min":1.413},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59699999999999998,"15min":0.65600000000000003},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78800000000000003,"5min":0.92100000000000004,"15min":1.3280000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61299999999999999,"15min":0.66400000000000003},"min":{"1min":0.46400000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82699999999999996,"5min":0.94299999999999995,"15min":0.99299999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63200000000000001,"15min":0.66500000000000004},"min":{"1min":0.45500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85999999999999999,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73599999999999999,"15min":0.79000000000000004},"min":{"1min":0.65500000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.86699999999999999,"5min":1.008,"15min":1.6839999999999999},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67900000000000005,"15min":0.72799999999999998},"min":{"1min":0.52700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80000000000000004,"5min":0.95699999999999996,"15min":1.0660000000000001},"last":0.749},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69799999999999995,"15min":0.749},"min":{"1min":0.50800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83899999999999997,"5min":1.079,"15min":1.21},"last":0.85299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1596,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1634304,"data_stored":1060248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.66200000000000003},"min":{"1min":0.55600000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82099999999999995,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67100000000000004,"15min":0.85199999999999998},"min":{"1min":0.59499999999999997,"5min":0.38100000000000001,"15min":0.29999999999999999},"max":{"1min":0.82799999999999996,"5min":0.93000000000000005,"15min":6.4059999999999997},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.60999999999999999,"5min":0.39700000000000002,"15min":0.307},"max":{"1min":0.90000000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.54900000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81999999999999995,"5min":1.1970000000000001,"15min":1.546},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68899999999999995,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.38500000000000001,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":1.071,"15min":1.071},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.47099999999999997,"5min":0.439,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.93000000000000005,"15min":1.0029999999999999},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.55200000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92900000000000005,"5min":1.0009999999999999,"15min":1.03},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73499999999999999,"15min":0.752},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.39600000000000002},"max":{"1min":0.86599999999999999,"5min":4.484,"15min":4.484},"last":1.3140000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.34699999999999998},"max":{"1min":0.77400000000000002,"5min":1.1359999999999999,"15min":1.6240000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65000000000000002,"15min":0.69599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79300000000000004,"5min":1.0569999999999999,"15min":2.718},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.49099999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68799999999999994,"15min":0.71899999999999997},"min":{"1min":0.51100000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88200000000000001,"5min":1.246,"15min":1.669},"last":0.753}]},{"osd":7,"last update":"Thu Jul 6 03:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72899999999999998,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89800000000000002,"5min":1.3360000000000001,"15min":2.0459999999999998},"last":0.622},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.749,"15min":0.78200000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.47699999999999998},"max":{"1min":0.94499999999999995,"5min":1.0609999999999999,"15min":2.2919999999999998},"last":0.58199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:52:39.826 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T03:52:39.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T03:52:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.058+0000 7fb5591ff700 1 Processor -- start 2023-07-06T03:52:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.059+0000 7fb5591ff700 1 -- start start 2023-07-06T03:52:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.060+0000 7fb5591ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb55415a100 0x7fb5540cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.060+0000 7fb5591ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb5540d0c10 0x7fb554160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.060+0000 7fb5591ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5540cd400 con 0x7fb5540b4790 2023-07-06T03:52:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.060+0000 7fb5591ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5540cda80 con 0x7fb5540d0c10 2023-07-06T03:52:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.060+0000 7fb5591ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb554161150 con 0x7fb55415a100 2023-07-06T03:52:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.060+0000 7fb552d9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb55415a100 0x7fb5540cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.060+0000 7fb55259c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb5540d0c10 0x7fb554160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.060+0000 7fb55259c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb5540d0c10 0x7fb554160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:51554/0 (socket says 172.21.15.97:51554) 2023-07-06T03:52:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.060+0000 7fb55259c700 1 -- 172.21.15.97:0/3033750874 learned_addr learned my addr 172.21.15.97:0/3033750874 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.060+0000 7fb552d9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb55415a100 0x7fb5540cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:43818/0 (socket says 172.21.15.97:43818) 2023-07-06T03:52:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.061+0000 7fb552d9d700 1 -- 172.21.15.97:0/3033750874 >> v2:172.21.15.146:3300/0 conn(0x7fb5540d0c10 msgr2=0x7fb554160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.061+0000 7fb552d9d700 1 --2- 172.21.15.97:0/3033750874 >> v2:172.21.15.146:3300/0 conn(0x7fb5540d0c10 0x7fb554160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.061+0000 7fb552d9d700 1 -- 172.21.15.97:0/3033750874 >> v1:172.21.15.146:6789/0 conn(0x7fb5540b4790 legacy=0x7fb55415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.061+0000 7fb552d9d700 1 -- 172.21.15.97:0/3033750874 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540d1060 con 0x7fb55415a100 2023-07-06T03:52:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.061+0000 7fb55259c700 1 --2- 172.21.15.97:0/3033750874 >> v2:172.21.15.146:3300/0 conn(0x7fb5540d0c10 0x7fb554160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.061+0000 7fb551d9b700 1 -- 172.21.15.97:0/3033750874 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193393531 0 0) 0x7fb5540cd400 con 0x7fb5540b4790 2023-07-06T03:52:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.061+0000 7fb552d9d700 1 --2- 172.21.15.97:0/3033750874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb55415a100 0x7fb5540cefd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb544009840 tx=0x7fb544009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23e14cdf2976fd90 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.061+0000 7fb551d9b700 1 -- 172.21.15.97:0/3033750874 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544011830 con 0x7fb55415a100 2023-07-06T03:52:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.062+0000 7fb551d9b700 1 -- 172.21.15.97:0/3033750874 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544011990 con 0x7fb55415a100 2023-07-06T03:52:40.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.062+0000 7fb551d9b700 1 -- 172.21.15.97:0/3033750874 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544011d20 con 0x7fb55415a100 2023-07-06T03:52:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.062+0000 7fb5591ff700 1 -- 172.21.15.97:0/3033750874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb55415a100 msgr2=0x7fb5540cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.062+0000 7fb5591ff700 1 --2- 172.21.15.97:0/3033750874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb55415a100 0x7fb5540cefd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb544009840 tx=0x7fb544009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.062+0000 7fb5591ff700 1 -- 172.21.15.97:0/3033750874 shutdown_connections 2023-07-06T03:52:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.062+0000 7fb5591ff700 1 --2- 172.21.15.97:0/3033750874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb55415a100 0x7fb5540cefd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.062+0000 7fb5591ff700 1 --2- 172.21.15.97:0/3033750874 >> v2:172.21.15.146:3300/0 conn(0x7fb5540d0c10 0x7fb554160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.062+0000 7fb5591ff700 1 -- 172.21.15.97:0/3033750874 >> 172.21.15.97:0/3033750874 conn(0x7fb55406b2d0 msgr2=0x7fb5540cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.062+0000 7fb5591ff700 1 -- 172.21.15.97:0/3033750874 shutdown_connections 2023-07-06T03:52:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.062+0000 7fb5591ff700 1 -- 172.21.15.97:0/3033750874 wait complete. 2023-07-06T03:52:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.064+0000 7fb5591ff700 1 Processor -- start 2023-07-06T03:52:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.064+0000 7fb5591ff700 1 -- start start 2023-07-06T03:52:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb5591ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5540b4790 0x7fb5540c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb5591ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5540d0c10 0x7fb55414dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb5591ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb55415a100 0x7fb5540b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb5591ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540c6d80 con 0x7fb55415a100 2023-07-06T03:52:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb5591ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5541f66a0 con 0x7fb5540b4790 2023-07-06T03:52:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb5591ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5541f5b20 con 0x7fb5540d0c10 2023-07-06T03:52:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb55359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb55415a100 0x7fb5540b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb55359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb55415a100 0x7fb5540b82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58984/0 (socket says 172.21.15.97:58984) 2023-07-06T03:52:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb552d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5540b4790 0x7fb5540c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb55359e700 1 -- 172.21.15.97:0/112056624 learned_addr learned my addr 172.21.15.97:0/112056624 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb55259c700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5540d0c10 0x7fb55414dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.065+0000 7fb55359e700 1 -- 172.21.15.97:0/112056624 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5540d0c10 msgr2=0x7fb55414dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.066+0000 7fb55359e700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5540d0c10 0x7fb55414dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.066+0000 7fb55359e700 1 -- 172.21.15.97:0/112056624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5540b4790 msgr2=0x7fb5540c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.066+0000 7fb55359e700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5540b4790 0x7fb5540c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.066+0000 7fb55359e700 1 -- 172.21.15.97:0/112056624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb548009550 con 0x7fb55415a100 2023-07-06T03:52:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.066+0000 7fb55359e700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb55415a100 0x7fb5540b82a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00c7a0 tx=0x7fb54c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.066+0000 7fb53b7fe700 1 -- 172.21.15.97:0/112056624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c003e30 con 0x7fb55415a100 2023-07-06T03:52:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.066+0000 7fb53b7fe700 1 -- 172.21.15.97:0/112056624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c005070 con 0x7fb55415a100 2023-07-06T03:52:40.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.066+0000 7fb5591ff700 1 -- 172.21.15.97:0/112056624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5440094a0 con 0x7fb55415a100 2023-07-06T03:52:40.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.066+0000 7fb53b7fe700 1 -- 172.21.15.97:0/112056624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c005420 con 0x7fb55415a100 2023-07-06T03:52:40.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.066+0000 7fb5591ff700 1 -- 172.21.15.97:0/112056624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541f60e0 con 0x7fb55415a100 2023-07-06T03:52:40.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.067+0000 7fb5591ff700 1 -- 172.21.15.97:0/112056624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb554067920 con 0x7fb55415a100 2023-07-06T03:52:40.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.068+0000 7fb53b7fe700 1 -- 172.21.15.97:0/112056624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb54c005580 con 0x7fb55415a100 2023-07-06T03:52:40.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.069+0000 7fb53b7fe700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb54006ace0 0x7fb54006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.069+0000 7fb53b7fe700 1 -- 172.21.15.97:0/112056624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb54c0889a0 con 0x7fb55415a100 2023-07-06T03:52:40.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.069+0000 7fb552d9d700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb54006ace0 0x7fb54006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.070+0000 7fb552d9d700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb54006ace0 0x7fb54006d190 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb544009fd0 tx=0x7fb54401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:40.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.073+0000 7fb53b7fe700 1 -- 172.21.15.97:0/112056624 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb54c057570 con 0x7fb55415a100 2023-07-06T03:52:40.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.344+0000 7fb5591ff700 1 -- 172.21.15.97:0/112056624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb55404e340 con 0x7fb55415a100 2023-07-06T03:52:40.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.349+0000 7fb53b7fe700 1 -- 172.21.15.97:0/112056624 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb54c017020 con 0x7fb55415a100 2023-07-06T03:52:40.351 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:52:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg== 2023-07-06T03:52:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-07-06T03:52:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-07-06T03:52:40.351 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T03:52:40.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.351+0000 7fb5591ff700 1 -- 172.21.15.97:0/112056624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb54006ace0 msgr2=0x7fb54006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.351+0000 7fb5591ff700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb54006ace0 0x7fb54006d190 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb544009fd0 tx=0x7fb54401c040 comp rx=0 tx=0).stop 2023-07-06T03:52:40.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.351+0000 7fb5591ff700 1 -- 172.21.15.97:0/112056624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb55415a100 msgr2=0x7fb5540b82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.351+0000 7fb5591ff700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb55415a100 0x7fb5540b82a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00c7a0 tx=0x7fb54c00cab0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.351+0000 7fb5591ff700 1 -- 172.21.15.97:0/112056624 shutdown_connections 2023-07-06T03:52:40.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.351+0000 7fb5591ff700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb54006ace0 0x7fb54006d190 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.351+0000 7fb5591ff700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb55415a100 0x7fb5540b82a0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.351+0000 7fb5591ff700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5540b4790 0x7fb5540c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.351+0000 7fb5591ff700 1 --2- 172.21.15.97:0/112056624 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5540d0c10 0x7fb55414dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.351+0000 7fb5591ff700 1 -- 172.21.15.97:0/112056624 >> 172.21.15.97:0/112056624 conn(0x7fb55406b2d0 msgr2=0x7fb5540b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.351+0000 7fb5591ff700 1 -- 172.21.15.97:0/112056624 shutdown_connections 2023-07-06T03:52:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.352+0000 7fb5591ff700 1 -- 172.21.15.97:0/112056624 wait complete. 2023-07-06T03:52:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-07-06T03:52:40.373 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:52:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.566+0000 7fef563ff700 1 Processor -- start 2023-07-06T03:52:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.566+0000 7fef563ff700 1 -- start start 2023-07-06T03:52:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.567+0000 7fef563ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef500cd390 0x7fef50160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.567+0000 7fef563ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fef50160dd0 0x7fef500b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.567+0000 7fef563ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef500cdae0 con 0x7fef5015c520 2023-07-06T03:52:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.567+0000 7fef563ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef500b6640 con 0x7fef50160dd0 2023-07-06T03:52:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.567+0000 7fef563ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef500b6920 con 0x7fef500cd390 2023-07-06T03:52:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.568+0000 7fef5499c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fef50160dd0 0x7fef500b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.568+0000 7fef4f7fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef500cd390 0x7fef50160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.568+0000 7fef5499c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fef50160dd0 0x7fef500b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:51578/0 (socket says 172.21.15.97:51578) 2023-07-06T03:52:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.568+0000 7fef5499c700 1 -- 172.21.15.97:0/3309702543 learned_addr learned my addr 172.21.15.97:0/3309702543 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.569+0000 7fef5499c700 1 -- 172.21.15.97:0/3309702543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef500cd390 msgr2=0x7fef50160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.569+0000 7fef5499c700 1 --2- 172.21.15.97:0/3309702543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef500cd390 0x7fef50160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.569+0000 7fef5499c700 1 -- 172.21.15.97:0/3309702543 >> v1:172.21.15.146:6789/0 conn(0x7fef5015c520 legacy=0x7fef50160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.569+0000 7fef5499c700 1 -- 172.21.15.97:0/3309702543 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef500b6c00 con 0x7fef50160dd0 2023-07-06T03:52:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.569+0000 7fef4f7fe700 1 --2- 172.21.15.97:0/3309702543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef500cd390 0x7fef50160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:52:40.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.569+0000 7fef4effd700 1 -- 172.21.15.97:0/3309702543 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306707315 0 0) 0x7fef500cdae0 con 0x7fef5015c520 2023-07-06T03:52:40.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.569+0000 7fef5499c700 1 --2- 172.21.15.97:0/3309702543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef50160dd0 0x7fef500b7400 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef48009e50 tx=0x7fef4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6d1a6106097df1c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:40.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.569+0000 7fef4effd700 1 -- 172.21.15.97:0/3309702543 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4800e070 con 0x7fef50160dd0 2023-07-06T03:52:40.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.569+0000 7fef4effd700 1 -- 172.21.15.97:0/3309702543 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4800b910 con 0x7fef50160dd0 2023-07-06T03:52:40.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.569+0000 7fef4effd700 1 -- 172.21.15.97:0/3309702543 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4800bcc0 con 0x7fef50160dd0 2023-07-06T03:52:40.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.570+0000 7fef563ff700 1 -- 172.21.15.97:0/3309702543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef50160dd0 msgr2=0x7fef500b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.570+0000 7fef563ff700 1 --2- 172.21.15.97:0/3309702543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef50160dd0 0x7fef500b7400 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef48009e50 tx=0x7fef4800b330 comp rx=0 tx=0).stop 2023-07-06T03:52:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.570+0000 7fef563ff700 1 -- 172.21.15.97:0/3309702543 shutdown_connections 2023-07-06T03:52:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.570+0000 7fef563ff700 1 --2- 172.21.15.97:0/3309702543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef50160dd0 0x7fef500b7400 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.570+0000 7fef563ff700 1 --2- 172.21.15.97:0/3309702543 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef500cd390 0x7fef50160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.570+0000 7fef563ff700 1 -- 172.21.15.97:0/3309702543 >> 172.21.15.97:0/3309702543 conn(0x7fef50071790 msgr2=0x7fef50071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:40.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.570+0000 7fef563ff700 1 -- 172.21.15.97:0/3309702543 shutdown_connections 2023-07-06T03:52:40.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.570+0000 7fef563ff700 1 -- 172.21.15.97:0/3309702543 wait complete. 2023-07-06T03:52:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.571+0000 7fef563ff700 1 Processor -- start 2023-07-06T03:52:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.572+0000 7fef563ff700 1 -- start start 2023-07-06T03:52:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef563ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef500cd390 0x7fef500d34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef563ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef5015c520 0x7fef501e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef563ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef50160dd0 0x7fef5015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef563ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef501e7980 con 0x7fef500cd390 2023-07-06T03:52:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef563ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef501e7c60 con 0x7fef50160dd0 2023-07-06T03:52:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef563ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef501e7f40 con 0x7fef5015c520 2023-07-06T03:52:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef500cd390 0x7fef500d34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef4ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef500cd390 0x7fef500d34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59000/0 (socket says 172.21.15.97:59000) 2023-07-06T03:52:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef4ffff700 1 -- 172.21.15.97:0/1633742322 learned_addr learned my addr 172.21.15.97:0/1633742322 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef5499c700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef50160dd0 0x7fef5015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef4f7fe700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef5015c520 0x7fef501e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef4ffff700 1 -- 172.21.15.97:0/1633742322 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef5015c520 msgr2=0x7fef501e6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef4ffff700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef5015c520 0x7fef501e6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef4ffff700 1 -- 172.21.15.97:0/1633742322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef50160dd0 msgr2=0x7fef5015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef4ffff700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef50160dd0 0x7fef5015ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef4ffff700 1 -- 172.21.15.97:0/1633742322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef440094a0 con 0x7fef500cd390 2023-07-06T03:52:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.573+0000 7fef5499c700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef50160dd0 0x7fef5015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.574+0000 7fef4ffff700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef500cd390 0x7fef500d34e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c550 tx=0x7fef4000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.574+0000 7fef4cff9700 1 -- 172.21.15.97:0/1633742322 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef400054e0 con 0x7fef500cd390 2023-07-06T03:52:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.574+0000 7fef4cff9700 1 -- 172.21.15.97:0/1633742322 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef40003680 con 0x7fef500cd390 2023-07-06T03:52:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.574+0000 7fef563ff700 1 -- 172.21.15.97:0/1633742322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef48009b30 con 0x7fef500cd390 2023-07-06T03:52:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.574+0000 7fef4cff9700 1 -- 172.21.15.97:0/1633742322 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40003a30 con 0x7fef500cd390 2023-07-06T03:52:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.574+0000 7fef563ff700 1 -- 172.21.15.97:0/1633742322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef501e8500 con 0x7fef500cd390 2023-07-06T03:52:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.575+0000 7fef563ff700 1 -- 172.21.15.97:0/1633742322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef501e8720 con 0x7fef500cd390 2023-07-06T03:52:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.576+0000 7fef4cff9700 1 -- 172.21.15.97:0/1633742322 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef40003b90 con 0x7fef500cd390 2023-07-06T03:52:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.577+0000 7fef4cff9700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fef3c06ace0 0x7fef3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.577+0000 7fef4cff9700 1 -- 172.21.15.97:0/1633742322 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef400889c0 con 0x7fef500cd390 2023-07-06T03:52:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.577+0000 7fef4f7fe700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fef3c06ace0 0x7fef3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.578+0000 7fef4f7fe700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fef3c06ace0 0x7fef3c06d190 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fef500c9d90 tx=0x7fef44009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.581+0000 7fef4cff9700 1 -- 172.21.15.97:0/1633742322 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fef40053940 con 0x7fef500cd390 2023-07-06T03:52:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.822+0000 7fef563ff700 1 -- 172.21.15.97:0/1633742322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef500ce9b0 con 0x7fef500cd390 2023-07-06T03:52:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.822+0000 7fef4cff9700 1 -- 172.21.15.97:0/1633742322 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4679 (secure 0 0 0) 0x7fef40017030 con 0x7fef500cd390 2023-07-06T03:52:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:40.824 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3945452207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3945452207},{"type":"v1","addr":"172.21.15.146:6835","nonce":3945452207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2428591387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2428591387},{"type":"v1","addr":"172.21.15.97:6837","nonce":2428591387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/2089126027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2089126027},{"type":"v1","addr":"172.21.15.146:6834","nonce":2089126027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:52:34.447330+0000","modified":"2023-07-06T03:52:37.985332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7497":{"gid":7497,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/2046478602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2046478602},{"type":"v1","addr":"172.21.15.97:6836","nonce":2046478602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-06T03:52:40.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.824+0000 7fef563ff700 1 -- 172.21.15.97:0/1633742322 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fef3c06ace0 msgr2=0x7fef3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.824+0000 7fef563ff700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fef3c06ace0 0x7fef3c06d190 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fef500c9d90 tx=0x7fef44009210 comp rx=0 tx=0).stop 2023-07-06T03:52:40.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.824+0000 7fef563ff700 1 -- 172.21.15.97:0/1633742322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef500cd390 msgr2=0x7fef500d34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:40.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.824+0000 7fef563ff700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef500cd390 0x7fef500d34e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c550 tx=0x7fef4000c860 comp rx=0 tx=0).stop 2023-07-06T03:52:40.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.825+0000 7fef563ff700 1 -- 172.21.15.97:0/1633742322 shutdown_connections 2023-07-06T03:52:40.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.825+0000 7fef563ff700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fef3c06ace0 0x7fef3c06d190 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.825+0000 7fef563ff700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef50160dd0 0x7fef5015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.825+0000 7fef563ff700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef500cd390 0x7fef500d34e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.825+0000 7fef563ff700 1 --2- 172.21.15.97:0/1633742322 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef5015c520 0x7fef501e6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:40.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.825+0000 7fef563ff700 1 -- 172.21.15.97:0/1633742322 >> 172.21.15.97:0/1633742322 conn(0x7fef50071790 msgr2=0x7fef500cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:40.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.825+0000 7fef563ff700 1 -- 172.21.15.97:0/1633742322 shutdown_connections 2023-07-06T03:52:40.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:40.825+0000 7fef563ff700 1 -- 172.21.15.97:0/1633742322 wait complete. 2023-07-06T03:52:40.829 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 56 2023-07-06T03:52:41.246 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T03:52:34.447330+0000', 'modified': '2023-07-06T03:52:37.985332+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7497}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7497': {'gid': 7497, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6836/2046478602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 2046478602}, {'type': 'v1', 'addr': '172.21.15.97:6836', 'nonce': 2046478602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T03:52:41.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T03:52:41.247 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-06T03:52:41.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.442+0000 7f2fff5ff700 1 Processor -- start 2023-07-06T03:52:41.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.442+0000 7f2fff5ff700 1 -- start start 2023-07-06T03:52:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.443+0000 7f2fff5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff80d2210 0x7f2ff80b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.443+0000 7f2fff5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff80cff90 con 0x7f2ff806b360 2023-07-06T03:52:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.443+0000 7f2fff5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff80b5650 con 0x7f2ff815c610 2023-07-06T03:52:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.443+0000 7f2fff5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80d0780 con 0x7f2ff80d2210 2023-07-06T03:52:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.443+0000 7f2ffcb9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2ff806b360 0x7f2ff80d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48714/0 (socket says 172.21.15.97:48714) 2023-07-06T03:52:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.443+0000 7f2ffcb9a700 1 -- 172.21.15.97:0/2265381327 learned_addr learned my addr 172.21.15.97:0/2265381327 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.444+0000 7f2ffdb9c700 1 --2- 172.21.15.97:0/2265381327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff80d2210 0x7f2ff80b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.444+0000 7f2feffff700 1 -- 172.21.15.97:0/2265381327 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756600878 0 0) 0x7f2ff80cff90 con 0x7f2ff806b360 2023-07-06T03:52:41.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.444+0000 7f2feffff700 1 -- 172.21.15.97:0/2265381327 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe40035b0 con 0x7f2ff806b360 2023-07-06T03:52:41.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.444+0000 7f2feffff700 1 -- 172.21.15.97:0/2265381327 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847239597 0 0) 0x7f2ff80b5650 con 0x7f2ff815c610 2023-07-06T03:52:41.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.444+0000 7f2feffff700 1 -- 172.21.15.97:0/2265381327 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff80cff90 con 0x7f2ff815c610 2023-07-06T03:52:41.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.444+0000 7f2ffdb9c700 1 -- 172.21.15.97:0/2265381327 >> v1:172.21.15.146:6789/0 conn(0x7f2ff815c610 legacy=0x7f2ff80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.444+0000 7f2ffdb9c700 1 -- 172.21.15.97:0/2265381327 >> v1:172.21.15.97:6789/0 conn(0x7f2ff806b360 legacy=0x7f2ff80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.444+0000 7f2ffdb9c700 1 -- 172.21.15.97:0/2265381327 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff80b5400 con 0x7f2ff80d2210 2023-07-06T03:52:41.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.444+0000 7f2ffdb9c700 1 --2- 172.21.15.97:0/2265381327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff80d2210 0x7f2ff80b4ec0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009dc0 tx=0x7f2ff000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3fc6452ce4e90de server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.445+0000 7f2feffff700 1 -- 172.21.15.97:0/2265381327 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff000e070 con 0x7f2ff80d2210 2023-07-06T03:52:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.445+0000 7f2feffff700 1 -- 172.21.15.97:0/2265381327 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff000bb20 con 0x7f2ff80d2210 2023-07-06T03:52:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.445+0000 7f2feffff700 1 -- 172.21.15.97:0/2265381327 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0012430 con 0x7f2ff80d2210 2023-07-06T03:52:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.445+0000 7f2fff5ff700 1 -- 172.21.15.97:0/2265381327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff80d2210 msgr2=0x7f2ff80b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.445+0000 7f2fff5ff700 1 --2- 172.21.15.97:0/2265381327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff80d2210 0x7f2ff80b4ec0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009dc0 tx=0x7f2ff000b330 comp rx=0 tx=0).stop 2023-07-06T03:52:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.445+0000 7f2fff5ff700 1 -- 172.21.15.97:0/2265381327 shutdown_connections 2023-07-06T03:52:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.445+0000 7f2fff5ff700 1 --2- 172.21.15.97:0/2265381327 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff80d2210 0x7f2ff80b4ec0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.445+0000 7f2fff5ff700 1 -- 172.21.15.97:0/2265381327 >> 172.21.15.97:0/2265381327 conn(0x7f2ff806a6c0 msgr2=0x7f2ff806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:41.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.445+0000 7f2fff5ff700 1 -- 172.21.15.97:0/2265381327 shutdown_connections 2023-07-06T03:52:41.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.445+0000 7f2fff5ff700 1 -- 172.21.15.97:0/2265381327 wait complete. 2023-07-06T03:52:41.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.447+0000 7f2fff5ff700 1 Processor -- start 2023-07-06T03:52:41.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.447+0000 7f2fff5ff700 1 -- start start 2023-07-06T03:52:41.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.448+0000 7f2fff5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff806b360 0x7f2ff80c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.448+0000 7f2fff5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80d2210 0x7f2ff814b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.448+0000 7f2fff5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff815c610 0x7f2ff815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.448+0000 7f2fff5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80c7360 con 0x7f2ff80d2210 2023-07-06T03:52:41.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.448+0000 7f2fff5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80c9e00 con 0x7f2ff806b360 2023-07-06T03:52:41.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.448+0000 7f2fff5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80ca050 con 0x7f2ff815c610 2023-07-06T03:52:41.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.448+0000 7f2ffcb9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80d2210 0x7f2ff814b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.448+0000 7f2ffcb9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80d2210 0x7f2ff814b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59014/0 (socket says 172.21.15.97:59014) 2023-07-06T03:52:41.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.448+0000 7f2ffcb9a700 1 -- 172.21.15.97:0/1989415339 learned_addr learned my addr 172.21.15.97:0/1989415339 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:41.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2ffdb9c700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff815c610 0x7f2ff815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff806b360 0x7f2ff80c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2ffcb9a700 1 -- 172.21.15.97:0/1989415339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff815c610 msgr2=0x7f2ff815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2ffcb9a700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff815c610 0x7f2ff815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2ffcb9a700 1 -- 172.21.15.97:0/1989415339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff806b360 msgr2=0x7f2ff80c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2ffcb9a700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff806b360 0x7f2ff80c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2ffcb9a700 1 -- 172.21.15.97:0/1989415339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0009a70 con 0x7f2ff80d2210 2023-07-06T03:52:41.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2ffcb9a700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80d2210 0x7f2ff814b0c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8001eb0 tx=0x7f2fe800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:41.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2fedffb700 1 -- 172.21.15.97:0/1989415339 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8005d70 con 0x7f2ff80d2210 2023-07-06T03:52:41.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2fedffb700 1 -- 172.21.15.97:0/1989415339 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe80070d0 con 0x7f2ff80d2210 2023-07-06T03:52:41.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2fff5ff700 1 -- 172.21.15.97:0/1989415339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff80ca330 con 0x7f2ff80d2210 2023-07-06T03:52:41.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2fedffb700 1 -- 172.21.15.97:0/1989415339 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8007460 con 0x7f2ff80d2210 2023-07-06T03:52:41.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.449+0000 7f2fff5ff700 1 -- 172.21.15.97:0/1989415339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff80ca8f0 con 0x7f2ff80d2210 2023-07-06T03:52:41.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.452+0000 7f2fedffb700 1 -- 172.21.15.97:0/1989415339 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2fe80075c0 con 0x7f2ff80d2210 2023-07-06T03:52:41.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.453+0000 7f2fedffb700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fdc06ace0 0x7f2fdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.453+0000 7f2fedffb700 1 -- 172.21.15.97:0/1989415339 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2fe8088850 con 0x7f2ff80d2210 2023-07-06T03:52:41.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.454+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fdc06ace0 0x7f2fdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.454+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fdc06ace0 0x7f2fdc06d190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40011e0 tx=0x7f2ff4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:41.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.455+0000 7f3012f9eb80 1 Processor -- start 2023-07-06T03:52:41.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.456+0000 7f3012f9eb80 1 -- start start 2023-07-06T03:52:41.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9d1190 0x557f1f9efa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9a6b50 0x557f1f9a9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1f91dea0 con 0x557f1f9cbd50 2023-07-06T03:52:41.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1f891600 con 0x557f1f9d1190 2023-07-06T03:52:41.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f1f900910 con 0x557f1f9a6b50 2023-07-06T03:52:41.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f2ffd39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f2ffd39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59024/0 (socket says 172.21.15.97:59024) 2023-07-06T03:52:41.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f2ffd39b700 1 -- 172.21.15.97:0/3169176270 learned_addr learned my addr 172.21.15.97:0/3169176270 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:41.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f2ffcb9a700 1 --2- 172.21.15.97:0/3169176270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9d1190 0x557f1f9efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f2ffdb9c700 1 --2- 172.21.15.97:0/3169176270 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9a6b50 0x557f1f9a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f2ffd39b700 1 -- 172.21.15.97:0/3169176270 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9a6b50 msgr2=0x557f1f9a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:41.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/3169176270 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9a6b50 0x557f1f9a9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f2ffd39b700 1 -- 172.21.15.97:0/3169176270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9d1190 msgr2=0x557f1f9efa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:41.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/3169176270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9d1190 0x557f1f9efa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f2ffd39b700 1 -- 172.21.15.97:0/3169176270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f1f6ed790 con 0x557f1f9cbd50 2023-07-06T03:52:41.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.457+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/3169176270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9cc120 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400f7a0 tx=0x7f2ff4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae7305b549599db server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:41.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.458+0000 7f2fe2ffd700 1 -- 172.21.15.97:0/3169176270 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4014450 con 0x557f1f9cbd50 2023-07-06T03:52:41.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.458+0000 7f2fe2ffd700 1 -- 172.21.15.97:0/3169176270 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff40145b0 con 0x557f1f9cbd50 2023-07-06T03:52:41.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.458+0000 7f2fe2ffd700 1 -- 172.21.15.97:0/3169176270 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4014920 con 0x557f1f9cbd50 2023-07-06T03:52:41.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.458+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3169176270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 msgr2=0x557f1f9cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.458+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3169176270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9cc120 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400f7a0 tx=0x7f2ff4012000 comp rx=0 tx=0).stop 2023-07-06T03:52:41.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.458+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3169176270 shutdown_connections 2023-07-06T03:52:41.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.458+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3169176270 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9a6b50 0x557f1f9a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.458+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3169176270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9cc120 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.458+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3169176270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9d1190 0x557f1f9efa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.458+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3169176270 >> 172.21.15.97:0/3169176270 conn(0x557f1f99e310 msgr2=0x557f1f9ee5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:41.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.458+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3169176270 shutdown_connections 2023-07-06T03:52:41.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.459+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3169176270 wait complete. 2023-07-06T03:52:41.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.459+0000 7f3012f9eb80 1 -- start start 2023-07-06T03:52:41.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.459+0000 7f3012f9eb80 1 Processor -- start 2023-07-06T03:52:41.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.460+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9a6b50 0x557f1f9d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.460+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9cbd50 0x557f1f9d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.460+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9d1190 0x557f1f9f1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.460+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1f8f9be0 con 0x557f1f9d1190 2023-07-06T03:52:41.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.460+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1f8b75b0 con 0x557f1f9a6b50 2023-07-06T03:52:41.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.460+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f1f89d910 con 0x557f1f9cbd50 2023-07-06T03:52:41.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.460+0000 7f3012f9eb80 10 client.0 _ll_register_callbacks cb 0x557f1f8b5b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:52:41.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2ffdb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9d1190 0x557f1f9f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2ffdb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9d1190 0x557f1f9f1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59036/0 (socket says 172.21.15.97:59036) 2023-07-06T03:52:41.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2ffdb9c700 1 -- 172.21.15.97:0/2833363416 learned_addr learned my addr 172.21.15.97:0/2833363416 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:41.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2ffcb9a700 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9cbd50 0x557f1f9d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9a6b50 0x557f1f9d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2ffdb9c700 1 -- 172.21.15.97:0/2833363416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9cbd50 msgr2=0x557f1f9d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:41.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2ffdb9c700 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9cbd50 0x557f1f9d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2ffdb9c700 1 -- 172.21.15.97:0/2833363416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9a6b50 msgr2=0x557f1f9d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:41.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2ffdb9c700 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9a6b50 0x557f1f9d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2ffdb9c700 1 -- 172.21.15.97:0/2833363416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff80cff90 con 0x557f1f9d1190 2023-07-06T03:52:41.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2ffdb9c700 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9d1190 0x557f1f9f1810 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2ff808b120 tx=0x7f2ff000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaac05575c7a5cf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:41.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.461+0000 7f2fd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:52:41.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f2fd7fff700 1 -- 172.21.15.97:0/2833363416 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff002a070 con 0x557f1f9d1190 2023-07-06T03:52:41.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f2fd7fff700 1 -- 172.21.15.97:0/2833363416 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2ff0021070 con 0x557f1f9d1190 2023-07-06T03:52:41.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f3012f9eb80 1 -- 172.21.15.97:0/2833363416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f1f99b920 con 0x557f1f9d1190 2023-07-06T03:52:41.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f2fd7fff700 1 -- 172.21.15.97:0/2833363416 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ff00098f0 con 0x557f1f9d1190 2023-07-06T03:52:41.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f3012f9eb80 10 client.7632 fetch_fsmap learned FSMap version 56 2023-07-06T03:52:41.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f3012f9eb80 1 -- 172.21.15.97:0/2833363416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x557f1f7da0a0 con 0x557f1f9d1190 2023-07-06T03:52:41.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f2fd7fff700 1 -- 172.21.15.97:0/2833363416 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557f1f99b920 con 0x557f1f9d1190 2023-07-06T03:52:41.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f2fd7fff700 1 -- 172.21.15.97:0/2833363416 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 56) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2ff002d020 con 0x557f1f9d1190 2023-07-06T03:52:41.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f3012f9eb80 10 client.7632 fetch_fsmap finished waiting for FSMap version 56 2023-07-06T03:52:41.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f3012f9eb80 10 client.7632 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-06T03:52:41.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f3012f9eb80 20 client.7632 populate_metadata read hostname 'smithi097' 2023-07-06T03:52:41.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.97:6834/2046478602,v1:172.21.15.97:6836/2046478602] conn(0x557f1f9f1dd0 0x557f1f9f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f3012f9eb80 4 client.7632 mds_command: new command op to 7497 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T03:52:41.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.462+0000 7f3012f9eb80 1 -- 172.21.15.97:0/2833363416 --> [v2:172.21.15.97:6834/2046478602,v1:172.21.15.97:6836/2046478602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f1f9780d0 con 0x557f1f9f1dd0 2023-07-06T03:52:41.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.463+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.97:6834/2046478602,v1:172.21.15.97:6836/2046478602] conn(0x557f1f9f1dd0 0x557f1f9f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.463+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.97:6834/2046478602,v1:172.21.15.97:6836/2046478602] conn(0x557f1f9f1dd0 0x557f1f9f21a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f8b2764b39b102 server_cookie=58648f850284701 in_seq=0 out_seq=0 2023-07-06T03:52:41.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.463+0000 7f2fd7fff700 10 client.7632 ms_handle_connect on v2:172.21.15.97:6834/2046478602 2023-07-06T03:52:41.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f2fd7fff700 1 -- 172.21.15.97:0/2833363416 <== mds.0 v2:172.21.15.97:6834/2046478602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f1f9780d0 con 0x557f1f9f1dd0 2023-07-06T03:52:41.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f2fd7fff700 10 client.7632 handle_command_reply: tid=0 2023-07-06T03:52:41.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 1 client.7632 shutdown 2023-07-06T03:52:41.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 1 -- 172.21.15.97:0/2833363416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9d1190 msgr2=0x557f1f9f1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9d1190 0x557f1f9f1810 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2ff808b120 tx=0x7f2ff000e000 comp rx=0 tx=0).stop 2023-07-06T03:52:41.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 1 -- 172.21.15.97:0/2833363416 shutdown_connections 2023-07-06T03:52:41.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.97:6834/2046478602,v1:172.21.15.97:6836/2046478602] conn(0x557f1f9f1dd0 0x557f1f9f21a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9d1190 0x557f1f9f1810 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9a6b50 0x557f1f9d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/2833363416 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9cbd50 0x557f1f9d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f2fd7fff700 0 client.7632 ms_handle_reset on v2:172.21.15.97:6834/2046478602 2023-07-06T03:52:41.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 1 -- 172.21.15.97:0/2833363416 >> 172.21.15.97:0/2833363416 conn(0x557f1f99e310 msgr2=0x557f1f9d2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:41.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 1 -- 172.21.15.97:0/2833363416 shutdown_connections 2023-07-06T03:52:41.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 1 -- 172.21.15.97:0/2833363416 wait complete. 2023-07-06T03:52:41.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.464+0000 7f3012f9eb80 20 client.7632 trim_cache size 0 max 16384 2023-07-06T03:52:41.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.482+0000 7f3012f9eb80 1 Processor -- start 2023-07-06T03:52:41.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.482+0000 7f3012f9eb80 1 -- start start 2023-07-06T03:52:41.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.483+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9a6b50 0x557f1f9d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.483+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.483+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9d1190 0x557f1f9c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.483+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1f6ee390 con 0x557f1f9cbd50 2023-07-06T03:52:41.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.483+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1f985be0 con 0x557f1f9a6b50 2023-07-06T03:52:41.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.483+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f1f9d2100 con 0x557f1f9d1190 2023-07-06T03:52:41.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffcb9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffcb9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9c5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59052/0 (socket says 172.21.15.97:59052) 2023-07-06T03:52:41.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffcb9a700 1 -- 172.21.15.97:0/3763711646 learned_addr learned my addr 172.21.15.97:0/3763711646 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:41.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/3763711646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9a6b50 0x557f1f9d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffdb9c700 1 --2- 172.21.15.97:0/3763711646 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9d1190 0x557f1f9c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffcb9a700 1 -- 172.21.15.97:0/3763711646 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9d1190 msgr2=0x557f1f9c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:41.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffcb9a700 1 --2- 172.21.15.97:0/3763711646 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9d1190 0x557f1f9c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffcb9a700 1 -- 172.21.15.97:0/3763711646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9a6b50 msgr2=0x557f1f9d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:41.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffcb9a700 1 --2- 172.21.15.97:0/3763711646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9a6b50 0x557f1f9d1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffcb9a700 1 -- 172.21.15.97:0/3763711646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f1f8bfb10 con 0x557f1f9cbd50 2023-07-06T03:52:41.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/3763711646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9a6b50 0x557f1f9d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:41.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2ffcb9a700 1 --2- 172.21.15.97:0/3763711646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9c5600 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80572d0 tx=0x7f2fe8057110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fbd06c2a72d3a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:41.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2fe37fe700 1 -- 172.21.15.97:0/3763711646 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe805b590 con 0x557f1f9cbd50 2023-07-06T03:52:41.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2fe37fe700 1 -- 172.21.15.97:0/3763711646 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe805f920 con 0x557f1f9cbd50 2023-07-06T03:52:41.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.484+0000 7f2fe37fe700 1 -- 172.21.15.97:0/3763711646 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8055970 con 0x557f1f9cbd50 2023-07-06T03:52:41.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.485+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3763711646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 msgr2=0x557f1f9c5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.485+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3763711646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9c5600 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80572d0 tx=0x7f2fe8057110 comp rx=0 tx=0).stop 2023-07-06T03:52:41.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.485+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3763711646 shutdown_connections 2023-07-06T03:52:41.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.485+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3763711646 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9d1190 0x557f1f9c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.485+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3763711646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9a6b50 0x557f1f9d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.485+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3763711646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9cbd50 0x557f1f9c5600 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.485+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3763711646 >> 172.21.15.97:0/3763711646 conn(0x557f1f99e310 msgr2=0x557f1f99f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:41.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.485+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3763711646 shutdown_connections 2023-07-06T03:52:41.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.485+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3763711646 wait complete. 2023-07-06T03:52:41.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.486+0000 7f3012f9eb80 1 -- start start 2023-07-06T03:52:41.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.486+0000 7f3012f9eb80 1 Processor -- start 2023-07-06T03:52:41.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9a6b50 0x557f1f9e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9cbd50 0x557f1f9a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f3012f9eb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9d1190 0x557f1f9e5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1f9e60c0 con 0x557f1f9a6b50 2023-07-06T03:52:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1f9ca270 con 0x557f1f9d1190 2023-07-06T03:52:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f3012f9eb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f1f9c88f0 con 0x557f1f9cbd50 2023-07-06T03:52:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f3012f9eb80 10 client.0 _ll_register_callbacks cb 0x7f2fe4003740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:52:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f2ffd39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9a6b50 0x557f1f9e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f2ffd39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9a6b50 0x557f1f9e7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59060/0 (socket says 172.21.15.97:59060) 2023-07-06T03:52:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f2ffd39b700 1 -- 172.21.15.97:0/3512835686 learned_addr learned my addr 172.21.15.97:0/3512835686 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:41.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f2ffd39b700 1 -- 172.21.15.97:0/3512835686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9cbd50 msgr2=0x557f1f9a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:41.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/3512835686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9cbd50 0x557f1f9a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f2ffd39b700 1 -- 172.21.15.97:0/3512835686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9d1190 msgr2=0x557f1f9e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:41.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/3512835686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9d1190 0x557f1f9e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f2ffd39b700 1 -- 172.21.15.97:0/3512835686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff80ca8f0 con 0x557f1f9a6b50 2023-07-06T03:52:41.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f2ffd39b700 1 --2- 172.21.15.97:0/3512835686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9a6b50 0x557f1f9e7f50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4001eb0 tx=0x7f2ff4001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33a946cc8e67a23 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:41.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.488+0000 7f2fd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:52:41.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f2fd7fff700 1 -- 172.21.15.97:0/3512835686 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4014dd0 con 0x557f1f9a6b50 2023-07-06T03:52:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f2fd7fff700 1 -- 172.21.15.97:0/3512835686 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2ff401c040 con 0x557f1f9a6b50 2023-07-06T03:52:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3512835686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ff40145b0 con 0x557f1f9a6b50 2023-07-06T03:52:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f2fd7fff700 1 -- 172.21.15.97:0/3512835686 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ff4031260 con 0x557f1f9a6b50 2023-07-06T03:52:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f3012f9eb80 10 client.7638 fetch_fsmap learned FSMap version 56 2023-07-06T03:52:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3512835686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x557f1f74e1d0 con 0x557f1f9a6b50 2023-07-06T03:52:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f2fd7fff700 1 -- 172.21.15.97:0/3512835686 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff40145b0 con 0x557f1f9a6b50 2023-07-06T03:52:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f2fd7fff700 1 -- 172.21.15.97:0/3512835686 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 56) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2ff40313c0 con 0x557f1f9a6b50 2023-07-06T03:52:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f3012f9eb80 10 client.7638 fetch_fsmap finished waiting for FSMap version 56 2023-07-06T03:52:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f3012f9eb80 10 client.7638 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-06T03:52:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f3012f9eb80 20 client.7638 populate_metadata read hostname 'smithi097' 2023-07-06T03:52:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3512835686 >> [v2:172.21.15.97:6834/2046478602,v1:172.21.15.97:6836/2046478602] conn(0x557f1f9c5600 0x557f1f9c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f3012f9eb80 4 client.7638 mds_command: new command op to 7497 tid=0[{"prefix": "status"}] 2023-07-06T03:52:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.489+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3512835686 --> [v2:172.21.15.97:6834/2046478602,v1:172.21.15.97:6836/2046478602] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557f1f8f2d20 con 0x557f1f9c5600 2023-07-06T03:52:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f2ffcb9a700 1 --2- 172.21.15.97:0/3512835686 >> [v2:172.21.15.97:6834/2046478602,v1:172.21.15.97:6836/2046478602] conn(0x557f1f9c5600 0x557f1f9c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f2ffcb9a700 1 --2- 172.21.15.97:0/3512835686 >> [v2:172.21.15.97:6834/2046478602,v1:172.21.15.97:6836/2046478602] conn(0x557f1f9c5600 0x557f1f9c59f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1346362f00060c02 server_cookie=c5aaf4ca9eb1bd48 in_seq=0 out_seq=0 2023-07-06T03:52:41.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f2fd7fff700 10 client.7638 ms_handle_connect on v2:172.21.15.97:6834/2046478602 2023-07-06T03:52:41.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f2fd7fff700 1 -- 172.21.15.97:0/3512835686 <== mds.0 v2:172.21.15.97:6834/2046478602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557f1f8f2d20 con 0x557f1f9c5600 2023-07-06T03:52:41.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f2fd7fff700 10 client.7638 handle_command_reply: tid=0 2023-07-06T03:52:41.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f3012f9eb80 1 client.7638 shutdown 2023-07-06T03:52:41.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3512835686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9a6b50 msgr2=0x557f1f9e7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3512835686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9a6b50 0x557f1f9e7f50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4001eb0 tx=0x7f2ff4001150 comp rx=0 tx=0).stop 2023-07-06T03:52:41.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3512835686 shutdown_connections 2023-07-06T03:52:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3512835686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f1f9d1190 0x557f1f9e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3512835686 >> [v2:172.21.15.97:6834/2046478602,v1:172.21.15.97:6836/2046478602] conn(0x557f1f9c5600 0x557f1f9c59f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3512835686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f1f9a6b50 0x557f1f9e7f50 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f3012f9eb80 1 --2- 172.21.15.97:0/3512835686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x557f1f9cbd50 0x557f1f9a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f2fd7fff700 0 client.7638 ms_handle_reset on v2:172.21.15.97:6834/2046478602 2023-07-06T03:52:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.490+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3512835686 >> 172.21.15.97:0/3512835686 conn(0x557f1f99e310 msgr2=0x557f1f99f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3512835686 shutdown_connections 2023-07-06T03:52:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f3012f9eb80 1 -- 172.21.15.97:0/3512835686 wait complete. 2023-07-06T03:52:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f3012f9eb80 20 client.7638 trim_cache size 0 max 16384 2023-07-06T03:52:41.501 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:52:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36", 2023-07-06T03:52:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-07-06T03:52:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7497, 2023-07-06T03:52:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-07-06T03:52:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-07-06T03:52:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-07-06T03:52:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.5370899610000004, 2023-07-06T03:52:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 56, 2023-07-06T03:52:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 86, 2023-07-06T03:52:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 86, 2023-07-06T03:52:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 13.046215545000001 2023-07-06T03:52:41.503 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:52:41.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f2fff5ff700 1 -- 172.21.15.97:0/1989415339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fdc06ace0 msgr2=0x7f2fdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f2fff5ff700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fdc06ace0 0x7f2fdc06d190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40011e0 tx=0x7f2ff4009040 comp rx=0 tx=0).stop 2023-07-06T03:52:41.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f2fff5ff700 1 -- 172.21.15.97:0/1989415339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80d2210 msgr2=0x7f2ff814b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f2fff5ff700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80d2210 0x7f2ff814b0c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8001eb0 tx=0x7f2fe800c7b0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f2fff5ff700 1 -- 172.21.15.97:0/1989415339 shutdown_connections 2023-07-06T03:52:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f2fff5ff700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2fdc06ace0 0x7f2fdc06d190 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f2fff5ff700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ff815c610 0x7f2ff815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f2fff5ff700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ff806b360 0x7f2ff80c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f2fff5ff700 1 --2- 172.21.15.97:0/1989415339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff80d2210 0x7f2ff814b0c0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.491+0000 7f2fff5ff700 1 -- 172.21.15.97:0/1989415339 >> 172.21.15.97:0/1989415339 conn(0x7f2ff806a6c0 msgr2=0x7f2ff815a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.492+0000 7f2fff5ff700 1 -- 172.21.15.97:0/1989415339 shutdown_connections 2023-07-06T03:52:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.492+0000 7f2fff5ff700 1 -- 172.21.15.97:0/1989415339 wait complete. 2023-07-06T03:52:41.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:52:41.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.702+0000 7f5931dff700 1 Processor -- start 2023-07-06T03:52:41.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.702+0000 7f5931dff700 1 -- start start 2023-07-06T03:52:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.703+0000 7f5931dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f592c0cd390 0x7f592c160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.703+0000 7f5931dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c0cdae0 con 0x7f592c15c520 2023-07-06T03:52:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.703+0000 7f5931dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c0b6640 con 0x7f592c160dd0 2023-07-06T03:52:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.703+0000 7f5931dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c0b6920 con 0x7f592c0cd390 2023-07-06T03:52:41.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.703+0000 7f592b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f592c15c520 0x7f592c160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48720/0 (socket says 172.21.15.97:48720) 2023-07-06T03:52:41.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.703+0000 7f592b7fe700 1 -- 172.21.15.97:0/1265143744 learned_addr learned my addr 172.21.15.97:0/1265143744 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:41.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.703+0000 7f5923fff700 1 --2- 172.21.15.97:0/1265143744 >> v2:172.21.15.97:3300/0 conn(0x7f592c0cd390 0x7f592c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.704+0000 7f592affd700 1 -- 172.21.15.97:0/1265143744 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173682193 0 0) 0x7f592c0cdae0 con 0x7f592c15c520 2023-07-06T03:52:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.704+0000 7f592affd700 1 -- 172.21.15.97:0/1265143744 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59180035b0 con 0x7f592c15c520 2023-07-06T03:52:41.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.704+0000 7f5923fff700 1 -- 172.21.15.97:0/1265143744 >> v1:172.21.15.146:6789/0 conn(0x7f592c160dd0 legacy=0x7f592c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.704+0000 7f5923fff700 1 -- 172.21.15.97:0/1265143744 >> v1:172.21.15.97:6789/0 conn(0x7f592c15c520 legacy=0x7f592c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.704+0000 7f5923fff700 1 -- 172.21.15.97:0/1265143744 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c0b6c00 con 0x7f592c0cd390 2023-07-06T03:52:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.704+0000 7f5923fff700 1 --2- 172.21.15.97:0/1265143744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c0cd390 0x7f592c160890 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5914009840 tx=0x7f5914009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90af88fb25d86982 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.704+0000 7f592affd700 1 -- 172.21.15.97:0/1265143744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914011830 con 0x7f592c0cd390 2023-07-06T03:52:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.704+0000 7f592affd700 1 -- 172.21.15.97:0/1265143744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914011990 con 0x7f592c0cd390 2023-07-06T03:52:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.704+0000 7f592affd700 1 -- 172.21.15.97:0/1265143744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914011d00 con 0x7f592c0cd390 2023-07-06T03:52:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.705+0000 7f5931dff700 1 -- 172.21.15.97:0/1265143744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c0cd390 msgr2=0x7f592c160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.705+0000 7f5931dff700 1 --2- 172.21.15.97:0/1265143744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c0cd390 0x7f592c160890 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5914009840 tx=0x7f5914009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.705+0000 7f5931dff700 1 -- 172.21.15.97:0/1265143744 shutdown_connections 2023-07-06T03:52:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.705+0000 7f5931dff700 1 --2- 172.21.15.97:0/1265143744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c0cd390 0x7f592c160890 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.705+0000 7f5931dff700 1 -- 172.21.15.97:0/1265143744 >> 172.21.15.97:0/1265143744 conn(0x7f592c071790 msgr2=0x7f592c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.705+0000 7f5931dff700 1 -- 172.21.15.97:0/1265143744 shutdown_connections 2023-07-06T03:52:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.705+0000 7f5931dff700 1 -- 172.21.15.97:0/1265143744 wait complete. 2023-07-06T03:52:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.707+0000 7f5931dff700 1 Processor -- start 2023-07-06T03:52:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.707+0000 7f5931dff700 1 -- start start 2023-07-06T03:52:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f5931dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f592c0cd390 0x7f592c0ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f5931dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c15c520 0x7f592c1f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f5931dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f592c160dd0 0x7f592c1f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f5931dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1f2840 con 0x7f592c15c520 2023-07-06T03:52:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f5931dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1f2b20 con 0x7f592c160dd0 2023-07-06T03:52:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f5931dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1f2e00 con 0x7f592c0cd390 2023-07-06T03:52:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f5923fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c15c520 0x7f592c1f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f5923fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c15c520 0x7f592c1f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59090/0 (socket says 172.21.15.97:59090) 2023-07-06T03:52:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f5923fff700 1 -- 172.21.15.97:0/40616187 learned_addr learned my addr 172.21.15.97:0/40616187 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f592b7fe700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f592c0cd390 0x7f592c0ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f592bfff700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f592c160dd0 0x7f592c1f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f5923fff700 1 -- 172.21.15.97:0/40616187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f592c0cd390 msgr2=0x7f592c0ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.708+0000 7f5923fff700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f592c0cd390 0x7f592c0ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.709+0000 7f5923fff700 1 -- 172.21.15.97:0/40616187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f592c160dd0 msgr2=0x7f592c1f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.709+0000 7f5923fff700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f592c160dd0 0x7f592c1f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.709+0000 7f5923fff700 1 -- 172.21.15.97:0/40616187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59140094a0 con 0x7f592c15c520 2023-07-06T03:52:41.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.709+0000 7f592bfff700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f592c160dd0 0x7f592c1f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:41.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.709+0000 7f5923fff700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c15c520 0x7f592c1f4ec0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5914009fd0 tx=0x7f5914004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:41.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.709+0000 7f5928ff9700 1 -- 172.21.15.97:0/40616187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59140036a0 con 0x7f592c15c520 2023-07-06T03:52:41.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.709+0000 7f5928ff9700 1 -- 172.21.15.97:0/40616187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591401e070 con 0x7f592c15c520 2023-07-06T03:52:41.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.709+0000 7f5928ff9700 1 -- 172.21.15.97:0/40616187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914003a40 con 0x7f592c15c520 2023-07-06T03:52:41.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.709+0000 7f5931dff700 1 -- 172.21.15.97:0/40616187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c1f30e0 con 0x7f592c15c520 2023-07-06T03:52:41.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.709+0000 7f5931dff700 1 -- 172.21.15.97:0/40616187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c1f2490 con 0x7f592c15c520 2023-07-06T03:52:41.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.710+0000 7f5931dff700 1 -- 172.21.15.97:0/40616187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c1f3320 con 0x7f592c15c520 2023-07-06T03:52:41.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.711+0000 7f5928ff9700 1 -- 172.21.15.97:0/40616187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5914003ba0 con 0x7f592c15c520 2023-07-06T03:52:41.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.712+0000 7f5928ff9700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f590c06ace0 0x7f590c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:41.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.712+0000 7f5928ff9700 1 -- 172.21.15.97:0/40616187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f591408a5e0 con 0x7f592c15c520 2023-07-06T03:52:41.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.712+0000 7f592b7fe700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f590c06ace0 0x7f590c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:41.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.713+0000 7f592b7fe700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f590c06ace0 0x7f590c06d190 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f591c0011e0 tx=0x7f591c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:41.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.716+0000 7f5928ff9700 1 -- 172.21.15.97:0/40616187 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f592c1f3320 con 0x7f592c15c520 2023-07-06T03:52:41.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.957+0000 7f5931dff700 1 -- 172.21.15.97:0/40616187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f592c1f3320 con 0x7f592c15c520 2023-07-06T03:52:41.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.958+0000 7f5928ff9700 1 -- 172.21.15.97:0/40616187 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f592c1f3320 con 0x7f592c15c520 2023-07-06T03:52:41.960 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:52:41.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 -- 172.21.15.97:0/40616187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f590c06ace0 msgr2=0x7f590c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f590c06ace0 0x7f590c06d190 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f591c0011e0 tx=0x7f591c009040 comp rx=0 tx=0).stop 2023-07-06T03:52:41.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 -- 172.21.15.97:0/40616187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c15c520 msgr2=0x7f592c1f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:41.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c15c520 0x7f592c1f4ec0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5914009fd0 tx=0x7f5914004750 comp rx=0 tx=0).stop 2023-07-06T03:52:41.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 -- 172.21.15.97:0/40616187 shutdown_connections 2023-07-06T03:52:41.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f590c06ace0 0x7f590c06d190 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f592c160dd0 0x7f592c1f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f592c0cd390 0x7f592c0ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 --2- 172.21.15.97:0/40616187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c15c520 0x7f592c1f4ec0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:41.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 -- 172.21.15.97:0/40616187 >> 172.21.15.97:0/40616187 conn(0x7f592c071790 msgr2=0x7f592c0b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:41.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 -- 172.21.15.97:0/40616187 shutdown_connections 2023-07-06T03:52:41.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:41.960+0000 7f5931dff700 1 -- 172.21.15.97:0/40616187 wait complete. 2023-07-06T03:52:41.980 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:52:41.981 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:52:41.981 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:52:41.981 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:52:41.981 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:41.982 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:52:41.982 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:52:41.982 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:52:41.982 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:52:41.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:41.983 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:52:42.029 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:52:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:52:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:52:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:52:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:42.030 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:52:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:52:42.030 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:52:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:52:42.031 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:52:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:42.031 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:52:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:52:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41707sec preferred_lft 41707sec 2023-07-06T03:52:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:52:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:42.032 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:52:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:52:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:52:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:42.032 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:52:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:52:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:52:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:42.033 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:52:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:52:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:52:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:42.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:42.036 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:52:42.142 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:52:42.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:42.145 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:52:42.194 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:52:42.194 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:52:42.194 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:52:42.194 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:52:42.194 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:52:42.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:42.196 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:52:42.196 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:52:42.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:52:42.197 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:52:42.197 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:52:42.197 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:52:42.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:42.364 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:52:42.409 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:52:42.410 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:52:42.410 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:52:42.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:42.411 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:52:42.480 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:52:42.481 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:52:42.481 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:42.527 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:52:42.527 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:52:42.528 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:42.611 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:52:42.720 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:52:42.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:52:42.788 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:52:42.790 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:52:42.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:42.791 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:52:42.836 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:52:42.837 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:52:42.881 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:52:42.951 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:52:42.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:52:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:52:43.026 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:52:43.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:43.027 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:52:43.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:52:43.049+0000 7fec2f6bc5c0 -1 init, newargv = 0x5623b364d110 newargc=15 2023-07-06T03:52:43.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[149203]: starting ceph client 2023-07-06T03:52:43.058 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[149203]: starting fuse 2023-07-06T03:52:43.076 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:52:43.077 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:52:43.077 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:52:43.077 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:52:43.078 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:52:43.078 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:52:43.078 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:52:43.079 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:43.079 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:52:43.079 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:52:43.080 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:52:43.080 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:52:43.080 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:43.080 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:52:43.081 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:52:43.081 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:52:43.081 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:52:43.082 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:43.082 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:52:43.082 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:52:43.083 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:52:43.083 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:52:43.083 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:52:43.083 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:52:43.084 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:52:43.084 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:52:43.084 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:43.085 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:52:43.085 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:52:43.259 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.149203.asok 2023-07-06T03:52:43.436 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.149203.asok 2023-07-06T03:52:43.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:43.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.149203.asok status 2023-07-06T03:52:43.659 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:52:43.660 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:52:43.660 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:52:43.660 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:52:43.660 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:52:43.661 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:52:43.661 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:52:43.661 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "149203", 2023-07-06T03:52:43.661 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:52:43.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:43.661 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:52:43.661 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:52:43.662 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7653, 2023-07-06T03:52:43.662 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:52:43.662 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:52:43.662 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:52:43.662 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 7653 2023-07-06T03:52:43.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:43.662 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:52:43.663 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:52:43.663 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:52:43.663 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1311806518 2023-07-06T03:52:43.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:43.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:43.663 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:52:43.663 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:52:43.664 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:52:43.664 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1311806518 2023-07-06T03:52:43.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:43.664 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.7653 192.168.0.1:0/1311806518", 2023-07-06T03:52:43.664 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/1311806518", 2023-07-06T03:52:43.664 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:52:43.664 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 56, 2023-07-06T03:52:43.665 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 86, 2023-07-06T03:52:43.665 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:52:43.665 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:52:43.665 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:52:43.665 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:52:43.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:43.674 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:52:43.675 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:43.722 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:52:43.722 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:43.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:43.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:43.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:52:43.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.985+0000 7f7512fff700 1 Processor -- start 2023-07-06T03:52:43.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.985+0000 7f7512fff700 1 -- start start 2023-07-06T03:52:43.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.986+0000 7f7512fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f750c0cd390 0x7f750c160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:43.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.986+0000 7f7512fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f750c160dd0 0x7f750c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:43.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.986+0000 7f7512fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750c0cdae0 con 0x7f750c15c520 2023-07-06T03:52:43.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.986+0000 7f7512fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f750c0b6640 con 0x7f750c0cd390 2023-07-06T03:52:43.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.986+0000 7f7512fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f750c0b6920 con 0x7f750c160dd0 2023-07-06T03:52:43.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.986+0000 7f750bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f750c0cd390 0x7f750c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:43.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.986+0000 7f750bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f750c0cd390 0x7f750c160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59102/0 (socket says 172.21.15.97:59102) 2023-07-06T03:52:43.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.986+0000 7f750bfff700 1 -- 172.21.15.97:0/2543547925 learned_addr learned my addr 172.21.15.97:0/2543547925 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:43.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.986+0000 7f7510d9b700 1 --1- 172.21.15.97:0/2543547925 >> v1:172.21.15.97:6789/0 conn(0x7f750c15c520 0x7f750c160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48722/0 (socket says 172.21.15.97:48722) 2023-07-06T03:52:43.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.987+0000 7f751159c700 1 --2- 172.21.15.97:0/2543547925 >> v2:172.21.15.146:3300/0 conn(0x7f750c160dd0 0x7f750c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:43.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.987+0000 7f750bfff700 1 -- 172.21.15.97:0/2543547925 >> v2:172.21.15.146:3300/0 conn(0x7f750c160dd0 msgr2=0x7f750c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:43.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.987+0000 7f750bfff700 1 --2- 172.21.15.97:0/2543547925 >> v2:172.21.15.146:3300/0 conn(0x7f750c160dd0 0x7f750c0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:43.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.987+0000 7f750bfff700 1 -- 172.21.15.97:0/2543547925 >> v1:172.21.15.97:6789/0 conn(0x7f750c15c520 legacy=0x7f750c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:43.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.987+0000 7f750b7fe700 1 -- 172.21.15.97:0/2543547925 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265114984 0 0) 0x7f750c0cdae0 con 0x7f750c15c520 2023-07-06T03:52:43.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.987+0000 7f750bfff700 1 -- 172.21.15.97:0/2543547925 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c0b6c00 con 0x7f750c0cd390 2023-07-06T03:52:43.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.987+0000 7f750bfff700 1 --2- 172.21.15.97:0/2543547925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f750c0cd390 0x7f750c160890 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f750c0cd320 tx=0x7f74f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702ac8d45124863a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:43.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.987+0000 7f750b7fe700 1 -- 172.21.15.97:0/2543547925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f800b870 con 0x7f750c0cd390 2023-07-06T03:52:43.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.987+0000 7f750b7fe700 1 -- 172.21.15.97:0/2543547925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f800b9d0 con 0x7f750c0cd390 2023-07-06T03:52:43.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.987+0000 7f750b7fe700 1 -- 172.21.15.97:0/2543547925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f800bd60 con 0x7f750c0cd390 2023-07-06T03:52:43.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.988+0000 7f7512fff700 1 -- 172.21.15.97:0/2543547925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f750c0cd390 msgr2=0x7f750c160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:43.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.988+0000 7f7512fff700 1 --2- 172.21.15.97:0/2543547925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f750c0cd390 0x7f750c160890 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f750c0cd320 tx=0x7f74f8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:52:43.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.988+0000 7f7512fff700 1 -- 172.21.15.97:0/2543547925 shutdown_connections 2023-07-06T03:52:43.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.988+0000 7f7512fff700 1 --2- 172.21.15.97:0/2543547925 >> v2:172.21.15.146:3300/0 conn(0x7f750c160dd0 0x7f750c0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:43.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.988+0000 7f7512fff700 1 --2- 172.21.15.97:0/2543547925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f750c0cd390 0x7f750c160890 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:43.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.988+0000 7f7512fff700 1 -- 172.21.15.97:0/2543547925 >> 172.21.15.97:0/2543547925 conn(0x7f750c071790 msgr2=0x7f750c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:43.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.988+0000 7f7512fff700 1 -- 172.21.15.97:0/2543547925 shutdown_connections 2023-07-06T03:52:43.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.988+0000 7f7512fff700 1 -- 172.21.15.97:0/2543547925 wait complete. 2023-07-06T03:52:43.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.990+0000 7f7512fff700 1 Processor -- start 2023-07-06T03:52:43.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.990+0000 7f7512fff700 1 -- start start 2023-07-06T03:52:43.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.991+0000 7f7512fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f750c0cd390 0x7f750c0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:43.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.991+0000 7f7512fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f750c15c520 0x7f750c0c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:43.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.991+0000 7f7512fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c160dd0 0x7f750c147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:43.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.991+0000 7f7512fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c0c9c70 con 0x7f750c0cd390 2023-07-06T03:52:43.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.991+0000 7f7512fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c15f590 con 0x7f750c160dd0 2023-07-06T03:52:43.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.991+0000 7f7512fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750c15f870 con 0x7f750c15c520 2023-07-06T03:52:43.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.991+0000 7f7510d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f750c0cd390 0x7f750c0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:43.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.991+0000 7f7510d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f750c0cd390 0x7f750c0c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59114/0 (socket says 172.21.15.97:59114) 2023-07-06T03:52:43.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.991+0000 7f7510d9b700 1 -- 172.21.15.97:0/2408010447 learned_addr learned my addr 172.21.15.97:0/2408010447 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:43.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.991+0000 7f751159c700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c160dd0 0x7f750c147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:43.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.991+0000 7f750bfff700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f750c15c520 0x7f750c0c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:43.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f7510d9b700 1 -- 172.21.15.97:0/2408010447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f750c15c520 msgr2=0x7f750c0c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:43.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f7510d9b700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f750c15c520 0x7f750c0c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:43.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f7510d9b700 1 -- 172.21.15.97:0/2408010447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c160dd0 msgr2=0x7f750c147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:43.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f7510d9b700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c160dd0 0x7f750c147bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:43.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f7510d9b700 1 -- 172.21.15.97:0/2408010447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f8009550 con 0x7f750c0cd390 2023-07-06T03:52:43.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f751159c700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c160dd0 0x7f750c147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:43.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f750bfff700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f750c15c520 0x7f750c0c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:43.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f7510d9b700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f750c0cd390 0x7f750c0c91f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f75000011e0 tx=0x7f750000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:43.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f75097fa700 1 -- 172.21.15.97:0/2408010447 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500004c80 con 0x7f750c0cd390 2023-07-06T03:52:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f75097fa700 1 -- 172.21.15.97:0/2408010447 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7500004e00 con 0x7f750c0cd390 2023-07-06T03:52:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f75097fa700 1 -- 172.21.15.97:0/2408010447 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500007310 con 0x7f750c0cd390 2023-07-06T03:52:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f7512fff700 1 -- 172.21.15.97:0/2408010447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f750c0d3660 con 0x7f750c0cd390 2023-07-06T03:52:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.992+0000 7f7512fff700 1 -- 172.21.15.97:0/2408010447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750c15f280 con 0x7f750c0cd390 2023-07-06T03:52:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.993+0000 7f7512fff700 1 -- 172.21.15.97:0/2408010447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f750c0d3b80 con 0x7f750c0cd390 2023-07-06T03:52:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.994+0000 7f75097fa700 1 -- 172.21.15.97:0/2408010447 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7500007470 con 0x7f750c0cd390 2023-07-06T03:52:44.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.995+0000 7f75097fa700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74f006ace0 0x7f74f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:44.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.995+0000 7f75097fa700 1 -- 172.21.15.97:0/2408010447 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7500089280 con 0x7f750c0cd390 2023-07-06T03:52:44.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.995+0000 7f750bfff700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74f006ace0 0x7f74f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:44.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:43.996+0000 7f750bfff700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74f006ace0 0x7f74f006d190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f750c0cbb00 tx=0x7f74f8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:44.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.000+0000 7f75097fa700 1 -- 172.21.15.97:0/2408010447 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f750c0d3b80 con 0x7f750c0cd390 2023-07-06T03:52:44.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.237+0000 7f7512fff700 1 -- 172.21.15.97:0/2408010447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f750c0d3b80 con 0x7f750c0cd390 2023-07-06T03:52:44.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.237+0000 7f75097fa700 1 -- 172.21.15.97:0/2408010447 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f750c0d3b80 con 0x7f750c0cd390 2023-07-06T03:52:44.239 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:52:44.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 -- 172.21.15.97:0/2408010447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74f006ace0 msgr2=0x7f74f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:44.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74f006ace0 0x7f74f006d190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f750c0cbb00 tx=0x7f74f8008030 comp rx=0 tx=0).stop 2023-07-06T03:52:44.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 -- 172.21.15.97:0/2408010447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f750c0cd390 msgr2=0x7f750c0c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:44.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f750c0cd390 0x7f750c0c91f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f75000011e0 tx=0x7f750000cae0 comp rx=0 tx=0).stop 2023-07-06T03:52:44.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 -- 172.21.15.97:0/2408010447 shutdown_connections 2023-07-06T03:52:44.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74f006ace0 0x7f74f006d190 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:44.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f750c160dd0 0x7f750c147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:44.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f750c0cd390 0x7f750c0c91f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:44.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 --2- 172.21.15.97:0/2408010447 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f750c15c520 0x7f750c0c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:44.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 -- 172.21.15.97:0/2408010447 >> 172.21.15.97:0/2408010447 conn(0x7f750c071790 msgr2=0x7f750c0b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:44.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 -- 172.21.15.97:0/2408010447 shutdown_connections 2023-07-06T03:52:44.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:44.240+0000 7f7512fff700 1 -- 172.21.15.97:0/2408010447 wait complete. 2023-07-06T03:52:44.259 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:52:44.259 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:52:44.260 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2023-07-06T03:52:44.260 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T03:52:44.260 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T03:52:44.260 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:52:44.261 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:52:44.261 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:52:44.261 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:52:44.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:44.262 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2023-07-06T03:52:44.314 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:52:44.315 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:52:44.315 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:52:44.315 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:44.315 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:52:44.316 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:44.316 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:52:44.316 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:44.316 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2023-07-06T03:52:44.317 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:52:44.317 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:44.317 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2023-07-06T03:52:44.317 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:52:44.318 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:44.318 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:52:44.318 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:44.319 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:52:44.319 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 41720sec preferred_lft 41720sec 2023-07-06T03:52:44.319 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2023-07-06T03:52:44.319 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:44.320 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:52:44.320 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:44.320 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:52:44.321 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:44.321 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::58f0:6eff:fe04:3be0/64 scope link 2023-07-06T03:52:44.321 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:44.321 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:52:44.322 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:52:44.322 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::106d:9cff:fed5:9d0f/64 scope link 2023-07-06T03:52:44.322 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:44.322 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:52:44.323 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether a2:c5:04:8e:a2:a7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T03:52:44.323 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::a0c5:4ff:fe8e:a2a7/64 scope link 2023-07-06T03:52:44.324 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:44.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:44.326 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:52:44.397 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:52:44.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:44.400 DEBUG:teuthology.orchestra.run.smithi146:> route 2023-07-06T03:52:44.450 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2023-07-06T03:52:44.451 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:52:44.451 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:52:44.451 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:52:44.451 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:52:44.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:44.452 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:52:44.453 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:52:44.453 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:52:44.453 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:52:44.454 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:52:44.454 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:52:44.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:44.621 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:52:44.667 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T03:52:44.667 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T03:52:44.668 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:52:44.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:44.669 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2023-07-06T03:52:44.740 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-06T03:52:44.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:52:44.741 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T03:52:44.788 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:52:44.788 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:52:44.789 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:52:44.873 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:52:44.982 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:52:44.983 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:52:45.050 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:52:45.053 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:52:45.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:45.055 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:52:45.100 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:52:45.101 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-06T03:52:45.145 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:52:45.216 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:52:45.217 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:52:45.288 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:52:45.291 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:52:45.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:45.292 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:52:45.316 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2023-07-06T03:52:45.315+0000 7f0680c085c0 -1 init, newargv = 0x55636a881270 newargc=15 2023-07-06T03:52:45.317 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[124181]: starting ceph client 2023-07-06T03:52:45.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[124181]: starting fuse 2023-07-06T03:52:45.343 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:52:45.343 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T03:52:45.344 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:52:45.344 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2023-07-06T03:52:45.344 DEBUG:teuthology.orchestra.run.smithi146:> import re 2023-07-06T03:52:45.345 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:52:45.345 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2023-07-06T03:52:45.345 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:52:45.346 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2023-07-06T03:52:45.346 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-06T03:52:45.346 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2023-07-06T03:52:45.346 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-06T03:52:45.347 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:52:45.347 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2023-07-06T03:52:45.347 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2023-07-06T03:52:45.348 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2023-07-06T03:52:45.348 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2023-07-06T03:52:45.348 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:52:45.349 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2023-07-06T03:52:45.349 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:52:45.349 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:52:45.349 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:52:45.350 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2023-07-06T03:52:45.350 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2023-07-06T03:52:45.351 DEBUG:teuthology.orchestra.run.smithi146:> return f 2023-07-06T03:52:45.351 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:52:45.351 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:52:45.351 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2023-07-06T03:52:45.352 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:52:45.527 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.124181.asok 2023-07-06T03:52:45.704 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.124181.asok 2023-07-06T03:52:45.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:45.705 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.124181.asok status 2023-07-06T03:52:45.927 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-07-06T03:52:45.927 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2023-07-06T03:52:45.927 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:52:45.928 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:52:45.928 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2023-07-06T03:52:45.928 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2023-07-06T03:52:45.928 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-06T03:52:45.928 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "124181", 2023-07-06T03:52:45.928 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2023-07-06T03:52:45.928 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:52:45.929 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2023-07-06T03:52:45.929 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2023-07-06T03:52:45.929 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 7114, 2023-07-06T03:52:45.929 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2023-07-06T03:52:45.929 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2023-07-06T03:52:45.929 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2023-07-06T03:52:45.929 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 7114 2023-07-06T03:52:45.929 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:52:45.930 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:52:45.930 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:52:45.930 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:52:45.930 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 3754810850 2023-07-06T03:52:45.930 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-07-06T03:52:45.930 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:52:45.930 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:52:45.931 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:52:45.931 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:52:45.931 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 3754810850 2023-07-06T03:52:45.931 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:52:45.931 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.7114 192.168.0.1:0/3754810850", 2023-07-06T03:52:45.932 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/3754810850", 2023-07-06T03:52:45.932 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2023-07-06T03:52:45.932 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 56, 2023-07-06T03:52:45.932 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 86, 2023-07-06T03:52:45.932 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:52:45.932 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2023-07-06T03:52:45.932 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2023-07-06T03:52:45.933 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-07-06T03:52:45.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:45.943 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2023-07-06T03:52:45.943 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-06T03:52:45.989 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2023-07-06T03:52:45.990 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-06T03:52:45.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:45.991 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:52:46.060 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:52:46.066 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:52:46.183 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2023-07-06T03:52:46.183 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2023-07-06T03:52:46.183 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:52:46.183 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2023-07-06T03:52:46.184 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.184 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2023-07-06T03:52:46.184 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.184 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2023-07-06T03:52:46.184 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.184 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2023-07-06T03:52:46.184 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2023-07-06T03:52:46.185 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.185 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2023-07-06T03:52:46.185 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2023-07-06T03:52:46.185 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.185 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:52:46.185 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2023-07-06T03:52:46.185 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.185 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:52:46.186 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:52:46.186 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.186 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2023-07-06T03:52:46.186 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.186 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2023-07-06T03:52:46.186 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2023-07-06T03:52:46.186 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:52:46.186 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:52:46.187 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2023-07-06T03:52:46.187 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:52:46.187 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2023-07-06T03:52:46.187 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:52:46.187 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:52:46.187 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2023-07-06T03:52:46.187 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.187 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:52:46.188 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.188 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2023-07-06T03:52:46.188 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.188 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2023-07-06T03:52:46.188 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.188 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2023-07-06T03:52:46.188 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2023-07-06T03:52:46.189 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2023-07-06T03:52:46.189 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:52:46.189 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:52:46.189 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.189 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2023-07-06T03:52:46.189 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2023-07-06T03:52:46.189 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.189 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2023-07-06T03:52:46.190 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2023-07-06T03:52:46.190 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:52:46.190 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.190 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2023-07-06T03:52:46.190 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2023-07-06T03:52:46.190 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.190 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2023-07-06T03:52:46.190 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2023-07-06T03:52:46.191 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2023-07-06T03:52:46.191 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.191 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2023-07-06T03:52:46.191 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2023-07-06T03:52:46.191 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.191 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:52:46.191 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.192 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2023-07-06T03:52:46.192 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.192 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2023-07-06T03:52:46.192 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.192 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:52:46.192 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2023-07-06T03:52:46.192 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.192 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2023-07-06T03:52:46.193 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2023-07-06T03:52:46.193 INFO:teuthology.orchestra.run.smithi146.stdout: bdev async discard = True 2023-07-06T03:52:46.193 INFO:teuthology.orchestra.run.smithi146.stdout: bdev enable discard = True 2023-07-06T03:52:46.193 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore allocator = bitmap 2023-07-06T03:52:46.193 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2023-07-06T03:52:46.193 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2023-07-06T03:52:46.193 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 1/20 2023-07-06T03:52:46.193 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 1/20 2023-07-06T03:52:46.194 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:52:46.194 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2023-07-06T03:52:46.194 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 4/10 2023-07-06T03:52:46.194 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:52:46.194 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2023-07-06T03:52:46.194 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:52:46.194 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:52:46.194 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2023-07-06T03:52:46.195 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:52:46.195 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.195 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2023-07-06T03:52:46.195 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:52:46.195 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2023-07-06T03:52:46.195 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:52:46.195 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:52:46.195 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:52:46.196 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:52:46.196 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2023-07-06T03:52:46.196 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.196 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2023-07-06T03:52:46.196 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:52:46.196 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:52:46.196 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2023-07-06T03:52:46.196 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:52:46.197 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2023-07-06T03:52:46.197 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2023-07-06T03:52:46.197 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:52:46.197 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2023-07-06T03:52:46.197 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2023-07-06T03:52:46.197 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:52:46.197 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.197 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:52:46.198 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:52:46.198 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:52:46.198 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.198 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:52:46.198 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:52:46.198 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:52:46.198 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.198 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2023-07-06T03:52:46.199 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:52:46.199 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.199 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:52:46.199 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2023-07-06T03:52:46.199 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:52:46.199 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2023-07-06T03:52:46.199 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2023-07-06T03:52:46.200 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2023-07-06T03:52:46.200 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2023-07-06T03:52:46.200 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:52:46.200 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:52:46.200 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2023-07-06T03:52:46.200 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2023-07-06T03:52:46.200 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:52:46.200 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2023-07-06T03:52:46.201 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:52:46.201 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:52:46.201 INFO:teuthology.orchestra.run.smithi146.stdout: client inject fixed oldest tid = true 2023-07-06T03:52:46.201 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2023-07-06T03:52:46.201 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2023-07-06T03:52:46.201 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2023-07-06T03:52:46.201 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2023-07-06T03:52:46.201 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2023-07-06T03:52:46.202 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2023-07-06T03:52:46.202 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2023-07-06T03:52:46.202 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2023-07-06T03:52:46.202 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2023-07-06T03:52:46.202 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2023-07-06T03:52:46.202 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:52:46.202 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2023-07-06T03:52:46.202 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2023-07-06T03:52:46.203 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2023-07-06T03:52:46.203 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2023-07-06T03:52:46.203 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:52:46.203 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:52:46.203 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:52:46.203 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2023-07-06T03:52:46.204 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2023-07-06T03:52:46.204 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:52:46.204 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2023-07-06T03:52:46.204 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.204 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2023-07-06T03:52:46.204 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.205 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2023-07-06T03:52:46.205 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.205 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2023-07-06T03:52:46.205 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2023-07-06T03:52:46.205 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.205 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2023-07-06T03:52:46.205 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2023-07-06T03:52:46.205 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.206 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:52:46.206 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2023-07-06T03:52:46.206 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.206 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:52:46.206 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:52:46.206 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.206 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2023-07-06T03:52:46.206 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.207 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2023-07-06T03:52:46.207 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2023-07-06T03:52:46.207 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:52:46.207 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:52:46.207 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2023-07-06T03:52:46.207 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:52:46.207 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2023-07-06T03:52:46.207 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:52:46.208 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:52:46.208 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2023-07-06T03:52:46.208 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.208 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:52:46.208 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.208 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2023-07-06T03:52:46.208 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.209 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2023-07-06T03:52:46.209 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.209 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2023-07-06T03:52:46.209 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2023-07-06T03:52:46.209 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2023-07-06T03:52:46.209 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:52:46.209 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:52:46.209 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.210 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2023-07-06T03:52:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2023-07-06T03:52:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2023-07-06T03:52:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2023-07-06T03:52:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:52:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.210 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2023-07-06T03:52:46.211 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2023-07-06T03:52:46.211 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.211 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2023-07-06T03:52:46.211 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2023-07-06T03:52:46.211 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2023-07-06T03:52:46.211 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.211 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2023-07-06T03:52:46.211 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2023-07-06T03:52:46.212 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.212 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:52:46.212 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.212 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2023-07-06T03:52:46.212 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.212 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2023-07-06T03:52:46.212 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.212 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:52:46.213 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2023-07-06T03:52:46.213 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.213 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2023-07-06T03:52:46.213 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2023-07-06T03:52:46.213 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2023-07-06T03:52:46.213 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2023-07-06T03:52:46.213 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2023-07-06T03:52:46.213 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2023-07-06T03:52:46.214 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2023-07-06T03:52:46.214 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2023-07-06T03:52:46.214 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2023-07-06T03:52:46.214 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:52:46.214 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2023-07-06T03:52:46.214 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2023-07-06T03:52:46.214 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:52:46.214 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2023-07-06T03:52:46.215 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:52:46.215 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:52:46.215 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2023-07-06T03:52:46.215 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:52:46.215 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.215 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2023-07-06T03:52:46.215 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:52:46.215 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2023-07-06T03:52:46.216 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:52:46.216 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:52:46.216 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:52:46.216 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:52:46.216 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2023-07-06T03:52:46.216 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.216 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2023-07-06T03:52:46.216 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:52:46.217 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:52:46.217 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2023-07-06T03:52:46.217 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:52:46.217 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2023-07-06T03:52:46.217 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2023-07-06T03:52:46.217 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:52:46.217 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2023-07-06T03:52:46.217 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2023-07-06T03:52:46.218 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:52:46.218 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.218 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:52:46.218 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:52:46.218 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:52:46.218 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.218 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:52:46.219 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:52:46.219 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:52:46.219 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.219 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2023-07-06T03:52:46.219 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:52:46.219 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.219 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:52:46.219 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2023-07-06T03:52:46.220 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.220 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2023-07-06T03:52:46.220 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2023-07-06T03:52:46.220 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2023-07-06T03:52:46.220 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2023-07-06T03:52:46.220 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:52:46.220 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:52:46.220 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2023-07-06T03:52:46.221 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2023-07-06T03:52:46.221 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:52:46.221 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2023-07-06T03:52:46.221 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:52:46.221 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:52:46.221 INFO:teuthology.orchestra.run.smithi097.stdout: client inject fixed oldest tid = true 2023-07-06T03:52:46.221 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2023-07-06T03:52:46.221 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2023-07-06T03:52:46.222 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2023-07-06T03:52:46.222 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-07-06T03:52:46.222 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-07-06T03:52:46.222 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-07-06T03:52:46.222 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-07-06T03:52:46.222 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2023-07-06T03:52:46.222 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2023-07-06T03:52:46.222 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2023-07-06T03:52:46.223 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:52:46.223 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2023-07-06T03:52:46.223 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2023-07-06T03:52:46.223 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2023-07-06T03:52:46.223 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2023-07-06T03:52:46.223 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:52:46.223 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:52:46.223 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:52:46.225 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:52:46.225 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:52:46.270 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:52:46.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.503+0000 7f54345ff700 1 Processor -- start 2023-07-06T03:52:46.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.503+0000 7f54345ff700 1 -- start start 2023-07-06T03:52:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.505+0000 7f54345ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f542c0b0560 0x7f542c0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.505+0000 7f54345ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c0b0ea0 con 0x7f542c15a480 2023-07-06T03:52:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.505+0000 7f54345ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c0ae1e0 con 0x7f542c15a850 2023-07-06T03:52:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.505+0000 7f54345ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c071050 con 0x7f542c0b0560 2023-07-06T03:52:46.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.505+0000 7f543239b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f542c15a480 0x7f542c071b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48730/0 (socket says 172.21.15.97:48730) 2023-07-06T03:52:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.505+0000 7f543239b700 1 -- 172.21.15.97:0/3313878106 learned_addr learned my addr 172.21.15.97:0/3313878106 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.505+0000 7f5432b9c700 1 --2- 172.21.15.97:0/3313878106 >> v2:172.21.15.146:3300/0 conn(0x7f542c0b0560 0x7f542c0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:46.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.505+0000 7f5431399700 1 -- 172.21.15.97:0/3313878106 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617342614 0 0) 0x7f542c0b0ea0 con 0x7f542c15a480 2023-07-06T03:52:46.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.505+0000 7f5431399700 1 -- 172.21.15.97:0/3313878106 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54180035b0 con 0x7f542c15a480 2023-07-06T03:52:46.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.506+0000 7f5432b9c700 1 -- 172.21.15.97:0/3313878106 >> v1:172.21.15.146:6789/0 conn(0x7f542c15a850 legacy=0x7f542c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:46.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.506+0000 7f5432b9c700 1 -- 172.21.15.97:0/3313878106 >> v1:172.21.15.97:6789/0 conn(0x7f542c15a480 legacy=0x7f542c071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:46.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.506+0000 7f5432b9c700 1 -- 172.21.15.97:0/3313878106 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c071330 con 0x7f542c0b0560 2023-07-06T03:52:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.506+0000 7f5432b9c700 1 --2- 172.21.15.97:0/3313878106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f542c0b0560 0x7f542c0b0960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5424009d90 tx=0x7f542400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5832c465f92e588d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.506+0000 7f5431399700 1 -- 172.21.15.97:0/3313878106 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400e070 con 0x7f542c0b0560 2023-07-06T03:52:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.506+0000 7f5431399700 1 -- 172.21.15.97:0/3313878106 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542400bb20 con 0x7f542c0b0560 2023-07-06T03:52:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.506+0000 7f5431399700 1 -- 172.21.15.97:0/3313878106 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424012430 con 0x7f542c0b0560 2023-07-06T03:52:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.507+0000 7f54345ff700 1 -- 172.21.15.97:0/3313878106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f542c0b0560 msgr2=0x7f542c0b0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.507+0000 7f54345ff700 1 --2- 172.21.15.97:0/3313878106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f542c0b0560 0x7f542c0b0960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5424009d90 tx=0x7f542400b330 comp rx=0 tx=0).stop 2023-07-06T03:52:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.507+0000 7f54345ff700 1 -- 172.21.15.97:0/3313878106 shutdown_connections 2023-07-06T03:52:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.507+0000 7f54345ff700 1 --2- 172.21.15.97:0/3313878106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f542c0b0560 0x7f542c0b0960 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.507+0000 7f54345ff700 1 -- 172.21.15.97:0/3313878106 >> 172.21.15.97:0/3313878106 conn(0x7f542c0cb560 msgr2=0x7f542c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.507+0000 7f54345ff700 1 -- 172.21.15.97:0/3313878106 shutdown_connections 2023-07-06T03:52:46.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.507+0000 7f54345ff700 1 -- 172.21.15.97:0/3313878106 wait complete. 2023-07-06T03:52:46.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.509+0000 7f54345ff700 1 Processor -- start 2023-07-06T03:52:46.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.509+0000 7f54345ff700 1 -- start start 2023-07-06T03:52:46.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f54345ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f542c0b0560 0x7f542c153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:46.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f54345ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f542c15a480 0x7f542c15ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:46.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f54345ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f542c15a850 0x7f542c145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:46.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f54345ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c15b1a0 con 0x7f542c15a480 2023-07-06T03:52:46.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f54345ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c15dfc0 con 0x7f542c15a850 2023-07-06T03:52:46.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f54345ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c15d3b0 con 0x7f542c0b0560 2023-07-06T03:52:46.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f5431b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f542c15a480 0x7f542c15ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:46.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f5431b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f542c15a480 0x7f542c15ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59120/0 (socket says 172.21.15.97:59120) 2023-07-06T03:52:46.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f5431b9a700 1 -- 172.21.15.97:0/4100644341 learned_addr learned my addr 172.21.15.97:0/4100644341 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:46.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f5432b9c700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f542c15a850 0x7f542c145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:46.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f543239b700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f542c0b0560 0x7f542c153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:46.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f5431b9a700 1 -- 172.21.15.97:0/4100644341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f542c0b0560 msgr2=0x7f542c153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:46.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f5431b9a700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f542c0b0560 0x7f542c153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f5431b9a700 1 -- 172.21.15.97:0/4100644341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f542c15a850 msgr2=0x7f542c145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:46.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f5431b9a700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f542c15a850 0x7f542c145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.510+0000 7f5431b9a700 1 -- 172.21.15.97:0/4100644341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424009a70 con 0x7f542c15a480 2023-07-06T03:52:46.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.511+0000 7f5431b9a700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f542c15a480 0x7f542c15ac60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c430 tx=0x7f541c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:46.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.511+0000 7f5422ffd700 1 -- 172.21.15.97:0/4100644341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c0052d0 con 0x7f542c15a480 2023-07-06T03:52:46.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.511+0000 7f5422ffd700 1 -- 172.21.15.97:0/4100644341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c005430 con 0x7f542c15a480 2023-07-06T03:52:46.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.511+0000 7f54345ff700 1 -- 172.21.15.97:0/4100644341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c15d690 con 0x7f542c15a480 2023-07-06T03:52:46.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.511+0000 7f5422ffd700 1 -- 172.21.15.97:0/4100644341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c0038c0 con 0x7f542c15a480 2023-07-06T03:52:46.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.511+0000 7f54345ff700 1 -- 172.21.15.97:0/4100644341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c15dc50 con 0x7f542c15a480 2023-07-06T03:52:46.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.512+0000 7f54345ff700 1 -- 172.21.15.97:0/4100644341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c074350 con 0x7f542c15a480 2023-07-06T03:52:46.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.513+0000 7f5422ffd700 1 -- 172.21.15.97:0/4100644341 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f541c003a20 con 0x7f542c15a480 2023-07-06T03:52:46.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.515+0000 7f5422ffd700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541006ace0 0x7f541006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:46.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.515+0000 7f5422ffd700 1 -- 172.21.15.97:0/4100644341 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f541c0887d0 con 0x7f542c15a480 2023-07-06T03:52:46.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.515+0000 7f543239b700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541006ace0 0x7f541006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:46.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.515+0000 7f543239b700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541006ace0 0x7f541006d190 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5428007fd0 tx=0x7f5428007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:46.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.519+0000 7f5422ffd700 1 -- 172.21.15.97:0/4100644341 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f541c0537e0 con 0x7f542c15a480 2023-07-06T03:52:46.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.717+0000 7f54345ff700 1 -- 172.21.15.97:0/4100644341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f542c0cc1f0 con 0x7f542c15a480 2023-07-06T03:52:46.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.718+0000 7f5422ffd700 1 -- 172.21.15.97:0/4100644341 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+18368 (secure 0 0 0) 0x7f541c056e00 con 0x7f542c15a480 2023-07-06T03:52:46.720 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:46.720 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:52:35.953109+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:52:32.448828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-06T03:52:33.428266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:52:46.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 -- 172.21.15.97:0/4100644341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541006ace0 msgr2=0x7f541006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541006ace0 0x7f541006d190 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5428007fd0 tx=0x7f5428007dc0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 -- 172.21.15.97:0/4100644341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f542c15a480 msgr2=0x7f542c15ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f542c15a480 0x7f542c15ac60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c430 tx=0x7f541c00c740 comp rx=0 tx=0).stop 2023-07-06T03:52:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 -- 172.21.15.97:0/4100644341 shutdown_connections 2023-07-06T03:52:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f541006ace0 0x7f541006d190 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f542c15a850 0x7f542c145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f542c0b0560 0x7f542c153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 --2- 172.21.15.97:0/4100644341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f542c15a480 0x7f542c15ac60 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 -- 172.21.15.97:0/4100644341 >> 172.21.15.97:0/4100644341 conn(0x7f542c0cb560 msgr2=0x7f542c0afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 -- 172.21.15.97:0/4100644341 shutdown_connections 2023-07-06T03:52:46.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.721+0000 7f54345ff700 1 -- 172.21.15.97:0/4100644341 wait complete. 2023-07-06T03:52:46.741 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:52:32.448828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:52:33.428266+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:52:46.742 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:52:46.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.940+0000 7f953ddff700 1 Processor -- start 2023-07-06T03:52:46.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.940+0000 7f953ddff700 1 -- start start 2023-07-06T03:52:46.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.941+0000 7f953ddff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95380b1fc0 0x7f95380cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:46.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.941+0000 7f953ddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95380cf080 0x7f95380cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:46.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.941+0000 7f953ddff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953815d610 con 0x7f95380b2390 2023-07-06T03:52:46.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.941+0000 7f953ddff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f953815db40 con 0x7f95380b1fc0 2023-07-06T03:52:46.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.941+0000 7f953ddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953815d860 con 0x7f95380cf080 2023-07-06T03:52:46.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.941+0000 7f95377fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95380b1fc0 0x7f95380cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:46.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.941+0000 7f95377fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95380b1fc0 0x7f95380cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59128/0 (socket says 172.21.15.97:59128) 2023-07-06T03:52:46.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.942+0000 7f95377fe700 1 -- 172.21.15.97:0/2044766042 learned_addr learned my addr 172.21.15.97:0/2044766042 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:46.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.942+0000 7f9537fff700 1 --2- 172.21.15.97:0/2044766042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95380cf080 0x7f95380cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:46.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.942+0000 7f95377fe700 1 -- 172.21.15.97:0/2044766042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95380cf080 msgr2=0x7f95380cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:46.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.942+0000 7f95377fe700 1 --2- 172.21.15.97:0/2044766042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95380cf080 0x7f95380cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.942+0000 7f95377fe700 1 -- 172.21.15.97:0/2044766042 >> v1:172.21.15.97:6789/0 conn(0x7f95380b2390 legacy=0x7f95380b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:46.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.942+0000 7f95377fe700 1 -- 172.21.15.97:0/2044766042 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95380cd630 con 0x7f95380b1fc0 2023-07-06T03:52:46.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.942+0000 7f95367fc700 1 -- 172.21.15.97:0/2044766042 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879447831 0 0) 0x7f953815d610 con 0x7f95380b2390 2023-07-06T03:52:46.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.942+0000 7f95377fe700 1 --2- 172.21.15.97:0/2044766042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380b1fc0 0x7f95380cf8b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f95280098f0 tx=0x7f9528009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab05d085697912b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:46.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.942+0000 7f95367fc700 1 -- 172.21.15.97:0/2044766042 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9528011830 con 0x7f95380b1fc0 2023-07-06T03:52:46.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.942+0000 7f95367fc700 1 -- 172.21.15.97:0/2044766042 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9528011990 con 0x7f95380b1fc0 2023-07-06T03:52:46.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.942+0000 7f95367fc700 1 -- 172.21.15.97:0/2044766042 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9528011d20 con 0x7f95380b1fc0 2023-07-06T03:52:46.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.943+0000 7f953ddff700 1 -- 172.21.15.97:0/2044766042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380b1fc0 msgr2=0x7f95380cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:46.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.943+0000 7f953ddff700 1 --2- 172.21.15.97:0/2044766042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380b1fc0 0x7f95380cf8b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f95280098f0 tx=0x7f9528009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:46.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.943+0000 7f953ddff700 1 -- 172.21.15.97:0/2044766042 shutdown_connections 2023-07-06T03:52:46.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.943+0000 7f953ddff700 1 --2- 172.21.15.97:0/2044766042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380b1fc0 0x7f95380cf8b0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.943+0000 7f953ddff700 1 --2- 172.21.15.97:0/2044766042 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95380cf080 0x7f95380cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.943+0000 7f953ddff700 1 -- 172.21.15.97:0/2044766042 >> 172.21.15.97:0/2044766042 conn(0x7f953806c210 msgr2=0x7f953806c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:46.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.943+0000 7f953ddff700 1 -- 172.21.15.97:0/2044766042 shutdown_connections 2023-07-06T03:52:46.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.943+0000 7f953ddff700 1 -- 172.21.15.97:0/2044766042 wait complete. 2023-07-06T03:52:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.945+0000 7f953ddff700 1 Processor -- start 2023-07-06T03:52:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.945+0000 7f953ddff700 1 -- start start 2023-07-06T03:52:46.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.946+0000 7f953ddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95380b1fc0 0x7f95381f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:46.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.946+0000 7f953ddff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95380b2390 0x7f95381581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:46.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.946+0000 7f953ddff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cf080 0x7f9538159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:46.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.946+0000 7f953ddff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9538155d30 con 0x7f95380cf080 2023-07-06T03:52:46.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.946+0000 7f953ddff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95381568b0 con 0x7f95380b2390 2023-07-06T03:52:46.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.946+0000 7f953ddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9538156d40 con 0x7f95380b1fc0 2023-07-06T03:52:46.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f9537fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cf080 0x7f9538159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:46.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f9537fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cf080 0x7f9538159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59134/0 (socket says 172.21.15.97:59134) 2023-07-06T03:52:46.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f95377fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95380b1fc0 0x7f95381f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:46.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f9537fff700 1 -- 172.21.15.97:0/956147588 learned_addr learned my addr 172.21.15.97:0/956147588 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:46.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f9536ffd700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95380b2390 0x7f95381581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:46.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f9537fff700 1 -- 172.21.15.97:0/956147588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95380b1fc0 msgr2=0x7f95381f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:46.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f9537fff700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95380b1fc0 0x7f95381f28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f9537fff700 1 -- 172.21.15.97:0/956147588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95380b2390 msgr2=0x7f95381581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:46.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f9537fff700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95380b2390 0x7f95381581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f9537fff700 1 -- 172.21.15.97:0/956147588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9528009550 con 0x7f95380cf080 2023-07-06T03:52:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f9536ffd700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95380b2390 0x7f95381581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f9537fff700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cf080 0x7f9538159b10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9530009dd0 tx=0x7f953000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f951ffff700 1 -- 172.21.15.97:0/956147588 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530009c40 con 0x7f95380cf080 2023-07-06T03:52:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f951ffff700 1 -- 172.21.15.97:0/956147588 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953000f040 con 0x7f95380cf080 2023-07-06T03:52:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.947+0000 7f953ddff700 1 -- 172.21.15.97:0/956147588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9538158720 con 0x7f95380cf080 2023-07-06T03:52:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.948+0000 7f951ffff700 1 -- 172.21.15.97:0/956147588 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530014480 con 0x7f95380cf080 2023-07-06T03:52:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.948+0000 7f953ddff700 1 -- 172.21.15.97:0/956147588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9538156260 con 0x7f95380cf080 2023-07-06T03:52:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.949+0000 7f953ddff700 1 -- 172.21.15.97:0/956147588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9538156a40 con 0x7f95380cf080 2023-07-06T03:52:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.950+0000 7f951ffff700 1 -- 172.21.15.97:0/956147588 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f95300145e0 con 0x7f95380cf080 2023-07-06T03:52:46.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.950+0000 7f951ffff700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f952406ace0 0x7f952406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:46.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.951+0000 7f951ffff700 1 -- 172.21.15.97:0/956147588 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9530089d60 con 0x7f95380cf080 2023-07-06T03:52:46.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.951+0000 7f95377fe700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f952406ace0 0x7f952406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:46.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.951+0000 7f95377fe700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f952406ace0 0x7f952406d190 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9528014040 tx=0x7f9528008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:46.955+0000 7f951ffff700 1 -- 172.21.15.97:0/956147588 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9530054e00 con 0x7f95380cf080 2023-07-06T03:52:47.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.149+0000 7f953ddff700 1 -- 172.21.15.97:0/956147588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9538074350 con 0x7f95380cf080 2023-07-06T03:52:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.150+0000 7f951ffff700 1 -- 172.21.15.97:0/956147588 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9530018020 con 0x7f95380cf080 2023-07-06T03:52:47.152 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:52:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 -- 172.21.15.97:0/956147588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f952406ace0 msgr2=0x7f952406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f952406ace0 0x7f952406d190 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9528014040 tx=0x7f9528008030 comp rx=0 tx=0).stop 2023-07-06T03:52:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 -- 172.21.15.97:0/956147588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cf080 msgr2=0x7f9538159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cf080 0x7f9538159b10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9530009dd0 tx=0x7f953000c3a0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 -- 172.21.15.97:0/956147588 shutdown_connections 2023-07-06T03:52:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f952406ace0 0x7f952406d190 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cf080 0x7f9538159b10 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95380b1fc0 0x7f95381f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 --2- 172.21.15.97:0/956147588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95380b2390 0x7f95381581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 -- 172.21.15.97:0/956147588 >> 172.21.15.97:0/956147588 conn(0x7f953806c210 msgr2=0x7f953815cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 -- 172.21.15.97:0/956147588 shutdown_connections 2023-07-06T03:52:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.152+0000 7f953ddff700 1 -- 172.21.15.97:0/956147588 wait complete. 2023-07-06T03:52:47.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:52:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.368+0000 7fb80a3ff700 1 Processor -- start 2023-07-06T03:52:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.368+0000 7fb80a3ff700 1 -- start start 2023-07-06T03:52:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.369+0000 7fb80a3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb8040b2390 0x7fb8040d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.369+0000 7fb80a3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8040cffc0 0x7fb8040d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.369+0000 7fb80a3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80415d610 con 0x7fb8040b1fc0 2023-07-06T03:52:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.369+0000 7fb80a3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80415db40 con 0x7fb8040cffc0 2023-07-06T03:52:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.369+0000 7fb80a3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80415d860 con 0x7fb8040b2390 2023-07-06T03:52:47.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.369+0000 7fb80899c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8040cffc0 0x7fb8040d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:47.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.369+0000 7fb80899c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8040cffc0 0x7fb8040d10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59146/0 (socket says 172.21.15.97:59146) 2023-07-06T03:52:47.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.369+0000 7fb80899c700 1 -- 172.21.15.97:0/1229067733 learned_addr learned my addr 172.21.15.97:0/1229067733 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:47.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.370+0000 7fb8037fe700 1 --2- 172.21.15.97:0/1229067733 >> v2:172.21.15.146:3300/0 conn(0x7fb8040b2390 0x7fb8040d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:47.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.370+0000 7fb80899c700 1 -- 172.21.15.97:0/1229067733 >> v2:172.21.15.146:3300/0 conn(0x7fb8040b2390 msgr2=0x7fb8040d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.370+0000 7fb80899c700 1 --2- 172.21.15.97:0/1229067733 >> v2:172.21.15.146:3300/0 conn(0x7fb8040b2390 0x7fb8040d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.370+0000 7fb80899c700 1 -- 172.21.15.97:0/1229067733 >> v1:172.21.15.146:6789/0 conn(0x7fb8040b1fc0 legacy=0x7fb8040cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.370+0000 7fb80899c700 1 -- 172.21.15.97:0/1229067733 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040cd630 con 0x7fb8040cffc0 2023-07-06T03:52:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.370+0000 7fb8037fe700 1 --2- 172.21.15.97:0/1229067733 >> v2:172.21.15.146:3300/0 conn(0x7fb8040b2390 0x7fb8040d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.370+0000 7fb80899c700 1 --2- 172.21.15.97:0/1229067733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8040cffc0 0x7fb8040d10e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fb8040cd4a0 tx=0x7fb7fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980144de969e2b47 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.370+0000 7fb802ffd700 1 -- 172.21.15.97:0/1229067733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc00e070 con 0x7fb8040cffc0 2023-07-06T03:52:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.370+0000 7fb802ffd700 1 -- 172.21.15.97:0/1229067733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc00b9c0 con 0x7fb8040cffc0 2023-07-06T03:52:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.370+0000 7fb802ffd700 1 -- 172.21.15.97:0/1229067733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc00bd70 con 0x7fb8040cffc0 2023-07-06T03:52:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.371+0000 7fb80a3ff700 1 -- 172.21.15.97:0/1229067733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8040cffc0 msgr2=0x7fb8040d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.371+0000 7fb80a3ff700 1 --2- 172.21.15.97:0/1229067733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8040cffc0 0x7fb8040d10e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fb8040cd4a0 tx=0x7fb7fc00b330 comp rx=0 tx=0).stop 2023-07-06T03:52:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.371+0000 7fb80a3ff700 1 -- 172.21.15.97:0/1229067733 shutdown_connections 2023-07-06T03:52:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.371+0000 7fb80a3ff700 1 --2- 172.21.15.97:0/1229067733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8040cffc0 0x7fb8040d10e0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.371+0000 7fb80a3ff700 1 --2- 172.21.15.97:0/1229067733 >> v2:172.21.15.146:3300/0 conn(0x7fb8040b2390 0x7fb8040d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.371+0000 7fb80a3ff700 1 -- 172.21.15.97:0/1229067733 >> 172.21.15.97:0/1229067733 conn(0x7fb80406c210 msgr2=0x7fb80406c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:47.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.371+0000 7fb80a3ff700 1 -- 172.21.15.97:0/1229067733 shutdown_connections 2023-07-06T03:52:47.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.371+0000 7fb80a3ff700 1 -- 172.21.15.97:0/1229067733 wait complete. 2023-07-06T03:52:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.373+0000 7fb80a3ff700 1 Processor -- start 2023-07-06T03:52:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.373+0000 7fb80a3ff700 1 -- start start 2023-07-06T03:52:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.374+0000 7fb80a3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8040b1fc0 0x7fb804152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.374+0000 7fb80a3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8040b2390 0x7fb8040b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.374+0000 7fb80a3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8040cffc0 0x7fb8040b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.374+0000 7fb80a3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8041f1fe0 con 0x7fb8040b2390 2023-07-06T03:52:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.374+0000 7fb80a3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb804159cd0 con 0x7fb8040cffc0 2023-07-06T03:52:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.374+0000 7fb80a3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb804159fb0 con 0x7fb8040b1fc0 2023-07-06T03:52:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.374+0000 7fb8037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8040b2390 0x7fb8040b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.374+0000 7fb8037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8040b2390 0x7fb8040b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59152/0 (socket says 172.21.15.97:59152) 2023-07-06T03:52:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.374+0000 7fb8037fe700 1 -- 172.21.15.97:0/3371796043 learned_addr learned my addr 172.21.15.97:0/3371796043 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.374+0000 7fb803fff700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8040b1fc0 0x7fb804152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb80899c700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8040cffc0 0x7fb8040b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb8037fe700 1 -- 172.21.15.97:0/3371796043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8040b1fc0 msgr2=0x7fb804152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb8037fe700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8040b1fc0 0x7fb804152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb8037fe700 1 -- 172.21.15.97:0/3371796043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8040cffc0 msgr2=0x7fb8040b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb8037fe700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8040cffc0 0x7fb8040b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb8037fe700 1 -- 172.21.15.97:0/3371796043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc009b30 con 0x7fb8040b2390 2023-07-06T03:52:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb8037fe700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8040b2390 0x7fb8040b4110 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800e7a0 tx=0x7fb7f800eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb800ff9700 1 -- 172.21.15.97:0/3371796043 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8013610 con 0x7fb8040b2390 2023-07-06T03:52:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb800ff9700 1 -- 172.21.15.97:0/3371796043 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f8013770 con 0x7fb8040b2390 2023-07-06T03:52:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb80a3ff700 1 -- 172.21.15.97:0/3371796043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80415a290 con 0x7fb8040b2390 2023-07-06T03:52:47.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb800ff9700 1 -- 172.21.15.97:0/3371796043 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8013b00 con 0x7fb8040b2390 2023-07-06T03:52:47.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.375+0000 7fb80a3ff700 1 -- 172.21.15.97:0/3371796043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80415a570 con 0x7fb8040b2390 2023-07-06T03:52:47.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.376+0000 7fb80a3ff700 1 -- 172.21.15.97:0/3371796043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8041f21b0 con 0x7fb8040b2390 2023-07-06T03:52:47.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.377+0000 7fb800ff9700 1 -- 172.21.15.97:0/3371796043 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7f8013c60 con 0x7fb8040b2390 2023-07-06T03:52:47.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.378+0000 7fb800ff9700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb7f006ace0 0x7fb7f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:47.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.378+0000 7fb800ff9700 1 -- 172.21.15.97:0/3371796043 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7f80891c0 con 0x7fb8040b2390 2023-07-06T03:52:47.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.378+0000 7fb803fff700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb7f006ace0 0x7fb7f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:47.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.379+0000 7fb803fff700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb7f006ace0 0x7fb7f006d190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4000f60 tx=0x7fb7f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:47.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.382+0000 7fb800ff9700 1 -- 172.21.15.97:0/3371796043 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb8041f21b0 con 0x7fb8040b2390 2023-07-06T03:52:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.575+0000 7fb80a3ff700 1 -- 172.21.15.97:0/3371796043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb804074350 con 0x7fb8040b2390 2023-07-06T03:52:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.576+0000 7fb800ff9700 1 -- 172.21.15.97:0/3371796043 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb7f8057c60 con 0x7fb8040b2390 2023-07-06T03:52:47.578 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:52:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 -- 172.21.15.97:0/3371796043 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb7f006ace0 msgr2=0x7fb7f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb7f006ace0 0x7fb7f006d190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4000f60 tx=0x7fb7f4009040 comp rx=0 tx=0).stop 2023-07-06T03:52:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 -- 172.21.15.97:0/3371796043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8040b2390 msgr2=0x7fb8040b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8040b2390 0x7fb8040b4110 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800e7a0 tx=0x7fb7f800eb60 comp rx=0 tx=0).stop 2023-07-06T03:52:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 -- 172.21.15.97:0/3371796043 shutdown_connections 2023-07-06T03:52:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb7f006ace0 0x7fb7f006d190 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8040cffc0 0x7fb8040b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb8040b1fc0 0x7fb804152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 --2- 172.21.15.97:0/3371796043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8040b2390 0x7fb8040b4110 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 -- 172.21.15.97:0/3371796043 >> 172.21.15.97:0/3371796043 conn(0x7fb80406c210 msgr2=0x7fb8040b7580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 -- 172.21.15.97:0/3371796043 shutdown_connections 2023-07-06T03:52:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.578+0000 7fb80a3ff700 1 -- 172.21.15.97:0/3371796043 wait complete. 2023-07-06T03:52:47.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:52:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.793+0000 7f78799ff700 1 Processor -- start 2023-07-06T03:52:47.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.794+0000 7f78799ff700 1 -- start start 2023-07-06T03:52:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.795+0000 7f78799ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 0x7f7874160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.795+0000 7f78799ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78741608c0 con 0x7f78740aef10 2023-07-06T03:52:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.795+0000 7f78799ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78741602b0 con 0x7f78740af2e0 2023-07-06T03:52:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.795+0000 7f78799ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78740c6610 con 0x7f78740b1e20 2023-07-06T03:52:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.795+0000 7f7872ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f78740aef10 0x7f78740c5ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48748/0 (socket says 172.21.15.97:48748) 2023-07-06T03:52:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.795+0000 7f7872ffd700 1 -- 172.21.15.97:0/1713657436 learned_addr learned my addr 172.21.15.97:0/1713657436 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.795+0000 7f78737fe700 1 --2- 172.21.15.97:0/1713657436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 0x7f7874160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.796+0000 7f7871ffb700 1 -- 172.21.15.97:0/1713657436 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44420163 0 0) 0x7f78741608c0 con 0x7f78740aef10 2023-07-06T03:52:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.796+0000 7f7871ffb700 1 -- 172.21.15.97:0/1713657436 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f785c0035b0 con 0x7f78740aef10 2023-07-06T03:52:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.796+0000 7f7871ffb700 1 -- 172.21.15.97:0/1713657436 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229233615 0 0) 0x7f78741602b0 con 0x7f78740af2e0 2023-07-06T03:52:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.796+0000 7f7871ffb700 1 -- 172.21.15.97:0/1713657436 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78741608c0 con 0x7f78740af2e0 2023-07-06T03:52:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.796+0000 7f78737fe700 1 -- 172.21.15.97:0/1713657436 >> v1:172.21.15.146:6789/0 conn(0x7f78740af2e0 legacy=0x7f78740b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.796+0000 7f78737fe700 1 -- 172.21.15.97:0/1713657436 >> v1:172.21.15.97:6789/0 conn(0x7f78740aef10 legacy=0x7f78740c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.796+0000 7f78737fe700 1 -- 172.21.15.97:0/1713657436 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7874160550 con 0x7f78740b1e20 2023-07-06T03:52:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.796+0000 7f78737fe700 1 --2- 172.21.15.97:0/1713657436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 0x7f7874160d50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f786c009dc0 tx=0x7f786c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=743d08d8d52d5934 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.797+0000 7f7871ffb700 1 -- 172.21.15.97:0/1713657436 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00e070 con 0x7f78740b1e20 2023-07-06T03:52:47.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.797+0000 7f7871ffb700 1 -- 172.21.15.97:0/1713657436 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c00bb20 con 0x7f78740b1e20 2023-07-06T03:52:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.797+0000 7f7871ffb700 1 -- 172.21.15.97:0/1713657436 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c012430 con 0x7f78740b1e20 2023-07-06T03:52:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.797+0000 7f78799ff700 1 -- 172.21.15.97:0/1713657436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 msgr2=0x7f7874160d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.797+0000 7f78799ff700 1 --2- 172.21.15.97:0/1713657436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 0x7f7874160d50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f786c009dc0 tx=0x7f786c00b330 comp rx=0 tx=0).stop 2023-07-06T03:52:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.797+0000 7f78799ff700 1 -- 172.21.15.97:0/1713657436 shutdown_connections 2023-07-06T03:52:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.797+0000 7f78799ff700 1 --2- 172.21.15.97:0/1713657436 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 0x7f7874160d50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.797+0000 7f78799ff700 1 -- 172.21.15.97:0/1713657436 >> 172.21.15.97:0/1713657436 conn(0x7f7874066950 msgr2=0x7f78740cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.797+0000 7f78799ff700 1 -- 172.21.15.97:0/1713657436 shutdown_connections 2023-07-06T03:52:47.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.797+0000 7f78799ff700 1 -- 172.21.15.97:0/1713657436 wait complete. 2023-07-06T03:52:47.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.799+0000 7f78799ff700 1 Processor -- start 2023-07-06T03:52:47.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.799+0000 7f78799ff700 1 -- start start 2023-07-06T03:52:47.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.800+0000 7f78799ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78740aef10 0x7f7874158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:47.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.800+0000 7f78799ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740af2e0 0x7f78740ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:47.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.800+0000 7f78799ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 0x7f78740ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:47.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.800+0000 7f78799ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78741f5f50 con 0x7f78740af2e0 2023-07-06T03:52:47.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.800+0000 7f78799ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740b4b70 con 0x7f78740aef10 2023-07-06T03:52:47.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.800+0000 7f78799ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78740b4e50 con 0x7f78740b1e20 2023-07-06T03:52:47.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.800+0000 7f78727fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740af2e0 0x7f78740ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:47.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.800+0000 7f78727fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740af2e0 0x7f78740ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59156/0 (socket says 172.21.15.97:59156) 2023-07-06T03:52:47.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.800+0000 7f78727fc700 1 -- 172.21.15.97:0/1083708971 learned_addr learned my addr 172.21.15.97:0/1083708971 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:47.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f7872ffd700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78740aef10 0x7f7874158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:47.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f78727fc700 1 -- 172.21.15.97:0/1083708971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 msgr2=0x7f78740ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f78737fe700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 0x7f78740ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:47.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f78727fc700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 0x7f78740ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f78727fc700 1 -- 172.21.15.97:0/1083708971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78740aef10 msgr2=0x7f7874158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:47.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f78727fc700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78740aef10 0x7f7874158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:47.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f78727fc700 1 -- 172.21.15.97:0/1083708971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c009a70 con 0x7f78740af2e0 2023-07-06T03:52:47.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f7872ffd700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78740aef10 0x7f7874158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:47.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f78737fe700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 0x7f78740ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:47.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f78727fc700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740af2e0 0x7f78740ad170 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f786800c520 tx=0x7f786800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f785bfff700 1 -- 172.21.15.97:0/1083708971 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680052d0 con 0x7f78740af2e0 2023-07-06T03:52:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f785bfff700 1 -- 172.21.15.97:0/1083708971 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7868005430 con 0x7f78740af2e0 2023-07-06T03:52:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f785bfff700 1 -- 172.21.15.97:0/1083708971 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680038c0 con 0x7f78740af2e0 2023-07-06T03:52:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f78799ff700 1 -- 172.21.15.97:0/1083708971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78740b5130 con 0x7f78740af2e0 2023-07-06T03:52:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.801+0000 7f78799ff700 1 -- 172.21.15.97:0/1083708971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7874158a60 con 0x7f78740af2e0 2023-07-06T03:52:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.802+0000 7f78799ff700 1 -- 172.21.15.97:0/1083708971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78740b5370 con 0x7f78740af2e0 2023-07-06T03:52:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.803+0000 7f785bfff700 1 -- 172.21.15.97:0/1083708971 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7868003a20 con 0x7f78740af2e0 2023-07-06T03:52:47.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.804+0000 7f785bfff700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f786006ace0 0x7f786006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.804+0000 7f785bfff700 1 -- 172.21.15.97:0/1083708971 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7868088a60 con 0x7f78740af2e0 2023-07-06T03:52:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.804+0000 7f7872ffd700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f786006ace0 0x7f786006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.805+0000 7f7872ffd700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f786006ace0 0x7f786006d190 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f78640011e0 tx=0x7f7864009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:47.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:47.809+0000 7f785bfff700 1 -- 172.21.15.97:0/1083708971 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f78740b5370 con 0x7f78740af2e0 2023-07-06T03:52:48.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.005+0000 7f78799ff700 1 -- 172.21.15.97:0/1083708971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78740b5370 con 0x7f78740af2e0 2023-07-06T03:52:48.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.005+0000 7f785bfff700 1 -- 172.21.15.97:0/1083708971 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78740b5370 con 0x7f78740af2e0 2023-07-06T03:52:48.007 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:52:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 -- 172.21.15.97:0/1083708971 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f786006ace0 msgr2=0x7f786006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f786006ace0 0x7f786006d190 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f78640011e0 tx=0x7f7864009040 comp rx=0 tx=0).stop 2023-07-06T03:52:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 -- 172.21.15.97:0/1083708971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740af2e0 msgr2=0x7f78740ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740af2e0 0x7f78740ad170 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f786800c520 tx=0x7f786800c8e0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 -- 172.21.15.97:0/1083708971 shutdown_connections 2023-07-06T03:52:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f786006ace0 0x7f786006d190 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78740b1e20 0x7f78740ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78740aef10 0x7f7874158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 --2- 172.21.15.97:0/1083708971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740af2e0 0x7f78740ad170 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 -- 172.21.15.97:0/1083708971 >> 172.21.15.97:0/1083708971 conn(0x7f7874066950 msgr2=0x7f787406e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 -- 172.21.15.97:0/1083708971 shutdown_connections 2023-07-06T03:52:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.008+0000 7f78799ff700 1 -- 172.21.15.97:0/1083708971 wait complete. 2023-07-06T03:52:48.027 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:52:48.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.223+0000 7f2bfd5ff700 1 Processor -- start 2023-07-06T03:52:48.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.223+0000 7f2bfd5ff700 1 -- start start 2023-07-06T03:52:48.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.224+0000 7f2bfd5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2bf80b1fc0 0x7f2bf80cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.224+0000 7f2bfd5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bf80cf080 0x7f2bf80cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.224+0000 7f2bfd5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf815d610 con 0x7f2bf80b2390 2023-07-06T03:52:48.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.224+0000 7f2bfd5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf815db40 con 0x7f2bf80b1fc0 2023-07-06T03:52:48.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.224+0000 7f2bfd5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf815d860 con 0x7f2bf80cf080 2023-07-06T03:52:48.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.225+0000 7f2bf6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2bf80b1fc0 0x7f2bf80cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:48.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.225+0000 7f2bf67fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2bf80b2390 0x7f2bf80b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:48752/0 (socket says 172.21.15.97:48752) 2023-07-06T03:52:48.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.225+0000 7f2bf67fc700 1 -- 172.21.15.97:0/3335255099 learned_addr learned my addr 172.21.15.97:0/3335255099 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:48.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.225+0000 7f2bf77fe700 1 --2- 172.21.15.97:0/3335255099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bf80cf080 0x7f2bf80cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:48.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.225+0000 7f2bf6ffd700 1 -- 172.21.15.97:0/3335255099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bf80cf080 msgr2=0x7f2bf80cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:48.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.225+0000 7f2bf6ffd700 1 --2- 172.21.15.97:0/3335255099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bf80cf080 0x7f2bf80cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.225+0000 7f2bf6ffd700 1 -- 172.21.15.97:0/3335255099 >> v1:172.21.15.97:6789/0 conn(0x7f2bf80b2390 legacy=0x7f2bf80b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.225+0000 7f2bf6ffd700 1 -- 172.21.15.97:0/3335255099 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf80cd630 con 0x7f2bf80b1fc0 2023-07-06T03:52:48.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.225+0000 7f2bf5ffb700 1 -- 172.21.15.97:0/3335255099 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176635939 0 0) 0x7f2bf815d610 con 0x7f2bf80b2390 2023-07-06T03:52:48.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.225+0000 7f2bf77fe700 1 --2- 172.21.15.97:0/3335255099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bf80cf080 0x7f2bf80cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:48.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.225+0000 7f2bf6ffd700 1 --2- 172.21.15.97:0/3335255099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf80b1fc0 0x7f2bf80cf8b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009840 tx=0x7f2be8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af694fe598599f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:48.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.226+0000 7f2bf5ffb700 1 -- 172.21.15.97:0/3335255099 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8011830 con 0x7f2bf80b1fc0 2023-07-06T03:52:48.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.226+0000 7f2bf5ffb700 1 -- 172.21.15.97:0/3335255099 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be8011990 con 0x7f2bf80b1fc0 2023-07-06T03:52:48.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.226+0000 7f2bf5ffb700 1 -- 172.21.15.97:0/3335255099 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8011d20 con 0x7f2bf80b1fc0 2023-07-06T03:52:48.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.226+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/3335255099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf80b1fc0 msgr2=0x7f2bf80cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.226+0000 7f2bfd5ff700 1 --2- 172.21.15.97:0/3335255099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf80b1fc0 0x7f2bf80cf8b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009840 tx=0x7f2be8009b50 comp rx=0 tx=0).stop 2023-07-06T03:52:48.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.226+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/3335255099 shutdown_connections 2023-07-06T03:52:48.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.226+0000 7f2bfd5ff700 1 --2- 172.21.15.97:0/3335255099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf80b1fc0 0x7f2bf80cf8b0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.226+0000 7f2bfd5ff700 1 --2- 172.21.15.97:0/3335255099 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bf80cf080 0x7f2bf80cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.226+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/3335255099 >> 172.21.15.97:0/3335255099 conn(0x7f2bf806c210 msgr2=0x7f2bf806c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:48.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.226+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/3335255099 shutdown_connections 2023-07-06T03:52:48.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.226+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/3335255099 wait complete. 2023-07-06T03:52:48.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.228+0000 7f2bfd5ff700 1 Processor -- start 2023-07-06T03:52:48.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.228+0000 7f2bfd5ff700 1 -- start start 2023-07-06T03:52:48.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.229+0000 7f2bfd5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bf80b1fc0 0x7f2bf81503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.229+0000 7f2bfd5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bf80b2390 0x7f2bf81e37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.229+0000 7f2bfd5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf80cf080 0x7f2bf80cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.229+0000 7f2bfd5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf8150910 con 0x7f2bf80cf080 2023-07-06T03:52:48.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.229+0000 7f2bfd5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf81e4960 con 0x7f2bf80b2390 2023-07-06T03:52:48.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.229+0000 7f2bfd5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf81e4bb0 con 0x7f2bf80b1fc0 2023-07-06T03:52:48.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.230+0000 7f2bf77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf80cf080 0x7f2bf80cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:48.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.230+0000 7f2bf6ffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bf80b1fc0 0x7f2bf81503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:48.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.230+0000 7f2bf77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf80cf080 0x7f2bf80cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59178/0 (socket says 172.21.15.97:59178) 2023-07-06T03:52:48.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.230+0000 7f2bf77fe700 1 -- 172.21.15.97:0/922795839 learned_addr learned my addr 172.21.15.97:0/922795839 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:48.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.230+0000 7f2bf67fc700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bf80b2390 0x7f2bf81e37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:48.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.230+0000 7f2bf77fe700 1 -- 172.21.15.97:0/922795839 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bf80b1fc0 msgr2=0x7f2bf81503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.230+0000 7f2bf77fe700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bf80b1fc0 0x7f2bf81503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.230+0000 7f2bf77fe700 1 -- 172.21.15.97:0/922795839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bf80b2390 msgr2=0x7f2bf81e37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.230+0000 7f2bf77fe700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bf80b2390 0x7f2bf81e37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.230+0000 7f2bf77fe700 1 -- 172.21.15.97:0/922795839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be80094a0 con 0x7f2bf80cf080 2023-07-06T03:52:48.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.230+0000 7f2bf77fe700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf80cf080 0x7f2bf80cb3b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000cef0 tx=0x7f2bf000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:48.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.231+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/922795839 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf000ad00 con 0x7f2bf80cf080 2023-07-06T03:52:48.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.231+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/922795839 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf000ae60 con 0x7f2bf80cf080 2023-07-06T03:52:48.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.231+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/922795839 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0015680 con 0x7f2bf80cf080 2023-07-06T03:52:48.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.231+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/922795839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf81e4e90 con 0x7f2bf80cf080 2023-07-06T03:52:48.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.231+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/922795839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf81e5450 con 0x7f2bf80cf080 2023-07-06T03:52:48.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.232+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/922795839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf8074350 con 0x7f2bf80cf080 2023-07-06T03:52:48.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.233+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/922795839 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2bf000d430 con 0x7f2bf80cf080 2023-07-06T03:52:48.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.234+0000 7f2bdf7fe700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2be406ace0 0x7f2be406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.234+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/922795839 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2bf00899d0 con 0x7f2bf80cf080 2023-07-06T03:52:48.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.234+0000 7f2bf6ffd700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2be406ace0 0x7f2be406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:48.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.235+0000 7f2bf6ffd700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2be406ace0 0x7f2be406d190 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2be80090b0 tx=0x7f2be801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:48.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.238+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/922795839 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2bf00549e0 con 0x7f2bf80cf080 2023-07-06T03:52:48.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.430+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/922795839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bf806b580 con 0x7f2bf80cf080 2023-07-06T03:52:48.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.431+0000 7f2bdf7fe700 1 -- 172.21.15.97:0/922795839 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bf001a0a0 con 0x7f2bf80cf080 2023-07-06T03:52:48.433 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:52:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/922795839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2be406ace0 msgr2=0x7f2be406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2be406ace0 0x7f2be406d190 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2be80090b0 tx=0x7f2be801c040 comp rx=0 tx=0).stop 2023-07-06T03:52:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/922795839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf80cf080 msgr2=0x7f2bf80cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf80cf080 0x7f2bf80cb3b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000cef0 tx=0x7f2bf000cf20 comp rx=0 tx=0).stop 2023-07-06T03:52:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/922795839 shutdown_connections 2023-07-06T03:52:48.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2be406ace0 0x7f2be406d190 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bf80cf080 0x7f2bf80cb3b0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bf80b1fc0 0x7f2bf81503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 --2- 172.21.15.97:0/922795839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bf80b2390 0x7f2bf81e37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/922795839 >> 172.21.15.97:0/922795839 conn(0x7f2bf806c210 msgr2=0x7f2bf80c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/922795839 shutdown_connections 2023-07-06T03:52:48.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.433+0000 7f2bfd5ff700 1 -- 172.21.15.97:0/922795839 wait complete. 2023-07-06T03:52:48.452 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:52:48.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.645+0000 7f0cce7ff700 1 Processor -- start 2023-07-06T03:52:48.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.646+0000 7f0cce7ff700 1 -- start start 2023-07-06T03:52:48.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cce7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cc80cd670 0x7f0cc80cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cce7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0cc8160190 0x7f0cc806c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cce7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc80cdf80 con 0x7f0cc806c8b0 2023-07-06T03:52:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cce7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc806b2a0 con 0x7f0cc80cd670 2023-07-06T03:52:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cce7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc806b580 con 0x7f0cc8160190 2023-07-06T03:52:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cc7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cc80cd670 0x7f0cc80cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cc7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cc80cd670 0x7f0cc80cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59188/0 (socket says 172.21.15.97:59188) 2023-07-06T03:52:48.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cc7fff700 1 -- 172.21.15.97:0/3542373235 learned_addr learned my addr 172.21.15.97:0/3542373235 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:48.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cc7fff700 1 -- 172.21.15.97:0/3542373235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0cc8160190 msgr2=0x7f0cc806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:48.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cc7fff700 1 --2- 172.21.15.97:0/3542373235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0cc8160190 0x7f0cc806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cc7fff700 1 -- 172.21.15.97:0/3542373235 >> v1:172.21.15.146:6789/0 conn(0x7f0cc806c8b0 legacy=0x7f0cc815cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:48.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.647+0000 7f0cc7fff700 1 -- 172.21.15.97:0/3542373235 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc806b860 con 0x7f0cc80cd670 2023-07-06T03:52:48.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.648+0000 7f0cc7fff700 1 --2- 172.21.15.97:0/3542373235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cc80cd670 0x7f0cc80cda40 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80098f0 tx=0x7f0cb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322a2daa1592787 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:48.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.648+0000 7f0cc6ffd700 1 -- 172.21.15.97:0/3542373235 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8004230 con 0x7f0cc80cd670 2023-07-06T03:52:48.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.648+0000 7f0cc6ffd700 1 -- 172.21.15.97:0/3542373235 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb8004390 con 0x7f0cc80cd670 2023-07-06T03:52:48.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.648+0000 7f0cc6ffd700 1 -- 172.21.15.97:0/3542373235 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb80036a0 con 0x7f0cc80cd670 2023-07-06T03:52:48.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.648+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3542373235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cc80cd670 msgr2=0x7f0cc80cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.648+0000 7f0cce7ff700 1 --2- 172.21.15.97:0/3542373235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cc80cd670 0x7f0cc80cda40 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80098f0 tx=0x7f0cb8009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:48.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.648+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3542373235 shutdown_connections 2023-07-06T03:52:48.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.648+0000 7f0cce7ff700 1 --2- 172.21.15.97:0/3542373235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cc80cd670 0x7f0cc80cda40 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.648+0000 7f0cce7ff700 1 --2- 172.21.15.97:0/3542373235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0cc8160190 0x7f0cc806c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.648+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3542373235 >> 172.21.15.97:0/3542373235 conn(0x7f0cc806e080 msgr2=0x7f0cc806e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:48.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.649+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3542373235 shutdown_connections 2023-07-06T03:52:48.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.649+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3542373235 wait complete. 2023-07-06T03:52:48.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.650+0000 7f0cce7ff700 1 Processor -- start 2023-07-06T03:52:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.650+0000 7f0cce7ff700 1 -- start start 2023-07-06T03:52:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.651+0000 7f0cce7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0cc806c8b0 0x7f0cc815f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.651+0000 7f0cce7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cc80cd670 0x7f0cc815b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.651+0000 7f0cce7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cc8160190 0x7f0cc815b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.651+0000 7f0cce7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80acc90 con 0x7f0cc80cd670 2023-07-06T03:52:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.651+0000 7f0cce7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80acf70 con 0x7f0cc8160190 2023-07-06T03:52:48.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.651+0000 7f0cce7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80ad250 con 0x7f0cc806c8b0 2023-07-06T03:52:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.651+0000 7f0cc77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cc80cd670 0x7f0cc815b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cc80cd670 0x7f0cc815b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59194/0 (socket says 172.21.15.97:59194) 2023-07-06T03:52:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc77fe700 1 -- 172.21.15.97:0/3269317459 learned_addr learned my addr 172.21.15.97:0/3269317459 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc7fff700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0cc806c8b0 0x7f0cc815f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cccd9c700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cc8160190 0x7f0cc815b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc77fe700 1 -- 172.21.15.97:0/3269317459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0cc806c8b0 msgr2=0x7f0cc815f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc77fe700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0cc806c8b0 0x7f0cc815f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc77fe700 1 -- 172.21.15.97:0/3269317459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cc8160190 msgr2=0x7f0cc815b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc77fe700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cc8160190 0x7f0cc815b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc77fe700 1 -- 172.21.15.97:0/3269317459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb8009550 con 0x7f0cc80cd670 2023-07-06T03:52:48.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cccd9c700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cc8160190 0x7f0cc815b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:48.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc77fe700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cc80cd670 0x7f0cc815b420 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00b4a0 tx=0x7f0cbc00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:48.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc4ff9700 1 -- 172.21.15.97:0/3269317459 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc011830 con 0x7f0cc80cd670 2023-07-06T03:52:48.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc4ff9700 1 -- 172.21.15.97:0/3269317459 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc011990 con 0x7f0cc80cd670 2023-07-06T03:52:48.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3269317459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc80ad530 con 0x7f0cc80cd670 2023-07-06T03:52:48.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.652+0000 7f0cc4ff9700 1 -- 172.21.15.97:0/3269317459 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc011d40 con 0x7f0cc80cd670 2023-07-06T03:52:48.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.653+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3269317459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc815d050 con 0x7f0cc80cd670 2023-07-06T03:52:48.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.654+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3269317459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc8074350 con 0x7f0cc80cd670 2023-07-06T03:52:48.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.655+0000 7f0cc4ff9700 1 -- 172.21.15.97:0/3269317459 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0cbc01a400 con 0x7f0cc80cd670 2023-07-06T03:52:48.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.656+0000 7f0cc4ff9700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cb406ace0 0x7f0cb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:48.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.656+0000 7f0cc4ff9700 1 -- 172.21.15.97:0/3269317459 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0cbc0894c0 con 0x7f0cc80cd670 2023-07-06T03:52:48.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.656+0000 7f0cc7fff700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cb406ace0 0x7f0cb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:48.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.656+0000 7f0cc7fff700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cb406ace0 0x7f0cb406d190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009840 tx=0x7f0cb8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:48.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.660+0000 7f0cc4ff9700 1 -- 172.21.15.97:0/3269317459 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0cbc0544e0 con 0x7f0cc80cd670 2023-07-06T03:52:48.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.872+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3269317459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0cc80adb20 con 0x7f0cc80cd670 2023-07-06T03:52:48.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.872+0000 7f0cc4ff9700 1 -- 172.21.15.97:0/3269317459 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v86) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0cbc057b00 con 0x7f0cc80cd670 2023-07-06T03:52:48.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3269317459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cb406ace0 msgr2=0x7f0cb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cb406ace0 0x7f0cb406d190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009840 tx=0x7f0cb8008030 comp rx=0 tx=0).stop 2023-07-06T03:52:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3269317459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cc80cd670 msgr2=0x7f0cc815b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cc80cd670 0x7f0cc815b420 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00b4a0 tx=0x7f0cbc00b7b0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3269317459 shutdown_connections 2023-07-06T03:52:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0cb406ace0 0x7f0cb406d190 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cc8160190 0x7f0cc815b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0cc806c8b0 0x7f0cc815f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 --2- 172.21.15.97:0/3269317459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cc80cd670 0x7f0cc815b420 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3269317459 >> 172.21.15.97:0/3269317459 conn(0x7f0cc806e080 msgr2=0x7f0cc80b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3269317459 shutdown_connections 2023-07-06T03:52:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:48.875+0000 7f0cce7ff700 1 -- 172.21.15.97:0/3269317459 wait complete. 2023-07-06T03:52:48.878 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:52:48.894 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:52:48.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:48.895 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:48.960 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[149203]: fuse finished with error 0 and tester_r 0 2023-07-06T03:52:48.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:52:48.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:48.964 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:52:49.040 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:52:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.273+0000 7fe3a8fff700 1 Processor -- start 2023-07-06T03:52:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.273+0000 7fe3a8fff700 1 -- start start 2023-07-06T03:52:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.274+0000 7fe3a8fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3a415c830 0x7fe3a40b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.274+0000 7fe3a8fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a40d1250 0x7fe3a40afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:49.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a8fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a40b3f90 con 0x7fe3a40b23f0 2023-07-06T03:52:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a8fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a40b3380 con 0x7fe3a415c830 2023-07-06T03:52:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a8fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40b3660 con 0x7fe3a40d1250 2023-07-06T03:52:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3a415c830 0x7fe3a40b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3a415c830 0x7fe3a40b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59202/0 (socket says 172.21.15.97:59202) 2023-07-06T03:52:49.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a259c700 1 -- 172.21.15.97:0/334623916 learned_addr learned my addr 172.21.15.97:0/334623916 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:49.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a2d9d700 1 --2- 172.21.15.97:0/334623916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a40d1250 0x7fe3a40afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:49.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a259c700 1 -- 172.21.15.97:0/334623916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a40d1250 msgr2=0x7fe3a40afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:49.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a259c700 1 --2- 172.21.15.97:0/334623916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a40d1250 0x7fe3a40afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:49.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a259c700 1 -- 172.21.15.97:0/334623916 >> v1:172.21.15.97:6789/0 conn(0x7fe3a40b23f0 legacy=0x7fe3a40d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:49.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a259c700 1 -- 172.21.15.97:0/334623916 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a40b3940 con 0x7fe3a415c830 2023-07-06T03:52:49.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a2d9d700 1 --2- 172.21.15.97:0/334623916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a40d1250 0x7fe3a40afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:49.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a1d9b700 1 -- 172.21.15.97:0/334623916 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323372135 0 0) 0x7fe3a40b3f90 con 0x7fe3a40b23f0 2023-07-06T03:52:49.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.275+0000 7fe3a259c700 1 --2- 172.21.15.97:0/334623916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a415c830 0x7fe3a40b4420 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe3940098f0 tx=0x7fe394009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78d173161aa2457 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:49.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.276+0000 7fe3a1d9b700 1 -- 172.21.15.97:0/334623916 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400b8a0 con 0x7fe3a415c830 2023-07-06T03:52:49.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.276+0000 7fe3a1d9b700 1 -- 172.21.15.97:0/334623916 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39400ba00 con 0x7fe3a415c830 2023-07-06T03:52:49.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.276+0000 7fe3a1d9b700 1 -- 172.21.15.97:0/334623916 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400bd90 con 0x7fe3a415c830 2023-07-06T03:52:49.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.276+0000 7fe3a8fff700 1 -- 172.21.15.97:0/334623916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a415c830 msgr2=0x7fe3a40b4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:49.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.276+0000 7fe3a8fff700 1 --2- 172.21.15.97:0/334623916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a415c830 0x7fe3a40b4420 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe3940098f0 tx=0x7fe394009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:49.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.276+0000 7fe3a8fff700 1 -- 172.21.15.97:0/334623916 shutdown_connections 2023-07-06T03:52:49.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.276+0000 7fe3a8fff700 1 --2- 172.21.15.97:0/334623916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a415c830 0x7fe3a40b4420 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:49.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.276+0000 7fe3a8fff700 1 --2- 172.21.15.97:0/334623916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a40d1250 0x7fe3a40afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:49.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.276+0000 7fe3a8fff700 1 -- 172.21.15.97:0/334623916 >> 172.21.15.97:0/334623916 conn(0x7fe3a409c1e0 msgr2=0x7fe3a40d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:49.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.276+0000 7fe3a8fff700 1 -- 172.21.15.97:0/334623916 shutdown_connections 2023-07-06T03:52:49.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.276+0000 7fe3a8fff700 1 -- 172.21.15.97:0/334623916 wait complete. 2023-07-06T03:52:49.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.278+0000 7fe3a8fff700 1 Processor -- start 2023-07-06T03:52:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.278+0000 7fe3a8fff700 1 -- start start 2023-07-06T03:52:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.279+0000 7fe3a8fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3a40b23f0 0x7fe3a415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.279+0000 7fe3a8fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a40d1250 0x7fe3a4157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.279+0000 7fe3a8fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a415c830 0x7fe3a40cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.279+0000 7fe3a8fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41f2270 con 0x7fe3a415c830 2023-07-06T03:52:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.279+0000 7fe3a8fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41f24d0 con 0x7fe3a40b23f0 2023-07-06T03:52:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.279+0000 7fe3a8fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41f2770 con 0x7fe3a40d1250 2023-07-06T03:52:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.279+0000 7fe3a2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a415c830 0x7fe3a40cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.279+0000 7fe39bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a40d1250 0x7fe3a4157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.279+0000 7fe3a259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3a40b23f0 0x7fe3a415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.279+0000 7fe3a2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a415c830 0x7fe3a40cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59210/0 (socket says 172.21.15.97:59210) 2023-07-06T03:52:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.280+0000 7fe3a2d9d700 1 -- 172.21.15.97:0/2368155526 learned_addr learned my addr 172.21.15.97:0/2368155526 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.280+0000 7fe3a2d9d700 1 -- 172.21.15.97:0/2368155526 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a40d1250 msgr2=0x7fe3a4157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.280+0000 7fe3a2d9d700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a40d1250 0x7fe3a4157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:49.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.280+0000 7fe3a2d9d700 1 -- 172.21.15.97:0/2368155526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3a40b23f0 msgr2=0x7fe3a415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:49.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.280+0000 7fe3a2d9d700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3a40b23f0 0x7fe3a415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:49.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.280+0000 7fe3a2d9d700 1 -- 172.21.15.97:0/2368155526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe394009550 con 0x7fe3a415c830 2023-07-06T03:52:49.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.280+0000 7fe3a2d9d700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a415c830 0x7fe3a40cf7a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00cef0 tx=0x7fe39c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:49.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.280+0000 7fe39affd700 1 -- 172.21.15.97:0/2368155526 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00ad00 con 0x7fe3a415c830 2023-07-06T03:52:49.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.280+0000 7fe39affd700 1 -- 172.21.15.97:0/2368155526 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c00ae60 con 0x7fe3a415c830 2023-07-06T03:52:49.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.280+0000 7fe3a8fff700 1 -- 172.21.15.97:0/2368155526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a41f2a50 con 0x7fe3a415c830 2023-07-06T03:52:49.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.280+0000 7fe39affd700 1 -- 172.21.15.97:0/2368155526 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c015680 con 0x7fe3a415c830 2023-07-06T03:52:49.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.281+0000 7fe3a8fff700 1 -- 172.21.15.97:0/2368155526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a41f3010 con 0x7fe3a415c830 2023-07-06T03:52:49.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.282+0000 7fe3a8fff700 1 -- 172.21.15.97:0/2368155526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a41f58b0 con 0x7fe3a415c830 2023-07-06T03:52:49.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.282+0000 7fe39affd700 1 -- 172.21.15.97:0/2368155526 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe39c00d430 con 0x7fe3a415c830 2023-07-06T03:52:49.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.283+0000 7fe39affd700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe38406ace0 0x7fe38406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:49.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.283+0000 7fe39affd700 1 -- 172.21.15.97:0/2368155526 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe39c0899d0 con 0x7fe3a415c830 2023-07-06T03:52:49.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.284+0000 7fe3a259c700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe38406ace0 0x7fe38406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:49.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.284+0000 7fe3a259c700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe38406ace0 0x7fe38406d190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe394014040 tx=0x7fe394008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:49.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.288+0000 7fe39affd700 1 -- 172.21.15.97:0/2368155526 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe39c0549e0 con 0x7fe3a415c830 2023-07-06T03:52:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.530+0000 7fe3a8fff700 1 -- 172.21.15.97:0/2368155526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe3a4074350 con 0x7fe3a415c830 2023-07-06T03:52:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.530+0000 7fe39affd700 1 -- 172.21.15.97:0/2368155526 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe39c01a070 con 0x7fe3a415c830 2023-07-06T03:52:49.532 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:52:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.532+0000 7fe3a8fff700 1 -- 172.21.15.97:0/2368155526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe38406ace0 msgr2=0x7fe38406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.532+0000 7fe3a8fff700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe38406ace0 0x7fe38406d190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe394014040 tx=0x7fe394008030 comp rx=0 tx=0).stop 2023-07-06T03:52:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.532+0000 7fe3a8fff700 1 -- 172.21.15.97:0/2368155526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a415c830 msgr2=0x7fe3a40cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.532+0000 7fe3a8fff700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a415c830 0x7fe3a40cf7a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00cef0 tx=0x7fe39c00cf20 comp rx=0 tx=0).stop 2023-07-06T03:52:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.532+0000 7fe3a8fff700 1 -- 172.21.15.97:0/2368155526 shutdown_connections 2023-07-06T03:52:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.532+0000 7fe3a8fff700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe38406ace0 0x7fe38406d190 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.532+0000 7fe3a8fff700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a415c830 0x7fe3a40cf7a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.532+0000 7fe3a8fff700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3a40b23f0 0x7fe3a415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.532+0000 7fe3a8fff700 1 --2- 172.21.15.97:0/2368155526 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3a40d1250 0x7fe3a4157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.533+0000 7fe3a8fff700 1 -- 172.21.15.97:0/2368155526 >> 172.21.15.97:0/2368155526 conn(0x7fe3a409c1e0 msgr2=0x7fe3a40afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:49.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.533+0000 7fe3a8fff700 1 -- 172.21.15.97:0/2368155526 shutdown_connections 2023-07-06T03:52:49.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:49.533+0000 7fe3a8fff700 1 -- 172.21.15.97:0/2368155526 wait complete. 2023-07-06T03:52:49.552 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:52:49.552 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:52:49.552 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:52:49.553 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:52:49.553 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:49.553 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:52:49.553 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:52:49.554 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:52:49.554 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:52:49.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:49.555 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:52:49.600 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:52:49.601 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:52:49.601 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:52:49.601 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:49.601 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:52:49.601 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:49.601 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:52:49.602 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:49.602 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:52:49.602 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:52:49.602 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:49.602 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:52:49.602 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:52:49.602 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:49.602 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:52:49.603 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:49.603 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:52:49.603 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41699sec preferred_lft 41699sec 2023-07-06T03:52:49.603 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:52:49.603 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:49.603 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:52:49.603 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:52:49.604 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:52:49.604 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:49.604 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:52:49.604 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:49.604 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:52:49.604 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:52:49.604 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:52:49.604 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:49.605 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:52:49.605 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:52:49.605 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:52:49.605 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:52:49.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:49.606 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:52:49.712 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:52:49.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:49.715 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:52:49.764 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:52:49.765 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:52:49.765 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:52:49.765 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:52:49.765 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:52:49.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:49.766 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:52:49.767 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:52:49.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:52:49.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:52:49.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:52:49.768 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:52:49.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:49.935 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:52:49.982 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:52:49.982 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:52:49.983 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:52:49.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:49.984 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:52:50.053 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:52:50.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:52:50.054 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:50.099 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:52:50.099 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:52:50.099 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:50.184 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:52:50.293 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:52:50.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:52:50.360 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:52:50.362 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:52:50.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:50.364 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:52:50.409 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:52:50.409 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:52:50.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:52:50.527 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:52:50.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:52:50.597 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:52:50.599 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:52:50.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:50.601 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:52:50.622 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:52:50.620+0000 7fd7b1e405c0 -1 init, newargv = 0x5600d2784370 newargc=15 2023-07-06T03:52:50.622 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[150207]: starting ceph client 2023-07-06T03:52:50.628 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[150207]: starting fuse 2023-07-06T03:52:50.650 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:52:50.651 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:52:50.651 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:52:50.652 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:52:50.652 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:52:50.652 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:52:50.653 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:52:50.653 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:50.653 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:52:50.654 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:52:50.654 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:52:50.654 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:52:50.655 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:50.655 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:52:50.655 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:52:50.656 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:52:50.656 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:52:50.657 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:50.657 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:52:50.657 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:52:50.658 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:52:50.658 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:52:50.658 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:52:50.658 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:52:50.659 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:52:50.659 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:52:50.659 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:50.660 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:52:50.660 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:52:50.833 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.150207.asok 2023-07-06T03:52:51.011 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.150207.asok 2023-07-06T03:52:51.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:51.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.150207.asok status 2023-07-06T03:52:51.234 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:52:51.234 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:52:51.234 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:52:51.235 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:52:51.235 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:52:51.235 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:52:51.235 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:52:51.235 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "150207", 2023-07-06T03:52:51.235 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:52:51.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:51.235 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:52:51.236 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:52:51.236 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7725, 2023-07-06T03:52:51.236 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:52:51.236 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:52:51.236 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:52:51.236 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 7725 2023-07-06T03:52:51.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:51.237 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:52:51.237 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:52:51.237 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:52:51.237 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3455588533 2023-07-06T03:52:51.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:52:51.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:51.237 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:52:51.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:52:51.238 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:52:51.238 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3455588533 2023-07-06T03:52:51.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:52:51.238 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.7725 192.168.0.1:0/3455588533", 2023-07-06T03:52:51.238 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/3455588533", 2023-07-06T03:52:51.238 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:52:51.238 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 56, 2023-07-06T03:52:51.239 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 86, 2023-07-06T03:52:51.239 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:52:51.239 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:52:51.239 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:52:51.240 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:52:51.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:51.249 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:52:51.249 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:51.295 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:52:51.296 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:51.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:52:51.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:52:51.364 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:52:51.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.556+0000 7ff9437ff700 1 Processor -- start 2023-07-06T03:52:51.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.556+0000 7ff9437ff700 1 -- start start 2023-07-06T03:52:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.557+0000 7ff9437ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff93c15cce0 0x7ff93c0c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.557+0000 7ff9437ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93c0ce370 con 0x7ff93c0cb0b0 2023-07-06T03:52:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.557+0000 7ff9437ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93c070170 con 0x7ff93c0cace0 2023-07-06T03:52:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.557+0000 7ff9437ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93c0ce5c0 con 0x7ff93c15cce0 2023-07-06T03:52:51.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.557+0000 7ff940d9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff93c15cce0 0x7ff93c0c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:51.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.558+0000 7ff941d9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff93c0cb0b0 0x7ff93c0cd6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49018/0 (socket says 172.21.15.97:49018) 2023-07-06T03:52:51.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.558+0000 7ff940d9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff93c15cce0 0x7ff93c0c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33602/0 (socket says 172.21.15.97:33602) 2023-07-06T03:52:51.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.558+0000 7ff941d9c700 1 -- 172.21.15.97:0/3207886195 learned_addr learned my addr 172.21.15.97:0/3207886195 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:52:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.558+0000 7ff940d9a700 1 -- 172.21.15.97:0/3207886195 >> v1:172.21.15.146:6789/0 conn(0x7ff93c0cace0 legacy=0x7ff93c15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.558+0000 7ff940d9a700 1 -- 172.21.15.97:0/3207886195 >> v1:172.21.15.97:6789/0 conn(0x7ff93c0cb0b0 legacy=0x7ff93c0cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.558+0000 7ff940d9a700 1 -- 172.21.15.97:0/3207886195 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c0ce8a0 con 0x7ff93c15cce0 2023-07-06T03:52:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.558+0000 7ff933fff700 1 -- 172.21.15.97:0/3207886195 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283413286 0 0) 0x7ff93c0ce370 con 0x7ff93c0cb0b0 2023-07-06T03:52:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.558+0000 7ff940d9a700 1 --2- 172.21.15.97:0/3207886195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff93c15cce0 0x7ff93c0c9130 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00a9e0 tx=0x7ff92c00acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1faec86d99169bf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.558+0000 7ff933fff700 1 -- 172.21.15.97:0/3207886195 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c00cb70 con 0x7ff93c15cce0 2023-07-06T03:52:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.558+0000 7ff933fff700 1 -- 172.21.15.97:0/3207886195 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c00ccd0 con 0x7ff93c15cce0 2023-07-06T03:52:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.558+0000 7ff933fff700 1 -- 172.21.15.97:0/3207886195 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c018620 con 0x7ff93c15cce0 2023-07-06T03:52:51.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.559+0000 7ff9437ff700 1 -- 172.21.15.97:0/3207886195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff93c15cce0 msgr2=0x7ff93c0c9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.559+0000 7ff9437ff700 1 --2- 172.21.15.97:0/3207886195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff93c15cce0 0x7ff93c0c9130 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00a9e0 tx=0x7ff92c00acf0 comp rx=0 tx=0).stop 2023-07-06T03:52:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.559+0000 7ff9437ff700 1 -- 172.21.15.97:0/3207886195 shutdown_connections 2023-07-06T03:52:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.559+0000 7ff9437ff700 1 --2- 172.21.15.97:0/3207886195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff93c15cce0 0x7ff93c0c9130 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.559+0000 7ff9437ff700 1 -- 172.21.15.97:0/3207886195 >> 172.21.15.97:0/3207886195 conn(0x7ff93c0cb560 msgr2=0x7ff93c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.559+0000 7ff9437ff700 1 -- 172.21.15.97:0/3207886195 shutdown_connections 2023-07-06T03:52:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.559+0000 7ff9437ff700 1 -- 172.21.15.97:0/3207886195 wait complete. 2023-07-06T03:52:51.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.561+0000 7ff9437ff700 1 Processor -- start 2023-07-06T03:52:51.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.561+0000 7ff9437ff700 1 -- start start 2023-07-06T03:52:51.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.562+0000 7ff9437ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff93c0cace0 0x7ff93c153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:51.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.562+0000 7ff9437ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff93c0cb0b0 0x7ff93c0c2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.562+0000 7ff9437ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff93c15cce0 0x7ff93c0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.562+0000 7ff9437ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0c4ff0 con 0x7ff93c0cace0 2023-07-06T03:52:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.562+0000 7ff9437ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0c52d0 con 0x7ff93c0cb0b0 2023-07-06T03:52:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.562+0000 7ff9437ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0c31c0 con 0x7ff93c15cce0 2023-07-06T03:52:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.562+0000 7ff94159b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff93c0cace0 0x7ff93c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.562+0000 7ff94159b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff93c0cace0 0x7ff93c153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33614/0 (socket says 172.21.15.97:33614) 2023-07-06T03:52:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.562+0000 7ff94159b700 1 -- 172.21.15.97:0/3444336901 learned_addr learned my addr 172.21.15.97:0/3444336901 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.562+0000 7ff940d9a700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff93c0cb0b0 0x7ff93c0c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:51.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.562+0000 7ff941d9c700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff93c15cce0 0x7ff93c0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:51.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff94159b700 1 -- 172.21.15.97:0/3444336901 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff93c15cce0 msgr2=0x7ff93c0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:51.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff94159b700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff93c15cce0 0x7ff93c0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:51.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff94159b700 1 -- 172.21.15.97:0/3444336901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff93c0cb0b0 msgr2=0x7ff93c0c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:51.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff94159b700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff93c0cb0b0 0x7ff93c0c2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:51.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff94159b700 1 -- 172.21.15.97:0/3444336901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c00a640 con 0x7ff93c0cace0 2023-07-06T03:52:51.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff941d9c700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff93c15cce0 0x7ff93c0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:52:51.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff940d9a700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff93c0cb0b0 0x7ff93c0c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:52:51.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff94159b700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff93c0cace0 0x7ff93c153710 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff93800c450 tx=0x7ff93800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:51.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff931ffb700 1 -- 172.21.15.97:0/3444336901 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9380052d0 con 0x7ff93c0cace0 2023-07-06T03:52:51.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff931ffb700 1 -- 172.21.15.97:0/3444336901 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff938005430 con 0x7ff93c0cace0 2023-07-06T03:52:51.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff931ffb700 1 -- 172.21.15.97:0/3444336901 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9380038c0 con 0x7ff93c0cace0 2023-07-06T03:52:51.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff9437ff700 1 -- 172.21.15.97:0/3444336901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93c0c2b50 con 0x7ff93c0cace0 2023-07-06T03:52:51.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.563+0000 7ff9437ff700 1 -- 172.21.15.97:0/3444336901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c0c36e0 con 0x7ff93c0cace0 2023-07-06T03:52:51.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.564+0000 7ff9437ff700 1 -- 172.21.15.97:0/3444336901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93c070730 con 0x7ff93c0cace0 2023-07-06T03:52:51.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.565+0000 7ff931ffb700 1 -- 172.21.15.97:0/3444336901 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff938003a20 con 0x7ff93c0cace0 2023-07-06T03:52:51.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.566+0000 7ff931ffb700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff92006ace0 0x7ff92006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:51.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.566+0000 7ff931ffb700 1 -- 172.21.15.97:0/3444336901 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff938088830 con 0x7ff93c0cace0 2023-07-06T03:52:51.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.566+0000 7ff940d9a700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff92006ace0 0x7ff92006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:51.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.567+0000 7ff940d9a700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff92006ace0 0x7ff92006d190 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00a1c0 tx=0x7ff92c01a040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:51.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.571+0000 7ff931ffb700 1 -- 172.21.15.97:0/3444336901 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff9380538d0 con 0x7ff93c0cace0 2023-07-06T03:52:51.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.807+0000 7ff9437ff700 1 -- 172.21.15.97:0/3444336901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff93c0c5130 con 0x7ff93c0cace0 2023-07-06T03:52:51.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.808+0000 7ff931ffb700 1 -- 172.21.15.97:0/3444336901 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4679 (secure 0 0 0) 0x7ff93c0c5130 con 0x7ff93c0cace0 2023-07-06T03:52:51.810 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:51.810 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3945452207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3945452207},{"type":"v1","addr":"172.21.15.146:6835","nonce":3945452207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2428591387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2428591387},{"type":"v1","addr":"172.21.15.97:6837","nonce":2428591387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/2089126027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2089126027},{"type":"v1","addr":"172.21.15.146:6834","nonce":2089126027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:52:34.447330+0000","modified":"2023-07-06T03:52:37.985332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7497":{"gid":7497,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/2046478602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2046478602},{"type":"v1","addr":"172.21.15.97:6836","nonce":2046478602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-06T03:52:51.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 -- 172.21.15.97:0/3444336901 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff92006ace0 msgr2=0x7ff92006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:51.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff92006ace0 0x7ff92006d190 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00a1c0 tx=0x7ff92c01a040 comp rx=0 tx=0).stop 2023-07-06T03:52:51.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 -- 172.21.15.97:0/3444336901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff93c0cace0 msgr2=0x7ff93c153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:51.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff93c0cace0 0x7ff93c153710 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff93800c450 tx=0x7ff93800c810 comp rx=0 tx=0).stop 2023-07-06T03:52:51.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 -- 172.21.15.97:0/3444336901 shutdown_connections 2023-07-06T03:52:51.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff92006ace0 0x7ff92006d190 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:51.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff93c15cce0 0x7ff93c0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:51.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff93c0cace0 0x7ff93c153710 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:51.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 --2- 172.21.15.97:0/3444336901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff93c0cb0b0 0x7ff93c0c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:51.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 -- 172.21.15.97:0/3444336901 >> 172.21.15.97:0/3444336901 conn(0x7ff93c0cb560 msgr2=0x7ff93c15c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:51.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 -- 172.21.15.97:0/3444336901 shutdown_connections 2023-07-06T03:52:51.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:51.810+0000 7ff9437ff700 1 -- 172.21.15.97:0/3444336901 wait complete. 2023-07-06T03:52:51.814 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 56 2023-07-06T03:52:51.830 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_max_completed_requests 1000 2023-07-06T03:52:52.045 INFO:teuthology.orchestra.run.smithi097.stdout:{"success":"mds_max_completed_requests = '1000' (not observed, change may require restart) "} 2023-07-06T03:52:52.045 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_max_completed_requests = '1000' (not observed, change may require restart) " } 2023-07-06T03:52:52.045 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:52:52.046 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:52:52.091 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:52:52.092 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:52:52.092 DEBUG:teuthology.orchestra.run.smithi097:> import uuid 2023-07-06T03:52:52.092 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:52.093 DEBUG:teuthology.orchestra.run.smithi097:> n = 1100 2023-07-06T03:52:52.093 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = False 2023-07-06T03:52:52.093 DEBUG:teuthology.orchestra.run.smithi097:> if 0 > 0: 2023-07-06T03:52:52.093 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = True 2023-07-06T03:52:52.094 DEBUG:teuthology.orchestra.run.smithi097:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file1" 2023-07-06T03:52:52.096 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:52.096 DEBUG:teuthology.orchestra.run.smithi097:> dpath = os.path.dirname(path) 2023-07-06T03:52:52.096 DEBUG:teuthology.orchestra.run.smithi097:> fnameprefix = os.path.basename(path) 2023-07-06T03:52:52.097 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(dpath, exist_ok=True) 2023-07-06T03:52:52.097 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:52.098 DEBUG:teuthology.orchestra.run.smithi097:> try: 2023-07-06T03:52:52.098 DEBUG:teuthology.orchestra.run.smithi097:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-06T03:52:52.098 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:52.099 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(n): 2023-07-06T03:52:52.099 DEBUG:teuthology.orchestra.run.smithi097:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-06T03:52:52.099 DEBUG:teuthology.orchestra.run.smithi097:> with open(fpath, '"'"'w'"'"') as f: 2023-07-06T03:52:52.099 DEBUG:teuthology.orchestra.run.smithi097:> f.write(f"{i}") 2023-07-06T03:52:52.100 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:52:52.100 DEBUG:teuthology.orchestra.run.smithi097:> f.flush() 2023-07-06T03:52:52.100 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(f.fileno()) 2023-07-06T03:52:52.101 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:52:52.101 DEBUG:teuthology.orchestra.run.smithi097:> os.unlink(fpath) 2023-07-06T03:52:52.101 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:52:52.102 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:52:52.102 DEBUG:teuthology.orchestra.run.smithi097:> if create_hard_links: 2023-07-06T03:52:52.102 DEBUG:teuthology.orchestra.run.smithi097:> for j in range(0): 2023-07-06T03:52:52.102 DEBUG:teuthology.orchestra.run.smithi097:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-06T03:52:52.103 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:52:52.103 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:52:52.103 DEBUG:teuthology.orchestra.run.smithi097:> finally: 2023-07-06T03:52:52.104 DEBUG:teuthology.orchestra.run.smithi097:> os.close(dirfd) 2023-07-06T03:52:52.104 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:52:55.232 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:52:55.233 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:52:55.318 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:52:55.319 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:52:55.319 DEBUG:teuthology.orchestra.run.smithi097:> import uuid 2023-07-06T03:52:55.319 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:55.319 DEBUG:teuthology.orchestra.run.smithi097:> n = 5 2023-07-06T03:52:55.320 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = False 2023-07-06T03:52:55.320 DEBUG:teuthology.orchestra.run.smithi097:> if 0 > 0: 2023-07-06T03:52:55.320 DEBUG:teuthology.orchestra.run.smithi097:> create_hard_links = True 2023-07-06T03:52:55.321 DEBUG:teuthology.orchestra.run.smithi097:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2023-07-06T03:52:55.321 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:55.321 DEBUG:teuthology.orchestra.run.smithi097:> dpath = os.path.dirname(path) 2023-07-06T03:52:55.321 DEBUG:teuthology.orchestra.run.smithi097:> fnameprefix = os.path.basename(path) 2023-07-06T03:52:55.322 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(dpath, exist_ok=True) 2023-07-06T03:52:55.322 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:55.322 DEBUG:teuthology.orchestra.run.smithi097:> try: 2023-07-06T03:52:55.322 DEBUG:teuthology.orchestra.run.smithi097:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-06T03:52:55.323 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:52:55.323 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(n): 2023-07-06T03:52:55.323 DEBUG:teuthology.orchestra.run.smithi097:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-06T03:52:55.324 DEBUG:teuthology.orchestra.run.smithi097:> with open(fpath, '"'"'w'"'"') as f: 2023-07-06T03:52:55.324 DEBUG:teuthology.orchestra.run.smithi097:> f.write(f"{i}") 2023-07-06T03:52:55.324 DEBUG:teuthology.orchestra.run.smithi097:> if True: 2023-07-06T03:52:55.324 DEBUG:teuthology.orchestra.run.smithi097:> f.flush() 2023-07-06T03:52:55.325 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(f.fileno()) 2023-07-06T03:52:55.325 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:52:55.325 DEBUG:teuthology.orchestra.run.smithi097:> os.unlink(fpath) 2023-07-06T03:52:55.325 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:52:55.326 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:52:55.326 DEBUG:teuthology.orchestra.run.smithi097:> if create_hard_links: 2023-07-06T03:52:55.326 DEBUG:teuthology.orchestra.run.smithi097:> for j in range(0): 2023-07-06T03:52:55.326 DEBUG:teuthology.orchestra.run.smithi097:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-06T03:52:55.327 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2023-07-06T03:52:55.327 DEBUG:teuthology.orchestra.run.smithi097:> os.fsync(dirfd) 2023-07-06T03:52:55.327 DEBUG:teuthology.orchestra.run.smithi097:> finally: 2023-07-06T03:52:55.328 DEBUG:teuthology.orchestra.run.smithi097:> os.close(dirfd) 2023-07-06T03:52:55.328 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:52:55.655 INFO:tasks.ceph_test_case:waiting 100s for health warning matching MDS_CLIENT_OLDEST_TID 2023-07-06T03:52:55.655 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-07-06T03:52:55.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.889+0000 7fa4a6dff700 1 Processor -- start 2023-07-06T03:52:55.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.889+0000 7fa4a6dff700 1 -- start start 2023-07-06T03:52:55.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.890+0000 7fa4a6dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa4a00d11c0 0x7fa4a00b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:55.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.890+0000 7fa4a6dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4a00b44d0 0x7fa4a00b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:55.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.890+0000 7fa4a6dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a00afdd0 con 0x7fa4a015c960 2023-07-06T03:52:55.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.890+0000 7fa4a6dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a00b00b0 con 0x7fa4a00d11c0 2023-07-06T03:52:55.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.890+0000 7fa4a6dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a00b0390 con 0x7fa4a00b44d0 2023-07-06T03:52:55.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.891+0000 7fa49ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa4a00d11c0 0x7fa4a00b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:55.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.891+0000 7fa49ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa4a00d11c0 0x7fa4a00b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33622/0 (socket says 172.21.15.97:33622) 2023-07-06T03:52:55.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.891+0000 7fa49ffff700 1 -- 172.21.15.97:0/3643494634 learned_addr learned my addr 172.21.15.97:0/3643494634 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:55.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.891+0000 7fa4a539c700 1 --2- 172.21.15.97:0/3643494634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4a00b44d0 0x7fa4a00b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:55.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.891+0000 7fa49ffff700 1 -- 172.21.15.97:0/3643494634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4a00b44d0 msgr2=0x7fa4a00b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:52:55.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.891+0000 7fa49ffff700 1 --2- 172.21.15.97:0/3643494634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4a00b44d0 0x7fa4a00b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:55.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.891+0000 7fa49ffff700 1 -- 172.21.15.97:0/3643494634 >> v1:172.21.15.146:6789/0 conn(0x7fa4a015c960 legacy=0x7fa4a00d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:55.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.891+0000 7fa49ffff700 1 -- 172.21.15.97:0/3643494634 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a00b0670 con 0x7fa4a00d11c0 2023-07-06T03:52:55.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.891+0000 7fa49ffff700 1 --2- 172.21.15.97:0/3643494634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a00d11c0 0x7fa4a00b0e70 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0098f0 tx=0x7fa48c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc367bc9891b571e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:55.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.892+0000 7fa49f7fe700 1 -- 172.21.15.97:0/3643494634 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c011830 con 0x7fa4a00d11c0 2023-07-06T03:52:55.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.892+0000 7fa49f7fe700 1 -- 172.21.15.97:0/3643494634 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c011990 con 0x7fa4a00d11c0 2023-07-06T03:52:55.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.892+0000 7fa49f7fe700 1 -- 172.21.15.97:0/3643494634 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c011d20 con 0x7fa4a00d11c0 2023-07-06T03:52:55.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.892+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3643494634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a00d11c0 msgr2=0x7fa4a00b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:55.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.892+0000 7fa4a6dff700 1 --2- 172.21.15.97:0/3643494634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a00d11c0 0x7fa4a00b0e70 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0098f0 tx=0x7fa48c009c00 comp rx=0 tx=0).stop 2023-07-06T03:52:55.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.892+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3643494634 shutdown_connections 2023-07-06T03:52:55.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.892+0000 7fa4a6dff700 1 --2- 172.21.15.97:0/3643494634 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4a00b44d0 0x7fa4a00b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:55.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.892+0000 7fa4a6dff700 1 --2- 172.21.15.97:0/3643494634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a00d11c0 0x7fa4a00b0e70 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:55.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.892+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3643494634 >> 172.21.15.97:0/3643494634 conn(0x7fa4a009c1b0 msgr2=0x7fa4a00d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:55.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.892+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3643494634 shutdown_connections 2023-07-06T03:52:55.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.892+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3643494634 wait complete. 2023-07-06T03:52:55.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.894+0000 7fa4a6dff700 1 Processor -- start 2023-07-06T03:52:55.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.894+0000 7fa4a6dff700 1 -- start start 2023-07-06T03:52:55.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.895+0000 7fa4a6dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4a00b44d0 0x7fa4a00ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:55.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.895+0000 7fa4a6dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a00d11c0 0x7fa4a01598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:55.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.895+0000 7fa4a6dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4a015c960 0x7fa4a0159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:55.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.895+0000 7fa4a6dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a00b7850 con 0x7fa4a00d11c0 2023-07-06T03:52:55.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.895+0000 7fa4a6dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a0156c60 con 0x7fa4a015c960 2023-07-06T03:52:55.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.895+0000 7fa4a6dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a00c7680 con 0x7fa4a00b44d0 2023-07-06T03:52:55.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.895+0000 7fa49ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a00d11c0 0x7fa4a01598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:55.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.895+0000 7fa4a4b9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4a00b44d0 0x7fa4a00ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:55.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.895+0000 7fa49ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a00d11c0 0x7fa4a01598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33628/0 (socket says 172.21.15.97:33628) 2023-07-06T03:52:55.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.895+0000 7fa49ffff700 1 -- 172.21.15.97:0/3329045064 learned_addr learned my addr 172.21.15.97:0/3329045064 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:52:55.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.895+0000 7fa4a539c700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4a015c960 0x7fa4a0159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:55.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.896+0000 7fa49ffff700 1 -- 172.21.15.97:0/3329045064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4a00b44d0 msgr2=0x7fa4a00ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:55.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.896+0000 7fa49ffff700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4a00b44d0 0x7fa4a00ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:55.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.896+0000 7fa49ffff700 1 -- 172.21.15.97:0/3329045064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4a015c960 msgr2=0x7fa4a0159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:55.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.896+0000 7fa49ffff700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4a015c960 0x7fa4a0159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:55.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.896+0000 7fa49ffff700 1 -- 172.21.15.97:0/3329045064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c009550 con 0x7fa4a00d11c0 2023-07-06T03:52:55.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.896+0000 7fa49ffff700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a00d11c0 0x7fa4a01598a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa48c000c00 tx=0x7fa48c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:55.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.896+0000 7fa49d7fa700 1 -- 172.21.15.97:0/3329045064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c004400 con 0x7fa4a00d11c0 2023-07-06T03:52:55.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.896+0000 7fa49d7fa700 1 -- 172.21.15.97:0/3329045064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c01f070 con 0x7fa4a00d11c0 2023-07-06T03:52:55.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.896+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3329045064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a00b6960 con 0x7fa4a00d11c0 2023-07-06T03:52:55.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.896+0000 7fa49d7fa700 1 -- 172.21.15.97:0/3329045064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c0038e0 con 0x7fa4a00d11c0 2023-07-06T03:52:55.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.896+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3329045064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a00b7470 con 0x7fa4a00d11c0 2023-07-06T03:52:55.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.897+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3329045064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a0074350 con 0x7fa4a00d11c0 2023-07-06T03:52:55.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.898+0000 7fa49d7fa700 1 -- 172.21.15.97:0/3329045064 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa48c021030 con 0x7fa4a00d11c0 2023-07-06T03:52:55.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.900+0000 7fa49d7fa700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa48406ace0 0x7fa48406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:52:55.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.900+0000 7fa49d7fa700 1 -- 172.21.15.97:0/3329045064 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fa48c08a990 con 0x7fa4a00d11c0 2023-07-06T03:52:55.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.900+0000 7fa4a4b9b700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa48406ace0 0x7fa48406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:52:55.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.900+0000 7fa4a4b9b700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa48406ace0 0x7fa48406d190 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa4a015a590 tx=0x7fa494009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:52:55.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:55.904+0000 7fa49d7fa700 1 -- 172.21.15.97:0/3329045064 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa48c055a00 con 0x7fa4a00d11c0 2023-07-06T03:52:56.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.179+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3329045064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7fa4a006bbb0 con 0x7fa4a00d11c0 2023-07-06T03:52:56.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.179+0000 7fa49d7fa700 1 -- 172.21.15.97:0/3329045064 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+46 (secure 0 0 0) 0x7fa48c0150a0 con 0x7fa4a00d11c0 2023-07-06T03:52:56.182 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:52:56.182 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:52:56.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3329045064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa48406ace0 msgr2=0x7fa48406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:56.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa48406ace0 0x7fa48406d190 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa4a015a590 tx=0x7fa494009040 comp rx=0 tx=0).stop 2023-07-06T03:52:56.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3329045064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a00d11c0 msgr2=0x7fa4a01598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:52:56.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a00d11c0 0x7fa4a01598a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa48c000c00 tx=0x7fa48c004750 comp rx=0 tx=0).stop 2023-07-06T03:52:56.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3329045064 shutdown_connections 2023-07-06T03:52:56.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa48406ace0 0x7fa48406d190 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:56.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4a015c960 0x7fa4a0159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:56.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa4a00b44d0 0x7fa4a00ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:56.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 --2- 172.21.15.97:0/3329045064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a00d11c0 0x7fa4a01598a0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:52:56.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3329045064 >> 172.21.15.97:0/3329045064 conn(0x7fa4a009c1b0 msgr2=0x7fa4a006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:52:56.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3329045064 shutdown_connections 2023-07-06T03:52:56.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:52:56.182+0000 7fa4a6dff700 1 -- 172.21.15.97:0/3329045064 wait complete. 2023-07-06T03:52:56.202 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:52:56.202 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-07-06T03:53:01.203 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-07-06T03:53:01.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.397+0000 7ff480dff700 1 Processor -- start 2023-07-06T03:53:01.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.397+0000 7ff480dff700 1 -- start start 2023-07-06T03:53:01.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.398+0000 7ff480dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff47c15a480 0x7ff47c071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.398+0000 7ff480dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff47c15a850 0x7ff47c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.398+0000 7ff480dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47c072070 con 0x7ff47c0adc60 2023-07-06T03:53:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.398+0000 7ff480dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47c0ae1e0 con 0x7ff47c15a480 2023-07-06T03:53:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff480dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c071050 con 0x7ff47c15a850 2023-07-06T03:53:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff47ad9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff47c0adc60 0x7ff47c0b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45572/0 (socket says 172.21.15.97:45572) 2023-07-06T03:53:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff47ad9d700 1 -- 172.21.15.97:0/3905931208 learned_addr learned my addr 172.21.15.97:0/3905931208 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff47a59c700 1 --2- 172.21.15.97:0/3905931208 >> v2:172.21.15.146:3300/0 conn(0x7ff47c15a480 0x7ff47c071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff473fff700 1 --2- 172.21.15.97:0/3905931208 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff47c15a850 0x7ff47c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff479d9b700 1 -- 172.21.15.97:0/3905931208 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793451651 0 0) 0x7ff47c072070 con 0x7ff47c0adc60 2023-07-06T03:53:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff479d9b700 1 -- 172.21.15.97:0/3905931208 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4680035b0 con 0x7ff47c0adc60 2023-07-06T03:53:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff47a59c700 1 -- 172.21.15.97:0/3905931208 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff47c15a850 msgr2=0x7ff47c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff47a59c700 1 --2- 172.21.15.97:0/3905931208 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff47c15a850 0x7ff47c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff47a59c700 1 -- 172.21.15.97:0/3905931208 >> v1:172.21.15.97:6789/0 conn(0x7ff47c0adc60 legacy=0x7ff47c0b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff479d9b700 1 -- 172.21.15.97:0/3905931208 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3037580317 0 0) 0x7ff4680035b0 con 0x7ff47c0adc60 2023-07-06T03:53:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff47a59c700 1 -- 172.21.15.97:0/3905931208 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c071330 con 0x7ff47c15a480 2023-07-06T03:53:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.399+0000 7ff473fff700 1 --2- 172.21.15.97:0/3905931208 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff47c15a850 0x7ff47c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:53:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.400+0000 7ff47a59c700 1 --2- 172.21.15.97:0/3905931208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff47c15a480 0x7ff47c071b30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009840 tx=0x7ff46c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf6ef32d9c85a6da server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.400+0000 7ff479d9b700 1 -- 172.21.15.97:0/3905931208 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c011830 con 0x7ff47c15a480 2023-07-06T03:53:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.400+0000 7ff479d9b700 1 -- 172.21.15.97:0/3905931208 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c011990 con 0x7ff47c15a480 2023-07-06T03:53:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.400+0000 7ff479d9b700 1 -- 172.21.15.97:0/3905931208 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c011d20 con 0x7ff47c15a480 2023-07-06T03:53:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.401+0000 7ff480dff700 1 -- 172.21.15.97:0/3905931208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff47c15a480 msgr2=0x7ff47c071b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.401+0000 7ff480dff700 1 --2- 172.21.15.97:0/3905931208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff47c15a480 0x7ff47c071b30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009840 tx=0x7ff46c009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.401+0000 7ff480dff700 1 -- 172.21.15.97:0/3905931208 shutdown_connections 2023-07-06T03:53:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.401+0000 7ff480dff700 1 --2- 172.21.15.97:0/3905931208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff47c15a480 0x7ff47c071b30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.401+0000 7ff480dff700 1 --2- 172.21.15.97:0/3905931208 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff47c15a850 0x7ff47c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.401+0000 7ff480dff700 1 -- 172.21.15.97:0/3905931208 >> 172.21.15.97:0/3905931208 conn(0x7ff47c0cb560 msgr2=0x7ff47c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.401+0000 7ff480dff700 1 -- 172.21.15.97:0/3905931208 shutdown_connections 2023-07-06T03:53:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.401+0000 7ff480dff700 1 -- 172.21.15.97:0/3905931208 wait complete. 2023-07-06T03:53:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.402+0000 7ff480dff700 1 Processor -- start 2023-07-06T03:53:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.403+0000 7ff480dff700 1 -- start start 2023-07-06T03:53:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.403+0000 7ff480dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c0adc60 0x7ff47c1547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.403+0000 7ff480dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff47c15a480 0x7ff47c0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff480dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff47c15a850 0x7ff47c0c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:01.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff480dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0b36d0 con 0x7ff47c0adc60 2023-07-06T03:53:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff480dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0b2ac0 con 0x7ff47c15a850 2023-07-06T03:53:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff480dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0b2da0 con 0x7ff47c15a480 2023-07-06T03:53:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff47a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c0adc60 0x7ff47c1547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff47a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c0adc60 0x7ff47c1547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35986/0 (socket says 172.21.15.97:35986) 2023-07-06T03:53:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff47a59c700 1 -- 172.21.15.97:0/1295443938 learned_addr learned my addr 172.21.15.97:0/1295443938 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff47a59c700 1 -- 172.21.15.97:0/1295443938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff47c15a480 msgr2=0x7ff47c0c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff47a59c700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff47c15a480 0x7ff47c0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff47a59c700 1 -- 172.21.15.97:0/1295443938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff47c15a850 msgr2=0x7ff47c0c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff47a59c700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff47c15a850 0x7ff47c0c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff47a59c700 1 -- 172.21.15.97:0/1295443938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4640094a0 con 0x7ff47c0adc60 2023-07-06T03:53:01.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.404+0000 7ff47a59c700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c0adc60 0x7ff47c1547b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0090b0 tx=0x7ff46c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:01.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.405+0000 7ff472ffd700 1 -- 172.21.15.97:0/1295443938 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c01a860 con 0x7ff47c0adc60 2023-07-06T03:53:01.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.405+0000 7ff472ffd700 1 -- 172.21.15.97:0/1295443938 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c01a9c0 con 0x7ff47c0adc60 2023-07-06T03:53:01.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.405+0000 7ff472ffd700 1 -- 172.21.15.97:0/1295443938 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c01ad30 con 0x7ff47c0adc60 2023-07-06T03:53:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.405+0000 7ff480dff700 1 -- 172.21.15.97:0/1295443938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c0094a0 con 0x7ff47c0adc60 2023-07-06T03:53:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.405+0000 7ff480dff700 1 -- 172.21.15.97:0/1295443938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c0b3360 con 0x7ff47c0adc60 2023-07-06T03:53:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.406+0000 7ff480dff700 1 -- 172.21.15.97:0/1295443938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47c159ca0 con 0x7ff47c0adc60 2023-07-06T03:53:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.407+0000 7ff472ffd700 1 -- 172.21.15.97:0/1295443938 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff46c022070 con 0x7ff47c0adc60 2023-07-06T03:53:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.407+0000 7ff472ffd700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff45c06ace0 0x7ff45c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.407+0000 7ff472ffd700 1 -- 172.21.15.97:0/1295443938 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7ff46c08a280 con 0x7ff47c0adc60 2023-07-06T03:53:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.408+0000 7ff473fff700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff45c06ace0 0x7ff45c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.408+0000 7ff473fff700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff45c06ace0 0x7ff45c06d190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff464006f10 tx=0x7ff464006e60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.412+0000 7ff472ffd700 1 -- 172.21.15.97:0/1295443938 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff46c055420 con 0x7ff47c0adc60 2023-07-06T03:53:01.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.688+0000 7ff480dff700 1 -- 172.21.15.97:0/1295443938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7ff47c074350 con 0x7ff47c0adc60 2023-07-06T03:53:01.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.688+0000 7ff472ffd700 1 -- 172.21.15.97:0/1295443938 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+322 (secure 0 0 0) 0x7ff46c0150a0 con 0x7ff47c0adc60 2023-07-06T03:53:01.691 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:01.691 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CLIENT_OLDEST_TID":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to advance oldest client/flush tid","count":1},"detail":[{"message":"mds.a(mds.0): Client smithi097:0 failing to advance its oldest client/flush tid. client_id: 7725"}],"muted":false}},"mutes":[]} 2023-07-06T03:53:01.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 -- 172.21.15.97:0/1295443938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff45c06ace0 msgr2=0x7ff45c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:01.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff45c06ace0 0x7ff45c06d190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff464006f10 tx=0x7ff464006e60 comp rx=0 tx=0).stop 2023-07-06T03:53:01.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 -- 172.21.15.97:0/1295443938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c0adc60 msgr2=0x7ff47c1547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:01.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c0adc60 0x7ff47c1547b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0090b0 tx=0x7ff46c004750 comp rx=0 tx=0).stop 2023-07-06T03:53:01.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 -- 172.21.15.97:0/1295443938 shutdown_connections 2023-07-06T03:53:01.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff45c06ace0 0x7ff45c06d190 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:01.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff47c15a850 0x7ff47c0c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:01.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c0adc60 0x7ff47c1547b0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:01.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 --2- 172.21.15.97:0/1295443938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff47c15a480 0x7ff47c0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:01.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 -- 172.21.15.97:0/1295443938 >> 172.21.15.97:0/1295443938 conn(0x7ff47c0cb560 msgr2=0x7ff47c0b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:01.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 -- 172.21.15.97:0/1295443938 shutdown_connections 2023-07-06T03:53:01.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:01.690+0000 7ff480dff700 1 -- 172.21.15.97:0/1295443938 wait complete. 2023-07-06T03:53:01.710 DEBUG:tasks.ceph_test_case:detail_strings: ['mds.a(mds.0): Client smithi097:0 failing to advance its oldest client/flush tid. client_id: 7725'] 2023-07-06T03:53:01.711 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-07-06T03:53:01.711 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:53:01.711 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:53:01.816 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:53:01.927 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:53:01.928 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:53:01.928 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:53:01.974 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:53:02.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.205+0000 7f1a739ff700 1 Processor -- start 2023-07-06T03:53:02.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.205+0000 7f1a739ff700 1 -- start start 2023-07-06T03:53:02.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.206+0000 7f1a739ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a6c0d11c0 0x7f1a6c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:02.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.206+0000 7f1a739ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a6c0cd530 0x7f1a6c0b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:02.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.206+0000 7f1a739ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0d2ce0 con 0x7f1a6c0b45b0 2023-07-06T03:53:02.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.206+0000 7f1a739ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c0ceb30 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.206+0000 7f1a739ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0cee10 con 0x7f1a6c0cd530 2023-07-06T03:53:02.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.207+0000 7f1a70f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a6c0d11c0 0x7f1a6c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:02.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.207+0000 7f1a70f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a6c0d11c0 0x7f1a6c0ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35996/0 (socket says 172.21.15.97:35996) 2023-07-06T03:53:02.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.207+0000 7f1a70f9a700 1 -- 172.21.15.97:0/3713958706 learned_addr learned my addr 172.21.15.97:0/3713958706 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:02.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.207+0000 7f1a7179b700 1 --1- 172.21.15.97:0/3713958706 >> v1:172.21.15.97:6789/0 conn(0x7f1a6c0b45b0 0x7f1a6c0d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45574/0 (socket says 172.21.15.97:45574) 2023-07-06T03:53:02.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.207+0000 7f1a71f9c700 1 --2- 172.21.15.97:0/3713958706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a6c0cd530 0x7f1a6c0b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:02.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.207+0000 7f1a70f9a700 1 -- 172.21.15.97:0/3713958706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a6c0cd530 msgr2=0x7f1a6c0b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:02.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.207+0000 7f1a70f9a700 1 --2- 172.21.15.97:0/3713958706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a6c0cd530 0x7f1a6c0b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.207+0000 7f1a70f9a700 1 -- 172.21.15.97:0/3713958706 >> v1:172.21.15.97:6789/0 conn(0x7f1a6c0b45b0 legacy=0x7f1a6c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.207+0000 7f1a70f9a700 1 -- 172.21.15.97:0/3713958706 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c0cf0f0 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.207+0000 7f1a63fff700 1 -- 172.21.15.97:0/3713958706 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513090428 0 0) 0x7f1a6c0d2ce0 con 0x7f1a6c0b45b0 2023-07-06T03:53:02.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.207+0000 7f1a70f9a700 1 --2- 172.21.15.97:0/3713958706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d11c0 0x7f1a6c0ce420 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0098f0 tx=0x7f1a5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd763e418619049 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:02.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.208+0000 7f1a63fff700 1 -- 172.21.15.97:0/3713958706 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c00b8a0 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.208+0000 7f1a63fff700 1 -- 172.21.15.97:0/3713958706 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c00ba00 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.208+0000 7f1a63fff700 1 -- 172.21.15.97:0/3713958706 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c00bd90 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.208+0000 7f1a739ff700 1 -- 172.21.15.97:0/3713958706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d11c0 msgr2=0x7f1a6c0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.208+0000 7f1a739ff700 1 --2- 172.21.15.97:0/3713958706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d11c0 0x7f1a6c0ce420 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0098f0 tx=0x7f1a5c009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:02.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.208+0000 7f1a739ff700 1 -- 172.21.15.97:0/3713958706 shutdown_connections 2023-07-06T03:53:02.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.208+0000 7f1a739ff700 1 --2- 172.21.15.97:0/3713958706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a6c0cd530 0x7f1a6c0b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.208+0000 7f1a739ff700 1 --2- 172.21.15.97:0/3713958706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d11c0 0x7f1a6c0ce420 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.208+0000 7f1a739ff700 1 -- 172.21.15.97:0/3713958706 >> 172.21.15.97:0/3713958706 conn(0x7f1a6c09c1b0 msgr2=0x7f1a6c0d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:02.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.208+0000 7f1a739ff700 1 -- 172.21.15.97:0/3713958706 shutdown_connections 2023-07-06T03:53:02.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.208+0000 7f1a739ff700 1 -- 172.21.15.97:0/3713958706 wait complete. 2023-07-06T03:53:02.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.210+0000 7f1a739ff700 1 Processor -- start 2023-07-06T03:53:02.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.210+0000 7f1a739ff700 1 -- start start 2023-07-06T03:53:02.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.211+0000 7f1a739ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a6c0b45b0 0x7f1a6c0cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:02.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.211+0000 7f1a739ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a6c0cd530 0x7f1a6c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:02.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.211+0000 7f1a739ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d11c0 0x7f1a6c06ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:02.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.211+0000 7f1a739ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c1f2a00 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.211+0000 7f1a739ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0ca730 con 0x7f1a6c0cd530 2023-07-06T03:53:02.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.211+0000 7f1a739ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c1f2720 con 0x7f1a6c0b45b0 2023-07-06T03:53:02.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.211+0000 7f1a71f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d11c0 0x7f1a6c06ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:02.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.211+0000 7f1a71f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d11c0 0x7f1a6c06ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36000/0 (socket says 172.21.15.97:36000) 2023-07-06T03:53:02.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.211+0000 7f1a71f9c700 1 -- 172.21.15.97:0/3005108554 learned_addr learned my addr 172.21.15.97:0/3005108554 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:02.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.211+0000 7f1a7179b700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a6c0b45b0 0x7f1a6c0cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:02.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a70f9a700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a6c0cd530 0x7f1a6c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:02.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a71f9c700 1 -- 172.21.15.97:0/3005108554 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a6c0b45b0 msgr2=0x7f1a6c0cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a71f9c700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a6c0b45b0 0x7f1a6c0cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a71f9c700 1 -- 172.21.15.97:0/3005108554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a6c0cd530 msgr2=0x7f1a6c06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a71f9c700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a6c0cd530 0x7f1a6c06a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a71f9c700 1 -- 172.21.15.97:0/3005108554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c009550 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a7179b700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a6c0b45b0 0x7f1a6c0cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:02.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a70f9a700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a6c0cd530 0x7f1a6c06a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:02.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a71f9c700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d11c0 0x7f1a6c06ad10 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1a64007e70 tx=0x7f1a6400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:02.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a61ffb700 1 -- 172.21.15.97:0/3005108554 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400ee50 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a61ffb700 1 -- 172.21.15.97:0/3005108554 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a64014420 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a739ff700 1 -- 172.21.15.97:0/3005108554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c0ca0e0 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a61ffb700 1 -- 172.21.15.97:0/3005108554 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a640147d0 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.212+0000 7f1a739ff700 1 -- 172.21.15.97:0/3005108554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c0cb8e0 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.213+0000 7f1a739ff700 1 -- 172.21.15.97:0/3005108554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c0ca8c0 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.214+0000 7f1a61ffb700 1 -- 172.21.15.97:0/3005108554 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a64014930 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.216+0000 7f1a61ffb700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5006ace0 0x7f1a5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.216+0000 7f1a61ffb700 1 -- 172.21.15.97:0/3005108554 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f1a64089c90 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.216+0000 7f1a7179b700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5006ace0 0x7f1a5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:02.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.217+0000 7f1a7179b700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5006ace0 0x7f1a5006d190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1a68000f60 tx=0x7f1a68007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:02.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.220+0000 7f1a61ffb700 1 -- 172.21.15.97:0/3005108554 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1a64054d00 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.415+0000 7f1a739ff700 1 -- 172.21.15.97:0/3005108554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a6c074350 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.416+0000 7f1a61ffb700 1 -- 172.21.15.97:0/3005108554 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+18706 (secure 0 0 0) 0x7f1a64058320 con 0x7f1a6c0d11c0 2023-07-06T03:53:02.419 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:02.419 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:52:53.756629+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:52:32.448828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-06T03:52:33.428266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.12","mappings":[{"from":6,"to":4}]},{"pgid":"14.15","mappings":[{"from":1,"to":0}]},{"pgid":"14.1a","mappings":[{"from":7,"to":3}]},{"pgid":"14.26","mappings":[{"from":7,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.34","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:53:02.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.418+0000 7f1a739ff700 1 -- 172.21.15.97:0/3005108554 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5006ace0 msgr2=0x7f1a5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.418+0000 7f1a739ff700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5006ace0 0x7f1a5006d190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1a68000f60 tx=0x7f1a68007ca0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.418+0000 7f1a739ff700 1 -- 172.21.15.97:0/3005108554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d11c0 msgr2=0x7f1a6c06ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.418+0000 7f1a739ff700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d11c0 0x7f1a6c06ad10 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1a64007e70 tx=0x7f1a6400e3c0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.418+0000 7f1a739ff700 1 -- 172.21.15.97:0/3005108554 shutdown_connections 2023-07-06T03:53:02.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.418+0000 7f1a739ff700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a5006ace0 0x7f1a5006d190 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.418+0000 7f1a739ff700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d11c0 0x7f1a6c06ad10 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.418+0000 7f1a739ff700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a6c0b45b0 0x7f1a6c0cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.418+0000 7f1a739ff700 1 --2- 172.21.15.97:0/3005108554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a6c0cd530 0x7f1a6c06a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.418+0000 7f1a739ff700 1 -- 172.21.15.97:0/3005108554 >> 172.21.15.97:0/3005108554 conn(0x7f1a6c09c1b0 msgr2=0x7f1a6c0d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:02.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.419+0000 7f1a739ff700 1 -- 172.21.15.97:0/3005108554 shutdown_connections 2023-07-06T03:53:02.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.419+0000 7f1a739ff700 1 -- 172.21.15.97:0/3005108554 wait complete. 2023-07-06T03:53:02.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:52:32.448828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:52:33.428266+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:53:02.440 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:53:02.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.633+0000 7fb0c69ff700 1 Processor -- start 2023-07-06T03:53:02.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.633+0000 7fb0c69ff700 1 -- start start 2023-07-06T03:53:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.635+0000 7fb0c69ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0c00cbbe0 0x7fb0c00ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.635+0000 7fb0c69ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c015ace0 con 0x7fb0c00ca870 2023-07-06T03:53:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.635+0000 7fb0c69ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c00ce9e0 con 0x7fb0c015a910 2023-07-06T03:53:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.635+0000 7fb0c69ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c006c6a0 con 0x7fb0c00cbbe0 2023-07-06T03:53:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.635+0000 7fb0c4f9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb0c00ca870 0x7fb0c0070e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45586/0 (socket says 172.21.15.97:45586) 2023-07-06T03:53:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.635+0000 7fb0c4f9c700 1 -- 172.21.15.97:0/3159213281 learned_addr learned my addr 172.21.15.97:0/3159213281 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.635+0000 7fb0bf7fe700 1 --2- 172.21.15.97:0/3159213281 >> v2:172.21.15.146:3300/0 conn(0x7fb0c00cbbe0 0x7fb0c00ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.635+0000 7fb0beffd700 1 -- 172.21.15.97:0/3159213281 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950839092 0 0) 0x7fb0b8001490 con 0x7fb0c00ca870 2023-07-06T03:53:02.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.635+0000 7fb0beffd700 1 -- 172.21.15.97:0/3159213281 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a80035b0 con 0x7fb0c00ca870 2023-07-06T03:53:02.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.636+0000 7fb0bf7fe700 1 -- 172.21.15.97:0/3159213281 >> v1:172.21.15.146:6789/0 conn(0x7fb0c015a910 legacy=0x7fb0c00cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.636+0000 7fb0bf7fe700 1 -- 172.21.15.97:0/3159213281 >> v1:172.21.15.97:6789/0 conn(0x7fb0c00ca870 legacy=0x7fb0c0070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.636+0000 7fb0bf7fe700 1 -- 172.21.15.97:0/3159213281 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c00cafb0 con 0x7fb0c00cbbe0 2023-07-06T03:53:02.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.636+0000 7fb0bf7fe700 1 --2- 172.21.15.97:0/3159213281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c00cbbe0 0x7fb0c00ca330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009840 tx=0x7fb0b4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=431bb16691ca0690 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:02.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.636+0000 7fb0beffd700 1 -- 172.21.15.97:0/3159213281 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4011830 con 0x7fb0c00cbbe0 2023-07-06T03:53:02.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.636+0000 7fb0beffd700 1 -- 172.21.15.97:0/3159213281 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b4011990 con 0x7fb0c00cbbe0 2023-07-06T03:53:02.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.636+0000 7fb0beffd700 1 -- 172.21.15.97:0/3159213281 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4011d00 con 0x7fb0c00cbbe0 2023-07-06T03:53:02.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.637+0000 7fb0c69ff700 1 -- 172.21.15.97:0/3159213281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c00cbbe0 msgr2=0x7fb0c00ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.637+0000 7fb0c69ff700 1 --2- 172.21.15.97:0/3159213281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c00cbbe0 0x7fb0c00ca330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009840 tx=0x7fb0b4009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:02.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.637+0000 7fb0c69ff700 1 -- 172.21.15.97:0/3159213281 shutdown_connections 2023-07-06T03:53:02.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.637+0000 7fb0c69ff700 1 --2- 172.21.15.97:0/3159213281 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c00cbbe0 0x7fb0c00ca330 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.637+0000 7fb0c69ff700 1 -- 172.21.15.97:0/3159213281 >> 172.21.15.97:0/3159213281 conn(0x7fb0c0065d00 msgr2=0x7fb0c00660d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:02.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.637+0000 7fb0c69ff700 1 -- 172.21.15.97:0/3159213281 shutdown_connections 2023-07-06T03:53:02.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.637+0000 7fb0c69ff700 1 -- 172.21.15.97:0/3159213281 wait complete. 2023-07-06T03:53:02.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.639+0000 7fb0c69ff700 1 Processor -- start 2023-07-06T03:53:02.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.639+0000 7fb0c69ff700 1 -- start start 2023-07-06T03:53:02.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c69ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c00ca870 0x7fb0c0158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:02.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c69ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0c00cbbe0 0x7fb0c015c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:02.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c69ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c015a910 0x7fb0c00b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:02.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c69ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c01587a0 con 0x7fb0c015a910 2023-07-06T03:53:02.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c69ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00b2e40 con 0x7fb0c00ca870 2023-07-06T03:53:02.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c69ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00b2b10 con 0x7fb0c00cbbe0 2023-07-06T03:53:02.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c4f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c015a910 0x7fb0c00b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:02.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c4f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c015a910 0x7fb0c00b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36006/0 (socket says 172.21.15.97:36006) 2023-07-06T03:53:02.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c4f9c700 1 -- 172.21.15.97:0/4258752402 learned_addr learned my addr 172.21.15.97:0/4258752402 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:02.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0bf7fe700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0c00cbbe0 0x7fb0c015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c4f9c700 1 -- 172.21.15.97:0/4258752402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0c00cbbe0 msgr2=0x7fb0c015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c4f9c700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0c00cbbe0 0x7fb0c015c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c4f9c700 1 -- 172.21.15.97:0/4258752402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c00ca870 msgr2=0x7fb0c0158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0bffff700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c00ca870 0x7fb0c0158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c4f9c700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c00ca870 0x7fb0c0158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0c4f9c700 1 -- 172.21.15.97:0/4258752402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b40094a0 con 0x7fb0c015a910 2023-07-06T03:53:02.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.640+0000 7fb0bffff700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c00ca870 0x7fb0c0158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:02.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.641+0000 7fb0c4f9c700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c015a910 0x7fb0c00b32d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800cb40 tx=0x7fb0b800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.641+0000 7fb0bcff9700 1 -- 172.21.15.97:0/4258752402 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800ab50 con 0x7fb0c015a910 2023-07-06T03:53:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.641+0000 7fb0bcff9700 1 -- 172.21.15.97:0/4258752402 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b800acb0 con 0x7fb0c015a910 2023-07-06T03:53:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.641+0000 7fb0c69ff700 1 -- 172.21.15.97:0/4258752402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c015d1d0 con 0x7fb0c015a910 2023-07-06T03:53:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.641+0000 7fb0bcff9700 1 -- 172.21.15.97:0/4258752402 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8012620 con 0x7fb0c015a910 2023-07-06T03:53:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.641+0000 7fb0c69ff700 1 -- 172.21.15.97:0/4258752402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c015d790 con 0x7fb0c015a910 2023-07-06T03:53:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.642+0000 7fb0c69ff700 1 -- 172.21.15.97:0/4258752402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c0074350 con 0x7fb0c015a910 2023-07-06T03:53:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.643+0000 7fb0bcff9700 1 -- 172.21.15.97:0/4258752402 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb0b801b3b0 con 0x7fb0c015a910 2023-07-06T03:53:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.644+0000 7fb0bcff9700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0ac06ace0 0x7fb0ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.644+0000 7fb0bcff9700 1 -- 172.21.15.97:0/4258752402 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fb0b80891c0 con 0x7fb0c015a910 2023-07-06T03:53:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.644+0000 7fb0bffff700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0ac06ace0 0x7fb0ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.645+0000 7fb0bffff700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0ac06ace0 0x7fb0ac06d190 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0000f60 tx=0x7fb0b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.648+0000 7fb0bcff9700 1 -- 172.21.15.97:0/4258752402 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb0b8054190 con 0x7fb0c015a910 2023-07-06T03:53:02.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.839+0000 7fb0c69ff700 1 -- 172.21.15.97:0/4258752402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0c00b2f80 con 0x7fb0c015a910 2023-07-06T03:53:02.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.840+0000 7fb0bcff9700 1 -- 172.21.15.97:0/4258752402 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0b80577b0 con 0x7fb0c015a910 2023-07-06T03:53:02.842 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:53:02.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 -- 172.21.15.97:0/4258752402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0ac06ace0 msgr2=0x7fb0ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0ac06ace0 0x7fb0ac06d190 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0000f60 tx=0x7fb0b0009040 comp rx=0 tx=0).stop 2023-07-06T03:53:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 -- 172.21.15.97:0/4258752402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c015a910 msgr2=0x7fb0c00b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c015a910 0x7fb0c00b32d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800cb40 tx=0x7fb0b800cf00 comp rx=0 tx=0).stop 2023-07-06T03:53:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 -- 172.21.15.97:0/4258752402 shutdown_connections 2023-07-06T03:53:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0ac06ace0 0x7fb0ac06d190 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c015a910 0x7fb0c00b32d0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c00ca870 0x7fb0c0158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 --2- 172.21.15.97:0/4258752402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0c00cbbe0 0x7fb0c015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 -- 172.21.15.97:0/4258752402 >> 172.21.15.97:0/4258752402 conn(0x7fb0c0065d00 msgr2=0x7fb0c00660d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 -- 172.21.15.97:0/4258752402 shutdown_connections 2023-07-06T03:53:02.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:02.842+0000 7fb0c69ff700 1 -- 172.21.15.97:0/4258752402 wait complete. 2023-07-06T03:53:02.862 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:53:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.059+0000 7f79cc7ff700 1 Processor -- start 2023-07-06T03:53:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.059+0000 7f79cc7ff700 1 -- start start 2023-07-06T03:53:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.060+0000 7f79cc7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f79c40b1fc0 0x7f79c40cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.060+0000 7f79cc7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c415d610 con 0x7f79c40cf080 2023-07-06T03:53:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.060+0000 7f79cc7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c415db40 con 0x7f79c40b2390 2023-07-06T03:53:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.060+0000 7f79cc7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c40b5770 con 0x7f79c40b1fc0 2023-07-06T03:53:03.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.060+0000 7f79cad9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f79c40cf080 0x7f79c40b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45602/0 (socket says 172.21.15.97:45602) 2023-07-06T03:53:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.060+0000 7f79cad9c700 1 -- 172.21.15.97:0/3572668161 learned_addr learned my addr 172.21.15.97:0/3572668161 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:03.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.061+0000 7f79ca59b700 1 --2- 172.21.15.97:0/3572668161 >> v2:172.21.15.146:3300/0 conn(0x7f79c40b1fc0 0x7f79c40cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.061+0000 7f79c9599700 1 -- 172.21.15.97:0/3572668161 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309435820 0 0) 0x7f79c415d610 con 0x7f79c40cf080 2023-07-06T03:53:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.061+0000 7f79c9599700 1 -- 172.21.15.97:0/3572668161 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79b00035b0 con 0x7f79c40cf080 2023-07-06T03:53:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.061+0000 7f79ca59b700 1 -- 172.21.15.97:0/3572668161 >> v1:172.21.15.146:6789/0 conn(0x7f79c40b2390 legacy=0x7f79c40b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.061+0000 7f79ca59b700 1 -- 172.21.15.97:0/3572668161 >> v1:172.21.15.97:6789/0 conn(0x7f79c40cf080 legacy=0x7f79c40b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.061+0000 7f79ca59b700 1 -- 172.21.15.97:0/3572668161 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c415d860 con 0x7f79c40b1fc0 2023-07-06T03:53:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.061+0000 7f79ca59b700 1 --2- 172.21.15.97:0/3572668161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79c40b1fc0 0x7f79c40cf8b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009840 tx=0x7f79c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=120b95bc24f8d081 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.062+0000 7f79c9599700 1 -- 172.21.15.97:0/3572668161 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0011830 con 0x7f79c40b1fc0 2023-07-06T03:53:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.062+0000 7f79c9599700 1 -- 172.21.15.97:0/3572668161 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c0011990 con 0x7f79c40b1fc0 2023-07-06T03:53:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.062+0000 7f79c9599700 1 -- 172.21.15.97:0/3572668161 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0011d20 con 0x7f79c40b1fc0 2023-07-06T03:53:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.062+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3572668161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79c40b1fc0 msgr2=0x7f79c40cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.062+0000 7f79cc7ff700 1 --2- 172.21.15.97:0/3572668161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79c40b1fc0 0x7f79c40cf8b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009840 tx=0x7f79c0009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.062+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3572668161 shutdown_connections 2023-07-06T03:53:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.062+0000 7f79cc7ff700 1 --2- 172.21.15.97:0/3572668161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79c40b1fc0 0x7f79c40cf8b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.062+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3572668161 >> 172.21.15.97:0/3572668161 conn(0x7f79c406c210 msgr2=0x7f79c406c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.062+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3572668161 shutdown_connections 2023-07-06T03:53:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.062+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3572668161 wait complete. 2023-07-06T03:53:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.064+0000 7f79cc7ff700 1 Processor -- start 2023-07-06T03:53:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.064+0000 7f79cc7ff700 1 -- start start 2023-07-06T03:53:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.065+0000 7f79cc7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79c40b1fc0 0x7f79c41505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.065+0000 7f79cc7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c40b2390 0x7f79c40d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.065+0000 7f79cc7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79c40cf080 0x7f79c40cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.065+0000 7f79cc7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4150b10 con 0x7f79c40b2390 2023-07-06T03:53:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.065+0000 7f79cc7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40c9fa0 con 0x7f79c40cf080 2023-07-06T03:53:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.065+0000 7f79cc7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40cae00 con 0x7f79c40b1fc0 2023-07-06T03:53:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.065+0000 7f79c9d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c40b2390 0x7f79c40d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.065+0000 7f79c9d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c40b2390 0x7f79c40d2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36010/0 (socket says 172.21.15.97:36010) 2023-07-06T03:53:03.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.065+0000 7f79c9d9a700 1 -- 172.21.15.97:0/3945645379 learned_addr learned my addr 172.21.15.97:0/3945645379 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:03.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79cad9c700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79c40cf080 0x7f79c40cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79ca59b700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79c40b1fc0 0x7f79c41505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79c9d9a700 1 -- 172.21.15.97:0/3945645379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79c40b1fc0 msgr2=0x7f79c41505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79c9d9a700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79c40b1fc0 0x7f79c41505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79c9d9a700 1 -- 172.21.15.97:0/3945645379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79c40cf080 msgr2=0x7f79c40cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79c9d9a700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79c40cf080 0x7f79c40cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79c9d9a700 1 -- 172.21.15.97:0/3945645379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c00094a0 con 0x7f79c40b2390 2023-07-06T03:53:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79c9d9a700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c40b2390 0x7f79c40d2db0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f79b400c4e0 tx=0x7f79b400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79baffd700 1 -- 172.21.15.97:0/3945645379 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b40052d0 con 0x7f79c40b2390 2023-07-06T03:53:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79baffd700 1 -- 172.21.15.97:0/3945645379 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4005430 con 0x7f79c40b2390 2023-07-06T03:53:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3945645379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c40ca1f0 con 0x7f79c40b2390 2023-07-06T03:53:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.066+0000 7f79baffd700 1 -- 172.21.15.97:0/3945645379 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b40038c0 con 0x7f79c40b2390 2023-07-06T03:53:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.067+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3945645379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c40ca7b0 con 0x7f79c40b2390 2023-07-06T03:53:03.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.068+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3945645379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c4074350 con 0x7f79c40b2390 2023-07-06T03:53:03.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.069+0000 7f79baffd700 1 -- 172.21.15.97:0/3945645379 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79b4003a20 con 0x7f79c40b2390 2023-07-06T03:53:03.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.070+0000 7f79baffd700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79a806ace0 0x7f79a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.070+0000 7f79baffd700 1 -- 172.21.15.97:0/3945645379 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f79b4088930 con 0x7f79c40b2390 2023-07-06T03:53:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.070+0000 7f79ca59b700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79a806ace0 0x7f79a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.070+0000 7f79ca59b700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79a806ace0 0x7f79a806d190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f79c00090b0 tx=0x7f79c0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:03.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.074+0000 7f79baffd700 1 -- 172.21.15.97:0/3945645379 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f79b4053870 con 0x7f79c40b2390 2023-07-06T03:53:03.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.267+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3945645379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f79c40caf40 con 0x7f79c40b2390 2023-07-06T03:53:03.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.267+0000 7f79baffd700 1 -- 172.21.15.97:0/3945645379 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v88) v1 ==== 92+0+10 (secure 0 0 0) 0x7f79b4056e90 con 0x7f79c40b2390 2023-07-06T03:53:03.270 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:53:03.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.269+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3945645379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79a806ace0 msgr2=0x7f79a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.269+0000 7f79cc7ff700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79a806ace0 0x7f79a806d190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f79c00090b0 tx=0x7f79c0008030 comp rx=0 tx=0).stop 2023-07-06T03:53:03.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.269+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3945645379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c40b2390 msgr2=0x7f79c40d2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.269+0000 7f79cc7ff700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c40b2390 0x7f79c40d2db0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f79b400c4e0 tx=0x7f79b400c7f0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.270+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3945645379 shutdown_connections 2023-07-06T03:53:03.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.270+0000 7f79cc7ff700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f79a806ace0 0x7f79a806d190 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.270+0000 7f79cc7ff700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79c40cf080 0x7f79c40cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.270+0000 7f79cc7ff700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f79c40b1fc0 0x7f79c41505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.270+0000 7f79cc7ff700 1 --2- 172.21.15.97:0/3945645379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79c40b2390 0x7f79c40d2db0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.270+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3945645379 >> 172.21.15.97:0/3945645379 conn(0x7f79c406c210 msgr2=0x7f79c40b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:03.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.270+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3945645379 shutdown_connections 2023-07-06T03:53:03.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.270+0000 7f79cc7ff700 1 -- 172.21.15.97:0/3945645379 wait complete. 2023-07-06T03:53:03.290 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:53:03.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.492+0000 7f48d8fff700 1 Processor -- start 2023-07-06T03:53:03.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.492+0000 7f48d8fff700 1 -- start start 2023-07-06T03:53:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.493+0000 7f48d8fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f48d40b2390 0x7f48d40d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.493+0000 7f48d8fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d415d610 con 0x7f48d40b1fc0 2023-07-06T03:53:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.493+0000 7f48d8fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d415db40 con 0x7f48d40cffc0 2023-07-06T03:53:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.493+0000 7f48d8fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d415d860 con 0x7f48d40b2390 2023-07-06T03:53:03.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.493+0000 7f48d259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f48d40b1fc0 0x7f48d40cf8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45618/0 (socket says 172.21.15.97:45618) 2023-07-06T03:53:03.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.493+0000 7f48d259c700 1 -- 172.21.15.97:0/3939426060 learned_addr learned my addr 172.21.15.97:0/3939426060 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:03.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48cbfff700 1 --2- 172.21.15.97:0/3939426060 >> v2:172.21.15.146:3300/0 conn(0x7f48d40b2390 0x7f48d40d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984007263 0 0) 0x7f48d415d610 con 0x7f48d40b1fc0 2023-07-06T03:53:03.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48c00035b0 con 0x7f48d40b1fc0 2023-07-06T03:53:03.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3387227358 0 0) 0x7f48c00035b0 con 0x7f48d40b1fc0 2023-07-06T03:53:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 >> v2:172.21.15.146:3300/0 conn(0x7f48d40b2390 msgr2=0x7f48d40d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 --2- 172.21.15.97:0/3939426060 >> v2:172.21.15.146:3300/0 conn(0x7f48d40b2390 0x7f48d40d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 >> v1:172.21.15.146:6789/0 conn(0x7f48d40cffc0 legacy=0x7f48d40d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d40cd630 con 0x7f48d40b1fc0 2023-07-06T03:53:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f48c4001670 con 0x7f48d40b1fc0 2023-07-06T03:53:03.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 >> v1:172.21.15.97:6789/0 conn(0x7f48d40b1fc0 legacy=0x7f48d40cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48c0003a00 0x7f48c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 0x7f48c0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c0005200 0x7f48c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d415d860 con 0x7f48c0004850 2023-07-06T03:53:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c000c080 con 0x7f48c0003a00 2023-07-06T03:53:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.494+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c000c240 con 0x7f48c0005200 2023-07-06T03:53:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48cbfff700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 0x7f48c0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48d259c700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48c0003a00 0x7f48c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48d2d9d700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c0005200 0x7f48c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48cbfff700 1 -- 172.21.15.97:0/3939426060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c0005200 msgr2=0x7f48c000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:03.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48cbfff700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c0005200 0x7f48c000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48cbfff700 1 -- 172.21.15.97:0/3939426060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48c0003a00 msgr2=0x7f48c0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:03.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48cbfff700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48c0003a00 0x7f48c0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48cbfff700 1 -- 172.21.15.97:0/3939426060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00035b0 con 0x7f48c0004850 2023-07-06T03:53:03.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48d2d9d700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c0005200 0x7f48c000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48d2d9d700 1 -- 172.21.15.97:0/3939426060 reap_dead start 2023-07-06T03:53:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48cbfff700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 0x7f48c0004cc0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009b30 tx=0x7f48c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa8e9c8b675e1a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.495+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00e6c0 con 0x7f48c0004850 2023-07-06T03:53:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.496+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48bc00e820 con 0x7f48c0004850 2023-07-06T03:53:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.496+0000 7f48d1d9b700 1 -- 172.21.15.97:0/3939426060 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00eb90 con 0x7f48c0004850 2023-07-06T03:53:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.496+0000 7f48d8fff700 1 -- 172.21.15.97:0/3939426060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 msgr2=0x7f48c0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.496+0000 7f48d8fff700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 0x7f48c0004cc0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009b30 tx=0x7f48c0005880 comp rx=0 tx=0).stop 2023-07-06T03:53:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.496+0000 7f48d8fff700 1 -- 172.21.15.97:0/3939426060 shutdown_connections 2023-07-06T03:53:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.496+0000 7f48d8fff700 1 --2- 172.21.15.97:0/3939426060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 0x7f48c0004cc0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.496+0000 7f48d8fff700 1 -- 172.21.15.97:0/3939426060 >> 172.21.15.97:0/3939426060 conn(0x7f48d406c210 msgr2=0x7f48d406c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.496+0000 7f48d8fff700 1 -- 172.21.15.97:0/3939426060 shutdown_connections 2023-07-06T03:53:03.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.496+0000 7f48d8fff700 1 -- 172.21.15.97:0/3939426060 wait complete. 2023-07-06T03:53:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.499+0000 7f48d8fff700 1 Processor -- start 2023-07-06T03:53:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.499+0000 7f48d8fff700 1 -- start start 2023-07-06T03:53:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.500+0000 7f48d8fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 0x7f48d40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.500+0000 7f48d8fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48d41598f0 0x7f48d40b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.500+0000 7f48d8fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48d4159d60 0x7f48d415b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.500+0000 7f48d8fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40b46c0 con 0x7f48c0004850 2023-07-06T03:53:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.500+0000 7f48d8fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40c9210 con 0x7f48d41598f0 2023-07-06T03:53:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.500+0000 7f48d8fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40b3d50 con 0x7f48d4159d60 2023-07-06T03:53:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.500+0000 7f48d259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 0x7f48d40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.500+0000 7f48d259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 0x7f48d40c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36024/0 (socket says 172.21.15.97:36024) 2023-07-06T03:53:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.500+0000 7f48d259c700 1 -- 172.21.15.97:0/247533533 learned_addr learned my addr 172.21.15.97:0/247533533 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.500+0000 7f48cbfff700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48d41598f0 0x7f48d40b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.500+0000 7f48d2d9d700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48d4159d60 0x7f48d415b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.501+0000 7f48d259c700 1 -- 172.21.15.97:0/247533533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48d4159d60 msgr2=0x7f48d415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.501+0000 7f48d259c700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48d4159d60 0x7f48d415b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.501+0000 7f48d259c700 1 -- 172.21.15.97:0/247533533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48d41598f0 msgr2=0x7f48d40b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.501+0000 7f48d259c700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48d41598f0 0x7f48d40b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.501+0000 7f48d259c700 1 -- 172.21.15.97:0/247533533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d40b4030 con 0x7f48c0004850 2023-07-06T03:53:03.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.501+0000 7f48d259c700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 0x7f48d40c8cd0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f48c4007290 tx=0x7f48c40075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:03.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.501+0000 7f48caffd700 1 -- 172.21.15.97:0/247533533 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4011580 con 0x7f48c0004850 2023-07-06T03:53:03.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.501+0000 7f48caffd700 1 -- 172.21.15.97:0/247533533 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c40116e0 con 0x7f48c0004850 2023-07-06T03:53:03.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.501+0000 7f48caffd700 1 -- 172.21.15.97:0/247533533 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4011a90 con 0x7f48c0004850 2023-07-06T03:53:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.501+0000 7f48d8fff700 1 -- 172.21.15.97:0/247533533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d40b4310 con 0x7f48c0004850 2023-07-06T03:53:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.501+0000 7f48d8fff700 1 -- 172.21.15.97:0/247533533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d40b4bb0 con 0x7f48c0004850 2023-07-06T03:53:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.502+0000 7f48d8fff700 1 -- 172.21.15.97:0/247533533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d40b44a0 con 0x7f48c0004850 2023-07-06T03:53:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.503+0000 7f48caffd700 1 -- 172.21.15.97:0/247533533 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f48c4011bf0 con 0x7f48c0004850 2023-07-06T03:53:03.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.504+0000 7f48caffd700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48b406ace0 0x7f48b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.504+0000 7f48caffd700 1 -- 172.21.15.97:0/247533533 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f48c4089470 con 0x7f48c0004850 2023-07-06T03:53:03.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.505+0000 7f48cbfff700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48b406ace0 0x7f48b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.505+0000 7f48cbfff700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48b406ace0 0x7f48b406d190 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0090b0 tx=0x7f48bc01e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:03.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.509+0000 7f48caffd700 1 -- 172.21.15.97:0/247533533 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f48c4054450 con 0x7f48c0004850 2023-07-06T03:53:03.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.703+0000 7f48d8fff700 1 -- 172.21.15.97:0/247533533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48d4074350 con 0x7f48c0004850 2023-07-06T03:53:03.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.704+0000 7f48caffd700 1 -- 172.21.15.97:0/247533533 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f48c4057a70 con 0x7f48c0004850 2023-07-06T03:53:03.707 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:53:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.706+0000 7f48d8fff700 1 -- 172.21.15.97:0/247533533 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48b406ace0 msgr2=0x7f48b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.706+0000 7f48d8fff700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48b406ace0 0x7f48b406d190 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0090b0 tx=0x7f48bc01e040 comp rx=0 tx=0).stop 2023-07-06T03:53:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.706+0000 7f48d8fff700 1 -- 172.21.15.97:0/247533533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 msgr2=0x7f48d40c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.706+0000 7f48d8fff700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 0x7f48d40c8cd0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f48c4007290 tx=0x7f48c40075a0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.706+0000 7f48d8fff700 1 -- 172.21.15.97:0/247533533 shutdown_connections 2023-07-06T03:53:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.706+0000 7f48d8fff700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48b406ace0 0x7f48b406d190 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.706+0000 7f48d8fff700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48d4159d60 0x7f48d415b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.706+0000 7f48d8fff700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c0004850 0x7f48d40c8cd0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.706+0000 7f48d8fff700 1 --2- 172.21.15.97:0/247533533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48d41598f0 0x7f48d40b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.706+0000 7f48d8fff700 1 -- 172.21.15.97:0/247533533 >> 172.21.15.97:0/247533533 conn(0x7f48d406c210 msgr2=0x7f48d40b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.706+0000 7f48d8fff700 1 -- 172.21.15.97:0/247533533 shutdown_connections 2023-07-06T03:53:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.707+0000 7f48d8fff700 1 -- 172.21.15.97:0/247533533 wait complete. 2023-07-06T03:53:03.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:53:03.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.926+0000 7f84003ff700 1 Processor -- start 2023-07-06T03:53:03.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.926+0000 7f84003ff700 1 -- start start 2023-07-06T03:53:03.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.927+0000 7f84003ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f83f80c9a90 0x7f83f8071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.927+0000 7f84003ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f8159ab0 con 0x7f83f80cd530 2023-07-06T03:53:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.927+0000 7f84003ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f8070b70 con 0x7f83f81596e0 2023-07-06T03:53:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.927+0000 7f84003ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f80717f0 con 0x7f83f80c9a90 2023-07-06T03:53:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.927+0000 7f83fe99c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f83f80cd530 0x7f83f815d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45626/0 (socket says 172.21.15.97:45626) 2023-07-06T03:53:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.927+0000 7f83fe99c700 1 -- 172.21.15.97:0/1836955263 learned_addr learned my addr 172.21.15.97:0/1836955263 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.927+0000 7f83fd99a700 1 --2- 172.21.15.97:0/1836955263 >> v2:172.21.15.146:3300/0 conn(0x7f83f80c9a90 0x7f83f8071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.928+0000 7f83fd199700 1 -- 172.21.15.97:0/1836955263 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064395420 0 0) 0x7f83f8159ab0 con 0x7f83f80cd530 2023-07-06T03:53:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.928+0000 7f83fd199700 1 -- 172.21.15.97:0/1836955263 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e40035b0 con 0x7f83f80cd530 2023-07-06T03:53:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.928+0000 7f83fd99a700 1 -- 172.21.15.97:0/1836955263 >> v1:172.21.15.146:6789/0 conn(0x7f83f81596e0 legacy=0x7f83f80cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.928+0000 7f83fd99a700 1 -- 172.21.15.97:0/1836955263 >> v1:172.21.15.97:6789/0 conn(0x7f83f80cd530 legacy=0x7f83f815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.928+0000 7f83fd99a700 1 -- 172.21.15.97:0/1836955263 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f8070e50 con 0x7f83f80c9a90 2023-07-06T03:53:03.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.928+0000 7f83fd99a700 1 --2- 172.21.15.97:0/1836955263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f80c9a90 0x7f83f8071c10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009840 tx=0x7f83e8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14936599df998346 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:03.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.928+0000 7f83fd199700 1 -- 172.21.15.97:0/1836955263 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8011830 con 0x7f83f80c9a90 2023-07-06T03:53:03.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.928+0000 7f83fd199700 1 -- 172.21.15.97:0/1836955263 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e8011990 con 0x7f83f80c9a90 2023-07-06T03:53:03.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.928+0000 7f83fd199700 1 -- 172.21.15.97:0/1836955263 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8011d20 con 0x7f83f80c9a90 2023-07-06T03:53:03.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.929+0000 7f84003ff700 1 -- 172.21.15.97:0/1836955263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f80c9a90 msgr2=0x7f83f8071c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.929+0000 7f84003ff700 1 --2- 172.21.15.97:0/1836955263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f80c9a90 0x7f83f8071c10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009840 tx=0x7f83e8009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.929+0000 7f84003ff700 1 -- 172.21.15.97:0/1836955263 shutdown_connections 2023-07-06T03:53:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.929+0000 7f84003ff700 1 --2- 172.21.15.97:0/1836955263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f80c9a90 0x7f83f8071c10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.929+0000 7f84003ff700 1 -- 172.21.15.97:0/1836955263 >> 172.21.15.97:0/1836955263 conn(0x7f83f8065d00 msgr2=0x7f83f80660d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.929+0000 7f84003ff700 1 -- 172.21.15.97:0/1836955263 shutdown_connections 2023-07-06T03:53:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.929+0000 7f84003ff700 1 -- 172.21.15.97:0/1836955263 wait complete. 2023-07-06T03:53:03.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.931+0000 7f84003ff700 1 Processor -- start 2023-07-06T03:53:03.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.931+0000 7f84003ff700 1 -- start start 2023-07-06T03:53:03.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f84003ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83f80c9a90 0x7f83f8155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f84003ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f80cd530 0x7f83f814f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f84003ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f81596e0 0x7f83f81f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f84003ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f81530e0 con 0x7f83f80cd530 2023-07-06T03:53:03.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f84003ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80b1ef0 con 0x7f83f81596e0 2023-07-06T03:53:03.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f84003ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f814f6c0 con 0x7f83f80c9a90 2023-07-06T03:53:03.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fd99a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f80cd530 0x7f83f814f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fd99a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f80cd530 0x7f83f814f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36040/0 (socket says 172.21.15.97:36040) 2023-07-06T03:53:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fd99a700 1 -- 172.21.15.97:0/1486897933 learned_addr learned my addr 172.21.15.97:0/1486897933 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fe19b700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83f80c9a90 0x7f83f8155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fe99c700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f81596e0 0x7f83f81f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fd99a700 1 -- 172.21.15.97:0/1486897933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83f80c9a90 msgr2=0x7f83f8155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fd99a700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83f80c9a90 0x7f83f8155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fd99a700 1 -- 172.21.15.97:0/1486897933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f81596e0 msgr2=0x7f83f81f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:03.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fd99a700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f81596e0 0x7f83f81f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fd99a700 1 -- 172.21.15.97:0/1486897933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80094a0 con 0x7f83f80cd530 2023-07-06T03:53:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fe19b700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83f80c9a90 0x7f83f8155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.932+0000 7f83fe99c700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f81596e0 0x7f83f81f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.933+0000 7f83fd99a700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f80cd530 0x7f83f814f180 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009fd0 tx=0x7f83e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.933+0000 7f83eeffd700 1 -- 172.21.15.97:0/1486897933 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80038a0 con 0x7f83f80cd530 2023-07-06T03:53:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.933+0000 7f83eeffd700 1 -- 172.21.15.97:0/1486897933 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e801f070 con 0x7f83f80cd530 2023-07-06T03:53:03.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.933+0000 7f84003ff700 1 -- 172.21.15.97:0/1486897933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f81f2aa0 con 0x7f83f80cd530 2023-07-06T03:53:03.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.933+0000 7f83eeffd700 1 -- 172.21.15.97:0/1486897933 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8003c40 con 0x7f83f80cd530 2023-07-06T03:53:03.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.933+0000 7f84003ff700 1 -- 172.21.15.97:0/1486897933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f81f3020 con 0x7f83f80cd530 2023-07-06T03:53:03.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.934+0000 7f84003ff700 1 -- 172.21.15.97:0/1486897933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f8074350 con 0x7f83f80cd530 2023-07-06T03:53:03.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.935+0000 7f83eeffd700 1 -- 172.21.15.97:0/1486897933 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83e8026480 con 0x7f83f80cd530 2023-07-06T03:53:03.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.936+0000 7f83eeffd700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f83dc06ace0 0x7f83dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:03.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.936+0000 7f83eeffd700 1 -- 172.21.15.97:0/1486897933 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f83e8021070 con 0x7f83f80cd530 2023-07-06T03:53:03.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.936+0000 7f83fe19b700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f83dc06ace0 0x7f83dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:03.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.937+0000 7f83fe19b700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f83dc06ace0 0x7f83dc06d190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f83f4000f60 tx=0x7f83f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:03.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:03.941+0000 7f83eeffd700 1 -- 172.21.15.97:0/1486897933 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f83e8055e60 con 0x7f83f80cd530 2023-07-06T03:53:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.135+0000 7f84003ff700 1 -- 172.21.15.97:0/1486897933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f83f80cc690 con 0x7f83f80cd530 2023-07-06T03:53:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.135+0000 7f83eeffd700 1 -- 172.21.15.97:0/1486897933 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f83e80150a0 con 0x7f83f80cd530 2023-07-06T03:53:04.139 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:53:04.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 -- 172.21.15.97:0/1486897933 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f83dc06ace0 msgr2=0x7f83dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f83dc06ace0 0x7f83dc06d190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f83f4000f60 tx=0x7f83f4009040 comp rx=0 tx=0).stop 2023-07-06T03:53:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 -- 172.21.15.97:0/1486897933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f80cd530 msgr2=0x7f83f814f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f80cd530 0x7f83f814f180 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009fd0 tx=0x7f83e8004750 comp rx=0 tx=0).stop 2023-07-06T03:53:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 -- 172.21.15.97:0/1486897933 shutdown_connections 2023-07-06T03:53:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f83dc06ace0 0x7f83dc06d190 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83f81596e0 0x7f83f81f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f83f80c9a90 0x7f83f8155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 --2- 172.21.15.97:0/1486897933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f80cd530 0x7f83f814f180 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 -- 172.21.15.97:0/1486897933 >> 172.21.15.97:0/1486897933 conn(0x7f83f8065d00 msgr2=0x7f83f806fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 -- 172.21.15.97:0/1486897933 shutdown_connections 2023-07-06T03:53:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.138+0000 7f84003ff700 1 -- 172.21.15.97:0/1486897933 wait complete. 2023-07-06T03:53:04.158 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:53:04.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.352+0000 7f196dbff700 1 Processor -- start 2023-07-06T03:53:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.352+0000 7f196dbff700 1 -- start start 2023-07-06T03:53:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.353+0000 7f196dbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19680d0580 0x7f196806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.353+0000 7f196dbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1968160aa0 0x7f1968160f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.353+0000 7f196dbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196806bb40 con 0x7f196815cde0 2023-07-06T03:53:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.353+0000 7f196dbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196806c510 con 0x7f19680d0580 2023-07-06T03:53:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.353+0000 7f196dbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19681617f0 con 0x7f1968160aa0 2023-07-06T03:53:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.354+0000 7f1966ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19680d0580 0x7f196806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.354+0000 7f1966ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19680d0580 0x7f196806bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36044/0 (socket says 172.21.15.97:36044) 2023-07-06T03:53:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.354+0000 7f1966ffd700 1 -- 172.21.15.97:0/140683628 learned_addr learned my addr 172.21.15.97:0/140683628 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.354+0000 7f19677fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f196815cde0 0x7f19680cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45630/0 (socket says 172.21.15.97:45630) 2023-07-06T03:53:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.354+0000 7f1967fff700 1 --2- 172.21.15.97:0/140683628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1968160aa0 0x7f1968160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.354+0000 7f1966ffd700 1 -- 172.21.15.97:0/140683628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1968160aa0 msgr2=0x7f1968160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.354+0000 7f1966ffd700 1 --2- 172.21.15.97:0/140683628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1968160aa0 0x7f1968160f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.354+0000 7f1966ffd700 1 -- 172.21.15.97:0/140683628 >> v1:172.21.15.97:6789/0 conn(0x7f196815cde0 legacy=0x7f19680cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.354+0000 7f1966ffd700 1 -- 172.21.15.97:0/140683628 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196806b210 con 0x7f19680d0580 2023-07-06T03:53:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.354+0000 7f1966ffd700 1 --2- 172.21.15.97:0/140683628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19680d0580 0x7f196806bfd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f195c0098f0 tx=0x7f195c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a16362ce8261303 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.355+0000 7f19667fc700 1 -- 172.21.15.97:0/140683628 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c00b8a0 con 0x7f19680d0580 2023-07-06T03:53:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.355+0000 7f19667fc700 1 -- 172.21.15.97:0/140683628 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c00ba00 con 0x7f19680d0580 2023-07-06T03:53:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.355+0000 7f19667fc700 1 -- 172.21.15.97:0/140683628 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c00bd90 con 0x7f19680d0580 2023-07-06T03:53:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.355+0000 7f196dbff700 1 -- 172.21.15.97:0/140683628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19680d0580 msgr2=0x7f196806bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.355+0000 7f196dbff700 1 --2- 172.21.15.97:0/140683628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19680d0580 0x7f196806bfd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f195c0098f0 tx=0x7f195c009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.355+0000 7f196dbff700 1 -- 172.21.15.97:0/140683628 shutdown_connections 2023-07-06T03:53:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.355+0000 7f196dbff700 1 --2- 172.21.15.97:0/140683628 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1968160aa0 0x7f1968160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.355+0000 7f196dbff700 1 --2- 172.21.15.97:0/140683628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19680d0580 0x7f196806bfd0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.355+0000 7f196dbff700 1 -- 172.21.15.97:0/140683628 >> 172.21.15.97:0/140683628 conn(0x7f196806eca0 msgr2=0x7f1968065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.355+0000 7f196dbff700 1 -- 172.21.15.97:0/140683628 shutdown_connections 2023-07-06T03:53:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.355+0000 7f196dbff700 1 -- 172.21.15.97:0/140683628 wait complete. 2023-07-06T03:53:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.357+0000 7f196dbff700 1 Processor -- start 2023-07-06T03:53:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.357+0000 7f196dbff700 1 -- start start 2023-07-06T03:53:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f196dbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19680d0580 0x7f1968147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f196dbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f196815cde0 0x7f19680c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f196dbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968160aa0 0x7f1968149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f196dbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19680c9cd0 con 0x7f1968160aa0 2023-07-06T03:53:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f196dbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19681484d0 con 0x7f196815cde0 2023-07-06T03:53:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f196dbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196814a8c0 con 0x7f19680d0580 2023-07-06T03:53:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f1967fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968160aa0 0x7f1968149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f1967fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968160aa0 0x7f1968149ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36058/0 (socket says 172.21.15.97:36058) 2023-07-06T03:53:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f1967fff700 1 -- 172.21.15.97:0/752274904 learned_addr learned my addr 172.21.15.97:0/752274904 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f19677fe700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19680d0580 0x7f1968147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f1967fff700 1 -- 172.21.15.97:0/752274904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19680d0580 msgr2=0x7f1968147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f1966ffd700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f196815cde0 0x7f19680c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f1967fff700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19680d0580 0x7f1968147f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.358+0000 7f1967fff700 1 -- 172.21.15.97:0/752274904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f196815cde0 msgr2=0x7f19680c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.359+0000 7f1967fff700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f196815cde0 0x7f19680c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.359+0000 7f1967fff700 1 -- 172.21.15.97:0/752274904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c009550 con 0x7f1968160aa0 2023-07-06T03:53:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.359+0000 7f19677fe700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19680d0580 0x7f1968147f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.359+0000 7f1966ffd700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f196815cde0 0x7f19680c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.359+0000 7f1967fff700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968160aa0 0x7f1968149ff0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1960007e60 tx=0x7f196000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.359+0000 7f194ffff700 1 -- 172.21.15.97:0/752274904 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000ede0 con 0x7f1968160aa0 2023-07-06T03:53:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.359+0000 7f194ffff700 1 -- 172.21.15.97:0/752274904 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1960014420 con 0x7f1968160aa0 2023-07-06T03:53:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.359+0000 7f196dbff700 1 -- 172.21.15.97:0/752274904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196814aba0 con 0x7f1968160aa0 2023-07-06T03:53:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.359+0000 7f194ffff700 1 -- 172.21.15.97:0/752274904 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19600147d0 con 0x7f1968160aa0 2023-07-06T03:53:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.359+0000 7f196dbff700 1 -- 172.21.15.97:0/752274904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19680c70c0 con 0x7f1968160aa0 2023-07-06T03:53:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.360+0000 7f196dbff700 1 -- 172.21.15.97:0/752274904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19680c9e40 con 0x7f1968160aa0 2023-07-06T03:53:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.361+0000 7f194ffff700 1 -- 172.21.15.97:0/752274904 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1960014930 con 0x7f1968160aa0 2023-07-06T03:53:04.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.362+0000 7f194ffff700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195406ad10 0x7f195406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:04.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.362+0000 7f194ffff700 1 -- 172.21.15.97:0/752274904 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f1960089d80 con 0x7f1968160aa0 2023-07-06T03:53:04.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.362+0000 7f19677fe700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195406ad10 0x7f195406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:04.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.363+0000 7f19677fe700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195406ad10 0x7f195406d1c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1958000f60 tx=0x7f1958009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:04.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.366+0000 7f194ffff700 1 -- 172.21.15.97:0/752274904 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1960054d70 con 0x7f1968160aa0 2023-07-06T03:53:04.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.578+0000 7f196dbff700 1 -- 172.21.15.97:0/752274904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1968074350 con 0x7f1968160aa0 2023-07-06T03:53:04.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.578+0000 7f194ffff700 1 -- 172.21.15.97:0/752274904 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1960058390 con 0x7f1968160aa0 2023-07-06T03:53:04.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 -- 172.21.15.97:0/752274904 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195406ad10 msgr2=0x7f195406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:04.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195406ad10 0x7f195406d1c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1958000f60 tx=0x7f1958009040 comp rx=0 tx=0).stop 2023-07-06T03:53:04.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 -- 172.21.15.97:0/752274904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968160aa0 msgr2=0x7f1968149ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968160aa0 0x7f1968149ff0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1960007e60 tx=0x7f196000e3c0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 -- 172.21.15.97:0/752274904 shutdown_connections 2023-07-06T03:53:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f195406ad10 0x7f195406d1c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1968160aa0 0x7f1968149ff0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f19680d0580 0x7f1968147f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 --2- 172.21.15.97:0/752274904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f196815cde0 0x7f19680c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 -- 172.21.15.97:0/752274904 >> 172.21.15.97:0/752274904 conn(0x7f196806eca0 msgr2=0x7f19680b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:04.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 -- 172.21.15.97:0/752274904 shutdown_connections 2023-07-06T03:53:04.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:04.580+0000 7f196dbff700 1 -- 172.21.15.97:0/752274904 wait complete. 2023-07-06T03:53:04.585 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:53:04.601 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:53:04.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:04.602 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:53:04.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[150207]: fuse finished with error 0 and tester_r 0 2023-07-06T03:53:04.679 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:53:04.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:04.680 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:53:04.821 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:53:04.821 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:53:04.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:53:05.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.062+0000 7f38753ff700 1 Processor -- start 2023-07-06T03:53:05.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.062+0000 7f38753ff700 1 -- start start 2023-07-06T03:53:05.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f38753ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38700cce00 0x7f38700d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f38753ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f38700d1130 0x7f387006b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f38753ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f387006b6b0 0x7f38700cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f38753ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38700b2640 con 0x7f387006b6b0 2023-07-06T03:53:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f38753ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38700b0f50 con 0x7f38700d1130 2023-07-06T03:53:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f38753ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38700d1580 con 0x7f38700cce00 2023-07-06T03:53:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f386f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f387006b6b0 0x7f38700cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f386f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f387006b6b0 0x7f38700cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36068/0 (socket says 172.21.15.97:36068) 2023-07-06T03:53:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f386f7fe700 1 -- 172.21.15.97:0/2263864117 learned_addr learned my addr 172.21.15.97:0/2263864117 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:05.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f3867fff700 1 --2- 172.21.15.97:0/2263864117 >> v2:172.21.15.146:3300/0 conn(0x7f38700d1130 0x7f387006b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f386effd700 1 --2- 172.21.15.97:0/2263864117 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38700cce00 0x7f38700d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f386f7fe700 1 -- 172.21.15.97:0/2263864117 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38700cce00 msgr2=0x7f38700d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:05.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f386f7fe700 1 --2- 172.21.15.97:0/2263864117 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38700cce00 0x7f38700d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f386f7fe700 1 -- 172.21.15.97:0/2263864117 >> v2:172.21.15.146:3300/0 conn(0x7f38700d1130 msgr2=0x7f387006b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:05.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f386f7fe700 1 --2- 172.21.15.97:0/2263864117 >> v2:172.21.15.146:3300/0 conn(0x7f38700d1130 0x7f387006b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.064+0000 7f386f7fe700 1 -- 172.21.15.97:0/2263864117 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387006bb30 con 0x7f387006b6b0 2023-07-06T03:53:05.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.065+0000 7f386f7fe700 1 --2- 172.21.15.97:0/2263864117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387006b6b0 0x7f38700cd510 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f38700b3880 tx=0x7f386800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabc19973b7bbf9f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:05.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.065+0000 7f386e7fc700 1 -- 172.21.15.97:0/2263864117 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386800e070 con 0x7f387006b6b0 2023-07-06T03:53:05.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.065+0000 7f386e7fc700 1 -- 172.21.15.97:0/2263864117 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386800b9c0 con 0x7f387006b6b0 2023-07-06T03:53:05.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.065+0000 7f386e7fc700 1 -- 172.21.15.97:0/2263864117 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386800bd70 con 0x7f387006b6b0 2023-07-06T03:53:05.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.065+0000 7f38753ff700 1 -- 172.21.15.97:0/2263864117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387006b6b0 msgr2=0x7f38700cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.065+0000 7f38753ff700 1 --2- 172.21.15.97:0/2263864117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387006b6b0 0x7f38700cd510 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f38700b3880 tx=0x7f386800b330 comp rx=0 tx=0).stop 2023-07-06T03:53:05.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.066+0000 7f38753ff700 1 -- 172.21.15.97:0/2263864117 shutdown_connections 2023-07-06T03:53:05.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.066+0000 7f38753ff700 1 --2- 172.21.15.97:0/2263864117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387006b6b0 0x7f38700cd510 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.066+0000 7f38753ff700 1 --2- 172.21.15.97:0/2263864117 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38700cce00 0x7f38700d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.066+0000 7f38753ff700 1 --2- 172.21.15.97:0/2263864117 >> v2:172.21.15.146:3300/0 conn(0x7f38700d1130 0x7f387006b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.066+0000 7f38753ff700 1 -- 172.21.15.97:0/2263864117 >> 172.21.15.97:0/2263864117 conn(0x7f3870099a50 msgr2=0x7f3870099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:05.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.066+0000 7f38753ff700 1 -- 172.21.15.97:0/2263864117 shutdown_connections 2023-07-06T03:53:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.066+0000 7f38753ff700 1 -- 172.21.15.97:0/2263864117 wait complete. 2023-07-06T03:53:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.067+0000 7f38753ff700 1 Processor -- start 2023-07-06T03:53:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.067+0000 7f38753ff700 1 -- start start 2023-07-06T03:53:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.068+0000 7f38753ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387006b6b0 0x7f38701f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.068+0000 7f38753ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38700cce00 0x7f38700c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.068+0000 7f38753ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38700d1130 0x7f387014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.068+0000 7f38753ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38700c7b50 con 0x7f387006b6b0 2023-07-06T03:53:05.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.068+0000 7f38753ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387014aba0 con 0x7f38700cce00 2023-07-06T03:53:05.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.068+0000 7f38753ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38701f55f0 con 0x7f38700d1130 2023-07-06T03:53:05.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f386effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387006b6b0 0x7f38701f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f386effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387006b6b0 0x7f38701f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36080/0 (socket says 172.21.15.97:36080) 2023-07-06T03:53:05.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f386effd700 1 -- 172.21.15.97:0/520382850 learned_addr learned my addr 172.21.15.97:0/520382850 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:05.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f3867fff700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38700cce00 0x7f38700c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f386f7fe700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38700d1130 0x7f387014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f386effd700 1 -- 172.21.15.97:0/520382850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38700d1130 msgr2=0x7f387014cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f386effd700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38700d1130 0x7f387014cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f386effd700 1 -- 172.21.15.97:0/520382850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38700cce00 msgr2=0x7f38700c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f386effd700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38700cce00 0x7f38700c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f386effd700 1 -- 172.21.15.97:0/520382850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3868009b30 con 0x7f387006b6b0 2023-07-06T03:53:05.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f386effd700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387006b6b0 0x7f38701f60b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f386000d800 tx=0x7f386000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:05.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f38677fe700 1 -- 172.21.15.97:0/520382850 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600125f0 con 0x7f387006b6b0 2023-07-06T03:53:05.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f38677fe700 1 -- 172.21.15.97:0/520382850 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3860012750 con 0x7f387006b6b0 2023-07-06T03:53:05.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.069+0000 7f38753ff700 1 -- 172.21.15.97:0/520382850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3870150c40 con 0x7f387006b6b0 2023-07-06T03:53:05.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.070+0000 7f38677fe700 1 -- 172.21.15.97:0/520382850 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860012ae0 con 0x7f387006b6b0 2023-07-06T03:53:05.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.070+0000 7f38753ff700 1 -- 172.21.15.97:0/520382850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387014dd10 con 0x7f387006b6b0 2023-07-06T03:53:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.071+0000 7f38753ff700 1 -- 172.21.15.97:0/520382850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3870074350 con 0x7f387006b6b0 2023-07-06T03:53:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.072+0000 7f38677fe700 1 -- 172.21.15.97:0/520382850 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3860012c40 con 0x7f387006b6b0 2023-07-06T03:53:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.073+0000 7f38677fe700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f385006ace0 0x7f385006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.073+0000 7f38677fe700 1 -- 172.21.15.97:0/520382850 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f38600898b0 con 0x7f387006b6b0 2023-07-06T03:53:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.073+0000 7f3867fff700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f385006ace0 0x7f385006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.074+0000 7f3867fff700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f385006ace0 0x7f385006d190 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f38700cfdc0 tx=0x7f3858008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.077+0000 7f38677fe700 1 -- 172.21.15.97:0/520382850 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3860054890 con 0x7f387006b6b0 2023-07-06T03:53:05.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.272+0000 7f38753ff700 1 -- 172.21.15.97:0/520382850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f387015fd60 con 0x7f387006b6b0 2023-07-06T03:53:05.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.274+0000 7f38677fe700 1 -- 172.21.15.97:0/520382850 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+18706 (secure 0 0 0) 0x7f3860057eb0 con 0x7f387006b6b0 2023-07-06T03:53:05.277 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:05.277 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:52:53.756629+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:52:32.448828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-06T03:52:33.428266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.12","mappings":[{"from":6,"to":4}]},{"pgid":"14.15","mappings":[{"from":1,"to":0}]},{"pgid":"14.1a","mappings":[{"from":7,"to":3}]},{"pgid":"14.26","mappings":[{"from":7,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.34","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:53:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 -- 172.21.15.97:0/520382850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f385006ace0 msgr2=0x7f385006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f385006ace0 0x7f385006d190 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f38700cfdc0 tx=0x7f3858008040 comp rx=0 tx=0).stop 2023-07-06T03:53:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 -- 172.21.15.97:0/520382850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387006b6b0 msgr2=0x7f38701f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387006b6b0 0x7f38701f60b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f386000d800 tx=0x7f386000dbc0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 -- 172.21.15.97:0/520382850 shutdown_connections 2023-07-06T03:53:05.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f385006ace0 0x7f385006d190 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f38700d1130 0x7f387014cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387006b6b0 0x7f38701f60b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 --2- 172.21.15.97:0/520382850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38700cce00 0x7f38700c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 -- 172.21.15.97:0/520382850 >> 172.21.15.97:0/520382850 conn(0x7f3870099a50 msgr2=0x7f38700b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:05.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 -- 172.21.15.97:0/520382850 shutdown_connections 2023-07-06T03:53:05.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.276+0000 7f38753ff700 1 -- 172.21.15.97:0/520382850 wait complete. 2023-07-06T03:53:05.297 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:52:32.448828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:52:33.428266+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:53:05.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:53:05.478 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:53:05.483 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:53:05.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.498+0000 7f27c41ff700 1 Processor -- start 2023-07-06T03:53:05.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.499+0000 7f27c41ff700 1 -- start start 2023-07-06T03:53:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.500+0000 7f27c41ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27bc0b2390 0x7f27bc0d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.500+0000 7f27c41ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f27bc0cffc0 0x7f27bc0d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.500+0000 7f27c41ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27bc15d610 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.500+0000 7f27c41ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27bc15db40 con 0x7f27bc0cffc0 2023-07-06T03:53:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.500+0000 7f27c41ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27bc15d860 con 0x7f27bc0b2390 2023-07-06T03:53:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.500+0000 7f27c1f9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f27bc0b1fc0 0x7f27bc0cf8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45634/0 (socket says 172.21.15.97:45634) 2023-07-06T03:53:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.500+0000 7f27c1f9b700 1 -- 172.21.15.97:0/2040835291 learned_addr learned my addr 172.21.15.97:0/2040835291 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.500+0000 7f27c279c700 1 --2- 172.21.15.97:0/2040835291 >> v2:172.21.15.97:3300/0 conn(0x7f27bc0cffc0 0x7f27bc0d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.500+0000 7f27c179a700 1 --2- 172.21.15.97:0/2040835291 >> v2:172.21.15.146:3300/0 conn(0x7f27bc0b2390 0x7f27bc0d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.501+0000 7f27c279c700 1 -- 172.21.15.97:0/2040835291 >> v2:172.21.15.146:3300/0 conn(0x7f27bc0b2390 msgr2=0x7f27bc0d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.501+0000 7f27c279c700 1 --2- 172.21.15.97:0/2040835291 >> v2:172.21.15.146:3300/0 conn(0x7f27bc0b2390 0x7f27bc0d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.501+0000 7f27c279c700 1 -- 172.21.15.97:0/2040835291 >> v1:172.21.15.97:6789/0 conn(0x7f27bc0b1fc0 legacy=0x7f27bc0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.501+0000 7f27c279c700 1 -- 172.21.15.97:0/2040835291 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc0cd630 con 0x7f27bc0cffc0 2023-07-06T03:53:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.501+0000 7f27c0f99700 1 -- 172.21.15.97:0/2040835291 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101846736 0 0) 0x7f27bc15d610 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.501+0000 7f27c279c700 1 --2- 172.21.15.97:0/2040835291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc0cffc0 0x7f27bc0d10e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0cd4a0 tx=0x7f27b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5eaeba50c840c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.501+0000 7f27c0f99700 1 -- 172.21.15.97:0/2040835291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b400e070 con 0x7f27bc0cffc0 2023-07-06T03:53:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.501+0000 7f27c0f99700 1 -- 172.21.15.97:0/2040835291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b400bbd0 con 0x7f27bc0cffc0 2023-07-06T03:53:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.501+0000 7f27c0f99700 1 -- 172.21.15.97:0/2040835291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4012430 con 0x7f27bc0cffc0 2023-07-06T03:53:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.502+0000 7f27c41ff700 1 -- 172.21.15.97:0/2040835291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc0cffc0 msgr2=0x7f27bc0d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.502+0000 7f27c41ff700 1 --2- 172.21.15.97:0/2040835291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc0cffc0 0x7f27bc0d10e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0cd4a0 tx=0x7f27b400b330 comp rx=0 tx=0).stop 2023-07-06T03:53:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.502+0000 7f27c41ff700 1 -- 172.21.15.97:0/2040835291 shutdown_connections 2023-07-06T03:53:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.502+0000 7f27c41ff700 1 --2- 172.21.15.97:0/2040835291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc0cffc0 0x7f27bc0d10e0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.502+0000 7f27c41ff700 1 --2- 172.21.15.97:0/2040835291 >> v2:172.21.15.146:3300/0 conn(0x7f27bc0b2390 0x7f27bc0d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.502+0000 7f27c41ff700 1 -- 172.21.15.97:0/2040835291 >> 172.21.15.97:0/2040835291 conn(0x7f27bc06c210 msgr2=0x7f27bc06c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.502+0000 7f27c41ff700 1 -- 172.21.15.97:0/2040835291 shutdown_connections 2023-07-06T03:53:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.502+0000 7f27c41ff700 1 -- 172.21.15.97:0/2040835291 wait complete. 2023-07-06T03:53:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.504+0000 7f27c41ff700 1 Processor -- start 2023-07-06T03:53:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.504+0000 7f27c41ff700 1 -- start start 2023-07-06T03:53:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c41ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc0b1fc0 0x7f27bc155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c41ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc0b2390 0x7f27bc0c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c41ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc0cffc0 0x7f27bc0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c41ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc1e3ed0 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c41ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc0b5470 con 0x7f27bc0b2390 2023-07-06T03:53:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c41ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc1e4120 con 0x7f27bc0cffc0 2023-07-06T03:53:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c1f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc0b1fc0 0x7f27bc155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c1f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc0b1fc0 0x7f27bc155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36094/0 (socket says 172.21.15.97:36094) 2023-07-06T03:53:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c1f9b700 1 -- 172.21.15.97:0/100923526 learned_addr learned my addr 172.21.15.97:0/100923526 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c179a700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc0b2390 0x7f27bc0c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c279c700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc0cffc0 0x7f27bc0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c1f9b700 1 -- 172.21.15.97:0/100923526 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc0cffc0 msgr2=0x7f27bc0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c1f9b700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc0cffc0 0x7f27bc0b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c1f9b700 1 -- 172.21.15.97:0/100923526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc0b2390 msgr2=0x7f27bc0c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c1f9b700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc0b2390 0x7f27bc0c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.505+0000 7f27c1f9b700 1 -- 172.21.15.97:0/100923526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b4009a70 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.506+0000 7f27c1f9b700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc0b1fc0 0x7f27bc155490 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f27b80011e0 tx=0x7f27b800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:05.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.506+0000 7f27b27fc700 1 -- 172.21.15.97:0/100923526 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8004c80 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.506+0000 7f27b27fc700 1 -- 172.21.15.97:0/100923526 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b8004e00 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.506+0000 7f27c41ff700 1 -- 172.21.15.97:0/100923526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27bc1e4400 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.506+0000 7f27b27fc700 1 -- 172.21.15.97:0/100923526 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8007310 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.506+0000 7f27c41ff700 1 -- 172.21.15.97:0/100923526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27bc1e49c0 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.507+0000 7f27c41ff700 1 -- 172.21.15.97:0/100923526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27bc074350 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.508+0000 7f27b27fc700 1 -- 172.21.15.97:0/100923526 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f27b8007470 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.509+0000 7f27b27fc700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 0x7f27a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.509+0000 7f27b27fc700 1 -- 172.21.15.97:0/100923526 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f27b8089220 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.510+0000 7f27c179a700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 0x7f27a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.510+0000 7f27c179a700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 0x7f27a006d190 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0b59a0 tx=0x7f27ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:05.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.515+0000 7f27b27fc700 1 -- 172.21.15.97:0/100923526 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f27b8054160 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.715+0000 7f27c41ff700 1 -- 172.21.15.97:0/100923526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27bc06b580 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.715+0000 7f27b27fc700 1 -- 172.21.15.97:0/100923526 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27b8057780 con 0x7f27bc0b1fc0 2023-07-06T03:53:05.719 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:53:05.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 -- 172.21.15.97:0/100923526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 msgr2=0x7f27a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 0x7f27a006d190 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0b59a0 tx=0x7f27ac008040 comp rx=0 tx=0).stop 2023-07-06T03:53:05.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 -- 172.21.15.97:0/100923526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc0b1fc0 msgr2=0x7f27bc155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc0b1fc0 0x7f27bc155490 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f27b80011e0 tx=0x7f27b800ca30 comp rx=0 tx=0).stop 2023-07-06T03:53:05.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 -- 172.21.15.97:0/100923526 shutdown_connections 2023-07-06T03:53:05.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f27a006ace0 0x7f27a006d190 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f27bc0cffc0 0x7f27bc0b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27bc0b1fc0 0x7f27bc155490 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 --2- 172.21.15.97:0/100923526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27bc0b2390 0x7f27bc0c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 -- 172.21.15.97:0/100923526 >> 172.21.15.97:0/100923526 conn(0x7f27bc06c210 msgr2=0x7f27bc1e2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:05.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 -- 172.21.15.97:0/100923526 shutdown_connections 2023-07-06T03:53:05.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.718+0000 7f27c41ff700 1 -- 172.21.15.97:0/100923526 wait complete. 2023-07-06T03:53:05.739 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:53:05.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.980+0000 7f8e849fe700 1 Processor -- start 2023-07-06T03:53:05.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.981+0000 7f8e849fe700 1 -- start start 2023-07-06T03:53:05.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e849fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e8015c730 0x7f8e800d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e849fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8e800d0d50 0x7f8e800b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e849fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e800b11e0 0x7f8e800b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e849fe700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e800d11c0 con 0x7f8e800d0d50 2023-07-06T03:53:05.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e849fe700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e800cf340 con 0x7f8e8015c730 2023-07-06T03:53:05.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e849fe700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800b0810 con 0x7f8e800b11e0 2023-07-06T03:53:05.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e7e59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8e800d0d50 0x7f8e800b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e7e59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8e800d0d50 0x7f8e800b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36102/0 (socket says 172.21.15.97:36102) 2023-07-06T03:53:05.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e7e59c700 1 -- 172.21.15.97:0/2364276797 learned_addr learned my addr 172.21.15.97:0/2364276797 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:05.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e7ed9d700 1 --2- 172.21.15.97:0/2364276797 >> v2:172.21.15.146:3300/0 conn(0x7f8e8015c730 0x7f8e800d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e7f59e700 1 --2- 172.21.15.97:0/2364276797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e800b11e0 0x7f8e800b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.982+0000 7f8e7e59c700 1 -- 172.21.15.97:0/2364276797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e800b11e0 msgr2=0x7f8e800b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:05.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.983+0000 7f8e7e59c700 1 --2- 172.21.15.97:0/2364276797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e800b11e0 0x7f8e800b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.983+0000 7f8e7e59c700 1 -- 172.21.15.97:0/2364276797 >> v2:172.21.15.146:3300/0 conn(0x7f8e8015c730 msgr2=0x7f8e800d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:05.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.983+0000 7f8e7e59c700 1 --2- 172.21.15.97:0/2364276797 >> v2:172.21.15.146:3300/0 conn(0x7f8e8015c730 0x7f8e800d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.983+0000 7f8e7e59c700 1 -- 172.21.15.97:0/2364276797 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8015cb00 con 0x7f8e800d0d50 2023-07-06T03:53:05.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.983+0000 7f8e7e59c700 1 --2- 172.21.15.97:0/2364276797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e800d0d50 0x7f8e800b0ca0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8e8008c150 tx=0x7f8e74009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b7a4bf2a650dd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.983+0000 7f8e7dd9b700 1 -- 172.21.15.97:0/2364276797 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400b860 con 0x7f8e800d0d50 2023-07-06T03:53:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.983+0000 7f8e7dd9b700 1 -- 172.21.15.97:0/2364276797 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7400b9c0 con 0x7f8e800d0d50 2023-07-06T03:53:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.983+0000 7f8e7dd9b700 1 -- 172.21.15.97:0/2364276797 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400bd50 con 0x7f8e800d0d50 2023-07-06T03:53:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.984+0000 7f8e849fe700 1 -- 172.21.15.97:0/2364276797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e800d0d50 msgr2=0x7f8e800b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.984+0000 7f8e849fe700 1 --2- 172.21.15.97:0/2364276797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e800d0d50 0x7f8e800b0ca0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8e8008c150 tx=0x7f8e74009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:05.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.984+0000 7f8e849fe700 1 -- 172.21.15.97:0/2364276797 shutdown_connections 2023-07-06T03:53:05.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.984+0000 7f8e849fe700 1 --2- 172.21.15.97:0/2364276797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e800b11e0 0x7f8e800b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.984+0000 7f8e849fe700 1 --2- 172.21.15.97:0/2364276797 >> v2:172.21.15.146:3300/0 conn(0x7f8e8015c730 0x7f8e800d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.984+0000 7f8e849fe700 1 --2- 172.21.15.97:0/2364276797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e800d0d50 0x7f8e800b0ca0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.984+0000 7f8e849fe700 1 -- 172.21.15.97:0/2364276797 >> 172.21.15.97:0/2364276797 conn(0x7f8e8009c000 msgr2=0x7f8e800d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.984+0000 7f8e849fe700 1 -- 172.21.15.97:0/2364276797 shutdown_connections 2023-07-06T03:53:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.984+0000 7f8e849fe700 1 -- 172.21.15.97:0/2364276797 wait complete. 2023-07-06T03:53:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.986+0000 7f8e849fe700 1 Processor -- start 2023-07-06T03:53:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.986+0000 7f8e849fe700 1 -- start start 2023-07-06T03:53:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e849fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e800b11e0 0x7f8e801f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e849fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e800d0d50 0x7f8e800c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e849fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e8015c730 0x7f8e800c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e849fe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80142dd0 con 0x7f8e800d0d50 2023-07-06T03:53:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e849fe700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8013a700 con 0x7f8e8015c730 2023-07-06T03:53:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e849fe700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8013afa0 con 0x7f8e800b11e0 2023-07-06T03:53:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e7e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e800d0d50 0x7f8e800c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e7e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e800d0d50 0x7f8e800c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36108/0 (socket says 172.21.15.97:36108) 2023-07-06T03:53:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e7e59c700 1 -- 172.21.15.97:0/3481814755 learned_addr learned my addr 172.21.15.97:0/3481814755 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e7f59e700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e8015c730 0x7f8e800c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e7ed9d700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e800b11e0 0x7f8e801f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e7e59c700 1 -- 172.21.15.97:0/3481814755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e800b11e0 msgr2=0x7f8e801f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e7e59c700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e800b11e0 0x7f8e801f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e7e59c700 1 -- 172.21.15.97:0/3481814755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e8015c730 msgr2=0x7f8e800c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:05.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e7e59c700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e8015c730 0x7f8e800c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:05.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e7e59c700 1 -- 172.21.15.97:0/3481814755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e74009550 con 0x7f8e800d0d50 2023-07-06T03:53:05.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.987+0000 7f8e7f59e700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e8015c730 0x7f8e800c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:05.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.988+0000 7f8e7e59c700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e800d0d50 0x7f8e800c6a30 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f8e74000c00 tx=0x7f8e74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:05.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.988+0000 7f8e677fe700 1 -- 172.21.15.97:0/3481814755 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74004380 con 0x7f8e800d0d50 2023-07-06T03:53:05.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.988+0000 7f8e677fe700 1 -- 172.21.15.97:0/3481814755 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7401f070 con 0x7f8e800d0d50 2023-07-06T03:53:05.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.988+0000 7f8e677fe700 1 -- 172.21.15.97:0/3481814755 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e740036a0 con 0x7f8e800d0d50 2023-07-06T03:53:05.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.988+0000 7f8e849fe700 1 -- 172.21.15.97:0/3481814755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e80157270 con 0x7f8e800d0d50 2023-07-06T03:53:05.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.988+0000 7f8e849fe700 1 -- 172.21.15.97:0/3481814755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8013ac30 con 0x7f8e800d0d50 2023-07-06T03:53:05.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.989+0000 7f8e849fe700 1 -- 172.21.15.97:0/3481814755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8004eb90 con 0x7f8e800d0d50 2023-07-06T03:53:05.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.990+0000 7f8e677fe700 1 -- 172.21.15.97:0/3481814755 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e74021030 con 0x7f8e800d0d50 2023-07-06T03:53:05.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.991+0000 7f8e677fe700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8e6c06ace0 0x7f8e6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:05.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.991+0000 7f8e677fe700 1 -- 172.21.15.97:0/3481814755 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f8e7408a950 con 0x7f8e800d0d50 2023-07-06T03:53:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.991+0000 7f8e7ed9d700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8e6c06ace0 0x7f8e6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.992+0000 7f8e7ed9d700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8e6c06ace0 0x7f8e6c06d190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8e70006fd0 tx=0x7f8e70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:05.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:05.995+0000 7f8e677fe700 1 -- 172.21.15.97:0/3481814755 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8e74055a40 con 0x7f8e800d0d50 2023-07-06T03:53:06.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.196+0000 7f8e849fe700 1 -- 172.21.15.97:0/3481814755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8e8006a230 con 0x7f8e800d0d50 2023-07-06T03:53:06.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.197+0000 7f8e677fe700 1 -- 172.21.15.97:0/3481814755 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v88) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8e74015020 con 0x7f8e800d0d50 2023-07-06T03:53:06.200 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:53:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.199+0000 7f8e849fe700 1 -- 172.21.15.97:0/3481814755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8e6c06ace0 msgr2=0x7f8e6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.199+0000 7f8e849fe700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8e6c06ace0 0x7f8e6c06d190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8e70006fd0 tx=0x7f8e70008040 comp rx=0 tx=0).stop 2023-07-06T03:53:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.199+0000 7f8e849fe700 1 -- 172.21.15.97:0/3481814755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e800d0d50 msgr2=0x7f8e800c6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.199+0000 7f8e849fe700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e800d0d50 0x7f8e800c6a30 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f8e74000c00 tx=0x7f8e74004750 comp rx=0 tx=0).stop 2023-07-06T03:53:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.199+0000 7f8e849fe700 1 -- 172.21.15.97:0/3481814755 shutdown_connections 2023-07-06T03:53:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.199+0000 7f8e849fe700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8e6c06ace0 0x7f8e6c06d190 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.199+0000 7f8e849fe700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e8015c730 0x7f8e800c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.199+0000 7f8e849fe700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8e800b11e0 0x7f8e801f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.199+0000 7f8e849fe700 1 --2- 172.21.15.97:0/3481814755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e800d0d50 0x7f8e800c6a30 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.199+0000 7f8e849fe700 1 -- 172.21.15.97:0/3481814755 >> 172.21.15.97:0/3481814755 conn(0x7f8e8009c000 msgr2=0x7f8e80159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.199+0000 7f8e849fe700 1 -- 172.21.15.97:0/3481814755 shutdown_connections 2023-07-06T03:53:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.200+0000 7f8e849fe700 1 -- 172.21.15.97:0/3481814755 wait complete. 2023-07-06T03:53:06.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:53:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.414+0000 7fbc423ff700 1 Processor -- start 2023-07-06T03:53:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.415+0000 7fbc423ff700 1 -- start start 2023-07-06T03:53:06.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.416+0000 7fbc423ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc3c0b1fc0 0x7fbc3c0cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.416+0000 7fbc423ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc3c0b2390 0x7fbc3c0cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.416+0000 7fbc423ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3c15d610 con 0x7fbc3c0cf080 2023-07-06T03:53:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.416+0000 7fbc423ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3c15db40 con 0x7fbc3c0b2390 2023-07-06T03:53:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.416+0000 7fbc423ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c15d860 con 0x7fbc3c0b1fc0 2023-07-06T03:53:06.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.416+0000 7fbc3b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc3c0b2390 0x7fbc3c0cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.416+0000 7fbc3b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc3c0b2390 0x7fbc3c0cfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36124/0 (socket says 172.21.15.97:36124) 2023-07-06T03:53:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.416+0000 7fbc4099c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbc3c0cf080 0x7fbc3c0b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45638/0 (socket says 172.21.15.97:45638) 2023-07-06T03:53:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.416+0000 7fbc3b7fe700 1 -- 172.21.15.97:0/2585195801 learned_addr learned my addr 172.21.15.97:0/2585195801 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.416+0000 7fbc3bfff700 1 --2- 172.21.15.97:0/2585195801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc3c0b1fc0 0x7fbc3c0cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc3b7fe700 1 -- 172.21.15.97:0/2585195801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc3c0b1fc0 msgr2=0x7fbc3c0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc3b7fe700 1 --2- 172.21.15.97:0/2585195801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc3c0b1fc0 0x7fbc3c0cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc3b7fe700 1 -- 172.21.15.97:0/2585195801 >> v1:172.21.15.97:6789/0 conn(0x7fbc3c0cf080 legacy=0x7fbc3c0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc3b7fe700 1 -- 172.21.15.97:0/2585195801 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c0cd630 con 0x7fbc3c0b2390 2023-07-06T03:53:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc3bfff700 1 --2- 172.21.15.97:0/2585195801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc3c0b1fc0 0x7fbc3c0cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc3affd700 1 -- 172.21.15.97:0/2585195801 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701388281 0 0) 0x7fbc3c15d610 con 0x7fbc3c0cf080 2023-07-06T03:53:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc3b7fe700 1 --2- 172.21.15.97:0/2585195801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c0b2390 0x7fbc3c0cfdf0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fbc300098f0 tx=0x7fbc30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37b82b9959b7b6a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc3affd700 1 -- 172.21.15.97:0/2585195801 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000b8a0 con 0x7fbc3c0b2390 2023-07-06T03:53:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc3affd700 1 -- 172.21.15.97:0/2585195801 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3000ba00 con 0x7fbc3c0b2390 2023-07-06T03:53:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc3affd700 1 -- 172.21.15.97:0/2585195801 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000bd90 con 0x7fbc3c0b2390 2023-07-06T03:53:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc423ff700 1 -- 172.21.15.97:0/2585195801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c0b2390 msgr2=0x7fbc3c0cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.417+0000 7fbc423ff700 1 --2- 172.21.15.97:0/2585195801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c0b2390 0x7fbc3c0cfdf0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fbc300098f0 tx=0x7fbc30009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.418+0000 7fbc423ff700 1 -- 172.21.15.97:0/2585195801 shutdown_connections 2023-07-06T03:53:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.418+0000 7fbc423ff700 1 --2- 172.21.15.97:0/2585195801 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc3c0b1fc0 0x7fbc3c0cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.418+0000 7fbc423ff700 1 --2- 172.21.15.97:0/2585195801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c0b2390 0x7fbc3c0cfdf0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.418+0000 7fbc423ff700 1 -- 172.21.15.97:0/2585195801 >> 172.21.15.97:0/2585195801 conn(0x7fbc3c06c210 msgr2=0x7fbc3c06c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.418+0000 7fbc423ff700 1 -- 172.21.15.97:0/2585195801 shutdown_connections 2023-07-06T03:53:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.418+0000 7fbc423ff700 1 -- 172.21.15.97:0/2585195801 wait complete. 2023-07-06T03:53:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.419+0000 7fbc423ff700 1 Processor -- start 2023-07-06T03:53:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.420+0000 7fbc423ff700 1 -- start start 2023-07-06T03:53:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.420+0000 7fbc423ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc3c0b1fc0 0x7fbc3c1f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.420+0000 7fbc423ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c0b2390 0x7fbc3c0cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc423ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc3c0cf080 0x7fbc3c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc423ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0c9780 con 0x7fbc3c0b2390 2023-07-06T03:53:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc423ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0b28f0 con 0x7fbc3c0b1fc0 2023-07-06T03:53:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc423ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c1f5500 con 0x7fbc3c0cf080 2023-07-06T03:53:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc3b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c0b2390 0x7fbc3c0cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc4099c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc3c0cf080 0x7fbc3c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc3bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc3c0b1fc0 0x7fbc3c1f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc3b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c0b2390 0x7fbc3c0cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36126/0 (socket says 172.21.15.97:36126) 2023-07-06T03:53:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc3bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc3c0b1fc0 0x7fbc3c1f4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:36482/0 (socket says 172.21.15.97:36482) 2023-07-06T03:53:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc3b7fe700 1 -- 172.21.15.97:0/1756152366 learned_addr learned my addr 172.21.15.97:0/1756152366 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc3b7fe700 1 -- 172.21.15.97:0/1756152366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc3c0cf080 msgr2=0x7fbc3c0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc3b7fe700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc3c0cf080 0x7fbc3c0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc3b7fe700 1 -- 172.21.15.97:0/1756152366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc3c0b1fc0 msgr2=0x7fbc3c1f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc3b7fe700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc3c0b1fc0 0x7fbc3c1f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.421+0000 7fbc3b7fe700 1 -- 172.21.15.97:0/1756152366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc30009550 con 0x7fbc3c0b2390 2023-07-06T03:53:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.422+0000 7fbc3b7fe700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c0b2390 0x7fbc3c0cc170 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fbc30009c40 tx=0x7fbc30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.422+0000 7fbc38ff9700 1 -- 172.21.15.97:0/1756152366 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc30004240 con 0x7fbc3c0b2390 2023-07-06T03:53:06.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.422+0000 7fbc38ff9700 1 -- 172.21.15.97:0/1756152366 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3001f070 con 0x7fbc3c0b2390 2023-07-06T03:53:06.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.422+0000 7fbc423ff700 1 -- 172.21.15.97:0/1756152366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c1f57e0 con 0x7fbc3c0b2390 2023-07-06T03:53:06.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.422+0000 7fbc38ff9700 1 -- 172.21.15.97:0/1756152366 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc300045a0 con 0x7fbc3c0b2390 2023-07-06T03:53:06.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.422+0000 7fbc423ff700 1 -- 172.21.15.97:0/1756152366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c1f5da0 con 0x7fbc3c0b2390 2023-07-06T03:53:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.423+0000 7fbc423ff700 1 -- 172.21.15.97:0/1756152366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3c074350 con 0x7fbc3c0b2390 2023-07-06T03:53:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.424+0000 7fbc38ff9700 1 -- 172.21.15.97:0/1756152366 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbc30021030 con 0x7fbc3c0b2390 2023-07-06T03:53:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.425+0000 7fbc38ff9700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc2806ace0 0x7fbc2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.425+0000 7fbc38ff9700 1 -- 172.21.15.97:0/1756152366 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fbc3008a840 con 0x7fbc3c0b2390 2023-07-06T03:53:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.425+0000 7fbc3bfff700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc2806ace0 0x7fbc2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.426+0000 7fbc3bfff700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc2806ace0 0x7fbc2806d190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c006f10 tx=0x7fbc2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.429+0000 7fbc38ff9700 1 -- 172.21.15.97:0/1756152366 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbc30055790 con 0x7fbc3c0b2390 2023-07-06T03:53:06.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.621+0000 7fbc423ff700 1 -- 172.21.15.97:0/1756152366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc3c06b580 con 0x7fbc3c0b2390 2023-07-06T03:53:06.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.621+0000 7fbc38ff9700 1 -- 172.21.15.97:0/1756152366 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc30058db0 con 0x7fbc3c0b2390 2023-07-06T03:53:06.625 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:53:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 -- 172.21.15.97:0/1756152366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc2806ace0 msgr2=0x7fbc2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc2806ace0 0x7fbc2806d190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c006f10 tx=0x7fbc2c008040 comp rx=0 tx=0).stop 2023-07-06T03:53:06.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 -- 172.21.15.97:0/1756152366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c0b2390 msgr2=0x7fbc3c0cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c0b2390 0x7fbc3c0cc170 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fbc30009c40 tx=0x7fbc30004750 comp rx=0 tx=0).stop 2023-07-06T03:53:06.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 -- 172.21.15.97:0/1756152366 shutdown_connections 2023-07-06T03:53:06.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbc2806ace0 0x7fbc2806d190 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbc3c0cf080 0x7fbc3c0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc3c0b1fc0 0x7fbc3c1f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 --2- 172.21.15.97:0/1756152366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc3c0b2390 0x7fbc3c0cc170 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 -- 172.21.15.97:0/1756152366 >> 172.21.15.97:0/1756152366 conn(0x7fbc3c06c210 msgr2=0x7fbc3c0b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:06.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 -- 172.21.15.97:0/1756152366 shutdown_connections 2023-07-06T03:53:06.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.624+0000 7fbc423ff700 1 -- 172.21.15.97:0/1756152366 wait complete. 2023-07-06T03:53:06.644 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:53:06.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.840+0000 7feea27ff700 1 Processor -- start 2023-07-06T03:53:06.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.841+0000 7feea27ff700 1 -- start start 2023-07-06T03:53:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.842+0000 7feea27ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fee9c0b1a90 0x7fee9c0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.842+0000 7feea27ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee9c06c1e0 0x7fee9c06c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.842+0000 7feea27ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9c15de20 con 0x7fee9c15d9a0 2023-07-06T03:53:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.842+0000 7feea27ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9c15d510 con 0x7fee9c0b1a90 2023-07-06T03:53:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.842+0000 7feea27ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c06cb40 con 0x7fee9c06c1e0 2023-07-06T03:53:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.843+0000 7fee9bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fee9c0b1a90 0x7fee9c0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.843+0000 7fee9bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fee9c0b1a90 0x7fee9c0b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36128/0 (socket says 172.21.15.97:36128) 2023-07-06T03:53:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.843+0000 7fee9bfff700 1 -- 172.21.15.97:0/2373401213 learned_addr learned my addr 172.21.15.97:0/2373401213 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:06.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.843+0000 7feea0d9c700 1 --2- 172.21.15.97:0/2373401213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee9c06c1e0 0x7fee9c06c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.843+0000 7fee9bfff700 1 -- 172.21.15.97:0/2373401213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee9c06c1e0 msgr2=0x7fee9c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.843+0000 7fee9bfff700 1 --2- 172.21.15.97:0/2373401213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee9c06c1e0 0x7fee9c06c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.843+0000 7fee9bfff700 1 -- 172.21.15.97:0/2373401213 >> v1:172.21.15.97:6789/0 conn(0x7fee9c15d9a0 legacy=0x7fee9c160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.843+0000 7fee9bfff700 1 -- 172.21.15.97:0/2373401213 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c0b2660 con 0x7fee9c0b1a90 2023-07-06T03:53:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.843+0000 7fee9affd700 1 -- 172.21.15.97:0/2373401213 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081518975 0 0) 0x7fee90001110 con 0x7fee9c15d9a0 2023-07-06T03:53:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7fee9bfff700 1 --2- 172.21.15.97:0/2373401213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee9c0b1a90 0x7fee9c0b1e60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0098f0 tx=0x7fee8c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae093d358900d1ac server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7fee9affd700 1 -- 172.21.15.97:0/2373401213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c011830 con 0x7fee9c0b1a90 2023-07-06T03:53:06.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7fee9affd700 1 -- 172.21.15.97:0/2373401213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c011990 con 0x7fee9c0b1a90 2023-07-06T03:53:06.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7fee9affd700 1 -- 172.21.15.97:0/2373401213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c011d20 con 0x7fee9c0b1a90 2023-07-06T03:53:06.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7feea27ff700 1 -- 172.21.15.97:0/2373401213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee9c0b1a90 msgr2=0x7fee9c0b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7feea27ff700 1 --2- 172.21.15.97:0/2373401213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee9c0b1a90 0x7fee9c0b1e60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0098f0 tx=0x7fee8c009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:06.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7feea27ff700 1 -- 172.21.15.97:0/2373401213 shutdown_connections 2023-07-06T03:53:06.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7feea27ff700 1 --2- 172.21.15.97:0/2373401213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee9c0b1a90 0x7fee9c0b1e60 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7feea27ff700 1 --2- 172.21.15.97:0/2373401213 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee9c06c1e0 0x7fee9c06c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7feea27ff700 1 -- 172.21.15.97:0/2373401213 >> 172.21.15.97:0/2373401213 conn(0x7fee9c0d2f50 msgr2=0x7fee9c0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:06.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7feea27ff700 1 -- 172.21.15.97:0/2373401213 shutdown_connections 2023-07-06T03:53:06.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.844+0000 7feea27ff700 1 -- 172.21.15.97:0/2373401213 wait complete. 2023-07-06T03:53:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.846+0000 7feea27ff700 1 Processor -- start 2023-07-06T03:53:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.846+0000 7feea27ff700 1 -- start start 2023-07-06T03:53:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.847+0000 7feea27ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee9c06c1e0 0x7fee9c155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.847+0000 7feea27ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee9c0b1a90 0x7fee9c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.847+0000 7feea27ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee9c15d9a0 0x7fee9c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.847+0000 7feea27ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0cb1a0 con 0x7fee9c06c1e0 2023-07-06T03:53:06.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.847+0000 7feea27ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c155590 con 0x7fee9c0b1a90 2023-07-06T03:53:06.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.847+0000 7feea27ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0cb440 con 0x7fee9c15d9a0 2023-07-06T03:53:06.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.847+0000 7fee9bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee9c06c1e0 0x7fee9c155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee9bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee9c06c1e0 0x7fee9c155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36142/0 (socket says 172.21.15.97:36142) 2023-07-06T03:53:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee9bfff700 1 -- 172.21.15.97:0/1579983275 learned_addr learned my addr 172.21.15.97:0/1579983275 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee9b7fe700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee9c0b1a90 0x7fee9c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7feea0d9c700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee9c15d9a0 0x7fee9c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee9bfff700 1 -- 172.21.15.97:0/1579983275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee9c15d9a0 msgr2=0x7fee9c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee9bfff700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee9c15d9a0 0x7fee9c0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee9bfff700 1 -- 172.21.15.97:0/1579983275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee9c0b1a90 msgr2=0x7fee9c0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee9bfff700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee9c0b1a90 0x7fee9c0cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee9bfff700 1 -- 172.21.15.97:0/1579983275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c009550 con 0x7fee9c06c1e0 2023-07-06T03:53:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee9b7fe700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee9c0b1a90 0x7fee9c0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee9bfff700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee9c06c1e0 0x7fee9c155050 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009c40 tx=0x7fee8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee98ff9700 1 -- 172.21.15.97:0/1579983275 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c004480 con 0x7fee9c06c1e0 2023-07-06T03:53:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7fee98ff9700 1 -- 172.21.15.97:0/1579983275 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c01f070 con 0x7fee9c06c1e0 2023-07-06T03:53:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.848+0000 7feea27ff700 1 -- 172.21.15.97:0/1579983275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c0cb720 con 0x7fee9c06c1e0 2023-07-06T03:53:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.849+0000 7fee98ff9700 1 -- 172.21.15.97:0/1579983275 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0038c0 con 0x7fee9c06c1e0 2023-07-06T03:53:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.849+0000 7feea27ff700 1 -- 172.21.15.97:0/1579983275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c0cbce0 con 0x7fee9c06c1e0 2023-07-06T03:53:06.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.850+0000 7feea27ff700 1 -- 172.21.15.97:0/1579983275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9c0d3320 con 0x7fee9c06c1e0 2023-07-06T03:53:06.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.851+0000 7fee98ff9700 1 -- 172.21.15.97:0/1579983275 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fee8c021030 con 0x7fee9c06c1e0 2023-07-06T03:53:06.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.852+0000 7fee98ff9700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fee8806ace0 0x7fee8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.852+0000 7fee98ff9700 1 -- 172.21.15.97:0/1579983275 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fee8c08aa70 con 0x7fee9c06c1e0 2023-07-06T03:53:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.852+0000 7fee9b7fe700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fee8806ace0 0x7fee8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.853+0000 7fee9b7fe700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fee8806ace0 0x7fee8806d190 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0b8df0 tx=0x7fee90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:06.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:06.856+0000 7fee98ff9700 1 -- 172.21.15.97:0/1579983275 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fee8c055a40 con 0x7fee9c06c1e0 2023-07-06T03:53:07.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.060+0000 7feea27ff700 1 -- 172.21.15.97:0/1579983275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee9c0cbed0 con 0x7fee9c06c1e0 2023-07-06T03:53:07.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.060+0000 7fee98ff9700 1 -- 172.21.15.97:0/1579983275 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee8c0150a0 con 0x7fee9c06c1e0 2023-07-06T03:53:07.064 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:53:07.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 -- 172.21.15.97:0/1579983275 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fee8806ace0 msgr2=0x7fee8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:07.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fee8806ace0 0x7fee8806d190 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0b8df0 tx=0x7fee90009040 comp rx=0 tx=0).stop 2023-07-06T03:53:07.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 -- 172.21.15.97:0/1579983275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee9c06c1e0 msgr2=0x7fee9c155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:07.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee9c06c1e0 0x7fee9c155050 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009c40 tx=0x7fee8c004750 comp rx=0 tx=0).stop 2023-07-06T03:53:07.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 -- 172.21.15.97:0/1579983275 shutdown_connections 2023-07-06T03:53:07.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fee8806ace0 0x7fee8806d190 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fee9c15d9a0 0x7fee9c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee9c06c1e0 0x7fee9c155050 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 --2- 172.21.15.97:0/1579983275 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee9c0b1a90 0x7fee9c0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 -- 172.21.15.97:0/1579983275 >> 172.21.15.97:0/1579983275 conn(0x7fee9c0d2f50 msgr2=0x7fee9c15c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:07.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 -- 172.21.15.97:0/1579983275 shutdown_connections 2023-07-06T03:53:07.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.063+0000 7feea27ff700 1 -- 172.21.15.97:0/1579983275 wait complete. 2023-07-06T03:53:07.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:53:07.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.277+0000 7faa719ff700 1 Processor -- start 2023-07-06T03:53:07.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.277+0000 7faa719ff700 1 -- start start 2023-07-06T03:53:07.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa719ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa6c15c8b0 0x7faa6c0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:07.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa719ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faa6c0b5550 0x7faa6c0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:07.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa719ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c0cdaf0 con 0x7faa6c0b4e10 2023-07-06T03:53:07.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa719ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6c0b5250 con 0x7faa6c15c8b0 2023-07-06T03:53:07.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa719ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6c0cff30 con 0x7faa6c0b5550 2023-07-06T03:53:07.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa6affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa6c15c8b0 0x7faa6c0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:07.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa6affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa6c15c8b0 0x7faa6c0b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36148/0 (socket says 172.21.15.97:36148) 2023-07-06T03:53:07.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa6affd700 1 -- 172.21.15.97:0/3318050266 learned_addr learned my addr 172.21.15.97:0/3318050266 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:07.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa63fff700 1 --2- 172.21.15.97:0/3318050266 >> v2:172.21.15.146:3300/0 conn(0x7faa6c0b5550 0x7faa6c0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:07.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa6affd700 1 -- 172.21.15.97:0/3318050266 >> v2:172.21.15.146:3300/0 conn(0x7faa6c0b5550 msgr2=0x7faa6c0b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:07.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa6affd700 1 --2- 172.21.15.97:0/3318050266 >> v2:172.21.15.146:3300/0 conn(0x7faa6c0b5550 0x7faa6c0b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa6affd700 1 -- 172.21.15.97:0/3318050266 >> v1:172.21.15.146:6789/0 conn(0x7faa6c0b4e10 legacy=0x7faa6c1601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:07.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa6affd700 1 -- 172.21.15.97:0/3318050266 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c0d0210 con 0x7faa6c15c8b0 2023-07-06T03:53:07.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.279+0000 7faa63fff700 1 --2- 172.21.15.97:0/3318050266 >> v2:172.21.15.146:3300/0 conn(0x7faa6c0b5550 0x7faa6c0b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:07.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.280+0000 7faa6affd700 1 --2- 172.21.15.97:0/3318050266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6c15c8b0 0x7faa6c0b2280 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7faa6c08bbe0 tx=0x7faa5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531dcd6c46bd42b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:07.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.280+0000 7faa6a7fc700 1 -- 172.21.15.97:0/3318050266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c00b870 con 0x7faa6c15c8b0 2023-07-06T03:53:07.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.280+0000 7faa6a7fc700 1 -- 172.21.15.97:0/3318050266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c00b9d0 con 0x7faa6c15c8b0 2023-07-06T03:53:07.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.280+0000 7faa6a7fc700 1 -- 172.21.15.97:0/3318050266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c00bd60 con 0x7faa6c15c8b0 2023-07-06T03:53:07.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.280+0000 7faa719ff700 1 -- 172.21.15.97:0/3318050266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6c15c8b0 msgr2=0x7faa6c0b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:07.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.280+0000 7faa719ff700 1 --2- 172.21.15.97:0/3318050266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6c15c8b0 0x7faa6c0b2280 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7faa6c08bbe0 tx=0x7faa5c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.281+0000 7faa719ff700 1 -- 172.21.15.97:0/3318050266 shutdown_connections 2023-07-06T03:53:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.281+0000 7faa719ff700 1 --2- 172.21.15.97:0/3318050266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6c15c8b0 0x7faa6c0b2280 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.281+0000 7faa719ff700 1 --2- 172.21.15.97:0/3318050266 >> v2:172.21.15.146:3300/0 conn(0x7faa6c0b5550 0x7faa6c0b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.281+0000 7faa719ff700 1 -- 172.21.15.97:0/3318050266 >> 172.21.15.97:0/3318050266 conn(0x7faa6c066990 msgr2=0x7faa6c06a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.281+0000 7faa719ff700 1 -- 172.21.15.97:0/3318050266 shutdown_connections 2023-07-06T03:53:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.281+0000 7faa719ff700 1 -- 172.21.15.97:0/3318050266 wait complete. 2023-07-06T03:53:07.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.282+0000 7faa719ff700 1 Processor -- start 2023-07-06T03:53:07.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.282+0000 7faa719ff700 1 -- start start 2023-07-06T03:53:07.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.283+0000 7faa719ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6c0b4e10 0x7faa6c0cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:07.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.283+0000 7faa719ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6c0b5550 0x7faa6c1503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:07.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.283+0000 7faa719ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa6c15c8b0 0x7faa6c0b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:07.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.283+0000 7faa719ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0c9f20 con 0x7faa6c0b4e10 2023-07-06T03:53:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.283+0000 7faa719ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0b87f0 con 0x7faa6c15c8b0 2023-07-06T03:53:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.283+0000 7faa719ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0ca130 con 0x7faa6c0b5550 2023-07-06T03:53:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa6affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6c0b4e10 0x7faa6c0cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa6affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6c0b4e10 0x7faa6c0cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36150/0 (socket says 172.21.15.97:36150) 2023-07-06T03:53:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa6affd700 1 -- 172.21.15.97:0/2122974769 learned_addr learned my addr 172.21.15.97:0/2122974769 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa63fff700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6c0b5550 0x7faa6c1503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa6b7fe700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa6c15c8b0 0x7faa6c0b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa6affd700 1 -- 172.21.15.97:0/2122974769 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6c0b5550 msgr2=0x7faa6c1503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa6affd700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6c0b5550 0x7faa6c1503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa6affd700 1 -- 172.21.15.97:0/2122974769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa6c15c8b0 msgr2=0x7faa6c0b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa6affd700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa6c15c8b0 0x7faa6c0b7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa6affd700 1 -- 172.21.15.97:0/2122974769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c009550 con 0x7faa6c0b4e10 2023-07-06T03:53:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa6b7fe700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa6c15c8b0 0x7faa6c0b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa6affd700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6c0b4e10 0x7faa6c0cac10 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7faa5c000c00 tx=0x7faa5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:07.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.284+0000 7faa68ff9700 1 -- 172.21.15.97:0/2122974769 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c004520 con 0x7faa6c0b4e10 2023-07-06T03:53:07.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.285+0000 7faa68ff9700 1 -- 172.21.15.97:0/2122974769 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c01e070 con 0x7faa6c0b4e10 2023-07-06T03:53:07.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.285+0000 7faa719ff700 1 -- 172.21.15.97:0/2122974769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6c0ca410 con 0x7faa6c0b4e10 2023-07-06T03:53:07.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.285+0000 7faa68ff9700 1 -- 172.21.15.97:0/2122974769 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0038a0 con 0x7faa6c0b4e10 2023-07-06T03:53:07.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.285+0000 7faa719ff700 1 -- 172.21.15.97:0/2122974769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6c1f34a0 con 0x7faa6c0b4e10 2023-07-06T03:53:07.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.286+0000 7faa719ff700 1 -- 172.21.15.97:0/2122974769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6c0c67f0 con 0x7faa6c0b4e10 2023-07-06T03:53:07.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.287+0000 7faa68ff9700 1 -- 172.21.15.97:0/2122974769 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faa5c003a00 con 0x7faa6c0b4e10 2023-07-06T03:53:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.287+0000 7faa68ff9700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa4c06ace0 0x7faa4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.288+0000 7faa68ff9700 1 -- 172.21.15.97:0/2122974769 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7faa5c020070 con 0x7faa6c0b4e10 2023-07-06T03:53:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.288+0000 7faa63fff700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa4c06ace0 0x7faa4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.288+0000 7faa63fff700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa4c06ace0 0x7faa4c06d190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0b6df0 tx=0x7faa54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.292+0000 7faa68ff9700 1 -- 172.21.15.97:0/2122974769 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faa5c055a40 con 0x7faa6c0b4e10 2023-07-06T03:53:07.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.503+0000 7faa719ff700 1 -- 172.21.15.97:0/2122974769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa6c0ca930 con 0x7faa6c0b4e10 2023-07-06T03:53:07.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.504+0000 7faa68ff9700 1 -- 172.21.15.97:0/2122974769 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) v1 ==== 80+0+0 (secure 0 0 0) 0x7faa6c0ca930 con 0x7faa6c0b4e10 2023-07-06T03:53:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 -- 172.21.15.97:0/2122974769 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa4c06ace0 msgr2=0x7faa4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa4c06ace0 0x7faa4c06d190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0b6df0 tx=0x7faa54008040 comp rx=0 tx=0).stop 2023-07-06T03:53:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 -- 172.21.15.97:0/2122974769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6c0b4e10 msgr2=0x7faa6c0cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6c0b4e10 0x7faa6c0cac10 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7faa5c000c00 tx=0x7faa5c004750 comp rx=0 tx=0).stop 2023-07-06T03:53:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 -- 172.21.15.97:0/2122974769 shutdown_connections 2023-07-06T03:53:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faa4c06ace0 0x7faa4c06d190 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa6c15c8b0 0x7faa6c0b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6c0b4e10 0x7faa6c0cac10 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 --2- 172.21.15.97:0/2122974769 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faa6c0b5550 0x7faa6c1503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 -- 172.21.15.97:0/2122974769 >> 172.21.15.97:0/2122974769 conn(0x7faa6c066990 msgr2=0x7faa6c0b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:07.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 -- 172.21.15.97:0/2122974769 shutdown_connections 2023-07-06T03:53:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:07.506+0000 7faa719ff700 1 -- 172.21.15.97:0/2122974769 wait complete. 2023-07-06T03:53:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:53:07.527 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2023-07-06T03:53:07.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:07.528 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-06T03:53:07.601 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[124181]: fuse finished with error 0 and tester_r 0 2023-07-06T03:53:07.605 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:53:07.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:07.606 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T03:53:07.731 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:53:07.732 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:53:07.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:53:07.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:53:07.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:07.781 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T03:53:07.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:53:07.866 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T03:53:07.866 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:53:07.867 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:53:07.916 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:53:07.916 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:53:07.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:07.917 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T03:53:07.962 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T03:53:07.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:53:07.964 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:53:08.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.156+0000 7f95b09ff700 1 Processor -- start 2023-07-06T03:53:08.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.157+0000 7f95b09ff700 1 -- start start 2023-07-06T03:53:08.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95b09ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95ac15c8b0 0x7f95ac0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95b09ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f95ac0b4fe0 0x7f95ac0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95b09ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0cdaf0 con 0x7f95ac0b5550 2023-07-06T03:53:08.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95b09ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac0cff30 con 0x7f95ac15c8b0 2023-07-06T03:53:08.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95b09ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac0d0210 con 0x7f95ac0b4fe0 2023-07-06T03:53:08.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95aa59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95ac15c8b0 0x7f95ac0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95a3fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f95ac0b5550 0x7f95ac0b48d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45652/0 (socket says 172.21.15.97:45652) 2023-07-06T03:53:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95aa59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95ac15c8b0 0x7f95ac0b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36152/0 (socket says 172.21.15.97:36152) 2023-07-06T03:53:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95a3fff700 1 -- 172.21.15.97:0/3544215144 learned_addr learned my addr 172.21.15.97:0/3544215144 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95aad9d700 1 --2- 172.21.15.97:0/3544215144 >> v2:172.21.15.146:3300/0 conn(0x7f95ac0b4fe0 0x7f95ac0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95aa59c700 1 -- 172.21.15.97:0/3544215144 >> v2:172.21.15.146:3300/0 conn(0x7f95ac0b4fe0 msgr2=0x7f95ac0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:08.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95aa59c700 1 --2- 172.21.15.97:0/3544215144 >> v2:172.21.15.146:3300/0 conn(0x7f95ac0b4fe0 0x7f95ac0d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95aa59c700 1 -- 172.21.15.97:0/3544215144 >> v1:172.21.15.97:6789/0 conn(0x7f95ac0b5550 legacy=0x7f95ac0b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95aa59c700 1 -- 172.21.15.97:0/3544215144 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0cd4a0 con 0x7f95ac15c8b0 2023-07-06T03:53:08.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.158+0000 7f95a9d9b700 1 -- 172.21.15.97:0/3544215144 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568038827 0 0) 0x7f95ac0cdaf0 con 0x7f95ac0b5550 2023-07-06T03:53:08.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.159+0000 7f95aa59c700 1 --2- 172.21.15.97:0/3544215144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95ac15c8b0 0x7f95ac0b2280 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f95ac06af50 tx=0x7f959c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ff54b42626ed2b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:08.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.159+0000 7f95a9d9b700 1 -- 172.21.15.97:0/3544215144 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c011830 con 0x7f95ac15c8b0 2023-07-06T03:53:08.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.159+0000 7f95a9d9b700 1 -- 172.21.15.97:0/3544215144 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c011990 con 0x7f95ac15c8b0 2023-07-06T03:53:08.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.159+0000 7f95a9d9b700 1 -- 172.21.15.97:0/3544215144 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c011d20 con 0x7f95ac15c8b0 2023-07-06T03:53:08.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.159+0000 7f95b09ff700 1 -- 172.21.15.97:0/3544215144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95ac15c8b0 msgr2=0x7f95ac0b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.159+0000 7f95b09ff700 1 --2- 172.21.15.97:0/3544215144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95ac15c8b0 0x7f95ac0b2280 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f95ac06af50 tx=0x7f959c009b20 comp rx=0 tx=0).stop 2023-07-06T03:53:08.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.160+0000 7f95b09ff700 1 -- 172.21.15.97:0/3544215144 shutdown_connections 2023-07-06T03:53:08.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.160+0000 7f95b09ff700 1 --2- 172.21.15.97:0/3544215144 >> v2:172.21.15.146:3300/0 conn(0x7f95ac0b4fe0 0x7f95ac0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.160+0000 7f95b09ff700 1 --2- 172.21.15.97:0/3544215144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95ac15c8b0 0x7f95ac0b2280 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.160+0000 7f95b09ff700 1 -- 172.21.15.97:0/3544215144 >> 172.21.15.97:0/3544215144 conn(0x7f95ac066990 msgr2=0x7f95ac06a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:08.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.160+0000 7f95b09ff700 1 -- 172.21.15.97:0/3544215144 shutdown_connections 2023-07-06T03:53:08.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.160+0000 7f95b09ff700 1 -- 172.21.15.97:0/3544215144 wait complete. 2023-07-06T03:53:08.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.161+0000 7f95b09ff700 1 Processor -- start 2023-07-06T03:53:08.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.161+0000 7f95b09ff700 1 -- start start 2023-07-06T03:53:08.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.162+0000 7f95b09ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0b4fe0 0x7f95ac0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.162+0000 7f95b09ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95ac0b5550 0x7f95ac153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.162+0000 7f95b09ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95ac15c8b0 0x7f95ac157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.162+0000 7f95b09ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac153a50 con 0x7f95ac15c8b0 2023-07-06T03:53:08.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.162+0000 7f95b09ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0cac10 con 0x7f95ac0b4fe0 2023-07-06T03:53:08.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.162+0000 7f95b09ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0caef0 con 0x7f95ac0b5550 2023-07-06T03:53:08.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95aa59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0b4fe0 0x7f95ac0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95aad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95ac15c8b0 0x7f95ac157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95aa59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0b4fe0 0x7f95ac0cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:36538/0 (socket says 172.21.15.97:36538) 2023-07-06T03:53:08.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95aad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95ac15c8b0 0x7f95ac157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36168/0 (socket says 172.21.15.97:36168) 2023-07-06T03:53:08.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95aa59c700 1 -- 172.21.15.97:0/3695713984 learned_addr learned my addr 172.21.15.97:0/3695713984 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:08.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95a3fff700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95ac0b5550 0x7f95ac153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95aad9d700 1 -- 172.21.15.97:0/3695713984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95ac0b5550 msgr2=0x7f95ac153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95aad9d700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95ac0b5550 0x7f95ac153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95aad9d700 1 -- 172.21.15.97:0/3695713984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0b4fe0 msgr2=0x7f95ac0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95aad9d700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0b4fe0 0x7f95ac0cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95aad9d700 1 -- 172.21.15.97:0/3695713984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0094a0 con 0x7f95ac15c8b0 2023-07-06T03:53:08.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95aad9d700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95ac15c8b0 0x7f95ac157500 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f95a4007d60 tx=0x7f95a400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:08.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95a2ffd700 1 -- 172.21.15.97:0/3695713984 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4018070 con 0x7f95ac15c8b0 2023-07-06T03:53:08.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95a2ffd700 1 -- 172.21.15.97:0/3695713984 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a400eb30 con 0x7f95ac15c8b0 2023-07-06T03:53:08.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95b09ff700 1 -- 172.21.15.97:0/3695713984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac0cb1d0 con 0x7f95ac15c8b0 2023-07-06T03:53:08.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.163+0000 7f95a2ffd700 1 -- 172.21.15.97:0/3695713984 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4003a20 con 0x7f95ac15c8b0 2023-07-06T03:53:08.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.164+0000 7f95b09ff700 1 -- 172.21.15.97:0/3695713984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac0cb790 con 0x7f95ac15c8b0 2023-07-06T03:53:08.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.164+0000 7f95b09ff700 1 -- 172.21.15.97:0/3695713984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac0cb9b0 con 0x7f95ac15c8b0 2023-07-06T03:53:08.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.165+0000 7f95a2ffd700 1 -- 172.21.15.97:0/3695713984 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f95a4014420 con 0x7f95ac15c8b0 2023-07-06T03:53:08.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.166+0000 7f95a2ffd700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f958c06ace0 0x7f958c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.166+0000 7f95a2ffd700 1 -- 172.21.15.97:0/3695713984 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f95a4089510 con 0x7f95ac15c8b0 2023-07-06T03:53:08.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.166+0000 7f95aa59c700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f958c06ace0 0x7f958c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.167+0000 7f95aa59c700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f958c06ace0 0x7f958c06d190 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:08.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.171+0000 7f95a2ffd700 1 -- 172.21.15.97:0/3695713984 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f95a40544e0 con 0x7f95ac15c8b0 2023-07-06T03:53:08.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.407+0000 7f95b09ff700 1 -- 172.21.15.97:0/3695713984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95ac074350 con 0x7f95ac15c8b0 2023-07-06T03:53:08.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.408+0000 7f95a2ffd700 1 -- 172.21.15.97:0/3695713984 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f95a4057b00 con 0x7f95ac15c8b0 2023-07-06T03:53:08.411 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:08.411 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3945452207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3945452207},{"type":"v1","addr":"172.21.15.146:6835","nonce":3945452207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2428591387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2428591387},{"type":"v1","addr":"172.21.15.97:6837","nonce":2428591387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/2089126027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2089126027},{"type":"v1","addr":"172.21.15.146:6834","nonce":2089126027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:52:34.447330+0000","modified":"2023-07-06T03:52:59.797803+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7497":{"gid":7497,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6836/2046478602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2046478602},{"type":"v1","addr":"172.21.15.97:6836","nonce":2046478602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-06T03:53:08.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 -- 172.21.15.97:0/3695713984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f958c06ace0 msgr2=0x7f958c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f958c06ace0 0x7f958c06d190 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f959c000c00 tx=0x7f959c01c040 comp rx=0 tx=0).stop 2023-07-06T03:53:08.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 -- 172.21.15.97:0/3695713984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95ac15c8b0 msgr2=0x7f95ac157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95ac15c8b0 0x7f95ac157500 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f95a4007d60 tx=0x7f95a400e3c0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 -- 172.21.15.97:0/3695713984 shutdown_connections 2023-07-06T03:53:08.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f958c06ace0 0x7f958c06d190 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95ac15c8b0 0x7f95ac157500 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95ac0b4fe0 0x7f95ac0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 --2- 172.21.15.97:0/3695713984 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95ac0b5550 0x7f95ac153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 -- 172.21.15.97:0/3695713984 >> 172.21.15.97:0/3695713984 conn(0x7f95ac066990 msgr2=0x7f95ac15cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:08.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 -- 172.21.15.97:0/3695713984 shutdown_connections 2023-07-06T03:53:08.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.410+0000 7f95b09ff700 1 -- 172.21.15.97:0/3695713984 wait complete. 2023-07-06T03:53:08.415 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 57 2023-07-06T03:53:08.431 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:53:08.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.625+0000 7fc2519ff700 1 Processor -- start 2023-07-06T03:53:08.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.625+0000 7fc2519ff700 1 -- start start 2023-07-06T03:53:08.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.626+0000 7fc2519ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc24c15c830 0x7fc24c0b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.626+0000 7fc2519ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc24c0b23f0 0x7fc24c0d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.626+0000 7fc2519ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c0b3f90 con 0x7fc24c0b1c80 2023-07-06T03:53:08.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.626+0000 7fc2519ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c0b3380 con 0x7fc24c15c830 2023-07-06T03:53:08.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.626+0000 7fc2519ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c0b3660 con 0x7fc24c0b23f0 2023-07-06T03:53:08.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.626+0000 7fc24affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc24c15c830 0x7fc24c0b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.626+0000 7fc24affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc24c15c830 0x7fc24c0b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36170/0 (socket says 172.21.15.97:36170) 2023-07-06T03:53:08.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.626+0000 7fc24affd700 1 -- 172.21.15.97:0/3951869759 learned_addr learned my addr 172.21.15.97:0/3951869759 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:08.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.627+0000 7fc24a7fc700 1 --2- 172.21.15.97:0/3951869759 >> v2:172.21.15.146:3300/0 conn(0x7fc24c0b23f0 0x7fc24c0d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.627+0000 7fc24affd700 1 -- 172.21.15.97:0/3951869759 >> v2:172.21.15.146:3300/0 conn(0x7fc24c0b23f0 msgr2=0x7fc24c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:08.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.627+0000 7fc24affd700 1 --2- 172.21.15.97:0/3951869759 >> v2:172.21.15.146:3300/0 conn(0x7fc24c0b23f0 0x7fc24c0d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.627+0000 7fc24affd700 1 -- 172.21.15.97:0/3951869759 >> v1:172.21.15.97:6789/0 conn(0x7fc24c0b1c80 legacy=0x7fc24c0afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.627+0000 7fc24affd700 1 -- 172.21.15.97:0/3951869759 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c0b3940 con 0x7fc24c15c830 2023-07-06T03:53:08.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.627+0000 7fc24affd700 1 --2- 172.21.15.97:0/3951869759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc24c15c830 0x7fc24c0b4420 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fc24c072280 tx=0x7fc23c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8525848dafaf0d85 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:08.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.627+0000 7fc249ffb700 1 -- 172.21.15.97:0/3951869759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c00b870 con 0x7fc24c15c830 2023-07-06T03:53:08.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.627+0000 7fc249ffb700 1 -- 172.21.15.97:0/3951869759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c00b9d0 con 0x7fc24c15c830 2023-07-06T03:53:08.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.627+0000 7fc249ffb700 1 -- 172.21.15.97:0/3951869759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c00bd60 con 0x7fc24c15c830 2023-07-06T03:53:08.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.628+0000 7fc2519ff700 1 -- 172.21.15.97:0/3951869759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc24c15c830 msgr2=0x7fc24c0b4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.628+0000 7fc2519ff700 1 --2- 172.21.15.97:0/3951869759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc24c15c830 0x7fc24c0b4420 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fc24c072280 tx=0x7fc23c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.628+0000 7fc2519ff700 1 -- 172.21.15.97:0/3951869759 shutdown_connections 2023-07-06T03:53:08.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.628+0000 7fc2519ff700 1 --2- 172.21.15.97:0/3951869759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc24c15c830 0x7fc24c0b4420 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.628+0000 7fc2519ff700 1 --2- 172.21.15.97:0/3951869759 >> v2:172.21.15.146:3300/0 conn(0x7fc24c0b23f0 0x7fc24c0d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.628+0000 7fc2519ff700 1 -- 172.21.15.97:0/3951869759 >> 172.21.15.97:0/3951869759 conn(0x7fc24c09c1e0 msgr2=0x7fc24c0d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:08.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.628+0000 7fc2519ff700 1 -- 172.21.15.97:0/3951869759 shutdown_connections 2023-07-06T03:53:08.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.628+0000 7fc2519ff700 1 -- 172.21.15.97:0/3951869759 wait complete. 2023-07-06T03:53:08.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.630+0000 7fc2519ff700 1 Processor -- start 2023-07-06T03:53:08.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.630+0000 7fc2519ff700 1 -- start start 2023-07-06T03:53:08.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc2519ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc24c0b1c80 0x7fc24c1f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc2519ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc24c0b23f0 0x7fc24c0c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc2519ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc24c15c830 0x7fc24c1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc2519ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0cf3e0 con 0x7fc24c0b1c80 2023-07-06T03:53:08.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc2519ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0ceaf0 con 0x7fc24c0b23f0 2023-07-06T03:53:08.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc2519ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0ced90 con 0x7fc24c15c830 2023-07-06T03:53:08.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc24affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc24c0b1c80 0x7fc24c1f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc24affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc24c0b1c80 0x7fc24c1f2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36186/0 (socket says 172.21.15.97:36186) 2023-07-06T03:53:08.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc24affd700 1 -- 172.21.15.97:0/3257353130 learned_addr learned my addr 172.21.15.97:0/3257353130 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:08.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc24a7fc700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc24c0b23f0 0x7fc24c0c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc24b7fe700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc24c15c830 0x7fc24c1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc24affd700 1 -- 172.21.15.97:0/3257353130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc24c15c830 msgr2=0x7fc24c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc24affd700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc24c15c830 0x7fc24c1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc24affd700 1 -- 172.21.15.97:0/3257353130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc24c0b23f0 msgr2=0x7fc24c0c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc24affd700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc24c0b23f0 0x7fc24c0c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc24affd700 1 -- 172.21.15.97:0/3257353130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c009550 con 0x7fc24c0b1c80 2023-07-06T03:53:08.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.631+0000 7fc24a7fc700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc24c0b23f0 0x7fc24c0c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:08.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.632+0000 7fc24affd700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc24c0b1c80 0x7fc24c1f2300 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fc23c000c00 tx=0x7fc23c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:08.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.632+0000 7fc233fff700 1 -- 172.21.15.97:0/3257353130 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c004520 con 0x7fc24c0b1c80 2023-07-06T03:53:08.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.632+0000 7fc233fff700 1 -- 172.21.15.97:0/3257353130 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c01e070 con 0x7fc24c0b1c80 2023-07-06T03:53:08.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.632+0000 7fc233fff700 1 -- 172.21.15.97:0/3257353130 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0038a0 con 0x7fc24c0b1c80 2023-07-06T03:53:08.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.632+0000 7fc2519ff700 1 -- 172.21.15.97:0/3257353130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24c0cf070 con 0x7fc24c0b1c80 2023-07-06T03:53:08.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.632+0000 7fc2519ff700 1 -- 172.21.15.97:0/3257353130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c15e680 con 0x7fc24c0b1c80 2023-07-06T03:53:08.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.633+0000 7fc2519ff700 1 -- 172.21.15.97:0/3257353130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24c074350 con 0x7fc24c0b1c80 2023-07-06T03:53:08.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.634+0000 7fc233fff700 1 -- 172.21.15.97:0/3257353130 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc23c003a00 con 0x7fc24c0b1c80 2023-07-06T03:53:08.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.635+0000 7fc233fff700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc23806ace0 0x7fc23806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:08.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.635+0000 7fc233fff700 1 -- 172.21.15.97:0/3257353130 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fc23c020070 con 0x7fc24c0b1c80 2023-07-06T03:53:08.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.635+0000 7fc24a7fc700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc23806ace0 0x7fc23806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:08.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.636+0000 7fc24a7fc700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc23806ace0 0x7fc23806d190 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0ce980 tx=0x7fc240008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:08.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.639+0000 7fc233fff700 1 -- 172.21.15.97:0/3257353130 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc23c055bb0 con 0x7fc24c0b1c80 2023-07-06T03:53:08.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.838+0000 7fc2519ff700 1 -- 172.21.15.97:0/3257353130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc24c08c5b0 con 0x7fc24c0b1c80 2023-07-06T03:53:08.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.839+0000 7fc233fff700 1 -- 172.21.15.97:0/3257353130 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+18706 (secure 0 0 0) 0x7fc23c015090 con 0x7fc24c0b1c80 2023-07-06T03:53:08.842 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:08.842 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:52:53.756629+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:52:32.448828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-06T03:52:33.428266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.12","mappings":[{"from":6,"to":4}]},{"pgid":"14.15","mappings":[{"from":1,"to":0}]},{"pgid":"14.1a","mappings":[{"from":7,"to":3}]},{"pgid":"14.26","mappings":[{"from":7,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.34","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:53:08.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.841+0000 7fc2519ff700 1 -- 172.21.15.97:0/3257353130 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc23806ace0 msgr2=0x7fc23806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.841+0000 7fc2519ff700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc23806ace0 0x7fc23806d190 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0ce980 tx=0x7fc240008040 comp rx=0 tx=0).stop 2023-07-06T03:53:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.841+0000 7fc2519ff700 1 -- 172.21.15.97:0/3257353130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc24c0b1c80 msgr2=0x7fc24c1f2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.841+0000 7fc2519ff700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc24c0b1c80 0x7fc24c1f2300 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fc23c000c00 tx=0x7fc23c004750 comp rx=0 tx=0).stop 2023-07-06T03:53:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.842+0000 7fc2519ff700 1 -- 172.21.15.97:0/3257353130 shutdown_connections 2023-07-06T03:53:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.842+0000 7fc2519ff700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc23806ace0 0x7fc23806d190 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.842+0000 7fc2519ff700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc24c15c830 0x7fc24c1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.842+0000 7fc2519ff700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc24c0b1c80 0x7fc24c1f2300 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.842+0000 7fc2519ff700 1 --2- 172.21.15.97:0/3257353130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc24c0b23f0 0x7fc24c0c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:08.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.842+0000 7fc2519ff700 1 -- 172.21.15.97:0/3257353130 >> 172.21.15.97:0/3257353130 conn(0x7fc24c09c1e0 msgr2=0x7fc24c06bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:08.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.842+0000 7fc2519ff700 1 -- 172.21.15.97:0/3257353130 shutdown_connections 2023-07-06T03:53:08.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:08.842+0000 7fc2519ff700 1 -- 172.21.15.97:0/3257353130 wait complete. 2023-07-06T03:53:08.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:52:32.448828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:52:33.428266+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:53:08.863 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:53:09.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.059+0000 7fca305ff700 1 Processor -- start 2023-07-06T03:53:09.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.059+0000 7fca305ff700 1 -- start start 2023-07-06T03:53:09.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.060+0000 7fca305ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fca280b1fc0 0x7fca280cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.060+0000 7fca305ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca280b2390 0x7fca280cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.060+0000 7fca305ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2815d610 con 0x7fca280cf080 2023-07-06T03:53:09.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.060+0000 7fca305ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2815db40 con 0x7fca280b1fc0 2023-07-06T03:53:09.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.060+0000 7fca305ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2815d860 con 0x7fca280b2390 2023-07-06T03:53:09.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.061+0000 7fca2eb9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fca280cf080 0x7fca280b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45668/0 (socket says 172.21.15.97:45668) 2023-07-06T03:53:09.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.061+0000 7fca2eb9c700 1 -- 172.21.15.97:0/2750499163 learned_addr learned my addr 172.21.15.97:0/2750499163 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:09.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.061+0000 7fca2e39b700 1 --2- 172.21.15.97:0/2750499163 >> v2:172.21.15.146:3300/0 conn(0x7fca280b1fc0 0x7fca280cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.061+0000 7fca2db9a700 1 --2- 172.21.15.97:0/2750499163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca280b2390 0x7fca280cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.061+0000 7fca2d399700 1 -- 172.21.15.97:0/2750499163 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770661917 0 0) 0x7fca2815d610 con 0x7fca280cf080 2023-07-06T03:53:09.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.061+0000 7fca2d399700 1 -- 172.21.15.97:0/2750499163 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca140035b0 con 0x7fca280cf080 2023-07-06T03:53:09.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.061+0000 7fca2e39b700 1 -- 172.21.15.97:0/2750499163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca280b2390 msgr2=0x7fca280cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:09.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.061+0000 7fca2e39b700 1 --2- 172.21.15.97:0/2750499163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca280b2390 0x7fca280cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.061+0000 7fca2e39b700 1 -- 172.21.15.97:0/2750499163 >> v1:172.21.15.97:6789/0 conn(0x7fca280cf080 legacy=0x7fca280b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.061+0000 7fca2e39b700 1 -- 172.21.15.97:0/2750499163 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca280cd630 con 0x7fca280b1fc0 2023-07-06T03:53:09.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.062+0000 7fca2e39b700 1 --2- 172.21.15.97:0/2750499163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca280b1fc0 0x7fca280cf8b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fca24009840 tx=0x7fca24009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29ff0645a3ee0ef7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:09.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.062+0000 7fca2d399700 1 -- 172.21.15.97:0/2750499163 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca24011830 con 0x7fca280b1fc0 2023-07-06T03:53:09.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.062+0000 7fca2d399700 1 -- 172.21.15.97:0/2750499163 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca24011990 con 0x7fca280b1fc0 2023-07-06T03:53:09.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.062+0000 7fca2d399700 1 -- 172.21.15.97:0/2750499163 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca24011d20 con 0x7fca280b1fc0 2023-07-06T03:53:09.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.062+0000 7fca305ff700 1 -- 172.21.15.97:0/2750499163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca280b1fc0 msgr2=0x7fca280cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.062+0000 7fca305ff700 1 --2- 172.21.15.97:0/2750499163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca280b1fc0 0x7fca280cf8b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fca24009840 tx=0x7fca24009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:09.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.063+0000 7fca305ff700 1 -- 172.21.15.97:0/2750499163 shutdown_connections 2023-07-06T03:53:09.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.063+0000 7fca305ff700 1 --2- 172.21.15.97:0/2750499163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca280b1fc0 0x7fca280cf8b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.063+0000 7fca305ff700 1 --2- 172.21.15.97:0/2750499163 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca280b2390 0x7fca280cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.063+0000 7fca305ff700 1 -- 172.21.15.97:0/2750499163 >> 172.21.15.97:0/2750499163 conn(0x7fca2806c210 msgr2=0x7fca2806c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:09.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.063+0000 7fca305ff700 1 -- 172.21.15.97:0/2750499163 shutdown_connections 2023-07-06T03:53:09.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.063+0000 7fca305ff700 1 -- 172.21.15.97:0/2750499163 wait complete. 2023-07-06T03:53:09.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.064+0000 7fca305ff700 1 Processor -- start 2023-07-06T03:53:09.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.064+0000 7fca305ff700 1 -- start start 2023-07-06T03:53:09.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.065+0000 7fca305ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca280b1fc0 0x7fca280d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.065+0000 7fca305ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca280b2390 0x7fca281f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.065+0000 7fca305ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca280cf080 0x7fca28154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.065+0000 7fca305ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280d3dc0 con 0x7fca280cf080 2023-07-06T03:53:09.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.065+0000 7fca305ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca281f3020 con 0x7fca280b1fc0 2023-07-06T03:53:09.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.065+0000 7fca305ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca280d31b0 con 0x7fca280b2390 2023-07-06T03:53:09.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.066+0000 7fca2eb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca280cf080 0x7fca28154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.066+0000 7fca2eb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca280cf080 0x7fca28154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36200/0 (socket says 172.21.15.97:36200) 2023-07-06T03:53:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.066+0000 7fca2e39b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca280b1fc0 0x7fca280d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.066+0000 7fca2eb9c700 1 -- 172.21.15.97:0/3102952346 learned_addr learned my addr 172.21.15.97:0/3102952346 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.066+0000 7fca2db9a700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca280b2390 0x7fca281f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.066+0000 7fca2eb9c700 1 -- 172.21.15.97:0/3102952346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca280b2390 msgr2=0x7fca281f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.066+0000 7fca2eb9c700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca280b2390 0x7fca281f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.066+0000 7fca2eb9c700 1 -- 172.21.15.97:0/3102952346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca280b1fc0 msgr2=0x7fca280d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.066+0000 7fca2eb9c700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca280b1fc0 0x7fca280d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.066+0000 7fca2eb9c700 1 -- 172.21.15.97:0/3102952346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca240094a0 con 0x7fca280cf080 2023-07-06T03:53:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.066+0000 7fca2eb9c700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca280cf080 0x7fca28154310 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fca2000caa0 tx=0x7fca2000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.067+0000 7fca1effd700 1 -- 172.21.15.97:0/3102952346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000a9e0 con 0x7fca280cf080 2023-07-06T03:53:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.067+0000 7fca1effd700 1 -- 172.21.15.97:0/3102952346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2000ab40 con 0x7fca280cf080 2023-07-06T03:53:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.067+0000 7fca305ff700 1 -- 172.21.15.97:0/3102952346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca280d3490 con 0x7fca280cf080 2023-07-06T03:53:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.067+0000 7fca1effd700 1 -- 172.21.15.97:0/3102952346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20012400 con 0x7fca280cf080 2023-07-06T03:53:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.067+0000 7fca305ff700 1 -- 172.21.15.97:0/3102952346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca280d3a50 con 0x7fca280cf080 2023-07-06T03:53:09.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.068+0000 7fca305ff700 1 -- 172.21.15.97:0/3102952346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca28156090 con 0x7fca280cf080 2023-07-06T03:53:09.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.069+0000 7fca1effd700 1 -- 172.21.15.97:0/3102952346 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fca2001b3b0 con 0x7fca280cf080 2023-07-06T03:53:09.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.070+0000 7fca1effd700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca0c06ace0 0x7fca0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.070+0000 7fca1effd700 1 -- 172.21.15.97:0/3102952346 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fca20089210 con 0x7fca280cf080 2023-07-06T03:53:09.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.070+0000 7fca2e39b700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca0c06ace0 0x7fca0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.071+0000 7fca2e39b700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca0c06ace0 0x7fca0c06d190 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fca24000c00 tx=0x7fca24008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:09.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.075+0000 7fca1effd700 1 -- 172.21.15.97:0/3102952346 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fca28156090 con 0x7fca280cf080 2023-07-06T03:53:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.266+0000 7fca305ff700 1 -- 172.21.15.97:0/3102952346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca28156090 con 0x7fca280cf080 2023-07-06T03:53:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.266+0000 7fca1effd700 1 -- 172.21.15.97:0/3102952346 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca28156090 con 0x7fca280cf080 2023-07-06T03:53:09.269 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:53:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 -- 172.21.15.97:0/3102952346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca0c06ace0 msgr2=0x7fca0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca0c06ace0 0x7fca0c06d190 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fca24000c00 tx=0x7fca24008030 comp rx=0 tx=0).stop 2023-07-06T03:53:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 -- 172.21.15.97:0/3102952346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca280cf080 msgr2=0x7fca28154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca280cf080 0x7fca28154310 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fca2000caa0 tx=0x7fca2000cdb0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 -- 172.21.15.97:0/3102952346 shutdown_connections 2023-07-06T03:53:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fca0c06ace0 0x7fca0c06d190 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca280cf080 0x7fca28154310 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca280b1fc0 0x7fca280d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 --2- 172.21.15.97:0/3102952346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fca280b2390 0x7fca281f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 -- 172.21.15.97:0/3102952346 >> 172.21.15.97:0/3102952346 conn(0x7fca2806c210 msgr2=0x7fca280b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 -- 172.21.15.97:0/3102952346 shutdown_connections 2023-07-06T03:53:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.269+0000 7fca305ff700 1 -- 172.21.15.97:0/3102952346 wait complete. 2023-07-06T03:53:09.289 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:53:09.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.484+0000 7f0181dff700 1 Processor -- start 2023-07-06T03:53:09.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.484+0000 7f0181dff700 1 -- start start 2023-07-06T03:53:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.485+0000 7f0181dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f017c0b2390 0x7f017c0d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f0181dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c0cffc0 0x7f017c0d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f0181dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017c15d610 con 0x7f017c0b1fc0 2023-07-06T03:53:09.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f0181dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f017c15db40 con 0x7f017c0b2390 2023-07-06T03:53:09.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f0181dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017c15d860 con 0x7f017c0cffc0 2023-07-06T03:53:09.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f017b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f017c0b1fc0 0x7f017c0cf8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45678/0 (socket says 172.21.15.97:45678) 2023-07-06T03:53:09.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f017b7fe700 1 -- 172.21.15.97:0/4175372303 learned_addr learned my addr 172.21.15.97:0/4175372303 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f0173fff700 1 --2- 172.21.15.97:0/4175372303 >> v2:172.21.15.146:3300/0 conn(0x7f017c0b2390 0x7f017c0d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f017bfff700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c0cffc0 0x7f017c0d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193847484 0 0) 0x7f017c15d610 con 0x7f017c0b1fc0 2023-07-06T03:53:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01680035b0 con 0x7f017c0b1fc0 2023-07-06T03:53:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1776418634 0 0) 0x7f01680035b0 con 0x7f017c0b1fc0 2023-07-06T03:53:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.486+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c0cffc0 msgr2=0x7f017c0d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c0cffc0 0x7f017c0d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 >> v2:172.21.15.146:3300/0 conn(0x7f017c0b2390 msgr2=0x7f017c0d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 --2- 172.21.15.97:0/4175372303 >> v2:172.21.15.146:3300/0 conn(0x7f017c0b2390 0x7f017c0d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c0cd630 con 0x7f017c0b1fc0 2023-07-06T03:53:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f016c001670 con 0x7f017c0b1fc0 2023-07-06T03:53:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 >> v1:172.21.15.97:6789/0 conn(0x7f017c0b1fc0 legacy=0x7f017c0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0168003a00 0x7f0168003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0168004430 0x7f01680048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c0cffc0 0x7f0168004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c15db40 con 0x7f0168004430 2023-07-06T03:53:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c15d860 con 0x7f0168003a00 2023-07-06T03:53:09.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016800ba10 con 0x7f017c0cffc0 2023-07-06T03:53:09.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f0173fff700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0168004430 0x7f01680048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017b7fe700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0168003a00 0x7f0168003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f017bfff700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c0cffc0 0x7f0168004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f0173fff700 1 -- 172.21.15.97:0/4175372303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c0cffc0 msgr2=0x7f0168004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:09.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f0173fff700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c0cffc0 0x7f0168004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f0173fff700 1 -- 172.21.15.97:0/4175372303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0168003a00 msgr2=0x7f0168003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:09.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f0173fff700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0168003a00 0x7f0168003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.487+0000 7f0173fff700 1 -- 172.21.15.97:0/4175372303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01680035b0 con 0x7f0168004430 2023-07-06T03:53:09.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.488+0000 7f0173fff700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0168004430 0x7f01680048f0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f0164009b30 tx=0x7f0168005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b71c5c16715eeb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:09.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.488+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016400e6c0 con 0x7f0168004430 2023-07-06T03:53:09.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.488+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016400e820 con 0x7f0168004430 2023-07-06T03:53:09.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.488+0000 7f017affd700 1 -- 172.21.15.97:0/4175372303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016400eb90 con 0x7f0168004430 2023-07-06T03:53:09.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.488+0000 7f0181dff700 1 -- 172.21.15.97:0/4175372303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0168004430 msgr2=0x7f01680048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.488+0000 7f0181dff700 1 --2- 172.21.15.97:0/4175372303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0168004430 0x7f01680048f0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f0164009b30 tx=0x7f0168005880 comp rx=0 tx=0).stop 2023-07-06T03:53:09.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.488+0000 7f017bfff700 1 -- 172.21.15.97:0/4175372303 reap_dead start 2023-07-06T03:53:09.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.488+0000 7f0181dff700 1 -- 172.21.15.97:0/4175372303 shutdown_connections 2023-07-06T03:53:09.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.488+0000 7f0181dff700 1 -- 172.21.15.97:0/4175372303 >> 172.21.15.97:0/4175372303 conn(0x7f017c06c210 msgr2=0x7f017c06c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:09.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.489+0000 7f0181dff700 1 -- 172.21.15.97:0/4175372303 shutdown_connections 2023-07-06T03:53:09.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.489+0000 7f0181dff700 1 -- 172.21.15.97:0/4175372303 wait complete. 2023-07-06T03:53:09.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.491+0000 7f0181dff700 1 Processor -- start 2023-07-06T03:53:09.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.491+0000 7f0181dff700 1 -- start start 2023-07-06T03:53:09.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.492+0000 7f0181dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0168004430 0x7f017c147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.492+0000 7f0181dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f017c0b3500 0x7f017c0b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.492+0000 7f0181dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c1f2640 0x7f017c0cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.492+0000 7f0181dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c0b4d40 con 0x7f017c0b3500 2023-07-06T03:53:09.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.492+0000 7f0181dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c0b4130 con 0x7f0168004430 2023-07-06T03:53:09.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.492+0000 7f0181dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017c0b4410 con 0x7f017c1f2640 2023-07-06T03:53:09.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.493+0000 7f0173fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f017c0b3500 0x7f017c0b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.493+0000 7f0173fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f017c0b3500 0x7f017c0b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36218/0 (socket says 172.21.15.97:36218) 2023-07-06T03:53:09.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.493+0000 7f0173fff700 1 -- 172.21.15.97:0/3669887180 learned_addr learned my addr 172.21.15.97:0/3669887180 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.493+0000 7f017b7fe700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0168004430 0x7f017c147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.493+0000 7f017bfff700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c1f2640 0x7f017c0cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.493+0000 7f0173fff700 1 -- 172.21.15.97:0/3669887180 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c1f2640 msgr2=0x7f017c0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.493+0000 7f0173fff700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c1f2640 0x7f017c0cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.493+0000 7f0173fff700 1 -- 172.21.15.97:0/3669887180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0168004430 msgr2=0x7f017c147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.493+0000 7f0173fff700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0168004430 0x7f017c147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.493+0000 7f0173fff700 1 -- 172.21.15.97:0/3669887180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c0b46f0 con 0x7f017c0b3500 2023-07-06T03:53:09.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.493+0000 7f0173fff700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f017c0b3500 0x7f017c0b51d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f01680056d0 tx=0x7f01640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:09.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.494+0000 7f0178ff9700 1 -- 172.21.15.97:0/3669887180 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164021070 con 0x7f017c0b3500 2023-07-06T03:53:09.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.494+0000 7f0178ff9700 1 -- 172.21.15.97:0/3669887180 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01640044a0 con 0x7f017c0b3500 2023-07-06T03:53:09.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.494+0000 7f0181dff700 1 -- 172.21.15.97:0/3669887180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017c0b49d0 con 0x7f017c0b3500 2023-07-06T03:53:09.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.494+0000 7f0178ff9700 1 -- 172.21.15.97:0/3669887180 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016400e6c0 con 0x7f017c0b3500 2023-07-06T03:53:09.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.494+0000 7f0181dff700 1 -- 172.21.15.97:0/3669887180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017c1f2ae0 con 0x7f017c0b3500 2023-07-06T03:53:09.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.495+0000 7f0181dff700 1 -- 172.21.15.97:0/3669887180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017c0b4eb0 con 0x7f017c0b3500 2023-07-06T03:53:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.496+0000 7f0178ff9700 1 -- 172.21.15.97:0/3669887180 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0164025070 con 0x7f017c0b3500 2023-07-06T03:53:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.496+0000 7f0178ff9700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f015c06ace0 0x7f015c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.497+0000 7f0178ff9700 1 -- 172.21.15.97:0/3669887180 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f016408a630 con 0x7f017c0b3500 2023-07-06T03:53:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.497+0000 7f017b7fe700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f015c06ace0 0x7f015c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.497+0000 7f017b7fe700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f015c06ace0 0x7f015c06d190 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f016c001050 tx=0x7f016c007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:09.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.501+0000 7f0178ff9700 1 -- 172.21.15.97:0/3669887180 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0164055720 con 0x7f017c0b3500 2023-07-06T03:53:09.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.694+0000 7f0181dff700 1 -- 172.21.15.97:0/3669887180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f017c1f2cd0 con 0x7f017c0b3500 2023-07-06T03:53:09.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.694+0000 7f0178ff9700 1 -- 172.21.15.97:0/3669887180 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v88) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0164058d40 con 0x7f017c0b3500 2023-07-06T03:53:09.697 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:53:09.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.696+0000 7f0181dff700 1 -- 172.21.15.97:0/3669887180 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f015c06ace0 msgr2=0x7f015c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.696+0000 7f0181dff700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f015c06ace0 0x7f015c06d190 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f016c001050 tx=0x7f016c007980 comp rx=0 tx=0).stop 2023-07-06T03:53:09.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.696+0000 7f0181dff700 1 -- 172.21.15.97:0/3669887180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f017c0b3500 msgr2=0x7f017c0b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.696+0000 7f0181dff700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f017c0b3500 0x7f017c0b51d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f01680056d0 tx=0x7f01640047c0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.696+0000 7f0181dff700 1 -- 172.21.15.97:0/3669887180 shutdown_connections 2023-07-06T03:53:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.696+0000 7f0181dff700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f015c06ace0 0x7f015c06d190 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.696+0000 7f0181dff700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f017c1f2640 0x7f017c0cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.696+0000 7f0181dff700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0168004430 0x7f017c147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.696+0000 7f0181dff700 1 --2- 172.21.15.97:0/3669887180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f017c0b3500 0x7f017c0b51d0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.696+0000 7f0181dff700 1 -- 172.21.15.97:0/3669887180 >> 172.21.15.97:0/3669887180 conn(0x7f017c06c210 msgr2=0x7f017c0d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.696+0000 7f0181dff700 1 -- 172.21.15.97:0/3669887180 shutdown_connections 2023-07-06T03:53:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.697+0000 7f0181dff700 1 -- 172.21.15.97:0/3669887180 wait complete. 2023-07-06T03:53:09.717 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:53:09.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.913+0000 7f518c7ff700 1 Processor -- start 2023-07-06T03:53:09.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.913+0000 7f518c7ff700 1 -- start start 2023-07-06T03:53:09.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.914+0000 7f518c7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51840b2390 0x7f51840d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.914+0000 7f518c7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f51840cffc0 0x7f51840d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.914+0000 7f518c7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518415d610 con 0x7f51840b1fc0 2023-07-06T03:53:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.914+0000 7f518c7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518415db40 con 0x7f51840cffc0 2023-07-06T03:53:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.914+0000 7f518c7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518415d860 con 0x7f51840b2390 2023-07-06T03:53:09.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.914+0000 7f518ad9c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f51840cffc0 0x7f51840d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.914+0000 7f5189d9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51840b2390 0x7f51840d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.915+0000 7f518ad9c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f51840cffc0 0x7f51840d10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:37776/0 (socket says 172.21.15.97:37776) 2023-07-06T03:53:09.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.915+0000 7f518ad9c700 1 -- 172.21.15.97:0/4146681007 learned_addr learned my addr 172.21.15.97:0/4146681007 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.915+0000 7f518ad9c700 1 -- 172.21.15.97:0/4146681007 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51840b2390 msgr2=0x7f51840d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.915+0000 7f518ad9c700 1 --2- 172.21.15.97:0/4146681007 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51840b2390 0x7f51840d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.915+0000 7f518ad9c700 1 -- 172.21.15.97:0/4146681007 >> v1:172.21.15.146:6789/0 conn(0x7f51840b1fc0 legacy=0x7f51840cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.915+0000 7f518ad9c700 1 -- 172.21.15.97:0/4146681007 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51840cd630 con 0x7f51840cffc0 2023-07-06T03:53:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.915+0000 7f518ad9c700 1 --2- 172.21.15.97:0/4146681007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51840cffc0 0x7f51840d10e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f517c009e50 tx=0x7f517c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e4af1e49b738284 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.916+0000 7f5189599700 1 -- 172.21.15.97:0/4146681007 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c00e070 con 0x7f51840cffc0 2023-07-06T03:53:09.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.916+0000 7f5189599700 1 -- 172.21.15.97:0/4146681007 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c00b910 con 0x7f51840cffc0 2023-07-06T03:53:09.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.916+0000 7f5189599700 1 -- 172.21.15.97:0/4146681007 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c00bcc0 con 0x7f51840cffc0 2023-07-06T03:53:09.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.916+0000 7f518c7ff700 1 -- 172.21.15.97:0/4146681007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51840cffc0 msgr2=0x7f51840d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.916+0000 7f518c7ff700 1 --2- 172.21.15.97:0/4146681007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51840cffc0 0x7f51840d10e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f517c009e50 tx=0x7f517c00b330 comp rx=0 tx=0).stop 2023-07-06T03:53:09.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.916+0000 7f518c7ff700 1 -- 172.21.15.97:0/4146681007 shutdown_connections 2023-07-06T03:53:09.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.916+0000 7f518c7ff700 1 --2- 172.21.15.97:0/4146681007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51840cffc0 0x7f51840d10e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.916+0000 7f518c7ff700 1 --2- 172.21.15.97:0/4146681007 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51840b2390 0x7f51840d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.916+0000 7f518c7ff700 1 -- 172.21.15.97:0/4146681007 >> 172.21.15.97:0/4146681007 conn(0x7f518406c210 msgr2=0x7f518406c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:09.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.916+0000 7f518c7ff700 1 -- 172.21.15.97:0/4146681007 shutdown_connections 2023-07-06T03:53:09.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.916+0000 7f518c7ff700 1 -- 172.21.15.97:0/4146681007 wait complete. 2023-07-06T03:53:09.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.918+0000 7f518c7ff700 1 Processor -- start 2023-07-06T03:53:09.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.918+0000 7f518c7ff700 1 -- start start 2023-07-06T03:53:09.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.919+0000 7f518c7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51840b1fc0 0x7f51841e6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.919+0000 7f518c7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51840b2390 0x7f51841f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.919+0000 7f518c7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51840cffc0 0x7f51841f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.919+0000 7f518c7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5184148e30 con 0x7f51840b2390 2023-07-06T03:53:09.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.919+0000 7f518c7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51841e6b70 con 0x7f51840b1fc0 2023-07-06T03:53:09.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.919+0000 7f518c7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51841e6e50 con 0x7f51840cffc0 2023-07-06T03:53:09.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.919+0000 7f5189d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51840b2390 0x7f51841f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.919+0000 7f518a59b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51840b1fc0 0x7f51841e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.919+0000 7f5189d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51840b2390 0x7f51841f54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54532/0 (socket says 172.21.15.97:54532) 2023-07-06T03:53:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.919+0000 7f5189d9a700 1 -- 172.21.15.97:0/48535907 learned_addr learned my addr 172.21.15.97:0/48535907 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.920+0000 7f518ad9c700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51840cffc0 0x7f51841f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.920+0000 7f5189d9a700 1 -- 172.21.15.97:0/48535907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51840cffc0 msgr2=0x7f51841f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.920+0000 7f5189d9a700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51840cffc0 0x7f51841f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.920+0000 7f5189d9a700 1 -- 172.21.15.97:0/48535907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51840b1fc0 msgr2=0x7f51841e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.920+0000 7f5189d9a700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51840b1fc0 0x7f51841e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.920+0000 7f5189d9a700 1 -- 172.21.15.97:0/48535907 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c009b30 con 0x7f51840b2390 2023-07-06T03:53:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.920+0000 7f5189d9a700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51840b2390 0x7f51841f54c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f517400e980 tx=0x7f517400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.920+0000 7f517affd700 1 -- 172.21.15.97:0/48535907 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174009710 con 0x7f51840b2390 2023-07-06T03:53:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.920+0000 7f517affd700 1 -- 172.21.15.97:0/48535907 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517400c690 con 0x7f51840b2390 2023-07-06T03:53:09.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.920+0000 7f518c7ff700 1 -- 172.21.15.97:0/48535907 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51840b3920 con 0x7f51840b2390 2023-07-06T03:53:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.920+0000 7f517affd700 1 -- 172.21.15.97:0/48535907 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517400ca40 con 0x7f51840b2390 2023-07-06T03:53:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.921+0000 7f518c7ff700 1 -- 172.21.15.97:0/48535907 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51840b3ee0 con 0x7f51840b2390 2023-07-06T03:53:09.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.922+0000 7f518c7ff700 1 -- 172.21.15.97:0/48535907 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5184074350 con 0x7f51840b2390 2023-07-06T03:53:09.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.922+0000 7f517affd700 1 -- 172.21.15.97:0/48535907 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f517400cba0 con 0x7f51840b2390 2023-07-06T03:53:09.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.923+0000 7f517affd700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516806ace0 0x7f516806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:09.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.923+0000 7f517affd700 1 -- 172.21.15.97:0/48535907 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f5174089550 con 0x7f51840b2390 2023-07-06T03:53:09.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.924+0000 7f518a59b700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516806ace0 0x7f516806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:09.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.924+0000 7f518a59b700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516806ace0 0x7f516806d190 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5180007fd0 tx=0x7f5180009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:09.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:09.928+0000 7f517affd700 1 -- 172.21.15.97:0/48535907 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5174057f20 con 0x7f51840b2390 2023-07-06T03:53:10.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.121+0000 7f518c7ff700 1 -- 172.21.15.97:0/48535907 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f518406b580 con 0x7f51840b2390 2023-07-06T03:53:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.122+0000 7f517affd700 1 -- 172.21.15.97:0/48535907 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5174057ab0 con 0x7f51840b2390 2023-07-06T03:53:10.125 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:53:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 -- 172.21.15.97:0/48535907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516806ace0 msgr2=0x7f516806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516806ace0 0x7f516806d190 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5180007fd0 tx=0x7f5180009040 comp rx=0 tx=0).stop 2023-07-06T03:53:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 -- 172.21.15.97:0/48535907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51840b2390 msgr2=0x7f51841f54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51840b2390 0x7f51841f54c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f517400e980 tx=0x7f517400ec90 comp rx=0 tx=0).stop 2023-07-06T03:53:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 -- 172.21.15.97:0/48535907 shutdown_connections 2023-07-06T03:53:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f516806ace0 0x7f516806d190 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51840cffc0 0x7f51841f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51840b1fc0 0x7f51841e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 --2- 172.21.15.97:0/48535907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51840b2390 0x7f51841f54c0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 -- 172.21.15.97:0/48535907 >> 172.21.15.97:0/48535907 conn(0x7f518406c210 msgr2=0x7f51840b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 -- 172.21.15.97:0/48535907 shutdown_connections 2023-07-06T03:53:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.124+0000 7f518c7ff700 1 -- 172.21.15.97:0/48535907 wait complete. 2023-07-06T03:53:10.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:53:10.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.339+0000 7fdf3fdff700 1 Processor -- start 2023-07-06T03:53:10.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.339+0000 7fdf3fdff700 1 -- start start 2023-07-06T03:53:10.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.340+0000 7fdf3fdff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf380b1fc0 0x7fdf380cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:10.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.340+0000 7fdf3fdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf380b2390 0x7fdf380cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:10.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.340+0000 7fdf3fdff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3815d610 con 0x7fdf380cf080 2023-07-06T03:53:10.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.340+0000 7fdf3fdff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3815db40 con 0x7fdf380b1fc0 2023-07-06T03:53:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.340+0000 7fdf3fdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3815d860 con 0x7fdf380b2390 2023-07-06T03:53:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.340+0000 7fdf3db9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf380b1fc0 0x7fdf380cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.340+0000 7fdf3db9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf380b1fc0 0x7fdf380cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54544/0 (socket says 172.21.15.97:54544) 2023-07-06T03:53:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.340+0000 7fdf3db9b700 1 -- 172.21.15.97:0/3455942153 learned_addr learned my addr 172.21.15.97:0/3455942153 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:10.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.340+0000 7fdf3d39a700 1 --2- 172.21.15.97:0/3455942153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf380b2390 0x7fdf380cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:10.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.341+0000 7fdf3db9b700 1 -- 172.21.15.97:0/3455942153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf380b2390 msgr2=0x7fdf380cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.341+0000 7fdf3db9b700 1 --2- 172.21.15.97:0/3455942153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf380b2390 0x7fdf380cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.341+0000 7fdf3db9b700 1 -- 172.21.15.97:0/3455942153 >> v1:172.21.15.146:6789/0 conn(0x7fdf380cf080 legacy=0x7fdf380b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.341+0000 7fdf3db9b700 1 -- 172.21.15.97:0/3455942153 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380cd630 con 0x7fdf380b1fc0 2023-07-06T03:53:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.341+0000 7fdf3db9b700 1 --2- 172.21.15.97:0/3455942153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf380b1fc0 0x7fdf380cf8b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fdf340098f0 tx=0x7fdf34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e9a5df8d6ec11e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.341+0000 7fdf3cb99700 1 -- 172.21.15.97:0/3455942153 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34011830 con 0x7fdf380b1fc0 2023-07-06T03:53:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.341+0000 7fdf3cb99700 1 -- 172.21.15.97:0/3455942153 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf34011990 con 0x7fdf380b1fc0 2023-07-06T03:53:10.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.341+0000 7fdf3cb99700 1 -- 172.21.15.97:0/3455942153 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34011d20 con 0x7fdf380b1fc0 2023-07-06T03:53:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.342+0000 7fdf3fdff700 1 -- 172.21.15.97:0/3455942153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf380b1fc0 msgr2=0x7fdf380cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.342+0000 7fdf3fdff700 1 --2- 172.21.15.97:0/3455942153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf380b1fc0 0x7fdf380cf8b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fdf340098f0 tx=0x7fdf34009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.342+0000 7fdf3fdff700 1 -- 172.21.15.97:0/3455942153 shutdown_connections 2023-07-06T03:53:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.342+0000 7fdf3fdff700 1 --2- 172.21.15.97:0/3455942153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf380b1fc0 0x7fdf380cf8b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.342+0000 7fdf3fdff700 1 --2- 172.21.15.97:0/3455942153 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf380b2390 0x7fdf380cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.342+0000 7fdf3fdff700 1 -- 172.21.15.97:0/3455942153 >> 172.21.15.97:0/3455942153 conn(0x7fdf3806c210 msgr2=0x7fdf3806c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.342+0000 7fdf3fdff700 1 -- 172.21.15.97:0/3455942153 shutdown_connections 2023-07-06T03:53:10.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.342+0000 7fdf3fdff700 1 -- 172.21.15.97:0/3455942153 wait complete. 2023-07-06T03:53:10.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.344+0000 7fdf3fdff700 1 Processor -- start 2023-07-06T03:53:10.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.344+0000 7fdf3fdff700 1 -- start start 2023-07-06T03:53:10.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3fdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf380b1fc0 0x7fdf381f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:10.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3fdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf380b2390 0x7fdf380cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:10.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3fdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf380cf080 0x7fdf38158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:10.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3fdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380b3820 con 0x7fdf380cf080 2023-07-06T03:53:10.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3fdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380b2ca0 con 0x7fdf380b1fc0 2023-07-06T03:53:10.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3fdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380b2f80 con 0x7fdf380b2390 2023-07-06T03:53:10.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3e39c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf380cf080 0x7fdf38158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:10.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3e39c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf380cf080 0x7fdf38158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54556/0 (socket says 172.21.15.97:54556) 2023-07-06T03:53:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3e39c700 1 -- 172.21.15.97:0/1965541667 learned_addr learned my addr 172.21.15.97:0/1965541667 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3db9b700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf380b1fc0 0x7fdf381f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3d39a700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf380b2390 0x7fdf380cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3e39c700 1 -- 172.21.15.97:0/1965541667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf380b2390 msgr2=0x7fdf380cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.345+0000 7fdf3e39c700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf380b2390 0x7fdf380cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.346+0000 7fdf3e39c700 1 -- 172.21.15.97:0/1965541667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf380b1fc0 msgr2=0x7fdf381f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.346+0000 7fdf3e39c700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf380b1fc0 0x7fdf381f29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.346+0000 7fdf3e39c700 1 -- 172.21.15.97:0/1965541667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34009550 con 0x7fdf380cf080 2023-07-06T03:53:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.346+0000 7fdf3db9b700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf380b1fc0 0x7fdf381f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.346+0000 7fdf3e39c700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf380cf080 0x7fdf38158150 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000cb50 tx=0x7fdf3000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.346+0000 7fdf2e7fc700 1 -- 172.21.15.97:0/1965541667 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30003c00 con 0x7fdf380cf080 2023-07-06T03:53:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.346+0000 7fdf2e7fc700 1 -- 172.21.15.97:0/1965541667 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf30003d60 con 0x7fdf380cf080 2023-07-06T03:53:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.346+0000 7fdf2e7fc700 1 -- 172.21.15.97:0/1965541667 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3000ab70 con 0x7fdf380cf080 2023-07-06T03:53:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.346+0000 7fdf3fdff700 1 -- 172.21.15.97:0/1965541667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf380b3260 con 0x7fdf380cf080 2023-07-06T03:53:10.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.346+0000 7fdf3fdff700 1 -- 172.21.15.97:0/1965541667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf380b4750 con 0x7fdf380cf080 2023-07-06T03:53:10.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.347+0000 7fdf3fdff700 1 -- 172.21.15.97:0/1965541667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf38074350 con 0x7fdf380cf080 2023-07-06T03:53:10.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.348+0000 7fdf2e7fc700 1 -- 172.21.15.97:0/1965541667 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf3000acd0 con 0x7fdf380cf080 2023-07-06T03:53:10.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.349+0000 7fdf2e7fc700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf1c06ace0 0x7fdf1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:10.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.349+0000 7fdf2e7fc700 1 -- 172.21.15.97:0/1965541667 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fdf30089630 con 0x7fdf380cf080 2023-07-06T03:53:10.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.349+0000 7fdf3db9b700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf1c06ace0 0x7fdf1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:10.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.350+0000 7fdf3db9b700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf1c06ace0 0x7fdf1c06d190 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009c40 tx=0x7fdf34008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:10.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.353+0000 7fdf2e7fc700 1 -- 172.21.15.97:0/1965541667 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf30054590 con 0x7fdf380cf080 2023-07-06T03:53:10.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.546+0000 7fdf3fdff700 1 -- 172.21.15.97:0/1965541667 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf3806b580 con 0x7fdf380cf080 2023-07-06T03:53:10.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.547+0000 7fdf2e7fc700 1 -- 172.21.15.97:0/1965541667 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf30057bb0 con 0x7fdf380cf080 2023-07-06T03:53:10.550 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:53:10.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 -- 172.21.15.97:0/1965541667 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf1c06ace0 msgr2=0x7fdf1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf1c06ace0 0x7fdf1c06d190 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009c40 tx=0x7fdf34008030 comp rx=0 tx=0).stop 2023-07-06T03:53:10.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 -- 172.21.15.97:0/1965541667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf380cf080 msgr2=0x7fdf38158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf380cf080 0x7fdf38158150 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000cb50 tx=0x7fdf3000cf10 comp rx=0 tx=0).stop 2023-07-06T03:53:10.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 -- 172.21.15.97:0/1965541667 shutdown_connections 2023-07-06T03:53:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf1c06ace0 0x7fdf1c06d190 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf380cf080 0x7fdf38158150 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf380b1fc0 0x7fdf381f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 --2- 172.21.15.97:0/1965541667 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf380b2390 0x7fdf380cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 -- 172.21.15.97:0/1965541667 >> 172.21.15.97:0/1965541667 conn(0x7fdf3806c210 msgr2=0x7fdf380b6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 -- 172.21.15.97:0/1965541667 shutdown_connections 2023-07-06T03:53:10.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.549+0000 7fdf3fdff700 1 -- 172.21.15.97:0/1965541667 wait complete. 2023-07-06T03:53:10.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:53:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.760+0000 7f0fcdfff700 1 Processor -- start 2023-07-06T03:53:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.760+0000 7f0fcdfff700 1 -- start start 2023-07-06T03:53:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.761+0000 7f0fcdfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fc815c960 0x7f0fc80d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.762+0000 7f0fcdfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc80afdd0 con 0x7f0fc80b44d0 2023-07-06T03:53:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.762+0000 7f0fcdfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc80b00b0 con 0x7f0fc80b1e80 2023-07-06T03:53:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.762+0000 7f0fcdfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc80b0390 con 0x7f0fc815c960 2023-07-06T03:53:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.762+0000 7f0fbffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0fc80b44d0 0x7f0fc80b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36826/0 (socket says 172.21.15.97:36826) 2023-07-06T03:53:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.762+0000 7f0fbffff700 1 -- 172.21.15.97:0/1542965583 learned_addr learned my addr 172.21.15.97:0/1542965583 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.762+0000 7f0fc77fe700 1 --2- 172.21.15.97:0/1542965583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fc815c960 0x7f0fc80d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.762+0000 7f0fc6ffd700 1 -- 172.21.15.97:0/1542965583 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868437547 0 0) 0x7f0fc80afdd0 con 0x7f0fc80b44d0 2023-07-06T03:53:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.762+0000 7f0fc6ffd700 1 -- 172.21.15.97:0/1542965583 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fb40035b0 con 0x7f0fc80b44d0 2023-07-06T03:53:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.762+0000 7f0fc6ffd700 1 -- 172.21.15.97:0/1542965583 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222002199 0 0) 0x7f0fc80b00b0 con 0x7f0fc80b1e80 2023-07-06T03:53:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.762+0000 7f0fc6ffd700 1 -- 172.21.15.97:0/1542965583 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fc80afdd0 con 0x7f0fc80b1e80 2023-07-06T03:53:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.763+0000 7f0fc77fe700 1 -- 172.21.15.97:0/1542965583 >> v1:172.21.15.146:6789/0 conn(0x7f0fc80b1e80 legacy=0x7f0fc80b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.763+0000 7f0fc77fe700 1 -- 172.21.15.97:0/1542965583 >> v1:172.21.15.97:6789/0 conn(0x7f0fc80b44d0 legacy=0x7f0fc80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.763+0000 7f0fc77fe700 1 -- 172.21.15.97:0/1542965583 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc80b0670 con 0x7f0fc815c960 2023-07-06T03:53:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.763+0000 7f0fc77fe700 1 --2- 172.21.15.97:0/1542965583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fc815c960 0x7f0fc80d0ab0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009870 tx=0x7f0fb8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cf611bb4a34720f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.763+0000 7f0fc6ffd700 1 -- 172.21.15.97:0/1542965583 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8011830 con 0x7f0fc815c960 2023-07-06T03:53:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.763+0000 7f0fc6ffd700 1 -- 172.21.15.97:0/1542965583 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb8011990 con 0x7f0fc815c960 2023-07-06T03:53:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.763+0000 7f0fc6ffd700 1 -- 172.21.15.97:0/1542965583 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8011d20 con 0x7f0fc815c960 2023-07-06T03:53:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.764+0000 7f0fcdfff700 1 -- 172.21.15.97:0/1542965583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fc815c960 msgr2=0x7f0fc80d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.764+0000 7f0fcdfff700 1 --2- 172.21.15.97:0/1542965583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fc815c960 0x7f0fc80d0ab0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009870 tx=0x7f0fb8009b80 comp rx=0 tx=0).stop 2023-07-06T03:53:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.764+0000 7f0fcdfff700 1 -- 172.21.15.97:0/1542965583 shutdown_connections 2023-07-06T03:53:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.764+0000 7f0fcdfff700 1 --2- 172.21.15.97:0/1542965583 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fc815c960 0x7f0fc80d0ab0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.764+0000 7f0fcdfff700 1 -- 172.21.15.97:0/1542965583 >> 172.21.15.97:0/1542965583 conn(0x7f0fc809c1b0 msgr2=0x7f0fc80d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.764+0000 7f0fcdfff700 1 -- 172.21.15.97:0/1542965583 shutdown_connections 2023-07-06T03:53:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.764+0000 7f0fcdfff700 1 -- 172.21.15.97:0/1542965583 wait complete. 2023-07-06T03:53:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.766+0000 7f0fcdfff700 1 Processor -- start 2023-07-06T03:53:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.766+0000 7f0fcdfff700 1 -- start start 2023-07-06T03:53:10.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.766+0000 7f0fcdfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fc80b1e80 0x7f0fc8159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:10.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.766+0000 7f0fcdfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc80b44d0 0x7f0fc80c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fcdfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fc815c960 0x7f0fc80ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fcdfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc815a140 con 0x7f0fc80b44d0 2023-07-06T03:53:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fcdfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc815a330 con 0x7f0fc815c960 2023-07-06T03:53:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fcdfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc815a610 con 0x7f0fc80b1e80 2023-07-06T03:53:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fbffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc80b44d0 0x7f0fc80c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fbffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc80b44d0 0x7f0fc80c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54570/0 (socket says 172.21.15.97:54570) 2023-07-06T03:53:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fbffff700 1 -- 172.21.15.97:0/2077371840 learned_addr learned my addr 172.21.15.97:0/2077371840 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:10.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fc77fe700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fc80b1e80 0x7f0fc8159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:10.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fc7fff700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fc815c960 0x7f0fc80ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:10.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fbffff700 1 -- 172.21.15.97:0/2077371840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fc80b1e80 msgr2=0x7f0fc8159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fbffff700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fc80b1e80 0x7f0fc8159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fbffff700 1 -- 172.21.15.97:0/2077371840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fc815c960 msgr2=0x7f0fc80ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:10.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fbffff700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fc815c960 0x7f0fc80ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:10.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fbffff700 1 -- 172.21.15.97:0/2077371840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb80094a0 con 0x7f0fc80b44d0 2023-07-06T03:53:10.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fc7fff700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fc815c960 0x7f0fc80ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:10.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.767+0000 7f0fbffff700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc80b44d0 0x7f0fc80c9fb0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c4d0 tx=0x7f0fb000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:10.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.768+0000 7f0fc4ff9700 1 -- 172.21.15.97:0/2077371840 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0005d70 con 0x7f0fc80b44d0 2023-07-06T03:53:10.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.768+0000 7f0fc4ff9700 1 -- 172.21.15.97:0/2077371840 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb00070d0 con 0x7f0fc80b44d0 2023-07-06T03:53:10.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.768+0000 7f0fc4ff9700 1 -- 172.21.15.97:0/2077371840 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0007480 con 0x7f0fc80b44d0 2023-07-06T03:53:10.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.768+0000 7f0fcdfff700 1 -- 172.21.15.97:0/2077371840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc815a8f0 con 0x7f0fc80b44d0 2023-07-06T03:53:10.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.768+0000 7f0fcdfff700 1 -- 172.21.15.97:0/2077371840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc815aeb0 con 0x7f0fc80b44d0 2023-07-06T03:53:10.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.769+0000 7f0fcdfff700 1 -- 172.21.15.97:0/2077371840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc8074350 con 0x7f0fc80b44d0 2023-07-06T03:53:10.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.770+0000 7f0fc4ff9700 1 -- 172.21.15.97:0/2077371840 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0fb00075e0 con 0x7f0fc80b44d0 2023-07-06T03:53:10.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.771+0000 7f0fc4ff9700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fa806ace0 0x7f0fa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:10.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.771+0000 7f0fc4ff9700 1 -- 172.21.15.97:0/2077371840 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f0fb0088be0 con 0x7f0fc80b44d0 2023-07-06T03:53:10.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.771+0000 7f0fc77fe700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fa806ace0 0x7f0fa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.771+0000 7f0fc77fe700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fa806ace0 0x7f0fa806d190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80097c0 tx=0x7f0fb801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:10.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:10.775+0000 7f0fc4ff9700 1 -- 172.21.15.97:0/2077371840 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0fb0053b20 con 0x7f0fc80b44d0 2023-07-06T03:53:11.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.014+0000 7f0fcdfff700 1 -- 172.21.15.97:0/2077371840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fc80d1820 con 0x7f0fc80b44d0 2023-07-06T03:53:11.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.014+0000 7f0fc4ff9700 1 -- 172.21.15.97:0/2077371840 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f0fb0057140 con 0x7f0fc80b44d0 2023-07-06T03:53:11.017 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:11.017 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3945452207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3945452207},{"type":"v1","addr":"172.21.15.146:6835","nonce":3945452207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2428591387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2428591387},{"type":"v1","addr":"172.21.15.97:6837","nonce":2428591387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/2089126027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2089126027},{"type":"v1","addr":"172.21.15.146:6834","nonce":2089126027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:52:34.447330+0000","modified":"2023-07-06T03:52:59.797803+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7497":{"gid":7497,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6836/2046478602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2046478602},{"type":"v1","addr":"172.21.15.97:6836","nonce":2046478602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-06T03:53:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 -- 172.21.15.97:0/2077371840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fa806ace0 msgr2=0x7f0fa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fa806ace0 0x7f0fa806d190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80097c0 tx=0x7f0fb801c040 comp rx=0 tx=0).stop 2023-07-06T03:53:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 -- 172.21.15.97:0/2077371840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc80b44d0 msgr2=0x7f0fc80c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc80b44d0 0x7f0fc80c9fb0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c4d0 tx=0x7f0fb000c890 comp rx=0 tx=0).stop 2023-07-06T03:53:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 -- 172.21.15.97:0/2077371840 shutdown_connections 2023-07-06T03:53:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fa806ace0 0x7f0fa806d190 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fc815c960 0x7f0fc80ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fc80b1e80 0x7f0fc8159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 --2- 172.21.15.97:0/2077371840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc80b44d0 0x7f0fc80c9fb0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 -- 172.21.15.97:0/2077371840 >> 172.21.15.97:0/2077371840 conn(0x7f0fc809c1b0 msgr2=0x7f0fc80c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 -- 172.21.15.97:0/2077371840 shutdown_connections 2023-07-06T03:53:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.017+0000 7f0fcdfff700 1 -- 172.21.15.97:0/2077371840 wait complete. 2023-07-06T03:53:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 57 2023-07-06T03:53:11.037 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:53:11.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.230+0000 7ffa6cbff700 1 Processor -- start 2023-07-06T03:53:11.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.230+0000 7ffa6cbff700 1 -- start start 2023-07-06T03:53:11.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.231+0000 7ffa6cbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6815c8b0 0x7ffa680b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.231+0000 7ffa6cbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffa680b5550 0x7ffa680b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.231+0000 7ffa6cbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa680b4e10 0x7ffa681601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.231+0000 7ffa6cbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa680b5270 con 0x7ffa680b4e10 2023-07-06T03:53:11.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.231+0000 7ffa6cbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa680cdaf0 con 0x7ffa680b5550 2023-07-06T03:53:11.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.231+0000 7ffa6cbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680cff30 con 0x7ffa6815c8b0 2023-07-06T03:53:11.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.231+0000 7ffa66d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa680b4e10 0x7ffa681601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.231+0000 7ffa66d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa680b4e10 0x7ffa681601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54586/0 (socket says 172.21.15.97:54586) 2023-07-06T03:53:11.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.231+0000 7ffa66d9d700 1 -- 172.21.15.97:0/3845495054 learned_addr learned my addr 172.21.15.97:0/3845495054 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:11.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.231+0000 7ffa5ffff700 1 --2- 172.21.15.97:0/3845495054 >> v2:172.21.15.146:3300/0 conn(0x7ffa680b5550 0x7ffa680b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.231+0000 7ffa6659c700 1 --2- 172.21.15.97:0/3845495054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6815c8b0 0x7ffa680b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.232+0000 7ffa66d9d700 1 -- 172.21.15.97:0/3845495054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6815c8b0 msgr2=0x7ffa680b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.232+0000 7ffa66d9d700 1 --2- 172.21.15.97:0/3845495054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6815c8b0 0x7ffa680b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.232+0000 7ffa66d9d700 1 -- 172.21.15.97:0/3845495054 >> v2:172.21.15.146:3300/0 conn(0x7ffa680b5550 msgr2=0x7ffa680b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.232+0000 7ffa66d9d700 1 --2- 172.21.15.97:0/3845495054 >> v2:172.21.15.146:3300/0 conn(0x7ffa680b5550 0x7ffa680b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.232+0000 7ffa66d9d700 1 -- 172.21.15.97:0/3845495054 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa680d0210 con 0x7ffa680b4e10 2023-07-06T03:53:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.232+0000 7ffa5ffff700 1 --2- 172.21.15.97:0/3845495054 >> v2:172.21.15.146:3300/0 conn(0x7ffa680b5550 0x7ffa680b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.232+0000 7ffa66d9d700 1 --2- 172.21.15.97:0/3845495054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa680b4e10 0x7ffa681601f0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ffa6808c010 tx=0x7ffa6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3053b986d0246ae5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.232+0000 7ffa65d9b700 1 -- 172.21.15.97:0/3845495054 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000e070 con 0x7ffa680b4e10 2023-07-06T03:53:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.232+0000 7ffa65d9b700 1 -- 172.21.15.97:0/3845495054 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6000b910 con 0x7ffa680b4e10 2023-07-06T03:53:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.232+0000 7ffa65d9b700 1 -- 172.21.15.97:0/3845495054 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000bcc0 con 0x7ffa680b4e10 2023-07-06T03:53:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.233+0000 7ffa6cbff700 1 -- 172.21.15.97:0/3845495054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa680b4e10 msgr2=0x7ffa681601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.233+0000 7ffa6cbff700 1 --2- 172.21.15.97:0/3845495054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa680b4e10 0x7ffa681601f0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ffa6808c010 tx=0x7ffa6000b330 comp rx=0 tx=0).stop 2023-07-06T03:53:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.233+0000 7ffa6cbff700 1 -- 172.21.15.97:0/3845495054 shutdown_connections 2023-07-06T03:53:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.233+0000 7ffa6cbff700 1 --2- 172.21.15.97:0/3845495054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa680b4e10 0x7ffa681601f0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.233+0000 7ffa6cbff700 1 --2- 172.21.15.97:0/3845495054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa6815c8b0 0x7ffa680b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.233+0000 7ffa6cbff700 1 --2- 172.21.15.97:0/3845495054 >> v2:172.21.15.146:3300/0 conn(0x7ffa680b5550 0x7ffa680b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.233+0000 7ffa6cbff700 1 -- 172.21.15.97:0/3845495054 >> 172.21.15.97:0/3845495054 conn(0x7ffa68066990 msgr2=0x7ffa6806a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:11.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.233+0000 7ffa6cbff700 1 -- 172.21.15.97:0/3845495054 shutdown_connections 2023-07-06T03:53:11.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.233+0000 7ffa6cbff700 1 -- 172.21.15.97:0/3845495054 wait complete. 2023-07-06T03:53:11.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.234+0000 7ffa6cbff700 1 Processor -- start 2023-07-06T03:53:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.235+0000 7ffa6cbff700 1 -- start start 2023-07-06T03:53:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.235+0000 7ffa6cbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa680b4e10 0x7ffa681e8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.235+0000 7ffa6cbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa680b5550 0x7ffa68152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa6cbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa6815c8b0 0x7ffa680b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa6cbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa681e81a0 con 0x7ffa680b5550 2023-07-06T03:53:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa6cbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680b7a40 con 0x7ffa6815c8b0 2023-07-06T03:53:11.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa6cbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680b7110 con 0x7ffa680b4e10 2023-07-06T03:53:11.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa680b5550 0x7ffa68152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa680b5550 0x7ffa68152c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54588/0 (socket says 172.21.15.97:54588) 2023-07-06T03:53:11.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa5ffff700 1 -- 172.21.15.97:0/2694351985 learned_addr learned my addr 172.21.15.97:0/2694351985 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:11.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa6659c700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa680b4e10 0x7ffa681e8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa66d9d700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa6815c8b0 0x7ffa680b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa5ffff700 1 -- 172.21.15.97:0/2694351985 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa680b4e10 msgr2=0x7ffa681e8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa5ffff700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa680b4e10 0x7ffa681e8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa5ffff700 1 -- 172.21.15.97:0/2694351985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa6815c8b0 msgr2=0x7ffa680b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa5ffff700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa6815c8b0 0x7ffa680b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa5ffff700 1 -- 172.21.15.97:0/2694351985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa60009b30 con 0x7ffa680b5550 2023-07-06T03:53:11.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.236+0000 7ffa5ffff700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa680b5550 0x7ffa68152c70 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000e7a0 tx=0x7ffa5000eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:11.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.237+0000 7ffa5effd700 1 -- 172.21.15.97:0/2694351985 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50013610 con 0x7ffa680b5550 2023-07-06T03:53:11.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.237+0000 7ffa5effd700 1 -- 172.21.15.97:0/2694351985 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa50013770 con 0x7ffa680b5550 2023-07-06T03:53:11.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.237+0000 7ffa6cbff700 1 -- 172.21.15.97:0/2694351985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa680b73f0 con 0x7ffa680b5550 2023-07-06T03:53:11.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.237+0000 7ffa5effd700 1 -- 172.21.15.97:0/2694351985 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50013b00 con 0x7ffa680b5550 2023-07-06T03:53:11.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.237+0000 7ffa6cbff700 1 -- 172.21.15.97:0/2694351985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa681e79c0 con 0x7ffa680b5550 2023-07-06T03:53:11.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.238+0000 7ffa6cbff700 1 -- 172.21.15.97:0/2694351985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa680cf5b0 con 0x7ffa680b5550 2023-07-06T03:53:11.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.239+0000 7ffa5effd700 1 -- 172.21.15.97:0/2694351985 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa50013c60 con 0x7ffa680b5550 2023-07-06T03:53:11.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.240+0000 7ffa5effd700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa4806ace0 0x7ffa4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.240+0000 7ffa5effd700 1 -- 172.21.15.97:0/2694351985 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7ffa50089140 con 0x7ffa680b5550 2023-07-06T03:53:11.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.240+0000 7ffa6659c700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa4806ace0 0x7ffa4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.240+0000 7ffa6659c700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa4806ace0 0x7ffa4806d190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009b60 tx=0x7ffa58009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:11.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.244+0000 7ffa5effd700 1 -- 172.21.15.97:0/2694351985 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffa50057b10 con 0x7ffa680b5550 2023-07-06T03:53:11.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.436+0000 7ffa6cbff700 1 -- 172.21.15.97:0/2694351985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa6815f550 con 0x7ffa680b5550 2023-07-06T03:53:11.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.438+0000 7ffa5effd700 1 -- 172.21.15.97:0/2694351985 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+18706 (secure 0 0 0) 0x7ffa500576a0 con 0x7ffa680b5550 2023-07-06T03:53:11.441 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:11.441 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:52:53.756629+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:52:32.448828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-06T03:52:33.428266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.12","mappings":[{"from":6,"to":4}]},{"pgid":"14.15","mappings":[{"from":1,"to":0}]},{"pgid":"14.1a","mappings":[{"from":7,"to":3}]},{"pgid":"14.26","mappings":[{"from":7,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.34","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:53:11.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 -- 172.21.15.97:0/2694351985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa4806ace0 msgr2=0x7ffa4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa4806ace0 0x7ffa4806d190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009b60 tx=0x7ffa58009210 comp rx=0 tx=0).stop 2023-07-06T03:53:11.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 -- 172.21.15.97:0/2694351985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa680b5550 msgr2=0x7ffa68152c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa680b5550 0x7ffa68152c70 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000e7a0 tx=0x7ffa5000eab0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 -- 172.21.15.97:0/2694351985 shutdown_connections 2023-07-06T03:53:11.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa4806ace0 0x7ffa4806d190 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa6815c8b0 0x7ffa680b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa680b4e10 0x7ffa681e8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 --2- 172.21.15.97:0/2694351985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa680b5550 0x7ffa68152c70 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 -- 172.21.15.97:0/2694351985 >> 172.21.15.97:0/2694351985 conn(0x7ffa68066990 msgr2=0x7ffa6815cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:11.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 -- 172.21.15.97:0/2694351985 shutdown_connections 2023-07-06T03:53:11.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.440+0000 7ffa6cbff700 1 -- 172.21.15.97:0/2694351985 wait complete. 2023-07-06T03:53:11.462 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T03:53:11.462 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:53:11.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.654+0000 7f8de6fff700 1 Processor -- start 2023-07-06T03:53:11.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.654+0000 7f8de6fff700 1 -- start start 2023-07-06T03:53:11.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.655+0000 7f8de6fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8de00b4fe0 0x7f8de00b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.655+0000 7f8de6fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de00cdaf0 con 0x7f8de00b5550 2023-07-06T03:53:11.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.655+0000 7f8de6fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de00cff30 con 0x7f8de015c8b0 2023-07-06T03:53:11.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.655+0000 7f8de6fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de00d0210 con 0x7f8de00b4fe0 2023-07-06T03:53:11.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8de00b5550 0x7f8de00d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36830/0 (socket says 172.21.15.97:36830) 2023-07-06T03:53:11.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddffff700 1 -- 172.21.15.97:0/2249035970 learned_addr learned my addr 172.21.15.97:0/2249035970 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:11.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8de559c700 1 --2- 172.21.15.97:0/2249035970 >> v2:172.21.15.146:3300/0 conn(0x7f8de00b4fe0 0x7f8de00b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65447576 0 0) 0x7f8de00cdaf0 con 0x7f8de00b5550 2023-07-06T03:53:11.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dd00035b0 con 0x7f8de00b5550 2023-07-06T03:53:11.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 615997236 0 0) 0x7f8dd00035b0 con 0x7f8de00b5550 2023-07-06T03:53:11.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 >> v2:172.21.15.146:3300/0 conn(0x7f8de00b4fe0 msgr2=0x7f8de00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:11.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 --2- 172.21.15.97:0/2249035970 >> v2:172.21.15.146:3300/0 conn(0x7f8de00b4fe0 0x7f8de00b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 >> v1:172.21.15.146:6789/0 conn(0x7f8de015c8b0 legacy=0x7f8de00b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de00cd4a0 con 0x7f8de00b5550 2023-07-06T03:53:11.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f8dcc001670 con 0x7f8de00b5550 2023-07-06T03:53:11.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 >> v1:172.21.15.97:6789/0 conn(0x7f8de00b5550 legacy=0x7f8de00d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 --2- 172.21.15.97:0/2249035970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dd0003a50 0x7f8dd0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 --2- 172.21.15.97:0/2249035970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dd0004850 0x7f8dd0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 --2- 172.21.15.97:0/2249035970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd0005200 0x7f8dd000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00d0210 con 0x7f8dd0005200 2023-07-06T03:53:11.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd000c0e0 con 0x7f8dd0003a50 2023-07-06T03:53:11.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.656+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd000c2a0 con 0x7f8dd0004850 2023-07-06T03:53:11.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.657+0000 7f8de559c700 1 --2- 172.21.15.97:0/2249035970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd0005200 0x7f8dd000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.657+0000 7f8de4d9b700 1 --2- 172.21.15.97:0/2249035970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dd0003a50 0x7f8dd0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.657+0000 7f8ddffff700 1 --2- 172.21.15.97:0/2249035970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dd0004850 0x7f8dd0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.657+0000 7f8de559c700 1 -- 172.21.15.97:0/2249035970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dd0004850 msgr2=0x7f8dd0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.657+0000 7f8de559c700 1 --2- 172.21.15.97:0/2249035970 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dd0004850 0x7f8dd0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.657+0000 7f8de559c700 1 -- 172.21.15.97:0/2249035970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dd0003a50 msgr2=0x7f8dd0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.657+0000 7f8de559c700 1 --2- 172.21.15.97:0/2249035970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8dd0003a50 0x7f8dd0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.657+0000 7f8de559c700 1 -- 172.21.15.97:0/2249035970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd00035b0 con 0x7f8dd0005200 2023-07-06T03:53:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.657+0000 7f8de559c700 1 -- 172.21.15.97:0/2249035970 reap_dead start 2023-07-06T03:53:11.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.657+0000 7f8de559c700 1 --2- 172.21.15.97:0/2249035970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd0005200 0x7f8dd000ba10 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f8de006b880 tx=0x7f8dd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b869bd836a8e1e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:11.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.658+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800ebd0 con 0x7f8dd0005200 2023-07-06T03:53:11.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.658+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd800f040 con 0x7f8dd0005200 2023-07-06T03:53:11.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.658+0000 7f8ddf7fe700 1 -- 172.21.15.97:0/2249035970 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8014480 con 0x7f8dd0005200 2023-07-06T03:53:11.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.658+0000 7f8de6fff700 1 -- 172.21.15.97:0/2249035970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd0005200 msgr2=0x7f8dd000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.658+0000 7f8de6fff700 1 --2- 172.21.15.97:0/2249035970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd0005200 0x7f8dd000ba10 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f8de006b880 tx=0x7f8dd0005880 comp rx=0 tx=0).stop 2023-07-06T03:53:11.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.658+0000 7f8de6fff700 1 -- 172.21.15.97:0/2249035970 shutdown_connections 2023-07-06T03:53:11.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.658+0000 7f8de6fff700 1 --2- 172.21.15.97:0/2249035970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd0005200 0x7f8dd000ba10 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.658+0000 7f8de6fff700 1 -- 172.21.15.97:0/2249035970 >> 172.21.15.97:0/2249035970 conn(0x7f8de0066990 msgr2=0x7f8de006a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:11.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.658+0000 7f8de6fff700 1 -- 172.21.15.97:0/2249035970 shutdown_connections 2023-07-06T03:53:11.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.658+0000 7f8de6fff700 1 -- 172.21.15.97:0/2249035970 wait complete. 2023-07-06T03:53:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.660+0000 7f8de6fff700 1 Processor -- start 2023-07-06T03:53:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.660+0000 7f8de6fff700 1 -- start start 2023-07-06T03:53:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.661+0000 7f8de6fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dd0005200 0x7f8de01f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.661+0000 7f8de6fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8de01533f0 0x7f8de00b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.661+0000 7f8de6fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8de01f35b0 0x7f8de015e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.661+0000 7f8de6fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de0155850 con 0x7f8de01533f0 2023-07-06T03:53:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.661+0000 7f8de6fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de0153860 con 0x7f8de01f35b0 2023-07-06T03:53:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.661+0000 7f8de6fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de0153cc0 con 0x7f8dd0005200 2023-07-06T03:53:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.661+0000 7f8ddffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8de01533f0 0x7f8de00b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.661+0000 7f8ddffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8de01533f0 0x7f8de00b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54614/0 (socket says 172.21.15.97:54614) 2023-07-06T03:53:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.661+0000 7f8ddffff700 1 -- 172.21.15.97:0/491469756 learned_addr learned my addr 172.21.15.97:0/491469756 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8ddffff700 1 -- 172.21.15.97:0/491469756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dd0005200 msgr2=0x7f8de01f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8de4d9b700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dd0005200 0x7f8de01f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8de559c700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8de01f35b0 0x7f8de015e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8ddffff700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dd0005200 0x7f8de01f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8ddffff700 1 -- 172.21.15.97:0/491469756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8de01f35b0 msgr2=0x7f8de015e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8ddffff700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8de01f35b0 0x7f8de015e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8ddffff700 1 -- 172.21.15.97:0/491469756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de01f2a10 con 0x7f8de01533f0 2023-07-06T03:53:11.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8de4d9b700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dd0005200 0x7f8de01f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8de559c700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8de01f35b0 0x7f8de015e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8ddffff700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8de01533f0 0x7f8de00b6690 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc000c00 tx=0x7f8dcc007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8ddd7fa700 1 -- 172.21.15.97:0/491469756 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc011580 con 0x7f8de01533f0 2023-07-06T03:53:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8ddd7fa700 1 -- 172.21.15.97:0/491469756 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc011720 con 0x7f8de01533f0 2023-07-06T03:53:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8ddd7fa700 1 -- 172.21.15.97:0/491469756 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc011ad0 con 0x7f8de01533f0 2023-07-06T03:53:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8de6fff700 1 -- 172.21.15.97:0/491469756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de01f2cf0 con 0x7f8de01533f0 2023-07-06T03:53:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.662+0000 7f8de6fff700 1 -- 172.21.15.97:0/491469756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de01f32b0 con 0x7f8de01533f0 2023-07-06T03:53:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.663+0000 7f8de6fff700 1 -- 172.21.15.97:0/491469756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de0153e70 con 0x7f8de01533f0 2023-07-06T03:53:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.664+0000 7f8ddd7fa700 1 -- 172.21.15.97:0/491469756 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8dcc011c30 con 0x7f8de01533f0 2023-07-06T03:53:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.666+0000 7f8ddd7fa700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8dc406ace0 0x7f8dc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.666+0000 7f8ddd7fa700 1 -- 172.21.15.97:0/491469756 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f8dcc089570 con 0x7f8de01533f0 2023-07-06T03:53:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.666+0000 7f8de4d9b700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8dc406ace0 0x7f8dc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.667+0000 7f8de4d9b700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8dc406ace0 0x7f8dc406d190 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8de015f3a0 tx=0x7f8dd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.670+0000 7f8ddd7fa700 1 -- 172.21.15.97:0/491469756 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8de0153e70 con 0x7f8de01533f0 2023-07-06T03:53:11.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.908+0000 7f8de6fff700 1 -- 172.21.15.97:0/491469756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8de0153e30 con 0x7f8de01533f0 2023-07-06T03:53:11.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.909+0000 7f8ddd7fa700 1 -- 172.21.15.97:0/491469756 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4680 (secure 0 0 0) 0x7f8dcc0544b0 con 0x7f8de01533f0 2023-07-06T03:53:11.912 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:11.912 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3945452207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3945452207},{"type":"v1","addr":"172.21.15.146:6835","nonce":3945452207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2428591387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2428591387},{"type":"v1","addr":"172.21.15.97:6837","nonce":2428591387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/2089126027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2089126027},{"type":"v1","addr":"172.21.15.146:6834","nonce":2089126027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:52:34.447330+0000","modified":"2023-07-06T03:53:11.892749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7497":{"gid":7497,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6836/2046478602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2046478602},{"type":"v1","addr":"172.21.15.97:6836","nonce":2046478602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-06T03:53:11.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 -- 172.21.15.97:0/491469756 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8dc406ace0 msgr2=0x7f8dc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8dc406ace0 0x7f8dc406d190 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8de015f3a0 tx=0x7f8dd4009040 comp rx=0 tx=0).stop 2023-07-06T03:53:11.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 -- 172.21.15.97:0/491469756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8de01533f0 msgr2=0x7f8de00b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:11.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8de01533f0 0x7f8de00b6690 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc000c00 tx=0x7f8dcc007570 comp rx=0 tx=0).stop 2023-07-06T03:53:11.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 -- 172.21.15.97:0/491469756 shutdown_connections 2023-07-06T03:53:11.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8dc406ace0 0x7f8dc406d190 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8de01f35b0 0x7f8de015e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8dd0005200 0x7f8de01f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 --2- 172.21.15.97:0/491469756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8de01533f0 0x7f8de00b6690 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:11.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 -- 172.21.15.97:0/491469756 >> 172.21.15.97:0/491469756 conn(0x7f8de0066990 msgr2=0x7f8de015cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:11.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 -- 172.21.15.97:0/491469756 shutdown_connections 2023-07-06T03:53:11.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:11.911+0000 7f8de6fff700 1 -- 172.21.15.97:0/491469756 wait complete. 2023-07-06T03:53:11.916 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 58 2023-07-06T03:53:11.932 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:53:12.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.124+0000 7f505bbff700 1 Processor -- start 2023-07-06T03:53:12.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.124+0000 7f505bbff700 1 -- start start 2023-07-06T03:53:12.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.125+0000 7f505bbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f505415c8b0 0x7f50540b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.125+0000 7f505bbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f50540b4fe0 0x7f50540d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.125+0000 7f505bbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50540cdaf0 con 0x7f50540b5550 2023-07-06T03:53:12.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.125+0000 7f505bbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50540cff30 con 0x7f50540b4fe0 2023-07-06T03:53:12.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.125+0000 7f505bbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50540d0210 con 0x7f505415c8b0 2023-07-06T03:53:12.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.125+0000 7f505999b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f505415c8b0 0x7f50540b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.125+0000 7f505a19c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f50540b4fe0 0x7f50540d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.125+0000 7f505999b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f505415c8b0 0x7f50540b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:33422/0 (socket says 172.21.15.97:33422) 2023-07-06T03:53:12.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.125+0000 7f505999b700 1 -- 172.21.15.97:0/2313747018 learned_addr learned my addr 172.21.15.97:0/2313747018 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:12.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.126+0000 7f5058999700 1 -- 172.21.15.97:0/2313747018 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233598163 0 0) 0x7f50540cdaf0 con 0x7f50540b5550 2023-07-06T03:53:12.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.126+0000 7f505999b700 1 -- 172.21.15.97:0/2313747018 >> v2:172.21.15.146:3300/0 conn(0x7f50540b4fe0 msgr2=0x7f50540d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:12.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.126+0000 7f505999b700 1 --2- 172.21.15.97:0/2313747018 >> v2:172.21.15.146:3300/0 conn(0x7f50540b4fe0 0x7f50540d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.126+0000 7f505999b700 1 -- 172.21.15.97:0/2313747018 >> v1:172.21.15.146:6789/0 conn(0x7f50540b5550 legacy=0x7f50540b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.126+0000 7f505999b700 1 -- 172.21.15.97:0/2313747018 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50540cd4a0 con 0x7f505415c8b0 2023-07-06T03:53:12.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.126+0000 7f505a19c700 1 --2- 172.21.15.97:0/2313747018 >> v2:172.21.15.146:3300/0 conn(0x7f50540b4fe0 0x7f50540d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:53:12.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.126+0000 7f505999b700 1 --2- 172.21.15.97:0/2313747018 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f505415c8b0 0x7f50540b2280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f50500098f0 tx=0x7f5050009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb7905b51efe137 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:12.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.126+0000 7f5058999700 1 -- 172.21.15.97:0/2313747018 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050011830 con 0x7f505415c8b0 2023-07-06T03:53:12.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.127+0000 7f5058999700 1 -- 172.21.15.97:0/2313747018 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5050011990 con 0x7f505415c8b0 2023-07-06T03:53:12.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.127+0000 7f5058999700 1 -- 172.21.15.97:0/2313747018 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050011d20 con 0x7f505415c8b0 2023-07-06T03:53:12.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.127+0000 7f505bbff700 1 -- 172.21.15.97:0/2313747018 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f505415c8b0 msgr2=0x7f50540b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.127+0000 7f505bbff700 1 --2- 172.21.15.97:0/2313747018 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f505415c8b0 0x7f50540b2280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f50500098f0 tx=0x7f5050009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:12.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.127+0000 7f505bbff700 1 -- 172.21.15.97:0/2313747018 shutdown_connections 2023-07-06T03:53:12.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.127+0000 7f505bbff700 1 --2- 172.21.15.97:0/2313747018 >> v2:172.21.15.146:3300/0 conn(0x7f50540b4fe0 0x7f50540d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.127+0000 7f505bbff700 1 --2- 172.21.15.97:0/2313747018 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f505415c8b0 0x7f50540b2280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.127+0000 7f505bbff700 1 -- 172.21.15.97:0/2313747018 >> 172.21.15.97:0/2313747018 conn(0x7f5054066990 msgr2=0x7f505406a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:12.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.127+0000 7f505bbff700 1 -- 172.21.15.97:0/2313747018 shutdown_connections 2023-07-06T03:53:12.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.127+0000 7f505bbff700 1 -- 172.21.15.97:0/2313747018 wait complete. 2023-07-06T03:53:12.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.129+0000 7f505bbff700 1 Processor -- start 2023-07-06T03:53:12.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.129+0000 7f505bbff700 1 -- start start 2023-07-06T03:53:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.130+0000 7f505bbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50540b4fe0 0x7f50541582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.130+0000 7f505bbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5550 0x7f50540c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.130+0000 7f505bbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505415c8b0 0x7f505415e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.130+0000 7f505bbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50540cb510 con 0x7f50540b5550 2023-07-06T03:53:12.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.130+0000 7f505bbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50540ca890 con 0x7f505415c8b0 2023-07-06T03:53:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.130+0000 7f505bbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50540cab70 con 0x7f50540b4fe0 2023-07-06T03:53:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.130+0000 7f505919a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5550 0x7f50540c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.130+0000 7f505919a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5550 0x7f50540c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54620/0 (socket says 172.21.15.97:54620) 2023-07-06T03:53:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.130+0000 7f505919a700 1 -- 172.21.15.97:0/3703228375 learned_addr learned my addr 172.21.15.97:0/3703228375 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.130+0000 7f505a19c700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505415c8b0 0x7f505415e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.130+0000 7f505999b700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50540b4fe0 0x7f50541582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.131+0000 7f505919a700 1 -- 172.21.15.97:0/3703228375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50540b4fe0 msgr2=0x7f50541582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.131+0000 7f505919a700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50540b4fe0 0x7f50541582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.131+0000 7f505919a700 1 -- 172.21.15.97:0/3703228375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505415c8b0 msgr2=0x7f505415e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.131+0000 7f505919a700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505415c8b0 0x7f505415e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.131+0000 7f505919a700 1 -- 172.21.15.97:0/3703228375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c009a70 con 0x7f50540b5550 2023-07-06T03:53:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.131+0000 7f505919a700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5550 0x7f50540c9f20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f504400c520 tx=0x7f504400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:12.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.131+0000 7f504a7fc700 1 -- 172.21.15.97:0/3703228375 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50440054e0 con 0x7f50540b5550 2023-07-06T03:53:12.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.131+0000 7f504a7fc700 1 -- 172.21.15.97:0/3703228375 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5044003680 con 0x7f50540b5550 2023-07-06T03:53:12.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.131+0000 7f504a7fc700 1 -- 172.21.15.97:0/3703228375 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044003a30 con 0x7f50540b5550 2023-07-06T03:53:12.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.131+0000 7f505bbff700 1 -- 172.21.15.97:0/3703228375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5050009550 con 0x7f50540b5550 2023-07-06T03:53:12.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.131+0000 7f505bbff700 1 -- 172.21.15.97:0/3703228375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50540cb130 con 0x7f50540b5550 2023-07-06T03:53:12.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.132+0000 7f505bbff700 1 -- 172.21.15.97:0/3703228375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50540cb2f0 con 0x7f50540b5550 2023-07-06T03:53:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.133+0000 7f504a7fc700 1 -- 172.21.15.97:0/3703228375 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5044003b90 con 0x7f50540b5550 2023-07-06T03:53:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.134+0000 7f504a7fc700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f503806ace0 0x7f503806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.134+0000 7f504a7fc700 1 -- 172.21.15.97:0/3703228375 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f5044088a00 con 0x7f50540b5550 2023-07-06T03:53:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.134+0000 7f505999b700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f503806ace0 0x7f503806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.135+0000 7f505999b700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f503806ace0 0x7f503806d190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5050009c40 tx=0x7f5050008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.138+0000 7f504a7fc700 1 -- 172.21.15.97:0/3703228375 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5044053940 con 0x7f50540b5550 2023-07-06T03:53:12.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.378+0000 7f505bbff700 1 -- 172.21.15.97:0/3703228375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5054074350 con 0x7f50540b5550 2023-07-06T03:53:12.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.378+0000 7f504a7fc700 1 -- 172.21.15.97:0/3703228375 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4680 (secure 0 0 0) 0x7f5044017090 con 0x7f50540b5550 2023-07-06T03:53:12.381 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:12.381 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3945452207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3945452207},{"type":"v1","addr":"172.21.15.146:6835","nonce":3945452207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2428591387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2428591387},{"type":"v1","addr":"172.21.15.97:6837","nonce":2428591387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/2089126027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2089126027},{"type":"v1","addr":"172.21.15.146:6834","nonce":2089126027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:52:34.447330+0000","modified":"2023-07-06T03:53:11.892749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7497},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7497":{"gid":7497,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6836/2046478602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2046478602},{"type":"v1","addr":"172.21.15.97:6836","nonce":2046478602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-07-06T03:53:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 -- 172.21.15.97:0/3703228375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f503806ace0 msgr2=0x7f503806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f503806ace0 0x7f503806d190 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5050009c40 tx=0x7f5050008030 comp rx=0 tx=0).stop 2023-07-06T03:53:12.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 -- 172.21.15.97:0/3703228375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5550 msgr2=0x7f50540c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5550 0x7f50540c9f20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f504400c520 tx=0x7f504400c830 comp rx=0 tx=0).stop 2023-07-06T03:53:12.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 -- 172.21.15.97:0/3703228375 shutdown_connections 2023-07-06T03:53:12.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f503806ace0 0x7f503806d190 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505415c8b0 0x7f505415e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f50540b4fe0 0x7f50541582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 --2- 172.21.15.97:0/3703228375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5550 0x7f50540c9f20 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 -- 172.21.15.97:0/3703228375 >> 172.21.15.97:0/3703228375 conn(0x7f5054066990 msgr2=0x7f50540b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:12.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 -- 172.21.15.97:0/3703228375 shutdown_connections 2023-07-06T03:53:12.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.381+0000 7f505bbff700 1 -- 172.21.15.97:0/3703228375 wait complete. 2023-07-06T03:53:12.385 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 58 2023-07-06T03:53:12.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:53:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.592+0000 7f961b7ff700 1 Processor -- start 2023-07-06T03:53:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.593+0000 7f961b7ff700 1 -- start start 2023-07-06T03:53:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.594+0000 7f961b7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961415c930 0x7f96140cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.594+0000 7f961b7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140ccbe0 con 0x7f961406ced0 2023-07-06T03:53:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.594+0000 7f961b7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140c99c0 con 0x7f961415da60 2023-07-06T03:53:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.594+0000 7f961b7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140cce50 con 0x7f961415c930 2023-07-06T03:53:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.594+0000 7f9619d9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f961406ced0 0x7f961406fd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36840/0 (socket says 172.21.15.97:36840) 2023-07-06T03:53:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.594+0000 7f9619d9c700 1 -- 172.21.15.97:0/2369649633 learned_addr learned my addr 172.21.15.97:0/2369649633 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.594+0000 7f961959b700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961415c930 0x7f96140cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.594+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439965910 0 0) 0x7f96140ccbe0 con 0x7f961406ced0 2023-07-06T03:53:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96000035b0 con 0x7f961406ced0 2023-07-06T03:53:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75828698 0 0) 0x7f96140c99c0 con 0x7f961415da60 2023-07-06T03:53:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96140ccbe0 con 0x7f961415da60 2023-07-06T03:53:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2492514776 0 0) 0x7f96000035b0 con 0x7f961406ced0 2023-07-06T03:53:12.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961415c930 msgr2=0x7f96140cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:12.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961415c930 0x7f96140cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 >> v1:172.21.15.146:6789/0 conn(0x7f961415da60 legacy=0x7f96140c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96140cd130 con 0x7f961406ced0 2023-07-06T03:53:12.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f960c0019f0 con 0x7f961406ced0 2023-07-06T03:53:12.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 >> v1:172.21.15.97:6789/0 conn(0x7f961406ced0 legacy=0x7f961406fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600003bc0 0x7f9600003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961415c930 0x7f9600004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9600004d90 0x7f9600005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140cce50 con 0x7f9600004d90 2023-07-06T03:53:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960000bac0 con 0x7f9600003bc0 2023-07-06T03:53:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960000bc80 con 0x7f961415c930 2023-07-06T03:53:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f9619d9c700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9600004d90 0x7f9600005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f961959b700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600003bc0 0x7f9600003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.595+0000 7f9618d9a700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961415c930 0x7f9600004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.596+0000 7f9619d9c700 1 -- 172.21.15.97:0/2369649633 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961415c930 msgr2=0x7f9600004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.596+0000 7f9619d9c700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f961415c930 0x7f9600004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.596+0000 7f9619d9c700 1 -- 172.21.15.97:0/2369649633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600003bc0 msgr2=0x7f9600003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.596+0000 7f9619d9c700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600003bc0 0x7f9600003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.596+0000 7f9619d9c700 1 -- 172.21.15.97:0/2369649633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96000035b0 con 0x7f9600004d90 2023-07-06T03:53:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.596+0000 7f9619d9c700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9600004d90 0x7f9600005200 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f960c00a6e0 tx=0x7f9600005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7265179ecc7b3e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.596+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0072d0 con 0x7f9600004d90 2023-07-06T03:53:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.596+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c007430 con 0x7f9600004d90 2023-07-06T03:53:12.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.596+0000 7f960bfff700 1 -- 172.21.15.97:0/2369649633 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0077c0 con 0x7f9600004d90 2023-07-06T03:53:12.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.597+0000 7f961b7ff700 1 -- 172.21.15.97:0/2369649633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9600004d90 msgr2=0x7f9600005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.597+0000 7f961b7ff700 1 --2- 172.21.15.97:0/2369649633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9600004d90 0x7f9600005200 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f960c00a6e0 tx=0x7f9600005880 comp rx=0 tx=0).stop 2023-07-06T03:53:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.597+0000 7f9619d9c700 1 -- 172.21.15.97:0/2369649633 reap_dead start 2023-07-06T03:53:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.597+0000 7f961b7ff700 1 -- 172.21.15.97:0/2369649633 shutdown_connections 2023-07-06T03:53:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.597+0000 7f961b7ff700 1 -- 172.21.15.97:0/2369649633 >> 172.21.15.97:0/2369649633 conn(0x7f96140cb560 msgr2=0x7f96140cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.597+0000 7f961b7ff700 1 -- 172.21.15.97:0/2369649633 shutdown_connections 2023-07-06T03:53:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.597+0000 7f961b7ff700 1 -- 172.21.15.97:0/2369649633 wait complete. 2023-07-06T03:53:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.598+0000 7f961b7ff700 1 Processor -- start 2023-07-06T03:53:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.599+0000 7f961b7ff700 1 -- start start 2023-07-06T03:53:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f961b7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600004d90 0x7f96140b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f961b7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96140ac8d0 0x7f96140acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f961b7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96140b2d00 0x7f96140b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f961b7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961415b140 con 0x7f96140ac8d0 2023-07-06T03:53:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f961b7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140ad280 con 0x7f9600004d90 2023-07-06T03:53:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f961b7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140b36b0 con 0x7f96140b2d00 2023-07-06T03:53:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f9618d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96140ac8d0 0x7f96140acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f9618d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96140ac8d0 0x7f96140acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54636/0 (socket says 172.21.15.97:54636) 2023-07-06T03:53:12.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f9618d9a700 1 -- 172.21.15.97:0/3471755442 learned_addr learned my addr 172.21.15.97:0/3471755442 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:12.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f9619d9c700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96140b2d00 0x7f96140b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f961959b700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600004d90 0x7f96140b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f9618d9a700 1 -- 172.21.15.97:0/3471755442 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96140b2d00 msgr2=0x7f96140b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f9618d9a700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96140b2d00 0x7f96140b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f9618d9a700 1 -- 172.21.15.97:0/3471755442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600004d90 msgr2=0x7f96140b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f9618d9a700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600004d90 0x7f96140b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f9618d9a700 1 -- 172.21.15.97:0/3471755442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96140b14e0 con 0x7f96140ac8d0 2023-07-06T03:53:12.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.600+0000 7f9618d9a700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96140ac8d0 0x7f96140acd40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f960400f8e0 tx=0x7f960400fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:12.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.601+0000 7f9609ffb700 1 -- 172.21.15.97:0/3471755442 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960400fe80 con 0x7f96140ac8d0 2023-07-06T03:53:12.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.601+0000 7f9609ffb700 1 -- 172.21.15.97:0/3471755442 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96040052e0 con 0x7f96140ac8d0 2023-07-06T03:53:12.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.601+0000 7f9609ffb700 1 -- 172.21.15.97:0/3471755442 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604003680 con 0x7f96140ac8d0 2023-07-06T03:53:12.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.601+0000 7f961b7ff700 1 -- 172.21.15.97:0/3471755442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96140b3910 con 0x7f96140ac8d0 2023-07-06T03:53:12.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.601+0000 7f961b7ff700 1 -- 172.21.15.97:0/3471755442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961414bcd0 con 0x7f96140ac8d0 2023-07-06T03:53:12.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.602+0000 7f961b7ff700 1 -- 172.21.15.97:0/3471755442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9614074350 con 0x7f96140ac8d0 2023-07-06T03:53:12.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.603+0000 7f9609ffb700 1 -- 172.21.15.97:0/3471755442 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96040037e0 con 0x7f96140ac8d0 2023-07-06T03:53:12.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.603+0000 7f9609ffb700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f95f806ace0 0x7f95f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:12.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.604+0000 7f9609ffb700 1 -- 172.21.15.97:0/3471755442 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f9604089600 con 0x7f96140ac8d0 2023-07-06T03:53:12.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.604+0000 7f961959b700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f95f806ace0 0x7f95f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:12.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.604+0000 7f961959b700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f95f806ace0 0x7f95f806d190 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9610009ba0 tx=0x7f9610008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:12.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.608+0000 7f9609ffb700 1 -- 172.21.15.97:0/3471755442 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9604054540 con 0x7f96140ac8d0 2023-07-06T03:53:12.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.805+0000 7f961b7ff700 1 -- 172.21.15.97:0/3471755442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96140b4a60 con 0x7f96140ac8d0 2023-07-06T03:53:12.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.806+0000 7f9609ffb700 1 -- 172.21.15.97:0/3471755442 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+18706 (secure 0 0 0) 0x7f9604057b60 con 0x7f96140ac8d0 2023-07-06T03:53:12.809 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:12.809 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:52:53.756629+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:52:32.448828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-07-06T03:52:33.428266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.12","mappings":[{"from":6,"to":4}]},{"pgid":"14.15","mappings":[{"from":1,"to":0}]},{"pgid":"14.1a","mappings":[{"from":7,"to":3}]},{"pgid":"14.26","mappings":[{"from":7,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.34","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:53:12.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.808+0000 7f961b7ff700 1 -- 172.21.15.97:0/3471755442 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f95f806ace0 msgr2=0x7f95f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.808+0000 7f961b7ff700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f95f806ace0 0x7f95f806d190 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9610009ba0 tx=0x7f9610008040 comp rx=0 tx=0).stop 2023-07-06T03:53:12.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.808+0000 7f961b7ff700 1 -- 172.21.15.97:0/3471755442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96140ac8d0 msgr2=0x7f96140acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.808+0000 7f961b7ff700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96140ac8d0 0x7f96140acd40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f960400f8e0 tx=0x7f960400fca0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.808+0000 7f961b7ff700 1 -- 172.21.15.97:0/3471755442 shutdown_connections 2023-07-06T03:53:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.808+0000 7f961b7ff700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f95f806ace0 0x7f95f806d190 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.808+0000 7f961b7ff700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96140b2d00 0x7f96140b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.808+0000 7f961b7ff700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9600004d90 0x7f96140b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.808+0000 7f961b7ff700 1 --2- 172.21.15.97:0/3471755442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96140ac8d0 0x7f96140acd40 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:12.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.808+0000 7f961b7ff700 1 -- 172.21.15.97:0/3471755442 >> 172.21.15.97:0/3471755442 conn(0x7f96140cb560 msgr2=0x7f96140c2610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:12.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.809+0000 7f961b7ff700 1 -- 172.21.15.97:0/3471755442 shutdown_connections 2023-07-06T03:53:12.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:12.809+0000 7f961b7ff700 1 -- 172.21.15.97:0/3471755442 wait complete. 2023-07-06T03:53:12.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T03:53:13.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.023+0000 7fb06b3ff700 1 Processor -- start 2023-07-06T03:53:13.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.023+0000 7fb06b3ff700 1 -- start start 2023-07-06T03:53:13.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.025+0000 7fb06b3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb06415c8b0 0x7fb0640b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:13.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.025+0000 7fb06b3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0640b4fe0 0x7fb0640d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:13.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.025+0000 7fb06b3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0640cdaf0 con 0x7fb0640b5550 2023-07-06T03:53:13.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.025+0000 7fb06b3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0640cff30 con 0x7fb06415c8b0 2023-07-06T03:53:13.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.025+0000 7fb06b3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640d0210 con 0x7fb0640b4fe0 2023-07-06T03:53:13.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.025+0000 7fb06899a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb0640b5550 0x7fb0640b48d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36856/0 (socket says 172.21.15.97:36856) 2023-07-06T03:53:13.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.025+0000 7fb06899a700 1 -- 172.21.15.97:0/2596385781 learned_addr learned my addr 172.21.15.97:0/2596385781 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:13.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.025+0000 7fb06919b700 1 --2- 172.21.15.97:0/2596385781 >> v2:172.21.15.146:3300/0 conn(0x7fb06415c8b0 0x7fb0640b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:13.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.025+0000 7fb06999c700 1 --2- 172.21.15.97:0/2596385781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0640b4fe0 0x7fb0640d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:13.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.025+0000 7fb05bfff700 1 -- 172.21.15.97:0/2596385781 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770450933 0 0) 0x7fb0640cdaf0 con 0x7fb0640b5550 2023-07-06T03:53:13.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.026+0000 7fb05bfff700 1 -- 172.21.15.97:0/2596385781 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0500035b0 con 0x7fb0640b5550 2023-07-06T03:53:13.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.026+0000 7fb05bfff700 1 -- 172.21.15.97:0/2596385781 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 966105931 0 0) 0x7fb0500035b0 con 0x7fb0640b5550 2023-07-06T03:53:13.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.026+0000 7fb06919b700 1 -- 172.21.15.97:0/2596385781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0640b4fe0 msgr2=0x7fb0640d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:13.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.026+0000 7fb06919b700 1 --2- 172.21.15.97:0/2596385781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0640b4fe0 0x7fb0640d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.026+0000 7fb06919b700 1 -- 172.21.15.97:0/2596385781 >> v1:172.21.15.97:6789/0 conn(0x7fb0640b5550 legacy=0x7fb0640b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.026+0000 7fb06919b700 1 -- 172.21.15.97:0/2596385781 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0640cd4a0 con 0x7fb06415c8b0 2023-07-06T03:53:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.026+0000 7fb06919b700 1 --2- 172.21.15.97:0/2596385781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb06415c8b0 0x7fb0640b2280 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb060009840 tx=0x7fb060009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf419b415b6c36ce server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.026+0000 7fb05bfff700 1 -- 172.21.15.97:0/2596385781 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054001670 con 0x7fb06415c8b0 2023-07-06T03:53:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.026+0000 7fb05bfff700 1 -- 172.21.15.97:0/2596385781 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb060004030 con 0x7fb06415c8b0 2023-07-06T03:53:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.026+0000 7fb05bfff700 1 -- 172.21.15.97:0/2596385781 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0600043a0 con 0x7fb06415c8b0 2023-07-06T03:53:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.027+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2596385781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb06415c8b0 msgr2=0x7fb0640b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.027+0000 7fb06b3ff700 1 --2- 172.21.15.97:0/2596385781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb06415c8b0 0x7fb0640b2280 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb060009840 tx=0x7fb060009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.027+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2596385781 shutdown_connections 2023-07-06T03:53:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.027+0000 7fb06b3ff700 1 --2- 172.21.15.97:0/2596385781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb06415c8b0 0x7fb0640b2280 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.027+0000 7fb06b3ff700 1 --2- 172.21.15.97:0/2596385781 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0640b4fe0 0x7fb0640d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.027+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2596385781 >> 172.21.15.97:0/2596385781 conn(0x7fb064066990 msgr2=0x7fb06406a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.027+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2596385781 shutdown_connections 2023-07-06T03:53:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.027+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2596385781 wait complete. 2023-07-06T03:53:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.029+0000 7fb06b3ff700 1 Processor -- start 2023-07-06T03:53:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.029+0000 7fb06b3ff700 1 -- start start 2023-07-06T03:53:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06b3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0640b4fe0 0x7fb0640c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06b3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0640b5550 0x7fb06414b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06b3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06415c8b0 0x7fb0640c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06b3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06414b820 con 0x7fb0640b4fe0 2023-07-06T03:53:13.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06b3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640c8740 con 0x7fb0640b5550 2023-07-06T03:53:13.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06b3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640c7160 con 0x7fb06415c8b0 2023-07-06T03:53:13.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06919b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0640b4fe0 0x7fb0640c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:13.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06919b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0640b4fe0 0x7fb0640c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54644/0 (socket says 172.21.15.97:54644) 2023-07-06T03:53:13.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06919b700 1 -- 172.21.15.97:0/2708037338 learned_addr learned my addr 172.21.15.97:0/2708037338 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:13.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06919b700 1 -- 172.21.15.97:0/2708037338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06415c8b0 msgr2=0x7fb0640c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:13.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06919b700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06415c8b0 0x7fb0640c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06919b700 1 -- 172.21.15.97:0/2708037338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0640b5550 msgr2=0x7fb06414b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06919b700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0640b5550 0x7fb06414b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.030+0000 7fb06919b700 1 -- 172.21.15.97:0/2708037338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0600094a0 con 0x7fb0640b4fe0 2023-07-06T03:53:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.031+0000 7fb06919b700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0640b4fe0 0x7fb0640c9460 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb0600090b0 tx=0x7fb060004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.031+0000 7fb059ffb700 1 -- 172.21.15.97:0/2708037338 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060003d30 con 0x7fb0640b4fe0 2023-07-06T03:53:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.031+0000 7fb059ffb700 1 -- 172.21.15.97:0/2708037338 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06001f070 con 0x7fb0640b4fe0 2023-07-06T03:53:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.031+0000 7fb059ffb700 1 -- 172.21.15.97:0/2708037338 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060011870 con 0x7fb0640b4fe0 2023-07-06T03:53:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.031+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2708037338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0640c7440 con 0x7fb0640b4fe0 2023-07-06T03:53:13.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.031+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2708037338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0640c7a00 con 0x7fb0640b4fe0 2023-07-06T03:53:13.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.032+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2708037338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0640cde60 con 0x7fb0640b4fe0 2023-07-06T03:53:13.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.033+0000 7fb059ffb700 1 -- 172.21.15.97:0/2708037338 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb060021030 con 0x7fb0640b4fe0 2023-07-06T03:53:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.034+0000 7fb059ffb700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb04806ace0 0x7fb04806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.034+0000 7fb059ffb700 1 -- 172.21.15.97:0/2708037338 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fb06008a7e0 con 0x7fb0640b4fe0 2023-07-06T03:53:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.035+0000 7fb06899a700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb04806ace0 0x7fb04806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.035+0000 7fb06899a700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb04806ace0 0x7fb04806d190 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb054001eb0 tx=0x7fb054001a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.039+0000 7fb059ffb700 1 -- 172.21.15.97:0/2708037338 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb060055850 con 0x7fb0640b4fe0 2023-07-06T03:53:13.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.280+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2708037338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb064074350 con 0x7fb0640b4fe0 2023-07-06T03:53:13.283 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-07-06T03:53:13.281+0000 7fb463ff4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T03:53:13.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.286+0000 7fb059ffb700 1 -- 172.21.15.97:0/2708037338 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb060058e70 con 0x7fb0640b4fe0 2023-07-06T03:53:13.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.288+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2708037338 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb04806ace0 msgr2=0x7fb04806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:13.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.288+0000 7fb06b3ff700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb04806ace0 0x7fb04806d190 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb054001eb0 tx=0x7fb054001a20 comp rx=0 tx=0).stop 2023-07-06T03:53:13.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.288+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2708037338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0640b4fe0 msgr2=0x7fb0640c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:13.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.288+0000 7fb06b3ff700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0640b4fe0 0x7fb0640c9460 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb0600090b0 tx=0x7fb060004750 comp rx=0 tx=0).stop 2023-07-06T03:53:13.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.289+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2708037338 shutdown_connections 2023-07-06T03:53:13.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.289+0000 7fb06b3ff700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb04806ace0 0x7fb04806d190 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.289+0000 7fb06b3ff700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06415c8b0 0x7fb0640c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.289+0000 7fb06b3ff700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0640b4fe0 0x7fb0640c9460 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.289+0000 7fb06b3ff700 1 --2- 172.21.15.97:0/2708037338 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0640b5550 0x7fb06414b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.289+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2708037338 >> 172.21.15.97:0/2708037338 conn(0x7fb064066990 msgr2=0x7fb064158210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:13.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.289+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2708037338 shutdown_connections 2023-07-06T03:53:13.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.289+0000 7fb06b3ff700 1 -- 172.21.15.97:0/2708037338 wait complete. 2023-07-06T03:53:13.293 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T03:53:13.311 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T03:53:13.364 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:53:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.511+0000 7f5b06bff700 1 Processor -- start 2023-07-06T03:53:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.511+0000 7f5b06bff700 1 -- start start 2023-07-06T03:53:13.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.512+0000 7f5b06bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b000b1fc0 0x7f5b000cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:13.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.512+0000 7f5b06bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0015d610 con 0x7f5b000b2390 2023-07-06T03:53:13.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.512+0000 7f5b06bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0015db40 con 0x7f5b000cf080 2023-07-06T03:53:13.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.513+0000 7f5b06bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b000b5770 con 0x7f5b000b1fc0 2023-07-06T03:53:13.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.513+0000 7f5afffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5b000b2390 0x7f5b000b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36872/0 (socket says 172.21.15.97:36872) 2023-07-06T03:53:13.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.513+0000 7f5afffff700 1 -- 172.21.15.97:0/3557462000 learned_addr learned my addr 172.21.15.97:0/3557462000 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.513+0000 7f5b0499b700 1 --2- 172.21.15.97:0/3557462000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b000b1fc0 0x7f5b000cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.513+0000 7f5aff7fe700 1 -- 172.21.15.97:0/3557462000 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415648352 0 0) 0x7f5b0015d610 con 0x7f5b000b2390 2023-07-06T03:53:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.513+0000 7f5aff7fe700 1 -- 172.21.15.97:0/3557462000 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af00035b0 con 0x7f5b000b2390 2023-07-06T03:53:13.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.513+0000 7f5b0499b700 1 -- 172.21.15.97:0/3557462000 >> v1:172.21.15.146:6789/0 conn(0x7f5b000cf080 legacy=0x7f5b000b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.513+0000 7f5aff7fe700 1 -- 172.21.15.97:0/3557462000 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972646011 0 0) 0x7f5b0015db40 con 0x7f5b000cf080 2023-07-06T03:53:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.513+0000 7f5b0499b700 1 -- 172.21.15.97:0/3557462000 >> v1:172.21.15.97:6789/0 conn(0x7f5b000b2390 legacy=0x7f5b000b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.513+0000 7f5b0499b700 1 -- 172.21.15.97:0/3557462000 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0015d860 con 0x7f5b000b1fc0 2023-07-06T03:53:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.514+0000 7f5b0499b700 1 --2- 172.21.15.97:0/3557462000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b000b1fc0 0x7f5b000cf8b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009870 tx=0x7f5af4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f14d583254a713c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.514+0000 7f5aff7fe700 1 -- 172.21.15.97:0/3557462000 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4011830 con 0x7f5b000b1fc0 2023-07-06T03:53:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.514+0000 7f5aff7fe700 1 -- 172.21.15.97:0/3557462000 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af4011990 con 0x7f5b000b1fc0 2023-07-06T03:53:13.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.514+0000 7f5aff7fe700 1 -- 172.21.15.97:0/3557462000 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4011d00 con 0x7f5b000b1fc0 2023-07-06T03:53:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.514+0000 7f5b06bff700 1 -- 172.21.15.97:0/3557462000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b000b1fc0 msgr2=0x7f5b000cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.514+0000 7f5b06bff700 1 --2- 172.21.15.97:0/3557462000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b000b1fc0 0x7f5b000cf8b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009870 tx=0x7f5af4009b80 comp rx=0 tx=0).stop 2023-07-06T03:53:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.515+0000 7f5b06bff700 1 -- 172.21.15.97:0/3557462000 shutdown_connections 2023-07-06T03:53:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.515+0000 7f5b06bff700 1 --2- 172.21.15.97:0/3557462000 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b000b1fc0 0x7f5b000cf8b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.515+0000 7f5b06bff700 1 -- 172.21.15.97:0/3557462000 >> 172.21.15.97:0/3557462000 conn(0x7f5b0006c210 msgr2=0x7f5b0006c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.515+0000 7f5b06bff700 1 -- 172.21.15.97:0/3557462000 shutdown_connections 2023-07-06T03:53:13.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.515+0000 7f5b06bff700 1 -- 172.21.15.97:0/3557462000 wait complete. 2023-07-06T03:53:13.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.516+0000 7f5b06bff700 1 Processor -- start 2023-07-06T03:53:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.516+0000 7f5b06bff700 1 -- start start 2023-07-06T03:53:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.517+0000 7f5b06bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b000b1fc0 0x7f5b001541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.517+0000 7f5b06bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b000b2390 0x7f5b000c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.517+0000 7f5b06bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b000cf080 0x7f5b00155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.517+0000 7f5b06bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b00155f20 con 0x7f5b000b1fc0 2023-07-06T03:53:13.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.517+0000 7f5b06bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b00156250 con 0x7f5b000cf080 2023-07-06T03:53:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.517+0000 7f5b06bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b00156530 con 0x7f5b000b2390 2023-07-06T03:53:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b0499b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b000b1fc0 0x7f5b001541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b0499b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b000b1fc0 0x7f5b001541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54684/0 (socket says 172.21.15.97:54684) 2023-07-06T03:53:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b0499b700 1 -- 172.21.15.97:0/2790525675 learned_addr learned my addr 172.21.15.97:0/2790525675 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b0519c700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b000cf080 0x7f5b00155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5afffff700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b000b2390 0x7f5b000c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b0499b700 1 -- 172.21.15.97:0/2790525675 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b000b2390 msgr2=0x7f5b000c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:13.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b0499b700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b000b2390 0x7f5b000c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b0499b700 1 -- 172.21.15.97:0/2790525675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b000cf080 msgr2=0x7f5b00155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b0499b700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b000cf080 0x7f5b00155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b0499b700 1 -- 172.21.15.97:0/2790525675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40094a0 con 0x7f5b000b1fc0 2023-07-06T03:53:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b0519c700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b000cf080 0x7f5b00155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:13.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b0499b700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b000b1fc0 0x7f5b001541c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f5af4014040 tx=0x7f5af4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5afd7fa700 1 -- 172.21.15.97:0/2790525675 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af40036a0 con 0x7f5b000b1fc0 2023-07-06T03:53:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5afd7fa700 1 -- 172.21.15.97:0/2790525675 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af401e070 con 0x7f5b000b1fc0 2023-07-06T03:53:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5afd7fa700 1 -- 172.21.15.97:0/2790525675 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4003a20 con 0x7f5b000b1fc0 2023-07-06T03:53:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.518+0000 7f5b06bff700 1 -- 172.21.15.97:0/2790525675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b00156810 con 0x7f5b000b1fc0 2023-07-06T03:53:13.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.519+0000 7f5b06bff700 1 -- 172.21.15.97:0/2790525675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b00154700 con 0x7f5b000b1fc0 2023-07-06T03:53:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.520+0000 7f5b06bff700 1 -- 172.21.15.97:0/2790525675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b00156060 con 0x7f5b000b1fc0 2023-07-06T03:53:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.520+0000 7f5afd7fa700 1 -- 172.21.15.97:0/2790525675 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5af4003b80 con 0x7f5b000b1fc0 2023-07-06T03:53:13.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.522+0000 7f5afd7fa700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ae406ace0 0x7f5ae406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:13.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.522+0000 7f5afd7fa700 1 -- 172.21.15.97:0/2790525675 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6351+0+0 (secure 0 0 0) 0x7f5af408a630 con 0x7f5b000b1fc0 2023-07-06T03:53:13.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.522+0000 7f5afffff700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ae406ace0 0x7f5ae406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:13.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.523+0000 7f5afffff700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ae406ace0 0x7f5ae406d190 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5aec005910 tx=0x7f5aec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:13.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.526+0000 7f5afd7fa700 1 -- 172.21.15.97:0/2790525675 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5af40555b0 con 0x7f5b000b1fc0 2023-07-06T03:53:13.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:13.761+0000 7f5b06bff700 1 -- 172.21.15.97:0/2790525675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5b00074350 con 0x7f5b000b1fc0 2023-07-06T03:53:14.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.287+0000 7f5afd7fa700 1 -- 172.21.15.97:0/2790525675 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5af4058bd0 con 0x7f5b000b1fc0 2023-07-06T03:53:14.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 -- 172.21.15.97:0/2790525675 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ae406ace0 msgr2=0x7f5ae406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:14.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ae406ace0 0x7f5ae406d190 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5aec005910 tx=0x7f5aec009040 comp rx=0 tx=0).stop 2023-07-06T03:53:14.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 -- 172.21.15.97:0/2790525675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b000b1fc0 msgr2=0x7f5b001541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:14.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b000b1fc0 0x7f5b001541c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f5af4014040 tx=0x7f5af4004750 comp rx=0 tx=0).stop 2023-07-06T03:53:14.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 -- 172.21.15.97:0/2790525675 shutdown_connections 2023-07-06T03:53:14.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5ae406ace0 0x7f5ae406d190 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:14.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b000cf080 0x7f5b00155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b000b1fc0 0x7f5b001541c0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 --2- 172.21.15.97:0/2790525675 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5b000b2390 0x7f5b000c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 -- 172.21.15.97:0/2790525675 >> 172.21.15.97:0/2790525675 conn(0x7f5b0006c210 msgr2=0x7f5b000b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 -- 172.21.15.97:0/2790525675 shutdown_connections 2023-07-06T03:53:14.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.290+0000 7f5b06bff700 1 -- 172.21.15.97:0/2790525675 wait complete. 2023-07-06T03:53:14.312 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T03:53:14.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.509+0000 7fcc095ff700 1 Processor -- start 2023-07-06T03:53:14.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.509+0000 7fcc095ff700 1 -- start start 2023-07-06T03:53:14.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.510+0000 7fcc095ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc040cffc0 0x7fcc040d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:14.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.510+0000 7fcc095ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc040cf080 0x7fcc040d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:14.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.510+0000 7fcc095ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0415d610 con 0x7fcc040b2050 2023-07-06T03:53:14.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.510+0000 7fcc095ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0415db40 con 0x7fcc040cffc0 2023-07-06T03:53:14.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.510+0000 7fcc095ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0415d860 con 0x7fcc040cf080 2023-07-06T03:53:14.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.510+0000 7fcc027fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc040cffc0 0x7fcc040d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:14.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.510+0000 7fcc02ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcc040b2050 0x7fcc040cf8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36888/0 (socket says 172.21.15.97:36888) 2023-07-06T03:53:14.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.510+0000 7fcc02ffd700 1 -- 172.21.15.97:0/2983490173 learned_addr learned my addr 172.21.15.97:0/2983490173 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:14.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.511+0000 7fcc037fe700 1 --2- 172.21.15.97:0/2983490173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc040cf080 0x7fcc040d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:14.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.511+0000 7fcc027fc700 1 -- 172.21.15.97:0/2983490173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc040cf080 msgr2=0x7fcc040d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:14.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.511+0000 7fcc027fc700 1 --2- 172.21.15.97:0/2983490173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc040cf080 0x7fcc040d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:14.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.511+0000 7fcc027fc700 1 -- 172.21.15.97:0/2983490173 >> v1:172.21.15.97:6789/0 conn(0x7fcc040b2050 legacy=0x7fcc040cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:14.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.511+0000 7fcc027fc700 1 -- 172.21.15.97:0/2983490173 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040cd630 con 0x7fcc040cffc0 2023-07-06T03:53:14.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.511+0000 7fcc037fe700 1 --2- 172.21.15.97:0/2983490173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc040cf080 0x7fcc040d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:14.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.511+0000 7fcc01ffb700 1 -- 172.21.15.97:0/2983490173 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857482531 0 0) 0x7fcc0415d610 con 0x7fcc040b2050 2023-07-06T03:53:14.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.511+0000 7fcc027fc700 1 --2- 172.21.15.97:0/2983490173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc040cffc0 0x7fcc040d0ba0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009840 tx=0x7fcbf8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eec9c39045a8da server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:14.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.511+0000 7fcc01ffb700 1 -- 172.21.15.97:0/2983490173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8011830 con 0x7fcc040cffc0 2023-07-06T03:53:14.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.511+0000 7fcc01ffb700 1 -- 172.21.15.97:0/2983490173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf8011990 con 0x7fcc040cffc0 2023-07-06T03:53:14.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.511+0000 7fcc01ffb700 1 -- 172.21.15.97:0/2983490173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8011d20 con 0x7fcc040cffc0 2023-07-06T03:53:14.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.512+0000 7fcc095ff700 1 -- 172.21.15.97:0/2983490173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc040cffc0 msgr2=0x7fcc040d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:14.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.512+0000 7fcc095ff700 1 --2- 172.21.15.97:0/2983490173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc040cffc0 0x7fcc040d0ba0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009840 tx=0x7fcbf8009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:14.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.512+0000 7fcc095ff700 1 -- 172.21.15.97:0/2983490173 shutdown_connections 2023-07-06T03:53:14.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.512+0000 7fcc095ff700 1 --2- 172.21.15.97:0/2983490173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc040cf080 0x7fcc040d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:14.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.512+0000 7fcc095ff700 1 --2- 172.21.15.97:0/2983490173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc040cffc0 0x7fcc040d0ba0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:14.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.512+0000 7fcc095ff700 1 -- 172.21.15.97:0/2983490173 >> 172.21.15.97:0/2983490173 conn(0x7fcc0406c210 msgr2=0x7fcc0406c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:14.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.512+0000 7fcc095ff700 1 -- 172.21.15.97:0/2983490173 shutdown_connections 2023-07-06T03:53:14.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.512+0000 7fcc095ff700 1 -- 172.21.15.97:0/2983490173 wait complete. 2023-07-06T03:53:14.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.514+0000 7fcc095ff700 1 Processor -- start 2023-07-06T03:53:14.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.515+0000 7fcc095ff700 1 -- start start 2023-07-06T03:53:14.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc095ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc040b2050 0x7fcc04158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:14.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc095ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc040cf080 0x7fcc040cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:14.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc095ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc040cffc0 0x7fcc040cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:14.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc095ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040b2960 con 0x7fcc040b2050 2023-07-06T03:53:14.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc095ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040b2c40 con 0x7fcc040cf080 2023-07-06T03:53:14.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc095ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040b2f20 con 0x7fcc040cffc0 2023-07-06T03:53:14.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc02ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc040b2050 0x7fcc04158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:14.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc02ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc040b2050 0x7fcc04158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54706/0 (socket says 172.21.15.97:54706) 2023-07-06T03:53:14.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc02ffd700 1 -- 172.21.15.97:0/217541369 learned_addr learned my addr 172.21.15.97:0/217541369 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:14.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc02ffd700 1 -- 172.21.15.97:0/217541369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc040cffc0 msgr2=0x7fcc040cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:14.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc02ffd700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc040cffc0 0x7fcc040cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:14.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc02ffd700 1 -- 172.21.15.97:0/217541369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc040cf080 msgr2=0x7fcc040cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:14.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc02ffd700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc040cf080 0x7fcc040cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:14.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.516+0000 7fcc02ffd700 1 -- 172.21.15.97:0/217541369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf80094a0 con 0x7fcc040b2050 2023-07-06T03:53:14.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.517+0000 7fcc02ffd700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc040b2050 0x7fcc04158630 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c550 tx=0x7fcbf400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:14.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.517+0000 7fcbeb7fe700 1 -- 172.21.15.97:0/217541369 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4004c80 con 0x7fcc040b2050 2023-07-06T03:53:14.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.517+0000 7fcbeb7fe700 1 -- 172.21.15.97:0/217541369 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf4004e00 con 0x7fcc040b2050 2023-07-06T03:53:14.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.517+0000 7fcc095ff700 1 -- 172.21.15.97:0/217541369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc040b3200 con 0x7fcc040b2050 2023-07-06T03:53:14.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.517+0000 7fcbeb7fe700 1 -- 172.21.15.97:0/217541369 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4007310 con 0x7fcc040b2050 2023-07-06T03:53:14.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.517+0000 7fcc095ff700 1 -- 172.21.15.97:0/217541369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc04158b70 con 0x7fcc040b2050 2023-07-06T03:53:14.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.518+0000 7fcc095ff700 1 -- 172.21.15.97:0/217541369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc04074350 con 0x7fcc040b2050 2023-07-06T03:53:14.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.519+0000 7fcbeb7fe700 1 -- 172.21.15.97:0/217541369 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcbf4007470 con 0x7fcc040b2050 2023-07-06T03:53:14.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.520+0000 7fcbeb7fe700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbf006ace0 0x7fcbf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:14.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.520+0000 7fcbeb7fe700 1 -- 172.21.15.97:0/217541369 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6351+0+0 (secure 0 0 0) 0x7fcbf40892c0 con 0x7fcc040b2050 2023-07-06T03:53:14.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.520+0000 7fcc027fc700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbf006ace0 0x7fcbf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:14.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.521+0000 7fcc027fc700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbf006ace0 0x7fcbf006d190 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcc040d03e0 tx=0x7fcbf8010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:14.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.526+0000 7fcbeb7fe700 1 -- 172.21.15.97:0/217541369 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcbf4054240 con 0x7fcc040b2050 2023-07-06T03:53:14.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:14.722+0000 7fcc095ff700 1 -- 172.21.15.97:0/217541369 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcc040b3720 con 0x7fcc040b2050 2023-07-06T03:53:15.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.289+0000 7fcbeb7fe700 1 -- 172.21.15.97:0/217541369 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcc040b3720 con 0x7fcc040b2050 2023-07-06T03:53:15.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 -- 172.21.15.97:0/217541369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbf006ace0 msgr2=0x7fcbf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:15.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbf006ace0 0x7fcbf006d190 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcc040d03e0 tx=0x7fcbf8010000 comp rx=0 tx=0).stop 2023-07-06T03:53:15.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 -- 172.21.15.97:0/217541369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc040b2050 msgr2=0x7fcc04158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:15.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc040b2050 0x7fcc04158630 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c550 tx=0x7fcbf400c910 comp rx=0 tx=0).stop 2023-07-06T03:53:15.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 -- 172.21.15.97:0/217541369 shutdown_connections 2023-07-06T03:53:15.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbf006ace0 0x7fcbf006d190 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:15.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc040cffc0 0x7fcc040cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:15.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc040b2050 0x7fcc04158630 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:15.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 --2- 172.21.15.97:0/217541369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc040cf080 0x7fcc040cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:15.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 -- 172.21.15.97:0/217541369 >> 172.21.15.97:0/217541369 conn(0x7fcc0406c210 msgr2=0x7fcc040ca110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:15.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 -- 172.21.15.97:0/217541369 shutdown_connections 2023-07-06T03:53:15.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.293+0000 7fcc095ff700 1 -- 172.21.15.97:0/217541369 wait complete. 2023-07-06T03:53:15.298 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-07-06T03:53:15.315 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T03:53:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.513+0000 7f5cfc5ff700 1 Processor -- start 2023-07-06T03:53:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.513+0000 7f5cfc5ff700 1 -- start start 2023-07-06T03:53:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.513+0000 7f5cfc5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf40b2050 0x7f5cf40cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.513+0000 7f5cfc5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5cf40cf080 0x7f5cf40cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.513+0000 7f5cfc5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf415d610 con 0x7f5cf40d29b0 2023-07-06T03:53:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.513+0000 7f5cfc5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf415db40 con 0x7f5cf40cf080 2023-07-06T03:53:15.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.513+0000 7f5cfc5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf415d860 con 0x7f5cf40b2050 2023-07-06T03:53:15.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cf9b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5cf40cf080 0x7f5cf40cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:15.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cf9b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5cf40cf080 0x7f5cf40cfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54718/0 (socket says 172.21.15.97:54718) 2023-07-06T03:53:15.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cf9b9a700 1 -- 172.21.15.97:0/3853231373 learned_addr learned my addr 172.21.15.97:0/3853231373 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:15.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cfa39b700 1 --2- 172.21.15.97:0/3853231373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf40b2050 0x7f5cf40cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:15.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cf9b9a700 1 -- 172.21.15.97:0/3853231373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf40b2050 msgr2=0x7f5cf40cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:15.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cf9b9a700 1 --2- 172.21.15.97:0/3853231373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf40b2050 0x7f5cf40cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cf9b9a700 1 -- 172.21.15.97:0/3853231373 >> v1:172.21.15.146:6789/0 conn(0x7f5cf40d29b0 legacy=0x7f5cf40b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cf9b9a700 1 -- 172.21.15.97:0/3853231373 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40cd630 con 0x7f5cf40cf080 2023-07-06T03:53:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cf9b9a700 1 --2- 172.21.15.97:0/3853231373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf40cf080 0x7f5cf40cfdf0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40098f0 tx=0x7f5ce4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e8c2896ed8594a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cf9399700 1 -- 172.21.15.97:0/3853231373 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4004230 con 0x7f5cf40cf080 2023-07-06T03:53:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cf9399700 1 -- 172.21.15.97:0/3853231373 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce4004390 con 0x7f5cf40cf080 2023-07-06T03:53:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.514+0000 7f5cf9399700 1 -- 172.21.15.97:0/3853231373 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40036a0 con 0x7f5cf40cf080 2023-07-06T03:53:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.515+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3853231373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf40cf080 msgr2=0x7f5cf40cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.515+0000 7f5cfc5ff700 1 --2- 172.21.15.97:0/3853231373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf40cf080 0x7f5cf40cfdf0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40098f0 tx=0x7f5ce4009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.515+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3853231373 shutdown_connections 2023-07-06T03:53:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.515+0000 7f5cfc5ff700 1 --2- 172.21.15.97:0/3853231373 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf40b2050 0x7f5cf40cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.515+0000 7f5cfc5ff700 1 --2- 172.21.15.97:0/3853231373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf40cf080 0x7f5cf40cfdf0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.515+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3853231373 >> 172.21.15.97:0/3853231373 conn(0x7f5cf406c210 msgr2=0x7f5cf406c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.515+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3853231373 shutdown_connections 2023-07-06T03:53:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.515+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3853231373 wait complete. 2023-07-06T03:53:15.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.517+0000 7f5cfc5ff700 1 Processor -- start 2023-07-06T03:53:15.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.517+0000 7f5cfc5ff700 1 -- start start 2023-07-06T03:53:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cfc5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf40b2050 0x7f5cf40cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cfc5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf40cf080 0x7f5cf40c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cfc5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf40d29b0 0x7f5cf40c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cfc5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf41558c0 con 0x7f5cf40cf080 2023-07-06T03:53:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cfc5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40c9490 con 0x7f5cf40d29b0 2023-07-06T03:53:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cfc5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40b2830 con 0x7f5cf40b2050 2023-07-06T03:53:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cf9b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf40cf080 0x7f5cf40c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cf9b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf40cf080 0x7f5cf40c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54730/0 (socket says 172.21.15.97:54730) 2023-07-06T03:53:15.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cf9b9a700 1 -- 172.21.15.97:0/3813966849 learned_addr learned my addr 172.21.15.97:0/3813966849 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:15.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cfab9c700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf40d29b0 0x7f5cf40c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:15.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cfa39b700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf40b2050 0x7f5cf40cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:15.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cf9b9a700 1 -- 172.21.15.97:0/3813966849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf40b2050 msgr2=0x7f5cf40cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:15.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cf9b9a700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf40b2050 0x7f5cf40cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cf9b9a700 1 -- 172.21.15.97:0/3813966849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf40d29b0 msgr2=0x7f5cf40c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cf9b9a700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf40d29b0 0x7f5cf40c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.518+0000 7f5cf9b9a700 1 -- 172.21.15.97:0/3813966849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce4009550 con 0x7f5cf40cf080 2023-07-06T03:53:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.519+0000 7f5cf9b9a700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf40cf080 0x7f5cf40c8a10 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009130 tx=0x7f5ce4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.519+0000 7f5ceaffd700 1 -- 172.21.15.97:0/3813966849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4003c00 con 0x7f5cf40cf080 2023-07-06T03:53:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.519+0000 7f5ceaffd700 1 -- 172.21.15.97:0/3813966849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce401f070 con 0x7f5cf40cf080 2023-07-06T03:53:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.519+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3813966849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf40b30d0 con 0x7f5cf40cf080 2023-07-06T03:53:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.519+0000 7f5ceaffd700 1 -- 172.21.15.97:0/3813966849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce400b660 con 0x7f5cf40cf080 2023-07-06T03:53:15.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.519+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3813966849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf40b2d60 con 0x7f5cf40cf080 2023-07-06T03:53:15.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.520+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3813966849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf40b3260 con 0x7f5cf40cf080 2023-07-06T03:53:15.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.521+0000 7f5ceaffd700 1 -- 172.21.15.97:0/3813966849 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5ce4021030 con 0x7f5cf40cf080 2023-07-06T03:53:15.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.522+0000 7f5ceaffd700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cd806ace0 0x7f5cd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:15.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.522+0000 7f5ceaffd700 1 -- 172.21.15.97:0/3813966849 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5884+0+0 (secure 0 0 0) 0x7f5ce408a9a0 con 0x7f5cf40cf080 2023-07-06T03:53:15.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.522+0000 7f5cfa39b700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cd806ace0 0x7f5cd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:15.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.523+0000 7f5cfa39b700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cd806ace0 0x7f5cd806d190 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40ccf40 tx=0x7f5cf0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:15.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.526+0000 7f5ceaffd700 1 -- 172.21.15.97:0/3813966849 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5ce4055af0 con 0x7f5cf40cf080 2023-07-06T03:53:15.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:15.722+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3813966849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5cf40b3260 con 0x7f5cf40cf080 2023-07-06T03:53:16.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.295+0000 7f5ceaffd700 1 -- 172.21.15.97:0/3813966849 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5ce4059110 con 0x7f5cf40cf080 2023-07-06T03:53:16.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.301+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3813966849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cd806ace0 msgr2=0x7f5cd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:16.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.301+0000 7f5cfc5ff700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cd806ace0 0x7f5cd806d190 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40ccf40 tx=0x7f5cf0008040 comp rx=0 tx=0).stop 2023-07-06T03:53:16.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.301+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3813966849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf40cf080 msgr2=0x7f5cf40c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:16.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.301+0000 7f5cfc5ff700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf40cf080 0x7f5cf40c8a10 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009130 tx=0x7f5ce4004750 comp rx=0 tx=0).stop 2023-07-06T03:53:16.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.301+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3813966849 shutdown_connections 2023-07-06T03:53:16.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.301+0000 7f5cfc5ff700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cd806ace0 0x7f5cd806d190 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.301+0000 7f5cfc5ff700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf40d29b0 0x7f5cf40c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.301+0000 7f5cfc5ff700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf40b2050 0x7f5cf40cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.301+0000 7f5cfc5ff700 1 --2- 172.21.15.97:0/3813966849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf40cf080 0x7f5cf40c8a10 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.302+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3813966849 >> 172.21.15.97:0/3813966849 conn(0x7f5cf406c210 msgr2=0x7f5cf40b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:16.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.304+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3813966849 shutdown_connections 2023-07-06T03:53:16.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.304+0000 7f5cfc5ff700 1 -- 172.21.15.97:0/3813966849 wait complete. 2023-07-06T03:53:16.310 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-07-06T03:53:16.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:53:16.334 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:53:16.446 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2023-07-06T03:53:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2023-07-06T03:53:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:53:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2023-07-06T03:53:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2023-07-06T03:53:16.447 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2023-07-06T03:53:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2023-07-06T03:53:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2023-07-06T03:53:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2023-07-06T03:53:16.448 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2023-07-06T03:53:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:53:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2023-07-06T03:53:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:53:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:53:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.449 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2023-07-06T03:53:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2023-07-06T03:53:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2023-07-06T03:53:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:53:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:53:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2023-07-06T03:53:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:53:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2023-07-06T03:53:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:53:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:53:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2023-07-06T03:53:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:53:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2023-07-06T03:53:16.451 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2023-07-06T03:53:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2023-07-06T03:53:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2023-07-06T03:53:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2023-07-06T03:53:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:53:16.452 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:53:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.453 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2023-07-06T03:53:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2023-07-06T03:53:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2023-07-06T03:53:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2023-07-06T03:53:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:53:16.453 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2023-07-06T03:53:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2023-07-06T03:53:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2023-07-06T03:53:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2023-07-06T03:53:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2023-07-06T03:53:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.454 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2023-07-06T03:53:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2023-07-06T03:53:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:53:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2023-07-06T03:53:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2023-07-06T03:53:16.455 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:53:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2023-07-06T03:53:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2023-07-06T03:53:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2023-07-06T03:53:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2023-07-06T03:53:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2023-07-06T03:53:16.456 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2023-07-06T03:53:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2023-07-06T03:53:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2023-07-06T03:53:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2023-07-06T03:53:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2023-07-06T03:53:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:53:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2023-07-06T03:53:16.457 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2023-07-06T03:53:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:53:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2023-07-06T03:53:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:53:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:53:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2023-07-06T03:53:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:53:16.458 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.458 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2023-07-06T03:53:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:53:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2023-07-06T03:53:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:53:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:53:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:53:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:53:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2023-07-06T03:53:16.459 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.460 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2023-07-06T03:53:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:53:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:53:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2023-07-06T03:53:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:53:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2023-07-06T03:53:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2023-07-06T03:53:16.460 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:53:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2023-07-06T03:53:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2023-07-06T03:53:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:53:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:53:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:53:16.461 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:53:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:53:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:53:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:53:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2023-07-06T03:53:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:53:16.462 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:53:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2023-07-06T03:53:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:16.463 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2023-07-06T03:53:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2023-07-06T03:53:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2023-07-06T03:53:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2023-07-06T03:53:16.463 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:53:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:53:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2023-07-06T03:53:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2023-07-06T03:53:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:53:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2023-07-06T03:53:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:53:16.464 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:53:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2023-07-06T03:53:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2023-07-06T03:53:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2023-07-06T03:53:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-07-06T03:53:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-07-06T03:53:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-07-06T03:53:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-07-06T03:53:16.465 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2023-07-06T03:53:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2023-07-06T03:53:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2023-07-06T03:53:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:53:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2023-07-06T03:53:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2023-07-06T03:53:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2023-07-06T03:53:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2023-07-06T03:53:16.466 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:53:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:53:16.467 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:53:16.467 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2023-07-06T03:53:16.467 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2023-07-06T03:53:16.468 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:53:16.468 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2023-07-06T03:53:16.468 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.468 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2023-07-06T03:53:16.468 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.468 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2023-07-06T03:53:16.469 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.469 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2023-07-06T03:53:16.469 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2023-07-06T03:53:16.469 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.469 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2023-07-06T03:53:16.469 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2023-07-06T03:53:16.469 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.469 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:53:16.470 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2023-07-06T03:53:16.470 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.470 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:53:16.470 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:53:16.470 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.470 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2023-07-06T03:53:16.470 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.471 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2023-07-06T03:53:16.471 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2023-07-06T03:53:16.471 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:53:16.471 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:53:16.471 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2023-07-06T03:53:16.471 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:53:16.471 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2023-07-06T03:53:16.472 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:53:16.472 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:53:16.472 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2023-07-06T03:53:16.472 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.472 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:53:16.472 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.473 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2023-07-06T03:53:16.473 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.473 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2023-07-06T03:53:16.473 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.473 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2023-07-06T03:53:16.473 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2023-07-06T03:53:16.473 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2023-07-06T03:53:16.474 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:53:16.474 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:53:16.474 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.474 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2023-07-06T03:53:16.474 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2023-07-06T03:53:16.474 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.474 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2023-07-06T03:53:16.475 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2023-07-06T03:53:16.475 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:53:16.475 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.475 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2023-07-06T03:53:16.475 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2023-07-06T03:53:16.475 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.475 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2023-07-06T03:53:16.475 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2023-07-06T03:53:16.476 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2023-07-06T03:53:16.476 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.476 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2023-07-06T03:53:16.476 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2023-07-06T03:53:16.476 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.476 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:53:16.476 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.476 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2023-07-06T03:53:16.477 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.477 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2023-07-06T03:53:16.477 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.477 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:53:16.477 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2023-07-06T03:53:16.477 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.477 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2023-07-06T03:53:16.477 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2023-07-06T03:53:16.478 INFO:teuthology.orchestra.run.smithi146.stdout: bdev async discard = True 2023-07-06T03:53:16.478 INFO:teuthology.orchestra.run.smithi146.stdout: bdev enable discard = True 2023-07-06T03:53:16.478 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore allocator = bitmap 2023-07-06T03:53:16.478 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2023-07-06T03:53:16.478 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2023-07-06T03:53:16.478 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 1/20 2023-07-06T03:53:16.478 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 1/20 2023-07-06T03:53:16.478 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:53:16.479 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2023-07-06T03:53:16.479 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 4/10 2023-07-06T03:53:16.479 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:53:16.479 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2023-07-06T03:53:16.479 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:53:16.479 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:53:16.479 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2023-07-06T03:53:16.479 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:53:16.480 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.480 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2023-07-06T03:53:16.480 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:53:16.480 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2023-07-06T03:53:16.480 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:53:16.480 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:53:16.480 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:53:16.481 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:53:16.481 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2023-07-06T03:53:16.481 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.481 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2023-07-06T03:53:16.481 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:53:16.481 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:53:16.481 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2023-07-06T03:53:16.482 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:53:16.482 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2023-07-06T03:53:16.482 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2023-07-06T03:53:16.482 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:53:16.482 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2023-07-06T03:53:16.482 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2023-07-06T03:53:16.482 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:53:16.482 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.483 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:53:16.483 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:53:16.483 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:53:16.483 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.483 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:53:16.483 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:53:16.483 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:53:16.483 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.484 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2023-07-06T03:53:16.484 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:53:16.484 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.484 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:53:16.484 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2023-07-06T03:53:16.484 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:53:16.484 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2023-07-06T03:53:16.485 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2023-07-06T03:53:16.485 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2023-07-06T03:53:16.485 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2023-07-06T03:53:16.485 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:53:16.485 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:53:16.485 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2023-07-06T03:53:16.485 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2023-07-06T03:53:16.485 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:53:16.486 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2023-07-06T03:53:16.486 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:53:16.486 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:53:16.486 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2023-07-06T03:53:16.486 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2023-07-06T03:53:16.486 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2023-07-06T03:53:16.486 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2023-07-06T03:53:16.486 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2023-07-06T03:53:16.487 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2023-07-06T03:53:16.487 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2023-07-06T03:53:16.487 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2023-07-06T03:53:16.487 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2023-07-06T03:53:16.487 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2023-07-06T03:53:16.487 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:53:16.487 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2023-07-06T03:53:16.487 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2023-07-06T03:53:16.488 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2023-07-06T03:53:16.488 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2023-07-06T03:53:16.488 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:53:16.488 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:53:16.488 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:53:16.489 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T03:53:16.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.684+0000 7f60efbff700 1 Processor -- start 2023-07-06T03:53:16.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.684+0000 7f60efbff700 1 -- start start 2023-07-06T03:53:16.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.685+0000 7f60efbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60e80d2210 0x7f60e80b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:16.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.685+0000 7f60efbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e80cff90 con 0x7f60e815c610 2023-07-06T03:53:16.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.685+0000 7f60efbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e80b5650 con 0x7f60e806b360 2023-07-06T03:53:16.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.685+0000 7f60efbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e80d0780 con 0x7f60e80d2210 2023-07-06T03:53:16.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.685+0000 7f60ee19c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60e80d2210 0x7f60e80b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:16.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.685+0000 7f60ee19c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60e80d2210 0x7f60e80b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54732/0 (socket says 172.21.15.97:54732) 2023-07-06T03:53:16.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.685+0000 7f60ed99b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f60e815c610 0x7f60e80b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36896/0 (socket says 172.21.15.97:36896) 2023-07-06T03:53:16.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.685+0000 7f60ee19c700 1 -- 172.21.15.97:0/2506752135 learned_addr learned my addr 172.21.15.97:0/2506752135 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:16.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.686+0000 7f60ee19c700 1 -- 172.21.15.97:0/2506752135 >> v1:172.21.15.146:6789/0 conn(0x7f60e806b360 legacy=0x7f60e80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:16.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.686+0000 7f60ee19c700 1 -- 172.21.15.97:0/2506752135 >> v1:172.21.15.97:6789/0 conn(0x7f60e815c610 legacy=0x7f60e80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:16.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.686+0000 7f60ee19c700 1 -- 172.21.15.97:0/2506752135 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e80b5400 con 0x7f60e80d2210 2023-07-06T03:53:16.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.686+0000 7f60ee19c700 1 --2- 172.21.15.97:0/2506752135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60e80d2210 0x7f60e80b4ec0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009e50 tx=0x7f60e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c66bfd2c9e0204 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:16.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.686+0000 7f60ec999700 1 -- 172.21.15.97:0/2506752135 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e000e070 con 0x7f60e80d2210 2023-07-06T03:53:16.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.686+0000 7f60ec999700 1 -- 172.21.15.97:0/2506752135 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e0003a20 con 0x7f60e80d2210 2023-07-06T03:53:16.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.686+0000 7f60ec999700 1 -- 172.21.15.97:0/2506752135 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0003dd0 con 0x7f60e80d2210 2023-07-06T03:53:16.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.687+0000 7f60efbff700 1 -- 172.21.15.97:0/2506752135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60e80d2210 msgr2=0x7f60e80b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:16.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.687+0000 7f60efbff700 1 --2- 172.21.15.97:0/2506752135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60e80d2210 0x7f60e80b4ec0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009e50 tx=0x7f60e000b330 comp rx=0 tx=0).stop 2023-07-06T03:53:16.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.687+0000 7f60efbff700 1 -- 172.21.15.97:0/2506752135 shutdown_connections 2023-07-06T03:53:16.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.687+0000 7f60efbff700 1 --2- 172.21.15.97:0/2506752135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60e80d2210 0x7f60e80b4ec0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.687+0000 7f60efbff700 1 -- 172.21.15.97:0/2506752135 >> 172.21.15.97:0/2506752135 conn(0x7f60e806a6c0 msgr2=0x7f60e806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:16.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.687+0000 7f60efbff700 1 -- 172.21.15.97:0/2506752135 shutdown_connections 2023-07-06T03:53:16.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.687+0000 7f60efbff700 1 -- 172.21.15.97:0/2506752135 wait complete. 2023-07-06T03:53:16.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.689+0000 7f60efbff700 1 Processor -- start 2023-07-06T03:53:16.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.689+0000 7f60efbff700 1 -- start start 2023-07-06T03:53:16.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60efbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f60e806b360 0x7f60e814c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:16.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60efbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60e80d2210 0x7f60e81f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:16.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60efbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60e815c610 0x7f60e80c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:16.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60efbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e81f2ac0 con 0x7f60e815c610 2023-07-06T03:53:16.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60efbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e81592b0 con 0x7f60e80d2210 2023-07-06T03:53:16.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60efbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e815d6d0 con 0x7f60e806b360 2023-07-06T03:53:16.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60ee19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60e815c610 0x7f60e80c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:16.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60ee19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60e815c610 0x7f60e80c6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54746/0 (socket says 172.21.15.97:54746) 2023-07-06T03:53:16.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60ee19c700 1 -- 172.21.15.97:0/2830606539 learned_addr learned my addr 172.21.15.97:0/2830606539 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60ed99b700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f60e806b360 0x7f60e814c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60ed19a700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60e80d2210 0x7f60e81f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60ee19c700 1 -- 172.21.15.97:0/2830606539 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f60e806b360 msgr2=0x7f60e814c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60ee19c700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f60e806b360 0x7f60e814c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60ee19c700 1 -- 172.21.15.97:0/2830606539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60e80d2210 msgr2=0x7f60e81f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60ee19c700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60e80d2210 0x7f60e81f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60ee19c700 1 -- 172.21.15.97:0/2830606539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e0009b30 con 0x7f60e815c610 2023-07-06T03:53:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.691+0000 7f60ed19a700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60e80d2210 0x7f60e81f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.692+0000 7f60ee19c700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60e815c610 0x7f60e80c6e50 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f60e808b120 tx=0x7f60e0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.692+0000 7f60de7fc700 1 -- 172.21.15.97:0/2830606539 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0004440 con 0x7f60e815c610 2023-07-06T03:53:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.692+0000 7f60de7fc700 1 -- 172.21.15.97:0/2830606539 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e00045a0 con 0x7f60e815c610 2023-07-06T03:53:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.692+0000 7f60efbff700 1 -- 172.21.15.97:0/2830606539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e815d9b0 con 0x7f60e815c610 2023-07-06T03:53:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.692+0000 7f60de7fc700 1 -- 172.21.15.97:0/2830606539 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0004950 con 0x7f60e815c610 2023-07-06T03:53:16.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.692+0000 7f60efbff700 1 -- 172.21.15.97:0/2830606539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e815df70 con 0x7f60e815c610 2023-07-06T03:53:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.693+0000 7f60efbff700 1 -- 172.21.15.97:0/2830606539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e80ce040 con 0x7f60e815c610 2023-07-06T03:53:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.694+0000 7f60de7fc700 1 -- 172.21.15.97:0/2830606539 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f60e0012430 con 0x7f60e815c610 2023-07-06T03:53:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.695+0000 7f60de7fc700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f60cc06acb0 0x7f60cc06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.696+0000 7f60de7fc700 1 -- 172.21.15.97:0/2830606539 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f60e008a7b0 con 0x7f60e815c610 2023-07-06T03:53:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.696+0000 7f60ed99b700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f60cc06acb0 0x7f60cc06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.696+0000 7f60ed99b700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f60cc06acb0 0x7f60cc06d160 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f60e4000f60 tx=0x7f60e4007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:16.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.700+0000 7f60de7fc700 1 -- 172.21.15.97:0/2830606539 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f60e0055af0 con 0x7f60e815c610 2023-07-06T03:53:16.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.901+0000 7f60efbff700 1 -- 172.21.15.97:0/2830606539 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f60e804e340 con 0x7f60e815c610 2023-07-06T03:53:16.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.905+0000 7f60de7fc700 1 -- 172.21.15.97:0/2830606539 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f60e0059110 con 0x7f60e815c610 2023-07-06T03:53:16.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 -- 172.21.15.97:0/2830606539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f60cc06acb0 msgr2=0x7f60cc06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:16.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f60cc06acb0 0x7f60cc06d160 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f60e4000f60 tx=0x7f60e4007d50 comp rx=0 tx=0).stop 2023-07-06T03:53:16.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 -- 172.21.15.97:0/2830606539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60e815c610 msgr2=0x7f60e80c6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:16.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60e815c610 0x7f60e80c6e50 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f60e808b120 tx=0x7f60e0004ad0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 -- 172.21.15.97:0/2830606539 shutdown_connections 2023-07-06T03:53:16.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f60cc06acb0 0x7f60cc06d160 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60e815c610 0x7f60e80c6e50 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f60e806b360 0x7f60e814c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 --2- 172.21.15.97:0/2830606539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f60e80d2210 0x7f60e81f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 -- 172.21.15.97:0/2830606539 >> 172.21.15.97:0/2830606539 conn(0x7f60e806a6c0 msgr2=0x7f60e80b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 -- 172.21.15.97:0/2830606539 shutdown_connections 2023-07-06T03:53:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:16.908+0000 7f60efbff700 1 -- 172.21.15.97:0/2830606539 wait complete. 2023-07-06T03:53:16.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2023-07-06T03:53:17.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.125+0000 7feace5ff700 1 Processor -- start 2023-07-06T03:53:17.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.125+0000 7feace5ff700 1 -- start start 2023-07-06T03:53:17.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.127+0000 7feace5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7feac815d990 0x7feac815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:17.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.127+0000 7feace5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac80c97c0 con 0x7feac806c7a0 2023-07-06T03:53:17.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.127+0000 7feace5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac815ceb0 con 0x7feac8159390 2023-07-06T03:53:17.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.127+0000 7feace5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac8159760 con 0x7feac815d990 2023-07-06T03:53:17.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.127+0000 7feaccb9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7feac806c7a0 0x7feac80cc9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36900/0 (socket says 172.21.15.97:36900) 2023-07-06T03:53:17.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.127+0000 7feaccb9c700 1 -- 172.21.15.97:0/2166268567 learned_addr learned my addr 172.21.15.97:0/2166268567 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:17.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.127+0000 7feabffff700 1 --2- 172.21.15.97:0/2166268567 >> v2:172.21.15.146:3300/0 conn(0x7feac815d990 0x7feac815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:17.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.127+0000 7feac77fe700 1 -- 172.21.15.97:0/2166268567 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503793513 0 0) 0x7feac80c97c0 con 0x7feac806c7a0 2023-07-06T03:53:17.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.127+0000 7feac77fe700 1 -- 172.21.15.97:0/2166268567 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab40035b0 con 0x7feac806c7a0 2023-07-06T03:53:17.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.128+0000 7feabffff700 1 -- 172.21.15.97:0/2166268567 >> v1:172.21.15.146:6789/0 conn(0x7feac8159390 legacy=0x7feac80c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:17.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.128+0000 7feac77fe700 1 -- 172.21.15.97:0/2166268567 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3407448350 0 0) 0x7feab40035b0 con 0x7feac806c7a0 2023-07-06T03:53:17.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.128+0000 7feabffff700 1 -- 172.21.15.97:0/2166268567 >> v1:172.21.15.97:6789/0 conn(0x7feac806c7a0 legacy=0x7feac80cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:17.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.128+0000 7feabffff700 1 -- 172.21.15.97:0/2166268567 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac815d190 con 0x7feac815d990 2023-07-06T03:53:17.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.128+0000 7feabffff700 1 --2- 172.21.15.97:0/2166268567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feac815d990 0x7feac815a350 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7feab00097c0 tx=0x7feab0009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e6d7505ee8698a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:17.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.128+0000 7feac77fe700 1 -- 172.21.15.97:0/2166268567 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00019f0 con 0x7feac815d990 2023-07-06T03:53:17.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.128+0000 7feac77fe700 1 -- 172.21.15.97:0/2166268567 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab0004030 con 0x7feac815d990 2023-07-06T03:53:17.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.128+0000 7feac77fe700 1 -- 172.21.15.97:0/2166268567 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab00043a0 con 0x7feac815d990 2023-07-06T03:53:17.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.129+0000 7feace5ff700 1 -- 172.21.15.97:0/2166268567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feac815d990 msgr2=0x7feac815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:17.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.129+0000 7feace5ff700 1 --2- 172.21.15.97:0/2166268567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feac815d990 0x7feac815a350 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7feab00097c0 tx=0x7feab0009ad0 comp rx=0 tx=0).stop 2023-07-06T03:53:17.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.129+0000 7feace5ff700 1 -- 172.21.15.97:0/2166268567 shutdown_connections 2023-07-06T03:53:17.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.129+0000 7feace5ff700 1 --2- 172.21.15.97:0/2166268567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feac815d990 0x7feac815a350 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:17.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.129+0000 7feace5ff700 1 -- 172.21.15.97:0/2166268567 >> 172.21.15.97:0/2166268567 conn(0x7feac80b04a0 msgr2=0x7feac80b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:17.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.129+0000 7feace5ff700 1 -- 172.21.15.97:0/2166268567 shutdown_connections 2023-07-06T03:53:17.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.129+0000 7feace5ff700 1 -- 172.21.15.97:0/2166268567 wait complete. 2023-07-06T03:53:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.130+0000 7feace5ff700 1 Processor -- start 2023-07-06T03:53:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.131+0000 7feace5ff700 1 -- start start 2023-07-06T03:53:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.131+0000 7feace5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feac806c7a0 0x7feac8153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.131+0000 7feace5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feac8159390 0x7feac8151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.131+0000 7feace5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac815d990 0x7feac8156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.131+0000 7feace5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80ce1a0 con 0x7feac815d990 2023-07-06T03:53:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.131+0000 7feace5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80cd590 con 0x7feac806c7a0 2023-07-06T03:53:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.131+0000 7feace5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac8156ec0 con 0x7feac8159390 2023-07-06T03:53:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.132+0000 7feaccb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac815d990 0x7feac8156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.132+0000 7feac7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feac806c7a0 0x7feac8153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.132+0000 7feaccb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac815d990 0x7feac8156980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54752/0 (socket says 172.21.15.97:54752) 2023-07-06T03:53:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.132+0000 7feaccb9c700 1 -- 172.21.15.97:0/2614237900 learned_addr learned my addr 172.21.15.97:0/2614237900 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.132+0000 7feabffff700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feac8159390 0x7feac8151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.132+0000 7feaccb9c700 1 -- 172.21.15.97:0/2614237900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feac8159390 msgr2=0x7feac8151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.132+0000 7feaccb9c700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feac8159390 0x7feac8151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.132+0000 7feaccb9c700 1 -- 172.21.15.97:0/2614237900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feac806c7a0 msgr2=0x7feac8153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.132+0000 7feaccb9c700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feac806c7a0 0x7feac8153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.132+0000 7feaccb9c700 1 -- 172.21.15.97:0/2614237900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab00094a0 con 0x7feac815d990 2023-07-06T03:53:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.132+0000 7feaccb9c700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac815d990 0x7feac8156980 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7feac0001900 tx=0x7feac000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.133+0000 7feac57fa700 1 -- 172.21.15.97:0/2614237900 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac000ac50 con 0x7feac815d990 2023-07-06T03:53:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.133+0000 7feac57fa700 1 -- 172.21.15.97:0/2614237900 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac000add0 con 0x7feac815d990 2023-07-06T03:53:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.133+0000 7feace5ff700 1 -- 172.21.15.97:0/2614237900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac80ce6d0 con 0x7feac815d990 2023-07-06T03:53:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.133+0000 7feac57fa700 1 -- 172.21.15.97:0/2614237900 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0007310 con 0x7feac815d990 2023-07-06T03:53:17.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.133+0000 7feace5ff700 1 -- 172.21.15.97:0/2614237900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac80cd220 con 0x7feac815d990 2023-07-06T03:53:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.134+0000 7feace5ff700 1 -- 172.21.15.97:0/2614237900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac804e340 con 0x7feac815d990 2023-07-06T03:53:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.135+0000 7feac57fa700 1 -- 172.21.15.97:0/2614237900 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feac0007470 con 0x7feac815d990 2023-07-06T03:53:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.136+0000 7feac57fa700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feaa806ace0 0x7feaa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.136+0000 7feac57fa700 1 -- 172.21.15.97:0/2614237900 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7feac0088fb0 con 0x7feac815d990 2023-07-06T03:53:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.137+0000 7feac7fff700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feaa806ace0 0x7feaa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.137+0000 7feac7fff700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feaa806ace0 0x7feaa806d190 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7feac80702d0 tx=0x7feab8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:17.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.141+0000 7feac57fa700 1 -- 172.21.15.97:0/2614237900 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feac00bb310 con 0x7feac815d990 2023-07-06T03:53:17.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:17.411+0000 7feace5ff700 1 -- 172.21.15.97:0/2614237900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) v1 -- 0x7feac80b11f0 con 0x7feac815d990 2023-07-06T03:53:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.301+0000 7feac57fa700 1 -- 172.21.15.97:0/2614237900 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v413) v1 ==== 150+0+0 (secure 0 0 0) 0x7feac0018020 con 0x7feac815d990 2023-07-06T03:53:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.304+0000 7feace5ff700 1 -- 172.21.15.97:0/2614237900 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feaa806ace0 msgr2=0x7feaa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.305+0000 7feace5ff700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feaa806ace0 0x7feaa806d190 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7feac80702d0 tx=0x7feab8009040 comp rx=0 tx=0).stop 2023-07-06T03:53:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.305+0000 7feace5ff700 1 -- 172.21.15.97:0/2614237900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac815d990 msgr2=0x7feac8156980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.305+0000 7feace5ff700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac815d990 0x7feac8156980 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7feac0001900 tx=0x7feac000a340 comp rx=0 tx=0).stop 2023-07-06T03:53:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.305+0000 7feace5ff700 1 -- 172.21.15.97:0/2614237900 shutdown_connections 2023-07-06T03:53:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.305+0000 7feace5ff700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feaa806ace0 0x7feaa806d190 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.305+0000 7feace5ff700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac815d990 0x7feac8156980 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.305+0000 7feace5ff700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feac806c7a0 0x7feac8153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.305+0000 7feace5ff700 1 --2- 172.21.15.97:0/2614237900 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feac8159390 0x7feac8151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:18.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.305+0000 7feace5ff700 1 -- 172.21.15.97:0/2614237900 >> 172.21.15.97:0/2614237900 conn(0x7feac80b04a0 msgr2=0x7feac8152b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:18.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.305+0000 7feace5ff700 1 -- 172.21.15.97:0/2614237900 shutdown_connections 2023-07-06T03:53:18.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.305+0000 7feace5ff700 1 -- 172.21.15.97:0/2614237900 wait complete. 2023-07-06T03:53:18.327 INFO:tasks.cephfs_test_runner:test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-06T03:53:18.328 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:53:18.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2023-07-06T03:53:18.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.525+0000 7fa60cfff700 1 Processor -- start 2023-07-06T03:53:18.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.525+0000 7fa60cfff700 1 -- start start 2023-07-06T03:53:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.526+0000 7fa60cfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa60815a300 0x7fa6080c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.526+0000 7fa60cfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6080c9520 0x7fa608071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.526+0000 7fa60cfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa608070fc0 con 0x7fa608071990 2023-07-06T03:53:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.526+0000 7fa60cfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60806c470 con 0x7fa6080c9520 2023-07-06T03:53:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.526+0000 7fa60cfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60806c750 con 0x7fa60815a300 2023-07-06T03:53:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.526+0000 7fa605d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6080c9520 0x7fa608071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.526+0000 7fa606d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa608071990 0x7fa6080cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36904/0 (socket says 172.21.15.97:36904) 2023-07-06T03:53:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.526+0000 7fa606d9d700 1 -- 172.21.15.97:0/3279440805 learned_addr learned my addr 172.21.15.97:0/3279440805 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.527+0000 7fa60659c700 1 --2- 172.21.15.97:0/3279440805 >> v2:172.21.15.146:3300/0 conn(0x7fa60815a300 0x7fa6080c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.527+0000 7fa605d9b700 1 -- 172.21.15.97:0/3279440805 >> v2:172.21.15.146:3300/0 conn(0x7fa60815a300 msgr2=0x7fa6080c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.527+0000 7fa605d9b700 1 --2- 172.21.15.97:0/3279440805 >> v2:172.21.15.146:3300/0 conn(0x7fa60815a300 0x7fa6080c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.527+0000 7fa605d9b700 1 -- 172.21.15.97:0/3279440805 >> v1:172.21.15.97:6789/0 conn(0x7fa608071990 legacy=0x7fa6080cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.527+0000 7fa605d9b700 1 -- 172.21.15.97:0/3279440805 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa608070c50 con 0x7fa6080c9520 2023-07-06T03:53:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.527+0000 7fa60559a700 1 -- 172.21.15.97:0/3279440805 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213547826 0 0) 0x7fa608070fc0 con 0x7fa608071990 2023-07-06T03:53:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.527+0000 7fa605d9b700 1 --2- 172.21.15.97:0/3279440805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6080c9520 0x7fa608071450 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fa60806b530 tx=0x7fa5fc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696716b6b86da89d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:18.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.527+0000 7fa60559a700 1 -- 172.21.15.97:0/3279440805 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc011830 con 0x7fa6080c9520 2023-07-06T03:53:18.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.527+0000 7fa60559a700 1 -- 172.21.15.97:0/3279440805 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc011990 con 0x7fa6080c9520 2023-07-06T03:53:18.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.527+0000 7fa60559a700 1 -- 172.21.15.97:0/3279440805 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc011d20 con 0x7fa6080c9520 2023-07-06T03:53:18.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.528+0000 7fa60cfff700 1 -- 172.21.15.97:0/3279440805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6080c9520 msgr2=0x7fa608071450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:18.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.528+0000 7fa60cfff700 1 --2- 172.21.15.97:0/3279440805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6080c9520 0x7fa608071450 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fa60806b530 tx=0x7fa5fc009b20 comp rx=0 tx=0).stop 2023-07-06T03:53:18.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.528+0000 7fa60cfff700 1 -- 172.21.15.97:0/3279440805 shutdown_connections 2023-07-06T03:53:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.528+0000 7fa60cfff700 1 --2- 172.21.15.97:0/3279440805 >> v2:172.21.15.146:3300/0 conn(0x7fa60815a300 0x7fa6080c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.528+0000 7fa60cfff700 1 --2- 172.21.15.97:0/3279440805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6080c9520 0x7fa608071450 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.528+0000 7fa60cfff700 1 -- 172.21.15.97:0/3279440805 >> 172.21.15.97:0/3279440805 conn(0x7fa6080b04a0 msgr2=0x7fa6080b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.528+0000 7fa60cfff700 1 -- 172.21.15.97:0/3279440805 shutdown_connections 2023-07-06T03:53:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.528+0000 7fa60cfff700 1 -- 172.21.15.97:0/3279440805 wait complete. 2023-07-06T03:53:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.530+0000 7fa60cfff700 1 Processor -- start 2023-07-06T03:53:18.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.530+0000 7fa60cfff700 1 -- start start 2023-07-06T03:53:18.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.531+0000 7fa60cfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa608071990 0x7fa6080c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:18.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.531+0000 7fa60cfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6080c9520 0x7fa6080c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:18.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.531+0000 7fa60cfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60815a300 0x7fa608154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:18.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.531+0000 7fa60cfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6080c2c40 con 0x7fa60815a300 2023-07-06T03:53:18.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.531+0000 7fa60cfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6080c29e0 con 0x7fa608071990 2023-07-06T03:53:18.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.531+0000 7fa60cfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa608158fe0 con 0x7fa6080c9520 2023-07-06T03:53:18.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.531+0000 7fa606d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60815a300 0x7fa608154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:18.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.531+0000 7fa606d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60815a300 0x7fa608154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54772/0 (socket says 172.21.15.97:54772) 2023-07-06T03:53:18.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.531+0000 7fa606d9d700 1 -- 172.21.15.97:0/3276009739 learned_addr learned my addr 172.21.15.97:0/3276009739 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:18.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.531+0000 7fa605d9b700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6080c9520 0x7fa6080c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:18.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.531+0000 7fa60659c700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa608071990 0x7fa6080c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:18.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa606d9d700 1 -- 172.21.15.97:0/3276009739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6080c9520 msgr2=0x7fa6080c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:18.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa606d9d700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6080c9520 0x7fa6080c24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:18.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa606d9d700 1 -- 172.21.15.97:0/3276009739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa608071990 msgr2=0x7fa6080c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa606d9d700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa608071990 0x7fa6080c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa606d9d700 1 -- 172.21.15.97:0/3276009739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc0094a0 con 0x7fa60815a300 2023-07-06T03:53:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa60659c700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa608071990 0x7fa6080c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:18.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa606d9d700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60815a300 0x7fa608154b60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa600002230 tx=0x7fa60000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:18.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa5f6ffd700 1 -- 172.21.15.97:0/3276009739 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000ab70 con 0x7fa60815a300 2023-07-06T03:53:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa5f6ffd700 1 -- 172.21.15.97:0/3276009739 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60000acf0 con 0x7fa60815a300 2023-07-06T03:53:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa60cfff700 1 -- 172.21.15.97:0/3276009739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6081550a0 con 0x7fa60815a300 2023-07-06T03:53:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa5f6ffd700 1 -- 172.21.15.97:0/3276009739 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600007310 con 0x7fa60815a300 2023-07-06T03:53:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.532+0000 7fa60cfff700 1 -- 172.21.15.97:0/3276009739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa608159560 con 0x7fa60815a300 2023-07-06T03:53:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.533+0000 7fa60cfff700 1 -- 172.21.15.97:0/3276009739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60804e340 con 0x7fa60815a300 2023-07-06T03:53:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.534+0000 7fa5f6ffd700 1 -- 172.21.15.97:0/3276009739 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa600007470 con 0x7fa60815a300 2023-07-06T03:53:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.535+0000 7fa5f6ffd700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5e806ace0 0x7fa5e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:18.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.535+0000 7fa5f6ffd700 1 -- 172.21.15.97:0/3276009739 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa600088cc0 con 0x7fa60815a300 2023-07-06T03:53:18.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.535+0000 7fa60659c700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5e806ace0 0x7fa5e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:18.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.536+0000 7fa60659c700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5e806ace0 0x7fa5e806d190 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8007a50 tx=0x7fa5f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:18.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.540+0000 7fa5f6ffd700 1 -- 172.21.15.97:0/3276009739 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa60008f080 con 0x7fa60815a300 2023-07-06T03:53:18.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:18.813+0000 7fa60cfff700 1 -- 172.21.15.97:0/3276009739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) v1 -- 0x7fa6080b1080 con 0x7fa60815a300 2023-07-06T03:53:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.306+0000 7fa5f6ffd700 1 -- 172.21.15.97:0/3276009739 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v414) v1 ==== 146+0+0 (secure 0 0 0) 0x7fa6000180d0 con 0x7fa60815a300 2023-07-06T03:53:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 -- 172.21.15.97:0/3276009739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5e806ace0 msgr2=0x7fa5e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5e806ace0 0x7fa5e806d190 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8007a50 tx=0x7fa5f8008040 comp rx=0 tx=0).stop 2023-07-06T03:53:19.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 -- 172.21.15.97:0/3276009739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60815a300 msgr2=0x7fa608154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60815a300 0x7fa608154b60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa600002230 tx=0x7fa60000cf50 comp rx=0 tx=0).stop 2023-07-06T03:53:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 -- 172.21.15.97:0/3276009739 shutdown_connections 2023-07-06T03:53:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa5e806ace0 0x7fa5e806d190 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa60815a300 0x7fa608154b60 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa608071990 0x7fa6080c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 --2- 172.21.15.97:0/3276009739 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa6080c9520 0x7fa6080c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 -- 172.21.15.97:0/3276009739 >> 172.21.15.97:0/3276009739 conn(0x7fa6080b04a0 msgr2=0x7fa60806b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:19.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 -- 172.21.15.97:0/3276009739 shutdown_connections 2023-07-06T03:53:19.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.308+0000 7fa60cfff700 1 -- 172.21.15.97:0/3276009739 wait complete. 2023-07-06T03:53:19.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T03:53:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.524+0000 7f6ef23ff700 1 Processor -- start 2023-07-06T03:53:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.525+0000 7f6ef23ff700 1 -- start start 2023-07-06T03:53:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.526+0000 7f6ef23ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6eec15c610 0x7f6eec0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.526+0000 7f6ef23ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6eec06b360 0x7f6eec0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.526+0000 7f6ef23ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eec0cff90 con 0x7f6eec0d2210 2023-07-06T03:53:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.526+0000 7f6ef23ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eec0d0780 con 0x7f6eec15c610 2023-07-06T03:53:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.526+0000 7f6ef23ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec15c9e0 con 0x7f6eec06b360 2023-07-06T03:53:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.526+0000 7f6eebfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6eec15c610 0x7f6eec0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.526+0000 7f6eebfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6eec15c610 0x7f6eec0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54786/0 (socket says 172.21.15.97:54786) 2023-07-06T03:53:19.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.526+0000 7f6eebfff700 1 -- 172.21.15.97:0/2880094693 learned_addr learned my addr 172.21.15.97:0/2880094693 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:19.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.526+0000 7f6eebfff700 1 -- 172.21.15.97:0/2880094693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6eec06b360 msgr2=0x7f6eec0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:19.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.526+0000 7f6eebfff700 1 --2- 172.21.15.97:0/2880094693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6eec06b360 0x7f6eec0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.526+0000 7f6eebfff700 1 -- 172.21.15.97:0/2880094693 >> v1:172.21.15.146:6789/0 conn(0x7f6eec0d2210 legacy=0x7f6eec0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:19.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.527+0000 7f6eebfff700 1 -- 172.21.15.97:0/2880094693 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec0b5650 con 0x7f6eec15c610 2023-07-06T03:53:19.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.527+0000 7f6eebfff700 1 --2- 172.21.15.97:0/2880094693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec15c610 0x7f6eec0b47b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009920 tx=0x7f6edc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c614985bd040d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:19.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.527+0000 7f6eeb7fe700 1 -- 172.21.15.97:0/2880094693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc004230 con 0x7f6eec15c610 2023-07-06T03:53:19.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.527+0000 7f6eeb7fe700 1 -- 172.21.15.97:0/2880094693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc004390 con 0x7f6eec15c610 2023-07-06T03:53:19.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.527+0000 7f6eeb7fe700 1 -- 172.21.15.97:0/2880094693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc009e70 con 0x7f6eec15c610 2023-07-06T03:53:19.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.527+0000 7f6ef23ff700 1 -- 172.21.15.97:0/2880094693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec15c610 msgr2=0x7f6eec0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:19.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.527+0000 7f6ef23ff700 1 --2- 172.21.15.97:0/2880094693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec15c610 0x7f6eec0b47b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009920 tx=0x7f6edc009c30 comp rx=0 tx=0).stop 2023-07-06T03:53:19.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.528+0000 7f6ef23ff700 1 -- 172.21.15.97:0/2880094693 shutdown_connections 2023-07-06T03:53:19.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.528+0000 7f6ef23ff700 1 --2- 172.21.15.97:0/2880094693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec15c610 0x7f6eec0b47b0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.528+0000 7f6ef23ff700 1 --2- 172.21.15.97:0/2880094693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6eec06b360 0x7f6eec0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.528+0000 7f6ef23ff700 1 -- 172.21.15.97:0/2880094693 >> 172.21.15.97:0/2880094693 conn(0x7f6eec06a6c0 msgr2=0x7f6eec06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:19.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.528+0000 7f6ef23ff700 1 -- 172.21.15.97:0/2880094693 shutdown_connections 2023-07-06T03:53:19.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.528+0000 7f6ef23ff700 1 -- 172.21.15.97:0/2880094693 wait complete. 2023-07-06T03:53:19.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.529+0000 7f6ef23ff700 1 Processor -- start 2023-07-06T03:53:19.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.530+0000 7f6ef23ff700 1 -- start start 2023-07-06T03:53:19.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.530+0000 7f6ef23ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6eec06b360 0x7f6eec14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:19.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.530+0000 7f6ef23ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0d2210 0x7f6eec15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:19.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.530+0000 7f6ef23ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6eec15c610 0x7f6eec15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:19.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ef23ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0c8e70 con 0x7f6eec0d2210 2023-07-06T03:53:19.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ef23ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0cb020 con 0x7f6eec06b360 2023-07-06T03:53:19.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ef23ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0caaa0 con 0x7f6eec15c610 2023-07-06T03:53:19.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ee3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0d2210 0x7f6eec15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:19.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ee3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0d2210 0x7f6eec15cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54802/0 (socket says 172.21.15.97:54802) 2023-07-06T03:53:19.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ee3fff700 1 -- 172.21.15.97:0/4081114632 learned_addr learned my addr 172.21.15.97:0/4081114632 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:19.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6eebfff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6eec06b360 0x7f6eec14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:19.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ef099c700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6eec15c610 0x7f6eec15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:19.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ee3fff700 1 -- 172.21.15.97:0/4081114632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6eec15c610 msgr2=0x7f6eec15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:19.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ee3fff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6eec15c610 0x7f6eec15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ee3fff700 1 -- 172.21.15.97:0/4081114632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6eec06b360 msgr2=0x7f6eec14fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:19.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ee3fff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6eec06b360 0x7f6eec14fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ee3fff700 1 -- 172.21.15.97:0/4081114632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc009550 con 0x7f6eec0d2210 2023-07-06T03:53:19.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.531+0000 7f6ee3fff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0d2210 0x7f6eec15cb20 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400b500 tx=0x7f6ed400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.532+0000 7f6ee97fa700 1 -- 172.21.15.97:0/4081114632 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4011830 con 0x7f6eec0d2210 2023-07-06T03:53:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.532+0000 7f6ee97fa700 1 -- 172.21.15.97:0/4081114632 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed4011990 con 0x7f6eec0d2210 2023-07-06T03:53:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.532+0000 7f6ef23ff700 1 -- 172.21.15.97:0/4081114632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eec0c9080 con 0x7f6eec0d2210 2023-07-06T03:53:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.532+0000 7f6ee97fa700 1 -- 172.21.15.97:0/4081114632 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4011d40 con 0x7f6eec0d2210 2023-07-06T03:53:19.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.532+0000 7f6ef23ff700 1 -- 172.21.15.97:0/4081114632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eec0c9320 con 0x7f6eec0d2210 2023-07-06T03:53:19.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.533+0000 7f6ef23ff700 1 -- 172.21.15.97:0/4081114632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eec159dd0 con 0x7f6eec0d2210 2023-07-06T03:53:19.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.534+0000 7f6ee97fa700 1 -- 172.21.15.97:0/4081114632 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ed401a400 con 0x7f6eec0d2210 2023-07-06T03:53:19.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.535+0000 7f6ee97fa700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ecc06ace0 0x7f6ecc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:19.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.535+0000 7f6ee97fa700 1 -- 172.21.15.97:0/4081114632 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6ed4089120 con 0x7f6eec0d2210 2023-07-06T03:53:19.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.535+0000 7f6eebfff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ecc06ace0 0x7f6ecc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:19.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.536+0000 7f6eebfff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ecc06ace0 0x7f6ecc06d190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009130 tx=0x7f6edc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:19.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.540+0000 7f6ee97fa700 1 -- 172.21.15.97:0/4081114632 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6ed40544e0 con 0x7f6eec0d2210 2023-07-06T03:53:19.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.737+0000 7f6ef23ff700 1 -- 172.21.15.97:0/4081114632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6eec04e340 con 0x7f6eec0d2210 2023-07-06T03:53:19.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.742+0000 7f6ee97fa700 1 -- 172.21.15.97:0/4081114632 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ed4057b00 con 0x7f6eec0d2210 2023-07-06T03:53:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 -- 172.21.15.97:0/4081114632 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ecc06ace0 msgr2=0x7f6ecc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ecc06ace0 0x7f6ecc06d190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009130 tx=0x7f6edc008030 comp rx=0 tx=0).stop 2023-07-06T03:53:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 -- 172.21.15.97:0/4081114632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0d2210 msgr2=0x7f6eec15cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0d2210 0x7f6eec15cb20 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400b500 tx=0x7f6ed400b810 comp rx=0 tx=0).stop 2023-07-06T03:53:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 -- 172.21.15.97:0/4081114632 shutdown_connections 2023-07-06T03:53:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ecc06ace0 0x7f6ecc06d190 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6eec15c610 0x7f6eec15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6eec06b360 0x7f6eec14fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 --2- 172.21.15.97:0/4081114632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0d2210 0x7f6eec15cb20 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:19.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 -- 172.21.15.97:0/4081114632 >> 172.21.15.97:0/4081114632 conn(0x7f6eec06a6c0 msgr2=0x7f6eec15b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:19.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 -- 172.21.15.97:0/4081114632 shutdown_connections 2023-07-06T03:53:19.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:19.745+0000 7f6ef23ff700 1 -- 172.21.15.97:0/4081114632 wait complete. 2023-07-06T03:53:19.764 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:53:19.764 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:53:19.865 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:53:19.975 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:53:19.975 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:53:19.976 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:53:20.021 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:53:20.021 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:53:20.022 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:53:20.069 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:53:20.069 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:53:20.070 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:53:20.115 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:53:20.115 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:53:20.116 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:53:20.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:53:20.162 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T03:53:20.163 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:53:20.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.357+0000 7f0d827ff700 1 Processor -- start 2023-07-06T03:53:20.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.357+0000 7f0d827ff700 1 -- start start 2023-07-06T03:53:20.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.358+0000 7f0d827ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d7c15c520 0x7f0d7c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:20.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.358+0000 7f0d827ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d7c160d00 0x7f0d7c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:20.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.358+0000 7f0d827ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7c0cdae0 con 0x7f0d7c160180 2023-07-06T03:53:20.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.358+0000 7f0d827ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c0cd8d0 con 0x7f0d7c160d00 2023-07-06T03:53:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.358+0000 7f0d827ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0b6640 con 0x7f0d7c15c520 2023-07-06T03:53:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.358+0000 7f0d80d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d7c160d00 0x7f0d7c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.358+0000 7f0d80d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d7c160d00 0x7f0d7c0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38878/0 (socket says 172.21.15.97:38878) 2023-07-06T03:53:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.358+0000 7f0d80d9c700 1 -- 172.21.15.97:0/2863671692 learned_addr learned my addr 172.21.15.97:0/2863671692 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.358+0000 7f0d73fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0d7c160180 0x7f0d7c1605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36906/0 (socket says 172.21.15.97:36906) 2023-07-06T03:53:20.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.359+0000 7f0d7bfff700 1 --2- 172.21.15.97:0/2863671692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d7c15c520 0x7f0d7c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:20.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.359+0000 7f0d80d9c700 1 -- 172.21.15.97:0/2863671692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d7c15c520 msgr2=0x7f0d7c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:20.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.359+0000 7f0d80d9c700 1 --2- 172.21.15.97:0/2863671692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d7c15c520 0x7f0d7c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.359+0000 7f0d80d9c700 1 -- 172.21.15.97:0/2863671692 >> v1:172.21.15.97:6789/0 conn(0x7f0d7c160180 legacy=0x7f0d7c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:20.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.359+0000 7f0d80d9c700 1 -- 172.21.15.97:0/2863671692 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0b6920 con 0x7f0d7c160d00 2023-07-06T03:53:20.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.359+0000 7f0d80d9c700 1 --2- 172.21.15.97:0/2863671692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d7c160d00 0x7f0d7c0b7400 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009e50 tx=0x7f0d7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe57b6d267f1886 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:20.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.359+0000 7f0d7b7fe700 1 -- 172.21.15.97:0/2863671692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400e070 con 0x7f0d7c160d00 2023-07-06T03:53:20.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.359+0000 7f0d7b7fe700 1 -- 172.21.15.97:0/2863671692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7400b9c0 con 0x7f0d7c160d00 2023-07-06T03:53:20.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.359+0000 7f0d7b7fe700 1 -- 172.21.15.97:0/2863671692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400bd70 con 0x7f0d7c160d00 2023-07-06T03:53:20.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.360+0000 7f0d827ff700 1 -- 172.21.15.97:0/2863671692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d7c160d00 msgr2=0x7f0d7c0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:20.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.360+0000 7f0d827ff700 1 --2- 172.21.15.97:0/2863671692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d7c160d00 0x7f0d7c0b7400 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009e50 tx=0x7f0d7400b330 comp rx=0 tx=0).stop 2023-07-06T03:53:20.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.360+0000 7f0d827ff700 1 -- 172.21.15.97:0/2863671692 shutdown_connections 2023-07-06T03:53:20.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.360+0000 7f0d827ff700 1 --2- 172.21.15.97:0/2863671692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d7c160d00 0x7f0d7c0b7400 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.360+0000 7f0d827ff700 1 --2- 172.21.15.97:0/2863671692 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d7c15c520 0x7f0d7c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.360+0000 7f0d827ff700 1 -- 172.21.15.97:0/2863671692 >> 172.21.15.97:0/2863671692 conn(0x7f0d7c071790 msgr2=0x7f0d7c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:20.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.360+0000 7f0d827ff700 1 -- 172.21.15.97:0/2863671692 shutdown_connections 2023-07-06T03:53:20.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.360+0000 7f0d827ff700 1 -- 172.21.15.97:0/2863671692 wait complete. 2023-07-06T03:53:20.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.362+0000 7f0d827ff700 1 Processor -- start 2023-07-06T03:53:20.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.362+0000 7f0d827ff700 1 -- start start 2023-07-06T03:53:20.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d827ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d7c15c520 0x7f0d7c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:20.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d827ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d7c160180 0x7f0d7c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:20.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d827ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d7c160d00 0x7f0d7c1f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:20.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d827ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c157a60 con 0x7f0d7c15c520 2023-07-06T03:53:20.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d827ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0ca710 con 0x7f0d7c160180 2023-07-06T03:53:20.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d827ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0ca9f0 con 0x7f0d7c160d00 2023-07-06T03:53:20.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d7bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d7c15c520 0x7f0d7c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:20.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d7bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d7c15c520 0x7f0d7c157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38888/0 (socket says 172.21.15.97:38888) 2023-07-06T03:53:20.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d7bfff700 1 -- 172.21.15.97:0/2588187465 learned_addr learned my addr 172.21.15.97:0/2588187465 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:20.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d80d9c700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d7c160d00 0x7f0d7c1f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:20.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d73fff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d7c160180 0x7f0d7c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:20.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d7bfff700 1 -- 172.21.15.97:0/2588187465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d7c160d00 msgr2=0x7f0d7c1f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:20.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d7bfff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d7c160d00 0x7f0d7c1f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d7bfff700 1 -- 172.21.15.97:0/2588187465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d7c160180 msgr2=0x7f0d7c0c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:20.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d7bfff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d7c160180 0x7f0d7c0c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d7bfff700 1 -- 172.21.15.97:0/2588187465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74009b30 con 0x7f0d7c15c520 2023-07-06T03:53:20.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.363+0000 7f0d73fff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d7c160180 0x7f0d7c0c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:20.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.364+0000 7f0d7bfff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d7c15c520 0x7f0d7c157520 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00d800 tx=0x7f0d6c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:20.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.364+0000 7f0d797fa700 1 -- 172.21.15.97:0/2588187465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0125f0 con 0x7f0d7c15c520 2023-07-06T03:53:20.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.364+0000 7f0d797fa700 1 -- 172.21.15.97:0/2588187465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c012750 con 0x7f0d7c15c520 2023-07-06T03:53:20.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.364+0000 7f0d827ff700 1 -- 172.21.15.97:0/2588187465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c0cacd0 con 0x7f0d7c15c520 2023-07-06T03:53:20.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.364+0000 7f0d797fa700 1 -- 172.21.15.97:0/2588187465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c012b00 con 0x7f0d7c15c520 2023-07-06T03:53:20.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.364+0000 7f0d827ff700 1 -- 172.21.15.97:0/2588187465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c0cb290 con 0x7f0d7c15c520 2023-07-06T03:53:20.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.365+0000 7f0d827ff700 1 -- 172.21.15.97:0/2588187465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c0cb600 con 0x7f0d7c15c520 2023-07-06T03:53:20.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.366+0000 7f0d797fa700 1 -- 172.21.15.97:0/2588187465 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d6c012c60 con 0x7f0d7c15c520 2023-07-06T03:53:20.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.367+0000 7f0d797fa700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d5c06ace0 0x7f0d5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:20.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.367+0000 7f0d797fa700 1 -- 172.21.15.97:0/2588187465 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0d6c089550 con 0x7f0d7c15c520 2023-07-06T03:53:20.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.367+0000 7f0d73fff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d5c06ace0 0x7f0d5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:20.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.367+0000 7f0d73fff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d5c06ace0 0x7f0d5c06d190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0b47e0 tx=0x7f0d64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:20.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.371+0000 7f0d797fa700 1 -- 172.21.15.97:0/2588187465 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0d6c054910 con 0x7f0d7c15c520 2023-07-06T03:53:20.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.607+0000 7f0d827ff700 1 -- 172.21.15.97:0/2588187465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d7c04e340 con 0x7f0d7c15c520 2023-07-06T03:53:20.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.607+0000 7f0d797fa700 1 -- 172.21.15.97:0/2588187465 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f0d6c057f30 con 0x7f0d7c15c520 2023-07-06T03:53:20.609 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:20.609 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3945452207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3945452207},{"type":"v1","addr":"172.21.15.146:6835","nonce":3945452207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2428591387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2428591387},{"type":"v1","addr":"172.21.15.97:6837","nonce":2428591387}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/2089126027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":2089126027},{"type":"v1","addr":"172.21.15.146:6834","nonce":2089126027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7914,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3736180661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3736180661},{"type":"v1","addr":"172.21.15.97:6836","nonce":3736180661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[]} 2023-07-06T03:53:20.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 -- 172.21.15.97:0/2588187465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d5c06ace0 msgr2=0x7f0d5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:20.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d5c06ace0 0x7f0d5c06d190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0b47e0 tx=0x7f0d64009040 comp rx=0 tx=0).stop 2023-07-06T03:53:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 -- 172.21.15.97:0/2588187465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d7c15c520 msgr2=0x7f0d7c157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d7c15c520 0x7f0d7c157520 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00d800 tx=0x7f0d6c00dbc0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 -- 172.21.15.97:0/2588187465 shutdown_connections 2023-07-06T03:53:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0d5c06ace0 0x7f0d5c06d190 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0d7c160d00 0x7f0d7c1f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d7c15c520 0x7f0d7c157520 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 --2- 172.21.15.97:0/2588187465 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d7c160180 0x7f0d7c0c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 -- 172.21.15.97:0/2588187465 >> 172.21.15.97:0/2588187465 conn(0x7f0d7c071790 msgr2=0x7f0d7c0b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:20.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 -- 172.21.15.97:0/2588187465 shutdown_connections 2023-07-06T03:53:20.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.610+0000 7f0d827ff700 1 -- 172.21.15.97:0/2588187465 wait complete. 2023-07-06T03:53:20.614 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2023-07-06T03:53:20.630 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T03:53:20.630 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T03:53:20.630 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T03:53:20.630 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:53:20.631 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T03:53:20.631 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T03:53:20.631 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T03:53:20.631 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:53:20.632 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T03:53:20.632 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T03:53:20.632 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T03:53:20.632 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:53:20.632 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T03:53:20.633 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T03:53:20.633 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T03:53:20.633 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:53:20.643 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T03:53:20.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T03:53:20.645 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T03:53:20.645 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T03:53:20.646 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T03:53:20.646 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T03:53:20.647 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T03:53:20.648 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T03:53:20.688 INFO:tasks.ceph.mds.a:Started 2023-07-06T03:53:20.688 DEBUG:teuthology.parallel:result is None 2023-07-06T03:53:20.692 INFO:tasks.ceph.mds.b:Started 2023-07-06T03:53:20.692 DEBUG:teuthology.parallel:result is None 2023-07-06T03:53:20.693 INFO:tasks.ceph.mds.c:Started 2023-07-06T03:53:20.693 DEBUG:teuthology.parallel:result is None 2023-07-06T03:53:20.695 INFO:tasks.ceph.mds.d:Started 2023-07-06T03:53:20.696 DEBUG:teuthology.parallel:result is None 2023-07-06T03:53:20.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T03:53:20.836 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-07-06T03:53:20.836 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:53:20.837 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:53:20.838 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:53:20.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.901+0000 7f227c5ff700 1 Processor -- start 2023-07-06T03:53:20.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.901+0000 7f227c5ff700 1 -- start start 2023-07-06T03:53:20.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.903+0000 7f227c5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f227415c610 0x7f22740b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.903+0000 7f227c5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740cff90 con 0x7f22740d2210 2023-07-06T03:53:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.903+0000 7f227c5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740b5650 con 0x7f227406b360 2023-07-06T03:53:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.903+0000 7f227c5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22740d0780 con 0x7f227415c610 2023-07-06T03:53:20.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.903+0000 7f227ab9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f22740d2210 0x7f22740d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36920/0 (socket says 172.21.15.97:36920) 2023-07-06T03:53:20.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.903+0000 7f227ab9c700 1 -- 172.21.15.97:0/1097385671 learned_addr learned my addr 172.21.15.97:0/1097385671 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:20.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.904+0000 7f227a39b700 1 --2- 172.21.15.97:0/1097385671 >> v2:172.21.15.146:3300/0 conn(0x7f227415c610 0x7f22740b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.904+0000 7f2279399700 1 -- 172.21.15.97:0/1097385671 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188808163 0 0) 0x7f22740cff90 con 0x7f22740d2210 2023-07-06T03:53:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.904+0000 7f2279399700 1 -- 172.21.15.97:0/1097385671 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22600035b0 con 0x7f22740d2210 2023-07-06T03:53:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.904+0000 7f2279399700 1 -- 172.21.15.97:0/1097385671 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349466645 0 0) 0x7f22740b5650 con 0x7f227406b360 2023-07-06T03:53:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.904+0000 7f227a39b700 1 -- 172.21.15.97:0/1097385671 >> v1:172.21.15.146:6789/0 conn(0x7f227406b360 legacy=0x7f22740b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.904+0000 7f227a39b700 1 -- 172.21.15.97:0/1097385671 >> v1:172.21.15.97:6789/0 conn(0x7f22740d2210 legacy=0x7f22740d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.904+0000 7f227a39b700 1 -- 172.21.15.97:0/1097385671 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22740b5400 con 0x7f227415c610 2023-07-06T03:53:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.904+0000 7f227a39b700 1 --2- 172.21.15.97:0/1097385671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f227415c610 0x7f22740b47b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2270009840 tx=0x7f2270009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48654c6ec391ce3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.905+0000 7f2279399700 1 -- 172.21.15.97:0/1097385671 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270011830 con 0x7f227415c610 2023-07-06T03:53:20.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.905+0000 7f2279399700 1 -- 172.21.15.97:0/1097385671 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2270011990 con 0x7f227415c610 2023-07-06T03:53:20.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.905+0000 7f2279399700 1 -- 172.21.15.97:0/1097385671 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270011d20 con 0x7f227415c610 2023-07-06T03:53:20.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.905+0000 7f227c5ff700 1 -- 172.21.15.97:0/1097385671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f227415c610 msgr2=0x7f22740b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.905+0000 7f227c5ff700 1 --2- 172.21.15.97:0/1097385671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f227415c610 0x7f22740b47b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2270009840 tx=0x7f2270009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.905+0000 7f227c5ff700 1 -- 172.21.15.97:0/1097385671 shutdown_connections 2023-07-06T03:53:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.905+0000 7f227c5ff700 1 --2- 172.21.15.97:0/1097385671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f227415c610 0x7f22740b47b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.905+0000 7f227c5ff700 1 -- 172.21.15.97:0/1097385671 >> 172.21.15.97:0/1097385671 conn(0x7f227406a6c0 msgr2=0x7f227406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.905+0000 7f227c5ff700 1 -- 172.21.15.97:0/1097385671 shutdown_connections 2023-07-06T03:53:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.905+0000 7f227c5ff700 1 -- 172.21.15.97:0/1097385671 wait complete. 2023-07-06T03:53:20.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.907+0000 7f227c5ff700 1 Processor -- start 2023-07-06T03:53:20.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.907+0000 7f227c5ff700 1 -- start start 2023-07-06T03:53:20.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.908+0000 7f227c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f227406b360 0x7f227415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.908+0000 7f227c5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22740d2210 0x7f227415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.908+0000 7f227c5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f227415c610 0x7f22740b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.908+0000 7f227c5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227415dcc0 con 0x7f22740d2210 2023-07-06T03:53:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.908+0000 7f227c5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227415dab0 con 0x7f227415c610 2023-07-06T03:53:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.908+0000 7f227c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22740cb9e0 con 0x7f227406b360 2023-07-06T03:53:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.908+0000 7f2279b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22740d2210 0x7f227415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.908+0000 7f2279b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22740d2210 0x7f227415d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38926/0 (socket says 172.21.15.97:38926) 2023-07-06T03:53:20.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.908+0000 7f2279b9a700 1 -- 172.21.15.97:0/3667300418 learned_addr learned my addr 172.21.15.97:0/3667300418 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:20.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.908+0000 7f227a39b700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f227406b360 0x7f227415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:20.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.908+0000 7f227ab9c700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f227415c610 0x7f22740b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:20.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.909+0000 7f2279b9a700 1 -- 172.21.15.97:0/3667300418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f227406b360 msgr2=0x7f227415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:20.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.909+0000 7f2279b9a700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f227406b360 0x7f227415cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.909+0000 7f2279b9a700 1 -- 172.21.15.97:0/3667300418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f227415c610 msgr2=0x7f22740b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:20.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.909+0000 7f2279b9a700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f227415c610 0x7f22740b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:20.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.909+0000 7f2279b9a700 1 -- 172.21.15.97:0/3667300418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22700094a0 con 0x7f22740d2210 2023-07-06T03:53:20.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.909+0000 7f2279b9a700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22740d2210 0x7f227415d570 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f226400c550 tx=0x7f226400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:20.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.909+0000 7f226affd700 1 -- 172.21.15.97:0/3667300418 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22640054e0 con 0x7f22740d2210 2023-07-06T03:53:20.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.909+0000 7f226affd700 1 -- 172.21.15.97:0/3667300418 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2264003680 con 0x7f22740d2210 2023-07-06T03:53:20.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.909+0000 7f226affd700 1 -- 172.21.15.97:0/3667300418 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264003a30 con 0x7f22740d2210 2023-07-06T03:53:20.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.909+0000 7f227c5ff700 1 -- 172.21.15.97:0/3667300418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227415f9b0 con 0x7f22740d2210 2023-07-06T03:53:20.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.909+0000 7f227c5ff700 1 -- 172.21.15.97:0/3667300418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227415f3f0 con 0x7f22740d2210 2023-07-06T03:53:20.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.910+0000 7f227c5ff700 1 -- 172.21.15.97:0/3667300418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227404e340 con 0x7f22740d2210 2023-07-06T03:53:20.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.911+0000 7f226affd700 1 -- 172.21.15.97:0/3667300418 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2264003b90 con 0x7f22740d2210 2023-07-06T03:53:20.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.912+0000 7f226affd700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f225806ad10 0x7f225806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:20.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.912+0000 7f226affd700 1 -- 172.21.15.97:0/3667300418 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2264088500 con 0x7f22740d2210 2023-07-06T03:53:20.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.912+0000 7f227a39b700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f225806ad10 0x7f225806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:20.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.913+0000 7f227a39b700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f225806ad10 0x7f225806d1c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2270009fd0 tx=0x7f227001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:20.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:20.916+0000 7f226affd700 1 -- 172.21.15.97:0/3667300418 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2264053830 con 0x7f22740d2210 2023-07-06T03:53:21.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.128+0000 7f227c5ff700 1 -- 172.21.15.97:0/3667300418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f22740cbb20 con 0x7f22740d2210 2023-07-06T03:53:21.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.325+0000 7f226affd700 1 -- 172.21.15.97:0/3667300418 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v93) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2264017020 con 0x7f22740d2210 2023-07-06T03:53:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 -- 172.21.15.97:0/3667300418 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f225806ad10 msgr2=0x7f225806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f225806ad10 0x7f225806d1c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2270009fd0 tx=0x7f227001c040 comp rx=0 tx=0).stop 2023-07-06T03:53:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 -- 172.21.15.97:0/3667300418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22740d2210 msgr2=0x7f227415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22740d2210 0x7f227415d570 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f226400c550 tx=0x7f226400c860 comp rx=0 tx=0).stop 2023-07-06T03:53:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 -- 172.21.15.97:0/3667300418 shutdown_connections 2023-07-06T03:53:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f225806ad10 0x7f225806d1c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f227415c610 0x7f22740b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f227406b360 0x7f227415cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 --2- 172.21.15.97:0/3667300418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22740d2210 0x7f227415d570 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 -- 172.21.15.97:0/3667300418 >> 172.21.15.97:0/3667300418 conn(0x7f227406a6c0 msgr2=0x7f227415b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 -- 172.21.15.97:0/3667300418 shutdown_connections 2023-07-06T03:53:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.328+0000 7f227c5ff700 1 -- 172.21.15.97:0/3667300418 wait complete. 2023-07-06T03:53:21.331 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-07-06T03:53:21.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T03:53:21.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.541+0000 7f48cfbff700 1 Processor -- start 2023-07-06T03:53:21.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.542+0000 7f48cfbff700 1 -- start start 2023-07-06T03:53:21.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48cfbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c815c520 0x7f48c80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48cfbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48c8160d00 0x7f48c80b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48cfbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c80cdae0 con 0x7f48c8160180 2023-07-06T03:53:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48cfbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c80cd8d0 con 0x7f48c8160d00 2023-07-06T03:53:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48cfbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80b6640 con 0x7f48c815c520 2023-07-06T03:53:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48ce19c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48c8160d00 0x7f48c80b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48ce19c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48c8160d00 0x7f48c80b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38932/0 (socket says 172.21.15.97:38932) 2023-07-06T03:53:21.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48cd19a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f48c8160180 0x7f48c81605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36922/0 (socket says 172.21.15.97:36922) 2023-07-06T03:53:21.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48ce19c700 1 -- 172.21.15.97:0/2981917006 learned_addr learned my addr 172.21.15.97:0/2981917006 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:21.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48cd99b700 1 --2- 172.21.15.97:0/2981917006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c815c520 0x7f48c80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:21.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48ce19c700 1 -- 172.21.15.97:0/2981917006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c815c520 msgr2=0x7f48c80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:21.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48ce19c700 1 --2- 172.21.15.97:0/2981917006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c815c520 0x7f48c80cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48ce19c700 1 -- 172.21.15.97:0/2981917006 >> v1:172.21.15.97:6789/0 conn(0x7f48c8160180 legacy=0x7f48c81605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:21.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48ce19c700 1 -- 172.21.15.97:0/2981917006 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c80b6920 con 0x7f48c8160d00 2023-07-06T03:53:21.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.543+0000 7f48cc999700 1 -- 172.21.15.97:0/2981917006 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115593821 0 0) 0x7f48c80cdae0 con 0x7f48c8160180 2023-07-06T03:53:21.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.544+0000 7f48ce19c700 1 --2- 172.21.15.97:0/2981917006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c8160d00 0x7f48c80b7400 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009e50 tx=0x7f48c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72573a5530121919 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:21.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.544+0000 7f48cc999700 1 -- 172.21.15.97:0/2981917006 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c000e070 con 0x7f48c8160d00 2023-07-06T03:53:21.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.544+0000 7f48cc999700 1 -- 172.21.15.97:0/2981917006 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c000b9c0 con 0x7f48c8160d00 2023-07-06T03:53:21.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.544+0000 7f48cc999700 1 -- 172.21.15.97:0/2981917006 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c000bd70 con 0x7f48c8160d00 2023-07-06T03:53:21.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.544+0000 7f48cfbff700 1 -- 172.21.15.97:0/2981917006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c8160d00 msgr2=0x7f48c80b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:21.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.544+0000 7f48cfbff700 1 --2- 172.21.15.97:0/2981917006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c8160d00 0x7f48c80b7400 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009e50 tx=0x7f48c000b330 comp rx=0 tx=0).stop 2023-07-06T03:53:21.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.544+0000 7f48cfbff700 1 -- 172.21.15.97:0/2981917006 shutdown_connections 2023-07-06T03:53:21.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.544+0000 7f48cfbff700 1 --2- 172.21.15.97:0/2981917006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c8160d00 0x7f48c80b7400 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.544+0000 7f48cfbff700 1 --2- 172.21.15.97:0/2981917006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c815c520 0x7f48c80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.544+0000 7f48cfbff700 1 -- 172.21.15.97:0/2981917006 >> 172.21.15.97:0/2981917006 conn(0x7f48c8071790 msgr2=0x7f48c8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:21.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.545+0000 7f48cfbff700 1 -- 172.21.15.97:0/2981917006 shutdown_connections 2023-07-06T03:53:21.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.545+0000 7f48cfbff700 1 -- 172.21.15.97:0/2981917006 wait complete. 2023-07-06T03:53:21.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.546+0000 7f48cfbff700 1 Processor -- start 2023-07-06T03:53:21.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.546+0000 7f48cfbff700 1 -- start start 2023-07-06T03:53:21.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.547+0000 7f48cfbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48c815c520 0x7f48c80c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:21.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.547+0000 7f48cfbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c8160180 0x7f48c815f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:21.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.547+0000 7f48cfbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c8160d00 0x7f48c80cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:21.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.547+0000 7f48cfbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80c9060 con 0x7f48c8160d00 2023-07-06T03:53:21.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.547+0000 7f48cfbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80d1020 con 0x7f48c815c520 2023-07-06T03:53:21.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.547+0000 7f48cfbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80d1300 con 0x7f48c8160180 2023-07-06T03:53:21.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.547+0000 7f48ce19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c8160d00 0x7f48c80cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:21.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48ce19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c8160d00 0x7f48c80cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38948/0 (socket says 172.21.15.97:38948) 2023-07-06T03:53:21.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48ce19c700 1 -- 172.21.15.97:0/4268754424 learned_addr learned my addr 172.21.15.97:0/4268754424 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:21.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48cd99b700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48c815c520 0x7f48c80c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:21.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48cd19a700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c8160180 0x7f48c815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:21.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48ce19c700 1 -- 172.21.15.97:0/4268754424 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c8160180 msgr2=0x7f48c815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:21.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48ce19c700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c8160180 0x7f48c815f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48ce19c700 1 -- 172.21.15.97:0/4268754424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48c815c520 msgr2=0x7f48c80c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:21.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48ce19c700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48c815c520 0x7f48c80c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48ce19c700 1 -- 172.21.15.97:0/4268754424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c0009b30 con 0x7f48c8160d00 2023-07-06T03:53:21.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48cd99b700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48c815c520 0x7f48c80c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:21.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48ce19c700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c8160d00 0x7f48c80cac70 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f48c000b730 tx=0x7f48c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:21.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48be7fc700 1 -- 172.21.15.97:0/4268754424 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c00099b0 con 0x7f48c8160d00 2023-07-06T03:53:21.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48be7fc700 1 -- 172.21.15.97:0/4268754424 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c00092e0 con 0x7f48c8160d00 2023-07-06T03:53:21.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.548+0000 7f48cfbff700 1 -- 172.21.15.97:0/4268754424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c80d15e0 con 0x7f48c8160d00 2023-07-06T03:53:21.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.549+0000 7f48be7fc700 1 -- 172.21.15.97:0/4268754424 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c001c630 con 0x7f48c8160d00 2023-07-06T03:53:21.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.549+0000 7f48cfbff700 1 -- 172.21.15.97:0/4268754424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c80d1ba0 con 0x7f48c8160d00 2023-07-06T03:53:21.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.550+0000 7f48cfbff700 1 -- 172.21.15.97:0/4268754424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c804e340 con 0x7f48c8160d00 2023-07-06T03:53:21.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.551+0000 7f48be7fc700 1 -- 172.21.15.97:0/4268754424 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f48c001c790 con 0x7f48c8160d00 2023-07-06T03:53:21.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.551+0000 7f48be7fc700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48ac06ace0 0x7f48ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:21.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.552+0000 7f48be7fc700 1 -- 172.21.15.97:0/4268754424 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f48c008ae90 con 0x7f48c8160d00 2023-07-06T03:53:21.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.552+0000 7f48cd99b700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48ac06ace0 0x7f48ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:21.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.552+0000 7f48cd99b700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48ac06ace0 0x7f48ac06d190 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f48c4007a50 tx=0x7f48c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:21.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.556+0000 7f48be7fc700 1 -- 172.21.15.97:0/4268754424 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f48c0056280 con 0x7f48c8160d00 2023-07-06T03:53:21.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.824+0000 7f48cfbff700 1 -- 172.21.15.97:0/4268754424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f48c80b4f40 con 0x7f48c8160d00 2023-07-06T03:53:21.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.825+0000 7f48be7fc700 1 -- 172.21.15.97:0/4268754424 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f48c008c090 con 0x7f48c8160d00 2023-07-06T03:53:21.826 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:21.827 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:53:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-07-06T03:53:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-07-06T03:53:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkczuvGRAAUKjfmMyKOTCYzOQmtf3n3A==", 2023-07-06T03:53:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:53:21.827 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:53:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:53:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-07-06T03:53:21.828 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkFHvUMBAA2WgMcUUf+Qi8T2ZjhfWaHA==", 2023-07-06T03:53:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:53:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:53:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:53:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-07-06T03:53:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZk0N4+JRAA3tjqoIBPpZ6C7TMmOOg7lw==", 2023-07-06T03:53:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:53:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:53:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:53:21.830 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-07-06T03:53:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkJnHUABAAwIMH2DmoHaQN3frJCgRiQw==", 2023-07-06T03:53:21.831 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:53:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:53:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:53:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.832 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-07-06T03:53:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCLOKZkShOIABAAzIBHDcNkmnDOLSY7Qdb+FA==", 2023-07-06T03:53:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:53:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:53:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-07-06T03:53:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCOOKZkq1t7AxAAwihuhimbjitZDeki1QuMug==", 2023-07-06T03:53:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:53:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:53:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.834 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-07-06T03:53:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCROKZkdYv4BhAApTHg/pmlC0lO7hVTkimEgw==", 2023-07-06T03:53:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:53:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:53:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-07-06T03:53:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCUOKZkqhuMChAAsV9xG6g1dofrpWt3F8tw5w==", 2023-07-06T03:53:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:53:21.836 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:53:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-07-06T03:53:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCaOKZk1Q5aIBAABU1+afwIihm+g4xylryz/A==", 2023-07-06T03:53:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.837 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:53:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:53:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-07-06T03:53:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCdOKZkRskIJBAA+821aoG98aEV7h4FTINQFA==", 2023-07-06T03:53:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:53:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:53:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-07-06T03:53:21.839 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCgOKZk9qnMJxAATcWG+a/9Hl1AuIL/TSrdEA==", 2023-07-06T03:53:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:53:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:53:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.840 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-07-06T03:53:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCjOKZk6wJ/KxAAWnZxt7h7OJhQ+bqdel4+iQ==", 2023-07-06T03:53:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:53:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:53:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-07-06T03:53:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg==", 2023-07-06T03:53:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:53:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-07-06T03:53:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-06T03:53:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-07-06T03:53:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkq4/pGRAAucHmR1eBA8MNMy3/l7tl0g==", 2023-07-06T03:53:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:53:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:53:21.843 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:53:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:53:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-07-06T03:53:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkhL5fEhAAWhw4IbCPqTO2sXPIcpTVHA==", 2023-07-06T03:53:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.844 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:53:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:53:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:53:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:53:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-07-06T03:53:21.845 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkO81dIRAAffjVyvsb/ygBtx8UrYQK0w==", 2023-07-06T03:53:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:53:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:53:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:53:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:53:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-07-06T03:53:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCFOKZk6uVlHRAA92RDhdDKW/q65SEOQMWgiA==", 2023-07-06T03:53:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:53:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:53:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:53:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.847 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-07-06T03:53:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkT4ALEhAAXwwkcodQU9meJA0tfseJCg==", 2023-07-06T03:53:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:53:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:53:21.848 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-07-06T03:53:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkfnoEChAAuRxqXqPI33dgWkBJoUMh3w==", 2023-07-06T03:53:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:53:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:53:21.849 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:53:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:53:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:21.850 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:53:21.850 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:53:21.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 -- 172.21.15.97:0/4268754424 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48ac06ace0 msgr2=0x7f48ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48ac06ace0 0x7f48ac06d190 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f48c4007a50 tx=0x7f48c4008040 comp rx=0 tx=0).stop 2023-07-06T03:53:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 -- 172.21.15.97:0/4268754424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c8160d00 msgr2=0x7f48c80cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c8160d00 0x7f48c80cac70 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f48c000b730 tx=0x7f48c0004ad0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 -- 172.21.15.97:0/4268754424 shutdown_connections 2023-07-06T03:53:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f48ac06ace0 0x7f48ac06d190 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48c8160d00 0x7f48c80cac70 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48c815c520 0x7f48c80c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 --2- 172.21.15.97:0/4268754424 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48c8160180 0x7f48c815f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 -- 172.21.15.97:0/4268754424 >> 172.21.15.97:0/4268754424 conn(0x7f48c8071790 msgr2=0x7f48c8071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 -- 172.21.15.97:0/4268754424 shutdown_connections 2023-07-06T03:53:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:21.827+0000 7f48cfbff700 1 -- 172.21.15.97:0/4268754424 wait complete. 2023-07-06T03:53:21.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:53:22.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.048+0000 7f1ce87ff700 1 Processor -- start 2023-07-06T03:53:22.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.048+0000 7f1ce87ff700 1 -- start start 2023-07-06T03:53:22.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.050+0000 7f1ce87ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ce0071a00 0x7f1ce006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.050+0000 7f1ce87ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce006ddb0 con 0x7f1ce0158960 2023-07-06T03:53:22.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.050+0000 7f1ce87ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce015d310 con 0x7f1ce015ce90 2023-07-06T03:53:22.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.050+0000 7f1ce87ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce006bbf0 con 0x7f1ce0071a00 2023-07-06T03:53:22.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.050+0000 7f1ce5d9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ce0071a00 0x7f1ce006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.050+0000 7f1ce659b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1ce0158960 0x7f1ce00712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36932/0 (socket says 172.21.15.97:36932) 2023-07-06T03:53:22.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.050+0000 7f1ce659b700 1 -- 172.21.15.97:0/16784087 learned_addr learned my addr 172.21.15.97:0/16784087 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:22.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.050+0000 7f1ce5d9a700 1 -- 172.21.15.97:0/16784087 >> v1:172.21.15.146:6789/0 conn(0x7f1ce015ce90 legacy=0x7f1ce0158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:22.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.050+0000 7f1ce5d9a700 1 -- 172.21.15.97:0/16784087 >> v1:172.21.15.97:6789/0 conn(0x7f1ce0158960 legacy=0x7f1ce00712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.050+0000 7f1ce5d9a700 1 -- 172.21.15.97:0/16784087 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce006bed0 con 0x7f1ce0071a00 2023-07-06T03:53:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.050+0000 7f1ce5599700 1 -- 172.21.15.97:0/16784087 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812490119 0 0) 0x7f1ce006ddb0 con 0x7f1ce0158960 2023-07-06T03:53:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.051+0000 7f1ce5d9a700 1 --2- 172.21.15.97:0/16784087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0071a00 0x7f1ce006d6f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009840 tx=0x7f1cd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1de0123e4c431fc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.051+0000 7f1ce5599700 1 -- 172.21.15.97:0/16784087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0004230 con 0x7f1ce0071a00 2023-07-06T03:53:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.051+0000 7f1ce5599700 1 -- 172.21.15.97:0/16784087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd0004390 con 0x7f1ce0071a00 2023-07-06T03:53:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.051+0000 7f1ce5599700 1 -- 172.21.15.97:0/16784087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00036a0 con 0x7f1ce0071a00 2023-07-06T03:53:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.051+0000 7f1ce87ff700 1 -- 172.21.15.97:0/16784087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0071a00 msgr2=0x7f1ce006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.051+0000 7f1ce87ff700 1 --2- 172.21.15.97:0/16784087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0071a00 0x7f1ce006d6f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009840 tx=0x7f1cd0009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.051+0000 7f1ce87ff700 1 -- 172.21.15.97:0/16784087 shutdown_connections 2023-07-06T03:53:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.051+0000 7f1ce87ff700 1 --2- 172.21.15.97:0/16784087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce0071a00 0x7f1ce006d6f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.051+0000 7f1ce87ff700 1 -- 172.21.15.97:0/16784087 >> 172.21.15.97:0/16784087 conn(0x7f1ce00ce8c0 msgr2=0x7f1ce00cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.051+0000 7f1ce87ff700 1 -- 172.21.15.97:0/16784087 shutdown_connections 2023-07-06T03:53:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.052+0000 7f1ce87ff700 1 -- 172.21.15.97:0/16784087 wait complete. 2023-07-06T03:53:22.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.053+0000 7f1ce87ff700 1 Processor -- start 2023-07-06T03:53:22.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.053+0000 7f1ce87ff700 1 -- start start 2023-07-06T03:53:22.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.054+0000 7f1ce87ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ce0071a00 0x7f1ce00b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.054+0000 7f1ce87ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce0158960 0x7f1ce0156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.054+0000 7f1ce87ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce015ce90 0x7f1ce00af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.054+0000 7f1ce87ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce0144370 con 0x7f1ce015ce90 2023-07-06T03:53:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.054+0000 7f1ce87ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce0144650 con 0x7f1ce0071a00 2023-07-06T03:53:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.054+0000 7f1ce87ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce0144930 con 0x7f1ce0158960 2023-07-06T03:53:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.054+0000 7f1ce6d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce015ce90 0x7f1ce00af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.054+0000 7f1ce6d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce015ce90 0x7f1ce00af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38964/0 (socket says 172.21.15.97:38964) 2023-07-06T03:53:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.054+0000 7f1ce6d9c700 1 -- 172.21.15.97:0/3527274716 learned_addr learned my addr 172.21.15.97:0/3527274716 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1ce659b700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ce0071a00 0x7f1ce00b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1ce5d9a700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce0158960 0x7f1ce0156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1ce6d9c700 1 -- 172.21.15.97:0/3527274716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce0158960 msgr2=0x7f1ce0156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1ce6d9c700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce0158960 0x7f1ce0156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1ce6d9c700 1 -- 172.21.15.97:0/3527274716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ce0071a00 msgr2=0x7f1ce00b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1ce6d9c700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ce0071a00 0x7f1ce00b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1ce6d9c700 1 -- 172.21.15.97:0/3527274716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd00094a0 con 0x7f1ce015ce90 2023-07-06T03:53:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1ce659b700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ce0071a00 0x7f1ce00b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1ce6d9c700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce015ce90 0x7f1ce00af800 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800cc10 tx=0x7f1cd800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:22.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1cd6ffd700 1 -- 172.21.15.97:0/3527274716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd800ab10 con 0x7f1ce015ce90 2023-07-06T03:53:22.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1cd6ffd700 1 -- 172.21.15.97:0/3527274716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd800ac70 con 0x7f1ce015ce90 2023-07-06T03:53:22.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1ce87ff700 1 -- 172.21.15.97:0/3527274716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce0144c10 con 0x7f1ce015ce90 2023-07-06T03:53:22.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1cd6ffd700 1 -- 172.21.15.97:0/3527274716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8012420 con 0x7f1ce015ce90 2023-07-06T03:53:22.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.055+0000 7f1ce87ff700 1 -- 172.21.15.97:0/3527274716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce00b3e30 con 0x7f1ce015ce90 2023-07-06T03:53:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.056+0000 7f1ce87ff700 1 -- 172.21.15.97:0/3527274716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce0145130 con 0x7f1ce015ce90 2023-07-06T03:53:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.057+0000 7f1cd6ffd700 1 -- 172.21.15.97:0/3527274716 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1cd8012580 con 0x7f1ce015ce90 2023-07-06T03:53:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.058+0000 7f1cd6ffd700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cc406ace0 0x7f1cc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.058+0000 7f1cd6ffd700 1 -- 172.21.15.97:0/3527274716 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1cd80894f0 con 0x7f1ce015ce90 2023-07-06T03:53:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.059+0000 7f1ce659b700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cc406ace0 0x7f1cc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.059+0000 7f1ce659b700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cc406ace0 0x7f1cc406d190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc000f60 tx=0x7f1cdc007ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:22.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.062+0000 7f1cd6ffd700 1 -- 172.21.15.97:0/3527274716 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1cd80548f0 con 0x7f1ce015ce90 2023-07-06T03:53:22.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.258+0000 7f1ce87ff700 1 -- 172.21.15.97:0/3527274716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ce0067830 con 0x7f1ce015ce90 2023-07-06T03:53:22.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.259+0000 7f1cd6ffd700 1 -- 172.21.15.97:0/3527274716 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+14666 (secure 0 0 0) 0x7f1cd8057f10 con 0x7f1ce015ce90 2023-07-06T03:53:22.261 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:22.262 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":95,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:53:21.332093+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:53:22.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 -- 172.21.15.97:0/3527274716 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cc406ace0 msgr2=0x7f1cc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cc406ace0 0x7f1cc406d190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc000f60 tx=0x7f1cdc007ce0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 -- 172.21.15.97:0/3527274716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce015ce90 msgr2=0x7f1ce00af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce015ce90 0x7f1ce00af800 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800cc10 tx=0x7f1cd800cf20 comp rx=0 tx=0).stop 2023-07-06T03:53:22.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 -- 172.21.15.97:0/3527274716 shutdown_connections 2023-07-06T03:53:22.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cc406ace0 0x7f1cc406d190 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce015ce90 0x7f1ce00af800 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ce0071a00 0x7f1ce00b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 --2- 172.21.15.97:0/3527274716 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ce0158960 0x7f1ce0156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 -- 172.21.15.97:0/3527274716 >> 172.21.15.97:0/3527274716 conn(0x7f1ce00ce8c0 msgr2=0x7f1ce00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 -- 172.21.15.97:0/3527274716 shutdown_connections 2023-07-06T03:53:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.262+0000 7f1ce87ff700 1 -- 172.21.15.97:0/3527274716 wait complete. 2023-07-06T03:53:22.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:53:22.282 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:53:22.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.480+0000 7fb9453ff700 1 Processor -- start 2023-07-06T03:53:22.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.480+0000 7fb9453ff700 1 -- start start 2023-07-06T03:53:22.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.481+0000 7fb9453ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb94006b360 0x7fb9400b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.481+0000 7fb9453ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9400cff90 con 0x7fb94015c610 2023-07-06T03:53:22.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.481+0000 7fb9453ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9400b5650 con 0x7fb9400d2210 2023-07-06T03:53:22.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.481+0000 7fb9453ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400d0780 con 0x7fb94006b360 2023-07-06T03:53:22.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.481+0000 7fb93effd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb94015c610 0x7fb9400b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36936/0 (socket says 172.21.15.97:36936) 2023-07-06T03:53:22.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.481+0000 7fb93effd700 1 -- 172.21.15.97:0/660495463 learned_addr learned my addr 172.21.15.97:0/660495463 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:22.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.482+0000 7fb93e7fc700 1 --2- 172.21.15.97:0/660495463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb94006b360 0x7fb9400b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.482+0000 7fb93dffb700 1 -- 172.21.15.97:0/660495463 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128758922 0 0) 0x7fb9400cff90 con 0x7fb94015c610 2023-07-06T03:53:22.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.482+0000 7fb93dffb700 1 -- 172.21.15.97:0/660495463 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9280035b0 con 0x7fb94015c610 2023-07-06T03:53:22.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.482+0000 7fb93e7fc700 1 -- 172.21.15.97:0/660495463 >> v1:172.21.15.146:6789/0 conn(0x7fb9400d2210 legacy=0x7fb9400d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.482+0000 7fb93dffb700 1 -- 172.21.15.97:0/660495463 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413972480 0 0) 0x7fb9400b5650 con 0x7fb9400d2210 2023-07-06T03:53:22.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.482+0000 7fb93e7fc700 1 -- 172.21.15.97:0/660495463 >> v1:172.21.15.97:6789/0 conn(0x7fb94015c610 legacy=0x7fb9400b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.482+0000 7fb93e7fc700 1 -- 172.21.15.97:0/660495463 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9400b5400 con 0x7fb94006b360 2023-07-06T03:53:22.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.482+0000 7fb93e7fc700 1 --2- 172.21.15.97:0/660495463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb94006b360 0x7fb9400b4ec0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb934009870 tx=0x7fb934009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5836924d6cd6d23 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:22.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.483+0000 7fb93dffb700 1 -- 172.21.15.97:0/660495463 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011830 con 0x7fb94006b360 2023-07-06T03:53:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.483+0000 7fb93dffb700 1 -- 172.21.15.97:0/660495463 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb934011990 con 0x7fb94006b360 2023-07-06T03:53:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.483+0000 7fb93dffb700 1 -- 172.21.15.97:0/660495463 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011d20 con 0x7fb94006b360 2023-07-06T03:53:22.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.483+0000 7fb9453ff700 1 -- 172.21.15.97:0/660495463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb94006b360 msgr2=0x7fb9400b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.483+0000 7fb9453ff700 1 --2- 172.21.15.97:0/660495463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb94006b360 0x7fb9400b4ec0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb934009870 tx=0x7fb934009b80 comp rx=0 tx=0).stop 2023-07-06T03:53:22.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.483+0000 7fb9453ff700 1 -- 172.21.15.97:0/660495463 shutdown_connections 2023-07-06T03:53:22.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.483+0000 7fb9453ff700 1 --2- 172.21.15.97:0/660495463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb94006b360 0x7fb9400b4ec0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.483+0000 7fb9453ff700 1 -- 172.21.15.97:0/660495463 >> 172.21.15.97:0/660495463 conn(0x7fb94006a6c0 msgr2=0x7fb94006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:22.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.483+0000 7fb9453ff700 1 -- 172.21.15.97:0/660495463 shutdown_connections 2023-07-06T03:53:22.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.483+0000 7fb9453ff700 1 -- 172.21.15.97:0/660495463 wait complete. 2023-07-06T03:53:22.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.485+0000 7fb9453ff700 1 Processor -- start 2023-07-06T03:53:22.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.485+0000 7fb9453ff700 1 -- start start 2023-07-06T03:53:22.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.486+0000 7fb9453ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb94006b360 0x7fb94015db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.486+0000 7fb9453ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9400d2210 0x7fb9400cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.486+0000 7fb9453ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb94015c610 0x7fb9400c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.486+0000 7fb9453ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94015e070 con 0x7fb9400d2210 2023-07-06T03:53:22.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.486+0000 7fb9453ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94015e410 con 0x7fb94006b360 2023-07-06T03:53:22.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.486+0000 7fb9453ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400ccb10 con 0x7fb94015c610 2023-07-06T03:53:22.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.486+0000 7fb93e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9400d2210 0x7fb9400cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.486+0000 7fb93e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9400d2210 0x7fb9400cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38970/0 (socket says 172.21.15.97:38970) 2023-07-06T03:53:22.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.486+0000 7fb93e7fc700 1 -- 172.21.15.97:0/4185828332 learned_addr learned my addr 172.21.15.97:0/4185828332 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:22.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.486+0000 7fb93effd700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb94006b360 0x7fb94015db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.486+0000 7fb93f7fe700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb94015c610 0x7fb9400c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.487+0000 7fb93e7fc700 1 -- 172.21.15.97:0/4185828332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb94015c610 msgr2=0x7fb9400c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.487+0000 7fb93e7fc700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb94015c610 0x7fb9400c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.487+0000 7fb93e7fc700 1 -- 172.21.15.97:0/4185828332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb94006b360 msgr2=0x7fb94015db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.487+0000 7fb93e7fc700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb94006b360 0x7fb94015db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.487+0000 7fb93e7fc700 1 -- 172.21.15.97:0/4185828332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9340094a0 con 0x7fb9400d2210 2023-07-06T03:53:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.487+0000 7fb93e7fc700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9400d2210 0x7fb9400cc5d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fb9340090b0 tx=0x7fb934004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.487+0000 7fb9277fe700 1 -- 172.21.15.97:0/4185828332 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93401a7e0 con 0x7fb9400d2210 2023-07-06T03:53:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.487+0000 7fb9277fe700 1 -- 172.21.15.97:0/4185828332 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93401a940 con 0x7fb9400d2210 2023-07-06T03:53:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.487+0000 7fb9453ff700 1 -- 172.21.15.97:0/4185828332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94015e660 con 0x7fb9400d2210 2023-07-06T03:53:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.487+0000 7fb9277fe700 1 -- 172.21.15.97:0/4185828332 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93401acd0 con 0x7fb9400d2210 2023-07-06T03:53:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.487+0000 7fb9453ff700 1 -- 172.21.15.97:0/4185828332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94015ec20 con 0x7fb9400d2210 2023-07-06T03:53:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.488+0000 7fb9453ff700 1 -- 172.21.15.97:0/4185828332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9400cf650 con 0x7fb9400d2210 2023-07-06T03:53:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.489+0000 7fb9277fe700 1 -- 172.21.15.97:0/4185828332 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb934022070 con 0x7fb9400d2210 2023-07-06T03:53:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.490+0000 7fb9277fe700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb92c06ace0 0x7fb92c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.490+0000 7fb9277fe700 1 -- 172.21.15.97:0/4185828332 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb93408a0c0 con 0x7fb9400d2210 2023-07-06T03:53:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.490+0000 7fb93effd700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb92c06ace0 0x7fb92c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.491+0000 7fb93effd700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb92c06ace0 0x7fb92c06d190 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb930007fd0 tx=0x7fb930009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:22.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.494+0000 7fb9277fe700 1 -- 172.21.15.97:0/4185828332 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb934055550 con 0x7fb9400d2210 2023-07-06T03:53:22.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.687+0000 7fb9453ff700 1 -- 172.21.15.97:0/4185828332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb94004e340 con 0x7fb9400d2210 2023-07-06T03:53:22.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.688+0000 7fb9277fe700 1 -- 172.21.15.97:0/4185828332 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb934058b70 con 0x7fb9400d2210 2023-07-06T03:53:22.690 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:53:22.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.690+0000 7fb9453ff700 1 -- 172.21.15.97:0/4185828332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb92c06ace0 msgr2=0x7fb92c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.690+0000 7fb9453ff700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb92c06ace0 0x7fb92c06d190 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb930007fd0 tx=0x7fb930009040 comp rx=0 tx=0).stop 2023-07-06T03:53:22.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.690+0000 7fb9453ff700 1 -- 172.21.15.97:0/4185828332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9400d2210 msgr2=0x7fb9400cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.690+0000 7fb9453ff700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9400d2210 0x7fb9400cc5d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fb9340090b0 tx=0x7fb934004750 comp rx=0 tx=0).stop 2023-07-06T03:53:22.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.690+0000 7fb9453ff700 1 -- 172.21.15.97:0/4185828332 shutdown_connections 2023-07-06T03:53:22.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.690+0000 7fb9453ff700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb92c06ace0 0x7fb92c06d190 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.690+0000 7fb9453ff700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb94015c610 0x7fb9400c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.690+0000 7fb9453ff700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb94006b360 0x7fb94015db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.690+0000 7fb9453ff700 1 --2- 172.21.15.97:0/4185828332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9400d2210 0x7fb9400cc5d0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.690+0000 7fb9453ff700 1 -- 172.21.15.97:0/4185828332 >> 172.21.15.97:0/4185828332 conn(0x7fb94006a6c0 msgr2=0x7fb9400cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:22.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.690+0000 7fb9453ff700 1 -- 172.21.15.97:0/4185828332 shutdown_connections 2023-07-06T03:53:22.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.691+0000 7fb9453ff700 1 -- 172.21.15.97:0/4185828332 wait complete. 2023-07-06T03:53:22.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:53:22.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.905+0000 7f1dfd5ff700 1 Processor -- start 2023-07-06T03:53:22.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.905+0000 7f1dfd5ff700 1 -- start start 2023-07-06T03:53:22.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.906+0000 7f1dfd5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1df815c610 0x7f1df80b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1dfd5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1df80d2210 0x7f1df80d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1dfd5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df80cff90 con 0x7f1df806b360 2023-07-06T03:53:22.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1dfd5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df80d0780 con 0x7f1df815c610 2023-07-06T03:53:22.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1dfd5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df815c9e0 con 0x7f1df80d2210 2023-07-06T03:53:22.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1df6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1df815c610 0x7f1df80b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1df6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1df815c610 0x7f1df80b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38972/0 (socket says 172.21.15.97:38972) 2023-07-06T03:53:22.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1df6ffd700 1 -- 172.21.15.97:0/3482511773 learned_addr learned my addr 172.21.15.97:0/3482511773 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:22.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1df77fe700 1 --2- 172.21.15.97:0/3482511773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1df80d2210 0x7f1df80d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1df6ffd700 1 -- 172.21.15.97:0/3482511773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1df80d2210 msgr2=0x7f1df80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:22.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1df6ffd700 1 --2- 172.21.15.97:0/3482511773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1df80d2210 0x7f1df80d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1df6ffd700 1 -- 172.21.15.97:0/3482511773 >> v1:172.21.15.97:6789/0 conn(0x7f1df806b360 legacy=0x7f1df80b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1df6ffd700 1 -- 172.21.15.97:0/3482511773 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df80b5650 con 0x7f1df815c610 2023-07-06T03:53:22.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.907+0000 7f1df67fc700 1 -- 172.21.15.97:0/3482511773 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118245720 0 0) 0x7f1df80cff90 con 0x7f1df806b360 2023-07-06T03:53:22.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.908+0000 7f1df6ffd700 1 --2- 172.21.15.97:0/3482511773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df815c610 0x7f1df80b47b0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f1de80098f0 tx=0x7f1de8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d735f210d11811 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:22.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.908+0000 7f1df67fc700 1 -- 172.21.15.97:0/3482511773 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de800b8a0 con 0x7f1df815c610 2023-07-06T03:53:22.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.908+0000 7f1df67fc700 1 -- 172.21.15.97:0/3482511773 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de800ba00 con 0x7f1df815c610 2023-07-06T03:53:22.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.908+0000 7f1df67fc700 1 -- 172.21.15.97:0/3482511773 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de800bd90 con 0x7f1df815c610 2023-07-06T03:53:22.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.908+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/3482511773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df815c610 msgr2=0x7f1df80b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.908+0000 7f1dfd5ff700 1 --2- 172.21.15.97:0/3482511773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df815c610 0x7f1df80b47b0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f1de80098f0 tx=0x7f1de8009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:22.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.908+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/3482511773 shutdown_connections 2023-07-06T03:53:22.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.908+0000 7f1dfd5ff700 1 --2- 172.21.15.97:0/3482511773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df815c610 0x7f1df80b47b0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.908+0000 7f1dfd5ff700 1 --2- 172.21.15.97:0/3482511773 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1df80d2210 0x7f1df80d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.908+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/3482511773 >> 172.21.15.97:0/3482511773 conn(0x7f1df806a6c0 msgr2=0x7f1df806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:22.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.908+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/3482511773 shutdown_connections 2023-07-06T03:53:22.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.909+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/3482511773 wait complete. 2023-07-06T03:53:22.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.910+0000 7f1dfd5ff700 1 Processor -- start 2023-07-06T03:53:22.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.910+0000 7f1dfd5ff700 1 -- start start 2023-07-06T03:53:22.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.911+0000 7f1dfd5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1df806b360 0x7f1df8157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.911+0000 7f1dfd5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1df80d2210 0x7f1df815ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.911+0000 7f1dfd5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df815c610 0x7f1df80cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.911+0000 7f1dfd5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df815cb20 con 0x7f1df815c610 2023-07-06T03:53:22.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.911+0000 7f1dfd5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df815cd30 con 0x7f1df80d2210 2023-07-06T03:53:22.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.911+0000 7f1dfd5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df815d010 con 0x7f1df806b360 2023-07-06T03:53:22.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1df77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df815c610 0x7f1df80cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1df77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df815c610 0x7f1df80cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38982/0 (socket says 172.21.15.97:38982) 2023-07-06T03:53:22.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1df77fe700 1 -- 172.21.15.97:0/2043505894 learned_addr learned my addr 172.21.15.97:0/2043505894 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:22.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1df6ffd700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1df806b360 0x7f1df8157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1deffff700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1df80d2210 0x7f1df815ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1df77fe700 1 -- 172.21.15.97:0/2043505894 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1df806b360 msgr2=0x7f1df8157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1df77fe700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1df806b360 0x7f1df8157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1df77fe700 1 -- 172.21.15.97:0/2043505894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1df80d2210 msgr2=0x7f1df815ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:22.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1df77fe700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1df80d2210 0x7f1df815ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:22.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1df77fe700 1 -- 172.21.15.97:0/2043505894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de8009550 con 0x7f1df815c610 2023-07-06T03:53:22.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1df77fe700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df815c610 0x7f1df80cc1d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f1df0007e40 tx=0x7f1df000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:22.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1def7fe700 1 -- 172.21.15.97:0/2043505894 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df000ed30 con 0x7f1df815c610 2023-07-06T03:53:22.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1def7fe700 1 -- 172.21.15.97:0/2043505894 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df0014420 con 0x7f1df815c610 2023-07-06T03:53:22.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/2043505894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df815d2f0 con 0x7f1df815c610 2023-07-06T03:53:22.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.912+0000 7f1def7fe700 1 -- 172.21.15.97:0/2043505894 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df00147d0 con 0x7f1df815c610 2023-07-06T03:53:22.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.913+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/2043505894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df815d8b0 con 0x7f1df815c610 2023-07-06T03:53:22.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.914+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/2043505894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df80ce600 con 0x7f1df815c610 2023-07-06T03:53:22.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.914+0000 7f1def7fe700 1 -- 172.21.15.97:0/2043505894 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1df0014930 con 0x7f1df815c610 2023-07-06T03:53:22.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.915+0000 7f1def7fe700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dd806ace0 0x7f1dd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:22.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.915+0000 7f1def7fe700 1 -- 172.21.15.97:0/2043505894 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1df00897c0 con 0x7f1df815c610 2023-07-06T03:53:22.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.916+0000 7f1df6ffd700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dd806ace0 0x7f1dd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:22.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.916+0000 7f1df6ffd700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dd806ace0 0x7f1dd806d190 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1de8014040 tx=0x7f1de8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:22.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:22.920+0000 7f1def7fe700 1 -- 172.21.15.97:0/2043505894 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1df0054bb0 con 0x7f1df815c610 2023-07-06T03:53:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.115+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/2043505894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1df804e340 con 0x7f1df815c610 2023-07-06T03:53:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.115+0000 7f1def7fe700 1 -- 172.21.15.97:0/2043505894 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v95) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1df00581d0 con 0x7f1df815c610 2023-07-06T03:53:23.117 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:53:23.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/2043505894 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dd806ace0 msgr2=0x7f1dd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dd806ace0 0x7f1dd806d190 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1de8014040 tx=0x7f1de8008030 comp rx=0 tx=0).stop 2023-07-06T03:53:23.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/2043505894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df815c610 msgr2=0x7f1df80cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df815c610 0x7f1df80cc1d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f1df0007e40 tx=0x7f1df000e3c0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/2043505894 shutdown_connections 2023-07-06T03:53:23.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1dd806ace0 0x7f1dd806d190 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1df815c610 0x7f1df80cc1d0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1df806b360 0x7f1df8157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 --2- 172.21.15.97:0/2043505894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1df80d2210 0x7f1df815ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/2043505894 >> 172.21.15.97:0/2043505894 conn(0x7f1df806a6c0 msgr2=0x7f1df80b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:23.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/2043505894 shutdown_connections 2023-07-06T03:53:23.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.118+0000 7f1dfd5ff700 1 -- 172.21.15.97:0/2043505894 wait complete. 2023-07-06T03:53:23.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T03:53:23.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.333+0000 7fb3d69ff700 1 Processor -- start 2023-07-06T03:53:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.333+0000 7fb3d69ff700 1 -- start start 2023-07-06T03:53:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.334+0000 7fb3d69ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb3d015c520 0x7fb3d00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.334+0000 7fb3d69ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3d0160d00 0x7fb3d00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.334+0000 7fb3d69ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d00cdae0 con 0x7fb3d0160180 2023-07-06T03:53:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.334+0000 7fb3d69ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d00cd8d0 con 0x7fb3d015c520 2023-07-06T03:53:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.334+0000 7fb3d69ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00b6640 con 0x7fb3d0160d00 2023-07-06T03:53:23.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.335+0000 7fb3d4f9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3d0160d00 0x7fb3d00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:23.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.335+0000 7fb3cffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb3d015c520 0x7fb3d00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:23.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.335+0000 7fb3cf7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb3d0160180 0x7fb3d01605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:58644/0 (socket says 172.21.15.97:58644) 2023-07-06T03:53:23.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.335+0000 7fb3cf7fe700 1 -- 172.21.15.97:0/3951433971 learned_addr learned my addr 172.21.15.97:0/3951433971 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:23.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.336+0000 7fb3cffff700 1 -- 172.21.15.97:0/3951433971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3d0160d00 msgr2=0x7fb3d00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:23.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.336+0000 7fb3cffff700 1 --2- 172.21.15.97:0/3951433971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3d0160d00 0x7fb3d00b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.336+0000 7fb3cffff700 1 -- 172.21.15.97:0/3951433971 >> v1:172.21.15.146:6789/0 conn(0x7fb3d0160180 legacy=0x7fb3d01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.336+0000 7fb3cffff700 1 -- 172.21.15.97:0/3951433971 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d00b6920 con 0x7fb3d015c520 2023-07-06T03:53:23.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.336+0000 7fb3ceffd700 1 -- 172.21.15.97:0/3951433971 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251852763 0 0) 0x7fb3d00cdae0 con 0x7fb3d0160180 2023-07-06T03:53:23.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.336+0000 7fb3cffff700 1 --2- 172.21.15.97:0/3951433971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d015c520 0x7fb3d00cd390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009840 tx=0x7fb3c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19242c574725f33c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:23.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.336+0000 7fb3ceffd700 1 -- 172.21.15.97:0/3951433971 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0011830 con 0x7fb3d015c520 2023-07-06T03:53:23.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.336+0000 7fb3ceffd700 1 -- 172.21.15.97:0/3951433971 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c0011990 con 0x7fb3d015c520 2023-07-06T03:53:23.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.336+0000 7fb3ceffd700 1 -- 172.21.15.97:0/3951433971 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0011d20 con 0x7fb3d015c520 2023-07-06T03:53:23.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.337+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3951433971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d015c520 msgr2=0x7fb3d00cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.337+0000 7fb3d69ff700 1 --2- 172.21.15.97:0/3951433971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d015c520 0x7fb3d00cd390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009840 tx=0x7fb3c0009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:23.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.337+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3951433971 shutdown_connections 2023-07-06T03:53:23.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.337+0000 7fb3d69ff700 1 --2- 172.21.15.97:0/3951433971 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3d0160d00 0x7fb3d00b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.337+0000 7fb3d69ff700 1 --2- 172.21.15.97:0/3951433971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d015c520 0x7fb3d00cd390 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.337+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3951433971 >> 172.21.15.97:0/3951433971 conn(0x7fb3d0071790 msgr2=0x7fb3d0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.337+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3951433971 shutdown_connections 2023-07-06T03:53:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.337+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3951433971 wait complete. 2023-07-06T03:53:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.339+0000 7fb3d69ff700 1 Processor -- start 2023-07-06T03:53:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.339+0000 7fb3d69ff700 1 -- start start 2023-07-06T03:53:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d69ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3d015c520 0x7fb3d00cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d69ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d0160180 0x7fb3d015e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d69ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0160d00 0x7fb3d014cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d69ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00cb5d0 con 0x7fb3d0160d00 2023-07-06T03:53:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d69ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00ca380 con 0x7fb3d0160180 2023-07-06T03:53:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d69ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00d0c40 con 0x7fb3d015c520 2023-07-06T03:53:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d4f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0160d00 0x7fb3d014cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d4f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0160d00 0x7fb3d014cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38988/0 (socket says 172.21.15.97:38988) 2023-07-06T03:53:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d4f9c700 1 -- 172.21.15.97:0/3699235406 learned_addr learned my addr 172.21.15.97:0/3699235406 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3cffff700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3d015c520 0x7fb3d00cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:23.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3cf7fe700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d0160180 0x7fb3d015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:23.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d4f9c700 1 -- 172.21.15.97:0/3699235406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3d015c520 msgr2=0x7fb3d00cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d4f9c700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3d015c520 0x7fb3d00cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d4f9c700 1 -- 172.21.15.97:0/3699235406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d0160180 msgr2=0x7fb3d015e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d4f9c700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d0160180 0x7fb3d015e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.340+0000 7fb3d4f9c700 1 -- 172.21.15.97:0/3699235406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c00094a0 con 0x7fb3d0160d00 2023-07-06T03:53:23.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.341+0000 7fb3d4f9c700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0160d00 0x7fb3d014cde0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800efc0 tx=0x7fb3c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:23.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.341+0000 7fb3ccff9700 1 -- 172.21.15.97:0/3699235406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8009ea0 con 0x7fb3d0160d00 2023-07-06T03:53:23.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.341+0000 7fb3ccff9700 1 -- 172.21.15.97:0/3699235406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c800f040 con 0x7fb3d0160d00 2023-07-06T03:53:23.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.341+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3699235406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d00d0f20 con 0x7fb3d0160d00 2023-07-06T03:53:23.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.341+0000 7fb3ccff9700 1 -- 172.21.15.97:0/3699235406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800ced0 con 0x7fb3d0160d00 2023-07-06T03:53:23.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.341+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3699235406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d00d14e0 con 0x7fb3d0160d00 2023-07-06T03:53:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.342+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3699235406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d004e340 con 0x7fb3d0160d00 2023-07-06T03:53:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.343+0000 7fb3ccff9700 1 -- 172.21.15.97:0/3699235406 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb3c8014480 con 0x7fb3d0160d00 2023-07-06T03:53:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.344+0000 7fb3ccff9700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3bc06ace0 0x7fb3bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.344+0000 7fb3ccff9700 1 -- 172.21.15.97:0/3699235406 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb3c80899d0 con 0x7fb3d0160d00 2023-07-06T03:53:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.344+0000 7fb3cffff700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3bc06ace0 0x7fb3bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.344+0000 7fb3cffff700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3bc06ace0 0x7fb3bc06d190 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00090b0 tx=0x7fb3c001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.348+0000 7fb3ccff9700 1 -- 172.21.15.97:0/3699235406 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb3c8054d30 con 0x7fb3d0160d00 2023-07-06T03:53:23.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.542+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3699235406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb3d00b4f40 con 0x7fb3d0160d00 2023-07-06T03:53:23.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.542+0000 7fb3ccff9700 1 -- 172.21.15.97:0/3699235406 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v95) v1 ==== 84+0+126 (secure 0 0 0) 0x7fb3c8018090 con 0x7fb3d0160d00 2023-07-06T03:53:23.544 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:23.545 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-07-06T03:53:23.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:23.545 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-07-06T03:53:23.545 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-07-06T03:53:23.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:23.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:23.546 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 2, 2023-07-06T03:53:23.546 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": "rbd" 2023-07-06T03:53:23.546 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:23.546 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-07-06T03:53:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3699235406 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3bc06ace0 msgr2=0x7fb3bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3bc06ace0 0x7fb3bc06d190 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00090b0 tx=0x7fb3c001c040 comp rx=0 tx=0).stop 2023-07-06T03:53:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3699235406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0160d00 msgr2=0x7fb3d014cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0160d00 0x7fb3d014cde0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800efc0 tx=0x7fb3c800c3a0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3699235406 shutdown_connections 2023-07-06T03:53:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3bc06ace0 0x7fb3bc06d190 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0160d00 0x7fb3d014cde0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3d015c520 0x7fb3d00cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 --2- 172.21.15.97:0/3699235406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3d0160180 0x7fb3d015e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3699235406 >> 172.21.15.97:0/3699235406 conn(0x7fb3d0071790 msgr2=0x7fb3d00b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3699235406 shutdown_connections 2023-07-06T03:53:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.545+0000 7fb3d69ff700 1 -- 172.21.15.97:0/3699235406 wait complete. 2023-07-06T03:53:23.564 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T03:53:23.565 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T03:53:23.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.763+0000 7fc9e6bff700 1 Processor -- start 2023-07-06T03:53:23.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.764+0000 7fc9e6bff700 1 -- start start 2023-07-06T03:53:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.765+0000 7fc9e6bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e00addc0 0x7fc9e006b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.765+0000 7fc9e6bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e00ae330 con 0x7fc9e015c6e0 2023-07-06T03:53:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.765+0000 7fc9e6bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e00ae610 con 0x7fc9e006b170 2023-07-06T03:53:23.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.765+0000 7fc9e6bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e006bb50 con 0x7fc9e00addc0 2023-07-06T03:53:23.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.766+0000 7fc9e499b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc9e015c6e0 0x7fc9e00af3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36956/0 (socket says 172.21.15.97:36956) 2023-07-06T03:53:23.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.766+0000 7fc9e499b700 1 -- 172.21.15.97:0/3392465630 learned_addr learned my addr 172.21.15.97:0/3392465630 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:23.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.766+0000 7fc9e519c700 1 --2- 172.21.15.97:0/3392465630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e00addc0 0x7fc9e006b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:23.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.766+0000 7fc9df7fe700 1 -- 172.21.15.97:0/3392465630 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100720989 0 0) 0x7fc9e00ae330 con 0x7fc9e015c6e0 2023-07-06T03:53:23.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.766+0000 7fc9df7fe700 1 -- 172.21.15.97:0/3392465630 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9d00035b0 con 0x7fc9e015c6e0 2023-07-06T03:53:23.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.766+0000 7fc9df7fe700 1 -- 172.21.15.97:0/3392465630 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648069557 0 0) 0x7fc9e00ae610 con 0x7fc9e006b170 2023-07-06T03:53:23.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.766+0000 7fc9df7fe700 1 -- 172.21.15.97:0/3392465630 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9e00ae330 con 0x7fc9e006b170 2023-07-06T03:53:23.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.766+0000 7fc9e519c700 1 -- 172.21.15.97:0/3392465630 >> v1:172.21.15.146:6789/0 conn(0x7fc9e006b170 legacy=0x7fc9e00ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.766+0000 7fc9e519c700 1 -- 172.21.15.97:0/3392465630 >> v1:172.21.15.97:6789/0 conn(0x7fc9e015c6e0 legacy=0x7fc9e00af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.766+0000 7fc9df7fe700 1 -- 172.21.15.97:0/3392465630 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 534766048 0 0) 0x7fc9d00035b0 con 0x7fc9e015c6e0 2023-07-06T03:53:23.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.766+0000 7fc9e519c700 1 -- 172.21.15.97:0/3392465630 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e00ae8f0 con 0x7fc9e00addc0 2023-07-06T03:53:23.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.767+0000 7fc9e519c700 1 --2- 172.21.15.97:0/3392465630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e00addc0 0x7fc9e006b610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009dc0 tx=0x7fc9d800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f448f517f28735f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:23.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.767+0000 7fc9df7fe700 1 -- 172.21.15.97:0/3392465630 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800e070 con 0x7fc9e00addc0 2023-07-06T03:53:23.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.767+0000 7fc9df7fe700 1 -- 172.21.15.97:0/3392465630 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d8003a20 con 0x7fc9e00addc0 2023-07-06T03:53:23.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.767+0000 7fc9df7fe700 1 -- 172.21.15.97:0/3392465630 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4001670 con 0x7fc9e00addc0 2023-07-06T03:53:23.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.767+0000 7fc9e6bff700 1 -- 172.21.15.97:0/3392465630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e00addc0 msgr2=0x7fc9e006b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.767+0000 7fc9e6bff700 1 --2- 172.21.15.97:0/3392465630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e00addc0 0x7fc9e006b610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009dc0 tx=0x7fc9d800b330 comp rx=0 tx=0).stop 2023-07-06T03:53:23.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.768+0000 7fc9e6bff700 1 -- 172.21.15.97:0/3392465630 shutdown_connections 2023-07-06T03:53:23.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.768+0000 7fc9e6bff700 1 --2- 172.21.15.97:0/3392465630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e00addc0 0x7fc9e006b610 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.768+0000 7fc9e6bff700 1 -- 172.21.15.97:0/3392465630 >> 172.21.15.97:0/3392465630 conn(0x7fc9e0099a00 msgr2=0x7fc9e0099e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:23.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.768+0000 7fc9e6bff700 1 -- 172.21.15.97:0/3392465630 shutdown_connections 2023-07-06T03:53:23.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.768+0000 7fc9e6bff700 1 -- 172.21.15.97:0/3392465630 wait complete. 2023-07-06T03:53:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.769+0000 7fc9e6bff700 1 Processor -- start 2023-07-06T03:53:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.769+0000 7fc9e6bff700 1 -- start start 2023-07-06T03:53:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.770+0000 7fc9e6bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e006b170 0x7fc9e00b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.770+0000 7fc9e6bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e00addc0 0x7fc9e00b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.770+0000 7fc9e6bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e015c6e0 0x7fc9e015af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:23.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.770+0000 7fc9e6bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00b5220 con 0x7fc9e006b170 2023-07-06T03:53:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.770+0000 7fc9e6bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00b5500 con 0x7fc9e015c6e0 2023-07-06T03:53:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.770+0000 7fc9e6bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00b57e0 con 0x7fc9e00addc0 2023-07-06T03:53:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9e499b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e006b170 0x7fc9e00b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9e499b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e006b170 0x7fc9e00b62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39004/0 (socket says 172.21.15.97:39004) 2023-07-06T03:53:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9e499b700 1 -- 172.21.15.97:0/220093733 learned_addr learned my addr 172.21.15.97:0/220093733 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9dffff700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e00addc0 0x7fc9e00b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:23.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9e519c700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e015c6e0 0x7fc9e015af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9e499b700 1 -- 172.21.15.97:0/220093733 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e00addc0 msgr2=0x7fc9e00b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9e499b700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e00addc0 0x7fc9e00b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9e499b700 1 -- 172.21.15.97:0/220093733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e015c6e0 msgr2=0x7fc9e015af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9e499b700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e015c6e0 0x7fc9e015af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9e499b700 1 -- 172.21.15.97:0/220093733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d8009a70 con 0x7fc9e006b170 2023-07-06T03:53:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9e499b700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e006b170 0x7fc9e00b62c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400c6b0 tx=0x7fc9d400ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:23.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9dd7fa700 1 -- 172.21.15.97:0/220093733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4004c80 con 0x7fc9e006b170 2023-07-06T03:53:23.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9dd7fa700 1 -- 172.21.15.97:0/220093733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d4004e00 con 0x7fc9e006b170 2023-07-06T03:53:23.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9e6bff700 1 -- 172.21.15.97:0/220093733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e00b5ac0 con 0x7fc9e006b170 2023-07-06T03:53:23.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.771+0000 7fc9dd7fa700 1 -- 172.21.15.97:0/220093733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4007310 con 0x7fc9e006b170 2023-07-06T03:53:23.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.772+0000 7fc9e6bff700 1 -- 172.21.15.97:0/220093733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e00b4e60 con 0x7fc9e006b170 2023-07-06T03:53:23.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.773+0000 7fc9e6bff700 1 -- 172.21.15.97:0/220093733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e004e340 con 0x7fc9e006b170 2023-07-06T03:53:23.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.773+0000 7fc9dd7fa700 1 -- 172.21.15.97:0/220093733 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc9d4007470 con 0x7fc9e006b170 2023-07-06T03:53:23.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.774+0000 7fc9dd7fa700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc9c406ad10 0x7fc9c406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:23.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.775+0000 7fc9dd7fa700 1 -- 172.21.15.97:0/220093733 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc9d4088e20 con 0x7fc9e006b170 2023-07-06T03:53:23.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.775+0000 7fc9dffff700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc9c406ad10 0x7fc9c406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:23.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.775+0000 7fc9dffff700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc9c406ad10 0x7fc9c406d1c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0011e0 tx=0x7fc9cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:23.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.779+0000 7fc9dd7fa700 1 -- 172.21.15.97:0/220093733 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc9d40542b0 con 0x7fc9e006b170 2023-07-06T03:53:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:23.973+0000 7fc9e6bff700 1 -- 172.21.15.97:0/220093733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc9e0071600 con 0x7fc9e006b170 2023-07-06T03:53:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.739+0000 7fc9dd7fa700 1 -- 172.21.15.97:0/220093733 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v96) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc9d40578d0 con 0x7fc9e006b170 2023-07-06T03:53:24.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.742+0000 7fc9e6bff700 1 -- 172.21.15.97:0/220093733 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc9c406ad10 msgr2=0x7fc9c406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:24.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.742+0000 7fc9e6bff700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc9c406ad10 0x7fc9c406d1c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0011e0 tx=0x7fc9cc009040 comp rx=0 tx=0).stop 2023-07-06T03:53:24.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.742+0000 7fc9e6bff700 1 -- 172.21.15.97:0/220093733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e006b170 msgr2=0x7fc9e00b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:24.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.742+0000 7fc9e6bff700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e006b170 0x7fc9e00b62c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400c6b0 tx=0x7fc9d400ca70 comp rx=0 tx=0).stop 2023-07-06T03:53:24.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.743+0000 7fc9e6bff700 1 -- 172.21.15.97:0/220093733 shutdown_connections 2023-07-06T03:53:24.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.743+0000 7fc9e6bff700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc9c406ad10 0x7fc9c406d1c0 secure :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0011e0 tx=0x7fc9cc009040 comp rx=0 tx=0).stop 2023-07-06T03:53:24.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.743+0000 7fc9e6bff700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e015c6e0 0x7fc9e015af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:24.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.743+0000 7fc9e6bff700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e006b170 0x7fc9e00b62c0 secure :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400c6b0 tx=0x7fc9d400ca70 comp rx=0 tx=0).stop 2023-07-06T03:53:24.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.743+0000 7fc9e6bff700 1 --2- 172.21.15.97:0/220093733 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc9e00addc0 0x7fc9e00b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:24.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.743+0000 7fc9e6bff700 1 -- 172.21.15.97:0/220093733 >> 172.21.15.97:0/220093733 conn(0x7fc9e0099a00 msgr2=0x7fc9e00b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:24.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.746+0000 7fc9e6bff700 1 -- 172.21.15.97:0/220093733 shutdown_connections 2023-07-06T03:53:24.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.747+0000 7fc9e6bff700 1 -- 172.21.15.97:0/220093733 wait complete. 2023-07-06T03:53:24.750 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-07-06T03:53:24.769 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T03:53:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.973+0000 7f2f593ff700 1 Processor -- start 2023-07-06T03:53:24.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.973+0000 7f2f593ff700 1 -- start start 2023-07-06T03:53:24.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.974+0000 7f2f593ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f540b8590 0x7f2f540cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:24.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.974+0000 7f2f593ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2f54160a50 0x7f2f540b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:24.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.974+0000 7f2f593ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f540b4cd0 0x7f2f5406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:24.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.974+0000 7f2f593ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f54160ef0 con 0x7f2f540b8590 2023-07-06T03:53:24.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.974+0000 7f2f593ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f540b5150 con 0x7f2f54160a50 2023-07-06T03:53:24.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.974+0000 7f2f593ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540d0c10 con 0x7f2f540b4cd0 2023-07-06T03:53:24.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.974+0000 7f2f52ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f540b8590 0x7f2f540cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:24.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.974+0000 7f2f52ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f540b8590 0x7f2f540cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39008/0 (socket says 172.21.15.97:39008) 2023-07-06T03:53:24.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.974+0000 7f2f52ffd700 1 -- 172.21.15.97:0/2164212749 learned_addr learned my addr 172.21.15.97:0/2164212749 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:24.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f537fe700 1 --2- 172.21.15.97:0/2164212749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f540b4cd0 0x7f2f5406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:24.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f527fc700 1 --2- 172.21.15.97:0/2164212749 >> v2:172.21.15.146:3300/0 conn(0x7f2f54160a50 0x7f2f540b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f52ffd700 1 -- 172.21.15.97:0/2164212749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f540b4cd0 msgr2=0x7f2f5406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f52ffd700 1 --2- 172.21.15.97:0/2164212749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f540b4cd0 0x7f2f5406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f52ffd700 1 -- 172.21.15.97:0/2164212749 >> v2:172.21.15.146:3300/0 conn(0x7f2f54160a50 msgr2=0x7f2f540b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f52ffd700 1 --2- 172.21.15.97:0/2164212749 >> v2:172.21.15.146:3300/0 conn(0x7f2f54160a50 0x7f2f540b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f52ffd700 1 -- 172.21.15.97:0/2164212749 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f540d08e0 con 0x7f2f540b8590 2023-07-06T03:53:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f527fc700 1 --2- 172.21.15.97:0/2164212749 >> v2:172.21.15.146:3300/0 conn(0x7f2f54160a50 0x7f2f540b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f52ffd700 1 --2- 172.21.15.97:0/2164212749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f540b8590 0x7f2f540cef60 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f2f540b1980 tx=0x7f2f44009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7d6a17d58d5cf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f51ffb700 1 -- 172.21.15.97:0/2164212749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44011830 con 0x7f2f540b8590 2023-07-06T03:53:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f51ffb700 1 -- 172.21.15.97:0/2164212749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f44011990 con 0x7f2f540b8590 2023-07-06T03:53:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.975+0000 7f2f51ffb700 1 -- 172.21.15.97:0/2164212749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44011d20 con 0x7f2f540b8590 2023-07-06T03:53:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.976+0000 7f2f593ff700 1 -- 172.21.15.97:0/2164212749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f540b8590 msgr2=0x7f2f540cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.976+0000 7f2f593ff700 1 --2- 172.21.15.97:0/2164212749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f540b8590 0x7f2f540cef60 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f2f540b1980 tx=0x7f2f44009bd0 comp rx=0 tx=0).stop 2023-07-06T03:53:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.976+0000 7f2f593ff700 1 -- 172.21.15.97:0/2164212749 shutdown_connections 2023-07-06T03:53:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.976+0000 7f2f593ff700 1 --2- 172.21.15.97:0/2164212749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f540b4cd0 0x7f2f5406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.976+0000 7f2f593ff700 1 --2- 172.21.15.97:0/2164212749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f540b8590 0x7f2f540cef60 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.976+0000 7f2f593ff700 1 --2- 172.21.15.97:0/2164212749 >> v2:172.21.15.146:3300/0 conn(0x7f2f54160a50 0x7f2f540b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.976+0000 7f2f593ff700 1 -- 172.21.15.97:0/2164212749 >> 172.21.15.97:0/2164212749 conn(0x7f2f5406b290 msgr2=0x7f2f540cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.976+0000 7f2f593ff700 1 -- 172.21.15.97:0/2164212749 shutdown_connections 2023-07-06T03:53:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.976+0000 7f2f593ff700 1 -- 172.21.15.97:0/2164212749 wait complete. 2023-07-06T03:53:24.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.978+0000 7f2f593ff700 1 Processor -- start 2023-07-06T03:53:24.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.978+0000 7f2f593ff700 1 -- start start 2023-07-06T03:53:24.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.979+0000 7f2f593ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f540b4cd0 0x7f2f54159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:24.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.979+0000 7f2f593ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f540b8590 0x7f2f540aa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:24.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.979+0000 7f2f593ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f54160a50 0x7f2f540b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:24.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.979+0000 7f2f593ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540cc1a0 con 0x7f2f540b8590 2023-07-06T03:53:24.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.979+0000 7f2f593ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540ccfe0 con 0x7f2f54160a50 2023-07-06T03:53:24.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.979+0000 7f2f593ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5415a310 con 0x7f2f540b4cd0 2023-07-06T03:53:24.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.979+0000 7f2f527fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f540b8590 0x7f2f540aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:24.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.979+0000 7f2f527fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f540b8590 0x7f2f540aa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39010/0 (socket says 172.21.15.97:39010) 2023-07-06T03:53:24.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.979+0000 7f2f527fc700 1 -- 172.21.15.97:0/2041591992 learned_addr learned my addr 172.21.15.97:0/2041591992 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:24.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.979+0000 7f2f537fe700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f54160a50 0x7f2f540b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.979+0000 7f2f52ffd700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f540b4cd0 0x7f2f54159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f527fc700 1 -- 172.21.15.97:0/2041591992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f540b4cd0 msgr2=0x7f2f54159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f527fc700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f540b4cd0 0x7f2f54159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f527fc700 1 -- 172.21.15.97:0/2041591992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f54160a50 msgr2=0x7f2f540b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f527fc700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f54160a50 0x7f2f540b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f527fc700 1 -- 172.21.15.97:0/2041591992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f44009550 con 0x7f2f540b8590 2023-07-06T03:53:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f52ffd700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f540b4cd0 0x7f2f54159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f527fc700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f540b8590 0x7f2f540aa090 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800e810 tx=0x7f2f4800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f3b7fe700 1 -- 172.21.15.97:0/2041591992 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4800c020 con 0x7f2f540b8590 2023-07-06T03:53:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f3b7fe700 1 -- 172.21.15.97:0/2041591992 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48004250 con 0x7f2f540b8590 2023-07-06T03:53:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f3b7fe700 1 -- 172.21.15.97:0/2041591992 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f480036a0 con 0x7f2f540b8590 2023-07-06T03:53:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f593ff700 1 -- 172.21.15.97:0/2041591992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f540a9c30 con 0x7f2f540b8590 2023-07-06T03:53:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.980+0000 7f2f593ff700 1 -- 172.21.15.97:0/2041591992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f540cbae0 con 0x7f2f540b8590 2023-07-06T03:53:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.981+0000 7f2f593ff700 1 -- 172.21.15.97:0/2041591992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5404e340 con 0x7f2f540b8590 2023-07-06T03:53:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.983+0000 7f2f3b7fe700 1 -- 172.21.15.97:0/2041591992 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f48003800 con 0x7f2f540b8590 2023-07-06T03:53:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.984+0000 7f2f3b7fe700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4006ace0 0x7f2f4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.984+0000 7f2f3b7fe700 1 -- 172.21.15.97:0/2041591992 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f2f48089580 con 0x7f2f540b8590 2023-07-06T03:53:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.984+0000 7f2f52ffd700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4006ace0 0x7f2f4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.984+0000 7f2f52ffd700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4006ace0 0x7f2f4006d190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2f44014040 tx=0x7f2f44008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:24.987+0000 7f2f3b7fe700 1 -- 172.21.15.97:0/2041591992 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2f48054880 con 0x7f2f540b8590 2023-07-06T03:53:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.183+0000 7f2f593ff700 1 -- 172.21.15.97:0/2041591992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2f540d00c0 con 0x7f2f540b8590 2023-07-06T03:53:25.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.745+0000 7f2f3b7fe700 1 -- 172.21.15.97:0/2041591992 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v97) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2f48057ea0 con 0x7f2f540b8590 2023-07-06T03:53:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.748+0000 7f2f593ff700 1 -- 172.21.15.97:0/2041591992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4006ace0 msgr2=0x7f2f4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.748+0000 7f2f593ff700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4006ace0 0x7f2f4006d190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2f44014040 tx=0x7f2f44008030 comp rx=0 tx=0).stop 2023-07-06T03:53:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.748+0000 7f2f593ff700 1 -- 172.21.15.97:0/2041591992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f540b8590 msgr2=0x7f2f540aa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.748+0000 7f2f593ff700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f540b8590 0x7f2f540aa090 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800e810 tx=0x7f2f4800eb20 comp rx=0 tx=0).stop 2023-07-06T03:53:25.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.751+0000 7f2f593ff700 1 -- 172.21.15.97:0/2041591992 shutdown_connections 2023-07-06T03:53:25.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.751+0000 7f2f593ff700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2f4006ace0 0x7f2f4006d190 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:25.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.751+0000 7f2f593ff700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f54160a50 0x7f2f540b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:25.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.751+0000 7f2f593ff700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2f540b4cd0 0x7f2f54159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:25.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.751+0000 7f2f593ff700 1 --2- 172.21.15.97:0/2041591992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f540b8590 0x7f2f540aa090 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:25.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.751+0000 7f2f593ff700 1 -- 172.21.15.97:0/2041591992 >> 172.21.15.97:0/2041591992 conn(0x7f2f5406b290 msgr2=0x7f2f540b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:25.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.753+0000 7f2f593ff700 1 -- 172.21.15.97:0/2041591992 shutdown_connections 2023-07-06T03:53:25.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.755+0000 7f2f593ff700 1 -- 172.21.15.97:0/2041591992 wait complete. 2023-07-06T03:53:25.765 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-07-06T03:53:25.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T03:53:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.980+0000 7fdd699ff700 1 Processor -- start 2023-07-06T03:53:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.981+0000 7fdd699ff700 1 -- start start 2023-07-06T03:53:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.982+0000 7fdd699ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd6406b360 0x7fdd640b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:25.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.982+0000 7fdd699ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d2210 0x7fdd640d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:25.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.982+0000 7fdd699ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd640cff90 con 0x7fdd6415c610 2023-07-06T03:53:25.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.982+0000 7fdd699ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd640d0780 con 0x7fdd6406b360 2023-07-06T03:53:25.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.982+0000 7fdd699ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6415c9e0 con 0x7fdd640d2210 2023-07-06T03:53:25.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.982+0000 7fdd5bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd6406b360 0x7fdd640b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:25.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.982+0000 7fdd5bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd6406b360 0x7fdd640b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39026/0 (socket says 172.21.15.97:39026) 2023-07-06T03:53:25.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.982+0000 7fdd5bfff700 1 -- 172.21.15.97:0/2808501767 learned_addr learned my addr 172.21.15.97:0/2808501767 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:25.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.983+0000 7fdd637fe700 1 --2- 172.21.15.97:0/2808501767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d2210 0x7fdd640d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:25.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.983+0000 7fdd5bfff700 1 -- 172.21.15.97:0/2808501767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d2210 msgr2=0x7fdd640d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:25.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.983+0000 7fdd5bfff700 1 --2- 172.21.15.97:0/2808501767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d2210 0x7fdd640d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:25.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.983+0000 7fdd5bfff700 1 -- 172.21.15.97:0/2808501767 >> v1:172.21.15.146:6789/0 conn(0x7fdd6415c610 legacy=0x7fdd640b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:25.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.983+0000 7fdd5bfff700 1 -- 172.21.15.97:0/2808501767 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640b5650 con 0x7fdd6406b360 2023-07-06T03:53:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.983+0000 7fdd637fe700 1 --2- 172.21.15.97:0/2808501767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d2210 0x7fdd640d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.983+0000 7fdd5bfff700 1 --2- 172.21.15.97:0/2808501767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6406b360 0x7fdd640b4ec0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0098f0 tx=0x7fdd4c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d985312809e583 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.983+0000 7fdd627fc700 1 -- 172.21.15.97:0/2808501767 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004230 con 0x7fdd6406b360 2023-07-06T03:53:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.983+0000 7fdd627fc700 1 -- 172.21.15.97:0/2808501767 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c004390 con 0x7fdd6406b360 2023-07-06T03:53:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.983+0000 7fdd627fc700 1 -- 172.21.15.97:0/2808501767 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0036a0 con 0x7fdd6406b360 2023-07-06T03:53:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.984+0000 7fdd699ff700 1 -- 172.21.15.97:0/2808501767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6406b360 msgr2=0x7fdd640b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.984+0000 7fdd699ff700 1 --2- 172.21.15.97:0/2808501767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6406b360 0x7fdd640b4ec0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0098f0 tx=0x7fdd4c009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.984+0000 7fdd699ff700 1 -- 172.21.15.97:0/2808501767 shutdown_connections 2023-07-06T03:53:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.984+0000 7fdd699ff700 1 --2- 172.21.15.97:0/2808501767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d2210 0x7fdd640d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.984+0000 7fdd699ff700 1 --2- 172.21.15.97:0/2808501767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6406b360 0x7fdd640b4ec0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.984+0000 7fdd699ff700 1 -- 172.21.15.97:0/2808501767 >> 172.21.15.97:0/2808501767 conn(0x7fdd6406a6c0 msgr2=0x7fdd6406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.984+0000 7fdd699ff700 1 -- 172.21.15.97:0/2808501767 shutdown_connections 2023-07-06T03:53:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.984+0000 7fdd699ff700 1 -- 172.21.15.97:0/2808501767 wait complete. 2023-07-06T03:53:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.986+0000 7fdd699ff700 1 Processor -- start 2023-07-06T03:53:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.986+0000 7fdd699ff700 1 -- start start 2023-07-06T03:53:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd699ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6406b360 0x7fdd640cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd699ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d2210 0x7fdd640ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd699ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6415c610 0x7fdd64158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd699ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd64159150 con 0x7fdd6406b360 2023-07-06T03:53:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd699ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd640cae50 con 0x7fdd6415c610 2023-07-06T03:53:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd699ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd640cb130 con 0x7fdd640d2210 2023-07-06T03:53:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd62ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6406b360 0x7fdd640cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd62ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6406b360 0x7fdd640cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39040/0 (socket says 172.21.15.97:39040) 2023-07-06T03:53:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd62ffd700 1 -- 172.21.15.97:0/2063475435 learned_addr learned my addr 172.21.15.97:0/2063475435 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd62ffd700 1 -- 172.21.15.97:0/2063475435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d2210 msgr2=0x7fdd640ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd62ffd700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d2210 0x7fdd640ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd62ffd700 1 -- 172.21.15.97:0/2063475435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6415c610 msgr2=0x7fdd64158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd62ffd700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6415c610 0x7fdd64158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.987+0000 7fdd62ffd700 1 -- 172.21.15.97:0/2063475435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c009550 con 0x7fdd6406b360 2023-07-06T03:53:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.988+0000 7fdd62ffd700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6406b360 0x7fdd640cbef0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400b4a0 tx=0x7fdd5400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.988+0000 7fdd60ff9700 1 -- 172.21.15.97:0/2063475435 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54011830 con 0x7fdd6406b360 2023-07-06T03:53:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.988+0000 7fdd60ff9700 1 -- 172.21.15.97:0/2063475435 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd54011990 con 0x7fdd6406b360 2023-07-06T03:53:25.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.988+0000 7fdd60ff9700 1 -- 172.21.15.97:0/2063475435 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54011d40 con 0x7fdd6406b360 2023-07-06T03:53:25.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.988+0000 7fdd699ff700 1 -- 172.21.15.97:0/2063475435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd640cb410 con 0x7fdd6406b360 2023-07-06T03:53:25.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.988+0000 7fdd699ff700 1 -- 172.21.15.97:0/2063475435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd640cb9d0 con 0x7fdd6406b360 2023-07-06T03:53:25.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.989+0000 7fdd699ff700 1 -- 172.21.15.97:0/2063475435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6404e340 con 0x7fdd6406b360 2023-07-06T03:53:25.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.990+0000 7fdd60ff9700 1 -- 172.21.15.97:0/2063475435 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd54004020 con 0x7fdd6406b360 2023-07-06T03:53:25.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.991+0000 7fdd60ff9700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4406ace0 0x7fdd4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:25.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.991+0000 7fdd60ff9700 1 -- 172.21.15.97:0/2063475435 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fdd540896c0 con 0x7fdd6406b360 2023-07-06T03:53:25.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.992+0000 7fdd5bfff700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4406ace0 0x7fdd4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:25.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.992+0000 7fdd5bfff700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4406ace0 0x7fdd4406d190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdd640ccde0 tx=0x7fdd4c010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:25.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:25.995+0000 7fdd60ff9700 1 -- 172.21.15.97:0/2063475435 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdd540547b0 con 0x7fdd6406b360 2023-07-06T03:53:26.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.232+0000 7fdd699ff700 1 -- 172.21.15.97:0/2063475435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdd64066010 con 0x7fdd6406b360 2023-07-06T03:53:26.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.750+0000 7fdd60ff9700 1 -- 172.21.15.97:0/2063475435 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdd54057dd0 con 0x7fdd6406b360 2023-07-06T03:53:26.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.757+0000 7fdd699ff700 1 -- 172.21.15.97:0/2063475435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4406ace0 msgr2=0x7fdd4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:26.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.757+0000 7fdd699ff700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4406ace0 0x7fdd4406d190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdd640ccde0 tx=0x7fdd4c010000 comp rx=0 tx=0).stop 2023-07-06T03:53:26.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.757+0000 7fdd699ff700 1 -- 172.21.15.97:0/2063475435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6406b360 msgr2=0x7fdd640cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:26.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.757+0000 7fdd699ff700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6406b360 0x7fdd640cbef0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400b4a0 tx=0x7fdd5400b860 comp rx=0 tx=0).stop 2023-07-06T03:53:26.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.762+0000 7fdd699ff700 1 -- 172.21.15.97:0/2063475435 shutdown_connections 2023-07-06T03:53:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.762+0000 7fdd699ff700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd4406ace0 0x7fdd4406d190 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.762+0000 7fdd699ff700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd6415c610 0x7fdd64158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.762+0000 7fdd699ff700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd6406b360 0x7fdd640cbef0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.762+0000 7fdd699ff700 1 --2- 172.21.15.97:0/2063475435 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd640d2210 0x7fdd640ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.762+0000 7fdd699ff700 1 -- 172.21.15.97:0/2063475435 >> 172.21.15.97:0/2063475435 conn(0x7fdd6406a6c0 msgr2=0x7fdd640b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:26.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.766+0000 7fdd699ff700 1 -- 172.21.15.97:0/2063475435 shutdown_connections 2023-07-06T03:53:26.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.767+0000 7fdd699ff700 1 -- 172.21.15.97:0/2063475435 wait complete. 2023-07-06T03:53:26.778 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 15 and data pool 16 2023-07-06T03:53:26.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:53:26.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.992+0000 7f99985ff700 1 Processor -- start 2023-07-06T03:53:26.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.992+0000 7f99985ff700 1 -- start start 2023-07-06T03:53:26.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.994+0000 7f99985ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f99900cd8e0 0x7f999015e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:26.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.994+0000 7f99985ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9990070c30 0x7f99900adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:26.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.994+0000 7f99985ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99900710a0 con 0x7f99900cdcb0 2023-07-06T03:53:26.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.994+0000 7f99985ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99900ae3e0 con 0x7f99900cd8e0 2023-07-06T03:53:26.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.994+0000 7f99985ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9990071330 con 0x7f9990070c30 2023-07-06T03:53:26.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.994+0000 7f9996b9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f99900cdcb0 0x7f999006f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36958/0 (socket says 172.21.15.97:36958) 2023-07-06T03:53:26.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.994+0000 7f9996b9c700 1 -- 172.21.15.97:0/1412732331 learned_addr learned my addr 172.21.15.97:0/1412732331 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:26.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.994+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938908696 0 0) 0x7f99900710a0 con 0x7f99900cdcb0 2023-07-06T03:53:26.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.994+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f997c0035b0 con 0x7f99900cdcb0 2023-07-06T03:53:26.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.994+0000 7f9995b9a700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9990070c30 0x7f99900adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:26.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.994+0000 7f999639b700 1 --2- 172.21.15.97:0/1412732331 >> v2:172.21.15.146:3300/0 conn(0x7f99900cd8e0 0x7f999015e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:26.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967412347 0 0) 0x7f997c0035b0 con 0x7f99900cdcb0 2023-07-06T03:53:26.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9990070c30 msgr2=0x7f99900adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:26.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9990070c30 0x7f99900adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 >> v2:172.21.15.146:3300/0 conn(0x7f99900cd8e0 msgr2=0x7f999015e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:27.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 --2- 172.21.15.97:0/1412732331 >> v2:172.21.15.146:3300/0 conn(0x7f99900cd8e0 0x7f999015e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9990071610 con 0x7f99900cdcb0 2023-07-06T03:53:27.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f99880019f0 con 0x7f99900cdcb0 2023-07-06T03:53:27.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 >> v1:172.21.15.97:6789/0 conn(0x7f99900cdcb0 legacy=0x7f999006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f997c003a00 0x7f997c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c004430 0x7f997c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9990070c30 0x7f997c004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99900ae3e0 con 0x7f997c004430 2023-07-06T03:53:27.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9990071330 con 0x7f997c003a00 2023-07-06T03:53:27.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c00ba10 con 0x7f9990070c30 2023-07-06T03:53:27.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9995b9a700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c004430 0x7f997c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f999639b700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f997c003a00 0x7f997c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.995+0000 7f9996b9c700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9990070c30 0x7f997c004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.996+0000 7f9995b9a700 1 -- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9990070c30 msgr2=0x7f997c004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:27.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.996+0000 7f9995b9a700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9990070c30 0x7f997c004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.996+0000 7f9995b9a700 1 -- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f997c003a00 msgr2=0x7f997c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:27.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.996+0000 7f9995b9a700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f997c003a00 0x7f997c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.996+0000 7f9995b9a700 1 -- 172.21.15.97:0/1412732331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c0035b0 con 0x7f997c004430 2023-07-06T03:53:27.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.996+0000 7f9996b9c700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9990070c30 0x7f997c004fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:27.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.996+0000 7f9995b9a700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c004430 0x7f997c0048f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f99800096f0 tx=0x7f997c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea45400127f0854 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:27.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.996+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000e6c0 con 0x7f997c004430 2023-07-06T03:53:27.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.996+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998000e820 con 0x7f997c004430 2023-07-06T03:53:27.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.996+0000 7f9995399700 1 -- 172.21.15.97:0/1412732331 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000ebb0 con 0x7f997c004430 2023-07-06T03:53:27.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.997+0000 7f99985ff700 1 -- 172.21.15.97:0/1412732331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c004430 msgr2=0x7f997c0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.997+0000 7f99985ff700 1 --2- 172.21.15.97:0/1412732331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c004430 0x7f997c0048f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f99800096f0 tx=0x7f997c005880 comp rx=0 tx=0).stop 2023-07-06T03:53:27.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.997+0000 7f9996b9c700 1 -- 172.21.15.97:0/1412732331 reap_dead start 2023-07-06T03:53:27.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.997+0000 7f99985ff700 1 -- 172.21.15.97:0/1412732331 shutdown_connections 2023-07-06T03:53:27.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.997+0000 7f99985ff700 1 -- 172.21.15.97:0/1412732331 >> 172.21.15.97:0/1412732331 conn(0x7f99900ce8c0 msgr2=0x7f99900cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:27.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.997+0000 7f99985ff700 1 -- 172.21.15.97:0/1412732331 shutdown_connections 2023-07-06T03:53:27.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.997+0000 7f99985ff700 1 -- 172.21.15.97:0/1412732331 wait complete. 2023-07-06T03:53:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.999+0000 7f99985ff700 1 Processor -- start 2023-07-06T03:53:27.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:26.999+0000 7f99985ff700 1 -- start start 2023-07-06T03:53:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f99985ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f997c004430 0x7f999014d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f99985ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99900a8460 0x7f99900a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f99985ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99900a88d0 0x7f99900aa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f99985ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999014de60 con 0x7f99900a8460 2023-07-06T03:53:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f99985ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99900ad0f0 con 0x7f997c004430 2023-07-06T03:53:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f99985ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99900acb30 con 0x7f99900a88d0 2023-07-06T03:53:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f9995b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99900a8460 0x7f99900a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f9995b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99900a8460 0x7f99900a9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39056/0 (socket says 172.21.15.97:39056) 2023-07-06T03:53:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f9995b9a700 1 -- 172.21.15.97:0/3975761201 learned_addr learned my addr 172.21.15.97:0/3975761201 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f9996b9c700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99900a88d0 0x7f99900aa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f999639b700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f997c004430 0x7f999014d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f9995b9a700 1 -- 172.21.15.97:0/3975761201 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99900a88d0 msgr2=0x7f99900aa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f9995b9a700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99900a88d0 0x7f99900aa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f9995b9a700 1 -- 172.21.15.97:0/3975761201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f997c004430 msgr2=0x7f999014d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f9995b9a700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f997c004430 0x7f999014d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f9995b9a700 1 -- 172.21.15.97:0/3975761201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99900ace10 con 0x7f99900a8460 2023-07-06T03:53:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f999639b700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f997c004430 0x7f999014d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.000+0000 7f9996b9c700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99900a88d0 0x7f99900aa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.001+0000 7f9995b9a700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99900a8460 0x7f99900a9d10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f99800090b0 tx=0x7f99800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.001+0000 7f9986ffd700 1 -- 172.21.15.97:0/3975761201 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000e6c0 con 0x7f99900a8460 2023-07-06T03:53:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.001+0000 7f9986ffd700 1 -- 172.21.15.97:0/3975761201 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99800038c0 con 0x7f99900a8460 2023-07-06T03:53:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.001+0000 7f9986ffd700 1 -- 172.21.15.97:0/3975761201 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9980006e60 con 0x7f99900a8460 2023-07-06T03:53:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.001+0000 7f99985ff700 1 -- 172.21.15.97:0/3975761201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9990154c50 con 0x7f99900a8460 2023-07-06T03:53:27.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.001+0000 7f99985ff700 1 -- 172.21.15.97:0/3975761201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9990155450 con 0x7f99900a8460 2023-07-06T03:53:27.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.002+0000 7f99985ff700 1 -- 172.21.15.97:0/3975761201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9990070aa0 con 0x7f99900a8460 2023-07-06T03:53:27.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.003+0000 7f9986ffd700 1 -- 172.21.15.97:0/3975761201 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9980022040 con 0x7f99900a8460 2023-07-06T03:53:27.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.004+0000 7f9986ffd700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f997406ace0 0x7f997406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.004+0000 7f9986ffd700 1 -- 172.21.15.97:0/3975761201 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f998008de80 con 0x7f99900a8460 2023-07-06T03:53:27.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.005+0000 7f999639b700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f997406ace0 0x7f997406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.005+0000 7f999639b700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f997406ace0 0x7f997406d190 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f998c009bd0 tx=0x7f998c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:27.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.009+0000 7f9986ffd700 1 -- 172.21.15.97:0/3975761201 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9980058fc0 con 0x7f99900a8460 2023-07-06T03:53:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.204+0000 7f99985ff700 1 -- 172.21.15.97:0/3975761201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f999004e340 con 0x7f99900a8460 2023-07-06T03:53:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.205+0000 7f9986ffd700 1 -- 172.21.15.97:0/3975761201 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+18337 (secure 0 0 0) 0x7f998005c5e0 con 0x7f99900a8460 2023-07-06T03:53:27.207 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:27.207 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":98,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:53:26.233731+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:53:24.013743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-06T03:53:25.222178+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:53:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.207+0000 7f99985ff700 1 -- 172.21.15.97:0/3975761201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f997406ace0 msgr2=0x7f997406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.207+0000 7f99985ff700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f997406ace0 0x7f997406d190 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f998c009bd0 tx=0x7f998c008040 comp rx=0 tx=0).stop 2023-07-06T03:53:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.207+0000 7f99985ff700 1 -- 172.21.15.97:0/3975761201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99900a8460 msgr2=0x7f99900a9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.207+0000 7f99985ff700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99900a8460 0x7f99900a9d10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f99800090b0 tx=0x7f99800047c0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.207+0000 7f99985ff700 1 -- 172.21.15.97:0/3975761201 shutdown_connections 2023-07-06T03:53:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.207+0000 7f99985ff700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f997406ace0 0x7f997406d190 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.207+0000 7f99985ff700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99900a88d0 0x7f99900aa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.207+0000 7f99985ff700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f997c004430 0x7f999014d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.207+0000 7f99985ff700 1 --2- 172.21.15.97:0/3975761201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99900a8460 0x7f99900a9d10 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.207+0000 7f99985ff700 1 -- 172.21.15.97:0/3975761201 >> 172.21.15.97:0/3975761201 conn(0x7f99900ce8c0 msgr2=0x7f99900cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:27.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.208+0000 7f99985ff700 1 -- 172.21.15.97:0/3975761201 shutdown_connections 2023-07-06T03:53:27.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.208+0000 7f99985ff700 1 -- 172.21.15.97:0/3975761201 wait complete. 2023-07-06T03:53:27.228 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:53:27.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.420+0000 7f7ed6dff700 1 Processor -- start 2023-07-06T03:53:27.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.420+0000 7f7ed6dff700 1 -- start start 2023-07-06T03:53:27.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.421+0000 7f7ed6dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7ed00cd390 0x7f7ed0160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.421+0000 7f7ed6dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ed0160dd0 0x7f7ed00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.421+0000 7f7ed6dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed00cdae0 con 0x7f7ed015c520 2023-07-06T03:53:27.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.421+0000 7f7ed6dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed00b6640 con 0x7f7ed00cd390 2023-07-06T03:53:27.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.421+0000 7f7ed6dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed00b6920 con 0x7f7ed0160dd0 2023-07-06T03:53:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.422+0000 7f7ed4b9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7ed015c520 0x7f7ed0160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36968/0 (socket says 172.21.15.97:36968) 2023-07-06T03:53:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.422+0000 7f7ed4b9b700 1 -- 172.21.15.97:0/157490352 learned_addr learned my addr 172.21.15.97:0/157490352 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.422+0000 7f7ecffff700 1 --2- 172.21.15.97:0/157490352 >> v2:172.21.15.146:3300/0 conn(0x7f7ed00cd390 0x7f7ed0160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.422+0000 7f7ed539c700 1 --2- 172.21.15.97:0/157490352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ed0160dd0 0x7f7ed00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.422+0000 7f7ecf7fe700 1 -- 172.21.15.97:0/157490352 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606752484 0 0) 0x7f7ed00cdae0 con 0x7f7ed015c520 2023-07-06T03:53:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.422+0000 7f7ecf7fe700 1 -- 172.21.15.97:0/157490352 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec00035b0 con 0x7f7ed015c520 2023-07-06T03:53:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.422+0000 7f7ecffff700 1 -- 172.21.15.97:0/157490352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ed0160dd0 msgr2=0x7f7ed00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.422+0000 7f7ecffff700 1 --2- 172.21.15.97:0/157490352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ed0160dd0 0x7f7ed00b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.422+0000 7f7ecffff700 1 -- 172.21.15.97:0/157490352 >> v1:172.21.15.97:6789/0 conn(0x7f7ed015c520 legacy=0x7f7ed0160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.422+0000 7f7ecffff700 1 -- 172.21.15.97:0/157490352 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed00b6c00 con 0x7f7ed00cd390 2023-07-06T03:53:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.422+0000 7f7ecffff700 1 --2- 172.21.15.97:0/157490352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ed00cd390 0x7f7ed0160890 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009840 tx=0x7f7ebc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec5c19fd73f9bf0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.423+0000 7f7ecf7fe700 1 -- 172.21.15.97:0/157490352 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc011830 con 0x7f7ed00cd390 2023-07-06T03:53:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.423+0000 7f7ecf7fe700 1 -- 172.21.15.97:0/157490352 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc011990 con 0x7f7ed00cd390 2023-07-06T03:53:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.423+0000 7f7ecf7fe700 1 -- 172.21.15.97:0/157490352 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc011d20 con 0x7f7ed00cd390 2023-07-06T03:53:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.423+0000 7f7ed6dff700 1 -- 172.21.15.97:0/157490352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ed00cd390 msgr2=0x7f7ed0160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.423+0000 7f7ed6dff700 1 --2- 172.21.15.97:0/157490352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ed00cd390 0x7f7ed0160890 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009840 tx=0x7f7ebc009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.423+0000 7f7ed6dff700 1 -- 172.21.15.97:0/157490352 shutdown_connections 2023-07-06T03:53:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.423+0000 7f7ed6dff700 1 --2- 172.21.15.97:0/157490352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ed0160dd0 0x7f7ed00b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.423+0000 7f7ed6dff700 1 --2- 172.21.15.97:0/157490352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ed00cd390 0x7f7ed0160890 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.423+0000 7f7ed6dff700 1 -- 172.21.15.97:0/157490352 >> 172.21.15.97:0/157490352 conn(0x7f7ed0071790 msgr2=0x7f7ed0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:27.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.423+0000 7f7ed6dff700 1 -- 172.21.15.97:0/157490352 shutdown_connections 2023-07-06T03:53:27.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.423+0000 7f7ed6dff700 1 -- 172.21.15.97:0/157490352 wait complete. 2023-07-06T03:53:27.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.425+0000 7f7ed6dff700 1 Processor -- start 2023-07-06T03:53:27.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.425+0000 7f7ed6dff700 1 -- start start 2023-07-06T03:53:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.426+0000 7f7ed6dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ed00cd390 0x7f7ed014d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.426+0000 7f7ed6dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ed015c520 0x7f7ed00d1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.426+0000 7f7ed6dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed0160dd0 0x7f7ed00d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.426+0000 7f7ed6dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed00d13e0 con 0x7f7ed0160dd0 2023-07-06T03:53:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.426+0000 7f7ed6dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed00d1100 con 0x7f7ed015c520 2023-07-06T03:53:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.426+0000 7f7ed6dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed015d870 con 0x7f7ed00cd390 2023-07-06T03:53:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.426+0000 7f7ed539c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed0160dd0 0x7f7ed00d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.426+0000 7f7ed539c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed0160dd0 0x7f7ed00d30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39070/0 (socket says 172.21.15.97:39070) 2023-07-06T03:53:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.426+0000 7f7ed539c700 1 -- 172.21.15.97:0/1405224303 learned_addr learned my addr 172.21.15.97:0/1405224303 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.426+0000 7f7ecffff700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ed015c520 0x7f7ed00d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ed4b9b700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ed00cd390 0x7f7ed014d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ed539c700 1 -- 172.21.15.97:0/1405224303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ed00cd390 msgr2=0x7f7ed014d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ed539c700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ed00cd390 0x7f7ed014d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ed539c700 1 -- 172.21.15.97:0/1405224303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ed015c520 msgr2=0x7f7ed00d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ed539c700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ed015c520 0x7f7ed00d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ed539c700 1 -- 172.21.15.97:0/1405224303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc0094a0 con 0x7f7ed0160dd0 2023-07-06T03:53:27.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ed539c700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed0160dd0 0x7f7ed00d30d0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800efc0 tx=0x7f7ec800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:27.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ecd7fa700 1 -- 172.21.15.97:0/1405224303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8009ea0 con 0x7f7ed0160dd0 2023-07-06T03:53:27.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ecd7fa700 1 -- 172.21.15.97:0/1405224303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec800f040 con 0x7f7ed0160dd0 2023-07-06T03:53:27.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ed6dff700 1 -- 172.21.15.97:0/1405224303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed015db50 con 0x7f7ed0160dd0 2023-07-06T03:53:27.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ecd7fa700 1 -- 172.21.15.97:0/1405224303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec800ced0 con 0x7f7ed0160dd0 2023-07-06T03:53:27.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.427+0000 7f7ed6dff700 1 -- 172.21.15.97:0/1405224303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed014d6c0 con 0x7f7ed0160dd0 2023-07-06T03:53:27.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.428+0000 7f7ed6dff700 1 -- 172.21.15.97:0/1405224303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed015d6d0 con 0x7f7ed0160dd0 2023-07-06T03:53:27.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.429+0000 7f7ecd7fa700 1 -- 172.21.15.97:0/1405224303 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ec8014480 con 0x7f7ed0160dd0 2023-07-06T03:53:27.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.430+0000 7f7ecd7fa700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7eb406ace0 0x7f7eb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.431+0000 7f7ecd7fa700 1 -- 172.21.15.97:0/1405224303 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7ec8089f80 con 0x7f7ed0160dd0 2023-07-06T03:53:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.431+0000 7f7ed4b9b700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7eb406ace0 0x7f7eb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.431+0000 7f7ed4b9b700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7eb406ace0 0x7f7eb406d190 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4007fd0 tx=0x7f7ec4007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:27.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.434+0000 7f7ecd7fa700 1 -- 172.21.15.97:0/1405224303 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7ed015d6d0 con 0x7f7ed0160dd0 2023-07-06T03:53:27.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.630+0000 7f7ed6dff700 1 -- 172.21.15.97:0/1405224303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ed015e070 con 0x7f7ed0160dd0 2023-07-06T03:53:27.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.632+0000 7f7ecd7fa700 1 -- 172.21.15.97:0/1405224303 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+18337 (secure 0 0 0) 0x7f7ed015e070 con 0x7f7ed0160dd0 2023-07-06T03:53:27.634 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:27.634 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":98,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:53:26.233731+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:53:24.013743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-06T03:53:25.222178+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:53:27.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 -- 172.21.15.97:0/1405224303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7eb406ace0 msgr2=0x7f7eb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7eb406ace0 0x7f7eb406d190 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4007fd0 tx=0x7f7ec4007dc0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 -- 172.21.15.97:0/1405224303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed0160dd0 msgr2=0x7f7ed00d30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed0160dd0 0x7f7ed00d30d0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800efc0 tx=0x7f7ec800c3a0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 -- 172.21.15.97:0/1405224303 shutdown_connections 2023-07-06T03:53:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7eb406ace0 0x7f7eb406d190 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ed0160dd0 0x7f7ed00d30d0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7ed00cd390 0x7f7ed014d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 --2- 172.21.15.97:0/1405224303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ed015c520 0x7f7ed00d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 -- 172.21.15.97:0/1405224303 >> 172.21.15.97:0/1405224303 conn(0x7f7ed0071790 msgr2=0x7f7ed00b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 -- 172.21.15.97:0/1405224303 shutdown_connections 2023-07-06T03:53:27.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.634+0000 7f7ed6dff700 1 -- 172.21.15.97:0/1405224303 wait complete. 2023-07-06T03:53:27.654 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T03:53:27.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.855+0000 7f067a5ff700 1 Processor -- start 2023-07-06T03:53:27.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.855+0000 7f067a5ff700 1 -- start start 2023-07-06T03:53:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.856+0000 7f067a5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f067415eb90 0x7f067415b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f067a5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f067415b6e0 con 0x7f06740cbaa0 2023-07-06T03:53:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f067a5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f067415e0b0 con 0x7f0674154890 2023-07-06T03:53:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f067a5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067415e390 con 0x7f067415eb90 2023-07-06T03:53:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f0678b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f067415eb90 0x7f067415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f0678b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f067415eb90 0x7f067415b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39082/0 (socket says 172.21.15.97:39082) 2023-07-06T03:53:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f0678b9c700 1 -- 172.21.15.97:0/453607188 learned_addr learned my addr 172.21.15.97:0/453607188 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f06737fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f06740cbaa0 0x7f067415a940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36978/0 (socket says 172.21.15.97:36978) 2023-07-06T03:53:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f0678b9c700 1 -- 172.21.15.97:0/453607188 >> v1:172.21.15.146:6789/0 conn(0x7f0674154890 legacy=0x7f06740cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f0678b9c700 1 -- 172.21.15.97:0/453607188 >> v1:172.21.15.97:6789/0 conn(0x7f06740cbaa0 legacy=0x7f067415a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f0678b9c700 1 -- 172.21.15.97:0/453607188 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067415e670 con 0x7f067415eb90 2023-07-06T03:53:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f0672ffd700 1 -- 172.21.15.97:0/453607188 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682679971 0 0) 0x7f067415b6e0 con 0x7f06740cbaa0 2023-07-06T03:53:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.857+0000 7f0678b9c700 1 --2- 172.21.15.97:0/453607188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f067415eb90 0x7f067415b050 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f066c009e50 tx=0x7f066c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee67fdc221095a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.858+0000 7f0672ffd700 1 -- 172.21.15.97:0/453607188 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c00e070 con 0x7f067415eb90 2023-07-06T03:53:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.858+0000 7f0672ffd700 1 -- 172.21.15.97:0/453607188 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c003a20 con 0x7f067415eb90 2023-07-06T03:53:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.858+0000 7f0672ffd700 1 -- 172.21.15.97:0/453607188 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c003dd0 con 0x7f067415eb90 2023-07-06T03:53:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.858+0000 7f067a5ff700 1 -- 172.21.15.97:0/453607188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f067415eb90 msgr2=0x7f067415b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.858+0000 7f067a5ff700 1 --2- 172.21.15.97:0/453607188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f067415eb90 0x7f067415b050 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f066c009e50 tx=0x7f066c00b330 comp rx=0 tx=0).stop 2023-07-06T03:53:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.858+0000 7f067a5ff700 1 -- 172.21.15.97:0/453607188 shutdown_connections 2023-07-06T03:53:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.858+0000 7f067a5ff700 1 --2- 172.21.15.97:0/453607188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f067415eb90 0x7f067415b050 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.858+0000 7f067a5ff700 1 -- 172.21.15.97:0/453607188 >> 172.21.15.97:0/453607188 conn(0x7f067406a060 msgr2=0x7f067406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.858+0000 7f067a5ff700 1 -- 172.21.15.97:0/453607188 shutdown_connections 2023-07-06T03:53:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.858+0000 7f067a5ff700 1 -- 172.21.15.97:0/453607188 wait complete. 2023-07-06T03:53:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.860+0000 7f067a5ff700 1 Processor -- start 2023-07-06T03:53:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.860+0000 7f067a5ff700 1 -- start start 2023-07-06T03:53:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.861+0000 7f067a5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06740cbaa0 0x7f067414a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.861+0000 7f067a5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0674154890 0x7f0674151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.861+0000 7f067a5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f067415eb90 0x7f0674149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.861+0000 7f067a5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06740cc560 con 0x7f06740cbaa0 2023-07-06T03:53:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.861+0000 7f067a5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067415d760 con 0x7f067415eb90 2023-07-06T03:53:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.861+0000 7f067a5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06740ca000 con 0x7f0674154890 2023-07-06T03:53:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.861+0000 7f0673fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06740cbaa0 0x7f067414a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.861+0000 7f0673fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06740cbaa0 0x7f067414a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39094/0 (socket says 172.21.15.97:39094) 2023-07-06T03:53:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.861+0000 7f0673fff700 1 -- 172.21.15.97:0/3518169023 learned_addr learned my addr 172.21.15.97:0/3518169023 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.861+0000 7f06737fe700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0674154890 0x7f0674151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f0678b9c700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f067415eb90 0x7f0674149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f0673fff700 1 -- 172.21.15.97:0/3518169023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0674154890 msgr2=0x7f0674151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f0673fff700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0674154890 0x7f0674151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f0673fff700 1 -- 172.21.15.97:0/3518169023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f067415eb90 msgr2=0x7f0674149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f0673fff700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f067415eb90 0x7f0674149a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f0673fff700 1 -- 172.21.15.97:0/3518169023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c009b30 con 0x7f06740cbaa0 2023-07-06T03:53:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f06737fe700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0674154890 0x7f0674151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f0673fff700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06740cbaa0 0x7f067414a240 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f066400c4a0 tx=0x7f066400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f0670ff9700 1 -- 172.21.15.97:0/3518169023 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06640054e0 con 0x7f06740cbaa0 2023-07-06T03:53:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f0670ff9700 1 -- 172.21.15.97:0/3518169023 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0664003680 con 0x7f06740cbaa0 2023-07-06T03:53:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f0670ff9700 1 -- 172.21.15.97:0/3518169023 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664003a30 con 0x7f06740cbaa0 2023-07-06T03:53:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f067a5ff700 1 -- 172.21.15.97:0/3518169023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06740cc800 con 0x7f06740cbaa0 2023-07-06T03:53:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.862+0000 7f067a5ff700 1 -- 172.21.15.97:0/3518169023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06740ca2e0 con 0x7f06740cbaa0 2023-07-06T03:53:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.863+0000 7f067a5ff700 1 -- 172.21.15.97:0/3518169023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0674159f60 con 0x7f06740cbaa0 2023-07-06T03:53:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.864+0000 7f0670ff9700 1 -- 172.21.15.97:0/3518169023 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0664003b90 con 0x7f06740cbaa0 2023-07-06T03:53:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.865+0000 7f0670ff9700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f066006ace0 0x7f066006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.865+0000 7f0670ff9700 1 -- 172.21.15.97:0/3518169023 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f06640884d0 con 0x7f06740cbaa0 2023-07-06T03:53:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.865+0000 7f06737fe700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f066006ace0 0x7f066006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.866+0000 7f06737fe700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f066006ace0 0x7f066006d190 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0674159c50 tx=0x7f0668007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:27.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:27.869+0000 7f0670ff9700 1 -- 172.21.15.97:0/3518169023 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0664053570 con 0x7f06740cbaa0 2023-07-06T03:53:28.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.112+0000 7f067a5ff700 1 -- 172.21.15.97:0/3518169023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f067404e340 con 0x7f06740cbaa0 2023-07-06T03:53:28.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.775+0000 7f0670ff9700 1 -- 172.21.15.97:0/3518169023 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0664056b90 con 0x7f06740cbaa0 2023-07-06T03:53:28.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.777+0000 7f067a5ff700 1 -- 172.21.15.97:0/3518169023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f066006ace0 msgr2=0x7f066006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.777+0000 7f067a5ff700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f066006ace0 0x7f066006d190 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0674159c50 tx=0x7f0668007ca0 comp rx=0 tx=0).stop 2023-07-06T03:53:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.777+0000 7f067a5ff700 1 -- 172.21.15.97:0/3518169023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06740cbaa0 msgr2=0x7f067414a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.777+0000 7f067a5ff700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06740cbaa0 0x7f067414a240 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f066400c4a0 tx=0x7f066400c7b0 comp rx=0 tx=0).stop 2023-07-06T03:53:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.778+0000 7f067a5ff700 1 -- 172.21.15.97:0/3518169023 shutdown_connections 2023-07-06T03:53:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.778+0000 7f067a5ff700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f066006ace0 0x7f066006d190 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.778+0000 7f067a5ff700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f067415eb90 0x7f0674149a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.778+0000 7f067a5ff700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06740cbaa0 0x7f067414a240 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.778+0000 7f067a5ff700 1 --2- 172.21.15.97:0/3518169023 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0674154890 0x7f0674151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.778+0000 7f067a5ff700 1 -- 172.21.15.97:0/3518169023 >> 172.21.15.97:0/3518169023 conn(0x7f067406a060 msgr2=0x7f067406c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.778+0000 7f067a5ff700 1 -- 172.21.15.97:0/3518169023 shutdown_connections 2023-07-06T03:53:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.778+0000 7f067a5ff700 1 -- 172.21.15.97:0/3518169023 wait complete. 2023-07-06T03:53:28.798 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T03:53:28.798 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T03:53:28.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.993+0000 7f5688dff700 1 Processor -- start 2023-07-06T03:53:28.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.993+0000 7f5688dff700 1 -- start start 2023-07-06T03:53:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.994+0000 7f5688dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f568415c590 0x7f568415d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.994+0000 7f5688dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56840ccc30 con 0x7f56840719c0 2023-07-06T03:53:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.994+0000 7f5688dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56840ccf10 con 0x7f568415c1c0 2023-07-06T03:53:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.994+0000 7f5688dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56840cd1f0 con 0x7f568415c590 2023-07-06T03:53:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.995+0000 7f5681d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f568415c590 0x7f568415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.995+0000 7f5681d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f568415c590 0x7f568415d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39096/0 (socket says 172.21.15.97:39096) 2023-07-06T03:53:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.995+0000 7f5681d9b700 1 -- 172.21.15.97:0/2050354412 learned_addr learned my addr 172.21.15.97:0/2050354412 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.995+0000 7f5682d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f56840719c0 0x7f568406a060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36988/0 (socket says 172.21.15.97:36988) 2023-07-06T03:53:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.995+0000 7f5681d9b700 1 -- 172.21.15.97:0/2050354412 >> v1:172.21.15.146:6789/0 conn(0x7f568415c1c0 legacy=0x7f56840cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.995+0000 7f5681d9b700 1 -- 172.21.15.97:0/2050354412 >> v1:172.21.15.97:6789/0 conn(0x7f56840719c0 legacy=0x7f568406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.995+0000 7f5681d9b700 1 -- 172.21.15.97:0/2050354412 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568415cb00 con 0x7f568415c590 2023-07-06T03:53:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.995+0000 7f568159a700 1 -- 172.21.15.97:0/2050354412 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418001502 0 0) 0x7f56840ccc30 con 0x7f56840719c0 2023-07-06T03:53:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.995+0000 7f5681d9b700 1 --2- 172.21.15.97:0/2050354412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568415c590 0x7f568415d300 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f56780098f0 tx=0x7f5678009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466390d4cbe5e9b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.996+0000 7f568159a700 1 -- 172.21.15.97:0/2050354412 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567800b8a0 con 0x7f568415c590 2023-07-06T03:53:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.996+0000 7f568159a700 1 -- 172.21.15.97:0/2050354412 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567800ba00 con 0x7f568415c590 2023-07-06T03:53:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.996+0000 7f568159a700 1 -- 172.21.15.97:0/2050354412 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567800bd90 con 0x7f568415c590 2023-07-06T03:53:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.996+0000 7f5688dff700 1 -- 172.21.15.97:0/2050354412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568415c590 msgr2=0x7f568415d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.996+0000 7f5688dff700 1 --2- 172.21.15.97:0/2050354412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568415c590 0x7f568415d300 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f56780098f0 tx=0x7f5678009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.996+0000 7f5688dff700 1 -- 172.21.15.97:0/2050354412 shutdown_connections 2023-07-06T03:53:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.996+0000 7f5688dff700 1 --2- 172.21.15.97:0/2050354412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568415c590 0x7f568415d300 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.996+0000 7f5688dff700 1 -- 172.21.15.97:0/2050354412 >> 172.21.15.97:0/2050354412 conn(0x7f56840b1650 msgr2=0x7f56840b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.996+0000 7f5688dff700 1 -- 172.21.15.97:0/2050354412 shutdown_connections 2023-07-06T03:53:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.996+0000 7f5688dff700 1 -- 172.21.15.97:0/2050354412 wait complete. 2023-07-06T03:53:29.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.998+0000 7f5688dff700 1 Processor -- start 2023-07-06T03:53:29.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.998+0000 7f5688dff700 1 -- start start 2023-07-06T03:53:29.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.999+0000 7f5688dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56840719c0 0x7f568406df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:29.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.999+0000 7f5688dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f568415c1c0 0x7f56840b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:29.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.999+0000 7f5688dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568415c590 0x7f56841f10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:29.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.999+0000 7f5688dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5684157440 con 0x7f568415c590 2023-07-06T03:53:29.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.999+0000 7f5688dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56840b3e20 con 0x7f568415c1c0 2023-07-06T03:53:29.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.999+0000 7f5688dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56840b4100 con 0x7f56840719c0 2023-07-06T03:53:29.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.999+0000 7f5682d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568415c590 0x7f56841f10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:29.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.999+0000 7f5682d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568415c590 0x7f56841f10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39100/0 (socket says 172.21.15.97:39100) 2023-07-06T03:53:29.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.999+0000 7f5682d9d700 1 -- 172.21.15.97:0/1153659741 learned_addr learned my addr 172.21.15.97:0/1153659741 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:29.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:28.999+0000 7f5681d9b700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f568415c1c0 0x7f56840b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:29.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5682d9d700 1 -- 172.21.15.97:0/1153659741 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56840719c0 msgr2=0x7f568406df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:29.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f568259c700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56840719c0 0x7f568406df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:29.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5682d9d700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56840719c0 0x7f568406df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:29.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5682d9d700 1 -- 172.21.15.97:0/1153659741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f568415c1c0 msgr2=0x7f56840b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:29.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5682d9d700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f568415c1c0 0x7f56840b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:29.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5682d9d700 1 -- 172.21.15.97:0/1153659741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5678009550 con 0x7f568415c590 2023-07-06T03:53:29.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5681d9b700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f568415c1c0 0x7f56840b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:29.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f568259c700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56840719c0 0x7f568406df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:29.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5682d9d700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568415c590 0x7f56841f10e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f567c002230 tx=0x7f567c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5672ffd700 1 -- 172.21.15.97:0/1153659741 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c00ac20 con 0x7f568415c590 2023-07-06T03:53:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5672ffd700 1 -- 172.21.15.97:0/1153659741 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c00ada0 con 0x7f568415c590 2023-07-06T03:53:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5672ffd700 1 -- 172.21.15.97:0/1153659741 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c007310 con 0x7f568415c590 2023-07-06T03:53:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5688dff700 1 -- 172.21.15.97:0/1153659741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56840b43e0 con 0x7f568415c590 2023-07-06T03:53:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.000+0000 7f5688dff700 1 -- 172.21.15.97:0/1153659741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5684149160 con 0x7f568415c590 2023-07-06T03:53:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.001+0000 7f5688dff700 1 -- 172.21.15.97:0/1153659741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5684157580 con 0x7f568415c590 2023-07-06T03:53:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.002+0000 7f5672ffd700 1 -- 172.21.15.97:0/1153659741 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f567c007470 con 0x7f568415c590 2023-07-06T03:53:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.003+0000 7f5672ffd700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f566406ace0 0x7f566406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.003+0000 7f5672ffd700 1 -- 172.21.15.97:0/1153659741 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f567c0890e0 con 0x7f568415c590 2023-07-06T03:53:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.004+0000 7f568259c700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f566406ace0 0x7f566406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.004+0000 7f568259c700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f566406ace0 0x7f566406d190 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f568406eb20 tx=0x7f5674009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:29.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.008+0000 7f5672ffd700 1 -- 172.21.15.97:0/1153659741 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5684157580 con 0x7f568415c590 2023-07-06T03:53:29.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.248+0000 7f5688dff700 1 -- 172.21.15.97:0/1153659741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5684157580 con 0x7f568415c590 2023-07-06T03:53:29.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.781+0000 7f5672ffd700 1 -- 172.21.15.97:0/1153659741 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5684157580 con 0x7f568415c590 2023-07-06T03:53:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 -- 172.21.15.97:0/1153659741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f566406ace0 msgr2=0x7f566406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f566406ace0 0x7f566406d190 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f568406eb20 tx=0x7f5674009040 comp rx=0 tx=0).stop 2023-07-06T03:53:29.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 -- 172.21.15.97:0/1153659741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568415c590 msgr2=0x7f56841f10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:29.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568415c590 0x7f56841f10e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f567c002230 tx=0x7f567c00a060 comp rx=0 tx=0).stop 2023-07-06T03:53:29.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 -- 172.21.15.97:0/1153659741 shutdown_connections 2023-07-06T03:53:29.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f566406ace0 0x7f566406d190 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:29.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568415c590 0x7f56841f10e0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:29.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f56840719c0 0x7f568406df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:29.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 --2- 172.21.15.97:0/1153659741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f568415c1c0 0x7f56840b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:29.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 -- 172.21.15.97:0/1153659741 >> 172.21.15.97:0/1153659741 conn(0x7f56840b1650 msgr2=0x7f568406f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:29.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 -- 172.21.15.97:0/1153659741 shutdown_connections 2023-07-06T03:53:29.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.784+0000 7f5688dff700 1 -- 172.21.15.97:0/1153659741 wait complete. 2023-07-06T03:53:29.804 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T03:53:29.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:53:30.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.998+0000 7f2a097ff700 1 Processor -- start 2023-07-06T03:53:30.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.998+0000 7f2a097ff700 1 -- start start 2023-07-06T03:53:30.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:29.999+0000 7f2a097ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a04158f20 0x7f2a040caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a097ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a040b0c70 0x7f2a0406bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a097ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0415cc40 con 0x7f2a041592f0 2023-07-06T03:53:30.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a097ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0415cf20 con 0x7f2a04158f20 2023-07-06T03:53:30.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a097ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0415d200 con 0x7f2a040b0c70 2023-07-06T03:53:30.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a02ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a04158f20 0x7f2a040caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a027fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2a041592f0 0x7f2a040b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51242/0 (socket says 172.21.15.97:51242) 2023-07-06T03:53:30.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a027fc700 1 -- 172.21.15.97:0/3486528415 learned_addr learned my addr 172.21.15.97:0/3486528415 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:30.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a037fe700 1 --2- 172.21.15.97:0/3486528415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a040b0c70 0x7f2a0406bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a02ffd700 1 -- 172.21.15.97:0/3486528415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a040b0c70 msgr2=0x7f2a0406bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:30.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a02ffd700 1 --2- 172.21.15.97:0/3486528415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a040b0c70 0x7f2a0406bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a02ffd700 1 -- 172.21.15.97:0/3486528415 >> v1:172.21.15.97:6789/0 conn(0x7f2a041592f0 legacy=0x7f2a040b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a01ffb700 1 -- 172.21.15.97:0/3486528415 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118257227 0 0) 0x7f2a0415cc40 con 0x7f2a041592f0 2023-07-06T03:53:30.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a02ffd700 1 -- 172.21.15.97:0/3486528415 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0415d4e0 con 0x7f2a04158f20 2023-07-06T03:53:30.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.000+0000 7f2a037fe700 1 --2- 172.21.15.97:0/3486528415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a040b0c70 0x7f2a0406bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:30.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.001+0000 7f2a02ffd700 1 --2- 172.21.15.97:0/3486528415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a04158f20 0x7f2a040caac0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009840 tx=0x7f29f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c414ecc164c27e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:30.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.001+0000 7f2a01ffb700 1 -- 172.21.15.97:0/3486528415 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4011830 con 0x7f2a04158f20 2023-07-06T03:53:30.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.001+0000 7f2a01ffb700 1 -- 172.21.15.97:0/3486528415 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f4011990 con 0x7f2a04158f20 2023-07-06T03:53:30.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.001+0000 7f2a01ffb700 1 -- 172.21.15.97:0/3486528415 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4011d20 con 0x7f2a04158f20 2023-07-06T03:53:30.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.001+0000 7f2a097ff700 1 -- 172.21.15.97:0/3486528415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a04158f20 msgr2=0x7f2a040caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.001+0000 7f2a097ff700 1 --2- 172.21.15.97:0/3486528415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a04158f20 0x7f2a040caac0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009840 tx=0x7f29f4009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:30.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.001+0000 7f2a097ff700 1 -- 172.21.15.97:0/3486528415 shutdown_connections 2023-07-06T03:53:30.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.001+0000 7f2a097ff700 1 --2- 172.21.15.97:0/3486528415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a04158f20 0x7f2a040caac0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.001+0000 7f2a097ff700 1 --2- 172.21.15.97:0/3486528415 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a040b0c70 0x7f2a0406bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.001+0000 7f2a097ff700 1 -- 172.21.15.97:0/3486528415 >> 172.21.15.97:0/3486528415 conn(0x7f2a040ce8c0 msgr2=0x7f2a040cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:30.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.001+0000 7f2a097ff700 1 -- 172.21.15.97:0/3486528415 shutdown_connections 2023-07-06T03:53:30.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.002+0000 7f2a097ff700 1 -- 172.21.15.97:0/3486528415 wait complete. 2023-07-06T03:53:30.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.003+0000 7f2a097ff700 1 Processor -- start 2023-07-06T03:53:30.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.003+0000 7f2a097ff700 1 -- start start 2023-07-06T03:53:30.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.004+0000 7f2a097ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a040b0c70 0x7f2a041483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.004+0000 7f2a097ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a04158f20 0x7f2a040b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.004+0000 7f2a097ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a041592f0 0x7f2a040cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.004+0000 7f2a097ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040b22a0 con 0x7f2a04158f20 2023-07-06T03:53:30.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.004+0000 7f2a097ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04148930 con 0x7f2a041592f0 2023-07-06T03:53:30.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.004+0000 7f2a097ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040b4080 con 0x7f2a040b0c70 2023-07-06T03:53:30.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a027fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a04158f20 0x7f2a040b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a027fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a04158f20 0x7f2a040b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59392/0 (socket says 172.21.15.97:59392) 2023-07-06T03:53:30.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a027fc700 1 -- 172.21.15.97:0/2199310344 learned_addr learned my addr 172.21.15.97:0/2199310344 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:30.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a02ffd700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a040b0c70 0x7f2a041483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a037fe700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a041592f0 0x7f2a040cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a027fc700 1 -- 172.21.15.97:0/2199310344 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a040b0c70 msgr2=0x7f2a041483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a027fc700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a040b0c70 0x7f2a041483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a027fc700 1 -- 172.21.15.97:0/2199310344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a041592f0 msgr2=0x7f2a040cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a027fc700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a041592f0 0x7f2a040cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a027fc700 1 -- 172.21.15.97:0/2199310344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f40094a0 con 0x7f2a04158f20 2023-07-06T03:53:30.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a037fe700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a041592f0 0x7f2a040cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:30.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a027fc700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a04158f20 0x7f2a040b35c0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f29f800c550 tx=0x7f29f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:30.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f29eb7fe700 1 -- 172.21.15.97:0/2199310344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8004c80 con 0x7f2a04158f20 2023-07-06T03:53:30.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f29eb7fe700 1 -- 172.21.15.97:0/2199310344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f8004e00 con 0x7f2a04158f20 2023-07-06T03:53:30.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f29eb7fe700 1 -- 172.21.15.97:0/2199310344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8007310 con 0x7f2a04158f20 2023-07-06T03:53:30.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.005+0000 7f2a097ff700 1 -- 172.21.15.97:0/2199310344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a040b1970 con 0x7f2a04158f20 2023-07-06T03:53:30.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.006+0000 7f2a097ff700 1 -- 172.21.15.97:0/2199310344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a040b1f30 con 0x7f2a04158f20 2023-07-06T03:53:30.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.007+0000 7f2a097ff700 1 -- 172.21.15.97:0/2199310344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a04153b40 con 0x7f2a04158f20 2023-07-06T03:53:30.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.008+0000 7f29eb7fe700 1 -- 172.21.15.97:0/2199310344 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f29f8007470 con 0x7f2a04158f20 2023-07-06T03:53:30.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.008+0000 7f29eb7fe700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29f006ace0 0x7f29f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.009+0000 7f29eb7fe700 1 -- 172.21.15.97:0/2199310344 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29f8089070 con 0x7f2a04158f20 2023-07-06T03:53:30.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.009+0000 7f2a02ffd700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29f006ace0 0x7f29f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.009+0000 7f2a02ffd700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29f006ace0 0x7f29f006d190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f29f40090b0 tx=0x7f29f4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:30.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.013+0000 7f29eb7fe700 1 -- 172.21.15.97:0/2199310344 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f29f80541b0 con 0x7f2a04158f20 2023-07-06T03:53:30.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.248+0000 7f2a097ff700 1 -- 172.21.15.97:0/2199310344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a040cc200 con 0x7f2a04158f20 2023-07-06T03:53:30.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.249+0000 7f29eb7fe700 1 -- 172.21.15.97:0/2199310344 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4673 (secure 0 0 0) 0x7f29f80577d0 con 0x7f2a04158f20 2023-07-06T03:53:30.250 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:30.251 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":67,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/387292825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":387292825},{"type":"v1","addr":"172.21.15.146:6833","nonce":387292825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690380561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690380561},{"type":"v1","addr":"172.21.15.97:6837","nonce":690380561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3897834271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3897834271},{"type":"v1","addr":"172.21.15.146:6835","nonce":3897834271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:53:26.234190+0000","modified":"2023-07-06T03:53:29.774892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/3324055785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3324055785},{"type":"v1","addr":"172.21.15.97:6836","nonce":3324055785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-06T03:53:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 -- 172.21.15.97:0/2199310344 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29f006ace0 msgr2=0x7f29f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29f006ace0 0x7f29f006d190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f29f40090b0 tx=0x7f29f4008030 comp rx=0 tx=0).stop 2023-07-06T03:53:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 -- 172.21.15.97:0/2199310344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a04158f20 msgr2=0x7f2a040b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a04158f20 0x7f2a040b35c0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f29f800c550 tx=0x7f29f800c860 comp rx=0 tx=0).stop 2023-07-06T03:53:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 -- 172.21.15.97:0/2199310344 shutdown_connections 2023-07-06T03:53:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f29f006ace0 0x7f29f006d190 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a041592f0 0x7f2a040cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a040b0c70 0x7f2a041483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 --2- 172.21.15.97:0/2199310344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a04158f20 0x7f2a040b35c0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 -- 172.21.15.97:0/2199310344 >> 172.21.15.97:0/2199310344 conn(0x7f2a040ce8c0 msgr2=0x7f2a04153430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 -- 172.21.15.97:0/2199310344 shutdown_connections 2023-07-06T03:53:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.251+0000 7f2a097ff700 1 -- 172.21.15.97:0/2199310344 wait complete. 2023-07-06T03:53:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 67 2023-07-06T03:53:30.270 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:53:30.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.462+0000 7fd3353ff700 1 Processor -- start 2023-07-06T03:53:30.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.462+0000 7fd3353ff700 1 -- start start 2023-07-06T03:53:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.463+0000 7fd3353ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33015c520 0x7fd3300cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.463+0000 7fd3353ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd330160180 0x7fd3301605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.463+0000 7fd3353ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd330160b30 0x7fd3300cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd3353ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3300cd8d0 con 0x7fd330160b30 2023-07-06T03:53:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd3353ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3300cdae0 con 0x7fd330160180 2023-07-06T03:53:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd3353ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300b6640 con 0x7fd33015c520 2023-07-06T03:53:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd32f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd330160b30 0x7fd3300cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd32f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd330160b30 0x7fd3300cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59404/0 (socket says 172.21.15.97:59404) 2023-07-06T03:53:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd32f7fe700 1 -- 172.21.15.97:0/913164298 learned_addr learned my addr 172.21.15.97:0/913164298 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd327fff700 1 --2- 172.21.15.97:0/913164298 >> v2:172.21.15.146:3300/0 conn(0x7fd330160180 0x7fd3301605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd32effd700 1 --2- 172.21.15.97:0/913164298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33015c520 0x7fd3300cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd32f7fe700 1 -- 172.21.15.97:0/913164298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33015c520 msgr2=0x7fd3300cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd32f7fe700 1 --2- 172.21.15.97:0/913164298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33015c520 0x7fd3300cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd32f7fe700 1 -- 172.21.15.97:0/913164298 >> v2:172.21.15.146:3300/0 conn(0x7fd330160180 msgr2=0x7fd3301605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd32f7fe700 1 --2- 172.21.15.97:0/913164298 >> v2:172.21.15.146:3300/0 conn(0x7fd330160180 0x7fd3301605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd32f7fe700 1 -- 172.21.15.97:0/913164298 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300b6920 con 0x7fd330160b30 2023-07-06T03:53:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd327fff700 1 --2- 172.21.15.97:0/913164298 >> v2:172.21.15.146:3300/0 conn(0x7fd330160180 0x7fd3301605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.464+0000 7fd32f7fe700 1 --2- 172.21.15.97:0/913164298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd330160b30 0x7fd3300cf050 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd330072770 tx=0x7fd32800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3de48a75d71ceac server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd32e7fc700 1 -- 172.21.15.97:0/913164298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32800e070 con 0x7fd330160b30 2023-07-06T03:53:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd32e7fc700 1 -- 172.21.15.97:0/913164298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32800b9c0 con 0x7fd330160b30 2023-07-06T03:53:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd32e7fc700 1 -- 172.21.15.97:0/913164298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32800bd70 con 0x7fd330160b30 2023-07-06T03:53:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd3353ff700 1 -- 172.21.15.97:0/913164298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd330160b30 msgr2=0x7fd3300cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd3353ff700 1 --2- 172.21.15.97:0/913164298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd330160b30 0x7fd3300cf050 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd330072770 tx=0x7fd32800b330 comp rx=0 tx=0).stop 2023-07-06T03:53:30.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd3353ff700 1 -- 172.21.15.97:0/913164298 shutdown_connections 2023-07-06T03:53:30.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd3353ff700 1 --2- 172.21.15.97:0/913164298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd330160b30 0x7fd3300cf050 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd3353ff700 1 --2- 172.21.15.97:0/913164298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33015c520 0x7fd3300cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd3353ff700 1 --2- 172.21.15.97:0/913164298 >> v2:172.21.15.146:3300/0 conn(0x7fd330160180 0x7fd3301605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd3353ff700 1 -- 172.21.15.97:0/913164298 >> 172.21.15.97:0/913164298 conn(0x7fd330071790 msgr2=0x7fd330071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:30.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd3353ff700 1 -- 172.21.15.97:0/913164298 shutdown_connections 2023-07-06T03:53:30.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.465+0000 7fd3353ff700 1 -- 172.21.15.97:0/913164298 wait complete. 2023-07-06T03:53:30.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.467+0000 7fd3353ff700 1 Processor -- start 2023-07-06T03:53:30.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.467+0000 7fd3353ff700 1 -- start start 2023-07-06T03:53:30.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.468+0000 7fd3353ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33015c520 0x7fd3300caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.468+0000 7fd3353ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd330160180 0x7fd3300d2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.468+0000 7fd3353ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd330160b30 0x7fd3301f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.468+0000 7fd3353ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300d2ac0 con 0x7fd33015c520 2023-07-06T03:53:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.468+0000 7fd3353ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300d2da0 con 0x7fd330160180 2023-07-06T03:53:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.468+0000 7fd3353ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300d3080 con 0x7fd330160b30 2023-07-06T03:53:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.468+0000 7fd32effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33015c520 0x7fd3300caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd32effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33015c520 0x7fd3300caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59406/0 (socket says 172.21.15.97:59406) 2023-07-06T03:53:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd32effd700 1 -- 172.21.15.97:0/2752120586 learned_addr learned my addr 172.21.15.97:0/2752120586 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd327fff700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd330160180 0x7fd3300d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd32f7fe700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd330160b30 0x7fd3301f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd32effd700 1 -- 172.21.15.97:0/2752120586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd330160b30 msgr2=0x7fd3301f1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd32effd700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd330160b30 0x7fd3301f1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd32effd700 1 -- 172.21.15.97:0/2752120586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd330160180 msgr2=0x7fd3300d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd32effd700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd330160180 0x7fd3300d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd32effd700 1 -- 172.21.15.97:0/2752120586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd328009b30 con 0x7fd33015c520 2023-07-06T03:53:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd32f7fe700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd330160b30 0x7fd3301f1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd327fff700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd330160180 0x7fd3300d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:30.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd32effd700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33015c520 0x7fd3300caa20 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd32000e810 tx=0x7fd32000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:30.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd3277fe700 1 -- 172.21.15.97:0/2752120586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320013610 con 0x7fd33015c520 2023-07-06T03:53:30.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd3277fe700 1 -- 172.21.15.97:0/2752120586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd320013770 con 0x7fd33015c520 2023-07-06T03:53:30.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd3353ff700 1 -- 172.21.15.97:0/2752120586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3300d3360 con 0x7fd33015c520 2023-07-06T03:53:30.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.469+0000 7fd3277fe700 1 -- 172.21.15.97:0/2752120586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320013b00 con 0x7fd33015c520 2023-07-06T03:53:30.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.470+0000 7fd3353ff700 1 -- 172.21.15.97:0/2752120586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3300ca480 con 0x7fd33015c520 2023-07-06T03:53:30.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.470+0000 7fd3353ff700 1 -- 172.21.15.97:0/2752120586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3300d39b0 con 0x7fd33015c520 2023-07-06T03:53:30.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.472+0000 7fd3277fe700 1 -- 172.21.15.97:0/2752120586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd320013c60 con 0x7fd33015c520 2023-07-06T03:53:30.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.473+0000 7fd3277fe700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd31006ace0 0x7fd31006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.473+0000 7fd3277fe700 1 -- 172.21.15.97:0/2752120586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd320089130 con 0x7fd33015c520 2023-07-06T03:53:30.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.473+0000 7fd327fff700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd31006ace0 0x7fd31006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.473+0000 7fd327fff700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd31006ace0 0x7fd31006d190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd3300c6810 tx=0x7fd318008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:30.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.477+0000 7fd3277fe700 1 -- 172.21.15.97:0/2752120586 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd3200541e0 con 0x7fd33015c520 2023-07-06T03:53:30.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.670+0000 7fd3353ff700 1 -- 172.21.15.97:0/2752120586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3300c9880 con 0x7fd33015c520 2023-07-06T03:53:30.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.671+0000 7fd3277fe700 1 -- 172.21.15.97:0/2752120586 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+18324 (secure 0 0 0) 0x7fd320057800 con 0x7fd33015c520 2023-07-06T03:53:30.673 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:30.674 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":99,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:53:27.754704+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:53:24.013743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-06T03:53:25.222178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:53:30.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 -- 172.21.15.97:0/2752120586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd31006ace0 msgr2=0x7fd31006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd31006ace0 0x7fd31006d190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd3300c6810 tx=0x7fd318008040 comp rx=0 tx=0).stop 2023-07-06T03:53:30.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 -- 172.21.15.97:0/2752120586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33015c520 msgr2=0x7fd3300caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33015c520 0x7fd3300caa20 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd32000e810 tx=0x7fd32000ebd0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 -- 172.21.15.97:0/2752120586 shutdown_connections 2023-07-06T03:53:30.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd31006ace0 0x7fd31006d190 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd330160b30 0x7fd3301f1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33015c520 0x7fd3300caa20 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 --2- 172.21.15.97:0/2752120586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd330160180 0x7fd3300d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 -- 172.21.15.97:0/2752120586 >> 172.21.15.97:0/2752120586 conn(0x7fd330071790 msgr2=0x7fd3300b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:30.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 -- 172.21.15.97:0/2752120586 shutdown_connections 2023-07-06T03:53:30.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.674+0000 7fd3353ff700 1 -- 172.21.15.97:0/2752120586 wait complete. 2023-07-06T03:53:30.693 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T03:53:30.694 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:53:30.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.889+0000 7fb0485ff700 1 Processor -- start 2023-07-06T03:53:30.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.889+0000 7fb0485ff700 1 -- start start 2023-07-06T03:53:30.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.890+0000 7fb0485ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb0400cd390 0x7fb040160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.890+0000 7fb0485ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb040160dd0 0x7fb0400b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.890+0000 7fb0485ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0400cdae0 con 0x7fb04015c520 2023-07-06T03:53:30.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.890+0000 7fb0485ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0400b6640 con 0x7fb0400cd390 2023-07-06T03:53:30.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.890+0000 7fb0485ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0400b6920 con 0x7fb040160dd0 2023-07-06T03:53:30.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb04639b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb04015c520 0x7fb040160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51258/0 (socket says 172.21.15.97:51258) 2023-07-06T03:53:30.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb04639b700 1 -- 172.21.15.97:0/1481776409 learned_addr learned my addr 172.21.15.97:0/1481776409 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:30.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045b9a700 1 --2- 172.21.15.97:0/1481776409 >> v2:172.21.15.146:3300/0 conn(0x7fb0400cd390 0x7fb040160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb046b9c700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb040160dd0 0x7fb0400b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145228387 0 0) 0x7fb0400cdae0 con 0x7fb04015c520 2023-07-06T03:53:30.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb02c0035b0 con 0x7fb04015c520 2023-07-06T03:53:30.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4031939453 0 0) 0x7fb02c0035b0 con 0x7fb04015c520 2023-07-06T03:53:30.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb040160dd0 msgr2=0x7fb0400b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:30.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb040160dd0 0x7fb0400b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 >> v2:172.21.15.146:3300/0 conn(0x7fb0400cd390 msgr2=0x7fb040160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:30.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 --2- 172.21.15.97:0/1481776409 >> v2:172.21.15.146:3300/0 conn(0x7fb0400cd390 0x7fb040160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0400b6c00 con 0x7fb04015c520 2023-07-06T03:53:30.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fb03c001670 con 0x7fb04015c520 2023-07-06T03:53:30.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 >> v1:172.21.15.97:6789/0 conn(0x7fb04015c520 legacy=0x7fb040160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb040160dd0 0x7fb02c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02c003f80 0x7fb02c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb02c004d90 0x7fb02c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0400b6640 con 0x7fb02c004d90 2023-07-06T03:53:30.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0400b6920 con 0x7fb02c003f80 2023-07-06T03:53:30.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.891+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c00ba10 con 0x7fb040160dd0 2023-07-06T03:53:30.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.892+0000 7fb046b9c700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb02c004d90 0x7fb02c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.892+0000 7fb04639b700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb040160dd0 0x7fb02c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.892+0000 7fb045b9a700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02c003f80 0x7fb02c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.892+0000 7fb046b9c700 1 -- 172.21.15.97:0/1481776409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb040160dd0 msgr2=0x7fb02c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:30.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.892+0000 7fb046b9c700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb040160dd0 0x7fb02c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.892+0000 7fb046b9c700 1 -- 172.21.15.97:0/1481776409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02c003f80 msgr2=0x7fb02c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:30.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.892+0000 7fb046b9c700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02c003f80 0x7fb02c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.892+0000 7fb046b9c700 1 -- 172.21.15.97:0/1481776409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c0035b0 con 0x7fb02c004d90 2023-07-06T03:53:30.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.892+0000 7fb046b9c700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb02c004d90 0x7fb02c005200 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fb038009fc0 tx=0x7fb02c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b36f6f27e43e45a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:30.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.893+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03800ec30 con 0x7fb02c004d90 2023-07-06T03:53:30.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.893+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03800f040 con 0x7fb02c004d90 2023-07-06T03:53:30.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.893+0000 7fb045399700 1 -- 172.21.15.97:0/1481776409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038014480 con 0x7fb02c004d90 2023-07-06T03:53:30.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.893+0000 7fb0485ff700 1 -- 172.21.15.97:0/1481776409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb02c004d90 msgr2=0x7fb02c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.893+0000 7fb0485ff700 1 --2- 172.21.15.97:0/1481776409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb02c004d90 0x7fb02c005200 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fb038009fc0 tx=0x7fb02c005880 comp rx=0 tx=0).stop 2023-07-06T03:53:30.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.893+0000 7fb046b9c700 1 -- 172.21.15.97:0/1481776409 reap_dead start 2023-07-06T03:53:30.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.893+0000 7fb0485ff700 1 -- 172.21.15.97:0/1481776409 shutdown_connections 2023-07-06T03:53:30.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.893+0000 7fb0485ff700 1 -- 172.21.15.97:0/1481776409 >> 172.21.15.97:0/1481776409 conn(0x7fb040071790 msgr2=0x7fb040071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:30.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.893+0000 7fb0485ff700 1 -- 172.21.15.97:0/1481776409 shutdown_connections 2023-07-06T03:53:30.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.893+0000 7fb0485ff700 1 -- 172.21.15.97:0/1481776409 wait complete. 2023-07-06T03:53:30.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.896+0000 7fb0485ff700 1 Processor -- start 2023-07-06T03:53:30.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.896+0000 7fb0485ff700 1 -- start start 2023-07-06T03:53:30.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.897+0000 7fb0485ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02c004d90 0x7fb04015de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.897+0000 7fb0485ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0400d0c40 0x7fb040150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.897+0000 7fb0485ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0401f2ad0 0x7fb040150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.897+0000 7fb0485ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0401f1de0 con 0x7fb0400d0c40 2023-07-06T03:53:30.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.897+0000 7fb0485ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0400d1140 con 0x7fb02c004d90 2023-07-06T03:53:30.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.897+0000 7fb0485ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0401f1ff0 con 0x7fb0401f2ad0 2023-07-06T03:53:30.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.897+0000 7fb045b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0400d0c40 0x7fb040150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.897+0000 7fb045b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0400d0c40 0x7fb040150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59418/0 (socket says 172.21.15.97:59418) 2023-07-06T03:53:30.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.897+0000 7fb045b9a700 1 -- 172.21.15.97:0/3552382659 learned_addr learned my addr 172.21.15.97:0/3552382659 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:30.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.897+0000 7fb04639b700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02c004d90 0x7fb04015de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.897+0000 7fb046b9c700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0401f2ad0 0x7fb040150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb045b9a700 1 -- 172.21.15.97:0/3552382659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0401f2ad0 msgr2=0x7fb040150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb045b9a700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0401f2ad0 0x7fb040150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb045b9a700 1 -- 172.21.15.97:0/3552382659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02c004d90 msgr2=0x7fb04015de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:30.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb045b9a700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02c004d90 0x7fb04015de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:30.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb045b9a700 1 -- 172.21.15.97:0/3552382659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0401f22d0 con 0x7fb0400d0c40 2023-07-06T03:53:30.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb04639b700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02c004d90 0x7fb04015de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:30.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb045b9a700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0400d0c40 0x7fb040150040 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb030011aa0 tx=0x7fb030011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:30.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb036ffd700 1 -- 172.21.15.97:0/3552382659 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000f980 con 0x7fb0400d0c40 2023-07-06T03:53:30.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb036ffd700 1 -- 172.21.15.97:0/3552382659 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03000fb00 con 0x7fb0400d0c40 2023-07-06T03:53:30.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb0485ff700 1 -- 172.21.15.97:0/3552382659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0401f25b0 con 0x7fb0400d0c40 2023-07-06T03:53:30.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb036ffd700 1 -- 172.21.15.97:0/3552382659 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000feb0 con 0x7fb0400d0c40 2023-07-06T03:53:30.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.898+0000 7fb0485ff700 1 -- 172.21.15.97:0/3552382659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04015e570 con 0x7fb0400d0c40 2023-07-06T03:53:30.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.899+0000 7fb0485ff700 1 -- 172.21.15.97:0/3552382659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04004e340 con 0x7fb0400d0c40 2023-07-06T03:53:30.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.900+0000 7fb036ffd700 1 -- 172.21.15.97:0/3552382659 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb030014440 con 0x7fb0400d0c40 2023-07-06T03:53:30.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.901+0000 7fb036ffd700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb02406ace0 0x7fb02406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:30.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.901+0000 7fb036ffd700 1 -- 172.21.15.97:0/3552382659 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb030089ae0 con 0x7fb0400d0c40 2023-07-06T03:53:30.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.901+0000 7fb04639b700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb02406ace0 0x7fb02406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:30.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.902+0000 7fb04639b700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb02406ace0 0x7fb02406d190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb03c001050 tx=0x7fb03c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:30.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:30.906+0000 7fb036ffd700 1 -- 172.21.15.97:0/3552382659 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb030058640 con 0x7fb0400d0c40 2023-07-06T03:53:31.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.098+0000 7fb0485ff700 1 -- 172.21.15.97:0/3552382659 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0400cd800 con 0x7fb02406ace0 2023-07-06T03:53:31.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.107+0000 7fb036ffd700 1 -- 172.21.15.97:0/3552382659 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306045 (secure 0 0 0) 0x7fb03c0070c0 con 0x7fb02406ace0 2023-07-06T03:53:31.112 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:31.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 -- 172.21.15.97:0/3552382659 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb02406ace0 msgr2=0x7fb02406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:31.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb02406ace0 0x7fb02406d190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb03c001050 tx=0x7fb03c009040 comp rx=0 tx=0).stop 2023-07-06T03:53:31.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 -- 172.21.15.97:0/3552382659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0400d0c40 msgr2=0x7fb040150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:31.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0400d0c40 0x7fb040150040 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb030011aa0 tx=0x7fb030011e60 comp rx=0 tx=0).stop 2023-07-06T03:53:31.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 -- 172.21.15.97:0/3552382659 shutdown_connections 2023-07-06T03:53:31.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb02406ace0 0x7fb02406d190 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0401f2ad0 0x7fb040150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb02c004d90 0x7fb04015de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 --2- 172.21.15.97:0/3552382659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0400d0c40 0x7fb040150040 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 -- 172.21.15.97:0/3552382659 >> 172.21.15.97:0/3552382659 conn(0x7fb040071790 msgr2=0x7fb0400b7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:31.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 -- 172.21.15.97:0/3552382659 shutdown_connections 2023-07-06T03:53:31.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.112+0000 7fb0485ff700 1 -- 172.21.15.97:0/3552382659 wait complete. 2023-07-06T03:53:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:53:31.131 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2023-07-06T03:53:30.992395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283432,"kb_used_data":10472,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754691288,"statfs":{"total":773094113280,"available":772803878912,"internally_reserved":0,"allocated":10723328,"data_stored":5507465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001028"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755228+0000","last_change":"2023-07-06T03:53:25.755228+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755228+0000","last_undegraded":"2023-07-06T03:53:25.755228+0000","last_fullsized":"2023-07-06T03:53:25.755228+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750519+0000","last_change":"2023-07-06T03:53:25.750519+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750519+0000","last_undegraded":"2023-07-06T03:53:25.750519+0000","last_fullsized":"2023-07-06T03:53:25.750519+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.748603+0000","last_change":"2023-07-06T03:53:25.748603+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.748603+0000","last_undegraded":"2023-07-06T03:53:25.748603+0000","last_fullsized":"2023-07-06T03:53:25.748603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756199+0000","last_change":"2023-07-06T03:53:25.756199+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756199+0000","last_undegraded":"2023-07-06T03:53:25.756199+0000","last_fullsized":"2023-07-06T03:53:25.756199+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750436+0000","last_change":"2023-07-06T03:53:25.750436+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750436+0000","last_undegraded":"2023-07-06T03:53:25.750436+0000","last_fullsized":"2023-07-06T03:53:25.750436+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756992+0000","last_change":"2023-07-06T03:53:25.756992+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756992+0000","last_undegraded":"2023-07-06T03:53:25.756992+0000","last_fullsized":"2023-07-06T03:53:25.756992+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755696+0000","last_change":"2023-07-06T03:53:25.755696+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755696+0000","last_undegraded":"2023-07-06T03:53:25.755696+0000","last_fullsized":"2023-07-06T03:53:25.755696+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756528+0000","last_change":"2023-07-06T03:53:25.756528+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756528+0000","last_undegraded":"2023-07-06T03:53:25.756528+0000","last_fullsized":"2023-07-06T03:53:25.756528+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750207+0000","last_change":"2023-07-06T03:53:25.750207+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750207+0000","last_undegraded":"2023-07-06T03:53:25.750207+0000","last_fullsized":"2023-07-06T03:53:25.750207+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757208+0000","last_change":"2023-07-06T03:53:25.757208+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757208+0000","last_undegraded":"2023-07-06T03:53:25.757208+0000","last_fullsized":"2023-07-06T03:53:25.757208+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752148+0000","last_change":"2023-07-06T03:53:25.752148+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752148+0000","last_undegraded":"2023-07-06T03:53:25.752148+0000","last_fullsized":"2023-07-06T03:53:25.752148+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756929+0000","last_change":"2023-07-06T03:53:25.756929+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756929+0000","last_undegraded":"2023-07-06T03:53:25.756929+0000","last_fullsized":"2023-07-06T03:53:25.756929+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749997+0000","last_change":"2023-07-06T03:53:25.749997+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749997+0000","last_undegraded":"2023-07-06T03:53:25.749997+0000","last_fullsized":"2023-07-06T03:53:25.749997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756564+0000","last_change":"2023-07-06T03:53:25.756564+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756564+0000","last_undegraded":"2023-07-06T03:53:25.756564+0000","last_fullsized":"2023-07-06T03:53:25.756564+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750063+0000","last_change":"2023-07-06T03:53:25.750063+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750063+0000","last_undegraded":"2023-07-06T03:53:25.750063+0000","last_fullsized":"2023-07-06T03:53:25.750063+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747769+0000","last_change":"2023-07-06T03:53:25.747769+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747769+0000","last_undegraded":"2023-07-06T03:53:25.747769+0000","last_fullsized":"2023-07-06T03:53:25.747769+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749664+0000","last_change":"2023-07-06T03:53:25.749664+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749664+0000","last_undegraded":"2023-07-06T03:53:25.749664+0000","last_fullsized":"2023-07-06T03:53:25.749664+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751017+0000","last_change":"2023-07-06T03:53:25.751017+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751017+0000","last_undegraded":"2023-07-06T03:53:25.751017+0000","last_fullsized":"2023-07-06T03:53:25.751017+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756675+0000","last_change":"2023-07-06T03:53:25.756675+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756675+0000","last_undegraded":"2023-07-06T03:53:25.756675+0000","last_fullsized":"2023-07-06T03:53:25.756675+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747503+0000","last_change":"2023-07-06T03:53:25.747503+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747503+0000","last_undegraded":"2023-07-06T03:53:25.747503+0000","last_fullsized":"2023-07-06T03:53:25.747503+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750618+0000","last_change":"2023-07-06T03:53:25.750618+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750618+0000","last_undegraded":"2023-07-06T03:53:25.750618+0000","last_fullsized":"2023-07-06T03:53:25.750618+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751059+0000","last_change":"2023-07-06T03:53:25.751059+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751059+0000","last_undegraded":"2023-07-06T03:53:25.751059+0000","last_fullsized":"2023-07-06T03:53:25.751059+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756057+0000","last_change":"2023-07-06T03:53:25.756057+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756057+0000","last_undegraded":"2023-07-06T03:53:25.756057+0000","last_fullsized":"2023-07-06T03:53:25.756057+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756069+0000","last_change":"2023-07-06T03:53:25.756069+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756069+0000","last_undegraded":"2023-07-06T03:53:25.756069+0000","last_fullsized":"2023-07-06T03:53:25.756069+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747561+0000","last_change":"2023-07-06T03:53:25.747561+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747561+0000","last_undegraded":"2023-07-06T03:53:25.747561+0000","last_fullsized":"2023-07-06T03:53:25.747561+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747500+0000","last_change":"2023-07-06T03:53:25.747500+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747500+0000","last_undegraded":"2023-07-06T03:53:25.747500+0000","last_fullsized":"2023-07-06T03:53:25.747500+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747753+0000","last_change":"2023-07-06T03:53:25.747753+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747753+0000","last_undegraded":"2023-07-06T03:53:25.747753+0000","last_fullsized":"2023-07-06T03:53:25.747753+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756499+0000","last_change":"2023-07-06T03:53:25.756499+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756499+0000","last_undegraded":"2023-07-06T03:53:25.756499+0000","last_fullsized":"2023-07-06T03:53:25.756499+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757282+0000","last_change":"2023-07-06T03:53:25.757282+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757282+0000","last_undegraded":"2023-07-06T03:53:25.757282+0000","last_fullsized":"2023-07-06T03:53:25.757282+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750863+0000","last_change":"2023-07-06T03:53:25.750863+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750863+0000","last_undegraded":"2023-07-06T03:53:25.750863+0000","last_fullsized":"2023-07-06T03:53:25.750863+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757028+0000","last_change":"2023-07-06T03:53:25.757028+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757028+0000","last_undegraded":"2023-07-06T03:53:25.757028+0000","last_fullsized":"2023-07-06T03:53:25.757028+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751127+0000","last_change":"2023-07-06T03:53:25.751127+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751127+0000","last_undegraded":"2023-07-06T03:53:25.751127+0000","last_fullsized":"2023-07-06T03:53:25.751127+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750047+0000","last_change":"2023-07-06T03:53:25.750047+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750047+0000","last_undegraded":"2023-07-06T03:53:25.750047+0000","last_fullsized":"2023-07-06T03:53:25.750047+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.760618+0000","last_change":"2023-07-06T03:53:25.758687+0000","last_active":"2023-07-06T03:53:25.760618+0000","last_peered":"2023-07-06T03:53:25.760618+0000","last_clean":"2023-07-06T03:53:25.760618+0000","last_became_active":"2023-07-06T03:53:25.758351+0000","last_became_peered":"2023-07-06T03:53:25.758351+0000","last_unstale":"2023-07-06T03:53:25.760618+0000","last_undegraded":"2023-07-06T03:53:25.760618+0000","last_fullsized":"2023-07-06T03:53:25.760618+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:19:42.419643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755665+0000","last_change":"2023-07-06T03:53:25.755665+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755665+0000","last_undegraded":"2023-07-06T03:53:25.755665+0000","last_fullsized":"2023-07-06T03:53:25.755665+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768649+0000","last_change":"2023-07-06T03:53:25.765786+0000","last_active":"2023-07-06T03:53:25.768649+0000","last_peered":"2023-07-06T03:53:25.768649+0000","last_clean":"2023-07-06T03:53:25.768649+0000","last_became_active":"2023-07-06T03:53:25.764804+0000","last_became_peered":"2023-07-06T03:53:25.764804+0000","last_unstale":"2023-07-06T03:53:25.768649+0000","last_undegraded":"2023-07-06T03:53:25.768649+0000","last_fullsized":"2023-07-06T03:53:25.768649+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:41.234580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751044+0000","last_change":"2023-07-06T03:53:25.751044+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751044+0000","last_undegraded":"2023-07-06T03:53:25.751044+0000","last_fullsized":"2023-07-06T03:53:25.751044+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.763907+0000","last_change":"2023-07-06T03:53:25.761839+0000","last_active":"2023-07-06T03:53:25.763907+0000","last_peered":"2023-07-06T03:53:25.763907+0000","last_clean":"2023-07-06T03:53:25.763907+0000","last_became_active":"2023-07-06T03:53:25.761316+0000","last_became_peered":"2023-07-06T03:53:25.761316+0000","last_unstale":"2023-07-06T03:53:25.763907+0000","last_undegraded":"2023-07-06T03:53:25.763907+0000","last_fullsized":"2023-07-06T03:53:25.763907+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:40.155881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750909+0000","last_change":"2023-07-06T03:53:25.750909+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750909+0000","last_undegraded":"2023-07-06T03:53:25.750909+0000","last_fullsized":"2023-07-06T03:53:25.750909+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767922+0000","last_change":"2023-07-06T03:53:25.763450+0000","last_active":"2023-07-06T03:53:25.767922+0000","last_peered":"2023-07-06T03:53:25.767922+0000","last_clean":"2023-07-06T03:53:25.767922+0000","last_became_active":"2023-07-06T03:53:25.763153+0000","last_became_peered":"2023-07-06T03:53:25.763153+0000","last_unstale":"2023-07-06T03:53:25.767922+0000","last_undegraded":"2023-07-06T03:53:25.767922+0000","last_fullsized":"2023-07-06T03:53:25.767922+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:04:52.422334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.754346+0000","last_change":"2023-07-06T03:53:25.754346+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.754346+0000","last_undegraded":"2023-07-06T03:53:25.754346+0000","last_fullsized":"2023-07-06T03:53:25.754346+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.764932+0000","last_change":"2023-07-06T03:53:25.760397+0000","last_active":"2023-07-06T03:53:25.764932+0000","last_peered":"2023-07-06T03:53:25.764932+0000","last_clean":"2023-07-06T03:53:25.764932+0000","last_became_active":"2023-07-06T03:53:25.760151+0000","last_became_peered":"2023-07-06T03:53:25.760151+0000","last_unstale":"2023-07-06T03:53:25.764932+0000","last_undegraded":"2023-07-06T03:53:25.764932+0000","last_fullsized":"2023-07-06T03:53:25.764932+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:31:09.073491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752140+0000","last_change":"2023-07-06T03:53:25.752140+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752140+0000","last_undegraded":"2023-07-06T03:53:25.752140+0000","last_fullsized":"2023-07-06T03:53:25.752140+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.761642+0000","last_change":"2023-07-06T03:53:25.758308+0000","last_active":"2023-07-06T03:53:25.761642+0000","last_peered":"2023-07-06T03:53:25.761642+0000","last_clean":"2023-07-06T03:53:25.761642+0000","last_became_active":"2023-07-06T03:53:25.758128+0000","last_became_peered":"2023-07-06T03:53:25.758128+0000","last_unstale":"2023-07-06T03:53:25.761642+0000","last_undegraded":"2023-07-06T03:53:25.761642+0000","last_fullsized":"2023-07-06T03:53:25.761642+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:09:53.722217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751283+0000","last_change":"2023-07-06T03:53:25.751283+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751283+0000","last_undegraded":"2023-07-06T03:53:25.751283+0000","last_fullsized":"2023-07-06T03:53:25.751283+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.761614+0000","last_change":"2023-07-06T03:53:25.757461+0000","last_active":"2023-07-06T03:53:25.761614+0000","last_peered":"2023-07-06T03:53:25.761614+0000","last_clean":"2023-07-06T03:53:25.761614+0000","last_became_active":"2023-07-06T03:53:25.757185+0000","last_became_peered":"2023-07-06T03:53:25.757185+0000","last_unstale":"2023-07-06T03:53:25.761614+0000","last_undegraded":"2023-07-06T03:53:25.761614+0000","last_fullsized":"2023-07-06T03:53:25.761614+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:50:21.608182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752031+0000","last_change":"2023-07-06T03:53:25.752031+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752031+0000","last_undegraded":"2023-07-06T03:53:25.752031+0000","last_fullsized":"2023-07-06T03:53:25.752031+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768039+0000","last_change":"2023-07-06T03:53:25.764596+0000","last_active":"2023-07-06T03:53:25.768039+0000","last_peered":"2023-07-06T03:53:25.768039+0000","last_clean":"2023-07-06T03:53:25.768039+0000","last_became_active":"2023-07-06T03:53:25.764126+0000","last_became_peered":"2023-07-06T03:53:25.764126+0000","last_unstale":"2023-07-06T03:53:25.768039+0000","last_undegraded":"2023-07-06T03:53:25.768039+0000","last_fullsized":"2023-07-06T03:53:25.768039+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:29:26.930120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.748903+0000","last_change":"2023-07-06T03:53:25.748903+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.748903+0000","last_undegraded":"2023-07-06T03:53:25.748903+0000","last_fullsized":"2023-07-06T03:53:25.748903+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766515+0000","last_change":"2023-07-06T03:53:25.763023+0000","last_active":"2023-07-06T03:53:25.766515+0000","last_peered":"2023-07-06T03:53:25.766515+0000","last_clean":"2023-07-06T03:53:25.766515+0000","last_became_active":"2023-07-06T03:53:25.762691+0000","last_became_peered":"2023-07-06T03:53:25.762691+0000","last_unstale":"2023-07-06T03:53:25.766515+0000","last_undegraded":"2023-07-06T03:53:25.766515+0000","last_fullsized":"2023-07-06T03:53:25.766515+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:26:33.149240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749045+0000","last_change":"2023-07-06T03:53:25.749045+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749045+0000","last_undegraded":"2023-07-06T03:53:25.749045+0000","last_fullsized":"2023-07-06T03:53:25.749045+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766673+0000","last_change":"2023-07-06T03:53:25.761618+0000","last_active":"2023-07-06T03:53:25.766673+0000","last_peered":"2023-07-06T03:53:25.766673+0000","last_clean":"2023-07-06T03:53:25.766673+0000","last_became_active":"2023-07-06T03:53:25.761270+0000","last_became_peered":"2023-07-06T03:53:25.761270+0000","last_unstale":"2023-07-06T03:53:25.766673+0000","last_undegraded":"2023-07-06T03:53:25.766673+0000","last_fullsized":"2023-07-06T03:53:25.766673+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:22:05.629100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757009+0000","last_change":"2023-07-06T03:53:25.757009+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757009+0000","last_undegraded":"2023-07-06T03:53:25.757009+0000","last_fullsized":"2023-07-06T03:53:25.757009+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.769322+0000","last_change":"2023-07-06T03:53:25.764734+0000","last_active":"2023-07-06T03:53:25.769322+0000","last_peered":"2023-07-06T03:53:25.769322+0000","last_clean":"2023-07-06T03:53:25.769322+0000","last_became_active":"2023-07-06T03:53:25.764430+0000","last_became_peered":"2023-07-06T03:53:25.764430+0000","last_unstale":"2023-07-06T03:53:25.769322+0000","last_undegraded":"2023-07-06T03:53:25.769322+0000","last_fullsized":"2023-07-06T03:53:25.769322+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:57:02.493524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749095+0000","last_change":"2023-07-06T03:53:25.749095+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749095+0000","last_undegraded":"2023-07-06T03:53:25.749095+0000","last_fullsized":"2023-07-06T03:53:25.749095+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767656+0000","last_change":"2023-07-06T03:53:25.761999+0000","last_active":"2023-07-06T03:53:25.767656+0000","last_peered":"2023-07-06T03:53:25.767656+0000","last_clean":"2023-07-06T03:53:25.767656+0000","last_became_active":"2023-07-06T03:53:25.761778+0000","last_became_peered":"2023-07-06T03:53:25.761778+0000","last_unstale":"2023-07-06T03:53:25.767656+0000","last_undegraded":"2023-07-06T03:53:25.767656+0000","last_fullsized":"2023-07-06T03:53:25.767656+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:59:22.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750034+0000","last_change":"2023-07-06T03:53:25.750034+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750034+0000","last_undegraded":"2023-07-06T03:53:25.750034+0000","last_fullsized":"2023-07-06T03:53:25.750034+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767580+0000","last_change":"2023-07-06T03:53:25.760521+0000","last_active":"2023-07-06T03:53:25.767580+0000","last_peered":"2023-07-06T03:53:25.767580+0000","last_clean":"2023-07-06T03:53:25.767580+0000","last_became_active":"2023-07-06T03:53:25.760210+0000","last_became_peered":"2023-07-06T03:53:25.760210+0000","last_unstale":"2023-07-06T03:53:25.767580+0000","last_undegraded":"2023-07-06T03:53:25.767580+0000","last_fullsized":"2023-07-06T03:53:25.767580+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:59:34.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750939+0000","last_change":"2023-07-06T03:53:25.750939+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750939+0000","last_undegraded":"2023-07-06T03:53:25.750939+0000","last_fullsized":"2023-07-06T03:53:25.750939+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766899+0000","last_change":"2023-07-06T03:53:25.761696+0000","last_active":"2023-07-06T03:53:25.766899+0000","last_peered":"2023-07-06T03:53:25.766899+0000","last_clean":"2023-07-06T03:53:25.766899+0000","last_became_active":"2023-07-06T03:53:25.761373+0000","last_became_peered":"2023-07-06T03:53:25.761373+0000","last_unstale":"2023-07-06T03:53:25.766899+0000","last_undegraded":"2023-07-06T03:53:25.766899+0000","last_fullsized":"2023-07-06T03:53:25.766899+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:53.429422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750124+0000","last_change":"2023-07-06T03:53:25.750124+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750124+0000","last_undegraded":"2023-07-06T03:53:25.750124+0000","last_fullsized":"2023-07-06T03:53:25.750124+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768578+0000","last_change":"2023-07-06T03:53:25.763182+0000","last_active":"2023-07-06T03:53:25.768578+0000","last_peered":"2023-07-06T03:53:25.768578+0000","last_clean":"2023-07-06T03:53:25.768578+0000","last_became_active":"2023-07-06T03:53:25.762812+0000","last_became_peered":"2023-07-06T03:53:25.762812+0000","last_unstale":"2023-07-06T03:53:25.768578+0000","last_undegraded":"2023-07-06T03:53:25.768578+0000","last_fullsized":"2023-07-06T03:53:25.768578+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:09:53.722217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749081+0000","last_change":"2023-07-06T03:53:25.749081+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749081+0000","last_undegraded":"2023-07-06T03:53:25.749081+0000","last_fullsized":"2023-07-06T03:53:25.749081+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766721+0000","last_change":"2023-07-06T03:53:25.761393+0000","last_active":"2023-07-06T03:53:25.766721+0000","last_peered":"2023-07-06T03:53:25.766721+0000","last_clean":"2023-07-06T03:53:25.766721+0000","last_became_active":"2023-07-06T03:53:25.761142+0000","last_became_peered":"2023-07-06T03:53:25.761142+0000","last_unstale":"2023-07-06T03:53:25.766721+0000","last_undegraded":"2023-07-06T03:53:25.766721+0000","last_fullsized":"2023-07-06T03:53:25.766721+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:55:25.226706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750655+0000","last_change":"2023-07-06T03:53:25.750655+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750655+0000","last_undegraded":"2023-07-06T03:53:25.750655+0000","last_fullsized":"2023-07-06T03:53:25.750655+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.765261+0000","last_change":"2023-07-06T03:53:25.762326+0000","last_active":"2023-07-06T03:53:25.765261+0000","last_peered":"2023-07-06T03:53:25.765261+0000","last_clean":"2023-07-06T03:53:25.765261+0000","last_became_active":"2023-07-06T03:53:25.762045+0000","last_became_peered":"2023-07-06T03:53:25.762045+0000","last_unstale":"2023-07-06T03:53:25.765261+0000","last_undegraded":"2023-07-06T03:53:25.765261+0000","last_fullsized":"2023-07-06T03:53:25.765261+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:25:13.401196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750697+0000","last_change":"2023-07-06T03:53:25.750697+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750697+0000","last_undegraded":"2023-07-06T03:53:25.750697+0000","last_fullsized":"2023-07-06T03:53:25.750697+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768678+0000","last_change":"2023-07-06T03:53:25.766271+0000","last_active":"2023-07-06T03:53:25.768678+0000","last_peered":"2023-07-06T03:53:25.768678+0000","last_clean":"2023-07-06T03:53:25.768678+0000","last_became_active":"2023-07-06T03:53:25.765967+0000","last_became_peered":"2023-07-06T03:53:25.765967+0000","last_unstale":"2023-07-06T03:53:25.768678+0000","last_undegraded":"2023-07-06T03:53:25.768678+0000","last_fullsized":"2023-07-06T03:53:25.768678+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:04:53.517066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757521+0000","last_change":"2023-07-06T03:53:25.757521+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757521+0000","last_undegraded":"2023-07-06T03:53:25.757521+0000","last_fullsized":"2023-07-06T03:53:25.757521+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766346+0000","last_change":"2023-07-06T03:53:25.761337+0000","last_active":"2023-07-06T03:53:25.766346+0000","last_peered":"2023-07-06T03:53:25.766346+0000","last_clean":"2023-07-06T03:53:25.766346+0000","last_became_active":"2023-07-06T03:53:25.761078+0000","last_became_peered":"2023-07-06T03:53:25.761078+0000","last_unstale":"2023-07-06T03:53:25.766346+0000","last_undegraded":"2023-07-06T03:53:25.766346+0000","last_fullsized":"2023-07-06T03:53:25.766346+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:31:55.414395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747316+0000","last_change":"2023-07-06T03:53:25.747316+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747316+0000","last_undegraded":"2023-07-06T03:53:25.747316+0000","last_fullsized":"2023-07-06T03:53:25.747316+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767576+0000","last_change":"2023-07-06T03:53:25.760855+0000","last_active":"2023-07-06T03:53:25.767576+0000","last_peered":"2023-07-06T03:53:25.767576+0000","last_clean":"2023-07-06T03:53:25.767576+0000","last_became_active":"2023-07-06T03:53:25.760495+0000","last_became_peered":"2023-07-06T03:53:25.760495+0000","last_unstale":"2023-07-06T03:53:25.767576+0000","last_undegraded":"2023-07-06T03:53:25.767576+0000","last_fullsized":"2023-07-06T03:53:25.767576+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:11.379795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755808+0000","last_change":"2023-07-06T03:53:25.755808+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755808+0000","last_undegraded":"2023-07-06T03:53:25.755808+0000","last_fullsized":"2023-07-06T03:53:25.755808+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767720+0000","last_change":"2023-07-06T03:53:25.765885+0000","last_active":"2023-07-06T03:53:25.767720+0000","last_peered":"2023-07-06T03:53:25.767720+0000","last_clean":"2023-07-06T03:53:25.767720+0000","last_became_active":"2023-07-06T03:53:25.765567+0000","last_became_peered":"2023-07-06T03:53:25.765567+0000","last_unstale":"2023-07-06T03:53:25.767720+0000","last_undegraded":"2023-07-06T03:53:25.767720+0000","last_fullsized":"2023-07-06T03:53:25.767720+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:19:58.116780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755632+0000","last_change":"2023-07-06T03:53:25.755632+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755632+0000","last_undegraded":"2023-07-06T03:53:25.755632+0000","last_fullsized":"2023-07-06T03:53:25.755632+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768727+0000","last_change":"2023-07-06T03:53:25.766585+0000","last_active":"2023-07-06T03:53:25.768727+0000","last_peered":"2023-07-06T03:53:25.768727+0000","last_clean":"2023-07-06T03:53:25.768727+0000","last_became_active":"2023-07-06T03:53:25.765934+0000","last_became_peered":"2023-07-06T03:53:25.765934+0000","last_unstale":"2023-07-06T03:53:25.768727+0000","last_undegraded":"2023-07-06T03:53:25.768727+0000","last_fullsized":"2023-07-06T03:53:25.768727+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:49.408942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755831+0000","last_change":"2023-07-06T03:53:25.755831+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755831+0000","last_undegraded":"2023-07-06T03:53:25.755831+0000","last_fullsized":"2023-07-06T03:53:25.755831+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.765034+0000","last_change":"2023-07-06T03:53:25.762440+0000","last_active":"2023-07-06T03:53:25.765034+0000","last_peered":"2023-07-06T03:53:25.765034+0000","last_clean":"2023-07-06T03:53:25.765034+0000","last_became_active":"2023-07-06T03:53:25.762126+0000","last_became_peered":"2023-07-06T03:53:25.762126+0000","last_unstale":"2023-07-06T03:53:25.765034+0000","last_undegraded":"2023-07-06T03:53:25.765034+0000","last_fullsized":"2023-07-06T03:53:25.765034+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:59:34.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750599+0000","last_change":"2023-07-06T03:53:25.750599+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750599+0000","last_undegraded":"2023-07-06T03:53:25.750599+0000","last_fullsized":"2023-07-06T03:53:25.750599+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768675+0000","last_change":"2023-07-06T03:53:25.765395+0000","last_active":"2023-07-06T03:53:25.768675+0000","last_peered":"2023-07-06T03:53:25.768675+0000","last_clean":"2023-07-06T03:53:25.768675+0000","last_became_active":"2023-07-06T03:53:25.764440+0000","last_became_peered":"2023-07-06T03:53:25.764440+0000","last_unstale":"2023-07-06T03:53:25.768675+0000","last_undegraded":"2023-07-06T03:53:25.768675+0000","last_fullsized":"2023-07-06T03:53:25.768675+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:09:04.292040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750548+0000","last_change":"2023-07-06T03:53:25.750548+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750548+0000","last_undegraded":"2023-07-06T03:53:25.750548+0000","last_fullsized":"2023-07-06T03:53:25.750548+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.771940+0000","last_change":"2023-07-06T03:53:25.769061+0000","last_active":"2023-07-06T03:53:25.771940+0000","last_peered":"2023-07-06T03:53:25.771940+0000","last_clean":"2023-07-06T03:53:25.771940+0000","last_became_active":"2023-07-06T03:53:25.768721+0000","last_became_peered":"2023-07-06T03:53:25.768721+0000","last_unstale":"2023-07-06T03:53:25.771940+0000","last_undegraded":"2023-07-06T03:53:25.771940+0000","last_fullsized":"2023-07-06T03:53:25.771940+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:06:02.164844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.753416+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:53:25.753416+0000","last_peered":"2023-07-06T03:53:25.753416+0000","last_clean":"2023-07-06T03:53:25.753416+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:53:25.753416+0000","last_undegraded":"2023-07-06T03:53:25.753416+0000","last_fullsized":"2023-07-06T03:53:25.753416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:43:35.855728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755543+0000","last_change":"2023-07-06T03:53:25.755543+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755543+0000","last_undegraded":"2023-07-06T03:53:25.755543+0000","last_fullsized":"2023-07-06T03:53:25.755543+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.769357+0000","last_change":"2023-07-06T03:53:25.764949+0000","last_active":"2023-07-06T03:53:25.769357+0000","last_peered":"2023-07-06T03:53:25.769357+0000","last_clean":"2023-07-06T03:53:25.769357+0000","last_became_active":"2023-07-06T03:53:25.764699+0000","last_became_peered":"2023-07-06T03:53:25.764699+0000","last_unstale":"2023-07-06T03:53:25.769357+0000","last_undegraded":"2023-07-06T03:53:25.769357+0000","last_fullsized":"2023-07-06T03:53:25.769357+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:26:52.325913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.752342+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:53:25.752342+0000","last_peered":"2023-07-06T03:53:25.752342+0000","last_clean":"2023-07-06T03:53:25.752342+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:53:25.752342+0000","last_undegraded":"2023-07-06T03:53:25.752342+0000","last_fullsized":"2023-07-06T03:53:25.752342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:01:00.052091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749653+0000","last_change":"2023-07-06T03:53:25.749653+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749653+0000","last_undegraded":"2023-07-06T03:53:25.749653+0000","last_fullsized":"2023-07-06T03:53:25.749653+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766487+0000","last_change":"2023-07-06T03:53:25.763534+0000","last_active":"2023-07-06T03:53:25.766487+0000","last_peered":"2023-07-06T03:53:25.766487+0000","last_clean":"2023-07-06T03:53:25.766487+0000","last_became_active":"2023-07-06T03:53:25.762916+0000","last_became_peered":"2023-07-06T03:53:25.762916+0000","last_unstale":"2023-07-06T03:53:25.766487+0000","last_undegraded":"2023-07-06T03:53:25.766487+0000","last_fullsized":"2023-07-06T03:53:25.766487+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:53.585014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.751388+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:53:25.751388+0000","last_peered":"2023-07-06T03:53:25.751388+0000","last_clean":"2023-07-06T03:53:25.751388+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:53:25.751388+0000","last_undegraded":"2023-07-06T03:53:25.751388+0000","last_fullsized":"2023-07-06T03:53:25.751388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:54.214309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756568+0000","last_change":"2023-07-06T03:53:25.756568+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756568+0000","last_undegraded":"2023-07-06T03:53:25.756568+0000","last_fullsized":"2023-07-06T03:53:25.756568+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768632+0000","last_change":"2023-07-06T03:53:25.763463+0000","last_active":"2023-07-06T03:53:25.768632+0000","last_peered":"2023-07-06T03:53:25.768632+0000","last_clean":"2023-07-06T03:53:25.768632+0000","last_became_active":"2023-07-06T03:53:25.763132+0000","last_became_peered":"2023-07-06T03:53:25.763132+0000","last_unstale":"2023-07-06T03:53:25.768632+0000","last_undegraded":"2023-07-06T03:53:25.768632+0000","last_fullsized":"2023-07-06T03:53:25.768632+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:50:21.608182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.752190+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:53:25.752190+0000","last_peered":"2023-07-06T03:53:25.752190+0000","last_clean":"2023-07-06T03:53:25.752190+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:53:25.752190+0000","last_undegraded":"2023-07-06T03:53:25.752190+0000","last_fullsized":"2023-07-06T03:53:25.752190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:37:42.592191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":301,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.744734+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:53:25.744734+0000","last_peered":"2023-07-06T03:53:25.744734+0000","last_clean":"2023-07-06T03:53:25.744734+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:53:25.744734+0000","last_undegraded":"2023-07-06T03:53:25.744734+0000","last_fullsized":"2023-07-06T03:53:25.744734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:57:56.581947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747510+0000","last_change":"2023-07-06T03:53:25.747510+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747510+0000","last_undegraded":"2023-07-06T03:53:25.747510+0000","last_fullsized":"2023-07-06T03:53:25.747510+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.762283+0000","last_change":"2023-07-06T03:53:25.758578+0000","last_active":"2023-07-06T03:53:25.762283+0000","last_peered":"2023-07-06T03:53:25.762283+0000","last_clean":"2023-07-06T03:53:25.762283+0000","last_became_active":"2023-07-06T03:53:25.758327+0000","last_became_peered":"2023-07-06T03:53:25.758327+0000","last_unstale":"2023-07-06T03:53:25.762283+0000","last_undegraded":"2023-07-06T03:53:25.762283+0000","last_fullsized":"2023-07-06T03:53:25.762283+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:11.379795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.752104+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:53:25.752104+0000","last_peered":"2023-07-06T03:53:25.752104+0000","last_clean":"2023-07-06T03:53:25.752104+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:53:25.752104+0000","last_undegraded":"2023-07-06T03:53:25.752104+0000","last_fullsized":"2023-07-06T03:53:25.752104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:27:44.192794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.754828+0000","last_change":"2023-07-06T03:53:25.754828+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.754828+0000","last_undegraded":"2023-07-06T03:53:25.754828+0000","last_fullsized":"2023-07-06T03:53:25.754828+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767980+0000","last_change":"2023-07-06T03:53:25.763925+0000","last_active":"2023-07-06T03:53:25.767980+0000","last_peered":"2023-07-06T03:53:25.767980+0000","last_clean":"2023-07-06T03:53:25.767980+0000","last_became_active":"2023-07-06T03:53:25.763655+0000","last_became_peered":"2023-07-06T03:53:25.763655+0000","last_unstale":"2023-07-06T03:53:25.767980+0000","last_undegraded":"2023-07-06T03:53:25.767980+0000","last_fullsized":"2023-07-06T03:53:25.767980+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:33:43.766965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.753207+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:53:25.753207+0000","last_peered":"2023-07-06T03:53:25.753207+0000","last_clean":"2023-07-06T03:53:25.753207+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:53:25.753207+0000","last_undegraded":"2023-07-06T03:53:25.753207+0000","last_fullsized":"2023-07-06T03:53:25.753207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:25:29.753235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751739+0000","last_change":"2023-07-06T03:53:25.751739+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751739+0000","last_undegraded":"2023-07-06T03:53:25.751739+0000","last_fullsized":"2023-07-06T03:53:25.751739+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768560+0000","last_change":"2023-07-06T03:53:25.765118+0000","last_active":"2023-07-06T03:53:25.768560+0000","last_peered":"2023-07-06T03:53:25.768560+0000","last_clean":"2023-07-06T03:53:25.768560+0000","last_became_active":"2023-07-06T03:53:25.764820+0000","last_became_peered":"2023-07-06T03:53:25.764820+0000","last_unstale":"2023-07-06T03:53:25.768560+0000","last_undegraded":"2023-07-06T03:53:25.768560+0000","last_fullsized":"2023-07-06T03:53:25.768560+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:28.563800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":150,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.756204+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:53:25.756204+0000","last_peered":"2023-07-06T03:53:25.756204+0000","last_clean":"2023-07-06T03:53:25.756204+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:53:25.756204+0000","last_undegraded":"2023-07-06T03:53:25.756204+0000","last_fullsized":"2023-07-06T03:53:25.756204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:34:25.191156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752080+0000","last_change":"2023-07-06T03:53:25.752080+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752080+0000","last_undegraded":"2023-07-06T03:53:25.752080+0000","last_fullsized":"2023-07-06T03:53:25.752080+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768745+0000","last_change":"2023-07-06T03:53:25.766728+0000","last_active":"2023-07-06T03:53:25.768745+0000","last_peered":"2023-07-06T03:53:25.768745+0000","last_clean":"2023-07-06T03:53:25.768745+0000","last_became_active":"2023-07-06T03:53:25.766327+0000","last_became_peered":"2023-07-06T03:53:25.766327+0000","last_unstale":"2023-07-06T03:53:25.768745+0000","last_undegraded":"2023-07-06T03:53:25.768745+0000","last_fullsized":"2023-07-06T03:53:25.768745+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:07:23.865989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.754247+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:53:25.754247+0000","last_peered":"2023-07-06T03:53:25.754247+0000","last_clean":"2023-07-06T03:53:25.754247+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:53:25.754247+0000","last_undegraded":"2023-07-06T03:53:25.754247+0000","last_fullsized":"2023-07-06T03:53:25.754247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:07:44.809785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":1200128,"data_stored":545111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.45900000000000002,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":1.024,"5min":4.0570000000000004,"15min":4.0570000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67900000000000005,"15min":0.76600000000000001},"min":{"1min":0.51200000000000001,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.89700000000000002,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67800000000000005,"15min":0.72399999999999998},"min":{"1min":0.47199999999999998,"5min":0.45000000000000001,"15min":0.34000000000000002},"max":{"1min":1.091,"5min":1.091,"15min":2.6219999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70499999999999996,"15min":0.84599999999999997},"min":{"1min":0.47099999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94999999999999996,"5min":4.0389999999999997,"15min":6.6310000000000002},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.39700000000000002,"15min":0.35199999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":3.77},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.45000000000000001},"max":{"1min":0.93000000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.48699999999999999,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75,"15min":0.753},"min":{"1min":0.57599999999999996,"5min":0.45400000000000001,"15min":0.43099999999999999},"max":{"1min":0.92700000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.65800000000000003,"15min":0.70499999999999996},"min":{"1min":0.44600000000000001,"5min":0.38,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":0.95999999999999996,"15min":1.4590000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.69499999999999995},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0820000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82599999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.755}]},{"osd":6,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72499999999999998,"15min":0.73999999999999999},"min":{"1min":0.46400000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":1.03,"15min":1.5349999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.51800000000000002,"5min":0.45200000000000001,"15min":0.42299999999999999},"max":{"1min":0.98199999999999998,"5min":1.006,"15min":1.3779999999999999},"last":0.55300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":1200128,"data_stored":545111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70899999999999996,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.29399999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77200000000000002,"15min":0.73999999999999999},"min":{"1min":0.44700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.2889999999999999,"5min":3.3399999999999999,"15min":3.3399999999999999},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86099999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":1.1759999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.74399999999999999,"15min":0.68200000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":1.9890000000000001,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.9450000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.76100000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69999999999999996,"15min":0.745},"min":{"1min":0.502,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":2.1850000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.53400000000000003,"5min":0.40500000000000003,"15min":0.373},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":0.64300000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.70699999999999996},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.33200000000000002},"max":{"1min":0.92100000000000004,"5min":1.0309999999999999,"15min":1.1339999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67300000000000004,"15min":0.72499999999999998},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":0.98699999999999999,"15min":1.085},"last":0.623}]},{"osd":5,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.443,"15min":0.38600000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.218},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.77700000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":1.0489999999999999,"15min":1.198},"last":0.81599999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.80500000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97399999999999998,"5min":1.054,"15min":1.323},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.81599999999999995},"min":{"1min":0.54700000000000004,"5min":0.504,"15min":0.41499999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.4139999999999999},"last":0.74199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1752,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1794048,"data_stored":1135479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.52400000000000002},"min":{"1min":0.26900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65100000000000002,"5min":1.002,"15min":2.8879999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.501,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.313},"max":{"1min":0.58099999999999996,"5min":0.78300000000000003,"15min":1.96},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45500000000000002,"15min":0.83099999999999996},"min":{"1min":0.313,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.57799999999999996,"5min":0.70799999999999996,"15min":13.055999999999999},"last":0.375},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54500000000000004,"15min":0.621},"min":{"1min":0.35699999999999998,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":2.8479999999999999},"last":0.497}]},{"osd":3,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48099999999999998,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.71599999999999997,"15min":1.2709999999999999},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.73999999999999999,"15min":1.101},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.86199999999999999,"5min":1.0640000000000001,"15min":12.989000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.78700000000000003,"15min":0.82599999999999996},"min":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.46000000000000002},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.502,"5min":0.432,"15min":0.432},"max":{"1min":0.91200000000000003,"5min":1.0860000000000001,"15min":1.3100000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.71699999999999997,"15min":1.101},"min":{"1min":0.433,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.94499999999999995,"5min":1.1020000000000001,"15min":13.023999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":0.81000000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76700000000000002,"15min":1.147},"min":{"1min":0.54000000000000004,"5min":0.48699999999999999,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":1.0569999999999999,"15min":12.941000000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75800000000000001,"15min":0.81999999999999995},"min":{"1min":0.58399999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.071,"5min":1.071,"15min":2.806},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.751,"15min":0.80300000000000005},"min":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.83999999999999997,"5min":1.1319999999999999,"15min":1.337},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.28000000000000003,"5min":0.26400000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.753,"15min":0.753},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.439},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.248},"max":{"1min":0.51200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38}]},{"osd":2,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.29599999999999999},"max":{"1min":0.52400000000000002,"5min":0.69199999999999995,"15min":0.73699999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.48399999999999999,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.314},"max":{"1min":0.55800000000000005,"5min":0.67200000000000004,"15min":1.2030000000000001},"last":0.442}]},{"osd":3,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.32200000000000001,"5min":0.30099999999999999,"15min":0.29899999999999999},"max":{"1min":0.56200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.63400000000000001,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":1.1699999999999999,"15min":1.171},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":0.88800000000000001,"5min":1.032,"15min":1.034},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.745,"15min":0.73199999999999998},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":0.90600000000000003,"5min":1.1080000000000001,"15min":1.1419999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72899999999999998,"15min":0.747},"min":{"1min":0.56399999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93700000000000006,"5min":1.1040000000000001,"15min":1.1220000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.51600000000000001},"max":{"1min":0.93300000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73599999999999999,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81299999999999994,"5min":1.0649999999999999,"15min":1.546},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":0.86599999999999999,"5min":1.0820000000000001,"15min":1.6830000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77400000000000002,"15min":1.0269999999999999},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":0.86199999999999999,"5min":1.0669999999999999,"15min":9.5960000000000001},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40400,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331440,"statfs":{"total":96636764160,"available":96595394560,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.47999999999999998,"15min":0.443},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.50800000000000001,"15min":0.48199999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.093,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.375}]},{"osd":1,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.55000000000000004,"15min":0.51100000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":6.5960000000000001,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.62},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.57099999999999995,"15min":0.52400000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.657,"5min":3.657,"15min":3.657},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.63200000000000001,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":6.6390000000000002,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.1629999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":0.505}]},{"osd":4,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":0.94299999999999995,"15min":0.78100000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":12.007,"5min":12.007,"15min":12.007},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81899999999999995,"15min":0.73199999999999998},"min":{"1min":0.49299999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.2200000000000002,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.79300000000000004,"15min":0.749},"min":{"1min":0.495,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.1830000000000001,"5min":0.88300000000000001,"15min":0.79700000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.8440000000000003,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":0.94299999999999995,"15min":0.78600000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":11.696999999999999,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":1.4390000000000001,"5min":0.92400000000000004,"15min":0.80800000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":13.084,"5min":13.084,"15min":13.084},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.83299999999999996,"15min":0.83099999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.042,"5min":0.86499999999999999,"15min":0.81999999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":4.484,"5min":4.484,"15min":4.484},"last":0.58399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.44900000000000001,"15min":0.58499999999999996},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.501}]},{"osd":1,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45600000000000002,"15min":0.94899999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":15.911},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":1.0209999999999999},"min":{"1min":0.40300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.347,"5min":1.347,"15min":16.024000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.33600000000000002,"5min":0.32800000000000001,"15min":0.27400000000000002},"max":{"1min":3.25,"5min":3.25,"15min":3.25},"last":0.437},{"interface":"front","average":{"1min":0.63,"5min":0.54300000000000004,"15min":1.028},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.40699999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73999999999999999,"15min":0.75600000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.90300000000000002},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.36699999999999999},"max":{"1min":1.724,"5min":1.724,"15min":5.7300000000000004},"last":0.92400000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75600000000000001,"15min":0.90500000000000003},"min":{"1min":0.49199999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.411,"5min":1.411,"15min":5.077},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78900000000000003,"15min":0.95699999999999996},"min":{"1min":0.57999999999999996,"5min":0.499,"15min":0.44800000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.85199999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76200000000000001,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.91400000000000003},"min":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.45600000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":5.157},"last":0.70199999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.82199999999999995},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76600000000000001,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.492,"5min":1.492,"15min":5.1189999999999998},"last":0.72799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.41599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80700000000000005,"5min":1.784,"15min":1.784},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.38},"max":{"1min":0.81899999999999995,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.82799999999999996},"min":{"1min":0.39500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":6.258},"last":0.748},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68000000000000005,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":1.806,"15min":1.806},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.84599999999999997},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.29199999999999998},"max":{"1min":0.90100000000000002,"5min":0.91000000000000003,"15min":6.4740000000000002},"last":0.629},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73799999999999999,"15min":0.92300000000000004},"min":{"1min":0.57699999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":6.4859999999999998},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.48499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.97799999999999998,"15min":1.25},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.503,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":0.99099999999999999,"15min":1.413},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59999999999999998,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77800000000000002,"5min":0.92100000000000004,"15min":1.3280000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.66000000000000003},"min":{"1min":0.50600000000000001,"5min":0.40200000000000002,"15min":0.36399999999999999},"max":{"1min":0.80200000000000005,"5min":0.94299999999999995,"15min":0.99299999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63700000000000001,"15min":0.66200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.81999999999999995,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.78600000000000003},"min":{"1min":0.52400000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":1.008,"15min":1.6839999999999999},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.72299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41299999999999998},"max":{"1min":0.88200000000000001,"5min":0.95699999999999996,"15min":1.0660000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70399999999999996,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.99399999999999999,"5min":1.079,"15min":1.21},"last":0.53500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36880,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334960,"statfs":{"total":96636764160,"available":96598999040,"internally_reserved":0,"allocated":1777664,"data_stored":1127832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70599999999999996,"15min":0.66100000000000003},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.32100000000000001},"max":{"1min":0.871,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68300000000000005,"15min":0.83299999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":6.4059999999999997},"last":0.70699999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.745},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":1.1970000000000001,"15min":1.546},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.79100000000000004,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.376},"max":{"1min":8.1120000000000001,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.84399999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.80800000000000005,"15min":0.73999999999999999},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":8.1829999999999998,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73999999999999999,"15min":0.747},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":1.3140000000000001,"5min":4.484,"15min":4.484},"last":0.80000000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.627,"15min":0.628},"min":{"1min":0.46200000000000002,"5min":0.40899999999999997,"15min":0.34699999999999998},"max":{"1min":0.88500000000000001,"5min":1.1359999999999999,"15min":1.6240000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.42499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.89700000000000002,"5min":1.0569999999999999,"15min":2.718},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.48399999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91200000000000003,"5min":1.246,"15min":1.669},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.002,"5min":1.3360000000000001,"15min":2.0459999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.749,"15min":0.77800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0609999999999999,"15min":2.2919999999999998},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:53:31.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:53:31.323 INFO:tasks.ceph.mds.c.smithi097.stderr:2023-07-06T03:53:31.322+0000 7faae05f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:53:31.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.364+0000 7f26111ff700 1 Processor -- start 2023-07-06T03:53:31.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.364+0000 7f26111ff700 1 -- start start 2023-07-06T03:53:31.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.365+0000 7f26111ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260c158f20 0x7f260c0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.365+0000 7f26111ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f260c1592f0 0x7f260c0b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.365+0000 7f26111ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260c15cc40 con 0x7f260c06d860 2023-07-06T03:53:31.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.365+0000 7f26111ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260c15cf20 con 0x7f260c1592f0 2023-07-06T03:53:31.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.366+0000 7f26111ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c15d200 con 0x7f260c158f20 2023-07-06T03:53:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.366+0000 7f260a59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f260c1592f0 0x7f260c0b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.366+0000 7f260a59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f260c1592f0 0x7f260c0b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59430/0 (socket says 172.21.15.97:59430) 2023-07-06T03:53:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.366+0000 7f260a59c700 1 -- 172.21.15.97:0/1755927374 learned_addr learned my addr 172.21.15.97:0/1755927374 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.366+0000 7f260ad9d700 1 --2- 172.21.15.97:0/1755927374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260c158f20 0x7f260c0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.366+0000 7f260a59c700 1 -- 172.21.15.97:0/1755927374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260c158f20 msgr2=0x7f260c0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.366+0000 7f260a59c700 1 --2- 172.21.15.97:0/1755927374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260c158f20 0x7f260c0caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.366+0000 7f260a59c700 1 -- 172.21.15.97:0/1755927374 >> v1:172.21.15.146:6789/0 conn(0x7f260c06d860 legacy=0x7f260c0cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.366+0000 7f260a59c700 1 -- 172.21.15.97:0/1755927374 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c15d4e0 con 0x7f260c1592f0 2023-07-06T03:53:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.366+0000 7f260a59c700 1 --2- 172.21.15.97:0/1755927374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260c1592f0 0x7f260c0b0560 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f26000098f0 tx=0x7f2600009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d49d98081dbfb6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.367+0000 7f2609d9b700 1 -- 172.21.15.97:0/1755927374 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600004230 con 0x7f260c1592f0 2023-07-06T03:53:31.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.367+0000 7f2609d9b700 1 -- 172.21.15.97:0/1755927374 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2600004390 con 0x7f260c1592f0 2023-07-06T03:53:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.367+0000 7f2609d9b700 1 -- 172.21.15.97:0/1755927374 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26000036a0 con 0x7f260c1592f0 2023-07-06T03:53:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.367+0000 7f26111ff700 1 -- 172.21.15.97:0/1755927374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260c1592f0 msgr2=0x7f260c0b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.367+0000 7f26111ff700 1 --2- 172.21.15.97:0/1755927374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260c1592f0 0x7f260c0b0560 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f26000098f0 tx=0x7f2600009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.367+0000 7f26111ff700 1 -- 172.21.15.97:0/1755927374 shutdown_connections 2023-07-06T03:53:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.367+0000 7f26111ff700 1 --2- 172.21.15.97:0/1755927374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260c158f20 0x7f260c0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.367+0000 7f26111ff700 1 --2- 172.21.15.97:0/1755927374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260c1592f0 0x7f260c0b0560 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.367+0000 7f26111ff700 1 -- 172.21.15.97:0/1755927374 >> 172.21.15.97:0/1755927374 conn(0x7f260c0ce8c0 msgr2=0x7f260c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:31.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.367+0000 7f26111ff700 1 -- 172.21.15.97:0/1755927374 shutdown_connections 2023-07-06T03:53:31.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.367+0000 7f26111ff700 1 -- 172.21.15.97:0/1755927374 wait complete. 2023-07-06T03:53:31.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.369+0000 7f26111ff700 1 Processor -- start 2023-07-06T03:53:31.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.369+0000 7f26111ff700 1 -- start start 2023-07-06T03:53:31.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f26111ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260c06d860 0x7f260c137a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f26111ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f260c158f20 0x7f260c0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f26111ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260c1592f0 0x7f260c134660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f26111ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c0ce2b0 con 0x7f260c1592f0 2023-07-06T03:53:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f26111ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c15b710 con 0x7f260c158f20 2023-07-06T03:53:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f26111ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c134ba0 con 0x7f260c06d860 2023-07-06T03:53:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f260b59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260c1592f0 0x7f260c134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f260ad9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260c06d860 0x7f260c137a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f260a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f260c158f20 0x7f260c0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f260b59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260c1592f0 0x7f260c134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59438/0 (socket says 172.21.15.97:59438) 2023-07-06T03:53:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f260ad9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260c06d860 0x7f260c137a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:45946/0 (socket says 172.21.15.97:45946) 2023-07-06T03:53:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f260a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f260c158f20 0x7f260c0cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:60036/0 (socket says 172.21.15.97:60036) 2023-07-06T03:53:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.370+0000 7f260ad9d700 1 -- 172.21.15.97:0/3588741458 learned_addr learned my addr 172.21.15.97:0/3588741458 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:31.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.371+0000 7f260b59e700 1 -- 172.21.15.97:0/3588741458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260c06d860 msgr2=0x7f260c137a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.371+0000 7f260b59e700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260c06d860 0x7f260c137a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.371+0000 7f260b59e700 1 -- 172.21.15.97:0/3588741458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f260c158f20 msgr2=0x7f260c0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.371+0000 7f260b59e700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f260c158f20 0x7f260c0cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.371+0000 7f260b59e700 1 -- 172.21.15.97:0/3588741458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2600009550 con 0x7f260c1592f0 2023-07-06T03:53:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.371+0000 7f260b59e700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260c1592f0 0x7f260c134660 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f260400cb10 tx=0x7f260400ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.371+0000 7f25f37fe700 1 -- 172.21.15.97:0/3588741458 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400a9e0 con 0x7f260c1592f0 2023-07-06T03:53:31.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.371+0000 7f25f37fe700 1 -- 172.21.15.97:0/3588741458 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400ab40 con 0x7f260c1592f0 2023-07-06T03:53:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.371+0000 7f26111ff700 1 -- 172.21.15.97:0/3588741458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260c15b3e0 con 0x7f260c1592f0 2023-07-06T03:53:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.371+0000 7f25f37fe700 1 -- 172.21.15.97:0/3588741458 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604012420 con 0x7f260c1592f0 2023-07-06T03:53:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.372+0000 7f26111ff700 1 -- 172.21.15.97:0/3588741458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260c1599c0 con 0x7f260c1592f0 2023-07-06T03:53:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.372+0000 7f26111ff700 1 -- 172.21.15.97:0/3588741458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260c15b240 con 0x7f260c1592f0 2023-07-06T03:53:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.373+0000 7f25f37fe700 1 -- 172.21.15.97:0/3588741458 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2604012580 con 0x7f260c1592f0 2023-07-06T03:53:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.374+0000 7f25f37fe700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25f806ad10 0x7f25f806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.374+0000 7f25f37fe700 1 -- 172.21.15.97:0/3588741458 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2604089b20 con 0x7f260c1592f0 2023-07-06T03:53:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.374+0000 7f260ad9d700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25f806ad10 0x7f25f806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.375+0000 7f260ad9d700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25f806ad10 0x7f25f806d1c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009a90 tx=0x7f25fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:31.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.379+0000 7f25f37fe700 1 -- 172.21.15.97:0/3588741458 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f260c15b240 con 0x7f260c1592f0 2023-07-06T03:53:31.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.569+0000 7f26111ff700 1 -- 172.21.15.97:0/3588741458 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f260c15d7c0 con 0x7f25f806ad10 2023-07-06T03:53:31.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.578+0000 7f25f37fe700 1 -- 172.21.15.97:0/3588741458 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306045 (secure 0 0 0) 0x7f25fc011750 con 0x7f25f806ad10 2023-07-06T03:53:31.583 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:31.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 -- 172.21.15.97:0/3588741458 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25f806ad10 msgr2=0x7f25f806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:31.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25f806ad10 0x7f25f806d1c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009a90 tx=0x7f25fc008040 comp rx=0 tx=0).stop 2023-07-06T03:53:31.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 -- 172.21.15.97:0/3588741458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260c1592f0 msgr2=0x7f260c134660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:31.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260c1592f0 0x7f260c134660 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f260400cb10 tx=0x7f260400ce20 comp rx=0 tx=0).stop 2023-07-06T03:53:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 -- 172.21.15.97:0/3588741458 shutdown_connections 2023-07-06T03:53:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25f806ad10 0x7f25f806d1c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260c1592f0 0x7f260c134660 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260c06d860 0x7f260c137a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 --2- 172.21.15.97:0/3588741458 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f260c158f20 0x7f260c0cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 -- 172.21.15.97:0/3588741458 >> 172.21.15.97:0/3588741458 conn(0x7f260c0ce8c0 msgr2=0x7f260c153020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:31.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 -- 172.21.15.97:0/3588741458 shutdown_connections 2023-07-06T03:53:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.582+0000 7f26111ff700 1 -- 172.21.15.97:0/3588741458 wait complete. 2023-07-06T03:53:31.587 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:53:31.602 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2023-07-06T03:53:30.992395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283432,"kb_used_data":10472,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754691288,"statfs":{"total":773094113280,"available":772803878912,"internally_reserved":0,"allocated":10723328,"data_stored":5507465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001028"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755228+0000","last_change":"2023-07-06T03:53:25.755228+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755228+0000","last_undegraded":"2023-07-06T03:53:25.755228+0000","last_fullsized":"2023-07-06T03:53:25.755228+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750519+0000","last_change":"2023-07-06T03:53:25.750519+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750519+0000","last_undegraded":"2023-07-06T03:53:25.750519+0000","last_fullsized":"2023-07-06T03:53:25.750519+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.748603+0000","last_change":"2023-07-06T03:53:25.748603+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.748603+0000","last_undegraded":"2023-07-06T03:53:25.748603+0000","last_fullsized":"2023-07-06T03:53:25.748603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756199+0000","last_change":"2023-07-06T03:53:25.756199+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756199+0000","last_undegraded":"2023-07-06T03:53:25.756199+0000","last_fullsized":"2023-07-06T03:53:25.756199+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750436+0000","last_change":"2023-07-06T03:53:25.750436+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750436+0000","last_undegraded":"2023-07-06T03:53:25.750436+0000","last_fullsized":"2023-07-06T03:53:25.750436+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756992+0000","last_change":"2023-07-06T03:53:25.756992+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756992+0000","last_undegraded":"2023-07-06T03:53:25.756992+0000","last_fullsized":"2023-07-06T03:53:25.756992+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755696+0000","last_change":"2023-07-06T03:53:25.755696+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755696+0000","last_undegraded":"2023-07-06T03:53:25.755696+0000","last_fullsized":"2023-07-06T03:53:25.755696+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756528+0000","last_change":"2023-07-06T03:53:25.756528+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756528+0000","last_undegraded":"2023-07-06T03:53:25.756528+0000","last_fullsized":"2023-07-06T03:53:25.756528+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750207+0000","last_change":"2023-07-06T03:53:25.750207+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750207+0000","last_undegraded":"2023-07-06T03:53:25.750207+0000","last_fullsized":"2023-07-06T03:53:25.750207+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757208+0000","last_change":"2023-07-06T03:53:25.757208+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757208+0000","last_undegraded":"2023-07-06T03:53:25.757208+0000","last_fullsized":"2023-07-06T03:53:25.757208+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752148+0000","last_change":"2023-07-06T03:53:25.752148+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752148+0000","last_undegraded":"2023-07-06T03:53:25.752148+0000","last_fullsized":"2023-07-06T03:53:25.752148+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756929+0000","last_change":"2023-07-06T03:53:25.756929+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756929+0000","last_undegraded":"2023-07-06T03:53:25.756929+0000","last_fullsized":"2023-07-06T03:53:25.756929+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749997+0000","last_change":"2023-07-06T03:53:25.749997+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749997+0000","last_undegraded":"2023-07-06T03:53:25.749997+0000","last_fullsized":"2023-07-06T03:53:25.749997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756564+0000","last_change":"2023-07-06T03:53:25.756564+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756564+0000","last_undegraded":"2023-07-06T03:53:25.756564+0000","last_fullsized":"2023-07-06T03:53:25.756564+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750063+0000","last_change":"2023-07-06T03:53:25.750063+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750063+0000","last_undegraded":"2023-07-06T03:53:25.750063+0000","last_fullsized":"2023-07-06T03:53:25.750063+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747769+0000","last_change":"2023-07-06T03:53:25.747769+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747769+0000","last_undegraded":"2023-07-06T03:53:25.747769+0000","last_fullsized":"2023-07-06T03:53:25.747769+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749664+0000","last_change":"2023-07-06T03:53:25.749664+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749664+0000","last_undegraded":"2023-07-06T03:53:25.749664+0000","last_fullsized":"2023-07-06T03:53:25.749664+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751017+0000","last_change":"2023-07-06T03:53:25.751017+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751017+0000","last_undegraded":"2023-07-06T03:53:25.751017+0000","last_fullsized":"2023-07-06T03:53:25.751017+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756675+0000","last_change":"2023-07-06T03:53:25.756675+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756675+0000","last_undegraded":"2023-07-06T03:53:25.756675+0000","last_fullsized":"2023-07-06T03:53:25.756675+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747503+0000","last_change":"2023-07-06T03:53:25.747503+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747503+0000","last_undegraded":"2023-07-06T03:53:25.747503+0000","last_fullsized":"2023-07-06T03:53:25.747503+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750618+0000","last_change":"2023-07-06T03:53:25.750618+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750618+0000","last_undegraded":"2023-07-06T03:53:25.750618+0000","last_fullsized":"2023-07-06T03:53:25.750618+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751059+0000","last_change":"2023-07-06T03:53:25.751059+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751059+0000","last_undegraded":"2023-07-06T03:53:25.751059+0000","last_fullsized":"2023-07-06T03:53:25.751059+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756057+0000","last_change":"2023-07-06T03:53:25.756057+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756057+0000","last_undegraded":"2023-07-06T03:53:25.756057+0000","last_fullsized":"2023-07-06T03:53:25.756057+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756069+0000","last_change":"2023-07-06T03:53:25.756069+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756069+0000","last_undegraded":"2023-07-06T03:53:25.756069+0000","last_fullsized":"2023-07-06T03:53:25.756069+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747561+0000","last_change":"2023-07-06T03:53:25.747561+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747561+0000","last_undegraded":"2023-07-06T03:53:25.747561+0000","last_fullsized":"2023-07-06T03:53:25.747561+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747500+0000","last_change":"2023-07-06T03:53:25.747500+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747500+0000","last_undegraded":"2023-07-06T03:53:25.747500+0000","last_fullsized":"2023-07-06T03:53:25.747500+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747753+0000","last_change":"2023-07-06T03:53:25.747753+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747753+0000","last_undegraded":"2023-07-06T03:53:25.747753+0000","last_fullsized":"2023-07-06T03:53:25.747753+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756499+0000","last_change":"2023-07-06T03:53:25.756499+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756499+0000","last_undegraded":"2023-07-06T03:53:25.756499+0000","last_fullsized":"2023-07-06T03:53:25.756499+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757282+0000","last_change":"2023-07-06T03:53:25.757282+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757282+0000","last_undegraded":"2023-07-06T03:53:25.757282+0000","last_fullsized":"2023-07-06T03:53:25.757282+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750863+0000","last_change":"2023-07-06T03:53:25.750863+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750863+0000","last_undegraded":"2023-07-06T03:53:25.750863+0000","last_fullsized":"2023-07-06T03:53:25.750863+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757028+0000","last_change":"2023-07-06T03:53:25.757028+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757028+0000","last_undegraded":"2023-07-06T03:53:25.757028+0000","last_fullsized":"2023-07-06T03:53:25.757028+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751127+0000","last_change":"2023-07-06T03:53:25.751127+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751127+0000","last_undegraded":"2023-07-06T03:53:25.751127+0000","last_fullsized":"2023-07-06T03:53:25.751127+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750047+0000","last_change":"2023-07-06T03:53:25.750047+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750047+0000","last_undegraded":"2023-07-06T03:53:25.750047+0000","last_fullsized":"2023-07-06T03:53:25.750047+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.760618+0000","last_change":"2023-07-06T03:53:25.758687+0000","last_active":"2023-07-06T03:53:25.760618+0000","last_peered":"2023-07-06T03:53:25.760618+0000","last_clean":"2023-07-06T03:53:25.760618+0000","last_became_active":"2023-07-06T03:53:25.758351+0000","last_became_peered":"2023-07-06T03:53:25.758351+0000","last_unstale":"2023-07-06T03:53:25.760618+0000","last_undegraded":"2023-07-06T03:53:25.760618+0000","last_fullsized":"2023-07-06T03:53:25.760618+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:19:42.419643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755665+0000","last_change":"2023-07-06T03:53:25.755665+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755665+0000","last_undegraded":"2023-07-06T03:53:25.755665+0000","last_fullsized":"2023-07-06T03:53:25.755665+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768649+0000","last_change":"2023-07-06T03:53:25.765786+0000","last_active":"2023-07-06T03:53:25.768649+0000","last_peered":"2023-07-06T03:53:25.768649+0000","last_clean":"2023-07-06T03:53:25.768649+0000","last_became_active":"2023-07-06T03:53:25.764804+0000","last_became_peered":"2023-07-06T03:53:25.764804+0000","last_unstale":"2023-07-06T03:53:25.768649+0000","last_undegraded":"2023-07-06T03:53:25.768649+0000","last_fullsized":"2023-07-06T03:53:25.768649+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:41.234580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751044+0000","last_change":"2023-07-06T03:53:25.751044+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751044+0000","last_undegraded":"2023-07-06T03:53:25.751044+0000","last_fullsized":"2023-07-06T03:53:25.751044+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.763907+0000","last_change":"2023-07-06T03:53:25.761839+0000","last_active":"2023-07-06T03:53:25.763907+0000","last_peered":"2023-07-06T03:53:25.763907+0000","last_clean":"2023-07-06T03:53:25.763907+0000","last_became_active":"2023-07-06T03:53:25.761316+0000","last_became_peered":"2023-07-06T03:53:25.761316+0000","last_unstale":"2023-07-06T03:53:25.763907+0000","last_undegraded":"2023-07-06T03:53:25.763907+0000","last_fullsized":"2023-07-06T03:53:25.763907+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:40.155881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750909+0000","last_change":"2023-07-06T03:53:25.750909+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750909+0000","last_undegraded":"2023-07-06T03:53:25.750909+0000","last_fullsized":"2023-07-06T03:53:25.750909+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767922+0000","last_change":"2023-07-06T03:53:25.763450+0000","last_active":"2023-07-06T03:53:25.767922+0000","last_peered":"2023-07-06T03:53:25.767922+0000","last_clean":"2023-07-06T03:53:25.767922+0000","last_became_active":"2023-07-06T03:53:25.763153+0000","last_became_peered":"2023-07-06T03:53:25.763153+0000","last_unstale":"2023-07-06T03:53:25.767922+0000","last_undegraded":"2023-07-06T03:53:25.767922+0000","last_fullsized":"2023-07-06T03:53:25.767922+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:04:52.422334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.754346+0000","last_change":"2023-07-06T03:53:25.754346+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.754346+0000","last_undegraded":"2023-07-06T03:53:25.754346+0000","last_fullsized":"2023-07-06T03:53:25.754346+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.764932+0000","last_change":"2023-07-06T03:53:25.760397+0000","last_active":"2023-07-06T03:53:25.764932+0000","last_peered":"2023-07-06T03:53:25.764932+0000","last_clean":"2023-07-06T03:53:25.764932+0000","last_became_active":"2023-07-06T03:53:25.760151+0000","last_became_peered":"2023-07-06T03:53:25.760151+0000","last_unstale":"2023-07-06T03:53:25.764932+0000","last_undegraded":"2023-07-06T03:53:25.764932+0000","last_fullsized":"2023-07-06T03:53:25.764932+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:31:09.073491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752140+0000","last_change":"2023-07-06T03:53:25.752140+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752140+0000","last_undegraded":"2023-07-06T03:53:25.752140+0000","last_fullsized":"2023-07-06T03:53:25.752140+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.761642+0000","last_change":"2023-07-06T03:53:25.758308+0000","last_active":"2023-07-06T03:53:25.761642+0000","last_peered":"2023-07-06T03:53:25.761642+0000","last_clean":"2023-07-06T03:53:25.761642+0000","last_became_active":"2023-07-06T03:53:25.758128+0000","last_became_peered":"2023-07-06T03:53:25.758128+0000","last_unstale":"2023-07-06T03:53:25.761642+0000","last_undegraded":"2023-07-06T03:53:25.761642+0000","last_fullsized":"2023-07-06T03:53:25.761642+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:09:53.722217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751283+0000","last_change":"2023-07-06T03:53:25.751283+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751283+0000","last_undegraded":"2023-07-06T03:53:25.751283+0000","last_fullsized":"2023-07-06T03:53:25.751283+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.761614+0000","last_change":"2023-07-06T03:53:25.757461+0000","last_active":"2023-07-06T03:53:25.761614+0000","last_peered":"2023-07-06T03:53:25.761614+0000","last_clean":"2023-07-06T03:53:25.761614+0000","last_became_active":"2023-07-06T03:53:25.757185+0000","last_became_peered":"2023-07-06T03:53:25.757185+0000","last_unstale":"2023-07-06T03:53:25.761614+0000","last_undegraded":"2023-07-06T03:53:25.761614+0000","last_fullsized":"2023-07-06T03:53:25.761614+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:50:21.608182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752031+0000","last_change":"2023-07-06T03:53:25.752031+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752031+0000","last_undegraded":"2023-07-06T03:53:25.752031+0000","last_fullsized":"2023-07-06T03:53:25.752031+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768039+0000","last_change":"2023-07-06T03:53:25.764596+0000","last_active":"2023-07-06T03:53:25.768039+0000","last_peered":"2023-07-06T03:53:25.768039+0000","last_clean":"2023-07-06T03:53:25.768039+0000","last_became_active":"2023-07-06T03:53:25.764126+0000","last_became_peered":"2023-07-06T03:53:25.764126+0000","last_unstale":"2023-07-06T03:53:25.768039+0000","last_undegraded":"2023-07-06T03:53:25.768039+0000","last_fullsized":"2023-07-06T03:53:25.768039+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:29:26.930120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.748903+0000","last_change":"2023-07-06T03:53:25.748903+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.748903+0000","last_undegraded":"2023-07-06T03:53:25.748903+0000","last_fullsized":"2023-07-06T03:53:25.748903+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766515+0000","last_change":"2023-07-06T03:53:25.763023+0000","last_active":"2023-07-06T03:53:25.766515+0000","last_peered":"2023-07-06T03:53:25.766515+0000","last_clean":"2023-07-06T03:53:25.766515+0000","last_became_active":"2023-07-06T03:53:25.762691+0000","last_became_peered":"2023-07-06T03:53:25.762691+0000","last_unstale":"2023-07-06T03:53:25.766515+0000","last_undegraded":"2023-07-06T03:53:25.766515+0000","last_fullsized":"2023-07-06T03:53:25.766515+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:26:33.149240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749045+0000","last_change":"2023-07-06T03:53:25.749045+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749045+0000","last_undegraded":"2023-07-06T03:53:25.749045+0000","last_fullsized":"2023-07-06T03:53:25.749045+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766673+0000","last_change":"2023-07-06T03:53:25.761618+0000","last_active":"2023-07-06T03:53:25.766673+0000","last_peered":"2023-07-06T03:53:25.766673+0000","last_clean":"2023-07-06T03:53:25.766673+0000","last_became_active":"2023-07-06T03:53:25.761270+0000","last_became_peered":"2023-07-06T03:53:25.761270+0000","last_unstale":"2023-07-06T03:53:25.766673+0000","last_undegraded":"2023-07-06T03:53:25.766673+0000","last_fullsized":"2023-07-06T03:53:25.766673+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:22:05.629100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757009+0000","last_change":"2023-07-06T03:53:25.757009+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757009+0000","last_undegraded":"2023-07-06T03:53:25.757009+0000","last_fullsized":"2023-07-06T03:53:25.757009+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.769322+0000","last_change":"2023-07-06T03:53:25.764734+0000","last_active":"2023-07-06T03:53:25.769322+0000","last_peered":"2023-07-06T03:53:25.769322+0000","last_clean":"2023-07-06T03:53:25.769322+0000","last_became_active":"2023-07-06T03:53:25.764430+0000","last_became_peered":"2023-07-06T03:53:25.764430+0000","last_unstale":"2023-07-06T03:53:25.769322+0000","last_undegraded":"2023-07-06T03:53:25.769322+0000","last_fullsized":"2023-07-06T03:53:25.769322+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:57:02.493524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749095+0000","last_change":"2023-07-06T03:53:25.749095+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749095+0000","last_undegraded":"2023-07-06T03:53:25.749095+0000","last_fullsized":"2023-07-06T03:53:25.749095+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767656+0000","last_change":"2023-07-06T03:53:25.761999+0000","last_active":"2023-07-06T03:53:25.767656+0000","last_peered":"2023-07-06T03:53:25.767656+0000","last_clean":"2023-07-06T03:53:25.767656+0000","last_became_active":"2023-07-06T03:53:25.761778+0000","last_became_peered":"2023-07-06T03:53:25.761778+0000","last_unstale":"2023-07-06T03:53:25.767656+0000","last_undegraded":"2023-07-06T03:53:25.767656+0000","last_fullsized":"2023-07-06T03:53:25.767656+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:59:22.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750034+0000","last_change":"2023-07-06T03:53:25.750034+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750034+0000","last_undegraded":"2023-07-06T03:53:25.750034+0000","last_fullsized":"2023-07-06T03:53:25.750034+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767580+0000","last_change":"2023-07-06T03:53:25.760521+0000","last_active":"2023-07-06T03:53:25.767580+0000","last_peered":"2023-07-06T03:53:25.767580+0000","last_clean":"2023-07-06T03:53:25.767580+0000","last_became_active":"2023-07-06T03:53:25.760210+0000","last_became_peered":"2023-07-06T03:53:25.760210+0000","last_unstale":"2023-07-06T03:53:25.767580+0000","last_undegraded":"2023-07-06T03:53:25.767580+0000","last_fullsized":"2023-07-06T03:53:25.767580+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:59:34.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750939+0000","last_change":"2023-07-06T03:53:25.750939+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750939+0000","last_undegraded":"2023-07-06T03:53:25.750939+0000","last_fullsized":"2023-07-06T03:53:25.750939+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766899+0000","last_change":"2023-07-06T03:53:25.761696+0000","last_active":"2023-07-06T03:53:25.766899+0000","last_peered":"2023-07-06T03:53:25.766899+0000","last_clean":"2023-07-06T03:53:25.766899+0000","last_became_active":"2023-07-06T03:53:25.761373+0000","last_became_peered":"2023-07-06T03:53:25.761373+0000","last_unstale":"2023-07-06T03:53:25.766899+0000","last_undegraded":"2023-07-06T03:53:25.766899+0000","last_fullsized":"2023-07-06T03:53:25.766899+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:53.429422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750124+0000","last_change":"2023-07-06T03:53:25.750124+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750124+0000","last_undegraded":"2023-07-06T03:53:25.750124+0000","last_fullsized":"2023-07-06T03:53:25.750124+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768578+0000","last_change":"2023-07-06T03:53:25.763182+0000","last_active":"2023-07-06T03:53:25.768578+0000","last_peered":"2023-07-06T03:53:25.768578+0000","last_clean":"2023-07-06T03:53:25.768578+0000","last_became_active":"2023-07-06T03:53:25.762812+0000","last_became_peered":"2023-07-06T03:53:25.762812+0000","last_unstale":"2023-07-06T03:53:25.768578+0000","last_undegraded":"2023-07-06T03:53:25.768578+0000","last_fullsized":"2023-07-06T03:53:25.768578+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:09:53.722217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749081+0000","last_change":"2023-07-06T03:53:25.749081+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749081+0000","last_undegraded":"2023-07-06T03:53:25.749081+0000","last_fullsized":"2023-07-06T03:53:25.749081+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766721+0000","last_change":"2023-07-06T03:53:25.761393+0000","last_active":"2023-07-06T03:53:25.766721+0000","last_peered":"2023-07-06T03:53:25.766721+0000","last_clean":"2023-07-06T03:53:25.766721+0000","last_became_active":"2023-07-06T03:53:25.761142+0000","last_became_peered":"2023-07-06T03:53:25.761142+0000","last_unstale":"2023-07-06T03:53:25.766721+0000","last_undegraded":"2023-07-06T03:53:25.766721+0000","last_fullsized":"2023-07-06T03:53:25.766721+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:55:25.226706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750655+0000","last_change":"2023-07-06T03:53:25.750655+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750655+0000","last_undegraded":"2023-07-06T03:53:25.750655+0000","last_fullsized":"2023-07-06T03:53:25.750655+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.765261+0000","last_change":"2023-07-06T03:53:25.762326+0000","last_active":"2023-07-06T03:53:25.765261+0000","last_peered":"2023-07-06T03:53:25.765261+0000","last_clean":"2023-07-06T03:53:25.765261+0000","last_became_active":"2023-07-06T03:53:25.762045+0000","last_became_peered":"2023-07-06T03:53:25.762045+0000","last_unstale":"2023-07-06T03:53:25.765261+0000","last_undegraded":"2023-07-06T03:53:25.765261+0000","last_fullsized":"2023-07-06T03:53:25.765261+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:25:13.401196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750697+0000","last_change":"2023-07-06T03:53:25.750697+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750697+0000","last_undegraded":"2023-07-06T03:53:25.750697+0000","last_fullsized":"2023-07-06T03:53:25.750697+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768678+0000","last_change":"2023-07-06T03:53:25.766271+0000","last_active":"2023-07-06T03:53:25.768678+0000","last_peered":"2023-07-06T03:53:25.768678+0000","last_clean":"2023-07-06T03:53:25.768678+0000","last_became_active":"2023-07-06T03:53:25.765967+0000","last_became_peered":"2023-07-06T03:53:25.765967+0000","last_unstale":"2023-07-06T03:53:25.768678+0000","last_undegraded":"2023-07-06T03:53:25.768678+0000","last_fullsized":"2023-07-06T03:53:25.768678+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:04:53.517066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757521+0000","last_change":"2023-07-06T03:53:25.757521+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757521+0000","last_undegraded":"2023-07-06T03:53:25.757521+0000","last_fullsized":"2023-07-06T03:53:25.757521+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766346+0000","last_change":"2023-07-06T03:53:25.761337+0000","last_active":"2023-07-06T03:53:25.766346+0000","last_peered":"2023-07-06T03:53:25.766346+0000","last_clean":"2023-07-06T03:53:25.766346+0000","last_became_active":"2023-07-06T03:53:25.761078+0000","last_became_peered":"2023-07-06T03:53:25.761078+0000","last_unstale":"2023-07-06T03:53:25.766346+0000","last_undegraded":"2023-07-06T03:53:25.766346+0000","last_fullsized":"2023-07-06T03:53:25.766346+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:31:55.414395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747316+0000","last_change":"2023-07-06T03:53:25.747316+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747316+0000","last_undegraded":"2023-07-06T03:53:25.747316+0000","last_fullsized":"2023-07-06T03:53:25.747316+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767576+0000","last_change":"2023-07-06T03:53:25.760855+0000","last_active":"2023-07-06T03:53:25.767576+0000","last_peered":"2023-07-06T03:53:25.767576+0000","last_clean":"2023-07-06T03:53:25.767576+0000","last_became_active":"2023-07-06T03:53:25.760495+0000","last_became_peered":"2023-07-06T03:53:25.760495+0000","last_unstale":"2023-07-06T03:53:25.767576+0000","last_undegraded":"2023-07-06T03:53:25.767576+0000","last_fullsized":"2023-07-06T03:53:25.767576+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:11.379795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755808+0000","last_change":"2023-07-06T03:53:25.755808+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755808+0000","last_undegraded":"2023-07-06T03:53:25.755808+0000","last_fullsized":"2023-07-06T03:53:25.755808+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767720+0000","last_change":"2023-07-06T03:53:25.765885+0000","last_active":"2023-07-06T03:53:25.767720+0000","last_peered":"2023-07-06T03:53:25.767720+0000","last_clean":"2023-07-06T03:53:25.767720+0000","last_became_active":"2023-07-06T03:53:25.765567+0000","last_became_peered":"2023-07-06T03:53:25.765567+0000","last_unstale":"2023-07-06T03:53:25.767720+0000","last_undegraded":"2023-07-06T03:53:25.767720+0000","last_fullsized":"2023-07-06T03:53:25.767720+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:19:58.116780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755632+0000","last_change":"2023-07-06T03:53:25.755632+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755632+0000","last_undegraded":"2023-07-06T03:53:25.755632+0000","last_fullsized":"2023-07-06T03:53:25.755632+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768727+0000","last_change":"2023-07-06T03:53:25.766585+0000","last_active":"2023-07-06T03:53:25.768727+0000","last_peered":"2023-07-06T03:53:25.768727+0000","last_clean":"2023-07-06T03:53:25.768727+0000","last_became_active":"2023-07-06T03:53:25.765934+0000","last_became_peered":"2023-07-06T03:53:25.765934+0000","last_unstale":"2023-07-06T03:53:25.768727+0000","last_undegraded":"2023-07-06T03:53:25.768727+0000","last_fullsized":"2023-07-06T03:53:25.768727+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:49.408942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755831+0000","last_change":"2023-07-06T03:53:25.755831+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755831+0000","last_undegraded":"2023-07-06T03:53:25.755831+0000","last_fullsized":"2023-07-06T03:53:25.755831+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.765034+0000","last_change":"2023-07-06T03:53:25.762440+0000","last_active":"2023-07-06T03:53:25.765034+0000","last_peered":"2023-07-06T03:53:25.765034+0000","last_clean":"2023-07-06T03:53:25.765034+0000","last_became_active":"2023-07-06T03:53:25.762126+0000","last_became_peered":"2023-07-06T03:53:25.762126+0000","last_unstale":"2023-07-06T03:53:25.765034+0000","last_undegraded":"2023-07-06T03:53:25.765034+0000","last_fullsized":"2023-07-06T03:53:25.765034+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:59:34.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750599+0000","last_change":"2023-07-06T03:53:25.750599+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750599+0000","last_undegraded":"2023-07-06T03:53:25.750599+0000","last_fullsized":"2023-07-06T03:53:25.750599+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768675+0000","last_change":"2023-07-06T03:53:25.765395+0000","last_active":"2023-07-06T03:53:25.768675+0000","last_peered":"2023-07-06T03:53:25.768675+0000","last_clean":"2023-07-06T03:53:25.768675+0000","last_became_active":"2023-07-06T03:53:25.764440+0000","last_became_peered":"2023-07-06T03:53:25.764440+0000","last_unstale":"2023-07-06T03:53:25.768675+0000","last_undegraded":"2023-07-06T03:53:25.768675+0000","last_fullsized":"2023-07-06T03:53:25.768675+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:09:04.292040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750548+0000","last_change":"2023-07-06T03:53:25.750548+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750548+0000","last_undegraded":"2023-07-06T03:53:25.750548+0000","last_fullsized":"2023-07-06T03:53:25.750548+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.771940+0000","last_change":"2023-07-06T03:53:25.769061+0000","last_active":"2023-07-06T03:53:25.771940+0000","last_peered":"2023-07-06T03:53:25.771940+0000","last_clean":"2023-07-06T03:53:25.771940+0000","last_became_active":"2023-07-06T03:53:25.768721+0000","last_became_peered":"2023-07-06T03:53:25.768721+0000","last_unstale":"2023-07-06T03:53:25.771940+0000","last_undegraded":"2023-07-06T03:53:25.771940+0000","last_fullsized":"2023-07-06T03:53:25.771940+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:06:02.164844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.753416+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:53:25.753416+0000","last_peered":"2023-07-06T03:53:25.753416+0000","last_clean":"2023-07-06T03:53:25.753416+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:53:25.753416+0000","last_undegraded":"2023-07-06T03:53:25.753416+0000","last_fullsized":"2023-07-06T03:53:25.753416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:43:35.855728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755543+0000","last_change":"2023-07-06T03:53:25.755543+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755543+0000","last_undegraded":"2023-07-06T03:53:25.755543+0000","last_fullsized":"2023-07-06T03:53:25.755543+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.769357+0000","last_change":"2023-07-06T03:53:25.764949+0000","last_active":"2023-07-06T03:53:25.769357+0000","last_peered":"2023-07-06T03:53:25.769357+0000","last_clean":"2023-07-06T03:53:25.769357+0000","last_became_active":"2023-07-06T03:53:25.764699+0000","last_became_peered":"2023-07-06T03:53:25.764699+0000","last_unstale":"2023-07-06T03:53:25.769357+0000","last_undegraded":"2023-07-06T03:53:25.769357+0000","last_fullsized":"2023-07-06T03:53:25.769357+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:26:52.325913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.752342+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:53:25.752342+0000","last_peered":"2023-07-06T03:53:25.752342+0000","last_clean":"2023-07-06T03:53:25.752342+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:53:25.752342+0000","last_undegraded":"2023-07-06T03:53:25.752342+0000","last_fullsized":"2023-07-06T03:53:25.752342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:01:00.052091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749653+0000","last_change":"2023-07-06T03:53:25.749653+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749653+0000","last_undegraded":"2023-07-06T03:53:25.749653+0000","last_fullsized":"2023-07-06T03:53:25.749653+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766487+0000","last_change":"2023-07-06T03:53:25.763534+0000","last_active":"2023-07-06T03:53:25.766487+0000","last_peered":"2023-07-06T03:53:25.766487+0000","last_clean":"2023-07-06T03:53:25.766487+0000","last_became_active":"2023-07-06T03:53:25.762916+0000","last_became_peered":"2023-07-06T03:53:25.762916+0000","last_unstale":"2023-07-06T03:53:25.766487+0000","last_undegraded":"2023-07-06T03:53:25.766487+0000","last_fullsized":"2023-07-06T03:53:25.766487+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:53.585014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.751388+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:53:25.751388+0000","last_peered":"2023-07-06T03:53:25.751388+0000","last_clean":"2023-07-06T03:53:25.751388+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:53:25.751388+0000","last_undegraded":"2023-07-06T03:53:25.751388+0000","last_fullsized":"2023-07-06T03:53:25.751388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:54.214309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756568+0000","last_change":"2023-07-06T03:53:25.756568+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756568+0000","last_undegraded":"2023-07-06T03:53:25.756568+0000","last_fullsized":"2023-07-06T03:53:25.756568+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768632+0000","last_change":"2023-07-06T03:53:25.763463+0000","last_active":"2023-07-06T03:53:25.768632+0000","last_peered":"2023-07-06T03:53:25.768632+0000","last_clean":"2023-07-06T03:53:25.768632+0000","last_became_active":"2023-07-06T03:53:25.763132+0000","last_became_peered":"2023-07-06T03:53:25.763132+0000","last_unstale":"2023-07-06T03:53:25.768632+0000","last_undegraded":"2023-07-06T03:53:25.768632+0000","last_fullsized":"2023-07-06T03:53:25.768632+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:50:21.608182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.752190+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:53:25.752190+0000","last_peered":"2023-07-06T03:53:25.752190+0000","last_clean":"2023-07-06T03:53:25.752190+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:53:25.752190+0000","last_undegraded":"2023-07-06T03:53:25.752190+0000","last_fullsized":"2023-07-06T03:53:25.752190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:37:42.592191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":301,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.744734+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:53:25.744734+0000","last_peered":"2023-07-06T03:53:25.744734+0000","last_clean":"2023-07-06T03:53:25.744734+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:53:25.744734+0000","last_undegraded":"2023-07-06T03:53:25.744734+0000","last_fullsized":"2023-07-06T03:53:25.744734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:57:56.581947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747510+0000","last_change":"2023-07-06T03:53:25.747510+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747510+0000","last_undegraded":"2023-07-06T03:53:25.747510+0000","last_fullsized":"2023-07-06T03:53:25.747510+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.762283+0000","last_change":"2023-07-06T03:53:25.758578+0000","last_active":"2023-07-06T03:53:25.762283+0000","last_peered":"2023-07-06T03:53:25.762283+0000","last_clean":"2023-07-06T03:53:25.762283+0000","last_became_active":"2023-07-06T03:53:25.758327+0000","last_became_peered":"2023-07-06T03:53:25.758327+0000","last_unstale":"2023-07-06T03:53:25.762283+0000","last_undegraded":"2023-07-06T03:53:25.762283+0000","last_fullsized":"2023-07-06T03:53:25.762283+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:11.379795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.752104+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:53:25.752104+0000","last_peered":"2023-07-06T03:53:25.752104+0000","last_clean":"2023-07-06T03:53:25.752104+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:53:25.752104+0000","last_undegraded":"2023-07-06T03:53:25.752104+0000","last_fullsized":"2023-07-06T03:53:25.752104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:27:44.192794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.754828+0000","last_change":"2023-07-06T03:53:25.754828+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.754828+0000","last_undegraded":"2023-07-06T03:53:25.754828+0000","last_fullsized":"2023-07-06T03:53:25.754828+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767980+0000","last_change":"2023-07-06T03:53:25.763925+0000","last_active":"2023-07-06T03:53:25.767980+0000","last_peered":"2023-07-06T03:53:25.767980+0000","last_clean":"2023-07-06T03:53:25.767980+0000","last_became_active":"2023-07-06T03:53:25.763655+0000","last_became_peered":"2023-07-06T03:53:25.763655+0000","last_unstale":"2023-07-06T03:53:25.767980+0000","last_undegraded":"2023-07-06T03:53:25.767980+0000","last_fullsized":"2023-07-06T03:53:25.767980+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:33:43.766965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.753207+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:53:25.753207+0000","last_peered":"2023-07-06T03:53:25.753207+0000","last_clean":"2023-07-06T03:53:25.753207+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:53:25.753207+0000","last_undegraded":"2023-07-06T03:53:25.753207+0000","last_fullsized":"2023-07-06T03:53:25.753207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:25:29.753235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751739+0000","last_change":"2023-07-06T03:53:25.751739+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751739+0000","last_undegraded":"2023-07-06T03:53:25.751739+0000","last_fullsized":"2023-07-06T03:53:25.751739+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768560+0000","last_change":"2023-07-06T03:53:25.765118+0000","last_active":"2023-07-06T03:53:25.768560+0000","last_peered":"2023-07-06T03:53:25.768560+0000","last_clean":"2023-07-06T03:53:25.768560+0000","last_became_active":"2023-07-06T03:53:25.764820+0000","last_became_peered":"2023-07-06T03:53:25.764820+0000","last_unstale":"2023-07-06T03:53:25.768560+0000","last_undegraded":"2023-07-06T03:53:25.768560+0000","last_fullsized":"2023-07-06T03:53:25.768560+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:28.563800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":150,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.756204+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:53:25.756204+0000","last_peered":"2023-07-06T03:53:25.756204+0000","last_clean":"2023-07-06T03:53:25.756204+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:53:25.756204+0000","last_undegraded":"2023-07-06T03:53:25.756204+0000","last_fullsized":"2023-07-06T03:53:25.756204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:34:25.191156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752080+0000","last_change":"2023-07-06T03:53:25.752080+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752080+0000","last_undegraded":"2023-07-06T03:53:25.752080+0000","last_fullsized":"2023-07-06T03:53:25.752080+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768745+0000","last_change":"2023-07-06T03:53:25.766728+0000","last_active":"2023-07-06T03:53:25.768745+0000","last_peered":"2023-07-06T03:53:25.768745+0000","last_clean":"2023-07-06T03:53:25.768745+0000","last_became_active":"2023-07-06T03:53:25.766327+0000","last_became_peered":"2023-07-06T03:53:25.766327+0000","last_unstale":"2023-07-06T03:53:25.768745+0000","last_undegraded":"2023-07-06T03:53:25.768745+0000","last_fullsized":"2023-07-06T03:53:25.768745+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:07:23.865989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.754247+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:53:25.754247+0000","last_peered":"2023-07-06T03:53:25.754247+0000","last_clean":"2023-07-06T03:53:25.754247+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:53:25.754247+0000","last_undegraded":"2023-07-06T03:53:25.754247+0000","last_fullsized":"2023-07-06T03:53:25.754247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:07:44.809785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":1200128,"data_stored":545111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.45900000000000002,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":1.024,"5min":4.0570000000000004,"15min":4.0570000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67900000000000005,"15min":0.76600000000000001},"min":{"1min":0.51200000000000001,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.89700000000000002,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67800000000000005,"15min":0.72399999999999998},"min":{"1min":0.47199999999999998,"5min":0.45000000000000001,"15min":0.34000000000000002},"max":{"1min":1.091,"5min":1.091,"15min":2.6219999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70499999999999996,"15min":0.84599999999999997},"min":{"1min":0.47099999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94999999999999996,"5min":4.0389999999999997,"15min":6.6310000000000002},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.39700000000000002,"15min":0.35199999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":3.77},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.45000000000000001},"max":{"1min":0.93000000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.48699999999999999,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75,"15min":0.753},"min":{"1min":0.57599999999999996,"5min":0.45400000000000001,"15min":0.43099999999999999},"max":{"1min":0.92700000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.65800000000000003,"15min":0.70499999999999996},"min":{"1min":0.44600000000000001,"5min":0.38,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":0.95999999999999996,"15min":1.4590000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.69499999999999995},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0820000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82599999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.755}]},{"osd":6,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72499999999999998,"15min":0.73999999999999999},"min":{"1min":0.46400000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":1.03,"15min":1.5349999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.51800000000000002,"5min":0.45200000000000001,"15min":0.42299999999999999},"max":{"1min":0.98199999999999998,"5min":1.006,"15min":1.3779999999999999},"last":0.55300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":1200128,"data_stored":545111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70899999999999996,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.29399999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77200000000000002,"15min":0.73999999999999999},"min":{"1min":0.44700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.2889999999999999,"5min":3.3399999999999999,"15min":3.3399999999999999},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86099999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":1.1759999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.74399999999999999,"15min":0.68200000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":1.9890000000000001,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.9450000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.76100000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69999999999999996,"15min":0.745},"min":{"1min":0.502,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":2.1850000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.53400000000000003,"5min":0.40500000000000003,"15min":0.373},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":0.64300000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.70699999999999996},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.33200000000000002},"max":{"1min":0.92100000000000004,"5min":1.0309999999999999,"15min":1.1339999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67300000000000004,"15min":0.72499999999999998},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":0.98699999999999999,"15min":1.085},"last":0.623}]},{"osd":5,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.443,"15min":0.38600000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.218},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.77700000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":1.0489999999999999,"15min":1.198},"last":0.81599999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.80500000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97399999999999998,"5min":1.054,"15min":1.323},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.81599999999999995},"min":{"1min":0.54700000000000004,"5min":0.504,"15min":0.41499999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.4139999999999999},"last":0.74199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1752,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1794048,"data_stored":1135479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.52400000000000002},"min":{"1min":0.26900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65100000000000002,"5min":1.002,"15min":2.8879999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.501,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.313},"max":{"1min":0.58099999999999996,"5min":0.78300000000000003,"15min":1.96},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45500000000000002,"15min":0.83099999999999996},"min":{"1min":0.313,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.57799999999999996,"5min":0.70799999999999996,"15min":13.055999999999999},"last":0.375},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54500000000000004,"15min":0.621},"min":{"1min":0.35699999999999998,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":2.8479999999999999},"last":0.497}]},{"osd":3,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48099999999999998,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.71599999999999997,"15min":1.2709999999999999},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.73999999999999999,"15min":1.101},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.86199999999999999,"5min":1.0640000000000001,"15min":12.989000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.78700000000000003,"15min":0.82599999999999996},"min":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.46000000000000002},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.502,"5min":0.432,"15min":0.432},"max":{"1min":0.91200000000000003,"5min":1.0860000000000001,"15min":1.3100000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.71699999999999997,"15min":1.101},"min":{"1min":0.433,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.94499999999999995,"5min":1.1020000000000001,"15min":13.023999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":0.81000000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76700000000000002,"15min":1.147},"min":{"1min":0.54000000000000004,"5min":0.48699999999999999,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":1.0569999999999999,"15min":12.941000000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75800000000000001,"15min":0.81999999999999995},"min":{"1min":0.58399999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.071,"5min":1.071,"15min":2.806},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.751,"15min":0.80300000000000005},"min":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.83999999999999997,"5min":1.1319999999999999,"15min":1.337},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.28000000000000003,"5min":0.26400000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.753,"15min":0.753},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.439},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.248},"max":{"1min":0.51200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38}]},{"osd":2,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.29599999999999999},"max":{"1min":0.52400000000000002,"5min":0.69199999999999995,"15min":0.73699999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.48399999999999999,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.314},"max":{"1min":0.55800000000000005,"5min":0.67200000000000004,"15min":1.2030000000000001},"last":0.442}]},{"osd":3,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.32200000000000001,"5min":0.30099999999999999,"15min":0.29899999999999999},"max":{"1min":0.56200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.63400000000000001,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":1.1699999999999999,"15min":1.171},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":0.88800000000000001,"5min":1.032,"15min":1.034},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.745,"15min":0.73199999999999998},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":0.90600000000000003,"5min":1.1080000000000001,"15min":1.1419999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72899999999999998,"15min":0.747},"min":{"1min":0.56399999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93700000000000006,"5min":1.1040000000000001,"15min":1.1220000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.51600000000000001},"max":{"1min":0.93300000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73599999999999999,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81299999999999994,"5min":1.0649999999999999,"15min":1.546},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":0.86599999999999999,"5min":1.0820000000000001,"15min":1.6830000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77400000000000002,"15min":1.0269999999999999},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":0.86199999999999999,"5min":1.0669999999999999,"15min":9.5960000000000001},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40400,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331440,"statfs":{"total":96636764160,"available":96595394560,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.47999999999999998,"15min":0.443},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.50800000000000001,"15min":0.48199999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.093,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.375}]},{"osd":1,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.55000000000000004,"15min":0.51100000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":6.5960000000000001,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.62},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.57099999999999995,"15min":0.52400000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.657,"5min":3.657,"15min":3.657},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.63200000000000001,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":6.6390000000000002,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.1629999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":0.505}]},{"osd":4,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":0.94299999999999995,"15min":0.78100000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":12.007,"5min":12.007,"15min":12.007},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81899999999999995,"15min":0.73199999999999998},"min":{"1min":0.49299999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.2200000000000002,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.79300000000000004,"15min":0.749},"min":{"1min":0.495,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.1830000000000001,"5min":0.88300000000000001,"15min":0.79700000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.8440000000000003,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":0.94299999999999995,"15min":0.78600000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":11.696999999999999,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":1.4390000000000001,"5min":0.92400000000000004,"15min":0.80800000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":13.084,"5min":13.084,"15min":13.084},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.83299999999999996,"15min":0.83099999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.042,"5min":0.86499999999999999,"15min":0.81999999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":4.484,"5min":4.484,"15min":4.484},"last":0.58399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.44900000000000001,"15min":0.58499999999999996},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.501}]},{"osd":1,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45600000000000002,"15min":0.94899999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":15.911},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":1.0209999999999999},"min":{"1min":0.40300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.347,"5min":1.347,"15min":16.024000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.33600000000000002,"5min":0.32800000000000001,"15min":0.27400000000000002},"max":{"1min":3.25,"5min":3.25,"15min":3.25},"last":0.437},{"interface":"front","average":{"1min":0.63,"5min":0.54300000000000004,"15min":1.028},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.40699999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73999999999999999,"15min":0.75600000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.90300000000000002},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.36699999999999999},"max":{"1min":1.724,"5min":1.724,"15min":5.7300000000000004},"last":0.92400000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75600000000000001,"15min":0.90500000000000003},"min":{"1min":0.49199999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.411,"5min":1.411,"15min":5.077},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78900000000000003,"15min":0.95699999999999996},"min":{"1min":0.57999999999999996,"5min":0.499,"15min":0.44800000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.85199999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76200000000000001,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.91400000000000003},"min":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.45600000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":5.157},"last":0.70199999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.82199999999999995},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76600000000000001,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.492,"5min":1.492,"15min":5.1189999999999998},"last":0.72799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.41599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80700000000000005,"5min":1.784,"15min":1.784},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.38},"max":{"1min":0.81899999999999995,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.82799999999999996},"min":{"1min":0.39500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":6.258},"last":0.748},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68000000000000005,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":1.806,"15min":1.806},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.84599999999999997},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.29199999999999998},"max":{"1min":0.90100000000000002,"5min":0.91000000000000003,"15min":6.4740000000000002},"last":0.629},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73799999999999999,"15min":0.92300000000000004},"min":{"1min":0.57699999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":6.4859999999999998},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.48499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.97799999999999998,"15min":1.25},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.503,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":0.99099999999999999,"15min":1.413},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59999999999999998,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77800000000000002,"5min":0.92100000000000004,"15min":1.3280000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.66000000000000003},"min":{"1min":0.50600000000000001,"5min":0.40200000000000002,"15min":0.36399999999999999},"max":{"1min":0.80200000000000005,"5min":0.94299999999999995,"15min":0.99299999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63700000000000001,"15min":0.66200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.81999999999999995,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.78600000000000003},"min":{"1min":0.52400000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":1.008,"15min":1.6839999999999999},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.72299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41299999999999998},"max":{"1min":0.88200000000000001,"5min":0.95699999999999996,"15min":1.0660000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70399999999999996,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.99399999999999999,"5min":1.079,"15min":1.21},"last":0.53500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36880,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334960,"statfs":{"total":96636764160,"available":96598999040,"internally_reserved":0,"allocated":1777664,"data_stored":1127832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70599999999999996,"15min":0.66100000000000003},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.32100000000000001},"max":{"1min":0.871,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68300000000000005,"15min":0.83299999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":6.4059999999999997},"last":0.70699999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.745},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":1.1970000000000001,"15min":1.546},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.79100000000000004,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.376},"max":{"1min":8.1120000000000001,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.84399999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.80800000000000005,"15min":0.73999999999999999},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":8.1829999999999998,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73999999999999999,"15min":0.747},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":1.3140000000000001,"5min":4.484,"15min":4.484},"last":0.80000000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.627,"15min":0.628},"min":{"1min":0.46200000000000002,"5min":0.40899999999999997,"15min":0.34699999999999998},"max":{"1min":0.88500000000000001,"5min":1.1359999999999999,"15min":1.6240000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.42499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.89700000000000002,"5min":1.0569999999999999,"15min":2.718},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.48399999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91200000000000003,"5min":1.246,"15min":1.669},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.002,"5min":1.3360000000000001,"15min":2.0459999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.749,"15min":0.77800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0609999999999999,"15min":2.2919999999999998},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:53:31.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:53:31.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.835+0000 7faced5ff700 1 Processor -- start 2023-07-06T03:53:31.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.835+0000 7faced5ff700 1 -- start start 2023-07-06T03:53:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.836+0000 7faced5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7face815c520 0x7face80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.836+0000 7faced5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face8160d00 0x7face80b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.836+0000 7faced5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face80cdae0 con 0x7face8160180 2023-07-06T03:53:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.836+0000 7faced5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face80cd8d0 con 0x7face815c520 2023-07-06T03:53:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.836+0000 7faced5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face80b6640 con 0x7face8160d00 2023-07-06T03:53:31.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face67fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7face8160180 0x7face81605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51266/0 (socket says 172.21.15.97:51266) 2023-07-06T03:53:31.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face67fc700 1 -- 172.21.15.97:0/2420676173 learned_addr learned my addr 172.21.15.97:0/2420676173 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:31.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face77fe700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face8160d00 0x7face80b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face6ffd700 1 --2- 172.21.15.97:0/2420676173 >> v2:172.21.15.146:3300/0 conn(0x7face815c520 0x7face80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105588773 0 0) 0x7face80cdae0 con 0x7face8160180 2023-07-06T03:53:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facd00035b0 con 0x7face8160180 2023-07-06T03:53:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2846844205 0 0) 0x7facd00035b0 con 0x7face8160180 2023-07-06T03:53:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face8160d00 msgr2=0x7face80b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face5ffb700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face8160d00 0x7face80b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 >> v2:172.21.15.146:3300/0 conn(0x7face815c520 msgr2=0x7face80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:31.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face5ffb700 1 --2- 172.21.15.97:0/2420676173 >> v2:172.21.15.146:3300/0 conn(0x7face815c520 0x7face80cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face80b6920 con 0x7face8160180 2023-07-06T03:53:31.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.837+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7facdc001670 con 0x7face8160180 2023-07-06T03:53:31.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 >> v1:172.21.15.97:6789/0 conn(0x7face8160180 legacy=0x7face81605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:31.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face5ffb700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facd0003a00 0x7facd0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face5ffb700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd0004430 0x7facd00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face5ffb700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face8160d00 0x7facd0004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80cd8d0 con 0x7facd0003a00 2023-07-06T03:53:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80b6640 con 0x7facd0004430 2023-07-06T03:53:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd000ba10 con 0x7face8160d00 2023-07-06T03:53:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face6ffd700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facd0003a00 0x7facd0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face67fc700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd0004430 0x7facd00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face77fe700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face8160d00 0x7facd0004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face6ffd700 1 -- 172.21.15.97:0/2420676173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face8160d00 msgr2=0x7facd0004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face6ffd700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face8160d00 0x7facd0004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face6ffd700 1 -- 172.21.15.97:0/2420676173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd0004430 msgr2=0x7facd00048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face6ffd700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd0004430 0x7facd00048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face6ffd700 1 -- 172.21.15.97:0/2420676173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd00035b0 con 0x7facd0003a00 2023-07-06T03:53:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.838+0000 7face6ffd700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facd0003a00 0x7facd0003e10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7facd8009b60 tx=0x7facd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1249e211180a6053 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.839+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800e6c0 con 0x7facd0003a00 2023-07-06T03:53:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.839+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd800e820 con 0x7facd0003a00 2023-07-06T03:53:31.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.839+0000 7face5ffb700 1 -- 172.21.15.97:0/2420676173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800eb90 con 0x7facd0003a00 2023-07-06T03:53:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.839+0000 7faced5ff700 1 -- 172.21.15.97:0/2420676173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facd0003a00 msgr2=0x7facd0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.839+0000 7faced5ff700 1 --2- 172.21.15.97:0/2420676173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facd0003a00 0x7facd0003e10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7facd8009b60 tx=0x7facd0005880 comp rx=0 tx=0).stop 2023-07-06T03:53:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.839+0000 7face77fe700 1 -- 172.21.15.97:0/2420676173 reap_dead start 2023-07-06T03:53:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.839+0000 7faced5ff700 1 -- 172.21.15.97:0/2420676173 shutdown_connections 2023-07-06T03:53:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.839+0000 7faced5ff700 1 -- 172.21.15.97:0/2420676173 >> 172.21.15.97:0/2420676173 conn(0x7face8071790 msgr2=0x7face8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:31.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.839+0000 7faced5ff700 1 -- 172.21.15.97:0/2420676173 shutdown_connections 2023-07-06T03:53:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.839+0000 7faced5ff700 1 -- 172.21.15.97:0/2420676173 wait complete. 2023-07-06T03:53:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.841+0000 7faced5ff700 1 Processor -- start 2023-07-06T03:53:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.841+0000 7faced5ff700 1 -- start start 2023-07-06T03:53:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.842+0000 7faced5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd0003a00 0x7face814d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.842+0000 7faced5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face80c77b0 0x7face815f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.842+0000 7faced5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face815f710 0x7face80d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.842+0000 7faced5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face815edb0 con 0x7face815f710 2023-07-06T03:53:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.842+0000 7faced5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face815df00 con 0x7facd0003a00 2023-07-06T03:53:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.842+0000 7faced5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face815e130 con 0x7face80c77b0 2023-07-06T03:53:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.842+0000 7face77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face815f710 0x7face80d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.842+0000 7face77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face815f710 0x7face80d1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59454/0 (socket says 172.21.15.97:59454) 2023-07-06T03:53:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.842+0000 7face77fe700 1 -- 172.21.15.97:0/1069946377 learned_addr learned my addr 172.21.15.97:0/1069946377 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7face6ffd700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd0003a00 0x7face814d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7face67fc700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face80c77b0 0x7face815f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7face77fe700 1 -- 172.21.15.97:0/1069946377 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face80c77b0 msgr2=0x7face815f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:31.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7face77fe700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face80c77b0 0x7face815f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7face77fe700 1 -- 172.21.15.97:0/1069946377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd0003a00 msgr2=0x7face814d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:31.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7face77fe700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd0003a00 0x7face814d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:31.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7face77fe700 1 -- 172.21.15.97:0/1069946377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face815e410 con 0x7face815f710 2023-07-06T03:53:31.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7face6ffd700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd0003a00 0x7face814d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:31.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7face77fe700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face815f710 0x7face80d1dc0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7face81610d0 tx=0x7face00114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:31.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7faccf7fe700 1 -- 172.21.15.97:0/1069946377 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face000e070 con 0x7face815f710 2023-07-06T03:53:31.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7faccf7fe700 1 -- 172.21.15.97:0/1069946377 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face0003cf0 con 0x7face815f710 2023-07-06T03:53:31.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7faced5ff700 1 -- 172.21.15.97:0/1069946377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face815e6f0 con 0x7face815f710 2023-07-06T03:53:31.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7faccf7fe700 1 -- 172.21.15.97:0/1069946377 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0003ea0 con 0x7face815f710 2023-07-06T03:53:31.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.843+0000 7faced5ff700 1 -- 172.21.15.97:0/1069946377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face814d9e0 con 0x7face815f710 2023-07-06T03:53:31.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.845+0000 7faced5ff700 1 -- 172.21.15.97:0/1069946377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face815eef0 con 0x7face815f710 2023-07-06T03:53:31.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.845+0000 7faccf7fe700 1 -- 172.21.15.97:0/1069946377 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7face0011a10 con 0x7face815f710 2023-07-06T03:53:31.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.846+0000 7faccf7fe700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7facd406ace0 0x7facd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:31.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.846+0000 7faccf7fe700 1 -- 172.21.15.97:0/1069946377 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7face0089940 con 0x7face815f710 2023-07-06T03:53:31.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.847+0000 7face6ffd700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7facd406ace0 0x7facd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:31.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.847+0000 7face6ffd700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7facd406ace0 0x7facd406d190 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7facd80090b0 tx=0x7facd8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:31.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:31.850+0000 7faccf7fe700 1 -- 172.21.15.97:0/1069946377 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7face815eef0 con 0x7face815f710 2023-07-06T03:53:32.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.041+0000 7faced5ff700 1 -- 172.21.15.97:0/1069946377 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7face80b6ee0 con 0x7facd406ace0 2023-07-06T03:53:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.050+0000 7faccf7fe700 1 -- 172.21.15.97:0/1069946377 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306045 (secure 0 0 0) 0x7facd8022040 con 0x7facd406ace0 2023-07-06T03:53:32.054 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:32.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 -- 172.21.15.97:0/1069946377 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7facd406ace0 msgr2=0x7facd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:32.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7facd406ace0 0x7facd406d190 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7facd80090b0 tx=0x7facd8008030 comp rx=0 tx=0).stop 2023-07-06T03:53:32.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 -- 172.21.15.97:0/1069946377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face815f710 msgr2=0x7face80d1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:32.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face815f710 0x7face80d1dc0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7face81610d0 tx=0x7face00114b0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 -- 172.21.15.97:0/1069946377 shutdown_connections 2023-07-06T03:53:32.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7facd406ace0 0x7facd406d190 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face815f710 0x7face80d1dc0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facd0003a00 0x7face814d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 --2- 172.21.15.97:0/1069946377 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7face80c77b0 0x7face815f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 -- 172.21.15.97:0/1069946377 >> 172.21.15.97:0/1069946377 conn(0x7face8071790 msgr2=0x7face80b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:32.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 -- 172.21.15.97:0/1069946377 shutdown_connections 2023-07-06T03:53:32.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.054+0000 7faced5ff700 1 -- 172.21.15.97:0/1069946377 wait complete. 2023-07-06T03:53:32.058 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:53:32.074 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2023-07-06T03:53:30.992395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283432,"kb_used_data":10472,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754691288,"statfs":{"total":773094113280,"available":772803878912,"internally_reserved":0,"allocated":10723328,"data_stored":5507465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001028"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755228+0000","last_change":"2023-07-06T03:53:25.755228+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755228+0000","last_undegraded":"2023-07-06T03:53:25.755228+0000","last_fullsized":"2023-07-06T03:53:25.755228+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750519+0000","last_change":"2023-07-06T03:53:25.750519+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750519+0000","last_undegraded":"2023-07-06T03:53:25.750519+0000","last_fullsized":"2023-07-06T03:53:25.750519+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.748603+0000","last_change":"2023-07-06T03:53:25.748603+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.748603+0000","last_undegraded":"2023-07-06T03:53:25.748603+0000","last_fullsized":"2023-07-06T03:53:25.748603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756199+0000","last_change":"2023-07-06T03:53:25.756199+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756199+0000","last_undegraded":"2023-07-06T03:53:25.756199+0000","last_fullsized":"2023-07-06T03:53:25.756199+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750436+0000","last_change":"2023-07-06T03:53:25.750436+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750436+0000","last_undegraded":"2023-07-06T03:53:25.750436+0000","last_fullsized":"2023-07-06T03:53:25.750436+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756992+0000","last_change":"2023-07-06T03:53:25.756992+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756992+0000","last_undegraded":"2023-07-06T03:53:25.756992+0000","last_fullsized":"2023-07-06T03:53:25.756992+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755696+0000","last_change":"2023-07-06T03:53:25.755696+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755696+0000","last_undegraded":"2023-07-06T03:53:25.755696+0000","last_fullsized":"2023-07-06T03:53:25.755696+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756528+0000","last_change":"2023-07-06T03:53:25.756528+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756528+0000","last_undegraded":"2023-07-06T03:53:25.756528+0000","last_fullsized":"2023-07-06T03:53:25.756528+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750207+0000","last_change":"2023-07-06T03:53:25.750207+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750207+0000","last_undegraded":"2023-07-06T03:53:25.750207+0000","last_fullsized":"2023-07-06T03:53:25.750207+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757208+0000","last_change":"2023-07-06T03:53:25.757208+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757208+0000","last_undegraded":"2023-07-06T03:53:25.757208+0000","last_fullsized":"2023-07-06T03:53:25.757208+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752148+0000","last_change":"2023-07-06T03:53:25.752148+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752148+0000","last_undegraded":"2023-07-06T03:53:25.752148+0000","last_fullsized":"2023-07-06T03:53:25.752148+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756929+0000","last_change":"2023-07-06T03:53:25.756929+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756929+0000","last_undegraded":"2023-07-06T03:53:25.756929+0000","last_fullsized":"2023-07-06T03:53:25.756929+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749997+0000","last_change":"2023-07-06T03:53:25.749997+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749997+0000","last_undegraded":"2023-07-06T03:53:25.749997+0000","last_fullsized":"2023-07-06T03:53:25.749997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756564+0000","last_change":"2023-07-06T03:53:25.756564+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756564+0000","last_undegraded":"2023-07-06T03:53:25.756564+0000","last_fullsized":"2023-07-06T03:53:25.756564+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750063+0000","last_change":"2023-07-06T03:53:25.750063+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750063+0000","last_undegraded":"2023-07-06T03:53:25.750063+0000","last_fullsized":"2023-07-06T03:53:25.750063+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747769+0000","last_change":"2023-07-06T03:53:25.747769+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747769+0000","last_undegraded":"2023-07-06T03:53:25.747769+0000","last_fullsized":"2023-07-06T03:53:25.747769+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749664+0000","last_change":"2023-07-06T03:53:25.749664+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749664+0000","last_undegraded":"2023-07-06T03:53:25.749664+0000","last_fullsized":"2023-07-06T03:53:25.749664+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751017+0000","last_change":"2023-07-06T03:53:25.751017+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751017+0000","last_undegraded":"2023-07-06T03:53:25.751017+0000","last_fullsized":"2023-07-06T03:53:25.751017+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756675+0000","last_change":"2023-07-06T03:53:25.756675+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756675+0000","last_undegraded":"2023-07-06T03:53:25.756675+0000","last_fullsized":"2023-07-06T03:53:25.756675+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747503+0000","last_change":"2023-07-06T03:53:25.747503+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747503+0000","last_undegraded":"2023-07-06T03:53:25.747503+0000","last_fullsized":"2023-07-06T03:53:25.747503+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750618+0000","last_change":"2023-07-06T03:53:25.750618+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750618+0000","last_undegraded":"2023-07-06T03:53:25.750618+0000","last_fullsized":"2023-07-06T03:53:25.750618+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751059+0000","last_change":"2023-07-06T03:53:25.751059+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751059+0000","last_undegraded":"2023-07-06T03:53:25.751059+0000","last_fullsized":"2023-07-06T03:53:25.751059+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756057+0000","last_change":"2023-07-06T03:53:25.756057+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756057+0000","last_undegraded":"2023-07-06T03:53:25.756057+0000","last_fullsized":"2023-07-06T03:53:25.756057+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756069+0000","last_change":"2023-07-06T03:53:25.756069+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756069+0000","last_undegraded":"2023-07-06T03:53:25.756069+0000","last_fullsized":"2023-07-06T03:53:25.756069+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747561+0000","last_change":"2023-07-06T03:53:25.747561+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747561+0000","last_undegraded":"2023-07-06T03:53:25.747561+0000","last_fullsized":"2023-07-06T03:53:25.747561+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747500+0000","last_change":"2023-07-06T03:53:25.747500+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747500+0000","last_undegraded":"2023-07-06T03:53:25.747500+0000","last_fullsized":"2023-07-06T03:53:25.747500+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747753+0000","last_change":"2023-07-06T03:53:25.747753+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747753+0000","last_undegraded":"2023-07-06T03:53:25.747753+0000","last_fullsized":"2023-07-06T03:53:25.747753+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756499+0000","last_change":"2023-07-06T03:53:25.756499+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756499+0000","last_undegraded":"2023-07-06T03:53:25.756499+0000","last_fullsized":"2023-07-06T03:53:25.756499+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757282+0000","last_change":"2023-07-06T03:53:25.757282+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757282+0000","last_undegraded":"2023-07-06T03:53:25.757282+0000","last_fullsized":"2023-07-06T03:53:25.757282+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750863+0000","last_change":"2023-07-06T03:53:25.750863+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750863+0000","last_undegraded":"2023-07-06T03:53:25.750863+0000","last_fullsized":"2023-07-06T03:53:25.750863+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757028+0000","last_change":"2023-07-06T03:53:25.757028+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757028+0000","last_undegraded":"2023-07-06T03:53:25.757028+0000","last_fullsized":"2023-07-06T03:53:25.757028+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751127+0000","last_change":"2023-07-06T03:53:25.751127+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751127+0000","last_undegraded":"2023-07-06T03:53:25.751127+0000","last_fullsized":"2023-07-06T03:53:25.751127+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750047+0000","last_change":"2023-07-06T03:53:25.750047+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750047+0000","last_undegraded":"2023-07-06T03:53:25.750047+0000","last_fullsized":"2023-07-06T03:53:25.750047+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.760618+0000","last_change":"2023-07-06T03:53:25.758687+0000","last_active":"2023-07-06T03:53:25.760618+0000","last_peered":"2023-07-06T03:53:25.760618+0000","last_clean":"2023-07-06T03:53:25.760618+0000","last_became_active":"2023-07-06T03:53:25.758351+0000","last_became_peered":"2023-07-06T03:53:25.758351+0000","last_unstale":"2023-07-06T03:53:25.760618+0000","last_undegraded":"2023-07-06T03:53:25.760618+0000","last_fullsized":"2023-07-06T03:53:25.760618+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:19:42.419643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755665+0000","last_change":"2023-07-06T03:53:25.755665+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755665+0000","last_undegraded":"2023-07-06T03:53:25.755665+0000","last_fullsized":"2023-07-06T03:53:25.755665+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768649+0000","last_change":"2023-07-06T03:53:25.765786+0000","last_active":"2023-07-06T03:53:25.768649+0000","last_peered":"2023-07-06T03:53:25.768649+0000","last_clean":"2023-07-06T03:53:25.768649+0000","last_became_active":"2023-07-06T03:53:25.764804+0000","last_became_peered":"2023-07-06T03:53:25.764804+0000","last_unstale":"2023-07-06T03:53:25.768649+0000","last_undegraded":"2023-07-06T03:53:25.768649+0000","last_fullsized":"2023-07-06T03:53:25.768649+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:41.234580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751044+0000","last_change":"2023-07-06T03:53:25.751044+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751044+0000","last_undegraded":"2023-07-06T03:53:25.751044+0000","last_fullsized":"2023-07-06T03:53:25.751044+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.763907+0000","last_change":"2023-07-06T03:53:25.761839+0000","last_active":"2023-07-06T03:53:25.763907+0000","last_peered":"2023-07-06T03:53:25.763907+0000","last_clean":"2023-07-06T03:53:25.763907+0000","last_became_active":"2023-07-06T03:53:25.761316+0000","last_became_peered":"2023-07-06T03:53:25.761316+0000","last_unstale":"2023-07-06T03:53:25.763907+0000","last_undegraded":"2023-07-06T03:53:25.763907+0000","last_fullsized":"2023-07-06T03:53:25.763907+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:40.155881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750909+0000","last_change":"2023-07-06T03:53:25.750909+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750909+0000","last_undegraded":"2023-07-06T03:53:25.750909+0000","last_fullsized":"2023-07-06T03:53:25.750909+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767922+0000","last_change":"2023-07-06T03:53:25.763450+0000","last_active":"2023-07-06T03:53:25.767922+0000","last_peered":"2023-07-06T03:53:25.767922+0000","last_clean":"2023-07-06T03:53:25.767922+0000","last_became_active":"2023-07-06T03:53:25.763153+0000","last_became_peered":"2023-07-06T03:53:25.763153+0000","last_unstale":"2023-07-06T03:53:25.767922+0000","last_undegraded":"2023-07-06T03:53:25.767922+0000","last_fullsized":"2023-07-06T03:53:25.767922+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:04:52.422334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.754346+0000","last_change":"2023-07-06T03:53:25.754346+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.754346+0000","last_undegraded":"2023-07-06T03:53:25.754346+0000","last_fullsized":"2023-07-06T03:53:25.754346+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.764932+0000","last_change":"2023-07-06T03:53:25.760397+0000","last_active":"2023-07-06T03:53:25.764932+0000","last_peered":"2023-07-06T03:53:25.764932+0000","last_clean":"2023-07-06T03:53:25.764932+0000","last_became_active":"2023-07-06T03:53:25.760151+0000","last_became_peered":"2023-07-06T03:53:25.760151+0000","last_unstale":"2023-07-06T03:53:25.764932+0000","last_undegraded":"2023-07-06T03:53:25.764932+0000","last_fullsized":"2023-07-06T03:53:25.764932+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:31:09.073491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752140+0000","last_change":"2023-07-06T03:53:25.752140+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752140+0000","last_undegraded":"2023-07-06T03:53:25.752140+0000","last_fullsized":"2023-07-06T03:53:25.752140+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.761642+0000","last_change":"2023-07-06T03:53:25.758308+0000","last_active":"2023-07-06T03:53:25.761642+0000","last_peered":"2023-07-06T03:53:25.761642+0000","last_clean":"2023-07-06T03:53:25.761642+0000","last_became_active":"2023-07-06T03:53:25.758128+0000","last_became_peered":"2023-07-06T03:53:25.758128+0000","last_unstale":"2023-07-06T03:53:25.761642+0000","last_undegraded":"2023-07-06T03:53:25.761642+0000","last_fullsized":"2023-07-06T03:53:25.761642+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:09:53.722217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751283+0000","last_change":"2023-07-06T03:53:25.751283+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751283+0000","last_undegraded":"2023-07-06T03:53:25.751283+0000","last_fullsized":"2023-07-06T03:53:25.751283+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.761614+0000","last_change":"2023-07-06T03:53:25.757461+0000","last_active":"2023-07-06T03:53:25.761614+0000","last_peered":"2023-07-06T03:53:25.761614+0000","last_clean":"2023-07-06T03:53:25.761614+0000","last_became_active":"2023-07-06T03:53:25.757185+0000","last_became_peered":"2023-07-06T03:53:25.757185+0000","last_unstale":"2023-07-06T03:53:25.761614+0000","last_undegraded":"2023-07-06T03:53:25.761614+0000","last_fullsized":"2023-07-06T03:53:25.761614+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:50:21.608182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752031+0000","last_change":"2023-07-06T03:53:25.752031+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752031+0000","last_undegraded":"2023-07-06T03:53:25.752031+0000","last_fullsized":"2023-07-06T03:53:25.752031+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768039+0000","last_change":"2023-07-06T03:53:25.764596+0000","last_active":"2023-07-06T03:53:25.768039+0000","last_peered":"2023-07-06T03:53:25.768039+0000","last_clean":"2023-07-06T03:53:25.768039+0000","last_became_active":"2023-07-06T03:53:25.764126+0000","last_became_peered":"2023-07-06T03:53:25.764126+0000","last_unstale":"2023-07-06T03:53:25.768039+0000","last_undegraded":"2023-07-06T03:53:25.768039+0000","last_fullsized":"2023-07-06T03:53:25.768039+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:29:26.930120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.748903+0000","last_change":"2023-07-06T03:53:25.748903+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.748903+0000","last_undegraded":"2023-07-06T03:53:25.748903+0000","last_fullsized":"2023-07-06T03:53:25.748903+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766515+0000","last_change":"2023-07-06T03:53:25.763023+0000","last_active":"2023-07-06T03:53:25.766515+0000","last_peered":"2023-07-06T03:53:25.766515+0000","last_clean":"2023-07-06T03:53:25.766515+0000","last_became_active":"2023-07-06T03:53:25.762691+0000","last_became_peered":"2023-07-06T03:53:25.762691+0000","last_unstale":"2023-07-06T03:53:25.766515+0000","last_undegraded":"2023-07-06T03:53:25.766515+0000","last_fullsized":"2023-07-06T03:53:25.766515+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:26:33.149240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749045+0000","last_change":"2023-07-06T03:53:25.749045+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749045+0000","last_undegraded":"2023-07-06T03:53:25.749045+0000","last_fullsized":"2023-07-06T03:53:25.749045+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766673+0000","last_change":"2023-07-06T03:53:25.761618+0000","last_active":"2023-07-06T03:53:25.766673+0000","last_peered":"2023-07-06T03:53:25.766673+0000","last_clean":"2023-07-06T03:53:25.766673+0000","last_became_active":"2023-07-06T03:53:25.761270+0000","last_became_peered":"2023-07-06T03:53:25.761270+0000","last_unstale":"2023-07-06T03:53:25.766673+0000","last_undegraded":"2023-07-06T03:53:25.766673+0000","last_fullsized":"2023-07-06T03:53:25.766673+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:22:05.629100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757009+0000","last_change":"2023-07-06T03:53:25.757009+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757009+0000","last_undegraded":"2023-07-06T03:53:25.757009+0000","last_fullsized":"2023-07-06T03:53:25.757009+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.769322+0000","last_change":"2023-07-06T03:53:25.764734+0000","last_active":"2023-07-06T03:53:25.769322+0000","last_peered":"2023-07-06T03:53:25.769322+0000","last_clean":"2023-07-06T03:53:25.769322+0000","last_became_active":"2023-07-06T03:53:25.764430+0000","last_became_peered":"2023-07-06T03:53:25.764430+0000","last_unstale":"2023-07-06T03:53:25.769322+0000","last_undegraded":"2023-07-06T03:53:25.769322+0000","last_fullsized":"2023-07-06T03:53:25.769322+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:57:02.493524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749095+0000","last_change":"2023-07-06T03:53:25.749095+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749095+0000","last_undegraded":"2023-07-06T03:53:25.749095+0000","last_fullsized":"2023-07-06T03:53:25.749095+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767656+0000","last_change":"2023-07-06T03:53:25.761999+0000","last_active":"2023-07-06T03:53:25.767656+0000","last_peered":"2023-07-06T03:53:25.767656+0000","last_clean":"2023-07-06T03:53:25.767656+0000","last_became_active":"2023-07-06T03:53:25.761778+0000","last_became_peered":"2023-07-06T03:53:25.761778+0000","last_unstale":"2023-07-06T03:53:25.767656+0000","last_undegraded":"2023-07-06T03:53:25.767656+0000","last_fullsized":"2023-07-06T03:53:25.767656+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:59:22.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750034+0000","last_change":"2023-07-06T03:53:25.750034+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750034+0000","last_undegraded":"2023-07-06T03:53:25.750034+0000","last_fullsized":"2023-07-06T03:53:25.750034+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767580+0000","last_change":"2023-07-06T03:53:25.760521+0000","last_active":"2023-07-06T03:53:25.767580+0000","last_peered":"2023-07-06T03:53:25.767580+0000","last_clean":"2023-07-06T03:53:25.767580+0000","last_became_active":"2023-07-06T03:53:25.760210+0000","last_became_peered":"2023-07-06T03:53:25.760210+0000","last_unstale":"2023-07-06T03:53:25.767580+0000","last_undegraded":"2023-07-06T03:53:25.767580+0000","last_fullsized":"2023-07-06T03:53:25.767580+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:59:34.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750939+0000","last_change":"2023-07-06T03:53:25.750939+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750939+0000","last_undegraded":"2023-07-06T03:53:25.750939+0000","last_fullsized":"2023-07-06T03:53:25.750939+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766899+0000","last_change":"2023-07-06T03:53:25.761696+0000","last_active":"2023-07-06T03:53:25.766899+0000","last_peered":"2023-07-06T03:53:25.766899+0000","last_clean":"2023-07-06T03:53:25.766899+0000","last_became_active":"2023-07-06T03:53:25.761373+0000","last_became_peered":"2023-07-06T03:53:25.761373+0000","last_unstale":"2023-07-06T03:53:25.766899+0000","last_undegraded":"2023-07-06T03:53:25.766899+0000","last_fullsized":"2023-07-06T03:53:25.766899+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:53.429422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750124+0000","last_change":"2023-07-06T03:53:25.750124+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750124+0000","last_undegraded":"2023-07-06T03:53:25.750124+0000","last_fullsized":"2023-07-06T03:53:25.750124+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768578+0000","last_change":"2023-07-06T03:53:25.763182+0000","last_active":"2023-07-06T03:53:25.768578+0000","last_peered":"2023-07-06T03:53:25.768578+0000","last_clean":"2023-07-06T03:53:25.768578+0000","last_became_active":"2023-07-06T03:53:25.762812+0000","last_became_peered":"2023-07-06T03:53:25.762812+0000","last_unstale":"2023-07-06T03:53:25.768578+0000","last_undegraded":"2023-07-06T03:53:25.768578+0000","last_fullsized":"2023-07-06T03:53:25.768578+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:09:53.722217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749081+0000","last_change":"2023-07-06T03:53:25.749081+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749081+0000","last_undegraded":"2023-07-06T03:53:25.749081+0000","last_fullsized":"2023-07-06T03:53:25.749081+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766721+0000","last_change":"2023-07-06T03:53:25.761393+0000","last_active":"2023-07-06T03:53:25.766721+0000","last_peered":"2023-07-06T03:53:25.766721+0000","last_clean":"2023-07-06T03:53:25.766721+0000","last_became_active":"2023-07-06T03:53:25.761142+0000","last_became_peered":"2023-07-06T03:53:25.761142+0000","last_unstale":"2023-07-06T03:53:25.766721+0000","last_undegraded":"2023-07-06T03:53:25.766721+0000","last_fullsized":"2023-07-06T03:53:25.766721+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:55:25.226706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750655+0000","last_change":"2023-07-06T03:53:25.750655+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750655+0000","last_undegraded":"2023-07-06T03:53:25.750655+0000","last_fullsized":"2023-07-06T03:53:25.750655+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.765261+0000","last_change":"2023-07-06T03:53:25.762326+0000","last_active":"2023-07-06T03:53:25.765261+0000","last_peered":"2023-07-06T03:53:25.765261+0000","last_clean":"2023-07-06T03:53:25.765261+0000","last_became_active":"2023-07-06T03:53:25.762045+0000","last_became_peered":"2023-07-06T03:53:25.762045+0000","last_unstale":"2023-07-06T03:53:25.765261+0000","last_undegraded":"2023-07-06T03:53:25.765261+0000","last_fullsized":"2023-07-06T03:53:25.765261+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:25:13.401196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750697+0000","last_change":"2023-07-06T03:53:25.750697+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750697+0000","last_undegraded":"2023-07-06T03:53:25.750697+0000","last_fullsized":"2023-07-06T03:53:25.750697+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768678+0000","last_change":"2023-07-06T03:53:25.766271+0000","last_active":"2023-07-06T03:53:25.768678+0000","last_peered":"2023-07-06T03:53:25.768678+0000","last_clean":"2023-07-06T03:53:25.768678+0000","last_became_active":"2023-07-06T03:53:25.765967+0000","last_became_peered":"2023-07-06T03:53:25.765967+0000","last_unstale":"2023-07-06T03:53:25.768678+0000","last_undegraded":"2023-07-06T03:53:25.768678+0000","last_fullsized":"2023-07-06T03:53:25.768678+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:04:53.517066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.757521+0000","last_change":"2023-07-06T03:53:25.757521+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.757521+0000","last_undegraded":"2023-07-06T03:53:25.757521+0000","last_fullsized":"2023-07-06T03:53:25.757521+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766346+0000","last_change":"2023-07-06T03:53:25.761337+0000","last_active":"2023-07-06T03:53:25.766346+0000","last_peered":"2023-07-06T03:53:25.766346+0000","last_clean":"2023-07-06T03:53:25.766346+0000","last_became_active":"2023-07-06T03:53:25.761078+0000","last_became_peered":"2023-07-06T03:53:25.761078+0000","last_unstale":"2023-07-06T03:53:25.766346+0000","last_undegraded":"2023-07-06T03:53:25.766346+0000","last_fullsized":"2023-07-06T03:53:25.766346+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:31:55.414395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747316+0000","last_change":"2023-07-06T03:53:25.747316+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747316+0000","last_undegraded":"2023-07-06T03:53:25.747316+0000","last_fullsized":"2023-07-06T03:53:25.747316+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767576+0000","last_change":"2023-07-06T03:53:25.760855+0000","last_active":"2023-07-06T03:53:25.767576+0000","last_peered":"2023-07-06T03:53:25.767576+0000","last_clean":"2023-07-06T03:53:25.767576+0000","last_became_active":"2023-07-06T03:53:25.760495+0000","last_became_peered":"2023-07-06T03:53:25.760495+0000","last_unstale":"2023-07-06T03:53:25.767576+0000","last_undegraded":"2023-07-06T03:53:25.767576+0000","last_fullsized":"2023-07-06T03:53:25.767576+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:11.379795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755808+0000","last_change":"2023-07-06T03:53:25.755808+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755808+0000","last_undegraded":"2023-07-06T03:53:25.755808+0000","last_fullsized":"2023-07-06T03:53:25.755808+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767720+0000","last_change":"2023-07-06T03:53:25.765885+0000","last_active":"2023-07-06T03:53:25.767720+0000","last_peered":"2023-07-06T03:53:25.767720+0000","last_clean":"2023-07-06T03:53:25.767720+0000","last_became_active":"2023-07-06T03:53:25.765567+0000","last_became_peered":"2023-07-06T03:53:25.765567+0000","last_unstale":"2023-07-06T03:53:25.767720+0000","last_undegraded":"2023-07-06T03:53:25.767720+0000","last_fullsized":"2023-07-06T03:53:25.767720+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:19:58.116780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755632+0000","last_change":"2023-07-06T03:53:25.755632+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755632+0000","last_undegraded":"2023-07-06T03:53:25.755632+0000","last_fullsized":"2023-07-06T03:53:25.755632+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768727+0000","last_change":"2023-07-06T03:53:25.766585+0000","last_active":"2023-07-06T03:53:25.768727+0000","last_peered":"2023-07-06T03:53:25.768727+0000","last_clean":"2023-07-06T03:53:25.768727+0000","last_became_active":"2023-07-06T03:53:25.765934+0000","last_became_peered":"2023-07-06T03:53:25.765934+0000","last_unstale":"2023-07-06T03:53:25.768727+0000","last_undegraded":"2023-07-06T03:53:25.768727+0000","last_fullsized":"2023-07-06T03:53:25.768727+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:49.408942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755831+0000","last_change":"2023-07-06T03:53:25.755831+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755831+0000","last_undegraded":"2023-07-06T03:53:25.755831+0000","last_fullsized":"2023-07-06T03:53:25.755831+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.765034+0000","last_change":"2023-07-06T03:53:25.762440+0000","last_active":"2023-07-06T03:53:25.765034+0000","last_peered":"2023-07-06T03:53:25.765034+0000","last_clean":"2023-07-06T03:53:25.765034+0000","last_became_active":"2023-07-06T03:53:25.762126+0000","last_became_peered":"2023-07-06T03:53:25.762126+0000","last_unstale":"2023-07-06T03:53:25.765034+0000","last_undegraded":"2023-07-06T03:53:25.765034+0000","last_fullsized":"2023-07-06T03:53:25.765034+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:59:34.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750599+0000","last_change":"2023-07-06T03:53:25.750599+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750599+0000","last_undegraded":"2023-07-06T03:53:25.750599+0000","last_fullsized":"2023-07-06T03:53:25.750599+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768675+0000","last_change":"2023-07-06T03:53:25.765395+0000","last_active":"2023-07-06T03:53:25.768675+0000","last_peered":"2023-07-06T03:53:25.768675+0000","last_clean":"2023-07-06T03:53:25.768675+0000","last_became_active":"2023-07-06T03:53:25.764440+0000","last_became_peered":"2023-07-06T03:53:25.764440+0000","last_unstale":"2023-07-06T03:53:25.768675+0000","last_undegraded":"2023-07-06T03:53:25.768675+0000","last_fullsized":"2023-07-06T03:53:25.768675+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:09:04.292040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.750548+0000","last_change":"2023-07-06T03:53:25.750548+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.750548+0000","last_undegraded":"2023-07-06T03:53:25.750548+0000","last_fullsized":"2023-07-06T03:53:25.750548+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.771940+0000","last_change":"2023-07-06T03:53:25.769061+0000","last_active":"2023-07-06T03:53:25.771940+0000","last_peered":"2023-07-06T03:53:25.771940+0000","last_clean":"2023-07-06T03:53:25.771940+0000","last_became_active":"2023-07-06T03:53:25.768721+0000","last_became_peered":"2023-07-06T03:53:25.768721+0000","last_unstale":"2023-07-06T03:53:25.771940+0000","last_undegraded":"2023-07-06T03:53:25.771940+0000","last_fullsized":"2023-07-06T03:53:25.771940+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:06:02.164844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.753416+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:53:25.753416+0000","last_peered":"2023-07-06T03:53:25.753416+0000","last_clean":"2023-07-06T03:53:25.753416+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:53:25.753416+0000","last_undegraded":"2023-07-06T03:53:25.753416+0000","last_fullsized":"2023-07-06T03:53:25.753416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:43:35.855728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.755543+0000","last_change":"2023-07-06T03:53:25.755543+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.755543+0000","last_undegraded":"2023-07-06T03:53:25.755543+0000","last_fullsized":"2023-07-06T03:53:25.755543+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.769357+0000","last_change":"2023-07-06T03:53:25.764949+0000","last_active":"2023-07-06T03:53:25.769357+0000","last_peered":"2023-07-06T03:53:25.769357+0000","last_clean":"2023-07-06T03:53:25.769357+0000","last_became_active":"2023-07-06T03:53:25.764699+0000","last_became_peered":"2023-07-06T03:53:25.764699+0000","last_unstale":"2023-07-06T03:53:25.769357+0000","last_undegraded":"2023-07-06T03:53:25.769357+0000","last_fullsized":"2023-07-06T03:53:25.769357+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:26:52.325913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.752342+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:53:25.752342+0000","last_peered":"2023-07-06T03:53:25.752342+0000","last_clean":"2023-07-06T03:53:25.752342+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:53:25.752342+0000","last_undegraded":"2023-07-06T03:53:25.752342+0000","last_fullsized":"2023-07-06T03:53:25.752342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:01:00.052091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.749653+0000","last_change":"2023-07-06T03:53:25.749653+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.749653+0000","last_undegraded":"2023-07-06T03:53:25.749653+0000","last_fullsized":"2023-07-06T03:53:25.749653+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.766487+0000","last_change":"2023-07-06T03:53:25.763534+0000","last_active":"2023-07-06T03:53:25.766487+0000","last_peered":"2023-07-06T03:53:25.766487+0000","last_clean":"2023-07-06T03:53:25.766487+0000","last_became_active":"2023-07-06T03:53:25.762916+0000","last_became_peered":"2023-07-06T03:53:25.762916+0000","last_unstale":"2023-07-06T03:53:25.766487+0000","last_undegraded":"2023-07-06T03:53:25.766487+0000","last_fullsized":"2023-07-06T03:53:25.766487+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:53.585014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.751388+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:53:25.751388+0000","last_peered":"2023-07-06T03:53:25.751388+0000","last_clean":"2023-07-06T03:53:25.751388+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:53:25.751388+0000","last_undegraded":"2023-07-06T03:53:25.751388+0000","last_fullsized":"2023-07-06T03:53:25.751388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:54.214309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.756568+0000","last_change":"2023-07-06T03:53:25.756568+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.756568+0000","last_undegraded":"2023-07-06T03:53:25.756568+0000","last_fullsized":"2023-07-06T03:53:25.756568+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768632+0000","last_change":"2023-07-06T03:53:25.763463+0000","last_active":"2023-07-06T03:53:25.768632+0000","last_peered":"2023-07-06T03:53:25.768632+0000","last_clean":"2023-07-06T03:53:25.768632+0000","last_became_active":"2023-07-06T03:53:25.763132+0000","last_became_peered":"2023-07-06T03:53:25.763132+0000","last_unstale":"2023-07-06T03:53:25.768632+0000","last_undegraded":"2023-07-06T03:53:25.768632+0000","last_fullsized":"2023-07-06T03:53:25.768632+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:50:21.608182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.752190+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:53:25.752190+0000","last_peered":"2023-07-06T03:53:25.752190+0000","last_clean":"2023-07-06T03:53:25.752190+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:53:25.752190+0000","last_undegraded":"2023-07-06T03:53:25.752190+0000","last_fullsized":"2023-07-06T03:53:25.752190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:37:42.592191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":301,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.744734+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:53:25.744734+0000","last_peered":"2023-07-06T03:53:25.744734+0000","last_clean":"2023-07-06T03:53:25.744734+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:53:25.744734+0000","last_undegraded":"2023-07-06T03:53:25.744734+0000","last_fullsized":"2023-07-06T03:53:25.744734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:57:56.581947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.747510+0000","last_change":"2023-07-06T03:53:25.747510+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.747510+0000","last_undegraded":"2023-07-06T03:53:25.747510+0000","last_fullsized":"2023-07-06T03:53:25.747510+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.762283+0000","last_change":"2023-07-06T03:53:25.758578+0000","last_active":"2023-07-06T03:53:25.762283+0000","last_peered":"2023-07-06T03:53:25.762283+0000","last_clean":"2023-07-06T03:53:25.762283+0000","last_became_active":"2023-07-06T03:53:25.758327+0000","last_became_peered":"2023-07-06T03:53:25.758327+0000","last_unstale":"2023-07-06T03:53:25.762283+0000","last_undegraded":"2023-07-06T03:53:25.762283+0000","last_fullsized":"2023-07-06T03:53:25.762283+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:32:11.379795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.752104+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:53:25.752104+0000","last_peered":"2023-07-06T03:53:25.752104+0000","last_clean":"2023-07-06T03:53:25.752104+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:53:25.752104+0000","last_undegraded":"2023-07-06T03:53:25.752104+0000","last_fullsized":"2023-07-06T03:53:25.752104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:27:44.192794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.754828+0000","last_change":"2023-07-06T03:53:25.754828+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.754828+0000","last_undegraded":"2023-07-06T03:53:25.754828+0000","last_fullsized":"2023-07-06T03:53:25.754828+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.767980+0000","last_change":"2023-07-06T03:53:25.763925+0000","last_active":"2023-07-06T03:53:25.767980+0000","last_peered":"2023-07-06T03:53:25.767980+0000","last_clean":"2023-07-06T03:53:25.767980+0000","last_became_active":"2023-07-06T03:53:25.763655+0000","last_became_peered":"2023-07-06T03:53:25.763655+0000","last_unstale":"2023-07-06T03:53:25.767980+0000","last_undegraded":"2023-07-06T03:53:25.767980+0000","last_fullsized":"2023-07-06T03:53:25.767980+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:33:43.766965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.753207+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:53:25.753207+0000","last_peered":"2023-07-06T03:53:25.753207+0000","last_clean":"2023-07-06T03:53:25.753207+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:53:25.753207+0000","last_undegraded":"2023-07-06T03:53:25.753207+0000","last_fullsized":"2023-07-06T03:53:25.753207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:25:29.753235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.751739+0000","last_change":"2023-07-06T03:53:25.751739+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.751739+0000","last_undegraded":"2023-07-06T03:53:25.751739+0000","last_fullsized":"2023-07-06T03:53:25.751739+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768560+0000","last_change":"2023-07-06T03:53:25.765118+0000","last_active":"2023-07-06T03:53:25.768560+0000","last_peered":"2023-07-06T03:53:25.768560+0000","last_clean":"2023-07-06T03:53:25.768560+0000","last_became_active":"2023-07-06T03:53:25.764820+0000","last_became_peered":"2023-07-06T03:53:25.764820+0000","last_unstale":"2023-07-06T03:53:25.768560+0000","last_undegraded":"2023-07-06T03:53:25.768560+0000","last_fullsized":"2023-07-06T03:53:25.768560+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:28.563800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":150,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.756204+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:53:25.756204+0000","last_peered":"2023-07-06T03:53:25.756204+0000","last_clean":"2023-07-06T03:53:25.756204+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:53:25.756204+0000","last_undegraded":"2023-07-06T03:53:25.756204+0000","last_fullsized":"2023-07-06T03:53:25.756204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:34:25.191156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2023-07-06T03:53:25.752080+0000","last_change":"2023-07-06T03:53:25.752080+0000","last_active":"2023-07-06T03:53:25.737459+0000","last_peered":"2023-07-06T03:53:25.737459+0000","last_clean":"2023-07-06T03:53:25.737459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T03:53:25.752080+0000","last_undegraded":"2023-07-06T03:53:25.752080+0000","last_fullsized":"2023-07-06T03:53:25.752080+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.768745+0000","last_change":"2023-07-06T03:53:25.766728+0000","last_active":"2023-07-06T03:53:25.768745+0000","last_peered":"2023-07-06T03:53:25.768745+0000","last_clean":"2023-07-06T03:53:25.768745+0000","last_became_active":"2023-07-06T03:53:25.766327+0000","last_became_peered":"2023-07-06T03:53:25.766327+0000","last_unstale":"2023-07-06T03:53:25.768745+0000","last_undegraded":"2023-07-06T03:53:25.768745+0000","last_fullsized":"2023-07-06T03:53:25.768745+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:07:23.865989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-07-06T03:53:25.754247+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:53:25.754247+0000","last_peered":"2023-07-06T03:53:25.754247+0000","last_clean":"2023-07-06T03:53:25.754247+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:53:25.754247+0000","last_undegraded":"2023-07-06T03:53:25.754247+0000","last_fullsized":"2023-07-06T03:53:25.754247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:07:44.809785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":1200128,"data_stored":545111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.45900000000000002,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":1.024,"5min":4.0570000000000004,"15min":4.0570000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67900000000000005,"15min":0.76600000000000001},"min":{"1min":0.51200000000000001,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.89700000000000002,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67800000000000005,"15min":0.72399999999999998},"min":{"1min":0.47199999999999998,"5min":0.45000000000000001,"15min":0.34000000000000002},"max":{"1min":1.091,"5min":1.091,"15min":2.6219999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70499999999999996,"15min":0.84599999999999997},"min":{"1min":0.47099999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94999999999999996,"5min":4.0389999999999997,"15min":6.6310000000000002},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.39700000000000002,"15min":0.35199999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":3.77},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.45000000000000001},"max":{"1min":0.93000000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.48699999999999999,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75,"15min":0.753},"min":{"1min":0.57599999999999996,"5min":0.45400000000000001,"15min":0.43099999999999999},"max":{"1min":0.92700000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.65800000000000003,"15min":0.70499999999999996},"min":{"1min":0.44600000000000001,"5min":0.38,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":0.95999999999999996,"15min":1.4590000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.69499999999999995},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0820000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82599999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.755}]},{"osd":6,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72499999999999998,"15min":0.73999999999999999},"min":{"1min":0.46400000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":1.03,"15min":1.5349999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.51800000000000002,"5min":0.45200000000000001,"15min":0.42299999999999999},"max":{"1min":0.98199999999999998,"5min":1.006,"15min":1.3779999999999999},"last":0.55300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":1200128,"data_stored":545111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70899999999999996,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.29399999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77200000000000002,"15min":0.73999999999999999},"min":{"1min":0.44700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.2889999999999999,"5min":3.3399999999999999,"15min":3.3399999999999999},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86099999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":1.1759999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.74399999999999999,"15min":0.68200000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":1.9890000000000001,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.9450000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.76100000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69999999999999996,"15min":0.745},"min":{"1min":0.502,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":2.1850000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.53400000000000003,"5min":0.40500000000000003,"15min":0.373},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":0.64300000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.70699999999999996},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.33200000000000002},"max":{"1min":0.92100000000000004,"5min":1.0309999999999999,"15min":1.1339999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67300000000000004,"15min":0.72499999999999998},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":0.98699999999999999,"15min":1.085},"last":0.623}]},{"osd":5,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.443,"15min":0.38600000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.218},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.77700000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":1.0489999999999999,"15min":1.198},"last":0.81599999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.80500000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97399999999999998,"5min":1.054,"15min":1.323},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.81599999999999995},"min":{"1min":0.54700000000000004,"5min":0.504,"15min":0.41499999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.4139999999999999},"last":0.74199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1752,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1794048,"data_stored":1135479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.52400000000000002},"min":{"1min":0.26900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65100000000000002,"5min":1.002,"15min":2.8879999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.501,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.313},"max":{"1min":0.58099999999999996,"5min":0.78300000000000003,"15min":1.96},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45500000000000002,"15min":0.83099999999999996},"min":{"1min":0.313,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.57799999999999996,"5min":0.70799999999999996,"15min":13.055999999999999},"last":0.375},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54500000000000004,"15min":0.621},"min":{"1min":0.35699999999999998,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":2.8479999999999999},"last":0.497}]},{"osd":3,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48099999999999998,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.71599999999999997,"15min":1.2709999999999999},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.73999999999999999,"15min":1.101},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.86199999999999999,"5min":1.0640000000000001,"15min":12.989000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.78700000000000003,"15min":0.82599999999999996},"min":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.46000000000000002},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.502,"5min":0.432,"15min":0.432},"max":{"1min":0.91200000000000003,"5min":1.0860000000000001,"15min":1.3100000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.71699999999999997,"15min":1.101},"min":{"1min":0.433,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.94499999999999995,"5min":1.1020000000000001,"15min":13.023999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":0.81000000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76700000000000002,"15min":1.147},"min":{"1min":0.54000000000000004,"5min":0.48699999999999999,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":1.0569999999999999,"15min":12.941000000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75800000000000001,"15min":0.81999999999999995},"min":{"1min":0.58399999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.071,"5min":1.071,"15min":2.806},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.751,"15min":0.80300000000000005},"min":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.83999999999999997,"5min":1.1319999999999999,"15min":1.337},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.28000000000000003,"5min":0.26400000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.753,"15min":0.753},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.439},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.248},"max":{"1min":0.51200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38}]},{"osd":2,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.29599999999999999},"max":{"1min":0.52400000000000002,"5min":0.69199999999999995,"15min":0.73699999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.48399999999999999,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.314},"max":{"1min":0.55800000000000005,"5min":0.67200000000000004,"15min":1.2030000000000001},"last":0.442}]},{"osd":3,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.32200000000000001,"5min":0.30099999999999999,"15min":0.29899999999999999},"max":{"1min":0.56200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.63400000000000001,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":1.1699999999999999,"15min":1.171},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":0.88800000000000001,"5min":1.032,"15min":1.034},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.745,"15min":0.73199999999999998},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":0.90600000000000003,"5min":1.1080000000000001,"15min":1.1419999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72899999999999998,"15min":0.747},"min":{"1min":0.56399999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93700000000000006,"5min":1.1040000000000001,"15min":1.1220000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.51600000000000001},"max":{"1min":0.93300000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73599999999999999,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81299999999999994,"5min":1.0649999999999999,"15min":1.546},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":0.86599999999999999,"5min":1.0820000000000001,"15min":1.6830000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77400000000000002,"15min":1.0269999999999999},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":0.86199999999999999,"5min":1.0669999999999999,"15min":9.5960000000000001},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40400,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331440,"statfs":{"total":96636764160,"available":96595394560,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.47999999999999998,"15min":0.443},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.50800000000000001,"15min":0.48199999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.093,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.375}]},{"osd":1,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.55000000000000004,"15min":0.51100000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":6.5960000000000001,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.62},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.57099999999999995,"15min":0.52400000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.657,"5min":3.657,"15min":3.657},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.63200000000000001,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":6.6390000000000002,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.1629999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":0.505}]},{"osd":4,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":0.94299999999999995,"15min":0.78100000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":12.007,"5min":12.007,"15min":12.007},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81899999999999995,"15min":0.73199999999999998},"min":{"1min":0.49299999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.2200000000000002,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.79300000000000004,"15min":0.749},"min":{"1min":0.495,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.1830000000000001,"5min":0.88300000000000001,"15min":0.79700000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.8440000000000003,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":0.94299999999999995,"15min":0.78600000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":11.696999999999999,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":1.4390000000000001,"5min":0.92400000000000004,"15min":0.80800000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":13.084,"5min":13.084,"15min":13.084},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.83299999999999996,"15min":0.83099999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.042,"5min":0.86499999999999999,"15min":0.81999999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":4.484,"5min":4.484,"15min":4.484},"last":0.58399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.44900000000000001,"15min":0.58499999999999996},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.501}]},{"osd":1,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45600000000000002,"15min":0.94899999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":15.911},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":1.0209999999999999},"min":{"1min":0.40300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.347,"5min":1.347,"15min":16.024000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.33600000000000002,"5min":0.32800000000000001,"15min":0.27400000000000002},"max":{"1min":3.25,"5min":3.25,"15min":3.25},"last":0.437},{"interface":"front","average":{"1min":0.63,"5min":0.54300000000000004,"15min":1.028},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.40699999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73999999999999999,"15min":0.75600000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.90300000000000002},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.36699999999999999},"max":{"1min":1.724,"5min":1.724,"15min":5.7300000000000004},"last":0.92400000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75600000000000001,"15min":0.90500000000000003},"min":{"1min":0.49199999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.411,"5min":1.411,"15min":5.077},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78900000000000003,"15min":0.95699999999999996},"min":{"1min":0.57999999999999996,"5min":0.499,"15min":0.44800000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.85199999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76200000000000001,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.91400000000000003},"min":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.45600000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":5.157},"last":0.70199999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.82199999999999995},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76600000000000001,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.492,"5min":1.492,"15min":5.1189999999999998},"last":0.72799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":1187840,"data_stored":538483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.41599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80700000000000005,"5min":1.784,"15min":1.784},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.38},"max":{"1min":0.81899999999999995,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.82799999999999996},"min":{"1min":0.39500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":6.258},"last":0.748},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68000000000000005,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":1.806,"15min":1.806},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.84599999999999997},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.29199999999999998},"max":{"1min":0.90100000000000002,"5min":0.91000000000000003,"15min":6.4740000000000002},"last":0.629},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73799999999999999,"15min":0.92300000000000004},"min":{"1min":0.57699999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":6.4859999999999998},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.48499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.97799999999999998,"15min":1.25},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.503,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":0.99099999999999999,"15min":1.413},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59999999999999998,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77800000000000002,"5min":0.92100000000000004,"15min":1.3280000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.66000000000000003},"min":{"1min":0.50600000000000001,"5min":0.40200000000000002,"15min":0.36399999999999999},"max":{"1min":0.80200000000000005,"5min":0.94299999999999995,"15min":0.99299999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63700000000000001,"15min":0.66200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.81999999999999995,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.78600000000000003},"min":{"1min":0.52400000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":1.008,"15min":1.6839999999999999},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.72299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41299999999999998},"max":{"1min":0.88200000000000001,"5min":0.95699999999999996,"15min":1.0660000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70399999999999996,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.99399999999999999,"5min":1.079,"15min":1.21},"last":0.53500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36880,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334960,"statfs":{"total":96636764160,"available":96598999040,"internally_reserved":0,"allocated":1777664,"data_stored":1127832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70599999999999996,"15min":0.66100000000000003},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.32100000000000001},"max":{"1min":0.871,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68300000000000005,"15min":0.83299999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":6.4059999999999997},"last":0.70699999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.745},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":1.1970000000000001,"15min":1.546},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.79100000000000004,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.376},"max":{"1min":8.1120000000000001,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.84399999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.80800000000000005,"15min":0.73999999999999999},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":8.1829999999999998,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73999999999999999,"15min":0.747},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":1.3140000000000001,"5min":4.484,"15min":4.484},"last":0.80000000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.627,"15min":0.628},"min":{"1min":0.46200000000000002,"5min":0.40899999999999997,"15min":0.34699999999999998},"max":{"1min":0.88500000000000001,"5min":1.1359999999999999,"15min":1.6240000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.42499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.89700000000000002,"5min":1.0569999999999999,"15min":2.718},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.48399999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91200000000000003,"5min":1.246,"15min":1.669},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.002,"5min":1.3360000000000001,"15min":2.0459999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.749,"15min":0.77800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0609999999999999,"15min":2.2919999999999998},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:53:32.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2023-07-06T03:53:32.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.755228+0000', 'last_change': '2023-07-06T03:53:25.755228+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.755228+0000', 'last_undegraded': '2023-07-06T03:53:25.755228+0000', 'last_fullsized': '2023-07-06T03:53:25.755228+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3e is not active+clean 2023-07-06T03:53:32.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750519+0000', 'last_change': '2023-07-06T03:53:25.750519+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750519+0000', 'last_undegraded': '2023-07-06T03:53:25.750519+0000', 'last_fullsized': '2023-07-06T03:53:25.750519+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T03:53:32.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2023-07-06T03:53:32.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.748603+0000', 'last_change': '2023-07-06T03:53:25.748603+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.748603+0000', 'last_undegraded': '2023-07-06T03:53:25.748603+0000', 'last_fullsized': '2023-07-06T03:53:25.748603+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:53:32.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2023-07-06T03:53:32.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.756199+0000', 'last_change': '2023-07-06T03:53:25.756199+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.756199+0000', 'last_undegraded': '2023-07-06T03:53:25.756199+0000', 'last_fullsized': '2023-07-06T03:53:25.756199+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2023-07-06T03:53:32.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750436+0000', 'last_change': '2023-07-06T03:53:25.750436+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750436+0000', 'last_undegraded': '2023-07-06T03:53:25.750436+0000', 'last_fullsized': '2023-07-06T03:53:25.750436+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3a is not active+clean 2023-07-06T03:53:32.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.756992+0000', 'last_change': '2023-07-06T03:53:25.756992+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.756992+0000', 'last_undegraded': '2023-07-06T03:53:25.756992+0000', 'last_fullsized': '2023-07-06T03:53:25.756992+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:53:32.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.39 is not active+clean 2023-07-06T03:53:32.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.755696+0000', 'last_change': '2023-07-06T03:53:25.755696+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.755696+0000', 'last_undegraded': '2023-07-06T03:53:25.755696+0000', 'last_fullsized': '2023-07-06T03:53:25.755696+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:53:32.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2023-07-06T03:53:32.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.756528+0000', 'last_change': '2023-07-06T03:53:25.756528+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.756528+0000', 'last_undegraded': '2023-07-06T03:53:25.756528+0000', 'last_fullsized': '2023-07-06T03:53:25.756528+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:53:32.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.37 is not active+clean 2023-07-06T03:53:32.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750207+0000', 'last_change': '2023-07-06T03:53:25.750207+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750207+0000', 'last_undegraded': '2023-07-06T03:53:25.750207+0000', 'last_fullsized': '2023-07-06T03:53:25.750207+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T03:53:32.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2023-07-06T03:53:32.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.757208+0000', 'last_change': '2023-07-06T03:53:25.757208+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.757208+0000', 'last_undegraded': '2023-07-06T03:53:25.757208+0000', 'last_fullsized': '2023-07-06T03:53:25.757208+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:53:32.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2023-07-06T03:53:32.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.752148+0000', 'last_change': '2023-07-06T03:53:25.752148+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.752148+0000', 'last_undegraded': '2023-07-06T03:53:25.752148+0000', 'last_fullsized': '2023-07-06T03:53:25.752148+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T03:53:32.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2023-07-06T03:53:32.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.756929+0000', 'last_change': '2023-07-06T03:53:25.756929+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.756929+0000', 'last_undegraded': '2023-07-06T03:53:25.756929+0000', 'last_fullsized': '2023-07-06T03:53:25.756929+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:53:32.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2023-07-06T03:53:32.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.749997+0000', 'last_change': '2023-07-06T03:53:25.749997+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.749997+0000', 'last_undegraded': '2023-07-06T03:53:25.749997+0000', 'last_fullsized': '2023-07-06T03:53:25.749997+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.32 is not active+clean 2023-07-06T03:53:32.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.756564+0000', 'last_change': '2023-07-06T03:53:25.756564+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.756564+0000', 'last_undegraded': '2023-07-06T03:53:25.756564+0000', 'last_fullsized': '2023-07-06T03:53:25.756564+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:53:32.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.31 is not active+clean 2023-07-06T03:53:32.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750063+0000', 'last_change': '2023-07-06T03:53:25.750063+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750063+0000', 'last_undegraded': '2023-07-06T03:53:25.750063+0000', 'last_fullsized': '2023-07-06T03:53:25.750063+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T03:53:32.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.30 is not active+clean 2023-07-06T03:53:32.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.747769+0000', 'last_change': '2023-07-06T03:53:25.747769+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.747769+0000', 'last_undegraded': '2023-07-06T03:53:25.747769+0000', 'last_fullsized': '2023-07-06T03:53:25.747769+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:53:32.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2023-07-06T03:53:32.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.749664+0000', 'last_change': '2023-07-06T03:53:25.749664+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.749664+0000', 'last_undegraded': '2023-07-06T03:53:25.749664+0000', 'last_fullsized': '2023-07-06T03:53:25.749664+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:53:32.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2e is not active+clean 2023-07-06T03:53:32.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.751017+0000', 'last_change': '2023-07-06T03:53:25.751017+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.751017+0000', 'last_undegraded': '2023-07-06T03:53:25.751017+0000', 'last_fullsized': '2023-07-06T03:53:25.751017+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T03:53:32.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2023-07-06T03:53:32.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.756675+0000', 'last_change': '2023-07-06T03:53:25.756675+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.756675+0000', 'last_undegraded': '2023-07-06T03:53:25.756675+0000', 'last_fullsized': '2023-07-06T03:53:25.756675+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2023-07-06T03:53:32.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.747503+0000', 'last_change': '2023-07-06T03:53:25.747503+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.747503+0000', 'last_undegraded': '2023-07-06T03:53:25.747503+0000', 'last_fullsized': '2023-07-06T03:53:25.747503+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:53:32.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2023-07-06T03:53:32.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750618+0000', 'last_change': '2023-07-06T03:53:25.750618+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750618+0000', 'last_undegraded': '2023-07-06T03:53:25.750618+0000', 'last_fullsized': '2023-07-06T03:53:25.750618+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:53:32.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2023-07-06T03:53:32.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.751059+0000', 'last_change': '2023-07-06T03:53:25.751059+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.751059+0000', 'last_undegraded': '2023-07-06T03:53:25.751059+0000', 'last_fullsized': '2023-07-06T03:53:25.751059+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:53:32.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2023-07-06T03:53:32.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.756057+0000', 'last_change': '2023-07-06T03:53:25.756057+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.756057+0000', 'last_undegraded': '2023-07-06T03:53:25.756057+0000', 'last_fullsized': '2023-07-06T03:53:25.756057+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:53:32.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2023-07-06T03:53:32.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.756069+0000', 'last_change': '2023-07-06T03:53:25.756069+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.756069+0000', 'last_undegraded': '2023-07-06T03:53:25.756069+0000', 'last_fullsized': '2023-07-06T03:53:25.756069+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:53:32.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2023-07-06T03:53:32.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.747561+0000', 'last_change': '2023-07-06T03:53:25.747561+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.747561+0000', 'last_undegraded': '2023-07-06T03:53:25.747561+0000', 'last_fullsized': '2023-07-06T03:53:25.747561+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:53:32.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2023-07-06T03:53:32.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.747500+0000', 'last_change': '2023-07-06T03:53:25.747500+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.747500+0000', 'last_undegraded': '2023-07-06T03:53:25.747500+0000', 'last_fullsized': '2023-07-06T03:53:25.747500+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:53:32.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2023-07-06T03:53:32.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.747753+0000', 'last_change': '2023-07-06T03:53:25.747753+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.747753+0000', 'last_undegraded': '2023-07-06T03:53:25.747753+0000', 'last_fullsized': '2023-07-06T03:53:25.747753+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:53:32.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.24 is not active+clean 2023-07-06T03:53:32.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.756499+0000', 'last_change': '2023-07-06T03:53:25.756499+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.756499+0000', 'last_undegraded': '2023-07-06T03:53:25.756499+0000', 'last_fullsized': '2023-07-06T03:53:25.756499+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:53:32.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2023-07-06T03:53:32.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.757282+0000', 'last_change': '2023-07-06T03:53:25.757282+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.757282+0000', 'last_undegraded': '2023-07-06T03:53:25.757282+0000', 'last_fullsized': '2023-07-06T03:53:25.757282+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:53:32.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2023-07-06T03:53:32.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750863+0000', 'last_change': '2023-07-06T03:53:25.750863+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750863+0000', 'last_undegraded': '2023-07-06T03:53:25.750863+0000', 'last_fullsized': '2023-07-06T03:53:25.750863+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T03:53:32.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.21 is not active+clean 2023-07-06T03:53:32.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.757028+0000', 'last_change': '2023-07-06T03:53:25.757028+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.757028+0000', 'last_undegraded': '2023-07-06T03:53:25.757028+0000', 'last_fullsized': '2023-07-06T03:53:25.757028+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:53:32.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2023-07-06T03:53:32.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.751127+0000', 'last_change': '2023-07-06T03:53:25.751127+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.751127+0000', 'last_undegraded': '2023-07-06T03:53:25.751127+0000', 'last_fullsized': '2023-07-06T03:53:25.751127+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:53:32.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2023-07-06T03:53:32.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750047+0000', 'last_change': '2023-07-06T03:53:25.750047+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750047+0000', 'last_undegraded': '2023-07-06T03:53:25.750047+0000', 'last_fullsized': '2023-07-06T03:53:25.750047+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:53:32.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2023-07-06T03:53:32.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.755665+0000', 'last_change': '2023-07-06T03:53:25.755665+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.755665+0000', 'last_undegraded': '2023-07-06T03:53:25.755665+0000', 'last_fullsized': '2023-07-06T03:53:25.755665+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:53:32.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2023-07-06T03:53:32.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.751044+0000', 'last_change': '2023-07-06T03:53:25.751044+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.751044+0000', 'last_undegraded': '2023-07-06T03:53:25.751044+0000', 'last_fullsized': '2023-07-06T03:53:25.751044+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T03:53:32.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2023-07-06T03:53:32.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750909+0000', 'last_change': '2023-07-06T03:53:25.750909+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750909+0000', 'last_undegraded': '2023-07-06T03:53:25.750909+0000', 'last_fullsized': '2023-07-06T03:53:25.750909+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T03:53:32.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2023-07-06T03:53:32.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.754346+0000', 'last_change': '2023-07-06T03:53:25.754346+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.754346+0000', 'last_undegraded': '2023-07-06T03:53:25.754346+0000', 'last_fullsized': '2023-07-06T03:53:25.754346+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2023-07-06T03:53:32.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.752140+0000', 'last_change': '2023-07-06T03:53:25.752140+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.752140+0000', 'last_undegraded': '2023-07-06T03:53:25.752140+0000', 'last_fullsized': '2023-07-06T03:53:25.752140+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2023-07-06T03:53:32.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.751283+0000', 'last_change': '2023-07-06T03:53:25.751283+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.751283+0000', 'last_undegraded': '2023-07-06T03:53:25.751283+0000', 'last_fullsized': '2023-07-06T03:53:25.751283+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T03:53:32.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2023-07-06T03:53:32.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.752031+0000', 'last_change': '2023-07-06T03:53:25.752031+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.752031+0000', 'last_undegraded': '2023-07-06T03:53:25.752031+0000', 'last_fullsized': '2023-07-06T03:53:25.752031+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2023-07-06T03:53:32.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.748903+0000', 'last_change': '2023-07-06T03:53:25.748903+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.748903+0000', 'last_undegraded': '2023-07-06T03:53:25.748903+0000', 'last_fullsized': '2023-07-06T03:53:25.748903+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2023-07-06T03:53:32.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.749045+0000', 'last_change': '2023-07-06T03:53:25.749045+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.749045+0000', 'last_undegraded': '2023-07-06T03:53:25.749045+0000', 'last_fullsized': '2023-07-06T03:53:25.749045+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2023-07-06T03:53:32.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.757009+0000', 'last_change': '2023-07-06T03:53:25.757009+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.757009+0000', 'last_undegraded': '2023-07-06T03:53:25.757009+0000', 'last_fullsized': '2023-07-06T03:53:25.757009+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:53:32.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2023-07-06T03:53:32.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.749095+0000', 'last_change': '2023-07-06T03:53:25.749095+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.749095+0000', 'last_undegraded': '2023-07-06T03:53:25.749095+0000', 'last_fullsized': '2023-07-06T03:53:25.749095+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:53:32.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2023-07-06T03:53:32.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750034+0000', 'last_change': '2023-07-06T03:53:25.750034+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750034+0000', 'last_undegraded': '2023-07-06T03:53:25.750034+0000', 'last_fullsized': '2023-07-06T03:53:25.750034+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T03:53:32.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2023-07-06T03:53:32.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750939+0000', 'last_change': '2023-07-06T03:53:25.750939+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750939+0000', 'last_undegraded': '2023-07-06T03:53:25.750939+0000', 'last_fullsized': '2023-07-06T03:53:25.750939+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T03:53:32.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2023-07-06T03:53:32.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750124+0000', 'last_change': '2023-07-06T03:53:25.750124+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750124+0000', 'last_undegraded': '2023-07-06T03:53:25.750124+0000', 'last_fullsized': '2023-07-06T03:53:25.750124+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:53:32.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2023-07-06T03:53:32.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.749081+0000', 'last_change': '2023-07-06T03:53:25.749081+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.749081+0000', 'last_undegraded': '2023-07-06T03:53:25.749081+0000', 'last_fullsized': '2023-07-06T03:53:25.749081+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-07-06T03:53:32.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2023-07-06T03:53:32.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750655+0000', 'last_change': '2023-07-06T03:53:25.750655+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750655+0000', 'last_undegraded': '2023-07-06T03:53:25.750655+0000', 'last_fullsized': '2023-07-06T03:53:25.750655+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:53:32.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2023-07-06T03:53:32.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750697+0000', 'last_change': '2023-07-06T03:53:25.750697+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750697+0000', 'last_undegraded': '2023-07-06T03:53:25.750697+0000', 'last_fullsized': '2023-07-06T03:53:25.750697+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-07-06T03:53:32.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2023-07-06T03:53:32.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.757521+0000', 'last_change': '2023-07-06T03:53:25.757521+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.757521+0000', 'last_undegraded': '2023-07-06T03:53:25.757521+0000', 'last_fullsized': '2023-07-06T03:53:25.757521+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:53:32.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2023-07-06T03:53:32.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.747316+0000', 'last_change': '2023-07-06T03:53:25.747316+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.747316+0000', 'last_undegraded': '2023-07-06T03:53:25.747316+0000', 'last_fullsized': '2023-07-06T03:53:25.747316+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:53:32.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2023-07-06T03:53:32.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.755808+0000', 'last_change': '2023-07-06T03:53:25.755808+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.755808+0000', 'last_undegraded': '2023-07-06T03:53:25.755808+0000', 'last_fullsized': '2023-07-06T03:53:25.755808+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2023-07-06T03:53:32.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.755632+0000', 'last_change': '2023-07-06T03:53:25.755632+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.755632+0000', 'last_undegraded': '2023-07-06T03:53:25.755632+0000', 'last_fullsized': '2023-07-06T03:53:25.755632+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2023-07-06T03:53:32.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.755831+0000', 'last_change': '2023-07-06T03:53:25.755831+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.755831+0000', 'last_undegraded': '2023-07-06T03:53:25.755831+0000', 'last_fullsized': '2023-07-06T03:53:25.755831+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:53:32.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2023-07-06T03:53:32.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750599+0000', 'last_change': '2023-07-06T03:53:25.750599+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750599+0000', 'last_undegraded': '2023-07-06T03:53:25.750599+0000', 'last_fullsized': '2023-07-06T03:53:25.750599+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-07-06T03:53:32.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2023-07-06T03:53:32.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.750548+0000', 'last_change': '2023-07-06T03:53:25.750548+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.750548+0000', 'last_undegraded': '2023-07-06T03:53:25.750548+0000', 'last_fullsized': '2023-07-06T03:53:25.750548+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:53:32.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2023-07-06T03:53:32.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.755543+0000', 'last_change': '2023-07-06T03:53:25.755543+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.755543+0000', 'last_undegraded': '2023-07-06T03:53:25.755543+0000', 'last_fullsized': '2023-07-06T03:53:25.755543+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-07-06T03:53:32.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2023-07-06T03:53:32.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.749653+0000', 'last_change': '2023-07-06T03:53:25.749653+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.749653+0000', 'last_undegraded': '2023-07-06T03:53:25.749653+0000', 'last_fullsized': '2023-07-06T03:53:25.749653+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:53:32.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2023-07-06T03:53:32.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.756568+0000', 'last_change': '2023-07-06T03:53:25.756568+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.756568+0000', 'last_undegraded': '2023-07-06T03:53:25.756568+0000', 'last_fullsized': '2023-07-06T03:53:25.756568+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-07-06T03:53:32.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2023-07-06T03:53:32.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.747510+0000', 'last_change': '2023-07-06T03:53:25.747510+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.747510+0000', 'last_undegraded': '2023-07-06T03:53:25.747510+0000', 'last_fullsized': '2023-07-06T03:53:25.747510+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-07-06T03:53:32.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2023-07-06T03:53:32.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.754828+0000', 'last_change': '2023-07-06T03:53:25.754828+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.754828+0000', 'last_undegraded': '2023-07-06T03:53:25.754828+0000', 'last_fullsized': '2023-07-06T03:53:25.754828+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2023-07-06T03:53:32.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.751739+0000', 'last_change': '2023-07-06T03:53:25.751739+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.751739+0000', 'last_undegraded': '2023-07-06T03:53:25.751739+0000', 'last_fullsized': '2023-07-06T03:53:25.751739+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T03:53:32.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2023-07-06T03:53:32.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 97, 'state': 'creating+peering', 'last_fresh': '2023-07-06T03:53:25.752080+0000', 'last_change': '2023-07-06T03:53:25.752080+0000', 'last_active': '2023-07-06T03:53:25.737459+0000', 'last_peered': '2023-07-06T03:53:25.737459+0000', 'last_clean': '2023-07-06T03:53:25.737459+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T03:53:25.752080+0000', 'last_undegraded': '2023-07-06T03:53:25.752080+0000', 'last_fullsized': '2023-07-06T03:53:25.752080+0000', 'mapping_epoch': 97, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 97, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'last_clean_scrub_stamp': '2023-07-06T03:53:25.737459+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T03:53:32.124 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-07-06T03:53:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.316+0000 7f806b9ff700 1 Processor -- start 2023-07-06T03:53:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.316+0000 7f806b9ff700 1 -- start start 2023-07-06T03:53:32.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.317+0000 7f806b9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8064158f20 0x7f80640caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.317+0000 7f806b9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80641592f0 0x7f80640b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.317+0000 7f806b9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806415cc40 con 0x7f806406d860 2023-07-06T03:53:32.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f806b9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f806415cf20 con 0x7f80641592f0 2023-07-06T03:53:32.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f806b9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f806415d200 con 0x7f8064158f20 2023-07-06T03:53:32.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f8068f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80641592f0 0x7f80640b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f8069f9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f806406d860 0x7f80640cd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51270/0 (socket says 172.21.15.97:51270) 2023-07-06T03:53:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f8069f9c700 1 -- 172.21.15.97:0/1647252398 learned_addr learned my addr 172.21.15.97:0/1647252398 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f806979b700 1 --2- 172.21.15.97:0/1647252398 >> v2:172.21.15.146:3300/0 conn(0x7f8064158f20 0x7f80640caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f8068f9a700 1 -- 172.21.15.97:0/1647252398 >> v2:172.21.15.146:3300/0 conn(0x7f8064158f20 msgr2=0x7f80640caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f8068f9a700 1 --2- 172.21.15.97:0/1647252398 >> v2:172.21.15.146:3300/0 conn(0x7f8064158f20 0x7f80640caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f8068f9a700 1 -- 172.21.15.97:0/1647252398 >> v1:172.21.15.97:6789/0 conn(0x7f806406d860 legacy=0x7f80640cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f8068f9a700 1 -- 172.21.15.97:0/1647252398 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806415d4e0 con 0x7f80641592f0 2023-07-06T03:53:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f806979b700 1 --2- 172.21.15.97:0/1647252398 >> v2:172.21.15.146:3300/0 conn(0x7f8064158f20 0x7f80640caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.318+0000 7f805bfff700 1 -- 172.21.15.97:0/1647252398 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184578693 0 0) 0x7f806415cc40 con 0x7f806406d860 2023-07-06T03:53:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.319+0000 7f8068f9a700 1 --2- 172.21.15.97:0/1647252398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80641592f0 0x7f80640b0560 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f806406b810 tx=0x7f8054009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5865bfa41e7bf453 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.319+0000 7f805bfff700 1 -- 172.21.15.97:0/1647252398 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8054011830 con 0x7f80641592f0 2023-07-06T03:53:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.319+0000 7f805bfff700 1 -- 172.21.15.97:0/1647252398 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8054011990 con 0x7f80641592f0 2023-07-06T03:53:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.319+0000 7f805bfff700 1 -- 172.21.15.97:0/1647252398 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8054011d20 con 0x7f80641592f0 2023-07-06T03:53:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.319+0000 7f806b9ff700 1 -- 172.21.15.97:0/1647252398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80641592f0 msgr2=0x7f80640b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.319+0000 7f806b9ff700 1 --2- 172.21.15.97:0/1647252398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80641592f0 0x7f80640b0560 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f806406b810 tx=0x7f8054009b20 comp rx=0 tx=0).stop 2023-07-06T03:53:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.319+0000 7f806b9ff700 1 -- 172.21.15.97:0/1647252398 shutdown_connections 2023-07-06T03:53:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.319+0000 7f806b9ff700 1 --2- 172.21.15.97:0/1647252398 >> v2:172.21.15.146:3300/0 conn(0x7f8064158f20 0x7f80640caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.319+0000 7f806b9ff700 1 --2- 172.21.15.97:0/1647252398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80641592f0 0x7f80640b0560 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.319+0000 7f806b9ff700 1 -- 172.21.15.97:0/1647252398 >> 172.21.15.97:0/1647252398 conn(0x7f80640ce8c0 msgr2=0x7f80640cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.319+0000 7f806b9ff700 1 -- 172.21.15.97:0/1647252398 shutdown_connections 2023-07-06T03:53:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.320+0000 7f806b9ff700 1 -- 172.21.15.97:0/1647252398 wait complete. 2023-07-06T03:53:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.322+0000 7f806b9ff700 1 Processor -- start 2023-07-06T03:53:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.322+0000 7f806b9ff700 1 -- start start 2023-07-06T03:53:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.323+0000 7f806b9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f806406d860 0x7f806415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.323+0000 7f806b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8064158f20 0x7f80640aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.323+0000 7f806b9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80641592f0 0x7f80640b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.323+0000 7f806b9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80640aeff0 con 0x7f80641592f0 2023-07-06T03:53:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.323+0000 7f806b9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80640af290 con 0x7f806406d860 2023-07-06T03:53:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.323+0000 7f806b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80640af570 con 0x7f8064158f20 2023-07-06T03:53:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f8069f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80641592f0 0x7f80640b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f8069f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80641592f0 0x7f80640b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59468/0 (socket says 172.21.15.97:59468) 2023-07-06T03:53:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f8069f9c700 1 -- 172.21.15.97:0/749163837 learned_addr learned my addr 172.21.15.97:0/749163837 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:32.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f806979b700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f806406d860 0x7f806415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f8068f9a700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8064158f20 0x7f80640aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f8069f9c700 1 -- 172.21.15.97:0/749163837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8064158f20 msgr2=0x7f80640aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f8069f9c700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8064158f20 0x7f80640aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f8069f9c700 1 -- 172.21.15.97:0/749163837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f806406d860 msgr2=0x7f806415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f8069f9c700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f806406d860 0x7f806415c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f8069f9c700 1 -- 172.21.15.97:0/749163837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80540094a0 con 0x7f80641592f0 2023-07-06T03:53:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f8069f9c700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80641592f0 0x7f80640b0330 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f805c002230 tx=0x7f805c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:32.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.324+0000 7f8059ffb700 1 -- 172.21.15.97:0/749163837 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c00aac0 con 0x7f80641592f0 2023-07-06T03:53:32.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.325+0000 7f8059ffb700 1 -- 172.21.15.97:0/749163837 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805c00ac40 con 0x7f80641592f0 2023-07-06T03:53:32.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.325+0000 7f8059ffb700 1 -- 172.21.15.97:0/749163837 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c0070d0 con 0x7f80641592f0 2023-07-06T03:53:32.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.325+0000 7f806b9ff700 1 -- 172.21.15.97:0/749163837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80640af850 con 0x7f80641592f0 2023-07-06T03:53:32.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.325+0000 7f806b9ff700 1 -- 172.21.15.97:0/749163837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80640afe10 con 0x7f80641592f0 2023-07-06T03:53:32.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.326+0000 7f806b9ff700 1 -- 172.21.15.97:0/749163837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80640b0000 con 0x7f80641592f0 2023-07-06T03:53:32.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.327+0000 7f8059ffb700 1 -- 172.21.15.97:0/749163837 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f805c007250 con 0x7f80641592f0 2023-07-06T03:53:32.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.327+0000 7f8059ffb700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f804806ad10 0x7f804806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.328+0000 7f8059ffb700 1 -- 172.21.15.97:0/749163837 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f805c088f60 con 0x7f80641592f0 2023-07-06T03:53:32.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.328+0000 7f806979b700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f804806ad10 0x7f804806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.328+0000 7f806979b700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f804806ad10 0x7f804806d1c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8060006f10 tx=0x7f8060008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:32.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.332+0000 7f8059ffb700 1 -- 172.21.15.97:0/749163837 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f805c053f70 con 0x7f80641592f0 2023-07-06T03:53:32.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.604+0000 7f806b9ff700 1 -- 172.21.15.97:0/749163837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f806404e340 con 0x7f80641592f0 2023-07-06T03:53:32.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.605+0000 7f8059ffb700 1 -- 172.21.15.97:0/749163837 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1035 (secure 0 0 0) 0x7f805c018420 con 0x7f80641592f0 2023-07-06T03:53:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:32.607 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":528,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":99,"num_osds":8,"num_up_osds":8,"osd_up_since":1688615095,"num_in_osds":8,"osd_in_since":1688615087,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":290234368,"bytes_avail":772803878912,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":67,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7992}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2023-07-06T03:50:38.951587+0000","services":{}},"progress_events":{}} 2023-07-06T03:53:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.607+0000 7f806b9ff700 1 -- 172.21.15.97:0/749163837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f804806ad10 msgr2=0x7f804806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.607+0000 7f806b9ff700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f804806ad10 0x7f804806d1c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8060006f10 tx=0x7f8060008040 comp rx=0 tx=0).stop 2023-07-06T03:53:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.607+0000 7f806b9ff700 1 -- 172.21.15.97:0/749163837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80641592f0 msgr2=0x7f80640b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.607+0000 7f806b9ff700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80641592f0 0x7f80640b0330 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f805c002230 tx=0x7f805c00cf50 comp rx=0 tx=0).stop 2023-07-06T03:53:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.608+0000 7f806b9ff700 1 -- 172.21.15.97:0/749163837 shutdown_connections 2023-07-06T03:53:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.608+0000 7f806b9ff700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f804806ad10 0x7f804806d1c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.608+0000 7f806b9ff700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80641592f0 0x7f80640b0330 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.608+0000 7f806b9ff700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f806406d860 0x7f806415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.608+0000 7f806b9ff700 1 --2- 172.21.15.97:0/749163837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8064158f20 0x7f80640aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.608+0000 7f806b9ff700 1 -- 172.21.15.97:0/749163837 >> 172.21.15.97:0/749163837 conn(0x7f80640ce8c0 msgr2=0x7f80640cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:32.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.608+0000 7f806b9ff700 1 -- 172.21.15.97:0/749163837 shutdown_connections 2023-07-06T03:53:32.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.608+0000 7f806b9ff700 1 -- 172.21.15.97:0/749163837 wait complete. 2023-07-06T03:53:32.627 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-07-06T03:53:32.627 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:53:32.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.821+0000 7fe878fff700 1 Processor -- start 2023-07-06T03:53:32.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.821+0000 7fe878fff700 1 -- start start 2023-07-06T03:53:32.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.822+0000 7fe878fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe87415c740 0x7fe8740d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.822+0000 7fe878fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe874071820 0x7fe87406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.822+0000 7fe878fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe874071c90 0x7fe87406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.822+0000 7fe878fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8740d0610 con 0x7fe87415c740 2023-07-06T03:53:32.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.822+0000 7fe878fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8740cdc80 con 0x7fe874071820 2023-07-06T03:53:32.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.822+0000 7fe878fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740ce380 con 0x7fe874071c90 2023-07-06T03:53:32.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.822+0000 7fe87259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe87415c740 0x7fe8740d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe87259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe87415c740 0x7fe8740d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59476/0 (socket says 172.21.15.97:59476) 2023-07-06T03:53:32.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe87259c700 1 -- 172.21.15.97:0/3491989492 learned_addr learned my addr 172.21.15.97:0/3491989492 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:32.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe872d9d700 1 --2- 172.21.15.97:0/3491989492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe874071c90 0x7fe87406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe871d9b700 1 --2- 172.21.15.97:0/3491989492 >> v2:172.21.15.146:3300/0 conn(0x7fe874071820 0x7fe87406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe87259c700 1 -- 172.21.15.97:0/3491989492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe874071c90 msgr2=0x7fe87406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:32.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe87259c700 1 --2- 172.21.15.97:0/3491989492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe874071c90 0x7fe87406a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe87259c700 1 -- 172.21.15.97:0/3491989492 >> v2:172.21.15.146:3300/0 conn(0x7fe874071820 msgr2=0x7fe87406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:32.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe87259c700 1 --2- 172.21.15.97:0/3491989492 >> v2:172.21.15.146:3300/0 conn(0x7fe874071820 0x7fe87406a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe87259c700 1 -- 172.21.15.97:0/3491989492 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8740cd630 con 0x7fe87415c740 2023-07-06T03:53:32.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe872d9d700 1 --2- 172.21.15.97:0/3491989492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe874071c90 0x7fe87406a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:32.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe871d9b700 1 --2- 172.21.15.97:0/3491989492 >> v2:172.21.15.146:3300/0 conn(0x7fe874071820 0x7fe87406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:32.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe87259c700 1 --2- 172.21.15.97:0/3491989492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe87415c740 0x7fe8740d00d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe87406f310 tx=0x7fe864009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f578d9894c440c2d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:32.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe87159a700 1 -- 172.21.15.97:0/3491989492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86400b8a0 con 0x7fe87415c740 2023-07-06T03:53:32.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe87159a700 1 -- 172.21.15.97:0/3491989492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86400ba00 con 0x7fe87415c740 2023-07-06T03:53:32.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.823+0000 7fe87159a700 1 -- 172.21.15.97:0/3491989492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86400bdb0 con 0x7fe87415c740 2023-07-06T03:53:32.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.824+0000 7fe878fff700 1 -- 172.21.15.97:0/3491989492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe87415c740 msgr2=0x7fe8740d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:32.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.824+0000 7fe878fff700 1 --2- 172.21.15.97:0/3491989492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe87415c740 0x7fe8740d00d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe87406f310 tx=0x7fe864009bc0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.824+0000 7fe878fff700 1 -- 172.21.15.97:0/3491989492 shutdown_connections 2023-07-06T03:53:32.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.824+0000 7fe878fff700 1 --2- 172.21.15.97:0/3491989492 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe874071c90 0x7fe87406a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.824+0000 7fe878fff700 1 --2- 172.21.15.97:0/3491989492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe87415c740 0x7fe8740d00d0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.824+0000 7fe878fff700 1 --2- 172.21.15.97:0/3491989492 >> v2:172.21.15.146:3300/0 conn(0x7fe874071820 0x7fe87406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.824+0000 7fe878fff700 1 -- 172.21.15.97:0/3491989492 >> 172.21.15.97:0/3491989492 conn(0x7fe8740b2280 msgr2=0x7fe8740cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:32.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.824+0000 7fe878fff700 1 -- 172.21.15.97:0/3491989492 shutdown_connections 2023-07-06T03:53:32.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.824+0000 7fe878fff700 1 -- 172.21.15.97:0/3491989492 wait complete. 2023-07-06T03:53:32.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.826+0000 7fe878fff700 1 Processor -- start 2023-07-06T03:53:32.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.826+0000 7fe878fff700 1 -- start start 2023-07-06T03:53:32.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe878fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe874071820 0x7fe8740b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe878fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe87415c740 0x7fe874145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe878fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8740b3ee0 0x7fe8741460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe878fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740b5070 con 0x7fe8740b3ee0 2023-07-06T03:53:32.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe878fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740b4460 con 0x7fe87415c740 2023-07-06T03:53:32.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe878fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740b4740 con 0x7fe874071820 2023-07-06T03:53:32.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe872d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8740b3ee0 0x7fe8741460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe872d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8740b3ee0 0x7fe8741460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59492/0 (socket says 172.21.15.97:59492) 2023-07-06T03:53:32.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe872d9d700 1 -- 172.21.15.97:0/1488463355 learned_addr learned my addr 172.21.15.97:0/1488463355 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:32.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe87259c700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe874071820 0x7fe8740b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe871d9b700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe87415c740 0x7fe874145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe872d9d700 1 -- 172.21.15.97:0/1488463355 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe874071820 msgr2=0x7fe8740b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:32.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe872d9d700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe874071820 0x7fe8740b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe872d9d700 1 -- 172.21.15.97:0/1488463355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe87415c740 msgr2=0x7fe874145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:32.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe872d9d700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe87415c740 0x7fe874145b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:32.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.827+0000 7fe872d9d700 1 -- 172.21.15.97:0/1488463355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8640094f0 con 0x7fe8740b3ee0 2023-07-06T03:53:32.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.828+0000 7fe872d9d700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8740b3ee0 0x7fe8741460a0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe86c007ec0 tx=0x7fe86c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:32.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.828+0000 7fe862ffd700 1 -- 172.21.15.97:0/1488463355 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00de30 con 0x7fe8740b3ee0 2023-07-06T03:53:32.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.828+0000 7fe862ffd700 1 -- 172.21.15.97:0/1488463355 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c00f040 con 0x7fe8740b3ee0 2023-07-06T03:53:32.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.828+0000 7fe878fff700 1 -- 172.21.15.97:0/1488463355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8740b4a20 con 0x7fe8740b3ee0 2023-07-06T03:53:32.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.828+0000 7fe862ffd700 1 -- 172.21.15.97:0/1488463355 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c003c60 con 0x7fe8740b3ee0 2023-07-06T03:53:32.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.828+0000 7fe878fff700 1 -- 172.21.15.97:0/1488463355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8741495a0 con 0x7fe8740b3ee0 2023-07-06T03:53:32.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.829+0000 7fe878fff700 1 -- 172.21.15.97:0/1488463355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8740b5220 con 0x7fe8740b3ee0 2023-07-06T03:53:32.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.830+0000 7fe862ffd700 1 -- 172.21.15.97:0/1488463355 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe86c004030 con 0x7fe8740b3ee0 2023-07-06T03:53:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.831+0000 7fe862ffd700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe85406ace0 0x7fe85406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.831+0000 7fe862ffd700 1 -- 172.21.15.97:0/1488463355 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe86c089690 con 0x7fe8740b3ee0 2023-07-06T03:53:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.831+0000 7fe87259c700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe85406ace0 0x7fe85406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.831+0000 7fe87259c700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe85406ace0 0x7fe85406d190 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe864009c00 tx=0x7fe864008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:32.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:32.835+0000 7fe862ffd700 1 -- 172.21.15.97:0/1488463355 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe86c0546a0 con 0x7fe8740b3ee0 2023-07-06T03:53:33.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.027+0000 7fe878fff700 1 -- 172.21.15.97:0/1488463355 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8740aede0 con 0x7fe85406ace0 2023-07-06T03:53:33.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.036+0000 7fe862ffd700 1 -- 172.21.15.97:0/1488463355 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311411 (secure 0 0 0) 0x7fe86400bb70 con 0x7fe85406ace0 2023-07-06T03:53:33.041 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:33.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 -- 172.21.15.97:0/1488463355 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe85406ace0 msgr2=0x7fe85406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe85406ace0 0x7fe85406d190 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe864009c00 tx=0x7fe864008030 comp rx=0 tx=0).stop 2023-07-06T03:53:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 -- 172.21.15.97:0/1488463355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8740b3ee0 msgr2=0x7fe8741460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8740b3ee0 0x7fe8741460a0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe86c007ec0 tx=0x7fe86c00d3b0 comp rx=0 tx=0).stop 2023-07-06T03:53:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 -- 172.21.15.97:0/1488463355 shutdown_connections 2023-07-06T03:53:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe85406ace0 0x7fe85406d190 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:33.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8740b3ee0 0x7fe8741460a0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe874071820 0x7fe8740b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 --2- 172.21.15.97:0/1488463355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe87415c740 0x7fe874145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 -- 172.21.15.97:0/1488463355 >> 172.21.15.97:0/1488463355 conn(0x7fe8740b2280 msgr2=0x7fe8740cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 -- 172.21.15.97:0/1488463355 shutdown_connections 2023-07-06T03:53:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:33.040+0000 7fe878fff700 1 -- 172.21.15.97:0/1488463355 wait complete. 2023-07-06T03:53:33.044 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:53:33.060 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2023-07-06T03:53:32.992814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283748,"kb_used_data":10788,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754690972,"statfs":{"total":773094113280,"available":772803555328,"internally_reserved":0,"allocated":11046912,"data_stored":5655577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.249364"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985178+0000","last_change":"2023-07-06T03:53:26.775067+0000","last_active":"2023-07-06T03:53:27.985178+0000","last_peered":"2023-07-06T03:53:27.985178+0000","last_clean":"2023-07-06T03:53:27.985178+0000","last_became_active":"2023-07-06T03:53:26.774259+0000","last_became_peered":"2023-07-06T03:53:26.774259+0000","last_unstale":"2023-07-06T03:53:27.985178+0000","last_undegraded":"2023-07-06T03:53:27.985178+0000","last_fullsized":"2023-07-06T03:53:27.985178+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:47:37.797380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898845+0000","last_change":"2023-07-06T03:53:26.782230+0000","last_active":"2023-07-06T03:53:27.898845+0000","last_peered":"2023-07-06T03:53:27.898845+0000","last_clean":"2023-07-06T03:53:27.898845+0000","last_became_active":"2023-07-06T03:53:26.781962+0000","last_became_peered":"2023-07-06T03:53:26.781962+0000","last_unstale":"2023-07-06T03:53:27.898845+0000","last_undegraded":"2023-07-06T03:53:27.898845+0000","last_fullsized":"2023-07-06T03:53:27.898845+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:16.272580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985475+0000","last_change":"2023-07-06T03:53:26.782591+0000","last_active":"2023-07-06T03:53:27.985475+0000","last_peered":"2023-07-06T03:53:27.985475+0000","last_clean":"2023-07-06T03:53:27.985475+0000","last_became_active":"2023-07-06T03:53:26.782251+0000","last_became_peered":"2023-07-06T03:53:26.782251+0000","last_unstale":"2023-07-06T03:53:27.985475+0000","last_undegraded":"2023-07-06T03:53:27.985475+0000","last_fullsized":"2023-07-06T03:53:27.985475+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:40:27.706089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985466+0000","last_change":"2023-07-06T03:53:26.766314+0000","last_active":"2023-07-06T03:53:27.985466+0000","last_peered":"2023-07-06T03:53:27.985466+0000","last_clean":"2023-07-06T03:53:27.985466+0000","last_became_active":"2023-07-06T03:53:26.765989+0000","last_became_peered":"2023-07-06T03:53:26.765989+0000","last_unstale":"2023-07-06T03:53:27.985466+0000","last_undegraded":"2023-07-06T03:53:27.985466+0000","last_fullsized":"2023-07-06T03:53:27.985466+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:47:02.552002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985370+0000","last_change":"2023-07-06T03:53:26.778493+0000","last_active":"2023-07-06T03:53:27.985370+0000","last_peered":"2023-07-06T03:53:27.985370+0000","last_clean":"2023-07-06T03:53:27.985370+0000","last_became_active":"2023-07-06T03:53:26.778154+0000","last_became_peered":"2023-07-06T03:53:26.778154+0000","last_unstale":"2023-07-06T03:53:27.985370+0000","last_undegraded":"2023-07-06T03:53:27.985370+0000","last_fullsized":"2023-07-06T03:53:27.985370+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:28:19.541129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765535+0000","last_change":"2023-07-06T03:53:26.774404+0000","last_active":"2023-07-06T03:53:27.765535+0000","last_peered":"2023-07-06T03:53:27.765535+0000","last_clean":"2023-07-06T03:53:27.765535+0000","last_became_active":"2023-07-06T03:53:26.773965+0000","last_became_peered":"2023-07-06T03:53:26.773965+0000","last_unstale":"2023-07-06T03:53:27.765535+0000","last_undegraded":"2023-07-06T03:53:27.765535+0000","last_fullsized":"2023-07-06T03:53:27.765535+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:04:53.424979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765322+0000","last_change":"2023-07-06T03:53:26.769406+0000","last_active":"2023-07-06T03:53:27.765322+0000","last_peered":"2023-07-06T03:53:27.765322+0000","last_clean":"2023-07-06T03:53:27.765322+0000","last_became_active":"2023-07-06T03:53:26.768913+0000","last_became_peered":"2023-07-06T03:53:26.768913+0000","last_unstale":"2023-07-06T03:53:27.765322+0000","last_undegraded":"2023-07-06T03:53:27.765322+0000","last_fullsized":"2023-07-06T03:53:27.765322+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:33:44.769610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984341+0000","last_change":"2023-07-06T03:53:26.770178+0000","last_active":"2023-07-06T03:53:27.984341+0000","last_peered":"2023-07-06T03:53:27.984341+0000","last_clean":"2023-07-06T03:53:27.984341+0000","last_became_active":"2023-07-06T03:53:26.769900+0000","last_became_peered":"2023-07-06T03:53:26.769900+0000","last_unstale":"2023-07-06T03:53:27.984341+0000","last_undegraded":"2023-07-06T03:53:27.984341+0000","last_fullsized":"2023-07-06T03:53:27.984341+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:29.566445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.899045+0000","last_change":"2023-07-06T03:53:26.778308+0000","last_active":"2023-07-06T03:53:27.899045+0000","last_peered":"2023-07-06T03:53:27.899045+0000","last_clean":"2023-07-06T03:53:27.899045+0000","last_became_active":"2023-07-06T03:53:26.778012+0000","last_became_peered":"2023-07-06T03:53:26.778012+0000","last_unstale":"2023-07-06T03:53:27.899045+0000","last_undegraded":"2023-07-06T03:53:27.899045+0000","last_fullsized":"2023-07-06T03:53:27.899045+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:09:32.641258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985429+0000","last_change":"2023-07-06T03:53:26.769312+0000","last_active":"2023-07-06T03:53:27.985429+0000","last_peered":"2023-07-06T03:53:27.985429+0000","last_clean":"2023-07-06T03:53:27.985429+0000","last_became_active":"2023-07-06T03:53:26.769039+0000","last_became_peered":"2023-07-06T03:53:26.769039+0000","last_unstale":"2023-07-06T03:53:27.985429+0000","last_undegraded":"2023-07-06T03:53:27.985429+0000","last_fullsized":"2023-07-06T03:53:27.985429+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:42.237225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764992+0000","last_change":"2023-07-06T03:53:26.776295+0000","last_active":"2023-07-06T03:53:27.764992+0000","last_peered":"2023-07-06T03:53:27.764992+0000","last_clean":"2023-07-06T03:53:27.764992+0000","last_became_active":"2023-07-06T03:53:26.776001+0000","last_became_peered":"2023-07-06T03:53:26.776001+0000","last_unstale":"2023-07-06T03:53:27.764992+0000","last_undegraded":"2023-07-06T03:53:27.764992+0000","last_fullsized":"2023-07-06T03:53:27.764992+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:58:41.795378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765452+0000","last_change":"2023-07-06T03:53:26.764902+0000","last_active":"2023-07-06T03:53:27.765452+0000","last_peered":"2023-07-06T03:53:27.765452+0000","last_clean":"2023-07-06T03:53:27.765452+0000","last_became_active":"2023-07-06T03:53:26.764619+0000","last_became_peered":"2023-07-06T03:53:26.764619+0000","last_unstale":"2023-07-06T03:53:27.765452+0000","last_undegraded":"2023-07-06T03:53:27.765452+0000","last_fullsized":"2023-07-06T03:53:27.765452+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:22:06.631745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985419+0000","last_change":"2023-07-06T03:53:26.773421+0000","last_active":"2023-07-06T03:53:27.985419+0000","last_peered":"2023-07-06T03:53:27.985419+0000","last_clean":"2023-07-06T03:53:27.985419+0000","last_became_active":"2023-07-06T03:53:26.773138+0000","last_became_peered":"2023-07-06T03:53:26.773138+0000","last_unstale":"2023-07-06T03:53:27.985419+0000","last_undegraded":"2023-07-06T03:53:27.985419+0000","last_fullsized":"2023-07-06T03:53:27.985419+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:50:14.873622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765534+0000","last_change":"2023-07-06T03:53:26.769278+0000","last_active":"2023-07-06T03:53:27.765534+0000","last_peered":"2023-07-06T03:53:27.765534+0000","last_clean":"2023-07-06T03:53:27.765534+0000","last_became_active":"2023-07-06T03:53:26.768992+0000","last_became_peered":"2023-07-06T03:53:26.768992+0000","last_unstale":"2023-07-06T03:53:27.765534+0000","last_undegraded":"2023-07-06T03:53:27.765534+0000","last_fullsized":"2023-07-06T03:53:27.765534+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:54.432067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898693+0000","last_change":"2023-07-06T03:53:26.780150+0000","last_active":"2023-07-06T03:53:27.898693+0000","last_peered":"2023-07-06T03:53:27.898693+0000","last_clean":"2023-07-06T03:53:27.898693+0000","last_became_active":"2023-07-06T03:53:26.779845+0000","last_became_peered":"2023-07-06T03:53:26.779845+0000","last_unstale":"2023-07-06T03:53:27.898693+0000","last_undegraded":"2023-07-06T03:53:27.898693+0000","last_fullsized":"2023-07-06T03:53:27.898693+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:40:49.166824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984200+0000","last_change":"2023-07-06T03:53:26.770953+0000","last_active":"2023-07-06T03:53:27.984200+0000","last_peered":"2023-07-06T03:53:27.984200+0000","last_clean":"2023-07-06T03:53:27.984200+0000","last_became_active":"2023-07-06T03:53:26.770621+0000","last_became_peered":"2023-07-06T03:53:26.770621+0000","last_unstale":"2023-07-06T03:53:27.984200+0000","last_undegraded":"2023-07-06T03:53:27.984200+0000","last_fullsized":"2023-07-06T03:53:27.984200+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:45:55.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985031+0000","last_change":"2023-07-06T03:53:26.782452+0000","last_active":"2023-07-06T03:53:27.985031+0000","last_peered":"2023-07-06T03:53:27.985031+0000","last_clean":"2023-07-06T03:53:27.985031+0000","last_became_active":"2023-07-06T03:53:26.782166+0000","last_became_peered":"2023-07-06T03:53:26.782166+0000","last_unstale":"2023-07-06T03:53:27.985031+0000","last_undegraded":"2023-07-06T03:53:27.985031+0000","last_fullsized":"2023-07-06T03:53:27.985031+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:19.890315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898981+0000","last_change":"2023-07-06T03:53:26.771445+0000","last_active":"2023-07-06T03:53:27.898981+0000","last_peered":"2023-07-06T03:53:27.898981+0000","last_clean":"2023-07-06T03:53:27.898981+0000","last_became_active":"2023-07-06T03:53:26.771157+0000","last_became_peered":"2023-07-06T03:53:26.771157+0000","last_unstale":"2023-07-06T03:53:27.898981+0000","last_undegraded":"2023-07-06T03:53:27.898981+0000","last_fullsized":"2023-07-06T03:53:27.898981+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:26:05.048391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984889+0000","last_change":"2023-07-06T03:53:26.770316+0000","last_active":"2023-07-06T03:53:27.984889+0000","last_peered":"2023-07-06T03:53:27.984889+0000","last_clean":"2023-07-06T03:53:27.984889+0000","last_became_active":"2023-07-06T03:53:26.769996+0000","last_became_peered":"2023-07-06T03:53:26.769996+0000","last_unstale":"2023-07-06T03:53:27.984889+0000","last_undegraded":"2023-07-06T03:53:27.984889+0000","last_fullsized":"2023-07-06T03:53:27.984889+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:34:11.936629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984262+0000","last_change":"2023-07-06T03:53:26.770241+0000","last_active":"2023-07-06T03:53:27.984262+0000","last_peered":"2023-07-06T03:53:27.984262+0000","last_clean":"2023-07-06T03:53:27.984262+0000","last_became_active":"2023-07-06T03:53:26.770038+0000","last_became_peered":"2023-07-06T03:53:26.770038+0000","last_unstale":"2023-07-06T03:53:27.984262+0000","last_undegraded":"2023-07-06T03:53:27.984262+0000","last_fullsized":"2023-07-06T03:53:27.984262+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:58:54.719047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985942+0000","last_change":"2023-07-06T03:53:26.776930+0000","last_active":"2023-07-06T03:53:27.985942+0000","last_peered":"2023-07-06T03:53:27.985942+0000","last_clean":"2023-07-06T03:53:27.985942+0000","last_became_active":"2023-07-06T03:53:26.776545+0000","last_became_peered":"2023-07-06T03:53:26.776545+0000","last_unstale":"2023-07-06T03:53:27.985942+0000","last_undegraded":"2023-07-06T03:53:27.985942+0000","last_fullsized":"2023-07-06T03:53:27.985942+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:39:30.128086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766324+0000","last_change":"2023-07-06T03:53:26.778779+0000","last_active":"2023-07-06T03:53:27.766324+0000","last_peered":"2023-07-06T03:53:27.766324+0000","last_clean":"2023-07-06T03:53:27.766324+0000","last_became_active":"2023-07-06T03:53:26.778493+0000","last_became_peered":"2023-07-06T03:53:26.778493+0000","last_unstale":"2023-07-06T03:53:27.766324+0000","last_undegraded":"2023-07-06T03:53:27.766324+0000","last_fullsized":"2023-07-06T03:53:27.766324+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:01:39.874372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.986050+0000","last_change":"2023-07-06T03:53:26.767725+0000","last_active":"2023-07-06T03:53:27.986050+0000","last_peered":"2023-07-06T03:53:27.986050+0000","last_clean":"2023-07-06T03:53:27.986050+0000","last_became_active":"2023-07-06T03:53:26.767284+0000","last_became_peered":"2023-07-06T03:53:26.767284+0000","last_unstale":"2023-07-06T03:53:27.986050+0000","last_undegraded":"2023-07-06T03:53:27.986050+0000","last_fullsized":"2023-07-06T03:53:27.986050+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:01:09.555706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984512+0000","last_change":"2023-07-06T03:53:26.768498+0000","last_active":"2023-07-06T03:53:27.984512+0000","last_peered":"2023-07-06T03:53:27.984512+0000","last_clean":"2023-07-06T03:53:27.984512+0000","last_became_active":"2023-07-06T03:53:26.768222+0000","last_became_peered":"2023-07-06T03:53:26.768222+0000","last_unstale":"2023-07-06T03:53:27.984512+0000","last_undegraded":"2023-07-06T03:53:27.984512+0000","last_fullsized":"2023-07-06T03:53:27.984512+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:04:54.519711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984363+0000","last_change":"2023-07-06T03:53:26.767442+0000","last_active":"2023-07-06T03:53:27.984363+0000","last_peered":"2023-07-06T03:53:27.984363+0000","last_clean":"2023-07-06T03:53:27.984363+0000","last_became_active":"2023-07-06T03:53:26.767137+0000","last_became_peered":"2023-07-06T03:53:26.767137+0000","last_unstale":"2023-07-06T03:53:27.984363+0000","last_undegraded":"2023-07-06T03:53:27.984363+0000","last_fullsized":"2023-07-06T03:53:27.984363+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:50:40.924032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984268+0000","last_change":"2023-07-06T03:53:26.767855+0000","last_active":"2023-07-06T03:53:27.984268+0000","last_peered":"2023-07-06T03:53:27.984268+0000","last_clean":"2023-07-06T03:53:27.984268+0000","last_became_active":"2023-07-06T03:53:26.767574+0000","last_became_peered":"2023-07-06T03:53:26.767574+0000","last_unstale":"2023-07-06T03:53:27.984268+0000","last_undegraded":"2023-07-06T03:53:27.984268+0000","last_fullsized":"2023-07-06T03:53:27.984268+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:05:41.719802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984228+0000","last_change":"2023-07-06T03:53:26.775201+0000","last_active":"2023-07-06T03:53:27.984228+0000","last_peered":"2023-07-06T03:53:27.984228+0000","last_clean":"2023-07-06T03:53:27.984228+0000","last_became_active":"2023-07-06T03:53:26.774910+0000","last_became_peered":"2023-07-06T03:53:26.774910+0000","last_unstale":"2023-07-06T03:53:27.984228+0000","last_undegraded":"2023-07-06T03:53:27.984228+0000","last_fullsized":"2023-07-06T03:53:27.984228+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:34:11.936629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765625+0000","last_change":"2023-07-06T03:53:26.767402+0000","last_active":"2023-07-06T03:53:27.765625+0000","last_peered":"2023-07-06T03:53:27.765625+0000","last_clean":"2023-07-06T03:53:27.765625+0000","last_became_active":"2023-07-06T03:53:26.767157+0000","last_became_peered":"2023-07-06T03:53:26.767157+0000","last_unstale":"2023-07-06T03:53:27.765625+0000","last_undegraded":"2023-07-06T03:53:27.765625+0000","last_fullsized":"2023-07-06T03:53:27.765625+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:35:53.378827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985731+0000","last_change":"2023-07-06T03:53:26.779468+0000","last_active":"2023-07-06T03:53:27.985731+0000","last_peered":"2023-07-06T03:53:27.985731+0000","last_clean":"2023-07-06T03:53:27.985731+0000","last_became_active":"2023-07-06T03:53:26.779126+0000","last_became_peered":"2023-07-06T03:53:26.779126+0000","last_unstale":"2023-07-06T03:53:27.985731+0000","last_undegraded":"2023-07-06T03:53:27.985731+0000","last_fullsized":"2023-07-06T03:53:27.985731+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:16:22.855747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764729+0000","last_change":"2023-07-06T03:53:26.774539+0000","last_active":"2023-07-06T03:53:27.764729+0000","last_peered":"2023-07-06T03:53:27.764729+0000","last_clean":"2023-07-06T03:53:27.764729+0000","last_became_active":"2023-07-06T03:53:26.774190+0000","last_became_peered":"2023-07-06T03:53:26.774190+0000","last_unstale":"2023-07-06T03:53:27.764729+0000","last_undegraded":"2023-07-06T03:53:27.764729+0000","last_fullsized":"2023-07-06T03:53:27.764729+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:03.798172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765168+0000","last_change":"2023-07-06T03:53:26.769569+0000","last_active":"2023-07-06T03:53:27.765168+0000","last_peered":"2023-07-06T03:53:27.765168+0000","last_clean":"2023-07-06T03:53:27.765168+0000","last_became_active":"2023-07-06T03:53:26.769044+0000","last_became_peered":"2023-07-06T03:53:26.769044+0000","last_unstale":"2023-07-06T03:53:27.765168+0000","last_undegraded":"2023-07-06T03:53:27.765168+0000","last_fullsized":"2023-07-06T03:53:27.765168+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:24:58.843963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765854+0000","last_change":"2023-07-06T03:53:26.769788+0000","last_active":"2023-07-06T03:53:27.765854+0000","last_peered":"2023-07-06T03:53:27.765854+0000","last_clean":"2023-07-06T03:53:27.765854+0000","last_became_active":"2023-07-06T03:53:26.769238+0000","last_became_peered":"2023-07-06T03:53:26.769238+0000","last_unstale":"2023-07-06T03:53:27.765854+0000","last_undegraded":"2023-07-06T03:53:27.765854+0000","last_fullsized":"2023-07-06T03:53:27.765854+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:49:22.408985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985676+0000","last_change":"2023-07-06T03:53:26.781836+0000","last_active":"2023-07-06T03:53:27.985676+0000","last_peered":"2023-07-06T03:53:27.985676+0000","last_clean":"2023-07-06T03:53:27.985676+0000","last_became_active":"2023-07-06T03:53:26.781605+0000","last_became_peered":"2023-07-06T03:53:26.781605+0000","last_unstale":"2023-07-06T03:53:27.985676+0000","last_undegraded":"2023-07-06T03:53:27.985676+0000","last_fullsized":"2023-07-06T03:53:27.985676+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:07:09.272116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"98'4","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766323+0000","last_change":"2023-07-06T03:53:25.758687+0000","last_active":"2023-07-06T03:53:27.766323+0000","last_peered":"2023-07-06T03:53:27.766323+0000","last_clean":"2023-07-06T03:53:27.766323+0000","last_became_active":"2023-07-06T03:53:25.758351+0000","last_became_peered":"2023-07-06T03:53:25.758351+0000","last_unstale":"2023-07-06T03:53:27.766323+0000","last_undegraded":"2023-07-06T03:53:27.766323+0000","last_fullsized":"2023-07-06T03:53:27.766323+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:22.697286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.986146+0000","last_change":"2023-07-06T03:53:26.767550+0000","last_active":"2023-07-06T03:53:27.986146+0000","last_peered":"2023-07-06T03:53:27.986146+0000","last_clean":"2023-07-06T03:53:27.986146+0000","last_became_active":"2023-07-06T03:53:26.766905+0000","last_became_peered":"2023-07-06T03:53:26.766905+0000","last_unstale":"2023-07-06T03:53:27.986146+0000","last_undegraded":"2023-07-06T03:53:27.986146+0000","last_fullsized":"2023-07-06T03:53:27.986146+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:18:09.726375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"98'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984940+0000","last_change":"2023-07-06T03:53:25.765786+0000","last_active":"2023-07-06T03:53:27.984940+0000","last_peered":"2023-07-06T03:53:27.984940+0000","last_clean":"2023-07-06T03:53:27.984940+0000","last_became_active":"2023-07-06T03:53:25.764804+0000","last_became_peered":"2023-07-06T03:53:25.764804+0000","last_unstale":"2023-07-06T03:53:27.984940+0000","last_undegraded":"2023-07-06T03:53:27.984940+0000","last_fullsized":"2023-07-06T03:53:27.984940+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:32:42.638594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898694+0000","last_change":"2023-07-06T03:53:26.775778+0000","last_active":"2023-07-06T03:53:27.898694+0000","last_peered":"2023-07-06T03:53:27.898694+0000","last_clean":"2023-07-06T03:53:27.898694+0000","last_became_active":"2023-07-06T03:53:26.775531+0000","last_became_peered":"2023-07-06T03:53:26.775531+0000","last_unstale":"2023-07-06T03:53:27.898694+0000","last_undegraded":"2023-07-06T03:53:27.898694+0000","last_fullsized":"2023-07-06T03:53:27.898694+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:29:20.204759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.899017+0000","last_change":"2023-07-06T03:53:25.761839+0000","last_active":"2023-07-06T03:53:27.899017+0000","last_peered":"2023-07-06T03:53:27.899017+0000","last_clean":"2023-07-06T03:53:27.899017+0000","last_became_active":"2023-07-06T03:53:25.761316+0000","last_became_peered":"2023-07-06T03:53:25.761316+0000","last_unstale":"2023-07-06T03:53:27.899017+0000","last_undegraded":"2023-07-06T03:53:27.899017+0000","last_fullsized":"2023-07-06T03:53:27.899017+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:12:28.876525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764521+0000","last_change":"2023-07-06T03:53:26.772652+0000","last_active":"2023-07-06T03:53:27.764521+0000","last_peered":"2023-07-06T03:53:27.764521+0000","last_clean":"2023-07-06T03:53:27.764521+0000","last_became_active":"2023-07-06T03:53:26.767214+0000","last_became_peered":"2023-07-06T03:53:26.767214+0000","last_unstale":"2023-07-06T03:53:27.764521+0000","last_undegraded":"2023-07-06T03:53:27.764521+0000","last_fullsized":"2023-07-06T03:53:27.764521+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:15:03.164217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.986187+0000","last_change":"2023-07-06T03:53:25.763450+0000","last_active":"2023-07-06T03:53:27.986187+0000","last_peered":"2023-07-06T03:53:27.986187+0000","last_clean":"2023-07-06T03:53:27.986187+0000","last_became_active":"2023-07-06T03:53:25.763153+0000","last_became_peered":"2023-07-06T03:53:25.763153+0000","last_unstale":"2023-07-06T03:53:27.986187+0000","last_undegraded":"2023-07-06T03:53:27.986187+0000","last_fullsized":"2023-07-06T03:53:27.986187+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:59:38.714520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985751+0000","last_change":"2023-07-06T03:53:26.774591+0000","last_active":"2023-07-06T03:53:27.985751+0000","last_peered":"2023-07-06T03:53:27.985751+0000","last_clean":"2023-07-06T03:53:27.985751+0000","last_became_active":"2023-07-06T03:53:26.774245+0000","last_became_peered":"2023-07-06T03:53:26.774245+0000","last_unstale":"2023-07-06T03:53:27.985751+0000","last_undegraded":"2023-07-06T03:53:27.985751+0000","last_fullsized":"2023-07-06T03:53:27.985751+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:17.131280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"98'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764783+0000","last_change":"2023-07-06T03:53:25.760397+0000","last_active":"2023-07-06T03:53:27.764783+0000","last_peered":"2023-07-06T03:53:27.764783+0000","last_clean":"2023-07-06T03:53:27.764783+0000","last_became_active":"2023-07-06T03:53:25.760151+0000","last_became_peered":"2023-07-06T03:53:25.760151+0000","last_unstale":"2023-07-06T03:53:27.764783+0000","last_undegraded":"2023-07-06T03:53:27.764783+0000","last_fullsized":"2023-07-06T03:53:27.764783+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:52:42.874715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985038+0000","last_change":"2023-07-06T03:53:26.774771+0000","last_active":"2023-07-06T03:53:27.985038+0000","last_peered":"2023-07-06T03:53:27.985038+0000","last_clean":"2023-07-06T03:53:27.985038+0000","last_became_active":"2023-07-06T03:53:26.774459+0000","last_became_peered":"2023-07-06T03:53:26.774459+0000","last_unstale":"2023-07-06T03:53:27.985038+0000","last_undegraded":"2023-07-06T03:53:27.985038+0000","last_fullsized":"2023-07-06T03:53:27.985038+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:50:40.924032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"98'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984087+0000","last_change":"2023-07-06T03:53:25.758308+0000","last_active":"2023-07-06T03:53:27.984087+0000","last_peered":"2023-07-06T03:53:27.984087+0000","last_clean":"2023-07-06T03:53:27.984087+0000","last_became_active":"2023-07-06T03:53:25.758128+0000","last_became_peered":"2023-07-06T03:53:25.758128+0000","last_unstale":"2023-07-06T03:53:27.984087+0000","last_undegraded":"2023-07-06T03:53:27.984087+0000","last_fullsized":"2023-07-06T03:53:27.984087+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:41:41.289345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765134+0000","last_change":"2023-07-06T03:53:26.775444+0000","last_active":"2023-07-06T03:53:27.765134+0000","last_peered":"2023-07-06T03:53:27.765134+0000","last_clean":"2023-07-06T03:53:27.765134+0000","last_became_active":"2023-07-06T03:53:26.775174+0000","last_became_peered":"2023-07-06T03:53:26.775174+0000","last_unstale":"2023-07-06T03:53:27.765134+0000","last_undegraded":"2023-07-06T03:53:27.765134+0000","last_fullsized":"2023-07-06T03:53:27.765134+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:05:43.296706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.983889+0000","last_change":"2023-07-06T03:53:25.757461+0000","last_active":"2023-07-06T03:53:27.983889+0000","last_peered":"2023-07-06T03:53:27.983889+0000","last_clean":"2023-07-06T03:53:27.983889+0000","last_became_active":"2023-07-06T03:53:25.757185+0000","last_became_peered":"2023-07-06T03:53:25.757185+0000","last_unstale":"2023-07-06T03:53:27.983889+0000","last_undegraded":"2023-07-06T03:53:27.983889+0000","last_fullsized":"2023-07-06T03:53:27.983889+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:44:53.941233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.986009+0000","last_change":"2023-07-06T03:53:26.755601+0000","last_active":"2023-07-06T03:53:27.986009+0000","last_peered":"2023-07-06T03:53:27.986009+0000","last_clean":"2023-07-06T03:53:27.986009+0000","last_became_active":"2023-07-06T03:53:26.755320+0000","last_became_peered":"2023-07-06T03:53:26.755320+0000","last_unstale":"2023-07-06T03:53:27.986009+0000","last_undegraded":"2023-07-06T03:53:27.986009+0000","last_fullsized":"2023-07-06T03:53:27.986009+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:56.563331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766115+0000","last_change":"2023-07-06T03:53:25.764596+0000","last_active":"2023-07-06T03:53:27.766115+0000","last_peered":"2023-07-06T03:53:27.766115+0000","last_clean":"2023-07-06T03:53:27.766115+0000","last_became_active":"2023-07-06T03:53:25.764126+0000","last_became_peered":"2023-07-06T03:53:25.764126+0000","last_unstale":"2023-07-06T03:53:27.766115+0000","last_undegraded":"2023-07-06T03:53:27.766115+0000","last_fullsized":"2023-07-06T03:53:27.766115+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:10:08.489885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984619+0000","last_change":"2023-07-06T03:53:26.773822+0000","last_active":"2023-07-06T03:53:27.984619+0000","last_peered":"2023-07-06T03:53:27.984619+0000","last_clean":"2023-07-06T03:53:27.984619+0000","last_became_active":"2023-07-06T03:53:26.773420+0000","last_became_peered":"2023-07-06T03:53:26.773420+0000","last_unstale":"2023-07-06T03:53:27.984619+0000","last_undegraded":"2023-07-06T03:53:27.984619+0000","last_fullsized":"2023-07-06T03:53:27.984619+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:41:41.173551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765980+0000","last_change":"2023-07-06T03:53:25.763023+0000","last_active":"2023-07-06T03:53:27.765980+0000","last_peered":"2023-07-06T03:53:27.765980+0000","last_clean":"2023-07-06T03:53:27.765980+0000","last_became_active":"2023-07-06T03:53:25.762691+0000","last_became_peered":"2023-07-06T03:53:25.762691+0000","last_unstale":"2023-07-06T03:53:27.765980+0000","last_undegraded":"2023-07-06T03:53:27.765980+0000","last_fullsized":"2023-07-06T03:53:27.765980+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:00:21.226381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985054+0000","last_change":"2023-07-06T03:53:26.762058+0000","last_active":"2023-07-06T03:53:27.985054+0000","last_peered":"2023-07-06T03:53:27.985054+0000","last_clean":"2023-07-06T03:53:27.985054+0000","last_became_active":"2023-07-06T03:53:26.761785+0000","last_became_peered":"2023-07-06T03:53:26.761785+0000","last_unstale":"2023-07-06T03:53:27.985054+0000","last_undegraded":"2023-07-06T03:53:27.985054+0000","last_fullsized":"2023-07-06T03:53:27.985054+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:11:51.981810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985657+0000","last_change":"2023-07-06T03:53:25.761618+0000","last_active":"2023-07-06T03:53:27.985657+0000","last_peered":"2023-07-06T03:53:27.985657+0000","last_clean":"2023-07-06T03:53:27.985657+0000","last_became_active":"2023-07-06T03:53:25.761270+0000","last_became_peered":"2023-07-06T03:53:25.761270+0000","last_unstale":"2023-07-06T03:53:27.985657+0000","last_undegraded":"2023-07-06T03:53:27.985657+0000","last_fullsized":"2023-07-06T03:53:27.985657+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:03.806749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985781+0000","last_change":"2023-07-06T03:53:26.766451+0000","last_active":"2023-07-06T03:53:27.985781+0000","last_peered":"2023-07-06T03:53:27.985781+0000","last_clean":"2023-07-06T03:53:27.985781+0000","last_became_active":"2023-07-06T03:53:26.766172+0000","last_became_peered":"2023-07-06T03:53:26.766172+0000","last_unstale":"2023-07-06T03:53:27.985781+0000","last_undegraded":"2023-07-06T03:53:27.985781+0000","last_fullsized":"2023-07-06T03:53:27.985781+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:49:28.832109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"98'3","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765657+0000","last_change":"2023-07-06T03:53:25.764734+0000","last_active":"2023-07-06T03:53:27.765657+0000","last_peered":"2023-07-06T03:53:27.765657+0000","last_clean":"2023-07-06T03:53:27.765657+0000","last_became_active":"2023-07-06T03:53:25.764430+0000","last_became_peered":"2023-07-06T03:53:25.764430+0000","last_unstale":"2023-07-06T03:53:27.765657+0000","last_undegraded":"2023-07-06T03:53:27.765657+0000","last_fullsized":"2023-07-06T03:53:27.765657+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:20:39.109310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765941+0000","last_change":"2023-07-06T03:53:26.768657+0000","last_active":"2023-07-06T03:53:27.765941+0000","last_peered":"2023-07-06T03:53:27.765941+0000","last_clean":"2023-07-06T03:53:27.765941+0000","last_became_active":"2023-07-06T03:53:26.768246+0000","last_became_peered":"2023-07-06T03:53:26.768246+0000","last_unstale":"2023-07-06T03:53:27.765941+0000","last_undegraded":"2023-07-06T03:53:27.765941+0000","last_fullsized":"2023-07-06T03:53:27.765941+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:41:54.520076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"98'6","reported_seq":19,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764785+0000","last_change":"2023-07-06T03:53:25.761999+0000","last_active":"2023-07-06T03:53:27.764785+0000","last_peered":"2023-07-06T03:53:27.764785+0000","last_clean":"2023-07-06T03:53:27.764785+0000","last_became_active":"2023-07-06T03:53:25.761778+0000","last_became_peered":"2023-07-06T03:53:25.761778+0000","last_unstale":"2023-07-06T03:53:27.764785+0000","last_undegraded":"2023-07-06T03:53:27.764785+0000","last_fullsized":"2023-07-06T03:53:27.764785+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:27:05.659397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898985+0000","last_change":"2023-07-06T03:53:26.771608+0000","last_active":"2023-07-06T03:53:27.898985+0000","last_peered":"2023-07-06T03:53:27.898985+0000","last_clean":"2023-07-06T03:53:27.898985+0000","last_became_active":"2023-07-06T03:53:26.771346+0000","last_became_peered":"2023-07-06T03:53:26.771346+0000","last_unstale":"2023-07-06T03:53:27.898985+0000","last_undegraded":"2023-07-06T03:53:27.898985+0000","last_fullsized":"2023-07-06T03:53:27.898985+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:42:58.642834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985528+0000","last_change":"2023-07-06T03:53:25.760521+0000","last_active":"2023-07-06T03:53:27.985528+0000","last_peered":"2023-07-06T03:53:27.985528+0000","last_clean":"2023-07-06T03:53:27.985528+0000","last_became_active":"2023-07-06T03:53:25.760210+0000","last_became_peered":"2023-07-06T03:53:25.760210+0000","last_unstale":"2023-07-06T03:53:27.985528+0000","last_undegraded":"2023-07-06T03:53:27.985528+0000","last_fullsized":"2023-07-06T03:53:27.985528+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:15:24.380475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.899008+0000","last_change":"2023-07-06T03:53:26.775942+0000","last_active":"2023-07-06T03:53:27.899008+0000","last_peered":"2023-07-06T03:53:27.899008+0000","last_clean":"2023-07-06T03:53:27.899008+0000","last_became_active":"2023-07-06T03:53:26.775637+0000","last_became_peered":"2023-07-06T03:53:26.775637+0000","last_unstale":"2023-07-06T03:53:27.899008+0000","last_undegraded":"2023-07-06T03:53:27.899008+0000","last_fullsized":"2023-07-06T03:53:27.899008+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:13:13.552629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985925+0000","last_change":"2023-07-06T03:53:25.761696+0000","last_active":"2023-07-06T03:53:27.985925+0000","last_peered":"2023-07-06T03:53:27.985925+0000","last_clean":"2023-07-06T03:53:27.985925+0000","last_became_active":"2023-07-06T03:53:25.761373+0000","last_became_peered":"2023-07-06T03:53:25.761373+0000","last_unstale":"2023-07-06T03:53:27.985925+0000","last_undegraded":"2023-07-06T03:53:27.985925+0000","last_fullsized":"2023-07-06T03:53:27.985925+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:59:10.019933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766171+0000","last_change":"2023-07-06T03:53:26.771653+0000","last_active":"2023-07-06T03:53:27.766171+0000","last_peered":"2023-07-06T03:53:27.766171+0000","last_clean":"2023-07-06T03:53:27.766171+0000","last_became_active":"2023-07-06T03:53:26.771323+0000","last_became_peered":"2023-07-06T03:53:26.771323+0000","last_unstale":"2023-07-06T03:53:27.766171+0000","last_undegraded":"2023-07-06T03:53:27.766171+0000","last_fullsized":"2023-07-06T03:53:27.766171+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:12:58.253299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985239+0000","last_change":"2023-07-06T03:53:25.763182+0000","last_active":"2023-07-06T03:53:27.985239+0000","last_peered":"2023-07-06T03:53:27.985239+0000","last_clean":"2023-07-06T03:53:27.985239+0000","last_became_active":"2023-07-06T03:53:25.762812+0000","last_became_peered":"2023-07-06T03:53:25.762812+0000","last_unstale":"2023-07-06T03:53:27.985239+0000","last_undegraded":"2023-07-06T03:53:27.985239+0000","last_fullsized":"2023-07-06T03:53:27.985239+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:01:12.056695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985203+0000","last_change":"2023-07-06T03:53:26.781871+0000","last_active":"2023-07-06T03:53:27.985203+0000","last_peered":"2023-07-06T03:53:27.985203+0000","last_clean":"2023-07-06T03:53:27.985203+0000","last_became_active":"2023-07-06T03:53:26.781596+0000","last_became_peered":"2023-07-06T03:53:26.781596+0000","last_unstale":"2023-07-06T03:53:27.985203+0000","last_undegraded":"2023-07-06T03:53:27.985203+0000","last_fullsized":"2023-07-06T03:53:27.985203+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:46.741875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"98'8","reported_seq":21,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764621+0000","last_change":"2023-07-06T03:53:25.761393+0000","last_active":"2023-07-06T03:53:27.764621+0000","last_peered":"2023-07-06T03:53:27.764621+0000","last_clean":"2023-07-06T03:53:27.764621+0000","last_became_active":"2023-07-06T03:53:25.761142+0000","last_became_peered":"2023-07-06T03:53:25.761142+0000","last_unstale":"2023-07-06T03:53:27.764621+0000","last_undegraded":"2023-07-06T03:53:27.764621+0000","last_fullsized":"2023-07-06T03:53:27.764621+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:13:01.669680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765967+0000","last_change":"2023-07-06T03:53:26.769578+0000","last_active":"2023-07-06T03:53:27.765967+0000","last_peered":"2023-07-06T03:53:27.765967+0000","last_clean":"2023-07-06T03:53:27.765967+0000","last_became_active":"2023-07-06T03:53:26.768848+0000","last_became_peered":"2023-07-06T03:53:26.768848+0000","last_unstale":"2023-07-06T03:53:27.765967+0000","last_undegraded":"2023-07-06T03:53:27.765967+0000","last_fullsized":"2023-07-06T03:53:27.765967+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:25.503614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985365+0000","last_change":"2023-07-06T03:53:25.762326+0000","last_active":"2023-07-06T03:53:27.985365+0000","last_peered":"2023-07-06T03:53:27.985365+0000","last_clean":"2023-07-06T03:53:27.985365+0000","last_became_active":"2023-07-06T03:53:25.762045+0000","last_became_peered":"2023-07-06T03:53:25.762045+0000","last_unstale":"2023-07-06T03:53:27.985365+0000","last_undegraded":"2023-07-06T03:53:27.985365+0000","last_fullsized":"2023-07-06T03:53:27.985365+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:34:43.595364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764446+0000","last_change":"2023-07-06T03:53:26.754853+0000","last_active":"2023-07-06T03:53:27.764446+0000","last_peered":"2023-07-06T03:53:27.764446+0000","last_clean":"2023-07-06T03:53:27.764446+0000","last_became_active":"2023-07-06T03:53:26.754574+0000","last_became_peered":"2023-07-06T03:53:26.754574+0000","last_unstale":"2023-07-06T03:53:27.764446+0000","last_undegraded":"2023-07-06T03:53:27.764446+0000","last_fullsized":"2023-07-06T03:53:27.764446+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:10:51.643805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"99'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985130+0000","last_change":"2023-07-06T03:53:25.766271+0000","last_active":"2023-07-06T03:53:27.985130+0000","last_peered":"2023-07-06T03:53:27.985130+0000","last_clean":"2023-07-06T03:53:27.985130+0000","last_became_active":"2023-07-06T03:53:25.765967+0000","last_became_peered":"2023-07-06T03:53:25.765967+0000","last_unstale":"2023-07-06T03:53:27.985130+0000","last_undegraded":"2023-07-06T03:53:27.985130+0000","last_fullsized":"2023-07-06T03:53:27.985130+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:08:38.037063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985595+0000","last_change":"2023-07-06T03:53:26.779463+0000","last_active":"2023-07-06T03:53:27.985595+0000","last_peered":"2023-07-06T03:53:27.985595+0000","last_clean":"2023-07-06T03:53:27.985595+0000","last_became_active":"2023-07-06T03:53:26.779147+0000","last_became_peered":"2023-07-06T03:53:26.779147+0000","last_unstale":"2023-07-06T03:53:27.985595+0000","last_undegraded":"2023-07-06T03:53:27.985595+0000","last_fullsized":"2023-07-06T03:53:27.985595+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:49:17.417072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765121+0000","last_change":"2023-07-06T03:53:25.761337+0000","last_active":"2023-07-06T03:53:27.765121+0000","last_peered":"2023-07-06T03:53:27.765121+0000","last_clean":"2023-07-06T03:53:27.765121+0000","last_became_active":"2023-07-06T03:53:25.761078+0000","last_became_peered":"2023-07-06T03:53:25.761078+0000","last_unstale":"2023-07-06T03:53:27.765121+0000","last_undegraded":"2023-07-06T03:53:27.765121+0000","last_fullsized":"2023-07-06T03:53:27.765121+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:06.279786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984101+0000","last_change":"2023-07-06T03:53:26.775752+0000","last_active":"2023-07-06T03:53:27.984101+0000","last_peered":"2023-07-06T03:53:27.984101+0000","last_clean":"2023-07-06T03:53:27.984101+0000","last_became_active":"2023-07-06T03:53:26.773382+0000","last_became_peered":"2023-07-06T03:53:26.773382+0000","last_unstale":"2023-07-06T03:53:27.984101+0000","last_undegraded":"2023-07-06T03:53:27.984101+0000","last_fullsized":"2023-07-06T03:53:27.984101+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:44:27.300515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985918+0000","last_change":"2023-07-06T03:53:25.760855+0000","last_active":"2023-07-06T03:53:27.985918+0000","last_peered":"2023-07-06T03:53:27.985918+0000","last_clean":"2023-07-06T03:53:27.985918+0000","last_became_active":"2023-07-06T03:53:25.760495+0000","last_became_peered":"2023-07-06T03:53:25.760495+0000","last_unstale":"2023-07-06T03:53:27.985918+0000","last_undegraded":"2023-07-06T03:53:27.985918+0000","last_fullsized":"2023-07-06T03:53:27.985918+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:04:09.404201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985605+0000","last_change":"2023-07-06T03:53:26.761781+0000","last_active":"2023-07-06T03:53:27.985605+0000","last_peered":"2023-07-06T03:53:27.985605+0000","last_clean":"2023-07-06T03:53:27.985605+0000","last_became_active":"2023-07-06T03:53:26.761512+0000","last_became_peered":"2023-07-06T03:53:26.761512+0000","last_unstale":"2023-07-06T03:53:27.985605+0000","last_undegraded":"2023-07-06T03:53:27.985605+0000","last_fullsized":"2023-07-06T03:53:27.985605+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:19:36.149238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764633+0000","last_change":"2023-07-06T03:53:25.765885+0000","last_active":"2023-07-06T03:53:27.764633+0000","last_peered":"2023-07-06T03:53:27.764633+0000","last_clean":"2023-07-06T03:53:27.764633+0000","last_became_active":"2023-07-06T03:53:25.765567+0000","last_became_peered":"2023-07-06T03:53:25.765567+0000","last_unstale":"2023-07-06T03:53:27.764633+0000","last_undegraded":"2023-07-06T03:53:27.764633+0000","last_fullsized":"2023-07-06T03:53:27.764633+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:48:34.979836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984981+0000","last_change":"2023-07-06T03:53:26.762908+0000","last_active":"2023-07-06T03:53:27.984981+0000","last_peered":"2023-07-06T03:53:27.984981+0000","last_clean":"2023-07-06T03:53:27.984981+0000","last_became_active":"2023-07-06T03:53:26.762531+0000","last_became_peered":"2023-07-06T03:53:26.762531+0000","last_unstale":"2023-07-06T03:53:27.984981+0000","last_undegraded":"2023-07-06T03:53:27.984981+0000","last_fullsized":"2023-07-06T03:53:27.984981+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:05:41.719802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984775+0000","last_change":"2023-07-06T03:53:25.766585+0000","last_active":"2023-07-06T03:53:27.984775+0000","last_peered":"2023-07-06T03:53:27.984775+0000","last_clean":"2023-07-06T03:53:27.984775+0000","last_became_active":"2023-07-06T03:53:25.765934+0000","last_became_peered":"2023-07-06T03:53:25.765934+0000","last_unstale":"2023-07-06T03:53:27.984775+0000","last_undegraded":"2023-07-06T03:53:27.984775+0000","last_fullsized":"2023-07-06T03:53:27.984775+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:29:26.930120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765731+0000","last_change":"2023-07-06T03:53:26.767035+0000","last_active":"2023-07-06T03:53:27.765731+0000","last_peered":"2023-07-06T03:53:27.765731+0000","last_clean":"2023-07-06T03:53:27.765731+0000","last_became_active":"2023-07-06T03:53:26.766782+0000","last_became_peered":"2023-07-06T03:53:26.766782+0000","last_unstale":"2023-07-06T03:53:27.765731+0000","last_undegraded":"2023-07-06T03:53:27.765731+0000","last_fullsized":"2023-07-06T03:53:27.765731+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:54.804766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.983952+0000","last_change":"2023-07-06T03:53:25.762440+0000","last_active":"2023-07-06T03:53:27.983952+0000","last_peered":"2023-07-06T03:53:27.983952+0000","last_clean":"2023-07-06T03:53:27.983952+0000","last_became_active":"2023-07-06T03:53:25.762126+0000","last_became_peered":"2023-07-06T03:53:25.762126+0000","last_unstale":"2023-07-06T03:53:27.983952+0000","last_undegraded":"2023-07-06T03:53:27.983952+0000","last_fullsized":"2023-07-06T03:53:27.983952+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:40:07.152494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898887+0000","last_change":"2023-07-06T03:53:26.783048+0000","last_active":"2023-07-06T03:53:27.898887+0000","last_peered":"2023-07-06T03:53:27.898887+0000","last_clean":"2023-07-06T03:53:27.898887+0000","last_became_active":"2023-07-06T03:53:26.782676+0000","last_became_peered":"2023-07-06T03:53:26.782676+0000","last_unstale":"2023-07-06T03:53:27.898887+0000","last_undegraded":"2023-07-06T03:53:27.898887+0000","last_fullsized":"2023-07-06T03:53:27.898887+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:37:59.312364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984453+0000","last_change":"2023-07-06T03:53:25.765395+0000","last_active":"2023-07-06T03:53:27.984453+0000","last_peered":"2023-07-06T03:53:27.984453+0000","last_clean":"2023-07-06T03:53:27.984453+0000","last_became_active":"2023-07-06T03:53:25.764440+0000","last_became_peered":"2023-07-06T03:53:25.764440+0000","last_unstale":"2023-07-06T03:53:27.984453+0000","last_undegraded":"2023-07-06T03:53:27.984453+0000","last_fullsized":"2023-07-06T03:53:27.984453+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:57:14.835518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766090+0000","last_change":"2023-07-06T03:53:26.768459+0000","last_active":"2023-07-06T03:53:27.766090+0000","last_peered":"2023-07-06T03:53:27.766090+0000","last_clean":"2023-07-06T03:53:27.766090+0000","last_became_active":"2023-07-06T03:53:26.768132+0000","last_became_peered":"2023-07-06T03:53:26.768132+0000","last_unstale":"2023-07-06T03:53:27.766090+0000","last_undegraded":"2023-07-06T03:53:27.766090+0000","last_fullsized":"2023-07-06T03:53:27.766090+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:49:38.338342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765344+0000","last_change":"2023-07-06T03:53:25.769061+0000","last_active":"2023-07-06T03:53:27.765344+0000","last_peered":"2023-07-06T03:53:27.765344+0000","last_clean":"2023-07-06T03:53:27.765344+0000","last_became_active":"2023-07-06T03:53:25.768721+0000","last_became_peered":"2023-07-06T03:53:25.768721+0000","last_unstale":"2023-07-06T03:53:27.765344+0000","last_undegraded":"2023-07-06T03:53:27.765344+0000","last_fullsized":"2023-07-06T03:53:27.765344+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:36:12.901646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":155,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985716+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:53:27.985716+0000","last_peered":"2023-07-06T03:53:27.985716+0000","last_clean":"2023-07-06T03:53:27.985716+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:53:27.985716+0000","last_undegraded":"2023-07-06T03:53:27.985716+0000","last_fullsized":"2023-07-06T03:53:27.985716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:46:05.213430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985606+0000","last_change":"2023-07-06T03:53:26.772280+0000","last_active":"2023-07-06T03:53:27.985606+0000","last_peered":"2023-07-06T03:53:27.985606+0000","last_clean":"2023-07-06T03:53:27.985606+0000","last_became_active":"2023-07-06T03:53:26.771964+0000","last_became_peered":"2023-07-06T03:53:26.771964+0000","last_unstale":"2023-07-06T03:53:27.985606+0000","last_undegraded":"2023-07-06T03:53:27.985606+0000","last_fullsized":"2023-07-06T03:53:27.985606+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:51:57.479581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"98'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766081+0000","last_change":"2023-07-06T03:53:25.764949+0000","last_active":"2023-07-06T03:53:27.766081+0000","last_peered":"2023-07-06T03:53:27.766081+0000","last_clean":"2023-07-06T03:53:27.766081+0000","last_became_active":"2023-07-06T03:53:25.764699+0000","last_became_peered":"2023-07-06T03:53:25.764699+0000","last_unstale":"2023-07-06T03:53:27.766081+0000","last_undegraded":"2023-07-06T03:53:27.766081+0000","last_fullsized":"2023-07-06T03:53:27.766081+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:33:16.539361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":161,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764846+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:53:27.764846+0000","last_peered":"2023-07-06T03:53:27.764846+0000","last_clean":"2023-07-06T03:53:27.764846+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:53:27.764846+0000","last_undegraded":"2023-07-06T03:53:27.764846+0000","last_fullsized":"2023-07-06T03:53:27.764846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:59:20.018968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766063+0000","last_change":"2023-07-06T03:53:26.771097+0000","last_active":"2023-07-06T03:53:27.766063+0000","last_peered":"2023-07-06T03:53:27.766063+0000","last_clean":"2023-07-06T03:53:27.766063+0000","last_became_active":"2023-07-06T03:53:26.770713+0000","last_became_peered":"2023-07-06T03:53:26.770713+0000","last_unstale":"2023-07-06T03:53:27.766063+0000","last_undegraded":"2023-07-06T03:53:27.766063+0000","last_fullsized":"2023-07-06T03:53:27.766063+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:06:44.273593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765577+0000","last_change":"2023-07-06T03:53:25.763534+0000","last_active":"2023-07-06T03:53:27.765577+0000","last_peered":"2023-07-06T03:53:27.765577+0000","last_clean":"2023-07-06T03:53:27.765577+0000","last_became_active":"2023-07-06T03:53:25.762916+0000","last_became_peered":"2023-07-06T03:53:25.762916+0000","last_unstale":"2023-07-06T03:53:27.765577+0000","last_undegraded":"2023-07-06T03:53:27.765577+0000","last_fullsized":"2023-07-06T03:53:27.765577+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:45:54.953300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765932+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:53:27.765932+0000","last_peered":"2023-07-06T03:53:27.765932+0000","last_clean":"2023-07-06T03:53:27.765932+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:53:27.765932+0000","last_undegraded":"2023-07-06T03:53:27.765932+0000","last_fullsized":"2023-07-06T03:53:27.765932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:52:37.364641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765197+0000","last_change":"2023-07-06T03:53:26.769630+0000","last_active":"2023-07-06T03:53:27.765197+0000","last_peered":"2023-07-06T03:53:27.765197+0000","last_clean":"2023-07-06T03:53:27.765197+0000","last_became_active":"2023-07-06T03:53:26.769319+0000","last_became_peered":"2023-07-06T03:53:26.769319+0000","last_unstale":"2023-07-06T03:53:27.765197+0000","last_undegraded":"2023-07-06T03:53:27.765197+0000","last_fullsized":"2023-07-06T03:53:27.765197+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:06:49.760154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984831+0000","last_change":"2023-07-06T03:53:25.763463+0000","last_active":"2023-07-06T03:53:27.984831+0000","last_peered":"2023-07-06T03:53:27.984831+0000","last_clean":"2023-07-06T03:53:27.984831+0000","last_became_active":"2023-07-06T03:53:25.763132+0000","last_became_peered":"2023-07-06T03:53:25.763132+0000","last_unstale":"2023-07-06T03:53:27.984831+0000","last_undegraded":"2023-07-06T03:53:27.984831+0000","last_fullsized":"2023-07-06T03:53:27.984831+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:45:54.212957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":161,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764950+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:53:27.764950+0000","last_peered":"2023-07-06T03:53:27.764950+0000","last_clean":"2023-07-06T03:53:27.764950+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:53:27.764950+0000","last_undegraded":"2023-07-06T03:53:27.764950+0000","last_fullsized":"2023-07-06T03:53:27.764950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:34:40.976673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":305,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984031+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:53:27.984031+0000","last_peered":"2023-07-06T03:53:27.984031+0000","last_clean":"2023-07-06T03:53:27.984031+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:53:27.984031+0000","last_undegraded":"2023-07-06T03:53:27.984031+0000","last_fullsized":"2023-07-06T03:53:27.984031+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:19.715411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.983846+0000","last_change":"2023-07-06T03:53:26.767875+0000","last_active":"2023-07-06T03:53:27.983846+0000","last_peered":"2023-07-06T03:53:27.983846+0000","last_clean":"2023-07-06T03:53:27.983846+0000","last_became_active":"2023-07-06T03:53:26.767555+0000","last_became_peered":"2023-07-06T03:53:26.767555+0000","last_unstale":"2023-07-06T03:53:27.983846+0000","last_undegraded":"2023-07-06T03:53:27.983846+0000","last_fullsized":"2023-07-06T03:53:27.983846+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:23:50.020063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"98'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984134+0000","last_change":"2023-07-06T03:53:25.758578+0000","last_active":"2023-07-06T03:53:27.984134+0000","last_peered":"2023-07-06T03:53:27.984134+0000","last_clean":"2023-07-06T03:53:27.984134+0000","last_became_active":"2023-07-06T03:53:25.758327+0000","last_became_peered":"2023-07-06T03:53:25.758327+0000","last_unstale":"2023-07-06T03:53:27.984134+0000","last_undegraded":"2023-07-06T03:53:27.984134+0000","last_fullsized":"2023-07-06T03:53:27.984134+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:41:40.170906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":161,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764464+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:53:27.764464+0000","last_peered":"2023-07-06T03:53:27.764464+0000","last_clean":"2023-07-06T03:53:27.764464+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:53:27.764464+0000","last_undegraded":"2023-07-06T03:53:27.764464+0000","last_fullsized":"2023-07-06T03:53:27.764464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:35:08.792353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985145+0000","last_change":"2023-07-06T03:53:26.760601+0000","last_active":"2023-07-06T03:53:27.985145+0000","last_peered":"2023-07-06T03:53:27.985145+0000","last_clean":"2023-07-06T03:53:27.985145+0000","last_became_active":"2023-07-06T03:53:26.760320+0000","last_became_peered":"2023-07-06T03:53:26.760320+0000","last_unstale":"2023-07-06T03:53:27.985145+0000","last_undegraded":"2023-07-06T03:53:27.985145+0000","last_fullsized":"2023-07-06T03:53:27.985145+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:37:41.364027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985751+0000","last_change":"2023-07-06T03:53:25.763925+0000","last_active":"2023-07-06T03:53:27.985751+0000","last_peered":"2023-07-06T03:53:27.985751+0000","last_clean":"2023-07-06T03:53:27.985751+0000","last_became_active":"2023-07-06T03:53:25.763655+0000","last_became_peered":"2023-07-06T03:53:25.763655+0000","last_unstale":"2023-07-06T03:53:27.985751+0000","last_undegraded":"2023-07-06T03:53:27.985751+0000","last_fullsized":"2023-07-06T03:53:27.985751+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:20:49.520910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":161,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765235+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:53:27.765235+0000","last_peered":"2023-07-06T03:53:27.765235+0000","last_clean":"2023-07-06T03:53:27.765235+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:53:27.765235+0000","last_undegraded":"2023-07-06T03:53:27.765235+0000","last_fullsized":"2023-07-06T03:53:27.765235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:07:23.923774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985907+0000","last_change":"2023-07-06T03:53:26.755755+0000","last_active":"2023-07-06T03:53:27.985907+0000","last_peered":"2023-07-06T03:53:27.985907+0000","last_clean":"2023-07-06T03:53:27.985907+0000","last_became_active":"2023-07-06T03:53:26.755429+0000","last_became_peered":"2023-07-06T03:53:26.755429+0000","last_unstale":"2023-07-06T03:53:27.985907+0000","last_undegraded":"2023-07-06T03:53:27.985907+0000","last_fullsized":"2023-07-06T03:53:27.985907+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:06:32.140996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.986169+0000","last_change":"2023-07-06T03:53:25.765118+0000","last_active":"2023-07-06T03:53:27.986169+0000","last_peered":"2023-07-06T03:53:27.986169+0000","last_clean":"2023-07-06T03:53:27.986169+0000","last_became_active":"2023-07-06T03:53:25.764820+0000","last_became_peered":"2023-07-06T03:53:25.764820+0000","last_unstale":"2023-07-06T03:53:27.986169+0000","last_undegraded":"2023-07-06T03:53:27.986169+0000","last_fullsized":"2023-07-06T03:53:27.986169+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:18:37.051743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":154,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984979+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:53:27.984979+0000","last_peered":"2023-07-06T03:53:27.984979+0000","last_clean":"2023-07-06T03:53:27.984979+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:53:27.984979+0000","last_undegraded":"2023-07-06T03:53:27.984979+0000","last_fullsized":"2023-07-06T03:53:27.984979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:50:33.095193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766291+0000","last_change":"2023-07-06T03:53:26.760661+0000","last_active":"2023-07-06T03:53:27.766291+0000","last_peered":"2023-07-06T03:53:27.766291+0000","last_clean":"2023-07-06T03:53:27.766291+0000","last_became_active":"2023-07-06T03:53:26.760245+0000","last_became_peered":"2023-07-06T03:53:26.760245+0000","last_unstale":"2023-07-06T03:53:27.766291+0000","last_undegraded":"2023-07-06T03:53:27.766291+0000","last_fullsized":"2023-07-06T03:53:27.766291+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:08:49.844088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985500+0000","last_change":"2023-07-06T03:53:25.766728+0000","last_active":"2023-07-06T03:53:27.985500+0000","last_peered":"2023-07-06T03:53:27.985500+0000","last_clean":"2023-07-06T03:53:27.985500+0000","last_became_active":"2023-07-06T03:53:25.766327+0000","last_became_peered":"2023-07-06T03:53:25.766327+0000","last_unstale":"2023-07-06T03:53:27.985500+0000","last_undegraded":"2023-07-06T03:53:27.985500+0000","last_fullsized":"2023-07-06T03:53:27.985500+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:25.296456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":160,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765327+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:53:27.765327+0000","last_peered":"2023-07-06T03:53:27.765327+0000","last_clean":"2023-07-06T03:53:27.765327+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:53:27.765327+0000","last_undegraded":"2023-07-06T03:53:27.765327+0000","last_fullsized":"2023-07-06T03:53:27.765327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:37.190611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574954,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36348,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335492,"statfs":{"total":96636764160,"available":96599543808,"internally_reserved":0,"allocated":1232896,"data_stored":559544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.45900000000000002,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":1.024,"5min":4.0570000000000004,"15min":4.0570000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67900000000000005,"15min":0.76600000000000001},"min":{"1min":0.51200000000000001,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.89700000000000002,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67800000000000005,"15min":0.72399999999999998},"min":{"1min":0.47199999999999998,"5min":0.45000000000000001,"15min":0.34000000000000002},"max":{"1min":1.091,"5min":1.091,"15min":2.6219999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70499999999999996,"15min":0.84599999999999997},"min":{"1min":0.47099999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94999999999999996,"5min":4.0389999999999997,"15min":6.6310000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.39700000000000002,"15min":0.35199999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":3.77},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.45000000000000001},"max":{"1min":0.93000000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.753}]},{"osd":3,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.48699999999999999,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75,"15min":0.753},"min":{"1min":0.57599999999999996,"5min":0.45400000000000001,"15min":0.43099999999999999},"max":{"1min":0.92700000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.89700000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.65800000000000003,"15min":0.70499999999999996},"min":{"1min":0.44600000000000001,"5min":0.38,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":0.95999999999999996,"15min":1.4590000000000001},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.69499999999999995},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0820000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82599999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72499999999999998,"15min":0.73999999999999999},"min":{"1min":0.46400000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":1.03,"15min":1.5349999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.51800000000000002,"5min":0.45200000000000001,"15min":0.42299999999999999},"max":{"1min":0.98199999999999998,"5min":1.006,"15min":1.3779999999999999},"last":0.84199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36352,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335488,"statfs":{"total":96636764160,"available":96599539712,"internally_reserved":0,"allocated":1236992,"data_stored":560456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70899999999999996,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.29399999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77200000000000002,"15min":0.73999999999999999},"min":{"1min":0.44700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.2889999999999999,"5min":3.3399999999999999,"15min":3.3399999999999999},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86099999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":1.1759999999999999},"last":0.88100000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.74399999999999999,"15min":0.68200000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":1.9890000000000001,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.9450000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69999999999999996,"15min":0.745},"min":{"1min":0.502,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":2.1850000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.53400000000000003,"5min":0.40500000000000003,"15min":0.373},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.70699999999999996},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.33200000000000002},"max":{"1min":0.92100000000000004,"5min":1.0309999999999999,"15min":1.1339999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67300000000000004,"15min":0.72499999999999998},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":0.98699999999999999,"15min":1.085},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.443,"15min":0.38600000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.218},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.77700000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":1.0489999999999999,"15min":1.198},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.80500000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97399999999999998,"5min":1.054,"15min":1.323},"last":0.749},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.81599999999999995},"min":{"1min":0.54700000000000004,"5min":0.504,"15min":0.41499999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.4139999999999999},"last":0.71099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1826816,"data_stored":1149352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.52400000000000002},"min":{"1min":0.26900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65100000000000002,"5min":1.002,"15min":2.8879999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.501,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.313},"max":{"1min":0.58099999999999996,"5min":0.78300000000000003,"15min":1.96},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45500000000000002,"15min":0.83099999999999996},"min":{"1min":0.313,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.57799999999999996,"5min":0.70799999999999996,"15min":13.055999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54500000000000004,"15min":0.621},"min":{"1min":0.35699999999999998,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":2.8479999999999999},"last":0.435}]},{"osd":3,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48099999999999998,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.71599999999999997,"15min":1.2709999999999999},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.73999999999999999,"15min":1.101},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.86199999999999999,"5min":1.0640000000000001,"15min":12.989000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.78700000000000003,"15min":0.82599999999999996},"min":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.46000000000000002},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.502,"5min":0.432,"15min":0.432},"max":{"1min":0.91200000000000003,"5min":1.0860000000000001,"15min":1.3100000000000001},"last":0.501},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.71699999999999997,"15min":1.101},"min":{"1min":0.433,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.94499999999999995,"5min":1.1020000000000001,"15min":13.023999999999999},"last":0.625}]},{"osd":6,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":0.81000000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76700000000000002,"15min":1.147},"min":{"1min":0.54000000000000004,"5min":0.48699999999999999,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":1.0569999999999999,"15min":12.941000000000001},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75800000000000001,"15min":0.81999999999999995},"min":{"1min":0.58399999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.071,"5min":1.071,"15min":2.806},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.751,"15min":0.80300000000000005},"min":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.83999999999999997,"5min":1.1319999999999999,"15min":1.337},"last":0.58599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1228800,"data_stored":559474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.28000000000000003,"5min":0.26400000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.753,"15min":0.753},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.439},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.248},"max":{"1min":0.51200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.29599999999999999},"max":{"1min":0.52400000000000002,"5min":0.69199999999999995,"15min":0.73699999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.48399999999999999,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.314},"max":{"1min":0.55800000000000005,"5min":0.67200000000000004,"15min":1.2030000000000001},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.32200000000000001,"5min":0.30099999999999999,"15min":0.29899999999999999},"max":{"1min":0.56200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.497},{"interface":"front","average":{"1min":0.498,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.63400000000000001,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":1.1699999999999999,"15min":1.171},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":0.88800000000000001,"5min":1.032,"15min":1.034},"last":0.628}]},{"osd":5,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.745,"15min":0.73199999999999998},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":0.90600000000000003,"5min":1.1080000000000001,"15min":1.1419999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72899999999999998,"15min":0.747},"min":{"1min":0.56399999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93700000000000006,"5min":1.1040000000000001,"15min":1.1220000000000001},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.51600000000000001},"max":{"1min":0.93300000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73599999999999999,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81299999999999994,"5min":1.0649999999999999,"15min":1.546},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":0.86599999999999999,"5min":1.0820000000000001,"15min":1.6830000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77400000000000002,"15min":1.0269999999999999},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":0.86199999999999999,"5min":1.0669999999999999,"15min":9.5960000000000001},"last":0.70299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40444,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331396,"statfs":{"total":96636764160,"available":96595349504,"internally_reserved":0,"allocated":1232896,"data_stored":559004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.47999999999999998,"15min":0.443},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.371},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.50800000000000001,"15min":0.48199999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.093,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.55000000000000004,"15min":0.51100000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":6.5960000000000001,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.57099999999999995,"15min":0.52400000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.657,"5min":3.657,"15min":3.657},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.63200000000000001,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":6.6390000000000002,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.1629999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":0.94299999999999995,"15min":0.78100000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":12.007,"5min":12.007,"15min":12.007},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81899999999999995,"15min":0.73199999999999998},"min":{"1min":0.49299999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.2200000000000002,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.79300000000000004,"15min":0.749},"min":{"1min":0.495,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":0.88300000000000001,"15min":0.79700000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.8440000000000003,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":0.94299999999999995,"15min":0.78600000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":11.696999999999999,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.4390000000000001,"5min":0.92400000000000004,"15min":0.80800000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":13.084,"5min":13.084,"15min":13.084},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.83299999999999996,"15min":0.83099999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.042,"5min":0.86499999999999999,"15min":0.81999999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":4.484,"5min":4.484,"15min":4.484},"last":0.68999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1236992,"data_stored":560456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.44900000000000001,"15min":0.58499999999999996},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.442},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45600000000000002,"15min":0.94899999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":15.911},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":1.0209999999999999},"min":{"1min":0.40300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.347,"5min":1.347,"15min":16.024000000000001},"last":0.51500000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.33600000000000002,"5min":0.32800000000000001,"15min":0.27400000000000002},"max":{"1min":3.25,"5min":3.25,"15min":3.25},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.54300000000000004,"15min":1.028},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73999999999999999,"15min":0.75600000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.90300000000000002},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.36699999999999999},"max":{"1min":1.724,"5min":1.724,"15min":5.7300000000000004},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75600000000000001,"15min":0.90500000000000003},"min":{"1min":0.49199999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.411,"5min":1.411,"15min":5.077},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78900000000000003,"15min":0.95699999999999996},"min":{"1min":0.57999999999999996,"5min":0.499,"15min":0.44800000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.66400000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76200000000000001,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.91400000000000003},"min":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.45600000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":5.157},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.82199999999999995},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76600000000000001,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.492,"5min":1.492,"15min":5.1189999999999998},"last":0.72099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36344,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335496,"statfs":{"total":96636764160,"available":96599547904,"internally_reserved":0,"allocated":1228800,"data_stored":558926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.41599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80700000000000005,"5min":1.784,"15min":1.784},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.38},"max":{"1min":0.81899999999999995,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.82799999999999996},"min":{"1min":0.39500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":6.258},"last":0.748},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68000000000000005,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":1.806,"15min":1.806},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.84599999999999997},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.29199999999999998},"max":{"1min":0.90100000000000002,"5min":0.91000000000000003,"15min":6.4740000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73799999999999999,"15min":0.92300000000000004},"min":{"1min":0.57699999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":6.4859999999999998},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.48499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.97799999999999998,"15min":1.25},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.503,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":0.99099999999999999,"15min":1.413},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59999999999999998,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77800000000000002,"5min":0.92100000000000004,"15min":1.3280000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.66000000000000003},"min":{"1min":0.50600000000000001,"5min":0.40200000000000002,"15min":0.36399999999999999},"max":{"1min":0.80200000000000005,"5min":0.94299999999999995,"15min":0.99299999999999999},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63700000000000001,"15min":0.66200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.81999999999999995,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.78600000000000003},"min":{"1min":0.52400000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":1.008,"15min":1.6839999999999999},"last":0.77900000000000003}]},{"osd":7,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.72299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41299999999999998},"max":{"1min":0.88200000000000001,"5min":0.95699999999999996,"15min":1.0660000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70399999999999996,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.99399999999999999,"5min":1.079,"15min":1.21},"last":0.54700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36924,"kb_used_data":1780,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334916,"statfs":{"total":96636764160,"available":96598953984,"internally_reserved":0,"allocated":1822720,"data_stored":1148365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70599999999999996,"15min":0.66100000000000003},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.32100000000000001},"max":{"1min":0.871,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.753},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68300000000000005,"15min":0.83299999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":6.4059999999999997},"last":0.78100000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":1.1970000000000001,"15min":1.546},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.79100000000000004,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.376},"max":{"1min":8.1120000000000001,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.80800000000000005,"15min":0.73999999999999999},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":8.1829999999999998,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73999999999999999,"15min":0.747},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":1.3140000000000001,"5min":4.484,"15min":4.484},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.627,"15min":0.628},"min":{"1min":0.46200000000000002,"5min":0.40899999999999997,"15min":0.34699999999999998},"max":{"1min":0.88500000000000001,"5min":1.1359999999999999,"15min":1.6240000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.42499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.89700000000000002,"5min":1.0569999999999999,"15min":2.718},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.48399999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91200000000000003,"5min":1.246,"15min":1.669},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.002,"5min":1.3360000000000001,"15min":2.0459999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.749,"15min":0.77800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0609999999999999,"15min":2.2919999999999998},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:53:36.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:53:36.227 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:53:36.232 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:53:36.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.270+0000 7fe495bff700 1 Processor -- start 2023-07-06T03:53:36.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.270+0000 7fe495bff700 1 -- start start 2023-07-06T03:53:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.271+0000 7fe495bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe49015c520 0x7fe4900cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.271+0000 7fe495bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe490160180 0x7fe4901605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe495bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4900cdae0 con 0x7fe490160b30 2023-07-06T03:53:36.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe495bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4900cd8d0 con 0x7fe490160180 2023-07-06T03:53:36.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe495bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4900b6640 con 0x7fe49015c520 2023-07-06T03:53:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe48effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe490160180 0x7fe4901605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe48effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe490160180 0x7fe4901605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59504/0 (socket says 172.21.15.97:59504) 2023-07-06T03:53:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe48effd700 1 -- 172.21.15.97:0/1148742136 learned_addr learned my addr 172.21.15.97:0/1148742136 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe48f7fe700 1 --2- 172.21.15.97:0/1148742136 >> v2:172.21.15.146:3300/0 conn(0x7fe49015c520 0x7fe4900cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe48effd700 1 -- 172.21.15.97:0/1148742136 >> v2:172.21.15.146:3300/0 conn(0x7fe49015c520 msgr2=0x7fe4900cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe48effd700 1 --2- 172.21.15.97:0/1148742136 >> v2:172.21.15.146:3300/0 conn(0x7fe49015c520 0x7fe4900cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe48effd700 1 -- 172.21.15.97:0/1148742136 >> v1:172.21.15.97:6789/0 conn(0x7fe490160b30 legacy=0x7fe4900cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe48effd700 1 -- 172.21.15.97:0/1148742136 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4900b6920 con 0x7fe490160180 2023-07-06T03:53:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.272+0000 7fe48e7fc700 1 -- 172.21.15.97:0/1148742136 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326776642 0 0) 0x7fe4900cdae0 con 0x7fe490160b30 2023-07-06T03:53:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.273+0000 7fe48effd700 1 --2- 172.21.15.97:0/1148742136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe490160180 0x7fe4901605f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe4900b4c70 tx=0x7fe484009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de366c8395465e95 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.273+0000 7fe48e7fc700 1 -- 172.21.15.97:0/1148742136 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484011830 con 0x7fe490160180 2023-07-06T03:53:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.273+0000 7fe48e7fc700 1 -- 172.21.15.97:0/1148742136 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe484011990 con 0x7fe490160180 2023-07-06T03:53:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.273+0000 7fe48e7fc700 1 -- 172.21.15.97:0/1148742136 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484011d20 con 0x7fe490160180 2023-07-06T03:53:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.273+0000 7fe495bff700 1 -- 172.21.15.97:0/1148742136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe490160180 msgr2=0x7fe4901605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.273+0000 7fe495bff700 1 --2- 172.21.15.97:0/1148742136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe490160180 0x7fe4901605f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe4900b4c70 tx=0x7fe484009bd0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.274+0000 7fe495bff700 1 -- 172.21.15.97:0/1148742136 shutdown_connections 2023-07-06T03:53:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.274+0000 7fe495bff700 1 --2- 172.21.15.97:0/1148742136 >> v2:172.21.15.146:3300/0 conn(0x7fe49015c520 0x7fe4900cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.274+0000 7fe495bff700 1 --2- 172.21.15.97:0/1148742136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe490160180 0x7fe4901605f0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.274+0000 7fe495bff700 1 -- 172.21.15.97:0/1148742136 >> 172.21.15.97:0/1148742136 conn(0x7fe490071790 msgr2=0x7fe490071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.274+0000 7fe495bff700 1 -- 172.21.15.97:0/1148742136 shutdown_connections 2023-07-06T03:53:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.274+0000 7fe495bff700 1 -- 172.21.15.97:0/1148742136 wait complete. 2023-07-06T03:53:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.275+0000 7fe495bff700 1 Processor -- start 2023-07-06T03:53:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.276+0000 7fe495bff700 1 -- start start 2023-07-06T03:53:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.277+0000 7fe495bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49015c520 0x7fe49014fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.277+0000 7fe495bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe490160180 0x7fe4900d2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.277+0000 7fe495bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe490160b30 0x7fe49014f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.277+0000 7fe495bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900d3bd0 con 0x7fe490160180 2023-07-06T03:53:36.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.277+0000 7fe495bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900d3330 con 0x7fe49015c520 2023-07-06T03:53:36.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.277+0000 7fe495bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900d3610 con 0x7fe490160b30 2023-07-06T03:53:36.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.277+0000 7fe48effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe490160180 0x7fe4900d2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.277+0000 7fe48f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49015c520 0x7fe49014fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.277+0000 7fe48effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe490160180 0x7fe4900d2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59514/0 (socket says 172.21.15.97:59514) 2023-07-06T03:53:36.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.277+0000 7fe48effd700 1 -- 172.21.15.97:0/555992966 learned_addr learned my addr 172.21.15.97:0/555992966 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:36.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.277+0000 7fe48ffff700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe490160b30 0x7fe49014f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe48effd700 1 -- 172.21.15.97:0/555992966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe490160b30 msgr2=0x7fe49014f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:36.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe48effd700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe490160b30 0x7fe49014f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe48effd700 1 -- 172.21.15.97:0/555992966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49015c520 msgr2=0x7fe49014fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:36.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe48effd700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49015c520 0x7fe49014fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe48effd700 1 -- 172.21.15.97:0/555992966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe484009550 con 0x7fe490160180 2023-07-06T03:53:36.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe48f7fe700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49015c520 0x7fe49014fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:53:36.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe48effd700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe490160180 0x7fe4900d2ce0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe484009c10 tx=0x7fe484004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:36.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe477fff700 1 -- 172.21.15.97:0/555992966 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484004510 con 0x7fe490160180 2023-07-06T03:53:36.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe477fff700 1 -- 172.21.15.97:0/555992966 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4840036a0 con 0x7fe490160180 2023-07-06T03:53:36.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe495bff700 1 -- 172.21.15.97:0/555992966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4900d38f0 con 0x7fe490160180 2023-07-06T03:53:36.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe477fff700 1 -- 172.21.15.97:0/555992966 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484003a10 con 0x7fe490160180 2023-07-06T03:53:36.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.278+0000 7fe495bff700 1 -- 172.21.15.97:0/555992966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe490150520 con 0x7fe490160180 2023-07-06T03:53:36.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.280+0000 7fe495bff700 1 -- 172.21.15.97:0/555992966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4901f5ae0 con 0x7fe490160180 2023-07-06T03:53:36.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.281+0000 7fe477fff700 1 -- 172.21.15.97:0/555992966 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe484003b70 con 0x7fe490160180 2023-07-06T03:53:36.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.281+0000 7fe477fff700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe47c06ace0 0x7fe47c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.282+0000 7fe477fff700 1 -- 172.21.15.97:0/555992966 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe484021030 con 0x7fe490160180 2023-07-06T03:53:36.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.282+0000 7fe48f7fe700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe47c06ace0 0x7fe47c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.282+0000 7fe48f7fe700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe47c06ace0 0x7fe47c06d190 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe4901e66a0 tx=0x7fe480009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:36.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.286+0000 7fe477fff700 1 -- 172.21.15.97:0/555992966 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe4901f5ae0 con 0x7fe490160180 2023-07-06T03:53:36.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.483+0000 7fe495bff700 1 -- 172.21.15.97:0/555992966 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4900b6ee0 con 0x7fe47c06ace0 2023-07-06T03:53:36.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.492+0000 7fe477fff700 1 -- 172.21.15.97:0/555992966 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311411 (secure 0 0 0) 0x7fe480011c50 con 0x7fe47c06ace0 2023-07-06T03:53:36.497 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:36.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.496+0000 7fe495bff700 1 -- 172.21.15.97:0/555992966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe47c06ace0 msgr2=0x7fe47c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:36.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.496+0000 7fe495bff700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe47c06ace0 0x7fe47c06d190 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe4901e66a0 tx=0x7fe480009210 comp rx=0 tx=0).stop 2023-07-06T03:53:36.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.496+0000 7fe495bff700 1 -- 172.21.15.97:0/555992966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe490160180 msgr2=0x7fe4900d2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:36.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.496+0000 7fe495bff700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe490160180 0x7fe4900d2ce0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe484009c10 tx=0x7fe484004750 comp rx=0 tx=0).stop 2023-07-06T03:53:36.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.497+0000 7fe495bff700 1 -- 172.21.15.97:0/555992966 shutdown_connections 2023-07-06T03:53:36.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.497+0000 7fe495bff700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe47c06ace0 0x7fe47c06d190 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.497+0000 7fe495bff700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe490160b30 0x7fe49014f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.497+0000 7fe495bff700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe49015c520 0x7fe49014fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.497+0000 7fe495bff700 1 --2- 172.21.15.97:0/555992966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe490160180 0x7fe4900d2ce0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.497+0000 7fe495bff700 1 -- 172.21.15.97:0/555992966 >> 172.21.15.97:0/555992966 conn(0x7fe490071790 msgr2=0x7fe4901580b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:36.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.497+0000 7fe495bff700 1 -- 172.21.15.97:0/555992966 shutdown_connections 2023-07-06T03:53:36.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.497+0000 7fe495bff700 1 -- 172.21.15.97:0/555992966 wait complete. 2023-07-06T03:53:36.501 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:53:36.516 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2023-07-06T03:53:34.993442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283748,"kb_used_data":10788,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754690972,"statfs":{"total":773094113280,"available":772803555328,"internally_reserved":0,"allocated":11046912,"data_stored":5655577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.244734"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985178+0000","last_change":"2023-07-06T03:53:26.775067+0000","last_active":"2023-07-06T03:53:27.985178+0000","last_peered":"2023-07-06T03:53:27.985178+0000","last_clean":"2023-07-06T03:53:27.985178+0000","last_became_active":"2023-07-06T03:53:26.774259+0000","last_became_peered":"2023-07-06T03:53:26.774259+0000","last_unstale":"2023-07-06T03:53:27.985178+0000","last_undegraded":"2023-07-06T03:53:27.985178+0000","last_fullsized":"2023-07-06T03:53:27.985178+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:47:37.797380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898845+0000","last_change":"2023-07-06T03:53:26.782230+0000","last_active":"2023-07-06T03:53:27.898845+0000","last_peered":"2023-07-06T03:53:27.898845+0000","last_clean":"2023-07-06T03:53:27.898845+0000","last_became_active":"2023-07-06T03:53:26.781962+0000","last_became_peered":"2023-07-06T03:53:26.781962+0000","last_unstale":"2023-07-06T03:53:27.898845+0000","last_undegraded":"2023-07-06T03:53:27.898845+0000","last_fullsized":"2023-07-06T03:53:27.898845+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:16.272580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985475+0000","last_change":"2023-07-06T03:53:26.782591+0000","last_active":"2023-07-06T03:53:27.985475+0000","last_peered":"2023-07-06T03:53:27.985475+0000","last_clean":"2023-07-06T03:53:27.985475+0000","last_became_active":"2023-07-06T03:53:26.782251+0000","last_became_peered":"2023-07-06T03:53:26.782251+0000","last_unstale":"2023-07-06T03:53:27.985475+0000","last_undegraded":"2023-07-06T03:53:27.985475+0000","last_fullsized":"2023-07-06T03:53:27.985475+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:40:27.706089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985466+0000","last_change":"2023-07-06T03:53:26.766314+0000","last_active":"2023-07-06T03:53:27.985466+0000","last_peered":"2023-07-06T03:53:27.985466+0000","last_clean":"2023-07-06T03:53:27.985466+0000","last_became_active":"2023-07-06T03:53:26.765989+0000","last_became_peered":"2023-07-06T03:53:26.765989+0000","last_unstale":"2023-07-06T03:53:27.985466+0000","last_undegraded":"2023-07-06T03:53:27.985466+0000","last_fullsized":"2023-07-06T03:53:27.985466+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:47:02.552002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985370+0000","last_change":"2023-07-06T03:53:26.778493+0000","last_active":"2023-07-06T03:53:27.985370+0000","last_peered":"2023-07-06T03:53:27.985370+0000","last_clean":"2023-07-06T03:53:27.985370+0000","last_became_active":"2023-07-06T03:53:26.778154+0000","last_became_peered":"2023-07-06T03:53:26.778154+0000","last_unstale":"2023-07-06T03:53:27.985370+0000","last_undegraded":"2023-07-06T03:53:27.985370+0000","last_fullsized":"2023-07-06T03:53:27.985370+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:28:19.541129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765535+0000","last_change":"2023-07-06T03:53:26.774404+0000","last_active":"2023-07-06T03:53:27.765535+0000","last_peered":"2023-07-06T03:53:27.765535+0000","last_clean":"2023-07-06T03:53:27.765535+0000","last_became_active":"2023-07-06T03:53:26.773965+0000","last_became_peered":"2023-07-06T03:53:26.773965+0000","last_unstale":"2023-07-06T03:53:27.765535+0000","last_undegraded":"2023-07-06T03:53:27.765535+0000","last_fullsized":"2023-07-06T03:53:27.765535+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:04:53.424979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765322+0000","last_change":"2023-07-06T03:53:26.769406+0000","last_active":"2023-07-06T03:53:27.765322+0000","last_peered":"2023-07-06T03:53:27.765322+0000","last_clean":"2023-07-06T03:53:27.765322+0000","last_became_active":"2023-07-06T03:53:26.768913+0000","last_became_peered":"2023-07-06T03:53:26.768913+0000","last_unstale":"2023-07-06T03:53:27.765322+0000","last_undegraded":"2023-07-06T03:53:27.765322+0000","last_fullsized":"2023-07-06T03:53:27.765322+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:33:44.769610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984341+0000","last_change":"2023-07-06T03:53:26.770178+0000","last_active":"2023-07-06T03:53:27.984341+0000","last_peered":"2023-07-06T03:53:27.984341+0000","last_clean":"2023-07-06T03:53:27.984341+0000","last_became_active":"2023-07-06T03:53:26.769900+0000","last_became_peered":"2023-07-06T03:53:26.769900+0000","last_unstale":"2023-07-06T03:53:27.984341+0000","last_undegraded":"2023-07-06T03:53:27.984341+0000","last_fullsized":"2023-07-06T03:53:27.984341+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:29.566445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.899045+0000","last_change":"2023-07-06T03:53:26.778308+0000","last_active":"2023-07-06T03:53:27.899045+0000","last_peered":"2023-07-06T03:53:27.899045+0000","last_clean":"2023-07-06T03:53:27.899045+0000","last_became_active":"2023-07-06T03:53:26.778012+0000","last_became_peered":"2023-07-06T03:53:26.778012+0000","last_unstale":"2023-07-06T03:53:27.899045+0000","last_undegraded":"2023-07-06T03:53:27.899045+0000","last_fullsized":"2023-07-06T03:53:27.899045+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:09:32.641258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985429+0000","last_change":"2023-07-06T03:53:26.769312+0000","last_active":"2023-07-06T03:53:27.985429+0000","last_peered":"2023-07-06T03:53:27.985429+0000","last_clean":"2023-07-06T03:53:27.985429+0000","last_became_active":"2023-07-06T03:53:26.769039+0000","last_became_peered":"2023-07-06T03:53:26.769039+0000","last_unstale":"2023-07-06T03:53:27.985429+0000","last_undegraded":"2023-07-06T03:53:27.985429+0000","last_fullsized":"2023-07-06T03:53:27.985429+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:42.237225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764992+0000","last_change":"2023-07-06T03:53:26.776295+0000","last_active":"2023-07-06T03:53:27.764992+0000","last_peered":"2023-07-06T03:53:27.764992+0000","last_clean":"2023-07-06T03:53:27.764992+0000","last_became_active":"2023-07-06T03:53:26.776001+0000","last_became_peered":"2023-07-06T03:53:26.776001+0000","last_unstale":"2023-07-06T03:53:27.764992+0000","last_undegraded":"2023-07-06T03:53:27.764992+0000","last_fullsized":"2023-07-06T03:53:27.764992+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:58:41.795378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765452+0000","last_change":"2023-07-06T03:53:26.764902+0000","last_active":"2023-07-06T03:53:27.765452+0000","last_peered":"2023-07-06T03:53:27.765452+0000","last_clean":"2023-07-06T03:53:27.765452+0000","last_became_active":"2023-07-06T03:53:26.764619+0000","last_became_peered":"2023-07-06T03:53:26.764619+0000","last_unstale":"2023-07-06T03:53:27.765452+0000","last_undegraded":"2023-07-06T03:53:27.765452+0000","last_fullsized":"2023-07-06T03:53:27.765452+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:22:06.631745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985419+0000","last_change":"2023-07-06T03:53:26.773421+0000","last_active":"2023-07-06T03:53:27.985419+0000","last_peered":"2023-07-06T03:53:27.985419+0000","last_clean":"2023-07-06T03:53:27.985419+0000","last_became_active":"2023-07-06T03:53:26.773138+0000","last_became_peered":"2023-07-06T03:53:26.773138+0000","last_unstale":"2023-07-06T03:53:27.985419+0000","last_undegraded":"2023-07-06T03:53:27.985419+0000","last_fullsized":"2023-07-06T03:53:27.985419+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:50:14.873622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765534+0000","last_change":"2023-07-06T03:53:26.769278+0000","last_active":"2023-07-06T03:53:27.765534+0000","last_peered":"2023-07-06T03:53:27.765534+0000","last_clean":"2023-07-06T03:53:27.765534+0000","last_became_active":"2023-07-06T03:53:26.768992+0000","last_became_peered":"2023-07-06T03:53:26.768992+0000","last_unstale":"2023-07-06T03:53:27.765534+0000","last_undegraded":"2023-07-06T03:53:27.765534+0000","last_fullsized":"2023-07-06T03:53:27.765534+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:54.432067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898693+0000","last_change":"2023-07-06T03:53:26.780150+0000","last_active":"2023-07-06T03:53:27.898693+0000","last_peered":"2023-07-06T03:53:27.898693+0000","last_clean":"2023-07-06T03:53:27.898693+0000","last_became_active":"2023-07-06T03:53:26.779845+0000","last_became_peered":"2023-07-06T03:53:26.779845+0000","last_unstale":"2023-07-06T03:53:27.898693+0000","last_undegraded":"2023-07-06T03:53:27.898693+0000","last_fullsized":"2023-07-06T03:53:27.898693+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:40:49.166824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984200+0000","last_change":"2023-07-06T03:53:26.770953+0000","last_active":"2023-07-06T03:53:27.984200+0000","last_peered":"2023-07-06T03:53:27.984200+0000","last_clean":"2023-07-06T03:53:27.984200+0000","last_became_active":"2023-07-06T03:53:26.770621+0000","last_became_peered":"2023-07-06T03:53:26.770621+0000","last_unstale":"2023-07-06T03:53:27.984200+0000","last_undegraded":"2023-07-06T03:53:27.984200+0000","last_fullsized":"2023-07-06T03:53:27.984200+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:45:55.215602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985031+0000","last_change":"2023-07-06T03:53:26.782452+0000","last_active":"2023-07-06T03:53:27.985031+0000","last_peered":"2023-07-06T03:53:27.985031+0000","last_clean":"2023-07-06T03:53:27.985031+0000","last_became_active":"2023-07-06T03:53:26.782166+0000","last_became_peered":"2023-07-06T03:53:26.782166+0000","last_unstale":"2023-07-06T03:53:27.985031+0000","last_undegraded":"2023-07-06T03:53:27.985031+0000","last_fullsized":"2023-07-06T03:53:27.985031+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:19.890315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898981+0000","last_change":"2023-07-06T03:53:26.771445+0000","last_active":"2023-07-06T03:53:27.898981+0000","last_peered":"2023-07-06T03:53:27.898981+0000","last_clean":"2023-07-06T03:53:27.898981+0000","last_became_active":"2023-07-06T03:53:26.771157+0000","last_became_peered":"2023-07-06T03:53:26.771157+0000","last_unstale":"2023-07-06T03:53:27.898981+0000","last_undegraded":"2023-07-06T03:53:27.898981+0000","last_fullsized":"2023-07-06T03:53:27.898981+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:26:05.048391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984889+0000","last_change":"2023-07-06T03:53:26.770316+0000","last_active":"2023-07-06T03:53:27.984889+0000","last_peered":"2023-07-06T03:53:27.984889+0000","last_clean":"2023-07-06T03:53:27.984889+0000","last_became_active":"2023-07-06T03:53:26.769996+0000","last_became_peered":"2023-07-06T03:53:26.769996+0000","last_unstale":"2023-07-06T03:53:27.984889+0000","last_undegraded":"2023-07-06T03:53:27.984889+0000","last_fullsized":"2023-07-06T03:53:27.984889+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:34:11.936629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984262+0000","last_change":"2023-07-06T03:53:26.770241+0000","last_active":"2023-07-06T03:53:27.984262+0000","last_peered":"2023-07-06T03:53:27.984262+0000","last_clean":"2023-07-06T03:53:27.984262+0000","last_became_active":"2023-07-06T03:53:26.770038+0000","last_became_peered":"2023-07-06T03:53:26.770038+0000","last_unstale":"2023-07-06T03:53:27.984262+0000","last_undegraded":"2023-07-06T03:53:27.984262+0000","last_fullsized":"2023-07-06T03:53:27.984262+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:58:54.719047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985942+0000","last_change":"2023-07-06T03:53:26.776930+0000","last_active":"2023-07-06T03:53:27.985942+0000","last_peered":"2023-07-06T03:53:27.985942+0000","last_clean":"2023-07-06T03:53:27.985942+0000","last_became_active":"2023-07-06T03:53:26.776545+0000","last_became_peered":"2023-07-06T03:53:26.776545+0000","last_unstale":"2023-07-06T03:53:27.985942+0000","last_undegraded":"2023-07-06T03:53:27.985942+0000","last_fullsized":"2023-07-06T03:53:27.985942+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:39:30.128086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766324+0000","last_change":"2023-07-06T03:53:26.778779+0000","last_active":"2023-07-06T03:53:27.766324+0000","last_peered":"2023-07-06T03:53:27.766324+0000","last_clean":"2023-07-06T03:53:27.766324+0000","last_became_active":"2023-07-06T03:53:26.778493+0000","last_became_peered":"2023-07-06T03:53:26.778493+0000","last_unstale":"2023-07-06T03:53:27.766324+0000","last_undegraded":"2023-07-06T03:53:27.766324+0000","last_fullsized":"2023-07-06T03:53:27.766324+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:01:39.874372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.986050+0000","last_change":"2023-07-06T03:53:26.767725+0000","last_active":"2023-07-06T03:53:27.986050+0000","last_peered":"2023-07-06T03:53:27.986050+0000","last_clean":"2023-07-06T03:53:27.986050+0000","last_became_active":"2023-07-06T03:53:26.767284+0000","last_became_peered":"2023-07-06T03:53:26.767284+0000","last_unstale":"2023-07-06T03:53:27.986050+0000","last_undegraded":"2023-07-06T03:53:27.986050+0000","last_fullsized":"2023-07-06T03:53:27.986050+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:01:09.555706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984512+0000","last_change":"2023-07-06T03:53:26.768498+0000","last_active":"2023-07-06T03:53:27.984512+0000","last_peered":"2023-07-06T03:53:27.984512+0000","last_clean":"2023-07-06T03:53:27.984512+0000","last_became_active":"2023-07-06T03:53:26.768222+0000","last_became_peered":"2023-07-06T03:53:26.768222+0000","last_unstale":"2023-07-06T03:53:27.984512+0000","last_undegraded":"2023-07-06T03:53:27.984512+0000","last_fullsized":"2023-07-06T03:53:27.984512+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:04:54.519711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984363+0000","last_change":"2023-07-06T03:53:26.767442+0000","last_active":"2023-07-06T03:53:27.984363+0000","last_peered":"2023-07-06T03:53:27.984363+0000","last_clean":"2023-07-06T03:53:27.984363+0000","last_became_active":"2023-07-06T03:53:26.767137+0000","last_became_peered":"2023-07-06T03:53:26.767137+0000","last_unstale":"2023-07-06T03:53:27.984363+0000","last_undegraded":"2023-07-06T03:53:27.984363+0000","last_fullsized":"2023-07-06T03:53:27.984363+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:50:40.924032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984268+0000","last_change":"2023-07-06T03:53:26.767855+0000","last_active":"2023-07-06T03:53:27.984268+0000","last_peered":"2023-07-06T03:53:27.984268+0000","last_clean":"2023-07-06T03:53:27.984268+0000","last_became_active":"2023-07-06T03:53:26.767574+0000","last_became_peered":"2023-07-06T03:53:26.767574+0000","last_unstale":"2023-07-06T03:53:27.984268+0000","last_undegraded":"2023-07-06T03:53:27.984268+0000","last_fullsized":"2023-07-06T03:53:27.984268+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:05:41.719802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984228+0000","last_change":"2023-07-06T03:53:26.775201+0000","last_active":"2023-07-06T03:53:27.984228+0000","last_peered":"2023-07-06T03:53:27.984228+0000","last_clean":"2023-07-06T03:53:27.984228+0000","last_became_active":"2023-07-06T03:53:26.774910+0000","last_became_peered":"2023-07-06T03:53:26.774910+0000","last_unstale":"2023-07-06T03:53:27.984228+0000","last_undegraded":"2023-07-06T03:53:27.984228+0000","last_fullsized":"2023-07-06T03:53:27.984228+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:34:11.936629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765625+0000","last_change":"2023-07-06T03:53:26.767402+0000","last_active":"2023-07-06T03:53:27.765625+0000","last_peered":"2023-07-06T03:53:27.765625+0000","last_clean":"2023-07-06T03:53:27.765625+0000","last_became_active":"2023-07-06T03:53:26.767157+0000","last_became_peered":"2023-07-06T03:53:26.767157+0000","last_unstale":"2023-07-06T03:53:27.765625+0000","last_undegraded":"2023-07-06T03:53:27.765625+0000","last_fullsized":"2023-07-06T03:53:27.765625+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:35:53.378827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985731+0000","last_change":"2023-07-06T03:53:26.779468+0000","last_active":"2023-07-06T03:53:27.985731+0000","last_peered":"2023-07-06T03:53:27.985731+0000","last_clean":"2023-07-06T03:53:27.985731+0000","last_became_active":"2023-07-06T03:53:26.779126+0000","last_became_peered":"2023-07-06T03:53:26.779126+0000","last_unstale":"2023-07-06T03:53:27.985731+0000","last_undegraded":"2023-07-06T03:53:27.985731+0000","last_fullsized":"2023-07-06T03:53:27.985731+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:16:22.855747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764729+0000","last_change":"2023-07-06T03:53:26.774539+0000","last_active":"2023-07-06T03:53:27.764729+0000","last_peered":"2023-07-06T03:53:27.764729+0000","last_clean":"2023-07-06T03:53:27.764729+0000","last_became_active":"2023-07-06T03:53:26.774190+0000","last_became_peered":"2023-07-06T03:53:26.774190+0000","last_unstale":"2023-07-06T03:53:27.764729+0000","last_undegraded":"2023-07-06T03:53:27.764729+0000","last_fullsized":"2023-07-06T03:53:27.764729+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:03.798172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765168+0000","last_change":"2023-07-06T03:53:26.769569+0000","last_active":"2023-07-06T03:53:27.765168+0000","last_peered":"2023-07-06T03:53:27.765168+0000","last_clean":"2023-07-06T03:53:27.765168+0000","last_became_active":"2023-07-06T03:53:26.769044+0000","last_became_peered":"2023-07-06T03:53:26.769044+0000","last_unstale":"2023-07-06T03:53:27.765168+0000","last_undegraded":"2023-07-06T03:53:27.765168+0000","last_fullsized":"2023-07-06T03:53:27.765168+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:24:58.843963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765854+0000","last_change":"2023-07-06T03:53:26.769788+0000","last_active":"2023-07-06T03:53:27.765854+0000","last_peered":"2023-07-06T03:53:27.765854+0000","last_clean":"2023-07-06T03:53:27.765854+0000","last_became_active":"2023-07-06T03:53:26.769238+0000","last_became_peered":"2023-07-06T03:53:26.769238+0000","last_unstale":"2023-07-06T03:53:27.765854+0000","last_undegraded":"2023-07-06T03:53:27.765854+0000","last_fullsized":"2023-07-06T03:53:27.765854+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:49:22.408985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985676+0000","last_change":"2023-07-06T03:53:26.781836+0000","last_active":"2023-07-06T03:53:27.985676+0000","last_peered":"2023-07-06T03:53:27.985676+0000","last_clean":"2023-07-06T03:53:27.985676+0000","last_became_active":"2023-07-06T03:53:26.781605+0000","last_became_peered":"2023-07-06T03:53:26.781605+0000","last_unstale":"2023-07-06T03:53:27.985676+0000","last_undegraded":"2023-07-06T03:53:27.985676+0000","last_fullsized":"2023-07-06T03:53:27.985676+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:07:09.272116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"98'4","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766323+0000","last_change":"2023-07-06T03:53:25.758687+0000","last_active":"2023-07-06T03:53:27.766323+0000","last_peered":"2023-07-06T03:53:27.766323+0000","last_clean":"2023-07-06T03:53:27.766323+0000","last_became_active":"2023-07-06T03:53:25.758351+0000","last_became_peered":"2023-07-06T03:53:25.758351+0000","last_unstale":"2023-07-06T03:53:27.766323+0000","last_undegraded":"2023-07-06T03:53:27.766323+0000","last_fullsized":"2023-07-06T03:53:27.766323+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:27:22.697286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.986146+0000","last_change":"2023-07-06T03:53:26.767550+0000","last_active":"2023-07-06T03:53:27.986146+0000","last_peered":"2023-07-06T03:53:27.986146+0000","last_clean":"2023-07-06T03:53:27.986146+0000","last_became_active":"2023-07-06T03:53:26.766905+0000","last_became_peered":"2023-07-06T03:53:26.766905+0000","last_unstale":"2023-07-06T03:53:27.986146+0000","last_undegraded":"2023-07-06T03:53:27.986146+0000","last_fullsized":"2023-07-06T03:53:27.986146+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:18:09.726375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"98'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984940+0000","last_change":"2023-07-06T03:53:25.765786+0000","last_active":"2023-07-06T03:53:27.984940+0000","last_peered":"2023-07-06T03:53:27.984940+0000","last_clean":"2023-07-06T03:53:27.984940+0000","last_became_active":"2023-07-06T03:53:25.764804+0000","last_became_peered":"2023-07-06T03:53:25.764804+0000","last_unstale":"2023-07-06T03:53:27.984940+0000","last_undegraded":"2023-07-06T03:53:27.984940+0000","last_fullsized":"2023-07-06T03:53:27.984940+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:32:42.638594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898694+0000","last_change":"2023-07-06T03:53:26.775778+0000","last_active":"2023-07-06T03:53:27.898694+0000","last_peered":"2023-07-06T03:53:27.898694+0000","last_clean":"2023-07-06T03:53:27.898694+0000","last_became_active":"2023-07-06T03:53:26.775531+0000","last_became_peered":"2023-07-06T03:53:26.775531+0000","last_unstale":"2023-07-06T03:53:27.898694+0000","last_undegraded":"2023-07-06T03:53:27.898694+0000","last_fullsized":"2023-07-06T03:53:27.898694+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:29:20.204759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.899017+0000","last_change":"2023-07-06T03:53:25.761839+0000","last_active":"2023-07-06T03:53:27.899017+0000","last_peered":"2023-07-06T03:53:27.899017+0000","last_clean":"2023-07-06T03:53:27.899017+0000","last_became_active":"2023-07-06T03:53:25.761316+0000","last_became_peered":"2023-07-06T03:53:25.761316+0000","last_unstale":"2023-07-06T03:53:27.899017+0000","last_undegraded":"2023-07-06T03:53:27.899017+0000","last_fullsized":"2023-07-06T03:53:27.899017+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:12:28.876525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764521+0000","last_change":"2023-07-06T03:53:26.772652+0000","last_active":"2023-07-06T03:53:27.764521+0000","last_peered":"2023-07-06T03:53:27.764521+0000","last_clean":"2023-07-06T03:53:27.764521+0000","last_became_active":"2023-07-06T03:53:26.767214+0000","last_became_peered":"2023-07-06T03:53:26.767214+0000","last_unstale":"2023-07-06T03:53:27.764521+0000","last_undegraded":"2023-07-06T03:53:27.764521+0000","last_fullsized":"2023-07-06T03:53:27.764521+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:15:03.164217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.986187+0000","last_change":"2023-07-06T03:53:25.763450+0000","last_active":"2023-07-06T03:53:27.986187+0000","last_peered":"2023-07-06T03:53:27.986187+0000","last_clean":"2023-07-06T03:53:27.986187+0000","last_became_active":"2023-07-06T03:53:25.763153+0000","last_became_peered":"2023-07-06T03:53:25.763153+0000","last_unstale":"2023-07-06T03:53:27.986187+0000","last_undegraded":"2023-07-06T03:53:27.986187+0000","last_fullsized":"2023-07-06T03:53:27.986187+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:59:38.714520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985751+0000","last_change":"2023-07-06T03:53:26.774591+0000","last_active":"2023-07-06T03:53:27.985751+0000","last_peered":"2023-07-06T03:53:27.985751+0000","last_clean":"2023-07-06T03:53:27.985751+0000","last_became_active":"2023-07-06T03:53:26.774245+0000","last_became_peered":"2023-07-06T03:53:26.774245+0000","last_unstale":"2023-07-06T03:53:27.985751+0000","last_undegraded":"2023-07-06T03:53:27.985751+0000","last_fullsized":"2023-07-06T03:53:27.985751+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:17.131280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"98'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764783+0000","last_change":"2023-07-06T03:53:25.760397+0000","last_active":"2023-07-06T03:53:27.764783+0000","last_peered":"2023-07-06T03:53:27.764783+0000","last_clean":"2023-07-06T03:53:27.764783+0000","last_became_active":"2023-07-06T03:53:25.760151+0000","last_became_peered":"2023-07-06T03:53:25.760151+0000","last_unstale":"2023-07-06T03:53:27.764783+0000","last_undegraded":"2023-07-06T03:53:27.764783+0000","last_fullsized":"2023-07-06T03:53:27.764783+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:52:42.874715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985038+0000","last_change":"2023-07-06T03:53:26.774771+0000","last_active":"2023-07-06T03:53:27.985038+0000","last_peered":"2023-07-06T03:53:27.985038+0000","last_clean":"2023-07-06T03:53:27.985038+0000","last_became_active":"2023-07-06T03:53:26.774459+0000","last_became_peered":"2023-07-06T03:53:26.774459+0000","last_unstale":"2023-07-06T03:53:27.985038+0000","last_undegraded":"2023-07-06T03:53:27.985038+0000","last_fullsized":"2023-07-06T03:53:27.985038+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:50:40.924032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"98'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984087+0000","last_change":"2023-07-06T03:53:25.758308+0000","last_active":"2023-07-06T03:53:27.984087+0000","last_peered":"2023-07-06T03:53:27.984087+0000","last_clean":"2023-07-06T03:53:27.984087+0000","last_became_active":"2023-07-06T03:53:25.758128+0000","last_became_peered":"2023-07-06T03:53:25.758128+0000","last_unstale":"2023-07-06T03:53:27.984087+0000","last_undegraded":"2023-07-06T03:53:27.984087+0000","last_fullsized":"2023-07-06T03:53:27.984087+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:41:41.289345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765134+0000","last_change":"2023-07-06T03:53:26.775444+0000","last_active":"2023-07-06T03:53:27.765134+0000","last_peered":"2023-07-06T03:53:27.765134+0000","last_clean":"2023-07-06T03:53:27.765134+0000","last_became_active":"2023-07-06T03:53:26.775174+0000","last_became_peered":"2023-07-06T03:53:26.775174+0000","last_unstale":"2023-07-06T03:53:27.765134+0000","last_undegraded":"2023-07-06T03:53:27.765134+0000","last_fullsized":"2023-07-06T03:53:27.765134+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:05:43.296706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.983889+0000","last_change":"2023-07-06T03:53:25.757461+0000","last_active":"2023-07-06T03:53:27.983889+0000","last_peered":"2023-07-06T03:53:27.983889+0000","last_clean":"2023-07-06T03:53:27.983889+0000","last_became_active":"2023-07-06T03:53:25.757185+0000","last_became_peered":"2023-07-06T03:53:25.757185+0000","last_unstale":"2023-07-06T03:53:27.983889+0000","last_undegraded":"2023-07-06T03:53:27.983889+0000","last_fullsized":"2023-07-06T03:53:27.983889+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:44:53.941233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.986009+0000","last_change":"2023-07-06T03:53:26.755601+0000","last_active":"2023-07-06T03:53:27.986009+0000","last_peered":"2023-07-06T03:53:27.986009+0000","last_clean":"2023-07-06T03:53:27.986009+0000","last_became_active":"2023-07-06T03:53:26.755320+0000","last_became_peered":"2023-07-06T03:53:26.755320+0000","last_unstale":"2023-07-06T03:53:27.986009+0000","last_undegraded":"2023-07-06T03:53:27.986009+0000","last_fullsized":"2023-07-06T03:53:27.986009+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:47:56.563331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766115+0000","last_change":"2023-07-06T03:53:25.764596+0000","last_active":"2023-07-06T03:53:27.766115+0000","last_peered":"2023-07-06T03:53:27.766115+0000","last_clean":"2023-07-06T03:53:27.766115+0000","last_became_active":"2023-07-06T03:53:25.764126+0000","last_became_peered":"2023-07-06T03:53:25.764126+0000","last_unstale":"2023-07-06T03:53:27.766115+0000","last_undegraded":"2023-07-06T03:53:27.766115+0000","last_fullsized":"2023-07-06T03:53:27.766115+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:10:08.489885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984619+0000","last_change":"2023-07-06T03:53:26.773822+0000","last_active":"2023-07-06T03:53:27.984619+0000","last_peered":"2023-07-06T03:53:27.984619+0000","last_clean":"2023-07-06T03:53:27.984619+0000","last_became_active":"2023-07-06T03:53:26.773420+0000","last_became_peered":"2023-07-06T03:53:26.773420+0000","last_unstale":"2023-07-06T03:53:27.984619+0000","last_undegraded":"2023-07-06T03:53:27.984619+0000","last_fullsized":"2023-07-06T03:53:27.984619+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:41:41.173551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765980+0000","last_change":"2023-07-06T03:53:25.763023+0000","last_active":"2023-07-06T03:53:27.765980+0000","last_peered":"2023-07-06T03:53:27.765980+0000","last_clean":"2023-07-06T03:53:27.765980+0000","last_became_active":"2023-07-06T03:53:25.762691+0000","last_became_peered":"2023-07-06T03:53:25.762691+0000","last_unstale":"2023-07-06T03:53:27.765980+0000","last_undegraded":"2023-07-06T03:53:27.765980+0000","last_fullsized":"2023-07-06T03:53:27.765980+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:00:21.226381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985054+0000","last_change":"2023-07-06T03:53:26.762058+0000","last_active":"2023-07-06T03:53:27.985054+0000","last_peered":"2023-07-06T03:53:27.985054+0000","last_clean":"2023-07-06T03:53:27.985054+0000","last_became_active":"2023-07-06T03:53:26.761785+0000","last_became_peered":"2023-07-06T03:53:26.761785+0000","last_unstale":"2023-07-06T03:53:27.985054+0000","last_undegraded":"2023-07-06T03:53:27.985054+0000","last_fullsized":"2023-07-06T03:53:27.985054+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:11:51.981810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985657+0000","last_change":"2023-07-06T03:53:25.761618+0000","last_active":"2023-07-06T03:53:27.985657+0000","last_peered":"2023-07-06T03:53:27.985657+0000","last_clean":"2023-07-06T03:53:27.985657+0000","last_became_active":"2023-07-06T03:53:25.761270+0000","last_became_peered":"2023-07-06T03:53:25.761270+0000","last_unstale":"2023-07-06T03:53:27.985657+0000","last_undegraded":"2023-07-06T03:53:27.985657+0000","last_fullsized":"2023-07-06T03:53:27.985657+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:36:03.806749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985781+0000","last_change":"2023-07-06T03:53:26.766451+0000","last_active":"2023-07-06T03:53:27.985781+0000","last_peered":"2023-07-06T03:53:27.985781+0000","last_clean":"2023-07-06T03:53:27.985781+0000","last_became_active":"2023-07-06T03:53:26.766172+0000","last_became_peered":"2023-07-06T03:53:26.766172+0000","last_unstale":"2023-07-06T03:53:27.985781+0000","last_undegraded":"2023-07-06T03:53:27.985781+0000","last_fullsized":"2023-07-06T03:53:27.985781+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:49:28.832109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"98'3","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765657+0000","last_change":"2023-07-06T03:53:25.764734+0000","last_active":"2023-07-06T03:53:27.765657+0000","last_peered":"2023-07-06T03:53:27.765657+0000","last_clean":"2023-07-06T03:53:27.765657+0000","last_became_active":"2023-07-06T03:53:25.764430+0000","last_became_peered":"2023-07-06T03:53:25.764430+0000","last_unstale":"2023-07-06T03:53:27.765657+0000","last_undegraded":"2023-07-06T03:53:27.765657+0000","last_fullsized":"2023-07-06T03:53:27.765657+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:20:39.109310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765941+0000","last_change":"2023-07-06T03:53:26.768657+0000","last_active":"2023-07-06T03:53:27.765941+0000","last_peered":"2023-07-06T03:53:27.765941+0000","last_clean":"2023-07-06T03:53:27.765941+0000","last_became_active":"2023-07-06T03:53:26.768246+0000","last_became_peered":"2023-07-06T03:53:26.768246+0000","last_unstale":"2023-07-06T03:53:27.765941+0000","last_undegraded":"2023-07-06T03:53:27.765941+0000","last_fullsized":"2023-07-06T03:53:27.765941+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:41:54.520076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"98'6","reported_seq":19,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764785+0000","last_change":"2023-07-06T03:53:25.761999+0000","last_active":"2023-07-06T03:53:27.764785+0000","last_peered":"2023-07-06T03:53:27.764785+0000","last_clean":"2023-07-06T03:53:27.764785+0000","last_became_active":"2023-07-06T03:53:25.761778+0000","last_became_peered":"2023-07-06T03:53:25.761778+0000","last_unstale":"2023-07-06T03:53:27.764785+0000","last_undegraded":"2023-07-06T03:53:27.764785+0000","last_fullsized":"2023-07-06T03:53:27.764785+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:27:05.659397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898985+0000","last_change":"2023-07-06T03:53:26.771608+0000","last_active":"2023-07-06T03:53:27.898985+0000","last_peered":"2023-07-06T03:53:27.898985+0000","last_clean":"2023-07-06T03:53:27.898985+0000","last_became_active":"2023-07-06T03:53:26.771346+0000","last_became_peered":"2023-07-06T03:53:26.771346+0000","last_unstale":"2023-07-06T03:53:27.898985+0000","last_undegraded":"2023-07-06T03:53:27.898985+0000","last_fullsized":"2023-07-06T03:53:27.898985+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:42:58.642834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985528+0000","last_change":"2023-07-06T03:53:25.760521+0000","last_active":"2023-07-06T03:53:27.985528+0000","last_peered":"2023-07-06T03:53:27.985528+0000","last_clean":"2023-07-06T03:53:27.985528+0000","last_became_active":"2023-07-06T03:53:25.760210+0000","last_became_peered":"2023-07-06T03:53:25.760210+0000","last_unstale":"2023-07-06T03:53:27.985528+0000","last_undegraded":"2023-07-06T03:53:27.985528+0000","last_fullsized":"2023-07-06T03:53:27.985528+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:15:24.380475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.899008+0000","last_change":"2023-07-06T03:53:26.775942+0000","last_active":"2023-07-06T03:53:27.899008+0000","last_peered":"2023-07-06T03:53:27.899008+0000","last_clean":"2023-07-06T03:53:27.899008+0000","last_became_active":"2023-07-06T03:53:26.775637+0000","last_became_peered":"2023-07-06T03:53:26.775637+0000","last_unstale":"2023-07-06T03:53:27.899008+0000","last_undegraded":"2023-07-06T03:53:27.899008+0000","last_fullsized":"2023-07-06T03:53:27.899008+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:13:13.552629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985925+0000","last_change":"2023-07-06T03:53:25.761696+0000","last_active":"2023-07-06T03:53:27.985925+0000","last_peered":"2023-07-06T03:53:27.985925+0000","last_clean":"2023-07-06T03:53:27.985925+0000","last_became_active":"2023-07-06T03:53:25.761373+0000","last_became_peered":"2023-07-06T03:53:25.761373+0000","last_unstale":"2023-07-06T03:53:27.985925+0000","last_undegraded":"2023-07-06T03:53:27.985925+0000","last_fullsized":"2023-07-06T03:53:27.985925+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:59:10.019933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766171+0000","last_change":"2023-07-06T03:53:26.771653+0000","last_active":"2023-07-06T03:53:27.766171+0000","last_peered":"2023-07-06T03:53:27.766171+0000","last_clean":"2023-07-06T03:53:27.766171+0000","last_became_active":"2023-07-06T03:53:26.771323+0000","last_became_peered":"2023-07-06T03:53:26.771323+0000","last_unstale":"2023-07-06T03:53:27.766171+0000","last_undegraded":"2023-07-06T03:53:27.766171+0000","last_fullsized":"2023-07-06T03:53:27.766171+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:12:58.253299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985239+0000","last_change":"2023-07-06T03:53:25.763182+0000","last_active":"2023-07-06T03:53:27.985239+0000","last_peered":"2023-07-06T03:53:27.985239+0000","last_clean":"2023-07-06T03:53:27.985239+0000","last_became_active":"2023-07-06T03:53:25.762812+0000","last_became_peered":"2023-07-06T03:53:25.762812+0000","last_unstale":"2023-07-06T03:53:27.985239+0000","last_undegraded":"2023-07-06T03:53:27.985239+0000","last_fullsized":"2023-07-06T03:53:27.985239+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:01:12.056695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985203+0000","last_change":"2023-07-06T03:53:26.781871+0000","last_active":"2023-07-06T03:53:27.985203+0000","last_peered":"2023-07-06T03:53:27.985203+0000","last_clean":"2023-07-06T03:53:27.985203+0000","last_became_active":"2023-07-06T03:53:26.781596+0000","last_became_peered":"2023-07-06T03:53:26.781596+0000","last_unstale":"2023-07-06T03:53:27.985203+0000","last_undegraded":"2023-07-06T03:53:27.985203+0000","last_fullsized":"2023-07-06T03:53:27.985203+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:46.741875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"98'8","reported_seq":21,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764621+0000","last_change":"2023-07-06T03:53:25.761393+0000","last_active":"2023-07-06T03:53:27.764621+0000","last_peered":"2023-07-06T03:53:27.764621+0000","last_clean":"2023-07-06T03:53:27.764621+0000","last_became_active":"2023-07-06T03:53:25.761142+0000","last_became_peered":"2023-07-06T03:53:25.761142+0000","last_unstale":"2023-07-06T03:53:27.764621+0000","last_undegraded":"2023-07-06T03:53:27.764621+0000","last_fullsized":"2023-07-06T03:53:27.764621+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:13:01.669680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765967+0000","last_change":"2023-07-06T03:53:26.769578+0000","last_active":"2023-07-06T03:53:27.765967+0000","last_peered":"2023-07-06T03:53:27.765967+0000","last_clean":"2023-07-06T03:53:27.765967+0000","last_became_active":"2023-07-06T03:53:26.768848+0000","last_became_peered":"2023-07-06T03:53:26.768848+0000","last_unstale":"2023-07-06T03:53:27.765967+0000","last_undegraded":"2023-07-06T03:53:27.765967+0000","last_fullsized":"2023-07-06T03:53:27.765967+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:25.503614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985365+0000","last_change":"2023-07-06T03:53:25.762326+0000","last_active":"2023-07-06T03:53:27.985365+0000","last_peered":"2023-07-06T03:53:27.985365+0000","last_clean":"2023-07-06T03:53:27.985365+0000","last_became_active":"2023-07-06T03:53:25.762045+0000","last_became_peered":"2023-07-06T03:53:25.762045+0000","last_unstale":"2023-07-06T03:53:27.985365+0000","last_undegraded":"2023-07-06T03:53:27.985365+0000","last_fullsized":"2023-07-06T03:53:27.985365+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:34:43.595364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764446+0000","last_change":"2023-07-06T03:53:26.754853+0000","last_active":"2023-07-06T03:53:27.764446+0000","last_peered":"2023-07-06T03:53:27.764446+0000","last_clean":"2023-07-06T03:53:27.764446+0000","last_became_active":"2023-07-06T03:53:26.754574+0000","last_became_peered":"2023-07-06T03:53:26.754574+0000","last_unstale":"2023-07-06T03:53:27.764446+0000","last_undegraded":"2023-07-06T03:53:27.764446+0000","last_fullsized":"2023-07-06T03:53:27.764446+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:10:51.643805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"99'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985130+0000","last_change":"2023-07-06T03:53:25.766271+0000","last_active":"2023-07-06T03:53:27.985130+0000","last_peered":"2023-07-06T03:53:27.985130+0000","last_clean":"2023-07-06T03:53:27.985130+0000","last_became_active":"2023-07-06T03:53:25.765967+0000","last_became_peered":"2023-07-06T03:53:25.765967+0000","last_unstale":"2023-07-06T03:53:27.985130+0000","last_undegraded":"2023-07-06T03:53:27.985130+0000","last_fullsized":"2023-07-06T03:53:27.985130+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:08:38.037063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985595+0000","last_change":"2023-07-06T03:53:26.779463+0000","last_active":"2023-07-06T03:53:27.985595+0000","last_peered":"2023-07-06T03:53:27.985595+0000","last_clean":"2023-07-06T03:53:27.985595+0000","last_became_active":"2023-07-06T03:53:26.779147+0000","last_became_peered":"2023-07-06T03:53:26.779147+0000","last_unstale":"2023-07-06T03:53:27.985595+0000","last_undegraded":"2023-07-06T03:53:27.985595+0000","last_fullsized":"2023-07-06T03:53:27.985595+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:49:17.417072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765121+0000","last_change":"2023-07-06T03:53:25.761337+0000","last_active":"2023-07-06T03:53:27.765121+0000","last_peered":"2023-07-06T03:53:27.765121+0000","last_clean":"2023-07-06T03:53:27.765121+0000","last_became_active":"2023-07-06T03:53:25.761078+0000","last_became_peered":"2023-07-06T03:53:25.761078+0000","last_unstale":"2023-07-06T03:53:27.765121+0000","last_undegraded":"2023-07-06T03:53:27.765121+0000","last_fullsized":"2023-07-06T03:53:27.765121+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:06.279786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984101+0000","last_change":"2023-07-06T03:53:26.775752+0000","last_active":"2023-07-06T03:53:27.984101+0000","last_peered":"2023-07-06T03:53:27.984101+0000","last_clean":"2023-07-06T03:53:27.984101+0000","last_became_active":"2023-07-06T03:53:26.773382+0000","last_became_peered":"2023-07-06T03:53:26.773382+0000","last_unstale":"2023-07-06T03:53:27.984101+0000","last_undegraded":"2023-07-06T03:53:27.984101+0000","last_fullsized":"2023-07-06T03:53:27.984101+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:44:27.300515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985918+0000","last_change":"2023-07-06T03:53:25.760855+0000","last_active":"2023-07-06T03:53:27.985918+0000","last_peered":"2023-07-06T03:53:27.985918+0000","last_clean":"2023-07-06T03:53:27.985918+0000","last_became_active":"2023-07-06T03:53:25.760495+0000","last_became_peered":"2023-07-06T03:53:25.760495+0000","last_unstale":"2023-07-06T03:53:27.985918+0000","last_undegraded":"2023-07-06T03:53:27.985918+0000","last_fullsized":"2023-07-06T03:53:27.985918+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:04:09.404201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985605+0000","last_change":"2023-07-06T03:53:26.761781+0000","last_active":"2023-07-06T03:53:27.985605+0000","last_peered":"2023-07-06T03:53:27.985605+0000","last_clean":"2023-07-06T03:53:27.985605+0000","last_became_active":"2023-07-06T03:53:26.761512+0000","last_became_peered":"2023-07-06T03:53:26.761512+0000","last_unstale":"2023-07-06T03:53:27.985605+0000","last_undegraded":"2023-07-06T03:53:27.985605+0000","last_fullsized":"2023-07-06T03:53:27.985605+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:19:36.149238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764633+0000","last_change":"2023-07-06T03:53:25.765885+0000","last_active":"2023-07-06T03:53:27.764633+0000","last_peered":"2023-07-06T03:53:27.764633+0000","last_clean":"2023-07-06T03:53:27.764633+0000","last_became_active":"2023-07-06T03:53:25.765567+0000","last_became_peered":"2023-07-06T03:53:25.765567+0000","last_unstale":"2023-07-06T03:53:27.764633+0000","last_undegraded":"2023-07-06T03:53:27.764633+0000","last_fullsized":"2023-07-06T03:53:27.764633+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:48:34.979836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984981+0000","last_change":"2023-07-06T03:53:26.762908+0000","last_active":"2023-07-06T03:53:27.984981+0000","last_peered":"2023-07-06T03:53:27.984981+0000","last_clean":"2023-07-06T03:53:27.984981+0000","last_became_active":"2023-07-06T03:53:26.762531+0000","last_became_peered":"2023-07-06T03:53:26.762531+0000","last_unstale":"2023-07-06T03:53:27.984981+0000","last_undegraded":"2023-07-06T03:53:27.984981+0000","last_fullsized":"2023-07-06T03:53:27.984981+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:05:41.719802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984775+0000","last_change":"2023-07-06T03:53:25.766585+0000","last_active":"2023-07-06T03:53:27.984775+0000","last_peered":"2023-07-06T03:53:27.984775+0000","last_clean":"2023-07-06T03:53:27.984775+0000","last_became_active":"2023-07-06T03:53:25.765934+0000","last_became_peered":"2023-07-06T03:53:25.765934+0000","last_unstale":"2023-07-06T03:53:27.984775+0000","last_undegraded":"2023-07-06T03:53:27.984775+0000","last_fullsized":"2023-07-06T03:53:27.984775+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:29:26.930120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765731+0000","last_change":"2023-07-06T03:53:26.767035+0000","last_active":"2023-07-06T03:53:27.765731+0000","last_peered":"2023-07-06T03:53:27.765731+0000","last_clean":"2023-07-06T03:53:27.765731+0000","last_became_active":"2023-07-06T03:53:26.766782+0000","last_became_peered":"2023-07-06T03:53:26.766782+0000","last_unstale":"2023-07-06T03:53:27.765731+0000","last_undegraded":"2023-07-06T03:53:27.765731+0000","last_fullsized":"2023-07-06T03:53:27.765731+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:54.804766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.983952+0000","last_change":"2023-07-06T03:53:25.762440+0000","last_active":"2023-07-06T03:53:27.983952+0000","last_peered":"2023-07-06T03:53:27.983952+0000","last_clean":"2023-07-06T03:53:27.983952+0000","last_became_active":"2023-07-06T03:53:25.762126+0000","last_became_peered":"2023-07-06T03:53:25.762126+0000","last_unstale":"2023-07-06T03:53:27.983952+0000","last_undegraded":"2023-07-06T03:53:27.983952+0000","last_fullsized":"2023-07-06T03:53:27.983952+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:40:07.152494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.898887+0000","last_change":"2023-07-06T03:53:26.783048+0000","last_active":"2023-07-06T03:53:27.898887+0000","last_peered":"2023-07-06T03:53:27.898887+0000","last_clean":"2023-07-06T03:53:27.898887+0000","last_became_active":"2023-07-06T03:53:26.782676+0000","last_became_peered":"2023-07-06T03:53:26.782676+0000","last_unstale":"2023-07-06T03:53:27.898887+0000","last_undegraded":"2023-07-06T03:53:27.898887+0000","last_fullsized":"2023-07-06T03:53:27.898887+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:37:59.312364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984453+0000","last_change":"2023-07-06T03:53:25.765395+0000","last_active":"2023-07-06T03:53:27.984453+0000","last_peered":"2023-07-06T03:53:27.984453+0000","last_clean":"2023-07-06T03:53:27.984453+0000","last_became_active":"2023-07-06T03:53:25.764440+0000","last_became_peered":"2023-07-06T03:53:25.764440+0000","last_unstale":"2023-07-06T03:53:27.984453+0000","last_undegraded":"2023-07-06T03:53:27.984453+0000","last_fullsized":"2023-07-06T03:53:27.984453+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:57:14.835518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766090+0000","last_change":"2023-07-06T03:53:26.768459+0000","last_active":"2023-07-06T03:53:27.766090+0000","last_peered":"2023-07-06T03:53:27.766090+0000","last_clean":"2023-07-06T03:53:27.766090+0000","last_became_active":"2023-07-06T03:53:26.768132+0000","last_became_peered":"2023-07-06T03:53:26.768132+0000","last_unstale":"2023-07-06T03:53:27.766090+0000","last_undegraded":"2023-07-06T03:53:27.766090+0000","last_fullsized":"2023-07-06T03:53:27.766090+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:49:38.338342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765344+0000","last_change":"2023-07-06T03:53:25.769061+0000","last_active":"2023-07-06T03:53:27.765344+0000","last_peered":"2023-07-06T03:53:27.765344+0000","last_clean":"2023-07-06T03:53:27.765344+0000","last_became_active":"2023-07-06T03:53:25.768721+0000","last_became_peered":"2023-07-06T03:53:25.768721+0000","last_unstale":"2023-07-06T03:53:27.765344+0000","last_undegraded":"2023-07-06T03:53:27.765344+0000","last_fullsized":"2023-07-06T03:53:27.765344+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:36:12.901646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":155,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985716+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:53:27.985716+0000","last_peered":"2023-07-06T03:53:27.985716+0000","last_clean":"2023-07-06T03:53:27.985716+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:53:27.985716+0000","last_undegraded":"2023-07-06T03:53:27.985716+0000","last_fullsized":"2023-07-06T03:53:27.985716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:46:05.213430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985606+0000","last_change":"2023-07-06T03:53:26.772280+0000","last_active":"2023-07-06T03:53:27.985606+0000","last_peered":"2023-07-06T03:53:27.985606+0000","last_clean":"2023-07-06T03:53:27.985606+0000","last_became_active":"2023-07-06T03:53:26.771964+0000","last_became_peered":"2023-07-06T03:53:26.771964+0000","last_unstale":"2023-07-06T03:53:27.985606+0000","last_undegraded":"2023-07-06T03:53:27.985606+0000","last_fullsized":"2023-07-06T03:53:27.985606+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:51:57.479581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"98'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766081+0000","last_change":"2023-07-06T03:53:25.764949+0000","last_active":"2023-07-06T03:53:27.766081+0000","last_peered":"2023-07-06T03:53:27.766081+0000","last_clean":"2023-07-06T03:53:27.766081+0000","last_became_active":"2023-07-06T03:53:25.764699+0000","last_became_peered":"2023-07-06T03:53:25.764699+0000","last_unstale":"2023-07-06T03:53:27.766081+0000","last_undegraded":"2023-07-06T03:53:27.766081+0000","last_fullsized":"2023-07-06T03:53:27.766081+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:33:16.539361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":161,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764846+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:53:27.764846+0000","last_peered":"2023-07-06T03:53:27.764846+0000","last_clean":"2023-07-06T03:53:27.764846+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:53:27.764846+0000","last_undegraded":"2023-07-06T03:53:27.764846+0000","last_fullsized":"2023-07-06T03:53:27.764846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:59:20.018968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766063+0000","last_change":"2023-07-06T03:53:26.771097+0000","last_active":"2023-07-06T03:53:27.766063+0000","last_peered":"2023-07-06T03:53:27.766063+0000","last_clean":"2023-07-06T03:53:27.766063+0000","last_became_active":"2023-07-06T03:53:26.770713+0000","last_became_peered":"2023-07-06T03:53:26.770713+0000","last_unstale":"2023-07-06T03:53:27.766063+0000","last_undegraded":"2023-07-06T03:53:27.766063+0000","last_fullsized":"2023-07-06T03:53:27.766063+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:06:44.273593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765577+0000","last_change":"2023-07-06T03:53:25.763534+0000","last_active":"2023-07-06T03:53:27.765577+0000","last_peered":"2023-07-06T03:53:27.765577+0000","last_clean":"2023-07-06T03:53:27.765577+0000","last_became_active":"2023-07-06T03:53:25.762916+0000","last_became_peered":"2023-07-06T03:53:25.762916+0000","last_unstale":"2023-07-06T03:53:27.765577+0000","last_undegraded":"2023-07-06T03:53:27.765577+0000","last_fullsized":"2023-07-06T03:53:27.765577+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:45:54.953300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765932+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:53:27.765932+0000","last_peered":"2023-07-06T03:53:27.765932+0000","last_clean":"2023-07-06T03:53:27.765932+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:53:27.765932+0000","last_undegraded":"2023-07-06T03:53:27.765932+0000","last_fullsized":"2023-07-06T03:53:27.765932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:52:37.364641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765197+0000","last_change":"2023-07-06T03:53:26.769630+0000","last_active":"2023-07-06T03:53:27.765197+0000","last_peered":"2023-07-06T03:53:27.765197+0000","last_clean":"2023-07-06T03:53:27.765197+0000","last_became_active":"2023-07-06T03:53:26.769319+0000","last_became_peered":"2023-07-06T03:53:26.769319+0000","last_unstale":"2023-07-06T03:53:27.765197+0000","last_undegraded":"2023-07-06T03:53:27.765197+0000","last_fullsized":"2023-07-06T03:53:27.765197+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:06:49.760154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984831+0000","last_change":"2023-07-06T03:53:25.763463+0000","last_active":"2023-07-06T03:53:27.984831+0000","last_peered":"2023-07-06T03:53:27.984831+0000","last_clean":"2023-07-06T03:53:27.984831+0000","last_became_active":"2023-07-06T03:53:25.763132+0000","last_became_peered":"2023-07-06T03:53:25.763132+0000","last_unstale":"2023-07-06T03:53:27.984831+0000","last_undegraded":"2023-07-06T03:53:27.984831+0000","last_fullsized":"2023-07-06T03:53:27.984831+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:45:54.212957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":161,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764950+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:53:27.764950+0000","last_peered":"2023-07-06T03:53:27.764950+0000","last_clean":"2023-07-06T03:53:27.764950+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:53:27.764950+0000","last_undegraded":"2023-07-06T03:53:27.764950+0000","last_fullsized":"2023-07-06T03:53:27.764950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:34:40.976673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":305,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984031+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:53:27.984031+0000","last_peered":"2023-07-06T03:53:27.984031+0000","last_clean":"2023-07-06T03:53:27.984031+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:53:27.984031+0000","last_undegraded":"2023-07-06T03:53:27.984031+0000","last_fullsized":"2023-07-06T03:53:27.984031+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:19.715411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.983846+0000","last_change":"2023-07-06T03:53:26.767875+0000","last_active":"2023-07-06T03:53:27.983846+0000","last_peered":"2023-07-06T03:53:27.983846+0000","last_clean":"2023-07-06T03:53:27.983846+0000","last_became_active":"2023-07-06T03:53:26.767555+0000","last_became_peered":"2023-07-06T03:53:26.767555+0000","last_unstale":"2023-07-06T03:53:27.983846+0000","last_undegraded":"2023-07-06T03:53:27.983846+0000","last_fullsized":"2023-07-06T03:53:27.983846+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:23:50.020063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"98'1","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984134+0000","last_change":"2023-07-06T03:53:25.758578+0000","last_active":"2023-07-06T03:53:27.984134+0000","last_peered":"2023-07-06T03:53:27.984134+0000","last_clean":"2023-07-06T03:53:27.984134+0000","last_became_active":"2023-07-06T03:53:25.758327+0000","last_became_peered":"2023-07-06T03:53:25.758327+0000","last_unstale":"2023-07-06T03:53:27.984134+0000","last_undegraded":"2023-07-06T03:53:27.984134+0000","last_fullsized":"2023-07-06T03:53:27.984134+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:41:40.170906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":161,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.764464+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:53:27.764464+0000","last_peered":"2023-07-06T03:53:27.764464+0000","last_clean":"2023-07-06T03:53:27.764464+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:53:27.764464+0000","last_undegraded":"2023-07-06T03:53:27.764464+0000","last_fullsized":"2023-07-06T03:53:27.764464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:35:08.792353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985145+0000","last_change":"2023-07-06T03:53:26.760601+0000","last_active":"2023-07-06T03:53:27.985145+0000","last_peered":"2023-07-06T03:53:27.985145+0000","last_clean":"2023-07-06T03:53:27.985145+0000","last_became_active":"2023-07-06T03:53:26.760320+0000","last_became_peered":"2023-07-06T03:53:26.760320+0000","last_unstale":"2023-07-06T03:53:27.985145+0000","last_undegraded":"2023-07-06T03:53:27.985145+0000","last_fullsized":"2023-07-06T03:53:27.985145+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:37:41.364027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985751+0000","last_change":"2023-07-06T03:53:25.763925+0000","last_active":"2023-07-06T03:53:27.985751+0000","last_peered":"2023-07-06T03:53:27.985751+0000","last_clean":"2023-07-06T03:53:27.985751+0000","last_became_active":"2023-07-06T03:53:25.763655+0000","last_became_peered":"2023-07-06T03:53:25.763655+0000","last_unstale":"2023-07-06T03:53:27.985751+0000","last_undegraded":"2023-07-06T03:53:27.985751+0000","last_fullsized":"2023-07-06T03:53:27.985751+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:20:49.520910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":161,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765235+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:53:27.765235+0000","last_peered":"2023-07-06T03:53:27.765235+0000","last_clean":"2023-07-06T03:53:27.765235+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:53:27.765235+0000","last_undegraded":"2023-07-06T03:53:27.765235+0000","last_fullsized":"2023-07-06T03:53:27.765235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:07:23.923774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985907+0000","last_change":"2023-07-06T03:53:26.755755+0000","last_active":"2023-07-06T03:53:27.985907+0000","last_peered":"2023-07-06T03:53:27.985907+0000","last_clean":"2023-07-06T03:53:27.985907+0000","last_became_active":"2023-07-06T03:53:26.755429+0000","last_became_peered":"2023-07-06T03:53:26.755429+0000","last_unstale":"2023-07-06T03:53:27.985907+0000","last_undegraded":"2023-07-06T03:53:27.985907+0000","last_fullsized":"2023-07-06T03:53:27.985907+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:06:32.140996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"98'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.986169+0000","last_change":"2023-07-06T03:53:25.765118+0000","last_active":"2023-07-06T03:53:27.986169+0000","last_peered":"2023-07-06T03:53:27.986169+0000","last_clean":"2023-07-06T03:53:27.986169+0000","last_became_active":"2023-07-06T03:53:25.764820+0000","last_became_peered":"2023-07-06T03:53:25.764820+0000","last_unstale":"2023-07-06T03:53:27.986169+0000","last_undegraded":"2023-07-06T03:53:27.986169+0000","last_fullsized":"2023-07-06T03:53:27.986169+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:18:37.051743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":154,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.984979+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:53:27.984979+0000","last_peered":"2023-07-06T03:53:27.984979+0000","last_clean":"2023-07-06T03:53:27.984979+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:53:27.984979+0000","last_undegraded":"2023-07-06T03:53:27.984979+0000","last_fullsized":"2023-07-06T03:53:27.984979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:50:33.095193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.766291+0000","last_change":"2023-07-06T03:53:26.760661+0000","last_active":"2023-07-06T03:53:27.766291+0000","last_peered":"2023-07-06T03:53:27.766291+0000","last_clean":"2023-07-06T03:53:27.766291+0000","last_became_active":"2023-07-06T03:53:26.760245+0000","last_became_peered":"2023-07-06T03:53:26.760245+0000","last_unstale":"2023-07-06T03:53:27.766291+0000","last_undegraded":"2023-07-06T03:53:27.766291+0000","last_fullsized":"2023-07-06T03:53:27.766291+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:25.737459+0000","last_clean_scrub_stamp":"2023-07-06T03:53:25.737459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:08:49.844088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.985500+0000","last_change":"2023-07-06T03:53:25.766728+0000","last_active":"2023-07-06T03:53:27.985500+0000","last_peered":"2023-07-06T03:53:27.985500+0000","last_clean":"2023-07-06T03:53:27.985500+0000","last_became_active":"2023-07-06T03:53:25.766327+0000","last_became_peered":"2023-07-06T03:53:25.766327+0000","last_unstale":"2023-07-06T03:53:27.985500+0000","last_undegraded":"2023-07-06T03:53:27.985500+0000","last_fullsized":"2023-07-06T03:53:27.985500+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:53:24.734814+0000","last_clean_scrub_stamp":"2023-07-06T03:53:24.734814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:25.296456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":160,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-07-06T03:53:27.765327+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:53:27.765327+0000","last_peered":"2023-07-06T03:53:27.765327+0000","last_clean":"2023-07-06T03:53:27.765327+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:53:27.765327+0000","last_undegraded":"2023-07-06T03:53:27.765327+0000","last_fullsized":"2023-07-06T03:53:27.765327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:37.190611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574954,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36348,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335492,"statfs":{"total":96636764160,"available":96599543808,"internally_reserved":0,"allocated":1232896,"data_stored":559544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.45900000000000002,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":1.024,"5min":4.0570000000000004,"15min":4.0570000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67900000000000005,"15min":0.76600000000000001},"min":{"1min":0.51200000000000001,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.89700000000000002,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67800000000000005,"15min":0.72399999999999998},"min":{"1min":0.47199999999999998,"5min":0.45000000000000001,"15min":0.34000000000000002},"max":{"1min":1.091,"5min":1.091,"15min":2.6219999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70499999999999996,"15min":0.84599999999999997},"min":{"1min":0.47099999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94999999999999996,"5min":4.0389999999999997,"15min":6.6310000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.39700000000000002,"15min":0.35199999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":3.77},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.45000000000000001},"max":{"1min":0.93000000000000005,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.753}]},{"osd":3,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.48699999999999999,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75,"15min":0.753},"min":{"1min":0.57599999999999996,"5min":0.45400000000000001,"15min":0.43099999999999999},"max":{"1min":0.92700000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.89700000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.624},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.65800000000000003,"15min":0.70499999999999996},"min":{"1min":0.44600000000000001,"5min":0.38,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":0.95999999999999996,"15min":1.4590000000000001},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.69499999999999995},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0820000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82599999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72499999999999998,"15min":0.73999999999999999},"min":{"1min":0.46400000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.90700000000000003,"5min":1.03,"15min":1.5349999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.51800000000000002,"5min":0.45200000000000001,"15min":0.42299999999999999},"max":{"1min":0.98199999999999998,"5min":1.006,"15min":1.3779999999999999},"last":0.84199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36352,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335488,"statfs":{"total":96636764160,"available":96599539712,"internally_reserved":0,"allocated":1236992,"data_stored":560456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70899999999999996,"15min":0.67200000000000004},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.29399999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77200000000000002,"15min":0.73999999999999999},"min":{"1min":0.44700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.2889999999999999,"5min":3.3399999999999999,"15min":3.3399999999999999},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86099999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":1.1759999999999999},"last":0.88100000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.74399999999999999,"15min":0.68200000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":1.9890000000000001,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.9450000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69999999999999996,"15min":0.745},"min":{"1min":0.502,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":2.1850000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.53400000000000003,"5min":0.40500000000000003,"15min":0.373},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66600000000000004,"15min":0.70699999999999996},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.33200000000000002},"max":{"1min":0.92100000000000004,"5min":1.0309999999999999,"15min":1.1339999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67300000000000004,"15min":0.72499999999999998},"min":{"1min":0.44700000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":0.98699999999999999,"15min":1.085},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.443,"15min":0.38600000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.218},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.77700000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":1.0489999999999999,"15min":1.198},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.80500000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97399999999999998,"5min":1.054,"15min":1.323},"last":0.749},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.81599999999999995},"min":{"1min":0.54700000000000004,"5min":0.504,"15min":0.41499999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.4139999999999999},"last":0.71099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1826816,"data_stored":1149352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.52400000000000002},"min":{"1min":0.26900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65100000000000002,"5min":1.002,"15min":2.8879999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.501,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.31900000000000001,"15min":0.313},"max":{"1min":0.58099999999999996,"5min":0.78300000000000003,"15min":1.96},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45500000000000002,"15min":0.83099999999999996},"min":{"1min":0.313,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.57799999999999996,"5min":0.70799999999999996,"15min":13.055999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54500000000000004,"15min":0.621},"min":{"1min":0.35699999999999998,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":2.8479999999999999},"last":0.435}]},{"osd":3,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48099999999999998,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.71599999999999997,"15min":1.2709999999999999},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.73999999999999999,"15min":1.101},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.86199999999999999,"5min":1.0640000000000001,"15min":12.989000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.78700000000000003,"15min":0.82599999999999996},"min":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.46000000000000002},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.502,"5min":0.432,"15min":0.432},"max":{"1min":0.91200000000000003,"5min":1.0860000000000001,"15min":1.3100000000000001},"last":0.501},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.71699999999999997,"15min":1.101},"min":{"1min":0.433,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.94499999999999995,"5min":1.1020000000000001,"15min":13.023999999999999},"last":0.625}]},{"osd":6,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":0.81000000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76700000000000002,"15min":1.147},"min":{"1min":0.54000000000000004,"5min":0.48699999999999999,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":1.0569999999999999,"15min":12.941000000000001},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75800000000000001,"15min":0.81999999999999995},"min":{"1min":0.58399999999999996,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.071,"5min":1.071,"15min":2.806},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.751,"15min":0.80300000000000005},"min":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.83999999999999997,"5min":1.1319999999999999,"15min":1.337},"last":0.58599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1228800,"data_stored":559474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.28000000000000003,"5min":0.26400000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.753,"15min":0.753},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.439},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.248},"max":{"1min":0.51200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.29599999999999999},"max":{"1min":0.52400000000000002,"5min":0.69199999999999995,"15min":0.73699999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.48399999999999999,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.314},"max":{"1min":0.55800000000000005,"5min":0.67200000000000004,"15min":1.2030000000000001},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.51400000000000001,"15min":0.48599999999999999},"min":{"1min":0.32200000000000001,"5min":0.30099999999999999,"15min":0.29899999999999999},"max":{"1min":0.56200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.497},{"interface":"front","average":{"1min":0.498,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.63400000000000001,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":1.1699999999999999,"15min":1.171},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":0.88800000000000001,"5min":1.032,"15min":1.034},"last":0.628}]},{"osd":5,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.745,"15min":0.73199999999999998},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":0.90600000000000003,"5min":1.1080000000000001,"15min":1.1419999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72899999999999998,"15min":0.747},"min":{"1min":0.56399999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93700000000000006,"5min":1.1040000000000001,"15min":1.1220000000000001},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.51600000000000001},"max":{"1min":0.93300000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73599999999999999,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81299999999999994,"5min":1.0649999999999999,"15min":1.546},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":0.86599999999999999,"5min":1.0820000000000001,"15min":1.6830000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77400000000000002,"15min":1.0269999999999999},"min":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":0.86199999999999999,"5min":1.0669999999999999,"15min":9.5960000000000001},"last":0.70299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40444,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331396,"statfs":{"total":96636764160,"available":96595349504,"internally_reserved":0,"allocated":1232896,"data_stored":559004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.47999999999999998,"15min":0.443},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":4.9199999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.371},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.50800000000000001,"15min":0.48199999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.093,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.55000000000000004,"15min":0.51100000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":6.5960000000000001,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.57099999999999995,"15min":0.52400000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.657,"5min":3.657,"15min":3.657},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.63200000000000001,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":6.6390000000000002,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.1629999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":0.94299999999999995,"15min":0.78100000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":12.007,"5min":12.007,"15min":12.007},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81899999999999995,"15min":0.73199999999999998},"min":{"1min":0.49299999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.2200000000000002,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.79300000000000004,"15min":0.749},"min":{"1min":0.495,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":4.8719999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":0.88300000000000001,"15min":0.79700000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.8440000000000003,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":0.94299999999999995,"15min":0.78600000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":11.696999999999999,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.4390000000000001,"5min":0.92400000000000004,"15min":0.80800000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":13.084,"5min":13.084,"15min":13.084},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:53:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.83299999999999996,"15min":0.83099999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.042,"5min":0.86499999999999999,"15min":0.81999999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":4.484,"5min":4.484,"15min":4.484},"last":0.68999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1236992,"data_stored":560456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.44900000000000001,"15min":0.58499999999999996},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.442},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45600000000000002,"15min":0.94899999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":15.911},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":1.0209999999999999},"min":{"1min":0.40300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.347,"5min":1.347,"15min":16.024000000000001},"last":0.51500000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.33600000000000002,"5min":0.32800000000000001,"15min":0.27400000000000002},"max":{"1min":3.25,"5min":3.25,"15min":3.25},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.54300000000000004,"15min":1.028},"min":{"1min":0.36399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73999999999999999,"15min":0.75600000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.90300000000000002},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.36699999999999999},"max":{"1min":1.724,"5min":1.724,"15min":5.7300000000000004},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75600000000000001,"15min":0.90500000000000003},"min":{"1min":0.49199999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.411,"5min":1.411,"15min":5.077},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78900000000000003,"15min":0.95699999999999996},"min":{"1min":0.57999999999999996,"5min":0.499,"15min":0.44800000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.66400000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76200000000000001,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.91400000000000003},"min":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.45600000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":5.157},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:53:18 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.82199999999999995},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76600000000000001,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.492,"5min":1.492,"15min":5.1189999999999998},"last":0.72099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36344,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335496,"statfs":{"total":96636764160,"available":96599547904,"internally_reserved":0,"allocated":1228800,"data_stored":558926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.41599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80700000000000005,"5min":1.784,"15min":1.784},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.38},"max":{"1min":0.81899999999999995,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66000000000000003,"15min":0.82799999999999996},"min":{"1min":0.39500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.84799999999999998,"5min":0.878,"15min":6.258},"last":0.748},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.68000000000000005,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":1.806,"15min":1.806},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.84599999999999997},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.29199999999999998},"max":{"1min":0.90100000000000002,"5min":0.91000000000000003,"15min":6.4740000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73799999999999999,"15min":0.92300000000000004},"min":{"1min":0.57699999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":6.4859999999999998},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.48499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.97799999999999998,"15min":1.25},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.71499999999999997},"min":{"1min":0.503,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":0.99099999999999999,"15min":1.413},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59999999999999998,"15min":0.65000000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77800000000000002,"5min":0.92100000000000004,"15min":1.3280000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.66000000000000003},"min":{"1min":0.50600000000000001,"5min":0.40200000000000002,"15min":0.36399999999999999},"max":{"1min":0.80200000000000005,"5min":0.94299999999999995,"15min":0.99299999999999999},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63700000000000001,"15min":0.66200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.81999999999999995,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.78600000000000003},"min":{"1min":0.52400000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":1.008,"15min":1.6839999999999999},"last":0.77900000000000003}]},{"osd":7,"last update":"Thu Jul 6 03:53:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.72299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41299999999999998},"max":{"1min":0.88200000000000001,"5min":0.95699999999999996,"15min":1.0660000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70399999999999996,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.99399999999999999,"5min":1.079,"15min":1.21},"last":0.54700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36924,"kb_used_data":1780,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334916,"statfs":{"total":96636764160,"available":96598953984,"internally_reserved":0,"allocated":1822720,"data_stored":1148365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70599999999999996,"15min":0.66100000000000003},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.32100000000000001},"max":{"1min":0.871,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.753},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68300000000000005,"15min":0.83299999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":6.4059999999999997},"last":0.78100000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.307},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":1.1970000000000001,"15min":1.546},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.79100000000000004,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.376},"max":{"1min":8.1120000000000001,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.69099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.80800000000000005,"15min":0.73999999999999999},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":8.1829999999999998,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73999999999999999,"15min":0.747},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":1.3140000000000001,"5min":4.484,"15min":4.484},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.627,"15min":0.628},"min":{"1min":0.46200000000000002,"5min":0.40899999999999997,"15min":0.34699999999999998},"max":{"1min":0.88500000000000001,"5min":1.1359999999999999,"15min":1.6240000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.42499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.89700000000000002,"5min":1.0569999999999999,"15min":2.718},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.48399999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91200000000000003,"5min":1.246,"15min":1.669},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:53:04 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.002,"5min":1.3360000000000001,"15min":2.0459999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.749,"15min":0.77800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0609999999999999,"15min":2.2919999999999998},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:53:36.523 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T03:53:36.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T03:53:36.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.756+0000 7f09fdfff700 1 Processor -- start 2023-07-06T03:53:36.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.756+0000 7f09fdfff700 1 -- start start 2023-07-06T03:53:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.757+0000 7f09fdfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80d0920 0x7f09f80d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.757+0000 7f09fdfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09f80d2970 0x7f09f80cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.757+0000 7f09fdfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f80d0080 con 0x7f09f80b20c0 2023-07-06T03:53:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.757+0000 7f09fdfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f80d0360 con 0x7f09f80d2970 2023-07-06T03:53:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.757+0000 7f09fdfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f80d0640 con 0x7f09f80d0920 2023-07-06T03:53:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.757+0000 7f09f6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09f80d2970 0x7f09f80cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.757+0000 7f09f6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09f80d2970 0x7f09f80cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59522/0 (socket says 172.21.15.97:59522) 2023-07-06T03:53:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.757+0000 7f09f7fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f09f80b20c0 0x7f09f80ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51286/0 (socket says 172.21.15.97:51286) 2023-07-06T03:53:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.757+0000 7f09f6ffd700 1 -- 172.21.15.97:0/156578476 learned_addr learned my addr 172.21.15.97:0/156578476 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.757+0000 7f09f77fe700 1 --2- 172.21.15.97:0/156578476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80d0920 0x7f09f80d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.758+0000 7f09f6ffd700 1 -- 172.21.15.97:0/156578476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80d0920 msgr2=0x7f09f80d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.758+0000 7f09f6ffd700 1 --2- 172.21.15.97:0/156578476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80d0920 0x7f09f80d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.758+0000 7f09f6ffd700 1 -- 172.21.15.97:0/156578476 >> v1:172.21.15.97:6789/0 conn(0x7f09f80b20c0 legacy=0x7f09f80ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.758+0000 7f09f6ffd700 1 -- 172.21.15.97:0/156578476 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f80cd440 con 0x7f09f80d2970 2023-07-06T03:53:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.758+0000 7f09f67fc700 1 -- 172.21.15.97:0/156578476 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277070901 0 0) 0x7f09f80d0080 con 0x7f09f80b20c0 2023-07-06T03:53:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.758+0000 7f09f6ffd700 1 --2- 172.21.15.97:0/156578476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f80d2970 0x7f09f80cdc40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0098f0 tx=0x7f09ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142cf386559c8eba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.758+0000 7f09f67fc700 1 -- 172.21.15.97:0/156578476 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00b8a0 con 0x7f09f80d2970 2023-07-06T03:53:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.758+0000 7f09f67fc700 1 -- 172.21.15.97:0/156578476 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec00ba00 con 0x7f09f80d2970 2023-07-06T03:53:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.758+0000 7f09f67fc700 1 -- 172.21.15.97:0/156578476 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00bd90 con 0x7f09f80d2970 2023-07-06T03:53:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.759+0000 7f09fdfff700 1 -- 172.21.15.97:0/156578476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f80d2970 msgr2=0x7f09f80cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.759+0000 7f09fdfff700 1 --2- 172.21.15.97:0/156578476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f80d2970 0x7f09f80cdc40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0098f0 tx=0x7f09ec009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.759+0000 7f09fdfff700 1 -- 172.21.15.97:0/156578476 shutdown_connections 2023-07-06T03:53:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.759+0000 7f09fdfff700 1 --2- 172.21.15.97:0/156578476 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80d0920 0x7f09f80d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.759+0000 7f09fdfff700 1 --2- 172.21.15.97:0/156578476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f80d2970 0x7f09f80cdc40 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.759+0000 7f09fdfff700 1 -- 172.21.15.97:0/156578476 >> 172.21.15.97:0/156578476 conn(0x7f09f809c0d0 msgr2=0x7f09f80d2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.759+0000 7f09fdfff700 1 -- 172.21.15.97:0/156578476 shutdown_connections 2023-07-06T03:53:36.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.759+0000 7f09fdfff700 1 -- 172.21.15.97:0/156578476 wait complete. 2023-07-06T03:53:36.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.761+0000 7f09fdfff700 1 Processor -- start 2023-07-06T03:53:36.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.761+0000 7f09fdfff700 1 -- start start 2023-07-06T03:53:36.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09fdfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b20c0 0x7f09f80c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09fdfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09f80d0920 0x7f09f80c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09fdfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f81f1f80 0x7f09f8156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09fdfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f81f3140 con 0x7f09f81f1f80 2023-07-06T03:53:36.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09fdfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f81f24b0 con 0x7f09f80d0920 2023-07-06T03:53:36.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09fdfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f81f2790 con 0x7f09f80b20c0 2023-07-06T03:53:36.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09f7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f81f1f80 0x7f09f8156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09f7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f81f1f80 0x7f09f8156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59532/0 (socket says 172.21.15.97:59532) 2023-07-06T03:53:36.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09f7fff700 1 -- 172.21.15.97:0/2734080288 learned_addr learned my addr 172.21.15.97:0/2734080288 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:36.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09f77fe700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b20c0 0x7f09f80c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09f6ffd700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09f80d0920 0x7f09f80c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09f7fff700 1 -- 172.21.15.97:0/2734080288 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b20c0 msgr2=0x7f09f80c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:36.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09f7fff700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b20c0 0x7f09f80c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09f7fff700 1 -- 172.21.15.97:0/2734080288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09f80d0920 msgr2=0x7f09f80c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:36.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09f7fff700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09f80d0920 0x7f09f80c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:36.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09f7fff700 1 -- 172.21.15.97:0/2734080288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec009550 con 0x7f09f81f1f80 2023-07-06T03:53:36.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.762+0000 7f09f77fe700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b20c0 0x7f09f80c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:36.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.763+0000 7f09f7fff700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f81f1f80 0x7f09f8156cc0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f09f0002230 tx=0x7f09f000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:36.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.763+0000 7f09dffff700 1 -- 172.21.15.97:0/2734080288 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f000ac20 con 0x7f09f81f1f80 2023-07-06T03:53:36.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.763+0000 7f09dffff700 1 -- 172.21.15.97:0/2734080288 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f000ada0 con 0x7f09f81f1f80 2023-07-06T03:53:36.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.763+0000 7f09fdfff700 1 -- 172.21.15.97:0/2734080288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f81f2a70 con 0x7f09f81f1f80 2023-07-06T03:53:36.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.763+0000 7f09dffff700 1 -- 172.21.15.97:0/2734080288 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0007310 con 0x7f09f81f1f80 2023-07-06T03:53:36.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.763+0000 7f09fdfff700 1 -- 172.21.15.97:0/2734080288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f80c7760 con 0x7f09f81f1f80 2023-07-06T03:53:36.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.764+0000 7f09fdfff700 1 -- 172.21.15.97:0/2734080288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f81f2f90 con 0x7f09f81f1f80 2023-07-06T03:53:36.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.765+0000 7f09dffff700 1 -- 172.21.15.97:0/2734080288 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f09f0007470 con 0x7f09f81f1f80 2023-07-06T03:53:36.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.766+0000 7f09dffff700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f09e406ace0 0x7f09e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:36.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.766+0000 7f09dffff700 1 -- 172.21.15.97:0/2734080288 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f09f0089120 con 0x7f09f81f1f80 2023-07-06T03:53:36.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.766+0000 7f09f77fe700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f09e406ace0 0x7f09e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:36.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.767+0000 7f09f77fe700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f09e406ace0 0x7f09e406d190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f09f80cce20 tx=0x7f09e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:36.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:36.770+0000 7f09dffff700 1 -- 172.21.15.97:0/2734080288 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f09f0054130 con 0x7f09f81f1f80 2023-07-06T03:53:37.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.045+0000 7f09fdfff700 1 -- 172.21.15.97:0/2734080288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f09f81f3280 con 0x7f09f81f1f80 2023-07-06T03:53:37.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.051+0000 7f09dffff700 1 -- 172.21.15.97:0/2734080288 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f09f81f3280 con 0x7f09f81f1f80 2023-07-06T03:53:37.053 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:53:37.053 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg== 2023-07-06T03:53:37.053 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-07-06T03:53:37.053 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-07-06T03:53:37.053 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T03:53:37.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 -- 172.21.15.97:0/2734080288 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f09e406ace0 msgr2=0x7f09e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:37.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f09e406ace0 0x7f09e406d190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f09f80cce20 tx=0x7f09e8008040 comp rx=0 tx=0).stop 2023-07-06T03:53:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 -- 172.21.15.97:0/2734080288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f81f1f80 msgr2=0x7f09f8156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f81f1f80 0x7f09f8156cc0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f09f0002230 tx=0x7f09f000a060 comp rx=0 tx=0).stop 2023-07-06T03:53:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 -- 172.21.15.97:0/2734080288 shutdown_connections 2023-07-06T03:53:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f09e406ace0 0x7f09e406d190 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f81f1f80 0x7f09f8156cc0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f09f80b20c0 0x7f09f80c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 --2- 172.21.15.97:0/2734080288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09f80d0920 0x7f09f80c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 -- 172.21.15.97:0/2734080288 >> 172.21.15.97:0/2734080288 conn(0x7f09f809c0d0 msgr2=0x7f09f80b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:37.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 -- 172.21.15.97:0/2734080288 shutdown_connections 2023-07-06T03:53:37.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.054+0000 7f09fdfff700 1 -- 172.21.15.97:0/2734080288 wait complete. 2023-07-06T03:53:37.057 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-07-06T03:53:37.074 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:53:37.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.266+0000 7f9ace7ff700 1 Processor -- start 2023-07-06T03:53:37.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.266+0000 7f9ace7ff700 1 -- start start 2023-07-06T03:53:37.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.268+0000 7f9ace7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ac815c740 0x7f9ac80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:37.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.268+0000 7f9ace7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac8071820 0x7f9ac806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:37.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.268+0000 7f9ace7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac80cdc80 con 0x7f9ac8071c90 2023-07-06T03:53:37.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.268+0000 7f9ace7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac80d0610 con 0x7f9ac815c740 2023-07-06T03:53:37.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.268+0000 7f9ace7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80ce380 con 0x7f9ac8071820 2023-07-06T03:53:37.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.268+0000 7f9ac7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ac815c740 0x7f9ac80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:37.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.268+0000 7f9accd9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9ac8071c90 0x7f9ac80ca2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51292/0 (socket says 172.21.15.97:51292) 2023-07-06T03:53:37.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.268+0000 7f9accd9c700 1 -- 172.21.15.97:0/2625238010 learned_addr learned my addr 172.21.15.97:0/2625238010 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:37.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.268+0000 7f9ac77fe700 1 --2- 172.21.15.97:0/2625238010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac8071820 0x7f9ac806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:37.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.269+0000 7f9ac7fff700 1 -- 172.21.15.97:0/2625238010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac8071820 msgr2=0x7f9ac806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:37.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.269+0000 7f9ac7fff700 1 --2- 172.21.15.97:0/2625238010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac8071820 0x7f9ac806a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.269+0000 7f9ac7fff700 1 -- 172.21.15.97:0/2625238010 >> v1:172.21.15.97:6789/0 conn(0x7f9ac8071c90 legacy=0x7f9ac80ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:37.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.269+0000 7f9ac7fff700 1 -- 172.21.15.97:0/2625238010 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80cd630 con 0x7f9ac815c740 2023-07-06T03:53:37.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.269+0000 7f9ac77fe700 1 --2- 172.21.15.97:0/2625238010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac8071820 0x7f9ac806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:37.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.269+0000 7f9ac7fff700 1 --2- 172.21.15.97:0/2625238010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac815c740 0x7f9ac80d00d0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009830 tx=0x7f9ab8009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0684f7c3f12e64 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:37.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.269+0000 7f9ac6ffd700 1 -- 172.21.15.97:0/2625238010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8011830 con 0x7f9ac815c740 2023-07-06T03:53:37.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.269+0000 7f9ac6ffd700 1 -- 172.21.15.97:0/2625238010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab8011990 con 0x7f9ac815c740 2023-07-06T03:53:37.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.269+0000 7f9ac6ffd700 1 -- 172.21.15.97:0/2625238010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8011d40 con 0x7f9ac815c740 2023-07-06T03:53:37.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.270+0000 7f9ace7ff700 1 -- 172.21.15.97:0/2625238010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac815c740 msgr2=0x7f9ac80d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:37.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.270+0000 7f9ace7ff700 1 --2- 172.21.15.97:0/2625238010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac815c740 0x7f9ac80d00d0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009830 tx=0x7f9ab8009b40 comp rx=0 tx=0).stop 2023-07-06T03:53:37.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.270+0000 7f9ace7ff700 1 -- 172.21.15.97:0/2625238010 shutdown_connections 2023-07-06T03:53:37.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.270+0000 7f9ace7ff700 1 --2- 172.21.15.97:0/2625238010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac815c740 0x7f9ac80d00d0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.270+0000 7f9ace7ff700 1 --2- 172.21.15.97:0/2625238010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac8071820 0x7f9ac806a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.270+0000 7f9ace7ff700 1 -- 172.21.15.97:0/2625238010 >> 172.21.15.97:0/2625238010 conn(0x7f9ac80b2280 msgr2=0x7f9ac80cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:37.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.270+0000 7f9ace7ff700 1 -- 172.21.15.97:0/2625238010 shutdown_connections 2023-07-06T03:53:37.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.270+0000 7f9ace7ff700 1 -- 172.21.15.97:0/2625238010 wait complete. 2023-07-06T03:53:37.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.272+0000 7f9ace7ff700 1 Processor -- start 2023-07-06T03:53:37.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.272+0000 7f9ace7ff700 1 -- start start 2023-07-06T03:53:37.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ace7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8071820 0x7f9ac81446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:37.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ace7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac815c740 0x7f9ac80b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:37.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ace7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac80b2950 0x7f9ac80b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:37.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ace7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80b3ad0 con 0x7f9ac815c740 2023-07-06T03:53:37.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ace7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80b2ec0 con 0x7f9ac8071820 2023-07-06T03:53:37.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ace7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80b31a0 con 0x7f9ac80b2950 2023-07-06T03:53:37.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ac77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac815c740 0x7f9ac80b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:37.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ac77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac815c740 0x7f9ac80b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59552/0 (socket says 172.21.15.97:59552) 2023-07-06T03:53:37.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ac77fe700 1 -- 172.21.15.97:0/3926984474 learned_addr learned my addr 172.21.15.97:0/3926984474 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:37.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9accd9c700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac80b2950 0x7f9ac80b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:37.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ac77fe700 1 -- 172.21.15.97:0/3926984474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac80b2950 msgr2=0x7f9ac80b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:37.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ac77fe700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac80b2950 0x7f9ac80b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ac77fe700 1 -- 172.21.15.97:0/3926984474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8071820 msgr2=0x7f9ac81446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:37.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ac7fff700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8071820 0x7f9ac81446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:37.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ac77fe700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8071820 0x7f9ac81446e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ac77fe700 1 -- 172.21.15.97:0/3926984474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab8009440 con 0x7f9ac815c740 2023-07-06T03:53:37.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ac7fff700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8071820 0x7f9ac81446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:37.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.273+0000 7f9ac77fe700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac815c740 0x7f9ac80b3f60 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e810 tx=0x7f9abc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:37.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.274+0000 7f9ac4ff9700 1 -- 172.21.15.97:0/3926984474 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc004230 con 0x7f9ac815c740 2023-07-06T03:53:37.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.274+0000 7f9ac4ff9700 1 -- 172.21.15.97:0/3926984474 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc004390 con 0x7f9ac815c740 2023-07-06T03:53:37.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.274+0000 7f9ac4ff9700 1 -- 172.21.15.97:0/3926984474 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0036a0 con 0x7f9ac815c740 2023-07-06T03:53:37.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.274+0000 7f9ace7ff700 1 -- 172.21.15.97:0/3926984474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac80b3480 con 0x7f9ac815c740 2023-07-06T03:53:37.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.274+0000 7f9ace7ff700 1 -- 172.21.15.97:0/3926984474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac80b44a0 con 0x7f9ac815c740 2023-07-06T03:53:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.275+0000 7f9ace7ff700 1 -- 172.21.15.97:0/3926984474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac804e340 con 0x7f9ac815c740 2023-07-06T03:53:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.276+0000 7f9ac4ff9700 1 -- 172.21.15.97:0/3926984474 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9abc003800 con 0x7f9ac815c740 2023-07-06T03:53:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.277+0000 7f9ac4ff9700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ab406ace0 0x7f9ab406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.277+0000 7f9ac4ff9700 1 -- 172.21.15.97:0/3926984474 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9abc089060 con 0x7f9ac815c740 2023-07-06T03:53:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.277+0000 7f9ac7fff700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ab406ace0 0x7f9ab406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:37.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.277+0000 7f9ac7fff700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ab406ace0 0x7f9ab406d190 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80090b0 tx=0x7f9ab801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:37.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.281+0000 7f9ac4ff9700 1 -- 172.21.15.97:0/3926984474 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9abc054070 con 0x7f9ac815c740 2023-07-06T03:53:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.513+0000 7f9ace7ff700 1 -- 172.21.15.97:0/3926984474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ac80d0070 con 0x7f9ac815c740 2023-07-06T03:53:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.514+0000 7f9ac4ff9700 1 -- 172.21.15.97:0/3926984474 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4673 (secure 0 0 0) 0x7f9abc057690 con 0x7f9ac815c740 2023-07-06T03:53:37.516 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:37.516 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":67,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/387292825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":387292825},{"type":"v1","addr":"172.21.15.146:6833","nonce":387292825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690380561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690380561},{"type":"v1","addr":"172.21.15.97:6837","nonce":690380561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3897834271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3897834271},{"type":"v1","addr":"172.21.15.146:6835","nonce":3897834271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:53:26.234190+0000","modified":"2023-07-06T03:53:29.774892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/3324055785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3324055785},{"type":"v1","addr":"172.21.15.97:6836","nonce":3324055785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-06T03:53:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.516+0000 7f9ace7ff700 1 -- 172.21.15.97:0/3926984474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ab406ace0 msgr2=0x7f9ab406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.516+0000 7f9ace7ff700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ab406ace0 0x7f9ab406d190 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80090b0 tx=0x7f9ab801c040 comp rx=0 tx=0).stop 2023-07-06T03:53:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.516+0000 7f9ace7ff700 1 -- 172.21.15.97:0/3926984474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac815c740 msgr2=0x7f9ac80b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.516+0000 7f9ace7ff700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac815c740 0x7f9ac80b3f60 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e810 tx=0x7f9abc00ebd0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.517+0000 7f9ace7ff700 1 -- 172.21.15.97:0/3926984474 shutdown_connections 2023-07-06T03:53:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.517+0000 7f9ace7ff700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ab406ace0 0x7f9ab406d190 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.517+0000 7f9ace7ff700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ac80b2950 0x7f9ac80b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.517+0000 7f9ace7ff700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ac8071820 0x7f9ac81446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.517+0000 7f9ace7ff700 1 --2- 172.21.15.97:0/3926984474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ac815c740 0x7f9ac80b3f60 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.517+0000 7f9ace7ff700 1 -- 172.21.15.97:0/3926984474 >> 172.21.15.97:0/3926984474 conn(0x7f9ac80b2280 msgr2=0x7f9ac80cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.517+0000 7f9ace7ff700 1 -- 172.21.15.97:0/3926984474 shutdown_connections 2023-07-06T03:53:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:37.517+0000 7f9ace7ff700 1 -- 172.21.15.97:0/3926984474 wait complete. 2023-07-06T03:53:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 67 2023-07-06T03:53:37.937 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T03:53:26.234190+0000', 'modified': '2023-07-06T03:53:29.774892+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7992}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7992': {'gid': 7992, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6836/3324055785', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 3324055785}, {'type': 'v1', 'addr': '172.21.15.97:6836', 'nonce': 3324055785}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T03:53:37.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T03:53:37.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-06T03:53:38.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.135+0000 7f16735ff700 1 Processor -- start 2023-07-06T03:53:38.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.135+0000 7f16735ff700 1 -- start start 2023-07-06T03:53:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.136+0000 7f16735ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f166c15c610 0x7f166c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.136+0000 7f16735ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f166c0d2210 0x7f166c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.136+0000 7f16735ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166c0cff90 con 0x7f166c06b360 2023-07-06T03:53:38.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.136+0000 7f16735ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166c0d0780 con 0x7f166c15c610 2023-07-06T03:53:38.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.136+0000 7f16735ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c15c9e0 con 0x7f166c0d2210 2023-07-06T03:53:38.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.136+0000 7f167139b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f166c15c610 0x7f166c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.136+0000 7f167139b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f166c15c610 0x7f166c0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:60128/0 (socket says 172.21.15.97:60128) 2023-07-06T03:53:38.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.136+0000 7f167139b700 1 -- 172.21.15.97:0/3069557884 learned_addr learned my addr 172.21.15.97:0/3069557884 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:38.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.136+0000 7f1671b9c700 1 --2- 172.21.15.97:0/3069557884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f166c0d2210 0x7f166c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.137+0000 7f167139b700 1 -- 172.21.15.97:0/3069557884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f166c0d2210 msgr2=0x7f166c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.137+0000 7f167139b700 1 --2- 172.21.15.97:0/3069557884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f166c0d2210 0x7f166c0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.137+0000 7f167139b700 1 -- 172.21.15.97:0/3069557884 >> v1:172.21.15.146:6789/0 conn(0x7f166c06b360 legacy=0x7f166c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.137+0000 7f167139b700 1 -- 172.21.15.97:0/3069557884 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c0b5650 con 0x7f166c15c610 2023-07-06T03:53:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.137+0000 7f1663fff700 1 -- 172.21.15.97:0/3069557884 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179175627 0 0) 0x7f166c0cff90 con 0x7f166c06b360 2023-07-06T03:53:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.137+0000 7f167139b700 1 --2- 172.21.15.97:0/3069557884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166c15c610 0x7f166c0b47b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f16680098f0 tx=0x7f1668009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceea3a462b8a0ef7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.137+0000 7f1663fff700 1 -- 172.21.15.97:0/3069557884 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668011830 con 0x7f166c15c610 2023-07-06T03:53:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.138+0000 7f1663fff700 1 -- 172.21.15.97:0/3069557884 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1668011990 con 0x7f166c15c610 2023-07-06T03:53:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.138+0000 7f1663fff700 1 -- 172.21.15.97:0/3069557884 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668011d20 con 0x7f166c15c610 2023-07-06T03:53:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.138+0000 7f16735ff700 1 -- 172.21.15.97:0/3069557884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166c15c610 msgr2=0x7f166c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.138+0000 7f16735ff700 1 --2- 172.21.15.97:0/3069557884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166c15c610 0x7f166c0b47b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f16680098f0 tx=0x7f1668009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.138+0000 7f16735ff700 1 -- 172.21.15.97:0/3069557884 shutdown_connections 2023-07-06T03:53:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.138+0000 7f16735ff700 1 --2- 172.21.15.97:0/3069557884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f166c0d2210 0x7f166c0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.138+0000 7f16735ff700 1 --2- 172.21.15.97:0/3069557884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166c15c610 0x7f166c0b47b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.138+0000 7f16735ff700 1 -- 172.21.15.97:0/3069557884 >> 172.21.15.97:0/3069557884 conn(0x7f166c06a6c0 msgr2=0x7f166c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.138+0000 7f16735ff700 1 -- 172.21.15.97:0/3069557884 shutdown_connections 2023-07-06T03:53:38.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.138+0000 7f16735ff700 1 -- 172.21.15.97:0/3069557884 wait complete. 2023-07-06T03:53:38.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.140+0000 7f16735ff700 1 Processor -- start 2023-07-06T03:53:38.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.140+0000 7f16735ff700 1 -- start start 2023-07-06T03:53:38.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.141+0000 7f16735ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166c06b360 0x7f166c157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.141+0000 7f16735ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166c0d2210 0x7f166c15ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.141+0000 7f16735ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f166c15c610 0x7f166c0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.141+0000 7f16735ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c15cb20 con 0x7f166c0d2210 2023-07-06T03:53:38.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.141+0000 7f16735ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c15cd30 con 0x7f166c06b360 2023-07-06T03:53:38.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.141+0000 7f16735ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c15d010 con 0x7f166c15c610 2023-07-06T03:53:38.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.141+0000 7f1670b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166c0d2210 0x7f166c15ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.141+0000 7f1670b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166c0d2210 0x7f166c15ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59558/0 (socket says 172.21.15.97:59558) 2023-07-06T03:53:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.141+0000 7f1670b9a700 1 -- 172.21.15.97:0/1437328983 learned_addr learned my addr 172.21.15.97:0/1437328983 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.141+0000 7f167139b700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166c06b360 0x7f166c157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.141+0000 7f1671b9c700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f166c15c610 0x7f166c0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f1670b9a700 1 -- 172.21.15.97:0/1437328983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f166c15c610 msgr2=0x7f166c0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f1670b9a700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f166c15c610 0x7f166c0cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f1670b9a700 1 -- 172.21.15.97:0/1437328983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166c06b360 msgr2=0x7f166c157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f1670b9a700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166c06b360 0x7f166c157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f1670b9a700 1 -- 172.21.15.97:0/1437328983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1668009550 con 0x7f166c0d2210 2023-07-06T03:53:38.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f1671b9c700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f166c15c610 0x7f166c0cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:38.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f1670b9a700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166c0d2210 0x7f166c15ddd0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f165c0011e0 tx=0x7f165c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:38.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f1661ffb700 1 -- 172.21.15.97:0/1437328983 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c0054e0 con 0x7f166c0d2210 2023-07-06T03:53:38.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f1661ffb700 1 -- 172.21.15.97:0/1437328983 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c003680 con 0x7f166c0d2210 2023-07-06T03:53:38.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f1661ffb700 1 -- 172.21.15.97:0/1437328983 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c003a30 con 0x7f166c0d2210 2023-07-06T03:53:38.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f16735ff700 1 -- 172.21.15.97:0/1437328983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166c15d2f0 con 0x7f166c0d2210 2023-07-06T03:53:38.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.142+0000 7f16735ff700 1 -- 172.21.15.97:0/1437328983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166c15d8b0 con 0x7f166c0d2210 2023-07-06T03:53:38.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.144+0000 7f1661ffb700 1 -- 172.21.15.97:0/1437328983 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f165c003b90 con 0x7f166c0d2210 2023-07-06T03:53:38.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.145+0000 7f1661ffb700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165006ace0 0x7f165006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.145+0000 7f1661ffb700 1 -- 172.21.15.97:0/1437328983 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f165c0889d0 con 0x7f166c0d2210 2023-07-06T03:53:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.145+0000 7f167139b700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165006ace0 0x7f165006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.146+0000 7f167139b700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165006ace0 0x7f165006d190 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1668014040 tx=0x7f1668008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.147+0000 7f1686fc1b80 1 Processor -- start 2023-07-06T03:53:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.147+0000 7f1686fc1b80 1 -- start start 2023-07-06T03:53:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.148+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eeba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.148+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eedda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.148+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ee97110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.148+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5ee0bea0 con 0x560c5ee94b50 2023-07-06T03:53:38.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.148+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5ed7f600 con 0x560c5eebf190 2023-07-06T03:53:38.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.148+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c5edee910 con 0x560c5eeb9d50 2023-07-06T03:53:38.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.148+0000 7f1671b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ee97110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.148+0000 7f1671b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ee97110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59570/0 (socket says 172.21.15.97:59570) 2023-07-06T03:53:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f1671b9c700 1 -- 172.21.15.97:0/1818223108 learned_addr learned my addr 172.21.15.97:0/1818223108 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f167139b700 1 --2- 172.21.15.97:0/1818223108 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eeba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f1671b9c700 1 -- 172.21.15.97:0/1818223108 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 msgr2=0x560c5eeba120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f1671b9c700 1 --2- 172.21.15.97:0/1818223108 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eeba120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f1671b9c700 1 -- 172.21.15.97:0/1818223108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 msgr2=0x560c5eedda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f1671b9c700 1 --2- 172.21.15.97:0/1818223108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eedda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f1671b9c700 1 -- 172.21.15.97:0/1818223108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c5ebdb790 con 0x560c5ee94b50 2023-07-06T03:53:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f167139b700 1 --2- 172.21.15.97:0/1818223108 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eeba120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f1671b9c700 1 --2- 172.21.15.97:0/1818223108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ee97110 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f1664007550 tx=0x7f1664009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25295d2e68a422b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f164effd700 1 -- 172.21.15.97:0/1818223108 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640147e0 con 0x560c5ee94b50 2023-07-06T03:53:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f164effd700 1 -- 172.21.15.97:0/1818223108 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166400f050 con 0x560c5ee94b50 2023-07-06T03:53:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.149+0000 7f164effd700 1 -- 172.21.15.97:0/1818223108 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664014b60 con 0x560c5ee94b50 2023-07-06T03:53:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.150+0000 7f1686fc1b80 1 -- 172.21.15.97:0/1818223108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 msgr2=0x560c5ee97110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.150+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/1818223108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ee97110 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f1664007550 tx=0x7f1664009750 comp rx=0 tx=0).stop 2023-07-06T03:53:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.150+0000 7f1686fc1b80 1 -- 172.21.15.97:0/1818223108 shutdown_connections 2023-07-06T03:53:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.150+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/1818223108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ee97110 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.150+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/1818223108 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eeba120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.150+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/1818223108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eedda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.150+0000 7f1686fc1b80 1 -- 172.21.15.97:0/1818223108 >> 172.21.15.97:0/1818223108 conn(0x560c5ee8c310 msgr2=0x560c5eedc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.150+0000 7f1686fc1b80 1 -- 172.21.15.97:0/1818223108 shutdown_connections 2023-07-06T03:53:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.150+0000 7f1686fc1b80 1 -- 172.21.15.97:0/1818223108 wait complete. 2023-07-06T03:53:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.151+0000 7f1686fc1b80 1 -- start start 2023-07-06T03:53:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.151+0000 7f1686fc1b80 1 Processor -- start 2023-07-06T03:53:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.153+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eeb3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.153+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eeb9d50 0x560c5eedda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.153+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eebf190 0x560c5eedf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.153+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5ede7be0 con 0x560c5ee94b50 2023-07-06T03:53:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.153+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5eda55b0 con 0x560c5eeb9d50 2023-07-06T03:53:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.153+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c5ed8b910 con 0x560c5eebf190 2023-07-06T03:53:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.153+0000 7f1686fc1b80 10 client.0 _ll_register_callbacks cb 0x560c5eda3b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:53:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.153+0000 7f167139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eeb3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.153+0000 7f167139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eeb3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59574/0 (socket says 172.21.15.97:59574) 2023-07-06T03:53:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.153+0000 7f167139b700 1 -- 172.21.15.97:0/2310646742 learned_addr learned my addr 172.21.15.97:0/2310646742 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f1670b9a700 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eeb9d50 0x560c5eedda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f1671b9c700 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eebf190 0x560c5eedf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f167139b700 1 -- 172.21.15.97:0/2310646742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eebf190 msgr2=0x560c5eedf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f167139b700 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eebf190 0x560c5eedf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f167139b700 1 -- 172.21.15.97:0/2310646742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eeb9d50 msgr2=0x560c5eedda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f167139b700 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eeb9d50 0x560c5eedda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f167139b700 1 -- 172.21.15.97:0/2310646742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1664009e30 con 0x560c5ee94b50 2023-07-06T03:53:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f1670b9a700 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eeb9d50 0x560c5eedda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f1671b9c700 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eebf190 0x560c5eedf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f167139b700 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eeb3c50 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f1668009f50 tx=0x7f16680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb1b9fcf38c9831 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f1647fff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:53:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f1647fff700 1 -- 172.21.15.97:0/2310646742 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668003bb0 con 0x560c5ee94b50 2023-07-06T03:53:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f1647fff700 1 -- 172.21.15.97:0/2310646742 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1668003d10 con 0x560c5ee94b50 2023-07-06T03:53:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.154+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2310646742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c5ee89920 con 0x560c5ee94b50 2023-07-06T03:53:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1647fff700 1 -- 172.21.15.97:0/2310646742 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f166801b430 con 0x560c5ee94b50 2023-07-06T03:53:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1686fc1b80 10 client.8175 fetch_fsmap learned FSMap version 67 2023-07-06T03:53:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2310646742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x560c5ecc80a0 con 0x560c5ee94b50 2023-07-06T03:53:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1647fff700 1 -- 172.21.15.97:0/2310646742 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c5ee89920 con 0x560c5ee94b50 2023-07-06T03:53:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1647fff700 1 -- 172.21.15.97:0/2310646742 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 67) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f166801b590 con 0x560c5ee94b50 2023-07-06T03:53:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1686fc1b80 10 client.8175 fetch_fsmap finished waiting for FSMap version 67 2023-07-06T03:53:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1686fc1b80 10 client.8175 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T03:53:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1686fc1b80 20 client.8175 populate_metadata read hostname 'smithi097' 2023-07-06T03:53:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.97:6834/3324055785,v1:172.21.15.97:6836/3324055785] conn(0x560c5eeb4830 0x560c5eeb4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1686fc1b80 4 client.8175 mds_command: new command op to 7992 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T03:53:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2310646742 --> [v2:172.21.15.97:6834/3324055785,v1:172.21.15.97:6836/3324055785] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c5ee660d0 con 0x560c5eeb4830 2023-07-06T03:53:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.155+0000 7f1670b9a700 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.97:6834/3324055785,v1:172.21.15.97:6836/3324055785] conn(0x560c5eeb4830 0x560c5eeb4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.156+0000 7f1647fff700 10 client.8175 ms_handle_connect on v2:172.21.15.97:6834/3324055785 2023-07-06T03:53:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.156+0000 7f1670b9a700 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.97:6834/3324055785,v1:172.21.15.97:6836/3324055785] conn(0x560c5eeb4830 0x560c5eeb4c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca3faf190d06ced server_cookie=fd12a7b1b338a6e in_seq=0 out_seq=0 2023-07-06T03:53:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.156+0000 7f1647fff700 1 -- 172.21.15.97:0/2310646742 <== mds.0 v2:172.21.15.97:6834/3324055785 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560c5ee660d0 con 0x560c5eeb4830 2023-07-06T03:53:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.156+0000 7f1647fff700 10 client.8175 handle_command_reply: tid=0 2023-07-06T03:53:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.156+0000 7f1686fc1b80 1 client.8175 shutdown 2023-07-06T03:53:38.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2310646742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 msgr2=0x560c5eeb3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eeb3c50 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f1668009f50 tx=0x7f16680047c0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2310646742 shutdown_connections 2023-07-06T03:53:38.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.97:6834/3324055785,v1:172.21.15.97:6836/3324055785] conn(0x560c5eeb4830 0x560c5eeb4c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eebf190 0x560c5eedf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eeb3c50 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1647fff700 0 client.8175 ms_handle_reset on v2:172.21.15.97:6834/3324055785 2023-07-06T03:53:38.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2310646742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eeb9d50 0x560c5eedda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2310646742 >> 172.21.15.97:0/2310646742 conn(0x560c5ee8c310 msgr2=0x560c5ee90ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:38.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2310646742 shutdown_connections 2023-07-06T03:53:38.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2310646742 wait complete. 2023-07-06T03:53:38.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.157+0000 7f1686fc1b80 20 client.8175 trim_cache size 0 max 16384 2023-07-06T03:53:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.175+0000 7f1686fc1b80 1 Processor -- start 2023-07-06T03:53:38.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.175+0000 7f1686fc1b80 1 -- start start 2023-07-06T03:53:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.176+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eed60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.176+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eedef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.176+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eec11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.176+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5ebdc390 con 0x560c5ee94b50 2023-07-06T03:53:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.176+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5eede2a0 con 0x560c5eebf190 2023-07-06T03:53:38.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.176+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c5eec0da0 con 0x560c5eeb9d50 2023-07-06T03:53:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.176+0000 7f167139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eed60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.176+0000 7f167139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eed60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59578/0 (socket says 172.21.15.97:59578) 2023-07-06T03:53:38.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.176+0000 7f167139b700 1 -- 172.21.15.97:0/3508417017 learned_addr learned my addr 172.21.15.97:0/3508417017 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f1670b9a700 1 --2- 172.21.15.97:0/3508417017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eedef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f167139b700 1 -- 172.21.15.97:0/3508417017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 msgr2=0x560c5eedef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:38.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f1671b9c700 1 --2- 172.21.15.97:0/3508417017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eec11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f167139b700 1 --2- 172.21.15.97:0/3508417017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eedef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f167139b700 1 -- 172.21.15.97:0/3508417017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 msgr2=0x560c5eec11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:38.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f167139b700 1 --2- 172.21.15.97:0/3508417017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eec11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f167139b700 1 -- 172.21.15.97:0/3508417017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c5edadb10 con 0x560c5ee94b50 2023-07-06T03:53:38.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f1670b9a700 1 --2- 172.21.15.97:0/3508417017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eedef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:38.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f1671b9c700 1 --2- 172.21.15.97:0/3508417017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eec11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:38.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f167139b700 1 --2- 172.21.15.97:0/3508417017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eed60b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f1668011710 tx=0x7f1668029040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973c896131cf015c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f164f7fe700 1 -- 172.21.15.97:0/3508417017 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166801b630 con 0x560c5ee94b50 2023-07-06T03:53:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f164f7fe700 1 -- 172.21.15.97:0/3508417017 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16680039b0 con 0x560c5ee94b50 2023-07-06T03:53:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.177+0000 7f164f7fe700 1 -- 172.21.15.97:0/3508417017 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668003d60 con 0x560c5ee94b50 2023-07-06T03:53:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.178+0000 7f1686fc1b80 1 -- 172.21.15.97:0/3508417017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 msgr2=0x560c5eed60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.178+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/3508417017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eed60b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f1668011710 tx=0x7f1668029040 comp rx=0 tx=0).stop 2023-07-06T03:53:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.178+0000 7f1686fc1b80 1 -- 172.21.15.97:0/3508417017 shutdown_connections 2023-07-06T03:53:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.178+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/3508417017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eec11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.178+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/3508417017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5eed60b0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.178+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/3508417017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eedef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.178+0000 7f1686fc1b80 1 -- 172.21.15.97:0/3508417017 >> 172.21.15.97:0/3508417017 conn(0x560c5ee8c310 msgr2=0x560c5ee96fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:38.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.178+0000 7f1686fc1b80 1 -- 172.21.15.97:0/3508417017 shutdown_connections 2023-07-06T03:53:38.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.178+0000 7f1686fc1b80 1 -- 172.21.15.97:0/3508417017 wait complete. 2023-07-06T03:53:38.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.178+0000 7f1686fc1b80 1 -- start start 2023-07-06T03:53:38.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.179+0000 7f1686fc1b80 1 Processor -- start 2023-07-06T03:53:38.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.179+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ef446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eec3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f1686fc1b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eec4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5eec4ab0 con 0x560c5ee94b50 2023-07-06T03:53:38.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5eeedb20 con 0x560c5eebf190 2023-07-06T03:53:38.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f1686fc1b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c5eeedd70 con 0x560c5eeb9d50 2023-07-06T03:53:38.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f1686fc1b80 10 client.0 _ll_register_callbacks cb 0x560c5eda3b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:53:38.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f167139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ef446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f167139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ef446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59586/0 (socket says 172.21.15.97:59586) 2023-07-06T03:53:38.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f167139b700 1 -- 172.21.15.97:0/2653398825 learned_addr learned my addr 172.21.15.97:0/2653398825 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:38.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f1670b9a700 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eec3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f1671b9c700 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eec4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f167139b700 1 -- 172.21.15.97:0/2653398825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 msgr2=0x560c5eec3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:38.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f167139b700 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eec3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f167139b700 1 -- 172.21.15.97:0/2653398825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 msgr2=0x560c5eec4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:38.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f167139b700 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eec4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f167139b700 1 -- 172.21.15.97:0/2653398825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c5ecc80a0 con 0x560c5ee94b50 2023-07-06T03:53:38.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f1670b9a700 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eec3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:38.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f1671b9c700 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eec4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:38.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f167139b700 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ef446a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f16680098f0 tx=0x7f166801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c6b04b1fa5f46b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:38.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.180+0000 7f1647fff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:53:38.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1647fff700 1 -- 172.21.15.97:0/2653398825 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166803a880 con 0x560c5ee94b50 2023-07-06T03:53:38.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1647fff700 1 -- 172.21.15.97:0/2653398825 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f166803a9e0 con 0x560c5ee94b50 2023-07-06T03:53:38.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2653398825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f166400f050 con 0x560c5ee94b50 2023-07-06T03:53:38.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1647fff700 1 -- 172.21.15.97:0/2653398825 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f166803abc0 con 0x560c5ee94b50 2023-07-06T03:53:38.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1686fc1b80 10 client.8181 fetch_fsmap learned FSMap version 67 2023-07-06T03:53:38.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2653398825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x560c5ec3c1d0 con 0x560c5ee94b50 2023-07-06T03:53:38.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1647fff700 1 -- 172.21.15.97:0/2653398825 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166400f050 con 0x560c5ee94b50 2023-07-06T03:53:38.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1647fff700 1 -- 172.21.15.97:0/2653398825 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 67) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1668041030 con 0x560c5ee94b50 2023-07-06T03:53:38.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1686fc1b80 10 client.8181 fetch_fsmap finished waiting for FSMap version 67 2023-07-06T03:53:38.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1686fc1b80 10 client.8181 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T03:53:38.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1686fc1b80 20 client.8181 populate_metadata read hostname 'smithi097' 2023-07-06T03:53:38.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.97:6834/3324055785,v1:172.21.15.97:6836/3324055785] conn(0x560c5eedc1a0 0x560c5ee905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1686fc1b80 4 client.8181 mds_command: new command op to 7992 tid=0[{"prefix": "status"}] 2023-07-06T03:53:38.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.181+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2653398825 --> [v2:172.21.15.97:6834/3324055785,v1:172.21.15.97:6836/3324055785] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560c5ede0d20 con 0x560c5eedc1a0 2023-07-06T03:53:38.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1670b9a700 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.97:6834/3324055785,v1:172.21.15.97:6836/3324055785] conn(0x560c5eedc1a0 0x560c5ee905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1670b9a700 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.97:6834/3324055785,v1:172.21.15.97:6836/3324055785] conn(0x560c5eedc1a0 0x560c5ee905a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab6731855a38e1b server_cookie=2b140ef1b77295cb in_seq=0 out_seq=0 2023-07-06T03:53:38.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1647fff700 10 client.8181 ms_handle_connect on v2:172.21.15.97:6834/3324055785 2023-07-06T03:53:38.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1647fff700 1 -- 172.21.15.97:0/2653398825 <== mds.0 v2:172.21.15.97:6834/3324055785 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x560c5ede0d20 con 0x560c5eedc1a0 2023-07-06T03:53:38.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1647fff700 10 client.8181 handle_command_reply: tid=0 2023-07-06T03:53:38.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1686fc1b80 1 client.8181 shutdown 2023-07-06T03:53:38.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2653398825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 msgr2=0x560c5ef446a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ef446a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f16680098f0 tx=0x7f166801d040 comp rx=0 tx=0).stop 2023-07-06T03:53:38.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2653398825 shutdown_connections 2023-07-06T03:53:38.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.97:6834/3324055785,v1:172.21.15.97:6836/3324055785] conn(0x560c5eedc1a0 0x560c5ee905a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c5eebf190 0x560c5eec4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c5ee94b50 0x560c5ef446a0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1686fc1b80 1 --2- 172.21.15.97:0/2653398825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x560c5eeb9d50 0x560c5eec3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1647fff700 0 client.8181 ms_handle_reset on v2:172.21.15.97:6834/3324055785 2023-07-06T03:53:38.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.182+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2653398825 >> 172.21.15.97:0/2653398825 conn(0x560c5ee8c310 msgr2=0x560c5ee96fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:38.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2653398825 shutdown_connections 2023-07-06T03:53:38.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f1686fc1b80 1 -- 172.21.15.97:0/2653398825 wait complete. 2023-07-06T03:53:38.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f1686fc1b80 20 client.8181 trim_cache size 0 max 16384 2023-07-06T03:53:38.193 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:53:38.193 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36", 2023-07-06T03:53:38.193 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-07-06T03:53:38.193 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7992, 2023-07-06T03:53:38.193 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-07-06T03:53:38.194 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-07-06T03:53:38.194 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-07-06T03:53:38.194 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 11.437087533, 2023-07-06T03:53:38.194 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 67, 2023-07-06T03:53:38.194 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 99, 2023-07-06T03:53:38.194 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 99, 2023-07-06T03:53:38.194 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 17.344080447 2023-07-06T03:53:38.194 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:53:38.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 -- 172.21.15.97:0/1437328983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165006ace0 msgr2=0x7f165006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165006ace0 0x7f165006d190 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1668014040 tx=0x7f1668008030 comp rx=0 tx=0).stop 2023-07-06T03:53:38.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 -- 172.21.15.97:0/1437328983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166c0d2210 msgr2=0x7f166c15ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166c0d2210 0x7f166c15ddd0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f165c0011e0 tx=0x7f165c00cae0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 -- 172.21.15.97:0/1437328983 shutdown_connections 2023-07-06T03:53:38.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f165006ace0 0x7f165006d190 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f166c15c610 0x7f166c0cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f166c06b360 0x7f166c157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 --2- 172.21.15.97:0/1437328983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f166c0d2210 0x7f166c15ddd0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 -- 172.21.15.97:0/1437328983 >> 172.21.15.97:0/1437328983 conn(0x7f166c06a6c0 msgr2=0x7f166c0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:38.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 -- 172.21.15.97:0/1437328983 shutdown_connections 2023-07-06T03:53:38.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.183+0000 7f16735ff700 1 -- 172.21.15.97:0/1437328983 wait complete. 2023-07-06T03:53:38.202 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:53:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.394+0000 7f6eccbff700 1 Processor -- start 2023-07-06T03:53:38.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.395+0000 7f6eccbff700 1 -- start start 2023-07-06T03:53:38.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6eccbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ec80cd390 0x7f6ec8160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6eccbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec80cdae0 con 0x7f6ec8160dd0 2023-07-06T03:53:38.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6eccbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec80b6640 con 0x7f6ec815c520 2023-07-06T03:53:38.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6eccbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec80b6920 con 0x7f6ec80cd390 2023-07-06T03:53:38.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6ebffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ec80cd390 0x7f6ec8160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6ec6d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6ec8160dd0 0x7f6ec80b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51306/0 (socket says 172.21.15.97:51306) 2023-07-06T03:53:38.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6ebffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ec80cd390 0x7f6ec8160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59596/0 (socket says 172.21.15.97:59596) 2023-07-06T03:53:38.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6ec6d9d700 1 -- 172.21.15.97:0/1146210800 learned_addr learned my addr 172.21.15.97:0/1146210800 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:38.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6ebffff700 1 -- 172.21.15.97:0/1146210800 >> v1:172.21.15.146:6789/0 conn(0x7f6ec815c520 legacy=0x7f6ec8160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6ebffff700 1 -- 172.21.15.97:0/1146210800 >> v1:172.21.15.97:6789/0 conn(0x7f6ec8160dd0 legacy=0x7f6ec80b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6ebffff700 1 -- 172.21.15.97:0/1146210800 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec80b6c00 con 0x7f6ec80cd390 2023-07-06T03:53:38.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.396+0000 7f6ec5d9b700 1 -- 172.21.15.97:0/1146210800 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319281634 0 0) 0x7f6ec80cdae0 con 0x7f6ec8160dd0 2023-07-06T03:53:38.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.397+0000 7f6ebffff700 1 --2- 172.21.15.97:0/1146210800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ec80cd390 0x7f6ec8160890 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009840 tx=0x7f6eb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501fcbd1ee7eb97d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:38.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.397+0000 7f6ec5d9b700 1 -- 172.21.15.97:0/1146210800 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0011830 con 0x7f6ec80cd390 2023-07-06T03:53:38.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.397+0000 7f6ec5d9b700 1 -- 172.21.15.97:0/1146210800 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb0011990 con 0x7f6ec80cd390 2023-07-06T03:53:38.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.397+0000 7f6ec5d9b700 1 -- 172.21.15.97:0/1146210800 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0011d20 con 0x7f6ec80cd390 2023-07-06T03:53:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.397+0000 7f6eccbff700 1 -- 172.21.15.97:0/1146210800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ec80cd390 msgr2=0x7f6ec8160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.397+0000 7f6eccbff700 1 --2- 172.21.15.97:0/1146210800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ec80cd390 0x7f6ec8160890 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009840 tx=0x7f6eb0009b50 comp rx=0 tx=0).stop 2023-07-06T03:53:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.397+0000 7f6eccbff700 1 -- 172.21.15.97:0/1146210800 shutdown_connections 2023-07-06T03:53:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.397+0000 7f6eccbff700 1 --2- 172.21.15.97:0/1146210800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ec80cd390 0x7f6ec8160890 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.397+0000 7f6eccbff700 1 -- 172.21.15.97:0/1146210800 >> 172.21.15.97:0/1146210800 conn(0x7f6ec8071790 msgr2=0x7f6ec8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.398+0000 7f6eccbff700 1 -- 172.21.15.97:0/1146210800 shutdown_connections 2023-07-06T03:53:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.398+0000 7f6eccbff700 1 -- 172.21.15.97:0/1146210800 wait complete. 2023-07-06T03:53:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.399+0000 7f6eccbff700 1 Processor -- start 2023-07-06T03:53:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.399+0000 7f6eccbff700 1 -- start start 2023-07-06T03:53:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.400+0000 7f6eccbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cd390 0x7f6ec815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.400+0000 7f6eccbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ec815c520 0x7f6ec8142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.400+0000 7f6eccbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ec8160dd0 0x7f6ec81f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.400+0000 7f6eccbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec815d1f0 con 0x7f6ec8160dd0 2023-07-06T03:53:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.400+0000 7f6eccbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec815cc30 con 0x7f6ec80cd390 2023-07-06T03:53:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.400+0000 7f6eccbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec815cf10 con 0x7f6ec815c520 2023-07-06T03:53:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ec6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ec8160dd0 0x7f6ec81f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ec6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ec8160dd0 0x7f6ec81f5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59610/0 (socket says 172.21.15.97:59610) 2023-07-06T03:53:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ec6d9d700 1 -- 172.21.15.97:0/652394505 learned_addr learned my addr 172.21.15.97:0/652394505 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ec659c700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cd390 0x7f6ec815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ebffff700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ec815c520 0x7f6ec8142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ec6d9d700 1 -- 172.21.15.97:0/652394505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ec815c520 msgr2=0x7f6ec8142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ec6d9d700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ec815c520 0x7f6ec8142c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ec6d9d700 1 -- 172.21.15.97:0/652394505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cd390 msgr2=0x7f6ec815ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ec6d9d700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cd390 0x7f6ec815ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ec6d9d700 1 -- 172.21.15.97:0/652394505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb00094a0 con 0x7f6ec8160dd0 2023-07-06T03:53:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ec659c700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cd390 0x7f6ec815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ec6d9d700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ec8160dd0 0x7f6ec81f5750 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0002230 tx=0x7f6ec000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.401+0000 7f6ebeffd700 1 -- 172.21.15.97:0/652394505 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000ac20 con 0x7f6ec8160dd0 2023-07-06T03:53:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.402+0000 7f6ebeffd700 1 -- 172.21.15.97:0/652394505 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec000ada0 con 0x7f6ec8160dd0 2023-07-06T03:53:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.402+0000 7f6ebeffd700 1 -- 172.21.15.97:0/652394505 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0007310 con 0x7f6ec8160dd0 2023-07-06T03:53:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.402+0000 7f6eccbff700 1 -- 172.21.15.97:0/652394505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec815f2a0 con 0x7f6ec8160dd0 2023-07-06T03:53:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.402+0000 7f6eccbff700 1 -- 172.21.15.97:0/652394505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec81f5c90 con 0x7f6ec8160dd0 2023-07-06T03:53:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.403+0000 7f6eccbff700 1 -- 172.21.15.97:0/652394505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec815f0f0 con 0x7f6ec8160dd0 2023-07-06T03:53:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.404+0000 7f6ebeffd700 1 -- 172.21.15.97:0/652394505 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ec0007470 con 0x7f6ec8160dd0 2023-07-06T03:53:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.405+0000 7f6ebeffd700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ea806ace0 0x7f6ea806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.405+0000 7f6ebeffd700 1 -- 172.21.15.97:0/652394505 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6ec00890c0 con 0x7f6ec8160dd0 2023-07-06T03:53:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.405+0000 7f6ec659c700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ea806ace0 0x7f6ea806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.406+0000 7f6ec659c700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ea806ace0 0x7f6ea806d190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8000f60 tx=0x7f6eb8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.409+0000 7f6ebeffd700 1 -- 172.21.15.97:0/652394505 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6ec00540d0 con 0x7f6ec8160dd0 2023-07-06T03:53:38.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.644+0000 7f6eccbff700 1 -- 172.21.15.97:0/652394505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6ec804e340 con 0x7f6ec8160dd0 2023-07-06T03:53:38.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.644+0000 7f6ebeffd700 1 -- 172.21.15.97:0/652394505 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6ec00576f0 con 0x7f6ec8160dd0 2023-07-06T03:53:38.646 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:53:38.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 -- 172.21.15.97:0/652394505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ea806ace0 msgr2=0x7f6ea806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ea806ace0 0x7f6ea806d190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8000f60 tx=0x7f6eb8009040 comp rx=0 tx=0).stop 2023-07-06T03:53:38.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 -- 172.21.15.97:0/652394505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ec8160dd0 msgr2=0x7f6ec81f5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:38.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ec8160dd0 0x7f6ec81f5750 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0002230 tx=0x7f6ec000a060 comp rx=0 tx=0).stop 2023-07-06T03:53:38.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 -- 172.21.15.97:0/652394505 shutdown_connections 2023-07-06T03:53:38.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ea806ace0 0x7f6ea806d190 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ec8160dd0 0x7f6ec81f5750 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ec80cd390 0x7f6ec815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 --2- 172.21.15.97:0/652394505 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ec815c520 0x7f6ec8142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:38.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 -- 172.21.15.97:0/652394505 >> 172.21.15.97:0/652394505 conn(0x7f6ec8071790 msgr2=0x7f6ec80b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:38.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 -- 172.21.15.97:0/652394505 shutdown_connections 2023-07-06T03:53:38.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:38.647+0000 7f6eccbff700 1 -- 172.21.15.97:0/652394505 wait complete. 2023-07-06T03:53:38.666 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:53:38.667 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:53:38.667 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:53:38.667 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:53:38.668 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:53:38.668 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:53:38.668 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:53:38.668 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:53:38.669 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:53:38.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:38.669 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:53:38.714 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:53:38.715 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:53:38.715 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:53:38.715 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:38.715 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:53:38.715 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:38.715 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:53:38.715 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:53:38.716 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:53:38.716 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:53:38.716 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:53:38.716 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:53:38.716 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:53:38.716 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:53:38.716 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:53:38.717 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:53:38.717 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:53:38.717 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41650sec preferred_lft 41650sec 2023-07-06T03:53:38.717 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:53:38.717 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:38.717 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:53:38.717 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:53:38.717 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:53:38.718 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:38.718 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:53:38.718 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:38.718 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:53:38.718 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:53:38.718 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:53:38.718 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:38.718 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:53:38.719 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:53:38.719 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:53:38.719 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:38.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:38.720 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:53:38.828 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:53:38.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:38.831 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:53:38.878 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:53:38.879 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:53:38.879 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:53:38.879 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:53:38.879 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:53:38.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:38.880 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:53:38.881 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:53:38.881 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:53:38.881 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:53:38.881 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:53:38.882 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:53:39.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:39.049 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:53:39.094 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:53:39.095 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:53:39.096 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:53:39.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:39.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:53:39.204 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:53:39.205 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:53:39.205 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:53:39.250 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:53:39.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:53:39.250 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:53:39.334 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:53:39.443 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:53:39.443 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:53:39.510 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:53:39.513 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:53:39.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:39.514 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:53:39.559 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:53:39.559 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:53:39.603 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:53:39.673 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:53:39.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:53:39.741 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:53:39.743 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:53:39.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:39.744 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:53:39.768 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:53:39.766+0000 7f6c84fe55c0 -1 init, newargv = 0x5571cc33af80 newargc=15 2023-07-06T03:53:39.768 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[154228]: starting ceph client 2023-07-06T03:53:39.775 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[154228]: starting fuse 2023-07-06T03:53:39.793 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:53:39.794 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:53:39.794 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:53:39.795 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:53:39.795 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:53:39.795 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:53:39.795 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:53:39.796 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:39.796 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:53:39.796 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:53:39.797 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:53:39.797 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:53:39.797 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:39.798 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:53:39.798 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:53:39.798 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:53:39.798 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:53:39.799 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:39.799 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:53:39.799 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:53:39.799 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:53:39.800 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:53:39.800 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:53:39.800 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:53:39.801 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:53:39.801 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:53:39.801 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:39.802 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:53:39.802 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:53:39.975 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.154228.asok 2023-07-06T03:53:40.153 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.154228.asok 2023-07-06T03:53:40.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:40.154 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.154228.asok status 2023-07-06T03:53:40.376 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:53:40.376 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:53:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:53:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:53:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:53:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:53:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:53:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "154228", 2023-07-06T03:53:40.377 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:53:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:53:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:53:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8199, 2023-07-06T03:53:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:53:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:53:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:53:40.378 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 8199 2023-07-06T03:53:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:53:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:53:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:53:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2529580715 2023-07-06T03:53:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:40.379 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:53:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:53:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:53:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2529580715 2023-07-06T03:53:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.8199 192.168.0.1:0/2529580715", 2023-07-06T03:53:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2529580715", 2023-07-06T03:53:40.380 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:53:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 67, 2023-07-06T03:53:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 99, 2023-07-06T03:53:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:53:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:53:40.381 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:53:40.381 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:53:40.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:40.391 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:53:40.391 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:53:40.436 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:53:40.437 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:53:40.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:40.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:53:40.506 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:53:40.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.699+0000 7fe1cb5ff700 1 Processor -- start 2023-07-06T03:53:40.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.699+0000 7fe1cb5ff700 1 -- start start 2023-07-06T03:53:40.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.700+0000 7fe1cb5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe1c41592f0 0x7fe1c40b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:40.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.700+0000 7fe1cb5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1c406d860 0x7fe1c40b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:40.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1cb5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c415cc40 con 0x7fe1c4158f20 2023-07-06T03:53:40.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1cb5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c415cf20 con 0x7fe1c41592f0 2023-07-06T03:53:40.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1cb5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c415d200 con 0x7fe1c406d860 2023-07-06T03:53:40.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1c8b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe1c41592f0 0x7fe1c40b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:40.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1c939b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe1c4158f20 0x7fe1c40caac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55388/0 (socket says 172.21.15.97:55388) 2023-07-06T03:53:40.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1c8b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe1c41592f0 0x7fe1c40b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35288/0 (socket says 172.21.15.97:35288) 2023-07-06T03:53:40.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1c939b700 1 -- 172.21.15.97:0/443852348 learned_addr learned my addr 172.21.15.97:0/443852348 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:40.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1c9b9c700 1 --2- 172.21.15.97:0/443852348 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1c406d860 0x7fe1c40b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:40.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1c8b9a700 1 -- 172.21.15.97:0/443852348 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1c406d860 msgr2=0x7fe1c40b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:40.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1c8b9a700 1 --2- 172.21.15.97:0/443852348 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1c406d860 0x7fe1c40b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1bbfff700 1 -- 172.21.15.97:0/443852348 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083567842 0 0) 0x7fe1c415cc40 con 0x7fe1c4158f20 2023-07-06T03:53:40.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1c8b9a700 1 -- 172.21.15.97:0/443852348 >> v1:172.21.15.97:6789/0 conn(0x7fe1c4158f20 legacy=0x7fe1c40caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:40.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.701+0000 7fe1c8b9a700 1 -- 172.21.15.97:0/443852348 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c415d4e0 con 0x7fe1c41592f0 2023-07-06T03:53:40.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1c8b9a700 1 --2- 172.21.15.97:0/443852348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c41592f0 0x7fe1c40b0560 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400bae0 tx=0x7fe1b400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0ac7e9756e093a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:40.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1bbfff700 1 -- 172.21.15.97:0/443852348 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b400dd30 con 0x7fe1c41592f0 2023-07-06T03:53:40.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1bbfff700 1 -- 172.21.15.97:0/443852348 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b400de90 con 0x7fe1c41592f0 2023-07-06T03:53:40.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1bbfff700 1 -- 172.21.15.97:0/443852348 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4010690 con 0x7fe1c41592f0 2023-07-06T03:53:40.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/443852348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c41592f0 msgr2=0x7fe1c40b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:40.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1cb5ff700 1 --2- 172.21.15.97:0/443852348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c41592f0 0x7fe1c40b0560 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400bae0 tx=0x7fe1b400bdf0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/443852348 shutdown_connections 2023-07-06T03:53:40.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1cb5ff700 1 --2- 172.21.15.97:0/443852348 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1c406d860 0x7fe1c40b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1cb5ff700 1 --2- 172.21.15.97:0/443852348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c41592f0 0x7fe1c40b0560 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/443852348 >> 172.21.15.97:0/443852348 conn(0x7fe1c40ce8c0 msgr2=0x7fe1c40cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:40.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/443852348 shutdown_connections 2023-07-06T03:53:40.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.702+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/443852348 wait complete. 2023-07-06T03:53:40.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.704+0000 7fe1cb5ff700 1 Processor -- start 2023-07-06T03:53:40.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.704+0000 7fe1cb5ff700 1 -- start start 2023-07-06T03:53:40.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.705+0000 7fe1cb5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1c406d860 0x7fe1c4157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:40.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.705+0000 7fe1cb5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4158f20 0x7fe1c41596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:40.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.705+0000 7fe1cb5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c41592f0 0x7fe1c415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:40.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.705+0000 7fe1cb5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c414ec20 con 0x7fe1c41592f0 2023-07-06T03:53:40.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.705+0000 7fe1cb5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c415b450 con 0x7fe1c4158f20 2023-07-06T03:53:40.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.705+0000 7fe1cb5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c415b730 con 0x7fe1c406d860 2023-07-06T03:53:40.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.705+0000 7fe1c9b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c41592f0 0x7fe1c415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:40.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1c9b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c41592f0 0x7fe1c415c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35296/0 (socket says 172.21.15.97:35296) 2023-07-06T03:53:40.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1c9b9c700 1 -- 172.21.15.97:0/1850076052 learned_addr learned my addr 172.21.15.97:0/1850076052 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:40.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1c939b700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1c406d860 0x7fe1c4157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:40.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1c8b9a700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4158f20 0x7fe1c41596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:40.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1c9b9c700 1 -- 172.21.15.97:0/1850076052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1c406d860 msgr2=0x7fe1c4157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:40.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1c9b9c700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1c406d860 0x7fe1c4157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1c9b9c700 1 -- 172.21.15.97:0/1850076052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4158f20 msgr2=0x7fe1c41596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:40.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1c9b9c700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4158f20 0x7fe1c41596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1c9b9c700 1 -- 172.21.15.97:0/1850076052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b400b740 con 0x7fe1c41592f0 2023-07-06T03:53:40.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1c9b9c700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c41592f0 0x7fe1c415c210 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc007e40 tx=0x7fe1bc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:40.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1b9ffb700 1 -- 172.21.15.97:0/1850076052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00ed30 con 0x7fe1c41592f0 2023-07-06T03:53:40.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1b9ffb700 1 -- 172.21.15.97:0/1850076052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc014420 con 0x7fe1c41592f0 2023-07-06T03:53:40.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.706+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/1850076052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c415ba10 con 0x7fe1c41592f0 2023-07-06T03:53:40.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.707+0000 7fe1b9ffb700 1 -- 172.21.15.97:0/1850076052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc0147d0 con 0x7fe1c41592f0 2023-07-06T03:53:40.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.707+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/1850076052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c415b0b0 con 0x7fe1c41592f0 2023-07-06T03:53:40.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.708+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/1850076052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c415bf30 con 0x7fe1c41592f0 2023-07-06T03:53:40.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.709+0000 7fe1b9ffb700 1 -- 172.21.15.97:0/1850076052 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe1bc014930 con 0x7fe1c41592f0 2023-07-06T03:53:40.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.710+0000 7fe1b9ffb700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe1a806ace0 0x7fe1a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:40.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.710+0000 7fe1b9ffb700 1 -- 172.21.15.97:0/1850076052 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1bc089b80 con 0x7fe1c41592f0 2023-07-06T03:53:40.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.710+0000 7fe1c939b700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe1a806ace0 0x7fe1a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:40.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.710+0000 7fe1c939b700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe1a806ace0 0x7fe1a806d190 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0007fd0 tx=0x7fe1c0007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:40.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.714+0000 7fe1b9ffb700 1 -- 172.21.15.97:0/1850076052 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe1c415bf30 con 0x7fe1c41592f0 2023-07-06T03:53:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.959+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/1850076052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe1c415bf30 con 0x7fe1c41592f0 2023-07-06T03:53:40.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.960+0000 7fe1b9ffb700 1 -- 172.21.15.97:0/1850076052 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe1c415bf30 con 0x7fe1c41592f0 2023-07-06T03:53:40.962 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:53:40.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/1850076052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe1a806ace0 msgr2=0x7fe1a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:40.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe1a806ace0 0x7fe1a806d190 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0007fd0 tx=0x7fe1c0007de0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/1850076052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c41592f0 msgr2=0x7fe1c415c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:40.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c41592f0 0x7fe1c415c210 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc007e40 tx=0x7fe1bc00e3c0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/1850076052 shutdown_connections 2023-07-06T03:53:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe1a806ace0 0x7fe1a806d190 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c41592f0 0x7fe1c415c210 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe1c406d860 0x7fe1c4157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 --2- 172.21.15.97:0/1850076052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1c4158f20 0x7fe1c41596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/1850076052 >> 172.21.15.97:0/1850076052 conn(0x7fe1c40ce8c0 msgr2=0x7fe1c4151b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/1850076052 shutdown_connections 2023-07-06T03:53:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:40.962+0000 7fe1cb5ff700 1 -- 172.21.15.97:0/1850076052 wait complete. 2023-07-06T03:53:40.981 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:53:40.981 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:53:40.982 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2023-07-06T03:53:40.982 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T03:53:40.982 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T03:53:40.982 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:53:40.983 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:53:40.983 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:53:40.983 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:53:40.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:40.984 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2023-07-06T03:53:41.036 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:53:41.037 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:53:41.037 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:53:41.037 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:41.037 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:53:41.037 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:41.037 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:53:41.037 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:53:41.038 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2023-07-06T03:53:41.038 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:53:41.038 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:53:41.038 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2023-07-06T03:53:41.038 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:53:41.038 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2023-07-06T03:53:41.038 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:53:41.039 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2023-07-06T03:53:41.039 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:53:41.039 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 41664sec preferred_lft 41664sec 2023-07-06T03:53:41.039 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2023-07-06T03:53:41.039 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:41.039 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:53:41.040 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:53:41.040 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:53:41.040 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:41.040 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::58f0:6eff:fe04:3be0/64 scope link 2023-07-06T03:53:41.040 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:41.040 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:53:41.041 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:53:41.041 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::106d:9cff:fed5:9d0f/64 scope link 2023-07-06T03:53:41.041 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:41.041 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:53:41.041 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether a2:c5:04:8e:a2:a7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T03:53:41.041 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::a0c5:4ff:fe8e:a2a7/64 scope link 2023-07-06T03:53:41.041 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:53:41.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:41.043 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:53:41.112 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:53:41.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:41.116 DEBUG:teuthology.orchestra.run.smithi146:> route 2023-07-06T03:53:41.164 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2023-07-06T03:53:41.164 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:53:41.165 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:53:41.165 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:53:41.165 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:53:41.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:41.166 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:53:41.167 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:53:41.167 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:53:41.167 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:53:41.167 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:53:41.168 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:53:41.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:41.336 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:53:41.381 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T03:53:41.382 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T03:53:41.382 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:53:41.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:41.383 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2023-07-06T03:53:41.492 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-06T03:53:41.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:53:41.493 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T03:53:41.537 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:53:41.538 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:53:41.538 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:53:41.622 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:53:41.731 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:53:41.732 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:53:41.800 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:53:41.802 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:53:41.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:41.803 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:53:41.850 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:53:41.850 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-06T03:53:41.895 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:53:41.964 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:53:41.965 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:53:42.034 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:53:42.037 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:53:42.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:42.038 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:53:42.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2023-07-06T03:53:42.060+0000 7f92a5a575c0 -1 init, newargv = 0x559a2d4dee40 newargc=15 2023-07-06T03:53:42.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[125117]: starting ceph client 2023-07-06T03:53:42.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[125117]: starting fuse 2023-07-06T03:53:42.088 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:53:42.088 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T03:53:42.088 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:53:42.089 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2023-07-06T03:53:42.089 DEBUG:teuthology.orchestra.run.smithi146:> import re 2023-07-06T03:53:42.089 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:53:42.090 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2023-07-06T03:53:42.090 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:53:42.090 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2023-07-06T03:53:42.091 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-06T03:53:42.091 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2023-07-06T03:53:42.091 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-06T03:53:42.091 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:53:42.092 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2023-07-06T03:53:42.092 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2023-07-06T03:53:42.092 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2023-07-06T03:53:42.093 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2023-07-06T03:53:42.093 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:53:42.093 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2023-07-06T03:53:42.093 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:53:42.094 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:53:42.094 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:53:42.094 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2023-07-06T03:53:42.095 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2023-07-06T03:53:42.095 DEBUG:teuthology.orchestra.run.smithi146:> return f 2023-07-06T03:53:42.095 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:53:42.096 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:53:42.096 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2023-07-06T03:53:42.096 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:53:42.270 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.125117.asok 2023-07-06T03:53:42.448 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.125117.asok 2023-07-06T03:53:42.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:42.449 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.125117.asok status 2023-07-06T03:53:42.672 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-07-06T03:53:42.673 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2023-07-06T03:53:42.673 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:53:42.673 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:53:42.673 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2023-07-06T03:53:42.673 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2023-07-06T03:53:42.674 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-06T03:53:42.674 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "125117", 2023-07-06T03:53:42.674 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2023-07-06T03:53:42.674 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:53:42.674 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2023-07-06T03:53:42.674 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2023-07-06T03:53:42.674 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 7495, 2023-07-06T03:53:42.674 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2023-07-06T03:53:42.675 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2023-07-06T03:53:42.675 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2023-07-06T03:53:42.675 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 7495 2023-07-06T03:53:42.675 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:53:42.675 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:53:42.675 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:53:42.675 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:53:42.675 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1754496776 2023-07-06T03:53:42.676 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-07-06T03:53:42.676 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:53:42.676 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:53:42.676 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:53:42.676 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:53:42.676 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1754496776 2023-07-06T03:53:42.676 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:53:42.677 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.7495 192.168.0.1:0/1754496776", 2023-07-06T03:53:42.677 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/1754496776", 2023-07-06T03:53:42.677 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2023-07-06T03:53:42.677 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 67, 2023-07-06T03:53:42.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 99, 2023-07-06T03:53:42.677 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:53:42.677 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2023-07-06T03:53:42.677 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2023-07-06T03:53:42.678 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-07-06T03:53:42.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:42.687 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2023-07-06T03:53:42.687 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-06T03:53:42.733 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2023-07-06T03:53:42.734 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-06T03:53:42.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:42.734 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:53:42.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-07-06T03:53:43.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.051+0000 7f6b66bff700 1 Processor -- start 2023-07-06T03:53:43.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.051+0000 7f6b66bff700 1 -- start start 2023-07-06T03:53:43.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.053+0000 7f6b66bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6b6015c610 0x7f6b600b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.053+0000 7f6b66bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b600d2210 0x7f6b600d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.053+0000 7f6b66bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b600cff90 con 0x7f6b6006b360 2023-07-06T03:53:43.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.053+0000 7f6b66bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b600d0780 con 0x7f6b6015c610 2023-07-06T03:53:43.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.053+0000 7f6b66bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6015c9e0 con 0x7f6b600d2210 2023-07-06T03:53:43.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.053+0000 7f6b6519c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b600d2210 0x7f6b600d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.053+0000 7f6b6499b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6b6015c610 0x7f6b600b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.053+0000 7f6b6519c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b600d2210 0x7f6b600d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:37194/0 (socket says 172.21.15.97:37194) 2023-07-06T03:53:43.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.053+0000 7f6b6519c700 1 -- 172.21.15.97:0/588369241 learned_addr learned my addr 172.21.15.97:0/588369241 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:43.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.054+0000 7f6b5f7fe700 1 -- 172.21.15.97:0/588369241 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458007591 0 0) 0x7f6b600cff90 con 0x7f6b6006b360 2023-07-06T03:53:43.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.054+0000 7f6b5f7fe700 1 -- 172.21.15.97:0/588369241 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b500035b0 con 0x7f6b6006b360 2023-07-06T03:53:43.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.054+0000 7f6b6519c700 1 -- 172.21.15.97:0/588369241 >> v2:172.21.15.146:3300/0 conn(0x7f6b6015c610 msgr2=0x7f6b600b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:43.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.054+0000 7f6b6519c700 1 --2- 172.21.15.97:0/588369241 >> v2:172.21.15.146:3300/0 conn(0x7f6b6015c610 0x7f6b600b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.054+0000 7f6b6519c700 1 -- 172.21.15.97:0/588369241 >> v1:172.21.15.146:6789/0 conn(0x7f6b6006b360 legacy=0x7f6b600b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.054+0000 7f6b6519c700 1 -- 172.21.15.97:0/588369241 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600b5650 con 0x7f6b600d2210 2023-07-06T03:53:43.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.054+0000 7f6b6499b700 1 --2- 172.21.15.97:0/588369241 >> v2:172.21.15.146:3300/0 conn(0x7f6b6015c610 0x7f6b600b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:53:43.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.054+0000 7f6b6519c700 1 --2- 172.21.15.97:0/588369241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b600d2210 0x7f6b600d1310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009e50 tx=0x7f6b5800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5245fa3e17496ae9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:43.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.054+0000 7f6b5f7fe700 1 -- 172.21.15.97:0/588369241 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800e070 con 0x7f6b600d2210 2023-07-06T03:53:43.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.054+0000 7f6b5f7fe700 1 -- 172.21.15.97:0/588369241 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5800b910 con 0x7f6b600d2210 2023-07-06T03:53:43.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.054+0000 7f6b5f7fe700 1 -- 172.21.15.97:0/588369241 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800bcc0 con 0x7f6b600d2210 2023-07-06T03:53:43.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.055+0000 7f6b66bff700 1 -- 172.21.15.97:0/588369241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b600d2210 msgr2=0x7f6b600d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.055+0000 7f6b66bff700 1 --2- 172.21.15.97:0/588369241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b600d2210 0x7f6b600d1310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009e50 tx=0x7f6b5800b330 comp rx=0 tx=0).stop 2023-07-06T03:53:43.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.055+0000 7f6b66bff700 1 -- 172.21.15.97:0/588369241 shutdown_connections 2023-07-06T03:53:43.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.055+0000 7f6b66bff700 1 --2- 172.21.15.97:0/588369241 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b600d2210 0x7f6b600d1310 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.055+0000 7f6b66bff700 1 --2- 172.21.15.97:0/588369241 >> v2:172.21.15.146:3300/0 conn(0x7f6b6015c610 0x7f6b600b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.055+0000 7f6b66bff700 1 -- 172.21.15.97:0/588369241 >> 172.21.15.97:0/588369241 conn(0x7f6b6006a6c0 msgr2=0x7f6b6006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:43.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.055+0000 7f6b66bff700 1 -- 172.21.15.97:0/588369241 shutdown_connections 2023-07-06T03:53:43.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.055+0000 7f6b66bff700 1 -- 172.21.15.97:0/588369241 wait complete. 2023-07-06T03:53:43.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.057+0000 7f6b66bff700 1 Processor -- start 2023-07-06T03:53:43.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.057+0000 7f6b66bff700 1 -- start start 2023-07-06T03:53:43.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.058+0000 7f6b66bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b6006b360 0x7f6b60153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.058+0000 7f6b66bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b600d2210 0x7f6b600b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.058+0000 7f6b66bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b6015c610 0x7f6b600b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.058+0000 7f6b66bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600cbb50 con 0x7f6b600d2210 2023-07-06T03:53:43.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.058+0000 7f6b66bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600c6e20 con 0x7f6b6015c610 2023-07-06T03:53:43.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.058+0000 7f6b66bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6015e110 con 0x7f6b6006b360 2023-07-06T03:53:43.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.058+0000 7f6b5ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b600d2210 0x7f6b600b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.058+0000 7f6b6499b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b6006b360 0x7f6b60153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.058+0000 7f6b5ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b600d2210 0x7f6b600b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35298/0 (socket says 172.21.15.97:35298) 2023-07-06T03:53:43.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.058+0000 7f6b5ffff700 1 -- 172.21.15.97:0/3676174075 learned_addr learned my addr 172.21.15.97:0/3676174075 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:43.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.058+0000 7f6b6519c700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b6015c610 0x7f6b600b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b5ffff700 1 -- 172.21.15.97:0/3676174075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b6006b360 msgr2=0x7f6b60153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b5ffff700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b6006b360 0x7f6b60153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b5ffff700 1 -- 172.21.15.97:0/3676174075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b6015c610 msgr2=0x7f6b600b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b5ffff700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b6015c610 0x7f6b600b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b5ffff700 1 -- 172.21.15.97:0/3676174075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b540094a0 con 0x7f6b600d2210 2023-07-06T03:53:43.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b6499b700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b6006b360 0x7f6b60153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:53:43.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b5ffff700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b600d2210 0x7f6b600b5e40 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00c520 tx=0x7f6b4c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:43.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b5d7fa700 1 -- 172.21.15.97:0/3676174075 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c0052d0 con 0x7f6b600d2210 2023-07-06T03:53:43.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b5d7fa700 1 -- 172.21.15.97:0/3676174075 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c005430 con 0x7f6b600d2210 2023-07-06T03:53:43.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b5d7fa700 1 -- 172.21.15.97:0/3676174075 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c0038c0 con 0x7f6b600d2210 2023-07-06T03:53:43.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b66bff700 1 -- 172.21.15.97:0/3676174075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b58009b30 con 0x7f6b600d2210 2023-07-06T03:53:43.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.059+0000 7f6b66bff700 1 -- 172.21.15.97:0/3676174075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6015e6d0 con 0x7f6b600d2210 2023-07-06T03:53:43.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.060+0000 7f6b66bff700 1 -- 172.21.15.97:0/3676174075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b600c6bf0 con 0x7f6b600d2210 2023-07-06T03:53:43.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.061+0000 7f6b5d7fa700 1 -- 172.21.15.97:0/3676174075 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b4c003a20 con 0x7f6b600d2210 2023-07-06T03:53:43.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.062+0000 7f6b5d7fa700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b4406ad10 0x7f6b4406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.063+0000 7f6b5d7fa700 1 -- 172.21.15.97:0/3676174075 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6b4c0889d0 con 0x7f6b600d2210 2023-07-06T03:53:43.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.063+0000 7f6b6499b700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b4406ad10 0x7f6b4406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.063+0000 7f6b6499b700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b4406ad10 0x7f6b4406d1c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6b54011380 tx=0x7f6b54009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:43.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.067+0000 7f6b5d7fa700 1 -- 172.21.15.97:0/3676174075 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6b600c6bf0 con 0x7f6b600d2210 2023-07-06T03:53:43.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.261+0000 7f6b66bff700 1 -- 172.21.15.97:0/3676174075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7f6b600c6bf0 con 0x7f6b600d2210 2023-07-06T03:53:43.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.267+0000 7f6b5d7fa700 1 -- 172.21.15.97:0/3676174075 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v35)=0 v35) v1 ==== 121+0+0 (secure 0 0 0) 0x7f6b600c6bf0 con 0x7f6b600d2210 2023-07-06T03:53:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 -- 172.21.15.97:0/3676174075 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b4406ad10 msgr2=0x7f6b4406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b4406ad10 0x7f6b4406d1c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6b54011380 tx=0x7f6b54009210 comp rx=0 tx=0).stop 2023-07-06T03:53:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 -- 172.21.15.97:0/3676174075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b600d2210 msgr2=0x7f6b600b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b600d2210 0x7f6b600b5e40 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00c520 tx=0x7f6b4c00c830 comp rx=0 tx=0).stop 2023-07-06T03:53:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 -- 172.21.15.97:0/3676174075 shutdown_connections 2023-07-06T03:53:43.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6b4406ad10 0x7f6b4406d1c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b6015c610 0x7f6b600b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6b6006b360 0x7f6b60153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 --2- 172.21.15.97:0/3676174075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b600d2210 0x7f6b600b5e40 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 -- 172.21.15.97:0/3676174075 >> 172.21.15.97:0/3676174075 conn(0x7f6b6006a6c0 msgr2=0x7f6b600b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:43.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 -- 172.21.15.97:0/3676174075 shutdown_connections 2023-07-06T03:53:43.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.270+0000 7f6b66bff700 1 -- 172.21.15.97:0/3676174075 wait complete. 2023-07-06T03:53:43.289 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 400 2023-07-06T03:53:43.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.486+0000 7fe604dff700 1 Processor -- start 2023-07-06T03:53:43.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.486+0000 7fe604dff700 1 -- start start 2023-07-06T03:53:43.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.487+0000 7fe604dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe60015c610 0x7fe6000b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.487+0000 7fe604dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe60006b360 0x7fe6000b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.487+0000 7fe604dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6000cff90 con 0x7fe6000d2210 2023-07-06T03:53:43.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.487+0000 7fe604dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6000d0780 con 0x7fe60006b360 2023-07-06T03:53:43.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.487+0000 7fe604dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60015c9e0 con 0x7fe60015c610 2023-07-06T03:53:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.487+0000 7fe5fdd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe60006b360 0x7fe6000b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.487+0000 7fe5fed9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe6000d2210 0x7fe6000d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55404/0 (socket says 172.21.15.97:55404) 2023-07-06T03:53:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.487+0000 7fe5fed9d700 1 -- 172.21.15.97:0/821336159 learned_addr learned my addr 172.21.15.97:0/821336159 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.487+0000 7fe5fdd9b700 1 --2- 172.21.15.97:0/821336159 >> v2:172.21.15.97:3300/0 conn(0x7fe60006b360 0x7fe6000b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35308/0 (socket says 172.21.15.97:35308) 2023-07-06T03:53:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.488+0000 7fe5fe59c700 1 --2- 172.21.15.97:0/821336159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe60015c610 0x7fe6000b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.488+0000 7fe5fdd9b700 1 -- 172.21.15.97:0/821336159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe60015c610 msgr2=0x7fe6000b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.488+0000 7fe5fdd9b700 1 --2- 172.21.15.97:0/821336159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe60015c610 0x7fe6000b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.488+0000 7fe5fdd9b700 1 -- 172.21.15.97:0/821336159 >> v1:172.21.15.97:6789/0 conn(0x7fe6000d2210 legacy=0x7fe6000d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.488+0000 7fe5fd59a700 1 -- 172.21.15.97:0/821336159 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665747270 0 0) 0x7fe6000cff90 con 0x7fe6000d2210 2023-07-06T03:53:43.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.488+0000 7fe5fdd9b700 1 -- 172.21.15.97:0/821336159 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6000b5650 con 0x7fe60006b360 2023-07-06T03:53:43.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.488+0000 7fe5fdd9b700 1 --2- 172.21.15.97:0/821336159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60006b360 0x7fe6000b4cf0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009870 tx=0x7fe5f4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bee0e60042cfeb4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:43.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.488+0000 7fe5fd59a700 1 -- 172.21.15.97:0/821336159 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4011830 con 0x7fe60006b360 2023-07-06T03:53:43.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.488+0000 7fe5fd59a700 1 -- 172.21.15.97:0/821336159 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f4011990 con 0x7fe60006b360 2023-07-06T03:53:43.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.488+0000 7fe5fd59a700 1 -- 172.21.15.97:0/821336159 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4011d20 con 0x7fe60006b360 2023-07-06T03:53:43.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.489+0000 7fe604dff700 1 -- 172.21.15.97:0/821336159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60006b360 msgr2=0x7fe6000b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.489+0000 7fe604dff700 1 --2- 172.21.15.97:0/821336159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60006b360 0x7fe6000b4cf0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009870 tx=0x7fe5f4009b80 comp rx=0 tx=0).stop 2023-07-06T03:53:43.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.489+0000 7fe604dff700 1 -- 172.21.15.97:0/821336159 shutdown_connections 2023-07-06T03:53:43.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.489+0000 7fe604dff700 1 --2- 172.21.15.97:0/821336159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe60015c610 0x7fe6000b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.489+0000 7fe604dff700 1 --2- 172.21.15.97:0/821336159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60006b360 0x7fe6000b4cf0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.489+0000 7fe604dff700 1 -- 172.21.15.97:0/821336159 >> 172.21.15.97:0/821336159 conn(0x7fe60006a6c0 msgr2=0x7fe60006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:43.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.489+0000 7fe604dff700 1 -- 172.21.15.97:0/821336159 shutdown_connections 2023-07-06T03:53:43.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.489+0000 7fe604dff700 1 -- 172.21.15.97:0/821336159 wait complete. 2023-07-06T03:53:43.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.491+0000 7fe604dff700 1 Processor -- start 2023-07-06T03:53:43.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.492+0000 7fe604dff700 1 -- start start 2023-07-06T03:53:43.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe604dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60006b360 0x7fe6000c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe604dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6000d2210 0x7fe6000cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe604dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe60015c610 0x7fe600144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe604dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe600145780 con 0x7fe60006b360 2023-07-06T03:53:43.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe604dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6000cbda0 con 0x7fe6000d2210 2023-07-06T03:53:43.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe604dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6000cc070 con 0x7fe60015c610 2023-07-06T03:53:43.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fe59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60006b360 0x7fe6000c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fe59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60006b360 0x7fe6000c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35312/0 (socket says 172.21.15.97:35312) 2023-07-06T03:53:43.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fe59c700 1 -- 172.21.15.97:0/473624833 learned_addr learned my addr 172.21.15.97:0/473624833 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:43.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fdd9b700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6000d2210 0x7fe6000cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fed9d700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe60015c610 0x7fe600144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fe59c700 1 -- 172.21.15.97:0/473624833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe60015c610 msgr2=0x7fe600144c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fe59c700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe60015c610 0x7fe600144c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fe59c700 1 -- 172.21.15.97:0/473624833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6000d2210 msgr2=0x7fe6000cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fe59c700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6000d2210 0x7fe6000cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fe59c700 1 -- 172.21.15.97:0/473624833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f40094a0 con 0x7fe60006b360 2023-07-06T03:53:43.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fed9d700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe60015c610 0x7fe600144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:43.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.493+0000 7fe5fe59c700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60006b360 0x7fe6000c6d00 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000e810 tx=0x7fe5f000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:43.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.494+0000 7fe5eeffd700 1 -- 172.21.15.97:0/473624833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0013610 con 0x7fe60006b360 2023-07-06T03:53:43.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.494+0000 7fe5eeffd700 1 -- 172.21.15.97:0/473624833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f0013770 con 0x7fe60006b360 2023-07-06T03:53:43.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.494+0000 7fe604dff700 1 -- 172.21.15.97:0/473624833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6000cc350 con 0x7fe60006b360 2023-07-06T03:53:43.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.494+0000 7fe5eeffd700 1 -- 172.21.15.97:0/473624833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0013b20 con 0x7fe60006b360 2023-07-06T03:53:43.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.494+0000 7fe604dff700 1 -- 172.21.15.97:0/473624833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe600145190 con 0x7fe60006b360 2023-07-06T03:53:43.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.495+0000 7fe604dff700 1 -- 172.21.15.97:0/473624833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6000cc740 con 0x7fe60006b360 2023-07-06T03:53:43.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.496+0000 7fe5eeffd700 1 -- 172.21.15.97:0/473624833 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe5f0013c80 con 0x7fe60006b360 2023-07-06T03:53:43.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.497+0000 7fe5eeffd700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe5e006ace0 0x7fe5e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.497+0000 7fe5eeffd700 1 -- 172.21.15.97:0/473624833 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe5f00891f0 con 0x7fe60006b360 2023-07-06T03:53:43.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.497+0000 7fe5fdd9b700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe5e006ace0 0x7fe5e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.498+0000 7fe5fdd9b700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe5e006ace0 0x7fe5e006d190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe60006b790 tx=0x7fe5f401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:43.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.501+0000 7fe5eeffd700 1 -- 172.21.15.97:0/473624833 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe5f0054210 con 0x7fe60006b360 2023-07-06T03:53:43.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.695+0000 7fe604dff700 1 -- 172.21.15.97:0/473624833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7fe60004e340 con 0x7fe60006b360 2023-07-06T03:53:43.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.701+0000 7fe5eeffd700 1 -- 172.21.15.97:0/473624833 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v36)=0 v36) v1 ==== 119+0+0 (secure 0 0 0) 0x7fe5f0057830 con 0x7fe60006b360 2023-07-06T03:53:43.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 -- 172.21.15.97:0/473624833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe5e006ace0 msgr2=0x7fe5e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe5e006ace0 0x7fe5e006d190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe60006b790 tx=0x7fe5f401c040 comp rx=0 tx=0).stop 2023-07-06T03:53:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 -- 172.21.15.97:0/473624833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60006b360 msgr2=0x7fe6000c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60006b360 0x7fe6000c6d00 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000e810 tx=0x7fe5f000ebd0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 -- 172.21.15.97:0/473624833 shutdown_connections 2023-07-06T03:53:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe5e006ace0 0x7fe5e006d190 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe60015c610 0x7fe600144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60006b360 0x7fe6000c6d00 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 --2- 172.21.15.97:0/473624833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6000d2210 0x7fe6000cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 -- 172.21.15.97:0/473624833 >> 172.21.15.97:0/473624833 conn(0x7fe60006a6c0 msgr2=0x7fe6000cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:43.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 -- 172.21.15.97:0/473624833 shutdown_connections 2023-07-06T03:53:43.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.704+0000 7fe604dff700 1 -- 172.21.15.97:0/473624833 wait complete. 2023-07-06T03:53:43.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 800 2023-07-06T03:53:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.921+0000 7f98073ff700 1 Processor -- start 2023-07-06T03:53:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.921+0000 7f98073ff700 1 -- start start 2023-07-06T03:53:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f98073ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9800158b20 0x7f98000cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f98073ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f98000b0560 0x7f98000b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f98073ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98000b0f00 con 0x7f980006cab0 2023-07-06T03:53:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f98073ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98000ce820 con 0x7f9800158b20 2023-07-06T03:53:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f98073ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9800158ef0 con 0x7f98000b0560 2023-07-06T03:53:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f980519b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9800158b20 0x7f98000cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f980519b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9800158b20 0x7f98000cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35316/0 (socket says 172.21.15.97:35316) 2023-07-06T03:53:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f980519b700 1 -- 172.21.15.97:0/3072309305 learned_addr learned my addr 172.21.15.97:0/3072309305 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f980599c700 1 --2- 172.21.15.97:0/3072309305 >> v2:172.21.15.146:3300/0 conn(0x7f98000b0560 0x7f98000b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f980519b700 1 -- 172.21.15.97:0/3072309305 >> v2:172.21.15.146:3300/0 conn(0x7f98000b0560 msgr2=0x7f98000b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f980519b700 1 --2- 172.21.15.97:0/3072309305 >> v2:172.21.15.146:3300/0 conn(0x7f98000b0560 0x7f98000b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f980519b700 1 -- 172.21.15.97:0/3072309305 >> v1:172.21.15.146:6789/0 conn(0x7f980006cab0 legacy=0x7f98000ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f980519b700 1 -- 172.21.15.97:0/3072309305 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98000ccbc0 con 0x7f9800158b20 2023-07-06T03:53:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.923+0000 7f980599c700 1 --2- 172.21.15.97:0/3072309305 >> v2:172.21.15.146:3300/0 conn(0x7f98000b0560 0x7f98000b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.924+0000 7f980519b700 1 --2- 172.21.15.97:0/3072309305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9800158b20 0x7f98000cc680 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f980006afd0 tx=0x7f97fc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9e2c6b54749e21 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.924+0000 7f97f7fff700 1 -- 172.21.15.97:0/3072309305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc00dd10 con 0x7f9800158b20 2023-07-06T03:53:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.924+0000 7f97f7fff700 1 -- 172.21.15.97:0/3072309305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97fc00de70 con 0x7f9800158b20 2023-07-06T03:53:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.924+0000 7f97f7fff700 1 -- 172.21.15.97:0/3072309305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc004230 con 0x7f9800158b20 2023-07-06T03:53:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.924+0000 7f98073ff700 1 -- 172.21.15.97:0/3072309305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9800158b20 msgr2=0x7f98000cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.924+0000 7f98073ff700 1 --2- 172.21.15.97:0/3072309305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9800158b20 0x7f98000cc680 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f980006afd0 tx=0x7f97fc00bea0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.924+0000 7f98073ff700 1 -- 172.21.15.97:0/3072309305 shutdown_connections 2023-07-06T03:53:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.924+0000 7f98073ff700 1 --2- 172.21.15.97:0/3072309305 >> v2:172.21.15.146:3300/0 conn(0x7f98000b0560 0x7f98000b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.924+0000 7f98073ff700 1 --2- 172.21.15.97:0/3072309305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9800158b20 0x7f98000cc680 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.925+0000 7f98073ff700 1 -- 172.21.15.97:0/3072309305 >> 172.21.15.97:0/3072309305 conn(0x7f980006a060 msgr2=0x7f980006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.925+0000 7f98073ff700 1 -- 172.21.15.97:0/3072309305 shutdown_connections 2023-07-06T03:53:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.925+0000 7f98073ff700 1 -- 172.21.15.97:0/3072309305 wait complete. 2023-07-06T03:53:43.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.926+0000 7f98073ff700 1 Processor -- start 2023-07-06T03:53:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.926+0000 7f98073ff700 1 -- start start 2023-07-06T03:53:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.927+0000 7f98073ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980006cab0 0x7f98001497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.927+0000 7f98073ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98000b0560 0x7f98000c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.927+0000 7f98073ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9800158b20 0x7f98000ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.927+0000 7f98073ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98000b2b10 con 0x7f980006cab0 2023-07-06T03:53:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.927+0000 7f98073ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98000b2830 con 0x7f98000b0560 2023-07-06T03:53:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.927+0000 7f98073ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98000b2f00 con 0x7f9800158b20 2023-07-06T03:53:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f980519b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980006cab0 0x7f98001497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f980519b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980006cab0 0x7f98001497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35332/0 (socket says 172.21.15.97:35332) 2023-07-06T03:53:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f980519b700 1 -- 172.21.15.97:0/520724051 learned_addr learned my addr 172.21.15.97:0/520724051 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f980499a700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98000b0560 0x7f98000c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f980599c700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9800158b20 0x7f98000ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f980519b700 1 -- 172.21.15.97:0/520724051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9800158b20 msgr2=0x7f98000ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f980519b700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9800158b20 0x7f98000ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f980519b700 1 -- 172.21.15.97:0/520724051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98000b0560 msgr2=0x7f98000c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f980519b700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98000b0560 0x7f98000c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f980519b700 1 -- 172.21.15.97:0/520724051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc00b7f0 con 0x7f980006cab0 2023-07-06T03:53:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f980519b700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980006cab0 0x7f98001497c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00bb10 tx=0x7f97fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f97f5ffb700 1 -- 172.21.15.97:0/520724051 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0045f0 con 0x7f980006cab0 2023-07-06T03:53:43.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f97f5ffb700 1 -- 172.21.15.97:0/520724051 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97fc010690 con 0x7f980006cab0 2023-07-06T03:53:43.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f97f5ffb700 1 -- 172.21.15.97:0/520724051 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc010a40 con 0x7f980006cab0 2023-07-06T03:53:43.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.928+0000 7f98073ff700 1 -- 172.21.15.97:0/520724051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98000b31e0 con 0x7f980006cab0 2023-07-06T03:53:43.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.929+0000 7f98073ff700 1 -- 172.21.15.97:0/520724051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9800146c40 con 0x7f980006cab0 2023-07-06T03:53:43.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.930+0000 7f98073ff700 1 -- 172.21.15.97:0/520724051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9800146dd0 con 0x7f980006cab0 2023-07-06T03:53:43.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.930+0000 7f97f5ffb700 1 -- 172.21.15.97:0/520724051 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97fc0193f0 con 0x7f980006cab0 2023-07-06T03:53:43.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.931+0000 7f97f5ffb700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97e406ace0 0x7f97e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:43.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.931+0000 7f97f5ffb700 1 -- 172.21.15.97:0/520724051 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97fc08b500 con 0x7f980006cab0 2023-07-06T03:53:43.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.932+0000 7f980499a700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97e406ace0 0x7f97e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:43.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.932+0000 7f980499a700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97e406ace0 0x7f97e406d190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9800070f20 tx=0x7f97f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:43.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:43.936+0000 7f97f5ffb700 1 -- 172.21.15.97:0/520724051 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f97fc0565a0 con 0x7f980006cab0 2023-07-06T03:53:44.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.131+0000 7f98073ff700 1 -- 172.21.15.97:0/520724051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7f980004e340 con 0x7f980006cab0 2023-07-06T03:53:44.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.137+0000 7f97f5ffb700 1 -- 172.21.15.97:0/520724051 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v37)=0 v37) v1 ==== 128+0+0 (secure 0 0 0) 0x7f97fc059bc0 con 0x7f980006cab0 2023-07-06T03:53:44.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 -- 172.21.15.97:0/520724051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97e406ace0 msgr2=0x7f97e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97e406ace0 0x7f97e406d190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9800070f20 tx=0x7f97f0009040 comp rx=0 tx=0).stop 2023-07-06T03:53:44.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 -- 172.21.15.97:0/520724051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980006cab0 msgr2=0x7f98001497c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980006cab0 0x7f98001497c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00bb10 tx=0x7f97fc004750 comp rx=0 tx=0).stop 2023-07-06T03:53:44.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 -- 172.21.15.97:0/520724051 shutdown_connections 2023-07-06T03:53:44.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f97e406ace0 0x7f97e406d190 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9800158b20 0x7f98000ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980006cab0 0x7f98001497c0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 --2- 172.21.15.97:0/520724051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98000b0560 0x7f98000c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 -- 172.21.15.97:0/520724051 >> 172.21.15.97:0/520724051 conn(0x7f980006a060 msgr2=0x7f980006a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:44.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 -- 172.21.15.97:0/520724051 shutdown_connections 2023-07-06T03:53:44.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.139+0000 7f98073ff700 1 -- 172.21.15.97:0/520724051 wait complete. 2023-07-06T03:53:44.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-07-06T03:53:44.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.355+0000 7fcbd01ff700 1 Processor -- start 2023-07-06T03:53:44.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.355+0000 7fcbd01ff700 1 -- start start 2023-07-06T03:53:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.356+0000 7fcbd01ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcbc80c3cd0 0x7fcbc80cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.356+0000 7fcbd01ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcbc8159440 0x7fcbc80cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.356+0000 7fcbd01ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc80ca260 con 0x7fcbc8159070 2023-07-06T03:53:44.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.356+0000 7fcbd01ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc80c99c0 con 0x7fcbc8159440 2023-07-06T03:53:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.356+0000 7fcbd01ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc80c9ca0 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.357+0000 7fcbce79c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcbc8159440 0x7fcbc80cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.357+0000 7fcbcdf9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcbc8159070 0x7fcbc80c8f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55420/0 (socket says 172.21.15.97:55420) 2023-07-06T03:53:44.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.357+0000 7fcbcdf9b700 1 -- 172.21.15.97:0/2248165174 learned_addr learned my addr 172.21.15.97:0/2248165174 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.357+0000 7fcbcd79a700 1 --2- 172.21.15.97:0/2248165174 >> v2:172.21.15.146:3300/0 conn(0x7fcbc80c3cd0 0x7fcbc80cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.357+0000 7fcbce79c700 1 -- 172.21.15.97:0/2248165174 >> v2:172.21.15.146:3300/0 conn(0x7fcbc80c3cd0 msgr2=0x7fcbc80cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.357+0000 7fcbce79c700 1 --2- 172.21.15.97:0/2248165174 >> v2:172.21.15.146:3300/0 conn(0x7fcbc80c3cd0 0x7fcbc80cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.357+0000 7fcbce79c700 1 -- 172.21.15.97:0/2248165174 >> v1:172.21.15.97:6789/0 conn(0x7fcbc8159070 legacy=0x7fcbc80c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.357+0000 7fcbce79c700 1 -- 172.21.15.97:0/2248165174 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc80c9f80 con 0x7fcbc8159440 2023-07-06T03:53:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.357+0000 7fcbccf99700 1 -- 172.21.15.97:0/2248165174 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991611173 0 0) 0x7fcbc80ca260 con 0x7fcbc8159070 2023-07-06T03:53:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.357+0000 7fcbce79c700 1 --2- 172.21.15.97:0/2248165174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbc8159440 0x7fcbc80cca00 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80cf9f0 tx=0x7fcbc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f865204d8ceb39f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.357+0000 7fcbccf99700 1 -- 172.21.15.97:0/2248165174 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000e070 con 0x7fcbc8159440 2023-07-06T03:53:44.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.358+0000 7fcbccf99700 1 -- 172.21.15.97:0/2248165174 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc000bbd0 con 0x7fcbc8159440 2023-07-06T03:53:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.358+0000 7fcbccf99700 1 -- 172.21.15.97:0/2248165174 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0012430 con 0x7fcbc8159440 2023-07-06T03:53:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.358+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2248165174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbc8159440 msgr2=0x7fcbc80cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.358+0000 7fcbd01ff700 1 --2- 172.21.15.97:0/2248165174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbc8159440 0x7fcbc80cca00 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80cf9f0 tx=0x7fcbc000b330 comp rx=0 tx=0).stop 2023-07-06T03:53:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.358+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2248165174 shutdown_connections 2023-07-06T03:53:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.358+0000 7fcbd01ff700 1 --2- 172.21.15.97:0/2248165174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbc8159440 0x7fcbc80cca00 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.358+0000 7fcbd01ff700 1 --2- 172.21.15.97:0/2248165174 >> v2:172.21.15.146:3300/0 conn(0x7fcbc80c3cd0 0x7fcbc80cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.358+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2248165174 >> 172.21.15.97:0/2248165174 conn(0x7fcbc806a060 msgr2=0x7fcbc806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.358+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2248165174 shutdown_connections 2023-07-06T03:53:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.358+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2248165174 wait complete. 2023-07-06T03:53:44.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.360+0000 7fcbd01ff700 1 Processor -- start 2023-07-06T03:53:44.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.360+0000 7fcbd01ff700 1 -- start start 2023-07-06T03:53:44.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.361+0000 7fcbd01ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbc80c3cd0 0x7fcbc80b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.361+0000 7fcbd01ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc8159070 0x7fcbc80cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.361+0000 7fcbd01ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbc8159440 0x7fcbc80cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.361+0000 7fcbd01ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b1d70 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.361+0000 7fcbd01ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b1fc0 con 0x7fcbc8159440 2023-07-06T03:53:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.361+0000 7fcbd01ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b22a0 con 0x7fcbc8159070 2023-07-06T03:53:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.361+0000 7fcbcdf9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbc80c3cd0 0x7fcbc80b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.361+0000 7fcbcdf9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbc80c3cd0 0x7fcbc80b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35350/0 (socket says 172.21.15.97:35350) 2023-07-06T03:53:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.361+0000 7fcbcdf9b700 1 -- 172.21.15.97:0/2296623022 learned_addr learned my addr 172.21.15.97:0/2296623022 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:44.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.361+0000 7fcbce79c700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbc8159440 0x7fcbc80cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.361+0000 7fcbcd79a700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc8159070 0x7fcbc80cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.362+0000 7fcbcdf9b700 1 -- 172.21.15.97:0/2296623022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc8159070 msgr2=0x7fcbc80cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.362+0000 7fcbcdf9b700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc8159070 0x7fcbc80cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.362+0000 7fcbcdf9b700 1 -- 172.21.15.97:0/2296623022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbc8159440 msgr2=0x7fcbc80cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.362+0000 7fcbcdf9b700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbc8159440 0x7fcbc80cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.362+0000 7fcbcdf9b700 1 -- 172.21.15.97:0/2296623022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc0009a70 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.362+0000 7fcbcdf9b700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbc80c3cd0 0x7fcbc80b0b70 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40011e0 tx=0x7fcbc400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:44.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.362+0000 7fcbbe7fc700 1 -- 172.21.15.97:0/2296623022 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4004c80 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.362+0000 7fcbbe7fc700 1 -- 172.21.15.97:0/2296623022 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc4004e00 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.362+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2296623022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc80b2580 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.362+0000 7fcbbe7fc700 1 -- 172.21.15.97:0/2296623022 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4007310 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.362+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2296623022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc80b10b0 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.363+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2296623022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc804e340 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.364+0000 7fcbbe7fc700 1 -- 172.21.15.97:0/2296623022 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcbc4007470 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.366+0000 7fcbbe7fc700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbac06ace0 0x7fcbac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.366+0000 7fcbbe7fc700 1 -- 172.21.15.97:0/2296623022 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbc40892a0 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.366+0000 7fcbcd79a700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbac06ace0 0x7fcbac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.367+0000 7fcbcd79a700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbac06ace0 0x7fcbac06d190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80c40d0 tx=0x7fcbb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:44.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.370+0000 7fcbbe7fc700 1 -- 172.21.15.97:0/2296623022 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcbc40542b0 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.567+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2296623022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7fcbc806d220 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.568+0000 7fcbbe7fc700 1 -- 172.21.15.97:0/2296623022 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v37) v1 ==== 106+0+4 (secure 0 0 0) 0x7fcbc40578d0 con 0x7fcbc80c3cd0 2023-07-06T03:53:44.570 INFO:teuthology.orchestra.run.smithi097.stdout:100 2023-07-06T03:53:44.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2296623022 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbac06ace0 msgr2=0x7fcbac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbac06ace0 0x7fcbac06d190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80c40d0 tx=0x7fcbb8008040 comp rx=0 tx=0).stop 2023-07-06T03:53:44.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2296623022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbc80c3cd0 msgr2=0x7fcbc80b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbc80c3cd0 0x7fcbc80b0b70 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40011e0 tx=0x7fcbc400cae0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2296623022 shutdown_connections 2023-07-06T03:53:44.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcbac06ace0 0x7fcbac06d190 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcbc8159440 0x7fcbc80cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbc80c3cd0 0x7fcbc80b0b70 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 --2- 172.21.15.97:0/2296623022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcbc8159070 0x7fcbc80cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2296623022 >> 172.21.15.97:0/2296623022 conn(0x7fcbc806a060 msgr2=0x7fcbc8070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:44.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2296623022 shutdown_connections 2023-07-06T03:53:44.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.570+0000 7fcbd01ff700 1 -- 172.21.15.97:0/2296623022 wait complete. 2023-07-06T03:53:44.589 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2023-07-06T03:53:44.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.787+0000 7f105a1ff700 1 Processor -- start 2023-07-06T03:53:44.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.787+0000 7f105a1ff700 1 -- start start 2023-07-06T03:53:44.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.788+0000 7f105a1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f105415c610 0x7f10540b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.788+0000 7f105a1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f105406b360 0x7f10540b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.788+0000 7f105a1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10540cff90 con 0x7f10540d2210 2023-07-06T03:53:44.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f105a1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10540d0780 con 0x7f105406b360 2023-07-06T03:53:44.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f105a1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105415c9e0 con 0x7f105415c610 2023-07-06T03:53:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f1052ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f105406b360 0x7f10540b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f1052ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f105406b360 0x7f10540b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35362/0 (socket says 172.21.15.97:35362) 2023-07-06T03:53:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f1052ffd700 1 -- 172.21.15.97:0/2828019268 learned_addr learned my addr 172.21.15.97:0/2828019268 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f10537fe700 1 --2- 172.21.15.97:0/2828019268 >> v2:172.21.15.146:3300/0 conn(0x7f105415c610 0x7f10540b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f1052ffd700 1 -- 172.21.15.97:0/2828019268 >> v2:172.21.15.146:3300/0 conn(0x7f105415c610 msgr2=0x7f10540b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:44.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f1052ffd700 1 --2- 172.21.15.97:0/2828019268 >> v2:172.21.15.146:3300/0 conn(0x7f105415c610 0x7f10540b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f1052ffd700 1 -- 172.21.15.97:0/2828019268 >> v1:172.21.15.97:6789/0 conn(0x7f10540d2210 legacy=0x7f10540d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f1052ffd700 1 -- 172.21.15.97:0/2828019268 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10540b5650 con 0x7f105406b360 2023-07-06T03:53:44.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f10527fc700 1 -- 172.21.15.97:0/2828019268 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411681715 0 0) 0x7f10540cff90 con 0x7f10540d2210 2023-07-06T03:53:44.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.789+0000 7f1052ffd700 1 --2- 172.21.15.97:0/2828019268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f105406b360 0x7f10540b4cf0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f105408b000 tx=0x7f1048009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2820d9426bea9a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:44.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.790+0000 7f10527fc700 1 -- 172.21.15.97:0/2828019268 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104800b860 con 0x7f105406b360 2023-07-06T03:53:44.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.790+0000 7f10527fc700 1 -- 172.21.15.97:0/2828019268 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104800b9c0 con 0x7f105406b360 2023-07-06T03:53:44.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.790+0000 7f10527fc700 1 -- 172.21.15.97:0/2828019268 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104800bd50 con 0x7f105406b360 2023-07-06T03:53:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.790+0000 7f105a1ff700 1 -- 172.21.15.97:0/2828019268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f105406b360 msgr2=0x7f10540b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.790+0000 7f105a1ff700 1 --2- 172.21.15.97:0/2828019268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f105406b360 0x7f10540b4cf0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f105408b000 tx=0x7f1048009c00 comp rx=0 tx=0).stop 2023-07-06T03:53:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.790+0000 7f105a1ff700 1 -- 172.21.15.97:0/2828019268 shutdown_connections 2023-07-06T03:53:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.790+0000 7f105a1ff700 1 --2- 172.21.15.97:0/2828019268 >> v2:172.21.15.146:3300/0 conn(0x7f105415c610 0x7f10540b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.790+0000 7f105a1ff700 1 --2- 172.21.15.97:0/2828019268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f105406b360 0x7f10540b4cf0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.790+0000 7f105a1ff700 1 -- 172.21.15.97:0/2828019268 >> 172.21.15.97:0/2828019268 conn(0x7f105406a6c0 msgr2=0x7f105406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.790+0000 7f105a1ff700 1 -- 172.21.15.97:0/2828019268 shutdown_connections 2023-07-06T03:53:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.790+0000 7f105a1ff700 1 -- 172.21.15.97:0/2828019268 wait complete. 2023-07-06T03:53:44.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.793+0000 7f105a1ff700 1 Processor -- start 2023-07-06T03:53:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.793+0000 7f105a1ff700 1 -- start start 2023-07-06T03:53:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.794+0000 7f105a1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f105406b360 0x7f105415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.794+0000 7f105a1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540d2210 0x7f10540b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.794+0000 7f105a1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f105415c610 0x7f10540b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.794+0000 7f105a1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10541f5540 con 0x7f105406b360 2023-07-06T03:53:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.794+0000 7f105a1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10541f5790 con 0x7f10540d2210 2023-07-06T03:53:44.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.794+0000 7f105a1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10541f5a70 con 0x7f105415c610 2023-07-06T03:53:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.794+0000 7f10537fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f105406b360 0x7f105415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.794+0000 7f10537fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f105406b360 0x7f105415cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35366/0 (socket says 172.21.15.97:35366) 2023-07-06T03:53:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.794+0000 7f10537fe700 1 -- 172.21.15.97:0/1893360672 learned_addr learned my addr 172.21.15.97:0/1893360672 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f1052ffd700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540d2210 0x7f10540b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f1053fff700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f105415c610 0x7f10540b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f10537fe700 1 -- 172.21.15.97:0/1893360672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f105415c610 msgr2=0x7f10540b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f10537fe700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f105415c610 0x7f10540b7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f10537fe700 1 -- 172.21.15.97:0/1893360672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540d2210 msgr2=0x7f10540b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f10537fe700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540d2210 0x7f10540b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f10537fe700 1 -- 172.21.15.97:0/1893360672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1048009550 con 0x7f105406b360 2023-07-06T03:53:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f10537fe700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f105406b360 0x7f105415cb20 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f104400c7f0 tx=0x7f104400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:44.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f103bfff700 1 -- 172.21.15.97:0/1893360672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044011800 con 0x7f105406b360 2023-07-06T03:53:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f103bfff700 1 -- 172.21.15.97:0/1893360672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1044011960 con 0x7f105406b360 2023-07-06T03:53:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f105a1ff700 1 -- 172.21.15.97:0/1893360672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10541f5d50 con 0x7f105406b360 2023-07-06T03:53:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f103bfff700 1 -- 172.21.15.97:0/1893360672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044011cf0 con 0x7f105406b360 2023-07-06T03:53:44.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.795+0000 7f105a1ff700 1 -- 172.21.15.97:0/1893360672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10540cb320 con 0x7f105406b360 2023-07-06T03:53:44.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.796+0000 7f105a1ff700 1 -- 172.21.15.97:0/1893360672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10541f6270 con 0x7f105406b360 2023-07-06T03:53:44.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.797+0000 7f103bfff700 1 -- 172.21.15.97:0/1893360672 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1044018360 con 0x7f105406b360 2023-07-06T03:53:44.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.798+0000 7f103bfff700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f104006ace0 0x7f104006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:44.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.798+0000 7f103bfff700 1 -- 172.21.15.97:0/1893360672 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f10440893d0 con 0x7f105406b360 2023-07-06T03:53:44.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.798+0000 7f1052ffd700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f104006ace0 0x7f104006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:44.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.799+0000 7f1052ffd700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f104006ace0 0x7f104006d190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1048009870 tx=0x7f1048008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:44.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.802+0000 7f103bfff700 1 -- 172.21.15.97:0/1893360672 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f10541f6270 con 0x7f105406b360 2023-07-06T03:53:45.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:44.999+0000 7f105a1ff700 1 -- 172.21.15.97:0/1893360672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7f10541f6270 con 0x7f105406b360 2023-07-06T03:53:45.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.003+0000 7f103bfff700 1 -- 172.21.15.97:0/1893360672 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v38)=0 v38) v1 ==== 124+0+0 (secure 0 0 0) 0x7f10541f6270 con 0x7f105406b360 2023-07-06T03:53:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.005+0000 7f105a1ff700 1 -- 172.21.15.97:0/1893360672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f104006ace0 msgr2=0x7f104006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.005+0000 7f105a1ff700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f104006ace0 0x7f104006d190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1048009870 tx=0x7f1048008030 comp rx=0 tx=0).stop 2023-07-06T03:53:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.005+0000 7f105a1ff700 1 -- 172.21.15.97:0/1893360672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f105406b360 msgr2=0x7f105415cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.005+0000 7f105a1ff700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f105406b360 0x7f105415cb20 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f104400c7f0 tx=0x7f104400cbb0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.005+0000 7f105a1ff700 1 -- 172.21.15.97:0/1893360672 shutdown_connections 2023-07-06T03:53:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.005+0000 7f105a1ff700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f104006ace0 0x7f104006d190 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.005+0000 7f105a1ff700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f105415c610 0x7f10540b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.005+0000 7f105a1ff700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f105406b360 0x7f105415cb20 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.005+0000 7f105a1ff700 1 --2- 172.21.15.97:0/1893360672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10540d2210 0x7f10540b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.006+0000 7f105a1ff700 1 -- 172.21.15.97:0/1893360672 >> 172.21.15.97:0/1893360672 conn(0x7f105406a6c0 msgr2=0x7f105415b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.006+0000 7f105a1ff700 1 -- 172.21.15.97:0/1893360672 shutdown_connections 2023-07-06T03:53:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.006+0000 7f105a1ff700 1 -- 172.21.15.97:0/1893360672 wait complete. 2023-07-06T03:53:45.025 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2023-07-06T03:53:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.220+0000 7fbcddbff700 1 Processor -- start 2023-07-06T03:53:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.220+0000 7fbcddbff700 1 -- start start 2023-07-06T03:53:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.222+0000 7fbcddbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcd80d2210 0x7fbcd80b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.222+0000 7fbcddbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd80cff90 con 0x7fbcd815c610 2023-07-06T03:53:45.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.222+0000 7fbcddbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd80b5650 con 0x7fbcd806b360 2023-07-06T03:53:45.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.222+0000 7fbcddbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd80d0780 con 0x7fbcd80d2210 2023-07-06T03:53:45.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.222+0000 7fbcd77fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbcd815c610 0x7fbcd80b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55444/0 (socket says 172.21.15.97:55444) 2023-07-06T03:53:45.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.222+0000 7fbcd77fe700 1 -- 172.21.15.97:0/3233782638 learned_addr learned my addr 172.21.15.97:0/3233782638 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:45.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.222+0000 7fbcd7fff700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcd80d2210 0x7fbcd80b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.222+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 788675607 0 0) 0x7fbcd80cff90 con 0x7fbcd815c610 2023-07-06T03:53:45.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.222+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcc00035b0 con 0x7fbcd815c610 2023-07-06T03:53:45.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.222+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295569034 0 0) 0x7fbcd80b5650 con 0x7fbcd806b360 2023-07-06T03:53:45.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.222+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcd80cff90 con 0x7fbcd806b360 2023-07-06T03:53:45.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1108240714 0 0) 0x7fbcc00035b0 con 0x7fbcd815c610 2023-07-06T03:53:45.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcd80d2210 msgr2=0x7fbcd80b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:45.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcd80d2210 0x7fbcd80b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 >> v1:172.21.15.146:6789/0 conn(0x7fbcd806b360 legacy=0x7fbcd80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd80b5400 con 0x7fbcd815c610 2023-07-06T03:53:45.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fbcc8001670 con 0x7fbcd815c610 2023-07-06T03:53:45.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 >> v1:172.21.15.97:6789/0 conn(0x7fbcd815c610 legacy=0x7fbcd80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcc0003bc0 0x7fbcc0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcc0004850 0x7fbcc0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcd80d2210 0x7fbcc0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd80d0780 con 0x7fbcc0004850 2023-07-06T03:53:45.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc000bac0 con 0x7fbcc0003bc0 2023-07-06T03:53:45.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc000bc30 con 0x7fbcd80d2210 2023-07-06T03:53:45.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd6ffd700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcc0004850 0x7fbcc0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd77fe700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcc0003bc0 0x7fbcc0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd7fff700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcd80d2210 0x7fbcc0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd6ffd700 1 -- 172.21.15.97:0/3233782638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcd80d2210 msgr2=0x7fbcc0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:45.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd6ffd700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcd80d2210 0x7fbcc0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd6ffd700 1 -- 172.21.15.97:0/3233782638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcc0003bc0 msgr2=0x7fbcc0003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:45.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd6ffd700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcc0003bc0 0x7fbcc0003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.223+0000 7fbcd6ffd700 1 -- 172.21.15.97:0/3233782638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc00035b0 con 0x7fbcc0004850 2023-07-06T03:53:45.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.224+0000 7fbcd6ffd700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcc0004850 0x7fbcc0004cc0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fbccc001940 tx=0x7fbcc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624318eabd26e052 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:45.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.224+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc001430 con 0x7fbcc0004850 2023-07-06T03:53:45.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.224+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc001d70 con 0x7fbcc0004850 2023-07-06T03:53:45.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.224+0000 7fbcd67fc700 1 -- 172.21.15.97:0/3233782638 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc005510 con 0x7fbcc0004850 2023-07-06T03:53:45.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.224+0000 7fbcddbff700 1 -- 172.21.15.97:0/3233782638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcc0004850 msgr2=0x7fbcc0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.224+0000 7fbcddbff700 1 --2- 172.21.15.97:0/3233782638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcc0004850 0x7fbcc0004cc0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fbccc001940 tx=0x7fbcc0005880 comp rx=0 tx=0).stop 2023-07-06T03:53:45.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.224+0000 7fbcd7fff700 1 -- 172.21.15.97:0/3233782638 reap_dead start 2023-07-06T03:53:45.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.224+0000 7fbcddbff700 1 -- 172.21.15.97:0/3233782638 shutdown_connections 2023-07-06T03:53:45.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.224+0000 7fbcddbff700 1 -- 172.21.15.97:0/3233782638 >> 172.21.15.97:0/3233782638 conn(0x7fbcd806a6c0 msgr2=0x7fbcd806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:45.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.224+0000 7fbcddbff700 1 -- 172.21.15.97:0/3233782638 shutdown_connections 2023-07-06T03:53:45.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.224+0000 7fbcddbff700 1 -- 172.21.15.97:0/3233782638 wait complete. 2023-07-06T03:53:45.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.226+0000 7fbcddbff700 1 Processor -- start 2023-07-06T03:53:45.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.226+0000 7fbcddbff700 1 -- start start 2023-07-06T03:53:45.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.227+0000 7fbcddbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcc0004850 0x7fbcd814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.227+0000 7fbcddbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcd8150100 0x7fbcd815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.227+0000 7fbcddbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcd8150570 0x7fbcd8159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.227+0000 7fbcddbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd80cca60 con 0x7fbcd8150570 2023-07-06T03:53:45.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.227+0000 7fbcddbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd814feb0 con 0x7fbcd8150100 2023-07-06T03:53:45.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.227+0000 7fbcddbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd815de50 con 0x7fbcc0004850 2023-07-06T03:53:45.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.227+0000 7fbcd7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcd8150570 0x7fbcd8159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.228+0000 7fbcd7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcd8150570 0x7fbcd8159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35384/0 (socket says 172.21.15.97:35384) 2023-07-06T03:53:45.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.228+0000 7fbcd7fff700 1 -- 172.21.15.97:0/2158241143 learned_addr learned my addr 172.21.15.97:0/2158241143 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:45.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.228+0000 7fbcd6ffd700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcd8150100 0x7fbcd815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.228+0000 7fbcd77fe700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcc0004850 0x7fbcd814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.228+0000 7fbcd7fff700 1 -- 172.21.15.97:0/2158241143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcc0004850 msgr2=0x7fbcd814f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.228+0000 7fbcd7fff700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcc0004850 0x7fbcd814f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.228+0000 7fbcd7fff700 1 -- 172.21.15.97:0/2158241143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcd8150100 msgr2=0x7fbcd815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.228+0000 7fbcd7fff700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcd8150100 0x7fbcd815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.228+0000 7fbcd7fff700 1 -- 172.21.15.97:0/2158241143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd815e130 con 0x7fbcd8150570 2023-07-06T03:53:45.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.228+0000 7fbcd6ffd700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcd8150100 0x7fbcd815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:45.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.228+0000 7fbcd7fff700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcd8150570 0x7fbcd8159270 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80cea10 tx=0x7fbcd00114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:45.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.229+0000 7fbcbffff700 1 -- 172.21.15.97:0/2158241143 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000e070 con 0x7fbcd8150570 2023-07-06T03:53:45.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.229+0000 7fbcbffff700 1 -- 172.21.15.97:0/2158241143 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd0003cf0 con 0x7fbcd8150570 2023-07-06T03:53:45.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.229+0000 7fbcbffff700 1 -- 172.21.15.97:0/2158241143 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0003ea0 con 0x7fbcd8150570 2023-07-06T03:53:45.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.229+0000 7fbcddbff700 1 -- 172.21.15.97:0/2158241143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd815e410 con 0x7fbcd8150570 2023-07-06T03:53:45.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.229+0000 7fbcddbff700 1 -- 172.21.15.97:0/2158241143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd815e9d0 con 0x7fbcd8150570 2023-07-06T03:53:45.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.230+0000 7fbcddbff700 1 -- 172.21.15.97:0/2158241143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd815ebf0 con 0x7fbcd8150570 2023-07-06T03:53:45.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.231+0000 7fbcbffff700 1 -- 172.21.15.97:0/2158241143 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbcd0011a10 con 0x7fbcd8150570 2023-07-06T03:53:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.232+0000 7fbcbffff700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbcc406ace0 0x7fbcc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.232+0000 7fbcbffff700 1 -- 172.21.15.97:0/2158241143 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbcd0089b30 con 0x7fbcd8150570 2023-07-06T03:53:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.232+0000 7fbcd77fe700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbcc406ace0 0x7fbcc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.232+0000 7fbcd77fe700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbcc406ace0 0x7fbcc406d190 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8001eb0 tx=0x7fbcc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.236+0000 7fbcbffff700 1 -- 172.21.15.97:0/2158241143 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbcd0054b40 con 0x7fbcd8150570 2023-07-06T03:53:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.433+0000 7fbcddbff700 1 -- 172.21.15.97:0/2158241143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) v1 -- 0x7fbcd804e340 con 0x7fbcd8150570 2023-07-06T03:53:45.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.434+0000 7fbcbffff700 1 -- 172.21.15.97:0/2158241143 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v38) v1 ==== 106+0+8 (secure 0 0 0) 0x7fbcd0058160 con 0x7fbcd8150570 2023-07-06T03:53:45.436 INFO:teuthology.orchestra.run.smithi097.stdout:1048576 2023-07-06T03:53:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 -- 172.21.15.97:0/2158241143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbcc406ace0 msgr2=0x7fbcc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbcc406ace0 0x7fbcc406d190 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8001eb0 tx=0x7fbcc8009040 comp rx=0 tx=0).stop 2023-07-06T03:53:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 -- 172.21.15.97:0/2158241143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcd8150570 msgr2=0x7fbcd8159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcd8150570 0x7fbcd8159270 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80cea10 tx=0x7fbcd00114b0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 -- 172.21.15.97:0/2158241143 shutdown_connections 2023-07-06T03:53:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbcc406ace0 0x7fbcc406d190 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcd8150570 0x7fbcd8159270 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbcc0004850 0x7fbcd814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 --2- 172.21.15.97:0/2158241143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcd8150100 0x7fbcd815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 -- 172.21.15.97:0/2158241143 >> 172.21.15.97:0/2158241143 conn(0x7fbcd806a6c0 msgr2=0x7fbcd80b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 -- 172.21.15.97:0/2158241143 shutdown_connections 2023-07-06T03:53:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.436+0000 7fbcddbff700 1 -- 172.21.15.97:0/2158241143 wait complete. 2023-07-06T03:53:45.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-07-06T03:53:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.649+0000 7ffa777ff700 1 Processor -- start 2023-07-06T03:53:45.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.650+0000 7ffa777ff700 1 -- start start 2023-07-06T03:53:45.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa777ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa700d15a0 0x7ffa700cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa777ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa700cfed0 0x7ffa700b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa777ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa700d2d30 con 0x7ffa700b4a40 2023-07-06T03:53:45.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa777ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa700b6520 con 0x7ffa700cfed0 2023-07-06T03:53:45.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa777ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700d1a10 con 0x7ffa700d15a0 2023-07-06T03:53:45.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa75d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa700cfed0 0x7ffa700b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa75d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa700cfed0 0x7ffa700b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35394/0 (socket says 172.21.15.97:35394) 2023-07-06T03:53:45.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa75d9c700 1 -- 172.21.15.97:0/1385618666 learned_addr learned my addr 172.21.15.97:0/1385618666 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:45.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa74d9a700 1 --2- 172.21.15.97:0/1385618666 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa700d15a0 0x7ffa700cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa75d9c700 1 -- 172.21.15.97:0/1385618666 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa700d15a0 msgr2=0x7ffa700cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:45.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa75d9c700 1 --2- 172.21.15.97:0/1385618666 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa700d15a0 0x7ffa700cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa75d9c700 1 -- 172.21.15.97:0/1385618666 >> v1:172.21.15.146:6789/0 conn(0x7ffa700b4a40 legacy=0x7ffa700b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.651+0000 7ffa75d9c700 1 -- 172.21.15.97:0/1385618666 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700cf210 con 0x7ffa700cfed0 2023-07-06T03:53:45.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.652+0000 7ffa75d9c700 1 --2- 172.21.15.97:0/1385618666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa700cfed0 0x7ffa700b4e10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009e50 tx=0x7ffa6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87aea0fd94de987f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.652+0000 7ffa67fff700 1 -- 172.21.15.97:0/1385618666 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800e070 con 0x7ffa700cfed0 2023-07-06T03:53:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.652+0000 7ffa67fff700 1 -- 172.21.15.97:0/1385618666 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6800b9c0 con 0x7ffa700cfed0 2023-07-06T03:53:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.652+0000 7ffa67fff700 1 -- 172.21.15.97:0/1385618666 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800bd70 con 0x7ffa700cfed0 2023-07-06T03:53:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.652+0000 7ffa777ff700 1 -- 172.21.15.97:0/1385618666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa700cfed0 msgr2=0x7ffa700b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.652+0000 7ffa777ff700 1 --2- 172.21.15.97:0/1385618666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa700cfed0 0x7ffa700b4e10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009e50 tx=0x7ffa6800b330 comp rx=0 tx=0).stop 2023-07-06T03:53:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.653+0000 7ffa777ff700 1 -- 172.21.15.97:0/1385618666 shutdown_connections 2023-07-06T03:53:45.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.653+0000 7ffa777ff700 1 --2- 172.21.15.97:0/1385618666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa700cfed0 0x7ffa700b4e10 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.653+0000 7ffa777ff700 1 --2- 172.21.15.97:0/1385618666 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa700d15a0 0x7ffa700cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.653+0000 7ffa777ff700 1 -- 172.21.15.97:0/1385618666 >> 172.21.15.97:0/1385618666 conn(0x7ffa7006a900 msgr2=0x7ffa7006acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.653+0000 7ffa777ff700 1 -- 172.21.15.97:0/1385618666 shutdown_connections 2023-07-06T03:53:45.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.653+0000 7ffa777ff700 1 -- 172.21.15.97:0/1385618666 wait complete. 2023-07-06T03:53:45.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.654+0000 7ffa777ff700 1 Processor -- start 2023-07-06T03:53:45.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.654+0000 7ffa777ff700 1 -- start start 2023-07-06T03:53:45.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.655+0000 7ffa777ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa700b4a40 0x7ffa701559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.655+0000 7ffa777ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa700cfed0 0x7ffa700c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.655+0000 7ffa777ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa700d15a0 0x7ffa700b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.655+0000 7ffa777ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7015cad0 con 0x7ffa700cfed0 2023-07-06T03:53:45.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.655+0000 7ffa777ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700ccf90 con 0x7ffa700b4a40 2023-07-06T03:53:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.655+0000 7ffa777ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700c6c90 con 0x7ffa700d15a0 2023-07-06T03:53:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa74d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa700cfed0 0x7ffa700c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa74d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa700cfed0 0x7ffa700c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35400/0 (socket says 172.21.15.97:35400) 2023-07-06T03:53:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa74d9a700 1 -- 172.21.15.97:0/2734057817 learned_addr learned my addr 172.21.15.97:0/2734057817 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa7559b700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa700b4a40 0x7ffa701559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa75d9c700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa700d15a0 0x7ffa700b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa74d9a700 1 -- 172.21.15.97:0/2734057817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa700d15a0 msgr2=0x7ffa700b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa74d9a700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa700d15a0 0x7ffa700b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa74d9a700 1 -- 172.21.15.97:0/2734057817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa700b4a40 msgr2=0x7ffa701559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa74d9a700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa700b4a40 0x7ffa701559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa74d9a700 1 -- 172.21.15.97:0/2734057817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa68009b30 con 0x7ffa700cfed0 2023-07-06T03:53:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa7559b700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa700b4a40 0x7ffa701559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa75d9c700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa700d15a0 0x7ffa700b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa74d9a700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa700cfed0 0x7ffa700c9470 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000e810 tx=0x7ffa6000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:45.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.656+0000 7ffa65ffb700 1 -- 172.21.15.97:0/2734057817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60013610 con 0x7ffa700cfed0 2023-07-06T03:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.657+0000 7ffa65ffb700 1 -- 172.21.15.97:0/2734057817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60013770 con 0x7ffa700cfed0 2023-07-06T03:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.657+0000 7ffa65ffb700 1 -- 172.21.15.97:0/2734057817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60013b20 con 0x7ffa700cfed0 2023-07-06T03:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.657+0000 7ffa777ff700 1 -- 172.21.15.97:0/2734057817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa700c6f70 con 0x7ffa700cfed0 2023-07-06T03:53:45.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.657+0000 7ffa777ff700 1 -- 172.21.15.97:0/2734057817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa700c99b0 con 0x7ffa700cfed0 2023-07-06T03:53:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.658+0000 7ffa777ff700 1 -- 172.21.15.97:0/2734057817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa700c6af0 con 0x7ffa700cfed0 2023-07-06T03:53:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.659+0000 7ffa65ffb700 1 -- 172.21.15.97:0/2734057817 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa60013c80 con 0x7ffa700cfed0 2023-07-06T03:53:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.660+0000 7ffa65ffb700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa5406ace0 0x7ffa5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.660+0000 7ffa65ffb700 1 -- 172.21.15.97:0/2734057817 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa600891d0 con 0x7ffa700cfed0 2023-07-06T03:53:45.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.660+0000 7ffa7559b700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa5406ace0 0x7ffa5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:45.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.661+0000 7ffa7559b700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa5406ace0 0x7ffa5406d190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000f60 tx=0x7ffa6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:45.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.664+0000 7ffa65ffb700 1 -- 172.21.15.97:0/2734057817 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffa700c6af0 con 0x7ffa700cfed0 2023-07-06T03:53:45.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.862+0000 7ffa777ff700 1 -- 172.21.15.97:0/2734057817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7ffa700c6af0 con 0x7ffa700cfed0 2023-07-06T03:53:45.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.862+0000 7ffa65ffb700 1 -- 172.21.15.97:0/2734057817 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v38) v1 ==== 112+0+10 (secure 0 0 0) 0x7ffa700c6af0 con 0x7ffa700cfed0 2023-07-06T03:53:45.864 INFO:teuthology.orchestra.run.smithi097.stdout:60.000000 2023-07-06T03:53:45.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.864+0000 7ffa777ff700 1 -- 172.21.15.97:0/2734057817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa5406ace0 msgr2=0x7ffa5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.864+0000 7ffa777ff700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa5406ace0 0x7ffa5406d190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000f60 tx=0x7ffa6c009040 comp rx=0 tx=0).stop 2023-07-06T03:53:45.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.864+0000 7ffa777ff700 1 -- 172.21.15.97:0/2734057817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa700cfed0 msgr2=0x7ffa700c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:45.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.864+0000 7ffa777ff700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa700cfed0 0x7ffa700c9470 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000e810 tx=0x7ffa6000ebd0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.865+0000 7ffa777ff700 1 -- 172.21.15.97:0/2734057817 shutdown_connections 2023-07-06T03:53:45.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.865+0000 7ffa777ff700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffa5406ace0 0x7ffa5406d190 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.865+0000 7ffa777ff700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffa700d15a0 0x7ffa700b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.865+0000 7ffa777ff700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa700b4a40 0x7ffa701559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.865+0000 7ffa777ff700 1 --2- 172.21.15.97:0/2734057817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa700cfed0 0x7ffa700c9470 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:45.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.865+0000 7ffa777ff700 1 -- 172.21.15.97:0/2734057817 >> 172.21.15.97:0/2734057817 conn(0x7ffa7006a900 msgr2=0x7ffa70160b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:45.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.865+0000 7ffa777ff700 1 -- 172.21.15.97:0/2734057817 shutdown_connections 2023-07-06T03:53:45.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:45.865+0000 7ffa777ff700 1 -- 172.21.15.97:0/2734057817 wait complete. 2023-07-06T03:53:45.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:53:45.885 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:53:45.885 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:53:45.885 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:53:45.885 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:53:45.886 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:45.886 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:53:45.886 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:53:45.887 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:53:45.887 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:53:45.887 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:45.887 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:53:45.888 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:53:45.888 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:53:45.888 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:53:45.889 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:45.889 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:53:45.889 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:53:45.889 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:53:45.890 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:53:45.890 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:53:45.890 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:53:45.891 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:53:45.891 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:53:45.891 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:45.891 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:53:45.892 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:53:46.035 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.154228.asok 2023-07-06T03:53:46.212 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.154228.asok 2023-07-06T03:53:46.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:53:46.213 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.154228.asok mds_sessions 2023-07-06T03:53:46.435 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:53:46.435 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8199, 2023-07-06T03:53:46.435 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:53:46.435 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:53:46.435 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:53:46.436 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 8199 2023-07-06T03:53:46.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:46.436 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:53:46.436 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:53:46.436 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:53:46.436 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2529580715 2023-07-06T03:53:46.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:46.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:46.437 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.8199 v1:192.168.0.1:0/2529580715", 2023-07-06T03:53:46.437 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "v1:192.168.0.1:0/2529580715", 2023-07-06T03:53:46.437 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions": [ 2023-07-06T03:53:46.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:46.437 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": 0, 2023-07-06T03:53:46.437 INFO:teuthology.orchestra.run.smithi097.stdout: "addrs": { 2023-07-06T03:53:46.437 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-07-06T03:53:46.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:46.438 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-07-06T03:53:46.438 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6834", 2023-07-06T03:53:46.438 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3324055785 2023-07-06T03:53:46.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:46.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:53:46.438 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:53:46.439 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6836", 2023-07-06T03:53:46.439 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3324055785 2023-07-06T03:53:46.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:46.439 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:53:46.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:53:46.439 INFO:teuthology.orchestra.run.smithi097.stdout: "seq": 0, 2023-07-06T03:53:46.439 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_gen": 0, 2023-07-06T03:53:46.439 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_ttl": "2023-07-06T03:54:40.770427+0000", 2023-07-06T03:53:46.440 INFO:teuthology.orchestra.run.smithi097.stdout: "last_cap_renew_request": "2023-07-06T03:53:40.770427+0000", 2023-07-06T03:53:46.440 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_renew_seq": 2, 2023-07-06T03:53:46.440 INFO:teuthology.orchestra.run.smithi097.stdout: "num_caps": 1, 2023-07-06T03:53:46.440 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "open" 2023-07-06T03:53:46.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:53:46.440 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-07-06T03:53:46.440 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 67 2023-07-06T03:53:46.440 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:53:46.449 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:53:46.449 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:53:46.495 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:53:46.496 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2023-07-06T03:53:46.496 DEBUG:teuthology.orchestra.run.smithi097:> import time 2023-07-06T03:53:46.496 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:53:46.497 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:46.497 DEBUG:teuthology.orchestra.run.smithi097:> n = 800 2023-07-06T03:53:46.497 DEBUG:teuthology.orchestra.run.smithi097:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2023-07-06T03:53:46.498 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:46.498 DEBUG:teuthology.orchestra.run.smithi097:> if not os.path.exists(abs_path): 2023-07-06T03:53:46.498 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(abs_path) 2023-07-06T03:53:46.498 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:46.499 DEBUG:teuthology.orchestra.run.smithi097:> handles = [] 2023-07-06T03:53:46.499 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(0, n): 2023-07-06T03:53:46.500 DEBUG:teuthology.orchestra.run.smithi097:> fname = "file_"+str(i) 2023-07-06T03:53:46.500 DEBUG:teuthology.orchestra.run.smithi097:> path = os.path.join(abs_path, fname) 2023-07-06T03:53:46.500 DEBUG:teuthology.orchestra.run.smithi097:> handles.append(open(path, '"'"'w'"'"')) 2023-07-06T03:53:46.500 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:53:46.501 DEBUG:teuthology.orchestra.run.smithi097:> while True: 2023-07-06T03:53:46.501 DEBUG:teuthology.orchestra.run.smithi097:> time.sleep(1) 2023-07-06T03:53:46.501 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:53:46.541 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:53:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.746+0000 7f56331ff700 1 Processor -- start 2023-07-06T03:53:46.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.746+0000 7f56331ff700 1 -- start start 2023-07-06T03:53:46.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.748+0000 7f56331ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f562c0b8410 0x7f562c167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:46.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.748+0000 7f56331ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c0cdae0 con 0x7f562c15c520 2023-07-06T03:53:46.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.748+0000 7f56331ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c0b6640 con 0x7f562c0cd390 2023-07-06T03:53:46.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.748+0000 7f56331ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562c0b6920 con 0x7f562c0b8410 2023-07-06T03:53:46.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.748+0000 7f5630f9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f562c15c520 0x7f562c160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55448/0 (socket says 172.21.15.97:55448) 2023-07-06T03:53:46.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.748+0000 7f5630f9b700 1 -- 172.21.15.97:0/2760331092 learned_addr learned my addr 172.21.15.97:0/2760331092 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:46.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.748+0000 7f563179c700 1 --2- 172.21.15.97:0/2760331092 >> v2:172.21.15.97:3300/0 conn(0x7f562c0b8410 0x7f562c167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:46.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.749+0000 7f563179c700 1 -- 172.21.15.97:0/2760331092 >> v1:172.21.15.146:6789/0 conn(0x7f562c0cd390 legacy=0x7f562c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.749+0000 7f563179c700 1 -- 172.21.15.97:0/2760331092 >> v1:172.21.15.97:6789/0 conn(0x7f562c15c520 legacy=0x7f562c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.749+0000 7f563179c700 1 -- 172.21.15.97:0/2760331092 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c0b6c00 con 0x7f562c0b8410 2023-07-06T03:53:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.749+0000 7f563179c700 1 --2- 172.21.15.97:0/2760331092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c0b8410 0x7f562c167510 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f5624009d90 tx=0x7f562400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88baeca92d594a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.749+0000 7f562b7fe700 1 -- 172.21.15.97:0/2760331092 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562400e070 con 0x7f562c0b8410 2023-07-06T03:53:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.749+0000 7f562b7fe700 1 -- 172.21.15.97:0/2760331092 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562400bad0 con 0x7f562c0b8410 2023-07-06T03:53:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.749+0000 7f562b7fe700 1 -- 172.21.15.97:0/2760331092 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562400be60 con 0x7f562c0b8410 2023-07-06T03:53:46.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.750+0000 7f56331ff700 1 -- 172.21.15.97:0/2760331092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c0b8410 msgr2=0x7f562c167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.750+0000 7f56331ff700 1 --2- 172.21.15.97:0/2760331092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c0b8410 0x7f562c167510 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f5624009d90 tx=0x7f562400b330 comp rx=0 tx=0).stop 2023-07-06T03:53:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.750+0000 7f56331ff700 1 -- 172.21.15.97:0/2760331092 shutdown_connections 2023-07-06T03:53:46.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.750+0000 7f56331ff700 1 --2- 172.21.15.97:0/2760331092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c0b8410 0x7f562c167510 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.750+0000 7f56331ff700 1 -- 172.21.15.97:0/2760331092 >> 172.21.15.97:0/2760331092 conn(0x7f562c071790 msgr2=0x7f562c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.750+0000 7f56331ff700 1 -- 172.21.15.97:0/2760331092 shutdown_connections 2023-07-06T03:53:46.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.750+0000 7f56331ff700 1 -- 172.21.15.97:0/2760331092 wait complete. 2023-07-06T03:53:46.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.752+0000 7f56331ff700 1 Processor -- start 2023-07-06T03:53:46.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.752+0000 7f56331ff700 1 -- start start 2023-07-06T03:53:46.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.753+0000 7f56331ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f562c0b8410 0x7f562c148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.753+0000 7f56331ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c0cd390 0x7f562c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.753+0000 7f56331ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f562c15c520 0x7f562c0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.753+0000 7f56331ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c14c920 con 0x7f562c0cd390 2023-07-06T03:53:46.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.753+0000 7f56331ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0cd060 con 0x7f562c0b8410 2023-07-06T03:53:46.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.753+0000 7f56331ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0cc450 con 0x7f562c15c520 2023-07-06T03:53:46.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.753+0000 7f562bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c0cd390 0x7f562c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:46.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.753+0000 7f562bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c0cd390 0x7f562c0c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35416/0 (socket says 172.21.15.97:35416) 2023-07-06T03:53:46.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.753+0000 7f562bfff700 1 -- 172.21.15.97:0/1351743480 learned_addr learned my addr 172.21.15.97:0/1351743480 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:46.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f563179c700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f562c15c520 0x7f562c0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:46.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f5630f9b700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f562c0b8410 0x7f562c148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:46.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f562bfff700 1 -- 172.21.15.97:0/1351743480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f562c15c520 msgr2=0x7f562c0ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:46.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f562bfff700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f562c15c520 0x7f562c0ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f562bfff700 1 -- 172.21.15.97:0/1351743480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f562c0b8410 msgr2=0x7f562c148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f562bfff700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f562c0b8410 0x7f562c148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f562bfff700 1 -- 172.21.15.97:0/1351743480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5624009a70 con 0x7f562c0cd390 2023-07-06T03:53:46.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f562bfff700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c0cd390 0x7f562c0c9e10 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f561800c4d0 tx=0x7f561800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:46.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f56297fa700 1 -- 172.21.15.97:0/1351743480 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5618011790 con 0x7f562c0cd390 2023-07-06T03:53:46.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f56297fa700 1 -- 172.21.15.97:0/1351743480 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56180118f0 con 0x7f562c0cd390 2023-07-06T03:53:46.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f56331ff700 1 -- 172.21.15.97:0/1351743480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c0cc730 con 0x7f562c0cd390 2023-07-06T03:53:46.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.754+0000 7f56297fa700 1 -- 172.21.15.97:0/1351743480 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5618011ca0 con 0x7f562c0cd390 2023-07-06T03:53:46.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.755+0000 7f56331ff700 1 -- 172.21.15.97:0/1351743480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c0cccf0 con 0x7f562c0cd390 2023-07-06T03:53:46.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.756+0000 7f56331ff700 1 -- 172.21.15.97:0/1351743480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c0cce80 con 0x7f562c0cd390 2023-07-06T03:53:46.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.757+0000 7f56297fa700 1 -- 172.21.15.97:0/1351743480 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f561801c460 con 0x7f562c0cd390 2023-07-06T03:53:46.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.759+0000 7f56297fa700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f561006ace0 0x7f561006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:46.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.759+0000 7f56297fa700 1 -- 172.21.15.97:0/1351743480 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f56180892d0 con 0x7f562c0cd390 2023-07-06T03:53:46.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.759+0000 7f5630f9b700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f561006ace0 0x7f561006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:46.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.760+0000 7f5630f9b700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f561006ace0 0x7f561006d190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5620007c20 tx=0x7f5620007d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:46.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:46.763+0000 7f56297fa700 1 -- 172.21.15.97:0/1351743480 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5618054370 con 0x7f562c0cd390 2023-07-06T03:53:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.062+0000 7f56331ff700 1 -- 172.21.15.97:0/1351743480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f562c0cd7e0 con 0x7f562c0cd390 2023-07-06T03:53:47.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.063+0000 7f56297fa700 1 -- 172.21.15.97:0/1351743480 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4673 (secure 0 0 0) 0x7f562c0cd7e0 con 0x7f562c0cd390 2023-07-06T03:53:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:47.065 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":67,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/387292825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":387292825},{"type":"v1","addr":"172.21.15.146:6833","nonce":387292825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690380561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690380561},{"type":"v1","addr":"172.21.15.97:6837","nonce":690380561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3897834271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3897834271},{"type":"v1","addr":"172.21.15.146:6835","nonce":3897834271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:53:26.234190+0000","modified":"2023-07-06T03:53:29.774892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/3324055785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3324055785},{"type":"v1","addr":"172.21.15.97:6836","nonce":3324055785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-06T03:53:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 -- 172.21.15.97:0/1351743480 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f561006ace0 msgr2=0x7f561006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:47.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f561006ace0 0x7f561006d190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5620007c20 tx=0x7f5620007d70 comp rx=0 tx=0).stop 2023-07-06T03:53:47.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 -- 172.21.15.97:0/1351743480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c0cd390 msgr2=0x7f562c0c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:47.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c0cd390 0x7f562c0c9e10 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f561800c4d0 tx=0x7f561800c890 comp rx=0 tx=0).stop 2023-07-06T03:53:47.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 -- 172.21.15.97:0/1351743480 shutdown_connections 2023-07-06T03:53:47.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f561006ace0 0x7f561006d190 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:47.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f562c15c520 0x7f562c0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:47.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f562c0b8410 0x7f562c148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:47.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 --2- 172.21.15.97:0/1351743480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c0cd390 0x7f562c0c9e10 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:47.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 -- 172.21.15.97:0/1351743480 >> 172.21.15.97:0/1351743480 conn(0x7f562c071790 msgr2=0x7f562c15fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:47.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 -- 172.21.15.97:0/1351743480 shutdown_connections 2023-07-06T03:53:47.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:47.066+0000 7f56331ff700 1 -- 172.21.15.97:0/1351743480 wait complete. 2023-07-06T03:53:47.070 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 67 2023-07-06T03:53:47.087 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-07-06T03:53:47.346 INFO:teuthology.orchestra.run.smithi097.stdout:[{"id":7495,"entity":{"name":{"type":"client","num":7495},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1754496776}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":5.2619129850000004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9848038655302904,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7495 192.168.0.1:0/1754496776","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"125117","root":"/"}},{"id":8199,"entity":{"name":{"type":"client","num":8199},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2529580715}},"state":"open","num_leases":1,"num_caps":401,"request_load_avg":401,"uptime":7.555875049,"requests_in_flight":399,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":803.36392285034867,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8199 192.168.0.1:0/2529580715","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"154228","root":"/"}}] 2023-07-06T03:53:47.347 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "125117", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1754496776, "type": "any" }, "name": { "num": 7495, "type": "client" } }, "id": 7495, "inst": "client.7495 192.168.0.1:0/1754496776", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9848038655302904 }, "state": "open", "uptime": 5.261912985 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "154228", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2529580715, "type": "any" }, "name": { "num": 8199, "type": "client" } }, "id": 8199, "inst": "client.8199 192.168.0.1:0/2529580715", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 401, "requests_in_flight": 399, "session_cache_liveness": { "halflife": 300, "value": 803.3639228503487 }, "state": "open", "uptime": 7.555875049 } ] 2023-07-06T03:53:47.348 DEBUG:tasks.ceph_test_case:wait_until_equal: 401 != 802, waiting (timeout=600)... 2023-07-06T03:53:52.348 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:53:52.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.550+0000 7f08a87ff700 1 Processor -- start 2023-07-06T03:53:52.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.550+0000 7f08a87ff700 1 -- start start 2023-07-06T03:53:52.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.551+0000 7f08a87ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08a015c520 0x7f08a00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:52.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.551+0000 7f08a87ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f08a0160180 0x7f08a01605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:52.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.551+0000 7f08a87ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a00cdae0 con 0x7f08a0160b30 2023-07-06T03:53:52.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.551+0000 7f08a87ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a00cd8d0 con 0x7f08a015c520 2023-07-06T03:53:52.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.551+0000 7f08a87ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a00b6640 con 0x7f08a0160180 2023-07-06T03:53:52.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.551+0000 7f08a659b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08a015c520 0x7f08a00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:52.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.551+0000 7f08a659b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08a015c520 0x7f08a00cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60110/0 (socket says 172.21.15.97:60110) 2023-07-06T03:53:52.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.551+0000 7f08a659b700 1 -- 172.21.15.97:0/1353320580 learned_addr learned my addr 172.21.15.97:0/1353320580 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:52.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.552+0000 7f08a5d9a700 1 --2- 172.21.15.97:0/1353320580 >> v2:172.21.15.146:3300/0 conn(0x7f08a0160180 0x7f08a01605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:52.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.552+0000 7f08a659b700 1 -- 172.21.15.97:0/1353320580 >> v2:172.21.15.146:3300/0 conn(0x7f08a0160180 msgr2=0x7f08a01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:52.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.552+0000 7f08a659b700 1 --2- 172.21.15.97:0/1353320580 >> v2:172.21.15.146:3300/0 conn(0x7f08a0160180 0x7f08a01605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:52.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.552+0000 7f08a659b700 1 -- 172.21.15.97:0/1353320580 >> v1:172.21.15.146:6789/0 conn(0x7f08a0160b30 legacy=0x7f08a00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:52.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.552+0000 7f08a659b700 1 -- 172.21.15.97:0/1353320580 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a00b6920 con 0x7f08a015c520 2023-07-06T03:53:52.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.552+0000 7f08a659b700 1 --2- 172.21.15.97:0/1353320580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a015c520 0x7f08a00cd390 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f08a00b5930 tx=0x7f089c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a5fccab2585222 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:52.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.552+0000 7f08a5599700 1 -- 172.21.15.97:0/1353320580 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c00b870 con 0x7f08a015c520 2023-07-06T03:53:52.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.552+0000 7f08a5599700 1 -- 172.21.15.97:0/1353320580 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c00b9d0 con 0x7f08a015c520 2023-07-06T03:53:52.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.552+0000 7f08a5599700 1 -- 172.21.15.97:0/1353320580 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c00bd60 con 0x7f08a015c520 2023-07-06T03:53:52.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.553+0000 7f08a87ff700 1 -- 172.21.15.97:0/1353320580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a015c520 msgr2=0x7f08a00cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:52.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.553+0000 7f08a87ff700 1 --2- 172.21.15.97:0/1353320580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a015c520 0x7f08a00cd390 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f08a00b5930 tx=0x7f089c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:53:52.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.553+0000 7f08a87ff700 1 -- 172.21.15.97:0/1353320580 shutdown_connections 2023-07-06T03:53:52.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.553+0000 7f08a87ff700 1 --2- 172.21.15.97:0/1353320580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a015c520 0x7f08a00cd390 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:52.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.553+0000 7f08a87ff700 1 --2- 172.21.15.97:0/1353320580 >> v2:172.21.15.146:3300/0 conn(0x7f08a0160180 0x7f08a01605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:52.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.553+0000 7f08a87ff700 1 -- 172.21.15.97:0/1353320580 >> 172.21.15.97:0/1353320580 conn(0x7f08a0071790 msgr2=0x7f08a0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:52.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.553+0000 7f08a87ff700 1 -- 172.21.15.97:0/1353320580 shutdown_connections 2023-07-06T03:53:52.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.553+0000 7f08a87ff700 1 -- 172.21.15.97:0/1353320580 wait complete. 2023-07-06T03:53:52.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.555+0000 7f08a87ff700 1 Processor -- start 2023-07-06T03:53:52.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.555+0000 7f08a87ff700 1 -- start start 2023-07-06T03:53:52.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a87ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a015c520 0x7f08a014dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:52.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a87ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a0160180 0x7f08a00cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:52.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a87ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08a0160b30 0x7f08a014cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:52.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a87ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a014f120 con 0x7f08a0160180 2023-07-06T03:53:52.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a87ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a014e2b0 con 0x7f08a0160b30 2023-07-06T03:53:52.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a87ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a014e510 con 0x7f08a015c520 2023-07-06T03:53:52.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a5d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a0160180 0x7f08a00cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:52.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a5d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a0160180 0x7f08a00cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60126/0 (socket says 172.21.15.97:60126) 2023-07-06T03:53:52.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a5d9a700 1 -- 172.21.15.97:0/3328576909 learned_addr learned my addr 172.21.15.97:0/3328576909 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:52.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a659b700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a015c520 0x7f08a014dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:52.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a6d9c700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08a0160b30 0x7f08a014cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:52.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a5d9a700 1 -- 172.21.15.97:0/3328576909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a015c520 msgr2=0x7f08a014dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:52.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a5d9a700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a015c520 0x7f08a014dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:52.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a5d9a700 1 -- 172.21.15.97:0/3328576909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08a0160b30 msgr2=0x7f08a014cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:52.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a5d9a700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08a0160b30 0x7f08a014cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:52.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.556+0000 7f08a5d9a700 1 -- 172.21.15.97:0/3328576909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c009550 con 0x7f08a0160180 2023-07-06T03:53:52.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.557+0000 7f08a5d9a700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a0160180 0x7f08a00cc090 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f089000e810 tx=0x7f089000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:52.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.557+0000 7f0896ffd700 1 -- 172.21.15.97:0/3328576909 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890013610 con 0x7f08a0160180 2023-07-06T03:53:52.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.557+0000 7f0896ffd700 1 -- 172.21.15.97:0/3328576909 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0890013770 con 0x7f08a0160180 2023-07-06T03:53:52.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.557+0000 7f08a87ff700 1 -- 172.21.15.97:0/3328576909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a014e7f0 con 0x7f08a0160180 2023-07-06T03:53:52.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.557+0000 7f0896ffd700 1 -- 172.21.15.97:0/3328576909 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890013b20 con 0x7f08a0160180 2023-07-06T03:53:52.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.557+0000 7f08a87ff700 1 -- 172.21.15.97:0/3328576909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a014edb0 con 0x7f08a0160180 2023-07-06T03:53:52.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.558+0000 7f08a87ff700 1 -- 172.21.15.97:0/3328576909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a004e340 con 0x7f08a0160180 2023-07-06T03:53:52.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.559+0000 7f0896ffd700 1 -- 172.21.15.97:0/3328576909 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0890013c80 con 0x7f08a0160180 2023-07-06T03:53:52.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.560+0000 7f0896ffd700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088406ace0 0x7f088406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:52.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.560+0000 7f0896ffd700 1 -- 172.21.15.97:0/3328576909 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0890089250 con 0x7f08a0160180 2023-07-06T03:53:52.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.560+0000 7f08a659b700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088406ace0 0x7f088406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:52.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.561+0000 7f08a659b700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088406ace0 0x7f088406d190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f089c014040 tx=0x7f089c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:52.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.564+0000 7f0896ffd700 1 -- 172.21.15.97:0/3328576909 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0890057cf0 con 0x7f08a0160180 2023-07-06T03:53:52.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.808+0000 7f08a87ff700 1 -- 172.21.15.97:0/3328576909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08a00b4f40 con 0x7f08a0160180 2023-07-06T03:53:52.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.809+0000 7f0896ffd700 1 -- 172.21.15.97:0/3328576909 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4673 (secure 0 0 0) 0x7f0890057880 con 0x7f08a0160180 2023-07-06T03:53:52.812 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:52.812 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":68,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/387292825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":387292825},{"type":"v1","addr":"172.21.15.146:6833","nonce":387292825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690380561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690380561},{"type":"v1","addr":"172.21.15.97:6837","nonce":690380561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3897834271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3897834271},{"type":"v1","addr":"172.21.15.146:6835","nonce":3897834271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:53:26.234190+0000","modified":"2023-07-06T03:53:47.279223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":8,"addr":"172.21.15.97:6836/3324055785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3324055785},{"type":"v1","addr":"172.21.15.97:6836","nonce":3324055785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-06T03:53:52.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.811+0000 7f08a87ff700 1 -- 172.21.15.97:0/3328576909 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088406ace0 msgr2=0x7f088406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:52.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.811+0000 7f08a87ff700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088406ace0 0x7f088406d190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f089c014040 tx=0x7f089c008030 comp rx=0 tx=0).stop 2023-07-06T03:53:52.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.811+0000 7f08a87ff700 1 -- 172.21.15.97:0/3328576909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a0160180 msgr2=0x7f08a00cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:52.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.811+0000 7f08a87ff700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a0160180 0x7f08a00cc090 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f089000e810 tx=0x7f089000ebd0 comp rx=0 tx=0).stop 2023-07-06T03:53:52.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.811+0000 7f08a87ff700 1 -- 172.21.15.97:0/3328576909 shutdown_connections 2023-07-06T03:53:52.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.811+0000 7f08a87ff700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088406ace0 0x7f088406d190 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:52.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.812+0000 7f08a87ff700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08a0160b30 0x7f08a014cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:52.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.812+0000 7f08a87ff700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08a015c520 0x7f08a014dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:52.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.812+0000 7f08a87ff700 1 --2- 172.21.15.97:0/3328576909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a0160180 0x7f08a00cc090 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:52.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.812+0000 7f08a87ff700 1 -- 172.21.15.97:0/3328576909 >> 172.21.15.97:0/3328576909 conn(0x7f08a0071790 msgr2=0x7f08a00b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:52.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.812+0000 7f08a87ff700 1 -- 172.21.15.97:0/3328576909 shutdown_connections 2023-07-06T03:53:52.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:52.812+0000 7f08a87ff700 1 -- 172.21.15.97:0/3328576909 wait complete. 2023-07-06T03:53:52.816 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 68 2023-07-06T03:53:52.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-07-06T03:53:53.051 INFO:teuthology.orchestra.run.smithi097.stdout:[{"id":7495,"entity":{"name":{"type":"client","num":7495},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1754496776}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":10.969818593999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9456985149805495,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7495 192.168.0.1:0/1754496776","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"125117","root":"/"}},{"id":8199,"entity":{"name":{"type":"client","num":8199},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2529580715}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":1148,"uptime":13.263780658,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2324.9186869544787,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":895.70457466815822,"halflife":1.5},"recall_caps_throttle2o":{"value":368.59823451246325,"halflife":0.5},"session_cache_liveness":{"value":1585.1908890596892,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":506,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8199 192.168.0.1:0/2529580715","completed_requests":[{"tid":506,"created_ino":"0x100000003e9"},{"tid":507,"created_ino":"0x100000003ea"},{"tid":508,"created_ino":"0x100000003eb"},{"tid":509,"created_ino":"0x100000003ec"},{"tid":510,"created_ino":"0x100000003ed"},{"tid":511,"created_ino":"0x100000003ee"},{"tid":512,"created_ino":"0x100000003ef"},{"tid":513,"created_ino":"0x100000003f0"},{"tid":514,"created_ino":"0x100000003f1"},{"tid":515,"created_ino":"0x100000003f2"},{"tid":516,"created_ino":"0x100000003f3"},{"tid":517,"created_ino":"0x100000003f4"},{"tid":518,"created_ino":"0x100000003f5"},{"tid":519,"created_ino":"0x100000003f6"},{"tid":520,"created_ino":"0x100000003f7"},{"tid":521,"created_ino":"0x100000003f8"},{"tid":522,"created_ino":"0x100000003f9"},{"tid":523,"created_ino":"0x100000003fa"},{"tid":524,"created_ino":"0x100000003fb"},{"tid":525,"created_ino":"0x100000003fc"},{"tid":526,"created_ino":"0x100000003fd"},{"tid":527,"created_ino":"0x100000003fe"},{"tid":528,"created_ino":"0x100000003ff"},{"tid":529,"created_ino":"0x10000000400"},{"tid":530,"created_ino":"0x10000000401"},{"tid":531,"created_ino":"0x10000000402"},{"tid":532,"created_ino":"0x10000000403"},{"tid":533,"created_ino":"0x10000000404"},{"tid":534,"created_ino":"0x10000000405"},{"tid":535,"created_ino":"0x10000000406"},{"tid":536,"created_ino":"0x10000000407"},{"tid":537,"created_ino":"0x10000000408"},{"tid":538,"created_ino":"0x10000000409"},{"tid":539,"created_ino":"0x1000000040a"},{"tid":540,"created_ino":"0x1000000040b"},{"tid":541,"created_ino":"0x1000000040c"},{"tid":542,"created_ino":"0x1000000040d"},{"tid":543,"created_ino":"0x1000000040e"},{"tid":544,"created_ino":"0x1000000040f"},{"tid":545,"created_ino":"0x10000000410"},{"tid":546,"created_ino":"0x10000000411"},{"tid":547,"created_ino":"0x10000000412"},{"tid":548,"created_ino":"0x10000000413"},{"tid":549,"created_ino":"0x10000000414"},{"tid":550,"created_ino":"0x10000000415"},{"tid":551,"created_ino":"0x10000000416"},{"tid":552,"created_ino":"0x10000000417"},{"tid":553,"created_ino":"0x10000000418"},{"tid":554,"created_ino":"0x10000000419"},{"tid":555,"created_ino":"0x1000000041a"},{"tid":556,"created_ino":"0x1000000041b"},{"tid":557,"created_ino":"0x1000000041c"},{"tid":558,"created_ino":"0x1000000041d"},{"tid":559,"created_ino":"0x1000000041e"},{"tid":560,"created_ino":"0x1000000041f"},{"tid":561,"created_ino":"0x10000000420"},{"tid":562,"created_ino":"0x10000000421"},{"tid":563,"created_ino":"0x10000000422"},{"tid":564,"created_ino":"0x10000000423"},{"tid":565,"created_ino":"0x10000000424"},{"tid":566,"created_ino":"0x10000000425"},{"tid":567,"created_ino":"0x10000000426"},{"tid":568,"created_ino":"0x10000000427"},{"tid":569,"created_ino":"0x10000000428"},{"tid":570,"created_ino":"0x10000000429"},{"tid":571,"created_ino":"0x1000000042a"},{"tid":572,"created_ino":"0x1000000042b"},{"tid":573,"created_ino":"0x1000000042c"},{"tid":574,"created_ino":"0x1000000042d"},{"tid":575,"created_ino":"0x1000000042e"},{"tid":576,"created_ino":"0x1000000042f"},{"tid":577,"created_ino":"0x10000000430"},{"tid":578,"created_ino":"0x10000000431"},{"tid":579,"created_ino":"0x10000000432"},{"tid":580,"created_ino":"0x10000000433"},{"tid":581,"created_ino":"0x10000000434"},{"tid":582,"created_ino":"0x10000000435"},{"tid":583,"created_ino":"0x10000000436"},{"tid":584,"created_ino":"0x10000000437"},{"tid":585,"created_ino":"0x10000000438"},{"tid":586,"created_ino":"0x10000000439"},{"tid":587,"created_ino":"0x1000000043a"},{"tid":588,"created_ino":"0x1000000043b"},{"tid":589,"created_ino":"0x1000000043c"},{"tid":590,"created_ino":"0x1000000043d"},{"tid":591,"created_ino":"0x1000000043e"},{"tid":592,"created_ino":"0x1000000043f"},{"tid":593,"created_ino":"0x10000000440"},{"tid":594,"created_ino":"0x10000000441"},{"tid":595,"created_ino":"0x10000000442"},{"tid":598,"created_ino":"0x10000000443"},{"tid":602,"created_ino":"0x10000000444"},{"tid":605,"created_ino":"0x10000000445"},{"tid":608,"created_ino":"0x10000000446"},{"tid":611,"created_ino":"0x10000000447"},{"tid":614,"created_ino":"0x10000000448"},{"tid":617,"created_ino":"0x10000000449"},{"tid":620,"created_ino":"0x1000000044a"},{"tid":623,"created_ino":"0x1000000044b"},{"tid":626,"created_ino":"0x1000000044c"},{"tid":629,"created_ino":"0x1000000044d"},{"tid":632,"created_ino":"0x1000000044e"},{"tid":635,"created_ino":"0x1000000044f"},{"tid":638,"created_ino":"0x10000000450"},{"tid":641,"created_ino":"0x10000000451"},{"tid":644,"created_ino":"0x10000000452"},{"tid":647,"created_ino":"0x10000000453"},{"tid":650,"created_ino":"0x10000000454"},{"tid":653,"created_ino":"0x10000000455"},{"tid":656,"created_ino":"0x10000000456"},{"tid":659,"created_ino":"0x10000000457"},{"tid":662,"created_ino":"0x10000000458"},{"tid":665,"created_ino":"0x10000000459"},{"tid":668,"created_ino":"0x1000000045a"},{"tid":671,"created_ino":"0x1000000045b"},{"tid":674,"created_ino":"0x1000000045c"},{"tid":677,"created_ino":"0x1000000045d"},{"tid":680,"created_ino":"0x1000000045e"},{"tid":683,"created_ino":"0x1000000045f"},{"tid":686,"created_ino":"0x10000000460"},{"tid":689,"created_ino":"0x10000000461"},{"tid":692,"created_ino":"0x10000000462"},{"tid":695,"created_ino":"0x10000000463"},{"tid":698,"created_ino":"0x10000000464"},{"tid":701,"created_ino":"0x10000000465"},{"tid":704,"created_ino":"0x10000000466"},{"tid":707,"created_ino":"0x10000000467"},{"tid":710,"created_ino":"0x10000000468"},{"tid":713,"created_ino":"0x10000000469"},{"tid":716,"created_ino":"0x1000000046a"},{"tid":719,"created_ino":"0x1000000046b"},{"tid":722,"created_ino":"0x1000000046c"},{"tid":725,"created_ino":"0x1000000046d"},{"tid":728,"created_ino":"0x1000000046e"},{"tid":731,"created_ino":"0x1000000046f"},{"tid":734,"created_ino":"0x10000000470"},{"tid":737,"created_ino":"0x10000000471"},{"tid":740,"created_ino":"0x10000000472"},{"tid":743,"created_ino":"0x10000000473"},{"tid":746,"created_ino":"0x10000000474"},{"tid":749,"created_ino":"0x10000000475"},{"tid":752,"created_ino":"0x10000000476"},{"tid":755,"created_ino":"0x10000000477"},{"tid":758,"created_ino":"0x10000000478"},{"tid":761,"created_ino":"0x10000000479"},{"tid":764,"created_ino":"0x1000000047a"},{"tid":767,"created_ino":"0x1000000047b"},{"tid":770,"created_ino":"0x1000000047c"},{"tid":773,"created_ino":"0x1000000047d"},{"tid":776,"created_ino":"0x1000000047e"},{"tid":779,"created_ino":"0x1000000047f"},{"tid":782,"created_ino":"0x10000000480"},{"tid":785,"created_ino":"0x10000000481"},{"tid":788,"created_ino":"0x10000000482"},{"tid":791,"created_ino":"0x10000000483"},{"tid":794,"created_ino":"0x10000000484"},{"tid":797,"created_ino":"0x10000000485"},{"tid":800,"created_ino":"0x10000000486"},{"tid":803,"created_ino":"0x10000000487"},{"tid":806,"created_ino":"0x10000000488"},{"tid":809,"created_ino":"0x10000000489"},{"tid":812,"created_ino":"0x1000000048a"},{"tid":815,"created_ino":"0x1000000048b"},{"tid":818,"created_ino":"0x1000000048c"},{"tid":821,"created_ino":"0x1000000048d"},{"tid":824,"created_ino":"0x1000000048e"},{"tid":827,"created_ino":"0x1000000048f"},{"tid":830,"created_ino":"0x10000000490"},{"tid":833,"created_ino":"0x10000000491"},{"tid":836,"created_ino":"0x10000000492"},{"tid":839,"created_ino":"0x10000000493"},{"tid":842,"created_ino":"0x10000000494"},{"tid":845,"created_ino":"0x10000000495"},{"tid":848,"created_ino":"0x10000000496"},{"tid":851,"created_ino":"0x10000000497"},{"tid":854,"created_ino":"0x10000000498"},{"tid":857,"created_ino":"0x10000000499"},{"tid":860,"created_ino":"0x1000000049a"},{"tid":863,"created_ino":"0x1000000049b"},{"tid":866,"created_ino":"0x1000000049c"},{"tid":869,"created_ino":"0x1000000049d"},{"tid":872,"created_ino":"0x1000000049e"},{"tid":875,"created_ino":"0x1000000049f"},{"tid":878,"created_ino":"0x100000004a0"},{"tid":881,"created_ino":"0x100000004a1"},{"tid":884,"created_ino":"0x100000004a2"},{"tid":887,"created_ino":"0x100000004a3"},{"tid":890,"created_ino":"0x100000004a4"},{"tid":893,"created_ino":"0x100000004a5"},{"tid":896,"created_ino":"0x100000004a6"},{"tid":899,"created_ino":"0x100000004a7"},{"tid":902,"created_ino":"0x100000004a8"},{"tid":905,"created_ino":"0x100000004a9"},{"tid":908,"created_ino":"0x100000004aa"},{"tid":911,"created_ino":"0x100000004ab"},{"tid":914,"created_ino":"0x100000004ac"},{"tid":917,"created_ino":"0x100000004ad"},{"tid":920,"created_ino":"0x100000004ae"},{"tid":923,"created_ino":"0x100000004af"},{"tid":926,"created_ino":"0x100000004b0"},{"tid":929,"created_ino":"0x100000004b1"},{"tid":932,"created_ino":"0x100000004b2"},{"tid":935,"created_ino":"0x100000004b3"},{"tid":938,"created_ino":"0x100000004b4"},{"tid":941,"created_ino":"0x100000004b5"},{"tid":944,"created_ino":"0x100000004b6"},{"tid":947,"created_ino":"0x100000004b7"},{"tid":950,"created_ino":"0x100000004b8"},{"tid":953,"created_ino":"0x100000004b9"},{"tid":956,"created_ino":"0x100000004ba"},{"tid":959,"created_ino":"0x100000004bb"},{"tid":962,"created_ino":"0x100000004bc"},{"tid":965,"created_ino":"0x100000004bd"},{"tid":968,"created_ino":"0x100000004be"},{"tid":971,"created_ino":"0x100000004bf"},{"tid":974,"created_ino":"0x100000004c0"},{"tid":977,"created_ino":"0x100000004c1"},{"tid":980,"created_ino":"0x100000004c2"},{"tid":983,"created_ino":"0x100000004c3"},{"tid":986,"created_ino":"0x100000004c4"},{"tid":989,"created_ino":"0x100000004c5"},{"tid":992,"created_ino":"0x100000004c6"},{"tid":995,"created_ino":"0x100000004c7"},{"tid":998,"created_ino":"0x100000004c8"},{"tid":1001,"created_ino":"0x100000004c9"},{"tid":1004,"created_ino":"0x100000004ca"},{"tid":1007,"created_ino":"0x100000004cb"},{"tid":1010,"created_ino":"0x100000004cc"},{"tid":1013,"created_ino":"0x100000004cd"},{"tid":1016,"created_ino":"0x100000004ce"},{"tid":1019,"created_ino":"0x100000004cf"},{"tid":1022,"created_ino":"0x100000004d0"},{"tid":1025,"created_ino":"0x100000004d1"},{"tid":1028,"created_ino":"0x100000004d2"},{"tid":1031,"created_ino":"0x100000004d3"},{"tid":1034,"created_ino":"0x100000004d4"},{"tid":1037,"created_ino":"0x100000004d5"},{"tid":1040,"created_ino":"0x100000004d6"},{"tid":1043,"created_ino":"0x100000004d7"},{"tid":1046,"created_ino":"0x100000004d8"},{"tid":1049,"created_ino":"0x100000004d9"},{"tid":1052,"created_ino":"0x100000004da"},{"tid":1055,"created_ino":"0x100000004db"},{"tid":1058,"created_ino":"0x100000004dc"},{"tid":1061,"created_ino":"0x100000004dd"},{"tid":1064,"created_ino":"0x100000004de"},{"tid":1067,"created_ino":"0x100000004df"},{"tid":1070,"created_ino":"0x100000004e0"},{"tid":1073,"created_ino":"0x100000004e1"},{"tid":1076,"created_ino":"0x100000004e2"},{"tid":1079,"created_ino":"0x100000004e3"},{"tid":1082,"created_ino":"0x100000004e4"},{"tid":1085,"created_ino":"0x100000004e5"},{"tid":1088,"created_ino":"0x100000004e6"},{"tid":1091,"created_ino":"0x100000004e7"},{"tid":1094,"created_ino":"0x100000004e8"},{"tid":1097,"created_ino":"0x100000004e9"},{"tid":1100,"created_ino":"0x100000004ea"},{"tid":1103,"created_ino":"0x100000004eb"},{"tid":1106,"created_ino":"0x100000004ec"},{"tid":1109,"created_ino":"0x100000004ed"},{"tid":1112,"created_ino":"0x100000004ee"},{"tid":1115,"created_ino":"0x100000004ef"},{"tid":1118,"created_ino":"0x100000004f0"},{"tid":1121,"created_ino":"0x100000004f1"},{"tid":1124,"created_ino":"0x100000004f2"},{"tid":1127,"created_ino":"0x100000004f3"},{"tid":1130,"created_ino":"0x100000004f4"},{"tid":1133,"created_ino":"0x100000004f5"},{"tid":1136,"created_ino":"0x100000004f6"},{"tid":1139,"created_ino":"0x100000004f7"},{"tid":1142,"created_ino":"0x100000004f8"},{"tid":1145,"created_ino":"0x100000004f9"},{"tid":1148,"created_ino":"0x100000004fa"},{"tid":1151,"created_ino":"0x100000004fb"},{"tid":1154,"created_ino":"0x100000004fc"},{"tid":1157,"created_ino":"0x100000004fd"},{"tid":1160,"created_ino":"0x100000004fe"},{"tid":1163,"created_ino":"0x100000004ff"},{"tid":1166,"created_ino":"0x10000000500"},{"tid":1169,"created_ino":"0x10000000501"},{"tid":1172,"created_ino":"0x10000000502"},{"tid":1175,"created_ino":"0x10000000503"},{"tid":1178,"created_ino":"0x10000000504"},{"tid":1181,"created_ino":"0x10000000505"},{"tid":1184,"created_ino":"0x10000000506"},{"tid":1187,"created_ino":"0x10000000507"},{"tid":1190,"created_ino":"0x10000000508"},{"tid":1193,"created_ino":"0x10000000509"},{"tid":1196,"created_ino":"0x1000000050a"},{"tid":1199,"created_ino":"0x1000000050b"},{"tid":1202,"created_ino":"0x1000000050c"},{"tid":1205,"created_ino":"0x1000000050d"},{"tid":1208,"created_ino":"0x1000000050e"},{"tid":1211,"created_ino":"0x1000000050f"},{"tid":1214,"created_ino":"0x10000000510"},{"tid":1217,"created_ino":"0x10000000511"},{"tid":1220,"created_ino":"0x10000000512"},{"tid":1223,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"154228","root":"/"}}] 2023-07-06T03:53:53.054 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "125117", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1754496776, "type": "any" }, "name": { "num": 7495, "type": "client" } }, "id": 7495, "inst": "client.7495 192.168.0.1:0/1754496776", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9456985149805495 }, "state": "open", "uptime": 10.969818594 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "154228", "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 506 }, { "created_ino": "0x100000003ea", "tid": 507 }, { "created_ino": "0x100000003eb", "tid": 508 }, { "created_ino": "0x100000003ec", "tid": 509 }, { "created_ino": "0x100000003ed", "tid": 510 }, { "created_ino": "0x100000003ee", "tid": 511 }, { "created_ino": "0x100000003ef", "tid": 512 }, { "created_ino": "0x100000003f0", "tid": 513 }, { "created_ino": "0x100000003f1", "tid": 514 }, { "created_ino": "0x100000003f2", "tid": 515 }, { "created_ino": "0x100000003f3", "tid": 516 }, { "created_ino": "0x100000003f4", "tid": 517 }, { "created_ino": "0x100000003f5", "tid": 518 }, { "created_ino": "0x100000003f6", "tid": 519 }, { "created_ino": "0x100000003f7", "tid": 520 }, { "created_ino": "0x100000003f8", "tid": 521 }, { "created_ino": "0x100000003f9", "tid": 522 }, { "created_ino": "0x100000003fa", "tid": 523 }, { "created_ino": "0x100000003fb", "tid": 524 }, { "created_ino": "0x100000003fc", "tid": 525 }, { "created_ino": "0x100000003fd", "tid": 526 }, { "created_ino": "0x100000003fe", "tid": 527 }, { "created_ino": "0x100000003ff", "tid": 528 }, { "created_ino": "0x10000000400", "tid": 529 }, { "created_ino": "0x10000000401", "tid": 530 }, { "created_ino": "0x10000000402", "tid": 531 }, { "created_ino": "0x10000000403", "tid": 532 }, { "created_ino": "0x10000000404", "tid": 533 }, { "created_ino": "0x10000000405", "tid": 534 }, { "created_ino": "0x10000000406", "tid": 535 }, { "created_ino": "0x10000000407", "tid": 536 }, { "created_ino": "0x10000000408", "tid": 537 }, { "created_ino": "0x10000000409", "tid": 538 }, { "created_ino": "0x1000000040a", "tid": 539 }, { "created_ino": "0x1000000040b", "tid": 540 }, { "created_ino": "0x1000000040c", "tid": 541 }, { "created_ino": "0x1000000040d", "tid": 542 }, { "created_ino": "0x1000000040e", "tid": 543 }, { "created_ino": "0x1000000040f", "tid": 544 }, { "created_ino": "0x10000000410", "tid": 545 }, { "created_ino": "0x10000000411", "tid": 546 }, { "created_ino": "0x10000000412", "tid": 547 }, { "created_ino": "0x10000000413", "tid": 548 }, { "created_ino": "0x10000000414", "tid": 549 }, { "created_ino": "0x10000000415", "tid": 550 }, { "created_ino": "0x10000000416", "tid": 551 }, { "created_ino": "0x10000000417", "tid": 552 }, { "created_ino": "0x10000000418", "tid": 553 }, { "created_ino": "0x10000000419", "tid": 554 }, { "created_ino": "0x1000000041a", "tid": 555 }, { "created_ino": "0x1000000041b", "tid": 556 }, { "created_ino": "0x1000000041c", "tid": 557 }, { "created_ino": "0x1000000041d", "tid": 558 }, { "created_ino": "0x1000000041e", "tid": 559 }, { "created_ino": "0x1000000041f", "tid": 560 }, { "created_ino": "0x10000000420", "tid": 561 }, { "created_ino": "0x10000000421", "tid": 562 }, { "created_ino": "0x10000000422", "tid": 563 }, { "created_ino": "0x10000000423", "tid": 564 }, { "created_ino": "0x10000000424", "tid": 565 }, { "created_ino": "0x10000000425", "tid": 566 }, { "created_ino": "0x10000000426", "tid": 567 }, { "created_ino": "0x10000000427", "tid": 568 }, { "created_ino": "0x10000000428", "tid": 569 }, { "created_ino": "0x10000000429", "tid": 570 }, { "created_ino": "0x1000000042a", "tid": 571 }, { "created_ino": "0x1000000042b", "tid": 572 }, { "created_ino": "0x1000000042c", "tid": 573 }, { "created_ino": "0x1000000042d", "tid": 574 }, { "created_ino": "0x1000000042e", "tid": 575 }, { "created_ino": "0x1000000042f", "tid": 576 }, { "created_ino": "0x10000000430", "tid": 577 }, { "created_ino": "0x10000000431", "tid": 578 }, { "created_ino": "0x10000000432", "tid": 579 }, { "created_ino": "0x10000000433", "tid": 580 }, { "created_ino": "0x10000000434", "tid": 581 }, { "created_ino": "0x10000000435", "tid": 582 }, { "created_ino": "0x10000000436", "tid": 583 }, { "created_ino": "0x10000000437", "tid": 584 }, { "created_ino": "0x10000000438", "tid": 585 }, { "created_ino": "0x10000000439", "tid": 586 }, { "created_ino": "0x1000000043a", "tid": 587 }, { "created_ino": "0x1000000043b", "tid": 588 }, { "created_ino": "0x1000000043c", "tid": 589 }, { "created_ino": "0x1000000043d", "tid": 590 }, { "created_ino": "0x1000000043e", "tid": 591 }, { "created_ino": "0x1000000043f", "tid": 592 }, { "created_ino": "0x10000000440", "tid": 593 }, { "created_ino": "0x10000000441", "tid": 594 }, { "created_ino": "0x10000000442", "tid": 595 }, { "created_ino": "0x10000000443", "tid": 598 }, { "created_ino": "0x10000000444", "tid": 602 }, { "created_ino": "0x10000000445", "tid": 605 }, { "created_ino": "0x10000000446", "tid": 608 }, { "created_ino": "0x10000000447", "tid": 611 }, { "created_ino": "0x10000000448", "tid": 614 }, { "created_ino": "0x10000000449", "tid": 617 }, { "created_ino": "0x1000000044a", "tid": 620 }, { "created_ino": "0x1000000044b", "tid": 623 }, { "created_ino": "0x1000000044c", "tid": 626 }, { "created_ino": "0x1000000044d", "tid": 629 }, { "created_ino": "0x1000000044e", "tid": 632 }, { "created_ino": "0x1000000044f", "tid": 635 }, { "created_ino": "0x10000000450", "tid": 638 }, { "created_ino": "0x10000000451", "tid": 641 }, { "created_ino": "0x10000000452", "tid": 644 }, { "created_ino": "0x10000000453", "tid": 647 }, { "created_ino": "0x10000000454", "tid": 650 }, { "created_ino": "0x10000000455", "tid": 653 }, { "created_ino": "0x10000000456", "tid": 656 }, { "created_ino": "0x10000000457", "tid": 659 }, { "created_ino": "0x10000000458", "tid": 662 }, { "created_ino": "0x10000000459", "tid": 665 }, { "created_ino": "0x1000000045a", "tid": 668 }, { "created_ino": "0x1000000045b", "tid": 671 }, { "created_ino": "0x1000000045c", "tid": 674 }, { "created_ino": "0x1000000045d", "tid": 677 }, { "created_ino": "0x1000000045e", "tid": 680 }, { "created_ino": "0x1000000045f", "tid": 683 }, { "created_ino": "0x10000000460", "tid": 686 }, { "created_ino": "0x10000000461", "tid": 689 }, { "created_ino": "0x10000000462", "tid": 692 }, { "created_ino": "0x10000000463", "tid": 695 }, { "created_ino": "0x10000000464", "tid": 698 }, { "created_ino": "0x10000000465", "tid": 701 }, { "created_ino": "0x10000000466", "tid": 704 }, { "created_ino": "0x10000000467", "tid": 707 }, { "created_ino": "0x10000000468", "tid": 710 }, { "created_ino": "0x10000000469", "tid": 713 }, { "created_ino": "0x1000000046a", "tid": 716 }, { "created_ino": "0x1000000046b", "tid": 719 }, { "created_ino": "0x1000000046c", "tid": 722 }, { "created_ino": "0x1000000046d", "tid": 725 }, { "created_ino": "0x1000000046e", "tid": 728 }, { "created_ino": "0x1000000046f", "tid": 731 }, { "created_ino": "0x10000000470", "tid": 734 }, { "created_ino": "0x10000000471", "tid": 737 }, { "created_ino": "0x10000000472", "tid": 740 }, { "created_ino": "0x10000000473", "tid": 743 }, { "created_ino": "0x10000000474", "tid": 746 }, { "created_ino": "0x10000000475", "tid": 749 }, { "created_ino": "0x10000000476", "tid": 752 }, { "created_ino": "0x10000000477", "tid": 755 }, { "created_ino": "0x10000000478", "tid": 758 }, { "created_ino": "0x10000000479", "tid": 761 }, { "created_ino": "0x1000000047a", "tid": 764 }, { "created_ino": "0x1000000047b", "tid": 767 }, { "created_ino": "0x1000000047c", "tid": 770 }, { "created_ino": "0x1000000047d", "tid": 773 }, { "created_ino": "0x1000000047e", "tid": 776 }, { "created_ino": "0x1000000047f", "tid": 779 }, { "created_ino": "0x10000000480", "tid": 782 }, { "created_ino": "0x10000000481", "tid": 785 }, { "created_ino": "0x10000000482", "tid": 788 }, { "created_ino": "0x10000000483", "tid": 791 }, { "created_ino": "0x10000000484", "tid": 794 }, { "created_ino": "0x10000000485", "tid": 797 }, { "created_ino": "0x10000000486", "tid": 800 }, { "created_ino": "0x10000000487", "tid": 803 }, { "created_ino": "0x10000000488", "tid": 806 }, { "created_ino": "0x10000000489", "tid": 809 }, { "created_ino": "0x1000000048a", "tid": 812 }, { "created_ino": "0x1000000048b", "tid": 815 }, { "created_ino": "0x1000000048c", "tid": 818 }, { "created_ino": "0x1000000048d", "tid": 821 }, { "created_ino": "0x1000000048e", "tid": 824 }, { "created_ino": "0x1000000048f", "tid": 827 }, { "created_ino": "0x10000000490", "tid": 830 }, { "created_ino": "0x10000000491", "tid": 833 }, { "created_ino": "0x10000000492", "tid": 836 }, { "created_ino": "0x10000000493", "tid": 839 }, { "created_ino": "0x10000000494", "tid": 842 }, { "created_ino": "0x10000000495", "tid": 845 }, { "created_ino": "0x10000000496", "tid": 848 }, { "created_ino": "0x10000000497", "tid": 851 }, { "created_ino": "0x10000000498", "tid": 854 }, { "created_ino": "0x10000000499", "tid": 857 }, { "created_ino": "0x1000000049a", "tid": 860 }, { "created_ino": "0x1000000049b", "tid": 863 }, { "created_ino": "0x1000000049c", "tid": 866 }, { "created_ino": "0x1000000049d", "tid": 869 }, { "created_ino": "0x1000000049e", "tid": 872 }, { "created_ino": "0x1000000049f", "tid": 875 }, { "created_ino": "0x100000004a0", "tid": 878 }, { "created_ino": "0x100000004a1", "tid": 881 }, { "created_ino": "0x100000004a2", "tid": 884 }, { "created_ino": "0x100000004a3", "tid": 887 }, { "created_ino": "0x100000004a4", "tid": 890 }, { "created_ino": "0x100000004a5", "tid": 893 }, { "created_ino": "0x100000004a6", "tid": 896 }, { "created_ino": "0x100000004a7", "tid": 899 }, { "created_ino": "0x100000004a8", "tid": 902 }, { "created_ino": "0x100000004a9", "tid": 905 }, { "created_ino": "0x100000004aa", "tid": 908 }, { "created_ino": "0x100000004ab", "tid": 911 }, { "created_ino": "0x100000004ac", "tid": 914 }, { "created_ino": "0x100000004ad", "tid": 917 }, { "created_ino": "0x100000004ae", "tid": 920 }, { "created_ino": "0x100000004af", "tid": 923 }, { "created_ino": "0x100000004b0", "tid": 926 }, { "created_ino": "0x100000004b1", "tid": 929 }, { "created_ino": "0x100000004b2", "tid": 932 }, { "created_ino": "0x100000004b3", "tid": 935 }, { "created_ino": "0x100000004b4", "tid": 938 }, { "created_ino": "0x100000004b5", "tid": 941 }, { "created_ino": "0x100000004b6", "tid": 944 }, { "created_ino": "0x100000004b7", "tid": 947 }, { "created_ino": "0x100000004b8", "tid": 950 }, { "created_ino": "0x100000004b9", "tid": 953 }, { "created_ino": "0x100000004ba", "tid": 956 }, { "created_ino": "0x100000004bb", "tid": 959 }, { "created_ino": "0x100000004bc", "tid": 962 }, { "created_ino": "0x100000004bd", "tid": 965 }, { "created_ino": "0x100000004be", "tid": 968 }, { "created_ino": "0x100000004bf", "tid": 971 }, { "created_ino": "0x100000004c0", "tid": 974 }, { "created_ino": "0x100000004c1", "tid": 977 }, { "created_ino": "0x100000004c2", "tid": 980 }, { "created_ino": "0x100000004c3", "tid": 983 }, { "created_ino": "0x100000004c4", "tid": 986 }, { "created_ino": "0x100000004c5", "tid": 989 }, { "created_ino": "0x100000004c6", "tid": 992 }, { "created_ino": "0x100000004c7", "tid": 995 }, { "created_ino": "0x100000004c8", "tid": 998 }, { "created_ino": "0x100000004c9", "tid": 1001 }, { "created_ino": "0x100000004ca", "tid": 1004 }, { "created_ino": "0x100000004cb", "tid": 1007 }, { "created_ino": "0x100000004cc", "tid": 1010 }, { "created_ino": "0x100000004cd", "tid": 1013 }, { "created_ino": "0x100000004ce", "tid": 1016 }, { "created_ino": "0x100000004cf", "tid": 1019 }, { "created_ino": "0x100000004d0", "tid": 1022 }, { "created_ino": "0x100000004d1", "tid": 1025 }, { "created_ino": "0x100000004d2", "tid": 1028 }, { "created_ino": "0x100000004d3", "tid": 1031 }, { "created_ino": "0x100000004d4", "tid": 1034 }, { "created_ino": "0x100000004d5", "tid": 1037 }, { "created_ino": "0x100000004d6", "tid": 1040 }, { "created_ino": "0x100000004d7", "tid": 1043 }, { "created_ino": "0x100000004d8", "tid": 1046 }, { "created_ino": "0x100000004d9", "tid": 1049 }, { "created_ino": "0x100000004da", "tid": 1052 }, { "created_ino": "0x100000004db", "tid": 1055 }, { "created_ino": "0x100000004dc", "tid": 1058 }, { "created_ino": "0x100000004dd", "tid": 1061 }, { "created_ino": "0x100000004de", "tid": 1064 }, { "created_ino": "0x100000004df", "tid": 1067 }, { "created_ino": "0x100000004e0", "tid": 1070 }, { "created_ino": "0x100000004e1", "tid": 1073 }, { "created_ino": "0x100000004e2", "tid": 1076 }, { "created_ino": "0x100000004e3", "tid": 1079 }, { "created_ino": "0x100000004e4", "tid": 1082 }, { "created_ino": "0x100000004e5", "tid": 1085 }, { "created_ino": "0x100000004e6", "tid": 1088 }, { "created_ino": "0x100000004e7", "tid": 1091 }, { "created_ino": "0x100000004e8", "tid": 1094 }, { "created_ino": "0x100000004e9", "tid": 1097 }, { "created_ino": "0x100000004ea", "tid": 1100 }, { "created_ino": "0x100000004eb", "tid": 1103 }, { "created_ino": "0x100000004ec", "tid": 1106 }, { "created_ino": "0x100000004ed", "tid": 1109 }, { "created_ino": "0x100000004ee", "tid": 1112 }, { "created_ino": "0x100000004ef", "tid": 1115 }, { "created_ino": "0x100000004f0", "tid": 1118 }, { "created_ino": "0x100000004f1", "tid": 1121 }, { "created_ino": "0x100000004f2", "tid": 1124 }, { "created_ino": "0x100000004f3", "tid": 1127 }, { "created_ino": "0x100000004f4", "tid": 1130 }, { "created_ino": "0x100000004f5", "tid": 1133 }, { "created_ino": "0x100000004f6", "tid": 1136 }, { "created_ino": "0x100000004f7", "tid": 1139 }, { "created_ino": "0x100000004f8", "tid": 1142 }, { "created_ino": "0x100000004f9", "tid": 1145 }, { "created_ino": "0x100000004fa", "tid": 1148 }, { "created_ino": "0x100000004fb", "tid": 1151 }, { "created_ino": "0x100000004fc", "tid": 1154 }, { "created_ino": "0x100000004fd", "tid": 1157 }, { "created_ino": "0x100000004fe", "tid": 1160 }, { "created_ino": "0x100000004ff", "tid": 1163 }, { "created_ino": "0x10000000500", "tid": 1166 }, { "created_ino": "0x10000000501", "tid": 1169 }, { "created_ino": "0x10000000502", "tid": 1172 }, { "created_ino": "0x10000000503", "tid": 1175 }, { "created_ino": "0x10000000504", "tid": 1178 }, { "created_ino": "0x10000000505", "tid": 1181 }, { "created_ino": "0x10000000506", "tid": 1184 }, { "created_ino": "0x10000000507", "tid": 1187 }, { "created_ino": "0x10000000508", "tid": 1190 }, { "created_ino": "0x10000000509", "tid": 1193 }, { "created_ino": "0x1000000050a", "tid": 1196 }, { "created_ino": "0x1000000050b", "tid": 1199 }, { "created_ino": "0x1000000050c", "tid": 1202 }, { "created_ino": "0x1000000050d", "tid": 1205 }, { "created_ino": "0x1000000050e", "tid": 1208 }, { "created_ino": "0x1000000050f", "tid": 1211 }, { "created_ino": "0x10000000510", "tid": 1214 }, { "created_ino": "0x10000000511", "tid": 1217 }, { "created_ino": "0x10000000512", "tid": 1220 }, { "created_ino": "0x10000000513", "tid": 1223 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2529580715, "type": "any" }, "name": { "num": 8199, "type": "client" } }, "id": 8199, "inst": "client.8199 192.168.0.1:0/2529580715", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 506, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2324.9186869544787 }, "recall_caps_throttle": { "halflife": 1.5, "value": 895.7045746681582 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 368.59823451246325 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1148, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1585.1908890596892 }, "state": "open", "uptime": 13.263780658 } ] 2023-07-06T03:53:53.055 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-07-06T03:53:53.055 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:53:53.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.247+0000 7f6fb65ff700 1 Processor -- start 2023-07-06T03:53:53.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.248+0000 7f6fb65ff700 1 -- start start 2023-07-06T03:53:53.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.249+0000 7f6fb65ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00b0c70 0x7f6fb0163fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:53.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.249+0000 7f6fb65ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb015cc40 con 0x7f6fb0158f20 2023-07-06T03:53:53.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.249+0000 7f6fb65ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb015cf20 con 0x7f6fb01592f0 2023-07-06T03:53:53.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.249+0000 7f6fb65ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb015d200 con 0x7f6fb00b0c70 2023-07-06T03:53:53.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.249+0000 7f6faffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6fb0158f20 0x7f6fb00caac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33316/0 (socket says 172.21.15.97:33316) 2023-07-06T03:53:53.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.249+0000 7f6faffff700 1 -- 172.21.15.97:0/2898811662 learned_addr learned my addr 172.21.15.97:0/2898811662 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:53.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.249+0000 7f6fb4b9c700 1 --2- 172.21.15.97:0/2898811662 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00b0c70 0x7f6fb0163fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:53.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.250+0000 7f6faf7fe700 1 -- 172.21.15.97:0/2898811662 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272510373 0 0) 0x7f6fb015cc40 con 0x7f6fb0158f20 2023-07-06T03:53:53.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.250+0000 7f6faf7fe700 1 -- 172.21.15.97:0/2898811662 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f9c0057c0 con 0x7f6fb0158f20 2023-07-06T03:53:53.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.250+0000 7f6faf7fe700 1 -- 172.21.15.97:0/2898811662 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240569641 0 0) 0x7f6fb015cf20 con 0x7f6fb01592f0 2023-07-06T03:53:53.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.250+0000 7f6faf7fe700 1 -- 172.21.15.97:0/2898811662 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb015cc40 con 0x7f6fb01592f0 2023-07-06T03:53:53.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.250+0000 7f6fb4b9c700 1 -- 172.21.15.97:0/2898811662 >> v1:172.21.15.146:6789/0 conn(0x7f6fb01592f0 legacy=0x7f6fb00b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:53.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.250+0000 7f6fb4b9c700 1 -- 172.21.15.97:0/2898811662 >> v1:172.21.15.97:6789/0 conn(0x7f6fb0158f20 legacy=0x7f6fb00caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:53.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.250+0000 7f6faf7fe700 1 -- 172.21.15.97:0/2898811662 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1126368495 0 0) 0x7f6f9c0057c0 con 0x7f6fb0158f20 2023-07-06T03:53:53.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.250+0000 7f6fb4b9c700 1 -- 172.21.15.97:0/2898811662 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb015d4e0 con 0x7f6fb00b0c70 2023-07-06T03:53:53.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.250+0000 7f6fb4b9c700 1 --2- 172.21.15.97:0/2898811662 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00b0c70 0x7f6fb0163fd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009dc0 tx=0x7f6fa800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db16a1a56945db8f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:53.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.250+0000 7f6faf7fe700 1 -- 172.21.15.97:0/2898811662 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800e070 con 0x7f6fb00b0c70 2023-07-06T03:53:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.250+0000 7f6faf7fe700 1 -- 172.21.15.97:0/2898811662 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa8003a20 con 0x7f6fb00b0c70 2023-07-06T03:53:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.251+0000 7f6faf7fe700 1 -- 172.21.15.97:0/2898811662 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0001670 con 0x7f6fb00b0c70 2023-07-06T03:53:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.251+0000 7f6fb65ff700 1 -- 172.21.15.97:0/2898811662 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00b0c70 msgr2=0x7f6fb0163fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.251+0000 7f6fb65ff700 1 --2- 172.21.15.97:0/2898811662 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00b0c70 0x7f6fb0163fd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009dc0 tx=0x7f6fa800b330 comp rx=0 tx=0).stop 2023-07-06T03:53:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.251+0000 7f6fb65ff700 1 -- 172.21.15.97:0/2898811662 shutdown_connections 2023-07-06T03:53:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.251+0000 7f6fb65ff700 1 --2- 172.21.15.97:0/2898811662 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb00b0c70 0x7f6fb0163fd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.251+0000 7f6fb65ff700 1 -- 172.21.15.97:0/2898811662 >> 172.21.15.97:0/2898811662 conn(0x7f6fb00ce8c0 msgr2=0x7f6fb00cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:53.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.251+0000 7f6fb65ff700 1 -- 172.21.15.97:0/2898811662 shutdown_connections 2023-07-06T03:53:53.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.251+0000 7f6fb65ff700 1 -- 172.21.15.97:0/2898811662 wait complete. 2023-07-06T03:53:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.253+0000 7f6fb65ff700 1 Processor -- start 2023-07-06T03:53:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.253+0000 7f6fb65ff700 1 -- start start 2023-07-06T03:53:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6fb65ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb00b0c70 0x7f6fb0154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6fb65ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fb0158f20 0x7f6fb013b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6fb65ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb01592f0 0x7f6fb00af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6fb65ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb013b860 con 0x7f6fb00b0c70 2023-07-06T03:53:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6fb65ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb00cc400 con 0x7f6fb0158f20 2023-07-06T03:53:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6fb65ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb00cc650 con 0x7f6fb01592f0 2023-07-06T03:53:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6faffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb00b0c70 0x7f6fb0154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6faffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb00b0c70 0x7f6fb0154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60140/0 (socket says 172.21.15.97:60140) 2023-07-06T03:53:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6faffff700 1 -- 172.21.15.97:0/1677111494 learned_addr learned my addr 172.21.15.97:0/1677111494 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6fa7fff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fb0158f20 0x7f6fb013b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6fb4b9c700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb01592f0 0x7f6fb00af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6faffff700 1 -- 172.21.15.97:0/1677111494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb01592f0 msgr2=0x7f6fb00af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6faffff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb01592f0 0x7f6fb00af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6faffff700 1 -- 172.21.15.97:0/1677111494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fb0158f20 msgr2=0x7f6fb013b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6faffff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fb0158f20 0x7f6fb013b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.254+0000 7f6faffff700 1 -- 172.21.15.97:0/1677111494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa8009a70 con 0x7f6fb00b0c70 2023-07-06T03:53:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.255+0000 7f6fb4b9c700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb01592f0 0x7f6fb00af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.255+0000 7f6fa7fff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fb0158f20 0x7f6fb013b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:53:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.255+0000 7f6faffff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb00b0c70 0x7f6fb0154c30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000c730 tx=0x7f6fa000caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.255+0000 7f6fad7fa700 1 -- 172.21.15.97:0/1677111494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0004c80 con 0x7f6fb00b0c70 2023-07-06T03:53:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.255+0000 7f6fad7fa700 1 -- 172.21.15.97:0/1677111494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa0004e00 con 0x7f6fb00b0c70 2023-07-06T03:53:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.255+0000 7f6fb65ff700 1 -- 172.21.15.97:0/1677111494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb00cc930 con 0x7f6fb00b0c70 2023-07-06T03:53:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.255+0000 7f6fad7fa700 1 -- 172.21.15.97:0/1677111494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0007310 con 0x7f6fb00b0c70 2023-07-06T03:53:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.255+0000 7f6fb65ff700 1 -- 172.21.15.97:0/1677111494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb00ccef0 con 0x7f6fb00b0c70 2023-07-06T03:53:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.256+0000 7f6fb65ff700 1 -- 172.21.15.97:0/1677111494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb0156360 con 0x7f6fb00b0c70 2023-07-06T03:53:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.257+0000 7f6fad7fa700 1 -- 172.21.15.97:0/1677111494 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6fa0007470 con 0x7f6fb00b0c70 2023-07-06T03:53:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.259+0000 7f6fad7fa700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f9006ace0 0x7f6f9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.259+0000 7f6fad7fa700 1 -- 172.21.15.97:0/1677111494 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6fa00892d0 con 0x7f6fb00b0c70 2023-07-06T03:53:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.259+0000 7f6fa7fff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f9006ace0 0x7f6f9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.259+0000 7f6fa7fff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f9006ace0 0x7f6f9006d190 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00b0320 tx=0x7f6f98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:53.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.263+0000 7f6fad7fa700 1 -- 172.21.15.97:0/1677111494 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6fa0054410 con 0x7f6fb00b0c70 2023-07-06T03:53:53.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.498+0000 7f6fb65ff700 1 -- 172.21.15.97:0/1677111494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6fb004e340 con 0x7f6fb00b0c70 2023-07-06T03:53:53.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.498+0000 7f6fad7fa700 1 -- 172.21.15.97:0/1677111494 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+174 (secure 0 0 0) 0x7f6fa0057a30 con 0x7f6fb00b0c70 2023-07-06T03:53:53.501 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:53.502 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:53:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 -- 172.21.15.97:0/1677111494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f9006ace0 msgr2=0x7f6f9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f9006ace0 0x7f6f9006d190 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00b0320 tx=0x7f6f98009040 comp rx=0 tx=0).stop 2023-07-06T03:53:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 -- 172.21.15.97:0/1677111494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb00b0c70 msgr2=0x7f6fb0154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb00b0c70 0x7f6fb0154c30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000c730 tx=0x7f6fa000caf0 comp rx=0 tx=0).stop 2023-07-06T03:53:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 -- 172.21.15.97:0/1677111494 shutdown_connections 2023-07-06T03:53:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f9006ace0 0x7f6f9006d190 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6fb01592f0 0x7f6fb00af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fb00b0c70 0x7f6fb0154c30 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 --2- 172.21.15.97:0/1677111494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fb0158f20 0x7f6fb013b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 -- 172.21.15.97:0/1677111494 >> 172.21.15.97:0/1677111494 conn(0x7f6fb00ce8c0 msgr2=0x7f6fb00ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 -- 172.21.15.97:0/1677111494 shutdown_connections 2023-07-06T03:53:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:53.501+0000 7f6fb65ff700 1 -- 172.21.15.97:0/1677111494 wait complete. 2023-07-06T03:53:53.521 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2023-07-06T03:53:53.522 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:53:58.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:53:58.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.723+0000 7f440fbff700 1 Processor -- start 2023-07-06T03:53:58.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.723+0000 7f440fbff700 1 -- start start 2023-07-06T03:53:58.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.724+0000 7f440fbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f440815b440 0x7f440815e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:58.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.724+0000 7f440fbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4408070c30 0x7f44080adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.724+0000 7f440fbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4408071070 con 0x7f44080cd8e0 2023-07-06T03:53:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.725+0000 7f440fbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44080ae3e0 con 0x7f4408070c30 2023-07-06T03:53:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.725+0000 7f440fbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4408071330 con 0x7f440815b440 2023-07-06T03:53:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.725+0000 7f440e19c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4408070c30 0x7f44080adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.725+0000 7f440e19c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4408070c30 0x7f44080adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60152/0 (socket says 172.21.15.97:60152) 2023-07-06T03:53:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.725+0000 7f440e19c700 1 -- 172.21.15.97:0/3586362718 learned_addr learned my addr 172.21.15.97:0/3586362718 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:58.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.725+0000 7f440d19a700 1 --2- 172.21.15.97:0/3586362718 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f440815b440 0x7f440815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:58.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.725+0000 7f440e19c700 1 -- 172.21.15.97:0/3586362718 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f440815b440 msgr2=0x7f440815e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:58.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.725+0000 7f440e19c700 1 --2- 172.21.15.97:0/3586362718 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f440815b440 0x7f440815e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:58.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.725+0000 7f440e19c700 1 -- 172.21.15.97:0/3586362718 >> v1:172.21.15.146:6789/0 conn(0x7f44080cd8e0 legacy=0x7f44080cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:58.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.725+0000 7f440e19c700 1 -- 172.21.15.97:0/3586362718 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4408071610 con 0x7f4408070c30 2023-07-06T03:53:58.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.725+0000 7f440e19c700 1 --2- 172.21.15.97:0/3586362718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4408070c30 0x7f44080adea0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f4400009e50 tx=0x7f440000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baeb71400305496c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:58.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.726+0000 7f440c999700 1 -- 172.21.15.97:0/3586362718 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440000e070 con 0x7f4408070c30 2023-07-06T03:53:58.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.726+0000 7f440c999700 1 -- 172.21.15.97:0/3586362718 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440000b9c0 con 0x7f4408070c30 2023-07-06T03:53:58.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.726+0000 7f440c999700 1 -- 172.21.15.97:0/3586362718 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440000bd70 con 0x7f4408070c30 2023-07-06T03:53:58.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.726+0000 7f440fbff700 1 -- 172.21.15.97:0/3586362718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4408070c30 msgr2=0x7f44080adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:58.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.726+0000 7f440fbff700 1 --2- 172.21.15.97:0/3586362718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4408070c30 0x7f44080adea0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f4400009e50 tx=0x7f440000b330 comp rx=0 tx=0).stop 2023-07-06T03:53:58.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.726+0000 7f440fbff700 1 -- 172.21.15.97:0/3586362718 shutdown_connections 2023-07-06T03:53:58.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.726+0000 7f440fbff700 1 --2- 172.21.15.97:0/3586362718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4408070c30 0x7f44080adea0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:58.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.726+0000 7f440fbff700 1 --2- 172.21.15.97:0/3586362718 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f440815b440 0x7f440815e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:58.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.726+0000 7f440fbff700 1 -- 172.21.15.97:0/3586362718 >> 172.21.15.97:0/3586362718 conn(0x7f44080ce8c0 msgr2=0x7f44080cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:58.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.726+0000 7f440fbff700 1 -- 172.21.15.97:0/3586362718 shutdown_connections 2023-07-06T03:53:58.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.726+0000 7f440fbff700 1 -- 172.21.15.97:0/3586362718 wait complete. 2023-07-06T03:53:58.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.728+0000 7f440fbff700 1 Processor -- start 2023-07-06T03:53:58.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.728+0000 7f440fbff700 1 -- start start 2023-07-06T03:53:58.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.729+0000 7f440fbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4408070c30 0x7f440814aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:58.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.729+0000 7f440fbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44080cd8e0 0x7f440814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:58.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.729+0000 7f440fbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440815b440 0x7f440806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:58.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.729+0000 7f440fbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440814bdf0 con 0x7f440815b440 2023-07-06T03:53:58.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.729+0000 7f440fbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440806d500 con 0x7f44080cd8e0 2023-07-06T03:53:58.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.729+0000 7f440fbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440814b170 con 0x7f4408070c30 2023-07-06T03:53:58.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.729+0000 7f440e19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440815b440 0x7f440806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:58.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.729+0000 7f440e19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440815b440 0x7f440806cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60158/0 (socket says 172.21.15.97:60158) 2023-07-06T03:53:58.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.729+0000 7f440e19c700 1 -- 172.21.15.97:0/3071449379 learned_addr learned my addr 172.21.15.97:0/3071449379 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:58.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f440d19a700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44080cd8e0 0x7f440814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:58.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f440d99b700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4408070c30 0x7f440814aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:58.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f440e19c700 1 -- 172.21.15.97:0/3071449379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4408070c30 msgr2=0x7f440814aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:58.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f440e19c700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4408070c30 0x7f440814aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:58.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f440e19c700 1 -- 172.21.15.97:0/3071449379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44080cd8e0 msgr2=0x7f440814a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:58.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f440e19c700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44080cd8e0 0x7f440814a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:58.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f440e19c700 1 -- 172.21.15.97:0/3071449379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4400009b30 con 0x7f440815b440 2023-07-06T03:53:58.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f440d19a700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44080cd8e0 0x7f440814a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:58.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f440e19c700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440815b440 0x7f440806cfc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f440806b980 tx=0x7f4400004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:58.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f43fe7fc700 1 -- 172.21.15.97:0/3071449379 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400004960 con 0x7f440815b440 2023-07-06T03:53:58.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f43fe7fc700 1 -- 172.21.15.97:0/3071449379 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44000099b0 con 0x7f440815b440 2023-07-06T03:53:58.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f440fbff700 1 -- 172.21.15.97:0/3071449379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440814b450 con 0x7f440815b440 2023-07-06T03:53:58.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f43fe7fc700 1 -- 172.21.15.97:0/3071449379 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440001b670 con 0x7f440815b440 2023-07-06T03:53:58.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.730+0000 7f440fbff700 1 -- 172.21.15.97:0/3071449379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440814ba10 con 0x7f440815b440 2023-07-06T03:53:58.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.731+0000 7f440fbff700 1 -- 172.21.15.97:0/3071449379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4408070aa0 con 0x7f440815b440 2023-07-06T03:53:58.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.732+0000 7f43fe7fc700 1 -- 172.21.15.97:0/3071449379 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f440001b7d0 con 0x7f440815b440 2023-07-06T03:53:58.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.733+0000 7f43fe7fc700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f43ec06ace0 0x7f43ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:58.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.733+0000 7f43fe7fc700 1 -- 172.21.15.97:0/3071449379 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f440008aeb0 con 0x7f440815b440 2023-07-06T03:53:58.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.733+0000 7f440d99b700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f43ec06ace0 0x7f43ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:58.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.734+0000 7f440d99b700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f43ec06ace0 0x7f43ec06d190 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4404000f60 tx=0x7f4404009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:58.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.738+0000 7f43fe7fc700 1 -- 172.21.15.97:0/3071449379 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4400055ed0 con 0x7f440815b440 2023-07-06T03:53:58.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.985+0000 7f440fbff700 1 -- 172.21.15.97:0/3071449379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f440814bf30 con 0x7f440815b440 2023-07-06T03:53:58.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.985+0000 7f43fe7fc700 1 -- 172.21.15.97:0/3071449379 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f440814bf30 con 0x7f440815b440 2023-07-06T03:53:58.988 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:58.988 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:53:58.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 -- 172.21.15.97:0/3071449379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f43ec06ace0 msgr2=0x7f43ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:58.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f43ec06ace0 0x7f43ec06d190 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4404000f60 tx=0x7f4404009040 comp rx=0 tx=0).stop 2023-07-06T03:53:58.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 -- 172.21.15.97:0/3071449379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440815b440 msgr2=0x7f440806cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:58.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440815b440 0x7f440806cfc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f440806b980 tx=0x7f4400004ad0 comp rx=0 tx=0).stop 2023-07-06T03:53:58.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 -- 172.21.15.97:0/3071449379 shutdown_connections 2023-07-06T03:53:58.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f43ec06ace0 0x7f43ec06d190 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:58.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440815b440 0x7f440806cfc0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:58.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4408070c30 0x7f440814aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:58.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 --2- 172.21.15.97:0/3071449379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44080cd8e0 0x7f440814a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:58.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 -- 172.21.15.97:0/3071449379 >> 172.21.15.97:0/3071449379 conn(0x7f44080ce8c0 msgr2=0x7f4408070110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:58.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 -- 172.21.15.97:0/3071449379 shutdown_connections 2023-07-06T03:53:58.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:58.988+0000 7f440fbff700 1 -- 172.21.15.97:0/3071449379 wait complete. 2023-07-06T03:53:59.008 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-07-06T03:53:59.008 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-07-06T03:53:59.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:53:59.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.200+0000 7f4a2cbff700 1 Processor -- start 2023-07-06T03:53:59.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.201+0000 7f4a2cbff700 1 -- start start 2023-07-06T03:53:59.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.202+0000 7f4a2cbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815c520 0x7f4a280cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.202+0000 7f4a2cbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4a28160d00 0x7f4a280b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.202+0000 7f4a2cbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a280cdae0 con 0x7f4a28160180 2023-07-06T03:53:59.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.202+0000 7f4a2cbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a280cd8d0 con 0x7f4a28160d00 2023-07-06T03:53:59.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.202+0000 7f4a2cbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280b6640 con 0x7f4a2815c520 2023-07-06T03:53:59.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.202+0000 7f4a1ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4a28160180 0x7f4a281605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33330/0 (socket says 172.21.15.97:33330) 2023-07-06T03:53:59.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.202+0000 7f4a1ffff700 1 -- 172.21.15.97:0/3696974954 learned_addr learned my addr 172.21.15.97:0/3696974954 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:53:59.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.202+0000 7f4a2659c700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815c520 0x7f4a280cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.202+0000 7f4a26d9d700 1 --2- 172.21.15.97:0/3696974954 >> v2:172.21.15.146:3300/0 conn(0x7f4a28160d00 0x7f4a280b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.202+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133503756 0 0) 0x7f4a280cdae0 con 0x7f4a28160180 2023-07-06T03:53:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.202+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a140035b0 con 0x7f4a28160180 2023-07-06T03:53:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2153236147 0 0) 0x7f4a140035b0 con 0x7f4a28160180 2023-07-06T03:53:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815c520 msgr2=0x7f4a280cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815c520 0x7f4a280cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 >> v2:172.21.15.146:3300/0 conn(0x7f4a28160d00 msgr2=0x7f4a280b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 --2- 172.21.15.97:0/3696974954 >> v2:172.21.15.146:3300/0 conn(0x7f4a28160d00 0x7f4a280b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a280b6920 con 0x7f4a28160180 2023-07-06T03:53:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f4a10001670 con 0x7f4a28160180 2023-07-06T03:53:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 >> v1:172.21.15.97:6789/0 conn(0x7f4a28160180 legacy=0x7f4a281605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 0x7f4a14003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a14004430 0x7f4a140048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815c520 0x7f4a14004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280cd8d0 con 0x7f4a14003a00 2023-07-06T03:53:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280b6640 con 0x7f4a14004430 2023-07-06T03:53:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1400ba10 con 0x7f4a2815c520 2023-07-06T03:53:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a2659c700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 0x7f4a14003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a1ffff700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a14004430 0x7f4a140048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a26d9d700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815c520 0x7f4a14004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a2659c700 1 -- 172.21.15.97:0/3696974954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815c520 msgr2=0x7f4a14004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a2659c700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a2815c520 0x7f4a14004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a2659c700 1 -- 172.21.15.97:0/3696974954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a14004430 msgr2=0x7f4a140048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a2659c700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a14004430 0x7f4a140048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.203+0000 7f4a2659c700 1 -- 172.21.15.97:0/3696974954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a140035b0 con 0x7f4a14003a00 2023-07-06T03:53:59.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.204+0000 7f4a2659c700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 0x7f4a14003e10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009b60 tx=0x7f4a14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ab2462e3189a73 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:59.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.204+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1800e6c0 con 0x7f4a14003a00 2023-07-06T03:53:59.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.204+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1800e820 con 0x7f4a14003a00 2023-07-06T03:53:59.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.204+0000 7f4a25d9b700 1 -- 172.21.15.97:0/3696974954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1800eb90 con 0x7f4a14003a00 2023-07-06T03:53:59.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.204+0000 7f4a2cbff700 1 -- 172.21.15.97:0/3696974954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 msgr2=0x7f4a14003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.204+0000 7f4a2cbff700 1 --2- 172.21.15.97:0/3696974954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 0x7f4a14003e10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009b60 tx=0x7f4a14005880 comp rx=0 tx=0).stop 2023-07-06T03:53:59.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.204+0000 7f4a26d9d700 1 -- 172.21.15.97:0/3696974954 reap_dead start 2023-07-06T03:53:59.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.204+0000 7f4a2cbff700 1 -- 172.21.15.97:0/3696974954 shutdown_connections 2023-07-06T03:53:59.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.204+0000 7f4a2cbff700 1 -- 172.21.15.97:0/3696974954 >> 172.21.15.97:0/3696974954 conn(0x7f4a28071790 msgr2=0x7f4a28071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:59.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.205+0000 7f4a2cbff700 1 -- 172.21.15.97:0/3696974954 shutdown_connections 2023-07-06T03:53:59.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.205+0000 7f4a2cbff700 1 -- 172.21.15.97:0/3696974954 wait complete. 2023-07-06T03:53:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.206+0000 7f4a2cbff700 1 Processor -- start 2023-07-06T03:53:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.206+0000 7f4a2cbff700 1 -- start start 2023-07-06T03:53:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.207+0000 7f4a2cbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 0x7f4a281f1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.207+0000 7f4a2cbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a280d2400 0x7f4a280c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.207+0000 7f4a2cbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a280d3910 0x7f4a2815dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.207+0000 7f4a2cbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280d2870 con 0x7f4a14003a00 2023-07-06T03:53:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.207+0000 7f4a2cbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280d2b50 con 0x7f4a280d3910 2023-07-06T03:53:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.207+0000 7f4a2cbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280d2e30 con 0x7f4a280d2400 2023-07-06T03:53:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a2659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 0x7f4a281f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a2659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 0x7f4a281f1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60180/0 (socket says 172.21.15.97:60180) 2023-07-06T03:53:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a2659c700 1 -- 172.21.15.97:0/374634947 learned_addr learned my addr 172.21.15.97:0/374634947 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a1ffff700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a280d2400 0x7f4a280c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a26d9d700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a280d3910 0x7f4a2815dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a2659c700 1 -- 172.21.15.97:0/374634947 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a280d2400 msgr2=0x7f4a280c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a2659c700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a280d2400 0x7f4a280c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a2659c700 1 -- 172.21.15.97:0/374634947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a280d3910 msgr2=0x7f4a2815dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a2659c700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a280d3910 0x7f4a2815dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a2659c700 1 -- 172.21.15.97:0/374634947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a280d3110 con 0x7f4a14003a00 2023-07-06T03:53:59.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a26d9d700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a280d3910 0x7f4a2815dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:53:59.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.208+0000 7f4a2659c700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 0x7f4a281f1620 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4a18000c00 tx=0x7f4a180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:59.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.209+0000 7f4a1effd700 1 -- 172.21.15.97:0/374634947 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18021070 con 0x7f4a14003a00 2023-07-06T03:53:59.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.209+0000 7f4a1effd700 1 -- 172.21.15.97:0/374634947 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a180036a0 con 0x7f4a14003a00 2023-07-06T03:53:59.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.209+0000 7f4a2cbff700 1 -- 172.21.15.97:0/374634947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a280d33f0 con 0x7f4a14003a00 2023-07-06T03:53:59.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.209+0000 7f4a1effd700 1 -- 172.21.15.97:0/374634947 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1800e6c0 con 0x7f4a14003a00 2023-07-06T03:53:59.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.209+0000 7f4a2cbff700 1 -- 172.21.15.97:0/374634947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a281f1d40 con 0x7f4a14003a00 2023-07-06T03:53:59.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.210+0000 7f4a2cbff700 1 -- 172.21.15.97:0/374634947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a280d3610 con 0x7f4a14003a00 2023-07-06T03:53:59.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.210+0000 7f4a1effd700 1 -- 172.21.15.97:0/374634947 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a18025070 con 0x7f4a14003a00 2023-07-06T03:53:59.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.211+0000 7f4a1effd700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a0806ace0 0x7f4a0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.211+0000 7f4a1effd700 1 -- 172.21.15.97:0/374634947 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f4a1808a760 con 0x7f4a14003a00 2023-07-06T03:53:59.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.212+0000 7f4a1ffff700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a0806ace0 0x7f4a0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.212+0000 7f4a1ffff700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a0806ace0 0x7f4a0806d190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4a280d3d50 tx=0x7f4a10007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:59.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.216+0000 7f4a1effd700 1 -- 172.21.15.97:0/374634947 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4a180558a0 con 0x7f4a14003a00 2023-07-06T03:53:59.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.451+0000 7f4a2cbff700 1 -- 172.21.15.97:0/374634947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4a280ce390 con 0x7f4a14003a00 2023-07-06T03:53:59.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.452+0000 7f4a1effd700 1 -- 172.21.15.97:0/374634947 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f4a18058ec0 con 0x7f4a14003a00 2023-07-06T03:53:59.455 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:59.455 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:53:59.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 -- 172.21.15.97:0/374634947 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a0806ace0 msgr2=0x7f4a0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a0806ace0 0x7f4a0806d190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4a280d3d50 tx=0x7f4a10007040 comp rx=0 tx=0).stop 2023-07-06T03:53:59.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 -- 172.21.15.97:0/374634947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 msgr2=0x7f4a281f1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 0x7f4a281f1620 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4a18000c00 tx=0x7f4a180047c0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 -- 172.21.15.97:0/374634947 shutdown_connections 2023-07-06T03:53:59.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a0806ace0 0x7f4a0806d190 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a280d3910 0x7f4a2815dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a14003a00 0x7f4a281f1620 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 --2- 172.21.15.97:0/374634947 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a280d2400 0x7f4a280c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 -- 172.21.15.97:0/374634947 >> 172.21.15.97:0/374634947 conn(0x7f4a28071790 msgr2=0x7f4a280b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:59.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 -- 172.21.15.97:0/374634947 shutdown_connections 2023-07-06T03:53:59.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.454+0000 7f4a2cbff700 1 -- 172.21.15.97:0/374634947 wait complete. 2023-07-06T03:53:59.474 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-06T03:53:59.474 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2023-07-06T03:53:59.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:53:59.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.721+0000 7f639f5ff700 1 Processor -- start 2023-07-06T03:53:59.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.721+0000 7f639f5ff700 1 -- start start 2023-07-06T03:53:59.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.722+0000 7f639f5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f639815c520 0x7f63980cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639f5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6398160d00 0x7f63980b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639f5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63980cdae0 con 0x7f6398160180 2023-07-06T03:53:59.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639f5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63980cd8d0 con 0x7f6398160d00 2023-07-06T03:53:59.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639f5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63980b6640 con 0x7f639815c520 2023-07-06T03:53:59.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639db9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6398160d00 0x7f63980b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639db9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6398160d00 0x7f63980b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60188/0 (socket says 172.21.15.97:60188) 2023-07-06T03:53:59.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639db9c700 1 -- 172.21.15.97:0/1299244874 learned_addr learned my addr 172.21.15.97:0/1299244874 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:59.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639d39b700 1 --2- 172.21.15.97:0/1299244874 >> v2:172.21.15.146:3300/0 conn(0x7f639815c520 0x7f63980cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639db9c700 1 -- 172.21.15.97:0/1299244874 >> v2:172.21.15.146:3300/0 conn(0x7f639815c520 msgr2=0x7f63980cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:53:59.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639db9c700 1 --2- 172.21.15.97:0/1299244874 >> v2:172.21.15.146:3300/0 conn(0x7f639815c520 0x7f63980cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639db9c700 1 -- 172.21.15.97:0/1299244874 >> v1:172.21.15.146:6789/0 conn(0x7f6398160180 legacy=0x7f63981605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.723+0000 7f639db9c700 1 -- 172.21.15.97:0/1299244874 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63980b6920 con 0x7f6398160d00 2023-07-06T03:53:59.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.724+0000 7f639db9c700 1 --2- 172.21.15.97:0/1299244874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398160d00 0x7f63980b7400 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f63980b4c70 tx=0x7f639000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83109b75e25c6272 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:59.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.724+0000 7f638ffff700 1 -- 172.21.15.97:0/1299244874 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639000e070 con 0x7f6398160d00 2023-07-06T03:53:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.724+0000 7f638ffff700 1 -- 172.21.15.97:0/1299244874 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639000b9c0 con 0x7f6398160d00 2023-07-06T03:53:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.724+0000 7f638ffff700 1 -- 172.21.15.97:0/1299244874 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639000bd70 con 0x7f6398160d00 2023-07-06T03:53:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.724+0000 7f639f5ff700 1 -- 172.21.15.97:0/1299244874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398160d00 msgr2=0x7f63980b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.724+0000 7f639f5ff700 1 --2- 172.21.15.97:0/1299244874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398160d00 0x7f63980b7400 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f63980b4c70 tx=0x7f639000b330 comp rx=0 tx=0).stop 2023-07-06T03:53:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.724+0000 7f639f5ff700 1 -- 172.21.15.97:0/1299244874 shutdown_connections 2023-07-06T03:53:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.724+0000 7f639f5ff700 1 --2- 172.21.15.97:0/1299244874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398160d00 0x7f63980b7400 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.724+0000 7f639f5ff700 1 --2- 172.21.15.97:0/1299244874 >> v2:172.21.15.146:3300/0 conn(0x7f639815c520 0x7f63980cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.724+0000 7f639f5ff700 1 -- 172.21.15.97:0/1299244874 >> 172.21.15.97:0/1299244874 conn(0x7f6398071790 msgr2=0x7f6398071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:59.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.724+0000 7f639f5ff700 1 -- 172.21.15.97:0/1299244874 shutdown_connections 2023-07-06T03:53:59.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.725+0000 7f639f5ff700 1 -- 172.21.15.97:0/1299244874 wait complete. 2023-07-06T03:53:59.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.726+0000 7f639f5ff700 1 Processor -- start 2023-07-06T03:53:59.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.726+0000 7f639f5ff700 1 -- start start 2023-07-06T03:53:59.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.727+0000 7f639f5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f639815c520 0x7f63980d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.727+0000 7f639f5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398160180 0x7f639815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.727+0000 7f639f5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6398160d00 0x7f639814ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.727+0000 7f639f5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63980ccfd0 con 0x7f6398160180 2023-07-06T03:53:59.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.727+0000 7f639f5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63980d1180 con 0x7f6398160d00 2023-07-06T03:53:59.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.727+0000 7f639f5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63980d13b0 con 0x7f639815c520 2023-07-06T03:53:59.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639cb9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398160180 0x7f639815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639cb9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398160180 0x7f639815efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60200/0 (socket says 172.21.15.97:60200) 2023-07-06T03:53:59.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639cb9a700 1 -- 172.21.15.97:0/99425189 learned_addr learned my addr 172.21.15.97:0/99425189 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:53:59.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639d39b700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f639815c520 0x7f63980d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639db9c700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6398160d00 0x7f639814ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639cb9a700 1 -- 172.21.15.97:0/99425189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f639815c520 msgr2=0x7f63980d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639cb9a700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f639815c520 0x7f63980d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639cb9a700 1 -- 172.21.15.97:0/99425189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6398160d00 msgr2=0x7f639814ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639cb9a700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6398160d00 0x7f639814ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639cb9a700 1 -- 172.21.15.97:0/99425189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390009b30 con 0x7f6398160180 2023-07-06T03:53:59.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639cb9a700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398160180 0x7f639815efd0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f638800c4e0 tx=0x7f638800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:59.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f638dffb700 1 -- 172.21.15.97:0/99425189 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880052d0 con 0x7f6398160180 2023-07-06T03:53:59.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f638dffb700 1 -- 172.21.15.97:0/99425189 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6388005430 con 0x7f6398160180 2023-07-06T03:53:59.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f639f5ff700 1 -- 172.21.15.97:0/99425189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63980d1690 con 0x7f6398160180 2023-07-06T03:53:59.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.728+0000 7f638dffb700 1 -- 172.21.15.97:0/99425189 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880038c0 con 0x7f6398160180 2023-07-06T03:53:59.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.729+0000 7f639f5ff700 1 -- 172.21.15.97:0/99425189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63980d1c50 con 0x7f6398160180 2023-07-06T03:53:59.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.729+0000 7f639f5ff700 1 -- 172.21.15.97:0/99425189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639804e340 con 0x7f6398160180 2023-07-06T03:53:59.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.730+0000 7f638dffb700 1 -- 172.21.15.97:0/99425189 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6388003a20 con 0x7f6398160180 2023-07-06T03:53:59.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.732+0000 7f638dffb700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 0x7f637c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:53:59.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.732+0000 7f638dffb700 1 -- 172.21.15.97:0/99425189 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f63880889b0 con 0x7f6398160180 2023-07-06T03:53:59.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.732+0000 7f639d39b700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 0x7f637c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:53:59.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.733+0000 7f639d39b700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 0x7f637c06d190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6394007a30 tx=0x7f6394008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:53:59.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.736+0000 7f638dffb700 1 -- 172.21.15.97:0/99425189 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f63880539d0 con 0x7f6398160180 2023-07-06T03:53:59.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.982+0000 7f639f5ff700 1 -- 172.21.15.97:0/99425189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f639815bd40 con 0x7f6398160180 2023-07-06T03:53:59.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.982+0000 7f638dffb700 1 -- 172.21.15.97:0/99425189 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f6388056ff0 con 0x7f6398160180 2023-07-06T03:53:59.985 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:53:59.986 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/387292825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":387292825},{"type":"v1","addr":"172.21.15.146:6833","nonce":387292825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690380561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690380561},{"type":"v1","addr":"172.21.15.97:6837","nonce":690380561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3897834271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3897834271},{"type":"v1","addr":"172.21.15.146:6835","nonce":3897834271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:53:26.234190+0000","modified":"2023-07-06T03:53:55.763145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":10,"addr":"172.21.15.97:6836/3324055785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3324055785},{"type":"v1","addr":"172.21.15.97:6836","nonce":3324055785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-06T03:53:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.984+0000 7f639f5ff700 1 -- 172.21.15.97:0/99425189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 msgr2=0x7f637c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.984+0000 7f639f5ff700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 0x7f637c06d190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6394007a30 tx=0x7f6394008040 comp rx=0 tx=0).stop 2023-07-06T03:53:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.984+0000 7f639f5ff700 1 -- 172.21.15.97:0/99425189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398160180 msgr2=0x7f639815efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:53:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.984+0000 7f639f5ff700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398160180 0x7f639815efd0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f638800c4e0 tx=0x7f638800c7f0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.985+0000 7f639f5ff700 1 -- 172.21.15.97:0/99425189 shutdown_connections 2023-07-06T03:53:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.985+0000 7f639f5ff700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f637c06ace0 0x7f637c06d190 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.985+0000 7f639f5ff700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6398160d00 0x7f639814ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.985+0000 7f639f5ff700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f639815c520 0x7f63980d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.985+0000 7f639f5ff700 1 --2- 172.21.15.97:0/99425189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6398160180 0x7f639815efd0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:53:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.985+0000 7f639f5ff700 1 -- 172.21.15.97:0/99425189 >> 172.21.15.97:0/99425189 conn(0x7f6398071790 msgr2=0x7f63980b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:53:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.985+0000 7f639f5ff700 1 -- 172.21.15.97:0/99425189 shutdown_connections 2023-07-06T03:53:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:53:59.985+0000 7f639f5ff700 1 -- 172.21.15.97:0/99425189 wait complete. 2023-07-06T03:53:59.989 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 69 2023-07-06T03:54:00.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-07-06T03:54:00.224 INFO:teuthology.orchestra.run.smithi097.stdout:[{"id":7495,"entity":{"name":{"type":"client","num":7495},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1754496776}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":18.142699976999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.8972821291559478,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7495 192.168.0.1:0/1754496776","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"125117","root":"/"}},{"id":8199,"entity":{"name":{"type":"client","num":8199},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2529580715}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":1057,"uptime":20.436662041000002,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":4831.8174084088196,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":882.38463878078085,"halflife":1.5},"recall_caps_throttle2o":{"value":292.5946251684947,"halflife":0.5},"session_cache_liveness":{"value":1559.136215337975,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":506,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8199 192.168.0.1:0/2529580715","completed_requests":[{"tid":506,"created_ino":"0x100000003e9"},{"tid":507,"created_ino":"0x100000003ea"},{"tid":508,"created_ino":"0x100000003eb"},{"tid":509,"created_ino":"0x100000003ec"},{"tid":510,"created_ino":"0x100000003ed"},{"tid":511,"created_ino":"0x100000003ee"},{"tid":512,"created_ino":"0x100000003ef"},{"tid":513,"created_ino":"0x100000003f0"},{"tid":514,"created_ino":"0x100000003f1"},{"tid":515,"created_ino":"0x100000003f2"},{"tid":516,"created_ino":"0x100000003f3"},{"tid":517,"created_ino":"0x100000003f4"},{"tid":518,"created_ino":"0x100000003f5"},{"tid":519,"created_ino":"0x100000003f6"},{"tid":520,"created_ino":"0x100000003f7"},{"tid":521,"created_ino":"0x100000003f8"},{"tid":522,"created_ino":"0x100000003f9"},{"tid":523,"created_ino":"0x100000003fa"},{"tid":524,"created_ino":"0x100000003fb"},{"tid":525,"created_ino":"0x100000003fc"},{"tid":526,"created_ino":"0x100000003fd"},{"tid":527,"created_ino":"0x100000003fe"},{"tid":528,"created_ino":"0x100000003ff"},{"tid":529,"created_ino":"0x10000000400"},{"tid":530,"created_ino":"0x10000000401"},{"tid":531,"created_ino":"0x10000000402"},{"tid":532,"created_ino":"0x10000000403"},{"tid":533,"created_ino":"0x10000000404"},{"tid":534,"created_ino":"0x10000000405"},{"tid":535,"created_ino":"0x10000000406"},{"tid":536,"created_ino":"0x10000000407"},{"tid":537,"created_ino":"0x10000000408"},{"tid":538,"created_ino":"0x10000000409"},{"tid":539,"created_ino":"0x1000000040a"},{"tid":540,"created_ino":"0x1000000040b"},{"tid":541,"created_ino":"0x1000000040c"},{"tid":542,"created_ino":"0x1000000040d"},{"tid":543,"created_ino":"0x1000000040e"},{"tid":544,"created_ino":"0x1000000040f"},{"tid":545,"created_ino":"0x10000000410"},{"tid":546,"created_ino":"0x10000000411"},{"tid":547,"created_ino":"0x10000000412"},{"tid":548,"created_ino":"0x10000000413"},{"tid":549,"created_ino":"0x10000000414"},{"tid":550,"created_ino":"0x10000000415"},{"tid":551,"created_ino":"0x10000000416"},{"tid":552,"created_ino":"0x10000000417"},{"tid":553,"created_ino":"0x10000000418"},{"tid":554,"created_ino":"0x10000000419"},{"tid":555,"created_ino":"0x1000000041a"},{"tid":556,"created_ino":"0x1000000041b"},{"tid":557,"created_ino":"0x1000000041c"},{"tid":558,"created_ino":"0x1000000041d"},{"tid":559,"created_ino":"0x1000000041e"},{"tid":560,"created_ino":"0x1000000041f"},{"tid":561,"created_ino":"0x10000000420"},{"tid":562,"created_ino":"0x10000000421"},{"tid":563,"created_ino":"0x10000000422"},{"tid":564,"created_ino":"0x10000000423"},{"tid":565,"created_ino":"0x10000000424"},{"tid":566,"created_ino":"0x10000000425"},{"tid":567,"created_ino":"0x10000000426"},{"tid":568,"created_ino":"0x10000000427"},{"tid":569,"created_ino":"0x10000000428"},{"tid":570,"created_ino":"0x10000000429"},{"tid":571,"created_ino":"0x1000000042a"},{"tid":572,"created_ino":"0x1000000042b"},{"tid":573,"created_ino":"0x1000000042c"},{"tid":574,"created_ino":"0x1000000042d"},{"tid":575,"created_ino":"0x1000000042e"},{"tid":576,"created_ino":"0x1000000042f"},{"tid":577,"created_ino":"0x10000000430"},{"tid":578,"created_ino":"0x10000000431"},{"tid":579,"created_ino":"0x10000000432"},{"tid":580,"created_ino":"0x10000000433"},{"tid":581,"created_ino":"0x10000000434"},{"tid":582,"created_ino":"0x10000000435"},{"tid":583,"created_ino":"0x10000000436"},{"tid":584,"created_ino":"0x10000000437"},{"tid":585,"created_ino":"0x10000000438"},{"tid":586,"created_ino":"0x10000000439"},{"tid":587,"created_ino":"0x1000000043a"},{"tid":588,"created_ino":"0x1000000043b"},{"tid":589,"created_ino":"0x1000000043c"},{"tid":590,"created_ino":"0x1000000043d"},{"tid":591,"created_ino":"0x1000000043e"},{"tid":592,"created_ino":"0x1000000043f"},{"tid":593,"created_ino":"0x10000000440"},{"tid":594,"created_ino":"0x10000000441"},{"tid":595,"created_ino":"0x10000000442"},{"tid":598,"created_ino":"0x10000000443"},{"tid":602,"created_ino":"0x10000000444"},{"tid":605,"created_ino":"0x10000000445"},{"tid":608,"created_ino":"0x10000000446"},{"tid":611,"created_ino":"0x10000000447"},{"tid":614,"created_ino":"0x10000000448"},{"tid":617,"created_ino":"0x10000000449"},{"tid":620,"created_ino":"0x1000000044a"},{"tid":623,"created_ino":"0x1000000044b"},{"tid":626,"created_ino":"0x1000000044c"},{"tid":629,"created_ino":"0x1000000044d"},{"tid":632,"created_ino":"0x1000000044e"},{"tid":635,"created_ino":"0x1000000044f"},{"tid":638,"created_ino":"0x10000000450"},{"tid":641,"created_ino":"0x10000000451"},{"tid":644,"created_ino":"0x10000000452"},{"tid":647,"created_ino":"0x10000000453"},{"tid":650,"created_ino":"0x10000000454"},{"tid":653,"created_ino":"0x10000000455"},{"tid":656,"created_ino":"0x10000000456"},{"tid":659,"created_ino":"0x10000000457"},{"tid":662,"created_ino":"0x10000000458"},{"tid":665,"created_ino":"0x10000000459"},{"tid":668,"created_ino":"0x1000000045a"},{"tid":671,"created_ino":"0x1000000045b"},{"tid":674,"created_ino":"0x1000000045c"},{"tid":677,"created_ino":"0x1000000045d"},{"tid":680,"created_ino":"0x1000000045e"},{"tid":683,"created_ino":"0x1000000045f"},{"tid":686,"created_ino":"0x10000000460"},{"tid":689,"created_ino":"0x10000000461"},{"tid":692,"created_ino":"0x10000000462"},{"tid":695,"created_ino":"0x10000000463"},{"tid":698,"created_ino":"0x10000000464"},{"tid":701,"created_ino":"0x10000000465"},{"tid":704,"created_ino":"0x10000000466"},{"tid":707,"created_ino":"0x10000000467"},{"tid":710,"created_ino":"0x10000000468"},{"tid":713,"created_ino":"0x10000000469"},{"tid":716,"created_ino":"0x1000000046a"},{"tid":719,"created_ino":"0x1000000046b"},{"tid":722,"created_ino":"0x1000000046c"},{"tid":725,"created_ino":"0x1000000046d"},{"tid":728,"created_ino":"0x1000000046e"},{"tid":731,"created_ino":"0x1000000046f"},{"tid":734,"created_ino":"0x10000000470"},{"tid":737,"created_ino":"0x10000000471"},{"tid":740,"created_ino":"0x10000000472"},{"tid":743,"created_ino":"0x10000000473"},{"tid":746,"created_ino":"0x10000000474"},{"tid":749,"created_ino":"0x10000000475"},{"tid":752,"created_ino":"0x10000000476"},{"tid":755,"created_ino":"0x10000000477"},{"tid":758,"created_ino":"0x10000000478"},{"tid":761,"created_ino":"0x10000000479"},{"tid":764,"created_ino":"0x1000000047a"},{"tid":767,"created_ino":"0x1000000047b"},{"tid":770,"created_ino":"0x1000000047c"},{"tid":773,"created_ino":"0x1000000047d"},{"tid":776,"created_ino":"0x1000000047e"},{"tid":779,"created_ino":"0x1000000047f"},{"tid":782,"created_ino":"0x10000000480"},{"tid":785,"created_ino":"0x10000000481"},{"tid":788,"created_ino":"0x10000000482"},{"tid":791,"created_ino":"0x10000000483"},{"tid":794,"created_ino":"0x10000000484"},{"tid":797,"created_ino":"0x10000000485"},{"tid":800,"created_ino":"0x10000000486"},{"tid":803,"created_ino":"0x10000000487"},{"tid":806,"created_ino":"0x10000000488"},{"tid":809,"created_ino":"0x10000000489"},{"tid":812,"created_ino":"0x1000000048a"},{"tid":815,"created_ino":"0x1000000048b"},{"tid":818,"created_ino":"0x1000000048c"},{"tid":821,"created_ino":"0x1000000048d"},{"tid":824,"created_ino":"0x1000000048e"},{"tid":827,"created_ino":"0x1000000048f"},{"tid":830,"created_ino":"0x10000000490"},{"tid":833,"created_ino":"0x10000000491"},{"tid":836,"created_ino":"0x10000000492"},{"tid":839,"created_ino":"0x10000000493"},{"tid":842,"created_ino":"0x10000000494"},{"tid":845,"created_ino":"0x10000000495"},{"tid":848,"created_ino":"0x10000000496"},{"tid":851,"created_ino":"0x10000000497"},{"tid":854,"created_ino":"0x10000000498"},{"tid":857,"created_ino":"0x10000000499"},{"tid":860,"created_ino":"0x1000000049a"},{"tid":863,"created_ino":"0x1000000049b"},{"tid":866,"created_ino":"0x1000000049c"},{"tid":869,"created_ino":"0x1000000049d"},{"tid":872,"created_ino":"0x1000000049e"},{"tid":875,"created_ino":"0x1000000049f"},{"tid":878,"created_ino":"0x100000004a0"},{"tid":881,"created_ino":"0x100000004a1"},{"tid":884,"created_ino":"0x100000004a2"},{"tid":887,"created_ino":"0x100000004a3"},{"tid":890,"created_ino":"0x100000004a4"},{"tid":893,"created_ino":"0x100000004a5"},{"tid":896,"created_ino":"0x100000004a6"},{"tid":899,"created_ino":"0x100000004a7"},{"tid":902,"created_ino":"0x100000004a8"},{"tid":905,"created_ino":"0x100000004a9"},{"tid":908,"created_ino":"0x100000004aa"},{"tid":911,"created_ino":"0x100000004ab"},{"tid":914,"created_ino":"0x100000004ac"},{"tid":917,"created_ino":"0x100000004ad"},{"tid":920,"created_ino":"0x100000004ae"},{"tid":923,"created_ino":"0x100000004af"},{"tid":926,"created_ino":"0x100000004b0"},{"tid":929,"created_ino":"0x100000004b1"},{"tid":932,"created_ino":"0x100000004b2"},{"tid":935,"created_ino":"0x100000004b3"},{"tid":938,"created_ino":"0x100000004b4"},{"tid":941,"created_ino":"0x100000004b5"},{"tid":944,"created_ino":"0x100000004b6"},{"tid":947,"created_ino":"0x100000004b7"},{"tid":950,"created_ino":"0x100000004b8"},{"tid":953,"created_ino":"0x100000004b9"},{"tid":956,"created_ino":"0x100000004ba"},{"tid":959,"created_ino":"0x100000004bb"},{"tid":962,"created_ino":"0x100000004bc"},{"tid":965,"created_ino":"0x100000004bd"},{"tid":968,"created_ino":"0x100000004be"},{"tid":971,"created_ino":"0x100000004bf"},{"tid":974,"created_ino":"0x100000004c0"},{"tid":977,"created_ino":"0x100000004c1"},{"tid":980,"created_ino":"0x100000004c2"},{"tid":983,"created_ino":"0x100000004c3"},{"tid":986,"created_ino":"0x100000004c4"},{"tid":989,"created_ino":"0x100000004c5"},{"tid":992,"created_ino":"0x100000004c6"},{"tid":995,"created_ino":"0x100000004c7"},{"tid":998,"created_ino":"0x100000004c8"},{"tid":1001,"created_ino":"0x100000004c9"},{"tid":1004,"created_ino":"0x100000004ca"},{"tid":1007,"created_ino":"0x100000004cb"},{"tid":1010,"created_ino":"0x100000004cc"},{"tid":1013,"created_ino":"0x100000004cd"},{"tid":1016,"created_ino":"0x100000004ce"},{"tid":1019,"created_ino":"0x100000004cf"},{"tid":1022,"created_ino":"0x100000004d0"},{"tid":1025,"created_ino":"0x100000004d1"},{"tid":1028,"created_ino":"0x100000004d2"},{"tid":1031,"created_ino":"0x100000004d3"},{"tid":1034,"created_ino":"0x100000004d4"},{"tid":1037,"created_ino":"0x100000004d5"},{"tid":1040,"created_ino":"0x100000004d6"},{"tid":1043,"created_ino":"0x100000004d7"},{"tid":1046,"created_ino":"0x100000004d8"},{"tid":1049,"created_ino":"0x100000004d9"},{"tid":1052,"created_ino":"0x100000004da"},{"tid":1055,"created_ino":"0x100000004db"},{"tid":1058,"created_ino":"0x100000004dc"},{"tid":1061,"created_ino":"0x100000004dd"},{"tid":1064,"created_ino":"0x100000004de"},{"tid":1067,"created_ino":"0x100000004df"},{"tid":1070,"created_ino":"0x100000004e0"},{"tid":1073,"created_ino":"0x100000004e1"},{"tid":1076,"created_ino":"0x100000004e2"},{"tid":1079,"created_ino":"0x100000004e3"},{"tid":1082,"created_ino":"0x100000004e4"},{"tid":1085,"created_ino":"0x100000004e5"},{"tid":1088,"created_ino":"0x100000004e6"},{"tid":1091,"created_ino":"0x100000004e7"},{"tid":1094,"created_ino":"0x100000004e8"},{"tid":1097,"created_ino":"0x100000004e9"},{"tid":1100,"created_ino":"0x100000004ea"},{"tid":1103,"created_ino":"0x100000004eb"},{"tid":1106,"created_ino":"0x100000004ec"},{"tid":1109,"created_ino":"0x100000004ed"},{"tid":1112,"created_ino":"0x100000004ee"},{"tid":1115,"created_ino":"0x100000004ef"},{"tid":1118,"created_ino":"0x100000004f0"},{"tid":1121,"created_ino":"0x100000004f1"},{"tid":1124,"created_ino":"0x100000004f2"},{"tid":1127,"created_ino":"0x100000004f3"},{"tid":1130,"created_ino":"0x100000004f4"},{"tid":1133,"created_ino":"0x100000004f5"},{"tid":1136,"created_ino":"0x100000004f6"},{"tid":1139,"created_ino":"0x100000004f7"},{"tid":1142,"created_ino":"0x100000004f8"},{"tid":1145,"created_ino":"0x100000004f9"},{"tid":1148,"created_ino":"0x100000004fa"},{"tid":1151,"created_ino":"0x100000004fb"},{"tid":1154,"created_ino":"0x100000004fc"},{"tid":1157,"created_ino":"0x100000004fd"},{"tid":1160,"created_ino":"0x100000004fe"},{"tid":1163,"created_ino":"0x100000004ff"},{"tid":1166,"created_ino":"0x10000000500"},{"tid":1169,"created_ino":"0x10000000501"},{"tid":1172,"created_ino":"0x10000000502"},{"tid":1175,"created_ino":"0x10000000503"},{"tid":1178,"created_ino":"0x10000000504"},{"tid":1181,"created_ino":"0x10000000505"},{"tid":1184,"created_ino":"0x10000000506"},{"tid":1187,"created_ino":"0x10000000507"},{"tid":1190,"created_ino":"0x10000000508"},{"tid":1193,"created_ino":"0x10000000509"},{"tid":1196,"created_ino":"0x1000000050a"},{"tid":1199,"created_ino":"0x1000000050b"},{"tid":1202,"created_ino":"0x1000000050c"},{"tid":1205,"created_ino":"0x1000000050d"},{"tid":1208,"created_ino":"0x1000000050e"},{"tid":1211,"created_ino":"0x1000000050f"},{"tid":1214,"created_ino":"0x10000000510"},{"tid":1217,"created_ino":"0x10000000511"},{"tid":1220,"created_ino":"0x10000000512"},{"tid":1223,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"154228","root":"/"}}] 2023-07-06T03:54:00.227 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "125117", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1754496776, "type": "any" }, "name": { "num": 7495, "type": "client" } }, "id": 7495, "inst": "client.7495 192.168.0.1:0/1754496776", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.897282129155948 }, "state": "open", "uptime": 18.142699977 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "154228", "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 506 }, { "created_ino": "0x100000003ea", "tid": 507 }, { "created_ino": "0x100000003eb", "tid": 508 }, { "created_ino": "0x100000003ec", "tid": 509 }, { "created_ino": "0x100000003ed", "tid": 510 }, { "created_ino": "0x100000003ee", "tid": 511 }, { "created_ino": "0x100000003ef", "tid": 512 }, { "created_ino": "0x100000003f0", "tid": 513 }, { "created_ino": "0x100000003f1", "tid": 514 }, { "created_ino": "0x100000003f2", "tid": 515 }, { "created_ino": "0x100000003f3", "tid": 516 }, { "created_ino": "0x100000003f4", "tid": 517 }, { "created_ino": "0x100000003f5", "tid": 518 }, { "created_ino": "0x100000003f6", "tid": 519 }, { "created_ino": "0x100000003f7", "tid": 520 }, { "created_ino": "0x100000003f8", "tid": 521 }, { "created_ino": "0x100000003f9", "tid": 522 }, { "created_ino": "0x100000003fa", "tid": 523 }, { "created_ino": "0x100000003fb", "tid": 524 }, { "created_ino": "0x100000003fc", "tid": 525 }, { "created_ino": "0x100000003fd", "tid": 526 }, { "created_ino": "0x100000003fe", "tid": 527 }, { "created_ino": "0x100000003ff", "tid": 528 }, { "created_ino": "0x10000000400", "tid": 529 }, { "created_ino": "0x10000000401", "tid": 530 }, { "created_ino": "0x10000000402", "tid": 531 }, { "created_ino": "0x10000000403", "tid": 532 }, { "created_ino": "0x10000000404", "tid": 533 }, { "created_ino": "0x10000000405", "tid": 534 }, { "created_ino": "0x10000000406", "tid": 535 }, { "created_ino": "0x10000000407", "tid": 536 }, { "created_ino": "0x10000000408", "tid": 537 }, { "created_ino": "0x10000000409", "tid": 538 }, { "created_ino": "0x1000000040a", "tid": 539 }, { "created_ino": "0x1000000040b", "tid": 540 }, { "created_ino": "0x1000000040c", "tid": 541 }, { "created_ino": "0x1000000040d", "tid": 542 }, { "created_ino": "0x1000000040e", "tid": 543 }, { "created_ino": "0x1000000040f", "tid": 544 }, { "created_ino": "0x10000000410", "tid": 545 }, { "created_ino": "0x10000000411", "tid": 546 }, { "created_ino": "0x10000000412", "tid": 547 }, { "created_ino": "0x10000000413", "tid": 548 }, { "created_ino": "0x10000000414", "tid": 549 }, { "created_ino": "0x10000000415", "tid": 550 }, { "created_ino": "0x10000000416", "tid": 551 }, { "created_ino": "0x10000000417", "tid": 552 }, { "created_ino": "0x10000000418", "tid": 553 }, { "created_ino": "0x10000000419", "tid": 554 }, { "created_ino": "0x1000000041a", "tid": 555 }, { "created_ino": "0x1000000041b", "tid": 556 }, { "created_ino": "0x1000000041c", "tid": 557 }, { "created_ino": "0x1000000041d", "tid": 558 }, { "created_ino": "0x1000000041e", "tid": 559 }, { "created_ino": "0x1000000041f", "tid": 560 }, { "created_ino": "0x10000000420", "tid": 561 }, { "created_ino": "0x10000000421", "tid": 562 }, { "created_ino": "0x10000000422", "tid": 563 }, { "created_ino": "0x10000000423", "tid": 564 }, { "created_ino": "0x10000000424", "tid": 565 }, { "created_ino": "0x10000000425", "tid": 566 }, { "created_ino": "0x10000000426", "tid": 567 }, { "created_ino": "0x10000000427", "tid": 568 }, { "created_ino": "0x10000000428", "tid": 569 }, { "created_ino": "0x10000000429", "tid": 570 }, { "created_ino": "0x1000000042a", "tid": 571 }, { "created_ino": "0x1000000042b", "tid": 572 }, { "created_ino": "0x1000000042c", "tid": 573 }, { "created_ino": "0x1000000042d", "tid": 574 }, { "created_ino": "0x1000000042e", "tid": 575 }, { "created_ino": "0x1000000042f", "tid": 576 }, { "created_ino": "0x10000000430", "tid": 577 }, { "created_ino": "0x10000000431", "tid": 578 }, { "created_ino": "0x10000000432", "tid": 579 }, { "created_ino": "0x10000000433", "tid": 580 }, { "created_ino": "0x10000000434", "tid": 581 }, { "created_ino": "0x10000000435", "tid": 582 }, { "created_ino": "0x10000000436", "tid": 583 }, { "created_ino": "0x10000000437", "tid": 584 }, { "created_ino": "0x10000000438", "tid": 585 }, { "created_ino": "0x10000000439", "tid": 586 }, { "created_ino": "0x1000000043a", "tid": 587 }, { "created_ino": "0x1000000043b", "tid": 588 }, { "created_ino": "0x1000000043c", "tid": 589 }, { "created_ino": "0x1000000043d", "tid": 590 }, { "created_ino": "0x1000000043e", "tid": 591 }, { "created_ino": "0x1000000043f", "tid": 592 }, { "created_ino": "0x10000000440", "tid": 593 }, { "created_ino": "0x10000000441", "tid": 594 }, { "created_ino": "0x10000000442", "tid": 595 }, { "created_ino": "0x10000000443", "tid": 598 }, { "created_ino": "0x10000000444", "tid": 602 }, { "created_ino": "0x10000000445", "tid": 605 }, { "created_ino": "0x10000000446", "tid": 608 }, { "created_ino": "0x10000000447", "tid": 611 }, { "created_ino": "0x10000000448", "tid": 614 }, { "created_ino": "0x10000000449", "tid": 617 }, { "created_ino": "0x1000000044a", "tid": 620 }, { "created_ino": "0x1000000044b", "tid": 623 }, { "created_ino": "0x1000000044c", "tid": 626 }, { "created_ino": "0x1000000044d", "tid": 629 }, { "created_ino": "0x1000000044e", "tid": 632 }, { "created_ino": "0x1000000044f", "tid": 635 }, { "created_ino": "0x10000000450", "tid": 638 }, { "created_ino": "0x10000000451", "tid": 641 }, { "created_ino": "0x10000000452", "tid": 644 }, { "created_ino": "0x10000000453", "tid": 647 }, { "created_ino": "0x10000000454", "tid": 650 }, { "created_ino": "0x10000000455", "tid": 653 }, { "created_ino": "0x10000000456", "tid": 656 }, { "created_ino": "0x10000000457", "tid": 659 }, { "created_ino": "0x10000000458", "tid": 662 }, { "created_ino": "0x10000000459", "tid": 665 }, { "created_ino": "0x1000000045a", "tid": 668 }, { "created_ino": "0x1000000045b", "tid": 671 }, { "created_ino": "0x1000000045c", "tid": 674 }, { "created_ino": "0x1000000045d", "tid": 677 }, { "created_ino": "0x1000000045e", "tid": 680 }, { "created_ino": "0x1000000045f", "tid": 683 }, { "created_ino": "0x10000000460", "tid": 686 }, { "created_ino": "0x10000000461", "tid": 689 }, { "created_ino": "0x10000000462", "tid": 692 }, { "created_ino": "0x10000000463", "tid": 695 }, { "created_ino": "0x10000000464", "tid": 698 }, { "created_ino": "0x10000000465", "tid": 701 }, { "created_ino": "0x10000000466", "tid": 704 }, { "created_ino": "0x10000000467", "tid": 707 }, { "created_ino": "0x10000000468", "tid": 710 }, { "created_ino": "0x10000000469", "tid": 713 }, { "created_ino": "0x1000000046a", "tid": 716 }, { "created_ino": "0x1000000046b", "tid": 719 }, { "created_ino": "0x1000000046c", "tid": 722 }, { "created_ino": "0x1000000046d", "tid": 725 }, { "created_ino": "0x1000000046e", "tid": 728 }, { "created_ino": "0x1000000046f", "tid": 731 }, { "created_ino": "0x10000000470", "tid": 734 }, { "created_ino": "0x10000000471", "tid": 737 }, { "created_ino": "0x10000000472", "tid": 740 }, { "created_ino": "0x10000000473", "tid": 743 }, { "created_ino": "0x10000000474", "tid": 746 }, { "created_ino": "0x10000000475", "tid": 749 }, { "created_ino": "0x10000000476", "tid": 752 }, { "created_ino": "0x10000000477", "tid": 755 }, { "created_ino": "0x10000000478", "tid": 758 }, { "created_ino": "0x10000000479", "tid": 761 }, { "created_ino": "0x1000000047a", "tid": 764 }, { "created_ino": "0x1000000047b", "tid": 767 }, { "created_ino": "0x1000000047c", "tid": 770 }, { "created_ino": "0x1000000047d", "tid": 773 }, { "created_ino": "0x1000000047e", "tid": 776 }, { "created_ino": "0x1000000047f", "tid": 779 }, { "created_ino": "0x10000000480", "tid": 782 }, { "created_ino": "0x10000000481", "tid": 785 }, { "created_ino": "0x10000000482", "tid": 788 }, { "created_ino": "0x10000000483", "tid": 791 }, { "created_ino": "0x10000000484", "tid": 794 }, { "created_ino": "0x10000000485", "tid": 797 }, { "created_ino": "0x10000000486", "tid": 800 }, { "created_ino": "0x10000000487", "tid": 803 }, { "created_ino": "0x10000000488", "tid": 806 }, { "created_ino": "0x10000000489", "tid": 809 }, { "created_ino": "0x1000000048a", "tid": 812 }, { "created_ino": "0x1000000048b", "tid": 815 }, { "created_ino": "0x1000000048c", "tid": 818 }, { "created_ino": "0x1000000048d", "tid": 821 }, { "created_ino": "0x1000000048e", "tid": 824 }, { "created_ino": "0x1000000048f", "tid": 827 }, { "created_ino": "0x10000000490", "tid": 830 }, { "created_ino": "0x10000000491", "tid": 833 }, { "created_ino": "0x10000000492", "tid": 836 }, { "created_ino": "0x10000000493", "tid": 839 }, { "created_ino": "0x10000000494", "tid": 842 }, { "created_ino": "0x10000000495", "tid": 845 }, { "created_ino": "0x10000000496", "tid": 848 }, { "created_ino": "0x10000000497", "tid": 851 }, { "created_ino": "0x10000000498", "tid": 854 }, { "created_ino": "0x10000000499", "tid": 857 }, { "created_ino": "0x1000000049a", "tid": 860 }, { "created_ino": "0x1000000049b", "tid": 863 }, { "created_ino": "0x1000000049c", "tid": 866 }, { "created_ino": "0x1000000049d", "tid": 869 }, { "created_ino": "0x1000000049e", "tid": 872 }, { "created_ino": "0x1000000049f", "tid": 875 }, { "created_ino": "0x100000004a0", "tid": 878 }, { "created_ino": "0x100000004a1", "tid": 881 }, { "created_ino": "0x100000004a2", "tid": 884 }, { "created_ino": "0x100000004a3", "tid": 887 }, { "created_ino": "0x100000004a4", "tid": 890 }, { "created_ino": "0x100000004a5", "tid": 893 }, { "created_ino": "0x100000004a6", "tid": 896 }, { "created_ino": "0x100000004a7", "tid": 899 }, { "created_ino": "0x100000004a8", "tid": 902 }, { "created_ino": "0x100000004a9", "tid": 905 }, { "created_ino": "0x100000004aa", "tid": 908 }, { "created_ino": "0x100000004ab", "tid": 911 }, { "created_ino": "0x100000004ac", "tid": 914 }, { "created_ino": "0x100000004ad", "tid": 917 }, { "created_ino": "0x100000004ae", "tid": 920 }, { "created_ino": "0x100000004af", "tid": 923 }, { "created_ino": "0x100000004b0", "tid": 926 }, { "created_ino": "0x100000004b1", "tid": 929 }, { "created_ino": "0x100000004b2", "tid": 932 }, { "created_ino": "0x100000004b3", "tid": 935 }, { "created_ino": "0x100000004b4", "tid": 938 }, { "created_ino": "0x100000004b5", "tid": 941 }, { "created_ino": "0x100000004b6", "tid": 944 }, { "created_ino": "0x100000004b7", "tid": 947 }, { "created_ino": "0x100000004b8", "tid": 950 }, { "created_ino": "0x100000004b9", "tid": 953 }, { "created_ino": "0x100000004ba", "tid": 956 }, { "created_ino": "0x100000004bb", "tid": 959 }, { "created_ino": "0x100000004bc", "tid": 962 }, { "created_ino": "0x100000004bd", "tid": 965 }, { "created_ino": "0x100000004be", "tid": 968 }, { "created_ino": "0x100000004bf", "tid": 971 }, { "created_ino": "0x100000004c0", "tid": 974 }, { "created_ino": "0x100000004c1", "tid": 977 }, { "created_ino": "0x100000004c2", "tid": 980 }, { "created_ino": "0x100000004c3", "tid": 983 }, { "created_ino": "0x100000004c4", "tid": 986 }, { "created_ino": "0x100000004c5", "tid": 989 }, { "created_ino": "0x100000004c6", "tid": 992 }, { "created_ino": "0x100000004c7", "tid": 995 }, { "created_ino": "0x100000004c8", "tid": 998 }, { "created_ino": "0x100000004c9", "tid": 1001 }, { "created_ino": "0x100000004ca", "tid": 1004 }, { "created_ino": "0x100000004cb", "tid": 1007 }, { "created_ino": "0x100000004cc", "tid": 1010 }, { "created_ino": "0x100000004cd", "tid": 1013 }, { "created_ino": "0x100000004ce", "tid": 1016 }, { "created_ino": "0x100000004cf", "tid": 1019 }, { "created_ino": "0x100000004d0", "tid": 1022 }, { "created_ino": "0x100000004d1", "tid": 1025 }, { "created_ino": "0x100000004d2", "tid": 1028 }, { "created_ino": "0x100000004d3", "tid": 1031 }, { "created_ino": "0x100000004d4", "tid": 1034 }, { "created_ino": "0x100000004d5", "tid": 1037 }, { "created_ino": "0x100000004d6", "tid": 1040 }, { "created_ino": "0x100000004d7", "tid": 1043 }, { "created_ino": "0x100000004d8", "tid": 1046 }, { "created_ino": "0x100000004d9", "tid": 1049 }, { "created_ino": "0x100000004da", "tid": 1052 }, { "created_ino": "0x100000004db", "tid": 1055 }, { "created_ino": "0x100000004dc", "tid": 1058 }, { "created_ino": "0x100000004dd", "tid": 1061 }, { "created_ino": "0x100000004de", "tid": 1064 }, { "created_ino": "0x100000004df", "tid": 1067 }, { "created_ino": "0x100000004e0", "tid": 1070 }, { "created_ino": "0x100000004e1", "tid": 1073 }, { "created_ino": "0x100000004e2", "tid": 1076 }, { "created_ino": "0x100000004e3", "tid": 1079 }, { "created_ino": "0x100000004e4", "tid": 1082 }, { "created_ino": "0x100000004e5", "tid": 1085 }, { "created_ino": "0x100000004e6", "tid": 1088 }, { "created_ino": "0x100000004e7", "tid": 1091 }, { "created_ino": "0x100000004e8", "tid": 1094 }, { "created_ino": "0x100000004e9", "tid": 1097 }, { "created_ino": "0x100000004ea", "tid": 1100 }, { "created_ino": "0x100000004eb", "tid": 1103 }, { "created_ino": "0x100000004ec", "tid": 1106 }, { "created_ino": "0x100000004ed", "tid": 1109 }, { "created_ino": "0x100000004ee", "tid": 1112 }, { "created_ino": "0x100000004ef", "tid": 1115 }, { "created_ino": "0x100000004f0", "tid": 1118 }, { "created_ino": "0x100000004f1", "tid": 1121 }, { "created_ino": "0x100000004f2", "tid": 1124 }, { "created_ino": "0x100000004f3", "tid": 1127 }, { "created_ino": "0x100000004f4", "tid": 1130 }, { "created_ino": "0x100000004f5", "tid": 1133 }, { "created_ino": "0x100000004f6", "tid": 1136 }, { "created_ino": "0x100000004f7", "tid": 1139 }, { "created_ino": "0x100000004f8", "tid": 1142 }, { "created_ino": "0x100000004f9", "tid": 1145 }, { "created_ino": "0x100000004fa", "tid": 1148 }, { "created_ino": "0x100000004fb", "tid": 1151 }, { "created_ino": "0x100000004fc", "tid": 1154 }, { "created_ino": "0x100000004fd", "tid": 1157 }, { "created_ino": "0x100000004fe", "tid": 1160 }, { "created_ino": "0x100000004ff", "tid": 1163 }, { "created_ino": "0x10000000500", "tid": 1166 }, { "created_ino": "0x10000000501", "tid": 1169 }, { "created_ino": "0x10000000502", "tid": 1172 }, { "created_ino": "0x10000000503", "tid": 1175 }, { "created_ino": "0x10000000504", "tid": 1178 }, { "created_ino": "0x10000000505", "tid": 1181 }, { "created_ino": "0x10000000506", "tid": 1184 }, { "created_ino": "0x10000000507", "tid": 1187 }, { "created_ino": "0x10000000508", "tid": 1190 }, { "created_ino": "0x10000000509", "tid": 1193 }, { "created_ino": "0x1000000050a", "tid": 1196 }, { "created_ino": "0x1000000050b", "tid": 1199 }, { "created_ino": "0x1000000050c", "tid": 1202 }, { "created_ino": "0x1000000050d", "tid": 1205 }, { "created_ino": "0x1000000050e", "tid": 1208 }, { "created_ino": "0x1000000050f", "tid": 1211 }, { "created_ino": "0x10000000510", "tid": 1214 }, { "created_ino": "0x10000000511", "tid": 1217 }, { "created_ino": "0x10000000512", "tid": 1220 }, { "created_ino": "0x10000000513", "tid": 1223 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2529580715, "type": "any" }, "name": { "num": 8199, "type": "client" } }, "id": 8199, "inst": "client.8199 192.168.0.1:0/2529580715", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 506, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 4831.81740840882 }, "recall_caps_throttle": { "halflife": 1.5, "value": 882.3846387807808 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 292.5946251684947 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1057, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1559.136215337975 }, "state": "open", "uptime": 20.436662041 } ] 2023-07-06T03:54:00.227 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-06T03:54:00.228 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:54:00.228 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:54:00.326 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:54:00.431 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:54:00.432 INFO:tasks.cephfs.mount:Terminating background process 2023-07-06T03:54:00.432 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:54:00.433 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:00.478 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:00.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.709+0000 7f5cb33ff700 1 Processor -- start 2023-07-06T03:54:00.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.710+0000 7f5cb33ff700 1 -- start start 2023-07-06T03:54:00.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb33ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5cac15c520 0x7f5cac0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:00.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb33ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cac160180 0x7f5cac1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:00.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb33ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5cac160b30 0x7f5cac0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:00.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb33ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac0cd8d0 con 0x7f5cac15c520 2023-07-06T03:54:00.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb33ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac0cdae0 con 0x7f5cac160b30 2023-07-06T03:54:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb33ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0b6640 con 0x7f5cac160180 2023-07-06T03:54:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb119b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5cac15c520 0x7f5cac0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb119b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5cac15c520 0x7f5cac0cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55148/0 (socket says 172.21.15.97:55148) 2023-07-06T03:54:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb119b700 1 -- 172.21.15.97:0/527739614 learned_addr learned my addr 172.21.15.97:0/527739614 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb199c700 1 --2- 172.21.15.97:0/527739614 >> v2:172.21.15.146:3300/0 conn(0x7f5cac160b30 0x7f5cac0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb099a700 1 --2- 172.21.15.97:0/527739614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cac160180 0x7f5cac1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb119b700 1 -- 172.21.15.97:0/527739614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cac160180 msgr2=0x7f5cac1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb119b700 1 --2- 172.21.15.97:0/527739614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cac160180 0x7f5cac1605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb119b700 1 -- 172.21.15.97:0/527739614 >> v2:172.21.15.146:3300/0 conn(0x7f5cac160b30 msgr2=0x7f5cac0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb119b700 1 --2- 172.21.15.97:0/527739614 >> v2:172.21.15.146:3300/0 conn(0x7f5cac160b30 0x7f5cac0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.711+0000 7f5cb119b700 1 -- 172.21.15.97:0/527739614 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0b6920 con 0x7f5cac15c520 2023-07-06T03:54:00.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.712+0000 7f5cb119b700 1 --2- 172.21.15.97:0/527739614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac15c520 0x7f5cac0cd390 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f5cac08bfc0 tx=0x7f5ca8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c89ac0d3309882 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:00.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.712+0000 7f5ca3fff700 1 -- 172.21.15.97:0/527739614 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca800b870 con 0x7f5cac15c520 2023-07-06T03:54:00.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.712+0000 7f5ca3fff700 1 -- 172.21.15.97:0/527739614 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca800b9d0 con 0x7f5cac15c520 2023-07-06T03:54:00.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.712+0000 7f5ca3fff700 1 -- 172.21.15.97:0/527739614 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca800bd60 con 0x7f5cac15c520 2023-07-06T03:54:00.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.712+0000 7f5cb33ff700 1 -- 172.21.15.97:0/527739614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac15c520 msgr2=0x7f5cac0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:00.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.712+0000 7f5cb33ff700 1 --2- 172.21.15.97:0/527739614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac15c520 0x7f5cac0cd390 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f5cac08bfc0 tx=0x7f5ca8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.712+0000 7f5cb33ff700 1 -- 172.21.15.97:0/527739614 shutdown_connections 2023-07-06T03:54:00.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.712+0000 7f5cb33ff700 1 --2- 172.21.15.97:0/527739614 >> v2:172.21.15.146:3300/0 conn(0x7f5cac160b30 0x7f5cac0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.712+0000 7f5cb33ff700 1 --2- 172.21.15.97:0/527739614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac15c520 0x7f5cac0cd390 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.712+0000 7f5cb33ff700 1 --2- 172.21.15.97:0/527739614 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cac160180 0x7f5cac1605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.712+0000 7f5cb33ff700 1 -- 172.21.15.97:0/527739614 >> 172.21.15.97:0/527739614 conn(0x7f5cac071790 msgr2=0x7f5cac071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.713+0000 7f5cb33ff700 1 -- 172.21.15.97:0/527739614 shutdown_connections 2023-07-06T03:54:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.713+0000 7f5cb33ff700 1 -- 172.21.15.97:0/527739614 wait complete. 2023-07-06T03:54:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.714+0000 7f5cb33ff700 1 Processor -- start 2023-07-06T03:54:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.714+0000 7f5cb33ff700 1 -- start start 2023-07-06T03:54:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.715+0000 7f5cb33ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cac15c520 0x7f5cac14d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.715+0000 7f5cb33ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cac160180 0x7f5cac0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.715+0000 7f5cb33ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac160b30 0x7f5cac15e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.715+0000 7f5cb33ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac14d7c0 con 0x7f5cac160b30 2023-07-06T03:54:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.715+0000 7f5cb33ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac1f25b0 con 0x7f5cac15c520 2023-07-06T03:54:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.715+0000 7f5cb33ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0cb100 con 0x7f5cac160180 2023-07-06T03:54:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.715+0000 7f5cb199c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac160b30 0x7f5cac15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.715+0000 7f5cb199c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac160b30 0x7f5cac15e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55152/0 (socket says 172.21.15.97:55152) 2023-07-06T03:54:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5cb199c700 1 -- 172.21.15.97:0/1107026203 learned_addr learned my addr 172.21.15.97:0/1107026203 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5cb119b700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cac15c520 0x7f5cac14d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5cb099a700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cac160180 0x7f5cac0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5cb199c700 1 -- 172.21.15.97:0/1107026203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cac160180 msgr2=0x7f5cac0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5cb199c700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cac160180 0x7f5cac0ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5cb199c700 1 -- 172.21.15.97:0/1107026203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cac15c520 msgr2=0x7f5cac14d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5cb199c700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cac15c520 0x7f5cac14d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5cb199c700 1 -- 172.21.15.97:0/1107026203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca8009550 con 0x7f5cac160b30 2023-07-06T03:54:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5cb199c700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac160b30 0x7f5cac15e330 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4007e60 tx=0x7f5ca400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5ca1ffb700 1 -- 172.21.15.97:0/1107026203 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400ed30 con 0x7f5cac160b30 2023-07-06T03:54:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5ca1ffb700 1 -- 172.21.15.97:0/1107026203 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca4014420 con 0x7f5cac160b30 2023-07-06T03:54:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5cb33ff700 1 -- 172.21.15.97:0/1107026203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac0cb3e0 con 0x7f5cac160b30 2023-07-06T03:54:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5ca1ffb700 1 -- 172.21.15.97:0/1107026203 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40147d0 con 0x7f5cac160b30 2023-07-06T03:54:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.716+0000 7f5cb33ff700 1 -- 172.21.15.97:0/1107026203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac0cb9a0 con 0x7f5cac160b30 2023-07-06T03:54:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.718+0000 7f5cb33ff700 1 -- 172.21.15.97:0/1107026203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac04e340 con 0x7f5cac160b30 2023-07-06T03:54:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.718+0000 7f5ca1ffb700 1 -- 172.21.15.97:0/1107026203 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5ca4014930 con 0x7f5cac160b30 2023-07-06T03:54:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.719+0000 7f5ca1ffb700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c9006ace0 0x7f5c9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.719+0000 7f5ca1ffb700 1 -- 172.21.15.97:0/1107026203 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f5ca4089b40 con 0x7f5cac160b30 2023-07-06T03:54:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.720+0000 7f5cb119b700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c9006ace0 0x7f5c9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.720+0000 7f5cb119b700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c9006ace0 0x7f5c9006d190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8000c00 tx=0x7f5ca8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.723+0000 7f5ca1ffb700 1 -- 172.21.15.97:0/1107026203 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5ca4054ae0 con 0x7f5cac160b30 2023-07-06T03:54:00.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.946+0000 7f5cb33ff700 1 -- 172.21.15.97:0/1107026203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cac1f26f0 con 0x7f5cac160b30 2023-07-06T03:54:00.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.947+0000 7f5ca1ffb700 1 -- 172.21.15.97:0/1107026203 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+18516 (secure 0 0 0) 0x7f5ca4058100 con 0x7f5cac160b30 2023-07-06T03:54:00.950 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":101,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:53:54.754060+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:53:24.013743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-06T03:53:25.222178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":6}]},{"pgid":"16.9","mappings":[{"from":4,"to":3}]},{"pgid":"16.15","mappings":[{"from":4,"to":3}]},{"pgid":"16.24","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:00.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 -- 172.21.15.97:0/1107026203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c9006ace0 msgr2=0x7f5c9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:00.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c9006ace0 0x7f5c9006d190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8000c00 tx=0x7f5ca8008030 comp rx=0 tx=0).stop 2023-07-06T03:54:00.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 -- 172.21.15.97:0/1107026203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac160b30 msgr2=0x7f5cac15e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:00.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac160b30 0x7f5cac15e330 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4007e60 tx=0x7f5ca400e3c0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 -- 172.21.15.97:0/1107026203 shutdown_connections 2023-07-06T03:54:00.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c9006ace0 0x7f5c9006d190 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cac160b30 0x7f5cac15e330 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cac15c520 0x7f5cac14d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 --2- 172.21.15.97:0/1107026203 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cac160180 0x7f5cac0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:00.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 -- 172.21.15.97:0/1107026203 >> 172.21.15.97:0/1107026203 conn(0x7f5cac071790 msgr2=0x7f5cac1571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:00.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 -- 172.21.15.97:0/1107026203 shutdown_connections 2023-07-06T03:54:00.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:00.950+0000 7f5cb33ff700 1 -- 172.21.15.97:0/1107026203 wait complete. 2023-07-06T03:54:00.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:53:24.013743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:53:25.222178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:54:00.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:54:01.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.167+0000 7f1493bff700 1 Processor -- start 2023-07-06T03:54:01.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.167+0000 7f1493bff700 1 -- start start 2023-07-06T03:54:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f1493bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f148c15c610 0x7f148c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f1493bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f148c06b360 0x7f148c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f1493bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148c0cff90 con 0x7f148c0d2210 2023-07-06T03:54:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f1493bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148c0d0780 con 0x7f148c06b360 2023-07-06T03:54:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f1493bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148c15c9e0 con 0x7f148c15c610 2023-07-06T03:54:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f149119a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f148c06b360 0x7f148c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f149119a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f148c06b360 0x7f148c0b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55154/0 (socket says 172.21.15.97:55154) 2023-07-06T03:54:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f149119a700 1 -- 172.21.15.97:0/3796411651 learned_addr learned my addr 172.21.15.97:0/3796411651 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f149199b700 1 --2- 172.21.15.97:0/3796411651 >> v2:172.21.15.146:3300/0 conn(0x7f148c15c610 0x7f148c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f149119a700 1 -- 172.21.15.97:0/3796411651 >> v2:172.21.15.146:3300/0 conn(0x7f148c15c610 msgr2=0x7f148c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f149119a700 1 --2- 172.21.15.97:0/3796411651 >> v2:172.21.15.146:3300/0 conn(0x7f148c15c610 0x7f148c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f149119a700 1 -- 172.21.15.97:0/3796411651 >> v1:172.21.15.97:6789/0 conn(0x7f148c0d2210 legacy=0x7f148c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f149119a700 1 -- 172.21.15.97:0/3796411651 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c0b5650 con 0x7f148c06b360 2023-07-06T03:54:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.169+0000 7f1490999700 1 -- 172.21.15.97:0/3796411651 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972447774 0 0) 0x7f148c0cff90 con 0x7f148c0d2210 2023-07-06T03:54:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.170+0000 7f149119a700 1 --2- 172.21.15.97:0/3796411651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f148c06b360 0x7f148c0b4cf0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f148c08b000 tx=0x7f147c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f969df5545efd94 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.170+0000 7f1490999700 1 -- 172.21.15.97:0/3796411651 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c011830 con 0x7f148c06b360 2023-07-06T03:54:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.170+0000 7f1490999700 1 -- 172.21.15.97:0/3796411651 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c011990 con 0x7f148c06b360 2023-07-06T03:54:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.170+0000 7f1490999700 1 -- 172.21.15.97:0/3796411651 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c011d20 con 0x7f148c06b360 2023-07-06T03:54:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.170+0000 7f1493bff700 1 -- 172.21.15.97:0/3796411651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f148c06b360 msgr2=0x7f148c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.170+0000 7f1493bff700 1 --2- 172.21.15.97:0/3796411651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f148c06b360 0x7f148c0b4cf0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f148c08b000 tx=0x7f147c009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.171+0000 7f1493bff700 1 -- 172.21.15.97:0/3796411651 shutdown_connections 2023-07-06T03:54:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.171+0000 7f1493bff700 1 --2- 172.21.15.97:0/3796411651 >> v2:172.21.15.146:3300/0 conn(0x7f148c15c610 0x7f148c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.171+0000 7f1493bff700 1 --2- 172.21.15.97:0/3796411651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f148c06b360 0x7f148c0b4cf0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.171+0000 7f1493bff700 1 -- 172.21.15.97:0/3796411651 >> 172.21.15.97:0/3796411651 conn(0x7f148c06a6c0 msgr2=0x7f148c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.171+0000 7f1493bff700 1 -- 172.21.15.97:0/3796411651 shutdown_connections 2023-07-06T03:54:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.171+0000 7f1493bff700 1 -- 172.21.15.97:0/3796411651 wait complete. 2023-07-06T03:54:01.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.172+0000 7f1493bff700 1 Processor -- start 2023-07-06T03:54:01.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.172+0000 7f1493bff700 1 -- start start 2023-07-06T03:54:01.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.173+0000 7f1493bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f148c06b360 0x7f148c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:01.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.173+0000 7f1493bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f148c0d2210 0x7f148c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:01.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.173+0000 7f1493bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f148c15c610 0x7f148c145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:01.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.173+0000 7f1493bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c1f1fd0 con 0x7f148c15c610 2023-07-06T03:54:01.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.173+0000 7f1493bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0cb170 con 0x7f148c0d2210 2023-07-06T03:54:01.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.173+0000 7f1493bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0c9850 con 0x7f148c06b360 2023-07-06T03:54:01.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149219c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f148c15c610 0x7f148c145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:01.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149219c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f148c15c610 0x7f148c145fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55164/0 (socket says 172.21.15.97:55164) 2023-07-06T03:54:01.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149219c700 1 -- 172.21.15.97:0/1538482518 learned_addr learned my addr 172.21.15.97:0/1538482518 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:01.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149119a700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f148c0d2210 0x7f148c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:01.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149199b700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f148c06b360 0x7f148c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:01.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149219c700 1 -- 172.21.15.97:0/1538482518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f148c06b360 msgr2=0x7f148c15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149219c700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f148c06b360 0x7f148c15cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149219c700 1 -- 172.21.15.97:0/1538482518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f148c0d2210 msgr2=0x7f148c0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149219c700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f148c0d2210 0x7f148c0cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149219c700 1 -- 172.21.15.97:0/1538482518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147c009550 con 0x7f148c15c610 2023-07-06T03:54:01.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149119a700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f148c0d2210 0x7f148c0cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:01.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149199b700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f148c06b360 0x7f148c15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:01.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f149219c700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f148c15c610 0x7f148c145fd0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f148400c840 tx=0x7f148400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:01.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.174+0000 7f14827fc700 1 -- 172.21.15.97:0/1538482518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484003e30 con 0x7f148c15c610 2023-07-06T03:54:01.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.175+0000 7f14827fc700 1 -- 172.21.15.97:0/1538482518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1484005070 con 0x7f148c15c610 2023-07-06T03:54:01.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.175+0000 7f1493bff700 1 -- 172.21.15.97:0/1538482518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c0b2f10 con 0x7f148c15c610 2023-07-06T03:54:01.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.175+0000 7f14827fc700 1 -- 172.21.15.97:0/1538482518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484005420 con 0x7f148c15c610 2023-07-06T03:54:01.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.175+0000 7f1493bff700 1 -- 172.21.15.97:0/1538482518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c0b28c0 con 0x7f148c15c610 2023-07-06T03:54:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.176+0000 7f1493bff700 1 -- 172.21.15.97:0/1538482518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148c0b30a0 con 0x7f148c15c610 2023-07-06T03:54:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.176+0000 7f14827fc700 1 -- 172.21.15.97:0/1538482518 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1484005580 con 0x7f148c15c610 2023-07-06T03:54:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.177+0000 7f14827fc700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f147006ace0 0x7f147006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.178+0000 7f14827fc700 1 -- 172.21.15.97:0/1538482518 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f1484089140 con 0x7f148c15c610 2023-07-06T03:54:01.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.178+0000 7f149199b700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f147006ace0 0x7f147006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:01.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.178+0000 7f149199b700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f147006ace0 0x7f147006d190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1488009ac0 tx=0x7f1488008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:01.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.182+0000 7f14827fc700 1 -- 172.21.15.97:0/1538482518 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1484054200 con 0x7f148c15c610 2023-07-06T03:54:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.378+0000 7f1493bff700 1 -- 172.21.15.97:0/1538482518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f148c04e340 con 0x7f148c15c610 2023-07-06T03:54:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.378+0000 7f14827fc700 1 -- 172.21.15.97:0/1538482518 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1484057820 con 0x7f148c15c610 2023-07-06T03:54:01.381 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:54:01.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.380+0000 7f1493bff700 1 -- 172.21.15.97:0/1538482518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f147006ace0 msgr2=0x7f147006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.380+0000 7f1493bff700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f147006ace0 0x7f147006d190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1488009ac0 tx=0x7f1488008040 comp rx=0 tx=0).stop 2023-07-06T03:54:01.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.380+0000 7f1493bff700 1 -- 172.21.15.97:0/1538482518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f148c15c610 msgr2=0x7f148c145fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.380+0000 7f1493bff700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f148c15c610 0x7f148c145fd0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f148400c840 tx=0x7f148400cc00 comp rx=0 tx=0).stop 2023-07-06T03:54:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.380+0000 7f1493bff700 1 -- 172.21.15.97:0/1538482518 shutdown_connections 2023-07-06T03:54:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.380+0000 7f1493bff700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f147006ace0 0x7f147006d190 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.380+0000 7f1493bff700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f148c15c610 0x7f148c145fd0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.380+0000 7f1493bff700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f148c06b360 0x7f148c15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.380+0000 7f1493bff700 1 --2- 172.21.15.97:0/1538482518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f148c0d2210 0x7f148c0cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.380+0000 7f1493bff700 1 -- 172.21.15.97:0/1538482518 >> 172.21.15.97:0/1538482518 conn(0x7f148c06a6c0 msgr2=0x7f148c0b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.381+0000 7f1493bff700 1 -- 172.21.15.97:0/1538482518 shutdown_connections 2023-07-06T03:54:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.381+0000 7f1493bff700 1 -- 172.21.15.97:0/1538482518 wait complete. 2023-07-06T03:54:01.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:54:01.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.596+0000 7fe76b3ff700 1 Processor -- start 2023-07-06T03:54:01.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.596+0000 7fe76b3ff700 1 -- start start 2023-07-06T03:54:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.597+0000 7fe76b3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe76406b360 0x7fe7640b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.597+0000 7fe76b3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7640cff90 con 0x7fe7640d2210 2023-07-06T03:54:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.597+0000 7fe76b3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7640b5650 con 0x7fe76415c610 2023-07-06T03:54:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.597+0000 7fe76b3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7640d0780 con 0x7fe76406b360 2023-07-06T03:54:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.598+0000 7fe76899a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe76406b360 0x7fe7640b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.598+0000 7fe76919b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fe76415c610 0x7fe7640b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:53858/0 (socket says 172.21.15.97:53858) 2023-07-06T03:54:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.598+0000 7fe76919b700 1 -- 172.21.15.97:0/1932139603 learned_addr learned my addr 172.21.15.97:0/1932139603 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.598+0000 7fe76999c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe7640d2210 0x7fe7640d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56714/0 (socket says 172.21.15.97:56714) 2023-07-06T03:54:01.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.598+0000 7fe75bfff700 1 -- 172.21.15.97:0/1932139603 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524179477 0 0) 0x7fe7640b5650 con 0x7fe76415c610 2023-07-06T03:54:01.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.598+0000 7fe75bfff700 1 -- 172.21.15.97:0/1932139603 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7500035b0 con 0x7fe76415c610 2023-07-06T03:54:01.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.598+0000 7fe75bfff700 1 -- 172.21.15.97:0/1932139603 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060822858 0 0) 0x7fe7640cff90 con 0x7fe7640d2210 2023-07-06T03:54:01.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.598+0000 7fe75bfff700 1 -- 172.21.15.97:0/1932139603 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7640b5650 con 0x7fe7640d2210 2023-07-06T03:54:01.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.598+0000 7fe76899a700 1 -- 172.21.15.97:0/1932139603 >> v1:172.21.15.146:6789/0 conn(0x7fe76415c610 legacy=0x7fe7640b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.598+0000 7fe76899a700 1 -- 172.21.15.97:0/1932139603 >> v1:172.21.15.97:6789/0 conn(0x7fe7640d2210 legacy=0x7fe7640d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.598+0000 7fe76899a700 1 -- 172.21.15.97:0/1932139603 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7640b5400 con 0x7fe76406b360 2023-07-06T03:54:01.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.599+0000 7fe76899a700 1 --2- 172.21.15.97:0/1932139603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76406b360 0x7fe7640b4ec0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe754009840 tx=0x7fe754009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd7fde8a75daa73a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:01.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.599+0000 7fe75bfff700 1 -- 172.21.15.97:0/1932139603 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754011830 con 0x7fe76406b360 2023-07-06T03:54:01.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.599+0000 7fe75bfff700 1 -- 172.21.15.97:0/1932139603 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754011990 con 0x7fe76406b360 2023-07-06T03:54:01.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.599+0000 7fe75bfff700 1 -- 172.21.15.97:0/1932139603 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754011d20 con 0x7fe76406b360 2023-07-06T03:54:01.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.599+0000 7fe76b3ff700 1 -- 172.21.15.97:0/1932139603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76406b360 msgr2=0x7fe7640b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.599+0000 7fe76b3ff700 1 --2- 172.21.15.97:0/1932139603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76406b360 0x7fe7640b4ec0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe754009840 tx=0x7fe754009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:01.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.599+0000 7fe76b3ff700 1 -- 172.21.15.97:0/1932139603 shutdown_connections 2023-07-06T03:54:01.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.599+0000 7fe76b3ff700 1 --2- 172.21.15.97:0/1932139603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76406b360 0x7fe7640b4ec0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.599+0000 7fe76b3ff700 1 -- 172.21.15.97:0/1932139603 >> 172.21.15.97:0/1932139603 conn(0x7fe76406a6c0 msgr2=0x7fe76406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:01.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.600+0000 7fe76b3ff700 1 -- 172.21.15.97:0/1932139603 shutdown_connections 2023-07-06T03:54:01.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.600+0000 7fe76b3ff700 1 -- 172.21.15.97:0/1932139603 wait complete. 2023-07-06T03:54:01.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.601+0000 7fe76b3ff700 1 Processor -- start 2023-07-06T03:54:01.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.601+0000 7fe76b3ff700 1 -- start start 2023-07-06T03:54:01.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.602+0000 7fe76b3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76406b360 0x7fe7641541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:01.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.602+0000 7fe76b3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7640d2210 0x7fe76415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:01.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76919b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76406b360 0x7fe7641541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:01.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76b3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe76415c610 0x7fe7641f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:01.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76899a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7640d2210 0x7fe76415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:01.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76b3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7640cb0d0 con 0x7fe76415c610 2023-07-06T03:54:01.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76b3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76415d0a0 con 0x7fe76406b360 2023-07-06T03:54:01.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76b3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7641f1ff0 con 0x7fe7640d2210 2023-07-06T03:54:01.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76919b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76406b360 0x7fe7641541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:36930/0 (socket says 172.21.15.97:36930) 2023-07-06T03:54:01.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76919b700 1 -- 172.21.15.97:0/4163842048 learned_addr learned my addr 172.21.15.97:0/4163842048 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:01.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76999c700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe76415c610 0x7fe7641f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:01.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76899a700 1 -- 172.21.15.97:0/4163842048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76406b360 msgr2=0x7fe7641541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76899a700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76406b360 0x7fe7641541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76899a700 1 -- 172.21.15.97:0/4163842048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe76415c610 msgr2=0x7fe7641f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76899a700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe76415c610 0x7fe7641f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76899a700 1 -- 172.21.15.97:0/4163842048 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7540094a0 con 0x7fe7640d2210 2023-07-06T03:54:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76919b700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76406b360 0x7fe7641541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:54:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.603+0000 7fe76899a700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7640d2210 0x7fe76415cb60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe754016000 tx=0x7fe754004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.604+0000 7fe759ffb700 1 -- 172.21.15.97:0/4163842048 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75401a6e0 con 0x7fe7640d2210 2023-07-06T03:54:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.604+0000 7fe759ffb700 1 -- 172.21.15.97:0/4163842048 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75401a860 con 0x7fe7640d2210 2023-07-06T03:54:01.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.604+0000 7fe76b3ff700 1 -- 172.21.15.97:0/4163842048 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76415d2d0 con 0x7fe7640d2210 2023-07-06T03:54:01.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.604+0000 7fe759ffb700 1 -- 172.21.15.97:0/4163842048 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75401ac10 con 0x7fe7640d2210 2023-07-06T03:54:01.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.604+0000 7fe76b3ff700 1 -- 172.21.15.97:0/4163842048 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76415d890 con 0x7fe7640d2210 2023-07-06T03:54:01.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.605+0000 7fe76b3ff700 1 -- 172.21.15.97:0/4163842048 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76404e340 con 0x7fe7640d2210 2023-07-06T03:54:01.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.606+0000 7fe759ffb700 1 -- 172.21.15.97:0/4163842048 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe754022070 con 0x7fe7640d2210 2023-07-06T03:54:01.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.607+0000 7fe759ffb700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe74806ace0 0x7fe74806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:01.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.607+0000 7fe759ffb700 1 -- 172.21.15.97:0/4163842048 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fe75408a5c0 con 0x7fe7640d2210 2023-07-06T03:54:01.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.607+0000 7fe76919b700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe74806ace0 0x7fe74806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:01.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.608+0000 7fe76919b700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe74806ace0 0x7fe74806d190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe760000f60 tx=0x7fe760009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:01.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.610+0000 7fe759ffb700 1 -- 172.21.15.97:0/4163842048 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe754055660 con 0x7fe7640d2210 2023-07-06T03:54:01.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.808+0000 7fe76b3ff700 1 -- 172.21.15.97:0/4163842048 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe76406b9d0 con 0x7fe7640d2210 2023-07-06T03:54:01.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.808+0000 7fe759ffb700 1 -- 172.21.15.97:0/4163842048 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v101) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe754058c80 con 0x7fe7640d2210 2023-07-06T03:54:01.811 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:54:01.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 -- 172.21.15.97:0/4163842048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe74806ace0 msgr2=0x7fe74806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe74806ace0 0x7fe74806d190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe760000f60 tx=0x7fe760009040 comp rx=0 tx=0).stop 2023-07-06T03:54:01.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 -- 172.21.15.97:0/4163842048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7640d2210 msgr2=0x7fe76415cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:01.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7640d2210 0x7fe76415cb60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe754016000 tx=0x7fe754004750 comp rx=0 tx=0).stop 2023-07-06T03:54:01.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 -- 172.21.15.97:0/4163842048 shutdown_connections 2023-07-06T03:54:01.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe74806ace0 0x7fe74806d190 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe76415c610 0x7fe7641f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe76406b360 0x7fe7641541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 --2- 172.21.15.97:0/4163842048 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7640d2210 0x7fe76415cb60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:01.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 -- 172.21.15.97:0/4163842048 >> 172.21.15.97:0/4163842048 conn(0x7fe76406a6c0 msgr2=0x7fe7640b78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:01.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 -- 172.21.15.97:0/4163842048 shutdown_connections 2023-07-06T03:54:01.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:01.811+0000 7fe76b3ff700 1 -- 172.21.15.97:0/4163842048 wait complete. 2023-07-06T03:54:01.835 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:54:02.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.029+0000 7f9a5b7ff700 1 Processor -- start 2023-07-06T03:54:02.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.029+0000 7f9a5b7ff700 1 -- start start 2023-07-06T03:54:02.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.030+0000 7f9a5b7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9a5415c1c0 0x7f9a540cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.030+0000 7f9a5b7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9a540719c0 0x7f9a5406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.030+0000 7f9a5b7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5415da10 con 0x7f9a5415c590 2023-07-06T03:54:02.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.030+0000 7f9a5b7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a540ccc30 con 0x7f9a5415c1c0 2023-07-06T03:54:02.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.030+0000 7f9a5b7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a540ccf10 con 0x7f9a540719c0 2023-07-06T03:54:02.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.031+0000 7f9a5959b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9a5415c1c0 0x7f9a540cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.031+0000 7f9a5959b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9a5415c1c0 0x7f9a540cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55182/0 (socket says 172.21.15.97:55182) 2023-07-06T03:54:02.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.031+0000 7f9a5959b700 1 -- 172.21.15.97:0/646975486 learned_addr learned my addr 172.21.15.97:0/646975486 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.031+0000 7f9a59d9c700 1 --2- 172.21.15.97:0/646975486 >> v2:172.21.15.146:3300/0 conn(0x7f9a540719c0 0x7f9a5406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.031+0000 7f9a5959b700 1 -- 172.21.15.97:0/646975486 >> v2:172.21.15.146:3300/0 conn(0x7f9a540719c0 msgr2=0x7f9a5406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:02.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.031+0000 7f9a5959b700 1 --2- 172.21.15.97:0/646975486 >> v2:172.21.15.146:3300/0 conn(0x7f9a540719c0 0x7f9a5406a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.031+0000 7f9a5959b700 1 -- 172.21.15.97:0/646975486 >> v1:172.21.15.146:6789/0 conn(0x7f9a5415c590 legacy=0x7f9a5415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:02.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.031+0000 7f9a5959b700 1 -- 172.21.15.97:0/646975486 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a540cd1f0 con 0x7f9a5415c1c0 2023-07-06T03:54:02.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.031+0000 7f9a59d9c700 1 --2- 172.21.15.97:0/646975486 >> v2:172.21.15.146:3300/0 conn(0x7f9a540719c0 0x7f9a5406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:02.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.031+0000 7f9a5959b700 1 --2- 172.21.15.97:0/646975486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a5415c1c0 0x7f9a540cc3e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f9a540b26c0 tx=0x7f9a50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5afad460a8f651 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:02.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.031+0000 7f9a4bfff700 1 -- 172.21.15.97:0/646975486 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50004230 con 0x7f9a5415c1c0 2023-07-06T03:54:02.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.032+0000 7f9a4bfff700 1 -- 172.21.15.97:0/646975486 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a50004390 con 0x7f9a5415c1c0 2023-07-06T03:54:02.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.032+0000 7f9a4bfff700 1 -- 172.21.15.97:0/646975486 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a500036a0 con 0x7f9a5415c1c0 2023-07-06T03:54:02.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.032+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/646975486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a5415c1c0 msgr2=0x7f9a540cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.032+0000 7f9a5b7ff700 1 --2- 172.21.15.97:0/646975486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a5415c1c0 0x7f9a540cc3e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f9a540b26c0 tx=0x7f9a50009bd0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.032+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/646975486 shutdown_connections 2023-07-06T03:54:02.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.032+0000 7f9a5b7ff700 1 --2- 172.21.15.97:0/646975486 >> v2:172.21.15.146:3300/0 conn(0x7f9a540719c0 0x7f9a5406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.032+0000 7f9a5b7ff700 1 --2- 172.21.15.97:0/646975486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a5415c1c0 0x7f9a540cc3e0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.032+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/646975486 >> 172.21.15.97:0/646975486 conn(0x7f9a540b1650 msgr2=0x7f9a540b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:02.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.032+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/646975486 shutdown_connections 2023-07-06T03:54:02.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.032+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/646975486 wait complete. 2023-07-06T03:54:02.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.034+0000 7f9a5b7ff700 1 Processor -- start 2023-07-06T03:54:02.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.035+0000 7f9a5b7ff700 1 -- start start 2023-07-06T03:54:02.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.036+0000 7f9a5b7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a540719c0 0x7f9a541f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.036+0000 7f9a5b7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a5415c1c0 0x7f9a541f1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.036+0000 7f9a5b7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a5415c590 0x7f9a54158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.036+0000 7f9a5b7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a540b2a60 con 0x7f9a5415c590 2023-07-06T03:54:02.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.036+0000 7f9a5b7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a541f60c0 con 0x7f9a5415c1c0 2023-07-06T03:54:02.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.036+0000 7f9a5b7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a541f2b10 con 0x7f9a540719c0 2023-07-06T03:54:02.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.036+0000 7f9a59d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a5415c590 0x7f9a54158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.036+0000 7f9a59d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a5415c590 0x7f9a54158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55190/0 (socket says 172.21.15.97:55190) 2023-07-06T03:54:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.036+0000 7f9a59d9c700 1 -- 172.21.15.97:0/37981002 learned_addr learned my addr 172.21.15.97:0/37981002 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.036+0000 7f9a5959b700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a540719c0 0x7f9a541f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.036+0000 7f9a58d9a700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a5415c1c0 0x7f9a541f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a59d9c700 1 -- 172.21.15.97:0/37981002 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a540719c0 msgr2=0x7f9a541f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a59d9c700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a540719c0 0x7f9a541f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a59d9c700 1 -- 172.21.15.97:0/37981002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a5415c1c0 msgr2=0x7f9a541f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a59d9c700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a5415c1c0 0x7f9a541f1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a59d9c700 1 -- 172.21.15.97:0/37981002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a50009550 con 0x7f9a5415c590 2023-07-06T03:54:02.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a5959b700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a540719c0 0x7f9a541f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:02.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a59d9c700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a5415c590 0x7f9a54158590 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00cef0 tx=0x7f9a4c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:02.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a49ffb700 1 -- 172.21.15.97:0/37981002 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c00ae50 con 0x7f9a5415c590 2023-07-06T03:54:02.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a49ffb700 1 -- 172.21.15.97:0/37981002 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c015440 con 0x7f9a5415c590 2023-07-06T03:54:02.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a49ffb700 1 -- 172.21.15.97:0/37981002 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0157f0 con 0x7f9a5415c590 2023-07-06T03:54:02.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/37981002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a541f1f90 con 0x7f9a5415c590 2023-07-06T03:54:02.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.037+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/37981002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a541f2550 con 0x7f9a5415c590 2023-07-06T03:54:02.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.038+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/37981002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5404e340 con 0x7f9a5415c590 2023-07-06T03:54:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.039+0000 7f9a49ffb700 1 -- 172.21.15.97:0/37981002 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a4c00d430 con 0x7f9a5415c590 2023-07-06T03:54:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.040+0000 7f9a49ffb700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a3806ace0 0x7f9a3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.040+0000 7f9a49ffb700 1 -- 172.21.15.97:0/37981002 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f9a4c089b70 con 0x7f9a5415c590 2023-07-06T03:54:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.040+0000 7f9a5959b700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a3806ace0 0x7f9a3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.041+0000 7f9a5959b700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a3806ace0 0x7f9a3806d190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009130 tx=0x7f9a50008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.045+0000 7f9a49ffb700 1 -- 172.21.15.97:0/37981002 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9a4c054b90 con 0x7f9a5415c590 2023-07-06T03:54:02.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.241+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/37981002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a54071290 con 0x7f9a5415c590 2023-07-06T03:54:02.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.242+0000 7f9a49ffb700 1 -- 172.21.15.97:0/37981002 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a4c01a020 con 0x7f9a5415c590 2023-07-06T03:54:02.245 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:54:02.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.244+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/37981002 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a3806ace0 msgr2=0x7f9a3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.244+0000 7f9a5b7ff700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a3806ace0 0x7f9a3806d190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009130 tx=0x7f9a50008030 comp rx=0 tx=0).stop 2023-07-06T03:54:02.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.244+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/37981002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a5415c590 msgr2=0x7f9a54158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.244+0000 7f9a5b7ff700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a5415c590 0x7f9a54158590 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00cef0 tx=0x7f9a4c00a340 comp rx=0 tx=0).stop 2023-07-06T03:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.244+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/37981002 shutdown_connections 2023-07-06T03:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.244+0000 7f9a5b7ff700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9a3806ace0 0x7f9a3806d190 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.244+0000 7f9a5b7ff700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a5415c590 0x7f9a54158590 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.244+0000 7f9a5b7ff700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a540719c0 0x7f9a541f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.244+0000 7f9a5b7ff700 1 --2- 172.21.15.97:0/37981002 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a5415c1c0 0x7f9a541f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.244+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/37981002 >> 172.21.15.97:0/37981002 conn(0x7f9a540b1650 msgr2=0x7f9a5406a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.245+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/37981002 shutdown_connections 2023-07-06T03:54:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.245+0000 7f9a5b7ff700 1 -- 172.21.15.97:0/37981002 wait complete. 2023-07-06T03:54:02.264 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:54:02.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.458+0000 7f9b083ff700 1 Processor -- start 2023-07-06T03:54:02.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.458+0000 7f9b083ff700 1 -- start start 2023-07-06T03:54:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.459+0000 7f9b083ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000b8890 0x7f9b000b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.459+0000 7f9b083ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9b000b4d70 0x7f9b000cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.459+0000 7f9b083ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b000b5280 con 0x7f9b000cd940 2023-07-06T03:54:02.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.459+0000 7f9b083ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b000cdde0 con 0x7f9b000b4d70 2023-07-06T03:54:02.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.459+0000 7f9b083ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b000d2bb0 con 0x7f9b000b8890 2023-07-06T03:54:02.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.459+0000 7f9b0599a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9b000b4d70 0x7f9b000cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.459+0000 7f9b0699c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9b000cd940 0x7f9b0006c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56716/0 (socket says 172.21.15.97:56716) 2023-07-06T03:54:02.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.459+0000 7f9b0699c700 1 -- 172.21.15.97:0/45567916 learned_addr learned my addr 172.21.15.97:0/45567916 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:02.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.460+0000 7f9b0619b700 1 --2- 172.21.15.97:0/45567916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000b8890 0x7f9b000b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.460+0000 7f9b0599a700 1 -- 172.21.15.97:0/45567916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000b8890 msgr2=0x7f9b000b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:02.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.460+0000 7f9b0599a700 1 --2- 172.21.15.97:0/45567916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000b8890 0x7f9b000b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.460+0000 7f9b0599a700 1 -- 172.21.15.97:0/45567916 >> v1:172.21.15.97:6789/0 conn(0x7f9b000cd940 legacy=0x7f9b0006c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.460+0000 7f9b0599a700 1 -- 172.21.15.97:0/45567916 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b000d0de0 con 0x7f9b000b4d70 2023-07-06T03:54:02.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.460+0000 7f9b05199700 1 -- 172.21.15.97:0/45567916 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4096162118 0 0) 0x7f9b000b5280 con 0x7f9b000cd940 2023-07-06T03:54:02.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.460+0000 7f9b0619b700 1 --2- 172.21.15.97:0/45567916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000b8890 0x7f9b000b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:02.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.460+0000 7f9b0599a700 1 --2- 172.21.15.97:0/45567916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b000b4d70 0x7f9b000cd400 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009840 tx=0x7f9af0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cc3ae0c52e78a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:02.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.460+0000 7f9b05199700 1 -- 172.21.15.97:0/45567916 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0011830 con 0x7f9b000b4d70 2023-07-06T03:54:02.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.460+0000 7f9b05199700 1 -- 172.21.15.97:0/45567916 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0011990 con 0x7f9b000b4d70 2023-07-06T03:54:02.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.460+0000 7f9b05199700 1 -- 172.21.15.97:0/45567916 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0011d20 con 0x7f9b000b4d70 2023-07-06T03:54:02.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.461+0000 7f9b083ff700 1 -- 172.21.15.97:0/45567916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b000b4d70 msgr2=0x7f9b000cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.461+0000 7f9b083ff700 1 --2- 172.21.15.97:0/45567916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b000b4d70 0x7f9b000cd400 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009840 tx=0x7f9af0009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:02.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.461+0000 7f9b083ff700 1 -- 172.21.15.97:0/45567916 shutdown_connections 2023-07-06T03:54:02.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.461+0000 7f9b083ff700 1 --2- 172.21.15.97:0/45567916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000b8890 0x7f9b000b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.461+0000 7f9b083ff700 1 --2- 172.21.15.97:0/45567916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b000b4d70 0x7f9b000cd400 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.461+0000 7f9b083ff700 1 -- 172.21.15.97:0/45567916 >> 172.21.15.97:0/45567916 conn(0x7f9b0006b2d0 msgr2=0x7f9b000cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:02.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.461+0000 7f9b083ff700 1 -- 172.21.15.97:0/45567916 shutdown_connections 2023-07-06T03:54:02.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.461+0000 7f9b083ff700 1 -- 172.21.15.97:0/45567916 wait complete. 2023-07-06T03:54:02.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.463+0000 7f9b083ff700 1 Processor -- start 2023-07-06T03:54:02.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.463+0000 7f9b083ff700 1 -- start start 2023-07-06T03:54:02.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b083ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b000b4d70 0x7f9b001f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b083ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b000b8890 0x7f9b00158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b083ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000cd940 0x7f9b0015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b083ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0014cb20 con 0x7f9b000b8890 2023-07-06T03:54:02.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b083ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b00157f30 con 0x7f9b000b4d70 2023-07-06T03:54:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b083ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b00158210 con 0x7f9b000cd940 2023-07-06T03:54:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0599a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b000b8890 0x7f9b00158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0599a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b000b8890 0x7f9b00158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55202/0 (socket says 172.21.15.97:55202) 2023-07-06T03:54:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0599a700 1 -- 172.21.15.97:0/4138296875 learned_addr learned my addr 172.21.15.97:0/4138296875 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0699c700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000cd940 0x7f9b0015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0619b700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b000b4d70 0x7f9b001f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0599a700 1 -- 172.21.15.97:0/4138296875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000cd940 msgr2=0x7f9b0015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0599a700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000cd940 0x7f9b0015d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0599a700 1 -- 172.21.15.97:0/4138296875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b000b4d70 msgr2=0x7f9b001f15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0599a700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b000b4d70 0x7f9b001f15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0599a700 1 -- 172.21.15.97:0/4138296875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af00094a0 con 0x7f9b000b8890 2023-07-06T03:54:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0699c700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000cd940 0x7f9b0015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.464+0000 7f9b0619b700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b000b4d70 0x7f9b001f15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.465+0000 7f9b0599a700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b000b8890 0x7f9b00158fd0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009b90 tx=0x7f9af0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.465+0000 7f9af6ffd700 1 -- 172.21.15.97:0/4138296875 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00038a0 con 0x7f9b000b8890 2023-07-06T03:54:02.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.465+0000 7f9af6ffd700 1 -- 172.21.15.97:0/4138296875 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af001e070 con 0x7f9b000b8890 2023-07-06T03:54:02.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.465+0000 7f9af6ffd700 1 -- 172.21.15.97:0/4138296875 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0003c00 con 0x7f9b000b8890 2023-07-06T03:54:02.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.465+0000 7f9b083ff700 1 -- 172.21.15.97:0/4138296875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b001584f0 con 0x7f9b000b8890 2023-07-06T03:54:02.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.465+0000 7f9b083ff700 1 -- 172.21.15.97:0/4138296875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b00158ab0 con 0x7f9b000b8890 2023-07-06T03:54:02.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.466+0000 7f9b083ff700 1 -- 172.21.15.97:0/4138296875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b00158ca0 con 0x7f9b000b8890 2023-07-06T03:54:02.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.467+0000 7f9af6ffd700 1 -- 172.21.15.97:0/4138296875 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9af002d450 con 0x7f9b000b8890 2023-07-06T03:54:02.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.468+0000 7f9af6ffd700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ae406ace0 0x7f9ae406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.468+0000 7f9af6ffd700 1 -- 172.21.15.97:0/4138296875 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f9af008a650 con 0x7f9b000b8890 2023-07-06T03:54:02.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.468+0000 7f9b0619b700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ae406ace0 0x7f9ae406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.469+0000 7f9b0619b700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ae406ace0 0x7f9ae406d190 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009a00 tx=0x7f9afc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:02.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.472+0000 7f9af6ffd700 1 -- 172.21.15.97:0/4138296875 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9af0055670 con 0x7f9b000b8890 2023-07-06T03:54:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.667+0000 7f9b083ff700 1 -- 172.21.15.97:0/4138296875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9b0004e340 con 0x7f9b000b8890 2023-07-06T03:54:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.668+0000 7f9af6ffd700 1 -- 172.21.15.97:0/4138296875 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9af0058c90 con 0x7f9b000b8890 2023-07-06T03:54:02.671 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:54:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 -- 172.21.15.97:0/4138296875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ae406ace0 msgr2=0x7f9ae406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ae406ace0 0x7f9ae406d190 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009a00 tx=0x7f9afc008040 comp rx=0 tx=0).stop 2023-07-06T03:54:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 -- 172.21.15.97:0/4138296875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b000b8890 msgr2=0x7f9b00158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b000b8890 0x7f9b00158fd0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009b90 tx=0x7f9af0004750 comp rx=0 tx=0).stop 2023-07-06T03:54:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 -- 172.21.15.97:0/4138296875 shutdown_connections 2023-07-06T03:54:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ae406ace0 0x7f9ae406d190 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9b000cd940 0x7f9b0015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b000b4d70 0x7f9b001f15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 --2- 172.21.15.97:0/4138296875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b000b8890 0x7f9b00158fd0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 -- 172.21.15.97:0/4138296875 >> 172.21.15.97:0/4138296875 conn(0x7f9b0006b2d0 msgr2=0x7f9b000b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 -- 172.21.15.97:0/4138296875 shutdown_connections 2023-07-06T03:54:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.670+0000 7f9b083ff700 1 -- 172.21.15.97:0/4138296875 wait complete. 2023-07-06T03:54:02.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:54:02.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.909+0000 7f3d6c1ff700 1 Processor -- start 2023-07-06T03:54:02.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.910+0000 7f3d6c1ff700 1 -- start start 2023-07-06T03:54:02.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.911+0000 7f3d6c1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3d640d2210 0x7f3d640b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.911+0000 7f3d6c1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d640cff90 con 0x7f3d6406b360 2023-07-06T03:54:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.911+0000 7f3d6c1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d640b5650 con 0x7f3d6415c610 2023-07-06T03:54:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.911+0000 7f3d6c1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d640d0780 con 0x7f3d640d2210 2023-07-06T03:54:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.911+0000 7f3d6979a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3d6406b360 0x7f3d640d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56726/0 (socket says 172.21.15.97:56726) 2023-07-06T03:54:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.911+0000 7f3d6979a700 1 -- 172.21.15.97:0/2176043077 learned_addr learned my addr 172.21.15.97:0/2176043077 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:02.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.911+0000 7f3d6a79c700 1 --2- 172.21.15.97:0/2176043077 >> v2:172.21.15.97:3300/0 conn(0x7f3d640d2210 0x7f3d640b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.911+0000 7f3d68f99700 1 -- 172.21.15.97:0/2176043077 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201052537 0 0) 0x7f3d640cff90 con 0x7f3d6406b360 2023-07-06T03:54:02.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.912+0000 7f3d68f99700 1 -- 172.21.15.97:0/2176043077 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d500035b0 con 0x7f3d6406b360 2023-07-06T03:54:02.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.912+0000 7f3d6a79c700 1 -- 172.21.15.97:0/2176043077 >> v1:172.21.15.146:6789/0 conn(0x7f3d6415c610 legacy=0x7f3d640b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.912+0000 7f3d6a79c700 1 -- 172.21.15.97:0/2176043077 >> v1:172.21.15.97:6789/0 conn(0x7f3d6406b360 legacy=0x7f3d640d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.912+0000 7f3d6a79c700 1 -- 172.21.15.97:0/2176043077 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d640b5400 con 0x7f3d640d2210 2023-07-06T03:54:02.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.912+0000 7f3d6a79c700 1 --2- 172.21.15.97:0/2176043077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d640d2210 0x7f3d640b4ec0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009d90 tx=0x7f3d5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf9fc7ca2796f15 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:02.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.912+0000 7f3d68f99700 1 -- 172.21.15.97:0/2176043077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c00e070 con 0x7f3d640d2210 2023-07-06T03:54:02.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.912+0000 7f3d68f99700 1 -- 172.21.15.97:0/2176043077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c00bb20 con 0x7f3d640d2210 2023-07-06T03:54:02.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.912+0000 7f3d68f99700 1 -- 172.21.15.97:0/2176043077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c012430 con 0x7f3d640d2210 2023-07-06T03:54:02.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.913+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/2176043077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d640d2210 msgr2=0x7f3d640b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.913+0000 7f3d6c1ff700 1 --2- 172.21.15.97:0/2176043077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d640d2210 0x7f3d640b4ec0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009d90 tx=0x7f3d5c00b330 comp rx=0 tx=0).stop 2023-07-06T03:54:02.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.913+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/2176043077 shutdown_connections 2023-07-06T03:54:02.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.913+0000 7f3d6c1ff700 1 --2- 172.21.15.97:0/2176043077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d640d2210 0x7f3d640b4ec0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.913+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/2176043077 >> 172.21.15.97:0/2176043077 conn(0x7f3d6406a6c0 msgr2=0x7f3d6406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:02.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.913+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/2176043077 shutdown_connections 2023-07-06T03:54:02.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.913+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/2176043077 wait complete. 2023-07-06T03:54:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.914+0000 7f3d6c1ff700 1 Processor -- start 2023-07-06T03:54:02.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.915+0000 7f3d6c1ff700 1 -- start start 2023-07-06T03:54:02.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.915+0000 7f3d6c1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d6406b360 0x7f3d641591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.915+0000 7f3d6c1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d640d2210 0x7f3d6415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.915+0000 7f3d6c1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d6415c610 0x7f3d640cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.915+0000 7f3d6c1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d64159700 con 0x7f3d6406b360 2023-07-06T03:54:02.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.915+0000 7f3d6c1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6415e970 con 0x7f3d6415c610 2023-07-06T03:54:02.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.915+0000 7f3d6c1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6415ec50 con 0x7f3d640d2210 2023-07-06T03:54:02.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d69f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d6406b360 0x7f3d641591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d69f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d6406b360 0x7f3d641591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55228/0 (socket says 172.21.15.97:55228) 2023-07-06T03:54:02.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d69f9b700 1 -- 172.21.15.97:0/4236505630 learned_addr learned my addr 172.21.15.97:0/4236505630 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:02.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d6a79c700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d6415c610 0x7f3d640cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d6979a700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d640d2210 0x7f3d6415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d69f9b700 1 -- 172.21.15.97:0/4236505630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d640d2210 msgr2=0x7f3d6415d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d69f9b700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d640d2210 0x7f3d6415d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d69f9b700 1 -- 172.21.15.97:0/4236505630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d6415c610 msgr2=0x7f3d640cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d69f9b700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d6415c610 0x7f3d640cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d69f9b700 1 -- 172.21.15.97:0/4236505630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c009a70 con 0x7f3d6406b360 2023-07-06T03:54:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d69f9b700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d6406b360 0x7f3d641591c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c4e0 tx=0x7f3d6000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d5a7fc700 1 -- 172.21.15.97:0/4236505630 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d600052d0 con 0x7f3d6406b360 2023-07-06T03:54:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d5a7fc700 1 -- 172.21.15.97:0/4236505630 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d60005430 con 0x7f3d6406b360 2023-07-06T03:54:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/4236505630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6415ef30 con 0x7f3d6406b360 2023-07-06T03:54:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.916+0000 7f3d5a7fc700 1 -- 172.21.15.97:0/4236505630 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d600038c0 con 0x7f3d6406b360 2023-07-06T03:54:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.917+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/4236505630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6415f4f0 con 0x7f3d6406b360 2023-07-06T03:54:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.918+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/4236505630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d640c6860 con 0x7f3d6406b360 2023-07-06T03:54:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.919+0000 7f3d5a7fc700 1 -- 172.21.15.97:0/4236505630 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d60003a20 con 0x7f3d6406b360 2023-07-06T03:54:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.920+0000 7f3d5a7fc700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3d4806ace0 0x7f3d4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.920+0000 7f3d5a7fc700 1 -- 172.21.15.97:0/4236505630 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f3d60088a30 con 0x7f3d6406b360 2023-07-06T03:54:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.920+0000 7f3d6979a700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3d4806ace0 0x7f3d4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.920+0000 7f3d6979a700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3d4806ace0 0x7f3d4806d190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3d641f5540 tx=0x7f3d54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:02.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:02.924+0000 7f3d5a7fc700 1 -- 172.21.15.97:0/4236505630 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3d60053a50 con 0x7f3d6406b360 2023-07-06T03:54:03.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.138+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/4236505630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3d6415f6b0 con 0x7f3d6406b360 2023-07-06T03:54:03.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.139+0000 7f3d5a7fc700 1 -- 172.21.15.97:0/4236505630 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3d60057070 con 0x7f3d6406b360 2023-07-06T03:54:03.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/4236505630 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3d4806ace0 msgr2=0x7f3d4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:03.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3d4806ace0 0x7f3d4806d190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3d641f5540 tx=0x7f3d54009040 comp rx=0 tx=0).stop 2023-07-06T03:54:03.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/4236505630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d6406b360 msgr2=0x7f3d641591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:03.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d6406b360 0x7f3d641591c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c4e0 tx=0x7f3d6000c8a0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/4236505630 shutdown_connections 2023-07-06T03:54:03.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3d4806ace0 0x7f3d4806d190 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d6415c610 0x7f3d640cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d6406b360 0x7f3d641591c0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 --2- 172.21.15.97:0/4236505630 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d640d2210 0x7f3d6415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/4236505630 >> 172.21.15.97:0/4236505630 conn(0x7f3d6406a6c0 msgr2=0x7f3d640b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/4236505630 shutdown_connections 2023-07-06T03:54:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.141+0000 7f3d6c1ff700 1 -- 172.21.15.97:0/4236505630 wait complete. 2023-07-06T03:54:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:54:03.162 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:54:03.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:03.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:54:03.229 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[154228]: fuse finished with error 0 and tester_r 0 2023-07-06T03:54:03.233 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:54:03.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:03.235 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:54:03.380 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:54:03.381 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:03.433 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.624+0000 7faed43ff700 1 Processor -- start 2023-07-06T03:54:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.624+0000 7faed43ff700 1 -- start start 2023-07-06T03:54:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.625+0000 7faed43ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faecc15c520 0x7faecc0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.625+0000 7faed43ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faecc160d00 0x7faecc0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.625+0000 7faed43ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faecc0cdae0 con 0x7faecc160180 2023-07-06T03:54:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.625+0000 7faed43ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faecc0cd8d0 con 0x7faecc160d00 2023-07-06T03:54:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.625+0000 7faed43ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faecc0b6640 con 0x7faecc15c520 2023-07-06T03:54:03.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.626+0000 7faed299c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faecc160d00 0x7faecc0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:03.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.626+0000 7faed299c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faecc160d00 0x7faecc0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55242/0 (socket says 172.21.15.97:55242) 2023-07-06T03:54:03.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.626+0000 7faed299c700 1 -- 172.21.15.97:0/781565567 learned_addr learned my addr 172.21.15.97:0/781565567 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:03.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.626+0000 7faed219b700 1 --2- 172.21.15.97:0/781565567 >> v2:172.21.15.146:3300/0 conn(0x7faecc15c520 0x7faecc0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:03.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.626+0000 7faed299c700 1 -- 172.21.15.97:0/781565567 >> v2:172.21.15.146:3300/0 conn(0x7faecc15c520 msgr2=0x7faecc0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:03.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.626+0000 7faed299c700 1 --2- 172.21.15.97:0/781565567 >> v2:172.21.15.146:3300/0 conn(0x7faecc15c520 0x7faecc0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.626+0000 7faed299c700 1 -- 172.21.15.97:0/781565567 >> v1:172.21.15.146:6789/0 conn(0x7faecc160180 legacy=0x7faecc1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:03.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.626+0000 7faed299c700 1 -- 172.21.15.97:0/781565567 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc0b6920 con 0x7faecc160d00 2023-07-06T03:54:03.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.626+0000 7faed219b700 1 --2- 172.21.15.97:0/781565567 >> v2:172.21.15.146:3300/0 conn(0x7faecc15c520 0x7faecc0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:03.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.626+0000 7faed299c700 1 --2- 172.21.15.97:0/781565567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc160d00 0x7faecc0b7400 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7faecc0b4c70 tx=0x7faec400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efc2d01bfedcc7c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:03.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.627+0000 7faed1199700 1 -- 172.21.15.97:0/781565567 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec400e070 con 0x7faecc160d00 2023-07-06T03:54:03.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.627+0000 7faed1199700 1 -- 172.21.15.97:0/781565567 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec4003a20 con 0x7faecc160d00 2023-07-06T03:54:03.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.627+0000 7faed1199700 1 -- 172.21.15.97:0/781565567 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4003dd0 con 0x7faecc160d00 2023-07-06T03:54:03.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.627+0000 7faed43ff700 1 -- 172.21.15.97:0/781565567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc160d00 msgr2=0x7faecc0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:03.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.627+0000 7faed43ff700 1 --2- 172.21.15.97:0/781565567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc160d00 0x7faecc0b7400 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7faecc0b4c70 tx=0x7faec400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:03.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.627+0000 7faed43ff700 1 -- 172.21.15.97:0/781565567 shutdown_connections 2023-07-06T03:54:03.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.627+0000 7faed43ff700 1 --2- 172.21.15.97:0/781565567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc160d00 0x7faecc0b7400 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.627+0000 7faed43ff700 1 --2- 172.21.15.97:0/781565567 >> v2:172.21.15.146:3300/0 conn(0x7faecc15c520 0x7faecc0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.627+0000 7faed43ff700 1 -- 172.21.15.97:0/781565567 >> 172.21.15.97:0/781565567 conn(0x7faecc071790 msgr2=0x7faecc071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:03.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.627+0000 7faed43ff700 1 -- 172.21.15.97:0/781565567 shutdown_connections 2023-07-06T03:54:03.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.627+0000 7faed43ff700 1 -- 172.21.15.97:0/781565567 wait complete. 2023-07-06T03:54:03.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.629+0000 7faed43ff700 1 Processor -- start 2023-07-06T03:54:03.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.629+0000 7faed43ff700 1 -- start start 2023-07-06T03:54:03.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.630+0000 7faed43ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15c520 0x7faecc14cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:03.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.630+0000 7faed43ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faecc160180 0x7faecc0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:03.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.630+0000 7faed43ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faecc160d00 0x7faecc0d3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:03.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.630+0000 7faed43ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc155cf0 con 0x7faecc15c520 2023-07-06T03:54:03.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.630+0000 7faed43ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc155070 con 0x7faecc160180 2023-07-06T03:54:03.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.630+0000 7faed43ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faecc155350 con 0x7faecc160d00 2023-07-06T03:54:03.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.630+0000 7faed219b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15c520 0x7faecc14cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:03.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.630+0000 7faed219b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15c520 0x7faecc14cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55256/0 (socket says 172.21.15.97:55256) 2023-07-06T03:54:03.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.630+0000 7faed219b700 1 -- 172.21.15.97:0/1199086550 learned_addr learned my addr 172.21.15.97:0/1199086550 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:03.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.630+0000 7faed199a700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faecc160180 0x7faecc0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faed299c700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faecc160d00 0x7faecc0d3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faed219b700 1 -- 172.21.15.97:0/1199086550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faecc160d00 msgr2=0x7faecc0d3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faed219b700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faecc160d00 0x7faecc0d3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faed219b700 1 -- 172.21.15.97:0/1199086550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faecc160180 msgr2=0x7faecc0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faed219b700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faecc160180 0x7faecc0ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faed219b700 1 -- 172.21.15.97:0/1199086550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec4009b30 con 0x7faecc15c520 2023-07-06T03:54:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faed199a700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faecc160180 0x7faecc0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faed219b700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15c520 0x7faecc14cce0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7faec800c7f0 tx=0x7faec800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faec2ffd700 1 -- 172.21.15.97:0/1199086550 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8011800 con 0x7faecc15c520 2023-07-06T03:54:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faec2ffd700 1 -- 172.21.15.97:0/1199086550 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec8011960 con 0x7faecc15c520 2023-07-06T03:54:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faed43ff700 1 -- 172.21.15.97:0/1199086550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faecc155630 con 0x7faecc15c520 2023-07-06T03:54:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faec2ffd700 1 -- 172.21.15.97:0/1199086550 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8011d10 con 0x7faecc15c520 2023-07-06T03:54:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.631+0000 7faed43ff700 1 -- 172.21.15.97:0/1199086550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faecc0d37d0 con 0x7faecc15c520 2023-07-06T03:54:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.632+0000 7faed43ff700 1 -- 172.21.15.97:0/1199086550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faecc04e340 con 0x7faecc15c520 2023-07-06T03:54:03.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.633+0000 7faec2ffd700 1 -- 172.21.15.97:0/1199086550 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faec8018360 con 0x7faecc15c520 2023-07-06T03:54:03.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.634+0000 7faec2ffd700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faeb006ace0 0x7faeb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:03.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.634+0000 7faec2ffd700 1 -- 172.21.15.97:0/1199086550 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7faec80894f0 con 0x7faecc15c520 2023-07-06T03:54:03.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.634+0000 7faed199a700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faeb006ace0 0x7faeb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:03.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.635+0000 7faed199a700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faeb006ace0 0x7faeb006d190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7faecc15d240 tx=0x7faebc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.638+0000 7faec2ffd700 1 -- 172.21.15.97:0/1199086550 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faec8057fa0 con 0x7faecc15c520 2023-07-06T03:54:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.839+0000 7faed43ff700 1 -- 172.21.15.97:0/1199086550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faecc0d39c0 con 0x7faecc15c520 2023-07-06T03:54:03.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.841+0000 7faec2ffd700 1 -- 172.21.15.97:0/1199086550 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+18516 (secure 0 0 0) 0x7faec8057b30 con 0x7faecc15c520 2023-07-06T03:54:03.844 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:03.844 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":101,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:53:54.754060+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:53:24.013743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-06T03:53:25.222178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":6}]},{"pgid":"16.9","mappings":[{"from":4,"to":3}]},{"pgid":"16.15","mappings":[{"from":4,"to":3}]},{"pgid":"16.24","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 -- 172.21.15.97:0/1199086550 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faeb006ace0 msgr2=0x7faeb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faeb006ace0 0x7faeb006d190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7faecc15d240 tx=0x7faebc009040 comp rx=0 tx=0).stop 2023-07-06T03:54:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 -- 172.21.15.97:0/1199086550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15c520 msgr2=0x7faecc14cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15c520 0x7faecc14cce0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7faec800c7f0 tx=0x7faec800cbb0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 -- 172.21.15.97:0/1199086550 shutdown_connections 2023-07-06T03:54:03.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faeb006ace0 0x7faeb006d190 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faecc160d00 0x7faecc0d3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15c520 0x7faecc14cce0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 --2- 172.21.15.97:0/1199086550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faecc160180 0x7faecc0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:03.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 -- 172.21.15.97:0/1199086550 >> 172.21.15.97:0/1199086550 conn(0x7faecc071790 msgr2=0x7faecc156bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:03.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 -- 172.21.15.97:0/1199086550 shutdown_connections 2023-07-06T03:54:03.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:03.843+0000 7faed43ff700 1 -- 172.21.15.97:0/1199086550 wait complete. 2023-07-06T03:54:03.863 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:53:24.013743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:53:25.222178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:54:03.864 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:54:04.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.059+0000 7f0f717ff700 1 Processor -- start 2023-07-06T03:54:04.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.059+0000 7f0f717ff700 1 -- start start 2023-07-06T03:54:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.060+0000 7f0f717ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0f6c0d2210 0x7f0f6c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.060+0000 7f0f717ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6c0cff90 con 0x7f0f6c15c610 2023-07-06T03:54:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.060+0000 7f0f717ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6c0b5650 con 0x7f0f6c06b360 2023-07-06T03:54:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.060+0000 7f0f717ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6c0d0780 con 0x7f0f6c0d2210 2023-07-06T03:54:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.060+0000 7f0f6affd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0f6c15c610 0x7f0f6c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56736/0 (socket says 172.21.15.97:56736) 2023-07-06T03:54:04.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.060+0000 7f0f6affd700 1 -- 172.21.15.97:0/37046949 learned_addr learned my addr 172.21.15.97:0/37046949 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:04.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6b7fe700 1 --2- 172.21.15.97:0/37046949 >> v2:172.21.15.146:3300/0 conn(0x7f0f6c0d2210 0x7f0f6c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870589218 0 0) 0x7f0f6c0cff90 con 0x7f0f6c15c610 2023-07-06T03:54:04.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f580035b0 con 0x7f0f6c15c610 2023-07-06T03:54:04.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1977466364 0 0) 0x7f0f580035b0 con 0x7f0f6c15c610 2023-07-06T03:54:04.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 >> v2:172.21.15.146:3300/0 conn(0x7f0f6c0d2210 msgr2=0x7f0f6c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:04.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 --2- 172.21.15.97:0/37046949 >> v2:172.21.15.146:3300/0 conn(0x7f0f6c0d2210 0x7f0f6c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 >> v1:172.21.15.146:6789/0 conn(0x7f0f6c06b360 legacy=0x7f0f6c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c0b5400 con 0x7f0f6c15c610 2023-07-06T03:54:04.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f0f5c001670 con 0x7f0f6c15c610 2023-07-06T03:54:04.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 >> v1:172.21.15.97:6789/0 conn(0x7f0f6c15c610 legacy=0x7f0f6c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 --2- 172.21.15.97:0/37046949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f58003a00 0x7f0f58003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 --2- 172.21.15.97:0/37046949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 0x7f0f58004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 --2- 172.21.15.97:0/37046949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f58005200 0x7f0f5800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c0d0780 con 0x7f0f58004850 2023-07-06T03:54:04.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5800c080 con 0x7f0f58005200 2023-07-06T03:54:04.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5800c240 con 0x7f0f58003a00 2023-07-06T03:54:04.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.061+0000 7f0f63fff700 1 --2- 172.21.15.97:0/37046949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 0x7f0f58004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f6affd700 1 --2- 172.21.15.97:0/37046949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f58003a00 0x7f0f58003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f6b7fe700 1 --2- 172.21.15.97:0/37046949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f58005200 0x7f0f5800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f63fff700 1 -- 172.21.15.97:0/37046949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f58003a00 msgr2=0x7f0f58003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f63fff700 1 --2- 172.21.15.97:0/37046949 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f58003a00 0x7f0f58003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f63fff700 1 -- 172.21.15.97:0/37046949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f58005200 msgr2=0x7f0f5800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f63fff700 1 --2- 172.21.15.97:0/37046949 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f58005200 0x7f0f5800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f63fff700 1 -- 172.21.15.97:0/37046949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580035b0 con 0x7f0f58004850 2023-07-06T03:54:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f6b7fe700 1 -- 172.21.15.97:0/37046949 reap_dead start 2023-07-06T03:54:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f63fff700 1 --2- 172.21.15.97:0/37046949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 0x7f0f58004cc0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0f540017c0 tx=0x7f0f58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab2aa8104e8acb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:04.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f540052d0 con 0x7f0f58004850 2023-07-06T03:54:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f54005450 con 0x7f0f58004850 2023-07-06T03:54:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.062+0000 7f0f6a7fc700 1 -- 172.21.15.97:0/37046949 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f540038c0 con 0x7f0f58004850 2023-07-06T03:54:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.063+0000 7f0f717ff700 1 -- 172.21.15.97:0/37046949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 msgr2=0x7f0f58004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.063+0000 7f0f717ff700 1 --2- 172.21.15.97:0/37046949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 0x7f0f58004cc0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0f540017c0 tx=0x7f0f58005880 comp rx=0 tx=0).stop 2023-07-06T03:54:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.063+0000 7f0f717ff700 1 -- 172.21.15.97:0/37046949 shutdown_connections 2023-07-06T03:54:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.063+0000 7f0f717ff700 1 --2- 172.21.15.97:0/37046949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 0x7f0f58004cc0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.063+0000 7f0f717ff700 1 -- 172.21.15.97:0/37046949 >> 172.21.15.97:0/37046949 conn(0x7f0f6c06a6c0 msgr2=0x7f0f6c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:04.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.063+0000 7f0f717ff700 1 -- 172.21.15.97:0/37046949 shutdown_connections 2023-07-06T03:54:04.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.063+0000 7f0f717ff700 1 -- 172.21.15.97:0/37046949 wait complete. 2023-07-06T03:54:04.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.065+0000 7f0f717ff700 1 Processor -- start 2023-07-06T03:54:04.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.065+0000 7f0f717ff700 1 -- start start 2023-07-06T03:54:04.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f717ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 0x7f0f6c1f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f717ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f6c0cc420 0x7f0f6c0cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f717ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6c0b5f00 0x7f0f6c0b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f717ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c0cad30 con 0x7f0f58004850 2023-07-06T03:54:04.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f717ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c0b68b0 con 0x7f0f6c0b5f00 2023-07-06T03:54:04.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f717ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c1f2180 con 0x7f0f6c0cc420 2023-07-06T03:54:04.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f6affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 0x7f0f6c1f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f6affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 0x7f0f6c1f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55282/0 (socket says 172.21.15.97:55282) 2023-07-06T03:54:04.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f6affd700 1 -- 172.21.15.97:0/551932189 learned_addr learned my addr 172.21.15.97:0/551932189 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:04.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f63fff700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f6c0cc420 0x7f0f6c0cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f6b7fe700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6c0b5f00 0x7f0f6c0b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f6affd700 1 -- 172.21.15.97:0/551932189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f6c0cc420 msgr2=0x7f0f6c0cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f6affd700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f6c0cc420 0x7f0f6c0cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f6affd700 1 -- 172.21.15.97:0/551932189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6c0b5f00 msgr2=0x7f0f6c0b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f6affd700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6c0b5f00 0x7f0f6c0b6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f6affd700 1 -- 172.21.15.97:0/551932189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c0cbf90 con 0x7f0f58004850 2023-07-06T03:54:04.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.066+0000 7f0f6b7fe700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6c0b5f00 0x7f0f6c0b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:04.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.067+0000 7f0f6affd700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 0x7f0f6c1f1c40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0072c0 tx=0x7f0f5c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:04.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.067+0000 7f0f637fe700 1 -- 172.21.15.97:0/551932189 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c00f7f0 con 0x7f0f58004850 2023-07-06T03:54:04.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.067+0000 7f0f637fe700 1 -- 172.21.15.97:0/551932189 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5c00f970 con 0x7f0f58004850 2023-07-06T03:54:04.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.067+0000 7f0f637fe700 1 -- 172.21.15.97:0/551932189 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c00fd20 con 0x7f0f58004850 2023-07-06T03:54:04.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.067+0000 7f0f717ff700 1 -- 172.21.15.97:0/551932189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6c0caa50 con 0x7f0f58004850 2023-07-06T03:54:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.067+0000 7f0f717ff700 1 -- 172.21.15.97:0/551932189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6c0cbc20 con 0x7f0f58004850 2023-07-06T03:54:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.068+0000 7f0f717ff700 1 -- 172.21.15.97:0/551932189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f6c0cb7a0 con 0x7f0f58004850 2023-07-06T03:54:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.069+0000 7f0f637fe700 1 -- 172.21.15.97:0/551932189 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f5c01d360 con 0x7f0f58004850 2023-07-06T03:54:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.070+0000 7f0f637fe700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f4c06ad10 0x7f0f4c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.070+0000 7f0f637fe700 1 -- 172.21.15.97:0/551932189 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f0f5c088c80 con 0x7f0f58004850 2023-07-06T03:54:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.070+0000 7f0f63fff700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f4c06ad10 0x7f0f4c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.071+0000 7f0f63fff700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f4c06ad10 0x7f0f4c06d1c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0f54000c00 tx=0x7f0f54009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.075+0000 7f0f637fe700 1 -- 172.21.15.97:0/551932189 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0f6c0cb7a0 con 0x7f0f58004850 2023-07-06T03:54:04.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.269+0000 7f0f717ff700 1 -- 172.21.15.97:0/551932189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f6c0cb7a0 con 0x7f0f58004850 2023-07-06T03:54:04.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.269+0000 7f0f637fe700 1 -- 172.21.15.97:0/551932189 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f6c0cb7a0 con 0x7f0f58004850 2023-07-06T03:54:04.272 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:54:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 -- 172.21.15.97:0/551932189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f4c06ad10 msgr2=0x7f0f4c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f4c06ad10 0x7f0f4c06d1c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0f54000c00 tx=0x7f0f54009030 comp rx=0 tx=0).stop 2023-07-06T03:54:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 -- 172.21.15.97:0/551932189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 msgr2=0x7f0f6c1f1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 0x7f0f6c1f1c40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0072c0 tx=0x7f0f5c007680 comp rx=0 tx=0).stop 2023-07-06T03:54:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 -- 172.21.15.97:0/551932189 shutdown_connections 2023-07-06T03:54:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f4c06ad10 0x7f0f4c06d1c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f6c0b5f00 0x7f0f6c0b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58004850 0x7f0f6c1f1c40 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 --2- 172.21.15.97:0/551932189 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f6c0cc420 0x7f0f6c0cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 -- 172.21.15.97:0/551932189 >> 172.21.15.97:0/551932189 conn(0x7f0f6c06a6c0 msgr2=0x7f0f6c0b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 -- 172.21.15.97:0/551932189 shutdown_connections 2023-07-06T03:54:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.272+0000 7f0f717ff700 1 -- 172.21.15.97:0/551932189 wait complete. 2023-07-06T03:54:04.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:54:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.488+0000 7f0879fff700 1 Processor -- start 2023-07-06T03:54:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.488+0000 7f0879fff700 1 -- start start 2023-07-06T03:54:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.489+0000 7f0879fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f087406b360 0x7f08740b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0879fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08740d2210 0x7f08740d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0879fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08740cff90 con 0x7f087415c610 2023-07-06T03:54:04.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0879fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08740d0780 con 0x7f087406b360 2023-07-06T03:54:04.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0879fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087415c9e0 con 0x7f08740d2210 2023-07-06T03:54:04.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0872ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f087406b360 0x7f08740b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0873fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08740d2210 0x7f08740d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0872ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f087406b360 0x7f08740b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:37018/0 (socket says 172.21.15.97:37018) 2023-07-06T03:54:04.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0872ffd700 1 -- 172.21.15.97:0/1372297744 learned_addr learned my addr 172.21.15.97:0/1372297744 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0872ffd700 1 -- 172.21.15.97:0/1372297744 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08740d2210 msgr2=0x7f08740d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0872ffd700 1 --2- 172.21.15.97:0/1372297744 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08740d2210 0x7f08740d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0872ffd700 1 -- 172.21.15.97:0/1372297744 >> v1:172.21.15.146:6789/0 conn(0x7f087415c610 legacy=0x7f08740b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.490+0000 7f0872ffd700 1 -- 172.21.15.97:0/1372297744 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740b5650 con 0x7f087406b360 2023-07-06T03:54:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.491+0000 7f0872ffd700 1 --2- 172.21.15.97:0/1372297744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f087406b360 0x7f08740b4ec0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f08680098f0 tx=0x7f0868009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2446d35cbb02c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.491+0000 7f08727fc700 1 -- 172.21.15.97:0/1372297744 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868011830 con 0x7f087406b360 2023-07-06T03:54:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.491+0000 7f08727fc700 1 -- 172.21.15.97:0/1372297744 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0868011990 con 0x7f087406b360 2023-07-06T03:54:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.491+0000 7f08727fc700 1 -- 172.21.15.97:0/1372297744 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868011d20 con 0x7f087406b360 2023-07-06T03:54:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.492+0000 7f0879fff700 1 -- 172.21.15.97:0/1372297744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f087406b360 msgr2=0x7f08740b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.492+0000 7f0879fff700 1 --2- 172.21.15.97:0/1372297744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f087406b360 0x7f08740b4ec0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f08680098f0 tx=0x7f0868009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.492+0000 7f0879fff700 1 -- 172.21.15.97:0/1372297744 shutdown_connections 2023-07-06T03:54:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.492+0000 7f0879fff700 1 --2- 172.21.15.97:0/1372297744 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08740d2210 0x7f08740d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.492+0000 7f0879fff700 1 --2- 172.21.15.97:0/1372297744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f087406b360 0x7f08740b4ec0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.492+0000 7f0879fff700 1 -- 172.21.15.97:0/1372297744 >> 172.21.15.97:0/1372297744 conn(0x7f087406a6c0 msgr2=0x7f087406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.492+0000 7f0879fff700 1 -- 172.21.15.97:0/1372297744 shutdown_connections 2023-07-06T03:54:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.492+0000 7f0879fff700 1 -- 172.21.15.97:0/1372297744 wait complete. 2023-07-06T03:54:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.493+0000 7f0879fff700 1 Processor -- start 2023-07-06T03:54:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.494+0000 7f0879fff700 1 -- start start 2023-07-06T03:54:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.494+0000 7f0879fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f087406b360 0x7f0874153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.494+0000 7f0879fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08740d2210 0x7f08740ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.494+0000 7f0879fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f087415c610 0x7f087415e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.494+0000 7f0879fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087415ea60 con 0x7f08740d2210 2023-07-06T03:54:04.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.494+0000 7f0879fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740c9e00 con 0x7f087415c610 2023-07-06T03:54:04.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.494+0000 7f0879fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087415ece0 con 0x7f087406b360 2023-07-06T03:54:04.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.495+0000 7f0872ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08740d2210 0x7f08740ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.495+0000 7f0872ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08740d2210 0x7f08740ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55284/0 (socket says 172.21.15.97:55284) 2023-07-06T03:54:04.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.495+0000 7f0872ffd700 1 -- 172.21.15.97:0/959031847 learned_addr learned my addr 172.21.15.97:0/959031847 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:04.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.495+0000 7f08737fe700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f087406b360 0x7f0874153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.495+0000 7f0873fff700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f087415c610 0x7f087415e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.495+0000 7f0872ffd700 1 -- 172.21.15.97:0/959031847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f087406b360 msgr2=0x7f0874153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.495+0000 7f0872ffd700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f087406b360 0x7f0874153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.495+0000 7f0872ffd700 1 -- 172.21.15.97:0/959031847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f087415c610 msgr2=0x7f087415e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.495+0000 7f0872ffd700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f087415c610 0x7f087415e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.495+0000 7f0872ffd700 1 -- 172.21.15.97:0/959031847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868009550 con 0x7f08740d2210 2023-07-06T03:54:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.495+0000 7f0872ffd700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08740d2210 0x7f08740ca8a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f0868000c00 tx=0x7f0868004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.496+0000 7f085bfff700 1 -- 172.21.15.97:0/959031847 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868004400 con 0x7f08740d2210 2023-07-06T03:54:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.496+0000 7f085bfff700 1 -- 172.21.15.97:0/959031847 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086801f070 con 0x7f08740d2210 2023-07-06T03:54:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.496+0000 7f0879fff700 1 -- 172.21.15.97:0/959031847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087415efc0 con 0x7f08740d2210 2023-07-06T03:54:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.496+0000 7f085bfff700 1 -- 172.21.15.97:0/959031847 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680038e0 con 0x7f08740d2210 2023-07-06T03:54:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.496+0000 7f0879fff700 1 -- 172.21.15.97:0/959031847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08740ca380 con 0x7f08740d2210 2023-07-06T03:54:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.497+0000 7f0879fff700 1 -- 172.21.15.97:0/959031847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087404e340 con 0x7f08740d2210 2023-07-06T03:54:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.498+0000 7f085bfff700 1 -- 172.21.15.97:0/959031847 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0868021030 con 0x7f08740d2210 2023-07-06T03:54:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.499+0000 7f085bfff700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f086006ace0 0x7f086006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.499+0000 7f085bfff700 1 -- 172.21.15.97:0/959031847 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f086808a940 con 0x7f08740d2210 2023-07-06T03:54:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.499+0000 7f08737fe700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f086006ace0 0x7f086006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.500+0000 7f08737fe700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f086006ace0 0x7f086006d190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f08741f53d0 tx=0x7f0864009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.503+0000 7f085bfff700 1 -- 172.21.15.97:0/959031847 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0868055a00 con 0x7f08740d2210 2023-07-06T03:54:04.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.697+0000 7f0879fff700 1 -- 172.21.15.97:0/959031847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f08740b5b60 con 0x7f08740d2210 2023-07-06T03:54:04.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.697+0000 7f085bfff700 1 -- 172.21.15.97:0/959031847 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v101) v1 ==== 92+0+10 (secure 0 0 0) 0x7f08680150a0 con 0x7f08740d2210 2023-07-06T03:54:04.700 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:54:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.699+0000 7f0879fff700 1 -- 172.21.15.97:0/959031847 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f086006ace0 msgr2=0x7f086006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.699+0000 7f0879fff700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f086006ace0 0x7f086006d190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f08741f53d0 tx=0x7f0864009040 comp rx=0 tx=0).stop 2023-07-06T03:54:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.699+0000 7f0879fff700 1 -- 172.21.15.97:0/959031847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08740d2210 msgr2=0x7f08740ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.699+0000 7f0879fff700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08740d2210 0x7f08740ca8a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f0868000c00 tx=0x7f0868004750 comp rx=0 tx=0).stop 2023-07-06T03:54:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.699+0000 7f0879fff700 1 -- 172.21.15.97:0/959031847 shutdown_connections 2023-07-06T03:54:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.699+0000 7f0879fff700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f086006ace0 0x7f086006d190 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.699+0000 7f0879fff700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f087415c610 0x7f087415e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.699+0000 7f0879fff700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f087406b360 0x7f0874153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.700+0000 7f0879fff700 1 --2- 172.21.15.97:0/959031847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08740d2210 0x7f08740ca8a0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.700+0000 7f0879fff700 1 -- 172.21.15.97:0/959031847 >> 172.21.15.97:0/959031847 conn(0x7f087406a6c0 msgr2=0x7f08740d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.700+0000 7f0879fff700 1 -- 172.21.15.97:0/959031847 shutdown_connections 2023-07-06T03:54:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.700+0000 7f0879fff700 1 -- 172.21.15.97:0/959031847 wait complete. 2023-07-06T03:54:04.720 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:54:04.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.916+0000 7feb279ff700 1 Processor -- start 2023-07-06T03:54:04.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.917+0000 7feb279ff700 1 -- start start 2023-07-06T03:54:04.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.918+0000 7feb279ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb2006b360 0x7feb200b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.918+0000 7feb279ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feb200d2210 0x7feb200d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.918+0000 7feb279ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb200cff90 con 0x7feb2015c610 2023-07-06T03:54:04.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.918+0000 7feb279ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200d0780 con 0x7feb2006b360 2023-07-06T03:54:04.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.918+0000 7feb279ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2015c9e0 con 0x7feb200d2210 2023-07-06T03:54:04.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.918+0000 7feb24f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb2006b360 0x7feb200b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.918+0000 7feb24f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb2006b360 0x7feb200b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55298/0 (socket says 172.21.15.97:55298) 2023-07-06T03:54:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.918+0000 7feb24f9a700 1 -- 172.21.15.97:0/487251779 learned_addr learned my addr 172.21.15.97:0/487251779 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:04.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.919+0000 7feb25f9c700 1 --2- 172.21.15.97:0/487251779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feb200d2210 0x7feb200d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.919+0000 7feb24f9a700 1 -- 172.21.15.97:0/487251779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feb200d2210 msgr2=0x7feb200d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:04.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.919+0000 7feb24f9a700 1 --2- 172.21.15.97:0/487251779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feb200d2210 0x7feb200d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.919+0000 7feb24f9a700 1 -- 172.21.15.97:0/487251779 >> v1:172.21.15.146:6789/0 conn(0x7feb2015c610 legacy=0x7feb200b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:04.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.919+0000 7feb24f9a700 1 -- 172.21.15.97:0/487251779 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb200b5650 con 0x7feb2006b360 2023-07-06T03:54:04.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.919+0000 7feb24f9a700 1 --2- 172.21.15.97:0/487251779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb2006b360 0x7feb200b4ec0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7feb100098f0 tx=0x7feb10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15172486f09ec20 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:04.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.919+0000 7feb17fff700 1 -- 172.21.15.97:0/487251779 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10004230 con 0x7feb2006b360 2023-07-06T03:54:04.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.919+0000 7feb17fff700 1 -- 172.21.15.97:0/487251779 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb10004390 con 0x7feb2006b360 2023-07-06T03:54:04.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.919+0000 7feb17fff700 1 -- 172.21.15.97:0/487251779 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100036a0 con 0x7feb2006b360 2023-07-06T03:54:04.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.920+0000 7feb279ff700 1 -- 172.21.15.97:0/487251779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb2006b360 msgr2=0x7feb200b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.920+0000 7feb279ff700 1 --2- 172.21.15.97:0/487251779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb2006b360 0x7feb200b4ec0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7feb100098f0 tx=0x7feb10009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:04.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.920+0000 7feb279ff700 1 -- 172.21.15.97:0/487251779 shutdown_connections 2023-07-06T03:54:04.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.920+0000 7feb279ff700 1 --2- 172.21.15.97:0/487251779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feb200d2210 0x7feb200d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.920+0000 7feb279ff700 1 --2- 172.21.15.97:0/487251779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb2006b360 0x7feb200b4ec0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.920+0000 7feb279ff700 1 -- 172.21.15.97:0/487251779 >> 172.21.15.97:0/487251779 conn(0x7feb2006a6c0 msgr2=0x7feb2006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:04.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.920+0000 7feb279ff700 1 -- 172.21.15.97:0/487251779 shutdown_connections 2023-07-06T03:54:04.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.920+0000 7feb279ff700 1 -- 172.21.15.97:0/487251779 wait complete. 2023-07-06T03:54:04.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.922+0000 7feb279ff700 1 Processor -- start 2023-07-06T03:54:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.922+0000 7feb279ff700 1 -- start start 2023-07-06T03:54:04.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.923+0000 7feb279ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feb2006b360 0x7feb200c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb279ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb200d2210 0x7feb2015f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb279ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb2015c610 0x7feb20158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb279ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2015e650 con 0x7feb200d2210 2023-07-06T03:54:04.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb279ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2015ee60 con 0x7feb2015c610 2023-07-06T03:54:04.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb279ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2015e8a0 con 0x7feb2006b360 2023-07-06T03:54:04.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb24f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb200d2210 0x7feb2015f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb24f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb200d2210 0x7feb2015f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55308/0 (socket says 172.21.15.97:55308) 2023-07-06T03:54:04.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb24f9a700 1 -- 172.21.15.97:0/273413964 learned_addr learned my addr 172.21.15.97:0/273413964 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:04.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb25f9c700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb2015c610 0x7feb20158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb2579b700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feb2006b360 0x7feb200c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb24f9a700 1 -- 172.21.15.97:0/273413964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feb2006b360 msgr2=0x7feb200c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb24f9a700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feb2006b360 0x7feb200c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb24f9a700 1 -- 172.21.15.97:0/273413964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb2015c610 msgr2=0x7feb20158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:04.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb24f9a700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb2015c610 0x7feb20158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:04.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb24f9a700 1 -- 172.21.15.97:0/273413964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb10009550 con 0x7feb200d2210 2023-07-06T03:54:04.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb2579b700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feb2006b360 0x7feb200c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:04.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.924+0000 7feb24f9a700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb200d2210 0x7feb2015f2d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7feb10009130 tx=0x7feb10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:04.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.925+0000 7feb15ffb700 1 -- 172.21.15.97:0/273413964 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10003e30 con 0x7feb200d2210 2023-07-06T03:54:04.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.925+0000 7feb15ffb700 1 -- 172.21.15.97:0/273413964 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1001f070 con 0x7feb200d2210 2023-07-06T03:54:04.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.925+0000 7feb15ffb700 1 -- 172.21.15.97:0/273413964 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1000b8a0 con 0x7feb200d2210 2023-07-06T03:54:04.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.925+0000 7feb279ff700 1 -- 172.21.15.97:0/273413964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2015eb80 con 0x7feb200d2210 2023-07-06T03:54:04.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.925+0000 7feb279ff700 1 -- 172.21.15.97:0/273413964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb200cb260 con 0x7feb200d2210 2023-07-06T03:54:04.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.926+0000 7feb279ff700 1 -- 172.21.15.97:0/273413964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb200d01a0 con 0x7feb200d2210 2023-07-06T03:54:04.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.927+0000 7feb15ffb700 1 -- 172.21.15.97:0/273413964 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb10021030 con 0x7feb200d2210 2023-07-06T03:54:04.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.928+0000 7feb15ffb700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb0406ace0 0x7feb0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:04.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.928+0000 7feb15ffb700 1 -- 172.21.15.97:0/273413964 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7feb1008ac10 con 0x7feb200d2210 2023-07-06T03:54:04.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.929+0000 7feb2579b700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb0406ace0 0x7feb0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:04.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.929+0000 7feb2579b700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb0406ace0 0x7feb0406d190 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7feb2015bd30 tx=0x7feb1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:04.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:04.933+0000 7feb15ffb700 1 -- 172.21.15.97:0/273413964 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feb200d01a0 con 0x7feb200d2210 2023-07-06T03:54:05.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.124+0000 7feb279ff700 1 -- 172.21.15.97:0/273413964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb200d01a0 con 0x7feb200d2210 2023-07-06T03:54:05.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.125+0000 7feb15ffb700 1 -- 172.21.15.97:0/273413964 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb200d01a0 con 0x7feb200d2210 2023-07-06T03:54:05.128 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:54:05.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.127+0000 7feb279ff700 1 -- 172.21.15.97:0/273413964 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb0406ace0 msgr2=0x7feb0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.127+0000 7feb279ff700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb0406ace0 0x7feb0406d190 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7feb2015bd30 tx=0x7feb1c008040 comp rx=0 tx=0).stop 2023-07-06T03:54:05.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.127+0000 7feb279ff700 1 -- 172.21.15.97:0/273413964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb200d2210 msgr2=0x7feb2015f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.128+0000 7feb279ff700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb200d2210 0x7feb2015f2d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7feb10009130 tx=0x7feb10004750 comp rx=0 tx=0).stop 2023-07-06T03:54:05.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.128+0000 7feb279ff700 1 -- 172.21.15.97:0/273413964 shutdown_connections 2023-07-06T03:54:05.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.128+0000 7feb279ff700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feb0406ace0 0x7feb0406d190 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.128+0000 7feb279ff700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb2015c610 0x7feb20158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.128+0000 7feb279ff700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7feb2006b360 0x7feb200c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.128+0000 7feb279ff700 1 --2- 172.21.15.97:0/273413964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb200d2210 0x7feb2015f2d0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.128+0000 7feb279ff700 1 -- 172.21.15.97:0/273413964 >> 172.21.15.97:0/273413964 conn(0x7feb2006a6c0 msgr2=0x7feb200d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:05.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.128+0000 7feb279ff700 1 -- 172.21.15.97:0/273413964 shutdown_connections 2023-07-06T03:54:05.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.128+0000 7feb279ff700 1 -- 172.21.15.97:0/273413964 wait complete. 2023-07-06T03:54:05.149 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:54:05.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.343+0000 7f0feddff700 1 Processor -- start 2023-07-06T03:54:05.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.343+0000 7f0feddff700 1 -- start start 2023-07-06T03:54:05.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.344+0000 7f0feddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fe815c610 0x7f0fe80b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.344+0000 7f0feddff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0fe806b360 0x7f0fe80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.344+0000 7f0feddff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0fe80d2210 0x7f0fe80d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.344+0000 7f0feddff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe80d0780 con 0x7f0fe80d2210 2023-07-06T03:54:05.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.344+0000 7f0feddff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe815c9e0 con 0x7f0fe806b360 2023-07-06T03:54:05.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.344+0000 7f0feddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80b5650 con 0x7f0fe815c610 2023-07-06T03:54:05.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0fe80d2210 0x7f0fe80d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0fe80d2210 0x7f0fe80d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55318/0 (socket says 172.21.15.97:55318) 2023-07-06T03:54:05.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe7fff700 1 -- 172.21.15.97:0/242591325 learned_addr learned my addr 172.21.15.97:0/242591325 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe6ffd700 1 --2- 172.21.15.97:0/242591325 >> v2:172.21.15.146:3300/0 conn(0x7f0fe806b360 0x7f0fe80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe77fe700 1 --2- 172.21.15.97:0/242591325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fe815c610 0x7f0fe80b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe7fff700 1 -- 172.21.15.97:0/242591325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fe815c610 msgr2=0x7f0fe80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:05.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe7fff700 1 --2- 172.21.15.97:0/242591325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fe815c610 0x7f0fe80b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe7fff700 1 -- 172.21.15.97:0/242591325 >> v2:172.21.15.146:3300/0 conn(0x7f0fe806b360 msgr2=0x7f0fe80b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:05.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe7fff700 1 --2- 172.21.15.97:0/242591325 >> v2:172.21.15.146:3300/0 conn(0x7f0fe806b360 0x7f0fe80b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe7fff700 1 -- 172.21.15.97:0/242591325 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe80d01c0 con 0x7f0fe80d2210 2023-07-06T03:54:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe77fe700 1 --2- 172.21.15.97:0/242591325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fe815c610 0x7f0fe80b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.345+0000 7f0fe7fff700 1 --2- 172.21.15.97:0/242591325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fe80d2210 0x7f0fe80d1310 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80b58b0 tx=0x7f0fe000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159c03ab14f9e83f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0fe67fc700 1 -- 172.21.15.97:0/242591325 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe000e070 con 0x7f0fe80d2210 2023-07-06T03:54:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0fe67fc700 1 -- 172.21.15.97:0/242591325 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe000b8d0 con 0x7f0fe80d2210 2023-07-06T03:54:05.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0fe67fc700 1 -- 172.21.15.97:0/242591325 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe000bc80 con 0x7f0fe80d2210 2023-07-06T03:54:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0feddff700 1 -- 172.21.15.97:0/242591325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fe80d2210 msgr2=0x7f0fe80d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0feddff700 1 --2- 172.21.15.97:0/242591325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fe80d2210 0x7f0fe80d1310 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80b58b0 tx=0x7f0fe000b330 comp rx=0 tx=0).stop 2023-07-06T03:54:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0feddff700 1 -- 172.21.15.97:0/242591325 shutdown_connections 2023-07-06T03:54:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0feddff700 1 --2- 172.21.15.97:0/242591325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fe80d2210 0x7f0fe80d1310 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0feddff700 1 --2- 172.21.15.97:0/242591325 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fe815c610 0x7f0fe80b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0feddff700 1 --2- 172.21.15.97:0/242591325 >> v2:172.21.15.146:3300/0 conn(0x7f0fe806b360 0x7f0fe80b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0feddff700 1 -- 172.21.15.97:0/242591325 >> 172.21.15.97:0/242591325 conn(0x7f0fe806a6c0 msgr2=0x7f0fe806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:05.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0feddff700 1 -- 172.21.15.97:0/242591325 shutdown_connections 2023-07-06T03:54:05.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.346+0000 7f0feddff700 1 -- 172.21.15.97:0/242591325 wait complete. 2023-07-06T03:54:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.348+0000 7f0feddff700 1 Processor -- start 2023-07-06T03:54:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.348+0000 7f0feddff700 1 -- start start 2023-07-06T03:54:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.349+0000 7f0feddff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fe806b360 0x7f0fe8155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.349+0000 7f0feddff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fe80d2210 0x7f0fe80cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.349+0000 7f0feddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fe815c610 0x7f0fe8148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.349+0000 7f0feddff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80cccc0 con 0x7f0fe806b360 2023-07-06T03:54:05.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.349+0000 7f0feddff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe81f6970 con 0x7f0fe80d2210 2023-07-06T03:54:05.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.349+0000 7f0feddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe8148640 con 0x7f0fe815c610 2023-07-06T03:54:05.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.349+0000 7f0fe77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fe806b360 0x7f0fe8155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.349+0000 7f0fe77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fe806b360 0x7f0fe8155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55334/0 (socket says 172.21.15.97:55334) 2023-07-06T03:54:05.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.349+0000 7f0fe77fe700 1 -- 172.21.15.97:0/3291533445 learned_addr learned my addr 172.21.15.97:0/3291533445 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:05.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.349+0000 7f0fe6ffd700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fe80d2210 0x7f0fe80cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.349+0000 7f0fe7fff700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fe815c610 0x7f0fe8148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0fe77fe700 1 -- 172.21.15.97:0/3291533445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fe815c610 msgr2=0x7f0fe8148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0fe77fe700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fe815c610 0x7f0fe8148100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0fe77fe700 1 -- 172.21.15.97:0/3291533445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fe80d2210 msgr2=0x7f0fe80cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0fe77fe700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fe80d2210 0x7f0fe80cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0fe77fe700 1 -- 172.21.15.97:0/3291533445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe0009b30 con 0x7f0fe806b360 2023-07-06T03:54:05.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0fe6ffd700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fe80d2210 0x7f0fe80cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0fe77fe700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fe806b360 0x7f0fe8155b40 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800e840 tx=0x7f0fd800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0fcffff700 1 -- 172.21.15.97:0/3291533445 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd800c020 con 0x7f0fe806b360 2023-07-06T03:54:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0fcffff700 1 -- 172.21.15.97:0/3291533445 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd8004250 con 0x7f0fe806b360 2023-07-06T03:54:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0fcffff700 1 -- 172.21.15.97:0/3291533445 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd80036a0 con 0x7f0fe806b360 2023-07-06T03:54:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0feddff700 1 -- 172.21.15.97:0/3291533445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe8156200 con 0x7f0fe806b360 2023-07-06T03:54:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.350+0000 7f0feddff700 1 -- 172.21.15.97:0/3291533445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe81567c0 con 0x7f0fe806b360 2023-07-06T03:54:05.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.351+0000 7f0feddff700 1 -- 172.21.15.97:0/3291533445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe804e340 con 0x7f0fe806b360 2023-07-06T03:54:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.352+0000 7f0fcffff700 1 -- 172.21.15.97:0/3291533445 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0fd8003800 con 0x7f0fe806b360 2023-07-06T03:54:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.354+0000 7f0fcffff700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fd406ace0 0x7f0fd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.354+0000 7f0fcffff700 1 -- 172.21.15.97:0/3291533445 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f0fd8088f90 con 0x7f0fe806b360 2023-07-06T03:54:05.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.354+0000 7f0fe6ffd700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fd406ace0 0x7f0fd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.354+0000 7f0fe6ffd700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fd406ace0 0x7f0fd406d190 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80c9db0 tx=0x7f0fdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:05.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.358+0000 7f0fcffff700 1 -- 172.21.15.97:0/3291533445 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0fd8053fc0 con 0x7f0fe806b360 2023-07-06T03:54:05.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.552+0000 7f0feddff700 1 -- 172.21.15.97:0/3291533445 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0fe80c63c0 con 0x7f0fe806b360 2023-07-06T03:54:05.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.553+0000 7f0fcffff700 1 -- 172.21.15.97:0/3291533445 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fd80575e0 con 0x7f0fe806b360 2023-07-06T03:54:05.556 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:54:05.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 -- 172.21.15.97:0/3291533445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fd406ace0 msgr2=0x7f0fd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fd406ace0 0x7f0fd406d190 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80c9db0 tx=0x7f0fdc008040 comp rx=0 tx=0).stop 2023-07-06T03:54:05.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 -- 172.21.15.97:0/3291533445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fe806b360 msgr2=0x7f0fe8155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fe806b360 0x7f0fe8155b40 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800e840 tx=0x7f0fd800ec00 comp rx=0 tx=0).stop 2023-07-06T03:54:05.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 -- 172.21.15.97:0/3291533445 shutdown_connections 2023-07-06T03:54:05.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0fd406ace0 0x7f0fd406d190 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0fe815c610 0x7f0fe8148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fe806b360 0x7f0fe8155b40 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 --2- 172.21.15.97:0/3291533445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fe80d2210 0x7f0fe80cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 -- 172.21.15.97:0/3291533445 >> 172.21.15.97:0/3291533445 conn(0x7f0fe806a6c0 msgr2=0x7f0fe80b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:05.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 -- 172.21.15.97:0/3291533445 shutdown_connections 2023-07-06T03:54:05.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.555+0000 7f0feddff700 1 -- 172.21.15.97:0/3291533445 wait complete. 2023-07-06T03:54:05.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:54:05.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.769+0000 7f11f259d700 1 Processor -- start 2023-07-06T03:54:05.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.769+0000 7f11f259d700 1 -- start start 2023-07-06T03:54:05.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.770+0000 7f11f259d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f11ec15c4b0 0x7f11ec0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.770+0000 7f11f259d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ec0d08b0 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.770+0000 7f11f259d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ec0d2700 con 0x7f11ec15c0e0 2023-07-06T03:54:05.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.770+0000 7f11f259d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ec0d0b50 con 0x7f11ec15c4b0 2023-07-06T03:54:05.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.770+0000 7f11f1d9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f11ec0cd7d0 0x7f11ec0b46c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56738/0 (socket says 172.21.15.97:56738) 2023-07-06T03:54:05.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.770+0000 7f11f1d9c700 1 -- 172.21.15.97:0/731680631 learned_addr learned my addr 172.21.15.97:0/731680631 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:05.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.770+0000 7f11f0d9a700 1 --2- 172.21.15.97:0/731680631 >> v2:172.21.15.146:3300/0 conn(0x7f11ec15c4b0 0x7f11ec0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.771+0000 7f11e3fff700 1 -- 172.21.15.97:0/731680631 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661560566 0 0) 0x7f11ec0d08b0 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.771+0000 7f11e3fff700 1 -- 172.21.15.97:0/731680631 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11d80035b0 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.771+0000 7f11f0d9a700 1 -- 172.21.15.97:0/731680631 >> v1:172.21.15.146:6789/0 conn(0x7f11ec15c0e0 legacy=0x7f11ec0d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.771+0000 7f11f0d9a700 1 -- 172.21.15.97:0/731680631 >> v1:172.21.15.97:6789/0 conn(0x7f11ec0cd7d0 legacy=0x7f11ec0b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.771+0000 7f11f0d9a700 1 -- 172.21.15.97:0/731680631 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec0d0e30 con 0x7f11ec15c4b0 2023-07-06T03:54:05.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.771+0000 7f11f0d9a700 1 --2- 172.21.15.97:0/731680631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11ec15c4b0 0x7f11ec0cd290 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009870 tx=0x7f11dc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4df4b6cdc2495676 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:05.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.771+0000 7f11e3fff700 1 -- 172.21.15.97:0/731680631 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc011830 con 0x7f11ec15c4b0 2023-07-06T03:54:05.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.772+0000 7f11e3fff700 1 -- 172.21.15.97:0/731680631 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc011990 con 0x7f11ec15c4b0 2023-07-06T03:54:05.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.772+0000 7f11e3fff700 1 -- 172.21.15.97:0/731680631 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc011d20 con 0x7f11ec15c4b0 2023-07-06T03:54:05.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.772+0000 7f11f259d700 1 -- 172.21.15.97:0/731680631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11ec15c4b0 msgr2=0x7f11ec0cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.772+0000 7f11f259d700 1 --2- 172.21.15.97:0/731680631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11ec15c4b0 0x7f11ec0cd290 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009870 tx=0x7f11dc009b80 comp rx=0 tx=0).stop 2023-07-06T03:54:05.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.772+0000 7f11f259d700 1 -- 172.21.15.97:0/731680631 shutdown_connections 2023-07-06T03:54:05.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.772+0000 7f11f259d700 1 --2- 172.21.15.97:0/731680631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11ec15c4b0 0x7f11ec0cd290 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.772+0000 7f11f259d700 1 -- 172.21.15.97:0/731680631 >> 172.21.15.97:0/731680631 conn(0x7f11ec0cf6c0 msgr2=0x7f11ec0cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:05.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.772+0000 7f11f259d700 1 -- 172.21.15.97:0/731680631 shutdown_connections 2023-07-06T03:54:05.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.772+0000 7f11f259d700 1 -- 172.21.15.97:0/731680631 wait complete. 2023-07-06T03:54:05.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.774+0000 7f11f259d700 1 Processor -- start 2023-07-06T03:54:05.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.775+0000 7f11f259d700 1 -- start start 2023-07-06T03:54:05.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.776+0000 7f11f259d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11ec0cd7d0 0x7f11ec1f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.776+0000 7f11f259d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11ec15c0e0 0x7f11ec0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.776+0000 7f11f259d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11ec15c4b0 0x7f11ec0b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.776+0000 7f11f259d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec0b68d0 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.776+0000 7f11f259d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec1f23d0 con 0x7f11ec15c4b0 2023-07-06T03:54:05.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.776+0000 7f11f259d700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec14dbf0 con 0x7f11ec15c0e0 2023-07-06T03:54:05.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.776+0000 7f11f159b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11ec0cd7d0 0x7f11ec1f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.776+0000 7f11f159b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11ec0cd7d0 0x7f11ec1f1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55338/0 (socket says 172.21.15.97:55338) 2023-07-06T03:54:05.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.776+0000 7f11f159b700 1 -- 172.21.15.97:0/231393577 learned_addr learned my addr 172.21.15.97:0/231393577 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:05.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f0d9a700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11ec15c0e0 0x7f11ec0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f1d9c700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11ec15c4b0 0x7f11ec0b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f159b700 1 -- 172.21.15.97:0/231393577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11ec15c0e0 msgr2=0x7f11ec0b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f159b700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11ec15c0e0 0x7f11ec0b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f159b700 1 -- 172.21.15.97:0/231393577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11ec15c4b0 msgr2=0x7f11ec0b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:05.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f159b700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11ec15c4b0 0x7f11ec0b5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:05.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f159b700 1 -- 172.21.15.97:0/231393577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc0094a0 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f0d9a700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11ec15c0e0 0x7f11ec0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:05.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f1d9c700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11ec15c4b0 0x7f11ec0b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:05.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f159b700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11ec0cd7d0 0x7f11ec1f1e90 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f11e800c510 tx=0x7f11e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:05.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11e1ffb700 1 -- 172.21.15.97:0/231393577 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80052d0 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11e1ffb700 1 -- 172.21.15.97:0/231393577 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e8005430 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f259d700 1 -- 172.21.15.97:0/231393577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ec0c7020 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11e1ffb700 1 -- 172.21.15.97:0/231393577 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80038c0 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.777+0000 7f11f259d700 1 -- 172.21.15.97:0/231393577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ec0c75e0 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.778+0000 7f11f259d700 1 -- 172.21.15.97:0/231393577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ec0c63f0 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.779+0000 7f11e1ffb700 1 -- 172.21.15.97:0/231393577 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f11e8003a20 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.780+0000 7f11e1ffb700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f11d006ace0 0x7f11d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:05.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.780+0000 7f11e1ffb700 1 -- 172.21.15.97:0/231393577 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f11e8088a30 con 0x7f11ec0cd7d0 2023-07-06T03:54:05.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.780+0000 7f11f0d9a700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f11d006ace0 0x7f11d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:05.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.781+0000 7f11f0d9a700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f11d006ace0 0x7f11d006d190 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f11ec15c8a0 tx=0x7f11dc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:05.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:05.785+0000 7f11e1ffb700 1 -- 172.21.15.97:0/231393577 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f11e8053a50 con 0x7f11ec0cd7d0 2023-07-06T03:54:06.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.002+0000 7f11f259d700 1 -- 172.21.15.97:0/231393577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f11ec0ce4b0 con 0x7f11ec0cd7d0 2023-07-06T03:54:06.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.002+0000 7f11e1ffb700 1 -- 172.21.15.97:0/231393577 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) v1 ==== 80+0+0 (secure 0 0 0) 0x7f11e8057070 con 0x7f11ec0cd7d0 2023-07-06T03:54:06.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 -- 172.21.15.97:0/231393577 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f11d006ace0 msgr2=0x7f11d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:06.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f11d006ace0 0x7f11d006d190 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f11ec15c8a0 tx=0x7f11dc008030 comp rx=0 tx=0).stop 2023-07-06T03:54:06.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 -- 172.21.15.97:0/231393577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11ec0cd7d0 msgr2=0x7f11ec1f1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:06.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11ec0cd7d0 0x7f11ec1f1e90 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f11e800c510 tx=0x7f11e800c8d0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 -- 172.21.15.97:0/231393577 shutdown_connections 2023-07-06T03:54:06.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f11d006ace0 0x7f11d006d190 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11ec15c4b0 0x7f11ec0b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11ec0cd7d0 0x7f11ec1f1e90 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 --2- 172.21.15.97:0/231393577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f11ec15c0e0 0x7f11ec0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 -- 172.21.15.97:0/231393577 >> 172.21.15.97:0/231393577 conn(0x7f11ec0cf6c0 msgr2=0x7f11ec15acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:06.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 -- 172.21.15.97:0/231393577 shutdown_connections 2023-07-06T03:54:06.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.005+0000 7f11f259d700 1 -- 172.21.15.97:0/231393577 wait complete. 2023-07-06T03:54:06.009 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:54:06.026 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2023-07-06T03:54:06.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:06.027 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-06T03:54:06.100 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[125117]: fuse finished with error 0 and tester_r 0 2023-07-06T03:54:06.103 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:54:06.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:06.104 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T03:54:06.266 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:54:06.267 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:06.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:54:06.312 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:54:06.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:06.313 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T03:54:06.340 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:54:06.359 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:54:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T03:54:06.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:54:06.401 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:54:06.401 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:06.451 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:54:06.451 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:54:06.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:06.452 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T03:54:06.535 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T03:54:06.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:54:06.536 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:06.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.728+0000 7fb06d5ff700 1 Processor -- start 2023-07-06T03:54:06.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.728+0000 7fb06d5ff700 1 -- start start 2023-07-06T03:54:06.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.729+0000 7fb06d5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06815c520 0x7fb0680cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:06.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.729+0000 7fb06d5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb068160180 0x7fb0681605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:06.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.729+0000 7fb06d5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0680cdae0 con 0x7fb068160b30 2023-07-06T03:54:06.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.729+0000 7fb06d5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0680cd8d0 con 0x7fb068160180 2023-07-06T03:54:06.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.729+0000 7fb06d5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680b6640 con 0x7fb06815c520 2023-07-06T03:54:06.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.730+0000 7fb0667fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb068160180 0x7fb0681605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:06.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.730+0000 7fb0677fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb068160b30 0x7fb0680cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56746/0 (socket says 172.21.15.97:56746) 2023-07-06T03:54:06.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.730+0000 7fb0677fe700 1 -- 172.21.15.97:0/1980617419 learned_addr learned my addr 172.21.15.97:0/1980617419 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:06.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.730+0000 7fb0667fc700 1 --2- 172.21.15.97:0/1980617419 >> v2:172.21.15.97:3300/0 conn(0x7fb068160180 0x7fb0681605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55342/0 (socket says 172.21.15.97:55342) 2023-07-06T03:54:06.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.730+0000 7fb066ffd700 1 --2- 172.21.15.97:0/1980617419 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06815c520 0x7fb0680cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:06.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.730+0000 7fb0667fc700 1 -- 172.21.15.97:0/1980617419 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06815c520 msgr2=0x7fb0680cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:06.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.730+0000 7fb0667fc700 1 --2- 172.21.15.97:0/1980617419 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06815c520 0x7fb0680cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.730+0000 7fb0667fc700 1 -- 172.21.15.97:0/1980617419 >> v1:172.21.15.97:6789/0 conn(0x7fb068160b30 legacy=0x7fb0680cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:06.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.730+0000 7fb065ffb700 1 -- 172.21.15.97:0/1980617419 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893254711 0 0) 0x7fb0680cdae0 con 0x7fb068160b30 2023-07-06T03:54:06.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.730+0000 7fb0667fc700 1 -- 172.21.15.97:0/1980617419 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0680b6920 con 0x7fb068160180 2023-07-06T03:54:06.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.730+0000 7fb0667fc700 1 --2- 172.21.15.97:0/1980617419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb068160180 0x7fb0681605f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009840 tx=0x7fb05c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788b15d0045f8204 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:06.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.731+0000 7fb065ffb700 1 -- 172.21.15.97:0/1980617419 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c011830 con 0x7fb068160180 2023-07-06T03:54:06.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.731+0000 7fb065ffb700 1 -- 172.21.15.97:0/1980617419 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c011990 con 0x7fb068160180 2023-07-06T03:54:06.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.731+0000 7fb065ffb700 1 -- 172.21.15.97:0/1980617419 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c011d20 con 0x7fb068160180 2023-07-06T03:54:06.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.731+0000 7fb06d5ff700 1 -- 172.21.15.97:0/1980617419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb068160180 msgr2=0x7fb0681605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:06.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.731+0000 7fb06d5ff700 1 --2- 172.21.15.97:0/1980617419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb068160180 0x7fb0681605f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009840 tx=0x7fb05c009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:06.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.731+0000 7fb06d5ff700 1 -- 172.21.15.97:0/1980617419 shutdown_connections 2023-07-06T03:54:06.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.731+0000 7fb06d5ff700 1 --2- 172.21.15.97:0/1980617419 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06815c520 0x7fb0680cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.731+0000 7fb06d5ff700 1 --2- 172.21.15.97:0/1980617419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb068160180 0x7fb0681605f0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.731+0000 7fb06d5ff700 1 -- 172.21.15.97:0/1980617419 >> 172.21.15.97:0/1980617419 conn(0x7fb068071790 msgr2=0x7fb068071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.731+0000 7fb06d5ff700 1 -- 172.21.15.97:0/1980617419 shutdown_connections 2023-07-06T03:54:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.731+0000 7fb06d5ff700 1 -- 172.21.15.97:0/1980617419 wait complete. 2023-07-06T03:54:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.733+0000 7fb06d5ff700 1 Processor -- start 2023-07-06T03:54:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.733+0000 7fb06d5ff700 1 -- start start 2023-07-06T03:54:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.734+0000 7fb06d5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06815c520 0x7fb0681f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:06.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.734+0000 7fb06d5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb068160180 0x7fb0680d2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:06.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.734+0000 7fb06d5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb068160b30 0x7fb068156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:06.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.734+0000 7fb06d5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680d3c80 con 0x7fb068160180 2023-07-06T03:54:06.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.734+0000 7fb06d5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0681430f0 con 0x7fb068160b30 2023-07-06T03:54:06.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.734+0000 7fb06d5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680d33e0 con 0x7fb06815c520 2023-07-06T03:54:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.734+0000 7fb0667fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb068160180 0x7fb0680d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.734+0000 7fb0667fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb068160180 0x7fb0680d2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55356/0 (socket says 172.21.15.97:55356) 2023-07-06T03:54:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.734+0000 7fb0667fc700 1 -- 172.21.15.97:0/3210398277 learned_addr learned my addr 172.21.15.97:0/3210398277 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb066ffd700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06815c520 0x7fb0681f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb0677fe700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb068160b30 0x7fb068156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb0667fc700 1 -- 172.21.15.97:0/3210398277 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06815c520 msgr2=0x7fb0681f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb0667fc700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06815c520 0x7fb0681f27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb0667fc700 1 -- 172.21.15.97:0/3210398277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb068160b30 msgr2=0x7fb068156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb0667fc700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb068160b30 0x7fb068156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb0667fc700 1 -- 172.21.15.97:0/3210398277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c0094a0 con 0x7fb068160180 2023-07-06T03:54:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb066ffd700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06815c520 0x7fb0681f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb0667fc700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb068160180 0x7fb0680d2900 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009fd0 tx=0x7fb05c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb04f7fe700 1 -- 172.21.15.97:0/3210398277 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0036a0 con 0x7fb068160180 2023-07-06T03:54:06.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb04f7fe700 1 -- 172.21.15.97:0/3210398277 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c01e070 con 0x7fb068160180 2023-07-06T03:54:06.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb06d5ff700 1 -- 172.21.15.97:0/3210398277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0680d36c0 con 0x7fb068160180 2023-07-06T03:54:06.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb04f7fe700 1 -- 172.21.15.97:0/3210398277 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c003a60 con 0x7fb068160180 2023-07-06T03:54:06.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.735+0000 7fb06d5ff700 1 -- 172.21.15.97:0/3210398277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0681f2d00 con 0x7fb068160180 2023-07-06T03:54:06.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.736+0000 7fb06d5ff700 1 -- 172.21.15.97:0/3210398277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0681432b0 con 0x7fb068160180 2023-07-06T03:54:06.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.737+0000 7fb04f7fe700 1 -- 172.21.15.97:0/3210398277 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb05c003bc0 con 0x7fb068160180 2023-07-06T03:54:06.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.738+0000 7fb04f7fe700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb05406ace0 0x7fb05406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:06.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.738+0000 7fb04f7fe700 1 -- 172.21.15.97:0/3210398277 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fb05c08a580 con 0x7fb068160180 2023-07-06T03:54:06.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.739+0000 7fb066ffd700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb05406ace0 0x7fb05406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:06.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.739+0000 7fb066ffd700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb05406ace0 0x7fb05406d190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb058009b90 tx=0x7fb058008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.743+0000 7fb04f7fe700 1 -- 172.21.15.97:0/3210398277 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb05c0555a0 con 0x7fb068160180 2023-07-06T03:54:06.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.987+0000 7fb06d5ff700 1 -- 172.21.15.97:0/3210398277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb06804e340 con 0x7fb068160180 2023-07-06T03:54:06.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.987+0000 7fb04f7fe700 1 -- 172.21.15.97:0/3210398277 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fb05c058bc0 con 0x7fb068160180 2023-07-06T03:54:06.990 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:06.991 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/387292825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":387292825},{"type":"v1","addr":"172.21.15.146:6833","nonce":387292825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690380561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690380561},{"type":"v1","addr":"172.21.15.97:6837","nonce":690380561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3897834271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3897834271},{"type":"v1","addr":"172.21.15.146:6835","nonce":3897834271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:53:26.234190+0000","modified":"2023-07-06T03:54:06.823422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6836/3324055785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3324055785},{"type":"v1","addr":"172.21.15.97:6836","nonce":3324055785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-06T03:54:06.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 -- 172.21.15.97:0/3210398277 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb05406ace0 msgr2=0x7fb05406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb05406ace0 0x7fb05406d190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb058009b90 tx=0x7fb058008040 comp rx=0 tx=0).stop 2023-07-06T03:54:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 -- 172.21.15.97:0/3210398277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb068160180 msgr2=0x7fb0680d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb068160180 0x7fb0680d2900 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009fd0 tx=0x7fb05c004750 comp rx=0 tx=0).stop 2023-07-06T03:54:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 -- 172.21.15.97:0/3210398277 shutdown_connections 2023-07-06T03:54:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb05406ace0 0x7fb05406d190 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb068160b30 0x7fb068156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb06815c520 0x7fb0681f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 --2- 172.21.15.97:0/3210398277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb068160180 0x7fb0680d2900 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:06.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 -- 172.21.15.97:0/3210398277 >> 172.21.15.97:0/3210398277 conn(0x7fb068071790 msgr2=0x7fb0680b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:06.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 -- 172.21.15.97:0/3210398277 shutdown_connections 2023-07-06T03:54:06.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:06.990+0000 7fb06d5ff700 1 -- 172.21.15.97:0/3210398277 wait complete. 2023-07-06T03:54:06.994 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 71 2023-07-06T03:54:07.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:07.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.202+0000 7f84ed3ff700 1 Processor -- start 2023-07-06T03:54:07.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.202+0000 7f84ed3ff700 1 -- start start 2023-07-06T03:54:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.203+0000 7f84ed3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e80cd8e0 0x7f84e815e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.203+0000 7f84ed3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f84e815b440 0x7f84e80adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.203+0000 7f84ed3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e80710a0 con 0x7f84e8070c30 2023-07-06T03:54:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.203+0000 7f84ed3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e80ae3e0 con 0x7f84e815b440 2023-07-06T03:54:07.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.203+0000 7f84ed3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e8071330 con 0x7f84e80cd8e0 2023-07-06T03:54:07.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.204+0000 7f84e77fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f84e815b440 0x7f84e80adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.204+0000 7f84e6ffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e80cd8e0 0x7f84e815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.204+0000 7f84e77fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f84e815b440 0x7f84e80adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:37084/0 (socket says 172.21.15.97:37084) 2023-07-06T03:54:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.204+0000 7f84e77fe700 1 -- 172.21.15.97:0/1812091096 learned_addr learned my addr 172.21.15.97:0/1812091096 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.204+0000 7f84e6ffd700 1 -- 172.21.15.97:0/1812091096 >> v2:172.21.15.146:3300/0 conn(0x7f84e815b440 msgr2=0x7f84e80adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.204+0000 7f84e6ffd700 1 --2- 172.21.15.97:0/1812091096 >> v2:172.21.15.146:3300/0 conn(0x7f84e815b440 0x7f84e80adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.204+0000 7f84e6ffd700 1 -- 172.21.15.97:0/1812091096 >> v1:172.21.15.146:6789/0 conn(0x7f84e8070c30 legacy=0x7f84e80cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.204+0000 7f84e6ffd700 1 -- 172.21.15.97:0/1812091096 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e8071610 con 0x7f84e80cd8e0 2023-07-06T03:54:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.204+0000 7f84e77fe700 1 --2- 172.21.15.97:0/1812091096 >> v2:172.21.15.146:3300/0 conn(0x7f84e815b440 0x7f84e80adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:54:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.205+0000 7f84e6ffd700 1 --2- 172.21.15.97:0/1812091096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e80cd8e0 0x7f84e815e860 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009840 tx=0x7f84d8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c426a3ec7a8350 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.205+0000 7f84e5ffb700 1 -- 172.21.15.97:0/1812091096 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8011830 con 0x7f84e80cd8e0 2023-07-06T03:54:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.205+0000 7f84e5ffb700 1 -- 172.21.15.97:0/1812091096 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d8011990 con 0x7f84e80cd8e0 2023-07-06T03:54:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.205+0000 7f84e5ffb700 1 -- 172.21.15.97:0/1812091096 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8011d20 con 0x7f84e80cd8e0 2023-07-06T03:54:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.205+0000 7f84ed3ff700 1 -- 172.21.15.97:0/1812091096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e80cd8e0 msgr2=0x7f84e815e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.205+0000 7f84ed3ff700 1 --2- 172.21.15.97:0/1812091096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e80cd8e0 0x7f84e815e860 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009840 tx=0x7f84d8009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.205+0000 7f84ed3ff700 1 -- 172.21.15.97:0/1812091096 shutdown_connections 2023-07-06T03:54:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.205+0000 7f84ed3ff700 1 --2- 172.21.15.97:0/1812091096 >> v2:172.21.15.146:3300/0 conn(0x7f84e815b440 0x7f84e80adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.205+0000 7f84ed3ff700 1 --2- 172.21.15.97:0/1812091096 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e80cd8e0 0x7f84e815e860 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.205+0000 7f84ed3ff700 1 -- 172.21.15.97:0/1812091096 >> 172.21.15.97:0/1812091096 conn(0x7f84e80ce8c0 msgr2=0x7f84e80cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:07.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.206+0000 7f84ed3ff700 1 -- 172.21.15.97:0/1812091096 shutdown_connections 2023-07-06T03:54:07.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.206+0000 7f84ed3ff700 1 -- 172.21.15.97:0/1812091096 wait complete. 2023-07-06T03:54:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.207+0000 7f84ed3ff700 1 Processor -- start 2023-07-06T03:54:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.207+0000 7f84ed3ff700 1 -- start start 2023-07-06T03:54:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.208+0000 7f84ed3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e8070c30 0x7f84e8145230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.208+0000 7f84ed3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e80cd8e0 0x7f84e806c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.208+0000 7f84ed3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e815b440 0x7f84e806c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.208+0000 7f84ed3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e8150090 con 0x7f84e815b440 2023-07-06T03:54:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.208+0000 7f84ed3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e814fc10 con 0x7f84e80cd8e0 2023-07-06T03:54:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.208+0000 7f84ed3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e806cf90 con 0x7f84e8070c30 2023-07-06T03:54:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e815b440 0x7f84e806c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e67fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e80cd8e0 0x7f84e806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e6ffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e8070c30 0x7f84e8145230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e67fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e80cd8e0 0x7f84e806c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:37092/0 (socket says 172.21.15.97:37092) 2023-07-06T03:54:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e815b440 0x7f84e806c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55360/0 (socket says 172.21.15.97:55360) 2023-07-06T03:54:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e67fc700 1 -- 172.21.15.97:0/371546725 learned_addr learned my addr 172.21.15.97:0/371546725 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e67fc700 1 -- 172.21.15.97:0/371546725 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e8070c30 msgr2=0x7f84e8145230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e67fc700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e8070c30 0x7f84e8145230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e67fc700 1 -- 172.21.15.97:0/371546725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e815b440 msgr2=0x7f84e806c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e67fc700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e815b440 0x7f84e806c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e67fc700 1 -- 172.21.15.97:0/371546725 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e0009b30 con 0x7f84e80cd8e0 2023-07-06T03:54:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.209+0000 7f84e67fc700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e80cd8e0 0x7f84e806c420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00c450 tx=0x7f84dc00c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.210+0000 7f84cf7fe700 1 -- 172.21.15.97:0/371546725 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0052d0 con 0x7f84e80cd8e0 2023-07-06T03:54:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.210+0000 7f84cf7fe700 1 -- 172.21.15.97:0/371546725 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc005430 con 0x7f84e80cd8e0 2023-07-06T03:54:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.210+0000 7f84ed3ff700 1 -- 172.21.15.97:0/371546725 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d80094a0 con 0x7f84e80cd8e0 2023-07-06T03:54:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.210+0000 7f84cf7fe700 1 -- 172.21.15.97:0/371546725 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0038c0 con 0x7f84e80cd8e0 2023-07-06T03:54:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.210+0000 7f84ed3ff700 1 -- 172.21.15.97:0/371546725 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e806d550 con 0x7f84e80cd8e0 2023-07-06T03:54:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.211+0000 7f84ed3ff700 1 -- 172.21.15.97:0/371546725 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e8070aa0 con 0x7f84e80cd8e0 2023-07-06T03:54:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.212+0000 7f84cf7fe700 1 -- 172.21.15.97:0/371546725 <== mon.1 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f84dc003a20 con 0x7f84e80cd8e0 2023-07-06T03:54:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.213+0000 7f84cf7fe700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84d406ace0 0x7f84d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.213+0000 7f84cf7fe700 1 -- 172.21.15.97:0/371546725 <== mon.1 v2:172.21.15.146:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f84dc088890 con 0x7f84e80cd8e0 2023-07-06T03:54:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.213+0000 7f84e6ffd700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84d406ace0 0x7f84d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.214+0000 7f84e6ffd700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84d406ace0 0x7f84d406d190 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f84d8000c00 tx=0x7f84d8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:07.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.218+0000 7f84cf7fe700 1 -- 172.21.15.97:0/371546725 <== mon.1 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f84dc053830 con 0x7f84e80cd8e0 2023-07-06T03:54:07.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.415+0000 7f84ed3ff700 1 -- 172.21.15.97:0/371546725 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84e8067830 con 0x7f84e80cd8e0 2023-07-06T03:54:07.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.417+0000 7f84cf7fe700 1 -- 172.21.15.97:0/371546725 <== mon.1 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+18516 (secure 0 0 0) 0x7f84dc00ce60 con 0x7f84e80cd8e0 2023-07-06T03:54:07.420 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:07.420 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":101,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:53:54.754060+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:53:24.013743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-06T03:53:25.222178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":6}]},{"pgid":"16.9","mappings":[{"from":4,"to":3}]},{"pgid":"16.15","mappings":[{"from":4,"to":3}]},{"pgid":"16.24","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:07.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.419+0000 7f84ed3ff700 1 -- 172.21.15.97:0/371546725 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84d406ace0 msgr2=0x7f84d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.419+0000 7f84ed3ff700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84d406ace0 0x7f84d406d190 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f84d8000c00 tx=0x7f84d8008030 comp rx=0 tx=0).stop 2023-07-06T03:54:07.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.419+0000 7f84ed3ff700 1 -- 172.21.15.97:0/371546725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e80cd8e0 msgr2=0x7f84e806c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.419+0000 7f84ed3ff700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e80cd8e0 0x7f84e806c420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00c450 tx=0x7f84dc00c760 comp rx=0 tx=0).stop 2023-07-06T03:54:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.419+0000 7f84ed3ff700 1 -- 172.21.15.97:0/371546725 shutdown_connections 2023-07-06T03:54:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.419+0000 7f84ed3ff700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f84d406ace0 0x7f84d406d190 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.419+0000 7f84ed3ff700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e815b440 0x7f84e806c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.419+0000 7f84ed3ff700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f84e8070c30 0x7f84e8145230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.419+0000 7f84ed3ff700 1 --2- 172.21.15.97:0/371546725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e80cd8e0 0x7f84e806c420 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.419+0000 7f84ed3ff700 1 -- 172.21.15.97:0/371546725 >> 172.21.15.97:0/371546725 conn(0x7f84e80ce8c0 msgr2=0x7f84e80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.420+0000 7f84ed3ff700 1 -- 172.21.15.97:0/371546725 shutdown_connections 2023-07-06T03:54:07.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.420+0000 7f84ed3ff700 1 -- 172.21.15.97:0/371546725 wait complete. 2023-07-06T03:54:07.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:53:24.013743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:53:25.222178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:54:07.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:54:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.635+0000 7f3c081ff700 1 Processor -- start 2023-07-06T03:54:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.635+0000 7f3c081ff700 1 -- start start 2023-07-06T03:54:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.636+0000 7f3c081ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c0015c610 0x7f3c000b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.636+0000 7f3c081ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c000d2210 0x7f3c000d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.636+0000 7f3c081ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c000cff90 con 0x7f3c0006b360 2023-07-06T03:54:07.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.636+0000 7f3c081ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c000d0780 con 0x7f3c0015c610 2023-07-06T03:54:07.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.636+0000 7f3c081ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0015c9e0 con 0x7f3c000d2210 2023-07-06T03:54:07.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.637+0000 7f3c05f9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c0015c610 0x7f3c000b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.637+0000 7f3c0679c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c000d2210 0x7f3c000d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.637+0000 7f3c05f9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3c0015c610 0x7f3c000b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:37104/0 (socket says 172.21.15.97:37104) 2023-07-06T03:54:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.637+0000 7f3c05f9b700 1 -- 172.21.15.97:0/1757253846 learned_addr learned my addr 172.21.15.97:0/1757253846 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.637+0000 7f3c05f9b700 1 -- 172.21.15.97:0/1757253846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c000d2210 msgr2=0x7f3c000d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.637+0000 7f3c05f9b700 1 --2- 172.21.15.97:0/1757253846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c000d2210 0x7f3c000d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.637+0000 7f3c05f9b700 1 -- 172.21.15.97:0/1757253846 >> v1:172.21.15.146:6789/0 conn(0x7f3c0006b360 legacy=0x7f3c000b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.637+0000 7f3c05f9b700 1 -- 172.21.15.97:0/1757253846 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c000b5650 con 0x7f3c0015c610 2023-07-06T03:54:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.637+0000 7f3c0679c700 1 --2- 172.21.15.97:0/1757253846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c000d2210 0x7f3c000d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:54:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.637+0000 7f3c05f9b700 1 --2- 172.21.15.97:0/1757253846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c0015c610 0x7f3c000b47b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0098f0 tx=0x7f3bfc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dea78ff21b425e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.638+0000 7f3c04f99700 1 -- 172.21.15.97:0/1757253846 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc011830 con 0x7f3c0015c610 2023-07-06T03:54:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.638+0000 7f3c04f99700 1 -- 172.21.15.97:0/1757253846 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bfc011990 con 0x7f3c0015c610 2023-07-06T03:54:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.638+0000 7f3c04f99700 1 -- 172.21.15.97:0/1757253846 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc011d20 con 0x7f3c0015c610 2023-07-06T03:54:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.638+0000 7f3c081ff700 1 -- 172.21.15.97:0/1757253846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c0015c610 msgr2=0x7f3c000b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.638+0000 7f3c081ff700 1 --2- 172.21.15.97:0/1757253846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c0015c610 0x7f3c000b47b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0098f0 tx=0x7f3bfc009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:07.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.638+0000 7f3c081ff700 1 -- 172.21.15.97:0/1757253846 shutdown_connections 2023-07-06T03:54:07.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.638+0000 7f3c081ff700 1 --2- 172.21.15.97:0/1757253846 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c000d2210 0x7f3c000d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.638+0000 7f3c081ff700 1 --2- 172.21.15.97:0/1757253846 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c0015c610 0x7f3c000b47b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.638+0000 7f3c081ff700 1 -- 172.21.15.97:0/1757253846 >> 172.21.15.97:0/1757253846 conn(0x7f3c0006a6c0 msgr2=0x7f3c0006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:07.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.638+0000 7f3c081ff700 1 -- 172.21.15.97:0/1757253846 shutdown_connections 2023-07-06T03:54:07.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.638+0000 7f3c081ff700 1 -- 172.21.15.97:0/1757253846 wait complete. 2023-07-06T03:54:07.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.640+0000 7f3c081ff700 1 Processor -- start 2023-07-06T03:54:07.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.640+0000 7f3c081ff700 1 -- start start 2023-07-06T03:54:07.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.641+0000 7f3c081ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c0006b360 0x7f3c001f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.641+0000 7f3c081ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c000d2210 0x7f3c000c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.641+0000 7f3c081ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c0015c610 0x7f3c0015f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.641+0000 7f3c081ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c000cbc50 con 0x7f3c0006b360 2023-07-06T03:54:07.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.641+0000 7f3c081ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c000cafd0 con 0x7f3c000d2210 2023-07-06T03:54:07.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.641+0000 7f3c081ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c000cb2b0 con 0x7f3c0015c610 2023-07-06T03:54:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.641+0000 7f3c05f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c0006b360 0x7f3c001f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.641+0000 7f3c05f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c0006b360 0x7f3c001f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55372/0 (socket says 172.21.15.97:55372) 2023-07-06T03:54:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.641+0000 7f3c05f9b700 1 -- 172.21.15.97:0/324724165 learned_addr learned my addr 172.21.15.97:0/324724165 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3c0579a700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c000d2210 0x7f3c000c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3c0679c700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c0015c610 0x7f3c0015f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3c05f9b700 1 -- 172.21.15.97:0/324724165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c0015c610 msgr2=0x7f3c0015f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3c05f9b700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c0015c610 0x7f3c0015f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3c05f9b700 1 -- 172.21.15.97:0/324724165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c000d2210 msgr2=0x7f3c000c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3c05f9b700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c000d2210 0x7f3c000c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3c05f9b700 1 -- 172.21.15.97:0/324724165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf8009a70 con 0x7f3c0006b360 2023-07-06T03:54:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3c05f9b700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c0006b360 0x7f3c001f1440 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009c40 tx=0x7f3bfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:07.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3bf67fc700 1 -- 172.21.15.97:0/324724165 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc0036a0 con 0x7f3c0006b360 2023-07-06T03:54:07.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3bf67fc700 1 -- 172.21.15.97:0/324724165 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bfc01e070 con 0x7f3c0006b360 2023-07-06T03:54:07.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3bf67fc700 1 -- 172.21.15.97:0/324724165 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc003a60 con 0x7f3c0006b360 2023-07-06T03:54:07.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3c081ff700 1 -- 172.21.15.97:0/324724165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bfc009550 con 0x7f3c0006b360 2023-07-06T03:54:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.642+0000 7f3c081ff700 1 -- 172.21.15.97:0/324724165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c000cb870 con 0x7f3c0006b360 2023-07-06T03:54:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.643+0000 7f3c081ff700 1 -- 172.21.15.97:0/324724165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0004e340 con 0x7f3c0006b360 2023-07-06T03:54:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.644+0000 7f3bf67fc700 1 -- 172.21.15.97:0/324724165 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3bfc003bc0 con 0x7f3c0006b360 2023-07-06T03:54:07.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.645+0000 7f3bf67fc700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3be406ace0 0x7f3be406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:07.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.645+0000 7f3bf67fc700 1 -- 172.21.15.97:0/324724165 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f3bfc08a450 con 0x7f3c0006b360 2023-07-06T03:54:07.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.646+0000 7f3c0579a700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3be406ace0 0x7f3be406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:07.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.646+0000 7f3c0579a700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3be406ace0 0x7f3be406d190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3c00153d60 tx=0x7f3bf0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:07.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.650+0000 7f3bf67fc700 1 -- 172.21.15.97:0/324724165 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3bfc055510 con 0x7f3c0006b360 2023-07-06T03:54:07.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.853+0000 7f3c081ff700 1 -- 172.21.15.97:0/324724165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c000d16e0 con 0x7f3c0006b360 2023-07-06T03:54:07.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.854+0000 7f3bf67fc700 1 -- 172.21.15.97:0/324724165 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3bfc058b30 con 0x7f3c0006b360 2023-07-06T03:54:07.857 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:54:07.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 -- 172.21.15.97:0/324724165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3be406ace0 msgr2=0x7f3be406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3be406ace0 0x7f3be406d190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3c00153d60 tx=0x7f3bf0009040 comp rx=0 tx=0).stop 2023-07-06T03:54:07.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 -- 172.21.15.97:0/324724165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c0006b360 msgr2=0x7f3c001f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:07.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c0006b360 0x7f3c001f1440 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009c40 tx=0x7f3bfc004750 comp rx=0 tx=0).stop 2023-07-06T03:54:07.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 -- 172.21.15.97:0/324724165 shutdown_connections 2023-07-06T03:54:07.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3be406ace0 0x7f3be406d190 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3c0015c610 0x7f3c0015f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c0006b360 0x7f3c001f1440 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 --2- 172.21.15.97:0/324724165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3c000d2210 0x7f3c000c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:07.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 -- 172.21.15.97:0/324724165 >> 172.21.15.97:0/324724165 conn(0x7f3c0006a6c0 msgr2=0x7f3c000d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:07.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 -- 172.21.15.97:0/324724165 shutdown_connections 2023-07-06T03:54:07.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:07.856+0000 7f3c081ff700 1 -- 172.21.15.97:0/324724165 wait complete. 2023-07-06T03:54:07.877 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:54:08.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.071+0000 7f4979bff700 1 Processor -- start 2023-07-06T03:54:08.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.071+0000 7f4979bff700 1 -- start start 2023-07-06T03:54:08.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.072+0000 7f4979bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f497415c610 0x7f49740b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.072+0000 7f4979bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49740d2210 0x7f49740d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.072+0000 7f4979bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49740cff90 con 0x7f497406b360 2023-07-06T03:54:08.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.072+0000 7f4979bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49740d0780 con 0x7f49740d2210 2023-07-06T03:54:08.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.072+0000 7f4979bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497415c9e0 con 0x7f497415c610 2023-07-06T03:54:08.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.072+0000 7f4973fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49740d2210 0x7f49740d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.072+0000 7f4973fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49740d2210 0x7f49740d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55388/0 (socket says 172.21.15.97:55388) 2023-07-06T03:54:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.072+0000 7f4972ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f497406b360 0x7f49740b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56750/0 (socket says 172.21.15.97:56750) 2023-07-06T03:54:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.072+0000 7f4973fff700 1 -- 172.21.15.97:0/3471254685 learned_addr learned my addr 172.21.15.97:0/3471254685 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.073+0000 7f49737fe700 1 --2- 172.21.15.97:0/3471254685 >> v2:172.21.15.146:3300/0 conn(0x7f497415c610 0x7f49740b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.073+0000 7f4973fff700 1 -- 172.21.15.97:0/3471254685 >> v2:172.21.15.146:3300/0 conn(0x7f497415c610 msgr2=0x7f49740b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.073+0000 7f4973fff700 1 --2- 172.21.15.97:0/3471254685 >> v2:172.21.15.146:3300/0 conn(0x7f497415c610 0x7f49740b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.073+0000 7f4973fff700 1 -- 172.21.15.97:0/3471254685 >> v1:172.21.15.97:6789/0 conn(0x7f497406b360 legacy=0x7f49740b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.073+0000 7f4973fff700 1 -- 172.21.15.97:0/3471254685 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49740b5650 con 0x7f49740d2210 2023-07-06T03:54:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.073+0000 7f4973fff700 1 --2- 172.21.15.97:0/3471254685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49740d2210 0x7f49740d1310 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f497408b000 tx=0x7f496c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241af7ab3c88c0c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.073+0000 7f49727fc700 1 -- 172.21.15.97:0/3471254685 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c00e070 con 0x7f49740d2210 2023-07-06T03:54:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.073+0000 7f49727fc700 1 -- 172.21.15.97:0/3471254685 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c00b980 con 0x7f49740d2210 2023-07-06T03:54:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.073+0000 7f49727fc700 1 -- 172.21.15.97:0/3471254685 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c00bd30 con 0x7f49740d2210 2023-07-06T03:54:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.074+0000 7f4979bff700 1 -- 172.21.15.97:0/3471254685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49740d2210 msgr2=0x7f49740d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.074+0000 7f4979bff700 1 --2- 172.21.15.97:0/3471254685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49740d2210 0x7f49740d1310 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f497408b000 tx=0x7f496c00b330 comp rx=0 tx=0).stop 2023-07-06T03:54:08.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.074+0000 7f4979bff700 1 -- 172.21.15.97:0/3471254685 shutdown_connections 2023-07-06T03:54:08.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.074+0000 7f4979bff700 1 --2- 172.21.15.97:0/3471254685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49740d2210 0x7f49740d1310 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.074+0000 7f4979bff700 1 --2- 172.21.15.97:0/3471254685 >> v2:172.21.15.146:3300/0 conn(0x7f497415c610 0x7f49740b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.074+0000 7f4979bff700 1 -- 172.21.15.97:0/3471254685 >> 172.21.15.97:0/3471254685 conn(0x7f497406a6c0 msgr2=0x7f497406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:08.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.074+0000 7f4979bff700 1 -- 172.21.15.97:0/3471254685 shutdown_connections 2023-07-06T03:54:08.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.074+0000 7f4979bff700 1 -- 172.21.15.97:0/3471254685 wait complete. 2023-07-06T03:54:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.076+0000 7f4979bff700 1 Processor -- start 2023-07-06T03:54:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.076+0000 7f4979bff700 1 -- start start 2023-07-06T03:54:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4979bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f497406b360 0x7f497414b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4979bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49740d2210 0x7f49740c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4979bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497415c610 0x7f49740b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4979bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49740c9510 con 0x7f497415c610 2023-07-06T03:54:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4979bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49740c8890 con 0x7f49740d2210 2023-07-06T03:54:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4979bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49740c8b70 con 0x7f497406b360 2023-07-06T03:54:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4973fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497415c610 0x7f49740b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4973fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497415c610 0x7f49740b6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55394/0 (socket says 172.21.15.97:55394) 2023-07-06T03:54:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4973fff700 1 -- 172.21.15.97:0/3560575755 learned_addr learned my addr 172.21.15.97:0/3560575755 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f49737fe700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f497406b360 0x7f497414b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4972ffd700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49740d2210 0x7f49740c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4973fff700 1 -- 172.21.15.97:0/3560575755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f497406b360 msgr2=0x7f497414b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4973fff700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f497406b360 0x7f497414b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4973fff700 1 -- 172.21.15.97:0/3560575755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49740d2210 msgr2=0x7f49740c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4973fff700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49740d2210 0x7f49740c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.077+0000 7f4973fff700 1 -- 172.21.15.97:0/3560575755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c009b30 con 0x7f497415c610 2023-07-06T03:54:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.078+0000 7f4973fff700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497415c610 0x7f49740b6570 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f497408b120 tx=0x7f496c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.078+0000 7f495bfff700 1 -- 172.21.15.97:0/3560575755 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c0092e0 con 0x7f497415c610 2023-07-06T03:54:08.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.078+0000 7f495bfff700 1 -- 172.21.15.97:0/3560575755 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c004960 con 0x7f497415c610 2023-07-06T03:54:08.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.078+0000 7f4979bff700 1 -- 172.21.15.97:0/3560575755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49740c8e50 con 0x7f497415c610 2023-07-06T03:54:08.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.078+0000 7f495bfff700 1 -- 172.21.15.97:0/3560575755 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c01c680 con 0x7f497415c610 2023-07-06T03:54:08.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.078+0000 7f4979bff700 1 -- 172.21.15.97:0/3560575755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49740c9930 con 0x7f497415c610 2023-07-06T03:54:08.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.079+0000 7f4979bff700 1 -- 172.21.15.97:0/3560575755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497404e340 con 0x7f497415c610 2023-07-06T03:54:08.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.080+0000 7f495bfff700 1 -- 172.21.15.97:0/3560575755 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f496c01c7e0 con 0x7f497415c610 2023-07-06T03:54:08.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.081+0000 7f495bfff700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f496006ace0 0x7f496006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.081+0000 7f495bfff700 1 -- 172.21.15.97:0/3560575755 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f496c08b170 con 0x7f497415c610 2023-07-06T03:54:08.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.081+0000 7f49737fe700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f496006ace0 0x7f496006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.082+0000 7f49737fe700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f496006ace0 0x7f496006d190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f49741581e0 tx=0x7f4964008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:08.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.085+0000 7f495bfff700 1 -- 172.21.15.97:0/3560575755 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f496c056190 con 0x7f497415c610 2023-07-06T03:54:08.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.287+0000 7f4979bff700 1 -- 172.21.15.97:0/3560575755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4974159a40 con 0x7f497415c610 2023-07-06T03:54:08.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.287+0000 7f495bfff700 1 -- 172.21.15.97:0/3560575755 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v101) v1 ==== 92+0+10 (secure 0 0 0) 0x7f496c0597b0 con 0x7f497415c610 2023-07-06T03:54:08.290 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:54:08.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 -- 172.21.15.97:0/3560575755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f496006ace0 msgr2=0x7f496006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f496006ace0 0x7f496006d190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f49741581e0 tx=0x7f4964008040 comp rx=0 tx=0).stop 2023-07-06T03:54:08.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 -- 172.21.15.97:0/3560575755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497415c610 msgr2=0x7f49740b6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497415c610 0x7f49740b6570 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f497408b120 tx=0x7f496c004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 -- 172.21.15.97:0/3560575755 shutdown_connections 2023-07-06T03:54:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f496006ace0 0x7f496006d190 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f497415c610 0x7f49740b6570 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f497406b360 0x7f497414b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 --2- 172.21.15.97:0/3560575755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49740d2210 0x7f49740c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 -- 172.21.15.97:0/3560575755 >> 172.21.15.97:0/3560575755 conn(0x7f497406a6c0 msgr2=0x7f49740b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 -- 172.21.15.97:0/3560575755 shutdown_connections 2023-07-06T03:54:08.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.289+0000 7f4979bff700 1 -- 172.21.15.97:0/3560575755 wait complete. 2023-07-06T03:54:08.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:54:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.506+0000 7fdd353ff700 1 Processor -- start 2023-07-06T03:54:08.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.506+0000 7fdd353ff700 1 -- start start 2023-07-06T03:54:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd353ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd3015c610 0x7fdd300b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd353ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd300cff90 con 0x7fdd3006b360 2023-07-06T03:54:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd353ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd300b5650 con 0x7fdd300d2210 2023-07-06T03:54:08.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd353ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd300d0780 con 0x7fdd3015c610 2023-07-06T03:54:08.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd2effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd3015c610 0x7fdd300b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd27fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdd3006b360 0x7fdd300b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56756/0 (socket says 172.21.15.97:56756) 2023-07-06T03:54:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd27fff700 1 -- 172.21.15.97:0/2204431065 learned_addr learned my addr 172.21.15.97:0/2204431065 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd2e7fc700 1 -- 172.21.15.97:0/2204431065 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9805350 0 0) 0x7fdd300cff90 con 0x7fdd3006b360 2023-07-06T03:54:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd2e7fc700 1 -- 172.21.15.97:0/2204431065 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd1c0035b0 con 0x7fdd3006b360 2023-07-06T03:54:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd2effd700 1 -- 172.21.15.97:0/2204431065 >> v1:172.21.15.146:6789/0 conn(0x7fdd300d2210 legacy=0x7fdd300d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd2effd700 1 -- 172.21.15.97:0/2204431065 >> v1:172.21.15.97:6789/0 conn(0x7fdd3006b360 legacy=0x7fdd300b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.508+0000 7fdd2effd700 1 -- 172.21.15.97:0/2204431065 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd300b5400 con 0x7fdd3015c610 2023-07-06T03:54:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.509+0000 7fdd2effd700 1 --2- 172.21.15.97:0/2204431065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3015c610 0x7fdd300b47b0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009840 tx=0x7fdd20009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ca0edeaa07e18e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.509+0000 7fdd2e7fc700 1 -- 172.21.15.97:0/2204431065 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20011830 con 0x7fdd3015c610 2023-07-06T03:54:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.509+0000 7fdd2e7fc700 1 -- 172.21.15.97:0/2204431065 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd20011990 con 0x7fdd3015c610 2023-07-06T03:54:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.509+0000 7fdd2e7fc700 1 -- 172.21.15.97:0/2204431065 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20011d20 con 0x7fdd3015c610 2023-07-06T03:54:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.509+0000 7fdd353ff700 1 -- 172.21.15.97:0/2204431065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3015c610 msgr2=0x7fdd300b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.509+0000 7fdd353ff700 1 --2- 172.21.15.97:0/2204431065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3015c610 0x7fdd300b47b0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009840 tx=0x7fdd20009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:08.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.509+0000 7fdd353ff700 1 -- 172.21.15.97:0/2204431065 shutdown_connections 2023-07-06T03:54:08.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.510+0000 7fdd353ff700 1 --2- 172.21.15.97:0/2204431065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3015c610 0x7fdd300b47b0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.510+0000 7fdd353ff700 1 -- 172.21.15.97:0/2204431065 >> 172.21.15.97:0/2204431065 conn(0x7fdd3006a6c0 msgr2=0x7fdd3006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:08.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.510+0000 7fdd353ff700 1 -- 172.21.15.97:0/2204431065 shutdown_connections 2023-07-06T03:54:08.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.510+0000 7fdd353ff700 1 -- 172.21.15.97:0/2204431065 wait complete. 2023-07-06T03:54:08.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.511+0000 7fdd353ff700 1 Processor -- start 2023-07-06T03:54:08.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.511+0000 7fdd353ff700 1 -- start start 2023-07-06T03:54:08.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.512+0000 7fdd353ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd3006b360 0x7fdd3014f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.512+0000 7fdd353ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd300d2210 0x7fdd300c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.512+0000 7fdd353ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3015c610 0x7fdd300b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.512+0000 7fdd353ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3015e170 con 0x7fdd3015c610 2023-07-06T03:54:08.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.512+0000 7fdd353ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3015e450 con 0x7fdd300d2210 2023-07-06T03:54:08.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.512+0000 7fdd353ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3015e730 con 0x7fdd3006b360 2023-07-06T03:54:08.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd2f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3015c610 0x7fdd300b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd2f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3015c610 0x7fdd300b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55410/0 (socket says 172.21.15.97:55410) 2023-07-06T03:54:08.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd2f7fe700 1 -- 172.21.15.97:0/3992707407 learned_addr learned my addr 172.21.15.97:0/3992707407 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:08.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd2effd700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd3006b360 0x7fdd3014f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd27fff700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd300d2210 0x7fdd300c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd2f7fe700 1 -- 172.21.15.97:0/3992707407 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd3006b360 msgr2=0x7fdd3014f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd2f7fe700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd3006b360 0x7fdd3014f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd2f7fe700 1 -- 172.21.15.97:0/3992707407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd300d2210 msgr2=0x7fdd300c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd2f7fe700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd300d2210 0x7fdd300c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd2f7fe700 1 -- 172.21.15.97:0/3992707407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd200094a0 con 0x7fdd3015c610 2023-07-06T03:54:08.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd27fff700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd300d2210 0x7fdd300c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:08.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd2f7fe700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3015c610 0x7fdd300b38e0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800cc50 tx=0x7fdd2800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:08.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd277fe700 1 -- 172.21.15.97:0/3992707407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28003c00 con 0x7fdd3015c610 2023-07-06T03:54:08.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.513+0000 7fdd277fe700 1 -- 172.21.15.97:0/3992707407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd28003d60 con 0x7fdd3015c610 2023-07-06T03:54:08.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.514+0000 7fdd353ff700 1 -- 172.21.15.97:0/3992707407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3015ea10 con 0x7fdd3015c610 2023-07-06T03:54:08.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.514+0000 7fdd277fe700 1 -- 172.21.15.97:0/3992707407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2800ab60 con 0x7fdd3015c610 2023-07-06T03:54:08.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.514+0000 7fdd353ff700 1 -- 172.21.15.97:0/3992707407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd300c6da0 con 0x7fdd3015c610 2023-07-06T03:54:08.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.515+0000 7fdd353ff700 1 -- 172.21.15.97:0/3992707407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3015ec00 con 0x7fdd3015c610 2023-07-06T03:54:08.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.515+0000 7fdd277fe700 1 -- 172.21.15.97:0/3992707407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd2800acc0 con 0x7fdd3015c610 2023-07-06T03:54:08.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.516+0000 7fdd277fe700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd1006ace0 0x7fdd1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.516+0000 7fdd277fe700 1 -- 172.21.15.97:0/3992707407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fdd280895e0 con 0x7fdd3015c610 2023-07-06T03:54:08.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.517+0000 7fdd2effd700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd1006ace0 0x7fdd1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.517+0000 7fdd2effd700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd1006ace0 0x7fdd1006d190 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdd200090b0 tx=0x7fdd20008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:08.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.520+0000 7fdd277fe700 1 -- 172.21.15.97:0/3992707407 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdd28054570 con 0x7fdd3015c610 2023-07-06T03:54:08.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.712+0000 7fdd353ff700 1 -- 172.21.15.97:0/3992707407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd3015e2b0 con 0x7fdd3015c610 2023-07-06T03:54:08.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.713+0000 7fdd277fe700 1 -- 172.21.15.97:0/3992707407 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd3015e2b0 con 0x7fdd3015c610 2023-07-06T03:54:08.716 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:54:08.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 -- 172.21.15.97:0/3992707407 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd1006ace0 msgr2=0x7fdd1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd1006ace0 0x7fdd1006d190 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdd200090b0 tx=0x7fdd20008030 comp rx=0 tx=0).stop 2023-07-06T03:54:08.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 -- 172.21.15.97:0/3992707407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3015c610 msgr2=0x7fdd300b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3015c610 0x7fdd300b38e0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800cc50 tx=0x7fdd2800cf60 comp rx=0 tx=0).stop 2023-07-06T03:54:08.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 -- 172.21.15.97:0/3992707407 shutdown_connections 2023-07-06T03:54:08.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdd1006ace0 0x7fdd1006d190 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3015c610 0x7fdd300b38e0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdd3006b360 0x7fdd3014f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 --2- 172.21.15.97:0/3992707407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd300d2210 0x7fdd300c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 -- 172.21.15.97:0/3992707407 >> 172.21.15.97:0/3992707407 conn(0x7fdd3006a6c0 msgr2=0x7fdd300d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:08.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 -- 172.21.15.97:0/3992707407 shutdown_connections 2023-07-06T03:54:08.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.715+0000 7fdd353ff700 1 -- 172.21.15.97:0/3992707407 wait complete. 2023-07-06T03:54:08.735 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:54:08.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.932+0000 7f2aff5ff700 1 Processor -- start 2023-07-06T03:54:08.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.932+0000 7f2aff5ff700 1 -- start start 2023-07-06T03:54:08.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2aff5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2af815c1c0 0x7f2af80cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2aff5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af815c590 0x7f2af815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2aff5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2af80719c0 0x7f2af806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2aff5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af80ccc30 con 0x7f2af80719c0 2023-07-06T03:54:08.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2aff5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af80ccf10 con 0x7f2af815c1c0 2023-07-06T03:54:08.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2aff5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80cd1f0 con 0x7f2af815c590 2023-07-06T03:54:08.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2afdb9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2af80719c0 0x7f2af806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2afdb9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2af80719c0 0x7f2af806a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55426/0 (socket says 172.21.15.97:55426) 2023-07-06T03:54:08.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2afdb9c700 1 -- 172.21.15.97:0/1398697503 learned_addr learned my addr 172.21.15.97:0/1398697503 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:08.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2afcb9a700 1 --2- 172.21.15.97:0/1398697503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af815c590 0x7f2af815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2afd39b700 1 --2- 172.21.15.97:0/1398697503 >> v2:172.21.15.146:3300/0 conn(0x7f2af815c1c0 0x7f2af80cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2afdb9c700 1 -- 172.21.15.97:0/1398697503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af815c590 msgr2=0x7f2af815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:08.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2afdb9c700 1 --2- 172.21.15.97:0/1398697503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af815c590 0x7f2af815d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2afdb9c700 1 -- 172.21.15.97:0/1398697503 >> v2:172.21.15.146:3300/0 conn(0x7f2af815c1c0 msgr2=0x7f2af80cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:08.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2afdb9c700 1 --2- 172.21.15.97:0/1398697503 >> v2:172.21.15.146:3300/0 conn(0x7f2af815c1c0 0x7f2af80cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2afdb9c700 1 -- 172.21.15.97:0/1398697503 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af815cb00 con 0x7f2af80719c0 2023-07-06T03:54:08.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.934+0000 7f2afcb9a700 1 --2- 172.21.15.97:0/1398697503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af815c590 0x7f2af815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:08.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2afdb9c700 1 --2- 172.21.15.97:0/1398697503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af80719c0 0x7f2af806a060 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2af8072570 tx=0x7f2af000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cec99b7ef434d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:08.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2aeffff700 1 -- 172.21.15.97:0/1398697503 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000e070 con 0x7f2af80719c0 2023-07-06T03:54:08.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2aeffff700 1 -- 172.21.15.97:0/1398697503 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000b9c0 con 0x7f2af80719c0 2023-07-06T03:54:08.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2aeffff700 1 -- 172.21.15.97:0/1398697503 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000bd70 con 0x7f2af80719c0 2023-07-06T03:54:08.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2aff5ff700 1 -- 172.21.15.97:0/1398697503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af80719c0 msgr2=0x7f2af806a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2aff5ff700 1 --2- 172.21.15.97:0/1398697503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af80719c0 0x7f2af806a060 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2af8072570 tx=0x7f2af000b330 comp rx=0 tx=0).stop 2023-07-06T03:54:08.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2aff5ff700 1 -- 172.21.15.97:0/1398697503 shutdown_connections 2023-07-06T03:54:08.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2aff5ff700 1 --2- 172.21.15.97:0/1398697503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af80719c0 0x7f2af806a060 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2aff5ff700 1 --2- 172.21.15.97:0/1398697503 >> v2:172.21.15.146:3300/0 conn(0x7f2af815c1c0 0x7f2af80cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2aff5ff700 1 --2- 172.21.15.97:0/1398697503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af815c590 0x7f2af815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2aff5ff700 1 -- 172.21.15.97:0/1398697503 >> 172.21.15.97:0/1398697503 conn(0x7f2af80b1650 msgr2=0x7f2af80b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:08.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.935+0000 7f2aff5ff700 1 -- 172.21.15.97:0/1398697503 shutdown_connections 2023-07-06T03:54:08.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.936+0000 7f2aff5ff700 1 -- 172.21.15.97:0/1398697503 wait complete. 2023-07-06T03:54:08.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.937+0000 7f2aff5ff700 1 Processor -- start 2023-07-06T03:54:08.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.937+0000 7f2aff5ff700 1 -- start start 2023-07-06T03:54:08.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.938+0000 7f2aff5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af80719c0 0x7f2af81465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.938+0000 7f2aff5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af815c1c0 0x7f2af8148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.938+0000 7f2aff5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af815c590 0x7f2af806de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.938+0000 7f2aff5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af8146ee0 con 0x7f2af80719c0 2023-07-06T03:54:08.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.938+0000 7f2aff5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af8149a30 con 0x7f2af815c1c0 2023-07-06T03:54:08.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.938+0000 7f2aff5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af8149d10 con 0x7f2af815c590 2023-07-06T03:54:08.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.938+0000 7f2afd39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af80719c0 0x7f2af81465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.938+0000 7f2afd39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af80719c0 0x7f2af81465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55428/0 (socket says 172.21.15.97:55428) 2023-07-06T03:54:08.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2afd39b700 1 -- 172.21.15.97:0/3928239249 learned_addr learned my addr 172.21.15.97:0/3928239249 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:08.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2afdb9c700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af815c590 0x7f2af806de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2afcb9a700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af815c1c0 0x7f2af8148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2afd39b700 1 -- 172.21.15.97:0/3928239249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af815c590 msgr2=0x7f2af806de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2afd39b700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af815c590 0x7f2af806de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2afd39b700 1 -- 172.21.15.97:0/3928239249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af815c1c0 msgr2=0x7f2af8148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:08.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2afd39b700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af815c1c0 0x7f2af8148f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:08.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2afd39b700 1 -- 172.21.15.97:0/3928239249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af0009b30 con 0x7f2af80719c0 2023-07-06T03:54:08.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2afd39b700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af80719c0 0x7f2af81465b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2af400e810 tx=0x7f2af400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2aedffb700 1 -- 172.21.15.97:0/3928239249 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4013610 con 0x7f2af80719c0 2023-07-06T03:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2aedffb700 1 -- 172.21.15.97:0/3928239249 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af4013770 con 0x7f2af80719c0 2023-07-06T03:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2aff5ff700 1 -- 172.21.15.97:0/3928239249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af81470f0 con 0x7f2af80719c0 2023-07-06T03:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2aedffb700 1 -- 172.21.15.97:0/3928239249 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4013b00 con 0x7f2af80719c0 2023-07-06T03:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.939+0000 7f2aff5ff700 1 -- 172.21.15.97:0/3928239249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af80b52f0 con 0x7f2af80719c0 2023-07-06T03:54:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.940+0000 7f2aff5ff700 1 -- 172.21.15.97:0/3928239249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af804e340 con 0x7f2af80719c0 2023-07-06T03:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.942+0000 7f2aedffb700 1 -- 172.21.15.97:0/3928239249 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2af4013c60 con 0x7f2af80719c0 2023-07-06T03:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.943+0000 7f2aedffb700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2adc06ace0 0x7f2adc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.943+0000 7f2aedffb700 1 -- 172.21.15.97:0/3928239249 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f2af4089150 con 0x7f2af80719c0 2023-07-06T03:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.943+0000 7f2afcb9a700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2adc06ace0 0x7f2adc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.944+0000 7f2afcb9a700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2adc06ace0 0x7f2adc06d190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2af80c5610 tx=0x7f2ae8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:08.948+0000 7f2aedffb700 1 -- 172.21.15.97:0/3928239249 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2af4054180 con 0x7f2af80719c0 2023-07-06T03:54:09.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.143+0000 7f2aff5ff700 1 -- 172.21.15.97:0/3928239249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2af8149ed0 con 0x7f2af80719c0 2023-07-06T03:54:09.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.143+0000 7f2aedffb700 1 -- 172.21.15.97:0/3928239249 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2af40577a0 con 0x7f2af80719c0 2023-07-06T03:54:09.146 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:54:09.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.145+0000 7f2aff5ff700 1 -- 172.21.15.97:0/3928239249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2adc06ace0 msgr2=0x7f2adc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.145+0000 7f2aff5ff700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2adc06ace0 0x7f2adc06d190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2af80c5610 tx=0x7f2ae8008040 comp rx=0 tx=0).stop 2023-07-06T03:54:09.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.145+0000 7f2aff5ff700 1 -- 172.21.15.97:0/3928239249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af80719c0 msgr2=0x7f2af81465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.145+0000 7f2aff5ff700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af80719c0 0x7f2af81465b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2af400e810 tx=0x7f2af400ebd0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.146+0000 7f2aff5ff700 1 -- 172.21.15.97:0/3928239249 shutdown_connections 2023-07-06T03:54:09.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.146+0000 7f2aff5ff700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2adc06ace0 0x7f2adc06d190 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.146+0000 7f2aff5ff700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2af815c590 0x7f2af806de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.146+0000 7f2aff5ff700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af80719c0 0x7f2af81465b0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.146+0000 7f2aff5ff700 1 --2- 172.21.15.97:0/3928239249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2af815c1c0 0x7f2af8148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.146+0000 7f2aff5ff700 1 -- 172.21.15.97:0/3928239249 >> 172.21.15.97:0/3928239249 conn(0x7f2af80b1650 msgr2=0x7f2af806a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.146+0000 7f2aff5ff700 1 -- 172.21.15.97:0/3928239249 shutdown_connections 2023-07-06T03:54:09.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.146+0000 7f2aff5ff700 1 -- 172.21.15.97:0/3928239249 wait complete. 2023-07-06T03:54:09.166 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:09.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.359+0000 7f260abff700 1 Processor -- start 2023-07-06T03:54:09.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.359+0000 7f260abff700 1 -- start start 2023-07-06T03:54:09.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.360+0000 7f260abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260415c520 0x7f26040cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.360+0000 7f260abff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2604160180 0x7f26041605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.360+0000 7f260abff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26040cdae0 con 0x7f2604160b30 2023-07-06T03:54:09.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.360+0000 7f260abff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040cd8d0 con 0x7f2604160180 2023-07-06T03:54:09.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.360+0000 7f260abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26040b6640 con 0x7f260415c520 2023-07-06T03:54:09.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.360+0000 7f2603fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2604160180 0x7f26041605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.360+0000 7f260919c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2604160b30 0x7f26040cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56760/0 (socket says 172.21.15.97:56760) 2023-07-06T03:54:09.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.360+0000 7f260919c700 1 -- 172.21.15.97:0/3897982825 learned_addr learned my addr 172.21.15.97:0/3897982825 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:09.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.361+0000 7f260899b700 1 --2- 172.21.15.97:0/3897982825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260415c520 0x7f26040cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.361+0000 7f2603fff700 1 -- 172.21.15.97:0/3897982825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260415c520 msgr2=0x7f26040cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:09.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.361+0000 7f2603fff700 1 --2- 172.21.15.97:0/3897982825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260415c520 0x7f26040cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.361+0000 7f2603fff700 1 -- 172.21.15.97:0/3897982825 >> v1:172.21.15.97:6789/0 conn(0x7f2604160b30 legacy=0x7f26040cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.361+0000 7f2603fff700 1 -- 172.21.15.97:0/3897982825 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040b6920 con 0x7f2604160180 2023-07-06T03:54:09.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.361+0000 7f26037fe700 1 -- 172.21.15.97:0/3897982825 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494825161 0 0) 0x7f26040cdae0 con 0x7f2604160b30 2023-07-06T03:54:09.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.361+0000 7f2603fff700 1 --2- 172.21.15.97:0/3897982825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2604160180 0x7f26041605f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009840 tx=0x7f25f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68729ce4d7a3e25 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:09.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.361+0000 7f26037fe700 1 -- 172.21.15.97:0/3897982825 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0011830 con 0x7f2604160180 2023-07-06T03:54:09.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.361+0000 7f26037fe700 1 -- 172.21.15.97:0/3897982825 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f0011990 con 0x7f2604160180 2023-07-06T03:54:09.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.361+0000 7f26037fe700 1 -- 172.21.15.97:0/3897982825 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0011d20 con 0x7f2604160180 2023-07-06T03:54:09.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.362+0000 7f260abff700 1 -- 172.21.15.97:0/3897982825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2604160180 msgr2=0x7f26041605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.362+0000 7f260abff700 1 --2- 172.21.15.97:0/3897982825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2604160180 0x7f26041605f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009840 tx=0x7f25f0009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:09.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.362+0000 7f260abff700 1 -- 172.21.15.97:0/3897982825 shutdown_connections 2023-07-06T03:54:09.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.362+0000 7f260abff700 1 --2- 172.21.15.97:0/3897982825 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f260415c520 0x7f26040cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.362+0000 7f260abff700 1 --2- 172.21.15.97:0/3897982825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2604160180 0x7f26041605f0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.362+0000 7f260abff700 1 -- 172.21.15.97:0/3897982825 >> 172.21.15.97:0/3897982825 conn(0x7f2604071790 msgr2=0x7f2604071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:09.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.362+0000 7f260abff700 1 -- 172.21.15.97:0/3897982825 shutdown_connections 2023-07-06T03:54:09.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.362+0000 7f260abff700 1 -- 172.21.15.97:0/3897982825 wait complete. 2023-07-06T03:54:09.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.364+0000 7f260abff700 1 Processor -- start 2023-07-06T03:54:09.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.364+0000 7f260abff700 1 -- start start 2023-07-06T03:54:09.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260abff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260415c520 0x7f2604155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2604160180 0x7f26040cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260abff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2604160b30 0x7f260415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260abff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040cb500 con 0x7f260415c520 2023-07-06T03:54:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260abff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040d25c0 con 0x7f2604160b30 2023-07-06T03:54:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260415db30 con 0x7f2604160180 2023-07-06T03:54:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260899b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260415c520 0x7f2604155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260899b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260415c520 0x7f2604155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55442/0 (socket says 172.21.15.97:55442) 2023-07-06T03:54:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260899b700 1 -- 172.21.15.97:0/2422393455 learned_addr learned my addr 172.21.15.97:0/2422393455 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f2603fff700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2604160180 0x7f26040cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260919c700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2604160b30 0x7f260415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260899b700 1 -- 172.21.15.97:0/2422393455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2604160180 msgr2=0x7f26040cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260899b700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2604160180 0x7f26040cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260899b700 1 -- 172.21.15.97:0/2422393455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2604160b30 msgr2=0x7f260415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260899b700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2604160b30 0x7f260415d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.365+0000 7f260899b700 1 -- 172.21.15.97:0/2422393455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f00094a0 con 0x7f260415c520 2023-07-06T03:54:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.366+0000 7f260899b700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260415c520 0x7f2604155dc0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f25f800e870 tx=0x7f25f800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.366+0000 7f26017fa700 1 -- 172.21.15.97:0/2422393455 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8009590 con 0x7f260415c520 2023-07-06T03:54:09.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.366+0000 7f26017fa700 1 -- 172.21.15.97:0/2422393455 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f800ee70 con 0x7f260415c520 2023-07-06T03:54:09.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.366+0000 7f260abff700 1 -- 172.21.15.97:0/2422393455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26040d1c90 con 0x7f260415c520 2023-07-06T03:54:09.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.366+0000 7f26017fa700 1 -- 172.21.15.97:0/2422393455 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f800ca50 con 0x7f260415c520 2023-07-06T03:54:09.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.366+0000 7f260abff700 1 -- 172.21.15.97:0/2422393455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26040d2250 con 0x7f260415c520 2023-07-06T03:54:09.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.367+0000 7f260abff700 1 -- 172.21.15.97:0/2422393455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260404e340 con 0x7f260415c520 2023-07-06T03:54:09.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.368+0000 7f26017fa700 1 -- 172.21.15.97:0/2422393455 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f25f800cbb0 con 0x7f260415c520 2023-07-06T03:54:09.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.369+0000 7f26017fa700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25e806ace0 0x7f25e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.369+0000 7f26017fa700 1 -- 172.21.15.97:0/2422393455 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f25f8089470 con 0x7f260415c520 2023-07-06T03:54:09.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.369+0000 7f2603fff700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25e806ace0 0x7f25e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.370+0000 7f2603fff700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25e806ace0 0x7f25e806d190 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f25f00090b0 tx=0x7f25f0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:09.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.373+0000 7f26017fa700 1 -- 172.21.15.97:0/2422393455 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f25f8057f20 con 0x7f260415c520 2023-07-06T03:54:09.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.607+0000 7f260abff700 1 -- 172.21.15.97:0/2422393455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26040b4f40 con 0x7f260415c520 2023-07-06T03:54:09.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.607+0000 7f26017fa700 1 -- 172.21.15.97:0/2422393455 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f25f800ce60 con 0x7f260415c520 2023-07-06T03:54:09.610 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:09.610 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/387292825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":387292825},{"type":"v1","addr":"172.21.15.146:6833","nonce":387292825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690380561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690380561},{"type":"v1","addr":"172.21.15.97:6837","nonce":690380561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3897834271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3897834271},{"type":"v1","addr":"172.21.15.146:6835","nonce":3897834271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:53:26.234190+0000","modified":"2023-07-06T03:54:06.823422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6836/3324055785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3324055785},{"type":"v1","addr":"172.21.15.97:6836","nonce":3324055785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-06T03:54:09.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.609+0000 7f260abff700 1 -- 172.21.15.97:0/2422393455 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25e806ace0 msgr2=0x7f25e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.609+0000 7f260abff700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25e806ace0 0x7f25e806d190 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f25f00090b0 tx=0x7f25f0008030 comp rx=0 tx=0).stop 2023-07-06T03:54:09.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.609+0000 7f260abff700 1 -- 172.21.15.97:0/2422393455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260415c520 msgr2=0x7f2604155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.609+0000 7f260abff700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260415c520 0x7f2604155dc0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f25f800e870 tx=0x7f25f800ec30 comp rx=0 tx=0).stop 2023-07-06T03:54:09.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.610+0000 7f260abff700 1 -- 172.21.15.97:0/2422393455 shutdown_connections 2023-07-06T03:54:09.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.610+0000 7f260abff700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f25e806ace0 0x7f25e806d190 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.610+0000 7f260abff700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2604160b30 0x7f260415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.610+0000 7f260abff700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f260415c520 0x7f2604155dc0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.610+0000 7f260abff700 1 --2- 172.21.15.97:0/2422393455 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2604160180 0x7f26040cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.610+0000 7f260abff700 1 -- 172.21.15.97:0/2422393455 >> 172.21.15.97:0/2422393455 conn(0x7f2604071790 msgr2=0x7f2604158140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:09.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.610+0000 7f260abff700 1 -- 172.21.15.97:0/2422393455 shutdown_connections 2023-07-06T03:54:09.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.610+0000 7f260abff700 1 -- 172.21.15.97:0/2422393455 wait complete. 2023-07-06T03:54:09.614 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 71 2023-07-06T03:54:09.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:09.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.822+0000 7fdb7a3ff700 1 Processor -- start 2023-07-06T03:54:09.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.822+0000 7fdb7a3ff700 1 -- start start 2023-07-06T03:54:09.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.824+0000 7fdb7a3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdb740cd390 0x7fdb74160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.824+0000 7fdb7a3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb74160dd0 0x7fdb740b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.824+0000 7fdb7a3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb740cdae0 con 0x7fdb7415c520 2023-07-06T03:54:09.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.824+0000 7fdb7a3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb740b6640 con 0x7fdb740cd390 2023-07-06T03:54:09.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.824+0000 7fdb7a3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb740b6920 con 0x7fdb74160dd0 2023-07-06T03:54:09.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.824+0000 7fdb6b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdb740cd390 0x7fdb74160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.824+0000 7fdb6b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdb740cd390 0x7fdb74160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55452/0 (socket says 172.21.15.97:55452) 2023-07-06T03:54:09.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.824+0000 7fdb6b7fe700 1 -- 172.21.15.97:0/3535328095 learned_addr learned my addr 172.21.15.97:0/3535328095 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:09.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.824+0000 7fdb7899c700 1 --2- 172.21.15.97:0/3535328095 >> v2:172.21.15.146:3300/0 conn(0x7fdb74160dd0 0x7fdb740b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.825+0000 7fdb6b7fe700 1 -- 172.21.15.97:0/3535328095 >> v2:172.21.15.146:3300/0 conn(0x7fdb74160dd0 msgr2=0x7fdb740b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:09.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.825+0000 7fdb6b7fe700 1 --2- 172.21.15.97:0/3535328095 >> v2:172.21.15.146:3300/0 conn(0x7fdb74160dd0 0x7fdb740b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.825+0000 7fdb6b7fe700 1 -- 172.21.15.97:0/3535328095 >> v1:172.21.15.146:6789/0 conn(0x7fdb7415c520 legacy=0x7fdb74160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.825+0000 7fdb6b7fe700 1 -- 172.21.15.97:0/3535328095 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb740b6c00 con 0x7fdb740cd390 2023-07-06T03:54:09.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.825+0000 7fdb6b7fe700 1 --2- 172.21.15.97:0/3535328095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb740cd390 0x7fdb74160890 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fdb740cd320 tx=0x7fdb5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25987eb4e6eba789 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:09.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.825+0000 7fdb737fe700 1 -- 172.21.15.97:0/3535328095 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c011830 con 0x7fdb740cd390 2023-07-06T03:54:09.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.825+0000 7fdb737fe700 1 -- 172.21.15.97:0/3535328095 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c011990 con 0x7fdb740cd390 2023-07-06T03:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.825+0000 7fdb737fe700 1 -- 172.21.15.97:0/3535328095 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c011d20 con 0x7fdb740cd390 2023-07-06T03:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.826+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/3535328095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb740cd390 msgr2=0x7fdb74160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.826+0000 7fdb7a3ff700 1 --2- 172.21.15.97:0/3535328095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb740cd390 0x7fdb74160890 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fdb740cd320 tx=0x7fdb5c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.826+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/3535328095 shutdown_connections 2023-07-06T03:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.826+0000 7fdb7a3ff700 1 --2- 172.21.15.97:0/3535328095 >> v2:172.21.15.146:3300/0 conn(0x7fdb74160dd0 0x7fdb740b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.826+0000 7fdb7a3ff700 1 --2- 172.21.15.97:0/3535328095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb740cd390 0x7fdb74160890 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.826+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/3535328095 >> 172.21.15.97:0/3535328095 conn(0x7fdb74071790 msgr2=0x7fdb74071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:09.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.826+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/3535328095 shutdown_connections 2023-07-06T03:54:09.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.826+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/3535328095 wait complete. 2023-07-06T03:54:09.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.828+0000 7fdb7a3ff700 1 Processor -- start 2023-07-06T03:54:09.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.828+0000 7fdb7a3ff700 1 -- start start 2023-07-06T03:54:09.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.829+0000 7fdb7a3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb740cd390 0x7fdb740d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.829+0000 7fdb7a3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7415c520 0x7fdb74158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.829+0000 7fdb7a3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb74160dd0 0x7fdb74158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.829+0000 7fdb7a3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb740d2ab0 con 0x7fdb740cd390 2023-07-06T03:54:09.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.829+0000 7fdb7a3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb740c9380 con 0x7fdb7415c520 2023-07-06T03:54:09.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.829+0000 7fdb7a3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb740d1e30 con 0x7fdb74160dd0 2023-07-06T03:54:09.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.829+0000 7fdb73fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb740cd390 0x7fdb740d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.829+0000 7fdb73fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb740cd390 0x7fdb740d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55468/0 (socket says 172.21.15.97:55468) 2023-07-06T03:54:09.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.829+0000 7fdb73fff700 1 -- 172.21.15.97:0/2192177239 learned_addr learned my addr 172.21.15.97:0/2192177239 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:09.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.829+0000 7fdb7899c700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb74160dd0 0x7fdb74158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.829+0000 7fdb6b7fe700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7415c520 0x7fdb74158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb73fff700 1 -- 172.21.15.97:0/2192177239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb74160dd0 msgr2=0x7fdb74158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb73fff700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb74160dd0 0x7fdb74158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb73fff700 1 -- 172.21.15.97:0/2192177239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7415c520 msgr2=0x7fdb74158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:09.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb73fff700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7415c520 0x7fdb74158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:09.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb73fff700 1 -- 172.21.15.97:0/2192177239 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c009550 con 0x7fdb740cd390 2023-07-06T03:54:09.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb6b7fe700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7415c520 0x7fdb74158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:09.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb7899c700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb74160dd0 0x7fdb74158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:09.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb73fff700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb740cd390 0x7fdb740d2ed0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fdb64001ed0 tx=0x7fdb6400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:09.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb717fa700 1 -- 172.21.15.97:0/2192177239 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6400ade0 con 0x7fdb740cd390 2023-07-06T03:54:09.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb717fa700 1 -- 172.21.15.97:0/2192177239 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb640020d0 con 0x7fdb740cd390 2023-07-06T03:54:09.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/2192177239 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb740d2110 con 0x7fdb740cd390 2023-07-06T03:54:09.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb717fa700 1 -- 172.21.15.97:0/2192177239 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64003c20 con 0x7fdb740cd390 2023-07-06T03:54:09.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.830+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/2192177239 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb740d26d0 con 0x7fdb740cd390 2023-07-06T03:54:09.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.831+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/2192177239 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb740d2d20 con 0x7fdb740cd390 2023-07-06T03:54:09.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.832+0000 7fdb717fa700 1 -- 172.21.15.97:0/2192177239 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb640050d0 con 0x7fdb740cd390 2023-07-06T03:54:09.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.833+0000 7fdb717fa700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb5406ace0 0x7fdb5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:09.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.834+0000 7fdb717fa700 1 -- 172.21.15.97:0/2192177239 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fdb64089720 con 0x7fdb740cd390 2023-07-06T03:54:09.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.834+0000 7fdb6b7fe700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb5406ace0 0x7fdb5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:09.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.834+0000 7fdb6b7fe700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb5406ace0 0x7fdb5406d190 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c014040 tx=0x7fdb5c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:09.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:09.840+0000 7fdb717fa700 1 -- 172.21.15.97:0/2192177239 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdb640546b0 con 0x7fdb740cd390 2023-07-06T03:54:10.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.036+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/2192177239 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb74160180 con 0x7fdb740cd390 2023-07-06T03:54:10.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.038+0000 7fdb717fa700 1 -- 172.21.15.97:0/2192177239 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+18516 (secure 0 0 0) 0x7fdb64057cd0 con 0x7fdb740cd390 2023-07-06T03:54:10.041 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:10.041 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":101,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:53:54.754060+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:53:24.013743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-06T03:53:25.222178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":6}]},{"pgid":"16.9","mappings":[{"from":4,"to":3}]},{"pgid":"16.15","mappings":[{"from":4,"to":3}]},{"pgid":"16.24","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:10.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/2192177239 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb5406ace0 msgr2=0x7fdb5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb5406ace0 0x7fdb5406d190 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c014040 tx=0x7fdb5c008030 comp rx=0 tx=0).stop 2023-07-06T03:54:10.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/2192177239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb740cd390 msgr2=0x7fdb740d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb740cd390 0x7fdb740d2ed0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fdb64001ed0 tx=0x7fdb6400a340 comp rx=0 tx=0).stop 2023-07-06T03:54:10.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/2192177239 shutdown_connections 2023-07-06T03:54:10.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb5406ace0 0x7fdb5406d190 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb74160dd0 0x7fdb74158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb740cd390 0x7fdb740d2ed0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 --2- 172.21.15.97:0/2192177239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb7415c520 0x7fdb74158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/2192177239 >> 172.21.15.97:0/2192177239 conn(0x7fdb74071790 msgr2=0x7fdb740d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:10.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/2192177239 shutdown_connections 2023-07-06T03:54:10.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.040+0000 7fdb7a3ff700 1 -- 172.21.15.97:0/2192177239 wait complete. 2023-07-06T03:54:10.061 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T03:54:10.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.252+0000 7f31991ff700 1 Processor -- start 2023-07-06T03:54:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.253+0000 7f31991ff700 1 -- start start 2023-07-06T03:54:10.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.254+0000 7f31991ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f319415d940 0x7f3194160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:10.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.254+0000 7f31991ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31940d0590 con 0x7f31940d10b0 2023-07-06T03:54:10.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.254+0000 7f31991ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3194160f70 con 0x7f319406c9a0 2023-07-06T03:54:10.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.254+0000 7f31991ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31940d1480 con 0x7f319415d940 2023-07-06T03:54:10.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.254+0000 7f3192d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f31940d10b0 0x7f319406c290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59920/0 (socket says 172.21.15.97:59920) 2023-07-06T03:54:10.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.254+0000 7f3192d9d700 1 -- 172.21.15.97:0/2809968457 learned_addr learned my addr 172.21.15.97:0/2809968457 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:10.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.254+0000 7f319359e700 1 --2- 172.21.15.97:0/2809968457 >> v2:172.21.15.97:3300/0 conn(0x7f319415d940 0x7f3194160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:10.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.254+0000 7f319359e700 1 -- 172.21.15.97:0/2809968457 >> v1:172.21.15.146:6789/0 conn(0x7f319406c9a0 legacy=0x7f3194160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:10.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.254+0000 7f319359e700 1 -- 172.21.15.97:0/2809968457 >> v1:172.21.15.97:6789/0 conn(0x7f31940d10b0 legacy=0x7f319406c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.255+0000 7f319359e700 1 -- 172.21.15.97:0/2809968457 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319415ce60 con 0x7f319415d940 2023-07-06T03:54:10.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.254+0000 7f319259c700 1 -- 172.21.15.97:0/2809968457 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791612949 0 0) 0x7f3184001290 con 0x7f31940d10b0 2023-07-06T03:54:10.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.255+0000 7f319359e700 1 --2- 172.21.15.97:0/2809968457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319415d940 0x7f3194160910 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f318c009d90 tx=0x7f318c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a2e2de672e7a25 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:10.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.255+0000 7f319259c700 1 -- 172.21.15.97:0/2809968457 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c00e070 con 0x7f319415d940 2023-07-06T03:54:10.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.255+0000 7f319259c700 1 -- 172.21.15.97:0/2809968457 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c00bb90 con 0x7f319415d940 2023-07-06T03:54:10.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.255+0000 7f319259c700 1 -- 172.21.15.97:0/2809968457 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c003a20 con 0x7f319415d940 2023-07-06T03:54:10.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.255+0000 7f31991ff700 1 -- 172.21.15.97:0/2809968457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319415d940 msgr2=0x7f3194160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.255+0000 7f31991ff700 1 --2- 172.21.15.97:0/2809968457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319415d940 0x7f3194160910 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f318c009d90 tx=0x7f318c00b330 comp rx=0 tx=0).stop 2023-07-06T03:54:10.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.256+0000 7f31991ff700 1 -- 172.21.15.97:0/2809968457 shutdown_connections 2023-07-06T03:54:10.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.256+0000 7f31991ff700 1 --2- 172.21.15.97:0/2809968457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319415d940 0x7f3194160910 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.256+0000 7f31991ff700 1 -- 172.21.15.97:0/2809968457 >> 172.21.15.97:0/2809968457 conn(0x7f319406a060 msgr2=0x7f319406a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:10.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.256+0000 7f31991ff700 1 -- 172.21.15.97:0/2809968457 shutdown_connections 2023-07-06T03:54:10.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.256+0000 7f31991ff700 1 -- 172.21.15.97:0/2809968457 wait complete. 2023-07-06T03:54:10.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.257+0000 7f31991ff700 1 Processor -- start 2023-07-06T03:54:10.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.257+0000 7f31991ff700 1 -- start start 2023-07-06T03:54:10.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.258+0000 7f31991ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f319406c9a0 0x7f3194154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:10.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.258+0000 7f31991ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31940d10b0 0x7f31940b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:10.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.258+0000 7f31991ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319415d940 0x7f319415b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:10.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.258+0000 7f31991ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3194154cb0 con 0x7f319415d940 2023-07-06T03:54:10.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.258+0000 7f31991ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940b1230 con 0x7f31940d10b0 2023-07-06T03:54:10.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.258+0000 7f31991ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31940c51e0 con 0x7f319406c9a0 2023-07-06T03:54:10.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f319359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319415d940 0x7f319415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:10.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f319359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319415d940 0x7f319415b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40662/0 (socket says 172.21.15.97:40662) 2023-07-06T03:54:10.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f319359e700 1 -- 172.21.15.97:0/4241893402 learned_addr learned my addr 172.21.15.97:0/4241893402 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:10.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f3192d9d700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f319406c9a0 0x7f3194154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:10.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f319359e700 1 -- 172.21.15.97:0/4241893402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f319406c9a0 msgr2=0x7f3194154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f319359e700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f319406c9a0 0x7f3194154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f318bfff700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31940d10b0 0x7f31940b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:10.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f319359e700 1 -- 172.21.15.97:0/4241893402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31940d10b0 msgr2=0x7f31940b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f319359e700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31940d10b0 0x7f31940b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f319359e700 1 -- 172.21.15.97:0/4241893402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c009a70 con 0x7f319415d940 2023-07-06T03:54:10.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f3192d9d700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f319406c9a0 0x7f3194154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:10.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f318bfff700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31940d10b0 0x7f31940b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:10.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.259+0000 7f319359e700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319415d940 0x7f319415b150 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f319406d7d0 tx=0x7f318c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:10.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.260+0000 7f318b7fe700 1 -- 172.21.15.97:0/4241893402 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c012630 con 0x7f319415d940 2023-07-06T03:54:10.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.260+0000 7f318b7fe700 1 -- 172.21.15.97:0/4241893402 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c0127b0 con 0x7f319415d940 2023-07-06T03:54:10.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.260+0000 7f31991ff700 1 -- 172.21.15.97:0/4241893402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31940c54c0 con 0x7f319415d940 2023-07-06T03:54:10.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.260+0000 7f318b7fe700 1 -- 172.21.15.97:0/4241893402 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c012b60 con 0x7f319415d940 2023-07-06T03:54:10.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.260+0000 7f31991ff700 1 -- 172.21.15.97:0/4241893402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31940c5a80 con 0x7f319415d940 2023-07-06T03:54:10.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.261+0000 7f31991ff700 1 -- 172.21.15.97:0/4241893402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319404e340 con 0x7f319415d940 2023-07-06T03:54:10.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.262+0000 7f318b7fe700 1 -- 172.21.15.97:0/4241893402 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f318c012cc0 con 0x7f319415d940 2023-07-06T03:54:10.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.263+0000 7f318b7fe700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f317406ace0 0x7f317406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:10.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.263+0000 7f318b7fe700 1 -- 172.21.15.97:0/4241893402 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f318c08bb60 con 0x7f319415d940 2023-07-06T03:54:10.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.263+0000 7f3192d9d700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f317406ace0 0x7f317406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:10.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.264+0000 7f3192d9d700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f317406ace0 0x7f317406d190 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3184001780 tx=0x7f3184009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:10.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.267+0000 7f318b7fe700 1 -- 172.21.15.97:0/4241893402 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f318c05a6b0 con 0x7f319415d940 2023-07-06T03:54:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.500+0000 7f31991ff700 1 -- 172.21.15.97:0/4241893402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f319415ca90 con 0x7f319415d940 2023-07-06T03:54:10.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.501+0000 7f318b7fe700 1 -- 172.21.15.97:0/4241893402 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f319415ca90 con 0x7f319415d940 2023-07-06T03:54:10.504 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:10.504 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/387292825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":387292825},{"type":"v1","addr":"172.21.15.146:6833","nonce":387292825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690380561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690380561},{"type":"v1","addr":"172.21.15.97:6837","nonce":690380561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3897834271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3897834271},{"type":"v1","addr":"172.21.15.146:6835","nonce":3897834271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:53:26.234190+0000","modified":"2023-07-06T03:54:06.823422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6836/3324055785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3324055785},{"type":"v1","addr":"172.21.15.97:6836","nonce":3324055785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-06T03:54:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.503+0000 7f31991ff700 1 -- 172.21.15.97:0/4241893402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f317406ace0 msgr2=0x7f317406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.503+0000 7f31991ff700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f317406ace0 0x7f317406d190 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3184001780 tx=0x7f3184009040 comp rx=0 tx=0).stop 2023-07-06T03:54:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.503+0000 7f31991ff700 1 -- 172.21.15.97:0/4241893402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319415d940 msgr2=0x7f319415b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.503+0000 7f31991ff700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319415d940 0x7f319415b150 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f319406d7d0 tx=0x7f318c004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.503+0000 7f31991ff700 1 -- 172.21.15.97:0/4241893402 shutdown_connections 2023-07-06T03:54:10.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.503+0000 7f31991ff700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f317406ace0 0x7f317406d190 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.503+0000 7f31991ff700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319415d940 0x7f319415b150 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.503+0000 7f31991ff700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f319406c9a0 0x7f3194154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.503+0000 7f31991ff700 1 --2- 172.21.15.97:0/4241893402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31940d10b0 0x7f31940b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.503+0000 7f31991ff700 1 -- 172.21.15.97:0/4241893402 >> 172.21.15.97:0/4241893402 conn(0x7f319406a060 msgr2=0x7f319415c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:10.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.503+0000 7f31991ff700 1 -- 172.21.15.97:0/4241893402 shutdown_connections 2023-07-06T03:54:10.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.504+0000 7f31991ff700 1 -- 172.21.15.97:0/4241893402 wait complete. 2023-07-06T03:54:10.508 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 71 2023-07-06T03:54:10.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:10.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.716+0000 7fc1145ff700 1 Processor -- start 2023-07-06T03:54:10.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.716+0000 7fc1145ff700 1 -- start start 2023-07-06T03:54:10.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.717+0000 7fc1145ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc10c0cd390 0x7fc10c160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:10.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.717+0000 7fc1145ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc10c160dd0 0x7fc10c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:10.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.717+0000 7fc1145ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c0cdae0 con 0x7fc10c15c520 2023-07-06T03:54:10.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.717+0000 7fc1145ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10c0b6640 con 0x7fc10c160dd0 2023-07-06T03:54:10.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.717+0000 7fc1145ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10c0b6920 con 0x7fc10c0cd390 2023-07-06T03:54:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.718+0000 7fc112b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc10c160dd0 0x7fc10c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.718+0000 7fc112b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc10c160dd0 0x7fc10c0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40676/0 (socket says 172.21.15.97:40676) 2023-07-06T03:54:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.718+0000 7fc112b9c700 1 -- 172.21.15.97:0/4004721388 learned_addr learned my addr 172.21.15.97:0/4004721388 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.718+0000 7fc111b9a700 1 --2- 172.21.15.97:0/4004721388 >> v2:172.21.15.146:3300/0 conn(0x7fc10c0cd390 0x7fc10c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.718+0000 7fc112b9c700 1 -- 172.21.15.97:0/4004721388 >> v2:172.21.15.146:3300/0 conn(0x7fc10c0cd390 msgr2=0x7fc10c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.718+0000 7fc112b9c700 1 --2- 172.21.15.97:0/4004721388 >> v2:172.21.15.146:3300/0 conn(0x7fc10c0cd390 0x7fc10c160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.718+0000 7fc112b9c700 1 -- 172.21.15.97:0/4004721388 >> v1:172.21.15.146:6789/0 conn(0x7fc10c15c520 legacy=0x7fc10c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.718+0000 7fc112b9c700 1 -- 172.21.15.97:0/4004721388 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c0b6c00 con 0x7fc10c160dd0 2023-07-06T03:54:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.718+0000 7fc112b9c700 1 --2- 172.21.15.97:0/4004721388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10c160dd0 0x7fc10c0b7400 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0b5930 tx=0x7fc10400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe009b78bbbfb2a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:10.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.718+0000 7fc111399700 1 -- 172.21.15.97:0/4004721388 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10400e070 con 0x7fc10c160dd0 2023-07-06T03:54:10.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.719+0000 7fc111399700 1 -- 172.21.15.97:0/4004721388 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10400b9c0 con 0x7fc10c160dd0 2023-07-06T03:54:10.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.719+0000 7fc111399700 1 -- 172.21.15.97:0/4004721388 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10400bd70 con 0x7fc10c160dd0 2023-07-06T03:54:10.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.719+0000 7fc1145ff700 1 -- 172.21.15.97:0/4004721388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10c160dd0 msgr2=0x7fc10c0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.719+0000 7fc1145ff700 1 --2- 172.21.15.97:0/4004721388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10c160dd0 0x7fc10c0b7400 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0b5930 tx=0x7fc10400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:10.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.719+0000 7fc1145ff700 1 -- 172.21.15.97:0/4004721388 shutdown_connections 2023-07-06T03:54:10.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.719+0000 7fc1145ff700 1 --2- 172.21.15.97:0/4004721388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10c160dd0 0x7fc10c0b7400 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.719+0000 7fc1145ff700 1 --2- 172.21.15.97:0/4004721388 >> v2:172.21.15.146:3300/0 conn(0x7fc10c0cd390 0x7fc10c160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.719+0000 7fc1145ff700 1 -- 172.21.15.97:0/4004721388 >> 172.21.15.97:0/4004721388 conn(0x7fc10c071790 msgr2=0x7fc10c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:10.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.719+0000 7fc1145ff700 1 -- 172.21.15.97:0/4004721388 shutdown_connections 2023-07-06T03:54:10.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.719+0000 7fc1145ff700 1 -- 172.21.15.97:0/4004721388 wait complete. 2023-07-06T03:54:10.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.721+0000 7fc1145ff700 1 Processor -- start 2023-07-06T03:54:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.721+0000 7fc1145ff700 1 -- start start 2023-07-06T03:54:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.722+0000 7fc1145ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc10c0cd390 0x7fc10c0cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.722+0000 7fc1145ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc10c15c520 0x7fc10c158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.722+0000 7fc1145ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10c160dd0 0x7fc10c155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.722+0000 7fc1145ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0cb7f0 con 0x7fc10c160dd0 2023-07-06T03:54:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.722+0000 7fc1145ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0cb590 con 0x7fc10c0cd390 2023-07-06T03:54:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.722+0000 7fc1145ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0d2350 con 0x7fc10c15c520 2023-07-06T03:54:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.722+0000 7fc112b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10c160dd0 0x7fc10c155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.722+0000 7fc112b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10c160dd0 0x7fc10c155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40688/0 (socket says 172.21.15.97:40688) 2023-07-06T03:54:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.722+0000 7fc112b9c700 1 -- 172.21.15.97:0/2074280339 learned_addr learned my addr 172.21.15.97:0/2074280339 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.722+0000 7fc111b9a700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc10c15c520 0x7fc10c158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.722+0000 7fc11239b700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc10c0cd390 0x7fc10c0cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.723+0000 7fc112b9c700 1 -- 172.21.15.97:0/2074280339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc10c15c520 msgr2=0x7fc10c158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.723+0000 7fc112b9c700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc10c15c520 0x7fc10c158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.723+0000 7fc112b9c700 1 -- 172.21.15.97:0/2074280339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc10c0cd390 msgr2=0x7fc10c0cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.723+0000 7fc112b9c700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc10c0cd390 0x7fc10c0cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.723+0000 7fc112b9c700 1 -- 172.21.15.97:0/2074280339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104009b30 con 0x7fc10c160dd0 2023-07-06T03:54:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.723+0000 7fc112b9c700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10c160dd0 0x7fc10c155710 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fc10400b370 tx=0x7fc104004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.723+0000 7fc102ffd700 1 -- 172.21.15.97:0/2074280339 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104004880 con 0x7fc10c160dd0 2023-07-06T03:54:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.723+0000 7fc102ffd700 1 -- 172.21.15.97:0/2074280339 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1040099b0 con 0x7fc10c160dd0 2023-07-06T03:54:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.723+0000 7fc102ffd700 1 -- 172.21.15.97:0/2074280339 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10401c680 con 0x7fc10c160dd0 2023-07-06T03:54:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.723+0000 7fc1145ff700 1 -- 172.21.15.97:0/2074280339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c0d2630 con 0x7fc10c160dd0 2023-07-06T03:54:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.723+0000 7fc1145ff700 1 -- 172.21.15.97:0/2074280339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10c0d2bf0 con 0x7fc10c160dd0 2023-07-06T03:54:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.724+0000 7fc1145ff700 1 -- 172.21.15.97:0/2074280339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10c160180 con 0x7fc10c160dd0 2023-07-06T03:54:10.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.725+0000 7fc102ffd700 1 -- 172.21.15.97:0/2074280339 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc10401c7e0 con 0x7fc10c160dd0 2023-07-06T03:54:10.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.726+0000 7fc102ffd700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0f006ace0 0x7fc0f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:10.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.726+0000 7fc102ffd700 1 -- 172.21.15.97:0/2074280339 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7fc10408ae20 con 0x7fc10c160dd0 2023-07-06T03:54:10.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.727+0000 7fc11239b700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0f006ace0 0x7fc0f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:10.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.727+0000 7fc11239b700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0f006ace0 0x7fc0f006d190 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc108007fd0 tx=0x7fc108009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:10.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.731+0000 7fc102ffd700 1 -- 172.21.15.97:0/2074280339 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc104055e40 con 0x7fc10c160dd0 2023-07-06T03:54:10.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.973+0000 7fc1145ff700 1 -- 172.21.15.97:0/2074280339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc10c04e340 con 0x7fc10c160dd0 2023-07-06T03:54:10.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.974+0000 7fc102ffd700 1 -- 172.21.15.97:0/2074280339 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fc104059460 con 0x7fc10c160dd0 2023-07-06T03:54:10.977 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:10.977 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/387292825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":387292825},{"type":"v1","addr":"172.21.15.146:6833","nonce":387292825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690380561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690380561},{"type":"v1","addr":"172.21.15.97:6837","nonce":690380561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3897834271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3897834271},{"type":"v1","addr":"172.21.15.146:6835","nonce":3897834271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:53:26.234190+0000","modified":"2023-07-06T03:54:06.823422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6836/3324055785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3324055785},{"type":"v1","addr":"172.21.15.97:6836","nonce":3324055785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-07-06T03:54:10.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.976+0000 7fc1145ff700 1 -- 172.21.15.97:0/2074280339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0f006ace0 msgr2=0x7fc0f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.976+0000 7fc1145ff700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0f006ace0 0x7fc0f006d190 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc108007fd0 tx=0x7fc108009040 comp rx=0 tx=0).stop 2023-07-06T03:54:10.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.976+0000 7fc1145ff700 1 -- 172.21.15.97:0/2074280339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10c160dd0 msgr2=0x7fc10c155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:10.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.976+0000 7fc1145ff700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10c160dd0 0x7fc10c155710 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fc10400b370 tx=0x7fc104004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.976+0000 7fc1145ff700 1 -- 172.21.15.97:0/2074280339 shutdown_connections 2023-07-06T03:54:10.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.976+0000 7fc1145ff700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc0f006ace0 0x7fc0f006d190 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.976+0000 7fc1145ff700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc10c160dd0 0x7fc10c155710 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.976+0000 7fc1145ff700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc10c0cd390 0x7fc10c0cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.976+0000 7fc1145ff700 1 --2- 172.21.15.97:0/2074280339 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc10c15c520 0x7fc10c158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.976+0000 7fc1145ff700 1 -- 172.21.15.97:0/2074280339 >> 172.21.15.97:0/2074280339 conn(0x7fc10c071790 msgr2=0x7fc10c1605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.977+0000 7fc1145ff700 1 -- 172.21.15.97:0/2074280339 shutdown_connections 2023-07-06T03:54:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:10.977+0000 7fc1145ff700 1 -- 172.21.15.97:0/2074280339 wait complete. 2023-07-06T03:54:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 71 2023-07-06T03:54:10.997 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.189+0000 7f9d211ff700 1 Processor -- start 2023-07-06T03:54:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.190+0000 7f9d211ff700 1 -- start start 2023-07-06T03:54:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d211ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9d1c15c520 0x7f9d1c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d211ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c160180 0x7f9d1c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d211ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d1c160b30 0x7f9d1c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d211ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1c0cd8d0 con 0x7f9d1c160b30 2023-07-06T03:54:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d211ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1c0cdae0 con 0x7f9d1c15c520 2023-07-06T03:54:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d211ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0b6640 con 0x7f9d1c160180 2023-07-06T03:54:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d1b59e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d1c160b30 0x7f9d1c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d1b59e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d1c160b30 0x7f9d1c0cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40700/0 (socket says 172.21.15.97:40700) 2023-07-06T03:54:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d1b59e700 1 -- 172.21.15.97:0/683596150 learned_addr learned my addr 172.21.15.97:0/683596150 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:11.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d1ad9d700 1 --2- 172.21.15.97:0/683596150 >> v2:172.21.15.146:3300/0 conn(0x7f9d1c15c520 0x7f9d1c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d13fff700 1 --2- 172.21.15.97:0/683596150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c160180 0x7f9d1c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d1b59e700 1 -- 172.21.15.97:0/683596150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c160180 msgr2=0x7f9d1c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d1b59e700 1 --2- 172.21.15.97:0/683596150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c160180 0x7f9d1c1605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d1b59e700 1 -- 172.21.15.97:0/683596150 >> v2:172.21.15.146:3300/0 conn(0x7f9d1c15c520 msgr2=0x7f9d1c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d1b59e700 1 --2- 172.21.15.97:0/683596150 >> v2:172.21.15.146:3300/0 conn(0x7f9d1c15c520 0x7f9d1c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.191+0000 7f9d1b59e700 1 -- 172.21.15.97:0/683596150 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c0b6920 con 0x7f9d1c160b30 2023-07-06T03:54:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.192+0000 7f9d1b59e700 1 --2- 172.21.15.97:0/683596150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c160b30 0x7f9d1c0cf050 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0b4c70 tx=0x7f9d1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150f4cb26aabda5a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.192+0000 7f9d1a59c700 1 -- 172.21.15.97:0/683596150 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400e070 con 0x7f9d1c160b30 2023-07-06T03:54:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.192+0000 7f9d1a59c700 1 -- 172.21.15.97:0/683596150 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1400b9c0 con 0x7f9d1c160b30 2023-07-06T03:54:11.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.192+0000 7f9d1a59c700 1 -- 172.21.15.97:0/683596150 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400bd70 con 0x7f9d1c160b30 2023-07-06T03:54:11.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.192+0000 7f9d211ff700 1 -- 172.21.15.97:0/683596150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c160b30 msgr2=0x7f9d1c0cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:11.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.192+0000 7f9d211ff700 1 --2- 172.21.15.97:0/683596150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c160b30 0x7f9d1c0cf050 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0b4c70 tx=0x7f9d1400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:11.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.193+0000 7f9d211ff700 1 -- 172.21.15.97:0/683596150 shutdown_connections 2023-07-06T03:54:11.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.193+0000 7f9d211ff700 1 --2- 172.21.15.97:0/683596150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c160b30 0x7f9d1c0cf050 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.193+0000 7f9d211ff700 1 --2- 172.21.15.97:0/683596150 >> v2:172.21.15.146:3300/0 conn(0x7f9d1c15c520 0x7f9d1c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.193+0000 7f9d211ff700 1 --2- 172.21.15.97:0/683596150 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c160180 0x7f9d1c1605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.193+0000 7f9d211ff700 1 -- 172.21.15.97:0/683596150 >> 172.21.15.97:0/683596150 conn(0x7f9d1c071790 msgr2=0x7f9d1c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:11.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.193+0000 7f9d211ff700 1 -- 172.21.15.97:0/683596150 shutdown_connections 2023-07-06T03:54:11.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.193+0000 7f9d211ff700 1 -- 172.21.15.97:0/683596150 wait complete. 2023-07-06T03:54:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.194+0000 7f9d211ff700 1 Processor -- start 2023-07-06T03:54:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.194+0000 7f9d211ff700 1 -- start start 2023-07-06T03:54:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.195+0000 7f9d211ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c520 0x7f9d1c0d1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.195+0000 7f9d211ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d1c160180 0x7f9d1c15e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.195+0000 7f9d211ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c160b30 0x7f9d1c157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.195+0000 7f9d211ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c15f3b0 con 0x7f9d1c15c520 2023-07-06T03:54:11.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.195+0000 7f9d211ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c15e890 con 0x7f9d1c160180 2023-07-06T03:54:11.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.195+0000 7f9d211ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1577b0 con 0x7f9d1c160b30 2023-07-06T03:54:11.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d1ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c520 0x7f9d1c0d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d1ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c520 0x7f9d1c0d1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40706/0 (socket says 172.21.15.97:40706) 2023-07-06T03:54:11.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d1ad9d700 1 -- 172.21.15.97:0/4043275797 learned_addr learned my addr 172.21.15.97:0/4043275797 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:11.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d13fff700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d1c160180 0x7f9d1c15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d1b59e700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c160b30 0x7f9d1c157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d1ad9d700 1 -- 172.21.15.97:0/4043275797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c160b30 msgr2=0x7f9d1c157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d1ad9d700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c160b30 0x7f9d1c157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d1ad9d700 1 -- 172.21.15.97:0/4043275797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d1c160180 msgr2=0x7f9d1c15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d1ad9d700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d1c160180 0x7f9d1c15e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d1ad9d700 1 -- 172.21.15.97:0/4043275797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d14009b30 con 0x7f9d1c15c520 2023-07-06T03:54:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d13fff700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d1c160180 0x7f9d1c15e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d1ad9d700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c520 0x7f9d1c0d1750 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00e840 tx=0x7f9d0c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:11.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d137fe700 1 -- 172.21.15.97:0/4043275797 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c013610 con 0x7f9d1c15c520 2023-07-06T03:54:11.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d137fe700 1 -- 172.21.15.97:0/4043275797 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c013770 con 0x7f9d1c15c520 2023-07-06T03:54:11.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d137fe700 1 -- 172.21.15.97:0/4043275797 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c013b20 con 0x7f9d1c15c520 2023-07-06T03:54:11.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.196+0000 7f9d211ff700 1 -- 172.21.15.97:0/4043275797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c15eb10 con 0x7f9d1c15c520 2023-07-06T03:54:11.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.197+0000 7f9d211ff700 1 -- 172.21.15.97:0/4043275797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c15f0d0 con 0x7f9d1c15c520 2023-07-06T03:54:11.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.198+0000 7f9d211ff700 1 -- 172.21.15.97:0/4043275797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c04e340 con 0x7f9d1c15c520 2023-07-06T03:54:11.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.198+0000 7f9d137fe700 1 -- 172.21.15.97:0/4043275797 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d0c013c80 con 0x7f9d1c15c520 2023-07-06T03:54:11.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.199+0000 7f9d137fe700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 0x7f9cfc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.199+0000 7f9d137fe700 1 -- 172.21.15.97:0/4043275797 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f9d0c0891e0 con 0x7f9d1c15c520 2023-07-06T03:54:11.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.200+0000 7f9d13fff700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 0x7f9cfc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.200+0000 7f9d13fff700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 0x7f9cfc06d190 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0cd7e0 tx=0x7f9d04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:11.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.203+0000 7f9d137fe700 1 -- 172.21.15.97:0/4043275797 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9d0c054200 con 0x7f9d1c15c520 2023-07-06T03:54:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.400+0000 7f9d211ff700 1 -- 172.21.15.97:0/4043275797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d1c0c6d80 con 0x7f9d1c15c520 2023-07-06T03:54:11.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.402+0000 7f9d137fe700 1 -- 172.21.15.97:0/4043275797 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+18516 (secure 0 0 0) 0x7f9d0c057820 con 0x7f9d1c15c520 2023-07-06T03:54:11.404 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:11.405 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":101,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:53:54.754060+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:53:24.013743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-07-06T03:53:25.222178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":6}]},{"pgid":"16.9","mappings":[{"from":4,"to":3}]},{"pgid":"16.15","mappings":[{"from":4,"to":3}]},{"pgid":"16.24","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 -- 172.21.15.97:0/4043275797 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 msgr2=0x7f9cfc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:11.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 0x7f9cfc06d190 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0cd7e0 tx=0x7f9d04008040 comp rx=0 tx=0).stop 2023-07-06T03:54:11.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 -- 172.21.15.97:0/4043275797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c520 msgr2=0x7f9d1c0d1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:11.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c520 0x7f9d1c0d1750 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00e840 tx=0x7f9d0c00ec00 comp rx=0 tx=0).stop 2023-07-06T03:54:11.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 -- 172.21.15.97:0/4043275797 shutdown_connections 2023-07-06T03:54:11.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9cfc06ace0 0x7f9cfc06d190 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d1c160b30 0x7f9d1c157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d1c15c520 0x7f9d1c0d1750 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 --2- 172.21.15.97:0/4043275797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d1c160180 0x7f9d1c15e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 -- 172.21.15.97:0/4043275797 >> 172.21.15.97:0/4043275797 conn(0x7f9d1c071790 msgr2=0x7f9d1c071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:11.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 -- 172.21.15.97:0/4043275797 shutdown_connections 2023-07-06T03:54:11.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.404+0000 7f9d211ff700 1 -- 172.21.15.97:0/4043275797 wait complete. 2023-07-06T03:54:11.424 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T03:54:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.616+0000 7f06483ff700 1 Processor -- start 2023-07-06T03:54:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.617+0000 7f06483ff700 1 -- start start 2023-07-06T03:54:11.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.618+0000 7f06483ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f064015c610 0x7f06400b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.618+0000 7f06483ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06400d2210 0x7f06400d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.618+0000 7f06483ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06400cff90 con 0x7f064006b360 2023-07-06T03:54:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.618+0000 7f06483ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06400d0780 con 0x7f064015c610 2023-07-06T03:54:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.618+0000 7f06483ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064015c9e0 con 0x7f06400d2210 2023-07-06T03:54:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.618+0000 7f064599a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f064006b360 0x7f06400b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59924/0 (socket says 172.21.15.97:59924) 2023-07-06T03:54:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.618+0000 7f064599a700 1 -- 172.21.15.97:0/2196156710 learned_addr learned my addr 172.21.15.97:0/2196156710 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.618+0000 7f064619b700 1 --2- 172.21.15.97:0/2196156710 >> v2:172.21.15.146:3300/0 conn(0x7f064015c610 0x7f06400b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.618+0000 7f064699c700 1 --2- 172.21.15.97:0/2196156710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06400d2210 0x7f06400d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.619+0000 7f0645199700 1 -- 172.21.15.97:0/2196156710 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622365144 0 0) 0x7f06400cff90 con 0x7f064006b360 2023-07-06T03:54:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.619+0000 7f0645199700 1 -- 172.21.15.97:0/2196156710 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f062c0035b0 con 0x7f064006b360 2023-07-06T03:54:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.619+0000 7f064699c700 1 -- 172.21.15.97:0/2196156710 >> v2:172.21.15.146:3300/0 conn(0x7f064015c610 msgr2=0x7f06400b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.619+0000 7f064699c700 1 --2- 172.21.15.97:0/2196156710 >> v2:172.21.15.146:3300/0 conn(0x7f064015c610 0x7f06400b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.619+0000 7f064699c700 1 -- 172.21.15.97:0/2196156710 >> v1:172.21.15.97:6789/0 conn(0x7f064006b360 legacy=0x7f06400b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.619+0000 7f064699c700 1 -- 172.21.15.97:0/2196156710 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400b5650 con 0x7f06400d2210 2023-07-06T03:54:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.619+0000 7f064619b700 1 --2- 172.21.15.97:0/2196156710 >> v2:172.21.15.146:3300/0 conn(0x7f064015c610 0x7f06400b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:54:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.619+0000 7f064699c700 1 --2- 172.21.15.97:0/2196156710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06400d2210 0x7f06400d1310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0638009d90 tx=0x7f063800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5402c691346b3755 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.619+0000 7f0645199700 1 -- 172.21.15.97:0/2196156710 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063800e070 con 0x7f06400d2210 2023-07-06T03:54:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.619+0000 7f0645199700 1 -- 172.21.15.97:0/2196156710 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063800bb20 con 0x7f06400d2210 2023-07-06T03:54:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.619+0000 7f0645199700 1 -- 172.21.15.97:0/2196156710 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638012430 con 0x7f06400d2210 2023-07-06T03:54:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.620+0000 7f06483ff700 1 -- 172.21.15.97:0/2196156710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06400d2210 msgr2=0x7f06400d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.620+0000 7f06483ff700 1 --2- 172.21.15.97:0/2196156710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06400d2210 0x7f06400d1310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0638009d90 tx=0x7f063800b330 comp rx=0 tx=0).stop 2023-07-06T03:54:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.620+0000 7f06483ff700 1 -- 172.21.15.97:0/2196156710 shutdown_connections 2023-07-06T03:54:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.620+0000 7f06483ff700 1 --2- 172.21.15.97:0/2196156710 >> v2:172.21.15.146:3300/0 conn(0x7f064015c610 0x7f06400b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.620+0000 7f06483ff700 1 --2- 172.21.15.97:0/2196156710 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f06400d2210 0x7f06400d1310 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.620+0000 7f06483ff700 1 -- 172.21.15.97:0/2196156710 >> 172.21.15.97:0/2196156710 conn(0x7f064006a6c0 msgr2=0x7f064006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.620+0000 7f06483ff700 1 -- 172.21.15.97:0/2196156710 shutdown_connections 2023-07-06T03:54:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.620+0000 7f06483ff700 1 -- 172.21.15.97:0/2196156710 wait complete. 2023-07-06T03:54:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.622+0000 7f06483ff700 1 Processor -- start 2023-07-06T03:54:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.622+0000 7f06483ff700 1 -- start start 2023-07-06T03:54:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.623+0000 7f06483ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064006b360 0x7f06400caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.623+0000 7f06483ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06400d2210 0x7f06400cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.623+0000 7f06483ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064015c610 0x7f06400cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.623+0000 7f06483ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064015d9d0 con 0x7f064015c610 2023-07-06T03:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.623+0000 7f06483ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064015cb20 con 0x7f06400d2210 2023-07-06T03:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.623+0000 7f06483ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064015cdc0 con 0x7f064006b360 2023-07-06T03:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.623+0000 7f064699c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064015c610 0x7f06400cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.623+0000 7f064699c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064015c610 0x7f06400cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40718/0 (socket says 172.21.15.97:40718) 2023-07-06T03:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.623+0000 7f064699c700 1 -- 172.21.15.97:0/1872304334 learned_addr learned my addr 172.21.15.97:0/1872304334 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f064599a700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06400d2210 0x7f06400cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f064619b700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064006b360 0x7f06400caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f064699c700 1 -- 172.21.15.97:0/1872304334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064006b360 msgr2=0x7f06400caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f064699c700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064006b360 0x7f06400caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f064699c700 1 -- 172.21.15.97:0/1872304334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06400d2210 msgr2=0x7f06400cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f064699c700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06400d2210 0x7f06400cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f064699c700 1 -- 172.21.15.97:0/1872304334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063c0094a0 con 0x7f064015c610 2023-07-06T03:54:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f064619b700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064006b360 0x7f06400caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:11.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f064599a700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06400d2210 0x7f06400cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f064699c700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064015c610 0x7f06400cbc30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f0638014040 tx=0x7f0638004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f0636ffd700 1 -- 172.21.15.97:0/1872304334 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380129f0 con 0x7f064015c610 2023-07-06T03:54:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f0636ffd700 1 -- 172.21.15.97:0/1872304334 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0638020070 con 0x7f064015c610 2023-07-06T03:54:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f06483ff700 1 -- 172.21.15.97:0/1872304334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0638009a70 con 0x7f064015c610 2023-07-06T03:54:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f0636ffd700 1 -- 172.21.15.97:0/1872304334 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638012db0 con 0x7f064015c610 2023-07-06T03:54:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.624+0000 7f06483ff700 1 -- 172.21.15.97:0/1872304334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064015d380 con 0x7f064015c610 2023-07-06T03:54:11.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.625+0000 7f06483ff700 1 -- 172.21.15.97:0/1872304334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064004e340 con 0x7f064015c610 2023-07-06T03:54:11.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.627+0000 7f0636ffd700 1 -- 172.21.15.97:0/1872304334 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0638004040 con 0x7f064015c610 2023-07-06T03:54:11.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.628+0000 7f0636ffd700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 0x7f062406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:11.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.628+0000 7f0636ffd700 1 -- 172.21.15.97:0/1872304334 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f0638022070 con 0x7f064015c610 2023-07-06T03:54:11.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.628+0000 7f064619b700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 0x7f062406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:11.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.628+0000 7f064619b700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 0x7f062406d190 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f063c009c50 tx=0x7f063c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:11.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.631+0000 7f0636ffd700 1 -- 172.21.15.97:0/1872304334 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f06380568f0 con 0x7f064015c610 2023-07-06T03:54:11.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.874+0000 7f06483ff700 1 -- 172.21.15.97:0/1872304334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f064015d720 con 0x7f064015c610 2023-07-06T03:54:11.877 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-07-06T03:54:11.874+0000 7fb463ff4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T03:54:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.878+0000 7f0636ffd700 1 -- 172.21.15.97:0/1872304334 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7f063802c090 con 0x7f064015c610 2023-07-06T03:54:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 -- 172.21.15.97:0/1872304334 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 msgr2=0x7f062406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 0x7f062406d190 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f063c009c50 tx=0x7f063c008040 comp rx=0 tx=0).stop 2023-07-06T03:54:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 -- 172.21.15.97:0/1872304334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064015c610 msgr2=0x7f06400cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064015c610 0x7f06400cbc30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f0638014040 tx=0x7f0638004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 -- 172.21.15.97:0/1872304334 shutdown_connections 2023-07-06T03:54:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f062406ace0 0x7f062406d190 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064015c610 0x7f06400cbc30 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f064006b360 0x7f06400caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 --2- 172.21.15.97:0/1872304334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f06400d2210 0x7f06400cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 -- 172.21.15.97:0/1872304334 >> 172.21.15.97:0/1872304334 conn(0x7f064006a6c0 msgr2=0x7f06400b7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 -- 172.21.15.97:0/1872304334 shutdown_connections 2023-07-06T03:54:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:11.881+0000 7f06483ff700 1 -- 172.21.15.97:0/1872304334 wait complete. 2023-07-06T03:54:11.886 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T03:54:11.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T03:54:11.951 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:54:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.100+0000 7f2e48bff700 1 Processor -- start 2023-07-06T03:54:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.100+0000 7f2e48bff700 1 -- start start 2023-07-06T03:54:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.101+0000 7f2e48bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e4415c1c0 0x7f2e440cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.101+0000 7f2e48bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2e4415c590 0x7f2e4415d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.101+0000 7f2e48bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e440719c0 0x7f2e4406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.101+0000 7f2e48bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e440ccc30 con 0x7f2e440719c0 2023-07-06T03:54:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.101+0000 7f2e48bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e440ccf10 con 0x7f2e4415c590 2023-07-06T03:54:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.101+0000 7f2e48bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440cd1f0 con 0x7f2e4415c1c0 2023-07-06T03:54:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e42d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e440719c0 0x7f2e4406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e42d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e440719c0 0x7f2e4406a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40738/0 (socket says 172.21.15.97:40738) 2023-07-06T03:54:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e42d9d700 1 -- 172.21.15.97:0/145877804 learned_addr learned my addr 172.21.15.97:0/145877804 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e4259c700 1 --2- 172.21.15.97:0/145877804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e4415c1c0 0x7f2e440cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e41d9b700 1 --2- 172.21.15.97:0/145877804 >> v2:172.21.15.146:3300/0 conn(0x7f2e4415c590 0x7f2e4415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e42d9d700 1 -- 172.21.15.97:0/145877804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e4415c1c0 msgr2=0x7f2e440cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e42d9d700 1 --2- 172.21.15.97:0/145877804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e4415c1c0 0x7f2e440cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e42d9d700 1 -- 172.21.15.97:0/145877804 >> v2:172.21.15.146:3300/0 conn(0x7f2e4415c590 msgr2=0x7f2e4415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e42d9d700 1 --2- 172.21.15.97:0/145877804 >> v2:172.21.15.146:3300/0 conn(0x7f2e4415c590 0x7f2e4415d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e42d9d700 1 -- 172.21.15.97:0/145877804 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4415cb00 con 0x7f2e440719c0 2023-07-06T03:54:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e4259c700 1 --2- 172.21.15.97:0/145877804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e4415c1c0 0x7f2e440cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e41d9b700 1 --2- 172.21.15.97:0/145877804 >> v2:172.21.15.146:3300/0 conn(0x7f2e4415c590 0x7f2e4415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.102+0000 7f2e42d9d700 1 --2- 172.21.15.97:0/145877804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e440719c0 0x7f2e4406a060 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2e440cd4b0 tx=0x7f2e3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b896f0be9b04a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e4159a700 1 -- 172.21.15.97:0/145877804 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00e070 con 0x7f2e440719c0 2023-07-06T03:54:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e4159a700 1 -- 172.21.15.97:0/145877804 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c00b9c0 con 0x7f2e440719c0 2023-07-06T03:54:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e4159a700 1 -- 172.21.15.97:0/145877804 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00bd70 con 0x7f2e440719c0 2023-07-06T03:54:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e48bff700 1 -- 172.21.15.97:0/145877804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e440719c0 msgr2=0x7f2e4406a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e48bff700 1 --2- 172.21.15.97:0/145877804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e440719c0 0x7f2e4406a060 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2e440cd4b0 tx=0x7f2e3c00b330 comp rx=0 tx=0).stop 2023-07-06T03:54:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e48bff700 1 -- 172.21.15.97:0/145877804 shutdown_connections 2023-07-06T03:54:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e48bff700 1 --2- 172.21.15.97:0/145877804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e440719c0 0x7f2e4406a060 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e48bff700 1 --2- 172.21.15.97:0/145877804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e4415c1c0 0x7f2e440cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e48bff700 1 --2- 172.21.15.97:0/145877804 >> v2:172.21.15.146:3300/0 conn(0x7f2e4415c590 0x7f2e4415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e48bff700 1 -- 172.21.15.97:0/145877804 >> 172.21.15.97:0/145877804 conn(0x7f2e440b1650 msgr2=0x7f2e440b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e48bff700 1 -- 172.21.15.97:0/145877804 shutdown_connections 2023-07-06T03:54:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.103+0000 7f2e48bff700 1 -- 172.21.15.97:0/145877804 wait complete. 2023-07-06T03:54:12.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.105+0000 7f2e48bff700 1 Processor -- start 2023-07-06T03:54:12.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.105+0000 7f2e48bff700 1 -- start start 2023-07-06T03:54:12.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.106+0000 7f2e48bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440719c0 0x7f2e4415a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:12.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.106+0000 7f2e48bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e4415c1c0 0x7f2e440b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:12.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.106+0000 7f2e48bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e4415c590 0x7f2e440c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:12.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.106+0000 7f2e48bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440c54c0 con 0x7f2e4415c590 2023-07-06T03:54:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.106+0000 7f2e48bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4415a2e0 con 0x7f2e440719c0 2023-07-06T03:54:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.106+0000 7f2e48bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4415a000 con 0x7f2e4415c1c0 2023-07-06T03:54:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.106+0000 7f2e42d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e4415c590 0x7f2e440c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.106+0000 7f2e42d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e4415c590 0x7f2e440c6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40748/0 (socket says 172.21.15.97:40748) 2023-07-06T03:54:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.106+0000 7f2e42d9d700 1 -- 172.21.15.97:0/2822861462 learned_addr learned my addr 172.21.15.97:0/2822861462 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.106+0000 7f2e4259c700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440719c0 0x7f2e4415a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.106+0000 7f2e41d9b700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e4415c1c0 0x7f2e440b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.107+0000 7f2e42d9d700 1 -- 172.21.15.97:0/2822861462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e4415c1c0 msgr2=0x7f2e440b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.107+0000 7f2e42d9d700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e4415c1c0 0x7f2e440b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.107+0000 7f2e42d9d700 1 -- 172.21.15.97:0/2822861462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440719c0 msgr2=0x7f2e4415a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.107+0000 7f2e42d9d700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440719c0 0x7f2e4415a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.107+0000 7f2e42d9d700 1 -- 172.21.15.97:0/2822861462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c009b30 con 0x7f2e4415c590 2023-07-06T03:54:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.107+0000 7f2e42d9d700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e4415c590 0x7f2e440c6ea0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2e44072500 tx=0x7f2e3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:12.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.107+0000 7f2e32ffd700 1 -- 172.21.15.97:0/2822861462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0099b0 con 0x7f2e4415c590 2023-07-06T03:54:12.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.107+0000 7f2e32ffd700 1 -- 172.21.15.97:0/2822861462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c004250 con 0x7f2e4415c590 2023-07-06T03:54:12.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.107+0000 7f2e32ffd700 1 -- 172.21.15.97:0/2822861462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c004600 con 0x7f2e4415c590 2023-07-06T03:54:12.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.107+0000 7f2e48bff700 1 -- 172.21.15.97:0/2822861462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e440c4f00 con 0x7f2e4415c590 2023-07-06T03:54:12.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.108+0000 7f2e48bff700 1 -- 172.21.15.97:0/2822861462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e440b4550 con 0x7f2e4415c590 2023-07-06T03:54:12.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.108+0000 7f2e48bff700 1 -- 172.21.15.97:0/2822861462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e440b43b0 con 0x7f2e4415c590 2023-07-06T03:54:12.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.109+0000 7f2e32ffd700 1 -- 172.21.15.97:0/2822861462 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2e3c004760 con 0x7f2e4415c590 2023-07-06T03:54:12.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.110+0000 7f2e32ffd700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e2406ace0 0x7f2e2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.110+0000 7f2e32ffd700 1 -- 172.21.15.97:0/2822861462 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f2e3c08a910 con 0x7f2e4415c590 2023-07-06T03:54:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.110+0000 7f2e4259c700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e2406ace0 0x7f2e2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.111+0000 7f2e4259c700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e2406ace0 0x7f2e2406d190 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2e440c5620 tx=0x7f2e34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:12.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.114+0000 7f2e32ffd700 1 -- 172.21.15.97:0/2822861462 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2e440b43b0 con 0x7f2e4415c590 2023-07-06T03:54:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.349+0000 7f2e48bff700 1 -- 172.21.15.97:0/2822861462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2e440b43b0 con 0x7f2e4415c590 2023-07-06T03:54:12.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.879+0000 7f2e32ffd700 1 -- 172.21.15.97:0/2822861462 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2e440b43b0 con 0x7f2e4415c590 2023-07-06T03:54:12.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 -- 172.21.15.97:0/2822861462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e2406ace0 msgr2=0x7f2e2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:12.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e2406ace0 0x7f2e2406d190 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2e440c5620 tx=0x7f2e34008040 comp rx=0 tx=0).stop 2023-07-06T03:54:12.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 -- 172.21.15.97:0/2822861462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e4415c590 msgr2=0x7f2e440c6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:12.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e4415c590 0x7f2e440c6ea0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2e44072500 tx=0x7f2e3c004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 -- 172.21.15.97:0/2822861462 shutdown_connections 2023-07-06T03:54:12.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2e2406ace0 0x7f2e2406d190 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e4415c590 0x7f2e440c6ea0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e440719c0 0x7f2e4415a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 --2- 172.21.15.97:0/2822861462 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2e4415c1c0 0x7f2e440b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:12.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 -- 172.21.15.97:0/2822861462 >> 172.21.15.97:0/2822861462 conn(0x7f2e440b1650 msgr2=0x7f2e440c5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:12.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 -- 172.21.15.97:0/2822861462 shutdown_connections 2023-07-06T03:54:12.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:12.883+0000 7f2e48bff700 1 -- 172.21.15.97:0/2822861462 wait complete. 2023-07-06T03:54:12.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T03:54:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.100+0000 7fef0bfff700 1 Processor -- start 2023-07-06T03:54:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.101+0000 7fef0bfff700 1 -- start start 2023-07-06T03:54:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef0c15c3d0 0x7fef0c0d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fef0c0d1fd0 0x7fef0c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0bfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0c0d10d0 con 0x7fef0c0cedb0 2023-07-06T03:54:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0bfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0c0cd1e0 con 0x7fef0c0d1fd0 2023-07-06T03:54:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0bfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0cd450 con 0x7fef0c15c3d0 2023-07-06T03:54:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fef0c0d1fd0 0x7fef0c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fef0c0d1fd0 0x7fef0c0cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40760/0 (socket says 172.21.15.97:40760) 2023-07-06T03:54:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0b7fe700 1 -- 172.21.15.97:0/4023844631 learned_addr learned my addr 172.21.15.97:0/4023844631 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0a7fc700 1 --1- 172.21.15.97:0/4023844631 >> v1:172.21.15.97:6789/0 conn(0x7fef0c0cedb0 0x7fef0c0b5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59946/0 (socket says 172.21.15.97:59946) 2023-07-06T03:54:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0affd700 1 --2- 172.21.15.97:0/4023844631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef0c15c3d0 0x7fef0c0d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0b7fe700 1 -- 172.21.15.97:0/4023844631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef0c15c3d0 msgr2=0x7fef0c0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.102+0000 7fef0b7fe700 1 --2- 172.21.15.97:0/4023844631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef0c15c3d0 0x7fef0c0d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.103+0000 7fef0b7fe700 1 -- 172.21.15.97:0/4023844631 >> v1:172.21.15.97:6789/0 conn(0x7fef0c0cedb0 legacy=0x7fef0c0b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.103+0000 7fef0b7fe700 1 -- 172.21.15.97:0/4023844631 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c0cd730 con 0x7fef0c0d1fd0 2023-07-06T03:54:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.103+0000 7fef09ffb700 1 -- 172.21.15.97:0/4023844631 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288775389 0 0) 0x7fef0c0d10d0 con 0x7fef0c0cedb0 2023-07-06T03:54:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.103+0000 7fef0b7fe700 1 --2- 172.21.15.97:0/4023844631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0c0d1fd0 0x7fef0c0cf500 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fef04009e50 tx=0x7fef0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816ca4f481d31c7a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.103+0000 7fef09ffb700 1 -- 172.21.15.97:0/4023844631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0400e070 con 0x7fef0c0d1fd0 2023-07-06T03:54:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.103+0000 7fef09ffb700 1 -- 172.21.15.97:0/4023844631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0400b9c0 con 0x7fef0c0d1fd0 2023-07-06T03:54:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.103+0000 7fef09ffb700 1 -- 172.21.15.97:0/4023844631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0400bd70 con 0x7fef0c0d1fd0 2023-07-06T03:54:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.104+0000 7fef0bfff700 1 -- 172.21.15.97:0/4023844631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0c0d1fd0 msgr2=0x7fef0c0cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.104+0000 7fef0bfff700 1 --2- 172.21.15.97:0/4023844631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0c0d1fd0 0x7fef0c0cf500 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fef04009e50 tx=0x7fef0400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.104+0000 7fef0bfff700 1 -- 172.21.15.97:0/4023844631 shutdown_connections 2023-07-06T03:54:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.104+0000 7fef0bfff700 1 --2- 172.21.15.97:0/4023844631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0c0d1fd0 0x7fef0c0cf500 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.104+0000 7fef0bfff700 1 --2- 172.21.15.97:0/4023844631 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef0c15c3d0 0x7fef0c0d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.104+0000 7fef0bfff700 1 -- 172.21.15.97:0/4023844631 >> 172.21.15.97:0/4023844631 conn(0x7fef0c06bf70 msgr2=0x7fef0c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.104+0000 7fef0bfff700 1 -- 172.21.15.97:0/4023844631 shutdown_connections 2023-07-06T03:54:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.104+0000 7fef0bfff700 1 -- 172.21.15.97:0/4023844631 wait complete. 2023-07-06T03:54:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.105+0000 7fef0bfff700 1 Processor -- start 2023-07-06T03:54:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.105+0000 7fef0bfff700 1 -- start start 2023-07-06T03:54:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.106+0000 7fef0bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef0c0cedb0 0x7fef0c14c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.106+0000 7fef0bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef0c0d1fd0 0x7fef0c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.106+0000 7fef0bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0c15f4e0 0x7fef0c0b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.106+0000 7fef0bfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c14d320 con 0x7fef0c15f4e0 2023-07-06T03:54:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.106+0000 7fef0bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0b2910 con 0x7fef0c0d1fd0 2023-07-06T03:54:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.106+0000 7fef0bfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0b2bf0 con 0x7fef0c0cedb0 2023-07-06T03:54:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.107+0000 7fef0b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0c15f4e0 0x7fef0c0b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.107+0000 7fef0affd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef0c0cedb0 0x7fef0c14c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.107+0000 7fef0b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0c15f4e0 0x7fef0c0b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40764/0 (socket says 172.21.15.97:40764) 2023-07-06T03:54:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.107+0000 7fef0b7fe700 1 -- 172.21.15.97:0/3441758347 learned_addr learned my addr 172.21.15.97:0/3441758347 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.107+0000 7fef0a7fc700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef0c0d1fd0 0x7fef0c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.107+0000 7fef0b7fe700 1 -- 172.21.15.97:0/3441758347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef0c0cedb0 msgr2=0x7fef0c14c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.107+0000 7fef0b7fe700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef0c0cedb0 0x7fef0c14c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.107+0000 7fef0b7fe700 1 -- 172.21.15.97:0/3441758347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef0c0d1fd0 msgr2=0x7fef0c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.107+0000 7fef0b7fe700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef0c0d1fd0 0x7fef0c15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.107+0000 7fef0b7fe700 1 -- 172.21.15.97:0/3441758347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef04009b30 con 0x7fef0c15f4e0 2023-07-06T03:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.107+0000 7fef0b7fe700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0c15f4e0 0x7fef0c0b6b20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0b56e0 tx=0x7fef04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.108+0000 7feef37fe700 1 -- 172.21.15.97:0/3441758347 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04003ea0 con 0x7fef0c15f4e0 2023-07-06T03:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.108+0000 7feef37fe700 1 -- 172.21.15.97:0/3441758347 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef040048e0 con 0x7fef0c15f4e0 2023-07-06T03:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.108+0000 7feef37fe700 1 -- 172.21.15.97:0/3441758347 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0401c680 con 0x7fef0c15f4e0 2023-07-06T03:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.108+0000 7fef0bfff700 1 -- 172.21.15.97:0/3441758347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0c0b2ed0 con 0x7fef0c15f4e0 2023-07-06T03:54:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.108+0000 7fef0bfff700 1 -- 172.21.15.97:0/3441758347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c0b3490 con 0x7fef0c15f4e0 2023-07-06T03:54:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.109+0000 7fef0bfff700 1 -- 172.21.15.97:0/3441758347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0c0b3650 con 0x7fef0c15f4e0 2023-07-06T03:54:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.110+0000 7feef37fe700 1 -- 172.21.15.97:0/3441758347 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef0401c7e0 con 0x7fef0c15f4e0 2023-07-06T03:54:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.111+0000 7feef37fe700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 0x7feef806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.111+0000 7feef37fe700 1 -- 172.21.15.97:0/3441758347 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fef0408b170 con 0x7fef0c15f4e0 2023-07-06T03:54:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.111+0000 7fef0affd700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 0x7feef806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.112+0000 7fef0affd700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 0x7feef806d190 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feefc006b20 tx=0x7feefc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:13.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.116+0000 7feef37fe700 1 -- 172.21.15.97:0/3441758347 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fef04056140 con 0x7fef0c15f4e0 2023-07-06T03:54:13.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.315+0000 7fef0bfff700 1 -- 172.21.15.97:0/3441758347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fef0c04e340 con 0x7fef0c15f4e0 2023-07-06T03:54:13.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.884+0000 7feef37fe700 1 -- 172.21.15.97:0/3441758347 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) v1 ==== 181+0+0 (secure 0 0 0) 0x7fef04059760 con 0x7fef0c15f4e0 2023-07-06T03:54:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 -- 172.21.15.97:0/3441758347 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 msgr2=0x7feef806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 0x7feef806d190 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feefc006b20 tx=0x7feefc008040 comp rx=0 tx=0).stop 2023-07-06T03:54:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 -- 172.21.15.97:0/3441758347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0c15f4e0 msgr2=0x7fef0c0b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0c15f4e0 0x7fef0c0b6b20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0b56e0 tx=0x7fef04004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 -- 172.21.15.97:0/3441758347 shutdown_connections 2023-07-06T03:54:13.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7feef806ace0 0x7feef806d190 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0c15f4e0 0x7fef0c0b6b20 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fef0c0cedb0 0x7fef0c14c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 --2- 172.21.15.97:0/3441758347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fef0c0d1fd0 0x7fef0c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 -- 172.21.15.97:0/3441758347 >> 172.21.15.97:0/3441758347 conn(0x7fef0c06bf70 msgr2=0x7fef0c158e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 -- 172.21.15.97:0/3441758347 shutdown_connections 2023-07-06T03:54:13.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:13.887+0000 7fef0bfff700 1 -- 172.21.15.97:0/3441758347 wait complete. 2023-07-06T03:54:13.892 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-07-06T03:54:13.909 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T03:54:14.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.112+0000 7f3e591ff700 1 Processor -- start 2023-07-06T03:54:14.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.112+0000 7f3e591ff700 1 -- start start 2023-07-06T03:54:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.114+0000 7f3e591ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e54160f10 0x7f3e540ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:14.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.114+0000 7f3e591ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5415dbd0 con 0x7f3e54160b40 2023-07-06T03:54:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.114+0000 7f3e591ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e540cd140 con 0x7f3e540cb720 2023-07-06T03:54:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.114+0000 7f3e591ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e540cedf0 con 0x7f3e54160f10 2023-07-06T03:54:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.114+0000 7f3e52d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3e54160b40 0x7f3e540cb010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59962/0 (socket says 172.21.15.97:59962) 2023-07-06T03:54:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.114+0000 7f3e52d9d700 1 -- 172.21.15.97:0/3487634693 learned_addr learned my addr 172.21.15.97:0/3487634693 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.114+0000 7f3e5359e700 1 --2- 172.21.15.97:0/3487634693 >> v2:172.21.15.146:3300/0 conn(0x7f3e54160f10 0x7f3e540ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.114+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083125321 0 0) 0x7f3e5415dbd0 con 0x7f3e54160b40 2023-07-06T03:54:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e400035b0 con 0x7f3e54160b40 2023-07-06T03:54:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1544196484 0 0) 0x7f3e400035b0 con 0x7f3e54160b40 2023-07-06T03:54:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 >> v2:172.21.15.146:3300/0 conn(0x7f3e54160f10 msgr2=0x7f3e540ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 --2- 172.21.15.97:0/3487634693 >> v2:172.21.15.146:3300/0 conn(0x7f3e54160f10 0x7f3e540ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 >> v1:172.21.15.146:6789/0 conn(0x7f3e540cb720 legacy=0x7f3e540ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540cd3e0 con 0x7f3e54160b40 2023-07-06T03:54:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f3e440019f0 con 0x7f3e54160b40 2023-07-06T03:54:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 >> v1:172.21.15.97:6789/0 conn(0x7f3e54160b40 legacy=0x7f3e540cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 --2- 172.21.15.97:0/3487634693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e40003a00 0x7f3e40003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 --2- 172.21.15.97:0/3487634693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 0x7f3e40004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 --2- 172.21.15.97:0/3487634693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e40005200 0x7f3e4000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540cedf0 con 0x7f3e40004850 2023-07-06T03:54:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4000c080 con 0x7f3e40003a00 2023-07-06T03:54:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4000c240 con 0x7f3e40005200 2023-07-06T03:54:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e4a59c700 1 --2- 172.21.15.97:0/3487634693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 0x7f3e40004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e52d9d700 1 --2- 172.21.15.97:0/3487634693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e40003a00 0x7f3e40003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.115+0000 7f3e5359e700 1 --2- 172.21.15.97:0/3487634693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e40005200 0x7f3e4000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.116+0000 7f3e4a59c700 1 -- 172.21.15.97:0/3487634693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e40005200 msgr2=0x7f3e4000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.116+0000 7f3e4a59c700 1 --2- 172.21.15.97:0/3487634693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e40005200 0x7f3e4000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.116+0000 7f3e4a59c700 1 -- 172.21.15.97:0/3487634693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e40003a00 msgr2=0x7f3e40003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.116+0000 7f3e4a59c700 1 --2- 172.21.15.97:0/3487634693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e40003a00 0x7f3e40003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.116+0000 7f3e4a59c700 1 -- 172.21.15.97:0/3487634693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e400035b0 con 0x7f3e40004850 2023-07-06T03:54:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.116+0000 7f3e5359e700 1 -- 172.21.15.97:0/3487634693 reap_dead start 2023-07-06T03:54:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.116+0000 7f3e4a59c700 1 --2- 172.21.15.97:0/3487634693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 0x7f3e40004cc0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f3e5406b7b0 tx=0x7f3e40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdb8807641186e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.116+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0052d0 con 0x7f3e40004850 2023-07-06T03:54:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.116+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3c005430 con 0x7f3e40004850 2023-07-06T03:54:14.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.116+0000 7f3e5259c700 1 -- 172.21.15.97:0/3487634693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0038c0 con 0x7f3e40004850 2023-07-06T03:54:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.117+0000 7f3e591ff700 1 -- 172.21.15.97:0/3487634693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 msgr2=0x7f3e40004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.117+0000 7f3e591ff700 1 --2- 172.21.15.97:0/3487634693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 0x7f3e40004cc0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f3e5406b7b0 tx=0x7f3e40005880 comp rx=0 tx=0).stop 2023-07-06T03:54:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.117+0000 7f3e591ff700 1 -- 172.21.15.97:0/3487634693 shutdown_connections 2023-07-06T03:54:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.117+0000 7f3e591ff700 1 --2- 172.21.15.97:0/3487634693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 0x7f3e40004cc0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.117+0000 7f3e591ff700 1 -- 172.21.15.97:0/3487634693 >> 172.21.15.97:0/3487634693 conn(0x7f3e540cbc20 msgr2=0x7f3e540cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.117+0000 7f3e591ff700 1 -- 172.21.15.97:0/3487634693 shutdown_connections 2023-07-06T03:54:14.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.117+0000 7f3e591ff700 1 -- 172.21.15.97:0/3487634693 wait complete. 2023-07-06T03:54:14.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.119+0000 7f3e591ff700 1 Processor -- start 2023-07-06T03:54:14.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.119+0000 7f3e591ff700 1 -- start start 2023-07-06T03:54:14.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e591ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 0x7f3e54157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:14.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e591ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e540b2a60 0x7f3e540b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:14.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e591ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e540b2ed0 0x7f3e540b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:14.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e591ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415dc20 con 0x7f3e40004850 2023-07-06T03:54:14.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e591ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540b4fe0 con 0x7f3e540b2a60 2023-07-06T03:54:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e591ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415d5a0 con 0x7f3e540b2ed0 2023-07-06T03:54:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e52d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 0x7f3e54157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e52d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 0x7f3e54157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40796/0 (socket says 172.21.15.97:40796) 2023-07-06T03:54:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e52d9d700 1 -- 172.21.15.97:0/1269554219 learned_addr learned my addr 172.21.15.97:0/1269554219 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e4a59c700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e540b2a60 0x7f3e540b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e5359e700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e540b2ed0 0x7f3e540b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e52d9d700 1 -- 172.21.15.97:0/1269554219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e540b2ed0 msgr2=0x7f3e540b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:14.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e52d9d700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e540b2ed0 0x7f3e540b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e52d9d700 1 -- 172.21.15.97:0/1269554219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e540b2a60 msgr2=0x7f3e540b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e52d9d700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e540b2a60 0x7f3e540b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e52d9d700 1 -- 172.21.15.97:0/1269554219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e541577d0 con 0x7f3e40004850 2023-07-06T03:54:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.120+0000 7f3e4a59c700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e540b2a60 0x7f3e540b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.121+0000 7f3e52d9d700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 0x7f3e54157290 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f3e44001ac0 tx=0x7f3e4400a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.121+0000 7f3e4bfff700 1 -- 172.21.15.97:0/1269554219 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440133f0 con 0x7f3e40004850 2023-07-06T03:54:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.121+0000 7f3e4bfff700 1 -- 172.21.15.97:0/1269554219 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e44013550 con 0x7f3e40004850 2023-07-06T03:54:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.121+0000 7f3e591ff700 1 -- 172.21.15.97:0/1269554219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e540b24b0 con 0x7f3e40004850 2023-07-06T03:54:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.121+0000 7f3e4bfff700 1 -- 172.21.15.97:0/1269554219 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e44013900 con 0x7f3e40004850 2023-07-06T03:54:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.121+0000 7f3e591ff700 1 -- 172.21.15.97:0/1269554219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e540b2790 con 0x7f3e40004850 2023-07-06T03:54:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.122+0000 7f3e591ff700 1 -- 172.21.15.97:0/1269554219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5415de90 con 0x7f3e40004850 2023-07-06T03:54:14.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.123+0000 7f3e4bfff700 1 -- 172.21.15.97:0/1269554219 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e44013a60 con 0x7f3e40004850 2023-07-06T03:54:14.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.124+0000 7f3e4bfff700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e3406ace0 0x7f3e3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:14.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.124+0000 7f3e4bfff700 1 -- 172.21.15.97:0/1269554219 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5826+0+0 (secure 0 0 0) 0x7f3e4408a1f0 con 0x7f3e40004850 2023-07-06T03:54:14.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.124+0000 7f3e4a59c700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e3406ace0 0x7f3e3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:14.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.125+0000 7f3e4a59c700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e3406ace0 0x7f3e3406d190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3e540b5340 tx=0x7f3e3c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:14.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.129+0000 7f3e4bfff700 1 -- 172.21.15.97:0/1269554219 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3e44055410 con 0x7f3e40004850 2023-07-06T03:54:14.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.323+0000 7f3e591ff700 1 -- 172.21.15.97:0/1269554219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3e5404e340 con 0x7f3e40004850 2023-07-06T03:54:14.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.890+0000 7f3e4bfff700 1 -- 172.21.15.97:0/1269554219 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3e440170d0 con 0x7f3e40004850 2023-07-06T03:54:14.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.893+0000 7f3e591ff700 1 -- 172.21.15.97:0/1269554219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e3406ace0 msgr2=0x7f3e3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.893+0000 7f3e591ff700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e3406ace0 0x7f3e3406d190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3e540b5340 tx=0x7f3e3c009030 comp rx=0 tx=0).stop 2023-07-06T03:54:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.894+0000 7f3e591ff700 1 -- 172.21.15.97:0/1269554219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 msgr2=0x7f3e54157290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.894+0000 7f3e591ff700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 0x7f3e54157290 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f3e44001ac0 tx=0x7f3e4400a6a0 comp rx=0 tx=0).stop 2023-07-06T03:54:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.894+0000 7f3e591ff700 1 -- 172.21.15.97:0/1269554219 shutdown_connections 2023-07-06T03:54:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.894+0000 7f3e591ff700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e3406ace0 0x7f3e3406d190 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.894+0000 7f3e591ff700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e540b2ed0 0x7f3e540b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.894+0000 7f3e591ff700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e40004850 0x7f3e54157290 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.894+0000 7f3e591ff700 1 --2- 172.21.15.97:0/1269554219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e540b2a60 0x7f3e540b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.894+0000 7f3e591ff700 1 -- 172.21.15.97:0/1269554219 >> 172.21.15.97:0/1269554219 conn(0x7f3e540cbc20 msgr2=0x7f3e540ad740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.894+0000 7f3e591ff700 1 -- 172.21.15.97:0/1269554219 shutdown_connections 2023-07-06T03:54:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:14.894+0000 7f3e591ff700 1 -- 172.21.15.97:0/1269554219 wait complete. 2023-07-06T03:54:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-07-06T03:54:14.917 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-07-06T03:54:15.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.110+0000 7f8d723ff700 1 Processor -- start 2023-07-06T03:54:15.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.111+0000 7f8d723ff700 1 -- start start 2023-07-06T03:54:15.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.112+0000 7f8d723ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d6c15c610 0x7f8d6c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.112+0000 7f8d723ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8d6c06b360 0x7f8d6c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.112+0000 7f8d723ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d6c0cff90 con 0x7f8d6c0d2210 2023-07-06T03:54:15.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.112+0000 7f8d723ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d6c0d0780 con 0x7f8d6c06b360 2023-07-06T03:54:15.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.112+0000 7f8d723ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c15c9e0 con 0x7f8d6c15c610 2023-07-06T03:54:15.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.112+0000 7f8d7099c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8d6c0d2210 0x7f8d6c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59976/0 (socket says 172.21.15.97:59976) 2023-07-06T03:54:15.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.112+0000 7f8d7099c700 1 -- 172.21.15.97:0/1965528611 learned_addr learned my addr 172.21.15.97:0/1965528611 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:15.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.112+0000 7f8d6bfff700 1 --2- 172.21.15.97:0/1965528611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d6c15c610 0x7f8d6c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.112+0000 7f8d6b7fe700 1 --2- 172.21.15.97:0/1965528611 >> v2:172.21.15.146:3300/0 conn(0x7f8d6c06b360 0x7f8d6c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.113+0000 7f8d6affd700 1 -- 172.21.15.97:0/1965528611 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939664563 0 0) 0x7f8d6c0cff90 con 0x7f8d6c0d2210 2023-07-06T03:54:15.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.113+0000 7f8d6affd700 1 -- 172.21.15.97:0/1965528611 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d540035b0 con 0x7f8d6c0d2210 2023-07-06T03:54:15.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.113+0000 7f8d6bfff700 1 -- 172.21.15.97:0/1965528611 >> v2:172.21.15.146:3300/0 conn(0x7f8d6c06b360 msgr2=0x7f8d6c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:15.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.113+0000 7f8d6bfff700 1 --2- 172.21.15.97:0/1965528611 >> v2:172.21.15.146:3300/0 conn(0x7f8d6c06b360 0x7f8d6c0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.113+0000 7f8d6bfff700 1 -- 172.21.15.97:0/1965528611 >> v1:172.21.15.97:6789/0 conn(0x7f8d6c0d2210 legacy=0x7f8d6c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.113+0000 7f8d6bfff700 1 -- 172.21.15.97:0/1965528611 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c0b5650 con 0x7f8d6c15c610 2023-07-06T03:54:15.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.113+0000 7f8d6b7fe700 1 --2- 172.21.15.97:0/1965528611 >> v2:172.21.15.146:3300/0 conn(0x7f8d6c06b360 0x7f8d6c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:54:15.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.113+0000 7f8d6bfff700 1 --2- 172.21.15.97:0/1965528611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d6c15c610 0x7f8d6c0b47b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009840 tx=0x7f8d5c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=decd02914f96a796 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:15.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.113+0000 7f8d6affd700 1 -- 172.21.15.97:0/1965528611 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c011830 con 0x7f8d6c15c610 2023-07-06T03:54:15.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.113+0000 7f8d6affd700 1 -- 172.21.15.97:0/1965528611 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5c011990 con 0x7f8d6c15c610 2023-07-06T03:54:15.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.113+0000 7f8d6affd700 1 -- 172.21.15.97:0/1965528611 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c011d00 con 0x7f8d6c15c610 2023-07-06T03:54:15.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.114+0000 7f8d723ff700 1 -- 172.21.15.97:0/1965528611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d6c15c610 msgr2=0x7f8d6c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.114+0000 7f8d723ff700 1 --2- 172.21.15.97:0/1965528611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d6c15c610 0x7f8d6c0b47b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009840 tx=0x7f8d5c009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:15.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.114+0000 7f8d723ff700 1 -- 172.21.15.97:0/1965528611 shutdown_connections 2023-07-06T03:54:15.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.114+0000 7f8d723ff700 1 --2- 172.21.15.97:0/1965528611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d6c15c610 0x7f8d6c0b47b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.114+0000 7f8d723ff700 1 --2- 172.21.15.97:0/1965528611 >> v2:172.21.15.146:3300/0 conn(0x7f8d6c06b360 0x7f8d6c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.114+0000 7f8d723ff700 1 -- 172.21.15.97:0/1965528611 >> 172.21.15.97:0/1965528611 conn(0x7f8d6c06a6c0 msgr2=0x7f8d6c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:15.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.114+0000 7f8d723ff700 1 -- 172.21.15.97:0/1965528611 shutdown_connections 2023-07-06T03:54:15.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.114+0000 7f8d723ff700 1 -- 172.21.15.97:0/1965528611 wait complete. 2023-07-06T03:54:15.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.116+0000 7f8d723ff700 1 Processor -- start 2023-07-06T03:54:15.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.116+0000 7f8d723ff700 1 -- start start 2023-07-06T03:54:15.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d723ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d6c06b360 0x7f8d6c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d723ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c0d2210 0x7f8d6c0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d723ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d6c15c610 0x7f8d6c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d723ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c0ccc70 con 0x7f8d6c0d2210 2023-07-06T03:54:15.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d723ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c15d1b0 con 0x7f8d6c06b360 2023-07-06T03:54:15.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d723ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c15d490 con 0x7f8d6c15c610 2023-07-06T03:54:15.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d6b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c0d2210 0x7f8d6c0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d6b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c0d2210 0x7f8d6c0cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40806/0 (socket says 172.21.15.97:40806) 2023-07-06T03:54:15.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d6b7fe700 1 -- 172.21.15.97:0/2301058211 learned_addr learned my addr 172.21.15.97:0/2301058211 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:15.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d6b7fe700 1 -- 172.21.15.97:0/2301058211 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d6c15c610 msgr2=0x7f8d6c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d6b7fe700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d6c15c610 0x7f8d6c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d6b7fe700 1 -- 172.21.15.97:0/2301058211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d6c06b360 msgr2=0x7f8d6c15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d6b7fe700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d6c06b360 0x7f8d6c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.117+0000 7f8d6b7fe700 1 -- 172.21.15.97:0/2301058211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d600094a0 con 0x7f8d6c0d2210 2023-07-06T03:54:15.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.118+0000 7f8d6b7fe700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c0d2210 0x7f8d6c0cc730 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000ebe0 tx=0x7f8d60009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:15.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.118+0000 7f8d68ff9700 1 -- 172.21.15.97:0/2301058211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6000c890 con 0x7f8d6c0d2210 2023-07-06T03:54:15.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.118+0000 7f8d68ff9700 1 -- 172.21.15.97:0/2301058211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6000c9f0 con 0x7f8d6c0d2210 2023-07-06T03:54:15.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.118+0000 7f8d723ff700 1 -- 172.21.15.97:0/2301058211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5c0094a0 con 0x7f8d6c0d2210 2023-07-06T03:54:15.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.118+0000 7f8d68ff9700 1 -- 172.21.15.97:0/2301058211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6000cda0 con 0x7f8d6c0d2210 2023-07-06T03:54:15.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.118+0000 7f8d723ff700 1 -- 172.21.15.97:0/2301058211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6c15d820 con 0x7f8d6c0d2210 2023-07-06T03:54:15.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.119+0000 7f8d723ff700 1 -- 172.21.15.97:0/2301058211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6c0cd4b0 con 0x7f8d6c0d2210 2023-07-06T03:54:15.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.120+0000 7f8d68ff9700 1 -- 172.21.15.97:0/2301058211 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d60013470 con 0x7f8d6c0d2210 2023-07-06T03:54:15.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.121+0000 7f8d68ff9700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5806ace0 0x7f8d5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.121+0000 7f8d68ff9700 1 -- 172.21.15.97:0/2301058211 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8d60089060 con 0x7f8d6c0d2210 2023-07-06T03:54:15.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.121+0000 7f8d6bfff700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5806ace0 0x7f8d5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.122+0000 7f8d6bfff700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5806ace0 0x7f8d5806d190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009b90 tx=0x7f8d5c010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:15.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.125+0000 7f8d68ff9700 1 -- 172.21.15.97:0/2301058211 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8d600bb310 con 0x7f8d6c0d2210 2023-07-06T03:54:15.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.319+0000 7f8d723ff700 1 -- 172.21.15.97:0/2301058211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7f8d6c04e340 con 0x7f8d6c0d2210 2023-07-06T03:54:15.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.324+0000 7f8d68ff9700 1 -- 172.21.15.97:0/2301058211 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v39) v1 ==== 111+0+0 (secure 0 0 0) 0x7f8d60057e20 con 0x7f8d6c0d2210 2023-07-06T03:54:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.326+0000 7f8d723ff700 1 -- 172.21.15.97:0/2301058211 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5806ace0 msgr2=0x7f8d5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.326+0000 7f8d723ff700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5806ace0 0x7f8d5806d190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009b90 tx=0x7f8d5c010000 comp rx=0 tx=0).stop 2023-07-06T03:54:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.326+0000 7f8d723ff700 1 -- 172.21.15.97:0/2301058211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c0d2210 msgr2=0x7f8d6c0cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.326+0000 7f8d723ff700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c0d2210 0x7f8d6c0cc730 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000ebe0 tx=0x7f8d60009e60 comp rx=0 tx=0).stop 2023-07-06T03:54:15.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.326+0000 7f8d723ff700 1 -- 172.21.15.97:0/2301058211 shutdown_connections 2023-07-06T03:54:15.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.326+0000 7f8d723ff700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5806ace0 0x7f8d5806d190 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.326+0000 7f8d723ff700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d6c15c610 0x7f8d6c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.326+0000 7f8d723ff700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d6c06b360 0x7f8d6c15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.326+0000 7f8d723ff700 1 --2- 172.21.15.97:0/2301058211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d6c0d2210 0x7f8d6c0cc730 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.326+0000 7f8d723ff700 1 -- 172.21.15.97:0/2301058211 >> 172.21.15.97:0/2301058211 conn(0x7f8d6c06a6c0 msgr2=0x7f8d6c15b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:15.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.327+0000 7f8d723ff700 1 -- 172.21.15.97:0/2301058211 shutdown_connections 2023-07-06T03:54:15.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.327+0000 7f8d723ff700 1 -- 172.21.15.97:0/2301058211 wait complete. 2023-07-06T03:54:15.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-07-06T03:54:15.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.540+0000 7f3487fff700 1 Processor -- start 2023-07-06T03:54:15.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.540+0000 7f3487fff700 1 -- start start 2023-07-06T03:54:15.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f3487fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f348006b360 0x7f34800b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f3487fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34800d2210 0x7f34800d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f3487fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34800cff90 con 0x7f348015c610 2023-07-06T03:54:15.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f3487fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34800d0780 con 0x7f348006b360 2023-07-06T03:54:15.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f3487fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348015c9e0 con 0x7f34800d2210 2023-07-06T03:54:15.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f348559a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f348006b360 0x7f34800b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f348559a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f348006b360 0x7f34800b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40818/0 (socket says 172.21.15.97:40818) 2023-07-06T03:54:15.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f348559a700 1 -- 172.21.15.97:0/4003996443 learned_addr learned my addr 172.21.15.97:0/4003996443 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:15.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f3485d9b700 1 --1- 172.21.15.97:0/4003996443 >> v1:172.21.15.97:6789/0 conn(0x7f348015c610 0x7f34800b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59992/0 (socket says 172.21.15.97:59992) 2023-07-06T03:54:15.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f348659c700 1 --2- 172.21.15.97:0/4003996443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34800d2210 0x7f34800d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f348559a700 1 -- 172.21.15.97:0/4003996443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34800d2210 msgr2=0x7f34800d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:15.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f348559a700 1 --2- 172.21.15.97:0/4003996443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34800d2210 0x7f34800d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f348559a700 1 -- 172.21.15.97:0/4003996443 >> v1:172.21.15.97:6789/0 conn(0x7f348015c610 legacy=0x7f34800b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.542+0000 7f348559a700 1 -- 172.21.15.97:0/4003996443 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800b5650 con 0x7f348006b360 2023-07-06T03:54:15.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.543+0000 7f348559a700 1 --2- 172.21.15.97:0/4003996443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f348006b360 0x7f34800b4ec0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f34700098f0 tx=0x7f3470009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f98c29c0df8094 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:15.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.543+0000 7f3484d99700 1 -- 172.21.15.97:0/4003996443 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347000b8a0 con 0x7f348006b360 2023-07-06T03:54:15.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.543+0000 7f3484d99700 1 -- 172.21.15.97:0/4003996443 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347000ba00 con 0x7f348006b360 2023-07-06T03:54:15.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.543+0000 7f3484d99700 1 -- 172.21.15.97:0/4003996443 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347000bd90 con 0x7f348006b360 2023-07-06T03:54:15.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.543+0000 7f3487fff700 1 -- 172.21.15.97:0/4003996443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f348006b360 msgr2=0x7f34800b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.543+0000 7f3487fff700 1 --2- 172.21.15.97:0/4003996443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f348006b360 0x7f34800b4ec0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f34700098f0 tx=0x7f3470009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:15.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.544+0000 7f3487fff700 1 -- 172.21.15.97:0/4003996443 shutdown_connections 2023-07-06T03:54:15.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.544+0000 7f3487fff700 1 --2- 172.21.15.97:0/4003996443 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34800d2210 0x7f34800d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.544+0000 7f3487fff700 1 --2- 172.21.15.97:0/4003996443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f348006b360 0x7f34800b4ec0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.544+0000 7f3487fff700 1 -- 172.21.15.97:0/4003996443 >> 172.21.15.97:0/4003996443 conn(0x7f348006a6c0 msgr2=0x7f348006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:15.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.544+0000 7f3487fff700 1 -- 172.21.15.97:0/4003996443 shutdown_connections 2023-07-06T03:54:15.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.544+0000 7f3487fff700 1 -- 172.21.15.97:0/4003996443 wait complete. 2023-07-06T03:54:15.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.545+0000 7f3487fff700 1 Processor -- start 2023-07-06T03:54:15.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.545+0000 7f3487fff700 1 -- start start 2023-07-06T03:54:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.546+0000 7f3487fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f348006b360 0x7f3480142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.546+0000 7f3487fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34800d2210 0x7f348015ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.546+0000 7f3487fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f348015c610 0x7f348015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.546+0000 7f3487fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348015f9f0 con 0x7f34800d2210 2023-07-06T03:54:15.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.546+0000 7f3487fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480152370 con 0x7f348015c610 2023-07-06T03:54:15.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.546+0000 7f3487fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34801f2d40 con 0x7f348006b360 2023-07-06T03:54:15.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f348559a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34800d2210 0x7f348015ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f348559a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34800d2210 0x7f348015ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40826/0 (socket says 172.21.15.97:40826) 2023-07-06T03:54:15.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f348559a700 1 -- 172.21.15.97:0/100292133 learned_addr learned my addr 172.21.15.97:0/100292133 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:15.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f348659c700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f348015c610 0x7f348015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f3485d9b700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f348006b360 0x7f3480142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f348559a700 1 -- 172.21.15.97:0/100292133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f348006b360 msgr2=0x7f3480142480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f348559a700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f348006b360 0x7f3480142480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f348559a700 1 -- 172.21.15.97:0/100292133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f348015c610 msgr2=0x7f348015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f348559a700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f348015c610 0x7f348015f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f348559a700 1 -- 172.21.15.97:0/100292133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470009550 con 0x7f34800d2210 2023-07-06T03:54:15.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f348559a700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34800d2210 0x7f348015ee80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f3470000c00 tx=0x7f3470004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:15.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.547+0000 7f34767fc700 1 -- 172.21.15.97:0/100292133 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470004480 con 0x7f34800d2210 2023-07-06T03:54:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.548+0000 7f34767fc700 1 -- 172.21.15.97:0/100292133 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347001f070 con 0x7f34800d2210 2023-07-06T03:54:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.548+0000 7f3487fff700 1 -- 172.21.15.97:0/100292133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3480156090 con 0x7f34800d2210 2023-07-06T03:54:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.548+0000 7f34767fc700 1 -- 172.21.15.97:0/100292133 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34700038c0 con 0x7f34800d2210 2023-07-06T03:54:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.548+0000 7f3487fff700 1 -- 172.21.15.97:0/100292133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3480156650 con 0x7f34800d2210 2023-07-06T03:54:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.549+0000 7f3487fff700 1 -- 172.21.15.97:0/100292133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3480066fe0 con 0x7f34800d2210 2023-07-06T03:54:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.550+0000 7f34767fc700 1 -- 172.21.15.97:0/100292133 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3470021030 con 0x7f34800d2210 2023-07-06T03:54:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.550+0000 7f34767fc700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f346406ace0 0x7f346406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.551+0000 7f34767fc700 1 -- 172.21.15.97:0/100292133 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f347008a6a0 con 0x7f34800d2210 2023-07-06T03:54:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.551+0000 7f3485d9b700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f346406ace0 0x7f346406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.551+0000 7f3485d9b700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f346406ace0 0x7f346406d190 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f347c000f60 tx=0x7f347c007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:15.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.555+0000 7f34767fc700 1 -- 172.21.15.97:0/100292133 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3470055c00 con 0x7f34800d2210 2023-07-06T03:54:15.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.750+0000 7f3487fff700 1 -- 172.21.15.97:0/100292133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7f348004e340 con 0x7f34800d2210 2023-07-06T03:54:15.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.754+0000 7f34767fc700 1 -- 172.21.15.97:0/100292133 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v40) v1 ==== 102+0+0 (secure 0 0 0) 0x7f3470059220 con 0x7f34800d2210 2023-07-06T03:54:15.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.756+0000 7f3487fff700 1 -- 172.21.15.97:0/100292133 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f346406ace0 msgr2=0x7f346406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.756+0000 7f3487fff700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f346406ace0 0x7f346406d190 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f347c000f60 tx=0x7f347c007d10 comp rx=0 tx=0).stop 2023-07-06T03:54:15.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.757+0000 7f3487fff700 1 -- 172.21.15.97:0/100292133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34800d2210 msgr2=0x7f348015ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.757+0000 7f3487fff700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34800d2210 0x7f348015ee80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f3470000c00 tx=0x7f3470004750 comp rx=0 tx=0).stop 2023-07-06T03:54:15.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.757+0000 7f3487fff700 1 -- 172.21.15.97:0/100292133 shutdown_connections 2023-07-06T03:54:15.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.757+0000 7f3487fff700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f346406ace0 0x7f346406d190 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.757+0000 7f3487fff700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f348015c610 0x7f348015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.757+0000 7f3487fff700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f348006b360 0x7f3480142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.757+0000 7f3487fff700 1 --2- 172.21.15.97:0/100292133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34800d2210 0x7f348015ee80 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.757+0000 7f3487fff700 1 -- 172.21.15.97:0/100292133 >> 172.21.15.97:0/100292133 conn(0x7f348006a6c0 msgr2=0x7f34800b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:15.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.757+0000 7f3487fff700 1 -- 172.21.15.97:0/100292133 shutdown_connections 2023-07-06T03:54:15.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.757+0000 7f3487fff700 1 -- 172.21.15.97:0/100292133 wait complete. 2023-07-06T03:54:15.777 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-07-06T03:54:15.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.973+0000 7eff08dff700 1 Processor -- start 2023-07-06T03:54:15.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.973+0000 7eff08dff700 1 -- start start 2023-07-06T03:54:15.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.974+0000 7eff08dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff04158b20 0x7eff040cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.974+0000 7eff08dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7eff0406cab0 0x7eff040ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.974+0000 7eff08dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff040ccbc0 con 0x7eff040b0560 2023-07-06T03:54:15.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.974+0000 7eff08dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff040ce820 con 0x7eff04158b20 2023-07-06T03:54:15.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.974+0000 7eff08dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff04158ef0 con 0x7eff0406cab0 2023-07-06T03:54:15.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.975+0000 7eff0259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff04158b20 0x7eff040cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.975+0000 7eff0259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff04158b20 0x7eff040cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40834/0 (socket says 172.21.15.97:40834) 2023-07-06T03:54:15.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.975+0000 7eff0259c700 1 -- 172.21.15.97:0/2576578995 learned_addr learned my addr 172.21.15.97:0/2576578995 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:15.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.975+0000 7eff01d9b700 1 --2- 172.21.15.97:0/2576578995 >> v2:172.21.15.146:3300/0 conn(0x7eff0406cab0 0x7eff040ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.975+0000 7eff0259c700 1 -- 172.21.15.97:0/2576578995 >> v2:172.21.15.146:3300/0 conn(0x7eff0406cab0 msgr2=0x7eff040ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:15.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.975+0000 7eff0259c700 1 --2- 172.21.15.97:0/2576578995 >> v2:172.21.15.146:3300/0 conn(0x7eff0406cab0 0x7eff040ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.975+0000 7eff0259c700 1 -- 172.21.15.97:0/2576578995 >> v1:172.21.15.97:6789/0 conn(0x7eff040b0560 legacy=0x7eff0406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.975+0000 7eff0159a700 1 -- 172.21.15.97:0/2576578995 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785191800 0 0) 0x7eff040ccbc0 con 0x7eff040b0560 2023-07-06T03:54:15.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.975+0000 7eff0259c700 1 -- 172.21.15.97:0/2576578995 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff040ce560 con 0x7eff04158b20 2023-07-06T03:54:15.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.975+0000 7eff0259c700 1 --2- 172.21.15.97:0/2576578995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff04158b20 0x7eff040cc680 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7eff0406b040 tx=0x7efef400ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4189ace86547fce2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:15.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.976+0000 7eff0159a700 1 -- 172.21.15.97:0/2576578995 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400cb70 con 0x7eff04158b20 2023-07-06T03:54:15.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.976+0000 7eff0159a700 1 -- 172.21.15.97:0/2576578995 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef400ccd0 con 0x7eff04158b20 2023-07-06T03:54:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.976+0000 7eff0159a700 1 -- 172.21.15.97:0/2576578995 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4018620 con 0x7eff04158b20 2023-07-06T03:54:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.976+0000 7eff08dff700 1 -- 172.21.15.97:0/2576578995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff04158b20 msgr2=0x7eff040cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.976+0000 7eff08dff700 1 --2- 172.21.15.97:0/2576578995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff04158b20 0x7eff040cc680 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7eff0406b040 tx=0x7efef400ad70 comp rx=0 tx=0).stop 2023-07-06T03:54:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.976+0000 7eff08dff700 1 -- 172.21.15.97:0/2576578995 shutdown_connections 2023-07-06T03:54:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.976+0000 7eff08dff700 1 --2- 172.21.15.97:0/2576578995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff04158b20 0x7eff040cc680 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.976+0000 7eff08dff700 1 --2- 172.21.15.97:0/2576578995 >> v2:172.21.15.146:3300/0 conn(0x7eff0406cab0 0x7eff040ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.976+0000 7eff08dff700 1 -- 172.21.15.97:0/2576578995 >> 172.21.15.97:0/2576578995 conn(0x7eff0406a060 msgr2=0x7eff0406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:15.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.976+0000 7eff08dff700 1 -- 172.21.15.97:0/2576578995 shutdown_connections 2023-07-06T03:54:15.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.976+0000 7eff08dff700 1 -- 172.21.15.97:0/2576578995 wait complete. 2023-07-06T03:54:15.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.978+0000 7eff08dff700 1 Processor -- start 2023-07-06T03:54:15.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.978+0000 7eff08dff700 1 -- start start 2023-07-06T03:54:15.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.979+0000 7eff08dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff0406cab0 0x7eff041593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.979+0000 7eff08dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040b0560 0x7eff04159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.979+0000 7eff08dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff04158b20 0x7eff0414c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.979+0000 7eff08dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff040afa80 con 0x7eff040b0560 2023-07-06T03:54:15.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.979+0000 7eff08dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0414cd50 con 0x7eff0406cab0 2023-07-06T03:54:15.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.979+0000 7eff08dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff04157f90 con 0x7eff04158b20 2023-07-06T03:54:15.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff01d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040b0560 0x7eff04159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff01d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040b0560 0x7eff04159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40840/0 (socket says 172.21.15.97:40840) 2023-07-06T03:54:15.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff01d9b700 1 -- 172.21.15.97:0/2885896059 learned_addr learned my addr 172.21.15.97:0/2885896059 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:15.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff0259c700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff0406cab0 0x7eff041593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff02d9d700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff04158b20 0x7eff0414c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff01d9b700 1 -- 172.21.15.97:0/2885896059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff04158b20 msgr2=0x7eff0414c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff01d9b700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff04158b20 0x7eff0414c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff01d9b700 1 -- 172.21.15.97:0/2885896059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff0406cab0 msgr2=0x7eff041593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:15.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff01d9b700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff0406cab0 0x7eff041593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:15.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff01d9b700 1 -- 172.21.15.97:0/2885896059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef400a6f0 con 0x7eff040b0560 2023-07-06T03:54:15.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff02d9d700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff04158b20 0x7eff0414c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:15.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff0259c700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff0406cab0 0x7eff041593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:15.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff01d9b700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040b0560 0x7eff04159bf0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7efef800e870 tx=0x7efef800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:15.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7efef2ffd700 1 -- 172.21.15.97:0/2885896059 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef8009590 con 0x7eff040b0560 2023-07-06T03:54:15.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7efef2ffd700 1 -- 172.21.15.97:0/2885896059 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef800ee70 con 0x7eff040b0560 2023-07-06T03:54:15.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7eff08dff700 1 -- 172.21.15.97:0/2885896059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff04157cb0 con 0x7eff040b0560 2023-07-06T03:54:15.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.980+0000 7efef2ffd700 1 -- 172.21.15.97:0/2885896059 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef800ca50 con 0x7eff040b0560 2023-07-06T03:54:15.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.981+0000 7eff08dff700 1 -- 172.21.15.97:0/2885896059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff04157870 con 0x7eff040b0560 2023-07-06T03:54:15.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.982+0000 7eff08dff700 1 -- 172.21.15.97:0/2885896059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff04159a40 con 0x7eff040b0560 2023-07-06T03:54:15.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.983+0000 7efef2ffd700 1 -- 172.21.15.97:0/2885896059 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efef800cbb0 con 0x7eff040b0560 2023-07-06T03:54:15.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.984+0000 7efef2ffd700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee406ace0 0x7efee406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:15.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.984+0000 7efef2ffd700 1 -- 172.21.15.97:0/2885896059 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5296+0+0 (secure 0 0 0) 0x7efef8089190 con 0x7eff040b0560 2023-07-06T03:54:15.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.984+0000 7eff0259c700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee406ace0 0x7efee406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:15.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.984+0000 7eff0259c700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee406ace0 0x7efee406d190 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efef4000c00 tx=0x7efef401a040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:15.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:15.988+0000 7efef2ffd700 1 -- 172.21.15.97:0/2885896059 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efef8057fd0 con 0x7eff040b0560 2023-07-06T03:54:16.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.183+0000 7eff08dff700 1 -- 172.21.15.97:0/2885896059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7eff040afbc0 con 0x7eff040b0560 2023-07-06T03:54:16.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.188+0000 7efef2ffd700 1 -- 172.21.15.97:0/2885896059 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v41) v1 ==== 107+0+0 (secure 0 0 0) 0x7eff040afbc0 con 0x7eff040b0560 2023-07-06T03:54:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 -- 172.21.15.97:0/2885896059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee406ace0 msgr2=0x7efee406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee406ace0 0x7efee406d190 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efef4000c00 tx=0x7efef401a040 comp rx=0 tx=0).stop 2023-07-06T03:54:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 -- 172.21.15.97:0/2885896059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040b0560 msgr2=0x7eff04159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040b0560 0x7eff04159bf0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7efef800e870 tx=0x7efef800ec30 comp rx=0 tx=0).stop 2023-07-06T03:54:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 -- 172.21.15.97:0/2885896059 shutdown_connections 2023-07-06T03:54:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efee406ace0 0x7efee406d190 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7eff04158b20 0x7eff0414c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff0406cab0 0x7eff041593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 --2- 172.21.15.97:0/2885896059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040b0560 0x7eff04159bf0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 -- 172.21.15.97:0/2885896059 >> 172.21.15.97:0/2885896059 conn(0x7eff0406a060 msgr2=0x7eff04155270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 -- 172.21.15.97:0/2885896059 shutdown_connections 2023-07-06T03:54:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.190+0000 7eff08dff700 1 -- 172.21.15.97:0/2885896059 wait complete. 2023-07-06T03:54:16.211 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T03:54:16.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.405+0000 7fed43dff700 1 Processor -- start 2023-07-06T03:54:16.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.405+0000 7fed43dff700 1 -- start start 2023-07-06T03:54:16.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.405+0000 7fed43dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed3c0719c0 0x7fed3c0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:16.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.405+0000 7fed43dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3c15da10 con 0x7fed3c15c590 2023-07-06T03:54:16.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.405+0000 7fed43dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3c0ccc30 con 0x7fed3c15c1c0 2023-07-06T03:54:16.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.405+0000 7fed43dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3c0ccf10 con 0x7fed3c0719c0 2023-07-06T03:54:16.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.406+0000 7fed4239c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed3c0719c0 0x7fed3c0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:16.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.406+0000 7fed4139a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fed3c15c590 0x7fed3c15d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60020/0 (socket says 172.21.15.97:60020) 2023-07-06T03:54:16.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.406+0000 7fed4139a700 1 -- 172.21.15.97:0/3911001769 learned_addr learned my addr 172.21.15.97:0/3911001769 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:16.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.406+0000 7fed4239c700 1 -- 172.21.15.97:0/3911001769 >> v1:172.21.15.146:6789/0 conn(0x7fed3c15c1c0 legacy=0x7fed3c0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.406+0000 7fed4239c700 1 -- 172.21.15.97:0/3911001769 >> v1:172.21.15.97:6789/0 conn(0x7fed3c15c590 legacy=0x7fed3c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.406+0000 7fed4239c700 1 -- 172.21.15.97:0/3911001769 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c0cd1f0 con 0x7fed3c0719c0 2023-07-06T03:54:16.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.406+0000 7fed4239c700 1 --2- 172.21.15.97:0/3911001769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3c0719c0 0x7fed3c0cb080 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fed34009d90 tx=0x7fed3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3e60dd339a1cb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:16.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.406+0000 7fed40b99700 1 -- 172.21.15.97:0/3911001769 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3400e070 con 0x7fed3c0719c0 2023-07-06T03:54:16.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.407+0000 7fed40b99700 1 -- 172.21.15.97:0/3911001769 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3400bbd0 con 0x7fed3c0719c0 2023-07-06T03:54:16.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.407+0000 7fed40b99700 1 -- 172.21.15.97:0/3911001769 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed34012430 con 0x7fed3c0719c0 2023-07-06T03:54:16.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.407+0000 7fed43dff700 1 -- 172.21.15.97:0/3911001769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3c0719c0 msgr2=0x7fed3c0cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.407+0000 7fed43dff700 1 --2- 172.21.15.97:0/3911001769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3c0719c0 0x7fed3c0cb080 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fed34009d90 tx=0x7fed3400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:16.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.407+0000 7fed43dff700 1 -- 172.21.15.97:0/3911001769 shutdown_connections 2023-07-06T03:54:16.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.407+0000 7fed43dff700 1 --2- 172.21.15.97:0/3911001769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3c0719c0 0x7fed3c0cb080 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.407+0000 7fed43dff700 1 -- 172.21.15.97:0/3911001769 >> 172.21.15.97:0/3911001769 conn(0x7fed3c0b1650 msgr2=0x7fed3c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:16.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.407+0000 7fed43dff700 1 -- 172.21.15.97:0/3911001769 shutdown_connections 2023-07-06T03:54:16.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.407+0000 7fed43dff700 1 -- 172.21.15.97:0/3911001769 wait complete. 2023-07-06T03:54:16.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.409+0000 7fed43dff700 1 Processor -- start 2023-07-06T03:54:16.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.409+0000 7fed43dff700 1 -- start start 2023-07-06T03:54:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed43dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed3c0719c0 0x7fed3c14d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed43dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3c15c1c0 0x7fed3c14a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed43dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed3c15c590 0x7fed3c0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed43dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c14a9a0 con 0x7fed3c15c1c0 2023-07-06T03:54:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed43dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0b2e90 con 0x7fed3c0719c0 2023-07-06T03:54:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed43dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0b3140 con 0x7fed3c15c590 2023-07-06T03:54:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed4139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3c15c1c0 0x7fed3c14a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:16.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed4139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3c15c1c0 0x7fed3c14a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40858/0 (socket says 172.21.15.97:40858) 2023-07-06T03:54:16.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed4139a700 1 -- 172.21.15.97:0/3557445034 learned_addr learned my addr 172.21.15.97:0/3557445034 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:16.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed41b9b700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed3c0719c0 0x7fed3c14d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:16.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed4139a700 1 -- 172.21.15.97:0/3557445034 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed3c15c590 msgr2=0x7fed3c0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.410+0000 7fed4139a700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed3c15c590 0x7fed3c0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.411+0000 7fed4139a700 1 -- 172.21.15.97:0/3557445034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed3c0719c0 msgr2=0x7fed3c14d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.411+0000 7fed4139a700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed3c0719c0 0x7fed3c14d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.411+0000 7fed4139a700 1 -- 172.21.15.97:0/3557445034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed34009a70 con 0x7fed3c15c1c0 2023-07-06T03:54:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.411+0000 7fed41b9b700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed3c0719c0 0x7fed3c14d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.411+0000 7fed4139a700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3c15c1c0 0x7fed3c14a460 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00c450 tx=0x7fed2c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.411+0000 7fed327fc700 1 -- 172.21.15.97:0/3557445034 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c005d70 con 0x7fed3c15c1c0 2023-07-06T03:54:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.411+0000 7fed327fc700 1 -- 172.21.15.97:0/3557445034 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2c0070d0 con 0x7fed3c15c1c0 2023-07-06T03:54:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.411+0000 7fed43dff700 1 -- 172.21.15.97:0/3557445034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c0b3420 con 0x7fed3c15c1c0 2023-07-06T03:54:16.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.411+0000 7fed327fc700 1 -- 172.21.15.97:0/3557445034 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c007440 con 0x7fed3c15c1c0 2023-07-06T03:54:16.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.411+0000 7fed43dff700 1 -- 172.21.15.97:0/3557445034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c0b39e0 con 0x7fed3c15c1c0 2023-07-06T03:54:16.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.412+0000 7fed43dff700 1 -- 172.21.15.97:0/3557445034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3c04e340 con 0x7fed3c15c1c0 2023-07-06T03:54:16.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.413+0000 7fed327fc700 1 -- 172.21.15.97:0/3557445034 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed2c0075a0 con 0x7fed3c15c1c0 2023-07-06T03:54:16.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.414+0000 7fed327fc700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed2006ace0 0x7fed2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:16.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.414+0000 7fed327fc700 1 -- 172.21.15.97:0/3557445034 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fed2c0882e0 con 0x7fed3c15c1c0 2023-07-06T03:54:16.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.414+0000 7fed41b9b700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed2006ace0 0x7fed2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:16.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.415+0000 7fed41b9b700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed2006ace0 0x7fed2006d190 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fed38000f60 tx=0x7fed38009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.418+0000 7fed327fc700 1 -- 172.21.15.97:0/3557445034 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fed2c053740 con 0x7fed3c15c1c0 2023-07-06T03:54:16.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.612+0000 7fed43dff700 1 -- 172.21.15.97:0/3557445034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fed3c15da10 con 0x7fed3c15c1c0 2023-07-06T03:54:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.616+0000 7fed327fc700 1 -- 172.21.15.97:0/3557445034 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7fed3c15da10 con 0x7fed3c15c1c0 2023-07-06T03:54:16.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 -- 172.21.15.97:0/3557445034 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed2006ace0 msgr2=0x7fed2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed2006ace0 0x7fed2006d190 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fed38000f60 tx=0x7fed38009040 comp rx=0 tx=0).stop 2023-07-06T03:54:16.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 -- 172.21.15.97:0/3557445034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3c15c1c0 msgr2=0x7fed3c14a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3c15c1c0 0x7fed3c14a460 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00c450 tx=0x7fed2c00c810 comp rx=0 tx=0).stop 2023-07-06T03:54:16.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 -- 172.21.15.97:0/3557445034 shutdown_connections 2023-07-06T03:54:16.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fed2006ace0 0x7fed2006d190 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fed3c15c590 0x7fed3c0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed3c0719c0 0x7fed3c14d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 --2- 172.21.15.97:0/3557445034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3c15c1c0 0x7fed3c14a460 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 -- 172.21.15.97:0/3557445034 >> 172.21.15.97:0/3557445034 conn(0x7fed3c0b1650 msgr2=0x7fed3c0ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:16.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 -- 172.21.15.97:0/3557445034 shutdown_connections 2023-07-06T03:54:16.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.619+0000 7fed43dff700 1 -- 172.21.15.97:0/3557445034 wait complete. 2023-07-06T03:54:16.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-07-06T03:54:16.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.836+0000 7f7213dff700 1 Processor -- start 2023-07-06T03:54:16.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.836+0000 7f7213dff700 1 -- start start 2023-07-06T03:54:16.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7213dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f720c15c1c0 0x7f720c0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:16.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7213dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f720c0719c0 0x7f720c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:16.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7213dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720c15da10 con 0x7f720c15c590 2023-07-06T03:54:16.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7213dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720c0ccc30 con 0x7f720c0719c0 2023-07-06T03:54:16.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7213dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720c0ccf10 con 0x7f720c15c1c0 2023-07-06T03:54:16.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f721139a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f720c15c590 0x7f720c15d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60032/0 (socket says 172.21.15.97:60032) 2023-07-06T03:54:16.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f721139a700 1 -- 172.21.15.97:0/1516621971 learned_addr learned my addr 172.21.15.97:0/1516621971 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:16.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f721239c700 1 --2- 172.21.15.97:0/1516621971 >> v2:172.21.15.97:3300/0 conn(0x7f720c0719c0 0x7f720c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:16.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7211b9b700 1 --2- 172.21.15.97:0/1516621971 >> v2:172.21.15.146:3300/0 conn(0x7f720c15c1c0 0x7f720c0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:16.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7210b99700 1 -- 172.21.15.97:0/1516621971 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359350779 0 0) 0x7f71fc001110 con 0x7f720c15c590 2023-07-06T03:54:16.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f721239c700 1 -- 172.21.15.97:0/1516621971 >> v2:172.21.15.146:3300/0 conn(0x7f720c15c1c0 msgr2=0x7f720c0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:16.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f721239c700 1 --2- 172.21.15.97:0/1516621971 >> v2:172.21.15.146:3300/0 conn(0x7f720c15c1c0 0x7f720c0cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f721239c700 1 -- 172.21.15.97:0/1516621971 >> v1:172.21.15.97:6789/0 conn(0x7f720c15c590 legacy=0x7f720c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f721239c700 1 -- 172.21.15.97:0/1516621971 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c0cd1f0 con 0x7f720c0719c0 2023-07-06T03:54:16.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f721239c700 1 --2- 172.21.15.97:0/1516621971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c0719c0 0x7f720c06a060 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f720c072570 tx=0x7f720400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb16539a813f0741 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:16.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7210b99700 1 -- 172.21.15.97:0/1516621971 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720400e070 con 0x7f720c0719c0 2023-07-06T03:54:16.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7210b99700 1 -- 172.21.15.97:0/1516621971 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720400bbd0 con 0x7f720c0719c0 2023-07-06T03:54:16.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7210b99700 1 -- 172.21.15.97:0/1516621971 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204012430 con 0x7f720c0719c0 2023-07-06T03:54:16.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7213dff700 1 -- 172.21.15.97:0/1516621971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c0719c0 msgr2=0x7f720c06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7213dff700 1 --2- 172.21.15.97:0/1516621971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c0719c0 0x7f720c06a060 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f720c072570 tx=0x7f720400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:16.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7213dff700 1 -- 172.21.15.97:0/1516621971 shutdown_connections 2023-07-06T03:54:16.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7213dff700 1 --2- 172.21.15.97:0/1516621971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c0719c0 0x7f720c06a060 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.839+0000 7f7213dff700 1 --2- 172.21.15.97:0/1516621971 >> v2:172.21.15.146:3300/0 conn(0x7f720c15c1c0 0x7f720c0cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.840+0000 7f7213dff700 1 -- 172.21.15.97:0/1516621971 >> 172.21.15.97:0/1516621971 conn(0x7f720c0b1650 msgr2=0x7f720c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:16.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.840+0000 7f7213dff700 1 -- 172.21.15.97:0/1516621971 shutdown_connections 2023-07-06T03:54:16.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.840+0000 7f7213dff700 1 -- 172.21.15.97:0/1516621971 wait complete. 2023-07-06T03:54:16.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.841+0000 7f7213dff700 1 Processor -- start 2023-07-06T03:54:16.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.841+0000 7f7213dff700 1 -- start start 2023-07-06T03:54:16.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.842+0000 7f7213dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f720c0719c0 0x7f720c146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:16.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.842+0000 7f7213dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f720c15c1c0 0x7f720c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:16.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.842+0000 7f7213dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c15c590 0x7f720c158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:16.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f7213dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c15dd50 con 0x7f720c15c590 2023-07-06T03:54:16.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f7213dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c15e030 con 0x7f720c0719c0 2023-07-06T03:54:16.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f7213dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c15e310 con 0x7f720c15c1c0 2023-07-06T03:54:16.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f721239c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c15c590 0x7f720c158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:16.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f721239c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c15c590 0x7f720c158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40878/0 (socket says 172.21.15.97:40878) 2023-07-06T03:54:16.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f7211b9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f720c0719c0 0x7f720c146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:16.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f721239c700 1 -- 172.21.15.97:0/3819740711 learned_addr learned my addr 172.21.15.97:0/3819740711 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:16.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f721139a700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f720c15c1c0 0x7f720c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:16.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f721239c700 1 -- 172.21.15.97:0/3819740711 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f720c15c1c0 msgr2=0x7f720c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f721239c700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f720c15c1c0 0x7f720c15f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f721239c700 1 -- 172.21.15.97:0/3819740711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f720c0719c0 msgr2=0x7f720c146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:16.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f721239c700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f720c0719c0 0x7f720c146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:16.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f721239c700 1 -- 172.21.15.97:0/3819740711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7204009a70 con 0x7f720c15c590 2023-07-06T03:54:16.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.843+0000 7f721239c700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c15c590 0x7f720c158750 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f720c072500 tx=0x7f7204004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:16.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.844+0000 7f72027fc700 1 -- 172.21.15.97:0/3819740711 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204012430 con 0x7f720c15c590 2023-07-06T03:54:16.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.844+0000 7f7213dff700 1 -- 172.21.15.97:0/3819740711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c15e5f0 con 0x7f720c15c590 2023-07-06T03:54:16.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.844+0000 7f72027fc700 1 -- 172.21.15.97:0/3819740711 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7204020070 con 0x7f720c15c590 2023-07-06T03:54:16.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.844+0000 7f72027fc700 1 -- 172.21.15.97:0/3819740711 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204004280 con 0x7f720c15c590 2023-07-06T03:54:16.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.844+0000 7f7213dff700 1 -- 172.21.15.97:0/3819740711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c1469d0 con 0x7f720c15c590 2023-07-06T03:54:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.845+0000 7f7213dff700 1 -- 172.21.15.97:0/3819740711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c15eb10 con 0x7f720c15c590 2023-07-06T03:54:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.846+0000 7f72027fc700 1 -- 172.21.15.97:0/3819740711 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72040043e0 con 0x7f720c15c590 2023-07-06T03:54:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.847+0000 7f72027fc700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71f006ace0 0x7f71f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.847+0000 7f72027fc700 1 -- 172.21.15.97:0/3819740711 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7204022070 con 0x7f720c15c590 2023-07-06T03:54:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.847+0000 7f7211b9b700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71f006ace0 0x7f71f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.847+0000 7f7211b9b700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71f006ace0 0x7f71f006d190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7208006b20 tx=0x7f7208008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:16.851+0000 7f72027fc700 1 -- 172.21.15.97:0/3819740711 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f720c15eb10 con 0x7f720c15c590 2023-07-06T03:54:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.049+0000 7f7213dff700 1 -- 172.21.15.97:0/3819740711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7f720c15eb10 con 0x7f720c15c590 2023-07-06T03:54:17.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.054+0000 7f72027fc700 1 -- 172.21.15.97:0/3819740711 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v43) v1 ==== 105+0+0 (secure 0 0 0) 0x7f720c15eb10 con 0x7f720c15c590 2023-07-06T03:54:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 -- 172.21.15.97:0/3819740711 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71f006ace0 msgr2=0x7f71f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71f006ace0 0x7f71f006d190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7208006b20 tx=0x7f7208008040 comp rx=0 tx=0).stop 2023-07-06T03:54:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 -- 172.21.15.97:0/3819740711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c15c590 msgr2=0x7f720c158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c15c590 0x7f720c158750 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f720c072500 tx=0x7f7204004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 -- 172.21.15.97:0/3819740711 shutdown_connections 2023-07-06T03:54:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71f006ace0 0x7f71f006d190 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c15c590 0x7f720c158750 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f720c0719c0 0x7f720c146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 --2- 172.21.15.97:0/3819740711 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f720c15c1c0 0x7f720c15f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 -- 172.21.15.97:0/3819740711 >> 172.21.15.97:0/3819740711 conn(0x7f720c0b1650 msgr2=0x7f720c0aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:17.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 -- 172.21.15.97:0/3819740711 shutdown_connections 2023-07-06T03:54:17.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.056+0000 7f7213dff700 1 -- 172.21.15.97:0/3819740711 wait complete. 2023-07-06T03:54:17.077 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2023-07-06T03:54:17.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.271+0000 7f7ce25ff700 1 Processor -- start 2023-07-06T03:54:17.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.272+0000 7f7ce25ff700 1 -- start start 2023-07-06T03:54:17.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7ce25ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7cdc15a300 0x7f7cdc0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:17.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7ce25ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7cdc0c9520 0x7f7cdc071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:17.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7ce25ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cdc070fc0 con 0x7f7cdc071990 2023-07-06T03:54:17.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7ce25ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cdc06c470 con 0x7f7cdc15a300 2023-07-06T03:54:17.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7ce25ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cdc06c750 con 0x7f7cdc0c9520 2023-07-06T03:54:17.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7cdbfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7cdc15a300 0x7f7cdc0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:17.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7ce0b9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7cdc071990 0x7f7cdc0cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60048/0 (socket says 172.21.15.97:60048) 2023-07-06T03:54:17.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7ce0b9c700 1 -- 172.21.15.97:0/1799535122 learned_addr learned my addr 172.21.15.97:0/1799535122 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:17.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7cdb7fe700 1 --2- 172.21.15.97:0/1799535122 >> v2:172.21.15.146:3300/0 conn(0x7f7cdc0c9520 0x7f7cdc071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7cdbfff700 1 -- 172.21.15.97:0/1799535122 >> v2:172.21.15.146:3300/0 conn(0x7f7cdc0c9520 msgr2=0x7f7cdc071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7cdbfff700 1 --2- 172.21.15.97:0/1799535122 >> v2:172.21.15.146:3300/0 conn(0x7f7cdc0c9520 0x7f7cdc071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7cdbfff700 1 -- 172.21.15.97:0/1799535122 >> v1:172.21.15.97:6789/0 conn(0x7f7cdc071990 legacy=0x7f7cdc0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7cdbfff700 1 -- 172.21.15.97:0/1799535122 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc070c50 con 0x7f7cdc15a300 2023-07-06T03:54:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.273+0000 7f7cdaffd700 1 -- 172.21.15.97:0/1799535122 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903275012 0 0) 0x7f7cdc070fc0 con 0x7f7cdc071990 2023-07-06T03:54:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.274+0000 7f7cdbfff700 1 --2- 172.21.15.97:0/1799535122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cdc15a300 0x7f7cdc0c8fe0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0cc510 tx=0x7f7ccc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aeecb93fb35ff3f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.274+0000 7f7cdaffd700 1 -- 172.21.15.97:0/1799535122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc011830 con 0x7f7cdc15a300 2023-07-06T03:54:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.274+0000 7f7cdaffd700 1 -- 172.21.15.97:0/1799535122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc011990 con 0x7f7cdc15a300 2023-07-06T03:54:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.274+0000 7f7cdaffd700 1 -- 172.21.15.97:0/1799535122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc011d20 con 0x7f7cdc15a300 2023-07-06T03:54:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.274+0000 7f7ce25ff700 1 -- 172.21.15.97:0/1799535122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cdc15a300 msgr2=0x7f7cdc0c8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.274+0000 7f7ce25ff700 1 --2- 172.21.15.97:0/1799535122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cdc15a300 0x7f7cdc0c8fe0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0cc510 tx=0x7f7ccc009b20 comp rx=0 tx=0).stop 2023-07-06T03:54:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.275+0000 7f7ce25ff700 1 -- 172.21.15.97:0/1799535122 shutdown_connections 2023-07-06T03:54:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.275+0000 7f7ce25ff700 1 --2- 172.21.15.97:0/1799535122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cdc15a300 0x7f7cdc0c8fe0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.275+0000 7f7ce25ff700 1 --2- 172.21.15.97:0/1799535122 >> v2:172.21.15.146:3300/0 conn(0x7f7cdc0c9520 0x7f7cdc071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.275+0000 7f7ce25ff700 1 -- 172.21.15.97:0/1799535122 >> 172.21.15.97:0/1799535122 conn(0x7f7cdc0b04a0 msgr2=0x7f7cdc0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.275+0000 7f7ce25ff700 1 -- 172.21.15.97:0/1799535122 shutdown_connections 2023-07-06T03:54:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.275+0000 7f7ce25ff700 1 -- 172.21.15.97:0/1799535122 wait complete. 2023-07-06T03:54:17.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.276+0000 7f7ce25ff700 1 Processor -- start 2023-07-06T03:54:17.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.276+0000 7f7ce25ff700 1 -- start start 2023-07-06T03:54:17.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.277+0000 7f7ce25ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cdc071990 0x7f7cdc0c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:17.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.277+0000 7f7ce25ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc0c9520 0x7f7cdc0cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:17.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.277+0000 7f7ce25ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cdc15a300 0x7f7cdc0cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:17.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.277+0000 7f7ce25ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc0c4010 con 0x7f7cdc15a300 2023-07-06T03:54:17.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.277+0000 7f7ce25ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc1efbd0 con 0x7f7cdc0c9520 2023-07-06T03:54:17.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.277+0000 7f7ce25ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc0c3390 con 0x7f7cdc071990 2023-07-06T03:54:17.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7ce0b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cdc15a300 0x7f7cdc0cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:17.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7ce0b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cdc15a300 0x7f7cdc0cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40902/0 (socket says 172.21.15.97:40902) 2023-07-06T03:54:17.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7ce0b9c700 1 -- 172.21.15.97:0/223413906 learned_addr learned my addr 172.21.15.97:0/223413906 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7cdb7fe700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc0c9520 0x7f7cdc0cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7cdbfff700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cdc071990 0x7f7cdc0c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7ce0b9c700 1 -- 172.21.15.97:0/223413906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cdc071990 msgr2=0x7f7cdc0c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7ce0b9c700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cdc071990 0x7f7cdc0c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7ce0b9c700 1 -- 172.21.15.97:0/223413906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc0c9520 msgr2=0x7f7cdc0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7ce0b9c700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc0c9520 0x7f7cdc0cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7ce0b9c700 1 -- 172.21.15.97:0/223413906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc0094a0 con 0x7f7cdc15a300 2023-07-06T03:54:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7cdbfff700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cdc071990 0x7f7cdc0c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:17.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7cdb7fe700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc0c9520 0x7f7cdc0cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:17.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.278+0000 7f7ce0b9c700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cdc15a300 0x7f7cdc0cb070 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4002230 tx=0x7f7cd400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:17.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.279+0000 7f7cd8ff9700 1 -- 172.21.15.97:0/223413906 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd400ac20 con 0x7f7cdc15a300 2023-07-06T03:54:17.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.279+0000 7f7cd8ff9700 1 -- 172.21.15.97:0/223413906 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd400ada0 con 0x7f7cdc15a300 2023-07-06T03:54:17.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.279+0000 7f7ce25ff700 1 -- 172.21.15.97:0/223413906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cdc0c3670 con 0x7f7cdc15a300 2023-07-06T03:54:17.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.279+0000 7f7cd8ff9700 1 -- 172.21.15.97:0/223413906 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4007310 con 0x7f7cdc15a300 2023-07-06T03:54:17.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.279+0000 7f7ce25ff700 1 -- 172.21.15.97:0/223413906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cdc0c3c30 con 0x7f7cdc15a300 2023-07-06T03:54:17.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.280+0000 7f7ce25ff700 1 -- 172.21.15.97:0/223413906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cdc04e340 con 0x7f7cdc15a300 2023-07-06T03:54:17.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.281+0000 7f7cd8ff9700 1 -- 172.21.15.97:0/223413906 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7cd4007470 con 0x7f7cdc15a300 2023-07-06T03:54:17.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.282+0000 7f7cd8ff9700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc806ace0 0x7f7cc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.282+0000 7f7cd8ff9700 1 -- 172.21.15.97:0/223413906 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7cd4088d10 con 0x7f7cdc15a300 2023-07-06T03:54:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.282+0000 7f7cdbfff700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc806ace0 0x7f7cc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:17.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.283+0000 7f7cdbfff700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc806ace0 0x7f7cc806d190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0090b0 tx=0x7f7ccc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:17.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.286+0000 7f7cd8ff9700 1 -- 172.21.15.97:0/223413906 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7cd408e020 con 0x7f7cdc15a300 2023-07-06T03:54:17.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.558+0000 7f7ce25ff700 1 -- 172.21.15.97:0/223413906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) v1 -- 0x7f7cdc0b45f0 con 0x7f7cdc15a300 2023-07-06T03:54:17.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.910+0000 7f7cd8ff9700 1 -- 172.21.15.97:0/223413906 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v465) v1 ==== 143+0+0 (secure 0 0 0) 0x7f7cd40180d0 con 0x7f7cdc15a300 2023-07-06T03:54:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 -- 172.21.15.97:0/223413906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc806ace0 msgr2=0x7f7cc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc806ace0 0x7f7cc806d190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0090b0 tx=0x7f7ccc01c040 comp rx=0 tx=0).stop 2023-07-06T03:54:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 -- 172.21.15.97:0/223413906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cdc15a300 msgr2=0x7f7cdc0cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cdc15a300 0x7f7cdc0cb070 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4002230 tx=0x7f7cd400a060 comp rx=0 tx=0).stop 2023-07-06T03:54:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 -- 172.21.15.97:0/223413906 shutdown_connections 2023-07-06T03:54:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7cc806ace0 0x7f7cc806d190 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cdc15a300 0x7f7cdc0cb070 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7cdc071990 0x7f7cdc0c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 --2- 172.21.15.97:0/223413906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7cdc0c9520 0x7f7cdc0cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:17.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 -- 172.21.15.97:0/223413906 >> 172.21.15.97:0/223413906 conn(0x7f7cdc0b04a0 msgr2=0x7f7cdc159310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:17.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 -- 172.21.15.97:0/223413906 shutdown_connections 2023-07-06T03:54:17.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:17.913+0000 7f7ce25ff700 1 -- 172.21.15.97:0/223413906 wait complete. 2023-07-06T03:54:17.933 INFO:tasks.cephfs_test_runner:test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-06T03:54:17.933 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:54:17.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2023-07-06T03:54:18.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.126+0000 7fd2911ff700 1 Processor -- start 2023-07-06T03:54:18.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.126+0000 7fd2911ff700 1 -- start start 2023-07-06T03:54:18.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.127+0000 7fd2911ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd28c15ff00 0x7fd28c0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:18.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.127+0000 7fd2911ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28c0b4740 con 0x7fd28c0c8d70 2023-07-06T03:54:18.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.127+0000 7fd2911ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28c1602d0 con 0x7fd28c0d10c0 2023-07-06T03:54:18.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.127+0000 7fd2911ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd28c0b60b0 con 0x7fd28c15ff00 2023-07-06T03:54:18.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.127+0000 7fd28a59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd28c0c8d70 0x7fd28c0d09b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60062/0 (socket says 172.21.15.97:60062) 2023-07-06T03:54:18.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.127+0000 7fd28a59c700 1 -- 172.21.15.97:0/3294355972 learned_addr learned my addr 172.21.15.97:0/3294355972 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd28ad9d700 1 --2- 172.21.15.97:0/3294355972 >> v2:172.21.15.146:3300/0 conn(0x7fd28c15ff00 0x7fd28c0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396867664 0 0) 0x7fd28c0b4740 con 0x7fd28c0c8d70 2023-07-06T03:54:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2740035b0 con 0x7fd28c0c8d70 2023-07-06T03:54:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 31124926 0 0) 0x7fd2740035b0 con 0x7fd28c0c8d70 2023-07-06T03:54:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 >> v2:172.21.15.146:3300/0 conn(0x7fd28c15ff00 msgr2=0x7fd28c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 --2- 172.21.15.97:0/3294355972 >> v2:172.21.15.146:3300/0 conn(0x7fd28c15ff00 0x7fd28c0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 >> v1:172.21.15.146:6789/0 conn(0x7fd28c0d10c0 legacy=0x7fd28c0b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c0c9800 con 0x7fd28c0c8d70 2023-07-06T03:54:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fd280001670 con 0x7fd28c0c8d70 2023-07-06T03:54:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 >> v1:172.21.15.97:6789/0 conn(0x7fd28c0c8d70 legacy=0x7fd28c0d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 --2- 172.21.15.97:0/3294355972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd274003a50 0x7fd274003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 --2- 172.21.15.97:0/3294355972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd274004850 0x7fd274004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 --2- 172.21.15.97:0/3294355972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274005250 0x7fd27400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c0b60b0 con 0x7fd274003a50 2023-07-06T03:54:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27400c160 con 0x7fd274005250 2023-07-06T03:54:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.128+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27400c320 con 0x7fd274004850 2023-07-06T03:54:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd28ad9d700 1 --2- 172.21.15.97:0/3294355972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd274003a50 0x7fd274003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd28b59e700 1 --2- 172.21.15.97:0/3294355972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274005250 0x7fd27400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd28a59c700 1 --2- 172.21.15.97:0/3294355972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd274004850 0x7fd274004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd28ad9d700 1 -- 172.21.15.97:0/3294355972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd274004850 msgr2=0x7fd274004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:18.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd28ad9d700 1 --2- 172.21.15.97:0/3294355972 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd274004850 0x7fd274004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd28ad9d700 1 -- 172.21.15.97:0/3294355972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274005250 msgr2=0x7fd27400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd28ad9d700 1 --2- 172.21.15.97:0/3294355972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd274005250 0x7fd27400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd28ad9d700 1 -- 172.21.15.97:0/3294355972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2740035b0 con 0x7fd274003a50 2023-07-06T03:54:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd28b59e700 1 -- 172.21.15.97:0/3294355972 reap_dead start 2023-07-06T03:54:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd28ad9d700 1 --2- 172.21.15.97:0/3294355972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd274003a50 0x7fd274003e60 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009b40 tx=0x7fd274005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02cb0ec4b77f25c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c005a20 con 0x7fd274003a50 2023-07-06T03:54:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c005b80 con 0x7fd274003a50 2023-07-06T03:54:18.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.129+0000 7fd289d9b700 1 -- 172.21.15.97:0/3294355972 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c00e6c0 con 0x7fd274003a50 2023-07-06T03:54:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.130+0000 7fd2911ff700 1 -- 172.21.15.97:0/3294355972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd274003a50 msgr2=0x7fd274003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.130+0000 7fd2911ff700 1 --2- 172.21.15.97:0/3294355972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd274003a50 0x7fd274003e60 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009b40 tx=0x7fd274005880 comp rx=0 tx=0).stop 2023-07-06T03:54:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.130+0000 7fd2911ff700 1 -- 172.21.15.97:0/3294355972 shutdown_connections 2023-07-06T03:54:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.130+0000 7fd2911ff700 1 --2- 172.21.15.97:0/3294355972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd274003a50 0x7fd274003e60 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.130+0000 7fd2911ff700 1 -- 172.21.15.97:0/3294355972 >> 172.21.15.97:0/3294355972 conn(0x7fd28c09c130 msgr2=0x7fd28c0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.130+0000 7fd2911ff700 1 -- 172.21.15.97:0/3294355972 shutdown_connections 2023-07-06T03:54:18.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.130+0000 7fd2911ff700 1 -- 172.21.15.97:0/3294355972 wait complete. 2023-07-06T03:54:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.132+0000 7fd2911ff700 1 Processor -- start 2023-07-06T03:54:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.132+0000 7fd2911ff700 1 -- start start 2023-07-06T03:54:18.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.133+0000 7fd2911ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd274003a50 0x7fd28c15b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.133+0000 7fd2911ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ca8a0 0x7fd28c0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.133+0000 7fd2911ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd28c0b2b60 0x7fd28c0b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.133+0000 7fd2911ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c0cb250 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.133+0000 7fd2911ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c0b0130 con 0x7fd28c0b2b60 2023-07-06T03:54:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.133+0000 7fd2911ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c0b3b60 con 0x7fd274003a50 2023-07-06T03:54:18.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.133+0000 7fd28a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ca8a0 0x7fd28c0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.133+0000 7fd28a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ca8a0 0x7fd28c0cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40920/0 (socket says 172.21.15.97:40920) 2023-07-06T03:54:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.133+0000 7fd28a59c700 1 -- 172.21.15.97:0/722699804 learned_addr learned my addr 172.21.15.97:0/722699804 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.133+0000 7fd28ad9d700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd274003a50 0x7fd28c15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.133+0000 7fd28b59e700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd28c0b2b60 0x7fd28c0b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd28a59c700 1 -- 172.21.15.97:0/722699804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd274003a50 msgr2=0x7fd28c15b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:18.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd28a59c700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd274003a50 0x7fd28c15b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd28a59c700 1 -- 172.21.15.97:0/722699804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd28c0b2b60 msgr2=0x7fd28c0b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd28a59c700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd28c0b2b60 0x7fd28c0b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd28a59c700 1 -- 172.21.15.97:0/722699804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c0b3880 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd28b59e700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd28c0b2b60 0x7fd28c0b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd28a59c700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ca8a0 0x7fd28c0cad10 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd28000e700 tx=0x7fd28000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd2737fe700 1 -- 172.21.15.97:0/722699804 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280012590 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd2737fe700 1 -- 172.21.15.97:0/722699804 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2800126f0 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd2911ff700 1 -- 172.21.15.97:0/722699804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28c0afae0 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd2737fe700 1 -- 172.21.15.97:0/722699804 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280012aa0 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.134+0000 7fd2911ff700 1 -- 172.21.15.97:0/722699804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28c15bb20 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.135+0000 7fd2911ff700 1 -- 172.21.15.97:0/722699804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28c0b02e0 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.136+0000 7fd2737fe700 1 -- 172.21.15.97:0/722699804 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd280012c00 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.137+0000 7fd2737fe700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd27806ace0 0x7fd27806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:18.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.137+0000 7fd2737fe700 1 -- 172.21.15.97:0/722699804 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd280088de0 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.137+0000 7fd28ad9d700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd27806ace0 0x7fd27806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:18.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.138+0000 7fd28ad9d700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd27806ace0 0x7fd27806d190 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000c00 tx=0x7fd27c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:18.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.141+0000 7fd2737fe700 1 -- 172.21.15.97:0/722699804 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd28c0b02e0 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.412+0000 7fd2911ff700 1 -- 172.21.15.97:0/722699804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) v1 -- 0x7fd28c06c600 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.912+0000 7fd2737fe700 1 -- 172.21.15.97:0/722699804 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v466) v1 ==== 154+0+0 (secure 0 0 0) 0x7fd280057cb0 con 0x7fd28c0ca8a0 2023-07-06T03:54:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 -- 172.21.15.97:0/722699804 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd27806ace0 msgr2=0x7fd27806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd27806ace0 0x7fd27806d190 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000c00 tx=0x7fd27c008030 comp rx=0 tx=0).stop 2023-07-06T03:54:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 -- 172.21.15.97:0/722699804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ca8a0 msgr2=0x7fd28c0cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ca8a0 0x7fd28c0cad10 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd28000e700 tx=0x7fd28000ea10 comp rx=0 tx=0).stop 2023-07-06T03:54:18.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 -- 172.21.15.97:0/722699804 shutdown_connections 2023-07-06T03:54:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd27806ace0 0x7fd27806d190 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd28c0b2b60 0x7fd28c0b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd274003a50 0x7fd28c15b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 --2- 172.21.15.97:0/722699804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ca8a0 0x7fd28c0cad10 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 -- 172.21.15.97:0/722699804 >> 172.21.15.97:0/722699804 conn(0x7fd28c09c130 msgr2=0x7fd28c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 -- 172.21.15.97:0/722699804 shutdown_connections 2023-07-06T03:54:18.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:18.915+0000 7fd2911ff700 1 -- 172.21.15.97:0/722699804 wait complete. 2023-07-06T03:54:18.936 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T03:54:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.130+0000 7f07817ff700 1 Processor -- start 2023-07-06T03:54:19.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.130+0000 7f07817ff700 1 -- start start 2023-07-06T03:54:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.131+0000 7f07817ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f077c15c610 0x7f077c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.131+0000 7f07817ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0cff90 con 0x7f077c06b360 2023-07-06T03:54:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.131+0000 7f07817ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0b5650 con 0x7f077c0d2210 2023-07-06T03:54:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.131+0000 7f07817ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c0d0780 con 0x7f077c15c610 2023-07-06T03:54:19.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.131+0000 7f077a7fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f077c06b360 0x7f077c0b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60072/0 (socket says 172.21.15.97:60072) 2023-07-06T03:54:19.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.131+0000 7f077a7fc700 1 -- 172.21.15.97:0/1001997431 learned_addr learned my addr 172.21.15.97:0/1001997431 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:19.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f077affd700 1 --2- 172.21.15.97:0/1001997431 >> v2:172.21.15.146:3300/0 conn(0x7f077c15c610 0x7f077c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508384281 0 0) 0x7f077c0cff90 con 0x7f077c06b360 2023-07-06T03:54:19.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07640035b0 con 0x7f077c06b360 2023-07-06T03:54:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1981641896 0 0) 0x7f07640035b0 con 0x7f077c06b360 2023-07-06T03:54:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 >> v2:172.21.15.146:3300/0 conn(0x7f077c15c610 msgr2=0x7f077c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 --2- 172.21.15.97:0/1001997431 >> v2:172.21.15.146:3300/0 conn(0x7f077c15c610 0x7f077c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 >> v1:172.21.15.146:6789/0 conn(0x7f077c0d2210 legacy=0x7f077c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0b5400 con 0x7f077c06b360 2023-07-06T03:54:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f0770001670 con 0x7f077c06b360 2023-07-06T03:54:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 >> v1:172.21.15.97:6789/0 conn(0x7f077c06b360 legacy=0x7f077c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f0764003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07640043f0 0x7f07640048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07640050b0 0x7f076400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0d0780 con 0x7f07640043f0 2023-07-06T03:54:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076400c000 con 0x7f0764003a00 2023-07-06T03:54:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.132+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076400c1c0 con 0x7f07640050b0 2023-07-06T03:54:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f077a7fc700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07640043f0 0x7f07640048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f077affd700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f0764003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f077b7fe700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07640050b0 0x7f076400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f077a7fc700 1 -- 172.21.15.97:0/1001997431 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07640050b0 msgr2=0x7f076400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f077a7fc700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07640050b0 0x7f076400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f077a7fc700 1 -- 172.21.15.97:0/1001997431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 msgr2=0x7f0764003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f077a7fc700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f0764003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f077a7fc700 1 -- 172.21.15.97:0/1001997431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07640035b0 con 0x7f07640043f0 2023-07-06T03:54:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f077affd700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764003a00 0x7f0764003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f077b7fe700 1 -- 172.21.15.97:0/1001997431 reap_dead start 2023-07-06T03:54:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f077a7fc700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07640043f0 0x7f07640048b0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f077000c2f0 tx=0x7f0764005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0551121605d8830 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077000ce30 con 0x7f07640043f0 2023-07-06T03:54:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0770005960 con 0x7f07640043f0 2023-07-06T03:54:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.133+0000 7f0779ffb700 1 -- 172.21.15.97:0/1001997431 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770005d10 con 0x7f07640043f0 2023-07-06T03:54:19.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.134+0000 7f07817ff700 1 -- 172.21.15.97:0/1001997431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07640043f0 msgr2=0x7f07640048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:19.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.134+0000 7f07817ff700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07640043f0 0x7f07640048b0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f077000c2f0 tx=0x7f0764005880 comp rx=0 tx=0).stop 2023-07-06T03:54:19.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.134+0000 7f07817ff700 1 -- 172.21.15.97:0/1001997431 shutdown_connections 2023-07-06T03:54:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.134+0000 7f07817ff700 1 --2- 172.21.15.97:0/1001997431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07640043f0 0x7f07640048b0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.134+0000 7f07817ff700 1 -- 172.21.15.97:0/1001997431 >> 172.21.15.97:0/1001997431 conn(0x7f077c06a6c0 msgr2=0x7f077c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.134+0000 7f07817ff700 1 -- 172.21.15.97:0/1001997431 shutdown_connections 2023-07-06T03:54:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.134+0000 7f07817ff700 1 -- 172.21.15.97:0/1001997431 wait complete. 2023-07-06T03:54:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.136+0000 7f07817ff700 1 Processor -- start 2023-07-06T03:54:19.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.136+0000 7f07817ff700 1 -- start start 2023-07-06T03:54:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f07817ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07640043f0 0x7f077c0b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f07817ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f077c13fb70 0x7f077c0b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f07817ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f077c13ffe0 0x7f077c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f07817ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c142b40 con 0x7f077c13ffe0 2023-07-06T03:54:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f07817ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c15dce0 con 0x7f077c13fb70 2023-07-06T03:54:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f07817ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c15df80 con 0x7f07640043f0 2023-07-06T03:54:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f077b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f077c13ffe0 0x7f077c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f077b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f077c13ffe0 0x7f077c0b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40932/0 (socket says 172.21.15.97:40932) 2023-07-06T03:54:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f077b7fe700 1 -- 172.21.15.97:0/4152686319 learned_addr learned my addr 172.21.15.97:0/4152686319 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f077affd700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07640043f0 0x7f077c0b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f077a7fc700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f077c13fb70 0x7f077c0b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f077b7fe700 1 -- 172.21.15.97:0/4152686319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07640043f0 msgr2=0x7f077c0b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f077b7fe700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07640043f0 0x7f077c0b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f077b7fe700 1 -- 172.21.15.97:0/4152686319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f077c13fb70 msgr2=0x7f077c0b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.137+0000 7f077b7fe700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f077c13fb70 0x7f077c0b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.138+0000 7f077b7fe700 1 -- 172.21.15.97:0/4152686319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c15e260 con 0x7f077c13ffe0 2023-07-06T03:54:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.138+0000 7f077b7fe700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f077c13ffe0 0x7f077c0b60a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f077c0ce810 tx=0x7f077400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.138+0000 7f07637fe700 1 -- 172.21.15.97:0/4152686319 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774003c10 con 0x7f077c13ffe0 2023-07-06T03:54:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.138+0000 7f07637fe700 1 -- 172.21.15.97:0/4152686319 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0774003d70 con 0x7f077c13ffe0 2023-07-06T03:54:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.138+0000 7f07817ff700 1 -- 172.21.15.97:0/4152686319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c15e540 con 0x7f077c13ffe0 2023-07-06T03:54:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.138+0000 7f07637fe700 1 -- 172.21.15.97:0/4152686319 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774005310 con 0x7f077c13ffe0 2023-07-06T03:54:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.138+0000 7f07817ff700 1 -- 172.21.15.97:0/4152686319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c142070 con 0x7f077c13ffe0 2023-07-06T03:54:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.139+0000 7f07817ff700 1 -- 172.21.15.97:0/4152686319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c15e0c0 con 0x7f077c13ffe0 2023-07-06T03:54:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.140+0000 7f07637fe700 1 -- 172.21.15.97:0/4152686319 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0774005470 con 0x7f077c13ffe0 2023-07-06T03:54:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.142+0000 7f07637fe700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f076806ace0 0x7f076806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.142+0000 7f07637fe700 1 -- 172.21.15.97:0/4152686319 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0774089210 con 0x7f077c13ffe0 2023-07-06T03:54:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.142+0000 7f077affd700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f076806ace0 0x7f076806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.142+0000 7f077affd700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f076806ace0 0x7f076806d190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f077c063c30 tx=0x7f076c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:19.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.146+0000 7f07637fe700 1 -- 172.21.15.97:0/4152686319 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f077c15e0c0 con 0x7f077c13ffe0 2023-07-06T03:54:19.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.339+0000 7f07817ff700 1 -- 172.21.15.97:0/4152686319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f077c15e0c0 con 0x7f077c13ffe0 2023-07-06T03:54:19.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.343+0000 7f07637fe700 1 -- 172.21.15.97:0/4152686319 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f077c15e0c0 con 0x7f077c13ffe0 2023-07-06T03:54:19.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 -- 172.21.15.97:0/4152686319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f076806ace0 msgr2=0x7f076806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f076806ace0 0x7f076806d190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f077c063c30 tx=0x7f076c008040 comp rx=0 tx=0).stop 2023-07-06T03:54:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 -- 172.21.15.97:0/4152686319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f077c13ffe0 msgr2=0x7f077c0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f077c13ffe0 0x7f077c0b60a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f077c0ce810 tx=0x7f077400ee50 comp rx=0 tx=0).stop 2023-07-06T03:54:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 -- 172.21.15.97:0/4152686319 shutdown_connections 2023-07-06T03:54:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f076806ace0 0x7f076806d190 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f077c13ffe0 0x7f077c0b60a0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f07640043f0 0x7f077c0b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 --2- 172.21.15.97:0/4152686319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f077c13fb70 0x7f077c0b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 -- 172.21.15.97:0/4152686319 >> 172.21.15.97:0/4152686319 conn(0x7f077c06a6c0 msgr2=0x7f077c0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 -- 172.21.15.97:0/4152686319 shutdown_connections 2023-07-06T03:54:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.346+0000 7f07817ff700 1 -- 172.21.15.97:0/4152686319 wait complete. 2023-07-06T03:54:19.366 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:54:19.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:54:19.465 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:54:19.573 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:54:19.574 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:54:19.574 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:19.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:54:19.620 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:54:19.620 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:19.667 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:54:19.667 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:54:19.668 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:19.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:54:19.714 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:54:19.714 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:19.760 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:54:19.761 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T03:54:19.761 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:19.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.955+0000 7fda421ff700 1 Processor -- start 2023-07-06T03:54:19.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.955+0000 7fda421ff700 1 -- start start 2023-07-06T03:54:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.956+0000 7fda421ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c0d10b0 0x7fda3c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.956+0000 7fda421ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda3c15d940 0x7fda3c06c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda421ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3c160f70 con 0x7fda3c160200 2023-07-06T03:54:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda421ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3c160d90 con 0x7fda3c15d940 2023-07-06T03:54:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda421ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c0d1480 con 0x7fda3c0d10b0 2023-07-06T03:54:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda3affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda3c15d940 0x7fda3c06c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda3affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda3c15d940 0x7fda3c06c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56966/0 (socket says 172.21.15.97:56966) 2023-07-06T03:54:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda3affd700 1 -- 172.21.15.97:0/2037485250 learned_addr learned my addr 172.21.15.97:0/2037485250 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:19.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda3b7fe700 1 --2- 172.21.15.97:0/2037485250 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c0d10b0 0x7fda3c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda3affd700 1 -- 172.21.15.97:0/2037485250 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c0d10b0 msgr2=0x7fda3c06c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:19.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda3affd700 1 --2- 172.21.15.97:0/2037485250 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c0d10b0 0x7fda3c06c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda3a7fc700 1 -- 172.21.15.97:0/2037485250 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265162936 0 0) 0x7fda3c160f70 con 0x7fda3c160200 2023-07-06T03:54:19.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda3affd700 1 -- 172.21.15.97:0/2037485250 >> v1:172.21.15.97:6789/0 conn(0x7fda3c160200 legacy=0x7fda3c160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:19.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda3affd700 1 -- 172.21.15.97:0/2037485250 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c0d0590 con 0x7fda3c15d940 2023-07-06T03:54:19.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.957+0000 7fda3affd700 1 --2- 172.21.15.97:0/2037485250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c15d940 0x7fda3c06c7d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fda300098f0 tx=0x7fda30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b94cc6c3da011d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:19.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.958+0000 7fda3a7fc700 1 -- 172.21.15.97:0/2037485250 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3000b8a0 con 0x7fda3c15d940 2023-07-06T03:54:19.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.958+0000 7fda3a7fc700 1 -- 172.21.15.97:0/2037485250 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3000ba00 con 0x7fda3c15d940 2023-07-06T03:54:19.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.958+0000 7fda3a7fc700 1 -- 172.21.15.97:0/2037485250 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3000bd90 con 0x7fda3c15d940 2023-07-06T03:54:19.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.958+0000 7fda421ff700 1 -- 172.21.15.97:0/2037485250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c15d940 msgr2=0x7fda3c06c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:19.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.958+0000 7fda421ff700 1 --2- 172.21.15.97:0/2037485250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c15d940 0x7fda3c06c7d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fda300098f0 tx=0x7fda30009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:19.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.958+0000 7fda421ff700 1 -- 172.21.15.97:0/2037485250 shutdown_connections 2023-07-06T03:54:19.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.958+0000 7fda421ff700 1 --2- 172.21.15.97:0/2037485250 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c0d10b0 0x7fda3c06c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.958+0000 7fda421ff700 1 --2- 172.21.15.97:0/2037485250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c15d940 0x7fda3c06c7d0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.958+0000 7fda421ff700 1 -- 172.21.15.97:0/2037485250 >> 172.21.15.97:0/2037485250 conn(0x7fda3c06a060 msgr2=0x7fda3c06a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:19.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.958+0000 7fda421ff700 1 -- 172.21.15.97:0/2037485250 shutdown_connections 2023-07-06T03:54:19.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.958+0000 7fda421ff700 1 -- 172.21.15.97:0/2037485250 wait complete. 2023-07-06T03:54:19.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.960+0000 7fda421ff700 1 Processor -- start 2023-07-06T03:54:19.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.960+0000 7fda421ff700 1 -- start start 2023-07-06T03:54:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.961+0000 7fda421ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0d10b0 0x7fda3c0c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.961+0000 7fda421ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda3c15d940 0x7fda3c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.961+0000 7fda421ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c160200 0x7fda3c0b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.961+0000 7fda421ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c0c6790 con 0x7fda3c0d10b0 2023-07-06T03:54:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.961+0000 7fda421ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c0c5ec0 con 0x7fda3c15d940 2023-07-06T03:54:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.961+0000 7fda421ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c0aeac0 con 0x7fda3c160200 2023-07-06T03:54:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.961+0000 7fda3b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0d10b0 0x7fda3c0c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.961+0000 7fda3b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0d10b0 0x7fda3c0c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56970/0 (socket says 172.21.15.97:56970) 2023-07-06T03:54:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.961+0000 7fda3b7fe700 1 -- 172.21.15.97:0/1957470871 learned_addr learned my addr 172.21.15.97:0/1957470871 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.962+0000 7fda3b7fe700 1 -- 172.21.15.97:0/1957470871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c160200 msgr2=0x7fda3c0b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.962+0000 7fda3b7fe700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c160200 0x7fda3c0b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.962+0000 7fda3b7fe700 1 -- 172.21.15.97:0/1957470871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda3c15d940 msgr2=0x7fda3c156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.962+0000 7fda3b7fe700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda3c15d940 0x7fda3c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.962+0000 7fda3b7fe700 1 -- 172.21.15.97:0/1957470871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda30009550 con 0x7fda3c0d10b0 2023-07-06T03:54:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.962+0000 7fda3b7fe700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0d10b0 0x7fda3c0c7120 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00e730 tx=0x7fda2c00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.962+0000 7fda23fff700 1 -- 172.21.15.97:0/1957470871 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c013610 con 0x7fda3c0d10b0 2023-07-06T03:54:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.962+0000 7fda23fff700 1 -- 172.21.15.97:0/1957470871 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c013770 con 0x7fda3c0d10b0 2023-07-06T03:54:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.962+0000 7fda421ff700 1 -- 172.21.15.97:0/1957470871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3c0c6140 con 0x7fda3c0d10b0 2023-07-06T03:54:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.962+0000 7fda23fff700 1 -- 172.21.15.97:0/1957470871 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c013b20 con 0x7fda3c0d10b0 2023-07-06T03:54:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.962+0000 7fda421ff700 1 -- 172.21.15.97:0/1957470871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3c0ae720 con 0x7fda3c0d10b0 2023-07-06T03:54:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.963+0000 7fda421ff700 1 -- 172.21.15.97:0/1957470871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3c0cee80 con 0x7fda3c0d10b0 2023-07-06T03:54:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.964+0000 7fda23fff700 1 -- 172.21.15.97:0/1957470871 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda2c013c80 con 0x7fda3c0d10b0 2023-07-06T03:54:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.966+0000 7fda23fff700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2806ace0 0x7fda2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.966+0000 7fda23fff700 1 -- 172.21.15.97:0/1957470871 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fda2c088e00 con 0x7fda3c0d10b0 2023-07-06T03:54:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.966+0000 7fda3affd700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2806ace0 0x7fda2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:19.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.966+0000 7fda3affd700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2806ace0 0x7fda2806d190 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fda3c15dd30 tx=0x7fda30010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:19.970+0000 7fda23fff700 1 -- 172.21.15.97:0/1957470871 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda2c057cd0 con 0x7fda3c0d10b0 2023-07-06T03:54:20.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.207+0000 7fda421ff700 1 -- 172.21.15.97:0/1957470871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda3c04e340 con 0x7fda3c0d10b0 2023-07-06T03:54:20.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.207+0000 7fda23fff700 1 -- 172.21.15.97:0/1957470871 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fda2c021090 con 0x7fda3c0d10b0 2023-07-06T03:54:20.209 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:20.209 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/387292825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":387292825},{"type":"v1","addr":"172.21.15.146:6833","nonce":387292825}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690380561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690380561},{"type":"v1","addr":"172.21.15.97:6837","nonce":690380561}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3897834271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3897834271},{"type":"v1","addr":"172.21.15.146:6835","nonce":3897834271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3826869439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3826869439},{"type":"v1","addr":"172.21.15.97:6836","nonce":3826869439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[]} 2023-07-06T03:54:20.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 -- 172.21.15.97:0/1957470871 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2806ace0 msgr2=0x7fda2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:20.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2806ace0 0x7fda2806d190 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fda3c15dd30 tx=0x7fda30010000 comp rx=0 tx=0).stop 2023-07-06T03:54:20.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 -- 172.21.15.97:0/1957470871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0d10b0 msgr2=0x7fda3c0c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:20.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0d10b0 0x7fda3c0c7120 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00e730 tx=0x7fda2c00eaf0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 -- 172.21.15.97:0/1957470871 shutdown_connections 2023-07-06T03:54:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda2806ace0 0x7fda2806d190 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda3c160200 0x7fda3c0b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0d10b0 0x7fda3c0c7120 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 --2- 172.21.15.97:0/1957470871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda3c15d940 0x7fda3c156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 -- 172.21.15.97:0/1957470871 >> 172.21.15.97:0/1957470871 conn(0x7fda3c06a060 msgr2=0x7fda3c15dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 -- 172.21.15.97:0/1957470871 shutdown_connections 2023-07-06T03:54:20.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.210+0000 7fda421ff700 1 -- 172.21.15.97:0/1957470871 wait complete. 2023-07-06T03:54:20.214 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 73 2023-07-06T03:54:20.230 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T03:54:20.230 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T03:54:20.230 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T03:54:20.230 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:54:20.231 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T03:54:20.231 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T03:54:20.231 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T03:54:20.231 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:54:20.231 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T03:54:20.232 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T03:54:20.232 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T03:54:20.232 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:54:20.232 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T03:54:20.233 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T03:54:20.233 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T03:54:20.233 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:54:20.243 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T03:54:20.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T03:54:20.245 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T03:54:20.245 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T03:54:20.248 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T03:54:20.249 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T03:54:20.249 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T03:54:20.250 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T03:54:20.287 INFO:tasks.ceph.mds.a:Started 2023-07-06T03:54:20.288 DEBUG:teuthology.parallel:result is None 2023-07-06T03:54:20.290 INFO:tasks.ceph.mds.c:Started 2023-07-06T03:54:20.290 DEBUG:teuthology.parallel:result is None 2023-07-06T03:54:20.293 INFO:tasks.ceph.mds.b:Started 2023-07-06T03:54:20.293 DEBUG:teuthology.parallel:result is None 2023-07-06T03:54:20.296 INFO:tasks.ceph.mds.d:Started 2023-07-06T03:54:20.296 DEBUG:teuthology.parallel:result is None 2023-07-06T03:54:20.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T03:54:20.434 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:54:20.439 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:54:20.441 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:54:20.442 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-07-06T03:54:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.502+0000 7f71e93ff700 1 Processor -- start 2023-07-06T03:54:20.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.502+0000 7f71e93ff700 1 -- start start 2023-07-06T03:54:20.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.503+0000 7f71e93ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f71e415cae0 0x7f71e415d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:20.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.503+0000 7f71e93ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e415de00 con 0x7f71e406b170 2023-07-06T03:54:20.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.503+0000 7f71e93ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e40ceb20 con 0x7f71e40cfbc0 2023-07-06T03:54:20.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.503+0000 7f71e93ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e40b25a0 con 0x7f71e415cae0 2023-07-06T03:54:20.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.504+0000 7f71e27fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f71e415cae0 0x7f71e415d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:20.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.504+0000 7f71e2ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f71e406b170 0x7f71e406b540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53498/0 (socket says 172.21.15.97:53498) 2023-07-06T03:54:20.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.504+0000 7f71e2ffd700 1 -- 172.21.15.97:0/1623398531 learned_addr learned my addr 172.21.15.97:0/1623398531 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:20.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.504+0000 7f71e27fc700 1 -- 172.21.15.97:0/1623398531 >> v1:172.21.15.146:6789/0 conn(0x7f71e40cfbc0 legacy=0x7f71e415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:20.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.504+0000 7f71e27fc700 1 -- 172.21.15.97:0/1623398531 >> v1:172.21.15.97:6789/0 conn(0x7f71e406b170 legacy=0x7f71e406b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:20.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.504+0000 7f71e27fc700 1 -- 172.21.15.97:0/1623398531 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e40cee00 con 0x7f71e415cae0 2023-07-06T03:54:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.504+0000 7f71e1ffb700 1 -- 172.21.15.97:0/1623398531 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511427041 0 0) 0x7f71d4001290 con 0x7f71e406b170 2023-07-06T03:54:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.504+0000 7f71e27fc700 1 --2- 172.21.15.97:0/1623398531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e415cae0 0x7f71e415d8c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009840 tx=0x7f71d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a9ccdf9cef6434 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.505+0000 7f71e1ffb700 1 -- 172.21.15.97:0/1623398531 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8011830 con 0x7f71e415cae0 2023-07-06T03:54:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.505+0000 7f71e1ffb700 1 -- 172.21.15.97:0/1623398531 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d8011990 con 0x7f71e415cae0 2023-07-06T03:54:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.505+0000 7f71e1ffb700 1 -- 172.21.15.97:0/1623398531 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8011d20 con 0x7f71e415cae0 2023-07-06T03:54:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.505+0000 7f71e93ff700 1 -- 172.21.15.97:0/1623398531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e415cae0 msgr2=0x7f71e415d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.505+0000 7f71e93ff700 1 --2- 172.21.15.97:0/1623398531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e415cae0 0x7f71e415d8c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009840 tx=0x7f71d8009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:20.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.505+0000 7f71e93ff700 1 -- 172.21.15.97:0/1623398531 shutdown_connections 2023-07-06T03:54:20.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.505+0000 7f71e93ff700 1 --2- 172.21.15.97:0/1623398531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e415cae0 0x7f71e415d8c0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.505+0000 7f71e93ff700 1 -- 172.21.15.97:0/1623398531 >> 172.21.15.97:0/1623398531 conn(0x7f71e4099f10 msgr2=0x7f71e40d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:20.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.505+0000 7f71e93ff700 1 -- 172.21.15.97:0/1623398531 shutdown_connections 2023-07-06T03:54:20.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.505+0000 7f71e93ff700 1 -- 172.21.15.97:0/1623398531 wait complete. 2023-07-06T03:54:20.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.508+0000 7f71e93ff700 1 Processor -- start 2023-07-06T03:54:20.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.508+0000 7f71e93ff700 1 -- start start 2023-07-06T03:54:20.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.509+0000 7f71e93ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71e406b170 0x7f71e40c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:20.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.509+0000 7f71e93ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e40cfbc0 0x7f71e409a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:20.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.509+0000 7f71e93ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e415cae0 0x7f71e41f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:20.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.509+0000 7f71e93ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e415b710 con 0x7f71e40cfbc0 2023-07-06T03:54:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.509+0000 7f71e93ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e41e78b0 con 0x7f71e415cae0 2023-07-06T03:54:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.509+0000 7f71e93ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e41e7b90 con 0x7f71e406b170 2023-07-06T03:54:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.509+0000 7f71e27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e40cfbc0 0x7f71e409a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.509+0000 7f71e27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e40cfbc0 0x7f71e409a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57046/0 (socket says 172.21.15.97:57046) 2023-07-06T03:54:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.509+0000 7f71e27fc700 1 -- 172.21.15.97:0/3866292354 learned_addr learned my addr 172.21.15.97:0/3866292354 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71e2ffd700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71e406b170 0x7f71e40c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71e37fe700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e415cae0 0x7f71e41f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71e27fc700 1 -- 172.21.15.97:0/3866292354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71e406b170 msgr2=0x7f71e40c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71e27fc700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71e406b170 0x7f71e40c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71e27fc700 1 -- 172.21.15.97:0/3866292354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e415cae0 msgr2=0x7f71e41f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71e27fc700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e415cae0 0x7f71e41f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71e27fc700 1 -- 172.21.15.97:0/3866292354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d80094a0 con 0x7f71e40cfbc0 2023-07-06T03:54:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71e37fe700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e415cae0 0x7f71e41f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71e27fc700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e40cfbc0 0x7f71e409a880 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f71d8014040 tx=0x7f71d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71cb7fe700 1 -- 172.21.15.97:0/3866292354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d801a980 con 0x7f71e40cfbc0 2023-07-06T03:54:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71cb7fe700 1 -- 172.21.15.97:0/3866292354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d801aae0 con 0x7f71e40cfbc0 2023-07-06T03:54:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71cb7fe700 1 -- 172.21.15.97:0/3866292354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d801ae70 con 0x7f71e40cfbc0 2023-07-06T03:54:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71e93ff700 1 -- 172.21.15.97:0/3866292354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e41e7e70 con 0x7f71e40cfbc0 2023-07-06T03:54:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.510+0000 7f71e93ff700 1 -- 172.21.15.97:0/3866292354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e41e8430 con 0x7f71e40cfbc0 2023-07-06T03:54:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.511+0000 7f71e93ff700 1 -- 172.21.15.97:0/3866292354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e404e340 con 0x7f71e40cfbc0 2023-07-06T03:54:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.512+0000 7f71cb7fe700 1 -- 172.21.15.97:0/3866292354 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f71d8022070 con 0x7f71e40cfbc0 2023-07-06T03:54:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.513+0000 7f71cb7fe700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71d006ace0 0x7f71d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.513+0000 7f71cb7fe700 1 -- 172.21.15.97:0/3866292354 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f71d808a180 con 0x7f71e40cfbc0 2023-07-06T03:54:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.513+0000 7f71e2ffd700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71d006ace0 0x7f71d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.514+0000 7f71e2ffd700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71d006ace0 0x7f71d006d190 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f71d4000c00 tx=0x7f71d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:20.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.517+0000 7f71cb7fe700 1 -- 172.21.15.97:0/3866292354 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f71d8055540 con 0x7f71e40cfbc0 2023-07-06T03:54:20.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.727+0000 7f71e93ff700 1 -- 172.21.15.97:0/3866292354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f71e406b900 con 0x7f71e40cfbc0 2023-07-06T03:54:20.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.928+0000 7f71cb7fe700 1 -- 172.21.15.97:0/3866292354 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v106) v1 ==== 97+0+0 (secure 0 0 0) 0x7f71d8058b60 con 0x7f71e40cfbc0 2023-07-06T03:54:20.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 -- 172.21.15.97:0/3866292354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71d006ace0 msgr2=0x7f71d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71d006ace0 0x7f71d006d190 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f71d4000c00 tx=0x7f71d4009040 comp rx=0 tx=0).stop 2023-07-06T03:54:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 -- 172.21.15.97:0/3866292354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e40cfbc0 msgr2=0x7f71e409a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e40cfbc0 0x7f71e409a880 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f71d8014040 tx=0x7f71d8004750 comp rx=0 tx=0).stop 2023-07-06T03:54:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 -- 172.21.15.97:0/3866292354 shutdown_connections 2023-07-06T03:54:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f71d006ace0 0x7f71d006d190 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e415cae0 0x7f71e41f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f71e406b170 0x7f71e40c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 --2- 172.21.15.97:0/3866292354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e40cfbc0 0x7f71e409a880 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:20.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 -- 172.21.15.97:0/3866292354 >> 172.21.15.97:0/3866292354 conn(0x7f71e4099f10 msgr2=0x7f71e40b61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:20.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 -- 172.21.15.97:0/3866292354 shutdown_connections 2023-07-06T03:54:20.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:20.931+0000 7f71e93ff700 1 -- 172.21.15.97:0/3866292354 wait complete. 2023-07-06T03:54:20.935 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-07-06T03:54:20.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T03:54:21.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.145+0000 7f576afff700 1 Processor -- start 2023-07-06T03:54:21.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.145+0000 7f576afff700 1 -- start start 2023-07-06T03:54:21.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.146+0000 7f576afff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5764071a00 0x7f576406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.146+0000 7f576afff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f576415ce90 0x7f5764158d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.146+0000 7f576afff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576406ddb0 con 0x7f5764158960 2023-07-06T03:54:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.146+0000 7f576afff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576415d310 con 0x7f5764071a00 2023-07-06T03:54:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.146+0000 7f576afff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576406bbf0 con 0x7f576415ce90 2023-07-06T03:54:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f5763fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5764071a00 0x7f576406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f5763fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5764071a00 0x7f576406d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57058/0 (socket says 172.21.15.97:57058) 2023-07-06T03:54:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f5763fff700 1 -- 172.21.15.97:0/2507806880 learned_addr learned my addr 172.21.15.97:0/2507806880 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f576959c700 1 --2- 172.21.15.97:0/2507806880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f576415ce90 0x7f5764158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:21.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f5763fff700 1 -- 172.21.15.97:0/2507806880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f576415ce90 msgr2=0x7f5764158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:21.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f5763fff700 1 --2- 172.21.15.97:0/2507806880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f576415ce90 0x7f5764158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f5763fff700 1 -- 172.21.15.97:0/2507806880 >> v1:172.21.15.146:6789/0 conn(0x7f5764158960 legacy=0x7f57640712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:21.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f5763fff700 1 -- 172.21.15.97:0/2507806880 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576406bed0 con 0x7f5764071a00 2023-07-06T03:54:21.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f5763fff700 1 --2- 172.21.15.97:0/2507806880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764071a00 0x7f576406d6f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f575000aa90 tx=0x7f575000ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53dcf4bc4f1c909c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:21.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f57637fe700 1 -- 172.21.15.97:0/2507806880 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575000cb70 con 0x7f5764071a00 2023-07-06T03:54:21.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f57637fe700 1 -- 172.21.15.97:0/2507806880 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575000ccd0 con 0x7f5764071a00 2023-07-06T03:54:21.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.147+0000 7f57637fe700 1 -- 172.21.15.97:0/2507806880 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575000f690 con 0x7f5764071a00 2023-07-06T03:54:21.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.148+0000 7f576afff700 1 -- 172.21.15.97:0/2507806880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764071a00 msgr2=0x7f576406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:21.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.148+0000 7f576afff700 1 --2- 172.21.15.97:0/2507806880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764071a00 0x7f576406d6f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f575000aa90 tx=0x7f575000ada0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.148+0000 7f576afff700 1 -- 172.21.15.97:0/2507806880 shutdown_connections 2023-07-06T03:54:21.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.148+0000 7f576afff700 1 --2- 172.21.15.97:0/2507806880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f576415ce90 0x7f5764158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.148+0000 7f576afff700 1 --2- 172.21.15.97:0/2507806880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764071a00 0x7f576406d6f0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.148+0000 7f576afff700 1 -- 172.21.15.97:0/2507806880 >> 172.21.15.97:0/2507806880 conn(0x7f57640ce8c0 msgr2=0x7f57640cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:21.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.148+0000 7f576afff700 1 -- 172.21.15.97:0/2507806880 shutdown_connections 2023-07-06T03:54:21.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.148+0000 7f576afff700 1 -- 172.21.15.97:0/2507806880 wait complete. 2023-07-06T03:54:21.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.150+0000 7f576afff700 1 Processor -- start 2023-07-06T03:54:21.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.150+0000 7f576afff700 1 -- start start 2023-07-06T03:54:21.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.151+0000 7f576afff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5764071a00 0x7f57640b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.151+0000 7f576afff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764158960 0x7f5764151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.151+0000 7f576afff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f576415ce90 0x7f57640af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.151+0000 7f576afff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640b23d0 con 0x7f5764158960 2023-07-06T03:54:21.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.151+0000 7f576afff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57641563e0 con 0x7f576415ce90 2023-07-06T03:54:21.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.151+0000 7f576afff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57641566c0 con 0x7f5764071a00 2023-07-06T03:54:21.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.151+0000 7f5763fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764158960 0x7f5764151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:21.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.151+0000 7f5763fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764158960 0x7f5764151d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57072/0 (socket says 172.21.15.97:57072) 2023-07-06T03:54:21.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.151+0000 7f5763fff700 1 -- 172.21.15.97:0/2692219691 learned_addr learned my addr 172.21.15.97:0/2692219691 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:21.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f576959c700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f576415ce90 0x7f57640af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:21.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f5768d9b700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5764071a00 0x7f57640b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:21.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f5763fff700 1 -- 172.21.15.97:0/2692219691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5764071a00 msgr2=0x7f57640b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:21.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f5763fff700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5764071a00 0x7f57640b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f5763fff700 1 -- 172.21.15.97:0/2692219691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f576415ce90 msgr2=0x7f57640af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:21.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f5763fff700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f576415ce90 0x7f57640af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f5763fff700 1 -- 172.21.15.97:0/2692219691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575000a6f0 con 0x7f5764158960 2023-07-06T03:54:21.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f5763fff700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764158960 0x7f5764151d30 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f575000ade0 tx=0x7f5750004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:21.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f57617fa700 1 -- 172.21.15.97:0/2692219691 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575000a570 con 0x7f5764158960 2023-07-06T03:54:21.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f57617fa700 1 -- 172.21.15.97:0/2692219691 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57500045d0 con 0x7f5764158960 2023-07-06T03:54:21.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f576afff700 1 -- 172.21.15.97:0/2692219691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57641569a0 con 0x7f5764158960 2023-07-06T03:54:21.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f57617fa700 1 -- 172.21.15.97:0/2692219691 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575000fc70 con 0x7f5764158960 2023-07-06T03:54:21.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.152+0000 7f576afff700 1 -- 172.21.15.97:0/2692219691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5764156f60 con 0x7f5764158960 2023-07-06T03:54:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.153+0000 7f576afff700 1 -- 172.21.15.97:0/2692219691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5764067830 con 0x7f5764158960 2023-07-06T03:54:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.154+0000 7f57617fa700 1 -- 172.21.15.97:0/2692219691 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57500036a0 con 0x7f5764158960 2023-07-06T03:54:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.155+0000 7f57617fa700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574806ace0 0x7f574806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.155+0000 7f57617fa700 1 -- 172.21.15.97:0/2692219691 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5750013070 con 0x7f5764158960 2023-07-06T03:54:21.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.156+0000 7f5768d9b700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574806ace0 0x7f574806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:21.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.156+0000 7f5768d9b700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574806ace0 0x7f574806d190 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5758000f60 tx=0x7f5758009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:21.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.160+0000 7f57617fa700 1 -- 172.21.15.97:0/2692219691 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f57500564d0 con 0x7f5764158960 2023-07-06T03:54:21.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.432+0000 7f576afff700 1 -- 172.21.15.97:0/2692219691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f576404e340 con 0x7f5764158960 2023-07-06T03:54:21.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.433+0000 7f57617fa700 1 -- 172.21.15.97:0/2692219691 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5750059af0 con 0x7f5764158960 2023-07-06T03:54:21.435 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:21.435 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:54:21.435 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-07-06T03:54:21.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.435 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-07-06T03:54:21.435 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkczuvGRAAUKjfmMyKOTCYzOQmtf3n3A==", 2023-07-06T03:54:21.435 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.436 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:54:21.436 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:54:21.436 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:54:21.436 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.437 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-07-06T03:54:21.437 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkFHvUMBAA2WgMcUUf+Qi8T2ZjhfWaHA==", 2023-07-06T03:54:21.437 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.437 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:54:21.437 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:54:21.437 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:54:21.437 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.438 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.438 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-07-06T03:54:21.438 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZk0N4+JRAA3tjqoIBPpZ6C7TMmOOg7lw==", 2023-07-06T03:54:21.438 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.438 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:54:21.439 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:54:21.439 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:54:21.439 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.440 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-07-06T03:54:21.440 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkJnHUABAAwIMH2DmoHaQN3frJCgRiQw==", 2023-07-06T03:54:21.440 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.441 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:54:21.441 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:54:21.441 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:54:21.441 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.441 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.442 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-07-06T03:54:21.442 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCLOKZkShOIABAAzIBHDcNkmnDOLSY7Qdb+FA==", 2023-07-06T03:54:21.442 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.442 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:54:21.442 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:54:21.442 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.443 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.443 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-07-06T03:54:21.443 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCOOKZkq1t7AxAAwihuhimbjitZDeki1QuMug==", 2023-07-06T03:54:21.443 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.443 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:54:21.444 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:54:21.444 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.444 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.444 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-07-06T03:54:21.444 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCROKZkdYv4BhAApTHg/pmlC0lO7hVTkimEgw==", 2023-07-06T03:54:21.445 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.445 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:54:21.445 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:54:21.445 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.446 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-07-06T03:54:21.446 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCUOKZkqhuMChAAsV9xG6g1dofrpWt3F8tw5w==", 2023-07-06T03:54:21.446 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.446 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:54:21.446 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:54:21.446 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.446 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.447 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-07-06T03:54:21.447 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCaOKZk1Q5aIBAABU1+afwIihm+g4xylryz/A==", 2023-07-06T03:54:21.447 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.447 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:54:21.447 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:54:21.448 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.448 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.448 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-07-06T03:54:21.448 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCdOKZkRskIJBAA+821aoG98aEV7h4FTINQFA==", 2023-07-06T03:54:21.449 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.449 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:54:21.449 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:54:21.449 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.449 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-07-06T03:54:21.450 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCgOKZk9qnMJxAATcWG+a/9Hl1AuIL/TSrdEA==", 2023-07-06T03:54:21.450 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.450 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:54:21.450 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:54:21.450 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.450 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.451 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-07-06T03:54:21.451 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCjOKZk6wJ/KxAAWnZxt7h7OJhQ+bqdel4+iQ==", 2023-07-06T03:54:21.451 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.451 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:54:21.451 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:54:21.452 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.452 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-07-06T03:54:21.452 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg==", 2023-07-06T03:54:21.452 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.452 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:54:21.453 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-07-06T03:54:21.453 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-06T03:54:21.453 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.453 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-07-06T03:54:21.453 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkq4/pGRAAucHmR1eBA8MNMy3/l7tl0g==", 2023-07-06T03:54:21.454 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.454 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:54:21.454 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:54:21.454 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:54:21.454 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:54:21.454 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.455 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-07-06T03:54:21.455 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkhL5fEhAAWhw4IbCPqTO2sXPIcpTVHA==", 2023-07-06T03:54:21.455 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.455 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:54:21.455 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:54:21.455 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:54:21.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:54:21.455 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.456 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-07-06T03:54:21.456 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkO81dIRAAffjVyvsb/ygBtx8UrYQK0w==", 2023-07-06T03:54:21.456 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.456 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:54:21.456 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:54:21.457 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:54:21.457 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:54:21.457 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.457 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-07-06T03:54:21.457 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCFOKZk6uVlHRAA92RDhdDKW/q65SEOQMWgiA==", 2023-07-06T03:54:21.457 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.458 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:54:21.458 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:54:21.458 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:54:21.458 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.458 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.458 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-07-06T03:54:21.459 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkT4ALEhAAXwwkcodQU9meJA0tfseJCg==", 2023-07-06T03:54:21.459 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.459 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:54:21.459 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:54:21.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.459 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:21.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:21.460 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-07-06T03:54:21.460 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkfnoEChAAuRxqXqPI33dgWkBJoUMh3w==", 2023-07-06T03:54:21.460 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:54:21.460 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:54:21.461 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:54:21.461 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:54:21.461 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.461 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:21.461 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:54:21.461 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:54:21.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.435+0000 7f576afff700 1 -- 172.21.15.97:0/2692219691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574806ace0 msgr2=0x7f574806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:21.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.435+0000 7f576afff700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574806ace0 0x7f574806d190 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5758000f60 tx=0x7f5758009040 comp rx=0 tx=0).stop 2023-07-06T03:54:21.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.435+0000 7f576afff700 1 -- 172.21.15.97:0/2692219691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764158960 msgr2=0x7f5764151d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:21.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.435+0000 7f576afff700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764158960 0x7f5764151d30 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f575000ade0 tx=0x7f5750004750 comp rx=0 tx=0).stop 2023-07-06T03:54:21.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.435+0000 7f576afff700 1 -- 172.21.15.97:0/2692219691 shutdown_connections 2023-07-06T03:54:21.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.435+0000 7f576afff700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574806ace0 0x7f574806d190 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.435+0000 7f576afff700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f576415ce90 0x7f57640af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.435+0000 7f576afff700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5764071a00 0x7f57640b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.435+0000 7f576afff700 1 --2- 172.21.15.97:0/2692219691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764158960 0x7f5764151d30 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.435+0000 7f576afff700 1 -- 172.21.15.97:0/2692219691 >> 172.21.15.97:0/2692219691 conn(0x7f57640ce8c0 msgr2=0x7f5764154bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.436+0000 7f576afff700 1 -- 172.21.15.97:0/2692219691 shutdown_connections 2023-07-06T03:54:21.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.436+0000 7f576afff700 1 -- 172.21.15.97:0/2692219691 wait complete. 2023-07-06T03:54:21.465 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.657+0000 7fcf127ff700 1 Processor -- start 2023-07-06T03:54:21.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.657+0000 7fcf127ff700 1 -- start start 2023-07-06T03:54:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.658+0000 7fcf127ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf0c158960 0x7fcf0c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.658+0000 7fcf127ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcf0c15ce90 0x7fcf0c0712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.658+0000 7fcf127ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcf0c071830 0x7fcf0c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.658+0000 7fcf127ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0c06ddb0 con 0x7fcf0c071830 2023-07-06T03:54:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.658+0000 7fcf127ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0c06bbf0 con 0x7fcf0c15ce90 2023-07-06T03:54:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.658+0000 7fcf127ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c06bed0 con 0x7fcf0c158960 2023-07-06T03:54:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.659+0000 7fcf10d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcf0c071830 0x7fcf0c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.659+0000 7fcf10d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcf0c071830 0x7fcf0c0cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57088/0 (socket says 172.21.15.97:57088) 2023-07-06T03:54:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.659+0000 7fcf10d9c700 1 -- 172.21.15.97:0/1116000901 learned_addr learned my addr 172.21.15.97:0/1116000901 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.659+0000 7fcf0bfff700 1 --2- 172.21.15.97:0/1116000901 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf0c158960 0x7fcf0c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.659+0000 7fcf0b7fe700 1 --2- 172.21.15.97:0/1116000901 >> v2:172.21.15.146:3300/0 conn(0x7fcf0c15ce90 0x7fcf0c0712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.659+0000 7fcf10d9c700 1 -- 172.21.15.97:0/1116000901 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf0c158960 msgr2=0x7fcf0c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.659+0000 7fcf10d9c700 1 --2- 172.21.15.97:0/1116000901 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf0c158960 0x7fcf0c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.659+0000 7fcf10d9c700 1 -- 172.21.15.97:0/1116000901 >> v2:172.21.15.146:3300/0 conn(0x7fcf0c15ce90 msgr2=0x7fcf0c0712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.659+0000 7fcf10d9c700 1 --2- 172.21.15.97:0/1116000901 >> v2:172.21.15.146:3300/0 conn(0x7fcf0c15ce90 0x7fcf0c0712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.659+0000 7fcf10d9c700 1 -- 172.21.15.97:0/1116000901 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c06c1b0 con 0x7fcf0c071830 2023-07-06T03:54:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.659+0000 7fcf10d9c700 1 --2- 172.21.15.97:0/1116000901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf0c071830 0x7fcf0c0cc400 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0b1690 tx=0x7fcf0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da55375b37f94365 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf0affd700 1 -- 172.21.15.97:0/1116000901 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400e070 con 0x7fcf0c071830 2023-07-06T03:54:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf0affd700 1 -- 172.21.15.97:0/1116000901 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0400b9c0 con 0x7fcf0c071830 2023-07-06T03:54:21.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf0affd700 1 -- 172.21.15.97:0/1116000901 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400bd70 con 0x7fcf0c071830 2023-07-06T03:54:21.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf127ff700 1 -- 172.21.15.97:0/1116000901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf0c071830 msgr2=0x7fcf0c0cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:21.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf127ff700 1 --2- 172.21.15.97:0/1116000901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf0c071830 0x7fcf0c0cc400 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0b1690 tx=0x7fcf0400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:21.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf127ff700 1 -- 172.21.15.97:0/1116000901 shutdown_connections 2023-07-06T03:54:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf127ff700 1 --2- 172.21.15.97:0/1116000901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf0c071830 0x7fcf0c0cc400 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf127ff700 1 --2- 172.21.15.97:0/1116000901 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf0c158960 0x7fcf0c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf127ff700 1 --2- 172.21.15.97:0/1116000901 >> v2:172.21.15.146:3300/0 conn(0x7fcf0c15ce90 0x7fcf0c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf127ff700 1 -- 172.21.15.97:0/1116000901 >> 172.21.15.97:0/1116000901 conn(0x7fcf0c0ce8c0 msgr2=0x7fcf0c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:21.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf127ff700 1 -- 172.21.15.97:0/1116000901 shutdown_connections 2023-07-06T03:54:21.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.660+0000 7fcf127ff700 1 -- 172.21.15.97:0/1116000901 wait complete. 2023-07-06T03:54:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.663+0000 7fcf127ff700 1 Processor -- start 2023-07-06T03:54:21.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.663+0000 7fcf127ff700 1 -- start start 2023-07-06T03:54:21.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.664+0000 7fcf127ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf0c071830 0x7fcf0c13bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.664+0000 7fcf127ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf0c158960 0x7fcf0c15b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.664+0000 7fcf127ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf0c15ce90 0x7fcf0c15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.664+0000 7fcf127ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c13f7a0 con 0x7fcf0c071830 2023-07-06T03:54:21.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.664+0000 7fcf127ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c147b70 con 0x7fcf0c15ce90 2023-07-06T03:54:21.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.664+0000 7fcf127ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c155f50 con 0x7fcf0c158960 2023-07-06T03:54:21.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.664+0000 7fcf0bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf0c071830 0x7fcf0c13bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:21.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.664+0000 7fcf0bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf0c071830 0x7fcf0c13bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57104/0 (socket says 172.21.15.97:57104) 2023-07-06T03:54:21.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.664+0000 7fcf0bfff700 1 -- 172.21.15.97:0/1699900535 learned_addr learned my addr 172.21.15.97:0/1699900535 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:21.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.665+0000 7fcf0bfff700 1 -- 172.21.15.97:0/1699900535 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf0c158960 msgr2=0x7fcf0c15b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:21.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.665+0000 7fcf0bfff700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf0c158960 0x7fcf0c15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.665+0000 7fcf0bfff700 1 -- 172.21.15.97:0/1699900535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf0c15ce90 msgr2=0x7fcf0c15b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:21.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.665+0000 7fcf0bfff700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf0c15ce90 0x7fcf0c15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.665+0000 7fcf0bfff700 1 -- 172.21.15.97:0/1699900535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04009b30 con 0x7fcf0c071830 2023-07-06T03:54:21.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.665+0000 7fcf0bfff700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf0c071830 0x7fcf0c13bab0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00e810 tx=0x7fcefc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:21.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.665+0000 7fcf08ff9700 1 -- 172.21.15.97:0/1699900535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc013610 con 0x7fcf0c071830 2023-07-06T03:54:21.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.665+0000 7fcf08ff9700 1 -- 172.21.15.97:0/1699900535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc013770 con 0x7fcf0c071830 2023-07-06T03:54:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.665+0000 7fcf08ff9700 1 -- 172.21.15.97:0/1699900535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc013b00 con 0x7fcf0c071830 2023-07-06T03:54:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.665+0000 7fcf127ff700 1 -- 172.21.15.97:0/1699900535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0c13bff0 con 0x7fcf0c071830 2023-07-06T03:54:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.665+0000 7fcf127ff700 1 -- 172.21.15.97:0/1699900535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c1561a0 con 0x7fcf0c071830 2023-07-06T03:54:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.666+0000 7fcf127ff700 1 -- 172.21.15.97:0/1699900535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0c156ad0 con 0x7fcf0c071830 2023-07-06T03:54:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.667+0000 7fcf08ff9700 1 -- 172.21.15.97:0/1699900535 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcefc013c60 con 0x7fcf0c071830 2023-07-06T03:54:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.668+0000 7fcf08ff9700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcef806ace0 0x7fcef806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.668+0000 7fcf08ff9700 1 -- 172.21.15.97:0/1699900535 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcefc088e00 con 0x7fcf0c071830 2023-07-06T03:54:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.668+0000 7fcf0b7fe700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcef806ace0 0x7fcef806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.669+0000 7fcf0b7fe700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcef806ace0 0x7fcef806d190 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c14a0f0 tx=0x7fcf00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.672+0000 7fcf08ff9700 1 -- 172.21.15.97:0/1699900535 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcefc054160 con 0x7fcf0c071830 2023-07-06T03:54:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.868+0000 7fcf127ff700 1 -- 172.21.15.97:0/1699900535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf0c15d260 con 0x7fcf0c071830 2023-07-06T03:54:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.869+0000 7fcf08ff9700 1 -- 172.21.15.97:0/1699900535 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+14670 (secure 0 0 0) 0x7fcf0c15d260 con 0x7fcf0c071830 2023-07-06T03:54:21.871 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:21.871 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":108,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:54:20.935196+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:21.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.871+0000 7fcf127ff700 1 -- 172.21.15.97:0/1699900535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcef806ace0 msgr2=0x7fcef806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:21.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.871+0000 7fcf127ff700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcef806ace0 0x7fcef806d190 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c14a0f0 tx=0x7fcf00008040 comp rx=0 tx=0).stop 2023-07-06T03:54:21.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.871+0000 7fcf127ff700 1 -- 172.21.15.97:0/1699900535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf0c071830 msgr2=0x7fcf0c13bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:21.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.871+0000 7fcf127ff700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf0c071830 0x7fcf0c13bab0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00e810 tx=0x7fcefc00ebd0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.871+0000 7fcf127ff700 1 -- 172.21.15.97:0/1699900535 shutdown_connections 2023-07-06T03:54:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.871+0000 7fcf127ff700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcef806ace0 0x7fcef806d190 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.871+0000 7fcf127ff700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf0c15ce90 0x7fcf0c15b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.871+0000 7fcf127ff700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf0c071830 0x7fcf0c13bab0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.871+0000 7fcf127ff700 1 --2- 172.21.15.97:0/1699900535 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf0c158960 0x7fcf0c15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.871+0000 7fcf127ff700 1 -- 172.21.15.97:0/1699900535 >> 172.21.15.97:0/1699900535 conn(0x7fcf0c0ce8c0 msgr2=0x7fcf0c159740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.871+0000 7fcf127ff700 1 -- 172.21.15.97:0/1699900535 shutdown_connections 2023-07-06T03:54:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:21.872+0000 7fcf127ff700 1 -- 172.21.15.97:0/1699900535 wait complete. 2023-07-06T03:54:21.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:54:21.893 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:54:22.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.092+0000 7f42019ff700 1 Processor -- start 2023-07-06T03:54:22.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.092+0000 7f42019ff700 1 -- start start 2023-07-06T03:54:22.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f42019ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f41fc15c610 0x7f41fc0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f42019ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41fc0cff90 con 0x7f41fc06b360 2023-07-06T03:54:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f42019ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41fc0b5650 con 0x7f41fc0d2210 2023-07-06T03:54:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f42019ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41fc0d0780 con 0x7f41fc15c610 2023-07-06T03:54:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f41faffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f41fc15c610 0x7f41fc0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f41faffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f41fc15c610 0x7f41fc0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57120/0 (socket says 172.21.15.97:57120) 2023-07-06T03:54:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f41f3fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f41fc06b360 0x7f41fc0b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53512/0 (socket says 172.21.15.97:53512) 2023-07-06T03:54:22.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f41faffd700 1 -- 172.21.15.97:0/250691482 learned_addr learned my addr 172.21.15.97:0/250691482 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:22.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f41faffd700 1 -- 172.21.15.97:0/250691482 >> v1:172.21.15.146:6789/0 conn(0x7f41fc0d2210 legacy=0x7f41fc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:22.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f41faffd700 1 -- 172.21.15.97:0/250691482 >> v1:172.21.15.97:6789/0 conn(0x7f41fc06b360 legacy=0x7f41fc0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f41faffd700 1 -- 172.21.15.97:0/250691482 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc0b5400 con 0x7f41fc15c610 2023-07-06T03:54:22.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.094+0000 7f41fa7fc700 1 -- 172.21.15.97:0/250691482 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391816397 0 0) 0x7f41fc0cff90 con 0x7f41fc06b360 2023-07-06T03:54:22.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.095+0000 7f41faffd700 1 --2- 172.21.15.97:0/250691482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc15c610 0x7f41fc0b47b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0098f0 tx=0x7f41ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd2dddb267ceb33 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:22.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.095+0000 7f41fa7fc700 1 -- 172.21.15.97:0/250691482 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec004230 con 0x7f41fc15c610 2023-07-06T03:54:22.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.095+0000 7f41fa7fc700 1 -- 172.21.15.97:0/250691482 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec004390 con 0x7f41fc15c610 2023-07-06T03:54:22.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.095+0000 7f41fa7fc700 1 -- 172.21.15.97:0/250691482 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0036a0 con 0x7f41fc15c610 2023-07-06T03:54:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.095+0000 7f42019ff700 1 -- 172.21.15.97:0/250691482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc15c610 msgr2=0x7f41fc0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.095+0000 7f42019ff700 1 --2- 172.21.15.97:0/250691482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc15c610 0x7f41fc0b47b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0098f0 tx=0x7f41ec009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.096+0000 7f42019ff700 1 -- 172.21.15.97:0/250691482 shutdown_connections 2023-07-06T03:54:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.096+0000 7f42019ff700 1 --2- 172.21.15.97:0/250691482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc15c610 0x7f41fc0b47b0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.096+0000 7f42019ff700 1 -- 172.21.15.97:0/250691482 >> 172.21.15.97:0/250691482 conn(0x7f41fc06a6c0 msgr2=0x7f41fc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.096+0000 7f42019ff700 1 -- 172.21.15.97:0/250691482 shutdown_connections 2023-07-06T03:54:22.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.096+0000 7f42019ff700 1 -- 172.21.15.97:0/250691482 wait complete. 2023-07-06T03:54:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.097+0000 7f42019ff700 1 Processor -- start 2023-07-06T03:54:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.097+0000 7f42019ff700 1 -- start start 2023-07-06T03:54:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.098+0000 7f42019ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc06b360 0x7f41fc157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.098+0000 7f42019ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc0d2210 0x7f41fc0cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.098+0000 7f42019ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41fc15c610 0x7f41fc0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.098+0000 7f42019ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc15fa50 con 0x7f41fc0d2210 2023-07-06T03:54:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.098+0000 7f42019ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc15e4d0 con 0x7f41fc15c610 2023-07-06T03:54:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.098+0000 7f42019ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc15d710 con 0x7f41fc06b360 2023-07-06T03:54:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41f3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc0d2210 0x7f41fc0cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41f3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc0d2210 0x7f41fc0cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57124/0 (socket says 172.21.15.97:57124) 2023-07-06T03:54:22.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41f3fff700 1 -- 172.21.15.97:0/3249137206 learned_addr learned my addr 172.21.15.97:0/3249137206 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:22.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41faffd700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc06b360 0x7f41fc157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41fb7fe700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41fc15c610 0x7f41fc0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41f3fff700 1 -- 172.21.15.97:0/3249137206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc06b360 msgr2=0x7f41fc157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41f3fff700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc06b360 0x7f41fc157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41f3fff700 1 -- 172.21.15.97:0/3249137206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41fc15c610 msgr2=0x7f41fc0cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41f3fff700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41fc15c610 0x7f41fc0cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41f3fff700 1 -- 172.21.15.97:0/3249137206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec009550 con 0x7f41fc0d2210 2023-07-06T03:54:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41f3fff700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc0d2210 0x7f41fc0cbc10 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c550 tx=0x7f41e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.099+0000 7f41f8ff9700 1 -- 172.21.15.97:0/3249137206 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4004c80 con 0x7f41fc0d2210 2023-07-06T03:54:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.100+0000 7f41f8ff9700 1 -- 172.21.15.97:0/3249137206 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e4004e00 con 0x7f41fc0d2210 2023-07-06T03:54:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.100+0000 7f42019ff700 1 -- 172.21.15.97:0/3249137206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc15d9f0 con 0x7f41fc0d2210 2023-07-06T03:54:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.100+0000 7f41f8ff9700 1 -- 172.21.15.97:0/3249137206 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4007310 con 0x7f41fc0d2210 2023-07-06T03:54:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.100+0000 7f42019ff700 1 -- 172.21.15.97:0/3249137206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc15dfb0 con 0x7f41fc0d2210 2023-07-06T03:54:22.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.101+0000 7f42019ff700 1 -- 172.21.15.97:0/3249137206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41fc15e1a0 con 0x7f41fc0d2210 2023-07-06T03:54:22.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.102+0000 7f41f8ff9700 1 -- 172.21.15.97:0/3249137206 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41e4007470 con 0x7f41fc0d2210 2023-07-06T03:54:22.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.103+0000 7f41f8ff9700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41dc06ace0 0x7f41dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.103+0000 7f41f8ff9700 1 -- 172.21.15.97:0/3249137206 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f41e4088de0 con 0x7f41fc0d2210 2023-07-06T03:54:22.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.103+0000 7f41faffd700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41dc06ace0 0x7f41dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.103+0000 7f41faffd700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41dc06ace0 0x7f41dc06d190 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009130 tx=0x7f41ec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.107+0000 7f41f8ff9700 1 -- 172.21.15.97:0/3249137206 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f41e4054270 con 0x7f41fc0d2210 2023-07-06T03:54:22.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.300+0000 7f42019ff700 1 -- 172.21.15.97:0/3249137206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41fc04e340 con 0x7f41fc0d2210 2023-07-06T03:54:22.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.301+0000 7f41f8ff9700 1 -- 172.21.15.97:0/3249137206 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41e4057890 con 0x7f41fc0d2210 2023-07-06T03:54:22.303 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:54:22.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 -- 172.21.15.97:0/3249137206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41dc06ace0 msgr2=0x7f41dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41dc06ace0 0x7f41dc06d190 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009130 tx=0x7f41ec008030 comp rx=0 tx=0).stop 2023-07-06T03:54:22.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 -- 172.21.15.97:0/3249137206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc0d2210 msgr2=0x7f41fc0cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc0d2210 0x7f41fc0cbc10 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c550 tx=0x7f41e400c910 comp rx=0 tx=0).stop 2023-07-06T03:54:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 -- 172.21.15.97:0/3249137206 shutdown_connections 2023-07-06T03:54:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f41dc06ace0 0x7f41dc06d190 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41fc15c610 0x7f41fc0cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f41fc06b360 0x7f41fc157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 --2- 172.21.15.97:0/3249137206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc0d2210 0x7f41fc0cbc10 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 -- 172.21.15.97:0/3249137206 >> 172.21.15.97:0/3249137206 conn(0x7f41fc06a6c0 msgr2=0x7f41fc159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 -- 172.21.15.97:0/3249137206 shutdown_connections 2023-07-06T03:54:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.303+0000 7f42019ff700 1 -- 172.21.15.97:0/3249137206 wait complete. 2023-07-06T03:54:22.323 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:54:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.518+0000 7f82823ff700 1 Processor -- start 2023-07-06T03:54:22.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.518+0000 7f82823ff700 1 -- start start 2023-07-06T03:54:22.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.519+0000 7f82823ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f827c15c1c0 0x7f827c0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.519+0000 7f82823ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f827c0719c0 0x7f827c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.519+0000 7f82823ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827c15da10 con 0x7f827c15c590 2023-07-06T03:54:22.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.519+0000 7f82823ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c0ccc30 con 0x7f827c0719c0 2023-07-06T03:54:22.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.519+0000 7f82823ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0ccf10 con 0x7f827c15c1c0 2023-07-06T03:54:22.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.519+0000 7f828099c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f827c0719c0 0x7f827c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.519+0000 7f828099c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f827c0719c0 0x7f827c06a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57138/0 (socket says 172.21.15.97:57138) 2023-07-06T03:54:22.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.519+0000 7f828099c700 1 -- 172.21.15.97:0/3629261724 learned_addr learned my addr 172.21.15.97:0/3629261724 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:22.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.520+0000 7f827bfff700 1 --2- 172.21.15.97:0/3629261724 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f827c15c1c0 0x7f827c0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.520+0000 7f828099c700 1 -- 172.21.15.97:0/3629261724 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f827c15c1c0 msgr2=0x7f827c0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:22.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.520+0000 7f828099c700 1 --2- 172.21.15.97:0/3629261724 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f827c15c1c0 0x7f827c0cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.520+0000 7f828099c700 1 -- 172.21.15.97:0/3629261724 >> v1:172.21.15.97:6789/0 conn(0x7f827c15c590 legacy=0x7f827c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.520+0000 7f827affd700 1 -- 172.21.15.97:0/3629261724 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447895583 0 0) 0x7f8270001110 con 0x7f827c15c590 2023-07-06T03:54:22.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.520+0000 7f828099c700 1 -- 172.21.15.97:0/3629261724 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0cd1f0 con 0x7f827c0719c0 2023-07-06T03:54:22.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.520+0000 7f828099c700 1 --2- 172.21.15.97:0/3629261724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f827c0719c0 0x7f827c06a060 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8274009e50 tx=0x7f827400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a9b7f454335913 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:22.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.520+0000 7f827affd700 1 -- 172.21.15.97:0/3629261724 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827400e070 con 0x7f827c0719c0 2023-07-06T03:54:22.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.520+0000 7f827affd700 1 -- 172.21.15.97:0/3629261724 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827400b9c0 con 0x7f827c0719c0 2023-07-06T03:54:22.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.520+0000 7f827affd700 1 -- 172.21.15.97:0/3629261724 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827400bd70 con 0x7f827c0719c0 2023-07-06T03:54:22.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.521+0000 7f82823ff700 1 -- 172.21.15.97:0/3629261724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f827c0719c0 msgr2=0x7f827c06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.521+0000 7f82823ff700 1 --2- 172.21.15.97:0/3629261724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f827c0719c0 0x7f827c06a060 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8274009e50 tx=0x7f827400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.521+0000 7f82823ff700 1 -- 172.21.15.97:0/3629261724 shutdown_connections 2023-07-06T03:54:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.521+0000 7f82823ff700 1 --2- 172.21.15.97:0/3629261724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f827c0719c0 0x7f827c06a060 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.521+0000 7f82823ff700 1 --2- 172.21.15.97:0/3629261724 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f827c15c1c0 0x7f827c0cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.521+0000 7f82823ff700 1 -- 172.21.15.97:0/3629261724 >> 172.21.15.97:0/3629261724 conn(0x7f827c0b1650 msgr2=0x7f827c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.521+0000 7f82823ff700 1 -- 172.21.15.97:0/3629261724 shutdown_connections 2023-07-06T03:54:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.521+0000 7f82823ff700 1 -- 172.21.15.97:0/3629261724 wait complete. 2023-07-06T03:54:22.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.523+0000 7f82823ff700 1 Processor -- start 2023-07-06T03:54:22.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.523+0000 7f82823ff700 1 -- start start 2023-07-06T03:54:22.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f82823ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f827c0719c0 0x7f827c1f1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f82823ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f827c15c1c0 0x7f827c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f82823ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f827c15c590 0x7f827c0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f82823ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0b57a0 con 0x7f827c15c1c0 2023-07-06T03:54:22.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f82823ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c1f5400 con 0x7f827c15c590 2023-07-06T03:54:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f82823ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c1f56e0 con 0x7f827c0719c0 2023-07-06T03:54:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f827b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f827c15c1c0 0x7f827c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f827b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f827c15c1c0 0x7f827c0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57144/0 (socket says 172.21.15.97:57144) 2023-07-06T03:54:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f827b7fe700 1 -- 172.21.15.97:0/2632509770 learned_addr learned my addr 172.21.15.97:0/2632509770 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f827b7fe700 1 -- 172.21.15.97:0/2632509770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f827c0719c0 msgr2=0x7f827c1f1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f827b7fe700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f827c0719c0 0x7f827c1f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f827b7fe700 1 -- 172.21.15.97:0/2632509770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f827c15c590 msgr2=0x7f827c0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f827b7fe700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f827c15c590 0x7f827c0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.524+0000 7f827b7fe700 1 -- 172.21.15.97:0/2632509770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274009b30 con 0x7f827c15c1c0 2023-07-06T03:54:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.525+0000 7f827b7fe700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f827c15c1c0 0x7f827c0b4d50 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f827000c550 tx=0x7f827000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.525+0000 7f8278ff9700 1 -- 172.21.15.97:0/2632509770 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82700054e0 con 0x7f827c15c1c0 2023-07-06T03:54:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.525+0000 7f8278ff9700 1 -- 172.21.15.97:0/2632509770 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8270003680 con 0x7f827c15c1c0 2023-07-06T03:54:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.525+0000 7f82823ff700 1 -- 172.21.15.97:0/2632509770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c1f59c0 con 0x7f827c15c1c0 2023-07-06T03:54:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.525+0000 7f8278ff9700 1 -- 172.21.15.97:0/2632509770 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270003a30 con 0x7f827c15c1c0 2023-07-06T03:54:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.525+0000 7f82823ff700 1 -- 172.21.15.97:0/2632509770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c1f61a0 con 0x7f827c15c1c0 2023-07-06T03:54:22.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.526+0000 7f82823ff700 1 -- 172.21.15.97:0/2632509770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c04e340 con 0x7f827c15c1c0 2023-07-06T03:54:22.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.527+0000 7f8278ff9700 1 -- 172.21.15.97:0/2632509770 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8270003b90 con 0x7f827c15c1c0 2023-07-06T03:54:22.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.528+0000 7f8278ff9700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 0x7f826806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.528+0000 7f8278ff9700 1 -- 172.21.15.97:0/2632509770 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f82700884f0 con 0x7f827c15c1c0 2023-07-06T03:54:22.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.528+0000 7f827bfff700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 0x7f826806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.528+0000 7f827bfff700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 0x7f826806d190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f826c0097f0 tx=0x7f826c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:22.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.532+0000 7f8278ff9700 1 -- 172.21.15.97:0/2632509770 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8270053850 con 0x7f827c15c1c0 2023-07-06T03:54:22.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.724+0000 7f82823ff700 1 -- 172.21.15.97:0/2632509770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f827c15da10 con 0x7f827c15c1c0 2023-07-06T03:54:22.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.725+0000 7f8278ff9700 1 -- 172.21.15.97:0/2632509770 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8270017020 con 0x7f827c15c1c0 2023-07-06T03:54:22.727 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:54:22.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.727+0000 7f82823ff700 1 -- 172.21.15.97:0/2632509770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 msgr2=0x7f826806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.727+0000 7f82823ff700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 0x7f826806d190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f826c0097f0 tx=0x7f826c008040 comp rx=0 tx=0).stop 2023-07-06T03:54:22.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.727+0000 7f82823ff700 1 -- 172.21.15.97:0/2632509770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f827c15c1c0 msgr2=0x7f827c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.727+0000 7f82823ff700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f827c15c1c0 0x7f827c0b4d50 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f827000c550 tx=0x7f827000c910 comp rx=0 tx=0).stop 2023-07-06T03:54:22.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.727+0000 7f82823ff700 1 -- 172.21.15.97:0/2632509770 shutdown_connections 2023-07-06T03:54:22.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.727+0000 7f82823ff700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f826806ace0 0x7f826806d190 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.728+0000 7f82823ff700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f827c15c590 0x7f827c0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.728+0000 7f82823ff700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f827c0719c0 0x7f827c1f1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.728+0000 7f82823ff700 1 --2- 172.21.15.97:0/2632509770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f827c15c1c0 0x7f827c0b4d50 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.728+0000 7f82823ff700 1 -- 172.21.15.97:0/2632509770 >> 172.21.15.97:0/2632509770 conn(0x7f827c0b1650 msgr2=0x7f827c0c5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:22.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.728+0000 7f82823ff700 1 -- 172.21.15.97:0/2632509770 shutdown_connections 2023-07-06T03:54:22.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.728+0000 7f82823ff700 1 -- 172.21.15.97:0/2632509770 wait complete. 2023-07-06T03:54:22.747 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T03:54:22.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.941+0000 7f78a0bff700 1 Processor -- start 2023-07-06T03:54:22.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.942+0000 7f78a0bff700 1 -- start start 2023-07-06T03:54:22.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.943+0000 7f78a0bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f789c15c520 0x7f789c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.943+0000 7f78a0bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f789c160d00 0x7f789c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.943+0000 7f78a0bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f789c0cdae0 con 0x7f789c160180 2023-07-06T03:54:22.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.943+0000 7f78a0bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789c0cd8d0 con 0x7f789c160d00 2023-07-06T03:54:22.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.943+0000 7f78a0bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0b6640 con 0x7f789c15c520 2023-07-06T03:54:22.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.943+0000 7f7899d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f789c160180 0x7f789c1605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53528/0 (socket says 172.21.15.97:53528) 2023-07-06T03:54:22.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.943+0000 7f7899d9b700 1 -- 172.21.15.97:0/1574578857 learned_addr learned my addr 172.21.15.97:0/1574578857 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:22.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.943+0000 7f789ad9d700 1 --2- 172.21.15.97:0/1574578857 >> v2:172.21.15.97:3300/0 conn(0x7f789c160d00 0x7f789c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.943+0000 7f789a59c700 1 --2- 172.21.15.97:0/1574578857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f789c15c520 0x7f789c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.944+0000 7f789959a700 1 -- 172.21.15.97:0/1574578857 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396314619 0 0) 0x7f789c0cdae0 con 0x7f789c160180 2023-07-06T03:54:22.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.944+0000 7f789959a700 1 -- 172.21.15.97:0/1574578857 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78840035b0 con 0x7f789c160180 2023-07-06T03:54:22.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.944+0000 7f789ad9d700 1 -- 172.21.15.97:0/1574578857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f789c15c520 msgr2=0x7f789c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:22.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.944+0000 7f789ad9d700 1 --2- 172.21.15.97:0/1574578857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f789c15c520 0x7f789c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.944+0000 7f789ad9d700 1 -- 172.21.15.97:0/1574578857 >> v1:172.21.15.97:6789/0 conn(0x7f789c160180 legacy=0x7f789c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.944+0000 7f789ad9d700 1 -- 172.21.15.97:0/1574578857 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c0b6920 con 0x7f789c160d00 2023-07-06T03:54:22.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.944+0000 7f789ad9d700 1 --2- 172.21.15.97:0/1574578857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f789c160d00 0x7f789c0b7400 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7894009d90 tx=0x7f789400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be965f480a3f475 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:22.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.944+0000 7f789959a700 1 -- 172.21.15.97:0/1574578857 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789400e070 con 0x7f789c160d00 2023-07-06T03:54:22.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.944+0000 7f789959a700 1 -- 172.21.15.97:0/1574578857 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789400bb20 con 0x7f789c160d00 2023-07-06T03:54:22.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.944+0000 7f789959a700 1 -- 172.21.15.97:0/1574578857 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789400be90 con 0x7f789c160d00 2023-07-06T03:54:22.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.945+0000 7f78a0bff700 1 -- 172.21.15.97:0/1574578857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f789c160d00 msgr2=0x7f789c0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.945+0000 7f78a0bff700 1 --2- 172.21.15.97:0/1574578857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f789c160d00 0x7f789c0b7400 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7894009d90 tx=0x7f789400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:22.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.945+0000 7f78a0bff700 1 -- 172.21.15.97:0/1574578857 shutdown_connections 2023-07-06T03:54:22.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.945+0000 7f78a0bff700 1 --2- 172.21.15.97:0/1574578857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f789c160d00 0x7f789c0b7400 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.945+0000 7f78a0bff700 1 --2- 172.21.15.97:0/1574578857 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f789c15c520 0x7f789c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.945+0000 7f78a0bff700 1 -- 172.21.15.97:0/1574578857 >> 172.21.15.97:0/1574578857 conn(0x7f789c071790 msgr2=0x7f789c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:22.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.945+0000 7f78a0bff700 1 -- 172.21.15.97:0/1574578857 shutdown_connections 2023-07-06T03:54:22.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.945+0000 7f78a0bff700 1 -- 172.21.15.97:0/1574578857 wait complete. 2023-07-06T03:54:22.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.947+0000 7f78a0bff700 1 Processor -- start 2023-07-06T03:54:22.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.947+0000 7f78a0bff700 1 -- start start 2023-07-06T03:54:22.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f78a0bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f789c15c520 0x7f789c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f78a0bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f789c160180 0x7f789c1e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f78a0bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f789c160d00 0x7f789c1f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f78a0bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c1563b0 con 0x7f789c160180 2023-07-06T03:54:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f78a0bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0d1360 con 0x7f789c15c520 2023-07-06T03:54:22.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f78a0bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0d1640 con 0x7f789c160d00 2023-07-06T03:54:22.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f7899d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f789c160180 0x7f789c1e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f7899d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f789c160180 0x7f789c1e5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57162/0 (socket says 172.21.15.97:57162) 2023-07-06T03:54:22.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f7899d9b700 1 -- 172.21.15.97:0/495837481 learned_addr learned my addr 172.21.15.97:0/495837481 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:22.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f789a59c700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f789c15c520 0x7f789c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f789ad9d700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f789c160d00 0x7f789c1f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f7899d9b700 1 -- 172.21.15.97:0/495837481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f789c160d00 msgr2=0x7f789c1f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f7899d9b700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f789c160d00 0x7f789c1f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f7899d9b700 1 -- 172.21.15.97:0/495837481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f789c15c520 msgr2=0x7f789c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:22.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f7899d9b700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f789c15c520 0x7f789c0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:22.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.948+0000 7f7899d9b700 1 -- 172.21.15.97:0/495837481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7894009a70 con 0x7f789c160180 2023-07-06T03:54:22.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.949+0000 7f7899d9b700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f789c160180 0x7f789c1e5e10 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f789000c420 tx=0x7f789000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:22.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.949+0000 7f788affd700 1 -- 172.21.15.97:0/495837481 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890005d70 con 0x7f789c160180 2023-07-06T03:54:22.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.949+0000 7f788affd700 1 -- 172.21.15.97:0/495837481 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78900070d0 con 0x7f789c160180 2023-07-06T03:54:22.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.949+0000 7f788affd700 1 -- 172.21.15.97:0/495837481 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890007440 con 0x7f789c160180 2023-07-06T03:54:22.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.949+0000 7f78a0bff700 1 -- 172.21.15.97:0/495837481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c0d1920 con 0x7f789c160180 2023-07-06T03:54:22.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.949+0000 7f78a0bff700 1 -- 172.21.15.97:0/495837481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789c0d1ee0 con 0x7f789c160180 2023-07-06T03:54:22.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.950+0000 7f78a0bff700 1 -- 172.21.15.97:0/495837481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789c04e340 con 0x7f789c160180 2023-07-06T03:54:22.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.951+0000 7f788affd700 1 -- 172.21.15.97:0/495837481 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f78900075a0 con 0x7f789c160180 2023-07-06T03:54:22.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.952+0000 7f788affd700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f787c06ace0 0x7f787c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:22.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.952+0000 7f788affd700 1 -- 172.21.15.97:0/495837481 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7890088550 con 0x7f789c160180 2023-07-06T03:54:22.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.952+0000 7f789a59c700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f787c06ace0 0x7f787c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:22.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.953+0000 7f789a59c700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f787c06ace0 0x7f787c06d190 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f788c009ad0 tx=0x7f788c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:22.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:22.956+0000 7f788affd700 1 -- 172.21.15.97:0/495837481 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7890053960 con 0x7f789c160180 2023-07-06T03:54:23.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.155+0000 7f78a0bff700 1 -- 172.21.15.97:0/495837481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f789c072020 con 0x7f789c160180 2023-07-06T03:54:23.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.156+0000 7f788affd700 1 -- 172.21.15.97:0/495837481 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7890017090 con 0x7f789c160180 2023-07-06T03:54:23.158 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:23.158 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-07-06T03:54:23.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:23.158 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-07-06T03:54:23.159 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-07-06T03:54:23.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:23.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:23.159 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 2, 2023-07-06T03:54:23.159 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": "rbd" 2023-07-06T03:54:23.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:23.160 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-07-06T03:54:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.158+0000 7f78a0bff700 1 -- 172.21.15.97:0/495837481 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f787c06ace0 msgr2=0x7f787c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.158+0000 7f78a0bff700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f787c06ace0 0x7f787c06d190 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f788c009ad0 tx=0x7f788c008040 comp rx=0 tx=0).stop 2023-07-06T03:54:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.158+0000 7f78a0bff700 1 -- 172.21.15.97:0/495837481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f789c160180 msgr2=0x7f789c1e5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.158+0000 7f78a0bff700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f789c160180 0x7f789c1e5e10 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f789000c420 tx=0x7f789000c7e0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.158+0000 7f78a0bff700 1 -- 172.21.15.97:0/495837481 shutdown_connections 2023-07-06T03:54:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.158+0000 7f78a0bff700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f787c06ace0 0x7f787c06d190 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.158+0000 7f78a0bff700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f789c160d00 0x7f789c1f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.158+0000 7f78a0bff700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f789c15c520 0x7f789c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.158+0000 7f78a0bff700 1 --2- 172.21.15.97:0/495837481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f789c160180 0x7f789c1e5e10 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.159+0000 7f78a0bff700 1 -- 172.21.15.97:0/495837481 >> 172.21.15.97:0/495837481 conn(0x7f789c071790 msgr2=0x7f789c0b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:23.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.159+0000 7f78a0bff700 1 -- 172.21.15.97:0/495837481 shutdown_connections 2023-07-06T03:54:23.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.159+0000 7f78a0bff700 1 -- 172.21.15.97:0/495837481 wait complete. 2023-07-06T03:54:23.178 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T03:54:23.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T03:54:23.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.376+0000 7ff8983ff700 1 Processor -- start 2023-07-06T03:54:23.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.376+0000 7ff8983ff700 1 -- start start 2023-07-06T03:54:23.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.378+0000 7ff8983ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff89015c640 0x7ff8900cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:23.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.378+0000 7ff8983ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff8900cfde0 0x7ff8900b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:23.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.378+0000 7ff8983ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89015ca10 con 0x7ff8900b5270 2023-07-06T03:54:23.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.378+0000 7ff8983ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8900b56e0 con 0x7ff8900cfde0 2023-07-06T03:54:23.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.378+0000 7ff8983ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8900cda40 con 0x7ff89015c640 2023-07-06T03:54:23.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.378+0000 7ff89599a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff8900b5270 0x7ff8900cf6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53544/0 (socket says 172.21.15.97:53544) 2023-07-06T03:54:23.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.378+0000 7ff89599a700 1 -- 172.21.15.97:0/1462657586 learned_addr learned my addr 172.21.15.97:0/1462657586 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:23.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.378+0000 7ff89619b700 1 --2- 172.21.15.97:0/1462657586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff89015c640 0x7ff8900cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:23.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.378+0000 7ff89699c700 1 --2- 172.21.15.97:0/1462657586 >> v2:172.21.15.146:3300/0 conn(0x7ff8900cfde0 0x7ff8900b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:23.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.378+0000 7ff895199700 1 -- 172.21.15.97:0/1462657586 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290134070 0 0) 0x7ff89015ca10 con 0x7ff8900b5270 2023-07-06T03:54:23.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.378+0000 7ff895199700 1 -- 172.21.15.97:0/1462657586 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff87c0035b0 con 0x7ff8900b5270 2023-07-06T03:54:23.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.379+0000 7ff89619b700 1 -- 172.21.15.97:0/1462657586 >> v2:172.21.15.146:3300/0 conn(0x7ff8900cfde0 msgr2=0x7ff8900b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:23.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.379+0000 7ff89619b700 1 --2- 172.21.15.97:0/1462657586 >> v2:172.21.15.146:3300/0 conn(0x7ff8900cfde0 0x7ff8900b73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.379+0000 7ff89619b700 1 -- 172.21.15.97:0/1462657586 >> v1:172.21.15.97:6789/0 conn(0x7ff8900b5270 legacy=0x7ff8900cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.379+0000 7ff89619b700 1 -- 172.21.15.97:0/1462657586 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8900b65f0 con 0x7ff89015c640 2023-07-06T03:54:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.379+0000 7ff89619b700 1 --2- 172.21.15.97:0/1462657586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff89015c640 0x7ff8900cd340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009840 tx=0x7ff88c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c86385ceda64dfb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.379+0000 7ff895199700 1 -- 172.21.15.97:0/1462657586 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c011830 con 0x7ff89015c640 2023-07-06T03:54:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.379+0000 7ff895199700 1 -- 172.21.15.97:0/1462657586 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c011990 con 0x7ff89015c640 2023-07-06T03:54:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.379+0000 7ff895199700 1 -- 172.21.15.97:0/1462657586 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c011d00 con 0x7ff89015c640 2023-07-06T03:54:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.380+0000 7ff8983ff700 1 -- 172.21.15.97:0/1462657586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff89015c640 msgr2=0x7ff8900cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:23.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.380+0000 7ff8983ff700 1 --2- 172.21.15.97:0/1462657586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff89015c640 0x7ff8900cd340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009840 tx=0x7ff88c009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:23.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.380+0000 7ff8983ff700 1 -- 172.21.15.97:0/1462657586 shutdown_connections 2023-07-06T03:54:23.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.380+0000 7ff8983ff700 1 --2- 172.21.15.97:0/1462657586 >> v2:172.21.15.146:3300/0 conn(0x7ff8900cfde0 0x7ff8900b73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.380+0000 7ff8983ff700 1 --2- 172.21.15.97:0/1462657586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff89015c640 0x7ff8900cd340 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.380+0000 7ff8983ff700 1 -- 172.21.15.97:0/1462657586 >> 172.21.15.97:0/1462657586 conn(0x7ff890071790 msgr2=0x7ff890071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:23.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.380+0000 7ff8983ff700 1 -- 172.21.15.97:0/1462657586 shutdown_connections 2023-07-06T03:54:23.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.380+0000 7ff8983ff700 1 -- 172.21.15.97:0/1462657586 wait complete. 2023-07-06T03:54:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.382+0000 7ff8983ff700 1 Processor -- start 2023-07-06T03:54:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.382+0000 7ff8983ff700 1 -- start start 2023-07-06T03:54:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff8983ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8900b5270 0x7ff8901562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:23.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff8983ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8900cfde0 0x7ff89015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:23.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff8983ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015c640 0x7ff89015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:23.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff8983ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff890159d10 con 0x7ff89015c640 2023-07-06T03:54:23.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff8983ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff890159770 con 0x7ff8900b5270 2023-07-06T03:54:23.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff8983ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8901567e0 con 0x7ff8900cfde0 2023-07-06T03:54:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff89699c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015c640 0x7ff89015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff89699c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015c640 0x7ff89015a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57176/0 (socket says 172.21.15.97:57176) 2023-07-06T03:54:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff89699c700 1 -- 172.21.15.97:0/4192027700 learned_addr learned my addr 172.21.15.97:0/4192027700 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff89599a700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8900cfde0 0x7ff89015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff89619b700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8900b5270 0x7ff8901562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff89699c700 1 -- 172.21.15.97:0/4192027700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8900cfde0 msgr2=0x7ff89015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff89699c700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8900cfde0 0x7ff89015cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff89699c700 1 -- 172.21.15.97:0/4192027700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8900b5270 msgr2=0x7ff8901562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:23.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff89699c700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8900b5270 0x7ff8901562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff89699c700 1 -- 172.21.15.97:0/4192027700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c0094a0 con 0x7ff89015c640 2023-07-06T03:54:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.384+0000 7ff89699c700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015c640 0x7ff89015a130 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7ff88800ef90 tx=0x7ff88800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.385+0000 7ff886ffd700 1 -- 172.21.15.97:0/4192027700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888009ca0 con 0x7ff89015c640 2023-07-06T03:54:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.385+0000 7ff886ffd700 1 -- 172.21.15.97:0/4192027700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88800f040 con 0x7ff89015c640 2023-07-06T03:54:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.385+0000 7ff886ffd700 1 -- 172.21.15.97:0/4192027700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888014480 con 0x7ff89015c640 2023-07-06T03:54:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.385+0000 7ff8983ff700 1 -- 172.21.15.97:0/4192027700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff890159970 con 0x7ff89015c640 2023-07-06T03:54:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.385+0000 7ff8983ff700 1 -- 172.21.15.97:0/4192027700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff890159120 con 0x7ff89015c640 2023-07-06T03:54:23.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.386+0000 7ff8983ff700 1 -- 172.21.15.97:0/4192027700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89004e340 con 0x7ff89015c640 2023-07-06T03:54:23.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.387+0000 7ff886ffd700 1 -- 172.21.15.97:0/4192027700 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff8880145e0 con 0x7ff89015c640 2023-07-06T03:54:23.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.388+0000 7ff886ffd700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff87406acb0 0x7ff87406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:23.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.388+0000 7ff886ffd700 1 -- 172.21.15.97:0/4192027700 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff888089b30 con 0x7ff89015c640 2023-07-06T03:54:23.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.388+0000 7ff89619b700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff87406acb0 0x7ff87406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:23.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.389+0000 7ff89619b700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff87406acb0 0x7ff87406d160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009fd0 tx=0x7ff88c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:23.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.392+0000 7ff886ffd700 1 -- 172.21.15.97:0/4192027700 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff8880589b0 con 0x7ff89015c640 2023-07-06T03:54:23.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.587+0000 7ff8983ff700 1 -- 172.21.15.97:0/4192027700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff890159e50 con 0x7ff89015c640 2023-07-06T03:54:23.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.942+0000 7ff886ffd700 1 -- 172.21.15.97:0/4192027700 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff888058540 con 0x7ff89015c640 2023-07-06T03:54:23.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.946+0000 7ff8983ff700 1 -- 172.21.15.97:0/4192027700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff87406acb0 msgr2=0x7ff87406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:23.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.946+0000 7ff8983ff700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff87406acb0 0x7ff87406d160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009fd0 tx=0x7ff88c008030 comp rx=0 tx=0).stop 2023-07-06T03:54:23.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.946+0000 7ff8983ff700 1 -- 172.21.15.97:0/4192027700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015c640 msgr2=0x7ff89015a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:23.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.946+0000 7ff8983ff700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015c640 0x7ff89015a130 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7ff88800ef90 tx=0x7ff88800c3a0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.947+0000 7ff8983ff700 1 -- 172.21.15.97:0/4192027700 shutdown_connections 2023-07-06T03:54:23.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.947+0000 7ff8983ff700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff87406acb0 0x7ff87406d160 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.947+0000 7ff8983ff700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff89015c640 0x7ff89015a130 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.947+0000 7ff8983ff700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8900b5270 0x7ff8901562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.947+0000 7ff8983ff700 1 --2- 172.21.15.97:0/4192027700 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8900cfde0 0x7ff89015cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:23.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.947+0000 7ff8983ff700 1 -- 172.21.15.97:0/4192027700 >> 172.21.15.97:0/4192027700 conn(0x7ff890071790 msgr2=0x7ff8900b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:23.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.947+0000 7ff8983ff700 1 -- 172.21.15.97:0/4192027700 shutdown_connections 2023-07-06T03:54:23.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:23.947+0000 7ff8983ff700 1 -- 172.21.15.97:0/4192027700 wait complete. 2023-07-06T03:54:23.950 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-07-06T03:54:23.967 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T03:54:24.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.168+0000 7f93bedff700 1 Processor -- start 2023-07-06T03:54:24.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.168+0000 7f93bedff700 1 -- start start 2023-07-06T03:54:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.169+0000 7f93bedff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93b80b8590 0x7f93b80cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.169+0000 7f93bedff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93b80b4ea0 0x7f93b80d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.169+0000 7f93bedff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b80d0c10 con 0x7f93b8160a50 2023-07-06T03:54:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.169+0000 7f93bedff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b8160ec0 con 0x7f93b80b4ea0 2023-07-06T03:54:24.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.169+0000 7f93bedff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b80d08e0 con 0x7f93b80b8590 2023-07-06T03:54:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.169+0000 7f93b7fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f93b8160a50 0x7f93b80b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53546/0 (socket says 172.21.15.97:53546) 2023-07-06T03:54:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.169+0000 7f93b7fff700 1 -- 172.21.15.97:0/1662580882 learned_addr learned my addr 172.21.15.97:0/1662580882 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:24.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.170+0000 7f93bd39c700 1 --2- 172.21.15.97:0/1662580882 >> v2:172.21.15.97:3300/0 conn(0x7f93b80b4ea0 0x7f93b80d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:24.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.170+0000 7f93bcb9b700 1 --2- 172.21.15.97:0/1662580882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93b80b8590 0x7f93b80cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:24.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.170+0000 7f93b77fe700 1 -- 172.21.15.97:0/1662580882 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230466680 0 0) 0x7f93b80d0c10 con 0x7f93b8160a50 2023-07-06T03:54:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.170+0000 7f93b77fe700 1 -- 172.21.15.97:0/1662580882 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93a80035b0 con 0x7f93b8160a50 2023-07-06T03:54:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.170+0000 7f93bd39c700 1 -- 172.21.15.97:0/1662580882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93b80b8590 msgr2=0x7f93b80cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.170+0000 7f93bd39c700 1 --2- 172.21.15.97:0/1662580882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93b80b8590 0x7f93b80cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.170+0000 7f93bd39c700 1 -- 172.21.15.97:0/1662580882 >> v1:172.21.15.97:6789/0 conn(0x7f93b8160a50 legacy=0x7f93b80b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.170+0000 7f93bd39c700 1 -- 172.21.15.97:0/1662580882 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b80b2560 con 0x7f93b80b4ea0 2023-07-06T03:54:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.170+0000 7f93bd39c700 1 --2- 172.21.15.97:0/1662580882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93b80b4ea0 0x7f93b80d10a0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009d90 tx=0x7f93b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfb14e246466efa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.170+0000 7f93b77fe700 1 -- 172.21.15.97:0/1662580882 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b000e070 con 0x7f93b80b4ea0 2023-07-06T03:54:24.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.171+0000 7f93b77fe700 1 -- 172.21.15.97:0/1662580882 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b000bb20 con 0x7f93b80b4ea0 2023-07-06T03:54:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.171+0000 7f93b77fe700 1 -- 172.21.15.97:0/1662580882 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0012430 con 0x7f93b80b4ea0 2023-07-06T03:54:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.171+0000 7f93bedff700 1 -- 172.21.15.97:0/1662580882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93b80b4ea0 msgr2=0x7f93b80d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.171+0000 7f93bedff700 1 --2- 172.21.15.97:0/1662580882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93b80b4ea0 0x7f93b80d10a0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009d90 tx=0x7f93b000b330 comp rx=0 tx=0).stop 2023-07-06T03:54:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.171+0000 7f93bedff700 1 -- 172.21.15.97:0/1662580882 shutdown_connections 2023-07-06T03:54:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.171+0000 7f93bedff700 1 --2- 172.21.15.97:0/1662580882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93b80b4ea0 0x7f93b80d10a0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.171+0000 7f93bedff700 1 --2- 172.21.15.97:0/1662580882 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93b80b8590 0x7f93b80cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:24.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.171+0000 7f93bedff700 1 -- 172.21.15.97:0/1662580882 >> 172.21.15.97:0/1662580882 conn(0x7f93b806b290 msgr2=0x7f93b80cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:24.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.171+0000 7f93bedff700 1 -- 172.21.15.97:0/1662580882 shutdown_connections 2023-07-06T03:54:24.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.171+0000 7f93bedff700 1 -- 172.21.15.97:0/1662580882 wait complete. 2023-07-06T03:54:24.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.173+0000 7f93bedff700 1 Processor -- start 2023-07-06T03:54:24.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.173+0000 7f93bedff700 1 -- start start 2023-07-06T03:54:24.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.174+0000 7f93bedff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93b80b4ea0 0x7f93b8154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:24.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.174+0000 7f93bedff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93b80b8590 0x7f93b8153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:24.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.174+0000 7f93bedff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93b8160a50 0x7f93b8153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:24.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.174+0000 7f93bedff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8155280 con 0x7f93b80b8590 2023-07-06T03:54:24.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.174+0000 7f93bedff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8155560 con 0x7f93b8160a50 2023-07-06T03:54:24.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.174+0000 7f93bedff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8155840 con 0x7f93b80b4ea0 2023-07-06T03:54:24.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.174+0000 7f93b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93b80b8590 0x7f93b8153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.174+0000 7f93b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93b80b8590 0x7f93b8153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57202/0 (socket says 172.21.15.97:57202) 2023-07-06T03:54:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.174+0000 7f93b7fff700 1 -- 172.21.15.97:0/3894658124 learned_addr learned my addr 172.21.15.97:0/3894658124 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93bcb9b700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93b80b4ea0 0x7f93b8154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93bd39c700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93b8160a50 0x7f93b8153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93b7fff700 1 -- 172.21.15.97:0/3894658124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93b80b4ea0 msgr2=0x7f93b8154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93b7fff700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93b80b4ea0 0x7f93b8154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93b7fff700 1 -- 172.21.15.97:0/3894658124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93b8160a50 msgr2=0x7f93b8153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:24.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93b7fff700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93b8160a50 0x7f93b8153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93b7fff700 1 -- 172.21.15.97:0/3894658124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b0009a70 con 0x7f93b80b8590 2023-07-06T03:54:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93b7fff700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93b80b8590 0x7f93b8153d50 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f93a400c700 tx=0x7f93a400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93b57fa700 1 -- 172.21.15.97:0/3894658124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4005d70 con 0x7f93b80b8590 2023-07-06T03:54:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93b57fa700 1 -- 172.21.15.97:0/3894658124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a40070d0 con 0x7f93b80b8590 2023-07-06T03:54:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93bedff700 1 -- 172.21.15.97:0/3894658124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b8155b20 con 0x7f93b80b8590 2023-07-06T03:54:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.175+0000 7f93b57fa700 1 -- 172.21.15.97:0/3894658124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4007460 con 0x7f93b80b8590 2023-07-06T03:54:24.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.176+0000 7f93bedff700 1 -- 172.21.15.97:0/3894658124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b8154290 con 0x7f93b80b8590 2023-07-06T03:54:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.177+0000 7f93bedff700 1 -- 172.21.15.97:0/3894658124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b804e340 con 0x7f93b80b8590 2023-07-06T03:54:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.178+0000 7f93b57fa700 1 -- 172.21.15.97:0/3894658124 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f93a40075c0 con 0x7f93b80b8590 2023-07-06T03:54:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.179+0000 7f93b57fa700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939c06ace0 0x7f939c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.180+0000 7f93b57fa700 1 -- 172.21.15.97:0/3894658124 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f93a4088770 con 0x7f93b80b8590 2023-07-06T03:54:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.180+0000 7f93bcb9b700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939c06ace0 0x7f939c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.180+0000 7f93bcb9b700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939c06ace0 0x7f939c06d190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009b40 tx=0x7f93ac009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.184+0000 7f93b57fa700 1 -- 172.21.15.97:0/3894658124 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f93a4053970 con 0x7f93b80b8590 2023-07-06T03:54:24.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.382+0000 7f93bedff700 1 -- 172.21.15.97:0/3894658124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f93b80b6330 con 0x7f93b80b8590 2023-07-06T03:54:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.945+0000 7f93b57fa700 1 -- 172.21.15.97:0/3894658124 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) v1 ==== 170+0+0 (secure 0 0 0) 0x7f93a4017020 con 0x7f93b80b8590 2023-07-06T03:54:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.950+0000 7f93bedff700 1 -- 172.21.15.97:0/3894658124 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939c06ace0 msgr2=0x7f939c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:24.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.950+0000 7f93bedff700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939c06ace0 0x7f939c06d190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009b40 tx=0x7f93ac009210 comp rx=0 tx=0).stop 2023-07-06T03:54:24.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.951+0000 7f93bedff700 1 -- 172.21.15.97:0/3894658124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93b80b8590 msgr2=0x7f93b8153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:24.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.951+0000 7f93bedff700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93b80b8590 0x7f93b8153d50 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f93a400c700 tx=0x7f93a400ca10 comp rx=0 tx=0).stop 2023-07-06T03:54:24.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.952+0000 7f93bedff700 1 -- 172.21.15.97:0/3894658124 shutdown_connections 2023-07-06T03:54:24.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.952+0000 7f93bedff700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f939c06ace0 0x7f939c06d190 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.952+0000 7f93bedff700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93b8160a50 0x7f93b8153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.952+0000 7f93bedff700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93b80b4ea0 0x7f93b8154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.952+0000 7f93bedff700 1 --2- 172.21.15.97:0/3894658124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93b80b8590 0x7f93b8153d50 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.952+0000 7f93bedff700 1 -- 172.21.15.97:0/3894658124 >> 172.21.15.97:0/3894658124 conn(0x7f93b806b290 msgr2=0x7f93b80cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.957+0000 7f93bedff700 1 -- 172.21.15.97:0/3894658124 shutdown_connections 2023-07-06T03:54:24.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:24.961+0000 7f93bedff700 1 -- 172.21.15.97:0/3894658124 wait complete. 2023-07-06T03:54:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-07-06T03:54:24.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T03:54:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.184+0000 7fc5d45ff700 1 Processor -- start 2023-07-06T03:54:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.185+0000 7fc5d45ff700 1 -- start start 2023-07-06T03:54:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.186+0000 7fc5d45ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5cc15c610 0x7fc5cc0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.186+0000 7fc5d45ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc5cc06b360 0x7fc5cc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.186+0000 7fc5d45ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5cc0cff90 con 0x7fc5cc0d2210 2023-07-06T03:54:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.186+0000 7fc5d45ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc0d0780 con 0x7fc5cc06b360 2023-07-06T03:54:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.186+0000 7fc5d45ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc15c9e0 con 0x7fc5cc15c610 2023-07-06T03:54:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.186+0000 7fc5d1b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc5cc06b360 0x7fc5cc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.186+0000 7fc5d1b9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc5cc06b360 0x7fc5cc0b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57214/0 (socket says 172.21.15.97:57214) 2023-07-06T03:54:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.186+0000 7fc5d1b9a700 1 -- 172.21.15.97:0/2456430421 learned_addr learned my addr 172.21.15.97:0/2456430421 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.186+0000 7fc5d2b9c700 1 --1- 172.21.15.97:0/2456430421 >> v1:172.21.15.97:6789/0 conn(0x7fc5cc0d2210 0x7fc5cc0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53556/0 (socket says 172.21.15.97:53556) 2023-07-06T03:54:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.187+0000 7fc5d239b700 1 --2- 172.21.15.97:0/2456430421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5cc15c610 0x7fc5cc0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.187+0000 7fc5d1b9a700 1 -- 172.21.15.97:0/2456430421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5cc15c610 msgr2=0x7fc5cc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.187+0000 7fc5d1b9a700 1 --2- 172.21.15.97:0/2456430421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5cc15c610 0x7fc5cc0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.187+0000 7fc5d1b9a700 1 -- 172.21.15.97:0/2456430421 >> v1:172.21.15.97:6789/0 conn(0x7fc5cc0d2210 legacy=0x7fc5cc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.187+0000 7fc5d1b9a700 1 -- 172.21.15.97:0/2456430421 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc0b5650 con 0x7fc5cc06b360 2023-07-06T03:54:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.187+0000 7fc5d1399700 1 -- 172.21.15.97:0/2456430421 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970408998 0 0) 0x7fc5cc0cff90 con 0x7fc5cc0d2210 2023-07-06T03:54:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.187+0000 7fc5d1b9a700 1 --2- 172.21.15.97:0/2456430421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc06b360 0x7fc5cc0b4cf0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009920 tx=0x7fc5bc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c51877147e25523 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.187+0000 7fc5d1399700 1 -- 172.21.15.97:0/2456430421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc00ba20 con 0x7fc5cc06b360 2023-07-06T03:54:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.187+0000 7fc5d1399700 1 -- 172.21.15.97:0/2456430421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc00bb80 con 0x7fc5cc06b360 2023-07-06T03:54:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.187+0000 7fc5d1399700 1 -- 172.21.15.97:0/2456430421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc009e70 con 0x7fc5cc06b360 2023-07-06T03:54:25.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.188+0000 7fc5d45ff700 1 -- 172.21.15.97:0/2456430421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc06b360 msgr2=0x7fc5cc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:25.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.188+0000 7fc5d45ff700 1 --2- 172.21.15.97:0/2456430421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc06b360 0x7fc5cc0b4cf0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009920 tx=0x7fc5bc009c30 comp rx=0 tx=0).stop 2023-07-06T03:54:25.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.188+0000 7fc5d45ff700 1 -- 172.21.15.97:0/2456430421 shutdown_connections 2023-07-06T03:54:25.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.188+0000 7fc5d45ff700 1 --2- 172.21.15.97:0/2456430421 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5cc15c610 0x7fc5cc0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:25.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.188+0000 7fc5d45ff700 1 --2- 172.21.15.97:0/2456430421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc06b360 0x7fc5cc0b4cf0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:25.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.188+0000 7fc5d45ff700 1 -- 172.21.15.97:0/2456430421 >> 172.21.15.97:0/2456430421 conn(0x7fc5cc06a6c0 msgr2=0x7fc5cc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:25.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.188+0000 7fc5d45ff700 1 -- 172.21.15.97:0/2456430421 shutdown_connections 2023-07-06T03:54:25.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.188+0000 7fc5d45ff700 1 -- 172.21.15.97:0/2456430421 wait complete. 2023-07-06T03:54:25.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.190+0000 7fc5d45ff700 1 Processor -- start 2023-07-06T03:54:25.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.190+0000 7fc5d45ff700 1 -- start start 2023-07-06T03:54:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d45ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5cc06b360 0x7fc5cc15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d45ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5cc0d2210 0x7fc5cc0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d45ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc15c610 0x7fc5cc14b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d45ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0cab90 con 0x7fc5cc15c610 2023-07-06T03:54:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d45ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0c9f10 con 0x7fc5cc06b360 2023-07-06T03:54:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d45ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0ca1f0 con 0x7fc5cc0d2210 2023-07-06T03:54:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d2b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc15c610 0x7fc5cc14b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d2b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc15c610 0x7fc5cc14b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57230/0 (socket says 172.21.15.97:57230) 2023-07-06T03:54:25.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d2b9c700 1 -- 172.21.15.97:0/203534998 learned_addr learned my addr 172.21.15.97:0/203534998 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:25.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d1b9a700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5cc0d2210 0x7fc5cc0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:25.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d239b700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5cc06b360 0x7fc5cc15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:25.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d2b9c700 1 -- 172.21.15.97:0/203534998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5cc0d2210 msgr2=0x7fc5cc0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:25.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d2b9c700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5cc0d2210 0x7fc5cc0cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:25.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d2b9c700 1 -- 172.21.15.97:0/203534998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5cc06b360 msgr2=0x7fc5cc15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:25.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d2b9c700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5cc06b360 0x7fc5cc15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:25.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.191+0000 7fc5d2b9c700 1 -- 172.21.15.97:0/203534998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc009550 con 0x7fc5cc15c610 2023-07-06T03:54:25.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.192+0000 7fc5d2b9c700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc15c610 0x7fc5cc14b070 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4002230 tx=0x7fc5c400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:25.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.192+0000 7fc5c2ffd700 1 -- 172.21.15.97:0/203534998 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400ac20 con 0x7fc5cc15c610 2023-07-06T03:54:25.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.192+0000 7fc5c2ffd700 1 -- 172.21.15.97:0/203534998 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c400ada0 con 0x7fc5cc15c610 2023-07-06T03:54:25.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.192+0000 7fc5c2ffd700 1 -- 172.21.15.97:0/203534998 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4007310 con 0x7fc5cc15c610 2023-07-06T03:54:25.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.192+0000 7fc5d45ff700 1 -- 172.21.15.97:0/203534998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc0ca4d0 con 0x7fc5cc15c610 2023-07-06T03:54:25.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.192+0000 7fc5d45ff700 1 -- 172.21.15.97:0/203534998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc15ef30 con 0x7fc5cc15c610 2023-07-06T03:54:25.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.193+0000 7fc5d45ff700 1 -- 172.21.15.97:0/203534998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc0cacd0 con 0x7fc5cc15c610 2023-07-06T03:54:25.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.194+0000 7fc5c2ffd700 1 -- 172.21.15.97:0/203534998 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc5c4007470 con 0x7fc5cc15c610 2023-07-06T03:54:25.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.195+0000 7fc5c2ffd700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc5b006ace0 0x7fc5b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:25.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.196+0000 7fc5c2ffd700 1 -- 172.21.15.97:0/203534998 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fc5c4088fe0 con 0x7fc5cc15c610 2023-07-06T03:54:25.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.196+0000 7fc5d239b700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc5b006ace0 0x7fc5b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:25.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.196+0000 7fc5d239b700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc5b006ace0 0x7fc5b006d190 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800b9b0 tx=0x7fc5c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:25.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.200+0000 7fc5c2ffd700 1 -- 172.21.15.97:0/203534998 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc5cc0cacd0 con 0x7fc5cc15c610 2023-07-06T03:54:25.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.440+0000 7fc5d45ff700 1 -- 172.21.15.97:0/203534998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc5cc0cacd0 con 0x7fc5cc15c610 2023-07-06T03:54:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.964+0000 7fc5c2ffd700 1 -- 172.21.15.97:0/203534998 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v76) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc5cc0cacd0 con 0x7fc5cc15c610 2023-07-06T03:54:25.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.969+0000 7fc5d45ff700 1 -- 172.21.15.97:0/203534998 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc5b006ace0 msgr2=0x7fc5b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:25.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.969+0000 7fc5d45ff700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc5b006ace0 0x7fc5b006d190 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800b9b0 tx=0x7fc5c8008040 comp rx=0 tx=0).stop 2023-07-06T03:54:25.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.969+0000 7fc5d45ff700 1 -- 172.21.15.97:0/203534998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc15c610 msgr2=0x7fc5cc14b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.969+0000 7fc5d45ff700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc15c610 0x7fc5cc14b070 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4002230 tx=0x7fc5c400a060 comp rx=0 tx=0).stop 2023-07-06T03:54:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.969+0000 7fc5d45ff700 1 -- 172.21.15.97:0/203534998 shutdown_connections 2023-07-06T03:54:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.969+0000 7fc5d45ff700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc5b006ace0 0x7fc5b006d190 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.969+0000 7fc5d45ff700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5cc15c610 0x7fc5cc14b070 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.969+0000 7fc5d45ff700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5cc06b360 0x7fc5cc15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.969+0000 7fc5d45ff700 1 --2- 172.21.15.97:0/203534998 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5cc0d2210 0x7fc5cc0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.969+0000 7fc5d45ff700 1 -- 172.21.15.97:0/203534998 >> 172.21.15.97:0/203534998 conn(0x7fc5cc06a6c0 msgr2=0x7fc5cc0b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.971+0000 7fc5d45ff700 1 -- 172.21.15.97:0/203534998 shutdown_connections 2023-07-06T03:54:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:25.972+0000 7fc5d45ff700 1 -- 172.21.15.97:0/203534998 wait complete. 2023-07-06T03:54:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 17 and data pool 18 2023-07-06T03:54:26.012 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:26.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.207+0000 7fd4163ff700 1 Processor -- start 2023-07-06T03:54:26.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.207+0000 7fd4163ff700 1 -- start start 2023-07-06T03:54:26.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.208+0000 7fd4163ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4100cd8e0 0x7fd41015e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:26.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.208+0000 7fd4163ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd410070c30 0x7fd4100adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:26.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.208+0000 7fd4163ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4100710a0 con 0x7fd4100cdcb0 2023-07-06T03:54:26.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.208+0000 7fd4163ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4100ae3e0 con 0x7fd410070c30 2023-07-06T03:54:26.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.208+0000 7fd4163ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd410071330 con 0x7fd4100cd8e0 2023-07-06T03:54:26.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.208+0000 7fd40f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd410070c30 0x7fd4100adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:26.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.209+0000 7fd40f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd410070c30 0x7fd4100adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57234/0 (socket says 172.21.15.97:57234) 2023-07-06T03:54:26.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.209+0000 7fd40f7fe700 1 -- 172.21.15.97:0/1023014267 learned_addr learned my addr 172.21.15.97:0/1023014267 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:26.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.209+0000 7fd40ffff700 1 --2- 172.21.15.97:0/1023014267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4100cd8e0 0x7fd41015e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:26.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.209+0000 7fd40f7fe700 1 -- 172.21.15.97:0/1023014267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4100cd8e0 msgr2=0x7fd41015e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.209+0000 7fd40f7fe700 1 --2- 172.21.15.97:0/1023014267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4100cd8e0 0x7fd41015e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.209+0000 7fd40f7fe700 1 -- 172.21.15.97:0/1023014267 >> v1:172.21.15.146:6789/0 conn(0x7fd4100cdcb0 legacy=0x7fd41006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.209+0000 7fd40f7fe700 1 -- 172.21.15.97:0/1023014267 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd410071610 con 0x7fd410070c30 2023-07-06T03:54:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.209+0000 7fd40f7fe700 1 --2- 172.21.15.97:0/1023014267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd410070c30 0x7fd4100adea0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd4040098f0 tx=0x7fd404009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99bd63e05b10736 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.209+0000 7fd40effd700 1 -- 172.21.15.97:0/1023014267 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404011830 con 0x7fd410070c30 2023-07-06T03:54:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.210+0000 7fd40effd700 1 -- 172.21.15.97:0/1023014267 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd404011990 con 0x7fd410070c30 2023-07-06T03:54:26.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.210+0000 7fd40effd700 1 -- 172.21.15.97:0/1023014267 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404011d20 con 0x7fd410070c30 2023-07-06T03:54:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.210+0000 7fd4163ff700 1 -- 172.21.15.97:0/1023014267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd410070c30 msgr2=0x7fd4100adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.210+0000 7fd4163ff700 1 --2- 172.21.15.97:0/1023014267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd410070c30 0x7fd4100adea0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd4040098f0 tx=0x7fd404009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.210+0000 7fd4163ff700 1 -- 172.21.15.97:0/1023014267 shutdown_connections 2023-07-06T03:54:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.210+0000 7fd4163ff700 1 --2- 172.21.15.97:0/1023014267 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4100cd8e0 0x7fd41015e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.210+0000 7fd4163ff700 1 --2- 172.21.15.97:0/1023014267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd410070c30 0x7fd4100adea0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.210+0000 7fd4163ff700 1 -- 172.21.15.97:0/1023014267 >> 172.21.15.97:0/1023014267 conn(0x7fd4100ce8c0 msgr2=0x7fd4100cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.210+0000 7fd4163ff700 1 -- 172.21.15.97:0/1023014267 shutdown_connections 2023-07-06T03:54:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.210+0000 7fd4163ff700 1 -- 172.21.15.97:0/1023014267 wait complete. 2023-07-06T03:54:26.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.212+0000 7fd4163ff700 1 Processor -- start 2023-07-06T03:54:26.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.212+0000 7fd4163ff700 1 -- start start 2023-07-06T03:54:26.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.213+0000 7fd4163ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd410070c30 0x7fd410144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:26.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.213+0000 7fd4163ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4100cd8e0 0x7fd4100af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:26.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.213+0000 7fd4163ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4100cdcb0 0x7fd4100afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:26.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.213+0000 7fd4163ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4101448d0 con 0x7fd4100cdcb0 2023-07-06T03:54:26.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.213+0000 7fd4163ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4100acd20 con 0x7fd410070c30 2023-07-06T03:54:26.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.213+0000 7fd4163ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4100ad000 con 0x7fd4100cd8e0 2023-07-06T03:54:26.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.213+0000 7fd41499c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4100cdcb0 0x7fd4100afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:26.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.213+0000 7fd40f7fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4100cd8e0 0x7fd4100af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:26.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.213+0000 7fd41499c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4100cdcb0 0x7fd4100afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57240/0 (socket says 172.21.15.97:57240) 2023-07-06T03:54:26.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.213+0000 7fd41499c700 1 -- 172.21.15.97:0/629253119 learned_addr learned my addr 172.21.15.97:0/629253119 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:26.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.213+0000 7fd40ffff700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd410070c30 0x7fd410144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:26.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.214+0000 7fd40f7fe700 1 -- 172.21.15.97:0/629253119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd410070c30 msgr2=0x7fd410144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.214+0000 7fd40f7fe700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd410070c30 0x7fd410144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.214+0000 7fd40f7fe700 1 -- 172.21.15.97:0/629253119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4100cdcb0 msgr2=0x7fd4100afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.214+0000 7fd40f7fe700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4100cdcb0 0x7fd4100afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.214+0000 7fd40f7fe700 1 -- 172.21.15.97:0/629253119 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd404009550 con 0x7fd4100cd8e0 2023-07-06T03:54:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.214+0000 7fd41499c700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4100cdcb0 0x7fd4100afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:54:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.214+0000 7fd40f7fe700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4100cd8e0 0x7fd4100af830 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd404016000 tx=0x7fd404004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.214+0000 7fd40cff9700 1 -- 172.21.15.97:0/629253119 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404004510 con 0x7fd4100cd8e0 2023-07-06T03:54:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.214+0000 7fd40cff9700 1 -- 172.21.15.97:0/629253119 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40401e070 con 0x7fd4100cd8e0 2023-07-06T03:54:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.215+0000 7fd4163ff700 1 -- 172.21.15.97:0/629253119 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4100ad2e0 con 0x7fd4100cd8e0 2023-07-06T03:54:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.215+0000 7fd40cff9700 1 -- 172.21.15.97:0/629253119 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4040038a0 con 0x7fd4100cd8e0 2023-07-06T03:54:26.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.215+0000 7fd4163ff700 1 -- 172.21.15.97:0/629253119 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd410144ee0 con 0x7fd4100cd8e0 2023-07-06T03:54:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.216+0000 7fd4163ff700 1 -- 172.21.15.97:0/629253119 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd410070aa0 con 0x7fd4100cd8e0 2023-07-06T03:54:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.217+0000 7fd40cff9700 1 -- 172.21.15.97:0/629253119 <== mon.2 v2:172.21.15.146:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd404003a00 con 0x7fd4100cd8e0 2023-07-06T03:54:26.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.218+0000 7fd40cff9700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3fc06ace0 0x7fd3fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:26.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.218+0000 7fd40cff9700 1 -- 172.21.15.97:0/629253119 <== mon.2 v2:172.21.15.146:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd40408aa90 con 0x7fd4100cd8e0 2023-07-06T03:54:26.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.218+0000 7fd40ffff700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3fc06ace0 0x7fd3fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:26.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.218+0000 7fd40ffff700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3fc06ace0 0x7fd3fc06d190 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd400009be0 tx=0x7fd400009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:26.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.222+0000 7fd40cff9700 1 -- 172.21.15.97:0/629253119 <== mon.2 v2:172.21.15.146:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd404055b30 con 0x7fd4100cd8e0 2023-07-06T03:54:26.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.419+0000 7fd4163ff700 1 -- 172.21.15.97:0/629253119 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd410067830 con 0x7fd4100cd8e0 2023-07-06T03:54:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.421+0000 7fd40cff9700 1 -- 172.21.15.97:0/629253119 <== mon.2 v2:172.21.15.146:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+18404 (secure 0 0 0) 0x7fd404003cb0 con 0x7fd4100cd8e0 2023-07-06T03:54:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:26.423 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":111,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:54:25.441555+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:54:23.626811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-06T03:54:24.421787+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:26.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 -- 172.21.15.97:0/629253119 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3fc06ace0 msgr2=0x7fd3fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3fc06ace0 0x7fd3fc06d190 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd400009be0 tx=0x7fd400009210 comp rx=0 tx=0).stop 2023-07-06T03:54:26.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 -- 172.21.15.97:0/629253119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4100cd8e0 msgr2=0x7fd4100af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4100cd8e0 0x7fd4100af830 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd404016000 tx=0x7fd404004750 comp rx=0 tx=0).stop 2023-07-06T03:54:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 -- 172.21.15.97:0/629253119 shutdown_connections 2023-07-06T03:54:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3fc06ace0 0x7fd3fc06d190 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4100cdcb0 0x7fd4100afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd410070c30 0x7fd410144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 --2- 172.21.15.97:0/629253119 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd4100cd8e0 0x7fd4100af830 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 -- 172.21.15.97:0/629253119 >> 172.21.15.97:0/629253119 conn(0x7fd4100ce8c0 msgr2=0x7fd410159620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 -- 172.21.15.97:0/629253119 shutdown_connections 2023-07-06T03:54:26.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.423+0000 7fd4163ff700 1 -- 172.21.15.97:0/629253119 wait complete. 2023-07-06T03:54:26.443 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.635+0000 7fcac57ff700 1 Processor -- start 2023-07-06T03:54:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.636+0000 7fcac57ff700 1 -- start start 2023-07-06T03:54:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.637+0000 7fcac57ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcac006caa0 0x7fcac00d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.637+0000 7fcac57ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac00cfd00 con 0x7fcac015c920 2023-07-06T03:54:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.637+0000 7fcac57ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac00b28f0 con 0x7fcac015ccf0 2023-07-06T03:54:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.637+0000 7fcac57ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac01610b0 con 0x7fcac006caa0 2023-07-06T03:54:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.637+0000 7fcabeffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcac015c920 0x7fcac006c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53562/0 (socket says 172.21.15.97:53562) 2023-07-06T03:54:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.637+0000 7fcabeffd700 1 -- 172.21.15.97:0/4185381162 learned_addr learned my addr 172.21.15.97:0/4185381162 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.637+0000 7fcabf7fe700 1 --2- 172.21.15.97:0/4185381162 >> v2:172.21.15.146:3300/0 conn(0x7fcac006caa0 0x7fcac00d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.638+0000 7fcabdffb700 1 -- 172.21.15.97:0/4185381162 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477705616 0 0) 0x7fcac00cfd00 con 0x7fcac015c920 2023-07-06T03:54:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.638+0000 7fcabdffb700 1 -- 172.21.15.97:0/4185381162 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa80035b0 con 0x7fcac015c920 2023-07-06T03:54:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.638+0000 7fcabdffb700 1 -- 172.21.15.97:0/4185381162 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160872699 0 0) 0x7fcac00b28f0 con 0x7fcac015ccf0 2023-07-06T03:54:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.638+0000 7fcabf7fe700 1 -- 172.21.15.97:0/4185381162 >> v1:172.21.15.146:6789/0 conn(0x7fcac015ccf0 legacy=0x7fcac00d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.638+0000 7fcabf7fe700 1 -- 172.21.15.97:0/4185381162 >> v1:172.21.15.97:6789/0 conn(0x7fcac015c920 legacy=0x7fcac006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.638+0000 7fcabf7fe700 1 -- 172.21.15.97:0/4185381162 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac0161390 con 0x7fcac006caa0 2023-07-06T03:54:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.638+0000 7fcabf7fe700 1 --2- 172.21.15.97:0/4185381162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac006caa0 0x7fcac00d08a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009d90 tx=0x7fcab800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874cc597654ffa9f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.638+0000 7fcabdffb700 1 -- 172.21.15.97:0/4185381162 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab800e070 con 0x7fcac006caa0 2023-07-06T03:54:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.638+0000 7fcabdffb700 1 -- 172.21.15.97:0/4185381162 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab800bb20 con 0x7fcac006caa0 2023-07-06T03:54:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.638+0000 7fcabdffb700 1 -- 172.21.15.97:0/4185381162 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab800bed0 con 0x7fcac006caa0 2023-07-06T03:54:26.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.639+0000 7fcac57ff700 1 -- 172.21.15.97:0/4185381162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac006caa0 msgr2=0x7fcac00d08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.639+0000 7fcac57ff700 1 --2- 172.21.15.97:0/4185381162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac006caa0 0x7fcac00d08a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009d90 tx=0x7fcab800b330 comp rx=0 tx=0).stop 2023-07-06T03:54:26.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.639+0000 7fcac57ff700 1 -- 172.21.15.97:0/4185381162 shutdown_connections 2023-07-06T03:54:26.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.639+0000 7fcac57ff700 1 --2- 172.21.15.97:0/4185381162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac006caa0 0x7fcac00d08a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.639+0000 7fcac57ff700 1 -- 172.21.15.97:0/4185381162 >> 172.21.15.97:0/4185381162 conn(0x7fcac006ace0 msgr2=0x7fcac006dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:26.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.639+0000 7fcac57ff700 1 -- 172.21.15.97:0/4185381162 shutdown_connections 2023-07-06T03:54:26.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.639+0000 7fcac57ff700 1 -- 172.21.15.97:0/4185381162 wait complete. 2023-07-06T03:54:26.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.641+0000 7fcac57ff700 1 Processor -- start 2023-07-06T03:54:26.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.641+0000 7fcac57ff700 1 -- start start 2023-07-06T03:54:26.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcac57ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac015c920 0x7fcac00b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:26.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcac57ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac015ccf0 0x7fcac00b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:26.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcac57ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac00c97c0 0x7fcac00c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:26.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcac57ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00ca180 con 0x7fcac00c97c0 2023-07-06T03:54:26.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcac57ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00ae230 con 0x7fcac015ccf0 2023-07-06T03:54:26.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcac57ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00c5b00 con 0x7fcac015c920 2023-07-06T03:54:26.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac00c97c0 0x7fcac00c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:26.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac00c97c0 0x7fcac00c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57244/0 (socket says 172.21.15.97:57244) 2023-07-06T03:54:26.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcabf7fe700 1 -- 172.21.15.97:0/138739770 learned_addr learned my addr 172.21.15.97:0/138739770 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:26.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcabf7fe700 1 -- 172.21.15.97:0/138739770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac015c920 msgr2=0x7fcac00b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcabf7fe700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac015c920 0x7fcac00b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcabf7fe700 1 -- 172.21.15.97:0/138739770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac015ccf0 msgr2=0x7fcac00b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcabf7fe700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac015ccf0 0x7fcac00b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.642+0000 7fcabf7fe700 1 -- 172.21.15.97:0/138739770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab8009a70 con 0x7fcac00c97c0 2023-07-06T03:54:26.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.643+0000 7fcabf7fe700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac00c97c0 0x7fcac00c9c40 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fcac0067b20 tx=0x7fcab8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:26.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.643+0000 7fcaa77fe700 1 -- 172.21.15.97:0/138739770 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab801c810 con 0x7fcac00c97c0 2023-07-06T03:54:26.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.643+0000 7fcaa77fe700 1 -- 172.21.15.97:0/138739770 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab8020070 con 0x7fcac00c97c0 2023-07-06T03:54:26.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.643+0000 7fcaa77fe700 1 -- 172.21.15.97:0/138739770 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab801cbb0 con 0x7fcac00c97c0 2023-07-06T03:54:26.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.643+0000 7fcac57ff700 1 -- 172.21.15.97:0/138739770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac00c63a0 con 0x7fcac00c97c0 2023-07-06T03:54:26.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.643+0000 7fcac57ff700 1 -- 172.21.15.97:0/138739770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac00c5d50 con 0x7fcac00c97c0 2023-07-06T03:54:26.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.644+0000 7fcac57ff700 1 -- 172.21.15.97:0/138739770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac0066a40 con 0x7fcac00c97c0 2023-07-06T03:54:26.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.645+0000 7fcaa77fe700 1 -- 172.21.15.97:0/138739770 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcab801cd10 con 0x7fcac00c97c0 2023-07-06T03:54:26.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.646+0000 7fcaa77fe700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcaac06ace0 0x7fcaac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:26.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.646+0000 7fcaa77fe700 1 -- 172.21.15.97:0/138739770 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcab8022070 con 0x7fcac00c97c0 2023-07-06T03:54:26.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.646+0000 7fcabeffd700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcaac06ace0 0x7fcaac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:26.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.647+0000 7fcabeffd700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcaac06ace0 0x7fcaac06d190 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcab0007fd0 tx=0x7fcab0007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:26.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.650+0000 7fcaa77fe700 1 -- 172.21.15.97:0/138739770 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcab805a060 con 0x7fcac00c97c0 2023-07-06T03:54:26.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.844+0000 7fcac57ff700 1 -- 172.21.15.97:0/138739770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcac004e340 con 0x7fcac00c97c0 2023-07-06T03:54:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.845+0000 7fcaa77fe700 1 -- 172.21.15.97:0/138739770 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+18404 (secure 0 0 0) 0x7fcab8059bf0 con 0x7fcac00c97c0 2023-07-06T03:54:26.847 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:26.847 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":111,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:54:25.441555+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:54:23.626811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-06T03:54:24.421787+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:26.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 -- 172.21.15.97:0/138739770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcaac06ace0 msgr2=0x7fcaac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcaac06ace0 0x7fcaac06d190 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcab0007fd0 tx=0x7fcab0007980 comp rx=0 tx=0).stop 2023-07-06T03:54:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 -- 172.21.15.97:0/138739770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac00c97c0 msgr2=0x7fcac00c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac00c97c0 0x7fcac00c9c40 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fcac0067b20 tx=0x7fcab8004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 -- 172.21.15.97:0/138739770 shutdown_connections 2023-07-06T03:54:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcaac06ace0 0x7fcaac06d190 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac00c97c0 0x7fcac00c9c40 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcac015c920 0x7fcac00b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 --2- 172.21.15.97:0/138739770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcac015ccf0 0x7fcac00b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 -- 172.21.15.97:0/138739770 >> 172.21.15.97:0/138739770 conn(0x7fcac006caa0 msgr2=0x7fcac006b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 -- 172.21.15.97:0/138739770 shutdown_connections 2023-07-06T03:54:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:26.848+0000 7fcac57ff700 1 -- 172.21.15.97:0/138739770 wait complete. 2023-07-06T03:54:26.868 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T03:54:27.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.068+0000 7fb2303ff700 1 Processor -- start 2023-07-06T03:54:27.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.069+0000 7fb2303ff700 1 -- start start 2023-07-06T03:54:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.070+0000 7fb2303ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb22815c610 0x7fb2280b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.070+0000 7fb2303ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb22806b360 0x7fb2280b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.070+0000 7fb2303ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb2280d2210 0x7fb2280d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.070+0000 7fb2303ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2280d0780 con 0x7fb22806b360 2023-07-06T03:54:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.070+0000 7fb2303ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22815c9e0 con 0x7fb2280d2210 2023-07-06T03:54:27.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.070+0000 7fb2303ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280b5650 con 0x7fb22815c610 2023-07-06T03:54:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.070+0000 7fb22d99a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb22806b360 0x7fb2280b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.070+0000 7fb22d99a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb22806b360 0x7fb2280b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57258/0 (socket says 172.21.15.97:57258) 2023-07-06T03:54:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.070+0000 7fb22d99a700 1 -- 172.21.15.97:0/2437213352 learned_addr learned my addr 172.21.15.97:0/2437213352 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.070+0000 7fb22e19b700 1 --2- 172.21.15.97:0/2437213352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb22815c610 0x7fb2280b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:27.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.070+0000 7fb22e99c700 1 --2- 172.21.15.97:0/2437213352 >> v2:172.21.15.146:3300/0 conn(0x7fb2280d2210 0x7fb2280d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:27.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.071+0000 7fb22d99a700 1 -- 172.21.15.97:0/2437213352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb22815c610 msgr2=0x7fb2280b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:27.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.071+0000 7fb22d99a700 1 --2- 172.21.15.97:0/2437213352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb22815c610 0x7fb2280b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:27.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.071+0000 7fb22d99a700 1 -- 172.21.15.97:0/2437213352 >> v2:172.21.15.146:3300/0 conn(0x7fb2280d2210 msgr2=0x7fb2280d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:27.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.071+0000 7fb22d99a700 1 --2- 172.21.15.97:0/2437213352 >> v2:172.21.15.146:3300/0 conn(0x7fb2280d2210 0x7fb2280d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:27.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.071+0000 7fb22d99a700 1 -- 172.21.15.97:0/2437213352 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280d01c0 con 0x7fb22806b360 2023-07-06T03:54:27.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.071+0000 7fb22e99c700 1 --2- 172.21.15.97:0/2437213352 >> v2:172.21.15.146:3300/0 conn(0x7fb2280d2210 0x7fb2280d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:27.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.071+0000 7fb22d99a700 1 --2- 172.21.15.97:0/2437213352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb22806b360 0x7fb2280b4cf0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb2280b1b70 tx=0x7fb218009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303fd8749e38e3f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:27.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.071+0000 7fb22d199700 1 -- 172.21.15.97:0/2437213352 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218011830 con 0x7fb22806b360 2023-07-06T03:54:27.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.071+0000 7fb22d199700 1 -- 172.21.15.97:0/2437213352 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb218011990 con 0x7fb22806b360 2023-07-06T03:54:27.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.071+0000 7fb22d199700 1 -- 172.21.15.97:0/2437213352 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218011d20 con 0x7fb22806b360 2023-07-06T03:54:27.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.072+0000 7fb2303ff700 1 -- 172.21.15.97:0/2437213352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb22806b360 msgr2=0x7fb2280b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:27.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.072+0000 7fb2303ff700 1 --2- 172.21.15.97:0/2437213352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb22806b360 0x7fb2280b4cf0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb2280b1b70 tx=0x7fb218009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:27.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.072+0000 7fb2303ff700 1 -- 172.21.15.97:0/2437213352 shutdown_connections 2023-07-06T03:54:27.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.072+0000 7fb2303ff700 1 --2- 172.21.15.97:0/2437213352 >> v2:172.21.15.146:3300/0 conn(0x7fb2280d2210 0x7fb2280d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:27.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.072+0000 7fb2303ff700 1 --2- 172.21.15.97:0/2437213352 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb22815c610 0x7fb2280b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:27.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.072+0000 7fb2303ff700 1 --2- 172.21.15.97:0/2437213352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb22806b360 0x7fb2280b4cf0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:27.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.072+0000 7fb2303ff700 1 -- 172.21.15.97:0/2437213352 >> 172.21.15.97:0/2437213352 conn(0x7fb22806a6c0 msgr2=0x7fb22806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.072+0000 7fb2303ff700 1 -- 172.21.15.97:0/2437213352 shutdown_connections 2023-07-06T03:54:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.072+0000 7fb2303ff700 1 -- 172.21.15.97:0/2437213352 wait complete. 2023-07-06T03:54:27.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.074+0000 7fb2303ff700 1 Processor -- start 2023-07-06T03:54:27.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.074+0000 7fb2303ff700 1 -- start start 2023-07-06T03:54:27.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb2303ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb22806b360 0x7fb22815f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:27.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb2303ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2280d2210 0x7fb2280b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:27.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb2303ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb22815c610 0x7fb2280c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:27.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb2303ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280c9b10 con 0x7fb22806b360 2023-07-06T03:54:27.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb2303ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280b6da0 con 0x7fb2280d2210 2023-07-06T03:54:27.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb2303ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280b6ad0 con 0x7fb22815c610 2023-07-06T03:54:27.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22e19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb22806b360 0x7fb22815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:27.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22e19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb22806b360 0x7fb22815f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57274/0 (socket says 172.21.15.97:57274) 2023-07-06T03:54:27.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22e19b700 1 -- 172.21.15.97:0/1684353870 learned_addr learned my addr 172.21.15.97:0/1684353870 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:27.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22e99c700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb22815c610 0x7fb2280c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:27.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22e19b700 1 -- 172.21.15.97:0/1684353870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb22815c610 msgr2=0x7fb2280c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:27.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22e19b700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb22815c610 0x7fb2280c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:27.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22d99a700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2280d2210 0x7fb2280b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:27.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22e19b700 1 -- 172.21.15.97:0/1684353870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2280d2210 msgr2=0x7fb2280b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:27.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22e19b700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2280d2210 0x7fb2280b6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:27.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22e19b700 1 -- 172.21.15.97:0/1684353870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218009550 con 0x7fb22806b360 2023-07-06T03:54:27.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22e99c700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb22815c610 0x7fb2280c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:27.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.075+0000 7fb22e19b700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb22806b360 0x7fb22815f650 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fb22400e870 tx=0x7fb22400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:27.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.076+0000 7fb21effd700 1 -- 172.21.15.97:0/1684353870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224009590 con 0x7fb22806b360 2023-07-06T03:54:27.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.076+0000 7fb21effd700 1 -- 172.21.15.97:0/1684353870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22400ee70 con 0x7fb22806b360 2023-07-06T03:54:27.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.076+0000 7fb21effd700 1 -- 172.21.15.97:0/1684353870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22400ca50 con 0x7fb22806b360 2023-07-06T03:54:27.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.076+0000 7fb2303ff700 1 -- 172.21.15.97:0/1684353870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22815d950 con 0x7fb22806b360 2023-07-06T03:54:27.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.076+0000 7fb2303ff700 1 -- 172.21.15.97:0/1684353870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22815df10 con 0x7fb22806b360 2023-07-06T03:54:27.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.077+0000 7fb2303ff700 1 -- 172.21.15.97:0/1684353870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22804e340 con 0x7fb22806b360 2023-07-06T03:54:27.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.078+0000 7fb21effd700 1 -- 172.21.15.97:0/1684353870 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb22400cbb0 con 0x7fb22806b360 2023-07-06T03:54:27.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.079+0000 7fb21effd700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb20c06ace0 0x7fb20c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:27.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.079+0000 7fb21effd700 1 -- 172.21.15.97:0/1684353870 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb224089530 con 0x7fb22806b360 2023-07-06T03:54:27.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.079+0000 7fb22d99a700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb20c06ace0 0x7fb20c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:27.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.080+0000 7fb22d99a700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb20c06ace0 0x7fb20c06d190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb22806b790 tx=0x7fb218008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.083+0000 7fb21effd700 1 -- 172.21.15.97:0/1684353870 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb224057fd0 con 0x7fb22806b360 2023-07-06T03:54:27.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.322+0000 7fb2303ff700 1 -- 172.21.15.97:0/1684353870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb22815e560 con 0x7fb22806b360 2023-07-06T03:54:27.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.982+0000 7fb21effd700 1 -- 172.21.15.97:0/1684353870 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v79) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb22400ce60 con 0x7fb22806b360 2023-07-06T03:54:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 -- 172.21.15.97:0/1684353870 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb20c06ace0 msgr2=0x7fb20c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb20c06ace0 0x7fb20c06d190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb22806b790 tx=0x7fb218008030 comp rx=0 tx=0).stop 2023-07-06T03:54:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 -- 172.21.15.97:0/1684353870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb22806b360 msgr2=0x7fb22815f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb22806b360 0x7fb22815f650 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fb22400e870 tx=0x7fb22400ec30 comp rx=0 tx=0).stop 2023-07-06T03:54:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 -- 172.21.15.97:0/1684353870 shutdown_connections 2023-07-06T03:54:27.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb20c06ace0 0x7fb20c06d190 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:27.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb22815c610 0x7fb2280c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:27.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb22806b360 0x7fb22815f650 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:27.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 --2- 172.21.15.97:0/1684353870 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb2280d2210 0x7fb2280b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:27.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 -- 172.21.15.97:0/1684353870 >> 172.21.15.97:0/1684353870 conn(0x7fb22806a6c0 msgr2=0x7fb2280b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:27.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 -- 172.21.15.97:0/1684353870 shutdown_connections 2023-07-06T03:54:27.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:27.985+0000 7fb2303ff700 1 -- 172.21.15.97:0/1684353870 wait complete. 2023-07-06T03:54:28.006 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T03:54:28.006 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T03:54:28.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.205+0000 7f60531ff700 1 Processor -- start 2023-07-06T03:54:28.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.205+0000 7f60531ff700 1 -- start start 2023-07-06T03:54:28.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.206+0000 7f60531ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f604c06b360 0x7f604c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:28.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.206+0000 7f60531ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 0x7f604c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:28.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.206+0000 7f60531ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604c0cff90 con 0x7f604c15c610 2023-07-06T03:54:28.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.206+0000 7f60531ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604c0d0780 con 0x7f604c06b360 2023-07-06T03:54:28.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.206+0000 7f60531ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c15c9e0 con 0x7f604c0d2210 2023-07-06T03:54:28.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.207+0000 7f605179c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 0x7f604c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:28.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.207+0000 7f605179c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 0x7f604c0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54148/0 (socket says 172.21.15.97:54148) 2023-07-06T03:54:28.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.207+0000 7f605179c700 1 -- 172.21.15.97:0/681074524 learned_addr learned my addr 172.21.15.97:0/681074524 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:28.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.207+0000 7f604bfff700 1 --2- 172.21.15.97:0/681074524 >> v2:172.21.15.146:3300/0 conn(0x7f604c06b360 0x7f604c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:28.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.207+0000 7f604b7fe700 1 -- 172.21.15.97:0/681074524 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482360567 0 0) 0x7f604c0cff90 con 0x7f604c15c610 2023-07-06T03:54:28.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.207+0000 7f605179c700 1 -- 172.21.15.97:0/681074524 >> v2:172.21.15.146:3300/0 conn(0x7f604c06b360 msgr2=0x7f604c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:28.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.207+0000 7f605179c700 1 --2- 172.21.15.97:0/681074524 >> v2:172.21.15.146:3300/0 conn(0x7f604c06b360 0x7f604c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:28.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.207+0000 7f605179c700 1 -- 172.21.15.97:0/681074524 >> v1:172.21.15.146:6789/0 conn(0x7f604c15c610 legacy=0x7f604c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:28.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.207+0000 7f605179c700 1 -- 172.21.15.97:0/681074524 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c0b5650 con 0x7f604c0d2210 2023-07-06T03:54:28.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.208+0000 7f605179c700 1 --2- 172.21.15.97:0/681074524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 0x7f604c0d1310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6044009e50 tx=0x7f604400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b07c20c488be116 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:28.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.208+0000 7f604b7fe700 1 -- 172.21.15.97:0/681074524 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400e070 con 0x7f604c0d2210 2023-07-06T03:54:28.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.208+0000 7f604b7fe700 1 -- 172.21.15.97:0/681074524 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604400b910 con 0x7f604c0d2210 2023-07-06T03:54:28.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.208+0000 7f604b7fe700 1 -- 172.21.15.97:0/681074524 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400bcc0 con 0x7f604c0d2210 2023-07-06T03:54:28.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.209+0000 7f60531ff700 1 -- 172.21.15.97:0/681074524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 msgr2=0x7f604c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.209+0000 7f60531ff700 1 --2- 172.21.15.97:0/681074524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 0x7f604c0d1310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6044009e50 tx=0x7f604400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.209+0000 7f60531ff700 1 -- 172.21.15.97:0/681074524 shutdown_connections 2023-07-06T03:54:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.209+0000 7f60531ff700 1 --2- 172.21.15.97:0/681074524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 0x7f604c0d1310 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.209+0000 7f60531ff700 1 --2- 172.21.15.97:0/681074524 >> v2:172.21.15.146:3300/0 conn(0x7f604c06b360 0x7f604c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.209+0000 7f60531ff700 1 -- 172.21.15.97:0/681074524 >> 172.21.15.97:0/681074524 conn(0x7f604c06a6c0 msgr2=0x7f604c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:28.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.209+0000 7f60531ff700 1 -- 172.21.15.97:0/681074524 shutdown_connections 2023-07-06T03:54:28.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.209+0000 7f60531ff700 1 -- 172.21.15.97:0/681074524 wait complete. 2023-07-06T03:54:28.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.210+0000 7f60531ff700 1 Processor -- start 2023-07-06T03:54:28.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.211+0000 7f60531ff700 1 -- start start 2023-07-06T03:54:28.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.211+0000 7f60531ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f604c06b360 0x7f604c0c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.211+0000 7f60531ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 0x7f604c159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.211+0000 7f60531ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c15c610 0x7f604c1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.211+0000 7f60531ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c15f040 con 0x7f604c15c610 2023-07-06T03:54:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.211+0000 7f60531ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c15e430 con 0x7f604c06b360 2023-07-06T03:54:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.211+0000 7f60531ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c15e710 con 0x7f604c0d2210 2023-07-06T03:54:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f605179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c15c610 0x7f604c1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f605179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c15c610 0x7f604c1596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57278/0 (socket says 172.21.15.97:57278) 2023-07-06T03:54:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f605179c700 1 -- 172.21.15.97:0/3365910860 learned_addr learned my addr 172.21.15.97:0/3365910860 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f604bfff700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 0x7f604c159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f6050f9b700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f604c06b360 0x7f604c0c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f605179c700 1 -- 172.21.15.97:0/3365910860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 msgr2=0x7f604c159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f605179c700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 0x7f604c159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f605179c700 1 -- 172.21.15.97:0/3365910860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f604c06b360 msgr2=0x7f604c0c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f605179c700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f604c06b360 0x7f604c0c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f605179c700 1 -- 172.21.15.97:0/3365910860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6044009b30 con 0x7f604c15c610 2023-07-06T03:54:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f6050f9b700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f604c06b360 0x7f604c0c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.212+0000 7f605179c700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c15c610 0x7f604c1596d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f604c0d2c10 tx=0x7f6044004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.213+0000 7f60497fa700 1 -- 172.21.15.97:0/3365910860 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6044003ea0 con 0x7f604c15c610 2023-07-06T03:54:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.213+0000 7f60497fa700 1 -- 172.21.15.97:0/3365910860 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60440099b0 con 0x7f604c15c610 2023-07-06T03:54:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.213+0000 7f60531ff700 1 -- 172.21.15.97:0/3365910860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c15e9f0 con 0x7f604c15c610 2023-07-06T03:54:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.213+0000 7f60497fa700 1 -- 172.21.15.97:0/3365910860 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604401c680 con 0x7f604c15c610 2023-07-06T03:54:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.213+0000 7f60531ff700 1 -- 172.21.15.97:0/3365910860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c153800 con 0x7f604c15c610 2023-07-06T03:54:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.214+0000 7f60531ff700 1 -- 172.21.15.97:0/3365910860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604c15f1f0 con 0x7f604c15c610 2023-07-06T03:54:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.215+0000 7f60497fa700 1 -- 172.21.15.97:0/3365910860 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f604401c7e0 con 0x7f604c15c610 2023-07-06T03:54:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.217+0000 7f60497fa700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f603006ace0 0x7f603006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:28.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.217+0000 7f60497fa700 1 -- 172.21.15.97:0/3365910860 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f604408ade0 con 0x7f604c15c610 2023-07-06T03:54:28.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.217+0000 7f6050f9b700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f603006ace0 0x7f603006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:28.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.217+0000 7f6050f9b700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f603006ace0 0x7f603006d190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6040000f60 tx=0x7f6040009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:28.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.221+0000 7f60497fa700 1 -- 172.21.15.97:0/3365910860 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f604c15f1f0 con 0x7f604c15c610 2023-07-06T03:54:28.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.460+0000 7f60531ff700 1 -- 172.21.15.97:0/3365910860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f604c04e340 con 0x7f604c15c610 2023-07-06T03:54:28.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.984+0000 7f60497fa700 1 -- 172.21.15.97:0/3365910860 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v80) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6044055df0 con 0x7f604c15c610 2023-07-06T03:54:28.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 -- 172.21.15.97:0/3365910860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f603006ace0 msgr2=0x7f603006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:28.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f603006ace0 0x7f603006d190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6040000f60 tx=0x7f6040009040 comp rx=0 tx=0).stop 2023-07-06T03:54:28.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 -- 172.21.15.97:0/3365910860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c15c610 msgr2=0x7f604c1596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:28.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c15c610 0x7f604c1596d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f604c0d2c10 tx=0x7f6044004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:28.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 -- 172.21.15.97:0/3365910860 shutdown_connections 2023-07-06T03:54:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f603006ace0 0x7f603006d190 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c15c610 0x7f604c1596d0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f604c06b360 0x7f604c0c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 --2- 172.21.15.97:0/3365910860 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f604c0d2210 0x7f604c159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 -- 172.21.15.97:0/3365910860 >> 172.21.15.97:0/3365910860 conn(0x7f604c06a6c0 msgr2=0x7f604c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 -- 172.21.15.97:0/3365910860 shutdown_connections 2023-07-06T03:54:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:28.987+0000 7f60531ff700 1 -- 172.21.15.97:0/3365910860 wait complete. 2023-07-06T03:54:29.009 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T03:54:29.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:29.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.204+0000 7f45117ff700 1 Processor -- start 2023-07-06T03:54:29.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.204+0000 7f45117ff700 1 -- start start 2023-07-06T03:54:29.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.205+0000 7f45117ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f450c0b8410 0x7f450c167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.205+0000 7f45117ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450c0cdae0 con 0x7f450c0cd390 2023-07-06T03:54:29.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.205+0000 7f45117ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450c0b6640 con 0x7f450c15c520 2023-07-06T03:54:29.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.205+0000 7f45117ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0b6920 con 0x7f450c0b8410 2023-07-06T03:54:29.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.206+0000 7f450a7fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f450c0cd390 0x7f450c160890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53578/0 (socket says 172.21.15.97:53578) 2023-07-06T03:54:29.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.206+0000 7f450a7fc700 1 -- 172.21.15.97:0/60632800 learned_addr learned my addr 172.21.15.97:0/60632800 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:29.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.206+0000 7f450b7fe700 1 --2- 172.21.15.97:0/60632800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f450c0b8410 0x7f450c167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.206+0000 7f4509ffb700 1 -- 172.21.15.97:0/60632800 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103124454 0 0) 0x7f450c0cdae0 con 0x7f450c0cd390 2023-07-06T03:54:29.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.206+0000 7f4509ffb700 1 -- 172.21.15.97:0/60632800 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44f40035b0 con 0x7f450c0cd390 2023-07-06T03:54:29.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.206+0000 7f4509ffb700 1 -- 172.21.15.97:0/60632800 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289417084 0 0) 0x7f450c0b6640 con 0x7f450c15c520 2023-07-06T03:54:29.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.206+0000 7f4509ffb700 1 -- 172.21.15.97:0/60632800 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f450c0cdae0 con 0x7f450c15c520 2023-07-06T03:54:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.206+0000 7f450b7fe700 1 -- 172.21.15.97:0/60632800 >> v1:172.21.15.146:6789/0 conn(0x7f450c15c520 legacy=0x7f450c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.206+0000 7f450b7fe700 1 -- 172.21.15.97:0/60632800 >> v1:172.21.15.97:6789/0 conn(0x7f450c0cd390 legacy=0x7f450c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.206+0000 7f450b7fe700 1 -- 172.21.15.97:0/60632800 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c0b6c00 con 0x7f450c0b8410 2023-07-06T03:54:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.207+0000 7f450b7fe700 1 --2- 172.21.15.97:0/60632800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f450c0b8410 0x7f450c167510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4504009dc0 tx=0x7f450400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e10a0cf331f329 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.207+0000 7f4509ffb700 1 -- 172.21.15.97:0/60632800 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450400e070 con 0x7f450c0b8410 2023-07-06T03:54:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.207+0000 7f4509ffb700 1 -- 172.21.15.97:0/60632800 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450400bb20 con 0x7f450c0b8410 2023-07-06T03:54:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.207+0000 7f4509ffb700 1 -- 172.21.15.97:0/60632800 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504012430 con 0x7f450c0b8410 2023-07-06T03:54:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.207+0000 7f45117ff700 1 -- 172.21.15.97:0/60632800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f450c0b8410 msgr2=0x7f450c167510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.207+0000 7f45117ff700 1 --2- 172.21.15.97:0/60632800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f450c0b8410 0x7f450c167510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4504009dc0 tx=0x7f450400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.207+0000 7f45117ff700 1 -- 172.21.15.97:0/60632800 shutdown_connections 2023-07-06T03:54:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.207+0000 7f45117ff700 1 --2- 172.21.15.97:0/60632800 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f450c0b8410 0x7f450c167510 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.207+0000 7f45117ff700 1 -- 172.21.15.97:0/60632800 >> 172.21.15.97:0/60632800 conn(0x7f450c071790 msgr2=0x7f450c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.207+0000 7f45117ff700 1 -- 172.21.15.97:0/60632800 shutdown_connections 2023-07-06T03:54:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.208+0000 7f45117ff700 1 -- 172.21.15.97:0/60632800 wait complete. 2023-07-06T03:54:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.209+0000 7f45117ff700 1 Processor -- start 2023-07-06T03:54:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.209+0000 7f45117ff700 1 -- start start 2023-07-06T03:54:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.210+0000 7f45117ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f450c0b8410 0x7f450c142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.210+0000 7f45117ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f450c0cd390 0x7f450c0d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.210+0000 7f45117ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f450c15c520 0x7f450c15d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.210+0000 7f45117ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0ca550 con 0x7f450c0b8410 2023-07-06T03:54:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.210+0000 7f45117ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0d3bb0 con 0x7f450c0cd390 2023-07-06T03:54:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.210+0000 7f45117ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0c9f40 con 0x7f450c15c520 2023-07-06T03:54:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.210+0000 7f450affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f450c0b8410 0x7f450c142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.210+0000 7f450affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f450c0b8410 0x7f450c142ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57282/0 (socket says 172.21.15.97:57282) 2023-07-06T03:54:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.210+0000 7f450affd700 1 -- 172.21.15.97:0/3487474122 learned_addr learned my addr 172.21.15.97:0/3487474122 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f450b7fe700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f450c15c520 0x7f450c15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f450a7fc700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f450c0cd390 0x7f450c0d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f450affd700 1 -- 172.21.15.97:0/3487474122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f450c15c520 msgr2=0x7f450c15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f450affd700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f450c15c520 0x7f450c15d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f450affd700 1 -- 172.21.15.97:0/3487474122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f450c0cd390 msgr2=0x7f450c0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f450affd700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f450c0cd390 0x7f450c0d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f450affd700 1 -- 172.21.15.97:0/3487474122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4504009a70 con 0x7f450c0b8410 2023-07-06T03:54:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f450affd700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f450c0b8410 0x7f450c142ec0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00c520 tx=0x7f44fc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f44f37fe700 1 -- 172.21.15.97:0/3487474122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0052d0 con 0x7f450c0b8410 2023-07-06T03:54:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f44f37fe700 1 -- 172.21.15.97:0/3487474122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc005430 con 0x7f450c0b8410 2023-07-06T03:54:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f44f37fe700 1 -- 172.21.15.97:0/3487474122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0038c0 con 0x7f450c0b8410 2023-07-06T03:54:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.211+0000 7f45117ff700 1 -- 172.21.15.97:0/3487474122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c0d38d0 con 0x7f450c0b8410 2023-07-06T03:54:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.212+0000 7f45117ff700 1 -- 172.21.15.97:0/3487474122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450c14d180 con 0x7f450c0b8410 2023-07-06T03:54:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.212+0000 7f45117ff700 1 -- 172.21.15.97:0/3487474122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450c04e340 con 0x7f450c0b8410 2023-07-06T03:54:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.214+0000 7f44f37fe700 1 -- 172.21.15.97:0/3487474122 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f44fc003a20 con 0x7f450c0b8410 2023-07-06T03:54:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.214+0000 7f44f37fe700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f44f806ace0 0x7f44f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.214+0000 7f44f37fe700 1 -- 172.21.15.97:0/3487474122 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f44fc0889a0 con 0x7f450c0b8410 2023-07-06T03:54:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.215+0000 7f450a7fc700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f44f806ace0 0x7f44f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.215+0000 7f450a7fc700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f44f806ace0 0x7f44f806d190 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f450c156750 tx=0x7f4500007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.219+0000 7f44f37fe700 1 -- 172.21.15.97:0/3487474122 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f44fc0539b0 con 0x7f450c0b8410 2023-07-06T03:54:29.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.457+0000 7f45117ff700 1 -- 172.21.15.97:0/3487474122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f450c0b4f40 con 0x7f450c0b8410 2023-07-06T03:54:29.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.458+0000 7f44f37fe700 1 -- 172.21.15.97:0/3487474122 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4676 (secure 0 0 0) 0x7f44fc056fd0 con 0x7f450c0b8410 2023-07-06T03:54:29.460 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:29.460 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2796855133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2796855133},{"type":"v1","addr":"172.21.15.97:6835","nonce":2796855133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6851,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1092966074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1092966074},{"type":"v1","addr":"172.21.15.146:6835","nonce":1092966074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8577,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2594138359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2594138359},{"type":"v1","addr":"172.21.15.97:6837","nonce":2594138359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:54:25.441909+0000","modified":"2023-07-06T03:54:28.981530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/582690410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":582690410},{"type":"v1","addr":"172.21.15.146:6834","nonce":582690410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-06T03:54:29.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.460+0000 7f45117ff700 1 -- 172.21.15.97:0/3487474122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f44f806ace0 msgr2=0x7f44f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.460+0000 7f45117ff700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f44f806ace0 0x7f44f806d190 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f450c156750 tx=0x7f4500007dc0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.461+0000 7f45117ff700 1 -- 172.21.15.97:0/3487474122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f450c0b8410 msgr2=0x7f450c142ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.461+0000 7f45117ff700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f450c0b8410 0x7f450c142ec0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00c520 tx=0x7f44fc00c830 comp rx=0 tx=0).stop 2023-07-06T03:54:29.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.461+0000 7f45117ff700 1 -- 172.21.15.97:0/3487474122 shutdown_connections 2023-07-06T03:54:29.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.461+0000 7f45117ff700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f44f806ace0 0x7f44f806d190 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.461+0000 7f45117ff700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f450c15c520 0x7f450c15d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.461+0000 7f45117ff700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f450c0b8410 0x7f450c142ec0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.461+0000 7f45117ff700 1 --2- 172.21.15.97:0/3487474122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f450c0cd390 0x7f450c0d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.461+0000 7f45117ff700 1 -- 172.21.15.97:0/3487474122 >> 172.21.15.97:0/3487474122 conn(0x7f450c071790 msgr2=0x7f450c071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:29.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.461+0000 7f45117ff700 1 -- 172.21.15.97:0/3487474122 shutdown_connections 2023-07-06T03:54:29.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.461+0000 7f45117ff700 1 -- 172.21.15.97:0/3487474122 wait complete. 2023-07-06T03:54:29.465 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 80 2023-07-06T03:54:29.481 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:29.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.674+0000 7fe41b7ff700 1 Processor -- start 2023-07-06T03:54:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.674+0000 7fe41b7ff700 1 -- start start 2023-07-06T03:54:29.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.675+0000 7fe41b7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe41415c520 0x7fe4140cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.675+0000 7fe41b7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4140cdae0 con 0x7fe414160180 2023-07-06T03:54:29.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.675+0000 7fe41b7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4140b6640 con 0x7fe414160d00 2023-07-06T03:54:29.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.675+0000 7fe41b7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140cd8d0 con 0x7fe41415c520 2023-07-06T03:54:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe418d9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe414160180 0x7fe4141605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53586/0 (socket says 172.21.15.97:53586) 2023-07-06T03:54:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe418d9a700 1 -- 172.21.15.97:0/2408929284 learned_addr learned my addr 172.21.15.97:0/2408929284 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe41959b700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe41415c520 0x7fe4140cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401344422 0 0) 0x7fe4140cdae0 con 0x7fe414160180 2023-07-06T03:54:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4000035b0 con 0x7fe414160180 2023-07-06T03:54:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192358561 0 0) 0x7fe4140b6640 con 0x7fe414160d00 2023-07-06T03:54:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4140cdae0 con 0x7fe414160d00 2023-07-06T03:54:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 300217843 0 0) 0x7fe4000035b0 con 0x7fe414160180 2023-07-06T03:54:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe41415c520 msgr2=0x7fe4140cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe41415c520 0x7fe4140cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 >> v1:172.21.15.146:6789/0 conn(0x7fe414160d00 legacy=0x7fe4140b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4140b6920 con 0x7fe414160180 2023-07-06T03:54:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fe404001670 con 0x7fe414160180 2023-07-06T03:54:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 >> v1:172.21.15.97:6789/0 conn(0x7fe414160180 legacy=0x7fe4141605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe400003bc0 0x7fe400003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe400004850 0x7fe400004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe400005200 0x7fe40000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140cd8d0 con 0x7fe400005200 2023-07-06T03:54:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40000bfd0 con 0x7fe400003bc0 2023-07-06T03:54:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40000c190 con 0x7fe400004850 2023-07-06T03:54:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.676+0000 7fe41959b700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe41415c520 0x7fe4140cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:54:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe419d9c700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe400005200 0x7fe40000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe418d9a700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe400004850 0x7fe400004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe41959b700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe400003bc0 0x7fe400003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe419d9c700 1 -- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe400004850 msgr2=0x7fe400004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe419d9c700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe400004850 0x7fe400004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe419d9c700 1 -- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe400003bc0 msgr2=0x7fe400003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe419d9c700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe400003bc0 0x7fe400003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe419d9c700 1 -- 172.21.15.97:0/2408929284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4000035b0 con 0x7fe400005200 2023-07-06T03:54:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe419d9c700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe400005200 0x7fe40000ba10 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00c2f0 tx=0x7fe400005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce80c795975e9606 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:29.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c001f30 con 0x7fe400005200 2023-07-06T03:54:29.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c00ce90 con 0x7fe400005200 2023-07-06T03:54:29.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.677+0000 7fe40bfff700 1 -- 172.21.15.97:0/2408929284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c003c40 con 0x7fe400005200 2023-07-06T03:54:29.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.678+0000 7fe41b7ff700 1 -- 172.21.15.97:0/2408929284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe400005200 msgr2=0x7fe40000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.678+0000 7fe41b7ff700 1 --2- 172.21.15.97:0/2408929284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe400005200 0x7fe40000ba10 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00c2f0 tx=0x7fe400005880 comp rx=0 tx=0).stop 2023-07-06T03:54:29.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.678+0000 7fe419d9c700 1 -- 172.21.15.97:0/2408929284 reap_dead start 2023-07-06T03:54:29.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.678+0000 7fe41b7ff700 1 -- 172.21.15.97:0/2408929284 shutdown_connections 2023-07-06T03:54:29.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.678+0000 7fe41b7ff700 1 -- 172.21.15.97:0/2408929284 >> 172.21.15.97:0/2408929284 conn(0x7fe414071790 msgr2=0x7fe414071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:29.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.678+0000 7fe41b7ff700 1 -- 172.21.15.97:0/2408929284 shutdown_connections 2023-07-06T03:54:29.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.678+0000 7fe41b7ff700 1 -- 172.21.15.97:0/2408929284 wait complete. 2023-07-06T03:54:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.680+0000 7fe41b7ff700 1 Processor -- start 2023-07-06T03:54:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.680+0000 7fe41b7ff700 1 -- start start 2023-07-06T03:54:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.681+0000 7fe41b7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe400005200 0x7fe4140d3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.681+0000 7fe41b7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4141e6f60 0x7fe4140ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.681+0000 7fe41b7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4140c9f30 0x7fe4140ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.681+0000 7fe41b7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140cd1a0 con 0x7fe4140c9f30 2023-07-06T03:54:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.681+0000 7fe41b7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4141e8270 con 0x7fe4141e6f60 2023-07-06T03:54:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.681+0000 7fe41b7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140ca8e0 con 0x7fe400005200 2023-07-06T03:54:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.681+0000 7fe419d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4140c9f30 0x7fe4140ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.681+0000 7fe419d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4140c9f30 0x7fe4140ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57290/0 (socket says 172.21.15.97:57290) 2023-07-06T03:54:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.681+0000 7fe419d9c700 1 -- 172.21.15.97:0/436789994 learned_addr learned my addr 172.21.15.97:0/436789994 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.681+0000 7fe41959b700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe400005200 0x7fe4140d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.681+0000 7fe418d9a700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4141e6f60 0x7fe4140ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe419d9c700 1 -- 172.21.15.97:0/436789994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe400005200 msgr2=0x7fe4140d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe419d9c700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe400005200 0x7fe4140d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe419d9c700 1 -- 172.21.15.97:0/436789994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4141e6f60 msgr2=0x7fe4140ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe419d9c700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4141e6f60 0x7fe4140ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe419d9c700 1 -- 172.21.15.97:0/436789994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4141e73d0 con 0x7fe4140c9f30 2023-07-06T03:54:29.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe41959b700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe400005200 0x7fe4140d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:29.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe419d9c700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4140c9f30 0x7fe4140ca3a0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fe40c004000 tx=0x7fe40c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:29.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe409ffb700 1 -- 172.21.15.97:0/436789994 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00cc90 con 0x7fe4140c9f30 2023-07-06T03:54:29.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe409ffb700 1 -- 172.21.15.97:0/436789994 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c0050d0 con 0x7fe4140c9f30 2023-07-06T03:54:29.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe409ffb700 1 -- 172.21.15.97:0/436789994 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c005480 con 0x7fe4140c9f30 2023-07-06T03:54:29.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe41b7ff700 1 -- 172.21.15.97:0/436789994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4141e7660 con 0x7fe4140c9f30 2023-07-06T03:54:29.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.682+0000 7fe41b7ff700 1 -- 172.21.15.97:0/436789994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4141e7c20 con 0x7fe4140c9f30 2023-07-06T03:54:29.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.683+0000 7fe41b7ff700 1 -- 172.21.15.97:0/436789994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41404e340 con 0x7fe4140c9f30 2023-07-06T03:54:29.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.684+0000 7fe409ffb700 1 -- 172.21.15.97:0/436789994 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe40c019070 con 0x7fe4140c9f30 2023-07-06T03:54:29.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.685+0000 7fe409ffb700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3f806ace0 0x7fe3f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:29.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.685+0000 7fe409ffb700 1 -- 172.21.15.97:0/436789994 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe40c08a2a0 con 0x7fe4140c9f30 2023-07-06T03:54:29.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.685+0000 7fe41959b700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3f806ace0 0x7fe3f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:29.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.686+0000 7fe41959b700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3f806ace0 0x7fe3f806d190 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe4140b4c70 tx=0x7fe410008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:29.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.689+0000 7fe409ffb700 1 -- 172.21.15.97:0/436789994 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe40c055340 con 0x7fe4140c9f30 2023-07-06T03:54:29.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.884+0000 7fe41b7ff700 1 -- 172.21.15.97:0/436789994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4140b4f40 con 0x7fe4140c9f30 2023-07-06T03:54:29.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.885+0000 7fe409ffb700 1 -- 172.21.15.97:0/436789994 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18391 (secure 0 0 0) 0x7fe40c0011c0 con 0x7fe4140c9f30 2023-07-06T03:54:29.887 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:29.888 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":112,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:54:26.969421+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:54:23.626811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-06T03:54:24.421787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 -- 172.21.15.97:0/436789994 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3f806ace0 msgr2=0x7fe3f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3f806ace0 0x7fe3f806d190 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe4140b4c70 tx=0x7fe410008040 comp rx=0 tx=0).stop 2023-07-06T03:54:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 -- 172.21.15.97:0/436789994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4140c9f30 msgr2=0x7fe4140ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4140c9f30 0x7fe4140ca3a0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fe40c004000 tx=0x7fe40c001650 comp rx=0 tx=0).stop 2023-07-06T03:54:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 -- 172.21.15.97:0/436789994 shutdown_connections 2023-07-06T03:54:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3f806ace0 0x7fe3f806d190 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4140c9f30 0x7fe4140ca3a0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe400005200 0x7fe4140d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 --2- 172.21.15.97:0/436789994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4141e6f60 0x7fe4140ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:29.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 -- 172.21.15.97:0/436789994 >> 172.21.15.97:0/436789994 conn(0x7fe414071790 msgr2=0x7fe414157930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:29.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 -- 172.21.15.97:0/436789994 shutdown_connections 2023-07-06T03:54:29.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:29.888+0000 7fe41b7ff700 1 -- 172.21.15.97:0/436789994 wait complete. 2023-07-06T03:54:29.908 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T03:54:29.908 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:54:30.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.102+0000 7fdb6c7ff700 1 Processor -- start 2023-07-06T03:54:30.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.102+0000 7fdb6c7ff700 1 -- start start 2023-07-06T03:54:30.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.103+0000 7fdb6c7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb6415c520 0x7fdb640cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.103+0000 7fdb6c7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb640cdae0 con 0x7fdb64160d00 2023-07-06T03:54:30.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.103+0000 7fdb6c7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb640b6640 con 0x7fdb64160180 2023-07-06T03:54:30.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.103+0000 7fdb6c7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb640cd8d0 con 0x7fdb6415c520 2023-07-06T03:54:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb6ad9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdb64160d00 0x7fdb640b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37840/0 (socket says 172.21.15.97:37840) 2023-07-06T03:54:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb6ad9c700 1 -- 172.21.15.97:0/555722022 learned_addr learned my addr 172.21.15.97:0/555722022 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714972566 0 0) 0x7fdb640cdae0 con 0x7fdb64160d00 2023-07-06T03:54:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb6a59b700 1 --2- 172.21.15.97:0/555722022 >> v2:172.21.15.146:3300/0 conn(0x7fdb6415c520 0x7fdb640cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb500035b0 con 0x7fdb64160d00 2023-07-06T03:54:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2108055266 0 0) 0x7fdb500035b0 con 0x7fdb64160d00 2023-07-06T03:54:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 >> v2:172.21.15.146:3300/0 conn(0x7fdb6415c520 msgr2=0x7fdb640cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb69599700 1 --2- 172.21.15.97:0/555722022 >> v2:172.21.15.146:3300/0 conn(0x7fdb6415c520 0x7fdb640cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 >> v1:172.21.15.146:6789/0 conn(0x7fdb64160180 legacy=0x7fdb641605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb640b6920 con 0x7fdb64160d00 2023-07-06T03:54:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fdb5c0019f0 con 0x7fdb64160d00 2023-07-06T03:54:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 >> v1:172.21.15.97:6789/0 conn(0x7fdb64160d00 legacy=0x7fdb640b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb69599700 1 --2- 172.21.15.97:0/555722022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb50003a00 0x7fdb50003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.104+0000 7fdb69599700 1 --2- 172.21.15.97:0/555722022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb50004850 0x7fdb50004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69599700 1 --2- 172.21.15.97:0/555722022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb50005200 0x7fdb5000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb640cd8d0 con 0x7fdb50004850 2023-07-06T03:54:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5000c080 con 0x7fdb50005200 2023-07-06T03:54:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5000c240 con 0x7fdb50003a00 2023-07-06T03:54:30.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69d9a700 1 --2- 172.21.15.97:0/555722022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb50004850 0x7fdb50004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb6a59b700 1 --2- 172.21.15.97:0/555722022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb50003a00 0x7fdb50003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb6ad9c700 1 --2- 172.21.15.97:0/555722022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb50005200 0x7fdb5000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69d9a700 1 -- 172.21.15.97:0/555722022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb50003a00 msgr2=0x7fdb50003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69d9a700 1 --2- 172.21.15.97:0/555722022 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb50003a00 0x7fdb50003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69d9a700 1 -- 172.21.15.97:0/555722022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb50005200 msgr2=0x7fdb5000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69d9a700 1 --2- 172.21.15.97:0/555722022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb50005200 0x7fdb5000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69d9a700 1 -- 172.21.15.97:0/555722022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb500035b0 con 0x7fdb50004850 2023-07-06T03:54:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb6ad9c700 1 -- 172.21.15.97:0/555722022 reap_dead start 2023-07-06T03:54:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69d9a700 1 --2- 172.21.15.97:0/555722022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb50004850 0x7fdb50004cc0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fdb540017c0 tx=0x7fdb50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8591c0623516dc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.105+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb540052d0 con 0x7fdb50004850 2023-07-06T03:54:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.106+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb54005450 con 0x7fdb50004850 2023-07-06T03:54:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.106+0000 7fdb69599700 1 -- 172.21.15.97:0/555722022 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb540038c0 con 0x7fdb50004850 2023-07-06T03:54:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.106+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/555722022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb50004850 msgr2=0x7fdb50004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.106+0000 7fdb6c7ff700 1 --2- 172.21.15.97:0/555722022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb50004850 0x7fdb50004cc0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fdb540017c0 tx=0x7fdb50005880 comp rx=0 tx=0).stop 2023-07-06T03:54:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.106+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/555722022 shutdown_connections 2023-07-06T03:54:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.106+0000 7fdb6c7ff700 1 --2- 172.21.15.97:0/555722022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb50004850 0x7fdb50004cc0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.106+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/555722022 >> 172.21.15.97:0/555722022 conn(0x7fdb64071790 msgr2=0x7fdb64071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.106+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/555722022 shutdown_connections 2023-07-06T03:54:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.106+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/555722022 wait complete. 2023-07-06T03:54:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.108+0000 7fdb6c7ff700 1 Processor -- start 2023-07-06T03:54:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.108+0000 7fdb6c7ff700 1 -- start start 2023-07-06T03:54:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.109+0000 7fdb6c7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb50004850 0x7fdb640c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.109+0000 7fdb6c7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb6415b640 0x7fdb6415db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.109+0000 7fdb6c7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb6415e0c0 0x7fdb6415e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.109+0000 7fdb6c7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb640c6870 con 0x7fdb6415b640 2023-07-06T03:54:30.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.109+0000 7fdb6c7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb640cb270 con 0x7fdb50004850 2023-07-06T03:54:30.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.109+0000 7fdb6c7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb640d34f0 con 0x7fdb6415e0c0 2023-07-06T03:54:30.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.109+0000 7fdb69d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb6415b640 0x7fdb6415db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.109+0000 7fdb69d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb6415b640 0x7fdb6415db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34474/0 (socket says 172.21.15.97:34474) 2023-07-06T03:54:30.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.109+0000 7fdb69d9a700 1 -- 172.21.15.97:0/1564613686 learned_addr learned my addr 172.21.15.97:0/1564613686 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.109+0000 7fdb6ad9c700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb6415e0c0 0x7fdb6415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.109+0000 7fdb6a59b700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb50004850 0x7fdb640c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb69d9a700 1 -- 172.21.15.97:0/1564613686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb6415e0c0 msgr2=0x7fdb6415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb69d9a700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb6415e0c0 0x7fdb6415e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb69d9a700 1 -- 172.21.15.97:0/1564613686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb50004850 msgr2=0x7fdb640c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb69d9a700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb50004850 0x7fdb640c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb69d9a700 1 -- 172.21.15.97:0/1564613686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb640d37d0 con 0x7fdb6415b640 2023-07-06T03:54:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb6a59b700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb50004850 0x7fdb640c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb69d9a700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb6415b640 0x7fdb6415db80 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c050 tx=0x7fdb54001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb5affd700 1 -- 172.21.15.97:0/1564613686 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54001290 con 0x7fdb6415b640 2023-07-06T03:54:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb5affd700 1 -- 172.21.15.97:0/1564613686 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb54007040 con 0x7fdb6415b640 2023-07-06T03:54:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/1564613686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb640caf00 con 0x7fdb6415b640 2023-07-06T03:54:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb5affd700 1 -- 172.21.15.97:0/1564613686 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb540073d0 con 0x7fdb6415b640 2023-07-06T03:54:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.110+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/1564613686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb640d3ab0 con 0x7fdb6415b640 2023-07-06T03:54:30.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.111+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/1564613686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb640cb3e0 con 0x7fdb6415b640 2023-07-06T03:54:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.112+0000 7fdb5affd700 1 -- 172.21.15.97:0/1564613686 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb54007530 con 0x7fdb6415b640 2023-07-06T03:54:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.113+0000 7fdb5affd700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb4806ace0 0x7fdb4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.113+0000 7fdb5affd700 1 -- 172.21.15.97:0/1564613686 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdb54089f60 con 0x7fdb6415b640 2023-07-06T03:54:30.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.113+0000 7fdb6a59b700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb4806ace0 0x7fdb4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.114+0000 7fdb6a59b700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb4806ace0 0x7fdb4806d190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fdb64063c30 tx=0x7fdb60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:30.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.118+0000 7fdb5affd700 1 -- 172.21.15.97:0/1564613686 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdb540551e0 con 0x7fdb6415b640 2023-07-06T03:54:30.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.309+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/1564613686 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb640b6ee0 con 0x7fdb4806ace0 2023-07-06T03:54:30.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.318+0000 7fdb5affd700 1 -- 172.21.15.97:0/1564613686 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311787 (secure 0 0 0) 0x7fdb600127c0 con 0x7fdb4806ace0 2023-07-06T03:54:30.323 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:30.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.322+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/1564613686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb4806ace0 msgr2=0x7fdb4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.322+0000 7fdb6c7ff700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb4806ace0 0x7fdb4806d190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fdb64063c30 tx=0x7fdb60008040 comp rx=0 tx=0).stop 2023-07-06T03:54:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.322+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/1564613686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb6415b640 msgr2=0x7fdb6415db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.322+0000 7fdb6c7ff700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb6415b640 0x7fdb6415db80 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c050 tx=0x7fdb54001120 comp rx=0 tx=0).stop 2023-07-06T03:54:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.323+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/1564613686 shutdown_connections 2023-07-06T03:54:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.323+0000 7fdb6c7ff700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdb4806ace0 0x7fdb4806d190 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.323+0000 7fdb6c7ff700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdb6415e0c0 0x7fdb6415e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.323+0000 7fdb6c7ff700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb50004850 0x7fdb640c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.323+0000 7fdb6c7ff700 1 --2- 172.21.15.97:0/1564613686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb6415b640 0x7fdb6415db80 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.323+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/1564613686 >> 172.21.15.97:0/1564613686 conn(0x7fdb64071790 msgr2=0x7fdb640b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.323+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/1564613686 shutdown_connections 2023-07-06T03:54:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.323+0000 7fdb6c7ff700 1 -- 172.21.15.97:0/1564613686 wait complete. 2023-07-06T03:54:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:54:30.343 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2023-07-06T03:54:29.004322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284804,"kb_used_data":11844,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754689916,"statfs":{"total":773094113280,"available":772802473984,"internally_reserved":0,"allocated":12128256,"data_stored":6185621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.065409"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.989525+0000","last_change":"2023-07-06T03:54:25.982519+0000","last_active":"2023-07-06T03:54:25.989525+0000","last_peered":"2023-07-06T03:54:25.989525+0000","last_clean":"2023-07-06T03:54:25.989525+0000","last_became_active":"2023-07-06T03:54:25.982111+0000","last_became_peered":"2023-07-06T03:54:25.982111+0000","last_unstale":"2023-07-06T03:54:25.989525+0000","last_undegraded":"2023-07-06T03:54:25.989525+0000","last_fullsized":"2023-07-06T03:54:25.989525+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:58.078557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.996598+0000","last_change":"2023-07-06T03:54:25.991355+0000","last_active":"2023-07-06T03:54:25.996598+0000","last_peered":"2023-07-06T03:54:25.996598+0000","last_clean":"2023-07-06T03:54:25.996598+0000","last_became_active":"2023-07-06T03:54:25.991021+0000","last_became_peered":"2023-07-06T03:54:25.991021+0000","last_unstale":"2023-07-06T03:54:25.996598+0000","last_undegraded":"2023-07-06T03:54:25.996598+0000","last_fullsized":"2023-07-06T03:54:25.996598+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:20:54.856278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998120+0000","last_change":"2023-07-06T03:54:25.997569+0000","last_active":"2023-07-06T03:54:25.998120+0000","last_peered":"2023-07-06T03:54:25.998120+0000","last_clean":"2023-07-06T03:54:25.998120+0000","last_became_active":"2023-07-06T03:54:25.997212+0000","last_became_peered":"2023-07-06T03:54:25.997212+0000","last_unstale":"2023-07-06T03:54:25.998120+0000","last_undegraded":"2023-07-06T03:54:25.998120+0000","last_fullsized":"2023-07-06T03:54:25.998120+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:47:56.013868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997116+0000","last_change":"2023-07-06T03:54:25.993309+0000","last_active":"2023-07-06T03:54:25.997116+0000","last_peered":"2023-07-06T03:54:25.997116+0000","last_clean":"2023-07-06T03:54:25.997116+0000","last_became_active":"2023-07-06T03:54:25.992730+0000","last_became_peered":"2023-07-06T03:54:25.992730+0000","last_unstale":"2023-07-06T03:54:25.997116+0000","last_undegraded":"2023-07-06T03:54:25.997116+0000","last_fullsized":"2023-07-06T03:54:25.997116+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:48:36.996242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.995407+0000","last_change":"2023-07-06T03:54:25.991695+0000","last_active":"2023-07-06T03:54:25.995407+0000","last_peered":"2023-07-06T03:54:25.995407+0000","last_clean":"2023-07-06T03:54:25.995407+0000","last_became_active":"2023-07-06T03:54:25.991226+0000","last_became_peered":"2023-07-06T03:54:25.991226+0000","last_unstale":"2023-07-06T03:54:25.995407+0000","last_undegraded":"2023-07-06T03:54:25.995407+0000","last_fullsized":"2023-07-06T03:54:25.995407+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:21:09.135716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.991221+0000","last_change":"2023-07-06T03:54:25.987083+0000","last_active":"2023-07-06T03:54:25.991221+0000","last_peered":"2023-07-06T03:54:25.991221+0000","last_clean":"2023-07-06T03:54:25.991221+0000","last_became_active":"2023-07-06T03:54:25.986814+0000","last_became_peered":"2023-07-06T03:54:25.986814+0000","last_unstale":"2023-07-06T03:54:25.991221+0000","last_undegraded":"2023-07-06T03:54:25.991221+0000","last_fullsized":"2023-07-06T03:54:25.991221+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:17:14.901305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998632+0000","last_change":"2023-07-06T03:54:25.995189+0000","last_active":"2023-07-06T03:54:25.998632+0000","last_peered":"2023-07-06T03:54:25.998632+0000","last_clean":"2023-07-06T03:54:25.998632+0000","last_became_active":"2023-07-06T03:54:25.994873+0000","last_became_peered":"2023-07-06T03:54:25.994873+0000","last_unstale":"2023-07-06T03:54:25.998632+0000","last_undegraded":"2023-07-06T03:54:25.998632+0000","last_fullsized":"2023-07-06T03:54:25.998632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:40:17.141868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997758+0000","last_change":"2023-07-06T03:54:25.990602+0000","last_active":"2023-07-06T03:54:25.997758+0000","last_peered":"2023-07-06T03:54:25.997758+0000","last_clean":"2023-07-06T03:54:25.997758+0000","last_became_active":"2023-07-06T03:54:25.990333+0000","last_became_peered":"2023-07-06T03:54:25.990333+0000","last_unstale":"2023-07-06T03:54:25.997758+0000","last_undegraded":"2023-07-06T03:54:25.997758+0000","last_fullsized":"2023-07-06T03:54:25.997758+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:14:54.749612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.999536+0000","last_change":"2023-07-06T03:54:25.998902+0000","last_active":"2023-07-06T03:54:25.999536+0000","last_peered":"2023-07-06T03:54:25.999536+0000","last_clean":"2023-07-06T03:54:25.999536+0000","last_became_active":"2023-07-06T03:54:25.998620+0000","last_became_peered":"2023-07-06T03:54:25.998620+0000","last_unstale":"2023-07-06T03:54:25.999536+0000","last_undegraded":"2023-07-06T03:54:25.999536+0000","last_fullsized":"2023-07-06T03:54:25.999536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:51:40.122894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.996615+0000","last_change":"2023-07-06T03:54:25.994841+0000","last_active":"2023-07-06T03:54:25.996615+0000","last_peered":"2023-07-06T03:54:25.996615+0000","last_clean":"2023-07-06T03:54:25.996615+0000","last_became_active":"2023-07-06T03:54:25.994550+0000","last_became_peered":"2023-07-06T03:54:25.994550+0000","last_unstale":"2023-07-06T03:54:25.996615+0000","last_undegraded":"2023-07-06T03:54:25.996615+0000","last_fullsized":"2023-07-06T03:54:25.996615+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:47:56.013868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.977499+0000","last_change":"2023-07-06T03:54:25.970538+0000","last_active":"2023-07-06T03:54:25.977499+0000","last_peered":"2023-07-06T03:54:25.977499+0000","last_clean":"2023-07-06T03:54:25.977499+0000","last_became_active":"2023-07-06T03:54:25.970212+0000","last_became_peered":"2023-07-06T03:54:25.970212+0000","last_unstale":"2023-07-06T03:54:25.977499+0000","last_undegraded":"2023-07-06T03:54:25.977499+0000","last_fullsized":"2023-07-06T03:54:25.977499+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:36:51.075925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998043+0000","last_change":"2023-07-06T03:54:25.993224+0000","last_active":"2023-07-06T03:54:25.998043+0000","last_peered":"2023-07-06T03:54:25.998043+0000","last_clean":"2023-07-06T03:54:25.998043+0000","last_became_active":"2023-07-06T03:54:25.992950+0000","last_became_peered":"2023-07-06T03:54:25.992950+0000","last_unstale":"2023-07-06T03:54:25.998043+0000","last_undegraded":"2023-07-06T03:54:25.998043+0000","last_fullsized":"2023-07-06T03:54:25.998043+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:20:52.627131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987667+0000","last_change":"2023-07-06T03:54:25.984700+0000","last_active":"2023-07-06T03:54:25.987667+0000","last_peered":"2023-07-06T03:54:25.987667+0000","last_clean":"2023-07-06T03:54:25.987667+0000","last_became_active":"2023-07-06T03:54:25.984210+0000","last_became_peered":"2023-07-06T03:54:25.984210+0000","last_unstale":"2023-07-06T03:54:25.987667+0000","last_undegraded":"2023-07-06T03:54:25.987667+0000","last_fullsized":"2023-07-06T03:54:25.987667+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:14.865749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998083+0000","last_change":"2023-07-06T03:54:25.993970+0000","last_active":"2023-07-06T03:54:25.998083+0000","last_peered":"2023-07-06T03:54:25.998083+0000","last_clean":"2023-07-06T03:54:25.998083+0000","last_became_active":"2023-07-06T03:54:25.993658+0000","last_became_peered":"2023-07-06T03:54:25.993658+0000","last_unstale":"2023-07-06T03:54:25.998083+0000","last_undegraded":"2023-07-06T03:54:25.998083+0000","last_fullsized":"2023-07-06T03:54:25.998083+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:17:47.631783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.996313+0000","last_change":"2023-07-06T03:54:25.990688+0000","last_active":"2023-07-06T03:54:25.996313+0000","last_peered":"2023-07-06T03:54:25.996313+0000","last_clean":"2023-07-06T03:54:25.996313+0000","last_became_active":"2023-07-06T03:54:25.990416+0000","last_became_peered":"2023-07-06T03:54:25.990416+0000","last_unstale":"2023-07-06T03:54:25.996313+0000","last_undegraded":"2023-07-06T03:54:25.996313+0000","last_fullsized":"2023-07-06T03:54:25.996313+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:16:50.354940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998108+0000","last_change":"2023-07-06T03:54:25.993590+0000","last_active":"2023-07-06T03:54:25.998108+0000","last_peered":"2023-07-06T03:54:25.998108+0000","last_clean":"2023-07-06T03:54:25.998108+0000","last_became_active":"2023-07-06T03:54:25.993261+0000","last_became_peered":"2023-07-06T03:54:25.993261+0000","last_unstale":"2023-07-06T03:54:25.998108+0000","last_undegraded":"2023-07-06T03:54:25.998108+0000","last_fullsized":"2023-07-06T03:54:25.998108+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:58.823441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:26.000841+0000","last_change":"2023-07-06T03:54:25.998097+0000","last_active":"2023-07-06T03:54:26.000841+0000","last_peered":"2023-07-06T03:54:26.000841+0000","last_clean":"2023-07-06T03:54:26.000841+0000","last_became_active":"2023-07-06T03:54:25.997692+0000","last_became_peered":"2023-07-06T03:54:25.997692+0000","last_unstale":"2023-07-06T03:54:26.000841+0000","last_undegraded":"2023-07-06T03:54:26.000841+0000","last_fullsized":"2023-07-06T03:54:26.000841+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:27:23.778751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998163+0000","last_change":"2023-07-06T03:54:25.995292+0000","last_active":"2023-07-06T03:54:25.998163+0000","last_peered":"2023-07-06T03:54:25.998163+0000","last_clean":"2023-07-06T03:54:25.998163+0000","last_became_active":"2023-07-06T03:54:25.994786+0000","last_became_peered":"2023-07-06T03:54:25.994786+0000","last_unstale":"2023-07-06T03:54:25.998163+0000","last_undegraded":"2023-07-06T03:54:25.998163+0000","last_fullsized":"2023-07-06T03:54:25.998163+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:51:34.065573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997697+0000","last_change":"2023-07-06T03:54:25.996732+0000","last_active":"2023-07-06T03:54:25.997697+0000","last_peered":"2023-07-06T03:54:25.997697+0000","last_clean":"2023-07-06T03:54:25.997697+0000","last_became_active":"2023-07-06T03:54:25.996395+0000","last_became_peered":"2023-07-06T03:54:25.996395+0000","last_unstale":"2023-07-06T03:54:25.997697+0000","last_undegraded":"2023-07-06T03:54:25.997697+0000","last_fullsized":"2023-07-06T03:54:25.997697+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:46:50.022959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997594+0000","last_change":"2023-07-06T03:54:25.995336+0000","last_active":"2023-07-06T03:54:25.997594+0000","last_peered":"2023-07-06T03:54:25.997594+0000","last_clean":"2023-07-06T03:54:25.997594+0000","last_became_active":"2023-07-06T03:54:25.994096+0000","last_became_peered":"2023-07-06T03:54:25.994096+0000","last_unstale":"2023-07-06T03:54:25.997594+0000","last_undegraded":"2023-07-06T03:54:25.997594+0000","last_fullsized":"2023-07-06T03:54:25.997594+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:31:38.605779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987248+0000","last_change":"2023-07-06T03:54:25.985809+0000","last_active":"2023-07-06T03:54:25.987248+0000","last_peered":"2023-07-06T03:54:25.987248+0000","last_clean":"2023-07-06T03:54:25.987248+0000","last_became_active":"2023-07-06T03:54:25.985546+0000","last_became_peered":"2023-07-06T03:54:25.985546+0000","last_unstale":"2023-07-06T03:54:25.987248+0000","last_undegraded":"2023-07-06T03:54:25.987248+0000","last_fullsized":"2023-07-06T03:54:25.987248+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:50:53.351454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998749+0000","last_change":"2023-07-06T03:54:25.995805+0000","last_active":"2023-07-06T03:54:25.998749+0000","last_peered":"2023-07-06T03:54:25.998749+0000","last_clean":"2023-07-06T03:54:25.998749+0000","last_became_active":"2023-07-06T03:54:25.995323+0000","last_became_peered":"2023-07-06T03:54:25.995323+0000","last_unstale":"2023-07-06T03:54:25.998749+0000","last_undegraded":"2023-07-06T03:54:25.998749+0000","last_fullsized":"2023-07-06T03:54:25.998749+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.947240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.996294+0000","last_change":"2023-07-06T03:54:25.990495+0000","last_active":"2023-07-06T03:54:25.996294+0000","last_peered":"2023-07-06T03:54:25.996294+0000","last_clean":"2023-07-06T03:54:25.996294+0000","last_became_active":"2023-07-06T03:54:25.990215+0000","last_became_peered":"2023-07-06T03:54:25.990215+0000","last_unstale":"2023-07-06T03:54:25.996294+0000","last_undegraded":"2023-07-06T03:54:25.996294+0000","last_fullsized":"2023-07-06T03:54:25.996294+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:06:42.446772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998469+0000","last_change":"2023-07-06T03:54:25.995055+0000","last_active":"2023-07-06T03:54:25.998469+0000","last_peered":"2023-07-06T03:54:25.998469+0000","last_clean":"2023-07-06T03:54:25.998469+0000","last_became_active":"2023-07-06T03:54:25.994787+0000","last_became_peered":"2023-07-06T03:54:25.994787+0000","last_unstale":"2023-07-06T03:54:25.998469+0000","last_undegraded":"2023-07-06T03:54:25.998469+0000","last_fullsized":"2023-07-06T03:54:25.998469+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:37:46.155723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.989504+0000","last_change":"2023-07-06T03:54:25.982354+0000","last_active":"2023-07-06T03:54:25.989504+0000","last_peered":"2023-07-06T03:54:25.989504+0000","last_clean":"2023-07-06T03:54:25.989504+0000","last_became_active":"2023-07-06T03:54:25.981783+0000","last_became_peered":"2023-07-06T03:54:25.981783+0000","last_unstale":"2023-07-06T03:54:25.989504+0000","last_undegraded":"2023-07-06T03:54:25.989504+0000","last_fullsized":"2023-07-06T03:54:25.989504+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:19:51.476763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.999156+0000","last_change":"2023-07-06T03:54:25.992886+0000","last_active":"2023-07-06T03:54:25.999156+0000","last_peered":"2023-07-06T03:54:25.999156+0000","last_clean":"2023-07-06T03:54:25.999156+0000","last_became_active":"2023-07-06T03:54:25.992612+0000","last_became_peered":"2023-07-06T03:54:25.992612+0000","last_unstale":"2023-07-06T03:54:25.999156+0000","last_undegraded":"2023-07-06T03:54:25.999156+0000","last_fullsized":"2023-07-06T03:54:25.999156+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:49:09.971185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.986756+0000","last_change":"2023-07-06T03:54:25.983575+0000","last_active":"2023-07-06T03:54:25.986756+0000","last_peered":"2023-07-06T03:54:25.986756+0000","last_clean":"2023-07-06T03:54:25.986756+0000","last_became_active":"2023-07-06T03:54:25.983322+0000","last_became_peered":"2023-07-06T03:54:25.983322+0000","last_unstale":"2023-07-06T03:54:25.986756+0000","last_undegraded":"2023-07-06T03:54:25.986756+0000","last_fullsized":"2023-07-06T03:54:25.986756+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:38:49.789114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997946+0000","last_change":"2023-07-06T03:54:25.992619+0000","last_active":"2023-07-06T03:54:25.997946+0000","last_peered":"2023-07-06T03:54:25.997946+0000","last_clean":"2023-07-06T03:54:25.997946+0000","last_became_active":"2023-07-06T03:54:25.992313+0000","last_became_peered":"2023-07-06T03:54:25.992313+0000","last_unstale":"2023-07-06T03:54:25.997946+0000","last_undegraded":"2023-07-06T03:54:25.997946+0000","last_fullsized":"2023-07-06T03:54:25.997946+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:56:14.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.990735+0000","last_change":"2023-07-06T03:54:25.989681+0000","last_active":"2023-07-06T03:54:25.990735+0000","last_peered":"2023-07-06T03:54:25.990735+0000","last_clean":"2023-07-06T03:54:25.990735+0000","last_became_active":"2023-07-06T03:54:25.989390+0000","last_became_peered":"2023-07-06T03:54:25.989390+0000","last_unstale":"2023-07-06T03:54:25.990735+0000","last_undegraded":"2023-07-06T03:54:25.990735+0000","last_fullsized":"2023-07-06T03:54:25.990735+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:11:09.192506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.974115+0000","last_change":"2023-07-06T03:54:25.970734+0000","last_active":"2023-07-06T03:54:25.974115+0000","last_peered":"2023-07-06T03:54:25.974115+0000","last_clean":"2023-07-06T03:54:25.974115+0000","last_became_active":"2023-07-06T03:54:25.970228+0000","last_became_peered":"2023-07-06T03:54:25.970228+0000","last_unstale":"2023-07-06T03:54:25.974115+0000","last_undegraded":"2023-07-06T03:54:25.974115+0000","last_fullsized":"2023-07-06T03:54:25.974115+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:02:01.652970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998086+0000","last_change":"2023-07-06T03:54:25.994954+0000","last_active":"2023-07-06T03:54:25.998086+0000","last_peered":"2023-07-06T03:54:25.998086+0000","last_clean":"2023-07-06T03:54:25.998086+0000","last_became_active":"2023-07-06T03:54:25.994643+0000","last_became_peered":"2023-07-06T03:54:25.994643+0000","last_unstale":"2023-07-06T03:54:25.998086+0000","last_undegraded":"2023-07-06T03:54:25.998086+0000","last_fullsized":"2023-07-06T03:54:25.998086+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:35:37.623028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.980217+0000","last_change":"2023-07-06T03:54:25.970822+0000","last_active":"2023-07-06T03:54:25.980217+0000","last_peered":"2023-07-06T03:54:25.980217+0000","last_clean":"2023-07-06T03:54:25.980217+0000","last_became_active":"2023-07-06T03:54:25.970472+0000","last_became_peered":"2023-07-06T03:54:25.970472+0000","last_unstale":"2023-07-06T03:54:25.980217+0000","last_undegraded":"2023-07-06T03:54:25.980217+0000","last_fullsized":"2023-07-06T03:54:25.980217+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:49:04.425510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.985726+0000","last_change":"2023-07-06T03:54:25.977347+0000","last_active":"2023-07-06T03:54:25.985726+0000","last_peered":"2023-07-06T03:54:25.985726+0000","last_clean":"2023-07-06T03:54:25.985726+0000","last_became_active":"2023-07-06T03:54:25.977018+0000","last_became_peered":"2023-07-06T03:54:25.977018+0000","last_unstale":"2023-07-06T03:54:25.985726+0000","last_undegraded":"2023-07-06T03:54:25.985726+0000","last_fullsized":"2023-07-06T03:54:25.985726+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:25:02.352516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"111'4","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.994355+0000","last_change":"2023-07-06T03:54:24.960709+0000","last_active":"2023-07-06T03:54:25.994355+0000","last_peered":"2023-07-06T03:54:25.994355+0000","last_clean":"2023-07-06T03:54:25.994355+0000","last_became_active":"2023-07-06T03:54:24.960417+0000","last_became_peered":"2023-07-06T03:54:24.960417+0000","last_unstale":"2023-07-06T03:54:25.994355+0000","last_undegraded":"2023-07-06T03:54:25.994355+0000","last_fullsized":"2023-07-06T03:54:25.994355+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:29:30.498239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987620+0000","last_change":"2023-07-06T03:54:25.983631+0000","last_active":"2023-07-06T03:54:25.987620+0000","last_peered":"2023-07-06T03:54:25.987620+0000","last_clean":"2023-07-06T03:54:25.987620+0000","last_became_active":"2023-07-06T03:54:25.983294+0000","last_became_peered":"2023-07-06T03:54:25.983294+0000","last_unstale":"2023-07-06T03:54:25.987620+0000","last_undegraded":"2023-07-06T03:54:25.987620+0000","last_fullsized":"2023-07-06T03:54:25.987620+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:56:05.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.995954+0000","last_change":"2023-07-06T03:54:24.966719+0000","last_active":"2023-07-06T03:54:25.995954+0000","last_peered":"2023-07-06T03:54:25.995954+0000","last_clean":"2023-07-06T03:54:25.995954+0000","last_became_active":"2023-07-06T03:54:24.966294+0000","last_became_peered":"2023-07-06T03:54:24.966294+0000","last_unstale":"2023-07-06T03:54:25.995954+0000","last_undegraded":"2023-07-06T03:54:25.995954+0000","last_fullsized":"2023-07-06T03:54:25.995954+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:44:34.378195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987162+0000","last_change":"2023-07-06T03:54:25.981442+0000","last_active":"2023-07-06T03:54:25.987162+0000","last_peered":"2023-07-06T03:54:25.987162+0000","last_clean":"2023-07-06T03:54:25.987162+0000","last_became_active":"2023-07-06T03:54:25.981170+0000","last_became_peered":"2023-07-06T03:54:25.981170+0000","last_unstale":"2023-07-06T03:54:25.987162+0000","last_undegraded":"2023-07-06T03:54:25.987162+0000","last_fullsized":"2023-07-06T03:54:25.987162+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:14:50.726445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.981603+0000","last_change":"2023-07-06T03:54:24.961735+0000","last_active":"2023-07-06T03:54:25.981603+0000","last_peered":"2023-07-06T03:54:25.981603+0000","last_clean":"2023-07-06T03:54:25.981603+0000","last_became_active":"2023-07-06T03:54:24.961470+0000","last_became_peered":"2023-07-06T03:54:24.961470+0000","last_unstale":"2023-07-06T03:54:25.981603+0000","last_undegraded":"2023-07-06T03:54:25.981603+0000","last_fullsized":"2023-07-06T03:54:25.981603+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:58:10.515633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.994191+0000","last_change":"2023-07-06T03:54:25.992478+0000","last_active":"2023-07-06T03:54:25.994191+0000","last_peered":"2023-07-06T03:54:25.994191+0000","last_clean":"2023-07-06T03:54:25.994191+0000","last_became_active":"2023-07-06T03:54:25.992184+0000","last_became_peered":"2023-07-06T03:54:25.992184+0000","last_unstale":"2023-07-06T03:54:25.994191+0000","last_undegraded":"2023-07-06T03:54:25.994191+0000","last_fullsized":"2023-07-06T03:54:25.994191+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:59:53.917909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.993785+0000","last_change":"2023-07-06T03:54:24.968618+0000","last_active":"2023-07-06T03:54:25.993785+0000","last_peered":"2023-07-06T03:54:25.993785+0000","last_clean":"2023-07-06T03:54:25.993785+0000","last_became_active":"2023-07-06T03:54:24.968325+0000","last_became_peered":"2023-07-06T03:54:24.968325+0000","last_unstale":"2023-07-06T03:54:25.993785+0000","last_undegraded":"2023-07-06T03:54:25.993785+0000","last_fullsized":"2023-07-06T03:54:25.993785+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:01:36.146555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.977683+0000","last_change":"2023-07-06T03:54:25.970445+0000","last_active":"2023-07-06T03:54:25.977683+0000","last_peered":"2023-07-06T03:54:25.977683+0000","last_clean":"2023-07-06T03:54:25.977683+0000","last_became_active":"2023-07-06T03:54:25.970136+0000","last_became_peered":"2023-07-06T03:54:25.970136+0000","last_unstale":"2023-07-06T03:54:25.977683+0000","last_undegraded":"2023-07-06T03:54:25.977683+0000","last_fullsized":"2023-07-06T03:54:25.977683+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:57:58.526146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.981300+0000","last_change":"2023-07-06T03:54:24.967094+0000","last_active":"2023-07-06T03:54:25.981300+0000","last_peered":"2023-07-06T03:54:25.981300+0000","last_clean":"2023-07-06T03:54:25.981300+0000","last_became_active":"2023-07-06T03:54:24.966840+0000","last_became_peered":"2023-07-06T03:54:24.966840+0000","last_unstale":"2023-07-06T03:54:25.981300+0000","last_undegraded":"2023-07-06T03:54:25.981300+0000","last_fullsized":"2023-07-06T03:54:25.981300+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:36:34.286009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.993556+0000","last_change":"2023-07-06T03:54:25.991207+0000","last_active":"2023-07-06T03:54:25.993556+0000","last_peered":"2023-07-06T03:54:25.993556+0000","last_clean":"2023-07-06T03:54:25.993556+0000","last_became_active":"2023-07-06T03:54:25.990919+0000","last_became_peered":"2023-07-06T03:54:25.990919+0000","last_unstale":"2023-07-06T03:54:25.993556+0000","last_undegraded":"2023-07-06T03:54:25.993556+0000","last_fullsized":"2023-07-06T03:54:25.993556+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:59:02.039814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.990048+0000","last_change":"2023-07-06T03:54:24.968210+0000","last_active":"2023-07-06T03:54:25.990048+0000","last_peered":"2023-07-06T03:54:25.990048+0000","last_clean":"2023-07-06T03:54:25.990048+0000","last_became_active":"2023-07-06T03:54:24.967928+0000","last_became_peered":"2023-07-06T03:54:24.967928+0000","last_unstale":"2023-07-06T03:54:25.990048+0000","last_undegraded":"2023-07-06T03:54:25.990048+0000","last_fullsized":"2023-07-06T03:54:25.990048+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:11:31.759804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.988261+0000","last_change":"2023-07-06T03:54:25.980651+0000","last_active":"2023-07-06T03:54:25.988261+0000","last_peered":"2023-07-06T03:54:25.988261+0000","last_clean":"2023-07-06T03:54:25.988261+0000","last_became_active":"2023-07-06T03:54:25.980072+0000","last_became_peered":"2023-07-06T03:54:25.980072+0000","last_unstale":"2023-07-06T03:54:25.988261+0000","last_undegraded":"2023-07-06T03:54:25.988261+0000","last_fullsized":"2023-07-06T03:54:25.988261+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:03:50.931934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.963743+0000","last_change":"2023-07-06T03:54:24.964106+0000","last_active":"2023-07-06T03:54:25.963743+0000","last_peered":"2023-07-06T03:54:25.963743+0000","last_clean":"2023-07-06T03:54:25.963743+0000","last_became_active":"2023-07-06T03:54:24.963813+0000","last_became_peered":"2023-07-06T03:54:24.963813+0000","last_unstale":"2023-07-06T03:54:25.963743+0000","last_undegraded":"2023-07-06T03:54:25.963743+0000","last_fullsized":"2023-07-06T03:54:25.963743+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:34:56.688555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.983436+0000","last_change":"2023-07-06T03:54:25.977287+0000","last_active":"2023-07-06T03:54:25.983436+0000","last_peered":"2023-07-06T03:54:25.983436+0000","last_clean":"2023-07-06T03:54:25.983436+0000","last_became_active":"2023-07-06T03:54:25.976934+0000","last_became_peered":"2023-07-06T03:54:25.976934+0000","last_unstale":"2023-07-06T03:54:25.983436+0000","last_undegraded":"2023-07-06T03:54:25.983436+0000","last_fullsized":"2023-07-06T03:54:25.983436+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:27:51.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964317+0000","last_change":"2023-07-06T03:54:24.959348+0000","last_active":"2023-07-06T03:54:25.964317+0000","last_peered":"2023-07-06T03:54:25.964317+0000","last_clean":"2023-07-06T03:54:25.964317+0000","last_became_active":"2023-07-06T03:54:24.958739+0000","last_became_peered":"2023-07-06T03:54:24.958739+0000","last_unstale":"2023-07-06T03:54:25.964317+0000","last_undegraded":"2023-07-06T03:54:25.964317+0000","last_fullsized":"2023-07-06T03:54:25.964317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:17:13.892908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.993829+0000","last_change":"2023-07-06T03:54:25.989190+0000","last_active":"2023-07-06T03:54:25.993829+0000","last_peered":"2023-07-06T03:54:25.993829+0000","last_clean":"2023-07-06T03:54:25.993829+0000","last_became_active":"2023-07-06T03:54:25.988768+0000","last_became_peered":"2023-07-06T03:54:25.988768+0000","last_unstale":"2023-07-06T03:54:25.993829+0000","last_undegraded":"2023-07-06T03:54:25.993829+0000","last_fullsized":"2023-07-06T03:54:25.993829+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:38:40.562890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997498+0000","last_change":"2023-07-06T03:54:24.966629+0000","last_active":"2023-07-06T03:54:25.997498+0000","last_peered":"2023-07-06T03:54:25.997498+0000","last_clean":"2023-07-06T03:54:25.997498+0000","last_became_active":"2023-07-06T03:54:24.966254+0000","last_became_peered":"2023-07-06T03:54:24.966254+0000","last_unstale":"2023-07-06T03:54:25.997498+0000","last_undegraded":"2023-07-06T03:54:25.997498+0000","last_fullsized":"2023-07-06T03:54:25.997498+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:42:51.264374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.991272+0000","last_change":"2023-07-06T03:54:25.985957+0000","last_active":"2023-07-06T03:54:25.991272+0000","last_peered":"2023-07-06T03:54:25.991272+0000","last_clean":"2023-07-06T03:54:25.991272+0000","last_became_active":"2023-07-06T03:54:25.985688+0000","last_became_peered":"2023-07-06T03:54:25.985688+0000","last_unstale":"2023-07-06T03:54:25.991272+0000","last_undegraded":"2023-07-06T03:54:25.991272+0000","last_fullsized":"2023-07-06T03:54:25.991272+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:45:52.736915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997209+0000","last_change":"2023-07-06T03:54:24.963213+0000","last_active":"2023-07-06T03:54:25.997209+0000","last_peered":"2023-07-06T03:54:25.997209+0000","last_clean":"2023-07-06T03:54:25.997209+0000","last_became_active":"2023-07-06T03:54:24.962771+0000","last_became_peered":"2023-07-06T03:54:24.962771+0000","last_unstale":"2023-07-06T03:54:25.997209+0000","last_undegraded":"2023-07-06T03:54:25.997209+0000","last_fullsized":"2023-07-06T03:54:25.997209+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:43:14.831709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.996578+0000","last_change":"2023-07-06T03:54:25.991263+0000","last_active":"2023-07-06T03:54:25.996578+0000","last_peered":"2023-07-06T03:54:25.996578+0000","last_clean":"2023-07-06T03:54:25.996578+0000","last_became_active":"2023-07-06T03:54:25.990981+0000","last_became_peered":"2023-07-06T03:54:25.990981+0000","last_unstale":"2023-07-06T03:54:25.996578+0000","last_undegraded":"2023-07-06T03:54:25.996578+0000","last_fullsized":"2023-07-06T03:54:25.996578+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:03:50.931934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.988023+0000","last_change":"2023-07-06T03:54:24.960832+0000","last_active":"2023-07-06T03:54:25.988023+0000","last_peered":"2023-07-06T03:54:25.988023+0000","last_clean":"2023-07-06T03:54:25.988023+0000","last_became_active":"2023-07-06T03:54:24.960553+0000","last_became_peered":"2023-07-06T03:54:24.960553+0000","last_unstale":"2023-07-06T03:54:25.988023+0000","last_undegraded":"2023-07-06T03:54:25.988023+0000","last_fullsized":"2023-07-06T03:54:25.988023+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:55:32.535767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.986816+0000","last_change":"2023-07-06T03:54:25.982232+0000","last_active":"2023-07-06T03:54:25.986816+0000","last_peered":"2023-07-06T03:54:25.986816+0000","last_clean":"2023-07-06T03:54:25.986816+0000","last_became_active":"2023-07-06T03:54:25.981957+0000","last_became_peered":"2023-07-06T03:54:25.981957+0000","last_unstale":"2023-07-06T03:54:25.986816+0000","last_undegraded":"2023-07-06T03:54:25.986816+0000","last_fullsized":"2023-07-06T03:54:25.986816+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:21:14.625844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.963328+0000","last_change":"2023-07-06T03:54:24.968500+0000","last_active":"2023-07-06T03:54:25.963328+0000","last_peered":"2023-07-06T03:54:25.963328+0000","last_clean":"2023-07-06T03:54:25.963328+0000","last_became_active":"2023-07-06T03:54:24.968254+0000","last_became_peered":"2023-07-06T03:54:24.968254+0000","last_unstale":"2023-07-06T03:54:25.963328+0000","last_undegraded":"2023-07-06T03:54:25.963328+0000","last_fullsized":"2023-07-06T03:54:25.963328+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:07:50.951010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987360+0000","last_change":"2023-07-06T03:54:25.984331+0000","last_active":"2023-07-06T03:54:25.987360+0000","last_peered":"2023-07-06T03:54:25.987360+0000","last_clean":"2023-07-06T03:54:25.987360+0000","last_became_active":"2023-07-06T03:54:25.984045+0000","last_became_peered":"2023-07-06T03:54:25.984045+0000","last_unstale":"2023-07-06T03:54:25.987360+0000","last_undegraded":"2023-07-06T03:54:25.987360+0000","last_fullsized":"2023-07-06T03:54:25.987360+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:12:12.357992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.994878+0000","last_change":"2023-07-06T03:54:24.963620+0000","last_active":"2023-07-06T03:54:25.994878+0000","last_peered":"2023-07-06T03:54:25.994878+0000","last_clean":"2023-07-06T03:54:25.994878+0000","last_became_active":"2023-07-06T03:54:24.963272+0000","last_became_peered":"2023-07-06T03:54:24.963272+0000","last_unstale":"2023-07-06T03:54:25.994878+0000","last_undegraded":"2023-07-06T03:54:25.994878+0000","last_fullsized":"2023-07-06T03:54:25.994878+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:23.966416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997995+0000","last_change":"2023-07-06T03:54:25.992970+0000","last_active":"2023-07-06T03:54:25.997995+0000","last_peered":"2023-07-06T03:54:25.997995+0000","last_clean":"2023-07-06T03:54:25.997995+0000","last_became_active":"2023-07-06T03:54:25.992697+0000","last_became_peered":"2023-07-06T03:54:25.992697+0000","last_unstale":"2023-07-06T03:54:25.997995+0000","last_undegraded":"2023-07-06T03:54:25.997995+0000","last_fullsized":"2023-07-06T03:54:25.997995+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:01:43.196893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.967959+0000","last_change":"2023-07-06T03:54:24.952178+0000","last_active":"2023-07-06T03:54:25.967959+0000","last_peered":"2023-07-06T03:54:25.967959+0000","last_clean":"2023-07-06T03:54:25.967959+0000","last_became_active":"2023-07-06T03:54:24.951909+0000","last_became_peered":"2023-07-06T03:54:24.951909+0000","last_unstale":"2023-07-06T03:54:25.967959+0000","last_undegraded":"2023-07-06T03:54:25.967959+0000","last_fullsized":"2023-07-06T03:54:25.967959+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:53:18.590181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.982290+0000","last_change":"2023-07-06T03:54:25.979010+0000","last_active":"2023-07-06T03:54:25.982290+0000","last_peered":"2023-07-06T03:54:25.982290+0000","last_clean":"2023-07-06T03:54:25.982290+0000","last_became_active":"2023-07-06T03:54:25.978750+0000","last_became_peered":"2023-07-06T03:54:25.978750+0000","last_unstale":"2023-07-06T03:54:25.982290+0000","last_undegraded":"2023-07-06T03:54:25.982290+0000","last_fullsized":"2023-07-06T03:54:25.982290+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:20:54.856278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964437+0000","last_change":"2023-07-06T03:54:24.959070+0000","last_active":"2023-07-06T03:54:25.964437+0000","last_peered":"2023-07-06T03:54:25.964437+0000","last_clean":"2023-07-06T03:54:25.964437+0000","last_became_active":"2023-07-06T03:54:24.958759+0000","last_became_peered":"2023-07-06T03:54:24.958759+0000","last_unstale":"2023-07-06T03:54:25.964437+0000","last_undegraded":"2023-07-06T03:54:25.964437+0000","last_fullsized":"2023-07-06T03:54:25.964437+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:13.857352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.989308+0000","last_change":"2023-07-06T03:54:25.986506+0000","last_active":"2023-07-06T03:54:25.989308+0000","last_peered":"2023-07-06T03:54:25.989308+0000","last_clean":"2023-07-06T03:54:25.989308+0000","last_became_active":"2023-07-06T03:54:25.986023+0000","last_became_peered":"2023-07-06T03:54:25.986023+0000","last_unstale":"2023-07-06T03:54:25.989308+0000","last_undegraded":"2023-07-06T03:54:25.989308+0000","last_fullsized":"2023-07-06T03:54:25.989308+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:26.978184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.971868+0000","last_change":"2023-07-06T03:54:24.961481+0000","last_active":"2023-07-06T03:54:25.971868+0000","last_peered":"2023-07-06T03:54:25.971868+0000","last_clean":"2023-07-06T03:54:25.971868+0000","last_became_active":"2023-07-06T03:54:24.961231+0000","last_became_peered":"2023-07-06T03:54:24.961231+0000","last_unstale":"2023-07-06T03:54:25.971868+0000","last_undegraded":"2023-07-06T03:54:25.971868+0000","last_fullsized":"2023-07-06T03:54:25.971868+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:50:51.592794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:26.000318+0000","last_change":"2023-07-06T03:54:25.995711+0000","last_active":"2023-07-06T03:54:26.000318+0000","last_peered":"2023-07-06T03:54:26.000318+0000","last_clean":"2023-07-06T03:54:26.000318+0000","last_became_active":"2023-07-06T03:54:25.995438+0000","last_became_peered":"2023-07-06T03:54:25.995438+0000","last_unstale":"2023-07-06T03:54:26.000318+0000","last_undegraded":"2023-07-06T03:54:26.000318+0000","last_fullsized":"2023-07-06T03:54:26.000318+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:30:08.592198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.994766+0000","last_change":"2023-07-06T03:54:24.961955+0000","last_active":"2023-07-06T03:54:25.994766+0000","last_peered":"2023-07-06T03:54:25.994766+0000","last_clean":"2023-07-06T03:54:25.994766+0000","last_became_active":"2023-07-06T03:54:24.961661+0000","last_became_peered":"2023-07-06T03:54:24.961661+0000","last_unstale":"2023-07-06T03:54:25.994766+0000","last_undegraded":"2023-07-06T03:54:25.994766+0000","last_fullsized":"2023-07-06T03:54:25.994766+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:24:21.072667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.989086+0000","last_change":"2023-07-06T03:54:25.981562+0000","last_active":"2023-07-06T03:54:25.989086+0000","last_peered":"2023-07-06T03:54:25.989086+0000","last_clean":"2023-07-06T03:54:25.989086+0000","last_became_active":"2023-07-06T03:54:25.981273+0000","last_became_peered":"2023-07-06T03:54:25.981273+0000","last_unstale":"2023-07-06T03:54:25.989086+0000","last_undegraded":"2023-07-06T03:54:25.989086+0000","last_fullsized":"2023-07-06T03:54:25.989086+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:40:17.141868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.972199+0000","last_change":"2023-07-06T03:54:24.966105+0000","last_active":"2023-07-06T03:54:25.972199+0000","last_peered":"2023-07-06T03:54:25.972199+0000","last_clean":"2023-07-06T03:54:25.972199+0000","last_became_active":"2023-07-06T03:54:24.965741+0000","last_became_peered":"2023-07-06T03:54:24.965741+0000","last_unstale":"2023-07-06T03:54:25.972199+0000","last_undegraded":"2023-07-06T03:54:25.972199+0000","last_fullsized":"2023-07-06T03:54:25.972199+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:58:24.843850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.993579+0000","last_change":"2023-07-06T03:54:25.990457+0000","last_active":"2023-07-06T03:54:25.993579+0000","last_peered":"2023-07-06T03:54:25.993579+0000","last_clean":"2023-07-06T03:54:25.993579+0000","last_became_active":"2023-07-06T03:54:25.990185+0000","last_became_peered":"2023-07-06T03:54:25.990185+0000","last_unstale":"2023-07-06T03:54:25.993579+0000","last_undegraded":"2023-07-06T03:54:25.993579+0000","last_fullsized":"2023-07-06T03:54:25.993579+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:35:11.135492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.973269+0000","last_change":"2023-07-06T03:54:24.968361+0000","last_active":"2023-07-06T03:54:25.973269+0000","last_peered":"2023-07-06T03:54:25.973269+0000","last_clean":"2023-07-06T03:54:25.973269+0000","last_became_active":"2023-07-06T03:54:24.968079+0000","last_became_peered":"2023-07-06T03:54:24.968079+0000","last_unstale":"2023-07-06T03:54:25.973269+0000","last_undegraded":"2023-07-06T03:54:25.973269+0000","last_fullsized":"2023-07-06T03:54:25.973269+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:31:15.895823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.977635+0000","last_change":"2023-07-06T03:54:25.972893+0000","last_active":"2023-07-06T03:54:25.977635+0000","last_peered":"2023-07-06T03:54:25.977635+0000","last_clean":"2023-07-06T03:54:25.977635+0000","last_became_active":"2023-07-06T03:54:25.972615+0000","last_became_peered":"2023-07-06T03:54:25.972615+0000","last_unstale":"2023-07-06T03:54:25.977635+0000","last_undegraded":"2023-07-06T03:54:25.977635+0000","last_fullsized":"2023-07-06T03:54:25.977635+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:08.012638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.985130+0000","last_change":"2023-07-06T03:54:24.967367+0000","last_active":"2023-07-06T03:54:25.985130+0000","last_peered":"2023-07-06T03:54:25.985130+0000","last_clean":"2023-07-06T03:54:25.985130+0000","last_became_active":"2023-07-06T03:54:24.966941+0000","last_became_peered":"2023-07-06T03:54:24.966941+0000","last_unstale":"2023-07-06T03:54:25.985130+0000","last_undegraded":"2023-07-06T03:54:25.985130+0000","last_fullsized":"2023-07-06T03:54:25.985130+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:11:23.437909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998131+0000","last_change":"2023-07-06T03:54:25.994315+0000","last_active":"2023-07-06T03:54:25.998131+0000","last_peered":"2023-07-06T03:54:25.998131+0000","last_clean":"2023-07-06T03:54:25.998131+0000","last_became_active":"2023-07-06T03:54:25.994034+0000","last_became_peered":"2023-07-06T03:54:25.994034+0000","last_unstale":"2023-07-06T03:54:25.998131+0000","last_undegraded":"2023-07-06T03:54:25.998131+0000","last_fullsized":"2023-07-06T03:54:25.998131+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:00.309903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.972550+0000","last_change":"2023-07-06T03:54:24.963634+0000","last_active":"2023-07-06T03:54:25.972550+0000","last_peered":"2023-07-06T03:54:25.972550+0000","last_clean":"2023-07-06T03:54:25.972550+0000","last_became_active":"2023-07-06T03:54:24.963383+0000","last_became_peered":"2023-07-06T03:54:24.963383+0000","last_unstale":"2023-07-06T03:54:25.972550+0000","last_undegraded":"2023-07-06T03:54:25.972550+0000","last_fullsized":"2023-07-06T03:54:25.972550+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:49.510799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.990858+0000","last_change":"2023-07-06T03:54:25.985942+0000","last_active":"2023-07-06T03:54:25.990858+0000","last_peered":"2023-07-06T03:54:25.990858+0000","last_clean":"2023-07-06T03:54:25.990858+0000","last_became_active":"2023-07-06T03:54:25.985555+0000","last_became_peered":"2023-07-06T03:54:25.985555+0000","last_unstale":"2023-07-06T03:54:25.990858+0000","last_undegraded":"2023-07-06T03:54:25.990858+0000","last_fullsized":"2023-07-06T03:54:25.990858+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:08.012638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.968110+0000","last_change":"2023-07-06T03:54:24.958887+0000","last_active":"2023-07-06T03:54:25.968110+0000","last_peered":"2023-07-06T03:54:25.968110+0000","last_clean":"2023-07-06T03:54:25.968110+0000","last_became_active":"2023-07-06T03:54:24.958620+0000","last_became_peered":"2023-07-06T03:54:24.958620+0000","last_unstale":"2023-07-06T03:54:25.968110+0000","last_undegraded":"2023-07-06T03:54:25.968110+0000","last_fullsized":"2023-07-06T03:54:25.968110+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:48:47.700659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987045+0000","last_change":"2023-07-06T03:54:25.983250+0000","last_active":"2023-07-06T03:54:25.987045+0000","last_peered":"2023-07-06T03:54:25.987045+0000","last_clean":"2023-07-06T03:54:25.987045+0000","last_became_active":"2023-07-06T03:54:25.982954+0000","last_became_peered":"2023-07-06T03:54:25.982954+0000","last_unstale":"2023-07-06T03:54:25.987045+0000","last_undegraded":"2023-07-06T03:54:25.987045+0000","last_fullsized":"2023-07-06T03:54:25.987045+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:37:35.751863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.982021+0000","last_change":"2023-07-06T03:54:24.961554+0000","last_active":"2023-07-06T03:54:25.982021+0000","last_peered":"2023-07-06T03:54:25.982021+0000","last_clean":"2023-07-06T03:54:25.982021+0000","last_became_active":"2023-07-06T03:54:24.961221+0000","last_became_peered":"2023-07-06T03:54:24.961221+0000","last_unstale":"2023-07-06T03:54:25.982021+0000","last_undegraded":"2023-07-06T03:54:25.982021+0000","last_fullsized":"2023-07-06T03:54:25.982021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:11:08.184109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.994352+0000","last_change":"2023-07-06T03:54:25.989922+0000","last_active":"2023-07-06T03:54:25.994352+0000","last_peered":"2023-07-06T03:54:25.994352+0000","last_clean":"2023-07-06T03:54:25.994352+0000","last_became_active":"2023-07-06T03:54:25.989588+0000","last_became_peered":"2023-07-06T03:54:25.989588+0000","last_unstale":"2023-07-06T03:54:25.994352+0000","last_undegraded":"2023-07-06T03:54:25.994352+0000","last_fullsized":"2023-07-06T03:54:25.994352+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:09:31.465372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.972211+0000","last_change":"2023-07-06T03:54:24.957392+0000","last_active":"2023-07-06T03:54:25.972211+0000","last_peered":"2023-07-06T03:54:25.972211+0000","last_clean":"2023-07-06T03:54:25.972211+0000","last_became_active":"2023-07-06T03:54:24.957119+0000","last_became_peered":"2023-07-06T03:54:24.957119+0000","last_unstale":"2023-07-06T03:54:25.972211+0000","last_undegraded":"2023-07-06T03:54:25.972211+0000","last_fullsized":"2023-07-06T03:54:25.972211+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:09:36.909936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.984427+0000","last_change":"2023-07-06T03:54:25.981315+0000","last_active":"2023-07-06T03:54:25.984427+0000","last_peered":"2023-07-06T03:54:25.984427+0000","last_clean":"2023-07-06T03:54:25.984427+0000","last_became_active":"2023-07-06T03:54:25.981019+0000","last_became_peered":"2023-07-06T03:54:25.981019+0000","last_unstale":"2023-07-06T03:54:25.984427+0000","last_undegraded":"2023-07-06T03:54:25.984427+0000","last_fullsized":"2023-07-06T03:54:25.984427+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:46:36.047228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.977617+0000","last_change":"2023-07-06T03:54:24.960376+0000","last_active":"2023-07-06T03:54:25.977617+0000","last_peered":"2023-07-06T03:54:25.977617+0000","last_clean":"2023-07-06T03:54:25.977617+0000","last_became_active":"2023-07-06T03:54:24.960090+0000","last_became_peered":"2023-07-06T03:54:24.960090+0000","last_unstale":"2023-07-06T03:54:25.977617+0000","last_undegraded":"2023-07-06T03:54:25.977617+0000","last_fullsized":"2023-07-06T03:54:25.977617+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:12:50.172275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":173,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.974356+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:54:25.974356+0000","last_peered":"2023-07-06T03:54:25.974356+0000","last_clean":"2023-07-06T03:54:25.974356+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:54:25.974356+0000","last_undegraded":"2023-07-06T03:54:25.974356+0000","last_fullsized":"2023-07-06T03:54:25.974356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:43:21.029393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998125+0000","last_change":"2023-07-06T03:54:25.995178+0000","last_active":"2023-07-06T03:54:25.998125+0000","last_peered":"2023-07-06T03:54:25.998125+0000","last_clean":"2023-07-06T03:54:25.998125+0000","last_became_active":"2023-07-06T03:54:25.994895+0000","last_became_peered":"2023-07-06T03:54:25.994895+0000","last_unstale":"2023-07-06T03:54:25.998125+0000","last_undegraded":"2023-07-06T03:54:25.998125+0000","last_fullsized":"2023-07-06T03:54:25.998125+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:21:35.838671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964590+0000","last_change":"2023-07-06T03:54:24.960620+0000","last_active":"2023-07-06T03:54:25.964590+0000","last_peered":"2023-07-06T03:54:25.964590+0000","last_clean":"2023-07-06T03:54:25.964590+0000","last_became_active":"2023-07-06T03:54:24.958294+0000","last_became_peered":"2023-07-06T03:54:24.958294+0000","last_unstale":"2023-07-06T03:54:25.964590+0000","last_undegraded":"2023-07-06T03:54:25.964590+0000","last_fullsized":"2023-07-06T03:54:25.964590+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:46:43.436206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":185,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.963405+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:54:25.963405+0000","last_peered":"2023-07-06T03:54:25.963405+0000","last_clean":"2023-07-06T03:54:25.963405+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:54:25.963405+0000","last_undegraded":"2023-07-06T03:54:25.963405+0000","last_fullsized":"2023-07-06T03:54:25.963405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:59:25.373090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.991268+0000","last_change":"2023-07-06T03:54:25.988190+0000","last_active":"2023-07-06T03:54:25.991268+0000","last_peered":"2023-07-06T03:54:25.991268+0000","last_clean":"2023-07-06T03:54:25.991268+0000","last_became_active":"2023-07-06T03:54:25.987907+0000","last_became_peered":"2023-07-06T03:54:25.987907+0000","last_unstale":"2023-07-06T03:54:25.991268+0000","last_undegraded":"2023-07-06T03:54:25.991268+0000","last_fullsized":"2023-07-06T03:54:25.991268+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:02:01.652970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.965658+0000","last_change":"2023-07-06T03:54:24.960738+0000","last_active":"2023-07-06T03:54:25.965658+0000","last_peered":"2023-07-06T03:54:25.965658+0000","last_clean":"2023-07-06T03:54:25.965658+0000","last_became_active":"2023-07-06T03:54:24.960472+0000","last_became_peered":"2023-07-06T03:54:24.960472+0000","last_unstale":"2023-07-06T03:54:25.965658+0000","last_undegraded":"2023-07-06T03:54:25.965658+0000","last_fullsized":"2023-07-06T03:54:25.965658+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:55:51.507033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":189,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964539+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:54:25.964539+0000","last_peered":"2023-07-06T03:54:25.964539+0000","last_clean":"2023-07-06T03:54:25.964539+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:54:25.964539+0000","last_undegraded":"2023-07-06T03:54:25.964539+0000","last_fullsized":"2023-07-06T03:54:25.964539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:02:51.535276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987230+0000","last_change":"2023-07-06T03:54:25.984232+0000","last_active":"2023-07-06T03:54:25.987230+0000","last_peered":"2023-07-06T03:54:25.987230+0000","last_clean":"2023-07-06T03:54:25.987230+0000","last_became_active":"2023-07-06T03:54:25.983927+0000","last_became_peered":"2023-07-06T03:54:25.983927+0000","last_unstale":"2023-07-06T03:54:25.987230+0000","last_undegraded":"2023-07-06T03:54:25.987230+0000","last_fullsized":"2023-07-06T03:54:25.987230+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:00:05.139216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"111'8","reported_seq":19,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.984046+0000","last_change":"2023-07-06T03:54:24.960132+0000","last_active":"2023-07-06T03:54:25.984046+0000","last_peered":"2023-07-06T03:54:25.984046+0000","last_clean":"2023-07-06T03:54:25.984046+0000","last_became_active":"2023-07-06T03:54:24.959848+0000","last_became_peered":"2023-07-06T03:54:24.959848+0000","last_unstale":"2023-07-06T03:54:25.984046+0000","last_undegraded":"2023-07-06T03:54:25.984046+0000","last_fullsized":"2023-07-06T03:54:25.984046+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:32:46.189015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":185,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964100+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:54:25.964100+0000","last_peered":"2023-07-06T03:54:25.964100+0000","last_clean":"2023-07-06T03:54:25.964100+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:54:25.964100+0000","last_undegraded":"2023-07-06T03:54:25.964100+0000","last_fullsized":"2023-07-06T03:54:25.964100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:30:27.500176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":325,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.967066+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:54:25.967066+0000","last_peered":"2023-07-06T03:54:25.967066+0000","last_clean":"2023-07-06T03:54:25.967066+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:54:25.967066+0000","last_undegraded":"2023-07-06T03:54:25.967066+0000","last_fullsized":"2023-07-06T03:54:25.967066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:58:32.498146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.983007+0000","last_change":"2023-07-06T03:54:25.978797+0000","last_active":"2023-07-06T03:54:25.983007+0000","last_peered":"2023-07-06T03:54:25.983007+0000","last_clean":"2023-07-06T03:54:25.983007+0000","last_became_active":"2023-07-06T03:54:25.978509+0000","last_became_peered":"2023-07-06T03:54:25.978509+0000","last_unstale":"2023-07-06T03:54:25.983007+0000","last_undegraded":"2023-07-06T03:54:25.983007+0000","last_fullsized":"2023-07-06T03:54:25.983007+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:06:16.468463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.980551+0000","last_change":"2023-07-06T03:54:24.962985+0000","last_active":"2023-07-06T03:54:25.980551+0000","last_peered":"2023-07-06T03:54:25.980551+0000","last_clean":"2023-07-06T03:54:25.980551+0000","last_became_active":"2023-07-06T03:54:24.962602+0000","last_became_peered":"2023-07-06T03:54:24.962602+0000","last_unstale":"2023-07-06T03:54:25.980551+0000","last_undegraded":"2023-07-06T03:54:25.980551+0000","last_fullsized":"2023-07-06T03:54:25.980551+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:03:25.070510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":185,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964023+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:54:25.964023+0000","last_peered":"2023-07-06T03:54:25.964023+0000","last_clean":"2023-07-06T03:54:25.964023+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:54:25.964023+0000","last_undegraded":"2023-07-06T03:54:25.964023+0000","last_fullsized":"2023-07-06T03:54:25.964023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:42:47.254337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.993433+0000","last_change":"2023-07-06T03:54:25.989854+0000","last_active":"2023-07-06T03:54:25.993433+0000","last_peered":"2023-07-06T03:54:25.993433+0000","last_clean":"2023-07-06T03:54:25.993433+0000","last_became_active":"2023-07-06T03:54:25.989547+0000","last_became_peered":"2023-07-06T03:54:25.989547+0000","last_unstale":"2023-07-06T03:54:25.993433+0000","last_undegraded":"2023-07-06T03:54:25.993433+0000","last_fullsized":"2023-07-06T03:54:25.993433+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:54.414464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.976519+0000","last_change":"2023-07-06T03:54:24.966446+0000","last_active":"2023-07-06T03:54:25.976519+0000","last_peered":"2023-07-06T03:54:25.976519+0000","last_clean":"2023-07-06T03:54:25.976519+0000","last_became_active":"2023-07-06T03:54:24.966105+0000","last_became_peered":"2023-07-06T03:54:24.966105+0000","last_unstale":"2023-07-06T03:54:25.976519+0000","last_undegraded":"2023-07-06T03:54:25.976519+0000","last_fullsized":"2023-07-06T03:54:25.976519+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:06:39.910267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":185,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.963667+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:54:25.963667+0000","last_peered":"2023-07-06T03:54:25.963667+0000","last_clean":"2023-07-06T03:54:25.963667+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:54:25.963667+0000","last_undegraded":"2023-07-06T03:54:25.963667+0000","last_fullsized":"2023-07-06T03:54:25.963667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:44:51.517874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.973688+0000","last_change":"2023-07-06T03:54:25.971739+0000","last_active":"2023-07-06T03:54:25.973688+0000","last_peered":"2023-07-06T03:54:25.973688+0000","last_clean":"2023-07-06T03:54:25.973688+0000","last_became_active":"2023-07-06T03:54:25.971403+0000","last_became_peered":"2023-07-06T03:54:25.971403+0000","last_unstale":"2023-07-06T03:54:25.973688+0000","last_undegraded":"2023-07-06T03:54:25.973688+0000","last_fullsized":"2023-07-06T03:54:25.973688+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:02:43.079625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"111'3","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.984012+0000","last_change":"2023-07-06T03:54:24.965857+0000","last_active":"2023-07-06T03:54:25.984012+0000","last_peered":"2023-07-06T03:54:25.984012+0000","last_clean":"2023-07-06T03:54:25.984012+0000","last_became_active":"2023-07-06T03:54:24.965573+0000","last_became_peered":"2023-07-06T03:54:24.965573+0000","last_unstale":"2023-07-06T03:54:25.984012+0000","last_undegraded":"2023-07-06T03:54:25.984012+0000","last_fullsized":"2023-07-06T03:54:25.984012+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:43:57.859083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":177,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.966250+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:54:25.966250+0000","last_peered":"2023-07-06T03:54:25.966250+0000","last_clean":"2023-07-06T03:54:25.966250+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:54:25.966250+0000","last_undegraded":"2023-07-06T03:54:25.966250+0000","last_fullsized":"2023-07-06T03:54:25.966250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:47.710521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987337+0000","last_change":"2023-07-06T03:54:25.986326+0000","last_active":"2023-07-06T03:54:25.987337+0000","last_peered":"2023-07-06T03:54:25.987337+0000","last_clean":"2023-07-06T03:54:25.987337+0000","last_became_active":"2023-07-06T03:54:25.986079+0000","last_became_peered":"2023-07-06T03:54:25.986079+0000","last_unstale":"2023-07-06T03:54:25.987337+0000","last_undegraded":"2023-07-06T03:54:25.987337+0000","last_fullsized":"2023-07-06T03:54:25.987337+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:44.388360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"111'6","reported_seq":17,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:26.000501+0000","last_change":"2023-07-06T03:54:24.965263+0000","last_active":"2023-07-06T03:54:26.000501+0000","last_peered":"2023-07-06T03:54:26.000501+0000","last_clean":"2023-07-06T03:54:26.000501+0000","last_became_active":"2023-07-06T03:54:24.964956+0000","last_became_peered":"2023-07-06T03:54:24.964956+0000","last_unstale":"2023-07-06T03:54:26.000501+0000","last_undegraded":"2023-07-06T03:54:26.000501+0000","last_fullsized":"2023-07-06T03:54:26.000501+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:13:17.999257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":185,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.963425+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:54:25.963425+0000","last_peered":"2023-07-06T03:54:25.963425+0000","last_clean":"2023-07-06T03:54:25.963425+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:54:25.963425+0000","last_undegraded":"2023-07-06T03:54:25.963425+0000","last_fullsized":"2023-07-06T03:54:25.963425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:23:20.454826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68600000000000005,"15min":0.70499999999999996},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.88300000000000001,"5min":1.0620000000000001,"15min":4.0570000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67900000000000005,"15min":0.752},"min":{"1min":0.39900000000000002,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.85499999999999998,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.499}]},{"osd":1,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68300000000000005,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.45000000000000001,"15min":0.34000000000000002},"max":{"1min":1.016,"5min":1.091,"15min":2.6219999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69099999999999995,"15min":0.82699999999999996},"min":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68100000000000005,"15min":0.76000000000000001},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.35199999999999998},"max":{"1min":0.80600000000000005,"5min":0.98599999999999999,"15min":3.77},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.48099999999999998,"5min":0.47799999999999998,"15min":0.45000000000000001},"max":{"1min":0.90800000000000003,"5min":0.94399999999999995,"15min":0.98899999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.753,"15min":0.752},"min":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.43099999999999999},"max":{"1min":0.91700000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.62,"15min":0.626},"min":{"1min":0.501,"5min":0.36099999999999999,"15min":0.34599999999999997},"max":{"1min":0.89600000000000002,"5min":0.90200000000000002,"15min":0.94699999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66800000000000004,"15min":0.70399999999999996},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.374},"max":{"1min":0.94099999999999995,"5min":0.94399999999999995,"15min":1.4590000000000001},"last":0.71499999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.52800000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.747},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.45300000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.56799999999999995,"5min":0.438,"15min":0.438},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.5349999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.77700000000000002},"min":{"1min":0.49199999999999999,"5min":0.46500000000000002,"15min":0.42299999999999999},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.76300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72799999999999998,"15min":0.67400000000000004},"min":{"1min":0.39800000000000002,"5min":0.32800000000000001,"15min":0.29399999999999998},"max":{"1min":0.89500000000000002,"5min":1.24,"15min":1.24},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.77800000000000002,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.44700000000000001,"15min":0.26900000000000002},"max":{"1min":1.046,"5min":3.3399999999999999,"15min":3.3399999999999999},"last":0.88400000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68799999999999994,"15min":0.66900000000000004},"min":{"1min":0.44500000000000001,"5min":0.372,"15min":0.34599999999999997},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.872},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70199999999999996,"15min":0.67800000000000005},"min":{"1min":0.51800000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94499999999999995,"5min":0.94699999999999995,"15min":1.1759999999999999},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.752,"15min":0.68200000000000005},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.98499999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77500000000000002,"15min":0.72799999999999998},"min":{"1min":0.60499999999999998,"5min":0.47699999999999998,"15min":0.38700000000000001},"max":{"1min":1.153,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.72599999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":1.085,"5min":1.2110000000000001,"15min":2.1850000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.373},"max":{"1min":1.071,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.69899999999999995},"min":{"1min":0.44900000000000001,"5min":0.42799999999999999,"15min":0.33200000000000002},"max":{"1min":0.83499999999999996,"5min":1.0309999999999999,"15min":1.1339999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.71699999999999997},"min":{"1min":0.42399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":0.98699999999999999,"15min":1.085},"last":0.84699999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65400000000000003,"15min":0.69599999999999995},"min":{"1min":0.48699999999999999,"5min":0.46300000000000002,"15min":0.38600000000000001},"max":{"1min":0.89500000000000002,"5min":0.98799999999999999,"15min":1.218},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72399999999999998,"15min":0.77000000000000002},"min":{"1min":0.54700000000000004,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0489999999999999,"15min":1.198},"last":0.82499999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72499999999999998,"15min":0.79500000000000004},"min":{"1min":0.54700000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.323},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74399999999999999,"15min":0.80600000000000005},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.41499999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.4139999999999999},"last":0.80600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1961984,"data_stored":1216106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.42999999999999999,"15min":0.50800000000000001},"min":{"1min":0.27100000000000002,"5min":0.25900000000000001,"15min":0.23699999999999999},"max":{"1min":0.56000000000000005,"5min":0.72299999999999998,"15min":2.8879999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50700000000000001,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.32300000000000001,"15min":0.313},"max":{"1min":0.621,"5min":0.78300000000000003,"15min":1.96},"last":3.0139999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.44600000000000001,"15min":0.78100000000000003},"min":{"1min":0.32300000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.58599999999999997,"5min":0.66800000000000004,"15min":13.055999999999999},"last":3.3690000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.54100000000000004,"15min":0.60599999999999998},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.59799999999999998,"5min":0.96199999999999997,"15min":2.8479999999999999},"last":3.0590000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52800000000000002,"15min":0.58699999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48299999999999998,"15min":0.52600000000000002},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.56899999999999995,"5min":0.71599999999999997,"15min":1.2709999999999999},"last":3.4049999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.73599999999999999,"15min":1.05},"min":{"1min":0.54000000000000004,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.90400000000000003,"5min":1,"15min":12.989000000000001},"last":1.444},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78200000000000003,"15min":0.81399999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":0.875,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":1.139}]},{"osd":5,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.432},"max":{"1min":0.88300000000000001,"5min":1.0860000000000001,"15min":1.3100000000000001},"last":1.7390000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72199999999999998,"15min":1.052},"min":{"1min":0.56100000000000005,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.83299999999999996,"5min":1.077,"15min":13.023999999999999},"last":1.494}]},{"osd":6,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.76200000000000001,"15min":0.79800000000000004},"min":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88700000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":2.9670000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":1.0960000000000001},"min":{"1min":0.59399999999999997,"5min":0.54000000000000004,"15min":0.47799999999999998},"max":{"1min":0.96399999999999997,"5min":1.0569999999999999,"15min":12.941000000000001},"last":2.5409999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.80900000000000005},"min":{"1min":0.48699999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92600000000000005,"5min":1.071,"15min":2.806},"last":2.431},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.755,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94199999999999995,"5min":1.1319999999999999,"15min":1.337},"last":1.7849999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574966,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.28899999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":0.57099999999999995,"5min":0.753,"15min":0.753},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.58099999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.66200000000000003,"5min":0.69199999999999995,"15min":0.73699999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48699999999999999,"15min":0.496},"min":{"1min":0.376,"5min":0.315,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51100000000000001,"15min":0.48499999999999999},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.29899999999999999},"max":{"1min":0.59099999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.443,"5min":0.39400000000000002,"15min":0.38800000000000001},"max":{"1min":0.628,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72699999999999998,"15min":0.73799999999999999},"min":{"1min":0.48399999999999999,"5min":0.45700000000000002,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":1.1699999999999999,"15min":1.171},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.59199999999999997,"5min":0.52400000000000002,"15min":0.36799999999999999},"max":{"1min":0.92200000000000004,"5min":1.032,"15min":1.034},"last":0.751}]},{"osd":5,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.72799999999999998},"min":{"1min":0.51100000000000001,"5min":0.503,"15min":0.45600000000000002},"max":{"1min":0.95399999999999996,"5min":1.1080000000000001,"15min":1.1419999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.55100000000000005,"5min":0.53600000000000003,"15min":0.46999999999999997},"max":{"1min":0.82199999999999995,"5min":1.1040000000000001,"15min":1.1220000000000001},"last":0.91800000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.84599999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73599999999999999,"15min":0.748},"min":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.46200000000000002},"max":{"1min":0.93999999999999995,"5min":1.0649999999999999,"15min":1.546},"last":0.80900000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78200000000000003,"15min":0.79400000000000004},"min":{"1min":0.56000000000000005,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.6830000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.76900000000000002,"15min":0.98999999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53100000000000003},"max":{"1min":0.90500000000000003,"5min":1.0669999999999999,"15min":9.5960000000000001},"last":0.84699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40576,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331264,"statfs":{"total":96636764160,"available":96595214336,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.47699999999999998,"15min":0.442},"min":{"1min":0.30499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65500000000000003,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.443,"5min":0.504,"15min":0.47999999999999998},"min":{"1min":0.34200000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56000000000000005,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.55000000000000004,"15min":0.51000000000000001},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.629,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.56599999999999995,"15min":0.52400000000000002},"min":{"1min":0.38,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.64600000000000002,"5min":3.657,"15min":3.657},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.626,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69099999999999995,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59699999999999998,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":0.442}]},{"osd":4,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94499999999999995,"15min":0.78600000000000003},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.121,"5min":12.007,"15min":12.007},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.81299999999999994,"15min":0.73699999999999999},"min":{"1min":0.52200000000000002,"5min":0.49299999999999999,"15min":0.36399999999999999},"max":{"1min":1.1930000000000001,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79200000000000004,"15min":0.75},"min":{"1min":0.51100000000000001,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":1.1459999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.88100000000000001,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.137,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.623}]},{"osd":6,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.94599999999999995,"15min":0.78900000000000003},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.165,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.92300000000000004,"15min":0.80900000000000005},"min":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":1.107,"5min":13.084,"15min":13.084},"last":0.55000000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83299999999999996,"15min":0.82499999999999996},"min":{"1min":0.60599999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.089,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.871,"15min":0.81799999999999995},"min":{"1min":0.55800000000000005,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.2170000000000001,"5min":4.484,"15min":4.484},"last":0.56599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574965,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.44900000000000001,"15min":0.56399999999999995},"min":{"1min":0.25700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60099999999999998,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51800000000000002,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.85699999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.45200000000000001,"15min":0.88300000000000001},"min":{"1min":0.316,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.54700000000000004,"5min":1.3620000000000001,"15min":15.911},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.53400000000000003,"15min":0.95699999999999996},"min":{"1min":0.33400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76000000000000001,"5min":1.347,"15min":16.024000000000001},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.499,"15min":0.499},"min":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.27400000000000002},"max":{"1min":0.61099999999999999,"5min":3.25,"15min":3.25},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.54200000000000004,"15min":0.96199999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.64100000000000001,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.499}]},{"osd":4,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.52800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":1.782,"15min":1.782},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75900000000000001,"15min":0.88400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.36699999999999999},"max":{"1min":1.373,"5min":1.724,"15min":5.7300000000000004},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76300000000000001,"15min":0.88500000000000001},"min":{"1min":0.54000000000000004,"5min":0.49199999999999999,"15min":0.39700000000000002},"max":{"1min":0.96099999999999997,"5min":1.411,"15min":5.077},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.93400000000000005},"min":{"1min":0.56999999999999995,"5min":0.54500000000000004,"15min":0.44800000000000001},"max":{"1min":0.94699999999999995,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76800000000000002,"15min":0.78900000000000003},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.026,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.75900000000000001,"15min":0.89300000000000002},"min":{"1min":0.61199999999999999,"5min":0.503,"15min":0.45600000000000002},"max":{"1min":0.98199999999999998,"5min":1.4490000000000001,"15min":5.157},"last":0.73799999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76800000000000002,"15min":0.81399999999999995},"min":{"1min":0.58599999999999997,"5min":0.51700000000000002,"15min":0.47399999999999998},"max":{"1min":0.94599999999999995,"5min":1.863,"15min":1.863},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.76600000000000001,"15min":0.90400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.492,"15min":5.1189999999999998},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66200000000000003,"15min":0.64600000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.88800000000000001,"5min":0.90400000000000003,"15min":1.784},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.65000000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.38},"max":{"1min":0.90200000000000002,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.80900000000000005},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.38},"max":{"1min":0.996,"5min":0.996,"15min":6.258},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.52400000000000002,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.95499999999999996,"15min":1.806},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69999999999999996,"15min":0.82599999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.29199999999999998},"max":{"1min":0.90400000000000003,"5min":0.91000000000000003,"15min":6.4740000000000002},"last":0.625},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.90100000000000002},"min":{"1min":0.58199999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":6.4859999999999998},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.52500000000000002,"5min":0.442,"15min":0.40000000000000002},"max":{"1min":0.86199999999999999,"5min":0.97799999999999998,"15min":1.25},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.71299999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.99099999999999999,"15min":1.413},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.64700000000000002},"min":{"1min":0.50700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.92100000000000004,"15min":1.3280000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63700000000000001,"15min":0.65700000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94299999999999995,"15min":0.99299999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.41099999999999998,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75800000000000001,"15min":0.78300000000000003},"min":{"1min":0.56799999999999995,"5min":0.52400000000000002,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.008,"15min":1.6839999999999999},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.54100000000000004,"5min":0.44500000000000001,"15min":0.41299999999999998},"max":{"1min":0.89200000000000002,"5min":0.95699999999999996,"15min":1.0660000000000001},"last":0.442},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.73799999999999999},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.036,"5min":1.036,"15min":1.21},"last":0.65400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37056,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334784,"statfs":{"total":96636764160,"available":96598818816,"internally_reserved":0,"allocated":1957888,"data_stored":1215087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.66700000000000004,"15min":0.66000000000000003},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.32100000000000001},"max":{"1min":0.88,"5min":0.94899999999999995,"15min":4.5010000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68700000000000006,"15min":0.81200000000000006},"min":{"1min":0.46999999999999997,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.86599999999999999,"5min":0.96199999999999997,"15min":6.4059999999999997},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.44800000000000001,"5min":0.42099999999999999,"15min":0.307},"max":{"1min":0.82699999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.46999999999999997,"5min":0.46100000000000002,"15min":0.41399999999999998},"max":{"1min":0.83999999999999997,"5min":1.1970000000000001,"15min":1.546},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.80100000000000005,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.376},"max":{"1min":0.86399999999999999,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.53300000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.81399999999999995,"15min":0.73899999999999999},"min":{"1min":0.53900000000000003,"5min":0.47599999999999998,"15min":0.42199999999999999},"max":{"1min":0.89300000000000002,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71599999999999997,"15min":0.74399999999999999},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":0.98099999999999998,"5min":1.3140000000000001,"15min":4.484},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.34699999999999998},"max":{"1min":1.0229999999999999,"5min":1.1359999999999999,"15min":1.6240000000000001},"last":0.503},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66200000000000003,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.93300000000000005,"5min":1.0569999999999999,"15min":2.718},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89900000000000002,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69599999999999995,"15min":0.71599999999999997},"min":{"1min":0.49099999999999999,"5min":0.48399999999999999,"15min":0.41799999999999998},"max":{"1min":0.94099999999999995,"5min":1.246,"15min":1.669},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.76000000000000001},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.93400000000000005,"5min":1.3360000000000001,"15min":2.0459999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.748,"15min":0.77600000000000002},"min":{"1min":0.52300000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0609999999999999,"15min":2.2919999999999998},"last":0.75900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:54:30.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:54:30.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.575+0000 7f2d173ff700 1 Processor -- start 2023-07-06T03:54:30.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.575+0000 7f2d173ff700 1 -- start start 2023-07-06T03:54:30.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.576+0000 7f2d173ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2d100d2930 0x7f2d100d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.576+0000 7f2d173ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2d100b1990 0x7f2d100b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.576+0000 7f2d173ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d100d0c70 con 0x7f2d1015c980 2023-07-06T03:54:30.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.576+0000 7f2d173ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100b2310 con 0x7f2d100d2930 2023-07-06T03:54:30.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.576+0000 7f2d173ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100d0ec0 con 0x7f2d100b1990 2023-07-06T03:54:30.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.577+0000 7f2d1499a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2d100d2930 0x7f2d100d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.577+0000 7f2d1499a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2d100d2930 0x7f2d100d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34480/0 (socket says 172.21.15.97:34480) 2023-07-06T03:54:30.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.577+0000 7f2d1499a700 1 -- 172.21.15.97:0/1950912507 learned_addr learned my addr 172.21.15.97:0/1950912507 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:30.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.577+0000 7f2d1599c700 1 --2- 172.21.15.97:0/1950912507 >> v2:172.21.15.146:3300/0 conn(0x7f2d100b1990 0x7f2d100b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.577+0000 7f2d1499a700 1 -- 172.21.15.97:0/1950912507 >> v2:172.21.15.146:3300/0 conn(0x7f2d100b1990 msgr2=0x7f2d100b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:30.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.577+0000 7f2d1499a700 1 --2- 172.21.15.97:0/1950912507 >> v2:172.21.15.146:3300/0 conn(0x7f2d100b1990 0x7f2d100b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.577+0000 7f2d1499a700 1 -- 172.21.15.97:0/1950912507 >> v1:172.21.15.97:6789/0 conn(0x7f2d1015c980 legacy=0x7f2d100cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.577+0000 7f2d1499a700 1 -- 172.21.15.97:0/1950912507 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100d11a0 con 0x7f2d100d2930 2023-07-06T03:54:30.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.577+0000 7f2d07fff700 1 -- 172.21.15.97:0/1950912507 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242639544 0 0) 0x7f2d100d0c70 con 0x7f2d1015c980 2023-07-06T03:54:30.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.577+0000 7f2d1499a700 1 --2- 172.21.15.97:0/1950912507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d100d2930 0x7f2d100d1c80 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f2d1008bf90 tx=0x7f2d00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e33c1e49756ce4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:30.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.578+0000 7f2d07fff700 1 -- 172.21.15.97:0/1950912507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00011830 con 0x7f2d100d2930 2023-07-06T03:54:30.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.578+0000 7f2d07fff700 1 -- 172.21.15.97:0/1950912507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d00011990 con 0x7f2d100d2930 2023-07-06T03:54:30.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.578+0000 7f2d07fff700 1 -- 172.21.15.97:0/1950912507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00011d20 con 0x7f2d100d2930 2023-07-06T03:54:30.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.578+0000 7f2d173ff700 1 -- 172.21.15.97:0/1950912507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d100d2930 msgr2=0x7f2d100d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.578+0000 7f2d173ff700 1 --2- 172.21.15.97:0/1950912507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d100d2930 0x7f2d100d1c80 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f2d1008bf90 tx=0x7f2d00009bd0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.578+0000 7f2d173ff700 1 -- 172.21.15.97:0/1950912507 shutdown_connections 2023-07-06T03:54:30.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.578+0000 7f2d173ff700 1 --2- 172.21.15.97:0/1950912507 >> v2:172.21.15.146:3300/0 conn(0x7f2d100b1990 0x7f2d100b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.578+0000 7f2d173ff700 1 --2- 172.21.15.97:0/1950912507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d100d2930 0x7f2d100d1c80 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.578+0000 7f2d173ff700 1 -- 172.21.15.97:0/1950912507 >> 172.21.15.97:0/1950912507 conn(0x7f2d1006b2f0 msgr2=0x7f2d1006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:30.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.578+0000 7f2d173ff700 1 -- 172.21.15.97:0/1950912507 shutdown_connections 2023-07-06T03:54:30.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.578+0000 7f2d173ff700 1 -- 172.21.15.97:0/1950912507 wait complete. 2023-07-06T03:54:30.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.580+0000 7f2d173ff700 1 Processor -- start 2023-07-06T03:54:30.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.580+0000 7f2d173ff700 1 -- start start 2023-07-06T03:54:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.581+0000 7f2d173ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d100b1990 0x7f2d100b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.581+0000 7f2d173ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d100d2930 0x7f2d101f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.581+0000 7f2d173ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d1015c980 0x7f2d100ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.581+0000 7f2d173ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100c78f0 con 0x7f2d100d2930 2023-07-06T03:54:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.581+0000 7f2d173ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100c7eb0 con 0x7f2d1015c980 2023-07-06T03:54:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.581+0000 7f2d173ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100cc900 con 0x7f2d100b1990 2023-07-06T03:54:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.581+0000 7f2d1499a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d100d2930 0x7f2d101f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d1499a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d100d2930 0x7f2d101f2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34482/0 (socket says 172.21.15.97:34482) 2023-07-06T03:54:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d1499a700 1 -- 172.21.15.97:0/2489446918 learned_addr learned my addr 172.21.15.97:0/2489446918 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d1519b700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d100b1990 0x7f2d100b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d1599c700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d1015c980 0x7f2d100ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d1499a700 1 -- 172.21.15.97:0/2489446918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d100b1990 msgr2=0x7f2d100b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d1499a700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d100b1990 0x7f2d100b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d1499a700 1 -- 172.21.15.97:0/2489446918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d1015c980 msgr2=0x7f2d100ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d1499a700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d1015c980 0x7f2d100ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d1499a700 1 -- 172.21.15.97:0/2489446918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d00009550 con 0x7f2d100d2930 2023-07-06T03:54:30.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d1499a700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d100d2930 0x7f2d101f2bd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009c10 tx=0x7f2d00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:30.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d05ffb700 1 -- 172.21.15.97:0/2489446918 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00004510 con 0x7f2d100d2930 2023-07-06T03:54:30.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d05ffb700 1 -- 172.21.15.97:0/2489446918 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d000036a0 con 0x7f2d100d2930 2023-07-06T03:54:30.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d173ff700 1 -- 172.21.15.97:0/2489446918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d100ccbe0 con 0x7f2d100d2930 2023-07-06T03:54:30.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.582+0000 7f2d05ffb700 1 -- 172.21.15.97:0/2489446918 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00003a10 con 0x7f2d100d2930 2023-07-06T03:54:30.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.583+0000 7f2d173ff700 1 -- 172.21.15.97:0/2489446918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1015db70 con 0x7f2d100d2930 2023-07-06T03:54:30.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.584+0000 7f2d173ff700 1 -- 172.21.15.97:0/2489446918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1006a2d0 con 0x7f2d100d2930 2023-07-06T03:54:30.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.585+0000 7f2d05ffb700 1 -- 172.21.15.97:0/2489446918 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2d00003b70 con 0x7f2d100d2930 2023-07-06T03:54:30.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.585+0000 7f2d05ffb700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cf406ace0 0x7f2cf406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:30.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.586+0000 7f2d05ffb700 1 -- 172.21.15.97:0/2489446918 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2d00021030 con 0x7f2d100d2930 2023-07-06T03:54:30.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.586+0000 7f2d1519b700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cf406ace0 0x7f2cf406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:30.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.586+0000 7f2d1519b700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cf406ace0 0x7f2cf406d190 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000f60 tx=0x7f2d0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:30.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.590+0000 7f2d05ffb700 1 -- 172.21.15.97:0/2489446918 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2d00055bf0 con 0x7f2d100d2930 2023-07-06T03:54:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.782+0000 7f2d173ff700 1 -- 172.21.15.97:0/2489446918 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d100d1760 con 0x7f2cf406ace0 2023-07-06T03:54:30.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.790+0000 7f2d05ffb700 1 -- 172.21.15.97:0/2489446918 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311787 (secure 0 0 0) 0x7f2d0c0014e0 con 0x7f2cf406ace0 2023-07-06T03:54:30.795 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 -- 172.21.15.97:0/2489446918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cf406ace0 msgr2=0x7f2cf406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cf406ace0 0x7f2cf406d190 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000f60 tx=0x7f2d0c009040 comp rx=0 tx=0).stop 2023-07-06T03:54:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 -- 172.21.15.97:0/2489446918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d100d2930 msgr2=0x7f2d101f2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d100d2930 0x7f2d101f2bd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009c10 tx=0x7f2d00004750 comp rx=0 tx=0).stop 2023-07-06T03:54:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 -- 172.21.15.97:0/2489446918 shutdown_connections 2023-07-06T03:54:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2cf406ace0 0x7f2cf406d190 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d1015c980 0x7f2d100ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2d100b1990 0x7f2d100b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 --2- 172.21.15.97:0/2489446918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d100d2930 0x7f2d101f2bd0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 -- 172.21.15.97:0/2489446918 >> 172.21.15.97:0/2489446918 conn(0x7f2d1006b2f0 msgr2=0x7f2d10157290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 -- 172.21.15.97:0/2489446918 shutdown_connections 2023-07-06T03:54:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:30.795+0000 7f2d173ff700 1 -- 172.21.15.97:0/2489446918 wait complete. 2023-07-06T03:54:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:54:30.815 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2023-07-06T03:54:29.004322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284804,"kb_used_data":11844,"kb_used_omap":0,"kb_used_meta":272896,"kb_avail":754689916,"statfs":{"total":773094113280,"available":772802473984,"internally_reserved":0,"allocated":12128256,"data_stored":6185621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279445504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.065409"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.989525+0000","last_change":"2023-07-06T03:54:25.982519+0000","last_active":"2023-07-06T03:54:25.989525+0000","last_peered":"2023-07-06T03:54:25.989525+0000","last_clean":"2023-07-06T03:54:25.989525+0000","last_became_active":"2023-07-06T03:54:25.982111+0000","last_became_peered":"2023-07-06T03:54:25.982111+0000","last_unstale":"2023-07-06T03:54:25.989525+0000","last_undegraded":"2023-07-06T03:54:25.989525+0000","last_fullsized":"2023-07-06T03:54:25.989525+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:34:58.078557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.996598+0000","last_change":"2023-07-06T03:54:25.991355+0000","last_active":"2023-07-06T03:54:25.996598+0000","last_peered":"2023-07-06T03:54:25.996598+0000","last_clean":"2023-07-06T03:54:25.996598+0000","last_became_active":"2023-07-06T03:54:25.991021+0000","last_became_peered":"2023-07-06T03:54:25.991021+0000","last_unstale":"2023-07-06T03:54:25.996598+0000","last_undegraded":"2023-07-06T03:54:25.996598+0000","last_fullsized":"2023-07-06T03:54:25.996598+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:20:54.856278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998120+0000","last_change":"2023-07-06T03:54:25.997569+0000","last_active":"2023-07-06T03:54:25.998120+0000","last_peered":"2023-07-06T03:54:25.998120+0000","last_clean":"2023-07-06T03:54:25.998120+0000","last_became_active":"2023-07-06T03:54:25.997212+0000","last_became_peered":"2023-07-06T03:54:25.997212+0000","last_unstale":"2023-07-06T03:54:25.998120+0000","last_undegraded":"2023-07-06T03:54:25.998120+0000","last_fullsized":"2023-07-06T03:54:25.998120+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:47:56.013868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997116+0000","last_change":"2023-07-06T03:54:25.993309+0000","last_active":"2023-07-06T03:54:25.997116+0000","last_peered":"2023-07-06T03:54:25.997116+0000","last_clean":"2023-07-06T03:54:25.997116+0000","last_became_active":"2023-07-06T03:54:25.992730+0000","last_became_peered":"2023-07-06T03:54:25.992730+0000","last_unstale":"2023-07-06T03:54:25.997116+0000","last_undegraded":"2023-07-06T03:54:25.997116+0000","last_fullsized":"2023-07-06T03:54:25.997116+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:48:36.996242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.995407+0000","last_change":"2023-07-06T03:54:25.991695+0000","last_active":"2023-07-06T03:54:25.995407+0000","last_peered":"2023-07-06T03:54:25.995407+0000","last_clean":"2023-07-06T03:54:25.995407+0000","last_became_active":"2023-07-06T03:54:25.991226+0000","last_became_peered":"2023-07-06T03:54:25.991226+0000","last_unstale":"2023-07-06T03:54:25.995407+0000","last_undegraded":"2023-07-06T03:54:25.995407+0000","last_fullsized":"2023-07-06T03:54:25.995407+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:21:09.135716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.991221+0000","last_change":"2023-07-06T03:54:25.987083+0000","last_active":"2023-07-06T03:54:25.991221+0000","last_peered":"2023-07-06T03:54:25.991221+0000","last_clean":"2023-07-06T03:54:25.991221+0000","last_became_active":"2023-07-06T03:54:25.986814+0000","last_became_peered":"2023-07-06T03:54:25.986814+0000","last_unstale":"2023-07-06T03:54:25.991221+0000","last_undegraded":"2023-07-06T03:54:25.991221+0000","last_fullsized":"2023-07-06T03:54:25.991221+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:17:14.901305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998632+0000","last_change":"2023-07-06T03:54:25.995189+0000","last_active":"2023-07-06T03:54:25.998632+0000","last_peered":"2023-07-06T03:54:25.998632+0000","last_clean":"2023-07-06T03:54:25.998632+0000","last_became_active":"2023-07-06T03:54:25.994873+0000","last_became_peered":"2023-07-06T03:54:25.994873+0000","last_unstale":"2023-07-06T03:54:25.998632+0000","last_undegraded":"2023-07-06T03:54:25.998632+0000","last_fullsized":"2023-07-06T03:54:25.998632+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:40:17.141868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997758+0000","last_change":"2023-07-06T03:54:25.990602+0000","last_active":"2023-07-06T03:54:25.997758+0000","last_peered":"2023-07-06T03:54:25.997758+0000","last_clean":"2023-07-06T03:54:25.997758+0000","last_became_active":"2023-07-06T03:54:25.990333+0000","last_became_peered":"2023-07-06T03:54:25.990333+0000","last_unstale":"2023-07-06T03:54:25.997758+0000","last_undegraded":"2023-07-06T03:54:25.997758+0000","last_fullsized":"2023-07-06T03:54:25.997758+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:14:54.749612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.999536+0000","last_change":"2023-07-06T03:54:25.998902+0000","last_active":"2023-07-06T03:54:25.999536+0000","last_peered":"2023-07-06T03:54:25.999536+0000","last_clean":"2023-07-06T03:54:25.999536+0000","last_became_active":"2023-07-06T03:54:25.998620+0000","last_became_peered":"2023-07-06T03:54:25.998620+0000","last_unstale":"2023-07-06T03:54:25.999536+0000","last_undegraded":"2023-07-06T03:54:25.999536+0000","last_fullsized":"2023-07-06T03:54:25.999536+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:51:40.122894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.996615+0000","last_change":"2023-07-06T03:54:25.994841+0000","last_active":"2023-07-06T03:54:25.996615+0000","last_peered":"2023-07-06T03:54:25.996615+0000","last_clean":"2023-07-06T03:54:25.996615+0000","last_became_active":"2023-07-06T03:54:25.994550+0000","last_became_peered":"2023-07-06T03:54:25.994550+0000","last_unstale":"2023-07-06T03:54:25.996615+0000","last_undegraded":"2023-07-06T03:54:25.996615+0000","last_fullsized":"2023-07-06T03:54:25.996615+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:47:56.013868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.977499+0000","last_change":"2023-07-06T03:54:25.970538+0000","last_active":"2023-07-06T03:54:25.977499+0000","last_peered":"2023-07-06T03:54:25.977499+0000","last_clean":"2023-07-06T03:54:25.977499+0000","last_became_active":"2023-07-06T03:54:25.970212+0000","last_became_peered":"2023-07-06T03:54:25.970212+0000","last_unstale":"2023-07-06T03:54:25.977499+0000","last_undegraded":"2023-07-06T03:54:25.977499+0000","last_fullsized":"2023-07-06T03:54:25.977499+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:36:51.075925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998043+0000","last_change":"2023-07-06T03:54:25.993224+0000","last_active":"2023-07-06T03:54:25.998043+0000","last_peered":"2023-07-06T03:54:25.998043+0000","last_clean":"2023-07-06T03:54:25.998043+0000","last_became_active":"2023-07-06T03:54:25.992950+0000","last_became_peered":"2023-07-06T03:54:25.992950+0000","last_unstale":"2023-07-06T03:54:25.998043+0000","last_undegraded":"2023-07-06T03:54:25.998043+0000","last_fullsized":"2023-07-06T03:54:25.998043+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:20:52.627131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987667+0000","last_change":"2023-07-06T03:54:25.984700+0000","last_active":"2023-07-06T03:54:25.987667+0000","last_peered":"2023-07-06T03:54:25.987667+0000","last_clean":"2023-07-06T03:54:25.987667+0000","last_became_active":"2023-07-06T03:54:25.984210+0000","last_became_peered":"2023-07-06T03:54:25.984210+0000","last_unstale":"2023-07-06T03:54:25.987667+0000","last_undegraded":"2023-07-06T03:54:25.987667+0000","last_fullsized":"2023-07-06T03:54:25.987667+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:14.865749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998083+0000","last_change":"2023-07-06T03:54:25.993970+0000","last_active":"2023-07-06T03:54:25.998083+0000","last_peered":"2023-07-06T03:54:25.998083+0000","last_clean":"2023-07-06T03:54:25.998083+0000","last_became_active":"2023-07-06T03:54:25.993658+0000","last_became_peered":"2023-07-06T03:54:25.993658+0000","last_unstale":"2023-07-06T03:54:25.998083+0000","last_undegraded":"2023-07-06T03:54:25.998083+0000","last_fullsized":"2023-07-06T03:54:25.998083+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:17:47.631783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.996313+0000","last_change":"2023-07-06T03:54:25.990688+0000","last_active":"2023-07-06T03:54:25.996313+0000","last_peered":"2023-07-06T03:54:25.996313+0000","last_clean":"2023-07-06T03:54:25.996313+0000","last_became_active":"2023-07-06T03:54:25.990416+0000","last_became_peered":"2023-07-06T03:54:25.990416+0000","last_unstale":"2023-07-06T03:54:25.996313+0000","last_undegraded":"2023-07-06T03:54:25.996313+0000","last_fullsized":"2023-07-06T03:54:25.996313+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:16:50.354940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998108+0000","last_change":"2023-07-06T03:54:25.993590+0000","last_active":"2023-07-06T03:54:25.998108+0000","last_peered":"2023-07-06T03:54:25.998108+0000","last_clean":"2023-07-06T03:54:25.998108+0000","last_became_active":"2023-07-06T03:54:25.993261+0000","last_became_peered":"2023-07-06T03:54:25.993261+0000","last_unstale":"2023-07-06T03:54:25.998108+0000","last_undegraded":"2023-07-06T03:54:25.998108+0000","last_fullsized":"2023-07-06T03:54:25.998108+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:58.823441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:26.000841+0000","last_change":"2023-07-06T03:54:25.998097+0000","last_active":"2023-07-06T03:54:26.000841+0000","last_peered":"2023-07-06T03:54:26.000841+0000","last_clean":"2023-07-06T03:54:26.000841+0000","last_became_active":"2023-07-06T03:54:25.997692+0000","last_became_peered":"2023-07-06T03:54:25.997692+0000","last_unstale":"2023-07-06T03:54:26.000841+0000","last_undegraded":"2023-07-06T03:54:26.000841+0000","last_fullsized":"2023-07-06T03:54:26.000841+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:27:23.778751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998163+0000","last_change":"2023-07-06T03:54:25.995292+0000","last_active":"2023-07-06T03:54:25.998163+0000","last_peered":"2023-07-06T03:54:25.998163+0000","last_clean":"2023-07-06T03:54:25.998163+0000","last_became_active":"2023-07-06T03:54:25.994786+0000","last_became_peered":"2023-07-06T03:54:25.994786+0000","last_unstale":"2023-07-06T03:54:25.998163+0000","last_undegraded":"2023-07-06T03:54:25.998163+0000","last_fullsized":"2023-07-06T03:54:25.998163+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:51:34.065573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997697+0000","last_change":"2023-07-06T03:54:25.996732+0000","last_active":"2023-07-06T03:54:25.997697+0000","last_peered":"2023-07-06T03:54:25.997697+0000","last_clean":"2023-07-06T03:54:25.997697+0000","last_became_active":"2023-07-06T03:54:25.996395+0000","last_became_peered":"2023-07-06T03:54:25.996395+0000","last_unstale":"2023-07-06T03:54:25.997697+0000","last_undegraded":"2023-07-06T03:54:25.997697+0000","last_fullsized":"2023-07-06T03:54:25.997697+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:46:50.022959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997594+0000","last_change":"2023-07-06T03:54:25.995336+0000","last_active":"2023-07-06T03:54:25.997594+0000","last_peered":"2023-07-06T03:54:25.997594+0000","last_clean":"2023-07-06T03:54:25.997594+0000","last_became_active":"2023-07-06T03:54:25.994096+0000","last_became_peered":"2023-07-06T03:54:25.994096+0000","last_unstale":"2023-07-06T03:54:25.997594+0000","last_undegraded":"2023-07-06T03:54:25.997594+0000","last_fullsized":"2023-07-06T03:54:25.997594+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:31:38.605779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987248+0000","last_change":"2023-07-06T03:54:25.985809+0000","last_active":"2023-07-06T03:54:25.987248+0000","last_peered":"2023-07-06T03:54:25.987248+0000","last_clean":"2023-07-06T03:54:25.987248+0000","last_became_active":"2023-07-06T03:54:25.985546+0000","last_became_peered":"2023-07-06T03:54:25.985546+0000","last_unstale":"2023-07-06T03:54:25.987248+0000","last_undegraded":"2023-07-06T03:54:25.987248+0000","last_fullsized":"2023-07-06T03:54:25.987248+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:50:53.351454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998749+0000","last_change":"2023-07-06T03:54:25.995805+0000","last_active":"2023-07-06T03:54:25.998749+0000","last_peered":"2023-07-06T03:54:25.998749+0000","last_clean":"2023-07-06T03:54:25.998749+0000","last_became_active":"2023-07-06T03:54:25.995323+0000","last_became_peered":"2023-07-06T03:54:25.995323+0000","last_unstale":"2023-07-06T03:54:25.998749+0000","last_undegraded":"2023-07-06T03:54:25.998749+0000","last_fullsized":"2023-07-06T03:54:25.998749+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:47.947240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.996294+0000","last_change":"2023-07-06T03:54:25.990495+0000","last_active":"2023-07-06T03:54:25.996294+0000","last_peered":"2023-07-06T03:54:25.996294+0000","last_clean":"2023-07-06T03:54:25.996294+0000","last_became_active":"2023-07-06T03:54:25.990215+0000","last_became_peered":"2023-07-06T03:54:25.990215+0000","last_unstale":"2023-07-06T03:54:25.996294+0000","last_undegraded":"2023-07-06T03:54:25.996294+0000","last_fullsized":"2023-07-06T03:54:25.996294+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:06:42.446772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998469+0000","last_change":"2023-07-06T03:54:25.995055+0000","last_active":"2023-07-06T03:54:25.998469+0000","last_peered":"2023-07-06T03:54:25.998469+0000","last_clean":"2023-07-06T03:54:25.998469+0000","last_became_active":"2023-07-06T03:54:25.994787+0000","last_became_peered":"2023-07-06T03:54:25.994787+0000","last_unstale":"2023-07-06T03:54:25.998469+0000","last_undegraded":"2023-07-06T03:54:25.998469+0000","last_fullsized":"2023-07-06T03:54:25.998469+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:37:46.155723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.989504+0000","last_change":"2023-07-06T03:54:25.982354+0000","last_active":"2023-07-06T03:54:25.989504+0000","last_peered":"2023-07-06T03:54:25.989504+0000","last_clean":"2023-07-06T03:54:25.989504+0000","last_became_active":"2023-07-06T03:54:25.981783+0000","last_became_peered":"2023-07-06T03:54:25.981783+0000","last_unstale":"2023-07-06T03:54:25.989504+0000","last_undegraded":"2023-07-06T03:54:25.989504+0000","last_fullsized":"2023-07-06T03:54:25.989504+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:19:51.476763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.999156+0000","last_change":"2023-07-06T03:54:25.992886+0000","last_active":"2023-07-06T03:54:25.999156+0000","last_peered":"2023-07-06T03:54:25.999156+0000","last_clean":"2023-07-06T03:54:25.999156+0000","last_became_active":"2023-07-06T03:54:25.992612+0000","last_became_peered":"2023-07-06T03:54:25.992612+0000","last_unstale":"2023-07-06T03:54:25.999156+0000","last_undegraded":"2023-07-06T03:54:25.999156+0000","last_fullsized":"2023-07-06T03:54:25.999156+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:49:09.971185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.986756+0000","last_change":"2023-07-06T03:54:25.983575+0000","last_active":"2023-07-06T03:54:25.986756+0000","last_peered":"2023-07-06T03:54:25.986756+0000","last_clean":"2023-07-06T03:54:25.986756+0000","last_became_active":"2023-07-06T03:54:25.983322+0000","last_became_peered":"2023-07-06T03:54:25.983322+0000","last_unstale":"2023-07-06T03:54:25.986756+0000","last_undegraded":"2023-07-06T03:54:25.986756+0000","last_fullsized":"2023-07-06T03:54:25.986756+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:38:49.789114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997946+0000","last_change":"2023-07-06T03:54:25.992619+0000","last_active":"2023-07-06T03:54:25.997946+0000","last_peered":"2023-07-06T03:54:25.997946+0000","last_clean":"2023-07-06T03:54:25.997946+0000","last_became_active":"2023-07-06T03:54:25.992313+0000","last_became_peered":"2023-07-06T03:54:25.992313+0000","last_unstale":"2023-07-06T03:54:25.997946+0000","last_undegraded":"2023-07-06T03:54:25.997946+0000","last_fullsized":"2023-07-06T03:54:25.997946+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:56:14.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.990735+0000","last_change":"2023-07-06T03:54:25.989681+0000","last_active":"2023-07-06T03:54:25.990735+0000","last_peered":"2023-07-06T03:54:25.990735+0000","last_clean":"2023-07-06T03:54:25.990735+0000","last_became_active":"2023-07-06T03:54:25.989390+0000","last_became_peered":"2023-07-06T03:54:25.989390+0000","last_unstale":"2023-07-06T03:54:25.990735+0000","last_undegraded":"2023-07-06T03:54:25.990735+0000","last_fullsized":"2023-07-06T03:54:25.990735+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:11:09.192506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.974115+0000","last_change":"2023-07-06T03:54:25.970734+0000","last_active":"2023-07-06T03:54:25.974115+0000","last_peered":"2023-07-06T03:54:25.974115+0000","last_clean":"2023-07-06T03:54:25.974115+0000","last_became_active":"2023-07-06T03:54:25.970228+0000","last_became_peered":"2023-07-06T03:54:25.970228+0000","last_unstale":"2023-07-06T03:54:25.974115+0000","last_undegraded":"2023-07-06T03:54:25.974115+0000","last_fullsized":"2023-07-06T03:54:25.974115+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:02:01.652970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998086+0000","last_change":"2023-07-06T03:54:25.994954+0000","last_active":"2023-07-06T03:54:25.998086+0000","last_peered":"2023-07-06T03:54:25.998086+0000","last_clean":"2023-07-06T03:54:25.998086+0000","last_became_active":"2023-07-06T03:54:25.994643+0000","last_became_peered":"2023-07-06T03:54:25.994643+0000","last_unstale":"2023-07-06T03:54:25.998086+0000","last_undegraded":"2023-07-06T03:54:25.998086+0000","last_fullsized":"2023-07-06T03:54:25.998086+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:35:37.623028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.980217+0000","last_change":"2023-07-06T03:54:25.970822+0000","last_active":"2023-07-06T03:54:25.980217+0000","last_peered":"2023-07-06T03:54:25.980217+0000","last_clean":"2023-07-06T03:54:25.980217+0000","last_became_active":"2023-07-06T03:54:25.970472+0000","last_became_peered":"2023-07-06T03:54:25.970472+0000","last_unstale":"2023-07-06T03:54:25.980217+0000","last_undegraded":"2023-07-06T03:54:25.980217+0000","last_fullsized":"2023-07-06T03:54:25.980217+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:49:04.425510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.985726+0000","last_change":"2023-07-06T03:54:25.977347+0000","last_active":"2023-07-06T03:54:25.985726+0000","last_peered":"2023-07-06T03:54:25.985726+0000","last_clean":"2023-07-06T03:54:25.985726+0000","last_became_active":"2023-07-06T03:54:25.977018+0000","last_became_peered":"2023-07-06T03:54:25.977018+0000","last_unstale":"2023-07-06T03:54:25.985726+0000","last_undegraded":"2023-07-06T03:54:25.985726+0000","last_fullsized":"2023-07-06T03:54:25.985726+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:25:02.352516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"111'4","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.994355+0000","last_change":"2023-07-06T03:54:24.960709+0000","last_active":"2023-07-06T03:54:25.994355+0000","last_peered":"2023-07-06T03:54:25.994355+0000","last_clean":"2023-07-06T03:54:25.994355+0000","last_became_active":"2023-07-06T03:54:24.960417+0000","last_became_peered":"2023-07-06T03:54:24.960417+0000","last_unstale":"2023-07-06T03:54:25.994355+0000","last_undegraded":"2023-07-06T03:54:25.994355+0000","last_fullsized":"2023-07-06T03:54:25.994355+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:29:30.498239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987620+0000","last_change":"2023-07-06T03:54:25.983631+0000","last_active":"2023-07-06T03:54:25.987620+0000","last_peered":"2023-07-06T03:54:25.987620+0000","last_clean":"2023-07-06T03:54:25.987620+0000","last_became_active":"2023-07-06T03:54:25.983294+0000","last_became_peered":"2023-07-06T03:54:25.983294+0000","last_unstale":"2023-07-06T03:54:25.987620+0000","last_undegraded":"2023-07-06T03:54:25.987620+0000","last_fullsized":"2023-07-06T03:54:25.987620+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:56:05.531702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.995954+0000","last_change":"2023-07-06T03:54:24.966719+0000","last_active":"2023-07-06T03:54:25.995954+0000","last_peered":"2023-07-06T03:54:25.995954+0000","last_clean":"2023-07-06T03:54:25.995954+0000","last_became_active":"2023-07-06T03:54:24.966294+0000","last_became_peered":"2023-07-06T03:54:24.966294+0000","last_unstale":"2023-07-06T03:54:25.995954+0000","last_undegraded":"2023-07-06T03:54:25.995954+0000","last_fullsized":"2023-07-06T03:54:25.995954+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:44:34.378195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987162+0000","last_change":"2023-07-06T03:54:25.981442+0000","last_active":"2023-07-06T03:54:25.987162+0000","last_peered":"2023-07-06T03:54:25.987162+0000","last_clean":"2023-07-06T03:54:25.987162+0000","last_became_active":"2023-07-06T03:54:25.981170+0000","last_became_peered":"2023-07-06T03:54:25.981170+0000","last_unstale":"2023-07-06T03:54:25.987162+0000","last_undegraded":"2023-07-06T03:54:25.987162+0000","last_fullsized":"2023-07-06T03:54:25.987162+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:14:50.726445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.981603+0000","last_change":"2023-07-06T03:54:24.961735+0000","last_active":"2023-07-06T03:54:25.981603+0000","last_peered":"2023-07-06T03:54:25.981603+0000","last_clean":"2023-07-06T03:54:25.981603+0000","last_became_active":"2023-07-06T03:54:24.961470+0000","last_became_peered":"2023-07-06T03:54:24.961470+0000","last_unstale":"2023-07-06T03:54:25.981603+0000","last_undegraded":"2023-07-06T03:54:25.981603+0000","last_fullsized":"2023-07-06T03:54:25.981603+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:58:10.515633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.994191+0000","last_change":"2023-07-06T03:54:25.992478+0000","last_active":"2023-07-06T03:54:25.994191+0000","last_peered":"2023-07-06T03:54:25.994191+0000","last_clean":"2023-07-06T03:54:25.994191+0000","last_became_active":"2023-07-06T03:54:25.992184+0000","last_became_peered":"2023-07-06T03:54:25.992184+0000","last_unstale":"2023-07-06T03:54:25.994191+0000","last_undegraded":"2023-07-06T03:54:25.994191+0000","last_fullsized":"2023-07-06T03:54:25.994191+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:59:53.917909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.993785+0000","last_change":"2023-07-06T03:54:24.968618+0000","last_active":"2023-07-06T03:54:25.993785+0000","last_peered":"2023-07-06T03:54:25.993785+0000","last_clean":"2023-07-06T03:54:25.993785+0000","last_became_active":"2023-07-06T03:54:24.968325+0000","last_became_peered":"2023-07-06T03:54:24.968325+0000","last_unstale":"2023-07-06T03:54:25.993785+0000","last_undegraded":"2023-07-06T03:54:25.993785+0000","last_fullsized":"2023-07-06T03:54:25.993785+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:01:36.146555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.977683+0000","last_change":"2023-07-06T03:54:25.970445+0000","last_active":"2023-07-06T03:54:25.977683+0000","last_peered":"2023-07-06T03:54:25.977683+0000","last_clean":"2023-07-06T03:54:25.977683+0000","last_became_active":"2023-07-06T03:54:25.970136+0000","last_became_peered":"2023-07-06T03:54:25.970136+0000","last_unstale":"2023-07-06T03:54:25.977683+0000","last_undegraded":"2023-07-06T03:54:25.977683+0000","last_fullsized":"2023-07-06T03:54:25.977683+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:57:58.526146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.981300+0000","last_change":"2023-07-06T03:54:24.967094+0000","last_active":"2023-07-06T03:54:25.981300+0000","last_peered":"2023-07-06T03:54:25.981300+0000","last_clean":"2023-07-06T03:54:25.981300+0000","last_became_active":"2023-07-06T03:54:24.966840+0000","last_became_peered":"2023-07-06T03:54:24.966840+0000","last_unstale":"2023-07-06T03:54:25.981300+0000","last_undegraded":"2023-07-06T03:54:25.981300+0000","last_fullsized":"2023-07-06T03:54:25.981300+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:36:34.286009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.993556+0000","last_change":"2023-07-06T03:54:25.991207+0000","last_active":"2023-07-06T03:54:25.993556+0000","last_peered":"2023-07-06T03:54:25.993556+0000","last_clean":"2023-07-06T03:54:25.993556+0000","last_became_active":"2023-07-06T03:54:25.990919+0000","last_became_peered":"2023-07-06T03:54:25.990919+0000","last_unstale":"2023-07-06T03:54:25.993556+0000","last_undegraded":"2023-07-06T03:54:25.993556+0000","last_fullsized":"2023-07-06T03:54:25.993556+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:59:02.039814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.990048+0000","last_change":"2023-07-06T03:54:24.968210+0000","last_active":"2023-07-06T03:54:25.990048+0000","last_peered":"2023-07-06T03:54:25.990048+0000","last_clean":"2023-07-06T03:54:25.990048+0000","last_became_active":"2023-07-06T03:54:24.967928+0000","last_became_peered":"2023-07-06T03:54:24.967928+0000","last_unstale":"2023-07-06T03:54:25.990048+0000","last_undegraded":"2023-07-06T03:54:25.990048+0000","last_fullsized":"2023-07-06T03:54:25.990048+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:11:31.759804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.988261+0000","last_change":"2023-07-06T03:54:25.980651+0000","last_active":"2023-07-06T03:54:25.988261+0000","last_peered":"2023-07-06T03:54:25.988261+0000","last_clean":"2023-07-06T03:54:25.988261+0000","last_became_active":"2023-07-06T03:54:25.980072+0000","last_became_peered":"2023-07-06T03:54:25.980072+0000","last_unstale":"2023-07-06T03:54:25.988261+0000","last_undegraded":"2023-07-06T03:54:25.988261+0000","last_fullsized":"2023-07-06T03:54:25.988261+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:03:50.931934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.963743+0000","last_change":"2023-07-06T03:54:24.964106+0000","last_active":"2023-07-06T03:54:25.963743+0000","last_peered":"2023-07-06T03:54:25.963743+0000","last_clean":"2023-07-06T03:54:25.963743+0000","last_became_active":"2023-07-06T03:54:24.963813+0000","last_became_peered":"2023-07-06T03:54:24.963813+0000","last_unstale":"2023-07-06T03:54:25.963743+0000","last_undegraded":"2023-07-06T03:54:25.963743+0000","last_fullsized":"2023-07-06T03:54:25.963743+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:34:56.688555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.983436+0000","last_change":"2023-07-06T03:54:25.977287+0000","last_active":"2023-07-06T03:54:25.983436+0000","last_peered":"2023-07-06T03:54:25.983436+0000","last_clean":"2023-07-06T03:54:25.983436+0000","last_became_active":"2023-07-06T03:54:25.976934+0000","last_became_peered":"2023-07-06T03:54:25.976934+0000","last_unstale":"2023-07-06T03:54:25.983436+0000","last_undegraded":"2023-07-06T03:54:25.983436+0000","last_fullsized":"2023-07-06T03:54:25.983436+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:27:51.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964317+0000","last_change":"2023-07-06T03:54:24.959348+0000","last_active":"2023-07-06T03:54:25.964317+0000","last_peered":"2023-07-06T03:54:25.964317+0000","last_clean":"2023-07-06T03:54:25.964317+0000","last_became_active":"2023-07-06T03:54:24.958739+0000","last_became_peered":"2023-07-06T03:54:24.958739+0000","last_unstale":"2023-07-06T03:54:25.964317+0000","last_undegraded":"2023-07-06T03:54:25.964317+0000","last_fullsized":"2023-07-06T03:54:25.964317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:17:13.892908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.993829+0000","last_change":"2023-07-06T03:54:25.989190+0000","last_active":"2023-07-06T03:54:25.993829+0000","last_peered":"2023-07-06T03:54:25.993829+0000","last_clean":"2023-07-06T03:54:25.993829+0000","last_became_active":"2023-07-06T03:54:25.988768+0000","last_became_peered":"2023-07-06T03:54:25.988768+0000","last_unstale":"2023-07-06T03:54:25.993829+0000","last_undegraded":"2023-07-06T03:54:25.993829+0000","last_fullsized":"2023-07-06T03:54:25.993829+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:38:40.562890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997498+0000","last_change":"2023-07-06T03:54:24.966629+0000","last_active":"2023-07-06T03:54:25.997498+0000","last_peered":"2023-07-06T03:54:25.997498+0000","last_clean":"2023-07-06T03:54:25.997498+0000","last_became_active":"2023-07-06T03:54:24.966254+0000","last_became_peered":"2023-07-06T03:54:24.966254+0000","last_unstale":"2023-07-06T03:54:25.997498+0000","last_undegraded":"2023-07-06T03:54:25.997498+0000","last_fullsized":"2023-07-06T03:54:25.997498+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:42:51.264374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.991272+0000","last_change":"2023-07-06T03:54:25.985957+0000","last_active":"2023-07-06T03:54:25.991272+0000","last_peered":"2023-07-06T03:54:25.991272+0000","last_clean":"2023-07-06T03:54:25.991272+0000","last_became_active":"2023-07-06T03:54:25.985688+0000","last_became_peered":"2023-07-06T03:54:25.985688+0000","last_unstale":"2023-07-06T03:54:25.991272+0000","last_undegraded":"2023-07-06T03:54:25.991272+0000","last_fullsized":"2023-07-06T03:54:25.991272+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:45:52.736915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997209+0000","last_change":"2023-07-06T03:54:24.963213+0000","last_active":"2023-07-06T03:54:25.997209+0000","last_peered":"2023-07-06T03:54:25.997209+0000","last_clean":"2023-07-06T03:54:25.997209+0000","last_became_active":"2023-07-06T03:54:24.962771+0000","last_became_peered":"2023-07-06T03:54:24.962771+0000","last_unstale":"2023-07-06T03:54:25.997209+0000","last_undegraded":"2023-07-06T03:54:25.997209+0000","last_fullsized":"2023-07-06T03:54:25.997209+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:43:14.831709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.996578+0000","last_change":"2023-07-06T03:54:25.991263+0000","last_active":"2023-07-06T03:54:25.996578+0000","last_peered":"2023-07-06T03:54:25.996578+0000","last_clean":"2023-07-06T03:54:25.996578+0000","last_became_active":"2023-07-06T03:54:25.990981+0000","last_became_peered":"2023-07-06T03:54:25.990981+0000","last_unstale":"2023-07-06T03:54:25.996578+0000","last_undegraded":"2023-07-06T03:54:25.996578+0000","last_fullsized":"2023-07-06T03:54:25.996578+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:03:50.931934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"111'1","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.988023+0000","last_change":"2023-07-06T03:54:24.960832+0000","last_active":"2023-07-06T03:54:25.988023+0000","last_peered":"2023-07-06T03:54:25.988023+0000","last_clean":"2023-07-06T03:54:25.988023+0000","last_became_active":"2023-07-06T03:54:24.960553+0000","last_became_peered":"2023-07-06T03:54:24.960553+0000","last_unstale":"2023-07-06T03:54:25.988023+0000","last_undegraded":"2023-07-06T03:54:25.988023+0000","last_fullsized":"2023-07-06T03:54:25.988023+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:55:32.535767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.986816+0000","last_change":"2023-07-06T03:54:25.982232+0000","last_active":"2023-07-06T03:54:25.986816+0000","last_peered":"2023-07-06T03:54:25.986816+0000","last_clean":"2023-07-06T03:54:25.986816+0000","last_became_active":"2023-07-06T03:54:25.981957+0000","last_became_peered":"2023-07-06T03:54:25.981957+0000","last_unstale":"2023-07-06T03:54:25.986816+0000","last_undegraded":"2023-07-06T03:54:25.986816+0000","last_fullsized":"2023-07-06T03:54:25.986816+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:21:14.625844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.963328+0000","last_change":"2023-07-06T03:54:24.968500+0000","last_active":"2023-07-06T03:54:25.963328+0000","last_peered":"2023-07-06T03:54:25.963328+0000","last_clean":"2023-07-06T03:54:25.963328+0000","last_became_active":"2023-07-06T03:54:24.968254+0000","last_became_peered":"2023-07-06T03:54:24.968254+0000","last_unstale":"2023-07-06T03:54:25.963328+0000","last_undegraded":"2023-07-06T03:54:25.963328+0000","last_fullsized":"2023-07-06T03:54:25.963328+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:07:50.951010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987360+0000","last_change":"2023-07-06T03:54:25.984331+0000","last_active":"2023-07-06T03:54:25.987360+0000","last_peered":"2023-07-06T03:54:25.987360+0000","last_clean":"2023-07-06T03:54:25.987360+0000","last_became_active":"2023-07-06T03:54:25.984045+0000","last_became_peered":"2023-07-06T03:54:25.984045+0000","last_unstale":"2023-07-06T03:54:25.987360+0000","last_undegraded":"2023-07-06T03:54:25.987360+0000","last_fullsized":"2023-07-06T03:54:25.987360+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:12:12.357992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.994878+0000","last_change":"2023-07-06T03:54:24.963620+0000","last_active":"2023-07-06T03:54:25.994878+0000","last_peered":"2023-07-06T03:54:25.994878+0000","last_clean":"2023-07-06T03:54:25.994878+0000","last_became_active":"2023-07-06T03:54:24.963272+0000","last_became_peered":"2023-07-06T03:54:24.963272+0000","last_unstale":"2023-07-06T03:54:25.994878+0000","last_undegraded":"2023-07-06T03:54:25.994878+0000","last_fullsized":"2023-07-06T03:54:25.994878+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:23.966416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.997995+0000","last_change":"2023-07-06T03:54:25.992970+0000","last_active":"2023-07-06T03:54:25.997995+0000","last_peered":"2023-07-06T03:54:25.997995+0000","last_clean":"2023-07-06T03:54:25.997995+0000","last_became_active":"2023-07-06T03:54:25.992697+0000","last_became_peered":"2023-07-06T03:54:25.992697+0000","last_unstale":"2023-07-06T03:54:25.997995+0000","last_undegraded":"2023-07-06T03:54:25.997995+0000","last_fullsized":"2023-07-06T03:54:25.997995+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:01:43.196893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.967959+0000","last_change":"2023-07-06T03:54:24.952178+0000","last_active":"2023-07-06T03:54:25.967959+0000","last_peered":"2023-07-06T03:54:25.967959+0000","last_clean":"2023-07-06T03:54:25.967959+0000","last_became_active":"2023-07-06T03:54:24.951909+0000","last_became_peered":"2023-07-06T03:54:24.951909+0000","last_unstale":"2023-07-06T03:54:25.967959+0000","last_undegraded":"2023-07-06T03:54:25.967959+0000","last_fullsized":"2023-07-06T03:54:25.967959+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:53:18.590181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.982290+0000","last_change":"2023-07-06T03:54:25.979010+0000","last_active":"2023-07-06T03:54:25.982290+0000","last_peered":"2023-07-06T03:54:25.982290+0000","last_clean":"2023-07-06T03:54:25.982290+0000","last_became_active":"2023-07-06T03:54:25.978750+0000","last_became_peered":"2023-07-06T03:54:25.978750+0000","last_unstale":"2023-07-06T03:54:25.982290+0000","last_undegraded":"2023-07-06T03:54:25.982290+0000","last_fullsized":"2023-07-06T03:54:25.982290+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:20:54.856278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964437+0000","last_change":"2023-07-06T03:54:24.959070+0000","last_active":"2023-07-06T03:54:25.964437+0000","last_peered":"2023-07-06T03:54:25.964437+0000","last_clean":"2023-07-06T03:54:25.964437+0000","last_became_active":"2023-07-06T03:54:24.958759+0000","last_became_peered":"2023-07-06T03:54:24.958759+0000","last_unstale":"2023-07-06T03:54:25.964437+0000","last_undegraded":"2023-07-06T03:54:25.964437+0000","last_fullsized":"2023-07-06T03:54:25.964437+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:58:13.857352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.989308+0000","last_change":"2023-07-06T03:54:25.986506+0000","last_active":"2023-07-06T03:54:25.989308+0000","last_peered":"2023-07-06T03:54:25.989308+0000","last_clean":"2023-07-06T03:54:25.989308+0000","last_became_active":"2023-07-06T03:54:25.986023+0000","last_became_peered":"2023-07-06T03:54:25.986023+0000","last_unstale":"2023-07-06T03:54:25.989308+0000","last_undegraded":"2023-07-06T03:54:25.989308+0000","last_fullsized":"2023-07-06T03:54:25.989308+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:25:26.978184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.971868+0000","last_change":"2023-07-06T03:54:24.961481+0000","last_active":"2023-07-06T03:54:25.971868+0000","last_peered":"2023-07-06T03:54:25.971868+0000","last_clean":"2023-07-06T03:54:25.971868+0000","last_became_active":"2023-07-06T03:54:24.961231+0000","last_became_peered":"2023-07-06T03:54:24.961231+0000","last_unstale":"2023-07-06T03:54:25.971868+0000","last_undegraded":"2023-07-06T03:54:25.971868+0000","last_fullsized":"2023-07-06T03:54:25.971868+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:50:51.592794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:26.000318+0000","last_change":"2023-07-06T03:54:25.995711+0000","last_active":"2023-07-06T03:54:26.000318+0000","last_peered":"2023-07-06T03:54:26.000318+0000","last_clean":"2023-07-06T03:54:26.000318+0000","last_became_active":"2023-07-06T03:54:25.995438+0000","last_became_peered":"2023-07-06T03:54:25.995438+0000","last_unstale":"2023-07-06T03:54:26.000318+0000","last_undegraded":"2023-07-06T03:54:26.000318+0000","last_fullsized":"2023-07-06T03:54:26.000318+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:30:08.592198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.994766+0000","last_change":"2023-07-06T03:54:24.961955+0000","last_active":"2023-07-06T03:54:25.994766+0000","last_peered":"2023-07-06T03:54:25.994766+0000","last_clean":"2023-07-06T03:54:25.994766+0000","last_became_active":"2023-07-06T03:54:24.961661+0000","last_became_peered":"2023-07-06T03:54:24.961661+0000","last_unstale":"2023-07-06T03:54:25.994766+0000","last_undegraded":"2023-07-06T03:54:25.994766+0000","last_fullsized":"2023-07-06T03:54:25.994766+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:24:21.072667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.989086+0000","last_change":"2023-07-06T03:54:25.981562+0000","last_active":"2023-07-06T03:54:25.989086+0000","last_peered":"2023-07-06T03:54:25.989086+0000","last_clean":"2023-07-06T03:54:25.989086+0000","last_became_active":"2023-07-06T03:54:25.981273+0000","last_became_peered":"2023-07-06T03:54:25.981273+0000","last_unstale":"2023-07-06T03:54:25.989086+0000","last_undegraded":"2023-07-06T03:54:25.989086+0000","last_fullsized":"2023-07-06T03:54:25.989086+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:40:17.141868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.972199+0000","last_change":"2023-07-06T03:54:24.966105+0000","last_active":"2023-07-06T03:54:25.972199+0000","last_peered":"2023-07-06T03:54:25.972199+0000","last_clean":"2023-07-06T03:54:25.972199+0000","last_became_active":"2023-07-06T03:54:24.965741+0000","last_became_peered":"2023-07-06T03:54:24.965741+0000","last_unstale":"2023-07-06T03:54:25.972199+0000","last_undegraded":"2023-07-06T03:54:25.972199+0000","last_fullsized":"2023-07-06T03:54:25.972199+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:58:24.843850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.993579+0000","last_change":"2023-07-06T03:54:25.990457+0000","last_active":"2023-07-06T03:54:25.993579+0000","last_peered":"2023-07-06T03:54:25.993579+0000","last_clean":"2023-07-06T03:54:25.993579+0000","last_became_active":"2023-07-06T03:54:25.990185+0000","last_became_peered":"2023-07-06T03:54:25.990185+0000","last_unstale":"2023-07-06T03:54:25.993579+0000","last_undegraded":"2023-07-06T03:54:25.993579+0000","last_fullsized":"2023-07-06T03:54:25.993579+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:35:11.135492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.973269+0000","last_change":"2023-07-06T03:54:24.968361+0000","last_active":"2023-07-06T03:54:25.973269+0000","last_peered":"2023-07-06T03:54:25.973269+0000","last_clean":"2023-07-06T03:54:25.973269+0000","last_became_active":"2023-07-06T03:54:24.968079+0000","last_became_peered":"2023-07-06T03:54:24.968079+0000","last_unstale":"2023-07-06T03:54:25.973269+0000","last_undegraded":"2023-07-06T03:54:25.973269+0000","last_fullsized":"2023-07-06T03:54:25.973269+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:31:15.895823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.977635+0000","last_change":"2023-07-06T03:54:25.972893+0000","last_active":"2023-07-06T03:54:25.977635+0000","last_peered":"2023-07-06T03:54:25.977635+0000","last_clean":"2023-07-06T03:54:25.977635+0000","last_became_active":"2023-07-06T03:54:25.972615+0000","last_became_peered":"2023-07-06T03:54:25.972615+0000","last_unstale":"2023-07-06T03:54:25.977635+0000","last_undegraded":"2023-07-06T03:54:25.977635+0000","last_fullsized":"2023-07-06T03:54:25.977635+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:08.012638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.985130+0000","last_change":"2023-07-06T03:54:24.967367+0000","last_active":"2023-07-06T03:54:25.985130+0000","last_peered":"2023-07-06T03:54:25.985130+0000","last_clean":"2023-07-06T03:54:25.985130+0000","last_became_active":"2023-07-06T03:54:24.966941+0000","last_became_peered":"2023-07-06T03:54:24.966941+0000","last_unstale":"2023-07-06T03:54:25.985130+0000","last_undegraded":"2023-07-06T03:54:25.985130+0000","last_fullsized":"2023-07-06T03:54:25.985130+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:11:23.437909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998131+0000","last_change":"2023-07-06T03:54:25.994315+0000","last_active":"2023-07-06T03:54:25.998131+0000","last_peered":"2023-07-06T03:54:25.998131+0000","last_clean":"2023-07-06T03:54:25.998131+0000","last_became_active":"2023-07-06T03:54:25.994034+0000","last_became_peered":"2023-07-06T03:54:25.994034+0000","last_unstale":"2023-07-06T03:54:25.998131+0000","last_undegraded":"2023-07-06T03:54:25.998131+0000","last_fullsized":"2023-07-06T03:54:25.998131+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:10:00.309903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.972550+0000","last_change":"2023-07-06T03:54:24.963634+0000","last_active":"2023-07-06T03:54:25.972550+0000","last_peered":"2023-07-06T03:54:25.972550+0000","last_clean":"2023-07-06T03:54:25.972550+0000","last_became_active":"2023-07-06T03:54:24.963383+0000","last_became_peered":"2023-07-06T03:54:24.963383+0000","last_unstale":"2023-07-06T03:54:25.972550+0000","last_undegraded":"2023-07-06T03:54:25.972550+0000","last_fullsized":"2023-07-06T03:54:25.972550+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:49.510799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.990858+0000","last_change":"2023-07-06T03:54:25.985942+0000","last_active":"2023-07-06T03:54:25.990858+0000","last_peered":"2023-07-06T03:54:25.990858+0000","last_clean":"2023-07-06T03:54:25.990858+0000","last_became_active":"2023-07-06T03:54:25.985555+0000","last_became_peered":"2023-07-06T03:54:25.985555+0000","last_unstale":"2023-07-06T03:54:25.990858+0000","last_undegraded":"2023-07-06T03:54:25.990858+0000","last_fullsized":"2023-07-06T03:54:25.990858+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:43:08.012638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.968110+0000","last_change":"2023-07-06T03:54:24.958887+0000","last_active":"2023-07-06T03:54:25.968110+0000","last_peered":"2023-07-06T03:54:25.968110+0000","last_clean":"2023-07-06T03:54:25.968110+0000","last_became_active":"2023-07-06T03:54:24.958620+0000","last_became_peered":"2023-07-06T03:54:24.958620+0000","last_unstale":"2023-07-06T03:54:25.968110+0000","last_undegraded":"2023-07-06T03:54:25.968110+0000","last_fullsized":"2023-07-06T03:54:25.968110+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:48:47.700659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987045+0000","last_change":"2023-07-06T03:54:25.983250+0000","last_active":"2023-07-06T03:54:25.987045+0000","last_peered":"2023-07-06T03:54:25.987045+0000","last_clean":"2023-07-06T03:54:25.987045+0000","last_became_active":"2023-07-06T03:54:25.982954+0000","last_became_peered":"2023-07-06T03:54:25.982954+0000","last_unstale":"2023-07-06T03:54:25.987045+0000","last_undegraded":"2023-07-06T03:54:25.987045+0000","last_fullsized":"2023-07-06T03:54:25.987045+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:37:35.751863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.982021+0000","last_change":"2023-07-06T03:54:24.961554+0000","last_active":"2023-07-06T03:54:25.982021+0000","last_peered":"2023-07-06T03:54:25.982021+0000","last_clean":"2023-07-06T03:54:25.982021+0000","last_became_active":"2023-07-06T03:54:24.961221+0000","last_became_peered":"2023-07-06T03:54:24.961221+0000","last_unstale":"2023-07-06T03:54:25.982021+0000","last_undegraded":"2023-07-06T03:54:25.982021+0000","last_fullsized":"2023-07-06T03:54:25.982021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:11:08.184109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.994352+0000","last_change":"2023-07-06T03:54:25.989922+0000","last_active":"2023-07-06T03:54:25.994352+0000","last_peered":"2023-07-06T03:54:25.994352+0000","last_clean":"2023-07-06T03:54:25.994352+0000","last_became_active":"2023-07-06T03:54:25.989588+0000","last_became_peered":"2023-07-06T03:54:25.989588+0000","last_unstale":"2023-07-06T03:54:25.994352+0000","last_undegraded":"2023-07-06T03:54:25.994352+0000","last_fullsized":"2023-07-06T03:54:25.994352+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:09:31.465372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.972211+0000","last_change":"2023-07-06T03:54:24.957392+0000","last_active":"2023-07-06T03:54:25.972211+0000","last_peered":"2023-07-06T03:54:25.972211+0000","last_clean":"2023-07-06T03:54:25.972211+0000","last_became_active":"2023-07-06T03:54:24.957119+0000","last_became_peered":"2023-07-06T03:54:24.957119+0000","last_unstale":"2023-07-06T03:54:25.972211+0000","last_undegraded":"2023-07-06T03:54:25.972211+0000","last_fullsized":"2023-07-06T03:54:25.972211+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:09:36.909936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.984427+0000","last_change":"2023-07-06T03:54:25.981315+0000","last_active":"2023-07-06T03:54:25.984427+0000","last_peered":"2023-07-06T03:54:25.984427+0000","last_clean":"2023-07-06T03:54:25.984427+0000","last_became_active":"2023-07-06T03:54:25.981019+0000","last_became_peered":"2023-07-06T03:54:25.981019+0000","last_unstale":"2023-07-06T03:54:25.984427+0000","last_undegraded":"2023-07-06T03:54:25.984427+0000","last_fullsized":"2023-07-06T03:54:25.984427+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:46:36.047228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.977617+0000","last_change":"2023-07-06T03:54:24.960376+0000","last_active":"2023-07-06T03:54:25.977617+0000","last_peered":"2023-07-06T03:54:25.977617+0000","last_clean":"2023-07-06T03:54:25.977617+0000","last_became_active":"2023-07-06T03:54:24.960090+0000","last_became_peered":"2023-07-06T03:54:24.960090+0000","last_unstale":"2023-07-06T03:54:25.977617+0000","last_undegraded":"2023-07-06T03:54:25.977617+0000","last_fullsized":"2023-07-06T03:54:25.977617+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:12:50.172275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":173,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.974356+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:54:25.974356+0000","last_peered":"2023-07-06T03:54:25.974356+0000","last_clean":"2023-07-06T03:54:25.974356+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:54:25.974356+0000","last_undegraded":"2023-07-06T03:54:25.974356+0000","last_fullsized":"2023-07-06T03:54:25.974356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:43:21.029393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.998125+0000","last_change":"2023-07-06T03:54:25.995178+0000","last_active":"2023-07-06T03:54:25.998125+0000","last_peered":"2023-07-06T03:54:25.998125+0000","last_clean":"2023-07-06T03:54:25.998125+0000","last_became_active":"2023-07-06T03:54:25.994895+0000","last_became_peered":"2023-07-06T03:54:25.994895+0000","last_unstale":"2023-07-06T03:54:25.998125+0000","last_undegraded":"2023-07-06T03:54:25.998125+0000","last_fullsized":"2023-07-06T03:54:25.998125+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:21:35.838671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964590+0000","last_change":"2023-07-06T03:54:24.960620+0000","last_active":"2023-07-06T03:54:25.964590+0000","last_peered":"2023-07-06T03:54:25.964590+0000","last_clean":"2023-07-06T03:54:25.964590+0000","last_became_active":"2023-07-06T03:54:24.958294+0000","last_became_peered":"2023-07-06T03:54:24.958294+0000","last_unstale":"2023-07-06T03:54:25.964590+0000","last_undegraded":"2023-07-06T03:54:25.964590+0000","last_fullsized":"2023-07-06T03:54:25.964590+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:46:43.436206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":185,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.963405+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:54:25.963405+0000","last_peered":"2023-07-06T03:54:25.963405+0000","last_clean":"2023-07-06T03:54:25.963405+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:54:25.963405+0000","last_undegraded":"2023-07-06T03:54:25.963405+0000","last_fullsized":"2023-07-06T03:54:25.963405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:59:25.373090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.991268+0000","last_change":"2023-07-06T03:54:25.988190+0000","last_active":"2023-07-06T03:54:25.991268+0000","last_peered":"2023-07-06T03:54:25.991268+0000","last_clean":"2023-07-06T03:54:25.991268+0000","last_became_active":"2023-07-06T03:54:25.987907+0000","last_became_peered":"2023-07-06T03:54:25.987907+0000","last_unstale":"2023-07-06T03:54:25.991268+0000","last_undegraded":"2023-07-06T03:54:25.991268+0000","last_fullsized":"2023-07-06T03:54:25.991268+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:02:01.652970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.965658+0000","last_change":"2023-07-06T03:54:24.960738+0000","last_active":"2023-07-06T03:54:25.965658+0000","last_peered":"2023-07-06T03:54:25.965658+0000","last_clean":"2023-07-06T03:54:25.965658+0000","last_became_active":"2023-07-06T03:54:24.960472+0000","last_became_peered":"2023-07-06T03:54:24.960472+0000","last_unstale":"2023-07-06T03:54:25.965658+0000","last_undegraded":"2023-07-06T03:54:25.965658+0000","last_fullsized":"2023-07-06T03:54:25.965658+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:55:51.507033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":189,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964539+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:54:25.964539+0000","last_peered":"2023-07-06T03:54:25.964539+0000","last_clean":"2023-07-06T03:54:25.964539+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:54:25.964539+0000","last_undegraded":"2023-07-06T03:54:25.964539+0000","last_fullsized":"2023-07-06T03:54:25.964539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:02:51.535276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987230+0000","last_change":"2023-07-06T03:54:25.984232+0000","last_active":"2023-07-06T03:54:25.987230+0000","last_peered":"2023-07-06T03:54:25.987230+0000","last_clean":"2023-07-06T03:54:25.987230+0000","last_became_active":"2023-07-06T03:54:25.983927+0000","last_became_peered":"2023-07-06T03:54:25.983927+0000","last_unstale":"2023-07-06T03:54:25.987230+0000","last_undegraded":"2023-07-06T03:54:25.987230+0000","last_fullsized":"2023-07-06T03:54:25.987230+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:00:05.139216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"111'8","reported_seq":19,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.984046+0000","last_change":"2023-07-06T03:54:24.960132+0000","last_active":"2023-07-06T03:54:25.984046+0000","last_peered":"2023-07-06T03:54:25.984046+0000","last_clean":"2023-07-06T03:54:25.984046+0000","last_became_active":"2023-07-06T03:54:24.959848+0000","last_became_peered":"2023-07-06T03:54:24.959848+0000","last_unstale":"2023-07-06T03:54:25.984046+0000","last_undegraded":"2023-07-06T03:54:25.984046+0000","last_fullsized":"2023-07-06T03:54:25.984046+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:32:46.189015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":185,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964100+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:54:25.964100+0000","last_peered":"2023-07-06T03:54:25.964100+0000","last_clean":"2023-07-06T03:54:25.964100+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:54:25.964100+0000","last_undegraded":"2023-07-06T03:54:25.964100+0000","last_fullsized":"2023-07-06T03:54:25.964100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:30:27.500176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":325,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.967066+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:54:25.967066+0000","last_peered":"2023-07-06T03:54:25.967066+0000","last_clean":"2023-07-06T03:54:25.967066+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:54:25.967066+0000","last_undegraded":"2023-07-06T03:54:25.967066+0000","last_fullsized":"2023-07-06T03:54:25.967066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:58:32.498146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.983007+0000","last_change":"2023-07-06T03:54:25.978797+0000","last_active":"2023-07-06T03:54:25.983007+0000","last_peered":"2023-07-06T03:54:25.983007+0000","last_clean":"2023-07-06T03:54:25.983007+0000","last_became_active":"2023-07-06T03:54:25.978509+0000","last_became_peered":"2023-07-06T03:54:25.978509+0000","last_unstale":"2023-07-06T03:54:25.983007+0000","last_undegraded":"2023-07-06T03:54:25.983007+0000","last_fullsized":"2023-07-06T03:54:25.983007+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:06:16.468463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"111'2","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.980551+0000","last_change":"2023-07-06T03:54:24.962985+0000","last_active":"2023-07-06T03:54:25.980551+0000","last_peered":"2023-07-06T03:54:25.980551+0000","last_clean":"2023-07-06T03:54:25.980551+0000","last_became_active":"2023-07-06T03:54:24.962602+0000","last_became_peered":"2023-07-06T03:54:24.962602+0000","last_unstale":"2023-07-06T03:54:25.980551+0000","last_undegraded":"2023-07-06T03:54:25.980551+0000","last_fullsized":"2023-07-06T03:54:25.980551+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:03:25.070510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":185,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.964023+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:54:25.964023+0000","last_peered":"2023-07-06T03:54:25.964023+0000","last_clean":"2023-07-06T03:54:25.964023+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:54:25.964023+0000","last_undegraded":"2023-07-06T03:54:25.964023+0000","last_fullsized":"2023-07-06T03:54:25.964023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:42:47.254337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.993433+0000","last_change":"2023-07-06T03:54:25.989854+0000","last_active":"2023-07-06T03:54:25.993433+0000","last_peered":"2023-07-06T03:54:25.993433+0000","last_clean":"2023-07-06T03:54:25.993433+0000","last_became_active":"2023-07-06T03:54:25.989547+0000","last_became_peered":"2023-07-06T03:54:25.989547+0000","last_unstale":"2023-07-06T03:54:25.993433+0000","last_undegraded":"2023-07-06T03:54:25.993433+0000","last_fullsized":"2023-07-06T03:54:25.993433+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:54.414464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.976519+0000","last_change":"2023-07-06T03:54:24.966446+0000","last_active":"2023-07-06T03:54:25.976519+0000","last_peered":"2023-07-06T03:54:25.976519+0000","last_clean":"2023-07-06T03:54:25.976519+0000","last_became_active":"2023-07-06T03:54:24.966105+0000","last_became_peered":"2023-07-06T03:54:24.966105+0000","last_unstale":"2023-07-06T03:54:25.976519+0000","last_undegraded":"2023-07-06T03:54:25.976519+0000","last_fullsized":"2023-07-06T03:54:25.976519+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:06:39.910267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":185,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.963667+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:54:25.963667+0000","last_peered":"2023-07-06T03:54:25.963667+0000","last_clean":"2023-07-06T03:54:25.963667+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:54:25.963667+0000","last_undegraded":"2023-07-06T03:54:25.963667+0000","last_fullsized":"2023-07-06T03:54:25.963667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:44:51.517874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.973688+0000","last_change":"2023-07-06T03:54:25.971739+0000","last_active":"2023-07-06T03:54:25.973688+0000","last_peered":"2023-07-06T03:54:25.973688+0000","last_clean":"2023-07-06T03:54:25.973688+0000","last_became_active":"2023-07-06T03:54:25.971403+0000","last_became_peered":"2023-07-06T03:54:25.971403+0000","last_unstale":"2023-07-06T03:54:25.973688+0000","last_undegraded":"2023-07-06T03:54:25.973688+0000","last_fullsized":"2023-07-06T03:54:25.973688+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:02:43.079625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"111'3","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.984012+0000","last_change":"2023-07-06T03:54:24.965857+0000","last_active":"2023-07-06T03:54:25.984012+0000","last_peered":"2023-07-06T03:54:25.984012+0000","last_clean":"2023-07-06T03:54:25.984012+0000","last_became_active":"2023-07-06T03:54:24.965573+0000","last_became_peered":"2023-07-06T03:54:24.965573+0000","last_unstale":"2023-07-06T03:54:25.984012+0000","last_undegraded":"2023-07-06T03:54:25.984012+0000","last_fullsized":"2023-07-06T03:54:25.984012+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:43:57.859083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":177,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.966250+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:54:25.966250+0000","last_peered":"2023-07-06T03:54:25.966250+0000","last_clean":"2023-07-06T03:54:25.966250+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:54:25.966250+0000","last_undegraded":"2023-07-06T03:54:25.966250+0000","last_fullsized":"2023-07-06T03:54:25.966250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:47.710521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.987337+0000","last_change":"2023-07-06T03:54:25.986326+0000","last_active":"2023-07-06T03:54:25.987337+0000","last_peered":"2023-07-06T03:54:25.987337+0000","last_clean":"2023-07-06T03:54:25.987337+0000","last_became_active":"2023-07-06T03:54:25.986079+0000","last_became_peered":"2023-07-06T03:54:25.986079+0000","last_unstale":"2023-07-06T03:54:25.987337+0000","last_undegraded":"2023-07-06T03:54:25.987337+0000","last_fullsized":"2023-07-06T03:54:25.987337+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:24.936322+0000","last_clean_scrub_stamp":"2023-07-06T03:54:24.936322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:11:44.388360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"111'6","reported_seq":17,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:26.000501+0000","last_change":"2023-07-06T03:54:24.965263+0000","last_active":"2023-07-06T03:54:26.000501+0000","last_peered":"2023-07-06T03:54:26.000501+0000","last_clean":"2023-07-06T03:54:26.000501+0000","last_became_active":"2023-07-06T03:54:24.964956+0000","last_became_peered":"2023-07-06T03:54:24.964956+0000","last_unstale":"2023-07-06T03:54:26.000501+0000","last_undegraded":"2023-07-06T03:54:26.000501+0000","last_fullsized":"2023-07-06T03:54:26.000501+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:54:23.927925+0000","last_clean_scrub_stamp":"2023-07-06T03:54:23.927925+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:13:17.999257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":185,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-07-06T03:54:25.963425+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:54:25.963425+0000","last_peered":"2023-07-06T03:54:25.963425+0000","last_clean":"2023-07-06T03:54:25.963425+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:54:25.963425+0000","last_undegraded":"2023-07-06T03:54:25.963425+0000","last_fullsized":"2023-07-06T03:54:25.963425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:23:20.454826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68600000000000005,"15min":0.70499999999999996},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.88300000000000001,"5min":1.0620000000000001,"15min":4.0570000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67900000000000005,"15min":0.752},"min":{"1min":0.39900000000000002,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.85499999999999998,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.499}]},{"osd":1,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68300000000000005,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.45000000000000001,"15min":0.34000000000000002},"max":{"1min":1.016,"5min":1.091,"15min":2.6219999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69099999999999995,"15min":0.82699999999999996},"min":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68100000000000005,"15min":0.76000000000000001},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.35199999999999998},"max":{"1min":0.80600000000000005,"5min":0.98599999999999999,"15min":3.77},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.48099999999999998,"5min":0.47799999999999998,"15min":0.45000000000000001},"max":{"1min":0.90800000000000003,"5min":0.94399999999999995,"15min":0.98899999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.753,"15min":0.752},"min":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.43099999999999999},"max":{"1min":0.91700000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.62,"15min":0.626},"min":{"1min":0.501,"5min":0.36099999999999999,"15min":0.34599999999999997},"max":{"1min":0.89600000000000002,"5min":0.90200000000000002,"15min":0.94699999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66800000000000004,"15min":0.70399999999999996},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.374},"max":{"1min":0.94099999999999995,"5min":0.94399999999999995,"15min":1.4590000000000001},"last":0.71499999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.52800000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.747},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.45300000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.56799999999999995,"5min":0.438,"15min":0.438},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.5349999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.77700000000000002},"min":{"1min":0.49199999999999999,"5min":0.46500000000000002,"15min":0.42299999999999999},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.76300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72799999999999998,"15min":0.67400000000000004},"min":{"1min":0.39800000000000002,"5min":0.32800000000000001,"15min":0.29399999999999998},"max":{"1min":0.89500000000000002,"5min":1.24,"15min":1.24},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.77800000000000002,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.44700000000000001,"15min":0.26900000000000002},"max":{"1min":1.046,"5min":3.3399999999999999,"15min":3.3399999999999999},"last":0.88400000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68799999999999994,"15min":0.66900000000000004},"min":{"1min":0.44500000000000001,"5min":0.372,"15min":0.34599999999999997},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.872},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70199999999999996,"15min":0.67800000000000005},"min":{"1min":0.51800000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94499999999999995,"5min":0.94699999999999995,"15min":1.1759999999999999},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.752,"15min":0.68200000000000005},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.98499999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77500000000000002,"15min":0.72799999999999998},"min":{"1min":0.60499999999999998,"5min":0.47699999999999998,"15min":0.38700000000000001},"max":{"1min":1.153,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.72599999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":1.085,"5min":1.2110000000000001,"15min":2.1850000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.373},"max":{"1min":1.071,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.69899999999999995},"min":{"1min":0.44900000000000001,"5min":0.42799999999999999,"15min":0.33200000000000002},"max":{"1min":0.83499999999999996,"5min":1.0309999999999999,"15min":1.1339999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.71699999999999997},"min":{"1min":0.42399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":0.98699999999999999,"15min":1.085},"last":0.84699999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65400000000000003,"15min":0.69599999999999995},"min":{"1min":0.48699999999999999,"5min":0.46300000000000002,"15min":0.38600000000000001},"max":{"1min":0.89500000000000002,"5min":0.98799999999999999,"15min":1.218},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72399999999999998,"15min":0.77000000000000002},"min":{"1min":0.54700000000000004,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0489999999999999,"15min":1.198},"last":0.82499999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72499999999999998,"15min":0.79500000000000004},"min":{"1min":0.54700000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.323},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74399999999999999,"15min":0.80600000000000005},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.41499999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.4139999999999999},"last":0.80600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1961984,"data_stored":1216106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.42999999999999999,"15min":0.50800000000000001},"min":{"1min":0.27100000000000002,"5min":0.25900000000000001,"15min":0.23699999999999999},"max":{"1min":0.56000000000000005,"5min":0.72299999999999998,"15min":2.8879999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50700000000000001,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.32300000000000001,"15min":0.313},"max":{"1min":0.621,"5min":0.78300000000000003,"15min":1.96},"last":3.0139999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.44600000000000001,"15min":0.78100000000000003},"min":{"1min":0.32300000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.58599999999999997,"5min":0.66800000000000004,"15min":13.055999999999999},"last":3.3690000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.54100000000000004,"15min":0.60599999999999998},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.59799999999999998,"5min":0.96199999999999997,"15min":2.8479999999999999},"last":3.0590000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52800000000000002,"15min":0.58699999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48299999999999998,"15min":0.52600000000000002},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.56899999999999995,"5min":0.71599999999999997,"15min":1.2709999999999999},"last":3.4049999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.73599999999999999,"15min":1.05},"min":{"1min":0.54000000000000004,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.90400000000000003,"5min":1,"15min":12.989000000000001},"last":1.444},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78200000000000003,"15min":0.81399999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":0.875,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":1.139}]},{"osd":5,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.432},"max":{"1min":0.88300000000000001,"5min":1.0860000000000001,"15min":1.3100000000000001},"last":1.7390000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72199999999999998,"15min":1.052},"min":{"1min":0.56100000000000005,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.83299999999999996,"5min":1.077,"15min":13.023999999999999},"last":1.494}]},{"osd":6,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.76200000000000001,"15min":0.79800000000000004},"min":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88700000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":2.9670000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":1.0960000000000001},"min":{"1min":0.59399999999999997,"5min":0.54000000000000004,"15min":0.47799999999999998},"max":{"1min":0.96399999999999997,"5min":1.0569999999999999,"15min":12.941000000000001},"last":2.5409999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.80900000000000005},"min":{"1min":0.48699999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92600000000000005,"5min":1.071,"15min":2.806},"last":2.431},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.755,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94199999999999995,"5min":1.1319999999999999,"15min":1.337},"last":1.7849999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574966,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.28899999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":0.57099999999999995,"5min":0.753,"15min":0.753},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.58099999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.66200000000000003,"5min":0.69199999999999995,"15min":0.73699999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48699999999999999,"15min":0.496},"min":{"1min":0.376,"5min":0.315,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51100000000000001,"15min":0.48499999999999999},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.29899999999999999},"max":{"1min":0.59099999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.443,"5min":0.39400000000000002,"15min":0.38800000000000001},"max":{"1min":0.628,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72699999999999998,"15min":0.73799999999999999},"min":{"1min":0.48399999999999999,"5min":0.45700000000000002,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":1.1699999999999999,"15min":1.171},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.59199999999999997,"5min":0.52400000000000002,"15min":0.36799999999999999},"max":{"1min":0.92200000000000004,"5min":1.032,"15min":1.034},"last":0.751}]},{"osd":5,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.72799999999999998},"min":{"1min":0.51100000000000001,"5min":0.503,"15min":0.45600000000000002},"max":{"1min":0.95399999999999996,"5min":1.1080000000000001,"15min":1.1419999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.55100000000000005,"5min":0.53600000000000003,"15min":0.46999999999999997},"max":{"1min":0.82199999999999995,"5min":1.1040000000000001,"15min":1.1220000000000001},"last":0.91800000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.84599999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73599999999999999,"15min":0.748},"min":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.46200000000000002},"max":{"1min":0.93999999999999995,"5min":1.0649999999999999,"15min":1.546},"last":0.80900000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78200000000000003,"15min":0.79400000000000004},"min":{"1min":0.56000000000000005,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.6830000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.76900000000000002,"15min":0.98999999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53100000000000003},"max":{"1min":0.90500000000000003,"5min":1.0669999999999999,"15min":9.5960000000000001},"last":0.84699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40576,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331264,"statfs":{"total":96636764160,"available":96595214336,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.47699999999999998,"15min":0.442},"min":{"1min":0.30499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65500000000000003,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.443,"5min":0.504,"15min":0.47999999999999998},"min":{"1min":0.34200000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56000000000000005,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.55000000000000004,"15min":0.51000000000000001},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.629,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.56599999999999995,"15min":0.52400000000000002},"min":{"1min":0.38,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.64600000000000002,"5min":3.657,"15min":3.657},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.626,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69099999999999995,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59699999999999998,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":0.442}]},{"osd":4,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94499999999999995,"15min":0.78600000000000003},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.121,"5min":12.007,"15min":12.007},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.81299999999999994,"15min":0.73699999999999999},"min":{"1min":0.52200000000000002,"5min":0.49299999999999999,"15min":0.36399999999999999},"max":{"1min":1.1930000000000001,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79200000000000004,"15min":0.75},"min":{"1min":0.51100000000000001,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":1.1459999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.88100000000000001,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.137,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.623}]},{"osd":6,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.94599999999999995,"15min":0.78900000000000003},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.165,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.92300000000000004,"15min":0.80900000000000005},"min":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":1.107,"5min":13.084,"15min":13.084},"last":0.55000000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83299999999999996,"15min":0.82499999999999996},"min":{"1min":0.60599999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.089,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.871,"15min":0.81799999999999995},"min":{"1min":0.55800000000000005,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.2170000000000001,"5min":4.484,"15min":4.484},"last":0.56599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574965,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.44900000000000001,"15min":0.56399999999999995},"min":{"1min":0.25700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60099999999999998,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51800000000000002,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.85699999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.45200000000000001,"15min":0.88300000000000001},"min":{"1min":0.316,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.54700000000000004,"5min":1.3620000000000001,"15min":15.911},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.53400000000000003,"15min":0.95699999999999996},"min":{"1min":0.33400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76000000000000001,"5min":1.347,"15min":16.024000000000001},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.499,"15min":0.499},"min":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.27400000000000002},"max":{"1min":0.61099999999999999,"5min":3.25,"15min":3.25},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.54200000000000004,"15min":0.96199999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.64100000000000001,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.499}]},{"osd":4,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.52800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":1.782,"15min":1.782},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75900000000000001,"15min":0.88400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.36699999999999999},"max":{"1min":1.373,"5min":1.724,"15min":5.7300000000000004},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76300000000000001,"15min":0.88500000000000001},"min":{"1min":0.54000000000000004,"5min":0.49199999999999999,"15min":0.39700000000000002},"max":{"1min":0.96099999999999997,"5min":1.411,"15min":5.077},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.93400000000000005},"min":{"1min":0.56999999999999995,"5min":0.54500000000000004,"15min":0.44800000000000001},"max":{"1min":0.94699999999999995,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76800000000000002,"15min":0.78900000000000003},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.026,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.75900000000000001,"15min":0.89300000000000002},"min":{"1min":0.61199999999999999,"5min":0.503,"15min":0.45600000000000002},"max":{"1min":0.98199999999999998,"5min":1.4490000000000001,"15min":5.157},"last":0.73799999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76800000000000002,"15min":0.81399999999999995},"min":{"1min":0.58599999999999997,"5min":0.51700000000000002,"15min":0.47399999999999998},"max":{"1min":0.94599999999999995,"5min":1.863,"15min":1.863},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.76600000000000001,"15min":0.90400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.492,"15min":5.1189999999999998},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1368064,"data_stored":625738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66200000000000003,"15min":0.64600000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.88800000000000001,"5min":0.90400000000000003,"15min":1.784},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.65000000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.38},"max":{"1min":0.90200000000000002,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.80900000000000005},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.38},"max":{"1min":0.996,"5min":0.996,"15min":6.258},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.52400000000000002,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.95499999999999996,"15min":1.806},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69999999999999996,"15min":0.82599999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.29199999999999998},"max":{"1min":0.90400000000000003,"5min":0.91000000000000003,"15min":6.4740000000000002},"last":0.625},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.90100000000000002},"min":{"1min":0.58199999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":6.4859999999999998},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.52500000000000002,"5min":0.442,"15min":0.40000000000000002},"max":{"1min":0.86199999999999999,"5min":0.97799999999999998,"15min":1.25},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.71299999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.99099999999999999,"15min":1.413},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.64700000000000002},"min":{"1min":0.50700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.92100000000000004,"15min":1.3280000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63700000000000001,"15min":0.65700000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94299999999999995,"15min":0.99299999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.41099999999999998,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75800000000000001,"15min":0.78300000000000003},"min":{"1min":0.56799999999999995,"5min":0.52400000000000002,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.008,"15min":1.6839999999999999},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.54100000000000004,"5min":0.44500000000000001,"15min":0.41299999999999998},"max":{"1min":0.89200000000000002,"5min":0.95699999999999996,"15min":1.0660000000000001},"last":0.442},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.73799999999999999},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.036,"5min":1.036,"15min":1.21},"last":0.65400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37056,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334784,"statfs":{"total":96636764160,"available":96598818816,"internally_reserved":0,"allocated":1957888,"data_stored":1215087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.66700000000000004,"15min":0.66000000000000003},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.32100000000000001},"max":{"1min":0.88,"5min":0.94899999999999995,"15min":4.5010000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68700000000000006,"15min":0.81200000000000006},"min":{"1min":0.46999999999999997,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.86599999999999999,"5min":0.96199999999999997,"15min":6.4059999999999997},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.44800000000000001,"5min":0.42099999999999999,"15min":0.307},"max":{"1min":0.82699999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.46999999999999997,"5min":0.46100000000000002,"15min":0.41399999999999998},"max":{"1min":0.83999999999999997,"5min":1.1970000000000001,"15min":1.546},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.80100000000000005,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.376},"max":{"1min":0.86399999999999999,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.53300000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.81399999999999995,"15min":0.73899999999999999},"min":{"1min":0.53900000000000003,"5min":0.47599999999999998,"15min":0.42199999999999999},"max":{"1min":0.89300000000000002,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71599999999999997,"15min":0.74399999999999999},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":0.98099999999999998,"5min":1.3140000000000001,"15min":4.484},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.40899999999999997,"15min":0.34699999999999998},"max":{"1min":1.0229999999999999,"5min":1.1359999999999999,"15min":1.6240000000000001},"last":0.503},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66200000000000003,"15min":0.69199999999999995},"min":{"1min":0.48499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.93300000000000005,"5min":1.0569999999999999,"15min":2.718},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.44500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89900000000000002,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69599999999999995,"15min":0.71599999999999997},"min":{"1min":0.49099999999999999,"5min":0.48399999999999999,"15min":0.41799999999999998},"max":{"1min":0.94099999999999995,"5min":1.246,"15min":1.669},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:54:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.76000000000000001},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.93400000000000005,"5min":1.3360000000000001,"15min":2.0459999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.748,"15min":0.77600000000000002},"min":{"1min":0.52300000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0609999999999999,"15min":2.2919999999999998},"last":0.75900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:54:30.823 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T03:54:30.823 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T03:54:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.058+0000 7fd3495ff700 1 Processor -- start 2023-07-06T03:54:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.059+0000 7fd3495ff700 1 -- start start 2023-07-06T03:54:31.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.060+0000 7fd3495ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd344159390 0x7fd3440c9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.060+0000 7fd3495ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd3440c5060 0x7fd3440ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.060+0000 7fd3495ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd344159760 con 0x7fd3440c8fb0 2023-07-06T03:54:31.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.060+0000 7fd3495ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd344070670 con 0x7fd3440c5060 2023-07-06T03:54:31.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.060+0000 7fd3495ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd344070950 con 0x7fd344159390 2023-07-06T03:54:31.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.060+0000 7fd33bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd3440c8fb0 0x7fd3440c4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37860/0 (socket says 172.21.15.97:37860) 2023-07-06T03:54:31.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.060+0000 7fd33bfff700 1 -- 172.21.15.97:0/3655741309 learned_addr learned my addr 172.21.15.97:0/3655741309 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:31.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.060+0000 7fd3437fe700 1 --2- 172.21.15.97:0/3655741309 >> v2:172.21.15.146:3300/0 conn(0x7fd3440c5060 0x7fd3440ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.060+0000 7fd342ffd700 1 --2- 172.21.15.97:0/3655741309 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd344159390 0x7fd3440c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.061+0000 7fd3427fc700 1 -- 172.21.15.97:0/3655741309 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198231079 0 0) 0x7fd344159760 con 0x7fd3440c8fb0 2023-07-06T03:54:31.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.061+0000 7fd3427fc700 1 -- 172.21.15.97:0/3655741309 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3300035b0 con 0x7fd3440c8fb0 2023-07-06T03:54:31.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.061+0000 7fd3437fe700 1 -- 172.21.15.97:0/3655741309 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd344159390 msgr2=0x7fd3440c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:31.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.061+0000 7fd3437fe700 1 --2- 172.21.15.97:0/3655741309 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd344159390 0x7fd3440c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.061+0000 7fd3437fe700 1 -- 172.21.15.97:0/3655741309 >> v1:172.21.15.97:6789/0 conn(0x7fd3440c8fb0 legacy=0x7fd3440c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.061+0000 7fd3437fe700 1 -- 172.21.15.97:0/3655741309 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd344070c30 con 0x7fd3440c5060 2023-07-06T03:54:31.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.061+0000 7fd3437fe700 1 --2- 172.21.15.97:0/3655741309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3440c5060 0x7fd3440ccd60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009d90 tx=0x7fd33c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db56eef1e36a596e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:31.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.062+0000 7fd3427fc700 1 -- 172.21.15.97:0/3655741309 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c00e070 con 0x7fd3440c5060 2023-07-06T03:54:31.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.062+0000 7fd3427fc700 1 -- 172.21.15.97:0/3655741309 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c00bb20 con 0x7fd3440c5060 2023-07-06T03:54:31.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.062+0000 7fd3427fc700 1 -- 172.21.15.97:0/3655741309 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c012430 con 0x7fd3440c5060 2023-07-06T03:54:31.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.062+0000 7fd3495ff700 1 -- 172.21.15.97:0/3655741309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3440c5060 msgr2=0x7fd3440ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.062+0000 7fd3495ff700 1 --2- 172.21.15.97:0/3655741309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3440c5060 0x7fd3440ccd60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009d90 tx=0x7fd33c00b330 comp rx=0 tx=0).stop 2023-07-06T03:54:31.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.062+0000 7fd3495ff700 1 -- 172.21.15.97:0/3655741309 shutdown_connections 2023-07-06T03:54:31.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.062+0000 7fd3495ff700 1 --2- 172.21.15.97:0/3655741309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3440c5060 0x7fd3440ccd60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.062+0000 7fd3495ff700 1 --2- 172.21.15.97:0/3655741309 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd344159390 0x7fd3440c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.062+0000 7fd3495ff700 1 -- 172.21.15.97:0/3655741309 >> 172.21.15.97:0/3655741309 conn(0x7fd3440b05d0 msgr2=0x7fd3440b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:31.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.062+0000 7fd3495ff700 1 -- 172.21.15.97:0/3655741309 shutdown_connections 2023-07-06T03:54:31.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.062+0000 7fd3495ff700 1 -- 172.21.15.97:0/3655741309 wait complete. 2023-07-06T03:54:31.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.064+0000 7fd3495ff700 1 Processor -- start 2023-07-06T03:54:31.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.064+0000 7fd3495ff700 1 -- start start 2023-07-06T03:54:31.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.065+0000 7fd3495ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3440c5060 0x7fd3441554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.065+0000 7fd3495ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3440c8fb0 0x7fd3440c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.065+0000 7fd3495ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344159390 0x7fd34414d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.065+0000 7fd3495ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3440c6980 con 0x7fd344159390 2023-07-06T03:54:31.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.065+0000 7fd3495ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3440c6b90 con 0x7fd3440c5060 2023-07-06T03:54:31.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.065+0000 7fd3495ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3440c6e70 con 0x7fd3440c8fb0 2023-07-06T03:54:31.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.065+0000 7fd3437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344159390 0x7fd34414d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.066+0000 7fd3437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344159390 0x7fd34414d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34492/0 (socket says 172.21.15.97:34492) 2023-07-06T03:54:31.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.066+0000 7fd3437fe700 1 -- 172.21.15.97:0/2967439199 learned_addr learned my addr 172.21.15.97:0/2967439199 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:31.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.066+0000 7fd33bfff700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3440c8fb0 0x7fd3440c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.066+0000 7fd342ffd700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3440c5060 0x7fd3441554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.066+0000 7fd3437fe700 1 -- 172.21.15.97:0/2967439199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3440c8fb0 msgr2=0x7fd3440c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.066+0000 7fd3437fe700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3440c8fb0 0x7fd3440c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.066+0000 7fd3437fe700 1 -- 172.21.15.97:0/2967439199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3440c5060 msgr2=0x7fd3441554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.066+0000 7fd3437fe700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3440c5060 0x7fd3441554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.066+0000 7fd3437fe700 1 -- 172.21.15.97:0/2967439199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c009a70 con 0x7fd344159390 2023-07-06T03:54:31.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.066+0000 7fd3437fe700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344159390 0x7fd34414d400 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd33c014040 tx=0x7fd33c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:31.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.067+0000 7fd33b7fe700 1 -- 172.21.15.97:0/2967439199 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c0129f0 con 0x7fd344159390 2023-07-06T03:54:31.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.067+0000 7fd33b7fe700 1 -- 172.21.15.97:0/2967439199 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c020070 con 0x7fd344159390 2023-07-06T03:54:31.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.067+0000 7fd3495ff700 1 -- 172.21.15.97:0/2967439199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3440c7150 con 0x7fd344159390 2023-07-06T03:54:31.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.067+0000 7fd33b7fe700 1 -- 172.21.15.97:0/2967439199 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c012dd0 con 0x7fd344159390 2023-07-06T03:54:31.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.067+0000 7fd3495ff700 1 -- 172.21.15.97:0/2967439199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3440c7710 con 0x7fd344159390 2023-07-06T03:54:31.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.068+0000 7fd3495ff700 1 -- 172.21.15.97:0/2967439199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3440c78a0 con 0x7fd344159390 2023-07-06T03:54:31.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.069+0000 7fd33b7fe700 1 -- 172.21.15.97:0/2967439199 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd33c004040 con 0x7fd344159390 2023-07-06T03:54:31.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.070+0000 7fd33b7fe700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32406ace0 0x7fd32406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.070+0000 7fd33b7fe700 1 -- 172.21.15.97:0/2967439199 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd33c022070 con 0x7fd344159390 2023-07-06T03:54:31.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.070+0000 7fd342ffd700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32406ace0 0x7fd32406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.071+0000 7fd342ffd700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32406ace0 0x7fd32406d190 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd334009b40 tx=0x7fd334009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:31.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.074+0000 7fd33b7fe700 1 -- 172.21.15.97:0/2967439199 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd33c056870 con 0x7fd344159390 2023-07-06T03:54:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.345+0000 7fd3495ff700 1 -- 172.21.15.97:0/2967439199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd34404e340 con 0x7fd344159390 2023-07-06T03:54:31.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.349+0000 7fd33b7fe700 1 -- 172.21.15.97:0/2967439199 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd33c059e90 con 0x7fd344159390 2023-07-06T03:54:31.351 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:54:31.351 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg== 2023-07-06T03:54:31.352 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-07-06T03:54:31.352 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-07-06T03:54:31.352 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T03:54:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 -- 172.21.15.97:0/2967439199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32406ace0 msgr2=0x7fd32406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32406ace0 0x7fd32406d190 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd334009b40 tx=0x7fd334009210 comp rx=0 tx=0).stop 2023-07-06T03:54:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 -- 172.21.15.97:0/2967439199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344159390 msgr2=0x7fd34414d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344159390 0x7fd34414d400 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd33c014040 tx=0x7fd33c004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 -- 172.21.15.97:0/2967439199 shutdown_connections 2023-07-06T03:54:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32406ace0 0x7fd32406d190 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344159390 0x7fd34414d400 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3440c5060 0x7fd3441554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 --2- 172.21.15.97:0/2967439199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd3440c8fb0 0x7fd3440c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 -- 172.21.15.97:0/2967439199 >> 172.21.15.97:0/2967439199 conn(0x7fd3440b05d0 msgr2=0x7fd344156c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 -- 172.21.15.97:0/2967439199 shutdown_connections 2023-07-06T03:54:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.352+0000 7fd3495ff700 1 -- 172.21.15.97:0/2967439199 wait complete. 2023-07-06T03:54:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-07-06T03:54:31.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:31.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.564+0000 7f88f63ff700 1 Processor -- start 2023-07-06T03:54:31.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.565+0000 7f88f63ff700 1 -- start start 2023-07-06T03:54:31.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f63ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f88f015c520 0x7f88f00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f63ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f88f0160d00 0x7f88f00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f63ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f00cdae0 con 0x7f88f0160180 2023-07-06T03:54:31.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f63ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f00cd8d0 con 0x7f88f0160d00 2023-07-06T03:54:31.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f63ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f00b6640 con 0x7f88f015c520 2023-07-06T03:54:31.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f499c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f88f0160d00 0x7f88f00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f499c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f88f0160d00 0x7f88f00b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34498/0 (socket says 172.21.15.97:34498) 2023-07-06T03:54:31.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f499c700 1 -- 172.21.15.97:0/3222644718 learned_addr learned my addr 172.21.15.97:0/3222644718 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:31.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88ef7fe700 1 --1- 172.21.15.97:0/3222644718 >> v1:172.21.15.97:6789/0 conn(0x7f88f0160180 0x7f88f01605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37874/0 (socket says 172.21.15.97:37874) 2023-07-06T03:54:31.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88effff700 1 --2- 172.21.15.97:0/3222644718 >> v2:172.21.15.146:3300/0 conn(0x7f88f015c520 0x7f88f00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f499c700 1 -- 172.21.15.97:0/3222644718 >> v2:172.21.15.146:3300/0 conn(0x7f88f015c520 msgr2=0x7f88f00cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:31.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f499c700 1 --2- 172.21.15.97:0/3222644718 >> v2:172.21.15.146:3300/0 conn(0x7f88f015c520 0x7f88f00cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f499c700 1 -- 172.21.15.97:0/3222644718 >> v1:172.21.15.97:6789/0 conn(0x7f88f0160180 legacy=0x7f88f01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88f499c700 1 -- 172.21.15.97:0/3222644718 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f00b6920 con 0x7f88f0160d00 2023-07-06T03:54:31.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.566+0000 7f88eeffd700 1 -- 172.21.15.97:0/3222644718 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291246667 0 0) 0x7f88f00cdae0 con 0x7f88f0160180 2023-07-06T03:54:31.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.567+0000 7f88f499c700 1 --2- 172.21.15.97:0/3222644718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f0160d00 0x7f88f00b7400 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f88f00b4c70 tx=0x7f88e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3cce2e8516bcd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:31.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.567+0000 7f88eeffd700 1 -- 172.21.15.97:0/3222644718 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e800e070 con 0x7f88f0160d00 2023-07-06T03:54:31.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.567+0000 7f88eeffd700 1 -- 172.21.15.97:0/3222644718 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e800b910 con 0x7f88f0160d00 2023-07-06T03:54:31.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.567+0000 7f88eeffd700 1 -- 172.21.15.97:0/3222644718 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e800bcc0 con 0x7f88f0160d00 2023-07-06T03:54:31.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.567+0000 7f88f63ff700 1 -- 172.21.15.97:0/3222644718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f0160d00 msgr2=0x7f88f00b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.567+0000 7f88f63ff700 1 --2- 172.21.15.97:0/3222644718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f0160d00 0x7f88f00b7400 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f88f00b4c70 tx=0x7f88e800b330 comp rx=0 tx=0).stop 2023-07-06T03:54:31.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.568+0000 7f88f63ff700 1 -- 172.21.15.97:0/3222644718 shutdown_connections 2023-07-06T03:54:31.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.568+0000 7f88f63ff700 1 --2- 172.21.15.97:0/3222644718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f0160d00 0x7f88f00b7400 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.568+0000 7f88f63ff700 1 --2- 172.21.15.97:0/3222644718 >> v2:172.21.15.146:3300/0 conn(0x7f88f015c520 0x7f88f00cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.568+0000 7f88f63ff700 1 -- 172.21.15.97:0/3222644718 >> 172.21.15.97:0/3222644718 conn(0x7f88f0071790 msgr2=0x7f88f0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:31.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.568+0000 7f88f63ff700 1 -- 172.21.15.97:0/3222644718 shutdown_connections 2023-07-06T03:54:31.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.568+0000 7f88f63ff700 1 -- 172.21.15.97:0/3222644718 wait complete. 2023-07-06T03:54:31.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.569+0000 7f88f63ff700 1 Processor -- start 2023-07-06T03:54:31.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.570+0000 7f88f63ff700 1 -- start start 2023-07-06T03:54:31.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.570+0000 7f88f63ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88f015c520 0x7f88f0155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.570+0000 7f88f63ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f0160180 0x7f88f015cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.570+0000 7f88f63ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88f0160d00 0x7f88f015d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.570+0000 7f88f63ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f00cb920 con 0x7f88f0160180 2023-07-06T03:54:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.570+0000 7f88f63ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f0155ab0 con 0x7f88f0160d00 2023-07-06T03:54:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.570+0000 7f88f63ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f015dec0 con 0x7f88f015c520 2023-07-06T03:54:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.571+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f0160180 0x7f88f015cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.571+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f0160180 0x7f88f015cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34502/0 (socket says 172.21.15.97:34502) 2023-07-06T03:54:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.571+0000 7f88ef7fe700 1 -- 172.21.15.97:0/4098384010 learned_addr learned my addr 172.21.15.97:0/4098384010 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.571+0000 7f88effff700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88f015c520 0x7f88f0155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.571+0000 7f88f499c700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88f0160d00 0x7f88f015d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.571+0000 7f88ef7fe700 1 -- 172.21.15.97:0/4098384010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88f015c520 msgr2=0x7f88f0155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.571+0000 7f88ef7fe700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88f015c520 0x7f88f0155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.571+0000 7f88ef7fe700 1 -- 172.21.15.97:0/4098384010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88f0160d00 msgr2=0x7f88f015d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.571+0000 7f88ef7fe700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88f0160d00 0x7f88f015d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.571+0000 7f88ef7fe700 1 -- 172.21.15.97:0/4098384010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e8009b30 con 0x7f88f0160180 2023-07-06T03:54:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.571+0000 7f88ef7fe700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f0160180 0x7f88f015cb90 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f88e40011e0 tx=0x7f88e400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.572+0000 7f88ecff9700 1 -- 172.21.15.97:0/4098384010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4004c80 con 0x7f88f0160180 2023-07-06T03:54:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.572+0000 7f88ecff9700 1 -- 172.21.15.97:0/4098384010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e4004e00 con 0x7f88f0160180 2023-07-06T03:54:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.572+0000 7f88f63ff700 1 -- 172.21.15.97:0/4098384010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f00cbb00 con 0x7f88f0160180 2023-07-06T03:54:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.572+0000 7f88ecff9700 1 -- 172.21.15.97:0/4098384010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4007310 con 0x7f88f0160180 2023-07-06T03:54:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.572+0000 7f88f63ff700 1 -- 172.21.15.97:0/4098384010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f00cc0c0 con 0x7f88f0160180 2023-07-06T03:54:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.573+0000 7f88f63ff700 1 -- 172.21.15.97:0/4098384010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f004e340 con 0x7f88f0160180 2023-07-06T03:54:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.574+0000 7f88ecff9700 1 -- 172.21.15.97:0/4098384010 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f88e4007470 con 0x7f88f0160180 2023-07-06T03:54:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.575+0000 7f88ecff9700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88dc06ace0 0x7f88dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.575+0000 7f88ecff9700 1 -- 172.21.15.97:0/4098384010 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f88e40891c0 con 0x7f88f0160180 2023-07-06T03:54:31.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.575+0000 7f88effff700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88dc06ace0 0x7f88dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:31.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.575+0000 7f88effff700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88dc06ace0 0x7f88dc06d190 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009b40 tx=0x7f88e0009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:31.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.579+0000 7f88ecff9700 1 -- 172.21.15.97:0/4098384010 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f88e40541d0 con 0x7f88f0160180 2023-07-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.814+0000 7f88f63ff700 1 -- 172.21.15.97:0/4098384010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88f00b4f40 con 0x7f88f0160180 2023-07-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.815+0000 7f88ecff9700 1 -- 172.21.15.97:0/4098384010 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4676 (secure 0 0 0) 0x7f88e40577f0 con 0x7f88f0160180 2023-07-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2796855133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2796855133},{"type":"v1","addr":"172.21.15.97:6835","nonce":2796855133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6851,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1092966074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1092966074},{"type":"v1","addr":"172.21.15.146:6835","nonce":1092966074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8577,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2594138359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2594138359},{"type":"v1","addr":"172.21.15.97:6837","nonce":2594138359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:54:25.441909+0000","modified":"2023-07-06T03:54:28.981530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/582690410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":582690410},{"type":"v1","addr":"172.21.15.146:6834","nonce":582690410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 -- 172.21.15.97:0/4098384010 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88dc06ace0 msgr2=0x7f88dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88dc06ace0 0x7f88dc06d190 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009b40 tx=0x7f88e0009210 comp rx=0 tx=0).stop 2023-07-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 -- 172.21.15.97:0/4098384010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f0160180 msgr2=0x7f88f015cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f0160180 0x7f88f015cb90 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f88e40011e0 tx=0x7f88e400ca30 comp rx=0 tx=0).stop 2023-07-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 -- 172.21.15.97:0/4098384010 shutdown_connections 2023-07-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f88dc06ace0 0x7f88dc06d190 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88f0160d00 0x7f88f015d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f88f015c520 0x7f88f0155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 --2- 172.21.15.97:0/4098384010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f0160180 0x7f88f015cb90 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 -- 172.21.15.97:0/4098384010 >> 172.21.15.97:0/4098384010 conn(0x7f88f0071790 msgr2=0x7f88f0071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 -- 172.21.15.97:0/4098384010 shutdown_connections 2023-07-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:31.817+0000 7f88f63ff700 1 -- 172.21.15.97:0/4098384010 wait complete. 2023-07-06T03:54:31.821 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 80 2023-07-06T03:54:31.936 INFO:tasks.ceph.mds.d.smithi146.stderr:2023-07-06T03:54:31.934+0000 7f0bbb3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:54:32.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T03:54:25.441909+0000', 'modified': '2023-07-06T03:54:28.981530+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8580}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8580': {'gid': 8580, 'name': 'd', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6834/582690410', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 582690410}, {'type': 'v1', 'addr': '172.21.15.146:6834', 'nonce': 582690410}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T03:54:32.240 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T03:54:32.240 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-07-06T03:54:32.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.435+0000 7fb8431ff700 1 Processor -- start 2023-07-06T03:54:32.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.435+0000 7fb8431ff700 1 -- start start 2023-07-06T03:54:32.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.436+0000 7fb8431ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb83c06b360 0x7fb83c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.436+0000 7fb8431ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb83c0d2210 0x7fb83c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.436+0000 7fb8431ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83c0cff90 con 0x7fb83c15c610 2023-07-06T03:54:32.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.436+0000 7fb8431ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb83c0d0780 con 0x7fb83c06b360 2023-07-06T03:54:32.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.436+0000 7fb8431ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c15c9e0 con 0x7fb83c0d2210 2023-07-06T03:54:32.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.436+0000 7fb83bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb83c06b360 0x7fb83c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.436+0000 7fb83bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb83c06b360 0x7fb83c0b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34512/0 (socket says 172.21.15.97:34512) 2023-07-06T03:54:32.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.436+0000 7fb83bfff700 1 -- 172.21.15.97:0/2366170835 learned_addr learned my addr 172.21.15.97:0/2366170835 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:32.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.436+0000 7fb84179c700 1 --2- 172.21.15.97:0/2366170835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb83c0d2210 0x7fb83c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.437+0000 7fb83bfff700 1 -- 172.21.15.97:0/2366170835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb83c0d2210 msgr2=0x7fb83c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:32.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.437+0000 7fb83bfff700 1 --2- 172.21.15.97:0/2366170835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb83c0d2210 0x7fb83c0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.437+0000 7fb83bfff700 1 -- 172.21.15.97:0/2366170835 >> v1:172.21.15.146:6789/0 conn(0x7fb83c15c610 legacy=0x7fb83c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.437+0000 7fb83bfff700 1 -- 172.21.15.97:0/2366170835 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c0b5650 con 0x7fb83c06b360 2023-07-06T03:54:32.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.437+0000 7fb84179c700 1 --2- 172.21.15.97:0/2366170835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb83c0d2210 0x7fb83c0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:32.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.437+0000 7fb83bfff700 1 --2- 172.21.15.97:0/2366170835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c06b360 0x7fb83c0b4ec0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb8280098f0 tx=0x7fb828009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b205875849de2ce1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:32.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.437+0000 7fb83b7fe700 1 -- 172.21.15.97:0/2366170835 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82800b8a0 con 0x7fb83c06b360 2023-07-06T03:54:32.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.437+0000 7fb83b7fe700 1 -- 172.21.15.97:0/2366170835 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82800ba00 con 0x7fb83c06b360 2023-07-06T03:54:32.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.437+0000 7fb83b7fe700 1 -- 172.21.15.97:0/2366170835 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82800bd90 con 0x7fb83c06b360 2023-07-06T03:54:32.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.438+0000 7fb8431ff700 1 -- 172.21.15.97:0/2366170835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c06b360 msgr2=0x7fb83c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.438+0000 7fb8431ff700 1 --2- 172.21.15.97:0/2366170835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c06b360 0x7fb83c0b4ec0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb8280098f0 tx=0x7fb828009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:32.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.438+0000 7fb8431ff700 1 -- 172.21.15.97:0/2366170835 shutdown_connections 2023-07-06T03:54:32.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.438+0000 7fb8431ff700 1 --2- 172.21.15.97:0/2366170835 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb83c0d2210 0x7fb83c0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.438+0000 7fb8431ff700 1 --2- 172.21.15.97:0/2366170835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c06b360 0x7fb83c0b4ec0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.438+0000 7fb8431ff700 1 -- 172.21.15.97:0/2366170835 >> 172.21.15.97:0/2366170835 conn(0x7fb83c06a6c0 msgr2=0x7fb83c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:32.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.438+0000 7fb8431ff700 1 -- 172.21.15.97:0/2366170835 shutdown_connections 2023-07-06T03:54:32.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.438+0000 7fb8431ff700 1 -- 172.21.15.97:0/2366170835 wait complete. 2023-07-06T03:54:32.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.440+0000 7fb8431ff700 1 Processor -- start 2023-07-06T03:54:32.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.440+0000 7fb8431ff700 1 -- start start 2023-07-06T03:54:32.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb8431ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb83c06b360 0x7fb83c157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb8431ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c0d2210 0x7fb83c0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb8431ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb83c15c610 0x7fb83c0d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb8431ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c15cb20 con 0x7fb83c0d2210 2023-07-06T03:54:32.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb8431ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c1f2510 con 0x7fb83c15c610 2023-07-06T03:54:32.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb8431ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c1f27f0 con 0x7fb83c06b360 2023-07-06T03:54:32.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb83bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c0d2210 0x7fb83c0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb83bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c0d2210 0x7fb83c0cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34520/0 (socket says 172.21.15.97:34520) 2023-07-06T03:54:32.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb83bfff700 1 -- 172.21.15.97:0/3027724263 learned_addr learned my addr 172.21.15.97:0/3027724263 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:32.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb840f9b700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb83c06b360 0x7fb83c157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb84179c700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb83c15c610 0x7fb83c0d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb83bfff700 1 -- 172.21.15.97:0/3027724263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb83c06b360 msgr2=0x7fb83c157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb83bfff700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb83c06b360 0x7fb83c157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb83bfff700 1 -- 172.21.15.97:0/3027724263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb83c15c610 msgr2=0x7fb83c0d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb83bfff700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb83c15c610 0x7fb83c0d2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.441+0000 7fb83bfff700 1 -- 172.21.15.97:0/3027724263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb828009550 con 0x7fb83c0d2210 2023-07-06T03:54:32.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.442+0000 7fb83bfff700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c0d2210 0x7fb83c0cc7c0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb828000c00 tx=0x7fb828004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:32.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.442+0000 7fb8397fa700 1 -- 172.21.15.97:0/3027724263 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828004300 con 0x7fb83c0d2210 2023-07-06T03:54:32.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.442+0000 7fb8397fa700 1 -- 172.21.15.97:0/3027724263 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82801f070 con 0x7fb83c0d2210 2023-07-06T03:54:32.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.442+0000 7fb8431ff700 1 -- 172.21.15.97:0/3027724263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83c1f2ad0 con 0x7fb83c0d2210 2023-07-06T03:54:32.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.442+0000 7fb8397fa700 1 -- 172.21.15.97:0/3027724263 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8280036a0 con 0x7fb83c0d2210 2023-07-06T03:54:32.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.442+0000 7fb8431ff700 1 -- 172.21.15.97:0/3027724263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83c1585b0 con 0x7fb83c0d2210 2023-07-06T03:54:32.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.444+0000 7fb8397fa700 1 -- 172.21.15.97:0/3027724263 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb828021030 con 0x7fb83c0d2210 2023-07-06T03:54:32.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.445+0000 7fb8397fa700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 0x7fb82006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.445+0000 7fb8397fa700 1 -- 172.21.15.97:0/3027724263 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb82808a860 con 0x7fb83c0d2210 2023-07-06T03:54:32.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.445+0000 7fb840f9b700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 0x7fb82006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.446+0000 7fb840f9b700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 0x7fb82006d190 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0c9d60 tx=0x7fb830009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:32.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.447+0000 7fb856bf2b80 1 Processor -- start 2023-07-06T03:54:32.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.447+0000 7fb856bf2b80 1 -- start start 2023-07-06T03:54:32.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.448+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf6999d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.448+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf699a2190 0x55bf699c0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.448+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf6997a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.448+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf698eeea0 con 0x55bf6999cd50 2023-07-06T03:54:32.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.448+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf69862600 con 0x55bf69977b50 2023-07-06T03:54:32.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.448+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf698d1910 con 0x55bf699a2190 2023-07-06T03:54:32.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.448+0000 7fb840f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf6999d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.448+0000 7fb840f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf6999d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34534/0 (socket says 172.21.15.97:34534) 2023-07-06T03:54:32.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.448+0000 7fb840f9b700 1 -- 172.21.15.97:0/1867994706 learned_addr learned my addr 172.21.15.97:0/1867994706 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:32.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.448+0000 7fb83bfff700 1 --2- 172.21.15.97:0/1867994706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf699a2190 0x55bf699c0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.448+0000 7fb84179c700 1 --2- 172.21.15.97:0/1867994706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf6997a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.449+0000 7fb840f9b700 1 -- 172.21.15.97:0/1867994706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf699a2190 msgr2=0x55bf699c0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:32.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.449+0000 7fb840f9b700 1 --2- 172.21.15.97:0/1867994706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf699a2190 0x55bf699c0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.449+0000 7fb840f9b700 1 -- 172.21.15.97:0/1867994706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 msgr2=0x55bf6997a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:32.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.449+0000 7fb840f9b700 1 --2- 172.21.15.97:0/1867994706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf6997a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.449+0000 7fb840f9b700 1 -- 172.21.15.97:0/1867994706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf696be790 con 0x55bf6999cd50 2023-07-06T03:54:32.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.449+0000 7fb84179c700 1 --2- 172.21.15.97:0/1867994706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf6997a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:32.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.449+0000 7fb840f9b700 1 --2- 172.21.15.97:0/1867994706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf6999d120 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fb83000f6f0 tx=0x7fb830012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace967096b3463cd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:32.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.449+0000 7fb8267fc700 1 -- 172.21.15.97:0/1867994706 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000fdc0 con 0x55bf6999cd50 2023-07-06T03:54:32.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.449+0000 7fb8267fc700 1 -- 172.21.15.97:0/1867994706 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb830014450 con 0x55bf6999cd50 2023-07-06T03:54:32.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.449+0000 7fb8267fc700 1 -- 172.21.15.97:0/1867994706 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8300147c0 con 0x55bf6999cd50 2023-07-06T03:54:32.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.450+0000 7fb856bf2b80 1 -- 172.21.15.97:0/1867994706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 msgr2=0x55bf6999d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.450+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/1867994706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf6999d120 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fb83000f6f0 tx=0x7fb830012000 comp rx=0 tx=0).stop 2023-07-06T03:54:32.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.450+0000 7fb856bf2b80 1 -- 172.21.15.97:0/1867994706 shutdown_connections 2023-07-06T03:54:32.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.450+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/1867994706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf6997a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.450+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/1867994706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf6999d120 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.450+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/1867994706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf699a2190 0x55bf699c0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.450+0000 7fb856bf2b80 1 -- 172.21.15.97:0/1867994706 >> 172.21.15.97:0/1867994706 conn(0x55bf6996f310 msgr2=0x55bf699bf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:32.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.450+0000 7fb856bf2b80 1 -- 172.21.15.97:0/1867994706 shutdown_connections 2023-07-06T03:54:32.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.450+0000 7fb856bf2b80 1 -- 172.21.15.97:0/1867994706 wait complete. 2023-07-06T03:54:32.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.451+0000 7fb856bf2b80 1 -- start start 2023-07-06T03:54:32.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.451+0000 7fb856bf2b80 1 Processor -- start 2023-07-06T03:54:32.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf69999ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf6999cd50 0x55bf6999b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf6999bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf698cabe0 con 0x55bf699a2190 2023-07-06T03:54:32.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf698885b0 con 0x55bf69977b50 2023-07-06T03:54:32.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf6986e910 con 0x55bf6999cd50 2023-07-06T03:54:32.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb856bf2b80 10 client.0 _ll_register_callbacks cb 0x55bf69886b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:54:32.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb84179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf6999bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb84179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf6999bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34538/0 (socket says 172.21.15.97:34538) 2023-07-06T03:54:32.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb84179c700 1 -- 172.21.15.97:0/2481927676 learned_addr learned my addr 172.21.15.97:0/2481927676 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:32.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb83bfff700 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf6999cd50 0x55bf6999b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.452+0000 7fb840f9b700 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf69999ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb84179c700 1 -- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf6999cd50 msgr2=0x55bf6999b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:32.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb84179c700 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf6999cd50 0x55bf6999b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb84179c700 1 -- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 msgr2=0x55bf69999ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:32.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb84179c700 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf69999ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb84179c700 1 -- 172.21.15.97:0/2481927676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83c0cff90 con 0x55bf699a2190 2023-07-06T03:54:32.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb84179c700 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf6999bcb0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fb834004740 tx=0x7fb834009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3d92fc9805574c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:32.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb81b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:54:32.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb81b7fe700 1 -- 172.21.15.97:0/2481927676 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834007bd0 con 0x55bf699a2190 2023-07-06T03:54:32.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb81b7fe700 1 -- 172.21.15.97:0/2481927676 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb834007d30 con 0x55bf699a2190 2023-07-06T03:54:32.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2481927676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf6996c920 con 0x55bf699a2190 2023-07-06T03:54:32.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb81b7fe700 1 -- 172.21.15.97:0/2481927676 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb834016460 con 0x55bf699a2190 2023-07-06T03:54:32.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb856bf2b80 10 client.8730 fetch_fsmap learned FSMap version 80 2023-07-06T03:54:32.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.453+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2481927676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x55bf697ab0a0 con 0x55bf699a2190 2023-07-06T03:54:32.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.454+0000 7fb81b7fe700 1 -- 172.21.15.97:0/2481927676 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf6996c920 con 0x55bf699a2190 2023-07-06T03:54:32.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.454+0000 7fb81b7fe700 1 -- 172.21.15.97:0/2481927676 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 80) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb834014040 con 0x55bf699a2190 2023-07-06T03:54:32.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.454+0000 7fb856bf2b80 10 client.8730 fetch_fsmap finished waiting for FSMap version 80 2023-07-06T03:54:32.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.454+0000 7fb856bf2b80 10 client.8730 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T03:54:32.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.454+0000 7fb856bf2b80 20 client.8730 populate_metadata read hostname 'smithi097' 2023-07-06T03:54:32.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.454+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:6832/582690410,v1:172.21.15.146:6834/582690410] conn(0x55bf69993970 0x55bf69994ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.454+0000 7fb856bf2b80 4 client.8730 mds_command: new command op to 8580 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T03:54:32.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.454+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2481927676 --> [v2:172.21.15.146:6832/582690410,v1:172.21.15.146:6834/582690410] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf699490d0 con 0x55bf69993970 2023-07-06T03:54:32.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.454+0000 7fb840f9b700 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:6832/582690410,v1:172.21.15.146:6834/582690410] conn(0x55bf69993970 0x55bf69994ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.455+0000 7fb81b7fe700 10 client.8730 ms_handle_connect on v2:172.21.15.146:6832/582690410 2023-07-06T03:54:32.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.455+0000 7fb840f9b700 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:6832/582690410,v1:172.21.15.146:6834/582690410] conn(0x55bf69993970 0x55bf69994ba0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d494c3b413291893 server_cookie=2d24b65955ed33fa in_seq=0 out_seq=0 2023-07-06T03:54:32.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb81b7fe700 1 -- 172.21.15.97:0/2481927676 <== mds.0 v2:172.21.15.146:6832/582690410 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bf699490d0 con 0x55bf69993970 2023-07-06T03:54:32.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb81b7fe700 10 client.8730 handle_command_reply: tid=0 2023-07-06T03:54:32.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 1 client.8730 shutdown 2023-07-06T03:54:32.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2481927676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 msgr2=0x55bf6999bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf6999bcb0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fb834004740 tx=0x7fb834009ee0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2481927676 shutdown_connections 2023-07-06T03:54:32.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf6999bcb0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:6832/582690410,v1:172.21.15.146:6834/582690410] conn(0x55bf69993970 0x55bf69994ba0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf69999ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/2481927676 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf6999cd50 0x55bf6999b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb81b7fe700 0 client.8730 ms_handle_reset on v2:172.21.15.146:6832/582690410 2023-07-06T03:54:32.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2481927676 >> 172.21.15.97:0/2481927676 conn(0x55bf6996f310 msgr2=0x55bf69973ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:32.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2481927676 shutdown_connections 2023-07-06T03:54:32.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2481927676 wait complete. 2023-07-06T03:54:32.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.456+0000 7fb856bf2b80 20 client.8730 trim_cache size 0 max 16384 2023-07-06T03:54:32.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.474+0000 7fb856bf2b80 1 Processor -- start 2023-07-06T03:54:32.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.474+0000 7fb856bf2b80 1 -- start start 2023-07-06T03:54:32.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.475+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf69978930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.475+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf6999cd50 0x55bf699747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.475+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf699a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.475+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6999b110 con 0x55bf699a2190 2023-07-06T03:54:32.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.475+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf696bf390 con 0x55bf69977b50 2023-07-06T03:54:32.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.475+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf69995d50 con 0x55bf6999cd50 2023-07-06T03:54:32.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.475+0000 7fb84179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf699a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.475+0000 7fb84179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf699a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34548/0 (socket says 172.21.15.97:34548) 2023-07-06T03:54:32.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.475+0000 7fb84179c700 1 -- 172.21.15.97:0/2301357371 learned_addr learned my addr 172.21.15.97:0/2301357371 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:32.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.476+0000 7fb84179c700 1 -- 172.21.15.97:0/2301357371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf6999cd50 msgr2=0x55bf699747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:32.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.476+0000 7fb84179c700 1 --2- 172.21.15.97:0/2301357371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf6999cd50 0x55bf699747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.476+0000 7fb84179c700 1 -- 172.21.15.97:0/2301357371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 msgr2=0x55bf69978930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.476+0000 7fb84179c700 1 --2- 172.21.15.97:0/2301357371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf69978930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.476+0000 7fb84179c700 1 -- 172.21.15.97:0/2301357371 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf69890b10 con 0x55bf699a2190 2023-07-06T03:54:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.476+0000 7fb84179c700 1 --2- 172.21.15.97:0/2301357371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf699a19c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fb834002230 tx=0x7fb834010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99044a7bed5d6458 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.476+0000 7fb826ffd700 1 -- 172.21.15.97:0/2301357371 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834007610 con 0x55bf699a2190 2023-07-06T03:54:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.476+0000 7fb826ffd700 1 -- 172.21.15.97:0/2301357371 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83400f040 con 0x55bf699a2190 2023-07-06T03:54:32.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.476+0000 7fb826ffd700 1 -- 172.21.15.97:0/2301357371 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8340079d0 con 0x55bf699a2190 2023-07-06T03:54:32.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.477+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2301357371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 msgr2=0x55bf699a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.477+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/2301357371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf699a19c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fb834002230 tx=0x7fb834010000 comp rx=0 tx=0).stop 2023-07-06T03:54:32.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.477+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2301357371 shutdown_connections 2023-07-06T03:54:32.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.477+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/2301357371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf699a2190 0x55bf699a19c0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.477+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/2301357371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf69977b50 0x55bf69978930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.477+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/2301357371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf6999cd50 0x55bf699747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.477+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2301357371 >> 172.21.15.97:0/2301357371 conn(0x55bf6996f310 msgr2=0x55bf699e1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:32.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.477+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2301357371 shutdown_connections 2023-07-06T03:54:32.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.477+0000 7fb856bf2b80 1 -- 172.21.15.97:0/2301357371 wait complete. 2023-07-06T03:54:32.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.478+0000 7fb856bf2b80 1 -- start start 2023-07-06T03:54:32.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.478+0000 7fb856bf2b80 1 Processor -- start 2023-07-06T03:54:32.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf69977b50 0x55bf69a15eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf699b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb856bf2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf699a2190 0x55bf699b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf699956d0 con 0x55bf6999cd50 2023-07-06T03:54:32.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf69a163f0 con 0x55bf699a2190 2023-07-06T03:54:32.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb856bf2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf69a1ab10 con 0x55bf69977b50 2023-07-06T03:54:32.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb856bf2b80 10 client.0 _ll_register_callbacks cb 0x7fb834003dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:54:32.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb83bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf699b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb83bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf699b64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34560/0 (socket says 172.21.15.97:34560) 2023-07-06T03:54:32.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb83bfff700 1 -- 172.21.15.97:0/572814816 learned_addr learned my addr 172.21.15.97:0/572814816 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:32.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb84179c700 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf699a2190 0x55bf699b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb840f9b700 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf69977b50 0x55bf69a15eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb83bfff700 1 -- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf69977b50 msgr2=0x55bf69a15eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:32.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb83bfff700 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf69977b50 0x55bf69a15eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb83bfff700 1 -- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf699a2190 msgr2=0x55bf699b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:32.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb83bfff700 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf699a2190 0x55bf699b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb83bfff700 1 -- 172.21.15.97:0/572814816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf697ab0a0 con 0x55bf6999cd50 2023-07-06T03:54:32.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.479+0000 7fb84179c700 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf699a2190 0x55bf699b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:32.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb83bfff700 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf699b64d0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fb82800c010 tx=0x7fb82805d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a0c81950df3cbc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:32.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb81b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:54:32.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb81b7fe700 1 -- 172.21.15.97:0/572814816 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8280579a0 con 0x55bf6999cd50 2023-07-06T03:54:32.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb81b7fe700 1 -- 172.21.15.97:0/572814816 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb828058a30 con 0x55bf6999cd50 2023-07-06T03:54:32.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb856bf2b80 1 -- 172.21.15.97:0/572814816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb830014450 con 0x55bf6999cd50 2023-07-06T03:54:32.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb81b7fe700 1 -- 172.21.15.97:0/572814816 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb828058bf0 con 0x55bf6999cd50 2023-07-06T03:54:32.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb856bf2b80 10 client.8736 fetch_fsmap learned FSMap version 80 2023-07-06T03:54:32.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb856bf2b80 1 -- 172.21.15.97:0/572814816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x55bf6971f1d0 con 0x55bf6999cd50 2023-07-06T03:54:32.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb81b7fe700 1 -- 172.21.15.97:0/572814816 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb830014450 con 0x55bf6999cd50 2023-07-06T03:54:32.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb81b7fe700 1 -- 172.21.15.97:0/572814816 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 80) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb828061400 con 0x55bf6999cd50 2023-07-06T03:54:32.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb856bf2b80 10 client.8736 fetch_fsmap finished waiting for FSMap version 80 2023-07-06T03:54:32.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb856bf2b80 10 client.8736 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T03:54:32.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb856bf2b80 20 client.8736 populate_metadata read hostname 'smithi097' 2023-07-06T03:54:32.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:6832/582690410,v1:172.21.15.146:6834/582690410] conn(0x55bf69a19bb0 0x55bf69a19f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb856bf2b80 4 client.8736 mds_command: new command op to 8580 tid=0[{"prefix": "status"}] 2023-07-06T03:54:32.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.480+0000 7fb856bf2b80 1 -- 172.21.15.97:0/572814816 --> [v2:172.21.15.146:6832/582690410,v1:172.21.15.146:6834/582690410] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bf698c3d20 con 0x55bf69a19bb0 2023-07-06T03:54:32.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.481+0000 7fb840f9b700 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:6832/582690410,v1:172.21.15.146:6834/582690410] conn(0x55bf69a19bb0 0x55bf69a19f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.481+0000 7fb840f9b700 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:6832/582690410,v1:172.21.15.146:6834/582690410] conn(0x55bf69a19bb0 0x55bf69a19f80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f79d2584fdc75cc server_cookie=db7130e65ac187b6 in_seq=0 out_seq=0 2023-07-06T03:54:32.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.481+0000 7fb81b7fe700 10 client.8736 ms_handle_connect on v2:172.21.15.146:6832/582690410 2023-07-06T03:54:32.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb81b7fe700 1 -- 172.21.15.97:0/572814816 <== mds.0 v2:172.21.15.146:6832/582690410 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55bf698c3d20 con 0x55bf69a19bb0 2023-07-06T03:54:32.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb81b7fe700 10 client.8736 handle_command_reply: tid=0 2023-07-06T03:54:32.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 1 client.8736 shutdown 2023-07-06T03:54:32.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 1 -- 172.21.15.97:0/572814816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 msgr2=0x55bf699b64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf699b64d0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fb82800c010 tx=0x7fb82805d5b0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 1 -- 172.21.15.97:0/572814816 shutdown_connections 2023-07-06T03:54:32.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:6832/582690410,v1:172.21.15.146:6834/582690410] conn(0x55bf69a19bb0 0x55bf69a19f80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf699a2190 0x55bf699b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55bf69977b50 0x55bf69a15eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb81b7fe700 0 client.8736 ms_handle_reset on v2:172.21.15.146:6832/582690410 2023-07-06T03:54:32.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 1 --2- 172.21.15.97:0/572814816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf6999cd50 0x55bf699b64d0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 1 -- 172.21.15.97:0/572814816 >> 172.21.15.97:0/572814816 conn(0x55bf6996f310 msgr2=0x55bf699e1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:32.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 1 -- 172.21.15.97:0/572814816 shutdown_connections 2023-07-06T03:54:32.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 1 -- 172.21.15.97:0/572814816 wait complete. 2023-07-06T03:54:32.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.482+0000 7fb856bf2b80 20 client.8736 trim_cache size 0 max 16384 2023-07-06T03:54:32.491 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:54:32.491 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36", 2023-07-06T03:54:32.492 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-07-06T03:54:32.492 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8580, 2023-07-06T03:54:32.492 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-07-06T03:54:32.492 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-07-06T03:54:32.492 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-07-06T03:54:32.492 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.5222341339999996, 2023-07-06T03:54:32.492 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 80, 2023-07-06T03:54:32.493 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 112, 2023-07-06T03:54:32.493 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 112, 2023-07-06T03:54:32.493 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 12.037876981 2023-07-06T03:54:32.493 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:54:32.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 -- 172.21.15.97:0/3027724263 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 msgr2=0x7fb82006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 0x7fb82006d190 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0c9d60 tx=0x7fb830009040 comp rx=0 tx=0).stop 2023-07-06T03:54:32.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 -- 172.21.15.97:0/3027724263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c0d2210 msgr2=0x7fb83c0cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c0d2210 0x7fb83c0cc7c0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb828000c00 tx=0x7fb828004750 comp rx=0 tx=0).stop 2023-07-06T03:54:32.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 -- 172.21.15.97:0/3027724263 shutdown_connections 2023-07-06T03:54:32.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 0x7fb82006d190 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb83c15c610 0x7fb83c0d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb83c06b360 0x7fb83c157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 --2- 172.21.15.97:0/3027724263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c0d2210 0x7fb83c0cc7c0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 -- 172.21.15.97:0/3027724263 >> 172.21.15.97:0/3027724263 conn(0x7fb83c06a6c0 msgr2=0x7fb83c15b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:32.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 -- 172.21.15.97:0/3027724263 shutdown_connections 2023-07-06T03:54:32.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.483+0000 7fb8431ff700 1 -- 172.21.15.97:0/3027724263 wait complete. 2023-07-06T03:54:32.501 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:54:32.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.693+0000 7fd9497ff700 1 Processor -- start 2023-07-06T03:54:32.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.694+0000 7fd9497ff700 1 -- start start 2023-07-06T03:54:32.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9497ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd94415c520 0x7fd9440cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9497ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd944160d00 0x7fd9440b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9497ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9440cdae0 con 0x7fd944160180 2023-07-06T03:54:32.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9497ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9440cd8d0 con 0x7fd944160d00 2023-07-06T03:54:32.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9497ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9440b6640 con 0x7fd94415c520 2023-07-06T03:54:32.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9437fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd944160d00 0x7fd9440b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9437fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd944160d00 0x7fd9440b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34564/0 (socket says 172.21.15.97:34564) 2023-07-06T03:54:32.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9437fe700 1 -- 172.21.15.97:0/2550085536 learned_addr learned my addr 172.21.15.97:0/2550085536 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:32.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9437fe700 1 -- 172.21.15.97:0/2550085536 >> v2:172.21.15.146:3300/0 conn(0x7fd94415c520 msgr2=0x7fd9440cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:32.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9437fe700 1 --2- 172.21.15.97:0/2550085536 >> v2:172.21.15.146:3300/0 conn(0x7fd94415c520 0x7fd9440cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9437fe700 1 -- 172.21.15.97:0/2550085536 >> v1:172.21.15.146:6789/0 conn(0x7fd944160180 legacy=0x7fd9441605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:32.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.695+0000 7fd9437fe700 1 -- 172.21.15.97:0/2550085536 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9440b6920 con 0x7fd944160d00 2023-07-06T03:54:32.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.696+0000 7fd9437fe700 1 --2- 172.21.15.97:0/2550085536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd944160d00 0x7fd9440b7400 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd9440b4c70 tx=0x7fd93c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae51258de9c2c26 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:32.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.696+0000 7fd941ffb700 1 -- 172.21.15.97:0/2550085536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c00e070 con 0x7fd944160d00 2023-07-06T03:54:32.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.696+0000 7fd941ffb700 1 -- 172.21.15.97:0/2550085536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c003a20 con 0x7fd944160d00 2023-07-06T03:54:32.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.696+0000 7fd941ffb700 1 -- 172.21.15.97:0/2550085536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c003dd0 con 0x7fd944160d00 2023-07-06T03:54:32.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.696+0000 7fd9497ff700 1 -- 172.21.15.97:0/2550085536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd944160d00 msgr2=0x7fd9440b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.696+0000 7fd9497ff700 1 --2- 172.21.15.97:0/2550085536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd944160d00 0x7fd9440b7400 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd9440b4c70 tx=0x7fd93c00b330 comp rx=0 tx=0).stop 2023-07-06T03:54:32.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.696+0000 7fd9497ff700 1 -- 172.21.15.97:0/2550085536 shutdown_connections 2023-07-06T03:54:32.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.696+0000 7fd9497ff700 1 --2- 172.21.15.97:0/2550085536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd944160d00 0x7fd9440b7400 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.696+0000 7fd9497ff700 1 --2- 172.21.15.97:0/2550085536 >> v2:172.21.15.146:3300/0 conn(0x7fd94415c520 0x7fd9440cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.696+0000 7fd9497ff700 1 -- 172.21.15.97:0/2550085536 >> 172.21.15.97:0/2550085536 conn(0x7fd944071790 msgr2=0x7fd944071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:32.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.697+0000 7fd9497ff700 1 -- 172.21.15.97:0/2550085536 shutdown_connections 2023-07-06T03:54:32.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.697+0000 7fd9497ff700 1 -- 172.21.15.97:0/2550085536 wait complete. 2023-07-06T03:54:32.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.698+0000 7fd9497ff700 1 Processor -- start 2023-07-06T03:54:32.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.698+0000 7fd9497ff700 1 -- start start 2023-07-06T03:54:32.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.699+0000 7fd9497ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd94415c520 0x7fd9441e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.699+0000 7fd9497ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd944160180 0x7fd9440c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.699+0000 7fd9497ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd944160d00 0x7fd9440ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.699+0000 7fd9497ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9440c9310 con 0x7fd944160d00 2023-07-06T03:54:32.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.699+0000 7fd9497ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441e8810 con 0x7fd944160180 2023-07-06T03:54:32.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.699+0000 7fd9497ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94415b210 con 0x7fd94415c520 2023-07-06T03:54:32.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.699+0000 7fd9437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd944160d00 0x7fd9440ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd9437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd944160d00 0x7fd9440ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34578/0 (socket says 172.21.15.97:34578) 2023-07-06T03:54:32.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd9437fe700 1 -- 172.21.15.97:0/1117855586 learned_addr learned my addr 172.21.15.97:0/1117855586 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:32.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd942ffd700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd94415c520 0x7fd9441e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd9427fc700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd944160180 0x7fd9440c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd9437fe700 1 -- 172.21.15.97:0/1117855586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd94415c520 msgr2=0x7fd9441e82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd9437fe700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd94415c520 0x7fd9441e82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd9437fe700 1 -- 172.21.15.97:0/1117855586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd944160180 msgr2=0x7fd9440c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd9437fe700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd944160180 0x7fd9440c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd9437fe700 1 -- 172.21.15.97:0/1117855586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c009b30 con 0x7fd944160d00 2023-07-06T03:54:32.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd9437fe700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd944160d00 0x7fd9440ca350 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fd93c015040 tx=0x7fd93c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:32.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd92b7fe700 1 -- 172.21.15.97:0/1117855586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0094d0 con 0x7fd944160d00 2023-07-06T03:54:32.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd92b7fe700 1 -- 172.21.15.97:0/1117855586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c004240 con 0x7fd944160d00 2023-07-06T03:54:32.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd9497ff700 1 -- 172.21.15.97:0/1117855586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94415d390 con 0x7fd944160d00 2023-07-06T03:54:32.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.700+0000 7fd92b7fe700 1 -- 172.21.15.97:0/1117855586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0045f0 con 0x7fd944160d00 2023-07-06T03:54:32.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.701+0000 7fd9497ff700 1 -- 172.21.15.97:0/1117855586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94415d950 con 0x7fd944160d00 2023-07-06T03:54:32.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.701+0000 7fd9497ff700 1 -- 172.21.15.97:0/1117855586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94415d1f0 con 0x7fd944160d00 2023-07-06T03:54:32.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.702+0000 7fd92b7fe700 1 -- 172.21.15.97:0/1117855586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd93c019070 con 0x7fd944160d00 2023-07-06T03:54:32.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.703+0000 7fd92b7fe700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd93006ace0 0x7fd93006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:32.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.703+0000 7fd92b7fe700 1 -- 172.21.15.97:0/1117855586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd93c08a300 con 0x7fd944160d00 2023-07-06T03:54:32.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.704+0000 7fd942ffd700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd93006ace0 0x7fd93006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:32.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.704+0000 7fd942ffd700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd93006ace0 0x7fd93006d190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd934009970 tx=0x7fd934009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.708+0000 7fd92b7fe700 1 -- 172.21.15.97:0/1117855586 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd94415d1f0 con 0x7fd944160d00 2023-07-06T03:54:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.945+0000 7fd9497ff700 1 -- 172.21.15.97:0/1117855586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd94415d1f0 con 0x7fd944160d00 2023-07-06T03:54:32.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.945+0000 7fd92b7fe700 1 -- 172.21.15.97:0/1117855586 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd94415d1f0 con 0x7fd944160d00 2023-07-06T03:54:32.947 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:54:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.947+0000 7fd9497ff700 1 -- 172.21.15.97:0/1117855586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd93006ace0 msgr2=0x7fd93006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.947+0000 7fd9497ff700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd93006ace0 0x7fd93006d190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd934009970 tx=0x7fd934009110 comp rx=0 tx=0).stop 2023-07-06T03:54:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.947+0000 7fd9497ff700 1 -- 172.21.15.97:0/1117855586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd944160d00 msgr2=0x7fd9440ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.947+0000 7fd9497ff700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd944160d00 0x7fd9440ca350 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fd93c015040 tx=0x7fd93c004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.947+0000 7fd9497ff700 1 -- 172.21.15.97:0/1117855586 shutdown_connections 2023-07-06T03:54:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.947+0000 7fd9497ff700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd93006ace0 0x7fd93006d190 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.947+0000 7fd9497ff700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd944160d00 0x7fd9440ca350 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.947+0000 7fd9497ff700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd94415c520 0x7fd9441e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.947+0000 7fd9497ff700 1 --2- 172.21.15.97:0/1117855586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd944160180 0x7fd9440c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.947+0000 7fd9497ff700 1 -- 172.21.15.97:0/1117855586 >> 172.21.15.97:0/1117855586 conn(0x7fd944071790 msgr2=0x7fd944156b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.947+0000 7fd9497ff700 1 -- 172.21.15.97:0/1117855586 shutdown_connections 2023-07-06T03:54:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:32.948+0000 7fd9497ff700 1 -- 172.21.15.97:0/1117855586 wait complete. 2023-07-06T03:54:32.968 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:54:32.969 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:54:32.969 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:54:32.969 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:54:32.970 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:54:32.970 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:54:32.970 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:54:32.970 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:54:32.971 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:54:32.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:32.971 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:54:33.018 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:54:33.018 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:54:33.018 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:54:33.018 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:33.018 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:54:33.019 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:33.019 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:54:33.019 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:54:33.019 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:54:33.019 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:54:33.019 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:54:33.019 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:54:33.020 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:54:33.020 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:54:33.020 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:54:33.020 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:54:33.020 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:54:33.020 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41596sec preferred_lft 41596sec 2023-07-06T03:54:33.021 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:54:33.021 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:33.021 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:54:33.021 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:54:33.021 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:54:33.021 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:33.021 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:54:33.022 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:33.022 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:54:33.022 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:54:33.022 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:54:33.022 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:33.022 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:54:33.023 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:54:33.023 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:54:33.023 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:33.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:33.024 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:54:33.091 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:54:33.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:33.094 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:54:33.142 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:54:33.143 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:54:33.143 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:54:33.143 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:54:33.143 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:54:33.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:33.145 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:54:33.145 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:54:33.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:54:33.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:54:33.146 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:54:33.146 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:54:33.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:33.313 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:54:33.359 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:54:33.359 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:54:33.360 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:54:33.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:33.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:54:33.430 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:54:33.430 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:54:33.430 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:54:33.476 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:54:33.476 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:54:33.477 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:54:33.561 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:54:33.670 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:54:33.670 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:54:33.737 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:54:33.739 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:54:33.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:33.740 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:54:33.786 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:54:33.786 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:54:33.791 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:54:33.862 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:54:33.863 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:54:33.932 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:54:33.934 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:54:33.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:33.935 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:54:33.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:54:33.956+0000 7fe256d835c0 -1 init, newargv = 0x55918f6de160 newargc=15 2023-07-06T03:54:33.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[158938]: starting ceph client 2023-07-06T03:54:33.966 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[158938]: starting fuse 2023-07-06T03:54:33.984 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:54:33.984 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:54:33.985 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:54:33.985 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:54:33.985 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:54:33.986 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:54:33.986 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:54:33.986 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:33.986 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:54:33.987 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:54:33.987 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:54:33.987 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:54:33.988 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:33.988 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:54:33.988 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:54:33.988 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:54:33.989 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:54:33.989 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:33.989 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:54:33.990 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:54:33.990 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:54:33.990 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:54:33.991 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:54:33.991 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:54:33.991 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:54:33.991 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:54:33.992 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:33.992 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:54:33.992 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:54:34.167 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.158938.asok 2023-07-06T03:54:34.345 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.158938.asok 2023-07-06T03:54:34.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:34.346 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.158938.asok status 2023-07-06T03:54:34.568 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:54:34.568 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:54:34.568 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:54:34.569 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:54:34.569 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:54:34.569 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:54:34.569 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:54:34.569 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "158938", 2023-07-06T03:54:34.569 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:54:34.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:34.570 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:54:34.570 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:54:34.570 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8751, 2023-07-06T03:54:34.570 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:54:34.570 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:54:34.570 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:54:34.570 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 8751 2023-07-06T03:54:34.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:34.571 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:54:34.571 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:54:34.571 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:54:34.571 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 343344933 2023-07-06T03:54:34.571 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:34.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:34.571 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:54:34.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:54:34.572 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:54:34.572 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 343344933 2023-07-06T03:54:34.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:34.572 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.8751 192.168.0.1:0/343344933", 2023-07-06T03:54:34.572 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/343344933", 2023-07-06T03:54:34.572 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:54:34.572 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 80, 2023-07-06T03:54:34.573 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 112, 2023-07-06T03:54:34.573 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:54:34.573 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:54:34.573 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:54:34.573 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:54:34.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:34.582 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:54:34.582 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:54:34.627 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:54:34.628 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:54:34.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:34.628 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:54:34.697 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:54:34.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.889+0000 7f8b00dff700 1 Processor -- start 2023-07-06T03:54:34.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.889+0000 7f8b00dff700 1 -- start start 2023-07-06T03:54:34.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.891+0000 7f8b00dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8afc071a00 0x7f8afc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:34.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.891+0000 7f8b00dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8afc15ce90 0x7f8afc158d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:34.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.891+0000 7f8b00dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8afc06ddb0 con 0x7f8afc158960 2023-07-06T03:54:34.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.891+0000 7f8b00dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc15d310 con 0x7f8afc071a00 2023-07-06T03:54:34.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.891+0000 7f8b00dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc06bbf0 con 0x7f8afc15ce90 2023-07-06T03:54:34.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.891+0000 7f8afa59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8afc158960 0x7f8afc0712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37884/0 (socket says 172.21.15.97:37884) 2023-07-06T03:54:34.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.891+0000 7f8af9d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8afc071a00 0x7f8afc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:34.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.891+0000 7f8afa59c700 1 -- 172.21.15.97:0/2119564706 learned_addr learned my addr 172.21.15.97:0/2119564706 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:34.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.891+0000 7f8afad9d700 1 --2- 172.21.15.97:0/2119564706 >> v2:172.21.15.146:3300/0 conn(0x7f8afc15ce90 0x7f8afc158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:34.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.892+0000 7f8af959a700 1 -- 172.21.15.97:0/2119564706 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223360510 0 0) 0x7f8afc06ddb0 con 0x7f8afc158960 2023-07-06T03:54:34.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.892+0000 7f8af9d9b700 1 -- 172.21.15.97:0/2119564706 >> v2:172.21.15.146:3300/0 conn(0x7f8afc15ce90 msgr2=0x7f8afc158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:34.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.892+0000 7f8af9d9b700 1 --2- 172.21.15.97:0/2119564706 >> v2:172.21.15.146:3300/0 conn(0x7f8afc15ce90 0x7f8afc158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.892+0000 7f8af9d9b700 1 -- 172.21.15.97:0/2119564706 >> v1:172.21.15.97:6789/0 conn(0x7f8afc158960 legacy=0x7f8afc0712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.892+0000 7f8af9d9b700 1 -- 172.21.15.97:0/2119564706 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc06bed0 con 0x7f8afc071a00 2023-07-06T03:54:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.892+0000 7f8af9d9b700 1 --2- 172.21.15.97:0/2119564706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8afc071a00 0x7f8afc06d6f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0cb2e0 tx=0x7f8af0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e4ac4e964958bf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.892+0000 7f8af959a700 1 -- 172.21.15.97:0/2119564706 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0011830 con 0x7f8afc071a00 2023-07-06T03:54:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.892+0000 7f8af959a700 1 -- 172.21.15.97:0/2119564706 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af0011990 con 0x7f8afc071a00 2023-07-06T03:54:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.892+0000 7f8af959a700 1 -- 172.21.15.97:0/2119564706 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0011d20 con 0x7f8afc071a00 2023-07-06T03:54:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.893+0000 7f8b00dff700 1 -- 172.21.15.97:0/2119564706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8afc071a00 msgr2=0x7f8afc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:34.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.893+0000 7f8b00dff700 1 --2- 172.21.15.97:0/2119564706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8afc071a00 0x7f8afc06d6f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0cb2e0 tx=0x7f8af0009b20 comp rx=0 tx=0).stop 2023-07-06T03:54:34.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.893+0000 7f8b00dff700 1 -- 172.21.15.97:0/2119564706 shutdown_connections 2023-07-06T03:54:34.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.893+0000 7f8b00dff700 1 --2- 172.21.15.97:0/2119564706 >> v2:172.21.15.146:3300/0 conn(0x7f8afc15ce90 0x7f8afc158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:34.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.893+0000 7f8b00dff700 1 --2- 172.21.15.97:0/2119564706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8afc071a00 0x7f8afc06d6f0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:34.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.893+0000 7f8b00dff700 1 -- 172.21.15.97:0/2119564706 >> 172.21.15.97:0/2119564706 conn(0x7f8afc0ce8c0 msgr2=0x7f8afc0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:34.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.893+0000 7f8b00dff700 1 -- 172.21.15.97:0/2119564706 shutdown_connections 2023-07-06T03:54:34.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.893+0000 7f8b00dff700 1 -- 172.21.15.97:0/2119564706 wait complete. 2023-07-06T03:54:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.894+0000 7f8b00dff700 1 Processor -- start 2023-07-06T03:54:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.895+0000 7f8b00dff700 1 -- start start 2023-07-06T03:54:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.895+0000 7f8b00dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8afc071a00 0x7f8afc1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8b00dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8afc158960 0x7f8afc0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:34.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8b00dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8afc15ce90 0x7f8afc0afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8b00dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc151ef0 con 0x7f8afc158960 2023-07-06T03:54:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8b00dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc152160 con 0x7f8afc15ce90 2023-07-06T03:54:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8b00dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc152440 con 0x7f8afc071a00 2023-07-06T03:54:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8af9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8afc158960 0x7f8afc0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8af9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8afc158960 0x7f8afc0af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34606/0 (socket says 172.21.15.97:34606) 2023-07-06T03:54:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8af9d9b700 1 -- 172.21.15.97:0/1860627474 learned_addr learned my addr 172.21.15.97:0/1860627474 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:34.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8afa59c700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8afc071a00 0x7f8afc1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8afad9d700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8afc15ce90 0x7f8afc0afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8af9d9b700 1 -- 172.21.15.97:0/1860627474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8afc071a00 msgr2=0x7f8afc1511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8af9d9b700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8afc071a00 0x7f8afc1511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8af9d9b700 1 -- 172.21.15.97:0/1860627474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8afc15ce90 msgr2=0x7f8afc0afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8af9d9b700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8afc15ce90 0x7f8afc0afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:34.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.896+0000 7f8af9d9b700 1 -- 172.21.15.97:0/1860627474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af00094a0 con 0x7f8afc158960 2023-07-06T03:54:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.897+0000 7f8af9d9b700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8afc158960 0x7f8afc0af6c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f8afc06b980 tx=0x7f8af0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.897+0000 7f8aeaffd700 1 -- 172.21.15.97:0/1860627474 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af00036a0 con 0x7f8afc158960 2023-07-06T03:54:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.897+0000 7f8aeaffd700 1 -- 172.21.15.97:0/1860627474 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af001f070 con 0x7f8afc158960 2023-07-06T03:54:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.897+0000 7f8b00dff700 1 -- 172.21.15.97:0/1860627474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc152720 con 0x7f8afc158960 2023-07-06T03:54:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.897+0000 7f8aeaffd700 1 -- 172.21.15.97:0/1860627474 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0003a60 con 0x7f8afc158960 2023-07-06T03:54:34.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.897+0000 7f8b00dff700 1 -- 172.21.15.97:0/1860627474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc151700 con 0x7f8afc158960 2023-07-06T03:54:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.898+0000 7f8b00dff700 1 -- 172.21.15.97:0/1860627474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc067830 con 0x7f8afc158960 2023-07-06T03:54:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.899+0000 7f8aeaffd700 1 -- 172.21.15.97:0/1860627474 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8af0003bc0 con 0x7f8afc158960 2023-07-06T03:54:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.900+0000 7f8aeaffd700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8adc06ace0 0x7f8adc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.900+0000 7f8aeaffd700 1 -- 172.21.15.97:0/1860627474 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8af0021070 con 0x7f8afc158960 2023-07-06T03:54:34.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.900+0000 7f8afa59c700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8adc06ace0 0x7f8adc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:34.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.901+0000 7f8afa59c700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8adc06ace0 0x7f8adc06d190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8aec007fd0 tx=0x7f8aec007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:34.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:34.905+0000 7f8aeaffd700 1 -- 172.21.15.97:0/1860627474 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8af0055d50 con 0x7f8afc158960 2023-07-06T03:54:35.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.145+0000 7f8b00dff700 1 -- 172.21.15.97:0/1860627474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8afc04e340 con 0x7f8afc158960 2023-07-06T03:54:35.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.145+0000 7f8aeaffd700 1 -- 172.21.15.97:0/1860627474 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8af002b070 con 0x7f8afc158960 2023-07-06T03:54:35.147 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:54:35.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.147+0000 7f8b00dff700 1 -- 172.21.15.97:0/1860627474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8adc06ace0 msgr2=0x7f8adc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:35.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.147+0000 7f8b00dff700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8adc06ace0 0x7f8adc06d190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8aec007fd0 tx=0x7f8aec007de0 comp rx=0 tx=0).stop 2023-07-06T03:54:35.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.147+0000 7f8b00dff700 1 -- 172.21.15.97:0/1860627474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8afc158960 msgr2=0x7f8afc0af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:35.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.147+0000 7f8b00dff700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8afc158960 0x7f8afc0af6c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f8afc06b980 tx=0x7f8af0004750 comp rx=0 tx=0).stop 2023-07-06T03:54:35.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.147+0000 7f8b00dff700 1 -- 172.21.15.97:0/1860627474 shutdown_connections 2023-07-06T03:54:35.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.147+0000 7f8b00dff700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8adc06ace0 0x7f8adc06d190 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:35.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.147+0000 7f8b00dff700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8afc15ce90 0x7f8afc0afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:35.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.147+0000 7f8b00dff700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8afc071a00 0x7f8afc1511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:35.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.147+0000 7f8b00dff700 1 --2- 172.21.15.97:0/1860627474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8afc158960 0x7f8afc0af6c0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:35.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.147+0000 7f8b00dff700 1 -- 172.21.15.97:0/1860627474 >> 172.21.15.97:0/1860627474 conn(0x7f8afc0ce8c0 msgr2=0x7f8afc0cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:35.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.147+0000 7f8b00dff700 1 -- 172.21.15.97:0/1860627474 shutdown_connections 2023-07-06T03:54:35.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:35.148+0000 7f8b00dff700 1 -- 172.21.15.97:0/1860627474 wait complete. 2023-07-06T03:54:35.167 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:54:35.167 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:54:35.168 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2023-07-06T03:54:35.168 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T03:54:35.168 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T03:54:35.168 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:54:35.169 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:54:35.169 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:54:35.169 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:54:35.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:35.170 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2023-07-06T03:54:35.222 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:54:35.223 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:54:35.223 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:54:35.223 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:35.223 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:54:35.224 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:35.224 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:54:35.224 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:54:35.224 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2023-07-06T03:54:35.224 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:54:35.224 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:54:35.224 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2023-07-06T03:54:35.225 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:54:35.225 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2023-07-06T03:54:35.225 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:54:35.225 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2023-07-06T03:54:35.225 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:54:35.225 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 41609sec preferred_lft 41609sec 2023-07-06T03:54:35.225 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2023-07-06T03:54:35.225 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:35.226 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:54:35.226 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:54:35.226 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:54:35.226 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:35.226 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::58f0:6eff:fe04:3be0/64 scope link 2023-07-06T03:54:35.226 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:35.226 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:54:35.226 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:54:35.227 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::106d:9cff:fed5:9d0f/64 scope link 2023-07-06T03:54:35.227 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:35.227 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:54:35.227 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether a2:c5:04:8e:a2:a7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T03:54:35.227 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::a0c5:4ff:fe8e:a2a7/64 scope link 2023-07-06T03:54:35.227 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:54:35.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:35.228 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:54:35.298 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:54:35.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:35.325 DEBUG:teuthology.orchestra.run.smithi146:> route 2023-07-06T03:54:35.374 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2023-07-06T03:54:35.374 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:54:35.374 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:54:35.374 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:54:35.375 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:54:35.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:35.376 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:54:35.376 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:54:35.377 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:54:35.377 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:54:35.377 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:54:35.378 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:54:35.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:35.545 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:54:35.590 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T03:54:35.591 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T03:54:35.591 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:54:35.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:35.592 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2023-07-06T03:54:35.701 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-06T03:54:35.702 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:54:35.702 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T03:54:35.748 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:54:35.749 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:54:35.750 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:54:35.834 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:54:35.944 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:54:35.944 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:54:36.011 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:54:36.013 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:54:36.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:36.015 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:54:36.060 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:54:36.061 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-06T03:54:36.065 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:54:36.134 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:54:36.135 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:54:36.207 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:54:36.209 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:54:36.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:36.210 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:54:36.232 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2023-07-06T03:54:36.230+0000 7f8f262135c0 -1 init, newargv = 0x558cff278cc0 newargc=15 2023-07-06T03:54:36.232 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[125971]: starting ceph client 2023-07-06T03:54:36.239 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[125971]: starting fuse 2023-07-06T03:54:36.258 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:54:36.259 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T03:54:36.259 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:54:36.259 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2023-07-06T03:54:36.260 DEBUG:teuthology.orchestra.run.smithi146:> import re 2023-07-06T03:54:36.260 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:54:36.261 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2023-07-06T03:54:36.261 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:54:36.261 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2023-07-06T03:54:36.261 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-06T03:54:36.262 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2023-07-06T03:54:36.262 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-06T03:54:36.262 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:54:36.263 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2023-07-06T03:54:36.263 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2023-07-06T03:54:36.263 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2023-07-06T03:54:36.264 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2023-07-06T03:54:36.264 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:54:36.264 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2023-07-06T03:54:36.265 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:54:36.265 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:54:36.265 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:54:36.266 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2023-07-06T03:54:36.266 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2023-07-06T03:54:36.266 DEBUG:teuthology.orchestra.run.smithi146:> return f 2023-07-06T03:54:36.266 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:54:36.267 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:54:36.267 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2023-07-06T03:54:36.267 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:54:36.440 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.125971.asok 2023-07-06T03:54:36.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:54:36.597 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:54:36.618 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.125971.asok 2023-07-06T03:54:36.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:36.619 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.125971.asok status 2023-07-06T03:54:36.843 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-07-06T03:54:36.843 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2023-07-06T03:54:36.843 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:54:36.844 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:54:36.844 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2023-07-06T03:54:36.844 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2023-07-06T03:54:36.844 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-06T03:54:36.844 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "125971", 2023-07-06T03:54:36.844 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2023-07-06T03:54:36.845 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:54:36.845 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2023-07-06T03:54:36.845 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2023-07-06T03:54:36.845 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 6965, 2023-07-06T03:54:36.845 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2023-07-06T03:54:36.845 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2023-07-06T03:54:36.845 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2023-07-06T03:54:36.845 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 6965 2023-07-06T03:54:36.846 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:54:36.846 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:54:36.846 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:54:36.846 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:54:36.846 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 2393358819 2023-07-06T03:54:36.846 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-07-06T03:54:36.846 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:54:36.846 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:54:36.847 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:54:36.847 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:54:36.847 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 2393358819 2023-07-06T03:54:36.847 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:54:36.847 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.6965 192.168.0.1:0/2393358819", 2023-07-06T03:54:36.847 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/2393358819", 2023-07-06T03:54:36.847 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2023-07-06T03:54:36.847 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 80, 2023-07-06T03:54:36.848 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 112, 2023-07-06T03:54:36.848 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:54:36.848 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2023-07-06T03:54:36.848 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2023-07-06T03:54:36.848 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-07-06T03:54:36.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:36.858 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2023-07-06T03:54:36.859 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-06T03:54:36.944 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2023-07-06T03:54:36.945 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-06T03:54:36.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:36.945 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:54:37.014 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-07-06T03:54:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.218+0000 7f8d7d1ff700 1 Processor -- start 2023-07-06T03:54:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.218+0000 7f8d7d1ff700 1 -- start start 2023-07-06T03:54:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.219+0000 7f8d7d1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d7806cab0 0x7f8d780ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.219+0000 7f8d7d1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d780b0560 0x7f8d780b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.219+0000 7f8d7d1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d780ce560 con 0x7f8d78158b20 2023-07-06T03:54:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.219+0000 7f8d7d1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d780ce820 con 0x7f8d7806cab0 2023-07-06T03:54:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.219+0000 7f8d7d1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780b0f20 con 0x7f8d780b0560 2023-07-06T03:54:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.219+0000 7f8d7659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d7806cab0 0x7f8d780ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.220+0000 7f8d7659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d7806cab0 0x7f8d780ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34614/0 (socket says 172.21.15.97:34614) 2023-07-06T03:54:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.220+0000 7f8d7659c700 1 -- 172.21.15.97:0/2805906880 learned_addr learned my addr 172.21.15.97:0/2805906880 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.220+0000 7f8d7759e700 1 --2- 172.21.15.97:0/2805906880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d780b0560 0x7f8d780b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.220+0000 7f8d7659c700 1 -- 172.21.15.97:0/2805906880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d780b0560 msgr2=0x7f8d780b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.220+0000 7f8d7659c700 1 --2- 172.21.15.97:0/2805906880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d780b0560 0x7f8d780b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.220+0000 7f8d7659c700 1 -- 172.21.15.97:0/2805906880 >> v1:172.21.15.146:6789/0 conn(0x7f8d78158b20 legacy=0x7f8d780cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.220+0000 7f8d7659c700 1 -- 172.21.15.97:0/2805906880 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d780adcf0 con 0x7f8d7806cab0 2023-07-06T03:54:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.220+0000 7f8d7659c700 1 --2- 172.21.15.97:0/2805906880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d7806cab0 0x7f8d780ce020 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00bb90 tx=0x7f8d6c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7042193a4605f5ce server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.220+0000 7f8d75d9b700 1 -- 172.21.15.97:0/2805906880 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00dd50 con 0x7f8d7806cab0 2023-07-06T03:54:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.221+0000 7f8d75d9b700 1 -- 172.21.15.97:0/2805906880 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c010470 con 0x7f8d7806cab0 2023-07-06T03:54:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.221+0000 7f8d75d9b700 1 -- 172.21.15.97:0/2805906880 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c010800 con 0x7f8d7806cab0 2023-07-06T03:54:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.221+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2805906880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d7806cab0 msgr2=0x7f8d780ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.221+0000 7f8d7d1ff700 1 --2- 172.21.15.97:0/2805906880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d7806cab0 0x7f8d780ce020 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00bb90 tx=0x7f8d6c00bea0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.221+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2805906880 shutdown_connections 2023-07-06T03:54:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.221+0000 7f8d7d1ff700 1 --2- 172.21.15.97:0/2805906880 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d780b0560 0x7f8d780b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.221+0000 7f8d7d1ff700 1 --2- 172.21.15.97:0/2805906880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d7806cab0 0x7f8d780ce020 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.221+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2805906880 >> 172.21.15.97:0/2805906880 conn(0x7f8d7806a060 msgr2=0x7f8d7806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.221+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2805906880 shutdown_connections 2023-07-06T03:54:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.221+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2805906880 wait complete. 2023-07-06T03:54:37.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.224+0000 7f8d7d1ff700 1 Processor -- start 2023-07-06T03:54:37.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.224+0000 7f8d7d1ff700 1 -- start start 2023-07-06T03:54:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.225+0000 7f8d7d1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d7806cab0 0x7f8d78070d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.225+0000 7f8d7d1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b0560 0x7f8d781edde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.225+0000 7f8d7d1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d78158b20 0x7f8d7814bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.225+0000 7f8d7d1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d781eea30 con 0x7f8d780b0560 2023-07-06T03:54:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.225+0000 7f8d7d1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7806fcf0 con 0x7f8d7806cab0 2023-07-06T03:54:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.225+0000 7f8d7d1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7806ffd0 con 0x7f8d78158b20 2023-07-06T03:54:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.225+0000 7f8d7659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b0560 0x7f8d781edde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.225+0000 7f8d76d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d7806cab0 0x7f8d78070d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.225+0000 7f8d7659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b0560 0x7f8d781edde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34628/0 (socket says 172.21.15.97:34628) 2023-07-06T03:54:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.225+0000 7f8d7659c700 1 -- 172.21.15.97:0/2242824560 learned_addr learned my addr 172.21.15.97:0/2242824560 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.225+0000 7f8d7759e700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d78158b20 0x7f8d7814bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.226+0000 7f8d7659c700 1 -- 172.21.15.97:0/2242824560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d78158b20 msgr2=0x7f8d7814bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.226+0000 7f8d7659c700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d78158b20 0x7f8d7814bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.226+0000 7f8d7659c700 1 -- 172.21.15.97:0/2242824560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d7806cab0 msgr2=0x7f8d78070d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.226+0000 7f8d7659c700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d7806cab0 0x7f8d78070d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.226+0000 7f8d7659c700 1 -- 172.21.15.97:0/2242824560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c00b7f0 con 0x7f8d780b0560 2023-07-06T03:54:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.226+0000 7f8d7659c700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b0560 0x7f8d781edde0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00bee0 tx=0x7f8d6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.226+0000 7f8d5f7fe700 1 -- 172.21.15.97:0/2242824560 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c01f070 con 0x7f8d780b0560 2023-07-06T03:54:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.226+0000 7f8d5f7fe700 1 -- 172.21.15.97:0/2242824560 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c0038b0 con 0x7f8d780b0560 2023-07-06T03:54:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.226+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2242824560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d780702b0 con 0x7f8d780b0560 2023-07-06T03:54:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.226+0000 7f8d5f7fe700 1 -- 172.21.15.97:0/2242824560 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c003c80 con 0x7f8d780b0560 2023-07-06T03:54:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.226+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2242824560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d78070870 con 0x7f8d780b0560 2023-07-06T03:54:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.227+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2242824560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d78071d80 con 0x7f8d780b0560 2023-07-06T03:54:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.228+0000 7f8d5f7fe700 1 -- 172.21.15.97:0/2242824560 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d6c019400 con 0x7f8d780b0560 2023-07-06T03:54:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.229+0000 7f8d5f7fe700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d6406ace0 0x7f8d6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.229+0000 7f8d5f7fe700 1 -- 172.21.15.97:0/2242824560 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d6c08d090 con 0x7f8d780b0560 2023-07-06T03:54:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.229+0000 7f8d76d9d700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d6406ace0 0x7f8d6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.230+0000 7f8d76d9d700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d6406ace0 0x7f8d6406d190 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007fd0 tx=0x7f8d68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:37.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.234+0000 7f8d5f7fe700 1 -- 172.21.15.97:0/2242824560 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8d6c0580a0 con 0x7f8d780b0560 2023-07-06T03:54:37.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.430+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2242824560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7f8d780c4af0 con 0x7f8d780b0560 2023-07-06T03:54:37.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.434+0000 7f8d5f7fe700 1 -- 172.21.15.97:0/2242824560 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v45)=0 v45) v1 ==== 121+0+0 (secure 0 0 0) 0x7f8d6c05b6c0 con 0x7f8d780b0560 2023-07-06T03:54:37.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2242824560 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d6406ace0 msgr2=0x7f8d6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d6406ace0 0x7f8d6406d190 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007fd0 tx=0x7f8d68009040 comp rx=0 tx=0).stop 2023-07-06T03:54:37.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2242824560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b0560 msgr2=0x7f8d781edde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b0560 0x7f8d781edde0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00bee0 tx=0x7f8d6c004750 comp rx=0 tx=0).stop 2023-07-06T03:54:37.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2242824560 shutdown_connections 2023-07-06T03:54:37.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d6406ace0 0x7f8d6406d190 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d78158b20 0x7f8d7814bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d7806cab0 0x7f8d78070d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 --2- 172.21.15.97:0/2242824560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d780b0560 0x7f8d781edde0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2242824560 >> 172.21.15.97:0/2242824560 conn(0x7f8d7806a060 msgr2=0x7f8d7814e970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:37.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2242824560 shutdown_connections 2023-07-06T03:54:37.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.437+0000 7f8d7d1ff700 1 -- 172.21.15.97:0/2242824560 wait complete. 2023-07-06T03:54:37.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 100 2023-07-06T03:54:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.652+0000 7f6bd13ff700 1 Processor -- start 2023-07-06T03:54:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.652+0000 7f6bd13ff700 1 -- start start 2023-07-06T03:54:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.653+0000 7f6bd13ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6bcc0cbaa0 0x7f6bcc15a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.653+0000 7f6bd13ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6bcc15eb90 0x7f6bcc15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.653+0000 7f6bd13ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc15b6e0 con 0x7f6bcc154890 2023-07-06T03:54:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.653+0000 7f6bd13ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bcc15e0b0 con 0x7f6bcc15eb90 2023-07-06T03:54:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.653+0000 7f6bd13ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc15e390 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.653+0000 7f6bcb7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6bcc15eb90 0x7f6bcc15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.653+0000 7f6bcb7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6bcc15eb90 0x7f6bcc15ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34644/0 (socket says 172.21.15.97:34644) 2023-07-06T03:54:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.653+0000 7f6bcb7fe700 1 -- 172.21.15.97:0/1822593474 learned_addr learned my addr 172.21.15.97:0/1822593474 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:37.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.654+0000 7f6bca7fc700 1 --2- 172.21.15.97:0/1822593474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6bcc0cbaa0 0x7f6bcc15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.654+0000 7f6bcb7fe700 1 -- 172.21.15.97:0/1822593474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6bcc0cbaa0 msgr2=0x7f6bcc15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:37.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.654+0000 7f6bcb7fe700 1 --2- 172.21.15.97:0/1822593474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6bcc0cbaa0 0x7f6bcc15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.654+0000 7f6bcb7fe700 1 -- 172.21.15.97:0/1822593474 >> v1:172.21.15.146:6789/0 conn(0x7f6bcc154890 legacy=0x7f6bcc0cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.654+0000 7f6bcb7fe700 1 -- 172.21.15.97:0/1822593474 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc15e670 con 0x7f6bcc15eb90 2023-07-06T03:54:37.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.654+0000 7f6bcb7fe700 1 --2- 172.21.15.97:0/1822593474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bcc15eb90 0x7f6bcc15ae80 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009e50 tx=0x7f6bc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869fc08c29b7a21a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:37.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.654+0000 7f6bc9ffb700 1 -- 172.21.15.97:0/1822593474 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400e070 con 0x7f6bcc15eb90 2023-07-06T03:54:37.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.654+0000 7f6bc9ffb700 1 -- 172.21.15.97:0/1822593474 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc400b9c0 con 0x7f6bcc15eb90 2023-07-06T03:54:37.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.654+0000 7f6bc9ffb700 1 -- 172.21.15.97:0/1822593474 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400bd70 con 0x7f6bcc15eb90 2023-07-06T03:54:37.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.655+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1822593474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bcc15eb90 msgr2=0x7f6bcc15ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.655+0000 7f6bd13ff700 1 --2- 172.21.15.97:0/1822593474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bcc15eb90 0x7f6bcc15ae80 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009e50 tx=0x7f6bc400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.655+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1822593474 shutdown_connections 2023-07-06T03:54:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.655+0000 7f6bd13ff700 1 --2- 172.21.15.97:0/1822593474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bcc15eb90 0x7f6bcc15ae80 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.655+0000 7f6bd13ff700 1 --2- 172.21.15.97:0/1822593474 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6bcc0cbaa0 0x7f6bcc15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.655+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1822593474 >> 172.21.15.97:0/1822593474 conn(0x7f6bcc06a060 msgr2=0x7f6bcc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.655+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1822593474 shutdown_connections 2023-07-06T03:54:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.655+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1822593474 wait complete. 2023-07-06T03:54:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.657+0000 7f6bd13ff700 1 Processor -- start 2023-07-06T03:54:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.657+0000 7f6bd13ff700 1 -- start start 2023-07-06T03:54:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bd13ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bcc0cbaa0 0x7f6bcc151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bd13ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bcc154890 0x7f6bcc14f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bd13ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6bcc15eb90 0x7f6bcc15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bd13ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc14fce0 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bd13ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc1523c0 con 0x7f6bcc154890 2023-07-06T03:54:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bd13ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc1f5580 con 0x7f6bcc15eb90 2023-07-06T03:54:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bcaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bcc0cbaa0 0x7f6bcc151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bcaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bcc0cbaa0 0x7f6bcc151e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34650/0 (socket says 172.21.15.97:34650) 2023-07-06T03:54:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bcaffd700 1 -- 172.21.15.97:0/1901637475 learned_addr learned my addr 172.21.15.97:0/1901637475 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bcaffd700 1 -- 172.21.15.97:0/1901637475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6bcc15eb90 msgr2=0x7f6bcc15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bcb7fe700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6bcc15eb90 0x7f6bcc15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bca7fc700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bcc154890 0x7f6bcc14f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bcaffd700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6bcc15eb90 0x7f6bcc15cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bcaffd700 1 -- 172.21.15.97:0/1901637475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bcc154890 msgr2=0x7f6bcc14f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bcaffd700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bcc154890 0x7f6bcc14f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bcaffd700 1 -- 172.21.15.97:0/1901637475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc4009b30 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bcb7fe700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6bcc15eb90 0x7f6bcc15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.658+0000 7f6bca7fc700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bcc154890 0x7f6bcc14f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.659+0000 7f6bcaffd700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bcc0cbaa0 0x7f6bcc151e80 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00c4e0 tx=0x7f6bbc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:37.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.659+0000 7f6bb37fe700 1 -- 172.21.15.97:0/1901637475 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0052d0 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.659+0000 7f6bb37fe700 1 -- 172.21.15.97:0/1901637475 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bbc005430 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.659+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1901637475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc1f5860 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.659+0000 7f6bb37fe700 1 -- 172.21.15.97:0/1901637475 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0038c0 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.659+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1901637475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc1f5e20 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.660+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1901637475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bcc04e340 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.661+0000 7f6bb37fe700 1 -- 172.21.15.97:0/1901637475 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6bbc003a20 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.662+0000 7f6bb37fe700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6bb806ace0 0x7f6bb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:37.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.662+0000 7f6bb37fe700 1 -- 172.21.15.97:0/1901637475 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6bbc088970 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.662+0000 7f6bca7fc700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6bb806ace0 0x7f6bb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:37.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.663+0000 7f6bca7fc700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6bb806ace0 0x7f6bb806d190 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc1542a0 tx=0x7f6bc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:37.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.666+0000 7f6bb37fe700 1 -- 172.21.15.97:0/1901637475 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6bbc053a10 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.859+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1901637475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7f6bcc1f1bf0 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.863+0000 7f6bb37fe700 1 -- 172.21.15.97:0/1901637475 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v46)=0 v46) v1 ==== 119+0+0 (secure 0 0 0) 0x7f6bbc057030 con 0x7f6bcc0cbaa0 2023-07-06T03:54:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1901637475 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6bb806ace0 msgr2=0x7f6bb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6bb806ace0 0x7f6bb806d190 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc1542a0 tx=0x7f6bc0008040 comp rx=0 tx=0).stop 2023-07-06T03:54:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1901637475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bcc0cbaa0 msgr2=0x7f6bcc151e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bcc0cbaa0 0x7f6bcc151e80 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00c4e0 tx=0x7f6bbc00c8a0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1901637475 shutdown_connections 2023-07-06T03:54:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6bb806ace0 0x7f6bb806d190 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6bcc15eb90 0x7f6bcc15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bcc0cbaa0 0x7f6bcc151e80 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 --2- 172.21.15.97:0/1901637475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bcc154890 0x7f6bcc14f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1901637475 >> 172.21.15.97:0/1901637475 conn(0x7f6bcc06a060 msgr2=0x7f6bcc06a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1901637475 shutdown_connections 2023-07-06T03:54:37.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:37.866+0000 7f6bd13ff700 1 -- 172.21.15.97:0/1901637475 wait complete. 2023-07-06T03:54:37.886 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 200 2023-07-06T03:54:38.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.082+0000 7f5f00dff700 1 Processor -- start 2023-07-06T03:54:38.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.082+0000 7f5f00dff700 1 -- start start 2023-07-06T03:54:38.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.084+0000 7f5f00dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5efc06cab0 0x7f5efc0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.084+0000 7f5f00dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0ce560 con 0x7f5efc158b20 2023-07-06T03:54:38.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.084+0000 7f5f00dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0adcf0 con 0x7f5efc0b0560 2023-07-06T03:54:38.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.084+0000 7f5f00dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0ce820 con 0x7f5efc06cab0 2023-07-06T03:54:38.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.084+0000 7f5efa59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5efc158b20 0x7f5efc0cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37886/0 (socket says 172.21.15.97:37886) 2023-07-06T03:54:38.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.085+0000 7f5efa59c700 1 -- 172.21.15.97:0/3513022806 learned_addr learned my addr 172.21.15.97:0/3513022806 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:38.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.085+0000 7f5ef3fff700 1 --2- 172.21.15.97:0/3513022806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5efc06cab0 0x7f5efc0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.085+0000 7f5ef9d9b700 1 -- 172.21.15.97:0/3513022806 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669422451 0 0) 0x7f5efc0ce560 con 0x7f5efc158b20 2023-07-06T03:54:38.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.085+0000 7f5ef9d9b700 1 -- 172.21.15.97:0/3513022806 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee80035b0 con 0x7f5efc158b20 2023-07-06T03:54:38.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.085+0000 7f5ef9d9b700 1 -- 172.21.15.97:0/3513022806 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702724041 0 0) 0x7f5efc0adcf0 con 0x7f5efc0b0560 2023-07-06T03:54:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.085+0000 7f5ef9d9b700 1 -- 172.21.15.97:0/3513022806 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5efc0ce560 con 0x7f5efc0b0560 2023-07-06T03:54:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.085+0000 7f5ef3fff700 1 -- 172.21.15.97:0/3513022806 >> v1:172.21.15.146:6789/0 conn(0x7f5efc0b0560 legacy=0x7f5efc06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.085+0000 7f5ef3fff700 1 -- 172.21.15.97:0/3513022806 >> v1:172.21.15.97:6789/0 conn(0x7f5efc158b20 legacy=0x7f5efc0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.085+0000 7f5ef9d9b700 1 -- 172.21.15.97:0/3513022806 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 834261385 0 0) 0x7f5ee80035b0 con 0x7f5efc158b20 2023-07-06T03:54:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.085+0000 7f5ef3fff700 1 -- 172.21.15.97:0/3513022806 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0adfd0 con 0x7f5efc06cab0 2023-07-06T03:54:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.086+0000 7f5ef3fff700 1 --2- 172.21.15.97:0/3513022806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5efc06cab0 0x7f5efc0ce020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009870 tx=0x7f5ee4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b617a8483df6866e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:38.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.086+0000 7f5ef9d9b700 1 -- 172.21.15.97:0/3513022806 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec001670 con 0x7f5efc06cab0 2023-07-06T03:54:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.086+0000 7f5ef9d9b700 1 -- 172.21.15.97:0/3513022806 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee4004030 con 0x7f5efc06cab0 2023-07-06T03:54:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.086+0000 7f5ef9d9b700 1 -- 172.21.15.97:0/3513022806 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40043a0 con 0x7f5efc06cab0 2023-07-06T03:54:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.086+0000 7f5f00dff700 1 -- 172.21.15.97:0/3513022806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5efc06cab0 msgr2=0x7f5efc0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.086+0000 7f5f00dff700 1 --2- 172.21.15.97:0/3513022806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5efc06cab0 0x7f5efc0ce020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009870 tx=0x7f5ee4009b80 comp rx=0 tx=0).stop 2023-07-06T03:54:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.086+0000 7f5f00dff700 1 -- 172.21.15.97:0/3513022806 shutdown_connections 2023-07-06T03:54:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.086+0000 7f5f00dff700 1 --2- 172.21.15.97:0/3513022806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5efc06cab0 0x7f5efc0ce020 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.086+0000 7f5f00dff700 1 -- 172.21.15.97:0/3513022806 >> 172.21.15.97:0/3513022806 conn(0x7f5efc06a060 msgr2=0x7f5efc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.087+0000 7f5f00dff700 1 -- 172.21.15.97:0/3513022806 shutdown_connections 2023-07-06T03:54:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.087+0000 7f5f00dff700 1 -- 172.21.15.97:0/3513022806 wait complete. 2023-07-06T03:54:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.088+0000 7f5f00dff700 1 Processor -- start 2023-07-06T03:54:38.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.088+0000 7f5f00dff700 1 -- start start 2023-07-06T03:54:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.089+0000 7f5f00dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5efc06cab0 0x7f5efc1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.089+0000 7f5f00dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5efc0b0560 0x7f5efc14b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.089+0000 7f5f00dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5efc158b20 0x7f5efc0af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.089+0000 7f5f00dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc159070 con 0x7f5efc0b0560 2023-07-06T03:54:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.089+0000 7f5f00dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc159880 con 0x7f5efc06cab0 2023-07-06T03:54:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.089+0000 7f5f00dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0709b0 con 0x7f5efc158b20 2023-07-06T03:54:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.090+0000 7f5ef3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5efc0b0560 0x7f5efc14b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.090+0000 7f5ef3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5efc0b0560 0x7f5efc14b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34654/0 (socket says 172.21.15.97:34654) 2023-07-06T03:54:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.090+0000 7f5ef3fff700 1 -- 172.21.15.97:0/4116495205 learned_addr learned my addr 172.21.15.97:0/4116495205 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.090+0000 7f5efa59c700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5efc06cab0 0x7f5efc1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.090+0000 7f5efad9d700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5efc158b20 0x7f5efc0af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.090+0000 7f5ef3fff700 1 -- 172.21.15.97:0/4116495205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5efc158b20 msgr2=0x7f5efc0af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.090+0000 7f5ef3fff700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5efc158b20 0x7f5efc0af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.090+0000 7f5ef3fff700 1 -- 172.21.15.97:0/4116495205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5efc06cab0 msgr2=0x7f5efc1547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.090+0000 7f5ef3fff700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5efc06cab0 0x7f5efc1547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.090+0000 7f5ef3fff700 1 -- 172.21.15.97:0/4116495205 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee40094a0 con 0x7f5efc0b0560 2023-07-06T03:54:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.090+0000 7f5ef3fff700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5efc0b0560 0x7f5efc14b820 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009ed0 tx=0x7f5ee4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.091+0000 7f5ef2ffd700 1 -- 172.21.15.97:0/4116495205 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee401f070 con 0x7f5efc0b0560 2023-07-06T03:54:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.091+0000 7f5ef2ffd700 1 -- 172.21.15.97:0/4116495205 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee4004550 con 0x7f5efc0b0560 2023-07-06T03:54:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.091+0000 7f5f00dff700 1 -- 172.21.15.97:0/4116495205 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc1592c0 con 0x7f5efc0b0560 2023-07-06T03:54:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.091+0000 7f5ef2ffd700 1 -- 172.21.15.97:0/4116495205 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4010630 con 0x7f5efc0b0560 2023-07-06T03:54:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.091+0000 7f5f00dff700 1 -- 172.21.15.97:0/4116495205 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc154d10 con 0x7f5efc0b0560 2023-07-06T03:54:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.092+0000 7f5f00dff700 1 -- 172.21.15.97:0/4116495205 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc04e340 con 0x7f5efc0b0560 2023-07-06T03:54:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.093+0000 7f5ef2ffd700 1 -- 172.21.15.97:0/4116495205 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5ee4010790 con 0x7f5efc0b0560 2023-07-06T03:54:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.094+0000 7f5ef2ffd700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5edc06ace0 0x7f5edc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.094+0000 7f5ef2ffd700 1 -- 172.21.15.97:0/4116495205 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ee408a130 con 0x7f5efc0b0560 2023-07-06T03:54:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.094+0000 7f5efa59c700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5edc06ace0 0x7f5edc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.094+0000 7f5efa59c700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5edc06ace0 0x7f5edc06d190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5eec004a60 tx=0x7f5eec000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.098+0000 7f5ef2ffd700 1 -- 172.21.15.97:0/4116495205 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5ee4055260 con 0x7f5efc0b0560 2023-07-06T03:54:38.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.294+0000 7f5f00dff700 1 -- 172.21.15.97:0/4116495205 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7f5efc0adc60 con 0x7f5efc0b0560 2023-07-06T03:54:38.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.298+0000 7f5ef2ffd700 1 -- 172.21.15.97:0/4116495205 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v47)=0 v47) v1 ==== 128+0+0 (secure 0 0 0) 0x7f5ee4058880 con 0x7f5efc0b0560 2023-07-06T03:54:38.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 -- 172.21.15.97:0/4116495205 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5edc06ace0 msgr2=0x7f5edc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5edc06ace0 0x7f5edc06d190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5eec004a60 tx=0x7f5eec000bc0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 -- 172.21.15.97:0/4116495205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5efc0b0560 msgr2=0x7f5efc14b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5efc0b0560 0x7f5efc14b820 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009ed0 tx=0x7f5ee4004750 comp rx=0 tx=0).stop 2023-07-06T03:54:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 -- 172.21.15.97:0/4116495205 shutdown_connections 2023-07-06T03:54:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5edc06ace0 0x7f5edc06d190 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5efc158b20 0x7f5efc0af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5efc06cab0 0x7f5efc1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 --2- 172.21.15.97:0/4116495205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5efc0b0560 0x7f5efc14b820 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 -- 172.21.15.97:0/4116495205 >> 172.21.15.97:0/4116495205 conn(0x7f5efc06a060 msgr2=0x7f5efc0c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 -- 172.21.15.97:0/4116495205 shutdown_connections 2023-07-06T03:54:38.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.301+0000 7f5f00dff700 1 -- 172.21.15.97:0/4116495205 wait complete. 2023-07-06T03:54:38.320 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-07-06T03:54:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.516+0000 7ff2b39ff700 1 Processor -- start 2023-07-06T03:54:38.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.517+0000 7ff2b39ff700 1 -- start start 2023-07-06T03:54:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.518+0000 7ff2b39ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac06b360 0x7ff2ac0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.518+0000 7ff2b39ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff2ac0d2210 0x7ff2ac0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.518+0000 7ff2b39ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac0cff90 con 0x7ff2ac15c610 2023-07-06T03:54:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.518+0000 7ff2b39ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2ac0d0780 con 0x7ff2ac0d2210 2023-07-06T03:54:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.518+0000 7ff2b39ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac15c9e0 con 0x7ff2ac06b360 2023-07-06T03:54:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.518+0000 7ff2b179b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff2ac15c610 0x7ff2ac0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37892/0 (socket says 172.21.15.97:37892) 2023-07-06T03:54:38.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.518+0000 7ff2b179b700 1 -- 172.21.15.97:0/868197815 learned_addr learned my addr 172.21.15.97:0/868197815 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.518+0000 7ff2b0f9a700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac06b360 0x7ff2ac0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.518+0000 7ff2b1f9c700 1 --2- 172.21.15.97:0/868197815 >> v2:172.21.15.146:3300/0 conn(0x7ff2ac0d2210 0x7ff2ac0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.518+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962389954 0 0) 0x7ff2ac0cff90 con 0x7ff2ac15c610 2023-07-06T03:54:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.518+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2980035b0 con 0x7ff2ac15c610 2023-07-06T03:54:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3127916128 0 0) 0x7ff2980035b0 con 0x7ff2ac15c610 2023-07-06T03:54:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac06b360 msgr2=0x7ff2ac0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac06b360 0x7ff2ac0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 >> v2:172.21.15.146:3300/0 conn(0x7ff2ac0d2210 msgr2=0x7ff2ac0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 --2- 172.21.15.97:0/868197815 >> v2:172.21.15.146:3300/0 conn(0x7ff2ac0d2210 0x7ff2ac0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac0b5650 con 0x7ff2ac15c610 2023-07-06T03:54:38.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7ff2a8001670 con 0x7ff2ac15c610 2023-07-06T03:54:38.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 >> v1:172.21.15.97:6789/0 conn(0x7ff2ac15c610 legacy=0x7ff2ac0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac06b360 0x7ff298003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff298003fb0 0x7ff298004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff298004d90 0x7ff298005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0d0780 con 0x7ff298004d90 2023-07-06T03:54:38.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac15c9e0 con 0x7ff298003fb0 2023-07-06T03:54:38.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29800ba10 con 0x7ff2ac06b360 2023-07-06T03:54:38.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2b1f9c700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff298004d90 0x7ff298005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2b179b700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac06b360 0x7ff298003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2b0f9a700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff298003fb0 0x7ff298004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2b1f9c700 1 -- 172.21.15.97:0/868197815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac06b360 msgr2=0x7ff298003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:38.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2b1f9c700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac06b360 0x7ff298003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2b1f9c700 1 -- 172.21.15.97:0/868197815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff298003fb0 msgr2=0x7ff298004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:38.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.519+0000 7ff2b1f9c700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff298003fb0 0x7ff298004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.520+0000 7ff2b1f9c700 1 -- 172.21.15.97:0/868197815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2980035b0 con 0x7ff298004d90 2023-07-06T03:54:38.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.520+0000 7ff2b1f9c700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff298004d90 0x7ff298005200 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009fc0 tx=0x7ff298005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e5d9a9bcd54f78 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:38.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.520+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400ec30 con 0x7ff298004d90 2023-07-06T03:54:38.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.520+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a400f040 con 0x7ff298004d90 2023-07-06T03:54:38.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.520+0000 7ff2a3fff700 1 -- 172.21.15.97:0/868197815 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4014480 con 0x7ff298004d90 2023-07-06T03:54:38.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.520+0000 7ff2b39ff700 1 -- 172.21.15.97:0/868197815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff298004d90 msgr2=0x7ff298005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.520+0000 7ff2b39ff700 1 --2- 172.21.15.97:0/868197815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff298004d90 0x7ff298005200 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009fc0 tx=0x7ff298005880 comp rx=0 tx=0).stop 2023-07-06T03:54:38.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.520+0000 7ff2b1f9c700 1 -- 172.21.15.97:0/868197815 reap_dead start 2023-07-06T03:54:38.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.521+0000 7ff2b39ff700 1 -- 172.21.15.97:0/868197815 shutdown_connections 2023-07-06T03:54:38.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.521+0000 7ff2b39ff700 1 -- 172.21.15.97:0/868197815 >> 172.21.15.97:0/868197815 conn(0x7ff2ac06a6c0 msgr2=0x7ff2ac06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:38.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.521+0000 7ff2b39ff700 1 -- 172.21.15.97:0/868197815 shutdown_connections 2023-07-06T03:54:38.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.521+0000 7ff2b39ff700 1 -- 172.21.15.97:0/868197815 wait complete. 2023-07-06T03:54:38.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.522+0000 7ff2b39ff700 1 Processor -- start 2023-07-06T03:54:38.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.523+0000 7ff2b39ff700 1 -- start start 2023-07-06T03:54:38.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b39ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff298004d90 0x7ff2ac1f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b39ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac1f26f0 0x7ff2ac0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b39ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2ac0cc200 0x7ff2ac0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b39ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac153a80 con 0x7ff2ac0cc200 2023-07-06T03:54:38.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b39ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0b4130 con 0x7ff298004d90 2023-07-06T03:54:38.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b39ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0b3090 con 0x7ff2ac1f26f0 2023-07-06T03:54:38.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b1f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2ac0cc200 0x7ff2ac0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b1f9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2ac0cc200 0x7ff2ac0cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34672/0 (socket says 172.21.15.97:34672) 2023-07-06T03:54:38.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b1f9c700 1 -- 172.21.15.97:0/2000705198 learned_addr learned my addr 172.21.15.97:0/2000705198 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:38.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b0f9a700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac1f26f0 0x7ff2ac0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b179b700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff298004d90 0x7ff2ac1f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b1f9c700 1 -- 172.21.15.97:0/2000705198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac1f26f0 msgr2=0x7ff2ac0b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b1f9c700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac1f26f0 0x7ff2ac0b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b1f9c700 1 -- 172.21.15.97:0/2000705198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff298004d90 msgr2=0x7ff2ac1f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b1f9c700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff298004d90 0x7ff2ac1f14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.524+0000 7ff2b1f9c700 1 -- 172.21.15.97:0/2000705198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac0b3370 con 0x7ff2ac0cc200 2023-07-06T03:54:38.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.525+0000 7ff2b1f9c700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2ac0cc200 0x7ff2ac0cc680 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4017040 tx=0x7ff2a4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:38.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.525+0000 7ff2a1ffb700 1 -- 172.21.15.97:0/2000705198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4007000 con 0x7ff2ac0cc200 2023-07-06T03:54:38.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.525+0000 7ff2a1ffb700 1 -- 172.21.15.97:0/2000705198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a4004470 con 0x7ff2ac0cc200 2023-07-06T03:54:38.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.525+0000 7ff2b39ff700 1 -- 172.21.15.97:0/2000705198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac0b3650 con 0x7ff2ac0cc200 2023-07-06T03:54:38.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.525+0000 7ff2a1ffb700 1 -- 172.21.15.97:0/2000705198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4004820 con 0x7ff2ac0cc200 2023-07-06T03:54:38.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.525+0000 7ff2b39ff700 1 -- 172.21.15.97:0/2000705198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac0b3c10 con 0x7ff2ac0cc200 2023-07-06T03:54:38.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.526+0000 7ff2b39ff700 1 -- 172.21.15.97:0/2000705198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac04e340 con 0x7ff2ac0cc200 2023-07-06T03:54:38.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.527+0000 7ff2a1ffb700 1 -- 172.21.15.97:0/2000705198 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff2a4024070 con 0x7ff2ac0cc200 2023-07-06T03:54:38.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.528+0000 7ff2a1ffb700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff29006ace0 0x7ff29006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.528+0000 7ff2a1ffb700 1 -- 172.21.15.97:0/2000705198 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff2a408a7e0 con 0x7ff2ac0cc200 2023-07-06T03:54:38.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.528+0000 7ff2b179b700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff29006ace0 0x7ff29006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.529+0000 7ff2b179b700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff29006ace0 0x7ff29006d190 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac15d580 tx=0x7ff2a80016b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:38.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.532+0000 7ff2a1ffb700 1 -- 172.21.15.97:0/2000705198 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff2a4055880 con 0x7ff2ac0cc200 2023-07-06T03:54:38.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.729+0000 7ff2b39ff700 1 -- 172.21.15.97:0/2000705198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7ff2ac0d0c40 con 0x7ff2ac0cc200 2023-07-06T03:54:38.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.730+0000 7ff2a1ffb700 1 -- 172.21.15.97:0/2000705198 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v47) v1 ==== 106+0+4 (secure 0 0 0) 0x7ff2a4058ea0 con 0x7ff2ac0cc200 2023-07-06T03:54:38.732 INFO:teuthology.orchestra.run.smithi097.stdout:100 2023-07-06T03:54:38.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 -- 172.21.15.97:0/2000705198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff29006ace0 msgr2=0x7ff29006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff29006ace0 0x7ff29006d190 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac15d580 tx=0x7ff2a80016b0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 -- 172.21.15.97:0/2000705198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2ac0cc200 msgr2=0x7ff2ac0cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2ac0cc200 0x7ff2ac0cc680 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4017040 tx=0x7ff2a4009750 comp rx=0 tx=0).stop 2023-07-06T03:54:38.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 -- 172.21.15.97:0/2000705198 shutdown_connections 2023-07-06T03:54:38.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff29006ace0 0x7ff29006d190 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2ac0cc200 0x7ff2ac0cc680 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff298004d90 0x7ff2ac1f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 --2- 172.21.15.97:0/2000705198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff2ac1f26f0 0x7ff2ac0b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 -- 172.21.15.97:0/2000705198 >> 172.21.15.97:0/2000705198 conn(0x7ff2ac06a6c0 msgr2=0x7ff2ac0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 -- 172.21.15.97:0/2000705198 shutdown_connections 2023-07-06T03:54:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.732+0000 7ff2b39ff700 1 -- 172.21.15.97:0/2000705198 wait complete. 2023-07-06T03:54:38.752 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2023-07-06T03:54:38.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.948+0000 7fb3519ff700 1 Processor -- start 2023-07-06T03:54:38.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.949+0000 7fb3519ff700 1 -- start start 2023-07-06T03:54:38.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.950+0000 7fb3519ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb34c06b360 0x7fb34c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.950+0000 7fb3519ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34c0cff90 con 0x7fb34c0d2210 2023-07-06T03:54:38.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.950+0000 7fb3519ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34c0b5650 con 0x7fb34c15c610 2023-07-06T03:54:38.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.950+0000 7fb3519ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0d0780 con 0x7fb34c06b360 2023-07-06T03:54:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.950+0000 7fb34b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb34c0d2210 0x7fb34c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37896/0 (socket says 172.21.15.97:37896) 2023-07-06T03:54:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.950+0000 7fb34b7fe700 1 -- 172.21.15.97:0/880423926 learned_addr learned my addr 172.21.15.97:0/880423926 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.950+0000 7fb343fff700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb34c06b360 0x7fb34c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143653568 0 0) 0x7fb34c0cff90 con 0x7fb34c0d2210 2023-07-06T03:54:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3380035b0 con 0x7fb34c0d2210 2023-07-06T03:54:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739745488 0 0) 0x7fb34c0b5650 con 0x7fb34c15c610 2023-07-06T03:54:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb34c0cff90 con 0x7fb34c15c610 2023-07-06T03:54:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3240977656 0 0) 0x7fb3380035b0 con 0x7fb34c0d2210 2023-07-06T03:54:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb34c06b360 msgr2=0x7fb34c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb34c06b360 0x7fb34c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 >> v1:172.21.15.146:6789/0 conn(0x7fb34c15c610 legacy=0x7fb34c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c0b5400 con 0x7fb34c0d2210 2023-07-06T03:54:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fb3440019f0 con 0x7fb34c0d2210 2023-07-06T03:54:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 >> v1:172.21.15.97:6789/0 conn(0x7fb34c0d2210 legacy=0x7fb34c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb338003bc0 0x7fb338003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 0x7fb338004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb338005200 0x7fb33800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0d0780 con 0x7fb338004850 2023-07-06T03:54:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33800c030 con 0x7fb338003bc0 2023-07-06T03:54:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33800c1f0 con 0x7fb338005200 2023-07-06T03:54:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb343fff700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb34c06b360 0x7fb34c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:54:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34b7fe700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb338005200 0x7fb33800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb343fff700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 0x7fb338004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.951+0000 7fb34affd700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb338003bc0 0x7fb338003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.952+0000 7fb343fff700 1 -- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb338005200 msgr2=0x7fb33800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.952+0000 7fb343fff700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb338005200 0x7fb33800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.952+0000 7fb343fff700 1 -- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb338003bc0 msgr2=0x7fb338003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.952+0000 7fb343fff700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb338003bc0 0x7fb338003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.952+0000 7fb343fff700 1 -- 172.21.15.97:0/880423926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380035b0 con 0x7fb338004850 2023-07-06T03:54:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.952+0000 7fb343fff700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 0x7fb338004cc0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fb34c08bd00 tx=0x7fb338005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd98bc6b3694ed5e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.952+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334004030 con 0x7fb338004850 2023-07-06T03:54:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.952+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb334004190 con 0x7fb338004850 2023-07-06T03:54:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.952+0000 7fb34a7fc700 1 -- 172.21.15.97:0/880423926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334004540 con 0x7fb338004850 2023-07-06T03:54:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.953+0000 7fb3519ff700 1 -- 172.21.15.97:0/880423926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 msgr2=0x7fb338004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.953+0000 7fb3519ff700 1 --2- 172.21.15.97:0/880423926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 0x7fb338004cc0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fb34c08bd00 tx=0x7fb338005880 comp rx=0 tx=0).stop 2023-07-06T03:54:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.953+0000 7fb34b7fe700 1 -- 172.21.15.97:0/880423926 reap_dead start 2023-07-06T03:54:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.953+0000 7fb3519ff700 1 -- 172.21.15.97:0/880423926 shutdown_connections 2023-07-06T03:54:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.953+0000 7fb3519ff700 1 -- 172.21.15.97:0/880423926 >> 172.21.15.97:0/880423926 conn(0x7fb34c06a6c0 msgr2=0x7fb34c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.953+0000 7fb3519ff700 1 -- 172.21.15.97:0/880423926 shutdown_connections 2023-07-06T03:54:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.953+0000 7fb3519ff700 1 -- 172.21.15.97:0/880423926 wait complete. 2023-07-06T03:54:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.955+0000 7fb3519ff700 1 Processor -- start 2023-07-06T03:54:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.955+0000 7fb3519ff700 1 -- start start 2023-07-06T03:54:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb3519ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 0x7fb34c0cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb3519ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb34c14f540 0x7fb34c0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb3519ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb34c0b6010 0x7fb34c0b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb3519ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0cc860 con 0x7fb338004850 2023-07-06T03:54:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb3519ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0b69c0 con 0x7fb34c0b6010 2023-07-06T03:54:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb3519ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0b6e20 con 0x7fb34c14f540 2023-07-06T03:54:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb34affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 0x7fb34c0cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb34affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 0x7fb34c0cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34684/0 (socket says 172.21.15.97:34684) 2023-07-06T03:54:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb34affd700 1 -- 172.21.15.97:0/3639923767 learned_addr learned my addr 172.21.15.97:0/3639923767 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb34affd700 1 -- 172.21.15.97:0/3639923767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb34c14f540 msgr2=0x7fb34c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb34affd700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb34c14f540 0x7fb34c0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb34affd700 1 -- 172.21.15.97:0/3639923767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb34c0b6010 msgr2=0x7fb34c0b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb34affd700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb34c0b6010 0x7fb34c0b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.956+0000 7fb34affd700 1 -- 172.21.15.97:0/3639923767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c0cb860 con 0x7fb338004850 2023-07-06T03:54:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.957+0000 7fb34affd700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 0x7fb34c0cc320 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00f840 tx=0x7fb33c00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.957+0000 7fb348ff9700 1 -- 172.21.15.97:0/3639923767 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c005320 con 0x7fb338004850 2023-07-06T03:54:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.957+0000 7fb348ff9700 1 -- 172.21.15.97:0/3639923767 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c0054a0 con 0x7fb338004850 2023-07-06T03:54:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.957+0000 7fb3519ff700 1 -- 172.21.15.97:0/3639923767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c15ecc0 con 0x7fb338004850 2023-07-06T03:54:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.957+0000 7fb348ff9700 1 -- 172.21.15.97:0/3639923767 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c006aa0 con 0x7fb338004850 2023-07-06T03:54:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.957+0000 7fb3519ff700 1 -- 172.21.15.97:0/3639923767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34c0cbb20 con 0x7fb338004850 2023-07-06T03:54:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.958+0000 7fb3519ff700 1 -- 172.21.15.97:0/3639923767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34c15c9e0 con 0x7fb338004850 2023-07-06T03:54:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.959+0000 7fb348ff9700 1 -- 172.21.15.97:0/3639923767 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb33c006c00 con 0x7fb338004850 2023-07-06T03:54:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.960+0000 7fb348ff9700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb32c06ace0 0x7fb32c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.960+0000 7fb348ff9700 1 -- 172.21.15.97:0/3639923767 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb33c0889f0 con 0x7fb338004850 2023-07-06T03:54:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.960+0000 7fb343fff700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb32c06ace0 0x7fb32c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.961+0000 7fb343fff700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb32c06ace0 0x7fb32c06d190 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb34c152ef0 tx=0x7fb334013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:38.965+0000 7fb348ff9700 1 -- 172.21.15.97:0/3639923767 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb33c053aa0 con 0x7fb338004850 2023-07-06T03:54:39.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.162+0000 7fb3519ff700 1 -- 172.21.15.97:0/3639923767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7fb34c04e340 con 0x7fb338004850 2023-07-06T03:54:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.167+0000 7fb348ff9700 1 -- 172.21.15.97:0/3639923767 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v48)=0 v48) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb33c0570c0 con 0x7fb338004850 2023-07-06T03:54:39.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 -- 172.21.15.97:0/3639923767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb32c06ace0 msgr2=0x7fb32c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:39.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb32c06ace0 0x7fb32c06d190 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb34c152ef0 tx=0x7fb334013000 comp rx=0 tx=0).stop 2023-07-06T03:54:39.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 -- 172.21.15.97:0/3639923767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 msgr2=0x7fb34c0cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:39.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 0x7fb34c0cc320 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00f840 tx=0x7fb33c00fc00 comp rx=0 tx=0).stop 2023-07-06T03:54:39.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 -- 172.21.15.97:0/3639923767 shutdown_connections 2023-07-06T03:54:39.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb32c06ace0 0x7fb32c06d190 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb34c0b6010 0x7fb34c0b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb338004850 0x7fb34c0cc320 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 --2- 172.21.15.97:0/3639923767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb34c14f540 0x7fb34c0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 -- 172.21.15.97:0/3639923767 >> 172.21.15.97:0/3639923767 conn(0x7fb34c06a6c0 msgr2=0x7fb34c159330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:39.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 -- 172.21.15.97:0/3639923767 shutdown_connections 2023-07-06T03:54:39.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.169+0000 7fb3519ff700 1 -- 172.21.15.97:0/3639923767 wait complete. 2023-07-06T03:54:39.188 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2023-07-06T03:54:39.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.384+0000 7fcee11ff700 1 Processor -- start 2023-07-06T03:54:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.384+0000 7fcee11ff700 1 -- start start 2023-07-06T03:54:39.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.385+0000 7fcee11ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcedc15c610 0x7fcedc0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.385+0000 7fcee11ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcedc06b360 0x7fcedc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.385+0000 7fcee11ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcedc0cff90 con 0x7fcedc0d2210 2023-07-06T03:54:39.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.385+0000 7fcee11ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc0d0780 con 0x7fcedc15c610 2023-07-06T03:54:39.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.385+0000 7fcee11ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc15c9e0 con 0x7fcedc06b360 2023-07-06T03:54:39.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.385+0000 7fcedb59e700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcedc0d2210 0x7fcedc0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37902/0 (socket says 172.21.15.97:37902) 2023-07-06T03:54:39.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.385+0000 7fcedb59e700 1 -- 172.21.15.97:0/1477699660 learned_addr learned my addr 172.21.15.97:0/1477699660 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:39.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.385+0000 7fcedad9d700 1 --2- 172.21.15.97:0/1477699660 >> v2:172.21.15.97:3300/0 conn(0x7fcedc15c610 0x7fcedc0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:39.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.386+0000 7fceda59c700 1 --2- 172.21.15.97:0/1477699660 >> v2:172.21.15.146:3300/0 conn(0x7fcedc06b360 0x7fcedc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:39.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.386+0000 7fced9d9b700 1 -- 172.21.15.97:0/1477699660 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644394923 0 0) 0x7fcedc0cff90 con 0x7fcedc0d2210 2023-07-06T03:54:39.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.386+0000 7fcedad9d700 1 -- 172.21.15.97:0/1477699660 >> v2:172.21.15.146:3300/0 conn(0x7fcedc06b360 msgr2=0x7fcedc0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:39.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.386+0000 7fcedad9d700 1 --2- 172.21.15.97:0/1477699660 >> v2:172.21.15.146:3300/0 conn(0x7fcedc06b360 0x7fcedc0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.386+0000 7fcedad9d700 1 -- 172.21.15.97:0/1477699660 >> v1:172.21.15.97:6789/0 conn(0x7fcedc0d2210 legacy=0x7fcedc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:39.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.386+0000 7fcedad9d700 1 -- 172.21.15.97:0/1477699660 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc0b5650 con 0x7fcedc15c610 2023-07-06T03:54:39.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.386+0000 7fcedad9d700 1 --2- 172.21.15.97:0/1477699660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcedc15c610 0x7fcedc0b47b0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fcedc06a590 tx=0x7fcecc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef04b2947ed4e11 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:39.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.386+0000 7fced9d9b700 1 -- 172.21.15.97:0/1477699660 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc011830 con 0x7fcedc15c610 2023-07-06T03:54:39.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.386+0000 7fced9d9b700 1 -- 172.21.15.97:0/1477699660 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc011990 con 0x7fcedc15c610 2023-07-06T03:54:39.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.386+0000 7fced9d9b700 1 -- 172.21.15.97:0/1477699660 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc011d20 con 0x7fcedc15c610 2023-07-06T03:54:39.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.387+0000 7fcee11ff700 1 -- 172.21.15.97:0/1477699660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcedc15c610 msgr2=0x7fcedc0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:39.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.387+0000 7fcee11ff700 1 --2- 172.21.15.97:0/1477699660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcedc15c610 0x7fcedc0b47b0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fcedc06a590 tx=0x7fcecc009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:39.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.387+0000 7fcee11ff700 1 -- 172.21.15.97:0/1477699660 shutdown_connections 2023-07-06T03:54:39.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.387+0000 7fcee11ff700 1 --2- 172.21.15.97:0/1477699660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcedc15c610 0x7fcedc0b47b0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.387+0000 7fcee11ff700 1 --2- 172.21.15.97:0/1477699660 >> v2:172.21.15.146:3300/0 conn(0x7fcedc06b360 0x7fcedc0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.387+0000 7fcee11ff700 1 -- 172.21.15.97:0/1477699660 >> 172.21.15.97:0/1477699660 conn(0x7fcedc06a6c0 msgr2=0x7fcedc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.387+0000 7fcee11ff700 1 -- 172.21.15.97:0/1477699660 shutdown_connections 2023-07-06T03:54:39.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.387+0000 7fcee11ff700 1 -- 172.21.15.97:0/1477699660 wait complete. 2023-07-06T03:54:39.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.389+0000 7fcee11ff700 1 Processor -- start 2023-07-06T03:54:39.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.389+0000 7fcee11ff700 1 -- start start 2023-07-06T03:54:39.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcee11ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcedc06b360 0x7fcedc1f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcee11ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcedc0d2210 0x7fcedc0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcee11ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcedc15c610 0x7fcedc15d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcee11ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0cc440 con 0x7fcedc06b360 2023-07-06T03:54:39.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcee11ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0cb830 con 0x7fcedc15c610 2023-07-06T03:54:39.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcee11ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0cbb10 con 0x7fcedc0d2210 2023-07-06T03:54:39.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcedad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcedc06b360 0x7fcedc1f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:39.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcedad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcedc06b360 0x7fcedc1f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34702/0 (socket says 172.21.15.97:34702) 2023-07-06T03:54:39.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcedad9d700 1 -- 172.21.15.97:0/1270436672 learned_addr learned my addr 172.21.15.97:0/1270436672 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:39.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcedad9d700 1 -- 172.21.15.97:0/1270436672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcedc0d2210 msgr2=0x7fcedc0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:39.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcedad9d700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcedc0d2210 0x7fcedc0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcedad9d700 1 -- 172.21.15.97:0/1270436672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcedc15c610 msgr2=0x7fcedc15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:39.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcedad9d700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcedc15c610 0x7fcedc15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcedad9d700 1 -- 172.21.15.97:0/1270436672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc0094a0 con 0x7fcedc06b360 2023-07-06T03:54:39.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.390+0000 7fcedad9d700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcedc06b360 0x7fcedc1f4c90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0090b0 tx=0x7fcecc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:39.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.391+0000 7fcec37fe700 1 -- 172.21.15.97:0/1270436672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0038a0 con 0x7fcedc06b360 2023-07-06T03:54:39.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.391+0000 7fcec37fe700 1 -- 172.21.15.97:0/1270436672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc01e070 con 0x7fcedc06b360 2023-07-06T03:54:39.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.391+0000 7fcee11ff700 1 -- 172.21.15.97:0/1270436672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc0cbdf0 con 0x7fcedc06b360 2023-07-06T03:54:39.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.391+0000 7fcec37fe700 1 -- 172.21.15.97:0/1270436672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc003c20 con 0x7fcedc06b360 2023-07-06T03:54:39.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.391+0000 7fcee11ff700 1 -- 172.21.15.97:0/1270436672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc1f51d0 con 0x7fcedc06b360 2023-07-06T03:54:39.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.392+0000 7fcee11ff700 1 -- 172.21.15.97:0/1270436672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc0b19c0 con 0x7fcedc06b360 2023-07-06T03:54:39.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.393+0000 7fcec37fe700 1 -- 172.21.15.97:0/1270436672 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcecc02d450 con 0x7fcedc06b360 2023-07-06T03:54:39.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.394+0000 7fcec37fe700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcec806ace0 0x7fcec806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.394+0000 7fcec37fe700 1 -- 172.21.15.97:0/1270436672 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcecc08a560 con 0x7fcedc06b360 2023-07-06T03:54:39.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.394+0000 7fceda59c700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcec806ace0 0x7fcec806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:39.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.395+0000 7fceda59c700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcec806ace0 0x7fcec806d190 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcedc153ac0 tx=0x7fced0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:39.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.398+0000 7fcec37fe700 1 -- 172.21.15.97:0/1270436672 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcecc055600 con 0x7fcedc06b360 2023-07-06T03:54:39.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.593+0000 7fcee11ff700 1 -- 172.21.15.97:0/1270436672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) v1 -- 0x7fcedc04e340 con 0x7fcedc06b360 2023-07-06T03:54:39.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.593+0000 7fcec37fe700 1 -- 172.21.15.97:0/1270436672 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v48) v1 ==== 106+0+8 (secure 0 0 0) 0x7fcecc058c20 con 0x7fcedc06b360 2023-07-06T03:54:39.595 INFO:teuthology.orchestra.run.smithi097.stdout:1048576 2023-07-06T03:54:39.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.595+0000 7fcee11ff700 1 -- 172.21.15.97:0/1270436672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcec806ace0 msgr2=0x7fcec806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:39.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.595+0000 7fcee11ff700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcec806ace0 0x7fcec806d190 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcedc153ac0 tx=0x7fced0008040 comp rx=0 tx=0).stop 2023-07-06T03:54:39.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.595+0000 7fcee11ff700 1 -- 172.21.15.97:0/1270436672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcedc06b360 msgr2=0x7fcedc1f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.595+0000 7fcee11ff700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcedc06b360 0x7fcedc1f4c90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0090b0 tx=0x7fcecc004750 comp rx=0 tx=0).stop 2023-07-06T03:54:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.596+0000 7fcee11ff700 1 -- 172.21.15.97:0/1270436672 shutdown_connections 2023-07-06T03:54:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.596+0000 7fcee11ff700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcec806ace0 0x7fcec806d190 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.596+0000 7fcee11ff700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcedc15c610 0x7fcedc15d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.596+0000 7fcee11ff700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcedc06b360 0x7fcedc1f4c90 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.596+0000 7fcee11ff700 1 --2- 172.21.15.97:0/1270436672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcedc0d2210 0x7fcedc0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.596+0000 7fcee11ff700 1 -- 172.21.15.97:0/1270436672 >> 172.21.15.97:0/1270436672 conn(0x7fcedc06a6c0 msgr2=0x7fcedc0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:39.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.596+0000 7fcee11ff700 1 -- 172.21.15.97:0/1270436672 shutdown_connections 2023-07-06T03:54:39.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.596+0000 7fcee11ff700 1 -- 172.21.15.97:0/1270436672 wait complete. 2023-07-06T03:54:39.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-07-06T03:54:39.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.810+0000 7f2855bff700 1 Processor -- start 2023-07-06T03:54:39.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.811+0000 7f2855bff700 1 -- start start 2023-07-06T03:54:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f2855bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f285015c610 0x7f28500b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f2855bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f285006b360 0x7f28500b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f2855bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28500d2210 0x7f28500d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f2855bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28500d0780 con 0x7f285006b360 2023-07-06T03:54:39.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f2855bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285015c9e0 con 0x7f285015c610 2023-07-06T03:54:39.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f2855bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28500b5650 con 0x7f28500d2210 2023-07-06T03:54:39.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f284effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f285006b360 0x7f28500b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:39.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f284effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f285006b360 0x7f28500b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34712/0 (socket says 172.21.15.97:34712) 2023-07-06T03:54:39.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f284effd700 1 -- 172.21.15.97:0/1149821145 learned_addr learned my addr 172.21.15.97:0/1149821145 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:39.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f284ffff700 1 --2- 172.21.15.97:0/1149821145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28500d2210 0x7f28500d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:39.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f284f7fe700 1 --2- 172.21.15.97:0/1149821145 >> v2:172.21.15.146:3300/0 conn(0x7f285015c610 0x7f28500b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:39.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f284effd700 1 -- 172.21.15.97:0/1149821145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28500d2210 msgr2=0x7f28500d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:39.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f284effd700 1 --2- 172.21.15.97:0/1149821145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28500d2210 0x7f28500d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.812+0000 7f284effd700 1 -- 172.21.15.97:0/1149821145 >> v2:172.21.15.146:3300/0 conn(0x7f285015c610 msgr2=0x7f28500b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:39.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.813+0000 7f284effd700 1 --2- 172.21.15.97:0/1149821145 >> v2:172.21.15.146:3300/0 conn(0x7f285015c610 0x7f28500b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.813+0000 7f284effd700 1 -- 172.21.15.97:0/1149821145 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28500d01c0 con 0x7f285006b360 2023-07-06T03:54:39.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.813+0000 7f284effd700 1 --2- 172.21.15.97:0/1149821145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f285006b360 0x7f28500b4cf0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f285008b000 tx=0x7f2844009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf768c6e22244717 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:39.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.813+0000 7f284e7fc700 1 -- 172.21.15.97:0/1149821145 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400b860 con 0x7f285006b360 2023-07-06T03:54:39.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.813+0000 7f284e7fc700 1 -- 172.21.15.97:0/1149821145 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284400b9c0 con 0x7f285006b360 2023-07-06T03:54:39.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.813+0000 7f284e7fc700 1 -- 172.21.15.97:0/1149821145 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400bd50 con 0x7f285006b360 2023-07-06T03:54:39.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.813+0000 7f2855bff700 1 -- 172.21.15.97:0/1149821145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f285006b360 msgr2=0x7f28500b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:39.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.813+0000 7f2855bff700 1 --2- 172.21.15.97:0/1149821145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f285006b360 0x7f28500b4cf0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f285008b000 tx=0x7f2844009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:39.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.814+0000 7f2855bff700 1 -- 172.21.15.97:0/1149821145 shutdown_connections 2023-07-06T03:54:39.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.814+0000 7f2855bff700 1 --2- 172.21.15.97:0/1149821145 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f28500d2210 0x7f28500d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.814+0000 7f2855bff700 1 --2- 172.21.15.97:0/1149821145 >> v2:172.21.15.146:3300/0 conn(0x7f285015c610 0x7f28500b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.814+0000 7f2855bff700 1 --2- 172.21.15.97:0/1149821145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f285006b360 0x7f28500b4cf0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.814+0000 7f2855bff700 1 -- 172.21.15.97:0/1149821145 >> 172.21.15.97:0/1149821145 conn(0x7f285006a6c0 msgr2=0x7f285006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:39.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.814+0000 7f2855bff700 1 -- 172.21.15.97:0/1149821145 shutdown_connections 2023-07-06T03:54:39.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.814+0000 7f2855bff700 1 -- 172.21.15.97:0/1149821145 wait complete. 2023-07-06T03:54:39.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.815+0000 7f2855bff700 1 Processor -- start 2023-07-06T03:54:39.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.815+0000 7f2855bff700 1 -- start start 2023-07-06T03:54:39.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.816+0000 7f2855bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f285006b360 0x7f28500c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.816+0000 7f2855bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28500d2210 0x7f28501f1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.816+0000 7f2855bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285015c610 0x7f28501f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.816+0000 7f2855bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2850148f00 con 0x7f28500d2210 2023-07-06T03:54:39.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.816+0000 7f2855bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28500c8150 con 0x7f285006b360 2023-07-06T03:54:39.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.816+0000 7f2855bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28500c8430 con 0x7f285015c610 2023-07-06T03:54:39.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.817+0000 7f284effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28500d2210 0x7f28501f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:39.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.817+0000 7f284effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28500d2210 0x7f28501f1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34724/0 (socket says 172.21.15.97:34724) 2023-07-06T03:54:39.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.817+0000 7f284effd700 1 -- 172.21.15.97:0/479267615 learned_addr learned my addr 172.21.15.97:0/479267615 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:39.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.817+0000 7f284f7fe700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f285006b360 0x7f28500c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.817+0000 7f284ffff700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285015c610 0x7f28501f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.817+0000 7f284effd700 1 -- 172.21.15.97:0/479267615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285015c610 msgr2=0x7f28501f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.817+0000 7f284effd700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285015c610 0x7f28501f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.817+0000 7f284effd700 1 -- 172.21.15.97:0/479267615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f285006b360 msgr2=0x7f28500c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.817+0000 7f284effd700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f285006b360 0x7f28500c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.817+0000 7f284effd700 1 -- 172.21.15.97:0/479267615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2844009550 con 0x7f28500d2210 2023-07-06T03:54:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.817+0000 7f284effd700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28500d2210 0x7f28501f1050 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f2844009c40 tx=0x7f2844004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:39.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.818+0000 7f2837fff700 1 -- 172.21.15.97:0/479267615 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2844004240 con 0x7f28500d2210 2023-07-06T03:54:39.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.818+0000 7f2837fff700 1 -- 172.21.15.97:0/479267615 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284401f070 con 0x7f28500d2210 2023-07-06T03:54:39.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.818+0000 7f2855bff700 1 -- 172.21.15.97:0/479267615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28500c8710 con 0x7f28500d2210 2023-07-06T03:54:39.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.818+0000 7f2837fff700 1 -- 172.21.15.97:0/479267615 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28440036a0 con 0x7f28500d2210 2023-07-06T03:54:39.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.818+0000 7f2855bff700 1 -- 172.21.15.97:0/479267615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28500c8cd0 con 0x7f28500d2210 2023-07-06T03:54:39.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.819+0000 7f2855bff700 1 -- 172.21.15.97:0/479267615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28500c8ec0 con 0x7f28500d2210 2023-07-06T03:54:39.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.820+0000 7f2837fff700 1 -- 172.21.15.97:0/479267615 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2844021030 con 0x7f28500d2210 2023-07-06T03:54:39.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.821+0000 7f2837fff700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283c06ace0 0x7f283c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:39.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.821+0000 7f2837fff700 1 -- 172.21.15.97:0/479267615 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f284408a760 con 0x7f28500d2210 2023-07-06T03:54:39.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.821+0000 7f284f7fe700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283c06ace0 0x7f283c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:39.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.821+0000 7f284f7fe700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283c06ace0 0x7f283c06d190 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2840006b20 tx=0x7f2840008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:39.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:39.825+0000 7f2837fff700 1 -- 172.21.15.97:0/479267615 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2844055800 con 0x7f28500d2210 2023-07-06T03:54:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.023+0000 7f2855bff700 1 -- 172.21.15.97:0/479267615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7f285004e340 con 0x7f28500d2210 2023-07-06T03:54:40.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.024+0000 7f2837fff700 1 -- 172.21.15.97:0/479267615 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v48) v1 ==== 112+0+10 (secure 0 0 0) 0x7f2844058e20 con 0x7f28500d2210 2023-07-06T03:54:40.026 INFO:teuthology.orchestra.run.smithi097.stdout:60.000000 2023-07-06T03:54:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.026+0000 7f2855bff700 1 -- 172.21.15.97:0/479267615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283c06ace0 msgr2=0x7f283c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.026+0000 7f2855bff700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283c06ace0 0x7f283c06d190 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2840006b20 tx=0x7f2840008040 comp rx=0 tx=0).stop 2023-07-06T03:54:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.026+0000 7f2855bff700 1 -- 172.21.15.97:0/479267615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28500d2210 msgr2=0x7f28501f1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.026+0000 7f2855bff700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28500d2210 0x7f28501f1050 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f2844009c40 tx=0x7f2844004750 comp rx=0 tx=0).stop 2023-07-06T03:54:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.026+0000 7f2855bff700 1 -- 172.21.15.97:0/479267615 shutdown_connections 2023-07-06T03:54:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.026+0000 7f2855bff700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f283c06ace0 0x7f283c06d190 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.026+0000 7f2855bff700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f285015c610 0x7f28501f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.026+0000 7f2855bff700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f285006b360 0x7f28500c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.026+0000 7f2855bff700 1 --2- 172.21.15.97:0/479267615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28500d2210 0x7f28501f1050 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.026+0000 7f2855bff700 1 -- 172.21.15.97:0/479267615 >> 172.21.15.97:0/479267615 conn(0x7f285006a6c0 msgr2=0x7f28500b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:40.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.027+0000 7f2855bff700 1 -- 172.21.15.97:0/479267615 shutdown_connections 2023-07-06T03:54:40.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.027+0000 7f2855bff700 1 -- 172.21.15.97:0/479267615 wait complete. 2023-07-06T03:54:40.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:54:40.046 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:54:40.046 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:54:40.047 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:54:40.047 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:54:40.047 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:40.047 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:54:40.048 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:54:40.048 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:54:40.048 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:54:40.049 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:40.049 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:54:40.049 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:54:40.049 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:54:40.050 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:54:40.050 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:40.050 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:54:40.051 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:54:40.051 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:54:40.051 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:54:40.051 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:54:40.052 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:54:40.052 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:54:40.052 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:54:40.053 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:40.053 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:54:40.053 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:54:40.194 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.158938.asok 2023-07-06T03:54:40.372 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.158938.asok 2023-07-06T03:54:40.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:40.373 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.158938.asok mds_sessions 2023-07-06T03:54:40.594 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:54:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8751, 2023-07-06T03:54:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:54:40.594 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:54:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:54:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 8751 2023-07-06T03:54:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:54:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:54:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:54:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 343344933 2023-07-06T03:54:40.595 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.8751 v1:192.168.0.1:0/343344933", 2023-07-06T03:54:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "v1:192.168.0.1:0/343344933", 2023-07-06T03:54:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions": [ 2023-07-06T03:54:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": 0, 2023-07-06T03:54:40.596 INFO:teuthology.orchestra.run.smithi097.stdout: "addrs": { 2023-07-06T03:54:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-07-06T03:54:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-07-06T03:54:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.146:6832", 2023-07-06T03:54:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 582690410 2023-07-06T03:54:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:54:40.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:54:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.146:6834", 2023-07-06T03:54:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 582690410 2023-07-06T03:54:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:54:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:54:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "seq": 0, 2023-07-06T03:54:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_gen": 0, 2023-07-06T03:54:40.598 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_ttl": "2023-07-06T03:55:34.960540+0000", 2023-07-06T03:54:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "last_cap_renew_request": "2023-07-06T03:54:34.960540+0000", 2023-07-06T03:54:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_renew_seq": 2, 2023-07-06T03:54:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "num_caps": 1, 2023-07-06T03:54:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "open" 2023-07-06T03:54:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:54:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-07-06T03:54:40.599 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 80 2023-07-06T03:54:40.600 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:54:40.608 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:54:40.609 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:40.656 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:54:40.656 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2023-07-06T03:54:40.657 DEBUG:teuthology.orchestra.run.smithi097:> import time 2023-07-06T03:54:40.657 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:54:40.658 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:40.658 DEBUG:teuthology.orchestra.run.smithi097:> n = 200 2023-07-06T03:54:40.658 DEBUG:teuthology.orchestra.run.smithi097:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2023-07-06T03:54:40.659 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:40.659 DEBUG:teuthology.orchestra.run.smithi097:> if not os.path.exists(abs_path): 2023-07-06T03:54:40.659 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(abs_path) 2023-07-06T03:54:40.659 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:40.660 DEBUG:teuthology.orchestra.run.smithi097:> handles = [] 2023-07-06T03:54:40.660 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(0, n): 2023-07-06T03:54:40.660 DEBUG:teuthology.orchestra.run.smithi097:> fname = "file_"+str(i) 2023-07-06T03:54:40.661 DEBUG:teuthology.orchestra.run.smithi097:> path = os.path.join(abs_path, fname) 2023-07-06T03:54:40.661 DEBUG:teuthology.orchestra.run.smithi097:> handles.append(open(path, '"'"'w'"'"')) 2023-07-06T03:54:40.661 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:54:40.661 DEBUG:teuthology.orchestra.run.smithi097:> while True: 2023-07-06T03:54:40.662 DEBUG:teuthology.orchestra.run.smithi097:> time.sleep(1) 2023-07-06T03:54:40.662 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:54:40.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.901+0000 7f30e7bff700 1 Processor -- start 2023-07-06T03:54:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.901+0000 7f30e7bff700 1 -- start start 2023-07-06T03:54:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.903+0000 7f30e7bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30e00d00d0 0x7f30e006a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.903+0000 7f30e7bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e006a5a0 0x7f30e00ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.903+0000 7f30e7bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e00cdc80 con 0x7f30e015c740 2023-07-06T03:54:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.903+0000 7f30e7bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e0071f30 con 0x7f30e00d00d0 2023-07-06T03:54:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.903+0000 7f30e7bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e006aa20 con 0x7f30e006a5a0 2023-07-06T03:54:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.904+0000 7f30e619c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e006a5a0 0x7f30e00ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.904+0000 7f30e519a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30e00d00d0 0x7f30e006a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.904+0000 7f30e619c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e006a5a0 0x7f30e00ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:42124/0 (socket says 172.21.15.97:42124) 2023-07-06T03:54:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.904+0000 7f30e619c700 1 -- 172.21.15.97:0/3971260622 learned_addr learned my addr 172.21.15.97:0/3971260622 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.904+0000 7f30e599b700 1 --1- 172.21.15.97:0/3971260622 >> v1:172.21.15.146:6789/0 conn(0x7f30e015c740 0x7f30e0071820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:53324/0 (socket says 172.21.15.97:53324) 2023-07-06T03:54:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.904+0000 7f30e519a700 1 -- 172.21.15.97:0/3971260622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e006a5a0 msgr2=0x7f30e00ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.904+0000 7f30e519a700 1 --2- 172.21.15.97:0/3971260622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e006a5a0 0x7f30e00ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.904+0000 7f30e519a700 1 -- 172.21.15.97:0/3971260622 >> v1:172.21.15.146:6789/0 conn(0x7f30e015c740 legacy=0x7f30e0071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.904+0000 7f30e519a700 1 -- 172.21.15.97:0/3971260622 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00ce380 con 0x7f30e00d00d0 2023-07-06T03:54:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.904+0000 7f30e619c700 1 --2- 172.21.15.97:0/3971260622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e006a5a0 0x7f30e00ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:54:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.904+0000 7f30e519a700 1 --2- 172.21.15.97:0/3971260622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e00d00d0 0x7f30e006a060 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009840 tx=0x7f30d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1280f78f57f6529d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.905+0000 7f30e4999700 1 -- 172.21.15.97:0/3971260622 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0011830 con 0x7f30e00d00d0 2023-07-06T03:54:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.905+0000 7f30e4999700 1 -- 172.21.15.97:0/3971260622 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d0011990 con 0x7f30e00d00d0 2023-07-06T03:54:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.905+0000 7f30e4999700 1 -- 172.21.15.97:0/3971260622 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0011d20 con 0x7f30e00d00d0 2023-07-06T03:54:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.905+0000 7f30e7bff700 1 -- 172.21.15.97:0/3971260622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e00d00d0 msgr2=0x7f30e006a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.905+0000 7f30e7bff700 1 --2- 172.21.15.97:0/3971260622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e00d00d0 0x7f30e006a060 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009840 tx=0x7f30d0009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.905+0000 7f30e7bff700 1 -- 172.21.15.97:0/3971260622 shutdown_connections 2023-07-06T03:54:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.905+0000 7f30e7bff700 1 --2- 172.21.15.97:0/3971260622 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e006a5a0 0x7f30e00ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.905+0000 7f30e7bff700 1 --2- 172.21.15.97:0/3971260622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e00d00d0 0x7f30e006a060 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.905+0000 7f30e7bff700 1 -- 172.21.15.97:0/3971260622 >> 172.21.15.97:0/3971260622 conn(0x7f30e00b2280 msgr2=0x7f30e00cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.905+0000 7f30e7bff700 1 -- 172.21.15.97:0/3971260622 shutdown_connections 2023-07-06T03:54:40.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.906+0000 7f30e7bff700 1 -- 172.21.15.97:0/3971260622 wait complete. 2023-07-06T03:54:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.907+0000 7f30e7bff700 1 Processor -- start 2023-07-06T03:54:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.907+0000 7f30e7bff700 1 -- start start 2023-07-06T03:54:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e7bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e006a5a0 0x7f30e015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e7bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e00d00d0 0x7f30e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e7bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e015c740 0x7f30e0147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e7bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00b4990 con 0x7f30e006a5a0 2023-07-06T03:54:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e7bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e015d410 con 0x7f30e00d00d0 2023-07-06T03:54:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e7bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00b3ff0 con 0x7f30e015c740 2023-07-06T03:54:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e599b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e006a5a0 0x7f30e015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e599b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e006a5a0 0x7f30e015cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36578/0 (socket says 172.21.15.97:36578) 2023-07-06T03:54:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e599b700 1 -- 172.21.15.97:0/1684350401 learned_addr learned my addr 172.21.15.97:0/1684350401 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e519a700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e00d00d0 0x7f30e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e619c700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e015c740 0x7f30e0147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e599b700 1 -- 172.21.15.97:0/1684350401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e015c740 msgr2=0x7f30e0147c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e599b700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e015c740 0x7f30e0147c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e599b700 1 -- 172.21.15.97:0/1684350401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e00d00d0 msgr2=0x7f30e006d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e599b700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e00d00d0 0x7f30e006d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e599b700 1 -- 172.21.15.97:0/1684350401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d8009b30 con 0x7f30e006a5a0 2023-07-06T03:54:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.909+0000 7f30e619c700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e015c740 0x7f30e0147c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.910+0000 7f30e599b700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e006a5a0 0x7f30e015cba0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c460 tx=0x7f30dc00c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.910+0000 7f30d67fc700 1 -- 172.21.15.97:0/1684350401 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc005d00 con 0x7f30e006a5a0 2023-07-06T03:54:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.910+0000 7f30d67fc700 1 -- 172.21.15.97:0/1684350401 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc005e60 con 0x7f30e006a5a0 2023-07-06T03:54:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.910+0000 7f30d67fc700 1 -- 172.21.15.97:0/1684350401 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc007310 con 0x7f30e006a5a0 2023-07-06T03:54:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.910+0000 7f30e7bff700 1 -- 172.21.15.97:0/1684350401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d00094a0 con 0x7f30e006a5a0 2023-07-06T03:54:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.910+0000 7f30e7bff700 1 -- 172.21.15.97:0/1684350401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e00b45b0 con 0x7f30e006a5a0 2023-07-06T03:54:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.911+0000 7f30e7bff700 1 -- 172.21.15.97:0/1684350401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e004e340 con 0x7f30e006a5a0 2023-07-06T03:54:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.912+0000 7f30d67fc700 1 -- 172.21.15.97:0/1684350401 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30dc007470 con 0x7f30e006a5a0 2023-07-06T03:54:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.914+0000 7f30d67fc700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30c406ace0 0x7f30c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.914+0000 7f30d67fc700 1 -- 172.21.15.97:0/1684350401 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f30dc088900 con 0x7f30e006a5a0 2023-07-06T03:54:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.914+0000 7f30e519a700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30c406ace0 0x7f30c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.915+0000 7f30e519a700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30c406ace0 0x7f30c406d190 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f30d00090b0 tx=0x7f30d0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:40.918+0000 7f30d67fc700 1 -- 172.21.15.97:0/1684350401 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30dc053910 con 0x7f30e006a5a0 2023-07-06T03:54:41.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.183+0000 7f30e7bff700 1 -- 172.21.15.97:0/1684350401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30e00ae980 con 0x7f30e006a5a0 2023-07-06T03:54:41.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.184+0000 7f30d67fc700 1 -- 172.21.15.97:0/1684350401 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4676 (secure 0 0 0) 0x7f30dc017090 con 0x7f30e006a5a0 2023-07-06T03:54:41.186 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:41.186 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2796855133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2796855133},{"type":"v1","addr":"172.21.15.97:6835","nonce":2796855133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6851,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1092966074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1092966074},{"type":"v1","addr":"172.21.15.146:6835","nonce":1092966074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8577,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2594138359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2594138359},{"type":"v1","addr":"172.21.15.97:6837","nonce":2594138359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:54:25.441909+0000","modified":"2023-07-06T03:54:28.981530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/582690410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":582690410},{"type":"v1","addr":"172.21.15.146:6834","nonce":582690410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-06T03:54:41.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.186+0000 7f30e7bff700 1 -- 172.21.15.97:0/1684350401 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30c406ace0 msgr2=0x7f30c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:41.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.186+0000 7f30e7bff700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30c406ace0 0x7f30c406d190 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f30d00090b0 tx=0x7f30d0008030 comp rx=0 tx=0).stop 2023-07-06T03:54:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.186+0000 7f30e7bff700 1 -- 172.21.15.97:0/1684350401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e006a5a0 msgr2=0x7f30e015cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.186+0000 7f30e7bff700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e006a5a0 0x7f30e015cba0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c460 tx=0x7f30dc00c770 comp rx=0 tx=0).stop 2023-07-06T03:54:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.186+0000 7f30e7bff700 1 -- 172.21.15.97:0/1684350401 shutdown_connections 2023-07-06T03:54:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.186+0000 7f30e7bff700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30c406ace0 0x7f30c406d190 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.186+0000 7f30e7bff700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30e015c740 0x7f30e0147c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.186+0000 7f30e7bff700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e006a5a0 0x7f30e015cba0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.186+0000 7f30e7bff700 1 --2- 172.21.15.97:0/1684350401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30e00d00d0 0x7f30e006d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.187+0000 7f30e7bff700 1 -- 172.21.15.97:0/1684350401 >> 172.21.15.97:0/1684350401 conn(0x7f30e00b2280 msgr2=0x7f30e00ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:41.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.187+0000 7f30e7bff700 1 -- 172.21.15.97:0/1684350401 shutdown_connections 2023-07-06T03:54:41.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.187+0000 7f30e7bff700 1 -- 172.21.15.97:0/1684350401 wait complete. 2023-07-06T03:54:41.190 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 80 2023-07-06T03:54:41.206 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-07-06T03:54:41.433 INFO:teuthology.orchestra.run.smithi146.stdout:[{"id":6965,"entity":{"name":{"type":"client","num":6965},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2393358819}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":5.1819199339999997,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9853163771382398,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6965 192.168.0.1:0/2393358819","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"125971","root":"/"}},{"id":8751,"entity":{"name":{"type":"client","num":8751},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":343344933}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":202,"uptime":7.4548848120000004,"requests_in_flight":200,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":405.61629407934453,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8751 192.168.0.1:0/343344933","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"158938","root":"/"}}] 2023-07-06T03:54:41.433 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "125971", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2393358819, "type": "any" }, "name": { "num": 6965, "type": "client" } }, "id": 6965, "inst": "client.6965 192.168.0.1:0/2393358819", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.98531637713824 }, "state": "open", "uptime": 5.181919934 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "158938", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 343344933, "type": "any" }, "name": { "num": 8751, "type": "client" } }, "id": 8751, "inst": "client.8751 192.168.0.1:0/343344933", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 202, "requests_in_flight": 200, "session_cache_liveness": { "halflife": 300, "value": 405.6162940793445 }, "state": "open", "uptime": 7.454884812 } ] 2023-07-06T03:54:41.434 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-07-06T03:54:41.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:54:41.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.627+0000 7fd8089ff700 1 Processor -- start 2023-07-06T03:54:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.627+0000 7fd8089ff700 1 -- start start 2023-07-06T03:54:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.628+0000 7fd8089ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd804164fb0 0x7fd804161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.628+0000 7fd8089ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd804164b20 con 0x7fd8040b0c70 2023-07-06T03:54:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.628+0000 7fd8089ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8041644f0 con 0x7fd80415afe0 2023-07-06T03:54:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.628+0000 7fd8089ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd804161b20 con 0x7fd804164fb0 2023-07-06T03:54:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.629+0000 7fd801d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd8040b0c70 0x7fd804160d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59178/0 (socket says 172.21.15.97:59178) 2023-07-06T03:54:41.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.629+0000 7fd801d9b700 1 -- 172.21.15.97:0/2394031600 learned_addr learned my addr 172.21.15.97:0/2394031600 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:41.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.629+0000 7fd802d9d700 1 --2- 172.21.15.97:0/2394031600 >> v2:172.21.15.97:3300/0 conn(0x7fd804164fb0 0x7fd804161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:41.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.629+0000 7fd80159a700 1 -- 172.21.15.97:0/2394031600 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901428586 0 0) 0x7fd804164b20 con 0x7fd8040b0c70 2023-07-06T03:54:41.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.629+0000 7fd80159a700 1 -- 172.21.15.97:0/2394031600 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7ec0035b0 con 0x7fd8040b0c70 2023-07-06T03:54:41.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.629+0000 7fd802d9d700 1 -- 172.21.15.97:0/2394031600 >> v1:172.21.15.146:6789/0 conn(0x7fd80415afe0 legacy=0x7fd8040b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:41.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.629+0000 7fd802d9d700 1 -- 172.21.15.97:0/2394031600 >> v1:172.21.15.97:6789/0 conn(0x7fd8040b0c70 legacy=0x7fd804160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:41.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.629+0000 7fd802d9d700 1 -- 172.21.15.97:0/2394031600 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8041647b0 con 0x7fd804164fb0 2023-07-06T03:54:41.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.629+0000 7fd802d9d700 1 --2- 172.21.15.97:0/2394031600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd804164fb0 0x7fd804161490 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009d90 tx=0x7fd7fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc68860b72d276de server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:41.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.630+0000 7fd80159a700 1 -- 172.21.15.97:0/2394031600 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc00e070 con 0x7fd804164fb0 2023-07-06T03:54:41.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.630+0000 7fd80159a700 1 -- 172.21.15.97:0/2394031600 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc00bb20 con 0x7fd804164fb0 2023-07-06T03:54:41.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.630+0000 7fd80159a700 1 -- 172.21.15.97:0/2394031600 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc012430 con 0x7fd804164fb0 2023-07-06T03:54:41.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.630+0000 7fd8089ff700 1 -- 172.21.15.97:0/2394031600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd804164fb0 msgr2=0x7fd804161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:41.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.630+0000 7fd8089ff700 1 --2- 172.21.15.97:0/2394031600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd804164fb0 0x7fd804161490 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009d90 tx=0x7fd7fc00b330 comp rx=0 tx=0).stop 2023-07-06T03:54:41.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.630+0000 7fd8089ff700 1 -- 172.21.15.97:0/2394031600 shutdown_connections 2023-07-06T03:54:41.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.630+0000 7fd8089ff700 1 --2- 172.21.15.97:0/2394031600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd804164fb0 0x7fd804161490 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.630+0000 7fd8089ff700 1 -- 172.21.15.97:0/2394031600 >> 172.21.15.97:0/2394031600 conn(0x7fd8040ce8c0 msgr2=0x7fd8040cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:41.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.630+0000 7fd8089ff700 1 -- 172.21.15.97:0/2394031600 shutdown_connections 2023-07-06T03:54:41.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.631+0000 7fd8089ff700 1 -- 172.21.15.97:0/2394031600 wait complete. 2023-07-06T03:54:41.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.633+0000 7fd8089ff700 1 Processor -- start 2023-07-06T03:54:41.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.633+0000 7fd8089ff700 1 -- start start 2023-07-06T03:54:41.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.634+0000 7fd8089ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8040b0c70 0x7fd8040b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:41.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.634+0000 7fd8089ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd80415afe0 0x7fd8040b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:41.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.634+0000 7fd8089ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd804164fb0 0x7fd8040ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:41.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.634+0000 7fd8089ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040b46d0 con 0x7fd804164fb0 2023-07-06T03:54:41.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.634+0000 7fd8089ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040ae670 con 0x7fd80415afe0 2023-07-06T03:54:41.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.634+0000 7fd8089ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040ae950 con 0x7fd8040b0c70 2023-07-06T03:54:41.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.634+0000 7fd802d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd804164fb0 0x7fd8040ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:41.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.634+0000 7fd802d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd804164fb0 0x7fd8040ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36602/0 (socket says 172.21.15.97:36602) 2023-07-06T03:54:41.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd802d9d700 1 -- 172.21.15.97:0/421497238 learned_addr learned my addr 172.21.15.97:0/421497238 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:41.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd801d9b700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd80415afe0 0x7fd8040b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:41.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd80259c700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8040b0c70 0x7fd8040b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:41.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd802d9d700 1 -- 172.21.15.97:0/421497238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8040b0c70 msgr2=0x7fd8040b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:41.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd802d9d700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8040b0c70 0x7fd8040b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd802d9d700 1 -- 172.21.15.97:0/421497238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd80415afe0 msgr2=0x7fd8040b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:41.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd802d9d700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd80415afe0 0x7fd8040b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd802d9d700 1 -- 172.21.15.97:0/421497238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc009a70 con 0x7fd804164fb0 2023-07-06T03:54:41.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd80259c700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8040b0c70 0x7fd8040b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:41.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd802d9d700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd804164fb0 0x7fd8040ad900 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd80406b980 tx=0x7fd7fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:41.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd7f2ffd700 1 -- 172.21.15.97:0/421497238 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc012430 con 0x7fd804164fb0 2023-07-06T03:54:41.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd7f2ffd700 1 -- 172.21.15.97:0/421497238 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc020070 con 0x7fd804164fb0 2023-07-06T03:54:41.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.635+0000 7fd8089ff700 1 -- 172.21.15.97:0/421497238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8040aec30 con 0x7fd804164fb0 2023-07-06T03:54:41.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.636+0000 7fd7f2ffd700 1 -- 172.21.15.97:0/421497238 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc004280 con 0x7fd804164fb0 2023-07-06T03:54:41.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.636+0000 7fd8089ff700 1 -- 172.21.15.97:0/421497238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8040af1f0 con 0x7fd804164fb0 2023-07-06T03:54:41.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.637+0000 7fd8089ff700 1 -- 172.21.15.97:0/421497238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd804067830 con 0x7fd804164fb0 2023-07-06T03:54:41.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.637+0000 7fd7f2ffd700 1 -- 172.21.15.97:0/421497238 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd7fc0043e0 con 0x7fd804164fb0 2023-07-06T03:54:41.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.638+0000 7fd7f2ffd700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd7e406ace0 0x7fd7e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:41.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.638+0000 7fd7f2ffd700 1 -- 172.21.15.97:0/421497238 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd7fc022070 con 0x7fd804164fb0 2023-07-06T03:54:41.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.639+0000 7fd80259c700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd7e406ace0 0x7fd7e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:41.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.639+0000 7fd80259c700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd7e406ace0 0x7fd7e406d190 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4000f60 tx=0x7fd7f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:41.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.643+0000 7fd7f2ffd700 1 -- 172.21.15.97:0/421497238 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd7fc056250 con 0x7fd804164fb0 2023-07-06T03:54:41.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.877+0000 7fd8089ff700 1 -- 172.21.15.97:0/421497238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd8040af3e0 con 0x7fd804164fb0 2023-07-06T03:54:41.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.878+0000 7fd7f2ffd700 1 -- 172.21.15.97:0/421497238 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd7fc059870 con 0x7fd804164fb0 2023-07-06T03:54:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:41.880 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:54:41.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 -- 172.21.15.97:0/421497238 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd7e406ace0 msgr2=0x7fd7e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:41.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd7e406ace0 0x7fd7e406d190 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4000f60 tx=0x7fd7f4009040 comp rx=0 tx=0).stop 2023-07-06T03:54:41.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 -- 172.21.15.97:0/421497238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd804164fb0 msgr2=0x7fd8040ad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:41.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd804164fb0 0x7fd8040ad900 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd80406b980 tx=0x7fd7fc004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 -- 172.21.15.97:0/421497238 shutdown_connections 2023-07-06T03:54:41.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd7e406ace0 0x7fd7e406d190 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd804164fb0 0x7fd8040ad900 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8040b0c70 0x7fd8040b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 --2- 172.21.15.97:0/421497238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd80415afe0 0x7fd8040b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:41.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 -- 172.21.15.97:0/421497238 >> 172.21.15.97:0/421497238 conn(0x7fd8040ce8c0 msgr2=0x7fd804160bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:41.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 -- 172.21.15.97:0/421497238 shutdown_connections 2023-07-06T03:54:41.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:41.880+0000 7fd8089ff700 1 -- 172.21.15.97:0/421497238 wait complete. 2023-07-06T03:54:41.899 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:54:41.900 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2023-07-06T03:54:46.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:54:47.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.105+0000 7f5e8a1ff700 1 Processor -- start 2023-07-06T03:54:47.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.105+0000 7f5e8a1ff700 1 -- start start 2023-07-06T03:54:47.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.106+0000 7f5e8a1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8415c520 0x7f5e840cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.106+0000 7f5e8a1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e84160180 0x7f5e841605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.106+0000 7f5e8a1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5e84160b30 0x7f5e840cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.106+0000 7f5e8a1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e840cd8d0 con 0x7f5e84160180 2023-07-06T03:54:47.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.106+0000 7f5e8a1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e840cdae0 con 0x7f5e84160b30 2023-07-06T03:54:47.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.106+0000 7f5e8a1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840b6640 con 0x7f5e8415c520 2023-07-06T03:54:47.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e82ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e84160180 0x7f5e841605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e82ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e84160180 0x7f5e841605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36612/0 (socket says 172.21.15.97:36612) 2023-07-06T03:54:47.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e82ffd700 1 -- 172.21.15.97:0/3916086356 learned_addr learned my addr 172.21.15.97:0/3916086356 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:47.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e837fe700 1 --2- 172.21.15.97:0/3916086356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8415c520 0x7f5e840cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e83fff700 1 --2- 172.21.15.97:0/3916086356 >> v2:172.21.15.146:3300/0 conn(0x7f5e84160b30 0x7f5e840cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e82ffd700 1 -- 172.21.15.97:0/3916086356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8415c520 msgr2=0x7f5e840cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:47.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e82ffd700 1 --2- 172.21.15.97:0/3916086356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8415c520 0x7f5e840cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e82ffd700 1 -- 172.21.15.97:0/3916086356 >> v2:172.21.15.146:3300/0 conn(0x7f5e84160b30 msgr2=0x7f5e840cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:47.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e82ffd700 1 --2- 172.21.15.97:0/3916086356 >> v2:172.21.15.146:3300/0 conn(0x7f5e84160b30 0x7f5e840cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e82ffd700 1 -- 172.21.15.97:0/3916086356 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e840b6920 con 0x7f5e84160180 2023-07-06T03:54:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e83fff700 1 --2- 172.21.15.97:0/3916086356 >> v2:172.21.15.146:3300/0 conn(0x7f5e84160b30 0x7f5e840cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.107+0000 7f5e82ffd700 1 --2- 172.21.15.97:0/3916086356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e84160180 0x7f5e841605f0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f5e8408bfc0 tx=0x7f5e78009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe13fbb9aba131c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e827fc700 1 -- 172.21.15.97:0/3916086356 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800b870 con 0x7f5e84160180 2023-07-06T03:54:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e827fc700 1 -- 172.21.15.97:0/3916086356 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7800b9d0 con 0x7f5e84160180 2023-07-06T03:54:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e827fc700 1 -- 172.21.15.97:0/3916086356 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800bd60 con 0x7f5e84160180 2023-07-06T03:54:47.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/3916086356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e84160180 msgr2=0x7f5e841605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:47.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e8a1ff700 1 --2- 172.21.15.97:0/3916086356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e84160180 0x7f5e841605f0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f5e8408bfc0 tx=0x7f5e78009bd0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/3916086356 shutdown_connections 2023-07-06T03:54:47.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e8a1ff700 1 --2- 172.21.15.97:0/3916086356 >> v2:172.21.15.146:3300/0 conn(0x7f5e84160b30 0x7f5e840cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e8a1ff700 1 --2- 172.21.15.97:0/3916086356 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8415c520 0x7f5e840cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e8a1ff700 1 --2- 172.21.15.97:0/3916086356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e84160180 0x7f5e841605f0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/3916086356 >> 172.21.15.97:0/3916086356 conn(0x7f5e84071790 msgr2=0x7f5e84071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/3916086356 shutdown_connections 2023-07-06T03:54:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.108+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/3916086356 wait complete. 2023-07-06T03:54:47.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.110+0000 7f5e8a1ff700 1 Processor -- start 2023-07-06T03:54:47.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.110+0000 7f5e8a1ff700 1 -- start start 2023-07-06T03:54:47.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.111+0000 7f5e8a1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8415c520 0x7f5e8415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.111+0000 7f5e8a1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84160180 0x7f5e8415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.111+0000 7f5e8a1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e84160b30 0x7f5e8414ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.111+0000 7f5e8a1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8415b800 con 0x7f5e84160b30 2023-07-06T03:54:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.111+0000 7f5e8a1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8415e4d0 con 0x7f5e84160180 2023-07-06T03:54:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.111+0000 7f5e8a1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840cab40 con 0x7f5e8415c520 2023-07-06T03:54:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.111+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e84160b30 0x7f5e8414ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.111+0000 7f5e837fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8415c520 0x7f5e8415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e837fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8415c520 0x7f5e8415b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:42164/0 (socket says 172.21.15.97:42164) 2023-07-06T03:54:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e84160b30 0x7f5e8414ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36624/0 (socket says 172.21.15.97:36624) 2023-07-06T03:54:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e837fe700 1 -- 172.21.15.97:0/1458717867 learned_addr learned my addr 172.21.15.97:0/1458717867 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e82ffd700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84160180 0x7f5e8415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e83fff700 1 -- 172.21.15.97:0/1458717867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8415c520 msgr2=0x7f5e8415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e83fff700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8415c520 0x7f5e8415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e83fff700 1 -- 172.21.15.97:0/1458717867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84160180 msgr2=0x7f5e8415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e83fff700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84160180 0x7f5e8415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e83fff700 1 -- 172.21.15.97:0/1458717867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e78009550 con 0x7f5e84160b30 2023-07-06T03:54:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e83fff700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e84160b30 0x7f5e8414ffa0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00cc90 tx=0x7f5e7c00a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e6bfff700 1 -- 172.21.15.97:0/1458717867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c00acf0 con 0x7f5e84160b30 2023-07-06T03:54:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e6bfff700 1 -- 172.21.15.97:0/1458717867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7c00ae70 con 0x7f5e84160b30 2023-07-06T03:54:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e6bfff700 1 -- 172.21.15.97:0/1458717867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c00f6a0 con 0x7f5e84160b30 2023-07-06T03:54:47.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.112+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/1458717867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e840cae20 con 0x7f5e84160b30 2023-07-06T03:54:47.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.113+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/1458717867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8415f1f0 con 0x7f5e84160b30 2023-07-06T03:54:47.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.114+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/1458717867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8415e9d0 con 0x7f5e84160b30 2023-07-06T03:54:47.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.115+0000 7f5e6bfff700 1 -- 172.21.15.97:0/1458717867 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5e7c00b040 con 0x7f5e84160b30 2023-07-06T03:54:47.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.115+0000 7f5e6bfff700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e7006ace0 0x7f5e7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.115+0000 7f5e6bfff700 1 -- 172.21.15.97:0/1458717867 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e7c089ab0 con 0x7f5e84160b30 2023-07-06T03:54:47.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.116+0000 7f5e837fe700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e7006ace0 0x7f5e7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.116+0000 7f5e837fe700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e7006ace0 0x7f5e7006d190 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009a90 tx=0x7f5e74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:47.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.120+0000 7f5e6bfff700 1 -- 172.21.15.97:0/1458717867 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5e8415e9d0 con 0x7f5e84160b30 2023-07-06T03:54:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.355+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/1458717867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5e8415e9d0 con 0x7f5e84160b30 2023-07-06T03:54:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.356+0000 7f5e6bfff700 1 -- 172.21.15.97:0/1458717867 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f5e8415e9d0 con 0x7f5e84160b30 2023-07-06T03:54:47.358 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:47.358 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:54:47.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/1458717867 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e7006ace0 msgr2=0x7f5e7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:47.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e7006ace0 0x7f5e7006d190 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009a90 tx=0x7f5e74008040 comp rx=0 tx=0).stop 2023-07-06T03:54:47.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/1458717867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e84160b30 msgr2=0x7f5e8414ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:47.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e84160b30 0x7f5e8414ffa0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00cc90 tx=0x7f5e7c00a280 comp rx=0 tx=0).stop 2023-07-06T03:54:47.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/1458717867 shutdown_connections 2023-07-06T03:54:47.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5e7006ace0 0x7f5e7006d190 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e84160b30 0x7f5e8414ffa0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5e8415c520 0x7f5e8415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 --2- 172.21.15.97:0/1458717867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e84160180 0x7f5e8415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/1458717867 >> 172.21.15.97:0/1458717867 conn(0x7f5e84071790 msgr2=0x7f5e840cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:47.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/1458717867 shutdown_connections 2023-07-06T03:54:47.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.358+0000 7f5e8a1ff700 1 -- 172.21.15.97:0/1458717867 wait complete. 2023-07-06T03:54:47.381 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-07-06T03:54:47.381 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-07-06T03:54:47.382 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:54:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.573+0000 7f87907ff700 1 Processor -- start 2023-07-06T03:54:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.573+0000 7f87907ff700 1 -- start start 2023-07-06T03:54:47.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.574+0000 7f87907ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f878815c520 0x7f87880cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.574+0000 7f87907ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8788160d00 0x7f87880b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.574+0000 7f87907ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87880cdae0 con 0x7f8788160180 2023-07-06T03:54:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.574+0000 7f87907ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87880cd8d0 con 0x7f8788160d00 2023-07-06T03:54:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.575+0000 7f87907ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87880b6640 con 0x7f878815c520 2023-07-06T03:54:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.575+0000 7f878dd9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8788160180 0x7f87881605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59184/0 (socket says 172.21.15.97:59184) 2023-07-06T03:54:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.575+0000 7f878dd9a700 1 -- 172.21.15.97:0/2118564186 learned_addr learned my addr 172.21.15.97:0/2118564186 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.575+0000 7f878ed9c700 1 --2- 172.21.15.97:0/2118564186 >> v2:172.21.15.97:3300/0 conn(0x7f8788160d00 0x7f87880b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.575+0000 7f878e59b700 1 --2- 172.21.15.97:0/2118564186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f878815c520 0x7f87880cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.575+0000 7f878d599700 1 -- 172.21.15.97:0/2118564186 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838628084 0 0) 0x7f87880cdae0 con 0x7f8788160180 2023-07-06T03:54:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.575+0000 7f878d599700 1 -- 172.21.15.97:0/2118564186 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87740035b0 con 0x7f8788160180 2023-07-06T03:54:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.575+0000 7f878ed9c700 1 -- 172.21.15.97:0/2118564186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f878815c520 msgr2=0x7f87880cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.575+0000 7f878ed9c700 1 --2- 172.21.15.97:0/2118564186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f878815c520 0x7f87880cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.575+0000 7f878ed9c700 1 -- 172.21.15.97:0/2118564186 >> v1:172.21.15.97:6789/0 conn(0x7f8788160180 legacy=0x7f87881605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.575+0000 7f878ed9c700 1 -- 172.21.15.97:0/2118564186 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87880b6920 con 0x7f8788160d00 2023-07-06T03:54:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f878ed9c700 1 --2- 172.21.15.97:0/2118564186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8788160d00 0x7f87880b7400 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f8780009d90 tx=0x7f878000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e9f45919ace20d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f878d599700 1 -- 172.21.15.97:0/2118564186 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878000e070 con 0x7f8788160d00 2023-07-06T03:54:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f878d599700 1 -- 172.21.15.97:0/2118564186 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878000bb20 con 0x7f8788160d00 2023-07-06T03:54:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f878d599700 1 -- 172.21.15.97:0/2118564186 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8780012430 con 0x7f8788160d00 2023-07-06T03:54:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f87907ff700 1 -- 172.21.15.97:0/2118564186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8788160d00 msgr2=0x7f87880b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f87907ff700 1 --2- 172.21.15.97:0/2118564186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8788160d00 0x7f87880b7400 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f8780009d90 tx=0x7f878000b330 comp rx=0 tx=0).stop 2023-07-06T03:54:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f87907ff700 1 -- 172.21.15.97:0/2118564186 shutdown_connections 2023-07-06T03:54:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f87907ff700 1 --2- 172.21.15.97:0/2118564186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8788160d00 0x7f87880b7400 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f87907ff700 1 --2- 172.21.15.97:0/2118564186 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f878815c520 0x7f87880cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f87907ff700 1 -- 172.21.15.97:0/2118564186 >> 172.21.15.97:0/2118564186 conn(0x7f8788071790 msgr2=0x7f8788071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f87907ff700 1 -- 172.21.15.97:0/2118564186 shutdown_connections 2023-07-06T03:54:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.576+0000 7f87907ff700 1 -- 172.21.15.97:0/2118564186 wait complete. 2023-07-06T03:54:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.578+0000 7f87907ff700 1 Processor -- start 2023-07-06T03:54:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.578+0000 7f87907ff700 1 -- start start 2023-07-06T03:54:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.579+0000 7f87907ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f878815c520 0x7f8788158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.579+0000 7f87907ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8788160180 0x7f87880ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.579+0000 7f87907ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8788160d00 0x7f878814bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.579+0000 7f87907ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87880cd1a0 con 0x7f8788160180 2023-07-06T03:54:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.579+0000 7f87907ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878815a610 con 0x7f878815c520 2023-07-06T03:54:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.579+0000 7f87907ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878815a8f0 con 0x7f8788160d00 2023-07-06T03:54:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.579+0000 7f878dd9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8788160180 0x7f87880ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.579+0000 7f878dd9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8788160180 0x7f87880ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36652/0 (socket says 172.21.15.97:36652) 2023-07-06T03:54:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.579+0000 7f878dd9a700 1 -- 172.21.15.97:0/3907897052 learned_addr learned my addr 172.21.15.97:0/3907897052 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.579+0000 7f878e59b700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f878815c520 0x7f8788158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.579+0000 7f878ed9c700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8788160d00 0x7f878814bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.580+0000 7f878dd9a700 1 -- 172.21.15.97:0/3907897052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8788160d00 msgr2=0x7f878814bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.580+0000 7f878dd9a700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8788160d00 0x7f878814bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.580+0000 7f878dd9a700 1 -- 172.21.15.97:0/3907897052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f878815c520 msgr2=0x7f8788158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.580+0000 7f878dd9a700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f878815c520 0x7f8788158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.580+0000 7f878dd9a700 1 -- 172.21.15.97:0/3907897052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8780009a70 con 0x7f8788160180 2023-07-06T03:54:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.580+0000 7f878dd9a700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8788160180 0x7f87880ccc60 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f877800c700 tx=0x7f877800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.580+0000 7f877effd700 1 -- 172.21.15.97:0/3907897052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778005d70 con 0x7f8788160180 2023-07-06T03:54:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.580+0000 7f877effd700 1 -- 172.21.15.97:0/3907897052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87780070d0 con 0x7f8788160180 2023-07-06T03:54:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.580+0000 7f87907ff700 1 -- 172.21.15.97:0/3907897052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878815abd0 con 0x7f8788160180 2023-07-06T03:54:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.580+0000 7f877effd700 1 -- 172.21.15.97:0/3907897052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778007460 con 0x7f8788160180 2023-07-06T03:54:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.580+0000 7f87907ff700 1 -- 172.21.15.97:0/3907897052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878815b190 con 0x7f8788160180 2023-07-06T03:54:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.582+0000 7f87907ff700 1 -- 172.21.15.97:0/3907897052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878804e340 con 0x7f8788160180 2023-07-06T03:54:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.583+0000 7f877effd700 1 -- 172.21.15.97:0/3907897052 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f87780075c0 con 0x7f8788160180 2023-07-06T03:54:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.583+0000 7f877effd700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f876c06ace0 0x7f876c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.584+0000 7f877effd700 1 -- 172.21.15.97:0/3907897052 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8778088900 con 0x7f8788160180 2023-07-06T03:54:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.584+0000 7f878e59b700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f876c06ace0 0x7f876c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.584+0000 7f878e59b700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f876c06ace0 0x7f876c06d190 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8784009b40 tx=0x7f8784009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.588+0000 7f877effd700 1 -- 172.21.15.97:0/3907897052 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f87780539b0 con 0x7f8788160180 2023-07-06T03:54:47.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.828+0000 7f87907ff700 1 -- 172.21.15.97:0/3907897052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f87880b4f40 con 0x7f8788160180 2023-07-06T03:54:47.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.829+0000 7f877effd700 1 -- 172.21.15.97:0/3907897052 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f8778017150 con 0x7f8788160180 2023-07-06T03:54:47.831 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:47.831 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:54:47.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 -- 172.21.15.97:0/3907897052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f876c06ace0 msgr2=0x7f876c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:47.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f876c06ace0 0x7f876c06d190 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8784009b40 tx=0x7f8784009210 comp rx=0 tx=0).stop 2023-07-06T03:54:47.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 -- 172.21.15.97:0/3907897052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8788160180 msgr2=0x7f87880ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:47.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8788160180 0x7f87880ccc60 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f877800c700 tx=0x7f877800ca10 comp rx=0 tx=0).stop 2023-07-06T03:54:47.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 -- 172.21.15.97:0/3907897052 shutdown_connections 2023-07-06T03:54:47.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f876c06ace0 0x7f876c06d190 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8788160d00 0x7f878814bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f878815c520 0x7f8788158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 --2- 172.21.15.97:0/3907897052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8788160180 0x7f87880ccc60 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:47.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 -- 172.21.15.97:0/3907897052 >> 172.21.15.97:0/3907897052 conn(0x7f8788071790 msgr2=0x7f87880b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:47.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 -- 172.21.15.97:0/3907897052 shutdown_connections 2023-07-06T03:54:47.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:47.831+0000 7f87907ff700 1 -- 172.21.15.97:0/3907897052 wait complete. 2023-07-06T03:54:47.850 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-06T03:54:47.851 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2023-07-06T03:54:47.862 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.098+0000 7fd412fff700 1 Processor -- start 2023-07-06T03:54:48.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.098+0000 7fd412fff700 1 -- start start 2023-07-06T03:54:48.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.099+0000 7fd412fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd40c0cd390 0x7fd40c160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:48.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.099+0000 7fd412fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd40c160dd0 0x7fd40c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:48.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.099+0000 7fd412fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c0cdae0 con 0x7fd40c15c520 2023-07-06T03:54:48.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.099+0000 7fd412fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40c0b6640 con 0x7fd40c160dd0 2023-07-06T03:54:48.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.099+0000 7fd412fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0b6920 con 0x7fd40c0cd390 2023-07-06T03:54:48.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.099+0000 7fd410d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd40c15c520 0x7fd40c160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59192/0 (socket says 172.21.15.97:59192) 2023-07-06T03:54:48.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.099+0000 7fd410d9b700 1 -- 172.21.15.97:0/1587365446 learned_addr learned my addr 172.21.15.97:0/1587365446 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:48.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.100+0000 7fd40bfff700 1 --2- 172.21.15.97:0/1587365446 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd40c0cd390 0x7fd40c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:48.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.100+0000 7fd41159c700 1 --2- 172.21.15.97:0/1587365446 >> v2:172.21.15.146:3300/0 conn(0x7fd40c160dd0 0x7fd40c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:48.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.100+0000 7fd40b7fe700 1 -- 172.21.15.97:0/1587365446 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483922414 0 0) 0x7fd40c0cdae0 con 0x7fd40c15c520 2023-07-06T03:54:48.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.100+0000 7fd40b7fe700 1 -- 172.21.15.97:0/1587365446 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f40035b0 con 0x7fd40c15c520 2023-07-06T03:54:48.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.100+0000 7fd40bfff700 1 -- 172.21.15.97:0/1587365446 >> v2:172.21.15.146:3300/0 conn(0x7fd40c160dd0 msgr2=0x7fd40c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:48.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.100+0000 7fd40bfff700 1 --2- 172.21.15.97:0/1587365446 >> v2:172.21.15.146:3300/0 conn(0x7fd40c160dd0 0x7fd40c0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:48.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.100+0000 7fd40bfff700 1 -- 172.21.15.97:0/1587365446 >> v1:172.21.15.97:6789/0 conn(0x7fd40c15c520 legacy=0x7fd40c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:48.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.100+0000 7fd40bfff700 1 -- 172.21.15.97:0/1587365446 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0b6c00 con 0x7fd40c0cd390 2023-07-06T03:54:48.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.100+0000 7fd40bfff700 1 --2- 172.21.15.97:0/1587365446 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd40c0cd390 0x7fd40c160890 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009840 tx=0x7fd3fc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fda405cec0667125 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:48.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.101+0000 7fd40b7fe700 1 -- 172.21.15.97:0/1587365446 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc011830 con 0x7fd40c0cd390 2023-07-06T03:54:48.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.101+0000 7fd40b7fe700 1 -- 172.21.15.97:0/1587365446 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc011990 con 0x7fd40c0cd390 2023-07-06T03:54:48.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.101+0000 7fd40b7fe700 1 -- 172.21.15.97:0/1587365446 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc011d20 con 0x7fd40c0cd390 2023-07-06T03:54:48.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.101+0000 7fd412fff700 1 -- 172.21.15.97:0/1587365446 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd40c0cd390 msgr2=0x7fd40c160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:48.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.101+0000 7fd412fff700 1 --2- 172.21.15.97:0/1587365446 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd40c0cd390 0x7fd40c160890 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009840 tx=0x7fd3fc009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:48.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.101+0000 7fd412fff700 1 -- 172.21.15.97:0/1587365446 shutdown_connections 2023-07-06T03:54:48.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.101+0000 7fd412fff700 1 --2- 172.21.15.97:0/1587365446 >> v2:172.21.15.146:3300/0 conn(0x7fd40c160dd0 0x7fd40c0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:48.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.101+0000 7fd412fff700 1 --2- 172.21.15.97:0/1587365446 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd40c0cd390 0x7fd40c160890 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:48.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.101+0000 7fd412fff700 1 -- 172.21.15.97:0/1587365446 >> 172.21.15.97:0/1587365446 conn(0x7fd40c071790 msgr2=0x7fd40c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:48.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.101+0000 7fd412fff700 1 -- 172.21.15.97:0/1587365446 shutdown_connections 2023-07-06T03:54:48.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.102+0000 7fd412fff700 1 -- 172.21.15.97:0/1587365446 wait complete. 2023-07-06T03:54:48.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.103+0000 7fd412fff700 1 Processor -- start 2023-07-06T03:54:48.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.103+0000 7fd412fff700 1 -- start start 2023-07-06T03:54:48.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.104+0000 7fd412fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd40c0cd390 0x7fd40c150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:48.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.104+0000 7fd412fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd40c15c520 0x7fd40c0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:48.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.104+0000 7fd412fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd40c160dd0 0x7fd40c0b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:48.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.104+0000 7fd412fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0cc770 con 0x7fd40c0cd390 2023-07-06T03:54:48.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.104+0000 7fd412fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0cca50 con 0x7fd40c15c520 2023-07-06T03:54:48.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.104+0000 7fd412fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0d1060 con 0x7fd40c160dd0 2023-07-06T03:54:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.104+0000 7fd410d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd40c0cd390 0x7fd40c150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd410d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd40c0cd390 0x7fd40c150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36654/0 (socket says 172.21.15.97:36654) 2023-07-06T03:54:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd410d9b700 1 -- 172.21.15.97:0/2001482228 learned_addr learned my addr 172.21.15.97:0/2001482228 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd40bfff700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd40c15c520 0x7fd40c0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd41159c700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd40c160dd0 0x7fd40c0b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd410d9b700 1 -- 172.21.15.97:0/2001482228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd40c160dd0 msgr2=0x7fd40c0b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:48.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd410d9b700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd40c160dd0 0x7fd40c0b7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:48.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd410d9b700 1 -- 172.21.15.97:0/2001482228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd40c15c520 msgr2=0x7fd40c0cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:48.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd410d9b700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd40c15c520 0x7fd40c0cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:48.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd410d9b700 1 -- 172.21.15.97:0/2001482228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc0094a0 con 0x7fd40c0cd390 2023-07-06T03:54:48.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd410d9b700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd40c0cd390 0x7fd40c150880 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fd40000c700 tx=0x7fd40000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:48.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd4097fa700 1 -- 172.21.15.97:0/2001482228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400005d70 con 0x7fd40c0cd390 2023-07-06T03:54:48.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd4097fa700 1 -- 172.21.15.97:0/2001482228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4000070d0 con 0x7fd40c0cd390 2023-07-06T03:54:48.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd412fff700 1 -- 172.21.15.97:0/2001482228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c0d1340 con 0x7fd40c0cd390 2023-07-06T03:54:48.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.105+0000 7fd4097fa700 1 -- 172.21.15.97:0/2001482228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400007460 con 0x7fd40c0cd390 2023-07-06T03:54:48.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.106+0000 7fd412fff700 1 -- 172.21.15.97:0/2001482228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c0cc3b0 con 0x7fd40c0cd390 2023-07-06T03:54:48.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.107+0000 7fd412fff700 1 -- 172.21.15.97:0/2001482228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c0ce9b0 con 0x7fd40c0cd390 2023-07-06T03:54:48.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.107+0000 7fd4097fa700 1 -- 172.21.15.97:0/2001482228 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd4000075c0 con 0x7fd40c0cd390 2023-07-06T03:54:48.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.108+0000 7fd4097fa700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3f806ace0 0x7fd3f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.108+0000 7fd4097fa700 1 -- 172.21.15.97:0/2001482228 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd400088900 con 0x7fd40c0cd390 2023-07-06T03:54:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.109+0000 7fd40bfff700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3f806ace0 0x7fd3f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:48.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.109+0000 7fd40bfff700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3f806ace0 0x7fd3f806d190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc000c00 tx=0x7fd3fc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:48.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.113+0000 7fd4097fa700 1 -- 172.21.15.97:0/2001482228 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd4000539b0 con 0x7fd40c0cd390 2023-07-06T03:54:48.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.350+0000 7fd412fff700 1 -- 172.21.15.97:0/2001482228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd40c04e340 con 0x7fd40c0cd390 2023-07-06T03:54:48.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.351+0000 7fd4097fa700 1 -- 172.21.15.97:0/2001482228 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4676 (secure 0 0 0) 0x7fd400017090 con 0x7fd40c0cd390 2023-07-06T03:54:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:48.353 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2796855133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2796855133},{"type":"v1","addr":"172.21.15.97:6835","nonce":2796855133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6851,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1092966074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1092966074},{"type":"v1","addr":"172.21.15.146:6835","nonce":1092966074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8577,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2594138359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2594138359},{"type":"v1","addr":"172.21.15.97:6837","nonce":2594138359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:54:25.441909+0000","modified":"2023-07-06T03:54:46.757700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6834/582690410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":582690410},{"type":"v1","addr":"172.21.15.146:6834","nonce":582690410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-06T03:54:48.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 -- 172.21.15.97:0/2001482228 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3f806ace0 msgr2=0x7fd3f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:48.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3f806ace0 0x7fd3f806d190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc000c00 tx=0x7fd3fc01c040 comp rx=0 tx=0).stop 2023-07-06T03:54:48.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 -- 172.21.15.97:0/2001482228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd40c0cd390 msgr2=0x7fd40c150880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:48.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd40c0cd390 0x7fd40c150880 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fd40000c700 tx=0x7fd40000ca10 comp rx=0 tx=0).stop 2023-07-06T03:54:48.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 -- 172.21.15.97:0/2001482228 shutdown_connections 2023-07-06T03:54:48.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd3f806ace0 0x7fd3f806d190 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd40c160dd0 0x7fd40c0b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd40c0cd390 0x7fd40c150880 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 --2- 172.21.15.97:0/2001482228 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd40c15c520 0x7fd40c0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 -- 172.21.15.97:0/2001482228 >> 172.21.15.97:0/2001482228 conn(0x7fd40c071790 msgr2=0x7fd40c072020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 -- 172.21.15.97:0/2001482228 shutdown_connections 2023-07-06T03:54:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:48.353+0000 7fd412fff700 1 -- 172.21.15.97:0/2001482228 wait complete. 2023-07-06T03:54:48.357 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 82 2023-07-06T03:54:48.373 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-07-06T03:54:48.601 INFO:teuthology.orchestra.run.smithi146.stdout:[{"id":6965,"entity":{"name":{"type":"client","num":6965},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2393358819}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":12.351809152,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9362691196701323,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6965 192.168.0.1:0/2393358819","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"125971","root":"/"}},{"id":8751,"entity":{"name":{"type":"client","num":8751},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":343344933}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":187,"uptime":14.624774029999999,"requests_in_flight":0,"num_completed_requests":200,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":671.64491065702975,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":196.63480629657863,"halflife":1.5},"recall_caps_throttle2o":{"value":58.011405591719367,"halflife":0.5},"session_cache_liveness":{"value":398.95222089724507,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":5,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8751 192.168.0.1:0/343344933","completed_requests":[{"tid":5,"created_ino":"0x10000000001"},{"tid":6,"created_ino":"0x100000001f6"},{"tid":7,"created_ino":"0x100000001f7"},{"tid":8,"created_ino":"0x100000001f8"},{"tid":9,"created_ino":"0x100000001f9"},{"tid":10,"created_ino":"0x100000001fa"},{"tid":11,"created_ino":"0x100000001fb"},{"tid":12,"created_ino":"0x100000001fc"},{"tid":13,"created_ino":"0x100000001fd"},{"tid":14,"created_ino":"0x100000001fe"},{"tid":15,"created_ino":"0x100000001ff"},{"tid":16,"created_ino":"0x10000000200"},{"tid":17,"created_ino":"0x10000000201"},{"tid":18,"created_ino":"0x10000000202"},{"tid":19,"created_ino":"0x10000000203"},{"tid":20,"created_ino":"0x10000000204"},{"tid":21,"created_ino":"0x10000000205"},{"tid":22,"created_ino":"0x10000000206"},{"tid":23,"created_ino":"0x10000000207"},{"tid":24,"created_ino":"0x10000000208"},{"tid":25,"created_ino":"0x10000000209"},{"tid":26,"created_ino":"0x1000000020a"},{"tid":27,"created_ino":"0x1000000020b"},{"tid":28,"created_ino":"0x1000000020c"},{"tid":29,"created_ino":"0x1000000020d"},{"tid":30,"created_ino":"0x1000000020e"},{"tid":31,"created_ino":"0x1000000020f"},{"tid":32,"created_ino":"0x10000000210"},{"tid":33,"created_ino":"0x10000000211"},{"tid":34,"created_ino":"0x10000000212"},{"tid":35,"created_ino":"0x10000000213"},{"tid":36,"created_ino":"0x10000000214"},{"tid":37,"created_ino":"0x10000000215"},{"tid":38,"created_ino":"0x10000000216"},{"tid":39,"created_ino":"0x10000000217"},{"tid":40,"created_ino":"0x10000000218"},{"tid":41,"created_ino":"0x10000000219"},{"tid":42,"created_ino":"0x1000000021a"},{"tid":43,"created_ino":"0x1000000021b"},{"tid":44,"created_ino":"0x1000000021c"},{"tid":45,"created_ino":"0x1000000021d"},{"tid":46,"created_ino":"0x1000000021e"},{"tid":47,"created_ino":"0x1000000021f"},{"tid":48,"created_ino":"0x10000000220"},{"tid":49,"created_ino":"0x10000000221"},{"tid":50,"created_ino":"0x10000000222"},{"tid":51,"created_ino":"0x10000000223"},{"tid":52,"created_ino":"0x10000000224"},{"tid":53,"created_ino":"0x10000000225"},{"tid":54,"created_ino":"0x10000000226"},{"tid":55,"created_ino":"0x10000000227"},{"tid":56,"created_ino":"0x10000000228"},{"tid":57,"created_ino":"0x10000000229"},{"tid":58,"created_ino":"0x1000000022a"},{"tid":59,"created_ino":"0x1000000022b"},{"tid":60,"created_ino":"0x1000000022c"},{"tid":61,"created_ino":"0x1000000022d"},{"tid":62,"created_ino":"0x1000000022e"},{"tid":63,"created_ino":"0x1000000022f"},{"tid":64,"created_ino":"0x10000000230"},{"tid":65,"created_ino":"0x10000000231"},{"tid":66,"created_ino":"0x10000000232"},{"tid":67,"created_ino":"0x10000000233"},{"tid":68,"created_ino":"0x10000000234"},{"tid":69,"created_ino":"0x10000000235"},{"tid":70,"created_ino":"0x10000000236"},{"tid":71,"created_ino":"0x10000000237"},{"tid":72,"created_ino":"0x10000000238"},{"tid":73,"created_ino":"0x10000000239"},{"tid":74,"created_ino":"0x1000000023a"},{"tid":75,"created_ino":"0x1000000023b"},{"tid":76,"created_ino":"0x1000000023c"},{"tid":77,"created_ino":"0x1000000023d"},{"tid":78,"created_ino":"0x1000000023e"},{"tid":79,"created_ino":"0x1000000023f"},{"tid":80,"created_ino":"0x10000000240"},{"tid":81,"created_ino":"0x10000000241"},{"tid":82,"created_ino":"0x10000000242"},{"tid":83,"created_ino":"0x10000000243"},{"tid":84,"created_ino":"0x10000000244"},{"tid":85,"created_ino":"0x10000000245"},{"tid":86,"created_ino":"0x10000000246"},{"tid":87,"created_ino":"0x10000000247"},{"tid":88,"created_ino":"0x10000000248"},{"tid":89,"created_ino":"0x10000000249"},{"tid":90,"created_ino":"0x1000000024a"},{"tid":91,"created_ino":"0x1000000024b"},{"tid":92,"created_ino":"0x1000000024c"},{"tid":93,"created_ino":"0x1000000024d"},{"tid":94,"created_ino":"0x1000000024e"},{"tid":95,"created_ino":"0x1000000024f"},{"tid":96,"created_ino":"0x10000000250"},{"tid":97,"created_ino":"0x10000000251"},{"tid":98,"created_ino":"0x10000000252"},{"tid":99,"created_ino":"0x10000000253"},{"tid":100,"created_ino":"0x10000000254"},{"tid":101,"created_ino":"0x10000000255"},{"tid":102,"created_ino":"0x10000000256"},{"tid":103,"created_ino":"0x10000000257"},{"tid":104,"created_ino":"0x10000000258"},{"tid":105,"created_ino":"0x10000000259"},{"tid":106,"created_ino":"0x1000000025a"},{"tid":107,"created_ino":"0x1000000025b"},{"tid":108,"created_ino":"0x1000000025c"},{"tid":109,"created_ino":"0x1000000025d"},{"tid":110,"created_ino":"0x1000000025e"},{"tid":111,"created_ino":"0x1000000025f"},{"tid":112,"created_ino":"0x10000000260"},{"tid":113,"created_ino":"0x10000000261"},{"tid":114,"created_ino":"0x10000000262"},{"tid":115,"created_ino":"0x10000000263"},{"tid":116,"created_ino":"0x10000000264"},{"tid":117,"created_ino":"0x10000000265"},{"tid":118,"created_ino":"0x10000000266"},{"tid":119,"created_ino":"0x10000000267"},{"tid":120,"created_ino":"0x10000000268"},{"tid":121,"created_ino":"0x10000000269"},{"tid":122,"created_ino":"0x1000000026a"},{"tid":123,"created_ino":"0x1000000026b"},{"tid":124,"created_ino":"0x1000000026c"},{"tid":125,"created_ino":"0x1000000026d"},{"tid":126,"created_ino":"0x1000000026e"},{"tid":127,"created_ino":"0x1000000026f"},{"tid":128,"created_ino":"0x10000000270"},{"tid":129,"created_ino":"0x10000000271"},{"tid":130,"created_ino":"0x10000000272"},{"tid":131,"created_ino":"0x10000000273"},{"tid":132,"created_ino":"0x10000000274"},{"tid":133,"created_ino":"0x10000000275"},{"tid":134,"created_ino":"0x10000000276"},{"tid":135,"created_ino":"0x10000000277"},{"tid":136,"created_ino":"0x10000000278"},{"tid":137,"created_ino":"0x10000000279"},{"tid":138,"created_ino":"0x1000000027a"},{"tid":139,"created_ino":"0x1000000027b"},{"tid":140,"created_ino":"0x1000000027c"},{"tid":141,"created_ino":"0x1000000027d"},{"tid":142,"created_ino":"0x1000000027e"},{"tid":143,"created_ino":"0x1000000027f"},{"tid":144,"created_ino":"0x10000000280"},{"tid":145,"created_ino":"0x10000000281"},{"tid":146,"created_ino":"0x10000000282"},{"tid":147,"created_ino":"0x10000000283"},{"tid":148,"created_ino":"0x10000000284"},{"tid":149,"created_ino":"0x10000000285"},{"tid":150,"created_ino":"0x10000000286"},{"tid":151,"created_ino":"0x10000000287"},{"tid":152,"created_ino":"0x10000000288"},{"tid":153,"created_ino":"0x10000000289"},{"tid":154,"created_ino":"0x1000000028a"},{"tid":155,"created_ino":"0x1000000028b"},{"tid":156,"created_ino":"0x1000000028c"},{"tid":157,"created_ino":"0x1000000028d"},{"tid":158,"created_ino":"0x1000000028e"},{"tid":159,"created_ino":"0x1000000028f"},{"tid":160,"created_ino":"0x10000000290"},{"tid":161,"created_ino":"0x10000000291"},{"tid":162,"created_ino":"0x10000000292"},{"tid":163,"created_ino":"0x10000000293"},{"tid":164,"created_ino":"0x10000000294"},{"tid":165,"created_ino":"0x10000000295"},{"tid":166,"created_ino":"0x10000000296"},{"tid":167,"created_ino":"0x10000000297"},{"tid":168,"created_ino":"0x10000000298"},{"tid":169,"created_ino":"0x10000000299"},{"tid":170,"created_ino":"0x1000000029a"},{"tid":171,"created_ino":"0x1000000029b"},{"tid":172,"created_ino":"0x1000000029c"},{"tid":173,"created_ino":"0x1000000029d"},{"tid":174,"created_ino":"0x1000000029e"},{"tid":175,"created_ino":"0x1000000029f"},{"tid":176,"created_ino":"0x100000002a0"},{"tid":177,"created_ino":"0x100000002a1"},{"tid":178,"created_ino":"0x100000002a2"},{"tid":179,"created_ino":"0x100000002a3"},{"tid":180,"created_ino":"0x100000002a4"},{"tid":181,"created_ino":"0x100000002a5"},{"tid":182,"created_ino":"0x100000002a6"},{"tid":183,"created_ino":"0x100000002a7"},{"tid":184,"created_ino":"0x100000002a8"},{"tid":185,"created_ino":"0x100000002a9"},{"tid":186,"created_ino":"0x100000002aa"},{"tid":187,"created_ino":"0x100000002ab"},{"tid":188,"created_ino":"0x100000002ac"},{"tid":189,"created_ino":"0x100000002ad"},{"tid":190,"created_ino":"0x100000002ae"},{"tid":191,"created_ino":"0x100000002af"},{"tid":192,"created_ino":"0x100000002b0"},{"tid":193,"created_ino":"0x100000002b1"},{"tid":194,"created_ino":"0x100000002b2"},{"tid":195,"created_ino":"0x100000002b3"},{"tid":196,"created_ino":"0x100000002b4"},{"tid":197,"created_ino":"0x100000002b5"},{"tid":198,"created_ino":"0x100000002b6"},{"tid":199,"created_ino":"0x100000002b7"},{"tid":200,"created_ino":"0x100000002b8"},{"tid":201,"created_ino":"0x100000002b9"},{"tid":202,"created_ino":"0x100000002ba"},{"tid":203,"created_ino":"0x100000002bb"},{"tid":204,"created_ino":"0x100000002bc"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000002bd","length":802}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"158938","root":"/"}}] 2023-07-06T03:54:48.603 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "125971", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2393358819, "type": "any" }, "name": { "num": 6965, "type": "client" } }, "id": 6965, "inst": "client.6965 192.168.0.1:0/2393358819", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9362691196701323 }, "state": "open", "uptime": 12.351809152 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "158938", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 5 }, { "created_ino": "0x100000001f6", "tid": 6 }, { "created_ino": "0x100000001f7", "tid": 7 }, { "created_ino": "0x100000001f8", "tid": 8 }, { "created_ino": "0x100000001f9", "tid": 9 }, { "created_ino": "0x100000001fa", "tid": 10 }, { "created_ino": "0x100000001fb", "tid": 11 }, { "created_ino": "0x100000001fc", "tid": 12 }, { "created_ino": "0x100000001fd", "tid": 13 }, { "created_ino": "0x100000001fe", "tid": 14 }, { "created_ino": "0x100000001ff", "tid": 15 }, { "created_ino": "0x10000000200", "tid": 16 }, { "created_ino": "0x10000000201", "tid": 17 }, { "created_ino": "0x10000000202", "tid": 18 }, { "created_ino": "0x10000000203", "tid": 19 }, { "created_ino": "0x10000000204", "tid": 20 }, { "created_ino": "0x10000000205", "tid": 21 }, { "created_ino": "0x10000000206", "tid": 22 }, { "created_ino": "0x10000000207", "tid": 23 }, { "created_ino": "0x10000000208", "tid": 24 }, { "created_ino": "0x10000000209", "tid": 25 }, { "created_ino": "0x1000000020a", "tid": 26 }, { "created_ino": "0x1000000020b", "tid": 27 }, { "created_ino": "0x1000000020c", "tid": 28 }, { "created_ino": "0x1000000020d", "tid": 29 }, { "created_ino": "0x1000000020e", "tid": 30 }, { "created_ino": "0x1000000020f", "tid": 31 }, { "created_ino": "0x10000000210", "tid": 32 }, { "created_ino": "0x10000000211", "tid": 33 }, { "created_ino": "0x10000000212", "tid": 34 }, { "created_ino": "0x10000000213", "tid": 35 }, { "created_ino": "0x10000000214", "tid": 36 }, { "created_ino": "0x10000000215", "tid": 37 }, { "created_ino": "0x10000000216", "tid": 38 }, { "created_ino": "0x10000000217", "tid": 39 }, { "created_ino": "0x10000000218", "tid": 40 }, { "created_ino": "0x10000000219", "tid": 41 }, { "created_ino": "0x1000000021a", "tid": 42 }, { "created_ino": "0x1000000021b", "tid": 43 }, { "created_ino": "0x1000000021c", "tid": 44 }, { "created_ino": "0x1000000021d", "tid": 45 }, { "created_ino": "0x1000000021e", "tid": 46 }, { "created_ino": "0x1000000021f", "tid": 47 }, { "created_ino": "0x10000000220", "tid": 48 }, { "created_ino": "0x10000000221", "tid": 49 }, { "created_ino": "0x10000000222", "tid": 50 }, { "created_ino": "0x10000000223", "tid": 51 }, { "created_ino": "0x10000000224", "tid": 52 }, { "created_ino": "0x10000000225", "tid": 53 }, { "created_ino": "0x10000000226", "tid": 54 }, { "created_ino": "0x10000000227", "tid": 55 }, { "created_ino": "0x10000000228", "tid": 56 }, { "created_ino": "0x10000000229", "tid": 57 }, { "created_ino": "0x1000000022a", "tid": 58 }, { "created_ino": "0x1000000022b", "tid": 59 }, { "created_ino": "0x1000000022c", "tid": 60 }, { "created_ino": "0x1000000022d", "tid": 61 }, { "created_ino": "0x1000000022e", "tid": 62 }, { "created_ino": "0x1000000022f", "tid": 63 }, { "created_ino": "0x10000000230", "tid": 64 }, { "created_ino": "0x10000000231", "tid": 65 }, { "created_ino": "0x10000000232", "tid": 66 }, { "created_ino": "0x10000000233", "tid": 67 }, { "created_ino": "0x10000000234", "tid": 68 }, { "created_ino": "0x10000000235", "tid": 69 }, { "created_ino": "0x10000000236", "tid": 70 }, { "created_ino": "0x10000000237", "tid": 71 }, { "created_ino": "0x10000000238", "tid": 72 }, { "created_ino": "0x10000000239", "tid": 73 }, { "created_ino": "0x1000000023a", "tid": 74 }, { "created_ino": "0x1000000023b", "tid": 75 }, { "created_ino": "0x1000000023c", "tid": 76 }, { "created_ino": "0x1000000023d", "tid": 77 }, { "created_ino": "0x1000000023e", "tid": 78 }, { "created_ino": "0x1000000023f", "tid": 79 }, { "created_ino": "0x10000000240", "tid": 80 }, { "created_ino": "0x10000000241", "tid": 81 }, { "created_ino": "0x10000000242", "tid": 82 }, { "created_ino": "0x10000000243", "tid": 83 }, { "created_ino": "0x10000000244", "tid": 84 }, { "created_ino": "0x10000000245", "tid": 85 }, { "created_ino": "0x10000000246", "tid": 86 }, { "created_ino": "0x10000000247", "tid": 87 }, { "created_ino": "0x10000000248", "tid": 88 }, { "created_ino": "0x10000000249", "tid": 89 }, { "created_ino": "0x1000000024a", "tid": 90 }, { "created_ino": "0x1000000024b", "tid": 91 }, { "created_ino": "0x1000000024c", "tid": 92 }, { "created_ino": "0x1000000024d", "tid": 93 }, { "created_ino": "0x1000000024e", "tid": 94 }, { "created_ino": "0x1000000024f", "tid": 95 }, { "created_ino": "0x10000000250", "tid": 96 }, { "created_ino": "0x10000000251", "tid": 97 }, { "created_ino": "0x10000000252", "tid": 98 }, { "created_ino": "0x10000000253", "tid": 99 }, { "created_ino": "0x10000000254", "tid": 100 }, { "created_ino": "0x10000000255", "tid": 101 }, { "created_ino": "0x10000000256", "tid": 102 }, { "created_ino": "0x10000000257", "tid": 103 }, { "created_ino": "0x10000000258", "tid": 104 }, { "created_ino": "0x10000000259", "tid": 105 }, { "created_ino": "0x1000000025a", "tid": 106 }, { "created_ino": "0x1000000025b", "tid": 107 }, { "created_ino": "0x1000000025c", "tid": 108 }, { "created_ino": "0x1000000025d", "tid": 109 }, { "created_ino": "0x1000000025e", "tid": 110 }, { "created_ino": "0x1000000025f", "tid": 111 }, { "created_ino": "0x10000000260", "tid": 112 }, { "created_ino": "0x10000000261", "tid": 113 }, { "created_ino": "0x10000000262", "tid": 114 }, { "created_ino": "0x10000000263", "tid": 115 }, { "created_ino": "0x10000000264", "tid": 116 }, { "created_ino": "0x10000000265", "tid": 117 }, { "created_ino": "0x10000000266", "tid": 118 }, { "created_ino": "0x10000000267", "tid": 119 }, { "created_ino": "0x10000000268", "tid": 120 }, { "created_ino": "0x10000000269", "tid": 121 }, { "created_ino": "0x1000000026a", "tid": 122 }, { "created_ino": "0x1000000026b", "tid": 123 }, { "created_ino": "0x1000000026c", "tid": 124 }, { "created_ino": "0x1000000026d", "tid": 125 }, { "created_ino": "0x1000000026e", "tid": 126 }, { "created_ino": "0x1000000026f", "tid": 127 }, { "created_ino": "0x10000000270", "tid": 128 }, { "created_ino": "0x10000000271", "tid": 129 }, { "created_ino": "0x10000000272", "tid": 130 }, { "created_ino": "0x10000000273", "tid": 131 }, { "created_ino": "0x10000000274", "tid": 132 }, { "created_ino": "0x10000000275", "tid": 133 }, { "created_ino": "0x10000000276", "tid": 134 }, { "created_ino": "0x10000000277", "tid": 135 }, { "created_ino": "0x10000000278", "tid": 136 }, { "created_ino": "0x10000000279", "tid": 137 }, { "created_ino": "0x1000000027a", "tid": 138 }, { "created_ino": "0x1000000027b", "tid": 139 }, { "created_ino": "0x1000000027c", "tid": 140 }, { "created_ino": "0x1000000027d", "tid": 141 }, { "created_ino": "0x1000000027e", "tid": 142 }, { "created_ino": "0x1000000027f", "tid": 143 }, { "created_ino": "0x10000000280", "tid": 144 }, { "created_ino": "0x10000000281", "tid": 145 }, { "created_ino": "0x10000000282", "tid": 146 }, { "created_ino": "0x10000000283", "tid": 147 }, { "created_ino": "0x10000000284", "tid": 148 }, { "created_ino": "0x10000000285", "tid": 149 }, { "created_ino": "0x10000000286", "tid": 150 }, { "created_ino": "0x10000000287", "tid": 151 }, { "created_ino": "0x10000000288", "tid": 152 }, { "created_ino": "0x10000000289", "tid": 153 }, { "created_ino": "0x1000000028a", "tid": 154 }, { "created_ino": "0x1000000028b", "tid": 155 }, { "created_ino": "0x1000000028c", "tid": 156 }, { "created_ino": "0x1000000028d", "tid": 157 }, { "created_ino": "0x1000000028e", "tid": 158 }, { "created_ino": "0x1000000028f", "tid": 159 }, { "created_ino": "0x10000000290", "tid": 160 }, { "created_ino": "0x10000000291", "tid": 161 }, { "created_ino": "0x10000000292", "tid": 162 }, { "created_ino": "0x10000000293", "tid": 163 }, { "created_ino": "0x10000000294", "tid": 164 }, { "created_ino": "0x10000000295", "tid": 165 }, { "created_ino": "0x10000000296", "tid": 166 }, { "created_ino": "0x10000000297", "tid": 167 }, { "created_ino": "0x10000000298", "tid": 168 }, { "created_ino": "0x10000000299", "tid": 169 }, { "created_ino": "0x1000000029a", "tid": 170 }, { "created_ino": "0x1000000029b", "tid": 171 }, { "created_ino": "0x1000000029c", "tid": 172 }, { "created_ino": "0x1000000029d", "tid": 173 }, { "created_ino": "0x1000000029e", "tid": 174 }, { "created_ino": "0x1000000029f", "tid": 175 }, { "created_ino": "0x100000002a0", "tid": 176 }, { "created_ino": "0x100000002a1", "tid": 177 }, { "created_ino": "0x100000002a2", "tid": 178 }, { "created_ino": "0x100000002a3", "tid": 179 }, { "created_ino": "0x100000002a4", "tid": 180 }, { "created_ino": "0x100000002a5", "tid": 181 }, { "created_ino": "0x100000002a6", "tid": 182 }, { "created_ino": "0x100000002a7", "tid": 183 }, { "created_ino": "0x100000002a8", "tid": 184 }, { "created_ino": "0x100000002a9", "tid": 185 }, { "created_ino": "0x100000002aa", "tid": 186 }, { "created_ino": "0x100000002ab", "tid": 187 }, { "created_ino": "0x100000002ac", "tid": 188 }, { "created_ino": "0x100000002ad", "tid": 189 }, { "created_ino": "0x100000002ae", "tid": 190 }, { "created_ino": "0x100000002af", "tid": 191 }, { "created_ino": "0x100000002b0", "tid": 192 }, { "created_ino": "0x100000002b1", "tid": 193 }, { "created_ino": "0x100000002b2", "tid": 194 }, { "created_ino": "0x100000002b3", "tid": 195 }, { "created_ino": "0x100000002b4", "tid": 196 }, { "created_ino": "0x100000002b5", "tid": 197 }, { "created_ino": "0x100000002b6", "tid": 198 }, { "created_ino": "0x100000002b7", "tid": 199 }, { "created_ino": "0x100000002b8", "tid": 200 }, { "created_ino": "0x100000002b9", "tid": 201 }, { "created_ino": "0x100000002ba", "tid": 202 }, { "created_ino": "0x100000002bb", "tid": 203 }, { "created_ino": "0x100000002bc", "tid": 204 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 343344933, "type": "any" }, "name": { "num": 8751, "type": "client" } }, "id": 8751, "inst": "client.8751 192.168.0.1:0/343344933", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 5, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 200, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 802, "start": "0x100000002bd" } ], "recall_caps": { "halflife": 60, "value": 671.6449106570298 }, "recall_caps_throttle": { "halflife": 1.5, "value": 196.63480629657863 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 58.01140559171937 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 187, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 398.95222089724507 }, "state": "open", "uptime": 14.62477403 } ] 2023-07-06T03:54:48.604 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-06T03:54:48.604 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:54:48.604 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:54:48.704 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:54:48.801 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:54:48.802 INFO:tasks.cephfs.mount:Terminating background process 2023-07-06T03:54:48.802 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:54:48.803 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:48.850 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:49.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.087+0000 7fc484dff700 1 Processor -- start 2023-07-06T03:54:49.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.088+0000 7fc484dff700 1 -- start start 2023-07-06T03:54:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.089+0000 7fc484dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4801592f0 0x7fc4800b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.089+0000 7fc484dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48015cc40 con 0x7fc480158f20 2023-07-06T03:54:49.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.089+0000 7fc484dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48015cf20 con 0x7fc48006d860 2023-07-06T03:54:49.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.089+0000 7fc484dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48015d200 con 0x7fc4801592f0 2023-07-06T03:54:49.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.089+0000 7fc47e59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc480158f20 0x7fc4800caac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59194/0 (socket says 172.21.15.97:59194) 2023-07-06T03:54:49.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.089+0000 7fc47e59c700 1 -- 172.21.15.97:0/2083012846 learned_addr learned my addr 172.21.15.97:0/2083012846 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:49.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.089+0000 7fc47dd9b700 1 --2- 172.21.15.97:0/2083012846 >> v2:172.21.15.97:3300/0 conn(0x7fc4801592f0 0x7fc4800b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.089+0000 7fc47d59a700 1 -- 172.21.15.97:0/2083012846 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815537543 0 0) 0x7fc48015cc40 con 0x7fc480158f20 2023-07-06T03:54:49.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.090+0000 7fc47d59a700 1 -- 172.21.15.97:0/2083012846 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4680035b0 con 0x7fc480158f20 2023-07-06T03:54:49.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.090+0000 7fc47dd9b700 1 -- 172.21.15.97:0/2083012846 >> v1:172.21.15.146:6789/0 conn(0x7fc48006d860 legacy=0x7fc48006bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.090+0000 7fc47d59a700 1 -- 172.21.15.97:0/2083012846 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712830133 0 0) 0x7fc48015cf20 con 0x7fc48006d860 2023-07-06T03:54:49.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.090+0000 7fc47dd9b700 1 -- 172.21.15.97:0/2083012846 >> v1:172.21.15.97:6789/0 conn(0x7fc480158f20 legacy=0x7fc4800caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.090+0000 7fc47dd9b700 1 -- 172.21.15.97:0/2083012846 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48015d4e0 con 0x7fc4801592f0 2023-07-06T03:54:49.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.090+0000 7fc47dd9b700 1 --2- 172.21.15.97:0/2083012846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4801592f0 0x7fc4800b0560 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc474009840 tx=0x7fc474009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f0c5b902cae19 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:49.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.090+0000 7fc47d59a700 1 -- 172.21.15.97:0/2083012846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474011830 con 0x7fc4801592f0 2023-07-06T03:54:49.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.090+0000 7fc47d59a700 1 -- 172.21.15.97:0/2083012846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc474011990 con 0x7fc4801592f0 2023-07-06T03:54:49.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.090+0000 7fc47d59a700 1 -- 172.21.15.97:0/2083012846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474011d20 con 0x7fc4801592f0 2023-07-06T03:54:49.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.091+0000 7fc484dff700 1 -- 172.21.15.97:0/2083012846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4801592f0 msgr2=0x7fc4800b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.091+0000 7fc484dff700 1 --2- 172.21.15.97:0/2083012846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4801592f0 0x7fc4800b0560 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc474009840 tx=0x7fc474009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:49.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.091+0000 7fc484dff700 1 -- 172.21.15.97:0/2083012846 shutdown_connections 2023-07-06T03:54:49.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.091+0000 7fc484dff700 1 --2- 172.21.15.97:0/2083012846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4801592f0 0x7fc4800b0560 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.091+0000 7fc484dff700 1 -- 172.21.15.97:0/2083012846 >> 172.21.15.97:0/2083012846 conn(0x7fc4800ce8c0 msgr2=0x7fc4800cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:49.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.091+0000 7fc484dff700 1 -- 172.21.15.97:0/2083012846 shutdown_connections 2023-07-06T03:54:49.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.091+0000 7fc484dff700 1 -- 172.21.15.97:0/2083012846 wait complete. 2023-07-06T03:54:49.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.093+0000 7fc484dff700 1 Processor -- start 2023-07-06T03:54:49.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.093+0000 7fc484dff700 1 -- start start 2023-07-06T03:54:49.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.094+0000 7fc484dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc48006d860 0x7fc4801ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.094+0000 7fc484dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc480158f20 0x7fc48014f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc484dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4801592f0 0x7fc480150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc484dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48015a730 con 0x7fc4801592f0 2023-07-06T03:54:49.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc484dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800b3340 con 0x7fc48006d860 2023-07-06T03:54:49.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc484dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48015a980 con 0x7fc480158f20 2023-07-06T03:54:49.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc47ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4801592f0 0x7fc480150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc47ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4801592f0 0x7fc480150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36662/0 (socket says 172.21.15.97:36662) 2023-07-06T03:54:49.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc47ed9d700 1 -- 172.21.15.97:0/2945074463 learned_addr learned my addr 172.21.15.97:0/2945074463 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:49.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc47e59c700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc48006d860 0x7fc4801ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc47dd9b700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc480158f20 0x7fc48014f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc47ed9d700 1 -- 172.21.15.97:0/2945074463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc480158f20 msgr2=0x7fc48014f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc47ed9d700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc480158f20 0x7fc48014f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc47ed9d700 1 -- 172.21.15.97:0/2945074463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc48006d860 msgr2=0x7fc4801ed9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc47ed9d700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc48006d860 0x7fc4801ed9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc47ed9d700 1 -- 172.21.15.97:0/2945074463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4740094a0 con 0x7fc4801592f0 2023-07-06T03:54:49.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.095+0000 7fc47e59c700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc48006d860 0x7fc4801ed9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:49.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.096+0000 7fc47ed9d700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4801592f0 0x7fc480150260 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc478001560 tx=0x7fc47800cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:49.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.096+0000 7fc46effd700 1 -- 172.21.15.97:0/2945074463 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478003e30 con 0x7fc4801592f0 2023-07-06T03:54:49.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.096+0000 7fc46effd700 1 -- 172.21.15.97:0/2945074463 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc478005070 con 0x7fc4801592f0 2023-07-06T03:54:49.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.096+0000 7fc484dff700 1 -- 172.21.15.97:0/2945074463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48015ac60 con 0x7fc4801592f0 2023-07-06T03:54:49.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.096+0000 7fc46effd700 1 -- 172.21.15.97:0/2945074463 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478005420 con 0x7fc4801592f0 2023-07-06T03:54:49.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.096+0000 7fc484dff700 1 -- 172.21.15.97:0/2945074463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48015b220 con 0x7fc4801592f0 2023-07-06T03:54:49.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.097+0000 7fc484dff700 1 -- 172.21.15.97:0/2945074463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48004e340 con 0x7fc4801592f0 2023-07-06T03:54:49.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.098+0000 7fc46effd700 1 -- 172.21.15.97:0/2945074463 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc478005580 con 0x7fc4801592f0 2023-07-06T03:54:49.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.099+0000 7fc46effd700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc46006ace0 0x7fc46006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.099+0000 7fc46effd700 1 -- 172.21.15.97:0/2945074463 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc478088b30 con 0x7fc4801592f0 2023-07-06T03:54:49.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.099+0000 7fc47e59c700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc46006ace0 0x7fc46006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.100+0000 7fc47e59c700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc46006ace0 0x7fc46006d190 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc470005910 tx=0x7fc470009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:49.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.103+0000 7fc46effd700 1 -- 172.21.15.97:0/2945074463 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc478053c50 con 0x7fc4801592f0 2023-07-06T03:54:49.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.299+0000 7fc484dff700 1 -- 172.21.15.97:0/2945074463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4800b1410 con 0x7fc4801592f0 2023-07-06T03:54:49.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.301+0000 7fc46effd700 1 -- 172.21.15.97:0/2945074463 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18391 (secure 0 0 0) 0x7fc478017020 con 0x7fc4801592f0 2023-07-06T03:54:49.303 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:49.303 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":112,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:54:26.969421+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:54:23.626811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-06T03:54:24.421787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:49.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 -- 172.21.15.97:0/2945074463 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc46006ace0 msgr2=0x7fc46006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc46006ace0 0x7fc46006d190 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc470005910 tx=0x7fc470009040 comp rx=0 tx=0).stop 2023-07-06T03:54:49.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 -- 172.21.15.97:0/2945074463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4801592f0 msgr2=0x7fc480150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4801592f0 0x7fc480150260 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc478001560 tx=0x7fc47800cb50 comp rx=0 tx=0).stop 2023-07-06T03:54:49.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 -- 172.21.15.97:0/2945074463 shutdown_connections 2023-07-06T03:54:49.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc46006ace0 0x7fc46006d190 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4801592f0 0x7fc480150260 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc48006d860 0x7fc4801ed9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 --2- 172.21.15.97:0/2945074463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc480158f20 0x7fc48014f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 -- 172.21.15.97:0/2945074463 >> 172.21.15.97:0/2945074463 conn(0x7fc4800ce8c0 msgr2=0x7fc480155370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:49.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 -- 172.21.15.97:0/2945074463 shutdown_connections 2023-07-06T03:54:49.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.303+0000 7fc484dff700 1 -- 172.21.15.97:0/2945074463 wait complete. 2023-07-06T03:54:49.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:54:23.626811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:54:24.421787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:54:49.324 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:54:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.519+0000 7fd343dff700 1 Processor -- start 2023-07-06T03:54:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.519+0000 7fd343dff700 1 -- start start 2023-07-06T03:54:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.521+0000 7fd343dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd33c15c610 0x7fd33c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.521+0000 7fd343dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33c0d2210 0x7fd33c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.521+0000 7fd343dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33c0cff90 con 0x7fd33c06b360 2023-07-06T03:54:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.521+0000 7fd343dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33c0d0780 con 0x7fd33c15c610 2023-07-06T03:54:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.521+0000 7fd343dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c15c9e0 con 0x7fd33c0d2210 2023-07-06T03:54:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.521+0000 7fd34139a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd33c06b360 0x7fd33c0b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59206/0 (socket says 172.21.15.97:59206) 2023-07-06T03:54:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.521+0000 7fd34139a700 1 -- 172.21.15.97:0/615778768 learned_addr learned my addr 172.21.15.97:0/615778768 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.522+0000 7fd341b9b700 1 --2- 172.21.15.97:0/615778768 >> v2:172.21.15.146:3300/0 conn(0x7fd33c15c610 0x7fd33c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.522+0000 7fd34239c700 1 --2- 172.21.15.97:0/615778768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33c0d2210 0x7fd33c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.522+0000 7fd340b99700 1 -- 172.21.15.97:0/615778768 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196349847 0 0) 0x7fd33c0cff90 con 0x7fd33c06b360 2023-07-06T03:54:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.522+0000 7fd340b99700 1 -- 172.21.15.97:0/615778768 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3280035b0 con 0x7fd33c06b360 2023-07-06T03:54:49.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.522+0000 7fd341b9b700 1 -- 172.21.15.97:0/615778768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33c0d2210 msgr2=0x7fd33c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:49.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.522+0000 7fd341b9b700 1 --2- 172.21.15.97:0/615778768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33c0d2210 0x7fd33c0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.522+0000 7fd341b9b700 1 -- 172.21.15.97:0/615778768 >> v1:172.21.15.97:6789/0 conn(0x7fd33c06b360 legacy=0x7fd33c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.522+0000 7fd341b9b700 1 -- 172.21.15.97:0/615778768 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c0b5650 con 0x7fd33c15c610 2023-07-06T03:54:49.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.522+0000 7fd341b9b700 1 --2- 172.21.15.97:0/615778768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd33c15c610 0x7fd33c0b47b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd338009840 tx=0x7fd338009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22aaf53a80e87945 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:49.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.523+0000 7fd340b99700 1 -- 172.21.15.97:0/615778768 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338011830 con 0x7fd33c15c610 2023-07-06T03:54:49.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.523+0000 7fd340b99700 1 -- 172.21.15.97:0/615778768 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd338011990 con 0x7fd33c15c610 2023-07-06T03:54:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.523+0000 7fd340b99700 1 -- 172.21.15.97:0/615778768 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338011d00 con 0x7fd33c15c610 2023-07-06T03:54:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.523+0000 7fd343dff700 1 -- 172.21.15.97:0/615778768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd33c15c610 msgr2=0x7fd33c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.523+0000 7fd343dff700 1 --2- 172.21.15.97:0/615778768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd33c15c610 0x7fd33c0b47b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd338009840 tx=0x7fd338009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.523+0000 7fd343dff700 1 -- 172.21.15.97:0/615778768 shutdown_connections 2023-07-06T03:54:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.523+0000 7fd343dff700 1 --2- 172.21.15.97:0/615778768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd33c15c610 0x7fd33c0b47b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.523+0000 7fd343dff700 1 --2- 172.21.15.97:0/615778768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33c0d2210 0x7fd33c0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.523+0000 7fd343dff700 1 -- 172.21.15.97:0/615778768 >> 172.21.15.97:0/615778768 conn(0x7fd33c06a6c0 msgr2=0x7fd33c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.523+0000 7fd343dff700 1 -- 172.21.15.97:0/615778768 shutdown_connections 2023-07-06T03:54:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.523+0000 7fd343dff700 1 -- 172.21.15.97:0/615778768 wait complete. 2023-07-06T03:54:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.525+0000 7fd343dff700 1 Processor -- start 2023-07-06T03:54:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.525+0000 7fd343dff700 1 -- start start 2023-07-06T03:54:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.526+0000 7fd343dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd33c06b360 0x7fd33c1508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.526+0000 7fd343dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33c0d2210 0x7fd33c0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.526+0000 7fd343dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33c15c610 0x7fd33c0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.526+0000 7fd343dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c150e00 con 0x7fd33c15c610 2023-07-06T03:54:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.526+0000 7fd343dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c0c72c0 con 0x7fd33c06b360 2023-07-06T03:54:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.526+0000 7fd343dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c0cb250 con 0x7fd33c0d2210 2023-07-06T03:54:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.526+0000 7fd34239c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33c15c610 0x7fd33c0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.526+0000 7fd34239c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33c15c610 0x7fd33c0c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36678/0 (socket says 172.21.15.97:36678) 2023-07-06T03:54:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.526+0000 7fd34239c700 1 -- 172.21.15.97:0/3425349629 learned_addr learned my addr 172.21.15.97:0/3425349629 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.526+0000 7fd34139a700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33c0d2210 0x7fd33c0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.526+0000 7fd341b9b700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd33c06b360 0x7fd33c1508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd34239c700 1 -- 172.21.15.97:0/3425349629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33c0d2210 msgr2=0x7fd33c0cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd34239c700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33c0d2210 0x7fd33c0cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd34239c700 1 -- 172.21.15.97:0/3425349629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd33c06b360 msgr2=0x7fd33c1508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd34239c700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd33c06b360 0x7fd33c1508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd34239c700 1 -- 172.21.15.97:0/3425349629 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3380094a0 con 0x7fd33c15c610 2023-07-06T03:54:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd341b9b700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd33c06b360 0x7fd33c1508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd34139a700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33c0d2210 0x7fd33c0cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd34239c700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33c15c610 0x7fd33c0c6860 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd33400efc0 tx=0x7fd33400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd3327fc700 1 -- 172.21.15.97:0/3425349629 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400cd40 con 0x7fd33c15c610 2023-07-06T03:54:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd3327fc700 1 -- 172.21.15.97:0/3425349629 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33400f040 con 0x7fd33c15c610 2023-07-06T03:54:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd3327fc700 1 -- 172.21.15.97:0/3425349629 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3340146c0 con 0x7fd33c15c610 2023-07-06T03:54:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd343dff700 1 -- 172.21.15.97:0/3425349629 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33c0cb530 con 0x7fd33c15c610 2023-07-06T03:54:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.527+0000 7fd343dff700 1 -- 172.21.15.97:0/3425349629 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33c0cbaf0 con 0x7fd33c15c610 2023-07-06T03:54:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.528+0000 7fd343dff700 1 -- 172.21.15.97:0/3425349629 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33c04e340 con 0x7fd33c15c610 2023-07-06T03:54:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.529+0000 7fd3327fc700 1 -- 172.21.15.97:0/3425349629 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd334014820 con 0x7fd33c15c610 2023-07-06T03:54:49.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.530+0000 7fd3327fc700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32006ad10 0x7fd32006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.530+0000 7fd3327fc700 1 -- 172.21.15.97:0/3425349629 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd334089ee0 con 0x7fd33c15c610 2023-07-06T03:54:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.530+0000 7fd341b9b700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32006ad10 0x7fd32006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.531+0000 7fd341b9b700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32006ad10 0x7fd32006d1c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd338000c00 tx=0x7fd33801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:49.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.534+0000 7fd3327fc700 1 -- 172.21.15.97:0/3425349629 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd334054e60 con 0x7fd33c15c610 2023-07-06T03:54:49.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.730+0000 7fd343dff700 1 -- 172.21.15.97:0/3425349629 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd33c15a2b0 con 0x7fd33c15c610 2023-07-06T03:54:49.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.731+0000 7fd3327fc700 1 -- 172.21.15.97:0/3425349629 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3340180a0 con 0x7fd33c15c610 2023-07-06T03:54:49.733 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:54:49.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.733+0000 7fd343dff700 1 -- 172.21.15.97:0/3425349629 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32006ad10 msgr2=0x7fd32006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.733+0000 7fd343dff700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32006ad10 0x7fd32006d1c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd338000c00 tx=0x7fd33801c040 comp rx=0 tx=0).stop 2023-07-06T03:54:49.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.733+0000 7fd343dff700 1 -- 172.21.15.97:0/3425349629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33c15c610 msgr2=0x7fd33c0c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.733+0000 7fd343dff700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33c15c610 0x7fd33c0c6860 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd33400efc0 tx=0x7fd33400c3a0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.733+0000 7fd343dff700 1 -- 172.21.15.97:0/3425349629 shutdown_connections 2023-07-06T03:54:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.733+0000 7fd343dff700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd32006ad10 0x7fd32006d1c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.733+0000 7fd343dff700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd33c15c610 0x7fd33c0c6860 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.733+0000 7fd343dff700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd33c06b360 0x7fd33c1508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.733+0000 7fd343dff700 1 --2- 172.21.15.97:0/3425349629 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd33c0d2210 0x7fd33c0cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.733+0000 7fd343dff700 1 -- 172.21.15.97:0/3425349629 >> 172.21.15.97:0/3425349629 conn(0x7fd33c06a6c0 msgr2=0x7fd33c15b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.733+0000 7fd343dff700 1 -- 172.21.15.97:0/3425349629 shutdown_connections 2023-07-06T03:54:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.734+0000 7fd343dff700 1 -- 172.21.15.97:0/3425349629 wait complete. 2023-07-06T03:54:49.753 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:54:49.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.949+0000 7fe94c1ff700 1 Processor -- start 2023-07-06T03:54:49.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.950+0000 7fe94c1ff700 1 -- start start 2023-07-06T03:54:49.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.951+0000 7fe94c1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe94415c590 0x7fe94415d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.951+0000 7fe94c1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9440719c0 0x7fe94406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.951+0000 7fe94c1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9440ccc30 con 0x7fe94415c1c0 2023-07-06T03:54:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.951+0000 7fe94c1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9440ccf10 con 0x7fe94415c590 2023-07-06T03:54:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.951+0000 7fe94c1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440cd1f0 con 0x7fe9440719c0 2023-07-06T03:54:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.951+0000 7fe949f9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe94415c1c0 0x7fe9440cc3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44340/0 (socket says 172.21.15.97:44340) 2023-07-06T03:54:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.951+0000 7fe949f9b700 1 -- 172.21.15.97:0/2534522266 learned_addr learned my addr 172.21.15.97:0/2534522266 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:49.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.951+0000 7fe94979a700 1 --2- 172.21.15.97:0/2534522266 >> v2:172.21.15.97:3300/0 conn(0x7fe94415c590 0x7fe94415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.951+0000 7fe94a79c700 1 --2- 172.21.15.97:0/2534522266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9440719c0 0x7fe94406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.951+0000 7fe948f99700 1 -- 172.21.15.97:0/2534522266 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380987712 0 0) 0x7fe9440ccc30 con 0x7fe94415c1c0 2023-07-06T03:54:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.952+0000 7fe948f99700 1 -- 172.21.15.97:0/2534522266 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9300035b0 con 0x7fe94415c1c0 2023-07-06T03:54:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.952+0000 7fe94979a700 1 -- 172.21.15.97:0/2534522266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9440719c0 msgr2=0x7fe94406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.952+0000 7fe94979a700 1 --2- 172.21.15.97:0/2534522266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9440719c0 0x7fe94406a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.952+0000 7fe94979a700 1 -- 172.21.15.97:0/2534522266 >> v1:172.21.15.97:6789/0 conn(0x7fe94415c1c0 legacy=0x7fe9440cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.952+0000 7fe94979a700 1 -- 172.21.15.97:0/2534522266 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94415cb00 con 0x7fe94415c590 2023-07-06T03:54:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.952+0000 7fe94979a700 1 --2- 172.21.15.97:0/2534522266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94415c590 0x7fe94415d300 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fe934009840 tx=0x7fe934009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49811e5c521490b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.952+0000 7fe948f99700 1 -- 172.21.15.97:0/2534522266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934011830 con 0x7fe94415c590 2023-07-06T03:54:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.952+0000 7fe948f99700 1 -- 172.21.15.97:0/2534522266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe934011990 con 0x7fe94415c590 2023-07-06T03:54:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.952+0000 7fe948f99700 1 -- 172.21.15.97:0/2534522266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934011d20 con 0x7fe94415c590 2023-07-06T03:54:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.953+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2534522266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94415c590 msgr2=0x7fe94415d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.953+0000 7fe94c1ff700 1 --2- 172.21.15.97:0/2534522266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94415c590 0x7fe94415d300 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fe934009840 tx=0x7fe934009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.953+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2534522266 shutdown_connections 2023-07-06T03:54:49.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.953+0000 7fe94c1ff700 1 --2- 172.21.15.97:0/2534522266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9440719c0 0x7fe94406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.953+0000 7fe94c1ff700 1 --2- 172.21.15.97:0/2534522266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94415c590 0x7fe94415d300 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.953+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2534522266 >> 172.21.15.97:0/2534522266 conn(0x7fe9440b1650 msgr2=0x7fe9440b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:49.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.953+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2534522266 shutdown_connections 2023-07-06T03:54:49.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.953+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2534522266 wait complete. 2023-07-06T03:54:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.954+0000 7fe94c1ff700 1 Processor -- start 2023-07-06T03:54:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.955+0000 7fe94c1ff700 1 -- start start 2023-07-06T03:54:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94c1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9440719c0 0x7fe94414e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94c1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe94415c1c0 0x7fe9440b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94c1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94415c590 0x7fe9440c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94c1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94414eb70 con 0x7fe94415c590 2023-07-06T03:54:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94c1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9441f53d0 con 0x7fe9440719c0 2023-07-06T03:54:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94c1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9441f56b0 con 0x7fe94415c1c0 2023-07-06T03:54:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94a79c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94415c590 0x7fe9440c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94a79c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94415c590 0x7fe9440c5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57566/0 (socket says 172.21.15.97:57566) 2023-07-06T03:54:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94a79c700 1 -- 172.21.15.97:0/2458009235 learned_addr learned my addr 172.21.15.97:0/2458009235 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe949f9b700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9440719c0 0x7fe94414e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94979a700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe94415c1c0 0x7fe9440b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94a79c700 1 -- 172.21.15.97:0/2458009235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe94415c1c0 msgr2=0x7fe9440b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94a79c700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe94415c1c0 0x7fe9440b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94a79c700 1 -- 172.21.15.97:0/2458009235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9440719c0 msgr2=0x7fe94414e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94a79c700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9440719c0 0x7fe94414e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.956+0000 7fe94a79c700 1 -- 172.21.15.97:0/2458009235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9340094a0 con 0x7fe94415c590 2023-07-06T03:54:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.957+0000 7fe94a79c700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94415c590 0x7fe9440c5a40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe93c007e60 tx=0x7fe93c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.957+0000 7fe93a7fc700 1 -- 172.21.15.97:0/2458009235 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c00ed30 con 0x7fe94415c590 2023-07-06T03:54:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.957+0000 7fe93a7fc700 1 -- 172.21.15.97:0/2458009235 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c014420 con 0x7fe94415c590 2023-07-06T03:54:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.957+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2458009235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9441f5990 con 0x7fe94415c590 2023-07-06T03:54:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.957+0000 7fe93a7fc700 1 -- 172.21.15.97:0/2458009235 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c0147d0 con 0x7fe94415c590 2023-07-06T03:54:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.957+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2458009235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9441f4ff0 con 0x7fe94415c590 2023-07-06T03:54:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.958+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2458009235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94404e340 con 0x7fe94415c590 2023-07-06T03:54:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.959+0000 7fe93a7fc700 1 -- 172.21.15.97:0/2458009235 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe93c014930 con 0x7fe94415c590 2023-07-06T03:54:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.960+0000 7fe93a7fc700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe92806ace0 0x7fe92806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.961+0000 7fe93a7fc700 1 -- 172.21.15.97:0/2458009235 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe93c089ba0 con 0x7fe94415c590 2023-07-06T03:54:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.961+0000 7fe949f9b700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe92806ace0 0x7fe92806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.961+0000 7fe949f9b700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe92806ace0 0x7fe92806d190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe940007fd0 tx=0x7fe940007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:49.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:49.965+0000 7fe93a7fc700 1 -- 172.21.15.97:0/2458009235 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe93c054c60 con 0x7fe94415c590 2023-07-06T03:54:50.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.165+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2458009235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe944071290 con 0x7fe94415c590 2023-07-06T03:54:50.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.166+0000 7fe93a7fc700 1 -- 172.21.15.97:0/2458009235 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v112) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe93c058280 con 0x7fe94415c590 2023-07-06T03:54:50.168 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:54:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2458009235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe92806ace0 msgr2=0x7fe92806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe92806ace0 0x7fe92806d190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe940007fd0 tx=0x7fe940007dc0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2458009235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94415c590 msgr2=0x7fe9440c5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94415c590 0x7fe9440c5a40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe93c007e60 tx=0x7fe93c00e3c0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2458009235 shutdown_connections 2023-07-06T03:54:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe92806ace0 0x7fe92806d190 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94415c590 0x7fe9440c5a40 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9440719c0 0x7fe94414e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 --2- 172.21.15.97:0/2458009235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe94415c1c0 0x7fe9440b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2458009235 >> 172.21.15.97:0/2458009235 conn(0x7fe9440b1650 msgr2=0x7fe9440acc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2458009235 shutdown_connections 2023-07-06T03:54:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.168+0000 7fe94c1ff700 1 -- 172.21.15.97:0/2458009235 wait complete. 2023-07-06T03:54:50.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:54:50.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.383+0000 7fb4a2dff700 1 Processor -- start 2023-07-06T03:54:50.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.383+0000 7fb4a2dff700 1 -- start start 2023-07-06T03:54:50.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.384+0000 7fb4a2dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb49c06b360 0x7fb49c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.384+0000 7fb4a2dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49c0cff90 con 0x7fb49c15c610 2023-07-06T03:54:50.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.384+0000 7fb4a2dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49c0b5650 con 0x7fb49c0d2210 2023-07-06T03:54:50.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.384+0000 7fb4a2dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49c0d0780 con 0x7fb49c06b360 2023-07-06T03:54:50.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb4a0b9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb49c15c610 0x7fb49c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44352/0 (socket says 172.21.15.97:44352) 2023-07-06T03:54:50.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb4a0b9b700 1 -- 172.21.15.97:0/1508041806 learned_addr learned my addr 172.21.15.97:0/1508041806 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:50.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942412062 0 0) 0x7fb49c0cff90 con 0x7fb49c15c610 2023-07-06T03:54:50.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49bfff700 1 --2- 172.21.15.97:0/1508041806 >> v2:172.21.15.146:3300/0 conn(0x7fb49c06b360 0x7fb49c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb48c0035b0 con 0x7fb49c15c610 2023-07-06T03:54:50.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 210962399 0 0) 0x7fb48c0035b0 con 0x7fb49c15c610 2023-07-06T03:54:50.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 >> v2:172.21.15.146:3300/0 conn(0x7fb49c06b360 msgr2=0x7fb49c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:50.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49b7fe700 1 --2- 172.21.15.97:0/1508041806 >> v2:172.21.15.146:3300/0 conn(0x7fb49c06b360 0x7fb49c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 >> v1:172.21.15.146:6789/0 conn(0x7fb49c0d2210 legacy=0x7fb49c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c0b5400 con 0x7fb49c15c610 2023-07-06T03:54:50.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fb490001670 con 0x7fb49c15c610 2023-07-06T03:54:50.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 >> v1:172.21.15.97:6789/0 conn(0x7fb49c15c610 legacy=0x7fb49c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49b7fe700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb48c003a50 0x7fb48c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.385+0000 7fb49b7fe700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 0x7fb48c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb49b7fe700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb48c005200 0x7fb48c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0d0780 con 0x7fb48c004850 2023-07-06T03:54:50.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c00c0e0 con 0x7fb48c003a50 2023-07-06T03:54:50.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c00c2a0 con 0x7fb48c005200 2023-07-06T03:54:50.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb49bfff700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 0x7fb48c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb4a0b9b700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb48c003a50 0x7fb48c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb4a139c700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb48c005200 0x7fb48c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb49bfff700 1 -- 172.21.15.97:0/1508041806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb48c005200 msgr2=0x7fb48c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:50.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb49bfff700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb48c005200 0x7fb48c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb49bfff700 1 -- 172.21.15.97:0/1508041806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb48c003a50 msgr2=0x7fb48c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:50.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb49bfff700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb48c003a50 0x7fb48c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb49bfff700 1 -- 172.21.15.97:0/1508041806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c0035b0 con 0x7fb48c004850 2023-07-06T03:54:50.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb4a0b9b700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb48c003a50 0x7fb48c003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:50.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb4a139c700 1 -- 172.21.15.97:0/1508041806 reap_dead start 2023-07-06T03:54:50.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.386+0000 7fb49bfff700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 0x7fb48c004cc0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fb488009b30 tx=0x7fb48c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec7f1dd56940998 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:50.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.387+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48800e6c0 con 0x7fb48c004850 2023-07-06T03:54:50.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.387+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48800e820 con 0x7fb48c004850 2023-07-06T03:54:50.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.387+0000 7fb49b7fe700 1 -- 172.21.15.97:0/1508041806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48800eb90 con 0x7fb48c004850 2023-07-06T03:54:50.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.387+0000 7fb4a2dff700 1 -- 172.21.15.97:0/1508041806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 msgr2=0x7fb48c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.387+0000 7fb4a2dff700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 0x7fb48c004cc0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fb488009b30 tx=0x7fb48c005880 comp rx=0 tx=0).stop 2023-07-06T03:54:50.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.387+0000 7fb4a2dff700 1 -- 172.21.15.97:0/1508041806 shutdown_connections 2023-07-06T03:54:50.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.387+0000 7fb4a2dff700 1 --2- 172.21.15.97:0/1508041806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 0x7fb48c004cc0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.387+0000 7fb4a2dff700 1 -- 172.21.15.97:0/1508041806 >> 172.21.15.97:0/1508041806 conn(0x7fb49c06a6c0 msgr2=0x7fb49c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:50.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.387+0000 7fb4a2dff700 1 -- 172.21.15.97:0/1508041806 shutdown_connections 2023-07-06T03:54:50.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.387+0000 7fb4a2dff700 1 -- 172.21.15.97:0/1508041806 wait complete. 2023-07-06T03:54:50.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.390+0000 7fb4a2dff700 1 Processor -- start 2023-07-06T03:54:50.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.390+0000 7fb4a2dff700 1 -- start start 2023-07-06T03:54:50.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.391+0000 7fb4a2dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 0x7fb49c0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.391+0000 7fb4a2dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb49c152ed0 0x7fb49c0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.391+0000 7fb4a2dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49c0cadc0 0x7fb49c15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.391+0000 7fb4a2dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0cb230 con 0x7fb48c004850 2023-07-06T03:54:50.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.391+0000 7fb4a2dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c15dc90 con 0x7fb49c152ed0 2023-07-06T03:54:50.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.391+0000 7fb4a2dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0c9260 con 0x7fb49c0cadc0 2023-07-06T03:54:50.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.391+0000 7fb4a0b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 0x7fb49c0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.391+0000 7fb4a0b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 0x7fb49c0c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57574/0 (socket says 172.21.15.97:57574) 2023-07-06T03:54:50.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.391+0000 7fb4a0b9b700 1 -- 172.21.15.97:0/3625113493 learned_addr learned my addr 172.21.15.97:0/3625113493 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:50.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.391+0000 7fb49bfff700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb49c152ed0 0x7fb49c0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.391+0000 7fb4a139c700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49c0cadc0 0x7fb49c15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4a0b9b700 1 -- 172.21.15.97:0/3625113493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49c0cadc0 msgr2=0x7fb49c15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4a0b9b700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49c0cadc0 0x7fb49c15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4a0b9b700 1 -- 172.21.15.97:0/3625113493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb49c152ed0 msgr2=0x7fb49c0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4a0b9b700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb49c152ed0 0x7fb49c0ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4a0b9b700 1 -- 172.21.15.97:0/3625113493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c0c9540 con 0x7fb48c004850 2023-07-06T03:54:50.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4a139c700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49c0cadc0 0x7fb49c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:50.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb49bfff700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb49c152ed0 0x7fb49c0ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:50.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4a0b9b700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 0x7fb49c0c9e00 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fb49000e780 tx=0x7fb49000eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:50.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4997fa700 1 -- 172.21.15.97:0/3625113493 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490012590 con 0x7fb48c004850 2023-07-06T03:54:50.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4997fa700 1 -- 172.21.15.97:0/3625113493 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4900126f0 con 0x7fb48c004850 2023-07-06T03:54:50.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4a2dff700 1 -- 172.21.15.97:0/3625113493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c15dee0 con 0x7fb48c004850 2023-07-06T03:54:50.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4997fa700 1 -- 172.21.15.97:0/3625113493 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490012a80 con 0x7fb48c004850 2023-07-06T03:54:50.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.392+0000 7fb4a2dff700 1 -- 172.21.15.97:0/3625113493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c153340 con 0x7fb48c004850 2023-07-06T03:54:50.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.393+0000 7fb4a2dff700 1 -- 172.21.15.97:0/3625113493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c15e400 con 0x7fb48c004850 2023-07-06T03:54:50.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.394+0000 7fb4997fa700 1 -- 172.21.15.97:0/3625113493 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb490012be0 con 0x7fb48c004850 2023-07-06T03:54:50.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.395+0000 7fb4997fa700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb48006ace0 0x7fb48006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.395+0000 7fb4997fa700 1 -- 172.21.15.97:0/3625113493 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4900892d0 con 0x7fb48c004850 2023-07-06T03:54:50.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.396+0000 7fb49bfff700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb48006ace0 0x7fb48006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.396+0000 7fb49bfff700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb48006ace0 0x7fb48006d190 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0c8bb0 tx=0x7fb488008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:50.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.399+0000 7fb4997fa700 1 -- 172.21.15.97:0/3625113493 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb49c15e400 con 0x7fb48c004850 2023-07-06T03:54:50.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.594+0000 7fb4a2dff700 1 -- 172.21.15.97:0/3625113493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb49c15e400 con 0x7fb48c004850 2023-07-06T03:54:50.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.594+0000 7fb4997fa700 1 -- 172.21.15.97:0/3625113493 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb49c15e400 con 0x7fb48c004850 2023-07-06T03:54:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:54:50.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 -- 172.21.15.97:0/3625113493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb48006ace0 msgr2=0x7fb48006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb48006ace0 0x7fb48006d190 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0c8bb0 tx=0x7fb488008030 comp rx=0 tx=0).stop 2023-07-06T03:54:50.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 -- 172.21.15.97:0/3625113493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 msgr2=0x7fb49c0c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 0x7fb49c0c9e00 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fb49000e780 tx=0x7fb49000eb40 comp rx=0 tx=0).stop 2023-07-06T03:54:50.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 -- 172.21.15.97:0/3625113493 shutdown_connections 2023-07-06T03:54:50.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb48006ace0 0x7fb48006d190 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb49c0cadc0 0x7fb49c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb48c004850 0x7fb49c0c9e00 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 --2- 172.21.15.97:0/3625113493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb49c152ed0 0x7fb49c0ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 -- 172.21.15.97:0/3625113493 >> 172.21.15.97:0/3625113493 conn(0x7fb49c06a6c0 msgr2=0x7fb49c0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:50.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 -- 172.21.15.97:0/3625113493 shutdown_connections 2023-07-06T03:54:50.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.597+0000 7fb4a2dff700 1 -- 172.21.15.97:0/3625113493 wait complete. 2023-07-06T03:54:50.616 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:54:50.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.813+0000 7f5d433ff700 1 Processor -- start 2023-07-06T03:54:50.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.813+0000 7f5d433ff700 1 -- start start 2023-07-06T03:54:50.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.814+0000 7f5d433ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c06b360 0x7f5d3c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.814+0000 7f5d433ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c0cff90 con 0x7f5d3c0d2210 2023-07-06T03:54:50.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.814+0000 7f5d433ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c0b5650 con 0x7f5d3c15c610 2023-07-06T03:54:50.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.814+0000 7f5d433ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0d0780 con 0x7f5d3c06b360 2023-07-06T03:54:50.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d4199c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5d3c0d2210 0x7f5d3c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44358/0 (socket says 172.21.15.97:44358) 2023-07-06T03:54:50.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d4199c700 1 -- 172.21.15.97:0/3767137852 learned_addr learned my addr 172.21.15.97:0/3767137852 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:50.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d4099a700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c06b360 0x7f5d3c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241928343 0 0) 0x7f5d3c0cff90 con 0x7f5d3c0d2210 2023-07-06T03:54:50.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d280035b0 con 0x7f5d3c0d2210 2023-07-06T03:54:50.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069395926 0 0) 0x7f5d3c0b5650 con 0x7f5d3c15c610 2023-07-06T03:54:50.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d3c0cff90 con 0x7f5d3c15c610 2023-07-06T03:54:50.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4036802371 0 0) 0x7f5d280035b0 con 0x7f5d3c0d2210 2023-07-06T03:54:50.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c06b360 msgr2=0x7f5d3c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:50.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d33fff700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c06b360 0x7f5d3c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 >> v1:172.21.15.146:6789/0 conn(0x7f5d3c15c610 legacy=0x7f5d3c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0b5400 con 0x7f5d3c0d2210 2023-07-06T03:54:50.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f5d340019f0 con 0x7f5d3c0d2210 2023-07-06T03:54:50.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.815+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 >> v1:172.21.15.97:6789/0 conn(0x7f5d3c0d2210 legacy=0x7f5d3c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d33fff700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 0x7f5d28003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d33fff700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c06b360 0x7f5d28004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d33fff700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d28004d90 0x7f5d28005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0d0780 con 0x7f5d28003bc0 2023-07-06T03:54:50.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2800bac0 con 0x7f5d28004d90 2023-07-06T03:54:50.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2800bc80 con 0x7f5d3c06b360 2023-07-06T03:54:50.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d4119b700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 0x7f5d28003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d4199c700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d28004d90 0x7f5d28005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d4099a700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c06b360 0x7f5d28004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d4119b700 1 -- 172.21.15.97:0/3767137852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c06b360 msgr2=0x7f5d28004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:50.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d4119b700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c06b360 0x7f5d28004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d4119b700 1 -- 172.21.15.97:0/3767137852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d28004d90 msgr2=0x7f5d28005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:50.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d4119b700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d28004d90 0x7f5d28005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d4119b700 1 -- 172.21.15.97:0/3767137852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d280035b0 con 0x7f5d28003bc0 2023-07-06T03:54:50.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.816+0000 7f5d4119b700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 0x7f5d28003fb0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f5d38007fd0 tx=0x7f5d28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353ac81477610d53 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:50.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.817+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38001d80 con 0x7f5d28003bc0 2023-07-06T03:54:50.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.817+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d380052d0 con 0x7f5d28003bc0 2023-07-06T03:54:50.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.817+0000 7f5d33fff700 1 -- 172.21.15.97:0/3767137852 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38001470 con 0x7f5d28003bc0 2023-07-06T03:54:50.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.817+0000 7f5d433ff700 1 -- 172.21.15.97:0/3767137852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 msgr2=0x7f5d28003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.817+0000 7f5d433ff700 1 --2- 172.21.15.97:0/3767137852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 0x7f5d28003fb0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f5d38007fd0 tx=0x7f5d28005880 comp rx=0 tx=0).stop 2023-07-06T03:54:50.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.817+0000 7f5d4199c700 1 -- 172.21.15.97:0/3767137852 reap_dead start 2023-07-06T03:54:50.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.817+0000 7f5d433ff700 1 -- 172.21.15.97:0/3767137852 shutdown_connections 2023-07-06T03:54:50.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.817+0000 7f5d433ff700 1 -- 172.21.15.97:0/3767137852 >> 172.21.15.97:0/3767137852 conn(0x7f5d3c06a6c0 msgr2=0x7f5d3c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:50.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.817+0000 7f5d433ff700 1 -- 172.21.15.97:0/3767137852 shutdown_connections 2023-07-06T03:54:50.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.817+0000 7f5d433ff700 1 -- 172.21.15.97:0/3767137852 wait complete. 2023-07-06T03:54:50.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.819+0000 7f5d433ff700 1 Processor -- start 2023-07-06T03:54:50.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.819+0000 7f5d433ff700 1 -- start start 2023-07-06T03:54:50.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.820+0000 7f5d433ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 0x7f5d3c14fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.820+0000 7f5d433ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c0cab40 0x7f5d3c0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.820+0000 7f5d433ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c0b3c60 0x7f5d3c0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.820+0000 7f5d433ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0b4620 con 0x7f5d28003bc0 2023-07-06T03:54:50.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.820+0000 7f5d433ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0cbdc0 con 0x7f5d3c0cab40 2023-07-06T03:54:50.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.820+0000 7f5d433ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0cc0a0 con 0x7f5d3c0b3c60 2023-07-06T03:54:50.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.820+0000 7f5d4119b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 0x7f5d3c14fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.820+0000 7f5d4119b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 0x7f5d3c14fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57584/0 (socket says 172.21.15.97:57584) 2023-07-06T03:54:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.820+0000 7f5d4119b700 1 -- 172.21.15.97:0/743676887 learned_addr learned my addr 172.21.15.97:0/743676887 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d4099a700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c0cab40 0x7f5d3c0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d4199c700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c0b3c60 0x7f5d3c0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d4119b700 1 -- 172.21.15.97:0/743676887 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c0b3c60 msgr2=0x7f5d3c0b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d4119b700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c0b3c60 0x7f5d3c0b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d4119b700 1 -- 172.21.15.97:0/743676887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c0cab40 msgr2=0x7f5d3c0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d4119b700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c0cab40 0x7f5d3c0cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d4119b700 1 -- 172.21.15.97:0/743676887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0cc380 con 0x7f5d28003bc0 2023-07-06T03:54:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d4099a700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c0cab40 0x7f5d3c0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d4119b700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 0x7f5d3c14fe70 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006b10 tx=0x7f5d38001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d31ffb700 1 -- 172.21.15.97:0/743676887 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38001d80 con 0x7f5d28003bc0 2023-07-06T03:54:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d31ffb700 1 -- 172.21.15.97:0/743676887 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38001410 con 0x7f5d28003bc0 2023-07-06T03:54:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d433ff700 1 -- 172.21.15.97:0/743676887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c0cc660 con 0x7f5d28003bc0 2023-07-06T03:54:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.821+0000 7f5d31ffb700 1 -- 172.21.15.97:0/743676887 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38007280 con 0x7f5d28003bc0 2023-07-06T03:54:50.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.822+0000 7f5d433ff700 1 -- 172.21.15.97:0/743676887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c0cb990 con 0x7f5d28003bc0 2023-07-06T03:54:50.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.823+0000 7f5d433ff700 1 -- 172.21.15.97:0/743676887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c0cbb20 con 0x7f5d28003bc0 2023-07-06T03:54:50.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.823+0000 7f5d31ffb700 1 -- 172.21.15.97:0/743676887 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d380073e0 con 0x7f5d28003bc0 2023-07-06T03:54:50.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.824+0000 7f5d31ffb700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d2006ace0 0x7f5d2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:50.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.824+0000 7f5d31ffb700 1 -- 172.21.15.97:0/743676887 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d3808a8e0 con 0x7f5d28003bc0 2023-07-06T03:54:50.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.825+0000 7f5d4099a700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d2006ace0 0x7f5d2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:50.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.825+0000 7f5d4099a700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d2006ace0 0x7f5d2006d190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0caf70 tx=0x7f5d2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:50.828+0000 7f5d31ffb700 1 -- 172.21.15.97:0/743676887 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d38055a20 con 0x7f5d28003bc0 2023-07-06T03:54:51.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.031+0000 7f5d433ff700 1 -- 172.21.15.97:0/743676887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d3c04e340 con 0x7f5d28003bc0 2023-07-06T03:54:51.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.032+0000 7f5d31ffb700 1 -- 172.21.15.97:0/743676887 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d38017020 con 0x7f5d28003bc0 2023-07-06T03:54:51.034 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:54:51.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.034+0000 7f5d433ff700 1 -- 172.21.15.97:0/743676887 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d2006ace0 msgr2=0x7f5d2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:51.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.034+0000 7f5d433ff700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d2006ace0 0x7f5d2006d190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0caf70 tx=0x7f5d2c008040 comp rx=0 tx=0).stop 2023-07-06T03:54:51.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.034+0000 7f5d433ff700 1 -- 172.21.15.97:0/743676887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 msgr2=0x7f5d3c14fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.034+0000 7f5d433ff700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 0x7f5d3c14fe70 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006b10 tx=0x7f5d38001310 comp rx=0 tx=0).stop 2023-07-06T03:54:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.034+0000 7f5d433ff700 1 -- 172.21.15.97:0/743676887 shutdown_connections 2023-07-06T03:54:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.034+0000 7f5d433ff700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d2006ace0 0x7f5d2006d190 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.034+0000 7f5d433ff700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d3c0b3c60 0x7f5d3c0b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.034+0000 7f5d433ff700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d28003bc0 0x7f5d3c14fe70 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.034+0000 7f5d433ff700 1 --2- 172.21.15.97:0/743676887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c0cab40 0x7f5d3c0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.034+0000 7f5d433ff700 1 -- 172.21.15.97:0/743676887 >> 172.21.15.97:0/743676887 conn(0x7f5d3c06a6c0 msgr2=0x7f5d3c15a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.034+0000 7f5d433ff700 1 -- 172.21.15.97:0/743676887 shutdown_connections 2023-07-06T03:54:51.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.035+0000 7f5d433ff700 1 -- 172.21.15.97:0/743676887 wait complete. 2023-07-06T03:54:51.054 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:54:51.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.247+0000 7fe6531ff700 1 Processor -- start 2023-07-06T03:54:51.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.248+0000 7fe6531ff700 1 -- start start 2023-07-06T03:54:51.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.249+0000 7fe6531ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe64c0c3cd0 0x7fe64c0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:51.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.249+0000 7fe6531ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe64c159440 0x7fe64c0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:51.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.249+0000 7fe6531ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64c0ca260 con 0x7fe64c159070 2023-07-06T03:54:51.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.249+0000 7fe6531ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64c0c99c0 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.249+0000 7fe6531ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64c0c9ca0 con 0x7fe64c159440 2023-07-06T03:54:51.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.249+0000 7fe64bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe64c0c3cd0 0x7fe64c0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:51.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.249+0000 7fe64bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe64c0c3cd0 0x7fe64c0cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57590/0 (socket says 172.21.15.97:57590) 2023-07-06T03:54:51.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.249+0000 7fe64bfff700 1 -- 172.21.15.97:0/2642327759 learned_addr learned my addr 172.21.15.97:0/2642327759 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:51.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.250+0000 7fe65179c700 1 --2- 172.21.15.97:0/2642327759 >> v2:172.21.15.146:3300/0 conn(0x7fe64c159440 0x7fe64c0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:51.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.250+0000 7fe64bfff700 1 -- 172.21.15.97:0/2642327759 >> v2:172.21.15.146:3300/0 conn(0x7fe64c159440 msgr2=0x7fe64c0cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:51.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.250+0000 7fe64bfff700 1 --2- 172.21.15.97:0/2642327759 >> v2:172.21.15.146:3300/0 conn(0x7fe64c159440 0x7fe64c0cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.250+0000 7fe64bfff700 1 -- 172.21.15.97:0/2642327759 >> v1:172.21.15.146:6789/0 conn(0x7fe64c159070 legacy=0x7fe64c0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:51.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.250+0000 7fe64bfff700 1 -- 172.21.15.97:0/2642327759 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c0c9f80 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.250+0000 7fe65179c700 1 --2- 172.21.15.97:0/2642327759 >> v2:172.21.15.146:3300/0 conn(0x7fe64c159440 0x7fe64c0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:51.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.250+0000 7fe64bfff700 1 --2- 172.21.15.97:0/2642327759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe64c0c3cd0 0x7fe64c0cc4c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe64c071f70 tx=0x7fe63800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180f17f62da06b16 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.250+0000 7fe64b7fe700 1 -- 172.21.15.97:0/2642327759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63800dcf0 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.250+0000 7fe64b7fe700 1 -- 172.21.15.97:0/2642327759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63800de50 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.250+0000 7fe64b7fe700 1 -- 172.21.15.97:0/2642327759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638004250 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.251+0000 7fe6531ff700 1 -- 172.21.15.97:0/2642327759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe64c0c3cd0 msgr2=0x7fe64c0cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.251+0000 7fe6531ff700 1 --2- 172.21.15.97:0/2642327759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe64c0c3cd0 0x7fe64c0cc4c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe64c071f70 tx=0x7fe63800be70 comp rx=0 tx=0).stop 2023-07-06T03:54:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.251+0000 7fe6531ff700 1 -- 172.21.15.97:0/2642327759 shutdown_connections 2023-07-06T03:54:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.251+0000 7fe6531ff700 1 --2- 172.21.15.97:0/2642327759 >> v2:172.21.15.146:3300/0 conn(0x7fe64c159440 0x7fe64c0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.251+0000 7fe6531ff700 1 --2- 172.21.15.97:0/2642327759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe64c0c3cd0 0x7fe64c0cc4c0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.251+0000 7fe6531ff700 1 -- 172.21.15.97:0/2642327759 >> 172.21.15.97:0/2642327759 conn(0x7fe64c06a060 msgr2=0x7fe64c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:51.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.251+0000 7fe6531ff700 1 -- 172.21.15.97:0/2642327759 shutdown_connections 2023-07-06T03:54:51.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.251+0000 7fe6531ff700 1 -- 172.21.15.97:0/2642327759 wait complete. 2023-07-06T03:54:51.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.253+0000 7fe6531ff700 1 Processor -- start 2023-07-06T03:54:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.253+0000 7fe6531ff700 1 -- start start 2023-07-06T03:54:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe6531ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe64c0c3cd0 0x7fe64c151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe6531ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe64c159070 0x7fe64c0aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe6531ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe64c159440 0x7fe64c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe6531ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c0b0300 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe6531ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c0af780 con 0x7fe64c159440 2023-07-06T03:54:51.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe6531ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c0afa60 con 0x7fe64c159070 2023-07-06T03:54:51.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe650f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe64c0c3cd0 0x7fe64c151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:51.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe650f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe64c0c3cd0 0x7fe64c151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57600/0 (socket says 172.21.15.97:57600) 2023-07-06T03:54:51.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe650f9b700 1 -- 172.21.15.97:0/2947242131 learned_addr learned my addr 172.21.15.97:0/2947242131 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:51.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe650f9b700 1 -- 172.21.15.97:0/2947242131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe64c159070 msgr2=0x7fe64c0aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:51.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe650f9b700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe64c159070 0x7fe64c0aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe650f9b700 1 -- 172.21.15.97:0/2947242131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe64c159440 msgr2=0x7fe64c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:51.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe650f9b700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe64c159440 0x7fe64c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.255+0000 7fe650f9b700 1 -- 172.21.15.97:0/2947242131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63800b7f0 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.256+0000 7fe650f9b700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe64c0c3cd0 0x7fe64c151c40 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c4d0 tx=0x7fe64000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:51.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.256+0000 7fe6497fa700 1 -- 172.21.15.97:0/2947242131 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640011790 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.256+0000 7fe6497fa700 1 -- 172.21.15.97:0/2947242131 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6400118f0 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.256+0000 7fe6531ff700 1 -- 172.21.15.97:0/2947242131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64c0afd40 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.256+0000 7fe6497fa700 1 -- 172.21.15.97:0/2947242131 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640011ca0 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.256+0000 7fe6531ff700 1 -- 172.21.15.97:0/2947242131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64c0af490 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.257+0000 7fe6531ff700 1 -- 172.21.15.97:0/2947242131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64c04e340 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.258+0000 7fe6497fa700 1 -- 172.21.15.97:0/2947242131 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe64001c460 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.259+0000 7fe6497fa700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe63006ace0 0x7fe63006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.259+0000 7fe6497fa700 1 -- 172.21.15.97:0/2947242131 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe640089360 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.259+0000 7fe64bfff700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe63006ace0 0x7fe63006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.260+0000 7fe64bfff700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe63006ace0 0x7fe63006d190 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0c40d0 tx=0x7fe63800bf90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:51.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.263+0000 7fe6497fa700 1 -- 172.21.15.97:0/2947242131 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe640054370 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.477+0000 7fe6531ff700 1 -- 172.21.15.97:0/2947242131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe64c06d220 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.477+0000 7fe6497fa700 1 -- 172.21.15.97:0/2947242131 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe640057990 con 0x7fe64c0c3cd0 2023-07-06T03:54:51.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 -- 172.21.15.97:0/2947242131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe63006ace0 msgr2=0x7fe63006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:51.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe63006ace0 0x7fe63006d190 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0c40d0 tx=0x7fe63800bf90 comp rx=0 tx=0).stop 2023-07-06T03:54:51.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 -- 172.21.15.97:0/2947242131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe64c0c3cd0 msgr2=0x7fe64c151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:51.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe64c0c3cd0 0x7fe64c151c40 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c4d0 tx=0x7fe64000c890 comp rx=0 tx=0).stop 2023-07-06T03:54:51.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 -- 172.21.15.97:0/2947242131 shutdown_connections 2023-07-06T03:54:51.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe63006ace0 0x7fe63006d190 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe64c159440 0x7fe64c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe64c0c3cd0 0x7fe64c151c40 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 --2- 172.21.15.97:0/2947242131 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe64c159070 0x7fe64c0aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 -- 172.21.15.97:0/2947242131 >> 172.21.15.97:0/2947242131 conn(0x7fe64c06a060 msgr2=0x7fe64c06bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 -- 172.21.15.97:0/2947242131 shutdown_connections 2023-07-06T03:54:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:51.480+0000 7fe6531ff700 1 -- 172.21.15.97:0/2947242131 wait complete. 2023-07-06T03:54:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:54:51.500 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:54:51.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:51.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:54:51.566 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[158938]: fuse finished with error 0 and tester_r 0 2023-07-06T03:54:51.570 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:54:51.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:51.571 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:54:51.755 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:54:51.756 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:51.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:52.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.000+0000 7f481dfff700 1 Processor -- start 2023-07-06T03:54:52.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.001+0000 7f481dfff700 1 -- start start 2023-07-06T03:54:52.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f481dfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48180cd390 0x7f4818160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f481dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4818160dd0 0x7f48180b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f481dfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48180cdae0 con 0x7f481815c520 2023-07-06T03:54:52.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f481dfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48180b6640 con 0x7f48180cd390 2023-07-06T03:54:52.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f481dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48180b6920 con 0x7f4818160dd0 2023-07-06T03:54:52.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f4816ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48180cd390 0x7f4818160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f4816ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48180cd390 0x7f4818160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57606/0 (socket says 172.21.15.97:57606) 2023-07-06T03:54:52.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f4816ffd700 1 -- 172.21.15.97:0/3987674779 learned_addr learned my addr 172.21.15.97:0/3987674779 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:52.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f4817fff700 1 --2- 172.21.15.97:0/3987674779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4818160dd0 0x7f48180b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f4816ffd700 1 -- 172.21.15.97:0/3987674779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4818160dd0 msgr2=0x7f48180b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:52.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f4816ffd700 1 --2- 172.21.15.97:0/3987674779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4818160dd0 0x7f48180b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f4816ffd700 1 -- 172.21.15.97:0/3987674779 >> v1:172.21.15.146:6789/0 conn(0x7f481815c520 legacy=0x7f4818160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.003+0000 7f4816ffd700 1 -- 172.21.15.97:0/3987674779 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48180b6c00 con 0x7f48180cd390 2023-07-06T03:54:52.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.002+0000 7f48167fc700 1 -- 172.21.15.97:0/3987674779 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628214413 0 0) 0x7f48180cdae0 con 0x7f481815c520 2023-07-06T03:54:52.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.003+0000 7f4816ffd700 1 --2- 172.21.15.97:0/3987674779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48180cd390 0x7f4818160890 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f480c0098f0 tx=0x7f480c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d77a4f4ddffbfc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:52.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.003+0000 7f48167fc700 1 -- 172.21.15.97:0/3987674779 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c011830 con 0x7f48180cd390 2023-07-06T03:54:52.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.003+0000 7f48167fc700 1 -- 172.21.15.97:0/3987674779 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c011990 con 0x7f48180cd390 2023-07-06T03:54:52.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.003+0000 7f48167fc700 1 -- 172.21.15.97:0/3987674779 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c011d20 con 0x7f48180cd390 2023-07-06T03:54:52.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.003+0000 7f481dfff700 1 -- 172.21.15.97:0/3987674779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48180cd390 msgr2=0x7f4818160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.003+0000 7f481dfff700 1 --2- 172.21.15.97:0/3987674779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48180cd390 0x7f4818160890 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f480c0098f0 tx=0x7f480c009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:52.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.004+0000 7f481dfff700 1 -- 172.21.15.97:0/3987674779 shutdown_connections 2023-07-06T03:54:52.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.004+0000 7f481dfff700 1 --2- 172.21.15.97:0/3987674779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4818160dd0 0x7f48180b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.004+0000 7f481dfff700 1 --2- 172.21.15.97:0/3987674779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48180cd390 0x7f4818160890 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.004+0000 7f481dfff700 1 -- 172.21.15.97:0/3987674779 >> 172.21.15.97:0/3987674779 conn(0x7f4818071790 msgr2=0x7f4818071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:52.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.004+0000 7f481dfff700 1 -- 172.21.15.97:0/3987674779 shutdown_connections 2023-07-06T03:54:52.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.004+0000 7f481dfff700 1 -- 172.21.15.97:0/3987674779 wait complete. 2023-07-06T03:54:52.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.005+0000 7f481dfff700 1 Processor -- start 2023-07-06T03:54:52.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.005+0000 7f481dfff700 1 -- start start 2023-07-06T03:54:52.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.006+0000 7f481dfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48180cd390 0x7f481815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.006+0000 7f481dfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481815c520 0x7f48180c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.006+0000 7f481dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4818160dd0 0x7f48181f5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.006+0000 7f481dfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481815eaf0 con 0x7f48180cd390 2023-07-06T03:54:52.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.006+0000 7f481dfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481815bcc0 con 0x7f481815c520 2023-07-06T03:54:52.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.006+0000 7f481dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481815d800 con 0x7f4818160dd0 2023-07-06T03:54:52.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f48177fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48180cd390 0x7f481815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f48177fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48180cd390 0x7f481815b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57612/0 (socket says 172.21.15.97:57612) 2023-07-06T03:54:52.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f48177fe700 1 -- 172.21.15.97:0/3681869029 learned_addr learned my addr 172.21.15.97:0/3681869029 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f4816ffd700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481815c520 0x7f48180c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f4817fff700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4818160dd0 0x7f48181f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f48177fe700 1 -- 172.21.15.97:0/3681869029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4818160dd0 msgr2=0x7f48181f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f48177fe700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4818160dd0 0x7f48181f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f48177fe700 1 -- 172.21.15.97:0/3681869029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481815c520 msgr2=0x7f48180c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f48177fe700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481815c520 0x7f48180c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f48177fe700 1 -- 172.21.15.97:0/3681869029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c009550 con 0x7f48180cd390 2023-07-06T03:54:52.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f48177fe700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48180cd390 0x7f481815b780 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f480800c750 tx=0x7f480800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:52.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f47fffff700 1 -- 172.21.15.97:0/3681869029 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48080054e0 con 0x7f48180cd390 2023-07-06T03:54:52.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f47fffff700 1 -- 172.21.15.97:0/3681869029 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4808003680 con 0x7f48180cd390 2023-07-06T03:54:52.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f47fffff700 1 -- 172.21.15.97:0/3681869029 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808003a30 con 0x7f48180cd390 2023-07-06T03:54:52.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.007+0000 7f481dfff700 1 -- 172.21.15.97:0/3681869029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481815e660 con 0x7f48180cd390 2023-07-06T03:54:52.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.008+0000 7f481dfff700 1 -- 172.21.15.97:0/3681869029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481815dd30 con 0x7f48180cd390 2023-07-06T03:54:52.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.009+0000 7f481dfff700 1 -- 172.21.15.97:0/3681869029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481804e340 con 0x7f48180cd390 2023-07-06T03:54:52.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.009+0000 7f47fffff700 1 -- 172.21.15.97:0/3681869029 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4808003b90 con 0x7f48180cd390 2023-07-06T03:54:52.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.010+0000 7f47fffff700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f480406ace0 0x7f480406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.010+0000 7f47fffff700 1 -- 172.21.15.97:0/3681869029 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4808088a80 con 0x7f48180cd390 2023-07-06T03:54:52.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.011+0000 7f4816ffd700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f480406ace0 0x7f480406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.011+0000 7f4816ffd700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f480406ace0 0x7f480406d190 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f480c014040 tx=0x7f480c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:52.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.015+0000 7f47fffff700 1 -- 172.21.15.97:0/3681869029 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4808053a90 con 0x7f48180cd390 2023-07-06T03:54:52.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.211+0000 7f481dfff700 1 -- 172.21.15.97:0/3681869029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48180b4f40 con 0x7f48180cd390 2023-07-06T03:54:52.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.213+0000 7f47fffff700 1 -- 172.21.15.97:0/3681869029 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18391 (secure 0 0 0) 0x7f4808017090 con 0x7f48180cd390 2023-07-06T03:54:52.215 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:52.215 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":112,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:54:26.969421+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:54:23.626811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-06T03:54:24.421787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:52.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 -- 172.21.15.97:0/3681869029 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f480406ace0 msgr2=0x7f480406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f480406ace0 0x7f480406d190 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f480c014040 tx=0x7f480c008030 comp rx=0 tx=0).stop 2023-07-06T03:54:52.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 -- 172.21.15.97:0/3681869029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48180cd390 msgr2=0x7f481815b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48180cd390 0x7f481815b780 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f480800c750 tx=0x7f480800cb10 comp rx=0 tx=0).stop 2023-07-06T03:54:52.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 -- 172.21.15.97:0/3681869029 shutdown_connections 2023-07-06T03:54:52.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f480406ace0 0x7f480406d190 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4818160dd0 0x7f48181f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48180cd390 0x7f481815b780 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 --2- 172.21.15.97:0/3681869029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f481815c520 0x7f48180c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 -- 172.21.15.97:0/3681869029 >> 172.21.15.97:0/3681869029 conn(0x7f4818071790 msgr2=0x7f48180b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:52.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 -- 172.21.15.97:0/3681869029 shutdown_connections 2023-07-06T03:54:52.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.215+0000 7f481dfff700 1 -- 172.21.15.97:0/3681869029 wait complete. 2023-07-06T03:54:52.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:54:23.626811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:54:24.421787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:54:52.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:54:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.431+0000 7fbdc57ff700 1 Processor -- start 2023-07-06T03:54:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.431+0000 7fbdc57ff700 1 -- start start 2023-07-06T03:54:52.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.432+0000 7fbdc57ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdc0159070 0x7fbdc00c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.432+0000 7fbdc57ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbdc00c3cd0 0x7fbdc00cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.432+0000 7fbdc57ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc00ca260 con 0x7fbdc0159440 2023-07-06T03:54:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.432+0000 7fbdc57ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc00c99c0 con 0x7fbdc00c3cd0 2023-07-06T03:54:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.432+0000 7fbdc57ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00c9ca0 con 0x7fbdc0159070 2023-07-06T03:54:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.432+0000 7fbdbf7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbdc0159440 0x7fbdc00b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44364/0 (socket says 172.21.15.97:44364) 2023-07-06T03:54:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.433+0000 7fbdbf7fe700 1 -- 172.21.15.97:0/936946918 learned_addr learned my addr 172.21.15.97:0/936946918 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.433+0000 7fbdbeffd700 1 --2- 172.21.15.97:0/936946918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdc0159070 0x7fbdc00c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.433+0000 7fbdbe7fc700 1 --2- 172.21.15.97:0/936946918 >> v2:172.21.15.146:3300/0 conn(0x7fbdc00c3cd0 0x7fbdc00cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.433+0000 7fbdbdffb700 1 -- 172.21.15.97:0/936946918 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460022117 0 0) 0x7fbdc00ca260 con 0x7fbdc0159440 2023-07-06T03:54:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.433+0000 7fbdbdffb700 1 -- 172.21.15.97:0/936946918 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbda80035b0 con 0x7fbdc0159440 2023-07-06T03:54:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.433+0000 7fbdbeffd700 1 -- 172.21.15.97:0/936946918 >> v2:172.21.15.146:3300/0 conn(0x7fbdc00c3cd0 msgr2=0x7fbdc00cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.433+0000 7fbdbeffd700 1 --2- 172.21.15.97:0/936946918 >> v2:172.21.15.146:3300/0 conn(0x7fbdc00c3cd0 0x7fbdc00cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.433+0000 7fbdbeffd700 1 -- 172.21.15.97:0/936946918 >> v1:172.21.15.97:6789/0 conn(0x7fbdc0159440 legacy=0x7fbdc00b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.433+0000 7fbdbeffd700 1 -- 172.21.15.97:0/936946918 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc00c9f80 con 0x7fbdc0159070 2023-07-06T03:54:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.433+0000 7fbdbeffd700 1 --2- 172.21.15.97:0/936946918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdc0159070 0x7fbdc00c8f40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009840 tx=0x7fbdb0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d08d4a6795edded4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.434+0000 7fbdbdffb700 1 -- 172.21.15.97:0/936946918 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0011830 con 0x7fbdc0159070 2023-07-06T03:54:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.434+0000 7fbdbdffb700 1 -- 172.21.15.97:0/936946918 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb0011990 con 0x7fbdc0159070 2023-07-06T03:54:52.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.434+0000 7fbdbdffb700 1 -- 172.21.15.97:0/936946918 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0011d20 con 0x7fbdc0159070 2023-07-06T03:54:52.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.434+0000 7fbdc57ff700 1 -- 172.21.15.97:0/936946918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdc0159070 msgr2=0x7fbdc00c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.434+0000 7fbdc57ff700 1 --2- 172.21.15.97:0/936946918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdc0159070 0x7fbdc00c8f40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009840 tx=0x7fbdb0009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:52.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.434+0000 7fbdc57ff700 1 -- 172.21.15.97:0/936946918 shutdown_connections 2023-07-06T03:54:52.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.434+0000 7fbdc57ff700 1 --2- 172.21.15.97:0/936946918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdc0159070 0x7fbdc00c8f40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.434+0000 7fbdc57ff700 1 --2- 172.21.15.97:0/936946918 >> v2:172.21.15.146:3300/0 conn(0x7fbdc00c3cd0 0x7fbdc00cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.434+0000 7fbdc57ff700 1 -- 172.21.15.97:0/936946918 >> 172.21.15.97:0/936946918 conn(0x7fbdc006a060 msgr2=0x7fbdc006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:52.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.434+0000 7fbdc57ff700 1 -- 172.21.15.97:0/936946918 shutdown_connections 2023-07-06T03:54:52.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.434+0000 7fbdc57ff700 1 -- 172.21.15.97:0/936946918 wait complete. 2023-07-06T03:54:52.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.436+0000 7fbdc57ff700 1 Processor -- start 2023-07-06T03:54:52.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.436+0000 7fbdc57ff700 1 -- start start 2023-07-06T03:54:52.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.437+0000 7fbdc57ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdc00c3cd0 0x7fbdc014e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.437+0000 7fbdc57ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc0159070 0x7fbdc00b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.437+0000 7fbdc57ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdc0159440 0x7fbdc00ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.437+0000 7fbdc57ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00aef90 con 0x7fbdc0159070 2023-07-06T03:54:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.437+0000 7fbdc57ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00af270 con 0x7fbdc00c3cd0 2023-07-06T03:54:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.437+0000 7fbdc57ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00af550 con 0x7fbdc0159440 2023-07-06T03:54:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.437+0000 7fbdbe7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc0159070 0x7fbdc00b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.437+0000 7fbdbe7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc0159070 0x7fbdc00b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57614/0 (socket says 172.21.15.97:57614) 2023-07-06T03:54:52.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.437+0000 7fbdbe7fc700 1 -- 172.21.15.97:0/2837688003 learned_addr learned my addr 172.21.15.97:0/2837688003 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.437+0000 7fbdbeffd700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdc00c3cd0 0x7fbdc014e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbdbf7fe700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdc0159440 0x7fbdc00ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbdbe7fc700 1 -- 172.21.15.97:0/2837688003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdc0159440 msgr2=0x7fbdc00ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbdbe7fc700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdc0159440 0x7fbdc00ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbdbe7fc700 1 -- 172.21.15.97:0/2837688003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdc00c3cd0 msgr2=0x7fbdc014e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbdbe7fc700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdc00c3cd0 0x7fbdc014e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbdbe7fc700 1 -- 172.21.15.97:0/2837688003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb00094a0 con 0x7fbdc0159070 2023-07-06T03:54:52.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbdbe7fc700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc0159070 0x7fbdc00b0030 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400e9b0 tx=0x7fbdb400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbda77fe700 1 -- 172.21.15.97:0/2837688003 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4009710 con 0x7fbdc0159070 2023-07-06T03:54:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbda77fe700 1 -- 172.21.15.97:0/2837688003 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb400c850 con 0x7fbdc0159070 2023-07-06T03:54:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbdc57ff700 1 -- 172.21.15.97:0/2837688003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc00af830 con 0x7fbdc0159070 2023-07-06T03:54:52.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbda77fe700 1 -- 172.21.15.97:0/2837688003 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb400cbc0 con 0x7fbdc0159070 2023-07-06T03:54:52.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.438+0000 7fbdc57ff700 1 -- 172.21.15.97:0/2837688003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc014e9e0 con 0x7fbdc0159070 2023-07-06T03:54:52.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.439+0000 7fbdc57ff700 1 -- 172.21.15.97:0/2837688003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc004e340 con 0x7fbdc0159070 2023-07-06T03:54:52.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.441+0000 7fbda77fe700 1 -- 172.21.15.97:0/2837688003 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbdb400cd20 con 0x7fbdc0159070 2023-07-06T03:54:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.441+0000 7fbda77fe700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbdac06ace0 0x7fbdac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.442+0000 7fbda77fe700 1 -- 172.21.15.97:0/2837688003 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbdb4089520 con 0x7fbdc0159070 2023-07-06T03:54:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.442+0000 7fbdbeffd700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbdac06ace0 0x7fbdac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.442+0000 7fbdbeffd700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbdac06ace0 0x7fbdac06d190 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0000c00 tx=0x7fbdb0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:52.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.446+0000 7fbda77fe700 1 -- 172.21.15.97:0/2837688003 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbdb40545c0 con 0x7fbdc0159070 2023-07-06T03:54:52.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.639+0000 7fbdc57ff700 1 -- 172.21.15.97:0/2837688003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbdc006d220 con 0x7fbdc0159070 2023-07-06T03:54:52.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.640+0000 7fbda77fe700 1 -- 172.21.15.97:0/2837688003 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbdb4057be0 con 0x7fbdc0159070 2023-07-06T03:54:52.642 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:54:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 -- 172.21.15.97:0/2837688003 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbdac06ace0 msgr2=0x7fbdac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbdac06ace0 0x7fbdac06d190 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0000c00 tx=0x7fbdb0008030 comp rx=0 tx=0).stop 2023-07-06T03:54:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 -- 172.21.15.97:0/2837688003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc0159070 msgr2=0x7fbdc00b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc0159070 0x7fbdc00b0030 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400e9b0 tx=0x7fbdb400ecc0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 -- 172.21.15.97:0/2837688003 shutdown_connections 2023-07-06T03:54:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbdac06ace0 0x7fbdac06d190 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbdc0159440 0x7fbdc00ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbdc00c3cd0 0x7fbdc014e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 --2- 172.21.15.97:0/2837688003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc0159070 0x7fbdc00b0030 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 -- 172.21.15.97:0/2837688003 >> 172.21.15.97:0/2837688003 conn(0x7fbdc006a060 msgr2=0x7fbdc00cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 -- 172.21.15.97:0/2837688003 shutdown_connections 2023-07-06T03:54:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.642+0000 7fbdc57ff700 1 -- 172.21.15.97:0/2837688003 wait complete. 2023-07-06T03:54:52.662 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:54:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.858+0000 7f959f3ff700 1 Processor -- start 2023-07-06T03:54:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.858+0000 7f959f3ff700 1 -- start start 2023-07-06T03:54:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959f3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f959815c610 0x7f95980b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959f3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f959806b360 0x7f95980b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959f3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95980cff90 con 0x7f95980d2210 2023-07-06T03:54:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959f3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95980d0780 con 0x7f959806b360 2023-07-06T03:54:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959f3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959815c9e0 con 0x7f959815c610 2023-07-06T03:54:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959c99a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f959806b360 0x7f95980b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959d99c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f95980d2210 0x7f95980d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44378/0 (socket says 172.21.15.97:44378) 2023-07-06T03:54:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959d99c700 1 -- 172.21.15.97:0/1380917762 learned_addr learned my addr 172.21.15.97:0/1380917762 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959d19b700 1 --2- 172.21.15.97:0/1380917762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f959815c610 0x7f95980b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959c99a700 1 -- 172.21.15.97:0/1380917762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f959815c610 msgr2=0x7f95980b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959c99a700 1 --2- 172.21.15.97:0/1380917762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f959815c610 0x7f95980b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.859+0000 7f959c99a700 1 -- 172.21.15.97:0/1380917762 >> v1:172.21.15.97:6789/0 conn(0x7f95980d2210 legacy=0x7f95980d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.860+0000 7f959c99a700 1 -- 172.21.15.97:0/1380917762 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95980b5650 con 0x7f959806b360 2023-07-06T03:54:52.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.860+0000 7f959c99a700 1 --2- 172.21.15.97:0/1380917762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f959806b360 0x7f95980b4cf0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f9588009870 tx=0x7f9588009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ab0a4467b4cfea server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:52.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.860+0000 7f958ffff700 1 -- 172.21.15.97:0/1380917762 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588011830 con 0x7f959806b360 2023-07-06T03:54:52.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.860+0000 7f958ffff700 1 -- 172.21.15.97:0/1380917762 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9588011990 con 0x7f959806b360 2023-07-06T03:54:52.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.860+0000 7f958ffff700 1 -- 172.21.15.97:0/1380917762 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588011d20 con 0x7f959806b360 2023-07-06T03:54:52.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.860+0000 7f959f3ff700 1 -- 172.21.15.97:0/1380917762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f959806b360 msgr2=0x7f95980b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.860+0000 7f959f3ff700 1 --2- 172.21.15.97:0/1380917762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f959806b360 0x7f95980b4cf0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f9588009870 tx=0x7f9588009b80 comp rx=0 tx=0).stop 2023-07-06T03:54:52.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.861+0000 7f959f3ff700 1 -- 172.21.15.97:0/1380917762 shutdown_connections 2023-07-06T03:54:52.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.861+0000 7f959f3ff700 1 --2- 172.21.15.97:0/1380917762 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f959815c610 0x7f95980b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.861+0000 7f959f3ff700 1 --2- 172.21.15.97:0/1380917762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f959806b360 0x7f95980b4cf0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.861+0000 7f959f3ff700 1 -- 172.21.15.97:0/1380917762 >> 172.21.15.97:0/1380917762 conn(0x7f959806a6c0 msgr2=0x7f959806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:52.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.861+0000 7f959f3ff700 1 -- 172.21.15.97:0/1380917762 shutdown_connections 2023-07-06T03:54:52.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.861+0000 7f959f3ff700 1 -- 172.21.15.97:0/1380917762 wait complete. 2023-07-06T03:54:52.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.862+0000 7f959f3ff700 1 Processor -- start 2023-07-06T03:54:52.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.863+0000 7f959f3ff700 1 -- start start 2023-07-06T03:54:52.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959f3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f959806b360 0x7f959815f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959f3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95980d2210 0x7f9598155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959f3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f959815c610 0x7f95981f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959f3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95980cbb20 con 0x7f959806b360 2023-07-06T03:54:52.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959f3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95980caea0 con 0x7f95980d2210 2023-07-06T03:54:52.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959f3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95980cb180 con 0x7f959815c610 2023-07-06T03:54:52.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959d19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f959806b360 0x7f959815f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959d19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f959806b360 0x7f959815f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57638/0 (socket says 172.21.15.97:57638) 2023-07-06T03:54:52.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959d19b700 1 -- 172.21.15.97:0/1985995017 learned_addr learned my addr 172.21.15.97:0/1985995017 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:52.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959c99a700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95980d2210 0x7f9598155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959d99c700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f959815c610 0x7f95981f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959d19b700 1 -- 172.21.15.97:0/1985995017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f959815c610 msgr2=0x7f95981f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959d19b700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f959815c610 0x7f95981f1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959d19b700 1 -- 172.21.15.97:0/1985995017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95980d2210 msgr2=0x7f9598155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:52.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959d19b700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95980d2210 0x7f9598155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:52.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959d19b700 1 -- 172.21.15.97:0/1985995017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880094a0 con 0x7f959806b360 2023-07-06T03:54:52.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.864+0000 7f959d19b700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f959806b360 0x7f959815f400 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f959400e810 tx=0x7f959400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:52.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.865+0000 7f958dffb700 1 -- 172.21.15.97:0/1985995017 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9594013610 con 0x7f959806b360 2023-07-06T03:54:52.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.865+0000 7f958dffb700 1 -- 172.21.15.97:0/1985995017 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9594013770 con 0x7f959806b360 2023-07-06T03:54:52.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.865+0000 7f958dffb700 1 -- 172.21.15.97:0/1985995017 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9594013b20 con 0x7f959806b360 2023-07-06T03:54:52.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.865+0000 7f959f3ff700 1 -- 172.21.15.97:0/1985995017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95980cb460 con 0x7f959806b360 2023-07-06T03:54:52.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.865+0000 7f959f3ff700 1 -- 172.21.15.97:0/1985995017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959815f940 con 0x7f959806b360 2023-07-06T03:54:52.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.866+0000 7f959f3ff700 1 -- 172.21.15.97:0/1985995017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95980d0180 con 0x7f959806b360 2023-07-06T03:54:52.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.867+0000 7f958dffb700 1 -- 172.21.15.97:0/1985995017 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9594013c80 con 0x7f959806b360 2023-07-06T03:54:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.868+0000 7f958dffb700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 0x7f957c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.868+0000 7f958dffb700 1 -- 172.21.15.97:0/1985995017 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f95940891c0 con 0x7f959806b360 2023-07-06T03:54:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.868+0000 7f959c99a700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 0x7f957c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.869+0000 7f959c99a700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 0x7f957c06d190 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f959806b790 tx=0x7f958801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:52.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:52.872+0000 7f958dffb700 1 -- 172.21.15.97:0/1985995017 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9594057c70 con 0x7f959806b360 2023-07-06T03:54:53.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.071+0000 7f959f3ff700 1 -- 172.21.15.97:0/1985995017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f959804e340 con 0x7f959806b360 2023-07-06T03:54:53.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.071+0000 7f958dffb700 1 -- 172.21.15.97:0/1985995017 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v112) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9594057800 con 0x7f959806b360 2023-07-06T03:54:53.074 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:54:53.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.073+0000 7f959f3ff700 1 -- 172.21.15.97:0/1985995017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 msgr2=0x7f957c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.073+0000 7f959f3ff700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 0x7f957c06d190 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f959806b790 tx=0x7f958801c040 comp rx=0 tx=0).stop 2023-07-06T03:54:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.073+0000 7f959f3ff700 1 -- 172.21.15.97:0/1985995017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f959806b360 msgr2=0x7f959815f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.073+0000 7f959f3ff700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f959806b360 0x7f959815f400 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f959400e810 tx=0x7f959400eb20 comp rx=0 tx=0).stop 2023-07-06T03:54:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.074+0000 7f959f3ff700 1 -- 172.21.15.97:0/1985995017 shutdown_connections 2023-07-06T03:54:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.074+0000 7f959f3ff700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 0x7f957c06d190 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.074+0000 7f959f3ff700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f959815c610 0x7f95981f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.074+0000 7f959f3ff700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f959806b360 0x7f959815f400 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.074+0000 7f959f3ff700 1 --2- 172.21.15.97:0/1985995017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95980d2210 0x7f9598155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.074+0000 7f959f3ff700 1 -- 172.21.15.97:0/1985995017 >> 172.21.15.97:0/1985995017 conn(0x7f959806a6c0 msgr2=0x7f95980d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:53.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.074+0000 7f959f3ff700 1 -- 172.21.15.97:0/1985995017 shutdown_connections 2023-07-06T03:54:53.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.074+0000 7f959f3ff700 1 -- 172.21.15.97:0/1985995017 wait complete. 2023-07-06T03:54:53.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:54:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.288+0000 7f4a403ff700 1 Processor -- start 2023-07-06T03:54:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.289+0000 7f4a403ff700 1 -- start start 2023-07-06T03:54:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.290+0000 7f4a403ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4a380c3cd0 0x7f4a380cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.290+0000 7f4a403ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4a38159440 0x7f4a380cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.290+0000 7f4a403ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a380ca260 con 0x7f4a38159070 2023-07-06T03:54:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.290+0000 7f4a403ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380c99c0 con 0x7f4a380c3cd0 2023-07-06T03:54:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.290+0000 7f4a403ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380c9ca0 con 0x7f4a38159440 2023-07-06T03:54:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.290+0000 7f4a3d99a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4a380c3cd0 0x7f4a380cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.290+0000 7f4a3e19b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4a38159070 0x7f4a380c8f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44394/0 (socket says 172.21.15.97:44394) 2023-07-06T03:54:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.290+0000 7f4a3e19b700 1 -- 172.21.15.97:0/403939502 learned_addr learned my addr 172.21.15.97:0/403939502 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.291+0000 7f4a3e99c700 1 --2- 172.21.15.97:0/403939502 >> v2:172.21.15.146:3300/0 conn(0x7f4a38159440 0x7f4a380cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.291+0000 7f4a3d99a700 1 -- 172.21.15.97:0/403939502 >> v2:172.21.15.146:3300/0 conn(0x7f4a38159440 msgr2=0x7f4a380cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.291+0000 7f4a3d99a700 1 --2- 172.21.15.97:0/403939502 >> v2:172.21.15.146:3300/0 conn(0x7f4a38159440 0x7f4a380cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.291+0000 7f4a3d99a700 1 -- 172.21.15.97:0/403939502 >> v1:172.21.15.97:6789/0 conn(0x7f4a38159070 legacy=0x7f4a380c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.291+0000 7f4a3d99a700 1 -- 172.21.15.97:0/403939502 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380c9f80 con 0x7f4a380c3cd0 2023-07-06T03:54:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.291+0000 7f4a3d99a700 1 --2- 172.21.15.97:0/403939502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a380c3cd0 0x7f4a380cc4c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4a38071f70 tx=0x7f4a28009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960bedc27a1fa72d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.291+0000 7f4a3d199700 1 -- 172.21.15.97:0/403939502 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28011830 con 0x7f4a380c3cd0 2023-07-06T03:54:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.291+0000 7f4a3d199700 1 -- 172.21.15.97:0/403939502 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28011990 con 0x7f4a380c3cd0 2023-07-06T03:54:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.291+0000 7f4a3d199700 1 -- 172.21.15.97:0/403939502 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28011d20 con 0x7f4a380c3cd0 2023-07-06T03:54:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.292+0000 7f4a403ff700 1 -- 172.21.15.97:0/403939502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a380c3cd0 msgr2=0x7f4a380cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.292+0000 7f4a403ff700 1 --2- 172.21.15.97:0/403939502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a380c3cd0 0x7f4a380cc4c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4a38071f70 tx=0x7f4a28009b20 comp rx=0 tx=0).stop 2023-07-06T03:54:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.292+0000 7f4a403ff700 1 -- 172.21.15.97:0/403939502 shutdown_connections 2023-07-06T03:54:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.292+0000 7f4a403ff700 1 --2- 172.21.15.97:0/403939502 >> v2:172.21.15.146:3300/0 conn(0x7f4a38159440 0x7f4a380cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.292+0000 7f4a403ff700 1 --2- 172.21.15.97:0/403939502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a380c3cd0 0x7f4a380cc4c0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.292+0000 7f4a403ff700 1 -- 172.21.15.97:0/403939502 >> 172.21.15.97:0/403939502 conn(0x7f4a3806a060 msgr2=0x7f4a3806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.292+0000 7f4a403ff700 1 -- 172.21.15.97:0/403939502 shutdown_connections 2023-07-06T03:54:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.292+0000 7f4a403ff700 1 -- 172.21.15.97:0/403939502 wait complete. 2023-07-06T03:54:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.294+0000 7f4a403ff700 1 Processor -- start 2023-07-06T03:54:53.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.294+0000 7f4a403ff700 1 -- start start 2023-07-06T03:54:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a403ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a380c3cd0 0x7f4a38159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a403ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a38159070 0x7f4a380b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a403ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a38159440 0x7f4a380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a403ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380b4760 con 0x7f4a38159440 2023-07-06T03:54:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a403ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3815b220 con 0x7f4a380c3cd0 2023-07-06T03:54:53.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a403ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380b2540 con 0x7f4a38159070 2023-07-06T03:54:53.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a3e99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a38159440 0x7f4a380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:53.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a3e99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a38159440 0x7f4a380b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57658/0 (socket says 172.21.15.97:57658) 2023-07-06T03:54:53.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a3e99c700 1 -- 172.21.15.97:0/1366257698 learned_addr learned my addr 172.21.15.97:0/1366257698 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:53.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a3d99a700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a38159070 0x7f4a380b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:53.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a3e19b700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a380c3cd0 0x7f4a38159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:53.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a3e99c700 1 -- 172.21.15.97:0/1366257698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a38159070 msgr2=0x7f4a380b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a3e99c700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a38159070 0x7f4a380b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a3e99c700 1 -- 172.21.15.97:0/1366257698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a380c3cd0 msgr2=0x7f4a38159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a3e99c700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a380c3cd0 0x7f4a38159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.296+0000 7f4a3e99c700 1 -- 172.21.15.97:0/1366257698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a280094a0 con 0x7f4a38159440 2023-07-06T03:54:53.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.297+0000 7f4a3e99c700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a38159440 0x7f4a380b2000 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f4a30007e60 tx=0x7f4a3000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:53.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.297+0000 7f4a2effd700 1 -- 172.21.15.97:0/1366257698 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000ed30 con 0x7f4a38159440 2023-07-06T03:54:53.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.297+0000 7f4a2effd700 1 -- 172.21.15.97:0/1366257698 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a30014420 con 0x7f4a38159440 2023-07-06T03:54:53.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.297+0000 7f4a403ff700 1 -- 172.21.15.97:0/1366257698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3815a610 con 0x7f4a38159440 2023-07-06T03:54:53.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.297+0000 7f4a2effd700 1 -- 172.21.15.97:0/1366257698 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a300147d0 con 0x7f4a38159440 2023-07-06T03:54:53.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.297+0000 7f4a403ff700 1 -- 172.21.15.97:0/1366257698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3815abd0 con 0x7f4a38159440 2023-07-06T03:54:53.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.298+0000 7f4a403ff700 1 -- 172.21.15.97:0/1366257698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3815b3d0 con 0x7f4a38159440 2023-07-06T03:54:53.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.299+0000 7f4a2effd700 1 -- 172.21.15.97:0/1366257698 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a30014930 con 0x7f4a38159440 2023-07-06T03:54:53.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.300+0000 7f4a2effd700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a1c06ace0 0x7f4a1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:53.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.300+0000 7f4a2effd700 1 -- 172.21.15.97:0/1366257698 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a30089b50 con 0x7f4a38159440 2023-07-06T03:54:53.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.300+0000 7f4a3e19b700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a1c06ace0 0x7f4a1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:53.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.301+0000 7f4a3e19b700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a1c06ace0 0x7f4a1c06d190 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4a38152c60 tx=0x7f4a34007f10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:53.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.304+0000 7f4a2effd700 1 -- 172.21.15.97:0/1366257698 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4a3815b3d0 con 0x7f4a38159440 2023-07-06T03:54:53.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.499+0000 7f4a403ff700 1 -- 172.21.15.97:0/1366257698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a380ca860 con 0x7f4a38159440 2023-07-06T03:54:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.499+0000 7f4a2effd700 1 -- 172.21.15.97:0/1366257698 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a30054c10 con 0x7f4a38159440 2023-07-06T03:54:53.502 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:54:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 -- 172.21.15.97:0/1366257698 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a1c06ace0 msgr2=0x7f4a1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a1c06ace0 0x7f4a1c06d190 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4a38152c60 tx=0x7f4a34007f10 comp rx=0 tx=0).stop 2023-07-06T03:54:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 -- 172.21.15.97:0/1366257698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a38159440 msgr2=0x7f4a380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a38159440 0x7f4a380b2000 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f4a30007e60 tx=0x7f4a3000e3c0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 -- 172.21.15.97:0/1366257698 shutdown_connections 2023-07-06T03:54:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4a1c06ace0 0x7f4a1c06d190 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a38159440 0x7f4a380b2000 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a380c3cd0 0x7f4a38159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 --2- 172.21.15.97:0/1366257698 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4a38159070 0x7f4a380b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 -- 172.21.15.97:0/1366257698 >> 172.21.15.97:0/1366257698 conn(0x7f4a3806a060 msgr2=0x7f4a3806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 -- 172.21.15.97:0/1366257698 shutdown_connections 2023-07-06T03:54:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.502+0000 7f4a403ff700 1 -- 172.21.15.97:0/1366257698 wait complete. 2023-07-06T03:54:53.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:54:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.716+0000 7fd2651ff700 1 Processor -- start 2023-07-06T03:54:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.717+0000 7fd2651ff700 1 -- start start 2023-07-06T03:54:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.718+0000 7fd2651ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd26015c610 0x7fd2600b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.718+0000 7fd2651ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2600cff90 con 0x7fd2600d2210 2023-07-06T03:54:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.718+0000 7fd2651ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2600b5650 con 0x7fd26006b360 2023-07-06T03:54:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.718+0000 7fd2651ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2600d0780 con 0x7fd26015c610 2023-07-06T03:54:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.718+0000 7fd25ed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd26015c610 0x7fd2600b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.718+0000 7fd25ed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd26015c610 0x7fd2600b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57664/0 (socket says 172.21.15.97:57664) 2023-07-06T03:54:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.718+0000 7fd25f59e700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd2600d2210 0x7fd2600d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44406/0 (socket says 172.21.15.97:44406) 2023-07-06T03:54:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.718+0000 7fd25ed9d700 1 -- 172.21.15.97:0/2976583163 learned_addr learned my addr 172.21.15.97:0/2976583163 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.718+0000 7fd25ed9d700 1 -- 172.21.15.97:0/2976583163 >> v1:172.21.15.146:6789/0 conn(0x7fd26006b360 legacy=0x7fd2600b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.718+0000 7fd25ed9d700 1 -- 172.21.15.97:0/2976583163 >> v1:172.21.15.97:6789/0 conn(0x7fd2600d2210 legacy=0x7fd2600d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.719+0000 7fd25ed9d700 1 -- 172.21.15.97:0/2976583163 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2600b5400 con 0x7fd26015c610 2023-07-06T03:54:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.718+0000 7fd25e59c700 1 -- 172.21.15.97:0/2976583163 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910662587 0 0) 0x7fd2600cff90 con 0x7fd2600d2210 2023-07-06T03:54:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.719+0000 7fd25ed9d700 1 --2- 172.21.15.97:0/2976583163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26015c610 0x7fd2600b47b0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd2500098f0 tx=0x7fd250009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3e98344ffba0d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.719+0000 7fd25e59c700 1 -- 172.21.15.97:0/2976583163 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250004230 con 0x7fd26015c610 2023-07-06T03:54:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.719+0000 7fd25e59c700 1 -- 172.21.15.97:0/2976583163 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250004390 con 0x7fd26015c610 2023-07-06T03:54:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.719+0000 7fd25e59c700 1 -- 172.21.15.97:0/2976583163 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500036a0 con 0x7fd26015c610 2023-07-06T03:54:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.719+0000 7fd2651ff700 1 -- 172.21.15.97:0/2976583163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26015c610 msgr2=0x7fd2600b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.719+0000 7fd2651ff700 1 --2- 172.21.15.97:0/2976583163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26015c610 0x7fd2600b47b0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd2500098f0 tx=0x7fd250009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.720+0000 7fd2651ff700 1 -- 172.21.15.97:0/2976583163 shutdown_connections 2023-07-06T03:54:53.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.720+0000 7fd2651ff700 1 --2- 172.21.15.97:0/2976583163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26015c610 0x7fd2600b47b0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.720+0000 7fd2651ff700 1 -- 172.21.15.97:0/2976583163 >> 172.21.15.97:0/2976583163 conn(0x7fd26006a6c0 msgr2=0x7fd26006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:53.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.720+0000 7fd2651ff700 1 -- 172.21.15.97:0/2976583163 shutdown_connections 2023-07-06T03:54:53.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.720+0000 7fd2651ff700 1 -- 172.21.15.97:0/2976583163 wait complete. 2023-07-06T03:54:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.722+0000 7fd2651ff700 1 Processor -- start 2023-07-06T03:54:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.722+0000 7fd2651ff700 1 -- start start 2023-07-06T03:54:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.723+0000 7fd2651ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd26006b360 0x7fd260157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.723+0000 7fd2651ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2600d2210 0x7fd26015e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.723+0000 7fd2651ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26015c610 0x7fd26015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.723+0000 7fd2651ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26015d610 con 0x7fd26015c610 2023-07-06T03:54:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.723+0000 7fd2651ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600cb110 con 0x7fd26006b360 2023-07-06T03:54:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.723+0000 7fd2651ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26015d8b0 con 0x7fd2600d2210 2023-07-06T03:54:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.724+0000 7fd25f59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26015c610 0x7fd26015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.724+0000 7fd25f59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26015c610 0x7fd26015f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57666/0 (socket says 172.21.15.97:57666) 2023-07-06T03:54:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.724+0000 7fd25f59e700 1 -- 172.21.15.97:0/1780781387 learned_addr learned my addr 172.21.15.97:0/1780781387 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.724+0000 7fd257fff700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2600d2210 0x7fd26015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.724+0000 7fd25ed9d700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd26006b360 0x7fd260157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.724+0000 7fd25f59e700 1 -- 172.21.15.97:0/1780781387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2600d2210 msgr2=0x7fd26015e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.724+0000 7fd25f59e700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2600d2210 0x7fd26015e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.724+0000 7fd25f59e700 1 -- 172.21.15.97:0/1780781387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd26006b360 msgr2=0x7fd260157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.724+0000 7fd25f59e700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd26006b360 0x7fd260157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.724+0000 7fd25f59e700 1 -- 172.21.15.97:0/1780781387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250009550 con 0x7fd26015c610 2023-07-06T03:54:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.724+0000 7fd25f59e700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26015c610 0x7fd26015f2a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fd25800cb60 tx=0x7fd25800ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.725+0000 7fd2577fe700 1 -- 172.21.15.97:0/1780781387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25800a9e0 con 0x7fd26015c610 2023-07-06T03:54:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.725+0000 7fd2577fe700 1 -- 172.21.15.97:0/1780781387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25800ab40 con 0x7fd26015c610 2023-07-06T03:54:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.725+0000 7fd2651ff700 1 -- 172.21.15.97:0/1780781387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26015db90 con 0x7fd26015c610 2023-07-06T03:54:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.725+0000 7fd2577fe700 1 -- 172.21.15.97:0/1780781387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2580070d0 con 0x7fd26015c610 2023-07-06T03:54:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.725+0000 7fd2651ff700 1 -- 172.21.15.97:0/1780781387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26015f930 con 0x7fd26015c610 2023-07-06T03:54:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.726+0000 7fd2651ff700 1 -- 172.21.15.97:0/1780781387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26015e0b0 con 0x7fd26015c610 2023-07-06T03:54:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.727+0000 7fd2577fe700 1 -- 172.21.15.97:0/1780781387 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd258007230 con 0x7fd26015c610 2023-07-06T03:54:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.728+0000 7fd2577fe700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd24006ace0 0x7fd24006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.728+0000 7fd2577fe700 1 -- 172.21.15.97:0/1780781387 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd258088f90 con 0x7fd26015c610 2023-07-06T03:54:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.728+0000 7fd25ed9d700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd24006ace0 0x7fd24006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.729+0000 7fd25ed9d700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd24006ace0 0x7fd24006d190 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd250009130 tx=0x7fd250008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.733+0000 7fd2577fe700 1 -- 172.21.15.97:0/1780781387 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd258053fa0 con 0x7fd26015c610 2023-07-06T03:54:53.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.935+0000 7fd2651ff700 1 -- 172.21.15.97:0/1780781387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd2600cdfe0 con 0x7fd26015c610 2023-07-06T03:54:53.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.935+0000 7fd2577fe700 1 -- 172.21.15.97:0/1780781387 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd2580575c0 con 0x7fd26015c610 2023-07-06T03:54:53.938 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:54:53.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.937+0000 7fd2651ff700 1 -- 172.21.15.97:0/1780781387 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd24006ace0 msgr2=0x7fd24006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.937+0000 7fd2651ff700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd24006ace0 0x7fd24006d190 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd250009130 tx=0x7fd250008030 comp rx=0 tx=0).stop 2023-07-06T03:54:53.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.937+0000 7fd2651ff700 1 -- 172.21.15.97:0/1780781387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26015c610 msgr2=0x7fd26015f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:53.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.937+0000 7fd2651ff700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26015c610 0x7fd26015f2a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fd25800cb60 tx=0x7fd25800ce70 comp rx=0 tx=0).stop 2023-07-06T03:54:53.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.938+0000 7fd2651ff700 1 -- 172.21.15.97:0/1780781387 shutdown_connections 2023-07-06T03:54:53.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.938+0000 7fd2651ff700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd24006ace0 0x7fd24006d190 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.938+0000 7fd2651ff700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26015c610 0x7fd26015f2a0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.938+0000 7fd2651ff700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd26006b360 0x7fd260157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.938+0000 7fd2651ff700 1 --2- 172.21.15.97:0/1780781387 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd2600d2210 0x7fd26015e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:53.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.938+0000 7fd2651ff700 1 -- 172.21.15.97:0/1780781387 >> 172.21.15.97:0/1780781387 conn(0x7fd26006a6c0 msgr2=0x7fd2600b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:53.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.938+0000 7fd2651ff700 1 -- 172.21.15.97:0/1780781387 shutdown_connections 2023-07-06T03:54:53.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:53.938+0000 7fd2651ff700 1 -- 172.21.15.97:0/1780781387 wait complete. 2023-07-06T03:54:53.959 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:54:54.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.150+0000 7fcb853ff700 1 Processor -- start 2023-07-06T03:54:54.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.151+0000 7fcb853ff700 1 -- start start 2023-07-06T03:54:54.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.152+0000 7fcb853ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcb800719c0 0x7fcb800cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:54.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.152+0000 7fcb853ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8015da10 con 0x7fcb8015c590 2023-07-06T03:54:54.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.152+0000 7fcb853ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb800ccc30 con 0x7fcb8015c1c0 2023-07-06T03:54:54.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.152+0000 7fcb853ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb800ccf10 con 0x7fcb800719c0 2023-07-06T03:54:54.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.152+0000 7fcb77fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcb8015c590 0x7fcb8015d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44418/0 (socket says 172.21.15.97:44418) 2023-07-06T03:54:54.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.152+0000 7fcb77fff700 1 -- 172.21.15.97:0/1352359135 learned_addr learned my addr 172.21.15.97:0/1352359135 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:54.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.152+0000 7fcb7f7fe700 1 --2- 172.21.15.97:0/1352359135 >> v2:172.21.15.97:3300/0 conn(0x7fcb800719c0 0x7fcb800cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:54.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.153+0000 7fcb7e7fc700 1 -- 172.21.15.97:0/1352359135 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697140964 0 0) 0x7fcb68001110 con 0x7fcb8015c590 2023-07-06T03:54:54.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.153+0000 7fcb7e7fc700 1 -- 172.21.15.97:0/1352359135 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb6c0035b0 con 0x7fcb8015c590 2023-07-06T03:54:54.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.153+0000 7fcb7f7fe700 1 -- 172.21.15.97:0/1352359135 >> v1:172.21.15.146:6789/0 conn(0x7fcb8015c1c0 legacy=0x7fcb800cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:54.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.153+0000 7fcb7f7fe700 1 -- 172.21.15.97:0/1352359135 >> v1:172.21.15.97:6789/0 conn(0x7fcb8015c590 legacy=0x7fcb8015d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:54.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.153+0000 7fcb7f7fe700 1 -- 172.21.15.97:0/1352359135 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb800cd1f0 con 0x7fcb800719c0 2023-07-06T03:54:54.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.153+0000 7fcb7f7fe700 1 --2- 172.21.15.97:0/1352359135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb800719c0 0x7fcb800cb080 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009d90 tx=0x7fcb7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47723a379ad08a6d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.153+0000 7fcb7e7fc700 1 -- 172.21.15.97:0/1352359135 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7800e070 con 0x7fcb800719c0 2023-07-06T03:54:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.153+0000 7fcb7e7fc700 1 -- 172.21.15.97:0/1352359135 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7800bb20 con 0x7fcb800719c0 2023-07-06T03:54:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.153+0000 7fcb7e7fc700 1 -- 172.21.15.97:0/1352359135 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78012430 con 0x7fcb800719c0 2023-07-06T03:54:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.154+0000 7fcb853ff700 1 -- 172.21.15.97:0/1352359135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb800719c0 msgr2=0x7fcb800cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.154+0000 7fcb853ff700 1 --2- 172.21.15.97:0/1352359135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb800719c0 0x7fcb800cb080 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009d90 tx=0x7fcb7800b330 comp rx=0 tx=0).stop 2023-07-06T03:54:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.154+0000 7fcb853ff700 1 -- 172.21.15.97:0/1352359135 shutdown_connections 2023-07-06T03:54:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.154+0000 7fcb853ff700 1 --2- 172.21.15.97:0/1352359135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb800719c0 0x7fcb800cb080 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:54.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.154+0000 7fcb853ff700 1 -- 172.21.15.97:0/1352359135 >> 172.21.15.97:0/1352359135 conn(0x7fcb800b1650 msgr2=0x7fcb800b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.154+0000 7fcb853ff700 1 -- 172.21.15.97:0/1352359135 shutdown_connections 2023-07-06T03:54:54.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.154+0000 7fcb853ff700 1 -- 172.21.15.97:0/1352359135 wait complete. 2023-07-06T03:54:54.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.156+0000 7fcb853ff700 1 Processor -- start 2023-07-06T03:54:54.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.156+0000 7fcb853ff700 1 -- start start 2023-07-06T03:54:54.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.157+0000 7fcb853ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb800719c0 0x7fcb801505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.157+0000 7fcb853ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb8015c1c0 0x7fcb80160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.157+0000 7fcb853ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb8015c590 0x7fcb800add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.157+0000 7fcb853ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800b3a20 con 0x7fcb8015c1c0 2023-07-06T03:54:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.157+0000 7fcb853ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb80150b00 con 0x7fcb800719c0 2023-07-06T03:54:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.157+0000 7fcb853ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800b3e40 con 0x7fcb8015c590 2023-07-06T03:54:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.157+0000 7fcb77fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb8015c1c0 0x7fcb80160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.157+0000 7fcb77fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb8015c1c0 0x7fcb80160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57690/0 (socket says 172.21.15.97:57690) 2023-07-06T03:54:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.157+0000 7fcb77fff700 1 -- 172.21.15.97:0/1269144814 learned_addr learned my addr 172.21.15.97:0/1269144814 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.157+0000 7fcb7effd700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb800719c0 0x7fcb801505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.157+0000 7fcb7f7fe700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb8015c590 0x7fcb800add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.158+0000 7fcb77fff700 1 -- 172.21.15.97:0/1269144814 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb8015c590 msgr2=0x7fcb800add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.158+0000 7fcb77fff700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb8015c590 0x7fcb800add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.158+0000 7fcb77fff700 1 -- 172.21.15.97:0/1269144814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb800719c0 msgr2=0x7fcb801505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.158+0000 7fcb77fff700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb800719c0 0x7fcb801505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.158+0000 7fcb77fff700 1 -- 172.21.15.97:0/1269144814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb78009a70 con 0x7fcb8015c1c0 2023-07-06T03:54:54.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.158+0000 7fcb77fff700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb8015c1c0 0x7fcb80160270 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800c500 tx=0x7fcb6800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:54.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.158+0000 7fcb777fe700 1 -- 172.21.15.97:0/1269144814 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68005d70 con 0x7fcb8015c1c0 2023-07-06T03:54:54.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.158+0000 7fcb777fe700 1 -- 172.21.15.97:0/1269144814 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb680070d0 con 0x7fcb8015c1c0 2023-07-06T03:54:54.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.158+0000 7fcb853ff700 1 -- 172.21.15.97:0/1269144814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb800b2da0 con 0x7fcb8015c1c0 2023-07-06T03:54:54.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.158+0000 7fcb777fe700 1 -- 172.21.15.97:0/1269144814 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68007460 con 0x7fcb8015c1c0 2023-07-06T03:54:54.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.158+0000 7fcb853ff700 1 -- 172.21.15.97:0/1269144814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb800b3360 con 0x7fcb8015c1c0 2023-07-06T03:54:54.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.159+0000 7fcb853ff700 1 -- 172.21.15.97:0/1269144814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb800b3b60 con 0x7fcb8015c1c0 2023-07-06T03:54:54.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.160+0000 7fcb777fe700 1 -- 172.21.15.97:0/1269144814 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb680075c0 con 0x7fcb8015c1c0 2023-07-06T03:54:54.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.162+0000 7fcb777fe700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb6006ace0 0x7fcb6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:54.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.162+0000 7fcb777fe700 1 -- 172.21.15.97:0/1269144814 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb680887a0 con 0x7fcb8015c1c0 2023-07-06T03:54:54.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.162+0000 7fcb7effd700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb6006ace0 0x7fcb6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:54.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.162+0000 7fcb7effd700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb6006ace0 0x7fcb6006d190 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcb70007fd0 tx=0x7fcb70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:54.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.166+0000 7fcb777fe700 1 -- 172.21.15.97:0/1269144814 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcb800b3b60 con 0x7fcb8015c1c0 2023-07-06T03:54:54.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.375+0000 7fcb853ff700 1 -- 172.21.15.97:0/1269144814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcb8015da10 con 0x7fcb8015c1c0 2023-07-06T03:54:54.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.375+0000 7fcb777fe700 1 -- 172.21.15.97:0/1269144814 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcb8015da10 con 0x7fcb8015c1c0 2023-07-06T03:54:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 -- 172.21.15.97:0/1269144814 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb6006ace0 msgr2=0x7fcb6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb6006ace0 0x7fcb6006d190 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcb70007fd0 tx=0x7fcb70009040 comp rx=0 tx=0).stop 2023-07-06T03:54:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 -- 172.21.15.97:0/1269144814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb8015c1c0 msgr2=0x7fcb80160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb8015c1c0 0x7fcb80160270 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800c500 tx=0x7fcb6800c810 comp rx=0 tx=0).stop 2023-07-06T03:54:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 -- 172.21.15.97:0/1269144814 shutdown_connections 2023-07-06T03:54:54.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb6006ace0 0x7fcb6006d190 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:54.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb8015c590 0x7fcb800add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:54.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb800719c0 0x7fcb801505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:54.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 --2- 172.21.15.97:0/1269144814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb8015c1c0 0x7fcb80160270 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:54.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 -- 172.21.15.97:0/1269144814 >> 172.21.15.97:0/1269144814 conn(0x7fcb800b1650 msgr2=0x7fcb8015b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:54.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 -- 172.21.15.97:0/1269144814 shutdown_connections 2023-07-06T03:54:54.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:54.378+0000 7fcb853ff700 1 -- 172.21.15.97:0/1269144814 wait complete. 2023-07-06T03:54:54.383 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:54:54.398 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2023-07-06T03:54:54.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:54.399 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-06T03:54:54.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[125971]: fuse finished with error 0 and tester_r 0 2023-07-06T03:54:54.476 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:54:54.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:54.476 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T03:54:54.638 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:54:54.638 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:54.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:54:54.685 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:54:54.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:54.686 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T03:54:54.769 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T03:54:54.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:54:54.770 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:54:54.771 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:54:54.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:54:54.820 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:54:54.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:54:54.821 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T03:54:54.904 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T03:54:54.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:54:54.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:55.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.098+0000 7f4ad51ff700 1 Processor -- start 2023-07-06T03:54:55.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.098+0000 7f4ad51ff700 1 -- start start 2023-07-06T03:54:55.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.099+0000 7f4ad51ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015c520 0x7f4ad00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:55.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4ad51ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad00cdae0 con 0x7f4ad0160180 2023-07-06T03:54:55.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4ad51ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad00b6640 con 0x7f4ad0160d00 2023-07-06T03:54:55.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4ad51ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00cd8d0 con 0x7f4ad015c520 2023-07-06T03:54:55.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4ace59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4ad0160180 0x7f4ad01605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44422/0 (socket says 172.21.15.97:44422) 2023-07-06T03:54:55.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4ace59c700 1 -- 172.21.15.97:0/141037187 learned_addr learned my addr 172.21.15.97:0/141037187 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:55.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4aced9d700 1 --2- 172.21.15.97:0/141037187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015c520 0x7f4ad00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:55.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4acdd9b700 1 -- 172.21.15.97:0/141037187 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063096348 0 0) 0x7f4ad00cdae0 con 0x7f4ad0160180 2023-07-06T03:54:55.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4acdd9b700 1 -- 172.21.15.97:0/141037187 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ab80035b0 con 0x7f4ad0160180 2023-07-06T03:54:55.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4acdd9b700 1 -- 172.21.15.97:0/141037187 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874122427 0 0) 0x7f4ad00b6640 con 0x7f4ad0160d00 2023-07-06T03:54:55.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4aced9d700 1 -- 172.21.15.97:0/141037187 >> v1:172.21.15.146:6789/0 conn(0x7f4ad0160d00 legacy=0x7f4ad00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4aced9d700 1 -- 172.21.15.97:0/141037187 >> v1:172.21.15.97:6789/0 conn(0x7f4ad0160180 legacy=0x7f4ad01605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.100+0000 7f4aced9d700 1 -- 172.21.15.97:0/141037187 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad00b6920 con 0x7f4ad015c520 2023-07-06T03:54:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.101+0000 7f4aced9d700 1 --2- 172.21.15.97:0/141037187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015c520 0x7f4ad00cd390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009870 tx=0x7f4ac0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62a84d98245ce361 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:55.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.101+0000 7f4acdd9b700 1 -- 172.21.15.97:0/141037187 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0011830 con 0x7f4ad015c520 2023-07-06T03:54:55.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.101+0000 7f4acdd9b700 1 -- 172.21.15.97:0/141037187 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac0011990 con 0x7f4ad015c520 2023-07-06T03:54:55.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.101+0000 7f4acdd9b700 1 -- 172.21.15.97:0/141037187 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0011d20 con 0x7f4ad015c520 2023-07-06T03:54:55.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.101+0000 7f4ad51ff700 1 -- 172.21.15.97:0/141037187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015c520 msgr2=0x7f4ad00cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.101+0000 7f4ad51ff700 1 --2- 172.21.15.97:0/141037187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015c520 0x7f4ad00cd390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009870 tx=0x7f4ac0009b80 comp rx=0 tx=0).stop 2023-07-06T03:54:55.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.102+0000 7f4ad51ff700 1 -- 172.21.15.97:0/141037187 shutdown_connections 2023-07-06T03:54:55.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.102+0000 7f4ad51ff700 1 --2- 172.21.15.97:0/141037187 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015c520 0x7f4ad00cd390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.102+0000 7f4ad51ff700 1 -- 172.21.15.97:0/141037187 >> 172.21.15.97:0/141037187 conn(0x7f4ad0071790 msgr2=0x7f4ad0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:55.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.102+0000 7f4ad51ff700 1 -- 172.21.15.97:0/141037187 shutdown_connections 2023-07-06T03:54:55.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.102+0000 7f4ad51ff700 1 -- 172.21.15.97:0/141037187 wait complete. 2023-07-06T03:54:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.103+0000 7f4ad51ff700 1 Processor -- start 2023-07-06T03:54:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.103+0000 7f4ad51ff700 1 -- start start 2023-07-06T03:54:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.104+0000 7f4ad51ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015c520 0x7f4ad00cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.104+0000 7f4ad51ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad0160180 0x7f4ad01f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.104+0000 7f4ad51ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad0160d00 0x7f4ad00c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.104+0000 7f4ad51ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad01f5450 con 0x7f4ad0160180 2023-07-06T03:54:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.104+0000 7f4ad51ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad015dcb0 con 0x7f4ad0160d00 2023-07-06T03:54:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.104+0000 7f4ad51ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00ccb50 con 0x7f4ad015c520 2023-07-06T03:54:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4ace59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad0160180 0x7f4ad01f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4ace59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad0160180 0x7f4ad01f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57692/0 (socket says 172.21.15.97:57692) 2023-07-06T03:54:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4ace59c700 1 -- 172.21.15.97:0/3218787975 learned_addr learned my addr 172.21.15.97:0/3218787975 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4acf59e700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad0160d00 0x7f4ad00c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4aced9d700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015c520 0x7f4ad00cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4ace59c700 1 -- 172.21.15.97:0/3218787975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015c520 msgr2=0x7f4ad00cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4ace59c700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015c520 0x7f4ad00cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4ace59c700 1 -- 172.21.15.97:0/3218787975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad0160d00 msgr2=0x7f4ad00c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4ace59c700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad0160d00 0x7f4ad00c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4ace59c700 1 -- 172.21.15.97:0/3218787975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac00094a0 con 0x7f4ad0160180 2023-07-06T03:54:55.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4ace59c700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad0160180 0x7f4ad01f4f10 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400c4d0 tx=0x7f4ac400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:55.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.105+0000 7f4ab77fe700 1 -- 172.21.15.97:0/3218787975 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4005d70 con 0x7f4ad0160180 2023-07-06T03:54:55.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.106+0000 7f4ab77fe700 1 -- 172.21.15.97:0/3218787975 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac40070d0 con 0x7f4ad0160180 2023-07-06T03:54:55.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.106+0000 7f4ab77fe700 1 -- 172.21.15.97:0/3218787975 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4007440 con 0x7f4ad0160180 2023-07-06T03:54:55.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.106+0000 7f4ad51ff700 1 -- 172.21.15.97:0/3218787975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad015df20 con 0x7f4ad0160180 2023-07-06T03:54:55.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.106+0000 7f4ad51ff700 1 -- 172.21.15.97:0/3218787975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad00ca510 con 0x7f4ad0160180 2023-07-06T03:54:55.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.107+0000 7f4ad51ff700 1 -- 172.21.15.97:0/3218787975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad015e440 con 0x7f4ad0160180 2023-07-06T03:54:55.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.108+0000 7f4ab77fe700 1 -- 172.21.15.97:0/3218787975 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ac40075a0 con 0x7f4ad0160180 2023-07-06T03:54:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.109+0000 7f4ab77fe700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4abc06ace0 0x7f4abc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.109+0000 7f4ab77fe700 1 -- 172.21.15.97:0/3218787975 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ac40887f0 con 0x7f4ad0160180 2023-07-06T03:54:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.109+0000 7f4aced9d700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4abc06ace0 0x7f4abc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.110+0000 7f4aced9d700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4abc06ace0 0x7f4abc06d190 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00090b0 tx=0x7f4ac001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:55.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.113+0000 7f4ab77fe700 1 -- 172.21.15.97:0/3218787975 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4ac40538d0 con 0x7f4ad0160180 2023-07-06T03:54:55.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.347+0000 7f4ad51ff700 1 -- 172.21.15.97:0/3218787975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ad004e340 con 0x7f4ad0160180 2023-07-06T03:54:55.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.348+0000 7f4ab77fe700 1 -- 172.21.15.97:0/3218787975 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4677 (secure 0 0 0) 0x7f4ac4056ef0 con 0x7f4ad0160180 2023-07-06T03:54:55.351 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:55.351 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":83,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2796855133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2796855133},{"type":"v1","addr":"172.21.15.97:6835","nonce":2796855133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6851,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1092966074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1092966074},{"type":"v1","addr":"172.21.15.146:6835","nonce":1092966074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8577,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2594138359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2594138359},{"type":"v1","addr":"172.21.15.97:6837","nonce":2594138359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:54:25.441909+0000","modified":"2023-07-06T03:54:54.793074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6834/582690410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":582690410},{"type":"v1","addr":"172.21.15.146:6834","nonce":582690410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-06T03:54:55.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 -- 172.21.15.97:0/3218787975 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4abc06ace0 msgr2=0x7f4abc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4abc06ace0 0x7f4abc06d190 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00090b0 tx=0x7f4ac001c040 comp rx=0 tx=0).stop 2023-07-06T03:54:55.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 -- 172.21.15.97:0/3218787975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad0160180 msgr2=0x7f4ad01f4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad0160180 0x7f4ad01f4f10 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400c4d0 tx=0x7f4ac400c7e0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 -- 172.21.15.97:0/3218787975 shutdown_connections 2023-07-06T03:54:55.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4abc06ace0 0x7f4abc06d190 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ad0160d00 0x7f4ad00c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4ad015c520 0x7f4ad00cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 --2- 172.21.15.97:0/3218787975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad0160180 0x7f4ad01f4f10 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 -- 172.21.15.97:0/3218787975 >> 172.21.15.97:0/3218787975 conn(0x7f4ad0071790 msgr2=0x7f4ad01605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 -- 172.21.15.97:0/3218787975 shutdown_connections 2023-07-06T03:54:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.350+0000 7f4ad51ff700 1 -- 172.21.15.97:0/3218787975 wait complete. 2023-07-06T03:54:55.355 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 83 2023-07-06T03:54:55.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:55.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.563+0000 7f7be9dff700 1 Processor -- start 2023-07-06T03:54:55.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.563+0000 7f7be9dff700 1 -- start start 2023-07-06T03:54:55.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.564+0000 7f7be9dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7be415c520 0x7f7be40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.564+0000 7f7be9dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be40cdae0 con 0x7f7be4160180 2023-07-06T03:54:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.564+0000 7f7be9dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be40b6640 con 0x7f7be4160d00 2023-07-06T03:54:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.564+0000 7f7be9dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be40cd8d0 con 0x7f7be415c520 2023-07-06T03:54:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.564+0000 7f7bdbfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7be4160180 0x7f7be41605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44430/0 (socket says 172.21.15.97:44430) 2023-07-06T03:54:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.564+0000 7f7bdbfff700 1 -- 172.21.15.97:0/243219102 learned_addr learned my addr 172.21.15.97:0/243219102 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:55.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.564+0000 7f7be37fe700 1 --2- 172.21.15.97:0/243219102 >> v2:172.21.15.146:3300/0 conn(0x7f7be415c520 0x7f7be40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:55.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.564+0000 7f7be2ffd700 1 -- 172.21.15.97:0/243219102 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652965208 0 0) 0x7f7be40cdae0 con 0x7f7be4160180 2023-07-06T03:54:55.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.565+0000 7f7be2ffd700 1 -- 172.21.15.97:0/243219102 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bd00035b0 con 0x7f7be4160180 2023-07-06T03:54:55.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.565+0000 7f7be2ffd700 1 -- 172.21.15.97:0/243219102 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261028138 0 0) 0x7f7be40b6640 con 0x7f7be4160d00 2023-07-06T03:54:55.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.565+0000 7f7be2ffd700 1 -- 172.21.15.97:0/243219102 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7be40cdae0 con 0x7f7be4160d00 2023-07-06T03:54:55.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.565+0000 7f7be2ffd700 1 -- 172.21.15.97:0/243219102 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 174023947 0 0) 0x7f7bd00035b0 con 0x7f7be4160180 2023-07-06T03:54:55.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.565+0000 7f7be37fe700 1 -- 172.21.15.97:0/243219102 >> v1:172.21.15.146:6789/0 conn(0x7f7be4160d00 legacy=0x7f7be40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.565+0000 7f7be37fe700 1 -- 172.21.15.97:0/243219102 >> v1:172.21.15.97:6789/0 conn(0x7f7be4160180 legacy=0x7f7be41605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.565+0000 7f7be37fe700 1 -- 172.21.15.97:0/243219102 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40b6920 con 0x7f7be415c520 2023-07-06T03:54:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.565+0000 7f7be37fe700 1 --2- 172.21.15.97:0/243219102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7be415c520 0x7f7be40cd390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009840 tx=0x7f7bd4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6029a8d69d723c63 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.565+0000 7f7be2ffd700 1 -- 172.21.15.97:0/243219102 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc001670 con 0x7f7be415c520 2023-07-06T03:54:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.565+0000 7f7be2ffd700 1 -- 172.21.15.97:0/243219102 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd4004030 con 0x7f7be415c520 2023-07-06T03:54:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.565+0000 7f7be2ffd700 1 -- 172.21.15.97:0/243219102 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd40043c0 con 0x7f7be415c520 2023-07-06T03:54:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.566+0000 7f7be9dff700 1 -- 172.21.15.97:0/243219102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7be415c520 msgr2=0x7f7be40cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.566+0000 7f7be9dff700 1 --2- 172.21.15.97:0/243219102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7be415c520 0x7f7be40cd390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009840 tx=0x7f7bd4009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.566+0000 7f7be9dff700 1 -- 172.21.15.97:0/243219102 shutdown_connections 2023-07-06T03:54:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.566+0000 7f7be9dff700 1 --2- 172.21.15.97:0/243219102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7be415c520 0x7f7be40cd390 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.566+0000 7f7be9dff700 1 -- 172.21.15.97:0/243219102 >> 172.21.15.97:0/243219102 conn(0x7f7be4071790 msgr2=0x7f7be4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.566+0000 7f7be9dff700 1 -- 172.21.15.97:0/243219102 shutdown_connections 2023-07-06T03:54:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.566+0000 7f7be9dff700 1 -- 172.21.15.97:0/243219102 wait complete. 2023-07-06T03:54:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.568+0000 7f7be9dff700 1 Processor -- start 2023-07-06T03:54:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.568+0000 7f7be9dff700 1 -- start start 2023-07-06T03:54:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.569+0000 7f7be9dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be415c520 0x7f7be414d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.569+0000 7f7be9dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7be4160180 0x7f7be414dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.569+0000 7f7be9dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7be4160d00 0x7f7be40d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:55.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.569+0000 7f7be9dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40caf70 con 0x7f7be415c520 2023-07-06T03:54:55.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.569+0000 7f7be9dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40cb250 con 0x7f7be4160d00 2023-07-06T03:54:55.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.569+0000 7f7be9dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40cb530 con 0x7f7be4160180 2023-07-06T03:54:55.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.569+0000 7f7be37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be415c520 0x7f7be414d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:55.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.569+0000 7f7be37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be415c520 0x7f7be414d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57708/0 (socket says 172.21.15.97:57708) 2023-07-06T03:54:55.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.569+0000 7f7be37fe700 1 -- 172.21.15.97:0/327856427 learned_addr learned my addr 172.21.15.97:0/327856427 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:55.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.569+0000 7f7bdbfff700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7be4160180 0x7f7be414dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.569+0000 7f7be3fff700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7be4160d00 0x7f7be40d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be37fe700 1 -- 172.21.15.97:0/327856427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7be4160180 msgr2=0x7f7be414dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be37fe700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7be4160180 0x7f7be414dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be37fe700 1 -- 172.21.15.97:0/327856427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7be4160d00 msgr2=0x7f7be40d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be37fe700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7be4160d00 0x7f7be40d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be37fe700 1 -- 172.21.15.97:0/327856427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd40094a0 con 0x7f7be415c520 2023-07-06T03:54:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be3fff700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7be4160d00 0x7f7be40d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be37fe700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be415c520 0x7f7be414d050 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400a010 tx=0x7f7bd4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be0ff9700 1 -- 172.21.15.97:0/327856427 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4003cc0 con 0x7f7be415c520 2023-07-06T03:54:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be0ff9700 1 -- 172.21.15.97:0/327856427 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd401f070 con 0x7f7be415c520 2023-07-06T03:54:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be9dff700 1 -- 172.21.15.97:0/327856427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be40cb810 con 0x7f7be415c520 2023-07-06T03:54:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be0ff9700 1 -- 172.21.15.97:0/327856427 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4011870 con 0x7f7be415c520 2023-07-06T03:54:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.570+0000 7f7be9dff700 1 -- 172.21.15.97:0/327856427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be40d1180 con 0x7f7be415c520 2023-07-06T03:54:55.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.571+0000 7f7be9dff700 1 -- 172.21.15.97:0/327856427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be40cbd30 con 0x7f7be415c520 2023-07-06T03:54:55.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.572+0000 7f7be0ff9700 1 -- 172.21.15.97:0/327856427 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7bd4021030 con 0x7f7be415c520 2023-07-06T03:54:55.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.573+0000 7f7be0ff9700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7bc406ace0 0x7f7bc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:55.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.573+0000 7f7be0ff9700 1 -- 172.21.15.97:0/327856427 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7bd408a6c0 con 0x7f7be415c520 2023-07-06T03:54:55.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.573+0000 7f7bdbfff700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7bc406ace0 0x7f7bc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:55.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.574+0000 7f7bdbfff700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7bc406ace0 0x7f7bc406d190 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7be40b1a40 tx=0x7f7bcc000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:55.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.577+0000 7f7be0ff9700 1 -- 172.21.15.97:0/327856427 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7be40cbd30 con 0x7f7be415c520 2023-07-06T03:54:55.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.772+0000 7f7be9dff700 1 -- 172.21.15.97:0/327856427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7be40cbd30 con 0x7f7be415c520 2023-07-06T03:54:55.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.774+0000 7f7be0ff9700 1 -- 172.21.15.97:0/327856427 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18391 (secure 0 0 0) 0x7f7be40cbd30 con 0x7f7be415c520 2023-07-06T03:54:55.800 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:55.800 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":112,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:54:26.969421+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:54:23.626811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-06T03:54:24.421787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:55.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 -- 172.21.15.97:0/327856427 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7bc406ace0 msgr2=0x7f7bc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7bc406ace0 0x7f7bc406d190 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7be40b1a40 tx=0x7f7bcc000ed0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 -- 172.21.15.97:0/327856427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be415c520 msgr2=0x7f7be414d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:55.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be415c520 0x7f7be414d050 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400a010 tx=0x7f7bd4004750 comp rx=0 tx=0).stop 2023-07-06T03:54:55.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 -- 172.21.15.97:0/327856427 shutdown_connections 2023-07-06T03:54:55.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7bc406ace0 0x7f7bc406d190 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7be4160d00 0x7f7be40d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7be415c520 0x7f7be414d050 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 --2- 172.21.15.97:0/327856427 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7be4160180 0x7f7be414dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:55.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 -- 172.21.15.97:0/327856427 >> 172.21.15.97:0/327856427 conn(0x7f7be4071790 msgr2=0x7f7be4071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:55.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 -- 172.21.15.97:0/327856427 shutdown_connections 2023-07-06T03:54:55.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:55.776+0000 7f7be9dff700 1 -- 172.21.15.97:0/327856427 wait complete. 2023-07-06T03:54:55.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:54:23.626811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:54:24.421787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:54:55.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:54:56.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.004+0000 7fcf77dff700 1 Processor -- start 2023-07-06T03:54:56.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.004+0000 7fcf77dff700 1 -- start start 2023-07-06T03:54:56.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.005+0000 7fcf77dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 0x7fcf700cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.005+0000 7fcf77dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7015b490 con 0x7fcf7015b050 2023-07-06T03:54:56.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.005+0000 7fcf77dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7015b6e0 con 0x7fcf7015eb90 2023-07-06T03:54:56.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.005+0000 7fcf77dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7015e0b0 con 0x7fcf70154890 2023-07-06T03:54:56.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.006+0000 7fcf7639c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcf7015b050 0x7fcf700c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44444/0 (socket says 172.21.15.97:44444) 2023-07-06T03:54:56.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.006+0000 7fcf7639c700 1 -- 172.21.15.97:0/3916249574 learned_addr learned my addr 172.21.15.97:0/3916249574 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:56.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.006+0000 7fcf75b9b700 1 --2- 172.21.15.97:0/3916249574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 0x7fcf700cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.006+0000 7fcf74b99700 1 -- 172.21.15.97:0/3916249574 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040680979 0 0) 0x7fcf7015b490 con 0x7fcf7015b050 2023-07-06T03:54:56.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.006+0000 7fcf74b99700 1 -- 172.21.15.97:0/3916249574 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf5c0035b0 con 0x7fcf7015b050 2023-07-06T03:54:56.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.006+0000 7fcf74b99700 1 -- 172.21.15.97:0/3916249574 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700285627 0 0) 0x7fcf7015b6e0 con 0x7fcf7015eb90 2023-07-06T03:54:56.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.006+0000 7fcf74b99700 1 -- 172.21.15.97:0/3916249574 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf7015b490 con 0x7fcf7015eb90 2023-07-06T03:54:56.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.006+0000 7fcf75b9b700 1 -- 172.21.15.97:0/3916249574 >> v1:172.21.15.146:6789/0 conn(0x7fcf7015eb90 legacy=0x7fcf7015a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.006+0000 7fcf75b9b700 1 -- 172.21.15.97:0/3916249574 >> v1:172.21.15.97:6789/0 conn(0x7fcf7015b050 legacy=0x7fcf700c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.006+0000 7fcf75b9b700 1 -- 172.21.15.97:0/3916249574 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7015e390 con 0x7fcf70154890 2023-07-06T03:54:56.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.007+0000 7fcf75b9b700 1 --2- 172.21.15.97:0/3916249574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 0x7fcf700cb390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009870 tx=0x7fcf6c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80c7f11f0a30df9a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:56.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.007+0000 7fcf74b99700 1 -- 172.21.15.97:0/3916249574 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c011830 con 0x7fcf70154890 2023-07-06T03:54:56.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.007+0000 7fcf74b99700 1 -- 172.21.15.97:0/3916249574 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c011990 con 0x7fcf70154890 2023-07-06T03:54:56.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.007+0000 7fcf74b99700 1 -- 172.21.15.97:0/3916249574 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c011d20 con 0x7fcf70154890 2023-07-06T03:54:56.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.007+0000 7fcf77dff700 1 -- 172.21.15.97:0/3916249574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 msgr2=0x7fcf700cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.007+0000 7fcf77dff700 1 --2- 172.21.15.97:0/3916249574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 0x7fcf700cb390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009870 tx=0x7fcf6c009b80 comp rx=0 tx=0).stop 2023-07-06T03:54:56.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.007+0000 7fcf77dff700 1 -- 172.21.15.97:0/3916249574 shutdown_connections 2023-07-06T03:54:56.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.007+0000 7fcf77dff700 1 --2- 172.21.15.97:0/3916249574 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 0x7fcf700cb390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.007+0000 7fcf77dff700 1 -- 172.21.15.97:0/3916249574 >> 172.21.15.97:0/3916249574 conn(0x7fcf7006a060 msgr2=0x7fcf7006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:56.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.008+0000 7fcf77dff700 1 -- 172.21.15.97:0/3916249574 shutdown_connections 2023-07-06T03:54:56.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.008+0000 7fcf77dff700 1 -- 172.21.15.97:0/3916249574 wait complete. 2023-07-06T03:54:56.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.009+0000 7fcf77dff700 1 Processor -- start 2023-07-06T03:54:56.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.009+0000 7fcf77dff700 1 -- start start 2023-07-06T03:54:56.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.010+0000 7fcf77dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 0x7fcf701578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.010+0000 7fcf77dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf7015b050 0x7fcf700ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.010+0000 7fcf77dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7015eb90 0x7fcf700cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.010+0000 7fcf77dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf701ec670 con 0x7fcf7015eb90 2023-07-06T03:54:56.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.010+0000 7fcf77dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf70157e00 con 0x7fcf7015b050 2023-07-06T03:54:56.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.010+0000 7fcf77dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7014c960 con 0x7fcf70154890 2023-07-06T03:54:56.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf7639c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7015eb90 0x7fcf700cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf7639c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7015eb90 0x7fcf700cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57724/0 (socket says 172.21.15.97:57724) 2023-07-06T03:54:56.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf7639c700 1 -- 172.21.15.97:0/3489230774 learned_addr learned my addr 172.21.15.97:0/3489230774 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf7539a700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf7015b050 0x7fcf700ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf75b9b700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 0x7fcf701578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf7639c700 1 -- 172.21.15.97:0/3489230774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 msgr2=0x7fcf701578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf7639c700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 0x7fcf701578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf7639c700 1 -- 172.21.15.97:0/3489230774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf7015b050 msgr2=0x7fcf700ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf7639c700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf7015b050 0x7fcf700ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf7639c700 1 -- 172.21.15.97:0/3489230774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c0094a0 con 0x7fcf7015eb90 2023-07-06T03:54:56.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf75b9b700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 0x7fcf701578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.011+0000 7fcf7639c700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7015eb90 0x7fcf700cab60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800cba0 tx=0x7fcf6800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.012+0000 7fcf667fc700 1 -- 172.21.15.97:0/3489230774 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800ab20 con 0x7fcf7015eb90 2023-07-06T03:54:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.012+0000 7fcf667fc700 1 -- 172.21.15.97:0/3489230774 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6800ac80 con 0x7fcf7015eb90 2023-07-06T03:54:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.012+0000 7fcf667fc700 1 -- 172.21.15.97:0/3489230774 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68012400 con 0x7fcf7015eb90 2023-07-06T03:54:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.012+0000 7fcf77dff700 1 -- 172.21.15.97:0/3489230774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7014cc40 con 0x7fcf7015eb90 2023-07-06T03:54:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.012+0000 7fcf77dff700 1 -- 172.21.15.97:0/3489230774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7014d200 con 0x7fcf7015eb90 2023-07-06T03:54:56.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.013+0000 7fcf77dff700 1 -- 172.21.15.97:0/3489230774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7004e340 con 0x7fcf7015eb90 2023-07-06T03:54:56.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.014+0000 7fcf667fc700 1 -- 172.21.15.97:0/3489230774 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcf6801b3b0 con 0x7fcf7015eb90 2023-07-06T03:54:56.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.015+0000 7fcf667fc700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcf5406ace0 0x7fcf5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.015+0000 7fcf667fc700 1 -- 172.21.15.97:0/3489230774 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf68089330 con 0x7fcf7015eb90 2023-07-06T03:54:56.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.015+0000 7fcf75b9b700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcf5406ace0 0x7fcf5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.015+0000 7fcf75b9b700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcf5406ace0 0x7fcf5406d190 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000c00 tx=0x7fcf6c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:56.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.019+0000 7fcf667fc700 1 -- 172.21.15.97:0/3489230774 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcf68057de0 con 0x7fcf7015eb90 2023-07-06T03:54:56.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.214+0000 7fcf77dff700 1 -- 172.21.15.97:0/3489230774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf70071da0 con 0x7fcf7015eb90 2023-07-06T03:54:56.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.214+0000 7fcf667fc700 1 -- 172.21.15.97:0/3489230774 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf68017020 con 0x7fcf7015eb90 2023-07-06T03:54:56.218 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:54:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 -- 172.21.15.97:0/3489230774 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcf5406ace0 msgr2=0x7fcf5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcf5406ace0 0x7fcf5406d190 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000c00 tx=0x7fcf6c01c040 comp rx=0 tx=0).stop 2023-07-06T03:54:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 -- 172.21.15.97:0/3489230774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7015eb90 msgr2=0x7fcf700cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7015eb90 0x7fcf700cab60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800cba0 tx=0x7fcf6800cf60 comp rx=0 tx=0).stop 2023-07-06T03:54:56.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 -- 172.21.15.97:0/3489230774 shutdown_connections 2023-07-06T03:54:56.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcf5406ace0 0x7fcf5406d190 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7015eb90 0x7fcf700cab60 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcf70154890 0x7fcf701578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 --2- 172.21.15.97:0/3489230774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf7015b050 0x7fcf700ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 -- 172.21.15.97:0/3489230774 >> 172.21.15.97:0/3489230774 conn(0x7fcf7006a060 msgr2=0x7fcf70153680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:56.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 -- 172.21.15.97:0/3489230774 shutdown_connections 2023-07-06T03:54:56.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.217+0000 7fcf77dff700 1 -- 172.21.15.97:0/3489230774 wait complete. 2023-07-06T03:54:56.238 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:54:56.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.433+0000 7fdc24fff700 1 Processor -- start 2023-07-06T03:54:56.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.433+0000 7fdc24fff700 1 -- start start 2023-07-06T03:54:56.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.434+0000 7fdc24fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdc2015c610 0x7fdc200b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.434+0000 7fdc24fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc2006b360 0x7fdc200b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.434+0000 7fdc24fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc200cff90 con 0x7fdc200d2210 2023-07-06T03:54:56.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.434+0000 7fdc24fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc200d0780 con 0x7fdc2015c610 2023-07-06T03:54:56.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.434+0000 7fdc24fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2015c9e0 con 0x7fdc2006b360 2023-07-06T03:54:56.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.434+0000 7fdc1ed9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdc200d2210 0x7fdc200d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44450/0 (socket says 172.21.15.97:44450) 2023-07-06T03:54:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.434+0000 7fdc1ed9d700 1 -- 172.21.15.97:0/649432988 learned_addr learned my addr 172.21.15.97:0/649432988 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1dd9b700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc2006b360 0x7fdc200b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1e59c700 1 --2- 172.21.15.97:0/649432988 >> v2:172.21.15.146:3300/0 conn(0x7fdc2015c610 0x7fdc200b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656371683 0 0) 0x7fdc200cff90 con 0x7fdc200d2210 2023-07-06T03:54:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc080035b0 con 0x7fdc200d2210 2023-07-06T03:54:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 219165658 0 0) 0x7fdc080035b0 con 0x7fdc200d2210 2023-07-06T03:54:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc2006b360 msgr2=0x7fdc200b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc2006b360 0x7fdc200b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 >> v2:172.21.15.146:3300/0 conn(0x7fdc2015c610 msgr2=0x7fdc200b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 --2- 172.21.15.97:0/649432988 >> v2:172.21.15.146:3300/0 conn(0x7fdc2015c610 0x7fdc200b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc200b5650 con 0x7fdc200d2210 2023-07-06T03:54:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fdc180019f0 con 0x7fdc200d2210 2023-07-06T03:54:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 >> v1:172.21.15.97:6789/0 conn(0x7fdc200d2210 legacy=0x7fdc200d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 0x7fdc08003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc2006b360 0x7fdc08004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc08004d90 0x7fdc08005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200d0780 con 0x7fdc08003a30 2023-07-06T03:54:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2015c9e0 con 0x7fdc08004d90 2023-07-06T03:54:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0800ba10 con 0x7fdc2006b360 2023-07-06T03:54:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1e59c700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 0x7fdc08003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.435+0000 7fdc1dd9b700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc2006b360 0x7fdc08004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.436+0000 7fdc1ed9d700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc08004d90 0x7fdc08005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.436+0000 7fdc1e59c700 1 -- 172.21.15.97:0/649432988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc2006b360 msgr2=0x7fdc08004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:56.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.436+0000 7fdc1e59c700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc2006b360 0x7fdc08004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.436+0000 7fdc1e59c700 1 -- 172.21.15.97:0/649432988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc08004d90 msgr2=0x7fdc08005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:56.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.436+0000 7fdc1e59c700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc08004d90 0x7fdc08005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.436+0000 7fdc1e59c700 1 -- 172.21.15.97:0/649432988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc080035b0 con 0x7fdc08003a30 2023-07-06T03:54:56.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.436+0000 7fdc1e59c700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 0x7fdc08003e40 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009b60 tx=0x7fdc08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90613277f8ee5e54 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:56.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.436+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000e6c0 con 0x7fdc08003a30 2023-07-06T03:54:56.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.436+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1000e820 con 0x7fdc08003a30 2023-07-06T03:54:56.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.436+0000 7fdc1d59a700 1 -- 172.21.15.97:0/649432988 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000eb90 con 0x7fdc08003a30 2023-07-06T03:54:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.437+0000 7fdc24fff700 1 -- 172.21.15.97:0/649432988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 msgr2=0x7fdc08003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.437+0000 7fdc24fff700 1 --2- 172.21.15.97:0/649432988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 0x7fdc08003e40 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009b60 tx=0x7fdc08005880 comp rx=0 tx=0).stop 2023-07-06T03:54:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.437+0000 7fdc1ed9d700 1 -- 172.21.15.97:0/649432988 reap_dead start 2023-07-06T03:54:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.437+0000 7fdc24fff700 1 -- 172.21.15.97:0/649432988 shutdown_connections 2023-07-06T03:54:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.437+0000 7fdc24fff700 1 -- 172.21.15.97:0/649432988 >> 172.21.15.97:0/649432988 conn(0x7fdc2006a6c0 msgr2=0x7fdc2006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:56.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.437+0000 7fdc24fff700 1 -- 172.21.15.97:0/649432988 shutdown_connections 2023-07-06T03:54:56.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.437+0000 7fdc24fff700 1 -- 172.21.15.97:0/649432988 wait complete. 2023-07-06T03:54:56.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.439+0000 7fdc24fff700 1 Processor -- start 2023-07-06T03:54:56.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.439+0000 7fdc24fff700 1 -- start start 2023-07-06T03:54:56.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc24fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 0x7fdc2014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc24fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc201f2f40 0x7fdc200b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc24fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc200b60a0 0x7fdc200b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc24fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2014f7d0 con 0x7fdc08003a30 2023-07-06T03:54:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc24fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200b6520 con 0x7fdc201f2f40 2023-07-06T03:54:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc24fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200b8a60 con 0x7fdc200b60a0 2023-07-06T03:54:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc1e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 0x7fdc2014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc1e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 0x7fdc2014f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57738/0 (socket says 172.21.15.97:57738) 2023-07-06T03:54:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc1e59c700 1 -- 172.21.15.97:0/3453564346 learned_addr learned my addr 172.21.15.97:0/3453564346 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc1dd9b700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc201f2f40 0x7fdc200b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc1ed9d700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc200b60a0 0x7fdc200b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc1e59c700 1 -- 172.21.15.97:0/3453564346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc200b60a0 msgr2=0x7fdc200b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc1e59c700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc200b60a0 0x7fdc200b7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc1e59c700 1 -- 172.21.15.97:0/3453564346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc201f2f40 msgr2=0x7fdc200b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc1e59c700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc201f2f40 0x7fdc200b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc1e59c700 1 -- 172.21.15.97:0/3453564346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2015dc50 con 0x7fdc08003a30 2023-07-06T03:54:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.440+0000 7fdc1dd9b700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc201f2f40 0x7fdc200b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.441+0000 7fdc1e59c700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 0x7fdc2014f290 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fdc100090b0 tx=0x7fdc100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.441+0000 7fdc0effd700 1 -- 172.21.15.97:0/3453564346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000e6c0 con 0x7fdc08003a30 2023-07-06T03:54:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.441+0000 7fdc0effd700 1 -- 172.21.15.97:0/3453564346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc10003ad0 con 0x7fdc08003a30 2023-07-06T03:54:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.441+0000 7fdc24fff700 1 -- 172.21.15.97:0/3453564346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2015df30 con 0x7fdc08003a30 2023-07-06T03:54:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.441+0000 7fdc0effd700 1 -- 172.21.15.97:0/3453564346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10003e80 con 0x7fdc08003a30 2023-07-06T03:54:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.441+0000 7fdc24fff700 1 -- 172.21.15.97:0/3453564346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2015e4f0 con 0x7fdc08003a30 2023-07-06T03:54:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.442+0000 7fdc24fff700 1 -- 172.21.15.97:0/3453564346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2015e680 con 0x7fdc08003a30 2023-07-06T03:54:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.443+0000 7fdc0effd700 1 -- 172.21.15.97:0/3453564346 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc10013470 con 0x7fdc08003a30 2023-07-06T03:54:56.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.444+0000 7fdc0effd700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc0006ace0 0x7fdc0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.444+0000 7fdc0effd700 1 -- 172.21.15.97:0/3453564346 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc1008ae00 con 0x7fdc08003a30 2023-07-06T03:54:56.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.444+0000 7fdc1dd9b700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc0006ace0 0x7fdc0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.445+0000 7fdc1dd9b700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc0006ace0 0x7fdc0006d190 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdc2006b790 tx=0x7fdc14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:56.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.449+0000 7fdc0effd700 1 -- 172.21.15.97:0/3453564346 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdc10055f40 con 0x7fdc08003a30 2023-07-06T03:54:56.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.640+0000 7fdc24fff700 1 -- 172.21.15.97:0/3453564346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdc2004e340 con 0x7fdc08003a30 2023-07-06T03:54:56.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.641+0000 7fdc0effd700 1 -- 172.21.15.97:0/3453564346 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v112) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdc10059560 con 0x7fdc08003a30 2023-07-06T03:54:56.644 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:54:56.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 -- 172.21.15.97:0/3453564346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc0006ace0 msgr2=0x7fdc0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc0006ace0 0x7fdc0006d190 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdc2006b790 tx=0x7fdc14008040 comp rx=0 tx=0).stop 2023-07-06T03:54:56.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 -- 172.21.15.97:0/3453564346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 msgr2=0x7fdc2014f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 0x7fdc2014f290 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fdc100090b0 tx=0x7fdc100047c0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 -- 172.21.15.97:0/3453564346 shutdown_connections 2023-07-06T03:54:56.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdc0006ace0 0x7fdc0006d190 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdc200b60a0 0x7fdc200b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc08003a30 0x7fdc2014f290 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 --2- 172.21.15.97:0/3453564346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc201f2f40 0x7fdc200b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 -- 172.21.15.97:0/3453564346 >> 172.21.15.97:0/3453564346 conn(0x7fdc2006a6c0 msgr2=0x7fdc2015ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:56.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 -- 172.21.15.97:0/3453564346 shutdown_connections 2023-07-06T03:54:56.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.643+0000 7fdc24fff700 1 -- 172.21.15.97:0/3453564346 wait complete. 2023-07-06T03:54:56.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:54:56.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.859+0000 7fc3921ff700 1 Processor -- start 2023-07-06T03:54:56.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.859+0000 7fc3921ff700 1 -- start start 2023-07-06T03:54:56.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.860+0000 7fc3921ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc38c06b360 0x7fc38c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc3921ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38c0cff90 con 0x7fc38c0d2210 2023-07-06T03:54:56.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc3921ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38c0b5650 con 0x7fc38c15c610 2023-07-06T03:54:56.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc3921ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c0d0780 con 0x7fc38c06b360 2023-07-06T03:54:56.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc38bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc38c0d2210 0x7fc38c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44452/0 (socket says 172.21.15.97:44452) 2023-07-06T03:54:56.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc38bfff700 1 -- 172.21.15.97:0/553461796 learned_addr learned my addr 172.21.15.97:0/553461796 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:56.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc38affd700 1 --2- 172.21.15.97:0/553461796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc38c06b360 0x7fc38c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc38a7fc700 1 -- 172.21.15.97:0/553461796 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402973543 0 0) 0x7fc38c0cff90 con 0x7fc38c0d2210 2023-07-06T03:54:56.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc38a7fc700 1 -- 172.21.15.97:0/553461796 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3740035b0 con 0x7fc38c0d2210 2023-07-06T03:54:56.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc38affd700 1 -- 172.21.15.97:0/553461796 >> v1:172.21.15.146:6789/0 conn(0x7fc38c15c610 legacy=0x7fc38c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc38a7fc700 1 -- 172.21.15.97:0/553461796 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060934221 0 0) 0x7fc38c0b5650 con 0x7fc38c15c610 2023-07-06T03:54:56.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc38affd700 1 -- 172.21.15.97:0/553461796 >> v1:172.21.15.97:6789/0 conn(0x7fc38c0d2210 legacy=0x7fc38c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.861+0000 7fc38affd700 1 -- 172.21.15.97:0/553461796 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c0b5400 con 0x7fc38c06b360 2023-07-06T03:54:56.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.862+0000 7fc38affd700 1 --2- 172.21.15.97:0/553461796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc38c06b360 0x7fc38c0b4ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc380009870 tx=0x7fc380009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe0ef39616fc43be server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:56.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.862+0000 7fc38a7fc700 1 -- 172.21.15.97:0/553461796 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380011830 con 0x7fc38c06b360 2023-07-06T03:54:56.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.862+0000 7fc38a7fc700 1 -- 172.21.15.97:0/553461796 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380011990 con 0x7fc38c06b360 2023-07-06T03:54:56.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.862+0000 7fc38a7fc700 1 -- 172.21.15.97:0/553461796 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380011d20 con 0x7fc38c06b360 2023-07-06T03:54:56.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.863+0000 7fc3921ff700 1 -- 172.21.15.97:0/553461796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc38c06b360 msgr2=0x7fc38c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.863+0000 7fc3921ff700 1 --2- 172.21.15.97:0/553461796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc38c06b360 0x7fc38c0b4ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc380009870 tx=0x7fc380009b80 comp rx=0 tx=0).stop 2023-07-06T03:54:56.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.863+0000 7fc3921ff700 1 -- 172.21.15.97:0/553461796 shutdown_connections 2023-07-06T03:54:56.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.863+0000 7fc3921ff700 1 --2- 172.21.15.97:0/553461796 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc38c06b360 0x7fc38c0b4ec0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.863+0000 7fc3921ff700 1 -- 172.21.15.97:0/553461796 >> 172.21.15.97:0/553461796 conn(0x7fc38c06a6c0 msgr2=0x7fc38c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:56.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.863+0000 7fc3921ff700 1 -- 172.21.15.97:0/553461796 shutdown_connections 2023-07-06T03:54:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.863+0000 7fc3921ff700 1 -- 172.21.15.97:0/553461796 wait complete. 2023-07-06T03:54:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.865+0000 7fc3921ff700 1 Processor -- start 2023-07-06T03:54:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.865+0000 7fc3921ff700 1 -- start start 2023-07-06T03:54:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.866+0000 7fc3921ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c06b360 0x7fc38c0cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.866+0000 7fc3921ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc38c0d2210 0x7fc38c145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.866+0000 7fc3921ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc38c15c610 0x7fc38c0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.866+0000 7fc3921ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c1f1f40 con 0x7fc38c15c610 2023-07-06T03:54:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.866+0000 7fc3921ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c1f64b0 con 0x7fc38c06b360 2023-07-06T03:54:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.866+0000 7fc3921ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c1f5930 con 0x7fc38c0d2210 2023-07-06T03:54:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.867+0000 7fc38bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc38c15c610 0x7fc38c0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.867+0000 7fc38bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc38c15c610 0x7fc38c0b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57742/0 (socket says 172.21.15.97:57742) 2023-07-06T03:54:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.867+0000 7fc38bfff700 1 -- 172.21.15.97:0/1908627316 learned_addr learned my addr 172.21.15.97:0/1908627316 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.867+0000 7fc38affd700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc38c0d2210 0x7fc38c145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.867+0000 7fc38b7fe700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c06b360 0x7fc38c0cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.867+0000 7fc38bfff700 1 -- 172.21.15.97:0/1908627316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc38c0d2210 msgr2=0x7fc38c145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.867+0000 7fc38bfff700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc38c0d2210 0x7fc38c145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.867+0000 7fc38bfff700 1 -- 172.21.15.97:0/1908627316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c06b360 msgr2=0x7fc38c0cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.867+0000 7fc38bfff700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c06b360 0x7fc38c0cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.867+0000 7fc38bfff700 1 -- 172.21.15.97:0/1908627316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3800094a0 con 0x7fc38c15c610 2023-07-06T03:54:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.868+0000 7fc38bfff700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc38c15c610 0x7fc38c0b61b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fc38400cb40 tx=0x7fc38400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.868+0000 7fc373fff700 1 -- 172.21.15.97:0/1908627316 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38400ab50 con 0x7fc38c15c610 2023-07-06T03:54:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.868+0000 7fc373fff700 1 -- 172.21.15.97:0/1908627316 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38400acb0 con 0x7fc38c15c610 2023-07-06T03:54:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.868+0000 7fc3921ff700 1 -- 172.21.15.97:0/1908627316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38c1f5c10 con 0x7fc38c15c610 2023-07-06T03:54:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.868+0000 7fc373fff700 1 -- 172.21.15.97:0/1908627316 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384012620 con 0x7fc38c15c610 2023-07-06T03:54:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.868+0000 7fc3921ff700 1 -- 172.21.15.97:0/1908627316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38c1f61d0 con 0x7fc38c15c610 2023-07-06T03:54:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.869+0000 7fc3921ff700 1 -- 172.21.15.97:0/1908627316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38c15dd80 con 0x7fc38c15c610 2023-07-06T03:54:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.870+0000 7fc373fff700 1 -- 172.21.15.97:0/1908627316 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc38401b3b0 con 0x7fc38c15c610 2023-07-06T03:54:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.871+0000 7fc373fff700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc37806ace0 0x7fc37806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.871+0000 7fc373fff700 1 -- 172.21.15.97:0/1908627316 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3840891d0 con 0x7fc38c15c610 2023-07-06T03:54:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.872+0000 7fc38b7fe700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc37806ace0 0x7fc37806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:56.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.872+0000 7fc38b7fe700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc37806ace0 0x7fc37806d190 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc37c007fd0 tx=0x7fc37c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:56.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:56.876+0000 7fc373fff700 1 -- 172.21.15.97:0/1908627316 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc38c15dd80 con 0x7fc38c15c610 2023-07-06T03:54:57.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.075+0000 7fc3921ff700 1 -- 172.21.15.97:0/1908627316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc38c15dd80 con 0x7fc38c15c610 2023-07-06T03:54:57.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.075+0000 7fc373fff700 1 -- 172.21.15.97:0/1908627316 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc38c15dd80 con 0x7fc38c15c610 2023-07-06T03:54:57.078 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:54:57.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.077+0000 7fc3921ff700 1 -- 172.21.15.97:0/1908627316 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc37806ace0 msgr2=0x7fc37806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.077+0000 7fc3921ff700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc37806ace0 0x7fc37806d190 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc37c007fd0 tx=0x7fc37c009040 comp rx=0 tx=0).stop 2023-07-06T03:54:57.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.077+0000 7fc3921ff700 1 -- 172.21.15.97:0/1908627316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc38c15c610 msgr2=0x7fc38c0b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.077+0000 7fc3921ff700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc38c15c610 0x7fc38c0b61b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fc38400cb40 tx=0x7fc38400cf00 comp rx=0 tx=0).stop 2023-07-06T03:54:57.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.077+0000 7fc3921ff700 1 -- 172.21.15.97:0/1908627316 shutdown_connections 2023-07-06T03:54:57.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.077+0000 7fc3921ff700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc37806ace0 0x7fc37806d190 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.077+0000 7fc3921ff700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc38c15c610 0x7fc38c0b61b0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.077+0000 7fc3921ff700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c06b360 0x7fc38c0cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.077+0000 7fc3921ff700 1 --2- 172.21.15.97:0/1908627316 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc38c0d2210 0x7fc38c145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.078+0000 7fc3921ff700 1 -- 172.21.15.97:0/1908627316 >> 172.21.15.97:0/1908627316 conn(0x7fc38c06a6c0 msgr2=0x7fc38c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:57.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.078+0000 7fc3921ff700 1 -- 172.21.15.97:0/1908627316 shutdown_connections 2023-07-06T03:54:57.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.078+0000 7fc3921ff700 1 -- 172.21.15.97:0/1908627316 wait complete. 2023-07-06T03:54:57.098 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:54:57.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.295+0000 7f43a9fff700 1 Processor -- start 2023-07-06T03:54:57.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.295+0000 7f43a9fff700 1 -- start start 2023-07-06T03:54:57.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a9fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f43a415c610 0x7f43a40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a9fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43a406b360 0x7f43a40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a9fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f43a40d2210 0x7f43a40d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a9fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a40d0780 con 0x7f43a40d2210 2023-07-06T03:54:57.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a9fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a415c9e0 con 0x7f43a415c610 2023-07-06T03:54:57.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a9fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40b5650 con 0x7f43a406b360 2023-07-06T03:54:57.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f43a40d2210 0x7f43a40d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f43a40d2210 0x7f43a40d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57758/0 (socket says 172.21.15.97:57758) 2023-07-06T03:54:57.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a3fff700 1 -- 172.21.15.97:0/793447577 learned_addr learned my addr 172.21.15.97:0/793447577 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:57.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a2ffd700 1 --2- 172.21.15.97:0/793447577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43a406b360 0x7f43a40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a37fe700 1 --2- 172.21.15.97:0/793447577 >> v2:172.21.15.146:3300/0 conn(0x7f43a415c610 0x7f43a40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a3fff700 1 -- 172.21.15.97:0/793447577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43a406b360 msgr2=0x7f43a40b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:57.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a3fff700 1 --2- 172.21.15.97:0/793447577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43a406b360 0x7f43a40b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.296+0000 7f43a3fff700 1 -- 172.21.15.97:0/793447577 >> v2:172.21.15.146:3300/0 conn(0x7f43a415c610 msgr2=0x7f43a40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:57.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.297+0000 7f43a3fff700 1 --2- 172.21.15.97:0/793447577 >> v2:172.21.15.146:3300/0 conn(0x7f43a415c610 0x7f43a40b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.297+0000 7f43a3fff700 1 -- 172.21.15.97:0/793447577 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a40d01c0 con 0x7f43a40d2210 2023-07-06T03:54:57.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.297+0000 7f43a37fe700 1 --2- 172.21.15.97:0/793447577 >> v2:172.21.15.146:3300/0 conn(0x7f43a415c610 0x7f43a40b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:54:57.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.297+0000 7f43a3fff700 1 --2- 172.21.15.97:0/793447577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a40d2210 0x7f43a40d1310 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f43a408b000 tx=0x7f439c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee0ae65063213e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:57.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.297+0000 7f43a27fc700 1 -- 172.21.15.97:0/793447577 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00e070 con 0x7f43a40d2210 2023-07-06T03:54:57.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.297+0000 7f43a27fc700 1 -- 172.21.15.97:0/793447577 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c00b980 con 0x7f43a40d2210 2023-07-06T03:54:57.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.297+0000 7f43a27fc700 1 -- 172.21.15.97:0/793447577 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00bd30 con 0x7f43a40d2210 2023-07-06T03:54:57.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.297+0000 7f43a9fff700 1 -- 172.21.15.97:0/793447577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a40d2210 msgr2=0x7f43a40d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.297+0000 7f43a9fff700 1 --2- 172.21.15.97:0/793447577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a40d2210 0x7f43a40d1310 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f43a408b000 tx=0x7f439c00b330 comp rx=0 tx=0).stop 2023-07-06T03:54:57.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.298+0000 7f43a9fff700 1 -- 172.21.15.97:0/793447577 shutdown_connections 2023-07-06T03:54:57.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.298+0000 7f43a9fff700 1 --2- 172.21.15.97:0/793447577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a40d2210 0x7f43a40d1310 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.298+0000 7f43a9fff700 1 --2- 172.21.15.97:0/793447577 >> v2:172.21.15.146:3300/0 conn(0x7f43a415c610 0x7f43a40b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.298+0000 7f43a9fff700 1 --2- 172.21.15.97:0/793447577 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43a406b360 0x7f43a40b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.298+0000 7f43a9fff700 1 -- 172.21.15.97:0/793447577 >> 172.21.15.97:0/793447577 conn(0x7f43a406a6c0 msgr2=0x7f43a406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:57.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.298+0000 7f43a9fff700 1 -- 172.21.15.97:0/793447577 shutdown_connections 2023-07-06T03:54:57.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.298+0000 7f43a9fff700 1 -- 172.21.15.97:0/793447577 wait complete. 2023-07-06T03:54:57.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.299+0000 7f43a9fff700 1 Processor -- start 2023-07-06T03:54:57.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.299+0000 7f43a9fff700 1 -- start start 2023-07-06T03:54:57.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.300+0000 7f43a9fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a406b360 0x7f43a41497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.300+0000 7f43a9fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43a40d2210 0x7f43a40c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.300+0000 7f43a9fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43a415c610 0x7f43a415d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.300+0000 7f43a9fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40cae00 con 0x7f43a406b360 2023-07-06T03:54:57.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.300+0000 7f43a9fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a415d770 con 0x7f43a415c610 2023-07-06T03:54:57.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.300+0000 7f43a9fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40ca4f0 con 0x7f43a40d2210 2023-07-06T03:54:57.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f43a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a406b360 0x7f43a41497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f43a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a406b360 0x7f43a41497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57760/0 (socket says 172.21.15.97:57760) 2023-07-06T03:54:57.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f43a37fe700 1 -- 172.21.15.97:0/3334489767 learned_addr learned my addr 172.21.15.97:0/3334489767 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:57.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f43a2ffd700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43a40d2210 0x7f43a40c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f43a3fff700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43a415c610 0x7f43a415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f43a37fe700 1 -- 172.21.15.97:0/3334489767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43a40d2210 msgr2=0x7f43a40c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f43a37fe700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43a40d2210 0x7f43a40c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f43a37fe700 1 -- 172.21.15.97:0/3334489767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43a415c610 msgr2=0x7f43a415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f43a37fe700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43a415c610 0x7f43a415d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f43a37fe700 1 -- 172.21.15.97:0/3334489767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c009b30 con 0x7f43a406b360 2023-07-06T03:54:57.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f43a37fe700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a406b360 0x7f43a41497c0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f439400c7f0 tx=0x7f439400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:57.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f438bfff700 1 -- 172.21.15.97:0/3334489767 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394011800 con 0x7f43a406b360 2023-07-06T03:54:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.301+0000 7f438bfff700 1 -- 172.21.15.97:0/3334489767 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4394011960 con 0x7f43a406b360 2023-07-06T03:54:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.302+0000 7f43a9fff700 1 -- 172.21.15.97:0/3334489767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a40c9960 con 0x7f43a406b360 2023-07-06T03:54:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.302+0000 7f438bfff700 1 -- 172.21.15.97:0/3334489767 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394011d10 con 0x7f43a406b360 2023-07-06T03:54:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.302+0000 7f43a9fff700 1 -- 172.21.15.97:0/3334489767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a40caa20 con 0x7f43a406b360 2023-07-06T03:54:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.303+0000 7f43a9fff700 1 -- 172.21.15.97:0/3334489767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a40cf650 con 0x7f43a406b360 2023-07-06T03:54:57.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.304+0000 7f438bfff700 1 -- 172.21.15.97:0/3334489767 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4394018360 con 0x7f43a406b360 2023-07-06T03:54:57.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.304+0000 7f438bfff700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f439006ace0 0x7f439006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.305+0000 7f438bfff700 1 -- 172.21.15.97:0/3334489767 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f43940893d0 con 0x7f43a406b360 2023-07-06T03:54:57.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.305+0000 7f43a2ffd700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f439006ace0 0x7f439006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.305+0000 7f43a2ffd700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f439006ace0 0x7f439006d190 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f43a40ccdd0 tx=0x7f4398008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:57.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.309+0000 7f438bfff700 1 -- 172.21.15.97:0/3334489767 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4394057fa0 con 0x7f43a406b360 2023-07-06T03:54:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.504+0000 7f43a9fff700 1 -- 172.21.15.97:0/3334489767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43a40b8440 con 0x7f43a406b360 2023-07-06T03:54:57.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.505+0000 7f438bfff700 1 -- 172.21.15.97:0/3334489767 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4394057b30 con 0x7f43a406b360 2023-07-06T03:54:57.508 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:54:57.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 -- 172.21.15.97:0/3334489767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f439006ace0 msgr2=0x7f439006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f439006ace0 0x7f439006d190 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f43a40ccdd0 tx=0x7f4398008040 comp rx=0 tx=0).stop 2023-07-06T03:54:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 -- 172.21.15.97:0/3334489767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a406b360 msgr2=0x7f43a41497c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a406b360 0x7f43a41497c0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f439400c7f0 tx=0x7f439400cbb0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 -- 172.21.15.97:0/3334489767 shutdown_connections 2023-07-06T03:54:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f439006ace0 0x7f439006d190 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43a415c610 0x7f43a415d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a406b360 0x7f43a41497c0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 --2- 172.21.15.97:0/3334489767 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43a40d2210 0x7f43a40c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 -- 172.21.15.97:0/3334489767 >> 172.21.15.97:0/3334489767 conn(0x7f43a406a6c0 msgr2=0x7f43a40cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:57.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 -- 172.21.15.97:0/3334489767 shutdown_connections 2023-07-06T03:54:57.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.507+0000 7f43a9fff700 1 -- 172.21.15.97:0/3334489767 wait complete. 2023-07-06T03:54:57.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.718+0000 7f539a5ff700 1 Processor -- start 2023-07-06T03:54:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.718+0000 7f539a5ff700 1 -- start start 2023-07-06T03:54:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.719+0000 7f539a5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f53940cd390 0x7f5394160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.719+0000 7f539a5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5394160dd0 0x7f53940b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.719+0000 7f539a5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53940cdae0 con 0x7f539415c520 2023-07-06T03:54:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.719+0000 7f539a5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53940b6640 con 0x7f53940cd390 2023-07-06T03:54:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.719+0000 7f539a5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53940b6920 con 0x7f5394160dd0 2023-07-06T03:54:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.720+0000 7f53937fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f53940cd390 0x7f5394160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.720+0000 7f53937fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f53940cd390 0x7f5394160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57764/0 (socket says 172.21.15.97:57764) 2023-07-06T03:54:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.720+0000 7f53937fe700 1 -- 172.21.15.97:0/3040116463 learned_addr learned my addr 172.21.15.97:0/3040116463 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.720+0000 7f5393fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f539415c520 0x7f5394160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44454/0 (socket says 172.21.15.97:44454) 2023-07-06T03:54:57.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.720+0000 7f5398b9c700 1 --2- 172.21.15.97:0/3040116463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5394160dd0 0x7f53940b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.720+0000 7f53937fe700 1 -- 172.21.15.97:0/3040116463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5394160dd0 msgr2=0x7f53940b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.720+0000 7f53937fe700 1 --2- 172.21.15.97:0/3040116463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5394160dd0 0x7f53940b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.720+0000 7f53937fe700 1 -- 172.21.15.97:0/3040116463 >> v1:172.21.15.97:6789/0 conn(0x7f539415c520 legacy=0x7f5394160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.720+0000 7f53937fe700 1 -- 172.21.15.97:0/3040116463 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940b6c00 con 0x7f53940cd390 2023-07-06T03:54:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.720+0000 7f53937fe700 1 --2- 172.21.15.97:0/3040116463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53940cd390 0x7f5394160890 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f53880098f0 tx=0x7f5388009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e842304601d9869c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.721+0000 7f5392ffd700 1 -- 172.21.15.97:0/3040116463 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538800b8a0 con 0x7f53940cd390 2023-07-06T03:54:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.721+0000 7f5392ffd700 1 -- 172.21.15.97:0/3040116463 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538800ba00 con 0x7f53940cd390 2023-07-06T03:54:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.721+0000 7f5392ffd700 1 -- 172.21.15.97:0/3040116463 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538800bd90 con 0x7f53940cd390 2023-07-06T03:54:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.721+0000 7f539a5ff700 1 -- 172.21.15.97:0/3040116463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53940cd390 msgr2=0x7f5394160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.721+0000 7f539a5ff700 1 --2- 172.21.15.97:0/3040116463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53940cd390 0x7f5394160890 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f53880098f0 tx=0x7f5388009c00 comp rx=0 tx=0).stop 2023-07-06T03:54:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.721+0000 7f539a5ff700 1 -- 172.21.15.97:0/3040116463 shutdown_connections 2023-07-06T03:54:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.721+0000 7f539a5ff700 1 --2- 172.21.15.97:0/3040116463 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5394160dd0 0x7f53940b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.721+0000 7f539a5ff700 1 --2- 172.21.15.97:0/3040116463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53940cd390 0x7f5394160890 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.721+0000 7f539a5ff700 1 -- 172.21.15.97:0/3040116463 >> 172.21.15.97:0/3040116463 conn(0x7f5394071790 msgr2=0x7f5394071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.721+0000 7f539a5ff700 1 -- 172.21.15.97:0/3040116463 shutdown_connections 2023-07-06T03:54:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.721+0000 7f539a5ff700 1 -- 172.21.15.97:0/3040116463 wait complete. 2023-07-06T03:54:57.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.723+0000 7f539a5ff700 1 Processor -- start 2023-07-06T03:54:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.723+0000 7f539a5ff700 1 -- start start 2023-07-06T03:54:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.724+0000 7f539a5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53940cd390 0x7f53940cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.724+0000 7f539a5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f539415c520 0x7f53940d3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.724+0000 7f539a5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5394160dd0 0x7f539415d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.724+0000 7f539a5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53940d3a70 con 0x7f5394160dd0 2023-07-06T03:54:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.724+0000 7f539a5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539415f1d0 con 0x7f539415c520 2023-07-06T03:54:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.724+0000 7f539a5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539415fa30 con 0x7f53940cd390 2023-07-06T03:54:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.724+0000 7f5398b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5394160dd0 0x7f539415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.724+0000 7f5398b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5394160dd0 0x7f539415d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57774/0 (socket says 172.21.15.97:57774) 2023-07-06T03:54:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.724+0000 7f5398b9c700 1 -- 172.21.15.97:0/4202765736 learned_addr learned my addr 172.21.15.97:0/4202765736 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.724+0000 7f5393fff700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53940cd390 0x7f53940cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f53937fe700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f539415c520 0x7f53940d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f5398b9c700 1 -- 172.21.15.97:0/4202765736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53940cd390 msgr2=0x7f53940cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f5398b9c700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53940cd390 0x7f53940cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f5398b9c700 1 -- 172.21.15.97:0/4202765736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f539415c520 msgr2=0x7f53940d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f5398b9c700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f539415c520 0x7f53940d3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f5398b9c700 1 -- 172.21.15.97:0/4202765736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5388009550 con 0x7f5394160dd0 2023-07-06T03:54:57.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f53937fe700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f539415c520 0x7f53940d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f5398b9c700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5394160dd0 0x7f539415d790 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f538c007e60 tx=0x7f538c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f5390ff9700 1 -- 172.21.15.97:0/4202765736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00ede0 con 0x7f5394160dd0 2023-07-06T03:54:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f5390ff9700 1 -- 172.21.15.97:0/4202765736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c014420 con 0x7f5394160dd0 2023-07-06T03:54:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f5390ff9700 1 -- 172.21.15.97:0/4202765736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c0147d0 con 0x7f5394160dd0 2023-07-06T03:54:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f539a5ff700 1 -- 172.21.15.97:0/4202765736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539415f470 con 0x7f5394160dd0 2023-07-06T03:54:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.725+0000 7f539a5ff700 1 -- 172.21.15.97:0/4202765736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53940cab00 con 0x7f5394160dd0 2023-07-06T03:54:57.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.726+0000 7f539a5ff700 1 -- 172.21.15.97:0/4202765736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539404e340 con 0x7f5394160dd0 2023-07-06T03:54:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.727+0000 7f5390ff9700 1 -- 172.21.15.97:0/4202765736 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f538c014930 con 0x7f5394160dd0 2023-07-06T03:54:57.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.728+0000 7f5390ff9700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f538006ad10 0x7f538006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.728+0000 7f5390ff9700 1 -- 172.21.15.97:0/4202765736 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f538c089c50 con 0x7f5394160dd0 2023-07-06T03:54:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.728+0000 7f5393fff700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f538006ad10 0x7f538006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.729+0000 7f5393fff700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f538006ad10 0x7f538006d1c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5384000f60 tx=0x7f5384007ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:57.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.732+0000 7f5390ff9700 1 -- 172.21.15.97:0/4202765736 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f538c054bd0 con 0x7f5394160dd0 2023-07-06T03:54:57.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.974+0000 7f539a5ff700 1 -- 172.21.15.97:0/4202765736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53940b4f40 con 0x7f5394160dd0 2023-07-06T03:54:57.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.974+0000 7f5390ff9700 1 -- 172.21.15.97:0/4202765736 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4677 (secure 0 0 0) 0x7f538c0581f0 con 0x7f5394160dd0 2023-07-06T03:54:57.977 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:57.978 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":83,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2796855133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2796855133},{"type":"v1","addr":"172.21.15.97:6835","nonce":2796855133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6851,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1092966074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1092966074},{"type":"v1","addr":"172.21.15.146:6835","nonce":1092966074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8577,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2594138359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2594138359},{"type":"v1","addr":"172.21.15.97:6837","nonce":2594138359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:54:25.441909+0000","modified":"2023-07-06T03:54:54.793074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6834/582690410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":582690410},{"type":"v1","addr":"172.21.15.146:6834","nonce":582690410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-06T03:54:57.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 -- 172.21.15.97:0/4202765736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f538006ad10 msgr2=0x7f538006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f538006ad10 0x7f538006d1c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5384000f60 tx=0x7f5384007ed0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 -- 172.21.15.97:0/4202765736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5394160dd0 msgr2=0x7f539415d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:57.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5394160dd0 0x7f539415d790 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f538c007e60 tx=0x7f538c00e3c0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 -- 172.21.15.97:0/4202765736 shutdown_connections 2023-07-06T03:54:57.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f538006ad10 0x7f538006d1c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5394160dd0 0x7f539415d790 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f53940cd390 0x7f53940cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 --2- 172.21.15.97:0/4202765736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f539415c520 0x7f53940d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:57.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 -- 172.21.15.97:0/4202765736 >> 172.21.15.97:0/4202765736 conn(0x7f5394071790 msgr2=0x7f5394071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:57.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 -- 172.21.15.97:0/4202765736 shutdown_connections 2023-07-06T03:54:57.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:57.977+0000 7f539a5ff700 1 -- 172.21.15.97:0/4202765736 wait complete. 2023-07-06T03:54:57.981 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 83 2023-07-06T03:54:57.999 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:58.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.191+0000 7fe2bbbff700 1 Processor -- start 2023-07-06T03:54:58.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.191+0000 7fe2bbbff700 1 -- start start 2023-07-06T03:54:58.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.192+0000 7fe2bbbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe2b415c520 0x7fe2b40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.192+0000 7fe2bbbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160d00 0x7fe2b40b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.192+0000 7fe2bbbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b40cdae0 con 0x7fe2b4160180 2023-07-06T03:54:58.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.192+0000 7fe2bbbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b40cd8d0 con 0x7fe2b415c520 2023-07-06T03:54:58.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.192+0000 7fe2bbbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40b6640 con 0x7fe2b4160d00 2023-07-06T03:54:58.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.193+0000 7fe2b999b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe2b415c520 0x7fe2b40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:58.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.193+0000 7fe2b919a700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fe2b4160180 0x7fe2b41605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:55938/0 (socket says 172.21.15.97:55938) 2023-07-06T03:54:58.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.193+0000 7fe2b919a700 1 -- 172.21.15.97:0/1480562168 learned_addr learned my addr 172.21.15.97:0/1480562168 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:58.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.193+0000 7fe2ba19c700 1 --2- 172.21.15.97:0/1480562168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160d00 0x7fe2b40b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.194+0000 7fe2b999b700 1 -- 172.21.15.97:0/1480562168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160d00 msgr2=0x7fe2b40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.194+0000 7fe2b999b700 1 --2- 172.21.15.97:0/1480562168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160d00 0x7fe2b40b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.194+0000 7fe2b999b700 1 -- 172.21.15.97:0/1480562168 >> v1:172.21.15.146:6789/0 conn(0x7fe2b4160180 legacy=0x7fe2b41605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.194+0000 7fe2b999b700 1 -- 172.21.15.97:0/1480562168 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b40b6920 con 0x7fe2b415c520 2023-07-06T03:54:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.194+0000 7fe2ba19c700 1 --2- 172.21.15.97:0/1480562168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160d00 0x7fe2b40b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:54:58.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.194+0000 7fe2b8999700 1 -- 172.21.15.97:0/1480562168 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029011005 0 0) 0x7fe2b40cdae0 con 0x7fe2b4160180 2023-07-06T03:54:58.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.194+0000 7fe2b999b700 1 --2- 172.21.15.97:0/1480562168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b415c520 0x7fe2b40cd390 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009840 tx=0x7fe2b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc87ba7f73adf5ee server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:58.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.194+0000 7fe2b8999700 1 -- 172.21.15.97:0/1480562168 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0011830 con 0x7fe2b415c520 2023-07-06T03:54:58.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.194+0000 7fe2b8999700 1 -- 172.21.15.97:0/1480562168 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b0011990 con 0x7fe2b415c520 2023-07-06T03:54:58.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.194+0000 7fe2b8999700 1 -- 172.21.15.97:0/1480562168 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0011d20 con 0x7fe2b415c520 2023-07-06T03:54:58.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.195+0000 7fe2bbbff700 1 -- 172.21.15.97:0/1480562168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b415c520 msgr2=0x7fe2b40cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.195+0000 7fe2bbbff700 1 --2- 172.21.15.97:0/1480562168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b415c520 0x7fe2b40cd390 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009840 tx=0x7fe2b0009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:58.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.195+0000 7fe2bbbff700 1 -- 172.21.15.97:0/1480562168 shutdown_connections 2023-07-06T03:54:58.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.195+0000 7fe2bbbff700 1 --2- 172.21.15.97:0/1480562168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160d00 0x7fe2b40b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.195+0000 7fe2bbbff700 1 --2- 172.21.15.97:0/1480562168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b415c520 0x7fe2b40cd390 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.195+0000 7fe2bbbff700 1 -- 172.21.15.97:0/1480562168 >> 172.21.15.97:0/1480562168 conn(0x7fe2b4071790 msgr2=0x7fe2b4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:58.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.195+0000 7fe2bbbff700 1 -- 172.21.15.97:0/1480562168 shutdown_connections 2023-07-06T03:54:58.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.195+0000 7fe2bbbff700 1 -- 172.21.15.97:0/1480562168 wait complete. 2023-07-06T03:54:58.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.196+0000 7fe2bbbff700 1 Processor -- start 2023-07-06T03:54:58.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.197+0000 7fe2bbbff700 1 -- start start 2023-07-06T03:54:58.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.197+0000 7fe2bbbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b415c520 0x7fe2b40ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.197+0000 7fe2bbbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160180 0x7fe2b414d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.197+0000 7fe2bbbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b4160d00 0x7fe2b40cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.197+0000 7fe2bbbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40cae90 con 0x7fe2b415c520 2023-07-06T03:54:58.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.197+0000 7fe2bbbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40ca4c0 con 0x7fe2b4160d00 2023-07-06T03:54:58.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.197+0000 7fe2bbbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40c9eb0 con 0x7fe2b4160180 2023-07-06T03:54:58.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2b999b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b415c520 0x7fe2b40ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:58.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2b999b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b415c520 0x7fe2b40ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57778/0 (socket says 172.21.15.97:57778) 2023-07-06T03:54:58.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2b999b700 1 -- 172.21.15.97:0/3537629444 learned_addr learned my addr 172.21.15.97:0/3537629444 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:58.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2ba19c700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b4160d00 0x7fe2b40cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:58.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2b919a700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160180 0x7fe2b414d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:58.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2b999b700 1 -- 172.21.15.97:0/3537629444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160180 msgr2=0x7fe2b414d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2b999b700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160180 0x7fe2b414d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2b999b700 1 -- 172.21.15.97:0/3537629444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b4160d00 msgr2=0x7fe2b40cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2b999b700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b4160d00 0x7fe2b40cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2b999b700 1 -- 172.21.15.97:0/3537629444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac009a70 con 0x7fe2b415c520 2023-07-06T03:54:58.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2b919a700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160180 0x7fe2b414d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:54:58.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.198+0000 7fe2b999b700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b415c520 0x7fe2b40ca950 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009fd0 tx=0x7fe2b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:58.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.199+0000 7fe2aa7fc700 1 -- 172.21.15.97:0/3537629444 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00036a0 con 0x7fe2b415c520 2023-07-06T03:54:58.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.199+0000 7fe2aa7fc700 1 -- 172.21.15.97:0/3537629444 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b001f070 con 0x7fe2b415c520 2023-07-06T03:54:58.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.199+0000 7fe2bbbff700 1 -- 172.21.15.97:0/3537629444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b00094a0 con 0x7fe2b415c520 2023-07-06T03:54:58.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.199+0000 7fe2aa7fc700 1 -- 172.21.15.97:0/3537629444 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0003a40 con 0x7fe2b415c520 2023-07-06T03:54:58.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.199+0000 7fe2bbbff700 1 -- 172.21.15.97:0/3537629444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b40ca150 con 0x7fe2b415c520 2023-07-06T03:54:58.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.200+0000 7fe2bbbff700 1 -- 172.21.15.97:0/3537629444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b40ca7a0 con 0x7fe2b415c520 2023-07-06T03:54:58.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.201+0000 7fe2aa7fc700 1 -- 172.21.15.97:0/3537629444 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe2b0003ba0 con 0x7fe2b415c520 2023-07-06T03:54:58.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.202+0000 7fe2aa7fc700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe29806ace0 0x7fe29806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.202+0000 7fe2aa7fc700 1 -- 172.21.15.97:0/3537629444 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe2b0021070 con 0x7fe2b415c520 2023-07-06T03:54:58.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.202+0000 7fe2b919a700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe29806ace0 0x7fe29806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:58.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.202+0000 7fe2b919a700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe29806ace0 0x7fe29806d190 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40cd5f0 tx=0x7fe2a4007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:58.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.206+0000 7fe2aa7fc700 1 -- 172.21.15.97:0/3537629444 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe2b0055de0 con 0x7fe2b415c520 2023-07-06T03:54:58.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.403+0000 7fe2bbbff700 1 -- 172.21.15.97:0/3537629444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2b40d0730 con 0x7fe2b415c520 2023-07-06T03:54:58.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.405+0000 7fe2aa7fc700 1 -- 172.21.15.97:0/3537629444 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18391 (secure 0 0 0) 0x7fe2b0015090 con 0x7fe2b415c520 2023-07-06T03:54:58.408 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:58.408 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":112,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:54:26.969421+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:54:23.626811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-06T03:54:24.421787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:58.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.407+0000 7fe2bbbff700 1 -- 172.21.15.97:0/3537629444 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe29806ace0 msgr2=0x7fe29806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.407+0000 7fe2bbbff700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe29806ace0 0x7fe29806d190 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40cd5f0 tx=0x7fe2a4007ca0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.407+0000 7fe2bbbff700 1 -- 172.21.15.97:0/3537629444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b415c520 msgr2=0x7fe2b40ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.407+0000 7fe2bbbff700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b415c520 0x7fe2b40ca950 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009fd0 tx=0x7fe2b0004750 comp rx=0 tx=0).stop 2023-07-06T03:54:58.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.407+0000 7fe2bbbff700 1 -- 172.21.15.97:0/3537629444 shutdown_connections 2023-07-06T03:54:58.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.407+0000 7fe2bbbff700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe29806ace0 0x7fe29806d190 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.407+0000 7fe2bbbff700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2b4160d00 0x7fe2b40cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.407+0000 7fe2bbbff700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2b415c520 0x7fe2b40ca950 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.407+0000 7fe2bbbff700 1 --2- 172.21.15.97:0/3537629444 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe2b4160180 0x7fe2b414d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.407+0000 7fe2bbbff700 1 -- 172.21.15.97:0/3537629444 >> 172.21.15.97:0/3537629444 conn(0x7fe2b4071790 msgr2=0x7fe2b40c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:58.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.408+0000 7fe2bbbff700 1 -- 172.21.15.97:0/3537629444 shutdown_connections 2023-07-06T03:54:58.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.408+0000 7fe2bbbff700 1 -- 172.21.15.97:0/3537629444 wait complete. 2023-07-06T03:54:58.428 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T03:54:58.429 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:58.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.620+0000 7f2bf41ff700 1 Processor -- start 2023-07-06T03:54:58.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.620+0000 7f2bf41ff700 1 -- start start 2023-07-06T03:54:58.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.621+0000 7f2bf41ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bec15c520 0x7f2bec0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.621+0000 7f2bf41ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2bec160180 0x7f2bec1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.621+0000 7f2bf41ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec0cdae0 con 0x7f2bec160b30 2023-07-06T03:54:58.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.621+0000 7f2bf41ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bec0cd8d0 con 0x7f2bec160180 2023-07-06T03:54:58.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.622+0000 7f2bf41ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0b6640 con 0x7f2bec15c520 2023-07-06T03:54:58.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.622+0000 7f2bf1f9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bec15c520 0x7f2bec0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:58.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.622+0000 7f2bf279c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f2bec160b30 0x7f2bec0cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:55952/0 (socket says 172.21.15.97:55952) 2023-07-06T03:54:58.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.622+0000 7f2bf279c700 1 -- 172.21.15.97:0/857240121 learned_addr learned my addr 172.21.15.97:0/857240121 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:58.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.622+0000 7f2bf179a700 1 --2- 172.21.15.97:0/857240121 >> v2:172.21.15.146:3300/0 conn(0x7f2bec160180 0x7f2bec1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:58.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.622+0000 7f2bf0f99700 1 -- 172.21.15.97:0/857240121 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355238220 0 0) 0x7f2bec0cdae0 con 0x7f2bec160b30 2023-07-06T03:54:58.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.623+0000 7f2bf0f99700 1 -- 172.21.15.97:0/857240121 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd80035b0 con 0x7f2bec160b30 2023-07-06T03:54:58.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.623+0000 7f2bf179a700 1 -- 172.21.15.97:0/857240121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bec15c520 msgr2=0x7f2bec0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:58.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.623+0000 7f2bf179a700 1 --2- 172.21.15.97:0/857240121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bec15c520 0x7f2bec0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.623+0000 7f2bf179a700 1 -- 172.21.15.97:0/857240121 >> v1:172.21.15.146:6789/0 conn(0x7f2bec160b30 legacy=0x7f2bec0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.623+0000 7f2bf179a700 1 -- 172.21.15.97:0/857240121 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec0b6920 con 0x7f2bec160180 2023-07-06T03:54:58.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.623+0000 7f2bf1f9b700 1 --2- 172.21.15.97:0/857240121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bec15c520 0x7f2bec0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:54:58.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.623+0000 7f2bf179a700 1 --2- 172.21.15.97:0/857240121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bec160180 0x7f2bec1605f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009840 tx=0x7f2bdc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5caa42ec6977c383 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:58.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.623+0000 7f2bf0f99700 1 -- 172.21.15.97:0/857240121 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011830 con 0x7f2bec160180 2023-07-06T03:54:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.623+0000 7f2bf0f99700 1 -- 172.21.15.97:0/857240121 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc011990 con 0x7f2bec160180 2023-07-06T03:54:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.623+0000 7f2bf0f99700 1 -- 172.21.15.97:0/857240121 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011d20 con 0x7f2bec160180 2023-07-06T03:54:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.624+0000 7f2bf41ff700 1 -- 172.21.15.97:0/857240121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bec160180 msgr2=0x7f2bec1605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.624+0000 7f2bf41ff700 1 --2- 172.21.15.97:0/857240121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bec160180 0x7f2bec1605f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009840 tx=0x7f2bdc009b50 comp rx=0 tx=0).stop 2023-07-06T03:54:58.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.624+0000 7f2bf41ff700 1 -- 172.21.15.97:0/857240121 shutdown_connections 2023-07-06T03:54:58.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.624+0000 7f2bf41ff700 1 --2- 172.21.15.97:0/857240121 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bec15c520 0x7f2bec0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.624+0000 7f2bf41ff700 1 --2- 172.21.15.97:0/857240121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bec160180 0x7f2bec1605f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.624+0000 7f2bf41ff700 1 -- 172.21.15.97:0/857240121 >> 172.21.15.97:0/857240121 conn(0x7f2bec071790 msgr2=0x7f2bec071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.624+0000 7f2bf41ff700 1 -- 172.21.15.97:0/857240121 shutdown_connections 2023-07-06T03:54:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.624+0000 7f2bf41ff700 1 -- 172.21.15.97:0/857240121 wait complete. 2023-07-06T03:54:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.625+0000 7f2bf41ff700 1 Processor -- start 2023-07-06T03:54:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.626+0000 7f2bf41ff700 1 -- start start 2023-07-06T03:54:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf41ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec15c520 0x7f2bec15f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf41ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bec160180 0x7f2bec155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf41ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bec160b30 0x7f2bec15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf41ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec15f270 con 0x7f2bec15c520 2023-07-06T03:54:58.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf41ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec15e5f0 con 0x7f2bec160b30 2023-07-06T03:54:58.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf41ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec15e8d0 con 0x7f2bec160180 2023-07-06T03:54:58.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf1f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec15c520 0x7f2bec15f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:58.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf1f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec15c520 0x7f2bec15f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57780/0 (socket says 172.21.15.97:57780) 2023-07-06T03:54:58.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf1f9b700 1 -- 172.21.15.97:0/56406608 learned_addr learned my addr 172.21.15.97:0/56406608 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:58.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf1f9b700 1 -- 172.21.15.97:0/56406608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bec160180 msgr2=0x7f2bec155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf1f9b700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bec160180 0x7f2bec155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf1f9b700 1 -- 172.21.15.97:0/56406608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bec160b30 msgr2=0x7f2bec15dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf1f9b700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bec160b30 0x7f2bec15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf1f9b700 1 -- 172.21.15.97:0/56406608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be80094a0 con 0x7f2bec15c520 2023-07-06T03:54:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.627+0000 7f2bf1f9b700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec15c520 0x7f2bec15f690 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f2be800ebe0 tx=0x7f2be8009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.628+0000 7f2be27fc700 1 -- 172.21.15.97:0/56406608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be800c8b0 con 0x7f2bec15c520 2023-07-06T03:54:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.628+0000 7f2bf41ff700 1 -- 172.21.15.97:0/56406608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc0094a0 con 0x7f2bec15c520 2023-07-06T03:54:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.628+0000 7f2be27fc700 1 -- 172.21.15.97:0/56406608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be800ca10 con 0x7f2bec15c520 2023-07-06T03:54:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.628+0000 7f2be27fc700 1 -- 172.21.15.97:0/56406608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be800cda0 con 0x7f2bec15c520 2023-07-06T03:54:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.628+0000 7f2bf41ff700 1 -- 172.21.15.97:0/56406608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec15ee90 con 0x7f2bec15c520 2023-07-06T03:54:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.629+0000 7f2bf41ff700 1 -- 172.21.15.97:0/56406608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec15f3b0 con 0x7f2bec15c520 2023-07-06T03:54:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.630+0000 7f2be27fc700 1 -- 172.21.15.97:0/56406608 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2be8013470 con 0x7f2bec15c520 2023-07-06T03:54:58.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.631+0000 7f2be27fc700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2bd006ad10 0x7f2bd006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:58.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.631+0000 7f2be27fc700 1 -- 172.21.15.97:0/56406608 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2be8089640 con 0x7f2bec15c520 2023-07-06T03:54:58.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.631+0000 7f2bf179a700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2bd006ad10 0x7f2bd006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:58.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.632+0000 7f2bf179a700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2bd006ad10 0x7f2bd006d1c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0cd5e0 tx=0x7f2bdc010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.635+0000 7f2be27fc700 1 -- 172.21.15.97:0/56406608 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2be8054670 con 0x7f2bec15c520 2023-07-06T03:54:58.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.878+0000 7f2bf41ff700 1 -- 172.21.15.97:0/56406608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bec04e340 con 0x7f2bec15c520 2023-07-06T03:54:58.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.878+0000 7f2be27fc700 1 -- 172.21.15.97:0/56406608 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4677 (secure 0 0 0) 0x7f2be8057c90 con 0x7f2bec15c520 2023-07-06T03:54:58.882 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:58.882 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2796855133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2796855133},{"type":"v1","addr":"172.21.15.97:6835","nonce":2796855133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6851,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1092966074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1092966074},{"type":"v1","addr":"172.21.15.146:6835","nonce":1092966074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8577,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2594138359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2594138359},{"type":"v1","addr":"172.21.15.97:6837","nonce":2594138359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:54:25.441909+0000","modified":"2023-07-06T03:54:58.816193+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6834/582690410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":582690410},{"type":"v1","addr":"172.21.15.146:6834","nonce":582690410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-06T03:54:58.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 -- 172.21.15.97:0/56406608 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2bd006ad10 msgr2=0x7f2bd006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2bd006ad10 0x7f2bd006d1c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0cd5e0 tx=0x7f2bdc010000 comp rx=0 tx=0).stop 2023-07-06T03:54:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 -- 172.21.15.97:0/56406608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec15c520 msgr2=0x7f2bec15f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec15c520 0x7f2bec15f690 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f2be800ebe0 tx=0x7f2be8009e20 comp rx=0 tx=0).stop 2023-07-06T03:54:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 -- 172.21.15.97:0/56406608 shutdown_connections 2023-07-06T03:54:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2bd006ad10 0x7f2bd006d1c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2bec160b30 0x7f2bec15dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec15c520 0x7f2bec15f690 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 --2- 172.21.15.97:0/56406608 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2bec160180 0x7f2bec155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:58.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 -- 172.21.15.97:0/56406608 >> 172.21.15.97:0/56406608 conn(0x7f2bec071790 msgr2=0x7f2bec0b81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:58.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 -- 172.21.15.97:0/56406608 shutdown_connections 2023-07-06T03:54:58.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:58.881+0000 7f2bf41ff700 1 -- 172.21.15.97:0/56406608 wait complete. 2023-07-06T03:54:58.886 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 84 2023-07-06T03:54:58.902 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:54:59.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.093+0000 7f2c679ff700 1 Processor -- start 2023-07-06T03:54:59.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.093+0000 7f2c679ff700 1 -- start start 2023-07-06T03:54:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.094+0000 7f2c679ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2c600cd390 0x7f2c60160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.094+0000 7f2c679ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2c60160dd0 0x7f2c600b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.094+0000 7f2c679ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c600cdae0 con 0x7f2c6015c520 2023-07-06T03:54:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.094+0000 7f2c679ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c600b6640 con 0x7f2c600cd390 2023-07-06T03:54:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.094+0000 7f2c679ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c600b6920 con 0x7f2c60160dd0 2023-07-06T03:54:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.095+0000 7f2c64f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2c600cd390 0x7f2c60160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.095+0000 7f2c64f9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2c600cd390 0x7f2c60160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57796/0 (socket says 172.21.15.97:57796) 2023-07-06T03:54:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.095+0000 7f2c64f9a700 1 -- 172.21.15.97:0/885473437 learned_addr learned my addr 172.21.15.97:0/885473437 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:59.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.095+0000 7f2c65f9c700 1 --2- 172.21.15.97:0/885473437 >> v2:172.21.15.146:3300/0 conn(0x7f2c60160dd0 0x7f2c600b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.095+0000 7f2c64f9a700 1 -- 172.21.15.97:0/885473437 >> v2:172.21.15.146:3300/0 conn(0x7f2c60160dd0 msgr2=0x7f2c600b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:59.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.095+0000 7f2c64f9a700 1 --2- 172.21.15.97:0/885473437 >> v2:172.21.15.146:3300/0 conn(0x7f2c60160dd0 0x7f2c600b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.095+0000 7f2c64f9a700 1 -- 172.21.15.97:0/885473437 >> v1:172.21.15.146:6789/0 conn(0x7f2c6015c520 legacy=0x7f2c60160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.095+0000 7f2c64f9a700 1 -- 172.21.15.97:0/885473437 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c600b6c00 con 0x7f2c600cd390 2023-07-06T03:54:59.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.095+0000 7f2c64f9a700 1 --2- 172.21.15.97:0/885473437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c600cd390 0x7f2c60160890 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f2c600cd320 tx=0x7f2c50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0666341428e6ba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:59.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.095+0000 7f2c57fff700 1 -- 172.21.15.97:0/885473437 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50011830 con 0x7f2c600cd390 2023-07-06T03:54:59.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.096+0000 7f2c57fff700 1 -- 172.21.15.97:0/885473437 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c50011990 con 0x7f2c600cd390 2023-07-06T03:54:59.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.096+0000 7f2c57fff700 1 -- 172.21.15.97:0/885473437 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50011d20 con 0x7f2c600cd390 2023-07-06T03:54:59.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.096+0000 7f2c679ff700 1 -- 172.21.15.97:0/885473437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c600cd390 msgr2=0x7f2c60160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.096+0000 7f2c679ff700 1 --2- 172.21.15.97:0/885473437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c600cd390 0x7f2c60160890 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f2c600cd320 tx=0x7f2c50009bd0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.096+0000 7f2c679ff700 1 -- 172.21.15.97:0/885473437 shutdown_connections 2023-07-06T03:54:59.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.096+0000 7f2c679ff700 1 --2- 172.21.15.97:0/885473437 >> v2:172.21.15.146:3300/0 conn(0x7f2c60160dd0 0x7f2c600b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.096+0000 7f2c679ff700 1 --2- 172.21.15.97:0/885473437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c600cd390 0x7f2c60160890 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.096+0000 7f2c679ff700 1 -- 172.21.15.97:0/885473437 >> 172.21.15.97:0/885473437 conn(0x7f2c60071790 msgr2=0x7f2c60071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:59.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.096+0000 7f2c679ff700 1 -- 172.21.15.97:0/885473437 shutdown_connections 2023-07-06T03:54:59.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.096+0000 7f2c679ff700 1 -- 172.21.15.97:0/885473437 wait complete. 2023-07-06T03:54:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.098+0000 7f2c679ff700 1 Processor -- start 2023-07-06T03:54:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.099+0000 7f2c679ff700 1 -- start start 2023-07-06T03:54:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.099+0000 7f2c679ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c600cd390 0x7f2c600d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.099+0000 7f2c679ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6015c520 0x7f2c60158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.099+0000 7f2c679ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c60160dd0 0x7f2c60158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.099+0000 7f2c679ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600d2ab0 con 0x7f2c6015c520 2023-07-06T03:54:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.099+0000 7f2c679ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600c9380 con 0x7f2c600cd390 2023-07-06T03:54:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.099+0000 7f2c679ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600d1e30 con 0x7f2c60160dd0 2023-07-06T03:54:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.100+0000 7f2c64f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6015c520 0x7f2c60158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.100+0000 7f2c64f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6015c520 0x7f2c60158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57802/0 (socket says 172.21.15.97:57802) 2023-07-06T03:54:59.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.100+0000 7f2c64f9a700 1 -- 172.21.15.97:0/3815141884 learned_addr learned my addr 172.21.15.97:0/3815141884 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.100+0000 7f2c6579b700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c600cd390 0x7f2c600d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.100+0000 7f2c65f9c700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c60160dd0 0x7f2c60158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.100+0000 7f2c64f9a700 1 -- 172.21.15.97:0/3815141884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c60160dd0 msgr2=0x7f2c60158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.100+0000 7f2c64f9a700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c60160dd0 0x7f2c60158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.100+0000 7f2c64f9a700 1 -- 172.21.15.97:0/3815141884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c600cd390 msgr2=0x7f2c600d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.100+0000 7f2c64f9a700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c600cd390 0x7f2c600d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.100+0000 7f2c64f9a700 1 -- 172.21.15.97:0/3815141884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50009550 con 0x7f2c6015c520 2023-07-06T03:54:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.100+0000 7f2c64f9a700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6015c520 0x7f2c60158100 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009c10 tx=0x7f2c50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:59.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.101+0000 7f2c55ffb700 1 -- 172.21.15.97:0/3815141884 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c500036a0 con 0x7f2c6015c520 2023-07-06T03:54:59.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.101+0000 7f2c55ffb700 1 -- 172.21.15.97:0/3815141884 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5001e070 con 0x7f2c6015c520 2023-07-06T03:54:59.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.101+0000 7f2c679ff700 1 -- 172.21.15.97:0/3815141884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c600d2110 con 0x7f2c6015c520 2023-07-06T03:54:59.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.101+0000 7f2c55ffb700 1 -- 172.21.15.97:0/3815141884 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50003a40 con 0x7f2c6015c520 2023-07-06T03:54:59.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.101+0000 7f2c679ff700 1 -- 172.21.15.97:0/3815141884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c600d26d0 con 0x7f2c6015c520 2023-07-06T03:54:59.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.102+0000 7f2c679ff700 1 -- 172.21.15.97:0/3815141884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c60160180 con 0x7f2c6015c520 2023-07-06T03:54:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.103+0000 7f2c55ffb700 1 -- 172.21.15.97:0/3815141884 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2c50003ba0 con 0x7f2c6015c520 2023-07-06T03:54:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.104+0000 7f2c55ffb700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c4406ace0 0x7f2c4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.104+0000 7f2c55ffb700 1 -- 172.21.15.97:0/3815141884 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2c5008a5f0 con 0x7f2c6015c520 2023-07-06T03:54:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.104+0000 7f2c6579b700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c4406ace0 0x7f2c4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.105+0000 7f2c6579b700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c4406ace0 0x7f2c4406d190 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000f60 tx=0x7f2c5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.108+0000 7f2c55ffb700 1 -- 172.21.15.97:0/3815141884 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2c50055690 con 0x7f2c6015c520 2023-07-06T03:54:59.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.345+0000 7f2c679ff700 1 -- 172.21.15.97:0/3815141884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c6004e340 con 0x7f2c6015c520 2023-07-06T03:54:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.345+0000 7f2c55ffb700 1 -- 172.21.15.97:0/3815141884 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4677 (secure 0 0 0) 0x7f2c50058cb0 con 0x7f2c6015c520 2023-07-06T03:54:59.348 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:59.348 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2796855133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2796855133},{"type":"v1","addr":"172.21.15.97:6835","nonce":2796855133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6851,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1092966074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1092966074},{"type":"v1","addr":"172.21.15.146:6835","nonce":1092966074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8577,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2594138359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2594138359},{"type":"v1","addr":"172.21.15.97:6837","nonce":2594138359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:54:25.441909+0000","modified":"2023-07-06T03:54:58.816193+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6834/582690410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":582690410},{"type":"v1","addr":"172.21.15.146:6834","nonce":582690410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-07-06T03:54:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.347+0000 7f2c679ff700 1 -- 172.21.15.97:0/3815141884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c4406ace0 msgr2=0x7f2c4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.347+0000 7f2c679ff700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c4406ace0 0x7f2c4406d190 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000f60 tx=0x7f2c5c009040 comp rx=0 tx=0).stop 2023-07-06T03:54:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.347+0000 7f2c679ff700 1 -- 172.21.15.97:0/3815141884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6015c520 msgr2=0x7f2c60158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.347+0000 7f2c679ff700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6015c520 0x7f2c60158100 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009c10 tx=0x7f2c50004750 comp rx=0 tx=0).stop 2023-07-06T03:54:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.347+0000 7f2c679ff700 1 -- 172.21.15.97:0/3815141884 shutdown_connections 2023-07-06T03:54:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.347+0000 7f2c679ff700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2c4406ace0 0x7f2c4406d190 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.347+0000 7f2c679ff700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2c60160dd0 0x7f2c60158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.347+0000 7f2c679ff700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2c600cd390 0x7f2c600d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.347+0000 7f2c679ff700 1 --2- 172.21.15.97:0/3815141884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6015c520 0x7f2c60158100 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.347+0000 7f2c679ff700 1 -- 172.21.15.97:0/3815141884 >> 172.21.15.97:0/3815141884 conn(0x7f2c60071790 msgr2=0x7f2c600d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.348+0000 7f2c679ff700 1 -- 172.21.15.97:0/3815141884 shutdown_connections 2023-07-06T03:54:59.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.348+0000 7f2c679ff700 1 -- 172.21.15.97:0/3815141884 wait complete. 2023-07-06T03:54:59.352 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 84 2023-07-06T03:54:59.369 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:54:59.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.561+0000 7ff1609ff700 1 Processor -- start 2023-07-06T03:54:59.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.561+0000 7ff1609ff700 1 -- start start 2023-07-06T03:54:59.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.562+0000 7ff1609ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff15c15c520 0x7ff15c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.562+0000 7ff1609ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c160d00 0x7ff15c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.562+0000 7ff1609ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15c0cdae0 con 0x7ff15c160180 2023-07-06T03:54:59.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.562+0000 7ff1609ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15c0cd8d0 con 0x7ff15c15c520 2023-07-06T03:54:59.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.562+0000 7ff1609ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c0b6640 con 0x7ff15c160d00 2023-07-06T03:54:59.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.562+0000 7ff15ad9d700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c160d00 0x7ff15c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.562+0000 7ff153fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff15c160180 0x7ff15c1605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:55968/0 (socket says 172.21.15.97:55968) 2023-07-06T03:54:59.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.563+0000 7ff153fff700 1 -- 172.21.15.97:0/3884847648 learned_addr learned my addr 172.21.15.97:0/3884847648 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.563+0000 7ff15a59c700 1 --2- 172.21.15.97:0/3884847648 >> v2:172.21.15.146:3300/0 conn(0x7ff15c15c520 0x7ff15c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.563+0000 7ff159d9b700 1 -- 172.21.15.97:0/3884847648 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625572675 0 0) 0x7ff15c0cdae0 con 0x7ff15c160180 2023-07-06T03:54:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.563+0000 7ff15ad9d700 1 -- 172.21.15.97:0/3884847648 >> v2:172.21.15.146:3300/0 conn(0x7ff15c15c520 msgr2=0x7ff15c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.563+0000 7ff15ad9d700 1 --2- 172.21.15.97:0/3884847648 >> v2:172.21.15.146:3300/0 conn(0x7ff15c15c520 0x7ff15c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.563+0000 7ff15ad9d700 1 -- 172.21.15.97:0/3884847648 >> v1:172.21.15.146:6789/0 conn(0x7ff15c160180 legacy=0x7ff15c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.563+0000 7ff15ad9d700 1 -- 172.21.15.97:0/3884847648 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c0b6920 con 0x7ff15c160d00 2023-07-06T03:54:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.563+0000 7ff15ad9d700 1 --2- 172.21.15.97:0/3884847648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c160d00 0x7ff15c0b7400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff154009d90 tx=0x7ff15400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c046ce7e50258023 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.563+0000 7ff159d9b700 1 -- 172.21.15.97:0/3884847648 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15400e070 con 0x7ff15c160d00 2023-07-06T03:54:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.564+0000 7ff159d9b700 1 -- 172.21.15.97:0/3884847648 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15400bb20 con 0x7ff15c160d00 2023-07-06T03:54:59.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.564+0000 7ff159d9b700 1 -- 172.21.15.97:0/3884847648 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154012430 con 0x7ff15c160d00 2023-07-06T03:54:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.564+0000 7ff1609ff700 1 -- 172.21.15.97:0/3884847648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c160d00 msgr2=0x7ff15c0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.564+0000 7ff1609ff700 1 --2- 172.21.15.97:0/3884847648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c160d00 0x7ff15c0b7400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff154009d90 tx=0x7ff15400b330 comp rx=0 tx=0).stop 2023-07-06T03:54:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.564+0000 7ff1609ff700 1 -- 172.21.15.97:0/3884847648 shutdown_connections 2023-07-06T03:54:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.564+0000 7ff1609ff700 1 --2- 172.21.15.97:0/3884847648 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c160d00 0x7ff15c0b7400 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.564+0000 7ff1609ff700 1 --2- 172.21.15.97:0/3884847648 >> v2:172.21.15.146:3300/0 conn(0x7ff15c15c520 0x7ff15c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.564+0000 7ff1609ff700 1 -- 172.21.15.97:0/3884847648 >> 172.21.15.97:0/3884847648 conn(0x7ff15c071790 msgr2=0x7ff15c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.564+0000 7ff1609ff700 1 -- 172.21.15.97:0/3884847648 shutdown_connections 2023-07-06T03:54:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.564+0000 7ff1609ff700 1 -- 172.21.15.97:0/3884847648 wait complete. 2023-07-06T03:54:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.566+0000 7ff1609ff700 1 Processor -- start 2023-07-06T03:54:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.566+0000 7ff1609ff700 1 -- start start 2023-07-06T03:54:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.567+0000 7ff1609ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c15c520 0x7ff15c1f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.567+0000 7ff1609ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff15c160180 0x7ff15c0d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.567+0000 7ff1609ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff15c160d00 0x7ff15c0d3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.567+0000 7ff1609ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c1f1c80 con 0x7ff15c160d00 2023-07-06T03:54:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.567+0000 7ff1609ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c0d3a70 con 0x7ff15c160180 2023-07-06T03:54:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.567+0000 7ff1609ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c15a520 con 0x7ff15c15c520 2023-07-06T03:54:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.567+0000 7ff15ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff15c160d00 0x7ff15c0d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.567+0000 7ff15ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff15c160d00 0x7ff15c0d3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57808/0 (socket says 172.21.15.97:57808) 2023-07-06T03:54:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.567+0000 7ff15ad9d700 1 -- 172.21.15.97:0/980154016 learned_addr learned my addr 172.21.15.97:0/980154016 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:54:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.567+0000 7ff15a59c700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c15c520 0x7ff15c1f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.567+0000 7ff153fff700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff15c160180 0x7ff15c0d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.568+0000 7ff15ad9d700 1 -- 172.21.15.97:0/980154016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c15c520 msgr2=0x7ff15c1f1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.568+0000 7ff15ad9d700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c15c520 0x7ff15c1f1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.568+0000 7ff15ad9d700 1 -- 172.21.15.97:0/980154016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff15c160180 msgr2=0x7ff15c0d2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.568+0000 7ff15ad9d700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff15c160180 0x7ff15c0d2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.568+0000 7ff15ad9d700 1 -- 172.21.15.97:0/980154016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff154009a70 con 0x7ff15c160d00 2023-07-06T03:54:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.568+0000 7ff15ad9d700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff15c160d00 0x7ff15c0d3530 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0cda10 tx=0x7ff154004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.568+0000 7ff152ffd700 1 -- 172.21.15.97:0/980154016 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154012430 con 0x7ff15c160d00 2023-07-06T03:54:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.568+0000 7ff152ffd700 1 -- 172.21.15.97:0/980154016 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff154020070 con 0x7ff15c160d00 2023-07-06T03:54:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.568+0000 7ff152ffd700 1 -- 172.21.15.97:0/980154016 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154004280 con 0x7ff15c160d00 2023-07-06T03:54:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.568+0000 7ff1609ff700 1 -- 172.21.15.97:0/980154016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15c15a800 con 0x7ff15c160d00 2023-07-06T03:54:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.569+0000 7ff1609ff700 1 -- 172.21.15.97:0/980154016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15c15adc0 con 0x7ff15c160d00 2023-07-06T03:54:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.570+0000 7ff1609ff700 1 -- 172.21.15.97:0/980154016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15c04e340 con 0x7ff15c160d00 2023-07-06T03:54:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.570+0000 7ff152ffd700 1 -- 172.21.15.97:0/980154016 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1540043e0 con 0x7ff15c160d00 2023-07-06T03:54:59.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.571+0000 7ff152ffd700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff13c06ace0 0x7ff13c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.571+0000 7ff152ffd700 1 -- 172.21.15.97:0/980154016 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff154022070 con 0x7ff15c160d00 2023-07-06T03:54:59.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.572+0000 7ff15a59c700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff13c06ace0 0x7ff13c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.572+0000 7ff15a59c700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff13c06ace0 0x7ff13c06d190 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0cd280 tx=0x7ff14c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:54:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.576+0000 7ff152ffd700 1 -- 172.21.15.97:0/980154016 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff154056190 con 0x7ff15c160d00 2023-07-06T03:54:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.769+0000 7ff1609ff700 1 -- 172.21.15.97:0/980154016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff15c0b4f40 con 0x7ff15c160d00 2023-07-06T03:54:59.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.770+0000 7ff152ffd700 1 -- 172.21.15.97:0/980154016 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+18391 (secure 0 0 0) 0x7ff15402c090 con 0x7ff15c160d00 2023-07-06T03:54:59.773 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:54:59.773 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":112,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:54:26.969421+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:54:23.626811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-07-06T03:54:24.421787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:54:59.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 -- 172.21.15.97:0/980154016 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff13c06ace0 msgr2=0x7ff13c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff13c06ace0 0x7ff13c06d190 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0cd280 tx=0x7ff14c009210 comp rx=0 tx=0).stop 2023-07-06T03:54:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 -- 172.21.15.97:0/980154016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff15c160d00 msgr2=0x7ff15c0d3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff15c160d00 0x7ff15c0d3530 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0cda10 tx=0x7ff154004ad0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 -- 172.21.15.97:0/980154016 shutdown_connections 2023-07-06T03:54:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff13c06ace0 0x7ff13c06d190 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff15c160d00 0x7ff15c0d3530 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff15c15c520 0x7ff15c1f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 --2- 172.21.15.97:0/980154016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff15c160180 0x7ff15c0d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 -- 172.21.15.97:0/980154016 >> 172.21.15.97:0/980154016 conn(0x7ff15c071790 msgr2=0x7ff15c158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:54:59.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 -- 172.21.15.97:0/980154016 shutdown_connections 2023-07-06T03:54:59.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.773+0000 7ff1609ff700 1 -- 172.21.15.97:0/980154016 wait complete. 2023-07-06T03:54:59.794 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T03:54:59.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.988+0000 7fcd76fff700 1 Processor -- start 2023-07-06T03:54:59.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.989+0000 7fcd76fff700 1 -- start start 2023-07-06T03:54:59.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.990+0000 7fcd76fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcd70154890 0x7fcd700cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.990+0000 7fcd76fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd7015eb90 0x7fcd700cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.990+0000 7fcd76fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7015b490 con 0x7fcd7015a940 2023-07-06T03:54:59.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.990+0000 7fcd76fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7015b6e0 con 0x7fcd70154890 2023-07-06T03:54:59.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.990+0000 7fcd76fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7015e0b0 con 0x7fcd7015eb90 2023-07-06T03:54:59.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.990+0000 7fcd7559c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcd7015a940 0x7fcd7015ad80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33348/0 (socket says 172.21.15.97:33348) 2023-07-06T03:54:59.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.990+0000 7fcd7559c700 1 -- 172.21.15.97:0/3564815592 learned_addr learned my addr 172.21.15.97:0/3564815592 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:54:59.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.990+0000 7fcd74d9b700 1 --2- 172.21.15.97:0/3564815592 >> v2:172.21.15.146:3300/0 conn(0x7fcd70154890 0x7fcd700cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.990+0000 7fcd6ffff700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd7015eb90 0x7fcd700cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455015399 0 0) 0x7fcd7015b490 con 0x7fcd7015a940 2023-07-06T03:54:59.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd600035b0 con 0x7fcd7015a940 2023-07-06T03:54:59.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1281887967 0 0) 0x7fcd600035b0 con 0x7fcd7015a940 2023-07-06T03:54:59.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd7015eb90 msgr2=0x7fcd700cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:59.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd7015eb90 0x7fcd700cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 >> v2:172.21.15.146:3300/0 conn(0x7fcd70154890 msgr2=0x7fcd700cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:59.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 --2- 172.21.15.97:0/3564815592 >> v2:172.21.15.146:3300/0 conn(0x7fcd70154890 0x7fcd700cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7015e390 con 0x7fcd7015a940 2023-07-06T03:54:59.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fcd680019f0 con 0x7fcd7015a940 2023-07-06T03:54:59.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 >> v1:172.21.15.97:6789/0 conn(0x7fcd7015a940 legacy=0x7fcd7015ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:54:59.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd7015eb90 0x7fcd60003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd60003f80 0x7fcd60004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd60004d90 0x7fcd60005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:54:59.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7015b6e0 con 0x7fcd60003f80 2023-07-06T03:54:59.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7015e0b0 con 0x7fcd60004d90 2023-07-06T03:54:59.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6000ba10 con 0x7fcd7015eb90 2023-07-06T03:54:59.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd6ffff700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd60003f80 0x7fcd60004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.991+0000 7fcd74d9b700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd7015eb90 0x7fcd60003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.992+0000 7fcd7559c700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd60004d90 0x7fcd60005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:54:59.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.992+0000 7fcd6ffff700 1 -- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd7015eb90 msgr2=0x7fcd60003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:59.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.992+0000 7fcd6ffff700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd7015eb90 0x7fcd60003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.992+0000 7fcd6ffff700 1 -- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd60004d90 msgr2=0x7fcd60005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:54:59.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.992+0000 7fcd6ffff700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd60004d90 0x7fcd60005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:54:59.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.992+0000 7fcd6ffff700 1 -- 172.21.15.97:0/3564815592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd600035b0 con 0x7fcd60003f80 2023-07-06T03:55:00.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.992+0000 7fcd74d9b700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd7015eb90 0x7fcd60003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:00.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.992+0000 7fcd6ffff700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd60003f80 0x7fcd60004850 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009b60 tx=0x7fcd60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fd3e0827f57146 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:00.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.992+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00e6c0 con 0x7fcd60003f80 2023-07-06T03:55:00.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.992+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c00e820 con 0x7fcd60003f80 2023-07-06T03:55:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.992+0000 7fcd6f7fe700 1 -- 172.21.15.97:0/3564815592 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00eb90 con 0x7fcd60003f80 2023-07-06T03:55:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.993+0000 7fcd76fff700 1 -- 172.21.15.97:0/3564815592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd60003f80 msgr2=0x7fcd60004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.993+0000 7fcd76fff700 1 --2- 172.21.15.97:0/3564815592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd60003f80 0x7fcd60004850 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009b60 tx=0x7fcd60005880 comp rx=0 tx=0).stop 2023-07-06T03:55:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.993+0000 7fcd7559c700 1 -- 172.21.15.97:0/3564815592 reap_dead start 2023-07-06T03:55:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.993+0000 7fcd76fff700 1 -- 172.21.15.97:0/3564815592 shutdown_connections 2023-07-06T03:55:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.993+0000 7fcd76fff700 1 -- 172.21.15.97:0/3564815592 >> 172.21.15.97:0/3564815592 conn(0x7fcd7006a060 msgr2=0x7fcd7006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.993+0000 7fcd76fff700 1 -- 172.21.15.97:0/3564815592 shutdown_connections 2023-07-06T03:55:00.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.993+0000 7fcd76fff700 1 -- 172.21.15.97:0/3564815592 wait complete. 2023-07-06T03:55:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.995+0000 7fcd76fff700 1 Processor -- start 2023-07-06T03:55:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.995+0000 7fcd76fff700 1 -- start start 2023-07-06T03:55:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd76fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd60003f80 0x7fcd701ef930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd76fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd7013db10 0x7fcd700b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd76fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd700b1e10 0x7fcd700ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd76fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7013ce90 con 0x7fcd7013db10 2023-07-06T03:55:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd76fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7013d070 con 0x7fcd700b1e10 2023-07-06T03:55:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd76fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700b2700 con 0x7fcd60003f80 2023-07-06T03:55:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd7013db10 0x7fcd700b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd7013db10 0x7fcd700b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60870/0 (socket says 172.21.15.97:60870) 2023-07-06T03:55:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd6ffff700 1 -- 172.21.15.97:0/3943898310 learned_addr learned my addr 172.21.15.97:0/3943898310 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd74d9b700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd60003f80 0x7fcd701ef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:00.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd7559c700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd700b1e10 0x7fcd700ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd6ffff700 1 -- 172.21.15.97:0/3943898310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd60003f80 msgr2=0x7fcd701ef930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd6ffff700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd60003f80 0x7fcd701ef930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd6ffff700 1 -- 172.21.15.97:0/3943898310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd700b1e10 msgr2=0x7fcd700ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd6ffff700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd700b1e10 0x7fcd700ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.997+0000 7fcd6ffff700 1 -- 172.21.15.97:0/3943898310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700b2420 con 0x7fcd7013db10 2023-07-06T03:55:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.998+0000 7fcd6ffff700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd7013db10 0x7fcd700b18d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0090b0 tx=0x7fcd5c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.998+0000 7fcd6d7fa700 1 -- 172.21.15.97:0/3943898310 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c021070 con 0x7fcd7013db10 2023-07-06T03:55:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.998+0000 7fcd6d7fa700 1 -- 172.21.15.97:0/3943898310 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c004590 con 0x7fcd7013db10 2023-07-06T03:55:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.998+0000 7fcd6d7fa700 1 -- 172.21.15.97:0/3943898310 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00e6c0 con 0x7fcd7013db10 2023-07-06T03:55:00.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.998+0000 7fcd76fff700 1 -- 172.21.15.97:0/3943898310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7013d310 con 0x7fcd7013db10 2023-07-06T03:55:00.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.998+0000 7fcd76fff700 1 -- 172.21.15.97:0/3943898310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd701411c0 con 0x7fcd7013db10 2023-07-06T03:55:00.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:54:59.999+0000 7fcd76fff700 1 -- 172.21.15.97:0/3943898310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd700b21e0 con 0x7fcd7013db10 2023-07-06T03:55:00.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.000+0000 7fcd6d7fa700 1 -- 172.21.15.97:0/3943898310 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd5c025070 con 0x7fcd7013db10 2023-07-06T03:55:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.001+0000 7fcd6d7fa700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd5406ace0 0x7fcd5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.001+0000 7fcd6d7fa700 1 -- 172.21.15.97:0/3943898310 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd5c08a710 con 0x7fcd7013db10 2023-07-06T03:55:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.001+0000 7fcd74d9b700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd5406ace0 0x7fcd5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:00.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.002+0000 7fcd74d9b700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd5406ace0 0x7fcd5406d190 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009bb0 tx=0x7fcd64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:00.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.005+0000 7fcd6d7fa700 1 -- 172.21.15.97:0/3943898310 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcd700b21e0 con 0x7fcd7013db10 2023-07-06T03:55:00.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.241+0000 7fcd76fff700 1 -- 172.21.15.97:0/3943898310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcd700b21e0 con 0x7fcd7013db10 2023-07-06T03:55:00.244 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-07-06T03:55:00.241+0000 7fb463ff4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T03:55:00.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.246+0000 7fcd6d7fa700 1 -- 172.21.15.97:0/3943898310 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v85) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcd700b21e0 con 0x7fcd7013db10 2023-07-06T03:55:00.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 -- 172.21.15.97:0/3943898310 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd5406ace0 msgr2=0x7fcd5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:00.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd5406ace0 0x7fcd5406d190 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009bb0 tx=0x7fcd64008040 comp rx=0 tx=0).stop 2023-07-06T03:55:00.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 -- 172.21.15.97:0/3943898310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd7013db10 msgr2=0x7fcd700b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:00.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd7013db10 0x7fcd700b18d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0090b0 tx=0x7fcd5c0047c0 comp rx=0 tx=0).stop 2023-07-06T03:55:00.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 -- 172.21.15.97:0/3943898310 shutdown_connections 2023-07-06T03:55:00.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd5406ace0 0x7fcd5406d190 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:00.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd700b1e10 0x7fcd700ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:00.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd60003f80 0x7fcd701ef930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:00.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 --2- 172.21.15.97:0/3943898310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd7013db10 0x7fcd700b18d0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:00.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 -- 172.21.15.97:0/3943898310 >> 172.21.15.97:0/3943898310 conn(0x7fcd7006a060 msgr2=0x7fcd70159830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:00.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 -- 172.21.15.97:0/3943898310 shutdown_connections 2023-07-06T03:55:00.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.248+0000 7fcd76fff700 1 -- 172.21.15.97:0/3943898310 wait complete. 2023-07-06T03:55:00.253 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T03:55:00.270 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T03:55:00.320 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:55:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.466+0000 7ff8305ff700 1 Processor -- start 2023-07-06T03:55:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.466+0000 7ff8305ff700 1 -- start start 2023-07-06T03:55:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.468+0000 7ff8305ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff82806b360 0x7ff8280b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:00.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.468+0000 7ff8305ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8280cff90 con 0x7ff8280d2210 2023-07-06T03:55:00.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.468+0000 7ff8305ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8280b5650 con 0x7ff82815c610 2023-07-06T03:55:00.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.468+0000 7ff8305ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8280d0780 con 0x7ff82806b360 2023-07-06T03:55:00.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.468+0000 7ff82db9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff82806b360 0x7ff8280b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:00.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.469+0000 7ff82db9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff82806b360 0x7ff8280b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60872/0 (socket says 172.21.15.97:60872) 2023-07-06T03:55:00.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.468+0000 7ff82eb9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff8280d2210 0x7ff8280d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33364/0 (socket says 172.21.15.97:33364) 2023-07-06T03:55:00.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.469+0000 7ff82db9a700 1 -- 172.21.15.97:0/2663249500 learned_addr learned my addr 172.21.15.97:0/2663249500 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:00.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.468+0000 7ff82e39b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff82815c610 0x7ff8280b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:52322/0 (socket says 172.21.15.97:52322) 2023-07-06T03:55:00.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.469+0000 7ff82db9a700 1 -- 172.21.15.97:0/2663249500 >> v1:172.21.15.146:6789/0 conn(0x7ff82815c610 legacy=0x7ff8280b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:00.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.469+0000 7ff82db9a700 1 -- 172.21.15.97:0/2663249500 >> v1:172.21.15.97:6789/0 conn(0x7ff8280d2210 legacy=0x7ff8280d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.469+0000 7ff82db9a700 1 -- 172.21.15.97:0/2663249500 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8280b5400 con 0x7ff82806b360 2023-07-06T03:55:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.469+0000 7ff82d399700 1 -- 172.21.15.97:0/2663249500 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802732564 0 0) 0x7ff8280b5650 con 0x7ff82815c610 2023-07-06T03:55:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.469+0000 7ff82db9a700 1 --2- 172.21.15.97:0/2663249500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff82806b360 0x7ff8280b4ec0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ff8180098f0 tx=0x7ff818009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e151ada7ed8f9c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.469+0000 7ff82d399700 1 -- 172.21.15.97:0/2663249500 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818004230 con 0x7ff82806b360 2023-07-06T03:55:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.469+0000 7ff82d399700 1 -- 172.21.15.97:0/2663249500 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff818004390 con 0x7ff82806b360 2023-07-06T03:55:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.469+0000 7ff82d399700 1 -- 172.21.15.97:0/2663249500 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8180036a0 con 0x7ff82806b360 2023-07-06T03:55:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.470+0000 7ff8305ff700 1 -- 172.21.15.97:0/2663249500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff82806b360 msgr2=0x7ff8280b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:00.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.470+0000 7ff8305ff700 1 --2- 172.21.15.97:0/2663249500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff82806b360 0x7ff8280b4ec0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ff8180098f0 tx=0x7ff818009c00 comp rx=0 tx=0).stop 2023-07-06T03:55:00.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.470+0000 7ff8305ff700 1 -- 172.21.15.97:0/2663249500 shutdown_connections 2023-07-06T03:55:00.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.470+0000 7ff8305ff700 1 --2- 172.21.15.97:0/2663249500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff82806b360 0x7ff8280b4ec0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:00.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.470+0000 7ff8305ff700 1 -- 172.21.15.97:0/2663249500 >> 172.21.15.97:0/2663249500 conn(0x7ff82806a6c0 msgr2=0x7ff82806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:00.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.470+0000 7ff8305ff700 1 -- 172.21.15.97:0/2663249500 shutdown_connections 2023-07-06T03:55:00.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.470+0000 7ff8305ff700 1 -- 172.21.15.97:0/2663249500 wait complete. 2023-07-06T03:55:00.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.472+0000 7ff8305ff700 1 Processor -- start 2023-07-06T03:55:00.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.472+0000 7ff8305ff700 1 -- start start 2023-07-06T03:55:00.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff8305ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff82806b360 0x7ff8280c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:00.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff8305ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8280d2210 0x7ff8280c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:00.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff8305ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff82815c610 0x7ff8280b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:00.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff8305ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8280cb490 con 0x7ff8280d2210 2023-07-06T03:55:00.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff8305ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8280b6a50 con 0x7ff82806b360 2023-07-06T03:55:00.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff8305ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8280cb730 con 0x7ff82815c610 2023-07-06T03:55:00.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff82db9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8280d2210 0x7ff8280c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:00.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff82db9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8280d2210 0x7ff8280c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60888/0 (socket says 172.21.15.97:60888) 2023-07-06T03:55:00.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff82db9a700 1 -- 172.21.15.97:0/572186677 learned_addr learned my addr 172.21.15.97:0/572186677 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:00.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff82e39b700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff82806b360 0x7ff8280c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:00.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff82eb9c700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff82815c610 0x7ff8280b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:00.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff82db9a700 1 -- 172.21.15.97:0/572186677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff82815c610 msgr2=0x7ff8280b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:00.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff82db9a700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff82815c610 0x7ff8280b60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:00.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff82db9a700 1 -- 172.21.15.97:0/572186677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff82806b360 msgr2=0x7ff8280c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:00.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff82db9a700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff82806b360 0x7ff8280c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:00.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff82db9a700 1 -- 172.21.15.97:0/572186677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff818009550 con 0x7ff8280d2210 2023-07-06T03:55:00.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.473+0000 7ff82e39b700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff82806b360 0x7ff8280c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:00.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.474+0000 7ff82db9a700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8280d2210 0x7ff8280c8eb0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ff82808b120 tx=0x7ff818004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:00.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.474+0000 7ff81effd700 1 -- 172.21.15.97:0/572186677 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818011630 con 0x7ff8280d2210 2023-07-06T03:55:00.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.474+0000 7ff81effd700 1 -- 172.21.15.97:0/572186677 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff818011790 con 0x7ff8280d2210 2023-07-06T03:55:00.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.474+0000 7ff8305ff700 1 -- 172.21.15.97:0/572186677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8280cba10 con 0x7ff8280d2210 2023-07-06T03:55:00.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.474+0000 7ff81effd700 1 -- 172.21.15.97:0/572186677 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818011b40 con 0x7ff8280d2210 2023-07-06T03:55:00.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.474+0000 7ff8305ff700 1 -- 172.21.15.97:0/572186677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8280c93f0 con 0x7ff8280d2210 2023-07-06T03:55:00.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.475+0000 7ff8305ff700 1 -- 172.21.15.97:0/572186677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8280c95e0 con 0x7ff8280d2210 2023-07-06T03:55:00.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.476+0000 7ff81effd700 1 -- 172.21.15.97:0/572186677 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff818021070 con 0x7ff8280d2210 2023-07-06T03:55:00.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.477+0000 7ff81effd700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff80c06ace0 0x7ff80c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.477+0000 7ff81effd700 1 -- 172.21.15.97:0/572186677 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ff81808ade0 con 0x7ff8280d2210 2023-07-06T03:55:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.477+0000 7ff82e39b700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff80c06ace0 0x7ff80c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.478+0000 7ff82e39b700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff80c06ace0 0x7ff80c06d190 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff824000f60 tx=0x7ff824007ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:00.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.481+0000 7ff81effd700 1 -- 172.21.15.97:0/572186677 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff818055e20 con 0x7ff8280d2210 2023-07-06T03:55:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:00.723+0000 7ff8305ff700 1 -- 172.21.15.97:0/572186677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff8280cbc20 con 0x7ff8280d2210 2023-07-06T03:55:01.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.248+0000 7ff81effd700 1 -- 172.21.15.97:0/572186677 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v86) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff818059440 con 0x7ff8280d2210 2023-07-06T03:55:01.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.250+0000 7ff8305ff700 1 -- 172.21.15.97:0/572186677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff80c06ace0 msgr2=0x7ff80c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:01.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.250+0000 7ff8305ff700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff80c06ace0 0x7ff80c06d190 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff824000f60 tx=0x7ff824007ce0 comp rx=0 tx=0).stop 2023-07-06T03:55:01.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.250+0000 7ff8305ff700 1 -- 172.21.15.97:0/572186677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8280d2210 msgr2=0x7ff8280c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:01.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.250+0000 7ff8305ff700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8280d2210 0x7ff8280c8eb0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ff82808b120 tx=0x7ff818004750 comp rx=0 tx=0).stop 2023-07-06T03:55:01.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.250+0000 7ff8305ff700 1 -- 172.21.15.97:0/572186677 shutdown_connections 2023-07-06T03:55:01.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.250+0000 7ff8305ff700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff80c06ace0 0x7ff80c06d190 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:01.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.250+0000 7ff8305ff700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff82815c610 0x7ff8280b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:01.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.250+0000 7ff8305ff700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff82806b360 0x7ff8280c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:01.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.250+0000 7ff8305ff700 1 --2- 172.21.15.97:0/572186677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8280d2210 0x7ff8280c8eb0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:01.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.250+0000 7ff8305ff700 1 -- 172.21.15.97:0/572186677 >> 172.21.15.97:0/572186677 conn(0x7ff82806a6c0 msgr2=0x7ff8280d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:01.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.251+0000 7ff8305ff700 1 -- 172.21.15.97:0/572186677 shutdown_connections 2023-07-06T03:55:01.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.251+0000 7ff8305ff700 1 -- 172.21.15.97:0/572186677 wait complete. 2023-07-06T03:55:01.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T03:55:01.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.467+0000 7f3e937ff700 1 Processor -- start 2023-07-06T03:55:01.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.467+0000 7f3e937ff700 1 -- start start 2023-07-06T03:55:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.468+0000 7f3e937ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e8c0b85f0 0x7f3e8c06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.468+0000 7f3e937ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 0x7f3e8c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.468+0000 7f3e937ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8c0cd330 con 0x7f3e8c0b4ca0 2023-07-06T03:55:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.468+0000 7f3e937ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8c0b89c0 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.468+0000 7f3e937ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0b5120 con 0x7f3e8c0cefa0 2023-07-06T03:55:01.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.469+0000 7f3e90d9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 0x7f3e8c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:01.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.469+0000 7f3e9159b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e8c0b85f0 0x7f3e8c06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:01.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.469+0000 7f3e9159b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3e8c0b85f0 0x7f3e8c06b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:54366/0 (socket says 172.21.15.97:54366) 2023-07-06T03:55:01.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.469+0000 7f3e9159b700 1 -- 172.21.15.97:0/3842453843 learned_addr learned my addr 172.21.15.97:0/3842453843 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:01.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.469+0000 7f3e90d9a700 1 --2- 172.21.15.97:0/3842453843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 0x7f3e8c0b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:57956/0 (socket says 172.21.15.97:57956) 2023-07-06T03:55:01.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.469+0000 7f3e90d9a700 1 -- 172.21.15.97:0/3842453843 >> v2:172.21.15.146:3300/0 conn(0x7f3e8c0b85f0 msgr2=0x7f3e8c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:01.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.469+0000 7f3e90d9a700 1 --2- 172.21.15.97:0/3842453843 >> v2:172.21.15.146:3300/0 conn(0x7f3e8c0b85f0 0x7f3e8c06b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:01.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.469+0000 7f3e90d9a700 1 -- 172.21.15.97:0/3842453843 >> v1:172.21.15.146:6789/0 conn(0x7f3e8c0b4ca0 legacy=0x7f3e8c15ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:01.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.469+0000 7f3e90d9a700 1 -- 172.21.15.97:0/3842453843 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c0d02a0 con 0x7f3e8c0cefa0 2023-07-06T03:55:01.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.469+0000 7f3e9159b700 1 --2- 172.21.15.97:0/3842453843 >> v2:172.21.15.146:3300/0 conn(0x7f3e8c0b85f0 0x7f3e8c06b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:55:01.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.469+0000 7f3e83fff700 1 -- 172.21.15.97:0/3842453843 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989080038 0 0) 0x7f3e8c0cd330 con 0x7f3e8c0b4ca0 2023-07-06T03:55:01.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.470+0000 7f3e90d9a700 1 --2- 172.21.15.97:0/3842453843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 0x7f3e8c0b4760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009840 tx=0x7f3e7c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8798bc865acbb76 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:01.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.470+0000 7f3e83fff700 1 -- 172.21.15.97:0/3842453843 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c011830 con 0x7f3e8c0cefa0 2023-07-06T03:55:01.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.470+0000 7f3e83fff700 1 -- 172.21.15.97:0/3842453843 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c011990 con 0x7f3e8c0cefa0 2023-07-06T03:55:01.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.470+0000 7f3e83fff700 1 -- 172.21.15.97:0/3842453843 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c011d20 con 0x7f3e8c0cefa0 2023-07-06T03:55:01.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.470+0000 7f3e937ff700 1 -- 172.21.15.97:0/3842453843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 msgr2=0x7f3e8c0b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:01.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.470+0000 7f3e937ff700 1 --2- 172.21.15.97:0/3842453843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 0x7f3e8c0b4760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009840 tx=0x7f3e7c009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:01.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.471+0000 7f3e937ff700 1 -- 172.21.15.97:0/3842453843 shutdown_connections 2023-07-06T03:55:01.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.471+0000 7f3e937ff700 1 --2- 172.21.15.97:0/3842453843 >> v2:172.21.15.146:3300/0 conn(0x7f3e8c0b85f0 0x7f3e8c06b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:01.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.471+0000 7f3e937ff700 1 --2- 172.21.15.97:0/3842453843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 0x7f3e8c0b4760 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:01.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.471+0000 7f3e937ff700 1 -- 172.21.15.97:0/3842453843 >> 172.21.15.97:0/3842453843 conn(0x7f3e8c06a5b0 msgr2=0x7f3e8c06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:01.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.471+0000 7f3e937ff700 1 -- 172.21.15.97:0/3842453843 shutdown_connections 2023-07-06T03:55:01.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.471+0000 7f3e937ff700 1 -- 172.21.15.97:0/3842453843 wait complete. 2023-07-06T03:55:01.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.472+0000 7f3e937ff700 1 Processor -- start 2023-07-06T03:55:01.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.473+0000 7f3e937ff700 1 -- start start 2023-07-06T03:55:01.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.473+0000 7f3e937ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e8c0b4ca0 0x7f3e8c14b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:01.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.473+0000 7f3e937ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c0b85f0 0x7f3e8c149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:01.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.473+0000 7f3e937ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 0x7f3e8c0c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:01.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e937ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0c8a70 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e937ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0c8740 con 0x7f3e8c0b4ca0 2023-07-06T03:55:01.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e937ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c14c1e0 con 0x7f3e8c0cefa0 2023-07-06T03:55:01.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e90d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c0b85f0 0x7f3e8c149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:01.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e90d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c0b85f0 0x7f3e8c149900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60892/0 (socket says 172.21.15.97:60892) 2023-07-06T03:55:01.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e90d9a700 1 -- 172.21.15.97:0/2227235459 learned_addr learned my addr 172.21.15.97:0/2227235459 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:01.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e9159b700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e8c0b4ca0 0x7f3e8c14b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:01.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e91d9c700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 0x7f3e8c0c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:01.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e90d9a700 1 -- 172.21.15.97:0/2227235459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 msgr2=0x7f3e8c0c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:01.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e90d9a700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 0x7f3e8c0c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:01.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e90d9a700 1 -- 172.21.15.97:0/2227235459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e8c0b4ca0 msgr2=0x7f3e8c14b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:01.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e90d9a700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e8c0b4ca0 0x7f3e8c14b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:01.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e90d9a700 1 -- 172.21.15.97:0/2227235459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e88009550 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.474+0000 7f3e90d9a700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c0b85f0 0x7f3e8c149900 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009fd0 tx=0x7f3e7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:01.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.475+0000 7f3e81ffb700 1 -- 172.21.15.97:0/2227235459 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c0036a0 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.475+0000 7f3e81ffb700 1 -- 172.21.15.97:0/2227235459 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c01e070 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.475+0000 7f3e81ffb700 1 -- 172.21.15.97:0/2227235459 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c003a40 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.475+0000 7f3e937ff700 1 -- 172.21.15.97:0/2227235459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c0094a0 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.475+0000 7f3e937ff700 1 -- 172.21.15.97:0/2227235459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8c14c7a0 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.476+0000 7f3e937ff700 1 -- 172.21.15.97:0/2227235459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8c04e340 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.477+0000 7f3e81ffb700 1 -- 172.21.15.97:0/2227235459 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e7c003ba0 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.478+0000 7f3e81ffb700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e7006ace0 0x7f3e7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:01.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.478+0000 7f3e81ffb700 1 -- 172.21.15.97:0/2227235459 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f3e7c08a520 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.478+0000 7f3e9159b700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e7006ace0 0x7f3e7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:01.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.479+0000 7f3e9159b700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e7006ace0 0x7f3e7006d190 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c159980 tx=0x7f3e880092c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:01.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.483+0000 7f3e81ffb700 1 -- 172.21.15.97:0/2227235459 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3e7c055560 con 0x7f3e8c0b85f0 2023-07-06T03:55:01.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:01.676+0000 7f3e937ff700 1 -- 172.21.15.97:0/2227235459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3e8c067a10 con 0x7f3e8c0b85f0 2023-07-06T03:55:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.249+0000 7f3e81ffb700 1 -- 172.21.15.97:0/2227235459 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3e7c058b80 con 0x7f3e8c0b85f0 2023-07-06T03:55:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 -- 172.21.15.97:0/2227235459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e7006ace0 msgr2=0x7f3e7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e7006ace0 0x7f3e7006d190 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c159980 tx=0x7f3e880092c0 comp rx=0 tx=0).stop 2023-07-06T03:55:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 -- 172.21.15.97:0/2227235459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c0b85f0 msgr2=0x7f3e8c149900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c0b85f0 0x7f3e8c149900 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009fd0 tx=0x7f3e7c004750 comp rx=0 tx=0).stop 2023-07-06T03:55:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 -- 172.21.15.97:0/2227235459 shutdown_connections 2023-07-06T03:55:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3e7006ace0 0x7f3e7006d190 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3e8c0cefa0 0x7f3e8c0c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e8c0b4ca0 0x7f3e8c14b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 --2- 172.21.15.97:0/2227235459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c0b85f0 0x7f3e8c149900 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 -- 172.21.15.97:0/2227235459 >> 172.21.15.97:0/2227235459 conn(0x7f3e8c06a5b0 msgr2=0x7f3e8c06a980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 -- 172.21.15.97:0/2227235459 shutdown_connections 2023-07-06T03:55:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.252+0000 7f3e937ff700 1 -- 172.21.15.97:0/2227235459 wait complete. 2023-07-06T03:55:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-07-06T03:55:02.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T03:55:02.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.471+0000 7fd311dff700 1 Processor -- start 2023-07-06T03:55:02.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.472+0000 7fd311dff700 1 -- start start 2023-07-06T03:55:02.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.473+0000 7fd311dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd30c071030 0x7fd30c0714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:02.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.473+0000 7fd311dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30c0ae140 con 0x7fd30c15a4a0 2023-07-06T03:55:02.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.473+0000 7fd311dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30c0ce360 con 0x7fd30c06c3f0 2023-07-06T03:55:02.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.473+0000 7fd311dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30c0ce640 con 0x7fd30c071030 2023-07-06T03:55:02.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.473+0000 7fd303fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd30c071030 0x7fd30c0714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:02.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.473+0000 7fd303fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd30c071030 0x7fd30c0714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60904/0 (socket says 172.21.15.97:60904) 2023-07-06T03:55:02.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.473+0000 7fd303fff700 1 -- 172.21.15.97:0/1311412931 learned_addr learned my addr 172.21.15.97:0/1311412931 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:02.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.473+0000 7fd30b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd30c15a4a0 0x7fd30c0cd1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33368/0 (socket says 172.21.15.97:33368) 2023-07-06T03:55:02.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.474+0000 7fd303fff700 1 -- 172.21.15.97:0/1311412931 >> v1:172.21.15.146:6789/0 conn(0x7fd30c06c3f0 legacy=0x7fd30c06c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:02.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.474+0000 7fd303fff700 1 -- 172.21.15.97:0/1311412931 >> v1:172.21.15.97:6789/0 conn(0x7fd30c15a4a0 legacy=0x7fd30c0cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:02.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.474+0000 7fd303fff700 1 -- 172.21.15.97:0/1311412931 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c06b910 con 0x7fd30c071030 2023-07-06T03:55:02.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.474+0000 7fd303fff700 1 --2- 172.21.15.97:0/1311412931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c071030 0x7fd30c0714d0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009920 tx=0x7fd2f4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b56a5b03e12194 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:02.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.474+0000 7fd30affd700 1 -- 172.21.15.97:0/1311412931 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4004230 con 0x7fd30c071030 2023-07-06T03:55:02.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.474+0000 7fd30affd700 1 -- 172.21.15.97:0/1311412931 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f4004390 con 0x7fd30c071030 2023-07-06T03:55:02.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.474+0000 7fd30affd700 1 -- 172.21.15.97:0/1311412931 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4009e70 con 0x7fd30c071030 2023-07-06T03:55:02.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.475+0000 7fd311dff700 1 -- 172.21.15.97:0/1311412931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c071030 msgr2=0x7fd30c0714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:02.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.475+0000 7fd311dff700 1 --2- 172.21.15.97:0/1311412931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c071030 0x7fd30c0714d0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009920 tx=0x7fd2f4009c30 comp rx=0 tx=0).stop 2023-07-06T03:55:02.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.475+0000 7fd311dff700 1 -- 172.21.15.97:0/1311412931 shutdown_connections 2023-07-06T03:55:02.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.475+0000 7fd311dff700 1 --2- 172.21.15.97:0/1311412931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c071030 0x7fd30c0714d0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:02.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.475+0000 7fd311dff700 1 -- 172.21.15.97:0/1311412931 >> 172.21.15.97:0/1311412931 conn(0x7fd30c0b0510 msgr2=0x7fd30c0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:02.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.475+0000 7fd311dff700 1 -- 172.21.15.97:0/1311412931 shutdown_connections 2023-07-06T03:55:02.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.475+0000 7fd311dff700 1 -- 172.21.15.97:0/1311412931 wait complete. 2023-07-06T03:55:02.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.476+0000 7fd311dff700 1 Processor -- start 2023-07-06T03:55:02.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.477+0000 7fd311dff700 1 -- start start 2023-07-06T03:55:02.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.477+0000 7fd311dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c06c3f0 0x7fd30c14c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.477+0000 7fd311dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c071030 0x7fd30c0c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.477+0000 7fd311dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c15a4a0 0x7fd30c158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.477+0000 7fd311dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0c64d0 con 0x7fd30c06c3f0 2023-07-06T03:55:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.477+0000 7fd311dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c153e60 con 0x7fd30c15a4a0 2023-07-06T03:55:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.477+0000 7fd311dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0c9450 con 0x7fd30c071030 2023-07-06T03:55:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c06c3f0 0x7fd30c14c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:02.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c06c3f0 0x7fd30c14c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60916/0 (socket says 172.21.15.97:60916) 2023-07-06T03:55:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd30b7fe700 1 -- 172.21.15.97:0/3180340106 learned_addr learned my addr 172.21.15.97:0/3180340106 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd30b7fe700 1 -- 172.21.15.97:0/3180340106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c071030 msgr2=0x7fd30c0c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd30b7fe700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c071030 0x7fd30c0c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd30b7fe700 1 -- 172.21.15.97:0/3180340106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c15a4a0 msgr2=0x7fd30c158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd30bfff700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c15a4a0 0x7fd30c158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd30b7fe700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c15a4a0 0x7fd30c158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd30b7fe700 1 -- 172.21.15.97:0/3180340106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4009550 con 0x7fd30c06c3f0 2023-07-06T03:55:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd30bfff700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c15a4a0 0x7fd30c158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd30b7fe700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c06c3f0 0x7fd30c14c0b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00c430 tx=0x7fd2fc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.478+0000 7fd308ff9700 1 -- 172.21.15.97:0/3180340106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc005cb0 con 0x7fd30c06c3f0 2023-07-06T03:55:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.479+0000 7fd308ff9700 1 -- 172.21.15.97:0/3180340106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc005e10 con 0x7fd30c06c3f0 2023-07-06T03:55:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.479+0000 7fd311dff700 1 -- 172.21.15.97:0/3180340106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c0c5b30 con 0x7fd30c06c3f0 2023-07-06T03:55:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.479+0000 7fd308ff9700 1 -- 172.21.15.97:0/3180340106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0072d0 con 0x7fd30c06c3f0 2023-07-06T03:55:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.479+0000 7fd311dff700 1 -- 172.21.15.97:0/3180340106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c0c60f0 con 0x7fd30c06c3f0 2023-07-06T03:55:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.480+0000 7fd311dff700 1 -- 172.21.15.97:0/3180340106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c071990 con 0x7fd30c06c3f0 2023-07-06T03:55:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.481+0000 7fd308ff9700 1 -- 172.21.15.97:0/3180340106 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd2fc007430 con 0x7fd30c06c3f0 2023-07-06T03:55:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.481+0000 7fd308ff9700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2ec06ace0 0x7fd2ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.482+0000 7fd308ff9700 1 -- 172.21.15.97:0/3180340106 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5710+0+0 (secure 0 0 0) 0x7fd2fc088630 con 0x7fd30c06c3f0 2023-07-06T03:55:02.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.482+0000 7fd303fff700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2ec06ace0 0x7fd2ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:02.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.482+0000 7fd303fff700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2ec06ace0 0x7fd2ec06d190 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd30c06dbf0 tx=0x7fd2f4010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:02.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.486+0000 7fd308ff9700 1 -- 172.21.15.97:0/3180340106 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd2fc0538e0 con 0x7fd30c06c3f0 2023-07-06T03:55:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:02.680+0000 7fd311dff700 1 -- 172.21.15.97:0/3180340106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd30c04e340 con 0x7fd30c06c3f0 2023-07-06T03:55:03.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.251+0000 7fd308ff9700 1 -- 172.21.15.97:0/3180340106 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd2fc056f00 con 0x7fd30c06c3f0 2023-07-06T03:55:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 -- 172.21.15.97:0/3180340106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2ec06ace0 msgr2=0x7fd2ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2ec06ace0 0x7fd2ec06d190 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd30c06dbf0 tx=0x7fd2f4010000 comp rx=0 tx=0).stop 2023-07-06T03:55:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 -- 172.21.15.97:0/3180340106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c06c3f0 msgr2=0x7fd30c14c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c06c3f0 0x7fd30c14c0b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00c430 tx=0x7fd2fc00c7f0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 -- 172.21.15.97:0/3180340106 shutdown_connections 2023-07-06T03:55:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd2ec06ace0 0x7fd2ec06d190 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd30c15a4a0 0x7fd30c158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c06c3f0 0x7fd30c14c0b0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 --2- 172.21.15.97:0/3180340106 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd30c071030 0x7fd30c0c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 -- 172.21.15.97:0/3180340106 >> 172.21.15.97:0/3180340106 conn(0x7fd30c0b0510 msgr2=0x7fd30c06c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 -- 172.21.15.97:0/3180340106 shutdown_connections 2023-07-06T03:55:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.255+0000 7fd311dff700 1 -- 172.21.15.97:0/3180340106 wait complete. 2023-07-06T03:55:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-07-06T03:55:03.278 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-07-06T03:55:03.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.472+0000 7fe7b4bff700 1 Processor -- start 2023-07-06T03:55:03.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.472+0000 7fe7b4bff700 1 -- start start 2023-07-06T03:55:03.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7b4bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7b006b170 0x7fe7b015d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7b4bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7b015cae0 0x7fe7b006b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7b4bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe7b00cfbc0 0x7fe7b015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7b4bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b00b25a0 con 0x7fe7b015cae0 2023-07-06T03:55:03.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7b4bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b015de00 con 0x7fe7b00cfbc0 2023-07-06T03:55:03.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7b4bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b006ba80 con 0x7fe7b006b170 2023-07-06T03:55:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7a7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7b015cae0 0x7fe7b006b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7a7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7b015cae0 0x7fe7b006b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60918/0 (socket says 172.21.15.97:60918) 2023-07-06T03:55:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7a7fff700 1 -- 172.21.15.97:0/2978091105 learned_addr learned my addr 172.21.15.97:0/2978091105 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:03.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7ae59c700 1 --2- 172.21.15.97:0/2978091105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7b006b170 0x7fe7b015d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7aed9d700 1 --2- 172.21.15.97:0/2978091105 >> v2:172.21.15.146:3300/0 conn(0x7fe7b00cfbc0 0x7fe7b015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7a7fff700 1 -- 172.21.15.97:0/2978091105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7b006b170 msgr2=0x7fe7b015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:03.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7a7fff700 1 --2- 172.21.15.97:0/2978091105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7b006b170 0x7fe7b015d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7a7fff700 1 -- 172.21.15.97:0/2978091105 >> v2:172.21.15.146:3300/0 conn(0x7fe7b00cfbc0 msgr2=0x7fe7b015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7a7fff700 1 --2- 172.21.15.97:0/2978091105 >> v2:172.21.15.146:3300/0 conn(0x7fe7b00cfbc0 0x7fe7b015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.474+0000 7fe7a7fff700 1 -- 172.21.15.97:0/2978091105 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00ceb20 con 0x7fe7b015cae0 2023-07-06T03:55:03.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.475+0000 7fe7a7fff700 1 --2- 172.21.15.97:0/2978091105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b015cae0 0x7fe7b006b540 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00ce600 tx=0x7fe798009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc7ca128c2ba8f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:03.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.475+0000 7fe7add9b700 1 -- 172.21.15.97:0/2978091105 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79800b870 con 0x7fe7b015cae0 2023-07-06T03:55:03.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.475+0000 7fe7add9b700 1 -- 172.21.15.97:0/2978091105 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79800b9d0 con 0x7fe7b015cae0 2023-07-06T03:55:03.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.475+0000 7fe7add9b700 1 -- 172.21.15.97:0/2978091105 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79800bd60 con 0x7fe7b015cae0 2023-07-06T03:55:03.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.475+0000 7fe7b4bff700 1 -- 172.21.15.97:0/2978091105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b015cae0 msgr2=0x7fe7b006b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:03.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.475+0000 7fe7b4bff700 1 --2- 172.21.15.97:0/2978091105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b015cae0 0x7fe7b006b540 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00ce600 tx=0x7fe798009bd0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.476+0000 7fe7b4bff700 1 -- 172.21.15.97:0/2978091105 shutdown_connections 2023-07-06T03:55:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.476+0000 7fe7b4bff700 1 --2- 172.21.15.97:0/2978091105 >> v2:172.21.15.146:3300/0 conn(0x7fe7b00cfbc0 0x7fe7b015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.476+0000 7fe7b4bff700 1 --2- 172.21.15.97:0/2978091105 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7b006b170 0x7fe7b015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.476+0000 7fe7b4bff700 1 --2- 172.21.15.97:0/2978091105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b015cae0 0x7fe7b006b540 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.476+0000 7fe7b4bff700 1 -- 172.21.15.97:0/2978091105 >> 172.21.15.97:0/2978091105 conn(0x7fe7b0099f10 msgr2=0x7fe7b00d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.476+0000 7fe7b4bff700 1 -- 172.21.15.97:0/2978091105 shutdown_connections 2023-07-06T03:55:03.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.476+0000 7fe7b4bff700 1 -- 172.21.15.97:0/2978091105 wait complete. 2023-07-06T03:55:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.477+0000 7fe7b4bff700 1 Processor -- start 2023-07-06T03:55:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.477+0000 7fe7b4bff700 1 -- start start 2023-07-06T03:55:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.478+0000 7fe7b4bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7b006b170 0x7fe7b01f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.478+0000 7fe7b4bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7b00cfbc0 0x7fe7b009c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.478+0000 7fe7b4bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b015cae0 0x7fe7b009cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.478+0000 7fe7b4bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b009d1c0 con 0x7fe7b015cae0 2023-07-06T03:55:03.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.478+0000 7fe7b4bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b009d450 con 0x7fe7b006b170 2023-07-06T03:55:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.478+0000 7fe7b4bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b009d730 con 0x7fe7b00cfbc0 2023-07-06T03:55:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7aed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b015cae0 0x7fe7b009cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7aed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b015cae0 0x7fe7b009cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60928/0 (socket says 172.21.15.97:60928) 2023-07-06T03:55:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7aed9d700 1 -- 172.21.15.97:0/1402031006 learned_addr learned my addr 172.21.15.97:0/1402031006 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7ae59c700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7b006b170 0x7fe7b01f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7a7fff700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7b00cfbc0 0x7fe7b009c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7aed9d700 1 -- 172.21.15.97:0/1402031006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7b00cfbc0 msgr2=0x7fe7b009c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7aed9d700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7b00cfbc0 0x7fe7b009c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7aed9d700 1 -- 172.21.15.97:0/1402031006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7b006b170 msgr2=0x7fe7b01f1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7aed9d700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7b006b170 0x7fe7b01f1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7aed9d700 1 -- 172.21.15.97:0/1402031006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798009550 con 0x7fe7b015cae0 2023-07-06T03:55:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7aed9d700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b015cae0 0x7fe7b009cc80 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8007e40 tx=0x7fe7a800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7a6ffd700 1 -- 172.21.15.97:0/1402031006 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800ed30 con 0x7fe7b015cae0 2023-07-06T03:55:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.479+0000 7fe7a6ffd700 1 -- 172.21.15.97:0/1402031006 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a8014420 con 0x7fe7b015cae0 2023-07-06T03:55:03.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.480+0000 7fe7b4bff700 1 -- 172.21.15.97:0/1402031006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b009da10 con 0x7fe7b015cae0 2023-07-06T03:55:03.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.480+0000 7fe7a6ffd700 1 -- 172.21.15.97:0/1402031006 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80147d0 con 0x7fe7b015cae0 2023-07-06T03:55:03.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.480+0000 7fe7b4bff700 1 -- 172.21.15.97:0/1402031006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b009dfd0 con 0x7fe7b015cae0 2023-07-06T03:55:03.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.481+0000 7fe7b4bff700 1 -- 172.21.15.97:0/1402031006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b004e340 con 0x7fe7b015cae0 2023-07-06T03:55:03.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.481+0000 7fe7a6ffd700 1 -- 172.21.15.97:0/1402031006 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe7a8014930 con 0x7fe7b015cae0 2023-07-06T03:55:03.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.483+0000 7fe7a6ffd700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe79006ace0 0x7fe79006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.483+0000 7fe7a6ffd700 1 -- 172.21.15.97:0/1402031006 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe7a8089840 con 0x7fe7b015cae0 2023-07-06T03:55:03.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.483+0000 7fe7ae59c700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe79006ace0 0x7fe79006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.484+0000 7fe7ae59c700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe79006ace0 0x7fe79006d190 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006fd0 tx=0x7fe7a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:03.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.487+0000 7fe7a6ffd700 1 -- 172.21.15.97:0/1402031006 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe7a8054bf0 con 0x7fe7b015cae0 2023-07-06T03:55:03.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.682+0000 7fe7b4bff700 1 -- 172.21.15.97:0/1402031006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7fe7b00b6240 con 0x7fe7b015cae0 2023-07-06T03:55:03.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.686+0000 7fe7a6ffd700 1 -- 172.21.15.97:0/1402031006 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v49) v1 ==== 111+0+0 (secure 0 0 0) 0x7fe7a8058210 con 0x7fe7b015cae0 2023-07-06T03:55:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 -- 172.21.15.97:0/1402031006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe79006ace0 msgr2=0x7fe79006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:03.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe79006ace0 0x7fe79006d190 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006fd0 tx=0x7fe7a0008040 comp rx=0 tx=0).stop 2023-07-06T03:55:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 -- 172.21.15.97:0/1402031006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b015cae0 msgr2=0x7fe7b009cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b015cae0 0x7fe7b009cc80 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8007e40 tx=0x7fe7a800e3c0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 -- 172.21.15.97:0/1402031006 shutdown_connections 2023-07-06T03:55:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe79006ace0 0x7fe79006d190 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b015cae0 0x7fe7b009cc80 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7b006b170 0x7fe7b01f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 --2- 172.21.15.97:0/1402031006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe7b00cfbc0 0x7fe7b009c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 -- 172.21.15.97:0/1402031006 >> 172.21.15.97:0/1402031006 conn(0x7fe7b0099f10 msgr2=0x7fe7b00b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:03.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 -- 172.21.15.97:0/1402031006 shutdown_connections 2023-07-06T03:55:03.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.689+0000 7fe7b4bff700 1 -- 172.21.15.97:0/1402031006 wait complete. 2023-07-06T03:55:03.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-07-06T03:55:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.904+0000 7f1a2cdff700 1 Processor -- start 2023-07-06T03:55:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.904+0000 7f1a2cdff700 1 -- start start 2023-07-06T03:55:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.906+0000 7f1a2cdff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1a2806b360 0x7f1a280b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.906+0000 7f1a2cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d2210 0x7f1a280d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.906+0000 7f1a2cdff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a280cff90 con 0x7f1a2815c610 2023-07-06T03:55:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.906+0000 7f1a2cdff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a280d0780 con 0x7f1a2806b360 2023-07-06T03:55:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.906+0000 7f1a2cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2815c9e0 con 0x7f1a280d2210 2023-07-06T03:55:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.906+0000 7f1a2659c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1a2815c610 0x7f1a280b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33380/0 (socket says 172.21.15.97:33380) 2023-07-06T03:55:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.906+0000 7f1a2659c700 1 -- 172.21.15.97:0/23782151 learned_addr learned my addr 172.21.15.97:0/23782151 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.906+0000 7f1a25d9b700 1 --2- 172.21.15.97:0/23782151 >> v2:172.21.15.146:3300/0 conn(0x7f1a2806b360 0x7f1a280b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.906+0000 7f1a26d9d700 1 --2- 172.21.15.97:0/23782151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d2210 0x7f1a280d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.906+0000 7f1a2559a700 1 -- 172.21.15.97:0/23782151 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351847481 0 0) 0x7f1a280cff90 con 0x7f1a2815c610 2023-07-06T03:55:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.906+0000 7f1a2559a700 1 -- 172.21.15.97:0/23782151 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a100035b0 con 0x7f1a2815c610 2023-07-06T03:55:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.907+0000 7f1a25d9b700 1 -- 172.21.15.97:0/23782151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d2210 msgr2=0x7f1a280d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.907+0000 7f1a25d9b700 1 --2- 172.21.15.97:0/23782151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d2210 0x7f1a280d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.907+0000 7f1a25d9b700 1 -- 172.21.15.97:0/23782151 >> v1:172.21.15.97:6789/0 conn(0x7f1a2815c610 legacy=0x7f1a280b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.907+0000 7f1a25d9b700 1 -- 172.21.15.97:0/23782151 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a280b5650 con 0x7f1a2806b360 2023-07-06T03:55:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.907+0000 7f1a26d9d700 1 --2- 172.21.15.97:0/23782151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d2210 0x7f1a280d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:55:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.907+0000 7f1a25d9b700 1 --2- 172.21.15.97:0/23782151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a2806b360 0x7f1a280b4ec0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009840 tx=0x7f1a1c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f905dcf37bc64a75 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.907+0000 7f1a2559a700 1 -- 172.21.15.97:0/23782151 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c011830 con 0x7f1a2806b360 2023-07-06T03:55:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.907+0000 7f1a2559a700 1 -- 172.21.15.97:0/23782151 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c011990 con 0x7f1a2806b360 2023-07-06T03:55:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.907+0000 7f1a2559a700 1 -- 172.21.15.97:0/23782151 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c011d20 con 0x7f1a2806b360 2023-07-06T03:55:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.908+0000 7f1a2cdff700 1 -- 172.21.15.97:0/23782151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a2806b360 msgr2=0x7f1a280b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.908+0000 7f1a2cdff700 1 --2- 172.21.15.97:0/23782151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a2806b360 0x7f1a280b4ec0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009840 tx=0x7f1a1c009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.908+0000 7f1a2cdff700 1 -- 172.21.15.97:0/23782151 shutdown_connections 2023-07-06T03:55:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.908+0000 7f1a2cdff700 1 --2- 172.21.15.97:0/23782151 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d2210 0x7f1a280d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.908+0000 7f1a2cdff700 1 --2- 172.21.15.97:0/23782151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a2806b360 0x7f1a280b4ec0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.908+0000 7f1a2cdff700 1 -- 172.21.15.97:0/23782151 >> 172.21.15.97:0/23782151 conn(0x7f1a2806a6c0 msgr2=0x7f1a2806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.908+0000 7f1a2cdff700 1 -- 172.21.15.97:0/23782151 shutdown_connections 2023-07-06T03:55:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.908+0000 7f1a2cdff700 1 -- 172.21.15.97:0/23782151 wait complete. 2023-07-06T03:55:03.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.910+0000 7f1a2cdff700 1 Processor -- start 2023-07-06T03:55:03.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.910+0000 7f1a2cdff700 1 -- start start 2023-07-06T03:55:03.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a2cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a2806b360 0x7f1a2815f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a2cdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280d2210 0x7f1a2814a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a2cdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a2815c610 0x7f1a2814a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a2cdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2815f8e0 con 0x7f1a280d2210 2023-07-06T03:55:03.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a2cdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280cb030 con 0x7f1a2815c610 2023-07-06T03:55:03.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a2cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280cb310 con 0x7f1a2806b360 2023-07-06T03:55:03.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a25d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280d2210 0x7f1a2814a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a2659c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a2806b360 0x7f1a2815f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a25d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280d2210 0x7f1a2814a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60938/0 (socket says 172.21.15.97:60938) 2023-07-06T03:55:03.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a25d9b700 1 -- 172.21.15.97:0/146883657 learned_addr learned my addr 172.21.15.97:0/146883657 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:03.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a26d9d700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a2815c610 0x7f1a2814a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a25d9b700 1 -- 172.21.15.97:0/146883657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a2806b360 msgr2=0x7f1a2815f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a25d9b700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a2806b360 0x7f1a2815f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a25d9b700 1 -- 172.21.15.97:0/146883657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a2815c610 msgr2=0x7f1a2814a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a25d9b700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a2815c610 0x7f1a2814a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.911+0000 7f1a25d9b700 1 -- 172.21.15.97:0/146883657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a20009a70 con 0x7f1a280d2210 2023-07-06T03:55:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.912+0000 7f1a25d9b700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280d2210 0x7f1a2814a960 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0090b0 tx=0x7f1a1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.912+0000 7f1a16ffd700 1 -- 172.21.15.97:0/146883657 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c004550 con 0x7f1a280d2210 2023-07-06T03:55:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.912+0000 7f1a16ffd700 1 -- 172.21.15.97:0/146883657 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c01f070 con 0x7f1a280d2210 2023-07-06T03:55:03.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.912+0000 7f1a2cdff700 1 -- 172.21.15.97:0/146883657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1c0094a0 con 0x7f1a280d2210 2023-07-06T03:55:03.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.912+0000 7f1a16ffd700 1 -- 172.21.15.97:0/146883657 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0038e0 con 0x7f1a280d2210 2023-07-06T03:55:03.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.912+0000 7f1a2cdff700 1 -- 172.21.15.97:0/146883657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a280cb8d0 con 0x7f1a280d2210 2023-07-06T03:55:03.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.913+0000 7f1a2cdff700 1 -- 172.21.15.97:0/146883657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2804e340 con 0x7f1a280d2210 2023-07-06T03:55:03.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.914+0000 7f1a16ffd700 1 -- 172.21.15.97:0/146883657 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a1c003a40 con 0x7f1a280d2210 2023-07-06T03:55:03.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.915+0000 7f1a16ffd700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0806ace0 0x7f1a0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:03.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.915+0000 7f1a16ffd700 1 -- 172.21.15.97:0/146883657 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1a1c021070 con 0x7f1a280d2210 2023-07-06T03:55:03.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.916+0000 7f1a2659c700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0806ace0 0x7f1a0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:03.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.916+0000 7f1a2659c700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0806ace0 0x7f1a0806d190 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1a18007fd0 tx=0x7f1a18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:03.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:03.920+0000 7f1a16ffd700 1 -- 172.21.15.97:0/146883657 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1a1c055cb0 con 0x7f1a280d2210 2023-07-06T03:55:04.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.115+0000 7f1a2cdff700 1 -- 172.21.15.97:0/146883657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7f1a280cd4f0 con 0x7f1a280d2210 2023-07-06T03:55:04.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.120+0000 7f1a16ffd700 1 -- 172.21.15.97:0/146883657 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v50) v1 ==== 102+0+0 (secure 0 0 0) 0x7f1a1c0592d0 con 0x7f1a280d2210 2023-07-06T03:55:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 -- 172.21.15.97:0/146883657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0806ace0 msgr2=0x7f1a0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0806ace0 0x7f1a0806d190 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1a18007fd0 tx=0x7f1a18009040 comp rx=0 tx=0).stop 2023-07-06T03:55:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 -- 172.21.15.97:0/146883657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280d2210 msgr2=0x7f1a2814a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280d2210 0x7f1a2814a960 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0090b0 tx=0x7f1a1c004750 comp rx=0 tx=0).stop 2023-07-06T03:55:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 -- 172.21.15.97:0/146883657 shutdown_connections 2023-07-06T03:55:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0806ace0 0x7f1a0806d190 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a2815c610 0x7f1a2814a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a2806b360 0x7f1a2815f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 --2- 172.21.15.97:0/146883657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280d2210 0x7f1a2814a960 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 -- 172.21.15.97:0/146883657 >> 172.21.15.97:0/146883657 conn(0x7f1a2806a6c0 msgr2=0x7f1a280cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 -- 172.21.15.97:0/146883657 shutdown_connections 2023-07-06T03:55:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.123+0000 7f1a2cdff700 1 -- 172.21.15.97:0/146883657 wait complete. 2023-07-06T03:55:04.143 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-07-06T03:55:04.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.349+0000 7f9d45dff700 1 Processor -- start 2023-07-06T03:55:04.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.350+0000 7f9d45dff700 1 -- start start 2023-07-06T03:55:04.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.351+0000 7f9d45dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d400d2210 0x7f9d400b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.351+0000 7f9d45dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d400cff90 con 0x7f9d4015c610 2023-07-06T03:55:04.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.351+0000 7f9d45dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d400b5650 con 0x7f9d4006b360 2023-07-06T03:55:04.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.351+0000 7f9d45dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400d0780 con 0x7f9d400d2210 2023-07-06T03:55:04.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.351+0000 7f9d3f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9d4015c610 0x7f9d400b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33396/0 (socket says 172.21.15.97:33396) 2023-07-06T03:55:04.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.351+0000 7f9d3f7fe700 1 -- 172.21.15.97:0/59482199 learned_addr learned my addr 172.21.15.97:0/59482199 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.351+0000 7f9d3ffff700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d400d2210 0x7f9d400b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.351+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116708120 0 0) 0x7f9d400cff90 con 0x7f9d4015c610 2023-07-06T03:55:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.351+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d2c0035b0 con 0x7f9d4015c610 2023-07-06T03:55:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484840436 0 0) 0x7f9d400b5650 con 0x7f9d4006b360 2023-07-06T03:55:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d400cff90 con 0x7f9d4006b360 2023-07-06T03:55:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2329368075 0 0) 0x7f9d2c0035b0 con 0x7f9d4015c610 2023-07-06T03:55:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d400d2210 msgr2=0x7f9d400b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d400d2210 0x7f9d400b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 >> v1:172.21.15.146:6789/0 conn(0x7f9d4006b360 legacy=0x7f9d400d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d400b5400 con 0x7f9d4015c610 2023-07-06T03:55:04.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f9d300015f0 con 0x7f9d4015c610 2023-07-06T03:55:04.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 >> v1:172.21.15.97:6789/0 conn(0x7f9d4015c610 legacy=0x7f9d400b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d2c003bc0 0x7f9d2c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d400d2210 0x7f9d2c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 0x7f9d2c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400d0780 con 0x7f9d2c004d90 2023-07-06T03:55:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c00bac0 con 0x7f9d2c003bc0 2023-07-06T03:55:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c00bc80 con 0x7f9d400d2210 2023-07-06T03:55:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3ffff700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 0x7f9d2c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d37fff700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d400d2210 0x7f9d2c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.352+0000 7f9d3f7fe700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d2c003bc0 0x7f9d2c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d3ffff700 1 -- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d400d2210 msgr2=0x7f9d2c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d3ffff700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d400d2210 0x7f9d2c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d3ffff700 1 -- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d2c003bc0 msgr2=0x7f9d2c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d3ffff700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d2c003bc0 0x7f9d2c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d3ffff700 1 -- 172.21.15.97:0/59482199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c0035b0 con 0x7f9d2c004d90 2023-07-06T03:55:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d37fff700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d400d2210 0x7f9d2c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d3f7fe700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d2c003bc0 0x7f9d2c003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d3ffff700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 0x7f9d2c005200 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009f90 tx=0x7f9d2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2841617f180135d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800ec30 con 0x7f9d2c004d90 2023-07-06T03:55:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3800f040 con 0x7f9d2c004d90 2023-07-06T03:55:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d3effd700 1 -- 172.21.15.97:0/59482199 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38014480 con 0x7f9d2c004d90 2023-07-06T03:55:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d45dff700 1 -- 172.21.15.97:0/59482199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 msgr2=0x7f9d2c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d45dff700 1 --2- 172.21.15.97:0/59482199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 0x7f9d2c005200 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009f90 tx=0x7f9d2c005880 comp rx=0 tx=0).stop 2023-07-06T03:55:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.353+0000 7f9d3ffff700 1 -- 172.21.15.97:0/59482199 reap_dead start 2023-07-06T03:55:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.354+0000 7f9d45dff700 1 -- 172.21.15.97:0/59482199 shutdown_connections 2023-07-06T03:55:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.354+0000 7f9d45dff700 1 -- 172.21.15.97:0/59482199 >> 172.21.15.97:0/59482199 conn(0x7f9d4006a6c0 msgr2=0x7f9d4006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.354+0000 7f9d45dff700 1 -- 172.21.15.97:0/59482199 shutdown_connections 2023-07-06T03:55:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.354+0000 7f9d45dff700 1 -- 172.21.15.97:0/59482199 wait complete. 2023-07-06T03:55:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.355+0000 7f9d45dff700 1 Processor -- start 2023-07-06T03:55:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.356+0000 7f9d45dff700 1 -- start start 2023-07-06T03:55:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d45dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 0x7f9d400c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d45dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d4014d170 0x7f9d4015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d45dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d4015d0a0 0x7f9d400b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d45dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4015db30 con 0x7f9d2c004d90 2023-07-06T03:55:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d45dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400c6da0 con 0x7f9d4015d0a0 2023-07-06T03:55:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d45dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4015e770 con 0x7f9d4014d170 2023-07-06T03:55:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d3f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 0x7f9d400c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d3f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 0x7f9d400c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60956/0 (socket says 172.21.15.97:60956) 2023-07-06T03:55:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d3f7fe700 1 -- 172.21.15.97:0/2877233350 learned_addr learned my addr 172.21.15.97:0/2877233350 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d3f7fe700 1 -- 172.21.15.97:0/2877233350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d4014d170 msgr2=0x7f9d4015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d3f7fe700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d4014d170 0x7f9d4015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d3f7fe700 1 -- 172.21.15.97:0/2877233350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d4015d0a0 msgr2=0x7f9d400b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d3f7fe700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d4015d0a0 0x7f9d400b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d3f7fe700 1 -- 172.21.15.97:0/2877233350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4015d570 con 0x7f9d2c004d90 2023-07-06T03:55:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.357+0000 7f9d3f7fe700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 0x7f9d400c6860 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000e7b0 tx=0x7f9d3000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.358+0000 7f9d3cff9700 1 -- 172.21.15.97:0/2877233350 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30012590 con 0x7f9d2c004d90 2023-07-06T03:55:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.358+0000 7f9d3cff9700 1 -- 172.21.15.97:0/2877233350 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d300126f0 con 0x7f9d2c004d90 2023-07-06T03:55:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.358+0000 7f9d3cff9700 1 -- 172.21.15.97:0/2877233350 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30012a60 con 0x7f9d2c004d90 2023-07-06T03:55:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.358+0000 7f9d45dff700 1 -- 172.21.15.97:0/2877233350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4015d850 con 0x7f9d2c004d90 2023-07-06T03:55:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.358+0000 7f9d45dff700 1 -- 172.21.15.97:0/2877233350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d40150140 con 0x7f9d2c004d90 2023-07-06T03:55:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.359+0000 7f9d45dff700 1 -- 172.21.15.97:0/2877233350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d40150300 con 0x7f9d2c004d90 2023-07-06T03:55:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.360+0000 7f9d3cff9700 1 -- 172.21.15.97:0/2877233350 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d30012bc0 con 0x7f9d2c004d90 2023-07-06T03:55:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.361+0000 7f9d3cff9700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9d2006ace0 0x7f9d2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.361+0000 7f9d3cff9700 1 -- 172.21.15.97:0/2877233350 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9d30088f10 con 0x7f9d2c004d90 2023-07-06T03:55:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.361+0000 7f9d37fff700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9d2006ace0 0x7f9d2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.361+0000 7f9d37fff700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9d2006ace0 0x7f9d2006d190 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9d400b1b90 tx=0x7f9d28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:04.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.365+0000 7f9d3cff9700 1 -- 172.21.15.97:0/2877233350 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9d30054240 con 0x7f9d2c004d90 2023-07-06T03:55:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.562+0000 7f9d45dff700 1 -- 172.21.15.97:0/2877233350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7f9d4015e260 con 0x7f9d2c004d90 2023-07-06T03:55:04.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.567+0000 7f9d3cff9700 1 -- 172.21.15.97:0/2877233350 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v51) v1 ==== 107+0+0 (secure 0 0 0) 0x7f9d4015e260 con 0x7f9d2c004d90 2023-07-06T03:55:04.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 -- 172.21.15.97:0/2877233350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9d2006ace0 msgr2=0x7f9d2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9d2006ace0 0x7f9d2006d190 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9d400b1b90 tx=0x7f9d28009040 comp rx=0 tx=0).stop 2023-07-06T03:55:04.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 -- 172.21.15.97:0/2877233350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 msgr2=0x7f9d400c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 0x7f9d400c6860 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000e7b0 tx=0x7f9d3000eb70 comp rx=0 tx=0).stop 2023-07-06T03:55:04.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 -- 172.21.15.97:0/2877233350 shutdown_connections 2023-07-06T03:55:04.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9d2006ace0 0x7f9d2006d190 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d4015d0a0 0x7f9d400b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d2c004d90 0x7f9d400c6860 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 --2- 172.21.15.97:0/2877233350 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9d4014d170 0x7f9d4015cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 -- 172.21.15.97:0/2877233350 >> 172.21.15.97:0/2877233350 conn(0x7f9d4006a6c0 msgr2=0x7f9d4015ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:04.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 -- 172.21.15.97:0/2877233350 shutdown_connections 2023-07-06T03:55:04.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.569+0000 7f9d45dff700 1 -- 172.21.15.97:0/2877233350 wait complete. 2023-07-06T03:55:04.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T03:55:04.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.784+0000 7f75217ff700 1 Processor -- start 2023-07-06T03:55:04.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.784+0000 7f75217ff700 1 -- start start 2023-07-06T03:55:04.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.785+0000 7f75217ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f751c0d2210 0x7f751c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.785+0000 7f75217ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751c0cff90 con 0x7f751c06b360 2023-07-06T03:55:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.785+0000 7f75217ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751c0b5650 con 0x7f751c15c610 2023-07-06T03:55:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.785+0000 7f75217ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751c0d0780 con 0x7f751c0d2210 2023-07-06T03:55:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.785+0000 7f7513fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f751c06b360 0x7f751c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33410/0 (socket says 172.21.15.97:33410) 2023-07-06T03:55:04.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.785+0000 7f7513fff700 1 -- 172.21.15.97:0/1460741779 learned_addr learned my addr 172.21.15.97:0/1460741779 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:04.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.785+0000 7f751b7fe700 1 --2- 172.21.15.97:0/1460741779 >> v2:172.21.15.146:3300/0 conn(0x7f751c0d2210 0x7f751c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307435799 0 0) 0x7f751c0cff90 con 0x7f751c06b360 2023-07-06T03:55:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75080035b0 con 0x7f751c06b360 2023-07-06T03:55:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1775936333 0 0) 0x7f75080035b0 con 0x7f751c06b360 2023-07-06T03:55:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 >> v2:172.21.15.146:3300/0 conn(0x7f751c0d2210 msgr2=0x7f751c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 --2- 172.21.15.97:0/1460741779 >> v2:172.21.15.146:3300/0 conn(0x7f751c0d2210 0x7f751c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 >> v1:172.21.15.146:6789/0 conn(0x7f751c15c610 legacy=0x7f751c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c0b5400 con 0x7f751c06b360 2023-07-06T03:55:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f7504001670 con 0x7f751c06b360 2023-07-06T03:55:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 >> v1:172.21.15.97:6789/0 conn(0x7f751c06b360 legacy=0x7f751c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7508003a50 0x7f7508003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 0x7f7508004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7508005200 0x7f750800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c0d0780 con 0x7f7508004850 2023-07-06T03:55:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750800c0e0 con 0x7f7508005200 2023-07-06T03:55:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750800c2a0 con 0x7f7508003a50 2023-07-06T03:55:04.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f7513fff700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 0x7f7508004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.786+0000 7f751b7fe700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7508005200 0x7f750800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f751affd700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7508003a50 0x7f7508003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f7513fff700 1 -- 172.21.15.97:0/1460741779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7508003a50 msgr2=0x7f7508003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f7513fff700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7508003a50 0x7f7508003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f7513fff700 1 -- 172.21.15.97:0/1460741779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7508005200 msgr2=0x7f750800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f7513fff700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7508005200 0x7f750800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f7513fff700 1 -- 172.21.15.97:0/1460741779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75080035b0 con 0x7f7508004850 2023-07-06T03:55:04.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f751b7fe700 1 -- 172.21.15.97:0/1460741779 reap_dead start 2023-07-06T03:55:04.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f751affd700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7508003a50 0x7f7508003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:04.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f7513fff700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 0x7f7508004cc0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7504001eb0 tx=0x7f7508005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ff64562e2e6bc7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:04.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504007b70 con 0x7f7508004850 2023-07-06T03:55:04.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750400cc50 con 0x7f7508004850 2023-07-06T03:55:04.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.787+0000 7f751a7fc700 1 -- 172.21.15.97:0/1460741779 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504005960 con 0x7f7508004850 2023-07-06T03:55:04.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.788+0000 7f75217ff700 1 -- 172.21.15.97:0/1460741779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 msgr2=0x7f7508004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.788+0000 7f75217ff700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 0x7f7508004cc0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7504001eb0 tx=0x7f7508005880 comp rx=0 tx=0).stop 2023-07-06T03:55:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.788+0000 7f75217ff700 1 -- 172.21.15.97:0/1460741779 shutdown_connections 2023-07-06T03:55:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.788+0000 7f75217ff700 1 --2- 172.21.15.97:0/1460741779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 0x7f7508004cc0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.788+0000 7f75217ff700 1 -- 172.21.15.97:0/1460741779 >> 172.21.15.97:0/1460741779 conn(0x7f751c06a6c0 msgr2=0x7f751c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.788+0000 7f75217ff700 1 -- 172.21.15.97:0/1460741779 shutdown_connections 2023-07-06T03:55:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.788+0000 7f75217ff700 1 -- 172.21.15.97:0/1460741779 wait complete. 2023-07-06T03:55:04.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.790+0000 7f75217ff700 1 Processor -- start 2023-07-06T03:55:04.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.790+0000 7f75217ff700 1 -- start start 2023-07-06T03:55:04.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f75217ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 0x7f751c1f1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f75217ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f751c15e8e0 0x7f751c15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f75217ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f751c15f290 0x7f751c0b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f75217ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c15dbb0 con 0x7f7508004850 2023-07-06T03:55:04.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f75217ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c15f810 con 0x7f751c15e8e0 2023-07-06T03:55:04.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f75217ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751c15de00 con 0x7f751c15f290 2023-07-06T03:55:04.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f751affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 0x7f751c1f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f751affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 0x7f751c1f1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60974/0 (socket says 172.21.15.97:60974) 2023-07-06T03:55:04.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f751affd700 1 -- 172.21.15.97:0/2263083511 learned_addr learned my addr 172.21.15.97:0/2263083511 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:04.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f751affd700 1 -- 172.21.15.97:0/2263083511 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f751c15f290 msgr2=0x7f751c0b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f751affd700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f751c15f290 0x7f751c0b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f7513fff700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f751c15e8e0 0x7f751c15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f751affd700 1 -- 172.21.15.97:0/2263083511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f751c15e8e0 msgr2=0x7f751c15ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:04.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f751affd700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f751c15e8e0 0x7f751c15ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f751affd700 1 -- 172.21.15.97:0/2263083511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c15e0e0 con 0x7f7508004850 2023-07-06T03:55:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.791+0000 7f7513fff700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f751c15e8e0 0x7f751c15ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.792+0000 7f751affd700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 0x7f751c1f1230 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f750c00e4d0 tx=0x7f750c00e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.792+0000 7f75137fe700 1 -- 172.21.15.97:0/2263083511 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c0052e0 con 0x7f7508004850 2023-07-06T03:55:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.792+0000 7f75137fe700 1 -- 172.21.15.97:0/2263083511 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750c005460 con 0x7f7508004850 2023-07-06T03:55:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.792+0000 7f75137fe700 1 -- 172.21.15.97:0/2263083511 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c0038c0 con 0x7f7508004850 2023-07-06T03:55:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.792+0000 7f75217ff700 1 -- 172.21.15.97:0/2263083511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751c15e3c0 con 0x7f7508004850 2023-07-06T03:55:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.792+0000 7f75217ff700 1 -- 172.21.15.97:0/2263083511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751c1f1770 con 0x7f7508004850 2023-07-06T03:55:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.793+0000 7f75217ff700 1 -- 172.21.15.97:0/2263083511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751c1f1960 con 0x7f7508004850 2023-07-06T03:55:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.794+0000 7f75137fe700 1 -- 172.21.15.97:0/2263083511 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f750c003a20 con 0x7f7508004850 2023-07-06T03:55:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.795+0000 7f75137fe700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74fc06ace0 0x7f74fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.795+0000 7f75137fe700 1 -- 172.21.15.97:0/2263083511 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f750c088890 con 0x7f7508004850 2023-07-06T03:55:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.795+0000 7f7513fff700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74fc06ace0 0x7f74fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.796+0000 7f7513fff700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74fc06ace0 0x7f74fc06d190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f751c1f6160 tx=0x7f7504007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.799+0000 7f75137fe700 1 -- 172.21.15.97:0/2263083511 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f750c053c70 con 0x7f7508004850 2023-07-06T03:55:05.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:04.997+0000 7f75217ff700 1 -- 172.21.15.97:0/2263083511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f751c04e340 con 0x7f7508004850 2023-07-06T03:55:05.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.001+0000 7f75137fe700 1 -- 172.21.15.97:0/2263083511 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7f750c057290 con 0x7f7508004850 2023-07-06T03:55:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 -- 172.21.15.97:0/2263083511 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74fc06ace0 msgr2=0x7f74fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74fc06ace0 0x7f74fc06d190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f751c1f6160 tx=0x7f7504007000 comp rx=0 tx=0).stop 2023-07-06T03:55:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 -- 172.21.15.97:0/2263083511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 msgr2=0x7f751c1f1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 0x7f751c1f1230 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f750c00e4d0 tx=0x7f750c00e890 comp rx=0 tx=0).stop 2023-07-06T03:55:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 -- 172.21.15.97:0/2263083511 shutdown_connections 2023-07-06T03:55:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f74fc06ace0 0x7f74fc06d190 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f751c15f290 0x7f751c0b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7508004850 0x7f751c1f1230 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 --2- 172.21.15.97:0/2263083511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f751c15e8e0 0x7f751c15ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 -- 172.21.15.97:0/2263083511 >> 172.21.15.97:0/2263083511 conn(0x7f751c06a6c0 msgr2=0x7f751c15b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:05.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 -- 172.21.15.97:0/2263083511 shutdown_connections 2023-07-06T03:55:05.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.004+0000 7f75217ff700 1 -- 172.21.15.97:0/2263083511 wait complete. 2023-07-06T03:55:05.025 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-07-06T03:55:05.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.219+0000 7f157d7ff700 1 Processor -- start 2023-07-06T03:55:05.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.219+0000 7f157d7ff700 1 -- start start 2023-07-06T03:55:05.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.220+0000 7f157d7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f157806b360 0x7f15780b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.220+0000 7f157d7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15780cff90 con 0x7f157815c610 2023-07-06T03:55:05.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.220+0000 7f157d7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15780b5650 con 0x7f15780d2210 2023-07-06T03:55:05.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.220+0000 7f157d7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15780d0780 con 0x7f157806b360 2023-07-06T03:55:05.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.220+0000 7f1576ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f157815c610 0x7f15780b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33414/0 (socket says 172.21.15.97:33414) 2023-07-06T03:55:05.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.220+0000 7f1576ffd700 1 -- 172.21.15.97:0/1212590999 learned_addr learned my addr 172.21.15.97:0/1212590999 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784699485 0 0) 0x7f15780cff90 con 0x7f157815c610 2023-07-06T03:55:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f15767fc700 1 --2- 172.21.15.97:0/1212590999 >> v2:172.21.15.146:3300/0 conn(0x7f157806b360 0x7f15780b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15600035b0 con 0x7f157815c610 2023-07-06T03:55:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3273356933 0 0) 0x7f15600035b0 con 0x7f157815c610 2023-07-06T03:55:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 >> v2:172.21.15.146:3300/0 conn(0x7f157806b360 msgr2=0x7f15780b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 --2- 172.21.15.97:0/1212590999 >> v2:172.21.15.146:3300/0 conn(0x7f157806b360 0x7f15780b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 >> v1:172.21.15.146:6789/0 conn(0x7f15780d2210 legacy=0x7f15780d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780b5400 con 0x7f157815c610 2023-07-06T03:55:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f1568001670 con 0x7f157815c610 2023-07-06T03:55:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 >> v1:172.21.15.97:6789/0 conn(0x7f157815c610 legacy=0x7f15780b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1560003a00 0x7f1560003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1560004850 0x7f1560004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1560005200 0x7f156000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15780d0780 con 0x7f1560005200 2023-07-06T03:55:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156000c080 con 0x7f1560004850 2023-07-06T03:55:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.221+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156000c240 con 0x7f1560003a00 2023-07-06T03:55:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f15777fe700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1560005200 0x7f156000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f15767fc700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1560004850 0x7f1560004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f1576ffd700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1560003a00 0x7f1560003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f15777fe700 1 -- 172.21.15.97:0/1212590999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1560003a00 msgr2=0x7f1560003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:05.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f15777fe700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1560003a00 0x7f1560003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f15777fe700 1 -- 172.21.15.97:0/1212590999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1560004850 msgr2=0x7f1560004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:05.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f15777fe700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1560004850 0x7f1560004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f15777fe700 1 -- 172.21.15.97:0/1212590999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15600035b0 con 0x7f1560005200 2023-07-06T03:55:05.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f1576ffd700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1560003a00 0x7f1560003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:05.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f15777fe700 1 -- 172.21.15.97:0/1212590999 reap_dead start 2023-07-06T03:55:05.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f15777fe700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1560005200 0x7f156000ba10 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f157000c190 tx=0x7f1560005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ffece6fa3612b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:05.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570001db0 con 0x7f1560005200 2023-07-06T03:55:05.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157000ccf0 con 0x7f1560005200 2023-07-06T03:55:05.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.222+0000 7f1575ffb700 1 -- 172.21.15.97:0/1212590999 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570003c40 con 0x7f1560005200 2023-07-06T03:55:05.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.223+0000 7f157d7ff700 1 -- 172.21.15.97:0/1212590999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1560005200 msgr2=0x7f156000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.223+0000 7f157d7ff700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1560005200 0x7f156000ba10 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f157000c190 tx=0x7f1560005880 comp rx=0 tx=0).stop 2023-07-06T03:55:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.223+0000 7f157d7ff700 1 -- 172.21.15.97:0/1212590999 shutdown_connections 2023-07-06T03:55:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.223+0000 7f157d7ff700 1 --2- 172.21.15.97:0/1212590999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1560005200 0x7f156000ba10 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.223+0000 7f157d7ff700 1 -- 172.21.15.97:0/1212590999 >> 172.21.15.97:0/1212590999 conn(0x7f157806a6c0 msgr2=0x7f157806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.223+0000 7f157d7ff700 1 -- 172.21.15.97:0/1212590999 shutdown_connections 2023-07-06T03:55:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.223+0000 7f157d7ff700 1 -- 172.21.15.97:0/1212590999 wait complete. 2023-07-06T03:55:05.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.225+0000 7f157d7ff700 1 Processor -- start 2023-07-06T03:55:05.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.225+0000 7f157d7ff700 1 -- start start 2023-07-06T03:55:05.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f157d7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1560005200 0x7f15780c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f157d7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15780cb3e0 0x7f15780b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f157d7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1578149a80 0x7f1578149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f157d7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15781f65b0 con 0x7f15780cb3e0 2023-07-06T03:55:05.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f157d7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15780cb850 con 0x7f1578149a80 2023-07-06T03:55:05.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f157d7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15781f5930 con 0x7f1560005200 2023-07-06T03:55:05.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f15767fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15780cb3e0 0x7f15780b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f15767fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15780cb3e0 0x7f15780b6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60992/0 (socket says 172.21.15.97:60992) 2023-07-06T03:55:05.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f15767fc700 1 -- 172.21.15.97:0/2102346925 learned_addr learned my addr 172.21.15.97:0/2102346925 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:05.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f1576ffd700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1560005200 0x7f15780c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f15777fe700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1578149a80 0x7f1578149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f15767fc700 1 -- 172.21.15.97:0/2102346925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1560005200 msgr2=0x7f15780c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f15767fc700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1560005200 0x7f15780c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f15767fc700 1 -- 172.21.15.97:0/2102346925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1578149a80 msgr2=0x7f1578149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f15767fc700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1578149a80 0x7f1578149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f15767fc700 1 -- 172.21.15.97:0/2102346925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15781f5c10 con 0x7f15780cb3e0 2023-07-06T03:55:05.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.226+0000 7f1576ffd700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1560005200 0x7f15780c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:05.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.227+0000 7f15767fc700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15780cb3e0 0x7f15780b6040 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f156c0119d0 tx=0x7f156c011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:05.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.227+0000 7f155f7fe700 1 -- 172.21.15.97:0/2102346925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00f940 con 0x7f15780cb3e0 2023-07-06T03:55:05.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.227+0000 7f155f7fe700 1 -- 172.21.15.97:0/2102346925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c00faa0 con 0x7f15780cb3e0 2023-07-06T03:55:05.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.227+0000 7f155f7fe700 1 -- 172.21.15.97:0/2102346925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00fe30 con 0x7f15780cb3e0 2023-07-06T03:55:05.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.227+0000 7f157d7ff700 1 -- 172.21.15.97:0/2102346925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15781f5ef0 con 0x7f15780cb3e0 2023-07-06T03:55:05.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.227+0000 7f157d7ff700 1 -- 172.21.15.97:0/2102346925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157814a440 con 0x7f15780cb3e0 2023-07-06T03:55:05.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.228+0000 7f157d7ff700 1 -- 172.21.15.97:0/2102346925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157804e340 con 0x7f15780cb3e0 2023-07-06T03:55:05.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.229+0000 7f155f7fe700 1 -- 172.21.15.97:0/2102346925 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f156c014440 con 0x7f15780cb3e0 2023-07-06T03:55:05.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.230+0000 7f155f7fe700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f156406ace0 0x7f156406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.230+0000 7f155f7fe700 1 -- 172.21.15.97:0/2102346925 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f156c089990 con 0x7f15780cb3e0 2023-07-06T03:55:05.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.230+0000 7f1576ffd700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f156406ace0 0x7f156406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.231+0000 7f1576ffd700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f156406ace0 0x7f156406d190 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1568005b00 tx=0x7f1568009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:05.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.234+0000 7f155f7fe700 1 -- 172.21.15.97:0/2102346925 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f156c054e00 con 0x7f15780cb3e0 2023-07-06T03:55:05.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.432+0000 7f157d7ff700 1 -- 172.21.15.97:0/2102346925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7f157806b9d0 con 0x7f15780cb3e0 2023-07-06T03:55:05.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.436+0000 7f155f7fe700 1 -- 172.21.15.97:0/2102346925 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v53) v1 ==== 105+0+0 (secure 0 0 0) 0x7f156c0220b0 con 0x7f15780cb3e0 2023-07-06T03:55:05.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.438+0000 7f157d7ff700 1 -- 172.21.15.97:0/2102346925 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f156406ace0 msgr2=0x7f156406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.438+0000 7f157d7ff700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f156406ace0 0x7f156406d190 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1568005b00 tx=0x7f1568009040 comp rx=0 tx=0).stop 2023-07-06T03:55:05.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.438+0000 7f157d7ff700 1 -- 172.21.15.97:0/2102346925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15780cb3e0 msgr2=0x7f15780b6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.438+0000 7f157d7ff700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15780cb3e0 0x7f15780b6040 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f156c0119d0 tx=0x7f156c011d90 comp rx=0 tx=0).stop 2023-07-06T03:55:05.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.439+0000 7f157d7ff700 1 -- 172.21.15.97:0/2102346925 shutdown_connections 2023-07-06T03:55:05.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.439+0000 7f157d7ff700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f156406ace0 0x7f156406d190 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.439+0000 7f157d7ff700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1578149a80 0x7f1578149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.439+0000 7f157d7ff700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1560005200 0x7f15780c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.439+0000 7f157d7ff700 1 --2- 172.21.15.97:0/2102346925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15780cb3e0 0x7f15780b6040 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.439+0000 7f157d7ff700 1 -- 172.21.15.97:0/2102346925 >> 172.21.15.97:0/2102346925 conn(0x7f157806a6c0 msgr2=0x7f15781592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:05.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.439+0000 7f157d7ff700 1 -- 172.21.15.97:0/2102346925 shutdown_connections 2023-07-06T03:55:05.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.439+0000 7f157d7ff700 1 -- 172.21.15.97:0/2102346925 wait complete. 2023-07-06T03:55:05.459 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2023-07-06T03:55:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.651+0000 7f30dc7ff700 1 Processor -- start 2023-07-06T03:55:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.652+0000 7f30dc7ff700 1 -- start start 2023-07-06T03:55:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.653+0000 7f30dc7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 0x7f30d40c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.653+0000 7f30dc7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30d40c8d70 0x7f30d40d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.653+0000 7f30dc7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d40b4740 con 0x7f30d40d0ef0 2023-07-06T03:55:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.653+0000 7f30dc7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d40d1370 con 0x7f30d40c8d70 2023-07-06T03:55:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.653+0000 7f30dc7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d41602d0 con 0x7f30d415ff00 2023-07-06T03:55:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.653+0000 7f30d9d9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30d40c8d70 0x7f30d40d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.653+0000 7f30da59b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 0x7f30d40c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.653+0000 7f30da59b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 0x7f30d40c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:58078/0 (socket says 172.21.15.97:58078) 2023-07-06T03:55:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.653+0000 7f30d9d9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30d40c8d70 0x7f30d40d09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:54494/0 (socket says 172.21.15.97:54494) 2023-07-06T03:55:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.653+0000 7f30da59b700 1 -- 172.21.15.97:0/661199782 learned_addr learned my addr 172.21.15.97:0/661199782 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.653+0000 7f30dad9c700 1 --1- 172.21.15.97:0/661199782 >> v1:172.21.15.146:6789/0 conn(0x7f30d40d0ef0 0x7f30d40c6930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:52376/0 (socket says 172.21.15.97:52376) 2023-07-06T03:55:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.654+0000 7f30da59b700 1 -- 172.21.15.97:0/661199782 >> v2:172.21.15.146:3300/0 conn(0x7f30d40c8d70 msgr2=0x7f30d40d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.654+0000 7f30da59b700 1 --2- 172.21.15.97:0/661199782 >> v2:172.21.15.146:3300/0 conn(0x7f30d40c8d70 0x7f30d40d09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.654+0000 7f30da59b700 1 -- 172.21.15.97:0/661199782 >> v1:172.21.15.146:6789/0 conn(0x7f30d40d0ef0 legacy=0x7f30d40c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.654+0000 7f30da59b700 1 -- 172.21.15.97:0/661199782 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d40b60b0 con 0x7f30d415ff00 2023-07-06T03:55:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.654+0000 7f30da59b700 1 --2- 172.21.15.97:0/661199782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 0x7f30d40c8830 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f30d0006fd0 tx=0x7f30d0009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e77e347dff5b6603 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.654+0000 7f30d9599700 1 -- 172.21.15.97:0/661199782 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0005e30 con 0x7f30d415ff00 2023-07-06T03:55:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.654+0000 7f30d9599700 1 -- 172.21.15.97:0/661199782 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d0004030 con 0x7f30d415ff00 2023-07-06T03:55:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.654+0000 7f30d9599700 1 -- 172.21.15.97:0/661199782 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d00043e0 con 0x7f30d415ff00 2023-07-06T03:55:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.655+0000 7f30dc7ff700 1 -- 172.21.15.97:0/661199782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 msgr2=0x7f30d40c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.655+0000 7f30dc7ff700 1 --2- 172.21.15.97:0/661199782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 0x7f30d40c8830 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f30d0006fd0 tx=0x7f30d0009bd0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.655+0000 7f30dc7ff700 1 -- 172.21.15.97:0/661199782 shutdown_connections 2023-07-06T03:55:05.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.655+0000 7f30dc7ff700 1 --2- 172.21.15.97:0/661199782 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 0x7f30d40c8830 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.655+0000 7f30dc7ff700 1 --2- 172.21.15.97:0/661199782 >> v2:172.21.15.146:3300/0 conn(0x7f30d40c8d70 0x7f30d40d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.655+0000 7f30dc7ff700 1 -- 172.21.15.97:0/661199782 >> 172.21.15.97:0/661199782 conn(0x7f30d409c130 msgr2=0x7f30d40d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:05.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.655+0000 7f30dc7ff700 1 -- 172.21.15.97:0/661199782 shutdown_connections 2023-07-06T03:55:05.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.655+0000 7f30dc7ff700 1 -- 172.21.15.97:0/661199782 wait complete. 2023-07-06T03:55:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.657+0000 7f30dc7ff700 1 Processor -- start 2023-07-06T03:55:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.657+0000 7f30dc7ff700 1 -- start start 2023-07-06T03:55:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30dc7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30d40c8d70 0x7f30d415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30dc7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40d0ef0 0x7f30d40ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30dc7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 0x7f30d40ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30dc7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40b1660 con 0x7f30d40d0ef0 2023-07-06T03:55:05.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30dc7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40cb0d0 con 0x7f30d40c8d70 2023-07-06T03:55:05.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30dc7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40b05c0 con 0x7f30d415ff00 2023-07-06T03:55:05.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30d9d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40d0ef0 0x7f30d40ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30d9d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40d0ef0 0x7f30d40ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60994/0 (socket says 172.21.15.97:60994) 2023-07-06T03:55:05.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30d9d9a700 1 -- 172.21.15.97:0/79596720 learned_addr learned my addr 172.21.15.97:0/79596720 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:05.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30da59b700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30d40c8d70 0x7f30d415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30dad9c700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 0x7f30d40ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30d9d9a700 1 -- 172.21.15.97:0/79596720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 msgr2=0x7f30d40ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30d9d9a700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 0x7f30d40ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30d9d9a700 1 -- 172.21.15.97:0/79596720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30d40c8d70 msgr2=0x7f30d415e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:05.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30d9d9a700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30d40c8d70 0x7f30d415e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:05.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30d9d9a700 1 -- 172.21.15.97:0/79596720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d0009500 con 0x7f30d40d0ef0 2023-07-06T03:55:05.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.658+0000 7f30dad9c700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 0x7f30d40ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:05.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.659+0000 7f30d9d9a700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40d0ef0 0x7f30d40ca250 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f30c400e9b0 tx=0x7f30c400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:05.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.659+0000 7f30caffd700 1 -- 172.21.15.97:0/79596720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4009710 con 0x7f30d40d0ef0 2023-07-06T03:55:05.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.659+0000 7f30caffd700 1 -- 172.21.15.97:0/79596720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c400c850 con 0x7f30d40d0ef0 2023-07-06T03:55:05.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.659+0000 7f30dc7ff700 1 -- 172.21.15.97:0/79596720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d40b08a0 con 0x7f30d40d0ef0 2023-07-06T03:55:05.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.659+0000 7f30caffd700 1 -- 172.21.15.97:0/79596720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c400cbc0 con 0x7f30d40d0ef0 2023-07-06T03:55:05.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.659+0000 7f30dc7ff700 1 -- 172.21.15.97:0/79596720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d40b0e60 con 0x7f30d40d0ef0 2023-07-06T03:55:05.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.660+0000 7f30dc7ff700 1 -- 172.21.15.97:0/79596720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d404e340 con 0x7f30d40d0ef0 2023-07-06T03:55:05.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.661+0000 7f30caffd700 1 -- 172.21.15.97:0/79596720 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30c400cd20 con 0x7f30d40d0ef0 2023-07-06T03:55:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.662+0000 7f30caffd700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30b806ace0 0x7f30b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.662+0000 7f30caffd700 1 -- 172.21.15.97:0/79596720 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f30c4089310 con 0x7f30d40d0ef0 2023-07-06T03:55:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.663+0000 7f30da59b700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30b806ace0 0x7f30b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.663+0000 7f30da59b700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30b806ace0 0x7f30b806d190 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f30d0006fd0 tx=0x7f30d0008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.666+0000 7f30caffd700 1 -- 172.21.15.97:0/79596720 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30c40580d0 con 0x7f30d40d0ef0 2023-07-06T03:55:05.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:05.941+0000 7f30dc7ff700 1 -- 172.21.15.97:0/79596720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) v1 -- 0x7f30d408b510 con 0x7f30d40d0ef0 2023-07-06T03:55:06.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.256+0000 7f30caffd700 1 -- 172.21.15.97:0/79596720 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v510) v1 ==== 151+0+0 (secure 0 0 0) 0x7f30c4057c60 con 0x7f30d40d0ef0 2023-07-06T03:55:06.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 -- 172.21.15.97:0/79596720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30b806ace0 msgr2=0x7f30b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:06.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30b806ace0 0x7f30b806d190 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f30d0006fd0 tx=0x7f30d0008070 comp rx=0 tx=0).stop 2023-07-06T03:55:06.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 -- 172.21.15.97:0/79596720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40d0ef0 msgr2=0x7f30d40ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:06.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40d0ef0 0x7f30d40ca250 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f30c400e9b0 tx=0x7f30c400ed70 comp rx=0 tx=0).stop 2023-07-06T03:55:06.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 -- 172.21.15.97:0/79596720 shutdown_connections 2023-07-06T03:55:06.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f30b806ace0 0x7f30b806d190 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:06.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f30d415ff00 0x7f30d40ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:06.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30d40c8d70 0x7f30d415e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:06.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 --2- 172.21.15.97:0/79596720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40d0ef0 0x7f30d40ca250 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:06.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 -- 172.21.15.97:0/79596720 >> 172.21.15.97:0/79596720 conn(0x7f30d409c130 msgr2=0x7f30d41e6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:06.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 -- 172.21.15.97:0/79596720 shutdown_connections 2023-07-06T03:55:06.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.259+0000 7f30dc7ff700 1 -- 172.21.15.97:0/79596720 wait complete. 2023-07-06T03:55:06.280 INFO:tasks.cephfs_test_runner:test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-06T03:55:06.281 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:55:06.281 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2023-07-06T03:55:06.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.475+0000 7f93e7bff700 1 Processor -- start 2023-07-06T03:55:06.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.475+0000 7f93e7bff700 1 -- start start 2023-07-06T03:55:06.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.476+0000 7f93e7bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f93e006ceb0 0x7f93e015a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:06.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.476+0000 7f93e7bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e00c97c0 con 0x7f93e015d990 2023-07-06T03:55:06.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.476+0000 7f93e7bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e015ceb0 con 0x7f93e0159390 2023-07-06T03:55:06.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.476+0000 7f93e7bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e0159760 con 0x7f93e006ceb0 2023-07-06T03:55:06.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.476+0000 7f93e519a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f93e015d990 0x7f93e006c7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33420/0 (socket says 172.21.15.97:33420) 2023-07-06T03:55:06.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.476+0000 7f93e519a700 1 -- 172.21.15.97:0/839922381 learned_addr learned my addr 172.21.15.97:0/839922381 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e619c700 1 --2- 172.21.15.97:0/839922381 >> v2:172.21.15.146:3300/0 conn(0x7f93e006ceb0 0x7f93e015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951249403 0 0) 0x7f93e00c97c0 con 0x7f93e015d990 2023-07-06T03:55:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93cc0035b0 con 0x7f93e015d990 2023-07-06T03:55:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2037830951 0 0) 0x7f93cc0035b0 con 0x7f93e015d990 2023-07-06T03:55:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 >> v2:172.21.15.146:3300/0 conn(0x7f93e006ceb0 msgr2=0x7f93e015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 --2- 172.21.15.97:0/839922381 >> v2:172.21.15.146:3300/0 conn(0x7f93e006ceb0 0x7f93e015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 >> v1:172.21.15.146:6789/0 conn(0x7f93e0159390 legacy=0x7f93e00c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e015d190 con 0x7f93e015d990 2023-07-06T03:55:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f93d0001670 con 0x7f93e015d990 2023-07-06T03:55:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e619c700 1 --2- 172.21.15.97:0/839922381 >> v2:172.21.15.146:3300/0 conn(0x7f93e006ceb0 0x7f93e015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:55:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 >> v1:172.21.15.97:6789/0 conn(0x7f93e015d990 legacy=0x7f93e006c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 --2- 172.21.15.97:0/839922381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93cc003a00 0x7f93cc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 --2- 172.21.15.97:0/839922381 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93cc004850 0x7f93cc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 --2- 172.21.15.97:0/839922381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93cc005200 0x7f93cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc00c050 con 0x7f93cc005200 2023-07-06T03:55:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc00c1c0 con 0x7f93cc003a00 2023-07-06T03:55:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.477+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc00c380 con 0x7f93cc004850 2023-07-06T03:55:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.478+0000 7f93e619c700 1 --2- 172.21.15.97:0/839922381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93cc005200 0x7f93cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.478+0000 7f93e519a700 1 --2- 172.21.15.97:0/839922381 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93cc004850 0x7f93cc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.478+0000 7f93e599b700 1 --2- 172.21.15.97:0/839922381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93cc003a00 0x7f93cc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.478+0000 7f93e619c700 1 -- 172.21.15.97:0/839922381 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93cc004850 msgr2=0x7f93cc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.478+0000 7f93e619c700 1 --2- 172.21.15.97:0/839922381 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93cc004850 0x7f93cc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:06.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.478+0000 7f93e619c700 1 -- 172.21.15.97:0/839922381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93cc003a00 msgr2=0x7f93cc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:06.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.478+0000 7f93e619c700 1 --2- 172.21.15.97:0/839922381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93cc003a00 0x7f93cc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:06.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.478+0000 7f93e619c700 1 -- 172.21.15.97:0/839922381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc0035b0 con 0x7f93cc005200 2023-07-06T03:55:06.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.478+0000 7f93e619c700 1 -- 172.21.15.97:0/839922381 reap_dead start 2023-07-06T03:55:06.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.478+0000 7f93e619c700 1 --2- 172.21.15.97:0/839922381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93cc005200 0x7f93cc00ba10 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009da0 tx=0x7f93cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4cefd74afaf954 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:06.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.478+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d800ed90 con 0x7f93cc005200 2023-07-06T03:55:06.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.479+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d800f040 con 0x7f93cc005200 2023-07-06T03:55:06.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.479+0000 7f93e4999700 1 -- 172.21.15.97:0/839922381 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8012670 con 0x7f93cc005200 2023-07-06T03:55:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.479+0000 7f93e7bff700 1 -- 172.21.15.97:0/839922381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93cc005200 msgr2=0x7f93cc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.479+0000 7f93e7bff700 1 --2- 172.21.15.97:0/839922381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93cc005200 0x7f93cc00ba10 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009da0 tx=0x7f93cc005880 comp rx=0 tx=0).stop 2023-07-06T03:55:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.479+0000 7f93e7bff700 1 -- 172.21.15.97:0/839922381 shutdown_connections 2023-07-06T03:55:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.479+0000 7f93e7bff700 1 --2- 172.21.15.97:0/839922381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93cc005200 0x7f93cc00ba10 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.479+0000 7f93e7bff700 1 -- 172.21.15.97:0/839922381 >> 172.21.15.97:0/839922381 conn(0x7f93e00b04a0 msgr2=0x7f93e00b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.479+0000 7f93e7bff700 1 -- 172.21.15.97:0/839922381 shutdown_connections 2023-07-06T03:55:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.479+0000 7f93e7bff700 1 -- 172.21.15.97:0/839922381 wait complete. 2023-07-06T03:55:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.481+0000 7f93e7bff700 1 Processor -- start 2023-07-06T03:55:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.481+0000 7f93e7bff700 1 -- start start 2023-07-06T03:55:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.482+0000 7f93e7bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93cc005200 0x7f93e0147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.482+0000 7f93e7bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e01f4f70 0x7f93e015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.482+0000 7f93e7bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93e01f53e0 0x7f93e015ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.482+0000 7f93e7bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e01f5ef0 con 0x7f93e01f4f70 2023-07-06T03:55:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.482+0000 7f93e7bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e0147bc0 con 0x7f93cc005200 2023-07-06T03:55:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.482+0000 7f93e7bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e01f5930 con 0x7f93e01f53e0 2023-07-06T03:55:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.482+0000 7f93e519a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e01f4f70 0x7f93e015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.482+0000 7f93e519a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e01f4f70 0x7f93e015b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32782/0 (socket says 172.21.15.97:32782) 2023-07-06T03:55:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.482+0000 7f93e519a700 1 -- 172.21.15.97:0/102023714 learned_addr learned my addr 172.21.15.97:0/102023714 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93e619c700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93e01f53e0 0x7f93e015ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93e599b700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93cc005200 0x7f93e0147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93e519a700 1 -- 172.21.15.97:0/102023714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93e01f53e0 msgr2=0x7f93e015ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93e519a700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93e01f53e0 0x7f93e015ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93e519a700 1 -- 172.21.15.97:0/102023714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93cc005200 msgr2=0x7f93e0147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93e519a700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93cc005200 0x7f93e0147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93e519a700 1 -- 172.21.15.97:0/102023714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e01f5c10 con 0x7f93e01f4f70 2023-07-06T03:55:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93e619c700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93e01f53e0 0x7f93e015ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93e519a700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e01f4f70 0x7f93e015b4d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f93d00072c0 tx=0x7f93d00075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93d67fc700 1 -- 172.21.15.97:0/102023714 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d000f640 con 0x7f93e01f4f70 2023-07-06T03:55:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93d67fc700 1 -- 172.21.15.97:0/102023714 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d000f7e0 con 0x7f93e01f4f70 2023-07-06T03:55:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93e7bff700 1 -- 172.21.15.97:0/102023714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e01477e0 con 0x7f93e01f4f70 2023-07-06T03:55:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93d67fc700 1 -- 172.21.15.97:0/102023714 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d000fb90 con 0x7f93e01f4f70 2023-07-06T03:55:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.483+0000 7f93e7bff700 1 -- 172.21.15.97:0/102023714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e014b7a0 con 0x7f93e01f4f70 2023-07-06T03:55:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.484+0000 7f93e7bff700 1 -- 172.21.15.97:0/102023714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e0147640 con 0x7f93e01f4f70 2023-07-06T03:55:06.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.485+0000 7f93d67fc700 1 -- 172.21.15.97:0/102023714 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f93d000fcf0 con 0x7f93e01f4f70 2023-07-06T03:55:06.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.486+0000 7f93d67fc700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f93c406ace0 0x7f93c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:06.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.486+0000 7f93d67fc700 1 -- 172.21.15.97:0/102023714 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f93d00890d0 con 0x7f93e01f4f70 2023-07-06T03:55:06.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.487+0000 7f93e599b700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f93c406ace0 0x7f93c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:06.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.487+0000 7f93e599b700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f93c406ace0 0x7f93c406d190 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f93e00b2010 tx=0x7f93dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:06.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.490+0000 7f93d67fc700 1 -- 172.21.15.97:0/102023714 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f93e0147640 con 0x7f93e01f4f70 2023-07-06T03:55:06.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:06.763+0000 7f93e7bff700 1 -- 172.21.15.97:0/102023714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) v1 -- 0x7f93e0147640 con 0x7f93e01f4f70 2023-07-06T03:55:06.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:55:06.834 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:55:07.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.261+0000 7f93d67fc700 1 -- 172.21.15.97:0/102023714 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v511) v1 ==== 151+0+0 (secure 0 0 0) 0x7f93e0147640 con 0x7f93e01f4f70 2023-07-06T03:55:07.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.263+0000 7f93e7bff700 1 -- 172.21.15.97:0/102023714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f93c406ace0 msgr2=0x7f93c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.263+0000 7f93e7bff700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f93c406ace0 0x7f93c406d190 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f93e00b2010 tx=0x7f93dc009040 comp rx=0 tx=0).stop 2023-07-06T03:55:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.263+0000 7f93e7bff700 1 -- 172.21.15.97:0/102023714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e01f4f70 msgr2=0x7f93e015b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.263+0000 7f93e7bff700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e01f4f70 0x7f93e015b4d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f93d00072c0 tx=0x7f93d00075d0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.264+0000 7f93e7bff700 1 -- 172.21.15.97:0/102023714 shutdown_connections 2023-07-06T03:55:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.264+0000 7f93e7bff700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f93c406ace0 0x7f93c406d190 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.264+0000 7f93e7bff700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f93e01f53e0 0x7f93e015ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.264+0000 7f93e7bff700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93cc005200 0x7f93e0147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.264+0000 7f93e7bff700 1 --2- 172.21.15.97:0/102023714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e01f4f70 0x7f93e015b4d0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.264+0000 7f93e7bff700 1 -- 172.21.15.97:0/102023714 >> 172.21.15.97:0/102023714 conn(0x7f93e00b04a0 msgr2=0x7f93e0159760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:07.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.264+0000 7f93e7bff700 1 -- 172.21.15.97:0/102023714 shutdown_connections 2023-07-06T03:55:07.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.264+0000 7f93e7bff700 1 -- 172.21.15.97:0/102023714 wait complete. 2023-07-06T03:55:07.284 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T03:55:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.480+0000 7f803cdff700 1 Processor -- start 2023-07-06T03:55:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.481+0000 7f803cdff700 1 -- start start 2023-07-06T03:55:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.482+0000 7f803cdff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f80380b4a40 0x7f80380d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.482+0000 7f803cdff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80380b5e10 0x7f80380cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.482+0000 7f803cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80380b6280 0x7f80380cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.482+0000 7f803cdff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380d0410 con 0x7f80380b5e10 2023-07-06T03:55:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.482+0000 7f803cdff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380d2d30 con 0x7f80380b4a40 2023-07-06T03:55:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.482+0000 7f803cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80380cf210 con 0x7f80380b6280 2023-07-06T03:55:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f802ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80380b5e10 0x7f80380cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f802ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80380b5e10 0x7f80380cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32798/0 (socket says 172.21.15.97:32798) 2023-07-06T03:55:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f802ffff700 1 -- 172.21.15.97:0/2496870003 learned_addr learned my addr 172.21.15.97:0/2496870003 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f803659c700 1 --2- 172.21.15.97:0/2496870003 >> v2:172.21.15.146:3300/0 conn(0x7f80380b4a40 0x7f80380d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f8036d9d700 1 --2- 172.21.15.97:0/2496870003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80380b6280 0x7f80380cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f802ffff700 1 -- 172.21.15.97:0/2496870003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80380b6280 msgr2=0x7f80380cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f802ffff700 1 --2- 172.21.15.97:0/2496870003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80380b6280 0x7f80380cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f802ffff700 1 -- 172.21.15.97:0/2496870003 >> v2:172.21.15.146:3300/0 conn(0x7f80380b4a40 msgr2=0x7f80380d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f802ffff700 1 --2- 172.21.15.97:0/2496870003 >> v2:172.21.15.146:3300/0 conn(0x7f80380b4a40 0x7f80380d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f802ffff700 1 -- 172.21.15.97:0/2496870003 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380cf4b0 con 0x7f80380b5e10 2023-07-06T03:55:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f803659c700 1 --2- 172.21.15.97:0/2496870003 >> v2:172.21.15.146:3300/0 conn(0x7f80380b4a40 0x7f80380d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f8036d9d700 1 --2- 172.21.15.97:0/2496870003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80380b6280 0x7f80380cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f802ffff700 1 --2- 172.21.15.97:0/2496870003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80380b5e10 0x7f80380cf990 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f803806b960 tx=0x7f8020009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93068253d4950aa3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:07.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f8035d9b700 1 -- 172.21.15.97:0/2496870003 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802000b870 con 0x7f80380b5e10 2023-07-06T03:55:07.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f8035d9b700 1 -- 172.21.15.97:0/2496870003 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802000b9d0 con 0x7f80380b5e10 2023-07-06T03:55:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.483+0000 7f8035d9b700 1 -- 172.21.15.97:0/2496870003 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802000bd60 con 0x7f80380b5e10 2023-07-06T03:55:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.484+0000 7f803cdff700 1 -- 172.21.15.97:0/2496870003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80380b5e10 msgr2=0x7f80380cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.484+0000 7f803cdff700 1 --2- 172.21.15.97:0/2496870003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80380b5e10 0x7f80380cf990 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f803806b960 tx=0x7f8020009bd0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.484+0000 7f803cdff700 1 -- 172.21.15.97:0/2496870003 shutdown_connections 2023-07-06T03:55:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.484+0000 7f803cdff700 1 --2- 172.21.15.97:0/2496870003 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80380b6280 0x7f80380cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.484+0000 7f803cdff700 1 --2- 172.21.15.97:0/2496870003 >> v2:172.21.15.146:3300/0 conn(0x7f80380b4a40 0x7f80380d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.484+0000 7f803cdff700 1 --2- 172.21.15.97:0/2496870003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80380b5e10 0x7f80380cf990 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.484+0000 7f803cdff700 1 -- 172.21.15.97:0/2496870003 >> 172.21.15.97:0/2496870003 conn(0x7f803806a900 msgr2=0x7f803806acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.484+0000 7f803cdff700 1 -- 172.21.15.97:0/2496870003 shutdown_connections 2023-07-06T03:55:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.484+0000 7f803cdff700 1 -- 172.21.15.97:0/2496870003 wait complete. 2023-07-06T03:55:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.486+0000 7f803cdff700 1 Processor -- start 2023-07-06T03:55:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.487+0000 7f803cdff700 1 -- start start 2023-07-06T03:55:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803cdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80380b4a40 0x7f803815e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80380b5e10 0x7f80380c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803cdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80380b6280 0x7f803815df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803cdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80381f3230 con 0x7f80380b4a40 2023-07-06T03:55:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803cdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803815e460 con 0x7f80380b6280 2023-07-06T03:55:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80381f25b0 con 0x7f80380b5e10 2023-07-06T03:55:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80380b4a40 0x7f803815e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80380b4a40 0x7f803815e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32806/0 (socket says 172.21.15.97:32806) 2023-07-06T03:55:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803659c700 1 -- 172.21.15.97:0/3176173953 learned_addr learned my addr 172.21.15.97:0/3176173953 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f8036d9d700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80380b6280 0x7f803815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f802ffff700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80380b5e10 0x7f80380c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803659c700 1 -- 172.21.15.97:0/3176173953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80380b5e10 msgr2=0x7f80380c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803659c700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80380b5e10 0x7f80380c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.488+0000 7f803659c700 1 -- 172.21.15.97:0/3176173953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80380b6280 msgr2=0x7f803815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.489+0000 7f803659c700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80380b6280 0x7f803815df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.489+0000 7f803659c700 1 -- 172.21.15.97:0/3176173953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8020009550 con 0x7f80380b4a40 2023-07-06T03:55:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.489+0000 7f803659c700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80380b4a40 0x7f803815e8b0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f802800c7f0 tx=0x7f802800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.489+0000 7f802effd700 1 -- 172.21.15.97:0/3176173953 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028011800 con 0x7f80380b4a40 2023-07-06T03:55:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.489+0000 7f802effd700 1 -- 172.21.15.97:0/3176173953 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8028011960 con 0x7f80380b4a40 2023-07-06T03:55:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.489+0000 7f803cdff700 1 -- 172.21.15.97:0/3176173953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80381f2890 con 0x7f80380b4a40 2023-07-06T03:55:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.489+0000 7f802effd700 1 -- 172.21.15.97:0/3176173953 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028011d10 con 0x7f80380b4a40 2023-07-06T03:55:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.489+0000 7f803cdff700 1 -- 172.21.15.97:0/3176173953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80381f2e50 con 0x7f80380b4a40 2023-07-06T03:55:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.490+0000 7f803cdff700 1 -- 172.21.15.97:0/3176173953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803804e340 con 0x7f80380b4a40 2023-07-06T03:55:07.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.491+0000 7f802effd700 1 -- 172.21.15.97:0/3176173953 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8028018360 con 0x7f80380b4a40 2023-07-06T03:55:07.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.492+0000 7f802effd700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f801806ace0 0x7f801806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:07.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.492+0000 7f802effd700 1 -- 172.21.15.97:0/3176173953 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8028089280 con 0x7f80380b4a40 2023-07-06T03:55:07.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.492+0000 7f802ffff700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f801806ace0 0x7f801806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:07.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.493+0000 7f802ffff700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f801806ace0 0x7f801806d190 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8020014040 tx=0x7f8020008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.496+0000 7f802effd700 1 -- 172.21.15.97:0/3176173953 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8028054520 con 0x7f80380b4a40 2023-07-06T03:55:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.693+0000 7f803cdff700 1 -- 172.21.15.97:0/3176173953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8038067210 con 0x7f80380b4a40 2023-07-06T03:55:07.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.697+0000 7f802effd700 1 -- 172.21.15.97:0/3176173953 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8028057b40 con 0x7f80380b4a40 2023-07-06T03:55:07.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 -- 172.21.15.97:0/3176173953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f801806ace0 msgr2=0x7f801806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f801806ace0 0x7f801806d190 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8020014040 tx=0x7f8020008030 comp rx=0 tx=0).stop 2023-07-06T03:55:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 -- 172.21.15.97:0/3176173953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80380b4a40 msgr2=0x7f803815e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80380b4a40 0x7f803815e8b0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f802800c7f0 tx=0x7f802800cbb0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 -- 172.21.15.97:0/3176173953 shutdown_connections 2023-07-06T03:55:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f801806ace0 0x7f801806d190 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80380b6280 0x7f803815df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80380b4a40 0x7f803815e8b0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 --2- 172.21.15.97:0/3176173953 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f80380b5e10 0x7f80380c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 -- 172.21.15.97:0/3176173953 >> 172.21.15.97:0/3176173953 conn(0x7f803806a900 msgr2=0x7f80380b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 -- 172.21.15.97:0/3176173953 shutdown_connections 2023-07-06T03:55:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:07.700+0000 7f803cdff700 1 -- 172.21.15.97:0/3176173953 wait complete. 2023-07-06T03:55:07.720 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:55:07.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:55:07.816 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:55:07.929 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:55:07.930 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:55:07.930 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:07.975 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:55:07.976 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:55:07.976 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:08.024 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:55:08.024 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:55:08.025 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:08.070 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:55:08.070 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:55:08.071 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:08.117 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:55:08.117 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T03:55:08.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:55:08.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.310+0000 7f1fd1bff700 1 Processor -- start 2023-07-06T03:55:08.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.311+0000 7f1fd1bff700 1 -- start start 2023-07-06T03:55:08.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.312+0000 7f1fd1bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fcc15c920 0x7f1fcc06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.312+0000 7f1fd1bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1fcc0d11a0 0x7f1fcc06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.312+0000 7f1fd1bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc0cfd00 con 0x7f1fcc15ccf0 2023-07-06T03:55:08.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.312+0000 7f1fd1bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc0b28f0 con 0x7f1fcc0d11a0 2023-07-06T03:55:08.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.312+0000 7f1fd1bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc1610b0 con 0x7f1fcc15c920 2023-07-06T03:55:08.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.312+0000 7f1fcbfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1fcc0d11a0 0x7f1fcc06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.312+0000 7f1fcb7fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fcc15c920 0x7f1fcc06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.312+0000 7f1fcbfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1fcc0d11a0 0x7f1fcc06c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:54550/0 (socket says 172.21.15.97:54550) 2023-07-06T03:55:08.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.312+0000 7f1fcbfff700 1 -- 172.21.15.97:0/1405485513 learned_addr learned my addr 172.21.15.97:0/1405485513 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:08.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.313+0000 7f1fcbfff700 1 -- 172.21.15.97:0/1405485513 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fcc15c920 msgr2=0x7f1fcc06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:08.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.313+0000 7f1fcbfff700 1 --2- 172.21.15.97:0/1405485513 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fcc15c920 0x7f1fcc06c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.313+0000 7f1fcbfff700 1 -- 172.21.15.97:0/1405485513 >> v1:172.21.15.146:6789/0 conn(0x7f1fcc15ccf0 legacy=0x7f1fcc06b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:08.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.313+0000 7f1fcbfff700 1 -- 172.21.15.97:0/1405485513 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc161390 con 0x7f1fcc0d11a0 2023-07-06T03:55:08.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.313+0000 7f1fcb7fe700 1 --2- 172.21.15.97:0/1405485513 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fcc15c920 0x7f1fcc06c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:55:08.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.313+0000 7f1fcbfff700 1 --2- 172.21.15.97:0/1405485513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc0d11a0 0x7f1fcc06c8d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009e50 tx=0x7f1fc400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2be8d1d9d4afe75b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:08.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.313+0000 7f1fcaffd700 1 -- 172.21.15.97:0/1405485513 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400e070 con 0x7f1fcc0d11a0 2023-07-06T03:55:08.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.313+0000 7f1fcaffd700 1 -- 172.21.15.97:0/1405485513 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc400b910 con 0x7f1fcc0d11a0 2023-07-06T03:55:08.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.313+0000 7f1fcaffd700 1 -- 172.21.15.97:0/1405485513 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400bcc0 con 0x7f1fcc0d11a0 2023-07-06T03:55:08.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.314+0000 7f1fd1bff700 1 -- 172.21.15.97:0/1405485513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc0d11a0 msgr2=0x7f1fcc06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:08.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.314+0000 7f1fd1bff700 1 --2- 172.21.15.97:0/1405485513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc0d11a0 0x7f1fcc06c8d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009e50 tx=0x7f1fc400b330 comp rx=0 tx=0).stop 2023-07-06T03:55:08.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.314+0000 7f1fd1bff700 1 -- 172.21.15.97:0/1405485513 shutdown_connections 2023-07-06T03:55:08.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.314+0000 7f1fd1bff700 1 --2- 172.21.15.97:0/1405485513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc0d11a0 0x7f1fcc06c8d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.314+0000 7f1fd1bff700 1 --2- 172.21.15.97:0/1405485513 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fcc15c920 0x7f1fcc06c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.314+0000 7f1fd1bff700 1 -- 172.21.15.97:0/1405485513 >> 172.21.15.97:0/1405485513 conn(0x7f1fcc06ace0 msgr2=0x7f1fcc06dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.314+0000 7f1fd1bff700 1 -- 172.21.15.97:0/1405485513 shutdown_connections 2023-07-06T03:55:08.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.314+0000 7f1fd1bff700 1 -- 172.21.15.97:0/1405485513 wait complete. 2023-07-06T03:55:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.316+0000 7f1fd1bff700 1 Processor -- start 2023-07-06T03:55:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.316+0000 7f1fd1bff700 1 -- start start 2023-07-06T03:55:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fd1bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc15c920 0x7f1fcc156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fd1bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fcc15ccf0 0x7f1fcc0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fd1bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc157580 0x7f1fcc0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fd1bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15fc40 con 0x7f1fcc15c920 2023-07-06T03:55:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fd1bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15f090 con 0x7f1fcc157580 2023-07-06T03:55:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fd1bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc156d90 con 0x7f1fcc15ccf0 2023-07-06T03:55:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fcb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc15c920 0x7f1fcc156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fcb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc15c920 0x7f1fcc156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32812/0 (socket says 172.21.15.97:32812) 2023-07-06T03:55:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fcb7fe700 1 -- 172.21.15.97:0/767048154 learned_addr learned my addr 172.21.15.97:0/767048154 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:08.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fcbfff700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc157580 0x7f1fcc0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fc3fff700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fcc15ccf0 0x7f1fcc0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fcb7fe700 1 -- 172.21.15.97:0/767048154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fcc15ccf0 msgr2=0x7f1fcc0ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:08.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fcb7fe700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fcc15ccf0 0x7f1fcc0ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fcb7fe700 1 -- 172.21.15.97:0/767048154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc157580 msgr2=0x7f1fcc0aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:08.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fcb7fe700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc157580 0x7f1fcc0aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fcb7fe700 1 -- 172.21.15.97:0/767048154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc0094a0 con 0x7f1fcc15c920 2023-07-06T03:55:08.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.317+0000 7f1fcbfff700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc157580 0x7f1fcc0aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:08.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.318+0000 7f1fcb7fe700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc15c920 0x7f1fcc156850 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00ec10 tx=0x7f1fbc009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:08.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.318+0000 7f1fc8ff9700 1 -- 172.21.15.97:0/767048154 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc00c8e0 con 0x7f1fcc15c920 2023-07-06T03:55:08.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.318+0000 7f1fc8ff9700 1 -- 172.21.15.97:0/767048154 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc00ca40 con 0x7f1fcc15c920 2023-07-06T03:55:08.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.318+0000 7f1fc8ff9700 1 -- 172.21.15.97:0/767048154 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc00cdb0 con 0x7f1fcc15c920 2023-07-06T03:55:08.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.318+0000 7f1fd1bff700 1 -- 172.21.15.97:0/767048154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc4009b30 con 0x7f1fcc15c920 2023-07-06T03:55:08.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.318+0000 7f1fd1bff700 1 -- 172.21.15.97:0/767048154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc15f580 con 0x7f1fcc15c920 2023-07-06T03:55:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.319+0000 7f1fd1bff700 1 -- 172.21.15.97:0/767048154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc066a40 con 0x7f1fcc15c920 2023-07-06T03:55:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.320+0000 7f1fc8ff9700 1 -- 172.21.15.97:0/767048154 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1fbc013470 con 0x7f1fcc15c920 2023-07-06T03:55:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.321+0000 7f1fc8ff9700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fac06ace0 0x7f1fac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.321+0000 7f1fc8ff9700 1 -- 172.21.15.97:0/767048154 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1fbc089490 con 0x7f1fcc15c920 2023-07-06T03:55:08.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.321+0000 7f1fc3fff700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fac06ace0 0x7f1fac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.322+0000 7f1fc3fff700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fac06ace0 0x7f1fac06d190 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc15ed50 tx=0x7f1fb4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:08.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.325+0000 7f1fc8ff9700 1 -- 172.21.15.97:0/767048154 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1fbc054730 con 0x7f1fcc15c920 2023-07-06T03:55:08.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.564+0000 7f1fd1bff700 1 -- 172.21.15.97:0/767048154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fcc15ff40 con 0x7f1fcc15c920 2023-07-06T03:55:08.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.564+0000 7f1fc8ff9700 1 -- 172.21.15.97:0/767048154 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f1fbc017de0 con 0x7f1fcc15c920 2023-07-06T03:55:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:08.568 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2796855133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2796855133},{"type":"v1","addr":"172.21.15.97:6835","nonce":2796855133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6851,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1092966074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":1092966074},{"type":"v1","addr":"172.21.15.146:6835","nonce":1092966074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75},{"gid":8113,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/3396642992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3396642992},{"type":"v1","addr":"172.21.15.146:6834","nonce":3396642992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":8577,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2594138359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2594138359},{"type":"v1","addr":"172.21.15.97:6837","nonce":2594138359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[]} 2023-07-06T03:55:08.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.566+0000 7f1fd1bff700 1 -- 172.21.15.97:0/767048154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fac06ace0 msgr2=0x7f1fac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:08.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.566+0000 7f1fd1bff700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fac06ace0 0x7f1fac06d190 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc15ed50 tx=0x7f1fb4009040 comp rx=0 tx=0).stop 2023-07-06T03:55:08.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.566+0000 7f1fd1bff700 1 -- 172.21.15.97:0/767048154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc15c920 msgr2=0x7f1fcc156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:08.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.566+0000 7f1fd1bff700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc15c920 0x7f1fcc156850 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00ec10 tx=0x7f1fbc009ec0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.566+0000 7f1fd1bff700 1 -- 172.21.15.97:0/767048154 shutdown_connections 2023-07-06T03:55:08.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.567+0000 7f1fd1bff700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1fac06ace0 0x7f1fac06d190 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.567+0000 7f1fd1bff700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc157580 0x7f1fcc0aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.567+0000 7f1fd1bff700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc15c920 0x7f1fcc156850 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.567+0000 7f1fd1bff700 1 --2- 172.21.15.97:0/767048154 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1fcc15ccf0 0x7f1fcc0ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.567+0000 7f1fd1bff700 1 -- 172.21.15.97:0/767048154 >> 172.21.15.97:0/767048154 conn(0x7f1fcc0d11a0 msgr2=0x7f1fcc0b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:08.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.567+0000 7f1fd1bff700 1 -- 172.21.15.97:0/767048154 shutdown_connections 2023-07-06T03:55:08.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.567+0000 7f1fd1bff700 1 -- 172.21.15.97:0/767048154 wait complete. 2023-07-06T03:55:08.571 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 86 2023-07-06T03:55:08.587 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T03:55:08.588 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T03:55:08.588 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T03:55:08.588 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:55:08.589 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T03:55:08.589 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T03:55:08.589 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T03:55:08.590 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:55:08.590 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T03:55:08.590 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T03:55:08.591 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T03:55:08.591 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:55:08.591 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T03:55:08.592 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T03:55:08.592 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T03:55:08.592 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:55:08.601 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T03:55:08.602 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T03:55:08.603 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T03:55:08.603 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T03:55:08.605 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T03:55:08.605 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T03:55:08.606 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T03:55:08.606 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T03:55:08.647 INFO:tasks.ceph.mds.a:Started 2023-07-06T03:55:08.648 DEBUG:teuthology.parallel:result is None 2023-07-06T03:55:08.649 INFO:tasks.ceph.mds.b:Started 2023-07-06T03:55:08.650 DEBUG:teuthology.parallel:result is None 2023-07-06T03:55:08.651 INFO:tasks.ceph.mds.c:Started 2023-07-06T03:55:08.651 DEBUG:teuthology.parallel:result is None 2023-07-06T03:55:08.653 INFO:tasks.ceph.mds.d:Started 2023-07-06T03:55:08.653 DEBUG:teuthology.parallel:result is None 2023-07-06T03:55:08.654 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T03:55:08.796 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:55:08.797 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:55:08.798 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-07-06T03:55:08.798 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:55:08.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.859+0000 7f86f5dff700 1 Processor -- start 2023-07-06T03:55:08.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.859+0000 7f86f5dff700 1 -- start start 2023-07-06T03:55:08.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.860+0000 7f86f5dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f86f015c610 0x7f86f00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.860+0000 7f86f5dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f00d2210 0x7f86f00d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.860+0000 7f86f5dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f00cff90 con 0x7f86f006b360 2023-07-06T03:55:08.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.860+0000 7f86f5dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f00d0780 con 0x7f86f015c610 2023-07-06T03:55:08.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.860+0000 7f86f5dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f015c9e0 con 0x7f86f00d2210 2023-07-06T03:55:08.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.861+0000 7f86ef7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f86f015c610 0x7f86f00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.861+0000 7f86eeffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f86f006b360 0x7f86f00b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:52408/0 (socket says 172.21.15.97:52408) 2023-07-06T03:55:08.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.861+0000 7f86eeffd700 1 -- 172.21.15.97:0/2510562294 learned_addr learned my addr 172.21.15.97:0/2510562294 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:08.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.861+0000 7f86effff700 1 --2- 172.21.15.97:0/2510562294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f00d2210 0x7f86f00d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.861+0000 7f86effff700 1 -- 172.21.15.97:0/2510562294 >> v2:172.21.15.146:3300/0 conn(0x7f86f015c610 msgr2=0x7f86f00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.861+0000 7f86effff700 1 --2- 172.21.15.97:0/2510562294 >> v2:172.21.15.146:3300/0 conn(0x7f86f015c610 0x7f86f00b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.861+0000 7f86effff700 1 -- 172.21.15.97:0/2510562294 >> v1:172.21.15.146:6789/0 conn(0x7f86f006b360 legacy=0x7f86f00b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.861+0000 7f86ee7fc700 1 -- 172.21.15.97:0/2510562294 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173957276 0 0) 0x7f86f00cff90 con 0x7f86f006b360 2023-07-06T03:55:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.861+0000 7f86effff700 1 -- 172.21.15.97:0/2510562294 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f00b5650 con 0x7f86f00d2210 2023-07-06T03:55:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.861+0000 7f86effff700 1 --2- 172.21.15.97:0/2510562294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f00d2210 0x7f86f00d1310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009d90 tx=0x7f86e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fbed7ee75aa2ab7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.862+0000 7f86ee7fc700 1 -- 172.21.15.97:0/2510562294 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e800e070 con 0x7f86f00d2210 2023-07-06T03:55:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.862+0000 7f86ee7fc700 1 -- 172.21.15.97:0/2510562294 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e800bb20 con 0x7f86f00d2210 2023-07-06T03:55:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.862+0000 7f86ee7fc700 1 -- 172.21.15.97:0/2510562294 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8012430 con 0x7f86f00d2210 2023-07-06T03:55:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.862+0000 7f86f5dff700 1 -- 172.21.15.97:0/2510562294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f00d2210 msgr2=0x7f86f00d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.862+0000 7f86f5dff700 1 --2- 172.21.15.97:0/2510562294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f00d2210 0x7f86f00d1310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009d90 tx=0x7f86e800b330 comp rx=0 tx=0).stop 2023-07-06T03:55:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.862+0000 7f86f5dff700 1 -- 172.21.15.97:0/2510562294 shutdown_connections 2023-07-06T03:55:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.862+0000 7f86f5dff700 1 --2- 172.21.15.97:0/2510562294 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f00d2210 0x7f86f00d1310 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.862+0000 7f86f5dff700 1 --2- 172.21.15.97:0/2510562294 >> v2:172.21.15.146:3300/0 conn(0x7f86f015c610 0x7f86f00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.862+0000 7f86f5dff700 1 -- 172.21.15.97:0/2510562294 >> 172.21.15.97:0/2510562294 conn(0x7f86f006a6c0 msgr2=0x7f86f006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.862+0000 7f86f5dff700 1 -- 172.21.15.97:0/2510562294 shutdown_connections 2023-07-06T03:55:08.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.862+0000 7f86f5dff700 1 -- 172.21.15.97:0/2510562294 wait complete. 2023-07-06T03:55:08.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.864+0000 7f86f5dff700 1 Processor -- start 2023-07-06T03:55:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.864+0000 7f86f5dff700 1 -- start start 2023-07-06T03:55:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.865+0000 7f86f5dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86f006b360 0x7f86f00c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.865+0000 7f86f5dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f00d2210 0x7f86f0151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.865+0000 7f86f5dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f015c610 0x7f86f01f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.865+0000 7f86f5dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f00cac80 con 0x7f86f00d2210 2023-07-06T03:55:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.865+0000 7f86f5dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01f54f0 con 0x7f86f006b360 2023-07-06T03:55:08.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.865+0000 7f86f5dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f00ca3e0 con 0x7f86f015c610 2023-07-06T03:55:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.865+0000 7f86eeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f00d2210 0x7f86f0151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.865+0000 7f86eeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f00d2210 0x7f86f0151d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32840/0 (socket says 172.21.15.97:32840) 2023-07-06T03:55:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.865+0000 7f86eeffd700 1 -- 172.21.15.97:0/3253308457 learned_addr learned my addr 172.21.15.97:0/3253308457 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.865+0000 7f86ef7fe700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86f006b360 0x7f86f00c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86effff700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f015c610 0x7f86f01f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86eeffd700 1 -- 172.21.15.97:0/3253308457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f015c610 msgr2=0x7f86f01f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:08.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86eeffd700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f015c610 0x7f86f01f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86eeffd700 1 -- 172.21.15.97:0/3253308457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86f006b360 msgr2=0x7f86f00c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:08.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86eeffd700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86f006b360 0x7f86f00c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:08.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86eeffd700 1 -- 172.21.15.97:0/3253308457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e8009a70 con 0x7f86f00d2210 2023-07-06T03:55:08.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86ef7fe700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86f006b360 0x7f86f00c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:08.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86effff700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f015c610 0x7f86f01f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:08.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86eeffd700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f00d2210 0x7f86f0151d70 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f86e40011e0 tx=0x7f86e400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86d7fff700 1 -- 172.21.15.97:0/3253308457 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4004c80 con 0x7f86f00d2210 2023-07-06T03:55:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86d7fff700 1 -- 172.21.15.97:0/3253308457 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4004e00 con 0x7f86f00d2210 2023-07-06T03:55:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86d7fff700 1 -- 172.21.15.97:0/3253308457 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4007310 con 0x7f86f00d2210 2023-07-06T03:55:08.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86f5dff700 1 -- 172.21.15.97:0/3253308457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f00ca6c0 con 0x7f86f00d2210 2023-07-06T03:55:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.866+0000 7f86f5dff700 1 -- 172.21.15.97:0/3253308457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f015ecc0 con 0x7f86f00d2210 2023-07-06T03:55:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.867+0000 7f86f5dff700 1 -- 172.21.15.97:0/3253308457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f01f5250 con 0x7f86f00d2210 2023-07-06T03:55:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.868+0000 7f86d7fff700 1 -- 172.21.15.97:0/3253308457 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f86e4007470 con 0x7f86f00d2210 2023-07-06T03:55:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.869+0000 7f86d7fff700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f86dc06ace0 0x7f86dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.869+0000 7f86d7fff700 1 -- 172.21.15.97:0/3253308457 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f86e4088f80 con 0x7f86f00d2210 2023-07-06T03:55:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.870+0000 7f86ef7fe700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f86dc06ace0 0x7f86dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.870+0000 7f86ef7fe700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f86dc06ace0 0x7f86dc06d190 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009b90 tx=0x7f86e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:08.874+0000 7f86d7fff700 1 -- 172.21.15.97:0/3253308457 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86f01f5250 con 0x7f86f00d2210 2023-07-06T03:55:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:09.085+0000 7f86f5dff700 1 -- 172.21.15.97:0/3253308457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f86f01f5250 con 0x7f86f00d2210 2023-07-06T03:55:10.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.285+0000 7f86d7fff700 1 -- 172.21.15.97:0/3253308457 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) v1 ==== 97+0+0 (secure 0 0 0) 0x7f86f01f5250 con 0x7f86f00d2210 2023-07-06T03:55:10.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 -- 172.21.15.97:0/3253308457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f86dc06ace0 msgr2=0x7f86dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:10.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f86dc06ace0 0x7f86dc06d190 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009b90 tx=0x7f86e0008040 comp rx=0 tx=0).stop 2023-07-06T03:55:10.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 -- 172.21.15.97:0/3253308457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f00d2210 msgr2=0x7f86f0151d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:10.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f00d2210 0x7f86f0151d70 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f86e40011e0 tx=0x7f86e400cae0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 -- 172.21.15.97:0/3253308457 shutdown_connections 2023-07-06T03:55:10.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f86dc06ace0 0x7f86dc06d190 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86f015c610 0x7f86f01f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86f006b360 0x7f86f00c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 --2- 172.21.15.97:0/3253308457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f00d2210 0x7f86f0151d70 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 -- 172.21.15.97:0/3253308457 >> 172.21.15.97:0/3253308457 conn(0x7f86f006a6c0 msgr2=0x7f86f015b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:10.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 -- 172.21.15.97:0/3253308457 shutdown_connections 2023-07-06T03:55:10.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.289+0000 7f86f5dff700 1 -- 172.21.15.97:0/3253308457 wait complete. 2023-07-06T03:55:10.293 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-07-06T03:55:10.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T03:55:10.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.508+0000 7ff4bb1ff700 1 Processor -- start 2023-07-06T03:55:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.508+0000 7ff4bb1ff700 1 -- start start 2023-07-06T03:55:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.509+0000 7ff4bb1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff4b40cd390 0x7ff4b4160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:10.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.509+0000 7ff4bb1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40cdae0 con 0x7ff4b415c520 2023-07-06T03:55:10.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.509+0000 7ff4bb1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40b6640 con 0x7ff4b4160dd0 2023-07-06T03:55:10.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.509+0000 7ff4bb1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40b6920 con 0x7ff4b40cd390 2023-07-06T03:55:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.509+0000 7ff4b8f9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff4b415c520 0x7ff4b4160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38312/0 (socket says 172.21.15.97:38312) 2023-07-06T03:55:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.509+0000 7ff4b8f9b700 1 -- 172.21.15.97:0/1212741925 learned_addr learned my addr 172.21.15.97:0/1212741925 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442882973 0 0) 0x7ff4b40cdae0 con 0x7ff4b415c520 2023-07-06T03:55:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a40035b0 con 0x7ff4b415c520 2023-07-06T03:55:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b3fff700 1 --2- 172.21.15.97:0/1212741925 >> v2:172.21.15.146:3300/0 conn(0x7ff4b40cd390 0x7ff4b4160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1249340068 0 0) 0x7ff4a40035b0 con 0x7ff4b415c520 2023-07-06T03:55:10.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 >> v2:172.21.15.146:3300/0 conn(0x7ff4b40cd390 msgr2=0x7ff4b4160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:10.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 --2- 172.21.15.97:0/1212741925 >> v2:172.21.15.146:3300/0 conn(0x7ff4b40cd390 0x7ff4b4160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 >> v1:172.21.15.146:6789/0 conn(0x7ff4b4160dd0 legacy=0x7ff4b40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:10.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40b6c00 con 0x7ff4b415c520 2023-07-06T03:55:10.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7ff4a8001670 con 0x7ff4b415c520 2023-07-06T03:55:10.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 >> v1:172.21.15.97:6789/0 conn(0x7ff4b415c520 legacy=0x7ff4b4160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:10.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 --2- 172.21.15.97:0/1212741925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4a4003a00 0x7ff4a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:10.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 --2- 172.21.15.97:0/1212741925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4a4004850 0x7ff4a4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 --2- 172.21.15.97:0/1212741925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4a4005200 0x7ff4a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40b6920 con 0x7ff4a4004850 2023-07-06T03:55:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a400c080 con 0x7ff4a4003a00 2023-07-06T03:55:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.510+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a400c240 con 0x7ff4a4005200 2023-07-06T03:55:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b3fff700 1 --2- 172.21.15.97:0/1212741925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4a4004850 0x7ff4a4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b8f9b700 1 --2- 172.21.15.97:0/1212741925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4a4003a00 0x7ff4a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b979c700 1 --2- 172.21.15.97:0/1212741925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4a4005200 0x7ff4a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b3fff700 1 -- 172.21.15.97:0/1212741925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4a4005200 msgr2=0x7ff4a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b3fff700 1 --2- 172.21.15.97:0/1212741925 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4a4005200 0x7ff4a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b3fff700 1 -- 172.21.15.97:0/1212741925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4a4003a00 msgr2=0x7ff4a4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b3fff700 1 --2- 172.21.15.97:0/1212741925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4a4003a00 0x7ff4a4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b3fff700 1 -- 172.21.15.97:0/1212741925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a40035b0 con 0x7ff4a4004850 2023-07-06T03:55:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b979c700 1 -- 172.21.15.97:0/1212741925 reap_dead start 2023-07-06T03:55:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b3fff700 1 --2- 172.21.15.97:0/1212741925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4a4004850 0x7ff4a4004cc0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009b60 tx=0x7ff4a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df77444ba07017ae server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000e6c0 con 0x7ff4a4004850 2023-07-06T03:55:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a000e820 con 0x7ff4a4004850 2023-07-06T03:55:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.511+0000 7ff4b37fe700 1 -- 172.21.15.97:0/1212741925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000eb90 con 0x7ff4a4004850 2023-07-06T03:55:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.512+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/1212741925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4a4004850 msgr2=0x7ff4a4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.512+0000 7ff4bb1ff700 1 --2- 172.21.15.97:0/1212741925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4a4004850 0x7ff4a4004cc0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009b60 tx=0x7ff4a4005880 comp rx=0 tx=0).stop 2023-07-06T03:55:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.512+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/1212741925 shutdown_connections 2023-07-06T03:55:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.512+0000 7ff4bb1ff700 1 --2- 172.21.15.97:0/1212741925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4a4004850 0x7ff4a4004cc0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.512+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/1212741925 >> 172.21.15.97:0/1212741925 conn(0x7ff4b4071790 msgr2=0x7ff4b4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.512+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/1212741925 shutdown_connections 2023-07-06T03:55:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.512+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/1212741925 wait complete. 2023-07-06T03:55:10.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.514+0000 7ff4bb1ff700 1 Processor -- start 2023-07-06T03:55:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.514+0000 7ff4bb1ff700 1 -- start start 2023-07-06T03:55:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4bb1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4a4004850 0x7ff4b4150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4bb1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b40c7c00 0x7ff4b40cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4bb1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4b40c8d40 0x7ff4b40c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4bb1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40c96f0 con 0x7ff4b40c7c00 2023-07-06T03:55:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4bb1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4154410 con 0x7ff4a4004850 2023-07-06T03:55:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4bb1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40ccc50 con 0x7ff4b40c8d40 2023-07-06T03:55:10.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4b3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b40c7c00 0x7ff4b40cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:10.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4b3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b40c7c00 0x7ff4b40cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39956/0 (socket says 172.21.15.97:39956) 2023-07-06T03:55:10.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4b3fff700 1 -- 172.21.15.97:0/3104011282 learned_addr learned my addr 172.21.15.97:0/3104011282 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:10.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4b979c700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4b40c8d40 0x7ff4b40c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:10.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4b8f9b700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4a4004850 0x7ff4b4150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:10.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4b3fff700 1 -- 172.21.15.97:0/3104011282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4b40c8d40 msgr2=0x7ff4b40c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:10.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4b3fff700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4b40c8d40 0x7ff4b40c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4b3fff700 1 -- 172.21.15.97:0/3104011282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4a4004850 msgr2=0x7ff4b4150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.515+0000 7ff4b3fff700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4a4004850 0x7ff4b4150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.516+0000 7ff4b3fff700 1 -- 172.21.15.97:0/3104011282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40cc690 con 0x7ff4b40c7c00 2023-07-06T03:55:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.516+0000 7ff4b8f9b700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4a4004850 0x7ff4b4150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.516+0000 7ff4b3fff700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b40c7c00 0x7ff4b40cbd60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c050 tx=0x7ff4a00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.516+0000 7ff4b17fa700 1 -- 172.21.15.97:0/3104011282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000e6c0 con 0x7ff4b40c7c00 2023-07-06T03:55:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.516+0000 7ff4b17fa700 1 -- 172.21.15.97:0/3104011282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a0003ad0 con 0x7ff4b40c7c00 2023-07-06T03:55:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.516+0000 7ff4b17fa700 1 -- 172.21.15.97:0/3104011282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0003e80 con 0x7ff4b40c7c00 2023-07-06T03:55:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.516+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/3104011282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b40cc970 con 0x7ff4b40c7c00 2023-07-06T03:55:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.516+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/3104011282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b4150780 con 0x7ff4b40c7c00 2023-07-06T03:55:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.517+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/3104011282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b404e340 con 0x7ff4b40c7c00 2023-07-06T03:55:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.518+0000 7ff4b17fa700 1 -- 172.21.15.97:0/3104011282 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff4a0013470 con 0x7ff4b40c7c00 2023-07-06T03:55:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.519+0000 7ff4b17fa700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff49806ace0 0x7ff49806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.519+0000 7ff4b17fa700 1 -- 172.21.15.97:0/3104011282 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff4a008aa40 con 0x7ff4b40c7c00 2023-07-06T03:55:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.519+0000 7ff4b8f9b700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff49806ace0 0x7ff49806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.520+0000 7ff4b8f9b700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff49806ace0 0x7ff49806d190 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8001490 tx=0x7ff4a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.523+0000 7ff4b17fa700 1 -- 172.21.15.97:0/3104011282 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff4a0055f50 con 0x7ff4b40c7c00 2023-07-06T03:55:10.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.790+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/3104011282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff4b40b4f40 con 0x7ff4b40c7c00 2023-07-06T03:55:10.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.790+0000 7ff4b17fa700 1 -- 172.21.15.97:0/3104011282 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff4a0017090 con 0x7ff4b40c7c00 2023-07-06T03:55:10.793 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:10.793 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:55:10.794 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-07-06T03:55:10.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.794 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-07-06T03:55:10.794 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkczuvGRAAUKjfmMyKOTCYzOQmtf3n3A==", 2023-07-06T03:55:10.794 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.794 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:55:10.794 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:55:10.795 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:55:10.795 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.795 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-07-06T03:55:10.795 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkFHvUMBAA2WgMcUUf+Qi8T2ZjhfWaHA==", 2023-07-06T03:55:10.795 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.796 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:55:10.796 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:55:10.796 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:55:10.796 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.797 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-07-06T03:55:10.797 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZk0N4+JRAA3tjqoIBPpZ6C7TMmOOg7lw==", 2023-07-06T03:55:10.797 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.797 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:55:10.797 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:55:10.797 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:55:10.797 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.798 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-07-06T03:55:10.798 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkJnHUABAAwIMH2DmoHaQN3frJCgRiQw==", 2023-07-06T03:55:10.798 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.798 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:55:10.798 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:55:10.799 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:55:10.799 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.799 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.799 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-07-06T03:55:10.799 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCLOKZkShOIABAAzIBHDcNkmnDOLSY7Qdb+FA==", 2023-07-06T03:55:10.799 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.800 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:55:10.800 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:55:10.800 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.800 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.800 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-07-06T03:55:10.800 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCOOKZkq1t7AxAAwihuhimbjitZDeki1QuMug==", 2023-07-06T03:55:10.801 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.801 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:55:10.801 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:55:10.801 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.801 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.802 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-07-06T03:55:10.802 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCROKZkdYv4BhAApTHg/pmlC0lO7hVTkimEgw==", 2023-07-06T03:55:10.802 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.802 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:55:10.802 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:55:10.802 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.802 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.803 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-07-06T03:55:10.803 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCUOKZkqhuMChAAsV9xG6g1dofrpWt3F8tw5w==", 2023-07-06T03:55:10.803 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.803 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:55:10.803 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:55:10.803 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.803 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.804 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-07-06T03:55:10.804 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCaOKZk1Q5aIBAABU1+afwIihm+g4xylryz/A==", 2023-07-06T03:55:10.804 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.804 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:55:10.804 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:55:10.804 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.805 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-07-06T03:55:10.805 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCdOKZkRskIJBAA+821aoG98aEV7h4FTINQFA==", 2023-07-06T03:55:10.805 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.805 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:55:10.806 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:55:10.806 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.806 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.806 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-07-06T03:55:10.806 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCgOKZk9qnMJxAATcWG+a/9Hl1AuIL/TSrdEA==", 2023-07-06T03:55:10.806 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.807 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:55:10.807 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:55:10.807 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.807 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-07-06T03:55:10.807 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCjOKZk6wJ/KxAAWnZxt7h7OJhQ+bqdel4+iQ==", 2023-07-06T03:55:10.808 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.808 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:55:10.808 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:55:10.808 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.808 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.808 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-07-06T03:55:10.809 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg==", 2023-07-06T03:55:10.809 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.809 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:55:10.809 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-07-06T03:55:10.809 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-06T03:55:10.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.810 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-07-06T03:55:10.810 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkq4/pGRAAucHmR1eBA8MNMy3/l7tl0g==", 2023-07-06T03:55:10.810 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.810 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:55:10.810 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:55:10.810 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:55:10.810 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:55:10.810 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.811 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-07-06T03:55:10.811 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkhL5fEhAAWhw4IbCPqTO2sXPIcpTVHA==", 2023-07-06T03:55:10.811 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.811 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:55:10.811 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:55:10.812 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:55:10.812 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:55:10.812 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.812 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-07-06T03:55:10.812 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkO81dIRAAffjVyvsb/ygBtx8UrYQK0w==", 2023-07-06T03:55:10.812 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.813 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:55:10.813 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:55:10.813 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:55:10.813 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:55:10.813 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.813 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-07-06T03:55:10.814 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCFOKZk6uVlHRAA92RDhdDKW/q65SEOQMWgiA==", 2023-07-06T03:55:10.814 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.814 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:55:10.814 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:55:10.814 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:55:10.814 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.814 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.815 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-07-06T03:55:10.815 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkT4ALEhAAXwwkcodQU9meJA0tfseJCg==", 2023-07-06T03:55:10.815 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.815 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:55:10.815 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:55:10.815 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.816 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:10.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:10.816 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-07-06T03:55:10.816 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkfnoEChAAuRxqXqPI33dgWkBJoUMh3w==", 2023-07-06T03:55:10.816 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:55:10.816 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:55:10.816 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:55:10.817 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:55:10.817 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.817 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:10.817 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:55:10.817 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:55:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.792+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/3104011282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff49806ace0 msgr2=0x7ff49806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:10.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.792+0000 7ff4bb1ff700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff49806ace0 0x7ff49806d190 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8001490 tx=0x7ff4a8009040 comp rx=0 tx=0).stop 2023-07-06T03:55:10.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.792+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/3104011282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b40c7c00 msgr2=0x7ff4b40cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:10.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.792+0000 7ff4bb1ff700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b40c7c00 0x7ff4b40cbd60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c050 tx=0x7ff4a00047c0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.793+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/3104011282 shutdown_connections 2023-07-06T03:55:10.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.793+0000 7ff4bb1ff700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff49806ace0 0x7ff49806d190 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.793+0000 7ff4bb1ff700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff4b40c8d40 0x7ff4b40c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.793+0000 7ff4bb1ff700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff4a4004850 0x7ff4b4150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.793+0000 7ff4bb1ff700 1 --2- 172.21.15.97:0/3104011282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b40c7c00 0x7ff4b40cbd60 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:10.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.793+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/3104011282 >> 172.21.15.97:0/3104011282 conn(0x7ff4b4071790 msgr2=0x7ff4b40b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:10.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.793+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/3104011282 shutdown_connections 2023-07-06T03:55:10.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:10.793+0000 7ff4bb1ff700 1 -- 172.21.15.97:0/3104011282 wait complete. 2023-07-06T03:55:10.821 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:55:11.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.013+0000 7fa0877ff700 1 Processor -- start 2023-07-06T03:55:11.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.013+0000 7fa0877ff700 1 -- start start 2023-07-06T03:55:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa0877ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa08015c520 0x7fa0800cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa0877ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa080160d00 0x7fa0800b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa0877ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0800cdae0 con 0x7fa080160180 2023-07-06T03:55:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa0877ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0800cd8d0 con 0x7fa080160d00 2023-07-06T03:55:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa0877ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0800b6640 con 0x7fa08015c520 2023-07-06T03:55:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa085d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa080160d00 0x7fa0800b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa085d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa080160d00 0x7fa0800b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39970/0 (socket says 172.21.15.97:39970) 2023-07-06T03:55:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa085d9c700 1 -- 172.21.15.97:0/2481232305 learned_addr learned my addr 172.21.15.97:0/2481232305 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa08559b700 1 --2- 172.21.15.97:0/2481232305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa08015c520 0x7fa0800cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa085d9c700 1 -- 172.21.15.97:0/2481232305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa08015c520 msgr2=0x7fa0800cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa085d9c700 1 --2- 172.21.15.97:0/2481232305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa08015c520 0x7fa0800cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa085d9c700 1 -- 172.21.15.97:0/2481232305 >> v1:172.21.15.146:6789/0 conn(0x7fa080160180 legacy=0x7fa0801605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.015+0000 7fa085d9c700 1 -- 172.21.15.97:0/2481232305 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0800b6920 con 0x7fa080160d00 2023-07-06T03:55:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.016+0000 7fa085d9c700 1 --2- 172.21.15.97:0/2481232305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa080160d00 0x7fa0800b7400 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa078009e50 tx=0x7fa07800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ac5ed6d65edde4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.016+0000 7fa08559b700 1 --2- 172.21.15.97:0/2481232305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa08015c520 0x7fa0800cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.016+0000 7fa077fff700 1 -- 172.21.15.97:0/2481232305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07800e070 con 0x7fa080160d00 2023-07-06T03:55:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.016+0000 7fa077fff700 1 -- 172.21.15.97:0/2481232305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07800b9c0 con 0x7fa080160d00 2023-07-06T03:55:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.016+0000 7fa077fff700 1 -- 172.21.15.97:0/2481232305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07800bd50 con 0x7fa080160d00 2023-07-06T03:55:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.016+0000 7fa0877ff700 1 -- 172.21.15.97:0/2481232305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa080160d00 msgr2=0x7fa0800b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.016+0000 7fa0877ff700 1 --2- 172.21.15.97:0/2481232305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa080160d00 0x7fa0800b7400 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa078009e50 tx=0x7fa07800b330 comp rx=0 tx=0).stop 2023-07-06T03:55:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.016+0000 7fa0877ff700 1 -- 172.21.15.97:0/2481232305 shutdown_connections 2023-07-06T03:55:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.017+0000 7fa0877ff700 1 --2- 172.21.15.97:0/2481232305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa080160d00 0x7fa0800b7400 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.017+0000 7fa0877ff700 1 --2- 172.21.15.97:0/2481232305 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa08015c520 0x7fa0800cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.017+0000 7fa0877ff700 1 -- 172.21.15.97:0/2481232305 >> 172.21.15.97:0/2481232305 conn(0x7fa080071790 msgr2=0x7fa080071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.017+0000 7fa0877ff700 1 -- 172.21.15.97:0/2481232305 shutdown_connections 2023-07-06T03:55:11.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.017+0000 7fa0877ff700 1 -- 172.21.15.97:0/2481232305 wait complete. 2023-07-06T03:55:11.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.018+0000 7fa0877ff700 1 Processor -- start 2023-07-06T03:55:11.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.018+0000 7fa0877ff700 1 -- start start 2023-07-06T03:55:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.019+0000 7fa0877ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa08015c520 0x7fa08013ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.019+0000 7fa0877ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa080160180 0x7fa080141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.019+0000 7fa0877ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa080160d00 0x7fa080142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.019+0000 7fa0877ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa080159bb0 con 0x7fa080160180 2023-07-06T03:55:11.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.019+0000 7fa0877ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa080159e20 con 0x7fa080160d00 2023-07-06T03:55:11.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.019+0000 7fa0877ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08015a100 con 0x7fa08015c520 2023-07-06T03:55:11.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.020+0000 7fa084d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa080160180 0x7fa080141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.020+0000 7fa084d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa080160180 0x7fa080141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39978/0 (socket says 172.21.15.97:39978) 2023-07-06T03:55:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.020+0000 7fa084d9a700 1 -- 172.21.15.97:0/1512250967 learned_addr learned my addr 172.21.15.97:0/1512250967 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.020+0000 7fa08559b700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa08015c520 0x7fa08013ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.020+0000 7fa085d9c700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa080160d00 0x7fa080142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.020+0000 7fa084d9a700 1 -- 172.21.15.97:0/1512250967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa08015c520 msgr2=0x7fa08013ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.020+0000 7fa084d9a700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa08015c520 0x7fa08013ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.020+0000 7fa084d9a700 1 -- 172.21.15.97:0/1512250967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa080160d00 msgr2=0x7fa080142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.020+0000 7fa084d9a700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa080160d00 0x7fa080142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.020+0000 7fa084d9a700 1 -- 172.21.15.97:0/1512250967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa078009b30 con 0x7fa080160180 2023-07-06T03:55:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.020+0000 7fa084d9a700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa080160180 0x7fa080141fe0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c4e0 tx=0x7fa07000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.021+0000 7fa075ffb700 1 -- 172.21.15.97:0/1512250967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0700052d0 con 0x7fa080160180 2023-07-06T03:55:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.021+0000 7fa075ffb700 1 -- 172.21.15.97:0/1512250967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa070005430 con 0x7fa080160180 2023-07-06T03:55:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.021+0000 7fa0877ff700 1 -- 172.21.15.97:0/1512250967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08015a3e0 con 0x7fa080160180 2023-07-06T03:55:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.021+0000 7fa075ffb700 1 -- 172.21.15.97:0/1512250967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0700038c0 con 0x7fa080160180 2023-07-06T03:55:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.021+0000 7fa0877ff700 1 -- 172.21.15.97:0/1512250967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08015f980 con 0x7fa080160180 2023-07-06T03:55:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.022+0000 7fa0877ff700 1 -- 172.21.15.97:0/1512250967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa08015a620 con 0x7fa080160180 2023-07-06T03:55:11.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.023+0000 7fa075ffb700 1 -- 172.21.15.97:0/1512250967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa070003a20 con 0x7fa080160180 2023-07-06T03:55:11.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.024+0000 7fa075ffb700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa06406ace0 0x7fa06406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.024+0000 7fa075ffb700 1 -- 172.21.15.97:0/1512250967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa0700884e0 con 0x7fa080160180 2023-07-06T03:55:11.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.024+0000 7fa08559b700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa06406ace0 0x7fa06406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.025+0000 7fa08559b700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa06406ace0 0x7fa06406d190 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa07c006f10 tx=0x7fa07c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:11.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.028+0000 7fa075ffb700 1 -- 172.21.15.97:0/1512250967 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa08015a620 con 0x7fa080160180 2023-07-06T03:55:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.225+0000 7fa0877ff700 1 -- 172.21.15.97:0/1512250967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa08015a5d0 con 0x7fa080160180 2023-07-06T03:55:11.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.226+0000 7fa075ffb700 1 -- 172.21.15.97:0/1512250967 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+14675 (secure 0 0 0) 0x7fa0700538d0 con 0x7fa080160180 2023-07-06T03:55:11.229 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:11.229 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":120,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:55:10.282680+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:55:11.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 -- 172.21.15.97:0/1512250967 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa06406ace0 msgr2=0x7fa06406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa06406ace0 0x7fa06406d190 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa07c006f10 tx=0x7fa07c008040 comp rx=0 tx=0).stop 2023-07-06T03:55:11.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 -- 172.21.15.97:0/1512250967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa080160180 msgr2=0x7fa080141fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa080160180 0x7fa080141fe0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c4e0 tx=0x7fa07000c7f0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 -- 172.21.15.97:0/1512250967 shutdown_connections 2023-07-06T03:55:11.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa06406ace0 0x7fa06406d190 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa080160d00 0x7fa080142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa08015c520 0x7fa08013ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 --2- 172.21.15.97:0/1512250967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa080160180 0x7fa080141fe0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 -- 172.21.15.97:0/1512250967 >> 172.21.15.97:0/1512250967 conn(0x7fa080071790 msgr2=0x7fa0800d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:11.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 -- 172.21.15.97:0/1512250967 shutdown_connections 2023-07-06T03:55:11.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.229+0000 7fa0877ff700 1 -- 172.21.15.97:0/1512250967 wait complete. 2023-07-06T03:55:11.249 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:55:11.250 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:55:11.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.447+0000 7fb4c21ff700 1 Processor -- start 2023-07-06T03:55:11.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.447+0000 7fb4c21ff700 1 -- start start 2023-07-06T03:55:11.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4c21ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4bc0cb2c0 0x7fb4bc0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4c21ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0b26f0 0x7fb4bc06c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4c21ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc0cb690 con 0x7fb4bc06b760 2023-07-06T03:55:11.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4c21ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4bc06aa70 con 0x7fb4bc0cb2c0 2023-07-06T03:55:11.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4c21ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc06dde0 con 0x7fb4bc0b26f0 2023-07-06T03:55:11.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4bb7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4bc0cb2c0 0x7fb4bc0cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4baffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb4bc06b760 0x7fb4bc0b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38326/0 (socket says 172.21.15.97:38326) 2023-07-06T03:55:11.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4baffd700 1 -- 172.21.15.97:0/3436503964 learned_addr learned my addr 172.21.15.97:0/3436503964 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:11.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4bb7fe700 1 -- 172.21.15.97:0/3436503964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0b26f0 msgr2=0x7fb4bc06c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:11.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4bbfff700 1 --2- 172.21.15.97:0/3436503964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0b26f0 0x7fb4bc06c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4bb7fe700 1 --2- 172.21.15.97:0/3436503964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0b26f0 0x7fb4bc06c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4ba7fc700 1 -- 172.21.15.97:0/3436503964 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282838444 0 0) 0x7fb4bc0cb690 con 0x7fb4bc06b760 2023-07-06T03:55:11.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4bb7fe700 1 -- 172.21.15.97:0/3436503964 >> v1:172.21.15.97:6789/0 conn(0x7fb4bc06b760 legacy=0x7fb4bc0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4bb7fe700 1 -- 172.21.15.97:0/3436503964 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc0cff50 con 0x7fb4bc0cb2c0 2023-07-06T03:55:11.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.449+0000 7fb4bbfff700 1 --2- 172.21.15.97:0/3436503964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0b26f0 0x7fb4bc06c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:11.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.450+0000 7fb4bb7fe700 1 --2- 172.21.15.97:0/3436503964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4bc0cb2c0 0x7fb4bc0cf970 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009840 tx=0x7fb4ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47d9ce05398aeaa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:11.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.450+0000 7fb4ba7fc700 1 -- 172.21.15.97:0/3436503964 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac011830 con 0x7fb4bc0cb2c0 2023-07-06T03:55:11.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.450+0000 7fb4ba7fc700 1 -- 172.21.15.97:0/3436503964 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac011990 con 0x7fb4bc0cb2c0 2023-07-06T03:55:11.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.450+0000 7fb4ba7fc700 1 -- 172.21.15.97:0/3436503964 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac011d20 con 0x7fb4bc0cb2c0 2023-07-06T03:55:11.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.450+0000 7fb4c21ff700 1 -- 172.21.15.97:0/3436503964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4bc0cb2c0 msgr2=0x7fb4bc0cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.450+0000 7fb4c21ff700 1 --2- 172.21.15.97:0/3436503964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4bc0cb2c0 0x7fb4bc0cf970 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009840 tx=0x7fb4ac009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:11.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.450+0000 7fb4c21ff700 1 -- 172.21.15.97:0/3436503964 shutdown_connections 2023-07-06T03:55:11.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.450+0000 7fb4c21ff700 1 --2- 172.21.15.97:0/3436503964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4bc0cb2c0 0x7fb4bc0cf970 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.450+0000 7fb4c21ff700 1 --2- 172.21.15.97:0/3436503964 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0b26f0 0x7fb4bc06c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.450+0000 7fb4c21ff700 1 -- 172.21.15.97:0/3436503964 >> 172.21.15.97:0/3436503964 conn(0x7fb4bc0cc890 msgr2=0x7fb4bc0cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:11.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.451+0000 7fb4c21ff700 1 -- 172.21.15.97:0/3436503964 shutdown_connections 2023-07-06T03:55:11.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.451+0000 7fb4c21ff700 1 -- 172.21.15.97:0/3436503964 wait complete. 2023-07-06T03:55:11.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.452+0000 7fb4c21ff700 1 Processor -- start 2023-07-06T03:55:11.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.452+0000 7fb4c21ff700 1 -- start start 2023-07-06T03:55:11.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.453+0000 7fb4c21ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4bc06b760 0x7fb4bc0c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.453+0000 7fb4c21ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4bc0b26f0 0x7fb4bc070e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.453+0000 7fb4c21ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0cb2c0 0x7fb4bc071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.453+0000 7fb4c21ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc0aef40 con 0x7fb4bc06b760 2023-07-06T03:55:11.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.453+0000 7fb4c21ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc0aec10 con 0x7fb4bc0b26f0 2023-07-06T03:55:11.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.453+0000 7fb4c21ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc14b550 con 0x7fb4bc0cb2c0 2023-07-06T03:55:11.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.453+0000 7fb4bb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4bc06b760 0x7fb4bc0c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4bb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4bc06b760 0x7fb4bc0c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39984/0 (socket says 172.21.15.97:39984) 2023-07-06T03:55:11.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4bb7fe700 1 -- 172.21.15.97:0/1355213199 learned_addr learned my addr 172.21.15.97:0/1355213199 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:11.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4bbfff700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0cb2c0 0x7fb4bc071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4baffd700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4bc0b26f0 0x7fb4bc070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4bb7fe700 1 -- 172.21.15.97:0/1355213199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0cb2c0 msgr2=0x7fb4bc071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4bb7fe700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0cb2c0 0x7fb4bc071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4bb7fe700 1 -- 172.21.15.97:0/1355213199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4bc0b26f0 msgr2=0x7fb4bc070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4bb7fe700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4bc0b26f0 0x7fb4bc070e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4bb7fe700 1 -- 172.21.15.97:0/1355213199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac0094a0 con 0x7fb4bc06b760 2023-07-06T03:55:11.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4bbfff700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0cb2c0 0x7fb4bc071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:11.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4baffd700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4bc0b26f0 0x7fb4bc070e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:11.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4bb7fe700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4bc06b760 0x7fb4bc0c8bd0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000c00 tx=0x7fb4ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:11.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4a3fff700 1 -- 172.21.15.97:0/1355213199 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0038a0 con 0x7fb4bc06b760 2023-07-06T03:55:11.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4a3fff700 1 -- 172.21.15.97:0/1355213199 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac01e070 con 0x7fb4bc06b760 2023-07-06T03:55:11.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4c21ff700 1 -- 172.21.15.97:0/1355213199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc14b830 con 0x7fb4bc06b760 2023-07-06T03:55:11.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.454+0000 7fb4a3fff700 1 -- 172.21.15.97:0/1355213199 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac003c60 con 0x7fb4bc06b760 2023-07-06T03:55:11.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.455+0000 7fb4c21ff700 1 -- 172.21.15.97:0/1355213199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc0c9110 con 0x7fb4bc06b760 2023-07-06T03:55:11.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.456+0000 7fb4c21ff700 1 -- 172.21.15.97:0/1355213199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc04e340 con 0x7fb4bc06b760 2023-07-06T03:55:11.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.456+0000 7fb4a3fff700 1 -- 172.21.15.97:0/1355213199 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb4ac02d450 con 0x7fb4bc06b760 2023-07-06T03:55:11.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.457+0000 7fb4a3fff700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4a806ace0 0x7fb4a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.457+0000 7fb4a3fff700 1 -- 172.21.15.97:0/1355213199 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb4ac08a230 con 0x7fb4bc06b760 2023-07-06T03:55:11.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.458+0000 7fb4baffd700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4a806ace0 0x7fb4a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.458+0000 7fb4baffd700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4a806ace0 0x7fb4a806d190 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc1f5540 tx=0x7fb4b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:11.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.462+0000 7fb4a3fff700 1 -- 172.21.15.97:0/1355213199 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb4ac0556c0 con 0x7fb4bc06b760 2023-07-06T03:55:11.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.657+0000 7fb4c21ff700 1 -- 172.21.15.97:0/1355213199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4bc0b5010 con 0x7fb4bc06b760 2023-07-06T03:55:11.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.658+0000 7fb4a3fff700 1 -- 172.21.15.97:0/1355213199 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4ac058ce0 con 0x7fb4bc06b760 2023-07-06T03:55:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:55:11.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.660+0000 7fb4c21ff700 1 -- 172.21.15.97:0/1355213199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4a806ace0 msgr2=0x7fb4a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.660+0000 7fb4c21ff700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4a806ace0 0x7fb4a806d190 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc1f5540 tx=0x7fb4b0009040 comp rx=0 tx=0).stop 2023-07-06T03:55:11.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.660+0000 7fb4c21ff700 1 -- 172.21.15.97:0/1355213199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4bc06b760 msgr2=0x7fb4bc0c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.660+0000 7fb4c21ff700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4bc06b760 0x7fb4bc0c8bd0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000c00 tx=0x7fb4ac004750 comp rx=0 tx=0).stop 2023-07-06T03:55:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.660+0000 7fb4c21ff700 1 -- 172.21.15.97:0/1355213199 shutdown_connections 2023-07-06T03:55:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.660+0000 7fb4c21ff700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4a806ace0 0x7fb4a806d190 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.660+0000 7fb4c21ff700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4bc0cb2c0 0x7fb4bc071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.660+0000 7fb4c21ff700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4bc06b760 0x7fb4bc0c8bd0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.660+0000 7fb4c21ff700 1 --2- 172.21.15.97:0/1355213199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4bc0b26f0 0x7fb4bc070e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.660+0000 7fb4c21ff700 1 -- 172.21.15.97:0/1355213199 >> 172.21.15.97:0/1355213199 conn(0x7fb4bc0cc890 msgr2=0x7fb4bc070320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:11.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.661+0000 7fb4c21ff700 1 -- 172.21.15.97:0/1355213199 shutdown_connections 2023-07-06T03:55:11.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.661+0000 7fb4c21ff700 1 -- 172.21.15.97:0/1355213199 wait complete. 2023-07-06T03:55:11.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:55:11.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.876+0000 7f866ddff700 1 Processor -- start 2023-07-06T03:55:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.877+0000 7f866ddff700 1 -- start start 2023-07-06T03:55:11.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.878+0000 7f866ddff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f866815cae0 0x7f866815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.878+0000 7f866ddff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866815de00 con 0x7f86680cfbc0 2023-07-06T03:55:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.878+0000 7f866ddff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86680ceb20 con 0x7f866806b170 2023-07-06T03:55:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.878+0000 7f866ddff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86680b25a0 con 0x7f866815cae0 2023-07-06T03:55:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.878+0000 7f8667fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f86680cfbc0 0x7f866815e1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38332/0 (socket says 172.21.15.97:38332) 2023-07-06T03:55:11.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.878+0000 7f8667fff700 1 -- 172.21.15.97:0/1782485194 learned_addr learned my addr 172.21.15.97:0/1782485194 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:11.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.878+0000 7f865ffff700 1 --2- 172.21.15.97:0/1782485194 >> v2:172.21.15.146:3300/0 conn(0x7f866815cae0 0x7f866815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359478151 0 0) 0x7f8660001490 con 0x7f86680cfbc0 2023-07-06T03:55:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86540035b0 con 0x7f86680cfbc0 2023-07-06T03:55:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1265000029 0 0) 0x7f86540035b0 con 0x7f86680cfbc0 2023-07-06T03:55:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 >> v2:172.21.15.146:3300/0 conn(0x7f866815cae0 msgr2=0x7f866815d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 --2- 172.21.15.97:0/1782485194 >> v2:172.21.15.146:3300/0 conn(0x7f866815cae0 0x7f866815d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 >> v1:172.21.15.146:6789/0 conn(0x7f866806b170 legacy=0x7f866806b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86680cee00 con 0x7f86680cfbc0 2023-07-06T03:55:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f8660001b80 con 0x7f86680cfbc0 2023-07-06T03:55:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 >> v1:172.21.15.97:6789/0 conn(0x7f86680cfbc0 legacy=0x7f866815e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 0x7f8654003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8654004850 0x7f8654004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8654005200 0x7f865400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865400c0e0 con 0x7f8654003a00 2023-07-06T03:55:11.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865400c250 con 0x7f8654004850 2023-07-06T03:55:11.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865400c410 con 0x7f8654005200 2023-07-06T03:55:11.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f86677fe700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 0x7f8654003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.879+0000 7f865ffff700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8654004850 0x7f8654004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f8667fff700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8654005200 0x7f865400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f86677fe700 1 -- 172.21.15.97:0/1782485194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8654005200 msgr2=0x7f865400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:11.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f86677fe700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8654005200 0x7f865400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f86677fe700 1 -- 172.21.15.97:0/1782485194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8654004850 msgr2=0x7f8654004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:11.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f86677fe700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8654004850 0x7f8654004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f86677fe700 1 -- 172.21.15.97:0/1782485194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540035b0 con 0x7f8654003a00 2023-07-06T03:55:11.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f8667fff700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8654005200 0x7f865400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:11.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f8667fff700 1 -- 172.21.15.97:0/1782485194 reap_dead start 2023-07-06T03:55:11.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f86677fe700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 0x7f8654003e10 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f8658001820 tx=0x7f8654005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361029a2e5fb2cdf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:11.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86580012e0 con 0x7f8654003a00 2023-07-06T03:55:11.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86580052d0 con 0x7f8654003a00 2023-07-06T03:55:11.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.880+0000 7f8666ffd700 1 -- 172.21.15.97:0/1782485194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86580036d0 con 0x7f8654003a00 2023-07-06T03:55:11.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.881+0000 7f866ddff700 1 -- 172.21.15.97:0/1782485194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 msgr2=0x7f8654003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.881+0000 7f866ddff700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 0x7f8654003e10 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f8658001820 tx=0x7f8654005880 comp rx=0 tx=0).stop 2023-07-06T03:55:11.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.881+0000 7f866ddff700 1 -- 172.21.15.97:0/1782485194 shutdown_connections 2023-07-06T03:55:11.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.881+0000 7f866ddff700 1 --2- 172.21.15.97:0/1782485194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 0x7f8654003e10 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.881+0000 7f866ddff700 1 -- 172.21.15.97:0/1782485194 >> 172.21.15.97:0/1782485194 conn(0x7f8668099f10 msgr2=0x7f86680d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:11.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.881+0000 7f866ddff700 1 -- 172.21.15.97:0/1782485194 shutdown_connections 2023-07-06T03:55:11.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.881+0000 7f866ddff700 1 -- 172.21.15.97:0/1782485194 wait complete. 2023-07-06T03:55:11.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.883+0000 7f866ddff700 1 Processor -- start 2023-07-06T03:55:11.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.884+0000 7f866ddff700 1 -- start start 2023-07-06T03:55:11.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f866ddff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 0x7f86680c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f866ddff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f866815b680 0x7f866814ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f866ddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f866814af40 0x7f866806f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f866ddff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86680c7100 con 0x7f8654003a00 2023-07-06T03:55:11.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f866ddff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86680c73e0 con 0x7f866815b680 2023-07-06T03:55:11.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f866ddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866809a880 con 0x7f866814af40 2023-07-06T03:55:11.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f86677fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 0x7f86680c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f86677fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 0x7f86680c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39988/0 (socket says 172.21.15.97:39988) 2023-07-06T03:55:11.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f86677fe700 1 -- 172.21.15.97:0/3740426466 learned_addr learned my addr 172.21.15.97:0/3740426466 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:11.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f8667fff700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f866814af40 0x7f866806f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f865ffff700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f866815b680 0x7f866814ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f86677fe700 1 -- 172.21.15.97:0/3740426466 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f866814af40 msgr2=0x7f866806f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f86677fe700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f866814af40 0x7f866806f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f86677fe700 1 -- 172.21.15.97:0/3740426466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f866815b680 msgr2=0x7f866814ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:11.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f86677fe700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f866815b680 0x7f866814ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:11.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f86677fe700 1 -- 172.21.15.97:0/3740426466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866809ab60 con 0x7f8654003a00 2023-07-06T03:55:11.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.885+0000 7f865ffff700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f866815b680 0x7f866814ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:11.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.886+0000 7f86677fe700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 0x7f86680c68d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f865400c220 tx=0x7f8658001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:11.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.886+0000 7f8664ff9700 1 -- 172.21.15.97:0/3740426466 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658001280 con 0x7f8654003a00 2023-07-06T03:55:11.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.886+0000 7f8664ff9700 1 -- 172.21.15.97:0/3740426466 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86580050d0 con 0x7f8654003a00 2023-07-06T03:55:11.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.886+0000 7f866ddff700 1 -- 172.21.15.97:0/3740426466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866809ae40 con 0x7f8654003a00 2023-07-06T03:55:11.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.886+0000 7f8664ff9700 1 -- 172.21.15.97:0/3740426466 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865801d420 con 0x7f8654003a00 2023-07-06T03:55:11.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.886+0000 7f866ddff700 1 -- 172.21.15.97:0/3740426466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866814bfd0 con 0x7f8654003a00 2023-07-06T03:55:11.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.887+0000 7f866ddff700 1 -- 172.21.15.97:0/3740426466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866809b1b0 con 0x7f8654003a00 2023-07-06T03:55:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.888+0000 7f8664ff9700 1 -- 172.21.15.97:0/3740426466 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f865801d580 con 0x7f8654003a00 2023-07-06T03:55:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.889+0000 7f8664ff9700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f864806ace0 0x7f864806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.889+0000 7f8664ff9700 1 -- 172.21.15.97:0/3740426466 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8658089a50 con 0x7f8654003a00 2023-07-06T03:55:11.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.889+0000 7f865ffff700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f864806ace0 0x7f864806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:11.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.890+0000 7f865ffff700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f864806ace0 0x7f864806d190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f86680d1960 tx=0x7f8650008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:11.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:11.893+0000 7f8664ff9700 1 -- 172.21.15.97:0/3740426466 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8658054e40 con 0x7f8654003a00 2023-07-06T03:55:12.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.089+0000 7f866ddff700 1 -- 172.21.15.97:0/3740426466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f866804e340 con 0x7f8654003a00 2023-07-06T03:55:12.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.089+0000 7f8664ff9700 1 -- 172.21.15.97:0/3740426466 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v120) v1 ==== 92+0+10 (secure 0 0 0) 0x7f865802b020 con 0x7f8654003a00 2023-07-06T03:55:12.092 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:55:12.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 -- 172.21.15.97:0/3740426466 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f864806ace0 msgr2=0x7f864806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f864806ace0 0x7f864806d190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f86680d1960 tx=0x7f8650008040 comp rx=0 tx=0).stop 2023-07-06T03:55:12.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 -- 172.21.15.97:0/3740426466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 msgr2=0x7f86680c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 0x7f86680c68d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f865400c220 tx=0x7f8658001170 comp rx=0 tx=0).stop 2023-07-06T03:55:12.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 -- 172.21.15.97:0/3740426466 shutdown_connections 2023-07-06T03:55:12.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f864806ace0 0x7f864806d190 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f866814af40 0x7f866806f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8654003a00 0x7f86680c68d0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 --2- 172.21.15.97:0/3740426466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f866815b680 0x7f866814ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 -- 172.21.15.97:0/3740426466 >> 172.21.15.97:0/3740426466 conn(0x7f8668099f10 msgr2=0x7f866806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:12.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 -- 172.21.15.97:0/3740426466 shutdown_connections 2023-07-06T03:55:12.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.092+0000 7f866ddff700 1 -- 172.21.15.97:0/3740426466 wait complete. 2023-07-06T03:55:12.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T03:55:12.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.305+0000 7f95ab1ff700 1 Processor -- start 2023-07-06T03:55:12.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.305+0000 7f95ab1ff700 1 -- start start 2023-07-06T03:55:12.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.306+0000 7f95ab1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95a415c520 0x7f95a40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:12.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.306+0000 7f95ab1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a40cdae0 con 0x7f95a4160d00 2023-07-06T03:55:12.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.306+0000 7f95ab1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a40b6640 con 0x7f95a4160180 2023-07-06T03:55:12.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.306+0000 7f95ab1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a40cd8d0 con 0x7f95a415c520 2023-07-06T03:55:12.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.306+0000 7f95a8f9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95a415c520 0x7f95a40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:12.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.307+0000 7f95a8f9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95a415c520 0x7f95a40cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39990/0 (socket says 172.21.15.97:39990) 2023-07-06T03:55:12.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.307+0000 7f95a8f9b700 1 -- 172.21.15.97:0/1874730378 learned_addr learned my addr 172.21.15.97:0/1874730378 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:12.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.307+0000 7f95a979c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f95a4160d00 0x7f95a40b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38340/0 (socket says 172.21.15.97:38340) 2023-07-06T03:55:12.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.307+0000 7f95a8f9b700 1 -- 172.21.15.97:0/1874730378 >> v1:172.21.15.146:6789/0 conn(0x7f95a4160180 legacy=0x7f95a41605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.307+0000 7f95a8f9b700 1 -- 172.21.15.97:0/1874730378 >> v1:172.21.15.97:6789/0 conn(0x7f95a4160d00 legacy=0x7f95a40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.307+0000 7f95a8f9b700 1 -- 172.21.15.97:0/1874730378 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a40b6920 con 0x7f95a415c520 2023-07-06T03:55:12.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.307+0000 7f95a8f9b700 1 --2- 172.21.15.97:0/1874730378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95a415c520 0x7f95a40cd390 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f95a00098f0 tx=0x7f95a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45061a7c1352693e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:12.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.307+0000 7f959b7fe700 1 -- 172.21.15.97:0/1874730378 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a000b8a0 con 0x7f95a415c520 2023-07-06T03:55:12.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.307+0000 7f959b7fe700 1 -- 172.21.15.97:0/1874730378 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a000ba00 con 0x7f95a415c520 2023-07-06T03:55:12.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.307+0000 7f959b7fe700 1 -- 172.21.15.97:0/1874730378 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a000bd90 con 0x7f95a415c520 2023-07-06T03:55:12.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.308+0000 7f95ab1ff700 1 -- 172.21.15.97:0/1874730378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95a415c520 msgr2=0x7f95a40cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.308+0000 7f95ab1ff700 1 --2- 172.21.15.97:0/1874730378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95a415c520 0x7f95a40cd390 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f95a00098f0 tx=0x7f95a0009c00 comp rx=0 tx=0).stop 2023-07-06T03:55:12.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.308+0000 7f95ab1ff700 1 -- 172.21.15.97:0/1874730378 shutdown_connections 2023-07-06T03:55:12.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.308+0000 7f95ab1ff700 1 --2- 172.21.15.97:0/1874730378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95a415c520 0x7f95a40cd390 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.308+0000 7f95ab1ff700 1 -- 172.21.15.97:0/1874730378 >> 172.21.15.97:0/1874730378 conn(0x7f95a4071790 msgr2=0x7f95a4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:12.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.308+0000 7f95ab1ff700 1 -- 172.21.15.97:0/1874730378 shutdown_connections 2023-07-06T03:55:12.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.308+0000 7f95ab1ff700 1 -- 172.21.15.97:0/1874730378 wait complete. 2023-07-06T03:55:12.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.310+0000 7f95ab1ff700 1 Processor -- start 2023-07-06T03:55:12.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.310+0000 7f95ab1ff700 1 -- start start 2023-07-06T03:55:12.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f95ab1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95a415c520 0x7f95a414d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:12.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f95ab1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95a4160180 0x7f95a415e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:12.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f95ab1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a4160d00 0x7f95a415e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f95ab1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a414da70 con 0x7f95a4160180 2023-07-06T03:55:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f95ab1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40d1ba0 con 0x7f95a415c520 2023-07-06T03:55:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f95ab1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40d0f90 con 0x7f95a4160d00 2023-07-06T03:55:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f959bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95a4160180 0x7f95a415e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:12.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f959bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95a4160180 0x7f95a415e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39994/0 (socket says 172.21.15.97:39994) 2023-07-06T03:55:12.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f959bfff700 1 -- 172.21.15.97:0/65907282 learned_addr learned my addr 172.21.15.97:0/65907282 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:12.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f95a8f9b700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95a415c520 0x7f95a414d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:12.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f95a979c700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a4160d00 0x7f95a415e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:12.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f959bfff700 1 -- 172.21.15.97:0/65907282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a4160d00 msgr2=0x7f95a415e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f959bfff700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a4160d00 0x7f95a415e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f959bfff700 1 -- 172.21.15.97:0/65907282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95a415c520 msgr2=0x7f95a414d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f959bfff700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95a415c520 0x7f95a414d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f959bfff700 1 -- 172.21.15.97:0/65907282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a0009550 con 0x7f95a4160180 2023-07-06T03:55:12.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.311+0000 7f95a8f9b700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95a415c520 0x7f95a414d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:12.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.312+0000 7f959bfff700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95a4160180 0x7f95a415e470 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f959400c4e0 tx=0x7f959400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:12.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.312+0000 7f95997fa700 1 -- 172.21.15.97:0/65907282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95940052d0 con 0x7f95a4160180 2023-07-06T03:55:12.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.312+0000 7f95997fa700 1 -- 172.21.15.97:0/65907282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9594005430 con 0x7f95a4160180 2023-07-06T03:55:12.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.312+0000 7f95ab1ff700 1 -- 172.21.15.97:0/65907282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a40d1270 con 0x7f95a4160180 2023-07-06T03:55:12.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.312+0000 7f95997fa700 1 -- 172.21.15.97:0/65907282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95940038c0 con 0x7f95a4160180 2023-07-06T03:55:12.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.312+0000 7f95ab1ff700 1 -- 172.21.15.97:0/65907282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a40d1830 con 0x7f95a4160180 2023-07-06T03:55:12.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.313+0000 7f95ab1ff700 1 -- 172.21.15.97:0/65907282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a40d2240 con 0x7f95a4160180 2023-07-06T03:55:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.314+0000 7f95997fa700 1 -- 172.21.15.97:0/65907282 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9594003a20 con 0x7f95a4160180 2023-07-06T03:55:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.315+0000 7f95997fa700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f959006ace0 0x7f959006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.315+0000 7f95997fa700 1 -- 172.21.15.97:0/65907282 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f95940886b0 con 0x7f95a4160180 2023-07-06T03:55:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.315+0000 7f95a8f9b700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f959006ace0 0x7f959006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.316+0000 7f95a8f9b700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f959006ace0 0x7f959006d190 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f95a0014040 tx=0x7f95a0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:12.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.319+0000 7f95997fa700 1 -- 172.21.15.97:0/65907282 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9594053a10 con 0x7f95a4160180 2023-07-06T03:55:12.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.517+0000 7f95ab1ff700 1 -- 172.21.15.97:0/65907282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f95a40d19c0 con 0x7f95a4160180 2023-07-06T03:55:12.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.517+0000 7f95997fa700 1 -- 172.21.15.97:0/65907282 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v120) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9594057030 con 0x7f95a4160180 2023-07-06T03:55:12.520 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:12.520 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-07-06T03:55:12.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:12.521 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-07-06T03:55:12.521 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-07-06T03:55:12.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:12.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:12.521 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 2, 2023-07-06T03:55:12.521 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": "rbd" 2023-07-06T03:55:12.522 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:12.522 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-07-06T03:55:12.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 -- 172.21.15.97:0/65907282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f959006ace0 msgr2=0x7f959006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f959006ace0 0x7f959006d190 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f95a0014040 tx=0x7f95a0008030 comp rx=0 tx=0).stop 2023-07-06T03:55:12.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 -- 172.21.15.97:0/65907282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95a4160180 msgr2=0x7f95a415e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95a4160180 0x7f95a415e470 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f959400c4e0 tx=0x7f959400c8a0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 -- 172.21.15.97:0/65907282 shutdown_connections 2023-07-06T03:55:12.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f959006ace0 0x7f959006d190 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95a4160d00 0x7f95a415e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95a415c520 0x7f95a414d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 --2- 172.21.15.97:0/65907282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95a4160180 0x7f95a415e470 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 -- 172.21.15.97:0/65907282 >> 172.21.15.97:0/65907282 conn(0x7f95a4071790 msgr2=0x7f95a40b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 -- 172.21.15.97:0/65907282 shutdown_connections 2023-07-06T03:55:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.520+0000 7f95ab1ff700 1 -- 172.21.15.97:0/65907282 wait complete. 2023-07-06T03:55:12.540 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T03:55:12.541 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T03:55:12.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.736+0000 7fd010dff700 1 Processor -- start 2023-07-06T03:55:12.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.737+0000 7fd010dff700 1 -- start start 2023-07-06T03:55:12.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.738+0000 7fd010dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd00c15c670 0x7fd00c0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:12.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.738+0000 7fd010dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd00c0cd4c0 0x7fd00c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:12.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.738+0000 7fd010dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c0cdc40 con 0x7fd00c0716a0 2023-07-06T03:55:12.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.738+0000 7fd010dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00c0cd960 con 0x7fd00c15c670 2023-07-06T03:55:12.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.738+0000 7fd010dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00c070bc0 con 0x7fd00c0cd4c0 2023-07-06T03:55:12.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.738+0000 7fd00a59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd00c15c670 0x7fd00c0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:12.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.738+0000 7fd00a59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd00c15c670 0x7fd00c0cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40010/0 (socket says 172.21.15.97:40010) 2023-07-06T03:55:12.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.738+0000 7fd00a59c700 1 -- 172.21.15.97:0/519256389 learned_addr learned my addr 172.21.15.97:0/519256389 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:12.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.739+0000 7fd00a59c700 1 -- 172.21.15.97:0/519256389 >> v2:172.21.15.146:3300/0 conn(0x7fd00c0cd4c0 msgr2=0x7fd00c06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:12.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.739+0000 7fd009d9b700 1 --2- 172.21.15.97:0/519256389 >> v2:172.21.15.146:3300/0 conn(0x7fd00c0cd4c0 0x7fd00c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:12.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.739+0000 7fd00a59c700 1 --2- 172.21.15.97:0/519256389 >> v2:172.21.15.146:3300/0 conn(0x7fd00c0cd4c0 0x7fd00c06a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.739+0000 7fd00a59c700 1 -- 172.21.15.97:0/519256389 >> v1:172.21.15.146:6789/0 conn(0x7fd00c0716a0 legacy=0x7fd00c0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.739+0000 7fd00a59c700 1 -- 172.21.15.97:0/519256389 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c070ea0 con 0x7fd00c15c670 2023-07-06T03:55:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.739+0000 7fd009d9b700 1 --2- 172.21.15.97:0/519256389 >> v2:172.21.15.146:3300/0 conn(0x7fd00c0cd4c0 0x7fd00c06a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.739+0000 7fd00a59c700 1 --2- 172.21.15.97:0/519256389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00c15c670 0x7fd00c0cbe00 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0cc340 tx=0x7fcffc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adadc82221a7ff2b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.739+0000 7fd00959a700 1 -- 172.21.15.97:0/519256389 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc004230 con 0x7fd00c15c670 2023-07-06T03:55:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.739+0000 7fd00959a700 1 -- 172.21.15.97:0/519256389 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc004390 con 0x7fd00c15c670 2023-07-06T03:55:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.739+0000 7fd00959a700 1 -- 172.21.15.97:0/519256389 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc0036a0 con 0x7fd00c15c670 2023-07-06T03:55:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.740+0000 7fd010dff700 1 -- 172.21.15.97:0/519256389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00c15c670 msgr2=0x7fd00c0cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.740+0000 7fd010dff700 1 --2- 172.21.15.97:0/519256389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00c15c670 0x7fd00c0cbe00 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0cc340 tx=0x7fcffc009bd0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.740+0000 7fd010dff700 1 -- 172.21.15.97:0/519256389 shutdown_connections 2023-07-06T03:55:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.740+0000 7fd010dff700 1 --2- 172.21.15.97:0/519256389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00c15c670 0x7fd00c0cbe00 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.740+0000 7fd010dff700 1 --2- 172.21.15.97:0/519256389 >> v2:172.21.15.146:3300/0 conn(0x7fd00c0cd4c0 0x7fd00c06a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.740+0000 7fd010dff700 1 -- 172.21.15.97:0/519256389 >> 172.21.15.97:0/519256389 conn(0x7fd00c0d05b0 msgr2=0x7fd00c0cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.740+0000 7fd010dff700 1 -- 172.21.15.97:0/519256389 shutdown_connections 2023-07-06T03:55:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.740+0000 7fd010dff700 1 -- 172.21.15.97:0/519256389 wait complete. 2023-07-06T03:55:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.742+0000 7fd010dff700 1 Processor -- start 2023-07-06T03:55:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.742+0000 7fd010dff700 1 -- start start 2023-07-06T03:55:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd010dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00c0716a0 0x7fd00c0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd010dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd00c0cd4c0 0x7fd00c1463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd010dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd00c15c670 0x7fd00c06dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd010dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c1f2100 con 0x7fd00c0716a0 2023-07-06T03:55:12.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd010dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0b3b40 con 0x7fd00c0cd4c0 2023-07-06T03:55:12.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd010dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0b32a0 con 0x7fd00c15c670 2023-07-06T03:55:12.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd00a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00c0716a0 0x7fd00c0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:12.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd00a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00c0716a0 0x7fd00c0b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40016/0 (socket says 172.21.15.97:40016) 2023-07-06T03:55:12.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd00a59c700 1 -- 172.21.15.97:0/3818429243 learned_addr learned my addr 172.21.15.97:0/3818429243 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:12.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd00a59c700 1 -- 172.21.15.97:0/3818429243 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd00c15c670 msgr2=0x7fd00c06dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd00a59c700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd00c15c670 0x7fd00c06dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd00a59c700 1 -- 172.21.15.97:0/3818429243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd00c0cd4c0 msgr2=0x7fd00c1463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:12.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd00a59c700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd00c0cd4c0 0x7fd00c1463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:12.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.744+0000 7fd00a59c700 1 -- 172.21.15.97:0/3818429243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc009550 con 0x7fd00c0716a0 2023-07-06T03:55:12.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.745+0000 7fd00a59c700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00c0716a0 0x7fd00c0b2950 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fd00c06c370 tx=0x7fcffc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.745+0000 7fcffaffd700 1 -- 172.21.15.97:0/3818429243 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc011630 con 0x7fd00c0716a0 2023-07-06T03:55:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.745+0000 7fcffaffd700 1 -- 172.21.15.97:0/3818429243 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc011790 con 0x7fd00c0716a0 2023-07-06T03:55:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.745+0000 7fcffaffd700 1 -- 172.21.15.97:0/3818429243 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc011b40 con 0x7fd00c0716a0 2023-07-06T03:55:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.745+0000 7fd010dff700 1 -- 172.21.15.97:0/3818429243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c0b3580 con 0x7fd00c0716a0 2023-07-06T03:55:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.745+0000 7fd010dff700 1 -- 172.21.15.97:0/3818429243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c0b2e90 con 0x7fd00c0716a0 2023-07-06T03:55:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.746+0000 7fd010dff700 1 -- 172.21.15.97:0/3818429243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c0b3070 con 0x7fd00c0716a0 2023-07-06T03:55:12.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.747+0000 7fcffaffd700 1 -- 172.21.15.97:0/3818429243 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcffc011ca0 con 0x7fd00c0716a0 2023-07-06T03:55:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.748+0000 7fcffaffd700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfec06ace0 0x7fcfec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.748+0000 7fcffaffd700 1 -- 172.21.15.97:0/3818429243 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcffc08a890 con 0x7fd00c0716a0 2023-07-06T03:55:12.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.748+0000 7fd009d9b700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfec06ace0 0x7fcfec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:12.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.749+0000 7fd009d9b700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfec06ace0 0x7fcfec06d190 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd00c1e6b90 tx=0x7fd000008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:12.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.752+0000 7fcffaffd700 1 -- 172.21.15.97:0/3818429243 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcffc055c80 con 0x7fd00c0716a0 2023-07-06T03:55:12.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:12.949+0000 7fd010dff700 1 -- 172.21.15.97:0/3818429243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd00c0cc160 con 0x7fd00c0716a0 2023-07-06T03:55:13.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.305+0000 7fcffaffd700 1 -- 172.21.15.97:0/3818429243 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v121) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcffc0592a0 con 0x7fd00c0716a0 2023-07-06T03:55:13.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.310+0000 7fd010dff700 1 -- 172.21.15.97:0/3818429243 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfec06ace0 msgr2=0x7fcfec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.310+0000 7fd010dff700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfec06ace0 0x7fcfec06d190 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd00c1e6b90 tx=0x7fd000008040 comp rx=0 tx=0).stop 2023-07-06T03:55:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.310+0000 7fd010dff700 1 -- 172.21.15.97:0/3818429243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00c0716a0 msgr2=0x7fd00c0b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.310+0000 7fd010dff700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00c0716a0 0x7fd00c0b2950 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fd00c06c370 tx=0x7fcffc004750 comp rx=0 tx=0).stop 2023-07-06T03:55:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.311+0000 7fd010dff700 1 -- 172.21.15.97:0/3818429243 shutdown_connections 2023-07-06T03:55:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.311+0000 7fd010dff700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfec06ace0 0x7fcfec06d190 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.311+0000 7fd010dff700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd00c15c670 0x7fd00c06dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:13.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.311+0000 7fd010dff700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd00c0716a0 0x7fd00c0b2950 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.311+0000 7fd010dff700 1 --2- 172.21.15.97:0/3818429243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd00c0cd4c0 0x7fd00c1463a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.311+0000 7fd010dff700 1 -- 172.21.15.97:0/3818429243 >> 172.21.15.97:0/3818429243 conn(0x7fd00c0d05b0 msgr2=0x7fd00c06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.311+0000 7fd010dff700 1 -- 172.21.15.97:0/3818429243 shutdown_connections 2023-07-06T03:55:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.311+0000 7fd010dff700 1 -- 172.21.15.97:0/3818429243 wait complete. 2023-07-06T03:55:13.315 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-07-06T03:55:13.332 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T03:55:13.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.532+0000 7f7021dff700 1 Processor -- start 2023-07-06T03:55:13.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.532+0000 7f7021dff700 1 -- start start 2023-07-06T03:55:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.533+0000 7f7021dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c0c51f0 0x7f701c071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.533+0000 7f7021dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701c0caa20 con 0x7f701c15a0d0 2023-07-06T03:55:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.533+0000 7f7021dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701c0cad00 con 0x7f701c15a4a0 2023-07-06T03:55:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.533+0000 7f7021dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c0c39e0 con 0x7f701c0c51f0 2023-07-06T03:55:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.533+0000 7f701b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f701c15a0d0 0x7f701c0ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38344/0 (socket says 172.21.15.97:38344) 2023-07-06T03:55:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.533+0000 7f701b7fe700 1 -- 172.21.15.97:0/1761587379 learned_addr learned my addr 172.21.15.97:0/1761587379 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701bfff700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c0c51f0 0x7f701c071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36270170 0 0) 0x7f701c0caa20 con 0x7f701c15a0d0 2023-07-06T03:55:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70040035b0 con 0x7f701c15a0d0 2023-07-06T03:55:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882935582 0 0) 0x7f701c0cad00 con 0x7f701c15a4a0 2023-07-06T03:55:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f701c0caa20 con 0x7f701c15a4a0 2023-07-06T03:55:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1384782886 0 0) 0x7f70040035b0 con 0x7f701c15a0d0 2023-07-06T03:55:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c0c51f0 msgr2=0x7f701c071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c0c51f0 0x7f701c071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 >> v1:172.21.15.146:6789/0 conn(0x7f701c15a4a0 legacy=0x7f701c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c0c3cc0 con 0x7f701c15a0d0 2023-07-06T03:55:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f700c001670 con 0x7f701c15a0d0 2023-07-06T03:55:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 >> v1:172.21.15.97:6789/0 conn(0x7f701c15a0d0 legacy=0x7f701c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:13.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7004003bc0 0x7f7004003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:13.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c0c51f0 0x7f7004004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:13.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7004004d90 0x7f7004005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:13.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c0c39e0 con 0x7f7004003bc0 2023-07-06T03:55:13.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700400bb10 con 0x7f7004004d90 2023-07-06T03:55:13.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.534+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700400bcd0 con 0x7f701c0c51f0 2023-07-06T03:55:13.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701b7fe700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7004003bc0 0x7f7004003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:13.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701affd700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c0c51f0 0x7f7004004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:13.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701bfff700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7004004d90 0x7f7004005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:13.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701b7fe700 1 -- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c0c51f0 msgr2=0x7f7004004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:13.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701b7fe700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c0c51f0 0x7f7004004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:13.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701b7fe700 1 -- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7004004d90 msgr2=0x7f7004005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:13.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701b7fe700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7004004d90 0x7f7004005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:13.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701b7fe700 1 -- 172.21.15.97:0/1761587379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70040035b0 con 0x7f7004003bc0 2023-07-06T03:55:13.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701affd700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c0c51f0 0x7f7004004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:13.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701b7fe700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7004003bc0 0x7f7004003fb0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f700c00c210 tx=0x7f7004005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa7560ad569f0f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:13.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c00cdb0 con 0x7f7004003bc0 2023-07-06T03:55:13.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c001af0 con 0x7f7004003bc0 2023-07-06T03:55:13.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.535+0000 7f701a7fc700 1 -- 172.21.15.97:0/1761587379 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c005ba0 con 0x7f7004003bc0 2023-07-06T03:55:13.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.536+0000 7f7021dff700 1 -- 172.21.15.97:0/1761587379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7004003bc0 msgr2=0x7f7004003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:13.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.536+0000 7f7021dff700 1 --2- 172.21.15.97:0/1761587379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7004003bc0 0x7f7004003fb0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f700c00c210 tx=0x7f7004005880 comp rx=0 tx=0).stop 2023-07-06T03:55:13.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.536+0000 7f701bfff700 1 -- 172.21.15.97:0/1761587379 reap_dead start 2023-07-06T03:55:13.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.536+0000 7f7021dff700 1 -- 172.21.15.97:0/1761587379 shutdown_connections 2023-07-06T03:55:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.536+0000 7f7021dff700 1 -- 172.21.15.97:0/1761587379 >> 172.21.15.97:0/1761587379 conn(0x7f701c0b05d0 msgr2=0x7f701c0b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.536+0000 7f7021dff700 1 -- 172.21.15.97:0/1761587379 shutdown_connections 2023-07-06T03:55:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.536+0000 7f7021dff700 1 -- 172.21.15.97:0/1761587379 wait complete. 2023-07-06T03:55:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.538+0000 7f7021dff700 1 Processor -- start 2023-07-06T03:55:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.538+0000 7f7021dff700 1 -- start start 2023-07-06T03:55:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.539+0000 7f7021dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7004003bc0 0x7f701c14f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.539+0000 7f7021dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c1e7870 0x7f701c0ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.539+0000 7f7021dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c0aef00 0x7f701c0b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.539+0000 7f7021dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c1e7450 con 0x7f701c0aef00 2023-07-06T03:55:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.539+0000 7f7021dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c152370 con 0x7f7004003bc0 2023-07-06T03:55:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.539+0000 7f7021dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c0af370 con 0x7f701c1e7870 2023-07-06T03:55:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.539+0000 7f701bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c0aef00 0x7f701c0b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.539+0000 7f701bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c0aef00 0x7f701c0b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40026/0 (socket says 172.21.15.97:40026) 2023-07-06T03:55:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.539+0000 7f701bfff700 1 -- 172.21.15.97:0/1085605820 learned_addr learned my addr 172.21.15.97:0/1085605820 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.539+0000 7f701affd700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c1e7870 0x7f701c0ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f701b7fe700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7004003bc0 0x7f701c14f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f701bfff700 1 -- 172.21.15.97:0/1085605820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c1e7870 msgr2=0x7f701c0ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f701bfff700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c1e7870 0x7f701c0ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f701bfff700 1 -- 172.21.15.97:0/1085605820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7004003bc0 msgr2=0x7f701c14f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f701bfff700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7004003bc0 0x7f701c14f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f701bfff700 1 -- 172.21.15.97:0/1085605820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c1e6ab0 con 0x7f701c0aef00 2023-07-06T03:55:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f701b7fe700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7004003bc0 0x7f701c14f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f701bfff700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c0aef00 0x7f701c0b3a30 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f7014007d10 tx=0x7f70140114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f7003fff700 1 -- 172.21.15.97:0/1085605820 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701400e070 con 0x7f701c0aef00 2023-07-06T03:55:13.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f7003fff700 1 -- 172.21.15.97:0/1085605820 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7014003c30 con 0x7f701c0aef00 2023-07-06T03:55:13.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f7003fff700 1 -- 172.21.15.97:0/1085605820 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014003de0 con 0x7f701c0aef00 2023-07-06T03:55:13.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f7021dff700 1 -- 172.21.15.97:0/1085605820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701c1e6d90 con 0x7f701c0aef00 2023-07-06T03:55:13.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.540+0000 7f7021dff700 1 -- 172.21.15.97:0/1085605820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c1e6760 con 0x7f701c0aef00 2023-07-06T03:55:13.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.541+0000 7f7021dff700 1 -- 172.21.15.97:0/1085605820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701c04e340 con 0x7f701c0aef00 2023-07-06T03:55:13.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.543+0000 7f7003fff700 1 -- 172.21.15.97:0/1085605820 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70140119e0 con 0x7f701c0aef00 2023-07-06T03:55:13.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.543+0000 7f7003fff700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f700806ace0 0x7f700806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:13.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.543+0000 7f7003fff700 1 -- 172.21.15.97:0/1085605820 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f70140897a0 con 0x7f701c0aef00 2023-07-06T03:55:13.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.544+0000 7f701b7fe700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f700806ace0 0x7f700806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:13.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.544+0000 7f701b7fe700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f700806ace0 0x7f700806d190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f700c015040 tx=0x7f700c00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:13.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.548+0000 7f7003fff700 1 -- 172.21.15.97:0/1085605820 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7014054a20 con 0x7f701c0aef00 2023-07-06T03:55:13.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:13.742+0000 7f7021dff700 1 -- 172.21.15.97:0/1085605820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f701c1e7590 con 0x7f701c0aef00 2023-07-06T03:55:14.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.315+0000 7f7003fff700 1 -- 172.21.15.97:0/1085605820 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v122) v1 ==== 170+0+0 (secure 0 0 0) 0x7f701c1e7590 con 0x7f701c0aef00 2023-07-06T03:55:14.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.319+0000 7f7021dff700 1 -- 172.21.15.97:0/1085605820 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f700806ace0 msgr2=0x7f700806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:14.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.319+0000 7f7021dff700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f700806ace0 0x7f700806d190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f700c015040 tx=0x7f700c00a000 comp rx=0 tx=0).stop 2023-07-06T03:55:14.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.319+0000 7f7021dff700 1 -- 172.21.15.97:0/1085605820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c0aef00 msgr2=0x7f701c0b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:14.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.319+0000 7f7021dff700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c0aef00 0x7f701c0b3a30 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f7014007d10 tx=0x7f70140114b0 comp rx=0 tx=0).stop 2023-07-06T03:55:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.320+0000 7f7021dff700 1 -- 172.21.15.97:0/1085605820 shutdown_connections 2023-07-06T03:55:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.320+0000 7f7021dff700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f700806ace0 0x7f700806d190 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.320+0000 7f7021dff700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c0aef00 0x7f701c0b3a30 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.320+0000 7f7021dff700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7004003bc0 0x7f701c14f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.320+0000 7f7021dff700 1 --2- 172.21.15.97:0/1085605820 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f701c1e7870 0x7f701c0ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.320+0000 7f7021dff700 1 -- 172.21.15.97:0/1085605820 >> 172.21.15.97:0/1085605820 conn(0x7f701c0b05d0 msgr2=0x7f701c0ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:14.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.321+0000 7f7021dff700 1 -- 172.21.15.97:0/1085605820 shutdown_connections 2023-07-06T03:55:14.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.321+0000 7f7021dff700 1 -- 172.21.15.97:0/1085605820 wait complete. 2023-07-06T03:55:14.336 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-07-06T03:55:14.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T03:55:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.550+0000 7f6f47fff700 1 Processor -- start 2023-07-06T03:55:14.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.550+0000 7f6f47fff700 1 -- start start 2023-07-06T03:55:14.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f47fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6f4006b360 0x7f6f400b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:14.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f47fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f400d2210 0x7f6f400d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:14.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f47fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f400cff90 con 0x7f6f4015c610 2023-07-06T03:55:14.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f47fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f400d0780 con 0x7f6f4006b360 2023-07-06T03:55:14.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f47fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4015c9e0 con 0x7f6f400d2210 2023-07-06T03:55:14.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f45d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6f4015c610 0x7f6f400b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38356/0 (socket says 172.21.15.97:38356) 2023-07-06T03:55:14.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f45d9b700 1 -- 172.21.15.97:0/2195309503 learned_addr learned my addr 172.21.15.97:0/2195309503 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:14.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f4559a700 1 --2- 172.21.15.97:0/2195309503 >> v2:172.21.15.97:3300/0 conn(0x7f6f4006b360 0x7f6f400b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:14.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f44d99700 1 -- 172.21.15.97:0/2195309503 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554693290 0 0) 0x7f6f400cff90 con 0x7f6f4015c610 2023-07-06T03:55:14.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f4559a700 1 -- 172.21.15.97:0/2195309503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f400d2210 msgr2=0x7f6f400d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:14.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f4559a700 1 --2- 172.21.15.97:0/2195309503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f400d2210 0x7f6f400d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:14.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f4559a700 1 -- 172.21.15.97:0/2195309503 >> v1:172.21.15.97:6789/0 conn(0x7f6f4015c610 legacy=0x7f6f400b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:14.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.552+0000 7f6f4559a700 1 -- 172.21.15.97:0/2195309503 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f400b5650 con 0x7f6f4006b360 2023-07-06T03:55:14.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.553+0000 7f6f4559a700 1 --2- 172.21.15.97:0/2195309503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4006b360 0x7f6f400b4ec0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009840 tx=0x7f6f30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6923c38b31685b45 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:14.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.553+0000 7f6f44d99700 1 -- 172.21.15.97:0/2195309503 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30011830 con 0x7f6f4006b360 2023-07-06T03:55:14.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.553+0000 7f6f44d99700 1 -- 172.21.15.97:0/2195309503 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f30011990 con 0x7f6f4006b360 2023-07-06T03:55:14.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.553+0000 7f6f44d99700 1 -- 172.21.15.97:0/2195309503 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30011d20 con 0x7f6f4006b360 2023-07-06T03:55:14.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.553+0000 7f6f47fff700 1 -- 172.21.15.97:0/2195309503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4006b360 msgr2=0x7f6f400b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:14.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.553+0000 7f6f47fff700 1 --2- 172.21.15.97:0/2195309503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4006b360 0x7f6f400b4ec0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009840 tx=0x7f6f30009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:14.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.554+0000 7f6f47fff700 1 -- 172.21.15.97:0/2195309503 shutdown_connections 2023-07-06T03:55:14.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.554+0000 7f6f47fff700 1 --2- 172.21.15.97:0/2195309503 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f400d2210 0x7f6f400d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:14.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.554+0000 7f6f47fff700 1 --2- 172.21.15.97:0/2195309503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4006b360 0x7f6f400b4ec0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:14.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.554+0000 7f6f47fff700 1 -- 172.21.15.97:0/2195309503 >> 172.21.15.97:0/2195309503 conn(0x7f6f4006a6c0 msgr2=0x7f6f4006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:14.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.554+0000 7f6f47fff700 1 -- 172.21.15.97:0/2195309503 shutdown_connections 2023-07-06T03:55:14.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.554+0000 7f6f47fff700 1 -- 172.21.15.97:0/2195309503 wait complete. 2023-07-06T03:55:14.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.555+0000 7f6f47fff700 1 Processor -- start 2023-07-06T03:55:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.555+0000 7f6f47fff700 1 -- start start 2023-07-06T03:55:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.556+0000 7f6f47fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f4006b360 0x7f6f400c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.556+0000 7f6f47fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f400d2210 0x7f6f400cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.556+0000 7f6f47fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4015c610 0x7f6f400c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.556+0000 7f6f47fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f40158460 con 0x7f6f4015c610 2023-07-06T03:55:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.556+0000 7f6f47fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4015f8d0 con 0x7f6f4006b360 2023-07-06T03:55:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.556+0000 7f6f47fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4015e830 con 0x7f6f400d2210 2023-07-06T03:55:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.556+0000 7f6f4659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4015c610 0x7f6f400c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f4659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4015c610 0x7f6f400c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40044/0 (socket says 172.21.15.97:40044) 2023-07-06T03:55:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f4659c700 1 -- 172.21.15.97:0/140267832 learned_addr learned my addr 172.21.15.97:0/140267832 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f4559a700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f400d2210 0x7f6f400cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f45d9b700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f4006b360 0x7f6f400c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f4659c700 1 -- 172.21.15.97:0/140267832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f400d2210 msgr2=0x7f6f400cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f4659c700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f400d2210 0x7f6f400cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f4659c700 1 -- 172.21.15.97:0/140267832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f4006b360 msgr2=0x7f6f400c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f4659c700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f4006b360 0x7f6f400c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f4659c700 1 -- 172.21.15.97:0/140267832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f300094a0 con 0x7f6f4015c610 2023-07-06T03:55:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f45d9b700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f4006b360 0x7f6f400c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f4659c700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4015c610 0x7f6f400c6860 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800b970 tx=0x7f6f3800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f367fc700 1 -- 172.21.15.97:0/140267832 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3800c9b0 con 0x7f6f4015c610 2023-07-06T03:55:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f367fc700 1 -- 172.21.15.97:0/140267832 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3800cb30 con 0x7f6f4015c610 2023-07-06T03:55:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f367fc700 1 -- 172.21.15.97:0/140267832 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38014440 con 0x7f6f4015c610 2023-07-06T03:55:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.557+0000 7f6f47fff700 1 -- 172.21.15.97:0/140267832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4015eb10 con 0x7f6f4015c610 2023-07-06T03:55:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.558+0000 7f6f47fff700 1 -- 172.21.15.97:0/140267832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4015f0d0 con 0x7f6f4015c610 2023-07-06T03:55:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.558+0000 7f6f47fff700 1 -- 172.21.15.97:0/140267832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4004e340 con 0x7f6f4015c610 2023-07-06T03:55:14.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.560+0000 7f6f367fc700 1 -- 172.21.15.97:0/140267832 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6f3801d3b0 con 0x7f6f4015c610 2023-07-06T03:55:14.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.561+0000 7f6f367fc700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f2406ace0 0x7f6f2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:14.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.561+0000 7f6f367fc700 1 -- 172.21.15.97:0/140267832 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f6f38006e40 con 0x7f6f4015c610 2023-07-06T03:55:14.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.561+0000 7f6f45d9b700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f2406ace0 0x7f6f2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:14.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.562+0000 7f6f45d9b700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f2406ace0 0x7f6f2406d190 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c000f60 tx=0x7f6f3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:14.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.565+0000 7f6f367fc700 1 -- 172.21.15.97:0/140267832 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6f38054750 con 0x7f6f4015c610 2023-07-06T03:55:14.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:14.804+0000 7f6f47fff700 1 -- 172.21.15.97:0/140267832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6f4006b9d0 con 0x7f6f4015c610 2023-07-06T03:55:15.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.330+0000 7f6f367fc700 1 -- 172.21.15.97:0/140267832 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v88) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6f38057d70 con 0x7f6f4015c610 2023-07-06T03:55:15.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.337+0000 7f6f47fff700 1 -- 172.21.15.97:0/140267832 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f2406ace0 msgr2=0x7f6f2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:15.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.337+0000 7f6f47fff700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f2406ace0 0x7f6f2406d190 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c000f60 tx=0x7f6f3c009040 comp rx=0 tx=0).stop 2023-07-06T03:55:15.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.337+0000 7f6f47fff700 1 -- 172.21.15.97:0/140267832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4015c610 msgr2=0x7f6f400c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:15.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.337+0000 7f6f47fff700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4015c610 0x7f6f400c6860 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800b970 tx=0x7f6f3800bd30 comp rx=0 tx=0).stop 2023-07-06T03:55:15.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.338+0000 7f6f47fff700 1 -- 172.21.15.97:0/140267832 shutdown_connections 2023-07-06T03:55:15.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.338+0000 7f6f47fff700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6f2406ace0 0x7f6f2406d190 secure :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c000f60 tx=0x7f6f3c009040 comp rx=0 tx=0).stop 2023-07-06T03:55:15.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.338+0000 7f6f47fff700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4015c610 0x7f6f400c6860 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.338+0000 7f6f47fff700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f4006b360 0x7f6f400c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.338+0000 7f6f47fff700 1 --2- 172.21.15.97:0/140267832 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6f400d2210 0x7f6f400cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.338+0000 7f6f47fff700 1 -- 172.21.15.97:0/140267832 >> 172.21.15.97:0/140267832 conn(0x7f6f4006a6c0 msgr2=0x7f6f400b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:15.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.340+0000 7f6f47fff700 1 -- 172.21.15.97:0/140267832 shutdown_connections 2023-07-06T03:55:15.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.340+0000 7f6f47fff700 1 -- 172.21.15.97:0/140267832 wait complete. 2023-07-06T03:55:15.345 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 19 and data pool 20 2023-07-06T03:55:15.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:55:15.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.563+0000 7fcd6f1ff700 1 Processor -- start 2023-07-06T03:55:15.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.563+0000 7fcd6f1ff700 1 -- start start 2023-07-06T03:55:15.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.564+0000 7fcd6f1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcd680cd390 0x7fcd68160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:15.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.564+0000 7fcd6f1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcd68160dd0 0x7fcd680b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:15.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.564+0000 7fcd6f1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd680cdae0 con 0x7fcd6815c520 2023-07-06T03:55:15.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.564+0000 7fcd6f1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd680b6640 con 0x7fcd68160dd0 2023-07-06T03:55:15.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.564+0000 7fcd6f1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd680b6920 con 0x7fcd680cd390 2023-07-06T03:55:15.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.564+0000 7fcd6d79c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcd68160dd0 0x7fcd680b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:15.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.565+0000 7fcd6d79c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcd68160dd0 0x7fcd680b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40054/0 (socket says 172.21.15.97:40054) 2023-07-06T03:55:15.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.565+0000 7fcd6d79c700 1 -- 172.21.15.97:0/3834296204 learned_addr learned my addr 172.21.15.97:0/3834296204 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:15.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.565+0000 7fcd67fff700 1 --2- 172.21.15.97:0/3834296204 >> v2:172.21.15.146:3300/0 conn(0x7fcd680cd390 0x7fcd68160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.565+0000 7fcd6d79c700 1 -- 172.21.15.97:0/3834296204 >> v2:172.21.15.146:3300/0 conn(0x7fcd680cd390 msgr2=0x7fcd68160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.565+0000 7fcd6d79c700 1 --2- 172.21.15.97:0/3834296204 >> v2:172.21.15.146:3300/0 conn(0x7fcd680cd390 0x7fcd68160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.565+0000 7fcd6d79c700 1 -- 172.21.15.97:0/3834296204 >> v1:172.21.15.146:6789/0 conn(0x7fcd6815c520 legacy=0x7fcd68160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.565+0000 7fcd6d79c700 1 -- 172.21.15.97:0/3834296204 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680b6c00 con 0x7fcd68160dd0 2023-07-06T03:55:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.565+0000 7fcd6d79c700 1 --2- 172.21.15.97:0/3834296204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd68160dd0 0x7fcd680b7400 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fcd680b5930 tx=0x7fcd6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f6d19bf09dd17d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.565+0000 7fcd677fe700 1 -- 172.21.15.97:0/3834296204 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000e070 con 0x7fcd68160dd0 2023-07-06T03:55:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.566+0000 7fcd677fe700 1 -- 172.21.15.97:0/3834296204 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6000b910 con 0x7fcd68160dd0 2023-07-06T03:55:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.566+0000 7fcd677fe700 1 -- 172.21.15.97:0/3834296204 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000bcc0 con 0x7fcd68160dd0 2023-07-06T03:55:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.566+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/3834296204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd68160dd0 msgr2=0x7fcd680b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.566+0000 7fcd6f1ff700 1 --2- 172.21.15.97:0/3834296204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd68160dd0 0x7fcd680b7400 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fcd680b5930 tx=0x7fcd6000b330 comp rx=0 tx=0).stop 2023-07-06T03:55:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.566+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/3834296204 shutdown_connections 2023-07-06T03:55:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.566+0000 7fcd6f1ff700 1 --2- 172.21.15.97:0/3834296204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd68160dd0 0x7fcd680b7400 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.566+0000 7fcd6f1ff700 1 --2- 172.21.15.97:0/3834296204 >> v2:172.21.15.146:3300/0 conn(0x7fcd680cd390 0x7fcd68160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.566+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/3834296204 >> 172.21.15.97:0/3834296204 conn(0x7fcd68071790 msgr2=0x7fcd68071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:15.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.566+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/3834296204 shutdown_connections 2023-07-06T03:55:15.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.566+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/3834296204 wait complete. 2023-07-06T03:55:15.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.569+0000 7fcd6f1ff700 1 Processor -- start 2023-07-06T03:55:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.569+0000 7fcd6f1ff700 1 -- start start 2023-07-06T03:55:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6f1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd680cd390 0x7fcd680cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6f1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd6815c520 0x7fcd68158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6f1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd68160dd0 0x7fcd68155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6f1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680cb7f0 con 0x7fcd680cd390 2023-07-06T03:55:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6f1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680cb590 con 0x7fcd68160dd0 2023-07-06T03:55:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6f1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680d2350 con 0x7fcd6815c520 2023-07-06T03:55:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6cf9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd680cd390 0x7fcd680cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6cf9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd680cd390 0x7fcd680cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40062/0 (socket says 172.21.15.97:40062) 2023-07-06T03:55:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6cf9b700 1 -- 172.21.15.97:0/746888126 learned_addr learned my addr 172.21.15.97:0/746888126 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd67fff700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd6815c520 0x7fcd68158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6d79c700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd68160dd0 0x7fcd68155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6cf9b700 1 -- 172.21.15.97:0/746888126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd6815c520 msgr2=0x7fcd68158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6cf9b700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd6815c520 0x7fcd68158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6cf9b700 1 -- 172.21.15.97:0/746888126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd68160dd0 msgr2=0x7fcd68155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6cf9b700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd68160dd0 0x7fcd68155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6cf9b700 1 -- 172.21.15.97:0/746888126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd60009b30 con 0x7fcd680cd390 2023-07-06T03:55:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6d79c700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd68160dd0 0x7fcd68155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.570+0000 7fcd6cf9b700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd680cd390 0x7fcd680cb050 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c4e0 tx=0x7fcd5c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.571+0000 7fcd657fa700 1 -- 172.21.15.97:0/746888126 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c0052d0 con 0x7fcd680cd390 2023-07-06T03:55:15.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.571+0000 7fcd657fa700 1 -- 172.21.15.97:0/746888126 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c005430 con 0x7fcd680cd390 2023-07-06T03:55:15.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.571+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/746888126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd680d2630 con 0x7fcd680cd390 2023-07-06T03:55:15.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.571+0000 7fcd657fa700 1 -- 172.21.15.97:0/746888126 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c0038c0 con 0x7fcd680cd390 2023-07-06T03:55:15.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.571+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/746888126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd680d2bf0 con 0x7fcd680cd390 2023-07-06T03:55:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.572+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/746888126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd68160180 con 0x7fcd680cd390 2023-07-06T03:55:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.573+0000 7fcd657fa700 1 -- 172.21.15.97:0/746888126 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd5c003a20 con 0x7fcd680cd390 2023-07-06T03:55:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.575+0000 7fcd657fa700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd4c06ace0 0x7fcd4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.575+0000 7fcd657fa700 1 -- 172.21.15.97:0/746888126 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd5c088910 con 0x7fcd680cd390 2023-07-06T03:55:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.575+0000 7fcd67fff700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd4c06ace0 0x7fcd4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.575+0000 7fcd67fff700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd4c06ace0 0x7fcd4c06d190 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcd6815b190 tx=0x7fcd54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:15.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.579+0000 7fcd657fa700 1 -- 172.21.15.97:0/746888126 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcd5c053a50 con 0x7fcd680cd390 2023-07-06T03:55:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.772+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/746888126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd6804e340 con 0x7fcd680cd390 2023-07-06T03:55:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.773+0000 7fcd657fa700 1 -- 172.21.15.97:0/746888126 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+18332 (secure 0 0 0) 0x7fcd5c057070 con 0x7fcd680cd390 2023-07-06T03:55:15.776 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:15.776 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":123,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:55:14.806607+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:55:12.989988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-06T03:55:13.782970+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:55:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/746888126 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd4c06ace0 msgr2=0x7fcd4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd4c06ace0 0x7fcd4c06d190 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcd6815b190 tx=0x7fcd54008040 comp rx=0 tx=0).stop 2023-07-06T03:55:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/746888126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd680cd390 msgr2=0x7fcd680cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd680cd390 0x7fcd680cb050 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c4e0 tx=0x7fcd5c00c8a0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/746888126 shutdown_connections 2023-07-06T03:55:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcd4c06ace0 0x7fcd4c06d190 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd68160dd0 0x7fcd68155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd680cd390 0x7fcd680cb050 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 --2- 172.21.15.97:0/746888126 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcd6815c520 0x7fcd68158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/746888126 >> 172.21.15.97:0/746888126 conn(0x7fcd68071790 msgr2=0x7fcd681605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:15.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/746888126 shutdown_connections 2023-07-06T03:55:15.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.776+0000 7fcd6f1ff700 1 -- 172.21.15.97:0/746888126 wait complete. 2023-07-06T03:55:15.797 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:55:15.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.990+0000 7f9fe0fff700 1 Processor -- start 2023-07-06T03:55:15.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.990+0000 7f9fe0fff700 1 -- start start 2023-07-06T03:55:15.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.990+0000 7f9fe0fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9fdc0cd390 0x7f9fdc160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:15.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.990+0000 7f9fe0fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc0cdae0 con 0x7f9fdc15c520 2023-07-06T03:55:15.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.990+0000 7f9fe0fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc0b6640 con 0x7f9fdc160dd0 2023-07-06T03:55:15.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.990+0000 7f9fe0fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc0b6920 con 0x7f9fdc0cd390 2023-07-06T03:55:15.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.990+0000 7f9fda59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9fdc15c520 0x7f9fdc160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38370/0 (socket says 172.21.15.97:38370) 2023-07-06T03:55:15.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.990+0000 7f9fda59c700 1 -- 172.21.15.97:0/72524883 learned_addr learned my addr 172.21.15.97:0/72524883 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:15.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.990+0000 7f9fd9d9b700 1 --2- 172.21.15.97:0/72524883 >> v2:172.21.15.97:3300/0 conn(0x7f9fdc0cd390 0x7f9fdc160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:15.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.991+0000 7f9fd959a700 1 -- 172.21.15.97:0/72524883 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783079908 0 0) 0x7f9fdc0cdae0 con 0x7f9fdc15c520 2023-07-06T03:55:15.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.991+0000 7f9fd959a700 1 -- 172.21.15.97:0/72524883 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc40035b0 con 0x7f9fdc15c520 2023-07-06T03:55:15.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.991+0000 7f9fd9d9b700 1 -- 172.21.15.97:0/72524883 >> v1:172.21.15.146:6789/0 conn(0x7f9fdc160dd0 legacy=0x7f9fdc0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:15.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.991+0000 7f9fd9d9b700 1 -- 172.21.15.97:0/72524883 >> v1:172.21.15.97:6789/0 conn(0x7f9fdc15c520 legacy=0x7f9fdc160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:15.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.991+0000 7f9fd9d9b700 1 -- 172.21.15.97:0/72524883 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0b6c00 con 0x7f9fdc0cd390 2023-07-06T03:55:15.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.991+0000 7f9fd9d9b700 1 --2- 172.21.15.97:0/72524883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fdc0cd390 0x7f9fdc160890 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009840 tx=0x7f9fd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71576e2380a3bd44 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:15.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.991+0000 7f9fd959a700 1 -- 172.21.15.97:0/72524883 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0011830 con 0x7f9fdc0cd390 2023-07-06T03:55:15.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.991+0000 7f9fd959a700 1 -- 172.21.15.97:0/72524883 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd0011990 con 0x7f9fdc0cd390 2023-07-06T03:55:15.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.991+0000 7f9fd959a700 1 -- 172.21.15.97:0/72524883 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0011d20 con 0x7f9fdc0cd390 2023-07-06T03:55:15.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.992+0000 7f9fe0fff700 1 -- 172.21.15.97:0/72524883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fdc0cd390 msgr2=0x7f9fdc160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:15.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.992+0000 7f9fe0fff700 1 --2- 172.21.15.97:0/72524883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fdc0cd390 0x7f9fdc160890 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009840 tx=0x7f9fd0009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:15.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.992+0000 7f9fe0fff700 1 -- 172.21.15.97:0/72524883 shutdown_connections 2023-07-06T03:55:15.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.992+0000 7f9fe0fff700 1 --2- 172.21.15.97:0/72524883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fdc0cd390 0x7f9fdc160890 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:15.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.992+0000 7f9fe0fff700 1 -- 172.21.15.97:0/72524883 >> 172.21.15.97:0/72524883 conn(0x7f9fdc071790 msgr2=0x7f9fdc071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:15.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.992+0000 7f9fe0fff700 1 -- 172.21.15.97:0/72524883 shutdown_connections 2023-07-06T03:55:15.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.992+0000 7f9fe0fff700 1 -- 172.21.15.97:0/72524883 wait complete. 2023-07-06T03:55:15.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.994+0000 7f9fe0fff700 1 Processor -- start 2023-07-06T03:55:15.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.994+0000 7f9fe0fff700 1 -- start start 2023-07-06T03:55:15.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fe0fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fdc0cd390 0x7f9fdc0d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:15.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fe0fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fdc15c520 0x7f9fdc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:15.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fe0fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fdc160dd0 0x7f9fdc1427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:15.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fe0fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc150500 con 0x7f9fdc0cd390 2023-07-06T03:55:15.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fe0fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0d3c60 con 0x7f9fdc160dd0 2023-07-06T03:55:15.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fe0fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0d36a0 con 0x7f9fdc15c520 2023-07-06T03:55:15.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fda59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fdc0cd390 0x7f9fdc0d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fda59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fdc0cd390 0x7f9fdc0d2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40074/0 (socket says 172.21.15.97:40074) 2023-07-06T03:55:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fda59c700 1 -- 172.21.15.97:0/16959931 learned_addr learned my addr 172.21.15.97:0/16959931 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fda59c700 1 -- 172.21.15.97:0/16959931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fdc15c520 msgr2=0x7f9fdc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fda59c700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fdc15c520 0x7f9fdc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fda59c700 1 -- 172.21.15.97:0/16959931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fdc160dd0 msgr2=0x7f9fdc1427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fda59c700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fdc160dd0 0x7f9fdc1427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fda59c700 1 -- 172.21.15.97:0/16959931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd00094a0 con 0x7f9fdc0cd390 2023-07-06T03:55:16.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.995+0000 7f9fda59c700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fdc0cd390 0x7f9fdc0d2d50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c700 tx=0x7f9fcc00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:16.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.996+0000 7f9fcaffd700 1 -- 172.21.15.97:0/16959931 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc005d70 con 0x7f9fdc0cd390 2023-07-06T03:55:16.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.996+0000 7f9fcaffd700 1 -- 172.21.15.97:0/16959931 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc0070d0 con 0x7f9fdc0cd390 2023-07-06T03:55:16.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.996+0000 7f9fcaffd700 1 -- 172.21.15.97:0/16959931 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc007460 con 0x7f9fdc0cd390 2023-07-06T03:55:16.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.996+0000 7f9fe0fff700 1 -- 172.21.15.97:0/16959931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc0d3980 con 0x7f9fdc0cd390 2023-07-06T03:55:16.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.996+0000 7f9fe0fff700 1 -- 172.21.15.97:0/16959931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc15d850 con 0x7f9fdc0cd390 2023-07-06T03:55:16.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.997+0000 7f9fe0fff700 1 -- 172.21.15.97:0/16959931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc15d3d0 con 0x7f9fdc0cd390 2023-07-06T03:55:16.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.998+0000 7f9fcaffd700 1 -- 172.21.15.97:0/16959931 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9fcc0075c0 con 0x7f9fdc0cd390 2023-07-06T03:55:16.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.999+0000 7f9fcaffd700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fbc06ace0 0x7f9fbc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:16.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.999+0000 7f9fcaffd700 1 -- 172.21.15.97:0/16959931 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9fcc088a50 con 0x7f9fdc0cd390 2023-07-06T03:55:16.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.999+0000 7f9fd9d9b700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fbc06ace0 0x7f9fbc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:16.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:15.999+0000 7f9fd9d9b700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fbc06ace0 0x7f9fbc06d190 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc15bb00 tx=0x7f9fd0010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:16.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.003+0000 7f9fcaffd700 1 -- 172.21.15.97:0/16959931 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9fdc15d3d0 con 0x7f9fdc0cd390 2023-07-06T03:55:16.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.199+0000 7f9fe0fff700 1 -- 172.21.15.97:0/16959931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fdc15d3d0 con 0x7f9fdc0cd390 2023-07-06T03:55:16.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.202+0000 7f9fcaffd700 1 -- 172.21.15.97:0/16959931 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+18332 (secure 0 0 0) 0x7f9fdc15d3d0 con 0x7f9fdc0cd390 2023-07-06T03:55:16.204 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:16.204 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":123,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:55:14.806607+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:55:12.989988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-06T03:55:13.782970+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:55:16.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 -- 172.21.15.97:0/16959931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fbc06ace0 msgr2=0x7f9fbc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:16.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fbc06ace0 0x7f9fbc06d190 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc15bb00 tx=0x7f9fd0010000 comp rx=0 tx=0).stop 2023-07-06T03:55:16.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 -- 172.21.15.97:0/16959931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fdc0cd390 msgr2=0x7f9fdc0d2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:16.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fdc0cd390 0x7f9fdc0d2d50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c700 tx=0x7f9fcc00cac0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 -- 172.21.15.97:0/16959931 shutdown_connections 2023-07-06T03:55:16.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9fbc06ace0 0x7f9fbc06d190 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9fdc160dd0 0x7f9fdc1427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fdc0cd390 0x7f9fdc0d2d50 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 --2- 172.21.15.97:0/16959931 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9fdc15c520 0x7f9fdc0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 -- 172.21.15.97:0/16959931 >> 172.21.15.97:0/16959931 conn(0x7f9fdc071790 msgr2=0x7f9fdc0c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:16.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 -- 172.21.15.97:0/16959931 shutdown_connections 2023-07-06T03:55:16.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.203+0000 7f9fe0fff700 1 -- 172.21.15.97:0/16959931 wait complete. 2023-07-06T03:55:16.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T03:55:16.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.422+0000 7f5121fff700 1 Processor -- start 2023-07-06T03:55:16.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.422+0000 7f5121fff700 1 -- start start 2023-07-06T03:55:16.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.424+0000 7f5121fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c06b360 0x7f511c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:16.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.424+0000 7f5121fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c0cff90 con 0x7f511c15c610 2023-07-06T03:55:16.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.424+0000 7f5121fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c0b5650 con 0x7f511c0d2210 2023-07-06T03:55:16.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.424+0000 7f5121fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0d0780 con 0x7f511c06b360 2023-07-06T03:55:16.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.424+0000 7f511b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f511c15c610 0x7f511c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38378/0 (socket says 172.21.15.97:38378) 2023-07-06T03:55:16.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.424+0000 7f511b7fe700 1 -- 172.21.15.97:0/739760146 learned_addr learned my addr 172.21.15.97:0/739760146 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:16.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.424+0000 7f5113fff700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c06b360 0x7f511c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.424+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105457694 0 0) 0x7f511c0cff90 con 0x7f511c15c610 2023-07-06T03:55:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.424+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51080035b0 con 0x7f511c15c610 2023-07-06T03:55:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4220446488 0 0) 0x7f51080035b0 con 0x7f511c15c610 2023-07-06T03:55:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c06b360 msgr2=0x7f511c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c06b360 0x7f511c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 >> v1:172.21.15.146:6789/0 conn(0x7f511c0d2210 legacy=0x7f511c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:16.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c0b5400 con 0x7f511c15c610 2023-07-06T03:55:16.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f510c001670 con 0x7f511c15c610 2023-07-06T03:55:16.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 >> v1:172.21.15.97:6789/0 conn(0x7f511c15c610 legacy=0x7f511c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:16.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c06b360 0x7f5108003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:16.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5108004000 0x7f5108004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:16.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5108004d90 0x7f5108005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:16.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0d0780 con 0x7f5108004000 2023-07-06T03:55:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510800bb20 con 0x7f5108004d90 2023-07-06T03:55:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510800bce0 con 0x7f511c06b360 2023-07-06T03:55:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f5113fff700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5108004000 0x7f5108004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511b7fe700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c06b360 0x7f5108003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511bfff700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5108004d90 0x7f5108005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:16.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f5113fff700 1 -- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c06b360 msgr2=0x7f5108003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f5113fff700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c06b360 0x7f5108003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f5113fff700 1 -- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5108004d90 msgr2=0x7f5108005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f5113fff700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5108004d90 0x7f5108005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f5113fff700 1 -- 172.21.15.97:0/739760146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080035b0 con 0x7f5108004000 2023-07-06T03:55:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511b7fe700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c06b360 0x7f5108003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:16.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.425+0000 7f511bfff700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5108004d90 0x7f5108005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:16.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.426+0000 7f5113fff700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5108004000 0x7f5108004850 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f5104009b30 tx=0x7f5108005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a6c082eab97870 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:16.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.426+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510400e6c0 con 0x7f5108004000 2023-07-06T03:55:16.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.426+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510400e820 con 0x7f5108004000 2023-07-06T03:55:16.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.426+0000 7f511affd700 1 -- 172.21.15.97:0/739760146 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510400eb90 con 0x7f5108004000 2023-07-06T03:55:16.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.426+0000 7f5121fff700 1 -- 172.21.15.97:0/739760146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5108004000 msgr2=0x7f5108004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:16.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.426+0000 7f5121fff700 1 --2- 172.21.15.97:0/739760146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5108004000 0x7f5108004850 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f5104009b30 tx=0x7f5108005880 comp rx=0 tx=0).stop 2023-07-06T03:55:16.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.426+0000 7f511bfff700 1 -- 172.21.15.97:0/739760146 reap_dead start 2023-07-06T03:55:16.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.427+0000 7f5121fff700 1 -- 172.21.15.97:0/739760146 shutdown_connections 2023-07-06T03:55:16.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.427+0000 7f5121fff700 1 -- 172.21.15.97:0/739760146 >> 172.21.15.97:0/739760146 conn(0x7f511c06a6c0 msgr2=0x7f511c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:16.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.427+0000 7f5121fff700 1 -- 172.21.15.97:0/739760146 shutdown_connections 2023-07-06T03:55:16.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.427+0000 7f5121fff700 1 -- 172.21.15.97:0/739760146 wait complete. 2023-07-06T03:55:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.429+0000 7f5121fff700 1 Processor -- start 2023-07-06T03:55:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.429+0000 7f5121fff700 1 -- start start 2023-07-06T03:55:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.430+0000 7f5121fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5108004000 0x7f511c15f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.430+0000 7f5121fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c14fe70 0x7f511c15e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.430+0000 7f5121fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0c6860 0x7f511c0b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:16.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f5121fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0c9e00 con 0x7f511c0c6860 2023-07-06T03:55:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f5121fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c1502e0 con 0x7f5108004000 2023-07-06T03:55:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f5121fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c15e5a0 con 0x7f511c14fe70 2023-07-06T03:55:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f511bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0c6860 0x7f511c0b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f511bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0c6860 0x7f511c0b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40100/0 (socket says 172.21.15.97:40100) 2023-07-06T03:55:16.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f511bfff700 1 -- 172.21.15.97:0/426401898 learned_addr learned my addr 172.21.15.97:0/426401898 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:16.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f5113fff700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c14fe70 0x7f511c15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:16.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f511b7fe700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5108004000 0x7f511c15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:16.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f511bfff700 1 -- 172.21.15.97:0/426401898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c14fe70 msgr2=0x7f511c15e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:16.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f511bfff700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c14fe70 0x7f511c15e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f511bfff700 1 -- 172.21.15.97:0/426401898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5108004000 msgr2=0x7f511c15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:16.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f511bfff700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5108004000 0x7f511c15f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f511bfff700 1 -- 172.21.15.97:0/426401898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c0ca050 con 0x7f511c0c6860 2023-07-06T03:55:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.431+0000 7f511bfff700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0c6860 0x7f511c0b6020 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f511c072770 tx=0x7f511400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.432+0000 7f5118ff9700 1 -- 172.21.15.97:0/426401898 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114003e10 con 0x7f511c0c6860 2023-07-06T03:55:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.432+0000 7f5118ff9700 1 -- 172.21.15.97:0/426401898 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5114005070 con 0x7f511c0c6860 2023-07-06T03:55:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.432+0000 7f5121fff700 1 -- 172.21.15.97:0/426401898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c0ca330 con 0x7f511c0c6860 2023-07-06T03:55:16.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.432+0000 7f5118ff9700 1 -- 172.21.15.97:0/426401898 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114005420 con 0x7f511c0c6860 2023-07-06T03:55:16.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.432+0000 7f5121fff700 1 -- 172.21.15.97:0/426401898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c0ca8f0 con 0x7f511c0c6860 2023-07-06T03:55:16.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.433+0000 7f5121fff700 1 -- 172.21.15.97:0/426401898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c04e340 con 0x7f511c0c6860 2023-07-06T03:55:16.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.434+0000 7f5118ff9700 1 -- 172.21.15.97:0/426401898 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5114005580 con 0x7f511c0c6860 2023-07-06T03:55:16.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.435+0000 7f5118ff9700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f50fc06ace0 0x7f50fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:16.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.435+0000 7f5118ff9700 1 -- 172.21.15.97:0/426401898 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5114088e50 con 0x7f511c0c6860 2023-07-06T03:55:16.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.435+0000 7f511b7fe700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f50fc06ace0 0x7f50fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:16.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.436+0000 7f511b7fe700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f50fc06ace0 0x7f50fc06d190 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f510c000ff0 tx=0x7f510c007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:16.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.439+0000 7f5118ff9700 1 -- 172.21.15.97:0/426401898 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5114053fb0 con 0x7f511c0c6860 2023-07-06T03:55:16.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:16.678+0000 7f5121fff700 1 -- 172.21.15.97:0/426401898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f511c0b17b0 con 0x7f511c0c6860 2023-07-06T03:55:17.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.343+0000 7f5118ff9700 1 -- 172.21.15.97:0/426401898 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v91) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5114018020 con 0x7f511c0c6860 2023-07-06T03:55:17.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.346+0000 7f5121fff700 1 -- 172.21.15.97:0/426401898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f50fc06ace0 msgr2=0x7f50fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:17.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.346+0000 7f5121fff700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f50fc06ace0 0x7f50fc06d190 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f510c000ff0 tx=0x7f510c007040 comp rx=0 tx=0).stop 2023-07-06T03:55:17.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.346+0000 7f5121fff700 1 -- 172.21.15.97:0/426401898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0c6860 msgr2=0x7f511c0b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:17.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.346+0000 7f5121fff700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0c6860 0x7f511c0b6020 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f511c072770 tx=0x7f511400ddc0 comp rx=0 tx=0).stop 2023-07-06T03:55:17.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.347+0000 7f5121fff700 1 -- 172.21.15.97:0/426401898 shutdown_connections 2023-07-06T03:55:17.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.347+0000 7f5121fff700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f50fc06ace0 0x7f50fc06d190 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:17.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.347+0000 7f5121fff700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0c6860 0x7f511c0b6020 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:17.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.347+0000 7f5121fff700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5108004000 0x7f511c15f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:17.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.347+0000 7f5121fff700 1 --2- 172.21.15.97:0/426401898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f511c14fe70 0x7f511c15e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:17.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.347+0000 7f5121fff700 1 -- 172.21.15.97:0/426401898 >> 172.21.15.97:0/426401898 conn(0x7f511c06a6c0 msgr2=0x7f511c0d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:17.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.347+0000 7f5121fff700 1 -- 172.21.15.97:0/426401898 shutdown_connections 2023-07-06T03:55:17.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.347+0000 7f5121fff700 1 -- 172.21.15.97:0/426401898 wait complete. 2023-07-06T03:55:17.369 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T03:55:17.369 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T03:55:17.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.569+0000 7f4327fff700 1 Processor -- start 2023-07-06T03:55:17.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.570+0000 7f4327fff700 1 -- start start 2023-07-06T03:55:17.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.571+0000 7f4327fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4320158b20 0x7f43200cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:17.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.571+0000 7f4327fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f432006cab0 0x7f43200ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:17.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.571+0000 7f4327fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43200ccbc0 con 0x7f43200b0560 2023-07-06T03:55:17.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.571+0000 7f4327fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43200ce820 con 0x7f432006cab0 2023-07-06T03:55:17.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.571+0000 7f4327fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4320158ef0 con 0x7f4320158b20 2023-07-06T03:55:17.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.572+0000 7f432559a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f432006cab0 0x7f43200ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:17.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.572+0000 7f432659c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f43200b0560 0x7f432006f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38384/0 (socket says 172.21.15.97:38384) 2023-07-06T03:55:17.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.572+0000 7f432659c700 1 -- 172.21.15.97:0/2913157242 learned_addr learned my addr 172.21.15.97:0/2913157242 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:17.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.572+0000 7f4325d9b700 1 --2- 172.21.15.97:0/2913157242 >> v2:172.21.15.146:3300/0 conn(0x7f4320158b20 0x7f43200cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:17.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.572+0000 7f432559a700 1 -- 172.21.15.97:0/2913157242 >> v2:172.21.15.146:3300/0 conn(0x7f4320158b20 msgr2=0x7f43200cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:17.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.572+0000 7f432559a700 1 --2- 172.21.15.97:0/2913157242 >> v2:172.21.15.146:3300/0 conn(0x7f4320158b20 0x7f43200cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:17.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.572+0000 7f432559a700 1 -- 172.21.15.97:0/2913157242 >> v1:172.21.15.97:6789/0 conn(0x7f43200b0560 legacy=0x7f432006f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:17.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.572+0000 7f4324d99700 1 -- 172.21.15.97:0/2913157242 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154018893 0 0) 0x7f43200ccbc0 con 0x7f43200b0560 2023-07-06T03:55:17.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.572+0000 7f432559a700 1 -- 172.21.15.97:0/2913157242 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43200ce560 con 0x7f432006cab0 2023-07-06T03:55:17.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.572+0000 7f432559a700 1 --2- 172.21.15.97:0/2913157242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432006cab0 0x7f43200ce020 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f432006f8e0 tx=0x7f4310009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d12d6f4cb738c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:17.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.573+0000 7f4324d99700 1 -- 172.21.15.97:0/2913157242 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310011830 con 0x7f432006cab0 2023-07-06T03:55:17.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.573+0000 7f4324d99700 1 -- 172.21.15.97:0/2913157242 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4310011990 con 0x7f432006cab0 2023-07-06T03:55:17.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.573+0000 7f4324d99700 1 -- 172.21.15.97:0/2913157242 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310011d20 con 0x7f432006cab0 2023-07-06T03:55:17.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.573+0000 7f4327fff700 1 -- 172.21.15.97:0/2913157242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432006cab0 msgr2=0x7f43200ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:17.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.573+0000 7f4327fff700 1 --2- 172.21.15.97:0/2913157242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432006cab0 0x7f43200ce020 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f432006f8e0 tx=0x7f4310009b20 comp rx=0 tx=0).stop 2023-07-06T03:55:17.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.573+0000 7f4327fff700 1 -- 172.21.15.97:0/2913157242 shutdown_connections 2023-07-06T03:55:17.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.573+0000 7f4327fff700 1 --2- 172.21.15.97:0/2913157242 >> v2:172.21.15.146:3300/0 conn(0x7f4320158b20 0x7f43200cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:17.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.573+0000 7f4327fff700 1 --2- 172.21.15.97:0/2913157242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432006cab0 0x7f43200ce020 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:17.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.573+0000 7f4327fff700 1 -- 172.21.15.97:0/2913157242 >> 172.21.15.97:0/2913157242 conn(0x7f432006a060 msgr2=0x7f432006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:17.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.573+0000 7f4327fff700 1 -- 172.21.15.97:0/2913157242 shutdown_connections 2023-07-06T03:55:17.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.573+0000 7f4327fff700 1 -- 172.21.15.97:0/2913157242 wait complete. 2023-07-06T03:55:17.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.575+0000 7f4327fff700 1 Processor -- start 2023-07-06T03:55:17.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.575+0000 7f4327fff700 1 -- start start 2023-07-06T03:55:17.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.576+0000 7f4327fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432006cab0 0x7f432014bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:17.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.576+0000 7f4327fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43200b0560 0x7f43200afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:17.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.576+0000 7f4327fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4320158b20 0x7f43200b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:17.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.576+0000 7f4327fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4320154f50 con 0x7f432006cab0 2023-07-06T03:55:17.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.576+0000 7f4327fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43200c6660 con 0x7f4320158b20 2023-07-06T03:55:17.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.576+0000 7f4327fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4320154340 con 0x7f43200b0560 2023-07-06T03:55:17.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.576+0000 7f4325d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432006cab0 0x7f432014bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.576+0000 7f4325d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432006cab0 0x7f432014bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40122/0 (socket says 172.21.15.97:40122) 2023-07-06T03:55:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.576+0000 7f4325d9b700 1 -- 172.21.15.97:0/4059068714 learned_addr learned my addr 172.21.15.97:0/4059068714 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.576+0000 7f432559a700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43200b0560 0x7f43200afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.576+0000 7f432659c700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4320158b20 0x7f43200b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.577+0000 7f4325d9b700 1 -- 172.21.15.97:0/4059068714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43200b0560 msgr2=0x7f43200afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.577+0000 7f4325d9b700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43200b0560 0x7f43200afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.577+0000 7f4325d9b700 1 -- 172.21.15.97:0/4059068714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4320158b20 msgr2=0x7f43200b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.577+0000 7f4325d9b700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4320158b20 0x7f43200b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:17.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.577+0000 7f4325d9b700 1 -- 172.21.15.97:0/4059068714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43100094a0 con 0x7f432006cab0 2023-07-06T03:55:17.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.577+0000 7f4325d9b700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432006cab0 0x7f432014bce0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f431c00e9e0 tx=0x7f431c00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:17.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.577+0000 7f43167fc700 1 -- 172.21.15.97:0/4059068714 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c009710 con 0x7f432006cab0 2023-07-06T03:55:17.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.577+0000 7f43167fc700 1 -- 172.21.15.97:0/4059068714 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c00c850 con 0x7f432006cab0 2023-07-06T03:55:17.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.577+0000 7f4327fff700 1 -- 172.21.15.97:0/4059068714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4320154620 con 0x7f432006cab0 2023-07-06T03:55:17.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.577+0000 7f43167fc700 1 -- 172.21.15.97:0/4059068714 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c00cbc0 con 0x7f432006cab0 2023-07-06T03:55:17.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.577+0000 7f4327fff700 1 -- 172.21.15.97:0/4059068714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4320154be0 con 0x7f432006cab0 2023-07-06T03:55:17.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.578+0000 7f4327fff700 1 -- 172.21.15.97:0/4059068714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432004e340 con 0x7f432006cab0 2023-07-06T03:55:17.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.579+0000 7f43167fc700 1 -- 172.21.15.97:0/4059068714 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f431c00cd20 con 0x7f432006cab0 2023-07-06T03:55:17.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.580+0000 7f43167fc700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430406ace0 0x7f430406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.580+0000 7f43167fc700 1 -- 172.21.15.97:0/4059068714 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f431c0895c0 con 0x7f432006cab0 2023-07-06T03:55:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.580+0000 7f432559a700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430406ace0 0x7f430406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.581+0000 7f432559a700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430406ace0 0x7f430406d190 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f43100090b0 tx=0x7f431001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:17.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.585+0000 7f43167fc700 1 -- 172.21.15.97:0/4059068714 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f431c058100 con 0x7f432006cab0 2023-07-06T03:55:17.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:17.824+0000 7f4327fff700 1 -- 172.21.15.97:0/4059068714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f432014c220 con 0x7f432006cab0 2023-07-06T03:55:18.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.348+0000 7f43167fc700 1 -- 172.21.15.97:0/4059068714 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v92) v1 ==== 122+0+0 (secure 0 0 0) 0x7f431c057c90 con 0x7f432006cab0 2023-07-06T03:55:18.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 -- 172.21.15.97:0/4059068714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430406ace0 msgr2=0x7f430406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:18.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430406ace0 0x7f430406d190 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f43100090b0 tx=0x7f431001c040 comp rx=0 tx=0).stop 2023-07-06T03:55:18.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 -- 172.21.15.97:0/4059068714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432006cab0 msgr2=0x7f432014bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:18.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432006cab0 0x7f432014bce0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f431c00e9e0 tx=0x7f431c00eda0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 -- 172.21.15.97:0/4059068714 shutdown_connections 2023-07-06T03:55:18.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f430406ace0 0x7f430406d190 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4320158b20 0x7f43200b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f432006cab0 0x7f432014bce0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 --2- 172.21.15.97:0/4059068714 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f43200b0560 0x7f43200afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 -- 172.21.15.97:0/4059068714 >> 172.21.15.97:0/4059068714 conn(0x7f432006a060 msgr2=0x7f4320152720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 -- 172.21.15.97:0/4059068714 shutdown_connections 2023-07-06T03:55:18.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.351+0000 7f4327fff700 1 -- 172.21.15.97:0/4059068714 wait complete. 2023-07-06T03:55:18.372 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T03:55:18.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:55:18.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.565+0000 7f743a9ff700 1 Processor -- start 2023-07-06T03:55:18.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.565+0000 7f743a9ff700 1 -- start start 2023-07-06T03:55:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.566+0000 7f743a9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f74340cd8e0 0x7f743415e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.566+0000 7f743a9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f743415b440 0x7f74340adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.566+0000 7f743a9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74340710d0 con 0x7f7434070c30 2023-07-06T03:55:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.566+0000 7f743a9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74340ae3e0 con 0x7f74340cd8e0 2023-07-06T03:55:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.566+0000 7f743a9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7434071330 con 0x7f743415b440 2023-07-06T03:55:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7433fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f74340cd8e0 0x7f743415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7433fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f74340cd8e0 0x7f743415e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40126/0 (socket says 172.21.15.97:40126) 2023-07-06T03:55:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7433fff700 1 -- 172.21.15.97:0/971871391 learned_addr learned my addr 172.21.15.97:0/971871391 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:18.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7438f9c700 1 --2- 172.21.15.97:0/971871391 >> v2:172.21.15.146:3300/0 conn(0x7f743415b440 0x7f74340adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:18.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7433fff700 1 -- 172.21.15.97:0/971871391 >> v2:172.21.15.146:3300/0 conn(0x7f743415b440 msgr2=0x7f74340adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:18.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7433fff700 1 --2- 172.21.15.97:0/971871391 >> v2:172.21.15.146:3300/0 conn(0x7f743415b440 0x7f74340adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7433fff700 1 -- 172.21.15.97:0/971871391 >> v1:172.21.15.146:6789/0 conn(0x7f7434070c30 legacy=0x7f74340cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:18.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7433fff700 1 -- 172.21.15.97:0/971871391 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7434071610 con 0x7f74340cd8e0 2023-07-06T03:55:18.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7438f9c700 1 --2- 172.21.15.97:0/971871391 >> v2:172.21.15.146:3300/0 conn(0x7f743415b440 0x7f74340adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:18.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7433fff700 1 --2- 172.21.15.97:0/971871391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74340cd8e0 0x7f743415e860 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f74340b4970 tx=0x7f7424009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa8438f01377f86 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:18.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7432ffd700 1 -- 172.21.15.97:0/971871391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424004230 con 0x7f74340cd8e0 2023-07-06T03:55:18.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.567+0000 7f7432ffd700 1 -- 172.21.15.97:0/971871391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7424004390 con 0x7f74340cd8e0 2023-07-06T03:55:18.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.568+0000 7f7432ffd700 1 -- 172.21.15.97:0/971871391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240036a0 con 0x7f74340cd8e0 2023-07-06T03:55:18.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.568+0000 7f743a9ff700 1 -- 172.21.15.97:0/971871391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74340cd8e0 msgr2=0x7f743415e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:18.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.568+0000 7f743a9ff700 1 --2- 172.21.15.97:0/971871391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74340cd8e0 0x7f743415e860 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f74340b4970 tx=0x7f7424009bd0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.568+0000 7f743a9ff700 1 -- 172.21.15.97:0/971871391 shutdown_connections 2023-07-06T03:55:18.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.568+0000 7f743a9ff700 1 --2- 172.21.15.97:0/971871391 >> v2:172.21.15.146:3300/0 conn(0x7f743415b440 0x7f74340adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.568+0000 7f743a9ff700 1 --2- 172.21.15.97:0/971871391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74340cd8e0 0x7f743415e860 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.568+0000 7f743a9ff700 1 -- 172.21.15.97:0/971871391 >> 172.21.15.97:0/971871391 conn(0x7f74340ce8c0 msgr2=0x7f74340cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:18.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.568+0000 7f743a9ff700 1 -- 172.21.15.97:0/971871391 shutdown_connections 2023-07-06T03:55:18.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.568+0000 7f743a9ff700 1 -- 172.21.15.97:0/971871391 wait complete. 2023-07-06T03:55:18.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.570+0000 7f743a9ff700 1 Processor -- start 2023-07-06T03:55:18.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.570+0000 7f743a9ff700 1 -- start start 2023-07-06T03:55:18.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f743a9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7434070c30 0x7f7434152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:18.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f743a9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74340cd8e0 0x7f7434148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:18.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f743a9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f743415b440 0x7f74341f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:18.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f743a9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340af570 con 0x7f74340cd8e0 2023-07-06T03:55:18.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f743a9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74341488c0 con 0x7f7434070c30 2023-07-06T03:55:18.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f743a9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74341f2d20 con 0x7f743415b440 2023-07-06T03:55:18.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f74337fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74340cd8e0 0x7f7434148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:18.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f74337fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74340cd8e0 0x7f7434148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40132/0 (socket says 172.21.15.97:40132) 2023-07-06T03:55:18.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f74337fe700 1 -- 172.21.15.97:0/2661704764 learned_addr learned my addr 172.21.15.97:0/2661704764 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:18.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f7433fff700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7434070c30 0x7f7434152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:18.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f7438f9c700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f743415b440 0x7f74341f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:18.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.571+0000 7f74337fe700 1 -- 172.21.15.97:0/2661704764 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f743415b440 msgr2=0x7f74341f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:18.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.572+0000 7f74337fe700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f743415b440 0x7f74341f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.572+0000 7f74337fe700 1 -- 172.21.15.97:0/2661704764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7434070c30 msgr2=0x7f7434152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:18.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.572+0000 7f74337fe700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7434070c30 0x7f7434152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.572+0000 7f74337fe700 1 -- 172.21.15.97:0/2661704764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7424009550 con 0x7f74340cd8e0 2023-07-06T03:55:18.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.572+0000 7f7433fff700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7434070c30 0x7f7434152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:18.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.572+0000 7f74337fe700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74340cd8e0 0x7f7434148380 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f742800b4d0 tx=0x7f742800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:18.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.572+0000 7f7430ff9700 1 -- 172.21.15.97:0/2661704764 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428011830 con 0x7f74340cd8e0 2023-07-06T03:55:18.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.572+0000 7f7430ff9700 1 -- 172.21.15.97:0/2661704764 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7428011990 con 0x7f74340cd8e0 2023-07-06T03:55:18.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.572+0000 7f743a9ff700 1 -- 172.21.15.97:0/2661704764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74341f2ff0 con 0x7f74340cd8e0 2023-07-06T03:55:18.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.572+0000 7f7430ff9700 1 -- 172.21.15.97:0/2661704764 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428011d40 con 0x7f74340cd8e0 2023-07-06T03:55:18.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.572+0000 7f743a9ff700 1 -- 172.21.15.97:0/2661704764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74341f35b0 con 0x7f74340cd8e0 2023-07-06T03:55:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.573+0000 7f743a9ff700 1 -- 172.21.15.97:0/2661704764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7434070aa0 con 0x7f74340cd8e0 2023-07-06T03:55:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.574+0000 7f7430ff9700 1 -- 172.21.15.97:0/2661704764 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f742801a400 con 0x7f74340cd8e0 2023-07-06T03:55:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.575+0000 7f7430ff9700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f742006ace0 0x7f742006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.575+0000 7f7430ff9700 1 -- 172.21.15.97:0/2661704764 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74280890e0 con 0x7f74340cd8e0 2023-07-06T03:55:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.575+0000 7f7433fff700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f742006ace0 0x7f742006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.576+0000 7f7433fff700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f742006ace0 0x7f742006d190 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7424009130 tx=0x7f7424008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.579+0000 7f7430ff9700 1 -- 172.21.15.97:0/2661704764 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7428054220 con 0x7f74340cd8e0 2023-07-06T03:55:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.816+0000 7f743a9ff700 1 -- 172.21.15.97:0/2661704764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74341f37d0 con 0x7f74340cd8e0 2023-07-06T03:55:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.817+0000 7f7430ff9700 1 -- 172.21.15.97:0/2661704764 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f7428057840 con 0x7f74340cd8e0 2023-07-06T03:55:18.819 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:18.819 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3059444241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3059444241},{"type":"v1","addr":"172.21.15.146:6835","nonce":3059444241}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2624688851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2624688851},{"type":"v1","addr":"172.21.15.97:6837","nonce":2624688851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3938376776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3938376776},{"type":"v1","addr":"172.21.15.97:6835","nonce":3938376776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:55:14.806943+0000","modified":"2023-07-06T03:55:18.341811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9099":{"gid":9099,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/3269084599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3269084599},{"type":"v1","addr":"172.21.15.146:6834","nonce":3269084599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-06T03:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.819+0000 7f743a9ff700 1 -- 172.21.15.97:0/2661704764 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f742006ace0 msgr2=0x7f742006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.819+0000 7f743a9ff700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f742006ace0 0x7f742006d190 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7424009130 tx=0x7f7424008030 comp rx=0 tx=0).stop 2023-07-06T03:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.819+0000 7f743a9ff700 1 -- 172.21.15.97:0/2661704764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74340cd8e0 msgr2=0x7f7434148380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:18.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.819+0000 7f743a9ff700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74340cd8e0 0x7f7434148380 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f742800b4d0 tx=0x7f742800b890 comp rx=0 tx=0).stop 2023-07-06T03:55:18.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.819+0000 7f743a9ff700 1 -- 172.21.15.97:0/2661704764 shutdown_connections 2023-07-06T03:55:18.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.819+0000 7f743a9ff700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f742006ace0 0x7f742006d190 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.819+0000 7f743a9ff700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f743415b440 0x7f74341f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.819+0000 7f743a9ff700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7434070c30 0x7f7434152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.819+0000 7f743a9ff700 1 --2- 172.21.15.97:0/2661704764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74340cd8e0 0x7f7434148380 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:18.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.819+0000 7f743a9ff700 1 -- 172.21.15.97:0/2661704764 >> 172.21.15.97:0/2661704764 conn(0x7f74340ce8c0 msgr2=0x7f743415c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:18.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.820+0000 7f743a9ff700 1 -- 172.21.15.97:0/2661704764 shutdown_connections 2023-07-06T03:55:18.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:18.820+0000 7f743a9ff700 1 -- 172.21.15.97:0/2661704764 wait complete. 2023-07-06T03:55:18.823 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 92 2023-07-06T03:55:18.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:55:19.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.032+0000 7f64acfff700 1 Processor -- start 2023-07-06T03:55:19.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.032+0000 7f64acfff700 1 -- start start 2023-07-06T03:55:19.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.033+0000 7f64acfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f64a80cd8e0 0x7f64a815e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.033+0000 7f64acfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a815b440 0x7f64a80adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.033+0000 7f64acfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a80710d0 con 0x7f64a8070c30 2023-07-06T03:55:19.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.033+0000 7f64acfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a80ae3e0 con 0x7f64a80cd8e0 2023-07-06T03:55:19.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.033+0000 7f64acfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a8071330 con 0x7f64a815b440 2023-07-06T03:55:19.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.034+0000 7f64a659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f64a80cd8e0 0x7f64a815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.034+0000 7f64a5d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f64a8070c30 0x7f64a80cdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38388/0 (socket says 172.21.15.97:38388) 2023-07-06T03:55:19.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.034+0000 7f64a5d9b700 1 -- 172.21.15.97:0/1876473848 learned_addr learned my addr 172.21.15.97:0/1876473848 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:19.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.034+0000 7f64a6d9d700 1 --2- 172.21.15.97:0/1876473848 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a815b440 0x7f64a80adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.034+0000 7f64a659c700 1 -- 172.21.15.97:0/1876473848 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a815b440 msgr2=0x7f64a80adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:19.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.034+0000 7f64a659c700 1 --2- 172.21.15.97:0/1876473848 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a815b440 0x7f64a80adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.034+0000 7f64a659c700 1 -- 172.21.15.97:0/1876473848 >> v1:172.21.15.97:6789/0 conn(0x7f64a8070c30 legacy=0x7f64a80cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.034+0000 7f64a659c700 1 -- 172.21.15.97:0/1876473848 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8071610 con 0x7f64a80cd8e0 2023-07-06T03:55:19.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.034+0000 7f64a6d9d700 1 --2- 172.21.15.97:0/1876473848 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a815b440 0x7f64a80adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:19.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.034+0000 7f64a559a700 1 -- 172.21.15.97:0/1876473848 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579290391 0 0) 0x7f64a80710d0 con 0x7f64a8070c30 2023-07-06T03:55:19.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64a659c700 1 --2- 172.21.15.97:0/1876473848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a80cd8e0 0x7f64a815e860 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f6498009840 tx=0x7f6498009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2c4156a03ce346 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:19.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64a559a700 1 -- 172.21.15.97:0/1876473848 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498011830 con 0x7f64a80cd8e0 2023-07-06T03:55:19.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64a559a700 1 -- 172.21.15.97:0/1876473848 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6498011990 con 0x7f64a80cd8e0 2023-07-06T03:55:19.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64a559a700 1 -- 172.21.15.97:0/1876473848 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498011d20 con 0x7f64a80cd8e0 2023-07-06T03:55:19.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64acfff700 1 -- 172.21.15.97:0/1876473848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a80cd8e0 msgr2=0x7f64a815e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64acfff700 1 --2- 172.21.15.97:0/1876473848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a80cd8e0 0x7f64a815e860 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f6498009840 tx=0x7f6498009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:19.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64acfff700 1 -- 172.21.15.97:0/1876473848 shutdown_connections 2023-07-06T03:55:19.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64acfff700 1 --2- 172.21.15.97:0/1876473848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a80cd8e0 0x7f64a815e860 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64acfff700 1 --2- 172.21.15.97:0/1876473848 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a815b440 0x7f64a80adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64acfff700 1 -- 172.21.15.97:0/1876473848 >> 172.21.15.97:0/1876473848 conn(0x7f64a80ce8c0 msgr2=0x7f64a80cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64acfff700 1 -- 172.21.15.97:0/1876473848 shutdown_connections 2023-07-06T03:55:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.035+0000 7f64acfff700 1 -- 172.21.15.97:0/1876473848 wait complete. 2023-07-06T03:55:19.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.037+0000 7f64acfff700 1 Processor -- start 2023-07-06T03:55:19.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.037+0000 7f64acfff700 1 -- start start 2023-07-06T03:55:19.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.038+0000 7f64acfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a8070c30 0x7f64a81453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.038+0000 7f64acfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64a80cd8e0 0x7f64a80ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.038+0000 7f64acfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a815b440 0x7f64a80acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.038+0000 7f64acfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a8145900 con 0x7f64a815b440 2023-07-06T03:55:19.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.038+0000 7f64acfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a806c650 con 0x7f64a80cd8e0 2023-07-06T03:55:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.038+0000 7f64acfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a806c930 con 0x7f64a8070c30 2023-07-06T03:55:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.038+0000 7f64a6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a815b440 0x7f64a80acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f64a6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a815b440 0x7f64a80acdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40150/0 (socket says 172.21.15.97:40150) 2023-07-06T03:55:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f64a6d9d700 1 -- 172.21.15.97:0/2168117245 learned_addr learned my addr 172.21.15.97:0/2168117245 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f64a659c700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a8070c30 0x7f64a81453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f64a5d9b700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64a80cd8e0 0x7f64a80ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f64a6d9d700 1 -- 172.21.15.97:0/2168117245 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a8070c30 msgr2=0x7f64a81453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f64a6d9d700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a8070c30 0x7f64a81453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f64a6d9d700 1 -- 172.21.15.97:0/2168117245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64a80cd8e0 msgr2=0x7f64a80ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f64a6d9d700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64a80cd8e0 0x7f64a80ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f64a6d9d700 1 -- 172.21.15.97:0/2168117245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64980094a0 con 0x7f64a815b440 2023-07-06T03:55:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f64a6d9d700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a815b440 0x7f64a80acdb0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f64a000cea0 tx=0x7f64a000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f6496ffd700 1 -- 172.21.15.97:0/2168117245 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a000ad40 con 0x7f64a815b440 2023-07-06T03:55:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f6496ffd700 1 -- 172.21.15.97:0/2168117245 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a0003a20 con 0x7f64a815b440 2023-07-06T03:55:19.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.039+0000 7f6496ffd700 1 -- 172.21.15.97:0/2168117245 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a0003dd0 con 0x7f64a815b440 2023-07-06T03:55:19.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.040+0000 7f64acfff700 1 -- 172.21.15.97:0/2168117245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a806cc10 con 0x7f64a815b440 2023-07-06T03:55:19.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.040+0000 7f64acfff700 1 -- 172.21.15.97:0/2168117245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a806d1d0 con 0x7f64a815b440 2023-07-06T03:55:19.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.041+0000 7f64acfff700 1 -- 172.21.15.97:0/2168117245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a8070aa0 con 0x7f64a815b440 2023-07-06T03:55:19.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.041+0000 7f6496ffd700 1 -- 172.21.15.97:0/2168117245 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f64a0003f30 con 0x7f64a815b440 2023-07-06T03:55:19.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.043+0000 7f6496ffd700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648806ace0 0x7f648806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.043+0000 7f6496ffd700 1 -- 172.21.15.97:0/2168117245 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64a00897b0 con 0x7f64a815b440 2023-07-06T03:55:19.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.043+0000 7f64a659c700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648806ace0 0x7f648806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.044+0000 7f64a659c700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648806ace0 0x7f648806d190 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f64980090b0 tx=0x7f649801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:19.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.048+0000 7f6496ffd700 1 -- 172.21.15.97:0/2168117245 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f64a0054860 con 0x7f64a815b440 2023-07-06T03:55:19.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.243+0000 7f64acfff700 1 -- 172.21.15.97:0/2168117245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64a8067830 con 0x7f64a815b440 2023-07-06T03:55:19.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.244+0000 7f6496ffd700 1 -- 172.21.15.97:0/2168117245 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18319 (secure 0 0 0) 0x7f64a0057e80 con 0x7f64a815b440 2023-07-06T03:55:19.246 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:19.246 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":124,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:55:16.327257+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:55:12.989988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-06T03:55:13.782970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:55:19.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 -- 172.21.15.97:0/2168117245 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648806ace0 msgr2=0x7f648806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648806ace0 0x7f648806d190 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f64980090b0 tx=0x7f649801c040 comp rx=0 tx=0).stop 2023-07-06T03:55:19.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 -- 172.21.15.97:0/2168117245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a815b440 msgr2=0x7f64a80acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a815b440 0x7f64a80acdb0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f64a000cea0 tx=0x7f64a000ced0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 -- 172.21.15.97:0/2168117245 shutdown_connections 2023-07-06T03:55:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f648806ace0 0x7f648806d190 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64a815b440 0x7f64a80acdb0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64a8070c30 0x7f64a81453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 --2- 172.21.15.97:0/2168117245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64a80cd8e0 0x7f64a80ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 -- 172.21.15.97:0/2168117245 >> 172.21.15.97:0/2168117245 conn(0x7f64a80ce8c0 msgr2=0x7f64a80b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 -- 172.21.15.97:0/2168117245 shutdown_connections 2023-07-06T03:55:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.246+0000 7f64acfff700 1 -- 172.21.15.97:0/2168117245 wait complete. 2023-07-06T03:55:19.266 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T03:55:19.266 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:55:19.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.459+0000 7f10a3dff700 1 Processor -- start 2023-07-06T03:55:19.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.460+0000 7f10a3dff700 1 -- start start 2023-07-06T03:55:19.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a3dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f109c15c520 0x7f109c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a3dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f109c160180 0x7f109c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a3dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109c160b30 0x7f109c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a3dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c0cd8d0 con 0x7f109c160180 2023-07-06T03:55:19.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a3dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c0cdae0 con 0x7f109c15c520 2023-07-06T03:55:19.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a3dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0b6640 con 0x7f109c160b30 2023-07-06T03:55:19.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a139a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f109c160180 0x7f109c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a139a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f109c160180 0x7f109c1605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40156/0 (socket says 172.21.15.97:40156) 2023-07-06T03:55:19.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a139a700 1 -- 172.21.15.97:0/1509378559 learned_addr learned my addr 172.21.15.97:0/1509378559 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:19.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a239c700 1 --2- 172.21.15.97:0/1509378559 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109c160b30 0x7f109c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a1b9b700 1 --2- 172.21.15.97:0/1509378559 >> v2:172.21.15.146:3300/0 conn(0x7f109c15c520 0x7f109c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.461+0000 7f10a139a700 1 -- 172.21.15.97:0/1509378559 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109c160b30 msgr2=0x7f109c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:19.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.462+0000 7f10a139a700 1 --2- 172.21.15.97:0/1509378559 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109c160b30 0x7f109c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.462+0000 7f10a139a700 1 -- 172.21.15.97:0/1509378559 >> v2:172.21.15.146:3300/0 conn(0x7f109c15c520 msgr2=0x7f109c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:19.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.462+0000 7f10a139a700 1 --2- 172.21.15.97:0/1509378559 >> v2:172.21.15.146:3300/0 conn(0x7f109c15c520 0x7f109c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.462+0000 7f10a139a700 1 -- 172.21.15.97:0/1509378559 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c0b6920 con 0x7f109c160180 2023-07-06T03:55:19.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.462+0000 7f10a239c700 1 --2- 172.21.15.97:0/1509378559 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109c160b30 0x7f109c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:19.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.462+0000 7f10a1b9b700 1 --2- 172.21.15.97:0/1509378559 >> v2:172.21.15.146:3300/0 conn(0x7f109c15c520 0x7f109c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:19.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.462+0000 7f10a139a700 1 --2- 172.21.15.97:0/1509378559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f109c160180 0x7f109c1605f0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f109c0b4c70 tx=0x7f108c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fccedcc4ef4973 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.462+0000 7f10a0b99700 1 -- 172.21.15.97:0/1509378559 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00b870 con 0x7f109c160180 2023-07-06T03:55:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.462+0000 7f10a0b99700 1 -- 172.21.15.97:0/1509378559 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c00b9d0 con 0x7f109c160180 2023-07-06T03:55:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.462+0000 7f10a0b99700 1 -- 172.21.15.97:0/1509378559 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00bd60 con 0x7f109c160180 2023-07-06T03:55:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.463+0000 7f10a3dff700 1 -- 172.21.15.97:0/1509378559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f109c160180 msgr2=0x7f109c1605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.463+0000 7f10a3dff700 1 --2- 172.21.15.97:0/1509378559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f109c160180 0x7f109c1605f0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f109c0b4c70 tx=0x7f108c009bd0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.463+0000 7f10a3dff700 1 -- 172.21.15.97:0/1509378559 shutdown_connections 2023-07-06T03:55:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.463+0000 7f10a3dff700 1 --2- 172.21.15.97:0/1509378559 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109c160b30 0x7f109c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.463+0000 7f10a3dff700 1 --2- 172.21.15.97:0/1509378559 >> v2:172.21.15.146:3300/0 conn(0x7f109c15c520 0x7f109c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.463+0000 7f10a3dff700 1 --2- 172.21.15.97:0/1509378559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f109c160180 0x7f109c1605f0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.463+0000 7f10a3dff700 1 -- 172.21.15.97:0/1509378559 >> 172.21.15.97:0/1509378559 conn(0x7f109c071790 msgr2=0x7f109c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.463+0000 7f10a3dff700 1 -- 172.21.15.97:0/1509378559 shutdown_connections 2023-07-06T03:55:19.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.463+0000 7f10a3dff700 1 -- 172.21.15.97:0/1509378559 wait complete. 2023-07-06T03:55:19.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.465+0000 7f10a3dff700 1 Processor -- start 2023-07-06T03:55:19.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.465+0000 7f10a3dff700 1 -- start start 2023-07-06T03:55:19.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a3dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f109c15c520 0x7f109c15f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a3dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f109c160180 0x7f109c0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a3dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109c160b30 0x7f109c1f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a3dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0d3ad0 con 0x7f109c160180 2023-07-06T03:55:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a3dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0d1180 con 0x7f109c15c520 2023-07-06T03:55:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a3dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c15f8a0 con 0x7f109c160b30 2023-07-06T03:55:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f109c160180 0x7f109c0d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a139a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f109c160180 0x7f109c0d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40158/0 (socket says 172.21.15.97:40158) 2023-07-06T03:55:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a139a700 1 -- 172.21.15.97:0/3932937530 learned_addr learned my addr 172.21.15.97:0/3932937530 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a239c700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109c160b30 0x7f109c1f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a1b9b700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f109c15c520 0x7f109c15f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a139a700 1 -- 172.21.15.97:0/3932937530 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109c160b30 msgr2=0x7f109c1f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a139a700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109c160b30 0x7f109c1f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a139a700 1 -- 172.21.15.97:0/3932937530 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f109c15c520 msgr2=0x7f109c15f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a139a700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f109c15c520 0x7f109c15f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.466+0000 7f10a139a700 1 -- 172.21.15.97:0/3932937530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c009550 con 0x7f109c160180 2023-07-06T03:55:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.467+0000 7f10a139a700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f109c160180 0x7f109c0d0c40 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f109c06c240 tx=0x7f108c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.467+0000 7f10927fc700 1 -- 172.21.15.97:0/3932937530 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c004310 con 0x7f109c160180 2023-07-06T03:55:19.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.467+0000 7f10927fc700 1 -- 172.21.15.97:0/3932937530 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c01f070 con 0x7f109c160180 2023-07-06T03:55:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.467+0000 7f10a3dff700 1 -- 172.21.15.97:0/3932937530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c0d31e0 con 0x7f109c160180 2023-07-06T03:55:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.467+0000 7f10927fc700 1 -- 172.21.15.97:0/3932937530 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0036a0 con 0x7f109c160180 2023-07-06T03:55:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.467+0000 7f10a3dff700 1 -- 172.21.15.97:0/3932937530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c0d3760 con 0x7f109c160180 2023-07-06T03:55:19.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.468+0000 7f10a3dff700 1 -- 172.21.15.97:0/3932937530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c04e340 con 0x7f109c160180 2023-07-06T03:55:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.469+0000 7f10927fc700 1 -- 172.21.15.97:0/3932937530 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f108c021030 con 0x7f109c160180 2023-07-06T03:55:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.470+0000 7f10927fc700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f108006ace0 0x7f108006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.470+0000 7f10927fc700 1 -- 172.21.15.97:0/3932937530 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f108c08aa00 con 0x7f109c160180 2023-07-06T03:55:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.470+0000 7f10a1b9b700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f108006ace0 0x7f108006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.471+0000 7f10a1b9b700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f108006ace0 0x7f108006d190 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1098006fd0 tx=0x7f1098008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:19.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.474+0000 7f10927fc700 1 -- 172.21.15.97:0/3932937530 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f108c055bc0 con 0x7f109c160180 2023-07-06T03:55:19.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.663+0000 7f10a3dff700 1 -- 172.21.15.97:0/3932937530 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f109c0b6ee0 con 0x7f108006ace0 2023-07-06T03:55:19.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.672+0000 7f10927fc700 1 -- 172.21.15.97:0/3932937530 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311787 (secure 0 0 0) 0x7f109800a720 con 0x7f108006ace0 2023-07-06T03:55:19.677 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:19.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 -- 172.21.15.97:0/3932937530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f108006ace0 msgr2=0x7f108006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f108006ace0 0x7f108006d190 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1098006fd0 tx=0x7f1098008040 comp rx=0 tx=0).stop 2023-07-06T03:55:19.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 -- 172.21.15.97:0/3932937530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f109c160180 msgr2=0x7f109c0d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f109c160180 0x7f109c0d0c40 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f109c06c240 tx=0x7f108c004750 comp rx=0 tx=0).stop 2023-07-06T03:55:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 -- 172.21.15.97:0/3932937530 shutdown_connections 2023-07-06T03:55:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f108006ace0 0x7f108006d190 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f109c160b30 0x7f109c1f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f109c15c520 0x7f109c15f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 --2- 172.21.15.97:0/3932937530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f109c160180 0x7f109c0d0c40 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 -- 172.21.15.97:0/3932937530 >> 172.21.15.97:0/3932937530 conn(0x7f109c071790 msgr2=0x7f109c0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 -- 172.21.15.97:0/3932937530 shutdown_connections 2023-07-06T03:55:19.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.677+0000 7f10a3dff700 1 -- 172.21.15.97:0/3932937530 wait complete. 2023-07-06T03:55:19.681 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:55:19.697 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2023-07-06T03:55:19.014579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290228,"kb_used_data":13172,"kb_used_omap":0,"kb_used_meta":276992,"kb_avail":754684492,"statfs":{"total":773094113280,"available":772796919808,"internally_reserved":0,"allocated":13488128,"data_stored":6807255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":283639808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.710362"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.352560+0000","last_change":"2023-07-06T03:55:15.343577+0000","last_active":"2023-07-06T03:55:15.352560+0000","last_peered":"2023-07-06T03:55:15.352560+0000","last_clean":"2023-07-06T03:55:15.352560+0000","last_became_active":"2023-07-06T03:55:15.343280+0000","last_became_peered":"2023-07-06T03:55:15.343280+0000","last_unstale":"2023-07-06T03:55:15.352560+0000","last_undegraded":"2023-07-06T03:55:15.352560+0000","last_fullsized":"2023-07-06T03:55:15.352560+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:03:00.154330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340336+0000","last_change":"2023-07-06T03:55:15.335701+0000","last_active":"2023-07-06T03:55:15.340336+0000","last_peered":"2023-07-06T03:55:15.340336+0000","last_clean":"2023-07-06T03:55:15.340336+0000","last_became_active":"2023-07-06T03:55:15.335388+0000","last_became_peered":"2023-07-06T03:55:15.335388+0000","last_unstale":"2023-07-06T03:55:15.340336+0000","last_undegraded":"2023-07-06T03:55:15.340336+0000","last_fullsized":"2023-07-06T03:55:15.340336+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:11:40.364030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349844+0000","last_change":"2023-07-06T03:55:15.340385+0000","last_active":"2023-07-06T03:55:15.349844+0000","last_peered":"2023-07-06T03:55:15.349844+0000","last_clean":"2023-07-06T03:55:15.349844+0000","last_became_active":"2023-07-06T03:55:15.339887+0000","last_became_peered":"2023-07-06T03:55:15.339887+0000","last_unstale":"2023-07-06T03:55:15.349844+0000","last_undegraded":"2023-07-06T03:55:15.349844+0000","last_fullsized":"2023-07-06T03:55:15.349844+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:06:54.459675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.344424+0000","last_change":"2023-07-06T03:55:15.339620+0000","last_active":"2023-07-06T03:55:15.344424+0000","last_peered":"2023-07-06T03:55:15.344424+0000","last_clean":"2023-07-06T03:55:15.344424+0000","last_became_active":"2023-07-06T03:55:15.339330+0000","last_became_peered":"2023-07-06T03:55:15.339330+0000","last_unstale":"2023-07-06T03:55:15.344424+0000","last_undegraded":"2023-07-06T03:55:15.344424+0000","last_fullsized":"2023-07-06T03:55:15.344424+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:38:32.416488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.348604+0000","last_change":"2023-07-06T03:55:15.345165+0000","last_active":"2023-07-06T03:55:15.348604+0000","last_peered":"2023-07-06T03:55:15.348604+0000","last_clean":"2023-07-06T03:55:15.348604+0000","last_became_active":"2023-07-06T03:55:15.344902+0000","last_became_peered":"2023-07-06T03:55:15.344902+0000","last_unstale":"2023-07-06T03:55:15.348604+0000","last_undegraded":"2023-07-06T03:55:15.348604+0000","last_fullsized":"2023-07-06T03:55:15.348604+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:28:25.690369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349077+0000","last_change":"2023-07-06T03:55:15.344276+0000","last_active":"2023-07-06T03:55:15.349077+0000","last_peered":"2023-07-06T03:55:15.349077+0000","last_clean":"2023-07-06T03:55:15.349077+0000","last_became_active":"2023-07-06T03:55:15.343896+0000","last_became_peered":"2023-07-06T03:55:15.343896+0000","last_unstale":"2023-07-06T03:55:15.349077+0000","last_undegraded":"2023-07-06T03:55:15.349077+0000","last_fullsized":"2023-07-06T03:55:15.349077+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:27:47.599953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.341267+0000","last_change":"2023-07-06T03:55:15.338317+0000","last_active":"2023-07-06T03:55:15.341267+0000","last_peered":"2023-07-06T03:55:15.341267+0000","last_clean":"2023-07-06T03:55:15.341267+0000","last_became_active":"2023-07-06T03:55:15.338062+0000","last_became_peered":"2023-07-06T03:55:15.338062+0000","last_unstale":"2023-07-06T03:55:15.341267+0000","last_undegraded":"2023-07-06T03:55:15.341267+0000","last_fullsized":"2023-07-06T03:55:15.341267+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:43:07.567807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.350665+0000","last_change":"2023-07-06T03:55:15.345178+0000","last_active":"2023-07-06T03:55:15.350665+0000","last_peered":"2023-07-06T03:55:15.350665+0000","last_clean":"2023-07-06T03:55:15.350665+0000","last_became_active":"2023-07-06T03:55:15.344887+0000","last_became_peered":"2023-07-06T03:55:15.344887+0000","last_unstale":"2023-07-06T03:55:15.350665+0000","last_undegraded":"2023-07-06T03:55:15.350665+0000","last_fullsized":"2023-07-06T03:55:15.350665+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:17:09.949975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353683+0000","last_change":"2023-07-06T03:55:15.348757+0000","last_active":"2023-07-06T03:55:15.353683+0000","last_peered":"2023-07-06T03:55:15.353683+0000","last_clean":"2023-07-06T03:55:15.353683+0000","last_became_active":"2023-07-06T03:55:15.348374+0000","last_became_peered":"2023-07-06T03:55:15.348374+0000","last_unstale":"2023-07-06T03:55:15.353683+0000","last_undegraded":"2023-07-06T03:55:15.353683+0000","last_fullsized":"2023-07-06T03:55:15.353683+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:24:53.959530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353386+0000","last_change":"2023-07-06T03:55:15.346780+0000","last_active":"2023-07-06T03:55:15.353386+0000","last_peered":"2023-07-06T03:55:15.353386+0000","last_clean":"2023-07-06T03:55:15.353386+0000","last_became_active":"2023-07-06T03:55:15.346507+0000","last_became_peered":"2023-07-06T03:55:15.346507+0000","last_unstale":"2023-07-06T03:55:15.353386+0000","last_undegraded":"2023-07-06T03:55:15.353386+0000","last_fullsized":"2023-07-06T03:55:15.353386+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:20:21.647465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.341628+0000","last_change":"2023-07-06T03:55:15.334194+0000","last_active":"2023-07-06T03:55:15.341628+0000","last_peered":"2023-07-06T03:55:15.341628+0000","last_clean":"2023-07-06T03:55:15.341628+0000","last_became_active":"2023-07-06T03:55:15.333933+0000","last_became_peered":"2023-07-06T03:55:15.333933+0000","last_unstale":"2023-07-06T03:55:15.341628+0000","last_undegraded":"2023-07-06T03:55:15.341628+0000","last_fullsized":"2023-07-06T03:55:15.341628+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:58:42.142451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.338401+0000","last_change":"2023-07-06T03:55:15.334254+0000","last_active":"2023-07-06T03:55:15.338401+0000","last_peered":"2023-07-06T03:55:15.338401+0000","last_clean":"2023-07-06T03:55:15.338401+0000","last_became_active":"2023-07-06T03:55:15.333945+0000","last_became_peered":"2023-07-06T03:55:15.333945+0000","last_unstale":"2023-07-06T03:55:15.338401+0000","last_undegraded":"2023-07-06T03:55:15.338401+0000","last_fullsized":"2023-07-06T03:55:15.338401+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:17:55.000000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.344695+0000","last_change":"2023-07-06T03:55:15.340327+0000","last_active":"2023-07-06T03:55:15.344695+0000","last_peered":"2023-07-06T03:55:15.344695+0000","last_clean":"2023-07-06T03:55:15.344695+0000","last_became_active":"2023-07-06T03:55:15.340004+0000","last_became_peered":"2023-07-06T03:55:15.340004+0000","last_unstale":"2023-07-06T03:55:15.344695+0000","last_undegraded":"2023-07-06T03:55:15.344695+0000","last_fullsized":"2023-07-06T03:55:15.344695+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:04:15.446407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.351933+0000","last_change":"2023-07-06T03:55:15.342731+0000","last_active":"2023-07-06T03:55:15.351933+0000","last_peered":"2023-07-06T03:55:15.351933+0000","last_clean":"2023-07-06T03:55:15.351933+0000","last_became_active":"2023-07-06T03:55:15.342455+0000","last_became_peered":"2023-07-06T03:55:15.342455+0000","last_unstale":"2023-07-06T03:55:15.351933+0000","last_undegraded":"2023-07-06T03:55:15.351933+0000","last_fullsized":"2023-07-06T03:55:15.351933+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:17.748239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.350389+0000","last_change":"2023-07-06T03:55:15.341040+0000","last_active":"2023-07-06T03:55:15.350389+0000","last_peered":"2023-07-06T03:55:15.350389+0000","last_clean":"2023-07-06T03:55:15.350389+0000","last_became_active":"2023-07-06T03:55:15.340748+0000","last_became_peered":"2023-07-06T03:55:15.340748+0000","last_unstale":"2023-07-06T03:55:15.350389+0000","last_undegraded":"2023-07-06T03:55:15.350389+0000","last_fullsized":"2023-07-06T03:55:15.350389+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:06:43.147768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349651+0000","last_change":"2023-07-06T03:55:15.338316+0000","last_active":"2023-07-06T03:55:15.349651+0000","last_peered":"2023-07-06T03:55:15.349651+0000","last_clean":"2023-07-06T03:55:15.349651+0000","last_became_active":"2023-07-06T03:55:15.338031+0000","last_became_peered":"2023-07-06T03:55:15.338031+0000","last_unstale":"2023-07-06T03:55:15.349651+0000","last_undegraded":"2023-07-06T03:55:15.349651+0000","last_fullsized":"2023-07-06T03:55:15.349651+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:40:57.335083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.352583+0000","last_change":"2023-07-06T03:55:15.343984+0000","last_active":"2023-07-06T03:55:15.352583+0000","last_peered":"2023-07-06T03:55:15.352583+0000","last_clean":"2023-07-06T03:55:15.352583+0000","last_became_active":"2023-07-06T03:55:15.343446+0000","last_became_peered":"2023-07-06T03:55:15.343446+0000","last_unstale":"2023-07-06T03:55:15.352583+0000","last_undegraded":"2023-07-06T03:55:15.352583+0000","last_fullsized":"2023-07-06T03:55:15.352583+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:03.763283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.338808+0000","last_change":"2023-07-06T03:55:15.336341+0000","last_active":"2023-07-06T03:55:15.338808+0000","last_peered":"2023-07-06T03:55:15.338808+0000","last_clean":"2023-07-06T03:55:15.338808+0000","last_became_active":"2023-07-06T03:55:15.336033+0000","last_became_peered":"2023-07-06T03:55:15.336033+0000","last_unstale":"2023-07-06T03:55:15.338808+0000","last_undegraded":"2023-07-06T03:55:15.338808+0000","last_fullsized":"2023-07-06T03:55:15.338808+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:18.056091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.345371+0000","last_change":"2023-07-06T03:55:15.342068+0000","last_active":"2023-07-06T03:55:15.345371+0000","last_peered":"2023-07-06T03:55:15.345371+0000","last_clean":"2023-07-06T03:55:15.345371+0000","last_became_active":"2023-07-06T03:55:15.341810+0000","last_became_peered":"2023-07-06T03:55:15.341810+0000","last_unstale":"2023-07-06T03:55:15.345371+0000","last_undegraded":"2023-07-06T03:55:15.345371+0000","last_fullsized":"2023-07-06T03:55:15.345371+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:18:59.607705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340295+0000","last_change":"2023-07-06T03:55:15.338238+0000","last_active":"2023-07-06T03:55:15.340295+0000","last_peered":"2023-07-06T03:55:15.340295+0000","last_clean":"2023-07-06T03:55:15.340295+0000","last_became_active":"2023-07-06T03:55:15.337935+0000","last_became_peered":"2023-07-06T03:55:15.337935+0000","last_unstale":"2023-07-06T03:55:15.340295+0000","last_undegraded":"2023-07-06T03:55:15.340295+0000","last_fullsized":"2023-07-06T03:55:15.340295+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:01:12.221859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349711+0000","last_change":"2023-07-06T03:55:15.340350+0000","last_active":"2023-07-06T03:55:15.349711+0000","last_peered":"2023-07-06T03:55:15.349711+0000","last_clean":"2023-07-06T03:55:15.349711+0000","last_became_active":"2023-07-06T03:55:15.340078+0000","last_became_peered":"2023-07-06T03:55:15.340078+0000","last_unstale":"2023-07-06T03:55:15.349711+0000","last_undegraded":"2023-07-06T03:55:15.349711+0000","last_fullsized":"2023-07-06T03:55:15.349711+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:24:35.800135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.344500+0000","last_change":"2023-07-06T03:55:15.340286+0000","last_active":"2023-07-06T03:55:15.344500+0000","last_peered":"2023-07-06T03:55:15.344500+0000","last_clean":"2023-07-06T03:55:15.344500+0000","last_became_active":"2023-07-06T03:55:15.339990+0000","last_became_peered":"2023-07-06T03:55:15.339990+0000","last_unstale":"2023-07-06T03:55:15.344500+0000","last_undegraded":"2023-07-06T03:55:15.344500+0000","last_fullsized":"2023-07-06T03:55:15.344500+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:16:29.789589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343421+0000","last_change":"2023-07-06T03:55:15.341897+0000","last_active":"2023-07-06T03:55:15.343421+0000","last_peered":"2023-07-06T03:55:15.343421+0000","last_clean":"2023-07-06T03:55:15.343421+0000","last_became_active":"2023-07-06T03:55:15.341618+0000","last_became_peered":"2023-07-06T03:55:15.341618+0000","last_unstale":"2023-07-06T03:55:15.343421+0000","last_undegraded":"2023-07-06T03:55:15.343421+0000","last_fullsized":"2023-07-06T03:55:15.343421+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:15:41.115884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354857+0000","last_change":"2023-07-06T03:55:15.349071+0000","last_active":"2023-07-06T03:55:15.354857+0000","last_peered":"2023-07-06T03:55:15.354857+0000","last_clean":"2023-07-06T03:55:15.354857+0000","last_became_active":"2023-07-06T03:55:15.348330+0000","last_became_peered":"2023-07-06T03:55:15.348330+0000","last_unstale":"2023-07-06T03:55:15.354857+0000","last_undegraded":"2023-07-06T03:55:15.354857+0000","last_fullsized":"2023-07-06T03:55:15.354857+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:57:38.531759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340636+0000","last_change":"2023-07-06T03:55:15.331929+0000","last_active":"2023-07-06T03:55:15.340636+0000","last_peered":"2023-07-06T03:55:15.340636+0000","last_clean":"2023-07-06T03:55:15.340636+0000","last_became_active":"2023-07-06T03:55:15.331641+0000","last_became_peered":"2023-07-06T03:55:15.331641+0000","last_unstale":"2023-07-06T03:55:15.340636+0000","last_undegraded":"2023-07-06T03:55:15.340636+0000","last_fullsized":"2023-07-06T03:55:15.340636+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:18:11.226972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340746+0000","last_change":"2023-07-06T03:55:15.334980+0000","last_active":"2023-07-06T03:55:15.340746+0000","last_peered":"2023-07-06T03:55:15.340746+0000","last_clean":"2023-07-06T03:55:15.340746+0000","last_became_active":"2023-07-06T03:55:15.334699+0000","last_became_peered":"2023-07-06T03:55:15.334699+0000","last_unstale":"2023-07-06T03:55:15.340746+0000","last_undegraded":"2023-07-06T03:55:15.340746+0000","last_fullsized":"2023-07-06T03:55:15.340746+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:42:25.047155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354884+0000","last_change":"2023-07-06T03:55:15.346424+0000","last_active":"2023-07-06T03:55:15.354884+0000","last_peered":"2023-07-06T03:55:15.354884+0000","last_clean":"2023-07-06T03:55:15.354884+0000","last_became_active":"2023-07-06T03:55:15.345464+0000","last_became_peered":"2023-07-06T03:55:15.345464+0000","last_unstale":"2023-07-06T03:55:15.354884+0000","last_undegraded":"2023-07-06T03:55:15.354884+0000","last_fullsized":"2023-07-06T03:55:15.354884+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:32:56.241778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.336430+0000","last_change":"2023-07-06T03:55:15.335155+0000","last_active":"2023-07-06T03:55:15.336430+0000","last_peered":"2023-07-06T03:55:15.336430+0000","last_clean":"2023-07-06T03:55:15.336430+0000","last_became_active":"2023-07-06T03:55:15.334898+0000","last_became_peered":"2023-07-06T03:55:15.334898+0000","last_unstale":"2023-07-06T03:55:15.336430+0000","last_undegraded":"2023-07-06T03:55:15.336430+0000","last_fullsized":"2023-07-06T03:55:15.336430+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:03.763283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353757+0000","last_change":"2023-07-06T03:55:15.350709+0000","last_active":"2023-07-06T03:55:15.353757+0000","last_peered":"2023-07-06T03:55:15.353757+0000","last_clean":"2023-07-06T03:55:15.353757+0000","last_became_active":"2023-07-06T03:55:15.350426+0000","last_became_peered":"2023-07-06T03:55:15.350426+0000","last_unstale":"2023-07-06T03:55:15.353757+0000","last_undegraded":"2023-07-06T03:55:15.353757+0000","last_fullsized":"2023-07-06T03:55:15.353757+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:10:57.317137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343401+0000","last_change":"2023-07-06T03:55:15.341531+0000","last_active":"2023-07-06T03:55:15.343401+0000","last_peered":"2023-07-06T03:55:15.343401+0000","last_clean":"2023-07-06T03:55:15.343401+0000","last_became_active":"2023-07-06T03:55:15.341233+0000","last_became_peered":"2023-07-06T03:55:15.341233+0000","last_unstale":"2023-07-06T03:55:15.343401+0000","last_undegraded":"2023-07-06T03:55:15.343401+0000","last_fullsized":"2023-07-06T03:55:15.343401+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:17:02.188812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354916+0000","last_change":"2023-07-06T03:55:15.352539+0000","last_active":"2023-07-06T03:55:15.354916+0000","last_peered":"2023-07-06T03:55:15.354916+0000","last_clean":"2023-07-06T03:55:15.354916+0000","last_became_active":"2023-07-06T03:55:15.352278+0000","last_became_peered":"2023-07-06T03:55:15.352278+0000","last_unstale":"2023-07-06T03:55:15.354916+0000","last_undegraded":"2023-07-06T03:55:15.354916+0000","last_fullsized":"2023-07-06T03:55:15.354916+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:11:52.405524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353772+0000","last_change":"2023-07-06T03:55:15.349545+0000","last_active":"2023-07-06T03:55:15.353772+0000","last_peered":"2023-07-06T03:55:15.353772+0000","last_clean":"2023-07-06T03:55:15.353772+0000","last_became_active":"2023-07-06T03:55:15.349240+0000","last_became_peered":"2023-07-06T03:55:15.349240+0000","last_unstale":"2023-07-06T03:55:15.353772+0000","last_undegraded":"2023-07-06T03:55:15.353772+0000","last_fullsized":"2023-07-06T03:55:15.353772+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:12:37.373095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340669+0000","last_change":"2023-07-06T03:55:15.332572+0000","last_active":"2023-07-06T03:55:15.340669+0000","last_peered":"2023-07-06T03:55:15.340669+0000","last_clean":"2023-07-06T03:55:15.340669+0000","last_became_active":"2023-07-06T03:55:15.332255+0000","last_became_peered":"2023-07-06T03:55:15.332255+0000","last_unstale":"2023-07-06T03:55:15.340669+0000","last_undegraded":"2023-07-06T03:55:15.340669+0000","last_fullsized":"2023-07-06T03:55:15.340669+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:47:18.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1f","version":"123'4","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.355169+0000","last_change":"2023-07-06T03:55:14.334903+0000","last_active":"2023-07-06T03:55:15.355169+0000","last_peered":"2023-07-06T03:55:15.355169+0000","last_clean":"2023-07-06T03:55:15.355169+0000","last_became_active":"2023-07-06T03:55:14.334552+0000","last_became_peered":"2023-07-06T03:55:14.334552+0000","last_unstale":"2023-07-06T03:55:15.355169+0000","last_undegraded":"2023-07-06T03:55:15.355169+0000","last_fullsized":"2023-07-06T03:55:15.355169+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:30:08.064899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353405+0000","last_change":"2023-07-06T03:55:15.347549+0000","last_active":"2023-07-06T03:55:15.353405+0000","last_peered":"2023-07-06T03:55:15.353405+0000","last_clean":"2023-07-06T03:55:15.353405+0000","last_became_active":"2023-07-06T03:55:15.347285+0000","last_became_peered":"2023-07-06T03:55:15.347285+0000","last_unstale":"2023-07-06T03:55:15.353405+0000","last_undegraded":"2023-07-06T03:55:15.353405+0000","last_fullsized":"2023-07-06T03:55:15.353405+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:01:12.002592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.1e","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.357175+0000","last_change":"2023-07-06T03:55:14.342032+0000","last_active":"2023-07-06T03:55:15.357175+0000","last_peered":"2023-07-06T03:55:15.357175+0000","last_clean":"2023-07-06T03:55:15.357175+0000","last_became_active":"2023-07-06T03:55:14.341762+0000","last_became_peered":"2023-07-06T03:55:14.341762+0000","last_unstale":"2023-07-06T03:55:15.357175+0000","last_undegraded":"2023-07-06T03:55:15.357175+0000","last_fullsized":"2023-07-06T03:55:15.357175+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:48:51.017885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343388+0000","last_change":"2023-07-06T03:55:15.341443+0000","last_active":"2023-07-06T03:55:15.343388+0000","last_peered":"2023-07-06T03:55:15.343388+0000","last_clean":"2023-07-06T03:55:15.343388+0000","last_became_active":"2023-07-06T03:55:15.341164+0000","last_became_peered":"2023-07-06T03:55:15.341164+0000","last_unstale":"2023-07-06T03:55:15.343388+0000","last_undegraded":"2023-07-06T03:55:15.343388+0000","last_fullsized":"2023-07-06T03:55:15.343388+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:27:12.742803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1d","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.355100+0000","last_change":"2023-07-06T03:55:14.332729+0000","last_active":"2023-07-06T03:55:15.355100+0000","last_peered":"2023-07-06T03:55:15.355100+0000","last_clean":"2023-07-06T03:55:15.355100+0000","last_became_active":"2023-07-06T03:55:14.332440+0000","last_became_peered":"2023-07-06T03:55:14.332440+0000","last_unstale":"2023-07-06T03:55:15.355100+0000","last_undegraded":"2023-07-06T03:55:15.355100+0000","last_fullsized":"2023-07-06T03:55:15.355100+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:54:46.655901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.337782+0000","last_change":"2023-07-06T03:55:15.335040+0000","last_active":"2023-07-06T03:55:15.337782+0000","last_peered":"2023-07-06T03:55:15.337782+0000","last_clean":"2023-07-06T03:55:15.337782+0000","last_became_active":"2023-07-06T03:55:15.334773+0000","last_became_peered":"2023-07-06T03:55:15.334773+0000","last_unstale":"2023-07-06T03:55:15.337782+0000","last_undegraded":"2023-07-06T03:55:15.337782+0000","last_fullsized":"2023-07-06T03:55:15.337782+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:11:10.242306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1c","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.356116+0000","last_change":"2023-07-06T03:55:14.337620+0000","last_active":"2023-07-06T03:55:15.356116+0000","last_peered":"2023-07-06T03:55:15.356116+0000","last_clean":"2023-07-06T03:55:15.356116+0000","last_became_active":"2023-07-06T03:55:14.337323+0000","last_became_peered":"2023-07-06T03:55:14.337323+0000","last_unstale":"2023-07-06T03:55:15.356116+0000","last_undegraded":"2023-07-06T03:55:15.356116+0000","last_fullsized":"2023-07-06T03:55:15.356116+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:04:02.874486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.351385+0000","last_change":"2023-07-06T03:55:15.347942+0000","last_active":"2023-07-06T03:55:15.351385+0000","last_peered":"2023-07-06T03:55:15.351385+0000","last_clean":"2023-07-06T03:55:15.351385+0000","last_became_active":"2023-07-06T03:55:15.347192+0000","last_became_peered":"2023-07-06T03:55:15.347192+0000","last_unstale":"2023-07-06T03:55:15.351385+0000","last_undegraded":"2023-07-06T03:55:15.351385+0000","last_fullsized":"2023-07-06T03:55:15.351385+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:43:31.772144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1b","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353895+0000","last_change":"2023-07-06T03:55:14.339675+0000","last_active":"2023-07-06T03:55:15.353895+0000","last_peered":"2023-07-06T03:55:15.353895+0000","last_clean":"2023-07-06T03:55:15.353895+0000","last_became_active":"2023-07-06T03:55:14.339335+0000","last_became_peered":"2023-07-06T03:55:14.339335+0000","last_unstale":"2023-07-06T03:55:15.353895+0000","last_undegraded":"2023-07-06T03:55:15.353895+0000","last_fullsized":"2023-07-06T03:55:15.353895+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:52:40.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.351690+0000","last_change":"2023-07-06T03:55:15.349764+0000","last_active":"2023-07-06T03:55:15.351690+0000","last_peered":"2023-07-06T03:55:15.351690+0000","last_clean":"2023-07-06T03:55:15.351690+0000","last_became_active":"2023-07-06T03:55:15.349451+0000","last_became_peered":"2023-07-06T03:55:15.349451+0000","last_unstale":"2023-07-06T03:55:15.351690+0000","last_undegraded":"2023-07-06T03:55:15.351690+0000","last_fullsized":"2023-07-06T03:55:15.351690+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:58:08.809003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1a","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.356321+0000","last_change":"2023-07-06T03:55:14.324495+0000","last_active":"2023-07-06T03:55:15.356321+0000","last_peered":"2023-07-06T03:55:15.356321+0000","last_clean":"2023-07-06T03:55:15.356321+0000","last_became_active":"2023-07-06T03:55:14.324172+0000","last_became_peered":"2023-07-06T03:55:14.324172+0000","last_unstale":"2023-07-06T03:55:15.356321+0000","last_undegraded":"2023-07-06T03:55:15.356321+0000","last_fullsized":"2023-07-06T03:55:15.356321+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:58:19.753702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.352821+0000","last_change":"2023-07-06T03:55:15.345661+0000","last_active":"2023-07-06T03:55:15.352821+0000","last_peered":"2023-07-06T03:55:15.352821+0000","last_clean":"2023-07-06T03:55:15.352821+0000","last_became_active":"2023-07-06T03:55:15.345377+0000","last_became_peered":"2023-07-06T03:55:15.345377+0000","last_unstale":"2023-07-06T03:55:15.352821+0000","last_undegraded":"2023-07-06T03:55:15.352821+0000","last_fullsized":"2023-07-06T03:55:15.352821+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:35:01.979264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.328899+0000","last_change":"2023-07-06T03:55:14.335681+0000","last_active":"2023-07-06T03:55:15.328899+0000","last_peered":"2023-07-06T03:55:15.328899+0000","last_clean":"2023-07-06T03:55:15.328899+0000","last_became_active":"2023-07-06T03:55:14.335042+0000","last_became_peered":"2023-07-06T03:55:14.335042+0000","last_unstale":"2023-07-06T03:55:15.328899+0000","last_undegraded":"2023-07-06T03:55:15.328899+0000","last_fullsized":"2023-07-06T03:55:15.328899+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:58:18.787436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.341889+0000","last_change":"2023-07-06T03:55:15.336689+0000","last_active":"2023-07-06T03:55:15.341889+0000","last_peered":"2023-07-06T03:55:15.341889+0000","last_clean":"2023-07-06T03:55:15.341889+0000","last_became_active":"2023-07-06T03:55:15.336383+0000","last_became_peered":"2023-07-06T03:55:15.336383+0000","last_unstale":"2023-07-06T03:55:15.341889+0000","last_undegraded":"2023-07-06T03:55:15.341889+0000","last_fullsized":"2023-07-06T03:55:15.341889+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:50:17.782805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.332124+0000","last_change":"2023-07-06T03:55:14.330764+0000","last_active":"2023-07-06T03:55:15.332124+0000","last_peered":"2023-07-06T03:55:15.332124+0000","last_clean":"2023-07-06T03:55:15.332124+0000","last_became_active":"2023-07-06T03:55:14.330521+0000","last_became_peered":"2023-07-06T03:55:14.330521+0000","last_unstale":"2023-07-06T03:55:15.332124+0000","last_undegraded":"2023-07-06T03:55:15.332124+0000","last_fullsized":"2023-07-06T03:55:15.332124+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:13:03.702548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.355783+0000","last_change":"2023-07-06T03:55:15.352105+0000","last_active":"2023-07-06T03:55:15.355783+0000","last_peered":"2023-07-06T03:55:15.355783+0000","last_clean":"2023-07-06T03:55:15.355783+0000","last_became_active":"2023-07-06T03:55:15.351760+0000","last_became_peered":"2023-07-06T03:55:15.351760+0000","last_unstale":"2023-07-06T03:55:15.355783+0000","last_undegraded":"2023-07-06T03:55:15.355783+0000","last_fullsized":"2023-07-06T03:55:15.355783+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:30:29.772585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.f","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.355682+0000","last_change":"2023-07-06T03:55:14.334289+0000","last_active":"2023-07-06T03:55:15.355682+0000","last_peered":"2023-07-06T03:55:15.355682+0000","last_clean":"2023-07-06T03:55:15.355682+0000","last_became_active":"2023-07-06T03:55:14.333890+0000","last_became_peered":"2023-07-06T03:55:14.333890+0000","last_unstale":"2023-07-06T03:55:15.355682+0000","last_undegraded":"2023-07-06T03:55:15.355682+0000","last_fullsized":"2023-07-06T03:55:15.355682+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:15.050981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343857+0000","last_change":"2023-07-06T03:55:15.338482+0000","last_active":"2023-07-06T03:55:15.343857+0000","last_peered":"2023-07-06T03:55:15.343857+0000","last_clean":"2023-07-06T03:55:15.343857+0000","last_became_active":"2023-07-06T03:55:15.338227+0000","last_became_peered":"2023-07-06T03:55:15.338227+0000","last_unstale":"2023-07-06T03:55:15.343857+0000","last_undegraded":"2023-07-06T03:55:15.343857+0000","last_fullsized":"2023-07-06T03:55:15.343857+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:12:49.936924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.e","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354619+0000","last_change":"2023-07-06T03:55:14.338984+0000","last_active":"2023-07-06T03:55:15.354619+0000","last_peered":"2023-07-06T03:55:15.354619+0000","last_clean":"2023-07-06T03:55:15.354619+0000","last_became_active":"2023-07-06T03:55:14.338704+0000","last_became_peered":"2023-07-06T03:55:14.338704+0000","last_unstale":"2023-07-06T03:55:15.354619+0000","last_undegraded":"2023-07-06T03:55:15.354619+0000","last_fullsized":"2023-07-06T03:55:15.354619+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:04:12.651513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343483+0000","last_change":"2023-07-06T03:55:15.335580+0000","last_active":"2023-07-06T03:55:15.343483+0000","last_peered":"2023-07-06T03:55:15.343483+0000","last_clean":"2023-07-06T03:55:15.343483+0000","last_became_active":"2023-07-06T03:55:15.334347+0000","last_became_peered":"2023-07-06T03:55:15.334347+0000","last_unstale":"2023-07-06T03:55:15.343483+0000","last_undegraded":"2023-07-06T03:55:15.343483+0000","last_fullsized":"2023-07-06T03:55:15.343483+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:59.852629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.d","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354199+0000","last_change":"2023-07-06T03:55:14.335419+0000","last_active":"2023-07-06T03:55:15.354199+0000","last_peered":"2023-07-06T03:55:15.354199+0000","last_clean":"2023-07-06T03:55:15.354199+0000","last_became_active":"2023-07-06T03:55:14.335135+0000","last_became_peered":"2023-07-06T03:55:14.335135+0000","last_unstale":"2023-07-06T03:55:15.354199+0000","last_undegraded":"2023-07-06T03:55:15.354199+0000","last_fullsized":"2023-07-06T03:55:15.354199+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:38:58.335529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.348578+0000","last_change":"2023-07-06T03:55:15.344256+0000","last_active":"2023-07-06T03:55:15.348578+0000","last_peered":"2023-07-06T03:55:15.348578+0000","last_clean":"2023-07-06T03:55:15.348578+0000","last_became_active":"2023-07-06T03:55:15.343887+0000","last_became_peered":"2023-07-06T03:55:15.343887+0000","last_unstale":"2023-07-06T03:55:15.348578+0000","last_undegraded":"2023-07-06T03:55:15.348578+0000","last_fullsized":"2023-07-06T03:55:15.348578+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:10.694869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.328190+0000","last_change":"2023-07-06T03:55:14.334539+0000","last_active":"2023-07-06T03:55:15.328190+0000","last_peered":"2023-07-06T03:55:15.328190+0000","last_clean":"2023-07-06T03:55:15.328190+0000","last_became_active":"2023-07-06T03:55:14.334232+0000","last_became_peered":"2023-07-06T03:55:14.334232+0000","last_unstale":"2023-07-06T03:55:15.328190+0000","last_undegraded":"2023-07-06T03:55:15.328190+0000","last_fullsized":"2023-07-06T03:55:15.328190+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:49:19.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.351784+0000","last_change":"2023-07-06T03:55:15.348686+0000","last_active":"2023-07-06T03:55:15.351784+0000","last_peered":"2023-07-06T03:55:15.351784+0000","last_clean":"2023-07-06T03:55:15.351784+0000","last_became_active":"2023-07-06T03:55:15.348414+0000","last_became_peered":"2023-07-06T03:55:15.348414+0000","last_unstale":"2023-07-06T03:55:15.351784+0000","last_undegraded":"2023-07-06T03:55:15.351784+0000","last_fullsized":"2023-07-06T03:55:15.351784+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:10:13.145178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.b","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.357265+0000","last_change":"2023-07-06T03:55:14.336950+0000","last_active":"2023-07-06T03:55:15.357265+0000","last_peered":"2023-07-06T03:55:15.357265+0000","last_clean":"2023-07-06T03:55:15.357265+0000","last_became_active":"2023-07-06T03:55:14.336584+0000","last_became_peered":"2023-07-06T03:55:14.336584+0000","last_unstale":"2023-07-06T03:55:15.357265+0000","last_undegraded":"2023-07-06T03:55:15.357265+0000","last_fullsized":"2023-07-06T03:55:15.357265+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:43:13.070358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349360+0000","last_change":"2023-07-06T03:55:15.339307+0000","last_active":"2023-07-06T03:55:15.349360+0000","last_peered":"2023-07-06T03:55:15.349360+0000","last_clean":"2023-07-06T03:55:15.349360+0000","last_became_active":"2023-07-06T03:55:15.339015+0000","last_became_peered":"2023-07-06T03:55:15.339015+0000","last_unstale":"2023-07-06T03:55:15.349360+0000","last_undegraded":"2023-07-06T03:55:15.349360+0000","last_fullsized":"2023-07-06T03:55:15.349360+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:13:05.423702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.328724+0000","last_change":"2023-07-06T03:55:14.335098+0000","last_active":"2023-07-06T03:55:15.328724+0000","last_peered":"2023-07-06T03:55:15.328724+0000","last_clean":"2023-07-06T03:55:15.328724+0000","last_became_active":"2023-07-06T03:55:14.334810+0000","last_became_peered":"2023-07-06T03:55:14.334810+0000","last_unstale":"2023-07-06T03:55:15.328724+0000","last_undegraded":"2023-07-06T03:55:15.328724+0000","last_fullsized":"2023-07-06T03:55:15.328724+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:15:58.117913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.347478+0000","last_change":"2023-07-06T03:55:15.340417+0000","last_active":"2023-07-06T03:55:15.347478+0000","last_peered":"2023-07-06T03:55:15.347478+0000","last_clean":"2023-07-06T03:55:15.347478+0000","last_became_active":"2023-07-06T03:55:15.340091+0000","last_became_peered":"2023-07-06T03:55:15.340091+0000","last_unstale":"2023-07-06T03:55:15.347478+0000","last_undegraded":"2023-07-06T03:55:15.347478+0000","last_fullsized":"2023-07-06T03:55:15.347478+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:50:05.835855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.332088+0000","last_change":"2023-07-06T03:55:14.333988+0000","last_active":"2023-07-06T03:55:15.332088+0000","last_peered":"2023-07-06T03:55:15.332088+0000","last_clean":"2023-07-06T03:55:15.332088+0000","last_became_active":"2023-07-06T03:55:14.333716+0000","last_became_peered":"2023-07-06T03:55:14.333716+0000","last_unstale":"2023-07-06T03:55:15.332088+0000","last_undegraded":"2023-07-06T03:55:15.332088+0000","last_fullsized":"2023-07-06T03:55:15.332088+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:29:50.378446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343994+0000","last_change":"2023-07-06T03:55:15.338488+0000","last_active":"2023-07-06T03:55:15.343994+0000","last_peered":"2023-07-06T03:55:15.343994+0000","last_clean":"2023-07-06T03:55:15.343994+0000","last_became_active":"2023-07-06T03:55:15.337648+0000","last_became_peered":"2023-07-06T03:55:15.337648+0000","last_unstale":"2023-07-06T03:55:15.343994+0000","last_undegraded":"2023-07-06T03:55:15.343994+0000","last_fullsized":"2023-07-06T03:55:15.343994+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:57.917700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.330325+0000","last_change":"2023-07-06T03:55:14.336520+0000","last_active":"2023-07-06T03:55:15.330325+0000","last_peered":"2023-07-06T03:55:15.330325+0000","last_clean":"2023-07-06T03:55:15.330325+0000","last_became_active":"2023-07-06T03:55:14.336237+0000","last_became_peered":"2023-07-06T03:55:14.336237+0000","last_unstale":"2023-07-06T03:55:15.330325+0000","last_undegraded":"2023-07-06T03:55:15.330325+0000","last_fullsized":"2023-07-06T03:55:15.330325+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:11:39.684006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354838+0000","last_change":"2023-07-06T03:55:15.345865+0000","last_active":"2023-07-06T03:55:15.354838+0000","last_peered":"2023-07-06T03:55:15.354838+0000","last_clean":"2023-07-06T03:55:15.354838+0000","last_became_active":"2023-07-06T03:55:15.344128+0000","last_became_peered":"2023-07-06T03:55:15.344128+0000","last_unstale":"2023-07-06T03:55:15.354838+0000","last_undegraded":"2023-07-06T03:55:15.354838+0000","last_fullsized":"2023-07-06T03:55:15.354838+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:36:15.916915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.7","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.356446+0000","last_change":"2023-07-06T03:55:14.333636+0000","last_active":"2023-07-06T03:55:15.356446+0000","last_peered":"2023-07-06T03:55:15.356446+0000","last_clean":"2023-07-06T03:55:15.356446+0000","last_became_active":"2023-07-06T03:55:14.333254+0000","last_became_peered":"2023-07-06T03:55:14.333254+0000","last_unstale":"2023-07-06T03:55:15.356446+0000","last_undegraded":"2023-07-06T03:55:15.356446+0000","last_fullsized":"2023-07-06T03:55:15.356446+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:55:21.319302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353707+0000","last_change":"2023-07-06T03:55:15.349910+0000","last_active":"2023-07-06T03:55:15.353707+0000","last_peered":"2023-07-06T03:55:15.353707+0000","last_clean":"2023-07-06T03:55:15.353707+0000","last_became_active":"2023-07-06T03:55:15.348656+0000","last_became_peered":"2023-07-06T03:55:15.348656+0000","last_unstale":"2023-07-06T03:55:15.353707+0000","last_undegraded":"2023-07-06T03:55:15.353707+0000","last_fullsized":"2023-07-06T03:55:15.353707+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:09:16.721736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.328064+0000","last_change":"2023-07-06T03:55:14.330046+0000","last_active":"2023-07-06T03:55:15.328064+0000","last_peered":"2023-07-06T03:55:15.328064+0000","last_clean":"2023-07-06T03:55:15.328064+0000","last_became_active":"2023-07-06T03:55:14.329777+0000","last_became_peered":"2023-07-06T03:55:14.329777+0000","last_unstale":"2023-07-06T03:55:15.328064+0000","last_undegraded":"2023-07-06T03:55:15.328064+0000","last_fullsized":"2023-07-06T03:55:15.328064+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:34:14.143126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340703+0000","last_change":"2023-07-06T03:55:15.335914+0000","last_active":"2023-07-06T03:55:15.340703+0000","last_peered":"2023-07-06T03:55:15.340703+0000","last_clean":"2023-07-06T03:55:15.340703+0000","last_became_active":"2023-07-06T03:55:15.335623+0000","last_became_peered":"2023-07-06T03:55:15.335623+0000","last_unstale":"2023-07-06T03:55:15.340703+0000","last_undegraded":"2023-07-06T03:55:15.340703+0000","last_fullsized":"2023-07-06T03:55:15.340703+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:23.315172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.324234+0000","last_change":"2023-07-06T03:55:14.335060+0000","last_active":"2023-07-06T03:55:15.324234+0000","last_peered":"2023-07-06T03:55:15.324234+0000","last_clean":"2023-07-06T03:55:15.324234+0000","last_became_active":"2023-07-06T03:55:14.334711+0000","last_became_peered":"2023-07-06T03:55:14.334711+0000","last_unstale":"2023-07-06T03:55:15.324234+0000","last_undegraded":"2023-07-06T03:55:15.324234+0000","last_fullsized":"2023-07-06T03:55:15.324234+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:07:59.933123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.344715+0000","last_change":"2023-07-06T03:55:15.340403+0000","last_active":"2023-07-06T03:55:15.344715+0000","last_peered":"2023-07-06T03:55:15.344715+0000","last_clean":"2023-07-06T03:55:15.344715+0000","last_became_active":"2023-07-06T03:55:15.340031+0000","last_became_peered":"2023-07-06T03:55:15.340031+0000","last_unstale":"2023-07-06T03:55:15.344715+0000","last_undegraded":"2023-07-06T03:55:15.344715+0000","last_fullsized":"2023-07-06T03:55:15.344715+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:40:57.335083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.331875+0000","last_change":"2023-07-06T03:55:14.339015+0000","last_active":"2023-07-06T03:55:15.331875+0000","last_peered":"2023-07-06T03:55:15.331875+0000","last_clean":"2023-07-06T03:55:15.331875+0000","last_became_active":"2023-07-06T03:55:14.338726+0000","last_became_peered":"2023-07-06T03:55:14.338726+0000","last_unstale":"2023-07-06T03:55:15.331875+0000","last_undegraded":"2023-07-06T03:55:15.331875+0000","last_fullsized":"2023-07-06T03:55:15.331875+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:51:35.103485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349860+0000","last_change":"2023-07-06T03:55:15.338210+0000","last_active":"2023-07-06T03:55:15.349860+0000","last_peered":"2023-07-06T03:55:15.349860+0000","last_clean":"2023-07-06T03:55:15.349860+0000","last_became_active":"2023-07-06T03:55:15.337915+0000","last_became_peered":"2023-07-06T03:55:15.337915+0000","last_unstale":"2023-07-06T03:55:15.349860+0000","last_undegraded":"2023-07-06T03:55:15.349860+0000","last_fullsized":"2023-07-06T03:55:15.349860+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:06:43.147768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.332480+0000","last_change":"2023-07-06T03:55:14.330713+0000","last_active":"2023-07-06T03:55:15.332480+0000","last_peered":"2023-07-06T03:55:15.332480+0000","last_clean":"2023-07-06T03:55:15.332480+0000","last_became_active":"2023-07-06T03:55:14.330452+0000","last_became_peered":"2023-07-06T03:55:14.330452+0000","last_unstale":"2023-07-06T03:55:15.332480+0000","last_undegraded":"2023-07-06T03:55:15.332480+0000","last_fullsized":"2023-07-06T03:55:15.332480+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:24:05.850586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349335+0000","last_change":"2023-07-06T03:55:15.338417+0000","last_active":"2023-07-06T03:55:15.349335+0000","last_peered":"2023-07-06T03:55:15.349335+0000","last_clean":"2023-07-06T03:55:15.349335+0000","last_became_active":"2023-07-06T03:55:15.337525+0000","last_became_peered":"2023-07-06T03:55:15.337525+0000","last_unstale":"2023-07-06T03:55:15.349335+0000","last_undegraded":"2023-07-06T03:55:15.349335+0000","last_fullsized":"2023-07-06T03:55:15.349335+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:16:29.789589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.332153+0000","last_change":"2023-07-06T03:55:14.332276+0000","last_active":"2023-07-06T03:55:15.332153+0000","last_peered":"2023-07-06T03:55:15.332153+0000","last_clean":"2023-07-06T03:55:15.332153+0000","last_became_active":"2023-07-06T03:55:14.332036+0000","last_became_peered":"2023-07-06T03:55:14.332036+0000","last_unstale":"2023-07-06T03:55:15.332153+0000","last_undegraded":"2023-07-06T03:55:15.332153+0000","last_fullsized":"2023-07-06T03:55:15.332153+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:20:15.921178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.338562+0000","last_change":"2023-07-06T03:55:15.333974+0000","last_active":"2023-07-06T03:55:15.338562+0000","last_peered":"2023-07-06T03:55:15.338562+0000","last_clean":"2023-07-06T03:55:15.338562+0000","last_became_active":"2023-07-06T03:55:15.333676+0000","last_became_peered":"2023-07-06T03:55:15.333676+0000","last_unstale":"2023-07-06T03:55:15.338562+0000","last_undegraded":"2023-07-06T03:55:15.338562+0000","last_fullsized":"2023-07-06T03:55:15.338562+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:13:05.423702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.329435+0000","last_change":"2023-07-06T03:55:14.339197+0000","last_active":"2023-07-06T03:55:15.329435+0000","last_peered":"2023-07-06T03:55:15.329435+0000","last_clean":"2023-07-06T03:55:15.329435+0000","last_became_active":"2023-07-06T03:55:14.338922+0000","last_became_peered":"2023-07-06T03:55:14.338922+0000","last_unstale":"2023-07-06T03:55:15.329435+0000","last_undegraded":"2023-07-06T03:55:15.329435+0000","last_fullsized":"2023-07-06T03:55:15.329435+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:30:10.278374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343644+0000","last_change":"2023-07-06T03:55:15.335004+0000","last_active":"2023-07-06T03:55:15.343644+0000","last_peered":"2023-07-06T03:55:15.343644+0000","last_clean":"2023-07-06T03:55:15.343644+0000","last_became_active":"2023-07-06T03:55:15.334747+0000","last_became_peered":"2023-07-06T03:55:15.334747+0000","last_unstale":"2023-07-06T03:55:15.343644+0000","last_undegraded":"2023-07-06T03:55:15.343644+0000","last_fullsized":"2023-07-06T03:55:15.343644+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:00:09.721559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.321990+0000","last_change":"2023-07-06T03:55:14.331689+0000","last_active":"2023-07-06T03:55:15.321990+0000","last_peered":"2023-07-06T03:55:15.321990+0000","last_clean":"2023-07-06T03:55:15.321990+0000","last_became_active":"2023-07-06T03:55:14.331398+0000","last_became_peered":"2023-07-06T03:55:14.331398+0000","last_unstale":"2023-07-06T03:55:15.321990+0000","last_undegraded":"2023-07-06T03:55:15.321990+0000","last_fullsized":"2023-07-06T03:55:15.321990+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:51:21.435963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.351144+0000","last_change":"2023-07-06T03:55:15.344625+0000","last_active":"2023-07-06T03:55:15.351144+0000","last_peered":"2023-07-06T03:55:15.351144+0000","last_clean":"2023-07-06T03:55:15.351144+0000","last_became_active":"2023-07-06T03:55:15.343908+0000","last_became_peered":"2023-07-06T03:55:15.343908+0000","last_unstale":"2023-07-06T03:55:15.351144+0000","last_undegraded":"2023-07-06T03:55:15.351144+0000","last_fullsized":"2023-07-06T03:55:15.351144+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:04:00.734485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"123'3","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.356761+0000","last_change":"2023-07-06T03:55:14.330576+0000","last_active":"2023-07-06T03:55:15.356761+0000","last_peered":"2023-07-06T03:55:15.356761+0000","last_clean":"2023-07-06T03:55:15.356761+0000","last_became_active":"2023-07-06T03:55:14.330111+0000","last_became_peered":"2023-07-06T03:55:14.330111+0000","last_unstale":"2023-07-06T03:55:15.356761+0000","last_undegraded":"2023-07-06T03:55:15.356761+0000","last_fullsized":"2023-07-06T03:55:15.356761+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:13.654894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":191,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.328030+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:55:15.328030+0000","last_peered":"2023-07-06T03:55:15.328030+0000","last_clean":"2023-07-06T03:55:15.328030+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:55:15.328030+0000","last_undegraded":"2023-07-06T03:55:15.328030+0000","last_fullsized":"2023-07-06T03:55:15.328030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:29:44.344080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.352355+0000","last_change":"2023-07-06T03:55:15.343507+0000","last_active":"2023-07-06T03:55:15.352355+0000","last_peered":"2023-07-06T03:55:15.352355+0000","last_clean":"2023-07-06T03:55:15.352355+0000","last_became_active":"2023-07-06T03:55:15.343202+0000","last_became_peered":"2023-07-06T03:55:15.343202+0000","last_unstale":"2023-07-06T03:55:15.352355+0000","last_undegraded":"2023-07-06T03:55:15.352355+0000","last_fullsized":"2023-07-06T03:55:15.352355+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:13:30.631195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.13","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.357455+0000","last_change":"2023-07-06T03:55:14.336329+0000","last_active":"2023-07-06T03:55:15.357455+0000","last_peered":"2023-07-06T03:55:15.357455+0000","last_clean":"2023-07-06T03:55:15.357455+0000","last_became_active":"2023-07-06T03:55:14.336019+0000","last_became_peered":"2023-07-06T03:55:14.336019+0000","last_unstale":"2023-07-06T03:55:15.357455+0000","last_undegraded":"2023-07-06T03:55:15.357455+0000","last_fullsized":"2023-07-06T03:55:15.357455+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:11:07.809332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":205,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.320479+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:55:15.320479+0000","last_peered":"2023-07-06T03:55:15.320479+0000","last_clean":"2023-07-06T03:55:15.320479+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:55:15.320479+0000","last_undegraded":"2023-07-06T03:55:15.320479+0000","last_fullsized":"2023-07-06T03:55:15.320479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:53:07.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.348615+0000","last_change":"2023-07-06T03:55:15.342873+0000","last_active":"2023-07-06T03:55:15.348615+0000","last_peered":"2023-07-06T03:55:15.348615+0000","last_clean":"2023-07-06T03:55:15.348615+0000","last_became_active":"2023-07-06T03:55:15.342532+0000","last_became_peered":"2023-07-06T03:55:15.342532+0000","last_unstale":"2023-07-06T03:55:15.348615+0000","last_undegraded":"2023-07-06T03:55:15.348615+0000","last_fullsized":"2023-07-06T03:55:15.348615+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:33:38.030192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"123'8","reported_seq":19,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.358638+0000","last_change":"2023-07-06T03:55:14.341914+0000","last_active":"2023-07-06T03:55:15.358638+0000","last_peered":"2023-07-06T03:55:15.358638+0000","last_clean":"2023-07-06T03:55:15.358638+0000","last_became_active":"2023-07-06T03:55:14.341678+0000","last_became_peered":"2023-07-06T03:55:14.341678+0000","last_unstale":"2023-07-06T03:55:15.358638+0000","last_undegraded":"2023-07-06T03:55:15.358638+0000","last_fullsized":"2023-07-06T03:55:15.358638+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:19:40.839313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":209,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.322805+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:55:15.322805+0000","last_peered":"2023-07-06T03:55:15.322805+0000","last_clean":"2023-07-06T03:55:15.322805+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:55:15.322805+0000","last_undegraded":"2023-07-06T03:55:15.322805+0000","last_fullsized":"2023-07-06T03:55:15.322805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:02:27.182880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354404+0000","last_change":"2023-07-06T03:55:15.350728+0000","last_active":"2023-07-06T03:55:15.354404+0000","last_peered":"2023-07-06T03:55:15.354404+0000","last_clean":"2023-07-06T03:55:15.354404+0000","last_became_active":"2023-07-06T03:55:15.350446+0000","last_became_peered":"2023-07-06T03:55:15.350446+0000","last_unstale":"2023-07-06T03:55:15.354404+0000","last_undegraded":"2023-07-06T03:55:15.354404+0000","last_fullsized":"2023-07-06T03:55:15.354404+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:20:01.214350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.320383+0000","last_change":"2023-07-06T03:55:14.335781+0000","last_active":"2023-07-06T03:55:15.320383+0000","last_peered":"2023-07-06T03:55:15.320383+0000","last_clean":"2023-07-06T03:55:15.320383+0000","last_became_active":"2023-07-06T03:55:14.334645+0000","last_became_peered":"2023-07-06T03:55:14.334645+0000","last_unstale":"2023-07-06T03:55:15.320383+0000","last_undegraded":"2023-07-06T03:55:15.320383+0000","last_fullsized":"2023-07-06T03:55:15.320383+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:32:11.846098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":205,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.324070+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:55:15.324070+0000","last_peered":"2023-07-06T03:55:15.324070+0000","last_clean":"2023-07-06T03:55:15.324070+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:55:15.324070+0000","last_undegraded":"2023-07-06T03:55:15.324070+0000","last_fullsized":"2023-07-06T03:55:15.324070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:47:19.788235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":344,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.327623+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:55:15.327623+0000","last_peered":"2023-07-06T03:55:15.327623+0000","last_clean":"2023-07-06T03:55:15.327623+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:55:15.327623+0000","last_undegraded":"2023-07-06T03:55:15.327623+0000","last_fullsized":"2023-07-06T03:55:15.327623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:49:16.388742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.355392+0000","last_change":"2023-07-06T03:55:15.353242+0000","last_active":"2023-07-06T03:55:15.355392+0000","last_peered":"2023-07-06T03:55:15.355392+0000","last_clean":"2023-07-06T03:55:15.355392+0000","last_became_active":"2023-07-06T03:55:15.352933+0000","last_became_peered":"2023-07-06T03:55:15.352933+0000","last_unstale":"2023-07-06T03:55:15.355392+0000","last_undegraded":"2023-07-06T03:55:15.355392+0000","last_fullsized":"2023-07-06T03:55:15.355392+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:06:08.221246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.331948+0000","last_change":"2023-07-06T03:55:14.332810+0000","last_active":"2023-07-06T03:55:15.331948+0000","last_peered":"2023-07-06T03:55:15.331948+0000","last_clean":"2023-07-06T03:55:15.331948+0000","last_became_active":"2023-07-06T03:55:14.332548+0000","last_became_peered":"2023-07-06T03:55:14.332548+0000","last_unstale":"2023-07-06T03:55:15.331948+0000","last_undegraded":"2023-07-06T03:55:15.331948+0000","last_fullsized":"2023-07-06T03:55:15.331948+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:53.023505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":206,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.320639+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:55:15.320639+0000","last_peered":"2023-07-06T03:55:15.320639+0000","last_clean":"2023-07-06T03:55:15.320639+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:55:15.320639+0000","last_undegraded":"2023-07-06T03:55:15.320639+0000","last_fullsized":"2023-07-06T03:55:15.320639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:12:13.805158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.344545+0000","last_change":"2023-07-06T03:55:15.338743+0000","last_active":"2023-07-06T03:55:15.344545+0000","last_peered":"2023-07-06T03:55:15.344545+0000","last_clean":"2023-07-06T03:55:15.344545+0000","last_became_active":"2023-07-06T03:55:15.338481+0000","last_became_peered":"2023-07-06T03:55:15.338481+0000","last_unstale":"2023-07-06T03:55:15.344545+0000","last_undegraded":"2023-07-06T03:55:15.344545+0000","last_fullsized":"2023-07-06T03:55:15.344545+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:57:40.184420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"123'6","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.358836+0000","last_change":"2023-07-06T03:55:14.329404+0000","last_active":"2023-07-06T03:55:15.358836+0000","last_peered":"2023-07-06T03:55:15.358836+0000","last_clean":"2023-07-06T03:55:15.358836+0000","last_became_active":"2023-07-06T03:55:14.329121+0000","last_became_peered":"2023-07-06T03:55:14.329121+0000","last_unstale":"2023-07-06T03:55:15.358836+0000","last_undegraded":"2023-07-06T03:55:15.358836+0000","last_fullsized":"2023-07-06T03:55:15.358836+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:47:09.939272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.324256+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:55:15.324256+0000","last_peered":"2023-07-06T03:55:15.324256+0000","last_clean":"2023-07-06T03:55:15.324256+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:55:15.324256+0000","last_undegraded":"2023-07-06T03:55:15.324256+0000","last_fullsized":"2023-07-06T03:55:15.324256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:40:44.784735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.339487+0000","last_change":"2023-07-06T03:55:15.335912+0000","last_active":"2023-07-06T03:55:15.339487+0000","last_peered":"2023-07-06T03:55:15.339487+0000","last_clean":"2023-07-06T03:55:15.339487+0000","last_became_active":"2023-07-06T03:55:15.335555+0000","last_became_peered":"2023-07-06T03:55:15.335555+0000","last_unstale":"2023-07-06T03:55:15.339487+0000","last_undegraded":"2023-07-06T03:55:15.339487+0000","last_fullsized":"2023-07-06T03:55:15.339487+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:35:39.943450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.321647+0000","last_change":"2023-07-06T03:55:14.326092+0000","last_active":"2023-07-06T03:55:15.321647+0000","last_peered":"2023-07-06T03:55:15.321647+0000","last_clean":"2023-07-06T03:55:15.321647+0000","last_became_active":"2023-07-06T03:55:14.324656+0000","last_became_peered":"2023-07-06T03:55:14.324656+0000","last_unstale":"2023-07-06T03:55:15.321647+0000","last_undegraded":"2023-07-06T03:55:15.321647+0000","last_fullsized":"2023-07-06T03:55:15.321647+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:24:09.584868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":197,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.327872+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:55:15.327872+0000","last_peered":"2023-07-06T03:55:15.327872+0000","last_clean":"2023-07-06T03:55:15.327872+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:55:15.327872+0000","last_undegraded":"2023-07-06T03:55:15.327872+0000","last_fullsized":"2023-07-06T03:55:15.327872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:27:35.438765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.350873+0000","last_change":"2023-07-06T03:55:15.344207+0000","last_active":"2023-07-06T03:55:15.350873+0000","last_peered":"2023-07-06T03:55:15.350873+0000","last_clean":"2023-07-06T03:55:15.350873+0000","last_became_active":"2023-07-06T03:55:15.343943+0000","last_became_peered":"2023-07-06T03:55:15.343943+0000","last_unstale":"2023-07-06T03:55:15.350873+0000","last_undegraded":"2023-07-06T03:55:15.350873+0000","last_fullsized":"2023-07-06T03:55:15.350873+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:17.748239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.327777+0000","last_change":"2023-07-06T03:55:14.334603+0000","last_active":"2023-07-06T03:55:15.327777+0000","last_peered":"2023-07-06T03:55:15.327777+0000","last_clean":"2023-07-06T03:55:15.327777+0000","last_became_active":"2023-07-06T03:55:14.334317+0000","last_became_peered":"2023-07-06T03:55:14.334317+0000","last_unstale":"2023-07-06T03:55:15.327777+0000","last_undegraded":"2023-07-06T03:55:15.327777+0000","last_fullsized":"2023-07-06T03:55:15.327777+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:21:08.819941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":205,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.321008+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:55:15.321008+0000","last_peered":"2023-07-06T03:55:15.321008+0000","last_clean":"2023-07-06T03:55:15.321008+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:55:15.321008+0000","last_undegraded":"2023-07-06T03:55:15.321008+0000","last_fullsized":"2023-07-06T03:55:15.321008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:18.889558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36644,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335196,"statfs":{"total":96636764160,"available":96599240704,"internally_reserved":0,"allocated":1536000,"data_stored":705609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68600000000000005,"15min":0.70499999999999996},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.88300000000000001,"5min":1.0620000000000001,"15min":4.0570000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67900000000000005,"15min":0.752},"min":{"1min":0.39900000000000002,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.85499999999999998,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68300000000000005,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.45000000000000001,"15min":0.34000000000000002},"max":{"1min":1.016,"5min":1.091,"15min":2.6219999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69099999999999995,"15min":0.82699999999999996},"min":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68100000000000005,"15min":0.76000000000000001},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.35199999999999998},"max":{"1min":0.80600000000000005,"5min":0.98599999999999999,"15min":3.77},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.48099999999999998,"5min":0.47799999999999998,"15min":0.45000000000000001},"max":{"1min":0.90800000000000003,"5min":0.94399999999999995,"15min":0.98899999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.753,"15min":0.752},"min":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.43099999999999999},"max":{"1min":0.91700000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.62,"15min":0.626},"min":{"1min":0.501,"5min":0.36099999999999999,"15min":0.34599999999999997},"max":{"1min":0.89600000000000002,"5min":0.90200000000000002,"15min":0.94699999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66800000000000004,"15min":0.70399999999999996},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.374},"max":{"1min":0.94099999999999995,"5min":0.94399999999999995,"15min":1.4590000000000001},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.52800000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.45300000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.56799999999999995,"5min":0.438,"15min":0.438},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.5349999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.77700000000000002},"min":{"1min":0.49199999999999999,"5min":0.46500000000000002,"15min":0.42299999999999999},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.80800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574975,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1540096,"data_stored":705643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72799999999999998,"15min":0.67400000000000004},"min":{"1min":0.39800000000000002,"5min":0.32800000000000001,"15min":0.29399999999999998},"max":{"1min":0.89500000000000002,"5min":1.24,"15min":1.24},"last":5.6269999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.77800000000000002,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.44700000000000001,"15min":0.26900000000000002},"max":{"1min":1.046,"5min":3.3399999999999999,"15min":3.3399999999999999},"last":3.3119999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68799999999999994,"15min":0.66900000000000004},"min":{"1min":0.44500000000000001,"5min":0.372,"15min":0.34599999999999997},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":5.8860000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70199999999999996,"15min":0.67800000000000005},"min":{"1min":0.51800000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94499999999999995,"5min":0.94699999999999995,"15min":1.1759999999999999},"last":5.96}]},{"osd":2,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.752,"15min":0.68200000000000005},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.98499999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":8.3740000000000006},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77500000000000002,"15min":0.72799999999999998},"min":{"1min":0.60499999999999998,"5min":0.47699999999999998,"15min":0.38700000000000001},"max":{"1min":1.153,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":5.8449999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":1.085,"5min":1.2110000000000001,"15min":2.1850000000000001},"last":3.4689999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.373},"max":{"1min":1.071,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":8.5269999999999992}]},{"osd":4,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.69899999999999995},"min":{"1min":0.44900000000000001,"5min":0.42799999999999999,"15min":0.33200000000000002},"max":{"1min":0.83499999999999996,"5min":1.0309999999999999,"15min":1.1339999999999999},"last":5.7759999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.71699999999999997},"min":{"1min":0.42399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":0.98699999999999999,"15min":1.085},"last":3.5190000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65400000000000003,"15min":0.69599999999999995},"min":{"1min":0.48699999999999999,"5min":0.46300000000000002,"15min":0.38600000000000001},"max":{"1min":0.89500000000000002,"5min":0.98799999999999999,"15min":1.218},"last":8.4870000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72399999999999998,"15min":0.77000000000000002},"min":{"1min":0.54700000000000004,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0489999999999999,"15min":1.198},"last":5.923}]},{"osd":7,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72499999999999998,"15min":0.79500000000000004},"min":{"1min":0.54700000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.323},"last":3.3900000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74399999999999999,"15min":0.80600000000000005},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.41499999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.4139999999999999},"last":8.4429999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":2096,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":2146304,"data_stored":1297631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.42999999999999999,"15min":0.50800000000000001},"min":{"1min":0.27100000000000002,"5min":0.25900000000000001,"15min":0.23699999999999999},"max":{"1min":0.56000000000000005,"5min":0.72299999999999998,"15min":2.8879999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50700000000000001,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.32300000000000001,"15min":0.313},"max":{"1min":0.621,"5min":0.78300000000000003,"15min":1.96},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.44600000000000001,"15min":0.78100000000000003},"min":{"1min":0.32300000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.58599999999999997,"5min":0.66800000000000004,"15min":13.055999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.54100000000000004,"15min":0.60599999999999998},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.59799999999999998,"5min":0.96199999999999997,"15min":2.8479999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52800000000000002,"15min":0.58699999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48299999999999998,"15min":0.52600000000000002},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.56899999999999995,"5min":0.71599999999999997,"15min":1.2709999999999999},"last":0.44}]},{"osd":4,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.73599999999999999,"15min":1.05},"min":{"1min":0.54000000000000004,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.90400000000000003,"5min":1,"15min":12.989000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78200000000000003,"15min":0.81399999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":0.875,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.66500000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.432},"max":{"1min":0.88300000000000001,"5min":1.0860000000000001,"15min":1.3100000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72199999999999998,"15min":1.052},"min":{"1min":0.56100000000000005,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.83299999999999996,"5min":1.077,"15min":13.023999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.76200000000000001,"15min":0.79800000000000004},"min":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88700000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":1.0960000000000001},"min":{"1min":0.59399999999999997,"5min":0.54000000000000004,"15min":0.47799999999999998},"max":{"1min":0.96399999999999997,"5min":1.0569999999999999,"15min":12.941000000000001},"last":0.63}]},{"osd":7,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.80900000000000005},"min":{"1min":0.48699999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92600000000000005,"5min":1.071,"15min":2.806},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.755,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94199999999999995,"5min":1.1319999999999999,"15min":1.337},"last":0.67500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1552384,"data_stored":707263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.28899999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":0.57099999999999995,"5min":0.753,"15min":0.753},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.58099999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.436}]},{"osd":2,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.66200000000000003,"5min":0.69199999999999995,"15min":0.73699999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48699999999999999,"15min":0.496},"min":{"1min":0.376,"5min":0.315,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51100000000000001,"15min":0.48499999999999999},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.29899999999999999},"max":{"1min":0.59099999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.443,"5min":0.39400000000000002,"15min":0.38800000000000001},"max":{"1min":0.628,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72699999999999998,"15min":0.73799999999999999},"min":{"1min":0.48399999999999999,"5min":0.45700000000000002,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":1.1699999999999999,"15min":1.171},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.59199999999999997,"5min":0.52400000000000002,"15min":0.36799999999999999},"max":{"1min":0.92200000000000004,"5min":1.032,"15min":1.034},"last":0.627}]},{"osd":5,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.72799999999999998},"min":{"1min":0.51100000000000001,"5min":0.503,"15min":0.45600000000000002},"max":{"1min":0.95399999999999996,"5min":1.1080000000000001,"15min":1.1419999999999999},"last":0.755},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.55100000000000005,"5min":0.53600000000000003,"15min":0.46999999999999997},"max":{"1min":0.82199999999999995,"5min":1.1040000000000001,"15min":1.1220000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.84599999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73599999999999999,"15min":0.748},"min":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.46200000000000002},"max":{"1min":0.93999999999999995,"5min":1.0649999999999999,"15min":1.546},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78200000000000003,"15min":0.79400000000000004},"min":{"1min":0.56000000000000005,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.6830000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.76900000000000002,"15min":0.98999999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53100000000000003},"max":{"1min":0.90500000000000003,"5min":1.0669999999999999,"15min":9.5960000000000001},"last":0.80100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40728,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331112,"statfs":{"total":96636764160,"available":96595058688,"internally_reserved":0,"allocated":1523712,"data_stored":698481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.47699999999999998,"15min":0.442},"min":{"1min":0.30499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65500000000000003,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.504,"15min":0.47999999999999998},"min":{"1min":0.34200000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56000000000000005,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.55000000000000004,"15min":0.51000000000000001},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.629,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.56599999999999995,"15min":0.52400000000000002},"min":{"1min":0.38,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.64600000000000002,"5min":3.657,"15min":3.657},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.626,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69099999999999995,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59699999999999998,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94499999999999995,"15min":0.78600000000000003},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.121,"5min":12.007,"15min":12.007},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.81299999999999994,"15min":0.73699999999999999},"min":{"1min":0.52200000000000002,"5min":0.49299999999999999,"15min":0.36399999999999999},"max":{"1min":1.1930000000000001,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79200000000000004,"15min":0.75},"min":{"1min":0.51100000000000001,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":1.1459999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.747,"5min":0.88100000000000001,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.137,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.94599999999999995,"15min":0.78900000000000003},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.165,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.92300000000000004,"15min":0.80900000000000005},"min":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":1.107,"5min":13.084,"15min":13.084},"last":0.84299999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83299999999999996,"15min":0.82499999999999996},"min":{"1min":0.60599999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.089,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.871,"15min":0.81799999999999995},"min":{"1min":0.55800000000000005,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.2170000000000001,"5min":4.484,"15min":4.484},"last":0.79800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1523712,"data_stored":698481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.44900000000000001,"15min":0.56399999999999995},"min":{"1min":0.25700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60099999999999998,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.44},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51800000000000002,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.85699999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.45200000000000001,"15min":0.88300000000000001},"min":{"1min":0.316,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.54700000000000004,"5min":1.3620000000000001,"15min":15.911},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.53400000000000003,"15min":0.95699999999999996},"min":{"1min":0.33400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76000000000000001,"5min":1.347,"15min":16.024000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.499,"15min":0.499},"min":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.27400000000000002},"max":{"1min":0.61099999999999999,"5min":3.25,"15min":3.25},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.54200000000000004,"15min":0.96199999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.64100000000000001,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.379}]},{"osd":4,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.52800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":1.782,"15min":1.782},"last":0.622},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75900000000000001,"15min":0.88400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.36699999999999999},"max":{"1min":1.373,"5min":1.724,"15min":5.7300000000000004},"last":0.58299999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76300000000000001,"15min":0.88500000000000001},"min":{"1min":0.54000000000000004,"5min":0.49199999999999999,"15min":0.39700000000000002},"max":{"1min":0.96099999999999997,"5min":1.411,"15min":5.077},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.93400000000000005},"min":{"1min":0.56999999999999995,"5min":0.54500000000000004,"15min":0.44800000000000001},"max":{"1min":0.94699999999999995,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76800000000000002,"15min":0.78900000000000003},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.026,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.75900000000000001,"15min":0.89300000000000002},"min":{"1min":0.61199999999999999,"5min":0.503,"15min":0.45600000000000002},"max":{"1min":0.98199999999999998,"5min":1.4490000000000001,"15min":5.157},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76800000000000002,"15min":0.81399999999999995},"min":{"1min":0.58599999999999997,"5min":0.51700000000000002,"15min":0.47399999999999998},"max":{"1min":0.94599999999999995,"5min":1.863,"15min":1.863},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.76600000000000001,"15min":0.90400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.492,"15min":5.1189999999999998},"last":0.60399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36632,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335208,"statfs":{"total":96636764160,"available":96599252992,"internally_reserved":0,"allocated":1523712,"data_stored":698481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66200000000000003,"15min":0.64600000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.88800000000000001,"5min":0.90400000000000003,"15min":1.784},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.65000000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.38},"max":{"1min":0.90200000000000002,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.66200000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.80900000000000005},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.38},"max":{"1min":0.996,"5min":0.996,"15min":6.258},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.52400000000000002,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.95499999999999996,"15min":1.806},"last":0.753}]},{"osd":2,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69999999999999996,"15min":0.82599999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.29199999999999998},"max":{"1min":0.90400000000000003,"5min":0.91000000000000003,"15min":6.4740000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.90100000000000002},"min":{"1min":0.58199999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":6.4859999999999998},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.52500000000000002,"5min":0.442,"15min":0.40000000000000002},"max":{"1min":0.86199999999999999,"5min":0.97799999999999998,"15min":1.25},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.71299999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.99099999999999999,"15min":1.413},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.64700000000000002},"min":{"1min":0.50700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.92100000000000004,"15min":1.3280000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63700000000000001,"15min":0.65700000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94299999999999995,"15min":0.99299999999999999},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.41099999999999998,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75800000000000001,"15min":0.78300000000000003},"min":{"1min":0.56799999999999995,"5min":0.52400000000000002,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.008,"15min":1.6839999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.54100000000000004,"5min":0.44500000000000001,"15min":0.41299999999999998},"max":{"1min":0.89200000000000002,"5min":0.95699999999999996,"15min":1.0660000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.73799999999999999},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.036,"5min":1.036,"15min":1.21},"last":0.67100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":2092,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":2142208,"data_stored":1295666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.91100000000000003,"15min":4.5010000000000003},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.67400000000000004,"15min":0.78500000000000003},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.83399999999999996,"5min":0.96199999999999997,"15min":6.4059999999999997},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.502,"5min":0.44800000000000001,"15min":0.307},"max":{"1min":0.76200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.46100000000000002,"15min":0.41399999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.546},"last":0.70499999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79300000000000004,"15min":0.72799999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.376},"max":{"1min":0.86499999999999999,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.79100000000000004,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.47599999999999998,"15min":0.42199999999999999},"max":{"1min":0.78100000000000003,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71399999999999997,"15min":0.73899999999999999},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":0.85299999999999998,"5min":1.3140000000000001,"15min":4.484},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":1.0229999999999999,"15min":1.6240000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64000000000000001,"15min":0.68700000000000006},"min":{"1min":0.41899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":2.718},"last":0.60199999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83899999999999997,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.70999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.82599999999999996,"5min":0.94099999999999995,"15min":1.669},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71499999999999997,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.93799999999999994,"5min":1.002,"15min":2.0459999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73699999999999999,"15min":0.77000000000000002},"min":{"1min":0.47199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88,"5min":1.0249999999999999,"15min":2.2919999999999998},"last":0.63}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:55:19.745 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:55:19.772 INFO:tasks.ceph.mds.d.smithi146.stderr:2023-07-06T03:55:19.771+0000 7fcf4adf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:55:19.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.939+0000 7fea449ff700 1 Processor -- start 2023-07-06T03:55:19.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.939+0000 7fea449ff700 1 -- start start 2023-07-06T03:55:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.940+0000 7fea449ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea401592f0 0x7fea400b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.940+0000 7fea449ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4006d860 0x7fea400b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.940+0000 7fea449ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4015cc40 con 0x7fea40158f20 2023-07-06T03:55:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.940+0000 7fea449ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4015cf20 con 0x7fea401592f0 2023-07-06T03:55:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.940+0000 7fea449ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4015d200 con 0x7fea4006d860 2023-07-06T03:55:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.940+0000 7fea37fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea401592f0 0x7fea400b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.940+0000 7fea37fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea401592f0 0x7fea400b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57648/0 (socket says 172.21.15.97:57648) 2023-07-06T03:55:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.940+0000 7fea37fff700 1 -- 172.21.15.97:0/2114981811 learned_addr learned my addr 172.21.15.97:0/2114981811 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.941+0000 7fea37fff700 1 -- 172.21.15.97:0/2114981811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4006d860 msgr2=0x7fea400b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.941+0000 7fea37fff700 1 --2- 172.21.15.97:0/2114981811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4006d860 0x7fea400b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.941+0000 7fea37fff700 1 -- 172.21.15.97:0/2114981811 >> v1:172.21.15.146:6789/0 conn(0x7fea40158f20 legacy=0x7fea400caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.941+0000 7fea37fff700 1 -- 172.21.15.97:0/2114981811 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4015d4e0 con 0x7fea401592f0 2023-07-06T03:55:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.941+0000 7fea37fff700 1 --2- 172.21.15.97:0/2114981811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea401592f0 0x7fea400b0560 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fea280098f0 tx=0x7fea28009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e282cd2468f1a1f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.941+0000 7fea3dd9b700 1 -- 172.21.15.97:0/2114981811 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28004230 con 0x7fea401592f0 2023-07-06T03:55:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.941+0000 7fea3dd9b700 1 -- 172.21.15.97:0/2114981811 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea28004390 con 0x7fea401592f0 2023-07-06T03:55:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.941+0000 7fea3dd9b700 1 -- 172.21.15.97:0/2114981811 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea280036a0 con 0x7fea401592f0 2023-07-06T03:55:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.942+0000 7fea449ff700 1 -- 172.21.15.97:0/2114981811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea401592f0 msgr2=0x7fea400b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.942+0000 7fea449ff700 1 --2- 172.21.15.97:0/2114981811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea401592f0 0x7fea400b0560 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fea280098f0 tx=0x7fea28009c00 comp rx=0 tx=0).stop 2023-07-06T03:55:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.942+0000 7fea449ff700 1 -- 172.21.15.97:0/2114981811 shutdown_connections 2023-07-06T03:55:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.942+0000 7fea449ff700 1 --2- 172.21.15.97:0/2114981811 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4006d860 0x7fea400b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.942+0000 7fea449ff700 1 --2- 172.21.15.97:0/2114981811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea401592f0 0x7fea400b0560 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.942+0000 7fea449ff700 1 -- 172.21.15.97:0/2114981811 >> 172.21.15.97:0/2114981811 conn(0x7fea400ce8c0 msgr2=0x7fea400cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.942+0000 7fea449ff700 1 -- 172.21.15.97:0/2114981811 shutdown_connections 2023-07-06T03:55:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.942+0000 7fea449ff700 1 -- 172.21.15.97:0/2114981811 wait complete. 2023-07-06T03:55:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.944+0000 7fea449ff700 1 Processor -- start 2023-07-06T03:55:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.944+0000 7fea449ff700 1 -- start start 2023-07-06T03:55:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea449ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4006d860 0x7fea401510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea449ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea40158f20 0x7fea401596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea449ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea401592f0 0x7fea400afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea449ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea400af6c0 con 0x7fea401592f0 2023-07-06T03:55:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea449ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea400aea40 con 0x7fea40158f20 2023-07-06T03:55:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea449ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea400aed20 con 0x7fea4006d860 2023-07-06T03:55:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea3ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea401592f0 0x7fea400afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea3ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea401592f0 0x7fea400afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57656/0 (socket says 172.21.15.97:57656) 2023-07-06T03:55:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea3ed9d700 1 -- 172.21.15.97:0/914036318 learned_addr learned my addr 172.21.15.97:0/914036318 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea37fff700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea40158f20 0x7fea401596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea3e59c700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4006d860 0x7fea401510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea3ed9d700 1 -- 172.21.15.97:0/914036318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4006d860 msgr2=0x7fea401510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea3ed9d700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4006d860 0x7fea401510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea3ed9d700 1 -- 172.21.15.97:0/914036318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea40158f20 msgr2=0x7fea401596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:19.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea3ed9d700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea40158f20 0x7fea401596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:19.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea3ed9d700 1 -- 172.21.15.97:0/914036318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea28009550 con 0x7fea401592f0 2023-07-06T03:55:19.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea37fff700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea40158f20 0x7fea401596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:19.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.945+0000 7fea3e59c700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4006d860 0x7fea401510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.946+0000 7fea3ed9d700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea401592f0 0x7fea400afae0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fea3800b970 tx=0x7fea3800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.946+0000 7fea36ffd700 1 -- 172.21.15.97:0/914036318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3800c9b0 con 0x7fea401592f0 2023-07-06T03:55:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.946+0000 7fea36ffd700 1 -- 172.21.15.97:0/914036318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3800cb30 con 0x7fea401592f0 2023-07-06T03:55:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.946+0000 7fea449ff700 1 -- 172.21.15.97:0/914036318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea400af000 con 0x7fea401592f0 2023-07-06T03:55:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.946+0000 7fea36ffd700 1 -- 172.21.15.97:0/914036318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38014440 con 0x7fea401592f0 2023-07-06T03:55:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.946+0000 7fea449ff700 1 -- 172.21.15.97:0/914036318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea40151880 con 0x7fea401592f0 2023-07-06T03:55:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.947+0000 7fea449ff700 1 -- 172.21.15.97:0/914036318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4004e340 con 0x7fea401592f0 2023-07-06T03:55:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.948+0000 7fea36ffd700 1 -- 172.21.15.97:0/914036318 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea3801d3b0 con 0x7fea401592f0 2023-07-06T03:55:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.949+0000 7fea36ffd700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2006ace0 0x7fea2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.949+0000 7fea36ffd700 1 -- 172.21.15.97:0/914036318 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea38006e40 con 0x7fea401592f0 2023-07-06T03:55:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.949+0000 7fea3e59c700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2006ace0 0x7fea2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.950+0000 7fea3e59c700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2006ace0 0x7fea2006d190 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fea30003f40 tx=0x7fea30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:19.953+0000 7fea36ffd700 1 -- 172.21.15.97:0/914036318 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fea380542e0 con 0x7fea401592f0 2023-07-06T03:55:20.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.142+0000 7fea449ff700 1 -- 172.21.15.97:0/914036318 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea4015d7c0 con 0x7fea2006ace0 2023-07-06T03:55:20.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.151+0000 7fea36ffd700 1 -- 172.21.15.97:0/914036318 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311787 (secure 0 0 0) 0x7fea30014420 con 0x7fea2006ace0 2023-07-06T03:55:20.156 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:20.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 -- 172.21.15.97:0/914036318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2006ace0 msgr2=0x7fea2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2006ace0 0x7fea2006d190 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fea30003f40 tx=0x7fea30009040 comp rx=0 tx=0).stop 2023-07-06T03:55:20.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 -- 172.21.15.97:0/914036318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea401592f0 msgr2=0x7fea400afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea401592f0 0x7fea400afae0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fea3800b970 tx=0x7fea3800bd30 comp rx=0 tx=0).stop 2023-07-06T03:55:20.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 -- 172.21.15.97:0/914036318 shutdown_connections 2023-07-06T03:55:20.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fea2006ace0 0x7fea2006d190 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea401592f0 0x7fea400afae0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea4006d860 0x7fea401510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 --2- 172.21.15.97:0/914036318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea40158f20 0x7fea401596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 -- 172.21.15.97:0/914036318 >> 172.21.15.97:0/914036318 conn(0x7fea400ce8c0 msgr2=0x7fea4006cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 -- 172.21.15.97:0/914036318 shutdown_connections 2023-07-06T03:55:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.155+0000 7fea449ff700 1 -- 172.21.15.97:0/914036318 wait complete. 2023-07-06T03:55:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:55:20.175 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2023-07-06T03:55:19.014579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290228,"kb_used_data":13172,"kb_used_omap":0,"kb_used_meta":276992,"kb_avail":754684492,"statfs":{"total":773094113280,"available":772796919808,"internally_reserved":0,"allocated":13488128,"data_stored":6807255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":283639808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.710362"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.352560+0000","last_change":"2023-07-06T03:55:15.343577+0000","last_active":"2023-07-06T03:55:15.352560+0000","last_peered":"2023-07-06T03:55:15.352560+0000","last_clean":"2023-07-06T03:55:15.352560+0000","last_became_active":"2023-07-06T03:55:15.343280+0000","last_became_peered":"2023-07-06T03:55:15.343280+0000","last_unstale":"2023-07-06T03:55:15.352560+0000","last_undegraded":"2023-07-06T03:55:15.352560+0000","last_fullsized":"2023-07-06T03:55:15.352560+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:03:00.154330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340336+0000","last_change":"2023-07-06T03:55:15.335701+0000","last_active":"2023-07-06T03:55:15.340336+0000","last_peered":"2023-07-06T03:55:15.340336+0000","last_clean":"2023-07-06T03:55:15.340336+0000","last_became_active":"2023-07-06T03:55:15.335388+0000","last_became_peered":"2023-07-06T03:55:15.335388+0000","last_unstale":"2023-07-06T03:55:15.340336+0000","last_undegraded":"2023-07-06T03:55:15.340336+0000","last_fullsized":"2023-07-06T03:55:15.340336+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:11:40.364030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349844+0000","last_change":"2023-07-06T03:55:15.340385+0000","last_active":"2023-07-06T03:55:15.349844+0000","last_peered":"2023-07-06T03:55:15.349844+0000","last_clean":"2023-07-06T03:55:15.349844+0000","last_became_active":"2023-07-06T03:55:15.339887+0000","last_became_peered":"2023-07-06T03:55:15.339887+0000","last_unstale":"2023-07-06T03:55:15.349844+0000","last_undegraded":"2023-07-06T03:55:15.349844+0000","last_fullsized":"2023-07-06T03:55:15.349844+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:06:54.459675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.344424+0000","last_change":"2023-07-06T03:55:15.339620+0000","last_active":"2023-07-06T03:55:15.344424+0000","last_peered":"2023-07-06T03:55:15.344424+0000","last_clean":"2023-07-06T03:55:15.344424+0000","last_became_active":"2023-07-06T03:55:15.339330+0000","last_became_peered":"2023-07-06T03:55:15.339330+0000","last_unstale":"2023-07-06T03:55:15.344424+0000","last_undegraded":"2023-07-06T03:55:15.344424+0000","last_fullsized":"2023-07-06T03:55:15.344424+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:38:32.416488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.348604+0000","last_change":"2023-07-06T03:55:15.345165+0000","last_active":"2023-07-06T03:55:15.348604+0000","last_peered":"2023-07-06T03:55:15.348604+0000","last_clean":"2023-07-06T03:55:15.348604+0000","last_became_active":"2023-07-06T03:55:15.344902+0000","last_became_peered":"2023-07-06T03:55:15.344902+0000","last_unstale":"2023-07-06T03:55:15.348604+0000","last_undegraded":"2023-07-06T03:55:15.348604+0000","last_fullsized":"2023-07-06T03:55:15.348604+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:28:25.690369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349077+0000","last_change":"2023-07-06T03:55:15.344276+0000","last_active":"2023-07-06T03:55:15.349077+0000","last_peered":"2023-07-06T03:55:15.349077+0000","last_clean":"2023-07-06T03:55:15.349077+0000","last_became_active":"2023-07-06T03:55:15.343896+0000","last_became_peered":"2023-07-06T03:55:15.343896+0000","last_unstale":"2023-07-06T03:55:15.349077+0000","last_undegraded":"2023-07-06T03:55:15.349077+0000","last_fullsized":"2023-07-06T03:55:15.349077+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:27:47.599953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.341267+0000","last_change":"2023-07-06T03:55:15.338317+0000","last_active":"2023-07-06T03:55:15.341267+0000","last_peered":"2023-07-06T03:55:15.341267+0000","last_clean":"2023-07-06T03:55:15.341267+0000","last_became_active":"2023-07-06T03:55:15.338062+0000","last_became_peered":"2023-07-06T03:55:15.338062+0000","last_unstale":"2023-07-06T03:55:15.341267+0000","last_undegraded":"2023-07-06T03:55:15.341267+0000","last_fullsized":"2023-07-06T03:55:15.341267+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:43:07.567807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.350665+0000","last_change":"2023-07-06T03:55:15.345178+0000","last_active":"2023-07-06T03:55:15.350665+0000","last_peered":"2023-07-06T03:55:15.350665+0000","last_clean":"2023-07-06T03:55:15.350665+0000","last_became_active":"2023-07-06T03:55:15.344887+0000","last_became_peered":"2023-07-06T03:55:15.344887+0000","last_unstale":"2023-07-06T03:55:15.350665+0000","last_undegraded":"2023-07-06T03:55:15.350665+0000","last_fullsized":"2023-07-06T03:55:15.350665+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:17:09.949975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353683+0000","last_change":"2023-07-06T03:55:15.348757+0000","last_active":"2023-07-06T03:55:15.353683+0000","last_peered":"2023-07-06T03:55:15.353683+0000","last_clean":"2023-07-06T03:55:15.353683+0000","last_became_active":"2023-07-06T03:55:15.348374+0000","last_became_peered":"2023-07-06T03:55:15.348374+0000","last_unstale":"2023-07-06T03:55:15.353683+0000","last_undegraded":"2023-07-06T03:55:15.353683+0000","last_fullsized":"2023-07-06T03:55:15.353683+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:24:53.959530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353386+0000","last_change":"2023-07-06T03:55:15.346780+0000","last_active":"2023-07-06T03:55:15.353386+0000","last_peered":"2023-07-06T03:55:15.353386+0000","last_clean":"2023-07-06T03:55:15.353386+0000","last_became_active":"2023-07-06T03:55:15.346507+0000","last_became_peered":"2023-07-06T03:55:15.346507+0000","last_unstale":"2023-07-06T03:55:15.353386+0000","last_undegraded":"2023-07-06T03:55:15.353386+0000","last_fullsized":"2023-07-06T03:55:15.353386+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:20:21.647465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.341628+0000","last_change":"2023-07-06T03:55:15.334194+0000","last_active":"2023-07-06T03:55:15.341628+0000","last_peered":"2023-07-06T03:55:15.341628+0000","last_clean":"2023-07-06T03:55:15.341628+0000","last_became_active":"2023-07-06T03:55:15.333933+0000","last_became_peered":"2023-07-06T03:55:15.333933+0000","last_unstale":"2023-07-06T03:55:15.341628+0000","last_undegraded":"2023-07-06T03:55:15.341628+0000","last_fullsized":"2023-07-06T03:55:15.341628+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:58:42.142451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.338401+0000","last_change":"2023-07-06T03:55:15.334254+0000","last_active":"2023-07-06T03:55:15.338401+0000","last_peered":"2023-07-06T03:55:15.338401+0000","last_clean":"2023-07-06T03:55:15.338401+0000","last_became_active":"2023-07-06T03:55:15.333945+0000","last_became_peered":"2023-07-06T03:55:15.333945+0000","last_unstale":"2023-07-06T03:55:15.338401+0000","last_undegraded":"2023-07-06T03:55:15.338401+0000","last_fullsized":"2023-07-06T03:55:15.338401+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:17:55.000000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.344695+0000","last_change":"2023-07-06T03:55:15.340327+0000","last_active":"2023-07-06T03:55:15.344695+0000","last_peered":"2023-07-06T03:55:15.344695+0000","last_clean":"2023-07-06T03:55:15.344695+0000","last_became_active":"2023-07-06T03:55:15.340004+0000","last_became_peered":"2023-07-06T03:55:15.340004+0000","last_unstale":"2023-07-06T03:55:15.344695+0000","last_undegraded":"2023-07-06T03:55:15.344695+0000","last_fullsized":"2023-07-06T03:55:15.344695+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:04:15.446407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.351933+0000","last_change":"2023-07-06T03:55:15.342731+0000","last_active":"2023-07-06T03:55:15.351933+0000","last_peered":"2023-07-06T03:55:15.351933+0000","last_clean":"2023-07-06T03:55:15.351933+0000","last_became_active":"2023-07-06T03:55:15.342455+0000","last_became_peered":"2023-07-06T03:55:15.342455+0000","last_unstale":"2023-07-06T03:55:15.351933+0000","last_undegraded":"2023-07-06T03:55:15.351933+0000","last_fullsized":"2023-07-06T03:55:15.351933+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:17.748239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.350389+0000","last_change":"2023-07-06T03:55:15.341040+0000","last_active":"2023-07-06T03:55:15.350389+0000","last_peered":"2023-07-06T03:55:15.350389+0000","last_clean":"2023-07-06T03:55:15.350389+0000","last_became_active":"2023-07-06T03:55:15.340748+0000","last_became_peered":"2023-07-06T03:55:15.340748+0000","last_unstale":"2023-07-06T03:55:15.350389+0000","last_undegraded":"2023-07-06T03:55:15.350389+0000","last_fullsized":"2023-07-06T03:55:15.350389+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:06:43.147768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349651+0000","last_change":"2023-07-06T03:55:15.338316+0000","last_active":"2023-07-06T03:55:15.349651+0000","last_peered":"2023-07-06T03:55:15.349651+0000","last_clean":"2023-07-06T03:55:15.349651+0000","last_became_active":"2023-07-06T03:55:15.338031+0000","last_became_peered":"2023-07-06T03:55:15.338031+0000","last_unstale":"2023-07-06T03:55:15.349651+0000","last_undegraded":"2023-07-06T03:55:15.349651+0000","last_fullsized":"2023-07-06T03:55:15.349651+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:40:57.335083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.352583+0000","last_change":"2023-07-06T03:55:15.343984+0000","last_active":"2023-07-06T03:55:15.352583+0000","last_peered":"2023-07-06T03:55:15.352583+0000","last_clean":"2023-07-06T03:55:15.352583+0000","last_became_active":"2023-07-06T03:55:15.343446+0000","last_became_peered":"2023-07-06T03:55:15.343446+0000","last_unstale":"2023-07-06T03:55:15.352583+0000","last_undegraded":"2023-07-06T03:55:15.352583+0000","last_fullsized":"2023-07-06T03:55:15.352583+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:03.763283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.338808+0000","last_change":"2023-07-06T03:55:15.336341+0000","last_active":"2023-07-06T03:55:15.338808+0000","last_peered":"2023-07-06T03:55:15.338808+0000","last_clean":"2023-07-06T03:55:15.338808+0000","last_became_active":"2023-07-06T03:55:15.336033+0000","last_became_peered":"2023-07-06T03:55:15.336033+0000","last_unstale":"2023-07-06T03:55:15.338808+0000","last_undegraded":"2023-07-06T03:55:15.338808+0000","last_fullsized":"2023-07-06T03:55:15.338808+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:12:18.056091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.345371+0000","last_change":"2023-07-06T03:55:15.342068+0000","last_active":"2023-07-06T03:55:15.345371+0000","last_peered":"2023-07-06T03:55:15.345371+0000","last_clean":"2023-07-06T03:55:15.345371+0000","last_became_active":"2023-07-06T03:55:15.341810+0000","last_became_peered":"2023-07-06T03:55:15.341810+0000","last_unstale":"2023-07-06T03:55:15.345371+0000","last_undegraded":"2023-07-06T03:55:15.345371+0000","last_fullsized":"2023-07-06T03:55:15.345371+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:18:59.607705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340295+0000","last_change":"2023-07-06T03:55:15.338238+0000","last_active":"2023-07-06T03:55:15.340295+0000","last_peered":"2023-07-06T03:55:15.340295+0000","last_clean":"2023-07-06T03:55:15.340295+0000","last_became_active":"2023-07-06T03:55:15.337935+0000","last_became_peered":"2023-07-06T03:55:15.337935+0000","last_unstale":"2023-07-06T03:55:15.340295+0000","last_undegraded":"2023-07-06T03:55:15.340295+0000","last_fullsized":"2023-07-06T03:55:15.340295+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:01:12.221859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349711+0000","last_change":"2023-07-06T03:55:15.340350+0000","last_active":"2023-07-06T03:55:15.349711+0000","last_peered":"2023-07-06T03:55:15.349711+0000","last_clean":"2023-07-06T03:55:15.349711+0000","last_became_active":"2023-07-06T03:55:15.340078+0000","last_became_peered":"2023-07-06T03:55:15.340078+0000","last_unstale":"2023-07-06T03:55:15.349711+0000","last_undegraded":"2023-07-06T03:55:15.349711+0000","last_fullsized":"2023-07-06T03:55:15.349711+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:24:35.800135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.344500+0000","last_change":"2023-07-06T03:55:15.340286+0000","last_active":"2023-07-06T03:55:15.344500+0000","last_peered":"2023-07-06T03:55:15.344500+0000","last_clean":"2023-07-06T03:55:15.344500+0000","last_became_active":"2023-07-06T03:55:15.339990+0000","last_became_peered":"2023-07-06T03:55:15.339990+0000","last_unstale":"2023-07-06T03:55:15.344500+0000","last_undegraded":"2023-07-06T03:55:15.344500+0000","last_fullsized":"2023-07-06T03:55:15.344500+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:16:29.789589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343421+0000","last_change":"2023-07-06T03:55:15.341897+0000","last_active":"2023-07-06T03:55:15.343421+0000","last_peered":"2023-07-06T03:55:15.343421+0000","last_clean":"2023-07-06T03:55:15.343421+0000","last_became_active":"2023-07-06T03:55:15.341618+0000","last_became_peered":"2023-07-06T03:55:15.341618+0000","last_unstale":"2023-07-06T03:55:15.343421+0000","last_undegraded":"2023-07-06T03:55:15.343421+0000","last_fullsized":"2023-07-06T03:55:15.343421+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:15:41.115884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354857+0000","last_change":"2023-07-06T03:55:15.349071+0000","last_active":"2023-07-06T03:55:15.354857+0000","last_peered":"2023-07-06T03:55:15.354857+0000","last_clean":"2023-07-06T03:55:15.354857+0000","last_became_active":"2023-07-06T03:55:15.348330+0000","last_became_peered":"2023-07-06T03:55:15.348330+0000","last_unstale":"2023-07-06T03:55:15.354857+0000","last_undegraded":"2023-07-06T03:55:15.354857+0000","last_fullsized":"2023-07-06T03:55:15.354857+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:57:38.531759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340636+0000","last_change":"2023-07-06T03:55:15.331929+0000","last_active":"2023-07-06T03:55:15.340636+0000","last_peered":"2023-07-06T03:55:15.340636+0000","last_clean":"2023-07-06T03:55:15.340636+0000","last_became_active":"2023-07-06T03:55:15.331641+0000","last_became_peered":"2023-07-06T03:55:15.331641+0000","last_unstale":"2023-07-06T03:55:15.340636+0000","last_undegraded":"2023-07-06T03:55:15.340636+0000","last_fullsized":"2023-07-06T03:55:15.340636+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:18:11.226972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340746+0000","last_change":"2023-07-06T03:55:15.334980+0000","last_active":"2023-07-06T03:55:15.340746+0000","last_peered":"2023-07-06T03:55:15.340746+0000","last_clean":"2023-07-06T03:55:15.340746+0000","last_became_active":"2023-07-06T03:55:15.334699+0000","last_became_peered":"2023-07-06T03:55:15.334699+0000","last_unstale":"2023-07-06T03:55:15.340746+0000","last_undegraded":"2023-07-06T03:55:15.340746+0000","last_fullsized":"2023-07-06T03:55:15.340746+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:42:25.047155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354884+0000","last_change":"2023-07-06T03:55:15.346424+0000","last_active":"2023-07-06T03:55:15.354884+0000","last_peered":"2023-07-06T03:55:15.354884+0000","last_clean":"2023-07-06T03:55:15.354884+0000","last_became_active":"2023-07-06T03:55:15.345464+0000","last_became_peered":"2023-07-06T03:55:15.345464+0000","last_unstale":"2023-07-06T03:55:15.354884+0000","last_undegraded":"2023-07-06T03:55:15.354884+0000","last_fullsized":"2023-07-06T03:55:15.354884+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:32:56.241778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.336430+0000","last_change":"2023-07-06T03:55:15.335155+0000","last_active":"2023-07-06T03:55:15.336430+0000","last_peered":"2023-07-06T03:55:15.336430+0000","last_clean":"2023-07-06T03:55:15.336430+0000","last_became_active":"2023-07-06T03:55:15.334898+0000","last_became_peered":"2023-07-06T03:55:15.334898+0000","last_unstale":"2023-07-06T03:55:15.336430+0000","last_undegraded":"2023-07-06T03:55:15.336430+0000","last_fullsized":"2023-07-06T03:55:15.336430+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:03.763283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353757+0000","last_change":"2023-07-06T03:55:15.350709+0000","last_active":"2023-07-06T03:55:15.353757+0000","last_peered":"2023-07-06T03:55:15.353757+0000","last_clean":"2023-07-06T03:55:15.353757+0000","last_became_active":"2023-07-06T03:55:15.350426+0000","last_became_peered":"2023-07-06T03:55:15.350426+0000","last_unstale":"2023-07-06T03:55:15.353757+0000","last_undegraded":"2023-07-06T03:55:15.353757+0000","last_fullsized":"2023-07-06T03:55:15.353757+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:10:57.317137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343401+0000","last_change":"2023-07-06T03:55:15.341531+0000","last_active":"2023-07-06T03:55:15.343401+0000","last_peered":"2023-07-06T03:55:15.343401+0000","last_clean":"2023-07-06T03:55:15.343401+0000","last_became_active":"2023-07-06T03:55:15.341233+0000","last_became_peered":"2023-07-06T03:55:15.341233+0000","last_unstale":"2023-07-06T03:55:15.343401+0000","last_undegraded":"2023-07-06T03:55:15.343401+0000","last_fullsized":"2023-07-06T03:55:15.343401+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:17:02.188812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354916+0000","last_change":"2023-07-06T03:55:15.352539+0000","last_active":"2023-07-06T03:55:15.354916+0000","last_peered":"2023-07-06T03:55:15.354916+0000","last_clean":"2023-07-06T03:55:15.354916+0000","last_became_active":"2023-07-06T03:55:15.352278+0000","last_became_peered":"2023-07-06T03:55:15.352278+0000","last_unstale":"2023-07-06T03:55:15.354916+0000","last_undegraded":"2023-07-06T03:55:15.354916+0000","last_fullsized":"2023-07-06T03:55:15.354916+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:11:52.405524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353772+0000","last_change":"2023-07-06T03:55:15.349545+0000","last_active":"2023-07-06T03:55:15.353772+0000","last_peered":"2023-07-06T03:55:15.353772+0000","last_clean":"2023-07-06T03:55:15.353772+0000","last_became_active":"2023-07-06T03:55:15.349240+0000","last_became_peered":"2023-07-06T03:55:15.349240+0000","last_unstale":"2023-07-06T03:55:15.353772+0000","last_undegraded":"2023-07-06T03:55:15.353772+0000","last_fullsized":"2023-07-06T03:55:15.353772+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:12:37.373095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340669+0000","last_change":"2023-07-06T03:55:15.332572+0000","last_active":"2023-07-06T03:55:15.340669+0000","last_peered":"2023-07-06T03:55:15.340669+0000","last_clean":"2023-07-06T03:55:15.340669+0000","last_became_active":"2023-07-06T03:55:15.332255+0000","last_became_peered":"2023-07-06T03:55:15.332255+0000","last_unstale":"2023-07-06T03:55:15.340669+0000","last_undegraded":"2023-07-06T03:55:15.340669+0000","last_fullsized":"2023-07-06T03:55:15.340669+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:47:18.407557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1f","version":"123'4","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.355169+0000","last_change":"2023-07-06T03:55:14.334903+0000","last_active":"2023-07-06T03:55:15.355169+0000","last_peered":"2023-07-06T03:55:15.355169+0000","last_clean":"2023-07-06T03:55:15.355169+0000","last_became_active":"2023-07-06T03:55:14.334552+0000","last_became_peered":"2023-07-06T03:55:14.334552+0000","last_unstale":"2023-07-06T03:55:15.355169+0000","last_undegraded":"2023-07-06T03:55:15.355169+0000","last_fullsized":"2023-07-06T03:55:15.355169+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:30:08.064899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353405+0000","last_change":"2023-07-06T03:55:15.347549+0000","last_active":"2023-07-06T03:55:15.353405+0000","last_peered":"2023-07-06T03:55:15.353405+0000","last_clean":"2023-07-06T03:55:15.353405+0000","last_became_active":"2023-07-06T03:55:15.347285+0000","last_became_peered":"2023-07-06T03:55:15.347285+0000","last_unstale":"2023-07-06T03:55:15.353405+0000","last_undegraded":"2023-07-06T03:55:15.353405+0000","last_fullsized":"2023-07-06T03:55:15.353405+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:01:12.002592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.1e","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.357175+0000","last_change":"2023-07-06T03:55:14.342032+0000","last_active":"2023-07-06T03:55:15.357175+0000","last_peered":"2023-07-06T03:55:15.357175+0000","last_clean":"2023-07-06T03:55:15.357175+0000","last_became_active":"2023-07-06T03:55:14.341762+0000","last_became_peered":"2023-07-06T03:55:14.341762+0000","last_unstale":"2023-07-06T03:55:15.357175+0000","last_undegraded":"2023-07-06T03:55:15.357175+0000","last_fullsized":"2023-07-06T03:55:15.357175+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:48:51.017885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343388+0000","last_change":"2023-07-06T03:55:15.341443+0000","last_active":"2023-07-06T03:55:15.343388+0000","last_peered":"2023-07-06T03:55:15.343388+0000","last_clean":"2023-07-06T03:55:15.343388+0000","last_became_active":"2023-07-06T03:55:15.341164+0000","last_became_peered":"2023-07-06T03:55:15.341164+0000","last_unstale":"2023-07-06T03:55:15.343388+0000","last_undegraded":"2023-07-06T03:55:15.343388+0000","last_fullsized":"2023-07-06T03:55:15.343388+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:27:12.742803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1d","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.355100+0000","last_change":"2023-07-06T03:55:14.332729+0000","last_active":"2023-07-06T03:55:15.355100+0000","last_peered":"2023-07-06T03:55:15.355100+0000","last_clean":"2023-07-06T03:55:15.355100+0000","last_became_active":"2023-07-06T03:55:14.332440+0000","last_became_peered":"2023-07-06T03:55:14.332440+0000","last_unstale":"2023-07-06T03:55:15.355100+0000","last_undegraded":"2023-07-06T03:55:15.355100+0000","last_fullsized":"2023-07-06T03:55:15.355100+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:54:46.655901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.337782+0000","last_change":"2023-07-06T03:55:15.335040+0000","last_active":"2023-07-06T03:55:15.337782+0000","last_peered":"2023-07-06T03:55:15.337782+0000","last_clean":"2023-07-06T03:55:15.337782+0000","last_became_active":"2023-07-06T03:55:15.334773+0000","last_became_peered":"2023-07-06T03:55:15.334773+0000","last_unstale":"2023-07-06T03:55:15.337782+0000","last_undegraded":"2023-07-06T03:55:15.337782+0000","last_fullsized":"2023-07-06T03:55:15.337782+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:11:10.242306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1c","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.356116+0000","last_change":"2023-07-06T03:55:14.337620+0000","last_active":"2023-07-06T03:55:15.356116+0000","last_peered":"2023-07-06T03:55:15.356116+0000","last_clean":"2023-07-06T03:55:15.356116+0000","last_became_active":"2023-07-06T03:55:14.337323+0000","last_became_peered":"2023-07-06T03:55:14.337323+0000","last_unstale":"2023-07-06T03:55:15.356116+0000","last_undegraded":"2023-07-06T03:55:15.356116+0000","last_fullsized":"2023-07-06T03:55:15.356116+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:04:02.874486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.351385+0000","last_change":"2023-07-06T03:55:15.347942+0000","last_active":"2023-07-06T03:55:15.351385+0000","last_peered":"2023-07-06T03:55:15.351385+0000","last_clean":"2023-07-06T03:55:15.351385+0000","last_became_active":"2023-07-06T03:55:15.347192+0000","last_became_peered":"2023-07-06T03:55:15.347192+0000","last_unstale":"2023-07-06T03:55:15.351385+0000","last_undegraded":"2023-07-06T03:55:15.351385+0000","last_fullsized":"2023-07-06T03:55:15.351385+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:43:31.772144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1b","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353895+0000","last_change":"2023-07-06T03:55:14.339675+0000","last_active":"2023-07-06T03:55:15.353895+0000","last_peered":"2023-07-06T03:55:15.353895+0000","last_clean":"2023-07-06T03:55:15.353895+0000","last_became_active":"2023-07-06T03:55:14.339335+0000","last_became_peered":"2023-07-06T03:55:14.339335+0000","last_unstale":"2023-07-06T03:55:15.353895+0000","last_undegraded":"2023-07-06T03:55:15.353895+0000","last_fullsized":"2023-07-06T03:55:15.353895+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:52:40.343430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.351690+0000","last_change":"2023-07-06T03:55:15.349764+0000","last_active":"2023-07-06T03:55:15.351690+0000","last_peered":"2023-07-06T03:55:15.351690+0000","last_clean":"2023-07-06T03:55:15.351690+0000","last_became_active":"2023-07-06T03:55:15.349451+0000","last_became_peered":"2023-07-06T03:55:15.349451+0000","last_unstale":"2023-07-06T03:55:15.351690+0000","last_undegraded":"2023-07-06T03:55:15.351690+0000","last_fullsized":"2023-07-06T03:55:15.351690+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:58:08.809003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1a","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.356321+0000","last_change":"2023-07-06T03:55:14.324495+0000","last_active":"2023-07-06T03:55:15.356321+0000","last_peered":"2023-07-06T03:55:15.356321+0000","last_clean":"2023-07-06T03:55:15.356321+0000","last_became_active":"2023-07-06T03:55:14.324172+0000","last_became_peered":"2023-07-06T03:55:14.324172+0000","last_unstale":"2023-07-06T03:55:15.356321+0000","last_undegraded":"2023-07-06T03:55:15.356321+0000","last_fullsized":"2023-07-06T03:55:15.356321+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:58:19.753702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.352821+0000","last_change":"2023-07-06T03:55:15.345661+0000","last_active":"2023-07-06T03:55:15.352821+0000","last_peered":"2023-07-06T03:55:15.352821+0000","last_clean":"2023-07-06T03:55:15.352821+0000","last_became_active":"2023-07-06T03:55:15.345377+0000","last_became_peered":"2023-07-06T03:55:15.345377+0000","last_unstale":"2023-07-06T03:55:15.352821+0000","last_undegraded":"2023-07-06T03:55:15.352821+0000","last_fullsized":"2023-07-06T03:55:15.352821+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:35:01.979264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.328899+0000","last_change":"2023-07-06T03:55:14.335681+0000","last_active":"2023-07-06T03:55:15.328899+0000","last_peered":"2023-07-06T03:55:15.328899+0000","last_clean":"2023-07-06T03:55:15.328899+0000","last_became_active":"2023-07-06T03:55:14.335042+0000","last_became_peered":"2023-07-06T03:55:14.335042+0000","last_unstale":"2023-07-06T03:55:15.328899+0000","last_undegraded":"2023-07-06T03:55:15.328899+0000","last_fullsized":"2023-07-06T03:55:15.328899+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:58:18.787436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.341889+0000","last_change":"2023-07-06T03:55:15.336689+0000","last_active":"2023-07-06T03:55:15.341889+0000","last_peered":"2023-07-06T03:55:15.341889+0000","last_clean":"2023-07-06T03:55:15.341889+0000","last_became_active":"2023-07-06T03:55:15.336383+0000","last_became_peered":"2023-07-06T03:55:15.336383+0000","last_unstale":"2023-07-06T03:55:15.341889+0000","last_undegraded":"2023-07-06T03:55:15.341889+0000","last_fullsized":"2023-07-06T03:55:15.341889+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:50:17.782805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.332124+0000","last_change":"2023-07-06T03:55:14.330764+0000","last_active":"2023-07-06T03:55:15.332124+0000","last_peered":"2023-07-06T03:55:15.332124+0000","last_clean":"2023-07-06T03:55:15.332124+0000","last_became_active":"2023-07-06T03:55:14.330521+0000","last_became_peered":"2023-07-06T03:55:14.330521+0000","last_unstale":"2023-07-06T03:55:15.332124+0000","last_undegraded":"2023-07-06T03:55:15.332124+0000","last_fullsized":"2023-07-06T03:55:15.332124+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:13:03.702548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.355783+0000","last_change":"2023-07-06T03:55:15.352105+0000","last_active":"2023-07-06T03:55:15.355783+0000","last_peered":"2023-07-06T03:55:15.355783+0000","last_clean":"2023-07-06T03:55:15.355783+0000","last_became_active":"2023-07-06T03:55:15.351760+0000","last_became_peered":"2023-07-06T03:55:15.351760+0000","last_unstale":"2023-07-06T03:55:15.355783+0000","last_undegraded":"2023-07-06T03:55:15.355783+0000","last_fullsized":"2023-07-06T03:55:15.355783+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:30:29.772585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.f","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.355682+0000","last_change":"2023-07-06T03:55:14.334289+0000","last_active":"2023-07-06T03:55:15.355682+0000","last_peered":"2023-07-06T03:55:15.355682+0000","last_clean":"2023-07-06T03:55:15.355682+0000","last_became_active":"2023-07-06T03:55:14.333890+0000","last_became_peered":"2023-07-06T03:55:14.333890+0000","last_unstale":"2023-07-06T03:55:15.355682+0000","last_undegraded":"2023-07-06T03:55:15.355682+0000","last_fullsized":"2023-07-06T03:55:15.355682+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:15.050981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343857+0000","last_change":"2023-07-06T03:55:15.338482+0000","last_active":"2023-07-06T03:55:15.343857+0000","last_peered":"2023-07-06T03:55:15.343857+0000","last_clean":"2023-07-06T03:55:15.343857+0000","last_became_active":"2023-07-06T03:55:15.338227+0000","last_became_peered":"2023-07-06T03:55:15.338227+0000","last_unstale":"2023-07-06T03:55:15.343857+0000","last_undegraded":"2023-07-06T03:55:15.343857+0000","last_fullsized":"2023-07-06T03:55:15.343857+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:12:49.936924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.e","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354619+0000","last_change":"2023-07-06T03:55:14.338984+0000","last_active":"2023-07-06T03:55:15.354619+0000","last_peered":"2023-07-06T03:55:15.354619+0000","last_clean":"2023-07-06T03:55:15.354619+0000","last_became_active":"2023-07-06T03:55:14.338704+0000","last_became_peered":"2023-07-06T03:55:14.338704+0000","last_unstale":"2023-07-06T03:55:15.354619+0000","last_undegraded":"2023-07-06T03:55:15.354619+0000","last_fullsized":"2023-07-06T03:55:15.354619+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:04:12.651513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343483+0000","last_change":"2023-07-06T03:55:15.335580+0000","last_active":"2023-07-06T03:55:15.343483+0000","last_peered":"2023-07-06T03:55:15.343483+0000","last_clean":"2023-07-06T03:55:15.343483+0000","last_became_active":"2023-07-06T03:55:15.334347+0000","last_became_peered":"2023-07-06T03:55:15.334347+0000","last_unstale":"2023-07-06T03:55:15.343483+0000","last_undegraded":"2023-07-06T03:55:15.343483+0000","last_fullsized":"2023-07-06T03:55:15.343483+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:03:59.852629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.d","version":"123'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354199+0000","last_change":"2023-07-06T03:55:14.335419+0000","last_active":"2023-07-06T03:55:15.354199+0000","last_peered":"2023-07-06T03:55:15.354199+0000","last_clean":"2023-07-06T03:55:15.354199+0000","last_became_active":"2023-07-06T03:55:14.335135+0000","last_became_peered":"2023-07-06T03:55:14.335135+0000","last_unstale":"2023-07-06T03:55:15.354199+0000","last_undegraded":"2023-07-06T03:55:15.354199+0000","last_fullsized":"2023-07-06T03:55:15.354199+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:38:58.335529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.348578+0000","last_change":"2023-07-06T03:55:15.344256+0000","last_active":"2023-07-06T03:55:15.348578+0000","last_peered":"2023-07-06T03:55:15.348578+0000","last_clean":"2023-07-06T03:55:15.348578+0000","last_became_active":"2023-07-06T03:55:15.343887+0000","last_became_peered":"2023-07-06T03:55:15.343887+0000","last_unstale":"2023-07-06T03:55:15.348578+0000","last_undegraded":"2023-07-06T03:55:15.348578+0000","last_fullsized":"2023-07-06T03:55:15.348578+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:10.694869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.328190+0000","last_change":"2023-07-06T03:55:14.334539+0000","last_active":"2023-07-06T03:55:15.328190+0000","last_peered":"2023-07-06T03:55:15.328190+0000","last_clean":"2023-07-06T03:55:15.328190+0000","last_became_active":"2023-07-06T03:55:14.334232+0000","last_became_peered":"2023-07-06T03:55:14.334232+0000","last_unstale":"2023-07-06T03:55:15.328190+0000","last_undegraded":"2023-07-06T03:55:15.328190+0000","last_fullsized":"2023-07-06T03:55:15.328190+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:49:19.509940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.351784+0000","last_change":"2023-07-06T03:55:15.348686+0000","last_active":"2023-07-06T03:55:15.351784+0000","last_peered":"2023-07-06T03:55:15.351784+0000","last_clean":"2023-07-06T03:55:15.351784+0000","last_became_active":"2023-07-06T03:55:15.348414+0000","last_became_peered":"2023-07-06T03:55:15.348414+0000","last_unstale":"2023-07-06T03:55:15.351784+0000","last_undegraded":"2023-07-06T03:55:15.351784+0000","last_fullsized":"2023-07-06T03:55:15.351784+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:10:13.145178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.b","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.357265+0000","last_change":"2023-07-06T03:55:14.336950+0000","last_active":"2023-07-06T03:55:15.357265+0000","last_peered":"2023-07-06T03:55:15.357265+0000","last_clean":"2023-07-06T03:55:15.357265+0000","last_became_active":"2023-07-06T03:55:14.336584+0000","last_became_peered":"2023-07-06T03:55:14.336584+0000","last_unstale":"2023-07-06T03:55:15.357265+0000","last_undegraded":"2023-07-06T03:55:15.357265+0000","last_fullsized":"2023-07-06T03:55:15.357265+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:43:13.070358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349360+0000","last_change":"2023-07-06T03:55:15.339307+0000","last_active":"2023-07-06T03:55:15.349360+0000","last_peered":"2023-07-06T03:55:15.349360+0000","last_clean":"2023-07-06T03:55:15.349360+0000","last_became_active":"2023-07-06T03:55:15.339015+0000","last_became_peered":"2023-07-06T03:55:15.339015+0000","last_unstale":"2023-07-06T03:55:15.349360+0000","last_undegraded":"2023-07-06T03:55:15.349360+0000","last_fullsized":"2023-07-06T03:55:15.349360+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:13:05.423702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.328724+0000","last_change":"2023-07-06T03:55:14.335098+0000","last_active":"2023-07-06T03:55:15.328724+0000","last_peered":"2023-07-06T03:55:15.328724+0000","last_clean":"2023-07-06T03:55:15.328724+0000","last_became_active":"2023-07-06T03:55:14.334810+0000","last_became_peered":"2023-07-06T03:55:14.334810+0000","last_unstale":"2023-07-06T03:55:15.328724+0000","last_undegraded":"2023-07-06T03:55:15.328724+0000","last_fullsized":"2023-07-06T03:55:15.328724+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:15:58.117913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.347478+0000","last_change":"2023-07-06T03:55:15.340417+0000","last_active":"2023-07-06T03:55:15.347478+0000","last_peered":"2023-07-06T03:55:15.347478+0000","last_clean":"2023-07-06T03:55:15.347478+0000","last_became_active":"2023-07-06T03:55:15.340091+0000","last_became_peered":"2023-07-06T03:55:15.340091+0000","last_unstale":"2023-07-06T03:55:15.347478+0000","last_undegraded":"2023-07-06T03:55:15.347478+0000","last_fullsized":"2023-07-06T03:55:15.347478+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:50:05.835855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.332088+0000","last_change":"2023-07-06T03:55:14.333988+0000","last_active":"2023-07-06T03:55:15.332088+0000","last_peered":"2023-07-06T03:55:15.332088+0000","last_clean":"2023-07-06T03:55:15.332088+0000","last_became_active":"2023-07-06T03:55:14.333716+0000","last_became_peered":"2023-07-06T03:55:14.333716+0000","last_unstale":"2023-07-06T03:55:15.332088+0000","last_undegraded":"2023-07-06T03:55:15.332088+0000","last_fullsized":"2023-07-06T03:55:15.332088+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:29:50.378446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343994+0000","last_change":"2023-07-06T03:55:15.338488+0000","last_active":"2023-07-06T03:55:15.343994+0000","last_peered":"2023-07-06T03:55:15.343994+0000","last_clean":"2023-07-06T03:55:15.343994+0000","last_became_active":"2023-07-06T03:55:15.337648+0000","last_became_peered":"2023-07-06T03:55:15.337648+0000","last_unstale":"2023-07-06T03:55:15.343994+0000","last_undegraded":"2023-07-06T03:55:15.343994+0000","last_fullsized":"2023-07-06T03:55:15.343994+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:20:57.917700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.330325+0000","last_change":"2023-07-06T03:55:14.336520+0000","last_active":"2023-07-06T03:55:15.330325+0000","last_peered":"2023-07-06T03:55:15.330325+0000","last_clean":"2023-07-06T03:55:15.330325+0000","last_became_active":"2023-07-06T03:55:14.336237+0000","last_became_peered":"2023-07-06T03:55:14.336237+0000","last_unstale":"2023-07-06T03:55:15.330325+0000","last_undegraded":"2023-07-06T03:55:15.330325+0000","last_fullsized":"2023-07-06T03:55:15.330325+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:11:39.684006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354838+0000","last_change":"2023-07-06T03:55:15.345865+0000","last_active":"2023-07-06T03:55:15.354838+0000","last_peered":"2023-07-06T03:55:15.354838+0000","last_clean":"2023-07-06T03:55:15.354838+0000","last_became_active":"2023-07-06T03:55:15.344128+0000","last_became_peered":"2023-07-06T03:55:15.344128+0000","last_unstale":"2023-07-06T03:55:15.354838+0000","last_undegraded":"2023-07-06T03:55:15.354838+0000","last_fullsized":"2023-07-06T03:55:15.354838+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:36:15.916915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.7","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.356446+0000","last_change":"2023-07-06T03:55:14.333636+0000","last_active":"2023-07-06T03:55:15.356446+0000","last_peered":"2023-07-06T03:55:15.356446+0000","last_clean":"2023-07-06T03:55:15.356446+0000","last_became_active":"2023-07-06T03:55:14.333254+0000","last_became_peered":"2023-07-06T03:55:14.333254+0000","last_unstale":"2023-07-06T03:55:15.356446+0000","last_undegraded":"2023-07-06T03:55:15.356446+0000","last_fullsized":"2023-07-06T03:55:15.356446+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:55:21.319302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.353707+0000","last_change":"2023-07-06T03:55:15.349910+0000","last_active":"2023-07-06T03:55:15.353707+0000","last_peered":"2023-07-06T03:55:15.353707+0000","last_clean":"2023-07-06T03:55:15.353707+0000","last_became_active":"2023-07-06T03:55:15.348656+0000","last_became_peered":"2023-07-06T03:55:15.348656+0000","last_unstale":"2023-07-06T03:55:15.353707+0000","last_undegraded":"2023-07-06T03:55:15.353707+0000","last_fullsized":"2023-07-06T03:55:15.353707+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:09:16.721736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.328064+0000","last_change":"2023-07-06T03:55:14.330046+0000","last_active":"2023-07-06T03:55:15.328064+0000","last_peered":"2023-07-06T03:55:15.328064+0000","last_clean":"2023-07-06T03:55:15.328064+0000","last_became_active":"2023-07-06T03:55:14.329777+0000","last_became_peered":"2023-07-06T03:55:14.329777+0000","last_unstale":"2023-07-06T03:55:15.328064+0000","last_undegraded":"2023-07-06T03:55:15.328064+0000","last_fullsized":"2023-07-06T03:55:15.328064+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:34:14.143126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.340703+0000","last_change":"2023-07-06T03:55:15.335914+0000","last_active":"2023-07-06T03:55:15.340703+0000","last_peered":"2023-07-06T03:55:15.340703+0000","last_clean":"2023-07-06T03:55:15.340703+0000","last_became_active":"2023-07-06T03:55:15.335623+0000","last_became_peered":"2023-07-06T03:55:15.335623+0000","last_unstale":"2023-07-06T03:55:15.340703+0000","last_undegraded":"2023-07-06T03:55:15.340703+0000","last_fullsized":"2023-07-06T03:55:15.340703+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:23.315172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.324234+0000","last_change":"2023-07-06T03:55:14.335060+0000","last_active":"2023-07-06T03:55:15.324234+0000","last_peered":"2023-07-06T03:55:15.324234+0000","last_clean":"2023-07-06T03:55:15.324234+0000","last_became_active":"2023-07-06T03:55:14.334711+0000","last_became_peered":"2023-07-06T03:55:14.334711+0000","last_unstale":"2023-07-06T03:55:15.324234+0000","last_undegraded":"2023-07-06T03:55:15.324234+0000","last_fullsized":"2023-07-06T03:55:15.324234+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:07:59.933123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.344715+0000","last_change":"2023-07-06T03:55:15.340403+0000","last_active":"2023-07-06T03:55:15.344715+0000","last_peered":"2023-07-06T03:55:15.344715+0000","last_clean":"2023-07-06T03:55:15.344715+0000","last_became_active":"2023-07-06T03:55:15.340031+0000","last_became_peered":"2023-07-06T03:55:15.340031+0000","last_unstale":"2023-07-06T03:55:15.344715+0000","last_undegraded":"2023-07-06T03:55:15.344715+0000","last_fullsized":"2023-07-06T03:55:15.344715+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:40:57.335083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.331875+0000","last_change":"2023-07-06T03:55:14.339015+0000","last_active":"2023-07-06T03:55:15.331875+0000","last_peered":"2023-07-06T03:55:15.331875+0000","last_clean":"2023-07-06T03:55:15.331875+0000","last_became_active":"2023-07-06T03:55:14.338726+0000","last_became_peered":"2023-07-06T03:55:14.338726+0000","last_unstale":"2023-07-06T03:55:15.331875+0000","last_undegraded":"2023-07-06T03:55:15.331875+0000","last_fullsized":"2023-07-06T03:55:15.331875+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:51:35.103485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349860+0000","last_change":"2023-07-06T03:55:15.338210+0000","last_active":"2023-07-06T03:55:15.349860+0000","last_peered":"2023-07-06T03:55:15.349860+0000","last_clean":"2023-07-06T03:55:15.349860+0000","last_became_active":"2023-07-06T03:55:15.337915+0000","last_became_peered":"2023-07-06T03:55:15.337915+0000","last_unstale":"2023-07-06T03:55:15.349860+0000","last_undegraded":"2023-07-06T03:55:15.349860+0000","last_fullsized":"2023-07-06T03:55:15.349860+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:06:43.147768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.332480+0000","last_change":"2023-07-06T03:55:14.330713+0000","last_active":"2023-07-06T03:55:15.332480+0000","last_peered":"2023-07-06T03:55:15.332480+0000","last_clean":"2023-07-06T03:55:15.332480+0000","last_became_active":"2023-07-06T03:55:14.330452+0000","last_became_peered":"2023-07-06T03:55:14.330452+0000","last_unstale":"2023-07-06T03:55:15.332480+0000","last_undegraded":"2023-07-06T03:55:15.332480+0000","last_fullsized":"2023-07-06T03:55:15.332480+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:24:05.850586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.349335+0000","last_change":"2023-07-06T03:55:15.338417+0000","last_active":"2023-07-06T03:55:15.349335+0000","last_peered":"2023-07-06T03:55:15.349335+0000","last_clean":"2023-07-06T03:55:15.349335+0000","last_became_active":"2023-07-06T03:55:15.337525+0000","last_became_peered":"2023-07-06T03:55:15.337525+0000","last_unstale":"2023-07-06T03:55:15.349335+0000","last_undegraded":"2023-07-06T03:55:15.349335+0000","last_fullsized":"2023-07-06T03:55:15.349335+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:16:29.789589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.332153+0000","last_change":"2023-07-06T03:55:14.332276+0000","last_active":"2023-07-06T03:55:15.332153+0000","last_peered":"2023-07-06T03:55:15.332153+0000","last_clean":"2023-07-06T03:55:15.332153+0000","last_became_active":"2023-07-06T03:55:14.332036+0000","last_became_peered":"2023-07-06T03:55:14.332036+0000","last_unstale":"2023-07-06T03:55:15.332153+0000","last_undegraded":"2023-07-06T03:55:15.332153+0000","last_fullsized":"2023-07-06T03:55:15.332153+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:20:15.921178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.338562+0000","last_change":"2023-07-06T03:55:15.333974+0000","last_active":"2023-07-06T03:55:15.338562+0000","last_peered":"2023-07-06T03:55:15.338562+0000","last_clean":"2023-07-06T03:55:15.338562+0000","last_became_active":"2023-07-06T03:55:15.333676+0000","last_became_peered":"2023-07-06T03:55:15.333676+0000","last_unstale":"2023-07-06T03:55:15.338562+0000","last_undegraded":"2023-07-06T03:55:15.338562+0000","last_fullsized":"2023-07-06T03:55:15.338562+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:13:05.423702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.329435+0000","last_change":"2023-07-06T03:55:14.339197+0000","last_active":"2023-07-06T03:55:15.329435+0000","last_peered":"2023-07-06T03:55:15.329435+0000","last_clean":"2023-07-06T03:55:15.329435+0000","last_became_active":"2023-07-06T03:55:14.338922+0000","last_became_peered":"2023-07-06T03:55:14.338922+0000","last_unstale":"2023-07-06T03:55:15.329435+0000","last_undegraded":"2023-07-06T03:55:15.329435+0000","last_fullsized":"2023-07-06T03:55:15.329435+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:30:10.278374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.343644+0000","last_change":"2023-07-06T03:55:15.335004+0000","last_active":"2023-07-06T03:55:15.343644+0000","last_peered":"2023-07-06T03:55:15.343644+0000","last_clean":"2023-07-06T03:55:15.343644+0000","last_became_active":"2023-07-06T03:55:15.334747+0000","last_became_peered":"2023-07-06T03:55:15.334747+0000","last_unstale":"2023-07-06T03:55:15.343644+0000","last_undegraded":"2023-07-06T03:55:15.343644+0000","last_fullsized":"2023-07-06T03:55:15.343644+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:00:09.721559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.321990+0000","last_change":"2023-07-06T03:55:14.331689+0000","last_active":"2023-07-06T03:55:15.321990+0000","last_peered":"2023-07-06T03:55:15.321990+0000","last_clean":"2023-07-06T03:55:15.321990+0000","last_became_active":"2023-07-06T03:55:14.331398+0000","last_became_peered":"2023-07-06T03:55:14.331398+0000","last_unstale":"2023-07-06T03:55:15.321990+0000","last_undegraded":"2023-07-06T03:55:15.321990+0000","last_fullsized":"2023-07-06T03:55:15.321990+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:51:21.435963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.351144+0000","last_change":"2023-07-06T03:55:15.344625+0000","last_active":"2023-07-06T03:55:15.351144+0000","last_peered":"2023-07-06T03:55:15.351144+0000","last_clean":"2023-07-06T03:55:15.351144+0000","last_became_active":"2023-07-06T03:55:15.343908+0000","last_became_peered":"2023-07-06T03:55:15.343908+0000","last_unstale":"2023-07-06T03:55:15.351144+0000","last_undegraded":"2023-07-06T03:55:15.351144+0000","last_fullsized":"2023-07-06T03:55:15.351144+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:04:00.734485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"123'3","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.356761+0000","last_change":"2023-07-06T03:55:14.330576+0000","last_active":"2023-07-06T03:55:15.356761+0000","last_peered":"2023-07-06T03:55:15.356761+0000","last_clean":"2023-07-06T03:55:15.356761+0000","last_became_active":"2023-07-06T03:55:14.330111+0000","last_became_peered":"2023-07-06T03:55:14.330111+0000","last_unstale":"2023-07-06T03:55:15.356761+0000","last_undegraded":"2023-07-06T03:55:15.356761+0000","last_fullsized":"2023-07-06T03:55:15.356761+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:16:13.654894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":191,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.328030+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:55:15.328030+0000","last_peered":"2023-07-06T03:55:15.328030+0000","last_clean":"2023-07-06T03:55:15.328030+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:55:15.328030+0000","last_undegraded":"2023-07-06T03:55:15.328030+0000","last_fullsized":"2023-07-06T03:55:15.328030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:29:44.344080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.352355+0000","last_change":"2023-07-06T03:55:15.343507+0000","last_active":"2023-07-06T03:55:15.352355+0000","last_peered":"2023-07-06T03:55:15.352355+0000","last_clean":"2023-07-06T03:55:15.352355+0000","last_became_active":"2023-07-06T03:55:15.343202+0000","last_became_peered":"2023-07-06T03:55:15.343202+0000","last_unstale":"2023-07-06T03:55:15.352355+0000","last_undegraded":"2023-07-06T03:55:15.352355+0000","last_fullsized":"2023-07-06T03:55:15.352355+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:13:30.631195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.13","version":"123'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.357455+0000","last_change":"2023-07-06T03:55:14.336329+0000","last_active":"2023-07-06T03:55:15.357455+0000","last_peered":"2023-07-06T03:55:15.357455+0000","last_clean":"2023-07-06T03:55:15.357455+0000","last_became_active":"2023-07-06T03:55:14.336019+0000","last_became_peered":"2023-07-06T03:55:14.336019+0000","last_unstale":"2023-07-06T03:55:15.357455+0000","last_undegraded":"2023-07-06T03:55:15.357455+0000","last_fullsized":"2023-07-06T03:55:15.357455+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:11:07.809332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":205,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.320479+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:55:15.320479+0000","last_peered":"2023-07-06T03:55:15.320479+0000","last_clean":"2023-07-06T03:55:15.320479+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:55:15.320479+0000","last_undegraded":"2023-07-06T03:55:15.320479+0000","last_fullsized":"2023-07-06T03:55:15.320479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:53:07.731286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.348615+0000","last_change":"2023-07-06T03:55:15.342873+0000","last_active":"2023-07-06T03:55:15.348615+0000","last_peered":"2023-07-06T03:55:15.348615+0000","last_clean":"2023-07-06T03:55:15.348615+0000","last_became_active":"2023-07-06T03:55:15.342532+0000","last_became_peered":"2023-07-06T03:55:15.342532+0000","last_unstale":"2023-07-06T03:55:15.348615+0000","last_undegraded":"2023-07-06T03:55:15.348615+0000","last_fullsized":"2023-07-06T03:55:15.348615+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:33:38.030192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"123'8","reported_seq":19,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.358638+0000","last_change":"2023-07-06T03:55:14.341914+0000","last_active":"2023-07-06T03:55:15.358638+0000","last_peered":"2023-07-06T03:55:15.358638+0000","last_clean":"2023-07-06T03:55:15.358638+0000","last_became_active":"2023-07-06T03:55:14.341678+0000","last_became_peered":"2023-07-06T03:55:14.341678+0000","last_unstale":"2023-07-06T03:55:15.358638+0000","last_undegraded":"2023-07-06T03:55:15.358638+0000","last_fullsized":"2023-07-06T03:55:15.358638+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:19:40.839313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":209,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.322805+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:55:15.322805+0000","last_peered":"2023-07-06T03:55:15.322805+0000","last_clean":"2023-07-06T03:55:15.322805+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:55:15.322805+0000","last_undegraded":"2023-07-06T03:55:15.322805+0000","last_fullsized":"2023-07-06T03:55:15.322805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:02:27.182880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.354404+0000","last_change":"2023-07-06T03:55:15.350728+0000","last_active":"2023-07-06T03:55:15.354404+0000","last_peered":"2023-07-06T03:55:15.354404+0000","last_clean":"2023-07-06T03:55:15.354404+0000","last_became_active":"2023-07-06T03:55:15.350446+0000","last_became_peered":"2023-07-06T03:55:15.350446+0000","last_unstale":"2023-07-06T03:55:15.354404+0000","last_undegraded":"2023-07-06T03:55:15.354404+0000","last_fullsized":"2023-07-06T03:55:15.354404+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:20:01.214350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.320383+0000","last_change":"2023-07-06T03:55:14.335781+0000","last_active":"2023-07-06T03:55:15.320383+0000","last_peered":"2023-07-06T03:55:15.320383+0000","last_clean":"2023-07-06T03:55:15.320383+0000","last_became_active":"2023-07-06T03:55:14.334645+0000","last_became_peered":"2023-07-06T03:55:14.334645+0000","last_unstale":"2023-07-06T03:55:15.320383+0000","last_undegraded":"2023-07-06T03:55:15.320383+0000","last_fullsized":"2023-07-06T03:55:15.320383+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:32:11.846098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":205,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.324070+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:55:15.324070+0000","last_peered":"2023-07-06T03:55:15.324070+0000","last_clean":"2023-07-06T03:55:15.324070+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:55:15.324070+0000","last_undegraded":"2023-07-06T03:55:15.324070+0000","last_fullsized":"2023-07-06T03:55:15.324070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:47:19.788235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":344,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.327623+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:55:15.327623+0000","last_peered":"2023-07-06T03:55:15.327623+0000","last_clean":"2023-07-06T03:55:15.327623+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:55:15.327623+0000","last_undegraded":"2023-07-06T03:55:15.327623+0000","last_fullsized":"2023-07-06T03:55:15.327623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:49:16.388742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.355392+0000","last_change":"2023-07-06T03:55:15.353242+0000","last_active":"2023-07-06T03:55:15.355392+0000","last_peered":"2023-07-06T03:55:15.355392+0000","last_clean":"2023-07-06T03:55:15.355392+0000","last_became_active":"2023-07-06T03:55:15.352933+0000","last_became_peered":"2023-07-06T03:55:15.352933+0000","last_unstale":"2023-07-06T03:55:15.355392+0000","last_undegraded":"2023-07-06T03:55:15.355392+0000","last_fullsized":"2023-07-06T03:55:15.355392+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:06:08.221246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.331948+0000","last_change":"2023-07-06T03:55:14.332810+0000","last_active":"2023-07-06T03:55:15.331948+0000","last_peered":"2023-07-06T03:55:15.331948+0000","last_clean":"2023-07-06T03:55:15.331948+0000","last_became_active":"2023-07-06T03:55:14.332548+0000","last_became_peered":"2023-07-06T03:55:14.332548+0000","last_unstale":"2023-07-06T03:55:15.331948+0000","last_undegraded":"2023-07-06T03:55:15.331948+0000","last_fullsized":"2023-07-06T03:55:15.331948+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:48:53.023505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":206,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.320639+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:55:15.320639+0000","last_peered":"2023-07-06T03:55:15.320639+0000","last_clean":"2023-07-06T03:55:15.320639+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:55:15.320639+0000","last_undegraded":"2023-07-06T03:55:15.320639+0000","last_fullsized":"2023-07-06T03:55:15.320639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:12:13.805158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.344545+0000","last_change":"2023-07-06T03:55:15.338743+0000","last_active":"2023-07-06T03:55:15.344545+0000","last_peered":"2023-07-06T03:55:15.344545+0000","last_clean":"2023-07-06T03:55:15.344545+0000","last_became_active":"2023-07-06T03:55:15.338481+0000","last_became_peered":"2023-07-06T03:55:15.338481+0000","last_unstale":"2023-07-06T03:55:15.344545+0000","last_undegraded":"2023-07-06T03:55:15.344545+0000","last_fullsized":"2023-07-06T03:55:15.344545+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:57:40.184420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"123'6","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.358836+0000","last_change":"2023-07-06T03:55:14.329404+0000","last_active":"2023-07-06T03:55:15.358836+0000","last_peered":"2023-07-06T03:55:15.358836+0000","last_clean":"2023-07-06T03:55:15.358836+0000","last_became_active":"2023-07-06T03:55:14.329121+0000","last_became_peered":"2023-07-06T03:55:14.329121+0000","last_unstale":"2023-07-06T03:55:15.358836+0000","last_undegraded":"2023-07-06T03:55:15.358836+0000","last_fullsized":"2023-07-06T03:55:15.358836+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:47:09.939272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.324256+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:55:15.324256+0000","last_peered":"2023-07-06T03:55:15.324256+0000","last_clean":"2023-07-06T03:55:15.324256+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:55:15.324256+0000","last_undegraded":"2023-07-06T03:55:15.324256+0000","last_fullsized":"2023-07-06T03:55:15.324256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:40:44.784735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.339487+0000","last_change":"2023-07-06T03:55:15.335912+0000","last_active":"2023-07-06T03:55:15.339487+0000","last_peered":"2023-07-06T03:55:15.339487+0000","last_clean":"2023-07-06T03:55:15.339487+0000","last_became_active":"2023-07-06T03:55:15.335555+0000","last_became_peered":"2023-07-06T03:55:15.335555+0000","last_unstale":"2023-07-06T03:55:15.339487+0000","last_undegraded":"2023-07-06T03:55:15.339487+0000","last_fullsized":"2023-07-06T03:55:15.339487+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:35:39.943450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.321647+0000","last_change":"2023-07-06T03:55:14.326092+0000","last_active":"2023-07-06T03:55:15.321647+0000","last_peered":"2023-07-06T03:55:15.321647+0000","last_clean":"2023-07-06T03:55:15.321647+0000","last_became_active":"2023-07-06T03:55:14.324656+0000","last_became_peered":"2023-07-06T03:55:14.324656+0000","last_unstale":"2023-07-06T03:55:15.321647+0000","last_undegraded":"2023-07-06T03:55:15.321647+0000","last_fullsized":"2023-07-06T03:55:15.321647+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:24:09.584868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":197,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.327872+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:55:15.327872+0000","last_peered":"2023-07-06T03:55:15.327872+0000","last_clean":"2023-07-06T03:55:15.327872+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:55:15.327872+0000","last_undegraded":"2023-07-06T03:55:15.327872+0000","last_fullsized":"2023-07-06T03:55:15.327872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:27:35.438765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.350873+0000","last_change":"2023-07-06T03:55:15.344207+0000","last_active":"2023-07-06T03:55:15.350873+0000","last_peered":"2023-07-06T03:55:15.350873+0000","last_clean":"2023-07-06T03:55:15.350873+0000","last_became_active":"2023-07-06T03:55:15.343943+0000","last_became_peered":"2023-07-06T03:55:15.343943+0000","last_unstale":"2023-07-06T03:55:15.350873+0000","last_undegraded":"2023-07-06T03:55:15.350873+0000","last_fullsized":"2023-07-06T03:55:15.350873+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:14.303821+0000","last_clean_scrub_stamp":"2023-07-06T03:55:14.303821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:33:17.748239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.327777+0000","last_change":"2023-07-06T03:55:14.334603+0000","last_active":"2023-07-06T03:55:15.327777+0000","last_peered":"2023-07-06T03:55:15.327777+0000","last_clean":"2023-07-06T03:55:15.327777+0000","last_became_active":"2023-07-06T03:55:14.334317+0000","last_became_peered":"2023-07-06T03:55:14.334317+0000","last_unstale":"2023-07-06T03:55:15.327777+0000","last_undegraded":"2023-07-06T03:55:15.327777+0000","last_fullsized":"2023-07-06T03:55:15.327777+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:55:13.293819+0000","last_clean_scrub_stamp":"2023-07-06T03:55:13.293819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:21:08.819941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":205,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-07-06T03:55:15.321008+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:55:15.321008+0000","last_peered":"2023-07-06T03:55:15.321008+0000","last_clean":"2023-07-06T03:55:15.321008+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:55:15.321008+0000","last_undegraded":"2023-07-06T03:55:15.321008+0000","last_fullsized":"2023-07-06T03:55:15.321008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:15:18.889558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36644,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335196,"statfs":{"total":96636764160,"available":96599240704,"internally_reserved":0,"allocated":1536000,"data_stored":705609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68600000000000005,"15min":0.70499999999999996},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.88300000000000001,"5min":1.0620000000000001,"15min":4.0570000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67900000000000005,"15min":0.752},"min":{"1min":0.39900000000000002,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.85499999999999998,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68300000000000005,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.45000000000000001,"15min":0.34000000000000002},"max":{"1min":1.016,"5min":1.091,"15min":2.6219999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69099999999999995,"15min":0.82699999999999996},"min":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68100000000000005,"15min":0.76000000000000001},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.35199999999999998},"max":{"1min":0.80600000000000005,"5min":0.98599999999999999,"15min":3.77},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.48099999999999998,"5min":0.47799999999999998,"15min":0.45000000000000001},"max":{"1min":0.90800000000000003,"5min":0.94399999999999995,"15min":0.98899999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.753,"15min":0.752},"min":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.43099999999999999},"max":{"1min":0.91700000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.62,"15min":0.626},"min":{"1min":0.501,"5min":0.36099999999999999,"15min":0.34599999999999997},"max":{"1min":0.89600000000000002,"5min":0.90200000000000002,"15min":0.94699999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66800000000000004,"15min":0.70399999999999996},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.374},"max":{"1min":0.94099999999999995,"5min":0.94399999999999995,"15min":1.4590000000000001},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.52800000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.45300000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.56799999999999995,"5min":0.438,"15min":0.438},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.5349999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.77700000000000002},"min":{"1min":0.49199999999999999,"5min":0.46500000000000002,"15min":0.42299999999999999},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.80800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574975,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1540096,"data_stored":705643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72799999999999998,"15min":0.67400000000000004},"min":{"1min":0.39800000000000002,"5min":0.32800000000000001,"15min":0.29399999999999998},"max":{"1min":0.89500000000000002,"5min":1.24,"15min":1.24},"last":5.6269999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.77800000000000002,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.44700000000000001,"15min":0.26900000000000002},"max":{"1min":1.046,"5min":3.3399999999999999,"15min":3.3399999999999999},"last":3.3119999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68799999999999994,"15min":0.66900000000000004},"min":{"1min":0.44500000000000001,"5min":0.372,"15min":0.34599999999999997},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":5.8860000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70199999999999996,"15min":0.67800000000000005},"min":{"1min":0.51800000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94499999999999995,"5min":0.94699999999999995,"15min":1.1759999999999999},"last":5.96}]},{"osd":2,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.752,"15min":0.68200000000000005},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.98499999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":8.3740000000000006},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77500000000000002,"15min":0.72799999999999998},"min":{"1min":0.60499999999999998,"5min":0.47699999999999998,"15min":0.38700000000000001},"max":{"1min":1.153,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":5.8449999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":1.085,"5min":1.2110000000000001,"15min":2.1850000000000001},"last":3.4689999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.75,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.373},"max":{"1min":1.071,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":8.5269999999999992}]},{"osd":4,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.69899999999999995},"min":{"1min":0.44900000000000001,"5min":0.42799999999999999,"15min":0.33200000000000002},"max":{"1min":0.83499999999999996,"5min":1.0309999999999999,"15min":1.1339999999999999},"last":5.7759999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67000000000000004,"15min":0.71699999999999997},"min":{"1min":0.42399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":0.98699999999999999,"15min":1.085},"last":3.5190000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65400000000000003,"15min":0.69599999999999995},"min":{"1min":0.48699999999999999,"5min":0.46300000000000002,"15min":0.38600000000000001},"max":{"1min":0.89500000000000002,"5min":0.98799999999999999,"15min":1.218},"last":8.4870000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72399999999999998,"15min":0.77000000000000002},"min":{"1min":0.54700000000000004,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0489999999999999,"15min":1.198},"last":5.923}]},{"osd":7,"last update":"Thu Jul 6 03:54:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72499999999999998,"15min":0.79500000000000004},"min":{"1min":0.54700000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.323},"last":3.3900000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74399999999999999,"15min":0.80600000000000005},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.41499999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.4139999999999999},"last":8.4429999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":2096,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":2146304,"data_stored":1297631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.42999999999999999,"15min":0.50800000000000001},"min":{"1min":0.27100000000000002,"5min":0.25900000000000001,"15min":0.23699999999999999},"max":{"1min":0.56000000000000005,"5min":0.72299999999999998,"15min":2.8879999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50700000000000001,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.32300000000000001,"15min":0.313},"max":{"1min":0.621,"5min":0.78300000000000003,"15min":1.96},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.44600000000000001,"15min":0.78100000000000003},"min":{"1min":0.32300000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.58599999999999997,"5min":0.66800000000000004,"15min":13.055999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.54100000000000004,"15min":0.60599999999999998},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.59799999999999998,"5min":0.96199999999999997,"15min":2.8479999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.52800000000000002,"15min":0.58699999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48299999999999998,"15min":0.52600000000000002},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.56899999999999995,"5min":0.71599999999999997,"15min":1.2709999999999999},"last":0.44}]},{"osd":4,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.73599999999999999,"15min":1.05},"min":{"1min":0.54000000000000004,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":0.90400000000000003,"5min":1,"15min":12.989000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78200000000000003,"15min":0.81399999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":0.875,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.66500000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.432},"max":{"1min":0.88300000000000001,"5min":1.0860000000000001,"15min":1.3100000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72199999999999998,"15min":1.052},"min":{"1min":0.56100000000000005,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.83299999999999996,"5min":1.077,"15min":13.023999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.76200000000000001,"15min":0.79800000000000004},"min":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.88700000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76700000000000002,"15min":1.0960000000000001},"min":{"1min":0.59399999999999997,"5min":0.54000000000000004,"15min":0.47799999999999998},"max":{"1min":0.96399999999999997,"5min":1.0569999999999999,"15min":12.941000000000001},"last":0.63}]},{"osd":7,"last update":"Thu Jul 6 03:54:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.80900000000000005},"min":{"1min":0.48699999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92600000000000005,"5min":1.071,"15min":2.806},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.755,"15min":0.79500000000000004},"min":{"1min":0.58599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94199999999999995,"5min":1.1319999999999999,"15min":1.337},"last":0.67500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1552384,"data_stored":707263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.28899999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":0.57099999999999995,"5min":0.753,"15min":0.753},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.58099999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.436}]},{"osd":2,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.66200000000000003,"5min":0.69199999999999995,"15min":0.73699999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48699999999999999,"15min":0.496},"min":{"1min":0.376,"5min":0.315,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51100000000000001,"15min":0.48499999999999999},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.29899999999999999},"max":{"1min":0.59099999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.443,"5min":0.39400000000000002,"15min":0.38800000000000001},"max":{"1min":0.628,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72699999999999998,"15min":0.73799999999999999},"min":{"1min":0.48399999999999999,"5min":0.45700000000000002,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":1.1699999999999999,"15min":1.171},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.59199999999999997,"5min":0.52400000000000002,"15min":0.36799999999999999},"max":{"1min":0.92200000000000004,"5min":1.032,"15min":1.034},"last":0.627}]},{"osd":5,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.72799999999999998},"min":{"1min":0.51100000000000001,"5min":0.503,"15min":0.45600000000000002},"max":{"1min":0.95399999999999996,"5min":1.1080000000000001,"15min":1.1419999999999999},"last":0.755},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.55100000000000005,"5min":0.53600000000000003,"15min":0.46999999999999997},"max":{"1min":0.82199999999999995,"5min":1.1040000000000001,"15min":1.1220000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.84599999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73599999999999999,"15min":0.748},"min":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.46200000000000002},"max":{"1min":0.93999999999999995,"5min":1.0649999999999999,"15min":1.546},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:54:14 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78200000000000003,"15min":0.79400000000000004},"min":{"1min":0.56000000000000005,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.6830000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.76900000000000002,"15min":0.98999999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53100000000000003},"max":{"1min":0.90500000000000003,"5min":1.0669999999999999,"15min":9.5960000000000001},"last":0.80100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40728,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331112,"statfs":{"total":96636764160,"available":96595058688,"internally_reserved":0,"allocated":1523712,"data_stored":698481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.47699999999999998,"15min":0.442},"min":{"1min":0.30499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65500000000000003,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.504,"15min":0.47999999999999998},"min":{"1min":0.34200000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56000000000000005,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.55000000000000004,"15min":0.51000000000000001},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.629,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.56599999999999995,"15min":0.52400000000000002},"min":{"1min":0.38,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.64600000000000002,"5min":3.657,"15min":3.657},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.626,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69099999999999995,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59699999999999998,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94499999999999995,"15min":0.78600000000000003},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.121,"5min":12.007,"15min":12.007},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.81299999999999994,"15min":0.73699999999999999},"min":{"1min":0.52200000000000002,"5min":0.49299999999999999,"15min":0.36399999999999999},"max":{"1min":1.1930000000000001,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79200000000000004,"15min":0.75},"min":{"1min":0.51100000000000001,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":1.1459999999999999,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.747,"5min":0.88100000000000001,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.137,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.94599999999999995,"15min":0.78900000000000003},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.165,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.92300000000000004,"15min":0.80900000000000005},"min":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":1.107,"5min":13.084,"15min":13.084},"last":0.84299999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:54:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83299999999999996,"15min":0.82499999999999996},"min":{"1min":0.60599999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.089,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.871,"15min":0.81799999999999995},"min":{"1min":0.55800000000000005,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.2170000000000001,"5min":4.484,"15min":4.484},"last":0.79800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1523712,"data_stored":698481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.44900000000000001,"15min":0.56399999999999995},"min":{"1min":0.25700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60099999999999998,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.44},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51800000000000002,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.85699999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.45200000000000001,"15min":0.88300000000000001},"min":{"1min":0.316,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.54700000000000004,"5min":1.3620000000000001,"15min":15.911},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.53400000000000003,"15min":0.95699999999999996},"min":{"1min":0.33400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76000000000000001,"5min":1.347,"15min":16.024000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.499,"15min":0.499},"min":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.27400000000000002},"max":{"1min":0.61099999999999999,"5min":3.25,"15min":3.25},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.54200000000000004,"15min":0.96199999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.64100000000000001,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.379}]},{"osd":4,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.52800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":1.782,"15min":1.782},"last":0.622},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75900000000000001,"15min":0.88400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.36699999999999999},"max":{"1min":1.373,"5min":1.724,"15min":5.7300000000000004},"last":0.58299999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76300000000000001,"15min":0.88500000000000001},"min":{"1min":0.54000000000000004,"5min":0.49199999999999999,"15min":0.39700000000000002},"max":{"1min":0.96099999999999997,"5min":1.411,"15min":5.077},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.93400000000000005},"min":{"1min":0.56999999999999995,"5min":0.54500000000000004,"15min":0.44800000000000001},"max":{"1min":0.94699999999999995,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76800000000000002,"15min":0.78900000000000003},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.026,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.75900000000000001,"15min":0.89300000000000002},"min":{"1min":0.61199999999999999,"5min":0.503,"15min":0.45600000000000002},"max":{"1min":0.98199999999999998,"5min":1.4490000000000001,"15min":5.157},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76800000000000002,"15min":0.81399999999999995},"min":{"1min":0.58599999999999997,"5min":0.51700000000000002,"15min":0.47399999999999998},"max":{"1min":0.94599999999999995,"5min":1.863,"15min":1.863},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.76600000000000001,"15min":0.90400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.492,"15min":5.1189999999999998},"last":0.60399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36632,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335208,"statfs":{"total":96636764160,"available":96599252992,"internally_reserved":0,"allocated":1523712,"data_stored":698481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66200000000000003,"15min":0.64600000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.88800000000000001,"5min":0.90400000000000003,"15min":1.784},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.65000000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.38},"max":{"1min":0.90200000000000002,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.66200000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.80900000000000005},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.38},"max":{"1min":0.996,"5min":0.996,"15min":6.258},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.52400000000000002,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.95499999999999996,"15min":1.806},"last":0.753}]},{"osd":2,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69999999999999996,"15min":0.82599999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.29199999999999998},"max":{"1min":0.90400000000000003,"5min":0.91000000000000003,"15min":6.4740000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.90100000000000002},"min":{"1min":0.58199999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":6.4859999999999998},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.52500000000000002,"5min":0.442,"15min":0.40000000000000002},"max":{"1min":0.86199999999999999,"5min":0.97799999999999998,"15min":1.25},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.71299999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.99099999999999999,"15min":1.413},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.64700000000000002},"min":{"1min":0.50700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.92100000000000004,"15min":1.3280000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63700000000000001,"15min":0.65700000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94299999999999995,"15min":0.99299999999999999},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.41099999999999998,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75800000000000001,"15min":0.78300000000000003},"min":{"1min":0.56799999999999995,"5min":0.52400000000000002,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.008,"15min":1.6839999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.54100000000000004,"5min":0.44500000000000001,"15min":0.41299999999999998},"max":{"1min":0.89200000000000002,"5min":0.95699999999999996,"15min":1.0660000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.73799999999999999},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.036,"5min":1.036,"15min":1.21},"last":0.67100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":2092,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":2142208,"data_stored":1295666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.91100000000000003,"15min":4.5010000000000003},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.67400000000000004,"15min":0.78500000000000003},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.83399999999999996,"5min":0.96199999999999997,"15min":6.4059999999999997},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.502,"5min":0.44800000000000001,"15min":0.307},"max":{"1min":0.76200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.46100000000000002,"15min":0.41399999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.546},"last":0.70499999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79300000000000004,"15min":0.72799999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.376},"max":{"1min":0.86499999999999999,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.79100000000000004,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.47599999999999998,"15min":0.42199999999999999},"max":{"1min":0.78100000000000003,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71399999999999997,"15min":0.73899999999999999},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":0.85299999999999998,"5min":1.3140000000000001,"15min":4.484},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":1.0229999999999999,"15min":1.6240000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64000000000000001,"15min":0.68700000000000006},"min":{"1min":0.41899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":2.718},"last":0.60199999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83899999999999997,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.70999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.82599999999999996,"5min":0.94099999999999995,"15min":1.669},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71499999999999997,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.93799999999999994,"5min":1.002,"15min":2.0459999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73699999999999999,"15min":0.77000000000000002},"min":{"1min":0.47199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88,"5min":1.0249999999999999,"15min":2.2919999999999998},"last":0.63}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:55:20.182 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T03:55:20.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T03:55:20.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.415+0000 7fb4d33ff700 1 Processor -- start 2023-07-06T03:55:20.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.415+0000 7fb4d33ff700 1 -- start start 2023-07-06T03:55:20.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.416+0000 7fb4d33ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4cc15a100 0x7fb4cc0cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.416+0000 7fb4d33ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4cc0d0c10 0x7fb4cc160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.416+0000 7fb4d33ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb4cc0b4790 0x7fb4cc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.416+0000 7fb4d33ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc0cda80 con 0x7fb4cc15a100 2023-07-06T03:55:20.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.416+0000 7fb4d33ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc161150 con 0x7fb4cc0b4790 2023-07-06T03:55:20.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.416+0000 7fb4d33ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0b5130 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.416+0000 7fb4d119b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4cc15a100 0x7fb4cc0cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.416+0000 7fb4d119b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4cc15a100 0x7fb4cc0cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57660/0 (socket says 172.21.15.97:57660) 2023-07-06T03:55:20.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.416+0000 7fb4d119b700 1 -- 172.21.15.97:0/2545100763 learned_addr learned my addr 172.21.15.97:0/2545100763 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.417+0000 7fb4d099a700 1 --2- 172.21.15.97:0/2545100763 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4cc0d0c10 0x7fb4cc160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.417+0000 7fb4d199c700 1 --2- 172.21.15.97:0/2545100763 >> v2:172.21.15.146:3300/0 conn(0x7fb4cc0b4790 0x7fb4cc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.417+0000 7fb4d119b700 1 -- 172.21.15.97:0/2545100763 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4cc0d0c10 msgr2=0x7fb4cc160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:20.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.417+0000 7fb4d119b700 1 --2- 172.21.15.97:0/2545100763 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4cc0d0c10 0x7fb4cc160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.417+0000 7fb4d119b700 1 -- 172.21.15.97:0/2545100763 >> v2:172.21.15.146:3300/0 conn(0x7fb4cc0b4790 msgr2=0x7fb4cc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:20.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.417+0000 7fb4d119b700 1 --2- 172.21.15.97:0/2545100763 >> v2:172.21.15.146:3300/0 conn(0x7fb4cc0b4790 0x7fb4cc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.417+0000 7fb4d119b700 1 -- 172.21.15.97:0/2545100763 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0d1090 con 0x7fb4cc15a100 2023-07-06T03:55:20.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.417+0000 7fb4d119b700 1 --2- 172.21.15.97:0/2545100763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc15a100 0x7fb4cc0cefd0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc08b060 tx=0x7fb4c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89ef37e548f3ab2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:20.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.417+0000 7fb4c3fff700 1 -- 172.21.15.97:0/2545100763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800b860 con 0x7fb4cc15a100 2023-07-06T03:55:20.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.417+0000 7fb4c3fff700 1 -- 172.21.15.97:0/2545100763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c800b9c0 con 0x7fb4cc15a100 2023-07-06T03:55:20.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.417+0000 7fb4c3fff700 1 -- 172.21.15.97:0/2545100763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800bd50 con 0x7fb4cc15a100 2023-07-06T03:55:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.418+0000 7fb4d33ff700 1 -- 172.21.15.97:0/2545100763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc15a100 msgr2=0x7fb4cc0cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.418+0000 7fb4d33ff700 1 --2- 172.21.15.97:0/2545100763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc15a100 0x7fb4cc0cefd0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc08b060 tx=0x7fb4c8009c00 comp rx=0 tx=0).stop 2023-07-06T03:55:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.418+0000 7fb4d33ff700 1 -- 172.21.15.97:0/2545100763 shutdown_connections 2023-07-06T03:55:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.418+0000 7fb4d33ff700 1 --2- 172.21.15.97:0/2545100763 >> v2:172.21.15.146:3300/0 conn(0x7fb4cc0b4790 0x7fb4cc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.418+0000 7fb4d33ff700 1 --2- 172.21.15.97:0/2545100763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc15a100 0x7fb4cc0cefd0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.418+0000 7fb4d33ff700 1 --2- 172.21.15.97:0/2545100763 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4cc0d0c10 0x7fb4cc160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.418+0000 7fb4d33ff700 1 -- 172.21.15.97:0/2545100763 >> 172.21.15.97:0/2545100763 conn(0x7fb4cc06b2d0 msgr2=0x7fb4cc0cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.418+0000 7fb4d33ff700 1 -- 172.21.15.97:0/2545100763 shutdown_connections 2023-07-06T03:55:20.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.418+0000 7fb4d33ff700 1 -- 172.21.15.97:0/2545100763 wait complete. 2023-07-06T03:55:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.420+0000 7fb4d33ff700 1 Processor -- start 2023-07-06T03:55:20.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.420+0000 7fb4d33ff700 1 -- start start 2023-07-06T03:55:20.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d33ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4cc0b4790 0x7fb4cc1f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d33ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0d0c10 0x7fb4cc0c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d33ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc15a100 0x7fb4cc0b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d33ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc1f6210 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d33ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0c91a0 con 0x7fb4cc15a100 2023-07-06T03:55:20.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d33ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0c7bc0 con 0x7fb4cc0b4790 2023-07-06T03:55:20.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d099a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0d0c10 0x7fb4cc0c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d099a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0d0c10 0x7fb4cc0c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57668/0 (socket says 172.21.15.97:57668) 2023-07-06T03:55:20.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d099a700 1 -- 172.21.15.97:0/1623932818 learned_addr learned my addr 172.21.15.97:0/1623932818 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:20.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d119b700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4cc0b4790 0x7fb4cc1f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d199c700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc15a100 0x7fb4cc0b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d099a700 1 -- 172.21.15.97:0/1623932818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4cc0b4790 msgr2=0x7fb4cc1f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d099a700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4cc0b4790 0x7fb4cc1f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d099a700 1 -- 172.21.15.97:0/1623932818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc15a100 msgr2=0x7fb4cc0b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d099a700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc15a100 0x7fb4cc0b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d099a700 1 -- 172.21.15.97:0/1623932818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c8009550 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.421+0000 7fb4d099a700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0d0c10 0x7fb4cc0c8c60 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00d800 tx=0x7fb4bc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.422+0000 7fb4c1ffb700 1 -- 172.21.15.97:0/1623932818 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0125f0 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.422+0000 7fb4c1ffb700 1 -- 172.21.15.97:0/1623932818 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc012750 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.422+0000 7fb4d33ff700 1 -- 172.21.15.97:0/1623932818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc0c7ea0 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.422+0000 7fb4c1ffb700 1 -- 172.21.15.97:0/1623932818 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc012b00 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.422+0000 7fb4d33ff700 1 -- 172.21.15.97:0/1623932818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc0c8460 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.423+0000 7fb4d33ff700 1 -- 172.21.15.97:0/1623932818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc067920 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.424+0000 7fb4c1ffb700 1 -- 172.21.15.97:0/1623932818 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb4bc012c60 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.425+0000 7fb4c1ffb700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4b006ace0 0x7fb4b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.425+0000 7fb4c1ffb700 1 -- 172.21.15.97:0/1623932818 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4bc089860 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.425+0000 7fb4d119b700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4b006ace0 0x7fb4b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.426+0000 7fb4d119b700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4b006ace0 0x7fb4b006d190 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8000c00 tx=0x7fb4c8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.429+0000 7fb4c1ffb700 1 -- 172.21.15.97:0/1623932818 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb4bc054910 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.701+0000 7fb4d33ff700 1 -- 172.21.15.97:0/1623932818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb4cc04e340 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.705+0000 7fb4c1ffb700 1 -- 172.21.15.97:0/1623932818 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb4bc057f30 con 0x7fb4cc0d0c10 2023-07-06T03:55:20.707 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:55:20.707 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg== 2023-07-06T03:55:20.707 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-07-06T03:55:20.707 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-07-06T03:55:20.708 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T03:55:20.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 -- 172.21.15.97:0/1623932818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4b006ace0 msgr2=0x7fb4b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4b006ace0 0x7fb4b006d190 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8000c00 tx=0x7fb4c8008030 comp rx=0 tx=0).stop 2023-07-06T03:55:20.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 -- 172.21.15.97:0/1623932818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0d0c10 msgr2=0x7fb4cc0c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0d0c10 0x7fb4cc0c8c60 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00d800 tx=0x7fb4bc00dbc0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 -- 172.21.15.97:0/1623932818 shutdown_connections 2023-07-06T03:55:20.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb4b006ace0 0x7fb4b006d190 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4cc15a100 0x7fb4cc0b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb4cc0b4790 0x7fb4cc1f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 --2- 172.21.15.97:0/1623932818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0d0c10 0x7fb4cc0c8c60 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 -- 172.21.15.97:0/1623932818 >> 172.21.15.97:0/1623932818 conn(0x7fb4cc06b2d0 msgr2=0x7fb4cc15f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:20.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 -- 172.21.15.97:0/1623932818 shutdown_connections 2023-07-06T03:55:20.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.708+0000 7fb4d33ff700 1 -- 172.21.15.97:0/1623932818 wait complete. 2023-07-06T03:55:20.711 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-07-06T03:55:20.727 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:55:20.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.920+0000 7fbecf1ff700 1 Processor -- start 2023-07-06T03:55:20.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.920+0000 7fbecf1ff700 1 -- start start 2023-07-06T03:55:20.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.921+0000 7fbecf1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbec80cd390 0x7fbec8160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.921+0000 7fbecf1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec80cdae0 con 0x7fbec8160dd0 2023-07-06T03:55:20.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbecf1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec80b6640 con 0x7fbec815c520 2023-07-06T03:55:20.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbecf1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec80b6920 con 0x7fbec80cd390 2023-07-06T03:55:20.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbecd79c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbec8160dd0 0x7fbec80b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53510/0 (socket says 172.21.15.97:53510) 2023-07-06T03:55:20.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbecd79c700 1 -- 172.21.15.97:0/4073773336 learned_addr learned my addr 172.21.15.97:0/4073773336 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:20.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbec7fff700 1 --2- 172.21.15.97:0/4073773336 >> v2:172.21.15.146:3300/0 conn(0x7fbec80cd390 0x7fbec8160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435630387 0 0) 0x7fbec80cdae0 con 0x7fbec8160dd0 2023-07-06T03:55:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb80035b0 con 0x7fbec8160dd0 2023-07-06T03:55:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 708765288 0 0) 0x7fbeb80035b0 con 0x7fbec8160dd0 2023-07-06T03:55:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 >> v2:172.21.15.146:3300/0 conn(0x7fbec80cd390 msgr2=0x7fbec8160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbec77fe700 1 --2- 172.21.15.97:0/4073773336 >> v2:172.21.15.146:3300/0 conn(0x7fbec80cd390 0x7fbec8160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 >> v1:172.21.15.146:6789/0 conn(0x7fbec815c520 legacy=0x7fbec8160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec80b6c00 con 0x7fbec8160dd0 2023-07-06T03:55:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.922+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fbec0001670 con 0x7fbec8160dd0 2023-07-06T03:55:20.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 >> v1:172.21.15.97:6789/0 conn(0x7fbec8160dd0 legacy=0x7fbec80b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbec77fe700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb8003a00 0x7fbeb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbec77fe700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeb8004850 0x7fbeb8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbec77fe700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb8005200 0x7fbeb800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec80b6920 con 0x7fbeb8005200 2023-07-06T03:55:20.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb800c060 con 0x7fbeb8004850 2023-07-06T03:55:20.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb800c220 con 0x7fbeb8003a00 2023-07-06T03:55:20.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbecd79c700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb8005200 0x7fbeb800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbec7fff700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeb8004850 0x7fbeb8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbeccf9b700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb8003a00 0x7fbeb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbecd79c700 1 -- 172.21.15.97:0/4073773336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb8003a00 msgr2=0x7fbeb8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:20.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbecd79c700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb8003a00 0x7fbeb8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbecd79c700 1 -- 172.21.15.97:0/4073773336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeb8004850 msgr2=0x7fbeb8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:20.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbecd79c700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeb8004850 0x7fbeb8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbecd79c700 1 -- 172.21.15.97:0/4073773336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb80035b0 con 0x7fbeb8005200 2023-07-06T03:55:20.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbec7fff700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbeb8004850 0x7fbeb8004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:20.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.923+0000 7fbecd79c700 1 -- 172.21.15.97:0/4073773336 reap_dead start 2023-07-06T03:55:20.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.924+0000 7fbecd79c700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb8005200 0x7fbeb800ba10 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fbec000c210 tx=0x7fbeb8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1a3c83ac8d790c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:20.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.924+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0007b70 con 0x7fbeb8005200 2023-07-06T03:55:20.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.924+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec000cdf0 con 0x7fbeb8005200 2023-07-06T03:55:20.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.924+0000 7fbec77fe700 1 -- 172.21.15.97:0/4073773336 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0005ba0 con 0x7fbeb8005200 2023-07-06T03:55:20.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.924+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4073773336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb8005200 msgr2=0x7fbeb800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.924+0000 7fbecf1ff700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb8005200 0x7fbeb800ba10 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fbec000c210 tx=0x7fbeb8005880 comp rx=0 tx=0).stop 2023-07-06T03:55:20.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.924+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4073773336 shutdown_connections 2023-07-06T03:55:20.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.924+0000 7fbecf1ff700 1 --2- 172.21.15.97:0/4073773336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb8005200 0x7fbeb800ba10 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.924+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4073773336 >> 172.21.15.97:0/4073773336 conn(0x7fbec8071790 msgr2=0x7fbec8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.924+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4073773336 shutdown_connections 2023-07-06T03:55:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.925+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4073773336 wait complete. 2023-07-06T03:55:20.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.927+0000 7fbecf1ff700 1 Processor -- start 2023-07-06T03:55:20.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.927+0000 7fbecf1ff700 1 -- start start 2023-07-06T03:55:20.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.928+0000 7fbecf1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb8005200 0x7fbec80d2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.928+0000 7fbecf1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbec80cca10 0x7fbec815f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.928+0000 7fbecf1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8155300 0x7fbec80d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.928+0000 7fbecf1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec80d1fe0 con 0x7fbec8155300 2023-07-06T03:55:20.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.928+0000 7fbecf1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec80d16f0 con 0x7fbec80cca10 2023-07-06T03:55:20.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.928+0000 7fbecf1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec80d1990 con 0x7fbeb8005200 2023-07-06T03:55:20.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.929+0000 7fbecd79c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8155300 0x7fbec80d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.929+0000 7fbecd79c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8155300 0x7fbec80d2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57686/0 (socket says 172.21.15.97:57686) 2023-07-06T03:55:20.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.929+0000 7fbecd79c700 1 -- 172.21.15.97:0/4151102361 learned_addr learned my addr 172.21.15.97:0/4151102361 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:20.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.929+0000 7fbeccf9b700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb8005200 0x7fbec80d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.929+0000 7fbec7fff700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbec80cca10 0x7fbec815f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.929+0000 7fbecd79c700 1 -- 172.21.15.97:0/4151102361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb8005200 msgr2=0x7fbec80d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.929+0000 7fbecd79c700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb8005200 0x7fbec80d2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.929+0000 7fbecd79c700 1 -- 172.21.15.97:0/4151102361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbec80cca10 msgr2=0x7fbec815f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:20.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.929+0000 7fbecd79c700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbec80cca10 0x7fbec815f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:20.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.929+0000 7fbecd79c700 1 -- 172.21.15.97:0/4151102361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec80d1c70 con 0x7fbec8155300 2023-07-06T03:55:20.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.929+0000 7fbecd79c700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8155300 0x7fbec80d2b70 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fbec80b1ad0 tx=0x7fbec0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:20.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.930+0000 7fbec57fa700 1 -- 172.21.15.97:0/4151102361 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0000ed0 con 0x7fbec8155300 2023-07-06T03:55:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.930+0000 7fbec57fa700 1 -- 172.21.15.97:0/4151102361 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec0004c20 con 0x7fbec8155300 2023-07-06T03:55:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.930+0000 7fbec57fa700 1 -- 172.21.15.97:0/4151102361 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec00070d0 con 0x7fbec8155300 2023-07-06T03:55:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.930+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4151102361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec815f980 con 0x7fbec8155300 2023-07-06T03:55:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.930+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4151102361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec80d30b0 con 0x7fbec8155300 2023-07-06T03:55:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.931+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4151102361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec8160640 con 0x7fbec8155300 2023-07-06T03:55:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.932+0000 7fbec57fa700 1 -- 172.21.15.97:0/4151102361 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbec0007230 con 0x7fbec8155300 2023-07-06T03:55:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.933+0000 7fbec57fa700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbeac06ace0 0x7fbeac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.933+0000 7fbec57fa700 1 -- 172.21.15.97:0/4151102361 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbec0089890 con 0x7fbec8155300 2023-07-06T03:55:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.933+0000 7fbeccf9b700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbeac06ace0 0x7fbeac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.934+0000 7fbeccf9b700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbeac06ace0 0x7fbeac06d190 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbebc001850 tx=0x7fbebc0017c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:20.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:20.937+0000 7fbec57fa700 1 -- 172.21.15.97:0/4151102361 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbec00549d0 con 0x7fbec8155300 2023-07-06T03:55:21.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.179+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4151102361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbec804e340 con 0x7fbec8155300 2023-07-06T03:55:21.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.179+0000 7fbec57fa700 1 -- 172.21.15.97:0/4151102361 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4679 (secure 0 0 0) 0x7fbec0022020 con 0x7fbec8155300 2023-07-06T03:55:21.181 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:21.181 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3059444241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3059444241},{"type":"v1","addr":"172.21.15.146:6835","nonce":3059444241}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2624688851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2624688851},{"type":"v1","addr":"172.21.15.97:6837","nonce":2624688851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3938376776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3938376776},{"type":"v1","addr":"172.21.15.97:6835","nonce":3938376776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:55:14.806943+0000","modified":"2023-07-06T03:55:18.341811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9099":{"gid":9099,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/3269084599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3269084599},{"type":"v1","addr":"172.21.15.146:6834","nonce":3269084599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-06T03:55:21.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4151102361 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbeac06ace0 msgr2=0x7fbeac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbeac06ace0 0x7fbeac06d190 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbebc001850 tx=0x7fbebc0017c0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4151102361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8155300 msgr2=0x7fbec80d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8155300 0x7fbec80d2b70 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fbec80b1ad0 tx=0x7fbec0001930 comp rx=0 tx=0).stop 2023-07-06T03:55:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4151102361 shutdown_connections 2023-07-06T03:55:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbeac06ace0 0x7fbeac06d190 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8155300 0x7fbec80d2b70 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbeb8005200 0x7fbec80d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 --2- 172.21.15.97:0/4151102361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbec80cca10 0x7fbec815f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4151102361 >> 172.21.15.97:0/4151102361 conn(0x7fbec8071790 msgr2=0x7fbec80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:21.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4151102361 shutdown_connections 2023-07-06T03:55:21.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.182+0000 7fbecf1ff700 1 -- 172.21.15.97:0/4151102361 wait complete. 2023-07-06T03:55:21.185 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 92 2023-07-06T03:55:21.603 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T03:55:14.806943+0000', 'modified': '2023-07-06T03:55:18.341811+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9099}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9099': {'gid': 9099, 'name': 'd', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6834/3269084599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 3269084599}, {'type': 'v1', 'addr': '172.21.15.146:6834', 'nonce': 3269084599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T03:55:21.604 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T03:55:21.604 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-07-06T03:55:21.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.798+0000 7f182c3ff700 1 Processor -- start 2023-07-06T03:55:21.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.798+0000 7f182c3ff700 1 -- start start 2023-07-06T03:55:21.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.799+0000 7f182c3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f182415c610 0x7f18240b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182c3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 0x7f18240b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182c3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18240cff90 con 0x7f18240d2210 2023-07-06T03:55:21.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182c3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18240d0780 con 0x7f182415c610 2023-07-06T03:55:21.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182c3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182415c9e0 con 0x7f182406b360 2023-07-06T03:55:21.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182a19b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f182415c610 0x7f18240b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182a19b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f182415c610 0x7f18240b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57696/0 (socket says 172.21.15.97:57696) 2023-07-06T03:55:21.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182a19b700 1 -- 172.21.15.97:0/2096400790 learned_addr learned my addr 172.21.15.97:0/2096400790 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:21.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182999a700 1 --2- 172.21.15.97:0/2096400790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 0x7f18240b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182a19b700 1 -- 172.21.15.97:0/2096400790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 msgr2=0x7f18240b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182a19b700 1 --2- 172.21.15.97:0/2096400790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 0x7f18240b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182a19b700 1 -- 172.21.15.97:0/2096400790 >> v1:172.21.15.146:6789/0 conn(0x7f18240d2210 legacy=0x7f18240d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182a19b700 1 -- 172.21.15.97:0/2096400790 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18240b5650 con 0x7f182415c610 2023-07-06T03:55:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.800+0000 7f182999a700 1 --2- 172.21.15.97:0/2096400790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 0x7f18240b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.801+0000 7f182a19b700 1 --2- 172.21.15.97:0/2096400790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f182415c610 0x7f18240b47b0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f1820009920 tx=0x7f1820009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4186d157ee19939a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:21.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.801+0000 7f1829199700 1 -- 172.21.15.97:0/2096400790 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182000ba20 con 0x7f182415c610 2023-07-06T03:55:21.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.801+0000 7f1829199700 1 -- 172.21.15.97:0/2096400790 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182000bb80 con 0x7f182415c610 2023-07-06T03:55:21.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.801+0000 7f1829199700 1 -- 172.21.15.97:0/2096400790 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820009e70 con 0x7f182415c610 2023-07-06T03:55:21.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.801+0000 7f182c3ff700 1 -- 172.21.15.97:0/2096400790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f182415c610 msgr2=0x7f18240b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.801+0000 7f182c3ff700 1 --2- 172.21.15.97:0/2096400790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f182415c610 0x7f18240b47b0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f1820009920 tx=0x7f1820009c30 comp rx=0 tx=0).stop 2023-07-06T03:55:21.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.801+0000 7f182c3ff700 1 -- 172.21.15.97:0/2096400790 shutdown_connections 2023-07-06T03:55:21.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.801+0000 7f182c3ff700 1 --2- 172.21.15.97:0/2096400790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f182415c610 0x7f18240b47b0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.801+0000 7f182c3ff700 1 --2- 172.21.15.97:0/2096400790 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 0x7f18240b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.801+0000 7f182c3ff700 1 -- 172.21.15.97:0/2096400790 >> 172.21.15.97:0/2096400790 conn(0x7f182406a6c0 msgr2=0x7f182406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:21.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.802+0000 7f182c3ff700 1 -- 172.21.15.97:0/2096400790 shutdown_connections 2023-07-06T03:55:21.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.802+0000 7f182c3ff700 1 -- 172.21.15.97:0/2096400790 wait complete. 2023-07-06T03:55:21.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.803+0000 7f182c3ff700 1 Processor -- start 2023-07-06T03:55:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.803+0000 7f182c3ff700 1 -- start start 2023-07-06T03:55:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.804+0000 7f182c3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 0x7f18240c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.804+0000 7f182c3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18240d2210 0x7f18240b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.804+0000 7f182c3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f182415c610 0x7f18240b7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.804+0000 7f182c3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c8150 con 0x7f18240d2210 2023-07-06T03:55:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.804+0000 7f182c3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c70b0 con 0x7f182415c610 2023-07-06T03:55:21.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.804+0000 7f182c3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c7390 con 0x7f182406b360 2023-07-06T03:55:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182999a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18240d2210 0x7f18240b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182999a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18240d2210 0x7f18240b7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57702/0 (socket says 172.21.15.97:57702) 2023-07-06T03:55:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182999a700 1 -- 172.21.15.97:0/94043383 learned_addr learned my addr 172.21.15.97:0/94043383 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182999a700 1 -- 172.21.15.97:0/94043383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 msgr2=0x7f18240c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182a19b700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 0x7f18240c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182a99c700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f182415c610 0x7f18240b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182999a700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 0x7f18240c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182999a700 1 -- 172.21.15.97:0/94043383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f182415c610 msgr2=0x7f18240b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182999a700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f182415c610 0x7f18240b7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182999a700 1 -- 172.21.15.97:0/94043383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820009550 con 0x7f18240d2210 2023-07-06T03:55:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182a19b700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 0x7f18240c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182a99c700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f182415c610 0x7f18240b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f182999a700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18240d2210 0x7f18240b7100 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f181400e7e0 tx=0x7f181400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.805+0000 7f181affd700 1 -- 172.21.15.97:0/94043383 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814004230 con 0x7f18240d2210 2023-07-06T03:55:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.806+0000 7f181affd700 1 -- 172.21.15.97:0/94043383 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1814004390 con 0x7f18240d2210 2023-07-06T03:55:21.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.806+0000 7f182c3ff700 1 -- 172.21.15.97:0/94043383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18240c7670 con 0x7f18240d2210 2023-07-06T03:55:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.806+0000 7f181affd700 1 -- 172.21.15.97:0/94043383 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18140036a0 con 0x7f18240d2210 2023-07-06T03:55:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.806+0000 7f182c3ff700 1 -- 172.21.15.97:0/94043383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18240c7c30 con 0x7f18240d2210 2023-07-06T03:55:21.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.808+0000 7f181affd700 1 -- 172.21.15.97:0/94043383 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1814003800 con 0x7f18240d2210 2023-07-06T03:55:21.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.809+0000 7f181affd700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f180806ace0 0x7f180806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.810+0000 7f181affd700 1 -- 172.21.15.97:0/94043383 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1814088f70 con 0x7f18240d2210 2023-07-06T03:55:21.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.810+0000 7f182a19b700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f180806ace0 0x7f180806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.810+0000 7f182a19b700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f180806ace0 0x7f180806d190 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1820014040 tx=0x7f1820008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.811+0000 7f183fcbdb80 1 Processor -- start 2023-07-06T03:55:21.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.811+0000 7f183fcbdb80 1 -- start start 2023-07-06T03:55:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.812+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fcad50 0x55db90fcb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.812+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fd0190 0x55db90feea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.812+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fa8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.812+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db90f1cea0 con 0x55db90fa5b50 2023-07-06T03:55:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.812+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db90e90600 con 0x55db90fcad50 2023-07-06T03:55:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.812+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db90eff910 con 0x55db90fd0190 2023-07-06T03:55:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f182a99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fa8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f182a99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fa8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57710/0 (socket says 172.21.15.97:57710) 2023-07-06T03:55:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f182a99c700 1 -- 172.21.15.97:0/3747785344 learned_addr learned my addr 172.21.15.97:0/3747785344 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f182999a700 1 --2- 172.21.15.97:0/3747785344 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fd0190 0x55db90feea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f182a19b700 1 --2- 172.21.15.97:0/3747785344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fcad50 0x55db90fcb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f182a99c700 1 -- 172.21.15.97:0/3747785344 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fd0190 msgr2=0x55db90feea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f182a99c700 1 --2- 172.21.15.97:0/3747785344 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fd0190 0x55db90feea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f182a99c700 1 -- 172.21.15.97:0/3747785344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fcad50 msgr2=0x55db90fcb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f182a99c700 1 --2- 172.21.15.97:0/3747785344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fcad50 0x55db90fcb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f182a99c700 1 -- 172.21.15.97:0/3747785344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db90cec790 con 0x55db90fa5b50 2023-07-06T03:55:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f182a99c700 1 --2- 172.21.15.97:0/3747785344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fa8110 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f181c0021b0 tx=0x7f181c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfd09a233167307 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:21.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f1807fff700 1 -- 172.21.15.97:0/3747785344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00b070 con 0x55db90fa5b50 2023-07-06T03:55:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f1807fff700 1 -- 172.21.15.97:0/3747785344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c003e30 con 0x55db90fa5b50 2023-07-06T03:55:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.813+0000 7f1807fff700 1 -- 172.21.15.97:0/3747785344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00dc30 con 0x55db90fa5b50 2023-07-06T03:55:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.814+0000 7f183fcbdb80 1 -- 172.21.15.97:0/3747785344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 msgr2=0x55db90fa8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.814+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/3747785344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fa8110 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f181c0021b0 tx=0x7f181c00d430 comp rx=0 tx=0).stop 2023-07-06T03:55:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.814+0000 7f183fcbdb80 1 -- 172.21.15.97:0/3747785344 shutdown_connections 2023-07-06T03:55:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.814+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/3747785344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fa8110 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.814+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/3747785344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fcad50 0x55db90fcb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.814+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/3747785344 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fd0190 0x55db90feea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.814+0000 7f183fcbdb80 1 -- 172.21.15.97:0/3747785344 >> 172.21.15.97:0/3747785344 conn(0x55db90f9d310 msgr2=0x55db90fed5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:21.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.814+0000 7f183fcbdb80 1 -- 172.21.15.97:0/3747785344 shutdown_connections 2023-07-06T03:55:21.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.814+0000 7f183fcbdb80 1 -- 172.21.15.97:0/3747785344 wait complete. 2023-07-06T03:55:21.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.815+0000 7f183fcbdb80 1 -- start start 2023-07-06T03:55:21.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.815+0000 7f183fcbdb80 1 Processor -- start 2023-07-06T03:55:21.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.817+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fa5b50 0x55db90fbfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.817+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fcad50 0x55db90fe4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.817+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fd0190 0x55db90fe4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.817+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db90ef8be0 con 0x55db90fcad50 2023-07-06T03:55:21.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.817+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db90eb65b0 con 0x55db90fd0190 2023-07-06T03:55:21.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.817+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db90e9c910 con 0x55db90fa5b50 2023-07-06T03:55:21.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.817+0000 7f183fcbdb80 10 client.0 _ll_register_callbacks cb 0x55db90eb4b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:55:21.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.817+0000 7f182999a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fcad50 0x55db90fe4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.817+0000 7f182999a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fcad50 0x55db90fe4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57718/0 (socket says 172.21.15.97:57718) 2023-07-06T03:55:21.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.817+0000 7f182999a700 1 -- 172.21.15.97:0/2847779916 learned_addr learned my addr 172.21.15.97:0/2847779916 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:21.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f182a99c700 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fd0190 0x55db90fe4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f182999a700 1 -- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fa5b50 msgr2=0x55db90fbfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:21.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f182999a700 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fa5b50 0x55db90fbfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f182999a700 1 -- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fd0190 msgr2=0x55db90fe4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:21.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f182999a700 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fd0190 0x55db90fe4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f182999a700 1 -- 172.21.15.97:0/2847779916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18240cff90 con 0x55db90fcad50 2023-07-06T03:55:21.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f182a99c700 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fd0190 0x55db90fe4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:21.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f182999a700 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fcad50 0x55db90fe4820 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f1814057c60 tx=0x7f18140574a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860cce9f4d716c01 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:21.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f18057fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:55:21.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f18057fa700 1 -- 172.21.15.97:0/2847779916 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181406f070 con 0x55db90fcad50 2023-07-06T03:55:21.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f18057fa700 1 -- 172.21.15.97:0/2847779916 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f181406f3d0 con 0x55db90fcad50 2023-07-06T03:55:21.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f183fcbdb80 1 -- 172.21.15.97:0/2847779916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db90f9a920 con 0x55db90fcad50 2023-07-06T03:55:21.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f18057fa700 1 -- 172.21.15.97:0/2847779916 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1814053e50 con 0x55db90fcad50 2023-07-06T03:55:21.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f183fcbdb80 10 client.9246 fetch_fsmap learned FSMap version 92 2023-07-06T03:55:21.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f183fcbdb80 1 -- 172.21.15.97:0/2847779916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x55db90dd90a0 con 0x55db90fcad50 2023-07-06T03:55:21.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.818+0000 7f18057fa700 1 -- 172.21.15.97:0/2847779916 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55db90f9a920 con 0x55db90fcad50 2023-07-06T03:55:21.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.819+0000 7f18057fa700 1 -- 172.21.15.97:0/2847779916 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 92) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1814056c30 con 0x55db90fcad50 2023-07-06T03:55:21.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.819+0000 7f183fcbdb80 10 client.9246 fetch_fsmap finished waiting for FSMap version 92 2023-07-06T03:55:21.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.819+0000 7f183fcbdb80 10 client.9246 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T03:55:21.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.819+0000 7f183fcbdb80 20 client.9246 populate_metadata read hostname 'smithi097' 2023-07-06T03:55:21.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.819+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:6832/3269084599,v1:172.21.15.146:6834/3269084599] conn(0x55db90fc40d0 0x55db9100a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.819+0000 7f183fcbdb80 4 client.9246 mds_command: new command op to 9099 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T03:55:21.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.819+0000 7f183fcbdb80 1 -- 172.21.15.97:0/2847779916 --> [v2:172.21.15.146:6832/3269084599,v1:172.21.15.146:6834/3269084599] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db90f770d0 con 0x55db90fc40d0 2023-07-06T03:55:21.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.819+0000 7f182a19b700 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:6832/3269084599,v1:172.21.15.146:6834/3269084599] conn(0x55db90fc40d0 0x55db9100a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.820+0000 7f18057fa700 10 client.9246 ms_handle_connect on v2:172.21.15.146:6832/3269084599 2023-07-06T03:55:21.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.820+0000 7f182a19b700 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:6832/3269084599,v1:172.21.15.146:6834/3269084599] conn(0x55db90fc40d0 0x55db9100a0a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=800bc505093ad155 server_cookie=a827db1000a13c4d in_seq=0 out_seq=0 2023-07-06T03:55:21.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f18057fa700 1 -- 172.21.15.97:0/2847779916 <== mds.0 v2:172.21.15.146:6832/3269084599 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55db90f770d0 con 0x55db90fc40d0 2023-07-06T03:55:21.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f18057fa700 10 client.9246 handle_command_reply: tid=0 2023-07-06T03:55:21.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 1 client.9246 shutdown 2023-07-06T03:55:21.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 1 -- 172.21.15.97:0/2847779916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fcad50 msgr2=0x55db90fe4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fcad50 0x55db90fe4820 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f1814057c60 tx=0x7f18140574a0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 1 -- 172.21.15.97:0/2847779916 shutdown_connections 2023-07-06T03:55:21.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:6832/3269084599,v1:172.21.15.146:6834/3269084599] conn(0x55db90fc40d0 0x55db9100a0a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fd0190 0x55db90fe4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fa5b50 0x55db90fbfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f18057fa700 0 client.9246 ms_handle_reset on v2:172.21.15.146:6832/3269084599 2023-07-06T03:55:21.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/2847779916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fcad50 0x55db90fe4820 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 1 -- 172.21.15.97:0/2847779916 >> 172.21.15.97:0/2847779916 conn(0x55db90f9d310 msgr2=0x55db90fa1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:21.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 1 -- 172.21.15.97:0/2847779916 shutdown_connections 2023-07-06T03:55:21.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 1 -- 172.21.15.97:0/2847779916 wait complete. 2023-07-06T03:55:21.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.821+0000 7f183fcbdb80 20 client.9246 trim_cache size 0 max 16384 2023-07-06T03:55:21.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.839+0000 7f183fcbdb80 1 Processor -- start 2023-07-06T03:55:21.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.839+0000 7f183fcbdb80 1 -- start start 2023-07-06T03:55:21.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.840+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fa5b50 0x55db90fa8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.840+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fcad50 0x55db9100e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.840+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fd0190 0x55db90fc9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.840+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db90ced390 con 0x55db90fd0190 2023-07-06T03:55:21.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.840+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db90fc5fa0 con 0x55db90fa5b50 2023-07-06T03:55:21.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.840+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db90fd6830 con 0x55db90fcad50 2023-07-06T03:55:21.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.840+0000 7f182a99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fd0190 0x55db90fc9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f182a99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fd0190 0x55db90fc9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57730/0 (socket says 172.21.15.97:57730) 2023-07-06T03:55:21.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f182a99c700 1 -- 172.21.15.97:0/1509484929 learned_addr learned my addr 172.21.15.97:0/1509484929 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:21.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f182a19b700 1 --2- 172.21.15.97:0/1509484929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fa5b50 0x55db90fa8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f182999a700 1 --2- 172.21.15.97:0/1509484929 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fcad50 0x55db9100e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f182a99c700 1 -- 172.21.15.97:0/1509484929 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fcad50 msgr2=0x55db9100e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:21.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f182a99c700 1 --2- 172.21.15.97:0/1509484929 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fcad50 0x55db9100e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f182a99c700 1 -- 172.21.15.97:0/1509484929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fa5b50 msgr2=0x55db90fa8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:21.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f182a99c700 1 --2- 172.21.15.97:0/1509484929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fa5b50 0x55db90fa8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f182a99c700 1 -- 172.21.15.97:0/1509484929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db90ebeb10 con 0x55db90fd0190 2023-07-06T03:55:21.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f182999a700 1 --2- 172.21.15.97:0/1509484929 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fcad50 0x55db9100e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:21.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f182a99c700 1 --2- 172.21.15.97:0/1509484929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fd0190 0x55db90fc9300 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f181c00d780 tx=0x7f181c00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57812b2e6470a74c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:21.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f1818bff700 1 -- 172.21.15.97:0/1509484929 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c003c60 con 0x55db90fd0190 2023-07-06T03:55:21.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f1818bff700 1 -- 172.21.15.97:0/1509484929 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c00da30 con 0x55db90fd0190 2023-07-06T03:55:21.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.841+0000 7f1818bff700 1 -- 172.21.15.97:0/1509484929 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00de20 con 0x55db90fd0190 2023-07-06T03:55:21.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.842+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1509484929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fd0190 msgr2=0x55db90fc9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.842+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/1509484929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fd0190 0x55db90fc9300 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f181c00d780 tx=0x7f181c00b000 comp rx=0 tx=0).stop 2023-07-06T03:55:21.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.842+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1509484929 shutdown_connections 2023-07-06T03:55:21.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.842+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/1509484929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fd0190 0x55db90fc9300 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.842+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/1509484929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fa5b50 0x55db90fa8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.842+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/1509484929 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fcad50 0x55db9100e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.842+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1509484929 >> 172.21.15.97:0/1509484929 conn(0x55db90f9d310 msgr2=0x55db90fc15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:21.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.842+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1509484929 shutdown_connections 2023-07-06T03:55:21.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.842+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1509484929 wait complete. 2023-07-06T03:55:21.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.843+0000 7f183fcbdb80 1 -- start start 2023-07-06T03:55:21.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.843+0000 7f183fcbdb80 1 Processor -- start 2023-07-06T03:55:21.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fe5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fcad50 0x55db90fa7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f183fcbdb80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fd0190 0x55db90fc81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db91044250 con 0x55db90fa5b50 2023-07-06T03:55:21.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db91043a80 con 0x55db90fd0190 2023-07-06T03:55:21.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f183fcbdb80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db91048bf0 con 0x55db90fcad50 2023-07-06T03:55:21.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f183fcbdb80 10 client.0 _ll_register_callbacks cb 0x55db90eb4b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:55:21.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f182a19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fe5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f182a19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fe5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57744/0 (socket says 172.21.15.97:57744) 2023-07-06T03:55:21.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f182a19b700 1 -- 172.21.15.97:0/1271224259 learned_addr learned my addr 172.21.15.97:0/1271224259 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f182a19b700 1 -- 172.21.15.97:0/1271224259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fcad50 msgr2=0x55db90fa7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f182a19b700 1 --2- 172.21.15.97:0/1271224259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fcad50 0x55db90fa7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f182a19b700 1 -- 172.21.15.97:0/1271224259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fd0190 msgr2=0x55db90fc81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f182a19b700 1 --2- 172.21.15.97:0/1271224259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fd0190 0x55db90fc81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.844+0000 7f182a19b700 1 -- 172.21.15.97:0/1271224259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db90cec790 con 0x55db90fa5b50 2023-07-06T03:55:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.845+0000 7f182a19b700 1 --2- 172.21.15.97:0/1271224259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fe5e60 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f182000bee0 tx=0x7f18200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7379a1fe9f4bcf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.845+0000 7f18057fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:55:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.845+0000 7f18057fa700 1 -- 172.21.15.97:0/1271224259 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182002f520 con 0x55db90fa5b50 2023-07-06T03:55:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.845+0000 7f18057fa700 1 -- 172.21.15.97:0/1271224259 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f182002c050 con 0x55db90fa5b50 2023-07-06T03:55:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.845+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1271224259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f181c003e30 con 0x55db90fa5b50 2023-07-06T03:55:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.845+0000 7f18057fa700 1 -- 172.21.15.97:0/1271224259 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f182002f8d0 con 0x55db90fa5b50 2023-07-06T03:55:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.845+0000 7f183fcbdb80 10 client.9252 fetch_fsmap learned FSMap version 92 2023-07-06T03:55:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.845+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1271224259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x55db90d4d1d0 con 0x55db90fa5b50 2023-07-06T03:55:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.845+0000 7f18057fa700 1 -- 172.21.15.97:0/1271224259 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c003e30 con 0x55db90fa5b50 2023-07-06T03:55:21.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.846+0000 7f18057fa700 1 -- 172.21.15.97:0/1271224259 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 92) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f182002fa30 con 0x55db90fa5b50 2023-07-06T03:55:21.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.846+0000 7f183fcbdb80 10 client.9252 fetch_fsmap finished waiting for FSMap version 92 2023-07-06T03:55:21.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.846+0000 7f183fcbdb80 10 client.9252 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T03:55:21.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.846+0000 7f183fcbdb80 20 client.9252 populate_metadata read hostname 'smithi097' 2023-07-06T03:55:21.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.846+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/1271224259 >> [v2:172.21.15.146:6832/3269084599,v1:172.21.15.146:6834/3269084599] conn(0x55db91057ee0 0x55db910582d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:21.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.846+0000 7f183fcbdb80 4 client.9252 mds_command: new command op to 9099 tid=0[{"prefix": "status"}] 2023-07-06T03:55:21.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.846+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1271224259 --> [v2:172.21.15.146:6832/3269084599,v1:172.21.15.146:6834/3269084599] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55db90ef1d20 con 0x55db91057ee0 2023-07-06T03:55:21.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.846+0000 7f182999a700 1 --2- 172.21.15.97:0/1271224259 >> [v2:172.21.15.146:6832/3269084599,v1:172.21.15.146:6834/3269084599] conn(0x55db91057ee0 0x55db910582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:21.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f18057fa700 10 client.9252 ms_handle_connect on v2:172.21.15.146:6832/3269084599 2023-07-06T03:55:21.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f182999a700 1 --2- 172.21.15.97:0/1271224259 >> [v2:172.21.15.146:6832/3269084599,v1:172.21.15.146:6834/3269084599] conn(0x55db91057ee0 0x55db910582d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4723cc0ae1c3aa1a server_cookie=6b55544eb0d04088 in_seq=0 out_seq=0 2023-07-06T03:55:21.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f18057fa700 1 -- 172.21.15.97:0/1271224259 <== mds.0 v2:172.21.15.146:6832/3269084599 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55db90ef1d20 con 0x55db91057ee0 2023-07-06T03:55:21.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f18057fa700 10 client.9252 handle_command_reply: tid=0 2023-07-06T03:55:21.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f183fcbdb80 1 client.9252 shutdown 2023-07-06T03:55:21.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1271224259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 msgr2=0x55db90fe5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/1271224259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fe5e60 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f182000bee0 tx=0x7f18200047c0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1271224259 shutdown_connections 2023-07-06T03:55:21.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/1271224259 >> [v2:172.21.15.146:6832/3269084599,v1:172.21.15.146:6834/3269084599] conn(0x55db91057ee0 0x55db910582d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/1271224259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55db90fd0190 0x55db90fc81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/1271224259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db90fa5b50 0x55db90fe5e60 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f18057fa700 0 client.9252 ms_handle_reset on v2:172.21.15.146:6832/3269084599 2023-07-06T03:55:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f183fcbdb80 1 --2- 172.21.15.97:0/1271224259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x55db90fcad50 0x55db90fa7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1271224259 >> 172.21.15.97:0/1271224259 conn(0x55db90f9d310 msgr2=0x55db90fa86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1271224259 shutdown_connections 2023-07-06T03:55:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.847+0000 7f183fcbdb80 1 -- 172.21.15.97:0/1271224259 wait complete. 2023-07-06T03:55:21.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f183fcbdb80 20 client.9252 trim_cache size 0 max 16384 2023-07-06T03:55:21.858 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:55:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36", 2023-07-06T03:55:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-07-06T03:55:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9099, 2023-07-06T03:55:21.858 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-07-06T03:55:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-07-06T03:55:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-07-06T03:55:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.5169885829999998, 2023-07-06T03:55:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 92, 2023-07-06T03:55:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 124, 2023-07-06T03:55:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 124, 2023-07-06T03:55:21.859 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 13.047385121 2023-07-06T03:55:21.860 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:55:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 -- 172.21.15.97:0/94043383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f180806ace0 msgr2=0x7f180806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f180806ace0 0x7f180806d190 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1820014040 tx=0x7f1820008030 comp rx=0 tx=0).stop 2023-07-06T03:55:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 -- 172.21.15.97:0/94043383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18240d2210 msgr2=0x7f18240b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18240d2210 0x7f18240b7100 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f181400e7e0 tx=0x7f181400eba0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 -- 172.21.15.97:0/94043383 shutdown_connections 2023-07-06T03:55:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f180806ace0 0x7f180806d190 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f182415c610 0x7f18240b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f182406b360 0x7f18240c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 --2- 172.21.15.97:0/94043383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18240d2210 0x7f18240b7100 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 -- 172.21.15.97:0/94043383 >> 172.21.15.97:0/94043383 conn(0x7f182406a6c0 msgr2=0x7f182415b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 -- 172.21.15.97:0/94043383 shutdown_connections 2023-07-06T03:55:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:21.848+0000 7f182c3ff700 1 -- 172.21.15.97:0/94043383 wait complete. 2023-07-06T03:55:21.867 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:55:22.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.061+0000 7f0899dff700 1 Processor -- start 2023-07-06T03:55:22.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.061+0000 7f0899dff700 1 -- start start 2023-07-06T03:55:22.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.062+0000 7f0899dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08940cd390 0x7f0894160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:22.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.062+0000 7f0899dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0894160dd0 0x7f08940b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:22.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.062+0000 7f0899dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08940cdae0 con 0x7f089415c520 2023-07-06T03:55:22.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.062+0000 7f0899dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08940b6640 con 0x7f0894160dd0 2023-07-06T03:55:22.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.062+0000 7f0899dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08940b6920 con 0x7f08940cd390 2023-07-06T03:55:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.062+0000 7f08937fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f089415c520 0x7f0894160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53520/0 (socket says 172.21.15.97:53520) 2023-07-06T03:55:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.062+0000 7f08937fe700 1 -- 172.21.15.97:0/1597315766 learned_addr learned my addr 172.21.15.97:0/1597315766 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:22.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.063+0000 7f0892ffd700 1 --2- 172.21.15.97:0/1597315766 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08940cd390 0x7f0894160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:22.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.063+0000 7f0893fff700 1 --2- 172.21.15.97:0/1597315766 >> v2:172.21.15.146:3300/0 conn(0x7f0894160dd0 0x7f08940b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:22.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.063+0000 7f08927fc700 1 -- 172.21.15.97:0/1597315766 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895039031 0 0) 0x7f08940cdae0 con 0x7f089415c520 2023-07-06T03:55:22.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.063+0000 7f08927fc700 1 -- 172.21.15.97:0/1597315766 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f087c0035b0 con 0x7f089415c520 2023-07-06T03:55:22.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.063+0000 7f08927fc700 1 -- 172.21.15.97:0/1597315766 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4033319372 0 0) 0x7f087c0035b0 con 0x7f089415c520 2023-07-06T03:55:22.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.063+0000 7f0892ffd700 1 -- 172.21.15.97:0/1597315766 >> v2:172.21.15.146:3300/0 conn(0x7f0894160dd0 msgr2=0x7f08940b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:22.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.063+0000 7f0892ffd700 1 --2- 172.21.15.97:0/1597315766 >> v2:172.21.15.146:3300/0 conn(0x7f0894160dd0 0x7f08940b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:22.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.063+0000 7f0892ffd700 1 -- 172.21.15.97:0/1597315766 >> v1:172.21.15.97:6789/0 conn(0x7f089415c520 legacy=0x7f0894160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:22.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.063+0000 7f0892ffd700 1 -- 172.21.15.97:0/1597315766 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08940b6c00 con 0x7f08940cd390 2023-07-06T03:55:22.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.063+0000 7f0893fff700 1 --2- 172.21.15.97:0/1597315766 >> v2:172.21.15.146:3300/0 conn(0x7f0894160dd0 0x7f08940b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:55:22.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.063+0000 7f0892ffd700 1 --2- 172.21.15.97:0/1597315766 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08940cd390 0x7f0894160890 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0888009840 tx=0x7f0888009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df60c06b148866a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:22.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.064+0000 7f08927fc700 1 -- 172.21.15.97:0/1597315766 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884001670 con 0x7f08940cd390 2023-07-06T03:55:22.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.064+0000 7f08927fc700 1 -- 172.21.15.97:0/1597315766 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0888004030 con 0x7f08940cd390 2023-07-06T03:55:22.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.064+0000 7f08927fc700 1 -- 172.21.15.97:0/1597315766 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08880043a0 con 0x7f08940cd390 2023-07-06T03:55:22.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.064+0000 7f0899dff700 1 -- 172.21.15.97:0/1597315766 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08940cd390 msgr2=0x7f0894160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:22.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.064+0000 7f0899dff700 1 --2- 172.21.15.97:0/1597315766 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08940cd390 0x7f0894160890 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0888009840 tx=0x7f0888009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:22.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.064+0000 7f0899dff700 1 -- 172.21.15.97:0/1597315766 shutdown_connections 2023-07-06T03:55:22.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.064+0000 7f0899dff700 1 --2- 172.21.15.97:0/1597315766 >> v2:172.21.15.146:3300/0 conn(0x7f0894160dd0 0x7f08940b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:22.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.064+0000 7f0899dff700 1 --2- 172.21.15.97:0/1597315766 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08940cd390 0x7f0894160890 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:22.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.064+0000 7f0899dff700 1 -- 172.21.15.97:0/1597315766 >> 172.21.15.97:0/1597315766 conn(0x7f0894071790 msgr2=0x7f0894071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:22.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.064+0000 7f0899dff700 1 -- 172.21.15.97:0/1597315766 shutdown_connections 2023-07-06T03:55:22.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.064+0000 7f0899dff700 1 -- 172.21.15.97:0/1597315766 wait complete. 2023-07-06T03:55:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.066+0000 7f0899dff700 1 Processor -- start 2023-07-06T03:55:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.066+0000 7f0899dff700 1 -- start start 2023-07-06T03:55:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.067+0000 7f0899dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08940cd390 0x7f089415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.067+0000 7f0899dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089415c520 0x7f08940c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.067+0000 7f0899dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0894160dd0 0x7f0894159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.067+0000 7f0899dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089415b500 con 0x7f089415c520 2023-07-06T03:55:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.067+0000 7f0899dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08940ca5f0 con 0x7f0894160dd0 2023-07-06T03:55:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.067+0000 7f0899dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08940d16a0 con 0x7f08940cd390 2023-07-06T03:55:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.067+0000 7f0892ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089415c520 0x7f08940c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.067+0000 7f0892ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089415c520 0x7f08940c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57754/0 (socket says 172.21.15.97:57754) 2023-07-06T03:55:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f0892ffd700 1 -- 172.21.15.97:0/1083245850 learned_addr learned my addr 172.21.15.97:0/1083245850 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f08937fe700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08940cd390 0x7f089415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f0893fff700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0894160dd0 0x7f0894159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f0892ffd700 1 -- 172.21.15.97:0/1083245850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08940cd390 msgr2=0x7f089415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f0892ffd700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08940cd390 0x7f089415afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f0892ffd700 1 -- 172.21.15.97:0/1083245850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0894160dd0 msgr2=0x7f0894159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f0892ffd700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0894160dd0 0x7f0894159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f0892ffd700 1 -- 172.21.15.97:0/1083245850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c009a70 con 0x7f089415c520 2023-07-06T03:55:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f0892ffd700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089415c520 0x7f08940c9e10 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f08880090b0 tx=0x7f0888004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f087bfff700 1 -- 172.21.15.97:0/1083245850 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888003d40 con 0x7f089415c520 2023-07-06T03:55:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f087bfff700 1 -- 172.21.15.97:0/1083245850 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088801f070 con 0x7f089415c520 2023-07-06T03:55:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f0899dff700 1 -- 172.21.15.97:0/1083245850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08880094a0 con 0x7f089415c520 2023-07-06T03:55:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.068+0000 7f087bfff700 1 -- 172.21.15.97:0/1083245850 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888003ea0 con 0x7f089415c520 2023-07-06T03:55:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.069+0000 7f0899dff700 1 -- 172.21.15.97:0/1083245850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08940d0d70 con 0x7f089415c520 2023-07-06T03:55:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.069+0000 7f0899dff700 1 -- 172.21.15.97:0/1083245850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08940d1850 con 0x7f089415c520 2023-07-06T03:55:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.071+0000 7f087bfff700 1 -- 172.21.15.97:0/1083245850 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0888021030 con 0x7f089415c520 2023-07-06T03:55:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.071+0000 7f087bfff700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088006ace0 0x7f088006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.072+0000 7f087bfff700 1 -- 172.21.15.97:0/1083245850 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f088808a780 con 0x7f089415c520 2023-07-06T03:55:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.072+0000 7f08937fe700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088006ace0 0x7f088006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.072+0000 7f08937fe700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088006ace0 0x7f088006d190 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f089414d8e0 tx=0x7f0884001da0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.076+0000 7f087bfff700 1 -- 172.21.15.97:0/1083245850 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f08940d1850 con 0x7f089415c520 2023-07-06T03:55:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.311+0000 7f0899dff700 1 -- 172.21.15.97:0/1083245850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f08940d1810 con 0x7f089415c520 2023-07-06T03:55:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.312+0000 7f087bfff700 1 -- 172.21.15.97:0/1083245850 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0888055940 con 0x7f089415c520 2023-07-06T03:55:22.314 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:55:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 -- 172.21.15.97:0/1083245850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088006ace0 msgr2=0x7f088006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088006ace0 0x7f088006d190 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f089414d8e0 tx=0x7f0884001da0 comp rx=0 tx=0).stop 2023-07-06T03:55:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 -- 172.21.15.97:0/1083245850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089415c520 msgr2=0x7f08940c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089415c520 0x7f08940c9e10 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f08880090b0 tx=0x7f0888004750 comp rx=0 tx=0).stop 2023-07-06T03:55:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 -- 172.21.15.97:0/1083245850 shutdown_connections 2023-07-06T03:55:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f088006ace0 0x7f088006d190 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:22.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0894160dd0 0x7f0894159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08940cd390 0x7f089415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 --2- 172.21.15.97:0/1083245850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f089415c520 0x7f08940c9e10 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 -- 172.21.15.97:0/1083245850 >> 172.21.15.97:0/1083245850 conn(0x7f0894071790 msgr2=0x7f08940b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 -- 172.21.15.97:0/1083245850 shutdown_connections 2023-07-06T03:55:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:22.314+0000 7f0899dff700 1 -- 172.21.15.97:0/1083245850 wait complete. 2023-07-06T03:55:22.334 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:55:22.334 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:55:22.334 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:55:22.335 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:55:22.335 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:55:22.335 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:55:22.335 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:55:22.336 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:55:22.336 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:55:22.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:22.337 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:55:22.383 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:55:22.383 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:55:22.383 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:55:22.383 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:22.383 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:55:22.384 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:22.384 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:55:22.384 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:55:22.384 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:55:22.384 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:55:22.384 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:55:22.384 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:55:22.385 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:55:22.385 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:55:22.385 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:55:22.385 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:55:22.385 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:55:22.385 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41546sec preferred_lft 41546sec 2023-07-06T03:55:22.385 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:55:22.385 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:22.386 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:55:22.386 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:55:22.386 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:55:22.386 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:22.386 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:55:22.386 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:22.386 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:55:22.386 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:55:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:55:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:22.387 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:55:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:55:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:55:22.387 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:22.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:22.389 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:55:22.455 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:55:22.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:22.458 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:55:22.506 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:55:22.506 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:55:22.506 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:55:22.506 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:55:22.507 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:55:22.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:22.508 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:55:22.508 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:55:22.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:55:22.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:55:22.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:55:22.510 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:55:22.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:22.677 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:55:22.723 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:55:22.724 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:55:22.725 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:55:22.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:22.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:55:22.794 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:55:22.795 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:55:22.795 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:55:22.840 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:55:22.841 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:55:22.841 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:55:22.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:55:23.035 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:55:23.036 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:55:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:55:23.105 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:55:23.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:23.106 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:55:23.151 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:55:23.151 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:55:23.195 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:55:23.264 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:55:23.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:55:23.333 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:55:23.335 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:55:23.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:23.336 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:55:23.363 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:55:23.361+0000 7f93478335c0 -1 init, newargv = 0x5577a40320c0 newargc=15 2023-07-06T03:55:23.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[163390]: starting ceph client 2023-07-06T03:55:23.372 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[163390]: starting fuse 2023-07-06T03:55:23.387 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:55:23.388 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:55:23.388 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:55:23.388 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:55:23.389 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:55:23.389 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:55:23.389 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:55:23.390 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:23.390 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:55:23.390 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:55:23.391 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:55:23.391 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:55:23.391 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:23.392 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:55:23.392 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:55:23.392 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:55:23.393 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:55:23.393 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:23.393 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:55:23.393 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:55:23.394 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:55:23.394 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:55:23.394 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:55:23.395 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:55:23.395 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:55:23.395 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:55:23.396 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:23.396 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:55:23.396 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:55:23.570 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.163390.asok 2023-07-06T03:55:23.749 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.163390.asok 2023-07-06T03:55:23.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:23.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.163390.asok status 2023-07-06T03:55:23.973 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:55:23.973 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:55:23.973 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:55:23.973 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:55:23.974 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:55:23.974 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:55:23.974 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:55:23.974 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "163390", 2023-07-06T03:55:23.974 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:55:23.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:23.974 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:55:23.975 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:55:23.975 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9264, 2023-07-06T03:55:23.975 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:55:23.975 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:55:23.975 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:55:23.975 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 9264 2023-07-06T03:55:23.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:23.975 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:55:23.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:55:23.976 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:55:23.976 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1280308817 2023-07-06T03:55:23.976 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:23.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:23.976 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:55:23.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:55:23.977 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:55:23.977 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1280308817 2023-07-06T03:55:23.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:23.977 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.9264 192.168.0.1:0/1280308817", 2023-07-06T03:55:23.977 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/1280308817", 2023-07-06T03:55:23.977 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:55:23.977 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 92, 2023-07-06T03:55:23.977 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 124, 2023-07-06T03:55:23.978 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:55:23.978 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:55:23.978 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:55:23.978 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:55:23.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:23.987 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:55:23.988 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:55:24.034 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:55:24.035 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:55:24.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:24.036 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:55:24.106 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:55:24.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.298+0000 7f6ad83ff700 1 Processor -- start 2023-07-06T03:55:24.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.298+0000 7f6ad83ff700 1 -- start start 2023-07-06T03:55:24.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.299+0000 7f6ad83ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6ad00cd390 0x7f6ad0160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:24.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.299+0000 7f6ad83ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad00cdae0 con 0x7f6ad0160dd0 2023-07-06T03:55:24.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.299+0000 7f6ad83ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad00b6640 con 0x7f6ad015c520 2023-07-06T03:55:24.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.299+0000 7f6ad83ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad00b6920 con 0x7f6ad00cd390 2023-07-06T03:55:24.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad699c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6ad0160dd0 0x7f6ad00b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53522/0 (socket says 172.21.15.97:53522) 2023-07-06T03:55:24.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad699c700 1 -- 172.21.15.97:0/1768025706 learned_addr learned my addr 172.21.15.97:0/1768025706 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:24.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad599a700 1 --2- 172.21.15.97:0/1768025706 >> v2:172.21.15.146:3300/0 conn(0x7f6ad00cd390 0x7f6ad0160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:24.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519858270 0 0) 0x7f6ad00cdae0 con 0x7f6ad0160dd0 2023-07-06T03:55:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6abc0035b0 con 0x7f6ad0160dd0 2023-07-06T03:55:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 139798973 0 0) 0x7f6abc0035b0 con 0x7f6ad0160dd0 2023-07-06T03:55:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 >> v2:172.21.15.146:3300/0 conn(0x7f6ad00cd390 msgr2=0x7f6ad0160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad5199700 1 --2- 172.21.15.97:0/1768025706 >> v2:172.21.15.146:3300/0 conn(0x7f6ad00cd390 0x7f6ad0160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 >> v1:172.21.15.146:6789/0 conn(0x7f6ad015c520 legacy=0x7f6ad0160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad00b6c00 con 0x7f6ad0160dd0 2023-07-06T03:55:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f6ac80019f0 con 0x7f6ad0160dd0 2023-07-06T03:55:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 >> v1:172.21.15.97:6789/0 conn(0x7f6ad0160dd0 legacy=0x7f6ad00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.300+0000 7f6ad5199700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 0x7f6abc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad5199700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc004850 0x7f6abc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad5199700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc005200 0x7f6abc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad00b6920 con 0x7f6abc003a00 2023-07-06T03:55:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc00c080 con 0x7f6abc005200 2023-07-06T03:55:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc00c240 con 0x7f6abc004850 2023-07-06T03:55:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad619b700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 0x7f6abc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad699c700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc005200 0x7f6abc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad599a700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc004850 0x7f6abc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad619b700 1 -- 172.21.15.97:0/1768025706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc004850 msgr2=0x7f6abc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad619b700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6abc004850 0x7f6abc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad619b700 1 -- 172.21.15.97:0/1768025706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc005200 msgr2=0x7f6abc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad619b700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc005200 0x7f6abc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad619b700 1 -- 172.21.15.97:0/1768025706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc0035b0 con 0x7f6abc003a00 2023-07-06T03:55:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad699c700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc005200 0x7f6abc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:24.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad699c700 1 -- 172.21.15.97:0/1768025706 reap_dead start 2023-07-06T03:55:24.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.301+0000 7f6ad619b700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 0x7f6abc003e10 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0017c0 tx=0x7f6abc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0d1b1a4eab25e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:24.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.302+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0052d0 con 0x7f6abc003a00 2023-07-06T03:55:24.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.302+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc005430 con 0x7f6abc003a00 2023-07-06T03:55:24.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.302+0000 7f6ad5199700 1 -- 172.21.15.97:0/1768025706 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0038c0 con 0x7f6abc003a00 2023-07-06T03:55:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.302+0000 7f6ad83ff700 1 -- 172.21.15.97:0/1768025706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 msgr2=0x7f6abc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.302+0000 7f6ad83ff700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 0x7f6abc003e10 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0017c0 tx=0x7f6abc005880 comp rx=0 tx=0).stop 2023-07-06T03:55:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.302+0000 7f6ad83ff700 1 -- 172.21.15.97:0/1768025706 shutdown_connections 2023-07-06T03:55:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.302+0000 7f6ad83ff700 1 --2- 172.21.15.97:0/1768025706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 0x7f6abc003e10 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.302+0000 7f6ad83ff700 1 -- 172.21.15.97:0/1768025706 >> 172.21.15.97:0/1768025706 conn(0x7f6ad0071790 msgr2=0x7f6ad0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.302+0000 7f6ad83ff700 1 -- 172.21.15.97:0/1768025706 shutdown_connections 2023-07-06T03:55:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.302+0000 7f6ad83ff700 1 -- 172.21.15.97:0/1768025706 wait complete. 2023-07-06T03:55:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.304+0000 7f6ad83ff700 1 Processor -- start 2023-07-06T03:55:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.304+0000 7f6ad83ff700 1 -- start start 2023-07-06T03:55:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.305+0000 7f6ad83ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 0x7f6ad0152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.305+0000 7f6ad83ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ad00c6870 0x7f6ad00cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.305+0000 7f6ad83ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ad00cca80 0x7f6ad00d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.305+0000 7f6ad83ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0153810 con 0x7f6abc003a00 2023-07-06T03:55:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.305+0000 7f6ad83ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad00ccf00 con 0x7f6ad00cca80 2023-07-06T03:55:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.305+0000 7f6ad83ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad00d1ba0 con 0x7f6ad00c6870 2023-07-06T03:55:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.305+0000 7f6ad619b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 0x7f6ad0152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.305+0000 7f6ad619b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 0x7f6ad0152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57768/0 (socket says 172.21.15.97:57768) 2023-07-06T03:55:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.305+0000 7f6ad619b700 1 -- 172.21.15.97:0/4122250662 learned_addr learned my addr 172.21.15.97:0/4122250662 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:24.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.305+0000 7f6ad699c700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ad00cca80 0x7f6ad00d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:24.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.305+0000 7f6ad599a700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ad00c6870 0x7f6ad00cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:24.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ad619b700 1 -- 172.21.15.97:0/4122250662 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ad00c6870 msgr2=0x7f6ad00cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:24.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ad619b700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ad00c6870 0x7f6ad00cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:24.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ad619b700 1 -- 172.21.15.97:0/4122250662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ad00cca80 msgr2=0x7f6ad00d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:24.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ad619b700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ad00cca80 0x7f6ad00d2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:24.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ad619b700 1 -- 172.21.15.97:0/4122250662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad00d1e80 con 0x7f6abc003a00 2023-07-06T03:55:24.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ad699c700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ad00cca80 0x7f6ad00d2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:24.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ad619b700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 0x7f6ad0152940 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f6acc015040 tx=0x7f6acc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:24.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ac6ffd700 1 -- 172.21.15.97:0/4122250662 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc001290 con 0x7f6abc003a00 2023-07-06T03:55:24.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ac6ffd700 1 -- 172.21.15.97:0/4122250662 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc01d480 con 0x7f6abc003a00 2023-07-06T03:55:24.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ad83ff700 1 -- 172.21.15.97:0/4122250662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad00d2160 con 0x7f6abc003a00 2023-07-06T03:55:24.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ac6ffd700 1 -- 172.21.15.97:0/4122250662 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc01d830 con 0x7f6abc003a00 2023-07-06T03:55:24.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.306+0000 7f6ad83ff700 1 -- 172.21.15.97:0/4122250662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad00d2720 con 0x7f6abc003a00 2023-07-06T03:55:24.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.307+0000 7f6ad83ff700 1 -- 172.21.15.97:0/4122250662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad00d2a90 con 0x7f6abc003a00 2023-07-06T03:55:24.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.308+0000 7f6ac6ffd700 1 -- 172.21.15.97:0/4122250662 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6acc019040 con 0x7f6abc003a00 2023-07-06T03:55:24.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.309+0000 7f6ac6ffd700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ab406ace0 0x7f6ab406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:24.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.309+0000 7f6ac6ffd700 1 -- 172.21.15.97:0/4122250662 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6acc08a0f0 con 0x7f6abc003a00 2023-07-06T03:55:24.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.310+0000 7f6ad599a700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ab406ace0 0x7f6ab406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:24.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.310+0000 7f6ad599a700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ab406ace0 0x7f6ab406d190 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00c9980 tx=0x7f6ac0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:24.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.314+0000 7f6ac6ffd700 1 -- 172.21.15.97:0/4122250662 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6acc055190 con 0x7f6abc003a00 2023-07-06T03:55:24.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.551+0000 7f6ad83ff700 1 -- 172.21.15.97:0/4122250662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6ad00d28b0 con 0x7f6abc003a00 2023-07-06T03:55:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.551+0000 7f6ac6ffd700 1 -- 172.21.15.97:0/4122250662 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6acc017070 con 0x7f6abc003a00 2023-07-06T03:55:24.553 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:55:24.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.553+0000 7f6ad83ff700 1 -- 172.21.15.97:0/4122250662 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ab406ace0 msgr2=0x7f6ab406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:24.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.553+0000 7f6ad83ff700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ab406ace0 0x7f6ab406d190 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00c9980 tx=0x7f6ac0008040 comp rx=0 tx=0).stop 2023-07-06T03:55:24.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.553+0000 7f6ad83ff700 1 -- 172.21.15.97:0/4122250662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 msgr2=0x7f6ad0152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.553+0000 7f6ad83ff700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 0x7f6ad0152940 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f6acc015040 tx=0x7f6acc001120 comp rx=0 tx=0).stop 2023-07-06T03:55:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.553+0000 7f6ad83ff700 1 -- 172.21.15.97:0/4122250662 shutdown_connections 2023-07-06T03:55:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.553+0000 7f6ad83ff700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6ab406ace0 0x7f6ab406d190 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.553+0000 7f6ad83ff700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ad00cca80 0x7f6ad00d2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.553+0000 7f6ad83ff700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc003a00 0x7f6ad0152940 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.553+0000 7f6ad83ff700 1 --2- 172.21.15.97:0/4122250662 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6ad00c6870 0x7f6ad00cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.553+0000 7f6ad83ff700 1 -- 172.21.15.97:0/4122250662 >> 172.21.15.97:0/4122250662 conn(0x7f6ad0071790 msgr2=0x7f6ad00b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:24.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.554+0000 7f6ad83ff700 1 -- 172.21.15.97:0/4122250662 shutdown_connections 2023-07-06T03:55:24.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:24.554+0000 7f6ad83ff700 1 -- 172.21.15.97:0/4122250662 wait complete. 2023-07-06T03:55:24.573 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:55:24.574 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:55:24.574 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2023-07-06T03:55:24.575 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T03:55:24.576 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T03:55:24.576 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:55:24.577 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:55:24.577 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:55:24.578 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:55:24.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:24.579 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2023-07-06T03:55:24.631 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:55:24.631 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:55:24.631 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:55:24.631 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:24.631 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:55:24.631 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:24.632 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:55:24.632 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:55:24.632 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2023-07-06T03:55:24.632 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:55:24.632 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:55:24.632 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2023-07-06T03:55:24.633 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:55:24.633 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2023-07-06T03:55:24.633 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:55:24.633 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2023-07-06T03:55:24.633 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:55:24.633 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 41560sec preferred_lft 41560sec 2023-07-06T03:55:24.633 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2023-07-06T03:55:24.634 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:24.634 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:55:24.634 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:55:24.634 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:55:24.634 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:24.634 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::58f0:6eff:fe04:3be0/64 scope link 2023-07-06T03:55:24.634 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:24.634 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:55:24.635 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:55:24.635 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::106d:9cff:fed5:9d0f/64 scope link 2023-07-06T03:55:24.635 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:24.635 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:55:24.635 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether a2:c5:04:8e:a2:a7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T03:55:24.635 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::a0c5:4ff:fe8e:a2a7/64 scope link 2023-07-06T03:55:24.635 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:55:24.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:24.637 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:55:24.706 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:55:24.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:24.710 DEBUG:teuthology.orchestra.run.smithi146:> route 2023-07-06T03:55:24.759 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2023-07-06T03:55:24.759 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:55:24.759 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:55:24.759 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:55:24.759 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:55:24.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:24.760 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:55:24.760 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:55:24.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:55:24.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:55:24.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:55:24.762 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:55:24.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:24.931 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:55:24.977 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T03:55:24.977 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T03:55:24.978 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:55:24.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:24.979 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2023-07-06T03:55:25.048 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-06T03:55:25.049 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:55:25.049 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T03:55:25.096 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:55:25.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:55:25.097 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:55:25.181 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:55:25.290 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:55:25.291 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:55:25.360 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:55:25.362 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:55:25.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:25.364 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:55:25.410 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:55:25.411 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-06T03:55:25.456 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:55:25.529 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:55:25.529 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:55:25.598 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:55:25.600 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:55:25.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:25.601 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:55:25.625 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2023-07-06T03:55:25.622+0000 7f9ef51d05c0 -1 init, newargv = 0x55b766ac72c0 newargc=15 2023-07-06T03:55:25.625 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[126921]: starting ceph client 2023-07-06T03:55:25.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[126921]: starting fuse 2023-07-06T03:55:25.654 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:55:25.654 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T03:55:25.655 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:55:25.656 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2023-07-06T03:55:25.656 DEBUG:teuthology.orchestra.run.smithi146:> import re 2023-07-06T03:55:25.657 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:55:25.657 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2023-07-06T03:55:25.657 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:55:25.657 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2023-07-06T03:55:25.658 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-06T03:55:25.658 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2023-07-06T03:55:25.658 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-06T03:55:25.659 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:55:25.659 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2023-07-06T03:55:25.660 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2023-07-06T03:55:25.661 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2023-07-06T03:55:25.661 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2023-07-06T03:55:25.662 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:55:25.662 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2023-07-06T03:55:25.663 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:55:25.664 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:55:25.664 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:55:25.665 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2023-07-06T03:55:25.666 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2023-07-06T03:55:25.666 DEBUG:teuthology.orchestra.run.smithi146:> return f 2023-07-06T03:55:25.667 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:55:25.667 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:55:25.668 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2023-07-06T03:55:25.668 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:55:25.837 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.126921.asok 2023-07-06T03:55:26.016 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.126921.asok 2023-07-06T03:55:26.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:26.017 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.126921.asok status 2023-07-06T03:55:26.241 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-07-06T03:55:26.241 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2023-07-06T03:55:26.241 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:55:26.241 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:55:26.242 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2023-07-06T03:55:26.242 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2023-07-06T03:55:26.242 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-06T03:55:26.242 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "126921", 2023-07-06T03:55:26.242 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2023-07-06T03:55:26.242 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:55:26.242 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2023-07-06T03:55:26.242 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2023-07-06T03:55:26.243 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 7286, 2023-07-06T03:55:26.243 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2023-07-06T03:55:26.243 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2023-07-06T03:55:26.243 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2023-07-06T03:55:26.243 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 7286 2023-07-06T03:55:26.243 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:55:26.243 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:55:26.244 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:55:26.244 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:55:26.244 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1807327696 2023-07-06T03:55:26.244 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-07-06T03:55:26.244 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:55:26.244 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:55:26.244 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:55:26.245 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:55:26.245 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1807327696 2023-07-06T03:55:26.245 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:55:26.245 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.7286 192.168.0.1:0/1807327696", 2023-07-06T03:55:26.245 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/1807327696", 2023-07-06T03:55:26.245 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2023-07-06T03:55:26.245 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 92, 2023-07-06T03:55:26.245 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 124, 2023-07-06T03:55:26.246 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:55:26.246 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2023-07-06T03:55:26.246 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2023-07-06T03:55:26.246 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-07-06T03:55:26.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:26.256 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2023-07-06T03:55:26.256 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-06T03:55:26.303 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2023-07-06T03:55:26.304 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-06T03:55:26.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:26.305 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:55:26.414 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2023-07-06T03:55:26.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.619+0000 7f3f5d7ff700 1 Processor -- start 2023-07-06T03:55:26.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.620+0000 7f3f5d7ff700 1 -- start start 2023-07-06T03:55:26.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f5d7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f5815c610 0x7f3f580b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:26.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f5d7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3f5806b360 0x7f3f580b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:26.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f5d7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f580cff90 con 0x7f3f580d2210 2023-07-06T03:55:26.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f5d7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f580d0780 con 0x7f3f5815c610 2023-07-06T03:55:26.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f5d7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5815c9e0 con 0x7f3f5806b360 2023-07-06T03:55:26.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f56ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f5815c610 0x7f3f580b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:26.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f56ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f5815c610 0x7f3f580b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57784/0 (socket says 172.21.15.97:57784) 2023-07-06T03:55:26.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f56ffd700 1 -- 172.21.15.97:0/376172504 learned_addr learned my addr 172.21.15.97:0/376172504 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:26.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f4ffff700 1 --2- 172.21.15.97:0/376172504 >> v2:172.21.15.146:3300/0 conn(0x7f3f5806b360 0x7f3f580b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:26.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f56ffd700 1 -- 172.21.15.97:0/376172504 >> v2:172.21.15.146:3300/0 conn(0x7f3f5806b360 msgr2=0x7f3f580b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:26.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f56ffd700 1 --2- 172.21.15.97:0/376172504 >> v2:172.21.15.146:3300/0 conn(0x7f3f5806b360 0x7f3f580b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:26.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.621+0000 7f3f56ffd700 1 -- 172.21.15.97:0/376172504 >> v1:172.21.15.146:6789/0 conn(0x7f3f580d2210 legacy=0x7f3f580d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:26.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.622+0000 7f3f56ffd700 1 -- 172.21.15.97:0/376172504 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580b5650 con 0x7f3f5815c610 2023-07-06T03:55:26.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.622+0000 7f3f4ffff700 1 --2- 172.21.15.97:0/376172504 >> v2:172.21.15.146:3300/0 conn(0x7f3f5806b360 0x7f3f580b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:26.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.622+0000 7f3f56ffd700 1 --2- 172.21.15.97:0/376172504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5815c610 0x7f3f580b47b0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f3f5806a590 tx=0x7f3f48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577f76ef04b60e6f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.622+0000 7f3f567fc700 1 -- 172.21.15.97:0/376172504 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48004230 con 0x7f3f5815c610 2023-07-06T03:55:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.622+0000 7f3f567fc700 1 -- 172.21.15.97:0/376172504 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48004390 con 0x7f3f5815c610 2023-07-06T03:55:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.622+0000 7f3f567fc700 1 -- 172.21.15.97:0/376172504 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f480036a0 con 0x7f3f5815c610 2023-07-06T03:55:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.622+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/376172504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5815c610 msgr2=0x7f3f580b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.622+0000 7f3f5d7ff700 1 --2- 172.21.15.97:0/376172504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5815c610 0x7f3f580b47b0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f3f5806a590 tx=0x7f3f48009c00 comp rx=0 tx=0).stop 2023-07-06T03:55:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.623+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/376172504 shutdown_connections 2023-07-06T03:55:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.623+0000 7f3f5d7ff700 1 --2- 172.21.15.97:0/376172504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5815c610 0x7f3f580b47b0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.623+0000 7f3f5d7ff700 1 --2- 172.21.15.97:0/376172504 >> v2:172.21.15.146:3300/0 conn(0x7f3f5806b360 0x7f3f580b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.623+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/376172504 >> 172.21.15.97:0/376172504 conn(0x7f3f5806a6c0 msgr2=0x7f3f5806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.623+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/376172504 shutdown_connections 2023-07-06T03:55:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.623+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/376172504 wait complete. 2023-07-06T03:55:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.624+0000 7f3f5d7ff700 1 Processor -- start 2023-07-06T03:55:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.624+0000 7f3f5d7ff700 1 -- start start 2023-07-06T03:55:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.625+0000 7f3f5d7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5806b360 0x7f3f58153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.625+0000 7f3f5d7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f580d2210 0x7f3f580cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.625+0000 7f3f5d7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f5815c610 0x7f3f5815d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.625+0000 7f3f5d7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580cc270 con 0x7f3f5806b360 2023-07-06T03:55:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.625+0000 7f3f5d7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580ccc40 con 0x7f3f580d2210 2023-07-06T03:55:26.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.625+0000 7f3f5d7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580b4540 con 0x7f3f5815c610 2023-07-06T03:55:26.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f56ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5806b360 0x7f3f58153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:26.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f56ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5806b360 0x7f3f58153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57798/0 (socket says 172.21.15.97:57798) 2023-07-06T03:55:26.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f56ffd700 1 -- 172.21.15.97:0/661007963 learned_addr learned my addr 172.21.15.97:0/661007963 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f4ffff700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f580d2210 0x7f3f580cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f577fe700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f5815c610 0x7f3f5815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f56ffd700 1 -- 172.21.15.97:0/661007963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f5815c610 msgr2=0x7f3f5815d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f56ffd700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f5815c610 0x7f3f5815d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f56ffd700 1 -- 172.21.15.97:0/661007963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f580d2210 msgr2=0x7f3f580cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f56ffd700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f580d2210 0x7f3f580cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f56ffd700 1 -- 172.21.15.97:0/661007963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f48009550 con 0x7f3f5806b360 2023-07-06T03:55:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f4ffff700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f580d2210 0x7f3f580cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f56ffd700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5806b360 0x7f3f58153460 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009130 tx=0x7f3f48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.626+0000 7f3f4f7fe700 1 -- 172.21.15.97:0/661007963 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48011830 con 0x7f3f5806b360 2023-07-06T03:55:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.627+0000 7f3f4f7fe700 1 -- 172.21.15.97:0/661007963 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48011990 con 0x7f3f5806b360 2023-07-06T03:55:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.627+0000 7f3f4f7fe700 1 -- 172.21.15.97:0/661007963 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48011d00 con 0x7f3f5806b360 2023-07-06T03:55:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.627+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/661007963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f580cb940 con 0x7f3f5806b360 2023-07-06T03:55:26.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.627+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/661007963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f580cbf00 con 0x7f3f5806b360 2023-07-06T03:55:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.628+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/661007963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5804e340 con 0x7f3f5806b360 2023-07-06T03:55:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.629+0000 7f3f4f7fe700 1 -- 172.21.15.97:0/661007963 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f4801b440 con 0x7f3f5806b360 2023-07-06T03:55:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.630+0000 7f3f4f7fe700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3f3806ace0 0x7f3f3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:26.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.630+0000 7f3f4f7fe700 1 -- 172.21.15.97:0/661007963 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f4808aeb0 con 0x7f3f5806b360 2023-07-06T03:55:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.630+0000 7f3f4ffff700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3f3806ace0 0x7f3f3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.631+0000 7f3f4ffff700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3f3806ace0 0x7f3f3806d190 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3f581f52d0 tx=0x7f3f40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:26.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.634+0000 7f3f4f7fe700 1 -- 172.21.15.97:0/661007963 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3f48055f50 con 0x7f3f5806b360 2023-07-06T03:55:26.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.831+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/661007963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) v1 -- 0x7f3f58159840 con 0x7f3f5806b360 2023-07-06T03:55:26.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.834+0000 7f3f4f7fe700 1 -- 172.21.15.97:0/661007963 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v55)=0 v55) v1 ==== 121+0+0 (secure 0 0 0) 0x7f3f48011e60 con 0x7f3f5806b360 2023-07-06T03:55:26.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/661007963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3f3806ace0 msgr2=0x7f3f3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:26.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3f3806ace0 0x7f3f3806d190 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3f581f52d0 tx=0x7f3f40008040 comp rx=0 tx=0).stop 2023-07-06T03:55:26.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/661007963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5806b360 msgr2=0x7f3f58153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:26.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5806b360 0x7f3f58153460 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009130 tx=0x7f3f48004750 comp rx=0 tx=0).stop 2023-07-06T03:55:26.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/661007963 shutdown_connections 2023-07-06T03:55:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3f3806ace0 0x7f3f3806d190 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3f5815c610 0x7f3f5815d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f5806b360 0x7f3f58153460 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 --2- 172.21.15.97:0/661007963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f580d2210 0x7f3f580cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/661007963 >> 172.21.15.97:0/661007963 conn(0x7f3f5806a6c0 msgr2=0x7f3f580d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/661007963 shutdown_connections 2023-07-06T03:55:26.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:26.837+0000 7f3f5d7ff700 1 -- 172.21.15.97:0/661007963 wait complete. 2023-07-06T03:55:26.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2023-07-06T03:55:27.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.052+0000 7f0457fff700 1 Processor -- start 2023-07-06T03:55:27.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.052+0000 7f0457fff700 1 -- start start 2023-07-06T03:55:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.053+0000 7f0457fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f045015c610 0x7f04500b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.053+0000 7f0457fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04500cff90 con 0x7f04500d2210 2023-07-06T03:55:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.054+0000 7f0457fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04500b5650 con 0x7f045006b360 2023-07-06T03:55:27.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.054+0000 7f0457fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04500d0780 con 0x7f045015c610 2023-07-06T03:55:27.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.054+0000 7f0455d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f045015c610 0x7f04500b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.054+0000 7f045659c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f04500d2210 0x7f04500d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53536/0 (socket says 172.21.15.97:53536) 2023-07-06T03:55:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.054+0000 7f045659c700 1 -- 172.21.15.97:0/2475480232 learned_addr learned my addr 172.21.15.97:0/2475480232 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.054+0000 7f0455d9b700 1 -- 172.21.15.97:0/2475480232 >> v1:172.21.15.146:6789/0 conn(0x7f045006b360 legacy=0x7f04500b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.054+0000 7f0455d9b700 1 -- 172.21.15.97:0/2475480232 >> v1:172.21.15.97:6789/0 conn(0x7f04500d2210 legacy=0x7f04500d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.054+0000 7f0455d9b700 1 -- 172.21.15.97:0/2475480232 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04500b5400 con 0x7f045015c610 2023-07-06T03:55:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.054+0000 7f0454d99700 1 -- 172.21.15.97:0/2475480232 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3235247506 0 0) 0x7f04500cff90 con 0x7f04500d2210 2023-07-06T03:55:27.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.055+0000 7f0455d9b700 1 --2- 172.21.15.97:0/2475480232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f045015c610 0x7f04500b47b0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f044c009840 tx=0x7f044c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e038166f860edf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:27.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.055+0000 7f0454d99700 1 -- 172.21.15.97:0/2475480232 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c011830 con 0x7f045015c610 2023-07-06T03:55:27.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.055+0000 7f0454d99700 1 -- 172.21.15.97:0/2475480232 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c011990 con 0x7f045015c610 2023-07-06T03:55:27.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.055+0000 7f0454d99700 1 -- 172.21.15.97:0/2475480232 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c011d20 con 0x7f045015c610 2023-07-06T03:55:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.055+0000 7f0457fff700 1 -- 172.21.15.97:0/2475480232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f045015c610 msgr2=0x7f04500b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.055+0000 7f0457fff700 1 --2- 172.21.15.97:0/2475480232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f045015c610 0x7f04500b47b0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f044c009840 tx=0x7f044c009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.055+0000 7f0457fff700 1 -- 172.21.15.97:0/2475480232 shutdown_connections 2023-07-06T03:55:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.055+0000 7f0457fff700 1 --2- 172.21.15.97:0/2475480232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f045015c610 0x7f04500b47b0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.055+0000 7f0457fff700 1 -- 172.21.15.97:0/2475480232 >> 172.21.15.97:0/2475480232 conn(0x7f045006a6c0 msgr2=0x7f045006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.056+0000 7f0457fff700 1 -- 172.21.15.97:0/2475480232 shutdown_connections 2023-07-06T03:55:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.056+0000 7f0457fff700 1 -- 172.21.15.97:0/2475480232 wait complete. 2023-07-06T03:55:27.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.057+0000 7f0457fff700 1 Processor -- start 2023-07-06T03:55:27.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.057+0000 7f0457fff700 1 -- start start 2023-07-06T03:55:27.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.058+0000 7f0457fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f045006b360 0x7f045015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.058+0000 7f0457fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04500d2210 0x7f04500cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.058+0000 7f0457fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f045015c610 0x7f04500cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.058+0000 7f0457fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045015d770 con 0x7f045015c610 2023-07-06T03:55:27.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.058+0000 7f0457fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045015cb60 con 0x7f045006b360 2023-07-06T03:55:27.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.058+0000 7f0457fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045015ce40 con 0x7f04500d2210 2023-07-06T03:55:27.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.059+0000 7f045659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f045015c610 0x7f04500cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.059+0000 7f045659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f045015c610 0x7f04500cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57816/0 (socket says 172.21.15.97:57816) 2023-07-06T03:55:27.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.059+0000 7f045659c700 1 -- 172.21.15.97:0/4195572672 learned_addr learned my addr 172.21.15.97:0/4195572672 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:27.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.059+0000 7f0455d9b700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f045006b360 0x7f045015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.059+0000 7f045559a700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04500d2210 0x7f04500cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.059+0000 7f045659c700 1 -- 172.21.15.97:0/4195572672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04500d2210 msgr2=0x7f04500cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.059+0000 7f045659c700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04500d2210 0x7f04500cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.059+0000 7f045659c700 1 -- 172.21.15.97:0/4195572672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f045006b360 msgr2=0x7f045015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.059+0000 7f045659c700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f045006b360 0x7f045015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.059+0000 7f045659c700 1 -- 172.21.15.97:0/4195572672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c0094a0 con 0x7f045015c610 2023-07-06T03:55:27.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.059+0000 7f045659c700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f045015c610 0x7f04500cc510 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f0448002230 tx=0x7f044800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:27.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.060+0000 7f04467fc700 1 -- 172.21.15.97:0/4195572672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044800aac0 con 0x7f045015c610 2023-07-06T03:55:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.060+0000 7f04467fc700 1 -- 172.21.15.97:0/4195572672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044800ac40 con 0x7f045015c610 2023-07-06T03:55:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.060+0000 7f04467fc700 1 -- 172.21.15.97:0/4195572672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04480070d0 con 0x7f045015c610 2023-07-06T03:55:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.060+0000 7f0457fff700 1 -- 172.21.15.97:0/4195572672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045015d120 con 0x7f045015c610 2023-07-06T03:55:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.060+0000 7f0457fff700 1 -- 172.21.15.97:0/4195572672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04500d1470 con 0x7f045015c610 2023-07-06T03:55:27.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.061+0000 7f0457fff700 1 -- 172.21.15.97:0/4195572672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045015d8e0 con 0x7f045015c610 2023-07-06T03:55:27.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.062+0000 7f04467fc700 1 -- 172.21.15.97:0/4195572672 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0448007250 con 0x7f045015c610 2023-07-06T03:55:27.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.063+0000 7f04467fc700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f043406ace0 0x7f043406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.063+0000 7f04467fc700 1 -- 172.21.15.97:0/4195572672 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0448088f40 con 0x7f045015c610 2023-07-06T03:55:27.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.063+0000 7f0455d9b700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f043406ace0 0x7f043406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.064+0000 7f0455d9b700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f043406ace0 0x7f043406d190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f044c0090b0 tx=0x7f044c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:27.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.067+0000 7f04467fc700 1 -- 172.21.15.97:0/4195572672 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0448053ff0 con 0x7f045015c610 2023-07-06T03:55:27.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.258+0000 7f0457fff700 1 -- 172.21.15.97:0/4195572672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) v1 -- 0x7f045004e340 con 0x7f045015c610 2023-07-06T03:55:27.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.262+0000 7f04467fc700 1 -- 172.21.15.97:0/4195572672 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v56)=0 v56) v1 ==== 119+0+0 (secure 0 0 0) 0x7f04480180b0 con 0x7f045015c610 2023-07-06T03:55:27.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 -- 172.21.15.97:0/4195572672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f043406ace0 msgr2=0x7f043406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f043406ace0 0x7f043406d190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f044c0090b0 tx=0x7f044c01c040 comp rx=0 tx=0).stop 2023-07-06T03:55:27.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 -- 172.21.15.97:0/4195572672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f045015c610 msgr2=0x7f04500cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f045015c610 0x7f04500cc510 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f0448002230 tx=0x7f044800cf50 comp rx=0 tx=0).stop 2023-07-06T03:55:27.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 -- 172.21.15.97:0/4195572672 shutdown_connections 2023-07-06T03:55:27.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f043406ace0 0x7f043406d190 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f045015c610 0x7f04500cc510 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f045006b360 0x7f045015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 --2- 172.21.15.97:0/4195572672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f04500d2210 0x7f04500cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 -- 172.21.15.97:0/4195572672 >> 172.21.15.97:0/4195572672 conn(0x7f045006a6c0 msgr2=0x7f04500b6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:27.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 -- 172.21.15.97:0/4195572672 shutdown_connections 2023-07-06T03:55:27.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.265+0000 7f0457fff700 1 -- 172.21.15.97:0/4195572672 wait complete. 2023-07-06T03:55:27.284 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2023-07-06T03:55:27.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.480+0000 7f95319ff700 1 Processor -- start 2023-07-06T03:55:27.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.481+0000 7f95319ff700 1 -- start start 2023-07-06T03:55:27.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.482+0000 7f95319ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0d2210 0x7f952c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.482+0000 7f95319ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c0cff90 con 0x7f952c15c610 2023-07-06T03:55:27.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.482+0000 7f95319ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c0b5650 con 0x7f952c06b360 2023-07-06T03:55:27.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.482+0000 7f95319ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0d0780 con 0x7f952c0d2210 2023-07-06T03:55:27.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.482+0000 7f952affd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f952c15c610 0x7f952c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53544/0 (socket says 172.21.15.97:53544) 2023-07-06T03:55:27.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.482+0000 7f952affd700 1 -- 172.21.15.97:0/436192912 learned_addr learned my addr 172.21.15.97:0/436192912 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:27.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.482+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051871832 0 0) 0x7f952c0cff90 con 0x7f952c15c610 2023-07-06T03:55:27.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.482+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95180035b0 con 0x7f952c15c610 2023-07-06T03:55:27.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.482+0000 7f952b7fe700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0d2210 0x7f952c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 837254999 0 0) 0x7f95180035b0 con 0x7f952c15c610 2023-07-06T03:55:27.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0d2210 msgr2=0x7f952c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:27.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0d2210 0x7f952c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 >> v1:172.21.15.146:6789/0 conn(0x7f952c06b360 legacy=0x7f952c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c0b5400 con 0x7f952c15c610 2023-07-06T03:55:27.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f951c001670 con 0x7f952c15c610 2023-07-06T03:55:27.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 >> v1:172.21.15.97:6789/0 conn(0x7f952c15c610 legacy=0x7f952c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9518003a30 0x7f9518003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 0x7f9518004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0d2210 0x7f95180052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0d0780 con 0x7f9518004850 2023-07-06T03:55:27.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951800bba0 con 0x7f9518003a30 2023-07-06T03:55:27.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951800bd60 con 0x7f952c0d2210 2023-07-06T03:55:27.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f9523fff700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 0x7f9518004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952affd700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9518003a30 0x7f9518003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f952b7fe700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0d2210 0x7f95180052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f9523fff700 1 -- 172.21.15.97:0/436192912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0d2210 msgr2=0x7f95180052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:27.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.483+0000 7f9523fff700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0d2210 0x7f95180052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.484+0000 7f9523fff700 1 -- 172.21.15.97:0/436192912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9518003a30 msgr2=0x7f9518003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:27.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.484+0000 7f9523fff700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9518003a30 0x7f9518003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.484+0000 7f9523fff700 1 -- 172.21.15.97:0/436192912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95180035b0 con 0x7f9518004850 2023-07-06T03:55:27.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.484+0000 7f9523fff700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 0x7f9518004cc0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f95140017c0 tx=0x7f9518005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5c05a5ff7ffee3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:27.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.484+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140052d0 con 0x7f9518004850 2023-07-06T03:55:27.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.484+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9514005430 con 0x7f9518004850 2023-07-06T03:55:27.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.484+0000 7f952a7fc700 1 -- 172.21.15.97:0/436192912 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140038c0 con 0x7f9518004850 2023-07-06T03:55:27.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.484+0000 7f95319ff700 1 -- 172.21.15.97:0/436192912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 msgr2=0x7f9518004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.484+0000 7f95319ff700 1 --2- 172.21.15.97:0/436192912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 0x7f9518004cc0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f95140017c0 tx=0x7f9518005880 comp rx=0 tx=0).stop 2023-07-06T03:55:27.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.484+0000 7f952b7fe700 1 -- 172.21.15.97:0/436192912 reap_dead start 2023-07-06T03:55:27.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.485+0000 7f95319ff700 1 -- 172.21.15.97:0/436192912 shutdown_connections 2023-07-06T03:55:27.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.485+0000 7f95319ff700 1 -- 172.21.15.97:0/436192912 >> 172.21.15.97:0/436192912 conn(0x7f952c06a6c0 msgr2=0x7f952c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:27.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.485+0000 7f95319ff700 1 -- 172.21.15.97:0/436192912 shutdown_connections 2023-07-06T03:55:27.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.485+0000 7f95319ff700 1 -- 172.21.15.97:0/436192912 wait complete. 2023-07-06T03:55:27.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.486+0000 7f95319ff700 1 Processor -- start 2023-07-06T03:55:27.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.487+0000 7f95319ff700 1 -- start start 2023-07-06T03:55:27.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.487+0000 7f95319ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 0x7f952c14a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.487+0000 7f95319ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0cc690 0x7f952c15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.487+0000 7f95319ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f952c15aff0 0x7f952c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.487+0000 7f95319ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c15b9a0 con 0x7f9518004850 2023-07-06T03:55:27.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.487+0000 7f95319ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c14a6f0 con 0x7f952c15aff0 2023-07-06T03:55:27.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.487+0000 7f95319ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0cb3a0 con 0x7f952c0cc690 2023-07-06T03:55:27.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f952affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 0x7f952c14a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f952affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 0x7f952c14a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57838/0 (socket says 172.21.15.97:57838) 2023-07-06T03:55:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f952affd700 1 -- 172.21.15.97:0/2743019190 learned_addr learned my addr 172.21.15.97:0/2743019190 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f9523fff700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0cc690 0x7f952c15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f952b7fe700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f952c15aff0 0x7f952c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f952affd700 1 -- 172.21.15.97:0/2743019190 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0cc690 msgr2=0x7f952c15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f952affd700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0cc690 0x7f952c15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f952affd700 1 -- 172.21.15.97:0/2743019190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f952c15aff0 msgr2=0x7f952c15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f952affd700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f952c15aff0 0x7f952c15b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f952affd700 1 -- 172.21.15.97:0/2743019190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c0cb5f0 con 0x7f9518004850 2023-07-06T03:55:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f952affd700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 0x7f952c14a1b0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f951c001200 tx=0x7f951c007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f9528ff9700 1 -- 172.21.15.97:0/2743019190 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c011580 con 0x7f9518004850 2023-07-06T03:55:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f9528ff9700 1 -- 172.21.15.97:0/2743019190 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c011720 con 0x7f9518004850 2023-07-06T03:55:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.488+0000 7f9528ff9700 1 -- 172.21.15.97:0/2743019190 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c011ad0 con 0x7f9518004850 2023-07-06T03:55:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.489+0000 7f95319ff700 1 -- 172.21.15.97:0/2743019190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c0cb8d0 con 0x7f9518004850 2023-07-06T03:55:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.489+0000 7f95319ff700 1 -- 172.21.15.97:0/2743019190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c0cbe90 con 0x7f9518004850 2023-07-06T03:55:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.490+0000 7f95319ff700 1 -- 172.21.15.97:0/2743019190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c04e340 con 0x7f9518004850 2023-07-06T03:55:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.491+0000 7f9528ff9700 1 -- 172.21.15.97:0/2743019190 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f951c011c30 con 0x7f9518004850 2023-07-06T03:55:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.492+0000 7f9528ff9700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f950c06ace0 0x7f950c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.492+0000 7f9528ff9700 1 -- 172.21.15.97:0/2743019190 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f951c089590 con 0x7f9518004850 2023-07-06T03:55:27.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.492+0000 7f9523fff700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f950c06ace0 0x7f950c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.493+0000 7f9523fff700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f950c06ace0 0x7f950c06d190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9514000c00 tx=0x7f9514009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:27.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.496+0000 7f9528ff9700 1 -- 172.21.15.97:0/2743019190 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f951c0545a0 con 0x7f9518004850 2023-07-06T03:55:27.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.692+0000 7f95319ff700 1 -- 172.21.15.97:0/2743019190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) v1 -- 0x7f952c0cf220 con 0x7f9518004850 2023-07-06T03:55:27.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.696+0000 7f9528ff9700 1 -- 172.21.15.97:0/2743019190 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v57)=0 v57) v1 ==== 128+0+0 (secure 0 0 0) 0x7f951c057bc0 con 0x7f9518004850 2023-07-06T03:55:27.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 -- 172.21.15.97:0/2743019190 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f950c06ace0 msgr2=0x7f950c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f950c06ace0 0x7f950c06d190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9514000c00 tx=0x7f9514009030 comp rx=0 tx=0).stop 2023-07-06T03:55:27.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 -- 172.21.15.97:0/2743019190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 msgr2=0x7f952c14a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 0x7f952c14a1b0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f951c001200 tx=0x7f951c007570 comp rx=0 tx=0).stop 2023-07-06T03:55:27.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 -- 172.21.15.97:0/2743019190 shutdown_connections 2023-07-06T03:55:27.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f950c06ace0 0x7f950c06d190 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f952c15aff0 0x7f952c15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9518004850 0x7f952c14a1b0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 --2- 172.21.15.97:0/2743019190 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f952c0cc690 0x7f952c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 -- 172.21.15.97:0/2743019190 >> 172.21.15.97:0/2743019190 conn(0x7f952c06a6c0 msgr2=0x7f952c0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:27.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 -- 172.21.15.97:0/2743019190 shutdown_connections 2023-07-06T03:55:27.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.699+0000 7f95319ff700 1 -- 172.21.15.97:0/2743019190 wait complete. 2023-07-06T03:55:27.718 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2023-07-06T03:55:27.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.913+0000 7fedef5ff700 1 Processor -- start 2023-07-06T03:55:27.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.913+0000 7fedef5ff700 1 -- start start 2023-07-06T03:55:27.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.914+0000 7fedef5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fede80d15a0 0x7fede80cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.914+0000 7fedef5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede80cfed0 0x7fede80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.914+0000 7fedef5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede80d2d30 con 0x7fede80b4a40 2023-07-06T03:55:27.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.914+0000 7fedef5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede80b6520 con 0x7fede80d15a0 2023-07-06T03:55:27.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.914+0000 7fedef5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede80d1a10 con 0x7fede80cfed0 2023-07-06T03:55:27.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.914+0000 7feded39b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fede80b4a40 0x7fede80b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53550/0 (socket says 172.21.15.97:53550) 2023-07-06T03:55:27.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.914+0000 7feded39b700 1 -- 172.21.15.97:0/975774191 learned_addr learned my addr 172.21.15.97:0/975774191 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:27.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7fededb9c700 1 --2- 172.21.15.97:0/975774191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede80cfed0 0x7fede80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7fedecb9a700 1 --2- 172.21.15.97:0/975774191 >> v2:172.21.15.146:3300/0 conn(0x7fede80d15a0 0x7fede80cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897548141 0 0) 0x7fede80d2d30 con 0x7fede80b4a40 2023-07-06T03:55:27.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedd40035b0 con 0x7fede80b4a40 2023-07-06T03:55:27.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 249818515 0 0) 0x7fedd40035b0 con 0x7fede80b4a40 2023-07-06T03:55:27.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede80cfed0 msgr2=0x7fede80b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:27.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 --2- 172.21.15.97:0/975774191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede80cfed0 0x7fede80b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 >> v2:172.21.15.146:3300/0 conn(0x7fede80d15a0 msgr2=0x7fede80cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:27.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 --2- 172.21.15.97:0/975774191 >> v2:172.21.15.146:3300/0 conn(0x7fede80d15a0 0x7fede80cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede80cf210 con 0x7fede80b4a40 2023-07-06T03:55:27.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fede4001670 con 0x7fede80b4a40 2023-07-06T03:55:27.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 >> v1:172.21.15.97:6789/0 conn(0x7fede80b4a40 legacy=0x7fede80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 --2- 172.21.15.97:0/975774191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede80cfed0 0x7fedd4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 --2- 172.21.15.97:0/975774191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedd4003fd0 0x7fedd4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 --2- 172.21.15.97:0/975774191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 0x7fedd4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede80b6520 con 0x7fedd4004d90 2023-07-06T03:55:27.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede80d1a10 con 0x7fedd4003fd0 2023-07-06T03:55:27.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.915+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd400ba10 con 0x7fede80cfed0 2023-07-06T03:55:27.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.916+0000 7fededb9c700 1 --2- 172.21.15.97:0/975774191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 0x7fedd4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.916+0000 7fedecb9a700 1 --2- 172.21.15.97:0/975774191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedd4003fd0 0x7fedd4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.916+0000 7fededb9c700 1 -- 172.21.15.97:0/975774191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede80cfed0 msgr2=0x7fedd4003a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T03:55:27.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.916+0000 7fededb9c700 1 --2- 172.21.15.97:0/975774191 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede80cfed0 0x7fedd4003a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.916+0000 7fededb9c700 1 -- 172.21.15.97:0/975774191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedd4003fd0 msgr2=0x7fedd4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:27.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.916+0000 7fededb9c700 1 --2- 172.21.15.97:0/975774191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedd4003fd0 0x7fedd4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.916+0000 7fededb9c700 1 -- 172.21.15.97:0/975774191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd40035b0 con 0x7fedd4004d90 2023-07-06T03:55:27.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.916+0000 7fededb9c700 1 --2- 172.21.15.97:0/975774191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 0x7fedd4005200 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fede0009fc0 tx=0x7fedd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e31d19b58abe96 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:27.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.916+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede000ec30 con 0x7fedd4004d90 2023-07-06T03:55:27.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.916+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede000f040 con 0x7fedd4004d90 2023-07-06T03:55:27.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.916+0000 7feddffff700 1 -- 172.21.15.97:0/975774191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0014480 con 0x7fedd4004d90 2023-07-06T03:55:27.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.917+0000 7fedef5ff700 1 -- 172.21.15.97:0/975774191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 msgr2=0x7fedd4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.917+0000 7fedef5ff700 1 --2- 172.21.15.97:0/975774191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 0x7fedd4005200 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fede0009fc0 tx=0x7fedd4005880 comp rx=0 tx=0).stop 2023-07-06T03:55:27.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.917+0000 7fededb9c700 1 -- 172.21.15.97:0/975774191 reap_dead start 2023-07-06T03:55:27.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.917+0000 7fedef5ff700 1 -- 172.21.15.97:0/975774191 shutdown_connections 2023-07-06T03:55:27.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.917+0000 7fedef5ff700 1 -- 172.21.15.97:0/975774191 >> 172.21.15.97:0/975774191 conn(0x7fede806a900 msgr2=0x7fede806acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:27.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.917+0000 7fedef5ff700 1 -- 172.21.15.97:0/975774191 shutdown_connections 2023-07-06T03:55:27.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.917+0000 7fedef5ff700 1 -- 172.21.15.97:0/975774191 wait complete. 2023-07-06T03:55:27.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.919+0000 7fedef5ff700 1 Processor -- start 2023-07-06T03:55:27.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.919+0000 7fedef5ff700 1 -- start start 2023-07-06T03:55:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7fedef5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 0x7fede80ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7fedef5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede815bd80 0x7fede80c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7fedef5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede815eca0 0x7fede815f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7fedef5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede815f680 con 0x7fedd4004d90 2023-07-06T03:55:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7fedef5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede80cb360 con 0x7fede815bd80 2023-07-06T03:55:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7fedef5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede80c7ad0 con 0x7fede815eca0 2023-07-06T03:55:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7feded39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 0x7fede80ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7feded39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 0x7fede80ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57850/0 (socket says 172.21.15.97:57850) 2023-07-06T03:55:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7feded39b700 1 -- 172.21.15.97:0/2753630709 learned_addr learned my addr 172.21.15.97:0/2753630709 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7fedecb9a700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede815bd80 0x7fede80c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7fededb9c700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede815eca0 0x7fede815f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7feded39b700 1 -- 172.21.15.97:0/2753630709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede815eca0 msgr2=0x7fede815f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7feded39b700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede815eca0 0x7fede815f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7feded39b700 1 -- 172.21.15.97:0/2753630709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede815bd80 msgr2=0x7fede80c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7feded39b700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede815bd80 0x7fede80c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7feded39b700 1 -- 172.21.15.97:0/2753630709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede80ca750 con 0x7fedd4004d90 2023-07-06T03:55:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.920+0000 7fededb9c700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede815eca0 0x7fede815f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:27.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.921+0000 7feded39b700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 0x7fede80ca090 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fede400cad0 tx=0x7fede400ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.921+0000 7fedddffb700 1 -- 172.21.15.97:0/2753630709 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4004a80 con 0x7fedd4004d90 2023-07-06T03:55:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.921+0000 7fedddffb700 1 -- 172.21.15.97:0/2753630709 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede4004c00 con 0x7fedd4004d90 2023-07-06T03:55:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.921+0000 7fedef5ff700 1 -- 172.21.15.97:0/2753630709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede80caa30 con 0x7fedd4004d90 2023-07-06T03:55:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.921+0000 7fedddffb700 1 -- 172.21.15.97:0/2753630709 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40070d0 con 0x7fedd4004d90 2023-07-06T03:55:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.921+0000 7fedef5ff700 1 -- 172.21.15.97:0/2753630709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede80caff0 con 0x7fedd4004d90 2023-07-06T03:55:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.922+0000 7fedef5ff700 1 -- 172.21.15.97:0/2753630709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede80cb640 con 0x7fedd4004d90 2023-07-06T03:55:27.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.923+0000 7fedddffb700 1 -- 172.21.15.97:0/2753630709 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fede4007230 con 0x7fedd4004d90 2023-07-06T03:55:27.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.924+0000 7fedddffb700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fedcc06ace0 0x7fedcc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:27.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.924+0000 7fedddffb700 1 -- 172.21.15.97:0/2753630709 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fede4088af0 con 0x7fedd4004d90 2023-07-06T03:55:27.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.924+0000 7fedecb9a700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fedcc06ace0 0x7fedcc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:27.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.925+0000 7fedecb9a700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fedcc06ace0 0x7fedcc06d190 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fede806b110 tx=0x7fedd8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:27.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:27.928+0000 7fedddffb700 1 -- 172.21.15.97:0/2753630709 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fede4057590 con 0x7fedd4004d90 2023-07-06T03:55:28.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.125+0000 7fedef5ff700 1 -- 172.21.15.97:0/2753630709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) v1 -- 0x7fede804e340 con 0x7fedd4004d90 2023-07-06T03:55:28.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.126+0000 7fedddffb700 1 -- 172.21.15.97:0/2753630709 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v57) v1 ==== 106+0+4 (secure 0 0 0) 0x7fede40180b0 con 0x7fedd4004d90 2023-07-06T03:55:28.128 INFO:teuthology.orchestra.run.smithi097.stdout:100 2023-07-06T03:55:28.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 -- 172.21.15.97:0/2753630709 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fedcc06ace0 msgr2=0x7fedcc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fedcc06ace0 0x7fedcc06d190 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fede806b110 tx=0x7fedd8008040 comp rx=0 tx=0).stop 2023-07-06T03:55:28.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 -- 172.21.15.97:0/2753630709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 msgr2=0x7fede80ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 0x7fede80ca090 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fede400cad0 tx=0x7fede400ce90 comp rx=0 tx=0).stop 2023-07-06T03:55:28.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 -- 172.21.15.97:0/2753630709 shutdown_connections 2023-07-06T03:55:28.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fedcc06ace0 0x7fedcc06d190 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fede815eca0 0x7fede815f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedd4004d90 0x7fede80ca090 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 --2- 172.21.15.97:0/2753630709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fede815bd80 0x7fede80c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 -- 172.21.15.97:0/2753630709 >> 172.21.15.97:0/2753630709 conn(0x7fede806a900 msgr2=0x7fede8156ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:28.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 -- 172.21.15.97:0/2753630709 shutdown_connections 2023-07-06T03:55:28.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.128+0000 7fedef5ff700 1 -- 172.21.15.97:0/2753630709 wait complete. 2023-07-06T03:55:28.147 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2023-07-06T03:55:28.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.342+0000 7f19403ff700 1 Processor -- start 2023-07-06T03:55:28.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.342+0000 7f19403ff700 1 -- start start 2023-07-06T03:55:28.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.344+0000 7f19403ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f19380c3cd0 0x7f19380cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.344+0000 7f19403ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19380ca260 con 0x7f1938159440 2023-07-06T03:55:28.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.344+0000 7f19403ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19380c99c0 con 0x7f1938159070 2023-07-06T03:55:28.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.344+0000 7f19403ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19380c9ca0 con 0x7f19380c3cd0 2023-07-06T03:55:28.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.344+0000 7f193e99c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1938159440 0x7f19380cb0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53560/0 (socket says 172.21.15.97:53560) 2023-07-06T03:55:28.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.344+0000 7f193e99c700 1 -- 172.21.15.97:0/719897845 learned_addr learned my addr 172.21.15.97:0/719897845 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:28.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.344+0000 7f193d99a700 1 --2- 172.21.15.97:0/719897845 >> v2:172.21.15.146:3300/0 conn(0x7f19380c3cd0 0x7f19380cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:28.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.344+0000 7f193d199700 1 -- 172.21.15.97:0/719897845 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183682495 0 0) 0x7f19380ca260 con 0x7f1938159440 2023-07-06T03:55:28.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.345+0000 7f193d199700 1 -- 172.21.15.97:0/719897845 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19240035b0 con 0x7f1938159440 2023-07-06T03:55:28.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.345+0000 7f193d199700 1 -- 172.21.15.97:0/719897845 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983912339 0 0) 0x7f19380c99c0 con 0x7f1938159070 2023-07-06T03:55:28.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.345+0000 7f193d199700 1 -- 172.21.15.97:0/719897845 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19380ca260 con 0x7f1938159070 2023-07-06T03:55:28.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.345+0000 7f193d199700 1 -- 172.21.15.97:0/719897845 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4117315644 0 0) 0x7f19240035b0 con 0x7f1938159440 2023-07-06T03:55:28.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.345+0000 7f193d99a700 1 -- 172.21.15.97:0/719897845 >> v1:172.21.15.146:6789/0 conn(0x7f1938159070 legacy=0x7f19380c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.345+0000 7f193d99a700 1 -- 172.21.15.97:0/719897845 >> v1:172.21.15.97:6789/0 conn(0x7f1938159440 legacy=0x7f19380cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.345+0000 7f193d99a700 1 -- 172.21.15.97:0/719897845 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19380c9f80 con 0x7f19380c3cd0 2023-07-06T03:55:28.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.345+0000 7f193d99a700 1 --2- 172.21.15.97:0/719897845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19380c3cd0 0x7f19380cc4c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1928009840 tx=0x7f1928009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcfd3b1df676e3d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:28.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.345+0000 7f193d199700 1 -- 172.21.15.97:0/719897845 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19300019f0 con 0x7f19380c3cd0 2023-07-06T03:55:28.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.345+0000 7f193d199700 1 -- 172.21.15.97:0/719897845 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1928004030 con 0x7f19380c3cd0 2023-07-06T03:55:28.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.345+0000 7f193d199700 1 -- 172.21.15.97:0/719897845 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19280043c0 con 0x7f19380c3cd0 2023-07-06T03:55:28.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.346+0000 7f19403ff700 1 -- 172.21.15.97:0/719897845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19380c3cd0 msgr2=0x7f19380cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.346+0000 7f19403ff700 1 --2- 172.21.15.97:0/719897845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19380c3cd0 0x7f19380cc4c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1928009840 tx=0x7f1928009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:28.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.346+0000 7f19403ff700 1 -- 172.21.15.97:0/719897845 shutdown_connections 2023-07-06T03:55:28.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.346+0000 7f19403ff700 1 --2- 172.21.15.97:0/719897845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19380c3cd0 0x7f19380cc4c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.346+0000 7f19403ff700 1 -- 172.21.15.97:0/719897845 >> 172.21.15.97:0/719897845 conn(0x7f193806a060 msgr2=0x7f193806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:28.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.346+0000 7f19403ff700 1 -- 172.21.15.97:0/719897845 shutdown_connections 2023-07-06T03:55:28.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.346+0000 7f19403ff700 1 -- 172.21.15.97:0/719897845 wait complete. 2023-07-06T03:55:28.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.348+0000 7f19403ff700 1 Processor -- start 2023-07-06T03:55:28.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.348+0000 7f19403ff700 1 -- start start 2023-07-06T03:55:28.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f19403ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19380c3cd0 0x7f19380b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f19403ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1938159070 0x7f19380b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f19403ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938159440 0x7f1938142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f19403ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1938140250 con 0x7f1938159440 2023-07-06T03:55:28.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f19403ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19380b33e0 con 0x7f19380c3cd0 2023-07-06T03:55:28.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f19403ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19380b36c0 con 0x7f1938159070 2023-07-06T03:55:28.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f193e99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938159440 0x7f1938142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:28.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f193e99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938159440 0x7f1938142510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57866/0 (socket says 172.21.15.97:57866) 2023-07-06T03:55:28.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f193e99c700 1 -- 172.21.15.97:0/1179812318 learned_addr learned my addr 172.21.15.97:0/1179812318 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:28.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f193d99a700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1938159070 0x7f19380b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:28.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f193e19b700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19380c3cd0 0x7f19380b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:28.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f193e99c700 1 -- 172.21.15.97:0/1179812318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1938159070 msgr2=0x7f19380b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f193e99c700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1938159070 0x7f19380b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f193e99c700 1 -- 172.21.15.97:0/1179812318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19380c3cd0 msgr2=0x7f19380b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f193e99c700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19380c3cd0 0x7f19380b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.349+0000 7f193e99c700 1 -- 172.21.15.97:0/1179812318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19280094a0 con 0x7f1938159440 2023-07-06T03:55:28.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.350+0000 7f193e99c700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938159440 0x7f1938142510 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f193000cbc0 tx=0x7f193000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:28.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.350+0000 7f192effd700 1 -- 172.21.15.97:0/1179812318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193000abb0 con 0x7f1938159440 2023-07-06T03:55:28.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.350+0000 7f192effd700 1 -- 172.21.15.97:0/1179812318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193000ad10 con 0x7f1938159440 2023-07-06T03:55:28.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.350+0000 7f19403ff700 1 -- 172.21.15.97:0/1179812318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19380b39a0 con 0x7f1938159440 2023-07-06T03:55:28.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.350+0000 7f192effd700 1 -- 172.21.15.97:0/1179812318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930007310 con 0x7f1938159440 2023-07-06T03:55:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.350+0000 7f19403ff700 1 -- 172.21.15.97:0/1179812318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19380b3f60 con 0x7f1938159440 2023-07-06T03:55:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.351+0000 7f19403ff700 1 -- 172.21.15.97:0/1179812318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19380cd1a0 con 0x7f1938159440 2023-07-06T03:55:28.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.352+0000 7f192effd700 1 -- 172.21.15.97:0/1179812318 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1930007470 con 0x7f1938159440 2023-07-06T03:55:28.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.353+0000 7f192effd700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f191c06ad10 0x7f191c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.353+0000 7f192effd700 1 -- 172.21.15.97:0/1179812318 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1930089100 con 0x7f1938159440 2023-07-06T03:55:28.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.353+0000 7f193e19b700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f191c06ad10 0x7f191c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:28.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.354+0000 7f193e19b700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f191c06ad10 0x7f191c06d1c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f19340011e0 tx=0x7f1934009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:28.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.357+0000 7f192effd700 1 -- 172.21.15.97:0/1179812318 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1930054080 con 0x7f1938159440 2023-07-06T03:55:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.554+0000 7f19403ff700 1 -- 172.21.15.97:0/1179812318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) v1 -- 0x7f193804e340 con 0x7f1938159440 2023-07-06T03:55:28.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.558+0000 7f192effd700 1 -- 172.21.15.97:0/1179812318 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v58)=0 v58) v1 ==== 124+0+0 (secure 0 0 0) 0x7f19300180c0 con 0x7f1938159440 2023-07-06T03:55:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 -- 172.21.15.97:0/1179812318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f191c06ad10 msgr2=0x7f191c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f191c06ad10 0x7f191c06d1c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f19340011e0 tx=0x7f1934009040 comp rx=0 tx=0).stop 2023-07-06T03:55:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 -- 172.21.15.97:0/1179812318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938159440 msgr2=0x7f1938142510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938159440 0x7f1938142510 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f193000cbc0 tx=0x7f193000cf80 comp rx=0 tx=0).stop 2023-07-06T03:55:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 -- 172.21.15.97:0/1179812318 shutdown_connections 2023-07-06T03:55:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f191c06ad10 0x7f191c06d1c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938159440 0x7f1938142510 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19380c3cd0 0x7f19380b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 --2- 172.21.15.97:0/1179812318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1938159070 0x7f19380b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 -- 172.21.15.97:0/1179812318 >> 172.21.15.97:0/1179812318 conn(0x7f193806a060 msgr2=0x7f19380cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:28.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 -- 172.21.15.97:0/1179812318 shutdown_connections 2023-07-06T03:55:28.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.561+0000 7f19403ff700 1 -- 172.21.15.97:0/1179812318 wait complete. 2023-07-06T03:55:28.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2023-07-06T03:55:28.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.775+0000 7f64123ff700 1 Processor -- start 2023-07-06T03:55:28.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.775+0000 7f64123ff700 1 -- start start 2023-07-06T03:55:28.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.776+0000 7f64123ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f640c15c610 0x7f640c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.776+0000 7f64123ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06b360 0x7f640c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.776+0000 7f64123ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f640c0d2210 0x7f640c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.776+0000 7f64123ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640c0d0780 con 0x7f640c0d2210 2023-07-06T03:55:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.776+0000 7f64123ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640c15c9e0 con 0x7f640c15c610 2023-07-06T03:55:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.776+0000 7f64123ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0b5650 con 0x7f640c06b360 2023-07-06T03:55:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.776+0000 7f641099c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f640c0d2210 0x7f640c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.776+0000 7f641099c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f640c0d2210 0x7f640c0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57872/0 (socket says 172.21.15.97:57872) 2023-07-06T03:55:28.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.776+0000 7f641099c700 1 -- 172.21.15.97:0/3656805171 learned_addr learned my addr 172.21.15.97:0/3656805171 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.776+0000 7f640b7fe700 1 --2- 172.21.15.97:0/3656805171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06b360 0x7f640c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.776+0000 7f640bfff700 1 --2- 172.21.15.97:0/3656805171 >> v2:172.21.15.146:3300/0 conn(0x7f640c15c610 0x7f640c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f641099c700 1 -- 172.21.15.97:0/3656805171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06b360 msgr2=0x7f640c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f641099c700 1 --2- 172.21.15.97:0/3656805171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06b360 0x7f640c0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f641099c700 1 -- 172.21.15.97:0/3656805171 >> v2:172.21.15.146:3300/0 conn(0x7f640c15c610 msgr2=0x7f640c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f641099c700 1 --2- 172.21.15.97:0/3656805171 >> v2:172.21.15.146:3300/0 conn(0x7f640c15c610 0x7f640c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f641099c700 1 -- 172.21.15.97:0/3656805171 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0d01c0 con 0x7f640c0d2210 2023-07-06T03:55:28.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f640b7fe700 1 --2- 172.21.15.97:0/3656805171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06b360 0x7f640c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:28.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f640bfff700 1 --2- 172.21.15.97:0/3656805171 >> v2:172.21.15.146:3300/0 conn(0x7f640c15c610 0x7f640c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:28.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f641099c700 1 --2- 172.21.15.97:0/3656805171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f640c0d2210 0x7f640c0d1310 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f640c08b000 tx=0x7f640400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af2a3127395f824 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:28.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f640affd700 1 -- 172.21.15.97:0/3656805171 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640400e070 con 0x7f640c0d2210 2023-07-06T03:55:28.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f640affd700 1 -- 172.21.15.97:0/3656805171 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640400b980 con 0x7f640c0d2210 2023-07-06T03:55:28.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f640affd700 1 -- 172.21.15.97:0/3656805171 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640400bd30 con 0x7f640c0d2210 2023-07-06T03:55:28.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.777+0000 7f64123ff700 1 -- 172.21.15.97:0/3656805171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f640c0d2210 msgr2=0x7f640c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.778+0000 7f64123ff700 1 --2- 172.21.15.97:0/3656805171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f640c0d2210 0x7f640c0d1310 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f640c08b000 tx=0x7f640400b330 comp rx=0 tx=0).stop 2023-07-06T03:55:28.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.778+0000 7f64123ff700 1 -- 172.21.15.97:0/3656805171 shutdown_connections 2023-07-06T03:55:28.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.778+0000 7f64123ff700 1 --2- 172.21.15.97:0/3656805171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f640c0d2210 0x7f640c0d1310 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.778+0000 7f64123ff700 1 --2- 172.21.15.97:0/3656805171 >> v2:172.21.15.146:3300/0 conn(0x7f640c15c610 0x7f640c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.778+0000 7f64123ff700 1 --2- 172.21.15.97:0/3656805171 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c06b360 0x7f640c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.778+0000 7f64123ff700 1 -- 172.21.15.97:0/3656805171 >> 172.21.15.97:0/3656805171 conn(0x7f640c06a6c0 msgr2=0x7f640c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:28.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.778+0000 7f64123ff700 1 -- 172.21.15.97:0/3656805171 shutdown_connections 2023-07-06T03:55:28.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.778+0000 7f64123ff700 1 -- 172.21.15.97:0/3656805171 wait complete. 2023-07-06T03:55:28.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.779+0000 7f64123ff700 1 Processor -- start 2023-07-06T03:55:28.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.780+0000 7f64123ff700 1 -- start start 2023-07-06T03:55:28.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.780+0000 7f64123ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f640c06b360 0x7f640c1508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.780+0000 7f64123ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c0d2210 0x7f640c150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.780+0000 7f64123ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f640c15c610 0x7f640c0cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.780+0000 7f64123ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0ccc90 con 0x7f640c06b360 2023-07-06T03:55:28.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.780+0000 7f64123ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c15d6d0 con 0x7f640c15c610 2023-07-06T03:55:28.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.780+0000 7f64123ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640c15d9b0 con 0x7f640c0d2210 2023-07-06T03:55:28.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.781+0000 7f640bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f640c06b360 0x7f640c1508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:28.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.781+0000 7f640bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f640c06b360 0x7f640c1508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57878/0 (socket says 172.21.15.97:57878) 2023-07-06T03:55:28.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.781+0000 7f640bfff700 1 -- 172.21.15.97:0/2943458580 learned_addr learned my addr 172.21.15.97:0/2943458580 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:28.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.781+0000 7f640bfff700 1 -- 172.21.15.97:0/2943458580 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c0d2210 msgr2=0x7f640c150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.781+0000 7f640bfff700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c0d2210 0x7f640c150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.781+0000 7f640bfff700 1 -- 172.21.15.97:0/2943458580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f640c15c610 msgr2=0x7f640c0cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.781+0000 7f640bfff700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f640c15c610 0x7f640c0cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.781+0000 7f640bfff700 1 -- 172.21.15.97:0/2943458580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6404009b30 con 0x7f640c06b360 2023-07-06T03:55:28.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.781+0000 7f640bfff700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f640c06b360 0x7f640c1508a0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00c7f0 tx=0x7f63fc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:28.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.781+0000 7f6408ff9700 1 -- 172.21.15.97:0/2943458580 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc011800 con 0x7f640c06b360 2023-07-06T03:55:28.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.782+0000 7f6408ff9700 1 -- 172.21.15.97:0/2943458580 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc011960 con 0x7f640c06b360 2023-07-06T03:55:28.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.782+0000 7f64123ff700 1 -- 172.21.15.97:0/2943458580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c15dc90 con 0x7f640c06b360 2023-07-06T03:55:28.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.782+0000 7f6408ff9700 1 -- 172.21.15.97:0/2943458580 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc011d10 con 0x7f640c06b360 2023-07-06T03:55:28.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.782+0000 7f64123ff700 1 -- 172.21.15.97:0/2943458580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640c15e250 con 0x7f640c06b360 2023-07-06T03:55:28.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.783+0000 7f64123ff700 1 -- 172.21.15.97:0/2943458580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640c15e440 con 0x7f640c06b360 2023-07-06T03:55:28.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.784+0000 7f6408ff9700 1 -- 172.21.15.97:0/2943458580 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f63fc018360 con 0x7f640c06b360 2023-07-06T03:55:28.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.784+0000 7f6408ff9700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f63f806ace0 0x7f63f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:28.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.785+0000 7f6408ff9700 1 -- 172.21.15.97:0/2943458580 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f63fc0894c0 con 0x7f640c06b360 2023-07-06T03:55:28.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.785+0000 7f640b7fe700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f63f806ace0 0x7f63f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:28.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.785+0000 7f640b7fe700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f63f806ace0 0x7f63f806d190 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6400006fd0 tx=0x7f6400008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:28.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.789+0000 7f6408ff9700 1 -- 172.21.15.97:0/2943458580 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f63fc057ff0 con 0x7f640c06b360 2023-07-06T03:55:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.988+0000 7f64123ff700 1 -- 172.21.15.97:0/2943458580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) v1 -- 0x7f640c04e340 con 0x7f640c06b360 2023-07-06T03:55:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.988+0000 7f6408ff9700 1 -- 172.21.15.97:0/2943458580 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v58) v1 ==== 106+0+8 (secure 0 0 0) 0x7f63fc057b80 con 0x7f640c06b360 2023-07-06T03:55:28.990 INFO:teuthology.orchestra.run.smithi097.stdout:1048576 2023-07-06T03:55:28.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.990+0000 7f64123ff700 1 -- 172.21.15.97:0/2943458580 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f63f806ace0 msgr2=0x7f63f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.990+0000 7f64123ff700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f63f806ace0 0x7f63f806d190 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6400006fd0 tx=0x7f6400008040 comp rx=0 tx=0).stop 2023-07-06T03:55:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.990+0000 7f64123ff700 1 -- 172.21.15.97:0/2943458580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f640c06b360 msgr2=0x7f640c1508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.990+0000 7f64123ff700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f640c06b360 0x7f640c1508a0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00c7f0 tx=0x7f63fc00cbb0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.991+0000 7f64123ff700 1 -- 172.21.15.97:0/2943458580 shutdown_connections 2023-07-06T03:55:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.991+0000 7f64123ff700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f63f806ace0 0x7f63f806d190 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.991+0000 7f64123ff700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f640c15c610 0x7f640c0cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.991+0000 7f64123ff700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f640c06b360 0x7f640c1508a0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.991+0000 7f64123ff700 1 --2- 172.21.15.97:0/2943458580 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f640c0d2210 0x7f640c150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:28.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.991+0000 7f64123ff700 1 -- 172.21.15.97:0/2943458580 >> 172.21.15.97:0/2943458580 conn(0x7f640c06a6c0 msgr2=0x7f640c15b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:28.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.991+0000 7f64123ff700 1 -- 172.21.15.97:0/2943458580 shutdown_connections 2023-07-06T03:55:28.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:28.991+0000 7f64123ff700 1 -- 172.21.15.97:0/2943458580 wait complete. 2023-07-06T03:55:29.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2023-07-06T03:55:29.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.206+0000 7f452c9ff700 1 Processor -- start 2023-07-06T03:55:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.206+0000 7f452c9ff700 1 -- start start 2023-07-06T03:55:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.207+0000 7f452c9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f452815c610 0x7f45280b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.207+0000 7f452c9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452806b360 0x7f45280b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.207+0000 7f452c9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45280cff90 con 0x7f45280d2210 2023-07-06T03:55:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.207+0000 7f452c9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45280d0780 con 0x7f452815c610 2023-07-06T03:55:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.207+0000 7f452c9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452815c9e0 con 0x7f452806b360 2023-07-06T03:55:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.207+0000 7f4526d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f45280d2210 0x7f45280d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:53572/0 (socket says 172.21.15.97:53572) 2023-07-06T03:55:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.207+0000 7f4526d9d700 1 -- 172.21.15.97:0/2146697200 learned_addr learned my addr 172.21.15.97:0/2146697200 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.207+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466603899 0 0) 0x7f45280cff90 con 0x7f45280d2210 2023-07-06T03:55:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45140035b0 con 0x7f45280d2210 2023-07-06T03:55:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f452659c700 1 --2- 172.21.15.97:0/2146697200 >> v2:172.21.15.146:3300/0 conn(0x7f452815c610 0x7f45280b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f451ffff700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452806b360 0x7f45280b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1199995236 0 0) 0x7f45140035b0 con 0x7f45280d2210 2023-07-06T03:55:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452806b360 msgr2=0x7f45280b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452806b360 0x7f45280b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 >> v2:172.21.15.146:3300/0 conn(0x7f452815c610 msgr2=0x7f45280b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 --2- 172.21.15.97:0/2146697200 >> v2:172.21.15.146:3300/0 conn(0x7f452815c610 0x7f45280b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45280b5650 con 0x7f45280d2210 2023-07-06T03:55:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f45200019f0 con 0x7f45280d2210 2023-07-06T03:55:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 >> v1:172.21.15.97:6789/0 conn(0x7f45280d2210 legacy=0x7f45280d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4514003a30 0x7f4514003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:29.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 0x7f4514004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452806b360 0x7f4514005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45280d0780 con 0x7f4514004850 2023-07-06T03:55:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452815c9e0 con 0x7f4514003a30 2023-07-06T03:55:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451400ba10 con 0x7f452806b360 2023-07-06T03:55:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f451ffff700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 0x7f4514004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f452659c700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4514003a30 0x7f4514003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:29.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.208+0000 7f4526d9d700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452806b360 0x7f4514005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.209+0000 7f451ffff700 1 -- 172.21.15.97:0/2146697200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452806b360 msgr2=0x7f4514005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.209+0000 7f451ffff700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452806b360 0x7f4514005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.209+0000 7f451ffff700 1 -- 172.21.15.97:0/2146697200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4514003a30 msgr2=0x7f4514003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.209+0000 7f451ffff700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4514003a30 0x7f4514003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.209+0000 7f451ffff700 1 -- 172.21.15.97:0/2146697200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140035b0 con 0x7f4514004850 2023-07-06T03:55:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.209+0000 7f451ffff700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 0x7f4514004cc0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f4510009b30 tx=0x7f4514005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d54508c7fc0439 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.209+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451000e6c0 con 0x7f4514004850 2023-07-06T03:55:29.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.209+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451000e820 con 0x7f4514004850 2023-07-06T03:55:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.209+0000 7f4525d9b700 1 -- 172.21.15.97:0/2146697200 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451000eb90 con 0x7f4514004850 2023-07-06T03:55:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.210+0000 7f452c9ff700 1 -- 172.21.15.97:0/2146697200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 msgr2=0x7f4514004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.210+0000 7f452c9ff700 1 --2- 172.21.15.97:0/2146697200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 0x7f4514004cc0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f4510009b30 tx=0x7f4514005880 comp rx=0 tx=0).stop 2023-07-06T03:55:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.210+0000 7f4526d9d700 1 -- 172.21.15.97:0/2146697200 reap_dead start 2023-07-06T03:55:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.210+0000 7f452c9ff700 1 -- 172.21.15.97:0/2146697200 shutdown_connections 2023-07-06T03:55:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.210+0000 7f452c9ff700 1 -- 172.21.15.97:0/2146697200 >> 172.21.15.97:0/2146697200 conn(0x7f452806a6c0 msgr2=0x7f452806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:29.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.210+0000 7f452c9ff700 1 -- 172.21.15.97:0/2146697200 shutdown_connections 2023-07-06T03:55:29.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.210+0000 7f452c9ff700 1 -- 172.21.15.97:0/2146697200 wait complete. 2023-07-06T03:55:29.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.211+0000 7f452c9ff700 1 Processor -- start 2023-07-06T03:55:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.212+0000 7f452c9ff700 1 -- start start 2023-07-06T03:55:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.212+0000 7f452c9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 0x7f452814f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.212+0000 7f452c9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f452814f770 0x7f452814eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452c9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452815ef20 0x7f452815f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452c9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452814f050 con 0x7f4514004850 2023-07-06T03:55:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452c9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45280ccca0 con 0x7f452814f770 2023-07-06T03:55:29.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452c9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452815f8d0 con 0x7f452815ef20 2023-07-06T03:55:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 0x7f452814f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 0x7f452814f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57890/0 (socket says 172.21.15.97:57890) 2023-07-06T03:55:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452659c700 1 -- 172.21.15.97:0/417384318 learned_addr learned my addr 172.21.15.97:0/417384318 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f451ffff700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f452814f770 0x7f452814eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f4526d9d700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452815ef20 0x7f452815f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452659c700 1 -- 172.21.15.97:0/417384318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452815ef20 msgr2=0x7f452815f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:29.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452659c700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452815ef20 0x7f452815f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452659c700 1 -- 172.21.15.97:0/417384318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f452814f770 msgr2=0x7f452814eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452659c700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f452814f770 0x7f452814eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452659c700 1 -- 172.21.15.97:0/417384318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45280ca5f0 con 0x7f4514004850 2023-07-06T03:55:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f451ffff700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f452814f770 0x7f452814eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.213+0000 7f452659c700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 0x7f452814f1b0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f4518011ad0 tx=0x7f4518011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.214+0000 7f451effd700 1 -- 172.21.15.97:0/417384318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451800fa60 con 0x7f4514004850 2023-07-06T03:55:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.214+0000 7f451effd700 1 -- 172.21.15.97:0/417384318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451800fbc0 con 0x7f4514004850 2023-07-06T03:55:29.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.214+0000 7f452c9ff700 1 -- 172.21.15.97:0/417384318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45280caa80 con 0x7f4514004850 2023-07-06T03:55:29.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.214+0000 7f451effd700 1 -- 172.21.15.97:0/417384318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518014440 con 0x7f4514004850 2023-07-06T03:55:29.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.214+0000 7f452c9ff700 1 -- 172.21.15.97:0/417384318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45280c9fa0 con 0x7f4514004850 2023-07-06T03:55:29.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.215+0000 7f452c9ff700 1 -- 172.21.15.97:0/417384318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45280c9de0 con 0x7f4514004850 2023-07-06T03:55:29.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.216+0000 7f451effd700 1 -- 172.21.15.97:0/417384318 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f45180145a0 con 0x7f4514004850 2023-07-06T03:55:29.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.217+0000 7f451effd700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f450806ace0 0x7f450806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:29.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.217+0000 7f451effd700 1 -- 172.21.15.97:0/417384318 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4518089cc0 con 0x7f4514004850 2023-07-06T03:55:29.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.217+0000 7f451ffff700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f450806ace0 0x7f450806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:29.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.218+0000 7f451ffff700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f450806ace0 0x7f450806d190 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f45100090b0 tx=0x7f4510008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:29.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.221+0000 7f451effd700 1 -- 172.21.15.97:0/417384318 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4518090080 con 0x7f4514004850 2023-07-06T03:55:29.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.416+0000 7f452c9ff700 1 -- 172.21.15.97:0/417384318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) v1 -- 0x7f452804e340 con 0x7f4514004850 2023-07-06T03:55:29.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.417+0000 7f451effd700 1 -- 172.21.15.97:0/417384318 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v58) v1 ==== 112+0+10 (secure 0 0 0) 0x7f45180220b0 con 0x7f4514004850 2023-07-06T03:55:29.419 INFO:teuthology.orchestra.run.smithi097.stdout:60.000000 2023-07-06T03:55:29.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 -- 172.21.15.97:0/417384318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f450806ace0 msgr2=0x7f450806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:29.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f450806ace0 0x7f450806d190 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f45100090b0 tx=0x7f4510008030 comp rx=0 tx=0).stop 2023-07-06T03:55:29.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 -- 172.21.15.97:0/417384318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 msgr2=0x7f452814f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:29.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 0x7f452814f1b0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f4518011ad0 tx=0x7f4518011e90 comp rx=0 tx=0).stop 2023-07-06T03:55:29.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 -- 172.21.15.97:0/417384318 shutdown_connections 2023-07-06T03:55:29.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f450806ace0 0x7f450806d190 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:29.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f452815ef20 0x7f452815f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:29.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4514004850 0x7f452814f1b0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:29.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 --2- 172.21.15.97:0/417384318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f452814f770 0x7f452814eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:29.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 -- 172.21.15.97:0/417384318 >> 172.21.15.97:0/417384318 conn(0x7f452806a6c0 msgr2=0x7f452815b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:29.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 -- 172.21.15.97:0/417384318 shutdown_connections 2023-07-06T03:55:29.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:29.419+0000 7f452c9ff700 1 -- 172.21.15.97:0/417384318 wait complete. 2023-07-06T03:55:29.438 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:55:29.439 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:55:29.439 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:55:29.439 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:55:29.440 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:55:29.440 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:29.440 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:55:29.441 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:55:29.441 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:55:29.441 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:55:29.442 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:29.442 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:55:29.442 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:55:29.442 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:55:29.443 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:55:29.443 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:29.443 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:55:29.444 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:55:29.444 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:55:29.444 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:55:29.444 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:55:29.445 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:55:29.445 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:55:29.445 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:55:29.446 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:29.446 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:55:29.446 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:55:29.588 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.163390.asok 2023-07-06T03:55:29.766 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.163390.asok 2023-07-06T03:55:29.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:29.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.163390.asok mds_sessions 2023-07-06T03:55:29.989 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:55:29.989 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9264, 2023-07-06T03:55:29.990 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:55:29.990 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:55:29.990 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:55:29.991 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 9264 2023-07-06T03:55:29.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:29.991 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:55:29.991 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:55:29.991 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:55:29.991 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1280308817 2023-07-06T03:55:29.992 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:29.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:29.992 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.9264 v1:192.168.0.1:0/1280308817", 2023-07-06T03:55:29.992 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "v1:192.168.0.1:0/1280308817", 2023-07-06T03:55:29.992 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions": [ 2023-07-06T03:55:29.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:29.993 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": 0, 2023-07-06T03:55:29.993 INFO:teuthology.orchestra.run.smithi097.stdout: "addrs": { 2023-07-06T03:55:29.993 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-07-06T03:55:29.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:29.993 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-07-06T03:55:29.993 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.146:6832", 2023-07-06T03:55:29.993 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3269084599 2023-07-06T03:55:29.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:29.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:55:29.994 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:55:29.994 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.146:6834", 2023-07-06T03:55:29.994 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3269084599 2023-07-06T03:55:29.994 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:29.994 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:55:29.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:55:29.995 INFO:teuthology.orchestra.run.smithi097.stdout: "seq": 0, 2023-07-06T03:55:29.995 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_gen": 0, 2023-07-06T03:55:29.995 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_ttl": "2023-07-06T03:56:24.366251+0000", 2023-07-06T03:55:29.995 INFO:teuthology.orchestra.run.smithi097.stdout: "last_cap_renew_request": "2023-07-06T03:55:24.366251+0000", 2023-07-06T03:55:29.995 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_renew_seq": 2, 2023-07-06T03:55:29.995 INFO:teuthology.orchestra.run.smithi097.stdout: "num_caps": 1, 2023-07-06T03:55:29.995 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "open" 2023-07-06T03:55:29.996 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:55:29.996 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-07-06T03:55:29.996 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 92 2023-07-06T03:55:29.996 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:55:30.003 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:55:30.003 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:30.049 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:55:30.049 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2023-07-06T03:55:30.049 DEBUG:teuthology.orchestra.run.smithi097:> import time 2023-07-06T03:55:30.050 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:55:30.050 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:30.050 DEBUG:teuthology.orchestra.run.smithi097:> n = 400 2023-07-06T03:55:30.051 DEBUG:teuthology.orchestra.run.smithi097:> abs_path = "/home/ubuntu/cephtest/mnt.0/." 2023-07-06T03:55:30.051 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:30.051 DEBUG:teuthology.orchestra.run.smithi097:> if not os.path.exists(abs_path): 2023-07-06T03:55:30.051 DEBUG:teuthology.orchestra.run.smithi097:> os.makedirs(abs_path) 2023-07-06T03:55:30.052 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:30.052 DEBUG:teuthology.orchestra.run.smithi097:> handles = [] 2023-07-06T03:55:30.052 DEBUG:teuthology.orchestra.run.smithi097:> for i in range(0, n): 2023-07-06T03:55:30.053 DEBUG:teuthology.orchestra.run.smithi097:> fname = "file_"+str(i) 2023-07-06T03:55:30.053 DEBUG:teuthology.orchestra.run.smithi097:> path = os.path.join(abs_path, fname) 2023-07-06T03:55:30.053 DEBUG:teuthology.orchestra.run.smithi097:> handles.append(open(path, '"'"'w'"'"')) 2023-07-06T03:55:30.054 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:55:30.054 DEBUG:teuthology.orchestra.run.smithi097:> while True: 2023-07-06T03:55:30.054 DEBUG:teuthology.orchestra.run.smithi097:> time.sleep(1) 2023-07-06T03:55:30.055 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:55:30.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:55:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.293+0000 7f6a0a7ff700 1 Processor -- start 2023-07-06T03:55:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.293+0000 7f6a0a7ff700 1 -- start start 2023-07-06T03:55:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.294+0000 7f6a0a7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6a0415c520 0x7f6a040cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.294+0000 7f6a0a7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6a04160180 0x7f6a041605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.294+0000 7f6a0a7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a04160b30 0x7f6a040cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.294+0000 7f6a0a7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a040cd8d0 con 0x7f6a04160180 2023-07-06T03:55:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.294+0000 7f6a0a7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a040cdae0 con 0x7f6a0415c520 2023-07-06T03:55:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.294+0000 7f6a0a7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040b6640 con 0x7f6a04160b30 2023-07-06T03:55:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.295+0000 7f6a037fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6a04160180 0x7f6a041605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.295+0000 7f6a037fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6a04160180 0x7f6a041605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58280/0 (socket says 172.21.15.97:58280) 2023-07-06T03:55:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.295+0000 7f6a037fe700 1 -- 172.21.15.97:0/2500126006 learned_addr learned my addr 172.21.15.97:0/2500126006 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.295+0000 7f6a03fff700 1 --2- 172.21.15.97:0/2500126006 >> v2:172.21.15.146:3300/0 conn(0x7f6a0415c520 0x7f6a040cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:30.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.295+0000 7f6a08d9c700 1 --2- 172.21.15.97:0/2500126006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a04160b30 0x7f6a040cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:30.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.295+0000 7f6a037fe700 1 -- 172.21.15.97:0/2500126006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a04160b30 msgr2=0x7f6a040cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:30.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.295+0000 7f6a037fe700 1 --2- 172.21.15.97:0/2500126006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a04160b30 0x7f6a040cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.295+0000 7f6a037fe700 1 -- 172.21.15.97:0/2500126006 >> v2:172.21.15.146:3300/0 conn(0x7f6a0415c520 msgr2=0x7f6a040cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:30.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.295+0000 7f6a037fe700 1 --2- 172.21.15.97:0/2500126006 >> v2:172.21.15.146:3300/0 conn(0x7f6a0415c520 0x7f6a040cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.295+0000 7f6a037fe700 1 -- 172.21.15.97:0/2500126006 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a040b6920 con 0x7f6a04160180 2023-07-06T03:55:30.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.295+0000 7f6a037fe700 1 --2- 172.21.15.97:0/2500126006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a04160180 0x7f6a041605f0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f6a040b4c70 tx=0x7f69f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42086292e0dec02d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:30.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a02ffd700 1 -- 172.21.15.97:0/2500126006 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8011830 con 0x7f6a04160180 2023-07-06T03:55:30.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a02ffd700 1 -- 172.21.15.97:0/2500126006 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f8011990 con 0x7f6a04160180 2023-07-06T03:55:30.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a02ffd700 1 -- 172.21.15.97:0/2500126006 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8011d20 con 0x7f6a04160180 2023-07-06T03:55:30.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/2500126006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a04160180 msgr2=0x7f6a041605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:30.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a0a7ff700 1 --2- 172.21.15.97:0/2500126006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a04160180 0x7f6a041605f0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f6a040b4c70 tx=0x7f69f8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/2500126006 shutdown_connections 2023-07-06T03:55:30.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a0a7ff700 1 --2- 172.21.15.97:0/2500126006 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a04160b30 0x7f6a040cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a0a7ff700 1 --2- 172.21.15.97:0/2500126006 >> v2:172.21.15.146:3300/0 conn(0x7f6a0415c520 0x7f6a040cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a0a7ff700 1 --2- 172.21.15.97:0/2500126006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a04160180 0x7f6a041605f0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/2500126006 >> 172.21.15.97:0/2500126006 conn(0x7f6a04071790 msgr2=0x7f6a04071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:30.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/2500126006 shutdown_connections 2023-07-06T03:55:30.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.296+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/2500126006 wait complete. 2023-07-06T03:55:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.298+0000 7f6a0a7ff700 1 Processor -- start 2023-07-06T03:55:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.298+0000 7f6a0a7ff700 1 -- start start 2023-07-06T03:55:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.299+0000 7f6a0a7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a0415c520 0x7f6a040d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.299+0000 7f6a0a7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a04160180 0x7f6a040d1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.299+0000 7f6a0a7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a04160b30 0x7f6a041588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.299+0000 7f6a0a7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040d3bf0 con 0x7f6a04160b30 2023-07-06T03:55:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.299+0000 7f6a0a7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040d3630 con 0x7f6a0415c520 2023-07-06T03:55:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.299+0000 7f6a0a7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040d3910 con 0x7f6a04160180 2023-07-06T03:55:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.300+0000 7f6a08d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a04160b30 0x7f6a041588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.300+0000 7f6a08d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a04160b30 0x7f6a041588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58290/0 (socket says 172.21.15.97:58290) 2023-07-06T03:55:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.300+0000 7f6a08d9c700 1 -- 172.21.15.97:0/3727512685 learned_addr learned my addr 172.21.15.97:0/3727512685 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.300+0000 7f6a03fff700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a0415c520 0x7f6a040d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.300+0000 7f6a037fe700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a04160180 0x7f6a040d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.300+0000 7f6a08d9c700 1 -- 172.21.15.97:0/3727512685 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a04160180 msgr2=0x7f6a040d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.300+0000 7f6a08d9c700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a04160180 0x7f6a040d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.300+0000 7f6a08d9c700 1 -- 172.21.15.97:0/3727512685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a0415c520 msgr2=0x7f6a040d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:30.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.300+0000 7f6a08d9c700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a0415c520 0x7f6a040d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.300+0000 7f6a08d9c700 1 -- 172.21.15.97:0/3727512685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8009550 con 0x7f6a04160b30 2023-07-06T03:55:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.300+0000 7f6a08d9c700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a04160b30 0x7f6a041588b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00efc0 tx=0x7f69fc00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.301+0000 7f6a00ff9700 1 -- 172.21.15.97:0/3727512685 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00cad0 con 0x7f6a04160b30 2023-07-06T03:55:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.301+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/3727512685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0415c990 con 0x7f6a04160b30 2023-07-06T03:55:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.301+0000 7f6a00ff9700 1 -- 172.21.15.97:0/3727512685 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc00f040 con 0x7f6a04160b30 2023-07-06T03:55:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.301+0000 7f6a00ff9700 1 -- 172.21.15.97:0/3727512685 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00ce70 con 0x7f6a04160b30 2023-07-06T03:55:30.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.301+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/3727512685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a040d3300 con 0x7f6a04160b30 2023-07-06T03:55:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.302+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/3727512685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0415ceb0 con 0x7f6a04160b30 2023-07-06T03:55:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.303+0000 7f6a00ff9700 1 -- 172.21.15.97:0/3727512685 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f69fc014480 con 0x7f6a04160b30 2023-07-06T03:55:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.304+0000 7f6a00ff9700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f69f006ace0 0x7f69f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.304+0000 7f6a00ff9700 1 -- 172.21.15.97:0/3727512685 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f69fc089c60 con 0x7f6a04160b30 2023-07-06T03:55:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.304+0000 7f6a03fff700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f69f006ace0 0x7f69f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.305+0000 7f6a03fff700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f69f006ace0 0x7f69f006d190 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009be0 tx=0x7f69f4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.308+0000 7f6a00ff9700 1 -- 172.21.15.97:0/3727512685 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6a0415ceb0 con 0x7f6a04160b30 2023-07-06T03:55:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.584+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/3727512685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a0415ceb0 con 0x7f6a04160b30 2023-07-06T03:55:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.584+0000 7f6a00ff9700 1 -- 172.21.15.97:0/3727512685 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f6a0415ceb0 con 0x7f6a04160b30 2023-07-06T03:55:30.586 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:30.586 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":92,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3059444241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3059444241},{"type":"v1","addr":"172.21.15.146:6835","nonce":3059444241}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2624688851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2624688851},{"type":"v1","addr":"172.21.15.97:6837","nonce":2624688851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3938376776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3938376776},{"type":"v1","addr":"172.21.15.97:6835","nonce":3938376776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:55:14.806943+0000","modified":"2023-07-06T03:55:18.341811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9099":{"gid":9099,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6834/3269084599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3269084599},{"type":"v1","addr":"172.21.15.146:6834","nonce":3269084599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-06T03:55:30.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/3727512685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f69f006ace0 msgr2=0x7f69f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:30.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f69f006ace0 0x7f69f006d190 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009be0 tx=0x7f69f4009210 comp rx=0 tx=0).stop 2023-07-06T03:55:30.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/3727512685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a04160b30 msgr2=0x7f6a041588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:30.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a04160b30 0x7f6a041588b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00efc0 tx=0x7f69fc00c0c0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/3727512685 shutdown_connections 2023-07-06T03:55:30.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f69f006ace0 0x7f69f006d190 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a04160b30 0x7f6a041588b0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a0415c520 0x7f6a040d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 --2- 172.21.15.97:0/3727512685 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6a04160180 0x7f6a040d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:30.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/3727512685 >> 172.21.15.97:0/3727512685 conn(0x7f6a04071790 msgr2=0x7f6a040b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:30.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/3727512685 shutdown_connections 2023-07-06T03:55:30.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:30.587+0000 7f6a0a7ff700 1 -- 172.21.15.97:0/3727512685 wait complete. 2023-07-06T03:55:30.591 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 92 2023-07-06T03:55:30.607 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-07-06T03:55:30.843 INFO:teuthology.orchestra.run.smithi146.stdout:[{"id":7286,"entity":{"name":{"type":"client","num":7286},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1807327696}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":5.1979193979999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9852014066726489,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7286 192.168.0.1:0/1807327696","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"126921","root":"/"}},{"id":9264,"entity":{"name":{"type":"client","num":9264},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1280308817}},"state":"open","num_leases":228,"num_caps":229,"request_load_avg":683,"uptime":7.4578843539999999,"requests_in_flight":227,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":684.50492768054585,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":7,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.9264 192.168.0.1:0/1280308817","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"163390","root":"/"}}] 2023-07-06T03:55:30.844 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "126921", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1807327696, "type": "any" }, "name": { "num": 7286, "type": "client" } }, "id": 7286, "inst": "client.7286 192.168.0.1:0/1807327696", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.985201406672649 }, "state": "open", "uptime": 5.197919398 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "163390", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1280308817, "type": "any" }, "name": { "num": 9264, "type": "client" } }, "id": 9264, "inst": "client.9264 192.168.0.1:0/1280308817", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 7, "num_caps": 229, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 228, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 683, "requests_in_flight": 227, "session_cache_liveness": { "halflife": 300, "value": 684.5049276805458 }, "state": "open", "uptime": 7.457884354 } ] 2023-07-06T03:55:30.844 DEBUG:tasks.ceph_test_case:wait_until_equal: 229 != 401, waiting (timeout=600)... 2023-07-06T03:55:35.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:55:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.051+0000 7fea08fff700 1 Processor -- start 2023-07-06T03:55:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.052+0000 7fea08fff700 1 -- start start 2023-07-06T03:55:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.053+0000 7fea08fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea0415c520 0x7fea040cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.053+0000 7fea08fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea040cdae0 con 0x7fea04160180 2023-07-06T03:55:36.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.053+0000 7fea08fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea040b6640 con 0x7fea04160d00 2023-07-06T03:55:36.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.053+0000 7fea08fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea040cd8d0 con 0x7fea0415c520 2023-07-06T03:55:36.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.053+0000 7fe9fbfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fea04160180 0x7fea041605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33524/0 (socket says 172.21.15.97:33524) 2023-07-06T03:55:36.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.053+0000 7fe9fbfff700 1 -- 172.21.15.97:0/1159072371 learned_addr learned my addr 172.21.15.97:0/1159072371 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:36.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.053+0000 7fea0259c700 1 --2- 172.21.15.97:0/1159072371 >> v2:172.21.15.97:3300/0 conn(0x7fea0415c520 0x7fea040cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:36.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.054+0000 7fea0259c700 1 -- 172.21.15.97:0/1159072371 >> v1:172.21.15.146:6789/0 conn(0x7fea04160d00 legacy=0x7fea040b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:36.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.054+0000 7fea0259c700 1 -- 172.21.15.97:0/1159072371 >> v1:172.21.15.97:6789/0 conn(0x7fea04160180 legacy=0x7fea041605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:36.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.054+0000 7fea0259c700 1 -- 172.21.15.97:0/1159072371 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040b6920 con 0x7fea0415c520 2023-07-06T03:55:36.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.054+0000 7fea0259c700 1 --2- 172.21.15.97:0/1159072371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0415c520 0x7fea040cd390 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009840 tx=0x7fe9f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efbfc6286fe0f77 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:36.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.054+0000 7fea01d9b700 1 -- 172.21.15.97:0/1159072371 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4004230 con 0x7fea0415c520 2023-07-06T03:55:36.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.054+0000 7fea01d9b700 1 -- 172.21.15.97:0/1159072371 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4004390 con 0x7fea0415c520 2023-07-06T03:55:36.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.054+0000 7fea01d9b700 1 -- 172.21.15.97:0/1159072371 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40036a0 con 0x7fea0415c520 2023-07-06T03:55:36.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.055+0000 7fea08fff700 1 -- 172.21.15.97:0/1159072371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0415c520 msgr2=0x7fea040cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:36.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.055+0000 7fea08fff700 1 --2- 172.21.15.97:0/1159072371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0415c520 0x7fea040cd390 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009840 tx=0x7fe9f4009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:36.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.055+0000 7fea08fff700 1 -- 172.21.15.97:0/1159072371 shutdown_connections 2023-07-06T03:55:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.055+0000 7fea08fff700 1 --2- 172.21.15.97:0/1159072371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea0415c520 0x7fea040cd390 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.055+0000 7fea08fff700 1 -- 172.21.15.97:0/1159072371 >> 172.21.15.97:0/1159072371 conn(0x7fea04071790 msgr2=0x7fea04071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.055+0000 7fea08fff700 1 -- 172.21.15.97:0/1159072371 shutdown_connections 2023-07-06T03:55:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.055+0000 7fea08fff700 1 -- 172.21.15.97:0/1159072371 wait complete. 2023-07-06T03:55:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.056+0000 7fea08fff700 1 Processor -- start 2023-07-06T03:55:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.057+0000 7fea08fff700 1 -- start start 2023-07-06T03:55:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.057+0000 7fea08fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea0415c520 0x7fea0414d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea08fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea04160180 0x7fea0415f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea08fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04160d00 0x7fea04153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea08fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041531f0 con 0x7fea04160d00 2023-07-06T03:55:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea08fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea04152600 con 0x7fea0415c520 2023-07-06T03:55:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea08fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea041528c0 con 0x7fea04160180 2023-07-06T03:55:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea02d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04160d00 0x7fea04153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea02d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04160d00 0x7fea04153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58318/0 (socket says 172.21.15.97:58318) 2023-07-06T03:55:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea02d9d700 1 -- 172.21.15.97:0/3824159144 learned_addr learned my addr 172.21.15.97:0/3824159144 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea0259c700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea0415c520 0x7fea0414d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fe9fbfff700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea04160180 0x7fea0415f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea02d9d700 1 -- 172.21.15.97:0/3824159144 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea04160180 msgr2=0x7fea0415f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea02d9d700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea04160180 0x7fea0415f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea02d9d700 1 -- 172.21.15.97:0/3824159144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea0415c520 msgr2=0x7fea0414d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea02d9d700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea0415c520 0x7fea0414d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea02d9d700 1 -- 172.21.15.97:0/3824159144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f40094a0 con 0x7fea04160d00 2023-07-06T03:55:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fe9fbfff700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea04160180 0x7fea0415f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.058+0000 7fea0259c700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea0415c520 0x7fea0414d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.059+0000 7fea02d9d700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04160d00 0x7fea04153680 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00cc10 tx=0x7fe9fc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.059+0000 7fe9faffd700 1 -- 172.21.15.97:0/3824159144 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00abb0 con 0x7fea04160d00 2023-07-06T03:55:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.059+0000 7fe9faffd700 1 -- 172.21.15.97:0/3824159144 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc00ad10 con 0x7fea04160d00 2023-07-06T03:55:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.059+0000 7fea08fff700 1 -- 172.21.15.97:0/3824159144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea04152ba0 con 0x7fea04160d00 2023-07-06T03:55:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.059+0000 7fe9faffd700 1 -- 172.21.15.97:0/3824159144 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc012660 con 0x7fea04160d00 2023-07-06T03:55:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.059+0000 7fea08fff700 1 -- 172.21.15.97:0/3824159144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea041f2e20 con 0x7fea04160d00 2023-07-06T03:55:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.060+0000 7fea08fff700 1 -- 172.21.15.97:0/3824159144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea041f3010 con 0x7fea04160d00 2023-07-06T03:55:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.061+0000 7fe9faffd700 1 -- 172.21.15.97:0/3824159144 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe9fc0127c0 con 0x7fea04160d00 2023-07-06T03:55:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.062+0000 7fe9faffd700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9e406ace0 0x7fe9e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.062+0000 7fe9faffd700 1 -- 172.21.15.97:0/3824159144 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9fc089c40 con 0x7fea04160d00 2023-07-06T03:55:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.062+0000 7fea0259c700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9e406ace0 0x7fe9e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.063+0000 7fea0259c700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9e406ace0 0x7fe9e406d190 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40090b0 tx=0x7fe9f4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.067+0000 7fe9faffd700 1 -- 172.21.15.97:0/3824159144 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe9fc054ce0 con 0x7fea04160d00 2023-07-06T03:55:36.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.302+0000 7fea08fff700 1 -- 172.21.15.97:0/3824159144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea0404e340 con 0x7fea04160d00 2023-07-06T03:55:36.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.303+0000 7fe9faffd700 1 -- 172.21.15.97:0/3824159144 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4679 (secure 0 0 0) 0x7fe9fc016020 con 0x7fea04160d00 2023-07-06T03:55:36.305 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:36.305 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3059444241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3059444241},{"type":"v1","addr":"172.21.15.146:6835","nonce":3059444241}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2624688851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2624688851},{"type":"v1","addr":"172.21.15.97:6837","nonce":2624688851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3938376776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3938376776},{"type":"v1","addr":"172.21.15.97:6835","nonce":3938376776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:55:14.806943+0000","modified":"2023-07-06T03:55:35.906574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9099":{"gid":9099,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6834/3269084599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3269084599},{"type":"v1","addr":"172.21.15.146:6834","nonce":3269084599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-06T03:55:36.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 -- 172.21.15.97:0/3824159144 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9e406ace0 msgr2=0x7fe9e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:36.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9e406ace0 0x7fe9e406d190 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40090b0 tx=0x7fe9f4008030 comp rx=0 tx=0).stop 2023-07-06T03:55:36.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 -- 172.21.15.97:0/3824159144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04160d00 msgr2=0x7fea04153680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:36.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04160d00 0x7fea04153680 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00cc10 tx=0x7fe9fc00a060 comp rx=0 tx=0).stop 2023-07-06T03:55:36.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 -- 172.21.15.97:0/3824159144 shutdown_connections 2023-07-06T03:55:36.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe9e406ace0 0x7fe9e406d190 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea04160d00 0x7fea04153680 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea0415c520 0x7fea0414d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 --2- 172.21.15.97:0/3824159144 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fea04160180 0x7fea0415f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 -- 172.21.15.97:0/3824159144 >> 172.21.15.97:0/3824159144 conn(0x7fea04071790 msgr2=0x7fea040b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:36.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 -- 172.21.15.97:0/3824159144 shutdown_connections 2023-07-06T03:55:36.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.305+0000 7fea08fff700 1 -- 172.21.15.97:0/3824159144 wait complete. 2023-07-06T03:55:36.309 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 94 2023-07-06T03:55:36.324 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-07-06T03:55:36.552 INFO:teuthology.orchestra.run.smithi146.stdout:[{"id":7286,"entity":{"name":{"type":"client","num":7286},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1807327696}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":10.908830841,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9460703873591192,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7286 192.168.0.1:0/1807327696","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"126921","root":"/"}},{"id":9264,"entity":{"name":{"type":"client","num":9264},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1280308817}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":1124,"uptime":13.168795797,"requests_in_flight":0,"num_completed_requests":399,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1163.6283746887048,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":466.28879032422282,"halflife":1.5},"recall_caps_throttle2o":{"value":207.98889302344605,"halflife":0.5},"session_cache_liveness":{"value":1185.0601591212203,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":7,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.9264 192.168.0.1:0/1280308817","completed_requests":[{"tid":7,"created_ino":"0x10000000001"},{"tid":10,"created_ino":"0x100000001f6"},{"tid":13,"created_ino":"0x100000001f7"},{"tid":16,"created_ino":"0x100000001f8"},{"tid":19,"created_ino":"0x100000001f9"},{"tid":22,"created_ino":"0x100000001fa"},{"tid":25,"created_ino":"0x100000001fb"},{"tid":28,"created_ino":"0x100000001fc"},{"tid":31,"created_ino":"0x100000001fd"},{"tid":34,"created_ino":"0x100000001fe"},{"tid":37,"created_ino":"0x100000001ff"},{"tid":40,"created_ino":"0x10000000200"},{"tid":43,"created_ino":"0x10000000201"},{"tid":46,"created_ino":"0x10000000202"},{"tid":49,"created_ino":"0x10000000203"},{"tid":52,"created_ino":"0x10000000204"},{"tid":55,"created_ino":"0x10000000205"},{"tid":58,"created_ino":"0x10000000206"},{"tid":61,"created_ino":"0x10000000207"},{"tid":64,"created_ino":"0x10000000208"},{"tid":67,"created_ino":"0x10000000209"},{"tid":70,"created_ino":"0x1000000020a"},{"tid":73,"created_ino":"0x1000000020b"},{"tid":76,"created_ino":"0x1000000020c"},{"tid":79,"created_ino":"0x1000000020d"},{"tid":82,"created_ino":"0x1000000020e"},{"tid":85,"created_ino":"0x1000000020f"},{"tid":88,"created_ino":"0x10000000210"},{"tid":91,"created_ino":"0x10000000211"},{"tid":94,"created_ino":"0x10000000212"},{"tid":97,"created_ino":"0x10000000213"},{"tid":100,"created_ino":"0x10000000214"},{"tid":103,"created_ino":"0x10000000215"},{"tid":106,"created_ino":"0x10000000216"},{"tid":109,"created_ino":"0x10000000217"},{"tid":112,"created_ino":"0x10000000218"},{"tid":115,"created_ino":"0x10000000219"},{"tid":118,"created_ino":"0x1000000021a"},{"tid":121,"created_ino":"0x1000000021b"},{"tid":124,"created_ino":"0x1000000021c"},{"tid":127,"created_ino":"0x1000000021d"},{"tid":130,"created_ino":"0x1000000021e"},{"tid":133,"created_ino":"0x1000000021f"},{"tid":136,"created_ino":"0x10000000220"},{"tid":139,"created_ino":"0x10000000221"},{"tid":142,"created_ino":"0x10000000222"},{"tid":145,"created_ino":"0x10000000223"},{"tid":148,"created_ino":"0x10000000224"},{"tid":151,"created_ino":"0x10000000225"},{"tid":154,"created_ino":"0x10000000226"},{"tid":157,"created_ino":"0x10000000227"},{"tid":160,"created_ino":"0x10000000228"},{"tid":163,"created_ino":"0x10000000229"},{"tid":166,"created_ino":"0x1000000022a"},{"tid":169,"created_ino":"0x1000000022b"},{"tid":172,"created_ino":"0x1000000022c"},{"tid":175,"created_ino":"0x1000000022d"},{"tid":178,"created_ino":"0x1000000022e"},{"tid":181,"created_ino":"0x1000000022f"},{"tid":184,"created_ino":"0x10000000230"},{"tid":187,"created_ino":"0x10000000231"},{"tid":190,"created_ino":"0x10000000232"},{"tid":193,"created_ino":"0x10000000233"},{"tid":196,"created_ino":"0x10000000234"},{"tid":199,"created_ino":"0x10000000235"},{"tid":202,"created_ino":"0x10000000236"},{"tid":205,"created_ino":"0x10000000237"},{"tid":208,"created_ino":"0x10000000238"},{"tid":211,"created_ino":"0x10000000239"},{"tid":214,"created_ino":"0x1000000023a"},{"tid":217,"created_ino":"0x1000000023b"},{"tid":220,"created_ino":"0x1000000023c"},{"tid":223,"created_ino":"0x1000000023d"},{"tid":226,"created_ino":"0x1000000023e"},{"tid":229,"created_ino":"0x1000000023f"},{"tid":232,"created_ino":"0x10000000240"},{"tid":235,"created_ino":"0x10000000241"},{"tid":238,"created_ino":"0x10000000242"},{"tid":241,"created_ino":"0x10000000243"},{"tid":244,"created_ino":"0x10000000244"},{"tid":247,"created_ino":"0x10000000245"},{"tid":250,"created_ino":"0x10000000246"},{"tid":253,"created_ino":"0x10000000247"},{"tid":256,"created_ino":"0x10000000248"},{"tid":259,"created_ino":"0x10000000249"},{"tid":262,"created_ino":"0x1000000024a"},{"tid":265,"created_ino":"0x1000000024b"},{"tid":268,"created_ino":"0x1000000024c"},{"tid":271,"created_ino":"0x1000000024d"},{"tid":274,"created_ino":"0x1000000024e"},{"tid":277,"created_ino":"0x1000000024f"},{"tid":280,"created_ino":"0x10000000250"},{"tid":283,"created_ino":"0x10000000251"},{"tid":286,"created_ino":"0x10000000252"},{"tid":289,"created_ino":"0x10000000253"},{"tid":292,"created_ino":"0x10000000254"},{"tid":295,"created_ino":"0x10000000255"},{"tid":298,"created_ino":"0x10000000256"},{"tid":301,"created_ino":"0x10000000257"},{"tid":304,"created_ino":"0x10000000258"},{"tid":307,"created_ino":"0x10000000259"},{"tid":310,"created_ino":"0x1000000025a"},{"tid":313,"created_ino":"0x1000000025b"},{"tid":316,"created_ino":"0x1000000025c"},{"tid":319,"created_ino":"0x1000000025d"},{"tid":322,"created_ino":"0x1000000025e"},{"tid":325,"created_ino":"0x1000000025f"},{"tid":328,"created_ino":"0x10000000260"},{"tid":331,"created_ino":"0x10000000261"},{"tid":334,"created_ino":"0x10000000262"},{"tid":337,"created_ino":"0x10000000263"},{"tid":340,"created_ino":"0x10000000264"},{"tid":343,"created_ino":"0x10000000265"},{"tid":346,"created_ino":"0x10000000266"},{"tid":349,"created_ino":"0x10000000267"},{"tid":352,"created_ino":"0x10000000268"},{"tid":355,"created_ino":"0x10000000269"},{"tid":358,"created_ino":"0x1000000026a"},{"tid":361,"created_ino":"0x1000000026b"},{"tid":364,"created_ino":"0x1000000026c"},{"tid":367,"created_ino":"0x1000000026d"},{"tid":370,"created_ino":"0x1000000026e"},{"tid":373,"created_ino":"0x1000000026f"},{"tid":376,"created_ino":"0x10000000270"},{"tid":379,"created_ino":"0x10000000271"},{"tid":382,"created_ino":"0x10000000272"},{"tid":385,"created_ino":"0x10000000273"},{"tid":388,"created_ino":"0x10000000274"},{"tid":391,"created_ino":"0x10000000275"},{"tid":394,"created_ino":"0x10000000276"},{"tid":397,"created_ino":"0x10000000277"},{"tid":400,"created_ino":"0x10000000278"},{"tid":403,"created_ino":"0x10000000279"},{"tid":406,"created_ino":"0x1000000027a"},{"tid":409,"created_ino":"0x1000000027b"},{"tid":412,"created_ino":"0x1000000027c"},{"tid":415,"created_ino":"0x1000000027d"},{"tid":418,"created_ino":"0x1000000027e"},{"tid":421,"created_ino":"0x1000000027f"},{"tid":424,"created_ino":"0x10000000280"},{"tid":427,"created_ino":"0x10000000281"},{"tid":430,"created_ino":"0x10000000282"},{"tid":433,"created_ino":"0x10000000283"},{"tid":436,"created_ino":"0x10000000284"},{"tid":439,"created_ino":"0x10000000285"},{"tid":442,"created_ino":"0x10000000286"},{"tid":445,"created_ino":"0x10000000287"},{"tid":448,"created_ino":"0x10000000288"},{"tid":451,"created_ino":"0x10000000289"},{"tid":454,"created_ino":"0x1000000028a"},{"tid":457,"created_ino":"0x1000000028b"},{"tid":460,"created_ino":"0x1000000028c"},{"tid":463,"created_ino":"0x1000000028d"},{"tid":466,"created_ino":"0x1000000028e"},{"tid":469,"created_ino":"0x1000000028f"},{"tid":472,"created_ino":"0x10000000290"},{"tid":475,"created_ino":"0x10000000291"},{"tid":478,"created_ino":"0x10000000292"},{"tid":481,"created_ino":"0x10000000293"},{"tid":484,"created_ino":"0x10000000294"},{"tid":487,"created_ino":"0x10000000295"},{"tid":490,"created_ino":"0x10000000296"},{"tid":493,"created_ino":"0x10000000297"},{"tid":496,"created_ino":"0x10000000298"},{"tid":499,"created_ino":"0x10000000299"},{"tid":502,"created_ino":"0x1000000029a"},{"tid":505,"created_ino":"0x1000000029b"},{"tid":508,"created_ino":"0x1000000029c"},{"tid":511,"created_ino":"0x1000000029d"},{"tid":514,"created_ino":"0x1000000029e"},{"tid":517,"created_ino":"0x1000000029f"},{"tid":520,"created_ino":"0x100000002a0"},{"tid":523,"created_ino":"0x100000002a1"},{"tid":526,"created_ino":"0x100000002a2"},{"tid":529,"created_ino":"0x100000002a3"},{"tid":532,"created_ino":"0x100000002a4"},{"tid":535,"created_ino":"0x100000002a5"},{"tid":538,"created_ino":"0x100000002a6"},{"tid":541,"created_ino":"0x100000002a7"},{"tid":544,"created_ino":"0x100000002a8"},{"tid":547,"created_ino":"0x100000002a9"},{"tid":550,"created_ino":"0x100000002aa"},{"tid":553,"created_ino":"0x100000002ab"},{"tid":556,"created_ino":"0x100000002ac"},{"tid":559,"created_ino":"0x100000002ad"},{"tid":562,"created_ino":"0x100000002ae"},{"tid":565,"created_ino":"0x100000002af"},{"tid":568,"created_ino":"0x100000002b0"},{"tid":571,"created_ino":"0x100000002b1"},{"tid":574,"created_ino":"0x100000002b2"},{"tid":577,"created_ino":"0x100000002b3"},{"tid":580,"created_ino":"0x100000002b4"},{"tid":583,"created_ino":"0x100000002b5"},{"tid":586,"created_ino":"0x100000002b6"},{"tid":589,"created_ino":"0x100000002b7"},{"tid":592,"created_ino":"0x100000002b8"},{"tid":595,"created_ino":"0x100000002b9"},{"tid":598,"created_ino":"0x100000002ba"},{"tid":601,"created_ino":"0x100000002bb"},{"tid":604,"created_ino":"0x100000002bc"},{"tid":607,"created_ino":"0x100000002bd"},{"tid":610,"created_ino":"0x100000002be"},{"tid":613,"created_ino":"0x100000002bf"},{"tid":616,"created_ino":"0x100000002c0"},{"tid":619,"created_ino":"0x100000002c1"},{"tid":622,"created_ino":"0x100000002c2"},{"tid":625,"created_ino":"0x100000002c3"},{"tid":628,"created_ino":"0x100000002c4"},{"tid":631,"created_ino":"0x100000002c5"},{"tid":634,"created_ino":"0x100000002c6"},{"tid":637,"created_ino":"0x100000002c7"},{"tid":640,"created_ino":"0x100000002c8"},{"tid":643,"created_ino":"0x100000002c9"},{"tid":646,"created_ino":"0x100000002ca"},{"tid":649,"created_ino":"0x100000002cb"},{"tid":652,"created_ino":"0x100000002cc"},{"tid":655,"created_ino":"0x100000002cd"},{"tid":658,"created_ino":"0x100000002ce"},{"tid":661,"created_ino":"0x100000002cf"},{"tid":664,"created_ino":"0x100000002d0"},{"tid":667,"created_ino":"0x100000002d1"},{"tid":670,"created_ino":"0x100000002d2"},{"tid":673,"created_ino":"0x100000002d3"},{"tid":676,"created_ino":"0x100000002d4"},{"tid":679,"created_ino":"0x100000002d5"},{"tid":682,"created_ino":"0x100000002d6"},{"tid":685,"created_ino":"0x100000002d7"},{"tid":688,"created_ino":"0x100000002d8"},{"tid":691,"created_ino":"0x100000002d9"},{"tid":694,"created_ino":"0x100000002da"},{"tid":697,"created_ino":"0x100000002db"},{"tid":700,"created_ino":"0x100000002dc"},{"tid":703,"created_ino":"0x100000002dd"},{"tid":706,"created_ino":"0x100000002de"},{"tid":709,"created_ino":"0x100000002df"},{"tid":712,"created_ino":"0x100000002e0"},{"tid":715,"created_ino":"0x100000002e1"},{"tid":718,"created_ino":"0x100000002e2"},{"tid":721,"created_ino":"0x100000002e3"},{"tid":724,"created_ino":"0x100000002e4"},{"tid":727,"created_ino":"0x100000002e5"},{"tid":730,"created_ino":"0x100000002e6"},{"tid":733,"created_ino":"0x100000002e7"},{"tid":736,"created_ino":"0x100000002e8"},{"tid":739,"created_ino":"0x100000002e9"},{"tid":742,"created_ino":"0x100000002ea"},{"tid":745,"created_ino":"0x100000002eb"},{"tid":748,"created_ino":"0x100000002ec"},{"tid":751,"created_ino":"0x100000002ed"},{"tid":754,"created_ino":"0x100000002ee"},{"tid":757,"created_ino":"0x100000002ef"},{"tid":760,"created_ino":"0x100000002f0"},{"tid":763,"created_ino":"0x100000002f1"},{"tid":766,"created_ino":"0x100000002f2"},{"tid":769,"created_ino":"0x100000002f3"},{"tid":772,"created_ino":"0x100000002f4"},{"tid":775,"created_ino":"0x100000002f5"},{"tid":778,"created_ino":"0x100000002f6"},{"tid":781,"created_ino":"0x100000002f7"},{"tid":784,"created_ino":"0x100000002f8"},{"tid":787,"created_ino":"0x100000002f9"},{"tid":790,"created_ino":"0x100000002fa"},{"tid":793,"created_ino":"0x100000002fb"},{"tid":796,"created_ino":"0x100000002fc"},{"tid":799,"created_ino":"0x100000002fd"},{"tid":802,"created_ino":"0x100000002fe"},{"tid":805,"created_ino":"0x100000002ff"},{"tid":808,"created_ino":"0x10000000300"},{"tid":811,"created_ino":"0x10000000301"},{"tid":814,"created_ino":"0x10000000302"},{"tid":817,"created_ino":"0x10000000303"},{"tid":820,"created_ino":"0x10000000304"},{"tid":823,"created_ino":"0x10000000305"},{"tid":826,"created_ino":"0x10000000306"},{"tid":829,"created_ino":"0x10000000307"},{"tid":832,"created_ino":"0x10000000308"},{"tid":835,"created_ino":"0x10000000309"},{"tid":838,"created_ino":"0x1000000030a"},{"tid":841,"created_ino":"0x1000000030b"},{"tid":844,"created_ino":"0x1000000030c"},{"tid":847,"created_ino":"0x1000000030d"},{"tid":850,"created_ino":"0x1000000030e"},{"tid":853,"created_ino":"0x1000000030f"},{"tid":856,"created_ino":"0x10000000310"},{"tid":859,"created_ino":"0x10000000311"},{"tid":862,"created_ino":"0x10000000312"},{"tid":865,"created_ino":"0x10000000313"},{"tid":868,"created_ino":"0x10000000314"},{"tid":871,"created_ino":"0x10000000315"},{"tid":874,"created_ino":"0x10000000316"},{"tid":877,"created_ino":"0x10000000317"},{"tid":880,"created_ino":"0x10000000318"},{"tid":883,"created_ino":"0x10000000319"},{"tid":886,"created_ino":"0x1000000031a"},{"tid":889,"created_ino":"0x1000000031b"},{"tid":892,"created_ino":"0x1000000031c"},{"tid":895,"created_ino":"0x1000000031d"},{"tid":898,"created_ino":"0x1000000031e"},{"tid":901,"created_ino":"0x1000000031f"},{"tid":904,"created_ino":"0x10000000320"},{"tid":907,"created_ino":"0x10000000321"},{"tid":910,"created_ino":"0x10000000322"},{"tid":913,"created_ino":"0x10000000323"},{"tid":916,"created_ino":"0x10000000324"},{"tid":919,"created_ino":"0x10000000325"},{"tid":922,"created_ino":"0x10000000326"},{"tid":925,"created_ino":"0x10000000327"},{"tid":928,"created_ino":"0x10000000328"},{"tid":931,"created_ino":"0x10000000329"},{"tid":934,"created_ino":"0x1000000032a"},{"tid":937,"created_ino":"0x1000000032b"},{"tid":940,"created_ino":"0x1000000032c"},{"tid":943,"created_ino":"0x1000000032d"},{"tid":946,"created_ino":"0x1000000032e"},{"tid":949,"created_ino":"0x1000000032f"},{"tid":952,"created_ino":"0x10000000330"},{"tid":955,"created_ino":"0x10000000331"},{"tid":958,"created_ino":"0x10000000332"},{"tid":961,"created_ino":"0x10000000333"},{"tid":964,"created_ino":"0x10000000334"},{"tid":967,"created_ino":"0x10000000335"},{"tid":970,"created_ino":"0x10000000336"},{"tid":973,"created_ino":"0x10000000337"},{"tid":976,"created_ino":"0x10000000338"},{"tid":979,"created_ino":"0x10000000339"},{"tid":982,"created_ino":"0x1000000033a"},{"tid":985,"created_ino":"0x1000000033b"},{"tid":988,"created_ino":"0x1000000033c"},{"tid":991,"created_ino":"0x1000000033d"},{"tid":994,"created_ino":"0x1000000033e"},{"tid":997,"created_ino":"0x1000000033f"},{"tid":1000,"created_ino":"0x10000000340"},{"tid":1003,"created_ino":"0x10000000341"},{"tid":1006,"created_ino":"0x10000000342"},{"tid":1009,"created_ino":"0x10000000343"},{"tid":1012,"created_ino":"0x10000000344"},{"tid":1015,"created_ino":"0x10000000345"},{"tid":1018,"created_ino":"0x10000000346"},{"tid":1021,"created_ino":"0x10000000347"},{"tid":1024,"created_ino":"0x10000000348"},{"tid":1027,"created_ino":"0x10000000349"},{"tid":1030,"created_ino":"0x1000000034a"},{"tid":1033,"created_ino":"0x1000000034b"},{"tid":1036,"created_ino":"0x1000000034c"},{"tid":1039,"created_ino":"0x1000000034d"},{"tid":1042,"created_ino":"0x1000000034e"},{"tid":1045,"created_ino":"0x1000000034f"},{"tid":1048,"created_ino":"0x10000000350"},{"tid":1051,"created_ino":"0x10000000351"},{"tid":1054,"created_ino":"0x10000000352"},{"tid":1057,"created_ino":"0x10000000353"},{"tid":1060,"created_ino":"0x10000000354"},{"tid":1063,"created_ino":"0x10000000355"},{"tid":1066,"created_ino":"0x10000000356"},{"tid":1069,"created_ino":"0x10000000357"},{"tid":1072,"created_ino":"0x10000000358"},{"tid":1075,"created_ino":"0x10000000359"},{"tid":1078,"created_ino":"0x1000000035a"},{"tid":1081,"created_ino":"0x1000000035b"},{"tid":1084,"created_ino":"0x1000000035c"},{"tid":1087,"created_ino":"0x1000000035d"},{"tid":1090,"created_ino":"0x1000000035e"},{"tid":1093,"created_ino":"0x1000000035f"},{"tid":1096,"created_ino":"0x10000000360"},{"tid":1099,"created_ino":"0x10000000361"},{"tid":1102,"created_ino":"0x10000000362"},{"tid":1105,"created_ino":"0x10000000363"},{"tid":1108,"created_ino":"0x10000000364"},{"tid":1111,"created_ino":"0x10000000365"},{"tid":1114,"created_ino":"0x10000000366"},{"tid":1117,"created_ino":"0x10000000367"},{"tid":1120,"created_ino":"0x10000000368"},{"tid":1123,"created_ino":"0x10000000369"},{"tid":1126,"created_ino":"0x1000000036a"},{"tid":1129,"created_ino":"0x1000000036b"},{"tid":1132,"created_ino":"0x1000000036c"},{"tid":1135,"created_ino":"0x1000000036d"},{"tid":1138,"created_ino":"0x1000000036e"},{"tid":1141,"created_ino":"0x1000000036f"},{"tid":1144,"created_ino":"0x10000000370"},{"tid":1147,"created_ino":"0x10000000371"},{"tid":1150,"created_ino":"0x10000000372"},{"tid":1153,"created_ino":"0x10000000373"},{"tid":1156,"created_ino":"0x10000000374"},{"tid":1159,"created_ino":"0x10000000375"},{"tid":1162,"created_ino":"0x10000000376"},{"tid":1165,"created_ino":"0x10000000377"},{"tid":1168,"created_ino":"0x10000000378"},{"tid":1171,"created_ino":"0x10000000379"},{"tid":1174,"created_ino":"0x1000000037a"},{"tid":1177,"created_ino":"0x1000000037b"},{"tid":1180,"created_ino":"0x1000000037c"},{"tid":1183,"created_ino":"0x1000000037d"},{"tid":1186,"created_ino":"0x1000000037e"},{"tid":1189,"created_ino":"0x1000000037f"},{"tid":1192,"created_ino":"0x10000000380"},{"tid":1195,"created_ino":"0x10000000381"},{"tid":1198,"created_ino":"0x10000000382"},{"tid":1201,"created_ino":"0x10000000383"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"163390","root":"/"}}] 2023-07-06T03:55:36.555 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "126921", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1807327696, "type": "any" }, "name": { "num": 7286, "type": "client" } }, "id": 7286, "inst": "client.7286 192.168.0.1:0/1807327696", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.946070387359119 }, "state": "open", "uptime": 10.908830841 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "163390", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 7 }, { "created_ino": "0x100000001f6", "tid": 10 }, { "created_ino": "0x100000001f7", "tid": 13 }, { "created_ino": "0x100000001f8", "tid": 16 }, { "created_ino": "0x100000001f9", "tid": 19 }, { "created_ino": "0x100000001fa", "tid": 22 }, { "created_ino": "0x100000001fb", "tid": 25 }, { "created_ino": "0x100000001fc", "tid": 28 }, { "created_ino": "0x100000001fd", "tid": 31 }, { "created_ino": "0x100000001fe", "tid": 34 }, { "created_ino": "0x100000001ff", "tid": 37 }, { "created_ino": "0x10000000200", "tid": 40 }, { "created_ino": "0x10000000201", "tid": 43 }, { "created_ino": "0x10000000202", "tid": 46 }, { "created_ino": "0x10000000203", "tid": 49 }, { "created_ino": "0x10000000204", "tid": 52 }, { "created_ino": "0x10000000205", "tid": 55 }, { "created_ino": "0x10000000206", "tid": 58 }, { "created_ino": "0x10000000207", "tid": 61 }, { "created_ino": "0x10000000208", "tid": 64 }, { "created_ino": "0x10000000209", "tid": 67 }, { "created_ino": "0x1000000020a", "tid": 70 }, { "created_ino": "0x1000000020b", "tid": 73 }, { "created_ino": "0x1000000020c", "tid": 76 }, { "created_ino": "0x1000000020d", "tid": 79 }, { "created_ino": "0x1000000020e", "tid": 82 }, { "created_ino": "0x1000000020f", "tid": 85 }, { "created_ino": "0x10000000210", "tid": 88 }, { "created_ino": "0x10000000211", "tid": 91 }, { "created_ino": "0x10000000212", "tid": 94 }, { "created_ino": "0x10000000213", "tid": 97 }, { "created_ino": "0x10000000214", "tid": 100 }, { "created_ino": "0x10000000215", "tid": 103 }, { "created_ino": "0x10000000216", "tid": 106 }, { "created_ino": "0x10000000217", "tid": 109 }, { "created_ino": "0x10000000218", "tid": 112 }, { "created_ino": "0x10000000219", "tid": 115 }, { "created_ino": "0x1000000021a", "tid": 118 }, { "created_ino": "0x1000000021b", "tid": 121 }, { "created_ino": "0x1000000021c", "tid": 124 }, { "created_ino": "0x1000000021d", "tid": 127 }, { "created_ino": "0x1000000021e", "tid": 130 }, { "created_ino": "0x1000000021f", "tid": 133 }, { "created_ino": "0x10000000220", "tid": 136 }, { "created_ino": "0x10000000221", "tid": 139 }, { "created_ino": "0x10000000222", "tid": 142 }, { "created_ino": "0x10000000223", "tid": 145 }, { "created_ino": "0x10000000224", "tid": 148 }, { "created_ino": "0x10000000225", "tid": 151 }, { "created_ino": "0x10000000226", "tid": 154 }, { "created_ino": "0x10000000227", "tid": 157 }, { "created_ino": "0x10000000228", "tid": 160 }, { "created_ino": "0x10000000229", "tid": 163 }, { "created_ino": "0x1000000022a", "tid": 166 }, { "created_ino": "0x1000000022b", "tid": 169 }, { "created_ino": "0x1000000022c", "tid": 172 }, { "created_ino": "0x1000000022d", "tid": 175 }, { "created_ino": "0x1000000022e", "tid": 178 }, { "created_ino": "0x1000000022f", "tid": 181 }, { "created_ino": "0x10000000230", "tid": 184 }, { "created_ino": "0x10000000231", "tid": 187 }, { "created_ino": "0x10000000232", "tid": 190 }, { "created_ino": "0x10000000233", "tid": 193 }, { "created_ino": "0x10000000234", "tid": 196 }, { "created_ino": "0x10000000235", "tid": 199 }, { "created_ino": "0x10000000236", "tid": 202 }, { "created_ino": "0x10000000237", "tid": 205 }, { "created_ino": "0x10000000238", "tid": 208 }, { "created_ino": "0x10000000239", "tid": 211 }, { "created_ino": "0x1000000023a", "tid": 214 }, { "created_ino": "0x1000000023b", "tid": 217 }, { "created_ino": "0x1000000023c", "tid": 220 }, { "created_ino": "0x1000000023d", "tid": 223 }, { "created_ino": "0x1000000023e", "tid": 226 }, { "created_ino": "0x1000000023f", "tid": 229 }, { "created_ino": "0x10000000240", "tid": 232 }, { "created_ino": "0x10000000241", "tid": 235 }, { "created_ino": "0x10000000242", "tid": 238 }, { "created_ino": "0x10000000243", "tid": 241 }, { "created_ino": "0x10000000244", "tid": 244 }, { "created_ino": "0x10000000245", "tid": 247 }, { "created_ino": "0x10000000246", "tid": 250 }, { "created_ino": "0x10000000247", "tid": 253 }, { "created_ino": "0x10000000248", "tid": 256 }, { "created_ino": "0x10000000249", "tid": 259 }, { "created_ino": "0x1000000024a", "tid": 262 }, { "created_ino": "0x1000000024b", "tid": 265 }, { "created_ino": "0x1000000024c", "tid": 268 }, { "created_ino": "0x1000000024d", "tid": 271 }, { "created_ino": "0x1000000024e", "tid": 274 }, { "created_ino": "0x1000000024f", "tid": 277 }, { "created_ino": "0x10000000250", "tid": 280 }, { "created_ino": "0x10000000251", "tid": 283 }, { "created_ino": "0x10000000252", "tid": 286 }, { "created_ino": "0x10000000253", "tid": 289 }, { "created_ino": "0x10000000254", "tid": 292 }, { "created_ino": "0x10000000255", "tid": 295 }, { "created_ino": "0x10000000256", "tid": 298 }, { "created_ino": "0x10000000257", "tid": 301 }, { "created_ino": "0x10000000258", "tid": 304 }, { "created_ino": "0x10000000259", "tid": 307 }, { "created_ino": "0x1000000025a", "tid": 310 }, { "created_ino": "0x1000000025b", "tid": 313 }, { "created_ino": "0x1000000025c", "tid": 316 }, { "created_ino": "0x1000000025d", "tid": 319 }, { "created_ino": "0x1000000025e", "tid": 322 }, { "created_ino": "0x1000000025f", "tid": 325 }, { "created_ino": "0x10000000260", "tid": 328 }, { "created_ino": "0x10000000261", "tid": 331 }, { "created_ino": "0x10000000262", "tid": 334 }, { "created_ino": "0x10000000263", "tid": 337 }, { "created_ino": "0x10000000264", "tid": 340 }, { "created_ino": "0x10000000265", "tid": 343 }, { "created_ino": "0x10000000266", "tid": 346 }, { "created_ino": "0x10000000267", "tid": 349 }, { "created_ino": "0x10000000268", "tid": 352 }, { "created_ino": "0x10000000269", "tid": 355 }, { "created_ino": "0x1000000026a", "tid": 358 }, { "created_ino": "0x1000000026b", "tid": 361 }, { "created_ino": "0x1000000026c", "tid": 364 }, { "created_ino": "0x1000000026d", "tid": 367 }, { "created_ino": "0x1000000026e", "tid": 370 }, { "created_ino": "0x1000000026f", "tid": 373 }, { "created_ino": "0x10000000270", "tid": 376 }, { "created_ino": "0x10000000271", "tid": 379 }, { "created_ino": "0x10000000272", "tid": 382 }, { "created_ino": "0x10000000273", "tid": 385 }, { "created_ino": "0x10000000274", "tid": 388 }, { "created_ino": "0x10000000275", "tid": 391 }, { "created_ino": "0x10000000276", "tid": 394 }, { "created_ino": "0x10000000277", "tid": 397 }, { "created_ino": "0x10000000278", "tid": 400 }, { "created_ino": "0x10000000279", "tid": 403 }, { "created_ino": "0x1000000027a", "tid": 406 }, { "created_ino": "0x1000000027b", "tid": 409 }, { "created_ino": "0x1000000027c", "tid": 412 }, { "created_ino": "0x1000000027d", "tid": 415 }, { "created_ino": "0x1000000027e", "tid": 418 }, { "created_ino": "0x1000000027f", "tid": 421 }, { "created_ino": "0x10000000280", "tid": 424 }, { "created_ino": "0x10000000281", "tid": 427 }, { "created_ino": "0x10000000282", "tid": 430 }, { "created_ino": "0x10000000283", "tid": 433 }, { "created_ino": "0x10000000284", "tid": 436 }, { "created_ino": "0x10000000285", "tid": 439 }, { "created_ino": "0x10000000286", "tid": 442 }, { "created_ino": "0x10000000287", "tid": 445 }, { "created_ino": "0x10000000288", "tid": 448 }, { "created_ino": "0x10000000289", "tid": 451 }, { "created_ino": "0x1000000028a", "tid": 454 }, { "created_ino": "0x1000000028b", "tid": 457 }, { "created_ino": "0x1000000028c", "tid": 460 }, { "created_ino": "0x1000000028d", "tid": 463 }, { "created_ino": "0x1000000028e", "tid": 466 }, { "created_ino": "0x1000000028f", "tid": 469 }, { "created_ino": "0x10000000290", "tid": 472 }, { "created_ino": "0x10000000291", "tid": 475 }, { "created_ino": "0x10000000292", "tid": 478 }, { "created_ino": "0x10000000293", "tid": 481 }, { "created_ino": "0x10000000294", "tid": 484 }, { "created_ino": "0x10000000295", "tid": 487 }, { "created_ino": "0x10000000296", "tid": 490 }, { "created_ino": "0x10000000297", "tid": 493 }, { "created_ino": "0x10000000298", "tid": 496 }, { "created_ino": "0x10000000299", "tid": 499 }, { "created_ino": "0x1000000029a", "tid": 502 }, { "created_ino": "0x1000000029b", "tid": 505 }, { "created_ino": "0x1000000029c", "tid": 508 }, { "created_ino": "0x1000000029d", "tid": 511 }, { "created_ino": "0x1000000029e", "tid": 514 }, { "created_ino": "0x1000000029f", "tid": 517 }, { "created_ino": "0x100000002a0", "tid": 520 }, { "created_ino": "0x100000002a1", "tid": 523 }, { "created_ino": "0x100000002a2", "tid": 526 }, { "created_ino": "0x100000002a3", "tid": 529 }, { "created_ino": "0x100000002a4", "tid": 532 }, { "created_ino": "0x100000002a5", "tid": 535 }, { "created_ino": "0x100000002a6", "tid": 538 }, { "created_ino": "0x100000002a7", "tid": 541 }, { "created_ino": "0x100000002a8", "tid": 544 }, { "created_ino": "0x100000002a9", "tid": 547 }, { "created_ino": "0x100000002aa", "tid": 550 }, { "created_ino": "0x100000002ab", "tid": 553 }, { "created_ino": "0x100000002ac", "tid": 556 }, { "created_ino": "0x100000002ad", "tid": 559 }, { "created_ino": "0x100000002ae", "tid": 562 }, { "created_ino": "0x100000002af", "tid": 565 }, { "created_ino": "0x100000002b0", "tid": 568 }, { "created_ino": "0x100000002b1", "tid": 571 }, { "created_ino": "0x100000002b2", "tid": 574 }, { "created_ino": "0x100000002b3", "tid": 577 }, { "created_ino": "0x100000002b4", "tid": 580 }, { "created_ino": "0x100000002b5", "tid": 583 }, { "created_ino": "0x100000002b6", "tid": 586 }, { "created_ino": "0x100000002b7", "tid": 589 }, { "created_ino": "0x100000002b8", "tid": 592 }, { "created_ino": "0x100000002b9", "tid": 595 }, { "created_ino": "0x100000002ba", "tid": 598 }, { "created_ino": "0x100000002bb", "tid": 601 }, { "created_ino": "0x100000002bc", "tid": 604 }, { "created_ino": "0x100000002bd", "tid": 607 }, { "created_ino": "0x100000002be", "tid": 610 }, { "created_ino": "0x100000002bf", "tid": 613 }, { "created_ino": "0x100000002c0", "tid": 616 }, { "created_ino": "0x100000002c1", "tid": 619 }, { "created_ino": "0x100000002c2", "tid": 622 }, { "created_ino": "0x100000002c3", "tid": 625 }, { "created_ino": "0x100000002c4", "tid": 628 }, { "created_ino": "0x100000002c5", "tid": 631 }, { "created_ino": "0x100000002c6", "tid": 634 }, { "created_ino": "0x100000002c7", "tid": 637 }, { "created_ino": "0x100000002c8", "tid": 640 }, { "created_ino": "0x100000002c9", "tid": 643 }, { "created_ino": "0x100000002ca", "tid": 646 }, { "created_ino": "0x100000002cb", "tid": 649 }, { "created_ino": "0x100000002cc", "tid": 652 }, { "created_ino": "0x100000002cd", "tid": 655 }, { "created_ino": "0x100000002ce", "tid": 658 }, { "created_ino": "0x100000002cf", "tid": 661 }, { "created_ino": "0x100000002d0", "tid": 664 }, { "created_ino": "0x100000002d1", "tid": 667 }, { "created_ino": "0x100000002d2", "tid": 670 }, { "created_ino": "0x100000002d3", "tid": 673 }, { "created_ino": "0x100000002d4", "tid": 676 }, { "created_ino": "0x100000002d5", "tid": 679 }, { "created_ino": "0x100000002d6", "tid": 682 }, { "created_ino": "0x100000002d7", "tid": 685 }, { "created_ino": "0x100000002d8", "tid": 688 }, { "created_ino": "0x100000002d9", "tid": 691 }, { "created_ino": "0x100000002da", "tid": 694 }, { "created_ino": "0x100000002db", "tid": 697 }, { "created_ino": "0x100000002dc", "tid": 700 }, { "created_ino": "0x100000002dd", "tid": 703 }, { "created_ino": "0x100000002de", "tid": 706 }, { "created_ino": "0x100000002df", "tid": 709 }, { "created_ino": "0x100000002e0", "tid": 712 }, { "created_ino": "0x100000002e1", "tid": 715 }, { "created_ino": "0x100000002e2", "tid": 718 }, { "created_ino": "0x100000002e3", "tid": 721 }, { "created_ino": "0x100000002e4", "tid": 724 }, { "created_ino": "0x100000002e5", "tid": 727 }, { "created_ino": "0x100000002e6", "tid": 730 }, { "created_ino": "0x100000002e7", "tid": 733 }, { "created_ino": "0x100000002e8", "tid": 736 }, { "created_ino": "0x100000002e9", "tid": 739 }, { "created_ino": "0x100000002ea", "tid": 742 }, { "created_ino": "0x100000002eb", "tid": 745 }, { "created_ino": "0x100000002ec", "tid": 748 }, { "created_ino": "0x100000002ed", "tid": 751 }, { "created_ino": "0x100000002ee", "tid": 754 }, { "created_ino": "0x100000002ef", "tid": 757 }, { "created_ino": "0x100000002f0", "tid": 760 }, { "created_ino": "0x100000002f1", "tid": 763 }, { "created_ino": "0x100000002f2", "tid": 766 }, { "created_ino": "0x100000002f3", "tid": 769 }, { "created_ino": "0x100000002f4", "tid": 772 }, { "created_ino": "0x100000002f5", "tid": 775 }, { "created_ino": "0x100000002f6", "tid": 778 }, { "created_ino": "0x100000002f7", "tid": 781 }, { "created_ino": "0x100000002f8", "tid": 784 }, { "created_ino": "0x100000002f9", "tid": 787 }, { "created_ino": "0x100000002fa", "tid": 790 }, { "created_ino": "0x100000002fb", "tid": 793 }, { "created_ino": "0x100000002fc", "tid": 796 }, { "created_ino": "0x100000002fd", "tid": 799 }, { "created_ino": "0x100000002fe", "tid": 802 }, { "created_ino": "0x100000002ff", "tid": 805 }, { "created_ino": "0x10000000300", "tid": 808 }, { "created_ino": "0x10000000301", "tid": 811 }, { "created_ino": "0x10000000302", "tid": 814 }, { "created_ino": "0x10000000303", "tid": 817 }, { "created_ino": "0x10000000304", "tid": 820 }, { "created_ino": "0x10000000305", "tid": 823 }, { "created_ino": "0x10000000306", "tid": 826 }, { "created_ino": "0x10000000307", "tid": 829 }, { "created_ino": "0x10000000308", "tid": 832 }, { "created_ino": "0x10000000309", "tid": 835 }, { "created_ino": "0x1000000030a", "tid": 838 }, { "created_ino": "0x1000000030b", "tid": 841 }, { "created_ino": "0x1000000030c", "tid": 844 }, { "created_ino": "0x1000000030d", "tid": 847 }, { "created_ino": "0x1000000030e", "tid": 850 }, { "created_ino": "0x1000000030f", "tid": 853 }, { "created_ino": "0x10000000310", "tid": 856 }, { "created_ino": "0x10000000311", "tid": 859 }, { "created_ino": "0x10000000312", "tid": 862 }, { "created_ino": "0x10000000313", "tid": 865 }, { "created_ino": "0x10000000314", "tid": 868 }, { "created_ino": "0x10000000315", "tid": 871 }, { "created_ino": "0x10000000316", "tid": 874 }, { "created_ino": "0x10000000317", "tid": 877 }, { "created_ino": "0x10000000318", "tid": 880 }, { "created_ino": "0x10000000319", "tid": 883 }, { "created_ino": "0x1000000031a", "tid": 886 }, { "created_ino": "0x1000000031b", "tid": 889 }, { "created_ino": "0x1000000031c", "tid": 892 }, { "created_ino": "0x1000000031d", "tid": 895 }, { "created_ino": "0x1000000031e", "tid": 898 }, { "created_ino": "0x1000000031f", "tid": 901 }, { "created_ino": "0x10000000320", "tid": 904 }, { "created_ino": "0x10000000321", "tid": 907 }, { "created_ino": "0x10000000322", "tid": 910 }, { "created_ino": "0x10000000323", "tid": 913 }, { "created_ino": "0x10000000324", "tid": 916 }, { "created_ino": "0x10000000325", "tid": 919 }, { "created_ino": "0x10000000326", "tid": 922 }, { "created_ino": "0x10000000327", "tid": 925 }, { "created_ino": "0x10000000328", "tid": 928 }, { "created_ino": "0x10000000329", "tid": 931 }, { "created_ino": "0x1000000032a", "tid": 934 }, { "created_ino": "0x1000000032b", "tid": 937 }, { "created_ino": "0x1000000032c", "tid": 940 }, { "created_ino": "0x1000000032d", "tid": 943 }, { "created_ino": "0x1000000032e", "tid": 946 }, { "created_ino": "0x1000000032f", "tid": 949 }, { "created_ino": "0x10000000330", "tid": 952 }, { "created_ino": "0x10000000331", "tid": 955 }, { "created_ino": "0x10000000332", "tid": 958 }, { "created_ino": "0x10000000333", "tid": 961 }, { "created_ino": "0x10000000334", "tid": 964 }, { "created_ino": "0x10000000335", "tid": 967 }, { "created_ino": "0x10000000336", "tid": 970 }, { "created_ino": "0x10000000337", "tid": 973 }, { "created_ino": "0x10000000338", "tid": 976 }, { "created_ino": "0x10000000339", "tid": 979 }, { "created_ino": "0x1000000033a", "tid": 982 }, { "created_ino": "0x1000000033b", "tid": 985 }, { "created_ino": "0x1000000033c", "tid": 988 }, { "created_ino": "0x1000000033d", "tid": 991 }, { "created_ino": "0x1000000033e", "tid": 994 }, { "created_ino": "0x1000000033f", "tid": 997 }, { "created_ino": "0x10000000340", "tid": 1000 }, { "created_ino": "0x10000000341", "tid": 1003 }, { "created_ino": "0x10000000342", "tid": 1006 }, { "created_ino": "0x10000000343", "tid": 1009 }, { "created_ino": "0x10000000344", "tid": 1012 }, { "created_ino": "0x10000000345", "tid": 1015 }, { "created_ino": "0x10000000346", "tid": 1018 }, { "created_ino": "0x10000000347", "tid": 1021 }, { "created_ino": "0x10000000348", "tid": 1024 }, { "created_ino": "0x10000000349", "tid": 1027 }, { "created_ino": "0x1000000034a", "tid": 1030 }, { "created_ino": "0x1000000034b", "tid": 1033 }, { "created_ino": "0x1000000034c", "tid": 1036 }, { "created_ino": "0x1000000034d", "tid": 1039 }, { "created_ino": "0x1000000034e", "tid": 1042 }, { "created_ino": "0x1000000034f", "tid": 1045 }, { "created_ino": "0x10000000350", "tid": 1048 }, { "created_ino": "0x10000000351", "tid": 1051 }, { "created_ino": "0x10000000352", "tid": 1054 }, { "created_ino": "0x10000000353", "tid": 1057 }, { "created_ino": "0x10000000354", "tid": 1060 }, { "created_ino": "0x10000000355", "tid": 1063 }, { "created_ino": "0x10000000356", "tid": 1066 }, { "created_ino": "0x10000000357", "tid": 1069 }, { "created_ino": "0x10000000358", "tid": 1072 }, { "created_ino": "0x10000000359", "tid": 1075 }, { "created_ino": "0x1000000035a", "tid": 1078 }, { "created_ino": "0x1000000035b", "tid": 1081 }, { "created_ino": "0x1000000035c", "tid": 1084 }, { "created_ino": "0x1000000035d", "tid": 1087 }, { "created_ino": "0x1000000035e", "tid": 1090 }, { "created_ino": "0x1000000035f", "tid": 1093 }, { "created_ino": "0x10000000360", "tid": 1096 }, { "created_ino": "0x10000000361", "tid": 1099 }, { "created_ino": "0x10000000362", "tid": 1102 }, { "created_ino": "0x10000000363", "tid": 1105 }, { "created_ino": "0x10000000364", "tid": 1108 }, { "created_ino": "0x10000000365", "tid": 1111 }, { "created_ino": "0x10000000366", "tid": 1114 }, { "created_ino": "0x10000000367", "tid": 1117 }, { "created_ino": "0x10000000368", "tid": 1120 }, { "created_ino": "0x10000000369", "tid": 1123 }, { "created_ino": "0x1000000036a", "tid": 1126 }, { "created_ino": "0x1000000036b", "tid": 1129 }, { "created_ino": "0x1000000036c", "tid": 1132 }, { "created_ino": "0x1000000036d", "tid": 1135 }, { "created_ino": "0x1000000036e", "tid": 1138 }, { "created_ino": "0x1000000036f", "tid": 1141 }, { "created_ino": "0x10000000370", "tid": 1144 }, { "created_ino": "0x10000000371", "tid": 1147 }, { "created_ino": "0x10000000372", "tid": 1150 }, { "created_ino": "0x10000000373", "tid": 1153 }, { "created_ino": "0x10000000374", "tid": 1156 }, { "created_ino": "0x10000000375", "tid": 1159 }, { "created_ino": "0x10000000376", "tid": 1162 }, { "created_ino": "0x10000000377", "tid": 1165 }, { "created_ino": "0x10000000378", "tid": 1168 }, { "created_ino": "0x10000000379", "tid": 1171 }, { "created_ino": "0x1000000037a", "tid": 1174 }, { "created_ino": "0x1000000037b", "tid": 1177 }, { "created_ino": "0x1000000037c", "tid": 1180 }, { "created_ino": "0x1000000037d", "tid": 1183 }, { "created_ino": "0x1000000037e", "tid": 1186 }, { "created_ino": "0x1000000037f", "tid": 1189 }, { "created_ino": "0x10000000380", "tid": 1192 }, { "created_ino": "0x10000000381", "tid": 1195 }, { "created_ino": "0x10000000382", "tid": 1198 }, { "created_ino": "0x10000000383", "tid": 1201 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1280308817, "type": "any" }, "name": { "num": 9264, "type": "client" } }, "id": 9264, "inst": "client.9264 192.168.0.1:0/1280308817", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 7, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 399, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 1163.6283746887048 }, "recall_caps_throttle": { "halflife": 1.5, "value": 466.2887903242228 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 207.98889302344605 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1124, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1185.0601591212203 }, "state": "open", "uptime": 13.168795797 } ] 2023-07-06T03:55:36.556 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2023-07-06T03:55:36.557 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:55:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.746+0000 7f5828bff700 1 Processor -- start 2023-07-06T03:55:36.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.747+0000 7f5828bff700 1 -- start start 2023-07-06T03:55:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.748+0000 7f5828bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f582415c520 0x7f58240cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.748+0000 7f5828bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5824160180 0x7f58241605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.748+0000 7f5828bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58240cdae0 con 0x7f5824160b30 2023-07-06T03:55:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.748+0000 7f5828bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58240cd8d0 con 0x7f5824160180 2023-07-06T03:55:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.748+0000 7f5828bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58240b6640 con 0x7f582415c520 2023-07-06T03:55:36.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.748+0000 7f5822d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5824160b30 0x7f58240cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33540/0 (socket says 172.21.15.97:33540) 2023-07-06T03:55:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.748+0000 7f581bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5824160180 0x7f58241605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.748+0000 7f5822d9d700 1 -- 172.21.15.97:0/3677453390 learned_addr learned my addr 172.21.15.97:0/3677453390 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:36.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.748+0000 7f582259c700 1 --2- 172.21.15.97:0/3677453390 >> v2:172.21.15.146:3300/0 conn(0x7f582415c520 0x7f58240cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.748+0000 7f5821d9b700 1 -- 172.21.15.97:0/3677453390 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162880181 0 0) 0x7f58240cdae0 con 0x7f5824160b30 2023-07-06T03:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.749+0000 7f581bfff700 1 -- 172.21.15.97:0/3677453390 >> v2:172.21.15.146:3300/0 conn(0x7f582415c520 msgr2=0x7f58240cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.749+0000 7f581bfff700 1 --2- 172.21.15.97:0/3677453390 >> v2:172.21.15.146:3300/0 conn(0x7f582415c520 0x7f58240cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.749+0000 7f581bfff700 1 -- 172.21.15.97:0/3677453390 >> v1:172.21.15.97:6789/0 conn(0x7f5824160b30 legacy=0x7f58240cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:36.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.749+0000 7f581bfff700 1 -- 172.21.15.97:0/3677453390 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58240b6920 con 0x7f5824160180 2023-07-06T03:55:36.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.749+0000 7f582259c700 1 --2- 172.21.15.97:0/3677453390 >> v2:172.21.15.146:3300/0 conn(0x7f582415c520 0x7f58240cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:36.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.749+0000 7f581bfff700 1 --2- 172.21.15.97:0/3677453390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5824160180 0x7f58241605f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f58240b4c70 tx=0x7f580c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99b1dbdc712429f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:36.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.749+0000 7f5821d9b700 1 -- 172.21.15.97:0/3677453390 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c011830 con 0x7f5824160180 2023-07-06T03:55:36.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.749+0000 7f5821d9b700 1 -- 172.21.15.97:0/3677453390 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c011990 con 0x7f5824160180 2023-07-06T03:55:36.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.749+0000 7f5821d9b700 1 -- 172.21.15.97:0/3677453390 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c011d20 con 0x7f5824160180 2023-07-06T03:55:36.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.749+0000 7f5828bff700 1 -- 172.21.15.97:0/3677453390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5824160180 msgr2=0x7f58241605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:36.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.749+0000 7f5828bff700 1 --2- 172.21.15.97:0/3677453390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5824160180 0x7f58241605f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f58240b4c70 tx=0x7f580c009b20 comp rx=0 tx=0).stop 2023-07-06T03:55:36.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.750+0000 7f5828bff700 1 -- 172.21.15.97:0/3677453390 shutdown_connections 2023-07-06T03:55:36.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.750+0000 7f5828bff700 1 --2- 172.21.15.97:0/3677453390 >> v2:172.21.15.146:3300/0 conn(0x7f582415c520 0x7f58240cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.750+0000 7f5828bff700 1 --2- 172.21.15.97:0/3677453390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5824160180 0x7f58241605f0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.750+0000 7f5828bff700 1 -- 172.21.15.97:0/3677453390 >> 172.21.15.97:0/3677453390 conn(0x7f5824071790 msgr2=0x7f5824071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:36.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.750+0000 7f5828bff700 1 -- 172.21.15.97:0/3677453390 shutdown_connections 2023-07-06T03:55:36.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.750+0000 7f5828bff700 1 -- 172.21.15.97:0/3677453390 wait complete. 2023-07-06T03:55:36.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.751+0000 7f5828bff700 1 Processor -- start 2023-07-06T03:55:36.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.751+0000 7f5828bff700 1 -- start start 2023-07-06T03:55:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.752+0000 7f5828bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582415c520 0x7f582415e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.752+0000 7f5828bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5824160180 0x7f58241f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.752+0000 7f5828bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5824160b30 0x7f58241f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.752+0000 7f5828bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58240cafc0 con 0x7f5824160b30 2023-07-06T03:55:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.752+0000 7f5828bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58240ca340 con 0x7f5824160180 2023-07-06T03:55:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.752+0000 7f5828bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58240ca620 con 0x7f582415c520 2023-07-06T03:55:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f5822d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5824160b30 0x7f58241f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f5822d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5824160b30 0x7f58241f28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58340/0 (socket says 172.21.15.97:58340) 2023-07-06T03:55:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f5822d9d700 1 -- 172.21.15.97:0/367751375 learned_addr learned my addr 172.21.15.97:0/367751375 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f581bfff700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5824160180 0x7f58241f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f582259c700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582415c520 0x7f582415e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f5822d9d700 1 -- 172.21.15.97:0/367751375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582415c520 msgr2=0x7f582415e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:36.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f5822d9d700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582415c520 0x7f582415e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f5822d9d700 1 -- 172.21.15.97:0/367751375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5824160180 msgr2=0x7f58241f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f5822d9d700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5824160180 0x7f58241f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f5822d9d700 1 -- 172.21.15.97:0/367751375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c0094a0 con 0x7f5824160b30 2023-07-06T03:55:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f582259c700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582415c520 0x7f582415e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.753+0000 7f5822d9d700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5824160b30 0x7f58241f28e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f581c00cca0 tx=0x7f581c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.754+0000 7f581affd700 1 -- 172.21.15.97:0/367751375 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c00ac20 con 0x7f5824160b30 2023-07-06T03:55:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.754+0000 7f581affd700 1 -- 172.21.15.97:0/367751375 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581c00ad80 con 0x7f5824160b30 2023-07-06T03:55:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.754+0000 7f5828bff700 1 -- 172.21.15.97:0/367751375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58240ca900 con 0x7f5824160b30 2023-07-06T03:55:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.754+0000 7f581affd700 1 -- 172.21.15.97:0/367751375 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c007310 con 0x7f5824160b30 2023-07-06T03:55:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.754+0000 7f5828bff700 1 -- 172.21.15.97:0/367751375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582415ed10 con 0x7f5824160b30 2023-07-06T03:55:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.755+0000 7f5828bff700 1 -- 172.21.15.97:0/367751375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58240cb100 con 0x7f5824160b30 2023-07-06T03:55:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.756+0000 7f581affd700 1 -- 172.21.15.97:0/367751375 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f581c007470 con 0x7f5824160b30 2023-07-06T03:55:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.756+0000 7f581affd700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f580406ace0 0x7f580406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.757+0000 7f581affd700 1 -- 172.21.15.97:0/367751375 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f581c089160 con 0x7f5824160b30 2023-07-06T03:55:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.757+0000 7f582259c700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f580406ace0 0x7f580406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.757+0000 7f582259c700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f580406ace0 0x7f580406d190 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f58140099b0 tx=0x7f5814008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.761+0000 7f581affd700 1 -- 172.21.15.97:0/367751375 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f58240cb100 con 0x7f5824160b30 2023-07-06T03:55:36.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.997+0000 7f5828bff700 1 -- 172.21.15.97:0/367751375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f58240cb100 con 0x7f5824160b30 2023-07-06T03:55:36.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:36.998+0000 7f581affd700 1 -- 172.21.15.97:0/367751375 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f58240cb100 con 0x7f5824160b30 2023-07-06T03:55:37.000 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:37.000 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:55:37.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 -- 172.21.15.97:0/367751375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f580406ace0 msgr2=0x7f580406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f580406ace0 0x7f580406d190 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f58140099b0 tx=0x7f5814008040 comp rx=0 tx=0).stop 2023-07-06T03:55:37.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 -- 172.21.15.97:0/367751375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5824160b30 msgr2=0x7f58241f28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5824160b30 0x7f58241f28e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f581c00cca0 tx=0x7f581c00a060 comp rx=0 tx=0).stop 2023-07-06T03:55:37.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 -- 172.21.15.97:0/367751375 shutdown_connections 2023-07-06T03:55:37.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f580406ace0 0x7f580406d190 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5824160b30 0x7f58241f28e0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582415c520 0x7f582415e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 --2- 172.21.15.97:0/367751375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5824160180 0x7f58241f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 -- 172.21.15.97:0/367751375 >> 172.21.15.97:0/367751375 conn(0x7f5824071790 msgr2=0x7f58240b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:37.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 -- 172.21.15.97:0/367751375 shutdown_connections 2023-07-06T03:55:37.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.000+0000 7f5828bff700 1 -- 172.21.15.97:0/367751375 wait complete. 2023-07-06T03:55:37.019 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-06T03:55:37.020 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2023-07-06T03:55:37.020 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:55:37.026 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:55:37.069 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:55:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.217+0000 7f5c4b9ff700 1 Processor -- start 2023-07-06T03:55:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.217+0000 7f5c4b9ff700 1 -- start start 2023-07-06T03:55:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.218+0000 7f5c4b9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5c441592f0 0x7f5c440b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.218+0000 7f5c4b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c4406d860 0x7f5c440b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.218+0000 7f5c4b9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4415cc40 con 0x7f5c44158f20 2023-07-06T03:55:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.218+0000 7f5c4b9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4415cf20 con 0x7f5c441592f0 2023-07-06T03:55:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.218+0000 7f5c4b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4415d200 con 0x7f5c4406d860 2023-07-06T03:55:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.218+0000 7f5c4979b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5c44158f20 0x7f5c440caac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33552/0 (socket says 172.21.15.97:33552) 2023-07-06T03:55:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.218+0000 7f5c4979b700 1 -- 172.21.15.97:0/1654219475 learned_addr learned my addr 172.21.15.97:0/1654219475 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.219+0000 7f5c48f9a700 1 --2- 172.21.15.97:0/1654219475 >> v2:172.21.15.146:3300/0 conn(0x7f5c441592f0 0x7f5c440b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.219+0000 7f5c49f9c700 1 --2- 172.21.15.97:0/1654219475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c4406d860 0x7f5c440b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.219+0000 7f5c3bfff700 1 -- 172.21.15.97:0/1654219475 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456310296 0 0) 0x7f5c4415cc40 con 0x7f5c44158f20 2023-07-06T03:55:37.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.219+0000 7f5c3bfff700 1 -- 172.21.15.97:0/1654219475 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c300035b0 con 0x7f5c44158f20 2023-07-06T03:55:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.219+0000 7f5c49f9c700 1 -- 172.21.15.97:0/1654219475 >> v2:172.21.15.146:3300/0 conn(0x7f5c441592f0 msgr2=0x7f5c440b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.219+0000 7f5c49f9c700 1 --2- 172.21.15.97:0/1654219475 >> v2:172.21.15.146:3300/0 conn(0x7f5c441592f0 0x7f5c440b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.219+0000 7f5c49f9c700 1 -- 172.21.15.97:0/1654219475 >> v1:172.21.15.97:6789/0 conn(0x7f5c44158f20 legacy=0x7f5c440caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.219+0000 7f5c49f9c700 1 -- 172.21.15.97:0/1654219475 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4415d4e0 con 0x7f5c4406d860 2023-07-06T03:55:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.219+0000 7f5c48f9a700 1 --2- 172.21.15.97:0/1654219475 >> v2:172.21.15.146:3300/0 conn(0x7f5c441592f0 0x7f5c440b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:55:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.219+0000 7f5c49f9c700 1 --2- 172.21.15.97:0/1654219475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c4406d860 0x7f5c440b0aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009d90 tx=0x7f5c3c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f655f5d8b1aaea8b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.220+0000 7f5c3bfff700 1 -- 172.21.15.97:0/1654219475 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00e070 con 0x7f5c4406d860 2023-07-06T03:55:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.220+0000 7f5c3bfff700 1 -- 172.21.15.97:0/1654219475 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00bb20 con 0x7f5c4406d860 2023-07-06T03:55:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.220+0000 7f5c3bfff700 1 -- 172.21.15.97:0/1654219475 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c012430 con 0x7f5c4406d860 2023-07-06T03:55:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.220+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/1654219475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c4406d860 msgr2=0x7f5c440b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.220+0000 7f5c4b9ff700 1 --2- 172.21.15.97:0/1654219475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c4406d860 0x7f5c440b0aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009d90 tx=0x7f5c3c00b330 comp rx=0 tx=0).stop 2023-07-06T03:55:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.220+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/1654219475 shutdown_connections 2023-07-06T03:55:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.220+0000 7f5c4b9ff700 1 --2- 172.21.15.97:0/1654219475 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c4406d860 0x7f5c440b0aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.220+0000 7f5c4b9ff700 1 --2- 172.21.15.97:0/1654219475 >> v2:172.21.15.146:3300/0 conn(0x7f5c441592f0 0x7f5c440b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.220+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/1654219475 >> 172.21.15.97:0/1654219475 conn(0x7f5c440ce8c0 msgr2=0x7f5c440cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.220+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/1654219475 shutdown_connections 2023-07-06T03:55:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.220+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/1654219475 wait complete. 2023-07-06T03:55:37.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.222+0000 7f5c4b9ff700 1 Processor -- start 2023-07-06T03:55:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.222+0000 7f5c4b9ff700 1 -- start start 2023-07-06T03:55:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.223+0000 7f5c4b9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4406d860 0x7f5c44149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.223+0000 7f5c4b9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c44158f20 0x7f5c4415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.223+0000 7f5c4b9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c441592f0 0x7f5c440b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.223+0000 7f5c4b9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44150e70 con 0x7f5c4406d860 2023-07-06T03:55:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.223+0000 7f5c4b9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44151150 con 0x7f5c44158f20 2023-07-06T03:55:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.223+0000 7f5c4b9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44151430 con 0x7f5c441592f0 2023-07-06T03:55:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.223+0000 7f5c4979b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4406d860 0x7f5c44149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.223+0000 7f5c4979b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4406d860 0x7f5c44149970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58354/0 (socket says 172.21.15.97:58354) 2023-07-06T03:55:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.223+0000 7f5c4979b700 1 -- 172.21.15.97:0/261687532 learned_addr learned my addr 172.21.15.97:0/261687532 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.223+0000 7f5c48f9a700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c44158f20 0x7f5c4415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.223+0000 7f5c49f9c700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c441592f0 0x7f5c440b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.224+0000 7f5c4979b700 1 -- 172.21.15.97:0/261687532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c441592f0 msgr2=0x7f5c440b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.224+0000 7f5c4979b700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c441592f0 0x7f5c440b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.224+0000 7f5c4979b700 1 -- 172.21.15.97:0/261687532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c44158f20 msgr2=0x7f5c4415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.224+0000 7f5c4979b700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c44158f20 0x7f5c4415a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.224+0000 7f5c4979b700 1 -- 172.21.15.97:0/261687532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c340094a0 con 0x7f5c4406d860 2023-07-06T03:55:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.224+0000 7f5c4979b700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4406d860 0x7f5c44149970 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c700 tx=0x7f5c4000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.224+0000 7f5c39ffb700 1 -- 172.21.15.97:0/261687532 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40005d70 con 0x7f5c4406d860 2023-07-06T03:55:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.224+0000 7f5c39ffb700 1 -- 172.21.15.97:0/261687532 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c400070d0 con 0x7f5c4406d860 2023-07-06T03:55:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.224+0000 7f5c39ffb700 1 -- 172.21.15.97:0/261687532 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40007460 con 0x7f5c4406d860 2023-07-06T03:55:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.224+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/261687532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3c009a70 con 0x7f5c4406d860 2023-07-06T03:55:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.224+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/261687532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c441519f0 con 0x7f5c4406d860 2023-07-06T03:55:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.225+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/261687532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c440684f0 con 0x7f5c4406d860 2023-07-06T03:55:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.226+0000 7f5c39ffb700 1 -- 172.21.15.97:0/261687532 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c400075c0 con 0x7f5c4406d860 2023-07-06T03:55:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.227+0000 7f5c39ffb700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c2806ace0 0x7f5c2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.227+0000 7f5c39ffb700 1 -- 172.21.15.97:0/261687532 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c40088960 con 0x7f5c4406d860 2023-07-06T03:55:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.227+0000 7f5c48f9a700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c2806ace0 0x7f5c2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.228+0000 7f5c48f9a700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c2806ace0 0x7f5c2806d190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5c440b4940 tx=0x7f5c34009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.232+0000 7f5c39ffb700 1 -- 172.21.15.97:0/261687532 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5c40053970 con 0x7f5c4406d860 2023-07-06T03:55:37.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.472+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/261687532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5c4404e340 con 0x7f5c4406d860 2023-07-06T03:55:37.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.473+0000 7f5c39ffb700 1 -- 172.21.15.97:0/261687532 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+316 (secure 0 0 0) 0x7f5c40017150 con 0x7f5c4406d860 2023-07-06T03:55:37.475 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:37.475 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2023-07-06T03:55:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/261687532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c2806ace0 msgr2=0x7f5c2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c2806ace0 0x7f5c2806d190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5c440b4940 tx=0x7f5c34009210 comp rx=0 tx=0).stop 2023-07-06T03:55:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/261687532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4406d860 msgr2=0x7f5c44149970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4406d860 0x7f5c44149970 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c700 tx=0x7f5c4000ca10 comp rx=0 tx=0).stop 2023-07-06T03:55:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/261687532 shutdown_connections 2023-07-06T03:55:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5c2806ace0 0x7f5c2806d190 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5c441592f0 0x7f5c440b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4406d860 0x7f5c44149970 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 --2- 172.21.15.97:0/261687532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c44158f20 0x7f5c4415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/261687532 >> 172.21.15.97:0/261687532 conn(0x7f5c440ce8c0 msgr2=0x7f5c44156d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:37.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/261687532 shutdown_connections 2023-07-06T03:55:37.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.475+0000 7f5c4b9ff700 1 -- 172.21.15.97:0/261687532 wait complete. 2023-07-06T03:55:37.494 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-06T03:55:37.495 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2023-07-06T03:55:37.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:55:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.741+0000 7feca0dff700 1 Processor -- start 2023-07-06T03:55:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.741+0000 7feca0dff700 1 -- start start 2023-07-06T03:55:37.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.742+0000 7feca0dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fec9c15c740 0x7fec9c0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.742+0000 7feca0dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9c071820 0x7fec9c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.743+0000 7feca0dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c0cdc80 con 0x7fec9c071c90 2023-07-06T03:55:37.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.743+0000 7feca0dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9c0d0610 con 0x7fec9c15c740 2023-07-06T03:55:37.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.743+0000 7feca0dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0ce380 con 0x7fec9c071820 2023-07-06T03:55:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.743+0000 7fec9ad9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fec9c071c90 0x7fec9c0ca2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:41228/0 (socket says 172.21.15.97:41228) 2023-07-06T03:55:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.743+0000 7fec9ad9d700 1 -- 172.21.15.97:0/1083280374 learned_addr learned my addr 172.21.15.97:0/1083280374 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.743+0000 7fec93fff700 1 --2- 172.21.15.97:0/1083280374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9c071820 0x7fec9c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.743+0000 7fec9a59c700 1 --2- 172.21.15.97:0/1083280374 >> v2:172.21.15.146:3300/0 conn(0x7fec9c15c740 0x7fec9c0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.743+0000 7fec99d9b700 1 -- 172.21.15.97:0/1083280374 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229240080 0 0) 0x7fec9c0cdc80 con 0x7fec9c071c90 2023-07-06T03:55:37.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.743+0000 7fec99d9b700 1 -- 172.21.15.97:0/1083280374 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec880035b0 con 0x7fec9c071c90 2023-07-06T03:55:37.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.744+0000 7fec93fff700 1 -- 172.21.15.97:0/1083280374 >> v2:172.21.15.146:3300/0 conn(0x7fec9c15c740 msgr2=0x7fec9c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:37.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.744+0000 7fec93fff700 1 --2- 172.21.15.97:0/1083280374 >> v2:172.21.15.146:3300/0 conn(0x7fec9c15c740 0x7fec9c0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.744+0000 7fec93fff700 1 -- 172.21.15.97:0/1083280374 >> v1:172.21.15.146:6789/0 conn(0x7fec9c071c90 legacy=0x7fec9c0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.744+0000 7fec93fff700 1 -- 172.21.15.97:0/1083280374 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0cd630 con 0x7fec9c071820 2023-07-06T03:55:37.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.744+0000 7fec93fff700 1 --2- 172.21.15.97:0/1083280374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9c071820 0x7fec9c06a060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fec84009840 tx=0x7fec84009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a0ba0a93ac9434 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:37.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.744+0000 7fec99d9b700 1 -- 172.21.15.97:0/1083280374 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84011830 con 0x7fec9c071820 2023-07-06T03:55:37.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.744+0000 7fec99d9b700 1 -- 172.21.15.97:0/1083280374 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec84011990 con 0x7fec9c071820 2023-07-06T03:55:37.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.744+0000 7fec99d9b700 1 -- 172.21.15.97:0/1083280374 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84011d20 con 0x7fec9c071820 2023-07-06T03:55:37.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.745+0000 7feca0dff700 1 -- 172.21.15.97:0/1083280374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9c071820 msgr2=0x7fec9c06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.745+0000 7feca0dff700 1 --2- 172.21.15.97:0/1083280374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9c071820 0x7fec9c06a060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fec84009840 tx=0x7fec84009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:37.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.745+0000 7feca0dff700 1 -- 172.21.15.97:0/1083280374 shutdown_connections 2023-07-06T03:55:37.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.745+0000 7feca0dff700 1 --2- 172.21.15.97:0/1083280374 >> v2:172.21.15.146:3300/0 conn(0x7fec9c15c740 0x7fec9c0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.745+0000 7feca0dff700 1 --2- 172.21.15.97:0/1083280374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9c071820 0x7fec9c06a060 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.745+0000 7feca0dff700 1 -- 172.21.15.97:0/1083280374 >> 172.21.15.97:0/1083280374 conn(0x7fec9c0b2280 msgr2=0x7fec9c0cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:37.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.745+0000 7feca0dff700 1 -- 172.21.15.97:0/1083280374 shutdown_connections 2023-07-06T03:55:37.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.745+0000 7feca0dff700 1 -- 172.21.15.97:0/1083280374 wait complete. 2023-07-06T03:55:37.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.747+0000 7feca0dff700 1 Processor -- start 2023-07-06T03:55:37.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.747+0000 7feca0dff700 1 -- start start 2023-07-06T03:55:37.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7feca0dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9c071820 0x7fec9c153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7feca0dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9c15c740 0x7fec9c15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7feca0dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9c15d5d0 0x7fec9c15da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7feca0dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0b29f0 con 0x7fec9c15c740 2023-07-06T03:55:37.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7feca0dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0b2c90 con 0x7fec9c15d5d0 2023-07-06T03:55:37.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7feca0dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0b2f70 con 0x7fec9c071820 2023-07-06T03:55:37.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7fec93fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9c15c740 0x7fec9c15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7fec93fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9c15c740 0x7fec9c15eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58366/0 (socket says 172.21.15.97:58366) 2023-07-06T03:55:37.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7fec93fff700 1 -- 172.21.15.97:0/3103868903 learned_addr learned my addr 172.21.15.97:0/3103868903 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:37.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7fec9a59c700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9c071820 0x7fec9c153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7fec9ad9d700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9c15d5d0 0x7fec9c15da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7fec93fff700 1 -- 172.21.15.97:0/3103868903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9c071820 msgr2=0x7fec9c153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7fec93fff700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9c071820 0x7fec9c153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7fec93fff700 1 -- 172.21.15.97:0/3103868903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9c15d5d0 msgr2=0x7fec9c15da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:37.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7fec93fff700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9c15d5d0 0x7fec9c15da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:37.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.748+0000 7fec93fff700 1 -- 172.21.15.97:0/3103868903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840094a0 con 0x7fec9c15c740 2023-07-06T03:55:37.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.749+0000 7fec93fff700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9c15c740 0x7fec9c15eb40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fec84000c00 tx=0x7fec84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:37.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.749+0000 7fec92ffd700 1 -- 172.21.15.97:0/3103868903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec840036a0 con 0x7fec9c15c740 2023-07-06T03:55:37.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.749+0000 7fec92ffd700 1 -- 172.21.15.97:0/3103868903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8401f070 con 0x7fec9c15c740 2023-07-06T03:55:37.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.749+0000 7feca0dff700 1 -- 172.21.15.97:0/3103868903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c0b3250 con 0x7fec9c15c740 2023-07-06T03:55:37.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.749+0000 7fec92ffd700 1 -- 172.21.15.97:0/3103868903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84003a60 con 0x7fec9c15c740 2023-07-06T03:55:37.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.749+0000 7feca0dff700 1 -- 172.21.15.97:0/3103868903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c15f640 con 0x7fec9c15c740 2023-07-06T03:55:37.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.750+0000 7feca0dff700 1 -- 172.21.15.97:0/3103868903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c0b3460 con 0x7fec9c15c740 2023-07-06T03:55:37.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.751+0000 7fec92ffd700 1 -- 172.21.15.97:0/3103868903 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec84003bc0 con 0x7fec9c15c740 2023-07-06T03:55:37.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.752+0000 7fec92ffd700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec7c06ace0 0x7fec7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:37.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.752+0000 7fec92ffd700 1 -- 172.21.15.97:0/3103868903 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fec84021070 con 0x7fec9c15c740 2023-07-06T03:55:37.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.752+0000 7fec9a59c700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec7c06ace0 0x7fec7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:37.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.753+0000 7fec9a59c700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec7c06ace0 0x7fec7c06d190 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009b20 tx=0x7fec8c0091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:37.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.756+0000 7fec92ffd700 1 -- 172.21.15.97:0/3103868903 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fec84055d00 con 0x7fec9c15c740 2023-07-06T03:55:37.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.996+0000 7feca0dff700 1 -- 172.21.15.97:0/3103868903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec9c04e340 con 0x7fec9c15c740 2023-07-06T03:55:37.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.996+0000 7fec92ffd700 1 -- 172.21.15.97:0/3103868903 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4679 (secure 0 0 0) 0x7fec84059320 con 0x7fec9c15c740 2023-07-06T03:55:37.998 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:37.998 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3059444241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3059444241},{"type":"v1","addr":"172.21.15.146:6835","nonce":3059444241}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2624688851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2624688851},{"type":"v1","addr":"172.21.15.97:6837","nonce":2624688851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3938376776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3938376776},{"type":"v1","addr":"172.21.15.97:6835","nonce":3938376776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:55:14.806943+0000","modified":"2023-07-06T03:55:35.906574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9099":{"gid":9099,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6834/3269084599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3269084599},{"type":"v1","addr":"172.21.15.146:6834","nonce":3269084599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-06T03:55:38.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.998+0000 7feca0dff700 1 -- 172.21.15.97:0/3103868903 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec7c06ace0 msgr2=0x7fec7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:38.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.998+0000 7feca0dff700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec7c06ace0 0x7fec7c06d190 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009b20 tx=0x7fec8c0091c0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.999+0000 7feca0dff700 1 -- 172.21.15.97:0/3103868903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9c15c740 msgr2=0x7fec9c15eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:38.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.999+0000 7feca0dff700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9c15c740 0x7fec9c15eb40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fec84000c00 tx=0x7fec84004750 comp rx=0 tx=0).stop 2023-07-06T03:55:38.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.999+0000 7feca0dff700 1 -- 172.21.15.97:0/3103868903 shutdown_connections 2023-07-06T03:55:38.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.999+0000 7feca0dff700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fec7c06ace0 0x7fec7c06d190 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.999+0000 7feca0dff700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9c15d5d0 0x7fec9c15da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.999+0000 7feca0dff700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fec9c071820 0x7fec9c153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.999+0000 7feca0dff700 1 --2- 172.21.15.97:0/3103868903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9c15c740 0x7fec9c15eb40 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.999+0000 7feca0dff700 1 -- 172.21.15.97:0/3103868903 >> 172.21.15.97:0/3103868903 conn(0x7fec9c0b2280 msgr2=0x7fec9c0b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:38.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.999+0000 7feca0dff700 1 -- 172.21.15.97:0/3103868903 shutdown_connections 2023-07-06T03:55:38.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:37.999+0000 7feca0dff700 1 -- 172.21.15.97:0/3103868903 wait complete. 2023-07-06T03:55:38.003 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 94 2023-07-06T03:55:38.019 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-07-06T03:55:38.247 INFO:teuthology.orchestra.run.smithi146.stdout:[{"id":7286,"entity":{"name":{"type":"client","num":7286},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1807327696}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":12.601804589,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9345690476111739,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7286 192.168.0.1:0/1807327696","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"1","hostname":"smithi146","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"126921","root":"/"}},{"id":9264,"entity":{"name":{"type":"client","num":9264},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1280308817}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":1102,"uptime":14.861769545,"requests_in_flight":0,"num_completed_requests":399,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1339.0889248140434,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":346.98604207847904,"halflife":1.5},"recall_caps_throttle2o":{"value":79.688228957935152,"halflife":0.5},"session_cache_liveness":{"value":1180.4337321457178,"halflife":300},"cap_acquisition":{"value":0,"halflife":10},"last_trim_completed_requests_tid":7,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.9264 192.168.0.1:0/1280308817","completed_requests":[{"tid":7,"created_ino":"0x10000000001"},{"tid":10,"created_ino":"0x100000001f6"},{"tid":13,"created_ino":"0x100000001f7"},{"tid":16,"created_ino":"0x100000001f8"},{"tid":19,"created_ino":"0x100000001f9"},{"tid":22,"created_ino":"0x100000001fa"},{"tid":25,"created_ino":"0x100000001fb"},{"tid":28,"created_ino":"0x100000001fc"},{"tid":31,"created_ino":"0x100000001fd"},{"tid":34,"created_ino":"0x100000001fe"},{"tid":37,"created_ino":"0x100000001ff"},{"tid":40,"created_ino":"0x10000000200"},{"tid":43,"created_ino":"0x10000000201"},{"tid":46,"created_ino":"0x10000000202"},{"tid":49,"created_ino":"0x10000000203"},{"tid":52,"created_ino":"0x10000000204"},{"tid":55,"created_ino":"0x10000000205"},{"tid":58,"created_ino":"0x10000000206"},{"tid":61,"created_ino":"0x10000000207"},{"tid":64,"created_ino":"0x10000000208"},{"tid":67,"created_ino":"0x10000000209"},{"tid":70,"created_ino":"0x1000000020a"},{"tid":73,"created_ino":"0x1000000020b"},{"tid":76,"created_ino":"0x1000000020c"},{"tid":79,"created_ino":"0x1000000020d"},{"tid":82,"created_ino":"0x1000000020e"},{"tid":85,"created_ino":"0x1000000020f"},{"tid":88,"created_ino":"0x10000000210"},{"tid":91,"created_ino":"0x10000000211"},{"tid":94,"created_ino":"0x10000000212"},{"tid":97,"created_ino":"0x10000000213"},{"tid":100,"created_ino":"0x10000000214"},{"tid":103,"created_ino":"0x10000000215"},{"tid":106,"created_ino":"0x10000000216"},{"tid":109,"created_ino":"0x10000000217"},{"tid":112,"created_ino":"0x10000000218"},{"tid":115,"created_ino":"0x10000000219"},{"tid":118,"created_ino":"0x1000000021a"},{"tid":121,"created_ino":"0x1000000021b"},{"tid":124,"created_ino":"0x1000000021c"},{"tid":127,"created_ino":"0x1000000021d"},{"tid":130,"created_ino":"0x1000000021e"},{"tid":133,"created_ino":"0x1000000021f"},{"tid":136,"created_ino":"0x10000000220"},{"tid":139,"created_ino":"0x10000000221"},{"tid":142,"created_ino":"0x10000000222"},{"tid":145,"created_ino":"0x10000000223"},{"tid":148,"created_ino":"0x10000000224"},{"tid":151,"created_ino":"0x10000000225"},{"tid":154,"created_ino":"0x10000000226"},{"tid":157,"created_ino":"0x10000000227"},{"tid":160,"created_ino":"0x10000000228"},{"tid":163,"created_ino":"0x10000000229"},{"tid":166,"created_ino":"0x1000000022a"},{"tid":169,"created_ino":"0x1000000022b"},{"tid":172,"created_ino":"0x1000000022c"},{"tid":175,"created_ino":"0x1000000022d"},{"tid":178,"created_ino":"0x1000000022e"},{"tid":181,"created_ino":"0x1000000022f"},{"tid":184,"created_ino":"0x10000000230"},{"tid":187,"created_ino":"0x10000000231"},{"tid":190,"created_ino":"0x10000000232"},{"tid":193,"created_ino":"0x10000000233"},{"tid":196,"created_ino":"0x10000000234"},{"tid":199,"created_ino":"0x10000000235"},{"tid":202,"created_ino":"0x10000000236"},{"tid":205,"created_ino":"0x10000000237"},{"tid":208,"created_ino":"0x10000000238"},{"tid":211,"created_ino":"0x10000000239"},{"tid":214,"created_ino":"0x1000000023a"},{"tid":217,"created_ino":"0x1000000023b"},{"tid":220,"created_ino":"0x1000000023c"},{"tid":223,"created_ino":"0x1000000023d"},{"tid":226,"created_ino":"0x1000000023e"},{"tid":229,"created_ino":"0x1000000023f"},{"tid":232,"created_ino":"0x10000000240"},{"tid":235,"created_ino":"0x10000000241"},{"tid":238,"created_ino":"0x10000000242"},{"tid":241,"created_ino":"0x10000000243"},{"tid":244,"created_ino":"0x10000000244"},{"tid":247,"created_ino":"0x10000000245"},{"tid":250,"created_ino":"0x10000000246"},{"tid":253,"created_ino":"0x10000000247"},{"tid":256,"created_ino":"0x10000000248"},{"tid":259,"created_ino":"0x10000000249"},{"tid":262,"created_ino":"0x1000000024a"},{"tid":265,"created_ino":"0x1000000024b"},{"tid":268,"created_ino":"0x1000000024c"},{"tid":271,"created_ino":"0x1000000024d"},{"tid":274,"created_ino":"0x1000000024e"},{"tid":277,"created_ino":"0x1000000024f"},{"tid":280,"created_ino":"0x10000000250"},{"tid":283,"created_ino":"0x10000000251"},{"tid":286,"created_ino":"0x10000000252"},{"tid":289,"created_ino":"0x10000000253"},{"tid":292,"created_ino":"0x10000000254"},{"tid":295,"created_ino":"0x10000000255"},{"tid":298,"created_ino":"0x10000000256"},{"tid":301,"created_ino":"0x10000000257"},{"tid":304,"created_ino":"0x10000000258"},{"tid":307,"created_ino":"0x10000000259"},{"tid":310,"created_ino":"0x1000000025a"},{"tid":313,"created_ino":"0x1000000025b"},{"tid":316,"created_ino":"0x1000000025c"},{"tid":319,"created_ino":"0x1000000025d"},{"tid":322,"created_ino":"0x1000000025e"},{"tid":325,"created_ino":"0x1000000025f"},{"tid":328,"created_ino":"0x10000000260"},{"tid":331,"created_ino":"0x10000000261"},{"tid":334,"created_ino":"0x10000000262"},{"tid":337,"created_ino":"0x10000000263"},{"tid":340,"created_ino":"0x10000000264"},{"tid":343,"created_ino":"0x10000000265"},{"tid":346,"created_ino":"0x10000000266"},{"tid":349,"created_ino":"0x10000000267"},{"tid":352,"created_ino":"0x10000000268"},{"tid":355,"created_ino":"0x10000000269"},{"tid":358,"created_ino":"0x1000000026a"},{"tid":361,"created_ino":"0x1000000026b"},{"tid":364,"created_ino":"0x1000000026c"},{"tid":367,"created_ino":"0x1000000026d"},{"tid":370,"created_ino":"0x1000000026e"},{"tid":373,"created_ino":"0x1000000026f"},{"tid":376,"created_ino":"0x10000000270"},{"tid":379,"created_ino":"0x10000000271"},{"tid":382,"created_ino":"0x10000000272"},{"tid":385,"created_ino":"0x10000000273"},{"tid":388,"created_ino":"0x10000000274"},{"tid":391,"created_ino":"0x10000000275"},{"tid":394,"created_ino":"0x10000000276"},{"tid":397,"created_ino":"0x10000000277"},{"tid":400,"created_ino":"0x10000000278"},{"tid":403,"created_ino":"0x10000000279"},{"tid":406,"created_ino":"0x1000000027a"},{"tid":409,"created_ino":"0x1000000027b"},{"tid":412,"created_ino":"0x1000000027c"},{"tid":415,"created_ino":"0x1000000027d"},{"tid":418,"created_ino":"0x1000000027e"},{"tid":421,"created_ino":"0x1000000027f"},{"tid":424,"created_ino":"0x10000000280"},{"tid":427,"created_ino":"0x10000000281"},{"tid":430,"created_ino":"0x10000000282"},{"tid":433,"created_ino":"0x10000000283"},{"tid":436,"created_ino":"0x10000000284"},{"tid":439,"created_ino":"0x10000000285"},{"tid":442,"created_ino":"0x10000000286"},{"tid":445,"created_ino":"0x10000000287"},{"tid":448,"created_ino":"0x10000000288"},{"tid":451,"created_ino":"0x10000000289"},{"tid":454,"created_ino":"0x1000000028a"},{"tid":457,"created_ino":"0x1000000028b"},{"tid":460,"created_ino":"0x1000000028c"},{"tid":463,"created_ino":"0x1000000028d"},{"tid":466,"created_ino":"0x1000000028e"},{"tid":469,"created_ino":"0x1000000028f"},{"tid":472,"created_ino":"0x10000000290"},{"tid":475,"created_ino":"0x10000000291"},{"tid":478,"created_ino":"0x10000000292"},{"tid":481,"created_ino":"0x10000000293"},{"tid":484,"created_ino":"0x10000000294"},{"tid":487,"created_ino":"0x10000000295"},{"tid":490,"created_ino":"0x10000000296"},{"tid":493,"created_ino":"0x10000000297"},{"tid":496,"created_ino":"0x10000000298"},{"tid":499,"created_ino":"0x10000000299"},{"tid":502,"created_ino":"0x1000000029a"},{"tid":505,"created_ino":"0x1000000029b"},{"tid":508,"created_ino":"0x1000000029c"},{"tid":511,"created_ino":"0x1000000029d"},{"tid":514,"created_ino":"0x1000000029e"},{"tid":517,"created_ino":"0x1000000029f"},{"tid":520,"created_ino":"0x100000002a0"},{"tid":523,"created_ino":"0x100000002a1"},{"tid":526,"created_ino":"0x100000002a2"},{"tid":529,"created_ino":"0x100000002a3"},{"tid":532,"created_ino":"0x100000002a4"},{"tid":535,"created_ino":"0x100000002a5"},{"tid":538,"created_ino":"0x100000002a6"},{"tid":541,"created_ino":"0x100000002a7"},{"tid":544,"created_ino":"0x100000002a8"},{"tid":547,"created_ino":"0x100000002a9"},{"tid":550,"created_ino":"0x100000002aa"},{"tid":553,"created_ino":"0x100000002ab"},{"tid":556,"created_ino":"0x100000002ac"},{"tid":559,"created_ino":"0x100000002ad"},{"tid":562,"created_ino":"0x100000002ae"},{"tid":565,"created_ino":"0x100000002af"},{"tid":568,"created_ino":"0x100000002b0"},{"tid":571,"created_ino":"0x100000002b1"},{"tid":574,"created_ino":"0x100000002b2"},{"tid":577,"created_ino":"0x100000002b3"},{"tid":580,"created_ino":"0x100000002b4"},{"tid":583,"created_ino":"0x100000002b5"},{"tid":586,"created_ino":"0x100000002b6"},{"tid":589,"created_ino":"0x100000002b7"},{"tid":592,"created_ino":"0x100000002b8"},{"tid":595,"created_ino":"0x100000002b9"},{"tid":598,"created_ino":"0x100000002ba"},{"tid":601,"created_ino":"0x100000002bb"},{"tid":604,"created_ino":"0x100000002bc"},{"tid":607,"created_ino":"0x100000002bd"},{"tid":610,"created_ino":"0x100000002be"},{"tid":613,"created_ino":"0x100000002bf"},{"tid":616,"created_ino":"0x100000002c0"},{"tid":619,"created_ino":"0x100000002c1"},{"tid":622,"created_ino":"0x100000002c2"},{"tid":625,"created_ino":"0x100000002c3"},{"tid":628,"created_ino":"0x100000002c4"},{"tid":631,"created_ino":"0x100000002c5"},{"tid":634,"created_ino":"0x100000002c6"},{"tid":637,"created_ino":"0x100000002c7"},{"tid":640,"created_ino":"0x100000002c8"},{"tid":643,"created_ino":"0x100000002c9"},{"tid":646,"created_ino":"0x100000002ca"},{"tid":649,"created_ino":"0x100000002cb"},{"tid":652,"created_ino":"0x100000002cc"},{"tid":655,"created_ino":"0x100000002cd"},{"tid":658,"created_ino":"0x100000002ce"},{"tid":661,"created_ino":"0x100000002cf"},{"tid":664,"created_ino":"0x100000002d0"},{"tid":667,"created_ino":"0x100000002d1"},{"tid":670,"created_ino":"0x100000002d2"},{"tid":673,"created_ino":"0x100000002d3"},{"tid":676,"created_ino":"0x100000002d4"},{"tid":679,"created_ino":"0x100000002d5"},{"tid":682,"created_ino":"0x100000002d6"},{"tid":685,"created_ino":"0x100000002d7"},{"tid":688,"created_ino":"0x100000002d8"},{"tid":691,"created_ino":"0x100000002d9"},{"tid":694,"created_ino":"0x100000002da"},{"tid":697,"created_ino":"0x100000002db"},{"tid":700,"created_ino":"0x100000002dc"},{"tid":703,"created_ino":"0x100000002dd"},{"tid":706,"created_ino":"0x100000002de"},{"tid":709,"created_ino":"0x100000002df"},{"tid":712,"created_ino":"0x100000002e0"},{"tid":715,"created_ino":"0x100000002e1"},{"tid":718,"created_ino":"0x100000002e2"},{"tid":721,"created_ino":"0x100000002e3"},{"tid":724,"created_ino":"0x100000002e4"},{"tid":727,"created_ino":"0x100000002e5"},{"tid":730,"created_ino":"0x100000002e6"},{"tid":733,"created_ino":"0x100000002e7"},{"tid":736,"created_ino":"0x100000002e8"},{"tid":739,"created_ino":"0x100000002e9"},{"tid":742,"created_ino":"0x100000002ea"},{"tid":745,"created_ino":"0x100000002eb"},{"tid":748,"created_ino":"0x100000002ec"},{"tid":751,"created_ino":"0x100000002ed"},{"tid":754,"created_ino":"0x100000002ee"},{"tid":757,"created_ino":"0x100000002ef"},{"tid":760,"created_ino":"0x100000002f0"},{"tid":763,"created_ino":"0x100000002f1"},{"tid":766,"created_ino":"0x100000002f2"},{"tid":769,"created_ino":"0x100000002f3"},{"tid":772,"created_ino":"0x100000002f4"},{"tid":775,"created_ino":"0x100000002f5"},{"tid":778,"created_ino":"0x100000002f6"},{"tid":781,"created_ino":"0x100000002f7"},{"tid":784,"created_ino":"0x100000002f8"},{"tid":787,"created_ino":"0x100000002f9"},{"tid":790,"created_ino":"0x100000002fa"},{"tid":793,"created_ino":"0x100000002fb"},{"tid":796,"created_ino":"0x100000002fc"},{"tid":799,"created_ino":"0x100000002fd"},{"tid":802,"created_ino":"0x100000002fe"},{"tid":805,"created_ino":"0x100000002ff"},{"tid":808,"created_ino":"0x10000000300"},{"tid":811,"created_ino":"0x10000000301"},{"tid":814,"created_ino":"0x10000000302"},{"tid":817,"created_ino":"0x10000000303"},{"tid":820,"created_ino":"0x10000000304"},{"tid":823,"created_ino":"0x10000000305"},{"tid":826,"created_ino":"0x10000000306"},{"tid":829,"created_ino":"0x10000000307"},{"tid":832,"created_ino":"0x10000000308"},{"tid":835,"created_ino":"0x10000000309"},{"tid":838,"created_ino":"0x1000000030a"},{"tid":841,"created_ino":"0x1000000030b"},{"tid":844,"created_ino":"0x1000000030c"},{"tid":847,"created_ino":"0x1000000030d"},{"tid":850,"created_ino":"0x1000000030e"},{"tid":853,"created_ino":"0x1000000030f"},{"tid":856,"created_ino":"0x10000000310"},{"tid":859,"created_ino":"0x10000000311"},{"tid":862,"created_ino":"0x10000000312"},{"tid":865,"created_ino":"0x10000000313"},{"tid":868,"created_ino":"0x10000000314"},{"tid":871,"created_ino":"0x10000000315"},{"tid":874,"created_ino":"0x10000000316"},{"tid":877,"created_ino":"0x10000000317"},{"tid":880,"created_ino":"0x10000000318"},{"tid":883,"created_ino":"0x10000000319"},{"tid":886,"created_ino":"0x1000000031a"},{"tid":889,"created_ino":"0x1000000031b"},{"tid":892,"created_ino":"0x1000000031c"},{"tid":895,"created_ino":"0x1000000031d"},{"tid":898,"created_ino":"0x1000000031e"},{"tid":901,"created_ino":"0x1000000031f"},{"tid":904,"created_ino":"0x10000000320"},{"tid":907,"created_ino":"0x10000000321"},{"tid":910,"created_ino":"0x10000000322"},{"tid":913,"created_ino":"0x10000000323"},{"tid":916,"created_ino":"0x10000000324"},{"tid":919,"created_ino":"0x10000000325"},{"tid":922,"created_ino":"0x10000000326"},{"tid":925,"created_ino":"0x10000000327"},{"tid":928,"created_ino":"0x10000000328"},{"tid":931,"created_ino":"0x10000000329"},{"tid":934,"created_ino":"0x1000000032a"},{"tid":937,"created_ino":"0x1000000032b"},{"tid":940,"created_ino":"0x1000000032c"},{"tid":943,"created_ino":"0x1000000032d"},{"tid":946,"created_ino":"0x1000000032e"},{"tid":949,"created_ino":"0x1000000032f"},{"tid":952,"created_ino":"0x10000000330"},{"tid":955,"created_ino":"0x10000000331"},{"tid":958,"created_ino":"0x10000000332"},{"tid":961,"created_ino":"0x10000000333"},{"tid":964,"created_ino":"0x10000000334"},{"tid":967,"created_ino":"0x10000000335"},{"tid":970,"created_ino":"0x10000000336"},{"tid":973,"created_ino":"0x10000000337"},{"tid":976,"created_ino":"0x10000000338"},{"tid":979,"created_ino":"0x10000000339"},{"tid":982,"created_ino":"0x1000000033a"},{"tid":985,"created_ino":"0x1000000033b"},{"tid":988,"created_ino":"0x1000000033c"},{"tid":991,"created_ino":"0x1000000033d"},{"tid":994,"created_ino":"0x1000000033e"},{"tid":997,"created_ino":"0x1000000033f"},{"tid":1000,"created_ino":"0x10000000340"},{"tid":1003,"created_ino":"0x10000000341"},{"tid":1006,"created_ino":"0x10000000342"},{"tid":1009,"created_ino":"0x10000000343"},{"tid":1012,"created_ino":"0x10000000344"},{"tid":1015,"created_ino":"0x10000000345"},{"tid":1018,"created_ino":"0x10000000346"},{"tid":1021,"created_ino":"0x10000000347"},{"tid":1024,"created_ino":"0x10000000348"},{"tid":1027,"created_ino":"0x10000000349"},{"tid":1030,"created_ino":"0x1000000034a"},{"tid":1033,"created_ino":"0x1000000034b"},{"tid":1036,"created_ino":"0x1000000034c"},{"tid":1039,"created_ino":"0x1000000034d"},{"tid":1042,"created_ino":"0x1000000034e"},{"tid":1045,"created_ino":"0x1000000034f"},{"tid":1048,"created_ino":"0x10000000350"},{"tid":1051,"created_ino":"0x10000000351"},{"tid":1054,"created_ino":"0x10000000352"},{"tid":1057,"created_ino":"0x10000000353"},{"tid":1060,"created_ino":"0x10000000354"},{"tid":1063,"created_ino":"0x10000000355"},{"tid":1066,"created_ino":"0x10000000356"},{"tid":1069,"created_ino":"0x10000000357"},{"tid":1072,"created_ino":"0x10000000358"},{"tid":1075,"created_ino":"0x10000000359"},{"tid":1078,"created_ino":"0x1000000035a"},{"tid":1081,"created_ino":"0x1000000035b"},{"tid":1084,"created_ino":"0x1000000035c"},{"tid":1087,"created_ino":"0x1000000035d"},{"tid":1090,"created_ino":"0x1000000035e"},{"tid":1093,"created_ino":"0x1000000035f"},{"tid":1096,"created_ino":"0x10000000360"},{"tid":1099,"created_ino":"0x10000000361"},{"tid":1102,"created_ino":"0x10000000362"},{"tid":1105,"created_ino":"0x10000000363"},{"tid":1108,"created_ino":"0x10000000364"},{"tid":1111,"created_ino":"0x10000000365"},{"tid":1114,"created_ino":"0x10000000366"},{"tid":1117,"created_ino":"0x10000000367"},{"tid":1120,"created_ino":"0x10000000368"},{"tid":1123,"created_ino":"0x10000000369"},{"tid":1126,"created_ino":"0x1000000036a"},{"tid":1129,"created_ino":"0x1000000036b"},{"tid":1132,"created_ino":"0x1000000036c"},{"tid":1135,"created_ino":"0x1000000036d"},{"tid":1138,"created_ino":"0x1000000036e"},{"tid":1141,"created_ino":"0x1000000036f"},{"tid":1144,"created_ino":"0x10000000370"},{"tid":1147,"created_ino":"0x10000000371"},{"tid":1150,"created_ino":"0x10000000372"},{"tid":1153,"created_ino":"0x10000000373"},{"tid":1156,"created_ino":"0x10000000374"},{"tid":1159,"created_ino":"0x10000000375"},{"tid":1162,"created_ino":"0x10000000376"},{"tid":1165,"created_ino":"0x10000000377"},{"tid":1168,"created_ino":"0x10000000378"},{"tid":1171,"created_ino":"0x10000000379"},{"tid":1174,"created_ino":"0x1000000037a"},{"tid":1177,"created_ino":"0x1000000037b"},{"tid":1180,"created_ino":"0x1000000037c"},{"tid":1183,"created_ino":"0x1000000037d"},{"tid":1186,"created_ino":"0x1000000037e"},{"tid":1189,"created_ino":"0x1000000037f"},{"tid":1192,"created_ino":"0x10000000380"},{"tid":1195,"created_ino":"0x10000000381"},{"tid":1198,"created_ino":"0x10000000382"},{"tid":1201,"created_ino":"0x10000000383"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x00000000000fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"79365bb3b803780276d69efb2cc4afe5b160de9e","ceph_version":"ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)","entity_id":"0","hostname":"smithi097","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"163390","root":"/"}}] 2023-07-06T03:55:38.250 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "1", "hostname": "smithi146", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "126921", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1807327696, "type": "any" }, "name": { "num": 7286, "type": "client" } }, "id": 7286, "inst": "client.7286 192.168.0.1:0/1807327696", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.934569047611174 }, "state": "open", "uptime": 12.601804589 }, { "cap_acquisition": { "halflife": 10, "value": 0 }, "client_metadata": { "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", "client_features": { "feature_bits": "0x00000000000fffff" }, "entity_id": "0", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "163390", "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 7 }, { "created_ino": "0x100000001f6", "tid": 10 }, { "created_ino": "0x100000001f7", "tid": 13 }, { "created_ino": "0x100000001f8", "tid": 16 }, { "created_ino": "0x100000001f9", "tid": 19 }, { "created_ino": "0x100000001fa", "tid": 22 }, { "created_ino": "0x100000001fb", "tid": 25 }, { "created_ino": "0x100000001fc", "tid": 28 }, { "created_ino": "0x100000001fd", "tid": 31 }, { "created_ino": "0x100000001fe", "tid": 34 }, { "created_ino": "0x100000001ff", "tid": 37 }, { "created_ino": "0x10000000200", "tid": 40 }, { "created_ino": "0x10000000201", "tid": 43 }, { "created_ino": "0x10000000202", "tid": 46 }, { "created_ino": "0x10000000203", "tid": 49 }, { "created_ino": "0x10000000204", "tid": 52 }, { "created_ino": "0x10000000205", "tid": 55 }, { "created_ino": "0x10000000206", "tid": 58 }, { "created_ino": "0x10000000207", "tid": 61 }, { "created_ino": "0x10000000208", "tid": 64 }, { "created_ino": "0x10000000209", "tid": 67 }, { "created_ino": "0x1000000020a", "tid": 70 }, { "created_ino": "0x1000000020b", "tid": 73 }, { "created_ino": "0x1000000020c", "tid": 76 }, { "created_ino": "0x1000000020d", "tid": 79 }, { "created_ino": "0x1000000020e", "tid": 82 }, { "created_ino": "0x1000000020f", "tid": 85 }, { "created_ino": "0x10000000210", "tid": 88 }, { "created_ino": "0x10000000211", "tid": 91 }, { "created_ino": "0x10000000212", "tid": 94 }, { "created_ino": "0x10000000213", "tid": 97 }, { "created_ino": "0x10000000214", "tid": 100 }, { "created_ino": "0x10000000215", "tid": 103 }, { "created_ino": "0x10000000216", "tid": 106 }, { "created_ino": "0x10000000217", "tid": 109 }, { "created_ino": "0x10000000218", "tid": 112 }, { "created_ino": "0x10000000219", "tid": 115 }, { "created_ino": "0x1000000021a", "tid": 118 }, { "created_ino": "0x1000000021b", "tid": 121 }, { "created_ino": "0x1000000021c", "tid": 124 }, { "created_ino": "0x1000000021d", "tid": 127 }, { "created_ino": "0x1000000021e", "tid": 130 }, { "created_ino": "0x1000000021f", "tid": 133 }, { "created_ino": "0x10000000220", "tid": 136 }, { "created_ino": "0x10000000221", "tid": 139 }, { "created_ino": "0x10000000222", "tid": 142 }, { "created_ino": "0x10000000223", "tid": 145 }, { "created_ino": "0x10000000224", "tid": 148 }, { "created_ino": "0x10000000225", "tid": 151 }, { "created_ino": "0x10000000226", "tid": 154 }, { "created_ino": "0x10000000227", "tid": 157 }, { "created_ino": "0x10000000228", "tid": 160 }, { "created_ino": "0x10000000229", "tid": 163 }, { "created_ino": "0x1000000022a", "tid": 166 }, { "created_ino": "0x1000000022b", "tid": 169 }, { "created_ino": "0x1000000022c", "tid": 172 }, { "created_ino": "0x1000000022d", "tid": 175 }, { "created_ino": "0x1000000022e", "tid": 178 }, { "created_ino": "0x1000000022f", "tid": 181 }, { "created_ino": "0x10000000230", "tid": 184 }, { "created_ino": "0x10000000231", "tid": 187 }, { "created_ino": "0x10000000232", "tid": 190 }, { "created_ino": "0x10000000233", "tid": 193 }, { "created_ino": "0x10000000234", "tid": 196 }, { "created_ino": "0x10000000235", "tid": 199 }, { "created_ino": "0x10000000236", "tid": 202 }, { "created_ino": "0x10000000237", "tid": 205 }, { "created_ino": "0x10000000238", "tid": 208 }, { "created_ino": "0x10000000239", "tid": 211 }, { "created_ino": "0x1000000023a", "tid": 214 }, { "created_ino": "0x1000000023b", "tid": 217 }, { "created_ino": "0x1000000023c", "tid": 220 }, { "created_ino": "0x1000000023d", "tid": 223 }, { "created_ino": "0x1000000023e", "tid": 226 }, { "created_ino": "0x1000000023f", "tid": 229 }, { "created_ino": "0x10000000240", "tid": 232 }, { "created_ino": "0x10000000241", "tid": 235 }, { "created_ino": "0x10000000242", "tid": 238 }, { "created_ino": "0x10000000243", "tid": 241 }, { "created_ino": "0x10000000244", "tid": 244 }, { "created_ino": "0x10000000245", "tid": 247 }, { "created_ino": "0x10000000246", "tid": 250 }, { "created_ino": "0x10000000247", "tid": 253 }, { "created_ino": "0x10000000248", "tid": 256 }, { "created_ino": "0x10000000249", "tid": 259 }, { "created_ino": "0x1000000024a", "tid": 262 }, { "created_ino": "0x1000000024b", "tid": 265 }, { "created_ino": "0x1000000024c", "tid": 268 }, { "created_ino": "0x1000000024d", "tid": 271 }, { "created_ino": "0x1000000024e", "tid": 274 }, { "created_ino": "0x1000000024f", "tid": 277 }, { "created_ino": "0x10000000250", "tid": 280 }, { "created_ino": "0x10000000251", "tid": 283 }, { "created_ino": "0x10000000252", "tid": 286 }, { "created_ino": "0x10000000253", "tid": 289 }, { "created_ino": "0x10000000254", "tid": 292 }, { "created_ino": "0x10000000255", "tid": 295 }, { "created_ino": "0x10000000256", "tid": 298 }, { "created_ino": "0x10000000257", "tid": 301 }, { "created_ino": "0x10000000258", "tid": 304 }, { "created_ino": "0x10000000259", "tid": 307 }, { "created_ino": "0x1000000025a", "tid": 310 }, { "created_ino": "0x1000000025b", "tid": 313 }, { "created_ino": "0x1000000025c", "tid": 316 }, { "created_ino": "0x1000000025d", "tid": 319 }, { "created_ino": "0x1000000025e", "tid": 322 }, { "created_ino": "0x1000000025f", "tid": 325 }, { "created_ino": "0x10000000260", "tid": 328 }, { "created_ino": "0x10000000261", "tid": 331 }, { "created_ino": "0x10000000262", "tid": 334 }, { "created_ino": "0x10000000263", "tid": 337 }, { "created_ino": "0x10000000264", "tid": 340 }, { "created_ino": "0x10000000265", "tid": 343 }, { "created_ino": "0x10000000266", "tid": 346 }, { "created_ino": "0x10000000267", "tid": 349 }, { "created_ino": "0x10000000268", "tid": 352 }, { "created_ino": "0x10000000269", "tid": 355 }, { "created_ino": "0x1000000026a", "tid": 358 }, { "created_ino": "0x1000000026b", "tid": 361 }, { "created_ino": "0x1000000026c", "tid": 364 }, { "created_ino": "0x1000000026d", "tid": 367 }, { "created_ino": "0x1000000026e", "tid": 370 }, { "created_ino": "0x1000000026f", "tid": 373 }, { "created_ino": "0x10000000270", "tid": 376 }, { "created_ino": "0x10000000271", "tid": 379 }, { "created_ino": "0x10000000272", "tid": 382 }, { "created_ino": "0x10000000273", "tid": 385 }, { "created_ino": "0x10000000274", "tid": 388 }, { "created_ino": "0x10000000275", "tid": 391 }, { "created_ino": "0x10000000276", "tid": 394 }, { "created_ino": "0x10000000277", "tid": 397 }, { "created_ino": "0x10000000278", "tid": 400 }, { "created_ino": "0x10000000279", "tid": 403 }, { "created_ino": "0x1000000027a", "tid": 406 }, { "created_ino": "0x1000000027b", "tid": 409 }, { "created_ino": "0x1000000027c", "tid": 412 }, { "created_ino": "0x1000000027d", "tid": 415 }, { "created_ino": "0x1000000027e", "tid": 418 }, { "created_ino": "0x1000000027f", "tid": 421 }, { "created_ino": "0x10000000280", "tid": 424 }, { "created_ino": "0x10000000281", "tid": 427 }, { "created_ino": "0x10000000282", "tid": 430 }, { "created_ino": "0x10000000283", "tid": 433 }, { "created_ino": "0x10000000284", "tid": 436 }, { "created_ino": "0x10000000285", "tid": 439 }, { "created_ino": "0x10000000286", "tid": 442 }, { "created_ino": "0x10000000287", "tid": 445 }, { "created_ino": "0x10000000288", "tid": 448 }, { "created_ino": "0x10000000289", "tid": 451 }, { "created_ino": "0x1000000028a", "tid": 454 }, { "created_ino": "0x1000000028b", "tid": 457 }, { "created_ino": "0x1000000028c", "tid": 460 }, { "created_ino": "0x1000000028d", "tid": 463 }, { "created_ino": "0x1000000028e", "tid": 466 }, { "created_ino": "0x1000000028f", "tid": 469 }, { "created_ino": "0x10000000290", "tid": 472 }, { "created_ino": "0x10000000291", "tid": 475 }, { "created_ino": "0x10000000292", "tid": 478 }, { "created_ino": "0x10000000293", "tid": 481 }, { "created_ino": "0x10000000294", "tid": 484 }, { "created_ino": "0x10000000295", "tid": 487 }, { "created_ino": "0x10000000296", "tid": 490 }, { "created_ino": "0x10000000297", "tid": 493 }, { "created_ino": "0x10000000298", "tid": 496 }, { "created_ino": "0x10000000299", "tid": 499 }, { "created_ino": "0x1000000029a", "tid": 502 }, { "created_ino": "0x1000000029b", "tid": 505 }, { "created_ino": "0x1000000029c", "tid": 508 }, { "created_ino": "0x1000000029d", "tid": 511 }, { "created_ino": "0x1000000029e", "tid": 514 }, { "created_ino": "0x1000000029f", "tid": 517 }, { "created_ino": "0x100000002a0", "tid": 520 }, { "created_ino": "0x100000002a1", "tid": 523 }, { "created_ino": "0x100000002a2", "tid": 526 }, { "created_ino": "0x100000002a3", "tid": 529 }, { "created_ino": "0x100000002a4", "tid": 532 }, { "created_ino": "0x100000002a5", "tid": 535 }, { "created_ino": "0x100000002a6", "tid": 538 }, { "created_ino": "0x100000002a7", "tid": 541 }, { "created_ino": "0x100000002a8", "tid": 544 }, { "created_ino": "0x100000002a9", "tid": 547 }, { "created_ino": "0x100000002aa", "tid": 550 }, { "created_ino": "0x100000002ab", "tid": 553 }, { "created_ino": "0x100000002ac", "tid": 556 }, { "created_ino": "0x100000002ad", "tid": 559 }, { "created_ino": "0x100000002ae", "tid": 562 }, { "created_ino": "0x100000002af", "tid": 565 }, { "created_ino": "0x100000002b0", "tid": 568 }, { "created_ino": "0x100000002b1", "tid": 571 }, { "created_ino": "0x100000002b2", "tid": 574 }, { "created_ino": "0x100000002b3", "tid": 577 }, { "created_ino": "0x100000002b4", "tid": 580 }, { "created_ino": "0x100000002b5", "tid": 583 }, { "created_ino": "0x100000002b6", "tid": 586 }, { "created_ino": "0x100000002b7", "tid": 589 }, { "created_ino": "0x100000002b8", "tid": 592 }, { "created_ino": "0x100000002b9", "tid": 595 }, { "created_ino": "0x100000002ba", "tid": 598 }, { "created_ino": "0x100000002bb", "tid": 601 }, { "created_ino": "0x100000002bc", "tid": 604 }, { "created_ino": "0x100000002bd", "tid": 607 }, { "created_ino": "0x100000002be", "tid": 610 }, { "created_ino": "0x100000002bf", "tid": 613 }, { "created_ino": "0x100000002c0", "tid": 616 }, { "created_ino": "0x100000002c1", "tid": 619 }, { "created_ino": "0x100000002c2", "tid": 622 }, { "created_ino": "0x100000002c3", "tid": 625 }, { "created_ino": "0x100000002c4", "tid": 628 }, { "created_ino": "0x100000002c5", "tid": 631 }, { "created_ino": "0x100000002c6", "tid": 634 }, { "created_ino": "0x100000002c7", "tid": 637 }, { "created_ino": "0x100000002c8", "tid": 640 }, { "created_ino": "0x100000002c9", "tid": 643 }, { "created_ino": "0x100000002ca", "tid": 646 }, { "created_ino": "0x100000002cb", "tid": 649 }, { "created_ino": "0x100000002cc", "tid": 652 }, { "created_ino": "0x100000002cd", "tid": 655 }, { "created_ino": "0x100000002ce", "tid": 658 }, { "created_ino": "0x100000002cf", "tid": 661 }, { "created_ino": "0x100000002d0", "tid": 664 }, { "created_ino": "0x100000002d1", "tid": 667 }, { "created_ino": "0x100000002d2", "tid": 670 }, { "created_ino": "0x100000002d3", "tid": 673 }, { "created_ino": "0x100000002d4", "tid": 676 }, { "created_ino": "0x100000002d5", "tid": 679 }, { "created_ino": "0x100000002d6", "tid": 682 }, { "created_ino": "0x100000002d7", "tid": 685 }, { "created_ino": "0x100000002d8", "tid": 688 }, { "created_ino": "0x100000002d9", "tid": 691 }, { "created_ino": "0x100000002da", "tid": 694 }, { "created_ino": "0x100000002db", "tid": 697 }, { "created_ino": "0x100000002dc", "tid": 700 }, { "created_ino": "0x100000002dd", "tid": 703 }, { "created_ino": "0x100000002de", "tid": 706 }, { "created_ino": "0x100000002df", "tid": 709 }, { "created_ino": "0x100000002e0", "tid": 712 }, { "created_ino": "0x100000002e1", "tid": 715 }, { "created_ino": "0x100000002e2", "tid": 718 }, { "created_ino": "0x100000002e3", "tid": 721 }, { "created_ino": "0x100000002e4", "tid": 724 }, { "created_ino": "0x100000002e5", "tid": 727 }, { "created_ino": "0x100000002e6", "tid": 730 }, { "created_ino": "0x100000002e7", "tid": 733 }, { "created_ino": "0x100000002e8", "tid": 736 }, { "created_ino": "0x100000002e9", "tid": 739 }, { "created_ino": "0x100000002ea", "tid": 742 }, { "created_ino": "0x100000002eb", "tid": 745 }, { "created_ino": "0x100000002ec", "tid": 748 }, { "created_ino": "0x100000002ed", "tid": 751 }, { "created_ino": "0x100000002ee", "tid": 754 }, { "created_ino": "0x100000002ef", "tid": 757 }, { "created_ino": "0x100000002f0", "tid": 760 }, { "created_ino": "0x100000002f1", "tid": 763 }, { "created_ino": "0x100000002f2", "tid": 766 }, { "created_ino": "0x100000002f3", "tid": 769 }, { "created_ino": "0x100000002f4", "tid": 772 }, { "created_ino": "0x100000002f5", "tid": 775 }, { "created_ino": "0x100000002f6", "tid": 778 }, { "created_ino": "0x100000002f7", "tid": 781 }, { "created_ino": "0x100000002f8", "tid": 784 }, { "created_ino": "0x100000002f9", "tid": 787 }, { "created_ino": "0x100000002fa", "tid": 790 }, { "created_ino": "0x100000002fb", "tid": 793 }, { "created_ino": "0x100000002fc", "tid": 796 }, { "created_ino": "0x100000002fd", "tid": 799 }, { "created_ino": "0x100000002fe", "tid": 802 }, { "created_ino": "0x100000002ff", "tid": 805 }, { "created_ino": "0x10000000300", "tid": 808 }, { "created_ino": "0x10000000301", "tid": 811 }, { "created_ino": "0x10000000302", "tid": 814 }, { "created_ino": "0x10000000303", "tid": 817 }, { "created_ino": "0x10000000304", "tid": 820 }, { "created_ino": "0x10000000305", "tid": 823 }, { "created_ino": "0x10000000306", "tid": 826 }, { "created_ino": "0x10000000307", "tid": 829 }, { "created_ino": "0x10000000308", "tid": 832 }, { "created_ino": "0x10000000309", "tid": 835 }, { "created_ino": "0x1000000030a", "tid": 838 }, { "created_ino": "0x1000000030b", "tid": 841 }, { "created_ino": "0x1000000030c", "tid": 844 }, { "created_ino": "0x1000000030d", "tid": 847 }, { "created_ino": "0x1000000030e", "tid": 850 }, { "created_ino": "0x1000000030f", "tid": 853 }, { "created_ino": "0x10000000310", "tid": 856 }, { "created_ino": "0x10000000311", "tid": 859 }, { "created_ino": "0x10000000312", "tid": 862 }, { "created_ino": "0x10000000313", "tid": 865 }, { "created_ino": "0x10000000314", "tid": 868 }, { "created_ino": "0x10000000315", "tid": 871 }, { "created_ino": "0x10000000316", "tid": 874 }, { "created_ino": "0x10000000317", "tid": 877 }, { "created_ino": "0x10000000318", "tid": 880 }, { "created_ino": "0x10000000319", "tid": 883 }, { "created_ino": "0x1000000031a", "tid": 886 }, { "created_ino": "0x1000000031b", "tid": 889 }, { "created_ino": "0x1000000031c", "tid": 892 }, { "created_ino": "0x1000000031d", "tid": 895 }, { "created_ino": "0x1000000031e", "tid": 898 }, { "created_ino": "0x1000000031f", "tid": 901 }, { "created_ino": "0x10000000320", "tid": 904 }, { "created_ino": "0x10000000321", "tid": 907 }, { "created_ino": "0x10000000322", "tid": 910 }, { "created_ino": "0x10000000323", "tid": 913 }, { "created_ino": "0x10000000324", "tid": 916 }, { "created_ino": "0x10000000325", "tid": 919 }, { "created_ino": "0x10000000326", "tid": 922 }, { "created_ino": "0x10000000327", "tid": 925 }, { "created_ino": "0x10000000328", "tid": 928 }, { "created_ino": "0x10000000329", "tid": 931 }, { "created_ino": "0x1000000032a", "tid": 934 }, { "created_ino": "0x1000000032b", "tid": 937 }, { "created_ino": "0x1000000032c", "tid": 940 }, { "created_ino": "0x1000000032d", "tid": 943 }, { "created_ino": "0x1000000032e", "tid": 946 }, { "created_ino": "0x1000000032f", "tid": 949 }, { "created_ino": "0x10000000330", "tid": 952 }, { "created_ino": "0x10000000331", "tid": 955 }, { "created_ino": "0x10000000332", "tid": 958 }, { "created_ino": "0x10000000333", "tid": 961 }, { "created_ino": "0x10000000334", "tid": 964 }, { "created_ino": "0x10000000335", "tid": 967 }, { "created_ino": "0x10000000336", "tid": 970 }, { "created_ino": "0x10000000337", "tid": 973 }, { "created_ino": "0x10000000338", "tid": 976 }, { "created_ino": "0x10000000339", "tid": 979 }, { "created_ino": "0x1000000033a", "tid": 982 }, { "created_ino": "0x1000000033b", "tid": 985 }, { "created_ino": "0x1000000033c", "tid": 988 }, { "created_ino": "0x1000000033d", "tid": 991 }, { "created_ino": "0x1000000033e", "tid": 994 }, { "created_ino": "0x1000000033f", "tid": 997 }, { "created_ino": "0x10000000340", "tid": 1000 }, { "created_ino": "0x10000000341", "tid": 1003 }, { "created_ino": "0x10000000342", "tid": 1006 }, { "created_ino": "0x10000000343", "tid": 1009 }, { "created_ino": "0x10000000344", "tid": 1012 }, { "created_ino": "0x10000000345", "tid": 1015 }, { "created_ino": "0x10000000346", "tid": 1018 }, { "created_ino": "0x10000000347", "tid": 1021 }, { "created_ino": "0x10000000348", "tid": 1024 }, { "created_ino": "0x10000000349", "tid": 1027 }, { "created_ino": "0x1000000034a", "tid": 1030 }, { "created_ino": "0x1000000034b", "tid": 1033 }, { "created_ino": "0x1000000034c", "tid": 1036 }, { "created_ino": "0x1000000034d", "tid": 1039 }, { "created_ino": "0x1000000034e", "tid": 1042 }, { "created_ino": "0x1000000034f", "tid": 1045 }, { "created_ino": "0x10000000350", "tid": 1048 }, { "created_ino": "0x10000000351", "tid": 1051 }, { "created_ino": "0x10000000352", "tid": 1054 }, { "created_ino": "0x10000000353", "tid": 1057 }, { "created_ino": "0x10000000354", "tid": 1060 }, { "created_ino": "0x10000000355", "tid": 1063 }, { "created_ino": "0x10000000356", "tid": 1066 }, { "created_ino": "0x10000000357", "tid": 1069 }, { "created_ino": "0x10000000358", "tid": 1072 }, { "created_ino": "0x10000000359", "tid": 1075 }, { "created_ino": "0x1000000035a", "tid": 1078 }, { "created_ino": "0x1000000035b", "tid": 1081 }, { "created_ino": "0x1000000035c", "tid": 1084 }, { "created_ino": "0x1000000035d", "tid": 1087 }, { "created_ino": "0x1000000035e", "tid": 1090 }, { "created_ino": "0x1000000035f", "tid": 1093 }, { "created_ino": "0x10000000360", "tid": 1096 }, { "created_ino": "0x10000000361", "tid": 1099 }, { "created_ino": "0x10000000362", "tid": 1102 }, { "created_ino": "0x10000000363", "tid": 1105 }, { "created_ino": "0x10000000364", "tid": 1108 }, { "created_ino": "0x10000000365", "tid": 1111 }, { "created_ino": "0x10000000366", "tid": 1114 }, { "created_ino": "0x10000000367", "tid": 1117 }, { "created_ino": "0x10000000368", "tid": 1120 }, { "created_ino": "0x10000000369", "tid": 1123 }, { "created_ino": "0x1000000036a", "tid": 1126 }, { "created_ino": "0x1000000036b", "tid": 1129 }, { "created_ino": "0x1000000036c", "tid": 1132 }, { "created_ino": "0x1000000036d", "tid": 1135 }, { "created_ino": "0x1000000036e", "tid": 1138 }, { "created_ino": "0x1000000036f", "tid": 1141 }, { "created_ino": "0x10000000370", "tid": 1144 }, { "created_ino": "0x10000000371", "tid": 1147 }, { "created_ino": "0x10000000372", "tid": 1150 }, { "created_ino": "0x10000000373", "tid": 1153 }, { "created_ino": "0x10000000374", "tid": 1156 }, { "created_ino": "0x10000000375", "tid": 1159 }, { "created_ino": "0x10000000376", "tid": 1162 }, { "created_ino": "0x10000000377", "tid": 1165 }, { "created_ino": "0x10000000378", "tid": 1168 }, { "created_ino": "0x10000000379", "tid": 1171 }, { "created_ino": "0x1000000037a", "tid": 1174 }, { "created_ino": "0x1000000037b", "tid": 1177 }, { "created_ino": "0x1000000037c", "tid": 1180 }, { "created_ino": "0x1000000037d", "tid": 1183 }, { "created_ino": "0x1000000037e", "tid": 1186 }, { "created_ino": "0x1000000037f", "tid": 1189 }, { "created_ino": "0x10000000380", "tid": 1192 }, { "created_ino": "0x10000000381", "tid": 1195 }, { "created_ino": "0x10000000382", "tid": 1198 }, { "created_ino": "0x10000000383", "tid": 1201 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1280308817, "type": "any" }, "name": { "num": 9264, "type": "client" } }, "id": 9264, "inst": "client.9264 192.168.0.1:0/1280308817", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 7, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 399, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 1339.0889248140434 }, "recall_caps_throttle": { "halflife": 1.5, "value": 346.98604207847904 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 79.68822895793515 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 1102, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1180.4337321457178 }, "state": "open", "uptime": 14.861769545 } ] 2023-07-06T03:55:38.251 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-07-06T03:55:38.251 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:55:38.252 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:55:38.346 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:55:38.445 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:55:38.445 INFO:tasks.cephfs.mount:Terminating background process 2023-07-06T03:55:38.446 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:55:38.446 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:38.493 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:55:38.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.725+0000 7fc6b4dff700 1 Processor -- start 2023-07-06T03:55:38.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.725+0000 7fc6b4dff700 1 -- start start 2023-07-06T03:55:38.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.726+0000 7fc6b4dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6b006c9a0 0x7fc6b0160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:38.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.726+0000 7fc6b4dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b015d940 0x7fc6b0160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:38.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.726+0000 7fc6b4dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b0160f70 con 0x7fc6b00d10b0 2023-07-06T03:55:38.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.726+0000 7fc6b4dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b00d1480 con 0x7fc6b006c9a0 2023-07-06T03:55:38.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.726+0000 7fc6b4dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00d0590 con 0x7fc6b015d940 2023-07-06T03:55:38.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.726+0000 7fc6a7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6b006c9a0 0x7fc6b0160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:38.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.726+0000 7fc6a7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6b006c9a0 0x7fc6b0160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58378/0 (socket says 172.21.15.97:58378) 2023-07-06T03:55:38.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.726+0000 7fc6a7fff700 1 -- 172.21.15.97:0/1998052705 learned_addr learned my addr 172.21.15.97:0/1998052705 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:38.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.726+0000 7fc6aed9d700 1 --2- 172.21.15.97:0/1998052705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b015d940 0x7fc6b0160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:38.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.727+0000 7fc6a7fff700 1 -- 172.21.15.97:0/1998052705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b015d940 msgr2=0x7fc6b0160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:38.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.727+0000 7fc6a7fff700 1 --2- 172.21.15.97:0/1998052705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b015d940 0x7fc6b0160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.727+0000 7fc6a7fff700 1 -- 172.21.15.97:0/1998052705 >> v1:172.21.15.146:6789/0 conn(0x7fc6b00d10b0 legacy=0x7fc6b006c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:38.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.727+0000 7fc6a7fff700 1 -- 172.21.15.97:0/1998052705 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b015ce60 con 0x7fc6b006c9a0 2023-07-06T03:55:38.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.727+0000 7fc6aed9d700 1 --2- 172.21.15.97:0/1998052705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b015d940 0x7fc6b0160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:38.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.727+0000 7fc6a7fff700 1 --2- 172.21.15.97:0/1998052705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b006c9a0 0x7fc6b0160200 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fc6980098f0 tx=0x7fc698009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc012881903a14a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:38.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.727+0000 7fc6add9b700 1 -- 172.21.15.97:0/1998052705 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69800b8a0 con 0x7fc6b006c9a0 2023-07-06T03:55:38.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.727+0000 7fc6add9b700 1 -- 172.21.15.97:0/1998052705 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69800ba00 con 0x7fc6b006c9a0 2023-07-06T03:55:38.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.727+0000 7fc6add9b700 1 -- 172.21.15.97:0/1998052705 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69800bd90 con 0x7fc6b006c9a0 2023-07-06T03:55:38.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.728+0000 7fc6b4dff700 1 -- 172.21.15.97:0/1998052705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b006c9a0 msgr2=0x7fc6b0160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:38.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.728+0000 7fc6b4dff700 1 --2- 172.21.15.97:0/1998052705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b006c9a0 0x7fc6b0160200 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fc6980098f0 tx=0x7fc698009c00 comp rx=0 tx=0).stop 2023-07-06T03:55:38.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.728+0000 7fc6b4dff700 1 -- 172.21.15.97:0/1998052705 shutdown_connections 2023-07-06T03:55:38.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.728+0000 7fc6b4dff700 1 --2- 172.21.15.97:0/1998052705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b015d940 0x7fc6b0160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.728+0000 7fc6b4dff700 1 --2- 172.21.15.97:0/1998052705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b006c9a0 0x7fc6b0160200 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.728+0000 7fc6b4dff700 1 -- 172.21.15.97:0/1998052705 >> 172.21.15.97:0/1998052705 conn(0x7fc6b006a060 msgr2=0x7fc6b006a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:38.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.728+0000 7fc6b4dff700 1 -- 172.21.15.97:0/1998052705 shutdown_connections 2023-07-06T03:55:38.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.728+0000 7fc6b4dff700 1 -- 172.21.15.97:0/1998052705 wait complete. 2023-07-06T03:55:38.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.729+0000 7fc6b4dff700 1 Processor -- start 2023-07-06T03:55:38.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.730+0000 7fc6b4dff700 1 -- start start 2023-07-06T03:55:38.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.730+0000 7fc6b4dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b006c9a0 0x7fc6b015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:38.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.730+0000 7fc6b4dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b00d10b0 0x7fc6b01e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:38.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.730+0000 7fc6b4dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b015d940 0x7fc6b01e6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.730+0000 7fc6b4dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00b6240 con 0x7fc6b015d940 2023-07-06T03:55:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.730+0000 7fc6b4dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00b6750 con 0x7fc6b006c9a0 2023-07-06T03:55:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.730+0000 7fc6b4dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00b6470 con 0x7fc6b00d10b0 2023-07-06T03:55:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.731+0000 7fc6aed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b015d940 0x7fc6b01e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.731+0000 7fc6aed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b015d940 0x7fc6b01e6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58384/0 (socket says 172.21.15.97:58384) 2023-07-06T03:55:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.731+0000 7fc6aed9d700 1 -- 172.21.15.97:0/3559167918 learned_addr learned my addr 172.21.15.97:0/3559167918 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:38.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.731+0000 7fc6a7fff700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b00d10b0 0x7fc6b01e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:38.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.731+0000 7fc6ae59c700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b006c9a0 0x7fc6b015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:38.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.731+0000 7fc6aed9d700 1 -- 172.21.15.97:0/3559167918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b00d10b0 msgr2=0x7fc6b01e5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:38.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.731+0000 7fc6aed9d700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b00d10b0 0x7fc6b01e5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.731+0000 7fc6aed9d700 1 -- 172.21.15.97:0/3559167918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b006c9a0 msgr2=0x7fc6b015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:38.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.731+0000 7fc6aed9d700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b006c9a0 0x7fc6b015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.731+0000 7fc6aed9d700 1 -- 172.21.15.97:0/3559167918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698009550 con 0x7fc6b015d940 2023-07-06T03:55:38.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.731+0000 7fc6aed9d700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b015d940 0x7fc6b01e6530 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8007ef0 tx=0x7fc6a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:38.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.732+0000 7fc6a6ffd700 1 -- 172.21.15.97:0/3559167918 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a800dd80 con 0x7fc6b015d940 2023-07-06T03:55:38.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.732+0000 7fc6a6ffd700 1 -- 172.21.15.97:0/3559167918 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a800f040 con 0x7fc6b015d940 2023-07-06T03:55:38.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.732+0000 7fc6a6ffd700 1 -- 172.21.15.97:0/3559167918 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8003c60 con 0x7fc6b015d940 2023-07-06T03:55:38.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.732+0000 7fc6b4dff700 1 -- 172.21.15.97:0/3559167918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b01e81d0 con 0x7fc6b015d940 2023-07-06T03:55:38.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.732+0000 7fc6b4dff700 1 -- 172.21.15.97:0/3559167918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b01e8790 con 0x7fc6b015d940 2023-07-06T03:55:38.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.733+0000 7fc6b4dff700 1 -- 172.21.15.97:0/3559167918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b01e8980 con 0x7fc6b015d940 2023-07-06T03:55:38.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.734+0000 7fc6a6ffd700 1 -- 172.21.15.97:0/3559167918 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6a8004030 con 0x7fc6b015d940 2023-07-06T03:55:38.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.735+0000 7fc6a6ffd700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc69006ace0 0x7fc69006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:38.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.735+0000 7fc6a6ffd700 1 -- 172.21.15.97:0/3559167918 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6a8089530 con 0x7fc6b015d940 2023-07-06T03:55:38.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.735+0000 7fc6ae59c700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc69006ace0 0x7fc69006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:38.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.735+0000 7fc6ae59c700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc69006ace0 0x7fc69006d190 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0000fb0 tx=0x7fc6a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:38.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.739+0000 7fc6a6ffd700 1 -- 172.21.15.97:0/3559167918 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc6a80545e0 con 0x7fc6b015d940 2023-07-06T03:55:38.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.936+0000 7fc6b4dff700 1 -- 172.21.15.97:0/3559167918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6b0160c50 con 0x7fc6b015d940 2023-07-06T03:55:38.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.937+0000 7fc6a6ffd700 1 -- 172.21.15.97:0/3559167918 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18319 (secure 0 0 0) 0x7fc6a8057c00 con 0x7fc6b015d940 2023-07-06T03:55:38.939 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:38.939 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":124,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:55:16.327257+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:55:12.989988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-06T03:55:13.782970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:55:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.939+0000 7fc6b4dff700 1 -- 172.21.15.97:0/3559167918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc69006ace0 msgr2=0x7fc69006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.939+0000 7fc6b4dff700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc69006ace0 0x7fc69006d190 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0000fb0 tx=0x7fc6a0009040 comp rx=0 tx=0).stop 2023-07-06T03:55:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.939+0000 7fc6b4dff700 1 -- 172.21.15.97:0/3559167918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b015d940 msgr2=0x7fc6b01e6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.939+0000 7fc6b4dff700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b015d940 0x7fc6b01e6530 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8007ef0 tx=0x7fc6a800d3b0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.939+0000 7fc6b4dff700 1 -- 172.21.15.97:0/3559167918 shutdown_connections 2023-07-06T03:55:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.939+0000 7fc6b4dff700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc69006ace0 0x7fc69006d190 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.939+0000 7fc6b4dff700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b015d940 0x7fc6b01e6530 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.939+0000 7fc6b4dff700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b006c9a0 0x7fc6b015b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.939+0000 7fc6b4dff700 1 --2- 172.21.15.97:0/3559167918 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b00d10b0 0x7fc6b01e5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.939+0000 7fc6b4dff700 1 -- 172.21.15.97:0/3559167918 >> 172.21.15.97:0/3559167918 conn(0x7fc6b006a060 msgr2=0x7fc6b00ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.939+0000 7fc6b4dff700 1 -- 172.21.15.97:0/3559167918 shutdown_connections 2023-07-06T03:55:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:38.940+0000 7fc6b4dff700 1 -- 172.21.15.97:0/3559167918 wait complete. 2023-07-06T03:55:38.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:55:12.989988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:55:13.782970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:55:38.960 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:55:39.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.156+0000 7faf737ff700 1 Processor -- start 2023-07-06T03:55:39.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.156+0000 7faf737ff700 1 -- start start 2023-07-06T03:55:39.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.157+0000 7faf737ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf6c0c3cd0 0x7faf6c0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.157+0000 7faf737ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf6c159440 0x7faf6c0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.157+0000 7faf737ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6c0ca260 con 0x7faf6c159070 2023-07-06T03:55:39.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.157+0000 7faf737ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf6c0c99c0 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.157+0000 7faf737ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c0c9ca0 con 0x7faf6c159440 2023-07-06T03:55:39.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.158+0000 7faf7159b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faf6c159070 0x7faf6c0c8f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33556/0 (socket says 172.21.15.97:33556) 2023-07-06T03:55:39.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.158+0000 7faf70d9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf6c0c3cd0 0x7faf6c0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.158+0000 7faf7159b700 1 -- 172.21.15.97:0/1348318903 learned_addr learned my addr 172.21.15.97:0/1348318903 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.158+0000 7faf71d9c700 1 --2- 172.21.15.97:0/1348318903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf6c159440 0x7faf6c0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.158+0000 7faf70d9a700 1 -- 172.21.15.97:0/1348318903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf6c159440 msgr2=0x7faf6c0cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.158+0000 7faf70d9a700 1 --2- 172.21.15.97:0/1348318903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf6c159440 0x7faf6c0cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.158+0000 7faf70d9a700 1 -- 172.21.15.97:0/1348318903 >> v1:172.21.15.97:6789/0 conn(0x7faf6c159070 legacy=0x7faf6c0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.158+0000 7faf70d9a700 1 -- 172.21.15.97:0/1348318903 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c0c9f80 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.158+0000 7faf63fff700 1 -- 172.21.15.97:0/1348318903 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718429494 0 0) 0x7faf6c0ca260 con 0x7faf6c159070 2023-07-06T03:55:39.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.158+0000 7faf70d9a700 1 --2- 172.21.15.97:0/1348318903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf6c0c3cd0 0x7faf6c0cc4c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009840 tx=0x7faf5c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4104e63ecd3dd2db server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:39.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.159+0000 7faf63fff700 1 -- 172.21.15.97:0/1348318903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c011830 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.159+0000 7faf63fff700 1 -- 172.21.15.97:0/1348318903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c011990 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.159+0000 7faf63fff700 1 -- 172.21.15.97:0/1348318903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c011d20 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.159+0000 7faf737ff700 1 -- 172.21.15.97:0/1348318903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf6c0c3cd0 msgr2=0x7faf6c0cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.159+0000 7faf737ff700 1 --2- 172.21.15.97:0/1348318903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf6c0c3cd0 0x7faf6c0cc4c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009840 tx=0x7faf5c009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:39.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.159+0000 7faf737ff700 1 -- 172.21.15.97:0/1348318903 shutdown_connections 2023-07-06T03:55:39.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.159+0000 7faf737ff700 1 --2- 172.21.15.97:0/1348318903 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf6c159440 0x7faf6c0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.159+0000 7faf737ff700 1 --2- 172.21.15.97:0/1348318903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf6c0c3cd0 0x7faf6c0cc4c0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.159+0000 7faf737ff700 1 -- 172.21.15.97:0/1348318903 >> 172.21.15.97:0/1348318903 conn(0x7faf6c06a060 msgr2=0x7faf6c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:39.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.159+0000 7faf737ff700 1 -- 172.21.15.97:0/1348318903 shutdown_connections 2023-07-06T03:55:39.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.159+0000 7faf737ff700 1 -- 172.21.15.97:0/1348318903 wait complete. 2023-07-06T03:55:39.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.161+0000 7faf737ff700 1 Processor -- start 2023-07-06T03:55:39.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.161+0000 7faf737ff700 1 -- start start 2023-07-06T03:55:39.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.162+0000 7faf737ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf6c0c3cd0 0x7faf6c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.162+0000 7faf737ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf6c159070 0x7faf6c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.162+0000 7faf737ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf6c159440 0x7faf6c1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.162+0000 7faf737ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c1411d0 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.162+0000 7faf737ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c15b6f0 con 0x7faf6c159440 2023-07-06T03:55:39.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.162+0000 7faf737ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c140ef0 con 0x7faf6c159070 2023-07-06T03:55:39.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.162+0000 7faf7159b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf6c0c3cd0 0x7faf6c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.162+0000 7faf7159b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf6c0c3cd0 0x7faf6c15c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58410/0 (socket says 172.21.15.97:58410) 2023-07-06T03:55:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.162+0000 7faf7159b700 1 -- 172.21.15.97:0/3228988308 learned_addr learned my addr 172.21.15.97:0/3228988308 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.162+0000 7faf70d9a700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf6c159070 0x7faf6c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.162+0000 7faf71d9c700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf6c159440 0x7faf6c1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.163+0000 7faf7159b700 1 -- 172.21.15.97:0/3228988308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf6c159070 msgr2=0x7faf6c1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.163+0000 7faf7159b700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf6c159070 0x7faf6c1552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.163+0000 7faf7159b700 1 -- 172.21.15.97:0/3228988308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf6c159440 msgr2=0x7faf6c1557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.163+0000 7faf7159b700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf6c159440 0x7faf6c1557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.163+0000 7faf7159b700 1 -- 172.21.15.97:0/3228988308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c0094a0 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.163+0000 7faf7159b700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf6c0c3cd0 0x7faf6c15c470 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7faf680011e0 tx=0x7faf6800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:39.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.163+0000 7faf61ffb700 1 -- 172.21.15.97:0/3228988308 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68004c80 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.163+0000 7faf61ffb700 1 -- 172.21.15.97:0/3228988308 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf68004e00 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.163+0000 7faf61ffb700 1 -- 172.21.15.97:0/3228988308 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68007310 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.163+0000 7faf737ff700 1 -- 172.21.15.97:0/3228988308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6c15b990 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.163+0000 7faf737ff700 1 -- 172.21.15.97:0/3228988308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c15bf50 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.164+0000 7faf737ff700 1 -- 172.21.15.97:0/3228988308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6c15c170 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.165+0000 7faf61ffb700 1 -- 172.21.15.97:0/3228988308 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf68007470 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.166+0000 7faf61ffb700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf5006ace0 0x7faf5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.167+0000 7faf61ffb700 1 -- 172.21.15.97:0/3228988308 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf680891c0 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.167+0000 7faf70d9a700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf5006ace0 0x7faf5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.167+0000 7faf70d9a700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf5006ace0 0x7faf5006d190 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0c40d0 tx=0x7faf5c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:39.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.171+0000 7faf61ffb700 1 -- 172.21.15.97:0/3228988308 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faf680541d0 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.367+0000 7faf737ff700 1 -- 172.21.15.97:0/3228988308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf6c04e340 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.368+0000 7faf61ffb700 1 -- 172.21.15.97:0/3228988308 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf680577f0 con 0x7faf6c0c3cd0 2023-07-06T03:55:39.370 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:55:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 -- 172.21.15.97:0/3228988308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf5006ace0 msgr2=0x7faf5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf5006ace0 0x7faf5006d190 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0c40d0 tx=0x7faf5c008030 comp rx=0 tx=0).stop 2023-07-06T03:55:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 -- 172.21.15.97:0/3228988308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf6c0c3cd0 msgr2=0x7faf6c15c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf6c0c3cd0 0x7faf6c15c470 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7faf680011e0 tx=0x7faf6800ca30 comp rx=0 tx=0).stop 2023-07-06T03:55:39.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 -- 172.21.15.97:0/3228988308 shutdown_connections 2023-07-06T03:55:39.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faf5006ace0 0x7faf5006d190 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faf6c159440 0x7faf6c1557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf6c0c3cd0 0x7faf6c15c470 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 --2- 172.21.15.97:0/3228988308 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faf6c159070 0x7faf6c1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 -- 172.21.15.97:0/3228988308 >> 172.21.15.97:0/3228988308 conn(0x7faf6c06a060 msgr2=0x7faf6c0cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:39.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 -- 172.21.15.97:0/3228988308 shutdown_connections 2023-07-06T03:55:39.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.370+0000 7faf737ff700 1 -- 172.21.15.97:0/3228988308 wait complete. 2023-07-06T03:55:39.390 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:55:39.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.587+0000 7f4b489ff700 1 Processor -- start 2023-07-06T03:55:39.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.587+0000 7f4b489ff700 1 -- start start 2023-07-06T03:55:39.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.588+0000 7f4b489ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4b440cbaa0 0x7f4b4415a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.589+0000 7f4b489ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4415b6e0 con 0x7f4b44154890 2023-07-06T03:55:39.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.589+0000 7f4b489ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4415e0b0 con 0x7f4b4415eb90 2023-07-06T03:55:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.589+0000 7f4b489ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4415e390 con 0x7f4b440cbaa0 2023-07-06T03:55:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.589+0000 7f4b4259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4b44154890 0x7f4b440cb390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33564/0 (socket says 172.21.15.97:33564) 2023-07-06T03:55:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.589+0000 7f4b4259c700 1 -- 172.21.15.97:0/2734873014 learned_addr learned my addr 172.21.15.97:0/2734873014 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.589+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343534552 0 0) 0x7f4b4415b6e0 con 0x7f4b44154890 2023-07-06T03:55:39.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.589+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b2c0035b0 con 0x7f4b44154890 2023-07-06T03:55:39.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.589+0000 7f4b41d9b700 1 --2- 172.21.15.97:0/2734873014 >> v2:172.21.15.146:3300/0 conn(0x7f4b440cbaa0 0x7f4b4415a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 574716038 0 0) 0x7f4b2c0035b0 con 0x7f4b44154890 2023-07-06T03:55:39.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 >> v2:172.21.15.146:3300/0 conn(0x7f4b440cbaa0 msgr2=0x7f4b4415a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:39.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 --2- 172.21.15.97:0/2734873014 >> v2:172.21.15.146:3300/0 conn(0x7f4b440cbaa0 0x7f4b4415a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 >> v1:172.21.15.146:6789/0 conn(0x7f4b4415eb90 legacy=0x7f4b4415ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4415e670 con 0x7f4b44154890 2023-07-06T03:55:39.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f4b34001670 con 0x7f4b44154890 2023-07-06T03:55:39.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 >> v1:172.21.15.97:6789/0 conn(0x7f4b44154890 legacy=0x7f4b440cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c003a00 0x7f4b2c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b2c004850 0x7f4b2c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c005200 0x7f4b2c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4415e390 con 0x7f4b2c005200 2023-07-06T03:55:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c00c080 con 0x7f4b2c003a00 2023-07-06T03:55:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c00c240 con 0x7f4b2c004850 2023-07-06T03:55:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b4259c700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c003a00 0x7f4b2c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b42d9d700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c005200 0x7f4b2c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b41d9b700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b2c004850 0x7f4b2c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.590+0000 7f4b42d9d700 1 -- 172.21.15.97:0/2734873014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b2c004850 msgr2=0x7f4b2c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:39.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b42d9d700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b2c004850 0x7f4b2c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b42d9d700 1 -- 172.21.15.97:0/2734873014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c003a00 msgr2=0x7f4b2c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:39.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b42d9d700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c003a00 0x7f4b2c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b42d9d700 1 -- 172.21.15.97:0/2734873014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c0035b0 con 0x7f4b2c005200 2023-07-06T03:55:39.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b4259c700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c003a00 0x7f4b2c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:55:39.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b42d9d700 1 -- 172.21.15.97:0/2734873014 reap_dead start 2023-07-06T03:55:39.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b42d9d700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c005200 0x7f4b2c00ba10 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00c190 tx=0x7f4b2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cc2c50bdb60fa7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:39.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c001db0 con 0x7f4b2c005200 2023-07-06T03:55:39.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c00ccd0 con 0x7f4b2c005200 2023-07-06T03:55:39.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b4159a700 1 -- 172.21.15.97:0/2734873014 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c003c40 con 0x7f4b2c005200 2023-07-06T03:55:39.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b489ff700 1 -- 172.21.15.97:0/2734873014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c005200 msgr2=0x7f4b2c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.591+0000 7f4b489ff700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c005200 0x7f4b2c00ba10 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00c190 tx=0x7f4b2c005880 comp rx=0 tx=0).stop 2023-07-06T03:55:39.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.592+0000 7f4b489ff700 1 -- 172.21.15.97:0/2734873014 shutdown_connections 2023-07-06T03:55:39.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.592+0000 7f4b489ff700 1 --2- 172.21.15.97:0/2734873014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c005200 0x7f4b2c00ba10 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.592+0000 7f4b489ff700 1 -- 172.21.15.97:0/2734873014 >> 172.21.15.97:0/2734873014 conn(0x7f4b4406a060 msgr2=0x7f4b4406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:39.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.592+0000 7f4b489ff700 1 -- 172.21.15.97:0/2734873014 shutdown_connections 2023-07-06T03:55:39.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.592+0000 7f4b489ff700 1 -- 172.21.15.97:0/2734873014 wait complete. 2023-07-06T03:55:39.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.594+0000 7f4b489ff700 1 Processor -- start 2023-07-06T03:55:39.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.594+0000 7f4b489ff700 1 -- start start 2023-07-06T03:55:39.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.595+0000 7f4b489ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c005200 0x7f4b4415d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.595+0000 7f4b489ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4415cd90 0x7f4b4415b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b489ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b44155100 0x7f4b4415bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b489ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b44145cc0 con 0x7f4b44155100 2023-07-06T03:55:39.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b489ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b441525e0 con 0x7f4b2c005200 2023-07-06T03:55:39.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b489ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b44152020 con 0x7f4b4415cd90 2023-07-06T03:55:39.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b42d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b44155100 0x7f4b4415bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b4259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c005200 0x7f4b4415d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b4259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c005200 0x7f4b4415d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:39056/0 (socket says 172.21.15.97:39056) 2023-07-06T03:55:39.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b4259c700 1 -- 172.21.15.97:0/4043994058 learned_addr learned my addr 172.21.15.97:0/4043994058 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:39.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b41d9b700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4415cd90 0x7f4b4415b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b42d9d700 1 -- 172.21.15.97:0/4043994058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4415cd90 msgr2=0x7f4b4415b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b42d9d700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4415cd90 0x7f4b4415b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b42d9d700 1 -- 172.21.15.97:0/4043994058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c005200 msgr2=0x7f4b4415d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b42d9d700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c005200 0x7f4b4415d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b42d9d700 1 -- 172.21.15.97:0/4043994058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b44152300 con 0x7f4b44155100 2023-07-06T03:55:39.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.596+0000 7f4b42d9d700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b44155100 0x7f4b4415bf00 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f4b44158530 tx=0x7f4b3c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:39.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.597+0000 7f4b32ffd700 1 -- 172.21.15.97:0/4043994058 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c005070 con 0x7f4b44155100 2023-07-06T03:55:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.597+0000 7f4b32ffd700 1 -- 172.21.15.97:0/4043994058 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0051d0 con 0x7f4b44155100 2023-07-06T03:55:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.597+0000 7f4b489ff700 1 -- 172.21.15.97:0/4043994058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b441528c0 con 0x7f4b44155100 2023-07-06T03:55:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.597+0000 7f4b32ffd700 1 -- 172.21.15.97:0/4043994058 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c005360 con 0x7f4b44155100 2023-07-06T03:55:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.597+0000 7f4b489ff700 1 -- 172.21.15.97:0/4043994058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4415d8f0 con 0x7f4b44155100 2023-07-06T03:55:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.598+0000 7f4b489ff700 1 -- 172.21.15.97:0/4043994058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b44152de0 con 0x7f4b44155100 2023-07-06T03:55:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.599+0000 7f4b32ffd700 1 -- 172.21.15.97:0/4043994058 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b3c0054c0 con 0x7f4b44155100 2023-07-06T03:55:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.600+0000 7f4b32ffd700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b2406ace0 0x7f4b2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:39.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.600+0000 7f4b32ffd700 1 -- 172.21.15.97:0/4043994058 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4b3c089c40 con 0x7f4b44155100 2023-07-06T03:55:39.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.600+0000 7f4b4259c700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b2406ace0 0x7f4b2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:39.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.600+0000 7f4b4259c700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b2406ace0 0x7f4b2406d190 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4b34000fb0 tx=0x7f4b340011c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:39.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.604+0000 7f4b32ffd700 1 -- 172.21.15.97:0/4043994058 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4b44152de0 con 0x7f4b44155100 2023-07-06T03:55:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.799+0000 7f4b489ff700 1 -- 172.21.15.97:0/4043994058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4b44152de0 con 0x7f4b44155100 2023-07-06T03:55:39.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.800+0000 7f4b32ffd700 1 -- 172.21.15.97:0/4043994058 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v124) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4b44152de0 con 0x7f4b44155100 2023-07-06T03:55:39.802 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:55:39.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 -- 172.21.15.97:0/4043994058 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b2406ace0 msgr2=0x7f4b2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b2406ace0 0x7f4b2406d190 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4b34000fb0 tx=0x7f4b340011c0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 -- 172.21.15.97:0/4043994058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b44155100 msgr2=0x7f4b4415bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b44155100 0x7f4b4415bf00 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f4b44158530 tx=0x7f4b3c0014a0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 -- 172.21.15.97:0/4043994058 shutdown_connections 2023-07-06T03:55:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4b2406ace0 0x7f4b2406d190 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b44155100 0x7f4b4415bf00 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b2c005200 0x7f4b4415d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 --2- 172.21.15.97:0/4043994058 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4b4415cd90 0x7f4b4415b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 -- 172.21.15.97:0/4043994058 >> 172.21.15.97:0/4043994058 conn(0x7f4b4406a060 msgr2=0x7f4b440c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:39.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 -- 172.21.15.97:0/4043994058 shutdown_connections 2023-07-06T03:55:39.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:39.802+0000 7f4b489ff700 1 -- 172.21.15.97:0/4043994058 wait complete. 2023-07-06T03:55:39.822 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:55:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.020+0000 7fda78fff700 1 Processor -- start 2023-07-06T03:55:40.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.020+0000 7fda78fff700 1 -- start start 2023-07-06T03:55:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.021+0000 7fda78fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda740c3cd0 0x7fda740cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.021+0000 7fda78fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda740ca260 con 0x7fda74159070 2023-07-06T03:55:40.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.021+0000 7fda78fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda740c99c0 con 0x7fda74159440 2023-07-06T03:55:40.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.021+0000 7fda78fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda740c9ca0 con 0x7fda740c3cd0 2023-07-06T03:55:40.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.021+0000 7fda71d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda740c3cd0 0x7fda740cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.021+0000 7fda7259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fda74159070 0x7fda740c8f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43310/0 (socket says 172.21.15.97:43310) 2023-07-06T03:55:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.021+0000 7fda7259c700 1 -- 172.21.15.97:0/3562589278 learned_addr learned my addr 172.21.15.97:0/3562589278 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.022+0000 7fda71d9b700 1 -- 172.21.15.97:0/3562589278 >> v1:172.21.15.146:6789/0 conn(0x7fda74159440 legacy=0x7fda740cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.022+0000 7fda71d9b700 1 -- 172.21.15.97:0/3562589278 >> v1:172.21.15.97:6789/0 conn(0x7fda74159070 legacy=0x7fda740c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.022+0000 7fda71d9b700 1 -- 172.21.15.97:0/3562589278 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda740c9f80 con 0x7fda740c3cd0 2023-07-06T03:55:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.022+0000 7fda71d9b700 1 --2- 172.21.15.97:0/3562589278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda740c3cd0 0x7fda740cc4c0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fda68009840 tx=0x7fda68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15588ede7ced66c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.022+0000 7fda7159a700 1 -- 172.21.15.97:0/3562589278 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68011830 con 0x7fda740c3cd0 2023-07-06T03:55:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.022+0000 7fda7159a700 1 -- 172.21.15.97:0/3562589278 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda68011990 con 0x7fda740c3cd0 2023-07-06T03:55:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.022+0000 7fda7159a700 1 -- 172.21.15.97:0/3562589278 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68011d20 con 0x7fda740c3cd0 2023-07-06T03:55:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.023+0000 7fda78fff700 1 -- 172.21.15.97:0/3562589278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda740c3cd0 msgr2=0x7fda740cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.023+0000 7fda78fff700 1 --2- 172.21.15.97:0/3562589278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda740c3cd0 0x7fda740cc4c0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fda68009840 tx=0x7fda68009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.023+0000 7fda78fff700 1 -- 172.21.15.97:0/3562589278 shutdown_connections 2023-07-06T03:55:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.023+0000 7fda78fff700 1 --2- 172.21.15.97:0/3562589278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda740c3cd0 0x7fda740cc4c0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.023+0000 7fda78fff700 1 -- 172.21.15.97:0/3562589278 >> 172.21.15.97:0/3562589278 conn(0x7fda7406a060 msgr2=0x7fda7406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:40.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.023+0000 7fda78fff700 1 -- 172.21.15.97:0/3562589278 shutdown_connections 2023-07-06T03:55:40.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.023+0000 7fda78fff700 1 -- 172.21.15.97:0/3562589278 wait complete. 2023-07-06T03:55:40.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.025+0000 7fda78fff700 1 Processor -- start 2023-07-06T03:55:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.025+0000 7fda78fff700 1 -- start start 2023-07-06T03:55:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda78fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda740c3cd0 0x7fda741583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda78fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda74159070 0x7fda740b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda78fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda74159440 0x7fda7414d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda78fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda740b3000 con 0x7fda740c3cd0 2023-07-06T03:55:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda78fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda740b2820 con 0x7fda74159440 2023-07-06T03:55:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda78fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda74155990 con 0x7fda74159070 2023-07-06T03:55:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda7259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda740c3cd0 0x7fda741583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda7259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda740c3cd0 0x7fda741583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37138/0 (socket says 172.21.15.97:37138) 2023-07-06T03:55:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda7259c700 1 -- 172.21.15.97:0/3689653649 learned_addr learned my addr 172.21.15.97:0/3689653649 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda7259c700 1 -- 172.21.15.97:0/3689653649 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda74159070 msgr2=0x7fda740b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda7259c700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda74159070 0x7fda740b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda7259c700 1 -- 172.21.15.97:0/3689653649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda74159440 msgr2=0x7fda7414d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda7259c700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda74159440 0x7fda7414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.026+0000 7fda7259c700 1 -- 172.21.15.97:0/3689653649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda680094a0 con 0x7fda740c3cd0 2023-07-06T03:55:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.027+0000 7fda7259c700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda740c3cd0 0x7fda741583c0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fda6400c550 tx=0x7fda6400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.027+0000 7fda62ffd700 1 -- 172.21.15.97:0/3689653649 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64005d70 con 0x7fda740c3cd0 2023-07-06T03:55:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.027+0000 7fda62ffd700 1 -- 172.21.15.97:0/3689653649 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda640070d0 con 0x7fda740c3cd0 2023-07-06T03:55:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.027+0000 7fda62ffd700 1 -- 172.21.15.97:0/3689653649 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64007440 con 0x7fda740c3cd0 2023-07-06T03:55:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.027+0000 7fda78fff700 1 -- 172.21.15.97:0/3689653649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7414d580 con 0x7fda740c3cd0 2023-07-06T03:55:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.027+0000 7fda78fff700 1 -- 172.21.15.97:0/3689653649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda740b3250 con 0x7fda740c3cd0 2023-07-06T03:55:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.028+0000 7fda78fff700 1 -- 172.21.15.97:0/3689653649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda740c9600 con 0x7fda740c3cd0 2023-07-06T03:55:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.029+0000 7fda62ffd700 1 -- 172.21.15.97:0/3689653649 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda640075a0 con 0x7fda740c3cd0 2023-07-06T03:55:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.030+0000 7fda62ffd700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda5406ace0 0x7fda5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.030+0000 7fda62ffd700 1 -- 172.21.15.97:0/3689653649 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fda640888c0 con 0x7fda740c3cd0 2023-07-06T03:55:40.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.030+0000 7fda71d9b700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda5406ace0 0x7fda5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.031+0000 7fda71d9b700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda5406ace0 0x7fda5406d190 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fda740c40d0 tx=0x7fda68010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:40.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.034+0000 7fda62ffd700 1 -- 172.21.15.97:0/3689653649 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda64053a10 con 0x7fda740c3cd0 2023-07-06T03:55:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.227+0000 7fda78fff700 1 -- 172.21.15.97:0/3689653649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda740cd1a0 con 0x7fda740c3cd0 2023-07-06T03:55:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.227+0000 7fda62ffd700 1 -- 172.21.15.97:0/3689653649 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda64057030 con 0x7fda740c3cd0 2023-07-06T03:55:40.229 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:55:40.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 -- 172.21.15.97:0/3689653649 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda5406ace0 msgr2=0x7fda5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda5406ace0 0x7fda5406d190 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fda740c40d0 tx=0x7fda68010000 comp rx=0 tx=0).stop 2023-07-06T03:55:40.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 -- 172.21.15.97:0/3689653649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda740c3cd0 msgr2=0x7fda741583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda740c3cd0 0x7fda741583c0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fda6400c550 tx=0x7fda6400c910 comp rx=0 tx=0).stop 2023-07-06T03:55:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 -- 172.21.15.97:0/3689653649 shutdown_connections 2023-07-06T03:55:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda5406ace0 0x7fda5406d190 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fda74159440 0x7fda7414d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda740c3cd0 0x7fda741583c0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 --2- 172.21.15.97:0/3689653649 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fda74159070 0x7fda740b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 -- 172.21.15.97:0/3689653649 >> 172.21.15.97:0/3689653649 conn(0x7fda7406a060 msgr2=0x7fda7406f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 -- 172.21.15.97:0/3689653649 shutdown_connections 2023-07-06T03:55:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.230+0000 7fda78fff700 1 -- 172.21.15.97:0/3689653649 wait complete. 2023-07-06T03:55:40.249 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.447+0000 7fefe31ff700 1 Processor -- start 2023-07-06T03:55:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.447+0000 7fefe31ff700 1 -- start start 2023-07-06T03:55:40.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.448+0000 7fefe31ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fefdc06b360 0x7fefdc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.449+0000 7fefe31ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefdc0cff90 con 0x7fefdc15c610 2023-07-06T03:55:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.449+0000 7fefe31ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefdc0b5650 con 0x7fefdc0d2210 2023-07-06T03:55:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.449+0000 7fefe31ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefdc0d0780 con 0x7fefdc06b360 2023-07-06T03:55:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.449+0000 7fefe0f9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fefdc15c610 0x7fefdc0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43312/0 (socket says 172.21.15.97:43312) 2023-07-06T03:55:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.449+0000 7fefe0f9b700 1 -- 172.21.15.97:0/3947719265 learned_addr learned my addr 172.21.15.97:0/3947719265 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.449+0000 7fefdbfff700 1 --2- 172.21.15.97:0/3947719265 >> v2:172.21.15.97:3300/0 conn(0x7fefdc06b360 0x7fefdc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.449+0000 7fefdb7fe700 1 -- 172.21.15.97:0/3947719265 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766949153 0 0) 0x7fefdc0cff90 con 0x7fefdc15c610 2023-07-06T03:55:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.449+0000 7fefdb7fe700 1 -- 172.21.15.97:0/3947719265 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefcc0035b0 con 0x7fefdc15c610 2023-07-06T03:55:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.449+0000 7fefdbfff700 1 -- 172.21.15.97:0/3947719265 >> v1:172.21.15.146:6789/0 conn(0x7fefdc0d2210 legacy=0x7fefdc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.449+0000 7fefdbfff700 1 -- 172.21.15.97:0/3947719265 >> v1:172.21.15.97:6789/0 conn(0x7fefdc15c610 legacy=0x7fefdc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.449+0000 7fefdbfff700 1 -- 172.21.15.97:0/3947719265 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc0b5400 con 0x7fefdc06b360 2023-07-06T03:55:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.450+0000 7fefdbfff700 1 --2- 172.21.15.97:0/3947719265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefdc06b360 0x7fefdc0b4ec0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009840 tx=0x7fefc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2582710697c2485 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.450+0000 7fefdb7fe700 1 -- 172.21.15.97:0/3947719265 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8011830 con 0x7fefdc06b360 2023-07-06T03:55:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.450+0000 7fefdb7fe700 1 -- 172.21.15.97:0/3947719265 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc8011990 con 0x7fefdc06b360 2023-07-06T03:55:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.450+0000 7fefdb7fe700 1 -- 172.21.15.97:0/3947719265 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8011d20 con 0x7fefdc06b360 2023-07-06T03:55:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.450+0000 7fefe31ff700 1 -- 172.21.15.97:0/3947719265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefdc06b360 msgr2=0x7fefdc0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.450+0000 7fefe31ff700 1 --2- 172.21.15.97:0/3947719265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefdc06b360 0x7fefdc0b4ec0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009840 tx=0x7fefc8009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.450+0000 7fefe31ff700 1 -- 172.21.15.97:0/3947719265 shutdown_connections 2023-07-06T03:55:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.450+0000 7fefe31ff700 1 --2- 172.21.15.97:0/3947719265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefdc06b360 0x7fefdc0b4ec0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.450+0000 7fefe31ff700 1 -- 172.21.15.97:0/3947719265 >> 172.21.15.97:0/3947719265 conn(0x7fefdc06a6c0 msgr2=0x7fefdc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.450+0000 7fefe31ff700 1 -- 172.21.15.97:0/3947719265 shutdown_connections 2023-07-06T03:55:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.451+0000 7fefe31ff700 1 -- 172.21.15.97:0/3947719265 wait complete. 2023-07-06T03:55:40.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.452+0000 7fefe31ff700 1 Processor -- start 2023-07-06T03:55:40.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.452+0000 7fefe31ff700 1 -- start start 2023-07-06T03:55:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.453+0000 7fefe31ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefdc06b360 0x7fefdc149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.453+0000 7fefe31ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefdc0d2210 0x7fefdc0c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.453+0000 7fefe31ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefdc15c610 0x7fefdc148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.453+0000 7fefe31ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc15ddd0 con 0x7fefdc0d2210 2023-07-06T03:55:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.453+0000 7fefe31ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc149e90 con 0x7fefdc06b360 2023-07-06T03:55:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.453+0000 7fefe31ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc15e040 con 0x7fefdc15c610 2023-07-06T03:55:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.453+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefdc0d2210 0x7fefdc0c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.453+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefdc0d2210 0x7fefdc0c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37158/0 (socket says 172.21.15.97:37158) 2023-07-06T03:55:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.453+0000 7fefdbfff700 1 -- 172.21.15.97:0/308042510 learned_addr learned my addr 172.21.15.97:0/308042510 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefe0f9b700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefdc06b360 0x7fefdc149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefe179c700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefdc15c610 0x7fefdc148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefdbfff700 1 -- 172.21.15.97:0/308042510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefdc15c610 msgr2=0x7fefdc148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefdbfff700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefdc15c610 0x7fefdc148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefdbfff700 1 -- 172.21.15.97:0/308042510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefdc06b360 msgr2=0x7fefdc149950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefdbfff700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefdc06b360 0x7fefdc149950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefdbfff700 1 -- 172.21.15.97:0/308042510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc80094a0 con 0x7fefdc0d2210 2023-07-06T03:55:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefe179c700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefdc15c610 0x7fefdc148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefdbfff700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefdc0d2210 0x7fefdc0c73e0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009b90 tx=0x7fefc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefd97fa700 1 -- 172.21.15.97:0/308042510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80038a0 con 0x7fefdc0d2210 2023-07-06T03:55:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefd97fa700 1 -- 172.21.15.97:0/308042510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc801e070 con 0x7fefdc0d2210 2023-07-06T03:55:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefe31ff700 1 -- 172.21.15.97:0/308042510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc15e320 con 0x7fefdc0d2210 2023-07-06T03:55:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefd97fa700 1 -- 172.21.15.97:0/308042510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8003c40 con 0x7fefdc0d2210 2023-07-06T03:55:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.454+0000 7fefe31ff700 1 -- 172.21.15.97:0/308042510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc15e8e0 con 0x7fefdc0d2210 2023-07-06T03:55:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.455+0000 7fefe31ff700 1 -- 172.21.15.97:0/308042510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefdc0d0480 con 0x7fefdc0d2210 2023-07-06T03:55:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.456+0000 7fefd97fa700 1 -- 172.21.15.97:0/308042510 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fefc802d450 con 0x7fefdc0d2210 2023-07-06T03:55:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.457+0000 7fefd97fa700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fefc006ace0 0x7fefc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.458+0000 7fefd97fa700 1 -- 172.21.15.97:0/308042510 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fefc808a530 con 0x7fefdc0d2210 2023-07-06T03:55:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.458+0000 7fefe0f9b700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fefc006ace0 0x7fefc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.458+0000 7fefe0f9b700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fefc006ace0 0x7fefc006d190 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fefd00011e0 tx=0x7fefd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.461+0000 7fefd97fa700 1 -- 172.21.15.97:0/308042510 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fefdc0d0480 con 0x7fefdc0d2210 2023-07-06T03:55:40.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.655+0000 7fefe31ff700 1 -- 172.21.15.97:0/308042510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fefdc0d0480 con 0x7fefdc0d2210 2023-07-06T03:55:40.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.655+0000 7fefd97fa700 1 -- 172.21.15.97:0/308042510 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7fefdc0d0480 con 0x7fefdc0d2210 2023-07-06T03:55:40.657 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:55:40.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.657+0000 7fefe31ff700 1 -- 172.21.15.97:0/308042510 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fefc006ace0 msgr2=0x7fefc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.657+0000 7fefe31ff700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fefc006ace0 0x7fefc006d190 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fefd00011e0 tx=0x7fefd0009040 comp rx=0 tx=0).stop 2023-07-06T03:55:40.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.658+0000 7fefe31ff700 1 -- 172.21.15.97:0/308042510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefdc0d2210 msgr2=0x7fefdc0c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.658+0000 7fefe31ff700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefdc0d2210 0x7fefdc0c73e0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009b90 tx=0x7fefc8004750 comp rx=0 tx=0).stop 2023-07-06T03:55:40.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.658+0000 7fefe31ff700 1 -- 172.21.15.97:0/308042510 shutdown_connections 2023-07-06T03:55:40.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.658+0000 7fefe31ff700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fefc006ace0 0x7fefc006d190 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.658+0000 7fefe31ff700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fefdc15c610 0x7fefdc148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.658+0000 7fefe31ff700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefdc06b360 0x7fefdc149950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.658+0000 7fefe31ff700 1 --2- 172.21.15.97:0/308042510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefdc0d2210 0x7fefdc0c73e0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.658+0000 7fefe31ff700 1 -- 172.21.15.97:0/308042510 >> 172.21.15.97:0/308042510 conn(0x7fefdc06a6c0 msgr2=0x7fefdc0cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:40.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.658+0000 7fefe31ff700 1 -- 172.21.15.97:0/308042510 shutdown_connections 2023-07-06T03:55:40.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.658+0000 7fefe31ff700 1 -- 172.21.15.97:0/308042510 wait complete. 2023-07-06T03:55:40.677 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:55:40.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.870+0000 7f0c2cdff700 1 Processor -- start 2023-07-06T03:55:40.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.870+0000 7f0c2cdff700 1 -- start start 2023-07-06T03:55:40.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.872+0000 7f0c2cdff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0c28154890 0x7f0c280cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.872+0000 7f0c2cdff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c2815b050 0x7f0c280cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.872+0000 7f0c2cdff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2815b450 con 0x7f0c2815eb90 2023-07-06T03:55:40.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.872+0000 7f0c2cdff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2815b6e0 con 0x7f0c28154890 2023-07-06T03:55:40.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.872+0000 7f0c2cdff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2815e0b0 con 0x7f0c2815b050 2023-07-06T03:55:40.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.872+0000 7f0c1ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0c2815eb90 0x7f0c2815a940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43326/0 (socket says 172.21.15.97:43326) 2023-07-06T03:55:40.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.872+0000 7f0c2659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0c28154890 0x7f0c280cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.872+0000 7f0c1ffff700 1 -- 172.21.15.97:0/3162001977 learned_addr learned my addr 172.21.15.97:0/3162001977 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:40.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.872+0000 7f0c26d9d700 1 --2- 172.21.15.97:0/3162001977 >> v2:172.21.15.146:3300/0 conn(0x7f0c2815b050 0x7f0c280cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.873+0000 7f0c25d9b700 1 -- 172.21.15.97:0/3162001977 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899658754 0 0) 0x7f0c2815b450 con 0x7f0c2815eb90 2023-07-06T03:55:40.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.873+0000 7f0c2659c700 1 -- 172.21.15.97:0/3162001977 >> v2:172.21.15.146:3300/0 conn(0x7f0c2815b050 msgr2=0x7f0c280cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:40.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.873+0000 7f0c2659c700 1 --2- 172.21.15.97:0/3162001977 >> v2:172.21.15.146:3300/0 conn(0x7f0c2815b050 0x7f0c280cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.873+0000 7f0c2659c700 1 -- 172.21.15.97:0/3162001977 >> v1:172.21.15.97:6789/0 conn(0x7f0c2815eb90 legacy=0x7f0c2815a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.873+0000 7f0c2659c700 1 -- 172.21.15.97:0/3162001977 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2815e390 con 0x7f0c28154890 2023-07-06T03:55:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.873+0000 7f0c2659c700 1 --2- 172.21.15.97:0/3162001977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28154890 0x7f0c280cb390 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f0c280ce770 tx=0x7f0c18009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fe9b361d072c5f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.873+0000 7f0c25d9b700 1 -- 172.21.15.97:0/3162001977 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18011830 con 0x7f0c28154890 2023-07-06T03:55:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.873+0000 7f0c25d9b700 1 -- 172.21.15.97:0/3162001977 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c18011990 con 0x7f0c28154890 2023-07-06T03:55:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.873+0000 7f0c25d9b700 1 -- 172.21.15.97:0/3162001977 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18011d20 con 0x7f0c28154890 2023-07-06T03:55:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.874+0000 7f0c2cdff700 1 -- 172.21.15.97:0/3162001977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28154890 msgr2=0x7f0c280cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.874+0000 7f0c2cdff700 1 --2- 172.21.15.97:0/3162001977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28154890 0x7f0c280cb390 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f0c280ce770 tx=0x7f0c18009b20 comp rx=0 tx=0).stop 2023-07-06T03:55:40.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.874+0000 7f0c2cdff700 1 -- 172.21.15.97:0/3162001977 shutdown_connections 2023-07-06T03:55:40.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.874+0000 7f0c2cdff700 1 --2- 172.21.15.97:0/3162001977 >> v2:172.21.15.146:3300/0 conn(0x7f0c2815b050 0x7f0c280cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.874+0000 7f0c2cdff700 1 --2- 172.21.15.97:0/3162001977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28154890 0x7f0c280cb390 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.874+0000 7f0c2cdff700 1 -- 172.21.15.97:0/3162001977 >> 172.21.15.97:0/3162001977 conn(0x7f0c2806a060 msgr2=0x7f0c2806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:40.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.874+0000 7f0c2cdff700 1 -- 172.21.15.97:0/3162001977 shutdown_connections 2023-07-06T03:55:40.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.874+0000 7f0c2cdff700 1 -- 172.21.15.97:0/3162001977 wait complete. 2023-07-06T03:55:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.875+0000 7f0c2cdff700 1 Processor -- start 2023-07-06T03:55:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.876+0000 7f0c2cdff700 1 -- start start 2023-07-06T03:55:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.876+0000 7f0c2cdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28154890 0x7f0c28070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.876+0000 7f0c2cdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c2815b050 0x7f0c281f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.876+0000 7f0c2cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c2815eb90 0x7f0c2806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2cdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281f1fa0 con 0x7f0c28154890 2023-07-06T03:55:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2cdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281f5ad0 con 0x7f0c2815b050 2023-07-06T03:55:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281f2190 con 0x7f0c2815eb90 2023-07-06T03:55:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28154890 0x7f0c28070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28154890 0x7f0c28070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37170/0 (socket says 172.21.15.97:37170) 2023-07-06T03:55:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2659c700 1 -- 172.21.15.97:0/2841506965 learned_addr learned my addr 172.21.15.97:0/2841506965 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:40.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c26d9d700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c2815eb90 0x7f0c2806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c1ffff700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c2815b050 0x7f0c281f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2659c700 1 -- 172.21.15.97:0/2841506965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c2815eb90 msgr2=0x7f0c2806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2659c700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c2815eb90 0x7f0c2806f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2659c700 1 -- 172.21.15.97:0/2841506965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c2815b050 msgr2=0x7f0c281f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2659c700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c2815b050 0x7f0c281f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:40.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2659c700 1 -- 172.21.15.97:0/2841506965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c180094a0 con 0x7f0c28154890 2023-07-06T03:55:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.877+0000 7f0c2659c700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28154890 0x7f0c28070a30 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f0c180090b0 tx=0x7f0c18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.878+0000 7f0c1effd700 1 -- 172.21.15.97:0/2841506965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c180038a0 con 0x7f0c28154890 2023-07-06T03:55:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.878+0000 7f0c1effd700 1 -- 172.21.15.97:0/2841506965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1801f070 con 0x7f0c28154890 2023-07-06T03:55:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.878+0000 7f0c2cdff700 1 -- 172.21.15.97:0/2841506965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c281f5d00 con 0x7f0c28154890 2023-07-06T03:55:40.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.878+0000 7f0c1effd700 1 -- 172.21.15.97:0/2841506965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18003c20 con 0x7f0c28154890 2023-07-06T03:55:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.878+0000 7f0c2cdff700 1 -- 172.21.15.97:0/2841506965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c281f62c0 con 0x7f0c28154890 2023-07-06T03:55:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.879+0000 7f0c2cdff700 1 -- 172.21.15.97:0/2841506965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2804e340 con 0x7f0c28154890 2023-07-06T03:55:40.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.880+0000 7f0c1effd700 1 -- 172.21.15.97:0/2841506965 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0c1801a430 con 0x7f0c28154890 2023-07-06T03:55:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.881+0000 7f0c1effd700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c0806ace0 0x7f0c0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.881+0000 7f0c1effd700 1 -- 172.21.15.97:0/2841506965 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0c18021070 con 0x7f0c28154890 2023-07-06T03:55:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.881+0000 7f0c1ffff700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c0806ace0 0x7f0c0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:40.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.881+0000 7f0c1ffff700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c0806ace0 0x7f0c0806d190 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0c28157fc0 tx=0x7f0c10007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:40.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:40.885+0000 7f0c1effd700 1 -- 172.21.15.97:0/2841506965 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0c18055fa0 con 0x7f0c28154890 2023-07-06T03:55:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.100+0000 7f0c2cdff700 1 -- 172.21.15.97:0/2841506965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0c280c88d0 con 0x7f0c28154890 2023-07-06T03:55:41.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.100+0000 7f0c1effd700 1 -- 172.21.15.97:0/2841506965 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v124) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0c1802b080 con 0x7f0c28154890 2023-07-06T03:55:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.102+0000 7f0c2cdff700 1 -- 172.21.15.97:0/2841506965 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c0806ace0 msgr2=0x7f0c0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.102+0000 7f0c2cdff700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c0806ace0 0x7f0c0806d190 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0c28157fc0 tx=0x7f0c10007c90 comp rx=0 tx=0).stop 2023-07-06T03:55:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.102+0000 7f0c2cdff700 1 -- 172.21.15.97:0/2841506965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28154890 msgr2=0x7f0c28070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.102+0000 7f0c2cdff700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28154890 0x7f0c28070a30 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f0c180090b0 tx=0x7f0c18004750 comp rx=0 tx=0).stop 2023-07-06T03:55:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.102+0000 7f0c2cdff700 1 -- 172.21.15.97:0/2841506965 shutdown_connections 2023-07-06T03:55:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.102+0000 7f0c2cdff700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0c0806ace0 0x7f0c0806d190 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.102+0000 7f0c2cdff700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0c2815eb90 0x7f0c2806f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.102+0000 7f0c2cdff700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28154890 0x7f0c28070a30 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.102+0000 7f0c2cdff700 1 --2- 172.21.15.97:0/2841506965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c2815b050 0x7f0c281f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.102+0000 7f0c2cdff700 1 -- 172.21.15.97:0/2841506965 >> 172.21.15.97:0/2841506965 conn(0x7f0c2806a060 msgr2=0x7f0c280c4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.103+0000 7f0c2cdff700 1 -- 172.21.15.97:0/2841506965 shutdown_connections 2023-07-06T03:55:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.103+0000 7f0c2cdff700 1 -- 172.21.15.97:0/2841506965 wait complete. 2023-07-06T03:55:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:55:41.122 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:55:41.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:41.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:55:41.189 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[163390]: fuse finished with error 0 and tester_r 0 2023-07-06T03:55:41.192 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:55:41.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:41.193 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:55:41.367 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:55:41.368 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:41.415 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:55:41.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.607+0000 7f86081ff700 1 Processor -- start 2023-07-06T03:55:41.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.607+0000 7f86081ff700 1 -- start start 2023-07-06T03:55:41.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.608+0000 7f86081ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f860015c520 0x7f86000cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:41.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.608+0000 7f86081ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8600160180 0x7f86001605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:41.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.608+0000 7f86081ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86000cdae0 con 0x7f8600160b30 2023-07-06T03:55:41.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.608+0000 7f86081ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86000cd8d0 con 0x7f860015c520 2023-07-06T03:55:41.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.608+0000 7f86081ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86000b6640 con 0x7f8600160180 2023-07-06T03:55:41.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.609+0000 7f8605f9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f860015c520 0x7f86000cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:41.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.609+0000 7f8605f9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f860015c520 0x7f86000cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37176/0 (socket says 172.21.15.97:37176) 2023-07-06T03:55:41.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.609+0000 7f8605f9b700 1 -- 172.21.15.97:0/3910451423 learned_addr learned my addr 172.21.15.97:0/3910451423 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:41.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.609+0000 7f860679c700 1 --1- 172.21.15.97:0/3910451423 >> v1:172.21.15.97:6789/0 conn(0x7f8600160b30 0x7f86000cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43340/0 (socket says 172.21.15.97:43340) 2023-07-06T03:55:41.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.609+0000 7f860579a700 1 --2- 172.21.15.97:0/3910451423 >> v2:172.21.15.146:3300/0 conn(0x7f8600160180 0x7f86001605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:41.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.609+0000 7f8605f9b700 1 -- 172.21.15.97:0/3910451423 >> v2:172.21.15.146:3300/0 conn(0x7f8600160180 msgr2=0x7f86001605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:41.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.609+0000 7f8605f9b700 1 --2- 172.21.15.97:0/3910451423 >> v2:172.21.15.146:3300/0 conn(0x7f8600160180 0x7f86001605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.609+0000 7f8605f9b700 1 -- 172.21.15.97:0/3910451423 >> v1:172.21.15.97:6789/0 conn(0x7f8600160b30 legacy=0x7f86000cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.609+0000 7f8605f9b700 1 -- 172.21.15.97:0/3910451423 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86000b6920 con 0x7f860015c520 2023-07-06T03:55:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.609+0000 7f8604f99700 1 -- 172.21.15.97:0/3910451423 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445289940 0 0) 0x7f86000cdae0 con 0x7f8600160b30 2023-07-06T03:55:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.609+0000 7f8605f9b700 1 --2- 172.21.15.97:0/3910451423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f860015c520 0x7f86000cd390 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f86000b5930 tx=0x7f85fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53979114b94c13c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.610+0000 7f8604f99700 1 -- 172.21.15.97:0/3910451423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc011830 con 0x7f860015c520 2023-07-06T03:55:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.610+0000 7f8604f99700 1 -- 172.21.15.97:0/3910451423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc011990 con 0x7f860015c520 2023-07-06T03:55:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.610+0000 7f8604f99700 1 -- 172.21.15.97:0/3910451423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc011d20 con 0x7f860015c520 2023-07-06T03:55:41.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.610+0000 7f86081ff700 1 -- 172.21.15.97:0/3910451423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f860015c520 msgr2=0x7f86000cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:41.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.610+0000 7f86081ff700 1 --2- 172.21.15.97:0/3910451423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f860015c520 0x7f86000cd390 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f86000b5930 tx=0x7f85fc009bd0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.610+0000 7f86081ff700 1 -- 172.21.15.97:0/3910451423 shutdown_connections 2023-07-06T03:55:41.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.610+0000 7f86081ff700 1 --2- 172.21.15.97:0/3910451423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f860015c520 0x7f86000cd390 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.610+0000 7f86081ff700 1 --2- 172.21.15.97:0/3910451423 >> v2:172.21.15.146:3300/0 conn(0x7f8600160180 0x7f86001605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.610+0000 7f86081ff700 1 -- 172.21.15.97:0/3910451423 >> 172.21.15.97:0/3910451423 conn(0x7f8600071790 msgr2=0x7f8600071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:41.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.610+0000 7f86081ff700 1 -- 172.21.15.97:0/3910451423 shutdown_connections 2023-07-06T03:55:41.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.610+0000 7f86081ff700 1 -- 172.21.15.97:0/3910451423 wait complete. 2023-07-06T03:55:41.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.612+0000 7f86081ff700 1 Processor -- start 2023-07-06T03:55:41.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.612+0000 7f86081ff700 1 -- start start 2023-07-06T03:55:41.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.613+0000 7f86081ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f860015c520 0x7f860014cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:41.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.613+0000 7f86081ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8600160180 0x7f86000cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:41.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.613+0000 7f86081ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8600160b30 0x7f86000cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:41.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.613+0000 7f86081ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86000cad70 con 0x7f860015c520 2023-07-06T03:55:41.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.613+0000 7f86081ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86000c9e10 con 0x7f8600160180 2023-07-06T03:55:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.613+0000 7f86081ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86000ca0f0 con 0x7f8600160b30 2023-07-06T03:55:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.613+0000 7f8605f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f860015c520 0x7f860014cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.613+0000 7f8605f9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f860015c520 0x7f860014cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37186/0 (socket says 172.21.15.97:37186) 2023-07-06T03:55:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.613+0000 7f8605f9b700 1 -- 172.21.15.97:0/2345659073 learned_addr learned my addr 172.21.15.97:0/2345659073 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f860679c700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8600160b30 0x7f86000cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f860579a700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8600160180 0x7f86000cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f8605f9b700 1 -- 172.21.15.97:0/2345659073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8600160b30 msgr2=0x7f86000cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:41.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f8605f9b700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8600160b30 0x7f86000cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f8605f9b700 1 -- 172.21.15.97:0/2345659073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8600160180 msgr2=0x7f86000cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f8605f9b700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8600160180 0x7f86000cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f8605f9b700 1 -- 172.21.15.97:0/2345659073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc009550 con 0x7f860015c520 2023-07-06T03:55:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f860679c700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8600160b30 0x7f86000cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f860579a700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8600160180 0x7f86000cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f8605f9b700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f860015c520 0x7f860014cd30 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009c10 tx=0x7f85fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:41.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f85f67fc700 1 -- 172.21.15.97:0/2345659073 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0036a0 con 0x7f860015c520 2023-07-06T03:55:41.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f85f67fc700 1 -- 172.21.15.97:0/2345659073 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc01e070 con 0x7f860015c520 2023-07-06T03:55:41.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f85f67fc700 1 -- 172.21.15.97:0/2345659073 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc003a20 con 0x7f860015c520 2023-07-06T03:55:41.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f86081ff700 1 -- 172.21.15.97:0/2345659073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86000ca3d0 con 0x7f860015c520 2023-07-06T03:55:41.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.614+0000 7f86081ff700 1 -- 172.21.15.97:0/2345659073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86000ca990 con 0x7f860015c520 2023-07-06T03:55:41.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.615+0000 7f86081ff700 1 -- 172.21.15.97:0/2345659073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86000caeb0 con 0x7f860015c520 2023-07-06T03:55:41.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.616+0000 7f85f67fc700 1 -- 172.21.15.97:0/2345659073 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85fc003b80 con 0x7f860015c520 2023-07-06T03:55:41.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.617+0000 7f85f67fc700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f85e406ace0 0x7f85e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:41.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.617+0000 7f85f67fc700 1 -- 172.21.15.97:0/2345659073 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f85fc08a4f0 con 0x7f860015c520 2023-07-06T03:55:41.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.617+0000 7f860579a700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f85e406ace0 0x7f85e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:41.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.618+0000 7f860579a700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f85e406ace0 0x7f85e406d190 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f86000b7630 tx=0x7f85f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:41.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.621+0000 7f85f67fc700 1 -- 172.21.15.97:0/2345659073 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86000caeb0 con 0x7f860015c520 2023-07-06T03:55:41.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.815+0000 7f86081ff700 1 -- 172.21.15.97:0/2345659073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86000caeb0 con 0x7f860015c520 2023-07-06T03:55:41.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.816+0000 7f85f67fc700 1 -- 172.21.15.97:0/2345659073 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18319 (secure 0 0 0) 0x7f86000caeb0 con 0x7f860015c520 2023-07-06T03:55:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:41.819 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":124,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:55:16.327257+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:55:12.989988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-06T03:55:13.782970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:55:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 -- 172.21.15.97:0/2345659073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f85e406ace0 msgr2=0x7f85e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f85e406ace0 0x7f85e406d190 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f86000b7630 tx=0x7f85f0008040 comp rx=0 tx=0).stop 2023-07-06T03:55:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 -- 172.21.15.97:0/2345659073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f860015c520 msgr2=0x7f860014cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f860015c520 0x7f860014cd30 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009c10 tx=0x7f85fc004750 comp rx=0 tx=0).stop 2023-07-06T03:55:41.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 -- 172.21.15.97:0/2345659073 shutdown_connections 2023-07-06T03:55:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f85e406ace0 0x7f85e406d190 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8600160b30 0x7f86000cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f860015c520 0x7f860014cd30 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 --2- 172.21.15.97:0/2345659073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8600160180 0x7f86000cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 -- 172.21.15.97:0/2345659073 >> 172.21.15.97:0/2345659073 conn(0x7f8600071790 msgr2=0x7f8600071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 -- 172.21.15.97:0/2345659073 shutdown_connections 2023-07-06T03:55:41.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:41.819+0000 7f86081ff700 1 -- 172.21.15.97:0/2345659073 wait complete. 2023-07-06T03:55:41.839 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:55:12.989988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:55:13.782970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:55:41.840 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:55:42.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.037+0000 7ff9617ff700 1 Processor -- start 2023-07-06T03:55:42.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.037+0000 7ff9617ff700 1 -- start start 2023-07-06T03:55:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.038+0000 7ff9617ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff95c15c610 0x7ff95c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.038+0000 7ff9617ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff95c0d2210 0x7ff95c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.038+0000 7ff9617ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff95c0cff90 con 0x7ff95c06b360 2023-07-06T03:55:42.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.039+0000 7ff9617ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95c0d0780 con 0x7ff95c15c610 2023-07-06T03:55:42.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.039+0000 7ff9617ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c15c9e0 con 0x7ff95c0d2210 2023-07-06T03:55:42.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.039+0000 7ff95affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff95c15c610 0x7ff95c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.039+0000 7ff95affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff95c15c610 0x7ff95c0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37196/0 (socket says 172.21.15.97:37196) 2023-07-06T03:55:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.039+0000 7ff95affd700 1 -- 172.21.15.97:0/3701405847 learned_addr learned my addr 172.21.15.97:0/3701405847 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.039+0000 7ff95b7fe700 1 --2- 172.21.15.97:0/3701405847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff95c0d2210 0x7ff95c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.039+0000 7ff95affd700 1 -- 172.21.15.97:0/3701405847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff95c0d2210 msgr2=0x7ff95c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:42.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.039+0000 7ff95affd700 1 --2- 172.21.15.97:0/3701405847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff95c0d2210 0x7ff95c0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.039+0000 7ff95affd700 1 -- 172.21.15.97:0/3701405847 >> v1:172.21.15.146:6789/0 conn(0x7ff95c06b360 legacy=0x7ff95c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.039+0000 7ff95affd700 1 -- 172.21.15.97:0/3701405847 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c0b5650 con 0x7ff95c15c610 2023-07-06T03:55:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.039+0000 7ff95affd700 1 --2- 172.21.15.97:0/3701405847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff95c15c610 0x7ff95c0b47b0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0098f0 tx=0x7ff94c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab2b8e08c45a59d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:42.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.040+0000 7ff95a7fc700 1 -- 172.21.15.97:0/3701405847 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00b8a0 con 0x7ff95c15c610 2023-07-06T03:55:42.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.040+0000 7ff95a7fc700 1 -- 172.21.15.97:0/3701405847 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c00ba00 con 0x7ff95c15c610 2023-07-06T03:55:42.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.040+0000 7ff95a7fc700 1 -- 172.21.15.97:0/3701405847 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00bd90 con 0x7ff95c15c610 2023-07-06T03:55:42.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.040+0000 7ff9617ff700 1 -- 172.21.15.97:0/3701405847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff95c15c610 msgr2=0x7ff95c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.040+0000 7ff9617ff700 1 --2- 172.21.15.97:0/3701405847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff95c15c610 0x7ff95c0b47b0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0098f0 tx=0x7ff94c009c00 comp rx=0 tx=0).stop 2023-07-06T03:55:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.040+0000 7ff9617ff700 1 -- 172.21.15.97:0/3701405847 shutdown_connections 2023-07-06T03:55:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.040+0000 7ff9617ff700 1 --2- 172.21.15.97:0/3701405847 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff95c0d2210 0x7ff95c0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.040+0000 7ff9617ff700 1 --2- 172.21.15.97:0/3701405847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff95c15c610 0x7ff95c0b47b0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.040+0000 7ff9617ff700 1 -- 172.21.15.97:0/3701405847 >> 172.21.15.97:0/3701405847 conn(0x7ff95c06a6c0 msgr2=0x7ff95c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.040+0000 7ff9617ff700 1 -- 172.21.15.97:0/3701405847 shutdown_connections 2023-07-06T03:55:42.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.040+0000 7ff9617ff700 1 -- 172.21.15.97:0/3701405847 wait complete. 2023-07-06T03:55:42.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.042+0000 7ff9617ff700 1 Processor -- start 2023-07-06T03:55:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.042+0000 7ff9617ff700 1 -- start start 2023-07-06T03:55:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.043+0000 7ff9617ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff95c06b360 0x7ff95c15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.043+0000 7ff9617ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff95c0d2210 0x7ff95c158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.043+0000 7ff9617ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff95c15c610 0x7ff95c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.043+0000 7ff9617ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0c9ea0 con 0x7ff95c15c610 2023-07-06T03:55:42.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.043+0000 7ff9617ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0ca180 con 0x7ff95c06b360 2023-07-06T03:55:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.043+0000 7ff9617ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0ca460 con 0x7ff95c0d2210 2023-07-06T03:55:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.043+0000 7ff95b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff95c15c610 0x7ff95c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.043+0000 7ff95b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff95c15c610 0x7ff95c0caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37200/0 (socket says 172.21.15.97:37200) 2023-07-06T03:55:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.043+0000 7ff95b7fe700 1 -- 172.21.15.97:0/1971539546 learned_addr learned my addr 172.21.15.97:0/1971539546 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.043+0000 7ff95affd700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff95c06b360 0x7ff95c15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.043+0000 7ff953fff700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff95c0d2210 0x7ff95c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff95b7fe700 1 -- 172.21.15.97:0/1971539546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff95c0d2210 msgr2=0x7ff95c158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff95b7fe700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff95c0d2210 0x7ff95c158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff95b7fe700 1 -- 172.21.15.97:0/1971539546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff95c06b360 msgr2=0x7ff95c15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff95b7fe700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff95c06b360 0x7ff95c15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff95b7fe700 1 -- 172.21.15.97:0/1971539546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c009550 con 0x7ff95c15c610 2023-07-06T03:55:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff953fff700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff95c0d2210 0x7ff95c158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff95b7fe700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff95c15c610 0x7ff95c0caf40 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7ff954007e40 tx=0x7ff95400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff9537fe700 1 -- 172.21.15.97:0/1971539546 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400ede0 con 0x7ff95c15c610 2023-07-06T03:55:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff9537fe700 1 -- 172.21.15.97:0/1971539546 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff954014420 con 0x7ff95c15c610 2023-07-06T03:55:42.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff9617ff700 1 -- 172.21.15.97:0/1971539546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95c0ca740 con 0x7ff95c15c610 2023-07-06T03:55:42.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff9537fe700 1 -- 172.21.15.97:0/1971539546 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9540147d0 con 0x7ff95c15c610 2023-07-06T03:55:42.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.044+0000 7ff9617ff700 1 -- 172.21.15.97:0/1971539546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c15dec0 con 0x7ff95c15c610 2023-07-06T03:55:42.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.045+0000 7ff9617ff700 1 -- 172.21.15.97:0/1971539546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95c04e340 con 0x7ff95c15c610 2023-07-06T03:55:42.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.046+0000 7ff9537fe700 1 -- 172.21.15.97:0/1971539546 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff954014930 con 0x7ff95c15c610 2023-07-06T03:55:42.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.047+0000 7ff9537fe700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff93c06ace0 0x7ff93c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.047+0000 7ff9537fe700 1 -- 172.21.15.97:0/1971539546 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff954089cc0 con 0x7ff95c15c610 2023-07-06T03:55:42.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.047+0000 7ff95affd700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff93c06ace0 0x7ff93c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.048+0000 7ff95affd700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff93c06ace0 0x7ff93c06d190 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff94c014040 tx=0x7ff94c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:42.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.051+0000 7ff9537fe700 1 -- 172.21.15.97:0/1971539546 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff954090080 con 0x7ff95c15c610 2023-07-06T03:55:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.243+0000 7ff9617ff700 1 -- 172.21.15.97:0/1971539546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff95c066010 con 0x7ff95c15c610 2023-07-06T03:55:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.243+0000 7ff9537fe700 1 -- 172.21.15.97:0/1971539546 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff954058810 con 0x7ff95c15c610 2023-07-06T03:55:42.245 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:55:42.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 -- 172.21.15.97:0/1971539546 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff93c06ace0 msgr2=0x7ff93c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff93c06ace0 0x7ff93c06d190 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff94c014040 tx=0x7ff94c008030 comp rx=0 tx=0).stop 2023-07-06T03:55:42.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 -- 172.21.15.97:0/1971539546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff95c15c610 msgr2=0x7ff95c0caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff95c15c610 0x7ff95c0caf40 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7ff954007e40 tx=0x7ff95400e3c0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 -- 172.21.15.97:0/1971539546 shutdown_connections 2023-07-06T03:55:42.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff93c06ace0 0x7ff93c06d190 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff95c15c610 0x7ff95c0caf40 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff95c06b360 0x7ff95c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 --2- 172.21.15.97:0/1971539546 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff95c0d2210 0x7ff95c158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 -- 172.21.15.97:0/1971539546 >> 172.21.15.97:0/1971539546 conn(0x7ff95c06a6c0 msgr2=0x7ff95c0b5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:42.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 -- 172.21.15.97:0/1971539546 shutdown_connections 2023-07-06T03:55:42.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.246+0000 7ff9617ff700 1 -- 172.21.15.97:0/1971539546 wait complete. 2023-07-06T03:55:42.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:55:42.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.460+0000 7ff35f1ff700 1 Processor -- start 2023-07-06T03:55:42.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.460+0000 7ff35f1ff700 1 -- start start 2023-07-06T03:55:42.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.461+0000 7ff35f1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff35815c610 0x7ff3580b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.461+0000 7ff35f1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3580cff90 con 0x7ff3580d2210 2023-07-06T03:55:42.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.461+0000 7ff35f1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3580b5650 con 0x7ff35806b360 2023-07-06T03:55:42.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.461+0000 7ff35f1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580d0780 con 0x7ff35815c610 2023-07-06T03:55:42.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.461+0000 7ff35d79c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff3580d2210 0x7ff3580d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43354/0 (socket says 172.21.15.97:43354) 2023-07-06T03:55:42.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.461+0000 7ff35d79c700 1 -- 172.21.15.97:0/2906696642 learned_addr learned my addr 172.21.15.97:0/2906696642 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:42.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.461+0000 7ff35cf9b700 1 --2- 172.21.15.97:0/2906696642 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff35815c610 0x7ff3580b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.462+0000 7ff3577fe700 1 -- 172.21.15.97:0/2906696642 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271166446 0 0) 0x7ff3580cff90 con 0x7ff3580d2210 2023-07-06T03:55:42.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.462+0000 7ff3577fe700 1 -- 172.21.15.97:0/2906696642 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3480035b0 con 0x7ff3580d2210 2023-07-06T03:55:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.462+0000 7ff3577fe700 1 -- 172.21.15.97:0/2906696642 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687127521 0 0) 0x7ff3580b5650 con 0x7ff35806b360 2023-07-06T03:55:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.462+0000 7ff3577fe700 1 -- 172.21.15.97:0/2906696642 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3580cff90 con 0x7ff35806b360 2023-07-06T03:55:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.462+0000 7ff35cf9b700 1 -- 172.21.15.97:0/2906696642 >> v1:172.21.15.146:6789/0 conn(0x7ff35806b360 legacy=0x7ff3580b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.462+0000 7ff35cf9b700 1 -- 172.21.15.97:0/2906696642 >> v1:172.21.15.97:6789/0 conn(0x7ff3580d2210 legacy=0x7ff3580d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.462+0000 7ff35cf9b700 1 -- 172.21.15.97:0/2906696642 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3580b5400 con 0x7ff35815c610 2023-07-06T03:55:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.462+0000 7ff35cf9b700 1 --2- 172.21.15.97:0/2906696642 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff35815c610 0x7ff3580b47b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009870 tx=0x7ff34c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d08781199e3ec442 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.462+0000 7ff3577fe700 1 -- 172.21.15.97:0/2906696642 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c011830 con 0x7ff35815c610 2023-07-06T03:55:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.463+0000 7ff3577fe700 1 -- 172.21.15.97:0/2906696642 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c011990 con 0x7ff35815c610 2023-07-06T03:55:42.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.463+0000 7ff3577fe700 1 -- 172.21.15.97:0/2906696642 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c011d20 con 0x7ff35815c610 2023-07-06T03:55:42.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.463+0000 7ff35f1ff700 1 -- 172.21.15.97:0/2906696642 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff35815c610 msgr2=0x7ff3580b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.463+0000 7ff35f1ff700 1 --2- 172.21.15.97:0/2906696642 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff35815c610 0x7ff3580b47b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009870 tx=0x7ff34c009b80 comp rx=0 tx=0).stop 2023-07-06T03:55:42.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.463+0000 7ff35f1ff700 1 -- 172.21.15.97:0/2906696642 shutdown_connections 2023-07-06T03:55:42.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.463+0000 7ff35f1ff700 1 --2- 172.21.15.97:0/2906696642 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff35815c610 0x7ff3580b47b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.463+0000 7ff35f1ff700 1 -- 172.21.15.97:0/2906696642 >> 172.21.15.97:0/2906696642 conn(0x7ff35806a6c0 msgr2=0x7ff35806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:42.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.463+0000 7ff35f1ff700 1 -- 172.21.15.97:0/2906696642 shutdown_connections 2023-07-06T03:55:42.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.463+0000 7ff35f1ff700 1 -- 172.21.15.97:0/2906696642 wait complete. 2023-07-06T03:55:42.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.465+0000 7ff35f1ff700 1 Processor -- start 2023-07-06T03:55:42.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.465+0000 7ff35f1ff700 1 -- start start 2023-07-06T03:55:42.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.466+0000 7ff35f1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff35806b360 0x7ff35814b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.466+0000 7ff35f1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3580d2210 0x7ff35815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.466+0000 7ff35f1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff35815c610 0x7ff3580c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.466+0000 7ff35f1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580c91e0 con 0x7ff35815c610 2023-07-06T03:55:42.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.466+0000 7ff35f1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35814be00 con 0x7ff35806b360 2023-07-06T03:55:42.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.466+0000 7ff35f1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580c8560 con 0x7ff3580d2210 2023-07-06T03:55:42.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.466+0000 7ff35d79c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff35815c610 0x7ff3580c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.466+0000 7ff35d79c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff35815c610 0x7ff3580c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37214/0 (socket says 172.21.15.97:37214) 2023-07-06T03:55:42.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.466+0000 7ff35d79c700 1 -- 172.21.15.97:0/312626234 learned_addr learned my addr 172.21.15.97:0/312626234 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:42.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.466+0000 7ff35cf9b700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff35806b360 0x7ff35814b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.466+0000 7ff357fff700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3580d2210 0x7ff35815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff35d79c700 1 -- 172.21.15.97:0/312626234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3580d2210 msgr2=0x7ff35815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff35d79c700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3580d2210 0x7ff35815f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff35d79c700 1 -- 172.21.15.97:0/312626234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff35806b360 msgr2=0x7ff35814b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff35d79c700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff35806b360 0x7ff35814b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff35d79c700 1 -- 172.21.15.97:0/312626234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c0094a0 con 0x7ff35815c610 2023-07-06T03:55:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff35cf9b700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff35806b360 0x7ff35814b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff357fff700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3580d2210 0x7ff35815f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff35d79c700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff35815c610 0x7ff3580c9600 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7ff35000cba0 tx=0x7ff35000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff3557fa700 1 -- 172.21.15.97:0/312626234 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35000ab20 con 0x7ff35815c610 2023-07-06T03:55:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff3557fa700 1 -- 172.21.15.97:0/312626234 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35000ac80 con 0x7ff35815c610 2023-07-06T03:55:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff35f1ff700 1 -- 172.21.15.97:0/312626234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3580c8840 con 0x7ff35815c610 2023-07-06T03:55:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff3557fa700 1 -- 172.21.15.97:0/312626234 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff350012400 con 0x7ff35815c610 2023-07-06T03:55:42.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.467+0000 7ff35f1ff700 1 -- 172.21.15.97:0/312626234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3580c8e00 con 0x7ff35815c610 2023-07-06T03:55:42.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.468+0000 7ff35f1ff700 1 -- 172.21.15.97:0/312626234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3580d01a0 con 0x7ff35815c610 2023-07-06T03:55:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.469+0000 7ff3557fa700 1 -- 172.21.15.97:0/312626234 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff35001b3b0 con 0x7ff35815c610 2023-07-06T03:55:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.471+0000 7ff3557fa700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff33c06ace0 0x7ff33c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.471+0000 7ff3557fa700 1 -- 172.21.15.97:0/312626234 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff3500892e0 con 0x7ff35815c610 2023-07-06T03:55:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.471+0000 7ff35cf9b700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff33c06ace0 0x7ff33c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.472+0000 7ff35cf9b700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff33c06ace0 0x7ff33c06d190 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0090b0 tx=0x7ff34c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:42.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.475+0000 7ff3557fa700 1 -- 172.21.15.97:0/312626234 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff3580d01a0 con 0x7ff35815c610 2023-07-06T03:55:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.669+0000 7ff35f1ff700 1 -- 172.21.15.97:0/312626234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff3580d01a0 con 0x7ff35815c610 2023-07-06T03:55:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.669+0000 7ff3557fa700 1 -- 172.21.15.97:0/312626234 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v124) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff3580d01a0 con 0x7ff35815c610 2023-07-06T03:55:42.672 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:55:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.671+0000 7ff35f1ff700 1 -- 172.21.15.97:0/312626234 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff33c06ace0 msgr2=0x7ff33c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.671+0000 7ff35f1ff700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff33c06ace0 0x7ff33c06d190 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0090b0 tx=0x7ff34c01c040 comp rx=0 tx=0).stop 2023-07-06T03:55:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.672+0000 7ff35f1ff700 1 -- 172.21.15.97:0/312626234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff35815c610 msgr2=0x7ff3580c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.672+0000 7ff35f1ff700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff35815c610 0x7ff3580c9600 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7ff35000cba0 tx=0x7ff35000cf60 comp rx=0 tx=0).stop 2023-07-06T03:55:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.672+0000 7ff35f1ff700 1 -- 172.21.15.97:0/312626234 shutdown_connections 2023-07-06T03:55:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.672+0000 7ff35f1ff700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff33c06ace0 0x7ff33c06d190 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.672+0000 7ff35f1ff700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff35815c610 0x7ff3580c9600 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.672+0000 7ff35f1ff700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff35806b360 0x7ff35814b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.672+0000 7ff35f1ff700 1 --2- 172.21.15.97:0/312626234 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff3580d2210 0x7ff35815f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.672+0000 7ff35f1ff700 1 -- 172.21.15.97:0/312626234 >> 172.21.15.97:0/312626234 conn(0x7ff35806a6c0 msgr2=0x7ff35815b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.672+0000 7ff35f1ff700 1 -- 172.21.15.97:0/312626234 shutdown_connections 2023-07-06T03:55:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.672+0000 7ff35f1ff700 1 -- 172.21.15.97:0/312626234 wait complete. 2023-07-06T03:55:42.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:55:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.887+0000 7f83a3fff700 1 Processor -- start 2023-07-06T03:55:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.887+0000 7f83a3fff700 1 -- start start 2023-07-06T03:55:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.888+0000 7f83a3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f839c15c610 0x7f839c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.888+0000 7f83a3fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839c0cff90 con 0x7f839c0d2210 2023-07-06T03:55:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.888+0000 7f83a3fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839c0b5650 con 0x7f839c06b360 2023-07-06T03:55:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.888+0000 7f83a3fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839c0d0780 con 0x7f839c15c610 2023-07-06T03:55:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.888+0000 7f83a1d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f839c15c610 0x7f839c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.888+0000 7f83a1d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f839c15c610 0x7f839c0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37220/0 (socket says 172.21.15.97:37220) 2023-07-06T03:55:42.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.888+0000 7f83a1d9b700 1 -- 172.21.15.97:0/3821685099 learned_addr learned my addr 172.21.15.97:0/3821685099 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:42.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.888+0000 7f83a259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f839c0d2210 0x7f839c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43362/0 (socket says 172.21.15.97:43362) 2023-07-06T03:55:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.889+0000 7f83a1d9b700 1 -- 172.21.15.97:0/3821685099 >> v1:172.21.15.146:6789/0 conn(0x7f839c06b360 legacy=0x7f839c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.889+0000 7f83a1d9b700 1 -- 172.21.15.97:0/3821685099 >> v1:172.21.15.97:6789/0 conn(0x7f839c0d2210 legacy=0x7f839c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.889+0000 7f83a1d9b700 1 -- 172.21.15.97:0/3821685099 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c0b5400 con 0x7f839c15c610 2023-07-06T03:55:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.889+0000 7f83a0d99700 1 -- 172.21.15.97:0/3821685099 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895782201 0 0) 0x7f839c0cff90 con 0x7f839c0d2210 2023-07-06T03:55:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.889+0000 7f83a1d9b700 1 --2- 172.21.15.97:0/3821685099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f839c15c610 0x7f839c0b47b0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f83980098f0 tx=0x7f8398009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be22a19aaeb85b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.889+0000 7f83a0d99700 1 -- 172.21.15.97:0/3821685099 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800b8a0 con 0x7f839c15c610 2023-07-06T03:55:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.890+0000 7f83a0d99700 1 -- 172.21.15.97:0/3821685099 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839800ba00 con 0x7f839c15c610 2023-07-06T03:55:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.890+0000 7f83a0d99700 1 -- 172.21.15.97:0/3821685099 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800bd90 con 0x7f839c15c610 2023-07-06T03:55:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.890+0000 7f83a3fff700 1 -- 172.21.15.97:0/3821685099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f839c15c610 msgr2=0x7f839c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.890+0000 7f83a3fff700 1 --2- 172.21.15.97:0/3821685099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f839c15c610 0x7f839c0b47b0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f83980098f0 tx=0x7f8398009c00 comp rx=0 tx=0).stop 2023-07-06T03:55:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.890+0000 7f83a3fff700 1 -- 172.21.15.97:0/3821685099 shutdown_connections 2023-07-06T03:55:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.890+0000 7f83a3fff700 1 --2- 172.21.15.97:0/3821685099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f839c15c610 0x7f839c0b47b0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.890+0000 7f83a3fff700 1 -- 172.21.15.97:0/3821685099 >> 172.21.15.97:0/3821685099 conn(0x7f839c06a6c0 msgr2=0x7f839c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.890+0000 7f83a3fff700 1 -- 172.21.15.97:0/3821685099 shutdown_connections 2023-07-06T03:55:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.890+0000 7f83a3fff700 1 -- 172.21.15.97:0/3821685099 wait complete. 2023-07-06T03:55:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.892+0000 7f83a3fff700 1 Processor -- start 2023-07-06T03:55:42.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.892+0000 7f83a3fff700 1 -- start start 2023-07-06T03:55:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f839c06b360 0x7f839c1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a3fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f839c0d2210 0x7f839c15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a3fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f839c15c610 0x7f839c15aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a3fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c1f2e80 con 0x7f839c06b360 2023-07-06T03:55:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a3fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c15cd00 con 0x7f839c0d2210 2023-07-06T03:55:42.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a3fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c15cfe0 con 0x7f839c15c610 2023-07-06T03:55:42.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a1d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f839c06b360 0x7f839c1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a1d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f839c06b360 0x7f839c1583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37222/0 (socket says 172.21.15.97:37222) 2023-07-06T03:55:42.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a1d9b700 1 -- 172.21.15.97:0/2676181050 learned_addr learned my addr 172.21.15.97:0/2676181050 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:42.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a1d9b700 1 -- 172.21.15.97:0/2676181050 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f839c15c610 msgr2=0x7f839c15aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a159a700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f839c0d2210 0x7f839c15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a259c700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f839c15c610 0x7f839c15aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a1d9b700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f839c15c610 0x7f839c15aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a1d9b700 1 -- 172.21.15.97:0/2676181050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f839c0d2210 msgr2=0x7f839c15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a1d9b700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f839c0d2210 0x7f839c15dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a1d9b700 1 -- 172.21.15.97:0/2676181050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8398009550 con 0x7f839c06b360 2023-07-06T03:55:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a259c700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f839c15c610 0x7f839c15aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.893+0000 7f83a159a700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f839c0d2210 0x7f839c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.894+0000 7f83a1d9b700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f839c06b360 0x7f839c1583b0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f839800b0e0 tx=0x7f8398004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:42.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.894+0000 7f83927fc700 1 -- 172.21.15.97:0/2676181050 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398004480 con 0x7f839c06b360 2023-07-06T03:55:42.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.894+0000 7f83927fc700 1 -- 172.21.15.97:0/2676181050 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839801f070 con 0x7f839c06b360 2023-07-06T03:55:42.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.894+0000 7f83a3fff700 1 -- 172.21.15.97:0/2676181050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839c15d2c0 con 0x7f839c06b360 2023-07-06T03:55:42.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.894+0000 7f83927fc700 1 -- 172.21.15.97:0/2676181050 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83980038e0 con 0x7f839c06b360 2023-07-06T03:55:42.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.894+0000 7f83a3fff700 1 -- 172.21.15.97:0/2676181050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839c15e000 con 0x7f839c06b360 2023-07-06T03:55:42.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.895+0000 7f83a3fff700 1 -- 172.21.15.97:0/2676181050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839c15e190 con 0x7f839c06b360 2023-07-06T03:55:42.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.896+0000 7f83927fc700 1 -- 172.21.15.97:0/2676181050 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8398021030 con 0x7f839c06b360 2023-07-06T03:55:42.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.897+0000 7f83927fc700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f838006ace0 0x7f838006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:42.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.897+0000 7f83927fc700 1 -- 172.21.15.97:0/2676181050 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f839808aa20 con 0x7f839c06b360 2023-07-06T03:55:42.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.897+0000 7f83a159a700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f838006ace0 0x7f838006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:42.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.898+0000 7f83a159a700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f838006ace0 0x7f838006d190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f839c15eb60 tx=0x7f838c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:42.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:42.901+0000 7f83927fc700 1 -- 172.21.15.97:0/2676181050 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8398055ac0 con 0x7f839c06b360 2023-07-06T03:55:43.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.098+0000 7f83a3fff700 1 -- 172.21.15.97:0/2676181050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f839c04e340 con 0x7f839c06b360 2023-07-06T03:55:43.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.099+0000 7f83927fc700 1 -- 172.21.15.97:0/2676181050 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f839808a070 con 0x7f839c06b360 2023-07-06T03:55:43.101 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:55:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.101+0000 7f83a3fff700 1 -- 172.21.15.97:0/2676181050 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f838006ace0 msgr2=0x7f838006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.101+0000 7f83a3fff700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f838006ace0 0x7f838006d190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f839c15eb60 tx=0x7f838c009040 comp rx=0 tx=0).stop 2023-07-06T03:55:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.101+0000 7f83a3fff700 1 -- 172.21.15.97:0/2676181050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f839c06b360 msgr2=0x7f839c1583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.101+0000 7f83a3fff700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f839c06b360 0x7f839c1583b0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f839800b0e0 tx=0x7f8398004750 comp rx=0 tx=0).stop 2023-07-06T03:55:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.101+0000 7f83a3fff700 1 -- 172.21.15.97:0/2676181050 shutdown_connections 2023-07-06T03:55:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.101+0000 7f83a3fff700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f838006ace0 0x7f838006d190 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.101+0000 7f83a3fff700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f839c15c610 0x7f839c15aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.101+0000 7f83a3fff700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f839c06b360 0x7f839c1583b0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.101+0000 7f83a3fff700 1 --2- 172.21.15.97:0/2676181050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f839c0d2210 0x7f839c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.102+0000 7f83a3fff700 1 -- 172.21.15.97:0/2676181050 >> 172.21.15.97:0/2676181050 conn(0x7f839c06a6c0 msgr2=0x7f839c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.102+0000 7f83a3fff700 1 -- 172.21.15.97:0/2676181050 shutdown_connections 2023-07-06T03:55:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.102+0000 7f83a3fff700 1 -- 172.21.15.97:0/2676181050 wait complete. 2023-07-06T03:55:43.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:55:43.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.317+0000 7f786ffff700 1 Processor -- start 2023-07-06T03:55:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.318+0000 7f786ffff700 1 -- start start 2023-07-06T03:55:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f786815c610 0x7f78680b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f786806b360 0x7f78680b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78680d2210 0x7f78680d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786ffff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78680d0780 con 0x7f786815c610 2023-07-06T03:55:43.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786ffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786815c9e0 con 0x7f786806b360 2023-07-06T03:55:43.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786ffff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78680b5650 con 0x7f78680d2210 2023-07-06T03:55:43.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786dd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f786815c610 0x7f78680b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:43.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786dd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f786815c610 0x7f78680b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37234/0 (socket says 172.21.15.97:37234) 2023-07-06T03:55:43.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786dd9b700 1 -- 172.21.15.97:0/1988046132 learned_addr learned my addr 172.21.15.97:0/1988046132 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786d59a700 1 --2- 172.21.15.97:0/1988046132 >> v2:172.21.15.146:3300/0 conn(0x7f786806b360 0x7f78680b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786e59c700 1 --2- 172.21.15.97:0/1988046132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78680d2210 0x7f78680d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786dd9b700 1 -- 172.21.15.97:0/1988046132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78680d2210 msgr2=0x7f78680d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.319+0000 7f786dd9b700 1 --2- 172.21.15.97:0/1988046132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78680d2210 0x7f78680d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.320+0000 7f786dd9b700 1 -- 172.21.15.97:0/1988046132 >> v2:172.21.15.146:3300/0 conn(0x7f786806b360 msgr2=0x7f78680b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.320+0000 7f786dd9b700 1 --2- 172.21.15.97:0/1988046132 >> v2:172.21.15.146:3300/0 conn(0x7f786806b360 0x7f78680b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.320+0000 7f786dd9b700 1 -- 172.21.15.97:0/1988046132 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680d01c0 con 0x7f786815c610 2023-07-06T03:55:43.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.320+0000 7f786dd9b700 1 --2- 172.21.15.97:0/1988046132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f786815c610 0x7f78680b47b0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f786806a590 tx=0x7f7864009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98280721f73add72 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:43.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.320+0000 7f786cd99700 1 -- 172.21.15.97:0/1988046132 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400b860 con 0x7f786815c610 2023-07-06T03:55:43.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.320+0000 7f786cd99700 1 -- 172.21.15.97:0/1988046132 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786400b9c0 con 0x7f786815c610 2023-07-06T03:55:43.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.320+0000 7f786cd99700 1 -- 172.21.15.97:0/1988046132 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400bd50 con 0x7f786815c610 2023-07-06T03:55:43.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.320+0000 7f786ffff700 1 -- 172.21.15.97:0/1988046132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f786815c610 msgr2=0x7f78680b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.320+0000 7f786ffff700 1 --2- 172.21.15.97:0/1988046132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f786815c610 0x7f78680b47b0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f786806a590 tx=0x7f7864009c00 comp rx=0 tx=0).stop 2023-07-06T03:55:43.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.321+0000 7f786ffff700 1 -- 172.21.15.97:0/1988046132 shutdown_connections 2023-07-06T03:55:43.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.321+0000 7f786ffff700 1 --2- 172.21.15.97:0/1988046132 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78680d2210 0x7f78680d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.321+0000 7f786ffff700 1 --2- 172.21.15.97:0/1988046132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f786815c610 0x7f78680b47b0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.321+0000 7f786ffff700 1 --2- 172.21.15.97:0/1988046132 >> v2:172.21.15.146:3300/0 conn(0x7f786806b360 0x7f78680b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.321+0000 7f786ffff700 1 -- 172.21.15.97:0/1988046132 >> 172.21.15.97:0/1988046132 conn(0x7f786806a6c0 msgr2=0x7f786806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:43.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.321+0000 7f786ffff700 1 -- 172.21.15.97:0/1988046132 shutdown_connections 2023-07-06T03:55:43.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.321+0000 7f786ffff700 1 -- 172.21.15.97:0/1988046132 wait complete. 2023-07-06T03:55:43.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.322+0000 7f786ffff700 1 Processor -- start 2023-07-06T03:55:43.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.323+0000 7f786ffff700 1 -- start start 2023-07-06T03:55:43.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f786806b360 0x7f78681f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78680d2210 0x7f78681f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f786815c610 0x7f78680d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786ffff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78681f5a70 con 0x7f786815c610 2023-07-06T03:55:43.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78681f5d50 con 0x7f786806b360 2023-07-06T03:55:43.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786ffff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786815cca0 con 0x7f78680d2210 2023-07-06T03:55:43.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f786815c610 0x7f78680d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:43.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f786815c610 0x7f78680d11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37246/0 (socket says 172.21.15.97:37246) 2023-07-06T03:55:43.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786e59c700 1 -- 172.21.15.97:0/1186262109 learned_addr learned my addr 172.21.15.97:0/1186262109 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:43.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786e59c700 1 -- 172.21.15.97:0/1186262109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78680d2210 msgr2=0x7f78681f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786e59c700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78680d2210 0x7f78681f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786e59c700 1 -- 172.21.15.97:0/1186262109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f786806b360 msgr2=0x7f78681f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786e59c700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f786806b360 0x7f78681f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.324+0000 7f786e59c700 1 -- 172.21.15.97:0/1186262109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864009550 con 0x7f786815c610 2023-07-06T03:55:43.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.325+0000 7f786e59c700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f786815c610 0x7f78680d11f0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f7860007e40 tx=0x7f786000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:43.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.325+0000 7f785e7fc700 1 -- 172.21.15.97:0/1186262109 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786000eea0 con 0x7f786815c610 2023-07-06T03:55:43.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.325+0000 7f785e7fc700 1 -- 172.21.15.97:0/1186262109 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7860014420 con 0x7f786815c610 2023-07-06T03:55:43.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.325+0000 7f785e7fc700 1 -- 172.21.15.97:0/1186262109 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78600147b0 con 0x7f786815c610 2023-07-06T03:55:43.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.325+0000 7f786ffff700 1 -- 172.21.15.97:0/1186262109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786815cf80 con 0x7f786815c610 2023-07-06T03:55:43.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.325+0000 7f786ffff700 1 -- 172.21.15.97:0/1186262109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78681f1720 con 0x7f786815c610 2023-07-06T03:55:43.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.326+0000 7f786ffff700 1 -- 172.21.15.97:0/1186262109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78681f1940 con 0x7f786815c610 2023-07-06T03:55:43.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.327+0000 7f785e7fc700 1 -- 172.21.15.97:0/1186262109 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7860014910 con 0x7f786815c610 2023-07-06T03:55:43.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.328+0000 7f785e7fc700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f784c06ace0 0x7f784c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.328+0000 7f785e7fc700 1 -- 172.21.15.97:0/1186262109 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7860089c90 con 0x7f786815c610 2023-07-06T03:55:43.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.328+0000 7f786dd9b700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f784c06ace0 0x7f784c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:43.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.329+0000 7f786dd9b700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f784c06ace0 0x7f784c06d190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7864014040 tx=0x7f7864010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:43.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.332+0000 7f785e7fc700 1 -- 172.21.15.97:0/1186262109 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f78681f1940 con 0x7f786815c610 2023-07-06T03:55:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.527+0000 7f786ffff700 1 -- 172.21.15.97:0/1186262109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78681f1940 con 0x7f786815c610 2023-07-06T03:55:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.528+0000 7f785e7fc700 1 -- 172.21.15.97:0/1186262109 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78681f1940 con 0x7f786815c610 2023-07-06T03:55:43.530 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:55:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 -- 172.21.15.97:0/1186262109 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f784c06ace0 msgr2=0x7f784c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f784c06ace0 0x7f784c06d190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7864014040 tx=0x7f7864010000 comp rx=0 tx=0).stop 2023-07-06T03:55:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 -- 172.21.15.97:0/1186262109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f786815c610 msgr2=0x7f78680d11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f786815c610 0x7f78680d11f0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f7860007e40 tx=0x7f786000e3c0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 -- 172.21.15.97:0/1186262109 shutdown_connections 2023-07-06T03:55:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f784c06ace0 0x7f784c06d190 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f786815c610 0x7f78680d11f0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f786806b360 0x7f78681f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 --2- 172.21.15.97:0/1186262109 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f78680d2210 0x7f78681f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 -- 172.21.15.97:0/1186262109 >> 172.21.15.97:0/1186262109 conn(0x7f786806a6c0 msgr2=0x7f78680b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 -- 172.21.15.97:0/1186262109 shutdown_connections 2023-07-06T03:55:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.530+0000 7f786ffff700 1 -- 172.21.15.97:0/1186262109 wait complete. 2023-07-06T03:55:43.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:55:43.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.742+0000 7f09035ff700 1 Processor -- start 2023-07-06T03:55:43.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.743+0000 7f09035ff700 1 -- start start 2023-07-06T03:55:43.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f09035ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08fc15c610 0x7f08fc0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f09035ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f08fc0d2210 0x7f08fc0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f09035ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08fc0cff90 con 0x7f08fc06b360 2023-07-06T03:55:43.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f09035ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc0d0780 con 0x7f08fc15c610 2023-07-06T03:55:43.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f09035ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc15c9e0 con 0x7f08fc0d2210 2023-07-06T03:55:43.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f090139b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08fc15c610 0x7f08fc0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:43.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f090139b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08fc15c610 0x7f08fc0b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37258/0 (socket says 172.21.15.97:37258) 2023-07-06T03:55:43.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f090139b700 1 -- 172.21.15.97:0/152014935 learned_addr learned my addr 172.21.15.97:0/152014935 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f0901b9c700 1 --2- 172.21.15.97:0/152014935 >> v2:172.21.15.146:3300/0 conn(0x7f08fc0d2210 0x7f08fc0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f090139b700 1 -- 172.21.15.97:0/152014935 >> v2:172.21.15.146:3300/0 conn(0x7f08fc0d2210 msgr2=0x7f08fc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f090139b700 1 --2- 172.21.15.97:0/152014935 >> v2:172.21.15.146:3300/0 conn(0x7f08fc0d2210 0x7f08fc0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.744+0000 7f090139b700 1 -- 172.21.15.97:0/152014935 >> v1:172.21.15.146:6789/0 conn(0x7f08fc06b360 legacy=0x7f08fc0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.745+0000 7f090139b700 1 -- 172.21.15.97:0/152014935 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc0b5650 con 0x7f08fc15c610 2023-07-06T03:55:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.745+0000 7f090139b700 1 --2- 172.21.15.97:0/152014935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08fc15c610 0x7f08fc0b47b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0d2bc0 tx=0x7f08f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93305f32ccb50044 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.745+0000 7f08f3fff700 1 -- 172.21.15.97:0/152014935 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f800b860 con 0x7f08fc15c610 2023-07-06T03:55:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.745+0000 7f08f3fff700 1 -- 172.21.15.97:0/152014935 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f800b9c0 con 0x7f08fc15c610 2023-07-06T03:55:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.745+0000 7f08f3fff700 1 -- 172.21.15.97:0/152014935 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f800bd50 con 0x7f08fc15c610 2023-07-06T03:55:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.745+0000 7f09035ff700 1 -- 172.21.15.97:0/152014935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08fc15c610 msgr2=0x7f08fc0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.745+0000 7f09035ff700 1 --2- 172.21.15.97:0/152014935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08fc15c610 0x7f08fc0b47b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0d2bc0 tx=0x7f08f8009c00 comp rx=0 tx=0).stop 2023-07-06T03:55:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.746+0000 7f09035ff700 1 -- 172.21.15.97:0/152014935 shutdown_connections 2023-07-06T03:55:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.746+0000 7f09035ff700 1 --2- 172.21.15.97:0/152014935 >> v2:172.21.15.146:3300/0 conn(0x7f08fc0d2210 0x7f08fc0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.746+0000 7f09035ff700 1 --2- 172.21.15.97:0/152014935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08fc15c610 0x7f08fc0b47b0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.746+0000 7f09035ff700 1 -- 172.21.15.97:0/152014935 >> 172.21.15.97:0/152014935 conn(0x7f08fc06a6c0 msgr2=0x7f08fc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.746+0000 7f09035ff700 1 -- 172.21.15.97:0/152014935 shutdown_connections 2023-07-06T03:55:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.746+0000 7f09035ff700 1 -- 172.21.15.97:0/152014935 wait complete. 2023-07-06T03:55:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.747+0000 7f09035ff700 1 Processor -- start 2023-07-06T03:55:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.747+0000 7f09035ff700 1 -- start start 2023-07-06T03:55:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.748+0000 7f09035ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08fc06b360 0x7f08fc15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.748+0000 7f09035ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08fc0d2210 0x7f08fc15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.748+0000 7f09035ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08fc15c610 0x7f08fc15f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.748+0000 7f09035ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0c7750 con 0x7f08fc15c610 2023-07-06T03:55:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.748+0000 7f09035ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0c96a0 con 0x7f08fc06b360 2023-07-06T03:55:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.748+0000 7f09035ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc15d200 con 0x7f08fc0d2210 2023-07-06T03:55:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f090139b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08fc06b360 0x7f08fc15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f0901b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08fc15c610 0x7f08fc15f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f090139b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08fc06b360 0x7f08fc15cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:41744/0 (socket says 172.21.15.97:41744) 2023-07-06T03:55:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f090139b700 1 -- 172.21.15.97:0/3076427598 learned_addr learned my addr 172.21.15.97:0/3076427598 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f0900b9a700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08fc0d2210 0x7f08fc15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:43.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f0901b9c700 1 -- 172.21.15.97:0/3076427598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08fc0d2210 msgr2=0x7f08fc15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f0901b9c700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08fc0d2210 0x7f08fc15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f0901b9c700 1 -- 172.21.15.97:0/3076427598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08fc06b360 msgr2=0x7f08fc15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f0901b9c700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08fc06b360 0x7f08fc15cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f0901b9c700 1 -- 172.21.15.97:0/3076427598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f8009550 con 0x7f08fc15c610 2023-07-06T03:55:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f090139b700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08fc06b360 0x7f08fc15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:55:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.749+0000 7f0901b9c700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08fc15c610 0x7f08fc15f040 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f08f4007d90 tx=0x7f08f400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:43.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.750+0000 7f08f1ffb700 1 -- 172.21.15.97:0/3076427598 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4018070 con 0x7f08fc15c610 2023-07-06T03:55:43.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.750+0000 7f08f1ffb700 1 -- 172.21.15.97:0/3076427598 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f400eb30 con 0x7f08fc15c610 2023-07-06T03:55:43.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.750+0000 7f08f1ffb700 1 -- 172.21.15.97:0/3076427598 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4003a20 con 0x7f08fc15c610 2023-07-06T03:55:43.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.750+0000 7f09035ff700 1 -- 172.21.15.97:0/3076427598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc0c6db0 con 0x7f08fc15c610 2023-07-06T03:55:43.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.750+0000 7f09035ff700 1 -- 172.21.15.97:0/3076427598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc0c7370 con 0x7f08fc15c610 2023-07-06T03:55:43.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.751+0000 7f09035ff700 1 -- 172.21.15.97:0/3076427598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08fc04e340 con 0x7f08fc15c610 2023-07-06T03:55:43.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.752+0000 7f08f1ffb700 1 -- 172.21.15.97:0/3076427598 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f08f4014420 con 0x7f08fc15c610 2023-07-06T03:55:43.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.753+0000 7f08f1ffb700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f08e006ace0 0x7f08e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:43.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.753+0000 7f08f1ffb700 1 -- 172.21.15.97:0/3076427598 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f08f40893e0 con 0x7f08fc15c610 2023-07-06T03:55:43.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.753+0000 7f090139b700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f08e006ace0 0x7f08e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:43.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.754+0000 7f090139b700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f08e006ace0 0x7f08e006d190 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009c40 tx=0x7f08f8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:43.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.757+0000 7f08f1ffb700 1 -- 172.21.15.97:0/3076427598 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f08f4054480 con 0x7f08fc15c610 2023-07-06T03:55:43.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.968+0000 7f09035ff700 1 -- 172.21.15.97:0/3076427598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f08fc0cd4f0 con 0x7f08fc15c610 2023-07-06T03:55:43.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.968+0000 7f08f1ffb700 1 -- 172.21.15.97:0/3076427598 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v124) v1 ==== 80+0+0 (secure 0 0 0) 0x7f08f4057aa0 con 0x7f08fc15c610 2023-07-06T03:55:43.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.970+0000 7f09035ff700 1 -- 172.21.15.97:0/3076427598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f08e006ace0 msgr2=0x7f08e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.970+0000 7f09035ff700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f08e006ace0 0x7f08e006d190 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009c40 tx=0x7f08f8008030 comp rx=0 tx=0).stop 2023-07-06T03:55:43.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.970+0000 7f09035ff700 1 -- 172.21.15.97:0/3076427598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08fc15c610 msgr2=0x7f08fc15f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:43.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.970+0000 7f09035ff700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08fc15c610 0x7f08fc15f040 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f08f4007d90 tx=0x7f08f400e3c0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.971+0000 7f09035ff700 1 -- 172.21.15.97:0/3076427598 shutdown_connections 2023-07-06T03:55:43.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.971+0000 7f09035ff700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f08e006ace0 0x7f08e006d190 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.971+0000 7f09035ff700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08fc15c610 0x7f08fc15f040 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.971+0000 7f09035ff700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08fc06b360 0x7f08fc15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.971+0000 7f09035ff700 1 --2- 172.21.15.97:0/3076427598 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f08fc0d2210 0x7f08fc15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:43.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.971+0000 7f09035ff700 1 -- 172.21.15.97:0/3076427598 >> 172.21.15.97:0/3076427598 conn(0x7f08fc06a6c0 msgr2=0x7f08fc0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:43.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.971+0000 7f09035ff700 1 -- 172.21.15.97:0/3076427598 shutdown_connections 2023-07-06T03:55:43.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:43.971+0000 7f09035ff700 1 -- 172.21.15.97:0/3076427598 wait complete. 2023-07-06T03:55:43.974 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:55:43.991 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2023-07-06T03:55:43.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:43.992 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-06T03:55:44.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[126921]: fuse finished with error 0 and tester_r 0 2023-07-06T03:55:44.068 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:55:44.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:44.069 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T03:55:44.229 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:55:44.230 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:44.276 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:55:44.277 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:55:44.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:44.277 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T03:55:44.360 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T03:55:44.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:55:44.362 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:55:44.362 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:44.410 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:55:44.411 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:55:44.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:55:44.411 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T03:55:44.496 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T03:55:44.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:55:44.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:55:44.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.691+0000 7fd1543ff700 1 Processor -- start 2023-07-06T03:55:44.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.691+0000 7fd1543ff700 1 -- start start 2023-07-06T03:55:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.692+0000 7fd1543ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 0x7fd14c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.692+0000 7fd1543ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd14c160180 0x7fd14c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.692+0000 7fd1543ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0cdae0 con 0x7fd14c160b30 2023-07-06T03:55:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.692+0000 7fd1543ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14c0cd8d0 con 0x7fd14c160180 2023-07-06T03:55:44.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.692+0000 7fd1543ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0b6640 con 0x7fd14c15c520 2023-07-06T03:55:44.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.693+0000 7fd15219b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 0x7fd14c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:44.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.693+0000 7fd15219b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 0x7fd14c0cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:44536/0 (socket says 172.21.15.97:44536) 2023-07-06T03:55:44.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.693+0000 7fd15219b700 1 -- 172.21.15.97:0/1003876257 learned_addr learned my addr 172.21.15.97:0/1003876257 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:44.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.693+0000 7fd15199a700 1 --2- 172.21.15.97:0/1003876257 >> v2:172.21.15.146:3300/0 conn(0x7fd14c160180 0x7fd14c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:44.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.693+0000 7fd15199a700 1 -- 172.21.15.97:0/1003876257 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 msgr2=0x7fd14c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:44.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.693+0000 7fd15199a700 1 --2- 172.21.15.97:0/1003876257 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 0x7fd14c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:44.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.693+0000 7fd15199a700 1 -- 172.21.15.97:0/1003876257 >> v1:172.21.15.146:6789/0 conn(0x7fd14c160b30 legacy=0x7fd14c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:44.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.693+0000 7fd15199a700 1 -- 172.21.15.97:0/1003876257 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c0b6920 con 0x7fd14c160180 2023-07-06T03:55:44.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.693+0000 7fd15219b700 1 --2- 172.21.15.97:0/1003876257 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 0x7fd14c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:55:44.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.694+0000 7fd15199a700 1 --2- 172.21.15.97:0/1003876257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd14c160180 0x7fd14c1605f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009840 tx=0x7fd13c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=454f96a04316b7f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:44.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.694+0000 7fd151199700 1 -- 172.21.15.97:0/1003876257 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c011830 con 0x7fd14c160180 2023-07-06T03:55:44.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.694+0000 7fd151199700 1 -- 172.21.15.97:0/1003876257 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c011990 con 0x7fd14c160180 2023-07-06T03:55:44.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.694+0000 7fd151199700 1 -- 172.21.15.97:0/1003876257 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c011d20 con 0x7fd14c160180 2023-07-06T03:55:44.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.694+0000 7fd1543ff700 1 -- 172.21.15.97:0/1003876257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd14c160180 msgr2=0x7fd14c1605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:44.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.694+0000 7fd1543ff700 1 --2- 172.21.15.97:0/1003876257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd14c160180 0x7fd14c1605f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009840 tx=0x7fd13c009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:44.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.694+0000 7fd1543ff700 1 -- 172.21.15.97:0/1003876257 shutdown_connections 2023-07-06T03:55:44.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.694+0000 7fd1543ff700 1 --2- 172.21.15.97:0/1003876257 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 0x7fd14c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:44.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.694+0000 7fd1543ff700 1 --2- 172.21.15.97:0/1003876257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd14c160180 0x7fd14c1605f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:44.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.694+0000 7fd1543ff700 1 -- 172.21.15.97:0/1003876257 >> 172.21.15.97:0/1003876257 conn(0x7fd14c071790 msgr2=0x7fd14c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:44.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.695+0000 7fd1543ff700 1 -- 172.21.15.97:0/1003876257 shutdown_connections 2023-07-06T03:55:44.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.695+0000 7fd1543ff700 1 -- 172.21.15.97:0/1003876257 wait complete. 2023-07-06T03:55:44.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.696+0000 7fd1543ff700 1 Processor -- start 2023-07-06T03:55:44.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.696+0000 7fd1543ff700 1 -- start start 2023-07-06T03:55:44.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.697+0000 7fd1543ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 0x7fd14c15cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:44.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.697+0000 7fd1543ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd14c160180 0x7fd14c1e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:44.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.697+0000 7fd1543ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd14c160b30 0x7fd14c148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:44.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.697+0000 7fd1543ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c1e7f00 con 0x7fd14c160180 2023-07-06T03:55:44.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.697+0000 7fd1543ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c1e8130 con 0x7fd14c160b30 2023-07-06T03:55:44.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.697+0000 7fd1543ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c1e8410 con 0x7fd14c15c520 2023-07-06T03:55:44.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15199a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd14c160180 0x7fd14c1e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:44.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15199a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd14c160180 0x7fd14c1e79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37282/0 (socket says 172.21.15.97:37282) 2023-07-06T03:55:44.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15199a700 1 -- 172.21.15.97:0/171850939 learned_addr learned my addr 172.21.15.97:0/171850939 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:44.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15219b700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 0x7fd14c15cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:44.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15299c700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd14c160b30 0x7fd14c148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:44.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15199a700 1 -- 172.21.15.97:0/171850939 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 msgr2=0x7fd14c15cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:44.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15199a700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 0x7fd14c15cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:44.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15199a700 1 -- 172.21.15.97:0/171850939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd14c160b30 msgr2=0x7fd14c148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:44.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15199a700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd14c160b30 0x7fd14c148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:44.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15199a700 1 -- 172.21.15.97:0/171850939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd148009550 con 0x7fd14c160180 2023-07-06T03:55:44.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15299c700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd14c160b30 0x7fd14c148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:44.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd15199a700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd14c160180 0x7fd14c1e79c0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009fd0 tx=0x7fd13c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:44.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd142ffd700 1 -- 172.21.15.97:0/171850939 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0036a0 con 0x7fd14c160180 2023-07-06T03:55:44.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd142ffd700 1 -- 172.21.15.97:0/171850939 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c01f070 con 0x7fd14c160180 2023-07-06T03:55:44.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd142ffd700 1 -- 172.21.15.97:0/171850939 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c003a60 con 0x7fd14c160180 2023-07-06T03:55:44.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.698+0000 7fd1543ff700 1 -- 172.21.15.97:0/171850939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c0094a0 con 0x7fd14c160180 2023-07-06T03:55:44.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.699+0000 7fd1543ff700 1 -- 172.21.15.97:0/171850939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c146020 con 0x7fd14c160180 2023-07-06T03:55:44.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.699+0000 7fd1543ff700 1 -- 172.21.15.97:0/171850939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c04e340 con 0x7fd14c160180 2023-07-06T03:55:44.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.701+0000 7fd142ffd700 1 -- 172.21.15.97:0/171850939 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd13c003bc0 con 0x7fd14c160180 2023-07-06T03:55:44.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.702+0000 7fd142ffd700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd13006ace0 0x7fd13006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:44.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.703+0000 7fd142ffd700 1 -- 172.21.15.97:0/171850939 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd13c021070 con 0x7fd14c160180 2023-07-06T03:55:44.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.703+0000 7fd15219b700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd13006ace0 0x7fd13006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:44.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.703+0000 7fd15219b700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd13006ace0 0x7fd13006d190 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0d0060 tx=0x7fd148008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.706+0000 7fd142ffd700 1 -- 172.21.15.97:0/171850939 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd13c0562a0 con 0x7fd14c160180 2023-07-06T03:55:44.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.944+0000 7fd1543ff700 1 -- 172.21.15.97:0/171850939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd14c0b4f40 con 0x7fd14c160180 2023-07-06T03:55:44.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.945+0000 7fd142ffd700 1 -- 172.21.15.97:0/171850939 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4679 (secure 0 0 0) 0x7fd13c0598c0 con 0x7fd14c160180 2023-07-06T03:55:44.947 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:44.947 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3059444241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3059444241},{"type":"v1","addr":"172.21.15.146:6835","nonce":3059444241}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2624688851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2624688851},{"type":"v1","addr":"172.21.15.97:6837","nonce":2624688851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3938376776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3938376776},{"type":"v1","addr":"172.21.15.97:6835","nonce":3938376776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:55:14.806943+0000","modified":"2023-07-06T03:55:35.906574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9099":{"gid":9099,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6834/3269084599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3269084599},{"type":"v1","addr":"172.21.15.146:6834","nonce":3269084599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-06T03:55:44.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 -- 172.21.15.97:0/171850939 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd13006ace0 msgr2=0x7fd13006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:44.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd13006ace0 0x7fd13006d190 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0d0060 tx=0x7fd148008040 comp rx=0 tx=0).stop 2023-07-06T03:55:44.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 -- 172.21.15.97:0/171850939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd14c160180 msgr2=0x7fd14c1e79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:44.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd14c160180 0x7fd14c1e79c0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009fd0 tx=0x7fd13c004750 comp rx=0 tx=0).stop 2023-07-06T03:55:44.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 -- 172.21.15.97:0/171850939 shutdown_connections 2023-07-06T03:55:44.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd13006ace0 0x7fd13006d190 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:44.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd14c160b30 0x7fd14c148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:44.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd14c15c520 0x7fd14c15cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:44.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 --2- 172.21.15.97:0/171850939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd14c160180 0x7fd14c1e79c0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:44.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 -- 172.21.15.97:0/171850939 >> 172.21.15.97:0/171850939 conn(0x7fd14c071790 msgr2=0x7fd14c156a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:44.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 -- 172.21.15.97:0/171850939 shutdown_connections 2023-07-06T03:55:44.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:44.947+0000 7fd1543ff700 1 -- 172.21.15.97:0/171850939 wait complete. 2023-07-06T03:55:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 94 2023-07-06T03:55:44.968 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:55:45.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.162+0000 7f637b5ff700 1 Processor -- start 2023-07-06T03:55:45.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.162+0000 7f637b5ff700 1 -- start start 2023-07-06T03:55:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.163+0000 7f637b5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f637415c520 0x7f63740cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.163+0000 7f637b5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6374160d00 0x7f63740b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.163+0000 7f637b5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63740cdae0 con 0x7f6374160180 2023-07-06T03:55:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.163+0000 7f637b5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63740cd8d0 con 0x7f6374160d00 2023-07-06T03:55:45.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.163+0000 7f637b5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63740b6640 con 0x7f637415c520 2023-07-06T03:55:45.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.163+0000 7f6378b9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6374160180 0x7f63741605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43364/0 (socket says 172.21.15.97:43364) 2023-07-06T03:55:45.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.164+0000 7f6378b9a700 1 -- 172.21.15.97:0/1458716259 learned_addr learned my addr 172.21.15.97:0/1458716259 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:45.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.164+0000 7f637939b700 1 --2- 172.21.15.97:0/1458716259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f637415c520 0x7f63740cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:45.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.164+0000 7f6379b9c700 1 --2- 172.21.15.97:0/1458716259 >> v2:172.21.15.146:3300/0 conn(0x7f6374160d00 0x7f63740b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:45.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.164+0000 7f636bfff700 1 -- 172.21.15.97:0/1458716259 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705031603 0 0) 0x7f63740cdae0 con 0x7f6374160180 2023-07-06T03:55:45.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.164+0000 7f636bfff700 1 -- 172.21.15.97:0/1458716259 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63600035b0 con 0x7f6374160180 2023-07-06T03:55:45.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.164+0000 7f637939b700 1 -- 172.21.15.97:0/1458716259 >> v2:172.21.15.146:3300/0 conn(0x7f6374160d00 msgr2=0x7f63740b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.164+0000 7f637939b700 1 --2- 172.21.15.97:0/1458716259 >> v2:172.21.15.146:3300/0 conn(0x7f6374160d00 0x7f63740b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.164+0000 7f637939b700 1 -- 172.21.15.97:0/1458716259 >> v1:172.21.15.97:6789/0 conn(0x7f6374160180 legacy=0x7f63741605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.164+0000 7f637939b700 1 -- 172.21.15.97:0/1458716259 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63740b6920 con 0x7f637415c520 2023-07-06T03:55:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.164+0000 7f637939b700 1 --2- 172.21.15.97:0/1458716259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f637415c520 0x7f63740cd390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6370009840 tx=0x7f6370009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a95f1aa6a3644c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.165+0000 7f636bfff700 1 -- 172.21.15.97:0/1458716259 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370011830 con 0x7f637415c520 2023-07-06T03:55:45.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.165+0000 7f636bfff700 1 -- 172.21.15.97:0/1458716259 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6370011990 con 0x7f637415c520 2023-07-06T03:55:45.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.165+0000 7f636bfff700 1 -- 172.21.15.97:0/1458716259 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370011d20 con 0x7f637415c520 2023-07-06T03:55:45.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.165+0000 7f637b5ff700 1 -- 172.21.15.97:0/1458716259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f637415c520 msgr2=0x7f63740cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.165+0000 7f637b5ff700 1 --2- 172.21.15.97:0/1458716259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f637415c520 0x7f63740cd390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6370009840 tx=0x7f6370009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:45.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.165+0000 7f637b5ff700 1 -- 172.21.15.97:0/1458716259 shutdown_connections 2023-07-06T03:55:45.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.165+0000 7f637b5ff700 1 --2- 172.21.15.97:0/1458716259 >> v2:172.21.15.146:3300/0 conn(0x7f6374160d00 0x7f63740b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.165+0000 7f637b5ff700 1 --2- 172.21.15.97:0/1458716259 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f637415c520 0x7f63740cd390 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.165+0000 7f637b5ff700 1 -- 172.21.15.97:0/1458716259 >> 172.21.15.97:0/1458716259 conn(0x7f6374071790 msgr2=0x7f6374071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:45.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.165+0000 7f637b5ff700 1 -- 172.21.15.97:0/1458716259 shutdown_connections 2023-07-06T03:55:45.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.165+0000 7f637b5ff700 1 -- 172.21.15.97:0/1458716259 wait complete. 2023-07-06T03:55:45.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.167+0000 7f637b5ff700 1 Processor -- start 2023-07-06T03:55:45.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.167+0000 7f637b5ff700 1 -- start start 2023-07-06T03:55:45.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.168+0000 7f637b5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f637415c520 0x7f6374150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:45.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.168+0000 7f637b5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374160180 0x7f63740cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:45.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.168+0000 7f637b5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6374160d00 0x7f63740c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:45.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.168+0000 7f637b5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63740ca860 con 0x7f637415c520 2023-07-06T03:55:45.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.168+0000 7f637b5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63740cab40 con 0x7f6374160180 2023-07-06T03:55:45.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.168+0000 7f637b5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63740cae20 con 0x7f6374160d00 2023-07-06T03:55:45.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.168+0000 7f637939b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f637415c520 0x7f6374150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:45.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.168+0000 7f637939b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f637415c520 0x7f6374150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37286/0 (socket says 172.21.15.97:37286) 2023-07-06T03:55:45.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.169+0000 7f637939b700 1 -- 172.21.15.97:0/776568523 learned_addr learned my addr 172.21.15.97:0/776568523 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:45.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.169+0000 7f6379b9c700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6374160d00 0x7f63740c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:45.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.169+0000 7f6378b9a700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374160180 0x7f63740cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:45.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.169+0000 7f637939b700 1 -- 172.21.15.97:0/776568523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6374160d00 msgr2=0x7f63740c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.169+0000 7f637939b700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6374160d00 0x7f63740c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.169+0000 7f637939b700 1 -- 172.21.15.97:0/776568523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374160180 msgr2=0x7f63740cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.169+0000 7f637939b700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374160180 0x7f63740cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.169+0000 7f637939b700 1 -- 172.21.15.97:0/776568523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63700094a0 con 0x7f637415c520 2023-07-06T03:55:45.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.169+0000 7f6378b9a700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374160180 0x7f63740cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:45.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.169+0000 7f637939b700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f637415c520 0x7f6374150270 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f6370009fd0 tx=0x7f6370004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:45.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.169+0000 7f6369ffb700 1 -- 172.21.15.97:0/776568523 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63700036a0 con 0x7f637415c520 2023-07-06T03:55:45.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.170+0000 7f6369ffb700 1 -- 172.21.15.97:0/776568523 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637001e070 con 0x7f637415c520 2023-07-06T03:55:45.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.170+0000 7f6369ffb700 1 -- 172.21.15.97:0/776568523 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370003a60 con 0x7f637415c520 2023-07-06T03:55:45.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.170+0000 7f637b5ff700 1 -- 172.21.15.97:0/776568523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63740cb100 con 0x7f637415c520 2023-07-06T03:55:45.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.170+0000 7f637b5ff700 1 -- 172.21.15.97:0/776568523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6374150a70 con 0x7f637415c520 2023-07-06T03:55:45.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.171+0000 7f637b5ff700 1 -- 172.21.15.97:0/776568523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637404e340 con 0x7f637415c520 2023-07-06T03:55:45.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.172+0000 7f6369ffb700 1 -- 172.21.15.97:0/776568523 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6370003bc0 con 0x7f637415c520 2023-07-06T03:55:45.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.172+0000 7f6369ffb700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f635806ace0 0x7f635806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:45.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.173+0000 7f6369ffb700 1 -- 172.21.15.97:0/776568523 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f637008a2e0 con 0x7f637415c520 2023-07-06T03:55:45.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.173+0000 7f6378b9a700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f635806ace0 0x7f635806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:45.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.173+0000 7f6378b9a700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f635806ace0 0x7f635806d190 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f63640011e0 tx=0x7f6364009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:45.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.177+0000 7f6369ffb700 1 -- 172.21.15.97:0/776568523 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6370055420 con 0x7f637415c520 2023-07-06T03:55:45.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.376+0000 7f637b5ff700 1 -- 172.21.15.97:0/776568523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6374150e00 con 0x7f637415c520 2023-07-06T03:55:45.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.377+0000 7f6369ffb700 1 -- 172.21.15.97:0/776568523 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18319 (secure 0 0 0) 0x7f6370058a40 con 0x7f637415c520 2023-07-06T03:55:45.379 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:45.380 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":124,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:55:16.327257+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:55:12.989988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-06T03:55:13.782970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:55:45.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 -- 172.21.15.97:0/776568523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f635806ace0 msgr2=0x7f635806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f635806ace0 0x7f635806d190 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f63640011e0 tx=0x7f6364009040 comp rx=0 tx=0).stop 2023-07-06T03:55:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 -- 172.21.15.97:0/776568523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f637415c520 msgr2=0x7f6374150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f637415c520 0x7f6374150270 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f6370009fd0 tx=0x7f6370004750 comp rx=0 tx=0).stop 2023-07-06T03:55:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 -- 172.21.15.97:0/776568523 shutdown_connections 2023-07-06T03:55:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f635806ace0 0x7f635806d190 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6374160d00 0x7f63740c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f637415c520 0x7f6374150270 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 --2- 172.21.15.97:0/776568523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374160180 0x7f63740cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 -- 172.21.15.97:0/776568523 >> 172.21.15.97:0/776568523 conn(0x7f6374071790 msgr2=0x7f63741605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 -- 172.21.15.97:0/776568523 shutdown_connections 2023-07-06T03:55:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.380+0000 7f637b5ff700 1 -- 172.21.15.97:0/776568523 wait complete. 2023-07-06T03:55:45.400 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:55:12.989988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:55:13.782970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:55:45.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:55:45.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.596+0000 7fd91c5ff700 1 Processor -- start 2023-07-06T03:55:45.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.597+0000 7fd91c5ff700 1 -- start start 2023-07-06T03:55:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.598+0000 7fd91c5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9140d2210 0x7fd9140b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.598+0000 7fd91c5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9140cff90 con 0x7fd91406b360 2023-07-06T03:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.598+0000 7fd91c5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9140b5650 con 0x7fd91415c610 2023-07-06T03:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.598+0000 7fd91c5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140d0780 con 0x7fd9140d2210 2023-07-06T03:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.598+0000 7fd91ab9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9140d2210 0x7fd9140b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.598+0000 7fd91ab9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9140d2210 0x7fd9140b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37288/0 (socket says 172.21.15.97:37288) 2023-07-06T03:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.598+0000 7fd919b9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd91406b360 0x7fd9140d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43376/0 (socket says 172.21.15.97:43376) 2023-07-06T03:55:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.598+0000 7fd91ab9c700 1 -- 172.21.15.97:0/1819759675 learned_addr learned my addr 172.21.15.97:0/1819759675 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.598+0000 7fd91ab9c700 1 -- 172.21.15.97:0/1819759675 >> v1:172.21.15.146:6789/0 conn(0x7fd91415c610 legacy=0x7fd9140b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.598+0000 7fd91ab9c700 1 -- 172.21.15.97:0/1819759675 >> v1:172.21.15.97:6789/0 conn(0x7fd91406b360 legacy=0x7fd9140d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.598+0000 7fd91ab9c700 1 -- 172.21.15.97:0/1819759675 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9140b5400 con 0x7fd9140d2210 2023-07-06T03:55:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.599+0000 7fd91ab9c700 1 --2- 172.21.15.97:0/1819759675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9140d2210 0x7fd9140b4ec0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009e50 tx=0x7fd90c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc16fa28a0bc4e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.599+0000 7fd919399700 1 -- 172.21.15.97:0/1819759675 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00e070 con 0x7fd9140d2210 2023-07-06T03:55:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.599+0000 7fd919399700 1 -- 172.21.15.97:0/1819759675 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c00b9c0 con 0x7fd9140d2210 2023-07-06T03:55:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.599+0000 7fd919399700 1 -- 172.21.15.97:0/1819759675 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00bd70 con 0x7fd9140d2210 2023-07-06T03:55:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.599+0000 7fd91c5ff700 1 -- 172.21.15.97:0/1819759675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9140d2210 msgr2=0x7fd9140b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.599+0000 7fd91c5ff700 1 --2- 172.21.15.97:0/1819759675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9140d2210 0x7fd9140b4ec0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009e50 tx=0x7fd90c00b330 comp rx=0 tx=0).stop 2023-07-06T03:55:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.600+0000 7fd91c5ff700 1 -- 172.21.15.97:0/1819759675 shutdown_connections 2023-07-06T03:55:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.600+0000 7fd91c5ff700 1 --2- 172.21.15.97:0/1819759675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9140d2210 0x7fd9140b4ec0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.600+0000 7fd91c5ff700 1 -- 172.21.15.97:0/1819759675 >> 172.21.15.97:0/1819759675 conn(0x7fd91406a6c0 msgr2=0x7fd91406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.600+0000 7fd91c5ff700 1 -- 172.21.15.97:0/1819759675 shutdown_connections 2023-07-06T03:55:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.600+0000 7fd91c5ff700 1 -- 172.21.15.97:0/1819759675 wait complete. 2023-07-06T03:55:45.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.601+0000 7fd91c5ff700 1 Processor -- start 2023-07-06T03:55:45.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.601+0000 7fd91c5ff700 1 -- start start 2023-07-06T03:55:45.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91c5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91406b360 0x7fd9141f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:45.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91c5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9140d2210 0x7fd91414d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:45.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91c5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91415c610 0x7fd91415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:45.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91c5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141f5ca0 con 0x7fd91406b360 2023-07-06T03:55:45.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91c5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91415f580 con 0x7fd9140d2210 2023-07-06T03:55:45.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91c5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91415f860 con 0x7fd91415c610 2023-07-06T03:55:45.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91a39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91406b360 0x7fd9141f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:45.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91a39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91406b360 0x7fd9141f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37300/0 (socket says 172.21.15.97:37300) 2023-07-06T03:55:45.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91a39b700 1 -- 172.21.15.97:0/2125688258 learned_addr learned my addr 172.21.15.97:0/2125688258 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:45.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd919b9a700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9140d2210 0x7fd91414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:45.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91ab9c700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91415c610 0x7fd91415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:45.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91a39b700 1 -- 172.21.15.97:0/2125688258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91415c610 msgr2=0x7fd91415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91a39b700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91415c610 0x7fd91415eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91a39b700 1 -- 172.21.15.97:0/2125688258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9140d2210 msgr2=0x7fd91414d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91a39b700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9140d2210 0x7fd91414d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.603+0000 7fd91a39b700 1 -- 172.21.15.97:0/2125688258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c009b30 con 0x7fd91406b360 2023-07-06T03:55:45.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.604+0000 7fd91a39b700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91406b360 0x7fd9141f60c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fd91000c4e0 tx=0x7fd91000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:45.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.604+0000 7fd90affd700 1 -- 172.21.15.97:0/2125688258 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100052d0 con 0x7fd91406b360 2023-07-06T03:55:45.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.604+0000 7fd90affd700 1 -- 172.21.15.97:0/2125688258 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd910005430 con 0x7fd91406b360 2023-07-06T03:55:45.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.604+0000 7fd91c5ff700 1 -- 172.21.15.97:0/2125688258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9141f55e0 con 0x7fd91406b360 2023-07-06T03:55:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.604+0000 7fd90affd700 1 -- 172.21.15.97:0/2125688258 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100038c0 con 0x7fd91406b360 2023-07-06T03:55:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.604+0000 7fd91c5ff700 1 -- 172.21.15.97:0/2125688258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91414d580 con 0x7fd91406b360 2023-07-06T03:55:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.605+0000 7fd91c5ff700 1 -- 172.21.15.97:0/2125688258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91404e340 con 0x7fd91406b360 2023-07-06T03:55:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.606+0000 7fd90affd700 1 -- 172.21.15.97:0/2125688258 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd910003a20 con 0x7fd91406b360 2023-07-06T03:55:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.607+0000 7fd90affd700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8f806ace0 0x7fd8f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.607+0000 7fd90affd700 1 -- 172.21.15.97:0/2125688258 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd910088830 con 0x7fd91406b360 2023-07-06T03:55:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.607+0000 7fd919b9a700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8f806ace0 0x7fd8f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.608+0000 7fd919b9a700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8f806ace0 0x7fd8f806d190 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd9140b7da0 tx=0x7fd904007ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:45.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.611+0000 7fd90affd700 1 -- 172.21.15.97:0/2125688258 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd9100538d0 con 0x7fd91406b360 2023-07-06T03:55:45.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.802+0000 7fd91c5ff700 1 -- 172.21.15.97:0/2125688258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9140b7af0 con 0x7fd91406b360 2023-07-06T03:55:45.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.802+0000 7fd90affd700 1 -- 172.21.15.97:0/2125688258 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd910056ef0 con 0x7fd91406b360 2023-07-06T03:55:45.804 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:55:45.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 -- 172.21.15.97:0/2125688258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8f806ace0 msgr2=0x7fd8f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8f806ace0 0x7fd8f806d190 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd9140b7da0 tx=0x7fd904007ed0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 -- 172.21.15.97:0/2125688258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91406b360 msgr2=0x7fd9141f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:45.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91406b360 0x7fd9141f60c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fd91000c4e0 tx=0x7fd91000c7f0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 -- 172.21.15.97:0/2125688258 shutdown_connections 2023-07-06T03:55:45.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd8f806ace0 0x7fd8f806d190 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd91415c610 0x7fd91415eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd91406b360 0x7fd9141f60c0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 --2- 172.21.15.97:0/2125688258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9140d2210 0x7fd91414d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:45.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 -- 172.21.15.97:0/2125688258 >> 172.21.15.97:0/2125688258 conn(0x7fd91406a6c0 msgr2=0x7fd9140d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:45.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 -- 172.21.15.97:0/2125688258 shutdown_connections 2023-07-06T03:55:45.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:45.805+0000 7fd91c5ff700 1 -- 172.21.15.97:0/2125688258 wait complete. 2023-07-06T03:55:45.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:55:46.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.022+0000 7f8b415ff700 1 Processor -- start 2023-07-06T03:55:46.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.023+0000 7f8b415ff700 1 -- start start 2023-07-06T03:55:46.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.024+0000 7f8b415ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8b3c15c1c0 0x7f8b3c0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.024+0000 7f8b415ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3c0719c0 0x7f8b3c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.024+0000 7f8b415ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3c15da10 con 0x7f8b3c15c590 2023-07-06T03:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.024+0000 7f8b415ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c0ccc30 con 0x7f8b3c15c1c0 2023-07-06T03:55:46.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.024+0000 7f8b415ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0ccf10 con 0x7f8b3c0719c0 2023-07-06T03:55:46.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.024+0000 7f8b3a7fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8b3c15c590 0x7f8b3c15d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43384/0 (socket says 172.21.15.97:43384) 2023-07-06T03:55:46.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.024+0000 7f8b3a7fc700 1 -- 172.21.15.97:0/3793096793 learned_addr learned my addr 172.21.15.97:0/3793096793 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:46.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.024+0000 7f8b3affd700 1 --2- 172.21.15.97:0/3793096793 >> v2:172.21.15.146:3300/0 conn(0x7f8b3c15c1c0 0x7f8b3c0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.024+0000 7f8b3b7fe700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3c0719c0 0x7f8b3c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802053596 0 0) 0x7f8b30001110 con 0x7f8b3c15c590 2023-07-06T03:55:46.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b240035b0 con 0x7f8b3c15c590 2023-07-06T03:55:46.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4246059911 0 0) 0x7f8b240035b0 con 0x7f8b3c15c590 2023-07-06T03:55:46.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3c0719c0 msgr2=0x7f8b3c06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:46.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3c0719c0 0x7f8b3c06a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 >> v2:172.21.15.146:3300/0 conn(0x7f8b3c15c1c0 msgr2=0x7f8b3c0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:46.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 --2- 172.21.15.97:0/3793096793 >> v2:172.21.15.146:3300/0 conn(0x7f8b3c15c1c0 0x7f8b3c0cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c0cd1f0 con 0x7f8b3c15c590 2023-07-06T03:55:46.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f8b30001800 con 0x7f8b3c15c590 2023-07-06T03:55:46.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 >> v1:172.21.15.97:6789/0 conn(0x7f8b3c15c590 legacy=0x7f8b3c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b24003a00 0x7f8b24003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3c0719c0 0x7f8b24004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b24004d90 0x7f8b24005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0ccc30 con 0x7f8b24003a00 2023-07-06T03:55:46.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0ccf10 con 0x7f8b24004d90 2023-07-06T03:55:46.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2400ba10 con 0x7f8b3c0719c0 2023-07-06T03:55:46.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b3affd700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b24003a00 0x7f8b24003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b3b7fe700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b24004d90 0x7f8b24005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.025+0000 7f8b3a7fc700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3c0719c0 0x7f8b24004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.026+0000 7f8b3affd700 1 -- 172.21.15.97:0/3793096793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3c0719c0 msgr2=0x7f8b24004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:46.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.026+0000 7f8b3affd700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b3c0719c0 0x7f8b24004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.026+0000 7f8b3affd700 1 -- 172.21.15.97:0/3793096793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b24004d90 msgr2=0x7f8b24005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:46.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.026+0000 7f8b3affd700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b24004d90 0x7f8b24005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.026+0000 7f8b3affd700 1 -- 172.21.15.97:0/3793096793 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b240035b0 con 0x7f8b24003a00 2023-07-06T03:55:46.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.026+0000 7f8b3affd700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b24003a00 0x7f8b24003e10 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009b60 tx=0x7f8b24005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcb8fa7d35b2ec9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:46.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.026+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00e660 con 0x7f8b24003a00 2023-07-06T03:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.026+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c00e7c0 con 0x7f8b24003a00 2023-07-06T03:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.026+0000 7f8b39ffb700 1 -- 172.21.15.97:0/3793096793 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00eb30 con 0x7f8b24003a00 2023-07-06T03:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.027+0000 7f8b415ff700 1 -- 172.21.15.97:0/3793096793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b24003a00 msgr2=0x7f8b24003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.027+0000 7f8b415ff700 1 --2- 172.21.15.97:0/3793096793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b24003a00 0x7f8b24003e10 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009b60 tx=0x7f8b24005880 comp rx=0 tx=0).stop 2023-07-06T03:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.027+0000 7f8b3b7fe700 1 -- 172.21.15.97:0/3793096793 reap_dead start 2023-07-06T03:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.027+0000 7f8b415ff700 1 -- 172.21.15.97:0/3793096793 shutdown_connections 2023-07-06T03:55:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.027+0000 7f8b415ff700 1 -- 172.21.15.97:0/3793096793 >> 172.21.15.97:0/3793096793 conn(0x7f8b3c0b1650 msgr2=0x7f8b3c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:46.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.027+0000 7f8b415ff700 1 -- 172.21.15.97:0/3793096793 shutdown_connections 2023-07-06T03:55:46.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.027+0000 7f8b415ff700 1 -- 172.21.15.97:0/3793096793 wait complete. 2023-07-06T03:55:46.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.029+0000 7f8b415ff700 1 Processor -- start 2023-07-06T03:55:46.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.029+0000 7f8b415ff700 1 -- start start 2023-07-06T03:55:46.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.030+0000 7f8b415ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b24003a00 0x7f8b3c0b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.030+0000 7f8b415ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3c156e40 0x7f8b3c156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.030+0000 7f8b415ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b3c0b3460 0x7f8b3c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.030+0000 7f8b415ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0b3040 con 0x7f8b3c156e40 2023-07-06T03:55:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.030+0000 7f8b415ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0b29f0 con 0x7f8b3c0b3460 2023-07-06T03:55:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.030+0000 7f8b415ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c06f230 con 0x7f8b24003a00 2023-07-06T03:55:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.030+0000 7f8b3a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3c156e40 0x7f8b3c156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.030+0000 7f8b3a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3c156e40 0x7f8b3c156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37328/0 (socket says 172.21.15.97:37328) 2023-07-06T03:55:46.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.030+0000 7f8b3a7fc700 1 -- 172.21.15.97:0/793046315 learned_addr learned my addr 172.21.15.97:0/793046315 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:46.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.030+0000 7f8b3affd700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b24003a00 0x7f8b3c0b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.030+0000 7f8b3b7fe700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b3c0b3460 0x7f8b3c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.031+0000 7f8b3a7fc700 1 -- 172.21.15.97:0/793046315 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b24003a00 msgr2=0x7f8b3c0b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.031+0000 7f8b3a7fc700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b24003a00 0x7f8b3c0b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.031+0000 7f8b3a7fc700 1 -- 172.21.15.97:0/793046315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b3c0b3460 msgr2=0x7f8b3c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.031+0000 7f8b3a7fc700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b3c0b3460 0x7f8b3c0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.031+0000 7f8b3a7fc700 1 -- 172.21.15.97:0/793046315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c0b2c60 con 0x7f8b3c156e40 2023-07-06T03:55:46.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.031+0000 7f8b3a7fc700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3c156e40 0x7f8b3c156590 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000e7f0 tx=0x7f8b3000eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:46.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.031+0000 7f8b237fe700 1 -- 172.21.15.97:0/793046315 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30012590 con 0x7f8b3c156e40 2023-07-06T03:55:46.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.031+0000 7f8b237fe700 1 -- 172.21.15.97:0/793046315 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b300126f0 con 0x7f8b3c156e40 2023-07-06T03:55:46.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.031+0000 7f8b415ff700 1 -- 172.21.15.97:0/793046315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3c06ef00 con 0x7f8b3c156e40 2023-07-06T03:55:46.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.031+0000 7f8b237fe700 1 -- 172.21.15.97:0/793046315 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30012aa0 con 0x7f8b3c156e40 2023-07-06T03:55:46.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.031+0000 7f8b415ff700 1 -- 172.21.15.97:0/793046315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c156ad0 con 0x7f8b3c156e40 2023-07-06T03:55:46.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.032+0000 7f8b415ff700 1 -- 172.21.15.97:0/793046315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3c0b3180 con 0x7f8b3c156e40 2023-07-06T03:55:46.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.033+0000 7f8b237fe700 1 -- 172.21.15.97:0/793046315 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8b30012c00 con 0x7f8b3c156e40 2023-07-06T03:55:46.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.034+0000 7f8b237fe700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b2806ad10 0x7f8b2806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.034+0000 7f8b237fe700 1 -- 172.21.15.97:0/793046315 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b30089060 con 0x7f8b3c156e40 2023-07-06T03:55:46.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.034+0000 7f8b3affd700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b2806ad10 0x7f8b2806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.035+0000 7f8b3affd700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b2806ad10 0x7f8b2806d1c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0090b0 tx=0x7f8b2c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:46.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.038+0000 7f8b237fe700 1 -- 172.21.15.97:0/793046315 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8b300bb310 con 0x7f8b3c156e40 2023-07-06T03:55:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.233+0000 7f8b415ff700 1 -- 172.21.15.97:0/793046315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8b3c15da10 con 0x7f8b3c156e40 2023-07-06T03:55:46.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.234+0000 7f8b237fe700 1 -- 172.21.15.97:0/793046315 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v124) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8b3c15da10 con 0x7f8b3c156e40 2023-07-06T03:55:46.236 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:55:46.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.236+0000 7f8b415ff700 1 -- 172.21.15.97:0/793046315 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b2806ad10 msgr2=0x7f8b2806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.236+0000 7f8b415ff700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b2806ad10 0x7f8b2806d1c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0090b0 tx=0x7f8b2c008030 comp rx=0 tx=0).stop 2023-07-06T03:55:46.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.236+0000 7f8b415ff700 1 -- 172.21.15.97:0/793046315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3c156e40 msgr2=0x7f8b3c156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.236+0000 7f8b415ff700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3c156e40 0x7f8b3c156590 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000e7f0 tx=0x7f8b3000eb00 comp rx=0 tx=0).stop 2023-07-06T03:55:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.236+0000 7f8b415ff700 1 -- 172.21.15.97:0/793046315 shutdown_connections 2023-07-06T03:55:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.236+0000 7f8b415ff700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b2806ad10 0x7f8b2806d1c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.236+0000 7f8b415ff700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b3c0b3460 0x7f8b3c0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.236+0000 7f8b415ff700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b24003a00 0x7f8b3c0b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.236+0000 7f8b415ff700 1 --2- 172.21.15.97:0/793046315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b3c156e40 0x7f8b3c156590 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.236+0000 7f8b415ff700 1 -- 172.21.15.97:0/793046315 >> 172.21.15.97:0/793046315 conn(0x7f8b3c0b1650 msgr2=0x7f8b3c0c4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.237+0000 7f8b415ff700 1 -- 172.21.15.97:0/793046315 shutdown_connections 2023-07-06T03:55:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.237+0000 7f8b415ff700 1 -- 172.21.15.97:0/793046315 wait complete. 2023-07-06T03:55:46.256 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:55:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.453+0000 7f8d74dff700 1 Processor -- start 2023-07-06T03:55:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.453+0000 7f8d74dff700 1 -- start start 2023-07-06T03:55:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.455+0000 7f8d74dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8d70159070 0x7f8d700c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.455+0000 7f8d74dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d700c3cd0 0x7f8d700cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.455+0000 7f8d74dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700ca260 con 0x7f8d70159440 2023-07-06T03:55:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.455+0000 7f8d74dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d700c99c0 con 0x7f8d70159070 2023-07-06T03:55:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.455+0000 7f8d74dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700c9ca0 con 0x7f8d700c3cd0 2023-07-06T03:55:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.455+0000 7f8d67fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d700c3cd0 0x7f8d700cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.455+0000 7f8d67fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d700c3cd0 0x7f8d700cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:44594/0 (socket says 172.21.15.97:44594) 2023-07-06T03:55:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.455+0000 7f8d6e59c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8d70159070 0x7f8d700c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.455+0000 7f8d67fff700 1 -- 172.21.15.97:0/2367972084 learned_addr learned my addr 172.21.15.97:0/2367972084 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.455+0000 7f8d6ed9d700 1 --1- 172.21.15.97:0/2367972084 >> v1:172.21.15.146:6789/0 conn(0x7f8d70159440 0x7f8d700b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:40190/0 (socket says 172.21.15.97:40190) 2023-07-06T03:55:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.456+0000 7f8d6e59c700 1 -- 172.21.15.97:0/2367972084 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d700c3cd0 msgr2=0x7f8d700cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.456+0000 7f8d6e59c700 1 --2- 172.21.15.97:0/2367972084 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d700c3cd0 0x7f8d700cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.456+0000 7f8d6e59c700 1 -- 172.21.15.97:0/2367972084 >> v1:172.21.15.146:6789/0 conn(0x7f8d70159440 legacy=0x7f8d700b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.456+0000 7f8d6e59c700 1 -- 172.21.15.97:0/2367972084 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700c9f80 con 0x7f8d70159070 2023-07-06T03:55:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.456+0000 7f8d6e59c700 1 --2- 172.21.15.97:0/2367972084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d70159070 0x7f8d700c8f40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009840 tx=0x7f8d60009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c0d6a9e9971a465 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.456+0000 7f8d6dd9b700 1 -- 172.21.15.97:0/2367972084 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60011830 con 0x7f8d70159070 2023-07-06T03:55:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.456+0000 7f8d6dd9b700 1 -- 172.21.15.97:0/2367972084 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d60011990 con 0x7f8d70159070 2023-07-06T03:55:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.456+0000 7f8d6dd9b700 1 -- 172.21.15.97:0/2367972084 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60011d20 con 0x7f8d70159070 2023-07-06T03:55:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.457+0000 7f8d74dff700 1 -- 172.21.15.97:0/2367972084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d70159070 msgr2=0x7f8d700c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.457+0000 7f8d74dff700 1 --2- 172.21.15.97:0/2367972084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d70159070 0x7f8d700c8f40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009840 tx=0x7f8d60009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.457+0000 7f8d74dff700 1 -- 172.21.15.97:0/2367972084 shutdown_connections 2023-07-06T03:55:46.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.457+0000 7f8d74dff700 1 --2- 172.21.15.97:0/2367972084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d70159070 0x7f8d700c8f40 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.457+0000 7f8d74dff700 1 --2- 172.21.15.97:0/2367972084 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d700c3cd0 0x7f8d700cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.457+0000 7f8d74dff700 1 -- 172.21.15.97:0/2367972084 >> 172.21.15.97:0/2367972084 conn(0x7f8d7006a060 msgr2=0x7f8d7006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.457+0000 7f8d74dff700 1 -- 172.21.15.97:0/2367972084 shutdown_connections 2023-07-06T03:55:46.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.457+0000 7f8d74dff700 1 -- 172.21.15.97:0/2367972084 wait complete. 2023-07-06T03:55:46.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.459+0000 7f8d74dff700 1 Processor -- start 2023-07-06T03:55:46.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.459+0000 7f8d74dff700 1 -- start start 2023-07-06T03:55:46.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d74dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d700c3cd0 0x7f8d700aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d74dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d70159070 0x7f8d7014d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d74dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d70159440 0x7f8d7014ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d74dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700ae540 con 0x7f8d700c3cd0 2023-07-06T03:55:46.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d74dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700aeac0 con 0x7f8d70159440 2023-07-06T03:55:46.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d74dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7014e890 con 0x7f8d70159070 2023-07-06T03:55:46.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d6e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d700c3cd0 0x7f8d700aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d6e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d700c3cd0 0x7f8d700aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37334/0 (socket says 172.21.15.97:37334) 2023-07-06T03:55:46.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d6e59c700 1 -- 172.21.15.97:0/2604571938 learned_addr learned my addr 172.21.15.97:0/2604571938 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:46.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d6ed9d700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d70159440 0x7f8d7014ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d67fff700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d70159070 0x7f8d7014d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d6e59c700 1 -- 172.21.15.97:0/2604571938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d70159070 msgr2=0x7f8d7014d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.460+0000 7f8d6e59c700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d70159070 0x7f8d7014d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.461+0000 7f8d6e59c700 1 -- 172.21.15.97:0/2604571938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d70159440 msgr2=0x7f8d7014ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.461+0000 7f8d6e59c700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d70159440 0x7f8d7014ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.461+0000 7f8d6e59c700 1 -- 172.21.15.97:0/2604571938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d600094a0 con 0x7f8d700c3cd0 2023-07-06T03:55:46.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.461+0000 7f8d67fff700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d70159070 0x7f8d7014d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:46.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.461+0000 7f8d6ed9d700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d70159440 0x7f8d7014ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:46.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.461+0000 7f8d6e59c700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d700c3cd0 0x7f8d700aef50 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009fd0 tx=0x7f8d60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:46.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.461+0000 7f8d66ffd700 1 -- 172.21.15.97:0/2604571938 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600036a0 con 0x7f8d700c3cd0 2023-07-06T03:55:46.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.461+0000 7f8d66ffd700 1 -- 172.21.15.97:0/2604571938 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6001f070 con 0x7f8d700c3cd0 2023-07-06T03:55:46.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.461+0000 7f8d74dff700 1 -- 172.21.15.97:0/2604571938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7014eb70 con 0x7f8d700c3cd0 2023-07-06T03:55:46.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.461+0000 7f8d66ffd700 1 -- 172.21.15.97:0/2604571938 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60003a60 con 0x7f8d700c3cd0 2023-07-06T03:55:46.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.461+0000 7f8d74dff700 1 -- 172.21.15.97:0/2604571938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d701509b0 con 0x7f8d700c3cd0 2023-07-06T03:55:46.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.462+0000 7f8d74dff700 1 -- 172.21.15.97:0/2604571938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d700aeda0 con 0x7f8d700c3cd0 2023-07-06T03:55:46.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.463+0000 7f8d66ffd700 1 -- 172.21.15.97:0/2604571938 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d60003bc0 con 0x7f8d700c3cd0 2023-07-06T03:55:46.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.464+0000 7f8d66ffd700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 0x7f8d5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.464+0000 7f8d66ffd700 1 -- 172.21.15.97:0/2604571938 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d60021070 con 0x7f8d700c3cd0 2023-07-06T03:55:46.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.464+0000 7f8d67fff700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 0x7f8d5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.465+0000 7f8d67fff700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 0x7f8d5006d190 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8d701f1700 tx=0x7f8d5800b4d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:46.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.469+0000 7f8d66ffd700 1 -- 172.21.15.97:0/2604571938 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8d60055f30 con 0x7f8d700c3cd0 2023-07-06T03:55:46.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.662+0000 7f8d74dff700 1 -- 172.21.15.97:0/2604571938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d7014ed60 con 0x7f8d700c3cd0 2023-07-06T03:55:46.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.662+0000 7f8d66ffd700 1 -- 172.21.15.97:0/2604571938 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d60015020 con 0x7f8d700c3cd0 2023-07-06T03:55:46.664 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:55:46.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.664+0000 7f8d74dff700 1 -- 172.21.15.97:0/2604571938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 msgr2=0x7f8d5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.664+0000 7f8d74dff700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 0x7f8d5006d190 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8d701f1700 tx=0x7f8d5800b4d0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.664+0000 7f8d74dff700 1 -- 172.21.15.97:0/2604571938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d700c3cd0 msgr2=0x7f8d700aef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.664+0000 7f8d74dff700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d700c3cd0 0x7f8d700aef50 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009fd0 tx=0x7f8d60004750 comp rx=0 tx=0).stop 2023-07-06T03:55:46.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.665+0000 7f8d74dff700 1 -- 172.21.15.97:0/2604571938 shutdown_connections 2023-07-06T03:55:46.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.665+0000 7f8d74dff700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8d5006ace0 0x7f8d5006d190 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.665+0000 7f8d74dff700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d70159440 0x7f8d7014ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.665+0000 7f8d74dff700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d700c3cd0 0x7f8d700aef50 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.665+0000 7f8d74dff700 1 --2- 172.21.15.97:0/2604571938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8d70159070 0x7f8d7014d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.665+0000 7f8d74dff700 1 -- 172.21.15.97:0/2604571938 >> 172.21.15.97:0/2604571938 conn(0x7f8d7006a060 msgr2=0x7f8d70071020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:46.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.665+0000 7f8d74dff700 1 -- 172.21.15.97:0/2604571938 shutdown_connections 2023-07-06T03:55:46.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.665+0000 7f8d74dff700 1 -- 172.21.15.97:0/2604571938 wait complete. 2023-07-06T03:55:46.684 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:55:46.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.879+0000 7f86af3ff700 1 Processor -- start 2023-07-06T03:55:46.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.879+0000 7f86af3ff700 1 -- start start 2023-07-06T03:55:46.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.880+0000 7f86af3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86a815cae0 0x7f86a815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.880+0000 7f86af3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86a80cfbc0 0x7f86a815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.880+0000 7f86af3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a815de00 con 0x7f86a806b170 2023-07-06T03:55:46.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.880+0000 7f86af3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a80b25a0 con 0x7f86a815cae0 2023-07-06T03:55:46.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.880+0000 7f86af3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a80ceb20 con 0x7f86a80cfbc0 2023-07-06T03:55:46.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.881+0000 7f86ac99a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86a815cae0 0x7f86a815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.881+0000 7f86ac99a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86a815cae0 0x7f86a815d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37342/0 (socket says 172.21.15.97:37342) 2023-07-06T03:55:46.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.881+0000 7f86ac99a700 1 -- 172.21.15.97:0/2942224711 learned_addr learned my addr 172.21.15.97:0/2942224711 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:46.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.881+0000 7f86ad99c700 1 --2- 172.21.15.97:0/2942224711 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86a80cfbc0 0x7f86a815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.881+0000 7f86ac99a700 1 -- 172.21.15.97:0/2942224711 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86a80cfbc0 msgr2=0x7f86a815e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:46.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.881+0000 7f86ac99a700 1 --2- 172.21.15.97:0/2942224711 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86a80cfbc0 0x7f86a815e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.881+0000 7f86ac99a700 1 -- 172.21.15.97:0/2942224711 >> v1:172.21.15.146:6789/0 conn(0x7f86a806b170 legacy=0x7f86a806b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.881+0000 7f86ac99a700 1 -- 172.21.15.97:0/2942224711 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a80cee00 con 0x7f86a815cae0 2023-07-06T03:55:46.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.881+0000 7f86ac99a700 1 --2- 172.21.15.97:0/2942224711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86a815cae0 0x7f86a815d8c0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f86980098f0 tx=0x7f8698009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae77fb435444ec2a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:46.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.882+0000 7f869ffff700 1 -- 172.21.15.97:0/2942224711 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869800b8a0 con 0x7f86a815cae0 2023-07-06T03:55:46.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.882+0000 7f869ffff700 1 -- 172.21.15.97:0/2942224711 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869800ba00 con 0x7f86a815cae0 2023-07-06T03:55:46.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.882+0000 7f869ffff700 1 -- 172.21.15.97:0/2942224711 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869800bd90 con 0x7f86a815cae0 2023-07-06T03:55:46.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.882+0000 7f86af3ff700 1 -- 172.21.15.97:0/2942224711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86a815cae0 msgr2=0x7f86a815d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.882+0000 7f86af3ff700 1 --2- 172.21.15.97:0/2942224711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86a815cae0 0x7f86a815d8c0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f86980098f0 tx=0x7f8698009c00 comp rx=0 tx=0).stop 2023-07-06T03:55:46.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.882+0000 7f86af3ff700 1 -- 172.21.15.97:0/2942224711 shutdown_connections 2023-07-06T03:55:46.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.882+0000 7f86af3ff700 1 --2- 172.21.15.97:0/2942224711 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86a80cfbc0 0x7f86a815e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.882+0000 7f86af3ff700 1 --2- 172.21.15.97:0/2942224711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86a815cae0 0x7f86a815d8c0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.882+0000 7f86af3ff700 1 -- 172.21.15.97:0/2942224711 >> 172.21.15.97:0/2942224711 conn(0x7f86a8099f10 msgr2=0x7f86a80d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:46.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.882+0000 7f86af3ff700 1 -- 172.21.15.97:0/2942224711 shutdown_connections 2023-07-06T03:55:46.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.882+0000 7f86af3ff700 1 -- 172.21.15.97:0/2942224711 wait complete. 2023-07-06T03:55:46.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.885+0000 7f86af3ff700 1 Processor -- start 2023-07-06T03:55:46.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.885+0000 7f86af3ff700 1 -- start start 2023-07-06T03:55:46.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.886+0000 7f86af3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86a806b170 0x7f86a81577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.886+0000 7f86af3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86a80cfbc0 0x7f86a81e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.886+0000 7f86af3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86a815cae0 0x7f86a81e2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.886+0000 7f86af3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8159260 con 0x7f86a80cfbc0 2023-07-06T03:55:46.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.886+0000 7f86af3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8155e00 con 0x7f86a806b170 2023-07-06T03:55:46.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.886+0000 7f86af3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a81560e0 con 0x7f86a815cae0 2023-07-06T03:55:46.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.886+0000 7f86ac99a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86a80cfbc0 0x7f86a81e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.886+0000 7f86ac99a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86a80cfbc0 0x7f86a81e4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37354/0 (socket says 172.21.15.97:37354) 2023-07-06T03:55:46.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.886+0000 7f86ac99a700 1 -- 172.21.15.97:0/936531080 learned_addr learned my addr 172.21.15.97:0/936531080 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:46.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f86ad99c700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86a815cae0 0x7f86a81e2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f86ad19b700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86a806b170 0x7f86a81577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f86ac99a700 1 -- 172.21.15.97:0/936531080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86a815cae0 msgr2=0x7f86a81e2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f86ac99a700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86a815cae0 0x7f86a81e2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f86ac99a700 1 -- 172.21.15.97:0/936531080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86a806b170 msgr2=0x7f86a81577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:46.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f86ac99a700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86a806b170 0x7f86a81577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:46.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f86ac99a700 1 -- 172.21.15.97:0/936531080 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8698009550 con 0x7f86a80cfbc0 2023-07-06T03:55:46.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f86ac99a700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86a80cfbc0 0x7f86a81e4b90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f8698000c00 tx=0x7f8698004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:46.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f869dffb700 1 -- 172.21.15.97:0/936531080 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698004300 con 0x7f86a80cfbc0 2023-07-06T03:55:46.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f869dffb700 1 -- 172.21.15.97:0/936531080 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869801f070 con 0x7f86a80cfbc0 2023-07-06T03:55:46.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f86af3ff700 1 -- 172.21.15.97:0/936531080 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a81563c0 con 0x7f86a80cfbc0 2023-07-06T03:55:46.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.887+0000 7f869dffb700 1 -- 172.21.15.97:0/936531080 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980036a0 con 0x7f86a80cfbc0 2023-07-06T03:55:46.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.888+0000 7f86af3ff700 1 -- 172.21.15.97:0/936531080 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a8157d30 con 0x7f86a80cfbc0 2023-07-06T03:55:46.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.889+0000 7f86af3ff700 1 -- 172.21.15.97:0/936531080 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a81568e0 con 0x7f86a80cfbc0 2023-07-06T03:55:46.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.889+0000 7f869dffb700 1 -- 172.21.15.97:0/936531080 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8698021030 con 0x7f86a80cfbc0 2023-07-06T03:55:46.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.891+0000 7f869dffb700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868c06ace0 0x7f868c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:46.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.891+0000 7f869dffb700 1 -- 172.21.15.97:0/936531080 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f869808a660 con 0x7f86a80cfbc0 2023-07-06T03:55:46.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.891+0000 7f86ad19b700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868c06ace0 0x7f868c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:46.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.892+0000 7f86ad19b700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868c06ace0 0x7f868c06d190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f86a4000fb0 tx=0x7f86a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:46.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:46.895+0000 7f869dffb700 1 -- 172.21.15.97:0/936531080 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8698055850 con 0x7f86a80cfbc0 2023-07-06T03:55:47.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.096+0000 7f86af3ff700 1 -- 172.21.15.97:0/936531080 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86a804e340 con 0x7f86a80cfbc0 2023-07-06T03:55:47.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.096+0000 7f869dffb700 1 -- 172.21.15.97:0/936531080 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8698058e70 con 0x7f86a80cfbc0 2023-07-06T03:55:47.099 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:55:47.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 -- 172.21.15.97:0/936531080 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868c06ace0 msgr2=0x7f868c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868c06ace0 0x7f868c06d190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f86a4000fb0 tx=0x7f86a4009040 comp rx=0 tx=0).stop 2023-07-06T03:55:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 -- 172.21.15.97:0/936531080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86a80cfbc0 msgr2=0x7f86a81e4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86a80cfbc0 0x7f86a81e4b90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f8698000c00 tx=0x7f8698004750 comp rx=0 tx=0).stop 2023-07-06T03:55:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 -- 172.21.15.97:0/936531080 shutdown_connections 2023-07-06T03:55:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868c06ace0 0x7f868c06d190 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86a815cae0 0x7f86a81e2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86a806b170 0x7f86a81577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 --2- 172.21.15.97:0/936531080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86a80cfbc0 0x7f86a81e4b90 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 -- 172.21.15.97:0/936531080 >> 172.21.15.97:0/936531080 conn(0x7f86a8099f10 msgr2=0x7f86a80b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:47.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 -- 172.21.15.97:0/936531080 shutdown_connections 2023-07-06T03:55:47.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.099+0000 7f86af3ff700 1 -- 172.21.15.97:0/936531080 wait complete. 2023-07-06T03:55:47.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:55:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.311+0000 7f2df83ff700 1 Processor -- start 2023-07-06T03:55:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.311+0000 7f2df83ff700 1 -- start start 2023-07-06T03:55:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.312+0000 7f2df83ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2df00cd390 0x7f2df0160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df83ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df0160dd0 0x7f2df00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df83ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df00cdae0 con 0x7f2df015c520 2023-07-06T03:55:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df83ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df00b6640 con 0x7f2df00cd390 2023-07-06T03:55:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df83ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00b6920 con 0x7f2df0160dd0 2023-07-06T03:55:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df619b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2df015c520 0x7f2df0160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43394/0 (socket says 172.21.15.97:43394) 2023-07-06T03:55:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df619b700 1 -- 172.21.15.97:0/666097904 learned_addr learned my addr 172.21.15.97:0/666097904 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df699c700 1 --2- 172.21.15.97:0/666097904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df0160dd0 0x7f2df00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df599a700 1 --2- 172.21.15.97:0/666097904 >> v2:172.21.15.146:3300/0 conn(0x7f2df00cd390 0x7f2df0160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df5199700 1 -- 172.21.15.97:0/666097904 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127755589 0 0) 0x7f2df00cdae0 con 0x7f2df015c520 2023-07-06T03:55:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df5199700 1 -- 172.21.15.97:0/666097904 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ddc0035b0 con 0x7f2df015c520 2023-07-06T03:55:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df699c700 1 -- 172.21.15.97:0/666097904 >> v2:172.21.15.146:3300/0 conn(0x7f2df00cd390 msgr2=0x7f2df0160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df699c700 1 --2- 172.21.15.97:0/666097904 >> v2:172.21.15.146:3300/0 conn(0x7f2df00cd390 0x7f2df0160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.313+0000 7f2df699c700 1 -- 172.21.15.97:0/666097904 >> v1:172.21.15.97:6789/0 conn(0x7f2df015c520 legacy=0x7f2df0160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.314+0000 7f2df699c700 1 -- 172.21.15.97:0/666097904 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df00b6c00 con 0x7f2df0160dd0 2023-07-06T03:55:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.314+0000 7f2df699c700 1 --2- 172.21.15.97:0/666097904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df0160dd0 0x7f2df00b7400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009d90 tx=0x7f2de800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c8e35a366acbcd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.314+0000 7f2df5199700 1 -- 172.21.15.97:0/666097904 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800e070 con 0x7f2df0160dd0 2023-07-06T03:55:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.314+0000 7f2df5199700 1 -- 172.21.15.97:0/666097904 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de800bb20 con 0x7f2df0160dd0 2023-07-06T03:55:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.314+0000 7f2df5199700 1 -- 172.21.15.97:0/666097904 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8012430 con 0x7f2df0160dd0 2023-07-06T03:55:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.315+0000 7f2df83ff700 1 -- 172.21.15.97:0/666097904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df0160dd0 msgr2=0x7f2df00b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.315+0000 7f2df83ff700 1 --2- 172.21.15.97:0/666097904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df0160dd0 0x7f2df00b7400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009d90 tx=0x7f2de800b330 comp rx=0 tx=0).stop 2023-07-06T03:55:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.315+0000 7f2df83ff700 1 -- 172.21.15.97:0/666097904 shutdown_connections 2023-07-06T03:55:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.315+0000 7f2df83ff700 1 --2- 172.21.15.97:0/666097904 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df0160dd0 0x7f2df00b7400 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.315+0000 7f2df83ff700 1 --2- 172.21.15.97:0/666097904 >> v2:172.21.15.146:3300/0 conn(0x7f2df00cd390 0x7f2df0160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.315+0000 7f2df83ff700 1 -- 172.21.15.97:0/666097904 >> 172.21.15.97:0/666097904 conn(0x7f2df0071790 msgr2=0x7f2df0071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.315+0000 7f2df83ff700 1 -- 172.21.15.97:0/666097904 shutdown_connections 2023-07-06T03:55:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.315+0000 7f2df83ff700 1 -- 172.21.15.97:0/666097904 wait complete. 2023-07-06T03:55:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.317+0000 7f2df83ff700 1 Processor -- start 2023-07-06T03:55:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.317+0000 7f2df83ff700 1 -- start start 2023-07-06T03:55:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.318+0000 7f2df83ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2df00cd390 0x7f2df00c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df83ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df015c520 0x7f2df01f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df83ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df0160dd0 0x7f2df01f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df83ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00d3c20 con 0x7f2df015c520 2023-07-06T03:55:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df83ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df0159cd0 con 0x7f2df00cd390 2023-07-06T03:55:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df83ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00d3380 con 0x7f2df0160dd0 2023-07-06T03:55:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df599a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df015c520 0x7f2df01f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df599a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df015c520 0x7f2df01f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37358/0 (socket says 172.21.15.97:37358) 2023-07-06T03:55:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df599a700 1 -- 172.21.15.97:0/3933598160 learned_addr learned my addr 172.21.15.97:0/3933598160 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df619b700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2df00cd390 0x7f2df00c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df699c700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df0160dd0 0x7f2df01f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df599a700 1 -- 172.21.15.97:0/3933598160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df0160dd0 msgr2=0x7f2df01f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df599a700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df0160dd0 0x7f2df01f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df599a700 1 -- 172.21.15.97:0/3933598160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2df00cd390 msgr2=0x7f2df00c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df599a700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2df00cd390 0x7f2df00c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df599a700 1 -- 172.21.15.97:0/3933598160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de8009a70 con 0x7f2df015c520 2023-07-06T03:55:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df619b700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2df00cd390 0x7f2df00c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.319+0000 7f2df599a700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df015c520 0x7f2df01f55d0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f2de000e8a0 tx=0x7f2de000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.320+0000 7f2de6ffd700 1 -- 172.21.15.97:0/3933598160 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0009590 con 0x7f2df015c520 2023-07-06T03:55:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.320+0000 7f2de6ffd700 1 -- 172.21.15.97:0/3933598160 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de000eea0 con 0x7f2df015c520 2023-07-06T03:55:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.320+0000 7f2df83ff700 1 -- 172.21.15.97:0/3933598160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df00d3660 con 0x7f2df015c520 2023-07-06T03:55:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.320+0000 7f2de6ffd700 1 -- 172.21.15.97:0/3933598160 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de000ca50 con 0x7f2df015c520 2023-07-06T03:55:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.320+0000 7f2df83ff700 1 -- 172.21.15.97:0/3933598160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df00c8fa0 con 0x7f2df015c520 2023-07-06T03:55:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.321+0000 7f2df83ff700 1 -- 172.21.15.97:0/3933598160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df004e340 con 0x7f2df015c520 2023-07-06T03:55:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.322+0000 7f2de6ffd700 1 -- 172.21.15.97:0/3933598160 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2de000cbb0 con 0x7f2df015c520 2023-07-06T03:55:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.323+0000 7f2de6ffd700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2dd406ace0 0x7f2dd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.323+0000 7f2de6ffd700 1 -- 172.21.15.97:0/3933598160 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2de00893d0 con 0x7f2df015c520 2023-07-06T03:55:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.323+0000 7f2df619b700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2dd406ace0 0x7f2dd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.324+0000 7f2df619b700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2dd406ace0 0x7f2dd406d190 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0011e0 tx=0x7f2dec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.328+0000 7f2de6ffd700 1 -- 172.21.15.97:0/3933598160 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2de0057fa0 con 0x7f2df015c520 2023-07-06T03:55:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.566+0000 7f2df83ff700 1 -- 172.21.15.97:0/3933598160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2df00b4f40 con 0x7f2df015c520 2023-07-06T03:55:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.567+0000 7f2de6ffd700 1 -- 172.21.15.97:0/3933598160 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4679 (secure 0 0 0) 0x7f2de0057b30 con 0x7f2df015c520 2023-07-06T03:55:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:47.569 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3059444241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3059444241},{"type":"v1","addr":"172.21.15.146:6835","nonce":3059444241}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2624688851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2624688851},{"type":"v1","addr":"172.21.15.97:6837","nonce":2624688851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3938376776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3938376776},{"type":"v1","addr":"172.21.15.97:6835","nonce":3938376776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:55:14.806943+0000","modified":"2023-07-06T03:55:35.906574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9099":{"gid":9099,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":8,"addr":"172.21.15.146:6834/3269084599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3269084599},{"type":"v1","addr":"172.21.15.146:6834","nonce":3269084599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-06T03:55:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 -- 172.21.15.97:0/3933598160 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2dd406ace0 msgr2=0x7f2dd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2dd406ace0 0x7f2dd406d190 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0011e0 tx=0x7f2dec009040 comp rx=0 tx=0).stop 2023-07-06T03:55:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 -- 172.21.15.97:0/3933598160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df015c520 msgr2=0x7f2df01f55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df015c520 0x7f2df01f55d0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f2de000e8a0 tx=0x7f2de000ec60 comp rx=0 tx=0).stop 2023-07-06T03:55:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 -- 172.21.15.97:0/3933598160 shutdown_connections 2023-07-06T03:55:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2dd406ace0 0x7f2dd406d190 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2df0160dd0 0x7f2df01f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2df00cd390 0x7f2df00c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 --2- 172.21.15.97:0/3933598160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2df015c520 0x7f2df01f55d0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 -- 172.21.15.97:0/3933598160 >> 172.21.15.97:0/3933598160 conn(0x7f2df0071790 msgr2=0x7f2df00b8060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 -- 172.21.15.97:0/3933598160 shutdown_connections 2023-07-06T03:55:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.569+0000 7f2df83ff700 1 -- 172.21.15.97:0/3933598160 wait complete. 2023-07-06T03:55:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 94 2023-07-06T03:55:47.588 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:55:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.781+0000 7f514bdff700 1 Processor -- start 2023-07-06T03:55:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.781+0000 7f514bdff700 1 -- start start 2023-07-06T03:55:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.782+0000 7f514bdff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f51440b8410 0x7f5144167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.782+0000 7f514bdff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51440cdae0 con 0x7f514415c520 2023-07-06T03:55:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f514bdff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51440b6640 con 0x7f51440cd390 2023-07-06T03:55:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f514bdff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51440b6920 con 0x7f51440b8410 2023-07-06T03:55:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f5149b9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f514415c520 0x7f5144160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43400/0 (socket says 172.21.15.97:43400) 2023-07-06T03:55:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f5149b9b700 1 -- 172.21.15.97:0/3569923787 learned_addr learned my addr 172.21.15.97:0/3569923787 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f514a39c700 1 --2- 172.21.15.97:0/3569923787 >> v2:172.21.15.146:3300/0 conn(0x7f51440b8410 0x7f5144167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302137409 0 0) 0x7f51440cdae0 con 0x7f514415c520 2023-07-06T03:55:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51300035b0 con 0x7f514415c520 2023-07-06T03:55:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008259326 0 0) 0x7f51440b6640 con 0x7f51440cd390 2023-07-06T03:55:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51440cdae0 con 0x7f51440cd390 2023-07-06T03:55:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2597598299 0 0) 0x7f51300035b0 con 0x7f514415c520 2023-07-06T03:55:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 >> v2:172.21.15.146:3300/0 conn(0x7f51440b8410 msgr2=0x7f5144167510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f5148b99700 1 --2- 172.21.15.97:0/3569923787 >> v2:172.21.15.146:3300/0 conn(0x7f51440b8410 0x7f5144167510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.783+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 >> v1:172.21.15.146:6789/0 conn(0x7f51440cd390 legacy=0x7f5144160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440b6c00 con 0x7f514415c520 2023-07-06T03:55:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f5140001670 con 0x7f514415c520 2023-07-06T03:55:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 >> v1:172.21.15.97:6789/0 conn(0x7f514415c520 legacy=0x7f5144160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5148b99700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 0x7f5130003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5148b99700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5130004850 0x7f5130004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5148b99700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5130005200 0x7f513000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440b6920 con 0x7f5130003be0 2023-07-06T03:55:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513000c0e0 con 0x7f5130004850 2023-07-06T03:55:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513000c270 con 0x7f5130005200 2023-07-06T03:55:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5149b9b700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 0x7f5130003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f514a39c700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5130005200 0x7f513000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f514939a700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5130004850 0x7f5130004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5149b9b700 1 -- 172.21.15.97:0/3569923787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5130005200 msgr2=0x7f513000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5149b9b700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5130005200 0x7f513000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5149b9b700 1 -- 172.21.15.97:0/3569923787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5130004850 msgr2=0x7f5130004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5149b9b700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5130004850 0x7f5130004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f5149b9b700 1 -- 172.21.15.97:0/3569923787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300035b0 con 0x7f5130003be0 2023-07-06T03:55:47.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f514939a700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5130004850 0x7f5130004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f514a39c700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5130005200 0x7f513000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.784+0000 7f514a39c700 1 -- 172.21.15.97:0/3569923787 reap_dead start 2023-07-06T03:55:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.785+0000 7f5149b9b700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 0x7f5130003fd0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f5140001eb0 tx=0x7f5130005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd03fdc56e06dec9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.785+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514000cc70 con 0x7f5130003be0 2023-07-06T03:55:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.785+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514000cdf0 con 0x7f5130003be0 2023-07-06T03:55:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.785+0000 7f5148b99700 1 -- 172.21.15.97:0/3569923787 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140005ba0 con 0x7f5130003be0 2023-07-06T03:55:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.785+0000 7f514bdff700 1 -- 172.21.15.97:0/3569923787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 msgr2=0x7f5130003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.785+0000 7f514bdff700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 0x7f5130003fd0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f5140001eb0 tx=0x7f5130005880 comp rx=0 tx=0).stop 2023-07-06T03:55:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.785+0000 7f514bdff700 1 -- 172.21.15.97:0/3569923787 shutdown_connections 2023-07-06T03:55:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.785+0000 7f514bdff700 1 --2- 172.21.15.97:0/3569923787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 0x7f5130003fd0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.785+0000 7f514bdff700 1 -- 172.21.15.97:0/3569923787 >> 172.21.15.97:0/3569923787 conn(0x7f5144071790 msgr2=0x7f5144071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.785+0000 7f514bdff700 1 -- 172.21.15.97:0/3569923787 shutdown_connections 2023-07-06T03:55:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.785+0000 7f514bdff700 1 -- 172.21.15.97:0/3569923787 wait complete. 2023-07-06T03:55:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.788+0000 7f514bdff700 1 Processor -- start 2023-07-06T03:55:47.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.788+0000 7f514bdff700 1 -- start start 2023-07-06T03:55:47.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.789+0000 7f514bdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 0x7f5144156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.789+0000 7f514bdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440cbb10 0x7f51440ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.789+0000 7f514bdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51441565a0 0x7f514415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.789+0000 7f514bdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440cb680 con 0x7f5130003be0 2023-07-06T03:55:47.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.789+0000 7f514bdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441575b0 con 0x7f51441565a0 2023-07-06T03:55:47.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.789+0000 7f514bdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51440caa70 con 0x7f51440cbb10 2023-07-06T03:55:47.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.789+0000 7f5149b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 0x7f5144156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.789+0000 7f5149b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 0x7f5144156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37384/0 (socket says 172.21.15.97:37384) 2023-07-06T03:55:47.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.789+0000 7f5149b9b700 1 -- 172.21.15.97:0/3523827988 learned_addr learned my addr 172.21.15.97:0/3523827988 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:47.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.789+0000 7f514939a700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440cbb10 0x7f51440ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f514a39c700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51441565a0 0x7f514415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f5149b9b700 1 -- 172.21.15.97:0/3523827988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440cbb10 msgr2=0x7f51440ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f5149b9b700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440cbb10 0x7f51440ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f5149b9b700 1 -- 172.21.15.97:0/3523827988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51441565a0 msgr2=0x7f514415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:47.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f5149b9b700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51441565a0 0x7f514415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:47.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f5149b9b700 1 -- 172.21.15.97:0/3523827988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440cad50 con 0x7f5130003be0 2023-07-06T03:55:47.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f514a39c700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51441565a0 0x7f514415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:47.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f5149b9b700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 0x7f5144156dc0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f51300047a0 tx=0x7f5140001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f513a7fc700 1 -- 172.21.15.97:0/3523827988 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51400070d0 con 0x7f5130003be0 2023-07-06T03:55:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f513a7fc700 1 -- 172.21.15.97:0/3523827988 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5140007230 con 0x7f5130003be0 2023-07-06T03:55:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f514bdff700 1 -- 172.21.15.97:0/3523827988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51440cb680 con 0x7f5130003be0 2023-07-06T03:55:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f513a7fc700 1 -- 172.21.15.97:0/3523827988 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51400075c0 con 0x7f5130003be0 2023-07-06T03:55:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.790+0000 7f514bdff700 1 -- 172.21.15.97:0/3523827988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51440cb310 con 0x7f5130003be0 2023-07-06T03:55:47.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.792+0000 7f514bdff700 1 -- 172.21.15.97:0/3523827988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514404e340 con 0x7f5130003be0 2023-07-06T03:55:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.792+0000 7f513a7fc700 1 -- 172.21.15.97:0/3523827988 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f514001e430 con 0x7f5130003be0 2023-07-06T03:55:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.793+0000 7f513a7fc700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806ace0 0x7f512806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.793+0000 7f513a7fc700 1 -- 172.21.15.97:0/3523827988 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f514008a570 con 0x7f5130003be0 2023-07-06T03:55:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.794+0000 7f514939a700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806ace0 0x7f512806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:47.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.794+0000 7f514939a700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806ace0 0x7f512806d190 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f514415ddf0 tx=0x7f5134009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:47.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.798+0000 7f513a7fc700 1 -- 172.21.15.97:0/3523827988 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f51400556b0 con 0x7f5130003be0 2023-07-06T03:55:47.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.994+0000 7f514bdff700 1 -- 172.21.15.97:0/3523827988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51440b4f40 con 0x7f5130003be0 2023-07-06T03:55:47.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.996+0000 7f513a7fc700 1 -- 172.21.15.97:0/3523827988 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18319 (secure 0 0 0) 0x7f514001c320 con 0x7f5130003be0 2023-07-06T03:55:47.998 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:47.998 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":124,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:55:16.327257+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:55:12.989988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-06T03:55:13.782970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:55:48.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 -- 172.21.15.97:0/3523827988 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806ace0 msgr2=0x7f512806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806ace0 0x7f512806d190 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f514415ddf0 tx=0x7f5134009040 comp rx=0 tx=0).stop 2023-07-06T03:55:48.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 -- 172.21.15.97:0/3523827988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 msgr2=0x7f5144156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 0x7f5144156dc0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f51300047a0 tx=0x7f5140001930 comp rx=0 tx=0).stop 2023-07-06T03:55:48.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 -- 172.21.15.97:0/3523827988 shutdown_connections 2023-07-06T03:55:48.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f512806ace0 0x7f512806d190 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51441565a0 0x7f514415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130003be0 0x7f5144156dc0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 --2- 172.21.15.97:0/3523827988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51440cbb10 0x7f51440ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 -- 172.21.15.97:0/3523827988 >> 172.21.15.97:0/3523827988 conn(0x7f5144071790 msgr2=0x7f514415d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:48.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 -- 172.21.15.97:0/3523827988 shutdown_connections 2023-07-06T03:55:48.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:47.998+0000 7f514bdff700 1 -- 172.21.15.97:0/3523827988 wait complete. 2023-07-06T03:55:48.019 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T03:55:48.019 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:55:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.211+0000 7f24bddff700 1 Processor -- start 2023-07-06T03:55:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.211+0000 7f24bddff700 1 -- start start 2023-07-06T03:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.212+0000 7f24bddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24b815c520 0x7f24b80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.212+0000 7f24bddff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24b8160180 0x7f24b81605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.212+0000 7f24bddff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f24b8160b30 0x7f24b80cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.212+0000 7f24bddff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b80cd8d0 con 0x7f24b8160180 2023-07-06T03:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.212+0000 7f24bddff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b80cdae0 con 0x7f24b8160b30 2023-07-06T03:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.212+0000 7f24bddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80b6640 con 0x7f24b815c520 2023-07-06T03:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24b8160180 0x7f24b81605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24b8160180 0x7f24b81605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37400/0 (socket says 172.21.15.97:37400) 2023-07-06T03:55:48.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b6ffd700 1 -- 172.21.15.97:0/2667384843 learned_addr learned my addr 172.21.15.97:0/2667384843 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b7fff700 1 --2- 172.21.15.97:0/2667384843 >> v2:172.21.15.146:3300/0 conn(0x7f24b8160b30 0x7f24b80cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b77fe700 1 --2- 172.21.15.97:0/2667384843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24b815c520 0x7f24b80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b6ffd700 1 -- 172.21.15.97:0/2667384843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24b815c520 msgr2=0x7f24b80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b6ffd700 1 --2- 172.21.15.97:0/2667384843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24b815c520 0x7f24b80cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b6ffd700 1 -- 172.21.15.97:0/2667384843 >> v2:172.21.15.146:3300/0 conn(0x7f24b8160b30 msgr2=0x7f24b80cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:48.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b6ffd700 1 --2- 172.21.15.97:0/2667384843 >> v2:172.21.15.146:3300/0 conn(0x7f24b8160b30 0x7f24b80cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b6ffd700 1 -- 172.21.15.97:0/2667384843 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b80b6920 con 0x7f24b8160180 2023-07-06T03:55:48.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b7fff700 1 --2- 172.21.15.97:0/2667384843 >> v2:172.21.15.146:3300/0 conn(0x7f24b8160b30 0x7f24b80cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:48.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b77fe700 1 --2- 172.21.15.97:0/2667384843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24b815c520 0x7f24b80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:48.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.213+0000 7f24b6ffd700 1 --2- 172.21.15.97:0/2667384843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b8160180 0x7f24b81605f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f24b808bfc0 tx=0x7f24ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1bb36ea0d574b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:48.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.214+0000 7f24b67fc700 1 -- 172.21.15.97:0/2667384843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac00b870 con 0x7f24b8160180 2023-07-06T03:55:48.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.214+0000 7f24b67fc700 1 -- 172.21.15.97:0/2667384843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ac00b9d0 con 0x7f24b8160180 2023-07-06T03:55:48.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.214+0000 7f24b67fc700 1 -- 172.21.15.97:0/2667384843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac00bd60 con 0x7f24b8160180 2023-07-06T03:55:48.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.214+0000 7f24bddff700 1 -- 172.21.15.97:0/2667384843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b8160180 msgr2=0x7f24b81605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.214+0000 7f24bddff700 1 --2- 172.21.15.97:0/2667384843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b8160180 0x7f24b81605f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f24b808bfc0 tx=0x7f24ac009bd0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.214+0000 7f24bddff700 1 -- 172.21.15.97:0/2667384843 shutdown_connections 2023-07-06T03:55:48.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.214+0000 7f24bddff700 1 --2- 172.21.15.97:0/2667384843 >> v2:172.21.15.146:3300/0 conn(0x7f24b8160b30 0x7f24b80cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.214+0000 7f24bddff700 1 --2- 172.21.15.97:0/2667384843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24b815c520 0x7f24b80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.214+0000 7f24bddff700 1 --2- 172.21.15.97:0/2667384843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b8160180 0x7f24b81605f0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.214+0000 7f24bddff700 1 -- 172.21.15.97:0/2667384843 >> 172.21.15.97:0/2667384843 conn(0x7f24b8071790 msgr2=0x7f24b8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:48.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.214+0000 7f24bddff700 1 -- 172.21.15.97:0/2667384843 shutdown_connections 2023-07-06T03:55:48.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.215+0000 7f24bddff700 1 -- 172.21.15.97:0/2667384843 wait complete. 2023-07-06T03:55:48.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.216+0000 7f24bddff700 1 Processor -- start 2023-07-06T03:55:48.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.216+0000 7f24bddff700 1 -- start start 2023-07-06T03:55:48.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.217+0000 7f24bddff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24b815c520 0x7f24b815c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.217+0000 7f24bddff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b8160180 0x7f24b815b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.217+0000 7f24bddff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24b8160b30 0x7f24b81503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.217+0000 7f24bddff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b815b810 con 0x7f24b8160180 2023-07-06T03:55:48.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.217+0000 7f24bddff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b815cf80 con 0x7f24b815c520 2023-07-06T03:55:48.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.217+0000 7f24bddff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b815d260 con 0x7f24b8160b30 2023-07-06T03:55:48.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.217+0000 7f24b6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b8160180 0x7f24b815b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.217+0000 7f24b6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b8160180 0x7f24b815b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37408/0 (socket says 172.21.15.97:37408) 2023-07-06T03:55:48.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.217+0000 7f24b6ffd700 1 -- 172.21.15.97:0/814075165 learned_addr learned my addr 172.21.15.97:0/814075165 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:48.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f24b77fe700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24b815c520 0x7f24b815c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f24b7fff700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24b8160b30 0x7f24b81503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f24b6ffd700 1 -- 172.21.15.97:0/814075165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24b8160b30 msgr2=0x7f24b81503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f24b6ffd700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24b8160b30 0x7f24b81503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f24b6ffd700 1 -- 172.21.15.97:0/814075165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24b815c520 msgr2=0x7f24b815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f24b6ffd700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24b815c520 0x7f24b815c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f24b6ffd700 1 -- 172.21.15.97:0/814075165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac009550 con 0x7f24b8160180 2023-07-06T03:55:48.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f24b6ffd700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b8160180 0x7f24b815b2d0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f24b806c240 tx=0x7f24ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:48.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f249ffff700 1 -- 172.21.15.97:0/814075165 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac004310 con 0x7f24b8160180 2023-07-06T03:55:48.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f249ffff700 1 -- 172.21.15.97:0/814075165 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ac01f070 con 0x7f24b8160180 2023-07-06T03:55:48.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f24bddff700 1 -- 172.21.15.97:0/814075165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b815d540 con 0x7f24b8160180 2023-07-06T03:55:48.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.218+0000 7f249ffff700 1 -- 172.21.15.97:0/814075165 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac0036a0 con 0x7f24b8160180 2023-07-06T03:55:48.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.219+0000 7f24bddff700 1 -- 172.21.15.97:0/814075165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b815db00 con 0x7f24b8160180 2023-07-06T03:55:48.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.219+0000 7f24bddff700 1 -- 172.21.15.97:0/814075165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b80b4940 con 0x7f24b8160180 2023-07-06T03:55:48.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.221+0000 7f249ffff700 1 -- 172.21.15.97:0/814075165 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f24ac021030 con 0x7f24b8160180 2023-07-06T03:55:48.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.221+0000 7f249ffff700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f24a406ace0 0x7f24a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.222+0000 7f249ffff700 1 -- 172.21.15.97:0/814075165 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f24ac08a830 con 0x7f24b8160180 2023-07-06T03:55:48.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.222+0000 7f24b77fe700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f24a406ace0 0x7f24a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.222+0000 7f24b77fe700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f24a406ace0 0x7f24a406d190 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f24a8006fd0 tx=0x7f24a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:48.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.226+0000 7f249ffff700 1 -- 172.21.15.97:0/814075165 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f24ac0558d0 con 0x7f24b8160180 2023-07-06T03:55:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.467+0000 7f24bddff700 1 -- 172.21.15.97:0/814075165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24b80ce9b0 con 0x7f24b8160180 2023-07-06T03:55:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.468+0000 7f249ffff700 1 -- 172.21.15.97:0/814075165 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4680 (secure 0 0 0) 0x7f24ac058ef0 con 0x7f24b8160180 2023-07-06T03:55:48.470 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:48.470 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":95,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3059444241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3059444241},{"type":"v1","addr":"172.21.15.146:6835","nonce":3059444241}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2624688851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2624688851},{"type":"v1","addr":"172.21.15.97:6837","nonce":2624688851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3938376776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3938376776},{"type":"v1","addr":"172.21.15.97:6835","nonce":3938376776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:55:14.806943+0000","modified":"2023-07-06T03:55:48.000214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9099":{"gid":9099,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6834/3269084599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3269084599},{"type":"v1","addr":"172.21.15.146:6834","nonce":3269084599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-06T03:55:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 -- 172.21.15.97:0/814075165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f24a406ace0 msgr2=0x7f24a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f24a406ace0 0x7f24a406d190 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f24a8006fd0 tx=0x7f24a8008040 comp rx=0 tx=0).stop 2023-07-06T03:55:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 -- 172.21.15.97:0/814075165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b8160180 msgr2=0x7f24b815b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b8160180 0x7f24b815b2d0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f24b806c240 tx=0x7f24ac004750 comp rx=0 tx=0).stop 2023-07-06T03:55:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 -- 172.21.15.97:0/814075165 shutdown_connections 2023-07-06T03:55:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f24a406ace0 0x7f24a406d190 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f24b8160b30 0x7f24b81503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24b815c520 0x7f24b815c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 --2- 172.21.15.97:0/814075165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b8160180 0x7f24b815b2d0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 -- 172.21.15.97:0/814075165 >> 172.21.15.97:0/814075165 conn(0x7f24b8071790 msgr2=0x7f24b8156aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 -- 172.21.15.97:0/814075165 shutdown_connections 2023-07-06T03:55:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.470+0000 7f24bddff700 1 -- 172.21.15.97:0/814075165 wait complete. 2023-07-06T03:55:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 95 2023-07-06T03:55:48.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:55:48.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.684+0000 7f02f0bff700 1 Processor -- start 2023-07-06T03:55:48.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.684+0000 7f02f0bff700 1 -- start start 2023-07-06T03:55:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.685+0000 7f02f0bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02ec158f20 0x7f02ec0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.685+0000 7f02f0bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f02ec0b0c70 0x7f02ec06bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.685+0000 7f02f0bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ec15cc40 con 0x7f02ec1592f0 2023-07-06T03:55:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.685+0000 7f02f0bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ec15cf20 con 0x7f02ec158f20 2023-07-06T03:55:48.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.685+0000 7f02f0bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ec15d200 con 0x7f02ec0b0c70 2023-07-06T03:55:48.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.685+0000 7f02ea59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02ec158f20 0x7f02ec0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.685+0000 7f02ea59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02ec158f20 0x7f02ec0caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37418/0 (socket says 172.21.15.97:37418) 2023-07-06T03:55:48.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.685+0000 7f02ea59c700 1 -- 172.21.15.97:0/4237621876 learned_addr learned my addr 172.21.15.97:0/4237621876 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:48.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.685+0000 7f02e9d9b700 1 --1- 172.21.15.97:0/4237621876 >> v1:172.21.15.97:6789/0 conn(0x7f02ec1592f0 0x7f02ec0b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43416/0 (socket says 172.21.15.97:43416) 2023-07-06T03:55:48.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.686+0000 7f02ead9d700 1 --2- 172.21.15.97:0/4237621876 >> v2:172.21.15.146:3300/0 conn(0x7f02ec0b0c70 0x7f02ec06bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.686+0000 7f02ea59c700 1 -- 172.21.15.97:0/4237621876 >> v2:172.21.15.146:3300/0 conn(0x7f02ec0b0c70 msgr2=0x7f02ec06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.686+0000 7f02ea59c700 1 --2- 172.21.15.97:0/4237621876 >> v2:172.21.15.146:3300/0 conn(0x7f02ec0b0c70 0x7f02ec06bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.686+0000 7f02ea59c700 1 -- 172.21.15.97:0/4237621876 >> v1:172.21.15.97:6789/0 conn(0x7f02ec1592f0 legacy=0x7f02ec0b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.686+0000 7f02ea59c700 1 -- 172.21.15.97:0/4237621876 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec15d4e0 con 0x7f02ec158f20 2023-07-06T03:55:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.686+0000 7f02e959a700 1 -- 172.21.15.97:0/4237621876 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381235294 0 0) 0x7f02ec15cc40 con 0x7f02ec1592f0 2023-07-06T03:55:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.686+0000 7f02ea59c700 1 --2- 172.21.15.97:0/4237621876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02ec158f20 0x7f02ec0caac0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0ae4f0 tx=0x7f02dc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e43135d924f210e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.686+0000 7f02e959a700 1 -- 172.21.15.97:0/4237621876 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00dd30 con 0x7f02ec158f20 2023-07-06T03:55:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.686+0000 7f02e959a700 1 -- 172.21.15.97:0/4237621876 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc00de90 con 0x7f02ec158f20 2023-07-06T03:55:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.686+0000 7f02e959a700 1 -- 172.21.15.97:0/4237621876 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc010690 con 0x7f02ec158f20 2023-07-06T03:55:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.687+0000 7f02f0bff700 1 -- 172.21.15.97:0/4237621876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02ec158f20 msgr2=0x7f02ec0caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.687+0000 7f02f0bff700 1 --2- 172.21.15.97:0/4237621876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02ec158f20 0x7f02ec0caac0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0ae4f0 tx=0x7f02dc00be70 comp rx=0 tx=0).stop 2023-07-06T03:55:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.687+0000 7f02f0bff700 1 -- 172.21.15.97:0/4237621876 shutdown_connections 2023-07-06T03:55:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.687+0000 7f02f0bff700 1 --2- 172.21.15.97:0/4237621876 >> v2:172.21.15.146:3300/0 conn(0x7f02ec0b0c70 0x7f02ec06bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.687+0000 7f02f0bff700 1 --2- 172.21.15.97:0/4237621876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02ec158f20 0x7f02ec0caac0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.687+0000 7f02f0bff700 1 -- 172.21.15.97:0/4237621876 >> 172.21.15.97:0/4237621876 conn(0x7f02ec0ce8c0 msgr2=0x7f02ec0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.687+0000 7f02f0bff700 1 -- 172.21.15.97:0/4237621876 shutdown_connections 2023-07-06T03:55:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.687+0000 7f02f0bff700 1 -- 172.21.15.97:0/4237621876 wait complete. 2023-07-06T03:55:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.689+0000 7f02f0bff700 1 Processor -- start 2023-07-06T03:55:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.689+0000 7f02f0bff700 1 -- start start 2023-07-06T03:55:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.690+0000 7f02f0bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02ec0b0c70 0x7f02ec149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.690+0000 7f02f0bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02ec158f20 0x7f02ec14e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.690+0000 7f02f0bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ec1592f0 0x7f02ec1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.690+0000 7f02f0bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec14e500 con 0x7f02ec158f20 2023-07-06T03:55:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.690+0000 7f02f0bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec14a140 con 0x7f02ec0b0c70 2023-07-06T03:55:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.690+0000 7f02f0bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec14e1b0 con 0x7f02ec1592f0 2023-07-06T03:55:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.690+0000 7f02e9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02ec158f20 0x7f02ec14e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.690+0000 7f02e9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02ec158f20 0x7f02ec14e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37430/0 (socket says 172.21.15.97:37430) 2023-07-06T03:55:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.690+0000 7f02e9d9b700 1 -- 172.21.15.97:0/3578703956 learned_addr learned my addr 172.21.15.97:0/3578703956 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.690+0000 7f02ea59c700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02ec0b0c70 0x7f02ec149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.690+0000 7f02ead9d700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ec1592f0 0x7f02ec1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02e9d9b700 1 -- 172.21.15.97:0/3578703956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ec1592f0 msgr2=0x7f02ec1596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02e9d9b700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ec1592f0 0x7f02ec1596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02e9d9b700 1 -- 172.21.15.97:0/3578703956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02ec0b0c70 msgr2=0x7f02ec149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02e9d9b700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02ec0b0c70 0x7f02ec149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02e9d9b700 1 -- 172.21.15.97:0/3578703956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02dc00b7f0 con 0x7f02ec158f20 2023-07-06T03:55:48.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02ead9d700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ec1592f0 0x7f02ec1596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:48.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02e9d9b700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02ec158f20 0x7f02ec14e990 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f02e000c550 tx=0x7f02e000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:48.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02daffd700 1 -- 172.21.15.97:0/3578703956 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0004c80 con 0x7f02ec158f20 2023-07-06T03:55:48.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02daffd700 1 -- 172.21.15.97:0/3578703956 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e0004e00 con 0x7f02ec158f20 2023-07-06T03:55:48.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02daffd700 1 -- 172.21.15.97:0/3578703956 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0007310 con 0x7f02ec158f20 2023-07-06T03:55:48.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02f0bff700 1 -- 172.21.15.97:0/3578703956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec15a340 con 0x7f02ec158f20 2023-07-06T03:55:48.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.691+0000 7f02f0bff700 1 -- 172.21.15.97:0/3578703956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec15a900 con 0x7f02ec158f20 2023-07-06T03:55:48.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.692+0000 7f02f0bff700 1 -- 172.21.15.97:0/3578703956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ec04e340 con 0x7f02ec158f20 2023-07-06T03:55:48.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.693+0000 7f02daffd700 1 -- 172.21.15.97:0/3578703956 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f02e0007470 con 0x7f02ec158f20 2023-07-06T03:55:48.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.694+0000 7f02daffd700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f02cc06ace0 0x7f02cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:48.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.694+0000 7f02daffd700 1 -- 172.21.15.97:0/3578703956 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f02e00890e0 con 0x7f02ec158f20 2023-07-06T03:55:48.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.694+0000 7f02ea59c700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f02cc06ace0 0x7f02cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:48.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.695+0000 7f02ea59c700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f02cc06ace0 0x7f02cc06d190 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00b6f0 tx=0x7f02dc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:48.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.698+0000 7f02daffd700 1 -- 172.21.15.97:0/3578703956 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f02e0054190 con 0x7f02ec158f20 2023-07-06T03:55:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.939+0000 7f02f0bff700 1 -- 172.21.15.97:0/3578703956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02ec14e320 con 0x7f02ec158f20 2023-07-06T03:55:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.940+0000 7f02daffd700 1 -- 172.21.15.97:0/3578703956 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4680 (secure 0 0 0) 0x7f02e00577b0 con 0x7f02ec158f20 2023-07-06T03:55:48.942 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:48.942 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":95,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3059444241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3059444241},{"type":"v1","addr":"172.21.15.146:6835","nonce":3059444241}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2624688851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2624688851},{"type":"v1","addr":"172.21.15.97:6837","nonce":2624688851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3938376776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3938376776},{"type":"v1","addr":"172.21.15.97:6835","nonce":3938376776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:55:14.806943+0000","modified":"2023-07-06T03:55:48.000214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9099":{"gid":9099,"name":"d","rank":0,"incarnation":88,"state":"up:active","state_seq":11,"addr":"172.21.15.146:6834/3269084599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3269084599},{"type":"v1","addr":"172.21.15.146:6834","nonce":3269084599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-07-06T03:55:48.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 -- 172.21.15.97:0/3578703956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f02cc06ace0 msgr2=0x7f02cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f02cc06ace0 0x7f02cc06d190 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00b6f0 tx=0x7f02dc01c040 comp rx=0 tx=0).stop 2023-07-06T03:55:48.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 -- 172.21.15.97:0/3578703956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02ec158f20 msgr2=0x7f02ec14e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:48.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02ec158f20 0x7f02ec14e990 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f02e000c550 tx=0x7f02e000c860 comp rx=0 tx=0).stop 2023-07-06T03:55:48.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 -- 172.21.15.97:0/3578703956 shutdown_connections 2023-07-06T03:55:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f02cc06ace0 0x7f02cc06d190 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f02ec1592f0 0x7f02ec1596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02ec0b0c70 0x7f02ec149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 --2- 172.21.15.97:0/3578703956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02ec158f20 0x7f02ec14e990 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 -- 172.21.15.97:0/3578703956 >> 172.21.15.97:0/3578703956 conn(0x7f02ec0ce8c0 msgr2=0x7f02ec06f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 -- 172.21.15.97:0/3578703956 shutdown_connections 2023-07-06T03:55:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:48.942+0000 7f02f0bff700 1 -- 172.21.15.97:0/3578703956 wait complete. 2023-07-06T03:55:48.946 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 95 2023-07-06T03:55:48.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:55:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.155+0000 7f5a6b7ff700 1 Processor -- start 2023-07-06T03:55:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.155+0000 7f5a6b7ff700 1 -- start start 2023-07-06T03:55:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.156+0000 7f5a6b7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5a640cd390 0x7f5a64160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.156+0000 7f5a6b7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a64160dd0 0x7f5a640b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.156+0000 7f5a6b7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a640cdae0 con 0x7f5a6415c520 2023-07-06T03:55:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.156+0000 7f5a6b7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640b6640 con 0x7f5a640cd390 2023-07-06T03:55:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.157+0000 7f5a6b7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640b6920 con 0x7f5a64160dd0 2023-07-06T03:55:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.157+0000 7f5a69d9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a64160dd0 0x7f5a640b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.157+0000 7f5a69d9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a64160dd0 0x7f5a640b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:44694/0 (socket says 172.21.15.97:44694) 2023-07-06T03:55:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.157+0000 7f5a69d9c700 1 -- 172.21.15.97:0/561907874 learned_addr learned my addr 172.21.15.97:0/561907874 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.157+0000 7f5a68d9a700 1 --2- 172.21.15.97:0/561907874 >> v2:172.21.15.146:3300/0 conn(0x7f5a640cd390 0x7f5a64160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.157+0000 7f5a69d9c700 1 -- 172.21.15.97:0/561907874 >> v2:172.21.15.146:3300/0 conn(0x7f5a640cd390 msgr2=0x7f5a64160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.157+0000 7f5a69d9c700 1 --2- 172.21.15.97:0/561907874 >> v2:172.21.15.146:3300/0 conn(0x7f5a640cd390 0x7f5a64160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.157+0000 7f5a5bfff700 1 -- 172.21.15.97:0/561907874 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867522454 0 0) 0x7f5a640cdae0 con 0x7f5a6415c520 2023-07-06T03:55:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.157+0000 7f5a69d9c700 1 -- 172.21.15.97:0/561907874 >> v1:172.21.15.146:6789/0 conn(0x7f5a6415c520 legacy=0x7f5a64160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.157+0000 7f5a69d9c700 1 -- 172.21.15.97:0/561907874 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640b6c00 con 0x7f5a64160dd0 2023-07-06T03:55:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.158+0000 7f5a69d9c700 1 --2- 172.21.15.97:0/561907874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a64160dd0 0x7f5a640b7400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009e50 tx=0x7f5a5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f26e2bb076c319 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.158+0000 7f5a5bfff700 1 -- 172.21.15.97:0/561907874 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00e070 con 0x7f5a64160dd0 2023-07-06T03:55:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.158+0000 7f5a5bfff700 1 -- 172.21.15.97:0/561907874 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c00b910 con 0x7f5a64160dd0 2023-07-06T03:55:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.158+0000 7f5a5bfff700 1 -- 172.21.15.97:0/561907874 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00bcc0 con 0x7f5a64160dd0 2023-07-06T03:55:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.158+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/561907874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a64160dd0 msgr2=0x7f5a640b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.158+0000 7f5a6b7ff700 1 --2- 172.21.15.97:0/561907874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a64160dd0 0x7f5a640b7400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009e50 tx=0x7f5a5c00b330 comp rx=0 tx=0).stop 2023-07-06T03:55:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.159+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/561907874 shutdown_connections 2023-07-06T03:55:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.159+0000 7f5a6b7ff700 1 --2- 172.21.15.97:0/561907874 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a64160dd0 0x7f5a640b7400 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.159+0000 7f5a6b7ff700 1 --2- 172.21.15.97:0/561907874 >> v2:172.21.15.146:3300/0 conn(0x7f5a640cd390 0x7f5a64160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.159+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/561907874 >> 172.21.15.97:0/561907874 conn(0x7f5a64071790 msgr2=0x7f5a64071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.159+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/561907874 shutdown_connections 2023-07-06T03:55:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.159+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/561907874 wait complete. 2023-07-06T03:55:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.160+0000 7f5a6b7ff700 1 Processor -- start 2023-07-06T03:55:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.160+0000 7f5a6b7ff700 1 -- start start 2023-07-06T03:55:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.161+0000 7f5a6b7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a640cd390 0x7f5a640d2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.161+0000 7f5a6b7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a6415c520 0x7f5a641f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.161+0000 7f5a6b7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a64160dd0 0x7f5a640cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.161+0000 7f5a6b7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a641f3250 con 0x7f5a6415c520 2023-07-06T03:55:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.161+0000 7f5a6b7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640d3b80 con 0x7f5a64160dd0 2023-07-06T03:55:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.161+0000 7f5a6b7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640d3000 con 0x7f5a640cd390 2023-07-06T03:55:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.162+0000 7f5a68d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a6415c520 0x7f5a641f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.162+0000 7f5a68d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a6415c520 0x7f5a641f2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37442/0 (socket says 172.21.15.97:37442) 2023-07-06T03:55:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.162+0000 7f5a68d9a700 1 -- 172.21.15.97:0/1431266701 learned_addr learned my addr 172.21.15.97:0/1431266701 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.162+0000 7f5a6959b700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a640cd390 0x7f5a640d2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.162+0000 7f5a69d9c700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a64160dd0 0x7f5a640cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.162+0000 7f5a68d9a700 1 -- 172.21.15.97:0/1431266701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a640cd390 msgr2=0x7f5a640d2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.162+0000 7f5a68d9a700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a640cd390 0x7f5a640d2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.162+0000 7f5a68d9a700 1 -- 172.21.15.97:0/1431266701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a64160dd0 msgr2=0x7f5a640cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.162+0000 7f5a68d9a700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a64160dd0 0x7f5a640cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.162+0000 7f5a68d9a700 1 -- 172.21.15.97:0/1431266701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c009b30 con 0x7f5a6415c520 2023-07-06T03:55:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.162+0000 7f5a68d9a700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a6415c520 0x7f5a641f2d10 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e870 tx=0x7f5a5400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.163+0000 7f5a59ffb700 1 -- 172.21.15.97:0/1431266701 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54009590 con 0x7f5a6415c520 2023-07-06T03:55:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.163+0000 7f5a59ffb700 1 -- 172.21.15.97:0/1431266701 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a54013680 con 0x7f5a6415c520 2023-07-06T03:55:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.163+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/1431266701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a640d32e0 con 0x7f5a6415c520 2023-07-06T03:55:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.163+0000 7f5a59ffb700 1 -- 172.21.15.97:0/1431266701 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a540139f0 con 0x7f5a6415c520 2023-07-06T03:55:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.163+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/1431266701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a640d38a0 con 0x7f5a6415c520 2023-07-06T03:55:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.164+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/1431266701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6404e340 con 0x7f5a6415c520 2023-07-06T03:55:49.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.165+0000 7f5a59ffb700 1 -- 172.21.15.97:0/1431266701 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a54013b50 con 0x7f5a6415c520 2023-07-06T03:55:49.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.166+0000 7f5a59ffb700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.166+0000 7f5a59ffb700 1 -- 172.21.15.97:0/1431266701 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a54089280 con 0x7f5a6415c520 2023-07-06T03:55:49.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.166+0000 7f5a6959b700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.166+0000 7f5a6959b700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 0x7f5a4806d190 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5a60000f60 tx=0x7f5a60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:49.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.170+0000 7f5a59ffb700 1 -- 172.21.15.97:0/1431266701 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a54057dc0 con 0x7f5a6415c520 2023-07-06T03:55:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.367+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/1431266701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a640b4f40 con 0x7f5a6415c520 2023-07-06T03:55:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.368+0000 7f5a59ffb700 1 -- 172.21.15.97:0/1431266701 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+18319 (secure 0 0 0) 0x7f5a54057950 con 0x7f5a6415c520 2023-07-06T03:55:49.370 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:49.370 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":124,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:55:16.327257+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:55:12.989988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-07-06T03:55:13.782970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:55:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.370+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/1431266701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 msgr2=0x7f5a4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.370+0000 7f5a6b7ff700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 0x7f5a4806d190 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5a60000f60 tx=0x7f5a60009040 comp rx=0 tx=0).stop 2023-07-06T03:55:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.370+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/1431266701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a6415c520 msgr2=0x7f5a641f2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.370+0000 7f5a6b7ff700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a6415c520 0x7f5a641f2d10 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e870 tx=0x7f5a5400eb80 comp rx=0 tx=0).stop 2023-07-06T03:55:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.370+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/1431266701 shutdown_connections 2023-07-06T03:55:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.370+0000 7f5a6b7ff700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.370+0000 7f5a6b7ff700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5a64160dd0 0x7f5a640cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.370+0000 7f5a6b7ff700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5a640cd390 0x7f5a640d2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.370+0000 7f5a6b7ff700 1 --2- 172.21.15.97:0/1431266701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a6415c520 0x7f5a641f2d10 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.370+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/1431266701 >> 172.21.15.97:0/1431266701 conn(0x7f5a64071790 msgr2=0x7f5a640b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.370+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/1431266701 shutdown_connections 2023-07-06T03:55:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.371+0000 7f5a6b7ff700 1 -- 172.21.15.97:0/1431266701 wait complete. 2023-07-06T03:55:49.391 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T03:55:49.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.583+0000 7f27231ff700 1 Processor -- start 2023-07-06T03:55:49.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.583+0000 7f27231ff700 1 -- start start 2023-07-06T03:55:49.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.585+0000 7f27231ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f271c15c610 0x7f271c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.585+0000 7f27231ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f271c06b360 0x7f271c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.585+0000 7f27231ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271c0cff90 con 0x7f271c0d2210 2023-07-06T03:55:49.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.585+0000 7f27231ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c0d0780 con 0x7f271c15c610 2023-07-06T03:55:49.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.585+0000 7f27231ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c15c9e0 con 0x7f271c06b360 2023-07-06T03:55:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.585+0000 7f271bfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f271c06b360 0x7f271c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.585+0000 7f2720f9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f271c15c610 0x7f271c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.585+0000 7f272179c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f271c0d2210 0x7f271c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:40212/0 (socket says 172.21.15.97:40212) 2023-07-06T03:55:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.585+0000 7f272179c700 1 -- 172.21.15.97:0/2403041347 learned_addr learned my addr 172.21.15.97:0/2403041347 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.586+0000 7f2720f9b700 1 -- 172.21.15.97:0/2403041347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f271c06b360 msgr2=0x7f271c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.586+0000 7f2720f9b700 1 --2- 172.21.15.97:0/2403041347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f271c06b360 0x7f271c0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.586+0000 7f2720f9b700 1 -- 172.21.15.97:0/2403041347 >> v1:172.21.15.146:6789/0 conn(0x7f271c0d2210 legacy=0x7f271c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.586+0000 7f2720f9b700 1 -- 172.21.15.97:0/2403041347 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c0b5650 con 0x7f271c15c610 2023-07-06T03:55:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.586+0000 7f271b7fe700 1 -- 172.21.15.97:0/2403041347 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015273005 0 0) 0x7f271c0cff90 con 0x7f271c0d2210 2023-07-06T03:55:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.586+0000 7f2720f9b700 1 --2- 172.21.15.97:0/2403041347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f271c15c610 0x7f271c0b47b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2710009870 tx=0x7f2710009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e77ec875cb3dd20e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.586+0000 7f271b7fe700 1 -- 172.21.15.97:0/2403041347 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710011830 con 0x7f271c15c610 2023-07-06T03:55:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.586+0000 7f271b7fe700 1 -- 172.21.15.97:0/2403041347 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2710011990 con 0x7f271c15c610 2023-07-06T03:55:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.586+0000 7f271b7fe700 1 -- 172.21.15.97:0/2403041347 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710011d20 con 0x7f271c15c610 2023-07-06T03:55:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.587+0000 7f27231ff700 1 -- 172.21.15.97:0/2403041347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f271c15c610 msgr2=0x7f271c0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.587+0000 7f27231ff700 1 --2- 172.21.15.97:0/2403041347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f271c15c610 0x7f271c0b47b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2710009870 tx=0x7f2710009b80 comp rx=0 tx=0).stop 2023-07-06T03:55:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.587+0000 7f27231ff700 1 -- 172.21.15.97:0/2403041347 shutdown_connections 2023-07-06T03:55:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.587+0000 7f27231ff700 1 --2- 172.21.15.97:0/2403041347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f271c15c610 0x7f271c0b47b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.587+0000 7f27231ff700 1 --2- 172.21.15.97:0/2403041347 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f271c06b360 0x7f271c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.587+0000 7f27231ff700 1 -- 172.21.15.97:0/2403041347 >> 172.21.15.97:0/2403041347 conn(0x7f271c06a6c0 msgr2=0x7f271c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.587+0000 7f27231ff700 1 -- 172.21.15.97:0/2403041347 shutdown_connections 2023-07-06T03:55:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.587+0000 7f27231ff700 1 -- 172.21.15.97:0/2403041347 wait complete. 2023-07-06T03:55:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.589+0000 7f27231ff700 1 Processor -- start 2023-07-06T03:55:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.589+0000 7f27231ff700 1 -- start start 2023-07-06T03:55:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.590+0000 7f27231ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f271c06b360 0x7f271c0d1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.590+0000 7f27231ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f271c0d2210 0x7f271c0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.590+0000 7f27231ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f271c15c610 0x7f271c14ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.590+0000 7f27231ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0c89b0 con 0x7f271c15c610 2023-07-06T03:55:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.590+0000 7f27231ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0c7d30 con 0x7f271c06b360 2023-07-06T03:55:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.590+0000 7f27231ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0c8010 con 0x7f271c0d2210 2023-07-06T03:55:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.590+0000 7f272179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f271c15c610 0x7f271c14ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.590+0000 7f272179c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f271c15c610 0x7f271c14ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37456/0 (socket says 172.21.15.97:37456) 2023-07-06T03:55:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.590+0000 7f272179c700 1 -- 172.21.15.97:0/3813980220 learned_addr learned my addr 172.21.15.97:0/3813980220 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.590+0000 7f2720f9b700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f271c06b360 0x7f271c0d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.590+0000 7f271bfff700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f271c0d2210 0x7f271c0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f272179c700 1 -- 172.21.15.97:0/3813980220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f271c0d2210 msgr2=0x7f271c0c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f272179c700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f271c0d2210 0x7f271c0c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f272179c700 1 -- 172.21.15.97:0/3813980220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f271c06b360 msgr2=0x7f271c0d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f272179c700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f271c06b360 0x7f271c0d1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f272179c700 1 -- 172.21.15.97:0/3813980220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100094a0 con 0x7f271c15c610 2023-07-06T03:55:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f2720f9b700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f271c06b360 0x7f271c0d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:55:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f272179c700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f271c15c610 0x7f271c14ad20 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2714002230 tx=0x7f271400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f27197fa700 1 -- 172.21.15.97:0/3813980220 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271400aac0 con 0x7f271c15c610 2023-07-06T03:55:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f27197fa700 1 -- 172.21.15.97:0/3813980220 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271400ac40 con 0x7f271c15c610 2023-07-06T03:55:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f27197fa700 1 -- 172.21.15.97:0/3813980220 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27140070d0 con 0x7f271c15c610 2023-07-06T03:55:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f27231ff700 1 -- 172.21.15.97:0/3813980220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c0c82f0 con 0x7f271c15c610 2023-07-06T03:55:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.591+0000 7f27231ff700 1 -- 172.21.15.97:0/3813980220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c0cc060 con 0x7f271c15c610 2023-07-06T03:55:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.592+0000 7f27231ff700 1 -- 172.21.15.97:0/3813980220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c0c8af0 con 0x7f271c15c610 2023-07-06T03:55:49.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.593+0000 7f27197fa700 1 -- 172.21.15.97:0/3813980220 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2714007250 con 0x7f271c15c610 2023-07-06T03:55:49.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.595+0000 7f27197fa700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f270006ace0 0x7f270006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:49.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.595+0000 7f27197fa700 1 -- 172.21.15.97:0/3813980220 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2714088f50 con 0x7f271c15c610 2023-07-06T03:55:49.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.595+0000 7f2720f9b700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f270006ace0 0x7f270006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:49.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.596+0000 7f2720f9b700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f270006ace0 0x7f270006d190 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f27100097c0 tx=0x7f271001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:49.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.599+0000 7f27197fa700 1 -- 172.21.15.97:0/3813980220 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f271c0c8af0 con 0x7f271c15c610 2023-07-06T03:55:49.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.838+0000 7f27231ff700 1 -- 172.21.15.97:0/3813980220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f271c0c8af0 con 0x7f271c15c610 2023-07-06T03:55:49.840 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-07-06T03:55:49.839+0000 7fb463ff4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T03:55:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.845+0000 7f27197fa700 1 -- 172.21.15.97:0/3813980220 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v96) v1 ==== 161+0+0 (secure 0 0 0) 0x7f271c0c8af0 con 0x7f271c15c610 2023-07-06T03:55:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 -- 172.21.15.97:0/3813980220 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f270006ace0 msgr2=0x7f270006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f270006ace0 0x7f270006d190 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f27100097c0 tx=0x7f271001c040 comp rx=0 tx=0).stop 2023-07-06T03:55:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 -- 172.21.15.97:0/3813980220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f271c15c610 msgr2=0x7f271c14ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f271c15c610 0x7f271c14ad20 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2714002230 tx=0x7f271400cf50 comp rx=0 tx=0).stop 2023-07-06T03:55:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 -- 172.21.15.97:0/3813980220 shutdown_connections 2023-07-06T03:55:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f270006ace0 0x7f270006d190 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f271c15c610 0x7f271c14ad20 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f271c06b360 0x7f271c0d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 --2- 172.21.15.97:0/3813980220 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f271c0d2210 0x7f271c0c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:49.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 -- 172.21.15.97:0/3813980220 >> 172.21.15.97:0/3813980220 conn(0x7f271c06a6c0 msgr2=0x7f271c15ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:49.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 -- 172.21.15.97:0/3813980220 shutdown_connections 2023-07-06T03:55:49.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:49.848+0000 7f27231ff700 1 -- 172.21.15.97:0/3813980220 wait complete. 2023-07-06T03:55:49.852 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T03:55:49.868 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T03:55:49.915 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:55:50.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.065+0000 7f40fc9fe700 1 Processor -- start 2023-07-06T03:55:50.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.065+0000 7f40fc9fe700 1 -- start start 2023-07-06T03:55:50.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.066+0000 7f40fc9fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f40f80d1330 0x7f40f80b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:50.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.066+0000 7f40fc9fe700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f815cb70 con 0x7f40f80b8470 2023-07-06T03:55:50.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.066+0000 7f40fc9fe700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f80cd1d0 con 0x7f40f80cd660 2023-07-06T03:55:50.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.066+0000 7f40fc9fe700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f80d2690 con 0x7f40f80d1330 2023-07-06T03:55:50.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.066+0000 7f40f6d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f40f80b8470 0x7f40f80d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42454/0 (socket says 172.21.15.97:42454) 2023-07-06T03:55:50.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.066+0000 7f40f6d9d700 1 -- 172.21.15.97:0/3473126952 learned_addr learned my addr 172.21.15.97:0/3473126952 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:50.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.066+0000 7f40f659c700 1 --2- 172.21.15.97:0/3473126952 >> v2:172.21.15.97:3300/0 conn(0x7f40f80d1330 0x7f40f80b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:50.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.067+0000 7f40f5d9b700 1 -- 172.21.15.97:0/3473126952 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000907363 0 0) 0x7f40e8001110 con 0x7f40f80b8470 2023-07-06T03:55:50.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.067+0000 7f40f5d9b700 1 -- 172.21.15.97:0/3473126952 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40e00035b0 con 0x7f40f80b8470 2023-07-06T03:55:50.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.067+0000 7f40f659c700 1 -- 172.21.15.97:0/3473126952 >> v1:172.21.15.146:6789/0 conn(0x7f40f80cd660 legacy=0x7f40f80cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:50.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.067+0000 7f40f5d9b700 1 -- 172.21.15.97:0/3473126952 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924407979 0 0) 0x7f40f80cd1d0 con 0x7f40f80cd660 2023-07-06T03:55:50.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.067+0000 7f40f659c700 1 -- 172.21.15.97:0/3473126952 >> v1:172.21.15.97:6789/0 conn(0x7f40f80b8470 legacy=0x7f40f80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:50.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.067+0000 7f40f659c700 1 -- 172.21.15.97:0/3473126952 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f8160cc0 con 0x7f40f80d1330 2023-07-06T03:55:50.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.067+0000 7f40f659c700 1 --2- 172.21.15.97:0/3473126952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40f80d1330 0x7f40f80b2050 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009840 tx=0x7f40ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99123c3a14197f52 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:50.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.067+0000 7f40f5d9b700 1 -- 172.21.15.97:0/3473126952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec011830 con 0x7f40f80d1330 2023-07-06T03:55:50.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.067+0000 7f40f5d9b700 1 -- 172.21.15.97:0/3473126952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec011990 con 0x7f40f80d1330 2023-07-06T03:55:50.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.067+0000 7f40f5d9b700 1 -- 172.21.15.97:0/3473126952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec011d20 con 0x7f40f80d1330 2023-07-06T03:55:50.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.068+0000 7f40fc9fe700 1 -- 172.21.15.97:0/3473126952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40f80d1330 msgr2=0x7f40f80b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:50.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.068+0000 7f40fc9fe700 1 --2- 172.21.15.97:0/3473126952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40f80d1330 0x7f40f80b2050 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009840 tx=0x7f40ec009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:50.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.068+0000 7f40fc9fe700 1 -- 172.21.15.97:0/3473126952 shutdown_connections 2023-07-06T03:55:50.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.068+0000 7f40fc9fe700 1 --2- 172.21.15.97:0/3473126952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40f80d1330 0x7f40f80b2050 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:50.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.068+0000 7f40fc9fe700 1 -- 172.21.15.97:0/3473126952 >> 172.21.15.97:0/3473126952 conn(0x7f40f806b160 msgr2=0x7f40f8071790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:50.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.068+0000 7f40fc9fe700 1 -- 172.21.15.97:0/3473126952 shutdown_connections 2023-07-06T03:55:50.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.068+0000 7f40fc9fe700 1 -- 172.21.15.97:0/3473126952 wait complete. 2023-07-06T03:55:50.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.070+0000 7f40fc9fe700 1 Processor -- start 2023-07-06T03:55:50.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.070+0000 7f40fc9fe700 1 -- start start 2023-07-06T03:55:50.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40fc9fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f80b8470 0x7f40f815d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:50.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40fc9fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40f80cd660 0x7f40f814d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:50.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40fc9fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40f80d1330 0x7f40f80ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:50.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40fc9fe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f80cb590 con 0x7f40f80cd660 2023-07-06T03:55:50.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40fc9fe700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f80cb2b0 con 0x7f40f80b8470 2023-07-06T03:55:50.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40fc9fe700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f815ce90 con 0x7f40f80d1330 2023-07-06T03:55:50.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40f80cd660 0x7f40f814d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:50.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40f80cd660 0x7f40f814d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59878/0 (socket says 172.21.15.97:59878) 2023-07-06T03:55:50.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f659c700 1 -- 172.21.15.97:0/1437562054 learned_addr learned my addr 172.21.15.97:0/1437562054 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:50.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f759e700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40f80d1330 0x7f40f80ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:50.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f6d9d700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f80b8470 0x7f40f815d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:50.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f659c700 1 -- 172.21.15.97:0/1437562054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40f80d1330 msgr2=0x7f40f80ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:50.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f659c700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40f80d1330 0x7f40f80ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:50.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f659c700 1 -- 172.21.15.97:0/1437562054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f80b8470 msgr2=0x7f40f815d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:50.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f659c700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f80b8470 0x7f40f815d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:50.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f659c700 1 -- 172.21.15.97:0/1437562054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec0094a0 con 0x7f40f80cd660 2023-07-06T03:55:50.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f759e700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40f80d1330 0x7f40f80ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:50.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.071+0000 7f40f6d9d700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f80b8470 0x7f40f815d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:50.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.072+0000 7f40f659c700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40f80cd660 0x7f40f814d160 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0090b0 tx=0x7f40ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:50.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.072+0000 7f40df7fe700 1 -- 172.21.15.97:0/1437562054 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec01a910 con 0x7f40f80cd660 2023-07-06T03:55:50.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.072+0000 7f40df7fe700 1 -- 172.21.15.97:0/1437562054 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec01aa70 con 0x7f40f80cd660 2023-07-06T03:55:50.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.072+0000 7f40df7fe700 1 -- 172.21.15.97:0/1437562054 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec01ade0 con 0x7f40f80cd660 2023-07-06T03:55:50.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.072+0000 7f40fc9fe700 1 -- 172.21.15.97:0/1437562054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f815d170 con 0x7f40f80cd660 2023-07-06T03:55:50.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.072+0000 7f40fc9fe700 1 -- 172.21.15.97:0/1437562054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f814d6a0 con 0x7f40f80cd660 2023-07-06T03:55:50.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.073+0000 7f40fc9fe700 1 -- 172.21.15.97:0/1437562054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f80b7fe0 con 0x7f40f80cd660 2023-07-06T03:55:50.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.074+0000 7f40df7fe700 1 -- 172.21.15.97:0/1437562054 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f40ec022070 con 0x7f40f80cd660 2023-07-06T03:55:50.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.075+0000 7f40df7fe700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40e406ace0 0x7f40e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:50.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.075+0000 7f40df7fe700 1 -- 172.21.15.97:0/1437562054 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f40ec08a220 con 0x7f40f80cd660 2023-07-06T03:55:50.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.075+0000 7f40f6d9d700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40e406ace0 0x7f40e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:50.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.076+0000 7f40f6d9d700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40e406ace0 0x7f40e406d190 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f40e8007fd0 tx=0x7f40e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:50.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.079+0000 7f40df7fe700 1 -- 172.21.15.97:0/1437562054 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f40ec055430 con 0x7f40f80cd660 2023-07-06T03:55:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.318+0000 7f40fc9fe700 1 -- 172.21.15.97:0/1437562054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f40f8072120 con 0x7f40f80cd660 2023-07-06T03:55:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.845+0000 7f40df7fe700 1 -- 172.21.15.97:0/1437562054 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v97) v1 ==== 104+0+0 (secure 0 0 0) 0x7f40ec058a50 con 0x7f40f80cd660 2023-07-06T03:55:50.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 -- 172.21.15.97:0/1437562054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40e406ace0 msgr2=0x7f40e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:50.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40e406ace0 0x7f40e406d190 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f40e8007fd0 tx=0x7f40e8009040 comp rx=0 tx=0).stop 2023-07-06T03:55:50.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 -- 172.21.15.97:0/1437562054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40f80cd660 msgr2=0x7f40f814d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:50.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40f80cd660 0x7f40f814d160 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0090b0 tx=0x7f40ec004750 comp rx=0 tx=0).stop 2023-07-06T03:55:50.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 -- 172.21.15.97:0/1437562054 shutdown_connections 2023-07-06T03:55:50.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f40e406ace0 0x7f40e406d190 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:50.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f40f80d1330 0x7f40f80ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:50.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f40f80b8470 0x7f40f815d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:50.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 --2- 172.21.15.97:0/1437562054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40f80cd660 0x7f40f814d160 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:50.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 -- 172.21.15.97:0/1437562054 >> 172.21.15.97:0/1437562054 conn(0x7f40f806b160 msgr2=0x7f40f80b6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:50.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 -- 172.21.15.97:0/1437562054 shutdown_connections 2023-07-06T03:55:50.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:50.848+0000 7f40fc9fe700 1 -- 172.21.15.97:0/1437562054 wait complete. 2023-07-06T03:55:50.868 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T03:55:51.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.069+0000 7f4c251ff700 1 Processor -- start 2023-07-06T03:55:51.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.069+0000 7f4c251ff700 1 -- start start 2023-07-06T03:55:51.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.071+0000 7f4c251ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c200c9640 0x7f4c200c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:51.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.071+0000 7f4c251ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4c20159060 0x7f4c200ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:51.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.071+0000 7f4c251ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c200cc870 con 0x7f4c201531b0 2023-07-06T03:55:51.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.071+0000 7f4c251ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c200ca950 con 0x7f4c20159060 2023-07-06T03:55:51.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.071+0000 7f4c251ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200c4300 con 0x7f4c200c9640 2023-07-06T03:55:51.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.071+0000 7f4c1ed9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4c201531b0 0x7f4c200c8f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42460/0 (socket says 172.21.15.97:42460) 2023-07-06T03:55:51.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.071+0000 7f4c1ed9d700 1 -- 172.21.15.97:0/937258795 learned_addr learned my addr 172.21.15.97:0/937258795 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:51.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.071+0000 7f4c1f59e700 1 --2- 172.21.15.97:0/937258795 >> v2:172.21.15.97:3300/0 conn(0x7f4c20159060 0x7f4c200ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:51.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.071+0000 7f4c17fff700 1 --2- 172.21.15.97:0/937258795 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c200c9640 0x7f4c200c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:51.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.071+0000 7f4c1e59c700 1 -- 172.21.15.97:0/937258795 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865139131 0 0) 0x7f4c200cc870 con 0x7f4c201531b0 2023-07-06T03:55:51.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.071+0000 7f4c1e59c700 1 -- 172.21.15.97:0/937258795 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c0c0035b0 con 0x7f4c201531b0 2023-07-06T03:55:51.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.072+0000 7f4c1f59e700 1 -- 172.21.15.97:0/937258795 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c200c9640 msgr2=0x7f4c200c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:51.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.072+0000 7f4c1f59e700 1 --2- 172.21.15.97:0/937258795 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c200c9640 0x7f4c200c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:51.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.072+0000 7f4c1f59e700 1 -- 172.21.15.97:0/937258795 >> v1:172.21.15.97:6789/0 conn(0x7f4c201531b0 legacy=0x7f4c200c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:51.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.072+0000 7f4c1f59e700 1 -- 172.21.15.97:0/937258795 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200c45e0 con 0x7f4c20159060 2023-07-06T03:55:51.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.072+0000 7f4c1f59e700 1 --2- 172.21.15.97:0/937258795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c20159060 0x7f4c200ad5b0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f4c18009d90 tx=0x7f4c1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5049075e13e2941f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:51.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.072+0000 7f4c1e59c700 1 -- 172.21.15.97:0/937258795 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1800e070 con 0x7f4c20159060 2023-07-06T03:55:51.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.072+0000 7f4c1e59c700 1 -- 172.21.15.97:0/937258795 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1800bb20 con 0x7f4c20159060 2023-07-06T03:55:51.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.072+0000 7f4c1e59c700 1 -- 172.21.15.97:0/937258795 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18012430 con 0x7f4c20159060 2023-07-06T03:55:51.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.073+0000 7f4c251ff700 1 -- 172.21.15.97:0/937258795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c20159060 msgr2=0x7f4c200ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:51.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.073+0000 7f4c251ff700 1 --2- 172.21.15.97:0/937258795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c20159060 0x7f4c200ad5b0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f4c18009d90 tx=0x7f4c1800b330 comp rx=0 tx=0).stop 2023-07-06T03:55:51.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.073+0000 7f4c251ff700 1 -- 172.21.15.97:0/937258795 shutdown_connections 2023-07-06T03:55:51.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.073+0000 7f4c251ff700 1 --2- 172.21.15.97:0/937258795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c20159060 0x7f4c200ad5b0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:51.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.073+0000 7f4c251ff700 1 --2- 172.21.15.97:0/937258795 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c200c9640 0x7f4c200c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:51.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.073+0000 7f4c251ff700 1 -- 172.21.15.97:0/937258795 >> 172.21.15.97:0/937258795 conn(0x7f4c200b0510 msgr2=0x7f4c200b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:51.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.073+0000 7f4c251ff700 1 -- 172.21.15.97:0/937258795 shutdown_connections 2023-07-06T03:55:51.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.073+0000 7f4c251ff700 1 -- 172.21.15.97:0/937258795 wait complete. 2023-07-06T03:55:51.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.074+0000 7f4c251ff700 1 Processor -- start 2023-07-06T03:55:51.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.075+0000 7f4c251ff700 1 -- start start 2023-07-06T03:55:51.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.075+0000 7f4c251ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c200c9640 0x7f4c20070670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:51.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.075+0000 7f4c251ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c201531b0 0x7f4c200c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:51.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.075+0000 7f4c251ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c20159060 0x7f4c200cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:51.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.075+0000 7f4c251ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2014f360 con 0x7f4c201531b0 2023-07-06T03:55:51.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.075+0000 7f4c251ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2006d690 con 0x7f4c200c9640 2023-07-06T03:55:51.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.075+0000 7f4c251ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2014f080 con 0x7f4c20159060 2023-07-06T03:55:51.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.076+0000 7f4c17fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c201531b0 0x7f4c200c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:51.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.076+0000 7f4c17fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c201531b0 0x7f4c200c59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59898/0 (socket says 172.21.15.97:59898) 2023-07-06T03:55:51.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.076+0000 7f4c17fff700 1 -- 172.21.15.97:0/4052031425 learned_addr learned my addr 172.21.15.97:0/4052031425 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:51.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.076+0000 7f4c1ed9d700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c200c9640 0x7f4c20070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:51.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.076+0000 7f4c1f59e700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c20159060 0x7f4c200cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:51.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.076+0000 7f4c17fff700 1 -- 172.21.15.97:0/4052031425 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c20159060 msgr2=0x7f4c200cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:51.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.076+0000 7f4c17fff700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c20159060 0x7f4c200cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:51.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.076+0000 7f4c17fff700 1 -- 172.21.15.97:0/4052031425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c200c9640 msgr2=0x7f4c20070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:51.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.076+0000 7f4c17fff700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c200c9640 0x7f4c20070670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:51.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.076+0000 7f4c17fff700 1 -- 172.21.15.97:0/4052031425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c18009a70 con 0x7f4c201531b0 2023-07-06T03:55:51.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.076+0000 7f4c17fff700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c201531b0 0x7f4c200c59b0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800e7e0 tx=0x7f4c0800eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:51.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.077+0000 7f4c177fe700 1 -- 172.21.15.97:0/4052031425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08013610 con 0x7f4c201531b0 2023-07-06T03:55:51.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.077+0000 7f4c177fe700 1 -- 172.21.15.97:0/4052031425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c080137b0 con 0x7f4c201531b0 2023-07-06T03:55:51.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.077+0000 7f4c251ff700 1 -- 172.21.15.97:0/4052031425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2006fb90 con 0x7f4c201531b0 2023-07-06T03:55:51.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.077+0000 7f4c177fe700 1 -- 172.21.15.97:0/4052031425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08013b60 con 0x7f4c201531b0 2023-07-06T03:55:51.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.077+0000 7f4c251ff700 1 -- 172.21.15.97:0/4052031425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c20070150 con 0x7f4c201531b0 2023-07-06T03:55:51.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.078+0000 7f4c251ff700 1 -- 172.21.15.97:0/4052031425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2006f9f0 con 0x7f4c201531b0 2023-07-06T03:55:51.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.079+0000 7f4c177fe700 1 -- 172.21.15.97:0/4052031425 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c08013cc0 con 0x7f4c201531b0 2023-07-06T03:55:51.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.080+0000 7f4c177fe700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c0006ace0 0x7f4c0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:51.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.080+0000 7f4c177fe700 1 -- 172.21.15.97:0/4052031425 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4c080892c0 con 0x7f4c201531b0 2023-07-06T03:55:51.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.080+0000 7f4c1ed9d700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c0006ace0 0x7f4c0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:51.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.080+0000 7f4c1ed9d700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c0006ace0 0x7f4c0006d190 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4c100011e0 tx=0x7f4c10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.084+0000 7f4c177fe700 1 -- 172.21.15.97:0/4052031425 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4c2006f9f0 con 0x7f4c201531b0 2023-07-06T03:55:51.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:51.280+0000 7f4c251ff700 1 -- 172.21.15.97:0/4052031425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4c2006f9f0 con 0x7f4c201531b0 2023-07-06T03:55:52.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.021+0000 7f4c177fe700 1 -- 172.21.15.97:0/4052031425 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4c2006f9f0 con 0x7f4c201531b0 2023-07-06T03:55:52.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 -- 172.21.15.97:0/4052031425 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c0006ace0 msgr2=0x7f4c0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:52.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c0006ace0 0x7f4c0006d190 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4c100011e0 tx=0x7f4c10009040 comp rx=0 tx=0).stop 2023-07-06T03:55:52.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 -- 172.21.15.97:0/4052031425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c201531b0 msgr2=0x7f4c200c59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:52.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c201531b0 0x7f4c200c59b0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800e7e0 tx=0x7f4c0800eba0 comp rx=0 tx=0).stop 2023-07-06T03:55:52.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 -- 172.21.15.97:0/4052031425 shutdown_connections 2023-07-06T03:55:52.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c0006ace0 0x7f4c0006d190 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:52.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c20159060 0x7f4c200cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:52.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c200c9640 0x7f4c20070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:52.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 --2- 172.21.15.97:0/4052031425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c201531b0 0x7f4c200c59b0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:52.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 -- 172.21.15.97:0/4052031425 >> 172.21.15.97:0/4052031425 conn(0x7f4c200b0510 msgr2=0x7f4c2015c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:52.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 -- 172.21.15.97:0/4052031425 shutdown_connections 2023-07-06T03:55:52.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.025+0000 7f4c251ff700 1 -- 172.21.15.97:0/4052031425 wait complete. 2023-07-06T03:55:52.028 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-07-06T03:55:52.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T03:55:52.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.248+0000 7f34fb9ff700 1 Processor -- start 2023-07-06T03:55:52.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.249+0000 7f34fb9ff700 1 -- start start 2023-07-06T03:55:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.250+0000 7f34fb9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f40b85f0 0x7f34f406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.250+0000 7f34fb9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f34f40cefa0 0x7f34f40b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.250+0000 7f34fb9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f40cd330 con 0x7f34f40b4ca0 2023-07-06T03:55:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.250+0000 7f34fb9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f40b89c0 con 0x7f34f40cefa0 2023-07-06T03:55:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.250+0000 7f34fb9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f40b5120 con 0x7f34f40b85f0 2023-07-06T03:55:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.250+0000 7f34f9f9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f34f40b4ca0 0x7f34f415ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42472/0 (socket says 172.21.15.97:42472) 2023-07-06T03:55:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.250+0000 7f34f9f9c700 1 -- 172.21.15.97:0/1693010044 learned_addr learned my addr 172.21.15.97:0/1693010044 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:52.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.250+0000 7f34f979b700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f40b85f0 0x7f34f406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:52.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.250+0000 7f34f8f9a700 1 --2- 172.21.15.97:0/1693010044 >> v2:172.21.15.146:3300/0 conn(0x7f34f40cefa0 0x7f34f40b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:52.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.250+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114798908 0 0) 0x7f34f40cd330 con 0x7f34f40b4ca0 2023-07-06T03:55:52.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e00035b0 con 0x7f34f40b4ca0 2023-07-06T03:55:52.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1716476698 0 0) 0x7f34e00035b0 con 0x7f34f40b4ca0 2023-07-06T03:55:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f40b85f0 msgr2=0x7f34f406b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f40b85f0 0x7f34f406b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 >> v2:172.21.15.146:3300/0 conn(0x7f34f40cefa0 msgr2=0x7f34f40b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 --2- 172.21.15.97:0/1693010044 >> v2:172.21.15.146:3300/0 conn(0x7f34f40cefa0 0x7f34f40b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f40d02a0 con 0x7f34f40b4ca0 2023-07-06T03:55:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f34ec0019f0 con 0x7f34f40b4ca0 2023-07-06T03:55:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 >> v1:172.21.15.97:6789/0 conn(0x7f34f40b4ca0 legacy=0x7f34f415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0003a00 0x7f34e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34e0004430 0x7f34e00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:52.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f40b85f0 0x7f34e0004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:52.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f40b89c0 con 0x7f34e0004430 2023-07-06T03:55:52.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f40b5120 con 0x7f34e0003a00 2023-07-06T03:55:52.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e000ba10 con 0x7f34f40b85f0 2023-07-06T03:55:52.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34f8f9a700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34e0004430 0x7f34e00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:52.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34f979b700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0003a00 0x7f34e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:52.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.251+0000 7f34f9f9c700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f40b85f0 0x7f34e0004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:52.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.252+0000 7f34f8f9a700 1 -- 172.21.15.97:0/1693010044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f40b85f0 msgr2=0x7f34e0004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:52.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.252+0000 7f34f8f9a700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34f40b85f0 0x7f34e0004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:52.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.252+0000 7f34f8f9a700 1 -- 172.21.15.97:0/1693010044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0003a00 msgr2=0x7f34e0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:52.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.252+0000 7f34f8f9a700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e0003a00 0x7f34e0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:52.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.252+0000 7f34f8f9a700 1 -- 172.21.15.97:0/1693010044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e00035b0 con 0x7f34e0004430 2023-07-06T03:55:52.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.252+0000 7f34f8f9a700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34e0004430 0x7f34e00048f0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009b60 tx=0x7f34e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c788ab471d2a34af server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:52.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.252+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e400e6c0 con 0x7f34e0004430 2023-07-06T03:55:52.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.252+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e400e820 con 0x7f34e0004430 2023-07-06T03:55:52.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.252+0000 7f34ebfff700 1 -- 172.21.15.97:0/1693010044 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e400eb90 con 0x7f34e0004430 2023-07-06T03:55:52.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.253+0000 7f34fb9ff700 1 -- 172.21.15.97:0/1693010044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34e0004430 msgr2=0x7f34e00048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:52.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.253+0000 7f34fb9ff700 1 --2- 172.21.15.97:0/1693010044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34e0004430 0x7f34e00048f0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009b60 tx=0x7f34e0005880 comp rx=0 tx=0).stop 2023-07-06T03:55:52.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.253+0000 7f34f9f9c700 1 -- 172.21.15.97:0/1693010044 reap_dead start 2023-07-06T03:55:52.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.253+0000 7f34fb9ff700 1 -- 172.21.15.97:0/1693010044 shutdown_connections 2023-07-06T03:55:52.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.253+0000 7f34fb9ff700 1 -- 172.21.15.97:0/1693010044 >> 172.21.15.97:0/1693010044 conn(0x7f34f406a5b0 msgr2=0x7f34f406a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:52.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.253+0000 7f34fb9ff700 1 -- 172.21.15.97:0/1693010044 shutdown_connections 2023-07-06T03:55:52.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.253+0000 7f34fb9ff700 1 -- 172.21.15.97:0/1693010044 wait complete. 2023-07-06T03:55:52.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.255+0000 7f34fb9ff700 1 Processor -- start 2023-07-06T03:55:52.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.255+0000 7f34fb9ff700 1 -- start start 2023-07-06T03:55:52.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34fb9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34e0004430 0x7f34f415edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:52.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34fb9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f40c8c00 0x7f34f4158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:52.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34fb9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f40c9070 0x7f34f4156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:52.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34fb9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f415e960 con 0x7f34f40c8c00 2023-07-06T03:55:52.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34fb9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f415dd50 con 0x7f34f40c9070 2023-07-06T03:55:52.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34fb9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f415e030 con 0x7f34e0004430 2023-07-06T03:55:52.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34f8f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f40c8c00 0x7f34f4158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:52.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34f8f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f40c8c00 0x7f34f4158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59914/0 (socket says 172.21.15.97:59914) 2023-07-06T03:55:52.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34f8f9a700 1 -- 172.21.15.97:0/4194418242 learned_addr learned my addr 172.21.15.97:0/4194418242 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:52.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34f979b700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34e0004430 0x7f34f415edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:52.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34f9f9c700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f40c9070 0x7f34f4156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:52.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34f8f9a700 1 -- 172.21.15.97:0/4194418242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34e0004430 msgr2=0x7f34f415edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:52.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34f8f9a700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34e0004430 0x7f34f415edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:52.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34f8f9a700 1 -- 172.21.15.97:0/4194418242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f40c9070 msgr2=0x7f34f4156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:52.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34f8f9a700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f40c9070 0x7f34f4156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:52.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34f8f9a700 1 -- 172.21.15.97:0/4194418242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f415e310 con 0x7f34f40c8c00 2023-07-06T03:55:52.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.256+0000 7f34f8f9a700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f40c8c00 0x7f34f4158b60 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f34e00056d0 tx=0x7f34e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:52.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.257+0000 7f34e9ffb700 1 -- 172.21.15.97:0/4194418242 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e400e6c0 con 0x7f34f40c8c00 2023-07-06T03:55:52.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.257+0000 7f34e9ffb700 1 -- 172.21.15.97:0/4194418242 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e4003ab0 con 0x7f34f40c8c00 2023-07-06T03:55:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.257+0000 7f34e9ffb700 1 -- 172.21.15.97:0/4194418242 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4003e60 con 0x7f34f40c8c00 2023-07-06T03:55:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.257+0000 7f34fb9ff700 1 -- 172.21.15.97:0/4194418242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f415e5f0 con 0x7f34f40c8c00 2023-07-06T03:55:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.257+0000 7f34fb9ff700 1 -- 172.21.15.97:0/4194418242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f415d990 con 0x7f34f40c8c00 2023-07-06T03:55:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.258+0000 7f34fb9ff700 1 -- 172.21.15.97:0/4194418242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f415ec40 con 0x7f34f40c8c00 2023-07-06T03:55:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.259+0000 7f34e9ffb700 1 -- 172.21.15.97:0/4194418242 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34e4013470 con 0x7f34f40c8c00 2023-07-06T03:55:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.260+0000 7f34e9ffb700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f34d806ace0 0x7f34d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.260+0000 7f34e9ffb700 1 -- 172.21.15.97:0/4194418242 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f34e408ad50 con 0x7f34f40c8c00 2023-07-06T03:55:52.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.260+0000 7f34f979b700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f34d806ace0 0x7f34d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:52.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.261+0000 7f34f979b700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f34d806ace0 0x7f34d806d190 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009ac0 tx=0x7f34f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:52.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.264+0000 7f34e9ffb700 1 -- 172.21.15.97:0/4194418242 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f34e408e030 con 0x7f34f40c8c00 2023-07-06T03:55:52.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:52.457+0000 7f34fb9ff700 1 -- 172.21.15.97:0/4194418242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34f404e340 con 0x7f34f40c8c00 2023-07-06T03:55:53.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.024+0000 7f34e9ffb700 1 -- 172.21.15.97:0/4194418242 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v127) v1 ==== 169+0+0 (secure 0 0 0) 0x7f34e4055ff0 con 0x7f34f40c8c00 2023-07-06T03:55:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.031+0000 7f34fb9ff700 1 -- 172.21.15.97:0/4194418242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f34d806ace0 msgr2=0x7f34d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.031+0000 7f34fb9ff700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f34d806ace0 0x7f34d806d190 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009ac0 tx=0x7f34f0008040 comp rx=0 tx=0).stop 2023-07-06T03:55:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.031+0000 7f34fb9ff700 1 -- 172.21.15.97:0/4194418242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f40c8c00 msgr2=0x7f34f4158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.031+0000 7f34fb9ff700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f40c8c00 0x7f34f4158b60 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f34e00056d0 tx=0x7f34e40047c0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.032+0000 7f34fb9ff700 1 -- 172.21.15.97:0/4194418242 shutdown_connections 2023-07-06T03:55:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.032+0000 7f34fb9ff700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f34d806ace0 0x7f34d806d190 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.032+0000 7f34fb9ff700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34f40c9070 0x7f34f4156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.032+0000 7f34fb9ff700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34e0004430 0x7f34f415edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.032+0000 7f34fb9ff700 1 --2- 172.21.15.97:0/4194418242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f40c8c00 0x7f34f4158b60 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.032+0000 7f34fb9ff700 1 -- 172.21.15.97:0/4194418242 >> 172.21.15.97:0/4194418242 conn(0x7f34f406a5b0 msgr2=0x7f34f40b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.034+0000 7f34fb9ff700 1 -- 172.21.15.97:0/4194418242 shutdown_connections 2023-07-06T03:55:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.034+0000 7f34fb9ff700 1 -- 172.21.15.97:0/4194418242 wait complete. 2023-07-06T03:55:53.043 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-07-06T03:55:53.066 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2023-07-06T03:55:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.259+0000 7f73435ff700 1 Processor -- start 2023-07-06T03:55:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.259+0000 7f73435ff700 1 -- start start 2023-07-06T03:55:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f73435ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f733c06b360 0x7f733c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f73435ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f733c0d2210 0x7f733c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f73435ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733c0cff90 con 0x7f733c15c610 2023-07-06T03:55:53.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f73435ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733c0d0780 con 0x7f733c0d2210 2023-07-06T03:55:53.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f73435ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733c15c9e0 con 0x7f733c06b360 2023-07-06T03:55:53.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f7341b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f733c0d2210 0x7f733c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f734139b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f733c15c610 0x7f733c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42486/0 (socket says 172.21.15.97:42486) 2023-07-06T03:55:53.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f734139b700 1 -- 172.21.15.97:0/3638463805 learned_addr learned my addr 172.21.15.97:0/3638463805 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:53.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f7341b9c700 1 -- 172.21.15.97:0/3638463805 >> v2:172.21.15.146:3300/0 conn(0x7f733c06b360 msgr2=0x7f733c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:53.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f7341b9c700 1 --2- 172.21.15.97:0/3638463805 >> v2:172.21.15.146:3300/0 conn(0x7f733c06b360 0x7f733c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f7341b9c700 1 -- 172.21.15.97:0/3638463805 >> v1:172.21.15.97:6789/0 conn(0x7f733c15c610 legacy=0x7f733c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.261+0000 7f7341b9c700 1 -- 172.21.15.97:0/3638463805 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c0b5650 con 0x7f733c0d2210 2023-07-06T03:55:53.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f7341b9c700 1 --2- 172.21.15.97:0/3638463805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f733c0d2210 0x7f733c0d1310 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f733c06a590 tx=0x7f733400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27482d934a6a88f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:53.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f7333fff700 1 -- 172.21.15.97:0/3638463805 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733400e070 con 0x7f733c0d2210 2023-07-06T03:55:53.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f7333fff700 1 -- 172.21.15.97:0/3638463805 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733400bb90 con 0x7f733c0d2210 2023-07-06T03:55:53.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f7333fff700 1 -- 172.21.15.97:0/3638463805 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334012430 con 0x7f733c0d2210 2023-07-06T03:55:53.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f73435ff700 1 -- 172.21.15.97:0/3638463805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f733c0d2210 msgr2=0x7f733c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f73435ff700 1 --2- 172.21.15.97:0/3638463805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f733c0d2210 0x7f733c0d1310 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f733c06a590 tx=0x7f733400b330 comp rx=0 tx=0).stop 2023-07-06T03:55:53.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f73435ff700 1 -- 172.21.15.97:0/3638463805 shutdown_connections 2023-07-06T03:55:53.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f73435ff700 1 --2- 172.21.15.97:0/3638463805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f733c0d2210 0x7f733c0d1310 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f73435ff700 1 --2- 172.21.15.97:0/3638463805 >> v2:172.21.15.146:3300/0 conn(0x7f733c06b360 0x7f733c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f73435ff700 1 -- 172.21.15.97:0/3638463805 >> 172.21.15.97:0/3638463805 conn(0x7f733c06a6c0 msgr2=0x7f733c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:53.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f73435ff700 1 -- 172.21.15.97:0/3638463805 shutdown_connections 2023-07-06T03:55:53.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.262+0000 7f73435ff700 1 -- 172.21.15.97:0/3638463805 wait complete. 2023-07-06T03:55:53.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.264+0000 7f73435ff700 1 Processor -- start 2023-07-06T03:55:53.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.264+0000 7f73435ff700 1 -- start start 2023-07-06T03:55:53.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.265+0000 7f73435ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f733c06b360 0x7f733c0c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.265+0000 7f73435ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f733c0d2210 0x7f733c1f1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.265+0000 7f73435ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f733c15c610 0x7f733c0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.265+0000 7f73435ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0c9b10 con 0x7f733c06b360 2023-07-06T03:55:53.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.265+0000 7f73435ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0caff0 con 0x7f733c0d2210 2023-07-06T03:55:53.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.265+0000 7f73435ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0cb2d0 con 0x7f733c15c610 2023-07-06T03:55:53.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.265+0000 7f734139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f733c06b360 0x7f733c0c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.265+0000 7f734139b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f733c06b360 0x7f733c0c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59938/0 (socket says 172.21.15.97:59938) 2023-07-06T03:55:53.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.265+0000 7f734139b700 1 -- 172.21.15.97:0/2047797393 learned_addr learned my addr 172.21.15.97:0/2047797393 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:53.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f7340b9a700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f733c0d2210 0x7f733c1f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f7341b9c700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f733c15c610 0x7f733c0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f734139b700 1 -- 172.21.15.97:0/2047797393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f733c15c610 msgr2=0x7f733c0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f734139b700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f733c15c610 0x7f733c0cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f734139b700 1 -- 172.21.15.97:0/2047797393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f733c0d2210 msgr2=0x7f733c1f1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f734139b700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f733c0d2210 0x7f733c1f1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f734139b700 1 -- 172.21.15.97:0/2047797393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7334009a70 con 0x7f733c06b360 2023-07-06T03:55:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f7340b9a700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f733c0d2210 0x7f733c1f1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f7341b9c700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f733c15c610 0x7f733c0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f734139b700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f733c06b360 0x7f733c0c95d0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f733800c480 tx=0x7f733800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f7331ffb700 1 -- 172.21.15.97:0/2047797393 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338005d70 con 0x7f733c06b360 2023-07-06T03:55:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f7331ffb700 1 -- 172.21.15.97:0/2047797393 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73380070d0 con 0x7f733c06b360 2023-07-06T03:55:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f7331ffb700 1 -- 172.21.15.97:0/2047797393 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338007440 con 0x7f733c06b360 2023-07-06T03:55:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f73435ff700 1 -- 172.21.15.97:0/2047797393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c0cb5b0 con 0x7f733c06b360 2023-07-06T03:55:53.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.266+0000 7f73435ff700 1 -- 172.21.15.97:0/2047797393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c0cbb70 con 0x7f733c06b360 2023-07-06T03:55:53.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.267+0000 7f73435ff700 1 -- 172.21.15.97:0/2047797393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733c04e340 con 0x7f733c06b360 2023-07-06T03:55:53.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.268+0000 7f7331ffb700 1 -- 172.21.15.97:0/2047797393 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f73380075a0 con 0x7f733c06b360 2023-07-06T03:55:53.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.269+0000 7f7331ffb700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 0x7f732006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.269+0000 7f7331ffb700 1 -- 172.21.15.97:0/2047797393 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f73380885d0 con 0x7f733c06b360 2023-07-06T03:55:53.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.269+0000 7f7340b9a700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 0x7f732006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.270+0000 7f7340b9a700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 0x7f732006d190 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f733c0ccad0 tx=0x7f732c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:53.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.273+0000 7f7331ffb700 1 -- 172.21.15.97:0/2047797393 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7338053a40 con 0x7f733c06b360 2023-07-06T03:55:53.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.472+0000 7f73435ff700 1 -- 172.21.15.97:0/2047797393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) v1 -- 0x7f733c0cd040 con 0x7f733c06b360 2023-07-06T03:55:53.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.476+0000 7f7331ffb700 1 -- 172.21.15.97:0/2047797393 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v59) v1 ==== 111+0+0 (secure 0 0 0) 0x7f7338057060 con 0x7f733c06b360 2023-07-06T03:55:53.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 -- 172.21.15.97:0/2047797393 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 msgr2=0x7f732006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 0x7f732006d190 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f733c0ccad0 tx=0x7f732c008040 comp rx=0 tx=0).stop 2023-07-06T03:55:53.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 -- 172.21.15.97:0/2047797393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f733c06b360 msgr2=0x7f733c0c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f733c06b360 0x7f733c0c95d0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f733800c480 tx=0x7f733800c840 comp rx=0 tx=0).stop 2023-07-06T03:55:53.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 -- 172.21.15.97:0/2047797393 shutdown_connections 2023-07-06T03:55:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f732006ace0 0x7f732006d190 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f733c15c610 0x7f733c0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f733c06b360 0x7f733c0c95d0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 --2- 172.21.15.97:0/2047797393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f733c0d2210 0x7f733c1f1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 -- 172.21.15.97:0/2047797393 >> 172.21.15.97:0/2047797393 conn(0x7f733c06a6c0 msgr2=0x7f733c15b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 -- 172.21.15.97:0/2047797393 shutdown_connections 2023-07-06T03:55:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.478+0000 7f73435ff700 1 -- 172.21.15.97:0/2047797393 wait complete. 2023-07-06T03:55:53.499 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2023-07-06T03:55:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.692+0000 7f33c95ff700 1 Processor -- start 2023-07-06T03:55:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.692+0000 7f33c95ff700 1 -- start start 2023-07-06T03:55:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.693+0000 7f33c95ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33c415c610 0x7f33c40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.693+0000 7f33c95ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c40cff90 con 0x7f33c40d2210 2023-07-06T03:55:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.693+0000 7f33c95ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c40b5650 con 0x7f33c406b360 2023-07-06T03:55:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.693+0000 7f33c95ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40d0780 con 0x7f33c415c610 2023-07-06T03:55:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.693+0000 7f33c37fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f33c40d2210 0x7f33c40d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42488/0 (socket says 172.21.15.97:42488) 2023-07-06T03:55:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.693+0000 7f33c37fe700 1 -- 172.21.15.97:0/110930756 learned_addr learned my addr 172.21.15.97:0/110930756 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426897824 0 0) 0x7f33c40cff90 con 0x7f33c40d2210 2023-07-06T03:55:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c2ffd700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33c415c610 0x7f33c40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33ac0035b0 con 0x7f33c40d2210 2023-07-06T03:55:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2173224630 0 0) 0x7f33ac0035b0 con 0x7f33c40d2210 2023-07-06T03:55:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33c415c610 msgr2=0x7f33c40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33c415c610 0x7f33c40b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 >> v1:172.21.15.146:6789/0 conn(0x7f33c406b360 legacy=0x7f33c40b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c40b5400 con 0x7f33c40d2210 2023-07-06T03:55:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f33bc0019f0 con 0x7f33c40d2210 2023-07-06T03:55:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 >> v1:172.21.15.97:6789/0 conn(0x7f33c40d2210 legacy=0x7f33c40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33ac003a30 0x7f33ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33c415c610 0x7f33ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac004d90 0x7f33ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40d0780 con 0x7f33ac003a30 2023-07-06T03:55:53.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac00ba10 con 0x7f33ac004d90 2023-07-06T03:55:53.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac00bba0 con 0x7f33c415c610 2023-07-06T03:55:53.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c2ffd700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33ac003a30 0x7f33ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.694+0000 7f33c27fc700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33c415c610 0x7f33ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.695+0000 7f33c37fe700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac004d90 0x7f33ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.695+0000 7f33c2ffd700 1 -- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33c415c610 msgr2=0x7f33ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:53.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.695+0000 7f33c2ffd700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33c415c610 0x7f33ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.695+0000 7f33c2ffd700 1 -- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac004d90 msgr2=0x7f33ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:53.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.695+0000 7f33c2ffd700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33ac004d90 0x7f33ac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.695+0000 7f33c2ffd700 1 -- 172.21.15.97:0/110930756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac0035b0 con 0x7f33ac003a30 2023-07-06T03:55:53.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.695+0000 7f33c27fc700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33c415c610 0x7f33ac004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:53.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.695+0000 7f33c2ffd700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33ac003a30 0x7f33ac003e00 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009b60 tx=0x7f33ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99488b1e6e2b3b17 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:53.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.695+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400e6c0 con 0x7f33ac003a30 2023-07-06T03:55:53.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.695+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b400e820 con 0x7f33ac003a30 2023-07-06T03:55:53.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.695+0000 7f33c1ffb700 1 -- 172.21.15.97:0/110930756 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400eb90 con 0x7f33ac003a30 2023-07-06T03:55:53.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.696+0000 7f33c95ff700 1 -- 172.21.15.97:0/110930756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33ac003a30 msgr2=0x7f33ac003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.696+0000 7f33c95ff700 1 --2- 172.21.15.97:0/110930756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33ac003a30 0x7f33ac003e00 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009b60 tx=0x7f33ac005880 comp rx=0 tx=0).stop 2023-07-06T03:55:53.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.696+0000 7f33c37fe700 1 -- 172.21.15.97:0/110930756 reap_dead start 2023-07-06T03:55:53.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.696+0000 7f33c95ff700 1 -- 172.21.15.97:0/110930756 shutdown_connections 2023-07-06T03:55:53.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.696+0000 7f33c95ff700 1 -- 172.21.15.97:0/110930756 >> 172.21.15.97:0/110930756 conn(0x7f33c406a6c0 msgr2=0x7f33c406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:53.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.696+0000 7f33c95ff700 1 -- 172.21.15.97:0/110930756 shutdown_connections 2023-07-06T03:55:53.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.696+0000 7f33c95ff700 1 -- 172.21.15.97:0/110930756 wait complete. 2023-07-06T03:55:53.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.698+0000 7f33c95ff700 1 Processor -- start 2023-07-06T03:55:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.698+0000 7f33c95ff700 1 -- start start 2023-07-06T03:55:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c95ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33ac003a30 0x7f33c40b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c95ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40b6270 0x7f33c41f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c95ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c40b66e0 0x7f33c41f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c95ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c41f5960 con 0x7f33c40b6270 2023-07-06T03:55:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c95ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c415e8c0 con 0x7f33c40b66e0 2023-07-06T03:55:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c95ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c415eb60 con 0x7f33ac003a30 2023-07-06T03:55:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40b6270 0x7f33c41f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40b6270 0x7f33c41f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59956/0 (socket says 172.21.15.97:59956) 2023-07-06T03:55:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c27fc700 1 -- 172.21.15.97:0/1778697819 learned_addr learned my addr 172.21.15.97:0/1778697819 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c2ffd700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33ac003a30 0x7f33c40b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c37fe700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c40b66e0 0x7f33c41f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c27fc700 1 -- 172.21.15.97:0/1778697819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33ac003a30 msgr2=0x7f33c40b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c27fc700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33ac003a30 0x7f33c40b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c27fc700 1 -- 172.21.15.97:0/1778697819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c40b66e0 msgr2=0x7f33c41f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c27fc700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c40b66e0 0x7f33c41f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c27fc700 1 -- 172.21.15.97:0/1778697819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c415ee40 con 0x7f33c40b6270 2023-07-06T03:55:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.699+0000 7f33c37fe700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c40b66e0 0x7f33c41f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.700+0000 7f33c27fc700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40b6270 0x7f33c41f4ee0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f33b800e780 tx=0x7f33b800eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.700+0000 7f33ab7fe700 1 -- 172.21.15.97:0/1778697819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b80052e0 con 0x7f33c40b6270 2023-07-06T03:55:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.700+0000 7f33ab7fe700 1 -- 172.21.15.97:0/1778697819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b8005440 con 0x7f33c40b6270 2023-07-06T03:55:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.700+0000 7f33c95ff700 1 -- 172.21.15.97:0/1778697819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c415f120 con 0x7f33c40b6270 2023-07-06T03:55:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.700+0000 7f33ab7fe700 1 -- 172.21.15.97:0/1778697819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b80038c0 con 0x7f33c40b6270 2023-07-06T03:55:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.700+0000 7f33c95ff700 1 -- 172.21.15.97:0/1778697819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c40c6860 con 0x7f33c40b6270 2023-07-06T03:55:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.701+0000 7f33c95ff700 1 -- 172.21.15.97:0/1778697819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c415f640 con 0x7f33c40b6270 2023-07-06T03:55:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.702+0000 7f33ab7fe700 1 -- 172.21.15.97:0/1778697819 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f33b8003a20 con 0x7f33c40b6270 2023-07-06T03:55:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.703+0000 7f33ab7fe700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f33b006ace0 0x7f33b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.703+0000 7f33ab7fe700 1 -- 172.21.15.97:0/1778697819 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f33b8088fe0 con 0x7f33c40b6270 2023-07-06T03:55:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.703+0000 7f33c2ffd700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f33b006ace0 0x7f33b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.704+0000 7f33c2ffd700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f33b006ace0 0x7f33b006d190 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f33b40090b0 tx=0x7f33b4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.707+0000 7f33ab7fe700 1 -- 172.21.15.97:0/1778697819 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f33c415f640 con 0x7f33c40b6270 2023-07-06T03:55:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.904+0000 7f33c95ff700 1 -- 172.21.15.97:0/1778697819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) v1 -- 0x7f33c415f640 con 0x7f33c40b6270 2023-07-06T03:55:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.909+0000 7f33ab7fe700 1 -- 172.21.15.97:0/1778697819 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v60) v1 ==== 102+0+0 (secure 0 0 0) 0x7f33c415f640 con 0x7f33c40b6270 2023-07-06T03:55:53.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 -- 172.21.15.97:0/1778697819 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f33b006ace0 msgr2=0x7f33b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f33b006ace0 0x7f33b006d190 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f33b40090b0 tx=0x7f33b4008030 comp rx=0 tx=0).stop 2023-07-06T03:55:53.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 -- 172.21.15.97:0/1778697819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40b6270 msgr2=0x7f33c41f4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:53.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40b6270 0x7f33c41f4ee0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f33b800e780 tx=0x7f33b800eb40 comp rx=0 tx=0).stop 2023-07-06T03:55:53.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 -- 172.21.15.97:0/1778697819 shutdown_connections 2023-07-06T03:55:53.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f33b006ace0 0x7f33b006d190 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33c40b66e0 0x7f33c41f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f33ac003a30 0x7f33c40b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 --2- 172.21.15.97:0/1778697819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40b6270 0x7f33c41f4ee0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:53.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 -- 172.21.15.97:0/1778697819 >> 172.21.15.97:0/1778697819 conn(0x7f33c406a6c0 msgr2=0x7f33c415b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:53.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 -- 172.21.15.97:0/1778697819 shutdown_connections 2023-07-06T03:55:53.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:53.911+0000 7f33c95ff700 1 -- 172.21.15.97:0/1778697819 wait complete. 2023-07-06T03:55:53.932 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2023-07-06T03:55:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.127+0000 7f57427ff700 1 Processor -- start 2023-07-06T03:55:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.127+0000 7f57427ff700 1 -- start start 2023-07-06T03:55:54.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.128+0000 7f57427ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f573c159070 0x7f573c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.129+0000 7f57427ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c0ca260 con 0x7f573c0ccbd0 2023-07-06T03:55:54.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.129+0000 7f57427ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c0c99c0 con 0x7f573c0c3cd0 2023-07-06T03:55:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.129+0000 7f57427ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573c0c9ca0 con 0x7f573c159070 2023-07-06T03:55:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.129+0000 7f573bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f573c159070 0x7f573c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.129+0000 7f5740d9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f573c0ccbd0 0x7f573c0cb0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42496/0 (socket says 172.21.15.97:42496) 2023-07-06T03:55:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.129+0000 7f573bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f573c159070 0x7f573c0c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59964/0 (socket says 172.21.15.97:59964) 2023-07-06T03:55:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.129+0000 7f5740d9c700 1 -- 172.21.15.97:0/2335523899 learned_addr learned my addr 172.21.15.97:0/2335523899 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.129+0000 7f573bfff700 1 -- 172.21.15.97:0/2335523899 >> v1:172.21.15.146:6789/0 conn(0x7f573c0c3cd0 legacy=0x7f573c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.129+0000 7f573bfff700 1 -- 172.21.15.97:0/2335523899 >> v1:172.21.15.97:6789/0 conn(0x7f573c0ccbd0 legacy=0x7f573c0cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.129+0000 7f573bfff700 1 -- 172.21.15.97:0/2335523899 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0c9f80 con 0x7f573c159070 2023-07-06T03:55:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.130+0000 7f573bfff700 1 --2- 172.21.15.97:0/2335523899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f573c159070 0x7f573c0c8f40 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f572c00bae0 tx=0x7f572c00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e82d99b2730ddd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:54.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.130+0000 7f573affd700 1 -- 172.21.15.97:0/2335523899 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00dd30 con 0x7f573c159070 2023-07-06T03:55:54.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.130+0000 7f573affd700 1 -- 172.21.15.97:0/2335523899 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c00de90 con 0x7f573c159070 2023-07-06T03:55:54.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.130+0000 7f573affd700 1 -- 172.21.15.97:0/2335523899 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c010690 con 0x7f573c159070 2023-07-06T03:55:54.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.130+0000 7f57427ff700 1 -- 172.21.15.97:0/2335523899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f573c159070 msgr2=0x7f573c0c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.130+0000 7f57427ff700 1 --2- 172.21.15.97:0/2335523899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f573c159070 0x7f573c0c8f40 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f572c00bae0 tx=0x7f572c00bdf0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.130+0000 7f57427ff700 1 -- 172.21.15.97:0/2335523899 shutdown_connections 2023-07-06T03:55:54.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.130+0000 7f57427ff700 1 --2- 172.21.15.97:0/2335523899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f573c159070 0x7f573c0c8f40 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.130+0000 7f57427ff700 1 -- 172.21.15.97:0/2335523899 >> 172.21.15.97:0/2335523899 conn(0x7f573c06a060 msgr2=0x7f573c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:54.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.130+0000 7f57427ff700 1 -- 172.21.15.97:0/2335523899 shutdown_connections 2023-07-06T03:55:54.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.130+0000 7f57427ff700 1 -- 172.21.15.97:0/2335523899 wait complete. 2023-07-06T03:55:54.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.132+0000 7f57427ff700 1 Processor -- start 2023-07-06T03:55:54.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.132+0000 7f57427ff700 1 -- start start 2023-07-06T03:55:54.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.133+0000 7f57427ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f573c0c3cd0 0x7f573c14e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.133+0000 7f57427ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f573c0ccbd0 0x7f573c153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.133+0000 7f57427ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f573c159070 0x7f573c15bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.133+0000 7f57427ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c152500 con 0x7f573c159070 2023-07-06T03:55:54.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.133+0000 7f57427ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c15b8f0 con 0x7f573c0c3cd0 2023-07-06T03:55:54.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.133+0000 7f57427ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c1532f0 con 0x7f573c0ccbd0 2023-07-06T03:55:54.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.133+0000 7f5740d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f573c159070 0x7f573c15bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.133+0000 7f5740d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f573c159070 0x7f573c15bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59972/0 (socket says 172.21.15.97:59972) 2023-07-06T03:55:54.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.133+0000 7f5740d9c700 1 -- 172.21.15.97:0/280958252 learned_addr learned my addr 172.21.15.97:0/280958252 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:54.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f573bfff700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f573c0c3cd0 0x7f573c14e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f573b7fe700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f573c0ccbd0 0x7f573c153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f5740d9c700 1 -- 172.21.15.97:0/280958252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f573c0ccbd0 msgr2=0x7f573c153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f5740d9c700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f573c0ccbd0 0x7f573c153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f5740d9c700 1 -- 172.21.15.97:0/280958252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f573c0c3cd0 msgr2=0x7f573c14e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f5740d9c700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f573c0c3cd0 0x7f573c14e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f5740d9c700 1 -- 172.21.15.97:0/280958252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c00b740 con 0x7f573c159070 2023-07-06T03:55:54.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f573b7fe700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f573c0ccbd0 0x7f573c153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:54.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f5740d9c700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f573c159070 0x7f573c15bdf0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f573400cbc0 tx=0x7f573400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:54.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f5738ff9700 1 -- 172.21.15.97:0/280958252 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400aaa0 con 0x7f573c159070 2023-07-06T03:55:54.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f5738ff9700 1 -- 172.21.15.97:0/280958252 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573400ac00 con 0x7f573c159070 2023-07-06T03:55:54.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f57427ff700 1 -- 172.21.15.97:0/280958252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573c14ea70 con 0x7f573c159070 2023-07-06T03:55:54.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.134+0000 7f5738ff9700 1 -- 172.21.15.97:0/280958252 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734012400 con 0x7f573c159070 2023-07-06T03:55:54.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.135+0000 7f57427ff700 1 -- 172.21.15.97:0/280958252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573c152ca0 con 0x7f573c159070 2023-07-06T03:55:54.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.136+0000 7f57427ff700 1 -- 172.21.15.97:0/280958252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573c04e340 con 0x7f573c159070 2023-07-06T03:55:54.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.137+0000 7f5738ff9700 1 -- 172.21.15.97:0/280958252 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f573401b3b0 con 0x7f573c159070 2023-07-06T03:55:54.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.138+0000 7f5738ff9700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f572806ace0 0x7f572806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.138+0000 7f5738ff9700 1 -- 172.21.15.97:0/280958252 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5734088d60 con 0x7f573c159070 2023-07-06T03:55:54.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.138+0000 7f573bfff700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f572806ace0 0x7f572806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.139+0000 7f573bfff700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f572806ace0 0x7f572806d190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f572c00b2c0 tx=0x7f572c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:54.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.143+0000 7f5738ff9700 1 -- 172.21.15.97:0/280958252 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5734054140 con 0x7f573c159070 2023-07-06T03:55:54.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.338+0000 7f57427ff700 1 -- 172.21.15.97:0/280958252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) v1 -- 0x7f573c06d220 con 0x7f573c159070 2023-07-06T03:55:54.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.342+0000 7f5738ff9700 1 -- 172.21.15.97:0/280958252 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v61) v1 ==== 107+0+0 (secure 0 0 0) 0x7f5734057760 con 0x7f573c159070 2023-07-06T03:55:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 -- 172.21.15.97:0/280958252 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f572806ace0 msgr2=0x7f572806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f572806ace0 0x7f572806d190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f572c00b2c0 tx=0x7f572c01c040 comp rx=0 tx=0).stop 2023-07-06T03:55:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 -- 172.21.15.97:0/280958252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f573c159070 msgr2=0x7f573c15bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f573c159070 0x7f573c15bdf0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f573400cbc0 tx=0x7f573400ced0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 -- 172.21.15.97:0/280958252 shutdown_connections 2023-07-06T03:55:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f572806ace0 0x7f572806d190 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f573c159070 0x7f573c15bdf0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f573c0c3cd0 0x7f573c14e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 --2- 172.21.15.97:0/280958252 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f573c0ccbd0 0x7f573c153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 -- 172.21.15.97:0/280958252 >> 172.21.15.97:0/280958252 conn(0x7f573c06a060 msgr2=0x7f573c154130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 -- 172.21.15.97:0/280958252 shutdown_connections 2023-07-06T03:55:54.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.345+0000 7f57427ff700 1 -- 172.21.15.97:0/280958252 wait complete. 2023-07-06T03:55:54.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T03:55:54.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.559+0000 7f51a5fff700 1 Processor -- start 2023-07-06T03:55:54.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.559+0000 7f51a5fff700 1 -- start start 2023-07-06T03:55:54.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f51a5fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a006b360 0x7f51a00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f51a5fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a00cff90 con 0x7f51a015c610 2023-07-06T03:55:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f51a5fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a00b5650 con 0x7f51a00d2210 2023-07-06T03:55:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f51a5fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00d0780 con 0x7f51a006b360 2023-07-06T03:55:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f519f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f51a015c610 0x7f51a00b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42500/0 (socket says 172.21.15.97:42500) 2023-07-06T03:55:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f519f7fe700 1 -- 172.21.15.97:0/154469664 learned_addr learned my addr 172.21.15.97:0/154469664 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446555606 0 0) 0x7f51a00cff90 con 0x7f51a015c610 2023-07-06T03:55:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f518c0035b0 con 0x7f51a015c610 2023-07-06T03:55:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f5197fff700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a006b360 0x7f51a00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4107954464 0 0) 0x7f518c0035b0 con 0x7f51a015c610 2023-07-06T03:55:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a006b360 msgr2=0x7f51a00b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.561+0000 7f519effd700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a006b360 0x7f51a00b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 >> v1:172.21.15.146:6789/0 conn(0x7f51a00d2210 legacy=0x7f51a00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a00b5400 con 0x7f51a015c610 2023-07-06T03:55:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f5190001670 con 0x7f51a015c610 2023-07-06T03:55:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 >> v1:172.21.15.97:6789/0 conn(0x7f51a015c610 legacy=0x7f51a00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519effd700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f518c003a30 0x7f518c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519effd700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 0x7f518c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519effd700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a006b360 0x7f518c005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00d0780 con 0x7f518c004850 2023-07-06T03:55:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c00bb40 con 0x7f518c003a30 2023-07-06T03:55:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c00bd00 con 0x7f51a006b360 2023-07-06T03:55:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f5197fff700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 0x7f518c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519f7fe700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f518c003a30 0x7f518c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f519ffff700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a006b360 0x7f518c005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f5197fff700 1 -- 172.21.15.97:0/154469664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a006b360 msgr2=0x7f518c005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f5197fff700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a006b360 0x7f518c005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f5197fff700 1 -- 172.21.15.97:0/154469664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f518c003a30 msgr2=0x7f518c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f5197fff700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f518c003a30 0x7f518c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f5197fff700 1 -- 172.21.15.97:0/154469664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0035b0 con 0x7f518c004850 2023-07-06T03:55:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.562+0000 7f5197fff700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 0x7f518c004cc0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f5188009860 tx=0x7f518c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b18cede0709d0ce server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.563+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518800c070 con 0x7f518c004850 2023-07-06T03:55:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.563+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5188013610 con 0x7f518c004850 2023-07-06T03:55:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.563+0000 7f519effd700 1 -- 172.21.15.97:0/154469664 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51880139a0 con 0x7f518c004850 2023-07-06T03:55:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.563+0000 7f51a5fff700 1 -- 172.21.15.97:0/154469664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 msgr2=0x7f518c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.563+0000 7f51a5fff700 1 --2- 172.21.15.97:0/154469664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 0x7f518c004cc0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f5188009860 tx=0x7f518c005880 comp rx=0 tx=0).stop 2023-07-06T03:55:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.563+0000 7f519ffff700 1 -- 172.21.15.97:0/154469664 reap_dead start 2023-07-06T03:55:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.563+0000 7f51a5fff700 1 -- 172.21.15.97:0/154469664 shutdown_connections 2023-07-06T03:55:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.563+0000 7f51a5fff700 1 -- 172.21.15.97:0/154469664 >> 172.21.15.97:0/154469664 conn(0x7f51a006a6c0 msgr2=0x7f51a006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.563+0000 7f51a5fff700 1 -- 172.21.15.97:0/154469664 shutdown_connections 2023-07-06T03:55:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.563+0000 7f51a5fff700 1 -- 172.21.15.97:0/154469664 wait complete. 2023-07-06T03:55:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.566+0000 7f51a5fff700 1 Processor -- start 2023-07-06T03:55:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.566+0000 7f51a5fff700 1 -- start start 2023-07-06T03:55:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.567+0000 7f51a5fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 0x7f51a01f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.567+0000 7f51a5fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a0158220 0x7f51a015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.567+0000 7f51a5fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51a015f520 0x7f51a015d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.567+0000 7f51a5fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a015f990 con 0x7f518c004850 2023-07-06T03:55:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.567+0000 7f51a5fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00c6bb0 con 0x7f51a015f520 2023-07-06T03:55:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.567+0000 7f51a5fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a015e200 con 0x7f51a0158220 2023-07-06T03:55:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.567+0000 7f519f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 0x7f51a01f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.567+0000 7f519f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 0x7f51a01f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59986/0 (socket says 172.21.15.97:59986) 2023-07-06T03:55:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.567+0000 7f519f7fe700 1 -- 172.21.15.97:0/3082993303 learned_addr learned my addr 172.21.15.97:0/3082993303 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f519ffff700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51a015f520 0x7f51a015d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f5197fff700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a0158220 0x7f51a015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f519f7fe700 1 -- 172.21.15.97:0/3082993303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a0158220 msgr2=0x7f51a015efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f519f7fe700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a0158220 0x7f51a015efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f519f7fe700 1 -- 172.21.15.97:0/3082993303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51a015f520 msgr2=0x7f51a015d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f519f7fe700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51a015f520 0x7f51a015d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f519f7fe700 1 -- 172.21.15.97:0/3082993303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a00cafd0 con 0x7f518c004850 2023-07-06T03:55:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f519f7fe700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 0x7f51a01f6c60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f519000eaf0 tx=0x7f519000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f519cff9700 1 -- 172.21.15.97:0/3082993303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000ee70 con 0x7f518c004850 2023-07-06T03:55:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f519cff9700 1 -- 172.21.15.97:0/3082993303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5190005e00 con 0x7f518c004850 2023-07-06T03:55:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f519cff9700 1 -- 172.21.15.97:0/3082993303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000ca20 con 0x7f518c004850 2023-07-06T03:55:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.568+0000 7f51a5fff700 1 -- 172.21.15.97:0/3082993303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a00cb2b0 con 0x7f518c004850 2023-07-06T03:55:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.569+0000 7f51a5fff700 1 -- 172.21.15.97:0/3082993303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a0158690 con 0x7f518c004850 2023-07-06T03:55:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.569+0000 7f51a5fff700 1 -- 172.21.15.97:0/3082993303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a0158850 con 0x7f518c004850 2023-07-06T03:55:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.570+0000 7f519cff9700 1 -- 172.21.15.97:0/3082993303 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f519000cb80 con 0x7f518c004850 2023-07-06T03:55:54.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.571+0000 7f519cff9700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f518006ace0 0x7f518006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.571+0000 7f519cff9700 1 -- 172.21.15.97:0/3082993303 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5190088e90 con 0x7f518c004850 2023-07-06T03:55:54.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.572+0000 7f5197fff700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f518006ace0 0x7f518006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:54.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.572+0000 7f5197fff700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f518006ace0 0x7f518006d190 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5188016040 tx=0x7f518801f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:54.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.576+0000 7f519cff9700 1 -- 172.21.15.97:0/3082993303 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f51900541c0 con 0x7f518c004850 2023-07-06T03:55:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.772+0000 7f51a5fff700 1 -- 172.21.15.97:0/3082993303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f51a015c9e0 con 0x7f518c004850 2023-07-06T03:55:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.777+0000 7f519cff9700 1 -- 172.21.15.97:0/3082993303 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7f51900577e0 con 0x7f518c004850 2023-07-06T03:55:54.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.779+0000 7f51a5fff700 1 -- 172.21.15.97:0/3082993303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f518006ace0 msgr2=0x7f518006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.779+0000 7f51a5fff700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f518006ace0 0x7f518006d190 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5188016040 tx=0x7f518801f040 comp rx=0 tx=0).stop 2023-07-06T03:55:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.779+0000 7f51a5fff700 1 -- 172.21.15.97:0/3082993303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 msgr2=0x7f51a01f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.779+0000 7f51a5fff700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 0x7f51a01f6c60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f519000eaf0 tx=0x7f519000ee00 comp rx=0 tx=0).stop 2023-07-06T03:55:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.779+0000 7f51a5fff700 1 -- 172.21.15.97:0/3082993303 shutdown_connections 2023-07-06T03:55:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.779+0000 7f51a5fff700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f518006ace0 0x7f518006d190 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.779+0000 7f51a5fff700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51a015f520 0x7f51a015d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.779+0000 7f51a5fff700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f518c004850 0x7f51a01f6c60 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.779+0000 7f51a5fff700 1 --2- 172.21.15.97:0/3082993303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f51a0158220 0x7f51a015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.779+0000 7f51a5fff700 1 -- 172.21.15.97:0/3082993303 >> 172.21.15.97:0/3082993303 conn(0x7f51a006a6c0 msgr2=0x7f51a0159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.779+0000 7f51a5fff700 1 -- 172.21.15.97:0/3082993303 shutdown_connections 2023-07-06T03:55:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.780+0000 7f51a5fff700 1 -- 172.21.15.97:0/3082993303 wait complete. 2023-07-06T03:55:54.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2023-07-06T03:55:54.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.994+0000 7ff9047ff700 1 Processor -- start 2023-07-06T03:55:54.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.994+0000 7ff9047ff700 1 -- start start 2023-07-06T03:55:54.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.996+0000 7ff9047ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8fc06b360 0x7ff8fc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:54.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.996+0000 7ff9047ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc0cff90 con 0x7ff8fc15c610 2023-07-06T03:55:54.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.996+0000 7ff9047ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc0b5650 con 0x7ff8fc0d2210 2023-07-06T03:55:54.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.996+0000 7ff9047ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0d0780 con 0x7ff8fc06b360 2023-07-06T03:55:54.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.996+0000 7ff90259b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff8fc15c610 0x7ff8fc0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42510/0 (socket says 172.21.15.97:42510) 2023-07-06T03:55:54.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.996+0000 7ff90259b700 1 -- 172.21.15.97:0/3685739082 learned_addr learned my addr 172.21.15.97:0/3685739082 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.996+0000 7ff901d9a700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8fc06b360 0x7ff8fc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.996+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046885637 0 0) 0x7ff8fc0cff90 con 0x7ff8fc15c610 2023-07-06T03:55:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.996+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8e80035b0 con 0x7ff8fc15c610 2023-07-06T03:55:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.996+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259012543 0 0) 0x7ff8fc0b5650 con 0x7ff8fc0d2210 2023-07-06T03:55:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8fc0cff90 con 0x7ff8fc0d2210 2023-07-06T03:55:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1983421446 0 0) 0x7ff8e80035b0 con 0x7ff8fc15c610 2023-07-06T03:55:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8fc06b360 msgr2=0x7ff8fc0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8fc06b360 0x7ff8fc0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 >> v1:172.21.15.146:6789/0 conn(0x7ff8fc0d2210 legacy=0x7ff8fc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc0b5400 con 0x7ff8fc15c610 2023-07-06T03:55:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7ff8f8001670 con 0x7ff8fc15c610 2023-07-06T03:55:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 >> v1:172.21.15.97:6789/0 conn(0x7ff8fc15c610 legacy=0x7ff8fc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8e8003bc0 0x7ff8e8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8fc06b360 0x7ff8e8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8e8004d90 0x7ff8e8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0d0780 con 0x7ff8e8003bc0 2023-07-06T03:55:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e800bac0 con 0x7ff8e8004d90 2023-07-06T03:55:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e800bc30 con 0x7ff8fc06b360 2023-07-06T03:55:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff90259b700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8e8003bc0 0x7ff8e8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff902d9c700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8e8004d90 0x7ff8e8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901d9a700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8fc06b360 0x7ff8e8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff90259b700 1 -- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8fc06b360 msgr2=0x7ff8e8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff90259b700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8fc06b360 0x7ff8e8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff90259b700 1 -- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8e8004d90 msgr2=0x7ff8e8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff90259b700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8e8004d90 0x7ff8e8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff90259b700 1 -- 172.21.15.97:0/3685739082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e80035b0 con 0x7ff8e8003bc0 2023-07-06T03:55:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.997+0000 7ff901d9a700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8fc06b360 0x7ff8e8004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.998+0000 7ff90259b700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8e8003bc0 0x7ff8e8003fb0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800c210 tx=0x7ff8e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256f433e4192d39e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.998+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f800ccf0 con 0x7ff8e8003bc0 2023-07-06T03:55:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.998+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f800ce70 con 0x7ff8e8003bc0 2023-07-06T03:55:55.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.998+0000 7ff901599700 1 -- 172.21.15.97:0/3685739082 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8005b80 con 0x7ff8e8003bc0 2023-07-06T03:55:55.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.998+0000 7ff9047ff700 1 -- 172.21.15.97:0/3685739082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8e8003bc0 msgr2=0x7ff8e8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:55.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.998+0000 7ff9047ff700 1 --2- 172.21.15.97:0/3685739082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8e8003bc0 0x7ff8e8003fb0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800c210 tx=0x7ff8e8005880 comp rx=0 tx=0).stop 2023-07-06T03:55:55.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.998+0000 7ff902d9c700 1 -- 172.21.15.97:0/3685739082 reap_dead start 2023-07-06T03:55:55.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.998+0000 7ff9047ff700 1 -- 172.21.15.97:0/3685739082 shutdown_connections 2023-07-06T03:55:55.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.998+0000 7ff9047ff700 1 -- 172.21.15.97:0/3685739082 >> 172.21.15.97:0/3685739082 conn(0x7ff8fc06a6c0 msgr2=0x7ff8fc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:55.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.998+0000 7ff9047ff700 1 -- 172.21.15.97:0/3685739082 shutdown_connections 2023-07-06T03:55:55.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:54.999+0000 7ff9047ff700 1 -- 172.21.15.97:0/3685739082 wait complete. 2023-07-06T03:55:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.000+0000 7ff9047ff700 1 Processor -- start 2023-07-06T03:55:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.000+0000 7ff9047ff700 1 -- start start 2023-07-06T03:55:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.001+0000 7ff9047ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8e8003bc0 0x7ff8fc0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.001+0000 7ff9047ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8fc0c9e00 0x7ff8fc0b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.001+0000 7ff9047ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc0b4e40 0x7ff8fc0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.001+0000 7ff9047ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc15e650 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.001+0000 7ff9047ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc15eb50 con 0x7ff8fc0b4e40 2023-07-06T03:55:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.001+0000 7ff9047ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0ca270 con 0x7ff8e8003bc0 2023-07-06T03:55:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff901d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8fc0c9e00 0x7ff8fc0b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff901d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8fc0c9e00 0x7ff8fc0b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60014/0 (socket says 172.21.15.97:60014) 2023-07-06T03:55:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff901d9a700 1 -- 172.21.15.97:0/531193319 learned_addr learned my addr 172.21.15.97:0/531193319 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff90259b700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8e8003bc0 0x7ff8fc0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff902d9c700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc0b4e40 0x7ff8fc0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff901d9a700 1 -- 172.21.15.97:0/531193319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8e8003bc0 msgr2=0x7ff8fc0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff901d9a700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8e8003bc0 0x7ff8fc0c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff901d9a700 1 -- 172.21.15.97:0/531193319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc0b4e40 msgr2=0x7ff8fc0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff901d9a700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc0b4e40 0x7ff8fc0b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff901d9a700 1 -- 172.21.15.97:0/531193319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc15ed80 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff902d9c700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc0b4e40 0x7ff8fc0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff90259b700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8e8003bc0 0x7ff8fc0c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff901d9a700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8fc0c9e00 0x7ff8fc0b4900 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec010870 tx=0x7ff8ec010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff8f2ffd700 1 -- 172.21.15.97:0/531193319 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec010e70 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff8f2ffd700 1 -- 172.21.15.97:0/531193319 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec004020 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.002+0000 7ff9047ff700 1 -- 172.21.15.97:0/531193319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc15f060 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.003+0000 7ff8f2ffd700 1 -- 172.21.15.97:0/531193319 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0043b0 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.003+0000 7ff9047ff700 1 -- 172.21.15.97:0/531193319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc15f620 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.004+0000 7ff9047ff700 1 -- 172.21.15.97:0/531193319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc04e340 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.005+0000 7ff8f2ffd700 1 -- 172.21.15.97:0/531193319 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff8ec00e8d0 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.005+0000 7ff8f2ffd700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8e006ace0 0x7ff8e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.006+0000 7ff8f2ffd700 1 -- 172.21.15.97:0/531193319 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff8ec089230 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.006+0000 7ff90259b700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8e006ace0 0x7ff8e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.006+0000 7ff90259b700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8e006ace0 0x7ff8e006d190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc153a10 tx=0x7ff8f8007f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:55.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.010+0000 7ff8f2ffd700 1 -- 172.21.15.97:0/531193319 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff8ec057ff0 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.209+0000 7ff9047ff700 1 -- 172.21.15.97:0/531193319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) v1 -- 0x7ff8fc0c9670 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.213+0000 7ff8f2ffd700 1 -- 172.21.15.97:0/531193319 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v63) v1 ==== 105+0+0 (secure 0 0 0) 0x7ff8ec057b80 con 0x7ff8fc0c9e00 2023-07-06T03:55:55.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 -- 172.21.15.97:0/531193319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8e006ace0 msgr2=0x7ff8e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:55.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8e006ace0 0x7ff8e006d190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc153a10 tx=0x7ff8f8007f90 comp rx=0 tx=0).stop 2023-07-06T03:55:55.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 -- 172.21.15.97:0/531193319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8fc0c9e00 msgr2=0x7ff8fc0b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:55.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8fc0c9e00 0x7ff8fc0b4900 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec010870 tx=0x7ff8ec010c30 comp rx=0 tx=0).stop 2023-07-06T03:55:55.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 -- 172.21.15.97:0/531193319 shutdown_connections 2023-07-06T03:55:55.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff8e006ace0 0x7ff8e006d190 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8fc0b4e40 0x7ff8fc0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff8e8003bc0 0x7ff8fc0c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 --2- 172.21.15.97:0/531193319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8fc0c9e00 0x7ff8fc0b4900 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 -- 172.21.15.97:0/531193319 >> 172.21.15.97:0/531193319 conn(0x7ff8fc06a6c0 msgr2=0x7ff8fc0d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:55.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 -- 172.21.15.97:0/531193319 shutdown_connections 2023-07-06T03:55:55.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.216+0000 7ff9047ff700 1 -- 172.21.15.97:0/531193319 wait complete. 2023-07-06T03:55:55.239 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2023-07-06T03:55:55.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.433+0000 7fcc905ff700 1 Processor -- start 2023-07-06T03:55:55.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.433+0000 7fcc905ff700 1 -- start start 2023-07-06T03:55:55.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.434+0000 7fcc905ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc8815c7c0 0x7fcc880ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.434+0000 7fcc905ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc880cb470 0x7fcc880cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.434+0000 7fcc905ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc880ca670 con 0x7fcc880cd800 2023-07-06T03:55:55.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.434+0000 7fcc905ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc880ca950 con 0x7fcc880cb470 2023-07-06T03:55:55.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.434+0000 7fcc905ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880cac30 con 0x7fcc8815c7c0 2023-07-06T03:55:55.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.434+0000 7fcc8db9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc880cb470 0x7fcc880cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.434+0000 7fcc8eb9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcc880cd800 0x7fcc881603a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42524/0 (socket says 172.21.15.97:42524) 2023-07-06T03:55:55.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.434+0000 7fcc8eb9c700 1 -- 172.21.15.97:0/31251995 learned_addr learned my addr 172.21.15.97:0/31251995 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:55.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.434+0000 7fcc8db9a700 1 --2- 172.21.15.97:0/31251995 >> v2:172.21.15.97:3300/0 conn(0x7fcc880cb470 0x7fcc880cd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60024/0 (socket says 172.21.15.97:60024) 2023-07-06T03:55:55.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.435+0000 7fcc8e39b700 1 --2- 172.21.15.97:0/31251995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc8815c7c0 0x7fcc880ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.435+0000 7fcc8db9a700 1 -- 172.21.15.97:0/31251995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc8815c7c0 msgr2=0x7fcc880ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:55.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.435+0000 7fcc8db9a700 1 --2- 172.21.15.97:0/31251995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc8815c7c0 0x7fcc880ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.435+0000 7fcc8d399700 1 -- 172.21.15.97:0/31251995 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717427743 0 0) 0x7fcc880ca670 con 0x7fcc880cd800 2023-07-06T03:55:55.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.435+0000 7fcc8db9a700 1 -- 172.21.15.97:0/31251995 >> v1:172.21.15.97:6789/0 conn(0x7fcc880cd800 legacy=0x7fcc881603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:55.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.435+0000 7fcc8db9a700 1 -- 172.21.15.97:0/31251995 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc880caf10 con 0x7fcc880cb470 2023-07-06T03:55:55.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.435+0000 7fcc8e39b700 1 --2- 172.21.15.97:0/31251995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc8815c7c0 0x7fcc880ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:55.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.435+0000 7fcc8db9a700 1 --2- 172.21.15.97:0/31251995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc880cb470 0x7fcc880cd2c0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fcc78009840 tx=0x7fcc78009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d9ad17ec83f09 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:55.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.435+0000 7fcc8d399700 1 -- 172.21.15.97:0/31251995 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78011830 con 0x7fcc880cb470 2023-07-06T03:55:55.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.435+0000 7fcc8d399700 1 -- 172.21.15.97:0/31251995 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc78011990 con 0x7fcc880cb470 2023-07-06T03:55:55.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.435+0000 7fcc8d399700 1 -- 172.21.15.97:0/31251995 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78011d20 con 0x7fcc880cb470 2023-07-06T03:55:55.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.436+0000 7fcc905ff700 1 -- 172.21.15.97:0/31251995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc880cb470 msgr2=0x7fcc880cd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:55.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.436+0000 7fcc905ff700 1 --2- 172.21.15.97:0/31251995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc880cb470 0x7fcc880cd2c0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fcc78009840 tx=0x7fcc78009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:55.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.436+0000 7fcc905ff700 1 -- 172.21.15.97:0/31251995 shutdown_connections 2023-07-06T03:55:55.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.436+0000 7fcc905ff700 1 --2- 172.21.15.97:0/31251995 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc8815c7c0 0x7fcc880ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.436+0000 7fcc905ff700 1 --2- 172.21.15.97:0/31251995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc880cb470 0x7fcc880cd2c0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.436+0000 7fcc905ff700 1 -- 172.21.15.97:0/31251995 >> 172.21.15.97:0/31251995 conn(0x7fcc880cf1b0 msgr2=0x7fcc880cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:55.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.436+0000 7fcc905ff700 1 -- 172.21.15.97:0/31251995 shutdown_connections 2023-07-06T03:55:55.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.436+0000 7fcc905ff700 1 -- 172.21.15.97:0/31251995 wait complete. 2023-07-06T03:55:55.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.438+0000 7fcc905ff700 1 Processor -- start 2023-07-06T03:55:55.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.438+0000 7fcc905ff700 1 -- start start 2023-07-06T03:55:55.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc905ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc880cb470 0x7fcc880c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc905ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc880cd800 0x7fcc8815b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc905ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc8815c7c0 0x7fcc8809f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc905ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880c70a0 con 0x7fcc8815c7c0 2023-07-06T03:55:55.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc905ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880b1fe0 con 0x7fcc880cb470 2023-07-06T03:55:55.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc905ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8815b8d0 con 0x7fcc880cd800 2023-07-06T03:55:55.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc8eb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc8815c7c0 0x7fcc8809f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc8eb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc8815c7c0 0x7fcc8809f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60030/0 (socket says 172.21.15.97:60030) 2023-07-06T03:55:55.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc8eb9c700 1 -- 172.21.15.97:0/515400216 learned_addr learned my addr 172.21.15.97:0/515400216 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:55.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc8e39b700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc880cb470 0x7fcc880c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc8db9a700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc880cd800 0x7fcc8815b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc8eb9c700 1 -- 172.21.15.97:0/515400216 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc880cd800 msgr2=0x7fcc8815b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:55.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc8eb9c700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc880cd800 0x7fcc8815b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc8eb9c700 1 -- 172.21.15.97:0/515400216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc880cb470 msgr2=0x7fcc880c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:55.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc8eb9c700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc880cb470 0x7fcc880c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:55.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc8eb9c700 1 -- 172.21.15.97:0/515400216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc780094a0 con 0x7fcc8815c7c0 2023-07-06T03:55:55.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.439+0000 7fcc8eb9c700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc8815c7c0 0x7fcc8809f390 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fcc80002230 tx=0x7fcc8000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:55.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.440+0000 7fcc7effd700 1 -- 172.21.15.97:0/515400216 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8000ac20 con 0x7fcc8815c7c0 2023-07-06T03:55:55.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.440+0000 7fcc7effd700 1 -- 172.21.15.97:0/515400216 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8000ada0 con 0x7fcc8815c7c0 2023-07-06T03:55:55.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.440+0000 7fcc905ff700 1 -- 172.21.15.97:0/515400216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8809fd00 con 0x7fcc8815c7c0 2023-07-06T03:55:55.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.440+0000 7fcc7effd700 1 -- 172.21.15.97:0/515400216 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80007310 con 0x7fcc8815c7c0 2023-07-06T03:55:55.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.440+0000 7fcc905ff700 1 -- 172.21.15.97:0/515400216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc880b2230 con 0x7fcc8815c7c0 2023-07-06T03:55:55.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.441+0000 7fcc905ff700 1 -- 172.21.15.97:0/515400216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8804e340 con 0x7fcc8815c7c0 2023-07-06T03:55:55.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.442+0000 7fcc7effd700 1 -- 172.21.15.97:0/515400216 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc80007470 con 0x7fcc8815c7c0 2023-07-06T03:55:55.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.443+0000 7fcc7effd700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc6c06ace0 0x7fcc6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:55.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.443+0000 7fcc7effd700 1 -- 172.21.15.97:0/515400216 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcc80089120 con 0x7fcc8815c7c0 2023-07-06T03:55:55.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.444+0000 7fcc8e39b700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc6c06ace0 0x7fcc6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:55.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.444+0000 7fcc8e39b700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc6c06ace0 0x7fcc6c06d190 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcc84006fd0 tx=0x7fcc84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:55.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.448+0000 7fcc7effd700 1 -- 172.21.15.97:0/515400216 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcc80054450 con 0x7fcc8815c7c0 2023-07-06T03:55:55.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:55.726+0000 7fcc905ff700 1 -- 172.21.15.97:0/515400216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) v1 -- 0x7fcc8815fe60 con 0x7fcc8815c7c0 2023-07-06T03:55:56.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.030+0000 7fcc7effd700 1 -- 172.21.15.97:0/515400216 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v556) v1 ==== 148+0+0 (secure 0 0 0) 0x7fcc80057a70 con 0x7fcc8815c7c0 2023-07-06T03:55:56.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 -- 172.21.15.97:0/515400216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc6c06ace0 msgr2=0x7fcc6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:56.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc6c06ace0 0x7fcc6c06d190 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcc84006fd0 tx=0x7fcc84008040 comp rx=0 tx=0).stop 2023-07-06T03:55:56.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 -- 172.21.15.97:0/515400216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc8815c7c0 msgr2=0x7fcc8809f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc8815c7c0 0x7fcc8809f390 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fcc80002230 tx=0x7fcc8000a060 comp rx=0 tx=0).stop 2023-07-06T03:55:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 -- 172.21.15.97:0/515400216 shutdown_connections 2023-07-06T03:55:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc6c06ace0 0x7fcc6c06d190 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc8815c7c0 0x7fcc8809f390 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc880cb470 0x7fcc880c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 --2- 172.21.15.97:0/515400216 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc880cd800 0x7fcc8815b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:56.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 -- 172.21.15.97:0/515400216 >> 172.21.15.97:0/515400216 conn(0x7fcc880cf1b0 msgr2=0x7fcc88070240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:56.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 -- 172.21.15.97:0/515400216 shutdown_connections 2023-07-06T03:55:56.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.033+0000 7fcc905ff700 1 -- 172.21.15.97:0/515400216 wait complete. 2023-07-06T03:55:56.054 INFO:tasks.cephfs_test_runner:test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2023-07-06T03:55:56.055 INFO:tasks.cephfs_test_runner:Starting test: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:55:56.055 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2023-07-06T03:55:56.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.247+0000 7ff592dff700 1 Processor -- start 2023-07-06T03:55:56.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.247+0000 7ff592dff700 1 -- start start 2023-07-06T03:55:56.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.248+0000 7ff592dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff58c0d09b0 0x7ff58c0d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:56.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.248+0000 7ff592dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c0b4740 con 0x7ff58c15ff00 2023-07-06T03:55:56.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.248+0000 7ff592dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c1602d0 con 0x7ff58c0b6540 2023-07-06T03:55:56.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.248+0000 7ff592dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58c0d1360 con 0x7ff58c0d09b0 2023-07-06T03:55:56.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.249+0000 7ff590b9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff58c15ff00 0x7ff58c0c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42534/0 (socket says 172.21.15.97:42534) 2023-07-06T03:55:56.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.249+0000 7ff590b9b700 1 -- 172.21.15.97:0/1088858863 learned_addr learned my addr 172.21.15.97:0/1088858863 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:56.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.249+0000 7ff58bfff700 1 --2- 172.21.15.97:0/1088858863 >> v2:172.21.15.146:3300/0 conn(0x7ff58c0d09b0 0x7ff58c0d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:56.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.249+0000 7ff58b7fe700 1 -- 172.21.15.97:0/1088858863 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595597632 0 0) 0x7ff58c0b4740 con 0x7ff58c15ff00 2023-07-06T03:55:56.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.249+0000 7ff58b7fe700 1 -- 172.21.15.97:0/1088858863 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff57c0035b0 con 0x7ff58c15ff00 2023-07-06T03:55:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.249+0000 7ff58bfff700 1 -- 172.21.15.97:0/1088858863 >> v1:172.21.15.146:6789/0 conn(0x7ff58c0b6540 legacy=0x7ff58c0b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.249+0000 7ff58bfff700 1 -- 172.21.15.97:0/1088858863 >> v1:172.21.15.97:6789/0 conn(0x7ff58c15ff00 legacy=0x7ff58c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.249+0000 7ff58b7fe700 1 -- 172.21.15.97:0/1088858863 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3322307337 0 0) 0x7ff57c0035b0 con 0x7ff58c15ff00 2023-07-06T03:55:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.249+0000 7ff58bfff700 1 -- 172.21.15.97:0/1088858863 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c0b60b0 con 0x7ff58c0d09b0 2023-07-06T03:55:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.250+0000 7ff58bfff700 1 --2- 172.21.15.97:0/1088858863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff58c0d09b0 0x7ff58c0d0e20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff578009840 tx=0x7ff578009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=396a9ab6a5fd1c18 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.250+0000 7ff58b7fe700 1 -- 172.21.15.97:0/1088858863 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff578011830 con 0x7ff58c0d09b0 2023-07-06T03:55:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.250+0000 7ff58b7fe700 1 -- 172.21.15.97:0/1088858863 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff578011990 con 0x7ff58c0d09b0 2023-07-06T03:55:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.250+0000 7ff58b7fe700 1 -- 172.21.15.97:0/1088858863 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff578011d20 con 0x7ff58c0d09b0 2023-07-06T03:55:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.250+0000 7ff592dff700 1 -- 172.21.15.97:0/1088858863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff58c0d09b0 msgr2=0x7ff58c0d0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:56.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.250+0000 7ff592dff700 1 --2- 172.21.15.97:0/1088858863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff58c0d09b0 0x7ff58c0d0e20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff578009840 tx=0x7ff578009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:56.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.250+0000 7ff592dff700 1 -- 172.21.15.97:0/1088858863 shutdown_connections 2023-07-06T03:55:56.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.250+0000 7ff592dff700 1 --2- 172.21.15.97:0/1088858863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff58c0d09b0 0x7ff58c0d0e20 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:56.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.250+0000 7ff592dff700 1 -- 172.21.15.97:0/1088858863 >> 172.21.15.97:0/1088858863 conn(0x7ff58c09c130 msgr2=0x7ff58c0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.250+0000 7ff592dff700 1 -- 172.21.15.97:0/1088858863 shutdown_connections 2023-07-06T03:55:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.251+0000 7ff592dff700 1 -- 172.21.15.97:0/1088858863 wait complete. 2023-07-06T03:55:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.252+0000 7ff592dff700 1 Processor -- start 2023-07-06T03:55:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.252+0000 7ff592dff700 1 -- start start 2023-07-06T03:55:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.253+0000 7ff592dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c0b6540 0x7ff58c1f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.253+0000 7ff592dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58c0d09b0 0x7ff58c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.253+0000 7ff592dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff58c15ff00 0x7ff58c0cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.253+0000 7ff592dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0cfc30 con 0x7ff58c0b6540 2023-07-06T03:55:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.253+0000 7ff592dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c1f2760 con 0x7ff58c15ff00 2023-07-06T03:55:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.253+0000 7ff592dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c1f2a40 con 0x7ff58c0d09b0 2023-07-06T03:55:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.253+0000 7ff590b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c0b6540 0x7ff58c1f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.253+0000 7ff590b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c0b6540 0x7ff58c1f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60034/0 (socket says 172.21.15.97:60034) 2023-07-06T03:55:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff590b9b700 1 -- 172.21.15.97:0/3153366303 learned_addr learned my addr 172.21.15.97:0/3153366303 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff58bfff700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58c0d09b0 0x7ff58c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff59139c700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff58c15ff00 0x7ff58c0cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff590b9b700 1 -- 172.21.15.97:0/3153366303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58c0d09b0 msgr2=0x7ff58c0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff590b9b700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58c0d09b0 0x7ff58c0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff590b9b700 1 -- 172.21.15.97:0/3153366303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff58c15ff00 msgr2=0x7ff58c0cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff590b9b700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff58c15ff00 0x7ff58c0cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff590b9b700 1 -- 172.21.15.97:0/3153366303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5780094a0 con 0x7ff58c0b6540 2023-07-06T03:55:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff59139c700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff58c15ff00 0x7ff58c0cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:55:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff590b9b700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c0b6540 0x7ff58c1f2120 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7ff580007fd0 tx=0x7ff58000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff5897fa700 1 -- 172.21.15.97:0/3153366303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5800074e0 con 0x7ff58c0b6540 2023-07-06T03:55:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff5897fa700 1 -- 172.21.15.97:0/3153366303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff580007640 con 0x7ff58c0b6540 2023-07-06T03:55:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff592dff700 1 -- 172.21.15.97:0/3153366303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c1f2d20 con 0x7ff58c0b6540 2023-07-06T03:55:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.254+0000 7ff5897fa700 1 -- 172.21.15.97:0/3153366303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580005a50 con 0x7ff58c0b6540 2023-07-06T03:55:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.255+0000 7ff592dff700 1 -- 172.21.15.97:0/3153366303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c1f32e0 con 0x7ff58c0b6540 2023-07-06T03:55:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.255+0000 7ff592dff700 1 -- 172.21.15.97:0/3153366303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c1f3650 con 0x7ff58c0b6540 2023-07-06T03:55:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.256+0000 7ff5897fa700 1 -- 172.21.15.97:0/3153366303 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff580005bb0 con 0x7ff58c0b6540 2023-07-06T03:55:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.257+0000 7ff5897fa700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff57006ace0 0x7ff57006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.258+0000 7ff5897fa700 1 -- 172.21.15.97:0/3153366303 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff580088690 con 0x7ff58c0b6540 2023-07-06T03:55:56.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.258+0000 7ff58bfff700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff57006ace0 0x7ff57006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:56.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.258+0000 7ff58bfff700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff57006ace0 0x7ff57006d190 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff578009fd0 tx=0x7ff57801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:56.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.261+0000 7ff5897fa700 1 -- 172.21.15.97:0/3153366303 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff580053a40 con 0x7ff58c0b6540 2023-07-06T03:55:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:56.529+0000 7ff592dff700 1 -- 172.21.15.97:0/3153366303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) v1 -- 0x7ff58c1f3470 con 0x7ff58c0b6540 2023-07-06T03:55:57.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.036+0000 7ff5897fa700 1 -- 172.21.15.97:0/3153366303 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v557) v1 ==== 154+0+0 (secure 0 0 0) 0x7ff580007d80 con 0x7ff58c0b6540 2023-07-06T03:55:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 -- 172.21.15.97:0/3153366303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff57006ace0 msgr2=0x7ff57006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff57006ace0 0x7ff57006d190 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff578009fd0 tx=0x7ff57801c040 comp rx=0 tx=0).stop 2023-07-06T03:55:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 -- 172.21.15.97:0/3153366303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c0b6540 msgr2=0x7ff58c1f2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c0b6540 0x7ff58c1f2120 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7ff580007fd0 tx=0x7ff58000c7e0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 -- 172.21.15.97:0/3153366303 shutdown_connections 2023-07-06T03:55:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff57006ace0 0x7ff57006d190 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff58c15ff00 0x7ff58c0cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c0b6540 0x7ff58c1f2120 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 --2- 172.21.15.97:0/3153366303 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff58c0d09b0 0x7ff58c0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 -- 172.21.15.97:0/3153366303 >> 172.21.15.97:0/3153366303 conn(0x7ff58c09c130 msgr2=0x7ff58c0b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 -- 172.21.15.97:0/3153366303 shutdown_connections 2023-07-06T03:55:57.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.038+0000 7ff592dff700 1 -- 172.21.15.97:0/3153366303 wait complete. 2023-07-06T03:55:57.061 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T03:55:57.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.257+0000 7f7b003ff700 1 Processor -- start 2023-07-06T03:55:57.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.258+0000 7f7b003ff700 1 -- start start 2023-07-06T03:55:57.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.259+0000 7f7b003ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7af8159070 0x7f7af80c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:57.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.259+0000 7f7b003ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7af80c3cd0 0x7f7af80cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:57.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.259+0000 7f7b003ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af80ca260 con 0x7f7af8159440 2023-07-06T03:55:57.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.259+0000 7f7b003ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af80c99c0 con 0x7f7af8159070 2023-07-06T03:55:57.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.259+0000 7f7b003ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80c9ca0 con 0x7f7af80c3cd0 2023-07-06T03:55:57.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.259+0000 7f7afe99c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7af8159440 0x7f7af80b0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42548/0 (socket says 172.21.15.97:42548) 2023-07-06T03:55:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.259+0000 7f7afe99c700 1 -- 172.21.15.97:0/1346800867 learned_addr learned my addr 172.21.15.97:0/1346800867 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.259+0000 7f7afd99a700 1 --2- 172.21.15.97:0/1346800867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7af80c3cd0 0x7f7af80cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.259+0000 7f7afe19b700 1 --2- 172.21.15.97:0/1346800867 >> v2:172.21.15.146:3300/0 conn(0x7f7af8159070 0x7f7af80c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:57.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.259+0000 7f7afd199700 1 -- 172.21.15.97:0/1346800867 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554409745 0 0) 0x7f7af80ca260 con 0x7f7af8159440 2023-07-06T03:55:57.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.260+0000 7f7afd199700 1 -- 172.21.15.97:0/1346800867 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ae40057c0 con 0x7f7af8159440 2023-07-06T03:55:57.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.260+0000 7f7afe19b700 1 -- 172.21.15.97:0/1346800867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7af80c3cd0 msgr2=0x7f7af80cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:57.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.260+0000 7f7afe19b700 1 --2- 172.21.15.97:0/1346800867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7af80c3cd0 0x7f7af80cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.260+0000 7f7afe19b700 1 -- 172.21.15.97:0/1346800867 >> v1:172.21.15.97:6789/0 conn(0x7f7af8159440 legacy=0x7f7af80b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:57.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.260+0000 7f7afe19b700 1 -- 172.21.15.97:0/1346800867 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af80c9f80 con 0x7f7af8159070 2023-07-06T03:55:57.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.260+0000 7f7afe19b700 1 --2- 172.21.15.97:0/1346800867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7af8159070 0x7f7af80c8f40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009840 tx=0x7f7af4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b90ef48b2578ddd7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:57.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.260+0000 7f7afd199700 1 -- 172.21.15.97:0/1346800867 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4011830 con 0x7f7af8159070 2023-07-06T03:55:57.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.260+0000 7f7afd199700 1 -- 172.21.15.97:0/1346800867 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4011990 con 0x7f7af8159070 2023-07-06T03:55:57.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.260+0000 7f7afd199700 1 -- 172.21.15.97:0/1346800867 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4011d20 con 0x7f7af8159070 2023-07-06T03:55:57.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.261+0000 7f7b003ff700 1 -- 172.21.15.97:0/1346800867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7af8159070 msgr2=0x7f7af80c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:57.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.261+0000 7f7b003ff700 1 --2- 172.21.15.97:0/1346800867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7af8159070 0x7f7af80c8f40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009840 tx=0x7f7af4009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:57.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.261+0000 7f7b003ff700 1 -- 172.21.15.97:0/1346800867 shutdown_connections 2023-07-06T03:55:57.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.261+0000 7f7b003ff700 1 --2- 172.21.15.97:0/1346800867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7af8159070 0x7f7af80c8f40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.261+0000 7f7b003ff700 1 --2- 172.21.15.97:0/1346800867 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7af80c3cd0 0x7f7af80cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.261+0000 7f7b003ff700 1 -- 172.21.15.97:0/1346800867 >> 172.21.15.97:0/1346800867 conn(0x7f7af806a060 msgr2=0x7f7af806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:57.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.261+0000 7f7b003ff700 1 -- 172.21.15.97:0/1346800867 shutdown_connections 2023-07-06T03:55:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.261+0000 7f7b003ff700 1 -- 172.21.15.97:0/1346800867 wait complete. 2023-07-06T03:55:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.263+0000 7f7b003ff700 1 Processor -- start 2023-07-06T03:55:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.263+0000 7f7b003ff700 1 -- start start 2023-07-06T03:55:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7b003ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af80c3cd0 0x7f7af814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7b003ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7af8159070 0x7f7af814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7b003ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7af8159440 0x7f7af80ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7b003ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80b3230 con 0x7f7af80c3cd0 2023-07-06T03:55:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7b003ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af8150730 con 0x7f7af8159070 2023-07-06T03:55:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7b003ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80b2920 con 0x7f7af8159440 2023-07-06T03:55:57.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7afe19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af80c3cd0 0x7f7af814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:57.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7afe19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af80c3cd0 0x7f7af814d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60050/0 (socket says 172.21.15.97:60050) 2023-07-06T03:55:57.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7afe19b700 1 -- 172.21.15.97:0/3239378497 learned_addr learned my addr 172.21.15.97:0/3239378497 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:57.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7afd99a700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7af8159070 0x7f7af814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:57.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7afe99c700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7af8159440 0x7f7af80ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:57.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7afe19b700 1 -- 172.21.15.97:0/3239378497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7af8159440 msgr2=0x7f7af80ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:57.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7afe19b700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7af8159440 0x7f7af80ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7afe19b700 1 -- 172.21.15.97:0/3239378497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7af8159070 msgr2=0x7f7af814fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.264+0000 7f7afe19b700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7af8159070 0x7f7af814fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.265+0000 7f7afe19b700 1 -- 172.21.15.97:0/3239378497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af40094a0 con 0x7f7af80c3cd0 2023-07-06T03:55:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.265+0000 7f7afe99c700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7af8159440 0x7f7af80ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.265+0000 7f7afe19b700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af80c3cd0 0x7f7af814d490 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f7af4000c00 tx=0x7f7af4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.265+0000 7f7aeeffd700 1 -- 172.21.15.97:0/3239378497 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40036a0 con 0x7f7af80c3cd0 2023-07-06T03:55:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.265+0000 7f7aeeffd700 1 -- 172.21.15.97:0/3239378497 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af401f070 con 0x7f7af80c3cd0 2023-07-06T03:55:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.265+0000 7f7aeeffd700 1 -- 172.21.15.97:0/3239378497 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4003a60 con 0x7f7af80c3cd0 2023-07-06T03:55:57.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.265+0000 7f7b003ff700 1 -- 172.21.15.97:0/3239378497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af80af2f0 con 0x7f7af80c3cd0 2023-07-06T03:55:57.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.265+0000 7f7b003ff700 1 -- 172.21.15.97:0/3239378497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af80b2b70 con 0x7f7af80c3cd0 2023-07-06T03:55:57.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.266+0000 7f7b003ff700 1 -- 172.21.15.97:0/3239378497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af804e340 con 0x7f7af80c3cd0 2023-07-06T03:55:57.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.267+0000 7f7aeeffd700 1 -- 172.21.15.97:0/3239378497 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7af4003bc0 con 0x7f7af80c3cd0 2023-07-06T03:55:57.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.268+0000 7f7aeeffd700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7adc06ace0 0x7f7adc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:57.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.268+0000 7f7aeeffd700 1 -- 172.21.15.97:0/3239378497 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7af4021070 con 0x7f7af80c3cd0 2023-07-06T03:55:57.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.268+0000 7f7afd99a700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7adc06ace0 0x7f7adc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:57.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.269+0000 7f7afd99a700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7adc06ace0 0x7f7adc06d190 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7af80ade90 tx=0x7f7ae8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:57.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.272+0000 7f7aeeffd700 1 -- 172.21.15.97:0/3239378497 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7af4055e00 con 0x7f7af80c3cd0 2023-07-06T03:55:57.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.468+0000 7f7b003ff700 1 -- 172.21.15.97:0/3239378497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7af806d220 con 0x7f7af80c3cd0 2023-07-06T03:55:57.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.473+0000 7f7aeeffd700 1 -- 172.21.15.97:0/3239378497 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7af4059420 con 0x7f7af80c3cd0 2023-07-06T03:55:57.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 -- 172.21.15.97:0/3239378497 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7adc06ace0 msgr2=0x7f7adc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:57.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7adc06ace0 0x7f7adc06d190 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7af80ade90 tx=0x7f7ae8008040 comp rx=0 tx=0).stop 2023-07-06T03:55:57.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 -- 172.21.15.97:0/3239378497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af80c3cd0 msgr2=0x7f7af814d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:57.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af80c3cd0 0x7f7af814d490 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f7af4000c00 tx=0x7f7af4004750 comp rx=0 tx=0).stop 2023-07-06T03:55:57.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 -- 172.21.15.97:0/3239378497 shutdown_connections 2023-07-06T03:55:57.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7adc06ace0 0x7f7adc06d190 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7af8159440 0x7f7af80ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af80c3cd0 0x7f7af814d490 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 --2- 172.21.15.97:0/3239378497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7af8159070 0x7f7af814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:57.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 -- 172.21.15.97:0/3239378497 >> 172.21.15.97:0/3239378497 conn(0x7f7af806a060 msgr2=0x7f7af80cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:57.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 -- 172.21.15.97:0/3239378497 shutdown_connections 2023-07-06T03:55:57.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:57.475+0000 7f7b003ff700 1 -- 172.21.15.97:0/3239378497 wait complete. 2023-07-06T03:55:57.496 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:55:57.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:55:57.589 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:55:57.694 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:55:57.694 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:55:57.695 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:57.740 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:55:57.741 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:55:57.741 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:57.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:55:57.789 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:55:57.790 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:57.835 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:55:57.836 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:55:57.836 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:55:57.882 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:55:57.883 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T03:55:57.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:55:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.077+0000 7ffb579ff700 1 Processor -- start 2023-07-06T03:55:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.077+0000 7ffb579ff700 1 -- start start 2023-07-06T03:55:58.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.078+0000 7ffb579ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb500cd390 0x7ffb50160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.078+0000 7ffb579ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffb50160dd0 0x7ffb500b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.078+0000 7ffb579ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb500cdae0 con 0x7ffb5015c520 2023-07-06T03:55:58.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.078+0000 7ffb579ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb500b6640 con 0x7ffb50160dd0 2023-07-06T03:55:58.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.078+0000 7ffb579ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb500b6920 con 0x7ffb500cd390 2023-07-06T03:55:58.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.078+0000 7ffb55f9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffb50160dd0 0x7ffb500b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:58.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.078+0000 7ffb55f9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffb50160dd0 0x7ffb500b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60066/0 (socket says 172.21.15.97:60066) 2023-07-06T03:55:58.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.079+0000 7ffb55f9c700 1 -- 172.21.15.97:0/2551183588 learned_addr learned my addr 172.21.15.97:0/2551183588 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:58.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.079+0000 7ffb54f9a700 1 --2- 172.21.15.97:0/2551183588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb500cd390 0x7ffb50160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.079+0000 7ffb55f9c700 1 -- 172.21.15.97:0/2551183588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb500cd390 msgr2=0x7ffb50160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.079+0000 7ffb55f9c700 1 --2- 172.21.15.97:0/2551183588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb500cd390 0x7ffb50160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.079+0000 7ffb55f9c700 1 -- 172.21.15.97:0/2551183588 >> v1:172.21.15.146:6789/0 conn(0x7ffb5015c520 legacy=0x7ffb50160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.079+0000 7ffb55f9c700 1 -- 172.21.15.97:0/2551183588 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb500b6c00 con 0x7ffb50160dd0 2023-07-06T03:55:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.079+0000 7ffb47fff700 1 -- 172.21.15.97:0/2551183588 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164973613 0 0) 0x7ffb500cdae0 con 0x7ffb5015c520 2023-07-06T03:55:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.079+0000 7ffb55f9c700 1 --2- 172.21.15.97:0/2551183588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb50160dd0 0x7ffb500b7400 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009e50 tx=0x7ffb4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc2053fc2c75fce server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:58.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.079+0000 7ffb47fff700 1 -- 172.21.15.97:0/2551183588 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4800e070 con 0x7ffb50160dd0 2023-07-06T03:55:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.079+0000 7ffb47fff700 1 -- 172.21.15.97:0/2551183588 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4800b910 con 0x7ffb50160dd0 2023-07-06T03:55:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.079+0000 7ffb47fff700 1 -- 172.21.15.97:0/2551183588 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4800bcc0 con 0x7ffb50160dd0 2023-07-06T03:55:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.080+0000 7ffb579ff700 1 -- 172.21.15.97:0/2551183588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb50160dd0 msgr2=0x7ffb500b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.080+0000 7ffb579ff700 1 --2- 172.21.15.97:0/2551183588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb50160dd0 0x7ffb500b7400 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009e50 tx=0x7ffb4800b330 comp rx=0 tx=0).stop 2023-07-06T03:55:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.080+0000 7ffb579ff700 1 -- 172.21.15.97:0/2551183588 shutdown_connections 2023-07-06T03:55:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.080+0000 7ffb579ff700 1 --2- 172.21.15.97:0/2551183588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb50160dd0 0x7ffb500b7400 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.080+0000 7ffb579ff700 1 --2- 172.21.15.97:0/2551183588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb500cd390 0x7ffb50160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.080+0000 7ffb579ff700 1 -- 172.21.15.97:0/2551183588 >> 172.21.15.97:0/2551183588 conn(0x7ffb50071790 msgr2=0x7ffb50071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:58.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.080+0000 7ffb579ff700 1 -- 172.21.15.97:0/2551183588 shutdown_connections 2023-07-06T03:55:58.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.080+0000 7ffb579ff700 1 -- 172.21.15.97:0/2551183588 wait complete. 2023-07-06T03:55:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.082+0000 7ffb579ff700 1 Processor -- start 2023-07-06T03:55:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.083+0000 7ffb579ff700 1 -- start start 2023-07-06T03:55:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb579ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb500cd390 0x7ffb5015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb579ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb5015c520 0x7ffb500d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb579ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb50160dd0 0x7ffb500b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb579ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5015b540 con 0x7ffb500cd390 2023-07-06T03:55:58.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb579ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb50140120 con 0x7ffb50160dd0 2023-07-06T03:55:58.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb579ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb500d2ff0 con 0x7ffb5015c520 2023-07-06T03:55:58.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb5579b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb500cd390 0x7ffb5015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:58.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb5579b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb500cd390 0x7ffb5015b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60076/0 (socket says 172.21.15.97:60076) 2023-07-06T03:55:58.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb5579b700 1 -- 172.21.15.97:0/4224099523 learned_addr learned my addr 172.21.15.97:0/4224099523 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:58.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb54f9a700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb5015c520 0x7ffb500d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:58.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb55f9c700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb50160dd0 0x7ffb500b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:58.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb5579b700 1 -- 172.21.15.97:0/4224099523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb5015c520 msgr2=0x7ffb500d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:58.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb5579b700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb5015c520 0x7ffb500d2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb5579b700 1 -- 172.21.15.97:0/4224099523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb50160dd0 msgr2=0x7ffb500b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:58.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb5579b700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb50160dd0 0x7ffb500b8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.084+0000 7ffb5579b700 1 -- 172.21.15.97:0/4224099523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb48009b30 con 0x7ffb500cd390 2023-07-06T03:55:58.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.085+0000 7ffb5579b700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb500cd390 0x7ffb5015b000 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0011e0 tx=0x7ffb4c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:58.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.085+0000 7ffb45ffb700 1 -- 172.21.15.97:0/4224099523 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c0054e0 con 0x7ffb500cd390 2023-07-06T03:55:58.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.085+0000 7ffb45ffb700 1 -- 172.21.15.97:0/4224099523 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c003680 con 0x7ffb500cd390 2023-07-06T03:55:58.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.085+0000 7ffb579ff700 1 -- 172.21.15.97:0/4224099523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb500b8950 con 0x7ffb500cd390 2023-07-06T03:55:58.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.085+0000 7ffb45ffb700 1 -- 172.21.15.97:0/4224099523 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c003a30 con 0x7ffb500cd390 2023-07-06T03:55:58.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.085+0000 7ffb579ff700 1 -- 172.21.15.97:0/4224099523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb500d3520 con 0x7ffb500cd390 2023-07-06T03:55:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.086+0000 7ffb579ff700 1 -- 172.21.15.97:0/4224099523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5004e340 con 0x7ffb500cd390 2023-07-06T03:55:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.087+0000 7ffb45ffb700 1 -- 172.21.15.97:0/4224099523 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb4c003b90 con 0x7ffb500cd390 2023-07-06T03:55:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.088+0000 7ffb45ffb700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffb3406ace0 0x7ffb3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.088+0000 7ffb45ffb700 1 -- 172.21.15.97:0/4224099523 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ffb4c0886e0 con 0x7ffb500cd390 2023-07-06T03:55:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.088+0000 7ffb54f9a700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffb3406ace0 0x7ffb3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:58.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.089+0000 7ffb54f9a700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffb3406ace0 0x7ffb3406d190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ffb500d0100 tx=0x7ffb40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:58.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.092+0000 7ffb45ffb700 1 -- 172.21.15.97:0/4224099523 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffb4c053aa0 con 0x7ffb500cd390 2023-07-06T03:55:58.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.330+0000 7ffb579ff700 1 -- 172.21.15.97:0/4224099523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb500b4f40 con 0x7ffb500cd390 2023-07-06T03:55:58.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.330+0000 7ffb45ffb700 1 -- 172.21.15.97:0/4224099523 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+3535 (secure 0 0 0) 0x7ffb4c017090 con 0x7ffb500cd390 2023-07-06T03:55:58.333 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:55:58.334 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":97,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3059444241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":3059444241},{"type":"v1","addr":"172.21.15.146:6835","nonce":3059444241}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2624688851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2624688851},{"type":"v1","addr":"172.21.15.97:6837","nonce":2624688851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":8539,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/1058756285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1058756285},{"type":"v1","addr":"172.21.15.146:6834","nonce":1058756285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97},{"gid":9096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3938376776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3938376776},{"type":"v1","addr":"172.21.15.97:6835","nonce":3938376776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[]} 2023-07-06T03:55:58.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 -- 172.21.15.97:0/4224099523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffb3406ace0 msgr2=0x7ffb3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:58.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffb3406ace0 0x7ffb3406d190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ffb500d0100 tx=0x7ffb40008040 comp rx=0 tx=0).stop 2023-07-06T03:55:58.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 -- 172.21.15.97:0/4224099523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb500cd390 msgr2=0x7ffb5015b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:58.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb500cd390 0x7ffb5015b000 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0011e0 tx=0x7ffb4c00cae0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 -- 172.21.15.97:0/4224099523 shutdown_connections 2023-07-06T03:55:58.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ffb3406ace0 0x7ffb3406d190 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb50160dd0 0x7ffb500b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb500cd390 0x7ffb5015b000 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 --2- 172.21.15.97:0/4224099523 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ffb5015c520 0x7ffb500d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 -- 172.21.15.97:0/4224099523 >> 172.21.15.97:0/4224099523 conn(0x7ffb50071790 msgr2=0x7ffb500b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:58.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 -- 172.21.15.97:0/4224099523 shutdown_connections 2023-07-06T03:55:58.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.333+0000 7ffb579ff700 1 -- 172.21.15.97:0/4224099523 wait complete. 2023-07-06T03:55:58.338 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 97 2023-07-06T03:55:58.354 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T03:55:58.355 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T03:55:58.355 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T03:55:58.356 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:55:58.356 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T03:55:58.356 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T03:55:58.357 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T03:55:58.357 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:55:58.358 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T03:55:58.358 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T03:55:58.359 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T03:55:58.359 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:55:58.360 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T03:55:58.360 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T03:55:58.360 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T03:55:58.361 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:55:58.370 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T03:55:58.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T03:55:58.371 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T03:55:58.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T03:55:58.373 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T03:55:58.373 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T03:55:58.374 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T03:55:58.374 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T03:55:58.417 INFO:tasks.ceph.mds.a:Started 2023-07-06T03:55:58.417 DEBUG:teuthology.parallel:result is None 2023-07-06T03:55:58.419 INFO:tasks.ceph.mds.b:Started 2023-07-06T03:55:58.419 DEBUG:teuthology.parallel:result is None 2023-07-06T03:55:58.420 INFO:tasks.ceph.mds.c:Started 2023-07-06T03:55:58.421 DEBUG:teuthology.parallel:result is None 2023-07-06T03:55:58.422 INFO:tasks.ceph.mds.d:Started 2023-07-06T03:55:58.422 DEBUG:teuthology.parallel:result is None 2023-07-06T03:55:58.422 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T03:55:58.567 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-07-06T03:55:58.567 INFO:tasks.ceph.mds.b.smithi146.stdout:starting mds.b at 2023-07-06T03:55:58.568 INFO:tasks.ceph.mds.d.smithi146.stdout:starting mds.d at 2023-07-06T03:55:58.570 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:55:58.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.626+0000 7f235d3ff700 1 Processor -- start 2023-07-06T03:55:58.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.627+0000 7f235d3ff700 1 -- start start 2023-07-06T03:55:58.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.628+0000 7f235d3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f235815c1c0 0x7f23580cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.628+0000 7f235d3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f235815c590 0x7f235815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.628+0000 7f235d3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23580ccc30 con 0x7f23580719c0 2023-07-06T03:55:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.628+0000 7f235d3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23580ccf10 con 0x7f235815c590 2023-07-06T03:55:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.628+0000 7f235d3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23580cd1f0 con 0x7f235815c1c0 2023-07-06T03:55:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.628+0000 7f2356ffd700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f235815c1c0 0x7f23580cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.628+0000 7f23567fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f235815c590 0x7f235815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.628+0000 7f23577fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f23580719c0 0x7f235806a060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:39218/0 (socket says 172.21.15.97:39218) 2023-07-06T03:55:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.629+0000 7f23577fe700 1 -- 172.21.15.97:0/2406227749 learned_addr learned my addr 172.21.15.97:0/2406227749 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:55:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.629+0000 7f2355ffb700 1 -- 172.21.15.97:0/2406227749 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251161709 0 0) 0x7f23580ccc30 con 0x7f23580719c0 2023-07-06T03:55:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.629+0000 7f2355ffb700 1 -- 172.21.15.97:0/2406227749 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23400035b0 con 0x7f23580719c0 2023-07-06T03:55:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.629+0000 7f2356ffd700 1 -- 172.21.15.97:0/2406227749 >> v2:172.21.15.146:3300/0 conn(0x7f235815c590 msgr2=0x7f235815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:55:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.629+0000 7f2356ffd700 1 --2- 172.21.15.97:0/2406227749 >> v2:172.21.15.146:3300/0 conn(0x7f235815c590 0x7f235815d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.629+0000 7f2356ffd700 1 -- 172.21.15.97:0/2406227749 >> v1:172.21.15.146:6789/0 conn(0x7f23580719c0 legacy=0x7f235806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.629+0000 7f2356ffd700 1 -- 172.21.15.97:0/2406227749 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235815cb00 con 0x7f235815c1c0 2023-07-06T03:55:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.629+0000 7f23567fc700 1 --2- 172.21.15.97:0/2406227749 >> v2:172.21.15.146:3300/0 conn(0x7f235815c590 0x7f235815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:55:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.629+0000 7f2356ffd700 1 --2- 172.21.15.97:0/2406227749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f235815c1c0 0x7f23580cc3e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2348009840 tx=0x7f2348009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2e97b27d199640 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.629+0000 7f2355ffb700 1 -- 172.21.15.97:0/2406227749 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348011830 con 0x7f235815c1c0 2023-07-06T03:55:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.630+0000 7f2355ffb700 1 -- 172.21.15.97:0/2406227749 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2348011990 con 0x7f235815c1c0 2023-07-06T03:55:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.630+0000 7f2355ffb700 1 -- 172.21.15.97:0/2406227749 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348011d20 con 0x7f235815c1c0 2023-07-06T03:55:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.630+0000 7f235d3ff700 1 -- 172.21.15.97:0/2406227749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f235815c1c0 msgr2=0x7f23580cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:58.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.630+0000 7f235d3ff700 1 --2- 172.21.15.97:0/2406227749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f235815c1c0 0x7f23580cc3e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2348009840 tx=0x7f2348009b50 comp rx=0 tx=0).stop 2023-07-06T03:55:58.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.630+0000 7f235d3ff700 1 -- 172.21.15.97:0/2406227749 shutdown_connections 2023-07-06T03:55:58.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.630+0000 7f235d3ff700 1 --2- 172.21.15.97:0/2406227749 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f235815c1c0 0x7f23580cc3e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.630+0000 7f235d3ff700 1 --2- 172.21.15.97:0/2406227749 >> v2:172.21.15.146:3300/0 conn(0x7f235815c590 0x7f235815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.630+0000 7f235d3ff700 1 -- 172.21.15.97:0/2406227749 >> 172.21.15.97:0/2406227749 conn(0x7f23580b1650 msgr2=0x7f23580b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:58.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.630+0000 7f235d3ff700 1 -- 172.21.15.97:0/2406227749 shutdown_connections 2023-07-06T03:55:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.630+0000 7f235d3ff700 1 -- 172.21.15.97:0/2406227749 wait complete. 2023-07-06T03:55:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.632+0000 7f235d3ff700 1 Processor -- start 2023-07-06T03:55:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.632+0000 7f235d3ff700 1 -- start start 2023-07-06T03:55:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.633+0000 7f235d3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23580719c0 0x7f23580b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.633+0000 7f235d3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f235815c1c0 0x7f23580b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.633+0000 7f235d3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f235815c590 0x7f23580b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.633+0000 7f235d3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235813fd00 con 0x7f23580719c0 2023-07-06T03:55:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.633+0000 7f235d3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235813ffa0 con 0x7f235815c590 2023-07-06T03:55:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.633+0000 7f235d3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2358140280 con 0x7f235815c1c0 2023-07-06T03:55:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.633+0000 7f2356ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23580719c0 0x7f23580b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.633+0000 7f2356ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23580719c0 0x7f23580b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60138/0 (socket says 172.21.15.97:60138) 2023-07-06T03:55:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.633+0000 7f2356ffd700 1 -- 172.21.15.97:0/2103030808 learned_addr learned my addr 172.21.15.97:0/2103030808 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:55:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f2356ffd700 1 -- 172.21.15.97:0/2103030808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f235815c1c0 msgr2=0x7f23580b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:58.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f2356ffd700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f235815c1c0 0x7f23580b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f2356ffd700 1 -- 172.21.15.97:0/2103030808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f235815c590 msgr2=0x7f23580b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f23577fe700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f235815c590 0x7f23580b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f2356ffd700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f235815c590 0x7f23580b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f2356ffd700 1 -- 172.21.15.97:0/2103030808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c0094a0 con 0x7f23580719c0 2023-07-06T03:55:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f23577fe700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f235815c590 0x7f23580b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:55:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f2356ffd700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23580719c0 0x7f23580b3d50 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f23480090b0 tx=0x7f2348004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f233f7fe700 1 -- 172.21.15.97:0/2103030808 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23480036a0 con 0x7f23580719c0 2023-07-06T03:55:58.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f233f7fe700 1 -- 172.21.15.97:0/2103030808 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234801f070 con 0x7f23580719c0 2023-07-06T03:55:58.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f235d3ff700 1 -- 172.21.15.97:0/2103030808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23480094a0 con 0x7f23580719c0 2023-07-06T03:55:58.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f233f7fe700 1 -- 172.21.15.97:0/2103030808 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348003a40 con 0x7f23580719c0 2023-07-06T03:55:58.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.634+0000 7f235d3ff700 1 -- 172.21.15.97:0/2103030808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23580b38b0 con 0x7f23580719c0 2023-07-06T03:55:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.636+0000 7f235d3ff700 1 -- 172.21.15.97:0/2103030808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235804e340 con 0x7f23580719c0 2023-07-06T03:55:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.636+0000 7f233f7fe700 1 -- 172.21.15.97:0/2103030808 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2348003ba0 con 0x7f23580719c0 2023-07-06T03:55:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.637+0000 7f233f7fe700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f234406ace0 0x7f234406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:55:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.637+0000 7f233f7fe700 1 -- 172.21.15.97:0/2103030808 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2348021070 con 0x7f23580719c0 2023-07-06T03:55:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.637+0000 7f23567fc700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f234406ace0 0x7f234406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:55:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.638+0000 7f23567fc700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f234406ace0 0x7f234406d190 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f234c006f10 tx=0x7f234c006e60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:55:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.641+0000 7f233f7fe700 1 -- 172.21.15.97:0/2103030808 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2348055e60 con 0x7f23580719c0 2023-07-06T03:55:58.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:58.863+0000 7f235d3ff700 1 -- 172.21.15.97:0/2103030808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2358071290 con 0x7f23580719c0 2023-07-06T03:55:59.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.829+0000 7f233f7fe700 1 -- 172.21.15.97:0/2103030808 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v132) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2348015080 con 0x7f23580719c0 2023-07-06T03:55:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.832+0000 7f235d3ff700 1 -- 172.21.15.97:0/2103030808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f234406ace0 msgr2=0x7f234406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.832+0000 7f235d3ff700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f234406ace0 0x7f234406d190 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f234c006f10 tx=0x7f234c006e60 comp rx=0 tx=0).stop 2023-07-06T03:55:59.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.832+0000 7f235d3ff700 1 -- 172.21.15.97:0/2103030808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23580719c0 msgr2=0x7f23580b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:55:59.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.832+0000 7f235d3ff700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23580719c0 0x7f23580b3d50 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f23480090b0 tx=0x7f2348004750 comp rx=0 tx=0).stop 2023-07-06T03:55:59.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.832+0000 7f235d3ff700 1 -- 172.21.15.97:0/2103030808 shutdown_connections 2023-07-06T03:55:59.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.832+0000 7f235d3ff700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f234406ace0 0x7f234406d190 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:59.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.832+0000 7f235d3ff700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f235815c590 0x7f23580b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:59.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.832+0000 7f235d3ff700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23580719c0 0x7f23580b3d50 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:59.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.832+0000 7f235d3ff700 1 --2- 172.21.15.97:0/2103030808 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f235815c1c0 0x7f23580b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:55:59.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.833+0000 7f235d3ff700 1 -- 172.21.15.97:0/2103030808 >> 172.21.15.97:0/2103030808 conn(0x7f23580b1650 msgr2=0x7f235806a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:55:59.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.833+0000 7f235d3ff700 1 -- 172.21.15.97:0/2103030808 shutdown_connections 2023-07-06T03:55:59.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:55:59.833+0000 7f235d3ff700 1 -- 172.21.15.97:0/2103030808 wait complete. 2023-07-06T03:55:59.837 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-07-06T03:55:59.855 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T03:56:00.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.053+0000 7fe1821ff700 1 Processor -- start 2023-07-06T03:56:00.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.053+0000 7fe1821ff700 1 -- start start 2023-07-06T03:56:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.054+0000 7fe1821ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c15c520 0x7fe17c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.054+0000 7fe1821ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe17c160180 0x7fe17c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.054+0000 7fe1821ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c0cdae0 con 0x7fe17c160b30 2023-07-06T03:56:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.054+0000 7fe1821ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17c0cd8d0 con 0x7fe17c160180 2023-07-06T03:56:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.054+0000 7fe1821ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0b6640 con 0x7fe17c15c520 2023-07-06T03:56:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.054+0000 7fe17affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe17c160180 0x7fe17c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.054+0000 7fe17affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe17c160180 0x7fe17c1605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42106/0 (socket says 172.21.15.97:42106) 2023-07-06T03:56:00.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.054+0000 7fe17affd700 1 -- 172.21.15.97:0/2763643686 learned_addr learned my addr 172.21.15.97:0/2763643686 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:00.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.054+0000 7fe17b7fe700 1 --2- 172.21.15.97:0/2763643686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c15c520 0x7fe17c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.055+0000 7fe17affd700 1 -- 172.21.15.97:0/2763643686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c15c520 msgr2=0x7fe17c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:00.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.055+0000 7fe17affd700 1 --2- 172.21.15.97:0/2763643686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c15c520 0x7fe17c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.055+0000 7fe17affd700 1 -- 172.21.15.97:0/2763643686 >> v1:172.21.15.146:6789/0 conn(0x7fe17c160b30 legacy=0x7fe17c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.055+0000 7fe17affd700 1 -- 172.21.15.97:0/2763643686 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0b6920 con 0x7fe17c160180 2023-07-06T03:56:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.055+0000 7fe17affd700 1 --2- 172.21.15.97:0/2763643686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe17c160180 0x7fe17c1605f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098f0 tx=0x7fe170009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ca092a9d18aed7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.055+0000 7fe17a7fc700 1 -- 172.21.15.97:0/2763643686 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000b8a0 con 0x7fe17c160180 2023-07-06T03:56:00.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.055+0000 7fe17a7fc700 1 -- 172.21.15.97:0/2763643686 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17000ba00 con 0x7fe17c160180 2023-07-06T03:56:00.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.055+0000 7fe17a7fc700 1 -- 172.21.15.97:0/2763643686 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000bd90 con 0x7fe17c160180 2023-07-06T03:56:00.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.056+0000 7fe1821ff700 1 -- 172.21.15.97:0/2763643686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe17c160180 msgr2=0x7fe17c1605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.056+0000 7fe1821ff700 1 --2- 172.21.15.97:0/2763643686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe17c160180 0x7fe17c1605f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098f0 tx=0x7fe170009c00 comp rx=0 tx=0).stop 2023-07-06T03:56:00.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.056+0000 7fe1821ff700 1 -- 172.21.15.97:0/2763643686 shutdown_connections 2023-07-06T03:56:00.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.056+0000 7fe1821ff700 1 --2- 172.21.15.97:0/2763643686 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c15c520 0x7fe17c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.056+0000 7fe1821ff700 1 --2- 172.21.15.97:0/2763643686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe17c160180 0x7fe17c1605f0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.056+0000 7fe1821ff700 1 -- 172.21.15.97:0/2763643686 >> 172.21.15.97:0/2763643686 conn(0x7fe17c071790 msgr2=0x7fe17c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:00.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.056+0000 7fe1821ff700 1 -- 172.21.15.97:0/2763643686 shutdown_connections 2023-07-06T03:56:00.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.056+0000 7fe1821ff700 1 -- 172.21.15.97:0/2763643686 wait complete. 2023-07-06T03:56:00.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.057+0000 7fe1821ff700 1 Processor -- start 2023-07-06T03:56:00.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.058+0000 7fe1821ff700 1 -- start start 2023-07-06T03:56:00.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.058+0000 7fe1821ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe17c15c520 0x7fe17c0d3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.058+0000 7fe1821ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160180 0x7fe17c0c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.058+0000 7fe1821ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17c160b30 0x7fe17c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe1821ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0d3c40 con 0x7fe17c15c520 2023-07-06T03:56:00.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe1821ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c1f1bd0 con 0x7fe17c160b30 2023-07-06T03:56:00.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe1821ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0d1210 con 0x7fe17c160180 2023-07-06T03:56:00.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe17b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe17c15c520 0x7fe17c0d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe17b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe17c15c520 0x7fe17c0d3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42122/0 (socket says 172.21.15.97:42122) 2023-07-06T03:56:00.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe17b7fe700 1 -- 172.21.15.97:0/599556202 learned_addr learned my addr 172.21.15.97:0/599556202 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:00.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe17affd700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160180 0x7fe17c0c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe17bfff700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17c160b30 0x7fe17c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe17b7fe700 1 -- 172.21.15.97:0/599556202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160180 msgr2=0x7fe17c0c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe17b7fe700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160180 0x7fe17c0c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe17b7fe700 1 -- 172.21.15.97:0/599556202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17c160b30 msgr2=0x7fe17c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe17b7fe700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17c160b30 0x7fe17c0d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe17b7fe700 1 -- 172.21.15.97:0/599556202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe170009550 con 0x7fe17c15c520 2023-07-06T03:56:00.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.059+0000 7fe17b7fe700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe17c15c520 0x7fe17c0d3580 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00d830 tx=0x7fe16c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:00.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.060+0000 7fe163fff700 1 -- 172.21.15.97:0/599556202 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0125f0 con 0x7fe17c15c520 2023-07-06T03:56:00.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.060+0000 7fe163fff700 1 -- 172.21.15.97:0/599556202 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c012750 con 0x7fe17c15c520 2023-07-06T03:56:00.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.060+0000 7fe1821ff700 1 -- 172.21.15.97:0/599556202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c15ed80 con 0x7fe17c15c520 2023-07-06T03:56:00.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.060+0000 7fe163fff700 1 -- 172.21.15.97:0/599556202 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c012b00 con 0x7fe17c15c520 2023-07-06T03:56:00.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.060+0000 7fe1821ff700 1 -- 172.21.15.97:0/599556202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c15f340 con 0x7fe17c15c520 2023-07-06T03:56:00.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.061+0000 7fe1821ff700 1 -- 172.21.15.97:0/599556202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c15f990 con 0x7fe17c15c520 2023-07-06T03:56:00.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.062+0000 7fe163fff700 1 -- 172.21.15.97:0/599556202 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe16c012c60 con 0x7fe17c15c520 2023-07-06T03:56:00.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.062+0000 7fe163fff700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16806ace0 0x7fe16806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.063+0000 7fe163fff700 1 -- 172.21.15.97:0/599556202 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe16c0891b0 con 0x7fe17c15c520 2023-07-06T03:56:00.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.063+0000 7fe17affd700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16806ace0 0x7fe16806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.063+0000 7fe17affd700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16806ace0 0x7fe16806d190 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe170009c40 tx=0x7fe170008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:00.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.067+0000 7fe163fff700 1 -- 172.21.15.97:0/599556202 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe16c054510 con 0x7fe17c15c520 2023-07-06T03:56:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.336+0000 7fe1821ff700 1 -- 172.21.15.97:0/599556202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe17c04e340 con 0x7fe17c15c520 2023-07-06T03:56:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.336+0000 7fe163fff700 1 -- 172.21.15.97:0/599556202 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe16c057b30 con 0x7fe17c15c520 2023-07-06T03:56:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:00.340 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:56:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-07-06T03:56:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-07-06T03:56:00.340 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkczuvGRAAUKjfmMyKOTCYzOQmtf3n3A==", 2023-07-06T03:56:00.341 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.341 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:56:00.341 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:56:00.341 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:56:00.341 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.341 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.342 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-07-06T03:56:00.342 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkFHvUMBAA2WgMcUUf+Qi8T2ZjhfWaHA==", 2023-07-06T03:56:00.342 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.342 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:56:00.342 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:56:00.342 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:56:00.342 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.343 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-07-06T03:56:00.343 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZk0N4+JRAA3tjqoIBPpZ6C7TMmOOg7lw==", 2023-07-06T03:56:00.343 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.343 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:56:00.343 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:56:00.344 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:56:00.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.344 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-07-06T03:56:00.344 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkJnHUABAAwIMH2DmoHaQN3frJCgRiQw==", 2023-07-06T03:56:00.344 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.345 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:56:00.345 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:56:00.345 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:56:00.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.345 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-07-06T03:56:00.346 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCLOKZkShOIABAAzIBHDcNkmnDOLSY7Qdb+FA==", 2023-07-06T03:56:00.346 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.346 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:56:00.346 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:56:00.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.347 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-07-06T03:56:00.347 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCOOKZkq1t7AxAAwihuhimbjitZDeki1QuMug==", 2023-07-06T03:56:00.347 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.347 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:56:00.347 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:56:00.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.348 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-07-06T03:56:00.348 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCROKZkdYv4BhAApTHg/pmlC0lO7hVTkimEgw==", 2023-07-06T03:56:00.348 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.348 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:56:00.348 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:56:00.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.349 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-07-06T03:56:00.349 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCUOKZkqhuMChAAsV9xG6g1dofrpWt3F8tw5w==", 2023-07-06T03:56:00.349 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.349 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:56:00.350 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:56:00.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.350 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-07-06T03:56:00.350 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCaOKZk1Q5aIBAABU1+afwIihm+g4xylryz/A==", 2023-07-06T03:56:00.350 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.351 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:56:00.351 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:56:00.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.351 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.351 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-07-06T03:56:00.351 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCdOKZkRskIJBAA+821aoG98aEV7h4FTINQFA==", 2023-07-06T03:56:00.352 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.352 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:56:00.352 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:56:00.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.352 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-07-06T03:56:00.353 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCgOKZk9qnMJxAATcWG+a/9Hl1AuIL/TSrdEA==", 2023-07-06T03:56:00.353 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.353 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:56:00.353 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:56:00.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.353 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.354 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-07-06T03:56:00.354 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCjOKZk6wJ/KxAAWnZxt7h7OJhQ+bqdel4+iQ==", 2023-07-06T03:56:00.354 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.354 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-07-06T03:56:00.354 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-07-06T03:56:00.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.354 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.355 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-07-06T03:56:00.355 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg==", 2023-07-06T03:56:00.355 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.355 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:56:00.355 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-07-06T03:56:00.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-07-06T03:56:00.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.356 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-07-06T03:56:00.356 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkq4/pGRAAucHmR1eBA8MNMy3/l7tl0g==", 2023-07-06T03:56:00.356 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.356 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:56:00.356 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:56:00.357 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:56:00.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:56:00.357 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.357 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-07-06T03:56:00.357 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkhL5fEhAAWhw4IbCPqTO2sXPIcpTVHA==", 2023-07-06T03:56:00.357 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.358 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:56:00.358 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:56:00.358 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:56:00.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:56:00.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.358 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-07-06T03:56:00.359 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCHOKZkO81dIRAAffjVyvsb/ygBtx8UrYQK0w==", 2023-07-06T03:56:00.359 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.359 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-07-06T03:56:00.359 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-07-06T03:56:00.359 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-07-06T03:56:00.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-07-06T03:56:00.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.360 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-07-06T03:56:00.360 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCFOKZk6uVlHRAA92RDhdDKW/q65SEOQMWgiA==", 2023-07-06T03:56:00.360 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.360 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:56:00.360 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-07-06T03:56:00.360 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-07-06T03:56:00.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.361 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.361 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-07-06T03:56:00.361 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkT4ALEhAAXwwkcodQU9meJA0tfseJCg==", 2023-07-06T03:56:00.361 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.361 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:56:00.362 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:56:00.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:00.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:00.362 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-07-06T03:56:00.362 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQCGOKZkfnoEChAAuRxqXqPI33dgWkBJoUMh3w==", 2023-07-06T03:56:00.362 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-07-06T03:56:00.363 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-07-06T03:56:00.363 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-07-06T03:56:00.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-07-06T03:56:00.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:00.363 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:56:00.363 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:56:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 -- 172.21.15.97:0/599556202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16806ace0 msgr2=0x7fe16806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16806ace0 0x7fe16806d190 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe170009c40 tx=0x7fe170008030 comp rx=0 tx=0).stop 2023-07-06T03:56:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 -- 172.21.15.97:0/599556202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe17c15c520 msgr2=0x7fe17c0d3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe17c15c520 0x7fe17c0d3580 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00d830 tx=0x7fe16c00db40 comp rx=0 tx=0).stop 2023-07-06T03:56:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 -- 172.21.15.97:0/599556202 shutdown_connections 2023-07-06T03:56:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16806ace0 0x7fe16806d190 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17c160b30 0x7fe17c0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe17c15c520 0x7fe17c0d3580 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 --2- 172.21.15.97:0/599556202 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160180 0x7fe17c0c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 -- 172.21.15.97:0/599556202 >> 172.21.15.97:0/599556202 conn(0x7fe17c071790 msgr2=0x7fe17c157850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 -- 172.21.15.97:0/599556202 shutdown_connections 2023-07-06T03:56:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.339+0000 7fe1821ff700 1 -- 172.21.15.97:0/599556202 wait complete. 2023-07-06T03:56:00.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:56:00.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.559+0000 7fa57c9ff700 1 Processor -- start 2023-07-06T03:56:00.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.559+0000 7fa57c9ff700 1 -- start start 2023-07-06T03:56:00.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.560+0000 7fa57c9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa57815c520 0x7fa5780cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.560+0000 7fa57c9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa578160d00 0x7fa5780b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.560+0000 7fa57c9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5780cdae0 con 0x7fa578160180 2023-07-06T03:56:00.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.560+0000 7fa57c9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5780cd8d0 con 0x7fa57815c520 2023-07-06T03:56:00.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.560+0000 7fa57c9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5780b6640 con 0x7fa578160d00 2023-07-06T03:56:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.561+0000 7fa57659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa57815c520 0x7fa5780cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.561+0000 7fa56ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa578160180 0x7fa5781605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46460/0 (socket says 172.21.15.97:46460) 2023-07-06T03:56:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.561+0000 7fa56ffff700 1 -- 172.21.15.97:0/2077259226 learned_addr learned my addr 172.21.15.97:0/2077259226 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:00.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.561+0000 7fa576d9d700 1 --2- 172.21.15.97:0/2077259226 >> v2:172.21.15.146:3300/0 conn(0x7fa578160d00 0x7fa5780b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.561+0000 7fa57659c700 1 -- 172.21.15.97:0/2077259226 >> v2:172.21.15.146:3300/0 conn(0x7fa578160d00 msgr2=0x7fa5780b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:00.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.561+0000 7fa57659c700 1 --2- 172.21.15.97:0/2077259226 >> v2:172.21.15.146:3300/0 conn(0x7fa578160d00 0x7fa5780b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.561+0000 7fa57659c700 1 -- 172.21.15.97:0/2077259226 >> v1:172.21.15.97:6789/0 conn(0x7fa578160180 legacy=0x7fa5781605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.561+0000 7fa57659c700 1 -- 172.21.15.97:0/2077259226 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5780b6920 con 0x7fa57815c520 2023-07-06T03:56:00.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.561+0000 7fa575d9b700 1 -- 172.21.15.97:0/2077259226 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853569964 0 0) 0x7fa5780cdae0 con 0x7fa578160180 2023-07-06T03:56:00.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.561+0000 7fa57659c700 1 --2- 172.21.15.97:0/2077259226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57815c520 0x7fa5780cd390 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa5780cd320 tx=0x7fa568009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c95aaa1d99e14d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:00.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.562+0000 7fa575d9b700 1 -- 172.21.15.97:0/2077259226 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568011830 con 0x7fa57815c520 2023-07-06T03:56:00.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.562+0000 7fa575d9b700 1 -- 172.21.15.97:0/2077259226 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa568011990 con 0x7fa57815c520 2023-07-06T03:56:00.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.562+0000 7fa575d9b700 1 -- 172.21.15.97:0/2077259226 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568011d20 con 0x7fa57815c520 2023-07-06T03:56:00.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.562+0000 7fa57c9ff700 1 -- 172.21.15.97:0/2077259226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57815c520 msgr2=0x7fa5780cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.562+0000 7fa57c9ff700 1 --2- 172.21.15.97:0/2077259226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57815c520 0x7fa5780cd390 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa5780cd320 tx=0x7fa568009b20 comp rx=0 tx=0).stop 2023-07-06T03:56:00.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.562+0000 7fa57c9ff700 1 -- 172.21.15.97:0/2077259226 shutdown_connections 2023-07-06T03:56:00.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.562+0000 7fa57c9ff700 1 --2- 172.21.15.97:0/2077259226 >> v2:172.21.15.146:3300/0 conn(0x7fa578160d00 0x7fa5780b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.562+0000 7fa57c9ff700 1 --2- 172.21.15.97:0/2077259226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57815c520 0x7fa5780cd390 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.562+0000 7fa57c9ff700 1 -- 172.21.15.97:0/2077259226 >> 172.21.15.97:0/2077259226 conn(0x7fa578071790 msgr2=0x7fa578071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.562+0000 7fa57c9ff700 1 -- 172.21.15.97:0/2077259226 shutdown_connections 2023-07-06T03:56:00.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.562+0000 7fa57c9ff700 1 -- 172.21.15.97:0/2077259226 wait complete. 2023-07-06T03:56:00.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.564+0000 7fa57c9ff700 1 Processor -- start 2023-07-06T03:56:00.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.564+0000 7fa57c9ff700 1 -- start start 2023-07-06T03:56:00.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.565+0000 7fa57c9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57815c520 0x7fa57815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.565+0000 7fa57c9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa578160180 0x7fa5780cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.565+0000 7fa57c9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa578160d00 0x7fa5780cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.565+0000 7fa57c9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780cb0d0 con 0x7fa57815c520 2023-07-06T03:56:00.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.565+0000 7fa57c9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780cadf0 con 0x7fa578160180 2023-07-06T03:56:00.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.565+0000 7fa57c9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780cb4f0 con 0x7fa578160d00 2023-07-06T03:56:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.565+0000 7fa57659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57815c520 0x7fa57815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.565+0000 7fa57659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57815c520 0x7fa57815d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42154/0 (socket says 172.21.15.97:42154) 2023-07-06T03:56:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.565+0000 7fa57659c700 1 -- 172.21.15.97:0/3020534268 learned_addr learned my addr 172.21.15.97:0/3020534268 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.565+0000 7fa56ffff700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa578160180 0x7fa5780cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.565+0000 7fa576d9d700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa578160d00 0x7fa5780cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa57659c700 1 -- 172.21.15.97:0/3020534268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa578160d00 msgr2=0x7fa5780cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa57659c700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa578160d00 0x7fa5780cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa57659c700 1 -- 172.21.15.97:0/3020534268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa578160180 msgr2=0x7fa5780cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa57659c700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa578160180 0x7fa5780cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa57659c700 1 -- 172.21.15.97:0/3020534268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5680094a0 con 0x7fa57815c520 2023-07-06T03:56:00.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa576d9d700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa578160d00 0x7fa5780cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:00.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa57659c700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57815c520 0x7fa57815d5a0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fa568014040 tx=0x7fa568004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:00.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa56effd700 1 -- 172.21.15.97:0/3020534268 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680036a0 con 0x7fa57815c520 2023-07-06T03:56:00.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa56effd700 1 -- 172.21.15.97:0/3020534268 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56801e070 con 0x7fa57815c520 2023-07-06T03:56:00.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa56effd700 1 -- 172.21.15.97:0/3020534268 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568003a00 con 0x7fa57815c520 2023-07-06T03:56:00.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa57c9ff700 1 -- 172.21.15.97:0/3020534268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5780cb7d0 con 0x7fa57815c520 2023-07-06T03:56:00.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.566+0000 7fa57c9ff700 1 -- 172.21.15.97:0/3020534268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5780caa30 con 0x7fa57815c520 2023-07-06T03:56:00.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.568+0000 7fa57c9ff700 1 -- 172.21.15.97:0/3020534268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5780cbcf0 con 0x7fa57815c520 2023-07-06T03:56:00.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.568+0000 7fa56effd700 1 -- 172.21.15.97:0/3020534268 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa568003b60 con 0x7fa57815c520 2023-07-06T03:56:00.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.569+0000 7fa56effd700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa55806ace0 0x7fa55806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.569+0000 7fa56effd700 1 -- 172.21.15.97:0/3020534268 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa56808a190 con 0x7fa57815c520 2023-07-06T03:56:00.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.570+0000 7fa56ffff700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa55806ace0 0x7fa55806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.570+0000 7fa56ffff700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa55806ace0 0x7fa55806d190 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa57815f160 tx=0x7fa560007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:00.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.574+0000 7fa56effd700 1 -- 172.21.15.97:0/3020534268 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa5780cbcf0 con 0x7fa57815c520 2023-07-06T03:56:00.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.769+0000 7fa57c9ff700 1 -- 172.21.15.97:0/3020534268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5780b38f0 con 0x7fa57815c520 2023-07-06T03:56:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.771+0000 7fa56effd700 1 -- 172.21.15.97:0/3020534268 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+14675 (secure 0 0 0) 0x7fa568055620 con 0x7fa57815c520 2023-07-06T03:56:00.773 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:00.774 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":132,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:55:59.820840+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:56:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 -- 172.21.15.97:0/3020534268 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa55806ace0 msgr2=0x7fa55806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa55806ace0 0x7fa55806d190 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa57815f160 tx=0x7fa560007de0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 -- 172.21.15.97:0/3020534268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57815c520 msgr2=0x7fa57815d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57815c520 0x7fa57815d5a0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fa568014040 tx=0x7fa568004750 comp rx=0 tx=0).stop 2023-07-06T03:56:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 -- 172.21.15.97:0/3020534268 shutdown_connections 2023-07-06T03:56:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa55806ace0 0x7fa55806d190 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa578160d00 0x7fa5780cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57815c520 0x7fa57815d5a0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 --2- 172.21.15.97:0/3020534268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa578160180 0x7fa5780cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 -- 172.21.15.97:0/3020534268 >> 172.21.15.97:0/3020534268 conn(0x7fa578071790 msgr2=0x7fa578071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 -- 172.21.15.97:0/3020534268 shutdown_connections 2023-07-06T03:56:00.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.773+0000 7fa57c9ff700 1 -- 172.21.15.97:0/3020534268 wait complete. 2023-07-06T03:56:00.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:56:00.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:56:00.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.989+0000 7f10827ff700 1 Processor -- start 2023-07-06T03:56:00.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.989+0000 7f10827ff700 1 -- start start 2023-07-06T03:56:00.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.990+0000 7f10827ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f107c15c610 0x7f107c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.990+0000 7f10827ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107c0cff90 con 0x7f107c06b360 2023-07-06T03:56:00.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.990+0000 7f10827ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107c0b5650 con 0x7f107c0d2210 2023-07-06T03:56:00.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.990+0000 7f10827ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c0d0780 con 0x7f107c15c610 2023-07-06T03:56:00.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.990+0000 7f107b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f107c06b360 0x7f107c0b4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46474/0 (socket says 172.21.15.97:46474) 2023-07-06T03:56:00.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.990+0000 7f107b7fe700 1 -- 172.21.15.97:0/1878167921 learned_addr learned my addr 172.21.15.97:0/1878167921 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:00.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.990+0000 7f107bfff700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f107c15c610 0x7f107c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.990+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583709566 0 0) 0x7f107c0cff90 con 0x7f107c06b360 2023-07-06T03:56:00.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10640035b0 con 0x7f107c06b360 2023-07-06T03:56:00.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807832914 0 0) 0x7f107c0b5650 con 0x7f107c0d2210 2023-07-06T03:56:00.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f107c0cff90 con 0x7f107c0d2210 2023-07-06T03:56:00.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 612088514 0 0) 0x7f10640035b0 con 0x7f107c06b360 2023-07-06T03:56:00.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f107c15c610 msgr2=0x7f107c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:00.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f107c15c610 0x7f107c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 >> v1:172.21.15.146:6789/0 conn(0x7f107c0d2210 legacy=0x7f107c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c0b5400 con 0x7f107c06b360 2023-07-06T03:56:00.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f1070001670 con 0x7f107c06b360 2023-07-06T03:56:00.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 >> v1:172.21.15.97:6789/0 conn(0x7f107c06b360 legacy=0x7f107c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:00.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1064003bc0 0x7f1064003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1064004850 0x7f1064004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 0x7f106400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:00.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c0d0780 con 0x7f1064005200 2023-07-06T03:56:00.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106400bfd0 con 0x7f1064004850 2023-07-06T03:56:00.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106400c190 con 0x7f1064003bc0 2023-07-06T03:56:00.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f1080d9c700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 0x7f106400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107bfff700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1064003bc0 0x7f1064003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.991+0000 7f107b7fe700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1064004850 0x7f1064004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:00.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.992+0000 7f1080d9c700 1 -- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1064003bc0 msgr2=0x7f1064003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:00.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.992+0000 7f1080d9c700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1064003bc0 0x7f1064003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.992+0000 7f1080d9c700 1 -- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1064004850 msgr2=0x7f1064004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:00.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.992+0000 7f1080d9c700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1064004850 0x7f1064004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:00.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.992+0000 7f1080d9c700 1 -- 172.21.15.97:0/1878167921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10640035b0 con 0x7f1064005200 2023-07-06T03:56:00.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.992+0000 7f107bfff700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1064003bc0 0x7f1064003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:56:01.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.992+0000 7f1080d9c700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 0x7f106400ba10 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f107400c4c0 tx=0x7f1064005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed1baf75b507926 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:01.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.992+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074003a00 con 0x7f1064005200 2023-07-06T03:56:01.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.992+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1074003b60 con 0x7f1064005200 2023-07-06T03:56:01.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.992+0000 7f107affd700 1 -- 172.21.15.97:0/1878167921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10740050d0 con 0x7f1064005200 2023-07-06T03:56:01.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.993+0000 7f10827ff700 1 -- 172.21.15.97:0/1878167921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 msgr2=0x7f106400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.993+0000 7f10827ff700 1 --2- 172.21.15.97:0/1878167921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 0x7f106400ba10 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f107400c4c0 tx=0x7f1064005880 comp rx=0 tx=0).stop 2023-07-06T03:56:01.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.993+0000 7f1080d9c700 1 -- 172.21.15.97:0/1878167921 reap_dead start 2023-07-06T03:56:01.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.993+0000 7f10827ff700 1 -- 172.21.15.97:0/1878167921 shutdown_connections 2023-07-06T03:56:01.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.993+0000 7f10827ff700 1 -- 172.21.15.97:0/1878167921 >> 172.21.15.97:0/1878167921 conn(0x7f107c06a6c0 msgr2=0x7f107c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:01.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.993+0000 7f10827ff700 1 -- 172.21.15.97:0/1878167921 shutdown_connections 2023-07-06T03:56:01.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.993+0000 7f10827ff700 1 -- 172.21.15.97:0/1878167921 wait complete. 2023-07-06T03:56:01.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.995+0000 7f10827ff700 1 Processor -- start 2023-07-06T03:56:01.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.995+0000 7f10827ff700 1 -- start start 2023-07-06T03:56:01.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f10827ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 0x7f107c0b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f10827ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f107c13e0d0 0x7f107c0b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f10827ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f107c13e540 0x7f107c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f10827ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c14f870 con 0x7f1064005200 2023-07-06T03:56:01.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f10827ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c0cb080 con 0x7f107c13e540 2023-07-06T03:56:01.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f10827ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c0b84a0 con 0x7f107c13e0d0 2023-07-06T03:56:01.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f107bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 0x7f107c0b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f107bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 0x7f107c0b7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42170/0 (socket says 172.21.15.97:42170) 2023-07-06T03:56:01.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f107bfff700 1 -- 172.21.15.97:0/1229155363 learned_addr learned my addr 172.21.15.97:0/1229155363 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:01.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f107bfff700 1 -- 172.21.15.97:0/1229155363 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f107c13e0d0 msgr2=0x7f107c0b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f107bfff700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f107c13e0d0 0x7f107c0b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f107bfff700 1 -- 172.21.15.97:0/1229155363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f107c13e540 msgr2=0x7f107c0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f107bfff700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f107c13e540 0x7f107c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.997+0000 7f107bfff700 1 -- 172.21.15.97:0/1229155363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c0b8780 con 0x7f1064005200 2023-07-06T03:56:01.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.998+0000 7f107bfff700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 0x7f107c0b7b50 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f106c011c00 tx=0x7f106c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:01.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.998+0000 7f1078ff9700 1 -- 172.21.15.97:0/1229155363 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00f980 con 0x7f1064005200 2023-07-06T03:56:01.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.998+0000 7f1078ff9700 1 -- 172.21.15.97:0/1229155363 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c00fae0 con 0x7f1064005200 2023-07-06T03:56:01.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.998+0000 7f1078ff9700 1 -- 172.21.15.97:0/1229155363 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00fe50 con 0x7f1064005200 2023-07-06T03:56:01.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.998+0000 7f10827ff700 1 -- 172.21.15.97:0/1229155363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107c0b8a60 con 0x7f1064005200 2023-07-06T03:56:01.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.998+0000 7f10827ff700 1 -- 172.21.15.97:0/1229155363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107c0b8270 con 0x7f1064005200 2023-07-06T03:56:01.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:00.999+0000 7f10827ff700 1 -- 172.21.15.97:0/1229155363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107c0cae50 con 0x7f1064005200 2023-07-06T03:56:01.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.000+0000 7f1078ff9700 1 -- 172.21.15.97:0/1229155363 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f106c004020 con 0x7f1064005200 2023-07-06T03:56:01.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.001+0000 7f1078ff9700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106806ace0 0x7f106806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.001+0000 7f1078ff9700 1 -- 172.21.15.97:0/1229155363 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f106c089640 con 0x7f1064005200 2023-07-06T03:56:01.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.001+0000 7f107b7fe700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106806ace0 0x7f106806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.002+0000 7f107b7fe700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106806ace0 0x7f106806d190 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1070000f60 tx=0x7f1070000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:01.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.005+0000 7f1078ff9700 1 -- 172.21.15.97:0/1229155363 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f107c0cae50 con 0x7f1064005200 2023-07-06T03:56:01.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.204+0000 7f10827ff700 1 -- 172.21.15.97:0/1229155363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f107c0cae50 con 0x7f1064005200 2023-07-06T03:56:01.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.205+0000 7f1078ff9700 1 -- 172.21.15.97:0/1229155363 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7f107c0cae50 con 0x7f1064005200 2023-07-06T03:56:01.208 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:56:01.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 -- 172.21.15.97:0/1229155363 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106806ace0 msgr2=0x7f106806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106806ace0 0x7f106806d190 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1070000f60 tx=0x7f1070000ed0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 -- 172.21.15.97:0/1229155363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 msgr2=0x7f107c0b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 0x7f107c0b7b50 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f106c011c00 tx=0x7f106c009ed0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 -- 172.21.15.97:0/1229155363 shutdown_connections 2023-07-06T03:56:01.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f106806ace0 0x7f106806d190 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f107c13e540 0x7f107c0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1064005200 0x7f107c0b7b50 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 --2- 172.21.15.97:0/1229155363 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f107c13e0d0 0x7f107c0b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 -- 172.21.15.97:0/1229155363 >> 172.21.15.97:0/1229155363 conn(0x7f107c06a6c0 msgr2=0x7f107c0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:01.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 -- 172.21.15.97:0/1229155363 shutdown_connections 2023-07-06T03:56:01.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.207+0000 7f10827ff700 1 -- 172.21.15.97:0/1229155363 wait complete. 2023-07-06T03:56:01.228 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:56:01.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.422+0000 7fc52d1ff700 1 Processor -- start 2023-07-06T03:56:01.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.422+0000 7fc52d1ff700 1 -- start start 2023-07-06T03:56:01.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.423+0000 7fc52d1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc528159070 0x7fc5280c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.423+0000 7fc52d1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5280ccbd0 0x7fc5280cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.423+0000 7fc52d1ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280ca260 con 0x7fc5280c3cd0 2023-07-06T03:56:01.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.423+0000 7fc52d1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5280c99c0 con 0x7fc528159070 2023-07-06T03:56:01.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.423+0000 7fc52d1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280c9ca0 con 0x7fc5280ccbd0 2023-07-06T03:56:01.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.424+0000 7fc526d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc528159070 0x7fc5280c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.424+0000 7fc52659c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc5280c3cd0 0x7fc5280cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46478/0 (socket says 172.21.15.97:46478) 2023-07-06T03:56:01.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.424+0000 7fc52659c700 1 -- 172.21.15.97:0/1223103282 learned_addr learned my addr 172.21.15.97:0/1223103282 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:01.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.424+0000 7fc52759e700 1 --2- 172.21.15.97:0/1223103282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5280ccbd0 0x7fc5280cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.424+0000 7fc526d9d700 1 -- 172.21.15.97:0/1223103282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5280ccbd0 msgr2=0x7fc5280cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:01.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.424+0000 7fc526d9d700 1 --2- 172.21.15.97:0/1223103282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5280ccbd0 0x7fc5280cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.424+0000 7fc526d9d700 1 -- 172.21.15.97:0/1223103282 >> v1:172.21.15.97:6789/0 conn(0x7fc5280c3cd0 legacy=0x7fc5280cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.424+0000 7fc526d9d700 1 -- 172.21.15.97:0/1223103282 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280c9f80 con 0x7fc528159070 2023-07-06T03:56:01.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.424+0000 7fc525d9b700 1 -- 172.21.15.97:0/1223103282 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855325399 0 0) 0x7fc5280ca260 con 0x7fc5280c3cd0 2023-07-06T03:56:01.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.424+0000 7fc526d9d700 1 --2- 172.21.15.97:0/1223103282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc528159070 0x7fc5280c8f40 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fc518009840 tx=0x7fc518009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1ec86a1775d25a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:01.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.425+0000 7fc525d9b700 1 -- 172.21.15.97:0/1223103282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518011830 con 0x7fc528159070 2023-07-06T03:56:01.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.425+0000 7fc525d9b700 1 -- 172.21.15.97:0/1223103282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc518011990 con 0x7fc528159070 2023-07-06T03:56:01.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.425+0000 7fc525d9b700 1 -- 172.21.15.97:0/1223103282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518011d20 con 0x7fc528159070 2023-07-06T03:56:01.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.425+0000 7fc52d1ff700 1 -- 172.21.15.97:0/1223103282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc528159070 msgr2=0x7fc5280c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.425+0000 7fc52d1ff700 1 --2- 172.21.15.97:0/1223103282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc528159070 0x7fc5280c8f40 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fc518009840 tx=0x7fc518009b50 comp rx=0 tx=0).stop 2023-07-06T03:56:01.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.425+0000 7fc52d1ff700 1 -- 172.21.15.97:0/1223103282 shutdown_connections 2023-07-06T03:56:01.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.425+0000 7fc52d1ff700 1 --2- 172.21.15.97:0/1223103282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc528159070 0x7fc5280c8f40 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.425+0000 7fc52d1ff700 1 --2- 172.21.15.97:0/1223103282 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5280ccbd0 0x7fc5280cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.425+0000 7fc52d1ff700 1 -- 172.21.15.97:0/1223103282 >> 172.21.15.97:0/1223103282 conn(0x7fc52806a060 msgr2=0x7fc52806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:01.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.425+0000 7fc52d1ff700 1 -- 172.21.15.97:0/1223103282 shutdown_connections 2023-07-06T03:56:01.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.425+0000 7fc52d1ff700 1 -- 172.21.15.97:0/1223103282 wait complete. 2023-07-06T03:56:01.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.427+0000 7fc52d1ff700 1 Processor -- start 2023-07-06T03:56:01.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.427+0000 7fc52d1ff700 1 -- start start 2023-07-06T03:56:01.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.428+0000 7fc52d1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5280c3cd0 0x7fc52814f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.428+0000 7fc52d1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5280ccbd0 0x7fc52814e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.428+0000 7fc52d1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc528159070 0x7fc5280afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.428+0000 7fc52d1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5281f2e80 con 0x7fc528159070 2023-07-06T03:56:01.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.428+0000 7fc52d1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5281f6370 con 0x7fc5280c3cd0 2023-07-06T03:56:01.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.428+0000 7fc52d1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52814eb70 con 0x7fc5280ccbd0 2023-07-06T03:56:01.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc52759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc528159070 0x7fc5280afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc52759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc528159070 0x7fc5280afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42180/0 (socket says 172.21.15.97:42180) 2023-07-06T03:56:01.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc52759e700 1 -- 172.21.15.97:0/347364159 learned_addr learned my addr 172.21.15.97:0/347364159 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:01.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc52659c700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5280ccbd0 0x7fc52814e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc526d9d700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5280c3cd0 0x7fc52814f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc52759e700 1 -- 172.21.15.97:0/347364159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5280ccbd0 msgr2=0x7fc52814e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc52759e700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5280ccbd0 0x7fc52814e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc52759e700 1 -- 172.21.15.97:0/347364159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5280c3cd0 msgr2=0x7fc52814f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc52759e700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5280c3cd0 0x7fc52814f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc52759e700 1 -- 172.21.15.97:0/347364159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5180094a0 con 0x7fc528159070 2023-07-06T03:56:01.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc52759e700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc528159070 0x7fc5280afa60 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fc52000ef90 tx=0x7fc52000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:01.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc50f7fe700 1 -- 172.21.15.97:0/347364159 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52000cc90 con 0x7fc528159070 2023-07-06T03:56:01.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc50f7fe700 1 -- 172.21.15.97:0/347364159 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52000f040 con 0x7fc528159070 2023-07-06T03:56:01.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.429+0000 7fc50f7fe700 1 -- 172.21.15.97:0/347364159 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520014480 con 0x7fc528159070 2023-07-06T03:56:01.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.430+0000 7fc52d1ff700 1 -- 172.21.15.97:0/347364159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5281f2200 con 0x7fc528159070 2023-07-06T03:56:01.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.430+0000 7fc52d1ff700 1 -- 172.21.15.97:0/347364159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5281f27c0 con 0x7fc528159070 2023-07-06T03:56:01.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.431+0000 7fc52d1ff700 1 -- 172.21.15.97:0/347364159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5280b03c0 con 0x7fc528159070 2023-07-06T03:56:01.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.431+0000 7fc50f7fe700 1 -- 172.21.15.97:0/347364159 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc5200145e0 con 0x7fc528159070 2023-07-06T03:56:01.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.433+0000 7fc50f7fe700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc51406ace0 0x7fc51406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.433+0000 7fc50f7fe700 1 -- 172.21.15.97:0/347364159 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc520089d00 con 0x7fc528159070 2023-07-06T03:56:01.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.433+0000 7fc526d9d700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc51406ace0 0x7fc51406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.434+0000 7fc526d9d700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc51406ace0 0x7fc51406d190 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc518009fd0 tx=0x7fc51801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:01.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.437+0000 7fc50f7fe700 1 -- 172.21.15.97:0/347364159 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc5280b03c0 con 0x7fc528159070 2023-07-06T03:56:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.630+0000 7fc52d1ff700 1 -- 172.21.15.97:0/347364159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc5280b03c0 con 0x7fc528159070 2023-07-06T03:56:01.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.631+0000 7fc50f7fe700 1 -- 172.21.15.97:0/347364159 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc5280b03c0 con 0x7fc528159070 2023-07-06T03:56:01.634 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:56:01.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 -- 172.21.15.97:0/347364159 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc51406ace0 msgr2=0x7fc51406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc51406ace0 0x7fc51406d190 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc518009fd0 tx=0x7fc51801c040 comp rx=0 tx=0).stop 2023-07-06T03:56:01.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 -- 172.21.15.97:0/347364159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc528159070 msgr2=0x7fc5280afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc528159070 0x7fc5280afa60 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fc52000ef90 tx=0x7fc52000c3a0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 -- 172.21.15.97:0/347364159 shutdown_connections 2023-07-06T03:56:01.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc51406ace0 0x7fc51406d190 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc528159070 0x7fc5280afa60 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5280c3cd0 0x7fc52814f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 --2- 172.21.15.97:0/347364159 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc5280ccbd0 0x7fc52814e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 -- 172.21.15.97:0/347364159 >> 172.21.15.97:0/347364159 conn(0x7fc52806a060 msgr2=0x7fc528153d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:01.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 -- 172.21.15.97:0/347364159 shutdown_connections 2023-07-06T03:56:01.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.633+0000 7fc52d1ff700 1 -- 172.21.15.97:0/347364159 wait complete. 2023-07-06T03:56:01.653 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T03:56:01.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.845+0000 7fae07fff700 1 Processor -- start 2023-07-06T03:56:01.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.845+0000 7fae07fff700 1 -- start start 2023-07-06T03:56:01.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.846+0000 7fae07fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae000cd390 0x7fae00160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.846+0000 7fae07fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae000cdae0 con 0x7fae0015c520 2023-07-06T03:56:01.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.846+0000 7fae07fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae000b6640 con 0x7fae00160dd0 2023-07-06T03:56:01.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.846+0000 7fae07fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae000b6920 con 0x7fae000cd390 2023-07-06T03:56:01.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.846+0000 7fae05d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fae0015c520 0x7fae00160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46494/0 (socket says 172.21.15.97:46494) 2023-07-06T03:56:01.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.846+0000 7fae05d9b700 1 -- 172.21.15.97:0/1633386374 learned_addr learned my addr 172.21.15.97:0/1633386374 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:01.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.846+0000 7fae0559a700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae000cd390 0x7fae00160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872153107 0 0) 0x7fae000cdae0 con 0x7fae0015c520 2023-07-06T03:56:01.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadec0035b0 con 0x7fae0015c520 2023-07-06T03:56:01.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1140242914 0 0) 0x7fadec0035b0 con 0x7fae0015c520 2023-07-06T03:56:01.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae000cd390 msgr2=0x7fae00160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:01.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae000cd390 0x7fae00160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 >> v1:172.21.15.146:6789/0 conn(0x7fae00160dd0 legacy=0x7fae000b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae000b6c00 con 0x7fae0015c520 2023-07-06T03:56:01.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fadfc001670 con 0x7fae0015c520 2023-07-06T03:56:01.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 >> v1:172.21.15.97:6789/0 conn(0x7fae0015c520 legacy=0x7fae00160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae000cd390 0x7fadec003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadec003f70 0x7fadec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadec004d90 0x7fadec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae000b6920 con 0x7fadec003f70 2023-07-06T03:56:01.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec00ba40 con 0x7fadec004d90 2023-07-06T03:56:01.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec00bc00 con 0x7fae000cd390 2023-07-06T03:56:01.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae0559a700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadec003f70 0x7fadec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae05d9b700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae000cd390 0x7fadec003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.847+0000 7fae0659c700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadec004d90 0x7fadec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.848+0000 7fae0559a700 1 -- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae000cd390 msgr2=0x7fadec003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:01.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.848+0000 7fae0559a700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae000cd390 0x7fadec003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.848+0000 7fae0559a700 1 -- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadec004d90 msgr2=0x7fadec005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:01.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.848+0000 7fae0559a700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadec004d90 0x7fadec005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.848+0000 7fae0559a700 1 -- 172.21.15.97:0/1633386374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec0035b0 con 0x7fadec003f70 2023-07-06T03:56:01.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.848+0000 7fae05d9b700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fae000cd390 0x7fadec003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:01.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.848+0000 7fae0659c700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fadec004d90 0x7fadec005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:01.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.848+0000 7fae0559a700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadec003f70 0x7fadec004850 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009b60 tx=0x7fadec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a408c8c9ec2882 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:01.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.848+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf000e6c0 con 0x7fadec003f70 2023-07-06T03:56:01.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.848+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf000e820 con 0x7fadec003f70 2023-07-06T03:56:01.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.848+0000 7fae04d99700 1 -- 172.21.15.97:0/1633386374 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf000eb90 con 0x7fadec003f70 2023-07-06T03:56:01.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.849+0000 7fae07fff700 1 -- 172.21.15.97:0/1633386374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadec003f70 msgr2=0x7fadec004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.849+0000 7fae07fff700 1 --2- 172.21.15.97:0/1633386374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadec003f70 0x7fadec004850 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009b60 tx=0x7fadec005880 comp rx=0 tx=0).stop 2023-07-06T03:56:01.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.849+0000 7fae0659c700 1 -- 172.21.15.97:0/1633386374 reap_dead start 2023-07-06T03:56:01.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.849+0000 7fae07fff700 1 -- 172.21.15.97:0/1633386374 shutdown_connections 2023-07-06T03:56:01.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.849+0000 7fae07fff700 1 -- 172.21.15.97:0/1633386374 >> 172.21.15.97:0/1633386374 conn(0x7fae00071790 msgr2=0x7fae00071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:01.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.849+0000 7fae07fff700 1 -- 172.21.15.97:0/1633386374 shutdown_connections 2023-07-06T03:56:01.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.849+0000 7fae07fff700 1 -- 172.21.15.97:0/1633386374 wait complete. 2023-07-06T03:56:01.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.850+0000 7fae07fff700 1 Processor -- start 2023-07-06T03:56:01.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.851+0000 7fae07fff700 1 -- start start 2023-07-06T03:56:01.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.851+0000 7fae07fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadec003f70 0x7fae000cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.851+0000 7fae07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae001f6350 0x7fae00149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae07fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae001f5900 0x7fae00148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae07fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae000d0c40 con 0x7fae001f6350 2023-07-06T03:56:01.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae07fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015e6f0 con 0x7fae001f5900 2023-07-06T03:56:01.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae07fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0014c6d0 con 0x7fadec003f70 2023-07-06T03:56:01.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae0559a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae001f6350 0x7fae00149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae0559a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae001f6350 0x7fae00149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42196/0 (socket says 172.21.15.97:42196) 2023-07-06T03:56:01.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae0559a700 1 -- 172.21.15.97:0/3997788015 learned_addr learned my addr 172.21.15.97:0/3997788015 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:01.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae05d9b700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadec003f70 0x7fae000cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae0659c700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae001f5900 0x7fae00148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae0559a700 1 -- 172.21.15.97:0/3997788015 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadec003f70 msgr2=0x7fae000cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae0559a700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadec003f70 0x7fae000cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae0559a700 1 -- 172.21.15.97:0/3997788015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae001f5900 msgr2=0x7fae00148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:01.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae0559a700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae001f5900 0x7fae00148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:01.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae0559a700 1 -- 172.21.15.97:0/3997788015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0014c9b0 con 0x7fae001f6350 2023-07-06T03:56:01.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.852+0000 7fae0559a700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae001f6350 0x7fae00149b90 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fadf000e4b0 tx=0x7fadf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:01.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.853+0000 7fadf67fc700 1 -- 172.21.15.97:0/3997788015 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf000e6c0 con 0x7fae001f6350 2023-07-06T03:56:01.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.853+0000 7fadf67fc700 1 -- 172.21.15.97:0/3997788015 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf00038a0 con 0x7fae001f6350 2023-07-06T03:56:01.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.853+0000 7fae07fff700 1 -- 172.21.15.97:0/3997788015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae000d0e90 con 0x7fae001f6350 2023-07-06T03:56:01.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.853+0000 7fadf67fc700 1 -- 172.21.15.97:0/3997788015 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0003c50 con 0x7fae001f6350 2023-07-06T03:56:01.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.853+0000 7fae07fff700 1 -- 172.21.15.97:0/3997788015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0014a0d0 con 0x7fae001f6350 2023-07-06T03:56:01.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.854+0000 7fae07fff700 1 -- 172.21.15.97:0/3997788015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0014a290 con 0x7fae001f6350 2023-07-06T03:56:01.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.855+0000 7fadf67fc700 1 -- 172.21.15.97:0/3997788015 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fadf0013470 con 0x7fae001f6350 2023-07-06T03:56:01.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.856+0000 7fadf67fc700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fade406ace0 0x7fade406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:01.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.856+0000 7fadf67fc700 1 -- 172.21.15.97:0/3997788015 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fadf008aae0 con 0x7fae001f6350 2023-07-06T03:56:01.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.856+0000 7fae05d9b700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fade406ace0 0x7fade406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:01.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.857+0000 7fae05d9b700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fade406ace0 0x7fade406d190 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fadfc001eb0 tx=0x7fadfc007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:01.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:01.860+0000 7fadf67fc700 1 -- 172.21.15.97:0/3997788015 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fadf0059bd0 con 0x7fae001f6350 2023-07-06T03:56:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.060+0000 7fae07fff700 1 -- 172.21.15.97:0/3997788015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fae0004e340 con 0x7fae001f6350 2023-07-06T03:56:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.060+0000 7fadf67fc700 1 -- 172.21.15.97:0/3997788015 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v132) v1 ==== 84+0+126 (secure 0 0 0) 0x7fadf0017090 con 0x7fae001f6350 2023-07-06T03:56:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:02.063 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-07-06T03:56:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-07-06T03:56:02.063 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-07-06T03:56:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 2, 2023-07-06T03:56:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": "rbd" 2023-07-06T03:56:02.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:02.064 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-07-06T03:56:02.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 -- 172.21.15.97:0/3997788015 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fade406ace0 msgr2=0x7fade406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:02.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fade406ace0 0x7fade406d190 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fadfc001eb0 tx=0x7fadfc007040 comp rx=0 tx=0).stop 2023-07-06T03:56:02.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 -- 172.21.15.97:0/3997788015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae001f6350 msgr2=0x7fae00149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:02.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae001f6350 0x7fae00149b90 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fadf000e4b0 tx=0x7fadf00047c0 comp rx=0 tx=0).stop 2023-07-06T03:56:02.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 -- 172.21.15.97:0/3997788015 shutdown_connections 2023-07-06T03:56:02.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fade406ace0 0x7fade406d190 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:02.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae001f5900 0x7fae00148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:02.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fadec003f70 0x7fae000cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:02.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 --2- 172.21.15.97:0/3997788015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae001f6350 0x7fae00149b90 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:02.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 -- 172.21.15.97:0/3997788015 >> 172.21.15.97:0/3997788015 conn(0x7fae00071790 msgr2=0x7fae000b72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:02.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 -- 172.21.15.97:0/3997788015 shutdown_connections 2023-07-06T03:56:02.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.062+0000 7fae07fff700 1 -- 172.21.15.97:0/3997788015 wait complete. 2023-07-06T03:56:02.083 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T03:56:02.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T03:56:02.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.279+0000 7f63b17ff700 1 Processor -- start 2023-07-06T03:56:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.279+0000 7f63b17ff700 1 -- start start 2023-07-06T03:56:02.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.281+0000 7f63b17ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f63ac1531b0 0x7f63ac0c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.281+0000 7f63b17ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ac0cc870 con 0x7f63ac0c9640 2023-07-06T03:56:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.281+0000 7f63b17ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ac0c4300 con 0x7f63ac159060 2023-07-06T03:56:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.281+0000 7f63b17ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac0ca950 con 0x7f63ac1531b0 2023-07-06T03:56:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.281+0000 7f63ab7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f63ac0c9640 0x7f63ac0ad5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46496/0 (socket says 172.21.15.97:46496) 2023-07-06T03:56:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.281+0000 7f63aaffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f63ac1531b0 0x7f63ac0c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.281+0000 7f63ab7fe700 1 -- 172.21.15.97:0/1245616530 learned_addr learned my addr 172.21.15.97:0/1245616530 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.282+0000 7f63aaffd700 1 -- 172.21.15.97:0/1245616530 >> v1:172.21.15.146:6789/0 conn(0x7f63ac159060 legacy=0x7f63ac0c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.282+0000 7f63aaffd700 1 -- 172.21.15.97:0/1245616530 >> v1:172.21.15.97:6789/0 conn(0x7f63ac0c9640 legacy=0x7f63ac0ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.282+0000 7f63aaffd700 1 -- 172.21.15.97:0/1245616530 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac0c45e0 con 0x7f63ac1531b0 2023-07-06T03:56:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.282+0000 7f63aaffd700 1 --2- 172.21.15.97:0/1245616530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63ac1531b0 0x7f63ac0c3c90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f639c009870 tx=0x7f639c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7f3ac803dfac4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.282+0000 7f63a9ffb700 1 -- 172.21.15.97:0/1245616530 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c00f8a0 con 0x7f63ac1531b0 2023-07-06T03:56:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.282+0000 7f63a9ffb700 1 -- 172.21.15.97:0/1245616530 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c00fa00 con 0x7f63ac1531b0 2023-07-06T03:56:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.282+0000 7f63a9ffb700 1 -- 172.21.15.97:0/1245616530 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c00fd90 con 0x7f63ac1531b0 2023-07-06T03:56:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.283+0000 7f63b17ff700 1 -- 172.21.15.97:0/1245616530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63ac1531b0 msgr2=0x7f63ac0c3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:02.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.283+0000 7f63b17ff700 1 --2- 172.21.15.97:0/1245616530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63ac1531b0 0x7f63ac0c3c90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f639c009870 tx=0x7f639c009b80 comp rx=0 tx=0).stop 2023-07-06T03:56:02.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.283+0000 7f63b17ff700 1 -- 172.21.15.97:0/1245616530 shutdown_connections 2023-07-06T03:56:02.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.283+0000 7f63b17ff700 1 --2- 172.21.15.97:0/1245616530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63ac1531b0 0x7f63ac0c3c90 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:02.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.283+0000 7f63b17ff700 1 -- 172.21.15.97:0/1245616530 >> 172.21.15.97:0/1245616530 conn(0x7f63ac0b0510 msgr2=0x7f63ac0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:02.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.283+0000 7f63b17ff700 1 -- 172.21.15.97:0/1245616530 shutdown_connections 2023-07-06T03:56:02.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.283+0000 7f63b17ff700 1 -- 172.21.15.97:0/1245616530 wait complete. 2023-07-06T03:56:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.284+0000 7f63b17ff700 1 Processor -- start 2023-07-06T03:56:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.285+0000 7f63b17ff700 1 -- start start 2023-07-06T03:56:02.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.285+0000 7f63b17ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63ac0c9640 0x7f63ac0c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.285+0000 7f63b17ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63ac1531b0 0x7f63ac0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.285+0000 7f63b17ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63ac159060 0x7f63ac0c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.285+0000 7f63b17ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0c6cd0 con 0x7f63ac159060 2023-07-06T03:56:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.285+0000 7f63b17ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0b1a40 con 0x7f63ac1531b0 2023-07-06T03:56:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.285+0000 7f63b17ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0c6430 con 0x7f63ac0c9640 2023-07-06T03:56:02.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63ac159060 0x7f63ac0c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:02.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63ac159060 0x7f63ac0c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42218/0 (socket says 172.21.15.97:42218) 2023-07-06T03:56:02.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63ab7fe700 1 -- 172.21.15.97:0/414189515 learned_addr learned my addr 172.21.15.97:0/414189515 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:02.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63aa7fc700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63ac1531b0 0x7f63ac0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:02.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63aaffd700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63ac0c9640 0x7f63ac0c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:02.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63ab7fe700 1 -- 172.21.15.97:0/414189515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63ac0c9640 msgr2=0x7f63ac0c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:02.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63ab7fe700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63ac0c9640 0x7f63ac0c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:02.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63ab7fe700 1 -- 172.21.15.97:0/414189515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63ac1531b0 msgr2=0x7f63ac0c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:02.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63ab7fe700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63ac1531b0 0x7f63ac0c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:02.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63ab7fe700 1 -- 172.21.15.97:0/414189515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c0094a0 con 0x7f63ac159060 2023-07-06T03:56:02.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63aa7fc700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63ac1531b0 0x7f63ac0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:02.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63aaffd700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63ac0c9640 0x7f63ac0c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:02.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63ab7fe700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63ac159060 0x7f63ac0c7480 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f63a400cad0 tx=0x7f63a400ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:02.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.286+0000 7f63937fe700 1 -- 172.21.15.97:0/414189515 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a400a860 con 0x7f63ac159060 2023-07-06T03:56:02.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.287+0000 7f63937fe700 1 -- 172.21.15.97:0/414189515 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a400a9c0 con 0x7f63ac159060 2023-07-06T03:56:02.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.287+0000 7f63937fe700 1 -- 172.21.15.97:0/414189515 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a400ad30 con 0x7f63ac159060 2023-07-06T03:56:02.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.287+0000 7f63b17ff700 1 -- 172.21.15.97:0/414189515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac0c6710 con 0x7f63ac159060 2023-07-06T03:56:02.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.287+0000 7f63b17ff700 1 -- 172.21.15.97:0/414189515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac0b3740 con 0x7f63ac159060 2023-07-06T03:56:02.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.288+0000 7f63b17ff700 1 -- 172.21.15.97:0/414189515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac0b1810 con 0x7f63ac159060 2023-07-06T03:56:02.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.289+0000 7f63937fe700 1 -- 172.21.15.97:0/414189515 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f63a4006070 con 0x7f63ac159060 2023-07-06T03:56:02.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.290+0000 7f63937fe700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f639806ace0 0x7f639806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:02.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.290+0000 7f63937fe700 1 -- 172.21.15.97:0/414189515 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f63a4088b70 con 0x7f63ac159060 2023-07-06T03:56:02.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.290+0000 7f63aaffd700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f639806ace0 0x7f639806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:02.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.290+0000 7f63aaffd700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f639806ace0 0x7f639806d190 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c01b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:02.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.294+0000 7f63937fe700 1 -- 172.21.15.97:0/414189515 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f63ac0b1810 con 0x7f63ac159060 2023-07-06T03:56:02.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:02.490+0000 7f63b17ff700 1 -- 172.21.15.97:0/414189515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f63ac0b1810 con 0x7f63ac159060 2023-07-06T03:56:03.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.068+0000 7f63937fe700 1 -- 172.21.15.97:0/414189515 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v133) v1 ==== 138+0+0 (secure 0 0 0) 0x7f63ac0b1810 con 0x7f63ac159060 2023-07-06T03:56:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 -- 172.21.15.97:0/414189515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f639806ace0 msgr2=0x7f639806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f639806ace0 0x7f639806d190 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c01b040 comp rx=0 tx=0).stop 2023-07-06T03:56:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 -- 172.21.15.97:0/414189515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63ac159060 msgr2=0x7f63ac0c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63ac159060 0x7f63ac0c7480 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f63a400cad0 tx=0x7f63a400ce90 comp rx=0 tx=0).stop 2023-07-06T03:56:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 -- 172.21.15.97:0/414189515 shutdown_connections 2023-07-06T03:56:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f639806ace0 0x7f639806d190 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:03.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63ac159060 0x7f63ac0c7480 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:03.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f63ac0c9640 0x7f63ac0c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:03.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 --2- 172.21.15.97:0/414189515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63ac1531b0 0x7f63ac0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:03.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 -- 172.21.15.97:0/414189515 >> 172.21.15.97:0/414189515 conn(0x7f63ac0b0510 msgr2=0x7f63ac0cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:03.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 -- 172.21.15.97:0/414189515 shutdown_connections 2023-07-06T03:56:03.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.074+0000 7f63b17ff700 1 -- 172.21.15.97:0/414189515 wait complete. 2023-07-06T03:56:03.079 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-07-06T03:56:03.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T03:56:03.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.294+0000 7f53343ff700 1 Processor -- start 2023-07-06T03:56:03.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.295+0000 7f53343ff700 1 -- start start 2023-07-06T03:56:03.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.296+0000 7f53343ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f532c159390 0x7f532c0c9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:03.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.296+0000 7f53343ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c159760 con 0x7f532c0c5060 2023-07-06T03:56:03.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.296+0000 7f53343ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c070670 con 0x7f532c0c8fb0 2023-07-06T03:56:03.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.296+0000 7f53343ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c070950 con 0x7f532c159390 2023-07-06T03:56:03.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.296+0000 7f533299c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f532c0c5060 0x7f532c0ccd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46512/0 (socket says 172.21.15.97:46512) 2023-07-06T03:56:03.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.296+0000 7f533299c700 1 -- 172.21.15.97:0/3192016482 learned_addr learned my addr 172.21.15.97:0/3192016482 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.296+0000 7f533219b700 1 --2- 172.21.15.97:0/3192016482 >> v2:172.21.15.146:3300/0 conn(0x7f532c159390 0x7f532c0c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.297+0000 7f5331199700 1 -- 172.21.15.97:0/3192016482 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875001736 0 0) 0x7f532c159760 con 0x7f532c0c5060 2023-07-06T03:56:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.297+0000 7f5331199700 1 -- 172.21.15.97:0/3192016482 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53180035b0 con 0x7f532c0c5060 2023-07-06T03:56:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.297+0000 7f533219b700 1 -- 172.21.15.97:0/3192016482 >> v1:172.21.15.146:6789/0 conn(0x7f532c0c8fb0 legacy=0x7f532c0c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.297+0000 7f533219b700 1 -- 172.21.15.97:0/3192016482 >> v1:172.21.15.97:6789/0 conn(0x7f532c0c5060 legacy=0x7f532c0ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.297+0000 7f533219b700 1 -- 172.21.15.97:0/3192016482 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c070c30 con 0x7f532c159390 2023-07-06T03:56:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.297+0000 7f533219b700 1 --2- 172.21.15.97:0/3192016482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f532c159390 0x7f532c0c9850 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5328009840 tx=0x7f5328009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2394457b286aae59 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.297+0000 7f5331199700 1 -- 172.21.15.97:0/3192016482 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328011830 con 0x7f532c159390 2023-07-06T03:56:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.297+0000 7f5331199700 1 -- 172.21.15.97:0/3192016482 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5328011990 con 0x7f532c159390 2023-07-06T03:56:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.297+0000 7f5331199700 1 -- 172.21.15.97:0/3192016482 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328011d20 con 0x7f532c159390 2023-07-06T03:56:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.298+0000 7f53343ff700 1 -- 172.21.15.97:0/3192016482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f532c159390 msgr2=0x7f532c0c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.298+0000 7f53343ff700 1 --2- 172.21.15.97:0/3192016482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f532c159390 0x7f532c0c9850 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5328009840 tx=0x7f5328009b50 comp rx=0 tx=0).stop 2023-07-06T03:56:03.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.298+0000 7f53343ff700 1 -- 172.21.15.97:0/3192016482 shutdown_connections 2023-07-06T03:56:03.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.298+0000 7f53343ff700 1 --2- 172.21.15.97:0/3192016482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f532c159390 0x7f532c0c9850 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:03.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.298+0000 7f53343ff700 1 -- 172.21.15.97:0/3192016482 >> 172.21.15.97:0/3192016482 conn(0x7f532c0b05d0 msgr2=0x7f532c0b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:03.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.298+0000 7f53343ff700 1 -- 172.21.15.97:0/3192016482 shutdown_connections 2023-07-06T03:56:03.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.298+0000 7f53343ff700 1 -- 172.21.15.97:0/3192016482 wait complete. 2023-07-06T03:56:03.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.300+0000 7f53343ff700 1 Processor -- start 2023-07-06T03:56:03.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.300+0000 7f53343ff700 1 -- start start 2023-07-06T03:56:03.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.301+0000 7f53343ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532c0c5060 0x7f532c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:03.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.301+0000 7f53343ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f532c0c8fb0 0x7f532c143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.301+0000 7f53343ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f532c159390 0x7f532c0af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.301+0000 7f53343ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c0c6f00 con 0x7f532c0c5060 2023-07-06T03:56:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.301+0000 7f53343ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c15b290 con 0x7f532c159390 2023-07-06T03:56:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.301+0000 7f53343ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c15b570 con 0x7f532c0c8fb0 2023-07-06T03:56:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.301+0000 7f533219b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532c0c5060 0x7f532c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.301+0000 7f533219b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532c0c5060 0x7f532c15a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42234/0 (socket says 172.21.15.97:42234) 2023-07-06T03:56:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.301+0000 7f533219b700 1 -- 172.21.15.97:0/3768816566 learned_addr learned my addr 172.21.15.97:0/3768816566 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.301+0000 7f533199a700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f532c0c8fb0 0x7f532c143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.301+0000 7f533299c700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f532c159390 0x7f532c0af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.302+0000 7f533219b700 1 -- 172.21.15.97:0/3768816566 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f532c0c8fb0 msgr2=0x7f532c143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.302+0000 7f533219b700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f532c0c8fb0 0x7f532c143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.302+0000 7f533219b700 1 -- 172.21.15.97:0/3768816566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f532c159390 msgr2=0x7f532c0af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.302+0000 7f533219b700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f532c159390 0x7f532c0af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:03.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.302+0000 7f533219b700 1 -- 172.21.15.97:0/3768816566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53280094a0 con 0x7f532c0c5060 2023-07-06T03:56:03.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.302+0000 7f533219b700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532c0c5060 0x7f532c15a9b0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f5328009b90 tx=0x7f5328004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:03.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.302+0000 7f5322ffd700 1 -- 172.21.15.97:0/3768816566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53280036a0 con 0x7f532c0c5060 2023-07-06T03:56:03.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.302+0000 7f5322ffd700 1 -- 172.21.15.97:0/3768816566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532801e070 con 0x7f532c0c5060 2023-07-06T03:56:03.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.302+0000 7f53343ff700 1 -- 172.21.15.97:0/3768816566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c15b850 con 0x7f532c0c5060 2023-07-06T03:56:03.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.302+0000 7f5322ffd700 1 -- 172.21.15.97:0/3768816566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328003a60 con 0x7f532c0c5060 2023-07-06T03:56:03.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.302+0000 7f53343ff700 1 -- 172.21.15.97:0/3768816566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c15aef0 con 0x7f532c0c5060 2023-07-06T03:56:03.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.303+0000 7f53343ff700 1 -- 172.21.15.97:0/3768816566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c15ba60 con 0x7f532c0c5060 2023-07-06T03:56:03.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.305+0000 7f5322ffd700 1 -- 172.21.15.97:0/3768816566 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5328003bc0 con 0x7f532c0c5060 2023-07-06T03:56:03.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.306+0000 7f5322ffd700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f531006ace0 0x7f531006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.306+0000 7f5322ffd700 1 -- 172.21.15.97:0/3768816566 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f532808a250 con 0x7f532c0c5060 2023-07-06T03:56:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.306+0000 7f533199a700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f531006ace0 0x7f531006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.306+0000 7f533199a700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f531006ace0 0x7f531006d190 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f532c15c350 tx=0x7f531c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:03.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.310+0000 7f5322ffd700 1 -- 172.21.15.97:0/3768816566 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5328055440 con 0x7f532c0c5060 2023-07-06T03:56:03.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:03.507+0000 7f53343ff700 1 -- 172.21.15.97:0/3768816566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f532c04e340 con 0x7f532c0c5060 2023-07-06T03:56:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.073+0000 7f5322ffd700 1 -- 172.21.15.97:0/3768816566 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v134) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5328058a60 con 0x7f532c0c5060 2023-07-06T03:56:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.081+0000 7f53343ff700 1 -- 172.21.15.97:0/3768816566 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f531006ace0 msgr2=0x7f531006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.081+0000 7f53343ff700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f531006ace0 0x7f531006d190 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f532c15c350 tx=0x7f531c009040 comp rx=0 tx=0).stop 2023-07-06T03:56:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.081+0000 7f53343ff700 1 -- 172.21.15.97:0/3768816566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532c0c5060 msgr2=0x7f532c15a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.081+0000 7f53343ff700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532c0c5060 0x7f532c15a9b0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f5328009b90 tx=0x7f5328004750 comp rx=0 tx=0).stop 2023-07-06T03:56:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.084+0000 7f53343ff700 1 -- 172.21.15.97:0/3768816566 shutdown_connections 2023-07-06T03:56:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.084+0000 7f53343ff700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f531006ace0 0x7f531006d190 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.084+0000 7f53343ff700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f532c159390 0x7f532c0af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.084+0000 7f53343ff700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532c0c5060 0x7f532c15a9b0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.084+0000 7f53343ff700 1 --2- 172.21.15.97:0/3768816566 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f532c0c8fb0 0x7f532c143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.084+0000 7f53343ff700 1 -- 172.21.15.97:0/3768816566 >> 172.21.15.97:0/3768816566 conn(0x7f532c0b05d0 msgr2=0x7f532c0ae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.084+0000 7f53343ff700 1 -- 172.21.15.97:0/3768816566 shutdown_connections 2023-07-06T03:56:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.084+0000 7f53343ff700 1 -- 172.21.15.97:0/3768816566 wait complete. 2023-07-06T03:56:04.097 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-07-06T03:56:04.112 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T03:56:04.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.308+0000 7f4666fff700 1 Processor -- start 2023-07-06T03:56:04.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.308+0000 7f4666fff700 1 -- start start 2023-07-06T03:56:04.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.309+0000 7f4666fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466006b360 0x7f46600b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.309+0000 7f4666fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f46600d2210 0x7f46600d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.309+0000 7f4666fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46600cff90 con 0x7f466015c610 2023-07-06T03:56:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.309+0000 7f4666fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46600d0780 con 0x7f46600d2210 2023-07-06T03:56:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.309+0000 7f4666fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466015c9e0 con 0x7f466006b360 2023-07-06T03:56:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.309+0000 7f4664d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f466015c610 0x7f46600b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46518/0 (socket says 172.21.15.97:46518) 2023-07-06T03:56:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.309+0000 7f4664d9b700 1 -- 172.21.15.97:0/2323084555 learned_addr learned my addr 172.21.15.97:0/2323084555 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:04.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.309+0000 7f466559c700 1 --2- 172.21.15.97:0/2323084555 >> v2:172.21.15.146:3300/0 conn(0x7f46600d2210 0x7f46600d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:04.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465ffff700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466006b360 0x7f46600b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:04.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78319874 0 0) 0x7f46600cff90 con 0x7f466015c610 2023-07-06T03:56:04.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46500035b0 con 0x7f466015c610 2023-07-06T03:56:04.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2943654590 0 0) 0x7f46500035b0 con 0x7f466015c610 2023-07-06T03:56:04.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466006b360 msgr2=0x7f46600b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466006b360 0x7f46600b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 >> v2:172.21.15.146:3300/0 conn(0x7f46600d2210 msgr2=0x7f46600d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 --2- 172.21.15.97:0/2323084555 >> v2:172.21.15.146:3300/0 conn(0x7f46600d2210 0x7f46600d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600b5650 con 0x7f466015c610 2023-07-06T03:56:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f4654001670 con 0x7f466015c610 2023-07-06T03:56:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 >> v1:172.21.15.97:6789/0 conn(0x7f466015c610 legacy=0x7f46600b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 0x7f4650003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466006b360 0x7f4650004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4650004d90 0x7f4650005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46600d0780 con 0x7f4650003a30 2023-07-06T03:56:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466015c9e0 con 0x7f4650004d90 2023-07-06T03:56:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465000ba10 con 0x7f466006b360 2023-07-06T03:56:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.310+0000 7f4664d9b700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 0x7f4650003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f465ffff700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466006b360 0x7f4650004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f466559c700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4650004d90 0x7f4650005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f4664d9b700 1 -- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466006b360 msgr2=0x7f4650004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f4664d9b700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f466006b360 0x7f4650004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f4664d9b700 1 -- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4650004d90 msgr2=0x7f4650005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f4664d9b700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4650004d90 0x7f4650005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f4664d9b700 1 -- 172.21.15.97:0/2323084555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46500035b0 con 0x7f4650003a30 2023-07-06T03:56:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f466559c700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4650004d90 0x7f4650005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f4664d9b700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 0x7f4650003e40 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f465400c240 tx=0x7f4650005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11944fabf9d7678 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654007c10 con 0x7f4650003a30 2023-07-06T03:56:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465400ce00 con 0x7f4650003a30 2023-07-06T03:56:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.311+0000 7f465f7fe700 1 -- 172.21.15.97:0/2323084555 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654005ba0 con 0x7f4650003a30 2023-07-06T03:56:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.312+0000 7f4666fff700 1 -- 172.21.15.97:0/2323084555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 msgr2=0x7f4650003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.312+0000 7f4666fff700 1 --2- 172.21.15.97:0/2323084555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 0x7f4650003e40 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f465400c240 tx=0x7f4650005880 comp rx=0 tx=0).stop 2023-07-06T03:56:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.312+0000 7f466559c700 1 -- 172.21.15.97:0/2323084555 reap_dead start 2023-07-06T03:56:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.312+0000 7f4666fff700 1 -- 172.21.15.97:0/2323084555 shutdown_connections 2023-07-06T03:56:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.312+0000 7f4666fff700 1 -- 172.21.15.97:0/2323084555 >> 172.21.15.97:0/2323084555 conn(0x7f466006a6c0 msgr2=0x7f466006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.312+0000 7f4666fff700 1 -- 172.21.15.97:0/2323084555 shutdown_connections 2023-07-06T03:56:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.312+0000 7f4666fff700 1 -- 172.21.15.97:0/2323084555 wait complete. 2023-07-06T03:56:04.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.314+0000 7f4666fff700 1 Processor -- start 2023-07-06T03:56:04.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.314+0000 7f4666fff700 1 -- start start 2023-07-06T03:56:04.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4666fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 0x7f466015def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:04.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4666fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4660153170 0x7f466014a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:04.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4666fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466015adb0 0x7f466015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:04.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4666fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466015b790 con 0x7f4650003a30 2023-07-06T03:56:04.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4666fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466014a550 con 0x7f466015adb0 2023-07-06T03:56:04.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4666fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46600cabb0 con 0x7f4660153170 2023-07-06T03:56:04.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4664d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 0x7f466015def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:04.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4664d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 0x7f466015def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42250/0 (socket says 172.21.15.97:42250) 2023-07-06T03:56:04.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4664d9b700 1 -- 172.21.15.97:0/286578380 learned_addr learned my addr 172.21.15.97:0/286578380 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:04.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f466559c700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466015adb0 0x7f466015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:04.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f465ffff700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4660153170 0x7f466014a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:04.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4664d9b700 1 -- 172.21.15.97:0/286578380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4660153170 msgr2=0x7f466014a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:04.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4664d9b700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4660153170 0x7f466014a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:04.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4664d9b700 1 -- 172.21.15.97:0/286578380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466015adb0 msgr2=0x7f466015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:04.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4664d9b700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466015adb0 0x7f466015b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:04.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f4664d9b700 1 -- 172.21.15.97:0/286578380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4660153970 con 0x7f4650003a30 2023-07-06T03:56:04.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f466559c700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466015adb0 0x7f466015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:56:04.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.316+0000 7f465ffff700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4660153170 0x7f466014a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:56:04.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.317+0000 7f4664d9b700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 0x7f466015def0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f4654000f50 tx=0x7f4654001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:04.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.317+0000 7f465d7fa700 1 -- 172.21.15.97:0/286578380 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654001630 con 0x7f4650003a30 2023-07-06T03:56:04.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.317+0000 7f465d7fa700 1 -- 172.21.15.97:0/286578380 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46540070d0 con 0x7f4650003a30 2023-07-06T03:56:04.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.317+0000 7f465d7fa700 1 -- 172.21.15.97:0/286578380 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46540074a0 con 0x7f4650003a30 2023-07-06T03:56:04.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.317+0000 7f4666fff700 1 -- 172.21.15.97:0/286578380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46600c9fa0 con 0x7f4650003a30 2023-07-06T03:56:04.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.317+0000 7f4666fff700 1 -- 172.21.15.97:0/286578380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46600ca560 con 0x7f4650003a30 2023-07-06T03:56:04.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.318+0000 7f4666fff700 1 -- 172.21.15.97:0/286578380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46600b7d70 con 0x7f4650003a30 2023-07-06T03:56:04.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.320+0000 7f465d7fa700 1 -- 172.21.15.97:0/286578380 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4654019030 con 0x7f4650003a30 2023-07-06T03:56:04.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.321+0000 7f465d7fa700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f464406ace0 0x7f464406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:04.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.321+0000 7f465d7fa700 1 -- 172.21.15.97:0/286578380 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f465408a5b0 con 0x7f4650003a30 2023-07-06T03:56:04.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.321+0000 7f465ffff700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f464406ace0 0x7f464406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:04.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.322+0000 7f465ffff700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f464406ace0 0x7f464406d190 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f466015d140 tx=0x7f464c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:04.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.325+0000 7f465d7fa700 1 -- 172.21.15.97:0/286578380 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f46540556b0 con 0x7f4650003a30 2023-07-06T03:56:04.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:04.563+0000 7f4666fff700 1 -- 172.21.15.97:0/286578380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f466004e340 con 0x7f4650003a30 2023-07-06T03:56:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.089+0000 7f465d7fa700 1 -- 172.21.15.97:0/286578380 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v99) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4654058cd0 con 0x7f4650003a30 2023-07-06T03:56:05.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.095+0000 7f4666fff700 1 -- 172.21.15.97:0/286578380 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f464406ace0 msgr2=0x7f464406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.095+0000 7f4666fff700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f464406ace0 0x7f464406d190 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f466015d140 tx=0x7f464c008040 comp rx=0 tx=0).stop 2023-07-06T03:56:05.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.095+0000 7f4666fff700 1 -- 172.21.15.97:0/286578380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 msgr2=0x7f466015def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.095+0000 7f4666fff700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 0x7f466015def0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f4654000f50 tx=0x7f4654001a00 comp rx=0 tx=0).stop 2023-07-06T03:56:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.103+0000 7f4666fff700 1 -- 172.21.15.97:0/286578380 shutdown_connections 2023-07-06T03:56:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.103+0000 7f4666fff700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f464406ace0 0x7f464406d190 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.103+0000 7f4666fff700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f466015adb0 0x7f466015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.103+0000 7f4666fff700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4650003a30 0x7f466015def0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.103+0000 7f4666fff700 1 --2- 172.21.15.97:0/286578380 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4660153170 0x7f466014a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.103+0000 7f4666fff700 1 -- 172.21.15.97:0/286578380 >> 172.21.15.97:0/286578380 conn(0x7f466006a6c0 msgr2=0x7f46600d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.103+0000 7f4666fff700 1 -- 172.21.15.97:0/286578380 shutdown_connections 2023-07-06T03:56:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.104+0000 7f4666fff700 1 -- 172.21.15.97:0/286578380 wait complete. 2023-07-06T03:56:05.111 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 21 and data pool 22 2023-07-06T03:56:05.129 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:56:05.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.320+0000 7f0ee03ff700 1 Processor -- start 2023-07-06T03:56:05.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.321+0000 7f0ee03ff700 1 -- start start 2023-07-06T03:56:05.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.322+0000 7f0ee03ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0ed80b8410 0x7f0ed8167510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.322+0000 7f0ee03ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed80cdae0 con 0x7f0ed815c520 2023-07-06T03:56:05.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.322+0000 7f0ee03ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed80b6640 con 0x7f0ed80cd390 2023-07-06T03:56:05.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.322+0000 7f0ee03ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed80b6920 con 0x7f0ed80b8410 2023-07-06T03:56:05.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.322+0000 7f0ede19b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0ed815c520 0x7f0ed8160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46526/0 (socket says 172.21.15.97:46526) 2023-07-06T03:56:05.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.322+0000 7f0ede19b700 1 -- 172.21.15.97:0/2547308371 learned_addr learned my addr 172.21.15.97:0/2547308371 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:05.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.322+0000 7f0ede99c700 1 --2- 172.21.15.97:0/2547308371 >> v2:172.21.15.146:3300/0 conn(0x7f0ed80b8410 0x7f0ed8167510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.322+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835108571 0 0) 0x7f0ed80cdae0 con 0x7f0ed815c520 2023-07-06T03:56:05.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.322+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ec40035b0 con 0x7f0ed815c520 2023-07-06T03:56:05.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2448224349 0 0) 0x7f0ec40035b0 con 0x7f0ed815c520 2023-07-06T03:56:05.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 >> v2:172.21.15.146:3300/0 conn(0x7f0ed80b8410 msgr2=0x7f0ed8167510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:05.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 --2- 172.21.15.97:0/2547308371 >> v2:172.21.15.146:3300/0 conn(0x7f0ed80b8410 0x7f0ed8167510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 >> v1:172.21.15.146:6789/0 conn(0x7f0ed80cd390 legacy=0x7f0ed8160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80b6c00 con 0x7f0ed815c520 2023-07-06T03:56:05.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f0ed4001670 con 0x7f0ed815c520 2023-07-06T03:56:05.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 >> v1:172.21.15.97:6789/0 conn(0x7f0ed815c520 legacy=0x7f0ed8160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 0x7f0ec4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ec4004850 0x7f0ec4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ec4005200 0x7f0ec400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80b6920 con 0x7f0ec4003a00 2023-07-06T03:56:05.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec400c080 con 0x7f0ec4004850 2023-07-06T03:56:05.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec400c240 con 0x7f0ec4005200 2023-07-06T03:56:05.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0ede99c700 1 --2- 172.21.15.97:0/2547308371 >> v2:172.21.15.146:3300/0 conn(0x7f0ed80b8410 0x7f0ed8167510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:56:05.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0ede19b700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 0x7f0ec4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0edd99a700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ec4004850 0x7f0ec4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0ede99c700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ec4005200 0x7f0ec400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0ede19b700 1 -- 172.21.15.97:0/2547308371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ec4005200 msgr2=0x7f0ec400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0ede19b700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ec4005200 0x7f0ec400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0ede19b700 1 -- 172.21.15.97:0/2547308371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ec4004850 msgr2=0x7f0ec4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:05.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.323+0000 7f0ede19b700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ec4004850 0x7f0ec4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.324+0000 7f0ede19b700 1 -- 172.21.15.97:0/2547308371 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec40035b0 con 0x7f0ec4003a00 2023-07-06T03:56:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.324+0000 7f0edd99a700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ec4004850 0x7f0ec4004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.324+0000 7f0ede99c700 1 -- 172.21.15.97:0/2547308371 reap_dead start 2023-07-06T03:56:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.324+0000 7f0ede19b700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 0x7f0ec4003e10 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400c240 tx=0x7f0ec4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8eb3ed2552ed2d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.324+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4007c10 con 0x7f0ec4003a00 2023-07-06T03:56:05.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.324+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed400ce00 con 0x7f0ec4003a00 2023-07-06T03:56:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.324+0000 7f0edd199700 1 -- 172.21.15.97:0/2547308371 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4005ba0 con 0x7f0ec4003a00 2023-07-06T03:56:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.324+0000 7f0ee03ff700 1 -- 172.21.15.97:0/2547308371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 msgr2=0x7f0ec4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.324+0000 7f0ee03ff700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 0x7f0ec4003e10 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400c240 tx=0x7f0ec4005880 comp rx=0 tx=0).stop 2023-07-06T03:56:05.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.325+0000 7f0ee03ff700 1 -- 172.21.15.97:0/2547308371 shutdown_connections 2023-07-06T03:56:05.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.325+0000 7f0ee03ff700 1 --2- 172.21.15.97:0/2547308371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 0x7f0ec4003e10 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.325+0000 7f0ee03ff700 1 -- 172.21.15.97:0/2547308371 >> 172.21.15.97:0/2547308371 conn(0x7f0ed8071790 msgr2=0x7f0ed8071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:05.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.325+0000 7f0ee03ff700 1 -- 172.21.15.97:0/2547308371 shutdown_connections 2023-07-06T03:56:05.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.325+0000 7f0ee03ff700 1 -- 172.21.15.97:0/2547308371 wait complete. 2023-07-06T03:56:05.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.327+0000 7f0ee03ff700 1 Processor -- start 2023-07-06T03:56:05.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.327+0000 7f0ee03ff700 1 -- start start 2023-07-06T03:56:05.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.328+0000 7f0ee03ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 0x7f0ed80c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.328+0000 7f0ee03ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ed80cbeb0 0x7f0ed81f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ee03ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed80c7e60 0x7f0ed80c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ee03ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80c8810 con 0x7f0ec4003a00 2023-07-06T03:56:05.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ee03ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81420f0 con 0x7f0ed80c7e60 2023-07-06T03:56:05.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ee03ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80cc6e0 con 0x7f0ed80cbeb0 2023-07-06T03:56:05.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ede19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 0x7f0ed80c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ede19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 0x7f0ed80c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42272/0 (socket says 172.21.15.97:42272) 2023-07-06T03:56:05.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ede19b700 1 -- 172.21.15.97:0/1705367068 learned_addr learned my addr 172.21.15.97:0/1705367068 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:05.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0edd99a700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ed80cbeb0 0x7f0ed81f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ede99c700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed80c7e60 0x7f0ed80c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ede19b700 1 -- 172.21.15.97:0/1705367068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ed80cbeb0 msgr2=0x7f0ed81f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ede19b700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ed80cbeb0 0x7f0ed81f1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ede19b700 1 -- 172.21.15.97:0/1705367068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed80c7e60 msgr2=0x7f0ed80c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ede19b700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed80c7e60 0x7f0ed80c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ede19b700 1 -- 172.21.15.97:0/1705367068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80cc9c0 con 0x7f0ec4003a00 2023-07-06T03:56:05.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.329+0000 7f0ede19b700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 0x7f0ed80c9260 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400c830 tx=0x7f0ed4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:05.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.330+0000 7f0eceffd700 1 -- 172.21.15.97:0/1705367068 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4004d40 con 0x7f0ec4003a00 2023-07-06T03:56:05.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.330+0000 7f0eceffd700 1 -- 172.21.15.97:0/1705367068 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed40070d0 con 0x7f0ec4003a00 2023-07-06T03:56:05.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.330+0000 7f0ee03ff700 1 -- 172.21.15.97:0/1705367068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed80ccca0 con 0x7f0ec4003a00 2023-07-06T03:56:05.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.330+0000 7f0eceffd700 1 -- 172.21.15.97:0/1705367068 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4004ea0 con 0x7f0ec4003a00 2023-07-06T03:56:05.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.330+0000 7f0ee03ff700 1 -- 172.21.15.97:0/1705367068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed81f23d0 con 0x7f0ec4003a00 2023-07-06T03:56:05.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.331+0000 7f0ee03ff700 1 -- 172.21.15.97:0/1705367068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed804e340 con 0x7f0ec4003a00 2023-07-06T03:56:05.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.332+0000 7f0eceffd700 1 -- 172.21.15.97:0/1705367068 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ed4007440 con 0x7f0ec4003a00 2023-07-06T03:56:05.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.333+0000 7f0eceffd700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0ebc06ace0 0x7f0ebc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.333+0000 7f0eceffd700 1 -- 172.21.15.97:0/1705367068 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0ed4089c70 con 0x7f0ec4003a00 2023-07-06T03:56:05.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.333+0000 7f0edd99a700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0ebc06ace0 0x7f0ebc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.333+0000 7f0edd99a700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0ebc06ace0 0x7f0ebc06d190 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8141fc0 tx=0x7f0ec8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:05.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.337+0000 7f0eceffd700 1 -- 172.21.15.97:0/1705367068 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0ed4054d10 con 0x7f0ec4003a00 2023-07-06T03:56:05.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.535+0000 7f0ee03ff700 1 -- 172.21.15.97:0/1705367068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ed8157f20 con 0x7f0ec4003a00 2023-07-06T03:56:05.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.536+0000 7f0eceffd700 1 -- 172.21.15.97:0/1705367068 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+18404 (secure 0 0 0) 0x7f0ed4087800 con 0x7f0ec4003a00 2023-07-06T03:56:05.539 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:05.539 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":135,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:56:04.565954+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:56:02.531370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-06T03:56:03.548148+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:56:05.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.538+0000 7f0ee03ff700 1 -- 172.21.15.97:0/1705367068 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0ebc06ace0 msgr2=0x7f0ebc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.541 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.538+0000 7f0ee03ff700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0ebc06ace0 0x7f0ebc06d190 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8141fc0 tx=0x7f0ec8009040 comp rx=0 tx=0).stop 2023-07-06T03:56:05.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.538+0000 7f0ee03ff700 1 -- 172.21.15.97:0/1705367068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 msgr2=0x7f0ed80c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.538+0000 7f0ee03ff700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 0x7f0ed80c9260 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400c830 tx=0x7f0ed4001a00 comp rx=0 tx=0).stop 2023-07-06T03:56:05.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.538+0000 7f0ee03ff700 1 -- 172.21.15.97:0/1705367068 shutdown_connections 2023-07-06T03:56:05.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.538+0000 7f0ee03ff700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0ebc06ace0 0x7f0ebc06d190 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.538+0000 7f0ee03ff700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ed80c7e60 0x7f0ed80c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.539+0000 7f0ee03ff700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ec4003a00 0x7f0ed80c9260 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.539+0000 7f0ee03ff700 1 --2- 172.21.15.97:0/1705367068 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0ed80cbeb0 0x7f0ed81f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.539+0000 7f0ee03ff700 1 -- 172.21.15.97:0/1705367068 >> 172.21.15.97:0/1705367068 conn(0x7f0ed8071790 msgr2=0x7f0ed815f410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:05.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.539+0000 7f0ee03ff700 1 -- 172.21.15.97:0/1705367068 shutdown_connections 2023-07-06T03:56:05.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.539+0000 7f0ee03ff700 1 -- 172.21.15.97:0/1705367068 wait complete. 2023-07-06T03:56:05.560 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:56:05.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.752+0000 7f58e27ff700 1 Processor -- start 2023-07-06T03:56:05.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.752+0000 7f58e27ff700 1 -- start start 2023-07-06T03:56:05.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.753+0000 7f58e27ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f58dc0cd390 0x7f58dc160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.753+0000 7f58e27ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58dc160dd0 0x7f58dc0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.753+0000 7f58e27ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc0cdae0 con 0x7f58dc15c520 2023-07-06T03:56:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.753+0000 7f58e27ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc0b6640 con 0x7f58dc160dd0 2023-07-06T03:56:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.753+0000 7f58e27ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc0b6920 con 0x7f58dc0cd390 2023-07-06T03:56:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.754+0000 7f58e0d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58dc160dd0 0x7f58dc0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.754+0000 7f58e0d9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58dc160dd0 0x7f58dc0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42284/0 (socket says 172.21.15.97:42284) 2023-07-06T03:56:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.754+0000 7f58e0d9c700 1 -- 172.21.15.97:0/583141240 learned_addr learned my addr 172.21.15.97:0/583141240 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.754+0000 7f58d37fe700 1 --2- 172.21.15.97:0/583141240 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0cd390 0x7f58dc160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.754+0000 7f58e0d9c700 1 -- 172.21.15.97:0/583141240 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0cd390 msgr2=0x7f58dc160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.754+0000 7f58e0d9c700 1 --2- 172.21.15.97:0/583141240 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0cd390 0x7f58dc160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.754+0000 7f58e0d9c700 1 -- 172.21.15.97:0/583141240 >> v1:172.21.15.146:6789/0 conn(0x7f58dc15c520 legacy=0x7f58dc160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:05.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.754+0000 7f58e0d9c700 1 -- 172.21.15.97:0/583141240 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc0b6c00 con 0x7f58dc160dd0 2023-07-06T03:56:05.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.754+0000 7f58d37fe700 1 --2- 172.21.15.97:0/583141240 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0cd390 0x7f58dc160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:05.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.755+0000 7f58e0d9c700 1 --2- 172.21.15.97:0/583141240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58dc160dd0 0x7f58dc0b7400 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0b5930 tx=0x7f58d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882223e2523d5c82 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:05.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.755+0000 7f58db7fe700 1 -- 172.21.15.97:0/583141240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400b870 con 0x7f58dc160dd0 2023-07-06T03:56:05.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.755+0000 7f58db7fe700 1 -- 172.21.15.97:0/583141240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d400b9d0 con 0x7f58dc160dd0 2023-07-06T03:56:05.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.755+0000 7f58db7fe700 1 -- 172.21.15.97:0/583141240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400bd60 con 0x7f58dc160dd0 2023-07-06T03:56:05.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.755+0000 7f58e27ff700 1 -- 172.21.15.97:0/583141240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58dc160dd0 msgr2=0x7f58dc0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.755+0000 7f58e27ff700 1 --2- 172.21.15.97:0/583141240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58dc160dd0 0x7f58dc0b7400 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0b5930 tx=0x7f58d4009bd0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.755+0000 7f58e27ff700 1 -- 172.21.15.97:0/583141240 shutdown_connections 2023-07-06T03:56:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.755+0000 7f58e27ff700 1 --2- 172.21.15.97:0/583141240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58dc160dd0 0x7f58dc0b7400 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.755+0000 7f58e27ff700 1 --2- 172.21.15.97:0/583141240 >> v2:172.21.15.146:3300/0 conn(0x7f58dc0cd390 0x7f58dc160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.755+0000 7f58e27ff700 1 -- 172.21.15.97:0/583141240 >> 172.21.15.97:0/583141240 conn(0x7f58dc071790 msgr2=0x7f58dc071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.755+0000 7f58e27ff700 1 -- 172.21.15.97:0/583141240 shutdown_connections 2023-07-06T03:56:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.756+0000 7f58e27ff700 1 -- 172.21.15.97:0/583141240 wait complete. 2023-07-06T03:56:05.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.757+0000 7f58e27ff700 1 Processor -- start 2023-07-06T03:56:05.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.757+0000 7f58e27ff700 1 -- start start 2023-07-06T03:56:05.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.758+0000 7f58e27ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc0cd390 0x7f58dc0cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.758+0000 7f58e27ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc15c520 0x7f58dc14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.758+0000 7f58e27ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58dc160dd0 0x7f58dc0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.758+0000 7f58e27ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0d21d0 con 0x7f58dc160dd0 2023-07-06T03:56:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.758+0000 7f58e27ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0d24b0 con 0x7f58dc0cd390 2023-07-06T03:56:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.758+0000 7f58e27ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0d2790 con 0x7f58dc15c520 2023-07-06T03:56:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.758+0000 7f58e0d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58dc160dd0 0x7f58dc0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58e0d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58dc160dd0 0x7f58dc0cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42300/0 (socket says 172.21.15.97:42300) 2023-07-06T03:56:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58e0d9c700 1 -- 172.21.15.97:0/3820952 learned_addr learned my addr 172.21.15.97:0/3820952 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58d37fe700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc15c520 0x7f58dc14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58dbfff700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc0cd390 0x7f58dc0cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58e0d9c700 1 -- 172.21.15.97:0/3820952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc15c520 msgr2=0x7f58dc14c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58e0d9c700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc15c520 0x7f58dc14c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58e0d9c700 1 -- 172.21.15.97:0/3820952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc0cd390 msgr2=0x7f58dc0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58e0d9c700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc0cd390 0x7f58dc0cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58e0d9c700 1 -- 172.21.15.97:0/3820952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d4009550 con 0x7f58dc160dd0 2023-07-06T03:56:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58d37fe700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc15c520 0x7f58dc14c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58e0d9c700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58dc160dd0 0x7f58dc0cc930 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009c10 tx=0x7f58d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58d97fa700 1 -- 172.21.15.97:0/3820952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4004540 con 0x7f58dc160dd0 2023-07-06T03:56:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58d97fa700 1 -- 172.21.15.97:0/3820952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d40036a0 con 0x7f58dc160dd0 2023-07-06T03:56:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58e27ff700 1 -- 172.21.15.97:0/3820952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc0d2a70 con 0x7f58dc160dd0 2023-07-06T03:56:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.759+0000 7f58d97fa700 1 -- 172.21.15.97:0/3820952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4003a30 con 0x7f58dc160dd0 2023-07-06T03:56:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.760+0000 7f58e27ff700 1 -- 172.21.15.97:0/3820952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc14cdd0 con 0x7f58dc160dd0 2023-07-06T03:56:05.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.761+0000 7f58e27ff700 1 -- 172.21.15.97:0/3820952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc0d2f90 con 0x7f58dc160dd0 2023-07-06T03:56:05.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.761+0000 7f58d97fa700 1 -- 172.21.15.97:0/3820952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f58d4021070 con 0x7f58dc160dd0 2023-07-06T03:56:05.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.762+0000 7f58d97fa700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f58bc06ace0 0x7f58bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:05.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.762+0000 7f58d97fa700 1 -- 172.21.15.97:0/3820952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f58d408aca0 con 0x7f58dc160dd0 2023-07-06T03:56:05.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.763+0000 7f58dbfff700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f58bc06ace0 0x7f58bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:05.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.763+0000 7f58dbfff700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f58bc06ace0 0x7f58bc06d190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0012e0 tx=0x7f58cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:05.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.767+0000 7f58d97fa700 1 -- 172.21.15.97:0/3820952 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f58dc0d2f90 con 0x7f58dc160dd0 2023-07-06T03:56:05.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.965+0000 7f58e27ff700 1 -- 172.21.15.97:0/3820952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58dc0d2f90 con 0x7f58dc160dd0 2023-07-06T03:56:05.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.966+0000 7f58d97fa700 1 -- 172.21.15.97:0/3820952 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+18404 (secure 0 0 0) 0x7f58dc0d2f90 con 0x7f58dc160dd0 2023-07-06T03:56:05.969 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:05.969 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":135,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:56:04.565954+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:56:02.531370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-06T03:56:03.548148+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:56:05.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 -- 172.21.15.97:0/3820952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f58bc06ace0 msgr2=0x7f58bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f58bc06ace0 0x7f58bc06d190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0012e0 tx=0x7f58cc009040 comp rx=0 tx=0).stop 2023-07-06T03:56:05.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 -- 172.21.15.97:0/3820952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58dc160dd0 msgr2=0x7f58dc0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:05.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58dc160dd0 0x7f58dc0cc930 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009c10 tx=0x7f58d4004750 comp rx=0 tx=0).stop 2023-07-06T03:56:05.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 -- 172.21.15.97:0/3820952 shutdown_connections 2023-07-06T03:56:05.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f58bc06ace0 0x7f58bc06d190 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58dc160dd0 0x7f58dc0cc930 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f58dc0cd390 0x7f58dc0cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 --2- 172.21.15.97:0/3820952 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f58dc15c520 0x7f58dc14c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:05.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 -- 172.21.15.97:0/3820952 >> 172.21.15.97:0/3820952 conn(0x7f58dc071790 msgr2=0x7f58dc1605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:05.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 -- 172.21.15.97:0/3820952 shutdown_connections 2023-07-06T03:56:05.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:05.969+0000 7f58e27ff700 1 -- 172.21.15.97:0/3820952 wait complete. 2023-07-06T03:56:05.990 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T03:56:06.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.192+0000 7fdce47ff700 1 Processor -- start 2023-07-06T03:56:06.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.192+0000 7fdce47ff700 1 -- start start 2023-07-06T03:56:06.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.193+0000 7fdce47ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdcdc06b360 0x7fdcdc0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:06.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.193+0000 7fdce47ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc0cff90 con 0x7fdcdc15c610 2023-07-06T03:56:06.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.193+0000 7fdce47ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc0b5650 con 0x7fdcdc0d2210 2023-07-06T03:56:06.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.193+0000 7fdce47ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc0d0780 con 0x7fdcdc06b360 2023-07-06T03:56:06.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.193+0000 7fdce259b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdcdc15c610 0x7fdcdc0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46530/0 (socket says 172.21.15.97:46530) 2023-07-06T03:56:06.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.193+0000 7fdce259b700 1 -- 172.21.15.97:0/4029676122 learned_addr learned my addr 172.21.15.97:0/4029676122 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:06.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1d9a700 1 --2- 172.21.15.97:0/4029676122 >> v2:172.21.15.146:3300/0 conn(0x7fdcdc06b360 0x7fdcdc0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:06.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193118101 0 0) 0x7fdcdc0cff90 con 0x7fdcdc15c610 2023-07-06T03:56:06.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc80035b0 con 0x7fdcdc15c610 2023-07-06T03:56:06.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1898612114 0 0) 0x7fdcc80035b0 con 0x7fdcdc15c610 2023-07-06T03:56:06.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 >> v2:172.21.15.146:3300/0 conn(0x7fdcdc06b360 msgr2=0x7fdcdc0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:06.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 --2- 172.21.15.97:0/4029676122 >> v2:172.21.15.146:3300/0 conn(0x7fdcdc06b360 0x7fdcdc0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:06.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 >> v1:172.21.15.146:6789/0 conn(0x7fdcdc0d2210 legacy=0x7fdcdc0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:06.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0b5400 con 0x7fdcdc15c610 2023-07-06T03:56:06.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fdcd8001670 con 0x7fdcdc15c610 2023-07-06T03:56:06.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 >> v1:172.21.15.97:6789/0 conn(0x7fdcdc15c610 legacy=0x7fdcdc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:06.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 --2- 172.21.15.97:0/4029676122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcc8003a00 0x7fdcc8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:06.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 --2- 172.21.15.97:0/4029676122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcc80043f0 0x7fdcc80048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:06.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 --2- 172.21.15.97:0/4029676122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc80050b0 0x7fdcc800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:06.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc0d0780 con 0x7fdcc80050b0 2023-07-06T03:56:06.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc800c000 con 0x7fdcc80043f0 2023-07-06T03:56:06.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc800c1c0 con 0x7fdcc8003a00 2023-07-06T03:56:06.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.194+0000 7fdce1d9a700 1 --2- 172.21.15.97:0/4029676122 >> v2:172.21.15.146:3300/0 conn(0x7fdcdc06b360 0x7fdcdc0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:56:06.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.195+0000 7fdce2d9c700 1 --2- 172.21.15.97:0/4029676122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc80050b0 0x7fdcc800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:06.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.195+0000 7fdce259b700 1 --2- 172.21.15.97:0/4029676122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcc8003a00 0x7fdcc8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.195+0000 7fdce1d9a700 1 --2- 172.21.15.97:0/4029676122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcc80043f0 0x7fdcc80048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.195+0000 7fdce2d9c700 1 -- 172.21.15.97:0/4029676122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcc8003a00 msgr2=0x7fdcc8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.195+0000 7fdce2d9c700 1 --2- 172.21.15.97:0/4029676122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcc8003a00 0x7fdcc8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.195+0000 7fdce2d9c700 1 -- 172.21.15.97:0/4029676122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcc80043f0 msgr2=0x7fdcc80048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.195+0000 7fdce2d9c700 1 --2- 172.21.15.97:0/4029676122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcc80043f0 0x7fdcc80048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.195+0000 7fdce2d9c700 1 -- 172.21.15.97:0/4029676122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc80035b0 con 0x7fdcc80050b0 2023-07-06T03:56:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.195+0000 7fdce2d9c700 1 -- 172.21.15.97:0/4029676122 reap_dead start 2023-07-06T03:56:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.195+0000 7fdce2d9c700 1 --2- 172.21.15.97:0/4029676122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc80050b0 0x7fdcc800ba10 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400c290 tx=0x7fdcc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cff8785a43e4c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.196+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4001e70 con 0x7fdcc80050b0 2023-07-06T03:56:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.196+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd400ce70 con 0x7fdcc80050b0 2023-07-06T03:56:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.196+0000 7fdce1599700 1 -- 172.21.15.97:0/4029676122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4003c20 con 0x7fdcc80050b0 2023-07-06T03:56:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.196+0000 7fdce47ff700 1 -- 172.21.15.97:0/4029676122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc80050b0 msgr2=0x7fdcc800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.196+0000 7fdce47ff700 1 --2- 172.21.15.97:0/4029676122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc80050b0 0x7fdcc800ba10 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400c290 tx=0x7fdcc8005880 comp rx=0 tx=0).stop 2023-07-06T03:56:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.196+0000 7fdce47ff700 1 -- 172.21.15.97:0/4029676122 shutdown_connections 2023-07-06T03:56:06.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.196+0000 7fdce47ff700 1 --2- 172.21.15.97:0/4029676122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc80050b0 0x7fdcc800ba10 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.196+0000 7fdce47ff700 1 -- 172.21.15.97:0/4029676122 >> 172.21.15.97:0/4029676122 conn(0x7fdcdc06a6c0 msgr2=0x7fdcdc06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.196+0000 7fdce47ff700 1 -- 172.21.15.97:0/4029676122 shutdown_connections 2023-07-06T03:56:06.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.196+0000 7fdce47ff700 1 -- 172.21.15.97:0/4029676122 wait complete. 2023-07-06T03:56:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.198+0000 7fdce47ff700 1 Processor -- start 2023-07-06T03:56:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.198+0000 7fdce47ff700 1 -- start start 2023-07-06T03:56:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.199+0000 7fdce47ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcc80050b0 0x7fdcdc0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.199+0000 7fdce47ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcdc15ddd0 0x7fdcdc152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.199+0000 7fdce47ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc0d1090 0x7fdcdc0d1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.199+0000 7fdce47ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc0d1a70 con 0x7fdcdc0d1090 2023-07-06T03:56:06.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.199+0000 7fdce47ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc15e340 con 0x7fdcdc15ddd0 2023-07-06T03:56:06.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.199+0000 7fdce47ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc15e620 con 0x7fdcc80050b0 2023-07-06T03:56:06.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.199+0000 7fdce2d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc0d1090 0x7fdcdc0d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:06.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.199+0000 7fdce2d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc0d1090 0x7fdcdc0d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42314/0 (socket says 172.21.15.97:42314) 2023-07-06T03:56:06.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.199+0000 7fdce2d9c700 1 -- 172.21.15.97:0/2611779851 learned_addr learned my addr 172.21.15.97:0/2611779851 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:06.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.199+0000 7fdce259b700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcc80050b0 0x7fdcdc0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:06.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.199+0000 7fdce1d9a700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcdc15ddd0 0x7fdcdc152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:06.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.200+0000 7fdce2d9c700 1 -- 172.21.15.97:0/2611779851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcc80050b0 msgr2=0x7fdcdc0cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:06.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.200+0000 7fdce2d9c700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcc80050b0 0x7fdcdc0cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:06.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.200+0000 7fdce2d9c700 1 -- 172.21.15.97:0/2611779851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcdc15ddd0 msgr2=0x7fdcdc152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:06.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.200+0000 7fdce2d9c700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcdc15ddd0 0x7fdcdc152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:06.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.200+0000 7fdce2d9c700 1 -- 172.21.15.97:0/2611779851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc15e900 con 0x7fdcdc0d1090 2023-07-06T03:56:06.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.200+0000 7fdce2d9c700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc0d1090 0x7fdcdc0d1530 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0ce810 tx=0x7fdcd40015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:06.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.200+0000 7fdcd2ffd700 1 -- 172.21.15.97:0/2611779851 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4017070 con 0x7fdcdc0d1090 2023-07-06T03:56:06.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.200+0000 7fdcd2ffd700 1 -- 172.21.15.97:0/2611779851 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd4001250 con 0x7fdcdc0d1090 2023-07-06T03:56:06.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.200+0000 7fdcd2ffd700 1 -- 172.21.15.97:0/2611779851 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40054e0 con 0x7fdcdc0d1090 2023-07-06T03:56:06.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.200+0000 7fdce47ff700 1 -- 172.21.15.97:0/2611779851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc15ebe0 con 0x7fdcdc0d1090 2023-07-06T03:56:06.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.200+0000 7fdce47ff700 1 -- 172.21.15.97:0/2611779851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc0ccbe0 con 0x7fdcdc0d1090 2023-07-06T03:56:06.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.201+0000 7fdce47ff700 1 -- 172.21.15.97:0/2611779851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc15f100 con 0x7fdcdc0d1090 2023-07-06T03:56:06.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.202+0000 7fdcd2ffd700 1 -- 172.21.15.97:0/2611779851 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdcd4005640 con 0x7fdcdc0d1090 2023-07-06T03:56:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.204+0000 7fdcd2ffd700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdcc006ace0 0x7fdcc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.204+0000 7fdcd2ffd700 1 -- 172.21.15.97:0/2611779851 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdcd401a040 con 0x7fdcdc0d1090 2023-07-06T03:56:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.204+0000 7fdce259b700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdcc006ace0 0x7fdcc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.205+0000 7fdce259b700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdcc006ace0 0x7fdcc006d190 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8001050 tx=0x7fdcd8001c30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.208+0000 7fdcd2ffd700 1 -- 172.21.15.97:0/2611779851 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdcdc15f100 con 0x7fdcdc0d1090 2023-07-06T03:56:06.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:06.445+0000 7fdce47ff700 1 -- 172.21.15.97:0/2611779851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdcdc15f100 con 0x7fdcdc0d1090 2023-07-06T03:56:07.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.097+0000 7fdcd2ffd700 1 -- 172.21.15.97:0/2611779851 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v102) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdcdc15f100 con 0x7fdcdc0d1090 2023-07-06T03:56:07.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 -- 172.21.15.97:0/2611779851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdcc006ace0 msgr2=0x7fdcc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:07.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdcc006ace0 0x7fdcc006d190 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8001050 tx=0x7fdcd8001c30 comp rx=0 tx=0).stop 2023-07-06T03:56:07.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 -- 172.21.15.97:0/2611779851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc0d1090 msgr2=0x7fdcdc0d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:07.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc0d1090 0x7fdcdc0d1530 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0ce810 tx=0x7fdcd40015b0 comp rx=0 tx=0).stop 2023-07-06T03:56:07.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 -- 172.21.15.97:0/2611779851 shutdown_connections 2023-07-06T03:56:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdcc006ace0 0x7fdcc006d190 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcdc0d1090 0x7fdcdc0d1530 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdcc80050b0 0x7fdcdc0cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 --2- 172.21.15.97:0/2611779851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdcdc15ddd0 0x7fdcdc152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 -- 172.21.15.97:0/2611779851 >> 172.21.15.97:0/2611779851 conn(0x7fdcdc06a6c0 msgr2=0x7fdcdc15ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 -- 172.21.15.97:0/2611779851 shutdown_connections 2023-07-06T03:56:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.100+0000 7fdce47ff700 1 -- 172.21.15.97:0/2611779851 wait complete. 2023-07-06T03:56:07.124 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T03:56:07.125 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T03:56:07.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:56:07.237 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:56:07.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.328+0000 7fc1a95ff700 1 Processor -- start 2023-07-06T03:56:07.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.328+0000 7fc1a95ff700 1 -- start start 2023-07-06T03:56:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.329+0000 7fc1a95ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a415c590 0x7fc1a415d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.329+0000 7fc1a95ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc1a40719c0 0x7fc1a406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.329+0000 7fc1a95ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a40ccc30 con 0x7fc1a415c1c0 2023-07-06T03:56:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.329+0000 7fc1a95ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a40ccf10 con 0x7fc1a40719c0 2023-07-06T03:56:07.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.329+0000 7fc1a95ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40cd1f0 con 0x7fc1a415c590 2023-07-06T03:56:07.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.329+0000 7fc1a2ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc1a415c1c0 0x7fc1a40cc3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46536/0 (socket says 172.21.15.97:46536) 2023-07-06T03:56:07.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.329+0000 7fc1a2ffd700 1 -- 172.21.15.97:0/2001813911 learned_addr learned my addr 172.21.15.97:0/2001813911 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:07.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.330+0000 7fc1a37fe700 1 --2- 172.21.15.97:0/2001813911 >> v2:172.21.15.146:3300/0 conn(0x7fc1a40719c0 0x7fc1a406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:07.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.330+0000 7fc19bfff700 1 --2- 172.21.15.97:0/2001813911 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a415c590 0x7fc1a415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:07.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.330+0000 7fc1a27fc700 1 -- 172.21.15.97:0/2001813911 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934325001 0 0) 0x7fc1a40ccc30 con 0x7fc1a415c1c0 2023-07-06T03:56:07.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.330+0000 7fc1a27fc700 1 -- 172.21.15.97:0/2001813911 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1900035b0 con 0x7fc1a415c1c0 2023-07-06T03:56:07.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.330+0000 7fc1a37fe700 1 -- 172.21.15.97:0/2001813911 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a415c590 msgr2=0x7fc1a415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:07.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.330+0000 7fc1a37fe700 1 --2- 172.21.15.97:0/2001813911 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a415c590 0x7fc1a415d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:07.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.330+0000 7fc1a37fe700 1 -- 172.21.15.97:0/2001813911 >> v1:172.21.15.97:6789/0 conn(0x7fc1a415c1c0 legacy=0x7fc1a40cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:07.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.330+0000 7fc1a27fc700 1 -- 172.21.15.97:0/2001813911 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1476908123 0 0) 0x7fc1900035b0 con 0x7fc1a415c1c0 2023-07-06T03:56:07.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.330+0000 7fc1a37fe700 1 -- 172.21.15.97:0/2001813911 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a415cb00 con 0x7fc1a40719c0 2023-07-06T03:56:07.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a37fe700 1 --2- 172.21.15.97:0/2001813911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a40719c0 0x7fc1a406a060 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009d90 tx=0x7fc19c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93589154e9fae441 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:07.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a27fc700 1 -- 172.21.15.97:0/2001813911 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00e070 con 0x7fc1a40719c0 2023-07-06T03:56:07.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a27fc700 1 -- 172.21.15.97:0/2001813911 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c003a20 con 0x7fc1a40719c0 2023-07-06T03:56:07.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a27fc700 1 -- 172.21.15.97:0/2001813911 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194001670 con 0x7fc1a40719c0 2023-07-06T03:56:07.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a95ff700 1 -- 172.21.15.97:0/2001813911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a40719c0 msgr2=0x7fc1a406a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:07.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a95ff700 1 --2- 172.21.15.97:0/2001813911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a40719c0 0x7fc1a406a060 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009d90 tx=0x7fc19c00b330 comp rx=0 tx=0).stop 2023-07-06T03:56:07.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a95ff700 1 -- 172.21.15.97:0/2001813911 shutdown_connections 2023-07-06T03:56:07.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a95ff700 1 --2- 172.21.15.97:0/2001813911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a40719c0 0x7fc1a406a060 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:07.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a95ff700 1 --2- 172.21.15.97:0/2001813911 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a415c590 0x7fc1a415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:07.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a95ff700 1 -- 172.21.15.97:0/2001813911 >> 172.21.15.97:0/2001813911 conn(0x7fc1a40b1650 msgr2=0x7fc1a40b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:07.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a95ff700 1 -- 172.21.15.97:0/2001813911 shutdown_connections 2023-07-06T03:56:07.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.331+0000 7fc1a95ff700 1 -- 172.21.15.97:0/2001813911 wait complete. 2023-07-06T03:56:07.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.333+0000 7fc1a95ff700 1 Processor -- start 2023-07-06T03:56:07.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.333+0000 7fc1a95ff700 1 -- start start 2023-07-06T03:56:07.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.334+0000 7fc1a95ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a40719c0 0x7fc1a40c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:07.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.334+0000 7fc1a95ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1a415c1c0 0x7fc1a415ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:07.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.334+0000 7fc1a95ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a415c590 0x7fc1a40b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:07.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.334+0000 7fc1a95ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40c6830 con 0x7fc1a415c1c0 2023-07-06T03:56:07.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.334+0000 7fc1a95ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a415f0f0 con 0x7fc1a40719c0 2023-07-06T03:56:07.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.334+0000 7fc1a95ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40b3f10 con 0x7fc1a415c590 2023-07-06T03:56:07.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.334+0000 7fc19bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1a415c1c0 0x7fc1a415ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:07.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.334+0000 7fc1a2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a40719c0 0x7fc1a40c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:07.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.334+0000 7fc19bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1a415c1c0 0x7fc1a415ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42322/0 (socket says 172.21.15.97:42322) 2023-07-06T03:56:07.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.334+0000 7fc19bfff700 1 -- 172.21.15.97:0/3997701319 learned_addr learned my addr 172.21.15.97:0/3997701319 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:07.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc1a37fe700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a415c590 0x7fc1a40b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:07.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc19bfff700 1 -- 172.21.15.97:0/3997701319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a415c590 msgr2=0x7fc1a40b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:07.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc19bfff700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a415c590 0x7fc1a40b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc19bfff700 1 -- 172.21.15.97:0/3997701319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a40719c0 msgr2=0x7fc1a40c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc19bfff700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a40719c0 0x7fc1a40c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc19bfff700 1 -- 172.21.15.97:0/3997701319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c009a70 con 0x7fc1a415c1c0 2023-07-06T03:56:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc19bfff700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1a415c1c0 0x7fc1a415ebb0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00e9b0 tx=0x7fc18c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc19b7fe700 1 -- 172.21.15.97:0/3997701319 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c009710 con 0x7fc1a415c1c0 2023-07-06T03:56:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc19b7fe700 1 -- 172.21.15.97:0/3997701319 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c00ee50 con 0x7fc1a415c1c0 2023-07-06T03:56:07.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc1a95ff700 1 -- 172.21.15.97:0/3997701319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a415f300 con 0x7fc1a415c1c0 2023-07-06T03:56:07.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc19b7fe700 1 -- 172.21.15.97:0/3997701319 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00ca50 con 0x7fc1a415c1c0 2023-07-06T03:56:07.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.335+0000 7fc1a95ff700 1 -- 172.21.15.97:0/3997701319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a415f910 con 0x7fc1a415c1c0 2023-07-06T03:56:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.337+0000 7fc1a95ff700 1 -- 172.21.15.97:0/3997701319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a404e340 con 0x7fc1a415c1c0 2023-07-06T03:56:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.338+0000 7fc19b7fe700 1 -- 172.21.15.97:0/3997701319 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc18c00cbb0 con 0x7fc1a415c1c0 2023-07-06T03:56:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.338+0000 7fc19b7fe700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc18406ad10 0x7fc18406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.338+0000 7fc19b7fe700 1 -- 172.21.15.97:0/3997701319 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc18c089410 con 0x7fc1a415c1c0 2023-07-06T03:56:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.339+0000 7fc1a2ffd700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc18406ad10 0x7fc18406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:07.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.339+0000 7fc1a2ffd700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc18406ad10 0x7fc18406d1c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc194001eb0 tx=0x7fc194001e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:07.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.343+0000 7fc19b7fe700 1 -- 172.21.15.97:0/3997701319 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc18c057f50 con 0x7fc1a415c1c0 2023-07-06T03:56:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:07.579+0000 7fc1a95ff700 1 -- 172.21.15.97:0/3997701319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc1a4071290 con 0x7fc1a415c1c0 2023-07-06T03:56:08.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.098+0000 7fc19b7fe700 1 -- 172.21.15.97:0/3997701319 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v103) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc18c00ce60 con 0x7fc1a415c1c0 2023-07-06T03:56:08.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 -- 172.21.15.97:0/3997701319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc18406ad10 msgr2=0x7fc18406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:08.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc18406ad10 0x7fc18406d1c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc194001eb0 tx=0x7fc194001e40 comp rx=0 tx=0).stop 2023-07-06T03:56:08.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 -- 172.21.15.97:0/3997701319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1a415c1c0 msgr2=0x7fc1a415ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:08.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1a415c1c0 0x7fc1a415ebb0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00e9b0 tx=0x7fc18c00ecc0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 -- 172.21.15.97:0/3997701319 shutdown_connections 2023-07-06T03:56:08.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc18406ad10 0x7fc18406d1c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1a415c590 0x7fc1a40b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1a40719c0 0x7fc1a40c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 --2- 172.21.15.97:0/3997701319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1a415c1c0 0x7fc1a415ebb0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 -- 172.21.15.97:0/3997701319 >> 172.21.15.97:0/3997701319 conn(0x7fc1a40b1650 msgr2=0x7fc1a415b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:08.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 -- 172.21.15.97:0/3997701319 shutdown_connections 2023-07-06T03:56:08.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.101+0000 7fc1a95ff700 1 -- 172.21.15.97:0/3997701319 wait complete. 2023-07-06T03:56:08.124 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T03:56:08.124 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:56:08.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.359+0000 7fdf9f3ff700 1 Processor -- start 2023-07-06T03:56:08.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.359+0000 7fdf9f3ff700 1 -- start start 2023-07-06T03:56:08.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.360+0000 7fdf9f3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdf98158f20 0x7fdf980caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.360+0000 7fdf9f3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf981592f0 0x7fdf980b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.360+0000 7fdf9f3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf9806d860 0x7fdf980b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.360+0000 7fdf9f3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9815cc40 con 0x7fdf9806d860 2023-07-06T03:56:08.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.360+0000 7fdf9f3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9815cf20 con 0x7fdf98158f20 2023-07-06T03:56:08.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.360+0000 7fdf9f3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9815d200 con 0x7fdf981592f0 2023-07-06T03:56:08.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9d99c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf9806d860 0x7fdf980b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9d99c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf9806d860 0x7fdf980b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42332/0 (socket says 172.21.15.97:42332) 2023-07-06T03:56:08.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9d99c700 1 -- 172.21.15.97:0/4010535660 learned_addr learned my addr 172.21.15.97:0/4010535660 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:08.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9d19b700 1 --2- 172.21.15.97:0/4010535660 >> v2:172.21.15.146:3300/0 conn(0x7fdf98158f20 0x7fdf980caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9c99a700 1 --2- 172.21.15.97:0/4010535660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf981592f0 0x7fdf980b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9d99c700 1 -- 172.21.15.97:0/4010535660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf981592f0 msgr2=0x7fdf980b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:08.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9d99c700 1 --2- 172.21.15.97:0/4010535660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf981592f0 0x7fdf980b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9d99c700 1 -- 172.21.15.97:0/4010535660 >> v2:172.21.15.146:3300/0 conn(0x7fdf98158f20 msgr2=0x7fdf980caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:08.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9d99c700 1 --2- 172.21.15.97:0/4010535660 >> v2:172.21.15.146:3300/0 conn(0x7fdf98158f20 0x7fdf980caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9d99c700 1 -- 172.21.15.97:0/4010535660 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9815d4e0 con 0x7fdf9806d860 2023-07-06T03:56:08.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9c99a700 1 --2- 172.21.15.97:0/4010535660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf981592f0 0x7fdf980b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:08.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.361+0000 7fdf9d99c700 1 --2- 172.21.15.97:0/4010535660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf9806d860 0x7fdf980b0aa0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fdf9806b810 tx=0x7fdf9000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbf3885c9366c11 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf8ffff700 1 -- 172.21.15.97:0/4010535660 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90010070 con 0x7fdf9806d860 2023-07-06T03:56:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf8ffff700 1 -- 172.21.15.97:0/4010535660 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9000dce0 con 0x7fdf9806d860 2023-07-06T03:56:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf8ffff700 1 -- 172.21.15.97:0/4010535660 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000b310 con 0x7fdf9806d860 2023-07-06T03:56:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/4010535660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf9806d860 msgr2=0x7fdf980b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf9f3ff700 1 --2- 172.21.15.97:0/4010535660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf9806d860 0x7fdf980b0aa0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fdf9806b810 tx=0x7fdf9000d430 comp rx=0 tx=0).stop 2023-07-06T03:56:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/4010535660 shutdown_connections 2023-07-06T03:56:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf9f3ff700 1 --2- 172.21.15.97:0/4010535660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf9806d860 0x7fdf980b0aa0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf9f3ff700 1 --2- 172.21.15.97:0/4010535660 >> v2:172.21.15.146:3300/0 conn(0x7fdf98158f20 0x7fdf980caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf9f3ff700 1 --2- 172.21.15.97:0/4010535660 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf981592f0 0x7fdf980b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/4010535660 >> 172.21.15.97:0/4010535660 conn(0x7fdf980ce8c0 msgr2=0x7fdf980cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/4010535660 shutdown_connections 2023-07-06T03:56:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.362+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/4010535660 wait complete. 2023-07-06T03:56:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.364+0000 7fdf9f3ff700 1 Processor -- start 2023-07-06T03:56:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.364+0000 7fdf9f3ff700 1 -- start start 2023-07-06T03:56:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.365+0000 7fdf9f3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf9806d860 0x7fdf980b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.365+0000 7fdf9f3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf98158f20 0x7fdf9813f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.365+0000 7fdf9f3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf981592f0 0x7fdf980af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.365+0000 7fdf9f3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf980b4a10 con 0x7fdf981592f0 2023-07-06T03:56:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.365+0000 7fdf9f3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9815aa30 con 0x7fdf98158f20 2023-07-06T03:56:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.365+0000 7fdf9f3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf980b1c70 con 0x7fdf9806d860 2023-07-06T03:56:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.365+0000 7fdf9d99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf981592f0 0x7fdf980af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.365+0000 7fdf9d99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf981592f0 0x7fdf980af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42334/0 (socket says 172.21.15.97:42334) 2023-07-06T03:56:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.365+0000 7fdf9d99c700 1 -- 172.21.15.97:0/2753405333 learned_addr learned my addr 172.21.15.97:0/2753405333 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.365+0000 7fdf9d19b700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf9806d860 0x7fdf980b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.365+0000 7fdf9c99a700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf98158f20 0x7fdf9813f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf9d99c700 1 -- 172.21.15.97:0/2753405333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf9806d860 msgr2=0x7fdf980b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf9d99c700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf9806d860 0x7fdf980b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf9d99c700 1 -- 172.21.15.97:0/2753405333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf98158f20 msgr2=0x7fdf9813f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf9d99c700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf98158f20 0x7fdf9813f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf9d99c700 1 -- 172.21.15.97:0/2753405333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9000bd40 con 0x7fdf981592f0 2023-07-06T03:56:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf9c99a700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf98158f20 0x7fdf9813f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf9d99c700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf981592f0 0x7fdf980af900 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fdf9806b980 tx=0x7fdf90004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf8dffb700 1 -- 172.21.15.97:0/2753405333 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90009a10 con 0x7fdf981592f0 2023-07-06T03:56:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf8dffb700 1 -- 172.21.15.97:0/2753405333 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf90009b70 con 0x7fdf981592f0 2023-07-06T03:56:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf8dffb700 1 -- 172.21.15.97:0/2753405333 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90004040 con 0x7fdf981592f0 2023-07-06T03:56:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/2753405333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9815ac80 con 0x7fdf981592f0 2023-07-06T03:56:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.366+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/2753405333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9815b240 con 0x7fdf981592f0 2023-07-06T03:56:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.367+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/2753405333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9815b460 con 0x7fdf981592f0 2023-07-06T03:56:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.368+0000 7fdf8dffb700 1 -- 172.21.15.97:0/2753405333 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf900041a0 con 0x7fdf981592f0 2023-07-06T03:56:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.369+0000 7fdf8dffb700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf7c06ace0 0x7fdf7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.369+0000 7fdf8dffb700 1 -- 172.21.15.97:0/2753405333 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdf9008cf20 con 0x7fdf981592f0 2023-07-06T03:56:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.369+0000 7fdf9d19b700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf7c06ace0 0x7fdf7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.370+0000 7fdf9d19b700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf7c06ace0 0x7fdf7c06d190 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdf98152ed0 tx=0x7fdf94009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.373+0000 7fdf8dffb700 1 -- 172.21.15.97:0/2753405333 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf90057fc0 con 0x7fdf981592f0 2023-07-06T03:56:08.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.611+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/2753405333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf9804e340 con 0x7fdf981592f0 2023-07-06T03:56:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.611+0000 7fdf8dffb700 1 -- 172.21.15.97:0/2753405333 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4680 (secure 0 0 0) 0x7fdf9005b5e0 con 0x7fdf981592f0 2023-07-06T03:56:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:08.614 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/3313682816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3313682816},{"type":"v1","addr":"172.21.15.146:6834","nonce":3313682816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9624,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/387461585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":387461585},{"type":"v1","addr":"172.21.15.146:6835","nonce":387461585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9630,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3109027091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3109027091},{"type":"v1","addr":"172.21.15.97:6835","nonce":3109027091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:56:04.566296+0000","modified":"2023-07-06T03:56:08.096494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9633":{"gid":9633,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2797569799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2797569799},{"type":"v1","addr":"172.21.15.97:6837","nonce":2797569799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-06T03:56:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.613+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/2753405333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf7c06ace0 msgr2=0x7fdf7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.614+0000 7fdf9f3ff700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf7c06ace0 0x7fdf7c06d190 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdf98152ed0 tx=0x7fdf94009210 comp rx=0 tx=0).stop 2023-07-06T03:56:08.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.614+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/2753405333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf981592f0 msgr2=0x7fdf980af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:08.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.614+0000 7fdf9f3ff700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf981592f0 0x7fdf980af900 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fdf9806b980 tx=0x7fdf90004ad0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.614+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/2753405333 shutdown_connections 2023-07-06T03:56:08.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.614+0000 7fdf9f3ff700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdf7c06ace0 0x7fdf7c06d190 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.614+0000 7fdf9f3ff700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf981592f0 0x7fdf980af900 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.614+0000 7fdf9f3ff700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdf9806d860 0x7fdf980b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.614+0000 7fdf9f3ff700 1 --2- 172.21.15.97:0/2753405333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf98158f20 0x7fdf9813f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.614+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/2753405333 >> 172.21.15.97:0/2753405333 conn(0x7fdf980ce8c0 msgr2=0x7fdf980cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:08.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.614+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/2753405333 shutdown_connections 2023-07-06T03:56:08.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.614+0000 7fdf9f3ff700 1 -- 172.21.15.97:0/2753405333 wait complete. 2023-07-06T03:56:08.618 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 103 2023-07-06T03:56:08.634 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:56:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.826+0000 7fc1be9ff700 1 Processor -- start 2023-07-06T03:56:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.826+0000 7fc1be9ff700 1 -- start start 2023-07-06T03:56:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1be9ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc1b815c740 0x7fc1b80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1be9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc1b8071820 0x7fc1b806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1be9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b80cdc80 con 0x7fc1b8071c90 2023-07-06T03:56:08.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1be9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b80d0610 con 0x7fc1b8071820 2023-07-06T03:56:08.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1be9ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b80ce380 con 0x7fc1b815c740 2023-07-06T03:56:08.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1b77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc1b8071820 0x7fc1b806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1b77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc1b8071820 0x7fc1b806a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42344/0 (socket says 172.21.15.97:42344) 2023-07-06T03:56:08.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1b77fe700 1 -- 172.21.15.97:0/2015276514 learned_addr learned my addr 172.21.15.97:0/2015276514 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:08.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1b7fff700 1 --2- 172.21.15.97:0/2015276514 >> v2:172.21.15.146:3300/0 conn(0x7fc1b815c740 0x7fc1b80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1b77fe700 1 -- 172.21.15.97:0/2015276514 >> v2:172.21.15.146:3300/0 conn(0x7fc1b815c740 msgr2=0x7fc1b80d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:08.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1b77fe700 1 --2- 172.21.15.97:0/2015276514 >> v2:172.21.15.146:3300/0 conn(0x7fc1b815c740 0x7fc1b80d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1b77fe700 1 -- 172.21.15.97:0/2015276514 >> v1:172.21.15.146:6789/0 conn(0x7fc1b8071c90 legacy=0x7fc1b80ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:08.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1b77fe700 1 -- 172.21.15.97:0/2015276514 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b80cd630 con 0x7fc1b8071820 2023-07-06T03:56:08.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.828+0000 7fc1b7fff700 1 --2- 172.21.15.97:0/2015276514 >> v2:172.21.15.146:3300/0 conn(0x7fc1b815c740 0x7fc1b80d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:56:08.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.829+0000 7fc1b77fe700 1 --2- 172.21.15.97:0/2015276514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1b8071820 0x7fc1b806a060 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0098f0 tx=0x7fc1ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd926dfbfac4a35 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:08.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.829+0000 7fc1b6ffd700 1 -- 172.21.15.97:0/2015276514 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac004230 con 0x7fc1b8071820 2023-07-06T03:56:08.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.829+0000 7fc1b6ffd700 1 -- 172.21.15.97:0/2015276514 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac004390 con 0x7fc1b8071820 2023-07-06T03:56:08.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.829+0000 7fc1b6ffd700 1 -- 172.21.15.97:0/2015276514 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0036a0 con 0x7fc1b8071820 2023-07-06T03:56:08.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.829+0000 7fc1be9ff700 1 -- 172.21.15.97:0/2015276514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1b8071820 msgr2=0x7fc1b806a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:08.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.829+0000 7fc1be9ff700 1 --2- 172.21.15.97:0/2015276514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1b8071820 0x7fc1b806a060 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0098f0 tx=0x7fc1ac009c00 comp rx=0 tx=0).stop 2023-07-06T03:56:08.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.830+0000 7fc1be9ff700 1 -- 172.21.15.97:0/2015276514 shutdown_connections 2023-07-06T03:56:08.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.830+0000 7fc1be9ff700 1 --2- 172.21.15.97:0/2015276514 >> v2:172.21.15.146:3300/0 conn(0x7fc1b815c740 0x7fc1b80d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.830+0000 7fc1be9ff700 1 --2- 172.21.15.97:0/2015276514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1b8071820 0x7fc1b806a060 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.830+0000 7fc1be9ff700 1 -- 172.21.15.97:0/2015276514 >> 172.21.15.97:0/2015276514 conn(0x7fc1b80b2280 msgr2=0x7fc1b80cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:08.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.830+0000 7fc1be9ff700 1 -- 172.21.15.97:0/2015276514 shutdown_connections 2023-07-06T03:56:08.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.830+0000 7fc1be9ff700 1 -- 172.21.15.97:0/2015276514 wait complete. 2023-07-06T03:56:08.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.831+0000 7fc1be9ff700 1 Processor -- start 2023-07-06T03:56:08.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.831+0000 7fc1be9ff700 1 -- start start 2023-07-06T03:56:08.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.832+0000 7fc1be9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1b8071820 0x7fc1b8150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.832+0000 7fc1be9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1b815c740 0x7fc1b806df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.832+0000 7fc1be9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1b8150000 0x7fc1b81f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.832+0000 7fc1be9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80b3e50 con 0x7fc1b8150000 2023-07-06T03:56:08.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.832+0000 7fc1be9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80b4130 con 0x7fc1b8071820 2023-07-06T03:56:08.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.832+0000 7fc1be9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80b4410 con 0x7fc1b815c740 2023-07-06T03:56:08.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.833+0000 7fc1bcf9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1b8150000 0x7fc1b81f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.833+0000 7fc1bcf9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1b8150000 0x7fc1b81f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42354/0 (socket says 172.21.15.97:42354) 2023-07-06T03:56:08.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.833+0000 7fc1bcf9c700 1 -- 172.21.15.97:0/203106611 learned_addr learned my addr 172.21.15.97:0/203106611 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:08.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.833+0000 7fc1b7fff700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1b8071820 0x7fc1b8150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.833+0000 7fc1b77fe700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1b815c740 0x7fc1b806df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.833+0000 7fc1bcf9c700 1 -- 172.21.15.97:0/203106611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1b815c740 msgr2=0x7fc1b806df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:08.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.833+0000 7fc1bcf9c700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1b815c740 0x7fc1b806df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.833+0000 7fc1bcf9c700 1 -- 172.21.15.97:0/203106611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1b8071820 msgr2=0x7fc1b8150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:08.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.833+0000 7fc1bcf9c700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1b8071820 0x7fc1b8150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:08.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.833+0000 7fc1bcf9c700 1 -- 172.21.15.97:0/203106611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac009550 con 0x7fc1b8150000 2023-07-06T03:56:08.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.833+0000 7fc1bcf9c700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1b8150000 0x7fc1b81f20a0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000c780 tx=0x7fc1b000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:08.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.834+0000 7fc1b4ff9700 1 -- 172.21.15.97:0/203106611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0003e10 con 0x7fc1b8150000 2023-07-06T03:56:08.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.834+0000 7fc1b4ff9700 1 -- 172.21.15.97:0/203106611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b00050d0 con 0x7fc1b8150000 2023-07-06T03:56:08.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.834+0000 7fc1be9ff700 1 -- 172.21.15.97:0/203106611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b80b46f0 con 0x7fc1b8150000 2023-07-06T03:56:08.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.834+0000 7fc1b4ff9700 1 -- 172.21.15.97:0/203106611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0005480 con 0x7fc1b8150000 2023-07-06T03:56:08.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.834+0000 7fc1be9ff700 1 -- 172.21.15.97:0/203106611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b8150b10 con 0x7fc1b8150000 2023-07-06T03:56:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.835+0000 7fc1be9ff700 1 -- 172.21.15.97:0/203106611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b80b4c10 con 0x7fc1b8150000 2023-07-06T03:56:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.836+0000 7fc1b4ff9700 1 -- 172.21.15.97:0/203106611 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc1b00055e0 con 0x7fc1b8150000 2023-07-06T03:56:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.837+0000 7fc1b4ff9700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1a406ace0 0x7fc1a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.837+0000 7fc1b4ff9700 1 -- 172.21.15.97:0/203106611 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc1b0088830 con 0x7fc1b8150000 2023-07-06T03:56:08.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.837+0000 7fc1b7fff700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1a406ace0 0x7fc1a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:08.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.838+0000 7fc1b7fff700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1a406ace0 0x7fc1a406d190 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800abe0 tx=0x7fc1a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:08.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:08.841+0000 7fc1b4ff9700 1 -- 172.21.15.97:0/203106611 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc1b80b4c10 con 0x7fc1b8150000 2023-07-06T03:56:09.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.039+0000 7fc1be9ff700 1 -- 172.21.15.97:0/203106611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1b80b4c10 con 0x7fc1b8150000 2023-07-06T03:56:09.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.040+0000 7fc1b4ff9700 1 -- 172.21.15.97:0/203106611 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18391 (secure 0 0 0) 0x7fc1b80b4c10 con 0x7fc1b8150000 2023-07-06T03:56:09.043 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:09.043 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":136,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:56:06.083480+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:56:02.531370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-06T03:56:03.548148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:56:09.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 -- 172.21.15.97:0/203106611 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1a406ace0 msgr2=0x7fc1a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1a406ace0 0x7fc1a406d190 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800abe0 tx=0x7fc1a8008040 comp rx=0 tx=0).stop 2023-07-06T03:56:09.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 -- 172.21.15.97:0/203106611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1b8150000 msgr2=0x7fc1b81f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1b8150000 0x7fc1b81f20a0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000c780 tx=0x7fc1b000ca90 comp rx=0 tx=0).stop 2023-07-06T03:56:09.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 -- 172.21.15.97:0/203106611 shutdown_connections 2023-07-06T03:56:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1a406ace0 0x7fc1a406d190 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1b8150000 0x7fc1b81f20a0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1b8071820 0x7fc1b8150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 --2- 172.21.15.97:0/203106611 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc1b815c740 0x7fc1b806df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 -- 172.21.15.97:0/203106611 >> 172.21.15.97:0/203106611 conn(0x7fc1b80b2280 msgr2=0x7fc1b806a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 -- 172.21.15.97:0/203106611 shutdown_connections 2023-07-06T03:56:09.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.042+0000 7fc1be9ff700 1 -- 172.21.15.97:0/203106611 wait complete. 2023-07-06T03:56:09.054 INFO:tasks.ceph.mds.c.smithi097.stderr:2023-07-06T03:56:09.052+0000 7f880a1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:56:09.063 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T03:56:09.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:56:09.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.256+0000 7f9a119ff700 1 Processor -- start 2023-07-06T03:56:09.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.257+0000 7f9a119ff700 1 -- start start 2023-07-06T03:56:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.258+0000 7f9a119ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9a0c0b0c70 0x7f9a0c163fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.258+0000 7f9a119ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c15cc40 con 0x7f9a0c158f20 2023-07-06T03:56:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.258+0000 7f9a119ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c15cf20 con 0x7f9a0c1592f0 2023-07-06T03:56:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.258+0000 7f9a119ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c15d200 con 0x7f9a0c0b0c70 2023-07-06T03:56:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.258+0000 7f9a0affd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9a0c158f20 0x7f9a0c0caac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46538/0 (socket says 172.21.15.97:46538) 2023-07-06T03:56:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.258+0000 7f9a0affd700 1 -- 172.21.15.97:0/1265908082 learned_addr learned my addr 172.21.15.97:0/1265908082 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.258+0000 7f9a0b7fe700 1 --2- 172.21.15.97:0/1265908082 >> v2:172.21.15.97:3300/0 conn(0x7f9a0c0b0c70 0x7f9a0c163fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.259+0000 7f9a0b7fe700 1 -- 172.21.15.97:0/1265908082 >> v1:172.21.15.146:6789/0 conn(0x7f9a0c1592f0 legacy=0x7f9a0c0b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.259+0000 7f9a0a7fc700 1 -- 172.21.15.97:0/1265908082 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524401388 0 0) 0x7f9a0c15cc40 con 0x7f9a0c158f20 2023-07-06T03:56:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.259+0000 7f9a0b7fe700 1 -- 172.21.15.97:0/1265908082 >> v1:172.21.15.97:6789/0 conn(0x7f9a0c158f20 legacy=0x7f9a0c0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.259+0000 7f9a0b7fe700 1 -- 172.21.15.97:0/1265908082 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c15d4e0 con 0x7f9a0c0b0c70 2023-07-06T03:56:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.259+0000 7f9a0b7fe700 1 --2- 172.21.15.97:0/1265908082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a0c0b0c70 0x7f9a0c163fd0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009d90 tx=0x7f9a0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6998c117b12cdc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.259+0000 7f9a0a7fc700 1 -- 172.21.15.97:0/1265908082 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400e070 con 0x7f9a0c0b0c70 2023-07-06T03:56:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.259+0000 7f9a0a7fc700 1 -- 172.21.15.97:0/1265908082 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0400bbd0 con 0x7f9a0c0b0c70 2023-07-06T03:56:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.259+0000 7f9a0a7fc700 1 -- 172.21.15.97:0/1265908082 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04012430 con 0x7f9a0c0b0c70 2023-07-06T03:56:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.260+0000 7f9a119ff700 1 -- 172.21.15.97:0/1265908082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a0c0b0c70 msgr2=0x7f9a0c163fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.260+0000 7f9a119ff700 1 --2- 172.21.15.97:0/1265908082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a0c0b0c70 0x7f9a0c163fd0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009d90 tx=0x7f9a0400b330 comp rx=0 tx=0).stop 2023-07-06T03:56:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.260+0000 7f9a119ff700 1 -- 172.21.15.97:0/1265908082 shutdown_connections 2023-07-06T03:56:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.260+0000 7f9a119ff700 1 --2- 172.21.15.97:0/1265908082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a0c0b0c70 0x7f9a0c163fd0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.260+0000 7f9a119ff700 1 -- 172.21.15.97:0/1265908082 >> 172.21.15.97:0/1265908082 conn(0x7f9a0c0ce8c0 msgr2=0x7f9a0c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.260+0000 7f9a119ff700 1 -- 172.21.15.97:0/1265908082 shutdown_connections 2023-07-06T03:56:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.260+0000 7f9a119ff700 1 -- 172.21.15.97:0/1265908082 wait complete. 2023-07-06T03:56:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.261+0000 7f9a119ff700 1 Processor -- start 2023-07-06T03:56:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.262+0000 7f9a119ff700 1 -- start start 2023-07-06T03:56:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.262+0000 7f9a119ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a0c0b0c70 0x7f9a0c0b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.262+0000 7f9a119ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a0c158f20 0x7f9a0c15c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a119ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a0c1592f0 0x7f9a0c144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a119ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c15c640 con 0x7f9a0c158f20 2023-07-06T03:56:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a119ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c159800 con 0x7f9a0c1592f0 2023-07-06T03:56:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a119ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0b4480 con 0x7f9a0c0b0c70 2023-07-06T03:56:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a03fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a0c158f20 0x7f9a0c15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a03fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a0c158f20 0x7f9a0c15c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42378/0 (socket says 172.21.15.97:42378) 2023-07-06T03:56:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a03fff700 1 -- 172.21.15.97:0/3559874840 learned_addr learned my addr 172.21.15.97:0/3559874840 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a0b7fe700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a0c1592f0 0x7f9a0c144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a0affd700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a0c0b0c70 0x7f9a0c0b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a03fff700 1 -- 172.21.15.97:0/3559874840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a0c0b0c70 msgr2=0x7f9a0c0b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a03fff700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a0c0b0c70 0x7f9a0c0b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a03fff700 1 -- 172.21.15.97:0/3559874840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a0c1592f0 msgr2=0x7f9a0c144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a03fff700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a0c1592f0 0x7f9a0c144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a03fff700 1 -- 172.21.15.97:0/3559874840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04009a70 con 0x7f9a0c158f20 2023-07-06T03:56:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a0affd700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a0c0b0c70 0x7f9a0c0b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:56:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.263+0000 7f9a03fff700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a0c158f20 0x7f9a0c15c100 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c450 tx=0x7f99f400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.264+0000 7f9a08ff9700 1 -- 172.21.15.97:0/3559874840 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f40052d0 con 0x7f9a0c158f20 2023-07-06T03:56:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.264+0000 7f9a08ff9700 1 -- 172.21.15.97:0/3559874840 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f4005430 con 0x7f9a0c158f20 2023-07-06T03:56:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.264+0000 7f9a119ff700 1 -- 172.21.15.97:0/3559874840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c0b4760 con 0x7f9a0c158f20 2023-07-06T03:56:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.264+0000 7f9a08ff9700 1 -- 172.21.15.97:0/3559874840 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f40038c0 con 0x7f9a0c158f20 2023-07-06T03:56:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.264+0000 7f9a119ff700 1 -- 172.21.15.97:0/3559874840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c159d60 con 0x7f9a0c158f20 2023-07-06T03:56:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.265+0000 7f9a119ff700 1 -- 172.21.15.97:0/3559874840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c04e340 con 0x7f9a0c158f20 2023-07-06T03:56:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.266+0000 7f9a08ff9700 1 -- 172.21.15.97:0/3559874840 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f99f4003a20 con 0x7f9a0c158f20 2023-07-06T03:56:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.267+0000 7f9a08ff9700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99ec06ace0 0x7f99ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.267+0000 7f9a08ff9700 1 -- 172.21.15.97:0/3559874840 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99f4088830 con 0x7f9a0c158f20 2023-07-06T03:56:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.267+0000 7f9a0affd700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99ec06ace0 0x7f99ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.268+0000 7f9a0affd700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99ec06ace0 0x7f99ec06d190 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f99fc000f60 tx=0x7f99fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.271+0000 7f9a08ff9700 1 -- 172.21.15.97:0/3559874840 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f99f40538d0 con 0x7f9a0c158f20 2023-07-06T03:56:09.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.462+0000 7f9a119ff700 1 -- 172.21.15.97:0/3559874840 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a0c15d7c0 con 0x7f99ec06ace0 2023-07-06T03:56:09.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.471+0000 7f9a08ff9700 1 -- 172.21.15.97:0/3559874840 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311894 (secure 0 0 0) 0x7f99fc001c90 con 0x7f99ec06ace0 2023-07-06T03:56:09.479 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:09.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 -- 172.21.15.97:0/3559874840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99ec06ace0 msgr2=0x7f99ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99ec06ace0 0x7f99ec06d190 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f99fc000f60 tx=0x7f99fc009040 comp rx=0 tx=0).stop 2023-07-06T03:56:09.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 -- 172.21.15.97:0/3559874840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a0c158f20 msgr2=0x7f9a0c15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a0c158f20 0x7f9a0c15c100 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c450 tx=0x7f99f400c810 comp rx=0 tx=0).stop 2023-07-06T03:56:09.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 -- 172.21.15.97:0/3559874840 shutdown_connections 2023-07-06T03:56:09.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f99ec06ace0 0x7f99ec06d190 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a0c1592f0 0x7f9a0c144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9a0c0b0c70 0x7f9a0c0b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 --2- 172.21.15.97:0/3559874840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a0c158f20 0x7f9a0c15c100 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 -- 172.21.15.97:0/3559874840 >> 172.21.15.97:0/3559874840 conn(0x7f9a0c0ce8c0 msgr2=0x7f9a0c06bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 -- 172.21.15.97:0/3559874840 shutdown_connections 2023-07-06T03:56:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.475+0000 7f9a119ff700 1 -- 172.21.15.97:0/3559874840 wait complete. 2023-07-06T03:56:09.486 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:56:09.496 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2023-07-06T03:56:09.024961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":291492,"kb_used_data":14436,"kb_used_omap":0,"kb_used_meta":276992,"kb_avail":754683228,"statfs":{"total":773094113280,"available":772795625472,"internally_reserved":0,"allocated":14782464,"data_stored":7411321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":283639808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.959979"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106091+0000","last_change":"2023-07-06T03:56:05.102566+0000","last_active":"2023-07-06T03:56:05.106091+0000","last_peered":"2023-07-06T03:56:05.106091+0000","last_clean":"2023-07-06T03:56:05.106091+0000","last_became_active":"2023-07-06T03:56:05.102312+0000","last_became_peered":"2023-07-06T03:56:05.102312+0000","last_unstale":"2023-07-06T03:56:05.106091+0000","last_undegraded":"2023-07-06T03:56:05.106091+0000","last_fullsized":"2023-07-06T03:56:05.106091+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:23:24.373621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.104555+0000","last_change":"2023-07-06T03:56:05.101011+0000","last_active":"2023-07-06T03:56:05.104555+0000","last_peered":"2023-07-06T03:56:05.104555+0000","last_clean":"2023-07-06T03:56:05.104555+0000","last_became_active":"2023-07-06T03:56:05.100690+0000","last_became_peered":"2023-07-06T03:56:05.100690+0000","last_unstale":"2023-07-06T03:56:05.104555+0000","last_undegraded":"2023-07-06T03:56:05.104555+0000","last_fullsized":"2023-07-06T03:56:05.104555+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:40:44.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.114050+0000","last_change":"2023-07-06T03:56:05.110365+0000","last_active":"2023-07-06T03:56:05.114050+0000","last_peered":"2023-07-06T03:56:05.114050+0000","last_clean":"2023-07-06T03:56:05.114050+0000","last_became_active":"2023-07-06T03:56:05.110123+0000","last_became_peered":"2023-07-06T03:56:05.110123+0000","last_unstale":"2023-07-06T03:56:05.114050+0000","last_undegraded":"2023-07-06T03:56:05.114050+0000","last_fullsized":"2023-07-06T03:56:05.114050+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:12.875713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.114977+0000","last_change":"2023-07-06T03:56:05.109510+0000","last_active":"2023-07-06T03:56:05.114977+0000","last_peered":"2023-07-06T03:56:05.114977+0000","last_clean":"2023-07-06T03:56:05.114977+0000","last_became_active":"2023-07-06T03:56:05.109215+0000","last_became_peered":"2023-07-06T03:56:05.109215+0000","last_unstale":"2023-07-06T03:56:05.114977+0000","last_undegraded":"2023-07-06T03:56:05.114977+0000","last_fullsized":"2023-07-06T03:56:05.114977+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:14:23.567839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.095034+0000","last_change":"2023-07-06T03:56:05.092060+0000","last_active":"2023-07-06T03:56:05.095034+0000","last_peered":"2023-07-06T03:56:05.095034+0000","last_clean":"2023-07-06T03:56:05.095034+0000","last_became_active":"2023-07-06T03:56:05.091762+0000","last_became_peered":"2023-07-06T03:56:05.091762+0000","last_unstale":"2023-07-06T03:56:05.095034+0000","last_undegraded":"2023-07-06T03:56:05.095034+0000","last_fullsized":"2023-07-06T03:56:05.095034+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:50:10.281115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.102559+0000","last_change":"2023-07-06T03:56:05.099340+0000","last_active":"2023-07-06T03:56:05.102559+0000","last_peered":"2023-07-06T03:56:05.102559+0000","last_clean":"2023-07-06T03:56:05.102559+0000","last_became_active":"2023-07-06T03:56:05.098815+0000","last_became_peered":"2023-07-06T03:56:05.098815+0000","last_unstale":"2023-07-06T03:56:05.102559+0000","last_undegraded":"2023-07-06T03:56:05.102559+0000","last_fullsized":"2023-07-06T03:56:05.102559+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:54:14.973450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106217+0000","last_change":"2023-07-06T03:56:05.100869+0000","last_active":"2023-07-06T03:56:05.106217+0000","last_peered":"2023-07-06T03:56:05.106217+0000","last_clean":"2023-07-06T03:56:05.106217+0000","last_became_active":"2023-07-06T03:56:05.100629+0000","last_became_peered":"2023-07-06T03:56:05.100629+0000","last_unstale":"2023-07-06T03:56:05.106217+0000","last_undegraded":"2023-07-06T03:56:05.106217+0000","last_fullsized":"2023-07-06T03:56:05.106217+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:27:55.441438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.107882+0000","last_change":"2023-07-06T03:56:05.095474+0000","last_active":"2023-07-06T03:56:05.107882+0000","last_peered":"2023-07-06T03:56:05.107882+0000","last_clean":"2023-07-06T03:56:05.107882+0000","last_became_active":"2023-07-06T03:56:05.095226+0000","last_became_peered":"2023-07-06T03:56:05.095226+0000","last_unstale":"2023-07-06T03:56:05.107882+0000","last_undegraded":"2023-07-06T03:56:05.107882+0000","last_fullsized":"2023-07-06T03:56:05.107882+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:18:27.163996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.104365+0000","last_change":"2023-07-06T03:56:05.101045+0000","last_active":"2023-07-06T03:56:05.104365+0000","last_peered":"2023-07-06T03:56:05.104365+0000","last_clean":"2023-07-06T03:56:05.104365+0000","last_became_active":"2023-07-06T03:56:05.100752+0000","last_became_peered":"2023-07-06T03:56:05.100752+0000","last_unstale":"2023-07-06T03:56:05.104365+0000","last_undegraded":"2023-07-06T03:56:05.104365+0000","last_fullsized":"2023-07-06T03:56:05.104365+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:29.971340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.097318+0000","last_change":"2023-07-06T03:56:05.094856+0000","last_active":"2023-07-06T03:56:05.097318+0000","last_peered":"2023-07-06T03:56:05.097318+0000","last_clean":"2023-07-06T03:56:05.097318+0000","last_became_active":"2023-07-06T03:56:05.094546+0000","last_became_peered":"2023-07-06T03:56:05.094546+0000","last_unstale":"2023-07-06T03:56:05.097318+0000","last_undegraded":"2023-07-06T03:56:05.097318+0000","last_fullsized":"2023-07-06T03:56:05.097318+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:52:56.136942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.098383+0000","last_change":"2023-07-06T03:56:05.088677+0000","last_active":"2023-07-06T03:56:05.098383+0000","last_peered":"2023-07-06T03:56:05.098383+0000","last_clean":"2023-07-06T03:56:05.098383+0000","last_became_active":"2023-07-06T03:56:05.088380+0000","last_became_peered":"2023-07-06T03:56:05.088380+0000","last_unstale":"2023-07-06T03:56:05.098383+0000","last_undegraded":"2023-07-06T03:56:05.098383+0000","last_fullsized":"2023-07-06T03:56:05.098383+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:38:39.597004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105716+0000","last_change":"2023-07-06T03:56:05.100990+0000","last_active":"2023-07-06T03:56:05.105716+0000","last_peered":"2023-07-06T03:56:05.105716+0000","last_clean":"2023-07-06T03:56:05.105716+0000","last_became_active":"2023-07-06T03:56:05.100738+0000","last_became_peered":"2023-07-06T03:56:05.100738+0000","last_unstale":"2023-07-06T03:56:05.105716+0000","last_undegraded":"2023-07-06T03:56:05.105716+0000","last_fullsized":"2023-07-06T03:56:05.105716+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:03:49.915504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.097813+0000","last_change":"2023-07-06T03:56:05.094945+0000","last_active":"2023-07-06T03:56:05.097813+0000","last_peered":"2023-07-06T03:56:05.097813+0000","last_clean":"2023-07-06T03:56:05.097813+0000","last_became_active":"2023-07-06T03:56:05.094372+0000","last_became_peered":"2023-07-06T03:56:05.094372+0000","last_unstale":"2023-07-06T03:56:05.097813+0000","last_undegraded":"2023-07-06T03:56:05.097813+0000","last_fullsized":"2023-07-06T03:56:05.097813+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:16:48.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106483+0000","last_change":"2023-07-06T03:56:05.092638+0000","last_active":"2023-07-06T03:56:05.106483+0000","last_peered":"2023-07-06T03:56:05.106483+0000","last_clean":"2023-07-06T03:56:05.106483+0000","last_became_active":"2023-07-06T03:56:05.091776+0000","last_became_peered":"2023-07-06T03:56:05.091776+0000","last_unstale":"2023-07-06T03:56:05.106483+0000","last_undegraded":"2023-07-06T03:56:05.106483+0000","last_fullsized":"2023-07-06T03:56:05.106483+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:36.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.099942+0000","last_change":"2023-07-06T03:56:05.098266+0000","last_active":"2023-07-06T03:56:05.099942+0000","last_peered":"2023-07-06T03:56:05.099942+0000","last_clean":"2023-07-06T03:56:05.099942+0000","last_became_active":"2023-07-06T03:56:05.097946+0000","last_became_peered":"2023-07-06T03:56:05.097946+0000","last_unstale":"2023-07-06T03:56:05.099942+0000","last_undegraded":"2023-07-06T03:56:05.099942+0000","last_fullsized":"2023-07-06T03:56:05.099942+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:40.999860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.099069+0000","last_change":"2023-07-06T03:56:05.094602+0000","last_active":"2023-07-06T03:56:05.099069+0000","last_peered":"2023-07-06T03:56:05.099069+0000","last_clean":"2023-07-06T03:56:05.099069+0000","last_became_active":"2023-07-06T03:56:05.094238+0000","last_became_peered":"2023-07-06T03:56:05.094238+0000","last_unstale":"2023-07-06T03:56:05.099069+0000","last_undegraded":"2023-07-06T03:56:05.099069+0000","last_fullsized":"2023-07-06T03:56:05.099069+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:08.743741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105730+0000","last_change":"2023-07-06T03:56:05.101635+0000","last_active":"2023-07-06T03:56:05.105730+0000","last_peered":"2023-07-06T03:56:05.105730+0000","last_clean":"2023-07-06T03:56:05.105730+0000","last_became_active":"2023-07-06T03:56:05.101388+0000","last_became_peered":"2023-07-06T03:56:05.101388+0000","last_unstale":"2023-07-06T03:56:05.105730+0000","last_undegraded":"2023-07-06T03:56:05.105730+0000","last_fullsized":"2023-07-06T03:56:05.105730+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:53.524456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.104536+0000","last_change":"2023-07-06T03:56:05.099850+0000","last_active":"2023-07-06T03:56:05.104536+0000","last_peered":"2023-07-06T03:56:05.104536+0000","last_clean":"2023-07-06T03:56:05.104536+0000","last_became_active":"2023-07-06T03:56:05.099517+0000","last_became_peered":"2023-07-06T03:56:05.099517+0000","last_unstale":"2023-07-06T03:56:05.104536+0000","last_undegraded":"2023-07-06T03:56:05.104536+0000","last_fullsized":"2023-07-06T03:56:05.104536+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:38:39.597004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105976+0000","last_change":"2023-07-06T03:56:05.101527+0000","last_active":"2023-07-06T03:56:05.105976+0000","last_peered":"2023-07-06T03:56:05.105976+0000","last_clean":"2023-07-06T03:56:05.105976+0000","last_became_active":"2023-07-06T03:56:05.101274+0000","last_became_peered":"2023-07-06T03:56:05.101274+0000","last_unstale":"2023-07-06T03:56:05.105976+0000","last_undegraded":"2023-07-06T03:56:05.105976+0000","last_fullsized":"2023-07-06T03:56:05.105976+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:53.326549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.092944+0000","last_change":"2023-07-06T03:56:05.087765+0000","last_active":"2023-07-06T03:56:05.092944+0000","last_peered":"2023-07-06T03:56:05.092944+0000","last_clean":"2023-07-06T03:56:05.092944+0000","last_became_active":"2023-07-06T03:56:05.087368+0000","last_became_peered":"2023-07-06T03:56:05.087368+0000","last_unstale":"2023-07-06T03:56:05.092944+0000","last_undegraded":"2023-07-06T03:56:05.092944+0000","last_fullsized":"2023-07-06T03:56:05.092944+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:59:57.954018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103511+0000","last_change":"2023-07-06T03:56:05.102189+0000","last_active":"2023-07-06T03:56:05.103511+0000","last_peered":"2023-07-06T03:56:05.103511+0000","last_clean":"2023-07-06T03:56:05.103511+0000","last_became_active":"2023-07-06T03:56:05.101927+0000","last_became_peered":"2023-07-06T03:56:05.101927+0000","last_unstale":"2023-07-06T03:56:05.103511+0000","last_undegraded":"2023-07-06T03:56:05.103511+0000","last_fullsized":"2023-07-06T03:56:05.103511+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:59:09.558612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103750+0000","last_change":"2023-07-06T03:56:05.098329+0000","last_active":"2023-07-06T03:56:05.103750+0000","last_peered":"2023-07-06T03:56:05.103750+0000","last_clean":"2023-07-06T03:56:05.103750+0000","last_became_active":"2023-07-06T03:56:05.098019+0000","last_became_peered":"2023-07-06T03:56:05.098019+0000","last_unstale":"2023-07-06T03:56:05.103750+0000","last_undegraded":"2023-07-06T03:56:05.103750+0000","last_fullsized":"2023-07-06T03:56:05.103750+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:05:07.560883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109655+0000","last_change":"2023-07-06T03:56:05.102409+0000","last_active":"2023-07-06T03:56:05.109655+0000","last_peered":"2023-07-06T03:56:05.109655+0000","last_clean":"2023-07-06T03:56:05.109655+0000","last_became_active":"2023-07-06T03:56:05.102093+0000","last_became_peered":"2023-07-06T03:56:05.102093+0000","last_unstale":"2023-07-06T03:56:05.109655+0000","last_undegraded":"2023-07-06T03:56:05.109655+0000","last_fullsized":"2023-07-06T03:56:05.109655+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:28.963453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.097489+0000","last_change":"2023-07-06T03:56:05.095474+0000","last_active":"2023-07-06T03:56:05.097489+0000","last_peered":"2023-07-06T03:56:05.097489+0000","last_clean":"2023-07-06T03:56:05.097489+0000","last_became_active":"2023-07-06T03:56:05.095200+0000","last_became_peered":"2023-07-06T03:56:05.095200+0000","last_unstale":"2023-07-06T03:56:05.097489+0000","last_undegraded":"2023-07-06T03:56:05.097489+0000","last_fullsized":"2023-07-06T03:56:05.097489+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:34:34.744575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106075+0000","last_change":"2023-07-06T03:56:05.101234+0000","last_active":"2023-07-06T03:56:05.106075+0000","last_peered":"2023-07-06T03:56:05.106075+0000","last_clean":"2023-07-06T03:56:05.106075+0000","last_became_active":"2023-07-06T03:56:05.100984+0000","last_became_peered":"2023-07-06T03:56:05.100984+0000","last_unstale":"2023-07-06T03:56:05.106075+0000","last_undegraded":"2023-07-06T03:56:05.106075+0000","last_fullsized":"2023-07-06T03:56:05.106075+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:59:05.963284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.112087+0000","last_change":"2023-07-06T03:56:05.107186+0000","last_active":"2023-07-06T03:56:05.112087+0000","last_peered":"2023-07-06T03:56:05.112087+0000","last_clean":"2023-07-06T03:56:05.112087+0000","last_became_active":"2023-07-06T03:56:05.106939+0000","last_became_peered":"2023-07-06T03:56:05.106939+0000","last_unstale":"2023-07-06T03:56:05.112087+0000","last_undegraded":"2023-07-06T03:56:05.112087+0000","last_fullsized":"2023-07-06T03:56:05.112087+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:36:38.071390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103769+0000","last_change":"2023-07-06T03:56:05.101832+0000","last_active":"2023-07-06T03:56:05.103769+0000","last_peered":"2023-07-06T03:56:05.103769+0000","last_clean":"2023-07-06T03:56:05.103769+0000","last_became_active":"2023-07-06T03:56:05.101533+0000","last_became_peered":"2023-07-06T03:56:05.101533+0000","last_unstale":"2023-07-06T03:56:05.103769+0000","last_undegraded":"2023-07-06T03:56:05.103769+0000","last_fullsized":"2023-07-06T03:56:05.103769+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:26:56.657153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.094726+0000","last_change":"2023-07-06T03:56:05.090513+0000","last_active":"2023-07-06T03:56:05.094726+0000","last_peered":"2023-07-06T03:56:05.094726+0000","last_clean":"2023-07-06T03:56:05.094726+0000","last_became_active":"2023-07-06T03:56:05.090184+0000","last_became_peered":"2023-07-06T03:56:05.090184+0000","last_unstale":"2023-07-06T03:56:05.094726+0000","last_undegraded":"2023-07-06T03:56:05.094726+0000","last_fullsized":"2023-07-06T03:56:05.094726+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:00.715722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.110243+0000","last_change":"2023-07-06T03:56:05.099956+0000","last_active":"2023-07-06T03:56:05.110243+0000","last_peered":"2023-07-06T03:56:05.110243+0000","last_clean":"2023-07-06T03:56:05.110243+0000","last_became_active":"2023-07-06T03:56:05.099440+0000","last_became_peered":"2023-07-06T03:56:05.099440+0000","last_unstale":"2023-07-06T03:56:05.110243+0000","last_undegraded":"2023-07-06T03:56:05.110243+0000","last_fullsized":"2023-07-06T03:56:05.110243+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:06:56.996144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103736+0000","last_change":"2023-07-06T03:56:05.101805+0000","last_active":"2023-07-06T03:56:05.103736+0000","last_peered":"2023-07-06T03:56:05.103736+0000","last_clean":"2023-07-06T03:56:05.103736+0000","last_became_active":"2023-07-06T03:56:05.101484+0000","last_became_peered":"2023-07-06T03:56:05.101484+0000","last_unstale":"2023-07-06T03:56:05.103736+0000","last_undegraded":"2023-07-06T03:56:05.103736+0000","last_fullsized":"2023-07-06T03:56:05.103736+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:33:48.403671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.098978+0000","last_change":"2023-07-06T03:56:05.094534+0000","last_active":"2023-07-06T03:56:05.098978+0000","last_peered":"2023-07-06T03:56:05.098978+0000","last_clean":"2023-07-06T03:56:05.098978+0000","last_became_active":"2023-07-06T03:56:05.093962+0000","last_became_peered":"2023-07-06T03:56:05.093962+0000","last_unstale":"2023-07-06T03:56:05.098978+0000","last_undegraded":"2023-07-06T03:56:05.098978+0000","last_fullsized":"2023-07-06T03:56:05.098978+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:16:48.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105926+0000","last_change":"2023-07-06T03:56:05.092257+0000","last_active":"2023-07-06T03:56:05.105926+0000","last_peered":"2023-07-06T03:56:05.105926+0000","last_clean":"2023-07-06T03:56:05.105926+0000","last_became_active":"2023-07-06T03:56:05.091600+0000","last_became_peered":"2023-07-06T03:56:05.091600+0000","last_unstale":"2023-07-06T03:56:05.105926+0000","last_undegraded":"2023-07-06T03:56:05.105926+0000","last_fullsized":"2023-07-06T03:56:05.105926+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:40:10.012028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.098988+0000","last_change":"2023-07-06T03:56:05.095256+0000","last_active":"2023-07-06T03:56:05.098988+0000","last_peered":"2023-07-06T03:56:05.098988+0000","last_clean":"2023-07-06T03:56:05.098988+0000","last_became_active":"2023-07-06T03:56:05.094971+0000","last_became_peered":"2023-07-06T03:56:05.094971+0000","last_unstale":"2023-07-06T03:56:05.098988+0000","last_undegraded":"2023-07-06T03:56:05.098988+0000","last_fullsized":"2023-07-06T03:56:05.098988+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:00.715722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1f","version":"135'4","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.112060+0000","last_change":"2023-07-06T03:56:04.090531+0000","last_active":"2023-07-06T03:56:05.112060+0000","last_peered":"2023-07-06T03:56:05.112060+0000","last_clean":"2023-07-06T03:56:05.112060+0000","last_became_active":"2023-07-06T03:56:04.090226+0000","last_became_peered":"2023-07-06T03:56:04.090226+0000","last_unstale":"2023-07-06T03:56:05.112060+0000","last_undegraded":"2023-07-06T03:56:05.112060+0000","last_fullsized":"2023-07-06T03:56:05.112060+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:28.963054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.100055+0000","last_change":"2023-07-06T03:56:05.096745+0000","last_active":"2023-07-06T03:56:05.100055+0000","last_peered":"2023-07-06T03:56:05.100055+0000","last_clean":"2023-07-06T03:56:05.100055+0000","last_became_active":"2023-07-06T03:56:05.096433+0000","last_became_peered":"2023-07-06T03:56:05.096433+0000","last_unstale":"2023-07-06T03:56:05.100055+0000","last_undegraded":"2023-07-06T03:56:05.100055+0000","last_fullsized":"2023-07-06T03:56:05.100055+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:03:05.882071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.113153+0000","last_change":"2023-07-06T03:56:04.088217+0000","last_active":"2023-07-06T03:56:05.113153+0000","last_peered":"2023-07-06T03:56:05.113153+0000","last_clean":"2023-07-06T03:56:05.113153+0000","last_became_active":"2023-07-06T03:56:04.087967+0000","last_became_peered":"2023-07-06T03:56:04.087967+0000","last_unstale":"2023-07-06T03:56:05.113153+0000","last_undegraded":"2023-07-06T03:56:05.113153+0000","last_fullsized":"2023-07-06T03:56:05.113153+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:59:40.517315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105704+0000","last_change":"2023-07-06T03:56:05.102191+0000","last_active":"2023-07-06T03:56:05.105704+0000","last_peered":"2023-07-06T03:56:05.105704+0000","last_clean":"2023-07-06T03:56:05.105704+0000","last_became_active":"2023-07-06T03:56:05.101955+0000","last_became_peered":"2023-07-06T03:56:05.101955+0000","last_unstale":"2023-07-06T03:56:05.105704+0000","last_undegraded":"2023-07-06T03:56:05.105704+0000","last_fullsized":"2023-07-06T03:56:05.105704+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:00:11.725190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1d","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.108145+0000","last_change":"2023-07-06T03:56:04.090809+0000","last_active":"2023-07-06T03:56:05.108145+0000","last_peered":"2023-07-06T03:56:05.108145+0000","last_clean":"2023-07-06T03:56:05.108145+0000","last_became_active":"2023-07-06T03:56:04.090507+0000","last_became_peered":"2023-07-06T03:56:04.090507+0000","last_unstale":"2023-07-06T03:56:05.108145+0000","last_undegraded":"2023-07-06T03:56:05.108145+0000","last_fullsized":"2023-07-06T03:56:05.108145+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:17:46.967407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.101033+0000","last_change":"2023-07-06T03:56:05.090898+0000","last_active":"2023-07-06T03:56:05.101033+0000","last_peered":"2023-07-06T03:56:05.101033+0000","last_clean":"2023-07-06T03:56:05.101033+0000","last_became_active":"2023-07-06T03:56:05.090638+0000","last_became_peered":"2023-07-06T03:56:05.090638+0000","last_unstale":"2023-07-06T03:56:05.101033+0000","last_undegraded":"2023-07-06T03:56:05.101033+0000","last_fullsized":"2023-07-06T03:56:05.101033+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:03:11.267195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1c","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.110647+0000","last_change":"2023-07-06T03:56:04.093434+0000","last_active":"2023-07-06T03:56:05.110647+0000","last_peered":"2023-07-06T03:56:05.110647+0000","last_clean":"2023-07-06T03:56:05.110647+0000","last_became_active":"2023-07-06T03:56:04.093126+0000","last_became_peered":"2023-07-06T03:56:04.093126+0000","last_unstale":"2023-07-06T03:56:05.110647+0000","last_undegraded":"2023-07-06T03:56:05.110647+0000","last_fullsized":"2023-07-06T03:56:05.110647+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:05:32.312327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.098368+0000","last_change":"2023-07-06T03:56:05.090648+0000","last_active":"2023-07-06T03:56:05.098368+0000","last_peered":"2023-07-06T03:56:05.098368+0000","last_clean":"2023-07-06T03:56:05.098368+0000","last_became_active":"2023-07-06T03:56:05.090392+0000","last_became_peered":"2023-07-06T03:56:05.090392+0000","last_unstale":"2023-07-06T03:56:05.098368+0000","last_undegraded":"2023-07-06T03:56:05.098368+0000","last_fullsized":"2023-07-06T03:56:05.098368+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:40:44.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1b","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.107799+0000","last_change":"2023-07-06T03:56:04.090894+0000","last_active":"2023-07-06T03:56:05.107799+0000","last_peered":"2023-07-06T03:56:05.107799+0000","last_clean":"2023-07-06T03:56:05.107799+0000","last_became_active":"2023-07-06T03:56:04.090607+0000","last_became_peered":"2023-07-06T03:56:04.090607+0000","last_unstale":"2023-07-06T03:56:05.107799+0000","last_undegraded":"2023-07-06T03:56:05.107799+0000","last_fullsized":"2023-07-06T03:56:05.107799+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:58:22.191968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.101416+0000","last_change":"2023-07-06T03:56:05.090478+0000","last_active":"2023-07-06T03:56:05.101416+0000","last_peered":"2023-07-06T03:56:05.101416+0000","last_clean":"2023-07-06T03:56:05.101416+0000","last_became_active":"2023-07-06T03:56:05.090192+0000","last_became_peered":"2023-07-06T03:56:05.090192+0000","last_unstale":"2023-07-06T03:56:05.101416+0000","last_undegraded":"2023-07-06T03:56:05.101416+0000","last_fullsized":"2023-07-06T03:56:05.101416+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:19:26.415133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109983+0000","last_change":"2023-07-06T03:56:04.089431+0000","last_active":"2023-07-06T03:56:05.109983+0000","last_peered":"2023-07-06T03:56:05.109983+0000","last_clean":"2023-07-06T03:56:05.109983+0000","last_became_active":"2023-07-06T03:56:04.089127+0000","last_became_peered":"2023-07-06T03:56:04.089127+0000","last_unstale":"2023-07-06T03:56:05.109983+0000","last_undegraded":"2023-07-06T03:56:05.109983+0000","last_fullsized":"2023-07-06T03:56:05.109983+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:34.275532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.111597+0000","last_change":"2023-07-06T03:56:05.107762+0000","last_active":"2023-07-06T03:56:05.111597+0000","last_peered":"2023-07-06T03:56:05.111597+0000","last_clean":"2023-07-06T03:56:05.111597+0000","last_became_active":"2023-07-06T03:56:05.107464+0000","last_became_peered":"2023-07-06T03:56:05.107464+0000","last_unstale":"2023-07-06T03:56:05.111597+0000","last_undegraded":"2023-07-06T03:56:05.111597+0000","last_fullsized":"2023-07-06T03:56:05.111597+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:43:14.304018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.078809+0000","last_change":"2023-07-06T03:56:04.089266+0000","last_active":"2023-07-06T03:56:05.078809+0000","last_peered":"2023-07-06T03:56:05.078809+0000","last_clean":"2023-07-06T03:56:05.078809+0000","last_became_active":"2023-07-06T03:56:04.089012+0000","last_became_peered":"2023-07-06T03:56:04.089012+0000","last_unstale":"2023-07-06T03:56:05.078809+0000","last_undegraded":"2023-07-06T03:56:05.078809+0000","last_fullsized":"2023-07-06T03:56:05.078809+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:51:23.674938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105861+0000","last_change":"2023-07-06T03:56:05.091804+0000","last_active":"2023-07-06T03:56:05.105861+0000","last_peered":"2023-07-06T03:56:05.105861+0000","last_clean":"2023-07-06T03:56:05.105861+0000","last_became_active":"2023-07-06T03:56:05.091527+0000","last_became_peered":"2023-07-06T03:56:05.091527+0000","last_unstale":"2023-07-06T03:56:05.105861+0000","last_undegraded":"2023-07-06T03:56:05.105861+0000","last_fullsized":"2023-07-06T03:56:05.105861+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:46:03.435316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.081889+0000","last_change":"2023-07-06T03:56:04.095698+0000","last_active":"2023-07-06T03:56:05.081889+0000","last_peered":"2023-07-06T03:56:05.081889+0000","last_clean":"2023-07-06T03:56:05.081889+0000","last_became_active":"2023-07-06T03:56:04.095342+0000","last_became_peered":"2023-07-06T03:56:04.095342+0000","last_unstale":"2023-07-06T03:56:05.081889+0000","last_undegraded":"2023-07-06T03:56:05.081889+0000","last_fullsized":"2023-07-06T03:56:05.081889+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:15:56.489035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106023+0000","last_change":"2023-07-06T03:56:05.101450+0000","last_active":"2023-07-06T03:56:05.106023+0000","last_peered":"2023-07-06T03:56:05.106023+0000","last_clean":"2023-07-06T03:56:05.106023+0000","last_became_active":"2023-07-06T03:56:05.101193+0000","last_became_peered":"2023-07-06T03:56:05.101193+0000","last_unstale":"2023-07-06T03:56:05.106023+0000","last_undegraded":"2023-07-06T03:56:05.106023+0000","last_fullsized":"2023-07-06T03:56:05.106023+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:41:34.959137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.f","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.110159+0000","last_change":"2023-07-06T03:56:04.088074+0000","last_active":"2023-07-06T03:56:05.110159+0000","last_peered":"2023-07-06T03:56:05.110159+0000","last_clean":"2023-07-06T03:56:05.110159+0000","last_became_active":"2023-07-06T03:56:04.087740+0000","last_became_peered":"2023-07-06T03:56:04.087740+0000","last_unstale":"2023-07-06T03:56:05.110159+0000","last_undegraded":"2023-07-06T03:56:05.110159+0000","last_fullsized":"2023-07-06T03:56:05.110159+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:07.735456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.108792+0000","last_change":"2023-07-06T03:56:05.097349+0000","last_active":"2023-07-06T03:56:05.108792+0000","last_peered":"2023-07-06T03:56:05.108792+0000","last_clean":"2023-07-06T03:56:05.108792+0000","last_became_active":"2023-07-06T03:56:05.096893+0000","last_became_peered":"2023-07-06T03:56:05.096893+0000","last_unstale":"2023-07-06T03:56:05.108792+0000","last_undegraded":"2023-07-06T03:56:05.108792+0000","last_fullsized":"2023-07-06T03:56:05.108792+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:43:39.208495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.e","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.112705+0000","last_change":"2023-07-06T03:56:04.095085+0000","last_active":"2023-07-06T03:56:05.112705+0000","last_peered":"2023-07-06T03:56:05.112705+0000","last_clean":"2023-07-06T03:56:05.112705+0000","last_became_active":"2023-07-06T03:56:04.094822+0000","last_became_peered":"2023-07-06T03:56:04.094822+0000","last_unstale":"2023-07-06T03:56:05.112705+0000","last_undegraded":"2023-07-06T03:56:05.112705+0000","last_fullsized":"2023-07-06T03:56:05.112705+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:19.124771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105684+0000","last_change":"2023-07-06T03:56:05.101133+0000","last_active":"2023-07-06T03:56:05.105684+0000","last_peered":"2023-07-06T03:56:05.105684+0000","last_clean":"2023-07-06T03:56:05.105684+0000","last_became_active":"2023-07-06T03:56:05.100876+0000","last_became_peered":"2023-07-06T03:56:05.100876+0000","last_unstale":"2023-07-06T03:56:05.105684+0000","last_undegraded":"2023-07-06T03:56:05.105684+0000","last_fullsized":"2023-07-06T03:56:05.105684+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:20.392368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.d","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.112490+0000","last_change":"2023-07-06T03:56:04.093254+0000","last_active":"2023-07-06T03:56:05.112490+0000","last_peered":"2023-07-06T03:56:05.112490+0000","last_clean":"2023-07-06T03:56:05.112490+0000","last_became_active":"2023-07-06T03:56:04.092953+0000","last_became_peered":"2023-07-06T03:56:04.092953+0000","last_unstale":"2023-07-06T03:56:05.112490+0000","last_undegraded":"2023-07-06T03:56:05.112490+0000","last_fullsized":"2023-07-06T03:56:05.112490+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:58:03.548601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109789+0000","last_change":"2023-07-06T03:56:05.102505+0000","last_active":"2023-07-06T03:56:05.109789+0000","last_peered":"2023-07-06T03:56:05.109789+0000","last_clean":"2023-07-06T03:56:05.109789+0000","last_became_active":"2023-07-06T03:56:05.102259+0000","last_became_peered":"2023-07-06T03:56:05.102259+0000","last_unstale":"2023-07-06T03:56:05.109789+0000","last_undegraded":"2023-07-06T03:56:05.109789+0000","last_fullsized":"2023-07-06T03:56:05.109789+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:38:04.098842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.089921+0000","last_change":"2023-07-06T03:56:04.093165+0000","last_active":"2023-07-06T03:56:05.089921+0000","last_peered":"2023-07-06T03:56:05.089921+0000","last_clean":"2023-07-06T03:56:05.089921+0000","last_became_active":"2023-07-06T03:56:04.092884+0000","last_became_peered":"2023-07-06T03:56:04.092884+0000","last_unstale":"2023-07-06T03:56:05.089921+0000","last_undegraded":"2023-07-06T03:56:05.089921+0000","last_fullsized":"2023-07-06T03:56:05.089921+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:08:37.967648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.098617+0000","last_change":"2023-07-06T03:56:05.096458+0000","last_active":"2023-07-06T03:56:05.098617+0000","last_peered":"2023-07-06T03:56:05.098617+0000","last_clean":"2023-07-06T03:56:05.098617+0000","last_became_active":"2023-07-06T03:56:05.096197+0000","last_became_peered":"2023-07-06T03:56:05.096197+0000","last_unstale":"2023-07-06T03:56:05.098617+0000","last_undegraded":"2023-07-06T03:56:05.098617+0000","last_fullsized":"2023-07-06T03:56:05.098617+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:34:44.981824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.b","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.110841+0000","last_change":"2023-07-06T03:56:04.089565+0000","last_active":"2023-07-06T03:56:05.110841+0000","last_peered":"2023-07-06T03:56:05.110841+0000","last_clean":"2023-07-06T03:56:05.110841+0000","last_became_active":"2023-07-06T03:56:04.089271+0000","last_became_peered":"2023-07-06T03:56:04.089271+0000","last_unstale":"2023-07-06T03:56:05.110841+0000","last_undegraded":"2023-07-06T03:56:05.110841+0000","last_fullsized":"2023-07-06T03:56:05.110841+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:59:08.550327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.111567+0000","last_change":"2023-07-06T03:56:05.107541+0000","last_active":"2023-07-06T03:56:05.111567+0000","last_peered":"2023-07-06T03:56:05.111567+0000","last_clean":"2023-07-06T03:56:05.111567+0000","last_became_active":"2023-07-06T03:56:05.107228+0000","last_became_peered":"2023-07-06T03:56:05.107228+0000","last_unstale":"2023-07-06T03:56:05.111567+0000","last_undegraded":"2023-07-06T03:56:05.111567+0000","last_fullsized":"2023-07-06T03:56:05.111567+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:12:08.635554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.085847+0000","last_change":"2023-07-06T03:56:04.095177+0000","last_active":"2023-07-06T03:56:05.085847+0000","last_peered":"2023-07-06T03:56:05.085847+0000","last_clean":"2023-07-06T03:56:05.085847+0000","last_became_active":"2023-07-06T03:56:04.094886+0000","last_became_peered":"2023-07-06T03:56:04.094886+0000","last_unstale":"2023-07-06T03:56:05.085847+0000","last_undegraded":"2023-07-06T03:56:05.085847+0000","last_fullsized":"2023-07-06T03:56:05.085847+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:30:13.899596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103769+0000","last_change":"2023-07-06T03:56:05.098785+0000","last_active":"2023-07-06T03:56:05.103769+0000","last_peered":"2023-07-06T03:56:05.103769+0000","last_clean":"2023-07-06T03:56:05.103769+0000","last_became_active":"2023-07-06T03:56:05.098528+0000","last_became_peered":"2023-07-06T03:56:05.098528+0000","last_unstale":"2023-07-06T03:56:05.103769+0000","last_undegraded":"2023-07-06T03:56:05.103769+0000","last_fullsized":"2023-07-06T03:56:05.103769+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:25:52.346892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.080696+0000","last_change":"2023-07-06T03:56:04.085393+0000","last_active":"2023-07-06T03:56:05.080696+0000","last_peered":"2023-07-06T03:56:05.080696+0000","last_clean":"2023-07-06T03:56:05.080696+0000","last_became_active":"2023-07-06T03:56:04.085076+0000","last_became_peered":"2023-07-06T03:56:04.085076+0000","last_unstale":"2023-07-06T03:56:05.080696+0000","last_undegraded":"2023-07-06T03:56:05.080696+0000","last_fullsized":"2023-07-06T03:56:05.080696+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:21:58.582831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103454+0000","last_change":"2023-07-06T03:56:05.101013+0000","last_active":"2023-07-06T03:56:05.103454+0000","last_peered":"2023-07-06T03:56:05.103454+0000","last_clean":"2023-07-06T03:56:05.103454+0000","last_became_active":"2023-07-06T03:56:05.100735+0000","last_became_peered":"2023-07-06T03:56:05.100735+0000","last_unstale":"2023-07-06T03:56:05.103454+0000","last_undegraded":"2023-07-06T03:56:05.103454+0000","last_fullsized":"2023-07-06T03:56:05.103454+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:35.283817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.085702+0000","last_change":"2023-07-06T03:56:04.092939+0000","last_active":"2023-07-06T03:56:05.085702+0000","last_peered":"2023-07-06T03:56:05.085702+0000","last_clean":"2023-07-06T03:56:05.085702+0000","last_became_active":"2023-07-06T03:56:04.092588+0000","last_became_peered":"2023-07-06T03:56:04.092588+0000","last_unstale":"2023-07-06T03:56:05.085702+0000","last_undegraded":"2023-07-06T03:56:05.085702+0000","last_fullsized":"2023-07-06T03:56:05.085702+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:40:43.287184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103377+0000","last_change":"2023-07-06T03:56:05.100900+0000","last_active":"2023-07-06T03:56:05.103377+0000","last_peered":"2023-07-06T03:56:05.103377+0000","last_clean":"2023-07-06T03:56:05.103377+0000","last_became_active":"2023-07-06T03:56:05.100629+0000","last_became_peered":"2023-07-06T03:56:05.100629+0000","last_unstale":"2023-07-06T03:56:05.103377+0000","last_undegraded":"2023-07-06T03:56:05.103377+0000","last_fullsized":"2023-07-06T03:56:05.103377+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:58:01.135846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.7","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.122286+0000","last_change":"2023-07-06T03:56:04.090746+0000","last_active":"2023-07-06T03:56:05.122286+0000","last_peered":"2023-07-06T03:56:05.122286+0000","last_clean":"2023-07-06T03:56:05.122286+0000","last_became_active":"2023-07-06T03:56:04.090444+0000","last_became_peered":"2023-07-06T03:56:04.090444+0000","last_unstale":"2023-07-06T03:56:05.122286+0000","last_undegraded":"2023-07-06T03:56:05.122286+0000","last_fullsized":"2023-07-06T03:56:05.122286+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:12:39.160706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.107936+0000","last_change":"2023-07-06T03:56:05.094632+0000","last_active":"2023-07-06T03:56:05.107936+0000","last_peered":"2023-07-06T03:56:05.107936+0000","last_clean":"2023-07-06T03:56:05.107936+0000","last_became_active":"2023-07-06T03:56:05.093861+0000","last_became_peered":"2023-07-06T03:56:05.093861+0000","last_unstale":"2023-07-06T03:56:05.107936+0000","last_undegraded":"2023-07-06T03:56:05.107936+0000","last_fullsized":"2023-07-06T03:56:05.107936+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:14:38.102238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.079684+0000","last_change":"2023-07-06T03:56:04.090622+0000","last_active":"2023-07-06T03:56:05.079684+0000","last_peered":"2023-07-06T03:56:05.079684+0000","last_clean":"2023-07-06T03:56:05.079684+0000","last_became_active":"2023-07-06T03:56:04.090339+0000","last_became_peered":"2023-07-06T03:56:04.090339+0000","last_unstale":"2023-07-06T03:56:05.079684+0000","last_undegraded":"2023-07-06T03:56:05.079684+0000","last_fullsized":"2023-07-06T03:56:05.079684+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:16:38.519342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.122421+0000","last_change":"2023-07-06T03:56:05.112839+0000","last_active":"2023-07-06T03:56:05.122421+0000","last_peered":"2023-07-06T03:56:05.122421+0000","last_clean":"2023-07-06T03:56:05.122421+0000","last_became_active":"2023-07-06T03:56:05.112499+0000","last_became_peered":"2023-07-06T03:56:05.112499+0000","last_unstale":"2023-07-06T03:56:05.122421+0000","last_undegraded":"2023-07-06T03:56:05.122421+0000","last_fullsized":"2023-07-06T03:56:05.122421+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:25:44.026540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.084990+0000","last_change":"2023-07-06T03:56:04.090547+0000","last_active":"2023-07-06T03:56:05.084990+0000","last_peered":"2023-07-06T03:56:05.084990+0000","last_clean":"2023-07-06T03:56:05.084990+0000","last_became_active":"2023-07-06T03:56:04.090262+0000","last_became_peered":"2023-07-06T03:56:04.090262+0000","last_unstale":"2023-07-06T03:56:05.084990+0000","last_undegraded":"2023-07-06T03:56:05.084990+0000","last_fullsized":"2023-07-06T03:56:05.084990+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:17.973105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.097784+0000","last_change":"2023-07-06T03:56:05.094478+0000","last_active":"2023-07-06T03:56:05.097784+0000","last_peered":"2023-07-06T03:56:05.097784+0000","last_clean":"2023-07-06T03:56:05.097784+0000","last_became_active":"2023-07-06T03:56:05.094164+0000","last_became_peered":"2023-07-06T03:56:05.094164+0000","last_unstale":"2023-07-06T03:56:05.097784+0000","last_undegraded":"2023-07-06T03:56:05.097784+0000","last_fullsized":"2023-07-06T03:56:05.097784+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:09:32.189142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.084788+0000","last_change":"2023-07-06T03:56:04.088850+0000","last_active":"2023-07-06T03:56:05.084788+0000","last_peered":"2023-07-06T03:56:05.084788+0000","last_clean":"2023-07-06T03:56:05.084788+0000","last_became_active":"2023-07-06T03:56:04.088555+0000","last_became_peered":"2023-07-06T03:56:04.088555+0000","last_unstale":"2023-07-06T03:56:05.084788+0000","last_undegraded":"2023-07-06T03:56:05.084788+0000","last_fullsized":"2023-07-06T03:56:05.084788+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:25:17.423773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106126+0000","last_change":"2023-07-06T03:56:05.092775+0000","last_active":"2023-07-06T03:56:05.106126+0000","last_peered":"2023-07-06T03:56:05.106126+0000","last_clean":"2023-07-06T03:56:05.106126+0000","last_became_active":"2023-07-06T03:56:05.091941+0000","last_became_peered":"2023-07-06T03:56:05.091941+0000","last_unstale":"2023-07-06T03:56:05.106126+0000","last_undegraded":"2023-07-06T03:56:05.106126+0000","last_fullsized":"2023-07-06T03:56:05.106126+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:34:59.543438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.079578+0000","last_change":"2023-07-06T03:56:04.090435+0000","last_active":"2023-07-06T03:56:05.079578+0000","last_peered":"2023-07-06T03:56:05.079578+0000","last_clean":"2023-07-06T03:56:05.079578+0000","last_became_active":"2023-07-06T03:56:04.090050+0000","last_became_peered":"2023-07-06T03:56:04.090050+0000","last_unstale":"2023-07-06T03:56:05.079578+0000","last_undegraded":"2023-07-06T03:56:05.079578+0000","last_fullsized":"2023-07-06T03:56:05.079578+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:20:48.464045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.108708+0000","last_change":"2023-07-06T03:56:05.100061+0000","last_active":"2023-07-06T03:56:05.108708+0000","last_peered":"2023-07-06T03:56:05.108708+0000","last_clean":"2023-07-06T03:56:05.108708+0000","last_became_active":"2023-07-06T03:56:05.099762+0000","last_became_peered":"2023-07-06T03:56:05.099762+0000","last_unstale":"2023-07-06T03:56:05.108708+0000","last_undegraded":"2023-07-06T03:56:05.108708+0000","last_fullsized":"2023-07-06T03:56:05.108708+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:55:16.541099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.078692+0000","last_change":"2023-07-06T03:56:04.095116+0000","last_active":"2023-07-06T03:56:05.078692+0000","last_peered":"2023-07-06T03:56:05.078692+0000","last_clean":"2023-07-06T03:56:05.078692+0000","last_became_active":"2023-07-06T03:56:04.094806+0000","last_became_peered":"2023-07-06T03:56:04.094806+0000","last_unstale":"2023-07-06T03:56:05.078692+0000","last_undegraded":"2023-07-06T03:56:05.078692+0000","last_fullsized":"2023-07-06T03:56:05.078692+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:15:56.489035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.102537+0000","last_change":"2023-07-06T03:56:05.099792+0000","last_active":"2023-07-06T03:56:05.102537+0000","last_peered":"2023-07-06T03:56:05.102537+0000","last_clean":"2023-07-06T03:56:05.102537+0000","last_became_active":"2023-07-06T03:56:05.099533+0000","last_became_peered":"2023-07-06T03:56:05.099533+0000","last_unstale":"2023-07-06T03:56:05.102537+0000","last_undegraded":"2023-07-06T03:56:05.102537+0000","last_fullsized":"2023-07-06T03:56:05.102537+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:42:14.854718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.085842+0000","last_change":"2023-07-06T03:56:04.087825+0000","last_active":"2023-07-06T03:56:05.085842+0000","last_peered":"2023-07-06T03:56:05.085842+0000","last_clean":"2023-07-06T03:56:05.085842+0000","last_became_active":"2023-07-06T03:56:04.087552+0000","last_became_peered":"2023-07-06T03:56:04.087552+0000","last_unstale":"2023-07-06T03:56:05.085842+0000","last_undegraded":"2023-07-06T03:56:05.085842+0000","last_fullsized":"2023-07-06T03:56:05.085842+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:16.895520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.110214+0000","last_change":"2023-07-06T03:56:05.099745+0000","last_active":"2023-07-06T03:56:05.110214+0000","last_peered":"2023-07-06T03:56:05.110214+0000","last_clean":"2023-07-06T03:56:05.110214+0000","last_became_active":"2023-07-06T03:56:05.099135+0000","last_became_peered":"2023-07-06T03:56:05.099135+0000","last_unstale":"2023-07-06T03:56:05.110214+0000","last_undegraded":"2023-07-06T03:56:05.110214+0000","last_fullsized":"2023-07-06T03:56:05.110214+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:09:07.509941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.078884+0000","last_change":"2023-07-06T03:56:04.089104+0000","last_active":"2023-07-06T03:56:05.078884+0000","last_peered":"2023-07-06T03:56:05.078884+0000","last_clean":"2023-07-06T03:56:05.078884+0000","last_became_active":"2023-07-06T03:56:04.088843+0000","last_became_peered":"2023-07-06T03:56:04.088843+0000","last_unstale":"2023-07-06T03:56:05.078884+0000","last_undegraded":"2023-07-06T03:56:05.078884+0000","last_fullsized":"2023-07-06T03:56:05.078884+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:35:38.087268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109629+0000","last_change":"2023-07-06T03:56:05.102989+0000","last_active":"2023-07-06T03:56:05.109629+0000","last_peered":"2023-07-06T03:56:05.109629+0000","last_clean":"2023-07-06T03:56:05.109629+0000","last_became_active":"2023-07-06T03:56:05.102713+0000","last_became_peered":"2023-07-06T03:56:05.102713+0000","last_unstale":"2023-07-06T03:56:05.109629+0000","last_undegraded":"2023-07-06T03:56:05.109629+0000","last_fullsized":"2023-07-06T03:56:05.109629+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:03:06.630621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.111591+0000","last_change":"2023-07-06T03:56:04.095792+0000","last_active":"2023-07-06T03:56:05.111591+0000","last_peered":"2023-07-06T03:56:05.111591+0000","last_clean":"2023-07-06T03:56:05.111591+0000","last_became_active":"2023-07-06T03:56:04.095378+0000","last_became_peered":"2023-07-06T03:56:04.095378+0000","last_unstale":"2023-07-06T03:56:05.111591+0000","last_undegraded":"2023-07-06T03:56:05.111591+0000","last_fullsized":"2023-07-06T03:56:05.111591+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:11.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":214,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.081952+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:56:05.081952+0000","last_peered":"2023-07-06T03:56:05.081952+0000","last_clean":"2023-07-06T03:56:05.081952+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:56:05.081952+0000","last_undegraded":"2023-07-06T03:56:05.081952+0000","last_fullsized":"2023-07-06T03:56:05.081952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:01:42.174318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.094719+0000","last_change":"2023-07-06T03:56:05.092821+0000","last_active":"2023-07-06T03:56:05.094719+0000","last_peered":"2023-07-06T03:56:05.094719+0000","last_clean":"2023-07-06T03:56:05.094719+0000","last_became_active":"2023-07-06T03:56:05.092439+0000","last_became_peered":"2023-07-06T03:56:05.092439+0000","last_unstale":"2023-07-06T03:56:05.094719+0000","last_undegraded":"2023-07-06T03:56:05.094719+0000","last_fullsized":"2023-07-06T03:56:05.094719+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:59:57.954018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"135'3","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.111914+0000","last_change":"2023-07-06T03:56:04.091569+0000","last_active":"2023-07-06T03:56:05.111914+0000","last_peered":"2023-07-06T03:56:05.111914+0000","last_clean":"2023-07-06T03:56:05.111914+0000","last_became_active":"2023-07-06T03:56:04.091106+0000","last_became_peered":"2023-07-06T03:56:04.091106+0000","last_unstale":"2023-07-06T03:56:05.111914+0000","last_undegraded":"2023-07-06T03:56:05.111914+0000","last_fullsized":"2023-07-06T03:56:05.111914+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:09:31.180856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":224,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.084958+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:56:05.084958+0000","last_peered":"2023-07-06T03:56:05.084958+0000","last_clean":"2023-07-06T03:56:05.084958+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:56:05.084958+0000","last_undegraded":"2023-07-06T03:56:05.084958+0000","last_fullsized":"2023-07-06T03:56:05.084958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:01:49.333338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103539+0000","last_change":"2023-07-06T03:56:05.101894+0000","last_active":"2023-07-06T03:56:05.103539+0000","last_peered":"2023-07-06T03:56:05.103539+0000","last_clean":"2023-07-06T03:56:05.103539+0000","last_became_active":"2023-07-06T03:56:05.101633+0000","last_became_peered":"2023-07-06T03:56:05.101633+0000","last_unstale":"2023-07-06T03:56:05.103539+0000","last_undegraded":"2023-07-06T03:56:05.103539+0000","last_fullsized":"2023-07-06T03:56:05.103539+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:03:05.882071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.079658+0000","last_change":"2023-07-06T03:56:04.093046+0000","last_active":"2023-07-06T03:56:05.079658+0000","last_peered":"2023-07-06T03:56:05.079658+0000","last_clean":"2023-07-06T03:56:05.079658+0000","last_became_active":"2023-07-06T03:56:04.091847+0000","last_became_peered":"2023-07-06T03:56:04.091847+0000","last_unstale":"2023-07-06T03:56:05.079658+0000","last_undegraded":"2023-07-06T03:56:05.079658+0000","last_fullsized":"2023-07-06T03:56:05.079658+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:47:06.931215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":230,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.080791+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:56:05.080791+0000","last_peered":"2023-07-06T03:56:05.080791+0000","last_clean":"2023-07-06T03:56:05.080791+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:56:05.080791+0000","last_undegraded":"2023-07-06T03:56:05.080791+0000","last_fullsized":"2023-07-06T03:56:05.080791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:27:35.438765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109769+0000","last_change":"2023-07-06T03:56:05.099617+0000","last_active":"2023-07-06T03:56:05.109769+0000","last_peered":"2023-07-06T03:56:05.109769+0000","last_clean":"2023-07-06T03:56:05.109769+0000","last_became_active":"2023-07-06T03:56:05.099259+0000","last_became_peered":"2023-07-06T03:56:05.099259+0000","last_unstale":"2023-07-06T03:56:05.109769+0000","last_undegraded":"2023-07-06T03:56:05.109769+0000","last_fullsized":"2023-07-06T03:56:05.109769+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:49.370060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.14","version":"135'6","reported_seq":17,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.124460+0000","last_change":"2023-07-06T03:56:04.092768+0000","last_active":"2023-07-06T03:56:05.124460+0000","last_peered":"2023-07-06T03:56:05.124460+0000","last_clean":"2023-07-06T03:56:05.124460+0000","last_became_active":"2023-07-06T03:56:04.092270+0000","last_became_peered":"2023-07-06T03:56:04.092270+0000","last_unstale":"2023-07-06T03:56:05.124460+0000","last_undegraded":"2023-07-06T03:56:05.124460+0000","last_fullsized":"2023-07-06T03:56:05.124460+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:51:13.301731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":224,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.086893+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:56:05.086893+0000","last_peered":"2023-07-06T03:56:05.086893+0000","last_clean":"2023-07-06T03:56:05.086893+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:56:05.086893+0000","last_undegraded":"2023-07-06T03:56:05.086893+0000","last_fullsized":"2023-07-06T03:56:05.086893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:00:36.125441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":364,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.083945+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:56:05.083945+0000","last_peered":"2023-07-06T03:56:05.083945+0000","last_clean":"2023-07-06T03:56:05.083945+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:56:05.083945+0000","last_undegraded":"2023-07-06T03:56:05.083945+0000","last_fullsized":"2023-07-06T03:56:05.083945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:42:04.322836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109821+0000","last_change":"2023-07-06T03:56:05.105251+0000","last_active":"2023-07-06T03:56:05.109821+0000","last_peered":"2023-07-06T03:56:05.109821+0000","last_clean":"2023-07-06T03:56:05.109821+0000","last_became_active":"2023-07-06T03:56:05.104976+0000","last_became_peered":"2023-07-06T03:56:05.104976+0000","last_unstale":"2023-07-06T03:56:05.109821+0000","last_undegraded":"2023-07-06T03:56:05.109821+0000","last_fullsized":"2023-07-06T03:56:05.109821+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:13:57.430942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.083949+0000","last_change":"2023-07-06T03:56:04.091701+0000","last_active":"2023-07-06T03:56:05.083949+0000","last_peered":"2023-07-06T03:56:05.083949+0000","last_clean":"2023-07-06T03:56:05.083949+0000","last_became_active":"2023-07-06T03:56:04.091437+0000","last_became_peered":"2023-07-06T03:56:04.091437+0000","last_unstale":"2023-07-06T03:56:05.083949+0000","last_undegraded":"2023-07-06T03:56:05.083949+0000","last_fullsized":"2023-07-06T03:56:05.083949+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:07:31.900656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":224,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.084109+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:56:05.084109+0000","last_peered":"2023-07-06T03:56:05.084109+0000","last_clean":"2023-07-06T03:56:05.084109+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:56:05.084109+0000","last_undegraded":"2023-07-06T03:56:05.084109+0000","last_fullsized":"2023-07-06T03:56:05.084109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:32:06.252387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.100322+0000","last_change":"2023-07-06T03:56:05.098013+0000","last_active":"2023-07-06T03:56:05.100322+0000","last_peered":"2023-07-06T03:56:05.100322+0000","last_clean":"2023-07-06T03:56:05.100322+0000","last_became_active":"2023-07-06T03:56:05.097722+0000","last_became_peered":"2023-07-06T03:56:05.097722+0000","last_unstale":"2023-07-06T03:56:05.100322+0000","last_undegraded":"2023-07-06T03:56:05.100322+0000","last_fullsized":"2023-07-06T03:56:05.100322+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:22:37.446960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.078878+0000","last_change":"2023-07-06T03:56:04.092797+0000","last_active":"2023-07-06T03:56:05.078878+0000","last_peered":"2023-07-06T03:56:05.078878+0000","last_clean":"2023-07-06T03:56:05.078878+0000","last_became_active":"2023-07-06T03:56:04.092532+0000","last_became_peered":"2023-07-06T03:56:04.092532+0000","last_unstale":"2023-07-06T03:56:05.078878+0000","last_undegraded":"2023-07-06T03:56:05.078878+0000","last_fullsized":"2023-07-06T03:56:05.078878+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:29:35.935289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":224,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.087502+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:56:05.087502+0000","last_peered":"2023-07-06T03:56:05.087502+0000","last_clean":"2023-07-06T03:56:05.087502+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:56:05.087502+0000","last_undegraded":"2023-07-06T03:56:05.087502+0000","last_fullsized":"2023-07-06T03:56:05.087502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:28:41.530100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.094846+0000","last_change":"2023-07-06T03:56:05.091933+0000","last_active":"2023-07-06T03:56:05.094846+0000","last_peered":"2023-07-06T03:56:05.094846+0000","last_clean":"2023-07-06T03:56:05.094846+0000","last_became_active":"2023-07-06T03:56:05.091615+0000","last_became_peered":"2023-07-06T03:56:05.091615+0000","last_unstale":"2023-07-06T03:56:05.094846+0000","last_undegraded":"2023-07-06T03:56:05.094846+0000","last_fullsized":"2023-07-06T03:56:05.094846+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:58:01.135846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.078705+0000","last_change":"2023-07-06T03:56:04.089470+0000","last_active":"2023-07-06T03:56:05.078705+0000","last_peered":"2023-07-06T03:56:05.078705+0000","last_clean":"2023-07-06T03:56:05.078705+0000","last_became_active":"2023-07-06T03:56:04.089213+0000","last_became_peered":"2023-07-06T03:56:04.089213+0000","last_unstale":"2023-07-06T03:56:05.078705+0000","last_undegraded":"2023-07-06T03:56:05.078705+0000","last_fullsized":"2023-07-06T03:56:05.078705+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:43.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":218,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.084404+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:56:05.084404+0000","last_peered":"2023-07-06T03:56:05.084404+0000","last_clean":"2023-07-06T03:56:05.084404+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:56:05.084404+0000","last_undegraded":"2023-07-06T03:56:05.084404+0000","last_fullsized":"2023-07-06T03:56:05.084404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:53:31.183596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.096941+0000","last_change":"2023-07-06T03:56:05.093444+0000","last_active":"2023-07-06T03:56:05.096941+0000","last_peered":"2023-07-06T03:56:05.096941+0000","last_clean":"2023-07-06T03:56:05.096941+0000","last_became_active":"2023-07-06T03:56:05.093034+0000","last_became_peered":"2023-07-06T03:56:05.093034+0000","last_unstale":"2023-07-06T03:56:05.096941+0000","last_undegraded":"2023-07-06T03:56:05.096941+0000","last_fullsized":"2023-07-06T03:56:05.096941+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:58.939632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.10","version":"135'8","reported_seq":19,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.122657+0000","last_change":"2023-07-06T03:56:04.090878+0000","last_active":"2023-07-06T03:56:05.122657+0000","last_peered":"2023-07-06T03:56:05.122657+0000","last_clean":"2023-07-06T03:56:05.122657+0000","last_became_active":"2023-07-06T03:56:04.090528+0000","last_became_peered":"2023-07-06T03:56:04.090528+0000","last_unstale":"2023-07-06T03:56:05.122657+0000","last_undegraded":"2023-07-06T03:56:05.122657+0000","last_fullsized":"2023-07-06T03:56:05.122657+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:46:59.004584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":224,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.085899+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:56:05.085899+0000","last_peered":"2023-07-06T03:56:05.085899+0000","last_clean":"2023-07-06T03:56:05.085899+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:56:05.085899+0000","last_undegraded":"2023-07-06T03:56:05.085899+0000","last_fullsized":"2023-07-06T03:56:05.085899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:50:56.461158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574985,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1699840,"data_stored":778472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.85299999999999998,"5min":1.024,"15min":4.0570000000000004},"last":0.625},{"interface":"front","average":{"1min":0.62,"5min":0.67500000000000004,"15min":0.73699999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.48599999999999999,"5min":0.45400000000000001,"15min":0.34000000000000002},"max":{"1min":0.90400000000000003,"5min":1.091,"15min":2.6219999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.68300000000000005,"15min":0.80300000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.38800000000000001},"max":{"1min":0.83699999999999997,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68400000000000005,"15min":0.747},"min":{"1min":0.47899999999999998,"5min":0.39700000000000002,"15min":0.35199999999999998},"max":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":3.77},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.45000000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.98899999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.47799999999999998,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":0.875,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.752,"15min":0.751},"min":{"1min":0.625,"5min":0.54500000000000004,"15min":0.43099999999999999},"max":{"1min":0.91000000000000003,"5min":0.98699999999999999,"15min":1.0209999999999999},"last":0.71399999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.90200000000000002,"15min":0.94699999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68700000000000006,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.40400000000000003,"15min":0.374},"max":{"1min":0.90500000000000003,"5min":0.94399999999999995,"15min":1.4590000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.54200000000000004,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.97099999999999997,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69699999999999995,"15min":0.68500000000000005},"min":{"1min":0.53500000000000003,"5min":0.45300000000000001,"15min":0.40000000000000002},"max":{"1min":0.93500000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.59099999999999997,"5min":0.46400000000000002,"15min":0.438},"max":{"1min":0.89600000000000002,"5min":1.0089999999999999,"15min":1.5349999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.748,"15min":0.77500000000000002},"min":{"1min":0.623,"5min":0.49199999999999999,"15min":0.42299999999999999},"max":{"1min":1.046,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.80400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36800,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335040,"statfs":{"total":96636764160,"available":96599080960,"internally_reserved":0,"allocated":1695744,"data_stored":778430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.76400000000000001,"15min":0.68999999999999995},"min":{"1min":0.39700000000000002,"5min":0.32800000000000001,"15min":0.29399999999999998},"max":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.76400000000000001,"15min":0.748},"min":{"1min":0.52200000000000002,"5min":0.44700000000000001,"15min":0.26900000000000002},"max":{"1min":3.3119999999999998,"5min":3.3119999999999998,"15min":3.3399999999999999},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.73599999999999999,"15min":0.68799999999999994},"min":{"1min":0.45500000000000002,"5min":0.372,"15min":0.34599999999999997},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.755,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":5.96,"5min":5.96,"15min":5.96},"last":0.76300000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.79800000000000004,"15min":0.71099999999999997},"min":{"1min":0.504,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":8.3740000000000006,"5min":8.3740000000000006,"15min":8.3740000000000006},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.79400000000000004,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.47699999999999998,"15min":0.38700000000000001},"max":{"1min":5.8449999999999998,"5min":5.8449999999999998,"15min":5.8449999999999998},"last":0.81799999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.48699999999999999,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.109,"5min":0.82999999999999996,"15min":0.75900000000000001},"min":{"1min":0.55200000000000005,"5min":0.51000000000000001,"15min":0.373},"max":{"1min":8.5269999999999992,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.72399999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.33200000000000002},"max":{"1min":5.7759999999999998,"5min":5.7759999999999998,"15min":5.7759999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.69599999999999995,"15min":0.72099999999999997},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.89000000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.73799999999999999,"15min":0.71799999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.38600000000000001},"max":{"1min":8.4870000000000001,"5min":8.4870000000000001,"15min":8.4870000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":5.923,"5min":5.923,"15min":5.923},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.76000000000000001,"15min":0.79700000000000004},"min":{"1min":0.5,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.129,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":8.4429999999999996,"5min":8.4429999999999996,"15min":8.4429999999999996},"last":0.78200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574985,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37380,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334460,"statfs":{"total":96636764160,"available":96598487040,"internally_reserved":0,"allocated":2289664,"data_stored":1368750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42699999999999999,"15min":0.496},"min":{"1min":0.29099999999999998,"5min":0.26900000000000002,"15min":0.23699999999999999},"max":{"1min":0.69499999999999995,"5min":0.72299999999999998,"15min":2.8879999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.52700000000000002,"15min":0.54900000000000004},"min":{"1min":0.34999999999999998,"5min":0.32300000000000001,"15min":0.313},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.46300000000000002,"15min":0.73899999999999999},"min":{"1min":0.28199999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.3690000000000002,"5min":3.3690000000000002,"15min":13.055999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.56299999999999994,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.752},{"interface":"front","average":{"1min":0.624,"5min":0.51500000000000001,"15min":0.52800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.29499999999999998},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.74199999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72499999999999998,"15min":1.002},"min":{"1min":0.442,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":12.989000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.55900000000000005,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":1.139,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.753},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.432},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71099999999999997,"15min":1.002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.494,"5min":1.494,"15min":13.023999999999999},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77800000000000002,"15min":0.79700000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.9670000000000001,"5min":2.9670000000000001,"15min":2.9670000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.77000000000000002,"15min":1.05},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47799999999999998},"max":{"1min":2.5409999999999999,"5min":2.5409999999999999,"15min":12.941000000000001},"last":0.88900000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.76600000000000001,"15min":0.80300000000000005},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.431,"5min":2.431,"15min":2.806},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.751,"15min":0.78900000000000003},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.85999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574986,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1699840,"data_stored":778464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.40899999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.25800000000000001},"max":{"1min":0.54900000000000004,"5min":0.753,"15min":0.753},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.441,"15min":0.437},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.58699999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.5}]},{"osd":2,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.46999999999999997},"min":{"1min":0.32100000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.56699999999999995,"5min":0.69199999999999995,"15min":0.73699999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.314},"max":{"1min":0.61499999999999999,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.32200000000000001,"15min":0.29899999999999999},"max":{"1min":0.627,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.38800000000000001},"max":{"1min":0.77600000000000002,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":1.1699999999999999,"15min":1.171},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.748,"15min":0.72399999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.76300000000000001,"15min":0.73599999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45600000000000002},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.46999999999999997},"max":{"1min":2.7549999999999999,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.51100000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.46200000000000002},"max":{"1min":0.96099999999999997,"5min":1.0649999999999999,"15min":1.546},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.628,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.623},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.78700000000000003,"15min":0.96299999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":9.5960000000000001},"last":0.76400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574985,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40892,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330948,"statfs":{"total":96636764160,"available":96594890752,"internally_reserved":0,"allocated":1691648,"data_stored":778360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.47299999999999998,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56299999999999994,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.30099999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58299999999999996,"5min":1.093,"15min":2.4630000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.54200000000000004,"15min":0.50700000000000001},"min":{"1min":0.33600000000000002,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.61899999999999999,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.57699999999999996,"5min":3.657,"15min":3.657},"last":0.441}]},{"osd":3,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.61699999999999999,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.42299999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65400000000000003,"5min":2.1629999999999998,"15min":2.7679999999999998},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.92000000000000004,"15min":0.78900000000000003},"min":{"1min":0.58099999999999996,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.103,"5min":12.007,"15min":12.007},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.79000000000000004,"15min":0.74299999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0389999999999999,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79500000000000004,"15min":0.754},"min":{"1min":0.58899999999999997,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":1.089,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.85799999999999998,"15min":0.79800000000000004},"min":{"1min":0.54700000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0680000000000001,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.92300000000000004,"15min":0.79300000000000004},"min":{"1min":0.50900000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.079,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.90900000000000003,"15min":0.81100000000000005},"min":{"1min":0.55000000000000004,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":1.0489999999999999,"5min":13.084,"15min":13.084},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81999999999999995,"15min":0.82299999999999995},"min":{"1min":0.60099999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.022,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.85299999999999998,"15min":0.81899999999999995},"min":{"1min":0.56599999999999995,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.573,"5min":4.484,"15min":4.484},"last":0.66400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574985,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1703936,"data_stored":779924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45000000000000001,"15min":0.54500000000000004},"min":{"1min":0.28499999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56699999999999995,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.59099999999999997,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.44700000000000001,"15min":0.81599999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57499999999999996,"5min":1.3620000000000001,"15min":15.911},"last":0.376},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.89300000000000002},"min":{"1min":0.38800000000000001,"5min":0.32100000000000001,"15min":0.29199999999999998},"max":{"1min":0.72599999999999998,"5min":1.347,"15min":16.024000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.49299999999999999,"15min":0.49199999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.60899999999999999,"5min":3.25,"15min":3.25},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.54300000000000004,"15min":0.89800000000000002},"min":{"1min":0.379,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80100000000000005,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73199999999999998,"15min":0.74299999999999999},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.99099999999999999,"5min":1.782,"15min":1.782},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.754,"15min":0.85899999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.36699999999999999},"max":{"1min":0.98499999999999999,"5min":1.724,"15min":5.7300000000000004},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.75800000000000001,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.49199999999999999,"15min":0.39700000000000002},"max":{"1min":1.036,"5min":1.411,"15min":5.077},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.79100000000000004,"15min":0.90700000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.44800000000000001},"max":{"1min":1.0680000000000001,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.76300000000000001,"15min":0.78000000000000003},"min":{"1min":0.52800000000000002,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.135,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.75900000000000001,"15min":0.86899999999999999},"min":{"1min":0.55300000000000005,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.0960000000000001,"5min":1.4490000000000001,"15min":5.157},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.76000000000000001,"15min":0.80100000000000005},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.47399999999999998},"max":{"1min":1.1639999999999999,"5min":1.863,"15min":1.863},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76300000000000001,"15min":0.88},"min":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.49399999999999999},"max":{"1min":0.97099999999999997,"5min":1.492,"15min":5.1189999999999998},"last":0.75}]}]},{"osd":4,"up_from":13,"seq":55834574984,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1699840,"data_stored":778976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.784},"last":0.755},{"interface":"front","average":{"1min":0.622,"5min":0.67400000000000004,"15min":0.64900000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.38},"max":{"1min":0.95199999999999996,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.503}]},{"osd":1,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.79000000000000004},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.38},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":6.258},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.806},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.80700000000000005},"min":{"1min":0.48599999999999999,"5min":0.42199999999999999,"15min":0.29199999999999998},"max":{"1min":1.012,"5min":1.012,"15min":6.4740000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.877},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.51900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":6.4859999999999998},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.68200000000000005},"min":{"1min":0.56599999999999995,"5min":0.45400000000000001,"15min":0.40000000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.25},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.84099999999999997,"5min":0.874,"15min":1.413},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61099999999999999,"15min":0.64100000000000001},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88300000000000001,"5min":0.89100000000000001,"15min":1.3280000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.441,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.84599999999999997,"5min":0.93000000000000005,"15min":0.99299999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.41899999999999998,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.91100000000000003,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.753,"15min":0.77600000000000002},"min":{"1min":0.57699999999999996,"5min":0.52400000000000002,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":0.96299999999999997,"15min":1.6839999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68700000000000006,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.41299999999999998},"max":{"1min":0.84399999999999997,"5min":0.89200000000000002,"15min":1.0660000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.51200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":1.036,"15min":1.21},"last":0.88200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574986,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":2301952,"data_stored":1369945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.91100000000000003,"15min":4.5010000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.67400000000000004,"15min":0.78500000000000003},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.83399999999999996,"5min":0.96199999999999997,"15min":6.4059999999999997},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.502,"5min":0.44800000000000001,"15min":0.307},"max":{"1min":0.76200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.46100000000000002,"15min":0.41399999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.546},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79300000000000004,"15min":0.72799999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.376},"max":{"1min":0.86499999999999999,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.79100000000000004,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.47599999999999998,"15min":0.42199999999999999},"max":{"1min":0.78100000000000003,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71399999999999997,"15min":0.73899999999999999},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":0.85299999999999998,"5min":1.3140000000000001,"15min":4.484},"last":0.72899999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":1.0229999999999999,"15min":1.6240000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64000000000000001,"15min":0.68700000000000006},"min":{"1min":0.41899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":2.718},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83899999999999997,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.70999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.82599999999999996,"5min":0.94099999999999995,"15min":1.669},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71499999999999997,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.93799999999999994,"5min":1.002,"15min":2.0459999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73699999999999999,"15min":0.77000000000000002},"min":{"1min":0.47199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88,"5min":1.0249999999999999,"15min":2.2919999999999998},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:56:09.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:56:09.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.727+0000 7fcc4d7ff700 1 Processor -- start 2023-07-06T03:56:09.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.728+0000 7fcc4d7ff700 1 -- start start 2023-07-06T03:56:09.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.729+0000 7fcc4d7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc4815c520 0x7fcc480cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:09.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.729+0000 7fcc4d7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc480cdae0 con 0x7fcc48160180 2023-07-06T03:56:09.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.729+0000 7fcc4d7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc480b6640 con 0x7fcc48160d00 2023-07-06T03:56:09.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.729+0000 7fcc4d7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc480cd8d0 con 0x7fcc4815c520 2023-07-06T03:56:09.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.729+0000 7fcc46ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc4815c520 0x7fcc480cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.729+0000 7fcc467fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcc48160180 0x7fcc481605f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46550/0 (socket says 172.21.15.97:46550) 2023-07-06T03:56:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.729+0000 7fcc467fc700 1 -- 172.21.15.97:0/2047495459 learned_addr learned my addr 172.21.15.97:0/2047495459 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.729+0000 7fcc46ffd700 1 -- 172.21.15.97:0/2047495459 >> v1:172.21.15.146:6789/0 conn(0x7fcc48160d00 legacy=0x7fcc480b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.729+0000 7fcc46ffd700 1 -- 172.21.15.97:0/2047495459 >> v1:172.21.15.97:6789/0 conn(0x7fcc48160180 legacy=0x7fcc481605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.729+0000 7fcc46ffd700 1 -- 172.21.15.97:0/2047495459 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc480b6920 con 0x7fcc4815c520 2023-07-06T03:56:09.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.729+0000 7fcc45ffb700 1 -- 172.21.15.97:0/2047495459 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398608574 0 0) 0x7fcc480cdae0 con 0x7fcc48160180 2023-07-06T03:56:09.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.730+0000 7fcc46ffd700 1 --2- 172.21.15.97:0/2047495459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4815c520 0x7fcc480cd390 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009840 tx=0x7fcc38009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce4d87174ec187 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:09.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.730+0000 7fcc45ffb700 1 -- 172.21.15.97:0/2047495459 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38004230 con 0x7fcc4815c520 2023-07-06T03:56:09.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.730+0000 7fcc45ffb700 1 -- 172.21.15.97:0/2047495459 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc38004390 con 0x7fcc4815c520 2023-07-06T03:56:09.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.730+0000 7fcc45ffb700 1 -- 172.21.15.97:0/2047495459 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc380036a0 con 0x7fcc4815c520 2023-07-06T03:56:09.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.730+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/2047495459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4815c520 msgr2=0x7fcc480cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.730+0000 7fcc4d7ff700 1 --2- 172.21.15.97:0/2047495459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4815c520 0x7fcc480cd390 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009840 tx=0x7fcc38009b50 comp rx=0 tx=0).stop 2023-07-06T03:56:09.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.730+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/2047495459 shutdown_connections 2023-07-06T03:56:09.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.730+0000 7fcc4d7ff700 1 --2- 172.21.15.97:0/2047495459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4815c520 0x7fcc480cd390 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.730+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/2047495459 >> 172.21.15.97:0/2047495459 conn(0x7fcc48071790 msgr2=0x7fcc48071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:09.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.731+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/2047495459 shutdown_connections 2023-07-06T03:56:09.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.731+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/2047495459 wait complete. 2023-07-06T03:56:09.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.733+0000 7fcc4d7ff700 1 Processor -- start 2023-07-06T03:56:09.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.733+0000 7fcc4d7ff700 1 -- start start 2023-07-06T03:56:09.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.734+0000 7fcc4d7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4815c520 0x7fcc480d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:09.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.734+0000 7fcc4d7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc48160180 0x7fcc48151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:09.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.734+0000 7fcc4d7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc48160d00 0x7fcc48153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:09.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.734+0000 7fcc4d7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc48154030 con 0x7fcc4815c520 2023-07-06T03:56:09.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.734+0000 7fcc4d7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc48153760 con 0x7fcc48160d00 2023-07-06T03:56:09.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.734+0000 7fcc4d7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc480d1b20 con 0x7fcc48160180 2023-07-06T03:56:09.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.735+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4815c520 0x7fcc480d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:09.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.735+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4815c520 0x7fcc480d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42400/0 (socket says 172.21.15.97:42400) 2023-07-06T03:56:09.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.735+0000 7fcc46ffd700 1 -- 172.21.15.97:0/1409785687 learned_addr learned my addr 172.21.15.97:0/1409785687 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:09.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.735+0000 7fcc467fc700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc48160180 0x7fcc48151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:09.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.735+0000 7fcc477fe700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc48160d00 0x7fcc48153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:09.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.735+0000 7fcc46ffd700 1 -- 172.21.15.97:0/1409785687 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc48160180 msgr2=0x7fcc48151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.735+0000 7fcc46ffd700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc48160180 0x7fcc48151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.735+0000 7fcc46ffd700 1 -- 172.21.15.97:0/1409785687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc48160d00 msgr2=0x7fcc48153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.735+0000 7fcc46ffd700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc48160d00 0x7fcc48153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.735+0000 7fcc46ffd700 1 -- 172.21.15.97:0/1409785687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc380094a0 con 0x7fcc4815c520 2023-07-06T03:56:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.735+0000 7fcc46ffd700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4815c520 0x7fcc480d1540 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fcc4806c240 tx=0x7fcc380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.736+0000 7fcc2f7fe700 1 -- 172.21.15.97:0/1409785687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38011630 con 0x7fcc4815c520 2023-07-06T03:56:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.736+0000 7fcc2f7fe700 1 -- 172.21.15.97:0/1409785687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc38011790 con 0x7fcc4815c520 2023-07-06T03:56:09.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.736+0000 7fcc2f7fe700 1 -- 172.21.15.97:0/1409785687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38011b40 con 0x7fcc4815c520 2023-07-06T03:56:09.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.736+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/1409785687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc480d1e00 con 0x7fcc4815c520 2023-07-06T03:56:09.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.736+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/1409785687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc480d23c0 con 0x7fcc4815c520 2023-07-06T03:56:09.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.737+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/1409785687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4804e340 con 0x7fcc4815c520 2023-07-06T03:56:09.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.738+0000 7fcc2f7fe700 1 -- 172.21.15.97:0/1409785687 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc38011ca0 con 0x7fcc4815c520 2023-07-06T03:56:09.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.739+0000 7fcc2f7fe700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc3406ace0 0x7fcc3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:09.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.739+0000 7fcc2f7fe700 1 -- 172.21.15.97:0/1409785687 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc3808a850 con 0x7fcc4815c520 2023-07-06T03:56:09.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.739+0000 7fcc467fc700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc3406ace0 0x7fcc3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:09.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.740+0000 7fcc467fc700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc3406ace0 0x7fcc3406d190 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fcc480d14d0 tx=0x7fcc3c007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:09.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.743+0000 7fcc2f7fe700 1 -- 172.21.15.97:0/1409785687 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcc38055980 con 0x7fcc4815c520 2023-07-06T03:56:09.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.934+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/1409785687 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc480b6ee0 con 0x7fcc3406ace0 2023-07-06T03:56:09.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.943+0000 7fcc2f7fe700 1 -- 172.21.15.97:0/1409785687 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311894 (secure 0 0 0) 0x7fcc3c001290 con 0x7fcc3406ace0 2023-07-06T03:56:09.949 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:09.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.947+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/1409785687 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc3406ace0 msgr2=0x7fcc3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.947+0000 7fcc4d7ff700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc3406ace0 0x7fcc3406d190 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fcc480d14d0 tx=0x7fcc3c007e70 comp rx=0 tx=0).stop 2023-07-06T03:56:09.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.947+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/1409785687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4815c520 msgr2=0x7fcc480d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:09.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.947+0000 7fcc4d7ff700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4815c520 0x7fcc480d1540 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fcc4806c240 tx=0x7fcc380047c0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.948+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/1409785687 shutdown_connections 2023-07-06T03:56:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.948+0000 7fcc4d7ff700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcc3406ace0 0x7fcc3406d190 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.948+0000 7fcc4d7ff700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc48160d00 0x7fcc48153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.948+0000 7fcc4d7ff700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4815c520 0x7fcc480d1540 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.948+0000 7fcc4d7ff700 1 --2- 172.21.15.97:0/1409785687 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcc48160180 0x7fcc48151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.948+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/1409785687 >> 172.21.15.97:0/1409785687 conn(0x7fcc48071790 msgr2=0x7fcc481605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:09.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.948+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/1409785687 shutdown_connections 2023-07-06T03:56:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:09.948+0000 7fcc4d7ff700 1 -- 172.21.15.97:0/1409785687 wait complete. 2023-07-06T03:56:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:56:09.968 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2023-07-06T03:56:09.024961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":291492,"kb_used_data":14436,"kb_used_omap":0,"kb_used_meta":276992,"kb_avail":754683228,"statfs":{"total":773094113280,"available":772795625472,"internally_reserved":0,"allocated":14782464,"data_stored":7411321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":283639808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.959979"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106091+0000","last_change":"2023-07-06T03:56:05.102566+0000","last_active":"2023-07-06T03:56:05.106091+0000","last_peered":"2023-07-06T03:56:05.106091+0000","last_clean":"2023-07-06T03:56:05.106091+0000","last_became_active":"2023-07-06T03:56:05.102312+0000","last_became_peered":"2023-07-06T03:56:05.102312+0000","last_unstale":"2023-07-06T03:56:05.106091+0000","last_undegraded":"2023-07-06T03:56:05.106091+0000","last_fullsized":"2023-07-06T03:56:05.106091+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:23:24.373621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.104555+0000","last_change":"2023-07-06T03:56:05.101011+0000","last_active":"2023-07-06T03:56:05.104555+0000","last_peered":"2023-07-06T03:56:05.104555+0000","last_clean":"2023-07-06T03:56:05.104555+0000","last_became_active":"2023-07-06T03:56:05.100690+0000","last_became_peered":"2023-07-06T03:56:05.100690+0000","last_unstale":"2023-07-06T03:56:05.104555+0000","last_undegraded":"2023-07-06T03:56:05.104555+0000","last_fullsized":"2023-07-06T03:56:05.104555+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:40:44.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.114050+0000","last_change":"2023-07-06T03:56:05.110365+0000","last_active":"2023-07-06T03:56:05.114050+0000","last_peered":"2023-07-06T03:56:05.114050+0000","last_clean":"2023-07-06T03:56:05.114050+0000","last_became_active":"2023-07-06T03:56:05.110123+0000","last_became_peered":"2023-07-06T03:56:05.110123+0000","last_unstale":"2023-07-06T03:56:05.114050+0000","last_undegraded":"2023-07-06T03:56:05.114050+0000","last_fullsized":"2023-07-06T03:56:05.114050+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:12.875713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.114977+0000","last_change":"2023-07-06T03:56:05.109510+0000","last_active":"2023-07-06T03:56:05.114977+0000","last_peered":"2023-07-06T03:56:05.114977+0000","last_clean":"2023-07-06T03:56:05.114977+0000","last_became_active":"2023-07-06T03:56:05.109215+0000","last_became_peered":"2023-07-06T03:56:05.109215+0000","last_unstale":"2023-07-06T03:56:05.114977+0000","last_undegraded":"2023-07-06T03:56:05.114977+0000","last_fullsized":"2023-07-06T03:56:05.114977+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:14:23.567839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.095034+0000","last_change":"2023-07-06T03:56:05.092060+0000","last_active":"2023-07-06T03:56:05.095034+0000","last_peered":"2023-07-06T03:56:05.095034+0000","last_clean":"2023-07-06T03:56:05.095034+0000","last_became_active":"2023-07-06T03:56:05.091762+0000","last_became_peered":"2023-07-06T03:56:05.091762+0000","last_unstale":"2023-07-06T03:56:05.095034+0000","last_undegraded":"2023-07-06T03:56:05.095034+0000","last_fullsized":"2023-07-06T03:56:05.095034+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:50:10.281115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.102559+0000","last_change":"2023-07-06T03:56:05.099340+0000","last_active":"2023-07-06T03:56:05.102559+0000","last_peered":"2023-07-06T03:56:05.102559+0000","last_clean":"2023-07-06T03:56:05.102559+0000","last_became_active":"2023-07-06T03:56:05.098815+0000","last_became_peered":"2023-07-06T03:56:05.098815+0000","last_unstale":"2023-07-06T03:56:05.102559+0000","last_undegraded":"2023-07-06T03:56:05.102559+0000","last_fullsized":"2023-07-06T03:56:05.102559+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:54:14.973450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106217+0000","last_change":"2023-07-06T03:56:05.100869+0000","last_active":"2023-07-06T03:56:05.106217+0000","last_peered":"2023-07-06T03:56:05.106217+0000","last_clean":"2023-07-06T03:56:05.106217+0000","last_became_active":"2023-07-06T03:56:05.100629+0000","last_became_peered":"2023-07-06T03:56:05.100629+0000","last_unstale":"2023-07-06T03:56:05.106217+0000","last_undegraded":"2023-07-06T03:56:05.106217+0000","last_fullsized":"2023-07-06T03:56:05.106217+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:27:55.441438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.107882+0000","last_change":"2023-07-06T03:56:05.095474+0000","last_active":"2023-07-06T03:56:05.107882+0000","last_peered":"2023-07-06T03:56:05.107882+0000","last_clean":"2023-07-06T03:56:05.107882+0000","last_became_active":"2023-07-06T03:56:05.095226+0000","last_became_peered":"2023-07-06T03:56:05.095226+0000","last_unstale":"2023-07-06T03:56:05.107882+0000","last_undegraded":"2023-07-06T03:56:05.107882+0000","last_fullsized":"2023-07-06T03:56:05.107882+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:18:27.163996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.104365+0000","last_change":"2023-07-06T03:56:05.101045+0000","last_active":"2023-07-06T03:56:05.104365+0000","last_peered":"2023-07-06T03:56:05.104365+0000","last_clean":"2023-07-06T03:56:05.104365+0000","last_became_active":"2023-07-06T03:56:05.100752+0000","last_became_peered":"2023-07-06T03:56:05.100752+0000","last_unstale":"2023-07-06T03:56:05.104365+0000","last_undegraded":"2023-07-06T03:56:05.104365+0000","last_fullsized":"2023-07-06T03:56:05.104365+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:29.971340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.097318+0000","last_change":"2023-07-06T03:56:05.094856+0000","last_active":"2023-07-06T03:56:05.097318+0000","last_peered":"2023-07-06T03:56:05.097318+0000","last_clean":"2023-07-06T03:56:05.097318+0000","last_became_active":"2023-07-06T03:56:05.094546+0000","last_became_peered":"2023-07-06T03:56:05.094546+0000","last_unstale":"2023-07-06T03:56:05.097318+0000","last_undegraded":"2023-07-06T03:56:05.097318+0000","last_fullsized":"2023-07-06T03:56:05.097318+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:52:56.136942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.098383+0000","last_change":"2023-07-06T03:56:05.088677+0000","last_active":"2023-07-06T03:56:05.098383+0000","last_peered":"2023-07-06T03:56:05.098383+0000","last_clean":"2023-07-06T03:56:05.098383+0000","last_became_active":"2023-07-06T03:56:05.088380+0000","last_became_peered":"2023-07-06T03:56:05.088380+0000","last_unstale":"2023-07-06T03:56:05.098383+0000","last_undegraded":"2023-07-06T03:56:05.098383+0000","last_fullsized":"2023-07-06T03:56:05.098383+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:38:39.597004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105716+0000","last_change":"2023-07-06T03:56:05.100990+0000","last_active":"2023-07-06T03:56:05.105716+0000","last_peered":"2023-07-06T03:56:05.105716+0000","last_clean":"2023-07-06T03:56:05.105716+0000","last_became_active":"2023-07-06T03:56:05.100738+0000","last_became_peered":"2023-07-06T03:56:05.100738+0000","last_unstale":"2023-07-06T03:56:05.105716+0000","last_undegraded":"2023-07-06T03:56:05.105716+0000","last_fullsized":"2023-07-06T03:56:05.105716+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:03:49.915504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.097813+0000","last_change":"2023-07-06T03:56:05.094945+0000","last_active":"2023-07-06T03:56:05.097813+0000","last_peered":"2023-07-06T03:56:05.097813+0000","last_clean":"2023-07-06T03:56:05.097813+0000","last_became_active":"2023-07-06T03:56:05.094372+0000","last_became_peered":"2023-07-06T03:56:05.094372+0000","last_unstale":"2023-07-06T03:56:05.097813+0000","last_undegraded":"2023-07-06T03:56:05.097813+0000","last_fullsized":"2023-07-06T03:56:05.097813+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:16:48.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106483+0000","last_change":"2023-07-06T03:56:05.092638+0000","last_active":"2023-07-06T03:56:05.106483+0000","last_peered":"2023-07-06T03:56:05.106483+0000","last_clean":"2023-07-06T03:56:05.106483+0000","last_became_active":"2023-07-06T03:56:05.091776+0000","last_became_peered":"2023-07-06T03:56:05.091776+0000","last_unstale":"2023-07-06T03:56:05.106483+0000","last_undegraded":"2023-07-06T03:56:05.106483+0000","last_fullsized":"2023-07-06T03:56:05.106483+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:36.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.099942+0000","last_change":"2023-07-06T03:56:05.098266+0000","last_active":"2023-07-06T03:56:05.099942+0000","last_peered":"2023-07-06T03:56:05.099942+0000","last_clean":"2023-07-06T03:56:05.099942+0000","last_became_active":"2023-07-06T03:56:05.097946+0000","last_became_peered":"2023-07-06T03:56:05.097946+0000","last_unstale":"2023-07-06T03:56:05.099942+0000","last_undegraded":"2023-07-06T03:56:05.099942+0000","last_fullsized":"2023-07-06T03:56:05.099942+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:40.999860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.099069+0000","last_change":"2023-07-06T03:56:05.094602+0000","last_active":"2023-07-06T03:56:05.099069+0000","last_peered":"2023-07-06T03:56:05.099069+0000","last_clean":"2023-07-06T03:56:05.099069+0000","last_became_active":"2023-07-06T03:56:05.094238+0000","last_became_peered":"2023-07-06T03:56:05.094238+0000","last_unstale":"2023-07-06T03:56:05.099069+0000","last_undegraded":"2023-07-06T03:56:05.099069+0000","last_fullsized":"2023-07-06T03:56:05.099069+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:08.743741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105730+0000","last_change":"2023-07-06T03:56:05.101635+0000","last_active":"2023-07-06T03:56:05.105730+0000","last_peered":"2023-07-06T03:56:05.105730+0000","last_clean":"2023-07-06T03:56:05.105730+0000","last_became_active":"2023-07-06T03:56:05.101388+0000","last_became_peered":"2023-07-06T03:56:05.101388+0000","last_unstale":"2023-07-06T03:56:05.105730+0000","last_undegraded":"2023-07-06T03:56:05.105730+0000","last_fullsized":"2023-07-06T03:56:05.105730+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:53.524456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.104536+0000","last_change":"2023-07-06T03:56:05.099850+0000","last_active":"2023-07-06T03:56:05.104536+0000","last_peered":"2023-07-06T03:56:05.104536+0000","last_clean":"2023-07-06T03:56:05.104536+0000","last_became_active":"2023-07-06T03:56:05.099517+0000","last_became_peered":"2023-07-06T03:56:05.099517+0000","last_unstale":"2023-07-06T03:56:05.104536+0000","last_undegraded":"2023-07-06T03:56:05.104536+0000","last_fullsized":"2023-07-06T03:56:05.104536+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:38:39.597004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105976+0000","last_change":"2023-07-06T03:56:05.101527+0000","last_active":"2023-07-06T03:56:05.105976+0000","last_peered":"2023-07-06T03:56:05.105976+0000","last_clean":"2023-07-06T03:56:05.105976+0000","last_became_active":"2023-07-06T03:56:05.101274+0000","last_became_peered":"2023-07-06T03:56:05.101274+0000","last_unstale":"2023-07-06T03:56:05.105976+0000","last_undegraded":"2023-07-06T03:56:05.105976+0000","last_fullsized":"2023-07-06T03:56:05.105976+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:29:53.326549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.092944+0000","last_change":"2023-07-06T03:56:05.087765+0000","last_active":"2023-07-06T03:56:05.092944+0000","last_peered":"2023-07-06T03:56:05.092944+0000","last_clean":"2023-07-06T03:56:05.092944+0000","last_became_active":"2023-07-06T03:56:05.087368+0000","last_became_peered":"2023-07-06T03:56:05.087368+0000","last_unstale":"2023-07-06T03:56:05.092944+0000","last_undegraded":"2023-07-06T03:56:05.092944+0000","last_fullsized":"2023-07-06T03:56:05.092944+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:59:57.954018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103511+0000","last_change":"2023-07-06T03:56:05.102189+0000","last_active":"2023-07-06T03:56:05.103511+0000","last_peered":"2023-07-06T03:56:05.103511+0000","last_clean":"2023-07-06T03:56:05.103511+0000","last_became_active":"2023-07-06T03:56:05.101927+0000","last_became_peered":"2023-07-06T03:56:05.101927+0000","last_unstale":"2023-07-06T03:56:05.103511+0000","last_undegraded":"2023-07-06T03:56:05.103511+0000","last_fullsized":"2023-07-06T03:56:05.103511+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:59:09.558612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103750+0000","last_change":"2023-07-06T03:56:05.098329+0000","last_active":"2023-07-06T03:56:05.103750+0000","last_peered":"2023-07-06T03:56:05.103750+0000","last_clean":"2023-07-06T03:56:05.103750+0000","last_became_active":"2023-07-06T03:56:05.098019+0000","last_became_peered":"2023-07-06T03:56:05.098019+0000","last_unstale":"2023-07-06T03:56:05.103750+0000","last_undegraded":"2023-07-06T03:56:05.103750+0000","last_fullsized":"2023-07-06T03:56:05.103750+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:05:07.560883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109655+0000","last_change":"2023-07-06T03:56:05.102409+0000","last_active":"2023-07-06T03:56:05.109655+0000","last_peered":"2023-07-06T03:56:05.109655+0000","last_clean":"2023-07-06T03:56:05.109655+0000","last_became_active":"2023-07-06T03:56:05.102093+0000","last_became_peered":"2023-07-06T03:56:05.102093+0000","last_unstale":"2023-07-06T03:56:05.109655+0000","last_undegraded":"2023-07-06T03:56:05.109655+0000","last_fullsized":"2023-07-06T03:56:05.109655+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:48:28.963453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.097489+0000","last_change":"2023-07-06T03:56:05.095474+0000","last_active":"2023-07-06T03:56:05.097489+0000","last_peered":"2023-07-06T03:56:05.097489+0000","last_clean":"2023-07-06T03:56:05.097489+0000","last_became_active":"2023-07-06T03:56:05.095200+0000","last_became_peered":"2023-07-06T03:56:05.095200+0000","last_unstale":"2023-07-06T03:56:05.097489+0000","last_undegraded":"2023-07-06T03:56:05.097489+0000","last_fullsized":"2023-07-06T03:56:05.097489+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:34:34.744575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106075+0000","last_change":"2023-07-06T03:56:05.101234+0000","last_active":"2023-07-06T03:56:05.106075+0000","last_peered":"2023-07-06T03:56:05.106075+0000","last_clean":"2023-07-06T03:56:05.106075+0000","last_became_active":"2023-07-06T03:56:05.100984+0000","last_became_peered":"2023-07-06T03:56:05.100984+0000","last_unstale":"2023-07-06T03:56:05.106075+0000","last_undegraded":"2023-07-06T03:56:05.106075+0000","last_fullsized":"2023-07-06T03:56:05.106075+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:59:05.963284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.112087+0000","last_change":"2023-07-06T03:56:05.107186+0000","last_active":"2023-07-06T03:56:05.112087+0000","last_peered":"2023-07-06T03:56:05.112087+0000","last_clean":"2023-07-06T03:56:05.112087+0000","last_became_active":"2023-07-06T03:56:05.106939+0000","last_became_peered":"2023-07-06T03:56:05.106939+0000","last_unstale":"2023-07-06T03:56:05.112087+0000","last_undegraded":"2023-07-06T03:56:05.112087+0000","last_fullsized":"2023-07-06T03:56:05.112087+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:36:38.071390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103769+0000","last_change":"2023-07-06T03:56:05.101832+0000","last_active":"2023-07-06T03:56:05.103769+0000","last_peered":"2023-07-06T03:56:05.103769+0000","last_clean":"2023-07-06T03:56:05.103769+0000","last_became_active":"2023-07-06T03:56:05.101533+0000","last_became_peered":"2023-07-06T03:56:05.101533+0000","last_unstale":"2023-07-06T03:56:05.103769+0000","last_undegraded":"2023-07-06T03:56:05.103769+0000","last_fullsized":"2023-07-06T03:56:05.103769+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:26:56.657153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.094726+0000","last_change":"2023-07-06T03:56:05.090513+0000","last_active":"2023-07-06T03:56:05.094726+0000","last_peered":"2023-07-06T03:56:05.094726+0000","last_clean":"2023-07-06T03:56:05.094726+0000","last_became_active":"2023-07-06T03:56:05.090184+0000","last_became_peered":"2023-07-06T03:56:05.090184+0000","last_unstale":"2023-07-06T03:56:05.094726+0000","last_undegraded":"2023-07-06T03:56:05.094726+0000","last_fullsized":"2023-07-06T03:56:05.094726+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:00.715722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.110243+0000","last_change":"2023-07-06T03:56:05.099956+0000","last_active":"2023-07-06T03:56:05.110243+0000","last_peered":"2023-07-06T03:56:05.110243+0000","last_clean":"2023-07-06T03:56:05.110243+0000","last_became_active":"2023-07-06T03:56:05.099440+0000","last_became_peered":"2023-07-06T03:56:05.099440+0000","last_unstale":"2023-07-06T03:56:05.110243+0000","last_undegraded":"2023-07-06T03:56:05.110243+0000","last_fullsized":"2023-07-06T03:56:05.110243+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:06:56.996144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103736+0000","last_change":"2023-07-06T03:56:05.101805+0000","last_active":"2023-07-06T03:56:05.103736+0000","last_peered":"2023-07-06T03:56:05.103736+0000","last_clean":"2023-07-06T03:56:05.103736+0000","last_became_active":"2023-07-06T03:56:05.101484+0000","last_became_peered":"2023-07-06T03:56:05.101484+0000","last_unstale":"2023-07-06T03:56:05.103736+0000","last_undegraded":"2023-07-06T03:56:05.103736+0000","last_fullsized":"2023-07-06T03:56:05.103736+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:33:48.403671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.098978+0000","last_change":"2023-07-06T03:56:05.094534+0000","last_active":"2023-07-06T03:56:05.098978+0000","last_peered":"2023-07-06T03:56:05.098978+0000","last_clean":"2023-07-06T03:56:05.098978+0000","last_became_active":"2023-07-06T03:56:05.093962+0000","last_became_peered":"2023-07-06T03:56:05.093962+0000","last_unstale":"2023-07-06T03:56:05.098978+0000","last_undegraded":"2023-07-06T03:56:05.098978+0000","last_fullsized":"2023-07-06T03:56:05.098978+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:16:48.889088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105926+0000","last_change":"2023-07-06T03:56:05.092257+0000","last_active":"2023-07-06T03:56:05.105926+0000","last_peered":"2023-07-06T03:56:05.105926+0000","last_clean":"2023-07-06T03:56:05.105926+0000","last_became_active":"2023-07-06T03:56:05.091600+0000","last_became_peered":"2023-07-06T03:56:05.091600+0000","last_unstale":"2023-07-06T03:56:05.105926+0000","last_undegraded":"2023-07-06T03:56:05.105926+0000","last_fullsized":"2023-07-06T03:56:05.105926+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:40:10.012028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.098988+0000","last_change":"2023-07-06T03:56:05.095256+0000","last_active":"2023-07-06T03:56:05.098988+0000","last_peered":"2023-07-06T03:56:05.098988+0000","last_clean":"2023-07-06T03:56:05.098988+0000","last_became_active":"2023-07-06T03:56:05.094971+0000","last_became_peered":"2023-07-06T03:56:05.094971+0000","last_unstale":"2023-07-06T03:56:05.098988+0000","last_undegraded":"2023-07-06T03:56:05.098988+0000","last_fullsized":"2023-07-06T03:56:05.098988+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:04:00.715722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1f","version":"135'4","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.112060+0000","last_change":"2023-07-06T03:56:04.090531+0000","last_active":"2023-07-06T03:56:05.112060+0000","last_peered":"2023-07-06T03:56:05.112060+0000","last_clean":"2023-07-06T03:56:05.112060+0000","last_became_active":"2023-07-06T03:56:04.090226+0000","last_became_peered":"2023-07-06T03:56:04.090226+0000","last_unstale":"2023-07-06T03:56:05.112060+0000","last_undegraded":"2023-07-06T03:56:05.112060+0000","last_fullsized":"2023-07-06T03:56:05.112060+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:13:28.963054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.100055+0000","last_change":"2023-07-06T03:56:05.096745+0000","last_active":"2023-07-06T03:56:05.100055+0000","last_peered":"2023-07-06T03:56:05.100055+0000","last_clean":"2023-07-06T03:56:05.100055+0000","last_became_active":"2023-07-06T03:56:05.096433+0000","last_became_peered":"2023-07-06T03:56:05.096433+0000","last_unstale":"2023-07-06T03:56:05.100055+0000","last_undegraded":"2023-07-06T03:56:05.100055+0000","last_fullsized":"2023-07-06T03:56:05.100055+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:03:05.882071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.113153+0000","last_change":"2023-07-06T03:56:04.088217+0000","last_active":"2023-07-06T03:56:05.113153+0000","last_peered":"2023-07-06T03:56:05.113153+0000","last_clean":"2023-07-06T03:56:05.113153+0000","last_became_active":"2023-07-06T03:56:04.087967+0000","last_became_peered":"2023-07-06T03:56:04.087967+0000","last_unstale":"2023-07-06T03:56:05.113153+0000","last_undegraded":"2023-07-06T03:56:05.113153+0000","last_fullsized":"2023-07-06T03:56:05.113153+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:59:40.517315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105704+0000","last_change":"2023-07-06T03:56:05.102191+0000","last_active":"2023-07-06T03:56:05.105704+0000","last_peered":"2023-07-06T03:56:05.105704+0000","last_clean":"2023-07-06T03:56:05.105704+0000","last_became_active":"2023-07-06T03:56:05.101955+0000","last_became_peered":"2023-07-06T03:56:05.101955+0000","last_unstale":"2023-07-06T03:56:05.105704+0000","last_undegraded":"2023-07-06T03:56:05.105704+0000","last_fullsized":"2023-07-06T03:56:05.105704+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:00:11.725190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1d","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.108145+0000","last_change":"2023-07-06T03:56:04.090809+0000","last_active":"2023-07-06T03:56:05.108145+0000","last_peered":"2023-07-06T03:56:05.108145+0000","last_clean":"2023-07-06T03:56:05.108145+0000","last_became_active":"2023-07-06T03:56:04.090507+0000","last_became_peered":"2023-07-06T03:56:04.090507+0000","last_unstale":"2023-07-06T03:56:05.108145+0000","last_undegraded":"2023-07-06T03:56:05.108145+0000","last_fullsized":"2023-07-06T03:56:05.108145+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:17:46.967407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.101033+0000","last_change":"2023-07-06T03:56:05.090898+0000","last_active":"2023-07-06T03:56:05.101033+0000","last_peered":"2023-07-06T03:56:05.101033+0000","last_clean":"2023-07-06T03:56:05.101033+0000","last_became_active":"2023-07-06T03:56:05.090638+0000","last_became_peered":"2023-07-06T03:56:05.090638+0000","last_unstale":"2023-07-06T03:56:05.101033+0000","last_undegraded":"2023-07-06T03:56:05.101033+0000","last_fullsized":"2023-07-06T03:56:05.101033+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:03:11.267195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1c","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.110647+0000","last_change":"2023-07-06T03:56:04.093434+0000","last_active":"2023-07-06T03:56:05.110647+0000","last_peered":"2023-07-06T03:56:05.110647+0000","last_clean":"2023-07-06T03:56:05.110647+0000","last_became_active":"2023-07-06T03:56:04.093126+0000","last_became_peered":"2023-07-06T03:56:04.093126+0000","last_unstale":"2023-07-06T03:56:05.110647+0000","last_undegraded":"2023-07-06T03:56:05.110647+0000","last_fullsized":"2023-07-06T03:56:05.110647+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:05:32.312327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.098368+0000","last_change":"2023-07-06T03:56:05.090648+0000","last_active":"2023-07-06T03:56:05.098368+0000","last_peered":"2023-07-06T03:56:05.098368+0000","last_clean":"2023-07-06T03:56:05.098368+0000","last_became_active":"2023-07-06T03:56:05.090392+0000","last_became_peered":"2023-07-06T03:56:05.090392+0000","last_unstale":"2023-07-06T03:56:05.098368+0000","last_undegraded":"2023-07-06T03:56:05.098368+0000","last_fullsized":"2023-07-06T03:56:05.098368+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:40:44.088146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1b","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.107799+0000","last_change":"2023-07-06T03:56:04.090894+0000","last_active":"2023-07-06T03:56:05.107799+0000","last_peered":"2023-07-06T03:56:05.107799+0000","last_clean":"2023-07-06T03:56:05.107799+0000","last_became_active":"2023-07-06T03:56:04.090607+0000","last_became_peered":"2023-07-06T03:56:04.090607+0000","last_unstale":"2023-07-06T03:56:05.107799+0000","last_undegraded":"2023-07-06T03:56:05.107799+0000","last_fullsized":"2023-07-06T03:56:05.107799+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:58:22.191968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.101416+0000","last_change":"2023-07-06T03:56:05.090478+0000","last_active":"2023-07-06T03:56:05.101416+0000","last_peered":"2023-07-06T03:56:05.101416+0000","last_clean":"2023-07-06T03:56:05.101416+0000","last_became_active":"2023-07-06T03:56:05.090192+0000","last_became_peered":"2023-07-06T03:56:05.090192+0000","last_unstale":"2023-07-06T03:56:05.101416+0000","last_undegraded":"2023-07-06T03:56:05.101416+0000","last_fullsized":"2023-07-06T03:56:05.101416+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:19:26.415133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109983+0000","last_change":"2023-07-06T03:56:04.089431+0000","last_active":"2023-07-06T03:56:05.109983+0000","last_peered":"2023-07-06T03:56:05.109983+0000","last_clean":"2023-07-06T03:56:05.109983+0000","last_became_active":"2023-07-06T03:56:04.089127+0000","last_became_peered":"2023-07-06T03:56:04.089127+0000","last_unstale":"2023-07-06T03:56:05.109983+0000","last_undegraded":"2023-07-06T03:56:05.109983+0000","last_fullsized":"2023-07-06T03:56:05.109983+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:34.275532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.111597+0000","last_change":"2023-07-06T03:56:05.107762+0000","last_active":"2023-07-06T03:56:05.111597+0000","last_peered":"2023-07-06T03:56:05.111597+0000","last_clean":"2023-07-06T03:56:05.111597+0000","last_became_active":"2023-07-06T03:56:05.107464+0000","last_became_peered":"2023-07-06T03:56:05.107464+0000","last_unstale":"2023-07-06T03:56:05.111597+0000","last_undegraded":"2023-07-06T03:56:05.111597+0000","last_fullsized":"2023-07-06T03:56:05.111597+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:43:14.304018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.078809+0000","last_change":"2023-07-06T03:56:04.089266+0000","last_active":"2023-07-06T03:56:05.078809+0000","last_peered":"2023-07-06T03:56:05.078809+0000","last_clean":"2023-07-06T03:56:05.078809+0000","last_became_active":"2023-07-06T03:56:04.089012+0000","last_became_peered":"2023-07-06T03:56:04.089012+0000","last_unstale":"2023-07-06T03:56:05.078809+0000","last_undegraded":"2023-07-06T03:56:05.078809+0000","last_fullsized":"2023-07-06T03:56:05.078809+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:51:23.674938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105861+0000","last_change":"2023-07-06T03:56:05.091804+0000","last_active":"2023-07-06T03:56:05.105861+0000","last_peered":"2023-07-06T03:56:05.105861+0000","last_clean":"2023-07-06T03:56:05.105861+0000","last_became_active":"2023-07-06T03:56:05.091527+0000","last_became_peered":"2023-07-06T03:56:05.091527+0000","last_unstale":"2023-07-06T03:56:05.105861+0000","last_undegraded":"2023-07-06T03:56:05.105861+0000","last_fullsized":"2023-07-06T03:56:05.105861+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:46:03.435316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.081889+0000","last_change":"2023-07-06T03:56:04.095698+0000","last_active":"2023-07-06T03:56:05.081889+0000","last_peered":"2023-07-06T03:56:05.081889+0000","last_clean":"2023-07-06T03:56:05.081889+0000","last_became_active":"2023-07-06T03:56:04.095342+0000","last_became_peered":"2023-07-06T03:56:04.095342+0000","last_unstale":"2023-07-06T03:56:05.081889+0000","last_undegraded":"2023-07-06T03:56:05.081889+0000","last_fullsized":"2023-07-06T03:56:05.081889+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:15:56.489035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106023+0000","last_change":"2023-07-06T03:56:05.101450+0000","last_active":"2023-07-06T03:56:05.106023+0000","last_peered":"2023-07-06T03:56:05.106023+0000","last_clean":"2023-07-06T03:56:05.106023+0000","last_became_active":"2023-07-06T03:56:05.101193+0000","last_became_peered":"2023-07-06T03:56:05.101193+0000","last_unstale":"2023-07-06T03:56:05.106023+0000","last_undegraded":"2023-07-06T03:56:05.106023+0000","last_fullsized":"2023-07-06T03:56:05.106023+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:41:34.959137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.f","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.110159+0000","last_change":"2023-07-06T03:56:04.088074+0000","last_active":"2023-07-06T03:56:05.110159+0000","last_peered":"2023-07-06T03:56:05.110159+0000","last_clean":"2023-07-06T03:56:05.110159+0000","last_became_active":"2023-07-06T03:56:04.087740+0000","last_became_peered":"2023-07-06T03:56:04.087740+0000","last_unstale":"2023-07-06T03:56:05.110159+0000","last_undegraded":"2023-07-06T03:56:05.110159+0000","last_fullsized":"2023-07-06T03:56:05.110159+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:46:07.735456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.108792+0000","last_change":"2023-07-06T03:56:05.097349+0000","last_active":"2023-07-06T03:56:05.108792+0000","last_peered":"2023-07-06T03:56:05.108792+0000","last_clean":"2023-07-06T03:56:05.108792+0000","last_became_active":"2023-07-06T03:56:05.096893+0000","last_became_peered":"2023-07-06T03:56:05.096893+0000","last_unstale":"2023-07-06T03:56:05.108792+0000","last_undegraded":"2023-07-06T03:56:05.108792+0000","last_fullsized":"2023-07-06T03:56:05.108792+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:43:39.208495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.e","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.112705+0000","last_change":"2023-07-06T03:56:04.095085+0000","last_active":"2023-07-06T03:56:05.112705+0000","last_peered":"2023-07-06T03:56:05.112705+0000","last_clean":"2023-07-06T03:56:05.112705+0000","last_became_active":"2023-07-06T03:56:04.094822+0000","last_became_peered":"2023-07-06T03:56:04.094822+0000","last_unstale":"2023-07-06T03:56:05.112705+0000","last_undegraded":"2023-07-06T03:56:05.112705+0000","last_fullsized":"2023-07-06T03:56:05.112705+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:19.124771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.105684+0000","last_change":"2023-07-06T03:56:05.101133+0000","last_active":"2023-07-06T03:56:05.105684+0000","last_peered":"2023-07-06T03:56:05.105684+0000","last_clean":"2023-07-06T03:56:05.105684+0000","last_became_active":"2023-07-06T03:56:05.100876+0000","last_became_peered":"2023-07-06T03:56:05.100876+0000","last_unstale":"2023-07-06T03:56:05.105684+0000","last_undegraded":"2023-07-06T03:56:05.105684+0000","last_fullsized":"2023-07-06T03:56:05.105684+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:14:20.392368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.d","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.112490+0000","last_change":"2023-07-06T03:56:04.093254+0000","last_active":"2023-07-06T03:56:05.112490+0000","last_peered":"2023-07-06T03:56:05.112490+0000","last_clean":"2023-07-06T03:56:05.112490+0000","last_became_active":"2023-07-06T03:56:04.092953+0000","last_became_peered":"2023-07-06T03:56:04.092953+0000","last_unstale":"2023-07-06T03:56:05.112490+0000","last_undegraded":"2023-07-06T03:56:05.112490+0000","last_fullsized":"2023-07-06T03:56:05.112490+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:58:03.548601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109789+0000","last_change":"2023-07-06T03:56:05.102505+0000","last_active":"2023-07-06T03:56:05.109789+0000","last_peered":"2023-07-06T03:56:05.109789+0000","last_clean":"2023-07-06T03:56:05.109789+0000","last_became_active":"2023-07-06T03:56:05.102259+0000","last_became_peered":"2023-07-06T03:56:05.102259+0000","last_unstale":"2023-07-06T03:56:05.109789+0000","last_undegraded":"2023-07-06T03:56:05.109789+0000","last_fullsized":"2023-07-06T03:56:05.109789+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:38:04.098842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.089921+0000","last_change":"2023-07-06T03:56:04.093165+0000","last_active":"2023-07-06T03:56:05.089921+0000","last_peered":"2023-07-06T03:56:05.089921+0000","last_clean":"2023-07-06T03:56:05.089921+0000","last_became_active":"2023-07-06T03:56:04.092884+0000","last_became_peered":"2023-07-06T03:56:04.092884+0000","last_unstale":"2023-07-06T03:56:05.089921+0000","last_undegraded":"2023-07-06T03:56:05.089921+0000","last_fullsized":"2023-07-06T03:56:05.089921+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:08:37.967648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.098617+0000","last_change":"2023-07-06T03:56:05.096458+0000","last_active":"2023-07-06T03:56:05.098617+0000","last_peered":"2023-07-06T03:56:05.098617+0000","last_clean":"2023-07-06T03:56:05.098617+0000","last_became_active":"2023-07-06T03:56:05.096197+0000","last_became_peered":"2023-07-06T03:56:05.096197+0000","last_unstale":"2023-07-06T03:56:05.098617+0000","last_undegraded":"2023-07-06T03:56:05.098617+0000","last_fullsized":"2023-07-06T03:56:05.098617+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:34:44.981824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.b","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.110841+0000","last_change":"2023-07-06T03:56:04.089565+0000","last_active":"2023-07-06T03:56:05.110841+0000","last_peered":"2023-07-06T03:56:05.110841+0000","last_clean":"2023-07-06T03:56:05.110841+0000","last_became_active":"2023-07-06T03:56:04.089271+0000","last_became_peered":"2023-07-06T03:56:04.089271+0000","last_unstale":"2023-07-06T03:56:05.110841+0000","last_undegraded":"2023-07-06T03:56:05.110841+0000","last_fullsized":"2023-07-06T03:56:05.110841+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:59:08.550327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.111567+0000","last_change":"2023-07-06T03:56:05.107541+0000","last_active":"2023-07-06T03:56:05.111567+0000","last_peered":"2023-07-06T03:56:05.111567+0000","last_clean":"2023-07-06T03:56:05.111567+0000","last_became_active":"2023-07-06T03:56:05.107228+0000","last_became_peered":"2023-07-06T03:56:05.107228+0000","last_unstale":"2023-07-06T03:56:05.111567+0000","last_undegraded":"2023-07-06T03:56:05.111567+0000","last_fullsized":"2023-07-06T03:56:05.111567+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:12:08.635554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.085847+0000","last_change":"2023-07-06T03:56:04.095177+0000","last_active":"2023-07-06T03:56:05.085847+0000","last_peered":"2023-07-06T03:56:05.085847+0000","last_clean":"2023-07-06T03:56:05.085847+0000","last_became_active":"2023-07-06T03:56:04.094886+0000","last_became_peered":"2023-07-06T03:56:04.094886+0000","last_unstale":"2023-07-06T03:56:05.085847+0000","last_undegraded":"2023-07-06T03:56:05.085847+0000","last_fullsized":"2023-07-06T03:56:05.085847+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:30:13.899596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103769+0000","last_change":"2023-07-06T03:56:05.098785+0000","last_active":"2023-07-06T03:56:05.103769+0000","last_peered":"2023-07-06T03:56:05.103769+0000","last_clean":"2023-07-06T03:56:05.103769+0000","last_became_active":"2023-07-06T03:56:05.098528+0000","last_became_peered":"2023-07-06T03:56:05.098528+0000","last_unstale":"2023-07-06T03:56:05.103769+0000","last_undegraded":"2023-07-06T03:56:05.103769+0000","last_fullsized":"2023-07-06T03:56:05.103769+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:25:52.346892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.080696+0000","last_change":"2023-07-06T03:56:04.085393+0000","last_active":"2023-07-06T03:56:05.080696+0000","last_peered":"2023-07-06T03:56:05.080696+0000","last_clean":"2023-07-06T03:56:05.080696+0000","last_became_active":"2023-07-06T03:56:04.085076+0000","last_became_peered":"2023-07-06T03:56:04.085076+0000","last_unstale":"2023-07-06T03:56:05.080696+0000","last_undegraded":"2023-07-06T03:56:05.080696+0000","last_fullsized":"2023-07-06T03:56:05.080696+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:21:58.582831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103454+0000","last_change":"2023-07-06T03:56:05.101013+0000","last_active":"2023-07-06T03:56:05.103454+0000","last_peered":"2023-07-06T03:56:05.103454+0000","last_clean":"2023-07-06T03:56:05.103454+0000","last_became_active":"2023-07-06T03:56:05.100735+0000","last_became_peered":"2023-07-06T03:56:05.100735+0000","last_unstale":"2023-07-06T03:56:05.103454+0000","last_undegraded":"2023-07-06T03:56:05.103454+0000","last_fullsized":"2023-07-06T03:56:05.103454+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:04:35.283817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.085702+0000","last_change":"2023-07-06T03:56:04.092939+0000","last_active":"2023-07-06T03:56:05.085702+0000","last_peered":"2023-07-06T03:56:05.085702+0000","last_clean":"2023-07-06T03:56:05.085702+0000","last_became_active":"2023-07-06T03:56:04.092588+0000","last_became_peered":"2023-07-06T03:56:04.092588+0000","last_unstale":"2023-07-06T03:56:05.085702+0000","last_undegraded":"2023-07-06T03:56:05.085702+0000","last_fullsized":"2023-07-06T03:56:05.085702+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:40:43.287184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103377+0000","last_change":"2023-07-06T03:56:05.100900+0000","last_active":"2023-07-06T03:56:05.103377+0000","last_peered":"2023-07-06T03:56:05.103377+0000","last_clean":"2023-07-06T03:56:05.103377+0000","last_became_active":"2023-07-06T03:56:05.100629+0000","last_became_peered":"2023-07-06T03:56:05.100629+0000","last_unstale":"2023-07-06T03:56:05.103377+0000","last_undegraded":"2023-07-06T03:56:05.103377+0000","last_fullsized":"2023-07-06T03:56:05.103377+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:58:01.135846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.7","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.122286+0000","last_change":"2023-07-06T03:56:04.090746+0000","last_active":"2023-07-06T03:56:05.122286+0000","last_peered":"2023-07-06T03:56:05.122286+0000","last_clean":"2023-07-06T03:56:05.122286+0000","last_became_active":"2023-07-06T03:56:04.090444+0000","last_became_peered":"2023-07-06T03:56:04.090444+0000","last_unstale":"2023-07-06T03:56:05.122286+0000","last_undegraded":"2023-07-06T03:56:05.122286+0000","last_fullsized":"2023-07-06T03:56:05.122286+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:12:39.160706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.107936+0000","last_change":"2023-07-06T03:56:05.094632+0000","last_active":"2023-07-06T03:56:05.107936+0000","last_peered":"2023-07-06T03:56:05.107936+0000","last_clean":"2023-07-06T03:56:05.107936+0000","last_became_active":"2023-07-06T03:56:05.093861+0000","last_became_peered":"2023-07-06T03:56:05.093861+0000","last_unstale":"2023-07-06T03:56:05.107936+0000","last_undegraded":"2023-07-06T03:56:05.107936+0000","last_fullsized":"2023-07-06T03:56:05.107936+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:14:38.102238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.079684+0000","last_change":"2023-07-06T03:56:04.090622+0000","last_active":"2023-07-06T03:56:05.079684+0000","last_peered":"2023-07-06T03:56:05.079684+0000","last_clean":"2023-07-06T03:56:05.079684+0000","last_became_active":"2023-07-06T03:56:04.090339+0000","last_became_peered":"2023-07-06T03:56:04.090339+0000","last_unstale":"2023-07-06T03:56:05.079684+0000","last_undegraded":"2023-07-06T03:56:05.079684+0000","last_fullsized":"2023-07-06T03:56:05.079684+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:16:38.519342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.122421+0000","last_change":"2023-07-06T03:56:05.112839+0000","last_active":"2023-07-06T03:56:05.122421+0000","last_peered":"2023-07-06T03:56:05.122421+0000","last_clean":"2023-07-06T03:56:05.122421+0000","last_became_active":"2023-07-06T03:56:05.112499+0000","last_became_peered":"2023-07-06T03:56:05.112499+0000","last_unstale":"2023-07-06T03:56:05.122421+0000","last_undegraded":"2023-07-06T03:56:05.122421+0000","last_fullsized":"2023-07-06T03:56:05.122421+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:25:44.026540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.084990+0000","last_change":"2023-07-06T03:56:04.090547+0000","last_active":"2023-07-06T03:56:05.084990+0000","last_peered":"2023-07-06T03:56:05.084990+0000","last_clean":"2023-07-06T03:56:05.084990+0000","last_became_active":"2023-07-06T03:56:04.090262+0000","last_became_peered":"2023-07-06T03:56:04.090262+0000","last_unstale":"2023-07-06T03:56:05.084990+0000","last_undegraded":"2023-07-06T03:56:05.084990+0000","last_fullsized":"2023-07-06T03:56:05.084990+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:17.973105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.097784+0000","last_change":"2023-07-06T03:56:05.094478+0000","last_active":"2023-07-06T03:56:05.097784+0000","last_peered":"2023-07-06T03:56:05.097784+0000","last_clean":"2023-07-06T03:56:05.097784+0000","last_became_active":"2023-07-06T03:56:05.094164+0000","last_became_peered":"2023-07-06T03:56:05.094164+0000","last_unstale":"2023-07-06T03:56:05.097784+0000","last_undegraded":"2023-07-06T03:56:05.097784+0000","last_fullsized":"2023-07-06T03:56:05.097784+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:09:32.189142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.084788+0000","last_change":"2023-07-06T03:56:04.088850+0000","last_active":"2023-07-06T03:56:05.084788+0000","last_peered":"2023-07-06T03:56:05.084788+0000","last_clean":"2023-07-06T03:56:05.084788+0000","last_became_active":"2023-07-06T03:56:04.088555+0000","last_became_peered":"2023-07-06T03:56:04.088555+0000","last_unstale":"2023-07-06T03:56:05.084788+0000","last_undegraded":"2023-07-06T03:56:05.084788+0000","last_fullsized":"2023-07-06T03:56:05.084788+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:25:17.423773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.106126+0000","last_change":"2023-07-06T03:56:05.092775+0000","last_active":"2023-07-06T03:56:05.106126+0000","last_peered":"2023-07-06T03:56:05.106126+0000","last_clean":"2023-07-06T03:56:05.106126+0000","last_became_active":"2023-07-06T03:56:05.091941+0000","last_became_peered":"2023-07-06T03:56:05.091941+0000","last_unstale":"2023-07-06T03:56:05.106126+0000","last_undegraded":"2023-07-06T03:56:05.106126+0000","last_fullsized":"2023-07-06T03:56:05.106126+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:34:59.543438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.079578+0000","last_change":"2023-07-06T03:56:04.090435+0000","last_active":"2023-07-06T03:56:05.079578+0000","last_peered":"2023-07-06T03:56:05.079578+0000","last_clean":"2023-07-06T03:56:05.079578+0000","last_became_active":"2023-07-06T03:56:04.090050+0000","last_became_peered":"2023-07-06T03:56:04.090050+0000","last_unstale":"2023-07-06T03:56:05.079578+0000","last_undegraded":"2023-07-06T03:56:05.079578+0000","last_fullsized":"2023-07-06T03:56:05.079578+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:20:48.464045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.108708+0000","last_change":"2023-07-06T03:56:05.100061+0000","last_active":"2023-07-06T03:56:05.108708+0000","last_peered":"2023-07-06T03:56:05.108708+0000","last_clean":"2023-07-06T03:56:05.108708+0000","last_became_active":"2023-07-06T03:56:05.099762+0000","last_became_peered":"2023-07-06T03:56:05.099762+0000","last_unstale":"2023-07-06T03:56:05.108708+0000","last_undegraded":"2023-07-06T03:56:05.108708+0000","last_fullsized":"2023-07-06T03:56:05.108708+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:55:16.541099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.078692+0000","last_change":"2023-07-06T03:56:04.095116+0000","last_active":"2023-07-06T03:56:05.078692+0000","last_peered":"2023-07-06T03:56:05.078692+0000","last_clean":"2023-07-06T03:56:05.078692+0000","last_became_active":"2023-07-06T03:56:04.094806+0000","last_became_peered":"2023-07-06T03:56:04.094806+0000","last_unstale":"2023-07-06T03:56:05.078692+0000","last_undegraded":"2023-07-06T03:56:05.078692+0000","last_fullsized":"2023-07-06T03:56:05.078692+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:15:56.489035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.102537+0000","last_change":"2023-07-06T03:56:05.099792+0000","last_active":"2023-07-06T03:56:05.102537+0000","last_peered":"2023-07-06T03:56:05.102537+0000","last_clean":"2023-07-06T03:56:05.102537+0000","last_became_active":"2023-07-06T03:56:05.099533+0000","last_became_peered":"2023-07-06T03:56:05.099533+0000","last_unstale":"2023-07-06T03:56:05.102537+0000","last_undegraded":"2023-07-06T03:56:05.102537+0000","last_fullsized":"2023-07-06T03:56:05.102537+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:42:14.854718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.085842+0000","last_change":"2023-07-06T03:56:04.087825+0000","last_active":"2023-07-06T03:56:05.085842+0000","last_peered":"2023-07-06T03:56:05.085842+0000","last_clean":"2023-07-06T03:56:05.085842+0000","last_became_active":"2023-07-06T03:56:04.087552+0000","last_became_peered":"2023-07-06T03:56:04.087552+0000","last_unstale":"2023-07-06T03:56:05.085842+0000","last_undegraded":"2023-07-06T03:56:05.085842+0000","last_fullsized":"2023-07-06T03:56:05.085842+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:25:16.895520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.110214+0000","last_change":"2023-07-06T03:56:05.099745+0000","last_active":"2023-07-06T03:56:05.110214+0000","last_peered":"2023-07-06T03:56:05.110214+0000","last_clean":"2023-07-06T03:56:05.110214+0000","last_became_active":"2023-07-06T03:56:05.099135+0000","last_became_peered":"2023-07-06T03:56:05.099135+0000","last_unstale":"2023-07-06T03:56:05.110214+0000","last_undegraded":"2023-07-06T03:56:05.110214+0000","last_fullsized":"2023-07-06T03:56:05.110214+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:09:07.509941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.078884+0000","last_change":"2023-07-06T03:56:04.089104+0000","last_active":"2023-07-06T03:56:05.078884+0000","last_peered":"2023-07-06T03:56:05.078884+0000","last_clean":"2023-07-06T03:56:05.078884+0000","last_became_active":"2023-07-06T03:56:04.088843+0000","last_became_peered":"2023-07-06T03:56:04.088843+0000","last_unstale":"2023-07-06T03:56:05.078884+0000","last_undegraded":"2023-07-06T03:56:05.078884+0000","last_fullsized":"2023-07-06T03:56:05.078884+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:35:38.087268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109629+0000","last_change":"2023-07-06T03:56:05.102989+0000","last_active":"2023-07-06T03:56:05.109629+0000","last_peered":"2023-07-06T03:56:05.109629+0000","last_clean":"2023-07-06T03:56:05.109629+0000","last_became_active":"2023-07-06T03:56:05.102713+0000","last_became_peered":"2023-07-06T03:56:05.102713+0000","last_unstale":"2023-07-06T03:56:05.109629+0000","last_undegraded":"2023-07-06T03:56:05.109629+0000","last_fullsized":"2023-07-06T03:56:05.109629+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:03:06.630621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"135'2","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.111591+0000","last_change":"2023-07-06T03:56:04.095792+0000","last_active":"2023-07-06T03:56:05.111591+0000","last_peered":"2023-07-06T03:56:05.111591+0000","last_clean":"2023-07-06T03:56:05.111591+0000","last_became_active":"2023-07-06T03:56:04.095378+0000","last_became_peered":"2023-07-06T03:56:04.095378+0000","last_unstale":"2023-07-06T03:56:05.111591+0000","last_undegraded":"2023-07-06T03:56:05.111591+0000","last_fullsized":"2023-07-06T03:56:05.111591+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:31:11.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":214,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.081952+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:56:05.081952+0000","last_peered":"2023-07-06T03:56:05.081952+0000","last_clean":"2023-07-06T03:56:05.081952+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:56:05.081952+0000","last_undegraded":"2023-07-06T03:56:05.081952+0000","last_fullsized":"2023-07-06T03:56:05.081952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:01:42.174318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.094719+0000","last_change":"2023-07-06T03:56:05.092821+0000","last_active":"2023-07-06T03:56:05.094719+0000","last_peered":"2023-07-06T03:56:05.094719+0000","last_clean":"2023-07-06T03:56:05.094719+0000","last_became_active":"2023-07-06T03:56:05.092439+0000","last_became_peered":"2023-07-06T03:56:05.092439+0000","last_unstale":"2023-07-06T03:56:05.094719+0000","last_undegraded":"2023-07-06T03:56:05.094719+0000","last_fullsized":"2023-07-06T03:56:05.094719+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:59:57.954018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"135'3","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.111914+0000","last_change":"2023-07-06T03:56:04.091569+0000","last_active":"2023-07-06T03:56:05.111914+0000","last_peered":"2023-07-06T03:56:05.111914+0000","last_clean":"2023-07-06T03:56:05.111914+0000","last_became_active":"2023-07-06T03:56:04.091106+0000","last_became_peered":"2023-07-06T03:56:04.091106+0000","last_unstale":"2023-07-06T03:56:05.111914+0000","last_undegraded":"2023-07-06T03:56:05.111914+0000","last_fullsized":"2023-07-06T03:56:05.111914+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:09:31.180856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":224,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.084958+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:56:05.084958+0000","last_peered":"2023-07-06T03:56:05.084958+0000","last_clean":"2023-07-06T03:56:05.084958+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:56:05.084958+0000","last_undegraded":"2023-07-06T03:56:05.084958+0000","last_fullsized":"2023-07-06T03:56:05.084958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:01:49.333338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.103539+0000","last_change":"2023-07-06T03:56:05.101894+0000","last_active":"2023-07-06T03:56:05.103539+0000","last_peered":"2023-07-06T03:56:05.103539+0000","last_clean":"2023-07-06T03:56:05.103539+0000","last_became_active":"2023-07-06T03:56:05.101633+0000","last_became_peered":"2023-07-06T03:56:05.101633+0000","last_unstale":"2023-07-06T03:56:05.103539+0000","last_undegraded":"2023-07-06T03:56:05.103539+0000","last_fullsized":"2023-07-06T03:56:05.103539+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:03:05.882071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.079658+0000","last_change":"2023-07-06T03:56:04.093046+0000","last_active":"2023-07-06T03:56:05.079658+0000","last_peered":"2023-07-06T03:56:05.079658+0000","last_clean":"2023-07-06T03:56:05.079658+0000","last_became_active":"2023-07-06T03:56:04.091847+0000","last_became_peered":"2023-07-06T03:56:04.091847+0000","last_unstale":"2023-07-06T03:56:05.079658+0000","last_undegraded":"2023-07-06T03:56:05.079658+0000","last_fullsized":"2023-07-06T03:56:05.079658+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:47:06.931215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":230,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.080791+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:56:05.080791+0000","last_peered":"2023-07-06T03:56:05.080791+0000","last_clean":"2023-07-06T03:56:05.080791+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:56:05.080791+0000","last_undegraded":"2023-07-06T03:56:05.080791+0000","last_fullsized":"2023-07-06T03:56:05.080791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:27:35.438765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109769+0000","last_change":"2023-07-06T03:56:05.099617+0000","last_active":"2023-07-06T03:56:05.109769+0000","last_peered":"2023-07-06T03:56:05.109769+0000","last_clean":"2023-07-06T03:56:05.109769+0000","last_became_active":"2023-07-06T03:56:05.099259+0000","last_became_peered":"2023-07-06T03:56:05.099259+0000","last_unstale":"2023-07-06T03:56:05.109769+0000","last_undegraded":"2023-07-06T03:56:05.109769+0000","last_fullsized":"2023-07-06T03:56:05.109769+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:09:49.370060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.14","version":"135'6","reported_seq":17,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.124460+0000","last_change":"2023-07-06T03:56:04.092768+0000","last_active":"2023-07-06T03:56:05.124460+0000","last_peered":"2023-07-06T03:56:05.124460+0000","last_clean":"2023-07-06T03:56:05.124460+0000","last_became_active":"2023-07-06T03:56:04.092270+0000","last_became_peered":"2023-07-06T03:56:04.092270+0000","last_unstale":"2023-07-06T03:56:05.124460+0000","last_undegraded":"2023-07-06T03:56:05.124460+0000","last_fullsized":"2023-07-06T03:56:05.124460+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:51:13.301731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":224,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.086893+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:56:05.086893+0000","last_peered":"2023-07-06T03:56:05.086893+0000","last_clean":"2023-07-06T03:56:05.086893+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:56:05.086893+0000","last_undegraded":"2023-07-06T03:56:05.086893+0000","last_fullsized":"2023-07-06T03:56:05.086893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:00:36.125441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":364,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.083945+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:56:05.083945+0000","last_peered":"2023-07-06T03:56:05.083945+0000","last_clean":"2023-07-06T03:56:05.083945+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:56:05.083945+0000","last_undegraded":"2023-07-06T03:56:05.083945+0000","last_fullsized":"2023-07-06T03:56:05.083945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:42:04.322836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.109821+0000","last_change":"2023-07-06T03:56:05.105251+0000","last_active":"2023-07-06T03:56:05.109821+0000","last_peered":"2023-07-06T03:56:05.109821+0000","last_clean":"2023-07-06T03:56:05.109821+0000","last_became_active":"2023-07-06T03:56:05.104976+0000","last_became_peered":"2023-07-06T03:56:05.104976+0000","last_unstale":"2023-07-06T03:56:05.109821+0000","last_undegraded":"2023-07-06T03:56:05.109821+0000","last_fullsized":"2023-07-06T03:56:05.109821+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:13:57.430942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.083949+0000","last_change":"2023-07-06T03:56:04.091701+0000","last_active":"2023-07-06T03:56:05.083949+0000","last_peered":"2023-07-06T03:56:05.083949+0000","last_clean":"2023-07-06T03:56:05.083949+0000","last_became_active":"2023-07-06T03:56:04.091437+0000","last_became_peered":"2023-07-06T03:56:04.091437+0000","last_unstale":"2023-07-06T03:56:05.083949+0000","last_undegraded":"2023-07-06T03:56:05.083949+0000","last_fullsized":"2023-07-06T03:56:05.083949+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:07:31.900656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":224,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.084109+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:56:05.084109+0000","last_peered":"2023-07-06T03:56:05.084109+0000","last_clean":"2023-07-06T03:56:05.084109+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:56:05.084109+0000","last_undegraded":"2023-07-06T03:56:05.084109+0000","last_fullsized":"2023-07-06T03:56:05.084109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:32:06.252387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.100322+0000","last_change":"2023-07-06T03:56:05.098013+0000","last_active":"2023-07-06T03:56:05.100322+0000","last_peered":"2023-07-06T03:56:05.100322+0000","last_clean":"2023-07-06T03:56:05.100322+0000","last_became_active":"2023-07-06T03:56:05.097722+0000","last_became_peered":"2023-07-06T03:56:05.097722+0000","last_unstale":"2023-07-06T03:56:05.100322+0000","last_undegraded":"2023-07-06T03:56:05.100322+0000","last_fullsized":"2023-07-06T03:56:05.100322+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:22:37.446960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.078878+0000","last_change":"2023-07-06T03:56:04.092797+0000","last_active":"2023-07-06T03:56:05.078878+0000","last_peered":"2023-07-06T03:56:05.078878+0000","last_clean":"2023-07-06T03:56:05.078878+0000","last_became_active":"2023-07-06T03:56:04.092532+0000","last_became_peered":"2023-07-06T03:56:04.092532+0000","last_unstale":"2023-07-06T03:56:05.078878+0000","last_undegraded":"2023-07-06T03:56:05.078878+0000","last_fullsized":"2023-07-06T03:56:05.078878+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:29:35.935289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":224,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.087502+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:56:05.087502+0000","last_peered":"2023-07-06T03:56:05.087502+0000","last_clean":"2023-07-06T03:56:05.087502+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:56:05.087502+0000","last_undegraded":"2023-07-06T03:56:05.087502+0000","last_fullsized":"2023-07-06T03:56:05.087502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T15:28:41.530100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.094846+0000","last_change":"2023-07-06T03:56:05.091933+0000","last_active":"2023-07-06T03:56:05.094846+0000","last_peered":"2023-07-06T03:56:05.094846+0000","last_clean":"2023-07-06T03:56:05.094846+0000","last_became_active":"2023-07-06T03:56:05.091615+0000","last_became_peered":"2023-07-06T03:56:05.091615+0000","last_unstale":"2023-07-06T03:56:05.094846+0000","last_undegraded":"2023-07-06T03:56:05.094846+0000","last_fullsized":"2023-07-06T03:56:05.094846+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:58:01.135846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.078705+0000","last_change":"2023-07-06T03:56:04.089470+0000","last_active":"2023-07-06T03:56:05.078705+0000","last_peered":"2023-07-06T03:56:05.078705+0000","last_clean":"2023-07-06T03:56:05.078705+0000","last_became_active":"2023-07-06T03:56:04.089213+0000","last_became_peered":"2023-07-06T03:56:04.089213+0000","last_unstale":"2023-07-06T03:56:05.078705+0000","last_undegraded":"2023-07-06T03:56:05.078705+0000","last_fullsized":"2023-07-06T03:56:05.078705+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:39:43.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":218,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.084404+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:56:05.084404+0000","last_peered":"2023-07-06T03:56:05.084404+0000","last_clean":"2023-07-06T03:56:05.084404+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:56:05.084404+0000","last_undegraded":"2023-07-06T03:56:05.084404+0000","last_fullsized":"2023-07-06T03:56:05.084404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:53:31.183596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.096941+0000","last_change":"2023-07-06T03:56:05.093444+0000","last_active":"2023-07-06T03:56:05.096941+0000","last_peered":"2023-07-06T03:56:05.096941+0000","last_clean":"2023-07-06T03:56:05.096941+0000","last_became_active":"2023-07-06T03:56:05.093034+0000","last_became_peered":"2023-07-06T03:56:05.093034+0000","last_unstale":"2023-07-06T03:56:05.096941+0000","last_undegraded":"2023-07-06T03:56:05.096941+0000","last_fullsized":"2023-07-06T03:56:05.096941+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:04.064994+0000","last_clean_scrub_stamp":"2023-07-06T03:56:04.064994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:45:58.939632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.10","version":"135'8","reported_seq":19,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.122657+0000","last_change":"2023-07-06T03:56:04.090878+0000","last_active":"2023-07-06T03:56:05.122657+0000","last_peered":"2023-07-06T03:56:05.122657+0000","last_clean":"2023-07-06T03:56:05.122657+0000","last_became_active":"2023-07-06T03:56:04.090528+0000","last_became_peered":"2023-07-06T03:56:04.090528+0000","last_unstale":"2023-07-06T03:56:05.122657+0000","last_undegraded":"2023-07-06T03:56:05.122657+0000","last_fullsized":"2023-07-06T03:56:05.122657+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:56:03.056709+0000","last_clean_scrub_stamp":"2023-07-06T03:56:03.056709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:46:59.004584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":224,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-07-06T03:56:05.085899+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:56:05.085899+0000","last_peered":"2023-07-06T03:56:05.085899+0000","last_clean":"2023-07-06T03:56:05.085899+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:56:05.085899+0000","last_undegraded":"2023-07-06T03:56:05.085899+0000","last_fullsized":"2023-07-06T03:56:05.085899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:50:56.461158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574985,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1699840,"data_stored":778472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.85299999999999998,"5min":1.024,"15min":4.0570000000000004},"last":0.625},{"interface":"front","average":{"1min":0.62,"5min":0.67500000000000004,"15min":0.73699999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":0.83299999999999996,"5min":0.95499999999999996,"15min":3.8660000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.48599999999999999,"5min":0.45400000000000001,"15min":0.34000000000000002},"max":{"1min":0.90400000000000003,"5min":1.091,"15min":2.6219999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.68300000000000005,"15min":0.80300000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.38800000000000001},"max":{"1min":0.83699999999999997,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68400000000000005,"15min":0.747},"min":{"1min":0.47899999999999998,"5min":0.39700000000000002,"15min":0.35199999999999998},"max":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":3.77},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.45000000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.98899999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.47799999999999998,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":0.875,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.752,"15min":0.751},"min":{"1min":0.625,"5min":0.54500000000000004,"15min":0.43099999999999999},"max":{"1min":0.91000000000000003,"5min":0.98699999999999999,"15min":1.0209999999999999},"last":0.71399999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.46600000000000003,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.90200000000000002,"15min":0.94699999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68700000000000006,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.40400000000000003,"15min":0.374},"max":{"1min":0.90500000000000003,"5min":0.94399999999999995,"15min":1.4590000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.54200000000000004,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.97099999999999997,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69699999999999995,"15min":0.68500000000000005},"min":{"1min":0.53500000000000003,"5min":0.45300000000000001,"15min":0.40000000000000002},"max":{"1min":0.93500000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.59099999999999997,"5min":0.46400000000000002,"15min":0.438},"max":{"1min":0.89600000000000002,"5min":1.0089999999999999,"15min":1.5349999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.748,"15min":0.77500000000000002},"min":{"1min":0.623,"5min":0.49199999999999999,"15min":0.42299999999999999},"max":{"1min":1.046,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.80400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36800,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335040,"statfs":{"total":96636764160,"available":96599080960,"internally_reserved":0,"allocated":1695744,"data_stored":778430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.76400000000000001,"15min":0.68999999999999995},"min":{"1min":0.39700000000000002,"5min":0.32800000000000001,"15min":0.29399999999999998},"max":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.76400000000000001,"15min":0.748},"min":{"1min":0.52200000000000002,"5min":0.44700000000000001,"15min":0.26900000000000002},"max":{"1min":3.3119999999999998,"5min":3.3119999999999998,"15min":3.3399999999999999},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.73599999999999999,"15min":0.68799999999999994},"min":{"1min":0.45500000000000002,"5min":0.372,"15min":0.34599999999999997},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.755,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":5.96,"5min":5.96,"15min":5.96},"last":0.76300000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.79800000000000004,"15min":0.71099999999999997},"min":{"1min":0.504,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":8.3740000000000006,"5min":8.3740000000000006,"15min":8.3740000000000006},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.79400000000000004,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.47699999999999998,"15min":0.38700000000000001},"max":{"1min":5.8449999999999998,"5min":5.8449999999999998,"15min":5.8449999999999998},"last":0.81799999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.48699999999999999,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.109,"5min":0.82999999999999996,"15min":0.75900000000000001},"min":{"1min":0.55200000000000005,"5min":0.51000000000000001,"15min":0.373},"max":{"1min":8.5269999999999992,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.72399999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.33200000000000002},"max":{"1min":5.7759999999999998,"5min":5.7759999999999998,"15min":5.7759999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.69599999999999995,"15min":0.72099999999999997},"min":{"1min":0.44400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.5190000000000001,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.89000000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.73799999999999999,"15min":0.71799999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.38600000000000001},"max":{"1min":8.4870000000000001,"5min":8.4870000000000001,"15min":8.4870000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.77500000000000002,"15min":0.78100000000000003},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":5.923,"5min":5.923,"15min":5.923},"last":0.86699999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.76000000000000001,"15min":0.79700000000000004},"min":{"1min":0.5,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.129,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":8.4429999999999996,"5min":8.4429999999999996,"15min":8.4429999999999996},"last":0.78200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574985,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37380,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334460,"statfs":{"total":96636764160,"available":96598487040,"internally_reserved":0,"allocated":2289664,"data_stored":1368750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42699999999999999,"15min":0.496},"min":{"1min":0.29099999999999998,"5min":0.26900000000000002,"15min":0.23699999999999999},"max":{"1min":0.69499999999999995,"5min":0.72299999999999998,"15min":2.8879999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.52700000000000002,"15min":0.54900000000000004},"min":{"1min":0.34999999999999998,"5min":0.32300000000000001,"15min":0.313},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.46300000000000002,"15min":0.73899999999999999},"min":{"1min":0.28199999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.3690000000000002,"5min":3.3690000000000002,"15min":13.055999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.56299999999999994,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":3.0590000000000002,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.752},{"interface":"front","average":{"1min":0.624,"5min":0.51500000000000001,"15min":0.52800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.29499999999999998},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.74199999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72499999999999998,"15min":1.002},"min":{"1min":0.442,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":12.989000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.55900000000000005,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":1.139,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.753},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.432},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71099999999999997,"15min":1.002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.494,"5min":1.494,"15min":13.023999999999999},"last":0.81000000000000005}]},{"osd":6,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77800000000000002,"15min":0.79700000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.9670000000000001,"5min":2.9670000000000001,"15min":2.9670000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.77000000000000002,"15min":1.05},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47799999999999998},"max":{"1min":2.5409999999999999,"5min":2.5409999999999999,"15min":12.941000000000001},"last":0.88900000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.76600000000000001,"15min":0.80300000000000005},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.431,"5min":2.431,"15min":2.806},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.751,"15min":0.78900000000000003},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.85999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574986,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1699840,"data_stored":778464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.40899999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.25800000000000001},"max":{"1min":0.54900000000000004,"5min":0.753,"15min":0.753},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.441,"15min":0.437},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.58699999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.5}]},{"osd":2,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.46999999999999997},"min":{"1min":0.32100000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.56699999999999995,"5min":0.69199999999999995,"15min":0.73699999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47999999999999998,"15min":0.49199999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.314},"max":{"1min":0.61499999999999999,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.32200000000000001,"15min":0.29899999999999999},"max":{"1min":0.627,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.38800000000000001},"max":{"1min":0.77600000000000002,"5min":0.80500000000000005,"15min":0.84399999999999997},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":1.1699999999999999,"15min":1.171},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.748,"15min":0.72399999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.76300000000000001,"15min":0.73599999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45600000000000002},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.46999999999999997},"max":{"1min":2.7549999999999999,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.51100000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.46200000000000002},"max":{"1min":0.96099999999999997,"5min":1.0649999999999999,"15min":1.546},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jul 6 03:55:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.628,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.623},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.78700000000000003,"15min":0.96299999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":9.5960000000000001},"last":0.76400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574985,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40892,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330948,"statfs":{"total":96636764160,"available":96594890752,"internally_reserved":0,"allocated":1691648,"data_stored":778360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.47299999999999998,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56299999999999994,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.30099999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58299999999999996,"5min":1.093,"15min":2.4630000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.54200000000000004,"15min":0.50700000000000001},"min":{"1min":0.33600000000000002,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.61899999999999999,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.57699999999999996,"5min":3.657,"15min":3.657},"last":0.441}]},{"osd":3,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.61699999999999999,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.69999999999999996,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.42299999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65400000000000003,"5min":2.1629999999999998,"15min":2.7679999999999998},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.92000000000000004,"15min":0.78900000000000003},"min":{"1min":0.58099999999999996,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.103,"5min":12.007,"15min":12.007},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.79000000000000004,"15min":0.74299999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0389999999999999,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79500000000000004,"15min":0.754},"min":{"1min":0.58899999999999997,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":1.089,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.85799999999999998,"15min":0.79800000000000004},"min":{"1min":0.54700000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0680000000000001,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.92300000000000004,"15min":0.79300000000000004},"min":{"1min":0.50900000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.079,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.90900000000000003,"15min":0.81100000000000005},"min":{"1min":0.55000000000000004,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":1.0489999999999999,"5min":13.084,"15min":13.084},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:55:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81999999999999995,"15min":0.82299999999999995},"min":{"1min":0.60099999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.022,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.85299999999999998,"15min":0.81899999999999995},"min":{"1min":0.56599999999999995,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.573,"5min":4.484,"15min":4.484},"last":0.66400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574985,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1703936,"data_stored":779924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45000000000000001,"15min":0.54500000000000004},"min":{"1min":0.28499999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56699999999999995,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.59099999999999997,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.44700000000000001,"15min":0.81599999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57499999999999996,"5min":1.3620000000000001,"15min":15.911},"last":0.376},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.89300000000000002},"min":{"1min":0.38800000000000001,"5min":0.32100000000000001,"15min":0.29199999999999998},"max":{"1min":0.72599999999999998,"5min":1.347,"15min":16.024000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.49299999999999999,"15min":0.49199999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.60899999999999999,"5min":3.25,"15min":3.25},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.54300000000000004,"15min":0.89800000000000002},"min":{"1min":0.379,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80100000000000005,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73199999999999998,"15min":0.74299999999999999},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.99099999999999999,"5min":1.782,"15min":1.782},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.754,"15min":0.85899999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.36699999999999999},"max":{"1min":0.98499999999999999,"5min":1.724,"15min":5.7300000000000004},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.75800000000000001,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.49199999999999999,"15min":0.39700000000000002},"max":{"1min":1.036,"5min":1.411,"15min":5.077},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.79100000000000004,"15min":0.90700000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.44800000000000001},"max":{"1min":1.0680000000000001,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.76300000000000001,"15min":0.78000000000000003},"min":{"1min":0.52800000000000002,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.135,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.75900000000000001,"15min":0.86899999999999999},"min":{"1min":0.55300000000000005,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.0960000000000001,"5min":1.4490000000000001,"15min":5.157},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:55:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.76000000000000001,"15min":0.80100000000000005},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.47399999999999998},"max":{"1min":1.1639999999999999,"5min":1.863,"15min":1.863},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76300000000000001,"15min":0.88},"min":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.49399999999999999},"max":{"1min":0.97099999999999997,"5min":1.492,"15min":5.1189999999999998},"last":0.75}]}]},{"osd":4,"up_from":13,"seq":55834574984,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1699840,"data_stored":778976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.784},"last":0.755},{"interface":"front","average":{"1min":0.622,"5min":0.67400000000000004,"15min":0.64900000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.38},"max":{"1min":0.95199999999999996,"5min":0.96599999999999997,"15min":1.0509999999999999},"last":0.503}]},{"osd":1,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.79000000000000004},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.38},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":6.258},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.806},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.80700000000000005},"min":{"1min":0.48599999999999999,"5min":0.42199999999999999,"15min":0.29199999999999998},"max":{"1min":1.012,"5min":1.012,"15min":6.4740000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.877},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.51900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":6.4859999999999998},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.68200000000000005},"min":{"1min":0.56599999999999995,"5min":0.45400000000000001,"15min":0.40000000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.25},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.84099999999999997,"5min":0.874,"15min":1.413},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61099999999999999,"15min":0.64100000000000001},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88300000000000001,"5min":0.89100000000000001,"15min":1.3280000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.441,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.84599999999999997,"5min":0.93000000000000005,"15min":0.99299999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.41899999999999998,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.91100000000000003,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.753,"15min":0.77600000000000002},"min":{"1min":0.57699999999999996,"5min":0.52400000000000002,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":0.96299999999999997,"15min":1.6839999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:55:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68700000000000006,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.41299999999999998},"max":{"1min":0.84399999999999997,"5min":0.89200000000000002,"15min":1.0660000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.51200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.85999999999999999,"5min":1.036,"15min":1.21},"last":0.88200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574986,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":2301952,"data_stored":1369945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.91100000000000003,"15min":4.5010000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.67400000000000004,"15min":0.78500000000000003},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.83399999999999996,"5min":0.96199999999999997,"15min":6.4059999999999997},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.502,"5min":0.44800000000000001,"15min":0.307},"max":{"1min":0.76200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67600000000000005,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.46100000000000002,"15min":0.41399999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.546},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79300000000000004,"15min":0.72799999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.376},"max":{"1min":0.86499999999999999,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.93799999999999994,"15min":1.0029999999999999},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.79100000000000004,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.47599999999999998,"15min":0.42199999999999999},"max":{"1min":0.78100000000000003,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71399999999999997,"15min":0.73899999999999999},"min":{"1min":0.495,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":0.85299999999999998,"5min":1.3140000000000001,"15min":4.484},"last":0.72899999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61699999999999999,"15min":0.626},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":1.0229999999999999,"15min":1.6240000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64000000000000001,"15min":0.68700000000000006},"min":{"1min":0.41899999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":2.718},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83899999999999997,"5min":0.98499999999999999,"15min":2.5249999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.70999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.82599999999999996,"5min":0.94099999999999995,"15min":1.669},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:55:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71499999999999997,"15min":0.754},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.93799999999999994,"5min":1.002,"15min":2.0459999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73699999999999999,"15min":0.77000000000000002},"min":{"1min":0.47199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88,"5min":1.0249999999999999,"15min":2.2919999999999998},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:56:09.975 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T03:56:09.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T03:56:10.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.208+0000 7fd0fafff700 1 Processor -- start 2023-07-06T03:56:10.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.208+0000 7fd0fafff700 1 -- start start 2023-07-06T03:56:10.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.209+0000 7fd0fafff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40b4ea0 0x7fd0f40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.209+0000 7fd0fafff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd0f4160a50 0x7fd0f40d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.209+0000 7fd0fafff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f40d0c10 con 0x7fd0f40b8590 2023-07-06T03:56:10.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.209+0000 7fd0fafff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f4160ef0 con 0x7fd0f4160a50 2023-07-06T03:56:10.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.209+0000 7fd0fafff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f40d08e0 con 0x7fd0f40b4ea0 2023-07-06T03:56:10.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.209+0000 7fd0f8d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd0f40b8590 0x7fd0f40b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35828/0 (socket says 172.21.15.97:35828) 2023-07-06T03:56:10.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.209+0000 7fd0f8d9b700 1 -- 172.21.15.97:0/2548745357 learned_addr learned my addr 172.21.15.97:0/2548745357 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:10.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f3fff700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40b4ea0 0x7fd0f40cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141332915 0 0) 0x7fd0f40d0c10 con 0x7fd0f40b8590 2023-07-06T03:56:10.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f959c700 1 --2- 172.21.15.97:0/2548745357 >> v2:172.21.15.146:3300/0 conn(0x7fd0f4160a50 0x7fd0f40d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0e40035b0 con 0x7fd0f40b8590 2023-07-06T03:56:10.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 507074220 0 0) 0x7fd0e40035b0 con 0x7fd0f40b8590 2023-07-06T03:56:10.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40b4ea0 msgr2=0x7fd0f40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:10.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40b4ea0 0x7fd0f40cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 >> v2:172.21.15.146:3300/0 conn(0x7fd0f4160a50 msgr2=0x7fd0f40d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:10.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 --2- 172.21.15.97:0/2548745357 >> v2:172.21.15.146:3300/0 conn(0x7fd0f4160a50 0x7fd0f40d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f40b2560 con 0x7fd0f40b8590 2023-07-06T03:56:10.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fd0e8001670 con 0x7fd0f40b8590 2023-07-06T03:56:10.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 >> v1:172.21.15.97:6789/0 conn(0x7fd0f40b8590 legacy=0x7fd0f40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40b4ea0 0x7fd0e4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0e4003f80 0x7fd0e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 0x7fd0e4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f40d08e0 con 0x7fd0e4004d90 2023-07-06T03:56:10.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e400ba10 con 0x7fd0e4003f80 2023-07-06T03:56:10.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e400bb50 con 0x7fd0f40b4ea0 2023-07-06T03:56:10.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.210+0000 7fd0f959c700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 0x7fd0e4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f3fff700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0e4003f80 0x7fd0e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f8d9b700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40b4ea0 0x7fd0e4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f959c700 1 -- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40b4ea0 msgr2=0x7fd0e4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:10.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f959c700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40b4ea0 0x7fd0e4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f959c700 1 -- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0e4003f80 msgr2=0x7fd0e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:10.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f959c700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0e4003f80 0x7fd0e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f959c700 1 -- 172.21.15.97:0/2548745357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40035b0 con 0x7fd0e4004d90 2023-07-06T03:56:10.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f8d9b700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40b4ea0 0x7fd0e4003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:10.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f959c700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 0x7fd0e4005200 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009fc0 tx=0x7fd0e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a164cd1f3c30db5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:10.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec00ec30 con 0x7fd0e4004d90 2023-07-06T03:56:10.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ec00f040 con 0x7fd0e4004d90 2023-07-06T03:56:10.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.211+0000 7fd0f37fe700 1 -- 172.21.15.97:0/2548745357 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec014480 con 0x7fd0e4004d90 2023-07-06T03:56:10.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.212+0000 7fd0fafff700 1 -- 172.21.15.97:0/2548745357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 msgr2=0x7fd0e4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.212+0000 7fd0fafff700 1 --2- 172.21.15.97:0/2548745357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 0x7fd0e4005200 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009fc0 tx=0x7fd0e4005880 comp rx=0 tx=0).stop 2023-07-06T03:56:10.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.212+0000 7fd0f959c700 1 -- 172.21.15.97:0/2548745357 reap_dead start 2023-07-06T03:56:10.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.212+0000 7fd0fafff700 1 -- 172.21.15.97:0/2548745357 shutdown_connections 2023-07-06T03:56:10.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.212+0000 7fd0fafff700 1 -- 172.21.15.97:0/2548745357 >> 172.21.15.97:0/2548745357 conn(0x7fd0f406b290 msgr2=0x7fd0f40cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:10.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.212+0000 7fd0fafff700 1 -- 172.21.15.97:0/2548745357 shutdown_connections 2023-07-06T03:56:10.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.212+0000 7fd0fafff700 1 -- 172.21.15.97:0/2548745357 wait complete. 2023-07-06T03:56:10.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.214+0000 7fd0fafff700 1 Processor -- start 2023-07-06T03:56:10.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.214+0000 7fd0fafff700 1 -- start start 2023-07-06T03:56:10.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0fafff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 0x7fd0f415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0fafff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40aa7b0 0x7fd0f40aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0fafff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0f40b3020 0x7fd0f40b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0fafff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f40b39e0 con 0x7fd0e4004d90 2023-07-06T03:56:10.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0fafff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f40ab270 con 0x7fd0f40b3020 2023-07-06T03:56:10.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0fafff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f415cc30 con 0x7fd0f40aa7b0 2023-07-06T03:56:10.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0f8d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 0x7fd0f415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0f8d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 0x7fd0f415a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59172/0 (socket says 172.21.15.97:59172) 2023-07-06T03:56:10.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0f8d9b700 1 -- 172.21.15.97:0/1340539638 learned_addr learned my addr 172.21.15.97:0/1340539638 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:10.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0f8d9b700 1 -- 172.21.15.97:0/1340539638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40aa7b0 msgr2=0x7fd0f40aac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0f8d9b700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40aa7b0 0x7fd0f40aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0f8d9b700 1 -- 172.21.15.97:0/1340539638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0f40b3020 msgr2=0x7fd0f40b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0f8d9b700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0f40b3020 0x7fd0f40b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.215+0000 7fd0f8d9b700 1 -- 172.21.15.97:0/1340539638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f415cf10 con 0x7fd0e4004d90 2023-07-06T03:56:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.216+0000 7fd0f8d9b700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 0x7fd0f415a040 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800eaf0 tx=0x7fd0e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.216+0000 7fd0f17fa700 1 -- 172.21.15.97:0/1340539638 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8017070 con 0x7fd0e4004d90 2023-07-06T03:56:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.216+0000 7fd0f17fa700 1 -- 172.21.15.97:0/1340539638 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e8004870 con 0x7fd0e4004d90 2023-07-06T03:56:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.216+0000 7fd0f17fa700 1 -- 172.21.15.97:0/1340539638 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e80049e0 con 0x7fd0e4004d90 2023-07-06T03:56:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.216+0000 7fd0fafff700 1 -- 172.21.15.97:0/1340539638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f415d1f0 con 0x7fd0e4004d90 2023-07-06T03:56:10.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.216+0000 7fd0fafff700 1 -- 172.21.15.97:0/1340539638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f415d9f0 con 0x7fd0e4004d90 2023-07-06T03:56:10.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.217+0000 7fd0fafff700 1 -- 172.21.15.97:0/1340539638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f408be70 con 0x7fd0e4004d90 2023-07-06T03:56:10.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.218+0000 7fd0f17fa700 1 -- 172.21.15.97:0/1340539638 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd0e8004b40 con 0x7fd0e4004d90 2023-07-06T03:56:10.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.219+0000 7fd0f17fa700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 0x7fd0d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.219+0000 7fd0f17fa700 1 -- 172.21.15.97:0/1340539638 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd0e8088ce0 con 0x7fd0e4004d90 2023-07-06T03:56:10.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.219+0000 7fd0f3fff700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 0x7fd0d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.220+0000 7fd0f3fff700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 0x7fd0d806d190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40b5290 tx=0x7fd0e00093c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:10.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.223+0000 7fd0f17fa700 1 -- 172.21.15.97:0/1340539638 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd0e8057960 con 0x7fd0e4004d90 2023-07-06T03:56:10.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.495+0000 7fd0fafff700 1 -- 172.21.15.97:0/1340539638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd0f408be70 con 0x7fd0e4004d90 2023-07-06T03:56:10.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.500+0000 7fd0f17fa700 1 -- 172.21.15.97:0/1340539638 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd0e80574f0 con 0x7fd0e4004d90 2023-07-06T03:56:10.502 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:56:10.503 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQCHOKZk/YrsChAAQkWOWDBjr1azAg80+oPPMg== 2023-07-06T03:56:10.503 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-07-06T03:56:10.503 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-07-06T03:56:10.503 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T03:56:10.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 -- 172.21.15.97:0/1340539638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 msgr2=0x7fd0d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 0x7fd0d806d190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40b5290 tx=0x7fd0e00093c0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 -- 172.21.15.97:0/1340539638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 msgr2=0x7fd0f415a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 0x7fd0f415a040 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800eaf0 tx=0x7fd0e800eeb0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 -- 172.21.15.97:0/1340539638 shutdown_connections 2023-07-06T03:56:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd0d806ace0 0x7fd0d806d190 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0f40b3020 0x7fd0f40b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e4004d90 0x7fd0f415a040 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 --2- 172.21.15.97:0/1340539638 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0f40aa7b0 0x7fd0f40aac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 -- 172.21.15.97:0/1340539638 >> 172.21.15.97:0/1340539638 conn(0x7fd0f406b290 msgr2=0x7fd0f40b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 -- 172.21.15.97:0/1340539638 shutdown_connections 2023-07-06T03:56:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.502+0000 7fd0fafff700 1 -- 172.21.15.97:0/1340539638 wait complete. 2023-07-06T03:56:10.507 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-07-06T03:56:10.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:56:10.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.714+0000 7fab155ff700 1 Processor -- start 2023-07-06T03:56:10.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.714+0000 7fab155ff700 1 -- start start 2023-07-06T03:56:10.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab155ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fab1006d6f0 0x7fab1015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab155ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fab100b0a40 0x7fab10159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab155ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1006ddb0 con 0x7fab100b05d0 2023-07-06T03:56:10.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab155ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1015d3d0 con 0x7fab1006d6f0 2023-07-06T03:56:10.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab155ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1006bbf0 con 0x7fab100b0a40 2023-07-06T03:56:10.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab07fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fab100b05d0 0x7fab100712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35842/0 (socket says 172.21.15.97:35842) 2023-07-06T03:56:10.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab07fff700 1 -- 172.21.15.97:0/4102683167 learned_addr learned my addr 172.21.15.97:0/4102683167 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:10.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab0effd700 1 --2- 172.21.15.97:0/4102683167 >> v2:172.21.15.97:3300/0 conn(0x7fab1006d6f0 0x7fab1015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab0f7fe700 1 --2- 172.21.15.97:0/4102683167 >> v2:172.21.15.146:3300/0 conn(0x7fab100b0a40 0x7fab10159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab0e7fc700 1 -- 172.21.15.97:0/4102683167 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578503267 0 0) 0x7fab1006ddb0 con 0x7fab100b05d0 2023-07-06T03:56:10.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab0e7fc700 1 -- 172.21.15.97:0/4102683167 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faafc0057c0 con 0x7fab100b05d0 2023-07-06T03:56:10.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab0effd700 1 -- 172.21.15.97:0/4102683167 >> v2:172.21.15.146:3300/0 conn(0x7fab100b0a40 msgr2=0x7fab10159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab0effd700 1 --2- 172.21.15.97:0/4102683167 >> v2:172.21.15.146:3300/0 conn(0x7fab100b0a40 0x7fab10159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.716+0000 7fab0effd700 1 -- 172.21.15.97:0/4102683167 >> v1:172.21.15.97:6789/0 conn(0x7fab100b05d0 legacy=0x7fab100712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.717+0000 7fab0effd700 1 -- 172.21.15.97:0/4102683167 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1006bed0 con 0x7fab1006d6f0 2023-07-06T03:56:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.717+0000 7fab0f7fe700 1 --2- 172.21.15.97:0/4102683167 >> v2:172.21.15.146:3300/0 conn(0x7fab100b0a40 0x7fab10159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.717+0000 7fab0effd700 1 --2- 172.21.15.97:0/4102683167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab1006d6f0 0x7fab1015ce90 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fab100cb310 tx=0x7fab00009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f740a0f38d78f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.717+0000 7fab0e7fc700 1 -- 172.21.15.97:0/4102683167 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00011830 con 0x7fab1006d6f0 2023-07-06T03:56:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.717+0000 7fab0e7fc700 1 -- 172.21.15.97:0/4102683167 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab00011990 con 0x7fab1006d6f0 2023-07-06T03:56:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.717+0000 7fab0e7fc700 1 -- 172.21.15.97:0/4102683167 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00011d00 con 0x7fab1006d6f0 2023-07-06T03:56:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.717+0000 7fab155ff700 1 -- 172.21.15.97:0/4102683167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab1006d6f0 msgr2=0x7fab1015ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.717+0000 7fab155ff700 1 --2- 172.21.15.97:0/4102683167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab1006d6f0 0x7fab1015ce90 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fab100cb310 tx=0x7fab00009b20 comp rx=0 tx=0).stop 2023-07-06T03:56:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.718+0000 7fab155ff700 1 -- 172.21.15.97:0/4102683167 shutdown_connections 2023-07-06T03:56:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.718+0000 7fab155ff700 1 --2- 172.21.15.97:0/4102683167 >> v2:172.21.15.146:3300/0 conn(0x7fab100b0a40 0x7fab10159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.718+0000 7fab155ff700 1 --2- 172.21.15.97:0/4102683167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab1006d6f0 0x7fab1015ce90 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.718+0000 7fab155ff700 1 -- 172.21.15.97:0/4102683167 >> 172.21.15.97:0/4102683167 conn(0x7fab100ce8c0 msgr2=0x7fab100cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:10.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.718+0000 7fab155ff700 1 -- 172.21.15.97:0/4102683167 shutdown_connections 2023-07-06T03:56:10.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.718+0000 7fab155ff700 1 -- 172.21.15.97:0/4102683167 wait complete. 2023-07-06T03:56:10.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.719+0000 7fab155ff700 1 Processor -- start 2023-07-06T03:56:10.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.719+0000 7fab155ff700 1 -- start start 2023-07-06T03:56:10.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.720+0000 7fab155ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab1006d6f0 0x7fab10159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.720+0000 7fab155ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab100b05d0 0x7fab10147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.720+0000 7fab155ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab100b0a40 0x7fab1015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.720+0000 7fab155ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1015b3b0 con 0x7fab100b05d0 2023-07-06T03:56:10.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.720+0000 7fab155ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1015a730 con 0x7fab1006d6f0 2023-07-06T03:56:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.720+0000 7fab155ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1015aa10 con 0x7fab100b0a40 2023-07-06T03:56:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab100b05d0 0x7fab10147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab100b05d0 0x7fab10147f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59190/0 (socket says 172.21.15.97:59190) 2023-07-06T03:56:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab07fff700 1 -- 172.21.15.97:0/4291441872 learned_addr learned my addr 172.21.15.97:0/4291441872 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab0effd700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab1006d6f0 0x7fab10159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab0f7fe700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab100b0a40 0x7fab1015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab07fff700 1 -- 172.21.15.97:0/4291441872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab100b0a40 msgr2=0x7fab1015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab07fff700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab100b0a40 0x7fab1015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab07fff700 1 -- 172.21.15.97:0/4291441872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab1006d6f0 msgr2=0x7fab10159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab07fff700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab1006d6f0 0x7fab10159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab07fff700 1 -- 172.21.15.97:0/4291441872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab000094a0 con 0x7fab100b05d0 2023-07-06T03:56:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab0effd700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab1006d6f0 0x7fab10159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:10.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab0f7fe700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab100b0a40 0x7fab1015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.721+0000 7fab07fff700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab100b05d0 0x7fab10147f30 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c4d0 tx=0x7faaf800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.722+0000 7fab077fe700 1 -- 172.21.15.97:0/4291441872 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8005d70 con 0x7fab100b05d0 2023-07-06T03:56:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.722+0000 7fab077fe700 1 -- 172.21.15.97:0/4291441872 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf80070d0 con 0x7fab100b05d0 2023-07-06T03:56:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.722+0000 7fab077fe700 1 -- 172.21.15.97:0/4291441872 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8007440 con 0x7fab100b05d0 2023-07-06T03:56:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.722+0000 7fab155ff700 1 -- 172.21.15.97:0/4291441872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1015acf0 con 0x7fab100b05d0 2023-07-06T03:56:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.722+0000 7fab155ff700 1 -- 172.21.15.97:0/4291441872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1015a420 con 0x7fab100b05d0 2023-07-06T03:56:10.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.723+0000 7fab155ff700 1 -- 172.21.15.97:0/4291441872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1015b4f0 con 0x7fab100b05d0 2023-07-06T03:56:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.724+0000 7fab077fe700 1 -- 172.21.15.97:0/4291441872 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faaf80075a0 con 0x7fab100b05d0 2023-07-06T03:56:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.725+0000 7fab077fe700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faaf006ace0 0x7faaf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:10.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.725+0000 7fab077fe700 1 -- 172.21.15.97:0/4291441872 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faaf8088950 con 0x7fab100b05d0 2023-07-06T03:56:10.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.725+0000 7fab0effd700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faaf006ace0 0x7faaf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:10.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.726+0000 7fab0effd700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faaf006ace0 0x7faaf006d190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fab000090b0 tx=0x7fab00008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:10.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.729+0000 7fab077fe700 1 -- 172.21.15.97:0/4291441872 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fab1015b4f0 con 0x7fab100b05d0 2023-07-06T03:56:10.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.970+0000 7fab155ff700 1 -- 172.21.15.97:0/4291441872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab1015b4f0 con 0x7fab100b05d0 2023-07-06T03:56:10.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.970+0000 7fab077fe700 1 -- 172.21.15.97:0/4291441872 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4680 (secure 0 0 0) 0x7fab1015b4f0 con 0x7fab100b05d0 2023-07-06T03:56:10.974 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:10.974 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/3313682816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3313682816},{"type":"v1","addr":"172.21.15.146:6834","nonce":3313682816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9624,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/387461585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":387461585},{"type":"v1","addr":"172.21.15.146:6835","nonce":387461585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9630,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3109027091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3109027091},{"type":"v1","addr":"172.21.15.97:6835","nonce":3109027091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:56:04.566296+0000","modified":"2023-07-06T03:56:08.096494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9633":{"gid":9633,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2797569799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2797569799},{"type":"v1","addr":"172.21.15.97:6837","nonce":2797569799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-06T03:56:10.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 -- 172.21.15.97:0/4291441872 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faaf006ace0 msgr2=0x7faaf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faaf006ace0 0x7faaf006d190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fab000090b0 tx=0x7fab00008030 comp rx=0 tx=0).stop 2023-07-06T03:56:10.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 -- 172.21.15.97:0/4291441872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab100b05d0 msgr2=0x7fab10147f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:10.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab100b05d0 0x7fab10147f30 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c4d0 tx=0x7faaf800c890 comp rx=0 tx=0).stop 2023-07-06T03:56:10.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 -- 172.21.15.97:0/4291441872 shutdown_connections 2023-07-06T03:56:10.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7faaf006ace0 0x7faaf006d190 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab100b0a40 0x7fab1015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab1006d6f0 0x7fab10159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 --2- 172.21.15.97:0/4291441872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab100b05d0 0x7fab10147f30 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:10.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 -- 172.21.15.97:0/4291441872 >> 172.21.15.97:0/4291441872 conn(0x7fab100ce8c0 msgr2=0x7fab10154040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:10.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 -- 172.21.15.97:0/4291441872 shutdown_connections 2023-07-06T03:56:10.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:10.973+0000 7fab155ff700 1 -- 172.21.15.97:0/4291441872 wait complete. 2023-07-06T03:56:10.977 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 103 2023-07-06T03:56:11.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T03:56:04.566296+0000', 'modified': '2023-07-06T03:56:08.096494+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9633}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9633': {'gid': 9633, 'name': 'c', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/2797569799', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 2797569799}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 2797569799}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T03:56:11.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T03:56:11.397 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-06T03:56:11.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.591+0000 7f76d1fff700 1 Processor -- start 2023-07-06T03:56:11.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.591+0000 7f76d1fff700 1 -- start start 2023-07-06T03:56:11.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.592+0000 7f76d1fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f76cc154890 0x7f76cc0cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.592+0000 7f76d1fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76cc15b450 con 0x7f76cc15b050 2023-07-06T03:56:11.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.592+0000 7f76d1fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76cc15b6e0 con 0x7f76cc15eb90 2023-07-06T03:56:11.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.592+0000 7f76d1fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76cc15e0b0 con 0x7f76cc154890 2023-07-06T03:56:11.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.593+0000 7f76cbfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f76cc15b050 0x7f76cc0c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35854/0 (socket says 172.21.15.97:35854) 2023-07-06T03:56:11.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.593+0000 7f76cbfff700 1 -- 172.21.15.97:0/3510814668 learned_addr learned my addr 172.21.15.97:0/3510814668 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.593+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/3510814668 >> v2:172.21.15.146:3300/0 conn(0x7f76cc154890 0x7f76cc0cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.593+0000 7f76ca7fc700 1 -- 172.21.15.97:0/3510814668 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963928860 0 0) 0x7f76cc15b450 con 0x7f76cc15b050 2023-07-06T03:56:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.593+0000 7f76ca7fc700 1 -- 172.21.15.97:0/3510814668 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76b40035b0 con 0x7f76cc15b050 2023-07-06T03:56:11.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.593+0000 7f76cb7fe700 1 -- 172.21.15.97:0/3510814668 >> v1:172.21.15.146:6789/0 conn(0x7f76cc15eb90 legacy=0x7f76cc15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.593+0000 7f76cb7fe700 1 -- 172.21.15.97:0/3510814668 >> v1:172.21.15.97:6789/0 conn(0x7f76cc15b050 legacy=0x7f76cc0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.593+0000 7f76ca7fc700 1 -- 172.21.15.97:0/3510814668 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 118230146 0 0) 0x7f76b40035b0 con 0x7f76cc15b050 2023-07-06T03:56:11.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.593+0000 7f76cb7fe700 1 -- 172.21.15.97:0/3510814668 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc15e390 con 0x7f76cc154890 2023-07-06T03:56:11.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.594+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/3510814668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76cc154890 0x7f76cc0cb390 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009840 tx=0x7f76bc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63ad07b25af5fdf0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:11.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.594+0000 7f76ca7fc700 1 -- 172.21.15.97:0/3510814668 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c40019f0 con 0x7f76cc154890 2023-07-06T03:56:11.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.594+0000 7f76ca7fc700 1 -- 172.21.15.97:0/3510814668 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc004030 con 0x7f76cc154890 2023-07-06T03:56:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.594+0000 7f76ca7fc700 1 -- 172.21.15.97:0/3510814668 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0043c0 con 0x7f76cc154890 2023-07-06T03:56:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.594+0000 7f76d1fff700 1 -- 172.21.15.97:0/3510814668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76cc154890 msgr2=0x7f76cc0cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.594+0000 7f76d1fff700 1 --2- 172.21.15.97:0/3510814668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76cc154890 0x7f76cc0cb390 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009840 tx=0x7f76bc009b50 comp rx=0 tx=0).stop 2023-07-06T03:56:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.595+0000 7f76d1fff700 1 -- 172.21.15.97:0/3510814668 shutdown_connections 2023-07-06T03:56:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.595+0000 7f76d1fff700 1 --2- 172.21.15.97:0/3510814668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76cc154890 0x7f76cc0cb390 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.595+0000 7f76d1fff700 1 -- 172.21.15.97:0/3510814668 >> 172.21.15.97:0/3510814668 conn(0x7f76cc06a060 msgr2=0x7f76cc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:11.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.595+0000 7f76d1fff700 1 -- 172.21.15.97:0/3510814668 shutdown_connections 2023-07-06T03:56:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.595+0000 7f76d1fff700 1 -- 172.21.15.97:0/3510814668 wait complete. 2023-07-06T03:56:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.596+0000 7f76d1fff700 1 Processor -- start 2023-07-06T03:56:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.596+0000 7f76d1fff700 1 -- start start 2023-07-06T03:56:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.597+0000 7f76d1fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76cc154890 0x7f76cc0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.597+0000 7f76d1fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76cc15b050 0x7f76cc14bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.597+0000 7f76d1fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15eb90 0x7f76cc0cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.597+0000 7f76d1fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc15d8e0 con 0x7f76cc15eb90 2023-07-06T03:56:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.597+0000 7f76d1fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc0c5a60 con 0x7f76cc15b050 2023-07-06T03:56:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.597+0000 7f76d1fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc0c5d40 con 0x7f76cc154890 2023-07-06T03:56:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76cbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15eb90 0x7f76cc0cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76cbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15eb90 0x7f76cc0cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59198/0 (socket says 172.21.15.97:59198) 2023-07-06T03:56:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76cbfff700 1 -- 172.21.15.97:0/197464041 learned_addr learned my addr 172.21.15.97:0/197464041 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76caffd700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76cc15b050 0x7f76cc14bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76cc154890 0x7f76cc0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76cbfff700 1 -- 172.21.15.97:0/197464041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76cc154890 msgr2=0x7f76cc0c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76cbfff700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76cc154890 0x7f76cc0c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76cbfff700 1 -- 172.21.15.97:0/197464041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76cc15b050 msgr2=0x7f76cc14bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76cbfff700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76cc15b050 0x7f76cc14bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76cbfff700 1 -- 172.21.15.97:0/197464041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc0094a0 con 0x7f76cc15eb90 2023-07-06T03:56:11.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76cbfff700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15eb90 0x7f76cc0cd510 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f76c4001ba0 tx=0x7f76c400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:11.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76b3fff700 1 -- 172.21.15.97:0/197464041 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c400ac20 con 0x7f76cc15eb90 2023-07-06T03:56:11.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76b3fff700 1 -- 172.21.15.97:0/197464041 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c400ad80 con 0x7f76cc15eb90 2023-07-06T03:56:11.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.598+0000 7f76d1fff700 1 -- 172.21.15.97:0/197464041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc0c6020 con 0x7f76cc15eb90 2023-07-06T03:56:11.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.599+0000 7f76b3fff700 1 -- 172.21.15.97:0/197464041 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4007310 con 0x7f76cc15eb90 2023-07-06T03:56:11.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.599+0000 7f76d1fff700 1 -- 172.21.15.97:0/197464041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc14c300 con 0x7f76cc15eb90 2023-07-06T03:56:11.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.601+0000 7f76b3fff700 1 -- 172.21.15.97:0/197464041 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f76c4007470 con 0x7f76cc15eb90 2023-07-06T03:56:11.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.602+0000 7f76b3fff700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f76b806ad10 0x7f76b806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.602+0000 7f76b3fff700 1 -- 172.21.15.97:0/197464041 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76c40890e0 con 0x7f76cc15eb90 2023-07-06T03:56:11.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.602+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f76b806ad10 0x7f76b806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.602+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f76b806ad10 0x7f76b806d1c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0090b0 tx=0x7f76bc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:11.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.603+0000 7f76e59b2b80 1 Processor -- start 2023-07-06T03:56:11.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.604+0000 7f76e59b2b80 1 -- start start 2023-07-06T03:56:11.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.604+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x56297878ad50 0x56297878b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.604+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562978790190 0x5629787aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.604+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978768110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.604+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629786dcea0 con 0x562978765b50 2023-07-06T03:56:11.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.604+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562978650600 con 0x562978790190 2023-07-06T03:56:11.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.604+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629786bf910 con 0x56297878ad50 2023-07-06T03:56:11.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76cbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978768110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76cbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978768110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59212/0 (socket says 172.21.15.97:59212) 2023-07-06T03:56:11.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76cbfff700 1 -- 172.21.15.97:0/2837555988 learned_addr learned my addr 172.21.15.97:0/2837555988 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:11.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/2837555988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x56297878ad50 0x56297878b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76caffd700 1 --2- 172.21.15.97:0/2837555988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562978790190 0x5629787aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76cbfff700 1 -- 172.21.15.97:0/2837555988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x56297878ad50 msgr2=0x56297878b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76cbfff700 1 --2- 172.21.15.97:0/2837555988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x56297878ad50 0x56297878b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76cbfff700 1 -- 172.21.15.97:0/2837555988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562978790190 msgr2=0x5629787aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76cbfff700 1 --2- 172.21.15.97:0/2837555988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562978790190 0x5629787aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76cbfff700 1 -- 172.21.15.97:0/2837555988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629784ac790 con 0x562978765b50 2023-07-06T03:56:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76caffd700 1 --2- 172.21.15.97:0/2837555988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562978790190 0x5629787aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76cbfff700 1 --2- 172.21.15.97:0/2837555988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978768110 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f76c4057810 tx=0x7f76c4057840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1fea6ae03142b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76b17fa700 1 -- 172.21.15.97:0/2837555988 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4060100 con 0x562978765b50 2023-07-06T03:56:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76b17fa700 1 -- 172.21.15.97:0/2837555988 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c4056150 con 0x562978765b50 2023-07-06T03:56:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.605+0000 7f76b17fa700 1 -- 172.21.15.97:0/2837555988 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c405fe70 con 0x562978765b50 2023-07-06T03:56:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.606+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2837555988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 msgr2=0x562978768110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.606+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/2837555988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978768110 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f76c4057810 tx=0x7f76c4057840 comp rx=0 tx=0).stop 2023-07-06T03:56:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.606+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2837555988 shutdown_connections 2023-07-06T03:56:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.606+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/2837555988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978768110 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.606+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/2837555988 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x56297878ad50 0x56297878b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.606+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/2837555988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562978790190 0x5629787aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.606+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2837555988 >> 172.21.15.97:0/2837555988 conn(0x56297875d310 msgr2=0x5629787ad5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:11.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.606+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2837555988 shutdown_connections 2023-07-06T03:56:11.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.606+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2837555988 wait complete. 2023-07-06T03:56:11.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.607+0000 7f76e59b2b80 1 -- start start 2023-07-06T03:56:11.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.607+0000 7f76e59b2b80 1 Processor -- start 2023-07-06T03:56:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.608+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978785560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.608+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56297878ad50 0x562978792e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.608+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 0x562978791a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.608+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629786b8be0 con 0x562978765b50 2023-07-06T03:56:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.608+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629786765b0 con 0x56297878ad50 2023-07-06T03:56:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.608+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x56297865c910 con 0x562978790190 2023-07-06T03:56:11.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.608+0000 7f76e59b2b80 10 client.0 _ll_register_callbacks cb 0x562978674b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:56:11.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76cb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978785560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76cb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978785560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59214/0 (socket says 172.21.15.97:59214) 2023-07-06T03:56:11.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76cb7fe700 1 -- 172.21.15.97:0/2154210898 learned_addr learned my addr 172.21.15.97:0/2154210898 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:11.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76cb7fe700 1 -- 172.21.15.97:0/2154210898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 msgr2=0x562978791a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 0x562978791a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76caffd700 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56297878ad50 0x562978792e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76cb7fe700 1 -- 172.21.15.97:0/2154210898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56297878ad50 msgr2=0x562978792e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56297878ad50 0x562978792e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76cb7fe700 1 -- 172.21.15.97:0/2154210898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc14c300 con 0x562978765b50 2023-07-06T03:56:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978785560 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0037e0 tx=0x7f76bc00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cbea270dc776a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76aa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:56:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.609+0000 7f76aa7fc700 1 -- 172.21.15.97:0/2154210898 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc02a040 con 0x562978765b50 2023-07-06T03:56:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76aa7fc700 1 -- 172.21.15.97:0/2154210898 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76bc011840 con 0x562978765b50 2023-07-06T03:56:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2154210898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56297875a920 con 0x562978765b50 2023-07-06T03:56:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76aa7fc700 1 -- 172.21.15.97:0/2154210898 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76bc01f070 con 0x562978765b50 2023-07-06T03:56:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76e59b2b80 10 client.9774 fetch_fsmap learned FSMap version 103 2023-07-06T03:56:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2154210898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x5629785990a0 con 0x562978765b50 2023-07-06T03:56:11.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76aa7fc700 1 -- 172.21.15.97:0/2154210898 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56297875a920 con 0x562978765b50 2023-07-06T03:56:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76aa7fc700 1 -- 172.21.15.97:0/2154210898 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 103) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f76bc02d030 con 0x562978765b50 2023-07-06T03:56:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76e59b2b80 10 client.9774 fetch_fsmap finished waiting for FSMap version 103 2023-07-06T03:56:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76e59b2b80 10 client.9774 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T03:56:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76e59b2b80 20 client.9774 populate_metadata read hostname 'smithi097' 2023-07-06T03:56:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.97:6836/2797569799,v1:172.21.15.97:6837/2797569799] conn(0x562978789600 0x5629787899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76e59b2b80 4 client.9774 mds_command: new command op to 9633 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T03:56:11.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2154210898 --> [v2:172.21.15.97:6836/2797569799,v1:172.21.15.97:6837/2797569799] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629787370d0 con 0x562978789600 2023-07-06T03:56:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.610+0000 7f76caffd700 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.97:6836/2797569799,v1:172.21.15.97:6837/2797569799] conn(0x562978789600 0x5629787899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.611+0000 7f76caffd700 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.97:6836/2797569799,v1:172.21.15.97:6837/2797569799] conn(0x562978789600 0x5629787899d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78dc162b0ef82da5 server_cookie=ee385b62b3119876 in_seq=0 out_seq=0 2023-07-06T03:56:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.611+0000 7f76aa7fc700 10 client.9774 ms_handle_connect on v2:172.21.15.97:6836/2797569799 2023-07-06T03:56:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76aa7fc700 1 -- 172.21.15.97:0/2154210898 <== mds.0 v2:172.21.15.97:6836/2797569799 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5629787370d0 con 0x562978789600 2023-07-06T03:56:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76aa7fc700 10 client.9774 handle_command_reply: tid=0 2023-07-06T03:56:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 1 client.9774 shutdown 2023-07-06T03:56:11.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2154210898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 msgr2=0x562978785560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978785560 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0037e0 tx=0x7f76bc00e000 comp rx=0 tx=0).stop 2023-07-06T03:56:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2154210898 shutdown_connections 2023-07-06T03:56:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 0x562978791a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978785560 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.97:6836/2797569799,v1:172.21.15.97:6837/2797569799] conn(0x562978789600 0x5629787899d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/2154210898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56297878ad50 0x562978792e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2154210898 >> 172.21.15.97:0/2154210898 conn(0x56297875d310 msgr2=0x562978790590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76aa7fc700 0 client.9774 ms_handle_reset on v2:172.21.15.97:6836/2797569799 2023-07-06T03:56:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2154210898 shutdown_connections 2023-07-06T03:56:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 1 -- 172.21.15.97:0/2154210898 wait complete. 2023-07-06T03:56:11.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.612+0000 7f76e59b2b80 20 client.9774 trim_cache size 0 max 16384 2023-07-06T03:56:11.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.630+0000 7f76e59b2b80 1 Processor -- start 2023-07-06T03:56:11.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.630+0000 7f76e59b2b80 1 -- start start 2023-07-06T03:56:11.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.631+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562978765b50 0x5629787cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.631+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56297878ad50 0x562978762fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.631+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 0x562978763a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.631+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629784ad390 con 0x56297878ad50 2023-07-06T03:56:11.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.631+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x562978787e20 con 0x562978765b50 2023-07-06T03:56:11.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.631+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x562978762660 con 0x562978790190 2023-07-06T03:56:11.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.631+0000 7f76caffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56297878ad50 0x562978762fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.631+0000 7f76caffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56297878ad50 0x562978762fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59226/0 (socket says 172.21.15.97:59226) 2023-07-06T03:56:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.631+0000 7f76caffd700 1 -- 172.21.15.97:0/221810674 learned_addr learned my addr 172.21.15.97:0/221810674 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.631+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/221810674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562978765b50 0x5629787cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.631+0000 7f76cbfff700 1 --2- 172.21.15.97:0/221810674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 0x562978763a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.632+0000 7f76caffd700 1 -- 172.21.15.97:0/221810674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 msgr2=0x562978763a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.632+0000 7f76caffd700 1 --2- 172.21.15.97:0/221810674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 0x562978763a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.632+0000 7f76caffd700 1 -- 172.21.15.97:0/221810674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562978765b50 msgr2=0x5629787cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.632+0000 7f76caffd700 1 --2- 172.21.15.97:0/221810674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562978765b50 0x5629787cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.632+0000 7f76caffd700 1 -- 172.21.15.97:0/221810674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56297867eb10 con 0x56297878ad50 2023-07-06T03:56:11.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.632+0000 7f76caffd700 1 --2- 172.21.15.97:0/221810674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56297878ad50 0x562978762fc0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f76c00056b0 tx=0x7f76c00186c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305bb04965b746a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:11.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.632+0000 7f76b1ffb700 1 -- 172.21.15.97:0/221810674 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0003d20 con 0x56297878ad50 2023-07-06T03:56:11.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.632+0000 7f76b1ffb700 1 -- 172.21.15.97:0/221810674 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c0003e80 con 0x56297878ad50 2023-07-06T03:56:11.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.632+0000 7f76b1ffb700 1 -- 172.21.15.97:0/221810674 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0018cc0 con 0x56297878ad50 2023-07-06T03:56:11.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.633+0000 7f76e59b2b80 1 -- 172.21.15.97:0/221810674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56297878ad50 msgr2=0x562978762fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.633+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/221810674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56297878ad50 0x562978762fc0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f76c00056b0 tx=0x7f76c00186c0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.633+0000 7f76e59b2b80 1 -- 172.21.15.97:0/221810674 shutdown_connections 2023-07-06T03:56:11.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.633+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/221810674 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 0x562978763a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.633+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/221810674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562978765b50 0x5629787cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.633+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/221810674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56297878ad50 0x562978762fc0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.633+0000 7f76e59b2b80 1 -- 172.21.15.97:0/221810674 >> 172.21.15.97:0/221810674 conn(0x56297875d310 msgr2=0x5629787a6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:11.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.633+0000 7f76e59b2b80 1 -- 172.21.15.97:0/221810674 shutdown_connections 2023-07-06T03:56:11.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.633+0000 7f76e59b2b80 1 -- 172.21.15.97:0/221810674 wait complete. 2023-07-06T03:56:11.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.633+0000 7f76e59b2b80 1 -- start start 2023-07-06T03:56:11.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.634+0000 7f76e59b2b80 1 Processor -- start 2023-07-06T03:56:11.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.634+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978787f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56297878ad50 0x562978811620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76e59b2b80 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 0x562978791290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x562978788560 con 0x562978765b50 2023-07-06T03:56:11.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629788176a0 con 0x56297878ad50 2023-07-06T03:56:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76e59b2b80 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629788178f0 con 0x562978790190 2023-07-06T03:56:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76e59b2b80 10 client.0 _ll_register_callbacks cb 0x562978674b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T03:56:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76cb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978787f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76cb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978787f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59232/0 (socket says 172.21.15.97:59232) 2023-07-06T03:56:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76cb7fe700 1 -- 172.21.15.97:0/462982385 learned_addr learned my addr 172.21.15.97:0/462982385 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76cb7fe700 1 -- 172.21.15.97:0/462982385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 msgr2=0x562978791290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/462982385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 0x562978791290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76cb7fe700 1 -- 172.21.15.97:0/462982385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56297878ad50 msgr2=0x562978811620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:11.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/462982385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56297878ad50 0x562978811620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76cb7fe700 1 -- 172.21.15.97:0/462982385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c000f040 con 0x562978765b50 2023-07-06T03:56:11.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76cb7fe700 1 --2- 172.21.15.97:0/462982385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978787f60 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f76bc011a70 tx=0x7f76bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5498bd189d16634 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:11.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.635+0000 7f76aa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-06T03:56:11.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76aa7fc700 1 -- 172.21.15.97:0/462982385 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc01a740 con 0x562978765b50 2023-07-06T03:56:11.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76aa7fc700 1 -- 172.21.15.97:0/462982385 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76bc01a8a0 con 0x562978765b50 2023-07-06T03:56:11.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76e59b2b80 1 -- 172.21.15.97:0/462982385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629786b1d20 con 0x562978765b50 2023-07-06T03:56:11.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76aa7fc700 1 -- 172.21.15.97:0/462982385 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76bc01aa60 con 0x562978765b50 2023-07-06T03:56:11.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76e59b2b80 10 client.9780 fetch_fsmap learned FSMap version 103 2023-07-06T03:56:11.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76e59b2b80 1 -- 172.21.15.97:0/462982385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x56297850d1d0 con 0x562978765b50 2023-07-06T03:56:11.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76aa7fc700 1 -- 172.21.15.97:0/462982385 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5629786b1d20 con 0x562978765b50 2023-07-06T03:56:11.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76aa7fc700 1 -- 172.21.15.97:0/462982385 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 103) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f76bc018070 con 0x562978765b50 2023-07-06T03:56:11.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76e59b2b80 10 client.9780 fetch_fsmap finished waiting for FSMap version 103 2023-07-06T03:56:11.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76e59b2b80 10 client.9780 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T03:56:11.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76e59b2b80 20 client.9780 populate_metadata read hostname 'smithi097' 2023-07-06T03:56:11.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/462982385 >> [v2:172.21.15.97:6836/2797569799,v1:172.21.15.97:6837/2797569799] conn(0x562978768470 0x562978768860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76e59b2b80 4 client.9780 mds_command: new command op to 9633 tid=0[{"prefix": "status"}] 2023-07-06T03:56:11.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76e59b2b80 1 -- 172.21.15.97:0/462982385 --> [v2:172.21.15.97:6836/2797569799,v1:172.21.15.97:6837/2797569799] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562978507ef0 con 0x562978768470 2023-07-06T03:56:11.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.636+0000 7f76caffd700 1 --2- 172.21.15.97:0/462982385 >> [v2:172.21.15.97:6836/2797569799,v1:172.21.15.97:6837/2797569799] conn(0x562978768470 0x562978768860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76aa7fc700 10 client.9780 ms_handle_connect on v2:172.21.15.97:6836/2797569799 2023-07-06T03:56:11.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76caffd700 1 --2- 172.21.15.97:0/462982385 >> [v2:172.21.15.97:6836/2797569799,v1:172.21.15.97:6837/2797569799] conn(0x562978768470 0x562978768860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f625be667921b5 server_cookie=568775f6d486ddc1 in_seq=0 out_seq=0 2023-07-06T03:56:11.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76aa7fc700 1 -- 172.21.15.97:0/462982385 <== mds.0 v2:172.21.15.97:6836/2797569799 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x562978507ef0 con 0x562978768470 2023-07-06T03:56:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76aa7fc700 10 client.9780 handle_command_reply: tid=0 2023-07-06T03:56:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76e59b2b80 1 client.9780 shutdown 2023-07-06T03:56:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76e59b2b80 1 -- 172.21.15.97:0/462982385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 msgr2=0x562978787f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/462982385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978787f60 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f76bc011a70 tx=0x7f76bc0047c0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76e59b2b80 1 -- 172.21.15.97:0/462982385 shutdown_connections 2023-07-06T03:56:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/462982385 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x562978790190 0x562978791290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/462982385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562978765b50 0x562978787f60 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/462982385 >> [v2:172.21.15.97:6836/2797569799,v1:172.21.15.97:6837/2797569799] conn(0x562978768470 0x562978768860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76e59b2b80 1 --2- 172.21.15.97:0/462982385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56297878ad50 0x562978811620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76aa7fc700 0 client.9780 ms_handle_reset on v2:172.21.15.97:6836/2797569799 2023-07-06T03:56:11.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76e59b2b80 1 -- 172.21.15.97:0/462982385 >> 172.21.15.97:0/462982385 conn(0x56297875d310 msgr2=0x5629787ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:11.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.637+0000 7f76e59b2b80 1 -- 172.21.15.97:0/462982385 shutdown_connections 2023-07-06T03:56:11.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76e59b2b80 1 -- 172.21.15.97:0/462982385 wait complete. 2023-07-06T03:56:11.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76e59b2b80 20 client.9780 trim_cache size 0 max 16384 2023-07-06T03:56:11.649 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:56:11.649 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36", 2023-07-06T03:56:11.649 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-07-06T03:56:11.649 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9633, 2023-07-06T03:56:11.649 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-07-06T03:56:11.650 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-07-06T03:56:11.650 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-07-06T03:56:11.650 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.5526810339999999, 2023-07-06T03:56:11.650 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 103, 2023-07-06T03:56:11.650 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 136, 2023-07-06T03:56:11.650 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 136, 2023-07-06T03:56:11.650 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 13.064093931 2023-07-06T03:56:11.650 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:56:11.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 -- 172.21.15.97:0/197464041 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f76b806ad10 msgr2=0x7f76b806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f76b806ad10 0x7f76b806d1c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0090b0 tx=0x7f76bc008030 comp rx=0 tx=0).stop 2023-07-06T03:56:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 -- 172.21.15.97:0/197464041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15eb90 msgr2=0x7f76cc0cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15eb90 0x7f76cc0cd510 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f76c4001ba0 tx=0x7f76c400a060 comp rx=0 tx=0).stop 2023-07-06T03:56:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 -- 172.21.15.97:0/197464041 shutdown_connections 2023-07-06T03:56:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f76b806ad10 0x7f76b806d1c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76cc15eb90 0x7f76cc0cd510 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f76cc154890 0x7f76cc0c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 --2- 172.21.15.97:0/197464041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76cc15b050 0x7f76cc14bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 -- 172.21.15.97:0/197464041 >> 172.21.15.97:0/197464041 conn(0x7f76cc06a060 msgr2=0x7f76cc0c38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 -- 172.21.15.97:0/197464041 shutdown_connections 2023-07-06T03:56:11.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.638+0000 7f76d1fff700 1 -- 172.21.15.97:0/197464041 wait complete. 2023-07-06T03:56:11.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:56:11.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.855+0000 7f6da97ff700 1 Processor -- start 2023-07-06T03:56:11.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.855+0000 7f6da97ff700 1 -- start start 2023-07-06T03:56:11.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.856+0000 7f6da97ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6da40cd390 0x7f6da4160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.857+0000 7f6da97ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da40cdae0 con 0x7f6da4160dd0 2023-07-06T03:56:11.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.857+0000 7f6da97ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da40b6640 con 0x7f6da415c520 2023-07-06T03:56:11.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.857+0000 7f6da97ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40b6920 con 0x7f6da40cd390 2023-07-06T03:56:11.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.857+0000 7f6da37fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6da4160dd0 0x7f6da40b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35856/0 (socket says 172.21.15.97:35856) 2023-07-06T03:56:11.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.857+0000 7f6da37fe700 1 -- 172.21.15.97:0/3427790160 learned_addr learned my addr 172.21.15.97:0/3427790160 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.857+0000 7f6d9bfff700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6da40cd390 0x7f6da4160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.857+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001316509 0 0) 0x7f6da40cdae0 con 0x7f6da4160dd0 2023-07-06T03:56:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.857+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d900035b0 con 0x7f6da4160dd0 2023-07-06T03:56:11.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1646033220 0 0) 0x7f6d900035b0 con 0x7f6da4160dd0 2023-07-06T03:56:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6da40cd390 msgr2=0x7f6da4160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6da40cd390 0x7f6da4160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 >> v1:172.21.15.146:6789/0 conn(0x7f6da415c520 legacy=0x7f6da4160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da40b6c00 con 0x7f6da4160dd0 2023-07-06T03:56:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f6d9c0019f0 con 0x7f6da4160dd0 2023-07-06T03:56:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 >> v1:172.21.15.97:6789/0 conn(0x7f6da4160dd0 legacy=0x7f6da40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6da40cd390 0x7f6d90003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d90003f70 0x7f6d90004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d90004d90 0x7f6d90005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40b6920 con 0x7f6d90003f70 2023-07-06T03:56:11.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9000ba40 con 0x7f6d90004d90 2023-07-06T03:56:11.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9000bc00 con 0x7f6da40cd390 2023-07-06T03:56:11.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6d9bfff700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d90003f70 0x7f6d90004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da2ffd700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6da40cd390 0x7f6d90003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da37fe700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d90004d90 0x7f6d90005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6d9bfff700 1 -- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6da40cd390 msgr2=0x7f6d90003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6d9bfff700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6da40cd390 0x7f6d90003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6d9bfff700 1 -- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d90004d90 msgr2=0x7f6d90005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6d9bfff700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d90004d90 0x7f6d90005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6d9bfff700 1 -- 172.21.15.97:0/3427790160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d900035b0 con 0x7f6d90003f70 2023-07-06T03:56:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.858+0000 7f6da2ffd700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6da40cd390 0x7f6d90003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:11.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.859+0000 7f6d9bfff700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d90003f70 0x7f6d90004850 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009b60 tx=0x7f6d90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be727f55a459255 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.859+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00e6c0 con 0x7f6d90003f70 2023-07-06T03:56:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.859+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c00e820 con 0x7f6d90003f70 2023-07-06T03:56:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.859+0000 7f6da27fc700 1 -- 172.21.15.97:0/3427790160 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00eb90 con 0x7f6d90003f70 2023-07-06T03:56:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.859+0000 7f6da97ff700 1 -- 172.21.15.97:0/3427790160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d90003f70 msgr2=0x7f6d90004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.859+0000 7f6da97ff700 1 --2- 172.21.15.97:0/3427790160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d90003f70 0x7f6d90004850 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009b60 tx=0x7f6d90005880 comp rx=0 tx=0).stop 2023-07-06T03:56:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.859+0000 7f6da37fe700 1 -- 172.21.15.97:0/3427790160 reap_dead start 2023-07-06T03:56:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.859+0000 7f6da97ff700 1 -- 172.21.15.97:0/3427790160 shutdown_connections 2023-07-06T03:56:11.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.859+0000 7f6da97ff700 1 -- 172.21.15.97:0/3427790160 >> 172.21.15.97:0/3427790160 conn(0x7f6da4071790 msgr2=0x7f6da4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.860+0000 7f6da97ff700 1 -- 172.21.15.97:0/3427790160 shutdown_connections 2023-07-06T03:56:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.860+0000 7f6da97ff700 1 -- 172.21.15.97:0/3427790160 wait complete. 2023-07-06T03:56:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.862+0000 7f6da97ff700 1 Processor -- start 2023-07-06T03:56:11.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.862+0000 7f6da97ff700 1 -- start start 2023-07-06T03:56:11.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.863+0000 7f6da97ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d90003f70 0x7f6da415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.863+0000 7f6da97ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6da415eda0 0x7f6da40d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.863+0000 7f6da97ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6da40d34c0 0x7f6da40cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.863+0000 7f6da97ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da4158830 con 0x7f6da415eda0 2023-07-06T03:56:11.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.863+0000 7f6da97ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da4157f20 con 0x7f6da40d34c0 2023-07-06T03:56:11.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.863+0000 7f6da97ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40d3a20 con 0x7f6d90003f70 2023-07-06T03:56:11.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6d9bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6da415eda0 0x7f6da40d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6d9bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6da415eda0 0x7f6da40d2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59242/0 (socket says 172.21.15.97:59242) 2023-07-06T03:56:11.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6d9bfff700 1 -- 172.21.15.97:0/2140930677 learned_addr learned my addr 172.21.15.97:0/2140930677 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:11.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6da2ffd700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d90003f70 0x7f6da415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6da37fe700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6da40d34c0 0x7f6da40cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6d9bfff700 1 -- 172.21.15.97:0/2140930677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d90003f70 msgr2=0x7f6da415a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6d9bfff700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d90003f70 0x7f6da415a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6d9bfff700 1 -- 172.21.15.97:0/2140930677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6da40d34c0 msgr2=0x7f6da40cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:11.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6d9bfff700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6da40d34c0 0x7f6da40cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:11.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6d9bfff700 1 -- 172.21.15.97:0/2140930677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da4158170 con 0x7f6da415eda0 2023-07-06T03:56:11.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6da2ffd700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d90003f70 0x7f6da415a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:11.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6da37fe700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6da40d34c0 0x7f6da40cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:11.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6d9bfff700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6da415eda0 0x7f6da40d2f80 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000ba10 tx=0x7f6d8c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:11.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.864+0000 7f6d9b7fe700 1 -- 172.21.15.97:0/2140930677 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00e6c0 con 0x7f6da415eda0 2023-07-06T03:56:11.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.865+0000 7f6d9b7fe700 1 -- 172.21.15.97:0/2140930677 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c003ab0 con 0x7f6da415eda0 2023-07-06T03:56:11.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.865+0000 7f6da97ff700 1 -- 172.21.15.97:0/2140930677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da4158450 con 0x7f6da415eda0 2023-07-06T03:56:11.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.865+0000 7f6d9b7fe700 1 -- 172.21.15.97:0/2140930677 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c003e60 con 0x7f6da415eda0 2023-07-06T03:56:11.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.865+0000 7f6da97ff700 1 -- 172.21.15.97:0/2140930677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da40cc520 con 0x7f6da415eda0 2023-07-06T03:56:11.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.866+0000 7f6da97ff700 1 -- 172.21.15.97:0/2140930677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da404e340 con 0x7f6da415eda0 2023-07-06T03:56:11.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.867+0000 7f6d9b7fe700 1 -- 172.21.15.97:0/2140930677 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6d8c013470 con 0x7f6da415eda0 2023-07-06T03:56:11.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.867+0000 7f6d9b7fe700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8406ace0 0x7f6d8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:11.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.868+0000 7f6d9b7fe700 1 -- 172.21.15.97:0/2140930677 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6d8c08af10 con 0x7f6da415eda0 2023-07-06T03:56:11.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.868+0000 7f6da2ffd700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8406ace0 0x7f6d8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:11.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.868+0000 7f6da2ffd700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8406ace0 0x7f6d8406d190 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6da40c6810 tx=0x7f6d94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:11.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:11.872+0000 7f6d9b7fe700 1 -- 172.21.15.97:0/2140930677 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6d8c0560d0 con 0x7f6da415eda0 2023-07-06T03:56:12.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.116+0000 7f6da97ff700 1 -- 172.21.15.97:0/2140930677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6da40b4f40 con 0x7f6da415eda0 2023-07-06T03:56:12.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.117+0000 7f6d9b7fe700 1 -- 172.21.15.97:0/2140930677 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6d8c0596f0 con 0x7f6da415eda0 2023-07-06T03:56:12.120 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:56:12.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 -- 172.21.15.97:0/2140930677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8406ace0 msgr2=0x7f6d8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:12.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8406ace0 0x7f6d8406d190 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6da40c6810 tx=0x7f6d94009040 comp rx=0 tx=0).stop 2023-07-06T03:56:12.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 -- 172.21.15.97:0/2140930677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6da415eda0 msgr2=0x7f6da40d2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:12.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6da415eda0 0x7f6da40d2f80 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000ba10 tx=0x7f6d8c0047c0 comp rx=0 tx=0).stop 2023-07-06T03:56:12.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 -- 172.21.15.97:0/2140930677 shutdown_connections 2023-07-06T03:56:12.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f6d8406ace0 0x7f6d8406d190 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:12.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6da40d34c0 0x7f6da40cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:12.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f6d90003f70 0x7f6da415a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:12.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 --2- 172.21.15.97:0/2140930677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6da415eda0 0x7f6da40d2f80 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:12.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 -- 172.21.15.97:0/2140930677 >> 172.21.15.97:0/2140930677 conn(0x7f6da4071790 msgr2=0x7f6da40b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:12.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 -- 172.21.15.97:0/2140930677 shutdown_connections 2023-07-06T03:56:12.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:12.119+0000 7f6da97ff700 1 -- 172.21.15.97:0/2140930677 wait complete. 2023-07-06T03:56:12.139 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:56:12.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:56:12.140 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:56:12.140 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:56:12.141 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:12.141 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:56:12.141 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:56:12.141 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:56:12.142 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:56:12.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:12.142 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:56:12.189 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:56:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:56:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:56:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:56:12.189 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:12.190 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:56:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:56:12.190 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:56:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:12.190 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:56:12.190 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:56:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:12.191 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:56:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:56:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41497sec preferred_lft 41497sec 2023-07-06T03:56:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:56:12.191 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:12.191 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:56:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:56:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:56:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:12.192 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:56:12.192 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:56:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:56:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:12.193 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:56:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:56:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:56:12.193 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:12.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:12.195 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:56:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:56:12.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:12.264 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:56:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:56:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:56:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:56:12.312 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:56:12.313 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:56:12.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:12.313 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:56:12.314 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:56:12.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:56:12.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:56:12.315 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:56:12.315 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:56:12.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:12.481 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:56:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:56:12.527 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:56:12.527 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:56:12.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:12.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:56:12.636 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:56:12.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:56:12.637 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:12.682 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:56:12.683 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:56:12.683 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:12.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:56:12.877 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:56:12.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:56:12.945 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:56:12.947 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:56:12.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:12.948 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:56:12.994 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:56:12.994 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:56:13.038 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:56:13.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:56:13.111 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:56:13.179 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:56:13.181 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:56:13.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:13.182 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:56:13.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:56:13.203+0000 7f50946a25c0 -1 init, newargv = 0x556169468ed0 newargc=15 2023-07-06T03:56:13.206 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[167878]: starting ceph client 2023-07-06T03:56:13.212 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[167878]: starting fuse 2023-07-06T03:56:13.231 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:56:13.232 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:56:13.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:56:13.233 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:56:13.233 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:56:13.233 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:56:13.234 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:56:13.234 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:13.234 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:56:13.235 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:56:13.235 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:56:13.235 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:56:13.235 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:13.236 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:56:13.236 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:56:13.236 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:56:13.237 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:56:13.237 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:13.238 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:56:13.238 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:56:13.238 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:56:13.239 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:56:13.239 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:56:13.239 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:56:13.239 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:56:13.240 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:56:13.240 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:13.241 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:56:13.241 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:56:13.415 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.167878.asok 2023-07-06T03:56:13.593 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.167878.asok 2023-07-06T03:56:13.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:13.594 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.167878.asok status 2023-07-06T03:56:13.818 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:56:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:56:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:56:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:56:13.818 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:56:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:56:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:56:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "167878", 2023-07-06T03:56:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:56:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:56:13.819 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:56:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9792, 2023-07-06T03:56:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:56:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:56:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:56:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 9792 2023-07-06T03:56:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:13.820 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:56:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:56:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:56:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 809271656 2023-07-06T03:56:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:56:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:56:13.821 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:56:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 809271656 2023-07-06T03:56:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.9792 192.168.0.1:0/809271656", 2023-07-06T03:56:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/809271656", 2023-07-06T03:56:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:56:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 103, 2023-07-06T03:56:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 136, 2023-07-06T03:56:13.822 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:56:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:56:13.823 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:56:13.823 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:56:13.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:13.833 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:56:13.833 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:13.879 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:56:13.879 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:13.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:13.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:13.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:56:14.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.181+0000 7f869dfff700 1 Processor -- start 2023-07-06T03:56:14.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.181+0000 7f869dfff700 1 -- start start 2023-07-06T03:56:14.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.182+0000 7f869dfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86980cd8e0 0x7f869815e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:14.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f869dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8698070c30 0x7f86980adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:14.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f869dfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86980710a0 con 0x7f86980cdcb0 2023-07-06T03:56:14.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f869dfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86980ae3e0 con 0x7f86980cd8e0 2023-07-06T03:56:14.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f869dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8698071330 con 0x7f8698070c30 2023-07-06T03:56:14.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f86977fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86980cd8e0 0x7f869815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:14.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f86977fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86980cd8e0 0x7f869815e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59256/0 (socket says 172.21.15.97:59256) 2023-07-06T03:56:14.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f8697fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f86980cdcb0 0x7f869806f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35862/0 (socket says 172.21.15.97:35862) 2023-07-06T03:56:14.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f86977fe700 1 -- 172.21.15.97:0/4237658963 learned_addr learned my addr 172.21.15.97:0/4237658963 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:14.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f868ffff700 1 --2- 172.21.15.97:0/4237658963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8698070c30 0x7f86980adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:14.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f86977fe700 1 -- 172.21.15.97:0/4237658963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8698070c30 msgr2=0x7f86980adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:14.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f86977fe700 1 --2- 172.21.15.97:0/4237658963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8698070c30 0x7f86980adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:14.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f86977fe700 1 -- 172.21.15.97:0/4237658963 >> v1:172.21.15.97:6789/0 conn(0x7f86980cdcb0 legacy=0x7f869806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:14.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f86977fe700 1 -- 172.21.15.97:0/4237658963 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8698071610 con 0x7f86980cd8e0 2023-07-06T03:56:14.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.183+0000 7f868ffff700 1 --2- 172.21.15.97:0/4237658963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8698070c30 0x7f86980adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:56:14.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.184+0000 7f86977fe700 1 --2- 172.21.15.97:0/4237658963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd8e0 0x7f869815e860 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f86880098f0 tx=0x7f8688009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6644b180b6ecbcf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:14.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.184+0000 7f8696ffd700 1 -- 172.21.15.97:0/4237658963 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800b8a0 con 0x7f86980cd8e0 2023-07-06T03:56:14.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.184+0000 7f8696ffd700 1 -- 172.21.15.97:0/4237658963 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868800ba00 con 0x7f86980cd8e0 2023-07-06T03:56:14.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.184+0000 7f8696ffd700 1 -- 172.21.15.97:0/4237658963 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800bd90 con 0x7f86980cd8e0 2023-07-06T03:56:14.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.184+0000 7f869dfff700 1 -- 172.21.15.97:0/4237658963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd8e0 msgr2=0x7f869815e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:14.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.184+0000 7f869dfff700 1 --2- 172.21.15.97:0/4237658963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd8e0 0x7f869815e860 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f86880098f0 tx=0x7f8688009c00 comp rx=0 tx=0).stop 2023-07-06T03:56:14.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.184+0000 7f869dfff700 1 -- 172.21.15.97:0/4237658963 shutdown_connections 2023-07-06T03:56:14.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.184+0000 7f869dfff700 1 --2- 172.21.15.97:0/4237658963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd8e0 0x7f869815e860 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:14.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.184+0000 7f869dfff700 1 --2- 172.21.15.97:0/4237658963 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8698070c30 0x7f86980adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:14.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.184+0000 7f869dfff700 1 -- 172.21.15.97:0/4237658963 >> 172.21.15.97:0/4237658963 conn(0x7f86980ce8c0 msgr2=0x7f86980cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.184+0000 7f869dfff700 1 -- 172.21.15.97:0/4237658963 shutdown_connections 2023-07-06T03:56:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.185+0000 7f869dfff700 1 -- 172.21.15.97:0/4237658963 wait complete. 2023-07-06T03:56:14.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.186+0000 7f869dfff700 1 Processor -- start 2023-07-06T03:56:14.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.186+0000 7f869dfff700 1 -- start start 2023-07-06T03:56:14.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.187+0000 7f869dfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698070c30 0x7f86981f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:14.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.187+0000 7f869dfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd8e0 0x7f8698148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:14.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.187+0000 7f869dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86980cdcb0 0x7f86980af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:14.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.187+0000 7f869dfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8698148bf0 con 0x7f86980cd8e0 2023-07-06T03:56:14.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.187+0000 7f869dfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8698148ed0 con 0x7f8698070c30 2023-07-06T03:56:14.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.187+0000 7f869dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86981491b0 con 0x7f86980cdcb0 2023-07-06T03:56:14.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.187+0000 7f868ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd8e0 0x7f8698148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:14.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.187+0000 7f868ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd8e0 0x7f8698148170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59260/0 (socket says 172.21.15.97:59260) 2023-07-06T03:56:14.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.187+0000 7f868ffff700 1 -- 172.21.15.97:0/2245071349 learned_addr learned my addr 172.21.15.97:0/2245071349 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:14.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.187+0000 7f86977fe700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698070c30 0x7f86981f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:14.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.188+0000 7f8697fff700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86980cdcb0 0x7f86980af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:14.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.188+0000 7f868ffff700 1 -- 172.21.15.97:0/2245071349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86980cdcb0 msgr2=0x7f86980af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:14.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.188+0000 7f868ffff700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86980cdcb0 0x7f86980af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:14.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.188+0000 7f868ffff700 1 -- 172.21.15.97:0/2245071349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698070c30 msgr2=0x7f86981f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:14.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.188+0000 7f868ffff700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698070c30 0x7f86981f1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:14.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.188+0000 7f868ffff700 1 -- 172.21.15.97:0/2245071349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8688009550 con 0x7f86980cd8e0 2023-07-06T03:56:14.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.188+0000 7f868ffff700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd8e0 0x7f8698148170 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f868000c7f0 tx=0x7f868000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:14.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.188+0000 7f8694ff9700 1 -- 172.21.15.97:0/2245071349 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680011800 con 0x7f86980cd8e0 2023-07-06T03:56:14.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.188+0000 7f8694ff9700 1 -- 172.21.15.97:0/2245071349 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8680011960 con 0x7f86980cd8e0 2023-07-06T03:56:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.188+0000 7f869dfff700 1 -- 172.21.15.97:0/2245071349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8698149490 con 0x7f86980cd8e0 2023-07-06T03:56:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.188+0000 7f8694ff9700 1 -- 172.21.15.97:0/2245071349 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680011cf0 con 0x7f86980cd8e0 2023-07-06T03:56:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.189+0000 7f869dfff700 1 -- 172.21.15.97:0/2245071349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86980b0c20 con 0x7f86980cd8e0 2023-07-06T03:56:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.189+0000 7f869dfff700 1 -- 172.21.15.97:0/2245071349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8698070aa0 con 0x7f86980cd8e0 2023-07-06T03:56:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.190+0000 7f8694ff9700 1 -- 172.21.15.97:0/2245071349 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8680018360 con 0x7f86980cd8e0 2023-07-06T03:56:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.191+0000 7f8694ff9700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 0x7f867806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:14.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.191+0000 7f8694ff9700 1 -- 172.21.15.97:0/2245071349 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8680089350 con 0x7f86980cd8e0 2023-07-06T03:56:14.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.191+0000 7f86977fe700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 0x7f867806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:14.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.192+0000 7f86977fe700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 0x7f867806d190 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8688009c40 tx=0x7f8688008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:14.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.196+0000 7f8694ff9700 1 -- 172.21.15.97:0/2245071349 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86800543f0 con 0x7f86980cd8e0 2023-07-06T03:56:14.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.436+0000 7f869dfff700 1 -- 172.21.15.97:0/2245071349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f86981499b0 con 0x7f86980cd8e0 2023-07-06T03:56:14.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.436+0000 7f8694ff9700 1 -- 172.21.15.97:0/2245071349 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) v1 ==== 53+0+73 (secure 0 0 0) 0x7f86981499b0 con 0x7f86980cd8e0 2023-07-06T03:56:14.439 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:56:14.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.438+0000 7f869dfff700 1 -- 172.21.15.97:0/2245071349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 msgr2=0x7f867806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:14.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.438+0000 7f869dfff700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 0x7f867806d190 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8688009c40 tx=0x7f8688008030 comp rx=0 tx=0).stop 2023-07-06T03:56:14.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.438+0000 7f869dfff700 1 -- 172.21.15.97:0/2245071349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd8e0 msgr2=0x7f8698148170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:14.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.438+0000 7f869dfff700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd8e0 0x7f8698148170 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f868000c7f0 tx=0x7f868000cb00 comp rx=0 tx=0).stop 2023-07-06T03:56:14.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.439+0000 7f869dfff700 1 -- 172.21.15.97:0/2245071349 shutdown_connections 2023-07-06T03:56:14.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.439+0000 7f869dfff700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f867806ace0 0x7f867806d190 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:14.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.439+0000 7f869dfff700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f86980cdcb0 0x7f86980af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:14.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.439+0000 7f869dfff700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8698070c30 0x7f86981f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:14.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.439+0000 7f869dfff700 1 --2- 172.21.15.97:0/2245071349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86980cd8e0 0x7f8698148170 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:14.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.439+0000 7f869dfff700 1 -- 172.21.15.97:0/2245071349 >> 172.21.15.97:0/2245071349 conn(0x7f86980ce8c0 msgr2=0x7f86980b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:14.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.439+0000 7f869dfff700 1 -- 172.21.15.97:0/2245071349 shutdown_connections 2023-07-06T03:56:14.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:14.439+0000 7f869dfff700 1 -- 172.21.15.97:0/2245071349 wait complete. 2023-07-06T03:56:14.459 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:56:14.459 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:56:14.460 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2023-07-06T03:56:14.460 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T03:56:14.460 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T03:56:14.460 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:56:14.461 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:56:14.461 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:56:14.461 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:56:14.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:14.462 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2023-07-06T03:56:14.513 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:56:14.514 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:56:14.514 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:56:14.514 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:14.514 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2023-07-06T03:56:14.514 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:14.514 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:56:14.515 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:14.515 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f0 2023-07-06T03:56:14.515 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:56:14.515 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:14.515 INFO:teuthology.orchestra.run.smithi146.stdout: altname enp7s0f1 2023-07-06T03:56:14.515 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:56:14.515 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:14.515 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:56:14.516 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:14.516 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:56:14.516 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 41510sec preferred_lft 41510sec 2023-07-06T03:56:14.516 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2023-07-06T03:56:14.516 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:14.516 INFO:teuthology.orchestra.run.smithi146.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:56:14.516 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:14.517 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:56:14.517 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:14.517 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::58f0:6eff:fe04:3be0/64 scope link 2023-07-06T03:56:14.517 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:14.517 INFO:teuthology.orchestra.run.smithi146.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:56:14.517 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 12:6d:9c:d5:9d:0f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:56:14.517 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::106d:9cff:fed5:9d0f/64 scope link 2023-07-06T03:56:14.517 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:14.518 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:56:14.518 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether a2:c5:04:8e:a2:a7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T03:56:14.518 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::a0c5:4ff:fe8e:a2a7/64 scope link 2023-07-06T03:56:14.518 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:14.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:14.520 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:56:14.590 INFO:teuthology.orchestra.run.smithi146.stdout:1 2023-07-06T03:56:14.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:14.594 DEBUG:teuthology.orchestra.run.smithi146:> route 2023-07-06T03:56:14.643 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2023-07-06T03:56:14.643 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:56:14.643 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:56:14.644 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:56:14.644 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:56:14.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:14.645 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2023-07-06T03:56:14.645 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2023-07-06T03:56:14.646 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:56:14.646 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:56:14.647 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:56:14.647 DEBUG:teuthology.orchestra.run.smithi146:> ') 2023-07-06T03:56:14.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:14.815 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:56:14.862 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T03:56:14.862 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T03:56:14.863 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:56:14.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:14.864 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2023-07-06T03:56:14.934 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-06T03:56:14.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:56:14.935 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T03:56:14.980 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T03:56:14.981 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:56:14.981 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:56:15.065 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:56:15.175 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:56:15.176 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:56:15.243 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:56:15.246 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:56:15.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:15.248 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:56:15.294 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:56:15.295 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-06T03:56:15.340 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2023-07-06T03:56:15.409 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:56:15.410 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:56:15.477 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:56:15.480 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:56:15.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:15.481 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2023-07-06T03:56:15.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:2023-07-06T03:56:15.502+0000 7f25c289f5c0 -1 init, newargv = 0x55d1506bb170 newargc=15 2023-07-06T03:56:15.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[127880]: starting ceph client 2023-07-06T03:56:15.512 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[127880]: starting fuse 2023-07-06T03:56:15.528 INFO:teuthology.orchestra.run.smithi146.stdout:42 2023-07-06T03:56:15.529 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T03:56:15.529 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:56:15.530 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2023-07-06T03:56:15.530 DEBUG:teuthology.orchestra.run.smithi146:> import re 2023-07-06T03:56:15.530 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:56:15.531 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2023-07-06T03:56:15.531 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:56:15.531 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2023-07-06T03:56:15.532 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-06T03:56:15.532 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2023-07-06T03:56:15.532 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-06T03:56:15.533 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:56:15.533 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2023-07-06T03:56:15.533 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2023-07-06T03:56:15.533 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2023-07-06T03:56:15.534 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2023-07-06T03:56:15.534 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:56:15.534 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2023-07-06T03:56:15.535 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:56:15.535 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:56:15.535 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:56:15.535 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2023-07-06T03:56:15.536 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2023-07-06T03:56:15.536 DEBUG:teuthology.orchestra.run.smithi146:> return f 2023-07-06T03:56:15.536 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:56:15.537 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:56:15.537 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.1")) 2023-07-06T03:56:15.537 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:56:15.713 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.1.127880.asok 2023-07-06T03:56:15.892 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.127880.asok 2023-07-06T03:56:15.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:15.893 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.127880.asok status 2023-07-06T03:56:16.116 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2023-07-06T03:56:16.116 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2023-07-06T03:56:16.117 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:56:16.117 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:56:16.117 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "1", 2023-07-06T03:56:16.117 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2023-07-06T03:56:16.117 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-06T03:56:16.117 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "127880", 2023-07-06T03:56:16.117 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2023-07-06T03:56:16.118 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:56:16.118 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2023-07-06T03:56:16.118 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2023-07-06T03:56:16.118 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 8758, 2023-07-06T03:56:16.118 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2023-07-06T03:56:16.118 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2023-07-06T03:56:16.118 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2023-07-06T03:56:16.118 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 8758 2023-07-06T03:56:16.119 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:56:16.119 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:56:16.119 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:56:16.119 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:56:16.119 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1007163085 2023-07-06T03:56:16.119 INFO:teuthology.orchestra.run.smithi146.stdout: } 2023-07-06T03:56:16.119 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:56:16.119 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2023-07-06T03:56:16.120 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2023-07-06T03:56:16.120 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:56:16.120 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1007163085 2023-07-06T03:56:16.120 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2023-07-06T03:56:16.120 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.8758 192.168.0.1:0/1007163085", 2023-07-06T03:56:16.120 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/1007163085", 2023-07-06T03:56:16.120 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2023-07-06T03:56:16.121 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 103, 2023-07-06T03:56:16.121 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 136, 2023-07-06T03:56:16.121 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:56:16.121 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2023-07-06T03:56:16.121 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2023-07-06T03:56:16.121 INFO:teuthology.orchestra.run.smithi146.stdout:} 2023-07-06T03:56:16.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:16.131 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2023-07-06T03:56:16.131 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-06T03:56:16.177 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2023-07-06T03:56:16.178 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-06T03:56:16.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:16.179 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T03:56:16.248 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:56:16.253 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:56:16.371 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2023-07-06T03:56:16.372 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2023-07-06T03:56:16.372 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:56:16.372 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2023-07-06T03:56:16.372 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.372 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2023-07-06T03:56:16.372 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.373 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2023-07-06T03:56:16.373 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.373 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2023-07-06T03:56:16.373 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2023-07-06T03:56:16.373 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.373 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2023-07-06T03:56:16.373 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2023-07-06T03:56:16.374 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.374 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:56:16.374 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2023-07-06T03:56:16.374 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.374 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:56:16.374 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:56:16.374 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.374 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2023-07-06T03:56:16.375 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.375 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2023-07-06T03:56:16.375 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2023-07-06T03:56:16.375 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:56:16.375 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:56:16.375 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2023-07-06T03:56:16.375 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:56:16.375 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2023-07-06T03:56:16.376 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:56:16.376 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:56:16.376 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2023-07-06T03:56:16.376 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.376 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:56:16.376 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.376 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2023-07-06T03:56:16.376 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.377 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2023-07-06T03:56:16.377 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.377 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2023-07-06T03:56:16.377 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2023-07-06T03:56:16.377 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2023-07-06T03:56:16.377 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:56:16.377 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:56:16.378 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.378 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2023-07-06T03:56:16.378 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2023-07-06T03:56:16.378 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.378 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2023-07-06T03:56:16.378 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2023-07-06T03:56:16.378 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:56:16.378 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.379 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2023-07-06T03:56:16.379 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2023-07-06T03:56:16.379 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.379 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2023-07-06T03:56:16.379 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2023-07-06T03:56:16.379 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2023-07-06T03:56:16.379 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.379 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2023-07-06T03:56:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2023-07-06T03:56:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:56:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2023-07-06T03:56:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2023-07-06T03:56:16.380 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.381 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:56:16.381 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2023-07-06T03:56:16.381 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.381 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2023-07-06T03:56:16.381 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2023-07-06T03:56:16.381 INFO:teuthology.orchestra.run.smithi146.stdout: bdev async discard = True 2023-07-06T03:56:16.382 INFO:teuthology.orchestra.run.smithi146.stdout: bdev enable discard = True 2023-07-06T03:56:16.382 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore allocator = bitmap 2023-07-06T03:56:16.382 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2023-07-06T03:56:16.382 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2023-07-06T03:56:16.382 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 1/20 2023-07-06T03:56:16.383 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 1/20 2023-07-06T03:56:16.383 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:56:16.383 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2023-07-06T03:56:16.383 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 4/10 2023-07-06T03:56:16.383 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:56:16.383 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2023-07-06T03:56:16.383 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:56:16.383 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:56:16.384 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2023-07-06T03:56:16.384 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:56:16.384 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.384 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2023-07-06T03:56:16.384 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:56:16.384 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2023-07-06T03:56:16.384 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:56:16.384 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:56:16.385 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:56:16.385 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:56:16.385 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2023-07-06T03:56:16.385 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.385 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2023-07-06T03:56:16.385 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:56:16.385 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:56:16.385 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2023-07-06T03:56:16.386 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:56:16.386 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2023-07-06T03:56:16.386 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2023-07-06T03:56:16.386 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:56:16.386 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2023-07-06T03:56:16.386 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2023-07-06T03:56:16.386 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:56:16.386 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.387 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:56:16.387 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:56:16.387 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:56:16.387 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.387 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:56:16.387 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:56:16.387 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:56:16.388 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.388 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2023-07-06T03:56:16.388 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:56:16.388 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.388 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:56:16.388 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2023-07-06T03:56:16.388 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:56:16.388 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2023-07-06T03:56:16.389 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2023-07-06T03:56:16.389 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2023-07-06T03:56:16.389 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2023-07-06T03:56:16.389 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:56:16.389 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:56:16.389 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2023-07-06T03:56:16.389 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2023-07-06T03:56:16.389 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:56:16.390 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2023-07-06T03:56:16.390 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:56:16.390 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:56:16.390 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2023-07-06T03:56:16.390 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2023-07-06T03:56:16.390 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2023-07-06T03:56:16.391 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2023-07-06T03:56:16.391 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2023-07-06T03:56:16.391 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2023-07-06T03:56:16.391 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2023-07-06T03:56:16.391 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2023-07-06T03:56:16.391 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2023-07-06T03:56:16.391 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2023-07-06T03:56:16.391 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:56:16.392 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2023-07-06T03:56:16.392 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2023-07-06T03:56:16.392 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2023-07-06T03:56:16.392 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2023-07-06T03:56:16.393 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:56:16.393 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:56:16.393 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:56:16.393 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2023-07-06T03:56:16.393 INFO:teuthology.orchestra.run.smithi146.stdout: client inject release failure = true 2023-07-06T03:56:16.394 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2023-07-06T03:56:16.394 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2023-07-06T03:56:16.394 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:56:16.394 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2023-07-06T03:56:16.394 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.394 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2023-07-06T03:56:16.395 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.395 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2023-07-06T03:56:16.395 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.395 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2023-07-06T03:56:16.395 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2023-07-06T03:56:16.395 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.395 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2023-07-06T03:56:16.395 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2023-07-06T03:56:16.396 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.396 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:56:16.396 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2023-07-06T03:56:16.396 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.396 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:56:16.396 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:56:16.396 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.396 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2023-07-06T03:56:16.397 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.397 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2023-07-06T03:56:16.397 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2023-07-06T03:56:16.397 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:56:16.397 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:56:16.397 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2023-07-06T03:56:16.397 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:56:16.398 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2023-07-06T03:56:16.398 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:56:16.398 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:56:16.398 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2023-07-06T03:56:16.398 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.398 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:56:16.398 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.398 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2023-07-06T03:56:16.399 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.399 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2023-07-06T03:56:16.399 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.399 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2023-07-06T03:56:16.399 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2023-07-06T03:56:16.399 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2023-07-06T03:56:16.399 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:56:16.399 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:56:16.400 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.400 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2023-07-06T03:56:16.400 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2023-07-06T03:56:16.400 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.400 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2023-07-06T03:56:16.400 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2023-07-06T03:56:16.400 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:56:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2023-07-06T03:56:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2023-07-06T03:56:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2023-07-06T03:56:16.401 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2023-07-06T03:56:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2023-07-06T03:56:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2023-07-06T03:56:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2023-07-06T03:56:16.402 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:56:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2023-07-06T03:56:16.403 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2023-07-06T03:56:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:56:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2023-07-06T03:56:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2023-07-06T03:56:16.404 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2023-07-06T03:56:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2023-07-06T03:56:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2023-07-06T03:56:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2023-07-06T03:56:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2023-07-06T03:56:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2023-07-06T03:56:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2023-07-06T03:56:16.405 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2023-07-06T03:56:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:56:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2023-07-06T03:56:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2023-07-06T03:56:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:56:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2023-07-06T03:56:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:56:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:56:16.406 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2023-07-06T03:56:16.407 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:56:16.407 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.407 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2023-07-06T03:56:16.407 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:56:16.407 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2023-07-06T03:56:16.407 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:56:16.407 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:56:16.407 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:56:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:56:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2023-07-06T03:56:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.408 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2023-07-06T03:56:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:56:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:56:16.408 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2023-07-06T03:56:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:56:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2023-07-06T03:56:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2023-07-06T03:56:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:56:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2023-07-06T03:56:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2023-07-06T03:56:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:56:16.409 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:56:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:56:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:56:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:56:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:56:16.410 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:56:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2023-07-06T03:56:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:56:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:56:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2023-07-06T03:56:16.411 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.412 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2023-07-06T03:56:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2023-07-06T03:56:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2023-07-06T03:56:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2023-07-06T03:56:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:56:16.412 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:56:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2023-07-06T03:56:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2023-07-06T03:56:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:56:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2023-07-06T03:56:16.413 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:56:16.414 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:56:16.414 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2023-07-06T03:56:16.414 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2023-07-06T03:56:16.414 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2023-07-06T03:56:16.414 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-07-06T03:56:16.414 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-07-06T03:56:16.414 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-07-06T03:56:16.414 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-07-06T03:56:16.415 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2023-07-06T03:56:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2023-07-06T03:56:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2023-07-06T03:56:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:56:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2023-07-06T03:56:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2023-07-06T03:56:16.415 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2023-07-06T03:56:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2023-07-06T03:56:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:56:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:56:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:56:16.416 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:56:16.416 INFO:teuthology.orchestra.run.smithi097.stdout: client inject release failure = true 2023-07-06T03:56:16.418 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:56:16.418 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:56:16.464 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:56:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.695+0000 7fa9a79ff700 1 Processor -- start 2023-07-06T03:56:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.695+0000 7fa9a79ff700 1 -- start start 2023-07-06T03:56:16.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.696+0000 7fa9a79ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa9a00d09d0 0x7fa9a00b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:16.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.697+0000 7fa9a79ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a00cf0e0 con 0x7fa9a00cde60 2023-07-06T03:56:16.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.697+0000 7fa9a79ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a00ce280 con 0x7fa9a015ca10 2023-07-06T03:56:16.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.697+0000 7fa9a79ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a00cf570 con 0x7fa9a00d09d0 2023-07-06T03:56:16.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.697+0000 7fa9a5f9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa9a00cde60 0x7fa9a00afa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35878/0 (socket says 172.21.15.97:35878) 2023-07-06T03:56:16.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.697+0000 7fa9a5f9c700 1 -- 172.21.15.97:0/576717363 learned_addr learned my addr 172.21.15.97:0/576717363 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:16.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.697+0000 7fa9a579b700 1 --2- 172.21.15.97:0/576717363 >> v2:172.21.15.146:3300/0 conn(0x7fa9a00d09d0 0x7fa9a00b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:16.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.697+0000 7fa997fff700 1 -- 172.21.15.97:0/576717363 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159929906 0 0) 0x7fa9a00cf0e0 con 0x7fa9a00cde60 2023-07-06T03:56:16.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.697+0000 7fa997fff700 1 -- 172.21.15.97:0/576717363 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa98c0035b0 con 0x7fa9a00cde60 2023-07-06T03:56:16.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.697+0000 7fa9a579b700 1 -- 172.21.15.97:0/576717363 >> v1:172.21.15.146:6789/0 conn(0x7fa9a015ca10 legacy=0x7fa9a00d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:16.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.697+0000 7fa997fff700 1 -- 172.21.15.97:0/576717363 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099385393 0 0) 0x7fa9a00ce280 con 0x7fa9a015ca10 2023-07-06T03:56:16.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.697+0000 7fa9a579b700 1 -- 172.21.15.97:0/576717363 >> v1:172.21.15.97:6789/0 conn(0x7fa9a00cde60 legacy=0x7fa9a00afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:16.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.698+0000 7fa9a579b700 1 -- 172.21.15.97:0/576717363 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a00d3d00 con 0x7fa9a00d09d0 2023-07-06T03:56:16.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.698+0000 7fa9a579b700 1 --2- 172.21.15.97:0/576717363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9a00d09d0 0x7fa9a00b1dd0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009840 tx=0x7fa99c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d5f4461bd8950b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:16.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.698+0000 7fa997fff700 1 -- 172.21.15.97:0/576717363 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c011830 con 0x7fa9a00d09d0 2023-07-06T03:56:16.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.698+0000 7fa997fff700 1 -- 172.21.15.97:0/576717363 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c011990 con 0x7fa9a00d09d0 2023-07-06T03:56:16.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.698+0000 7fa997fff700 1 -- 172.21.15.97:0/576717363 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c011d20 con 0x7fa9a00d09d0 2023-07-06T03:56:16.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.699+0000 7fa9a79ff700 1 -- 172.21.15.97:0/576717363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9a00d09d0 msgr2=0x7fa9a00b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:16.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.699+0000 7fa9a79ff700 1 --2- 172.21.15.97:0/576717363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9a00d09d0 0x7fa9a00b1dd0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009840 tx=0x7fa99c009b50 comp rx=0 tx=0).stop 2023-07-06T03:56:16.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.699+0000 7fa9a79ff700 1 -- 172.21.15.97:0/576717363 shutdown_connections 2023-07-06T03:56:16.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.699+0000 7fa9a79ff700 1 --2- 172.21.15.97:0/576717363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9a00d09d0 0x7fa9a00b1dd0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:16.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.699+0000 7fa9a79ff700 1 -- 172.21.15.97:0/576717363 >> 172.21.15.97:0/576717363 conn(0x7fa9a00720b0 msgr2=0x7fa9a00d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:16.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.699+0000 7fa9a79ff700 1 -- 172.21.15.97:0/576717363 shutdown_connections 2023-07-06T03:56:16.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.699+0000 7fa9a79ff700 1 -- 172.21.15.97:0/576717363 wait complete. 2023-07-06T03:56:16.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.700+0000 7fa9a79ff700 1 Processor -- start 2023-07-06T03:56:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.700+0000 7fa9a79ff700 1 -- start start 2023-07-06T03:56:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.701+0000 7fa9a79ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a00cde60 0x7fa9a00b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.701+0000 7fa9a79ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9a00d09d0 0x7fa9a00b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.701+0000 7fa9a79ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9a0159120 0x7fa9a015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.701+0000 7fa9a79ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00b3000 con 0x7fa9a00cde60 2023-07-06T03:56:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.701+0000 7fa9a79ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00caf20 con 0x7fa9a0159120 2023-07-06T03:56:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.701+0000 7fa9a79ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00b3280 con 0x7fa9a00d09d0 2023-07-06T03:56:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.702+0000 7fa9a579b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a00cde60 0x7fa9a00b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.702+0000 7fa9a579b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a00cde60 0x7fa9a00b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59276/0 (socket says 172.21.15.97:59276) 2023-07-06T03:56:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.702+0000 7fa9a579b700 1 -- 172.21.15.97:0/229906009 learned_addr learned my addr 172.21.15.97:0/229906009 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.702+0000 7fa9a4f9a700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9a00d09d0 0x7fa9a00b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.702+0000 7fa9a5f9c700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9a0159120 0x7fa9a015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.702+0000 7fa9a579b700 1 -- 172.21.15.97:0/229906009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9a00d09d0 msgr2=0x7fa9a00b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.702+0000 7fa9a579b700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9a00d09d0 0x7fa9a00b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:16.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.702+0000 7fa9a579b700 1 -- 172.21.15.97:0/229906009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9a0159120 msgr2=0x7fa9a015b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:16.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.702+0000 7fa9a579b700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9a0159120 0x7fa9a015b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:16.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.702+0000 7fa9a579b700 1 -- 172.21.15.97:0/229906009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c0094a0 con 0x7fa9a00cde60 2023-07-06T03:56:16.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.702+0000 7fa9a579b700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a00cde60 0x7fa9a00b06b0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0090b0 tx=0x7fa99c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:16.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.703+0000 7fa995ffb700 1 -- 172.21.15.97:0/229906009 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c01a7e0 con 0x7fa9a00cde60 2023-07-06T03:56:16.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.703+0000 7fa995ffb700 1 -- 172.21.15.97:0/229906009 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c01a940 con 0x7fa9a00cde60 2023-07-06T03:56:16.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.703+0000 7fa9a79ff700 1 -- 172.21.15.97:0/229906009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a00b3560 con 0x7fa9a00cde60 2023-07-06T03:56:16.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.703+0000 7fa995ffb700 1 -- 172.21.15.97:0/229906009 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c01acd0 con 0x7fa9a00cde60 2023-07-06T03:56:16.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.703+0000 7fa9a79ff700 1 -- 172.21.15.97:0/229906009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a00cb1e0 con 0x7fa9a00cde60 2023-07-06T03:56:16.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.704+0000 7fa9a79ff700 1 -- 172.21.15.97:0/229906009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a004e340 con 0x7fa9a00cde60 2023-07-06T03:56:16.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.705+0000 7fa995ffb700 1 -- 172.21.15.97:0/229906009 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa99c022070 con 0x7fa9a00cde60 2023-07-06T03:56:16.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.706+0000 7fa995ffb700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa98406ace0 0x7fa98406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:16.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.706+0000 7fa995ffb700 1 -- 172.21.15.97:0/229906009 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa99c08a1d0 con 0x7fa9a00cde60 2023-07-06T03:56:16.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.706+0000 7fa9a4f9a700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa98406ace0 0x7fa98406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:16.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.706+0000 7fa9a4f9a700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa98406ace0 0x7fa98406d190 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa9a015a370 tx=0x7fa990009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:16.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.710+0000 7fa995ffb700 1 -- 172.21.15.97:0/229906009 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa99c055270 con 0x7fa9a00cde60 2023-07-06T03:56:16.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.908+0000 7fa9a79ff700 1 -- 172.21.15.97:0/229906009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9a00c98e0 con 0x7fa9a00cde60 2023-07-06T03:56:16.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.910+0000 7fa995ffb700 1 -- 172.21.15.97:0/229906009 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18391 (secure 0 0 0) 0x7fa99c015090 con 0x7fa9a00cde60 2023-07-06T03:56:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:16.913 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":136,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:56:06.083480+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:56:02.531370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-06T03:56:03.548148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:56:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 -- 172.21.15.97:0/229906009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa98406ace0 msgr2=0x7fa98406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa98406ace0 0x7fa98406d190 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa9a015a370 tx=0x7fa990009040 comp rx=0 tx=0).stop 2023-07-06T03:56:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 -- 172.21.15.97:0/229906009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a00cde60 msgr2=0x7fa9a00b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a00cde60 0x7fa9a00b06b0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0090b0 tx=0x7fa99c004750 comp rx=0 tx=0).stop 2023-07-06T03:56:16.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 -- 172.21.15.97:0/229906009 shutdown_connections 2023-07-06T03:56:16.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa98406ace0 0x7fa98406d190 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:16.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9a0159120 0x7fa9a015b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:16.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a00cde60 0x7fa9a00b06b0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:16.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 --2- 172.21.15.97:0/229906009 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa9a00d09d0 0x7fa9a00b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 -- 172.21.15.97:0/229906009 >> 172.21.15.97:0/229906009 conn(0x7fa9a00720b0 msgr2=0x7fa9a006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 -- 172.21.15.97:0/229906009 shutdown_connections 2023-07-06T03:56:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:16.912+0000 7fa9a79ff700 1 -- 172.21.15.97:0/229906009 wait complete. 2023-07-06T03:56:16.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:56:02.531370+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.75, 'score_stable': 2.75, 'optimal_score': 1, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:56:03.548148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:56:16.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:56:17.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.129+0000 7f3d11bff700 1 Processor -- start 2023-07-06T03:56:17.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.129+0000 7f3d11bff700 1 -- start start 2023-07-06T03:56:17.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.131+0000 7f3d11bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3d0c158ad0 0x7f3d0c0ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d0c0adb70 0x7f3d0c06c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3d0c06c630 0x7f3d0c0c3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0c0cc590 con 0x7f3d0c158ad0 2023-07-06T03:56:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0c158ea0 con 0x7f3d0c06c630 2023-07-06T03:56:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c06cdc0 con 0x7f3d0c0adb70 2023-07-06T03:56:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3d0c158ad0 0x7f3d0c0ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3d0c158ad0 0x7f3d0c0ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59278/0 (socket says 172.21.15.97:59278) 2023-07-06T03:56:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0b7fe700 1 -- 172.21.15.97:0/109838524 learned_addr learned my addr 172.21.15.97:0/109838524 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0bfff700 1 --2- 172.21.15.97:0/109838524 >> v2:172.21.15.146:3300/0 conn(0x7f3d0c06c630 0x7f3d0c0c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0affd700 1 --2- 172.21.15.97:0/109838524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d0c0adb70 0x7f3d0c06c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0b7fe700 1 -- 172.21.15.97:0/109838524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d0c0adb70 msgr2=0x7f3d0c06c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0b7fe700 1 --2- 172.21.15.97:0/109838524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d0c0adb70 0x7f3d0c06c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0b7fe700 1 -- 172.21.15.97:0/109838524 >> v2:172.21.15.146:3300/0 conn(0x7f3d0c06c630 msgr2=0x7f3d0c0c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0b7fe700 1 --2- 172.21.15.97:0/109838524 >> v2:172.21.15.146:3300/0 conn(0x7f3d0c06c630 0x7f3d0c0c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0b7fe700 1 -- 172.21.15.97:0/109838524 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c06d0a0 con 0x7f3d0c158ad0 2023-07-06T03:56:17.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0b7fe700 1 --2- 172.21.15.97:0/109838524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c158ad0 0x7f3d0c0ce200 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0adb20 tx=0x7f3cfc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2d63fd534bfd29 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0a7fc700 1 -- 172.21.15.97:0/109838524 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00b870 con 0x7f3d0c158ad0 2023-07-06T03:56:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0a7fc700 1 -- 172.21.15.97:0/109838524 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc00b9d0 con 0x7f3d0c158ad0 2023-07-06T03:56:17.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d0a7fc700 1 -- 172.21.15.97:0/109838524 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00bd60 con 0x7f3d0c158ad0 2023-07-06T03:56:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 -- 172.21.15.97:0/109838524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c158ad0 msgr2=0x7f3d0c0ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 --2- 172.21.15.97:0/109838524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c158ad0 0x7f3d0c0ce200 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0adb20 tx=0x7f3cfc009bd0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 -- 172.21.15.97:0/109838524 shutdown_connections 2023-07-06T03:56:17.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 --2- 172.21.15.97:0/109838524 >> v2:172.21.15.146:3300/0 conn(0x7f3d0c06c630 0x7f3d0c0c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 --2- 172.21.15.97:0/109838524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c158ad0 0x7f3d0c0ce200 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 --2- 172.21.15.97:0/109838524 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d0c0adb70 0x7f3d0c06c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 -- 172.21.15.97:0/109838524 >> 172.21.15.97:0/109838524 conn(0x7f3d0c06a060 msgr2=0x7f3d0c06a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:17.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 -- 172.21.15.97:0/109838524 shutdown_connections 2023-07-06T03:56:17.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.132+0000 7f3d11bff700 1 -- 172.21.15.97:0/109838524 wait complete. 2023-07-06T03:56:17.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.134+0000 7f3d11bff700 1 Processor -- start 2023-07-06T03:56:17.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.134+0000 7f3d11bff700 1 -- start start 2023-07-06T03:56:17.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.136+0000 7f3d11bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06c630 0x7f3d0c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.136+0000 7f3d11bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d0c0adb70 0x7f3d0c0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.136+0000 7f3d11bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d0c158ad0 0x7f3d0c0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.136+0000 7f3d11bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c0ca580 con 0x7f3d0c06c630 2023-07-06T03:56:17.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.136+0000 7f3d11bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c1e5ba0 con 0x7f3d0c0adb70 2023-07-06T03:56:17.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.136+0000 7f3d11bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c0c9ec0 con 0x7f3d0c158ad0 2023-07-06T03:56:17.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.136+0000 7f3d0b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06c630 0x7f3d0c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.136+0000 7f3d0b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06c630 0x7f3d0c1539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59294/0 (socket says 172.21.15.97:59294) 2023-07-06T03:56:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.136+0000 7f3d0b7fe700 1 -- 172.21.15.97:0/3351881745 learned_addr learned my addr 172.21.15.97:0/3351881745 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3d0affd700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d0c0adb70 0x7f3d0c0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3d0bfff700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d0c158ad0 0x7f3d0c0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3d0b7fe700 1 -- 172.21.15.97:0/3351881745 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d0c158ad0 msgr2=0x7f3d0c0aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3d0b7fe700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d0c158ad0 0x7f3d0c0aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3d0b7fe700 1 -- 172.21.15.97:0/3351881745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d0c0adb70 msgr2=0x7f3d0c0ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3d0b7fe700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d0c0adb70 0x7f3d0c0ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3d0b7fe700 1 -- 172.21.15.97:0/3351881745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc009550 con 0x7f3d0c06c630 2023-07-06T03:56:17.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3d0b7fe700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06c630 0x7f3d0c1539f0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:17.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3cf3fff700 1 -- 172.21.15.97:0/3351881745 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc004520 con 0x7f3d0c06c630 2023-07-06T03:56:17.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3cf3fff700 1 -- 172.21.15.97:0/3351881745 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc01e070 con 0x7f3d0c06c630 2023-07-06T03:56:17.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3cf3fff700 1 -- 172.21.15.97:0/3351881745 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0038c0 con 0x7f3d0c06c630 2023-07-06T03:56:17.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3d11bff700 1 -- 172.21.15.97:0/3351881745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0c0ca1a0 con 0x7f3d0c06c630 2023-07-06T03:56:17.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.137+0000 7f3d11bff700 1 -- 172.21.15.97:0/3351881745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0c0c9b70 con 0x7f3d0c06c630 2023-07-06T03:56:17.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.138+0000 7f3d11bff700 1 -- 172.21.15.97:0/3351881745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0c04e340 con 0x7f3d0c06c630 2023-07-06T03:56:17.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.138+0000 7f3cf3fff700 1 -- 172.21.15.97:0/3351881745 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3cfc003a20 con 0x7f3d0c06c630 2023-07-06T03:56:17.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.139+0000 7f3cf3fff700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3cf806ace0 0x7f3cf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.139+0000 7f3cf3fff700 1 -- 172.21.15.97:0/3351881745 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3cfc020070 con 0x7f3d0c06c630 2023-07-06T03:56:17.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.139+0000 7f3d0affd700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3cf806ace0 0x7f3cf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.140+0000 7f3d0affd700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3cf806ace0 0x7f3cf806d190 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0cb640 tx=0x7f3d00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:17.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.144+0000 7f3cf3fff700 1 -- 172.21.15.97:0/3351881745 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3cfc055af0 con 0x7f3d0c06c630 2023-07-06T03:56:17.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.340+0000 7f3d11bff700 1 -- 172.21.15.97:0/3351881745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d0c0ca7f0 con 0x7f3d0c06c630 2023-07-06T03:56:17.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.340+0000 7f3cf3fff700 1 -- 172.21.15.97:0/3351881745 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3cfc059110 con 0x7f3d0c06c630 2023-07-06T03:56:17.342 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:56:17.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.342+0000 7f3d11bff700 1 -- 172.21.15.97:0/3351881745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3cf806ace0 msgr2=0x7f3cf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.343+0000 7f3d11bff700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3cf806ace0 0x7f3cf806d190 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0cb640 tx=0x7f3d00008040 comp rx=0 tx=0).stop 2023-07-06T03:56:17.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.343+0000 7f3d11bff700 1 -- 172.21.15.97:0/3351881745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06c630 msgr2=0x7f3d0c1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.343+0000 7f3d11bff700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06c630 0x7f3d0c1539f0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc004750 comp rx=0 tx=0).stop 2023-07-06T03:56:17.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.343+0000 7f3d11bff700 1 -- 172.21.15.97:0/3351881745 shutdown_connections 2023-07-06T03:56:17.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.343+0000 7f3d11bff700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3cf806ace0 0x7f3cf806d190 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.343+0000 7f3d11bff700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3d0c158ad0 0x7f3d0c0aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.343+0000 7f3d11bff700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06c630 0x7f3d0c1539f0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.343+0000 7f3d11bff700 1 --2- 172.21.15.97:0/3351881745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d0c0adb70 0x7f3d0c0ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.343+0000 7f3d11bff700 1 -- 172.21.15.97:0/3351881745 >> 172.21.15.97:0/3351881745 conn(0x7f3d0c06a060 msgr2=0x7f3d0c0ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:17.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.343+0000 7f3d11bff700 1 -- 172.21.15.97:0/3351881745 shutdown_connections 2023-07-06T03:56:17.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.343+0000 7f3d11bff700 1 -- 172.21.15.97:0/3351881745 wait complete. 2023-07-06T03:56:17.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:56:17.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.558+0000 7ff1ea5ff700 1 Processor -- start 2023-07-06T03:56:17.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.558+0000 7ff1ea5ff700 1 -- start start 2023-07-06T03:56:17.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.559+0000 7ff1ea5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff1e40cf7e0 0x7ff1e40d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.559+0000 7ff1ea5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff1e40cd490 0x7ff1e40b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1ea5ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e406b7f0 con 0x7ff1e406b380 2023-07-06T03:56:17.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1ea5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e40d2460 con 0x7ff1e40cd490 2023-07-06T03:56:17.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1ea5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e40d1950 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1e8b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff1e40cd490 0x7ff1e40b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1e8b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff1e40cd490 0x7ff1e40b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59298/0 (socket says 172.21.15.97:59298) 2023-07-06T03:56:17.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1e8b9c700 1 -- 172.21.15.97:0/3166293493 learned_addr learned my addr 172.21.15.97:0/3166293493 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:17.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1e3fff700 1 --2- 172.21.15.97:0/3166293493 >> v2:172.21.15.146:3300/0 conn(0x7ff1e40cf7e0 0x7ff1e40d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1e8b9c700 1 -- 172.21.15.97:0/3166293493 >> v2:172.21.15.146:3300/0 conn(0x7ff1e40cf7e0 msgr2=0x7ff1e40d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:17.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1e8b9c700 1 --2- 172.21.15.97:0/3166293493 >> v2:172.21.15.146:3300/0 conn(0x7ff1e40cf7e0 0x7ff1e40d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1e8b9c700 1 -- 172.21.15.97:0/3166293493 >> v1:172.21.15.146:6789/0 conn(0x7ff1e406b380 legacy=0x7ff1e40cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:17.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1e8b9c700 1 -- 172.21.15.97:0/3166293493 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40d2670 con 0x7ff1e40cd490 2023-07-06T03:56:17.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.560+0000 7ff1e8b9c700 1 --2- 172.21.15.97:0/3166293493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e40cd490 0x7ff1e40b5c20 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40b59c0 tx=0x7ff1dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eead5e51b57accd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:17.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.561+0000 7ff1e2ffd700 1 -- 172.21.15.97:0/3166293493 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00e070 con 0x7ff1e40cd490 2023-07-06T03:56:17.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.561+0000 7ff1e2ffd700 1 -- 172.21.15.97:0/3166293493 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc003a20 con 0x7ff1e40cd490 2023-07-06T03:56:17.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.561+0000 7ff1e2ffd700 1 -- 172.21.15.97:0/3166293493 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc003dd0 con 0x7ff1e40cd490 2023-07-06T03:56:17.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.561+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/3166293493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e40cd490 msgr2=0x7ff1e40b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.561+0000 7ff1ea5ff700 1 --2- 172.21.15.97:0/3166293493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e40cd490 0x7ff1e40b5c20 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40b59c0 tx=0x7ff1dc00b330 comp rx=0 tx=0).stop 2023-07-06T03:56:17.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.561+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/3166293493 shutdown_connections 2023-07-06T03:56:17.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.561+0000 7ff1ea5ff700 1 --2- 172.21.15.97:0/3166293493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e40cd490 0x7ff1e40b5c20 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.561+0000 7ff1ea5ff700 1 --2- 172.21.15.97:0/3166293493 >> v2:172.21.15.146:3300/0 conn(0x7ff1e40cf7e0 0x7ff1e40d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.561+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/3166293493 >> 172.21.15.97:0/3166293493 conn(0x7ff1e406cf30 msgr2=0x7ff1e406a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:17.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.561+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/3166293493 shutdown_connections 2023-07-06T03:56:17.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.561+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/3166293493 wait complete. 2023-07-06T03:56:17.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.563+0000 7ff1ea5ff700 1 Processor -- start 2023-07-06T03:56:17.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.563+0000 7ff1ea5ff700 1 -- start start 2023-07-06T03:56:17.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.564+0000 7ff1ea5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1e406b380 0x7ff1e41f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.564+0000 7ff1ea5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1e40cd490 0x7ff1e41f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.564+0000 7ff1ea5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e40cf7e0 0x7ff1e4152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.564+0000 7ff1ea5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40b6c50 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.564+0000 7ff1ea5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e41f3450 con 0x7ff1e406b380 2023-07-06T03:56:17.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.564+0000 7ff1ea5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e4155dc0 con 0x7ff1e40cd490 2023-07-06T03:56:17.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.564+0000 7ff1e8b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e40cf7e0 0x7ff1e4152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.564+0000 7ff1e8b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e40cf7e0 0x7ff1e4152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59308/0 (socket says 172.21.15.97:59308) 2023-07-06T03:56:17.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.564+0000 7ff1e8b9c700 1 -- 172.21.15.97:0/1450595621 learned_addr learned my addr 172.21.15.97:0/1450595621 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:17.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.564+0000 7ff1e37fe700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1e40cd490 0x7ff1e41f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.564+0000 7ff1e3fff700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1e406b380 0x7ff1e41f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.565+0000 7ff1e8b9c700 1 -- 172.21.15.97:0/1450595621 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1e40cd490 msgr2=0x7ff1e41f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.565+0000 7ff1e8b9c700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1e40cd490 0x7ff1e41f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.565+0000 7ff1e8b9c700 1 -- 172.21.15.97:0/1450595621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1e406b380 msgr2=0x7ff1e41f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.565+0000 7ff1e8b9c700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1e406b380 0x7ff1e41f28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.565+0000 7ff1e8b9c700 1 -- 172.21.15.97:0/1450595621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc009b30 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.565+0000 7ff1e8b9c700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e40cf7e0 0x7ff1e4152c30 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc015040 tx=0x7ff1dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:17.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.565+0000 7ff1e0ff9700 1 -- 172.21.15.97:0/1450595621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc004440 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.565+0000 7ff1e0ff9700 1 -- 172.21.15.97:0/1450595621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc0045c0 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.565+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/1450595621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e40b6ea0 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.565+0000 7ff1e0ff9700 1 -- 172.21.15.97:0/1450595621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc004970 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.565+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/1450595621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e40b7460 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.567+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/1450595621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e40b7680 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.567+0000 7ff1e0ff9700 1 -- 172.21.15.97:0/1450595621 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1dc012430 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.568+0000 7ff1e0ff9700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1d006ace0 0x7ff1d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.568+0000 7ff1e0ff9700 1 -- 172.21.15.97:0/1450595621 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1dc08a7d0 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.569+0000 7ff1e3fff700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1d006ace0 0x7ff1d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.569+0000 7ff1e3fff700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1d006ace0 0x7ff1d006d190 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4152a50 tx=0x7ff1d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:17.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.573+0000 7ff1e0ff9700 1 -- 172.21.15.97:0/1450595621 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff1dc0557f0 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.767+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/1450595621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff1e406b7f0 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.767+0000 7ff1e0ff9700 1 -- 172.21.15.97:0/1450595621 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v136) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff1e406b7f0 con 0x7ff1e40cf7e0 2023-07-06T03:56:17.769 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:56:17.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.769+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/1450595621 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1d006ace0 msgr2=0x7ff1d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.769+0000 7ff1ea5ff700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1d006ace0 0x7ff1d006d190 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4152a50 tx=0x7ff1d4008040 comp rx=0 tx=0).stop 2023-07-06T03:56:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.769+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/1450595621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e40cf7e0 msgr2=0x7ff1e4152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.769+0000 7ff1ea5ff700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e40cf7e0 0x7ff1e4152c30 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc015040 tx=0x7ff1dc004ad0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.769+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/1450595621 shutdown_connections 2023-07-06T03:56:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.769+0000 7ff1ea5ff700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff1d006ace0 0x7ff1d006d190 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.769+0000 7ff1ea5ff700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1e40cf7e0 0x7ff1e4152c30 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.769+0000 7ff1ea5ff700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1e406b380 0x7ff1e41f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.769+0000 7ff1ea5ff700 1 --2- 172.21.15.97:0/1450595621 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff1e40cd490 0x7ff1e41f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.769+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/1450595621 >> 172.21.15.97:0/1450595621 conn(0x7ff1e406cf30 msgr2=0x7ff1e40c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:17.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.770+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/1450595621 shutdown_connections 2023-07-06T03:56:17.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.770+0000 7ff1ea5ff700 1 -- 172.21.15.97:0/1450595621 wait complete. 2023-07-06T03:56:17.789 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:56:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.987+0000 7ff61cdff700 1 Processor -- start 2023-07-06T03:56:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.987+0000 7ff61cdff700 1 -- start start 2023-07-06T03:56:17.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.988+0000 7ff61cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180cf7e0 0x7ff6180d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.988+0000 7ff61cdff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61806b7f0 con 0x7ff61806b380 2023-07-06T03:56:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.988+0000 7ff61cdff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6180d1950 con 0x7ff6180cd490 2023-07-06T03:56:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.988+0000 7ff61cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180d2460 con 0x7ff6180cf7e0 2023-07-06T03:56:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.988+0000 7ff60ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff61806b380 0x7ff6180cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35892/0 (socket says 172.21.15.97:35892) 2023-07-06T03:56:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.988+0000 7ff60ffff700 1 -- 172.21.15.97:0/2452603672 learned_addr learned my addr 172.21.15.97:0/2452603672 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740194692 0 0) 0x7ff600001110 con 0x7ff61806b380 2023-07-06T03:56:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6040035b0 con 0x7ff61806b380 2023-07-06T03:56:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff61659c700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180cf7e0 0x7ff6180d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1224155480 0 0) 0x7ff6040035b0 con 0x7ff61806b380 2023-07-06T03:56:17.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180cf7e0 msgr2=0x7ff6180d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:17.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180cf7e0 0x7ff6180d1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 >> v1:172.21.15.146:6789/0 conn(0x7ff6180cd490 legacy=0x7ff6180b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6180d2670 con 0x7ff61806b380 2023-07-06T03:56:17.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7ff600001800 con 0x7ff61806b380 2023-07-06T03:56:17.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 >> v1:172.21.15.97:6789/0 conn(0x7ff61806b380 legacy=0x7ff6180cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180cf7e0 0x7ff604003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff604003f70 0x7ff604004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff604004d90 0x7ff604005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:17.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180d2460 con 0x7ff604004d90 2023-07-06T03:56:17.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60400ba40 con 0x7ff604003f70 2023-07-06T03:56:17.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60400bc00 con 0x7ff6180cf7e0 2023-07-06T03:56:17.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.989+0000 7ff616d9d700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff604004d90 0x7ff604005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.990+0000 7ff61659c700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180cf7e0 0x7ff604003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.990+0000 7ff60ffff700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff604003f70 0x7ff604004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.990+0000 7ff616d9d700 1 -- 172.21.15.97:0/2452603672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180cf7e0 msgr2=0x7ff604003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.990+0000 7ff616d9d700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff6180cf7e0 0x7ff604003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.990+0000 7ff616d9d700 1 -- 172.21.15.97:0/2452603672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff604003f70 msgr2=0x7ff604004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.990+0000 7ff616d9d700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff604003f70 0x7ff604004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.990+0000 7ff616d9d700 1 -- 172.21.15.97:0/2452603672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6040035b0 con 0x7ff604004d90 2023-07-06T03:56:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.990+0000 7ff616d9d700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff604004d90 0x7ff604005200 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7ff61000c290 tx=0x7ff604005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79405cb5fc0db33a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.990+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610001e70 con 0x7ff604004d90 2023-07-06T03:56:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.990+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61000ce50 con 0x7ff604004d90 2023-07-06T03:56:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.990+0000 7ff615d9b700 1 -- 172.21.15.97:0/2452603672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610012660 con 0x7ff604004d90 2023-07-06T03:56:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.991+0000 7ff61cdff700 1 -- 172.21.15.97:0/2452603672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff604004d90 msgr2=0x7ff604005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.991+0000 7ff61cdff700 1 --2- 172.21.15.97:0/2452603672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff604004d90 0x7ff604005200 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7ff61000c290 tx=0x7ff604005880 comp rx=0 tx=0).stop 2023-07-06T03:56:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.991+0000 7ff616d9d700 1 -- 172.21.15.97:0/2452603672 reap_dead start 2023-07-06T03:56:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.991+0000 7ff61cdff700 1 -- 172.21.15.97:0/2452603672 shutdown_connections 2023-07-06T03:56:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.991+0000 7ff61cdff700 1 -- 172.21.15.97:0/2452603672 >> 172.21.15.97:0/2452603672 conn(0x7ff61806cf30 msgr2=0x7ff61806a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:17.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.991+0000 7ff61cdff700 1 -- 172.21.15.97:0/2452603672 shutdown_connections 2023-07-06T03:56:17.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.991+0000 7ff61cdff700 1 -- 172.21.15.97:0/2452603672 wait complete. 2023-07-06T03:56:18.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.993+0000 7ff61cdff700 1 Processor -- start 2023-07-06T03:56:18.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.993+0000 7ff61cdff700 1 -- start start 2023-07-06T03:56:18.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff61cdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff604004d90 0x7ff61815d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff61cdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6181f24f0 0x7ff618151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff61cdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6180b66b0 0x7ff6180b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff61cdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180cacb0 con 0x7ff6181f24f0 2023-07-06T03:56:18.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff61cdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180b7070 con 0x7ff6180b66b0 2023-07-06T03:56:18.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff61cdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61815ddc0 con 0x7ff604004d90 2023-07-06T03:56:18.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff60ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6181f24f0 0x7ff618151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff60ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6181f24f0 0x7ff618151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59328/0 (socket says 172.21.15.97:59328) 2023-07-06T03:56:18.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff60ffff700 1 -- 172.21.15.97:0/2763963817 learned_addr learned my addr 172.21.15.97:0/2763963817 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:18.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff61659c700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff604004d90 0x7ff61815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff616d9d700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6180b66b0 0x7ff6180b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff60ffff700 1 -- 172.21.15.97:0/2763963817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff604004d90 msgr2=0x7ff61815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff60ffff700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff604004d90 0x7ff61815d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff60ffff700 1 -- 172.21.15.97:0/2763963817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6180b66b0 msgr2=0x7ff6180b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff60ffff700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6180b66b0 0x7ff6180b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff60ffff700 1 -- 172.21.15.97:0/2763963817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61815f610 con 0x7ff6181f24f0 2023-07-06T03:56:18.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff61659c700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff604004d90 0x7ff61815d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:18.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.994+0000 7ff616d9d700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6180b66b0 0x7ff6180b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:18.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.995+0000 7ff60ffff700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6181f24f0 0x7ff618151710 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7ff6000072c0 tx=0x7ff600007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:18.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.995+0000 7ff60effd700 1 -- 172.21.15.97:0/2763963817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6000077a0 con 0x7ff6181f24f0 2023-07-06T03:56:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.995+0000 7ff60effd700 1 -- 172.21.15.97:0/2763963817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60000f870 con 0x7ff6181f24f0 2023-07-06T03:56:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.995+0000 7ff61cdff700 1 -- 172.21.15.97:0/2763963817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6181f2a20 con 0x7ff6181f24f0 2023-07-06T03:56:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.995+0000 7ff60effd700 1 -- 172.21.15.97:0/2763963817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000fc20 con 0x7ff6181f24f0 2023-07-06T03:56:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.995+0000 7ff61cdff700 1 -- 172.21.15.97:0/2763963817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6181f2fe0 con 0x7ff6181f24f0 2023-07-06T03:56:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.996+0000 7ff61cdff700 1 -- 172.21.15.97:0/2763963817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6180cae20 con 0x7ff6181f24f0 2023-07-06T03:56:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.997+0000 7ff60effd700 1 -- 172.21.15.97:0/2763963817 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff60000fd80 con 0x7ff6181f24f0 2023-07-06T03:56:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.998+0000 7ff60effd700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5f806ace0 0x7ff5f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.998+0000 7ff60effd700 1 -- 172.21.15.97:0/2763963817 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff600089470 con 0x7ff6181f24f0 2023-07-06T03:56:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.998+0000 7ff61659c700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5f806ace0 0x7ff5f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:17.999+0000 7ff61659c700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5f806ace0 0x7ff5f806d190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff6180734e0 tx=0x7ff608008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.002+0000 7ff60effd700 1 -- 172.21.15.97:0/2763963817 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff6180cae20 con 0x7ff6181f24f0 2023-07-06T03:56:18.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.198+0000 7ff61cdff700 1 -- 172.21.15.97:0/2763963817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff61806b7f0 con 0x7ff6181f24f0 2023-07-06T03:56:18.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.198+0000 7ff60effd700 1 -- 172.21.15.97:0/2763963817 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v136) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff61806b7f0 con 0x7ff6181f24f0 2023-07-06T03:56:18.200 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:56:18.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 -- 172.21.15.97:0/2763963817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5f806ace0 msgr2=0x7ff5f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5f806ace0 0x7ff5f806d190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff6180734e0 tx=0x7ff608008040 comp rx=0 tx=0).stop 2023-07-06T03:56:18.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 -- 172.21.15.97:0/2763963817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6181f24f0 msgr2=0x7ff618151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6181f24f0 0x7ff618151710 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7ff6000072c0 tx=0x7ff600007680 comp rx=0 tx=0).stop 2023-07-06T03:56:18.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 -- 172.21.15.97:0/2763963817 shutdown_connections 2023-07-06T03:56:18.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff5f806ace0 0x7ff5f806d190 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6180b66b0 0x7ff6180b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff604004d90 0x7ff61815d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 --2- 172.21.15.97:0/2763963817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6181f24f0 0x7ff618151710 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 -- 172.21.15.97:0/2763963817 >> 172.21.15.97:0/2763963817 conn(0x7ff61806cf30 msgr2=0x7ff6180d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:18.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 -- 172.21.15.97:0/2763963817 shutdown_connections 2023-07-06T03:56:18.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.201+0000 7ff61cdff700 1 -- 172.21.15.97:0/2763963817 wait complete. 2023-07-06T03:56:18.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:56:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.417+0000 7faff8bff700 1 Processor -- start 2023-07-06T03:56:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.417+0000 7faff8bff700 1 -- start start 2023-07-06T03:56:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.418+0000 7faff8bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faff415a3c0 0x7faff40cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.418+0000 7faff8bff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7faff406cce0 0x7faff40cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.418+0000 7faff8bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faff40cdae0 0x7faff40cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.418+0000 7faff8bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff406d430 con 0x7faff415a3c0 2023-07-06T03:56:18.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.418+0000 7faff8bff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff40ce450 con 0x7faff406cce0 2023-07-06T03:56:18.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.418+0000 7faff8bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff415a790 con 0x7faff40cdae0 2023-07-06T03:56:18.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.418+0000 7faff259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faff415a3c0 0x7faff40cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faff415a3c0 0x7faff40cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59338/0 (socket says 172.21.15.97:59338) 2023-07-06T03:56:18.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff259c700 1 -- 172.21.15.97:0/3776397409 learned_addr learned my addr 172.21.15.97:0/3776397409 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff1d9b700 1 --2- 172.21.15.97:0/3776397409 >> v2:172.21.15.146:3300/0 conn(0x7faff406cce0 0x7faff40cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff2d9d700 1 --2- 172.21.15.97:0/3776397409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faff40cdae0 0x7faff40cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff259c700 1 -- 172.21.15.97:0/3776397409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faff40cdae0 msgr2=0x7faff40cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff259c700 1 --2- 172.21.15.97:0/3776397409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faff40cdae0 0x7faff40cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff259c700 1 -- 172.21.15.97:0/3776397409 >> v2:172.21.15.146:3300/0 conn(0x7faff406cce0 msgr2=0x7faff40cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff259c700 1 --2- 172.21.15.97:0/3776397409 >> v2:172.21.15.146:3300/0 conn(0x7faff406cce0 0x7faff40cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff259c700 1 -- 172.21.15.97:0/3776397409 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff40adf30 con 0x7faff415a3c0 2023-07-06T03:56:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff259c700 1 --2- 172.21.15.97:0/3776397409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff415a3c0 0x7faff40cf210 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7faff40c9940 tx=0x7fafe4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70e7b34b33df1a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:18.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff159a700 1 -- 172.21.15.97:0/3776397409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe400b870 con 0x7faff415a3c0 2023-07-06T03:56:18.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff159a700 1 -- 172.21.15.97:0/3776397409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe400b9d0 con 0x7faff415a3c0 2023-07-06T03:56:18.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.419+0000 7faff159a700 1 -- 172.21.15.97:0/3776397409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe400bd60 con 0x7faff415a3c0 2023-07-06T03:56:18.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.420+0000 7faff8bff700 1 -- 172.21.15.97:0/3776397409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff415a3c0 msgr2=0x7faff40cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.420+0000 7faff8bff700 1 --2- 172.21.15.97:0/3776397409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff415a3c0 0x7faff40cf210 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7faff40c9940 tx=0x7fafe4009bd0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.420+0000 7faff8bff700 1 -- 172.21.15.97:0/3776397409 shutdown_connections 2023-07-06T03:56:18.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.420+0000 7faff8bff700 1 --2- 172.21.15.97:0/3776397409 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faff40cdae0 0x7faff40cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.420+0000 7faff8bff700 1 --2- 172.21.15.97:0/3776397409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff415a3c0 0x7faff40cf210 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.420+0000 7faff8bff700 1 --2- 172.21.15.97:0/3776397409 >> v2:172.21.15.146:3300/0 conn(0x7faff406cce0 0x7faff40cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.420+0000 7faff8bff700 1 -- 172.21.15.97:0/3776397409 >> 172.21.15.97:0/3776397409 conn(0x7faff406a060 msgr2=0x7faff406a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:18.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.420+0000 7faff8bff700 1 -- 172.21.15.97:0/3776397409 shutdown_connections 2023-07-06T03:56:18.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.420+0000 7faff8bff700 1 -- 172.21.15.97:0/3776397409 wait complete. 2023-07-06T03:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.422+0000 7faff8bff700 1 Processor -- start 2023-07-06T03:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.422+0000 7faff8bff700 1 -- start start 2023-07-06T03:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.423+0000 7faff8bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff406cce0 0x7faff414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.423+0000 7faff8bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff40cdae0 0x7faff40b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.423+0000 7faff8bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faff415a3c0 0x7faff40cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.423+0000 7faff8bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff41573b0 con 0x7faff406cce0 2023-07-06T03:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.423+0000 7faff8bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff414ea70 con 0x7faff40cdae0 2023-07-06T03:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.423+0000 7faff8bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff41567a0 con 0x7faff415a3c0 2023-07-06T03:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.423+0000 7faff259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff406cce0 0x7faff414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.423+0000 7faff259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff406cce0 0x7faff414e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59350/0 (socket says 172.21.15.97:59350) 2023-07-06T03:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.423+0000 7faff259c700 1 -- 172.21.15.97:0/4234612588 learned_addr learned my addr 172.21.15.97:0/4234612588 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.423+0000 7faff1d9b700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff40cdae0 0x7faff40b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.423+0000 7faff2d9d700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faff415a3c0 0x7faff40cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.424+0000 7faff259c700 1 -- 172.21.15.97:0/4234612588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faff415a3c0 msgr2=0x7faff40cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.424+0000 7faff259c700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faff415a3c0 0x7faff40cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.424+0000 7faff259c700 1 -- 172.21.15.97:0/4234612588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff40cdae0 msgr2=0x7faff40b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.424+0000 7faff259c700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff40cdae0 0x7faff40b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.424+0000 7faff259c700 1 -- 172.21.15.97:0/4234612588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe4009550 con 0x7faff406cce0 2023-07-06T03:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.424+0000 7faff259c700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff406cce0 0x7faff414e530 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fafe4000c00 tx=0x7fafe4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.424+0000 7fafe2ffd700 1 -- 172.21.15.97:0/4234612588 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4004310 con 0x7faff406cce0 2023-07-06T03:56:18.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.424+0000 7fafe2ffd700 1 -- 172.21.15.97:0/4234612588 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe401e070 con 0x7faff406cce0 2023-07-06T03:56:18.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.424+0000 7fafe2ffd700 1 -- 172.21.15.97:0/4234612588 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe40036a0 con 0x7faff406cce0 2023-07-06T03:56:18.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.424+0000 7faff8bff700 1 -- 172.21.15.97:0/4234612588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff4156a80 con 0x7faff406cce0 2023-07-06T03:56:18.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.425+0000 7faff8bff700 1 -- 172.21.15.97:0/4234612588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff4157040 con 0x7faff406cce0 2023-07-06T03:56:18.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.425+0000 7faff8bff700 1 -- 172.21.15.97:0/4234612588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff404e340 con 0x7faff406cce0 2023-07-06T03:56:18.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.426+0000 7fafe2ffd700 1 -- 172.21.15.97:0/4234612588 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fafe4020030 con 0x7faff406cce0 2023-07-06T03:56:18.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.427+0000 7fafe2ffd700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafd406ace0 0x7fafd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.427+0000 7fafe2ffd700 1 -- 172.21.15.97:0/4234612588 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fafe408a280 con 0x7faff406cce0 2023-07-06T03:56:18.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.428+0000 7faff1d9b700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafd406ace0 0x7fafd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.428+0000 7faff1d9b700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafd406ace0 0x7fafd406d190 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7faff40b3600 tx=0x7fafe8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.432+0000 7fafe2ffd700 1 -- 172.21.15.97:0/4234612588 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fafe40553c0 con 0x7faff406cce0 2023-07-06T03:56:18.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.626+0000 7faff8bff700 1 -- 172.21.15.97:0/4234612588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faff406c5b0 con 0x7faff406cce0 2023-07-06T03:56:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.626+0000 7fafe2ffd700 1 -- 172.21.15.97:0/4234612588 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v136) v1 ==== 100+0+11 (secure 0 0 0) 0x7fafe40589e0 con 0x7faff406cce0 2023-07-06T03:56:18.628 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:56:18.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.628+0000 7faff8bff700 1 -- 172.21.15.97:0/4234612588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafd406ace0 msgr2=0x7fafd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.628+0000 7faff8bff700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafd406ace0 0x7fafd406d190 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7faff40b3600 tx=0x7fafe8008040 comp rx=0 tx=0).stop 2023-07-06T03:56:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.628+0000 7faff8bff700 1 -- 172.21.15.97:0/4234612588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff406cce0 msgr2=0x7faff414e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.628+0000 7faff8bff700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff406cce0 0x7faff414e530 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fafe4000c00 tx=0x7fafe4004750 comp rx=0 tx=0).stop 2023-07-06T03:56:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.629+0000 7faff8bff700 1 -- 172.21.15.97:0/4234612588 shutdown_connections 2023-07-06T03:56:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.629+0000 7faff8bff700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fafd406ace0 0x7fafd406d190 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.629+0000 7faff8bff700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7faff415a3c0 0x7faff40cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.629+0000 7faff8bff700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faff406cce0 0x7faff414e530 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.629+0000 7faff8bff700 1 --2- 172.21.15.97:0/4234612588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faff40cdae0 0x7faff40b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.629+0000 7faff8bff700 1 -- 172.21.15.97:0/4234612588 >> 172.21.15.97:0/4234612588 conn(0x7faff406a060 msgr2=0x7faff40c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.629+0000 7faff8bff700 1 -- 172.21.15.97:0/4234612588 shutdown_connections 2023-07-06T03:56:18.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.629+0000 7faff8bff700 1 -- 172.21.15.97:0/4234612588 wait complete. 2023-07-06T03:56:18.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:56:18.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.844+0000 7f15137ff700 1 Processor -- start 2023-07-06T03:56:18.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.844+0000 7f15137ff700 1 -- start start 2023-07-06T03:56:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.845+0000 7f15137ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f150c0d09d0 0x7f150c0b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.845+0000 7f15137ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f150c15ca10 0x7f150c0cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.845+0000 7f15137ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150c0cf0e0 con 0x7f150c0d0da0 2023-07-06T03:56:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.845+0000 7f15137ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c0cf570 con 0x7f150c0d09d0 2023-07-06T03:56:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.845+0000 7f15137ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c0d3d00 con 0x7f150c15ca10 2023-07-06T03:56:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.845+0000 7f151159b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f150c0d09d0 0x7f150c0b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.845+0000 7f151159b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f150c0d09d0 0x7f150c0b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59358/0 (socket says 172.21.15.97:59358) 2023-07-06T03:56:18.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.845+0000 7f151159b700 1 -- 172.21.15.97:0/178938110 learned_addr learned my addr 172.21.15.97:0/178938110 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:18.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.846+0000 7f151159b700 1 -- 172.21.15.97:0/178938110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f150c15ca10 msgr2=0x7f150c0cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:18.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.846+0000 7f1510d9a700 1 --2- 172.21.15.97:0/178938110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f150c15ca10 0x7f150c0cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.846+0000 7f151159b700 1 --2- 172.21.15.97:0/178938110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f150c15ca10 0x7f150c0cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.846+0000 7f151159b700 1 -- 172.21.15.97:0/178938110 >> v1:172.21.15.146:6789/0 conn(0x7f150c0d0da0 legacy=0x7f150c0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:18.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.846+0000 7f151159b700 1 -- 172.21.15.97:0/178938110 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c0ce4d0 con 0x7f150c0d09d0 2023-07-06T03:56:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.846+0000 7f1510d9a700 1 --2- 172.21.15.97:0/178938110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f150c15ca10 0x7f150c0cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:56:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.846+0000 7f151159b700 1 --2- 172.21.15.97:0/178938110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c0d09d0 0x7f150c0b1dd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f15080098f0 tx=0x7f1508009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc00e2a6cc69526 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.846+0000 7f1503fff700 1 -- 172.21.15.97:0/178938110 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508004230 con 0x7f150c0d09d0 2023-07-06T03:56:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.846+0000 7f1503fff700 1 -- 172.21.15.97:0/178938110 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1508004390 con 0x7f150c0d09d0 2023-07-06T03:56:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.846+0000 7f1503fff700 1 -- 172.21.15.97:0/178938110 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15080036a0 con 0x7f150c0d09d0 2023-07-06T03:56:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.847+0000 7f15137ff700 1 -- 172.21.15.97:0/178938110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c0d09d0 msgr2=0x7f150c0b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.847+0000 7f15137ff700 1 --2- 172.21.15.97:0/178938110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c0d09d0 0x7f150c0b1dd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f15080098f0 tx=0x7f1508009c00 comp rx=0 tx=0).stop 2023-07-06T03:56:18.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.847+0000 7f15137ff700 1 -- 172.21.15.97:0/178938110 shutdown_connections 2023-07-06T03:56:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.847+0000 7f15137ff700 1 --2- 172.21.15.97:0/178938110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c0d09d0 0x7f150c0b1dd0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.847+0000 7f15137ff700 1 --2- 172.21.15.97:0/178938110 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f150c15ca10 0x7f150c0cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.847+0000 7f15137ff700 1 -- 172.21.15.97:0/178938110 >> 172.21.15.97:0/178938110 conn(0x7f150c0720b0 msgr2=0x7f150c0d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.847+0000 7f15137ff700 1 -- 172.21.15.97:0/178938110 shutdown_connections 2023-07-06T03:56:18.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.847+0000 7f15137ff700 1 -- 172.21.15.97:0/178938110 wait complete. 2023-07-06T03:56:18.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.848+0000 7f15137ff700 1 Processor -- start 2023-07-06T03:56:18.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.849+0000 7f15137ff700 1 -- start start 2023-07-06T03:56:18.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.849+0000 7f15137ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0d09d0 0x7f150c0c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.849+0000 7f15137ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c0d0da0 0x7f150c15d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.849+0000 7f15137ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f150c1561a0 0x7f150c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.849+0000 7f15137ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c1f2400 con 0x7f150c0d0da0 2023-07-06T03:56:18.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.849+0000 7f15137ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c15e8d0 con 0x7f150c0d09d0 2023-07-06T03:56:18.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.849+0000 7f15137ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c15e030 con 0x7f150c1561a0 2023-07-06T03:56:18.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.850+0000 7f1510d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c0d0da0 0x7f150c15d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.850+0000 7f1510d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c0d0da0 0x7f150c15d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59372/0 (socket says 172.21.15.97:59372) 2023-07-06T03:56:18.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.850+0000 7f1510d9a700 1 -- 172.21.15.97:0/1082149174 learned_addr learned my addr 172.21.15.97:0/1082149174 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:18.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.850+0000 7f151159b700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0d09d0 0x7f150c0c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.850+0000 7f1511d9c700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f150c1561a0 0x7f150c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.850+0000 7f1510d9a700 1 -- 172.21.15.97:0/1082149174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f150c1561a0 msgr2=0x7f150c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.850+0000 7f1510d9a700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f150c1561a0 0x7f150c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.850+0000 7f1510d9a700 1 -- 172.21.15.97:0/1082149174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0d09d0 msgr2=0x7f150c0c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.850+0000 7f1510d9a700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0d09d0 0x7f150c0c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.850+0000 7f1510d9a700 1 -- 172.21.15.97:0/1082149174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508009550 con 0x7f150c0d0da0 2023-07-06T03:56:18.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.850+0000 7f1510d9a700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c0d0da0 0x7f150c15d800 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00c750 tx=0x7f14fc00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.851+0000 7f1501ffb700 1 -- 172.21.15.97:0/1082149174 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc011800 con 0x7f150c0d0da0 2023-07-06T03:56:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.851+0000 7f1501ffb700 1 -- 172.21.15.97:0/1082149174 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc011960 con 0x7f150c0d0da0 2023-07-06T03:56:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.851+0000 7f15137ff700 1 -- 172.21.15.97:0/1082149174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c15e310 con 0x7f150c0d0da0 2023-07-06T03:56:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.851+0000 7f1501ffb700 1 -- 172.21.15.97:0/1082149174 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc011cd0 con 0x7f150c0d0da0 2023-07-06T03:56:18.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.851+0000 7f15137ff700 1 -- 172.21.15.97:0/1082149174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c15dd40 con 0x7f150c0d0da0 2023-07-06T03:56:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.852+0000 7f15137ff700 1 -- 172.21.15.97:0/1082149174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c1f21d0 con 0x7f150c0d0da0 2023-07-06T03:56:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.853+0000 7f1501ffb700 1 -- 172.21.15.97:0/1082149174 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f14fc021350 con 0x7f150c0d0da0 2023-07-06T03:56:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.854+0000 7f1501ffb700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f14f006ace0 0x7f14f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.854+0000 7f1501ffb700 1 -- 172.21.15.97:0/1082149174 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f14fc089320 con 0x7f150c0d0da0 2023-07-06T03:56:18.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.854+0000 7f151159b700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f14f006ace0 0x7f14f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:18.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.855+0000 7f151159b700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f14f006ace0 0x7f14f006d190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1508009130 tx=0x7f1508008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:18.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:18.858+0000 7f1501ffb700 1 -- 172.21.15.97:0/1082149174 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f150c1f21d0 con 0x7f150c0d0da0 2023-07-06T03:56:19.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.070+0000 7f15137ff700 1 -- 172.21.15.97:0/1082149174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f150c1f21d0 con 0x7f150c0d0da0 2023-07-06T03:56:19.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.071+0000 7f1501ffb700 1 -- 172.21.15.97:0/1082149174 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v136) v1 ==== 80+0+0 (secure 0 0 0) 0x7f150c1f21d0 con 0x7f150c0d0da0 2023-07-06T03:56:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 -- 172.21.15.97:0/1082149174 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f14f006ace0 msgr2=0x7f14f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f14f006ace0 0x7f14f006d190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1508009130 tx=0x7f1508008030 comp rx=0 tx=0).stop 2023-07-06T03:56:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 -- 172.21.15.97:0/1082149174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c0d0da0 msgr2=0x7f150c15d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c0d0da0 0x7f150c15d800 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00c750 tx=0x7f14fc00ca60 comp rx=0 tx=0).stop 2023-07-06T03:56:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 -- 172.21.15.97:0/1082149174 shutdown_connections 2023-07-06T03:56:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f14f006ace0 0x7f14f006d190 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f150c1561a0 0x7f150c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f150c0d09d0 0x7f150c0c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 --2- 172.21.15.97:0/1082149174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f150c0d0da0 0x7f150c15d800 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 -- 172.21.15.97:0/1082149174 >> 172.21.15.97:0/1082149174 conn(0x7f150c0720b0 msgr2=0x7f150c15bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 -- 172.21.15.97:0/1082149174 shutdown_connections 2023-07-06T03:56:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.073+0000 7f15137ff700 1 -- 172.21.15.97:0/1082149174 wait complete. 2023-07-06T03:56:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:56:19.093 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:56:19.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:19.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:19.159 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[167878]: fuse finished with error 0 and tester_r 0 2023-07-06T03:56:19.162 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:56:19.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:19.164 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:56:19.211 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T03:56:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.443+0000 7f86a47ff700 1 Processor -- start 2023-07-06T03:56:19.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.443+0000 7f86a47ff700 1 -- start start 2023-07-06T03:56:19.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.444+0000 7f86a47ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f869c153a70 0x7f869c0c9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:19.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.444+0000 7f86a47ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869c0c9ac0 con 0x7f869c0706a0 2023-07-06T03:56:19.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.444+0000 7f86a47ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869c0c9da0 con 0x7f869c0c9560 2023-07-06T03:56:19.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.444+0000 7f86a47ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869c0ca080 con 0x7f869c153a70 2023-07-06T03:56:19.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a2d9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f869c0706a0 0x7f869c0ad4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35902/0 (socket says 172.21.15.97:35902) 2023-07-06T03:56:19.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a2d9c700 1 -- 172.21.15.97:0/2079180426 learned_addr learned my addr 172.21.15.97:0/2079180426 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418981636 0 0) 0x7f869c0c9ac0 con 0x7f869c0706a0 2023-07-06T03:56:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86880035b0 con 0x7f869c0706a0 2023-07-06T03:56:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a259b700 1 --2- 172.21.15.97:0/2079180426 >> v2:172.21.15.146:3300/0 conn(0x7f869c153a70 0x7f869c0c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:19.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2124133937 0 0) 0x7f86880035b0 con 0x7f869c0706a0 2023-07-06T03:56:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 >> v2:172.21.15.146:3300/0 conn(0x7f869c153a70 msgr2=0x7f869c0c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 --2- 172.21.15.97:0/2079180426 >> v2:172.21.15.146:3300/0 conn(0x7f869c153a70 0x7f869c0c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 >> v1:172.21.15.146:6789/0 conn(0x7f869c0c9560 legacy=0x7f869c06ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0ca360 con 0x7f869c0706a0 2023-07-06T03:56:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f86940019f0 con 0x7f869c0706a0 2023-07-06T03:56:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 >> v1:172.21.15.97:6789/0 conn(0x7f869c0706a0 legacy=0x7f869c0ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:19.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8688003a00 0x7f8688003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8688004850 0x7f8688004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688005200 0x7f868800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0ca080 con 0x7f8688005200 2023-07-06T03:56:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868800c190 con 0x7f8688003a00 2023-07-06T03:56:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.445+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868800c350 con 0x7f8688004850 2023-07-06T03:56:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.446+0000 7f86a259b700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8688003a00 0x7f8688003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.446+0000 7f86a2d9c700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688005200 0x7f868800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:19.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.446+0000 7f86a1d9a700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8688004850 0x7f8688004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.446+0000 7f86a259b700 1 -- 172.21.15.97:0/2079180426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8688004850 msgr2=0x7f8688004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.446+0000 7f86a259b700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8688004850 0x7f8688004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.446+0000 7f86a259b700 1 -- 172.21.15.97:0/2079180426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688005200 msgr2=0x7f868800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.446+0000 7f86a259b700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688005200 0x7f868800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.446+0000 7f86a259b700 1 -- 172.21.15.97:0/2079180426 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86880035b0 con 0x7f8688003a00 2023-07-06T03:56:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.446+0000 7f86a2d9c700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688005200 0x7f868800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:56:19.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.446+0000 7f86a2d9c700 1 -- 172.21.15.97:0/2079180426 reap_dead start 2023-07-06T03:56:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.446+0000 7f86a259b700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8688003a00 0x7f8688003e10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8698009860 tx=0x7f8688005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6ae50b6a1b4148a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.447+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869800c070 con 0x7f8688003a00 2023-07-06T03:56:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.447+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8698013610 con 0x7f8688003a00 2023-07-06T03:56:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.447+0000 7f86a1599700 1 -- 172.21.15.97:0/2079180426 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980139a0 con 0x7f8688003a00 2023-07-06T03:56:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.447+0000 7f86a47ff700 1 -- 172.21.15.97:0/2079180426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8688003a00 msgr2=0x7f8688003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.447+0000 7f86a47ff700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8688003a00 0x7f8688003e10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8698009860 tx=0x7f8688005880 comp rx=0 tx=0).stop 2023-07-06T03:56:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.447+0000 7f86a47ff700 1 -- 172.21.15.97:0/2079180426 shutdown_connections 2023-07-06T03:56:19.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.447+0000 7f86a47ff700 1 --2- 172.21.15.97:0/2079180426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8688003a00 0x7f8688003e10 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.447+0000 7f86a47ff700 1 -- 172.21.15.97:0/2079180426 >> 172.21.15.97:0/2079180426 conn(0x7f869c0cc0e0 msgr2=0x7f869c0ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:19.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.447+0000 7f86a47ff700 1 -- 172.21.15.97:0/2079180426 shutdown_connections 2023-07-06T03:56:19.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.447+0000 7f86a47ff700 1 -- 172.21.15.97:0/2079180426 wait complete. 2023-07-06T03:56:19.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.449+0000 7f86a47ff700 1 Processor -- start 2023-07-06T03:56:19.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.449+0000 7f86a47ff700 1 -- start start 2023-07-06T03:56:19.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.450+0000 7f86a47ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688003a00 0x7f869c152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:19.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.450+0000 7f86a47ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f869c0af810 0x7f869c0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:19.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.450+0000 7f86a47ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f869c0c8450 0x7f869c0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:19.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.450+0000 7f86a47ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0c66e0 con 0x7f8688003a00 2023-07-06T03:56:19.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.450+0000 7f86a47ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0b23c0 con 0x7f869c0af810 2023-07-06T03:56:19.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.450+0000 7f86a47ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0b26a0 con 0x7f869c0c8450 2023-07-06T03:56:19.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.450+0000 7f86a259b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688003a00 0x7f869c152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:19.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.450+0000 7f86a259b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688003a00 0x7f869c152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59388/0 (socket says 172.21.15.97:59388) 2023-07-06T03:56:19.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.450+0000 7f86a259b700 1 -- 172.21.15.97:0/2958540768 learned_addr learned my addr 172.21.15.97:0/2958540768 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f86a2d9c700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f869c0c8450 0x7f869c0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f86a1d9a700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f869c0af810 0x7f869c0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f86a259b700 1 -- 172.21.15.97:0/2958540768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f869c0c8450 msgr2=0x7f869c0c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f86a259b700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f869c0c8450 0x7f869c0c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f86a259b700 1 -- 172.21.15.97:0/2958540768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f869c0af810 msgr2=0x7f869c0c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f86a259b700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f869c0af810 0x7f869c0c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f86a259b700 1 -- 172.21.15.97:0/2958540768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0b2980 con 0x7f8688003a00 2023-07-06T03:56:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f86a259b700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688003a00 0x7f869c152360 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f868800c320 tx=0x7f86980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f8692ffd700 1 -- 172.21.15.97:0/2958540768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698004460 con 0x7f8688003a00 2023-07-06T03:56:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f8692ffd700 1 -- 172.21.15.97:0/2958540768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86980045c0 con 0x7f8688003a00 2023-07-06T03:56:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f86a47ff700 1 -- 172.21.15.97:0/2958540768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869c0c88f0 con 0x7f8688003a00 2023-07-06T03:56:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f8692ffd700 1 -- 172.21.15.97:0/2958540768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980038c0 con 0x7f8688003a00 2023-07-06T03:56:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.451+0000 7f86a47ff700 1 -- 172.21.15.97:0/2958540768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869c153110 con 0x7f8688003a00 2023-07-06T03:56:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.452+0000 7f86a47ff700 1 -- 172.21.15.97:0/2958540768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869c0b2cf0 con 0x7f8688003a00 2023-07-06T03:56:19.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.453+0000 7f8692ffd700 1 -- 172.21.15.97:0/2958540768 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8698003a20 con 0x7f8688003a00 2023-07-06T03:56:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.454+0000 7f8692ffd700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868006ace0 0x7f868006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.454+0000 7f8692ffd700 1 -- 172.21.15.97:0/2958540768 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f869808a640 con 0x7f8688003a00 2023-07-06T03:56:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.454+0000 7f86a1d9a700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868006ace0 0x7f868006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:19.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.455+0000 7f86a1d9a700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868006ace0 0x7f868006d190 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f869c159410 tx=0x7f868c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:19.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.458+0000 7f8692ffd700 1 -- 172.21.15.97:0/2958540768 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86980556e0 con 0x7f8688003a00 2023-07-06T03:56:19.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.698+0000 7f86a47ff700 1 -- 172.21.15.97:0/2958540768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f869c0b2b10 con 0x7f8688003a00 2023-07-06T03:56:19.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.698+0000 7f8692ffd700 1 -- 172.21.15.97:0/2958540768 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8698058d00 con 0x7f8688003a00 2023-07-06T03:56:19.700 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T03:56:19.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 -- 172.21.15.97:0/2958540768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868006ace0 msgr2=0x7f868006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:19.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868006ace0 0x7f868006d190 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f869c159410 tx=0x7f868c009040 comp rx=0 tx=0).stop 2023-07-06T03:56:19.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 -- 172.21.15.97:0/2958540768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688003a00 msgr2=0x7f869c152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:19.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688003a00 0x7f869c152360 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f868800c320 tx=0x7f86980047c0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 -- 172.21.15.97:0/2958540768 shutdown_connections 2023-07-06T03:56:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f868006ace0 0x7f868006d190 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f869c0c8450 0x7f869c0c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8688003a00 0x7f869c152360 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 --2- 172.21.15.97:0/2958540768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f869c0af810 0x7f869c0c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 -- 172.21.15.97:0/2958540768 >> 172.21.15.97:0/2958540768 conn(0x7f869c0cc0e0 msgr2=0x7f869c06d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 -- 172.21.15.97:0/2958540768 shutdown_connections 2023-07-06T03:56:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:19.700+0000 7f86a47ff700 1 -- 172.21.15.97:0/2958540768 wait complete. 2023-07-06T03:56:19.720 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T03:56:19.720 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T03:56:19.720 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-07-06T03:56:19.721 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T03:56:19.721 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:19.721 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T03:56:19.721 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T03:56:19.722 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T03:56:19.722 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:56:19.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:19.723 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-07-06T03:56:19.768 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T03:56:19.769 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T03:56:19.769 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T03:56:19.769 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:19.769 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-07-06T03:56:19.770 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:19.770 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:56:19.771 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:19.771 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-07-06T03:56:19.771 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:56:19.771 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:19.771 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-07-06T03:56:19.771 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T03:56:19.771 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:19.772 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T03:56:19.772 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:19.772 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T03:56:19.772 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41489sec preferred_lft 41489sec 2023-07-06T03:56:19.772 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-07-06T03:56:19.772 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:19.772 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T03:56:19.773 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff 2023-07-06T03:56:19.773 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T03:56:19.773 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:19.773 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c84d:f5ff:fe06:1957/64 scope link 2023-07-06T03:56:19.773 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:19.773 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:56:19.773 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:e8:c8:a7:f7:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:56:19.773 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2ce8:c8ff:fea7:f7f9/64 scope link 2023-07-06T03:56:19.774 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:19.774 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T03:56:19.774 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f6:cd:24:c1:e6:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:56:19.774 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f4cd:24ff:fec1:e6a3/64 scope link 2023-07-06T03:56:19.774 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-07-06T03:56:19.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:19.776 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T03:56:19.883 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-07-06T03:56:19.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:19.886 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-07-06T03:56:19.934 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-07-06T03:56:19.935 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T03:56:19.935 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T03:56:19.935 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T03:56:19.935 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T03:56:19.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:19.936 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-07-06T03:56:19.936 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-07-06T03:56:19.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T03:56:19.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T03:56:19.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T03:56:19.937 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-07-06T03:56:20.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:20.105 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:56:20.150 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:56:20.151 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:56:20.152 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:56:20.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:20.153 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-07-06T03:56:20.222 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T03:56:20.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:56:20.223 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:20.268 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T03:56:20.268 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T03:56:20.269 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:20.354 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:56:20.463 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:56:20.464 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:56:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:56:20.534 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:56:20.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:20.535 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:56:20.581 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T03:56:20.581 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T03:56:20.625 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2023-07-06T03:56:20.694 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T03:56:20.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T03:56:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T03:56:20.765 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T03:56:20.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:20.766 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2023-07-06T03:56:20.796 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2023-07-06T03:56:20.794+0000 7f1fb7a7c5c0 -1 init, newargv = 0x5587ee81e070 newargc=15 2023-07-06T03:56:20.796 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[168891]: starting ceph client 2023-07-06T03:56:20.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[168891]: starting fuse 2023-07-06T03:56:20.817 INFO:teuthology.orchestra.run.smithi097.stdout:43 2023-07-06T03:56:20.817 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T03:56:20.818 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:56:20.818 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:56:20.819 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:56:20.819 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:56:20.819 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:56:20.819 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:20.820 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:56:20.820 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:56:20.820 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:56:20.821 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:56:20.821 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:20.821 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:56:20.822 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:56:20.822 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:56:20.823 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:56:20.823 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:20.825 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:56:20.825 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:56:20.825 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:56:20.826 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:56:20.826 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:56:20.827 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:56:20.827 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:56:20.827 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:56:20.828 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:20.828 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:56:20.828 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:56:21.001 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.168891.asok 2023-07-06T03:56:21.179 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.168891.asok 2023-07-06T03:56:21.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:21.180 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.168891.asok status 2023-07-06T03:56:21.405 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:56:21.405 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2023-07-06T03:56:21.405 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "79365bb3b803780276d69efb2cc4afe5b160de9e", 2023-07-06T03:56:21.405 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4742-g79365bb3 (79365bb3b803780276d69efb2cc4afe5b160de9e) reef (dev)", 2023-07-06T03:56:21.405 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2023-07-06T03:56:21.406 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2023-07-06T03:56:21.406 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T03:56:21.406 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "168891", 2023-07-06T03:56:21.406 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2023-07-06T03:56:21.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:21.406 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2023-07-06T03:56:21.406 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2023-07-06T03:56:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9855, 2023-07-06T03:56:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:56:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:56:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:56:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 9855 2023-07-06T03:56:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:21.407 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:56:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:56:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:56:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 929081145 2023-07-06T03:56:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:56:21.408 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:56:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:56:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 929081145 2023-07-06T03:56:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.9855 192.168.0.1:0/929081145", 2023-07-06T03:56:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/929081145", 2023-07-06T03:56:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2023-07-06T03:56:21.409 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 103, 2023-07-06T03:56:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 136, 2023-07-06T03:56:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2023-07-06T03:56:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2023-07-06T03:56:21.410 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2023-07-06T03:56:21.410 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:56:21.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:21.419 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2023-07-06T03:56:21.419 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:21.465 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2023-07-06T03:56:21.466 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:21.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:21.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T03:56:21.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:56:21.576 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2023-07-06T03:56:21.576 DEBUG:teuthology.orchestra.run.smithi097:> import re 2023-07-06T03:56:21.577 DEBUG:teuthology.orchestra.run.smithi097:> import os 2023-07-06T03:56:21.577 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2023-07-06T03:56:21.577 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:21.577 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2023-07-06T03:56:21.578 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T03:56:21.578 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2023-07-06T03:56:21.579 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T03:56:21.579 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:21.579 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2023-07-06T03:56:21.579 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2023-07-06T03:56:21.580 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2023-07-06T03:56:21.580 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2023-07-06T03:56:21.580 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:21.581 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2023-07-06T03:56:21.581 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T03:56:21.581 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T03:56:21.582 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T03:56:21.582 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2023-07-06T03:56:21.582 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2023-07-06T03:56:21.583 DEBUG:teuthology.orchestra.run.smithi097:> return f 2023-07-06T03:56:21.583 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T03:56:21.583 DEBUG:teuthology.orchestra.run.smithi097:> 2023-07-06T03:56:21.583 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2023-07-06T03:56:21.584 DEBUG:teuthology.orchestra.run.smithi097:> ' 2023-07-06T03:56:21.725 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.168891.asok 2023-07-06T03:56:21.903 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.168891.asok 2023-07-06T03:56:21.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:21.904 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.168891.asok mds_sessions 2023-07-06T03:56:22.126 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:56:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9855, 2023-07-06T03:56:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2023-07-06T03:56:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2023-07-06T03:56:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2023-07-06T03:56:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 9855 2023-07-06T03:56:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2023-07-06T03:56:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:56:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2023-07-06T03:56:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 929081145 2023-07-06T03:56:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.9855 v1:192.168.0.1:0/929081145", 2023-07-06T03:56:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "v1:192.168.0.1:0/929081145", 2023-07-06T03:56:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions": [ 2023-07-06T03:56:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": 0, 2023-07-06T03:56:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "addrs": { 2023-07-06T03:56:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2023-07-06T03:56:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2023-07-06T03:56:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6836", 2023-07-06T03:56:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2797569799 2023-07-06T03:56:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-07-06T03:56:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2023-07-06T03:56:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6837", 2023-07-06T03:56:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2797569799 2023-07-06T03:56:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-06T03:56:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-07-06T03:56:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "seq": 0, 2023-07-06T03:56:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_gen": 0, 2023-07-06T03:56:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_ttl": "2023-07-06T03:57:21.798227+0000", 2023-07-06T03:56:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "last_cap_renew_request": "2023-07-06T03:56:21.798227+0000", 2023-07-06T03:56:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_renew_seq": 2, 2023-07-06T03:56:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "num_caps": 1, 2023-07-06T03:56:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "open" 2023-07-06T03:56:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-07-06T03:56:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-07-06T03:56:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 103 2023-07-06T03:56:22.135 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:56:22.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:56:22.141 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch file1) 2023-07-06T03:56:22.192 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:56:22.193 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:56:22.245 DEBUG:teuthology.orchestra.run.smithi146:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T03:56:22.246 DEBUG:teuthology.orchestra.run.smithi146:> import os 2023-07-06T03:56:22.246 DEBUG:teuthology.orchestra.run.smithi146:> import time 2023-07-06T03:56:22.247 DEBUG:teuthology.orchestra.run.smithi146:> 2023-07-06T03:56:22.247 DEBUG:teuthology.orchestra.run.smithi146:> fd = os.open("/home/ubuntu/cephtest/mnt.1/file1", os.O_RDWR | os.O_CREAT, 0o644) 2023-07-06T03:56:22.248 DEBUG:teuthology.orchestra.run.smithi146:> try: 2023-07-06T03:56:22.248 DEBUG:teuthology.orchestra.run.smithi146:> while True: 2023-07-06T03:56:22.249 DEBUG:teuthology.orchestra.run.smithi146:> os.write(fd, b'"'"'content'"'"') 2023-07-06T03:56:22.249 DEBUG:teuthology.orchestra.run.smithi146:> time.sleep(1) 2023-07-06T03:56:22.250 DEBUG:teuthology.orchestra.run.smithi146:> if not False: 2023-07-06T03:56:22.251 DEBUG:teuthology.orchestra.run.smithi146:> break 2023-07-06T03:56:22.251 DEBUG:teuthology.orchestra.run.smithi146:> except IOError as e: 2023-07-06T03:56:22.252 DEBUG:teuthology.orchestra.run.smithi146:> pass 2023-07-06T03:56:22.252 DEBUG:teuthology.orchestra.run.smithi146:> os.close(fd) 2023-07-06T03:56:22.253 DEBUG:teuthology.orchestra.run.smithi146:> ' 2023-07-06T03:56:22.258 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:56:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.454+0000 7f8b2b3ff700 1 Processor -- start 2023-07-06T03:56:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.454+0000 7f8b2b3ff700 1 -- start start 2023-07-06T03:56:22.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.455+0000 7f8b2b3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8b2415cc30 0x7f8b24161250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.455+0000 7f8b2b3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8b240b20d0 0x7f8b240cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.455+0000 7f8b2b3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b24160dc0 con 0x7f8b240cd4e0 2023-07-06T03:56:22.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.455+0000 7f8b2b3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b241607b0 con 0x7f8b2415cc30 2023-07-06T03:56:22.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.455+0000 7f8b2b3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2415d000 con 0x7f8b240b20d0 2023-07-06T03:56:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b2919b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8b2415cc30 0x7f8b24161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b2999c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8b240cd4e0 0x7f8b240d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50534/0 (socket says 172.21.15.97:50534) 2023-07-06T03:56:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b2919b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8b2415cc30 0x7f8b24161250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45174/0 (socket says 172.21.15.97:45174) 2023-07-06T03:56:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b2999c700 1 -- 172.21.15.97:0/2713078619 learned_addr learned my addr 172.21.15.97:0/2713078619 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b2899a700 1 --2- 172.21.15.97:0/2713078619 >> v2:172.21.15.146:3300/0 conn(0x7f8b240b20d0 0x7f8b240cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b2919b700 1 -- 172.21.15.97:0/2713078619 >> v2:172.21.15.146:3300/0 conn(0x7f8b240b20d0 msgr2=0x7f8b240cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b2919b700 1 --2- 172.21.15.97:0/2713078619 >> v2:172.21.15.146:3300/0 conn(0x7f8b240b20d0 0x7f8b240cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b2919b700 1 -- 172.21.15.97:0/2713078619 >> v1:172.21.15.97:6789/0 conn(0x7f8b240cd4e0 legacy=0x7f8b240d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b2919b700 1 -- 172.21.15.97:0/2713078619 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b240cfe30 con 0x7f8b2415cc30 2023-07-06T03:56:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b1bfff700 1 -- 172.21.15.97:0/2713078619 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922901202 0 0) 0x7f8b24160dc0 con 0x7f8b240cd4e0 2023-07-06T03:56:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b2919b700 1 --2- 172.21.15.97:0/2713078619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b2415cc30 0x7f8b24161250 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f8b240cd320 tx=0x7f8b20009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d7ee11a541826e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.456+0000 7f8b1bfff700 1 -- 172.21.15.97:0/2713078619 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b20011830 con 0x7f8b2415cc30 2023-07-06T03:56:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.457+0000 7f8b1bfff700 1 -- 172.21.15.97:0/2713078619 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b20011990 con 0x7f8b2415cc30 2023-07-06T03:56:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.457+0000 7f8b1bfff700 1 -- 172.21.15.97:0/2713078619 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b20011d20 con 0x7f8b2415cc30 2023-07-06T03:56:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.457+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/2713078619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b2415cc30 msgr2=0x7f8b24161250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.457+0000 7f8b2b3ff700 1 --2- 172.21.15.97:0/2713078619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b2415cc30 0x7f8b24161250 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f8b240cd320 tx=0x7f8b20009b20 comp rx=0 tx=0).stop 2023-07-06T03:56:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.457+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/2713078619 shutdown_connections 2023-07-06T03:56:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.457+0000 7f8b2b3ff700 1 --2- 172.21.15.97:0/2713078619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b2415cc30 0x7f8b24161250 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.457+0000 7f8b2b3ff700 1 --2- 172.21.15.97:0/2713078619 >> v2:172.21.15.146:3300/0 conn(0x7f8b240b20d0 0x7f8b240cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.457+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/2713078619 >> 172.21.15.97:0/2713078619 conn(0x7f8b2406c1c0 msgr2=0x7f8b2406c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.457+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/2713078619 shutdown_connections 2023-07-06T03:56:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.457+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/2713078619 wait complete. 2023-07-06T03:56:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.459+0000 7f8b2b3ff700 1 Processor -- start 2023-07-06T03:56:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.459+0000 7f8b2b3ff700 1 -- start start 2023-07-06T03:56:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2b3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b240b20d0 0x7f8b240b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2b3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b240cd4e0 0x7f8b241e61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2b3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b2415cc30 0x7f8b241e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2b3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b241e79d0 con 0x7f8b240cd4e0 2023-07-06T03:56:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2b3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b240b2fc0 con 0x7f8b2415cc30 2023-07-06T03:56:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2b3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b240b32a0 con 0x7f8b240b20d0 2023-07-06T03:56:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2899a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b240cd4e0 0x7f8b241e61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2899a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b240cd4e0 0x7f8b241e61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45176/0 (socket says 172.21.15.97:45176) 2023-07-06T03:56:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2899a700 1 -- 172.21.15.97:0/3304419320 learned_addr learned my addr 172.21.15.97:0/3304419320 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2899a700 1 -- 172.21.15.97:0/3304419320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b240b20d0 msgr2=0x7f8b240b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2899a700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b240b20d0 0x7f8b240b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2899a700 1 -- 172.21.15.97:0/3304419320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b2415cc30 msgr2=0x7f8b241e6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2899a700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b2415cc30 0x7f8b241e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.460+0000 7f8b2899a700 1 -- 172.21.15.97:0/3304419320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b200094a0 con 0x7f8b240cd4e0 2023-07-06T03:56:22.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.461+0000 7f8b2899a700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b240cd4e0 0x7f8b241e61c0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400e840 tx=0x7f8b1400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:22.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.461+0000 7f8b19ffb700 1 -- 172.21.15.97:0/3304419320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14009590 con 0x7f8b240cd4e0 2023-07-06T03:56:22.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.461+0000 7f8b19ffb700 1 -- 172.21.15.97:0/3304419320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14013680 con 0x7f8b240cd4e0 2023-07-06T03:56:22.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.461+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/3304419320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b240b3580 con 0x7f8b240cd4e0 2023-07-06T03:56:22.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.461+0000 7f8b19ffb700 1 -- 172.21.15.97:0/3304419320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14013a10 con 0x7f8b240cd4e0 2023-07-06T03:56:22.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.461+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/3304419320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b240b3b40 con 0x7f8b240cd4e0 2023-07-06T03:56:22.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.462+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/3304419320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2404e340 con 0x7f8b240cd4e0 2023-07-06T03:56:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.463+0000 7f8b19ffb700 1 -- 172.21.15.97:0/3304419320 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8b14013b70 con 0x7f8b240cd4e0 2023-07-06T03:56:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.464+0000 7f8b19ffb700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b0806ace0 0x7f8b0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.464+0000 7f8b19ffb700 1 -- 172.21.15.97:0/3304419320 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b14089410 con 0x7f8b240cd4e0 2023-07-06T03:56:22.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.464+0000 7f8b2919b700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b0806ace0 0x7f8b0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:22.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.465+0000 7f8b2919b700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b0806ace0 0x7f8b0806d190 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8b200090b0 tx=0x7f8b20010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:22.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.468+0000 7f8b19ffb700 1 -- 172.21.15.97:0/3304419320 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8b14057f40 con 0x7f8b240cd4e0 2023-07-06T03:56:22.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.703+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/3304419320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b240c61f0 con 0x7f8b240cd4e0 2023-07-06T03:56:22.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.703+0000 7f8b19ffb700 1 -- 172.21.15.97:0/3304419320 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4680 (secure 0 0 0) 0x7f8b14057ad0 con 0x7f8b240cd4e0 2023-07-06T03:56:22.706 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:22.706 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/3313682816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3313682816},{"type":"v1","addr":"172.21.15.146:6834","nonce":3313682816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9624,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/387461585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":387461585},{"type":"v1","addr":"172.21.15.146:6835","nonce":387461585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9630,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3109027091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3109027091},{"type":"v1","addr":"172.21.15.97:6835","nonce":3109027091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:56:04.566296+0000","modified":"2023-07-06T03:56:08.096494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9633":{"gid":9633,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2797569799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2797569799},{"type":"v1","addr":"172.21.15.97:6837","nonce":2797569799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-06T03:56:22.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/3304419320 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b0806ace0 msgr2=0x7f8b0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:22.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b0806ace0 0x7f8b0806d190 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8b200090b0 tx=0x7f8b20010000 comp rx=0 tx=0).stop 2023-07-06T03:56:22.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/3304419320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b240cd4e0 msgr2=0x7f8b241e61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:22.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b240cd4e0 0x7f8b241e61c0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400e840 tx=0x7f8b1400ec00 comp rx=0 tx=0).stop 2023-07-06T03:56:22.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/3304419320 shutdown_connections 2023-07-06T03:56:22.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f8b0806ace0 0x7f8b0806d190 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b2415cc30 0x7f8b241e6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f8b240b20d0 0x7f8b240b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 --2- 172.21.15.97:0/3304419320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b240cd4e0 0x7f8b241e61c0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/3304419320 >> 172.21.15.97:0/3304419320 conn(0x7f8b2406c1c0 msgr2=0x7f8b2415f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:22.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/3304419320 shutdown_connections 2023-07-06T03:56:22.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.706+0000 7f8b2b3ff700 1 -- 172.21.15.97:0/3304419320 wait complete. 2023-07-06T03:56:22.711 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 103 2023-07-06T03:56:22.725 INFO:tasks.ceph_test_case:waiting 70s for health warning matching MDS_CLIENT_LATE_RELEASE 2023-07-06T03:56:22.726 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:56:22.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.919+0000 7f9788dff700 1 Processor -- start 2023-07-06T03:56:22.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.919+0000 7f9788dff700 1 -- start start 2023-07-06T03:56:22.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.921+0000 7f9788dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f978415c950 0x7f97840b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.921+0000 7f9788dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f978406c920 0x7f97840d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.921+0000 7f9788dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97840b5160 con 0x7f97840b4850 2023-07-06T03:56:22.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.921+0000 7f9788dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97840cd4c0 con 0x7f978415c950 2023-07-06T03:56:22.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.921+0000 7f9788dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97840cd7a0 con 0x7f978406c920 2023-07-06T03:56:22.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.921+0000 7f978259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f97840b4850 0x7f97840cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50538/0 (socket says 172.21.15.97:50538) 2023-07-06T03:56:22.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.921+0000 7f978259c700 1 -- 172.21.15.97:0/1512202183 learned_addr learned my addr 172.21.15.97:0/1512202183 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:22.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.921+0000 7f9782d9d700 1 --2- 172.21.15.97:0/1512202183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f978406c920 0x7f97840d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:22.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.921+0000 7f977bfff700 1 --2- 172.21.15.97:0/1512202183 >> v2:172.21.15.146:3300/0 conn(0x7f978415c950 0x7f97840b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:22.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.921+0000 7f9781d9b700 1 -- 172.21.15.97:0/1512202183 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149882242 0 0) 0x7f97840b5160 con 0x7f97840b4850 2023-07-06T03:56:22.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.921+0000 7f9781d9b700 1 -- 172.21.15.97:0/1512202183 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97700035b0 con 0x7f97840b4850 2023-07-06T03:56:22.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.922+0000 7f9782d9d700 1 -- 172.21.15.97:0/1512202183 >> v2:172.21.15.146:3300/0 conn(0x7f978415c950 msgr2=0x7f97840b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:22.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.922+0000 7f9782d9d700 1 --2- 172.21.15.97:0/1512202183 >> v2:172.21.15.146:3300/0 conn(0x7f978415c950 0x7f97840b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.922+0000 7f9782d9d700 1 -- 172.21.15.97:0/1512202183 >> v1:172.21.15.97:6789/0 conn(0x7f97840b4850 legacy=0x7f97840cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:22.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.922+0000 7f9781d9b700 1 -- 172.21.15.97:0/1512202183 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3862401650 0 0) 0x7f97700035b0 con 0x7f97840b4850 2023-07-06T03:56:22.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.922+0000 7f9782d9d700 1 -- 172.21.15.97:0/1512202183 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97840cda80 con 0x7f978406c920 2023-07-06T03:56:22.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.922+0000 7f977bfff700 1 --2- 172.21.15.97:0/1512202183 >> v2:172.21.15.146:3300/0 conn(0x7f978415c950 0x7f97840b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:56:22.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.922+0000 7f9782d9d700 1 --2- 172.21.15.97:0/1512202183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f978406c920 0x7f97840d0d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f977c009d90 tx=0x7f977c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aea9e66dee09a626 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:22.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.922+0000 7f9781d9b700 1 -- 172.21.15.97:0/1512202183 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c00e070 con 0x7f978406c920 2023-07-06T03:56:22.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.922+0000 7f9781d9b700 1 -- 172.21.15.97:0/1512202183 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c003a20 con 0x7f978406c920 2023-07-06T03:56:22.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.922+0000 7f9781d9b700 1 -- 172.21.15.97:0/1512202183 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774001670 con 0x7f978406c920 2023-07-06T03:56:22.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.923+0000 7f9788dff700 1 -- 172.21.15.97:0/1512202183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f978406c920 msgr2=0x7f97840d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:22.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.923+0000 7f9788dff700 1 --2- 172.21.15.97:0/1512202183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f978406c920 0x7f97840d0d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f977c009d90 tx=0x7f977c00b330 comp rx=0 tx=0).stop 2023-07-06T03:56:22.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.923+0000 7f9788dff700 1 -- 172.21.15.97:0/1512202183 shutdown_connections 2023-07-06T03:56:22.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.923+0000 7f9788dff700 1 --2- 172.21.15.97:0/1512202183 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f978406c920 0x7f97840d0d80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.923+0000 7f9788dff700 1 --2- 172.21.15.97:0/1512202183 >> v2:172.21.15.146:3300/0 conn(0x7f978415c950 0x7f97840b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.923+0000 7f9788dff700 1 -- 172.21.15.97:0/1512202183 >> 172.21.15.97:0/1512202183 conn(0x7f978406b410 msgr2=0x7f97840cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:22.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.923+0000 7f9788dff700 1 -- 172.21.15.97:0/1512202183 shutdown_connections 2023-07-06T03:56:22.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.923+0000 7f9788dff700 1 -- 172.21.15.97:0/1512202183 wait complete. 2023-07-06T03:56:22.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.925+0000 7f9788dff700 1 Processor -- start 2023-07-06T03:56:22.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.925+0000 7f9788dff700 1 -- start start 2023-07-06T03:56:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f9788dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f978406c920 0x7f97840ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f9788dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840b4850 0x7f978415d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f9788dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f978415c950 0x7f9784160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f9788dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97841f55d0 con 0x7f978406c920 2023-07-06T03:56:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f9788dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840cc4c0 con 0x7f978415c950 2023-07-06T03:56:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f9788dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97840cc7a0 con 0x7f97840b4850 2023-07-06T03:56:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f978259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f978406c920 0x7f97840ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f978259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f978406c920 0x7f97840ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45186/0 (socket says 172.21.15.97:45186) 2023-07-06T03:56:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f978259c700 1 -- 172.21.15.97:0/378205515 learned_addr learned my addr 172.21.15.97:0/378205515 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f978259c700 1 -- 172.21.15.97:0/378205515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840b4850 msgr2=0x7f978415d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:22.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f978259c700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840b4850 0x7f978415d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f978259c700 1 -- 172.21.15.97:0/378205515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f978415c950 msgr2=0x7f9784160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:22.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f978259c700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f978415c950 0x7f9784160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:22.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f978259c700 1 -- 172.21.15.97:0/378205515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c0094a0 con 0x7f978406c920 2023-07-06T03:56:22.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.926+0000 7f978259c700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f978406c920 0x7f97840ca560 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f97740011e0 tx=0x7f977400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:22.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.927+0000 7f977affd700 1 -- 172.21.15.97:0/378205515 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774004c80 con 0x7f978406c920 2023-07-06T03:56:22.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.927+0000 7f977affd700 1 -- 172.21.15.97:0/378205515 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9774004e00 con 0x7f978406c920 2023-07-06T03:56:22.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.927+0000 7f977affd700 1 -- 172.21.15.97:0/378205515 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774007310 con 0x7f978406c920 2023-07-06T03:56:22.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.927+0000 7f9788dff700 1 -- 172.21.15.97:0/378205515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c009a70 con 0x7f978406c920 2023-07-06T03:56:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.927+0000 7f9788dff700 1 -- 172.21.15.97:0/378205515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97840ccd60 con 0x7f978406c920 2023-07-06T03:56:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.928+0000 7f9788dff700 1 -- 172.21.15.97:0/378205515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978404e340 con 0x7f978406c920 2023-07-06T03:56:22.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.929+0000 7f977affd700 1 -- 172.21.15.97:0/378205515 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9774007470 con 0x7f978406c920 2023-07-06T03:56:22.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.929+0000 7f977affd700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f976406ace0 0x7f976406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:22.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.930+0000 7f977affd700 1 -- 172.21.15.97:0/378205515 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97740892b0 con 0x7f978406c920 2023-07-06T03:56:22.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.930+0000 7f977bfff700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f976406ace0 0x7f976406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:22.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.930+0000 7f977bfff700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f976406ace0 0x7f976406d190 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f976c006f10 tx=0x7f976c006e60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:22.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:22.934+0000 7f977affd700 1 -- 172.21.15.97:0/378205515 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9774054360 con 0x7f978406c920 2023-07-06T03:56:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.176+0000 7f9788dff700 1 -- 172.21.15.97:0/378205515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f97840d01c0 con 0x7f978406c920 2023-07-06T03:56:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.176+0000 7f977affd700 1 -- 172.21.15.97:0/378205515 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9774057980 con 0x7f978406c920 2023-07-06T03:56:23.179 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:23.179 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:56:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.178+0000 7f9788dff700 1 -- 172.21.15.97:0/378205515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f976406ace0 msgr2=0x7f976406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.178+0000 7f9788dff700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f976406ace0 0x7f976406d190 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f976c006f10 tx=0x7f976c006e60 comp rx=0 tx=0).stop 2023-07-06T03:56:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.178+0000 7f9788dff700 1 -- 172.21.15.97:0/378205515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f978406c920 msgr2=0x7f97840ca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.178+0000 7f9788dff700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f978406c920 0x7f97840ca560 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f97740011e0 tx=0x7f977400ca90 comp rx=0 tx=0).stop 2023-07-06T03:56:23.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.179+0000 7f9788dff700 1 -- 172.21.15.97:0/378205515 shutdown_connections 2023-07-06T03:56:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.179+0000 7f9788dff700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f976406ace0 0x7f976406d190 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.179+0000 7f9788dff700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f978415c950 0x7f9784160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.179+0000 7f9788dff700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f978406c920 0x7f97840ca560 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.179+0000 7f9788dff700 1 --2- 172.21.15.97:0/378205515 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f97840b4850 0x7f978415d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.179+0000 7f9788dff700 1 -- 172.21.15.97:0/378205515 >> 172.21.15.97:0/378205515 conn(0x7f978406b410 msgr2=0x7f97840b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.179+0000 7f9788dff700 1 -- 172.21.15.97:0/378205515 shutdown_connections 2023-07-06T03:56:23.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:23.179+0000 7f9788dff700 1 -- 172.21.15.97:0/378205515 wait complete. 2023-07-06T03:56:23.198 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:56:23.199 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:56:28.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:56:28.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.560+0000 7fcb663ff700 1 Processor -- start 2023-07-06T03:56:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.560+0000 7fcb663ff700 1 -- start start 2023-07-06T03:56:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.561+0000 7fcb663ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcb60154a60 0x7fcb600ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.561+0000 7fcb663ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb6006cce0 0x7fcb600cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:28.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.561+0000 7fcb663ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6015aa10 con 0x7fcb600cb5c0 2023-07-06T03:56:28.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.561+0000 7fcb663ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6015a800 con 0x7fcb60154a60 2023-07-06T03:56:28.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.561+0000 7fcb663ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6015acf0 con 0x7fcb6006cce0 2023-07-06T03:56:28.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.562+0000 7fcb5ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcb60154a60 0x7fcb600ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:28.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.562+0000 7fcb5f7fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb6006cce0 0x7fcb600cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:28.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.562+0000 7fcb5f7fe700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb6006cce0 0x7fcb600cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:39932/0 (socket says 172.21.15.97:39932) 2023-07-06T03:56:28.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.562+0000 7fcb5f7fe700 1 -- 172.21.15.97:0/921456997 learned_addr learned my addr 172.21.15.97:0/921456997 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:28.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.562+0000 7fcb5ffff700 1 -- 172.21.15.97:0/921456997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb6006cce0 msgr2=0x7fcb600cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:28.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.562+0000 7fcb5ffff700 1 --2- 172.21.15.97:0/921456997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb6006cce0 0x7fcb600cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:28.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.562+0000 7fcb5ffff700 1 -- 172.21.15.97:0/921456997 >> v1:172.21.15.146:6789/0 conn(0x7fcb600cb5c0 legacy=0x7fcb600c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:28.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.562+0000 7fcb5ffff700 1 -- 172.21.15.97:0/921456997 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6015afd0 con 0x7fcb60154a60 2023-07-06T03:56:28.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.562+0000 7fcb5f7fe700 1 --2- 172.21.15.97:0/921456997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb6006cce0 0x7fcb600cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:56:28.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb5ffff700 1 --2- 172.21.15.97:0/921456997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb60154a60 0x7fcb600ca7e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009840 tx=0x7fcb50009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac889a7b468c56b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:28.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb5effd700 1 -- 172.21.15.97:0/921456997 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50011830 con 0x7fcb60154a60 2023-07-06T03:56:28.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb5effd700 1 -- 172.21.15.97:0/921456997 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50011990 con 0x7fcb60154a60 2023-07-06T03:56:28.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb5effd700 1 -- 172.21.15.97:0/921456997 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50011d20 con 0x7fcb60154a60 2023-07-06T03:56:28.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb663ff700 1 -- 172.21.15.97:0/921456997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb60154a60 msgr2=0x7fcb600ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:28.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb663ff700 1 --2- 172.21.15.97:0/921456997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb60154a60 0x7fcb600ca7e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009840 tx=0x7fcb50009b50 comp rx=0 tx=0).stop 2023-07-06T03:56:28.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb663ff700 1 -- 172.21.15.97:0/921456997 shutdown_connections 2023-07-06T03:56:28.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb663ff700 1 --2- 172.21.15.97:0/921456997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb60154a60 0x7fcb600ca7e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:28.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb663ff700 1 --2- 172.21.15.97:0/921456997 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb6006cce0 0x7fcb600cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:28.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb663ff700 1 -- 172.21.15.97:0/921456997 >> 172.21.15.97:0/921456997 conn(0x7fcb6006a060 msgr2=0x7fcb6006a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:28.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb663ff700 1 -- 172.21.15.97:0/921456997 shutdown_connections 2023-07-06T03:56:28.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.563+0000 7fcb663ff700 1 -- 172.21.15.97:0/921456997 wait complete. 2023-07-06T03:56:28.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.565+0000 7fcb663ff700 1 Processor -- start 2023-07-06T03:56:28.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.565+0000 7fcb663ff700 1 -- start start 2023-07-06T03:56:28.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.566+0000 7fcb663ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6006cce0 0x7fcb60150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:28.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.566+0000 7fcb663ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb600cb5c0 0x7fcb60152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:28.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.566+0000 7fcb663ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb60154a60 0x7fcb600ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:28.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.566+0000 7fcb663ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6015b900 con 0x7fcb6006cce0 2023-07-06T03:56:28.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.566+0000 7fcb663ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60159f40 con 0x7fcb60154a60 2023-07-06T03:56:28.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.566+0000 7fcb663ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6015a220 con 0x7fcb600cb5c0 2023-07-06T03:56:28.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.566+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6006cce0 0x7fcb60150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:28.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.566+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6006cce0 0x7fcb60150640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45196/0 (socket says 172.21.15.97:45196) 2023-07-06T03:56:28.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.566+0000 7fcb5ffff700 1 -- 172.21.15.97:0/2053923884 learned_addr learned my addr 172.21.15.97:0/2053923884 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:28.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb5f7fe700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb600cb5c0 0x7fcb60152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb6499c700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb60154a60 0x7fcb600ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb5ffff700 1 -- 172.21.15.97:0/2053923884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb600cb5c0 msgr2=0x7fcb60152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb5ffff700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb600cb5c0 0x7fcb60152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb5ffff700 1 -- 172.21.15.97:0/2053923884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb60154a60 msgr2=0x7fcb600ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb5ffff700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb60154a60 0x7fcb600ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb5ffff700 1 -- 172.21.15.97:0/2053923884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54009550 con 0x7fcb6006cce0 2023-07-06T03:56:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb6499c700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb60154a60 0x7fcb600ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:28.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb5ffff700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6006cce0 0x7fcb60150640 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009fd0 tx=0x7fcb50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:28.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb5cff9700 1 -- 172.21.15.97:0/2053923884 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb500036a0 con 0x7fcb6006cce0 2023-07-06T03:56:28.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb5cff9700 1 -- 172.21.15.97:0/2053923884 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5001e070 con 0x7fcb6006cce0 2023-07-06T03:56:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb5cff9700 1 -- 172.21.15.97:0/2053923884 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50003a60 con 0x7fcb6006cce0 2023-07-06T03:56:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb663ff700 1 -- 172.21.15.97:0/2053923884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb500094a0 con 0x7fcb6006cce0 2023-07-06T03:56:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.567+0000 7fcb663ff700 1 -- 172.21.15.97:0/2053923884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6015bbc0 con 0x7fcb6006cce0 2023-07-06T03:56:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.568+0000 7fcb663ff700 1 -- 172.21.15.97:0/2053923884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6004e340 con 0x7fcb6006cce0 2023-07-06T03:56:28.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.569+0000 7fcb5cff9700 1 -- 172.21.15.97:0/2053923884 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb50003bc0 con 0x7fcb6006cce0 2023-07-06T03:56:28.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.570+0000 7fcb5cff9700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb4c06ace0 0x7fcb4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:28.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.570+0000 7fcb5cff9700 1 -- 172.21.15.97:0/2053923884 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb5008a3a0 con 0x7fcb6006cce0 2023-07-06T03:56:28.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.570+0000 7fcb5f7fe700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb4c06ace0 0x7fcb4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:28.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.571+0000 7fcb5f7fe700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb4c06ace0 0x7fcb4c06d190 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fcb600cce50 tx=0x7fcb540092c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:28.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.575+0000 7fcb5cff9700 1 -- 172.21.15.97:0/2053923884 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcb500554f0 con 0x7fcb6006cce0 2023-07-06T03:56:28.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.815+0000 7fcb663ff700 1 -- 172.21.15.97:0/2053923884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcb6006cac0 con 0x7fcb6006cce0 2023-07-06T03:56:28.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.815+0000 7fcb5cff9700 1 -- 172.21.15.97:0/2053923884 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fcb50058b10 con 0x7fcb6006cce0 2023-07-06T03:56:28.818 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:28.818 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:56:28.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.817+0000 7fcb663ff700 1 -- 172.21.15.97:0/2053923884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb4c06ace0 msgr2=0x7fcb4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:28.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.817+0000 7fcb663ff700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb4c06ace0 0x7fcb4c06d190 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fcb600cce50 tx=0x7fcb540092c0 comp rx=0 tx=0).stop 2023-07-06T03:56:28.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.817+0000 7fcb663ff700 1 -- 172.21.15.97:0/2053923884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6006cce0 msgr2=0x7fcb60150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.817+0000 7fcb663ff700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6006cce0 0x7fcb60150640 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009fd0 tx=0x7fcb50004750 comp rx=0 tx=0).stop 2023-07-06T03:56:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.817+0000 7fcb663ff700 1 -- 172.21.15.97:0/2053923884 shutdown_connections 2023-07-06T03:56:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.817+0000 7fcb663ff700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcb4c06ace0 0x7fcb4c06d190 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.817+0000 7fcb663ff700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb60154a60 0x7fcb600ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.817+0000 7fcb663ff700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb6006cce0 0x7fcb60150640 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.817+0000 7fcb663ff700 1 --2- 172.21.15.97:0/2053923884 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fcb600cb5c0 0x7fcb60152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:28.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.817+0000 7fcb663ff700 1 -- 172.21.15.97:0/2053923884 >> 172.21.15.97:0/2053923884 conn(0x7fcb6006a060 msgr2=0x7fcb600cf070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:28.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.818+0000 7fcb663ff700 1 -- 172.21.15.97:0/2053923884 shutdown_connections 2023-07-06T03:56:28.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:28.818+0000 7fcb663ff700 1 -- 172.21.15.97:0/2053923884 wait complete. 2023-07-06T03:56:28.837 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:56:28.838 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:56:33.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:56:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.041+0000 7fc3f23ff700 1 Processor -- start 2023-07-06T03:56:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.041+0000 7fc3f23ff700 1 -- start start 2023-07-06T03:56:34.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.042+0000 7fc3f23ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3ec0cee80 0x7fc3ec0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:34.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3f23ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3ec0ccac0 0x7fc3ec070100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:34.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3f23ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ec0c9320 con 0x7fc3ec0adfb0 2023-07-06T03:56:34.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3f23ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec0cdce0 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3f23ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0ccf60 con 0x7fc3ec0cee80 2023-07-06T03:56:34.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3ebfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc3ec0adfb0 0x7fc3ec0ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:34322/0 (socket says 172.21.15.97:34322) 2023-07-06T03:56:34.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3ebfff700 1 -- 172.21.15.97:0/1192510181 learned_addr learned my addr 172.21.15.97:0/1192510181 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:56:34.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3f099c700 1 --2- 172.21.15.97:0/1192510181 >> v2:172.21.15.97:3300/0 conn(0x7fc3ec0ccac0 0x7fc3ec070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:34.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3eb7fe700 1 --2- 172.21.15.97:0/1192510181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3ec0cee80 0x7fc3ec0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:34.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3eaffd700 1 -- 172.21.15.97:0/1192510181 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719252720 0 0) 0x7fc3ec0c9320 con 0x7fc3ec0adfb0 2023-07-06T03:56:34.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3eaffd700 1 -- 172.21.15.97:0/1192510181 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d40035b0 con 0x7fc3ec0adfb0 2023-07-06T03:56:34.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3f099c700 1 -- 172.21.15.97:0/1192510181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3ec0cee80 msgr2=0x7fc3ec0cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:34.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3f099c700 1 --2- 172.21.15.97:0/1192510181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3ec0cee80 0x7fc3ec0cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:34.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3f099c700 1 -- 172.21.15.97:0/1192510181 >> v1:172.21.15.97:6789/0 conn(0x7fc3ec0adfb0 legacy=0x7fc3ec0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:34.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.043+0000 7fc3f099c700 1 -- 172.21.15.97:0/1192510181 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec0cd160 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.044+0000 7fc3f099c700 1 --2- 172.21.15.97:0/1192510181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0ccac0 0x7fc3ec070100 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009d90 tx=0x7fc3e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd451d6cb40c69e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:34.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.044+0000 7fc3eaffd700 1 -- 172.21.15.97:0/1192510181 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e400e070 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.044+0000 7fc3eaffd700 1 -- 172.21.15.97:0/1192510181 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e400bb20 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.044+0000 7fc3eaffd700 1 -- 172.21.15.97:0/1192510181 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e400be90 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.044+0000 7fc3f23ff700 1 -- 172.21.15.97:0/1192510181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0ccac0 msgr2=0x7fc3ec070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:34.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.044+0000 7fc3f23ff700 1 --2- 172.21.15.97:0/1192510181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0ccac0 0x7fc3ec070100 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009d90 tx=0x7fc3e400b330 comp rx=0 tx=0).stop 2023-07-06T03:56:34.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.044+0000 7fc3f23ff700 1 -- 172.21.15.97:0/1192510181 shutdown_connections 2023-07-06T03:56:34.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.044+0000 7fc3f23ff700 1 --2- 172.21.15.97:0/1192510181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0ccac0 0x7fc3ec070100 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:34.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.044+0000 7fc3f23ff700 1 --2- 172.21.15.97:0/1192510181 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3ec0cee80 0x7fc3ec0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:34.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.044+0000 7fc3f23ff700 1 -- 172.21.15.97:0/1192510181 >> 172.21.15.97:0/1192510181 conn(0x7fc3ec06a060 msgr2=0x7fc3ec06a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:34.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.044+0000 7fc3f23ff700 1 -- 172.21.15.97:0/1192510181 shutdown_connections 2023-07-06T03:56:34.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.045+0000 7fc3f23ff700 1 -- 172.21.15.97:0/1192510181 wait complete. 2023-07-06T03:56:34.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.047+0000 7fc3f23ff700 1 Processor -- start 2023-07-06T03:56:34.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.047+0000 7fc3f23ff700 1 -- start start 2023-07-06T03:56:34.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.048+0000 7fc3f23ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3ec0adfb0 0x7fc3ec14ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:34.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.048+0000 7fc3f23ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0ccac0 0x7fc3ec15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:34.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.048+0000 7fc3f23ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3ec0cee80 0x7fc3ec15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:34.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.048+0000 7fc3f23ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec159540 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.048+0000 7fc3f23ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0c30c0 con 0x7fc3ec0adfb0 2023-07-06T03:56:34.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.048+0000 7fc3f23ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0c33a0 con 0x7fc3ec0cee80 2023-07-06T03:56:34.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.048+0000 7fc3eb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0ccac0 0x7fc3ec15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.048+0000 7fc3eb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0ccac0 0x7fc3ec15b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59222/0 (socket says 172.21.15.97:59222) 2023-07-06T03:56:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3eb7fe700 1 -- 172.21.15.97:0/2199349143 learned_addr learned my addr 172.21.15.97:0/2199349143 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3ebfff700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3ec0adfb0 0x7fc3ec14ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:34.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3f099c700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3ec0cee80 0x7fc3ec15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3eb7fe700 1 -- 172.21.15.97:0/2199349143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3ec0cee80 msgr2=0x7fc3ec15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3eb7fe700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3ec0cee80 0x7fc3ec15b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3eb7fe700 1 -- 172.21.15.97:0/2199349143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3ec0adfb0 msgr2=0x7fc3ec14ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:34.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3eb7fe700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3ec0adfb0 0x7fc3ec14ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3eb7fe700 1 -- 172.21.15.97:0/2199349143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e4009a70 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3ebfff700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3ec0adfb0 0x7fc3ec14ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:34.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3eb7fe700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0ccac0 0x7fc3ec15b270 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000e810 tx=0x7fc3e000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:34.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3e8ff9700 1 -- 172.21.15.97:0/2199349143 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0013610 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3e8ff9700 1 -- 172.21.15.97:0/2199349143 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e0013790 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3e8ff9700 1 -- 172.21.15.97:0/2199349143 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0013b40 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.049+0000 7fc3f23ff700 1 -- 172.21.15.97:0/2199349143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec0c3680 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.050+0000 7fc3f23ff700 1 -- 172.21.15.97:0/2199349143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec0c3c40 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.051+0000 7fc3f23ff700 1 -- 172.21.15.97:0/2199349143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ec04e340 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.052+0000 7fc3e8ff9700 1 -- 172.21.15.97:0/2199349143 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3e0013ca0 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.053+0000 7fc3e8ff9700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc3d806ace0 0x7fc3d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:34.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.053+0000 7fc3e8ff9700 1 -- 172.21.15.97:0/2199349143 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3e0089120 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.053+0000 7fc3ebfff700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc3d806ace0 0x7fc3d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.054+0000 7fc3ebfff700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc3d806ace0 0x7fc3d806d190 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc005910 tx=0x7fc3dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:34.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.058+0000 7fc3e8ff9700 1 -- 172.21.15.97:0/2199349143 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc3e0057c60 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.296+0000 7fc3f23ff700 1 -- 172.21.15.97:0/2199349143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc3ec071fe0 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.297+0000 7fc3e8ff9700 1 -- 172.21.15.97:0/2199349143 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc3e00577f0 con 0x7fc3ec0ccac0 2023-07-06T03:56:34.300 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:34.300 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:56:34.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 -- 172.21.15.97:0/2199349143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc3d806ace0 msgr2=0x7fc3d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:34.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc3d806ace0 0x7fc3d806d190 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc005910 tx=0x7fc3dc009040 comp rx=0 tx=0).stop 2023-07-06T03:56:34.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 -- 172.21.15.97:0/2199349143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0ccac0 msgr2=0x7fc3ec15b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:34.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0ccac0 0x7fc3ec15b270 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000e810 tx=0x7fc3e000eb20 comp rx=0 tx=0).stop 2023-07-06T03:56:34.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 -- 172.21.15.97:0/2199349143 shutdown_connections 2023-07-06T03:56:34.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc3d806ace0 0x7fc3d806d190 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:34.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc3ec0cee80 0x7fc3ec15b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:34.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3ec0adfb0 0x7fc3ec14ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:34.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 --2- 172.21.15.97:0/2199349143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3ec0ccac0 0x7fc3ec15b270 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:34.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 -- 172.21.15.97:0/2199349143 >> 172.21.15.97:0/2199349143 conn(0x7fc3ec06a060 msgr2=0x7fc3ec06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:34.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 -- 172.21.15.97:0/2199349143 shutdown_connections 2023-07-06T03:56:34.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:34.299+0000 7fc3f23ff700 1 -- 172.21.15.97:0/2199349143 wait complete. 2023-07-06T03:56:34.318 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:56:34.319 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:56:37.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:56:37.352 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:56:39.320 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:56:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.521+0000 7f0f8dfff700 1 Processor -- start 2023-07-06T03:56:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.522+0000 7f0f8dfff700 1 -- start start 2023-07-06T03:56:39.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.523+0000 7f0f8dfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0f880ccc90 0x7f0f88163e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.523+0000 7f0f8dfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f880c9320 con 0x7f0f880adfb0 2023-07-06T03:56:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.523+0000 7f0f8dfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f880cd160 con 0x7f0f880cee80 2023-07-06T03:56:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.523+0000 7f0f8dfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f880cdce0 con 0x7f0f880ccc90 2023-07-06T03:56:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.523+0000 7f0f87fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0f880ccc90 0x7f0f88163e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.523+0000 7f0f87fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0f880ccc90 0x7f0f88163e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59238/0 (socket says 172.21.15.97:59238) 2023-07-06T03:56:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.523+0000 7f0f87fff700 1 -- 172.21.15.97:0/3143389161 learned_addr learned my addr 172.21.15.97:0/3143389161 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:39.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.523+0000 7f0f87fff700 1 -- 172.21.15.97:0/3143389161 >> v1:172.21.15.146:6789/0 conn(0x7f0f880cee80 legacy=0x7f0f880cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.523+0000 7f0f87fff700 1 -- 172.21.15.97:0/3143389161 >> v1:172.21.15.97:6789/0 conn(0x7f0f880adfb0 legacy=0x7f0f880ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.523+0000 7f0f87fff700 1 -- 172.21.15.97:0/3143389161 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f880cd440 con 0x7f0f880ccc90 2023-07-06T03:56:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.524+0000 7f0f87fff700 1 --2- 172.21.15.97:0/3143389161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f880ccc90 0x7f0f88163e80 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009e50 tx=0x7f0f8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f862bbca55331b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.524+0000 7f0f867fc700 1 -- 172.21.15.97:0/3143389161 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80003c20 con 0x7f0f880ccc90 2023-07-06T03:56:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.524+0000 7f0f867fc700 1 -- 172.21.15.97:0/3143389161 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f80003d80 con 0x7f0f880ccc90 2023-07-06T03:56:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.524+0000 7f0f867fc700 1 -- 172.21.15.97:0/3143389161 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8000bbc0 con 0x7f0f880ccc90 2023-07-06T03:56:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.524+0000 7f0f8dfff700 1 -- 172.21.15.97:0/3143389161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f880ccc90 msgr2=0x7f0f88163e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.524+0000 7f0f8dfff700 1 --2- 172.21.15.97:0/3143389161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f880ccc90 0x7f0f88163e80 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009e50 tx=0x7f0f8000b330 comp rx=0 tx=0).stop 2023-07-06T03:56:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.525+0000 7f0f8dfff700 1 -- 172.21.15.97:0/3143389161 shutdown_connections 2023-07-06T03:56:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.525+0000 7f0f8dfff700 1 --2- 172.21.15.97:0/3143389161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f880ccc90 0x7f0f88163e80 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.525+0000 7f0f8dfff700 1 -- 172.21.15.97:0/3143389161 >> 172.21.15.97:0/3143389161 conn(0x7f0f8806a060 msgr2=0x7f0f8806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.525+0000 7f0f8dfff700 1 -- 172.21.15.97:0/3143389161 shutdown_connections 2023-07-06T03:56:39.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.525+0000 7f0f8dfff700 1 -- 172.21.15.97:0/3143389161 wait complete. 2023-07-06T03:56:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.526+0000 7f0f8dfff700 1 Processor -- start 2023-07-06T03:56:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.526+0000 7f0f8dfff700 1 -- start start 2023-07-06T03:56:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.527+0000 7f0f8dfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f880adfb0 0x7f0f881571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:39.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.527+0000 7f0f8dfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f880ccc90 0x7f0f880c2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:39.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.527+0000 7f0f8dfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f880cee80 0x7f0f880c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:39.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.527+0000 7f0f8dfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f880b24a0 con 0x7f0f880ccc90 2023-07-06T03:56:39.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.527+0000 7f0f8dfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f880b2170 con 0x7f0f880adfb0 2023-07-06T03:56:39.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.527+0000 7f0f8dfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8815aaf0 con 0x7f0f880cee80 2023-07-06T03:56:39.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f86ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f880ccc90 0x7f0f880c2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:39.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f86ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f880ccc90 0x7f0f880c2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59240/0 (socket says 172.21.15.97:59240) 2023-07-06T03:56:39.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f86ffd700 1 -- 172.21.15.97:0/1534412101 learned_addr learned my addr 172.21.15.97:0/1534412101 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:39.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f877fe700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f880adfb0 0x7f0f881571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:39.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f86ffd700 1 -- 172.21.15.97:0/1534412101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f880cee80 msgr2=0x7f0f880c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:39.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f86ffd700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f880cee80 0x7f0f880c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:39.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f86ffd700 1 -- 172.21.15.97:0/1534412101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f880adfb0 msgr2=0x7f0f881571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:39.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f86ffd700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f880adfb0 0x7f0f881571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:39.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f86ffd700 1 -- 172.21.15.97:0/1534412101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80009b30 con 0x7f0f880ccc90 2023-07-06T03:56:39.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f877fe700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f880adfb0 0x7f0f881571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:56:39.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f86ffd700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f880ccc90 0x7f0f880c2ef0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b480 tx=0x7f0f7c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:39.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.528+0000 7f0f6ffff700 1 -- 172.21.15.97:0/1534412101 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c011830 con 0x7f0f880ccc90 2023-07-06T03:56:39.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.529+0000 7f0f6ffff700 1 -- 172.21.15.97:0/1534412101 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c011990 con 0x7f0f880ccc90 2023-07-06T03:56:39.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.529+0000 7f0f6ffff700 1 -- 172.21.15.97:0/1534412101 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c011d40 con 0x7f0f880ccc90 2023-07-06T03:56:39.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.529+0000 7f0f8dfff700 1 -- 172.21.15.97:0/1534412101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f880c3a60 con 0x7f0f880ccc90 2023-07-06T03:56:39.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.529+0000 7f0f8dfff700 1 -- 172.21.15.97:0/1534412101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f880c4020 con 0x7f0f880ccc90 2023-07-06T03:56:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.530+0000 7f0f8dfff700 1 -- 172.21.15.97:0/1534412101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f880c4240 con 0x7f0f880ccc90 2023-07-06T03:56:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.531+0000 7f0f6ffff700 1 -- 172.21.15.97:0/1534412101 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f7c01a400 con 0x7f0f880ccc90 2023-07-06T03:56:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.531+0000 7f0f6ffff700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f7406ace0 0x7f0f7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.532+0000 7f0f6ffff700 1 -- 172.21.15.97:0/1534412101 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f7c089190 con 0x7f0f880ccc90 2023-07-06T03:56:39.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.532+0000 7f0f877fe700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f7406ace0 0x7f0f7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:39.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.532+0000 7f0f877fe700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f7406ace0 0x7f0f7406d190 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0f8815c8c0 tx=0x7f0f78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:39.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.536+0000 7f0f6ffff700 1 -- 172.21.15.97:0/1534412101 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0f7c0542e0 con 0x7f0f880ccc90 2023-07-06T03:56:39.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.773+0000 7f0f8dfff700 1 -- 172.21.15.97:0/1534412101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0f880b25e0 con 0x7f0f880ccc90 2023-07-06T03:56:39.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.774+0000 7f0f6ffff700 1 -- 172.21.15.97:0/1534412101 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0f880b25e0 con 0x7f0f880ccc90 2023-07-06T03:56:39.776 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:39.776 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:56:39.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.776+0000 7f0f8dfff700 1 -- 172.21.15.97:0/1534412101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f7406ace0 msgr2=0x7f0f7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:39.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.776+0000 7f0f8dfff700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f7406ace0 0x7f0f7406d190 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0f8815c8c0 tx=0x7f0f78009040 comp rx=0 tx=0).stop 2023-07-06T03:56:39.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.776+0000 7f0f8dfff700 1 -- 172.21.15.97:0/1534412101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f880ccc90 msgr2=0x7f0f880c2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:39.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.776+0000 7f0f8dfff700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f880ccc90 0x7f0f880c2ef0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b480 tx=0x7f0f7c00b840 comp rx=0 tx=0).stop 2023-07-06T03:56:39.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.776+0000 7f0f8dfff700 1 -- 172.21.15.97:0/1534412101 shutdown_connections 2023-07-06T03:56:39.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.776+0000 7f0f8dfff700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f7406ace0 0x7f0f7406d190 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:39.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.776+0000 7f0f8dfff700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f880cee80 0x7f0f880c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:39.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.776+0000 7f0f8dfff700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f880adfb0 0x7f0f881571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:39.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.776+0000 7f0f8dfff700 1 --2- 172.21.15.97:0/1534412101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f880ccc90 0x7f0f880c2ef0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:39.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.776+0000 7f0f8dfff700 1 -- 172.21.15.97:0/1534412101 >> 172.21.15.97:0/1534412101 conn(0x7f0f8806a060 msgr2=0x7f0f8806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:39.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.777+0000 7f0f8dfff700 1 -- 172.21.15.97:0/1534412101 shutdown_connections 2023-07-06T03:56:39.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:39.777+0000 7f0f8dfff700 1 -- 172.21.15.97:0/1534412101 wait complete. 2023-07-06T03:56:39.796 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:56:39.796 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:56:44.797 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:56:45.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:44.999+0000 7efd67fff700 1 Processor -- start 2023-07-06T03:56:45.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.000+0000 7efd67fff700 1 -- start start 2023-07-06T03:56:45.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.001+0000 7efd67fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd600b4850 0x7efd600cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:45.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.001+0000 7efd67fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efd6015c950 0x7efd600b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:45.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.001+0000 7efd67fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd600cf640 con 0x7efd6006c920 2023-07-06T03:56:45.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.001+0000 7efd67fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd600b5160 con 0x7efd6015c950 2023-07-06T03:56:45.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.001+0000 7efd67fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd600cd4c0 con 0x7efd600b4850 2023-07-06T03:56:45.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.001+0000 7efd65d9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd600b4850 0x7efd600cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:45.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.001+0000 7efd6559a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7efd6015c950 0x7efd600b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:45.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.001+0000 7efd65d9b700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd600b4850 0x7efd600cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:42408/0 (socket says 172.21.15.97:42408) 2023-07-06T03:56:45.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.001+0000 7efd65d9b700 1 -- 172.21.15.97:0/3026461008 learned_addr learned my addr 172.21.15.97:0/3026461008 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:45.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.002+0000 7efd6559a700 1 -- 172.21.15.97:0/3026461008 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd600b4850 msgr2=0x7efd600cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:45.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.002+0000 7efd6559a700 1 --2- 172.21.15.97:0/3026461008 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd600b4850 0x7efd600cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:45.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.002+0000 7efd6559a700 1 -- 172.21.15.97:0/3026461008 >> v1:172.21.15.146:6789/0 conn(0x7efd6006c920 legacy=0x7efd60160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:45.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.002+0000 7efd6559a700 1 -- 172.21.15.97:0/3026461008 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd600cd7a0 con 0x7efd6015c950 2023-07-06T03:56:45.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.002+0000 7efd6559a700 1 --2- 172.21.15.97:0/3026461008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd6015c950 0x7efd600b4c20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7efd50009840 tx=0x7efd50009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d83dfbc4b9c4043f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:45.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.002+0000 7efd64d99700 1 -- 172.21.15.97:0/3026461008 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50011830 con 0x7efd6015c950 2023-07-06T03:56:45.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.002+0000 7efd64d99700 1 -- 172.21.15.97:0/3026461008 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd50011990 con 0x7efd6015c950 2023-07-06T03:56:45.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.002+0000 7efd64d99700 1 -- 172.21.15.97:0/3026461008 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50011d20 con 0x7efd6015c950 2023-07-06T03:56:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.003+0000 7efd67fff700 1 -- 172.21.15.97:0/3026461008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd6015c950 msgr2=0x7efd600b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.003+0000 7efd67fff700 1 --2- 172.21.15.97:0/3026461008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd6015c950 0x7efd600b4c20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7efd50009840 tx=0x7efd50009b50 comp rx=0 tx=0).stop 2023-07-06T03:56:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.003+0000 7efd67fff700 1 -- 172.21.15.97:0/3026461008 shutdown_connections 2023-07-06T03:56:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.003+0000 7efd67fff700 1 --2- 172.21.15.97:0/3026461008 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd600b4850 0x7efd600cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.003+0000 7efd67fff700 1 --2- 172.21.15.97:0/3026461008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd6015c950 0x7efd600b4c20 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.003+0000 7efd67fff700 1 -- 172.21.15.97:0/3026461008 >> 172.21.15.97:0/3026461008 conn(0x7efd6006b410 msgr2=0x7efd600cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.003+0000 7efd67fff700 1 -- 172.21.15.97:0/3026461008 shutdown_connections 2023-07-06T03:56:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.003+0000 7efd67fff700 1 -- 172.21.15.97:0/3026461008 wait complete. 2023-07-06T03:56:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.004+0000 7efd67fff700 1 Processor -- start 2023-07-06T03:56:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.005+0000 7efd67fff700 1 -- start start 2023-07-06T03:56:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.005+0000 7efd67fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd6006c920 0x7efd6015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.005+0000 7efd67fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd600b4850 0x7efd600c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd67fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd6015c950 0x7efd600ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd67fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd600c9110 con 0x7efd6015c950 2023-07-06T03:56:45.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd67fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd600c93b0 con 0x7efd600b4850 2023-07-06T03:56:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd67fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd600c9690 con 0x7efd6006c920 2023-07-06T03:56:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd6659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd6015c950 0x7efd600ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd6659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd6015c950 0x7efd600ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36110/0 (socket says 172.21.15.97:36110) 2023-07-06T03:56:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd6659c700 1 -- 172.21.15.97:0/1704520201 learned_addr learned my addr 172.21.15.97:0/1704520201 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd6559a700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd600b4850 0x7efd600c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd65d9b700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd6006c920 0x7efd6015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd6659c700 1 -- 172.21.15.97:0/1704520201 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd6006c920 msgr2=0x7efd6015b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd6659c700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd6006c920 0x7efd6015b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd6659c700 1 -- 172.21.15.97:0/1704520201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd600b4850 msgr2=0x7efd600c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd6659c700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd600b4850 0x7efd600c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.006+0000 7efd6659c700 1 -- 172.21.15.97:0/1704520201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd500094a0 con 0x7efd6015c950 2023-07-06T03:56:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.007+0000 7efd6659c700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd6015c950 0x7efd600ca3e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7efd5800cbc0 tx=0x7efd5800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.007+0000 7efd567fc700 1 -- 172.21.15.97:0/1704520201 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58003c00 con 0x7efd6015c950 2023-07-06T03:56:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.007+0000 7efd567fc700 1 -- 172.21.15.97:0/1704520201 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd58003d60 con 0x7efd6015c950 2023-07-06T03:56:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.007+0000 7efd567fc700 1 -- 172.21.15.97:0/1704520201 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5800aa20 con 0x7efd6015c950 2023-07-06T03:56:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.007+0000 7efd67fff700 1 -- 172.21.15.97:0/1704520201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6015aed0 con 0x7efd6015c950 2023-07-06T03:56:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.007+0000 7efd67fff700 1 -- 172.21.15.97:0/1704520201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd600b6d30 con 0x7efd6015c950 2023-07-06T03:56:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.008+0000 7efd67fff700 1 -- 172.21.15.97:0/1704520201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6004e340 con 0x7efd6015c950 2023-07-06T03:56:45.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.009+0000 7efd567fc700 1 -- 172.21.15.97:0/1704520201 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efd5800ab80 con 0x7efd6015c950 2023-07-06T03:56:45.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.010+0000 7efd567fc700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd4406ace0 0x7efd4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:45.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.010+0000 7efd567fc700 1 -- 172.21.15.97:0/1704520201 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efd580892a0 con 0x7efd6015c950 2023-07-06T03:56:45.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.010+0000 7efd65d9b700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd4406ace0 0x7efd4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:45.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.011+0000 7efd65d9b700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd4406ace0 0x7efd4406d190 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009bd0 tx=0x7efd5c0092c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.015+0000 7efd567fc700 1 -- 172.21.15.97:0/1704520201 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efd58054490 con 0x7efd6015c950 2023-07-06T03:56:45.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.260+0000 7efd67fff700 1 -- 172.21.15.97:0/1704520201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7efd600d01c0 con 0x7efd6015c950 2023-07-06T03:56:45.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.260+0000 7efd567fc700 1 -- 172.21.15.97:0/1704520201 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7efd58057ab0 con 0x7efd6015c950 2023-07-06T03:56:45.262 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:45.263 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:56:45.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 -- 172.21.15.97:0/1704520201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd4406ace0 msgr2=0x7efd4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:45.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd4406ace0 0x7efd4406d190 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009bd0 tx=0x7efd5c0092c0 comp rx=0 tx=0).stop 2023-07-06T03:56:45.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 -- 172.21.15.97:0/1704520201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd6015c950 msgr2=0x7efd600ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:45.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd6015c950 0x7efd600ca3e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7efd5800cbc0 tx=0x7efd5800ced0 comp rx=0 tx=0).stop 2023-07-06T03:56:45.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 -- 172.21.15.97:0/1704520201 shutdown_connections 2023-07-06T03:56:45.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7efd4406ace0 0x7efd4406d190 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:45.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd6015c950 0x7efd600ca3e0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:45.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7efd6006c920 0x7efd6015b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:45.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 --2- 172.21.15.97:0/1704520201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd600b4850 0x7efd600c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:45.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 -- 172.21.15.97:0/1704520201 >> 172.21.15.97:0/1704520201 conn(0x7efd6006b410 msgr2=0x7efd600d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:45.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 -- 172.21.15.97:0/1704520201 shutdown_connections 2023-07-06T03:56:45.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:45.263+0000 7efd67fff700 1 -- 172.21.15.97:0/1704520201 wait complete. 2023-07-06T03:56:45.282 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:56:45.282 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:56:50.284 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:56:50.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.486+0000 7f5cff1ff700 1 Processor -- start 2023-07-06T03:56:50.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.487+0000 7f5cff1ff700 1 -- start start 2023-07-06T03:56:50.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cff1ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5cf80b4850 0x7f5cf80cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:50.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cff1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5cf815c950 0x7f5cf80b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:50.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cff1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf80cf640 con 0x7f5cf806c920 2023-07-06T03:56:50.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cff1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf80b5160 con 0x7f5cf815c950 2023-07-06T03:56:50.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cff1ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf80cd4c0 con 0x7f5cf80b4850 2023-07-06T03:56:50.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cf7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5cf815c950 0x7f5cf80b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:50.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cf7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5cf815c950 0x7f5cf80b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42410/0 (socket says 172.21.15.97:42410) 2023-07-06T03:56:50.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cf7fff700 1 -- 172.21.15.97:0/1178143313 learned_addr learned my addr 172.21.15.97:0/1178143313 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:50.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cf7fff700 1 -- 172.21.15.97:0/1178143313 >> v2:172.21.15.146:3300/0 conn(0x7f5cf80b4850 msgr2=0x7f5cf80cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:50.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cfcf9b700 1 --2- 172.21.15.97:0/1178143313 >> v2:172.21.15.146:3300/0 conn(0x7f5cf80b4850 0x7f5cf80cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:50.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cf7fff700 1 --2- 172.21.15.97:0/1178143313 >> v2:172.21.15.146:3300/0 conn(0x7f5cf80b4850 0x7f5cf80cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:50.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cf7fff700 1 -- 172.21.15.97:0/1178143313 >> v1:172.21.15.146:6789/0 conn(0x7f5cf806c920 legacy=0x7f5cf8160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:50.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cf7fff700 1 -- 172.21.15.97:0/1178143313 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf80cd7a0 con 0x7f5cf815c950 2023-07-06T03:56:50.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.488+0000 7f5cfcf9b700 1 --2- 172.21.15.97:0/1178143313 >> v2:172.21.15.146:3300/0 conn(0x7f5cf80b4850 0x7f5cf80cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:56:50.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.489+0000 7f5cf7fff700 1 --2- 172.21.15.97:0/1178143313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf815c950 0x7f5cf80b4c20 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f5cf806a680 tx=0x7f5ce4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e43ca43d03794d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:50.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.489+0000 7f5cf77fe700 1 -- 172.21.15.97:0/1178143313 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4004230 con 0x7f5cf815c950 2023-07-06T03:56:50.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.489+0000 7f5cf77fe700 1 -- 172.21.15.97:0/1178143313 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce4004390 con 0x7f5cf815c950 2023-07-06T03:56:50.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.489+0000 7f5cf77fe700 1 -- 172.21.15.97:0/1178143313 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40036a0 con 0x7f5cf815c950 2023-07-06T03:56:50.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.489+0000 7f5cff1ff700 1 -- 172.21.15.97:0/1178143313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf815c950 msgr2=0x7f5cf80b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:50.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.489+0000 7f5cff1ff700 1 --2- 172.21.15.97:0/1178143313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf815c950 0x7f5cf80b4c20 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f5cf806a680 tx=0x7f5ce4009bd0 comp rx=0 tx=0).stop 2023-07-06T03:56:50.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.489+0000 7f5cff1ff700 1 -- 172.21.15.97:0/1178143313 shutdown_connections 2023-07-06T03:56:50.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.489+0000 7f5cff1ff700 1 --2- 172.21.15.97:0/1178143313 >> v2:172.21.15.146:3300/0 conn(0x7f5cf80b4850 0x7f5cf80cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:50.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.489+0000 7f5cff1ff700 1 --2- 172.21.15.97:0/1178143313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf815c950 0x7f5cf80b4c20 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:50.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.489+0000 7f5cff1ff700 1 -- 172.21.15.97:0/1178143313 >> 172.21.15.97:0/1178143313 conn(0x7f5cf806b410 msgr2=0x7f5cf80cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:50.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.490+0000 7f5cff1ff700 1 -- 172.21.15.97:0/1178143313 shutdown_connections 2023-07-06T03:56:50.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.490+0000 7f5cff1ff700 1 -- 172.21.15.97:0/1178143313 wait complete. 2023-07-06T03:56:50.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.491+0000 7f5cff1ff700 1 Processor -- start 2023-07-06T03:56:50.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.491+0000 7f5cff1ff700 1 -- start start 2023-07-06T03:56:50.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.492+0000 7f5cff1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf806c920 0x7f5cf80cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:50.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.492+0000 7f5cff1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf80b4850 0x7f5cf80c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:50.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.492+0000 7f5cff1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf815c950 0x7f5cf80ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:50.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.492+0000 7f5cff1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf815daf0 con 0x7f5cf806c920 2023-07-06T03:56:50.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.492+0000 7f5cff1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf815cee0 con 0x7f5cf815c950 2023-07-06T03:56:50.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.492+0000 7f5cff1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf815d1c0 con 0x7f5cf80b4850 2023-07-06T03:56:50.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.492+0000 7f5cfcf9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf806c920 0x7f5cf80cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:50.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cfcf9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf806c920 0x7f5cf80cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42418/0 (socket says 172.21.15.97:42418) 2023-07-06T03:56:50.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cfcf9b700 1 -- 172.21.15.97:0/2266101323 learned_addr learned my addr 172.21.15.97:0/2266101323 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:50.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cf7fff700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf80b4850 0x7f5cf80c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:50.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cfd79c700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf815c950 0x7f5cf80ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:50.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cfcf9b700 1 -- 172.21.15.97:0/2266101323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf80b4850 msgr2=0x7f5cf80c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:50.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cfcf9b700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf80b4850 0x7f5cf80c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:50.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cfcf9b700 1 -- 172.21.15.97:0/2266101323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf815c950 msgr2=0x7f5cf80ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:50.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cfcf9b700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf815c950 0x7f5cf80ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:50.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cfcf9b700 1 -- 172.21.15.97:0/2266101323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce4009550 con 0x7f5cf806c920 2023-07-06T03:56:50.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cfcf9b700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf806c920 0x7f5cf80cc750 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00c750 tx=0x7f5cec00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:50.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cf57fa700 1 -- 172.21.15.97:0/2266101323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec011800 con 0x7f5cf806c920 2023-07-06T03:56:50.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cf57fa700 1 -- 172.21.15.97:0/2266101323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec011960 con 0x7f5cf806c920 2023-07-06T03:56:50.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cff1ff700 1 -- 172.21.15.97:0/2266101323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf815d4a0 con 0x7f5cf806c920 2023-07-06T03:56:50.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.493+0000 7f5cf57fa700 1 -- 172.21.15.97:0/2266101323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec011cd0 con 0x7f5cf806c920 2023-07-06T03:56:50.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.494+0000 7f5cff1ff700 1 -- 172.21.15.97:0/2266101323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf80ccc90 con 0x7f5cf806c920 2023-07-06T03:56:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.494+0000 7f5cff1ff700 1 -- 172.21.15.97:0/2266101323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf804e340 con 0x7f5cf806c920 2023-07-06T03:56:50.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.495+0000 7f5cf57fa700 1 -- 172.21.15.97:0/2266101323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5cec021350 con 0x7f5cf806c920 2023-07-06T03:56:50.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.497+0000 7f5cf57fa700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cdc06ace0 0x7f5cdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:50.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.497+0000 7f5cf57fa700 1 -- 172.21.15.97:0/2266101323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5cec089520 con 0x7f5cf806c920 2023-07-06T03:56:50.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.497+0000 7f5cf7fff700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cdc06ace0 0x7f5cdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:50.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.497+0000 7f5cf7fff700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cdc06ace0 0x7f5cdc06d190 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009130 tx=0x7f5ce4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:50.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.501+0000 7f5cf57fa700 1 -- 172.21.15.97:0/2266101323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5cec054530 con 0x7f5cf806c920 2023-07-06T03:56:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.736+0000 7f5cff1ff700 1 -- 172.21.15.97:0/2266101323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5cf80d01c0 con 0x7f5cf806c920 2023-07-06T03:56:50.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.736+0000 7f5cf57fa700 1 -- 172.21.15.97:0/2266101323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5cec057b50 con 0x7f5cf806c920 2023-07-06T03:56:50.738 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:50.738 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:56:50.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.738+0000 7f5cff1ff700 1 -- 172.21.15.97:0/2266101323 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cdc06ace0 msgr2=0x7f5cdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:50.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.738+0000 7f5cff1ff700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cdc06ace0 0x7f5cdc06d190 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009130 tx=0x7f5ce4008030 comp rx=0 tx=0).stop 2023-07-06T03:56:50.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.738+0000 7f5cff1ff700 1 -- 172.21.15.97:0/2266101323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf806c920 msgr2=0x7f5cf80cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.738+0000 7f5cff1ff700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf806c920 0x7f5cf80cc750 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00c750 tx=0x7f5cec00cb10 comp rx=0 tx=0).stop 2023-07-06T03:56:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.738+0000 7f5cff1ff700 1 -- 172.21.15.97:0/2266101323 shutdown_connections 2023-07-06T03:56:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.738+0000 7f5cff1ff700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cdc06ace0 0x7f5cdc06d190 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.738+0000 7f5cff1ff700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf815c950 0x7f5cf80ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.738+0000 7f5cff1ff700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf806c920 0x7f5cf80cc750 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.738+0000 7f5cff1ff700 1 --2- 172.21.15.97:0/2266101323 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5cf80b4850 0x7f5cf80c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.738+0000 7f5cff1ff700 1 -- 172.21.15.97:0/2266101323 >> 172.21.15.97:0/2266101323 conn(0x7f5cf806b410 msgr2=0x7f5cf80d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.739+0000 7f5cff1ff700 1 -- 172.21.15.97:0/2266101323 shutdown_connections 2023-07-06T03:56:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:50.739+0000 7f5cff1ff700 1 -- 172.21.15.97:0/2266101323 wait complete. 2023-07-06T03:56:50.758 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:56:50.758 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:56:55.759 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:56:55.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.961+0000 7fe3fdfff700 1 Processor -- start 2023-07-06T03:56:55.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.961+0000 7fe3fdfff700 1 -- start start 2023-07-06T03:56:55.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.962+0000 7fe3fdfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3f80b0000 0x7fe3f80cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:55.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3fdfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3f806c060 0x7fe3f80cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:55.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3fdfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe3f806b170 0x7fe3f8160250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:55.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3fdfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f815cc00 con 0x7fe3f80b0000 2023-07-06T03:56:55.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3fdfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f806c5e0 con 0x7fe3f806b170 2023-07-06T03:56:55.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3fdfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f806c8c0 con 0x7fe3f806c060 2023-07-06T03:56:55.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3f77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3f80b0000 0x7fe3f80cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:55.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3f77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3f80b0000 0x7fe3f80cd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42424/0 (socket says 172.21.15.97:42424) 2023-07-06T03:56:55.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3f77fe700 1 -- 172.21.15.97:0/754693266 learned_addr learned my addr 172.21.15.97:0/754693266 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:55.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3f77fe700 1 -- 172.21.15.97:0/754693266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3f806c060 msgr2=0x7fe3f80cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:55.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3f77fe700 1 --2- 172.21.15.97:0/754693266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3f806c060 0x7fe3f80cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:55.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3f77fe700 1 -- 172.21.15.97:0/754693266 >> v2:172.21.15.146:3300/0 conn(0x7fe3f806b170 msgr2=0x7fe3f8160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:56:55.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3f77fe700 1 --2- 172.21.15.97:0/754693266 >> v2:172.21.15.146:3300/0 conn(0x7fe3f806b170 0x7fe3f8160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:55.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3f77fe700 1 -- 172.21.15.97:0/754693266 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f806cba0 con 0x7fe3f80b0000 2023-07-06T03:56:55.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.963+0000 7fe3f77fe700 1 --2- 172.21.15.97:0/754693266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f80b0000 0x7fe3f80cd4c0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80b2d60 tx=0x7fe3e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81485baf660c730 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:55.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3f67fc700 1 -- 172.21.15.97:0/754693266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800b870 con 0x7fe3f80b0000 2023-07-06T03:56:55.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3f67fc700 1 -- 172.21.15.97:0/754693266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e800b9d0 con 0x7fe3f80b0000 2023-07-06T03:56:55.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3f67fc700 1 -- 172.21.15.97:0/754693266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800bd60 con 0x7fe3f80b0000 2023-07-06T03:56:55.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3fdfff700 1 -- 172.21.15.97:0/754693266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f80b0000 msgr2=0x7fe3f80cd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:55.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3fdfff700 1 --2- 172.21.15.97:0/754693266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f80b0000 0x7fe3f80cd4c0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80b2d60 tx=0x7fe3e8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:56:55.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3fdfff700 1 -- 172.21.15.97:0/754693266 shutdown_connections 2023-07-06T03:56:55.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3fdfff700 1 --2- 172.21.15.97:0/754693266 >> v2:172.21.15.146:3300/0 conn(0x7fe3f806b170 0x7fe3f8160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:55.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3fdfff700 1 --2- 172.21.15.97:0/754693266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f80b0000 0x7fe3f80cd4c0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:55.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3fdfff700 1 --2- 172.21.15.97:0/754693266 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3f806c060 0x7fe3f80cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:55.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3fdfff700 1 -- 172.21.15.97:0/754693266 >> 172.21.15.97:0/754693266 conn(0x7fe3f8066960 msgr2=0x7fe3f806dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:55.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3fdfff700 1 -- 172.21.15.97:0/754693266 shutdown_connections 2023-07-06T03:56:55.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.964+0000 7fe3fdfff700 1 -- 172.21.15.97:0/754693266 wait complete. 2023-07-06T03:56:55.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.966+0000 7fe3fdfff700 1 Processor -- start 2023-07-06T03:56:55.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.966+0000 7fe3fdfff700 1 -- start start 2023-07-06T03:56:55.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.967+0000 7fe3fdfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3f806b170 0x7fe3f815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:55.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.967+0000 7fe3fdfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f806c060 0x7fe3f81534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:55.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.967+0000 7fe3fdfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3f80b0000 0x7fe3f80c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:55.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.967+0000 7fe3fdfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80adf20 con 0x7fe3f806c060 2023-07-06T03:56:55.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.967+0000 7fe3fdfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80c8740 con 0x7fe3f806b170 2023-07-06T03:56:55.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.967+0000 7fe3fdfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80c8a20 con 0x7fe3f80b0000 2023-07-06T03:56:55.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.967+0000 7fe3f6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f806c060 0x7fe3f81534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:55.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.967+0000 7fe3f6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f806c060 0x7fe3f81534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42430/0 (socket says 172.21.15.97:42430) 2023-07-06T03:56:55.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.967+0000 7fe3f6ffd700 1 -- 172.21.15.97:0/1795033064 learned_addr learned my addr 172.21.15.97:0/1795033064 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:56:55.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.967+0000 7fe3f77fe700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3f806b170 0x7fe3f815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:55.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3f7fff700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3f80b0000 0x7fe3f80c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:55.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3f6ffd700 1 -- 172.21.15.97:0/1795033064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3f80b0000 msgr2=0x7fe3f80c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:55.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3f6ffd700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3f80b0000 0x7fe3f80c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:55.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3f6ffd700 1 -- 172.21.15.97:0/1795033064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3f806b170 msgr2=0x7fe3f815cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:55.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3f6ffd700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3f806b170 0x7fe3f815cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:55.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3f6ffd700 1 -- 172.21.15.97:0/1795033064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e8009550 con 0x7fe3f806c060 2023-07-06T03:56:55.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3f7fff700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3f80b0000 0x7fe3f80c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:56:55.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3f6ffd700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f806c060 0x7fe3f81534e0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00b4a0 tx=0x7fe3ec00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:55.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3dffff700 1 -- 172.21.15.97:0/1795033064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec011830 con 0x7fe3f806c060 2023-07-06T03:56:55.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3dffff700 1 -- 172.21.15.97:0/1795033064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec011990 con 0x7fe3f806c060 2023-07-06T03:56:55.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3fdfff700 1 -- 172.21.15.97:0/1795033064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f80ad8d0 con 0x7fe3f806c060 2023-07-06T03:56:55.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3dffff700 1 -- 172.21.15.97:0/1795033064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec011d40 con 0x7fe3f806c060 2023-07-06T03:56:55.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.968+0000 7fe3fdfff700 1 -- 172.21.15.97:0/1795033064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f815d290 con 0x7fe3f806c060 2023-07-06T03:56:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.970+0000 7fe3fdfff700 1 -- 172.21.15.97:0/1795033064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f80ae090 con 0x7fe3f806c060 2023-07-06T03:56:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.971+0000 7fe3dffff700 1 -- 172.21.15.97:0/1795033064 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3ec01a400 con 0x7fe3f806c060 2023-07-06T03:56:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.971+0000 7fe3dffff700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3e406ace0 0x7fe3e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:56:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.972+0000 7fe3dffff700 1 -- 172.21.15.97:0/1795033064 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fe3ec089440 con 0x7fe3f806c060 2023-07-06T03:56:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.972+0000 7fe3f77fe700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3e406ace0 0x7fe3e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:56:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.972+0000 7fe3f77fe700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3e406ace0 0x7fe3e406d190 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8000c00 tx=0x7fe3e8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:56:55.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:55.976+0000 7fe3dffff700 1 -- 172.21.15.97:0/1795033064 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe3ec0544e0 con 0x7fe3f806c060 2023-07-06T03:56:56.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.216+0000 7fe3fdfff700 1 -- 172.21.15.97:0/1795033064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe3f80b2790 con 0x7fe3f806c060 2023-07-06T03:56:56.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.217+0000 7fe3dffff700 1 -- 172.21.15.97:0/1795033064 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe3ec057b00 con 0x7fe3f806c060 2023-07-06T03:56:56.220 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:56:56.220 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:56:56.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.219+0000 7fe3fdfff700 1 -- 172.21.15.97:0/1795033064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3e406ace0 msgr2=0x7fe3e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:56.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.219+0000 7fe3fdfff700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3e406ace0 0x7fe3e406d190 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8000c00 tx=0x7fe3e8008030 comp rx=0 tx=0).stop 2023-07-06T03:56:56.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.219+0000 7fe3fdfff700 1 -- 172.21.15.97:0/1795033064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f806c060 msgr2=0x7fe3f81534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:56:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.219+0000 7fe3fdfff700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f806c060 0x7fe3f81534e0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00b4a0 tx=0x7fe3ec00b7b0 comp rx=0 tx=0).stop 2023-07-06T03:56:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.219+0000 7fe3fdfff700 1 -- 172.21.15.97:0/1795033064 shutdown_connections 2023-07-06T03:56:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.219+0000 7fe3fdfff700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe3e406ace0 0x7fe3e406d190 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.219+0000 7fe3fdfff700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe3f80b0000 0x7fe3f80c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.219+0000 7fe3fdfff700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3f806b170 0x7fe3f815cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.219+0000 7fe3fdfff700 1 --2- 172.21.15.97:0/1795033064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f806c060 0x7fe3f81534e0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:56:56.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.219+0000 7fe3fdfff700 1 -- 172.21.15.97:0/1795033064 >> 172.21.15.97:0/1795033064 conn(0x7fe3f8066960 msgr2=0x7fe3f80b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:56:56.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.220+0000 7fe3fdfff700 1 -- 172.21.15.97:0/1795033064 shutdown_connections 2023-07-06T03:56:56.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:56:56.220+0000 7fe3fdfff700 1 -- 172.21.15.97:0/1795033064 wait complete. 2023-07-06T03:56:56.240 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:56:56.240 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:57:01.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:57:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.442+0000 7fdab6dff700 1 Processor -- start 2023-07-06T03:57:01.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.442+0000 7fdab6dff700 1 -- start start 2023-07-06T03:57:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.443+0000 7fdab6dff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdab015c950 0x7fdab00b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.443+0000 7fdab6dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdab006c920 0x7fdab00d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.443+0000 7fdab6dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab00b5160 con 0x7fdab00b4850 2023-07-06T03:57:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.443+0000 7fdab6dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab00cd4c0 con 0x7fdab006c920 2023-07-06T03:57:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.443+0000 7fdab6dff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab00cd7a0 con 0x7fdab015c950 2023-07-06T03:57:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.443+0000 7fdab4b9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdab00b4850 0x7fdab00cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60730/0 (socket says 172.21.15.97:60730) 2023-07-06T03:57:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.443+0000 7fdab4b9b700 1 -- 172.21.15.97:0/1211134760 learned_addr learned my addr 172.21.15.97:0/1211134760 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.443+0000 7fdab539c700 1 --2- 172.21.15.97:0/1211134760 >> v2:172.21.15.97:3300/0 conn(0x7fdab006c920 0x7fdab00d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:01.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.444+0000 7fdaaffff700 1 --2- 172.21.15.97:0/1211134760 >> v2:172.21.15.146:3300/0 conn(0x7fdab015c950 0x7fdab00b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:01.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.444+0000 7fdab539c700 1 -- 172.21.15.97:0/1211134760 >> v2:172.21.15.146:3300/0 conn(0x7fdab015c950 msgr2=0x7fdab00b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:01.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.444+0000 7fdab539c700 1 --2- 172.21.15.97:0/1211134760 >> v2:172.21.15.146:3300/0 conn(0x7fdab015c950 0x7fdab00b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:01.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.444+0000 7fdab539c700 1 -- 172.21.15.97:0/1211134760 >> v1:172.21.15.97:6789/0 conn(0x7fdab00b4850 legacy=0x7fdab00cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:01.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.444+0000 7fdab539c700 1 -- 172.21.15.97:0/1211134760 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab00cda80 con 0x7fdab006c920 2023-07-06T03:57:01.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.444+0000 7fdaaffff700 1 --2- 172.21.15.97:0/1211134760 >> v2:172.21.15.146:3300/0 conn(0x7fdab015c950 0x7fdab00b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:01.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.444+0000 7fdab539c700 1 --2- 172.21.15.97:0/1211134760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab006c920 0x7fdab00d0d80 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fdab00d3ea0 tx=0x7fdaa800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996889df56c7a202 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.444+0000 7fdaaf7fe700 1 -- 172.21.15.97:0/1211134760 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800e070 con 0x7fdab006c920 2023-07-06T03:57:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.444+0000 7fdaaf7fe700 1 -- 172.21.15.97:0/1211134760 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa800bbd0 con 0x7fdab006c920 2023-07-06T03:57:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.444+0000 7fdaaf7fe700 1 -- 172.21.15.97:0/1211134760 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8012430 con 0x7fdab006c920 2023-07-06T03:57:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.445+0000 7fdab6dff700 1 -- 172.21.15.97:0/1211134760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab006c920 msgr2=0x7fdab00d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.445+0000 7fdab6dff700 1 --2- 172.21.15.97:0/1211134760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab006c920 0x7fdab00d0d80 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fdab00d3ea0 tx=0x7fdaa800b330 comp rx=0 tx=0).stop 2023-07-06T03:57:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.445+0000 7fdab6dff700 1 -- 172.21.15.97:0/1211134760 shutdown_connections 2023-07-06T03:57:01.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.445+0000 7fdab6dff700 1 --2- 172.21.15.97:0/1211134760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab006c920 0x7fdab00d0d80 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.445+0000 7fdab6dff700 1 --2- 172.21.15.97:0/1211134760 >> v2:172.21.15.146:3300/0 conn(0x7fdab015c950 0x7fdab00b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.445+0000 7fdab6dff700 1 -- 172.21.15.97:0/1211134760 >> 172.21.15.97:0/1211134760 conn(0x7fdab006b410 msgr2=0x7fdab00cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.445+0000 7fdab6dff700 1 -- 172.21.15.97:0/1211134760 shutdown_connections 2023-07-06T03:57:01.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.445+0000 7fdab6dff700 1 -- 172.21.15.97:0/1211134760 wait complete. 2023-07-06T03:57:01.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.447+0000 7fdab6dff700 1 Processor -- start 2023-07-06T03:57:01.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.447+0000 7fdab6dff700 1 -- start start 2023-07-06T03:57:01.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab6dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab006c920 0x7fdab014dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:01.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab6dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdab00b4850 0x7fdab00c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:01.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab6dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab015c950 0x7fdab00c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:01.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab6dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab014e0e0 con 0x7fdab015c950 2023-07-06T03:57:01.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab6dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab00c8570 con 0x7fdab006c920 2023-07-06T03:57:01.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab6dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab00cb770 con 0x7fdab00b4850 2023-07-06T03:57:01.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab539c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab015c950 0x7fdab00c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:01.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab539c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab015c950 0x7fdab00c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37448/0 (socket says 172.21.15.97:37448) 2023-07-06T03:57:01.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab539c700 1 -- 172.21.15.97:0/910845432 learned_addr learned my addr 172.21.15.97:0/910845432 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:01.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab4b9b700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab006c920 0x7fdab014dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:01.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab539c700 1 -- 172.21.15.97:0/910845432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdab00b4850 msgr2=0x7fdab00c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:01.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdaaffff700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdab00b4850 0x7fdab00c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:01.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab539c700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdab00b4850 0x7fdab00c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:01.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab539c700 1 -- 172.21.15.97:0/910845432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab006c920 msgr2=0x7fdab014dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:01.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab539c700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab006c920 0x7fdab014dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:01.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab539c700 1 -- 172.21.15.97:0/910845432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa8009a70 con 0x7fdab015c950 2023-07-06T03:57:01.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.448+0000 7fdab4b9b700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab006c920 0x7fdab014dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:01.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.449+0000 7fdab539c700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab015c950 0x7fdab00c8030 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8014040 tx=0x7fdaa8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:01.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.449+0000 7fdaad7fa700 1 -- 172.21.15.97:0/910845432 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8012430 con 0x7fdab015c950 2023-07-06T03:57:01.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.449+0000 7fdaad7fa700 1 -- 172.21.15.97:0/910845432 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa8020070 con 0x7fdab015c950 2023-07-06T03:57:01.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.449+0000 7fdab6dff700 1 -- 172.21.15.97:0/910845432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab00cba50 con 0x7fdab015c950 2023-07-06T03:57:01.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.449+0000 7fdaad7fa700 1 -- 172.21.15.97:0/910845432 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8004280 con 0x7fdab015c950 2023-07-06T03:57:01.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.449+0000 7fdab6dff700 1 -- 172.21.15.97:0/910845432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab00b5fa0 con 0x7fdab015c950 2023-07-06T03:57:01.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.450+0000 7fdab6dff700 1 -- 172.21.15.97:0/910845432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab004e340 con 0x7fdab015c950 2023-07-06T03:57:01.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.451+0000 7fdaad7fa700 1 -- 172.21.15.97:0/910845432 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdaa80043e0 con 0x7fdab015c950 2023-07-06T03:57:01.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.452+0000 7fdaad7fa700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda9406ace0 0x7fda9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:01.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.452+0000 7fdaad7fa700 1 -- 172.21.15.97:0/910845432 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fdaa8022070 con 0x7fdab015c950 2023-07-06T03:57:01.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.452+0000 7fdab4b9b700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda9406ace0 0x7fda9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:01.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.453+0000 7fdab4b9b700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda9406ace0 0x7fda9406d190 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4005910 tx=0x7fdaa4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:01.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.457+0000 7fdaad7fa700 1 -- 172.21.15.97:0/910845432 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdaa8056740 con 0x7fdab015c950 2023-07-06T03:57:01.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.691+0000 7fdab6dff700 1 -- 172.21.15.97:0/910845432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdab00d01c0 con 0x7fdab015c950 2023-07-06T03:57:01.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.692+0000 7fdaad7fa700 1 -- 172.21.15.97:0/910845432 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fdaa8059d60 con 0x7fdab015c950 2023-07-06T03:57:01.695 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:01.695 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:57:01.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 -- 172.21.15.97:0/910845432 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda9406ace0 msgr2=0x7fda9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:01.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda9406ace0 0x7fda9406d190 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4005910 tx=0x7fdaa4009040 comp rx=0 tx=0).stop 2023-07-06T03:57:01.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 -- 172.21.15.97:0/910845432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab015c950 msgr2=0x7fdab00c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:01.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab015c950 0x7fdab00c8030 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8014040 tx=0x7fdaa8004ad0 comp rx=0 tx=0).stop 2023-07-06T03:57:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 -- 172.21.15.97:0/910845432 shutdown_connections 2023-07-06T03:57:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fda9406ace0 0x7fda9406d190 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdab015c950 0x7fdab00c8030 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdab006c920 0x7fdab014dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 --2- 172.21.15.97:0/910845432 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdab00b4850 0x7fdab00c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 -- 172.21.15.97:0/910845432 >> 172.21.15.97:0/910845432 conn(0x7fdab006b410 msgr2=0x7fdab00b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 -- 172.21.15.97:0/910845432 shutdown_connections 2023-07-06T03:57:01.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:01.694+0000 7fdab6dff700 1 -- 172.21.15.97:0/910845432 wait complete. 2023-07-06T03:57:01.714 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:57:01.715 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:57:06.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:57:06.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.917+0000 7f4da73ff700 1 Processor -- start 2023-07-06T03:57:06.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.917+0000 7f4da73ff700 1 -- start start 2023-07-06T03:57:06.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.918+0000 7f4da73ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4da006d520 0x7f4da01671f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:06.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.918+0000 7f4da73ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da015c9f0 con 0x7f4da00ac880 2023-07-06T03:57:06.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.918+0000 7f4da73ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da00cef80 con 0x7f4da006a060 2023-07-06T03:57:06.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.918+0000 7f4da73ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00cf260 con 0x7f4da006d520 2023-07-06T03:57:06.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.918+0000 7f4da499a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4da00ac880 0x7f4da00ce6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60736/0 (socket says 172.21.15.97:60736) 2023-07-06T03:57:06.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.918+0000 7f4da499a700 1 -- 172.21.15.97:0/436079938 learned_addr learned my addr 172.21.15.97:0/436079938 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:06.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.919+0000 7f4da599c700 1 --2- 172.21.15.97:0/436079938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4da006d520 0x7f4da01671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:06.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.919+0000 7f4d97fff700 1 -- 172.21.15.97:0/436079938 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666669969 0 0) 0x7f4da015c9f0 con 0x7f4da00ac880 2023-07-06T03:57:06.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.919+0000 7f4d97fff700 1 -- 172.21.15.97:0/436079938 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d8c0035b0 con 0x7f4da00ac880 2023-07-06T03:57:06.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.919+0000 7f4d97fff700 1 -- 172.21.15.97:0/436079938 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093795377 0 0) 0x7f4da00cef80 con 0x7f4da006a060 2023-07-06T03:57:06.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.919+0000 7f4d97fff700 1 -- 172.21.15.97:0/436079938 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4da015c9f0 con 0x7f4da006a060 2023-07-06T03:57:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.919+0000 7f4da599c700 1 -- 172.21.15.97:0/436079938 >> v1:172.21.15.146:6789/0 conn(0x7f4da006a060 legacy=0x7f4da006a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.919+0000 7f4da599c700 1 -- 172.21.15.97:0/436079938 >> v1:172.21.15.97:6789/0 conn(0x7f4da00ac880 legacy=0x7f4da00ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.919+0000 7f4d97fff700 1 -- 172.21.15.97:0/436079938 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3924378070 0 0) 0x7f4d8c0035b0 con 0x7f4da00ac880 2023-07-06T03:57:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.919+0000 7f4da599c700 1 -- 172.21.15.97:0/436079938 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da00cf540 con 0x7f4da006d520 2023-07-06T03:57:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.919+0000 7f4da599c700 1 --2- 172.21.15.97:0/436079938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4da006d520 0x7f4da01671f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009dc0 tx=0x7f4d9800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c777ee6707acfbde server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.920+0000 7f4d97fff700 1 -- 172.21.15.97:0/436079938 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800e070 con 0x7f4da006d520 2023-07-06T03:57:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.920+0000 7f4d97fff700 1 -- 172.21.15.97:0/436079938 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d98003a20 con 0x7f4da006d520 2023-07-06T03:57:06.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.920+0000 7f4d97fff700 1 -- 172.21.15.97:0/436079938 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90001670 con 0x7f4da006d520 2023-07-06T03:57:06.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.920+0000 7f4da73ff700 1 -- 172.21.15.97:0/436079938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4da006d520 msgr2=0x7f4da01671f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:06.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.920+0000 7f4da73ff700 1 --2- 172.21.15.97:0/436079938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4da006d520 0x7f4da01671f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009dc0 tx=0x7f4d9800b330 comp rx=0 tx=0).stop 2023-07-06T03:57:06.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.920+0000 7f4da73ff700 1 -- 172.21.15.97:0/436079938 shutdown_connections 2023-07-06T03:57:06.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.920+0000 7f4da73ff700 1 --2- 172.21.15.97:0/436079938 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4da006d520 0x7f4da01671f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:06.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.920+0000 7f4da73ff700 1 -- 172.21.15.97:0/436079938 >> 172.21.15.97:0/436079938 conn(0x7f4da00b16c0 msgr2=0x7f4da00b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:06.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.920+0000 7f4da73ff700 1 -- 172.21.15.97:0/436079938 shutdown_connections 2023-07-06T03:57:06.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.921+0000 7f4da73ff700 1 -- 172.21.15.97:0/436079938 wait complete. 2023-07-06T03:57:06.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.922+0000 7f4da73ff700 1 Processor -- start 2023-07-06T03:57:06.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.922+0000 7f4da73ff700 1 -- start start 2023-07-06T03:57:06.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.923+0000 7f4da73ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4da006a060 0x7f4da00b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:06.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.923+0000 7f4da73ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4da006d520 0x7f4da015a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:06.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.923+0000 7f4da73ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4da00ac880 0x7f4da015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:06.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.923+0000 7f4da73ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da015b0a0 con 0x7f4da006d520 2023-07-06T03:57:06.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.923+0000 7f4da73ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00c8570 con 0x7f4da006a060 2023-07-06T03:57:06.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.923+0000 7f4da73ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00c7960 con 0x7f4da00ac880 2023-07-06T03:57:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.923+0000 7f4da499a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4da006d520 0x7f4da015a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.923+0000 7f4da499a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4da006d520 0x7f4da015a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37452/0 (socket says 172.21.15.97:37452) 2023-07-06T03:57:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.923+0000 7f4da499a700 1 -- 172.21.15.97:0/3267714586 learned_addr learned my addr 172.21.15.97:0/3267714586 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4da519b700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4da006a060 0x7f4da00b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4da599c700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4da00ac880 0x7f4da015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4da499a700 1 -- 172.21.15.97:0/3267714586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4da00ac880 msgr2=0x7f4da015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4da499a700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4da00ac880 0x7f4da015ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4da499a700 1 -- 172.21.15.97:0/3267714586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4da006a060 msgr2=0x7f4da00b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4da499a700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4da006a060 0x7f4da00b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4da499a700 1 -- 172.21.15.97:0/3267714586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d98009a70 con 0x7f4da006d520 2023-07-06T03:57:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4da519b700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4da006a060 0x7f4da00b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4da499a700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4da006d520 0x7f4da015a620 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000c4d0 tx=0x7f4d9000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4d95ffb700 1 -- 172.21.15.97:0/3267714586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90004c80 con 0x7f4da006d520 2023-07-06T03:57:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4d95ffb700 1 -- 172.21.15.97:0/3267714586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d90004de0 con 0x7f4da006d520 2023-07-06T03:57:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4da73ff700 1 -- 172.21.15.97:0/3267714586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da00c7c40 con 0x7f4da006d520 2023-07-06T03:57:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4d95ffb700 1 -- 172.21.15.97:0/3267714586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d900072f0 con 0x7f4da006d520 2023-07-06T03:57:06.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.924+0000 7f4da73ff700 1 -- 172.21.15.97:0/3267714586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da00c8200 con 0x7f4da006d520 2023-07-06T03:57:06.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.925+0000 7f4da73ff700 1 -- 172.21.15.97:0/3267714586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da00c8700 con 0x7f4da006d520 2023-07-06T03:57:06.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.926+0000 7f4d95ffb700 1 -- 172.21.15.97:0/3267714586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d90007450 con 0x7f4da006d520 2023-07-06T03:57:06.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.927+0000 7f4d95ffb700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d8406ace0 0x7f4d8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:06.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.927+0000 7f4d95ffb700 1 -- 172.21.15.97:0/3267714586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f4d900893b0 con 0x7f4da006d520 2023-07-06T03:57:06.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.928+0000 7f4da519b700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d8406ace0 0x7f4d8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:06.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.928+0000 7f4da519b700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d8406ace0 0x7f4d8406d190 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c007fd0 tx=0x7f4d9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:06.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:06.932+0000 7f4d95ffb700 1 -- 172.21.15.97:0/3267714586 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4d90054310 con 0x7f4da006d520 2023-07-06T03:57:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.169+0000 7f4da73ff700 1 -- 172.21.15.97:0/3267714586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4da004e340 con 0x7f4da006d520 2023-07-06T03:57:07.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.170+0000 7f4d95ffb700 1 -- 172.21.15.97:0/3267714586 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4d90057930 con 0x7f4da006d520 2023-07-06T03:57:07.173 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:07.173 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:57:07.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 -- 172.21.15.97:0/3267714586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d8406ace0 msgr2=0x7f4d8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:07.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d8406ace0 0x7f4d8406d190 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c007fd0 tx=0x7f4d9c009040 comp rx=0 tx=0).stop 2023-07-06T03:57:07.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 -- 172.21.15.97:0/3267714586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4da006d520 msgr2=0x7f4da015a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:07.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4da006d520 0x7f4da015a620 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000c4d0 tx=0x7f4d9000c890 comp rx=0 tx=0).stop 2023-07-06T03:57:07.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 -- 172.21.15.97:0/3267714586 shutdown_connections 2023-07-06T03:57:07.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d8406ace0 0x7f4d8406d190 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4da00ac880 0x7f4da015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4da006a060 0x7f4da00b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 --2- 172.21.15.97:0/3267714586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4da006d520 0x7f4da015a620 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 -- 172.21.15.97:0/3267714586 >> 172.21.15.97:0/3267714586 conn(0x7f4da00b16c0 msgr2=0x7f4da00ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 -- 172.21.15.97:0/3267714586 shutdown_connections 2023-07-06T03:57:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:07.172+0000 7f4da73ff700 1 -- 172.21.15.97:0/3267714586 wait complete. 2023-07-06T03:57:07.192 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:57:07.193 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:57:07.462 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:57:07.466 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:57:12.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:57:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.395+0000 7f1b955ff700 1 Processor -- start 2023-07-06T03:57:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.395+0000 7f1b955ff700 1 -- start start 2023-07-06T03:57:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.396+0000 7f1b955ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1b900b4850 0x7f1b900cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.396+0000 7f1b955ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b9006c920 0x7f1b900d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b955ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b900cf640 con 0x7f1b9015c950 2023-07-06T03:57:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b955ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b900cd4c0 con 0x7f1b900b4850 2023-07-06T03:57:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b955ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900cd7a0 con 0x7f1b9006c920 2023-07-06T03:57:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b8e7fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1b9015c950 0x7f1b900b4c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58328/0 (socket says 172.21.15.97:58328) 2023-07-06T03:57:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b8e7fc700 1 -- 172.21.15.97:0/4227988723 learned_addr learned my addr 172.21.15.97:0/4227988723 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:12.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b8effd700 1 --2- 172.21.15.97:0/4227988723 >> v2:172.21.15.146:3300/0 conn(0x7f1b900b4850 0x7f1b900cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b8f7fe700 1 --2- 172.21.15.97:0/4227988723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b9006c920 0x7f1b900d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b8dffb700 1 -- 172.21.15.97:0/4227988723 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153588813 0 0) 0x7f1b900cf640 con 0x7f1b9015c950 2023-07-06T03:57:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b8dffb700 1 -- 172.21.15.97:0/4227988723 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b780035b0 con 0x7f1b9015c950 2023-07-06T03:57:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b8effd700 1 -- 172.21.15.97:0/4227988723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b9006c920 msgr2=0x7f1b900d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b8effd700 1 --2- 172.21.15.97:0/4227988723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b9006c920 0x7f1b900d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b8effd700 1 -- 172.21.15.97:0/4227988723 >> v1:172.21.15.97:6789/0 conn(0x7f1b9015c950 legacy=0x7f1b900b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.397+0000 7f1b8effd700 1 -- 172.21.15.97:0/4227988723 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900cda80 con 0x7f1b900b4850 2023-07-06T03:57:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.398+0000 7f1b8effd700 1 --2- 172.21.15.97:0/4227988723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b900b4850 0x7f1b900cf100 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009840 tx=0x7f1b80009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad78222a9e385ee9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.398+0000 7f1b8dffb700 1 -- 172.21.15.97:0/4227988723 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80011830 con 0x7f1b900b4850 2023-07-06T03:57:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.398+0000 7f1b8dffb700 1 -- 172.21.15.97:0/4227988723 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b80011990 con 0x7f1b900b4850 2023-07-06T03:57:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.398+0000 7f1b8dffb700 1 -- 172.21.15.97:0/4227988723 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80011d00 con 0x7f1b900b4850 2023-07-06T03:57:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.399+0000 7f1b955ff700 1 -- 172.21.15.97:0/4227988723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b900b4850 msgr2=0x7f1b900cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.399+0000 7f1b955ff700 1 --2- 172.21.15.97:0/4227988723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b900b4850 0x7f1b900cf100 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009840 tx=0x7f1b80009b50 comp rx=0 tx=0).stop 2023-07-06T03:57:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.399+0000 7f1b955ff700 1 -- 172.21.15.97:0/4227988723 shutdown_connections 2023-07-06T03:57:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.399+0000 7f1b955ff700 1 --2- 172.21.15.97:0/4227988723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b900b4850 0x7f1b900cf100 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.399+0000 7f1b955ff700 1 --2- 172.21.15.97:0/4227988723 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b9006c920 0x7f1b900d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.399+0000 7f1b955ff700 1 -- 172.21.15.97:0/4227988723 >> 172.21.15.97:0/4227988723 conn(0x7f1b9006b410 msgr2=0x7f1b900cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.399+0000 7f1b955ff700 1 -- 172.21.15.97:0/4227988723 shutdown_connections 2023-07-06T03:57:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.399+0000 7f1b955ff700 1 -- 172.21.15.97:0/4227988723 wait complete. 2023-07-06T03:57:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.400+0000 7f1b955ff700 1 Processor -- start 2023-07-06T03:57:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.401+0000 7f1b955ff700 1 -- start start 2023-07-06T03:57:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b955ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b9006c920 0x7f1b901563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b955ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b900b4850 0x7f1b900c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b955ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b9015c950 0x7f1b900caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b955ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900cb460 con 0x7f1b900b4850 2023-07-06T03:57:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b955ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900b70f0 con 0x7f1b9015c950 2023-07-06T03:57:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b955ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900b7390 con 0x7f1b9006c920 2023-07-06T03:57:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b8e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b900b4850 0x7f1b900c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b8e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b900b4850 0x7f1b900c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58018/0 (socket says 172.21.15.97:58018) 2023-07-06T03:57:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b8e7fc700 1 -- 172.21.15.97:0/2500129372 learned_addr learned my addr 172.21.15.97:0/2500129372 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b8effd700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b9006c920 0x7f1b901563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b8f7fe700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b9015c950 0x7f1b900caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b8e7fc700 1 -- 172.21.15.97:0/2500129372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b9006c920 msgr2=0x7f1b901563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b8e7fc700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b9006c920 0x7f1b901563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b8e7fc700 1 -- 172.21.15.97:0/2500129372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b9015c950 msgr2=0x7f1b900caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b8e7fc700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b9015c950 0x7f1b900caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b8e7fc700 1 -- 172.21.15.97:0/2500129372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b800094a0 con 0x7f1b900b4850 2023-07-06T03:57:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.402+0000 7f1b8f7fe700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b9015c950 0x7f1b900caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.403+0000 7f1b8e7fc700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b900b4850 0x7f1b900c7950 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400c420 tx=0x7f1b8400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.403+0000 7f1b777fe700 1 -- 172.21.15.97:0/2500129372 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84005d70 con 0x7f1b900b4850 2023-07-06T03:57:12.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.403+0000 7f1b777fe700 1 -- 172.21.15.97:0/2500129372 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b840070d0 con 0x7f1b900b4850 2023-07-06T03:57:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.403+0000 7f1b777fe700 1 -- 172.21.15.97:0/2500129372 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84007440 con 0x7f1b900b4850 2023-07-06T03:57:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.403+0000 7f1b955ff700 1 -- 172.21.15.97:0/2500129372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b900b7670 con 0x7f1b900b4850 2023-07-06T03:57:12.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.403+0000 7f1b955ff700 1 -- 172.21.15.97:0/2500129372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b900b7c30 con 0x7f1b900b4850 2023-07-06T03:57:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.404+0000 7f1b955ff700 1 -- 172.21.15.97:0/2500129372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9004e340 con 0x7f1b900b4850 2023-07-06T03:57:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.405+0000 7f1b777fe700 1 -- 172.21.15.97:0/2500129372 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b840075a0 con 0x7f1b900b4850 2023-07-06T03:57:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.406+0000 7f1b777fe700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b7c06ace0 0x7f1b7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.407+0000 7f1b777fe700 1 -- 172.21.15.97:0/2500129372 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f1b84088750 con 0x7f1b900b4850 2023-07-06T03:57:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.407+0000 7f1b8effd700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b7c06ace0 0x7f1b7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.407+0000 7f1b8effd700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b7c06ace0 0x7f1b7c06d190 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000c00 tx=0x7f1b80008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:12.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.411+0000 7f1b777fe700 1 -- 172.21.15.97:0/2500129372 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1b84053880 con 0x7f1b900b4850 2023-07-06T03:57:12.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.646+0000 7f1b955ff700 1 -- 172.21.15.97:0/2500129372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1b900d01c0 con 0x7f1b900b4850 2023-07-06T03:57:12.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.647+0000 7f1b777fe700 1 -- 172.21.15.97:0/2500129372 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1b84056ea0 con 0x7f1b900b4850 2023-07-06T03:57:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:12.650 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:57:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 -- 172.21.15.97:0/2500129372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b7c06ace0 msgr2=0x7f1b7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b7c06ace0 0x7f1b7c06d190 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000c00 tx=0x7f1b80008030 comp rx=0 tx=0).stop 2023-07-06T03:57:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 -- 172.21.15.97:0/2500129372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b900b4850 msgr2=0x7f1b900c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b900b4850 0x7f1b900c7950 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400c420 tx=0x7f1b8400c730 comp rx=0 tx=0).stop 2023-07-06T03:57:12.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 -- 172.21.15.97:0/2500129372 shutdown_connections 2023-07-06T03:57:12.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1b7c06ace0 0x7f1b7c06d190 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:12.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b9015c950 0x7f1b900caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:12.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1b9006c920 0x7f1b901563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:12.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 --2- 172.21.15.97:0/2500129372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b900b4850 0x7f1b900c7950 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:12.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 -- 172.21.15.97:0/2500129372 >> 172.21.15.97:0/2500129372 conn(0x7f1b9006b410 msgr2=0x7f1b900b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:12.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 -- 172.21.15.97:0/2500129372 shutdown_connections 2023-07-06T03:57:12.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:12.649+0000 7f1b955ff700 1 -- 172.21.15.97:0/2500129372 wait complete. 2023-07-06T03:57:12.670 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:57:12.670 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:57:17.671 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:57:17.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.873+0000 7f5bbcbff700 1 Processor -- start 2023-07-06T03:57:17.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.873+0000 7f5bbcbff700 1 -- start start 2023-07-06T03:57:17.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.874+0000 7f5bbcbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5bb80adfb0 0x7f5bb80ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:17.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.875+0000 7f5bbcbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb80c9320 con 0x7f5bb80cecb0 2023-07-06T03:57:17.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.875+0000 7f5bbcbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb80cd160 con 0x7f5bb80ccc90 2023-07-06T03:57:17.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.875+0000 7f5bbcbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb80cdce0 con 0x7f5bb80adfb0 2023-07-06T03:57:17.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.875+0000 7f5bb659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5bb80adfb0 0x7f5bb80ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:17.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.875+0000 7f5baffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5bb80cecb0 0x7f5bb80cc580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58340/0 (socket says 172.21.15.97:58340) 2023-07-06T03:57:17.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.875+0000 7f5baffff700 1 -- 172.21.15.97:0/3293298706 learned_addr learned my addr 172.21.15.97:0/3293298706 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:17.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.875+0000 7f5bb659c700 1 -- 172.21.15.97:0/3293298706 >> v1:172.21.15.146:6789/0 conn(0x7f5bb80ccc90 legacy=0x7f5bb8070100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:17.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.875+0000 7f5bb659c700 1 -- 172.21.15.97:0/3293298706 >> v1:172.21.15.97:6789/0 conn(0x7f5bb80cecb0 legacy=0x7f5bb80cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:17.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.875+0000 7f5bb659c700 1 -- 172.21.15.97:0/3293298706 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb80cd440 con 0x7f5bb80adfb0 2023-07-06T03:57:17.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.875+0000 7f5bb5d9b700 1 -- 172.21.15.97:0/3293298706 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828101326 0 0) 0x7f5bb80c9320 con 0x7f5bb80cecb0 2023-07-06T03:57:17.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.876+0000 7f5bb659c700 1 --2- 172.21.15.97:0/3293298706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bb80adfb0 0x7f5bb80ce770 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800bae0 tx=0x7f5ba800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d7fe7017a6aaaf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:17.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.876+0000 7f5bb5d9b700 1 -- 172.21.15.97:0/3293298706 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba800dcf0 con 0x7f5bb80adfb0 2023-07-06T03:57:17.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.876+0000 7f5bb5d9b700 1 -- 172.21.15.97:0/3293298706 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba800de50 con 0x7f5bb80adfb0 2023-07-06T03:57:17.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.876+0000 7f5bb5d9b700 1 -- 172.21.15.97:0/3293298706 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8004250 con 0x7f5bb80adfb0 2023-07-06T03:57:17.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.876+0000 7f5bbcbff700 1 -- 172.21.15.97:0/3293298706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bb80adfb0 msgr2=0x7f5bb80ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.876+0000 7f5bbcbff700 1 --2- 172.21.15.97:0/3293298706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bb80adfb0 0x7f5bb80ce770 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800bae0 tx=0x7f5ba800bdf0 comp rx=0 tx=0).stop 2023-07-06T03:57:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.876+0000 7f5bbcbff700 1 -- 172.21.15.97:0/3293298706 shutdown_connections 2023-07-06T03:57:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.876+0000 7f5bbcbff700 1 --2- 172.21.15.97:0/3293298706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bb80adfb0 0x7f5bb80ce770 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.876+0000 7f5bbcbff700 1 -- 172.21.15.97:0/3293298706 >> 172.21.15.97:0/3293298706 conn(0x7f5bb806a060 msgr2=0x7f5bb806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.876+0000 7f5bbcbff700 1 -- 172.21.15.97:0/3293298706 shutdown_connections 2023-07-06T03:57:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.877+0000 7f5bbcbff700 1 -- 172.21.15.97:0/3293298706 wait complete. 2023-07-06T03:57:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.879+0000 7f5bbcbff700 1 Processor -- start 2023-07-06T03:57:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.879+0000 7f5bbcbff700 1 -- start start 2023-07-06T03:57:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.880+0000 7f5bbcbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb80adfb0 0x7f5bb81505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.880+0000 7f5bbcbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bb80ccc90 0x7f5bb80c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.880+0000 7f5bbcbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bb80cecb0 0x7f5bb80c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.880+0000 7f5bbcbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb80c4910 con 0x7f5bb80cecb0 2023-07-06T03:57:17.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.880+0000 7f5bbcbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb80b2a80 con 0x7f5bb80adfb0 2023-07-06T03:57:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.880+0000 7f5bbcbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb80b2d60 con 0x7f5bb80ccc90 2023-07-06T03:57:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5bb6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bb80cecb0 0x7f5bb80c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5bb6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bb80cecb0 0x7f5bb80c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58048/0 (socket says 172.21.15.97:58048) 2023-07-06T03:57:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5bb6d9d700 1 -- 172.21.15.97:0/2787399586 learned_addr learned my addr 172.21.15.97:0/2787399586 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5bb659c700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb80adfb0 0x7f5bb81505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5baffff700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bb80ccc90 0x7f5bb80c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5bb6d9d700 1 -- 172.21.15.97:0/2787399586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bb80ccc90 msgr2=0x7f5bb80c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5bb6d9d700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bb80ccc90 0x7f5bb80c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5bb6d9d700 1 -- 172.21.15.97:0/2787399586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb80adfb0 msgr2=0x7f5bb81505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5bb6d9d700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb80adfb0 0x7f5bb81505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5bb6d9d700 1 -- 172.21.15.97:0/2787399586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba800b740 con 0x7f5bb80cecb0 2023-07-06T03:57:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5baffff700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bb80ccc90 0x7f5bb80c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:17.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5bb6d9d700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bb80cecb0 0x7f5bb80c3f00 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000cc10 tx=0x7f5bb0004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5baeffd700 1 -- 172.21.15.97:0/2787399586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000ab80 con 0x7f5bb80cecb0 2023-07-06T03:57:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5baeffd700 1 -- 172.21.15.97:0/2787399586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb000ace0 con 0x7f5bb80cecb0 2023-07-06T03:57:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.881+0000 7f5bbcbff700 1 -- 172.21.15.97:0/2787399586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb80b3040 con 0x7f5bb80cecb0 2023-07-06T03:57:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.882+0000 7f5baeffd700 1 -- 172.21.15.97:0/2787399586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0012660 con 0x7f5bb80cecb0 2023-07-06T03:57:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.882+0000 7f5bbcbff700 1 -- 172.21.15.97:0/2787399586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb8150b10 con 0x7f5bb80cecb0 2023-07-06T03:57:17.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.883+0000 7f5bbcbff700 1 -- 172.21.15.97:0/2787399586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb80b3560 con 0x7f5bb80cecb0 2023-07-06T03:57:17.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.883+0000 7f5baeffd700 1 -- 172.21.15.97:0/2787399586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5bb00127c0 con 0x7f5bb80cecb0 2023-07-06T03:57:17.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.884+0000 7f5baeffd700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5b9806ace0 0x7f5b9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:17.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.884+0000 7f5baeffd700 1 -- 172.21.15.97:0/2787399586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5bb0089990 con 0x7f5bb80cecb0 2023-07-06T03:57:17.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.885+0000 7f5bb659c700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5b9806ace0 0x7f5b9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:17.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.885+0000 7f5bb659c700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5b9806ace0 0x7f5b9806d190 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800b2c0 tx=0x7f5ba801b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:17.889+0000 7f5baeffd700 1 -- 172.21.15.97:0/2787399586 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5bb80b3560 con 0x7f5bb80cecb0 2023-07-06T03:57:18.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.126+0000 7f5bbcbff700 1 -- 172.21.15.97:0/2787399586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5bb80b3560 con 0x7f5bb80cecb0 2023-07-06T03:57:18.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.127+0000 7f5baeffd700 1 -- 172.21.15.97:0/2787399586 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5bb0016040 con 0x7f5bb80cecb0 2023-07-06T03:57:18.129 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:18.129 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:57:18.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 -- 172.21.15.97:0/2787399586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5b9806ace0 msgr2=0x7f5b9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:18.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5b9806ace0 0x7f5b9806d190 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800b2c0 tx=0x7f5ba801b040 comp rx=0 tx=0).stop 2023-07-06T03:57:18.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 -- 172.21.15.97:0/2787399586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bb80cecb0 msgr2=0x7f5bb80c3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:18.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bb80cecb0 0x7f5bb80c3f00 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000cc10 tx=0x7f5bb0004990 comp rx=0 tx=0).stop 2023-07-06T03:57:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 -- 172.21.15.97:0/2787399586 shutdown_connections 2023-07-06T03:57:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5b9806ace0 0x7f5b9806d190 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bb80cecb0 0x7f5bb80c3f00 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bb80adfb0 0x7f5bb81505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 --2- 172.21.15.97:0/2787399586 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5bb80ccc90 0x7f5bb80c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 -- 172.21.15.97:0/2787399586 >> 172.21.15.97:0/2787399586 conn(0x7f5bb806a060 msgr2=0x7f5bb80c94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:18.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 -- 172.21.15.97:0/2787399586 shutdown_connections 2023-07-06T03:57:18.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:18.129+0000 7f5bbcbff700 1 -- 172.21.15.97:0/2787399586 wait complete. 2023-07-06T03:57:18.149 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:57:18.149 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:57:23.150 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:57:23.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.350+0000 7f96bfdff700 1 Processor -- start 2023-07-06T03:57:23.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.350+0000 7f96bfdff700 1 -- start start 2023-07-06T03:57:23.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.351+0000 7f96bfdff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96b80adfb0 0x7f96b80ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:23.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.351+0000 7f96bfdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80ccc90 0x7f96b8070100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:23.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.351+0000 7f96bfdff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b80c9320 con 0x7f96b80cecb0 2023-07-06T03:57:23.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.351+0000 7f96bfdff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b80cdce0 con 0x7f96b80adfb0 2023-07-06T03:57:23.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.351+0000 7f96bfdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80cf120 con 0x7f96b80ccc90 2023-07-06T03:57:23.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.352+0000 7f96bdb9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96b80adfb0 0x7f96b80ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:23.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.352+0000 7f96be39c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80ccc90 0x7f96b8070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:23.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.352+0000 7f96bdb9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96b80adfb0 0x7f96b80ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:54834/0 (socket says 172.21.15.97:54834) 2023-07-06T03:57:23.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.352+0000 7f96be39c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80ccc90 0x7f96b8070100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:40396/0 (socket says 172.21.15.97:40396) 2023-07-06T03:57:23.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.352+0000 7f96bdb9b700 1 -- 172.21.15.97:0/2736527547 learned_addr learned my addr 172.21.15.97:0/2736527547 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:23.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.352+0000 7f96bd39a700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f96b80cecb0 0x7f96b80cc580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:54990/0 (socket says 172.21.15.97:54990) 2023-07-06T03:57:23.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.352+0000 7f96bdb9b700 1 -- 172.21.15.97:0/2736527547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80ccc90 msgr2=0x7f96b8070100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:23.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.352+0000 7f96bdb9b700 1 --2- 172.21.15.97:0/2736527547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80ccc90 0x7f96b8070100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:23.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.352+0000 7f96bdb9b700 1 -- 172.21.15.97:0/2736527547 >> v1:172.21.15.146:6789/0 conn(0x7f96b80cecb0 legacy=0x7f96b80cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:23.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.352+0000 7f96bdb9b700 1 -- 172.21.15.97:0/2736527547 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b80cd160 con 0x7f96b80adfb0 2023-07-06T03:57:23.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.352+0000 7f96be39c700 1 --2- 172.21.15.97:0/2736527547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80ccc90 0x7f96b8070100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:57:23.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.353+0000 7f96bdb9b700 1 --2- 172.21.15.97:0/2736527547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b80adfb0 0x7f96b80ce770 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f96b40098f0 tx=0x7f96b4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbcf70c6b0afe07c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:23.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.353+0000 7f96bcb99700 1 -- 172.21.15.97:0/2736527547 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4011830 con 0x7f96b80adfb0 2023-07-06T03:57:23.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.353+0000 7f96bcb99700 1 -- 172.21.15.97:0/2736527547 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b4011990 con 0x7f96b80adfb0 2023-07-06T03:57:23.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.353+0000 7f96bcb99700 1 -- 172.21.15.97:0/2736527547 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4011d20 con 0x7f96b80adfb0 2023-07-06T03:57:23.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.353+0000 7f96bfdff700 1 -- 172.21.15.97:0/2736527547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b80adfb0 msgr2=0x7f96b80ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:23.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.353+0000 7f96bfdff700 1 --2- 172.21.15.97:0/2736527547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b80adfb0 0x7f96b80ce770 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f96b40098f0 tx=0x7f96b4009c00 comp rx=0 tx=0).stop 2023-07-06T03:57:23.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.354+0000 7f96bfdff700 1 -- 172.21.15.97:0/2736527547 shutdown_connections 2023-07-06T03:57:23.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.354+0000 7f96bfdff700 1 --2- 172.21.15.97:0/2736527547 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80ccc90 0x7f96b8070100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:23.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.354+0000 7f96bfdff700 1 --2- 172.21.15.97:0/2736527547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b80adfb0 0x7f96b80ce770 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:23.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.354+0000 7f96bfdff700 1 -- 172.21.15.97:0/2736527547 >> 172.21.15.97:0/2736527547 conn(0x7f96b806a060 msgr2=0x7f96b806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:23.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.354+0000 7f96bfdff700 1 -- 172.21.15.97:0/2736527547 shutdown_connections 2023-07-06T03:57:23.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.354+0000 7f96bfdff700 1 -- 172.21.15.97:0/2736527547 wait complete. 2023-07-06T03:57:23.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.355+0000 7f96bfdff700 1 Processor -- start 2023-07-06T03:57:23.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.356+0000 7f96bfdff700 1 -- start start 2023-07-06T03:57:23.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.356+0000 7f96bfdff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80adfb0 0x7f96b81e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:23.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.356+0000 7f96bfdff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b80ccc90 0x7f96b8157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:23.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.356+0000 7f96bfdff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96b80cecb0 0x7f96b8157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.356+0000 7f96bfdff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80b0eb0 con 0x7f96b80cecb0 2023-07-06T03:57:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.356+0000 7f96bfdff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80b10e0 con 0x7f96b80ccc90 2023-07-06T03:57:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.356+0000 7f96bfdff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80b0790 con 0x7f96b80adfb0 2023-07-06T03:57:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96be39c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96b80cecb0 0x7f96b8157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96be39c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96b80cecb0 0x7f96b8157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52822/0 (socket says 172.21.15.97:52822) 2023-07-06T03:57:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96be39c700 1 -- 172.21.15.97:0/2819403376 learned_addr learned my addr 172.21.15.97:0/2819403376 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96bdb9b700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80adfb0 0x7f96b81e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:23.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96bd39a700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b80ccc90 0x7f96b8157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:23.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96be39c700 1 -- 172.21.15.97:0/2819403376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80adfb0 msgr2=0x7f96b81e1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:23.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96be39c700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80adfb0 0x7f96b81e1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:23.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96be39c700 1 -- 172.21.15.97:0/2819403376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b80ccc90 msgr2=0x7f96b8157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:23.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96be39c700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b80ccc90 0x7f96b8157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:23.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96be39c700 1 -- 172.21.15.97:0/2819403376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b0009a70 con 0x7f96b80cecb0 2023-07-06T03:57:23.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96bd39a700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b80ccc90 0x7f96b8157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:23.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.357+0000 7f96be39c700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96b80cecb0 0x7f96b8157e80 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f96b000efc0 tx=0x7f96b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:23.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.358+0000 7f96ae7fc700 1 -- 172.21.15.97:0/2819403376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b000cd40 con 0x7f96b80cecb0 2023-07-06T03:57:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.358+0000 7f96ae7fc700 1 -- 172.21.15.97:0/2819403376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b000f040 con 0x7f96b80cecb0 2023-07-06T03:57:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.358+0000 7f96bfdff700 1 -- 172.21.15.97:0/2819403376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b4009550 con 0x7f96b80cecb0 2023-07-06T03:57:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.358+0000 7f96ae7fc700 1 -- 172.21.15.97:0/2819403376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b00146c0 con 0x7f96b80cecb0 2023-07-06T03:57:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.358+0000 7f96bfdff700 1 -- 172.21.15.97:0/2819403376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b80b13c0 con 0x7f96b80cecb0 2023-07-06T03:57:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.359+0000 7f96bfdff700 1 -- 172.21.15.97:0/2819403376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b80b0cb0 con 0x7f96b80cecb0 2023-07-06T03:57:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.360+0000 7f96ae7fc700 1 -- 172.21.15.97:0/2819403376 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96b0014820 con 0x7f96b80cecb0 2023-07-06T03:57:23.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.361+0000 7f96ae7fc700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f969c06ace0 0x7f969c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:23.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.361+0000 7f96ae7fc700 1 -- 172.21.15.97:0/2819403376 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f96b008a170 con 0x7f96b80cecb0 2023-07-06T03:57:23.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.361+0000 7f96bdb9b700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f969c06ace0 0x7f969c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:23.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.362+0000 7f96bdb9b700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f969c06ace0 0x7f969c06d190 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f96b4014040 tx=0x7f96b4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:23.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.366+0000 7f96ae7fc700 1 -- 172.21.15.97:0/2819403376 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f96b80b0cb0 con 0x7f96b80cecb0 2023-07-06T03:57:23.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.605+0000 7f96bfdff700 1 -- 172.21.15.97:0/2819403376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f96b80b0cb0 con 0x7f96b80cecb0 2023-07-06T03:57:23.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.605+0000 7f96ae7fc700 1 -- 172.21.15.97:0/2819403376 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f96b80b0cb0 con 0x7f96b80cecb0 2023-07-06T03:57:23.607 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:23.608 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:57:23.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 -- 172.21.15.97:0/2819403376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f969c06ace0 msgr2=0x7f969c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:23.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f969c06ace0 0x7f969c06d190 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f96b4014040 tx=0x7f96b4008030 comp rx=0 tx=0).stop 2023-07-06T03:57:23.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 -- 172.21.15.97:0/2819403376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96b80cecb0 msgr2=0x7f96b8157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:23.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96b80cecb0 0x7f96b8157e80 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f96b000efc0 tx=0x7f96b000c3a0 comp rx=0 tx=0).stop 2023-07-06T03:57:23.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 -- 172.21.15.97:0/2819403376 shutdown_connections 2023-07-06T03:57:23.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f969c06ace0 0x7f969c06d190 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:23.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96b80cecb0 0x7f96b8157e80 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:23.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f96b80adfb0 0x7f96b81e1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:23.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 --2- 172.21.15.97:0/2819403376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96b80ccc90 0x7f96b8157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:23.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 -- 172.21.15.97:0/2819403376 >> 172.21.15.97:0/2819403376 conn(0x7f96b806a060 msgr2=0x7f96b80cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:23.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 -- 172.21.15.97:0/2819403376 shutdown_connections 2023-07-06T03:57:23.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:23.608+0000 7f96bfdff700 1 -- 172.21.15.97:0/2819403376 wait complete. 2023-07-06T03:57:23.627 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:57:23.628 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:57:28.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:57:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.832+0000 7f0f795ff700 1 Processor -- start 2023-07-06T03:57:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.833+0000 7f0f795ff700 1 -- start start 2023-07-06T03:57:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.834+0000 7f0f795ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0f740cdbd0 0x7f0f740cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.834+0000 7f0f795ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7406c5e0 con 0x7f0f7406c060 2023-07-06T03:57:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.834+0000 7f0f795ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7406c8c0 con 0x7f0f740b0000 2023-07-06T03:57:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.834+0000 7f0f795ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7415cc00 con 0x7f0f740cdbd0 2023-07-06T03:57:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.834+0000 7f0f737fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0f7406c060 0x7f0f740cbf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40244/0 (socket says 172.21.15.97:40244) 2023-07-06T03:57:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.834+0000 7f0f737fe700 1 -- 172.21.15.97:0/4247771754 learned_addr learned my addr 172.21.15.97:0/4247771754 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.834+0000 7f0f6bfff700 1 --2- 172.21.15.97:0/4247771754 >> v2:172.21.15.146:3300/0 conn(0x7f0f740cdbd0 0x7f0f740cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040020548 0 0) 0x7f0f7406c5e0 con 0x7f0f7406c060 2023-07-06T03:57:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f600035b0 con 0x7f0f7406c060 2023-07-06T03:57:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1077399042 0 0) 0x7f0f600035b0 con 0x7f0f7406c060 2023-07-06T03:57:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 >> v2:172.21.15.146:3300/0 conn(0x7f0f740cdbd0 msgr2=0x7f0f740cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 --2- 172.21.15.97:0/4247771754 >> v2:172.21.15.146:3300/0 conn(0x7f0f740cdbd0 0x7f0f740cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 >> v1:172.21.15.146:6789/0 conn(0x7f0f740b0000 legacy=0x7f0f740cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7406cba0 con 0x7f0f7406c060 2023-07-06T03:57:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f0f6c0019f0 con 0x7f0f7406c060 2023-07-06T03:57:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 >> v1:172.21.15.97:6789/0 conn(0x7f0f7406c060 legacy=0x7f0f740cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 --2- 172.21.15.97:0/4247771754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f60003a00 0x7f0f60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 --2- 172.21.15.97:0/4247771754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f600043f0 0x7f0f600048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 --2- 172.21.15.97:0/4247771754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f600050b0 0x7f0f6000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6000c000 con 0x7f0f600050b0 2023-07-06T03:57:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6000c140 con 0x7f0f600043f0 2023-07-06T03:57:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6000c300 con 0x7f0f60003a00 2023-07-06T03:57:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f737fe700 1 --2- 172.21.15.97:0/4247771754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f600050b0 0x7f0f6000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f72ffd700 1 --2- 172.21.15.97:0/4247771754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f60003a00 0x7f0f60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.835+0000 7f0f6bfff700 1 --2- 172.21.15.97:0/4247771754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f600043f0 0x7f0f600048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.836+0000 7f0f737fe700 1 -- 172.21.15.97:0/4247771754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f60003a00 msgr2=0x7f0f60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:28.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.836+0000 7f0f737fe700 1 --2- 172.21.15.97:0/4247771754 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f60003a00 0x7f0f60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:28.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.836+0000 7f0f737fe700 1 -- 172.21.15.97:0/4247771754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f600043f0 msgr2=0x7f0f600048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:28.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.836+0000 7f0f737fe700 1 --2- 172.21.15.97:0/4247771754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f600043f0 0x7f0f600048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:28.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.836+0000 7f0f737fe700 1 -- 172.21.15.97:0/4247771754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f600035b0 con 0x7f0f600050b0 2023-07-06T03:57:28.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.836+0000 7f0f737fe700 1 -- 172.21.15.97:0/4247771754 reap_dead start 2023-07-06T03:57:28.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.836+0000 7f0f737fe700 1 --2- 172.21.15.97:0/4247771754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f600050b0 0x7f0f6000ba10 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005b80 tx=0x7f0f60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e3876d4014a6ef server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:28.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.836+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0055f0 con 0x7f0f600050b0 2023-07-06T03:57:28.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.836+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c00db70 con 0x7f0f600050b0 2023-07-06T03:57:28.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.836+0000 7f0f727fc700 1 -- 172.21.15.97:0/4247771754 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c015420 con 0x7f0f600050b0 2023-07-06T03:57:28.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.837+0000 7f0f795ff700 1 -- 172.21.15.97:0/4247771754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f600050b0 msgr2=0x7f0f6000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:28.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.837+0000 7f0f795ff700 1 --2- 172.21.15.97:0/4247771754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f600050b0 0x7f0f6000ba10 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005b80 tx=0x7f0f60005880 comp rx=0 tx=0).stop 2023-07-06T03:57:28.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.837+0000 7f0f795ff700 1 -- 172.21.15.97:0/4247771754 shutdown_connections 2023-07-06T03:57:28.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.837+0000 7f0f795ff700 1 --2- 172.21.15.97:0/4247771754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f600050b0 0x7f0f6000ba10 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:28.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.837+0000 7f0f795ff700 1 -- 172.21.15.97:0/4247771754 >> 172.21.15.97:0/4247771754 conn(0x7f0f74066960 msgr2=0x7f0f7406dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:28.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.837+0000 7f0f795ff700 1 -- 172.21.15.97:0/4247771754 shutdown_connections 2023-07-06T03:57:28.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.837+0000 7f0f795ff700 1 -- 172.21.15.97:0/4247771754 wait complete. 2023-07-06T03:57:28.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.839+0000 7f0f795ff700 1 Processor -- start 2023-07-06T03:57:28.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.839+0000 7f0f795ff700 1 -- start start 2023-07-06T03:57:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f795ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f600050b0 0x7f0f740af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f795ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f740ca120 0x7f0f74151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f795ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f741e5f40 0x7f0f741e63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f795ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f740b3e80 con 0x7f0f741e5f40 2023-07-06T03:57:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f795ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f741e6920 con 0x7f0f740ca120 2023-07-06T03:57:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f795ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f740ac990 con 0x7f0f600050b0 2023-07-06T03:57:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f737fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f741e5f40 0x7f0f741e63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:28.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f737fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f741e5f40 0x7f0f741e63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52832/0 (socket says 172.21.15.97:52832) 2023-07-06T03:57:28.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f737fe700 1 -- 172.21.15.97:0/3800423578 learned_addr learned my addr 172.21.15.97:0/3800423578 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:28.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f737fe700 1 -- 172.21.15.97:0/3800423578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f600050b0 msgr2=0x7f0f740af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:28.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f737fe700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f600050b0 0x7f0f740af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:28.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f737fe700 1 -- 172.21.15.97:0/3800423578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f740ca120 msgr2=0x7f0f74151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:28.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f737fe700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f740ca120 0x7f0f74151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:28.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.840+0000 7f0f737fe700 1 -- 172.21.15.97:0/3800423578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f740caba0 con 0x7f0f741e5f40 2023-07-06T03:57:28.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.841+0000 7f0f737fe700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f741e5f40 0x7f0f741e63e0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c004ea0 tx=0x7f0f6c004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:28.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.841+0000 7f0f6b7fe700 1 -- 172.21.15.97:0/3800423578 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c01a030 con 0x7f0f741e5f40 2023-07-06T03:57:28.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.841+0000 7f0f6b7fe700 1 -- 172.21.15.97:0/3800423578 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c00dce0 con 0x7f0f741e5f40 2023-07-06T03:57:28.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.841+0000 7f0f6b7fe700 1 -- 172.21.15.97:0/3800423578 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c01d040 con 0x7f0f741e5f40 2023-07-06T03:57:28.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.841+0000 7f0f795ff700 1 -- 172.21.15.97:0/3800423578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f740ca5e0 con 0x7f0f741e5f40 2023-07-06T03:57:28.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.841+0000 7f0f795ff700 1 -- 172.21.15.97:0/3800423578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f740b3aa0 con 0x7f0f741e5f40 2023-07-06T03:57:28.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.842+0000 7f0f795ff700 1 -- 172.21.15.97:0/3800423578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7404e340 con 0x7f0f741e5f40 2023-07-06T03:57:28.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.843+0000 7f0f6b7fe700 1 -- 172.21.15.97:0/3800423578 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f6c015870 con 0x7f0f741e5f40 2023-07-06T03:57:28.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.844+0000 7f0f6b7fe700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f5406ace0 0x7f0f5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:28.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.844+0000 7f0f6b7fe700 1 -- 172.21.15.97:0/3800423578 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f0f6c08ab10 con 0x7f0f741e5f40 2023-07-06T03:57:28.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.844+0000 7f0f72ffd700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f5406ace0 0x7f0f5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:28.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.845+0000 7f0f72ffd700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f5406ace0 0x7f0f5406d190 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0f64007fd0 tx=0x7f0f64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:28.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:28.848+0000 7f0f6b7fe700 1 -- 172.21.15.97:0/3800423578 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0f6c055bd0 con 0x7f0f741e5f40 2023-07-06T03:57:29.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.091+0000 7f0f795ff700 1 -- 172.21.15.97:0/3800423578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0f740b3fc0 con 0x7f0f741e5f40 2023-07-06T03:57:29.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.091+0000 7f0f6b7fe700 1 -- 172.21.15.97:0/3800423578 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0f740b3fc0 con 0x7f0f741e5f40 2023-07-06T03:57:29.093 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:29.093 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:57:29.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.093+0000 7f0f795ff700 1 -- 172.21.15.97:0/3800423578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f5406ace0 msgr2=0x7f0f5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:29.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.093+0000 7f0f795ff700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f5406ace0 0x7f0f5406d190 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0f64007fd0 tx=0x7f0f64009040 comp rx=0 tx=0).stop 2023-07-06T03:57:29.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.093+0000 7f0f795ff700 1 -- 172.21.15.97:0/3800423578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f741e5f40 msgr2=0x7f0f741e63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.093+0000 7f0f795ff700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f741e5f40 0x7f0f741e63e0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c004ea0 tx=0x7f0f6c004e10 comp rx=0 tx=0).stop 2023-07-06T03:57:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.093+0000 7f0f795ff700 1 -- 172.21.15.97:0/3800423578 shutdown_connections 2023-07-06T03:57:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.093+0000 7f0f795ff700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f0f5406ace0 0x7f0f5406d190 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.093+0000 7f0f795ff700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f741e5f40 0x7f0f741e63e0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.093+0000 7f0f795ff700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f0f600050b0 0x7f0f740af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.093+0000 7f0f795ff700 1 --2- 172.21.15.97:0/3800423578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f740ca120 0x7f0f74151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.093+0000 7f0f795ff700 1 -- 172.21.15.97:0/3800423578 >> 172.21.15.97:0/3800423578 conn(0x7f0f74066960 msgr2=0x7f0f740d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.094+0000 7f0f795ff700 1 -- 172.21.15.97:0/3800423578 shutdown_connections 2023-07-06T03:57:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:29.094+0000 7f0f795ff700 1 -- 172.21.15.97:0/3800423578 wait complete. 2023-07-06T03:57:29.113 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:57:29.113 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:57:34.115 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:57:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.316+0000 7f5db63ff700 1 Processor -- start 2023-07-06T03:57:34.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.316+0000 7f5db63ff700 1 -- start start 2023-07-06T03:57:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.318+0000 7f5db63ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5db015e610 0x7f5db015a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.318+0000 7f5db63ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db006e7f0 con 0x7f5db015e240 2023-07-06T03:57:34.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.318+0000 7f5db63ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db00cedb0 con 0x7f5db015a670 2023-07-06T03:57:34.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.318+0000 7f5db63ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db00ce7a0 con 0x7f5db015e610 2023-07-06T03:57:34.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.318+0000 7f5daffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5db015e240 0x7f5db00d14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40084/0 (socket says 172.21.15.97:40084) 2023-07-06T03:57:34.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.318+0000 7f5daffff700 1 -- 172.21.15.97:0/1179565198 learned_addr learned my addr 172.21.15.97:0/1179565198 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.318+0000 7f5daf7fe700 1 --2- 172.21.15.97:0/1179565198 >> v2:172.21.15.146:3300/0 conn(0x7f5db015e610 0x7f5db015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.318+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101011422 0 0) 0x7f5db006e7f0 con 0x7f5db015e240 2023-07-06T03:57:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.318+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d980035b0 con 0x7f5db015e240 2023-07-06T03:57:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2344192536 0 0) 0x7f5d980035b0 con 0x7f5db015e240 2023-07-06T03:57:34.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 >> v2:172.21.15.146:3300/0 conn(0x7f5db015e610 msgr2=0x7f5db015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:34.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 --2- 172.21.15.97:0/1179565198 >> v2:172.21.15.146:3300/0 conn(0x7f5db015e610 0x7f5db015a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:34.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 >> v1:172.21.15.146:6789/0 conn(0x7f5db015a670 legacy=0x7f5db015aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:34.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db00d0250 con 0x7f5db015e240 2023-07-06T03:57:34.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f5da0001670 con 0x7f5db015e240 2023-07-06T03:57:34.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 >> v1:172.21.15.97:6789/0 conn(0x7f5db015e240 legacy=0x7f5db00d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:34.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 --2- 172.21.15.97:0/1179565198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d98003a00 0x7f5d98003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:34.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 --2- 172.21.15.97:0/1179565198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d98004850 0x7f5d98004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:34.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 --2- 172.21.15.97:0/1179565198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d98005200 0x7f5d9800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:34.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db00ce7a0 con 0x7f5d98004850 2023-07-06T03:57:34.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9800c080 con 0x7f5d98005200 2023-07-06T03:57:34.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9800c240 con 0x7f5d98003a00 2023-07-06T03:57:34.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daf7fe700 1 --2- 172.21.15.97:0/1179565198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d98004850 0x7f5d98004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:34.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daffff700 1 --2- 172.21.15.97:0/1179565198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d98003a00 0x7f5d98003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:34.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5db499c700 1 --2- 172.21.15.97:0/1179565198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d98005200 0x7f5d9800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:34.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daf7fe700 1 -- 172.21.15.97:0/1179565198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d98003a00 msgr2=0x7f5d98003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daf7fe700 1 --2- 172.21.15.97:0/1179565198 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d98003a00 0x7f5d98003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daf7fe700 1 -- 172.21.15.97:0/1179565198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d98005200 msgr2=0x7f5d9800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daf7fe700 1 --2- 172.21.15.97:0/1179565198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d98005200 0x7f5d9800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5db499c700 1 -- 172.21.15.97:0/1179565198 reap_dead start 2023-07-06T03:57:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.319+0000 7f5daf7fe700 1 -- 172.21.15.97:0/1179565198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d980035b0 con 0x7f5d98004850 2023-07-06T03:57:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.320+0000 7f5daf7fe700 1 --2- 172.21.15.97:0/1179565198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d98004850 0x7f5d98004cc0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009b60 tx=0x7f5d98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624c865f12e32189 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.320+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da400e6c0 con 0x7f5d98004850 2023-07-06T03:57:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.320+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da400e820 con 0x7f5d98004850 2023-07-06T03:57:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.320+0000 7f5daeffd700 1 -- 172.21.15.97:0/1179565198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da400eb90 con 0x7f5d98004850 2023-07-06T03:57:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.320+0000 7f5db63ff700 1 -- 172.21.15.97:0/1179565198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d98004850 msgr2=0x7f5d98004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.320+0000 7f5db63ff700 1 --2- 172.21.15.97:0/1179565198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d98004850 0x7f5d98004cc0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009b60 tx=0x7f5d98005880 comp rx=0 tx=0).stop 2023-07-06T03:57:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.320+0000 7f5db63ff700 1 -- 172.21.15.97:0/1179565198 shutdown_connections 2023-07-06T03:57:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.320+0000 7f5db63ff700 1 --2- 172.21.15.97:0/1179565198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d98004850 0x7f5d98004cc0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.320+0000 7f5db63ff700 1 -- 172.21.15.97:0/1179565198 >> 172.21.15.97:0/1179565198 conn(0x7f5db0066960 msgr2=0x7f5db006dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.321+0000 7f5db63ff700 1 -- 172.21.15.97:0/1179565198 shutdown_connections 2023-07-06T03:57:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.321+0000 7f5db63ff700 1 -- 172.21.15.97:0/1179565198 wait complete. 2023-07-06T03:57:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.322+0000 7f5db63ff700 1 Processor -- start 2023-07-06T03:57:34.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.322+0000 7f5db63ff700 1 -- start start 2023-07-06T03:57:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.323+0000 7f5db63ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d98004850 0x7f5db00cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.323+0000 7f5db63ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db00cdb70 0x7f5db00b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.323+0000 7f5db63ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5db01f2170 0x7f5db00b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.323+0000 7f5db63ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db00cdfe0 con 0x7f5db00cdb70 2023-07-06T03:57:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.323+0000 7f5db63ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db00ccad0 con 0x7f5db01f2170 2023-07-06T03:57:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.323+0000 7f5db63ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db00ccdb0 con 0x7f5d98004850 2023-07-06T03:57:34.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.323+0000 7f5daf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db00cdb70 0x7f5db00b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5daf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db00cdb70 0x7f5db00b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39612/0 (socket says 172.21.15.97:39612) 2023-07-06T03:57:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5daf7fe700 1 -- 172.21.15.97:0/4093884785 learned_addr learned my addr 172.21.15.97:0/4093884785 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5db499c700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5db01f2170 0x7f5db00b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5daffff700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d98004850 0x7f5db00cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5daf7fe700 1 -- 172.21.15.97:0/4093884785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d98004850 msgr2=0x7f5db00cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5daf7fe700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d98004850 0x7f5db00cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5daf7fe700 1 -- 172.21.15.97:0/4093884785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5db01f2170 msgr2=0x7f5db00b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5daf7fe700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5db01f2170 0x7f5db00b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5daf7fe700 1 -- 172.21.15.97:0/4093884785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db00cd090 con 0x7f5db00cdb70 2023-07-06T03:57:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5daf7fe700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db00cdb70 0x7f5db00b64b0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c050 tx=0x7f5da40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5dacff9700 1 -- 172.21.15.97:0/4093884785 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da400e6c0 con 0x7f5db00cdb70 2023-07-06T03:57:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5dacff9700 1 -- 172.21.15.97:0/4093884785 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da4003ad0 con 0x7f5db00cdb70 2023-07-06T03:57:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.324+0000 7f5db63ff700 1 -- 172.21.15.97:0/4093884785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db00cd370 con 0x7f5db00cdb70 2023-07-06T03:57:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.325+0000 7f5dacff9700 1 -- 172.21.15.97:0/4093884785 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4003e80 con 0x7f5db00cdb70 2023-07-06T03:57:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.325+0000 7f5db63ff700 1 -- 172.21.15.97:0/4093884785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db01f25e0 con 0x7f5db00cdb70 2023-07-06T03:57:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.326+0000 7f5db63ff700 1 -- 172.21.15.97:0/4093884785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db00cd9c0 con 0x7f5db00cdb70 2023-07-06T03:57:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.327+0000 7f5dacff9700 1 -- 172.21.15.97:0/4093884785 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5da4013470 con 0x7f5db00cdb70 2023-07-06T03:57:34.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.327+0000 7f5dacff9700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d9c06ace0 0x7f5d9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:34.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.328+0000 7f5dacff9700 1 -- 172.21.15.97:0/4093884785 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5da408ad20 con 0x7f5db00cdb70 2023-07-06T03:57:34.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.328+0000 7f5daffff700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d9c06ace0 0x7f5d9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:34.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.328+0000 7f5daffff700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d9c06ace0 0x7f5d9c06d190 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5da0000c00 tx=0x7f5da0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:34.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.332+0000 7f5dacff9700 1 -- 172.21.15.97:0/4093884785 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5da4055ec0 con 0x7f5db00cdb70 2023-07-06T03:57:34.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.569+0000 7f5db63ff700 1 -- 172.21.15.97:0/4093884785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5db00cdfe0 con 0x7f5db00cdb70 2023-07-06T03:57:34.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.569+0000 7f5dacff9700 1 -- 172.21.15.97:0/4093884785 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5db00cdfe0 con 0x7f5db00cdb70 2023-07-06T03:57:34.571 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:34.572 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:57:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 -- 172.21.15.97:0/4093884785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d9c06ace0 msgr2=0x7f5d9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:34.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d9c06ace0 0x7f5d9c06d190 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5da0000c00 tx=0x7f5da0009040 comp rx=0 tx=0).stop 2023-07-06T03:57:34.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 -- 172.21.15.97:0/4093884785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db00cdb70 msgr2=0x7f5db00b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:34.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db00cdb70 0x7f5db00b64b0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c050 tx=0x7f5da40047c0 comp rx=0 tx=0).stop 2023-07-06T03:57:34.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 -- 172.21.15.97:0/4093884785 shutdown_connections 2023-07-06T03:57:34.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5d9c06ace0 0x7f5d9c06d190 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:34.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5db01f2170 0x7f5db00b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:34.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d98004850 0x7f5db00cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:34.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 --2- 172.21.15.97:0/4093884785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5db00cdb70 0x7f5db00b64b0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:34.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 -- 172.21.15.97:0/4093884785 >> 172.21.15.97:0/4093884785 conn(0x7f5db0066960 msgr2=0x7f5db014ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:34.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 -- 172.21.15.97:0/4093884785 shutdown_connections 2023-07-06T03:57:34.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:34.572+0000 7f5db63ff700 1 -- 172.21.15.97:0/4093884785 wait complete. 2023-07-06T03:57:34.591 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:57:34.592 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=70 retry_count=0)... 2023-07-06T03:57:37.660 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:57:37.665 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:57:39.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T03:57:39.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.793+0000 7fb5fdfff700 1 Processor -- start 2023-07-06T03:57:39.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.793+0000 7fb5fdfff700 1 -- start start 2023-07-06T03:57:39.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.794+0000 7fb5fdfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb5f815c950 0x7fb5f80b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:39.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.794+0000 7fb5fdfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f806c920 0x7fb5f80d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:39.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.794+0000 7fb5fdfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f80b5160 con 0x7fb5f80b4850 2023-07-06T03:57:39.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.794+0000 7fb5fdfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f80cd4c0 con 0x7fb5f815c950 2023-07-06T03:57:39.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.794+0000 7fb5fdfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80cd7a0 con 0x7fb5f806c920 2023-07-06T03:57:39.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.795+0000 7fb5f6ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb5f815c950 0x7fb5f80b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:39.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.795+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f806c920 0x7fb5f80d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:39.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.795+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f806c920 0x7fb5f80d0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:54408/0 (socket says 172.21.15.97:54408) 2023-07-06T03:57:39.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.795+0000 7fb5f7fff700 1 -- 172.21.15.97:0/1704484235 learned_addr learned my addr 172.21.15.97:0/1704484235 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:39.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.795+0000 7fb5f7fff700 1 -- 172.21.15.97:0/1704484235 >> v2:172.21.15.146:3300/0 conn(0x7fb5f815c950 msgr2=0x7fb5f80b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:39.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.795+0000 7fb5f7fff700 1 --2- 172.21.15.97:0/1704484235 >> v2:172.21.15.146:3300/0 conn(0x7fb5f815c950 0x7fb5f80b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:39.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.795+0000 7fb5f7fff700 1 -- 172.21.15.97:0/1704484235 >> v1:172.21.15.146:6789/0 conn(0x7fb5f80b4850 legacy=0x7fb5f80cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:39.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.795+0000 7fb5f7fff700 1 -- 172.21.15.97:0/1704484235 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f80cda80 con 0x7fb5f806c920 2023-07-06T03:57:39.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.795+0000 7fb5f6ffd700 1 --2- 172.21.15.97:0/1704484235 >> v2:172.21.15.146:3300/0 conn(0x7fb5f815c950 0x7fb5f80b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:57:39.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.796+0000 7fb5f7fff700 1 --2- 172.21.15.97:0/1704484235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f806c920 0x7fb5f80d0d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009e50 tx=0x7fb5f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b15314f0c6ca61 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:39.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.796+0000 7fb5f67fc700 1 -- 172.21.15.97:0/1704484235 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000e070 con 0x7fb5f806c920 2023-07-06T03:57:39.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.796+0000 7fb5f67fc700 1 -- 172.21.15.97:0/1704484235 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f000b910 con 0x7fb5f806c920 2023-07-06T03:57:39.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.796+0000 7fb5f67fc700 1 -- 172.21.15.97:0/1704484235 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000bcc0 con 0x7fb5f806c920 2023-07-06T03:57:39.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.796+0000 7fb5fdfff700 1 -- 172.21.15.97:0/1704484235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f806c920 msgr2=0x7fb5f80d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.796+0000 7fb5fdfff700 1 --2- 172.21.15.97:0/1704484235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f806c920 0x7fb5f80d0d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009e50 tx=0x7fb5f000b330 comp rx=0 tx=0).stop 2023-07-06T03:57:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.796+0000 7fb5fdfff700 1 -- 172.21.15.97:0/1704484235 shutdown_connections 2023-07-06T03:57:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.796+0000 7fb5fdfff700 1 --2- 172.21.15.97:0/1704484235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f806c920 0x7fb5f80d0d80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.796+0000 7fb5fdfff700 1 --2- 172.21.15.97:0/1704484235 >> v2:172.21.15.146:3300/0 conn(0x7fb5f815c950 0x7fb5f80b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.797+0000 7fb5fdfff700 1 -- 172.21.15.97:0/1704484235 >> 172.21.15.97:0/1704484235 conn(0x7fb5f806b410 msgr2=0x7fb5f80cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.797+0000 7fb5fdfff700 1 -- 172.21.15.97:0/1704484235 shutdown_connections 2023-07-06T03:57:39.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.797+0000 7fb5fdfff700 1 -- 172.21.15.97:0/1704484235 wait complete. 2023-07-06T03:57:39.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.798+0000 7fb5fdfff700 1 Processor -- start 2023-07-06T03:57:39.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.798+0000 7fb5fdfff700 1 -- start start 2023-07-06T03:57:39.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.799+0000 7fb5fdfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f806c920 0x7fb5f81542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:39.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.799+0000 7fb5fdfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5f80b4850 0x7fb5f80c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:39.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.799+0000 7fb5fdfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f815c950 0x7fb5f80c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:39.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.799+0000 7fb5fdfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80cbe70 con 0x7fb5f806c920 2023-07-06T03:57:39.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.799+0000 7fb5fdfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80cc150 con 0x7fb5f80b4850 2023-07-06T03:57:39.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.799+0000 7fb5fdfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80cc430 con 0x7fb5f815c950 2023-07-06T03:57:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.800+0000 7fb5f77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f806c920 0x7fb5f81542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.800+0000 7fb5f77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f806c920 0x7fb5f81542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39628/0 (socket says 172.21.15.97:39628) 2023-07-06T03:57:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.800+0000 7fb5f77fe700 1 -- 172.21.15.97:0/4262873636 learned_addr learned my addr 172.21.15.97:0/4262873636 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.800+0000 7fb5f7fff700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f815c950 0x7fb5f80c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.800+0000 7fb5f6ffd700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5f80b4850 0x7fb5f80c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:39.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.800+0000 7fb5f77fe700 1 -- 172.21.15.97:0/4262873636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f815c950 msgr2=0x7fb5f80c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:39.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.800+0000 7fb5f77fe700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f815c950 0x7fb5f80c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:39.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.800+0000 7fb5f77fe700 1 -- 172.21.15.97:0/4262873636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5f80b4850 msgr2=0x7fb5f80c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:39.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.800+0000 7fb5f77fe700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5f80b4850 0x7fb5f80c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:39.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.800+0000 7fb5f77fe700 1 -- 172.21.15.97:0/4262873636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0094a0 con 0x7fb5f806c920 2023-07-06T03:57:39.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.800+0000 7fb5f77fe700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f806c920 0x7fb5f81542f0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c4e0 tx=0x7fb5e800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:39.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.801+0000 7fb5dffff700 1 -- 172.21.15.97:0/4262873636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80052d0 con 0x7fb5f806c920 2023-07-06T03:57:39.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.801+0000 7fb5dffff700 1 -- 172.21.15.97:0/4262873636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e8005430 con 0x7fb5f806c920 2023-07-06T03:57:39.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.801+0000 7fb5fdfff700 1 -- 172.21.15.97:0/4262873636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f0009b30 con 0x7fb5f806c920 2023-07-06T03:57:39.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.801+0000 7fb5dffff700 1 -- 172.21.15.97:0/4262873636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80038c0 con 0x7fb5f806c920 2023-07-06T03:57:39.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.801+0000 7fb5fdfff700 1 -- 172.21.15.97:0/4262873636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f80cc9f0 con 0x7fb5f806c920 2023-07-06T03:57:39.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.802+0000 7fb5fdfff700 1 -- 172.21.15.97:0/4262873636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f806d070 con 0x7fb5f806c920 2023-07-06T03:57:39.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.803+0000 7fb5dffff700 1 -- 172.21.15.97:0/4262873636 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb5e8003a20 con 0x7fb5f806c920 2023-07-06T03:57:39.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.804+0000 7fb5dffff700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5e406ad10 0x7fb5e406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:39.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.804+0000 7fb5dffff700 1 -- 172.21.15.97:0/4262873636 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fb5e8088850 con 0x7fb5f806c920 2023-07-06T03:57:39.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.804+0000 7fb5f6ffd700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5e406ad10 0x7fb5e406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:39.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.804+0000 7fb5f6ffd700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5e406ad10 0x7fb5e406d1c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80cafd0 tx=0x7fb5ec009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:39.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:39.808+0000 7fb5dffff700 1 -- 172.21.15.97:0/4262873636 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb5e80538d0 con 0x7fb5f806c920 2023-07-06T03:57:40.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.047+0000 7fb5fdfff700 1 -- 172.21.15.97:0/4262873636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb5f804e340 con 0x7fb5f806c920 2023-07-06T03:57:40.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.047+0000 7fb5dffff700 1 -- 172.21.15.97:0/4262873636 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb5e8056ef0 con 0x7fb5f806c920 2023-07-06T03:57:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:40.049 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T03:57:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.049+0000 7fb5fdfff700 1 -- 172.21.15.97:0/4262873636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5e406ad10 msgr2=0x7fb5e406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.049+0000 7fb5fdfff700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5e406ad10 0x7fb5e406d1c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80cafd0 tx=0x7fb5ec009210 comp rx=0 tx=0).stop 2023-07-06T03:57:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.049+0000 7fb5fdfff700 1 -- 172.21.15.97:0/4262873636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f806c920 msgr2=0x7fb5f81542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.049+0000 7fb5fdfff700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f806c920 0x7fb5f81542f0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c4e0 tx=0x7fb5e800c7f0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.049+0000 7fb5fdfff700 1 -- 172.21.15.97:0/4262873636 shutdown_connections 2023-07-06T03:57:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.049+0000 7fb5fdfff700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb5e406ad10 0x7fb5e406d1c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.049+0000 7fb5fdfff700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb5f815c950 0x7fb5f80c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.049+0000 7fb5fdfff700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f806c920 0x7fb5f81542f0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.049+0000 7fb5fdfff700 1 --2- 172.21.15.97:0/4262873636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb5f80b4850 0x7fb5f80c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.049+0000 7fb5fdfff700 1 -- 172.21.15.97:0/4262873636 >> 172.21.15.97:0/4262873636 conn(0x7fb5f806b410 msgr2=0x7fb5f80cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:40.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.049+0000 7fb5fdfff700 1 -- 172.21.15.97:0/4262873636 shutdown_connections 2023-07-06T03:57:40.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.050+0000 7fb5fdfff700 1 -- 172.21.15.97:0/4262873636 wait complete. 2023-07-06T03:57:40.068 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-07-06T03:57:40.069 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T03:57:40.069 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:57:40.183 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T03:57:40.289 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T03:57:40.289 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:57:40.289 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:57:40.335 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:57:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.567+0000 7fdecb3ff700 1 Processor -- start 2023-07-06T03:57:40.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.567+0000 7fdecb3ff700 1 -- start start 2023-07-06T03:57:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.568+0000 7fdecb3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec406a770 0x7fdec406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.568+0000 7fdecb3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec40cab30 con 0x7fdec415c540 2023-07-06T03:57:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.568+0000 7fdecb3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec40ca240 con 0x7fdec4158ac0 2023-07-06T03:57:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.568+0000 7fdecb3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40702a0 con 0x7fdec406a770 2023-07-06T03:57:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdec999c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdec415c540 0x7fdec406ce10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60206/0 (socket says 172.21.15.97:60206) 2023-07-06T03:57:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdec999c700 1 -- 172.21.15.97:0/1628492784 learned_addr learned my addr 172.21.15.97:0/1628492784 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:40.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdec899a700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec406a770 0x7fdec406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363054275 0 0) 0x7fdec40cab30 con 0x7fdec415c540 2023-07-06T03:57:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeb00034d0 con 0x7fdec415c540 2023-07-06T03:57:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071625480 0 0) 0x7fdec40ca240 con 0x7fdec4158ac0 2023-07-06T03:57:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec40cab30 con 0x7fdec4158ac0 2023-07-06T03:57:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 378686707 0 0) 0x7fdeb00034d0 con 0x7fdec415c540 2023-07-06T03:57:40.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec406a770 msgr2=0x7fdec406fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdebbfff700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec406a770 0x7fdec406fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 >> v1:172.21.15.146:6789/0 conn(0x7fdec4158ac0 legacy=0x7fdec406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec415c9b0 con 0x7fdec415c540 2023-07-06T03:57:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.569+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fdebc0019f0 con 0x7fdec415c540 2023-07-06T03:57:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 >> v1:172.21.15.97:6789/0 conn(0x7fdec415c540 legacy=0x7fdec406ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdebbfff700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdeb0003cb0 0x7fdeb00040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdebbfff700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 0x7fdeb000ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:40.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdebbfff700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec406a770 0x7fdeb000d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40702a0 con 0x7fdeb00089c0 2023-07-06T03:57:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb000dc60 con 0x7fdeb0003cb0 2023-07-06T03:57:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb000de20 con 0x7fdec406a770 2023-07-06T03:57:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdec899a700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 0x7fdeb000ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdec999c700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec406a770 0x7fdeb000d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdec919b700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdeb0003cb0 0x7fdeb00040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:40.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdec899a700 1 -- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec406a770 msgr2=0x7fdeb000d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdec899a700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec406a770 0x7fdeb000d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdec899a700 1 -- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdeb0003cb0 msgr2=0x7fdeb00040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdec899a700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdeb0003cb0 0x7fdeb00040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdec899a700 1 -- 172.21.15.97:0/1628492784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb00034d0 con 0x7fdeb00089c0 2023-07-06T03:57:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.570+0000 7fdec999c700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec406a770 0x7fdeb000d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.571+0000 7fdec899a700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 0x7fdeb000ae40 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009b10 tx=0x7fdeb0003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b11256eb91d0cea server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.571+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4005a20 con 0x7fdeb00089c0 2023-07-06T03:57:40.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.571+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb4005b80 con 0x7fdeb00089c0 2023-07-06T03:57:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.571+0000 7fdebbfff700 1 -- 172.21.15.97:0/1628492784 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400e6c0 con 0x7fdeb00089c0 2023-07-06T03:57:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.571+0000 7fdecb3ff700 1 -- 172.21.15.97:0/1628492784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 msgr2=0x7fdeb000ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.571+0000 7fdecb3ff700 1 --2- 172.21.15.97:0/1628492784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 0x7fdeb000ae40 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009b10 tx=0x7fdeb0003780 comp rx=0 tx=0).stop 2023-07-06T03:57:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.571+0000 7fdec999c700 1 -- 172.21.15.97:0/1628492784 reap_dead start 2023-07-06T03:57:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.571+0000 7fdecb3ff700 1 -- 172.21.15.97:0/1628492784 shutdown_connections 2023-07-06T03:57:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.571+0000 7fdecb3ff700 1 -- 172.21.15.97:0/1628492784 >> 172.21.15.97:0/1628492784 conn(0x7fdec40cc0e0 msgr2=0x7fdec40ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.571+0000 7fdecb3ff700 1 -- 172.21.15.97:0/1628492784 shutdown_connections 2023-07-06T03:57:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.571+0000 7fdecb3ff700 1 -- 172.21.15.97:0/1628492784 wait complete. 2023-07-06T03:57:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.573+0000 7fdecb3ff700 1 Processor -- start 2023-07-06T03:57:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.573+0000 7fdecb3ff700 1 -- start start 2023-07-06T03:57:40.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.574+0000 7fdecb3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 0x7fdec40c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.574+0000 7fdecb3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec40a8df0 0x7fdec41577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.574+0000 7fdecb3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdec4157d30 0x7fdec41f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.574+0000 7fdecb3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41f68a0 con 0x7fdeb00089c0 2023-07-06T03:57:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.574+0000 7fdecb3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40c3fd0 con 0x7fdec4157d30 2023-07-06T03:57:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.574+0000 7fdecb3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40ad7f0 con 0x7fdec40a8df0 2023-07-06T03:57:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.574+0000 7fdec919b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 0x7fdec40c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.574+0000 7fdec919b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 0x7fdec40c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43902/0 (socket says 172.21.15.97:43902) 2023-07-06T03:57:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.574+0000 7fdec919b700 1 -- 172.21.15.97:0/3081527609 learned_addr learned my addr 172.21.15.97:0/3081527609 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdec899a700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec40a8df0 0x7fdec41577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdec999c700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdec4157d30 0x7fdec41f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdec919b700 1 -- 172.21.15.97:0/3081527609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec40a8df0 msgr2=0x7fdec41577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdec919b700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec40a8df0 0x7fdec41577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdec919b700 1 -- 172.21.15.97:0/3081527609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdec4157d30 msgr2=0x7fdec41f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdec919b700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdec4157d30 0x7fdec41f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdec919b700 1 -- 172.21.15.97:0/3081527609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec40adad0 con 0x7fdeb00089c0 2023-07-06T03:57:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdec919b700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 0x7fdec40c4a20 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fdec000e7b0 tx=0x7fdec000eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdeb9ffb700 1 -- 172.21.15.97:0/3081527609 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00052e0 con 0x7fdeb00089c0 2023-07-06T03:57:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdeb9ffb700 1 -- 172.21.15.97:0/3081527609 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec0005460 con 0x7fdeb00089c0 2023-07-06T03:57:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdecb3ff700 1 -- 172.21.15.97:0/3081527609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec40c4220 con 0x7fdeb00089c0 2023-07-06T03:57:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.575+0000 7fdeb9ffb700 1 -- 172.21.15.97:0/3081527609 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00038c0 con 0x7fdeb00089c0 2023-07-06T03:57:40.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.576+0000 7fdecb3ff700 1 -- 172.21.15.97:0/3081527609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec40c4f60 con 0x7fdeb00089c0 2023-07-06T03:57:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.577+0000 7fdecb3ff700 1 -- 172.21.15.97:0/3081527609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec40c4740 con 0x7fdeb00089c0 2023-07-06T03:57:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.578+0000 7fdeb9ffb700 1 -- 172.21.15.97:0/3081527609 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdec0003a20 con 0x7fdeb00089c0 2023-07-06T03:57:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.579+0000 7fdeb9ffb700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdea806ace0 0x7fdea806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.579+0000 7fdeb9ffb700 1 -- 172.21.15.97:0/3081527609 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fdec0088d30 con 0x7fdeb00089c0 2023-07-06T03:57:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.579+0000 7fdec899a700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdea806ace0 0x7fdea806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:40.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.579+0000 7fdec899a700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdea806ace0 0x7fdea806d190 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4000c00 tx=0x7fdeb4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:40.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.583+0000 7fdeb9ffb700 1 -- 172.21.15.97:0/3081527609 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdec40c4740 con 0x7fdeb00089c0 2023-07-06T03:57:40.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.781+0000 7fdecb3ff700 1 -- 172.21.15.97:0/3081527609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdec40c4740 con 0x7fdeb00089c0 2023-07-06T03:57:40.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.783+0000 7fdeb9ffb700 1 -- 172.21.15.97:0/3081527609 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18436 (secure 0 0 0) 0x7fdec40c4740 con 0x7fdeb00089c0 2023-07-06T03:57:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:40.785 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":138,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:56:54.911710+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:56:02.531370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-06T03:56:03.548148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.7","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:57:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 -- 172.21.15.97:0/3081527609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdea806ace0 msgr2=0x7fdea806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdea806ace0 0x7fdea806d190 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4000c00 tx=0x7fdeb4008030 comp rx=0 tx=0).stop 2023-07-06T03:57:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 -- 172.21.15.97:0/3081527609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 msgr2=0x7fdec40c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 0x7fdec40c4a20 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fdec000e7b0 tx=0x7fdec000eac0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 -- 172.21.15.97:0/3081527609 shutdown_connections 2023-07-06T03:57:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdea806ace0 0x7fdea806d190 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdec4157d30 0x7fdec41f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb00089c0 0x7fdec40c4a20 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 --2- 172.21.15.97:0/3081527609 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdec40a8df0 0x7fdec41577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 -- 172.21.15.97:0/3081527609 >> 172.21.15.97:0/3081527609 conn(0x7fdec40cc0e0 msgr2=0x7fdec40af750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 -- 172.21.15.97:0/3081527609 shutdown_connections 2023-07-06T03:57:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:40.785+0000 7fdecb3ff700 1 -- 172.21.15.97:0/3081527609 wait complete. 2023-07-06T03:57:40.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:56:02.531370+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:56:03.548148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:57:40.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:57:41.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.002+0000 7f35c89ff700 1 Processor -- start 2023-07-06T03:57:41.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.002+0000 7f35c89ff700 1 -- start start 2023-07-06T03:57:41.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.003+0000 7f35c89ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35c4158ad0 0x7f35c40ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.003+0000 7f35c89ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35c40adb70 0x7f35c406c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.003+0000 7f35c89ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f35c406c630 0x7f35c40c3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.003+0000 7f35c89ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c40cc590 con 0x7f35c4158ad0 2023-07-06T03:57:41.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.003+0000 7f35c89ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c4158ea0 con 0x7f35c406c630 2023-07-06T03:57:41.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c89ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c406cdc0 con 0x7f35c40adb70 2023-07-06T03:57:41.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35c4158ad0 0x7f35c40ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35c4158ad0 0x7f35c40ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43916/0 (socket says 172.21.15.97:43916) 2023-07-06T03:57:41.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c259c700 1 -- 172.21.15.97:0/3462353227 learned_addr learned my addr 172.21.15.97:0/3462353227 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:41.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c1d9b700 1 --2- 172.21.15.97:0/3462353227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35c40adb70 0x7f35c406c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c2d9d700 1 --2- 172.21.15.97:0/3462353227 >> v2:172.21.15.146:3300/0 conn(0x7f35c406c630 0x7f35c40c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c259c700 1 -- 172.21.15.97:0/3462353227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35c40adb70 msgr2=0x7f35c406c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:41.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c259c700 1 --2- 172.21.15.97:0/3462353227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35c40adb70 0x7f35c406c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c259c700 1 -- 172.21.15.97:0/3462353227 >> v2:172.21.15.146:3300/0 conn(0x7f35c406c630 msgr2=0x7f35c40c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:41.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c259c700 1 --2- 172.21.15.97:0/3462353227 >> v2:172.21.15.146:3300/0 conn(0x7f35c406c630 0x7f35c40c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c259c700 1 -- 172.21.15.97:0/3462353227 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c406d0a0 con 0x7f35c4158ad0 2023-07-06T03:57:41.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c2d9d700 1 --2- 172.21.15.97:0/3462353227 >> v2:172.21.15.146:3300/0 conn(0x7f35c406c630 0x7f35c40c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:41.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.004+0000 7f35c259c700 1 --2- 172.21.15.97:0/3462353227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35c4158ad0 0x7f35c40ce200 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f35c40adb20 tx=0x7f35b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb01010991ebc90 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:41.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c159a700 1 -- 172.21.15.97:0/3462353227 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b400b870 con 0x7f35c4158ad0 2023-07-06T03:57:41.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c159a700 1 -- 172.21.15.97:0/3462353227 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b400b9d0 con 0x7f35c4158ad0 2023-07-06T03:57:41.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c159a700 1 -- 172.21.15.97:0/3462353227 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b400bd60 con 0x7f35c4158ad0 2023-07-06T03:57:41.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c89ff700 1 -- 172.21.15.97:0/3462353227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35c4158ad0 msgr2=0x7f35c40ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c89ff700 1 --2- 172.21.15.97:0/3462353227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35c4158ad0 0x7f35c40ce200 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f35c40adb20 tx=0x7f35b4009bd0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c89ff700 1 -- 172.21.15.97:0/3462353227 shutdown_connections 2023-07-06T03:57:41.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c89ff700 1 --2- 172.21.15.97:0/3462353227 >> v2:172.21.15.146:3300/0 conn(0x7f35c406c630 0x7f35c40c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c89ff700 1 --2- 172.21.15.97:0/3462353227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35c4158ad0 0x7f35c40ce200 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c89ff700 1 --2- 172.21.15.97:0/3462353227 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35c40adb70 0x7f35c406c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c89ff700 1 -- 172.21.15.97:0/3462353227 >> 172.21.15.97:0/3462353227 conn(0x7f35c406a060 msgr2=0x7f35c406a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:41.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c89ff700 1 -- 172.21.15.97:0/3462353227 shutdown_connections 2023-07-06T03:57:41.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.005+0000 7f35c89ff700 1 -- 172.21.15.97:0/3462353227 wait complete. 2023-07-06T03:57:41.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.007+0000 7f35c89ff700 1 Processor -- start 2023-07-06T03:57:41.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.007+0000 7f35c89ff700 1 -- start start 2023-07-06T03:57:41.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.008+0000 7f35c89ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35c406c630 0x7f35c4070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.008+0000 7f35c89ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35c40adb70 0x7f35c40714f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.008+0000 7f35c89ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35c4158ad0 0x7f35c40ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.008+0000 7f35c89ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c4142ed0 con 0x7f35c406c630 2023-07-06T03:57:41.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.008+0000 7f35c89ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c4142250 con 0x7f35c4158ad0 2023-07-06T03:57:41.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.008+0000 7f35c89ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c4142530 con 0x7f35c40adb70 2023-07-06T03:57:41.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.008+0000 7f35c259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35c406c630 0x7f35c4070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.008+0000 7f35c259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35c406c630 0x7f35c4070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43926/0 (socket says 172.21.15.97:43926) 2023-07-06T03:57:41.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.008+0000 7f35c259c700 1 -- 172.21.15.97:0/2853338922 learned_addr learned my addr 172.21.15.97:0/2853338922 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:41.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35c1d9b700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35c40adb70 0x7f35c40714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35c2d9d700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35c4158ad0 0x7f35c40ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35c259c700 1 -- 172.21.15.97:0/2853338922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35c40adb70 msgr2=0x7f35c40714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35c259c700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35c40adb70 0x7f35c40714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35c259c700 1 -- 172.21.15.97:0/2853338922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35c4158ad0 msgr2=0x7f35c40ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35c259c700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35c4158ad0 0x7f35c40ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35c259c700 1 -- 172.21.15.97:0/2853338922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b4009550 con 0x7f35c406c630 2023-07-06T03:57:41.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35c1d9b700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35c40adb70 0x7f35c40714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:41.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35c259c700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35c406c630 0x7f35c4070fb0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f35b4000c00 tx=0x7f35b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:41.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35b2ffd700 1 -- 172.21.15.97:0/2853338922 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4004520 con 0x7f35c406c630 2023-07-06T03:57:41.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35b2ffd700 1 -- 172.21.15.97:0/2853338922 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b401e070 con 0x7f35c406c630 2023-07-06T03:57:41.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35c89ff700 1 -- 172.21.15.97:0/2853338922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c4142810 con 0x7f35c406c630 2023-07-06T03:57:41.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35b2ffd700 1 -- 172.21.15.97:0/2853338922 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40038c0 con 0x7f35c406c630 2023-07-06T03:57:41.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.009+0000 7f35c89ff700 1 -- 172.21.15.97:0/2853338922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c40ca6b0 con 0x7f35c406c630 2023-07-06T03:57:41.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.010+0000 7f35c89ff700 1 -- 172.21.15.97:0/2853338922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c404e340 con 0x7f35c406c630 2023-07-06T03:57:41.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.011+0000 7f35b2ffd700 1 -- 172.21.15.97:0/2853338922 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35b4003a20 con 0x7f35c406c630 2023-07-06T03:57:41.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.012+0000 7f35b2ffd700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35a406ace0 0x7f35a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.012+0000 7f35b2ffd700 1 -- 172.21.15.97:0/2853338922 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f35b4020070 con 0x7f35c406c630 2023-07-06T03:57:41.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.012+0000 7f35c1d9b700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35a406ace0 0x7f35a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.013+0000 7f35c1d9b700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35a406ace0 0x7f35a406d190 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f35c4071f50 tx=0x7f35b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:41.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.016+0000 7f35b2ffd700 1 -- 172.21.15.97:0/2853338922 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f35b4055ad0 con 0x7f35c406c630 2023-07-06T03:57:41.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.217+0000 7f35c89ff700 1 -- 172.21.15.97:0/2853338922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35c40af150 con 0x7f35c406c630 2023-07-06T03:57:41.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.218+0000 7f35b2ffd700 1 -- 172.21.15.97:0/2853338922 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35b402a020 con 0x7f35c406c630 2023-07-06T03:57:41.220 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:57:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 -- 172.21.15.97:0/2853338922 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35a406ace0 msgr2=0x7f35a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35a406ace0 0x7f35a406d190 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f35c4071f50 tx=0x7f35b8008040 comp rx=0 tx=0).stop 2023-07-06T03:57:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 -- 172.21.15.97:0/2853338922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35c406c630 msgr2=0x7f35c4070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35c406c630 0x7f35c4070fb0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f35b4000c00 tx=0x7f35b4004750 comp rx=0 tx=0).stop 2023-07-06T03:57:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 -- 172.21.15.97:0/2853338922 shutdown_connections 2023-07-06T03:57:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f35a406ace0 0x7f35a406d190 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35c4158ad0 0x7f35c40ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35c406c630 0x7f35c4070fb0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 --2- 172.21.15.97:0/2853338922 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f35c40adb70 0x7f35c40714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 -- 172.21.15.97:0/2853338922 >> 172.21.15.97:0/2853338922 conn(0x7f35c406a060 msgr2=0x7f35c40ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 -- 172.21.15.97:0/2853338922 shutdown_connections 2023-07-06T03:57:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.220+0000 7f35c89ff700 1 -- 172.21.15.97:0/2853338922 wait complete. 2023-07-06T03:57:41.240 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:57:41.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.436+0000 7fc6cd9ff700 1 Processor -- start 2023-07-06T03:57:41.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.436+0000 7fc6cd9ff700 1 -- start start 2023-07-06T03:57:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.437+0000 7fc6cd9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6c80ce950 0x7fc6c80cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.437+0000 7fc6cd9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c80cab30 con 0x7fc6c8159460 2023-07-06T03:57:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.437+0000 7fc6cd9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c80cad80 con 0x7fc6c80c3a50 2023-07-06T03:57:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.437+0000 7fc6cd9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80cf300 con 0x7fc6c80ce950 2023-07-06T03:57:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.437+0000 7fc6c6ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc6c8159460 0x7fc6c80ccef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60218/0 (socket says 172.21.15.97:60218) 2023-07-06T03:57:41.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.437+0000 7fc6c6ffd700 1 -- 172.21.15.97:0/1139816690 learned_addr learned my addr 172.21.15.97:0/1139816690 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c67fc700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6c80ce950 0x7fc6c80cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347782850 0 0) 0x7fc6c80cab30 con 0x7fc6c8159460 2023-07-06T03:57:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6b00035b0 con 0x7fc6c8159460 2023-07-06T03:57:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4094168512 0 0) 0x7fc6b00035b0 con 0x7fc6c8159460 2023-07-06T03:57:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6c80ce950 msgr2=0x7fc6c80cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6c80ce950 0x7fc6c80cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 >> v1:172.21.15.146:6789/0 conn(0x7fc6c80c3a50 legacy=0x7fc6c80c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c80cb060 con 0x7fc6c8159460 2023-07-06T03:57:41.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fc6b8001670 con 0x7fc6c8159460 2023-07-06T03:57:41.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 >> v1:172.21.15.97:6789/0 conn(0x7fc6c8159460 legacy=0x7fc6c80ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6c80ce950 0x7fc6b0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b0003fb0 0x7fc6b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b0004d90 0x7fc6b0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80cf300 con 0x7fc6b0003fb0 2023-07-06T03:57:41.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b000dd00 con 0x7fc6b0004d90 2023-07-06T03:57:41.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.438+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b000dec0 con 0x7fc6c80ce950 2023-07-06T03:57:41.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c67fc700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b0003fb0 0x7fc6b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c6ffd700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6c80ce950 0x7fc6b0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c77fe700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b0004d90 0x7fc6b0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c67fc700 1 -- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6c80ce950 msgr2=0x7fc6b0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:41.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c67fc700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6c80ce950 0x7fc6b0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c67fc700 1 -- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b0004d90 msgr2=0x7fc6b0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c67fc700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b0004d90 0x7fc6b0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c67fc700 1 -- 172.21.15.97:0/1139816690 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00035b0 con 0x7fc6b0003fb0 2023-07-06T03:57:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c77fe700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b0004d90 0x7fc6b0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c67fc700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b0003fb0 0x7fc6b0004850 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009b30 tx=0x7fc6b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371743a854598404 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00e6c0 con 0x7fc6b0003fb0 2023-07-06T03:57:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc00e820 con 0x7fc6b0003fb0 2023-07-06T03:57:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.439+0000 7fc6c5ffb700 1 -- 172.21.15.97:0/1139816690 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00eb90 con 0x7fc6b0003fb0 2023-07-06T03:57:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.440+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1139816690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b0003fb0 msgr2=0x7fc6b0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.440+0000 7fc6cd9ff700 1 --2- 172.21.15.97:0/1139816690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6b0003fb0 0x7fc6b0004850 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009b30 tx=0x7fc6b0005880 comp rx=0 tx=0).stop 2023-07-06T03:57:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.440+0000 7fc6c77fe700 1 -- 172.21.15.97:0/1139816690 reap_dead start 2023-07-06T03:57:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.440+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1139816690 shutdown_connections 2023-07-06T03:57:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.440+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1139816690 >> 172.21.15.97:0/1139816690 conn(0x7fc6c806a060 msgr2=0x7fc6c806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.440+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1139816690 shutdown_connections 2023-07-06T03:57:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.440+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1139816690 wait complete. 2023-07-06T03:57:41.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.442+0000 7fc6cd9ff700 1 Processor -- start 2023-07-06T03:57:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.442+0000 7fc6cd9ff700 1 -- start start 2023-07-06T03:57:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.443+0000 7fc6cd9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b0003fb0 0x7fc6c80b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.443+0000 7fc6cd9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6c814a5a0 0x7fc6c80b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.443+0000 7fc6cd9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c815b840 0x7fc6c815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.443+0000 7fc6cd9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c815c220 con 0x7fc6c814a5a0 2023-07-06T03:57:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.443+0000 7fc6cd9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80b21b0 con 0x7fc6c815b840 2023-07-06T03:57:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.443+0000 7fc6cd9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c814aa10 con 0x7fc6b0003fb0 2023-07-06T03:57:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.443+0000 7fc6c67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6c814a5a0 0x7fc6c80b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.443+0000 7fc6c67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6c814a5a0 0x7fc6c80b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43946/0 (socket says 172.21.15.97:43946) 2023-07-06T03:57:41.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.443+0000 7fc6c67fc700 1 -- 172.21.15.97:0/1529549101 learned_addr learned my addr 172.21.15.97:0/1529549101 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:41.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.443+0000 7fc6c77fe700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c815b840 0x7fc6c815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.443+0000 7fc6c6ffd700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b0003fb0 0x7fc6c80b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6c67fc700 1 -- 172.21.15.97:0/1529549101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b0003fb0 msgr2=0x7fc6c80b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6c67fc700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b0003fb0 0x7fc6c80b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6c67fc700 1 -- 172.21.15.97:0/1529549101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c815b840 msgr2=0x7fc6c815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6c67fc700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c815b840 0x7fc6c815bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6c67fc700 1 -- 172.21.15.97:0/1529549101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c80b2400 con 0x7fc6c814a5a0 2023-07-06T03:57:41.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6c6ffd700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b0003fb0 0x7fc6c80b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:41.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6c77fe700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c815b840 0x7fc6c815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:41.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6c67fc700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6c814a5a0 0x7fc6c80b18d0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc004270 tx=0x7fc6bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:41.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6affff700 1 -- 172.21.15.97:0/1529549101 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc021070 con 0x7fc6c814a5a0 2023-07-06T03:57:41.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6affff700 1 -- 172.21.15.97:0/1529549101 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc0045b0 con 0x7fc6c814a5a0 2023-07-06T03:57:41.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1529549101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c80b26e0 con 0x7fc6c814a5a0 2023-07-06T03:57:41.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6affff700 1 -- 172.21.15.97:0/1529549101 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00e6c0 con 0x7fc6c814a5a0 2023-07-06T03:57:41.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.444+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1529549101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c80b2ca0 con 0x7fc6c814a5a0 2023-07-06T03:57:41.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.445+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1529549101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c804e340 con 0x7fc6c814a5a0 2023-07-06T03:57:41.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.446+0000 7fc6affff700 1 -- 172.21.15.97:0/1529549101 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6bc025070 con 0x7fc6c814a5a0 2023-07-06T03:57:41.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.447+0000 7fc6affff700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc6b406ace0 0x7fc6b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.447+0000 7fc6affff700 1 -- 172.21.15.97:0/1529549101 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fc6bc08a610 con 0x7fc6c814a5a0 2023-07-06T03:57:41.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.447+0000 7fc6c6ffd700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc6b406ace0 0x7fc6b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.448+0000 7fc6c6ffd700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc6b406ace0 0x7fc6b406d190 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8000c00 tx=0x7fc6b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:41.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.451+0000 7fc6affff700 1 -- 172.21.15.97:0/1529549101 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc6bc0557b0 con 0x7fc6c814a5a0 2023-07-06T03:57:41.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.649+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1529549101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc6c80cf580 con 0x7fc6c814a5a0 2023-07-06T03:57:41.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.649+0000 7fc6affff700 1 -- 172.21.15.97:0/1529549101 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc6bc058dd0 con 0x7fc6c814a5a0 2023-07-06T03:57:41.651 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:57:41.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.651+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1529549101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc6b406ace0 msgr2=0x7fc6b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.651+0000 7fc6cd9ff700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc6b406ace0 0x7fc6b406d190 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8000c00 tx=0x7fc6b8009040 comp rx=0 tx=0).stop 2023-07-06T03:57:41.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.651+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1529549101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6c814a5a0 msgr2=0x7fc6c80b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.651+0000 7fc6cd9ff700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6c814a5a0 0x7fc6c80b18d0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc004270 tx=0x7fc6bc0047c0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.651+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1529549101 shutdown_connections 2023-07-06T03:57:41.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.652+0000 7fc6cd9ff700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc6b406ace0 0x7fc6b406d190 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.652+0000 7fc6cd9ff700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6c815b840 0x7fc6c815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.652+0000 7fc6cd9ff700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc6b0003fb0 0x7fc6c80b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.652+0000 7fc6cd9ff700 1 --2- 172.21.15.97:0/1529549101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6c814a5a0 0x7fc6c80b18d0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.652+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1529549101 >> 172.21.15.97:0/1529549101 conn(0x7fc6c806a060 msgr2=0x7fc6c806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:41.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.652+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1529549101 shutdown_connections 2023-07-06T03:57:41.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.652+0000 7fc6cd9ff700 1 -- 172.21.15.97:0/1529549101 wait complete. 2023-07-06T03:57:41.670 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:57:41.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.867+0000 7fb40ffff700 1 Processor -- start 2023-07-06T03:57:41.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.867+0000 7fb40ffff700 1 -- start start 2023-07-06T03:57:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.868+0000 7fb40ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb4080b4850 0x7fb4080cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.868+0000 7fb40ffff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4080cf640 con 0x7fb40806c920 2023-07-06T03:57:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.868+0000 7fb40ffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4080cd4c0 con 0x7fb40815c950 2023-07-06T03:57:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.868+0000 7fb40ffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4080cd7a0 con 0x7fb4080b4850 2023-07-06T03:57:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.868+0000 7fb40e59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb40806c920 0x7fb4080d0d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60222/0 (socket says 172.21.15.97:60222) 2023-07-06T03:57:41.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.868+0000 7fb40e59c700 1 -- 172.21.15.97:0/3023494579 learned_addr learned my addr 172.21.15.97:0/3023494579 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:41.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.869+0000 7fb40dd9b700 1 --2- 172.21.15.97:0/3023494579 >> v2:172.21.15.146:3300/0 conn(0x7fb4080b4850 0x7fb4080cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.869+0000 7fb40cd99700 1 -- 172.21.15.97:0/3023494579 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122365834 0 0) 0x7fb4080cf640 con 0x7fb40806c920 2023-07-06T03:57:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.869+0000 7fb40cd99700 1 -- 172.21.15.97:0/3023494579 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f40035b0 con 0x7fb40806c920 2023-07-06T03:57:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.869+0000 7fb40dd9b700 1 -- 172.21.15.97:0/3023494579 >> v1:172.21.15.146:6789/0 conn(0x7fb40815c950 legacy=0x7fb4080b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.869+0000 7fb40dd9b700 1 -- 172.21.15.97:0/3023494579 >> v1:172.21.15.97:6789/0 conn(0x7fb40806c920 legacy=0x7fb4080d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.869+0000 7fb40dd9b700 1 -- 172.21.15.97:0/3023494579 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4080cda80 con 0x7fb4080b4850 2023-07-06T03:57:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.869+0000 7fb40dd9b700 1 --2- 172.21.15.97:0/3023494579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4080b4850 0x7fb4080cf100 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb404009840 tx=0x7fb404009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ededfcd18f34ba2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:41.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.870+0000 7fb40cd99700 1 -- 172.21.15.97:0/3023494579 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404011830 con 0x7fb4080b4850 2023-07-06T03:57:41.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.870+0000 7fb40cd99700 1 -- 172.21.15.97:0/3023494579 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb404011990 con 0x7fb4080b4850 2023-07-06T03:57:41.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.870+0000 7fb40cd99700 1 -- 172.21.15.97:0/3023494579 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404011d20 con 0x7fb4080b4850 2023-07-06T03:57:41.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.870+0000 7fb40ffff700 1 -- 172.21.15.97:0/3023494579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4080b4850 msgr2=0x7fb4080cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.870+0000 7fb40ffff700 1 --2- 172.21.15.97:0/3023494579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4080b4850 0x7fb4080cf100 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb404009840 tx=0x7fb404009b50 comp rx=0 tx=0).stop 2023-07-06T03:57:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.870+0000 7fb40ffff700 1 -- 172.21.15.97:0/3023494579 shutdown_connections 2023-07-06T03:57:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.870+0000 7fb40ffff700 1 --2- 172.21.15.97:0/3023494579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4080b4850 0x7fb4080cf100 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.870+0000 7fb40ffff700 1 -- 172.21.15.97:0/3023494579 >> 172.21.15.97:0/3023494579 conn(0x7fb40806b410 msgr2=0x7fb4080cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.870+0000 7fb40ffff700 1 -- 172.21.15.97:0/3023494579 shutdown_connections 2023-07-06T03:57:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.870+0000 7fb40ffff700 1 -- 172.21.15.97:0/3023494579 wait complete. 2023-07-06T03:57:41.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.872+0000 7fb40ffff700 1 Processor -- start 2023-07-06T03:57:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.872+0000 7fb40ffff700 1 -- start start 2023-07-06T03:57:41.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.873+0000 7fb40ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb40806c920 0x7fb4080cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.873+0000 7fb40ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4080b4850 0x7fb408152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.873+0000 7fb40ffff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40815c950 0x7fb4080c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.873+0000 7fb40ffff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4080cb620 con 0x7fb4080b4850 2023-07-06T03:57:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.873+0000 7fb40ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4080cb900 con 0x7fb40806c920 2023-07-06T03:57:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.873+0000 7fb40ffff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4080cbbe0 con 0x7fb40815c950 2023-07-06T03:57:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.873+0000 7fb40d59a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4080b4850 0x7fb408152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.873+0000 7fb40d59a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4080b4850 0x7fb408152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43952/0 (socket says 172.21.15.97:43952) 2023-07-06T03:57:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.873+0000 7fb40d59a700 1 -- 172.21.15.97:0/544205391 learned_addr learned my addr 172.21.15.97:0/544205391 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb40dd9b700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb40806c920 0x7fb4080cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb40e59c700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40815c950 0x7fb4080c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb40d59a700 1 -- 172.21.15.97:0/544205391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40815c950 msgr2=0x7fb4080c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb40d59a700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40815c950 0x7fb4080c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb40d59a700 1 -- 172.21.15.97:0/544205391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb40806c920 msgr2=0x7fb4080cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb40d59a700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb40806c920 0x7fb4080cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb40d59a700 1 -- 172.21.15.97:0/544205391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4040094a0 con 0x7fb4080b4850 2023-07-06T03:57:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb40e59c700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40815c950 0x7fb4080c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb40d59a700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4080b4850 0x7fb408152580 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c4d0 tx=0x7fb3f800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:41.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb3fe7fc700 1 -- 172.21.15.97:0/544205391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80052d0 con 0x7fb4080b4850 2023-07-06T03:57:41.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb3fe7fc700 1 -- 172.21.15.97:0/544205391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f8005430 con 0x7fb4080b4850 2023-07-06T03:57:41.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb3fe7fc700 1 -- 172.21.15.97:0/544205391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8003880 con 0x7fb4080b4850 2023-07-06T03:57:41.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb40ffff700 1 -- 172.21.15.97:0/544205391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4080cbec0 con 0x7fb4080b4850 2023-07-06T03:57:41.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.874+0000 7fb40ffff700 1 -- 172.21.15.97:0/544205391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4080c8610 con 0x7fb4080b4850 2023-07-06T03:57:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.875+0000 7fb40ffff700 1 -- 172.21.15.97:0/544205391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4080cc3e0 con 0x7fb4080b4850 2023-07-06T03:57:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.876+0000 7fb3fe7fc700 1 -- 172.21.15.97:0/544205391 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb3f80039e0 con 0x7fb4080b4850 2023-07-06T03:57:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.877+0000 7fb3fe7fc700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3ec06ace0 0x7fb3ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.877+0000 7fb3fe7fc700 1 -- 172.21.15.97:0/544205391 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fb3f8088840 con 0x7fb4080b4850 2023-07-06T03:57:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.878+0000 7fb40dd9b700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3ec06ace0 0x7fb3ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:41.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.878+0000 7fb40dd9b700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3ec06ace0 0x7fb3ec06d190 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb4040090b0 tx=0x7fb404008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:41.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:41.882+0000 7fb3fe7fc700 1 -- 172.21.15.97:0/544205391 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb4080cc3e0 con 0x7fb4080b4850 2023-07-06T03:57:42.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.080+0000 7fb40ffff700 1 -- 172.21.15.97:0/544205391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4080cc3e0 con 0x7fb4080b4850 2023-07-06T03:57:42.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.080+0000 7fb3fe7fc700 1 -- 172.21.15.97:0/544205391 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4080cc3e0 con 0x7fb4080b4850 2023-07-06T03:57:42.083 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:57:42.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 -- 172.21.15.97:0/544205391 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3ec06ace0 msgr2=0x7fb3ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3ec06ace0 0x7fb3ec06d190 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb4040090b0 tx=0x7fb404008030 comp rx=0 tx=0).stop 2023-07-06T03:57:42.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 -- 172.21.15.97:0/544205391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4080b4850 msgr2=0x7fb408152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4080b4850 0x7fb408152580 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c4d0 tx=0x7fb3f800c7e0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 -- 172.21.15.97:0/544205391 shutdown_connections 2023-07-06T03:57:42.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb3ec06ace0 0x7fb3ec06d190 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb40815c950 0x7fb4080c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb40806c920 0x7fb4080cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 --2- 172.21.15.97:0/544205391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4080b4850 0x7fb408152580 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 -- 172.21.15.97:0/544205391 >> 172.21.15.97:0/544205391 conn(0x7fb40806b410 msgr2=0x7fb4080c6900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:42.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 -- 172.21.15.97:0/544205391 shutdown_connections 2023-07-06T03:57:42.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.083+0000 7fb40ffff700 1 -- 172.21.15.97:0/544205391 wait complete. 2023-07-06T03:57:42.102 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:57:42.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.298+0000 7f9905fff700 1 Processor -- start 2023-07-06T03:57:42.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.298+0000 7f9905fff700 1 -- start start 2023-07-06T03:57:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f9905fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99000ac880 0x7f99000ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f9905fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f990006d520 0x7f9900071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f9905fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990015c9f0 con 0x7f990006a060 2023-07-06T03:57:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f9905fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99000cef80 con 0x7f990006d520 2023-07-06T03:57:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f9905fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99000cf260 con 0x7f99000ac880 2023-07-06T03:57:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f98fffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f990006d520 0x7f9900071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f98ff7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f990006a060 0x7f990006a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60232/0 (socket says 172.21.15.97:60232) 2023-07-06T03:57:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f98ff7fe700 1 -- 172.21.15.97:0/1923762644 learned_addr learned my addr 172.21.15.97:0/1923762644 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:42.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f98fffff700 1 -- 172.21.15.97:0/1923762644 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99000ac880 msgr2=0x7f99000ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:42.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f98feffd700 1 --2- 172.21.15.97:0/1923762644 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99000ac880 0x7f99000ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f98fffff700 1 --2- 172.21.15.97:0/1923762644 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99000ac880 0x7f99000ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f98fffff700 1 -- 172.21.15.97:0/1923762644 >> v1:172.21.15.97:6789/0 conn(0x7f990006a060 legacy=0x7f990006a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f98fe7fc700 1 -- 172.21.15.97:0/1923762644 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3712451998 0 0) 0x7f990015c9f0 con 0x7f990006a060 2023-07-06T03:57:42.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f98fffff700 1 -- 172.21.15.97:0/1923762644 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99000cf540 con 0x7f990006d520 2023-07-06T03:57:42.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.300+0000 7f98feffd700 1 --2- 172.21.15.97:0/1923762644 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99000ac880 0x7f99000ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:42.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.301+0000 7f98fffff700 1 --2- 172.21.15.97:0/1923762644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f990006d520 0x7f9900071370 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009d90 tx=0x7f98f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e99635e563596e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.301+0000 7f98fe7fc700 1 -- 172.21.15.97:0/1923762644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f800e070 con 0x7f990006d520 2023-07-06T03:57:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.301+0000 7f98fe7fc700 1 -- 172.21.15.97:0/1923762644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f800bbd0 con 0x7f990006d520 2023-07-06T03:57:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.301+0000 7f98fe7fc700 1 -- 172.21.15.97:0/1923762644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8012430 con 0x7f990006d520 2023-07-06T03:57:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.301+0000 7f9905fff700 1 -- 172.21.15.97:0/1923762644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f990006d520 msgr2=0x7f9900071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.301+0000 7f9905fff700 1 --2- 172.21.15.97:0/1923762644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f990006d520 0x7f9900071370 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009d90 tx=0x7f98f800b330 comp rx=0 tx=0).stop 2023-07-06T03:57:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.301+0000 7f9905fff700 1 -- 172.21.15.97:0/1923762644 shutdown_connections 2023-07-06T03:57:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.301+0000 7f9905fff700 1 --2- 172.21.15.97:0/1923762644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f990006d520 0x7f9900071370 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.301+0000 7f9905fff700 1 --2- 172.21.15.97:0/1923762644 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99000ac880 0x7f99000ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.301+0000 7f9905fff700 1 -- 172.21.15.97:0/1923762644 >> 172.21.15.97:0/1923762644 conn(0x7f99000b16c0 msgr2=0x7f99000b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:42.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.302+0000 7f9905fff700 1 -- 172.21.15.97:0/1923762644 shutdown_connections 2023-07-06T03:57:42.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.302+0000 7f9905fff700 1 -- 172.21.15.97:0/1923762644 wait complete. 2023-07-06T03:57:42.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.303+0000 7f9905fff700 1 Processor -- start 2023-07-06T03:57:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.303+0000 7f9905fff700 1 -- start start 2023-07-06T03:57:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.304+0000 7f9905fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f990006a060 0x7f99000c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.304+0000 7f9905fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f990006d520 0x7f99001f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.304+0000 7f9905fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99000ac880 0x7f9900071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.304+0000 7f9905fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9900145a20 con 0x7f990006d520 2023-07-06T03:57:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.304+0000 7f9905fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9900145c70 con 0x7f990006a060 2023-07-06T03:57:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.304+0000 7f9905fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9900145f50 con 0x7f99000ac880 2023-07-06T03:57:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98feffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f990006d520 0x7f99001f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98feffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f990006d520 0x7f99001f5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43974/0 (socket says 172.21.15.97:43974) 2023-07-06T03:57:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98feffd700 1 -- 172.21.15.97:0/2533113816 learned_addr learned my addr 172.21.15.97:0/2533113816 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98fffff700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99000ac880 0x7f9900071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98ff7fe700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f990006a060 0x7f99000c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98feffd700 1 -- 172.21.15.97:0/2533113816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99000ac880 msgr2=0x7f9900071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98feffd700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99000ac880 0x7f9900071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98feffd700 1 -- 172.21.15.97:0/2533113816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f990006a060 msgr2=0x7f99000c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98feffd700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f990006a060 0x7f99000c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98feffd700 1 -- 172.21.15.97:0/2533113816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f8009a70 con 0x7f990006d520 2023-07-06T03:57:42.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98ff7fe700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f990006a060 0x7f99000c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:42.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.305+0000 7f98feffd700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f990006d520 0x7f99001f5500 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f98f400c7f0 tx=0x7f98f400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:42.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.306+0000 7f98e7fff700 1 -- 172.21.15.97:0/2533113816 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4011800 con 0x7f990006d520 2023-07-06T03:57:42.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.306+0000 7f98e7fff700 1 -- 172.21.15.97:0/2533113816 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f4011960 con 0x7f990006d520 2023-07-06T03:57:42.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.306+0000 7f98e7fff700 1 -- 172.21.15.97:0/2533113816 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4011cf0 con 0x7f990006d520 2023-07-06T03:57:42.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.306+0000 7f9905fff700 1 -- 172.21.15.97:0/2533113816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9900146230 con 0x7f990006d520 2023-07-06T03:57:42.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.306+0000 7f9905fff700 1 -- 172.21.15.97:0/2533113816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99000c5f20 con 0x7f990006d520 2023-07-06T03:57:42.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.307+0000 7f9905fff700 1 -- 172.21.15.97:0/2533113816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9900146750 con 0x7f990006d520 2023-07-06T03:57:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.308+0000 7f98e7fff700 1 -- 172.21.15.97:0/2533113816 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f98f4018360 con 0x7f990006d520 2023-07-06T03:57:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.309+0000 7f98e7fff700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f98ec06ace0 0x7f98ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.309+0000 7f98e7fff700 1 -- 172.21.15.97:0/2533113816 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f98f4089430 con 0x7f990006d520 2023-07-06T03:57:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.310+0000 7f98ff7fe700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f98ec06ace0 0x7f98ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.310+0000 7f98ff7fe700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f98ec06ace0 0x7f98ec06d190 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000f60 tx=0x7f98f0007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.314+0000 7f98e7fff700 1 -- 172.21.15.97:0/2533113816 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9900146750 con 0x7f990006d520 2023-07-06T03:57:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.507+0000 7f9905fff700 1 -- 172.21.15.97:0/2533113816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9900146750 con 0x7f990006d520 2023-07-06T03:57:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.508+0000 7f98e7fff700 1 -- 172.21.15.97:0/2533113816 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9900146750 con 0x7f990006d520 2023-07-06T03:57:42.510 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:57:42.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.510+0000 7f9905fff700 1 -- 172.21.15.97:0/2533113816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f98ec06ace0 msgr2=0x7f98ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.510+0000 7f9905fff700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f98ec06ace0 0x7f98ec06d190 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000f60 tx=0x7f98f0007ca0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.510+0000 7f9905fff700 1 -- 172.21.15.97:0/2533113816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f990006d520 msgr2=0x7f99001f5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.510+0000 7f9905fff700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f990006d520 0x7f99001f5500 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f98f400c7f0 tx=0x7f98f400cb00 comp rx=0 tx=0).stop 2023-07-06T03:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.510+0000 7f9905fff700 1 -- 172.21.15.97:0/2533113816 shutdown_connections 2023-07-06T03:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.510+0000 7f9905fff700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f98ec06ace0 0x7f98ec06d190 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.510+0000 7f9905fff700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f99000ac880 0x7f9900071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.510+0000 7f9905fff700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f990006a060 0x7f99000c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.510+0000 7f9905fff700 1 --2- 172.21.15.97:0/2533113816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f990006d520 0x7f99001f5500 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.510+0000 7f9905fff700 1 -- 172.21.15.97:0/2533113816 >> 172.21.15.97:0/2533113816 conn(0x7f99000b16c0 msgr2=0x7f99000b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:42.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.511+0000 7f9905fff700 1 -- 172.21.15.97:0/2533113816 shutdown_connections 2023-07-06T03:57:42.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.511+0000 7f9905fff700 1 -- 172.21.15.97:0/2533113816 wait complete. 2023-07-06T03:57:42.530 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:57:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.723+0000 7f322b3ff700 1 Processor -- start 2023-07-06T03:57:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.723+0000 7f322b3ff700 1 -- start start 2023-07-06T03:57:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.724+0000 7f322b3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f32240d09b0 0x7f32240b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.724+0000 7f322b3ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f322415ce60 0x7f32240d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.724+0000 7f322b3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32240d16b0 con 0x7f32240cf780 2023-07-06T03:57:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.724+0000 7f322b3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32240d0e50 con 0x7f32240d09b0 2023-07-06T03:57:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.724+0000 7f322b3ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32240b24d0 con 0x7f322415ce60 2023-07-06T03:57:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.724+0000 7f322899a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f32240d09b0 0x7f32240b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.724+0000 7f322899a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f32240d09b0 0x7f32240b8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43988/0 (socket says 172.21.15.97:43988) 2023-07-06T03:57:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.724+0000 7f322899a700 1 -- 172.21.15.97:0/3053272690 learned_addr learned my addr 172.21.15.97:0/3053272690 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.724+0000 7f322999c700 1 --2- 172.21.15.97:0/3053272690 >> v2:172.21.15.146:3300/0 conn(0x7f322415ce60 0x7f32240d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.725+0000 7f322899a700 1 -- 172.21.15.97:0/3053272690 >> v2:172.21.15.146:3300/0 conn(0x7f322415ce60 msgr2=0x7f32240d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.725+0000 7f322899a700 1 --2- 172.21.15.97:0/3053272690 >> v2:172.21.15.146:3300/0 conn(0x7f322415ce60 0x7f32240d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.725+0000 7f322899a700 1 -- 172.21.15.97:0/3053272690 >> v1:172.21.15.97:6789/0 conn(0x7f32240cf780 legacy=0x7f32240cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.725+0000 7f322899a700 1 -- 172.21.15.97:0/3053272690 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240b8b60 con 0x7f32240d09b0 2023-07-06T03:57:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.725+0000 7f322899a700 1 --2- 172.21.15.97:0/3053272690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32240d09b0 0x7f32240b8230 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f32240d2640 tx=0x7f3214009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8234af1c18d84332 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.725+0000 7f321bfff700 1 -- 172.21.15.97:0/3053272690 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321400b870 con 0x7f32240d09b0 2023-07-06T03:57:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.725+0000 7f321bfff700 1 -- 172.21.15.97:0/3053272690 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321400b9d0 con 0x7f32240d09b0 2023-07-06T03:57:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.725+0000 7f321bfff700 1 -- 172.21.15.97:0/3053272690 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321400bd60 con 0x7f32240d09b0 2023-07-06T03:57:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.726+0000 7f322b3ff700 1 -- 172.21.15.97:0/3053272690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32240d09b0 msgr2=0x7f32240b8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.726+0000 7f322b3ff700 1 --2- 172.21.15.97:0/3053272690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32240d09b0 0x7f32240b8230 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f32240d2640 tx=0x7f3214009bd0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.726+0000 7f322b3ff700 1 -- 172.21.15.97:0/3053272690 shutdown_connections 2023-07-06T03:57:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.726+0000 7f322b3ff700 1 --2- 172.21.15.97:0/3053272690 >> v2:172.21.15.146:3300/0 conn(0x7f322415ce60 0x7f32240d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.726+0000 7f322b3ff700 1 --2- 172.21.15.97:0/3053272690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32240d09b0 0x7f32240b8230 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.726+0000 7f322b3ff700 1 -- 172.21.15.97:0/3053272690 >> 172.21.15.97:0/3053272690 conn(0x7f322406c1c0 msgr2=0x7f322406c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.726+0000 7f322b3ff700 1 -- 172.21.15.97:0/3053272690 shutdown_connections 2023-07-06T03:57:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.726+0000 7f322b3ff700 1 -- 172.21.15.97:0/3053272690 wait complete. 2023-07-06T03:57:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.727+0000 7f322b3ff700 1 Processor -- start 2023-07-06T03:57:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.728+0000 7f322b3ff700 1 -- start start 2023-07-06T03:57:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.728+0000 7f322b3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32240cf780 0x7f322414a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322b3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32240d09b0 0x7f32241f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322b3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f322415ce60 0x7f32240c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322b3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322414a040 con 0x7f32240cf780 2023-07-06T03:57:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322b3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32240c9700 con 0x7f32240d09b0 2023-07-06T03:57:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322b3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3224149a30 con 0x7f322415ce60 2023-07-06T03:57:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322919b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32240cf780 0x7f322414a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322919b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32240cf780 0x7f322414a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43992/0 (socket says 172.21.15.97:43992) 2023-07-06T03:57:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322919b700 1 -- 172.21.15.97:0/259185353 learned_addr learned my addr 172.21.15.97:0/259185353 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322999c700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f322415ce60 0x7f32240c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322899a700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32240d09b0 0x7f32241f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322919b700 1 -- 172.21.15.97:0/259185353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f322415ce60 msgr2=0x7f32240c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322919b700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f322415ce60 0x7f32240c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322919b700 1 -- 172.21.15.97:0/259185353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32240d09b0 msgr2=0x7f32241f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322919b700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32240d09b0 0x7f32241f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322919b700 1 -- 172.21.15.97:0/259185353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3214009550 con 0x7f32240cf780 2023-07-06T03:57:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.729+0000 7f322919b700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32240cf780 0x7f322414a4d0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f322000c4e0 tx=0x7f322000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.730+0000 7f3219ffb700 1 -- 172.21.15.97:0/259185353 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32200052d0 con 0x7f32240cf780 2023-07-06T03:57:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.730+0000 7f3219ffb700 1 -- 172.21.15.97:0/259185353 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3220005430 con 0x7f32240cf780 2023-07-06T03:57:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.730+0000 7f322b3ff700 1 -- 172.21.15.97:0/259185353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32240c9420 con 0x7f32240cf780 2023-07-06T03:57:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.730+0000 7f3219ffb700 1 -- 172.21.15.97:0/259185353 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32200038c0 con 0x7f32240cf780 2023-07-06T03:57:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.730+0000 7f322b3ff700 1 -- 172.21.15.97:0/259185353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32241f5540 con 0x7f32240cf780 2023-07-06T03:57:42.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.731+0000 7f322b3ff700 1 -- 172.21.15.97:0/259185353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322404e340 con 0x7f32240cf780 2023-07-06T03:57:42.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.732+0000 7f3219ffb700 1 -- 172.21.15.97:0/259185353 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3220003a20 con 0x7f32240cf780 2023-07-06T03:57:42.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.733+0000 7f3219ffb700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f320806ace0 0x7f320806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:42.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.733+0000 7f3219ffb700 1 -- 172.21.15.97:0/259185353 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f32200889d0 con 0x7f32240cf780 2023-07-06T03:57:42.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.733+0000 7f322899a700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f320806ace0 0x7f320806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:42.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.733+0000 7f322899a700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f320806ace0 0x7f320806d190 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f32240d0da0 tx=0x7f3214008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:42.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.737+0000 7f3219ffb700 1 -- 172.21.15.97:0/259185353 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3220053a50 con 0x7f32240cf780 2023-07-06T03:57:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.949+0000 7f322b3ff700 1 -- 172.21.15.97:0/259185353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f322406c730 con 0x7f32240cf780 2023-07-06T03:57:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.950+0000 7f3219ffb700 1 -- 172.21.15.97:0/259185353 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v138) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3220057070 con 0x7f32240cf780 2023-07-06T03:57:42.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.952+0000 7f322b3ff700 1 -- 172.21.15.97:0/259185353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f320806ace0 msgr2=0x7f320806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.952+0000 7f322b3ff700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f320806ace0 0x7f320806d190 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f32240d0da0 tx=0x7f3214008030 comp rx=0 tx=0).stop 2023-07-06T03:57:42.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.952+0000 7f322b3ff700 1 -- 172.21.15.97:0/259185353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32240cf780 msgr2=0x7f322414a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:42.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.952+0000 7f322b3ff700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32240cf780 0x7f322414a4d0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f322000c4e0 tx=0x7f322000c8a0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.952+0000 7f322b3ff700 1 -- 172.21.15.97:0/259185353 shutdown_connections 2023-07-06T03:57:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.952+0000 7f322b3ff700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f320806ace0 0x7f320806d190 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.952+0000 7f322b3ff700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f322415ce60 0x7f32240c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.952+0000 7f322b3ff700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32240cf780 0x7f322414a4d0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.952+0000 7f322b3ff700 1 --2- 172.21.15.97:0/259185353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32240d09b0 0x7f32241f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.952+0000 7f322b3ff700 1 -- 172.21.15.97:0/259185353 >> 172.21.15.97:0/259185353 conn(0x7f322406c1c0 msgr2=0x7f322415d280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.953+0000 7f322b3ff700 1 -- 172.21.15.97:0/259185353 shutdown_connections 2023-07-06T03:57:42.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:42.953+0000 7f322b3ff700 1 -- 172.21.15.97:0/259185353 wait complete. 2023-07-06T03:57:42.956 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:57:42.972 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2023-07-06T03:57:42.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:42.973 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T03:57:43.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[168891]: fuse finished with error 0 and tester_r 0 2023-07-06T03:57:43.041 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:57:43.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:43.041 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T03:57:43.091 INFO:tasks.cephfs.mount:Terminating background process 2023-07-06T03:57:43.092 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:57:43.092 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:57:43.143 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.335+0000 7fc21f5ff700 1 Processor -- start 2023-07-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.336+0000 7fc21f5ff700 1 -- start start 2023-07-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.337+0000 7fc21f5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc218153a70 0x7fc2180c9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.337+0000 7fc21f5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc2180706a0 0x7fc2180ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.337+0000 7fc21f5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180c9ac0 con 0x7fc2180c9560 2023-07-06T03:57:43.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.337+0000 7fc21f5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2180c9da0 con 0x7fc218153a70 2023-07-06T03:57:43.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.337+0000 7fc21f5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2180ca080 con 0x7fc2180706a0 2023-07-06T03:57:43.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.337+0000 7fc21cb9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc2180c9560 0x7fc21806ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60248/0 (socket says 172.21.15.97:60248) 2023-07-06T03:57:43.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.337+0000 7fc21cb9a700 1 -- 172.21.15.97:0/952352684 learned_addr learned my addr 172.21.15.97:0/952352684 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:43.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.337+0000 7fc21d39b700 1 --2- 172.21.15.97:0/952352684 >> v2:172.21.15.97:3300/0 conn(0x7fc218153a70 0x7fc2180c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.337+0000 7fc21db9c700 1 --2- 172.21.15.97:0/952352684 >> v2:172.21.15.146:3300/0 conn(0x7fc2180706a0 0x7fc2180ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.337+0000 7fc20ffff700 1 -- 172.21.15.97:0/952352684 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94717360 0 0) 0x7fc2180c9ac0 con 0x7fc2180c9560 2023-07-06T03:57:43.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.338+0000 7fc20ffff700 1 -- 172.21.15.97:0/952352684 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2040035b0 con 0x7fc2180c9560 2023-07-06T03:57:43.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.338+0000 7fc21d39b700 1 -- 172.21.15.97:0/952352684 >> v2:172.21.15.146:3300/0 conn(0x7fc2180706a0 msgr2=0x7fc2180ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:43.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.338+0000 7fc21d39b700 1 --2- 172.21.15.97:0/952352684 >> v2:172.21.15.146:3300/0 conn(0x7fc2180706a0 0x7fc2180ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.338+0000 7fc21d39b700 1 -- 172.21.15.97:0/952352684 >> v1:172.21.15.97:6789/0 conn(0x7fc2180c9560 legacy=0x7fc21806ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.338+0000 7fc21d39b700 1 -- 172.21.15.97:0/952352684 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180ca360 con 0x7fc218153a70 2023-07-06T03:57:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.338+0000 7fc21db9c700 1 --2- 172.21.15.97:0/952352684 >> v2:172.21.15.146:3300/0 conn(0x7fc2180706a0 0x7fc2180ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.338+0000 7fc21d39b700 1 --2- 172.21.15.97:0/952352684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc218153a70 0x7fc2180c9020 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fc2181575b0 tx=0x7fc214009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1476af58a63f29f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.338+0000 7fc20ffff700 1 -- 172.21.15.97:0/952352684 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214011830 con 0x7fc218153a70 2023-07-06T03:57:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.338+0000 7fc20ffff700 1 -- 172.21.15.97:0/952352684 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc214011990 con 0x7fc218153a70 2023-07-06T03:57:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.338+0000 7fc20ffff700 1 -- 172.21.15.97:0/952352684 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214011d20 con 0x7fc218153a70 2023-07-06T03:57:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.339+0000 7fc21f5ff700 1 -- 172.21.15.97:0/952352684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc218153a70 msgr2=0x7fc2180c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.339+0000 7fc21f5ff700 1 --2- 172.21.15.97:0/952352684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc218153a70 0x7fc2180c9020 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fc2181575b0 tx=0x7fc214009b20 comp rx=0 tx=0).stop 2023-07-06T03:57:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.339+0000 7fc21f5ff700 1 -- 172.21.15.97:0/952352684 shutdown_connections 2023-07-06T03:57:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.339+0000 7fc21f5ff700 1 --2- 172.21.15.97:0/952352684 >> v2:172.21.15.146:3300/0 conn(0x7fc2180706a0 0x7fc2180ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.339+0000 7fc21f5ff700 1 --2- 172.21.15.97:0/952352684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc218153a70 0x7fc2180c9020 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.339+0000 7fc21f5ff700 1 -- 172.21.15.97:0/952352684 >> 172.21.15.97:0/952352684 conn(0x7fc2180cc0e0 msgr2=0x7fc2180ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.339+0000 7fc21f5ff700 1 -- 172.21.15.97:0/952352684 shutdown_connections 2023-07-06T03:57:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.339+0000 7fc21f5ff700 1 -- 172.21.15.97:0/952352684 wait complete. 2023-07-06T03:57:43.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.341+0000 7fc21f5ff700 1 Processor -- start 2023-07-06T03:57:43.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.341+0000 7fc21f5ff700 1 -- start start 2023-07-06T03:57:43.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.341+0000 7fc21f5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2180706a0 0x7fc2180b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21f5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2180c9560 0x7fc218152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21f5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc218153a70 0x7fc2180c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21f5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180c6900 con 0x7fc218153a70 2023-07-06T03:57:43.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21f5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b3190 con 0x7fc2180706a0 2023-07-06T03:57:43.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21f5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b28f0 con 0x7fc2180c9560 2023-07-06T03:57:43.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21db9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc218153a70 0x7fc2180c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21db9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc218153a70 0x7fc2180c5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44018/0 (socket says 172.21.15.97:44018) 2023-07-06T03:57:43.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21db9c700 1 -- 172.21.15.97:0/2894566177 learned_addr learned my addr 172.21.15.97:0/2894566177 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:43.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21d39b700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2180706a0 0x7fc2180b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21cb9a700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2180c9560 0x7fc218152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21db9c700 1 -- 172.21.15.97:0/2894566177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2180c9560 msgr2=0x7fc218152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21db9c700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2180c9560 0x7fc218152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21db9c700 1 -- 172.21.15.97:0/2894566177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2180706a0 msgr2=0x7fc2180b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21db9c700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2180706a0 0x7fc2180b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21db9c700 1 -- 172.21.15.97:0/2894566177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2140094a0 con 0x7fc218153a70 2023-07-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21cb9a700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2180c9560 0x7fc218152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21d39b700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2180706a0 0x7fc2180b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.342+0000 7fc21db9c700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc218153a70 0x7fc2180c5aa0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fc21000cef0 tx=0x7fc21000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.343+0000 7fc20dffb700 1 -- 172.21.15.97:0/2894566177 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21000ae50 con 0x7fc218153a70 2023-07-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.343+0000 7fc20dffb700 1 -- 172.21.15.97:0/2894566177 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc210015440 con 0x7fc218153a70 2023-07-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.343+0000 7fc21f5ff700 1 -- 172.21.15.97:0/2894566177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2180b2bd0 con 0x7fc218153a70 2023-07-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.343+0000 7fc20dffb700 1 -- 172.21.15.97:0/2894566177 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100157f0 con 0x7fc218153a70 2023-07-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.343+0000 7fc21f5ff700 1 -- 172.21.15.97:0/2894566177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2180c31b0 con 0x7fc218153a70 2023-07-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.344+0000 7fc21f5ff700 1 -- 172.21.15.97:0/2894566177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2180c3010 con 0x7fc218153a70 2023-07-06T03:57:43.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.345+0000 7fc20dffb700 1 -- 172.21.15.97:0/2894566177 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc21000d430 con 0x7fc218153a70 2023-07-06T03:57:43.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.346+0000 7fc20dffb700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1fc06ace0 0x7fc1fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.346+0000 7fc20dffb700 1 -- 172.21.15.97:0/2894566177 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fc210089a90 con 0x7fc218153a70 2023-07-06T03:57:43.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.346+0000 7fc21d39b700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1fc06ace0 0x7fc1fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.347+0000 7fc21d39b700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1fc06ace0 0x7fc1fc06d190 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc2140090b0 tx=0x7fc214008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:43.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.350+0000 7fc20dffb700 1 -- 172.21.15.97:0/2894566177 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc2180c3010 con 0x7fc218153a70 2023-07-06T03:57:43.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.549+0000 7fc21f5ff700 1 -- 172.21.15.97:0/2894566177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2180c3010 con 0x7fc218153a70 2023-07-06T03:57:43.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.550+0000 7fc20dffb700 1 -- 172.21.15.97:0/2894566177 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18436 (secure 0 0 0) 0x7fc2180c3010 con 0x7fc218153a70 2023-07-06T03:57:43.552 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:43.552 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":138,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:56:54.911710+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:56:02.531370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-06T03:56:03.548148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.7","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:57:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.552+0000 7fc21f5ff700 1 -- 172.21.15.97:0/2894566177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1fc06ace0 msgr2=0x7fc1fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.552+0000 7fc21f5ff700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1fc06ace0 0x7fc1fc06d190 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc2140090b0 tx=0x7fc214008030 comp rx=0 tx=0).stop 2023-07-06T03:57:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.553+0000 7fc21f5ff700 1 -- 172.21.15.97:0/2894566177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc218153a70 msgr2=0x7fc2180c5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.553+0000 7fc21f5ff700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc218153a70 0x7fc2180c5aa0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fc21000cef0 tx=0x7fc21000a340 comp rx=0 tx=0).stop 2023-07-06T03:57:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.553+0000 7fc21f5ff700 1 -- 172.21.15.97:0/2894566177 shutdown_connections 2023-07-06T03:57:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.553+0000 7fc21f5ff700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fc1fc06ace0 0x7fc1fc06d190 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.553+0000 7fc21f5ff700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc218153a70 0x7fc2180c5aa0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.553+0000 7fc21f5ff700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2180706a0 0x7fc2180b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.553+0000 7fc21f5ff700 1 --2- 172.21.15.97:0/2894566177 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fc2180c9560 0x7fc218152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.553+0000 7fc21f5ff700 1 -- 172.21.15.97:0/2894566177 >> 172.21.15.97:0/2894566177 conn(0x7fc2180cc0e0 msgr2=0x7fc21806ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:43.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.553+0000 7fc21f5ff700 1 -- 172.21.15.97:0/2894566177 shutdown_connections 2023-07-06T03:57:43.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.553+0000 7fc21f5ff700 1 -- 172.21.15.97:0/2894566177 wait complete. 2023-07-06T03:57:43.572 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:56:02.531370+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:56:03.548148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:57:43.573 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:57:43.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.770+0000 7fb8453ff700 1 Processor -- start 2023-07-06T03:57:43.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.770+0000 7fb8453ff700 1 -- start start 2023-07-06T03:57:43.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.771+0000 7fb8453ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8400cdbd0 0x7fb8400cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.771+0000 7fb8453ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb84006c060 0x7fb8400cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.771+0000 7fb8453ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84006c5e0 con 0x7fb8400b0000 2023-07-06T03:57:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.771+0000 7fb8453ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84015cc00 con 0x7fb8400cdbd0 2023-07-06T03:57:43.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.771+0000 7fb8453ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84006c8c0 con 0x7fb84006c060 2023-07-06T03:57:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.771+0000 7fb837fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8400cdbd0 0x7fb8400cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.771+0000 7fb83effd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb8400b0000 0x7fb8400cd4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60250/0 (socket says 172.21.15.97:60250) 2023-07-06T03:57:43.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.771+0000 7fb83effd700 1 -- 172.21.15.97:0/1740084255 learned_addr learned my addr 172.21.15.97:0/1740084255 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:43.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.772+0000 7fb83f7fe700 1 --2- 172.21.15.97:0/1740084255 >> v2:172.21.15.146:3300/0 conn(0x7fb84006c060 0x7fb8400cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.772+0000 7fb83e7fc700 1 -- 172.21.15.97:0/1740084255 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28147451 0 0) 0x7fb84006c5e0 con 0x7fb8400b0000 2023-07-06T03:57:43.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.772+0000 7fb83e7fc700 1 -- 172.21.15.97:0/1740084255 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb82c0035b0 con 0x7fb8400b0000 2023-07-06T03:57:43.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.772+0000 7fb837fff700 1 -- 172.21.15.97:0/1740084255 >> v2:172.21.15.146:3300/0 conn(0x7fb84006c060 msgr2=0x7fb8400cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.772+0000 7fb837fff700 1 --2- 172.21.15.97:0/1740084255 >> v2:172.21.15.146:3300/0 conn(0x7fb84006c060 0x7fb8400cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.772+0000 7fb837fff700 1 -- 172.21.15.97:0/1740084255 >> v1:172.21.15.97:6789/0 conn(0x7fb8400b0000 legacy=0x7fb8400cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.772+0000 7fb837fff700 1 -- 172.21.15.97:0/1740084255 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84006cba0 con 0x7fb8400cdbd0 2023-07-06T03:57:43.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.772+0000 7fb837fff700 1 --2- 172.21.15.97:0/1740084255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8400cdbd0 0x7fb8400cba20 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fb840071cd0 tx=0x7fb828009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a2abb95ce8fed3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:43.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.772+0000 7fb83e7fc700 1 -- 172.21.15.97:0/1740084255 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828011830 con 0x7fb8400cdbd0 2023-07-06T03:57:43.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.772+0000 7fb83e7fc700 1 -- 172.21.15.97:0/1740084255 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb828011990 con 0x7fb8400cdbd0 2023-07-06T03:57:43.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.772+0000 7fb83e7fc700 1 -- 172.21.15.97:0/1740084255 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828011d20 con 0x7fb8400cdbd0 2023-07-06T03:57:43.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.773+0000 7fb8453ff700 1 -- 172.21.15.97:0/1740084255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8400cdbd0 msgr2=0x7fb8400cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.773+0000 7fb8453ff700 1 --2- 172.21.15.97:0/1740084255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8400cdbd0 0x7fb8400cba20 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fb840071cd0 tx=0x7fb828009b20 comp rx=0 tx=0).stop 2023-07-06T03:57:43.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.773+0000 7fb8453ff700 1 -- 172.21.15.97:0/1740084255 shutdown_connections 2023-07-06T03:57:43.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.773+0000 7fb8453ff700 1 --2- 172.21.15.97:0/1740084255 >> v2:172.21.15.146:3300/0 conn(0x7fb84006c060 0x7fb8400cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.773+0000 7fb8453ff700 1 --2- 172.21.15.97:0/1740084255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8400cdbd0 0x7fb8400cba20 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.773+0000 7fb8453ff700 1 -- 172.21.15.97:0/1740084255 >> 172.21.15.97:0/1740084255 conn(0x7fb840066960 msgr2=0x7fb84006dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:43.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.773+0000 7fb8453ff700 1 -- 172.21.15.97:0/1740084255 shutdown_connections 2023-07-06T03:57:43.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.773+0000 7fb8453ff700 1 -- 172.21.15.97:0/1740084255 wait complete. 2023-07-06T03:57:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.775+0000 7fb8453ff700 1 Processor -- start 2023-07-06T03:57:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.776+0000 7fb8453ff700 1 -- start start 2023-07-06T03:57:43.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb8453ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb84006c060 0x7fb84015cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb8453ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8400b0000 0x7fb8400c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb8453ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8400cdbd0 0x7fb8400ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb8453ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400c8cf0 con 0x7fb8400cdbd0 2023-07-06T03:57:43.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb8453ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84015d650 con 0x7fb8400b0000 2023-07-06T03:57:43.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb8453ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84015d930 con 0x7fb84006c060 2023-07-06T03:57:43.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb83f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8400cdbd0 0x7fb8400ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb83f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8400cdbd0 0x7fb8400ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44032/0 (socket says 172.21.15.97:44032) 2023-07-06T03:57:43.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb83f7fe700 1 -- 172.21.15.97:0/3413910892 learned_addr learned my addr 172.21.15.97:0/3413910892 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:43.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb83effd700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb84006c060 0x7fb84015cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb837fff700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8400b0000 0x7fb8400c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb83f7fe700 1 -- 172.21.15.97:0/3413910892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb84006c060 msgr2=0x7fb84015cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb83f7fe700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb84006c060 0x7fb84015cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb83f7fe700 1 -- 172.21.15.97:0/3413910892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8400b0000 msgr2=0x7fb8400c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb83f7fe700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8400b0000 0x7fb8400c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb83f7fe700 1 -- 172.21.15.97:0/3413910892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8280094a0 con 0x7fb8400cdbd0 2023-07-06T03:57:43.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.777+0000 7fb837fff700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8400b0000 0x7fb8400c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:43.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.778+0000 7fb83f7fe700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8400cdbd0 0x7fb8400ad790 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fb838007e60 tx=0x7fb83800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:43.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.778+0000 7fb8377fe700 1 -- 172.21.15.97:0/3413910892 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83800ede0 con 0x7fb8400cdbd0 2023-07-06T03:57:43.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.778+0000 7fb8377fe700 1 -- 172.21.15.97:0/3413910892 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb838014420 con 0x7fb8400cdbd0 2023-07-06T03:57:43.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.778+0000 7fb8453ff700 1 -- 172.21.15.97:0/3413910892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84015dc10 con 0x7fb8400cdbd0 2023-07-06T03:57:43.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.778+0000 7fb8377fe700 1 -- 172.21.15.97:0/3413910892 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8380147d0 con 0x7fb8400cdbd0 2023-07-06T03:57:43.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.778+0000 7fb8453ff700 1 -- 172.21.15.97:0/3413910892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84015d290 con 0x7fb8400cdbd0 2023-07-06T03:57:43.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.779+0000 7fb8453ff700 1 -- 172.21.15.97:0/3413910892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84004e340 con 0x7fb8400cdbd0 2023-07-06T03:57:43.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.780+0000 7fb8377fe700 1 -- 172.21.15.97:0/3413910892 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb838014930 con 0x7fb8400cdbd0 2023-07-06T03:57:43.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.781+0000 7fb8377fe700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 0x7fb82006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:43.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.781+0000 7fb8377fe700 1 -- 172.21.15.97:0/3413910892 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fb838089d20 con 0x7fb8400cdbd0 2023-07-06T03:57:43.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.781+0000 7fb83effd700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 0x7fb82006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:43.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.781+0000 7fb83effd700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 0x7fb82006d190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb8300011e0 tx=0x7fb830009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:43.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.785+0000 7fb8377fe700 1 -- 172.21.15.97:0/3413910892 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb838054dc0 con 0x7fb8400cdbd0 2023-07-06T03:57:43.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.982+0000 7fb8453ff700 1 -- 172.21.15.97:0/3413910892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8400b6b50 con 0x7fb8400cdbd0 2023-07-06T03:57:43.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.982+0000 7fb8377fe700 1 -- 172.21.15.97:0/3413910892 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8380583e0 con 0x7fb8400cdbd0 2023-07-06T03:57:43.984 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:57:43.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.984+0000 7fb8453ff700 1 -- 172.21.15.97:0/3413910892 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 msgr2=0x7fb82006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.984+0000 7fb8453ff700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 0x7fb82006d190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb8300011e0 tx=0x7fb830009040 comp rx=0 tx=0).stop 2023-07-06T03:57:43.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.984+0000 7fb8453ff700 1 -- 172.21.15.97:0/3413910892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8400cdbd0 msgr2=0x7fb8400ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:43.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.984+0000 7fb8453ff700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8400cdbd0 0x7fb8400ad790 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fb838007e60 tx=0x7fb83800e3c0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.985+0000 7fb8453ff700 1 -- 172.21.15.97:0/3413910892 shutdown_connections 2023-07-06T03:57:43.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.985+0000 7fb8453ff700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb82006ace0 0x7fb82006d190 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.985+0000 7fb8453ff700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8400cdbd0 0x7fb8400ad790 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.985+0000 7fb8453ff700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb84006c060 0x7fb84015cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.985+0000 7fb8453ff700 1 --2- 172.21.15.97:0/3413910892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8400b0000 0x7fb8400c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:43.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.985+0000 7fb8453ff700 1 -- 172.21.15.97:0/3413910892 >> 172.21.15.97:0/3413910892 conn(0x7fb840066960 msgr2=0x7fb8400b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:43.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.985+0000 7fb8453ff700 1 -- 172.21.15.97:0/3413910892 shutdown_connections 2023-07-06T03:57:43.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:43.985+0000 7fb8453ff700 1 -- 172.21.15.97:0/3413910892 wait complete. 2023-07-06T03:57:44.004 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:57:44.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.201+0000 7f7f2abff700 1 Processor -- start 2023-07-06T03:57:44.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.201+0000 7f7f2abff700 1 -- start start 2023-07-06T03:57:44.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.202+0000 7f7f2abff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7f24159460 0x7f7f240ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.202+0000 7f7f2abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f240ce950 0x7f7f240cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.202+0000 7f7f2abff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7f2406c0f0 0x7f7f2406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.202+0000 7f7f2abff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f240cf300 con 0x7f7f2406c0f0 2023-07-06T03:57:44.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.202+0000 7f7f2abff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f240cab30 con 0x7f7f24159460 2023-07-06T03:57:44.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.202+0000 7f7f2abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240cad80 con 0x7f7f240ce950 2023-07-06T03:57:44.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f2919c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7f2406c0f0 0x7f7f2406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f2919c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7f2406c0f0 0x7f7f2406c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44042/0 (socket says 172.21.15.97:44042) 2023-07-06T03:57:44.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f2919c700 1 -- 172.21.15.97:0/2164527635 learned_addr learned my addr 172.21.15.97:0/2164527635 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:44.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f2899b700 1 --2- 172.21.15.97:0/2164527635 >> v2:172.21.15.146:3300/0 conn(0x7f7f24159460 0x7f7f240ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f1bfff700 1 --2- 172.21.15.97:0/2164527635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f240ce950 0x7f7f240cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f2919c700 1 -- 172.21.15.97:0/2164527635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f240ce950 msgr2=0x7f7f240cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:44.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f2919c700 1 --2- 172.21.15.97:0/2164527635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f240ce950 0x7f7f240cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f2919c700 1 -- 172.21.15.97:0/2164527635 >> v2:172.21.15.146:3300/0 conn(0x7f7f24159460 msgr2=0x7f7f240ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:44.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f2919c700 1 --2- 172.21.15.97:0/2164527635 >> v2:172.21.15.146:3300/0 conn(0x7f7f24159460 0x7f7f240ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f2919c700 1 -- 172.21.15.97:0/2164527635 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f240cb060 con 0x7f7f2406c0f0 2023-07-06T03:57:44.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f2919c700 1 --2- 172.21.15.97:0/2164527635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2406c0f0 0x7f7f2406c570 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f7f2406fb40 tx=0x7f7f1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6939ed048f0984fb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:44.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.203+0000 7f7f1b7fe700 1 -- 172.21.15.97:0/2164527635 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00e070 con 0x7f7f2406c0f0 2023-07-06T03:57:44.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.204+0000 7f7f1b7fe700 1 -- 172.21.15.97:0/2164527635 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c00b9c0 con 0x7f7f2406c0f0 2023-07-06T03:57:44.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.204+0000 7f7f1b7fe700 1 -- 172.21.15.97:0/2164527635 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00bd70 con 0x7f7f2406c0f0 2023-07-06T03:57:44.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.204+0000 7f7f2abff700 1 -- 172.21.15.97:0/2164527635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2406c0f0 msgr2=0x7f7f2406c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:44.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.204+0000 7f7f2abff700 1 --2- 172.21.15.97:0/2164527635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2406c0f0 0x7f7f2406c570 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f7f2406fb40 tx=0x7f7f1c00b330 comp rx=0 tx=0).stop 2023-07-06T03:57:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.204+0000 7f7f2abff700 1 -- 172.21.15.97:0/2164527635 shutdown_connections 2023-07-06T03:57:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.204+0000 7f7f2abff700 1 --2- 172.21.15.97:0/2164527635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2406c0f0 0x7f7f2406c570 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.204+0000 7f7f2abff700 1 --2- 172.21.15.97:0/2164527635 >> v2:172.21.15.146:3300/0 conn(0x7f7f24159460 0x7f7f240ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.204+0000 7f7f2abff700 1 --2- 172.21.15.97:0/2164527635 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f240ce950 0x7f7f240cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.204+0000 7f7f2abff700 1 -- 172.21.15.97:0/2164527635 >> 172.21.15.97:0/2164527635 conn(0x7f7f2406a060 msgr2=0x7f7f2406a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.204+0000 7f7f2abff700 1 -- 172.21.15.97:0/2164527635 shutdown_connections 2023-07-06T03:57:44.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.204+0000 7f7f2abff700 1 -- 172.21.15.97:0/2164527635 wait complete. 2023-07-06T03:57:44.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.206+0000 7f7f2abff700 1 Processor -- start 2023-07-06T03:57:44.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.206+0000 7f7f2abff700 1 -- start start 2023-07-06T03:57:44.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.207+0000 7f7f2abff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2406c0f0 0x7f7f24151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.207+0000 7f7f2abff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f240ce950 0x7f7f240b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.207+0000 7f7f2abff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f24159460 0x7f7f24070f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.207+0000 7f7f2abff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240af8f0 con 0x7f7f2406c0f0 2023-07-06T03:57:44.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.207+0000 7f7f2abff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24151b70 con 0x7f7f240ce950 2023-07-06T03:57:44.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.207+0000 7f7f2abff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240b2880 con 0x7f7f24159460 2023-07-06T03:57:44.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.207+0000 7f7f2899b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2406c0f0 0x7f7f24151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.207+0000 7f7f2899b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2406c0f0 0x7f7f24151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44058/0 (socket says 172.21.15.97:44058) 2023-07-06T03:57:44.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.207+0000 7f7f2899b700 1 -- 172.21.15.97:0/1048347968 learned_addr learned my addr 172.21.15.97:0/1048347968 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:44.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.207+0000 7f7f1bfff700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f240ce950 0x7f7f240b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.207+0000 7f7f2919c700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f24159460 0x7f7f24070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.208+0000 7f7f2899b700 1 -- 172.21.15.97:0/1048347968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f24159460 msgr2=0x7f7f24070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:44.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.208+0000 7f7f2899b700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f24159460 0x7f7f24070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.208+0000 7f7f2899b700 1 -- 172.21.15.97:0/1048347968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f240ce950 msgr2=0x7f7f240b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:44.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.208+0000 7f7f2899b700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f240ce950 0x7f7f240b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.208+0000 7f7f2899b700 1 -- 172.21.15.97:0/1048347968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c009b30 con 0x7f7f2406c0f0 2023-07-06T03:57:44.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.208+0000 7f7f2899b700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2406c0f0 0x7f7f24151630 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000d800 tx=0x7f7f2000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:44.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.208+0000 7f7f197fa700 1 -- 172.21.15.97:0/1048347968 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f200125f0 con 0x7f7f2406c0f0 2023-07-06T03:57:44.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.208+0000 7f7f197fa700 1 -- 172.21.15.97:0/1048347968 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f20012750 con 0x7f7f2406c0f0 2023-07-06T03:57:44.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.208+0000 7f7f2abff700 1 -- 172.21.15.97:0/1048347968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f240b2b60 con 0x7f7f2406c0f0 2023-07-06T03:57:44.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.208+0000 7f7f197fa700 1 -- 172.21.15.97:0/1048347968 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20012b00 con 0x7f7f2406c0f0 2023-07-06T03:57:44.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.208+0000 7f7f2abff700 1 -- 172.21.15.97:0/1048347968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f240c5a90 con 0x7f7f2406c0f0 2023-07-06T03:57:44.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.209+0000 7f7f2abff700 1 -- 172.21.15.97:0/1048347968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f240afb60 con 0x7f7f2406c0f0 2023-07-06T03:57:44.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.210+0000 7f7f197fa700 1 -- 172.21.15.97:0/1048347968 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f20012c60 con 0x7f7f2406c0f0 2023-07-06T03:57:44.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.211+0000 7f7f197fa700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f1006ace0 0x7f7f1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.211+0000 7f7f197fa700 1 -- 172.21.15.97:0/1048347968 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f7f20089880 con 0x7f7f2406c0f0 2023-07-06T03:57:44.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.211+0000 7f7f1bfff700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f1006ace0 0x7f7f1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.212+0000 7f7f1bfff700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f1006ace0 0x7f7f1006d190 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7f24071b50 tx=0x7f7f14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:44.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.216+0000 7f7f197fa700 1 -- 172.21.15.97:0/1048347968 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7f20054870 con 0x7f7f2406c0f0 2023-07-06T03:57:44.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.412+0000 7f7f2abff700 1 -- 172.21.15.97:0/1048347968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7f2404e340 con 0x7f7f2406c0f0 2023-07-06T03:57:44.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.412+0000 7f7f197fa700 1 -- 172.21.15.97:0/1048347968 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7f20057e90 con 0x7f7f2406c0f0 2023-07-06T03:57:44.414 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:57:44.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 -- 172.21.15.97:0/1048347968 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f1006ace0 msgr2=0x7f7f1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:44.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f1006ace0 0x7f7f1006d190 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7f24071b50 tx=0x7f7f14008040 comp rx=0 tx=0).stop 2023-07-06T03:57:44.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 -- 172.21.15.97:0/1048347968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2406c0f0 msgr2=0x7f7f24151630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:44.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2406c0f0 0x7f7f24151630 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000d800 tx=0x7f7f2000dbc0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 -- 172.21.15.97:0/1048347968 shutdown_connections 2023-07-06T03:57:44.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7f1006ace0 0x7f7f1006d190 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7f24159460 0x7f7f24070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f2406c0f0 0x7f7f24151630 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 --2- 172.21.15.97:0/1048347968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f240ce950 0x7f7f240b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 -- 172.21.15.97:0/1048347968 >> 172.21.15.97:0/1048347968 conn(0x7f7f2406a060 msgr2=0x7f7f2406a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:44.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 -- 172.21.15.97:0/1048347968 shutdown_connections 2023-07-06T03:57:44.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.415+0000 7f7f2abff700 1 -- 172.21.15.97:0/1048347968 wait complete. 2023-07-06T03:57:44.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:57:44.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.630+0000 7f7a83bff700 1 Processor -- start 2023-07-06T03:57:44.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.630+0000 7f7a83bff700 1 -- start start 2023-07-06T03:57:44.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.631+0000 7f7a83bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a7c0cf7e0 0x7f7a7c0d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.631+0000 7f7a83bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a7c06b380 0x7f7a7c0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.631+0000 7f7a83bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7c06b7f0 con 0x7f7a7c0cd490 2023-07-06T03:57:44.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.631+0000 7f7a83bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7c0d2460 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.631+0000 7f7a83bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c0d00f0 con 0x7f7a7c06b380 2023-07-06T03:57:44.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.631+0000 7f7a8199b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a7c0cf7e0 0x7f7a7c0d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.631+0000 7f7a8199b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a7c0cf7e0 0x7f7a7c0d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44060/0 (socket says 172.21.15.97:44060) 2023-07-06T03:57:44.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.631+0000 7f7a8199b700 1 -- 172.21.15.97:0/683249871 learned_addr learned my addr 172.21.15.97:0/683249871 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:44.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.632+0000 7f7a8199b700 1 -- 172.21.15.97:0/683249871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a7c06b380 msgr2=0x7f7a7c0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:44.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.632+0000 7f7a8119a700 1 --2- 172.21.15.97:0/683249871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a7c06b380 0x7f7a7c0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.632+0000 7f7a8199b700 1 --2- 172.21.15.97:0/683249871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a7c06b380 0x7f7a7c0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.632+0000 7f7a8199b700 1 -- 172.21.15.97:0/683249871 >> v1:172.21.15.146:6789/0 conn(0x7f7a7c0cd490 legacy=0x7f7a7c0b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:44.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.632+0000 7f7a8199b700 1 -- 172.21.15.97:0/683249871 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c0d1950 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.632+0000 7f7a8119a700 1 --2- 172.21.15.97:0/683249871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a7c06b380 0x7f7a7c0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:44.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.632+0000 7f7a8199b700 1 --2- 172.21.15.97:0/683249871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7c0cf7e0 0x7f7a7c0d1360 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f7a780098f0 tx=0x7f7a78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca96cd963ace8fa7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:44.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.632+0000 7f7a80999700 1 -- 172.21.15.97:0/683249871 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78004230 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.632+0000 7f7a80999700 1 -- 172.21.15.97:0/683249871 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a78004390 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.632+0000 7f7a80999700 1 -- 172.21.15.97:0/683249871 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a780036a0 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.633+0000 7f7a83bff700 1 -- 172.21.15.97:0/683249871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7c0cf7e0 msgr2=0x7f7a7c0d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:44.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.633+0000 7f7a83bff700 1 --2- 172.21.15.97:0/683249871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7c0cf7e0 0x7f7a7c0d1360 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f7a780098f0 tx=0x7f7a78009c00 comp rx=0 tx=0).stop 2023-07-06T03:57:44.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.633+0000 7f7a83bff700 1 -- 172.21.15.97:0/683249871 shutdown_connections 2023-07-06T03:57:44.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.633+0000 7f7a83bff700 1 --2- 172.21.15.97:0/683249871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7c0cf7e0 0x7f7a7c0d1360 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.633+0000 7f7a83bff700 1 --2- 172.21.15.97:0/683249871 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a7c06b380 0x7f7a7c0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.633+0000 7f7a83bff700 1 -- 172.21.15.97:0/683249871 >> 172.21.15.97:0/683249871 conn(0x7f7a7c06cf30 msgr2=0x7f7a7c06a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:44.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.633+0000 7f7a83bff700 1 -- 172.21.15.97:0/683249871 shutdown_connections 2023-07-06T03:57:44.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.633+0000 7f7a83bff700 1 -- 172.21.15.97:0/683249871 wait complete. 2023-07-06T03:57:44.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.635+0000 7f7a83bff700 1 Processor -- start 2023-07-06T03:57:44.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.635+0000 7f7a83bff700 1 -- start start 2023-07-06T03:57:44.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a83bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a7c06b380 0x7f7a7c0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a83bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a7c0cd490 0x7f7a7c14e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a83bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7c0cf7e0 0x7f7a7c0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a83bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c14c970 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a83bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c14baa0 con 0x7f7a7c0cd490 2023-07-06T03:57:44.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a83bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c0b1770 con 0x7f7a7c06b380 2023-07-06T03:57:44.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a8219c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7c0cf7e0 0x7f7a7c0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a8219c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7c0cf7e0 0x7f7a7c0b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44072/0 (socket says 172.21.15.97:44072) 2023-07-06T03:57:44.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a8219c700 1 -- 172.21.15.97:0/2511174017 learned_addr learned my addr 172.21.15.97:0/2511174017 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:44.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a8199b700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a7c06b380 0x7f7a7c0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a8219c700 1 -- 172.21.15.97:0/2511174017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a7c06b380 msgr2=0x7f7a7c0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:44.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a8219c700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a7c06b380 0x7f7a7c0b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a8119a700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a7c0cd490 0x7f7a7c14e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a8219c700 1 -- 172.21.15.97:0/2511174017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a7c0cd490 msgr2=0x7f7a7c14e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:44.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a8219c700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a7c0cd490 0x7f7a7c14e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a8219c700 1 -- 172.21.15.97:0/2511174017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78009550 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.636+0000 7f7a8119a700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a7c0cd490 0x7f7a7c14e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:44.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.637+0000 7f7a8219c700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7c0cf7e0 0x7f7a7c0b1230 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400cc40 tx=0x7f7a7400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:44.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.637+0000 7f7a727fc700 1 -- 172.21.15.97:0/2511174017 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7400aba0 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.637+0000 7f7a727fc700 1 -- 172.21.15.97:0/2511174017 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7400ad00 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.637+0000 7f7a83bff700 1 -- 172.21.15.97:0/2511174017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7c14bd60 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.637+0000 7f7a727fc700 1 -- 172.21.15.97:0/2511174017 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74012660 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.637+0000 7f7a83bff700 1 -- 172.21.15.97:0/2511174017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7c14c320 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.638+0000 7f7a83bff700 1 -- 172.21.15.97:0/2511174017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7c067010 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.639+0000 7f7a727fc700 1 -- 172.21.15.97:0/2511174017 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7a740127c0 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.640+0000 7f7a727fc700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7a6006ace0 0x7f7a6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:44.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.641+0000 7f7a727fc700 1 -- 172.21.15.97:0/2511174017 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f7a74089d70 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.641+0000 7f7a8199b700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7a6006ace0 0x7f7a6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:44.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.641+0000 7f7a8199b700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7a6006ace0 0x7f7a6006d190 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009c40 tx=0x7f7a78008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:44.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.645+0000 7f7a727fc700 1 -- 172.21.15.97:0/2511174017 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7a74054cd0 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.837+0000 7f7a83bff700 1 -- 172.21.15.97:0/2511174017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a7c04e340 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.838+0000 7f7a727fc700 1 -- 172.21.15.97:0/2511174017 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a74016070 con 0x7f7a7c0cf7e0 2023-07-06T03:57:44.840 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:57:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 -- 172.21.15.97:0/2511174017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7a6006ace0 msgr2=0x7f7a6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7a6006ace0 0x7f7a6006d190 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009c40 tx=0x7f7a78008030 comp rx=0 tx=0).stop 2023-07-06T03:57:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 -- 172.21.15.97:0/2511174017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7c0cf7e0 msgr2=0x7f7a7c0b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7c0cf7e0 0x7f7a7c0b1230 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400cc40 tx=0x7f7a7400a060 comp rx=0 tx=0).stop 2023-07-06T03:57:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 -- 172.21.15.97:0/2511174017 shutdown_connections 2023-07-06T03:57:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f7a6006ace0 0x7f7a6006d190 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a7c0cf7e0 0x7f7a7c0b1230 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f7a7c06b380 0x7f7a7c0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 --2- 172.21.15.97:0/2511174017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a7c0cd490 0x7f7a7c14e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 -- 172.21.15.97:0/2511174017 >> 172.21.15.97:0/2511174017 conn(0x7f7a7c06cf30 msgr2=0x7f7a7c0b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 -- 172.21.15.97:0/2511174017 shutdown_connections 2023-07-06T03:57:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:44.840+0000 7f7a83bff700 1 -- 172.21.15.97:0/2511174017 wait complete. 2023-07-06T03:57:44.859 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:57:45.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.056+0000 7f20729ff700 1 Processor -- start 2023-07-06T03:57:45.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.057+0000 7f20729ff700 1 -- start start 2023-07-06T03:57:45.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f20729ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f206c0cf7e0 0x7f206c0d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f20729ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f206c06b380 0x7f206c0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f20729ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206c06b7f0 con 0x7f206c0cd490 2023-07-06T03:57:45.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f20729ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206c0d2460 con 0x7f206c0cf7e0 2023-07-06T03:57:45.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f20729ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0d00f0 con 0x7f206c06b380 2023-07-06T03:57:45.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f206bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f206c0cf7e0 0x7f206c0d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:45.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f206bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f206c0cf7e0 0x7f206c0d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44078/0 (socket says 172.21.15.97:44078) 2023-07-06T03:57:45.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f206bfff700 1 -- 172.21.15.97:0/216982092 learned_addr learned my addr 172.21.15.97:0/216982092 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:45.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f2063fff700 1 --2- 172.21.15.97:0/216982092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f206c06b380 0x7f206c0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:45.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f206bfff700 1 -- 172.21.15.97:0/216982092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f206c06b380 msgr2=0x7f206c0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:45.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f206bfff700 1 --2- 172.21.15.97:0/216982092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f206c06b380 0x7f206c0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.058+0000 7f206bfff700 1 -- 172.21.15.97:0/216982092 >> v1:172.21.15.146:6789/0 conn(0x7f206c0cd490 legacy=0x7f206c0b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:45.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.059+0000 7f206bfff700 1 -- 172.21.15.97:0/216982092 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c0d1950 con 0x7f206c0cf7e0 2023-07-06T03:57:45.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.059+0000 7f206bfff700 1 --2- 172.21.15.97:0/216982092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206c0cf7e0 0x7f206c0d1360 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f205c0098f0 tx=0x7f205c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2734cdb3da86fbe6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:45.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.059+0000 7f206b7fe700 1 -- 172.21.15.97:0/216982092 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c004230 con 0x7f206c0cf7e0 2023-07-06T03:57:45.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.059+0000 7f206b7fe700 1 -- 172.21.15.97:0/216982092 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205c004390 con 0x7f206c0cf7e0 2023-07-06T03:57:45.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.059+0000 7f206b7fe700 1 -- 172.21.15.97:0/216982092 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c0036a0 con 0x7f206c0cf7e0 2023-07-06T03:57:45.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.059+0000 7f20729ff700 1 -- 172.21.15.97:0/216982092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206c0cf7e0 msgr2=0x7f206c0d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:45.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.059+0000 7f20729ff700 1 --2- 172.21.15.97:0/216982092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206c0cf7e0 0x7f206c0d1360 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f205c0098f0 tx=0x7f205c009c00 comp rx=0 tx=0).stop 2023-07-06T03:57:45.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.060+0000 7f20729ff700 1 -- 172.21.15.97:0/216982092 shutdown_connections 2023-07-06T03:57:45.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.060+0000 7f20729ff700 1 --2- 172.21.15.97:0/216982092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206c0cf7e0 0x7f206c0d1360 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.060+0000 7f20729ff700 1 --2- 172.21.15.97:0/216982092 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f206c06b380 0x7f206c0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.060+0000 7f20729ff700 1 -- 172.21.15.97:0/216982092 >> 172.21.15.97:0/216982092 conn(0x7f206c06cf30 msgr2=0x7f206c06a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:45.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.060+0000 7f20729ff700 1 -- 172.21.15.97:0/216982092 shutdown_connections 2023-07-06T03:57:45.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.060+0000 7f20729ff700 1 -- 172.21.15.97:0/216982092 wait complete. 2023-07-06T03:57:45.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.061+0000 7f20729ff700 1 Processor -- start 2023-07-06T03:57:45.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.061+0000 7f20729ff700 1 -- start start 2023-07-06T03:57:45.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.062+0000 7f20729ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206c06b380 0x7f206c0b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.062+0000 7f20729ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f206c0cd490 0x7f206c148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.062+0000 7f20729ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f206c0cf7e0 0x7f206c15f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.062+0000 7f20729ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c15e160 con 0x7f206c06b380 2023-07-06T03:57:45.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.062+0000 7f20729ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c15e440 con 0x7f206c0cf7e0 2023-07-06T03:57:45.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.062+0000 7f20729ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c15e720 con 0x7f206c0cd490 2023-07-06T03:57:45.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f206bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206c06b380 0x7f206c0b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:45.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f206bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206c06b380 0x7f206c0b5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44080/0 (socket says 172.21.15.97:44080) 2023-07-06T03:57:45.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f206bfff700 1 -- 172.21.15.97:0/4211835248 learned_addr learned my addr 172.21.15.97:0/4211835248 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:45.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f206bfff700 1 -- 172.21.15.97:0/4211835248 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f206c0cd490 msgr2=0x7f206c148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:45.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f206bfff700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f206c0cd490 0x7f206c148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f206bfff700 1 -- 172.21.15.97:0/4211835248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f206c0cf7e0 msgr2=0x7f206c15f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:45.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f206bfff700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f206c0cf7e0 0x7f206c15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f206bfff700 1 -- 172.21.15.97:0/4211835248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c009550 con 0x7f206c06b380 2023-07-06T03:57:45.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f206bfff700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206c06b380 0x7f206c0b5260 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f205c009130 tx=0x7f205c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:45.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f20697fa700 1 -- 172.21.15.97:0/4211835248 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c003e50 con 0x7f206c06b380 2023-07-06T03:57:45.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f20697fa700 1 -- 172.21.15.97:0/4211835248 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205c00b660 con 0x7f206c06b380 2023-07-06T03:57:45.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f20729ff700 1 -- 172.21.15.97:0/4211835248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206c15ea00 con 0x7f206c06b380 2023-07-06T03:57:45.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.063+0000 7f20697fa700 1 -- 172.21.15.97:0/4211835248 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c00ba10 con 0x7f206c06b380 2023-07-06T03:57:45.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.064+0000 7f20729ff700 1 -- 172.21.15.97:0/4211835248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c1494c0 con 0x7f206c06b380 2023-07-06T03:57:45.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.065+0000 7f20729ff700 1 -- 172.21.15.97:0/4211835248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206c15ef20 con 0x7f206c06b380 2023-07-06T03:57:45.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.065+0000 7f20697fa700 1 -- 172.21.15.97:0/4211835248 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f205c00bb70 con 0x7f206c06b380 2023-07-06T03:57:45.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.066+0000 7f20697fa700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f204c06ace0 0x7f204c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.066+0000 7f20697fa700 1 -- 172.21.15.97:0/4211835248 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f205c08acb0 con 0x7f206c06b380 2023-07-06T03:57:45.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.067+0000 7f2063fff700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f204c06ace0 0x7f204c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:45.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.067+0000 7f2063fff700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f204c06ace0 0x7f204c06d190 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f206c15a1a0 tx=0x7f2054008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:45.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.071+0000 7f20697fa700 1 -- 172.21.15.97:0/4211835248 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f206c15ef20 con 0x7f206c06b380 2023-07-06T03:57:45.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.269+0000 7f20729ff700 1 -- 172.21.15.97:0/4211835248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f206c06b7f0 con 0x7f206c06b380 2023-07-06T03:57:45.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.269+0000 7f20697fa700 1 -- 172.21.15.97:0/4211835248 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+11 (secure 0 0 0) 0x7f205c01f020 con 0x7f206c06b380 2023-07-06T03:57:45.271 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:57:45.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 -- 172.21.15.97:0/4211835248 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f204c06ace0 msgr2=0x7f204c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f204c06ace0 0x7f204c06d190 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f206c15a1a0 tx=0x7f2054008040 comp rx=0 tx=0).stop 2023-07-06T03:57:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 -- 172.21.15.97:0/4211835248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206c06b380 msgr2=0x7f206c0b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206c06b380 0x7f206c0b5260 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f205c009130 tx=0x7f205c004750 comp rx=0 tx=0).stop 2023-07-06T03:57:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 -- 172.21.15.97:0/4211835248 shutdown_connections 2023-07-06T03:57:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f204c06ace0 0x7f204c06d190 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f206c0cf7e0 0x7f206c15f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f206c06b380 0x7f206c0b5260 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 --2- 172.21.15.97:0/4211835248 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f206c0cd490 0x7f206c148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 -- 172.21.15.97:0/4211835248 >> 172.21.15.97:0/4211835248 conn(0x7f206c06cf30 msgr2=0x7f206c0b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 -- 172.21.15.97:0/4211835248 shutdown_connections 2023-07-06T03:57:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.272+0000 7f20729ff700 1 -- 172.21.15.97:0/4211835248 wait complete. 2023-07-06T03:57:45.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T03:57:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.486+0000 7f1ec0dff700 1 Processor -- start 2023-07-06T03:57:45.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.486+0000 7f1ec0dff700 1 -- start start 2023-07-06T03:57:45.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.487+0000 7f1ec0dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ebc0c5160 0x7f1ebc06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.487+0000 7f1ec0dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ebc06dd60 0x7f1ebc06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.487+0000 7f1ec0dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc06e1b0 con 0x7f1ebc06a5a0 2023-07-06T03:57:45.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.487+0000 7f1ec0dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc15a860 con 0x7f1ebc0c5160 2023-07-06T03:57:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.487+0000 7f1ec0dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc06a9c0 con 0x7f1ebc06dd60 2023-07-06T03:57:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.487+0000 7f1eba59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ebc0c5160 0x7f1ebc06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.487+0000 7f1eba59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ebc0c5160 0x7f1ebc06d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44094/0 (socket says 172.21.15.97:44094) 2023-07-06T03:57:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.487+0000 7f1eba59c700 1 -- 172.21.15.97:0/2237451175 learned_addr learned my addr 172.21.15.97:0/2237451175 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.488+0000 7f1eb9d9b700 1 --2- 172.21.15.97:0/2237451175 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ebc06dd60 0x7f1ebc06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.488+0000 7f1eba59c700 1 -- 172.21.15.97:0/2237451175 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ebc06dd60 msgr2=0x7f1ebc06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.488+0000 7f1eba59c700 1 --2- 172.21.15.97:0/2237451175 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ebc06dd60 0x7f1ebc06a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.488+0000 7f1eba59c700 1 -- 172.21.15.97:0/2237451175 >> v1:172.21.15.146:6789/0 conn(0x7f1ebc06a5a0 legacy=0x7f1ebc06fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.488+0000 7f1eba59c700 1 -- 172.21.15.97:0/2237451175 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0ca410 con 0x7f1ebc0c5160 2023-07-06T03:57:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.488+0000 7f1eba59c700 1 --2- 172.21.15.97:0/2237451175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc0c5160 0x7f1ebc06d820 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0098f0 tx=0x7f1eac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c6c4a02014bc07 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.488+0000 7f1eb959a700 1 -- 172.21.15.97:0/2237451175 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac011830 con 0x7f1ebc0c5160 2023-07-06T03:57:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.488+0000 7f1eb959a700 1 -- 172.21.15.97:0/2237451175 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac011990 con 0x7f1ebc0c5160 2023-07-06T03:57:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.488+0000 7f1eb959a700 1 -- 172.21.15.97:0/2237451175 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac011d20 con 0x7f1ebc0c5160 2023-07-06T03:57:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.489+0000 7f1ec0dff700 1 -- 172.21.15.97:0/2237451175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc0c5160 msgr2=0x7f1ebc06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.489+0000 7f1ec0dff700 1 --2- 172.21.15.97:0/2237451175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc0c5160 0x7f1ebc06d820 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0098f0 tx=0x7f1eac009c00 comp rx=0 tx=0).stop 2023-07-06T03:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.489+0000 7f1ec0dff700 1 -- 172.21.15.97:0/2237451175 shutdown_connections 2023-07-06T03:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.489+0000 7f1ec0dff700 1 --2- 172.21.15.97:0/2237451175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc0c5160 0x7f1ebc06d820 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.489+0000 7f1ec0dff700 1 --2- 172.21.15.97:0/2237451175 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ebc06dd60 0x7f1ebc06a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.489+0000 7f1ec0dff700 1 -- 172.21.15.97:0/2237451175 >> 172.21.15.97:0/2237451175 conn(0x7f1ebc0cc0e0 msgr2=0x7f1ebc0ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.489+0000 7f1ec0dff700 1 -- 172.21.15.97:0/2237451175 shutdown_connections 2023-07-06T03:57:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.489+0000 7f1ec0dff700 1 -- 172.21.15.97:0/2237451175 wait complete. 2023-07-06T03:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.491+0000 7f1ec0dff700 1 Processor -- start 2023-07-06T03:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.491+0000 7f1ec0dff700 1 -- start start 2023-07-06T03:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1ec0dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ebc06a5a0 0x7f1ebc0c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1ec0dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc06dd60 0x7f1ebc0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1ec0dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ebc0c5160 0x7f1ebc144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1ec0dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0c4da0 con 0x7f1ebc06dd60 2023-07-06T03:57:45.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1ec0dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0c68f0 con 0x7f1ebc06a5a0 2023-07-06T03:57:45.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1ec0dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0c7340 con 0x7f1ebc0c5160 2023-07-06T03:57:45.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1eb9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc06dd60 0x7f1ebc0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:45.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1eb9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc06dd60 0x7f1ebc0c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44110/0 (socket says 172.21.15.97:44110) 2023-07-06T03:57:45.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1eb9d9b700 1 -- 172.21.15.97:0/4060076957 learned_addr learned my addr 172.21.15.97:0/4060076957 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1ebad9d700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ebc0c5160 0x7f1ebc144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1eba59c700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ebc06a5a0 0x7f1ebc0c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1eb9d9b700 1 -- 172.21.15.97:0/4060076957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ebc0c5160 msgr2=0x7f1ebc144ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1eb9d9b700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ebc0c5160 0x7f1ebc144ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1eb9d9b700 1 -- 172.21.15.97:0/4060076957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ebc06a5a0 msgr2=0x7f1ebc0c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1eb9d9b700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ebc06a5a0 0x7f1ebc0c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1eb9d9b700 1 -- 172.21.15.97:0/4060076957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac009550 con 0x7f1ebc06dd60 2023-07-06T03:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.492+0000 7f1ebad9d700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ebc0c5160 0x7f1ebc144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.493+0000 7f1eb9d9b700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc06dd60 0x7f1ebc0c8100 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000e990 tx=0x7f1eb000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.493+0000 7f1eaaffd700 1 -- 172.21.15.97:0/4060076957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00096f0 con 0x7f1ebc06dd60 2023-07-06T03:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.493+0000 7f1eaaffd700 1 -- 172.21.15.97:0/4060076957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb000c850 con 0x7f1ebc06dd60 2023-07-06T03:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.493+0000 7f1ec0dff700 1 -- 172.21.15.97:0/4060076957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc0c7620 con 0x7f1ebc06dd60 2023-07-06T03:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.493+0000 7f1eaaffd700 1 -- 172.21.15.97:0/4060076957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000cbc0 con 0x7f1ebc06dd60 2023-07-06T03:57:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.493+0000 7f1ec0dff700 1 -- 172.21.15.97:0/4060076957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc0c7be0 con 0x7f1ebc06dd60 2023-07-06T03:57:45.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.494+0000 7f1ec0dff700 1 -- 172.21.15.97:0/4060076957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc04e340 con 0x7f1ebc06dd60 2023-07-06T03:57:45.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.495+0000 7f1eaaffd700 1 -- 172.21.15.97:0/4060076957 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1eb000cd20 con 0x7f1ebc06dd60 2023-07-06T03:57:45.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.496+0000 7f1eaaffd700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1e9c06ace0 0x7f1e9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.496+0000 7f1eaaffd700 1 -- 172.21.15.97:0/4060076957 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f1eb00899e0 con 0x7f1ebc06dd60 2023-07-06T03:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.496+0000 7f1eba59c700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1e9c06ace0 0x7f1e9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:45.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.497+0000 7f1eba59c700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1e9c06ace0 0x7f1e9c06d190 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1eac014040 tx=0x7f1eac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:45.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.500+0000 7f1eaaffd700 1 -- 172.21.15.97:0/4060076957 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1eb00549d0 con 0x7f1ebc06dd60 2023-07-06T03:57:45.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.711+0000 7f1ec0dff700 1 -- 172.21.15.97:0/4060076957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1ebc0700e0 con 0x7f1ebc06dd60 2023-07-06T03:57:45.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.712+0000 7f1eaaffd700 1 -- 172.21.15.97:0/4060076957 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v138) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1eb0017080 con 0x7f1ebc06dd60 2023-07-06T03:57:45.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 -- 172.21.15.97:0/4060076957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1e9c06ace0 msgr2=0x7f1e9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:45.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1e9c06ace0 0x7f1e9c06d190 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1eac014040 tx=0x7f1eac008030 comp rx=0 tx=0).stop 2023-07-06T03:57:45.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 -- 172.21.15.97:0/4060076957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc06dd60 msgr2=0x7f1ebc0c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:45.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc06dd60 0x7f1ebc0c8100 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000e990 tx=0x7f1eb000ed50 comp rx=0 tx=0).stop 2023-07-06T03:57:45.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 -- 172.21.15.97:0/4060076957 shutdown_connections 2023-07-06T03:57:45.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1e9c06ace0 0x7f1e9c06d190 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1ebc0c5160 0x7f1ebc144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ebc06a5a0 0x7f1ebc0c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 --2- 172.21.15.97:0/4060076957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc06dd60 0x7f1ebc0c8100 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:45.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 -- 172.21.15.97:0/4060076957 >> 172.21.15.97:0/4060076957 conn(0x7f1ebc0cc0e0 msgr2=0x7f1ebc0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:45.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 -- 172.21.15.97:0/4060076957 shutdown_connections 2023-07-06T03:57:45.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:45.714+0000 7f1ec0dff700 1 -- 172.21.15.97:0/4060076957 wait complete. 2023-07-06T03:57:45.718 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-07-06T03:57:45.733 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2023-07-06T03:57:45.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:45.734 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-06T03:57:45.807 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi146.stderr:ceph-fuse[127880]: fuse finished with error 0 and tester_r 0 2023-07-06T03:57:45.810 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:57:45.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:45.811 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T03:57:45.973 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:57:45.974 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:57:46.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:57:46.019 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:57:46.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:46.020 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T03:57:46.104 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T03:57:46.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:57:46.106 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:57:46.106 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:57:46.153 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:57:46.154 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:57:46.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:46.154 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T03:57:46.239 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T03:57:46.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:57:46.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:57:46.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.432+0000 7f6496fff700 1 Processor -- start 2023-07-06T03:57:46.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.433+0000 7f6496fff700 1 -- start start 2023-07-06T03:57:46.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.434+0000 7f6496fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f64900d09d0 0x7f64900b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.434+0000 7f6496fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 0x7f64900afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.434+0000 7f6496fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64900cf0e0 con 0x7f649015ca10 2023-07-06T03:57:46.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.434+0000 7f6496fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64900ce2c0 con 0x7f64900d09d0 2023-07-06T03:57:46.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.434+0000 7f6496fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64900cf570 con 0x7f64900cde60 2023-07-06T03:57:46.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.434+0000 7f6494d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f64900d09d0 0x7f64900b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.434+0000 7f648ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f649015ca10 0x7f64900d0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60256/0 (socket says 172.21.15.97:60256) 2023-07-06T03:57:46.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.434+0000 7f648ffff700 1 -- 172.21.15.97:0/1342186849 learned_addr learned my addr 172.21.15.97:0/1342186849 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:46.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.434+0000 7f649559c700 1 --2- 172.21.15.97:0/1342186849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 0x7f64900afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.435+0000 7f6494d9b700 1 -- 172.21.15.97:0/1342186849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 msgr2=0x7f64900afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:46.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.435+0000 7f6494d9b700 1 --2- 172.21.15.97:0/1342186849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 0x7f64900afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.435+0000 7f6494d9b700 1 -- 172.21.15.97:0/1342186849 >> v1:172.21.15.97:6789/0 conn(0x7f649015ca10 legacy=0x7f64900d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:46.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.435+0000 7f6494d9b700 1 -- 172.21.15.97:0/1342186849 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900d3d00 con 0x7f64900d09d0 2023-07-06T03:57:46.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.435+0000 7f649559c700 1 --2- 172.21.15.97:0/1342186849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 0x7f64900afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:46.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.435+0000 7f6494d9b700 1 --2- 172.21.15.97:0/1342186849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64900d09d0 0x7f64900b1dd0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f6484009840 tx=0x7f6484009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ef6252be83a3b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:46.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.435+0000 7f648f7fe700 1 -- 172.21.15.97:0/1342186849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484011830 con 0x7f64900d09d0 2023-07-06T03:57:46.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.435+0000 7f648f7fe700 1 -- 172.21.15.97:0/1342186849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6484011990 con 0x7f64900d09d0 2023-07-06T03:57:46.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.435+0000 7f648f7fe700 1 -- 172.21.15.97:0/1342186849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484011d20 con 0x7f64900d09d0 2023-07-06T03:57:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.436+0000 7f6496fff700 1 -- 172.21.15.97:0/1342186849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64900d09d0 msgr2=0x7f64900b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.436+0000 7f6496fff700 1 --2- 172.21.15.97:0/1342186849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64900d09d0 0x7f64900b1dd0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f6484009840 tx=0x7f6484009b50 comp rx=0 tx=0).stop 2023-07-06T03:57:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.436+0000 7f6496fff700 1 -- 172.21.15.97:0/1342186849 shutdown_connections 2023-07-06T03:57:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.436+0000 7f6496fff700 1 --2- 172.21.15.97:0/1342186849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64900d09d0 0x7f64900b1dd0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.436+0000 7f6496fff700 1 --2- 172.21.15.97:0/1342186849 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 0x7f64900afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.436+0000 7f6496fff700 1 -- 172.21.15.97:0/1342186849 >> 172.21.15.97:0/1342186849 conn(0x7f64900720b0 msgr2=0x7f64900d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:46.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.436+0000 7f6496fff700 1 -- 172.21.15.97:0/1342186849 shutdown_connections 2023-07-06T03:57:46.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.436+0000 7f6496fff700 1 -- 172.21.15.97:0/1342186849 wait complete. 2023-07-06T03:57:46.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.438+0000 7f6496fff700 1 Processor -- start 2023-07-06T03:57:46.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.438+0000 7f6496fff700 1 -- start start 2023-07-06T03:57:46.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f6496fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 0x7f649014cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f6496fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64900d09d0 0x7f64900c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f6496fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f649015f560 0x7f649015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f6496fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649014c430 con 0x7f64900d09d0 2023-07-06T03:57:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f6496fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649015ff20 con 0x7f649015f560 2023-07-06T03:57:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f6496fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64900c8540 con 0x7f64900cde60 2023-07-06T03:57:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f648ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64900d09d0 0x7f64900c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f648ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64900d09d0 0x7f64900c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44122/0 (socket says 172.21.15.97:44122) 2023-07-06T03:57:46.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f648ffff700 1 -- 172.21.15.97:0/3664798532 learned_addr learned my addr 172.21.15.97:0/3664798532 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f6494d9b700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 0x7f649014cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f649559c700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f649015f560 0x7f649015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f648ffff700 1 -- 172.21.15.97:0/3664798532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 msgr2=0x7f649014cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f648ffff700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 0x7f649014cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f648ffff700 1 -- 172.21.15.97:0/3664798532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f649015f560 msgr2=0x7f649015f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:46.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f648ffff700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f649015f560 0x7f649015f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f648ffff700 1 -- 172.21.15.97:0/3664798532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64840094a0 con 0x7f64900d09d0 2023-07-06T03:57:46.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f6494d9b700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 0x7f649014cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:46.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.439+0000 7f649559c700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f649015f560 0x7f649015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:46.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.440+0000 7f648ffff700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64900d09d0 0x7f64900c9ed0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c4b0 tx=0x7f647c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:46.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.440+0000 7f648d7fa700 1 -- 172.21.15.97:0/3664798532 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c005d70 con 0x7f64900d09d0 2023-07-06T03:57:46.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.440+0000 7f648d7fa700 1 -- 172.21.15.97:0/3664798532 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647c0070d0 con 0x7f64900d09d0 2023-07-06T03:57:46.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.440+0000 7f6496fff700 1 -- 172.21.15.97:0/3664798532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64900c8b00 con 0x7f64900d09d0 2023-07-06T03:57:46.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.440+0000 7f648d7fa700 1 -- 172.21.15.97:0/3664798532 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c007440 con 0x7f64900d09d0 2023-07-06T03:57:46.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.440+0000 7f6496fff700 1 -- 172.21.15.97:0/3664798532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64900c7820 con 0x7f64900d09d0 2023-07-06T03:57:46.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.441+0000 7f6496fff700 1 -- 172.21.15.97:0/3664798532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64900c8c90 con 0x7f64900d09d0 2023-07-06T03:57:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.442+0000 7f648d7fa700 1 -- 172.21.15.97:0/3664798532 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f647c0075a0 con 0x7f64900d09d0 2023-07-06T03:57:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.443+0000 7f648d7fa700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647406ace0 0x7f647406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.443+0000 7f648d7fa700 1 -- 172.21.15.97:0/3664798532 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f647c0887e0 con 0x7f64900d09d0 2023-07-06T03:57:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.443+0000 7f6494d9b700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647406ace0 0x7f647406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.444+0000 7f6494d9b700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647406ace0 0x7f647406d190 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f64840090b0 tx=0x7f6484008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:46.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.447+0000 7f648d7fa700 1 -- 172.21.15.97:0/3664798532 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f647c053910 con 0x7f64900d09d0 2023-07-06T03:57:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.683+0000 7f6496fff700 1 -- 172.21.15.97:0/3664798532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f649004e340 con 0x7f64900d09d0 2023-07-06T03:57:46.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.683+0000 7f648d7fa700 1 -- 172.21.15.97:0/3664798532 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4680 (secure 0 0 0) 0x7f647c056f30 con 0x7f64900d09d0 2023-07-06T03:57:46.685 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:46.686 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/3313682816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3313682816},{"type":"v1","addr":"172.21.15.146:6834","nonce":3313682816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9624,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/387461585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":387461585},{"type":"v1","addr":"172.21.15.146:6835","nonce":387461585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9630,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3109027091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3109027091},{"type":"v1","addr":"172.21.15.97:6835","nonce":3109027091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:56:04.566296+0000","modified":"2023-07-06T03:56:08.096494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9633":{"gid":9633,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2797569799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2797569799},{"type":"v1","addr":"172.21.15.97:6837","nonce":2797569799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-06T03:57:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 -- 172.21.15.97:0/3664798532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647406ace0 msgr2=0x7f647406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647406ace0 0x7f647406d190 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f64840090b0 tx=0x7f6484008030 comp rx=0 tx=0).stop 2023-07-06T03:57:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 -- 172.21.15.97:0/3664798532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64900d09d0 msgr2=0x7f64900c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64900d09d0 0x7f64900c9ed0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c4b0 tx=0x7f647c00c870 comp rx=0 tx=0).stop 2023-07-06T03:57:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 -- 172.21.15.97:0/3664798532 shutdown_connections 2023-07-06T03:57:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f647406ace0 0x7f647406d190 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f649015f560 0x7f649015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f64900cde60 0x7f649014cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 --2- 172.21.15.97:0/3664798532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64900d09d0 0x7f64900c9ed0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 -- 172.21.15.97:0/3664798532 >> 172.21.15.97:0/3664798532 conn(0x7f64900720b0 msgr2=0x7f649015a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:46.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 -- 172.21.15.97:0/3664798532 shutdown_connections 2023-07-06T03:57:46.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.686+0000 7f6496fff700 1 -- 172.21.15.97:0/3664798532 wait complete. 2023-07-06T03:57:46.689 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 103 2023-07-06T03:57:46.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:57:46.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.899+0000 7f1d105ff700 1 Processor -- start 2023-07-06T03:57:46.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.899+0000 7f1d105ff700 1 -- start start 2023-07-06T03:57:46.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.901+0000 7f1d105ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1d08158ac0 0x7f1d0806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.901+0000 7f1d105ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d0815c540 0x7f1d0806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.901+0000 7f1d105ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080cab30 con 0x7f1d0806a5a0 2023-07-06T03:57:46.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.901+0000 7f1d105ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080702a0 con 0x7f1d0815c540 2023-07-06T03:57:46.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.901+0000 7f1d105ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0806aa20 con 0x7f1d08158ac0 2023-07-06T03:57:46.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.901+0000 7f1d0db9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d0815c540 0x7f1d0806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.901+0000 7f1d0db9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d0815c540 0x7f1d0806a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44130/0 (socket says 172.21.15.97:44130) 2023-07-06T03:57:46.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.901+0000 7f1d0db9a700 1 -- 172.21.15.97:0/2747918998 learned_addr learned my addr 172.21.15.97:0/2747918998 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:46.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.901+0000 7f1d0e39b700 1 --2- 172.21.15.97:0/2747918998 >> v2:172.21.15.146:3300/0 conn(0x7f1d08158ac0 0x7f1d0806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.901+0000 7f1d0db9a700 1 -- 172.21.15.97:0/2747918998 >> v2:172.21.15.146:3300/0 conn(0x7f1d08158ac0 msgr2=0x7f1d0806fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:46.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.901+0000 7f1d0db9a700 1 --2- 172.21.15.97:0/2747918998 >> v2:172.21.15.146:3300/0 conn(0x7f1d08158ac0 0x7f1d0806fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.902+0000 7f1d0db9a700 1 -- 172.21.15.97:0/2747918998 >> v1:172.21.15.146:6789/0 conn(0x7f1d0806a5a0 legacy=0x7f1d0806ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:46.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.902+0000 7f1d0db9a700 1 -- 172.21.15.97:0/2747918998 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080ca240 con 0x7f1d0815c540 2023-07-06T03:57:46.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.902+0000 7f1d0db9a700 1 --2- 172.21.15.97:0/2747918998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d0815c540 0x7f1d0806a060 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f1d080cdd30 tx=0x7f1cf8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14d57805426dcf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:46.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.902+0000 7f1d0d399700 1 -- 172.21.15.97:0/2747918998 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8005e30 con 0x7f1d0815c540 2023-07-06T03:57:46.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.902+0000 7f1d0d399700 1 -- 172.21.15.97:0/2747918998 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf8004030 con 0x7f1d0815c540 2023-07-06T03:57:46.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.902+0000 7f1d0d399700 1 -- 172.21.15.97:0/2747918998 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80043e0 con 0x7f1d0815c540 2023-07-06T03:57:46.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.903+0000 7f1d105ff700 1 -- 172.21.15.97:0/2747918998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d0815c540 msgr2=0x7f1d0806a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:46.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.903+0000 7f1d105ff700 1 --2- 172.21.15.97:0/2747918998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d0815c540 0x7f1d0806a060 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f1d080cdd30 tx=0x7f1cf8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.903+0000 7f1d105ff700 1 -- 172.21.15.97:0/2747918998 shutdown_connections 2023-07-06T03:57:46.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.903+0000 7f1d105ff700 1 --2- 172.21.15.97:0/2747918998 >> v2:172.21.15.146:3300/0 conn(0x7f1d08158ac0 0x7f1d0806fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.903+0000 7f1d105ff700 1 --2- 172.21.15.97:0/2747918998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d0815c540 0x7f1d0806a060 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.903+0000 7f1d105ff700 1 -- 172.21.15.97:0/2747918998 >> 172.21.15.97:0/2747918998 conn(0x7f1d080cc0e0 msgr2=0x7f1d080ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:46.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.903+0000 7f1d105ff700 1 -- 172.21.15.97:0/2747918998 shutdown_connections 2023-07-06T03:57:46.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.903+0000 7f1d105ff700 1 -- 172.21.15.97:0/2747918998 wait complete. 2023-07-06T03:57:46.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.905+0000 7f1d105ff700 1 Processor -- start 2023-07-06T03:57:46.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.905+0000 7f1d105ff700 1 -- start start 2023-07-06T03:57:46.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.906+0000 7f1d105ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d0806a5a0 0x7f1d080a8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.906+0000 7f1d105ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08158ac0 0x7f1d080af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.906+0000 7f1d105ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d0815c540 0x7f1d080a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.906+0000 7f1d105ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d08158680 con 0x7f1d0815c540 2023-07-06T03:57:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.906+0000 7f1d105ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080afa10 con 0x7f1d08158ac0 2023-07-06T03:57:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.906+0000 7f1d105ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080a8a50 con 0x7f1d0806a5a0 2023-07-06T03:57:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.907+0000 7f1d0eb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d0815c540 0x7f1d080a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.907+0000 7f1d0eb9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d0815c540 0x7f1d080a9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44134/0 (socket says 172.21.15.97:44134) 2023-07-06T03:57:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.907+0000 7f1d0eb9c700 1 -- 172.21.15.97:0/3350945535 learned_addr learned my addr 172.21.15.97:0/3350945535 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.907+0000 7f1d0e39b700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d0806a5a0 0x7f1d080a8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.907+0000 7f1d0db9a700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08158ac0 0x7f1d080af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.907+0000 7f1d0eb9c700 1 -- 172.21.15.97:0/3350945535 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d0806a5a0 msgr2=0x7f1d080a8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.907+0000 7f1d0eb9c700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d0806a5a0 0x7f1d080a8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.907+0000 7f1d0eb9c700 1 -- 172.21.15.97:0/3350945535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08158ac0 msgr2=0x7f1d080af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.907+0000 7f1d0eb9c700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08158ac0 0x7f1d080af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.907+0000 7f1d0eb9c700 1 -- 172.21.15.97:0/3350945535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf8009500 con 0x7f1d0815c540 2023-07-06T03:57:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.908+0000 7f1d0eb9c700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d0815c540 0x7f1d080a9af0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000cb50 tx=0x7f1d0000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:46.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.908+0000 7f1cfeffd700 1 -- 172.21.15.97:0/3350945535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00003c00 con 0x7f1d0815c540 2023-07-06T03:57:46.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.908+0000 7f1cfeffd700 1 -- 172.21.15.97:0/3350945535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d00003d60 con 0x7f1d0815c540 2023-07-06T03:57:46.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.908+0000 7f1d105ff700 1 -- 172.21.15.97:0/3350945535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d080a8d30 con 0x7f1d0815c540 2023-07-06T03:57:46.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.908+0000 7f1cfeffd700 1 -- 172.21.15.97:0/3350945535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000aa20 con 0x7f1d0815c540 2023-07-06T03:57:46.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.908+0000 7f1d105ff700 1 -- 172.21.15.97:0/3350945535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d080a92f0 con 0x7f1d0815c540 2023-07-06T03:57:46.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.909+0000 7f1d105ff700 1 -- 172.21.15.97:0/3350945535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0806bb20 con 0x7f1d0815c540 2023-07-06T03:57:46.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.910+0000 7f1cfeffd700 1 -- 172.21.15.97:0/3350945535 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d0000ab80 con 0x7f1d0815c540 2023-07-06T03:57:46.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.911+0000 7f1cfeffd700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cec06ace0 0x7f1cec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:46.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.911+0000 7f1cfeffd700 1 -- 172.21.15.97:0/3350945535 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f1d000894c0 con 0x7f1d0815c540 2023-07-06T03:57:46.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.911+0000 7f1d0e39b700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cec06ace0 0x7f1cec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:46.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.912+0000 7f1d0e39b700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cec06ace0 0x7f1cec06d190 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1d080c2430 tx=0x7f1d04009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:46.915+0000 7f1cfeffd700 1 -- 172.21.15.97:0/3350945535 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1d00054540 con 0x7f1d0815c540 2023-07-06T03:57:47.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.114+0000 7f1d105ff700 1 -- 172.21.15.97:0/3350945535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d08067010 con 0x7f1d0815c540 2023-07-06T03:57:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.116+0000 7f1cfeffd700 1 -- 172.21.15.97:0/3350945535 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18436 (secure 0 0 0) 0x7f1d00057b60 con 0x7f1d0815c540 2023-07-06T03:57:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:47.118 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":138,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:56:54.911710+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:56:02.531370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-06T03:56:03.548148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.7","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:57:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 -- 172.21.15.97:0/3350945535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cec06ace0 msgr2=0x7f1cec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cec06ace0 0x7f1cec06d190 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1d080c2430 tx=0x7f1d04009210 comp rx=0 tx=0).stop 2023-07-06T03:57:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 -- 172.21.15.97:0/3350945535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d0815c540 msgr2=0x7f1d080a9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d0815c540 0x7f1d080a9af0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000cb50 tx=0x7f1d0000ce60 comp rx=0 tx=0).stop 2023-07-06T03:57:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 -- 172.21.15.97:0/3350945535 shutdown_connections 2023-07-06T03:57:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1cec06ace0 0x7f1cec06d190 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d0815c540 0x7f1d080a9af0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1d0806a5a0 0x7f1d080a8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 --2- 172.21.15.97:0/3350945535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08158ac0 0x7f1d080af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 -- 172.21.15.97:0/3350945535 >> 172.21.15.97:0/3350945535 conn(0x7f1d080cc0e0 msgr2=0x7f1d08159080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 -- 172.21.15.97:0/3350945535 shutdown_connections 2023-07-06T03:57:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.118+0000 7f1d105ff700 1 -- 172.21.15.97:0/3350945535 wait complete. 2023-07-06T03:57:47.138 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T03:44:56.940879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T03:45:03.582725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T03:56:02.531370+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T03:56:03.548148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T03:57:47.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T03:57:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.334+0000 7fd83bbff700 1 Processor -- start 2023-07-06T03:57:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.334+0000 7fd83bbff700 1 -- start start 2023-07-06T03:57:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.335+0000 7fd83bbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd8340d1360 0x7fd83406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.335+0000 7fd83bbff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8340d1950 con 0x7fd8340cd490 2023-07-06T03:57:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.335+0000 7fd83bbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8340d2670 con 0x7fd8340cf7e0 2023-07-06T03:57:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.335+0000 7fd83bbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8340d2460 con 0x7fd8340d1360 2023-07-06T03:57:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.335+0000 7fd83919a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd8340d1360 0x7fd83406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.335+0000 7fd83a19c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd8340cd490 0x7fd8340b5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60258/0 (socket says 172.21.15.97:60258) 2023-07-06T03:57:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.335+0000 7fd83a19c700 1 -- 172.21.15.97:0/4210810948 learned_addr learned my addr 172.21.15.97:0/4210810948 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.336+0000 7fd83919a700 1 -- 172.21.15.97:0/4210810948 >> v1:172.21.15.146:6789/0 conn(0x7fd8340cf7e0 legacy=0x7fd8340cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.336+0000 7fd83919a700 1 -- 172.21.15.97:0/4210810948 >> v1:172.21.15.97:6789/0 conn(0x7fd8340cd490 legacy=0x7fd8340b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.336+0000 7fd838999700 1 -- 172.21.15.97:0/4210810948 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109302795 0 0) 0x7fd8340d1950 con 0x7fd8340cd490 2023-07-06T03:57:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.336+0000 7fd83919a700 1 -- 172.21.15.97:0/4210810948 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8340d2950 con 0x7fd8340d1360 2023-07-06T03:57:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.336+0000 7fd83919a700 1 --2- 172.21.15.97:0/4210810948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8340d1360 0x7fd83406b380 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fd824009840 tx=0x7fd824009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296f0e684e7a0fdc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.336+0000 7fd838999700 1 -- 172.21.15.97:0/4210810948 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824011830 con 0x7fd8340d1360 2023-07-06T03:57:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.336+0000 7fd838999700 1 -- 172.21.15.97:0/4210810948 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd824011990 con 0x7fd8340d1360 2023-07-06T03:57:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.336+0000 7fd838999700 1 -- 172.21.15.97:0/4210810948 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824011d20 con 0x7fd8340d1360 2023-07-06T03:57:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.337+0000 7fd83bbff700 1 -- 172.21.15.97:0/4210810948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8340d1360 msgr2=0x7fd83406b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.337+0000 7fd83bbff700 1 --2- 172.21.15.97:0/4210810948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8340d1360 0x7fd83406b380 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fd824009840 tx=0x7fd824009b50 comp rx=0 tx=0).stop 2023-07-06T03:57:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.337+0000 7fd83bbff700 1 -- 172.21.15.97:0/4210810948 shutdown_connections 2023-07-06T03:57:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.337+0000 7fd83bbff700 1 --2- 172.21.15.97:0/4210810948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8340d1360 0x7fd83406b380 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.337+0000 7fd83bbff700 1 -- 172.21.15.97:0/4210810948 >> 172.21.15.97:0/4210810948 conn(0x7fd83406cf30 msgr2=0x7fd83406a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.337+0000 7fd83bbff700 1 -- 172.21.15.97:0/4210810948 shutdown_connections 2023-07-06T03:57:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.337+0000 7fd83bbff700 1 -- 172.21.15.97:0/4210810948 wait complete. 2023-07-06T03:57:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.339+0000 7fd83bbff700 1 Processor -- start 2023-07-06T03:57:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.339+0000 7fd83bbff700 1 -- start start 2023-07-06T03:57:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.340+0000 7fd83bbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8340cd490 0x7fd8341f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.340+0000 7fd83bbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8340cf7e0 0x7fd8340cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.340+0000 7fd83bbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8340d1360 0x7fd8340caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.340+0000 7fd83bbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8340b3b00 con 0x7fd8340cd490 2023-07-06T03:57:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.340+0000 7fd83bbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8340b3de0 con 0x7fd8340cf7e0 2023-07-06T03:57:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.340+0000 7fd83bbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8340b40c0 con 0x7fd8340d1360 2023-07-06T03:57:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.340+0000 7fd83999b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8340cd490 0x7fd8341f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.340+0000 7fd83999b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8340cd490 0x7fd8341f2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44146/0 (socket says 172.21.15.97:44146) 2023-07-06T03:57:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.340+0000 7fd83999b700 1 -- 172.21.15.97:0/3242399843 learned_addr learned my addr 172.21.15.97:0/3242399843 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.341+0000 7fd83999b700 1 -- 172.21.15.97:0/3242399843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8340d1360 msgr2=0x7fd8340caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.341+0000 7fd83999b700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8340d1360 0x7fd8340caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.341+0000 7fd83999b700 1 -- 172.21.15.97:0/3242399843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8340cf7e0 msgr2=0x7fd8340cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.341+0000 7fd83999b700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8340cf7e0 0x7fd8340cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.341+0000 7fd83999b700 1 -- 172.21.15.97:0/3242399843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8240094a0 con 0x7fd8340cd490 2023-07-06T03:57:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.341+0000 7fd83999b700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8340cd490 0x7fd8341f2420 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fd83000c4e0 tx=0x7fd83000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.341+0000 7fd82a7fc700 1 -- 172.21.15.97:0/3242399843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8300052d0 con 0x7fd8340cd490 2023-07-06T03:57:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.341+0000 7fd82a7fc700 1 -- 172.21.15.97:0/3242399843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd830005430 con 0x7fd8340cd490 2023-07-06T03:57:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.341+0000 7fd83bbff700 1 -- 172.21.15.97:0/3242399843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8340b43a0 con 0x7fd8340cd490 2023-07-06T03:57:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.341+0000 7fd82a7fc700 1 -- 172.21.15.97:0/3242399843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8300038c0 con 0x7fd8340cd490 2023-07-06T03:57:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.341+0000 7fd83bbff700 1 -- 172.21.15.97:0/3242399843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8341f2960 con 0x7fd8340cd490 2023-07-06T03:57:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.342+0000 7fd83bbff700 1 -- 172.21.15.97:0/3242399843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8340b48c0 con 0x7fd8340cd490 2023-07-06T03:57:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.343+0000 7fd82a7fc700 1 -- 172.21.15.97:0/3242399843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd830003a20 con 0x7fd8340cd490 2023-07-06T03:57:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.344+0000 7fd82a7fc700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd81806ace0 0x7fd81806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.344+0000 7fd82a7fc700 1 -- 172.21.15.97:0/3242399843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd830088990 con 0x7fd8340cd490 2023-07-06T03:57:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.344+0000 7fd83919a700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd81806ace0 0x7fd81806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.345+0000 7fd83919a700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd81806ace0 0x7fd81806d190 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd8340b8380 tx=0x7fd824010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.348+0000 7fd82a7fc700 1 -- 172.21.15.97:0/3242399843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd8340b48c0 con 0x7fd8340cd490 2023-07-06T03:57:47.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.543+0000 7fd83bbff700 1 -- 172.21.15.97:0/3242399843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8340b48c0 con 0x7fd8340cd490 2023-07-06T03:57:47.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.543+0000 7fd82a7fc700 1 -- 172.21.15.97:0/3242399843 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd830017020 con 0x7fd8340cd490 2023-07-06T03:57:47.545 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-07-06T03:57:47.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 -- 172.21.15.97:0/3242399843 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd81806ace0 msgr2=0x7fd81806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd81806ace0 0x7fd81806d190 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd8340b8380 tx=0x7fd824010000 comp rx=0 tx=0).stop 2023-07-06T03:57:47.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 -- 172.21.15.97:0/3242399843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8340cd490 msgr2=0x7fd8341f2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8340cd490 0x7fd8341f2420 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fd83000c4e0 tx=0x7fd83000c8a0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 -- 172.21.15.97:0/3242399843 shutdown_connections 2023-07-06T03:57:47.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fd81806ace0 0x7fd81806d190 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd8340d1360 0x7fd8340caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8340cd490 0x7fd8341f2420 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 --2- 172.21.15.97:0/3242399843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8340cf7e0 0x7fd8340cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 -- 172.21.15.97:0/3242399843 >> 172.21.15.97:0/3242399843 conn(0x7fd83406cf30 msgr2=0x7fd8340b79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:47.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 -- 172.21.15.97:0/3242399843 shutdown_connections 2023-07-06T03:57:47.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.546+0000 7fd83bbff700 1 -- 172.21.15.97:0/3242399843 wait complete. 2023-07-06T03:57:47.565 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T03:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.761+0000 7f84445ff700 1 Processor -- start 2023-07-06T03:57:47.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.761+0000 7f84445ff700 1 -- start start 2023-07-06T03:57:47.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.762+0000 7f84445ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f843c0cf7a0 0x7f843c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.762+0000 7f84445ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f843c0cf0e0 0x7f843c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.762+0000 7f84445ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f843c0d2d70 0x7f843c0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.762+0000 7f84445ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c0d0a50 con 0x7f843c0cf7a0 2023-07-06T03:57:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.762+0000 7f84445ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c0d0d30 con 0x7f843c0cf0e0 2023-07-06T03:57:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.762+0000 7f84445ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0d1010 con 0x7f843c0d2d70 2023-07-06T03:57:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.763+0000 7f844239b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f843c0cf7a0 0x7f843c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:47.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.763+0000 7f844239b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f843c0cf7a0 0x7f843c0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44162/0 (socket says 172.21.15.97:44162) 2023-07-06T03:57:47.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.763+0000 7f844239b700 1 -- 172.21.15.97:0/795889104 learned_addr learned my addr 172.21.15.97:0/795889104 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:47.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.763+0000 7f8441b9a700 1 --2- 172.21.15.97:0/795889104 >> v2:172.21.15.146:3300/0 conn(0x7f843c0cf0e0 0x7f843c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:47.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.763+0000 7f8442b9c700 1 --2- 172.21.15.97:0/795889104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f843c0d2d70 0x7f843c0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:47.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.763+0000 7f844239b700 1 -- 172.21.15.97:0/795889104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f843c0d2d70 msgr2=0x7f843c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:47.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.763+0000 7f844239b700 1 --2- 172.21.15.97:0/795889104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f843c0d2d70 0x7f843c0d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.763+0000 7f844239b700 1 -- 172.21.15.97:0/795889104 >> v2:172.21.15.146:3300/0 conn(0x7f843c0cf0e0 msgr2=0x7f843c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:47.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.763+0000 7f844239b700 1 --2- 172.21.15.97:0/795889104 >> v2:172.21.15.146:3300/0 conn(0x7f843c0cf0e0 0x7f843c0cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.763+0000 7f844239b700 1 -- 172.21.15.97:0/795889104 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c0d12f0 con 0x7f843c0cf7a0 2023-07-06T03:57:47.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.763+0000 7f844239b700 1 --2- 172.21.15.97:0/795889104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f843c0cf7a0 0x7f843c0cfb70 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f843c06bc10 tx=0x7f8438009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c873ea424264db2e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:47.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.764+0000 7f8441399700 1 -- 172.21.15.97:0/795889104 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843800b870 con 0x7f843c0cf7a0 2023-07-06T03:57:47.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.764+0000 7f8441399700 1 -- 172.21.15.97:0/795889104 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843800b9d0 con 0x7f843c0cf7a0 2023-07-06T03:57:47.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.764+0000 7f8441399700 1 -- 172.21.15.97:0/795889104 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843800bd60 con 0x7f843c0cf7a0 2023-07-06T03:57:47.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.764+0000 7f84445ff700 1 -- 172.21.15.97:0/795889104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f843c0cf7a0 msgr2=0x7f843c0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.764+0000 7f84445ff700 1 --2- 172.21.15.97:0/795889104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f843c0cf7a0 0x7f843c0cfb70 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f843c06bc10 tx=0x7f8438009bd0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.764+0000 7f84445ff700 1 -- 172.21.15.97:0/795889104 shutdown_connections 2023-07-06T03:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.764+0000 7f84445ff700 1 --2- 172.21.15.97:0/795889104 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f843c0d2d70 0x7f843c0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.764+0000 7f84445ff700 1 --2- 172.21.15.97:0/795889104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f843c0cf7a0 0x7f843c0cfb70 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.764+0000 7f84445ff700 1 --2- 172.21.15.97:0/795889104 >> v2:172.21.15.146:3300/0 conn(0x7f843c0cf0e0 0x7f843c0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.764+0000 7f84445ff700 1 -- 172.21.15.97:0/795889104 >> 172.21.15.97:0/795889104 conn(0x7f843c06c1f0 msgr2=0x7f843c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.764+0000 7f84445ff700 1 -- 172.21.15.97:0/795889104 shutdown_connections 2023-07-06T03:57:47.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.765+0000 7f84445ff700 1 -- 172.21.15.97:0/795889104 wait complete. 2023-07-06T03:57:47.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.766+0000 7f84445ff700 1 Processor -- start 2023-07-06T03:57:47.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.766+0000 7f84445ff700 1 -- start start 2023-07-06T03:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.767+0000 7f84445ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f843c0cf0e0 0x7f843c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.767+0000 7f84445ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f843c0cf7a0 0x7f843c1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.767+0000 7f84445ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f843c0d2d70 0x7f843c0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.767+0000 7f84445ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c1568d0 con 0x7f843c0d2d70 2023-07-06T03:57:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.767+0000 7f84445ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c1565f0 con 0x7f843c0cf0e0 2023-07-06T03:57:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.767+0000 7f84445ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c1517e0 con 0x7f843c0cf7a0 2023-07-06T03:57:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8442b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f843c0d2d70 0x7f843c0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8442b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f843c0d2d70 0x7f843c0c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44170/0 (socket says 172.21.15.97:44170) 2023-07-06T03:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8442b9c700 1 -- 172.21.15.97:0/1128194450 learned_addr learned my addr 172.21.15.97:0/1128194450 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f844239b700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f843c0cf0e0 0x7f843c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8441b9a700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f843c0cf7a0 0x7f843c1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8442b9c700 1 -- 172.21.15.97:0/1128194450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f843c0cf7a0 msgr2=0x7f843c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8442b9c700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f843c0cf7a0 0x7f843c1534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8442b9c700 1 -- 172.21.15.97:0/1128194450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f843c0cf0e0 msgr2=0x7f843c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8442b9c700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f843c0cf0e0 0x7f843c156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8442b9c700 1 -- 172.21.15.97:0/1128194450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8438009550 con 0x7f843c0d2d70 2023-07-06T03:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8442b9c700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f843c0d2d70 0x7f843c0c8a40 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f8434007e60 tx=0x7f843400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8432ffd700 1 -- 172.21.15.97:0/1128194450 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400ede0 con 0x7f843c0d2d70 2023-07-06T03:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8432ffd700 1 -- 172.21.15.97:0/1128194450 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8434014420 con 0x7f843c0d2d70 2023-07-06T03:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f84445ff700 1 -- 172.21.15.97:0/1128194450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c151ac0 con 0x7f843c0d2d70 2023-07-06T03:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.768+0000 7f8432ffd700 1 -- 172.21.15.97:0/1128194450 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340147d0 con 0x7f843c0d2d70 2023-07-06T03:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.769+0000 7f84445ff700 1 -- 172.21.15.97:0/1128194450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c146730 con 0x7f843c0d2d70 2023-07-06T03:57:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.769+0000 7f84445ff700 1 -- 172.21.15.97:0/1128194450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c151640 con 0x7f843c0d2d70 2023-07-06T03:57:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.770+0000 7f8432ffd700 1 -- 172.21.15.97:0/1128194450 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8434014930 con 0x7f843c0d2d70 2023-07-06T03:57:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.771+0000 7f8432ffd700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f842006ace0 0x7f842006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.772+0000 7f8432ffd700 1 -- 172.21.15.97:0/1128194450 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8434089c90 con 0x7f843c0d2d70 2023-07-06T03:57:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.772+0000 7f844239b700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f842006ace0 0x7f842006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.772+0000 7f844239b700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f842006ace0 0x7f842006d190 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8438000c00 tx=0x7f8438008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.776+0000 7f8432ffd700 1 -- 172.21.15.97:0/1128194450 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f843c151640 con 0x7f843c0d2d70 2023-07-06T03:57:47.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.970+0000 7f84445ff700 1 -- 172.21.15.97:0/1128194450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f843c151640 con 0x7f843c0d2d70 2023-07-06T03:57:47.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.971+0000 7f8432ffd700 1 -- 172.21.15.97:0/1128194450 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7f843c151640 con 0x7f843c0d2d70 2023-07-06T03:57:47.973 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 8 2023-07-06T03:57:47.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 -- 172.21.15.97:0/1128194450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f842006ace0 msgr2=0x7f842006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f842006ace0 0x7f842006d190 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8438000c00 tx=0x7f8438008030 comp rx=0 tx=0).stop 2023-07-06T03:57:47.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 -- 172.21.15.97:0/1128194450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f843c0d2d70 msgr2=0x7f843c0c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:47.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f843c0d2d70 0x7f843c0c8a40 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f8434007e60 tx=0x7f843400e3c0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 -- 172.21.15.97:0/1128194450 shutdown_connections 2023-07-06T03:57:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f842006ace0 0x7f842006d190 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f843c0d2d70 0x7f843c0c8a40 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f843c0cf0e0 0x7f843c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 --2- 172.21.15.97:0/1128194450 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f843c0cf7a0 0x7f843c1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 -- 172.21.15.97:0/1128194450 >> 172.21.15.97:0/1128194450 conn(0x7f843c06c1f0 msgr2=0x7f843c0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 -- 172.21.15.97:0/1128194450 shutdown_connections 2023-07-06T03:57:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:47.973+0000 7f84445ff700 1 -- 172.21.15.97:0/1128194450 wait complete. 2023-07-06T03:57:47.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T03:57:48.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.190+0000 7f57649ff700 1 Processor -- start 2023-07-06T03:57:48.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.190+0000 7f57649ff700 1 -- start start 2023-07-06T03:57:48.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.191+0000 7f57649ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57600cf7e0 0x7f57600d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:48.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.191+0000 7f57649ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576006b7f0 con 0x7f576006b380 2023-07-06T03:57:48.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.191+0000 7f57649ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57600d1950 con 0x7f57600cd490 2023-07-06T03:57:48.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.191+0000 7f57649ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57600d2460 con 0x7f57600cf7e0 2023-07-06T03:57:48.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.192+0000 7f575dd9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f576006b380 0x7f57600cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60270/0 (socket says 172.21.15.97:60270) 2023-07-06T03:57:48.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.192+0000 7f575dd9b700 1 -- 172.21.15.97:0/2777912837 learned_addr learned my addr 172.21.15.97:0/2777912837 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:48.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.192+0000 7f575e59c700 1 --2- 172.21.15.97:0/2777912837 >> v2:172.21.15.97:3300/0 conn(0x7f57600cf7e0 0x7f57600d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:48.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.192+0000 7f575e59c700 1 -- 172.21.15.97:0/2777912837 >> v1:172.21.15.146:6789/0 conn(0x7f57600cd490 legacy=0x7f57600b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.192+0000 7f575e59c700 1 -- 172.21.15.97:0/2777912837 >> v1:172.21.15.97:6789/0 conn(0x7f576006b380 legacy=0x7f57600cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.192+0000 7f575e59c700 1 -- 172.21.15.97:0/2777912837 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600d2670 con 0x7f57600cf7e0 2023-07-06T03:57:48.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.192+0000 7f575e59c700 1 --2- 172.21.15.97:0/2777912837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57600cf7e0 0x7f57600d1360 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f5750009840 tx=0x7f5750009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32133b6451321c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:48.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.192+0000 7f575d59a700 1 -- 172.21.15.97:0/2777912837 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750011830 con 0x7f57600cf7e0 2023-07-06T03:57:48.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.193+0000 7f575d59a700 1 -- 172.21.15.97:0/2777912837 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5750011990 con 0x7f57600cf7e0 2023-07-06T03:57:48.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.193+0000 7f575d59a700 1 -- 172.21.15.97:0/2777912837 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750011d20 con 0x7f57600cf7e0 2023-07-06T03:57:48.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.193+0000 7f57649ff700 1 -- 172.21.15.97:0/2777912837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57600cf7e0 msgr2=0x7f57600d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.193+0000 7f57649ff700 1 --2- 172.21.15.97:0/2777912837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57600cf7e0 0x7f57600d1360 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f5750009840 tx=0x7f5750009b50 comp rx=0 tx=0).stop 2023-07-06T03:57:48.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.193+0000 7f57649ff700 1 -- 172.21.15.97:0/2777912837 shutdown_connections 2023-07-06T03:57:48.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.193+0000 7f57649ff700 1 --2- 172.21.15.97:0/2777912837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57600cf7e0 0x7f57600d1360 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.193+0000 7f57649ff700 1 -- 172.21.15.97:0/2777912837 >> 172.21.15.97:0/2777912837 conn(0x7f576006cf30 msgr2=0x7f576006a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:48.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.193+0000 7f57649ff700 1 -- 172.21.15.97:0/2777912837 shutdown_connections 2023-07-06T03:57:48.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.193+0000 7f57649ff700 1 -- 172.21.15.97:0/2777912837 wait complete. 2023-07-06T03:57:48.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.195+0000 7f57649ff700 1 Processor -- start 2023-07-06T03:57:48.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.195+0000 7f57649ff700 1 -- start start 2023-07-06T03:57:48.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.196+0000 7f57649ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f576006b380 0x7f576014e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:48.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.196+0000 7f57649ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57600cd490 0x7f57600cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:48.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.196+0000 7f57649ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57600cf7e0 0x7f57600b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:48.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.196+0000 7f57649ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57600b28c0 con 0x7f57600cd490 2023-07-06T03:57:48.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.196+0000 7f57649ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57601522c0 con 0x7f57600cf7e0 2023-07-06T03:57:48.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.196+0000 7f57649ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57600cb100 con 0x7f576006b380 2023-07-06T03:57:48.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.196+0000 7f575dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57600cd490 0x7f57600cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:48.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.196+0000 7f575dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57600cd490 0x7f57600cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44190/0 (socket says 172.21.15.97:44190) 2023-07-06T03:57:48.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.196+0000 7f575dd9b700 1 -- 172.21.15.97:0/1497809485 learned_addr learned my addr 172.21.15.97:0/1497809485 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:48.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.196+0000 7f575ed9d700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57600cf7e0 0x7f57600b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:48.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.196+0000 7f575e59c700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f576006b380 0x7f576014e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:48.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f575dd9b700 1 -- 172.21.15.97:0/1497809485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f576006b380 msgr2=0x7f576014e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f575dd9b700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f576006b380 0x7f576014e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f575dd9b700 1 -- 172.21.15.97:0/1497809485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57600cf7e0 msgr2=0x7f57600b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f575dd9b700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57600cf7e0 0x7f57600b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f575dd9b700 1 -- 172.21.15.97:0/1497809485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57500094a0 con 0x7f57600cd490 2023-07-06T03:57:48.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f575ed9d700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57600cf7e0 0x7f57600b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:48.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f575dd9b700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57600cd490 0x7f57600cc600 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f575400c550 tx=0x7f575400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:48.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f574effd700 1 -- 172.21.15.97:0/1497809485 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754005d70 con 0x7f57600cd490 2023-07-06T03:57:48.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f574effd700 1 -- 172.21.15.97:0/1497809485 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57540070d0 con 0x7f57600cd490 2023-07-06T03:57:48.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f57649ff700 1 -- 172.21.15.97:0/1497809485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57600cb3e0 con 0x7f57600cd490 2023-07-06T03:57:48.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f574effd700 1 -- 172.21.15.97:0/1497809485 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754007440 con 0x7f57600cd490 2023-07-06T03:57:48.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.197+0000 7f57649ff700 1 -- 172.21.15.97:0/1497809485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57600cb9a0 con 0x7f57600cd490 2023-07-06T03:57:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.198+0000 7f57649ff700 1 -- 172.21.15.97:0/1497809485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57600b2a00 con 0x7f57600cd490 2023-07-06T03:57:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.199+0000 7f574effd700 1 -- 172.21.15.97:0/1497809485 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57540075a0 con 0x7f57600cd490 2023-07-06T03:57:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.200+0000 7f574effd700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 0x7f574006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.200+0000 7f574effd700 1 -- 172.21.15.97:0/1497809485 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f57540886d0 con 0x7f57600cd490 2023-07-06T03:57:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.200+0000 7f575e59c700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 0x7f574006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.201+0000 7f575e59c700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 0x7f574006d190 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f57500090b0 tx=0x7f575001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:48.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.204+0000 7f574effd700 1 -- 172.21.15.97:0/1497809485 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f57600b2a00 con 0x7f57600cd490 2023-07-06T03:57:48.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.397+0000 7f57649ff700 1 -- 172.21.15.97:0/1497809485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f576006b7f0 con 0x7f57600cd490 2023-07-06T03:57:48.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.398+0000 7f574effd700 1 -- 172.21.15.97:0/1497809485 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+11 (secure 0 0 0) 0x7f576006b7f0 con 0x7f57600cd490 2023-07-06T03:57:48.400 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-07-06T03:57:48.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 -- 172.21.15.97:0/1497809485 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 msgr2=0x7f574006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 0x7f574006d190 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f57500090b0 tx=0x7f575001c040 comp rx=0 tx=0).stop 2023-07-06T03:57:48.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 -- 172.21.15.97:0/1497809485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57600cd490 msgr2=0x7f57600cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57600cd490 0x7f57600cc600 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f575400c550 tx=0x7f575400c910 comp rx=0 tx=0).stop 2023-07-06T03:57:48.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 -- 172.21.15.97:0/1497809485 shutdown_connections 2023-07-06T03:57:48.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 0x7f574006d190 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57600cf7e0 0x7f57600b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f576006b380 0x7f576014e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 --2- 172.21.15.97:0/1497809485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57600cd490 0x7f57600cc600 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 -- 172.21.15.97:0/1497809485 >> 172.21.15.97:0/1497809485 conn(0x7f576006cf30 msgr2=0x7f576015d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:48.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 -- 172.21.15.97:0/1497809485 shutdown_connections 2023-07-06T03:57:48.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.400+0000 7f57649ff700 1 -- 172.21.15.97:0/1497809485 wait complete. 2023-07-06T03:57:48.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T03:57:48.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.615+0000 7f58347ff700 1 Processor -- start 2023-07-06T03:57:48.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.615+0000 7f58347ff700 1 -- start start 2023-07-06T03:57:48.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.616+0000 7f58347ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f582c0cf7e0 0x7f582c0d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:48.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.616+0000 7f58347ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c06b380 0x7f582c0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:48.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.616+0000 7f58347ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582c06b7f0 con 0x7f582c0cd490 2023-07-06T03:57:48.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.616+0000 7f58347ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582c0d2460 con 0x7f582c0cf7e0 2023-07-06T03:57:48.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.616+0000 7f58347ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0d00f0 con 0x7f582c06b380 2023-07-06T03:57:48.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.617+0000 7f583259b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f582c0cf7e0 0x7f582c0d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:48.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.617+0000 7f5831d9a700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c06b380 0x7f582c0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:48.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.617+0000 7f583259b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f582c0cf7e0 0x7f582c0d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:42804/0 (socket says 172.21.15.97:42804) 2023-07-06T03:57:48.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.617+0000 7f583259b700 1 -- 172.21.15.97:0/633617728 learned_addr learned my addr 172.21.15.97:0/633617728 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:48.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.617+0000 7f5832d9c700 1 --1- 172.21.15.97:0/633617728 >> v1:172.21.15.146:6789/0 conn(0x7f582c0cd490 0x7f582c0b5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:46576/0 (socket says 172.21.15.97:46576) 2023-07-06T03:57:48.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.617+0000 7f583259b700 1 -- 172.21.15.97:0/633617728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c06b380 msgr2=0x7f582c0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:48.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.617+0000 7f583259b700 1 --2- 172.21.15.97:0/633617728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c06b380 0x7f582c0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.617+0000 7f583259b700 1 -- 172.21.15.97:0/633617728 >> v1:172.21.15.146:6789/0 conn(0x7f582c0cd490 legacy=0x7f582c0b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.617+0000 7f583259b700 1 -- 172.21.15.97:0/633617728 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c0d1950 con 0x7f582c0cf7e0 2023-07-06T03:57:48.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.617+0000 7f5831d9a700 1 --2- 172.21.15.97:0/633617728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c06b380 0x7f582c0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:57:48.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.617+0000 7f583259b700 1 --2- 172.21.15.97:0/633617728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c0cf7e0 0x7f582c0d1360 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f58280098f0 tx=0x7f5828009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3586774cc7eb9d76 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:48.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.618+0000 7f5831599700 1 -- 172.21.15.97:0/633617728 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828011830 con 0x7f582c0cf7e0 2023-07-06T03:57:48.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.618+0000 7f5831599700 1 -- 172.21.15.97:0/633617728 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5828011990 con 0x7f582c0cf7e0 2023-07-06T03:57:48.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.618+0000 7f5831599700 1 -- 172.21.15.97:0/633617728 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828011d20 con 0x7f582c0cf7e0 2023-07-06T03:57:48.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.618+0000 7f58347ff700 1 -- 172.21.15.97:0/633617728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c0cf7e0 msgr2=0x7f582c0d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.618+0000 7f58347ff700 1 --2- 172.21.15.97:0/633617728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c0cf7e0 0x7f582c0d1360 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f58280098f0 tx=0x7f5828009c00 comp rx=0 tx=0).stop 2023-07-06T03:57:48.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.618+0000 7f58347ff700 1 -- 172.21.15.97:0/633617728 shutdown_connections 2023-07-06T03:57:48.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.618+0000 7f58347ff700 1 --2- 172.21.15.97:0/633617728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c0cf7e0 0x7f582c0d1360 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.618+0000 7f58347ff700 1 --2- 172.21.15.97:0/633617728 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c06b380 0x7f582c0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.618+0000 7f58347ff700 1 -- 172.21.15.97:0/633617728 >> 172.21.15.97:0/633617728 conn(0x7f582c06cf30 msgr2=0x7f582c06a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:48.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.618+0000 7f58347ff700 1 -- 172.21.15.97:0/633617728 shutdown_connections 2023-07-06T03:57:48.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.618+0000 7f58347ff700 1 -- 172.21.15.97:0/633617728 wait complete. 2023-07-06T03:57:48.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.620+0000 7f58347ff700 1 Processor -- start 2023-07-06T03:57:48.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.620+0000 7f58347ff700 1 -- start start 2023-07-06T03:57:48.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.621+0000 7f58347ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c06b380 0x7f582c0b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:48.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.621+0000 7f58347ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c0cd490 0x7f582c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:48.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.621+0000 7f58347ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c0cf7e0 0x7f582c159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:48.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.621+0000 7f58347ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c15dbd0 con 0x7f582c0cd490 2023-07-06T03:57:48.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.621+0000 7f58347ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0cd060 con 0x7f582c06b380 2023-07-06T03:57:48.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.621+0000 7f58347ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0b4b20 con 0x7f582c0cf7e0 2023-07-06T03:57:48.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.621+0000 7f5831d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c0cd490 0x7f582c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:48.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.621+0000 7f5831d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c0cd490 0x7f582c0ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44202/0 (socket says 172.21.15.97:44202) 2023-07-06T03:57:48.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.621+0000 7f5831d9a700 1 -- 172.21.15.97:0/1484030899 learned_addr learned my addr 172.21.15.97:0/1484030899 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:48.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f583259b700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c06b380 0x7f582c0b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:48.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f5832d9c700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c0cf7e0 0x7f582c159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:48.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f5831d9a700 1 -- 172.21.15.97:0/1484030899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c0cf7e0 msgr2=0x7f582c159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f5831d9a700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c0cf7e0 0x7f582c159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f5831d9a700 1 -- 172.21.15.97:0/1484030899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c06b380 msgr2=0x7f582c0b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f5831d9a700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c06b380 0x7f582c0b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f5831d9a700 1 -- 172.21.15.97:0/1484030899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0094a0 con 0x7f582c0cd490 2023-07-06T03:57:48.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f5831d9a700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c0cd490 0x7f582c0ccb20 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f581c00ebe0 tx=0x7f581c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:48.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f5822ffd700 1 -- 172.21.15.97:0/1484030899 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c009800 con 0x7f582c0cd490 2023-07-06T03:57:48.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f5822ffd700 1 -- 172.21.15.97:0/1484030899 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581c00c910 con 0x7f582c0cd490 2023-07-06T03:57:48.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f58347ff700 1 -- 172.21.15.97:0/1484030899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5828009550 con 0x7f582c0cd490 2023-07-06T03:57:48.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f5822ffd700 1 -- 172.21.15.97:0/1484030899 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c00cc80 con 0x7f582c0cd490 2023-07-06T03:57:48.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.622+0000 7f58347ff700 1 -- 172.21.15.97:0/1484030899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c0b50e0 con 0x7f582c0cd490 2023-07-06T03:57:48.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.623+0000 7f58347ff700 1 -- 172.21.15.97:0/1484030899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c04e340 con 0x7f582c0cd490 2023-07-06T03:57:48.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.624+0000 7f5822ffd700 1 -- 172.21.15.97:0/1484030899 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f581c013470 con 0x7f582c0cd490 2023-07-06T03:57:48.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.625+0000 7f5822ffd700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 0x7f581006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:48.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.626+0000 7f5822ffd700 1 -- 172.21.15.97:0/1484030899 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f581c089710 con 0x7f582c0cd490 2023-07-06T03:57:48.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.626+0000 7f583259b700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 0x7f581006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:48.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.626+0000 7f583259b700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 0x7f581006d190 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5828014040 tx=0x7f5828008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:48.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.630+0000 7f5822ffd700 1 -- 172.21.15.97:0/1484030899 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f581c0581a0 con 0x7f582c0cd490 2023-07-06T03:57:48.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.825+0000 7f58347ff700 1 -- 172.21.15.97:0/1484030899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f582c0d2950 con 0x7f582c0cd490 2023-07-06T03:57:48.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.825+0000 7f5822ffd700 1 -- 172.21.15.97:0/1484030899 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+11 (secure 0 0 0) 0x7f581c057d30 con 0x7f582c0cd490 2023-07-06T03:57:48.827 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-07-06T03:57:48.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 -- 172.21.15.97:0/1484030899 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 msgr2=0x7f581006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 0x7f581006d190 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5828014040 tx=0x7f5828008030 comp rx=0 tx=0).stop 2023-07-06T03:57:48.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 -- 172.21.15.97:0/1484030899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c0cd490 msgr2=0x7f582c0ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:48.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c0cd490 0x7f582c0ccb20 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f581c00ebe0 tx=0x7f581c00eef0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 -- 172.21.15.97:0/1484030899 shutdown_connections 2023-07-06T03:57:48.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f581006ace0 0x7f581006d190 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f582c0cf7e0 0x7f582c159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f582c06b380 0x7f582c0b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 --2- 172.21.15.97:0/1484030899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f582c0cd490 0x7f582c0ccb20 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:48.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 -- 172.21.15.97:0/1484030899 >> 172.21.15.97:0/1484030899 conn(0x7f582c06cf30 msgr2=0x7f582c0b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:48.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 -- 172.21.15.97:0/1484030899 shutdown_connections 2023-07-06T03:57:48.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:48.828+0000 7f58347ff700 1 -- 172.21.15.97:0/1484030899 wait complete. 2023-07-06T03:57:48.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:57:49.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.041+0000 7f1a2f1ff700 1 Processor -- start 2023-07-06T03:57:49.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.041+0000 7f1a2f1ff700 1 -- start start 2023-07-06T03:57:49.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.042+0000 7f1a2f1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d09d0 0x7f1a280b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.042+0000 7f1a2f1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a2815ca10 0x7f1a280cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.042+0000 7f1a2f1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a280cf0e0 con 0x7f1a280d0da0 2023-07-06T03:57:49.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.042+0000 7f1a2f1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a280cf570 con 0x7f1a2815ca10 2023-07-06T03:57:49.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.042+0000 7f1a2f1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280d3d00 con 0x7f1a280d09d0 2023-07-06T03:57:49.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.042+0000 7f1a27fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a2815ca10 0x7f1a280cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.042+0000 7f1a27fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a2815ca10 0x7f1a280cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44216/0 (socket says 172.21.15.97:44216) 2023-07-06T03:57:49.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.042+0000 7f1a27fff700 1 -- 172.21.15.97:0/1461487268 learned_addr learned my addr 172.21.15.97:0/1461487268 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:49.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.042+0000 7f1a2cf9b700 1 --2- 172.21.15.97:0/1461487268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d09d0 0x7f1a280b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.043+0000 7f1a27fff700 1 -- 172.21.15.97:0/1461487268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d09d0 msgr2=0x7f1a280b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:49.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.043+0000 7f1a27fff700 1 --2- 172.21.15.97:0/1461487268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d09d0 0x7f1a280b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.043+0000 7f1a27fff700 1 -- 172.21.15.97:0/1461487268 >> v1:172.21.15.146:6789/0 conn(0x7f1a280d0da0 legacy=0x7f1a280cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.043+0000 7f1a27fff700 1 -- 172.21.15.97:0/1461487268 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a280ce4d0 con 0x7f1a2815ca10 2023-07-06T03:57:49.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.043+0000 7f1a27fff700 1 --2- 172.21.15.97:0/1461487268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a2815ca10 0x7f1a280cde60 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f1a140098f0 tx=0x7f1a14009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5130e4237ce176a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:49.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.043+0000 7f1a277fe700 1 -- 172.21.15.97:0/1461487268 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14011830 con 0x7f1a2815ca10 2023-07-06T03:57:49.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.043+0000 7f1a277fe700 1 -- 172.21.15.97:0/1461487268 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a14011990 con 0x7f1a2815ca10 2023-07-06T03:57:49.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.043+0000 7f1a277fe700 1 -- 172.21.15.97:0/1461487268 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14011d20 con 0x7f1a2815ca10 2023-07-06T03:57:49.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.044+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/1461487268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a2815ca10 msgr2=0x7f1a280cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.044+0000 7f1a2f1ff700 1 --2- 172.21.15.97:0/1461487268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a2815ca10 0x7f1a280cde60 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f1a140098f0 tx=0x7f1a14009c00 comp rx=0 tx=0).stop 2023-07-06T03:57:49.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.044+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/1461487268 shutdown_connections 2023-07-06T03:57:49.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.044+0000 7f1a2f1ff700 1 --2- 172.21.15.97:0/1461487268 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d09d0 0x7f1a280b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.044+0000 7f1a2f1ff700 1 --2- 172.21.15.97:0/1461487268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a2815ca10 0x7f1a280cde60 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.044+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/1461487268 >> 172.21.15.97:0/1461487268 conn(0x7f1a280720b0 msgr2=0x7f1a280d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:49.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.044+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/1461487268 shutdown_connections 2023-07-06T03:57:49.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.044+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/1461487268 wait complete. 2023-07-06T03:57:49.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.045+0000 7f1a2f1ff700 1 Processor -- start 2023-07-06T03:57:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.046+0000 7f1a2f1ff700 1 -- start start 2023-07-06T03:57:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2f1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a280d09d0 0x7f1a281f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2f1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d0da0 0x7f1a280b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2f1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280b6190 0x7f1a280c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2f1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280b6710 con 0x7f1a280b6190 2023-07-06T03:57:49.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2f1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280b69f0 con 0x7f1a280d09d0 2023-07-06T03:57:49.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2f1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280b6cd0 con 0x7f1a280d0da0 2023-07-06T03:57:49.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2d79c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280b6190 0x7f1a280c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2d79c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280b6190 0x7f1a280c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44232/0 (socket says 172.21.15.97:44232) 2023-07-06T03:57:49.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2d79c700 1 -- 172.21.15.97:0/998634991 learned_addr learned my addr 172.21.15.97:0/998634991 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:49.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2cf9b700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a280d09d0 0x7f1a281f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a27fff700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d0da0 0x7f1a280b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2d79c700 1 -- 172.21.15.97:0/998634991 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d0da0 msgr2=0x7f1a280b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2d79c700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d0da0 0x7f1a280b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2d79c700 1 -- 172.21.15.97:0/998634991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a280d09d0 msgr2=0x7f1a281f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2d79c700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a280d09d0 0x7f1a281f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2d79c700 1 -- 172.21.15.97:0/998634991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a14009550 con 0x7f1a280b6190 2023-07-06T03:57:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a2cf9b700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a280d09d0 0x7f1a281f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.047+0000 7f1a27fff700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d0da0 0x7f1a280b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.048+0000 7f1a2d79c700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280b6190 0x7f1a280c7f00 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000cb50 tx=0x7f1a2000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.048+0000 7f1a257fa700 1 -- 172.21.15.97:0/998634991 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20003c00 con 0x7f1a280b6190 2023-07-06T03:57:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.048+0000 7f1a257fa700 1 -- 172.21.15.97:0/998634991 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a20003d60 con 0x7f1a280b6190 2023-07-06T03:57:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.048+0000 7f1a257fa700 1 -- 172.21.15.97:0/998634991 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2000ab70 con 0x7f1a280b6190 2023-07-06T03:57:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.048+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/998634991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a280b6fb0 con 0x7f1a280b6190 2023-07-06T03:57:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.048+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/998634991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a280c9470 con 0x7f1a280b6190 2023-07-06T03:57:49.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.049+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/998634991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a280b74d0 con 0x7f1a280b6190 2023-07-06T03:57:49.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.050+0000 7f1a257fa700 1 -- 172.21.15.97:0/998634991 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a2000acd0 con 0x7f1a280b6190 2023-07-06T03:57:49.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.051+0000 7f1a257fa700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0c06ace0 0x7f1a0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.051+0000 7f1a257fa700 1 -- 172.21.15.97:0/998634991 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f1a20089540 con 0x7f1a280b6190 2023-07-06T03:57:49.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.051+0000 7f1a2cf9b700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0c06ace0 0x7f1a0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.052+0000 7f1a2cf9b700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0c06ace0 0x7f1a0c06d190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009ac0 tx=0x7f1a1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:49.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.056+0000 7f1a257fa700 1 -- 172.21.15.97:0/998634991 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1a200545d0 con 0x7f1a280b6190 2023-07-06T03:57:49.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.302+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/998634991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a2804e340 con 0x7f1a280b6190 2023-07-06T03:57:49.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.302+0000 7f1a257fa700 1 -- 172.21.15.97:0/998634991 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4680 (secure 0 0 0) 0x7f1a20057bf0 con 0x7f1a280b6190 2023-07-06T03:57:49.304 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:49.304 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/3313682816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3313682816},{"type":"v1","addr":"172.21.15.146:6834","nonce":3313682816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9624,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/387461585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":387461585},{"type":"v1","addr":"172.21.15.146:6835","nonce":387461585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9630,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3109027091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3109027091},{"type":"v1","addr":"172.21.15.97:6835","nonce":3109027091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:56:04.566296+0000","modified":"2023-07-06T03:56:08.096494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9633":{"gid":9633,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2797569799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2797569799},{"type":"v1","addr":"172.21.15.97:6837","nonce":2797569799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-06T03:57:49.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/998634991 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0c06ace0 msgr2=0x7f1a0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0c06ace0 0x7f1a0c06d190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009ac0 tx=0x7f1a1c008040 comp rx=0 tx=0).stop 2023-07-06T03:57:49.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/998634991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280b6190 msgr2=0x7f1a280c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280b6190 0x7f1a280c7f00 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000cb50 tx=0x7f1a2000cf10 comp rx=0 tx=0).stop 2023-07-06T03:57:49.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/998634991 shutdown_connections 2023-07-06T03:57:49.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f1a0c06ace0 0x7f1a0c06d190 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a280b6190 0x7f1a280c7f00 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a280d09d0 0x7f1a281f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 --2- 172.21.15.97:0/998634991 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f1a280d0da0 0x7f1a280b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/998634991 >> 172.21.15.97:0/998634991 conn(0x7f1a280720b0 msgr2=0x7f1a280cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:49.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/998634991 shutdown_connections 2023-07-06T03:57:49.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.305+0000 7f1a2f1ff700 1 -- 172.21.15.97:0/998634991 wait complete. 2023-07-06T03:57:49.308 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 103 2023-07-06T03:57:49.325 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:57:49.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.518+0000 7fb0be3ff700 1 Processor -- start 2023-07-06T03:57:49.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.518+0000 7fb0be3ff700 1 -- start start 2023-07-06T03:57:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.519+0000 7fb0be3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0b80cbe90 0x7fb0b8070e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.519+0000 7fb0be3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0b80713b0 0x7fb0b806a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.519+0000 7fb0be3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b8071810 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.519+0000 7fb0be3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b815cb20 con 0x7fb0b80713b0 2023-07-06T03:57:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.519+0000 7fb0be3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b81602f0 con 0x7fb0b80cbe90 2023-07-06T03:57:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.520+0000 7fb0bc99c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0b80713b0 0x7fb0b806a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.520+0000 7fb0bc99c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0b80713b0 0x7fb0b806a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44234/0 (socket says 172.21.15.97:44234) 2023-07-06T03:57:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.520+0000 7fb0bc99c700 1 -- 172.21.15.97:0/1658006257 learned_addr learned my addr 172.21.15.97:0/1658006257 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.520+0000 7fb0b77fe700 1 --2- 172.21.15.97:0/1658006257 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0b80cbe90 0x7fb0b8070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.520+0000 7fb0bc99c700 1 -- 172.21.15.97:0/1658006257 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0b80cbe90 msgr2=0x7fb0b8070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.520+0000 7fb0bc99c700 1 --2- 172.21.15.97:0/1658006257 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0b80cbe90 0x7fb0b8070e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.520+0000 7fb0bc99c700 1 -- 172.21.15.97:0/1658006257 >> v1:172.21.15.146:6789/0 conn(0x7fb0b80cf5e0 legacy=0x7fb0b80cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.520+0000 7fb0bc99c700 1 -- 172.21.15.97:0/1658006257 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8160590 con 0x7fb0b80713b0 2023-07-06T03:57:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.520+0000 7fb0bc99c700 1 --2- 172.21.15.97:0/1658006257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b80713b0 0x7fb0b806a060 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009e50 tx=0x7fb0b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58acfbc22709b76 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.521+0000 7fb0b6ffd700 1 -- 172.21.15.97:0/1658006257 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b000e070 con 0x7fb0b80713b0 2023-07-06T03:57:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.521+0000 7fb0b6ffd700 1 -- 172.21.15.97:0/1658006257 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b000b9c0 con 0x7fb0b80713b0 2023-07-06T03:57:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.521+0000 7fb0b6ffd700 1 -- 172.21.15.97:0/1658006257 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b000bd70 con 0x7fb0b80713b0 2023-07-06T03:57:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.521+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1658006257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b80713b0 msgr2=0x7fb0b806a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.521+0000 7fb0be3ff700 1 --2- 172.21.15.97:0/1658006257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b80713b0 0x7fb0b806a060 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009e50 tx=0x7fb0b000b330 comp rx=0 tx=0).stop 2023-07-06T03:57:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.521+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1658006257 shutdown_connections 2023-07-06T03:57:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.521+0000 7fb0be3ff700 1 --2- 172.21.15.97:0/1658006257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b80713b0 0x7fb0b806a060 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.521+0000 7fb0be3ff700 1 --2- 172.21.15.97:0/1658006257 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0b80cbe90 0x7fb0b8070e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.521+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1658006257 >> 172.21.15.97:0/1658006257 conn(0x7fb0b80cfae0 msgr2=0x7fb0b80cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.521+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1658006257 shutdown_connections 2023-07-06T03:57:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.521+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1658006257 wait complete. 2023-07-06T03:57:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.523+0000 7fb0be3ff700 1 Processor -- start 2023-07-06T03:57:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.523+0000 7fb0be3ff700 1 -- start start 2023-07-06T03:57:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.524+0000 7fb0be3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0b80713b0 0x7fb0b80ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.524+0000 7fb0be3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0b80cbe90 0x7fb0b815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.524+0000 7fb0be3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b80cf5e0 0x7fb0b815bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.524+0000 7fb0be3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b815ecd0 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.524+0000 7fb0be3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b815efb0 con 0x7fb0b80cbe90 2023-07-06T03:57:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.524+0000 7fb0be3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b815f290 con 0x7fb0b80713b0 2023-07-06T03:57:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.524+0000 7fb0bc99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b80cf5e0 0x7fb0b815bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.524+0000 7fb0bc99c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b80cf5e0 0x7fb0b815bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44250/0 (socket says 172.21.15.97:44250) 2023-07-06T03:57:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.524+0000 7fb0bc99c700 1 -- 172.21.15.97:0/1048474816 learned_addr learned my addr 172.21.15.97:0/1048474816 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.524+0000 7fb0b77fe700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0b80cbe90 0x7fb0b815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.524+0000 7fb0b7fff700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0b80713b0 0x7fb0b80ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.525+0000 7fb0bc99c700 1 -- 172.21.15.97:0/1048474816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0b80713b0 msgr2=0x7fb0b80ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.525+0000 7fb0bc99c700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0b80713b0 0x7fb0b80ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.525+0000 7fb0bc99c700 1 -- 172.21.15.97:0/1048474816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0b80cbe90 msgr2=0x7fb0b815ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.525+0000 7fb0bc99c700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0b80cbe90 0x7fb0b815ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.525+0000 7fb0bc99c700 1 -- 172.21.15.97:0/1048474816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b0009b30 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.525+0000 7fb0bc99c700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b80cf5e0 0x7fb0b815bf50 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80d0750 tx=0x7fb0b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.525+0000 7fb0b4ff9700 1 -- 172.21.15.97:0/1048474816 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0004800 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.525+0000 7fb0b4ff9700 1 -- 172.21.15.97:0/1048474816 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b0004960 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.525+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1048474816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b815f570 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.525+0000 7fb0b4ff9700 1 -- 172.21.15.97:0/1048474816 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b001c680 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.525+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1048474816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b80adf60 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.527+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1048474816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b80ae120 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.527+0000 7fb0b4ff9700 1 -- 172.21.15.97:0/1048474816 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb0b001c7e0 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.528+0000 7fb0b4ff9700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a406ace0 0x7fb0a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.528+0000 7fb0b4ff9700 1 -- 172.21.15.97:0/1048474816 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fb0b008b0f0 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.528+0000 7fb0b7fff700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a406ace0 0x7fb0a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.529+0000 7fb0b7fff700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a406ace0 0x7fb0a406d190 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8007fd0 tx=0x7fb0a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.532+0000 7fb0b4ff9700 1 -- 172.21.15.97:0/1048474816 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb0b0056170 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.727+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1048474816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0b815fa30 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.728+0000 7fb0b4ff9700 1 -- 172.21.15.97:0/1048474816 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18436 (secure 0 0 0) 0x7fb0b0059790 con 0x7fb0b80cf5e0 2023-07-06T03:57:49.730 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:49.730 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":138,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:56:54.911710+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:56:02.531370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-06T03:56:03.548148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.7","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:57:49.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1048474816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a406ace0 msgr2=0x7fb0a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a406ace0 0x7fb0a406d190 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8007fd0 tx=0x7fb0a8009040 comp rx=0 tx=0).stop 2023-07-06T03:57:49.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1048474816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b80cf5e0 msgr2=0x7fb0b815bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b80cf5e0 0x7fb0b815bf50 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80d0750 tx=0x7fb0b0004ad0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1048474816 shutdown_connections 2023-07-06T03:57:49.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb0a406ace0 0x7fb0a406d190 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b80cf5e0 0x7fb0b815bf50 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb0b80713b0 0x7fb0b80ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 --2- 172.21.15.97:0/1048474816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0b80cbe90 0x7fb0b815ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1048474816 >> 172.21.15.97:0/1048474816 conn(0x7fb0b80cfae0 msgr2=0x7fb0b806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:49.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1048474816 shutdown_connections 2023-07-06T03:57:49.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.731+0000 7fb0be3ff700 1 -- 172.21.15.97:0/1048474816 wait complete. 2023-07-06T03:57:49.750 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T03:57:49.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:57:49.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.943+0000 7f4d485ff700 1 Processor -- start 2023-07-06T03:57:49.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.943+0000 7f4d485ff700 1 -- start start 2023-07-06T03:57:49.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.944+0000 7f4d485ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4d400d09b0 0x7f4d400b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.944+0000 7f4d485ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d4015ce60 0x7f4d400d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.944+0000 7f4d485ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d400d16b0 con 0x7f4d400cf780 2023-07-06T03:57:49.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.944+0000 7f4d485ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d400d0e50 con 0x7f4d400d09b0 2023-07-06T03:57:49.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.944+0000 7f4d485ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400b24d0 con 0x7f4d4015ce60 2023-07-06T03:57:49.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.945+0000 7f4d45b9a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4d400d09b0 0x7f4d400b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.945+0000 7f4d4639b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4d400cf780 0x7f4d400cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:53926/0 (socket says 172.21.15.97:53926) 2023-07-06T03:57:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.945+0000 7f4d4639b700 1 -- 172.21.15.97:0/477843705 learned_addr learned my addr 172.21.15.97:0/477843705 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.945+0000 7f4d46b9c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d4015ce60 0x7f4d400d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.945+0000 7f4d45b9a700 1 -- 172.21.15.97:0/477843705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d4015ce60 msgr2=0x7f4d400d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.945+0000 7f4d45b9a700 1 --2- 172.21.15.97:0/477843705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d4015ce60 0x7f4d400d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.945+0000 7f4d45b9a700 1 -- 172.21.15.97:0/477843705 >> v1:172.21.15.146:6789/0 conn(0x7f4d400cf780 legacy=0x7f4d400cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.946+0000 7f4d45b9a700 1 -- 172.21.15.97:0/477843705 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d400b8b60 con 0x7f4d400d09b0 2023-07-06T03:57:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.946+0000 7f4d46b9c700 1 --2- 172.21.15.97:0/477843705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d4015ce60 0x7f4d400d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:57:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.946+0000 7f4d45399700 1 -- 172.21.15.97:0/477843705 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414079029 0 0) 0x7f4d400d16b0 con 0x7f4d400cf780 2023-07-06T03:57:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.946+0000 7f4d45b9a700 1 --2- 172.21.15.97:0/477843705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d400d09b0 0x7f4d400b8230 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009840 tx=0x7f4d30009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2177a899e7ddee7f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.946+0000 7f4d45399700 1 -- 172.21.15.97:0/477843705 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30011830 con 0x7f4d400d09b0 2023-07-06T03:57:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.946+0000 7f4d45399700 1 -- 172.21.15.97:0/477843705 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d30011990 con 0x7f4d400d09b0 2023-07-06T03:57:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.946+0000 7f4d45399700 1 -- 172.21.15.97:0/477843705 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30011d20 con 0x7f4d400d09b0 2023-07-06T03:57:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.947+0000 7f4d485ff700 1 -- 172.21.15.97:0/477843705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d400d09b0 msgr2=0x7f4d400b8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.947+0000 7f4d485ff700 1 --2- 172.21.15.97:0/477843705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d400d09b0 0x7f4d400b8230 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009840 tx=0x7f4d30009b50 comp rx=0 tx=0).stop 2023-07-06T03:57:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.947+0000 7f4d485ff700 1 -- 172.21.15.97:0/477843705 shutdown_connections 2023-07-06T03:57:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.947+0000 7f4d485ff700 1 --2- 172.21.15.97:0/477843705 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d4015ce60 0x7f4d400d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.947+0000 7f4d485ff700 1 --2- 172.21.15.97:0/477843705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d400d09b0 0x7f4d400b8230 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.947+0000 7f4d485ff700 1 -- 172.21.15.97:0/477843705 >> 172.21.15.97:0/477843705 conn(0x7f4d4006c1c0 msgr2=0x7f4d4006c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:49.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.947+0000 7f4d485ff700 1 -- 172.21.15.97:0/477843705 shutdown_connections 2023-07-06T03:57:49.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.947+0000 7f4d485ff700 1 -- 172.21.15.97:0/477843705 wait complete. 2023-07-06T03:57:49.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.949+0000 7f4d485ff700 1 Processor -- start 2023-07-06T03:57:49.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.949+0000 7f4d485ff700 1 -- start start 2023-07-06T03:57:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d485ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d400cf780 0x7f4d40147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d485ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d400d09b0 0x7f4d40147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d485ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d4015ce60 0x7f4d400b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d485ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4015ff70 con 0x7f4d400d09b0 2023-07-06T03:57:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d485ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4015d260 con 0x7f4d400cf780 2023-07-06T03:57:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d485ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4015d540 con 0x7f4d4015ce60 2023-07-06T03:57:49.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d45b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d400d09b0 0x7f4d40147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d45b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d400d09b0 0x7f4d40147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37492/0 (socket says 172.21.15.97:37492) 2023-07-06T03:57:49.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d45b9a700 1 -- 172.21.15.97:0/1975229883 learned_addr learned my addr 172.21.15.97:0/1975229883 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:49.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d46b9c700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d4015ce60 0x7f4d400b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d4639b700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d400cf780 0x7f4d40147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d45b9a700 1 -- 172.21.15.97:0/1975229883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d4015ce60 msgr2=0x7f4d400b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d45b9a700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d4015ce60 0x7f4d400b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d45b9a700 1 -- 172.21.15.97:0/1975229883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d400cf780 msgr2=0x7f4d40147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d45b9a700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d400cf780 0x7f4d40147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.950+0000 7f4d45b9a700 1 -- 172.21.15.97:0/1975229883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d38009a70 con 0x7f4d400d09b0 2023-07-06T03:57:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.951+0000 7f4d45b9a700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d400d09b0 0x7f4d40147020 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009fd0 tx=0x7f4d30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.951+0000 7f4d36ffd700 1 -- 172.21.15.97:0/1975229883 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d300038a0 con 0x7f4d400d09b0 2023-07-06T03:57:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.951+0000 7f4d36ffd700 1 -- 172.21.15.97:0/1975229883 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3001f070 con 0x7f4d400d09b0 2023-07-06T03:57:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.951+0000 7f4d485ff700 1 -- 172.21.15.97:0/1975229883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d300094a0 con 0x7f4d400d09b0 2023-07-06T03:57:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.951+0000 7f4d36ffd700 1 -- 172.21.15.97:0/1975229883 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30003c40 con 0x7f4d400d09b0 2023-07-06T03:57:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.951+0000 7f4d485ff700 1 -- 172.21.15.97:0/1975229883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4015db00 con 0x7f4d400d09b0 2023-07-06T03:57:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.952+0000 7f4d485ff700 1 -- 172.21.15.97:0/1975229883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4015dcf0 con 0x7f4d400d09b0 2023-07-06T03:57:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.953+0000 7f4d36ffd700 1 -- 172.21.15.97:0/1975229883 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d30026480 con 0x7f4d400d09b0 2023-07-06T03:57:49.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.954+0000 7f4d36ffd700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d2406ace0 0x7f4d2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:49.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.954+0000 7f4d36ffd700 1 -- 172.21.15.97:0/1975229883 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f4d30021070 con 0x7f4d400d09b0 2023-07-06T03:57:49.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.954+0000 7f4d4639b700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d2406ace0 0x7f4d2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:49.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.955+0000 7f4d4639b700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d2406ace0 0x7f4d2406d190 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c000f60 tx=0x7f4d3c007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:49.958+0000 7f4d36ffd700 1 -- 172.21.15.97:0/1975229883 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4d30055e70 con 0x7f4d400d09b0 2023-07-06T03:57:50.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.195+0000 7f4d485ff700 1 -- 172.21.15.97:0/1975229883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d4004e340 con 0x7f4d400d09b0 2023-07-06T03:57:50.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.196+0000 7f4d36ffd700 1 -- 172.21.15.97:0/1975229883 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4680 (secure 0 0 0) 0x7f4d30015090 con 0x7f4d400d09b0 2023-07-06T03:57:50.198 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:50.198 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/3313682816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3313682816},{"type":"v1","addr":"172.21.15.146:6834","nonce":3313682816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9624,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/387461585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":387461585},{"type":"v1","addr":"172.21.15.146:6835","nonce":387461585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9630,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3109027091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3109027091},{"type":"v1","addr":"172.21.15.97:6835","nonce":3109027091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:56:04.566296+0000","modified":"2023-07-06T03:56:08.096494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9633":{"gid":9633,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2797569799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2797569799},{"type":"v1","addr":"172.21.15.97:6837","nonce":2797569799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-06T03:57:50.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 -- 172.21.15.97:0/1975229883 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d2406ace0 msgr2=0x7f4d2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d2406ace0 0x7f4d2406d190 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c000f60 tx=0x7f4d3c007c90 comp rx=0 tx=0).stop 2023-07-06T03:57:50.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 -- 172.21.15.97:0/1975229883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d400d09b0 msgr2=0x7f4d40147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d400d09b0 0x7f4d40147020 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009fd0 tx=0x7f4d30004750 comp rx=0 tx=0).stop 2023-07-06T03:57:50.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 -- 172.21.15.97:0/1975229883 shutdown_connections 2023-07-06T03:57:50.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4d2406ace0 0x7f4d2406d190 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4d4015ce60 0x7f4d400b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4d400cf780 0x7f4d40147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 --2- 172.21.15.97:0/1975229883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d400d09b0 0x7f4d40147020 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 -- 172.21.15.97:0/1975229883 >> 172.21.15.97:0/1975229883 conn(0x7f4d4006c1c0 msgr2=0x7f4d400b7870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:50.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 -- 172.21.15.97:0/1975229883 shutdown_connections 2023-07-06T03:57:50.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.198+0000 7f4d485ff700 1 -- 172.21.15.97:0/1975229883 wait complete. 2023-07-06T03:57:50.202 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 103 2023-07-06T03:57:50.217 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:57:50.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.410+0000 7fa0137ff700 1 Processor -- start 2023-07-06T03:57:50.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.410+0000 7fa0137ff700 1 -- start start 2023-07-06T03:57:50.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.411+0000 7fa0137ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa00c06df30 0x7fa00c06a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa0137ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00c06aa00 con 0x7fa00c06a5a0 2023-07-06T03:57:50.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa0137ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00c071ca0 con 0x7fa00c0c5160 2023-07-06T03:57:50.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa0137ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00c15a860 con 0x7fa00c06df30 2023-07-06T03:57:50.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa011d9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa00c06a5a0 0x7fa00c0c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59220/0 (socket says 172.21.15.97:59220) 2023-07-06T03:57:50.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa011d9c700 1 -- 172.21.15.97:0/1432764149 learned_addr learned my addr 172.21.15.97:0/1432764149 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:50.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa010d9a700 1 --2- 172.21.15.97:0/1432764149 >> v2:172.21.15.146:3300/0 conn(0x7fa00c06df30 0x7fa00c06a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44651796 0 0) 0x7fa00c06aa00 con 0x7fa00c06a5a0 2023-07-06T03:57:50.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ff80035b0 con 0x7fa00c06a5a0 2023-07-06T03:57:50.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3563902258 0 0) 0x7f9ff80035b0 con 0x7fa00c06a5a0 2023-07-06T03:57:50.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 >> v2:172.21.15.146:3300/0 conn(0x7fa00c06df30 msgr2=0x7fa00c06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:50.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa003fff700 1 --2- 172.21.15.97:0/1432764149 >> v2:172.21.15.146:3300/0 conn(0x7fa00c06df30 0x7fa00c06a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 >> v1:172.21.15.146:6789/0 conn(0x7fa00c0c5160 legacy=0x7fa00c06d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.412+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c071120 con 0x7fa00c06a5a0 2023-07-06T03:57:50.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fa0040019f0 con 0x7fa00c06a5a0 2023-07-06T03:57:50.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 >> v1:172.21.15.97:6789/0 conn(0x7fa00c06a5a0 legacy=0x7fa00c0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa003fff700 1 --2- 172.21.15.97:0/1432764149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ff8003a00 0x7f9ff8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa003fff700 1 --2- 172.21.15.97:0/1432764149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ff8004850 0x7f9ff8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa003fff700 1 --2- 172.21.15.97:0/1432764149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8005200 0x7f9ff800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c15a860 con 0x7f9ff8005200 2023-07-06T03:57:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff800c080 con 0x7f9ff8004850 2023-07-06T03:57:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff800c240 con 0x7f9ff8003a00 2023-07-06T03:57:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa011d9c700 1 --2- 172.21.15.97:0/1432764149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8005200 0x7f9ff800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa01159b700 1 --2- 172.21.15.97:0/1432764149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ff8003a00 0x7f9ff8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa010d9a700 1 --2- 172.21.15.97:0/1432764149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ff8004850 0x7f9ff8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa011d9c700 1 -- 172.21.15.97:0/1432764149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ff8003a00 msgr2=0x7f9ff8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:50.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa011d9c700 1 --2- 172.21.15.97:0/1432764149 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ff8003a00 0x7f9ff8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa011d9c700 1 -- 172.21.15.97:0/1432764149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ff8004850 msgr2=0x7f9ff8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:50.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa011d9c700 1 --2- 172.21.15.97:0/1432764149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ff8004850 0x7f9ff8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa011d9c700 1 -- 172.21.15.97:0/1432764149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff80035b0 con 0x7f9ff8005200 2023-07-06T03:57:50.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.413+0000 7fa011d9c700 1 -- 172.21.15.97:0/1432764149 reap_dead start 2023-07-06T03:57:50.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.414+0000 7fa011d9c700 1 --2- 172.21.15.97:0/1432764149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8005200 0x7f9ff800ba10 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fa00400a830 tx=0x7f9ff8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26efd95bfdb8185 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:50.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.414+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00400a2f0 con 0x7f9ff8005200 2023-07-06T03:57:50.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.414+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa004012600 con 0x7f9ff8005200 2023-07-06T03:57:50.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.414+0000 7fa003fff700 1 -- 172.21.15.97:0/1432764149 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0040129b0 con 0x7f9ff8005200 2023-07-06T03:57:50.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.414+0000 7fa0137ff700 1 -- 172.21.15.97:0/1432764149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8005200 msgr2=0x7f9ff800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.414+0000 7fa0137ff700 1 --2- 172.21.15.97:0/1432764149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8005200 0x7f9ff800ba10 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fa00400a830 tx=0x7f9ff8005880 comp rx=0 tx=0).stop 2023-07-06T03:57:50.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.414+0000 7fa0137ff700 1 -- 172.21.15.97:0/1432764149 shutdown_connections 2023-07-06T03:57:50.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.414+0000 7fa0137ff700 1 --2- 172.21.15.97:0/1432764149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff8005200 0x7f9ff800ba10 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.414+0000 7fa0137ff700 1 -- 172.21.15.97:0/1432764149 >> 172.21.15.97:0/1432764149 conn(0x7fa00c0cc0e0 msgr2=0x7fa00c0ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.414+0000 7fa0137ff700 1 -- 172.21.15.97:0/1432764149 shutdown_connections 2023-07-06T03:57:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.414+0000 7fa0137ff700 1 -- 172.21.15.97:0/1432764149 wait complete. 2023-07-06T03:57:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.416+0000 7fa0137ff700 1 Processor -- start 2023-07-06T03:57:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.416+0000 7fa0137ff700 1 -- start start 2023-07-06T03:57:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.417+0000 7fa0137ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ff8005200 0x7fa00c148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.417+0000 7fa0137ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa00c15a9a0 0x7fa00c0c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.417+0000 7fa0137ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00c0c2730 0x7fa00c0c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.417+0000 7fa0137ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c15d780 con 0x7fa00c0c2730 2023-07-06T03:57:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.417+0000 7fa0137ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c149430 con 0x7fa00c15a9a0 2023-07-06T03:57:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.417+0000 7fa0137ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c0b45f0 con 0x7f9ff8005200 2023-07-06T03:57:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa011d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00c0c2730 0x7fa00c0c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa011d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00c0c2730 0x7fa00c0c2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37510/0 (socket says 172.21.15.97:37510) 2023-07-06T03:57:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa011d9c700 1 -- 172.21.15.97:0/3461246296 learned_addr learned my addr 172.21.15.97:0/3461246296 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa010d9a700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa00c15a9a0 0x7fa00c0c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa01159b700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ff8005200 0x7fa00c148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa011d9c700 1 -- 172.21.15.97:0/3461246296 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ff8005200 msgr2=0x7fa00c148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa011d9c700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ff8005200 0x7fa00c148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa011d9c700 1 -- 172.21.15.97:0/3461246296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa00c15a9a0 msgr2=0x7fa00c0c21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa011d9c700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa00c15a9a0 0x7fa00c0c21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa011d9c700 1 -- 172.21.15.97:0/3461246296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c0b1ec0 con 0x7fa00c0c2730 2023-07-06T03:57:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa011d9c700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00c0c2730 0x7fa00c0c2ba0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fa00c15de20 tx=0x7fa004001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa001ffb700 1 -- 172.21.15.97:0/3461246296 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0040017b0 con 0x7fa00c0c2730 2023-07-06T03:57:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa001ffb700 1 -- 172.21.15.97:0/3461246296 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa004001910 con 0x7fa00c0c2730 2023-07-06T03:57:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa0137ff700 1 -- 172.21.15.97:0/3461246296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00c0b21a0 con 0x7fa00c0c2730 2023-07-06T03:57:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.418+0000 7fa001ffb700 1 -- 172.21.15.97:0/3461246296 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00401d640 con 0x7fa00c0c2730 2023-07-06T03:57:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.419+0000 7fa0137ff700 1 -- 172.21.15.97:0/3461246296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00c0b2760 con 0x7fa00c0c2730 2023-07-06T03:57:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.420+0000 7fa0137ff700 1 -- 172.21.15.97:0/3461246296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00c04e340 con 0x7fa00c0c2730 2023-07-06T03:57:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.421+0000 7fa001ffb700 1 -- 172.21.15.97:0/3461246296 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa00401d7a0 con 0x7fa00c0c2730 2023-07-06T03:57:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.421+0000 7fa001ffb700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ff006ace0 0x7f9ff006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.422+0000 7fa001ffb700 1 -- 172.21.15.97:0/3461246296 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fa00408a8f0 con 0x7fa00c0c2730 2023-07-06T03:57:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.422+0000 7fa01159b700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ff006ace0 0x7f9ff006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.422+0000 7fa01159b700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ff006ace0 0x7f9ff006d190 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa008007fd0 tx=0x7fa008009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.426+0000 7fa001ffb700 1 -- 172.21.15.97:0/3461246296 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa004055a10 con 0x7fa00c0c2730 2023-07-06T03:57:50.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.667+0000 7fa0137ff700 1 -- 172.21.15.97:0/3461246296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa00c0ca510 con 0x7fa00c0c2730 2023-07-06T03:57:50.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.667+0000 7fa001ffb700 1 -- 172.21.15.97:0/3461246296 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4680 (secure 0 0 0) 0x7fa004026090 con 0x7fa00c0c2730 2023-07-06T03:57:50.669 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:50.669 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/3313682816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3313682816},{"type":"v1","addr":"172.21.15.146:6834","nonce":3313682816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9624,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/387461585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":387461585},{"type":"v1","addr":"172.21.15.146:6835","nonce":387461585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9630,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3109027091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3109027091},{"type":"v1","addr":"172.21.15.97:6835","nonce":3109027091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T03:56:04.566296+0000","modified":"2023-07-06T03:56:08.096494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9633":{"gid":9633,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/2797569799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2797569799},{"type":"v1","addr":"172.21.15.97:6837","nonce":2797569799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-07-06T03:57:50.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.669+0000 7fa0137ff700 1 -- 172.21.15.97:0/3461246296 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ff006ace0 msgr2=0x7f9ff006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.669+0000 7fa0137ff700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ff006ace0 0x7f9ff006d190 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa008007fd0 tx=0x7fa008009040 comp rx=0 tx=0).stop 2023-07-06T03:57:50.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.669+0000 7fa0137ff700 1 -- 172.21.15.97:0/3461246296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00c0c2730 msgr2=0x7fa00c0c2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.669+0000 7fa0137ff700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00c0c2730 0x7fa00c0c2ba0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fa00c15de20 tx=0x7fa004001d80 comp rx=0 tx=0).stop 2023-07-06T03:57:50.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.669+0000 7fa0137ff700 1 -- 172.21.15.97:0/3461246296 shutdown_connections 2023-07-06T03:57:50.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.669+0000 7fa0137ff700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f9ff006ace0 0x7f9ff006d190 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.669+0000 7fa0137ff700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa00c0c2730 0x7fa00c0c2ba0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.669+0000 7fa0137ff700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f9ff8005200 0x7fa00c148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.669+0000 7fa0137ff700 1 --2- 172.21.15.97:0/3461246296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa00c15a9a0 0x7fa00c0c21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.669+0000 7fa0137ff700 1 -- 172.21.15.97:0/3461246296 >> 172.21.15.97:0/3461246296 conn(0x7fa00c0cc0e0 msgr2=0x7fa00c0ad5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:50.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.670+0000 7fa0137ff700 1 -- 172.21.15.97:0/3461246296 shutdown_connections 2023-07-06T03:57:50.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.670+0000 7fa0137ff700 1 -- 172.21.15.97:0/3461246296 wait complete. 2023-07-06T03:57:50.673 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 103 2023-07-06T03:57:50.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:57:50.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.881+0000 7fe9ad1ff700 1 Processor -- start 2023-07-06T03:57:50.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.881+0000 7fe9ad1ff700 1 -- start start 2023-07-06T03:57:50.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.882+0000 7fe9ad1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9a815ca10 0x7fe9a80b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.882+0000 7fe9ad1ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9a80cde60 0x7fe9a80afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.882+0000 7fe9ad1ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a80cf0e0 con 0x7fe9a80d09d0 2023-07-06T03:57:50.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.882+0000 7fe9ad1ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a80ce2e0 con 0x7fe9a80cde60 2023-07-06T03:57:50.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.882+0000 7fe9ad1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80cf570 con 0x7fe9a815ca10 2023-07-06T03:57:50.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.882+0000 7fe9a759e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9a80cde60 0x7fe9a80afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.882+0000 7fe9a759e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9a80cde60 0x7fe9a80afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37524/0 (socket says 172.21.15.97:37524) 2023-07-06T03:57:50.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.883+0000 7fe9a759e700 1 -- 172.21.15.97:0/2931102004 learned_addr learned my addr 172.21.15.97:0/2931102004 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:50.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.883+0000 7fe9a659c700 1 --2- 172.21.15.97:0/2931102004 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9a815ca10 0x7fe9a80b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.883+0000 7fe9a759e700 1 -- 172.21.15.97:0/2931102004 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9a815ca10 msgr2=0x7fe9a80b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:50.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.883+0000 7fe9a759e700 1 --2- 172.21.15.97:0/2931102004 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9a815ca10 0x7fe9a80b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.883+0000 7fe9a759e700 1 -- 172.21.15.97:0/2931102004 >> v1:172.21.15.146:6789/0 conn(0x7fe9a80d09d0 legacy=0x7fe9a80d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.883+0000 7fe9a759e700 1 -- 172.21.15.97:0/2931102004 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a80d3d00 con 0x7fe9a80cde60 2023-07-06T03:57:50.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.883+0000 7fe9a759e700 1 --2- 172.21.15.97:0/2931102004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9a80cde60 0x7fe9a80afa90 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009e50 tx=0x7fe9a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431e7e2e01634ebe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:50.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.883+0000 7fe9a5d9b700 1 -- 172.21.15.97:0/2931102004 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000e070 con 0x7fe9a80cde60 2023-07-06T03:57:50.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.883+0000 7fe9a5d9b700 1 -- 172.21.15.97:0/2931102004 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a000b9c0 con 0x7fe9a80cde60 2023-07-06T03:57:50.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.883+0000 7fe9a5d9b700 1 -- 172.21.15.97:0/2931102004 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000bd70 con 0x7fe9a80cde60 2023-07-06T03:57:50.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.884+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/2931102004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9a80cde60 msgr2=0x7fe9a80afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.884+0000 7fe9ad1ff700 1 --2- 172.21.15.97:0/2931102004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9a80cde60 0x7fe9a80afa90 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009e50 tx=0x7fe9a000b330 comp rx=0 tx=0).stop 2023-07-06T03:57:50.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.884+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/2931102004 shutdown_connections 2023-07-06T03:57:50.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.884+0000 7fe9ad1ff700 1 --2- 172.21.15.97:0/2931102004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9a80cde60 0x7fe9a80afa90 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.884+0000 7fe9ad1ff700 1 --2- 172.21.15.97:0/2931102004 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9a815ca10 0x7fe9a80b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.884+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/2931102004 >> 172.21.15.97:0/2931102004 conn(0x7fe9a80720b0 msgr2=0x7fe9a80d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:50.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.884+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/2931102004 shutdown_connections 2023-07-06T03:57:50.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.884+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/2931102004 wait complete. 2023-07-06T03:57:50.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.886+0000 7fe9ad1ff700 1 Processor -- start 2023-07-06T03:57:50.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.887+0000 7fe9ad1ff700 1 -- start start 2023-07-06T03:57:50.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9ad1ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9a80cde60 0x7fe9a814d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9ad1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9a80d09d0 0x7fe9a815b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9ad1ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9a815b8a0 0x7fe9a80cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9ad1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80cacf0 con 0x7fe9a80d09d0 2023-07-06T03:57:50.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9ad1ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a815bd40 con 0x7fe9a815b8a0 2023-07-06T03:57:50.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9ad1ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80ca070 con 0x7fe9a80cde60 2023-07-06T03:57:50.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9a659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9a80d09d0 0x7fe9a815b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9a659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9a80d09d0 0x7fe9a815b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37540/0 (socket says 172.21.15.97:37540) 2023-07-06T03:57:50.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9a659c700 1 -- 172.21.15.97:0/3598136158 learned_addr learned my addr 172.21.15.97:0/3598136158 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:50.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9a6d9d700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9a80cde60 0x7fe9a814d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9a759e700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9a815b8a0 0x7fe9a80cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9a659c700 1 -- 172.21.15.97:0/3598136158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9a80cde60 msgr2=0x7fe9a814d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9a659c700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9a80cde60 0x7fe9a814d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9a659c700 1 -- 172.21.15.97:0/3598136158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9a815b8a0 msgr2=0x7fe9a80cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:50.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9a659c700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9a815b8a0 0x7fe9a80cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9a659c700 1 -- 172.21.15.97:0/3598136158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a0009b30 con 0x7fe9a80d09d0 2023-07-06T03:57:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.888+0000 7fe9a6d9d700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9a80cde60 0x7fe9a814d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.889+0000 7fe9a659c700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9a80d09d0 0x7fe9a815b360 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00e810 tx=0x7fe99c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.889+0000 7fe98f7fe700 1 -- 172.21.15.97:0/3598136158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c013610 con 0x7fe9a80d09d0 2023-07-06T03:57:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.889+0000 7fe98f7fe700 1 -- 172.21.15.97:0/3598136158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99c013770 con 0x7fe9a80d09d0 2023-07-06T03:57:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.889+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/3598136158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a80ca350 con 0x7fe9a80d09d0 2023-07-06T03:57:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.889+0000 7fe98f7fe700 1 -- 172.21.15.97:0/3598136158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c013b20 con 0x7fe9a80d09d0 2023-07-06T03:57:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.889+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/3598136158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a80ca910 con 0x7fe9a80d09d0 2023-07-06T03:57:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.890+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/3598136158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a80cae30 con 0x7fe9a80d09d0 2023-07-06T03:57:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.891+0000 7fe98f7fe700 1 -- 172.21.15.97:0/3598136158 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe99c013c80 con 0x7fe9a80d09d0 2023-07-06T03:57:50.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.892+0000 7fe98f7fe700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe99406ace0 0x7fe99406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:50.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.892+0000 7fe98f7fe700 1 -- 172.21.15.97:0/3598136158 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fe99c089120 con 0x7fe9a80d09d0 2023-07-06T03:57:50.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.892+0000 7fe9a6d9d700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe99406ace0 0x7fe99406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:50.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.893+0000 7fe9a6d9d700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe99406ace0 0x7fe99406d190 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe998000f60 tx=0x7fe998009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:50.896+0000 7fe98f7fe700 1 -- 172.21.15.97:0/3598136158 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe9a80cae30 con 0x7fe9a80d09d0 2023-07-06T03:57:51.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.097+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/3598136158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9a80cae30 con 0x7fe9a80d09d0 2023-07-06T03:57:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.098+0000 7fe98f7fe700 1 -- 172.21.15.97:0/3598136158 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18436 (secure 0 0 0) 0x7fe9a80cae30 con 0x7fe9a80d09d0 2023-07-06T03:57:51.100 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:51.100 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":138,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:56:54.911710+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-07-06T03:56:02.531370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-07-06T03:56:03.548148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.7","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:57:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.100+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/3598136158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe99406ace0 msgr2=0x7fe99406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.100+0000 7fe9ad1ff700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe99406ace0 0x7fe99406d190 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe998000f60 tx=0x7fe998009040 comp rx=0 tx=0).stop 2023-07-06T03:57:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.100+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/3598136158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9a80d09d0 msgr2=0x7fe9a815b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.100+0000 7fe9ad1ff700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9a80d09d0 0x7fe9a815b360 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00e810 tx=0x7fe99c00ebd0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.101+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/3598136158 shutdown_connections 2023-07-06T03:57:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.101+0000 7fe9ad1ff700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe99406ace0 0x7fe99406d190 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.101+0000 7fe9ad1ff700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9a815b8a0 0x7fe9a80cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.101+0000 7fe9ad1ff700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe9a80cde60 0x7fe9a814d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.101+0000 7fe9ad1ff700 1 --2- 172.21.15.97:0/3598136158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9a80d09d0 0x7fe9a815b360 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.101+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/3598136158 >> 172.21.15.97:0/3598136158 conn(0x7fe9a80720b0 msgr2=0x7fe9a806be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.101+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/3598136158 shutdown_connections 2023-07-06T03:57:51.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.101+0000 7fe9ad1ff700 1 -- 172.21.15.97:0/3598136158 wait complete. 2023-07-06T03:57:51.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T03:57:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.315+0000 7fecdd3ff700 1 Processor -- start 2023-07-06T03:57:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.316+0000 7fecdd3ff700 1 -- start start 2023-07-06T03:57:51.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.317+0000 7fecdd3ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fecd806f160 0x7fecd80cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:51.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.317+0000 7fecdd3ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd806bcf0 con 0x7fecd806b170 2023-07-06T03:57:51.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.317+0000 7fecdd3ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd806b730 con 0x7fecd806e500 2023-07-06T03:57:51.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.317+0000 7fecdd3ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd80ce470 con 0x7fecd806f160 2023-07-06T03:57:51.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.317+0000 7fecd6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fecd806f160 0x7fecd80cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:51.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.317+0000 7feccffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fecd806b170 0x7fecd806f530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59226/0 (socket says 172.21.15.97:59226) 2023-07-06T03:57:51.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.317+0000 7feccffff700 1 -- 172.21.15.97:0/126690053 learned_addr learned my addr 172.21.15.97:0/126690053 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:51.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.318+0000 7fecd6ffd700 1 -- 172.21.15.97:0/126690053 >> v1:172.21.15.146:6789/0 conn(0x7fecd806e500 legacy=0x7fecd80cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:51.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.318+0000 7fecd6ffd700 1 -- 172.21.15.97:0/126690053 >> v1:172.21.15.97:6789/0 conn(0x7fecd806b170 legacy=0x7fecd806f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.318+0000 7fecd67fc700 1 -- 172.21.15.97:0/126690053 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222581025 0 0) 0x7fecd806bcf0 con 0x7fecd806b170 2023-07-06T03:57:51.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.318+0000 7fecd6ffd700 1 -- 172.21.15.97:0/126690053 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd806ba10 con 0x7fecd806f160 2023-07-06T03:57:51.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.318+0000 7fecd6ffd700 1 --2- 172.21.15.97:0/126690053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd806f160 0x7fecd80cdf30 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009820 tx=0x7fecc8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5db085846cb935 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:51.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.318+0000 7fecd67fc700 1 -- 172.21.15.97:0/126690053 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8005e30 con 0x7fecd806f160 2023-07-06T03:57:51.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.318+0000 7fecd67fc700 1 -- 172.21.15.97:0/126690053 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc8004030 con 0x7fecd806f160 2023-07-06T03:57:51.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.318+0000 7fecd67fc700 1 -- 172.21.15.97:0/126690053 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc80043c0 con 0x7fecd806f160 2023-07-06T03:57:51.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.319+0000 7fecdd3ff700 1 -- 172.21.15.97:0/126690053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd806f160 msgr2=0x7fecd80cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.319+0000 7fecdd3ff700 1 --2- 172.21.15.97:0/126690053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd806f160 0x7fecd80cdf30 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009820 tx=0x7fecc8009b30 comp rx=0 tx=0).stop 2023-07-06T03:57:51.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.319+0000 7fecdd3ff700 1 -- 172.21.15.97:0/126690053 shutdown_connections 2023-07-06T03:57:51.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.319+0000 7fecdd3ff700 1 --2- 172.21.15.97:0/126690053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd806f160 0x7fecd80cdf30 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.319+0000 7fecdd3ff700 1 -- 172.21.15.97:0/126690053 >> 172.21.15.97:0/126690053 conn(0x7fecd8066620 msgr2=0x7fecd8099f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:51.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.319+0000 7fecdd3ff700 1 -- 172.21.15.97:0/126690053 shutdown_connections 2023-07-06T03:57:51.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.319+0000 7fecdd3ff700 1 -- 172.21.15.97:0/126690053 wait complete. 2023-07-06T03:57:51.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.321+0000 7fecdd3ff700 1 Processor -- start 2023-07-06T03:57:51.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.321+0000 7fecdd3ff700 1 -- start start 2023-07-06T03:57:51.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecdd3ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fecd806b170 0x7fecd8151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:51.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecdd3ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecd806e500 0x7fecd81560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:51.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecdd3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd806f160 0x7fecd81e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:51.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecdd3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd8152400 con 0x7fecd806f160 2023-07-06T03:57:51.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecdd3ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd80d1a40 con 0x7fecd806e500 2023-07-06T03:57:51.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecdd3ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd81e60f0 con 0x7fecd806b170 2023-07-06T03:57:51.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd806f160 0x7fecd81e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:51.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd806f160 0x7fecd81e6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37556/0 (socket says 172.21.15.97:37556) 2023-07-06T03:57:51.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecd77fe700 1 -- 172.21.15.97:0/1058927734 learned_addr learned my addr 172.21.15.97:0/1058927734 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:51.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7feccffff700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecd806e500 0x7fecd81560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:51.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecd6ffd700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fecd806b170 0x7fecd8151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:51.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecd77fe700 1 -- 172.21.15.97:0/1058927734 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fecd806b170 msgr2=0x7fecd8151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecd77fe700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fecd806b170 0x7fecd8151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecd77fe700 1 -- 172.21.15.97:0/1058927734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecd806e500 msgr2=0x7fecd81560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecd77fe700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecd806e500 0x7fecd81560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.322+0000 7fecd77fe700 1 -- 172.21.15.97:0/1058927734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc8009430 con 0x7fecd806f160 2023-07-06T03:57:51.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.323+0000 7fecd77fe700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd806f160 0x7fecd81e6eb0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fecd000cc10 tx=0x7fecd000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:51.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.323+0000 7feccf7fe700 1 -- 172.21.15.97:0/1058927734 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd000aa60 con 0x7fecd806f160 2023-07-06T03:57:51.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.323+0000 7feccf7fe700 1 -- 172.21.15.97:0/1058927734 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd000abc0 con 0x7fecd806f160 2023-07-06T03:57:51.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.323+0000 7fecdd3ff700 1 -- 172.21.15.97:0/1058927734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd81e63d0 con 0x7fecd806f160 2023-07-06T03:57:51.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.323+0000 7feccf7fe700 1 -- 172.21.15.97:0/1058927734 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0012420 con 0x7fecd806f160 2023-07-06T03:57:51.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.323+0000 7fecdd3ff700 1 -- 172.21.15.97:0/1058927734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd81e6990 con 0x7fecd806f160 2023-07-06T03:57:51.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.324+0000 7fecdd3ff700 1 -- 172.21.15.97:0/1058927734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd8071a80 con 0x7fecd806f160 2023-07-06T03:57:51.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.325+0000 7feccf7fe700 1 -- 172.21.15.97:0/1058927734 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fecd0012580 con 0x7fecd806f160 2023-07-06T03:57:51.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.326+0000 7feccf7fe700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fecb806ace0 0x7fecb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:51.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.326+0000 7feccf7fe700 1 -- 172.21.15.97:0/1058927734 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fecd0089760 con 0x7fecd806f160 2023-07-06T03:57:51.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.326+0000 7fecd6ffd700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fecb806ace0 0x7fecb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:51.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.327+0000 7fecd6ffd700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fecb806ace0 0x7fecb806d190 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fecc8000c00 tx=0x7fecc8008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:51.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.330+0000 7feccf7fe700 1 -- 172.21.15.97:0/1058927734 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fecd00547e0 con 0x7fecd806f160 2023-07-06T03:57:51.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.568+0000 7fecdd3ff700 1 -- 172.21.15.97:0/1058927734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fecd81e6bb0 con 0x7fecd806f160 2023-07-06T03:57:51.570 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-07-06T03:57:51.568+0000 7fb463ff4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T03:57:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.573+0000 7feccf7fe700 1 -- 172.21.15.97:0/1058927734 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) v1 ==== 161+0+0 (secure 0 0 0) 0x7fecd0057e00 con 0x7fecd806f160 2023-07-06T03:57:51.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 -- 172.21.15.97:0/1058927734 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fecb806ace0 msgr2=0x7fecb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fecb806ace0 0x7fecb806d190 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fecc8000c00 tx=0x7fecc8008070 comp rx=0 tx=0).stop 2023-07-06T03:57:51.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 -- 172.21.15.97:0/1058927734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd806f160 msgr2=0x7fecd81e6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd806f160 0x7fecd81e6eb0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fecd000cc10 tx=0x7fecd000cf20 comp rx=0 tx=0).stop 2023-07-06T03:57:51.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 -- 172.21.15.97:0/1058927734 shutdown_connections 2023-07-06T03:57:51.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fecb806ace0 0x7fecb806d190 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd806f160 0x7fecd81e6eb0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fecd806b170 0x7fecd8151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 --2- 172.21.15.97:0/1058927734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecd806e500 0x7fecd81560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 -- 172.21.15.97:0/1058927734 >> 172.21.15.97:0/1058927734 conn(0x7fecd8066620 msgr2=0x7fecd806f530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:51.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 -- 172.21.15.97:0/1058927734 shutdown_connections 2023-07-06T03:57:51.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.576+0000 7fecdd3ff700 1 -- 172.21.15.97:0/1058927734 wait complete. 2023-07-06T03:57:51.579 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T03:57:51.585 INFO:tasks.ceph.mds.c.smithi097.stderr: -6522> 2023-07-06T03:56:09.052+0000 7f880a1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T03:57:51.596 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T03:57:51.636 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-07-06T03:57:51.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.794+0000 7f3acb5ff700 1 Processor -- start 2023-07-06T03:57:51.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.795+0000 7f3acb5ff700 1 -- start start 2023-07-06T03:57:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.796+0000 7f3acb5ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ac40cf7e0 0x7f3ac40d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.796+0000 7f3acb5ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3ac406b380 0x7f3ac40cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.796+0000 7f3acb5ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac406b7f0 con 0x7f3ac40cd490 2023-07-06T03:57:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.796+0000 7f3acb5ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40d2460 con 0x7f3ac40cf7e0 2023-07-06T03:57:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.796+0000 7f3acb5ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40d00f0 con 0x7f3ac406b380 2023-07-06T03:57:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.796+0000 7f3ac939b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ac40cf7e0 0x7f3ac40d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.796+0000 7f3ac9b9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3ac40cd490 0x7f3ac40b5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59254/0 (socket says 172.21.15.97:59254) 2023-07-06T03:57:51.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.796+0000 7f3ac9b9c700 1 -- 172.21.15.97:0/1255431828 learned_addr learned my addr 172.21.15.97:0/1255431828 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:51.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.796+0000 7f3ac8b9a700 1 --2- 172.21.15.97:0/1255431828 >> v2:172.21.15.146:3300/0 conn(0x7f3ac406b380 0x7f3ac40cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:51.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.797+0000 7f3ac939b700 1 -- 172.21.15.97:0/1255431828 >> v2:172.21.15.146:3300/0 conn(0x7f3ac406b380 msgr2=0x7f3ac40cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:51.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.797+0000 7f3ac939b700 1 --2- 172.21.15.97:0/1255431828 >> v2:172.21.15.146:3300/0 conn(0x7f3ac406b380 0x7f3ac40cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.797+0000 7f3ac939b700 1 -- 172.21.15.97:0/1255431828 >> v1:172.21.15.97:6789/0 conn(0x7f3ac40cd490 legacy=0x7f3ac40b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.797+0000 7f3ac939b700 1 -- 172.21.15.97:0/1255431828 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac40d1950 con 0x7f3ac40cf7e0 2023-07-06T03:57:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.797+0000 7f3abbfff700 1 -- 172.21.15.97:0/1255431828 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456208629 0 0) 0x7f3abc001490 con 0x7f3ac40cd490 2023-07-06T03:57:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.797+0000 7f3ac939b700 1 --2- 172.21.15.97:0/1255431828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac40cf7e0 0x7f3ac40d1360 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40cf7b0 tx=0x7f3ac0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1f13187bc47280 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.797+0000 7f3abbfff700 1 -- 172.21.15.97:0/1255431828 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0011830 con 0x7f3ac40cf7e0 2023-07-06T03:57:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.797+0000 7f3abbfff700 1 -- 172.21.15.97:0/1255431828 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac0011990 con 0x7f3ac40cf7e0 2023-07-06T03:57:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.797+0000 7f3abbfff700 1 -- 172.21.15.97:0/1255431828 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0011d20 con 0x7f3ac40cf7e0 2023-07-06T03:57:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.798+0000 7f3acb5ff700 1 -- 172.21.15.97:0/1255431828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac40cf7e0 msgr2=0x7f3ac40d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.798+0000 7f3acb5ff700 1 --2- 172.21.15.97:0/1255431828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac40cf7e0 0x7f3ac40d1360 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40cf7b0 tx=0x7f3ac0009b20 comp rx=0 tx=0).stop 2023-07-06T03:57:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.798+0000 7f3acb5ff700 1 -- 172.21.15.97:0/1255431828 shutdown_connections 2023-07-06T03:57:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.798+0000 7f3acb5ff700 1 --2- 172.21.15.97:0/1255431828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac40cf7e0 0x7f3ac40d1360 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.798+0000 7f3acb5ff700 1 --2- 172.21.15.97:0/1255431828 >> v2:172.21.15.146:3300/0 conn(0x7f3ac406b380 0x7f3ac40cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.798+0000 7f3acb5ff700 1 -- 172.21.15.97:0/1255431828 >> 172.21.15.97:0/1255431828 conn(0x7f3ac406cf30 msgr2=0x7f3ac406a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.798+0000 7f3acb5ff700 1 -- 172.21.15.97:0/1255431828 shutdown_connections 2023-07-06T03:57:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.798+0000 7f3acb5ff700 1 -- 172.21.15.97:0/1255431828 wait complete. 2023-07-06T03:57:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.799+0000 7f3acb5ff700 1 Processor -- start 2023-07-06T03:57:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.800+0000 7f3acb5ff700 1 -- start start 2023-07-06T03:57:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.800+0000 7f3acb5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac406b380 0x7f3ac41f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.800+0000 7f3acb5ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ac40cd490 0x7f3ac40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.800+0000 7f3acb5ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ac40cf7e0 0x7f3ac40b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:51.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.800+0000 7f3acb5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac41577b0 con 0x7f3ac406b380 2023-07-06T03:57:51.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.800+0000 7f3acb5ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac4156e80 con 0x7f3ac40cd490 2023-07-06T03:57:51.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.800+0000 7f3acb5ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac4157160 con 0x7f3ac40cf7e0 2023-07-06T03:57:51.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac939b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac406b380 0x7f3ac41f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:51.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac939b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac406b380 0x7f3ac41f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37596/0 (socket says 172.21.15.97:37596) 2023-07-06T03:57:51.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac939b700 1 -- 172.21.15.97:0/2291301382 learned_addr learned my addr 172.21.15.97:0/2291301382 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:51.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac9b9c700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ac40cf7e0 0x7f3ac40b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:51.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac8b9a700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ac40cd490 0x7f3ac40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:51.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac939b700 1 -- 172.21.15.97:0/2291301382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ac40cf7e0 msgr2=0x7f3ac40b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac939b700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ac40cf7e0 0x7f3ac40b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac939b700 1 -- 172.21.15.97:0/2291301382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ac40cd490 msgr2=0x7f3ac40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:51.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac939b700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ac40cd490 0x7f3ac40b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:51.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac939b700 1 -- 172.21.15.97:0/2291301382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac00094a0 con 0x7f3ac406b380 2023-07-06T03:57:51.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac9b9c700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ac40cf7e0 0x7f3ac40b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:51.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac8b9a700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ac40cd490 0x7f3ac40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:51.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.801+0000 7f3ac939b700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac406b380 0x7f3ac41f5480 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0014040 tx=0x7f3ac0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:51.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.802+0000 7f3ab9ffb700 1 -- 172.21.15.97:0/2291301382 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00038a0 con 0x7f3ac406b380 2023-07-06T03:57:51.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.802+0000 7f3ab9ffb700 1 -- 172.21.15.97:0/2291301382 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac001e070 con 0x7f3ac406b380 2023-07-06T03:57:51.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.802+0000 7f3acb5ff700 1 -- 172.21.15.97:0/2291301382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac4157440 con 0x7f3ac406b380 2023-07-06T03:57:51.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.802+0000 7f3ab9ffb700 1 -- 172.21.15.97:0/2291301382 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0003c20 con 0x7f3ac406b380 2023-07-06T03:57:51.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.802+0000 7f3acb5ff700 1 -- 172.21.15.97:0/2291301382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac4157e20 con 0x7f3ac406b380 2023-07-06T03:57:51.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.803+0000 7f3acb5ff700 1 -- 172.21.15.97:0/2291301382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac404e340 con 0x7f3ac406b380 2023-07-06T03:57:51.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.804+0000 7f3ab9ffb700 1 -- 172.21.15.97:0/2291301382 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ac002d450 con 0x7f3ac406b380 2023-07-06T03:57:51.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.805+0000 7f3ab9ffb700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3aa806ace0 0x7f3aa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:51.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.805+0000 7f3ab9ffb700 1 -- 172.21.15.97:0/2291301382 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f3ac008a540 con 0x7f3ac406b380 2023-07-06T03:57:51.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.805+0000 7f3ac8b9a700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3aa806ace0 0x7f3aa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:51.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.806+0000 7f3ac8b9a700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3aa806ace0 0x7f3aa806d190 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3ac415c330 tx=0x7f3ab4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:51.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:51.810+0000 7f3ab9ffb700 1 -- 172.21.15.97:0/2291301382 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3ac0055560 con 0x7f3ac406b380 2023-07-06T03:57:52.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.051+0000 7f3acb5ff700 1 -- 172.21.15.97:0/2291301382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3ac40c62e0 con 0x7f3ac406b380 2023-07-06T03:57:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.574+0000 7f3ab9ffb700 1 -- 172.21.15.97:0/2291301382 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v105) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3ac0058b80 con 0x7f3ac406b380 2023-07-06T03:57:52.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 -- 172.21.15.97:0/2291301382 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3aa806ace0 msgr2=0x7f3aa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:52.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3aa806ace0 0x7f3aa806d190 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3ac415c330 tx=0x7f3ab4008040 comp rx=0 tx=0).stop 2023-07-06T03:57:52.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 -- 172.21.15.97:0/2291301382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac406b380 msgr2=0x7f3ac41f5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac406b380 0x7f3ac41f5480 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0014040 tx=0x7f3ac0004750 comp rx=0 tx=0).stop 2023-07-06T03:57:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 -- 172.21.15.97:0/2291301382 shutdown_connections 2023-07-06T03:57:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f3aa806ace0 0x7f3aa806d190 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f3ac40cf7e0 0x7f3ac40b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ac406b380 0x7f3ac41f5480 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 --2- 172.21.15.97:0/2291301382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3ac40cd490 0x7f3ac40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 -- 172.21.15.97:0/2291301382 >> 172.21.15.97:0/2291301382 conn(0x7f3ac406cf30 msgr2=0x7f3ac40b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 -- 172.21.15.97:0/2291301382 shutdown_connections 2023-07-06T03:57:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.577+0000 7f3acb5ff700 1 -- 172.21.15.97:0/2291301382 wait complete. 2023-07-06T03:57:52.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T03:57:52.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.837+0000 7f34253ff700 1 Processor -- start 2023-07-06T03:57:52.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.837+0000 7f34253ff700 1 -- start start 2023-07-06T03:57:52.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.838+0000 7f34253ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34200b21e0 0x7f3420160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:52.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.838+0000 7f34253ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f34200cd3c0 0x7f34200d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:52.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.838+0000 7f34253ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34200d1310 con 0x7f34200b47f0 2023-07-06T03:57:52.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.838+0000 7f34253ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34200b25b0 con 0x7f34200cd3c0 2023-07-06T03:57:52.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.838+0000 7f34253ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34200d09e0 con 0x7f34200b21e0 2023-07-06T03:57:52.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.839+0000 7f341f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f34200b47f0 0x7f34200b5bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59264/0 (socket says 172.21.15.97:59264) 2023-07-06T03:57:52.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.839+0000 7f341f7fe700 1 -- 172.21.15.97:0/1099753155 learned_addr learned my addr 172.21.15.97:0/1099753155 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:52.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.839+0000 7f341effd700 1 --2- 172.21.15.97:0/1099753155 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34200b21e0 0x7f3420160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:52.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.839+0000 7f341e7fc700 1 --2- 172.21.15.97:0/1099753155 >> v2:172.21.15.146:3300/0 conn(0x7f34200cd3c0 0x7f34200d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:52.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.839+0000 7f341dffb700 1 -- 172.21.15.97:0/1099753155 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384125686 0 0) 0x7f34200d1310 con 0x7f34200b47f0 2023-07-06T03:57:52.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.839+0000 7f341dffb700 1 -- 172.21.15.97:0/1099753155 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34080035b0 con 0x7f34200b47f0 2023-07-06T03:57:52.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.839+0000 7f341e7fc700 1 -- 172.21.15.97:0/1099753155 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34200b21e0 msgr2=0x7f3420160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:52.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.839+0000 7f341e7fc700 1 --2- 172.21.15.97:0/1099753155 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34200b21e0 0x7f3420160ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:52.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.839+0000 7f341e7fc700 1 -- 172.21.15.97:0/1099753155 >> v1:172.21.15.97:6789/0 conn(0x7f34200b47f0 legacy=0x7f34200b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:52.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.839+0000 7f341e7fc700 1 -- 172.21.15.97:0/1099753155 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34200d0cc0 con 0x7f34200cd3c0 2023-07-06T03:57:52.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.839+0000 7f341effd700 1 --2- 172.21.15.97:0/1099753155 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34200b21e0 0x7f3420160ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:57:52.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.840+0000 7f341e7fc700 1 --2- 172.21.15.97:0/1099753155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34200cd3c0 0x7f34200d17a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3414009840 tx=0x7f3414009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddab92b6dfaf81c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:52.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.840+0000 7f341dffb700 1 -- 172.21.15.97:0/1099753155 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414011830 con 0x7f34200cd3c0 2023-07-06T03:57:52.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.840+0000 7f341dffb700 1 -- 172.21.15.97:0/1099753155 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414011990 con 0x7f34200cd3c0 2023-07-06T03:57:52.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.840+0000 7f341dffb700 1 -- 172.21.15.97:0/1099753155 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414011d20 con 0x7f34200cd3c0 2023-07-06T03:57:52.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.840+0000 7f34253ff700 1 -- 172.21.15.97:0/1099753155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34200cd3c0 msgr2=0x7f34200d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:52.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.840+0000 7f34253ff700 1 --2- 172.21.15.97:0/1099753155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34200cd3c0 0x7f34200d17a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3414009840 tx=0x7f3414009b50 comp rx=0 tx=0).stop 2023-07-06T03:57:52.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.841+0000 7f34253ff700 1 -- 172.21.15.97:0/1099753155 shutdown_connections 2023-07-06T03:57:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.841+0000 7f34253ff700 1 --2- 172.21.15.97:0/1099753155 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34200b21e0 0x7f3420160ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.841+0000 7f34253ff700 1 --2- 172.21.15.97:0/1099753155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34200cd3c0 0x7f34200d17a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.841+0000 7f34253ff700 1 -- 172.21.15.97:0/1099753155 >> 172.21.15.97:0/1099753155 conn(0x7f34200cf7f0 msgr2=0x7f34200cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.841+0000 7f34253ff700 1 -- 172.21.15.97:0/1099753155 shutdown_connections 2023-07-06T03:57:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.841+0000 7f34253ff700 1 -- 172.21.15.97:0/1099753155 wait complete. 2023-07-06T03:57:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.842+0000 7f34253ff700 1 Processor -- start 2023-07-06T03:57:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.842+0000 7f34253ff700 1 -- start start 2023-07-06T03:57:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.843+0000 7f34253ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34200b21e0 0x7f342014c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.843+0000 7f34253ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34200b47f0 0x7f342015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.843+0000 7f34253ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34200cd3c0 0x7f342015e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.843+0000 7f34253ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342014c770 con 0x7f34200b47f0 2023-07-06T03:57:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.843+0000 7f34253ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342015d560 con 0x7f34200cd3c0 2023-07-06T03:57:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.843+0000 7f34253ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342015d840 con 0x7f34200b21e0 2023-07-06T03:57:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34200b47f0 0x7f342015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34200b47f0 0x7f342015f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37604/0 (socket says 172.21.15.97:37604) 2023-07-06T03:57:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341e7fc700 1 -- 172.21.15.97:0/3762312655 learned_addr learned my addr 172.21.15.97:0/3762312655 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341effd700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34200b21e0 0x7f342014c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341f7fe700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34200cd3c0 0x7f342015e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341e7fc700 1 -- 172.21.15.97:0/3762312655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34200b21e0 msgr2=0x7f342014c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:52.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341e7fc700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34200b21e0 0x7f342014c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:52.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341e7fc700 1 -- 172.21.15.97:0/3762312655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34200cd3c0 msgr2=0x7f342015e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:52.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341e7fc700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34200cd3c0 0x7f342015e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:52.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341e7fc700 1 -- 172.21.15.97:0/3762312655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34100094a0 con 0x7f34200b47f0 2023-07-06T03:57:52.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341f7fe700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34200cd3c0 0x7f342015e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:52.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.844+0000 7f341e7fc700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34200b47f0 0x7f342015f3b0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f34140090b0 tx=0x7f3414004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:52.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.845+0000 7f34077fe700 1 -- 172.21.15.97:0/3762312655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34140036a0 con 0x7f34200b47f0 2023-07-06T03:57:52.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.845+0000 7f34253ff700 1 -- 172.21.15.97:0/3762312655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34140094a0 con 0x7f34200b47f0 2023-07-06T03:57:52.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.845+0000 7f34077fe700 1 -- 172.21.15.97:0/3762312655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341401e070 con 0x7f34200b47f0 2023-07-06T03:57:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.845+0000 7f34077fe700 1 -- 172.21.15.97:0/3762312655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414003a40 con 0x7f34200b47f0 2023-07-06T03:57:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.845+0000 7f34253ff700 1 -- 172.21.15.97:0/3762312655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342015de00 con 0x7f34200b47f0 2023-07-06T03:57:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.846+0000 7f34253ff700 1 -- 172.21.15.97:0/3762312655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3420067010 con 0x7f34200b47f0 2023-07-06T03:57:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.847+0000 7f34077fe700 1 -- 172.21.15.97:0/3762312655 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3414003ba0 con 0x7f34200b47f0 2023-07-06T03:57:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.847+0000 7f34077fe700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340c06ad10 0x7f340c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.848+0000 7f34077fe700 1 -- 172.21.15.97:0/3762312655 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f341408a400 con 0x7f34200b47f0 2023-07-06T03:57:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.848+0000 7f341effd700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340c06ad10 0x7f340c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.848+0000 7f341effd700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340c06ad10 0x7f340c06d1c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3410009d70 tx=0x7f3410009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:52.852+0000 7f34077fe700 1 -- 172.21.15.97:0/3762312655 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3414055440 con 0x7f34200b47f0 2023-07-06T03:57:53.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.049+0000 7f34253ff700 1 -- 172.21.15.97:0/3762312655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34200b5ba0 con 0x7f34200b47f0 2023-07-06T03:57:53.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.952+0000 7f34077fe700 1 -- 172.21.15.97:0/3762312655 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v140) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3414058a60 con 0x7f34200b47f0 2023-07-06T03:57:53.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.956+0000 7f34253ff700 1 -- 172.21.15.97:0/3762312655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340c06ad10 msgr2=0x7f340c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:53.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.956+0000 7f34253ff700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340c06ad10 0x7f340c06d1c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3410009d70 tx=0x7f3410009210 comp rx=0 tx=0).stop 2023-07-06T03:57:53.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.956+0000 7f34253ff700 1 -- 172.21.15.97:0/3762312655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34200b47f0 msgr2=0x7f342015f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:53.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.956+0000 7f34253ff700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34200b47f0 0x7f342015f3b0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f34140090b0 tx=0x7f3414004750 comp rx=0 tx=0).stop 2023-07-06T03:57:53.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.956+0000 7f34253ff700 1 -- 172.21.15.97:0/3762312655 shutdown_connections 2023-07-06T03:57:53.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.956+0000 7f34253ff700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f340c06ad10 0x7f340c06d1c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:53.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.956+0000 7f34253ff700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34200cd3c0 0x7f342015e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:53.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.956+0000 7f34253ff700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34200b21e0 0x7f342014c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:53.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.956+0000 7f34253ff700 1 --2- 172.21.15.97:0/3762312655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34200b47f0 0x7f342015f3b0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:53.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.956+0000 7f34253ff700 1 -- 172.21.15.97:0/3762312655 >> 172.21.15.97:0/3762312655 conn(0x7f34200cf7f0 msgr2=0x7f34200b4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:53.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.956+0000 7f34253ff700 1 -- 172.21.15.97:0/3762312655 shutdown_connections 2023-07-06T03:57:53.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:53.957+0000 7f34253ff700 1 -- 172.21.15.97:0/3762312655 wait complete. 2023-07-06T03:57:53.961 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-07-06T03:57:53.981 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T03:57:54.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.179+0000 7fe183fff700 1 Processor -- start 2023-07-06T03:57:54.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.180+0000 7fe183fff700 1 -- start start 2023-07-06T03:57:54.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.181+0000 7fe183fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160ea0 0x7fe17c0cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:54.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.181+0000 7fe183fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c161310 con 0x7fe17c0b47f0 2023-07-06T03:57:54.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.181+0000 7fe183fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c0d1310 con 0x7fe17c0b21e0 2023-07-06T03:57:54.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.181+0000 7fe183fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0cd900 con 0x7fe17c160ea0 2023-07-06T03:57:54.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.181+0000 7fe18259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe17c0b47f0 0x7fe17c0b5bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59276/0 (socket says 172.21.15.97:59276) 2023-07-06T03:57:54.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.181+0000 7fe18259c700 1 -- 172.21.15.97:0/3409466161 learned_addr learned my addr 172.21.15.97:0/3409466161 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.181+0000 7fe18159a700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160ea0 0x7fe17c0cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.181+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302895446 0 0) 0x7fe17c161310 con 0x7fe17c0b47f0 2023-07-06T03:57:54.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.181+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1680035b0 con 0x7fe17c0b47f0 2023-07-06T03:57:54.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2814388045 0 0) 0x7fe1680035b0 con 0x7fe17c0b47f0 2023-07-06T03:57:54.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160ea0 msgr2=0x7fe17c0cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:54.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160ea0 0x7fe17c0cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:54.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 >> v1:172.21.15.146:6789/0 conn(0x7fe17c0b21e0 legacy=0x7fe17c0d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:54.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0b25b0 con 0x7fe17c0b47f0 2023-07-06T03:57:54.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fe1740019f0 con 0x7fe17c0b47f0 2023-07-06T03:57:54.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 >> v1:172.21.15.97:6789/0 conn(0x7fe17c0b47f0 legacy=0x7fe17c0b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:54.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 0x7fe168003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:54.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160ea0 0x7fe168004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:54.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe168004d90 0x7fe168005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:54.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0cd900 con 0x7fe168003a30 2023-07-06T03:57:54.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16800bac0 con 0x7fe168004d90 2023-07-06T03:57:54.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16800bc50 con 0x7fe17c160ea0 2023-07-06T03:57:54.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe181d9b700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 0x7fe168003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe18159a700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160ea0 0x7fe168004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe18259c700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe168004d90 0x7fe168005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe181d9b700 1 -- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160ea0 msgr2=0x7fe168004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe181d9b700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160ea0 0x7fe168004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe181d9b700 1 -- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe168004d90 msgr2=0x7fe168005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe181d9b700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe168004d90 0x7fe168005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe181d9b700 1 -- 172.21.15.97:0/3409466161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680035b0 con 0x7fe168003a30 2023-07-06T03:57:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.182+0000 7fe18159a700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c160ea0 0x7fe168004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.183+0000 7fe181d9b700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 0x7fe168003e40 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe1780017c0 tx=0x7fe168005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfff8fefe68ad138 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.183+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1780052d0 con 0x7fe168003a30 2023-07-06T03:57:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.183+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe178005430 con 0x7fe168003a30 2023-07-06T03:57:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.183+0000 7fe180d99700 1 -- 172.21.15.97:0/3409466161 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1780038c0 con 0x7fe168003a30 2023-07-06T03:57:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.183+0000 7fe183fff700 1 -- 172.21.15.97:0/3409466161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 msgr2=0x7fe168003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.183+0000 7fe183fff700 1 --2- 172.21.15.97:0/3409466161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 0x7fe168003e40 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe1780017c0 tx=0x7fe168005880 comp rx=0 tx=0).stop 2023-07-06T03:57:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.183+0000 7fe18259c700 1 -- 172.21.15.97:0/3409466161 reap_dead start 2023-07-06T03:57:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.183+0000 7fe183fff700 1 -- 172.21.15.97:0/3409466161 shutdown_connections 2023-07-06T03:57:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.184+0000 7fe183fff700 1 -- 172.21.15.97:0/3409466161 >> 172.21.15.97:0/3409466161 conn(0x7fe17c0cf7f0 msgr2=0x7fe17c0cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.184+0000 7fe183fff700 1 -- 172.21.15.97:0/3409466161 shutdown_connections 2023-07-06T03:57:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.184+0000 7fe183fff700 1 -- 172.21.15.97:0/3409466161 wait complete. 2023-07-06T03:57:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.185+0000 7fe183fff700 1 Processor -- start 2023-07-06T03:57:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.186+0000 7fe183fff700 1 -- start start 2023-07-06T03:57:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.186+0000 7fe183fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 0x7fe17c1e7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.186+0000 7fe183fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17c1544f0 0x7fe17c15e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.186+0000 7fe183fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c15ee30 0x7fe17c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.186+0000 7fe183fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c1e6ca0 con 0x7fe168003a30 2023-07-06T03:57:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.186+0000 7fe183fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c1e8280 con 0x7fe17c1544f0 2023-07-06T03:57:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.186+0000 7fe183fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c1e6020 con 0x7fe17c15ee30 2023-07-06T03:57:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe181d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 0x7fe17c1e7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe181d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 0x7fe17c1e7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37618/0 (socket says 172.21.15.97:37618) 2023-07-06T03:57:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe181d9b700 1 -- 172.21.15.97:0/1206533837 learned_addr learned my addr 172.21.15.97:0/1206533837 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe18159a700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17c1544f0 0x7fe17c15e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe18259c700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c15ee30 0x7fe17c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe181d9b700 1 -- 172.21.15.97:0/1206533837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c15ee30 msgr2=0x7fe17c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe181d9b700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c15ee30 0x7fe17c159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe181d9b700 1 -- 172.21.15.97:0/1206533837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17c1544f0 msgr2=0x7fe17c15e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe181d9b700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17c1544f0 0x7fe17c15e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe181d9b700 1 -- 172.21.15.97:0/1206533837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c1e6300 con 0x7fe168003a30 2023-07-06T03:57:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe181d9b700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 0x7fe17c1e7d40 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fe178000c00 tx=0x7fe178001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe1727fc700 1 -- 172.21.15.97:0/1206533837 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178007240 con 0x7fe168003a30 2023-07-06T03:57:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe1727fc700 1 -- 172.21.15.97:0/1206533837 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1780073a0 con 0x7fe168003a30 2023-07-06T03:57:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.187+0000 7fe183fff700 1 -- 172.21.15.97:0/1206533837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c1e65e0 con 0x7fe168003a30 2023-07-06T03:57:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.188+0000 7fe1727fc700 1 -- 172.21.15.97:0/1206533837 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178001290 con 0x7fe168003a30 2023-07-06T03:57:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.188+0000 7fe183fff700 1 -- 172.21.15.97:0/1206533837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c159db0 con 0x7fe168003a30 2023-07-06T03:57:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.189+0000 7fe183fff700 1 -- 172.21.15.97:0/1206533837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c04e340 con 0x7fe168003a30 2023-07-06T03:57:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.189+0000 7fe1727fc700 1 -- 172.21.15.97:0/1206533837 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe178003680 con 0x7fe168003a30 2023-07-06T03:57:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.190+0000 7fe1727fc700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16006ace0 0x7fe16006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:54.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.191+0000 7fe1727fc700 1 -- 172.21.15.97:0/1206533837 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5710+0+0 (secure 0 0 0) 0x7fe17808a300 con 0x7fe168003a30 2023-07-06T03:57:54.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.191+0000 7fe18159a700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16006ace0 0x7fe16006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:54.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.191+0000 7fe18159a700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16006ace0 0x7fe16006d190 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe17c1f5b50 tx=0x7fe16c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:54.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.195+0000 7fe1727fc700 1 -- 172.21.15.97:0/1206533837 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe178055620 con 0x7fe168003a30 2023-07-06T03:57:54.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.389+0000 7fe183fff700 1 -- 172.21.15.97:0/1206533837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe17c1e6f10 con 0x7fe168003a30 2023-07-06T03:57:54.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.960+0000 7fe1727fc700 1 -- 172.21.15.97:0/1206533837 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v141) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe178058c40 con 0x7fe168003a30 2023-07-06T03:57:54.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 -- 172.21.15.97:0/1206533837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16006ace0 msgr2=0x7fe16006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:54.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16006ace0 0x7fe16006d190 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe17c1f5b50 tx=0x7fe16c008040 comp rx=0 tx=0).stop 2023-07-06T03:57:54.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 -- 172.21.15.97:0/1206533837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 msgr2=0x7fe17c1e7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:54.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 0x7fe17c1e7d40 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fe178000c00 tx=0x7fe178001120 comp rx=0 tx=0).stop 2023-07-06T03:57:54.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 -- 172.21.15.97:0/1206533837 shutdown_connections 2023-07-06T03:57:54.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe16006ace0 0x7fe16006d190 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:54.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe17c15ee30 0x7fe17c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:54.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe168003a30 0x7fe17c1e7d40 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:54.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 --2- 172.21.15.97:0/1206533837 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe17c1544f0 0x7fe17c15e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:54.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 -- 172.21.15.97:0/1206533837 >> 172.21.15.97:0/1206533837 conn(0x7fe17c0cf7f0 msgr2=0x7fe17c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:54.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 -- 172.21.15.97:0/1206533837 shutdown_connections 2023-07-06T03:57:54.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:54.964+0000 7fe183fff700 1 -- 172.21.15.97:0/1206533837 wait complete. 2023-07-06T03:57:54.969 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-07-06T03:57:54.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:57:54.990 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T03:57:55.107 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2023-07-06T03:57:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2023-07-06T03:57:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:57:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2023-07-06T03:57:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2023-07-06T03:57:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2023-07-06T03:57:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2023-07-06T03:57:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2023-07-06T03:57:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2023-07-06T03:57:55.110 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2023-07-06T03:57:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:57:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2023-07-06T03:57:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:57:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:57:55.111 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2023-07-06T03:57:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2023-07-06T03:57:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2023-07-06T03:57:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:57:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:57:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2023-07-06T03:57:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:57:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2023-07-06T03:57:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:57:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:57:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2023-07-06T03:57:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:57:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2023-07-06T03:57:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2023-07-06T03:57:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2023-07-06T03:57:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2023-07-06T03:57:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2023-07-06T03:57:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:57:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:57:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.115 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2023-07-06T03:57:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2023-07-06T03:57:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2023-07-06T03:57:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2023-07-06T03:57:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:57:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2023-07-06T03:57:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2023-07-06T03:57:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2023-07-06T03:57:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2023-07-06T03:57:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2023-07-06T03:57:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2023-07-06T03:57:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2023-07-06T03:57:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:57:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2023-07-06T03:57:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2023-07-06T03:57:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:57:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2023-07-06T03:57:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2023-07-06T03:57:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2023-07-06T03:57:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2023-07-06T03:57:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2023-07-06T03:57:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2023-07-06T03:57:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2023-07-06T03:57:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2023-07-06T03:57:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2023-07-06T03:57:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2023-07-06T03:57:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:57:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2023-07-06T03:57:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2023-07-06T03:57:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:57:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2023-07-06T03:57:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:57:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:57:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2023-07-06T03:57:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:57:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.121 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2023-07-06T03:57:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:57:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2023-07-06T03:57:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:57:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:57:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:57:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:57:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2023-07-06T03:57:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.122 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2023-07-06T03:57:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:57:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-07-06T03:57:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2023-07-06T03:57:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-07-06T03:57:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2023-07-06T03:57:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2023-07-06T03:57:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:57:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2023-07-06T03:57:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2023-07-06T03:57:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:57:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:57:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:57:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:57:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:57:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:57:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:57:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2023-07-06T03:57:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:57:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:57:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2023-07-06T03:57:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:55.125 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2023-07-06T03:57:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2023-07-06T03:57:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2023-07-06T03:57:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2023-07-06T03:57:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:57:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:57:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2023-07-06T03:57:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2023-07-06T03:57:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:57:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2023-07-06T03:57:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:57:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:57:55.127 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2023-07-06T03:57:55.127 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2023-07-06T03:57:55.127 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2023-07-06T03:57:55.127 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-07-06T03:57:55.127 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-07-06T03:57:55.127 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-07-06T03:57:55.128 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-07-06T03:57:55.128 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2023-07-06T03:57:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2023-07-06T03:57:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2023-07-06T03:57:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-07-06T03:57:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2023-07-06T03:57:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2023-07-06T03:57:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2023-07-06T03:57:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2023-07-06T03:57:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-07-06T03:57:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-07-06T03:57:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-07-06T03:57:55.129 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-07-06T03:57:55.130 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2023-07-06T03:57:55.130 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2023-07-06T03:57:55.130 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T03:57:55.130 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2023-07-06T03:57:55.130 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.130 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2023-07-06T03:57:55.131 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.131 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2023-07-06T03:57:55.131 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.131 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2023-07-06T03:57:55.131 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2023-07-06T03:57:55.131 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.131 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2023-07-06T03:57:55.131 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2023-07-06T03:57:55.132 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.132 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T03:57:55.132 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2023-07-06T03:57:55.132 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.132 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T03:57:55.132 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T03:57:55.133 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.133 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2023-07-06T03:57:55.133 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.133 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2023-07-06T03:57:55.133 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2023-07-06T03:57:55.133 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2023-07-06T03:57:55.133 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2023-07-06T03:57:55.134 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2023-07-06T03:57:55.134 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2023-07-06T03:57:55.134 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2023-07-06T03:57:55.134 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T03:57:55.134 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T03:57:55.134 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2023-07-06T03:57:55.134 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.134 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T03:57:55.135 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.135 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2023-07-06T03:57:55.135 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.135 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2023-07-06T03:57:55.135 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.135 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2023-07-06T03:57:55.135 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2023-07-06T03:57:55.135 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2023-07-06T03:57:55.136 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.97,172.21.15.146,[v2:172.21.15.146:3301,v1:172.21.15.146:6790]" 2023-07-06T03:57:55.136 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36 2023-07-06T03:57:55.136 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.136 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2023-07-06T03:57:55.136 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2023-07-06T03:57:55.136 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.136 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2023-07-06T03:57:55.137 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2023-07-06T03:57:55.137 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2023-07-06T03:57:55.137 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.137 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2023-07-06T03:57:55.137 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2023-07-06T03:57:55.137 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.137 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2023-07-06T03:57:55.138 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2023-07-06T03:57:55.138 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2023-07-06T03:57:55.138 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.138 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2023-07-06T03:57:55.138 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2023-07-06T03:57:55.138 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.138 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2023-07-06T03:57:55.138 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.139 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2023-07-06T03:57:55.139 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.139 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2023-07-06T03:57:55.139 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.139 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2023-07-06T03:57:55.139 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2023-07-06T03:57:55.139 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.139 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2023-07-06T03:57:55.140 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2023-07-06T03:57:55.140 INFO:teuthology.orchestra.run.smithi146.stdout: bdev async discard = True 2023-07-06T03:57:55.140 INFO:teuthology.orchestra.run.smithi146.stdout: bdev enable discard = True 2023-07-06T03:57:55.140 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore allocator = bitmap 2023-07-06T03:57:55.140 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2023-07-06T03:57:55.140 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2023-07-06T03:57:55.140 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 1/20 2023-07-06T03:57:55.140 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 1/20 2023-07-06T03:57:55.141 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:57:55.141 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2023-07-06T03:57:55.141 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 4/10 2023-07-06T03:57:55.141 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T03:57:55.141 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2023-07-06T03:57:55.141 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2023-07-06T03:57:55.141 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2023-07-06T03:57:55.141 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2023-07-06T03:57:55.142 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:57:55.142 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.142 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2023-07-06T03:57:55.142 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:57:55.142 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2023-07-06T03:57:55.142 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:57:55.142 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:57:55.142 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:57:55.143 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:57:55.143 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2023-07-06T03:57:55.143 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.143 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2023-07-06T03:57:55.143 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:57:55.143 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2023-07-06T03:57:55.143 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2023-07-06T03:57:55.143 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2023-07-06T03:57:55.144 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2023-07-06T03:57:55.144 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2023-07-06T03:57:55.144 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2023-07-06T03:57:55.144 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2023-07-06T03:57:55.144 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2023-07-06T03:57:55.144 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2023-07-06T03:57:55.144 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.144 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T03:57:55.145 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T03:57:55.145 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T03:57:55.145 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.145 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T03:57:55.145 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T03:57:55.145 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T03:57:55.145 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.146 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2023-07-06T03:57:55.146 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2023-07-06T03:57:55.146 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.146 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2023-07-06T03:57:55.146 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2023-07-06T03:57:55.146 INFO:teuthology.orchestra.run.smithi146.stdout: 2023-07-06T03:57:55.146 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2023-07-06T03:57:55.147 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2023-07-06T03:57:55.147 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2023-07-06T03:57:55.147 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2023-07-06T03:57:55.147 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T03:57:55.147 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T03:57:55.147 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2023-07-06T03:57:55.147 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2023-07-06T03:57:55.148 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:57:55.148 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2023-07-06T03:57:55.148 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:57:55.148 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:57:55.148 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2023-07-06T03:57:55.148 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2023-07-06T03:57:55.148 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2023-07-06T03:57:55.148 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2023-07-06T03:57:55.149 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2023-07-06T03:57:55.149 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2023-07-06T03:57:55.149 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2023-07-06T03:57:55.149 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2023-07-06T03:57:55.149 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2023-07-06T03:57:55.149 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2023-07-06T03:57:55.149 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2023-07-06T03:57:55.150 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2023-07-06T03:57:55.150 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2023-07-06T03:57:55.150 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2023-07-06T03:57:55.150 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2023-07-06T03:57:55.150 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2023-07-06T03:57:55.150 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2023-07-06T03:57:55.150 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2023-07-06T03:57:55.151 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2023-07-06T03:57:55.152 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T03:57:55.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.347+0000 7fb3b6bff700 1 Processor -- start 2023-07-06T03:57:55.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.348+0000 7fb3b6bff700 1 -- start start 2023-07-06T03:57:55.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.349+0000 7fb3b6bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00b1e50 0x7fb3b00b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:55.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.349+0000 7fb3b6bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b015cc30 con 0x7fb3b00d28a0 2023-07-06T03:57:55.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.349+0000 7fb3b6bff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b00ce190 con 0x7fb3b00d0990 2023-07-06T03:57:55.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.349+0000 7fb3b6bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00cee10 con 0x7fb3b00b1e50 2023-07-06T03:57:55.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.349+0000 7fb3b519c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb3b00d28a0 0x7fb3b00b2c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59286/0 (socket says 172.21.15.97:59286) 2023-07-06T03:57:55.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.349+0000 7fb3b519c700 1 -- 172.21.15.97:0/3864918732 learned_addr learned my addr 172.21.15.97:0/3864918732 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:55.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.349+0000 7fb3affff700 1 --2- 172.21.15.97:0/3864918732 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00b1e50 0x7fb3b00b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:55.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.349+0000 7fb3af7fe700 1 -- 172.21.15.97:0/3864918732 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765161340 0 0) 0x7fb3b015cc30 con 0x7fb3b00d28a0 2023-07-06T03:57:55.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.349+0000 7fb3af7fe700 1 -- 172.21.15.97:0/3864918732 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3a00035b0 con 0x7fb3b00d28a0 2023-07-06T03:57:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.350+0000 7fb3af7fe700 1 -- 172.21.15.97:0/3864918732 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020086305 0 0) 0x7fb3b00ce190 con 0x7fb3b00d0990 2023-07-06T03:57:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.350+0000 7fb3af7fe700 1 -- 172.21.15.97:0/3864918732 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b015cc30 con 0x7fb3b00d0990 2023-07-06T03:57:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.350+0000 7fb3affff700 1 -- 172.21.15.97:0/3864918732 >> v1:172.21.15.146:6789/0 conn(0x7fb3b00d0990 legacy=0x7fb3b00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.350+0000 7fb3affff700 1 -- 172.21.15.97:0/3864918732 >> v1:172.21.15.97:6789/0 conn(0x7fb3b00d28a0 legacy=0x7fb3b00b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.350+0000 7fb3affff700 1 -- 172.21.15.97:0/3864918732 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00ce470 con 0x7fb3b00b1e50 2023-07-06T03:57:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.350+0000 7fb3affff700 1 --2- 172.21.15.97:0/3864918732 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00b1e50 0x7fb3b00b26f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009870 tx=0x7fb39c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baab046978b40fa9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.350+0000 7fb3af7fe700 1 -- 172.21.15.97:0/3864918732 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c011830 con 0x7fb3b00b1e50 2023-07-06T03:57:55.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.350+0000 7fb3af7fe700 1 -- 172.21.15.97:0/3864918732 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39c011990 con 0x7fb3b00b1e50 2023-07-06T03:57:55.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.350+0000 7fb3af7fe700 1 -- 172.21.15.97:0/3864918732 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c011d20 con 0x7fb3b00b1e50 2023-07-06T03:57:55.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.351+0000 7fb3b6bff700 1 -- 172.21.15.97:0/3864918732 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00b1e50 msgr2=0x7fb3b00b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.351+0000 7fb3b6bff700 1 --2- 172.21.15.97:0/3864918732 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00b1e50 0x7fb3b00b26f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009870 tx=0x7fb39c009b80 comp rx=0 tx=0).stop 2023-07-06T03:57:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.351+0000 7fb3b6bff700 1 -- 172.21.15.97:0/3864918732 shutdown_connections 2023-07-06T03:57:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.351+0000 7fb3b6bff700 1 --2- 172.21.15.97:0/3864918732 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00b1e50 0x7fb3b00b26f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.351+0000 7fb3b6bff700 1 -- 172.21.15.97:0/3864918732 >> 172.21.15.97:0/3864918732 conn(0x7fb3b009c120 msgr2=0x7fb3b00d2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.351+0000 7fb3b6bff700 1 -- 172.21.15.97:0/3864918732 shutdown_connections 2023-07-06T03:57:55.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.351+0000 7fb3b6bff700 1 -- 172.21.15.97:0/3864918732 wait complete. 2023-07-06T03:57:55.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.353+0000 7fb3b6bff700 1 Processor -- start 2023-07-06T03:57:55.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.353+0000 7fb3b6bff700 1 -- start start 2023-07-06T03:57:55.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b6bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00b1e50 0x7fb3b01524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:55.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b6bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b00d0990 0x7fb3b00b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b6bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00d28a0 0x7fb3b00b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b6bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00ca230 con 0x7fb3b00b1e50 2023-07-06T03:57:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b6bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00c70a0 con 0x7fb3b00d0990 2023-07-06T03:57:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b6bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00cca60 con 0x7fb3b00d28a0 2023-07-06T03:57:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b499b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00b1e50 0x7fb3b01524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b499b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00b1e50 0x7fb3b01524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37622/0 (socket says 172.21.15.97:37622) 2023-07-06T03:57:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b499b700 1 -- 172.21.15.97:0/1157740824 learned_addr learned my addr 172.21.15.97:0/1157740824 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3affff700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b00d0990 0x7fb3b00b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b519c700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00d28a0 0x7fb3b00b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b499b700 1 -- 172.21.15.97:0/1157740824 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00d28a0 msgr2=0x7fb3b00b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b499b700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00d28a0 0x7fb3b00b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b499b700 1 -- 172.21.15.97:0/1157740824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b00d0990 msgr2=0x7fb3b00b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b499b700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b00d0990 0x7fb3b00b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b499b700 1 -- 172.21.15.97:0/1157740824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c0094a0 con 0x7fb3b00b1e50 2023-07-06T03:57:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.354+0000 7fb3b519c700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00d28a0 0x7fb3b00b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.355+0000 7fb3b499b700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00b1e50 0x7fb3b01524e0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c500 tx=0x7fb3a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.355+0000 7fb3ad7fa700 1 -- 172.21.15.97:0/1157740824 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40052d0 con 0x7fb3b00b1e50 2023-07-06T03:57:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.355+0000 7fb3ad7fa700 1 -- 172.21.15.97:0/1157740824 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a4005450 con 0x7fb3b00b1e50 2023-07-06T03:57:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.355+0000 7fb3b6bff700 1 -- 172.21.15.97:0/1157740824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b0152a80 con 0x7fb3b00b1e50 2023-07-06T03:57:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.355+0000 7fb3ad7fa700 1 -- 172.21.15.97:0/1157740824 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40038c0 con 0x7fb3b00b1e50 2023-07-06T03:57:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.355+0000 7fb3b6bff700 1 -- 172.21.15.97:0/1157740824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b00c7850 con 0x7fb3b00b1e50 2023-07-06T03:57:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.356+0000 7fb3b6bff700 1 -- 172.21.15.97:0/1157740824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b004e340 con 0x7fb3b00b1e50 2023-07-06T03:57:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.357+0000 7fb3ad7fa700 1 -- 172.21.15.97:0/1157740824 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb3a4003a20 con 0x7fb3b00b1e50 2023-07-06T03:57:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.358+0000 7fb3ad7fa700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39406ace0 0x7fb39406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.358+0000 7fb3ad7fa700 1 -- 172.21.15.97:0/1157740824 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb3a40886e0 con 0x7fb3b00b1e50 2023-07-06T03:57:55.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.358+0000 7fb3affff700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39406ace0 0x7fb39406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:55.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.359+0000 7fb3affff700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39406ace0 0x7fb39406d190 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0097c0 tx=0x7fb39c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:55.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.362+0000 7fb3ad7fa700 1 -- 172.21.15.97:0/1157740824 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb3a4053a10 con 0x7fb3b00b1e50 2023-07-06T03:57:55.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.557+0000 7fb3b6bff700 1 -- 172.21.15.97:0/1157740824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb3b008c5a0 con 0x7fb3b00b1e50 2023-07-06T03:57:55.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.561+0000 7fb3ad7fa700 1 -- 172.21.15.97:0/1157740824 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb3a4057030 con 0x7fb3b00b1e50 2023-07-06T03:57:55.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.563+0000 7fb3b6bff700 1 -- 172.21.15.97:0/1157740824 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39406ace0 msgr2=0x7fb39406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:55.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.563+0000 7fb3b6bff700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39406ace0 0x7fb39406d190 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0097c0 tx=0x7fb39c01c040 comp rx=0 tx=0).stop 2023-07-06T03:57:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.563+0000 7fb3b6bff700 1 -- 172.21.15.97:0/1157740824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00b1e50 msgr2=0x7fb3b01524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.563+0000 7fb3b6bff700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00b1e50 0x7fb3b01524e0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c500 tx=0x7fb3a400c8c0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.564+0000 7fb3b6bff700 1 -- 172.21.15.97:0/1157740824 shutdown_connections 2023-07-06T03:57:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.564+0000 7fb3b6bff700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fb39406ace0 0x7fb39406d190 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.564+0000 7fb3b6bff700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fb3b00d28a0 0x7fb3b00b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.564+0000 7fb3b6bff700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00b1e50 0x7fb3b01524e0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.564+0000 7fb3b6bff700 1 --2- 172.21.15.97:0/1157740824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b00d0990 0x7fb3b00b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.564+0000 7fb3b6bff700 1 -- 172.21.15.97:0/1157740824 >> 172.21.15.97:0/1157740824 conn(0x7fb3b009c120 msgr2=0x7fb3b006a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:55.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.564+0000 7fb3b6bff700 1 -- 172.21.15.97:0/1157740824 shutdown_connections 2023-07-06T03:57:55.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.564+0000 7fb3b6bff700 1 -- 172.21.15.97:0/1157740824 wait complete. 2023-07-06T03:57:55.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2023-07-06T03:57:55.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.777+0000 7f4ee41ff700 1 Processor -- start 2023-07-06T03:57:55.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.777+0000 7f4ee41ff700 1 -- start start 2023-07-06T03:57:55.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.778+0000 7f4ee41ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4edc0702c0 0x7f4edc0cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:55.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.778+0000 7f4ee41ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4edc0cc510 0x7f4edc0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:55.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.778+0000 7f4ee41ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4edc159600 con 0x7f4edc159230 2023-07-06T03:57:55.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.778+0000 7f4ee41ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc0cd940 con 0x7f4edc0702c0 2023-07-06T03:57:55.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.779+0000 7f4ee41ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0cde80 con 0x7f4edc0cc510 2023-07-06T03:57:55.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.779+0000 7f4ee179a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4edc0702c0 0x7f4edc0cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:55.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.779+0000 7f4ee179a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4edc0702c0 0x7f4edc0cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37638/0 (socket says 172.21.15.97:37638) 2023-07-06T03:57:55.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.779+0000 7f4ee179a700 1 -- 172.21.15.97:0/695084342 learned_addr learned my addr 172.21.15.97:0/695084342 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:55.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.779+0000 7f4ee179a700 1 -- 172.21.15.97:0/695084342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4edc0cc510 msgr2=0x7f4edc0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:55.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.779+0000 7f4ee279c700 1 --2- 172.21.15.97:0/695084342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4edc0cc510 0x7f4edc0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:55.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.779+0000 7f4ee179a700 1 --2- 172.21.15.97:0/695084342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4edc0cc510 0x7f4edc0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.779+0000 7f4ee179a700 1 -- 172.21.15.97:0/695084342 >> v1:172.21.15.146:6789/0 conn(0x7f4edc159230 legacy=0x7f4edc06fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:55.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.779+0000 7f4ee179a700 1 -- 172.21.15.97:0/695084342 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc0cdba0 con 0x7f4edc0702c0 2023-07-06T03:57:55.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.779+0000 7f4ee279c700 1 --2- 172.21.15.97:0/695084342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4edc0cc510 0x7f4edc0c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:55.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.779+0000 7f4ee179a700 1 --2- 172.21.15.97:0/695084342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc0702c0 0x7f4edc0cd400 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0098f0 tx=0x7f4ecc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc0d32a31cf3bb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:55.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.780+0000 7f4ee0f99700 1 -- 172.21.15.97:0/695084342 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc004230 con 0x7f4edc0702c0 2023-07-06T03:57:55.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.780+0000 7f4ee0f99700 1 -- 172.21.15.97:0/695084342 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc004390 con 0x7f4edc0702c0 2023-07-06T03:57:55.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.780+0000 7f4ee0f99700 1 -- 172.21.15.97:0/695084342 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0036a0 con 0x7f4edc0702c0 2023-07-06T03:57:55.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.780+0000 7f4ee41ff700 1 -- 172.21.15.97:0/695084342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc0702c0 msgr2=0x7f4edc0cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:55.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.780+0000 7f4ee41ff700 1 --2- 172.21.15.97:0/695084342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc0702c0 0x7f4edc0cd400 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0098f0 tx=0x7f4ecc009c00 comp rx=0 tx=0).stop 2023-07-06T03:57:55.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.780+0000 7f4ee41ff700 1 -- 172.21.15.97:0/695084342 shutdown_connections 2023-07-06T03:57:55.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.780+0000 7f4ee41ff700 1 --2- 172.21.15.97:0/695084342 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4edc0cc510 0x7f4edc0c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.780+0000 7f4ee41ff700 1 --2- 172.21.15.97:0/695084342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc0702c0 0x7f4edc0cd400 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.780+0000 7f4ee41ff700 1 -- 172.21.15.97:0/695084342 >> 172.21.15.97:0/695084342 conn(0x7f4edc06aca0 msgr2=0x7f4edc0cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:55.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.780+0000 7f4ee41ff700 1 -- 172.21.15.97:0/695084342 shutdown_connections 2023-07-06T03:57:55.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.780+0000 7f4ee41ff700 1 -- 172.21.15.97:0/695084342 wait complete. 2023-07-06T03:57:55.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.782+0000 7f4ee41ff700 1 Processor -- start 2023-07-06T03:57:55.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.782+0000 7f4ee41ff700 1 -- start start 2023-07-06T03:57:55.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.783+0000 7f4ee41ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4edc0702c0 0x7f4edc151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.783+0000 7f4ee41ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc0cc510 0x7f4edc0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.783+0000 7f4ee41ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4edc159230 0x7f4edc158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.783+0000 7f4ee41ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1574e0 con 0x7f4edc0cc510 2023-07-06T03:57:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.783+0000 7f4ee41ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0b3250 con 0x7f4edc159230 2023-07-06T03:57:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.783+0000 7f4ee41ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc158a60 con 0x7f4edc0702c0 2023-07-06T03:57:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.783+0000 7f4ee179a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc0cc510 0x7f4edc0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:55.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.783+0000 7f4ee179a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc0cc510 0x7f4edc0b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37644/0 (socket says 172.21.15.97:37644) 2023-07-06T03:57:55.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.783+0000 7f4ee179a700 1 -- 172.21.15.97:0/3031400930 learned_addr learned my addr 172.21.15.97:0/3031400930 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:55.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ee1f9b700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4edc0702c0 0x7f4edc151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:55.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ee279c700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4edc159230 0x7f4edc158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:55.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ee179a700 1 -- 172.21.15.97:0/3031400930 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4edc0702c0 msgr2=0x7f4edc151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:55.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ee179a700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4edc0702c0 0x7f4edc151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ee179a700 1 -- 172.21.15.97:0/3031400930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4edc159230 msgr2=0x7f4edc158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:55.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ee179a700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4edc159230 0x7f4edc158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:55.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ee179a700 1 -- 172.21.15.97:0/3031400930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc009550 con 0x7f4edc0cc510 2023-07-06T03:57:55.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ee179a700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc0cc510 0x7f4edc0b2d10 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009130 tx=0x7f4ecc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:55.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ed27fc700 1 -- 172.21.15.97:0/3031400930 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc003cd0 con 0x7f4edc0cc510 2023-07-06T03:57:55.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ed27fc700 1 -- 172.21.15.97:0/3031400930 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc01f070 con 0x7f4edc0cc510 2023-07-06T03:57:55.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ee41ff700 1 -- 172.21.15.97:0/3031400930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc0b1c70 con 0x7f4edc0cc510 2023-07-06T03:57:55.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.784+0000 7f4ed27fc700 1 -- 172.21.15.97:0/3031400930 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00b8a0 con 0x7f4edc0cc510 2023-07-06T03:57:55.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.785+0000 7f4ee41ff700 1 -- 172.21.15.97:0/3031400930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc0b2230 con 0x7f4edc0cc510 2023-07-06T03:57:55.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.785+0000 7f4ee41ff700 1 -- 172.21.15.97:0/3031400930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc04e340 con 0x7f4edc0cc510 2023-07-06T03:57:55.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.786+0000 7f4ed27fc700 1 -- 172.21.15.97:0/3031400930 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ecc021030 con 0x7f4edc0cc510 2023-07-06T03:57:55.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.787+0000 7f4ed27fc700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4ec006ace0 0x7f4ec006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:55.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.787+0000 7f4ed27fc700 1 -- 172.21.15.97:0/3031400930 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4ecc08a6e0 con 0x7f4edc0cc510 2023-07-06T03:57:55.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.788+0000 7f4ee1f9b700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4ec006ace0 0x7f4ec006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:55.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.788+0000 7f4ee1f9b700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4ec006ace0 0x7f4ec006d190 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009960 tx=0x7f4ed80090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:55.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:55.791+0000 7f4ed27fc700 1 -- 172.21.15.97:0/3031400930 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4ecc055bc0 con 0x7f4edc0cc510 2023-07-06T03:57:56.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.065+0000 7f4ee41ff700 1 -- 172.21.15.97:0/3031400930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) v1 -- 0x7f4edc04e340 con 0x7f4edc0cc510 2023-07-06T03:57:56.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.970+0000 7f4ed27fc700 1 -- 172.21.15.97:0/3031400930 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v643) v1 ==== 151+0+0 (secure 0 0 0) 0x7f4ecc0591e0 con 0x7f4edc0cc510 2023-07-06T03:57:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 -- 172.21.15.97:0/3031400930 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4ec006ace0 msgr2=0x7f4ec006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4ec006ace0 0x7f4ec006d190 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009960 tx=0x7f4ed80090e0 comp rx=0 tx=0).stop 2023-07-06T03:57:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 -- 172.21.15.97:0/3031400930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc0cc510 msgr2=0x7f4edc0b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc0cc510 0x7f4edc0b2d10 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009130 tx=0x7f4ecc004750 comp rx=0 tx=0).stop 2023-07-06T03:57:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 -- 172.21.15.97:0/3031400930 shutdown_connections 2023-07-06T03:57:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4ec006ace0 0x7f4ec006d190 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4edc159230 0x7f4edc158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4edc0702c0 0x7f4edc151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 --2- 172.21.15.97:0/3031400930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc0cc510 0x7f4edc0b2d10 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 -- 172.21.15.97:0/3031400930 >> 172.21.15.97:0/3031400930 conn(0x7f4edc06aca0 msgr2=0x7f4edc0ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 -- 172.21.15.97:0/3031400930 shutdown_connections 2023-07-06T03:57:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:56.973+0000 7f4ee41ff700 1 -- 172.21.15.97:0/3031400930 wait complete. 2023-07-06T03:57:56.997 INFO:tasks.cephfs_test_runner:test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) ... ERROR 2023-07-06T03:57:56.997 INFO:tasks.cephfs_test_runner: 2023-07-06T03:57:56.997 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-06T03:57:56.998 INFO:tasks.cephfs_test_runner:ERROR: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:57:56.998 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T03:57:56.998 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-06T03:57:56.998 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/test_client_limits.py", line 208, in test_client_release_bug 2023-07-06T03:57:56.998 INFO:tasks.cephfs_test_runner: self.wait_for_health("MDS_CLIENT_LATE_RELEASE", session_timeout + 10) 2023-07-06T03:57:56.999 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/ceph_test_case.py", line 184, in wait_for_health 2023-07-06T03:57:56.999 INFO:tasks.cephfs_test_runner: self.wait_until_true(seen_health_warning, timeout) 2023-07-06T03:57:56.999 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/ceph_test_case.py", line 231, in wait_until_true 2023-07-06T03:57:56.999 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-07-06T03:57:56.999 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 70s and 0 retries 2023-07-06T03:57:56.999 INFO:tasks.cephfs_test_runner: 2023-07-06T03:57:57.000 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T03:57:57.000 INFO:tasks.cephfs_test_runner:Ran 9 tests in 719.831s 2023-07-06T03:57:57.000 INFO:tasks.cephfs_test_runner: 2023-07-06T03:57:57.000 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-07-06T03:57:57.000 INFO:tasks.cephfs_test_runner: 2023-07-06T03:57:57.001 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-06T03:57:57.001 INFO:tasks.cephfs_test_runner:ERROR: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:57:57.001 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T03:57:57.001 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-06T03:57:57.001 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/test_client_limits.py", line 208, in test_client_release_bug 2023-07-06T03:57:57.002 INFO:tasks.cephfs_test_runner: self.wait_for_health("MDS_CLIENT_LATE_RELEASE", session_timeout + 10) 2023-07-06T03:57:57.002 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/ceph_test_case.py", line 184, in wait_for_health 2023-07-06T03:57:57.002 INFO:tasks.cephfs_test_runner: self.wait_until_true(seen_health_warning, timeout) 2023-07-06T03:57:57.002 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/ceph_test_case.py", line 231, in wait_until_true 2023-07-06T03:57:57.002 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-07-06T03:57:57.003 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 70s and 0 retries 2023-07-06T03:57:57.003 INFO:tasks.cephfs_test_runner: 2023-07-06T03:57:57.003 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:57:57.061 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=3d2c1309dc4b4010b2fbf9af257db847 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2023-07-06T03:57:57.063 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-07-06T03:57:57.076 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-07-06T03:57:57.085 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-07-06T03:57:57.085 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:57:57.086 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:57:57.145 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:57:57.146 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-06T03:57:57.146 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:57:57.232 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345680k,nr_inodes=4086420,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T03:57:57.232 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:57:57.233 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:57:57.284 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:57:57.284 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2023-07-06T03:57:57.284 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T03:57:57.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345688k,nr_inodes=4086422,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T03:57:57.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:57.332 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2023-07-06T03:57:57.417 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T03:57:57.418 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T03:57:57.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:57.419 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T03:57:57.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:57.490 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T03:57:57.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:57.561 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2023-07-06T03:57:57.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:57.669 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-07-06T03:57:57.717 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T03:57:57.717 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T03:57:57.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:57.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T03:57:57.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:57.789 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T03:57:57.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T03:57:57.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-07-06T03:57:57.998 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-07-06T03:57:58.008 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-06T03:57:58.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:57:58.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.203+0000 7fa54c9ff700 1 Processor -- start 2023-07-06T03:57:58.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.203+0000 7fa54c9ff700 1 -- start start 2023-07-06T03:57:58.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.204+0000 7fa54c9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa548071230 0x7fa54806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.204+0000 7fa54c9ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54815cf90 con 0x7fa54806d260 2023-07-06T03:57:58.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.204+0000 7fa54c9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54806d6a0 con 0x7fa548159480 2023-07-06T03:57:58.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.204+0000 7fa54c9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa548071680 con 0x7fa548071230 2023-07-06T03:57:58.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.204+0000 7fa53ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa548071230 0x7fa54806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.204+0000 7fa546d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa54806d260 0x7fa5480aef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59292/0 (socket says 172.21.15.97:59292) 2023-07-06T03:57:58.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.204+0000 7fa546d9d700 1 -- 172.21.15.97:0/1710114218 learned_addr learned my addr 172.21.15.97:0/1710114218 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:58.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.205+0000 7fa53ffff700 1 -- 172.21.15.97:0/1710114218 >> v1:172.21.15.146:6789/0 conn(0x7fa548159480 legacy=0x7fa54806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.205+0000 7fa53ffff700 1 -- 172.21.15.97:0/1710114218 >> v1:172.21.15.97:6789/0 conn(0x7fa54806d260 legacy=0x7fa5480aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.205+0000 7fa53ffff700 1 -- 172.21.15.97:0/1710114218 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa548159850 con 0x7fa548071230 2023-07-06T03:57:58.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.205+0000 7fa545d9b700 1 -- 172.21.15.97:0/1710114218 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836260651 0 0) 0x7fa54815cf90 con 0x7fa54806d260 2023-07-06T03:57:58.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.205+0000 7fa53ffff700 1 --2- 172.21.15.97:0/1710114218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa548071230 0x7fa54806cd20 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fa530009840 tx=0x7fa530009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79d565df6b2de2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:58.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.205+0000 7fa545d9b700 1 -- 172.21.15.97:0/1710114218 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53000f8a0 con 0x7fa548071230 2023-07-06T03:57:58.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.205+0000 7fa545d9b700 1 -- 172.21.15.97:0/1710114218 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53000fa00 con 0x7fa548071230 2023-07-06T03:57:58.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.205+0000 7fa545d9b700 1 -- 172.21.15.97:0/1710114218 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53000fd90 con 0x7fa548071230 2023-07-06T03:57:58.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.206+0000 7fa54c9ff700 1 -- 172.21.15.97:0/1710114218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa548071230 msgr2=0x7fa54806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.206+0000 7fa54c9ff700 1 --2- 172.21.15.97:0/1710114218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa548071230 0x7fa54806cd20 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fa530009840 tx=0x7fa530009b50 comp rx=0 tx=0).stop 2023-07-06T03:57:58.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.206+0000 7fa54c9ff700 1 -- 172.21.15.97:0/1710114218 shutdown_connections 2023-07-06T03:57:58.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.206+0000 7fa54c9ff700 1 --2- 172.21.15.97:0/1710114218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa548071230 0x7fa54806cd20 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.206+0000 7fa54c9ff700 1 -- 172.21.15.97:0/1710114218 >> 172.21.15.97:0/1710114218 conn(0x7fa54806ac90 msgr2=0x7fa5480ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:58.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.206+0000 7fa54c9ff700 1 -- 172.21.15.97:0/1710114218 shutdown_connections 2023-07-06T03:57:58.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.206+0000 7fa54c9ff700 1 -- 172.21.15.97:0/1710114218 wait complete. 2023-07-06T03:57:58.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.208+0000 7fa54c9ff700 1 Processor -- start 2023-07-06T03:57:58.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.208+0000 7fa54c9ff700 1 -- start start 2023-07-06T03:57:58.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.209+0000 7fa54c9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa54806d260 0x7fa54815a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.209+0000 7fa54c9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa548071230 0x7fa548155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.209+0000 7fa54c9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa548159480 0x7fa5480c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.209+0000 7fa54c9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480b2090 con 0x7fa54806d260 2023-07-06T03:57:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.209+0000 7fa54c9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54815a6b0 con 0x7fa548071230 2023-07-06T03:57:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.209+0000 7fa54c9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480b1780 con 0x7fa548159480 2023-07-06T03:57:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.209+0000 7fa54659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa54806d260 0x7fa54815a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.209+0000 7fa54659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa54806d260 0x7fa54815a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37662/0 (socket says 172.21.15.97:37662) 2023-07-06T03:57:58.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.209+0000 7fa54659c700 1 -- 172.21.15.97:0/2895060206 learned_addr learned my addr 172.21.15.97:0/2895060206 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:58.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.209+0000 7fa53ffff700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa548071230 0x7fa548155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.209+0000 7fa546d9d700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa548159480 0x7fa5480c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.210+0000 7fa54659c700 1 -- 172.21.15.97:0/2895060206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa548159480 msgr2=0x7fa5480c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.210+0000 7fa54659c700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa548159480 0x7fa5480c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.210+0000 7fa54659c700 1 -- 172.21.15.97:0/2895060206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa548071230 msgr2=0x7fa548155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.210+0000 7fa54659c700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa548071230 0x7fa548155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.210+0000 7fa54659c700 1 -- 172.21.15.97:0/2895060206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5300094a0 con 0x7fa54806d260 2023-07-06T03:57:58.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.210+0000 7fa54659c700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa54806d260 0x7fa54815a170 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fa53800c4d0 tx=0x7fa53800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:58.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.210+0000 7fa53effd700 1 -- 172.21.15.97:0/2895060206 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53800cea0 con 0x7fa54806d260 2023-07-06T03:57:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.210+0000 7fa53effd700 1 -- 172.21.15.97:0/2895060206 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5380052d0 con 0x7fa54806d260 2023-07-06T03:57:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.210+0000 7fa53effd700 1 -- 172.21.15.97:0/2895060206 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538003680 con 0x7fa54806d260 2023-07-06T03:57:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.210+0000 7fa54c9ff700 1 -- 172.21.15.97:0/2895060206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa548156460 con 0x7fa54806d260 2023-07-06T03:57:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.210+0000 7fa54c9ff700 1 -- 172.21.15.97:0/2895060206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5480b19d0 con 0x7fa54806d260 2023-07-06T03:57:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.211+0000 7fa54c9ff700 1 -- 172.21.15.97:0/2895060206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54804e340 con 0x7fa54806d260 2023-07-06T03:57:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.212+0000 7fa53effd700 1 -- 172.21.15.97:0/2895060206 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5380037e0 con 0x7fa54806d260 2023-07-06T03:57:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.213+0000 7fa53effd700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa52806ace0 0x7fa52806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.213+0000 7fa53effd700 1 -- 172.21.15.97:0/2895060206 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa5380883d0 con 0x7fa54806d260 2023-07-06T03:57:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.213+0000 7fa53ffff700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa52806ace0 0x7fa52806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.214+0000 7fa53ffff700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa52806ace0 0x7fa52806d190 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa5300090b0 tx=0x7fa53001b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.217+0000 7fa53effd700 1 -- 172.21.15.97:0/2895060206 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa538053840 con 0x7fa54806d260 2023-07-06T03:57:58.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.406+0000 7fa54c9ff700 1 -- 172.21.15.97:0/2895060206 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa54815d1e0 con 0x7fa52806ace0 2023-07-06T03:57:58.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.409+0000 7fa53effd700 1 -- 172.21.15.97:0/2895060206 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72761 (secure 0 0 0) 0x7fa53000fb70 con 0x7fa52806ace0 2023-07-06T03:57:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:58.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 -- 172.21.15.97:0/2895060206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa52806ace0 msgr2=0x7fa52806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa52806ace0 0x7fa52806d190 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa5300090b0 tx=0x7fa53001b040 comp rx=0 tx=0).stop 2023-07-06T03:57:58.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 -- 172.21.15.97:0/2895060206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa54806d260 msgr2=0x7fa54815a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa54806d260 0x7fa54815a170 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fa53800c4d0 tx=0x7fa53800c7e0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 -- 172.21.15.97:0/2895060206 shutdown_connections 2023-07-06T03:57:58.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fa52806ace0 0x7fa52806d190 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fa548159480 0x7fa5480c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa54806d260 0x7fa54815a170 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 --2- 172.21.15.97:0/2895060206 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa548071230 0x7fa548155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 -- 172.21.15.97:0/2895060206 >> 172.21.15.97:0/2895060206 conn(0x7fa54806ac90 msgr2=0x7fa5480c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:58.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 -- 172.21.15.97:0/2895060206 shutdown_connections 2023-07-06T03:57:58.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.412+0000 7fa54c9ff700 1 -- 172.21.15.97:0/2895060206 wait complete. 2023-07-06T03:57:58.417 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:57:58.433 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2023-07-06T03:57:57.052464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296116,"kb_used_data":14964,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754678604,"statfs":{"total":773094113280,"available":772790890496,"internally_reserved":0,"allocated":15323136,"data_stored":7693515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14367,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001002"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":227,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:54.961258+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:57:54.961258+0000","last_peered":"2023-07-06T03:57:54.961258+0000","last_clean":"2023-07-06T03:57:54.961258+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:57:54.961258+0000","last_undegraded":"2023-07-06T03:57:54.961258+0000","last_fullsized":"2023-07-06T03:57:54.961258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:57:34.473968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956223+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:57:53.956223+0000","last_peered":"2023-07-06T03:57:53.956223+0000","last_clean":"2023-07-06T03:57:53.956223+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:57:53.956223+0000","last_undegraded":"2023-07-06T03:57:53.956223+0000","last_fullsized":"2023-07-06T03:57:53.956223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:25.741703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":243,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:54.964259+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:57:54.964259+0000","last_peered":"2023-07-06T03:57:54.964259+0000","last_clean":"2023-07-06T03:57:54.964259+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:57:54.964259+0000","last_undegraded":"2023-07-06T03:57:54.964259+0000","last_fullsized":"2023-07-06T03:57:54.964259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:50:57.912150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956011+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:57:53.956011+0000","last_peered":"2023-07-06T03:57:53.956011+0000","last_clean":"2023-07-06T03:57:53.956011+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:57:53.956011+0000","last_undegraded":"2023-07-06T03:57:53.956011+0000","last_fullsized":"2023-07-06T03:57:53.956011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:13.711531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":377,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:55.226464+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:57:55.226464+0000","last_peered":"2023-07-06T03:57:55.226464+0000","last_clean":"2023-07-06T03:57:55.226464+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:57:55.226464+0000","last_undegraded":"2023-07-06T03:57:55.226464+0000","last_fullsized":"2023-07-06T03:57:55.226464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:19:12.557397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956489+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:57:53.956489+0000","last_peered":"2023-07-06T03:57:53.956489+0000","last_clean":"2023-07-06T03:57:53.956489+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:57:53.956489+0000","last_undegraded":"2023-07-06T03:57:53.956489+0000","last_fullsized":"2023-07-06T03:57:53.956489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:26:45.174657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956669+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:57:53.956669+0000","last_peered":"2023-07-06T03:57:53.956669+0000","last_clean":"2023-07-06T03:57:53.956669+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:57:53.956669+0000","last_undegraded":"2023-07-06T03:57:53.956669+0000","last_fullsized":"2023-07-06T03:57:53.956669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:26:18.178369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":228,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.957919+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:57:53.957919+0000","last_peered":"2023-07-06T03:57:53.957919+0000","last_clean":"2023-07-06T03:57:53.957919+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:57:53.957919+0000","last_undegraded":"2023-07-06T03:57:53.957919+0000","last_fullsized":"2023-07-06T03:57:53.957919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:51:43.180770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":234,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.957279+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:57:53.957279+0000","last_peered":"2023-07-06T03:57:53.957279+0000","last_clean":"2023-07-06T03:57:53.957279+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:57:53.957279+0000","last_undegraded":"2023-07-06T03:57:53.957279+0000","last_fullsized":"2023-07-06T03:57:53.957279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:09:40.927896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.51000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.98499999999999999,"5min":1.024,"15min":4.0570000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.67200000000000004,"15min":0.72099999999999997},"min":{"1min":0.54700000000000004,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":3.8660000000000001},"last":0.623}]},{"osd":1,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.58299999999999996,"5min":0.45400000000000001,"15min":0.34000000000000002},"max":{"1min":0.96299999999999997,"5min":1.091,"15min":2.6219999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.67800000000000005,"15min":0.76700000000000002},"min":{"1min":0.52700000000000002,"5min":0.40000000000000002,"15min":0.38800000000000001},"max":{"1min":0.94399999999999995,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.68899999999999995,"15min":0.72999999999999998},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.35199999999999998},"max":{"1min":0.92000000000000004,"5min":1.0209999999999999,"15min":3.77},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.65000000000000002,"5min":0.47999999999999998,"15min":0.45000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71899999999999997,"15min":0.71099999999999997},"min":{"1min":0.60599999999999998,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98499999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.752,"15min":0.753},"min":{"1min":0.63700000000000001,"5min":0.54500000000000004,"15min":0.43099999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.0209999999999999},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65500000000000003,"15min":0.63100000000000001},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.94699999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":0.94399999999999995,"15min":1.4590000000000001},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.72199999999999998,"15min":0.69999999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.38},"max":{"1min":1.0329999999999999,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70499999999999996,"15min":0.68899999999999995},"min":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.40000000000000002},"max":{"1min":0.997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.74299999999999999},"min":{"1min":0.53600000000000003,"5min":0.46400000000000002,"15min":0.438},"max":{"1min":1.052,"5min":1.052,"15min":1.5349999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.502,"5min":0.49199999999999999,"15min":0.42299999999999999},"max":{"1min":1.0169999999999999,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.48299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":810526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.74299999999999999,"15min":0.69899999999999995},"min":{"1min":0.55300000000000005,"5min":0.32800000000000001,"15min":0.29399999999999998},"max":{"1min":0.95899999999999996,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.749,"15min":0.746},"min":{"1min":0.53600000000000003,"5min":0.32400000000000001,"15min":0.26900000000000002},"max":{"1min":0.91000000000000003,"5min":3.3119999999999998,"15min":3.3399999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.51700000000000002,"5min":0.372,"15min":0.34599999999999997},"max":{"1min":0.83399999999999996,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.70099999999999996},"min":{"1min":0.52300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96499999999999997,"5min":5.96,"15min":5.96},"last":0.63}]},{"osd":2,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77600000000000002,"15min":0.71299999999999997},"min":{"1min":0.56899999999999995,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.93899999999999995,"5min":8.3740000000000006,"15min":8.3740000000000006},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.77800000000000002,"15min":0.74199999999999999},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":5.8449999999999998,"15min":5.8449999999999998},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":0.96599999999999997,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.80800000000000005,"15min":0.75},"min":{"1min":0.58099999999999996,"5min":0.44500000000000001,"15min":0.373},"max":{"1min":0.874,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.33200000000000002},"max":{"1min":0.83699999999999997,"5min":5.7759999999999998,"15min":5.7759999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.46000000000000002,"5min":0.42399999999999999,"15min":0.39200000000000002},"max":{"1min":0.93400000000000005,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.83399999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.74199999999999999,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.38600000000000001},"max":{"1min":0.86799999999999999,"5min":8.4870000000000001,"15min":8.4870000000000001},"last":0.875},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":0.86499999999999999,"5min":5.923,"15min":5.923},"last":0.94999999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76100000000000001,"15min":0.77600000000000002},"min":{"1min":0.53900000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90300000000000002,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.81599999999999995,"15min":0.80000000000000004},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93700000000000006,"5min":8.4429999999999996,"15min":8.4429999999999996},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2312,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2367488,"data_stored":1406812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41599999999999998,"15min":0.46999999999999997},"min":{"1min":0.30199999999999999,"5min":0.26200000000000001,"15min":0.23699999999999999},"max":{"1min":0.53100000000000003,"5min":0.69499999999999995,"15min":2.8879999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.53200000000000003},"min":{"1min":0.40500000000000003,"5min":0.34999999999999998,"15min":0.313},"max":{"1min":0.70499999999999996,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.623}]},{"osd":2,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.64300000000000002},"min":{"1min":0.34599999999999997,"5min":0.28199999999999997,"15min":0.26500000000000001},"max":{"1min":0.58699999999999997,"5min":3.3690000000000002,"15min":13.055999999999999},"last":0.432},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.58099999999999996},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.66600000000000004,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.504,"15min":0.55500000000000005},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48699999999999999,"15min":0.51100000000000001},"min":{"1min":0.35999999999999999,"5min":0.318,"15min":0.29499999999999998},"max":{"1min":0.80100000000000005,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69899999999999995,"15min":0.90900000000000003},"min":{"1min":0.61199999999999999,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.444,"15min":12.989000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.747,"15min":0.78900000000000003},"min":{"1min":0.627,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":1.1020000000000001,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.879}]},{"osd":5,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.70099999999999996,"15min":0.74099999999999999},"min":{"1min":0.59299999999999997,"5min":0.44700000000000001,"15min":0.432},"max":{"1min":1.0429999999999999,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.90400000000000003},"min":{"1min":0.50600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.01,"5min":1.494,"15min":13.023999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.65600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99399999999999999,"5min":2.9670000000000001,"15min":2.9670000000000001},"last":0.746},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.754,"15min":0.95399999999999996},"min":{"1min":0.70599999999999996,"5min":0.47899999999999998,"15min":0.47799999999999998},"max":{"1min":1.012,"5min":2.5409999999999999,"15min":12.941000000000001},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.74099999999999999,"15min":0.78200000000000003},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.44700000000000001},"max":{"1min":1.1799999999999999,"5min":2.431,"15min":2.806},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.65500000000000003,"5min":0.50600000000000001,"15min":0.48399999999999999},"max":{"1min":1.1439999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.86599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.60699999999999998,"5min":0.70699999999999996,"15min":0.753},"last":0.432},{"interface":"front","average":{"1min":0.499,"5min":0.436,"15min":0.44},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.63,"5min":0.63,"15min":0.81200000000000006},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.73699999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.48899999999999999},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.314},"max":{"1min":0.65700000000000003,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47299999999999998,"15min":0.48899999999999999},"min":{"1min":0.33200000000000002,"5min":0.32200000000000001,"15min":0.29899999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.81200000000000006},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.55500000000000005},"min":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.38800000000000001},"max":{"1min":0.67300000000000004,"5min":0.77600000000000002,"15min":0.84399999999999997},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.72299999999999998},"min":{"1min":0.55800000000000005,"5min":0.48399999999999999,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.171},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52500000000000002,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":1.0109999999999999,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.52000000000000002,"5min":0.48099999999999998,"15min":0.45600000000000002},"max":{"1min":0.94899999999999995,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46999999999999997},"max":{"1min":1.032,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.749}]},{"osd":6,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.54600000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.093,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46200000000000002},"max":{"1min":0.94999999999999996,"5min":1.117,"15min":1.546},"last":0.621}]},{"osd":7,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77600000000000002,"15min":0.79000000000000004},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":1.0800000000000001,"5min":2.778,"15min":2.778},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.76600000000000001,"15min":0.89700000000000002},"min":{"1min":0.61499999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0680000000000001,"5min":3.7069999999999999,"15min":9.5960000000000001},"last":0.76400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575007,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40972,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330868,"statfs":{"total":96636764160,"available":96594808832,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47399999999999998,"15min":0.44400000000000001},"min":{"1min":0.33300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65100000000000002,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45800000000000002,"15min":0.47599999999999998},"min":{"1min":0.28699999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.628,"5min":1.093,"15min":2.4630000000000001},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.67900000000000005,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77800000000000002,"5min":3.657,"15min":3.657},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.56599999999999995},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.33900000000000002},"max":{"1min":0.68000000000000005,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76300000000000001,"5min":2.1629999999999998,"15min":2.7679999999999998},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.91400000000000003,"15min":0.79700000000000004},"min":{"1min":0.504,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.99299999999999999,"5min":12.007,"15min":12.007},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.48399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0529999999999999,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.79200000000000004,"15min":0.755},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":0.90400000000000003,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.79900000000000004},"min":{"1min":0.55900000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0660000000000001,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.873}]},{"osd":6,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.91400000000000003,"15min":0.80100000000000005},"min":{"1min":0.53500000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.022,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.90500000000000003,"15min":0.81399999999999995},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":1.0069999999999999,"5min":13.084,"15min":13.084},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81699999999999995,"15min":0.81599999999999995},"min":{"1min":0.56799999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0700000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.81499999999999995},"min":{"1min":0.61799999999999999,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.079,"5min":4.484,"15min":4.484},"last":0.96899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":810526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.435,"15min":0.50600000000000001},"min":{"1min":0.34799999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.72399999999999998,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.505},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.89600000000000002,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.64500000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.44,"15min":0.69399999999999995},"min":{"1min":0.308,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68000000000000005,"5min":1.3620000000000001,"15min":15.911},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.52600000000000002,"15min":0.77100000000000002},"min":{"1min":0.432,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":1.347,"15min":16.024000000000001},"last":0.621}]},{"osd":2,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.35099999999999998,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":0.67900000000000005,"5min":3.25,"15min":3.25},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.53200000000000003,"15min":0.77300000000000002},"min":{"1min":0.39400000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85399999999999998,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.71899999999999997,"15min":0.73899999999999999},"min":{"1min":0.627,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0329999999999999,"5min":1.782,"15min":1.782},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.73999999999999999,"15min":0.81799999999999995},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.36699999999999999},"max":{"1min":1.0229999999999999,"5min":1.724,"15min":5.7300000000000004},"last":1.0269999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.746,"15min":0.82499999999999996},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.39700000000000002},"max":{"1min":1.077,"5min":1.411,"15min":5.077},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.77200000000000002,"15min":0.86199999999999999},"min":{"1min":0.69099999999999995,"5min":0.52200000000000002,"15min":0.44800000000000001},"max":{"1min":1.0609999999999999,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.749,"15min":0.77100000000000002},"min":{"1min":0.58899999999999997,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.089,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.746,"15min":0.82799999999999996},"min":{"1min":0.63500000000000001,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.1319999999999999,"5min":1.4490000000000001,"15min":5.157},"last":1.1759999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.75800000000000001,"15min":0.78700000000000003},"min":{"1min":0.66400000000000003,"5min":0.51700000000000002,"15min":0.47399999999999998},"max":{"1min":1.073,"5min":1.863,"15min":1.863},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.75800000000000001,"15min":0.84099999999999997},"min":{"1min":0.64100000000000001,"5min":0.51100000000000001,"15min":0.49399999999999999},"max":{"1min":1.179,"5min":1.492,"15min":5.1189999999999998},"last":1.2230000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":810526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.66200000000000003,"15min":0.65500000000000003},"min":{"1min":0.55100000000000005,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.90700000000000003,"5min":0.999,"15min":1.784},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.56000000000000005,"5min":0.42099999999999999,"15min":0.38},"max":{"1min":0.878,"5min":0.95899999999999996,"15min":1.0509999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.39500000000000002,"15min":0.38},"max":{"1min":0.89700000000000002,"5min":1.0900000000000001,"15min":6.258},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.41399999999999998,"15min":0.35899999999999999},"max":{"1min":0.98399999999999999,"5min":1.0700000000000001,"15min":1.806},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69499999999999995,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.40600000000000003,"15min":0.29199999999999998},"max":{"1min":1.01,"5min":1.012,"15min":6.4740000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.82999999999999996},"min":{"1min":0.66000000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":0.98599999999999999,"5min":0.99399999999999999,"15min":6.4859999999999998},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68799999999999994},"min":{"1min":0.58099999999999996,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.0309999999999999,"15min":1.25},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.49099999999999999,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.413},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.627},"min":{"1min":0.44,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.3280000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.99299999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.629},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.76500000000000001},"min":{"1min":0.60199999999999998,"5min":0.5,"15min":0.47299999999999998},"max":{"1min":0.93600000000000005,"5min":0.96299999999999997,"15min":1.6839999999999999},"last":0.755}]},{"osd":7,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.48099999999999998,"5min":0.442,"15min":0.41299999999999998},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":1.036,"15min":1.21},"last":0.78600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37452,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334388,"statfs":{"total":96636764160,"available":96598413312,"internally_reserved":0,"allocated":2363392,"data_stored":1405793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.436,"5min":0.40200000000000002,"15min":0.32100000000000001},"max":{"1min":0.83299999999999996,"5min":0.92900000000000005,"15min":4.5010000000000003},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66200000000000003,"15min":0.746},"min":{"1min":0.57899999999999996,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.83999999999999997,"5min":0.99299999999999999,"15min":6.4059999999999997},"last":0.82999999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.307},"max":{"1min":0.92500000000000004,"5min":1.121,"15min":1.121},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.41399999999999998},"max":{"1min":0.93600000000000005,"5min":0.97299999999999998,"15min":1.546},"last":0.81799999999999995}]},{"osd":2,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.78700000000000003,"15min":0.72899999999999998},"min":{"1min":0.54100000000000004,"5min":0.49399999999999999,"15min":0.376},"max":{"1min":0.91000000000000003,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67900000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":1.069,"15min":1.069},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.78900000000000003,"15min":0.73799999999999999},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.42199999999999999},"max":{"1min":0.88400000000000001,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":0.93200000000000005,"5min":1.3140000000000001,"15min":4.484},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.995,"5min":1.0229999999999999,"15min":1.6240000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.379},"max":{"1min":0.95899999999999996,"5min":0.98699999999999999,"15min":2.718},"last":0.89400000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68500000000000005,"15min":0.68400000000000005},"min":{"1min":0.50600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.97399999999999998,"15min":2.5249999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.437,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.028,"5min":1.036,"15min":1.669},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73599999999999999,"15min":0.748},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.438},"max":{"1min":1.006,"5min":1.0529999999999999,"15min":2.0459999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74099999999999999,"15min":0.76200000000000001},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92700000000000005,"5min":1.034,"15min":2.2919999999999998},"last":0.95599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:57:58.435 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:57:58.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.628+0000 7fe82cfff700 1 Processor -- start 2023-07-06T03:57:58.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.628+0000 7fe82cfff700 1 -- start start 2023-07-06T03:57:58.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.629+0000 7fe82cfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe8280b8470 0x7fe8280cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.629+0000 7fe82cfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8280b2190 0x7fe8280cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.629+0000 7fe82cfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280cfc60 0x7fe828160120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.629+0000 7fe82cfff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8280b8840 con 0x7fe8280b2190 2023-07-06T03:57:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.629+0000 7fe82cfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8280d00c0 con 0x7fe8280b8470 2023-07-06T03:57:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.629+0000 7fe82cfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280d12f0 con 0x7fe8280cfc60 2023-07-06T03:57:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.629+0000 7fe81ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8280b2190 0x7fe8280cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.629+0000 7fe81ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8280b2190 0x7fe8280cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37678/0 (socket says 172.21.15.97:37678) 2023-07-06T03:57:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.629+0000 7fe81ffff700 1 -- 172.21.15.97:0/3115701684 learned_addr learned my addr 172.21.15.97:0/3115701684 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe826d9d700 1 --2- 172.21.15.97:0/3115701684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280cfc60 0x7fe828160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe82659c700 1 --2- 172.21.15.97:0/3115701684 >> v2:172.21.15.146:3300/0 conn(0x7fe8280b8470 0x7fe8280cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe81ffff700 1 -- 172.21.15.97:0/3115701684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280cfc60 msgr2=0x7fe828160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:58.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe81ffff700 1 --2- 172.21.15.97:0/3115701684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280cfc60 0x7fe828160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe81ffff700 1 -- 172.21.15.97:0/3115701684 >> v2:172.21.15.146:3300/0 conn(0x7fe8280b8470 msgr2=0x7fe8280cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:58.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe81ffff700 1 --2- 172.21.15.97:0/3115701684 >> v2:172.21.15.146:3300/0 conn(0x7fe8280b8470 0x7fe8280cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe81ffff700 1 -- 172.21.15.97:0/3115701684 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8280cf390 con 0x7fe8280b2190 2023-07-06T03:57:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe826d9d700 1 --2- 172.21.15.97:0/3115701684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280cfc60 0x7fe828160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe81ffff700 1 --2- 172.21.15.97:0/3115701684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8280b2190 0x7fe8280cf720 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fe8280d3d80 tx=0x7fe810009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1472cd4028376b20 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe825d9b700 1 -- 172.21.15.97:0/3115701684 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810011830 con 0x7fe8280b2190 2023-07-06T03:57:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe825d9b700 1 -- 172.21.15.97:0/3115701684 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe810011990 con 0x7fe8280b2190 2023-07-06T03:57:58.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.630+0000 7fe825d9b700 1 -- 172.21.15.97:0/3115701684 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810011d20 con 0x7fe8280b2190 2023-07-06T03:57:58.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.631+0000 7fe82cfff700 1 -- 172.21.15.97:0/3115701684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8280b2190 msgr2=0x7fe8280cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.631+0000 7fe82cfff700 1 --2- 172.21.15.97:0/3115701684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8280b2190 0x7fe8280cf720 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fe8280d3d80 tx=0x7fe810009bd0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.631+0000 7fe82cfff700 1 -- 172.21.15.97:0/3115701684 shutdown_connections 2023-07-06T03:57:58.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.631+0000 7fe82cfff700 1 --2- 172.21.15.97:0/3115701684 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280cfc60 0x7fe828160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.631+0000 7fe82cfff700 1 --2- 172.21.15.97:0/3115701684 >> v2:172.21.15.146:3300/0 conn(0x7fe8280b8470 0x7fe8280cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.631+0000 7fe82cfff700 1 --2- 172.21.15.97:0/3115701684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8280b2190 0x7fe8280cf720 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.631+0000 7fe82cfff700 1 -- 172.21.15.97:0/3115701684 >> 172.21.15.97:0/3115701684 conn(0x7fe828071740 msgr2=0x7fe828071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.631+0000 7fe82cfff700 1 -- 172.21.15.97:0/3115701684 shutdown_connections 2023-07-06T03:57:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.631+0000 7fe82cfff700 1 -- 172.21.15.97:0/3115701684 wait complete. 2023-07-06T03:57:58.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.633+0000 7fe82cfff700 1 Processor -- start 2023-07-06T03:57:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.634+0000 7fe82cfff700 1 -- start start 2023-07-06T03:57:58.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe82cfff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280b2190 0x7fe8280c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe82cfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8280b8470 0x7fe8280cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe82cfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8280cfc60 0x7fe8280cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe82cfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280c8c20 con 0x7fe8280cfc60 2023-07-06T03:57:58.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe82cfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe828153930 con 0x7fe8280b8470 2023-07-06T03:57:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe82cfff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe828152770 con 0x7fe8280b2190 2023-07-06T03:57:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe826d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8280cfc60 0x7fe8280cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe826d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8280cfc60 0x7fe8280cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37684/0 (socket says 172.21.15.97:37684) 2023-07-06T03:57:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe826d9d700 1 -- 172.21.15.97:0/1704274983 learned_addr learned my addr 172.21.15.97:0/1704274983 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:58.643 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe82659c700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280b2190 0x7fe8280c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe81ffff700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8280b8470 0x7fe8280cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe826d9d700 1 -- 172.21.15.97:0/1704274983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280b2190 msgr2=0x7fe8280c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe826d9d700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280b2190 0x7fe8280c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe826d9d700 1 -- 172.21.15.97:0/1704274983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8280b8470 msgr2=0x7fe8280cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe826d9d700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8280b8470 0x7fe8280cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe826d9d700 1 -- 172.21.15.97:0/1704274983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810009550 con 0x7fe8280cfc60 2023-07-06T03:57:58.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe81ffff700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8280b8470 0x7fe8280cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:58.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.635+0000 7fe82659c700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280b2190 0x7fe8280c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:57:58.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.636+0000 7fe826d9d700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8280cfc60 0x7fe8280cbd80 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fe820007df0 tx=0x7fe82000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:58.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.636+0000 7fe81effd700 1 -- 172.21.15.97:0/1704274983 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82000ddd0 con 0x7fe8280cfc60 2023-07-06T03:57:58.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.636+0000 7fe81effd700 1 -- 172.21.15.97:0/1704274983 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82000f040 con 0x7fe8280cfc60 2023-07-06T03:57:58.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.636+0000 7fe81effd700 1 -- 172.21.15.97:0/1704274983 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820003c60 con 0x7fe8280cfc60 2023-07-06T03:57:58.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.636+0000 7fe82cfff700 1 -- 172.21.15.97:0/1704274983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8280cc390 con 0x7fe8280cfc60 2023-07-06T03:57:58.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.636+0000 7fe82cfff700 1 -- 172.21.15.97:0/1704274983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe828153bf0 con 0x7fe8280cfc60 2023-07-06T03:57:58.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.637+0000 7fe82cfff700 1 -- 172.21.15.97:0/1704274983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe828068040 con 0x7fe8280cfc60 2023-07-06T03:57:58.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.638+0000 7fe81effd700 1 -- 172.21.15.97:0/1704274983 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe820004030 con 0x7fe8280cfc60 2023-07-06T03:57:58.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.639+0000 7fe81effd700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe80806ace0 0x7fe80806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:58.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.639+0000 7fe81effd700 1 -- 172.21.15.97:0/1704274983 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe8200894c0 con 0x7fe8280cfc60 2023-07-06T03:57:58.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.639+0000 7fe82659c700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe80806ace0 0x7fe80806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.640+0000 7fe82659c700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe80806ace0 0x7fe80806d190 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe818009bb0 tx=0x7fe818008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:58.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.643+0000 7fe81effd700 1 -- 172.21.15.97:0/1704274983 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe820054760 con 0x7fe8280cfc60 2023-07-06T03:57:58.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.833+0000 7fe82cfff700 1 -- 172.21.15.97:0/1704274983 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8280cd500 con 0x7fe80806ace0 2023-07-06T03:57:58.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.836+0000 7fe81effd700 1 -- 172.21.15.97:0/1704274983 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72761 (secure 0 0 0) 0x7fe818011800 con 0x7fe80806ace0 2023-07-06T03:57:58.840 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:58.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 -- 172.21.15.97:0/1704274983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe80806ace0 msgr2=0x7fe80806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe80806ace0 0x7fe80806d190 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe818009bb0 tx=0x7fe818008040 comp rx=0 tx=0).stop 2023-07-06T03:57:58.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 -- 172.21.15.97:0/1704274983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8280cfc60 msgr2=0x7fe8280cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:58.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8280cfc60 0x7fe8280cbd80 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fe820007df0 tx=0x7fe82000d3b0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 -- 172.21.15.97:0/1704274983 shutdown_connections 2023-07-06T03:57:58.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fe80806ace0 0x7fe80806d190 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8280cfc60 0x7fe8280cbd80 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fe8280b2190 0x7fe8280c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 --2- 172.21.15.97:0/1704274983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8280b8470 0x7fe8280cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:58.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 -- 172.21.15.97:0/1704274983 >> 172.21.15.97:0/1704274983 conn(0x7fe828071740 msgr2=0x7fe8280d2750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:58.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 -- 172.21.15.97:0/1704274983 shutdown_connections 2023-07-06T03:57:58.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:58.839+0000 7fe82cfff700 1 -- 172.21.15.97:0/1704274983 wait complete. 2023-07-06T03:57:58.844 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:57:58.859 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2023-07-06T03:57:57.052464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296116,"kb_used_data":14964,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754678604,"statfs":{"total":773094113280,"available":772790890496,"internally_reserved":0,"allocated":15323136,"data_stored":7693515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14367,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001002"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":227,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:54.961258+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:57:54.961258+0000","last_peered":"2023-07-06T03:57:54.961258+0000","last_clean":"2023-07-06T03:57:54.961258+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:57:54.961258+0000","last_undegraded":"2023-07-06T03:57:54.961258+0000","last_fullsized":"2023-07-06T03:57:54.961258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:57:34.473968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956223+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:57:53.956223+0000","last_peered":"2023-07-06T03:57:53.956223+0000","last_clean":"2023-07-06T03:57:53.956223+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:57:53.956223+0000","last_undegraded":"2023-07-06T03:57:53.956223+0000","last_fullsized":"2023-07-06T03:57:53.956223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:25.741703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":243,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:54.964259+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:57:54.964259+0000","last_peered":"2023-07-06T03:57:54.964259+0000","last_clean":"2023-07-06T03:57:54.964259+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:57:54.964259+0000","last_undegraded":"2023-07-06T03:57:54.964259+0000","last_fullsized":"2023-07-06T03:57:54.964259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:50:57.912150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956011+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:57:53.956011+0000","last_peered":"2023-07-06T03:57:53.956011+0000","last_clean":"2023-07-06T03:57:53.956011+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:57:53.956011+0000","last_undegraded":"2023-07-06T03:57:53.956011+0000","last_fullsized":"2023-07-06T03:57:53.956011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:13.711531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":377,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:55.226464+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:57:55.226464+0000","last_peered":"2023-07-06T03:57:55.226464+0000","last_clean":"2023-07-06T03:57:55.226464+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:57:55.226464+0000","last_undegraded":"2023-07-06T03:57:55.226464+0000","last_fullsized":"2023-07-06T03:57:55.226464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:19:12.557397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956489+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:57:53.956489+0000","last_peered":"2023-07-06T03:57:53.956489+0000","last_clean":"2023-07-06T03:57:53.956489+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:57:53.956489+0000","last_undegraded":"2023-07-06T03:57:53.956489+0000","last_fullsized":"2023-07-06T03:57:53.956489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:26:45.174657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956669+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:57:53.956669+0000","last_peered":"2023-07-06T03:57:53.956669+0000","last_clean":"2023-07-06T03:57:53.956669+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:57:53.956669+0000","last_undegraded":"2023-07-06T03:57:53.956669+0000","last_fullsized":"2023-07-06T03:57:53.956669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:26:18.178369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":228,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.957919+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:57:53.957919+0000","last_peered":"2023-07-06T03:57:53.957919+0000","last_clean":"2023-07-06T03:57:53.957919+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:57:53.957919+0000","last_undegraded":"2023-07-06T03:57:53.957919+0000","last_fullsized":"2023-07-06T03:57:53.957919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:51:43.180770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":234,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.957279+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:57:53.957279+0000","last_peered":"2023-07-06T03:57:53.957279+0000","last_clean":"2023-07-06T03:57:53.957279+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:57:53.957279+0000","last_undegraded":"2023-07-06T03:57:53.957279+0000","last_fullsized":"2023-07-06T03:57:53.957279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:09:40.927896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.51000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.98499999999999999,"5min":1.024,"15min":4.0570000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.67200000000000004,"15min":0.72099999999999997},"min":{"1min":0.54700000000000004,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":3.8660000000000001},"last":0.623}]},{"osd":1,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.58299999999999996,"5min":0.45400000000000001,"15min":0.34000000000000002},"max":{"1min":0.96299999999999997,"5min":1.091,"15min":2.6219999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.67800000000000005,"15min":0.76700000000000002},"min":{"1min":0.52700000000000002,"5min":0.40000000000000002,"15min":0.38800000000000001},"max":{"1min":0.94399999999999995,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.68899999999999995,"15min":0.72999999999999998},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.35199999999999998},"max":{"1min":0.92000000000000004,"5min":1.0209999999999999,"15min":3.77},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.65000000000000002,"5min":0.47999999999999998,"15min":0.45000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71899999999999997,"15min":0.71099999999999997},"min":{"1min":0.60599999999999998,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98499999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.752,"15min":0.753},"min":{"1min":0.63700000000000001,"5min":0.54500000000000004,"15min":0.43099999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.0209999999999999},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65500000000000003,"15min":0.63100000000000001},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.94699999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":0.94399999999999995,"15min":1.4590000000000001},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.72199999999999998,"15min":0.69999999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.38},"max":{"1min":1.0329999999999999,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70499999999999996,"15min":0.68899999999999995},"min":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.40000000000000002},"max":{"1min":0.997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.74299999999999999},"min":{"1min":0.53600000000000003,"5min":0.46400000000000002,"15min":0.438},"max":{"1min":1.052,"5min":1.052,"15min":1.5349999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.502,"5min":0.49199999999999999,"15min":0.42299999999999999},"max":{"1min":1.0169999999999999,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.48299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":810526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.74299999999999999,"15min":0.69899999999999995},"min":{"1min":0.55300000000000005,"5min":0.32800000000000001,"15min":0.29399999999999998},"max":{"1min":0.95899999999999996,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.749,"15min":0.746},"min":{"1min":0.53600000000000003,"5min":0.32400000000000001,"15min":0.26900000000000002},"max":{"1min":0.91000000000000003,"5min":3.3119999999999998,"15min":3.3399999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.51700000000000002,"5min":0.372,"15min":0.34599999999999997},"max":{"1min":0.83399999999999996,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.70099999999999996},"min":{"1min":0.52300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96499999999999997,"5min":5.96,"15min":5.96},"last":0.63}]},{"osd":2,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77600000000000002,"15min":0.71299999999999997},"min":{"1min":0.56899999999999995,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.93899999999999995,"5min":8.3740000000000006,"15min":8.3740000000000006},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.77800000000000002,"15min":0.74199999999999999},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":5.8449999999999998,"15min":5.8449999999999998},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":0.96599999999999997,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.80800000000000005,"15min":0.75},"min":{"1min":0.58099999999999996,"5min":0.44500000000000001,"15min":0.373},"max":{"1min":0.874,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.33200000000000002},"max":{"1min":0.83699999999999997,"5min":5.7759999999999998,"15min":5.7759999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.46000000000000002,"5min":0.42399999999999999,"15min":0.39200000000000002},"max":{"1min":0.93400000000000005,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.83399999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.74199999999999999,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.38600000000000001},"max":{"1min":0.86799999999999999,"5min":8.4870000000000001,"15min":8.4870000000000001},"last":0.875},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":0.86499999999999999,"5min":5.923,"15min":5.923},"last":0.94999999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76100000000000001,"15min":0.77600000000000002},"min":{"1min":0.53900000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90300000000000002,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.81599999999999995,"15min":0.80000000000000004},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93700000000000006,"5min":8.4429999999999996,"15min":8.4429999999999996},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2312,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2367488,"data_stored":1406812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41599999999999998,"15min":0.46999999999999997},"min":{"1min":0.30199999999999999,"5min":0.26200000000000001,"15min":0.23699999999999999},"max":{"1min":0.53100000000000003,"5min":0.69499999999999995,"15min":2.8879999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.53200000000000003},"min":{"1min":0.40500000000000003,"5min":0.34999999999999998,"15min":0.313},"max":{"1min":0.70499999999999996,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.623}]},{"osd":2,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.64300000000000002},"min":{"1min":0.34599999999999997,"5min":0.28199999999999997,"15min":0.26500000000000001},"max":{"1min":0.58699999999999997,"5min":3.3690000000000002,"15min":13.055999999999999},"last":0.432},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.58099999999999996},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.66600000000000004,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.504,"15min":0.55500000000000005},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48699999999999999,"15min":0.51100000000000001},"min":{"1min":0.35999999999999999,"5min":0.318,"15min":0.29499999999999998},"max":{"1min":0.80100000000000005,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69899999999999995,"15min":0.90900000000000003},"min":{"1min":0.61199999999999999,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.444,"15min":12.989000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.747,"15min":0.78900000000000003},"min":{"1min":0.627,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":1.1020000000000001,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.879}]},{"osd":5,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.70099999999999996,"15min":0.74099999999999999},"min":{"1min":0.59299999999999997,"5min":0.44700000000000001,"15min":0.432},"max":{"1min":1.0429999999999999,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.90400000000000003},"min":{"1min":0.50600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.01,"5min":1.494,"15min":13.023999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.65600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99399999999999999,"5min":2.9670000000000001,"15min":2.9670000000000001},"last":0.746},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.754,"15min":0.95399999999999996},"min":{"1min":0.70599999999999996,"5min":0.47899999999999998,"15min":0.47799999999999998},"max":{"1min":1.012,"5min":2.5409999999999999,"15min":12.941000000000001},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.74099999999999999,"15min":0.78200000000000003},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.44700000000000001},"max":{"1min":1.1799999999999999,"5min":2.431,"15min":2.806},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.65500000000000003,"5min":0.50600000000000001,"15min":0.48399999999999999},"max":{"1min":1.1439999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.86599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.60699999999999998,"5min":0.70699999999999996,"15min":0.753},"last":0.432},{"interface":"front","average":{"1min":0.499,"5min":0.436,"15min":0.44},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.63,"5min":0.63,"15min":0.81200000000000006},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.73699999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.48899999999999999},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.314},"max":{"1min":0.65700000000000003,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47299999999999998,"15min":0.48899999999999999},"min":{"1min":0.33200000000000002,"5min":0.32200000000000001,"15min":0.29899999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.81200000000000006},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.55500000000000005},"min":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.38800000000000001},"max":{"1min":0.67300000000000004,"5min":0.77600000000000002,"15min":0.84399999999999997},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.72299999999999998},"min":{"1min":0.55800000000000005,"5min":0.48399999999999999,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.171},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52500000000000002,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":1.0109999999999999,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.52000000000000002,"5min":0.48099999999999998,"15min":0.45600000000000002},"max":{"1min":0.94899999999999995,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46999999999999997},"max":{"1min":1.032,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.749}]},{"osd":6,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.54600000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.093,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46200000000000002},"max":{"1min":0.94999999999999996,"5min":1.117,"15min":1.546},"last":0.621}]},{"osd":7,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77600000000000002,"15min":0.79000000000000004},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":1.0800000000000001,"5min":2.778,"15min":2.778},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.76600000000000001,"15min":0.89700000000000002},"min":{"1min":0.61499999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0680000000000001,"5min":3.7069999999999999,"15min":9.5960000000000001},"last":0.76400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575007,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40972,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330868,"statfs":{"total":96636764160,"available":96594808832,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47399999999999998,"15min":0.44400000000000001},"min":{"1min":0.33300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65100000000000002,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45800000000000002,"15min":0.47599999999999998},"min":{"1min":0.28699999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.628,"5min":1.093,"15min":2.4630000000000001},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.67900000000000005,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77800000000000002,"5min":3.657,"15min":3.657},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.56599999999999995},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.33900000000000002},"max":{"1min":0.68000000000000005,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76300000000000001,"5min":2.1629999999999998,"15min":2.7679999999999998},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.91400000000000003,"15min":0.79700000000000004},"min":{"1min":0.504,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.99299999999999999,"5min":12.007,"15min":12.007},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.48399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0529999999999999,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.79200000000000004,"15min":0.755},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":0.90400000000000003,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.79900000000000004},"min":{"1min":0.55900000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0660000000000001,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.873}]},{"osd":6,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.91400000000000003,"15min":0.80100000000000005},"min":{"1min":0.53500000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.022,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.90500000000000003,"15min":0.81399999999999995},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":1.0069999999999999,"5min":13.084,"15min":13.084},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81699999999999995,"15min":0.81599999999999995},"min":{"1min":0.56799999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0700000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.81499999999999995},"min":{"1min":0.61799999999999999,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.079,"5min":4.484,"15min":4.484},"last":0.96899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":810526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.435,"15min":0.50600000000000001},"min":{"1min":0.34799999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.72399999999999998,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.505},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.89600000000000002,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.64500000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.44,"15min":0.69399999999999995},"min":{"1min":0.308,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68000000000000005,"5min":1.3620000000000001,"15min":15.911},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.52600000000000002,"15min":0.77100000000000002},"min":{"1min":0.432,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":1.347,"15min":16.024000000000001},"last":0.621}]},{"osd":2,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.35099999999999998,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":0.67900000000000005,"5min":3.25,"15min":3.25},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.53200000000000003,"15min":0.77300000000000002},"min":{"1min":0.39400000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85399999999999998,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.71899999999999997,"15min":0.73899999999999999},"min":{"1min":0.627,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0329999999999999,"5min":1.782,"15min":1.782},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.73999999999999999,"15min":0.81799999999999995},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.36699999999999999},"max":{"1min":1.0229999999999999,"5min":1.724,"15min":5.7300000000000004},"last":1.0269999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.746,"15min":0.82499999999999996},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.39700000000000002},"max":{"1min":1.077,"5min":1.411,"15min":5.077},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.77200000000000002,"15min":0.86199999999999999},"min":{"1min":0.69099999999999995,"5min":0.52200000000000002,"15min":0.44800000000000001},"max":{"1min":1.0609999999999999,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.749,"15min":0.77100000000000002},"min":{"1min":0.58899999999999997,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.089,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.746,"15min":0.82799999999999996},"min":{"1min":0.63500000000000001,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.1319999999999999,"5min":1.4490000000000001,"15min":5.157},"last":1.1759999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.75800000000000001,"15min":0.78700000000000003},"min":{"1min":0.66400000000000003,"5min":0.51700000000000002,"15min":0.47399999999999998},"max":{"1min":1.073,"5min":1.863,"15min":1.863},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.75800000000000001,"15min":0.84099999999999997},"min":{"1min":0.64100000000000001,"5min":0.51100000000000001,"15min":0.49399999999999999},"max":{"1min":1.179,"5min":1.492,"15min":5.1189999999999998},"last":1.2230000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":810526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.66200000000000003,"15min":0.65500000000000003},"min":{"1min":0.55100000000000005,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.90700000000000003,"5min":0.999,"15min":1.784},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.56000000000000005,"5min":0.42099999999999999,"15min":0.38},"max":{"1min":0.878,"5min":0.95899999999999996,"15min":1.0509999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.39500000000000002,"15min":0.38},"max":{"1min":0.89700000000000002,"5min":1.0900000000000001,"15min":6.258},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.41399999999999998,"15min":0.35899999999999999},"max":{"1min":0.98399999999999999,"5min":1.0700000000000001,"15min":1.806},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69499999999999995,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.40600000000000003,"15min":0.29199999999999998},"max":{"1min":1.01,"5min":1.012,"15min":6.4740000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.82999999999999996},"min":{"1min":0.66000000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":0.98599999999999999,"5min":0.99399999999999999,"15min":6.4859999999999998},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68799999999999994},"min":{"1min":0.58099999999999996,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.0309999999999999,"15min":1.25},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.49099999999999999,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.413},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.627},"min":{"1min":0.44,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.3280000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.99299999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.629},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.76500000000000001},"min":{"1min":0.60199999999999998,"5min":0.5,"15min":0.47299999999999998},"max":{"1min":0.93600000000000005,"5min":0.96299999999999997,"15min":1.6839999999999999},"last":0.755}]},{"osd":7,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.48099999999999998,"5min":0.442,"15min":0.41299999999999998},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":1.036,"15min":1.21},"last":0.78600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37452,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334388,"statfs":{"total":96636764160,"available":96598413312,"internally_reserved":0,"allocated":2363392,"data_stored":1405793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.436,"5min":0.40200000000000002,"15min":0.32100000000000001},"max":{"1min":0.83299999999999996,"5min":0.92900000000000005,"15min":4.5010000000000003},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66200000000000003,"15min":0.746},"min":{"1min":0.57899999999999996,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.83999999999999997,"5min":0.99299999999999999,"15min":6.4059999999999997},"last":0.82999999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.307},"max":{"1min":0.92500000000000004,"5min":1.121,"15min":1.121},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.41399999999999998},"max":{"1min":0.93600000000000005,"5min":0.97299999999999998,"15min":1.546},"last":0.81799999999999995}]},{"osd":2,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.78700000000000003,"15min":0.72899999999999998},"min":{"1min":0.54100000000000004,"5min":0.49399999999999999,"15min":0.376},"max":{"1min":0.91000000000000003,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67900000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":1.069,"15min":1.069},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.78900000000000003,"15min":0.73799999999999999},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.42199999999999999},"max":{"1min":0.88400000000000001,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":0.93200000000000005,"5min":1.3140000000000001,"15min":4.484},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.995,"5min":1.0229999999999999,"15min":1.6240000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.379},"max":{"1min":0.95899999999999996,"5min":0.98699999999999999,"15min":2.718},"last":0.89400000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68500000000000005,"15min":0.68400000000000005},"min":{"1min":0.50600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.97399999999999998,"15min":2.5249999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.437,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.028,"5min":1.036,"15min":1.669},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73599999999999999,"15min":0.748},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.438},"max":{"1min":1.006,"5min":1.0529999999999999,"15min":2.0459999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74099999999999999,"15min":0.76200000000000001},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92700000000000005,"5min":1.034,"15min":2.2919999999999998},"last":0.95599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:57:58.862 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-06T03:57:58.862 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:57:59.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.054+0000 7f707f9ff700 1 Processor -- start 2023-07-06T03:57:59.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.054+0000 7f707f9ff700 1 -- start start 2023-07-06T03:57:59.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.055+0000 7f707f9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70780cfe30 0x7f70780cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.055+0000 7f707f9ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f70780b2190 0x7f70780d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.055+0000 7f707f9ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70780d12f0 con 0x7f70780b8470 2023-07-06T03:57:59.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.055+0000 7f707f9ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70780b8840 con 0x7f70780b2190 2023-07-06T03:57:59.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.056+0000 7f707f9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70780cf390 con 0x7f70780cfe30 2023-07-06T03:57:59.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.056+0000 7f707df9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f70780b2190 0x7f70780d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.056+0000 7f707df9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f70780b2190 0x7f70780d2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37692/0 (socket says 172.21.15.97:37692) 2023-07-06T03:57:59.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.056+0000 7f707df9c700 1 -- 172.21.15.97:0/4074723715 learned_addr learned my addr 172.21.15.97:0/4074723715 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:59.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.056+0000 7f707df9c700 1 -- 172.21.15.97:0/4074723715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70780cfe30 msgr2=0x7f70780cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:59.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.056+0000 7f707cf9a700 1 --2- 172.21.15.97:0/4074723715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70780cfe30 0x7f70780cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.056+0000 7f707df9c700 1 --2- 172.21.15.97:0/4074723715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70780cfe30 0x7f70780cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.056+0000 7f707df9c700 1 -- 172.21.15.97:0/4074723715 >> v1:172.21.15.146:6789/0 conn(0x7f70780b8470 legacy=0x7f70780cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:59.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.056+0000 7f707df9c700 1 -- 172.21.15.97:0/4074723715 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780b8ca0 con 0x7f70780b2190 2023-07-06T03:57:59.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.056+0000 7f707cf9a700 1 --2- 172.21.15.97:0/4074723715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70780cfe30 0x7f70780cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:59.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.056+0000 7f707df9c700 1 --2- 172.21.15.97:0/4074723715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70780b2190 0x7f70780d2750 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f7070009e50 tx=0x7f707000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7234bd4a00505ae server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:59.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.057+0000 7f706ffff700 1 -- 172.21.15.97:0/4074723715 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707000e070 con 0x7f70780b2190 2023-07-06T03:57:59.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.057+0000 7f706ffff700 1 -- 172.21.15.97:0/4074723715 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7070003a20 con 0x7f70780b2190 2023-07-06T03:57:59.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.057+0000 7f706ffff700 1 -- 172.21.15.97:0/4074723715 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070003dd0 con 0x7f70780b2190 2023-07-06T03:57:59.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.057+0000 7f707f9ff700 1 -- 172.21.15.97:0/4074723715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70780b2190 msgr2=0x7f70780d2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:59.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.057+0000 7f707f9ff700 1 --2- 172.21.15.97:0/4074723715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70780b2190 0x7f70780d2750 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f7070009e50 tx=0x7f707000b330 comp rx=0 tx=0).stop 2023-07-06T03:57:59.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.057+0000 7f707f9ff700 1 -- 172.21.15.97:0/4074723715 shutdown_connections 2023-07-06T03:57:59.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.057+0000 7f707f9ff700 1 --2- 172.21.15.97:0/4074723715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70780b2190 0x7f70780d2750 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.057+0000 7f707f9ff700 1 --2- 172.21.15.97:0/4074723715 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70780cfe30 0x7f70780cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.057+0000 7f707f9ff700 1 -- 172.21.15.97:0/4074723715 >> 172.21.15.97:0/4074723715 conn(0x7f7078071740 msgr2=0x7f7078071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:59.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.057+0000 7f707f9ff700 1 -- 172.21.15.97:0/4074723715 shutdown_connections 2023-07-06T03:57:59.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.057+0000 7f707f9ff700 1 -- 172.21.15.97:0/4074723715 wait complete. 2023-07-06T03:57:59.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.059+0000 7f707f9ff700 1 Processor -- start 2023-07-06T03:57:59.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.059+0000 7f707f9ff700 1 -- start start 2023-07-06T03:57:59.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.060+0000 7f707f9ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70780b2190 0x7f7078159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.060+0000 7f707f9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70780b8470 0x7f70780c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.060+0000 7f707f9ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70780cfe30 0x7f707815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.060+0000 7f707f9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780cb920 con 0x7f70780b8470 2023-07-06T03:57:59.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.060+0000 7f707f9ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707815a180 con 0x7f70780cfe30 2023-07-06T03:57:59.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.060+0000 7f707f9ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70780ca880 con 0x7f70780b2190 2023-07-06T03:57:59.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.060+0000 7f707cf9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70780b8470 0x7f70780c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.060+0000 7f707cf9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70780b8470 0x7f70780c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37694/0 (socket says 172.21.15.97:37694) 2023-07-06T03:57:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.060+0000 7f707cf9a700 1 -- 172.21.15.97:0/2095693254 learned_addr learned my addr 172.21.15.97:0/2095693254 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f707d79b700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70780b2190 0x7f7078159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f707df9c700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70780cfe30 0x7f707815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f707cf9a700 1 -- 172.21.15.97:0/2095693254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70780b2190 msgr2=0x7f7078159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f707cf9a700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70780b2190 0x7f7078159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f707cf9a700 1 -- 172.21.15.97:0/2095693254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70780cfe30 msgr2=0x7f707815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f707cf9a700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70780cfe30 0x7f707815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f707cf9a700 1 -- 172.21.15.97:0/2095693254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070009b30 con 0x7f70780b8470 2023-07-06T03:57:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f707cf9a700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70780b8470 0x7f70780c9de0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f706800c7f0 tx=0x7f706800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f706dffb700 1 -- 172.21.15.97:0/2095693254 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068011800 con 0x7f70780b8470 2023-07-06T03:57:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f706dffb700 1 -- 172.21.15.97:0/2095693254 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7068011960 con 0x7f70780b8470 2023-07-06T03:57:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f706dffb700 1 -- 172.21.15.97:0/2095693254 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068011d10 con 0x7f70780b8470 2023-07-06T03:57:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f707f9ff700 1 -- 172.21.15.97:0/2095693254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70780cab60 con 0x7f70780b8470 2023-07-06T03:57:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.061+0000 7f707f9ff700 1 -- 172.21.15.97:0/2095693254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70780cb120 con 0x7f70780b8470 2023-07-06T03:57:59.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.062+0000 7f707f9ff700 1 -- 172.21.15.97:0/2095693254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70780cb770 con 0x7f70780b8470 2023-07-06T03:57:59.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.063+0000 7f706dffb700 1 -- 172.21.15.97:0/2095693254 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7068018360 con 0x7f70780b8470 2023-07-06T03:57:59.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.064+0000 7f706dffb700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f705c06ace0 0x7f705c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.064+0000 7f706dffb700 1 -- 172.21.15.97:0/2095693254 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7068089200 con 0x7f70780b8470 2023-07-06T03:57:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.065+0000 7f707d79b700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f705c06ace0 0x7f705c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.065+0000 7f707d79b700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f705c06ace0 0x7f705c06d190 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7074004f30 tx=0x7f7074009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.069+0000 7f706dffb700 1 -- 172.21.15.97:0/2095693254 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7068057fc0 con 0x7f70780b8470 2023-07-06T03:57:59.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.258+0000 7f707f9ff700 1 -- 172.21.15.97:0/2095693254 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70780cd7e0 con 0x7f705c06ace0 2023-07-06T03:57:59.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.261+0000 7f706dffb700 1 -- 172.21.15.97:0/2095693254 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72761 (secure 0 0 0) 0x7f7074015430 con 0x7f705c06ace0 2023-07-06T03:57:59.265 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:59.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 -- 172.21.15.97:0/2095693254 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f705c06ace0 msgr2=0x7f705c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:59.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f705c06ace0 0x7f705c06d190 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7074004f30 tx=0x7f7074009040 comp rx=0 tx=0).stop 2023-07-06T03:57:59.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 -- 172.21.15.97:0/2095693254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70780b8470 msgr2=0x7f70780c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:59.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70780b8470 0x7f70780c9de0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f706800c7f0 tx=0x7f706800cbb0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 -- 172.21.15.97:0/2095693254 shutdown_connections 2023-07-06T03:57:59.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f705c06ace0 0x7f705c06d190 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70780cfe30 0x7f707815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f70780b2190 0x7f7078159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 --2- 172.21.15.97:0/2095693254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70780b8470 0x7f70780c9de0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 -- 172.21.15.97:0/2095693254 >> 172.21.15.97:0/2095693254 conn(0x7f7078071740 msgr2=0x7f70780b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:59.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 -- 172.21.15.97:0/2095693254 shutdown_connections 2023-07-06T03:57:59.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.264+0000 7f707f9ff700 1 -- 172.21.15.97:0/2095693254 wait complete. 2023-07-06T03:57:59.269 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:57:59.285 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":526,"stamp":"2023-07-06T03:57:59.052823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296116,"kb_used_data":14964,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754678604,"statfs":{"total":773094113280,"available":772790890496,"internally_reserved":0,"allocated":15323136,"data_stored":7693515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14367,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.481182"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":227,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:54.961258+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:57:54.961258+0000","last_peered":"2023-07-06T03:57:54.961258+0000","last_clean":"2023-07-06T03:57:54.961258+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:57:54.961258+0000","last_undegraded":"2023-07-06T03:57:54.961258+0000","last_fullsized":"2023-07-06T03:57:54.961258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:57:34.473968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956223+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:57:53.956223+0000","last_peered":"2023-07-06T03:57:53.956223+0000","last_clean":"2023-07-06T03:57:53.956223+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:57:53.956223+0000","last_undegraded":"2023-07-06T03:57:53.956223+0000","last_fullsized":"2023-07-06T03:57:53.956223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:25.741703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":243,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:54.964259+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:57:54.964259+0000","last_peered":"2023-07-06T03:57:54.964259+0000","last_clean":"2023-07-06T03:57:54.964259+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:57:54.964259+0000","last_undegraded":"2023-07-06T03:57:54.964259+0000","last_fullsized":"2023-07-06T03:57:54.964259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:50:57.912150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956011+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:57:53.956011+0000","last_peered":"2023-07-06T03:57:53.956011+0000","last_clean":"2023-07-06T03:57:53.956011+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:57:53.956011+0000","last_undegraded":"2023-07-06T03:57:53.956011+0000","last_fullsized":"2023-07-06T03:57:53.956011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:13.711531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":377,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:55.226464+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:57:55.226464+0000","last_peered":"2023-07-06T03:57:55.226464+0000","last_clean":"2023-07-06T03:57:55.226464+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:57:55.226464+0000","last_undegraded":"2023-07-06T03:57:55.226464+0000","last_fullsized":"2023-07-06T03:57:55.226464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:19:12.557397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956489+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:57:53.956489+0000","last_peered":"2023-07-06T03:57:53.956489+0000","last_clean":"2023-07-06T03:57:53.956489+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:57:53.956489+0000","last_undegraded":"2023-07-06T03:57:53.956489+0000","last_fullsized":"2023-07-06T03:57:53.956489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:26:45.174657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":235,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.956669+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:57:53.956669+0000","last_peered":"2023-07-06T03:57:53.956669+0000","last_clean":"2023-07-06T03:57:53.956669+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:57:53.956669+0000","last_undegraded":"2023-07-06T03:57:53.956669+0000","last_fullsized":"2023-07-06T03:57:53.956669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:26:18.178369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":228,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.957919+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:57:53.957919+0000","last_peered":"2023-07-06T03:57:53.957919+0000","last_clean":"2023-07-06T03:57:53.957919+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:57:53.957919+0000","last_undegraded":"2023-07-06T03:57:53.957919+0000","last_fullsized":"2023-07-06T03:57:53.957919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:51:43.180770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":234,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-07-06T03:57:53.957279+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:57:53.957279+0000","last_peered":"2023-07-06T03:57:53.957279+0000","last_clean":"2023-07-06T03:57:53.957279+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:57:53.957279+0000","last_undegraded":"2023-07-06T03:57:53.957279+0000","last_fullsized":"2023-07-06T03:57:53.957279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:09:40.927896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.51000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.98499999999999999,"5min":1.024,"15min":4.0570000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.67200000000000004,"15min":0.72099999999999997},"min":{"1min":0.54700000000000004,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":3.8660000000000001},"last":0.623}]},{"osd":1,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.58299999999999996,"5min":0.45400000000000001,"15min":0.34000000000000002},"max":{"1min":0.96299999999999997,"5min":1.091,"15min":2.6219999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.67800000000000005,"15min":0.76700000000000002},"min":{"1min":0.52700000000000002,"5min":0.40000000000000002,"15min":0.38800000000000001},"max":{"1min":0.94399999999999995,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.68899999999999995,"15min":0.72999999999999998},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.35199999999999998},"max":{"1min":0.92000000000000004,"5min":1.0209999999999999,"15min":3.77},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.65000000000000002,"5min":0.47999999999999998,"15min":0.45000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71899999999999997,"15min":0.71099999999999997},"min":{"1min":0.60599999999999998,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98499999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.752,"15min":0.753},"min":{"1min":0.63700000000000001,"5min":0.54500000000000004,"15min":0.43099999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.0209999999999999},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65500000000000003,"15min":0.63100000000000001},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.94699999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":0.94399999999999995,"15min":1.4590000000000001},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.72199999999999998,"15min":0.69999999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.38},"max":{"1min":1.0329999999999999,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70499999999999996,"15min":0.68899999999999995},"min":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.40000000000000002},"max":{"1min":0.997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.74299999999999999},"min":{"1min":0.53600000000000003,"5min":0.46400000000000002,"15min":0.438},"max":{"1min":1.052,"5min":1.052,"15min":1.5349999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.502,"5min":0.49199999999999999,"15min":0.42299999999999999},"max":{"1min":1.0169999999999999,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.48299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":810526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.74299999999999999,"15min":0.69899999999999995},"min":{"1min":0.55300000000000005,"5min":0.32800000000000001,"15min":0.29399999999999998},"max":{"1min":0.95899999999999996,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.749,"15min":0.746},"min":{"1min":0.53600000000000003,"5min":0.32400000000000001,"15min":0.26900000000000002},"max":{"1min":0.91000000000000003,"5min":3.3119999999999998,"15min":3.3399999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.51700000000000002,"5min":0.372,"15min":0.34599999999999997},"max":{"1min":0.83399999999999996,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.70099999999999996},"min":{"1min":0.52300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96499999999999997,"5min":5.96,"15min":5.96},"last":0.63}]},{"osd":2,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77600000000000002,"15min":0.71299999999999997},"min":{"1min":0.56899999999999995,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.93899999999999995,"5min":8.3740000000000006,"15min":8.3740000000000006},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.77800000000000002,"15min":0.74199999999999999},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":5.8449999999999998,"15min":5.8449999999999998},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":0.96599999999999997,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.80800000000000005,"15min":0.75},"min":{"1min":0.58099999999999996,"5min":0.44500000000000001,"15min":0.373},"max":{"1min":0.874,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.33200000000000002},"max":{"1min":0.83699999999999997,"5min":5.7759999999999998,"15min":5.7759999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.46000000000000002,"5min":0.42399999999999999,"15min":0.39200000000000002},"max":{"1min":0.93400000000000005,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.83399999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.74199999999999999,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.38600000000000001},"max":{"1min":0.86799999999999999,"5min":8.4870000000000001,"15min":8.4870000000000001},"last":0.875},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":0.86499999999999999,"5min":5.923,"15min":5.923},"last":0.94999999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76100000000000001,"15min":0.77600000000000002},"min":{"1min":0.53900000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90300000000000002,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.81599999999999995,"15min":0.80000000000000004},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93700000000000006,"5min":8.4429999999999996,"15min":8.4429999999999996},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2312,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2367488,"data_stored":1406812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41599999999999998,"15min":0.46999999999999997},"min":{"1min":0.30199999999999999,"5min":0.26200000000000001,"15min":0.23699999999999999},"max":{"1min":0.53100000000000003,"5min":0.69499999999999995,"15min":2.8879999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.53200000000000003},"min":{"1min":0.40500000000000003,"5min":0.34999999999999998,"15min":0.313},"max":{"1min":0.70499999999999996,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.623}]},{"osd":2,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.64300000000000002},"min":{"1min":0.34599999999999997,"5min":0.28199999999999997,"15min":0.26500000000000001},"max":{"1min":0.58699999999999997,"5min":3.3690000000000002,"15min":13.055999999999999},"last":0.432},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.58099999999999996},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.66600000000000004,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.504,"15min":0.55500000000000005},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48699999999999999,"15min":0.51100000000000001},"min":{"1min":0.35999999999999999,"5min":0.318,"15min":0.29499999999999998},"max":{"1min":0.80100000000000005,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69899999999999995,"15min":0.90900000000000003},"min":{"1min":0.61199999999999999,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.444,"15min":12.989000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.747,"15min":0.78900000000000003},"min":{"1min":0.627,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":1.1020000000000001,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.879}]},{"osd":5,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.70099999999999996,"15min":0.74099999999999999},"min":{"1min":0.59299999999999997,"5min":0.44700000000000001,"15min":0.432},"max":{"1min":1.0429999999999999,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.90400000000000003},"min":{"1min":0.50600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.01,"5min":1.494,"15min":13.023999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.65600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99399999999999999,"5min":2.9670000000000001,"15min":2.9670000000000001},"last":0.746},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.754,"15min":0.95399999999999996},"min":{"1min":0.70599999999999996,"5min":0.47899999999999998,"15min":0.47799999999999998},"max":{"1min":1.012,"5min":2.5409999999999999,"15min":12.941000000000001},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.74099999999999999,"15min":0.78200000000000003},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.44700000000000001},"max":{"1min":1.1799999999999999,"5min":2.431,"15min":2.806},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.65500000000000003,"5min":0.50600000000000001,"15min":0.48399999999999999},"max":{"1min":1.1439999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.86599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575008,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.60699999999999998,"5min":0.70699999999999996,"15min":0.753},"last":0.432},{"interface":"front","average":{"1min":0.499,"5min":0.436,"15min":0.44},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.63,"5min":0.63,"15min":0.81200000000000006},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.73699999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.48899999999999999},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.314},"max":{"1min":0.65700000000000003,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47299999999999998,"15min":0.48899999999999999},"min":{"1min":0.33200000000000002,"5min":0.32200000000000001,"15min":0.29899999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.81200000000000006},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.55500000000000005},"min":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.38800000000000001},"max":{"1min":0.67300000000000004,"5min":0.77600000000000002,"15min":0.84399999999999997},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.72299999999999998},"min":{"1min":0.55800000000000005,"5min":0.48399999999999999,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.171},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52500000000000002,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":1.0109999999999999,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.52000000000000002,"5min":0.48099999999999998,"15min":0.45600000000000002},"max":{"1min":0.94899999999999995,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46999999999999997},"max":{"1min":1.032,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.749}]},{"osd":6,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.54600000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.093,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46200000000000002},"max":{"1min":0.94999999999999996,"5min":1.117,"15min":1.546},"last":0.621}]},{"osd":7,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77600000000000002,"15min":0.79000000000000004},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":1.0800000000000001,"5min":2.778,"15min":2.778},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.76600000000000001,"15min":0.89700000000000002},"min":{"1min":0.61499999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0680000000000001,"5min":3.7069999999999999,"15min":9.5960000000000001},"last":0.76400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575007,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40972,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330868,"statfs":{"total":96636764160,"available":96594808832,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47399999999999998,"15min":0.44400000000000001},"min":{"1min":0.33300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65100000000000002,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45800000000000002,"15min":0.47599999999999998},"min":{"1min":0.28699999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.628,"5min":1.093,"15min":2.4630000000000001},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.67900000000000005,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77800000000000002,"5min":3.657,"15min":3.657},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.56599999999999995},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.33900000000000002},"max":{"1min":0.68000000000000005,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76300000000000001,"5min":2.1629999999999998,"15min":2.7679999999999998},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.91400000000000003,"15min":0.79700000000000004},"min":{"1min":0.504,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.99299999999999999,"5min":12.007,"15min":12.007},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.48399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0529999999999999,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.79200000000000004,"15min":0.755},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":0.90400000000000003,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.79900000000000004},"min":{"1min":0.55900000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0660000000000001,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.873}]},{"osd":6,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.91400000000000003,"15min":0.80100000000000005},"min":{"1min":0.53500000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.022,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.90500000000000003,"15min":0.81399999999999995},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":1.0069999999999999,"5min":13.084,"15min":13.084},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81699999999999995,"15min":0.81599999999999995},"min":{"1min":0.56799999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0700000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.81499999999999995},"min":{"1min":0.61799999999999999,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.079,"5min":4.484,"15min":4.484},"last":0.96899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":810526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.435,"15min":0.50600000000000001},"min":{"1min":0.34799999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.72399999999999998,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.505},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.89600000000000002,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.64500000000000002}]},{"osd":1,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.44,"15min":0.69399999999999995},"min":{"1min":0.308,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68000000000000005,"5min":1.3620000000000001,"15min":15.911},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.52600000000000002,"15min":0.77100000000000002},"min":{"1min":0.432,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":1.347,"15min":16.024000000000001},"last":0.621}]},{"osd":2,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.35099999999999998,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":0.67900000000000005,"5min":3.25,"15min":3.25},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.53200000000000003,"15min":0.77300000000000002},"min":{"1min":0.39400000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85399999999999998,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.71899999999999997,"15min":0.73899999999999999},"min":{"1min":0.627,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0329999999999999,"5min":1.782,"15min":1.782},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.73999999999999999,"15min":0.81799999999999995},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.36699999999999999},"max":{"1min":1.0229999999999999,"5min":1.724,"15min":5.7300000000000004},"last":1.0269999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.746,"15min":0.82499999999999996},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.39700000000000002},"max":{"1min":1.077,"5min":1.411,"15min":5.077},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.77200000000000002,"15min":0.86199999999999999},"min":{"1min":0.69099999999999995,"5min":0.52200000000000002,"15min":0.44800000000000001},"max":{"1min":1.0609999999999999,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.749,"15min":0.77100000000000002},"min":{"1min":0.58899999999999997,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.089,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.746,"15min":0.82799999999999996},"min":{"1min":0.63500000000000001,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.1319999999999999,"5min":1.4490000000000001,"15min":5.157},"last":1.1759999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.75800000000000001,"15min":0.78700000000000003},"min":{"1min":0.66400000000000003,"5min":0.51700000000000002,"15min":0.47399999999999998},"max":{"1min":1.073,"5min":1.863,"15min":1.863},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.75800000000000001,"15min":0.84099999999999997},"min":{"1min":0.64100000000000001,"5min":0.51100000000000001,"15min":0.49399999999999999},"max":{"1min":1.179,"5min":1.492,"15min":5.1189999999999998},"last":1.2230000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1757184,"data_stored":810526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.66200000000000003,"15min":0.65500000000000003},"min":{"1min":0.55100000000000005,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.90700000000000003,"5min":0.999,"15min":1.784},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.56000000000000005,"5min":0.42099999999999999,"15min":0.38},"max":{"1min":0.878,"5min":0.95899999999999996,"15min":1.0509999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.39500000000000002,"15min":0.38},"max":{"1min":0.89700000000000002,"5min":1.0900000000000001,"15min":6.258},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.41399999999999998,"15min":0.35899999999999999},"max":{"1min":0.98399999999999999,"5min":1.0700000000000001,"15min":1.806},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69499999999999995,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.40600000000000003,"15min":0.29199999999999998},"max":{"1min":1.01,"5min":1.012,"15min":6.4740000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.82999999999999996},"min":{"1min":0.66000000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":0.98599999999999999,"5min":0.99399999999999999,"15min":6.4859999999999998},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68799999999999994},"min":{"1min":0.58099999999999996,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.0309999999999999,"15min":1.25},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.49099999999999999,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.413},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.627},"min":{"1min":0.44,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.3280000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.99299999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.629},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.76500000000000001},"min":{"1min":0.60199999999999998,"5min":0.5,"15min":0.47299999999999998},"max":{"1min":0.93600000000000005,"5min":0.96299999999999997,"15min":1.6839999999999999},"last":0.755}]},{"osd":7,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.48099999999999998,"5min":0.442,"15min":0.41299999999999998},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":1.036,"15min":1.21},"last":0.78600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37452,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334388,"statfs":{"total":96636764160,"available":96598413312,"internally_reserved":0,"allocated":2363392,"data_stored":1405793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.436,"5min":0.40200000000000002,"15min":0.32100000000000001},"max":{"1min":0.83299999999999996,"5min":0.92900000000000005,"15min":4.5010000000000003},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66200000000000003,"15min":0.746},"min":{"1min":0.57899999999999996,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.83999999999999997,"5min":0.99299999999999999,"15min":6.4059999999999997},"last":0.82999999999999996}]},{"osd":1,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.307},"max":{"1min":0.92500000000000004,"5min":1.121,"15min":1.121},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.41399999999999998},"max":{"1min":0.93600000000000005,"5min":0.97299999999999998,"15min":1.546},"last":0.81799999999999995}]},{"osd":2,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.78700000000000003,"15min":0.72899999999999998},"min":{"1min":0.54100000000000004,"5min":0.49399999999999999,"15min":0.376},"max":{"1min":0.91000000000000003,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67900000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":1.069,"15min":1.069},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.78900000000000003,"15min":0.73799999999999999},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.42199999999999999},"max":{"1min":0.88400000000000001,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":0.93200000000000005,"5min":1.3140000000000001,"15min":4.484},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.995,"5min":1.0229999999999999,"15min":1.6240000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.379},"max":{"1min":0.95899999999999996,"5min":0.98699999999999999,"15min":2.718},"last":0.89400000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68500000000000005,"15min":0.68400000000000005},"min":{"1min":0.50600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.97399999999999998,"15min":2.5249999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.437,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.028,"5min":1.036,"15min":1.669},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73599999999999999,"15min":0.748},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.438},"max":{"1min":1.006,"5min":1.0529999999999999,"15min":2.0459999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74099999999999999,"15min":0.76200000000000001},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92700000000000005,"5min":1.034,"15min":2.2919999999999998},"last":0.95599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:57:59.287 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T03:57:59.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.478+0000 7f2a629ff700 1 Processor -- start 2023-07-06T03:57:59.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.478+0000 7f2a629ff700 1 -- start start 2023-07-06T03:57:59.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.479+0000 7f2a629ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a5c0cfe30 0x7f2a5c0cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.479+0000 7f2a629ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2a5c0b2190 0x7f2a5c0d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.479+0000 7f2a629ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5c0d12f0 con 0x7f2a5c0b8470 2023-07-06T03:57:59.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.479+0000 7f2a629ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c0b8840 con 0x7f2a5c0cfe30 2023-07-06T03:57:59.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.479+0000 7f2a629ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c0cf390 con 0x7f2a5c0b2190 2023-07-06T03:57:59.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.480+0000 7f2a5bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2a5c0b8470 0x7f2a5c0cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59294/0 (socket says 172.21.15.97:59294) 2023-07-06T03:57:59.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.480+0000 7f2a5b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a5c0cfe30 0x7f2a5c0cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.480+0000 7f2a5bfff700 1 -- 172.21.15.97:0/140853934 learned_addr learned my addr 172.21.15.97:0/140853934 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:57:59.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.480+0000 7f2a60f9c700 1 --2- 172.21.15.97:0/140853934 >> v2:172.21.15.146:3300/0 conn(0x7f2a5c0b2190 0x7f2a5c0d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.480+0000 7f2a5b7fe700 1 -- 172.21.15.97:0/140853934 >> v2:172.21.15.146:3300/0 conn(0x7f2a5c0b2190 msgr2=0x7f2a5c0d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:57:59.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.480+0000 7f2a5b7fe700 1 --2- 172.21.15.97:0/140853934 >> v2:172.21.15.146:3300/0 conn(0x7f2a5c0b2190 0x7f2a5c0d2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.480+0000 7f2a5b7fe700 1 -- 172.21.15.97:0/140853934 >> v1:172.21.15.97:6789/0 conn(0x7f2a5c0b8470 legacy=0x7f2a5c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:59.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.480+0000 7f2a5b7fe700 1 -- 172.21.15.97:0/140853934 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c0b8ca0 con 0x7f2a5c0cfe30 2023-07-06T03:57:59.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.480+0000 7f2a5affd700 1 -- 172.21.15.97:0/140853934 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83266374 0 0) 0x7f2a5c0d12f0 con 0x7f2a5c0b8470 2023-07-06T03:57:59.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a5b7fe700 1 --2- 172.21.15.97:0/140853934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0cfe30 0x7f2a5c0cee50 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0d0200 tx=0x7f2a50009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109c880166d867a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:59.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a5affd700 1 -- 172.21.15.97:0/140853934 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011830 con 0x7f2a5c0cfe30 2023-07-06T03:57:59.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a5affd700 1 -- 172.21.15.97:0/140853934 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50011990 con 0x7f2a5c0cfe30 2023-07-06T03:57:59.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a5affd700 1 -- 172.21.15.97:0/140853934 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011d20 con 0x7f2a5c0cfe30 2023-07-06T03:57:59.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a629ff700 1 -- 172.21.15.97:0/140853934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0cfe30 msgr2=0x7f2a5c0cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:59.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a629ff700 1 --2- 172.21.15.97:0/140853934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0cfe30 0x7f2a5c0cee50 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0d0200 tx=0x7f2a50009b20 comp rx=0 tx=0).stop 2023-07-06T03:57:59.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a629ff700 1 -- 172.21.15.97:0/140853934 shutdown_connections 2023-07-06T03:57:59.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a629ff700 1 --2- 172.21.15.97:0/140853934 >> v2:172.21.15.146:3300/0 conn(0x7f2a5c0b2190 0x7f2a5c0d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a629ff700 1 --2- 172.21.15.97:0/140853934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0cfe30 0x7f2a5c0cee50 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a629ff700 1 -- 172.21.15.97:0/140853934 >> 172.21.15.97:0/140853934 conn(0x7f2a5c071740 msgr2=0x7f2a5c071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:59.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a629ff700 1 -- 172.21.15.97:0/140853934 shutdown_connections 2023-07-06T03:57:59.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.481+0000 7f2a629ff700 1 -- 172.21.15.97:0/140853934 wait complete. 2023-07-06T03:57:59.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.483+0000 7f2a629ff700 1 Processor -- start 2023-07-06T03:57:59.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.483+0000 7f2a629ff700 1 -- start start 2023-07-06T03:57:59.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.484+0000 7f2a629ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a5c0b2190 0x7f2a5c1532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.484+0000 7f2a629ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b8470 0x7f2a5c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.484+0000 7f2a629ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a5c0cfe30 0x7f2a5c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.484+0000 7f2a629ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c15fbc0 con 0x7f2a5c0b8470 2023-07-06T03:57:59.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.484+0000 7f2a629ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c15ef40 con 0x7f2a5c0cfe30 2023-07-06T03:57:59.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.484+0000 7f2a629ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c15f220 con 0x7f2a5c0b2190 2023-07-06T03:57:59.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.484+0000 7f2a5b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b8470 0x7f2a5c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.484+0000 7f2a5b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b8470 0x7f2a5c15e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37716/0 (socket says 172.21.15.97:37716) 2023-07-06T03:57:59.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.484+0000 7f2a5b7fe700 1 -- 172.21.15.97:0/3540934280 learned_addr learned my addr 172.21.15.97:0/3540934280 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:57:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a5bfff700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a5c0b2190 0x7f2a5c1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a60f9c700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a5c0cfe30 0x7f2a5c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a5b7fe700 1 -- 172.21.15.97:0/3540934280 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a5c0b2190 msgr2=0x7f2a5c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a5b7fe700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a5c0b2190 0x7f2a5c1532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a5b7fe700 1 -- 172.21.15.97:0/3540934280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a5c0cfe30 msgr2=0x7f2a5c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a5b7fe700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a5c0cfe30 0x7f2a5c0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a5b7fe700 1 -- 172.21.15.97:0/3540934280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a500094a0 con 0x7f2a5c0b8470 2023-07-06T03:57:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a5bfff700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a5c0b2190 0x7f2a5c1532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:57:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a5b7fe700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b8470 0x7f2a5c15e580 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f2a50014040 tx=0x7f2a50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a58ff9700 1 -- 172.21.15.97:0/3540934280 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a500038a0 con 0x7f2a5c0b8470 2023-07-06T03:57:59.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a58ff9700 1 -- 172.21.15.97:0/3540934280 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5001e070 con 0x7f2a5c0b8470 2023-07-06T03:57:59.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a629ff700 1 -- 172.21.15.97:0/3540934280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c15f500 con 0x7f2a5c0b8470 2023-07-06T03:57:59.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a58ff9700 1 -- 172.21.15.97:0/3540934280 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50003c20 con 0x7f2a5c0b8470 2023-07-06T03:57:59.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.485+0000 7f2a629ff700 1 -- 172.21.15.97:0/3540934280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c1537f0 con 0x7f2a5c0b8470 2023-07-06T03:57:59.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.486+0000 7f2a629ff700 1 -- 172.21.15.97:0/3540934280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c0672d0 con 0x7f2a5c0b8470 2023-07-06T03:57:59.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.487+0000 7f2a58ff9700 1 -- 172.21.15.97:0/3540934280 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a5002d450 con 0x7f2a5c0b8470 2023-07-06T03:57:59.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.488+0000 7f2a58ff9700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a4806ace0 0x7f2a4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:57:59.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.488+0000 7f2a58ff9700 1 -- 172.21.15.97:0/3540934280 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2a5008a1a0 con 0x7f2a5c0b8470 2023-07-06T03:57:59.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.488+0000 7f2a5bfff700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a4806ace0 0x7f2a4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:57:59.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.489+0000 7f2a5bfff700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a4806ace0 0x7f2a4806d190 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c000f60 tx=0x7f2a4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:57:59.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.492+0000 7f2a58ff9700 1 -- 172.21.15.97:0/3540934280 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2a50055560 con 0x7f2a5c0b8470 2023-07-06T03:57:59.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.685+0000 7f2a629ff700 1 -- 172.21.15.97:0/3540934280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a5c15fd00 con 0x7f2a5c0b8470 2023-07-06T03:57:59.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.686+0000 7f2a58ff9700 1 -- 172.21.15.97:0/3540934280 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+14804 (secure 0 0 0) 0x7f2a5c15fd00 con 0x7f2a5c0b8470 2023-07-06T03:57:59.689 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:57:59.690 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":141,"fsid":"eb623ba8-cc89-4cff-a8d2-c5e65cbc5c36","created":"2023-07-06T03:44:43.770758+0000","modified":"2023-07-06T03:57:54.952179+0000","last_up_change":"2023-07-06T03:44:55.910794+0000","last_in_change":"2023-07-06T03:44:47.208174+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T03:44:56.940879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T03:45:03.582725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aad6f41f-a3bc-4db9-a290-9a0fb9f44f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6801","nonce":3690379650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6803","nonce":3690379650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6807","nonce":3690379650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3690379650},{"type":"v1","addr":"172.21.15.97:6805","nonce":3690379650}]},"public_addr":"172.21.15.97:6801/3690379650","cluster_addr":"172.21.15.97:6803/3690379650","heartbeat_back_addr":"172.21.15.97:6807/3690379650","heartbeat_front_addr":"172.21.15.97:6805/3690379650","state":["exists","up"]},{"osd":1,"uuid":"bfd65829-a141-4776-93fa-ec4a853e8b5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6810","nonce":3448212434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6814","nonce":3448212434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6822","nonce":3448212434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":3448212434},{"type":"v1","addr":"172.21.15.97:6818","nonce":3448212434}]},"public_addr":"172.21.15.97:6810/3448212434","cluster_addr":"172.21.15.97:6814/3448212434","heartbeat_back_addr":"172.21.15.97:6822/3448212434","heartbeat_front_addr":"172.21.15.97:6818/3448212434","state":["exists","up"]},{"osd":2,"uuid":"236bb1dc-a348-413d-aac2-91bfceb3d132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6825","nonce":3131825859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6827","nonce":3131825859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6831","nonce":3131825859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3131825859},{"type":"v1","addr":"172.21.15.97:6829","nonce":3131825859}]},"public_addr":"172.21.15.97:6825/3131825859","cluster_addr":"172.21.15.97:6827/3131825859","heartbeat_back_addr":"172.21.15.97:6831/3131825859","heartbeat_front_addr":"172.21.15.97:6829/3131825859","state":["exists","up"]},{"osd":3,"uuid":"d6f4576e-fddf-480f-8a3d-535999a6363d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6811","nonce":1358667479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6815","nonce":1358667479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6823","nonce":1358667479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":1358667479},{"type":"v1","addr":"172.21.15.97:6819","nonce":1358667479}]},"public_addr":"172.21.15.97:6811/1358667479","cluster_addr":"172.21.15.97:6815/1358667479","heartbeat_back_addr":"172.21.15.97:6823/1358667479","heartbeat_front_addr":"172.21.15.97:6819/1358667479","state":["exists","up"]},{"osd":4,"uuid":"e5571845-c703-4465-9293-e87425028856","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6801","nonce":663749144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6803","nonce":663749144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6807","nonce":663749144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":663749144},{"type":"v1","addr":"172.21.15.146:6805","nonce":663749144}]},"public_addr":"172.21.15.146:6801/663749144","cluster_addr":"172.21.15.146:6803/663749144","heartbeat_back_addr":"172.21.15.146:6807/663749144","heartbeat_front_addr":"172.21.15.146:6805/663749144","state":["exists","up"]},{"osd":5,"uuid":"b45633e5-ab66-45fc-8b50-b90c1d5f38c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6809","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6812","nonce":3757512297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6817","nonce":3757512297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6823","nonce":3757512297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6819","nonce":3757512297},{"type":"v1","addr":"172.21.15.146:6821","nonce":3757512297}]},"public_addr":"172.21.15.146:6812/3757512297","cluster_addr":"172.21.15.146:6817/3757512297","heartbeat_back_addr":"172.21.15.146:6823/3757512297","heartbeat_front_addr":"172.21.15.146:6821/3757512297","state":["exists","up"]},{"osd":6,"uuid":"d938c560-a4b1-4d1a-a6a1-422594d32887","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6825","nonce":1310854864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6827","nonce":1310854864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6831","nonce":1310854864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1310854864},{"type":"v1","addr":"172.21.15.146:6829","nonce":1310854864}]},"public_addr":"172.21.15.146:6825/1310854864","cluster_addr":"172.21.15.146:6827/1310854864","heartbeat_back_addr":"172.21.15.146:6831/1310854864","heartbeat_front_addr":"172.21.15.146:6829/1310854864","state":["exists","up"]},{"osd":7,"uuid":"18496b76-0c09-484a-b379-90e9c991a7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6810","nonce":1299847983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6813","nonce":1299847983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6820","nonce":1299847983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6815","nonce":1299847983},{"type":"v1","addr":"172.21.15.146:6816","nonce":1299847983}]},"public_addr":"172.21.15.146:6810/1299847983","cluster_addr":"172.21.15.146:6813/1299847983","heartbeat_back_addr":"172.21.15.146:6820/1299847983","heartbeat_front_addr":"172.21.15.146:6816/1299847983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.763010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.860401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.756360+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.776830+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.747671+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.771711+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.787399+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T03:44:52.514047+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/2797569799":"2023-07-07T03:57:51.568995+0000","172.21.15.97:6836/2797569799":"2023-07-07T03:57:51.568995+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T03:57:59.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 -- 172.21.15.97:0/3540934280 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a4806ace0 msgr2=0x7f2a4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:59.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a4806ace0 0x7f2a4806d190 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c000f60 tx=0x7f2a4c009040 comp rx=0 tx=0).stop 2023-07-06T03:57:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 -- 172.21.15.97:0/3540934280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b8470 msgr2=0x7f2a5c15e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:57:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b8470 0x7f2a5c15e580 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f2a50014040 tx=0x7f2a50004750 comp rx=0 tx=0).stop 2023-07-06T03:57:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 -- 172.21.15.97:0/3540934280 shutdown_connections 2023-07-06T03:57:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2a4806ace0 0x7f2a4806d190 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a5c0cfe30 0x7f2a5c0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2a5c0b2190 0x7f2a5c1532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 --2- 172.21.15.97:0/3540934280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b8470 0x7f2a5c15e580 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:57:59.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 -- 172.21.15.97:0/3540934280 >> 172.21.15.97:0/3540934280 conn(0x7f2a5c071740 msgr2=0x7f2a5c0cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:57:59.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 -- 172.21.15.97:0/3540934280 shutdown_connections 2023-07-06T03:57:59.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:57:59.689+0000 7f2a629ff700 1 -- 172.21.15.97:0/3540934280 wait complete. 2023-07-06T03:58:00.711 INFO:tasks.ceph:Scrubbing osd.0 2023-07-06T03:58:00.711 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-07-06T03:58:00.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.917+0000 7fbee01ff700 1 Processor -- start 2023-07-06T03:58:00.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.918+0000 7fbee01ff700 1 -- start start 2023-07-06T03:58:00.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbee01ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbed80b47b0 0x7fbed80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:00.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbee01ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbed815c840 0x7fbed80b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:00.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbee01ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbed80d1ed0 0x7fbed80b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:00.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbee01ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed80d02b0 con 0x7fbed815c840 2023-07-06T03:58:00.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbee01ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed80cd420 con 0x7fbed80b47b0 2023-07-06T03:58:00.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbee01ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80b5600 con 0x7fbed80d1ed0 2023-07-06T03:58:00.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbedd79a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbed815c840 0x7fbed80b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:00.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbedd79a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbed815c840 0x7fbed80b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48198/0 (socket says 172.21.15.97:48198) 2023-07-06T03:58:00.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbedd79a700 1 -- 172.21.15.97:0/3999725951 learned_addr learned my addr 172.21.15.97:0/3999725951 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:00.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbeddf9b700 1 --2- 172.21.15.97:0/3999725951 >> v2:172.21.15.146:3300/0 conn(0x7fbed80b47b0 0x7fbed80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:00.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbede79c700 1 --2- 172.21.15.97:0/3999725951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbed80d1ed0 0x7fbed80b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:00.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.919+0000 7fbedd79a700 1 -- 172.21.15.97:0/3999725951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbed80d1ed0 msgr2=0x7fbed80b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:00.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.920+0000 7fbedd79a700 1 --2- 172.21.15.97:0/3999725951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbed80d1ed0 0x7fbed80b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:00.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.920+0000 7fbedd79a700 1 -- 172.21.15.97:0/3999725951 >> v2:172.21.15.146:3300/0 conn(0x7fbed80b47b0 msgr2=0x7fbed80cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:00.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.920+0000 7fbedd79a700 1 --2- 172.21.15.97:0/3999725951 >> v2:172.21.15.146:3300/0 conn(0x7fbed80b47b0 0x7fbed80cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:00.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.920+0000 7fbedd79a700 1 -- 172.21.15.97:0/3999725951 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed80d2420 con 0x7fbed815c840 2023-07-06T03:58:00.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.920+0000 7fbeddf9b700 1 --2- 172.21.15.97:0/3999725951 >> v2:172.21.15.146:3300/0 conn(0x7fbed80b47b0 0x7fbed80cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:00.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.920+0000 7fbede79c700 1 --2- 172.21.15.97:0/3999725951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbed80d1ed0 0x7fbed80b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:00.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.920+0000 7fbedd79a700 1 --2- 172.21.15.97:0/3999725951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed815c840 0x7fbed80b4b80 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fbed808c050 tx=0x7fbec8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767ba6eb862b8b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:00.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.920+0000 7fbedcf99700 1 -- 172.21.15.97:0/3999725951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec800b870 con 0x7fbed815c840 2023-07-06T03:58:00.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.920+0000 7fbedcf99700 1 -- 172.21.15.97:0/3999725951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec800b9d0 con 0x7fbed815c840 2023-07-06T03:58:00.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.920+0000 7fbedcf99700 1 -- 172.21.15.97:0/3999725951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec800bd60 con 0x7fbed815c840 2023-07-06T03:58:00.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.921+0000 7fbee01ff700 1 -- 172.21.15.97:0/3999725951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed815c840 msgr2=0x7fbed80b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:00.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.921+0000 7fbee01ff700 1 --2- 172.21.15.97:0/3999725951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed815c840 0x7fbed80b4b80 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fbed808c050 tx=0x7fbec8009bd0 comp rx=0 tx=0).stop 2023-07-06T03:58:00.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.921+0000 7fbee01ff700 1 -- 172.21.15.97:0/3999725951 shutdown_connections 2023-07-06T03:58:00.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.921+0000 7fbee01ff700 1 --2- 172.21.15.97:0/3999725951 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbed80d1ed0 0x7fbed80b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:00.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.921+0000 7fbee01ff700 1 --2- 172.21.15.97:0/3999725951 >> v2:172.21.15.146:3300/0 conn(0x7fbed80b47b0 0x7fbed80cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:00.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.921+0000 7fbee01ff700 1 --2- 172.21.15.97:0/3999725951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed815c840 0x7fbed80b4b80 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:00.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.921+0000 7fbee01ff700 1 -- 172.21.15.97:0/3999725951 >> 172.21.15.97:0/3999725951 conn(0x7fbed806a670 msgr2=0x7fbed806aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:00.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.921+0000 7fbee01ff700 1 -- 172.21.15.97:0/3999725951 shutdown_connections 2023-07-06T03:58:00.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.921+0000 7fbee01ff700 1 -- 172.21.15.97:0/3999725951 wait complete. 2023-07-06T03:58:00.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.922+0000 7fbee01ff700 1 Processor -- start 2023-07-06T03:58:00.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.923+0000 7fbee01ff700 1 -- start start 2023-07-06T03:58:00.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbee01ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbed80b47b0 0x7fbed80c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:00.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbee01ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed80d1ed0 0x7fbed814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:00.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbee01ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed815c840 0x7fbed80b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:00.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbee01ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed815ea10 con 0x7fbed815c840 2023-07-06T03:58:00.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbee01ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80c8a10 con 0x7fbed80d1ed0 2023-07-06T03:58:00.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbee01ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80c8730 con 0x7fbed80b47b0 2023-07-06T03:58:00.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbede79c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed815c840 0x7fbed80b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbede79c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed815c840 0x7fbed80b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48210/0 (socket says 172.21.15.97:48210) 2023-07-06T03:58:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbede79c700 1 -- 172.21.15.97:0/3929737923 learned_addr learned my addr 172.21.15.97:0/3929737923 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbede79c700 1 -- 172.21.15.97:0/3929737923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbed80b47b0 msgr2=0x7fbed80c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbede79c700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbed80b47b0 0x7fbed80c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbede79c700 1 -- 172.21.15.97:0/3929737923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed80d1ed0 msgr2=0x7fbed814f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:00.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbede79c700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbed80d1ed0 0x7fbed814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:00.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbede79c700 1 -- 172.21.15.97:0/3929737923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec8009550 con 0x7fbed815c840 2023-07-06T03:58:00.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.924+0000 7fbede79c700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed815c840 0x7fbed80b38b0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fbed000cd50 tx=0x7fbed000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:00.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.925+0000 7fbece7fc700 1 -- 172.21.15.97:0/3929737923 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000f480 con 0x7fbed815c840 2023-07-06T03:58:00.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.925+0000 7fbece7fc700 1 -- 172.21.15.97:0/3929737923 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed000f600 con 0x7fbed815c840 2023-07-06T03:58:00.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.925+0000 7fbee01ff700 1 -- 172.21.15.97:0/3929737923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed815e3c0 con 0x7fbed815c840 2023-07-06T03:58:00.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.925+0000 7fbece7fc700 1 -- 172.21.15.97:0/3929737923 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000f9b0 con 0x7fbed815c840 2023-07-06T03:58:00.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.925+0000 7fbee01ff700 1 -- 172.21.15.97:0/3929737923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed814f5a0 con 0x7fbed815c840 2023-07-06T03:58:00.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.926+0000 7fbee01ff700 1 -- 172.21.15.97:0/3929737923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbed80cd6c0 con 0x7fbed815c840 2023-07-06T03:58:00.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.927+0000 7fbece7fc700 1 -- 172.21.15.97:0/3929737923 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbed000b040 con 0x7fbed815c840 2023-07-06T03:58:00.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.928+0000 7fbece7fc700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbebc06ace0 0x7fbebc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.928+0000 7fbece7fc700 1 -- 172.21.15.97:0/3929737923 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbed0089a80 con 0x7fbed815c840 2023-07-06T03:58:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.928+0000 7fbeddf9b700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbebc06ace0 0x7fbebc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.928+0000 7fbece7fc700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] conn(0x7fbebc071150 0x7fbebc073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.928+0000 7fbece7fc700 1 -- 172.21.15.97:0/3929737923 --> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbebc073c30 con 0x7fbebc071150 2023-07-06T03:58:00.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.928+0000 7fbece7fc700 1 -- 172.21.15.97:0/3929737923 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbed008e2a0 con 0x7fbed815c840 2023-07-06T03:58:00.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.928+0000 7fbedd79a700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] conn(0x7fbebc071150 0x7fbebc073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:00.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.928+0000 7fbeddf9b700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbebc06ace0 0x7fbebc06d190 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbed4006ba0 tx=0x7fbed4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:00.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.929+0000 7fbedd79a700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] conn(0x7fbebc071150 0x7fbebc073560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:00.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.929+0000 7fbece7fc700 1 -- 172.21.15.97:0/3929737923 <== osd.0 v2:172.21.15.97:6800/3690379650 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbebc073c30 con 0x7fbebc071150 2023-07-06T03:58:00.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.944+0000 7fbee01ff700 1 -- 172.21.15.97:0/3929737923 --> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbed80d2700 con 0x7fbebc071150 2023-07-06T03:58:00.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.944+0000 7fbece7fc700 1 -- 172.21.15.97:0/3929737923 <== osd.0 v2:172.21.15.97:6800/3690379650 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbec8006c60 con 0x7fbebc071150 2023-07-06T03:58:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:58:00.948 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T03:58:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:58:00.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.945+0000 7fbee01ff700 1 -- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] conn(0x7fbebc071150 msgr2=0x7fbebc073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:00.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.945+0000 7fbee01ff700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:6800/3690379650,v1:172.21.15.97:6801/3690379650] conn(0x7fbebc071150 0x7fbebc073560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:00.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.945+0000 7fbee01ff700 1 -- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbebc06ace0 msgr2=0x7fbebc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:00.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.945+0000 7fbee01ff700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbebc06ace0 0x7fbebc06d190 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbed4006ba0 tx=0x7fbed4008040 comp rx=0 tx=0).stop 2023-07-06T03:58:00.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.945+0000 7fbee01ff700 1 -- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed815c840 msgr2=0x7fbed80b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:00.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.945+0000 7fbee01ff700 1 --2- 172.21.15.97:0/3929737923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed815c840 0x7fbed80b38b0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fbed000cd50 tx=0x7fbed000a560 comp rx=0 tx=0).stop 2023-07-06T03:58:00.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.945+0000 7fbede79c700 1 -- 172.21.15.97:0/3929737923 reap_dead start 2023-07-06T03:58:00.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.945+0000 7fbee01ff700 1 -- 172.21.15.97:0/3929737923 shutdown_connections 2023-07-06T03:58:00.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.945+0000 7fbee01ff700 1 -- 172.21.15.97:0/3929737923 >> 172.21.15.97:0/3929737923 conn(0x7fbed806a670 msgr2=0x7fbed8159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:00.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.945+0000 7fbee01ff700 1 -- 172.21.15.97:0/3929737923 shutdown_connections 2023-07-06T03:58:00.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:00.945+0000 7fbee01ff700 1 -- 172.21.15.97:0/3929737923 wait complete. 2023-07-06T03:58:00.965 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-07-06T03:58:01.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.158+0000 7f54075ff700 1 Processor -- start 2023-07-06T03:58:01.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.158+0000 7f54075ff700 1 -- start start 2023-07-06T03:58:01.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.159+0000 7f54075ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54000b8470 0x7f54000cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.159+0000 7f54075ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f54000b2190 0x7f54000cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.159+0000 7f54075ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54000cfc60 0x7f5400160120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.159+0000 7f54075ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54000b8840 con 0x7f54000cfc60 2023-07-06T03:58:01.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.159+0000 7f54075ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54000d12f0 con 0x7f54000b2190 2023-07-06T03:58:01.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.159+0000 7f54075ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54000cf390 con 0x7f54000b8470 2023-07-06T03:58:01.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f5405b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54000cfc60 0x7f5400160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f5405b9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54000cfc60 0x7f5400160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48224/0 (socket says 172.21.15.97:48224) 2023-07-06T03:58:01.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f5405b9c700 1 -- 172.21.15.97:0/134492613 learned_addr learned my addr 172.21.15.97:0/134492613 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:01.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f540539b700 1 --2- 172.21.15.97:0/134492613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54000b8470 0x7f54000cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f5404b9a700 1 --2- 172.21.15.97:0/134492613 >> v2:172.21.15.146:3300/0 conn(0x7f54000b2190 0x7f54000cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f5405b9c700 1 -- 172.21.15.97:0/134492613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54000b8470 msgr2=0x7f54000cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:01.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f5405b9c700 1 --2- 172.21.15.97:0/134492613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54000b8470 0x7f54000cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f5405b9c700 1 -- 172.21.15.97:0/134492613 >> v2:172.21.15.146:3300/0 conn(0x7f54000b2190 msgr2=0x7f54000cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:01.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f5405b9c700 1 --2- 172.21.15.97:0/134492613 >> v2:172.21.15.146:3300/0 conn(0x7f54000b2190 0x7f54000cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f5405b9c700 1 -- 172.21.15.97:0/134492613 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54000d00a0 con 0x7f54000cfc60 2023-07-06T03:58:01.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f540539b700 1 --2- 172.21.15.97:0/134492613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54000b8470 0x7f54000cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f5404b9a700 1 --2- 172.21.15.97:0/134492613 >> v2:172.21.15.146:3300/0 conn(0x7f54000b2190 0x7f54000cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.160+0000 7f5405b9c700 1 --2- 172.21.15.97:0/134492613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54000cfc60 0x7f5400160120 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f54000d3de0 tx=0x7f53f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919ef06f6cfcefa5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f53f7fff700 1 -- 172.21.15.97:0/134492613 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f800e070 con 0x7f54000cfc60 2023-07-06T03:58:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f53f7fff700 1 -- 172.21.15.97:0/134492613 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f800b9c0 con 0x7f54000cfc60 2023-07-06T03:58:01.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f53f7fff700 1 -- 172.21.15.97:0/134492613 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f800bd70 con 0x7f54000cfc60 2023-07-06T03:58:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f54075ff700 1 -- 172.21.15.97:0/134492613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54000cfc60 msgr2=0x7f5400160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f54075ff700 1 --2- 172.21.15.97:0/134492613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54000cfc60 0x7f5400160120 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f54000d3de0 tx=0x7f53f800b330 comp rx=0 tx=0).stop 2023-07-06T03:58:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f54075ff700 1 -- 172.21.15.97:0/134492613 shutdown_connections 2023-07-06T03:58:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f54075ff700 1 --2- 172.21.15.97:0/134492613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54000cfc60 0x7f5400160120 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f54075ff700 1 --2- 172.21.15.97:0/134492613 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54000b8470 0x7f54000cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f54075ff700 1 --2- 172.21.15.97:0/134492613 >> v2:172.21.15.146:3300/0 conn(0x7f54000b2190 0x7f54000cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f54075ff700 1 -- 172.21.15.97:0/134492613 >> 172.21.15.97:0/134492613 conn(0x7f5400071740 msgr2=0x7f5400071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f54075ff700 1 -- 172.21.15.97:0/134492613 shutdown_connections 2023-07-06T03:58:01.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.161+0000 7f54075ff700 1 -- 172.21.15.97:0/134492613 wait complete. 2023-07-06T03:58:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.163+0000 7f54075ff700 1 Processor -- start 2023-07-06T03:58:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.163+0000 7f54075ff700 1 -- start start 2023-07-06T03:58:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.164+0000 7f54075ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54000b2190 0x7f540015f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.164+0000 7f54075ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54000b8470 0x7f54001f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.164+0000 7f54075ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54000cfc60 0x7f5400156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.164+0000 7f54075ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54001f6470 con 0x7f54000b8470 2023-07-06T03:58:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.164+0000 7f54075ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540015b080 con 0x7f54000b2190 2023-07-06T03:58:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.164+0000 7f54075ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5400159730 con 0x7f54000cfc60 2023-07-06T03:58:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.164+0000 7f5404b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54000b8470 0x7f54001f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.164+0000 7f5404b9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54000b8470 0x7f54001f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48238/0 (socket says 172.21.15.97:48238) 2023-07-06T03:58:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.164+0000 7f5404b9a700 1 -- 172.21.15.97:0/3580942452 learned_addr learned my addr 172.21.15.97:0/3580942452 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.164+0000 7f540539b700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54000b2190 0x7f540015f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.164+0000 7f5405b9c700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54000cfc60 0x7f5400156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.165+0000 7f5404b9a700 1 -- 172.21.15.97:0/3580942452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54000cfc60 msgr2=0x7f5400156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.165+0000 7f5404b9a700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54000cfc60 0x7f5400156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.165+0000 7f5404b9a700 1 -- 172.21.15.97:0/3580942452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54000b2190 msgr2=0x7f540015f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.165+0000 7f5404b9a700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54000b2190 0x7f540015f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.165+0000 7f5404b9a700 1 -- 172.21.15.97:0/3580942452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f8009b30 con 0x7f54000b8470 2023-07-06T03:58:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.165+0000 7f5404b9a700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54000b8470 0x7f54001f5f30 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f53f000c770 tx=0x7f53f000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.165+0000 7f53f5ffb700 1 -- 172.21.15.97:0/3580942452 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0011800 con 0x7f54000b8470 2023-07-06T03:58:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.165+0000 7f53f5ffb700 1 -- 172.21.15.97:0/3580942452 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f0011960 con 0x7f54000b8470 2023-07-06T03:58:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.165+0000 7f54075ff700 1 -- 172.21.15.97:0/3580942452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5400158b20 con 0x7f54000b8470 2023-07-06T03:58:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.165+0000 7f53f5ffb700 1 -- 172.21.15.97:0/3580942452 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0011cf0 con 0x7f54000b8470 2023-07-06T03:58:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.165+0000 7f54075ff700 1 -- 172.21.15.97:0/3580942452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54001590e0 con 0x7f54000b8470 2023-07-06T03:58:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.166+0000 7f54075ff700 1 -- 172.21.15.97:0/3580942452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540004e340 con 0x7f54000b8470 2023-07-06T03:58:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.167+0000 7f53f5ffb700 1 -- 172.21.15.97:0/3580942452 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f53f0021350 con 0x7f54000b8470 2023-07-06T03:58:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.168+0000 7f53f5ffb700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53e406ace0 0x7f53e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.168+0000 7f53f5ffb700 1 -- 172.21.15.97:0/3580942452 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f53f0089110 con 0x7f54000b8470 2023-07-06T03:58:01.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.168+0000 7f540539b700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53e406ace0 0x7f53e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.169+0000 7f540539b700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53e406ace0 0x7f53e406d190 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f53fc006fd0 tx=0x7f53fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:01.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.172+0000 7f53f5ffb700 1 -- 172.21.15.97:0/3580942452 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f53f0057ed0 con 0x7f54000b8470 2023-07-06T03:58:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.364+0000 7f54075ff700 1 -- 172.21.15.97:0/3580942452 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f54000b8ca0 con 0x7f53e406ace0 2023-07-06T03:58:01.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.364+0000 7f53f5ffb700 1 -- 172.21.15.97:0/3580942452 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f53fc00c050 con 0x7f53e406ace0 2023-07-06T03:58:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 -- 172.21.15.97:0/3580942452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53e406ace0 msgr2=0x7f53e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53e406ace0 0x7f53e406d190 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f53fc006fd0 tx=0x7f53fc008040 comp rx=0 tx=0).stop 2023-07-06T03:58:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 -- 172.21.15.97:0/3580942452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54000b8470 msgr2=0x7f54001f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54000b8470 0x7f54001f5f30 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f53f000c770 tx=0x7f53f000cb30 comp rx=0 tx=0).stop 2023-07-06T03:58:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 -- 172.21.15.97:0/3580942452 shutdown_connections 2023-07-06T03:58:01.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f53e406ace0 0x7f53e406d190 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f54000cfc60 0x7f5400156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54000b2190 0x7f540015f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 --2- 172.21.15.97:0/3580942452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54000b8470 0x7f54001f5f30 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 -- 172.21.15.97:0/3580942452 >> 172.21.15.97:0/3580942452 conn(0x7f5400071740 msgr2=0x7f54000d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:01.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 -- 172.21.15.97:0/3580942452 shutdown_connections 2023-07-06T03:58:01.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.367+0000 7f54075ff700 1 -- 172.21.15.97:0/3580942452 wait complete. 2023-07-06T03:58:01.372 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 0 to deep-scrub 2023-07-06T03:58:01.387 INFO:tasks.ceph:Scrubbing osd.1 2023-07-06T03:58:01.388 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-07-06T03:58:01.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.584+0000 7f94a05ff700 1 Processor -- start 2023-07-06T03:58:01.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.584+0000 7f94a05ff700 1 -- start start 2023-07-06T03:58:01.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.585+0000 7f94a05ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94980b47b0 0x7f94980cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.585+0000 7f94a05ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f94980d1ed0 0x7f94980b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.585+0000 7f94a05ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94980d02b0 con 0x7f949815c840 2023-07-06T03:58:01.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.585+0000 7f94a05ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94980cd420 con 0x7f94980d1ed0 2023-07-06T03:58:01.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.585+0000 7f94a05ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94980d2420 con 0x7f94980b47b0 2023-07-06T03:58:01.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949db9a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f949815c840 0x7f94980b4b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51432/0 (socket says 172.21.15.97:51432) 2023-07-06T03:58:01.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949db9a700 1 -- 172.21.15.97:0/2879794196 learned_addr learned my addr 172.21.15.97:0/2879794196 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:01.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949e39b700 1 --2- 172.21.15.97:0/2879794196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94980b47b0 0x7f94980cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949eb9c700 1 --2- 172.21.15.97:0/2879794196 >> v2:172.21.15.146:3300/0 conn(0x7f94980d1ed0 0x7f94980b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949d399700 1 -- 172.21.15.97:0/2879794196 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184972102 0 0) 0x7f9488001110 con 0x7f949815c840 2023-07-06T03:58:01.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949d399700 1 -- 172.21.15.97:0/2879794196 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94840035b0 con 0x7f949815c840 2023-07-06T03:58:01.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949eb9c700 1 -- 172.21.15.97:0/2879794196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94980b47b0 msgr2=0x7f94980cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:01.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949eb9c700 1 --2- 172.21.15.97:0/2879794196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94980b47b0 0x7f94980cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949d399700 1 -- 172.21.15.97:0/2879794196 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892760424 0 0) 0x7f94840035b0 con 0x7f949815c840 2023-07-06T03:58:01.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949eb9c700 1 -- 172.21.15.97:0/2879794196 >> v1:172.21.15.97:6789/0 conn(0x7f949815c840 legacy=0x7f94980b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949eb9c700 1 -- 172.21.15.97:0/2879794196 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94980d2700 con 0x7f94980d1ed0 2023-07-06T03:58:01.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.586+0000 7f949e39b700 1 --2- 172.21.15.97:0/2879794196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94980b47b0 0x7f94980cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:58:01.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.587+0000 7f949eb9c700 1 --2- 172.21.15.97:0/2879794196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94980d1ed0 0x7f94980b5290 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9490009d90 tx=0x7f949000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff4cb17f0b03655e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:01.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.587+0000 7f949d399700 1 -- 172.21.15.97:0/2879794196 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949000e070 con 0x7f94980d1ed0 2023-07-06T03:58:01.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.587+0000 7f949d399700 1 -- 172.21.15.97:0/2879794196 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9490003a20 con 0x7f94980d1ed0 2023-07-06T03:58:01.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.587+0000 7f949d399700 1 -- 172.21.15.97:0/2879794196 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488001800 con 0x7f94980d1ed0 2023-07-06T03:58:01.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.588+0000 7f94a05ff700 1 -- 172.21.15.97:0/2879794196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94980d1ed0 msgr2=0x7f94980b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.588+0000 7f94a05ff700 1 --2- 172.21.15.97:0/2879794196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94980d1ed0 0x7f94980b5290 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9490009d90 tx=0x7f949000b330 comp rx=0 tx=0).stop 2023-07-06T03:58:01.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.588+0000 7f94a05ff700 1 -- 172.21.15.97:0/2879794196 shutdown_connections 2023-07-06T03:58:01.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.588+0000 7f94a05ff700 1 --2- 172.21.15.97:0/2879794196 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94980d1ed0 0x7f94980b5290 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.588+0000 7f94a05ff700 1 --2- 172.21.15.97:0/2879794196 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94980b47b0 0x7f94980cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.588+0000 7f94a05ff700 1 -- 172.21.15.97:0/2879794196 >> 172.21.15.97:0/2879794196 conn(0x7f949806a670 msgr2=0x7f949806aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:01.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.588+0000 7f94a05ff700 1 -- 172.21.15.97:0/2879794196 shutdown_connections 2023-07-06T03:58:01.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.588+0000 7f94a05ff700 1 -- 172.21.15.97:0/2879794196 wait complete. 2023-07-06T03:58:01.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.590+0000 7f94a05ff700 1 Processor -- start 2023-07-06T03:58:01.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.590+0000 7f94a05ff700 1 -- start start 2023-07-06T03:58:01.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f94a05ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94980b47b0 0x7f949814f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f94a05ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94980d1ed0 0x7f9498157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f94a05ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f949815c840 0x7f94980b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f94a05ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949815da60 con 0x7f94980d1ed0 2023-07-06T03:58:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f94a05ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949815cde0 con 0x7f949815c840 2023-07-06T03:58:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f94a05ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949815d0c0 con 0x7f94980b47b0 2023-07-06T03:58:01.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949db9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94980d1ed0 0x7f9498157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949db9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94980d1ed0 0x7f9498157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48246/0 (socket says 172.21.15.97:48246) 2023-07-06T03:58:01.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949db9a700 1 -- 172.21.15.97:0/1649081573 learned_addr learned my addr 172.21.15.97:0/1649081573 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:01.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949e39b700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94980b47b0 0x7f949814f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949eb9c700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f949815c840 0x7f94980b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949db9a700 1 -- 172.21.15.97:0/1649081573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94980b47b0 msgr2=0x7f949814f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949db9a700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f94980b47b0 0x7f949814f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949db9a700 1 -- 172.21.15.97:0/1649081573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f949815c840 msgr2=0x7f94980b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949db9a700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f949815c840 0x7f94980b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949db9a700 1 -- 172.21.15.97:0/1649081573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940094a0 con 0x7f94980d1ed0 2023-07-06T03:58:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949eb9c700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f949815c840 0x7f94980b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:58:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.591+0000 7f949db9a700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94980d1ed0 0x7f9498157f80 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f948800c780 tx=0x7f948800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.592+0000 7f948effd700 1 -- 172.21.15.97:0/1649081573 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488004c80 con 0x7f94980d1ed0 2023-07-06T03:58:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.592+0000 7f948effd700 1 -- 172.21.15.97:0/1649081573 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9488004de0 con 0x7f94980d1ed0 2023-07-06T03:58:01.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.592+0000 7f94a05ff700 1 -- 172.21.15.97:0/1649081573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9490009a70 con 0x7f94980d1ed0 2023-07-06T03:58:01.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.592+0000 7f948effd700 1 -- 172.21.15.97:0/1649081573 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94880072f0 con 0x7f94980d1ed0 2023-07-06T03:58:01.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.592+0000 7f94a05ff700 1 -- 172.21.15.97:0/1649081573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949815d680 con 0x7f94980d1ed0 2023-07-06T03:58:01.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.593+0000 7f94a05ff700 1 -- 172.21.15.97:0/1649081573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94980d29e0 con 0x7f94980d1ed0 2023-07-06T03:58:01.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.594+0000 7f948effd700 1 -- 172.21.15.97:0/1649081573 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9488007450 con 0x7f94980d1ed0 2023-07-06T03:58:01.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.595+0000 7f948effd700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f947c06ace0 0x7f947c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.595+0000 7f948effd700 1 -- 172.21.15.97:0/1649081573 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9488088db0 con 0x7f94980d1ed0 2023-07-06T03:58:01.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.595+0000 7f949e39b700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f947c06ace0 0x7f947c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.595+0000 7f948effd700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] conn(0x7f947c071150 0x7f947c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.595+0000 7f948effd700 1 -- 172.21.15.97:0/1649081573 --> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f947c073c30 con 0x7f947c071150 2023-07-06T03:58:01.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.595+0000 7f948effd700 1 -- 172.21.15.97:0/1649081573 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7f948808b390 con 0x7f94980d1ed0 2023-07-06T03:58:01.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.595+0000 7f949eb9c700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] conn(0x7f947c071150 0x7f947c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.596+0000 7f949e39b700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f947c06ace0 0x7f947c06d190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9494009cd0 tx=0x7f9494009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:01.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.596+0000 7f949eb9c700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] conn(0x7f947c071150 0x7f947c073560 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:01.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.597+0000 7f948effd700 1 -- 172.21.15.97:0/1649081573 <== osd.1 v2:172.21.15.97:6808/3448212434 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f947c073c30 con 0x7f947c071150 2023-07-06T03:58:01.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.611+0000 7f94a05ff700 1 -- 172.21.15.97:0/1649081573 --> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f94980cd6c0 con 0x7f947c071150 2023-07-06T03:58:01.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f948effd700 1 -- 172.21.15.97:0/1649081573 <== osd.1 v2:172.21.15.97:6808/3448212434 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9490003cf0 con 0x7f947c071150 2023-07-06T03:58:01.615 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:58:01.615 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T03:58:01.615 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:58:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f94a05ff700 1 -- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] conn(0x7f947c071150 msgr2=0x7f947c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f94a05ff700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:6808/3448212434,v1:172.21.15.97:6810/3448212434] conn(0x7f947c071150 0x7f947c073560 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f94a05ff700 1 -- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f947c06ace0 msgr2=0x7f947c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f94a05ff700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f947c06ace0 0x7f947c06d190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9494009cd0 tx=0x7f9494009210 comp rx=0 tx=0).stop 2023-07-06T03:58:01.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f94a05ff700 1 -- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94980d1ed0 msgr2=0x7f9498157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f94a05ff700 1 --2- 172.21.15.97:0/1649081573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94980d1ed0 0x7f9498157f80 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f948800c780 tx=0x7f948800ca90 comp rx=0 tx=0).stop 2023-07-06T03:58:01.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f949eb9c700 1 -- 172.21.15.97:0/1649081573 reap_dead start 2023-07-06T03:58:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f94a05ff700 1 -- 172.21.15.97:0/1649081573 shutdown_connections 2023-07-06T03:58:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f94a05ff700 1 -- 172.21.15.97:0/1649081573 >> 172.21.15.97:0/1649081573 conn(0x7f949806a670 msgr2=0x7f94980cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f94a05ff700 1 -- 172.21.15.97:0/1649081573 shutdown_connections 2023-07-06T03:58:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.612+0000 7f94a05ff700 1 -- 172.21.15.97:0/1649081573 wait complete. 2023-07-06T03:58:01.632 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-07-06T03:58:01.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.825+0000 7f347a7ff700 1 Processor -- start 2023-07-06T03:58:01.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.825+0000 7f347a7ff700 1 -- start start 2023-07-06T03:58:01.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.826+0000 7f347a7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34740b8470 0x7f34740cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.826+0000 7f347a7ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f34740cfe30 0x7f34740d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.826+0000 7f347a7ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34740d12f0 con 0x7f34740b2190 2023-07-06T03:58:01.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.826+0000 7f347a7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34740b8840 con 0x7f34740b8470 2023-07-06T03:58:01.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.826+0000 7f347a7ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34740cf390 con 0x7f34740cfe30 2023-07-06T03:58:01.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.827+0000 7f3473fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34740b8470 0x7f34740cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.827+0000 7f3473fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34740b8470 0x7f34740cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48254/0 (socket says 172.21.15.97:48254) 2023-07-06T03:58:01.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.827+0000 7f3473fff700 1 -- 172.21.15.97:0/443791040 learned_addr learned my addr 172.21.15.97:0/443791040 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:01.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.827+0000 7f3478d9c700 1 --2- 172.21.15.97:0/443791040 >> v2:172.21.15.146:3300/0 conn(0x7f34740cfe30 0x7f34740d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.827+0000 7f3473fff700 1 -- 172.21.15.97:0/443791040 >> v2:172.21.15.146:3300/0 conn(0x7f34740cfe30 msgr2=0x7f34740d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:01.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.827+0000 7f3473fff700 1 --2- 172.21.15.97:0/443791040 >> v2:172.21.15.146:3300/0 conn(0x7f34740cfe30 0x7f34740d2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.827+0000 7f3473fff700 1 -- 172.21.15.97:0/443791040 >> v1:172.21.15.146:6789/0 conn(0x7f34740b2190 legacy=0x7f34740cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:01.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.827+0000 7f3473fff700 1 -- 172.21.15.97:0/443791040 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34740b8ca0 con 0x7f34740b8470 2023-07-06T03:58:01.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.827+0000 7f3473fff700 1 --2- 172.21.15.97:0/443791040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34740b8470 0x7f34740cee50 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f34740b4d00 tx=0x7f3464009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753e56334a2305e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:01.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.828+0000 7f34737fe700 1 -- 172.21.15.97:0/443791040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346400b870 con 0x7f34740b8470 2023-07-06T03:58:01.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.828+0000 7f34737fe700 1 -- 172.21.15.97:0/443791040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346400b9d0 con 0x7f34740b8470 2023-07-06T03:58:01.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.828+0000 7f34737fe700 1 -- 172.21.15.97:0/443791040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346400bd60 con 0x7f34740b8470 2023-07-06T03:58:01.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.828+0000 7f347a7ff700 1 -- 172.21.15.97:0/443791040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34740b8470 msgr2=0x7f34740cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.828+0000 7f347a7ff700 1 --2- 172.21.15.97:0/443791040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34740b8470 0x7f34740cee50 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f34740b4d00 tx=0x7f3464009bd0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.828+0000 7f347a7ff700 1 -- 172.21.15.97:0/443791040 shutdown_connections 2023-07-06T03:58:01.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.828+0000 7f347a7ff700 1 --2- 172.21.15.97:0/443791040 >> v2:172.21.15.146:3300/0 conn(0x7f34740cfe30 0x7f34740d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.828+0000 7f347a7ff700 1 --2- 172.21.15.97:0/443791040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34740b8470 0x7f34740cee50 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.828+0000 7f347a7ff700 1 -- 172.21.15.97:0/443791040 >> 172.21.15.97:0/443791040 conn(0x7f3474071740 msgr2=0x7f3474071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:01.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.828+0000 7f347a7ff700 1 -- 172.21.15.97:0/443791040 shutdown_connections 2023-07-06T03:58:01.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.828+0000 7f347a7ff700 1 -- 172.21.15.97:0/443791040 wait complete. 2023-07-06T03:58:01.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.830+0000 7f347a7ff700 1 Processor -- start 2023-07-06T03:58:01.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.830+0000 7f347a7ff700 1 -- start start 2023-07-06T03:58:01.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.831+0000 7f347a7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34740b2190 0x7f34741f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.831+0000 7f347a7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34740b8470 0x7f34740b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.831+0000 7f347a7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34740cfe30 0x7f34740b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.831+0000 7f347a7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34740c78a0 con 0x7f34740cfe30 2023-07-06T03:58:01.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.831+0000 7f347a7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3474146e10 con 0x7f34740b2190 2023-07-06T03:58:01.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.831+0000 7f347a7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34741470b0 con 0x7f34740b8470 2023-07-06T03:58:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f3478d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34740cfe30 0x7f34740b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f3478d9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34740cfe30 0x7f34740b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48268/0 (socket says 172.21.15.97:48268) 2023-07-06T03:58:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f3478d9c700 1 -- 172.21.15.97:0/403714834 learned_addr learned my addr 172.21.15.97:0/403714834 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f3473fff700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34740b2190 0x7f34741f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f346bfff700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34740b8470 0x7f34740b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f3478d9c700 1 -- 172.21.15.97:0/403714834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34740b8470 msgr2=0x7f34740b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f3478d9c700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34740b8470 0x7f34740b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f3478d9c700 1 -- 172.21.15.97:0/403714834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34740b2190 msgr2=0x7f34741f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f3478d9c700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34740b2190 0x7f34741f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f3478d9c700 1 -- 172.21.15.97:0/403714834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3464009550 con 0x7f34740cfe30 2023-07-06T03:58:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f3478d9c700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34740cfe30 0x7f34740b3fd0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f346c007e60 tx=0x7f346c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.832+0000 7f34717fa700 1 -- 172.21.15.97:0/403714834 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c00ed30 con 0x7f34740cfe30 2023-07-06T03:58:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.833+0000 7f34717fa700 1 -- 172.21.15.97:0/403714834 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c014420 con 0x7f34740cfe30 2023-07-06T03:58:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.833+0000 7f34717fa700 1 -- 172.21.15.97:0/403714834 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c0147d0 con 0x7f34740cfe30 2023-07-06T03:58:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.833+0000 7f347a7ff700 1 -- 172.21.15.97:0/403714834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3474147390 con 0x7f34740cfe30 2023-07-06T03:58:01.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.833+0000 7f347a7ff700 1 -- 172.21.15.97:0/403714834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347415db40 con 0x7f34740cfe30 2023-07-06T03:58:01.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.834+0000 7f347a7ff700 1 -- 172.21.15.97:0/403714834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347415dd00 con 0x7f34740cfe30 2023-07-06T03:58:01.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.835+0000 7f34717fa700 1 -- 172.21.15.97:0/403714834 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f346c014930 con 0x7f34740cfe30 2023-07-06T03:58:01.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.836+0000 7f34717fa700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f345406ace0 0x7f345406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:01.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.836+0000 7f34717fa700 1 -- 172.21.15.97:0/403714834 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f346c0897a0 con 0x7f34740cfe30 2023-07-06T03:58:01.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.836+0000 7f3473fff700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f345406ace0 0x7f345406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:01.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.837+0000 7f3473fff700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f345406ace0 0x7f345406d190 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3464000c00 tx=0x7f3464008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:01.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:01.840+0000 7f34717fa700 1 -- 172.21.15.97:0/403714834 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f347415dd00 con 0x7f34740cfe30 2023-07-06T03:58:02.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.034+0000 7f347a7ff700 1 -- 172.21.15.97:0/403714834 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f34740cd7e0 con 0x7f345406ace0 2023-07-06T03:58:02.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.035+0000 7f34717fa700 1 -- 172.21.15.97:0/403714834 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f346400bb40 con 0x7f345406ace0 2023-07-06T03:58:02.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 -- 172.21.15.97:0/403714834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f345406ace0 msgr2=0x7f345406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f345406ace0 0x7f345406d190 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3464000c00 tx=0x7f3464008030 comp rx=0 tx=0).stop 2023-07-06T03:58:02.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 -- 172.21.15.97:0/403714834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34740cfe30 msgr2=0x7f34740b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34740cfe30 0x7f34740b3fd0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f346c007e60 tx=0x7f346c00e3c0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 -- 172.21.15.97:0/403714834 shutdown_connections 2023-07-06T03:58:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f345406ace0 0x7f345406d190 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34740cfe30 0x7f34740b3fd0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34740b2190 0x7f34741f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 --2- 172.21.15.97:0/403714834 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f34740b8470 0x7f34740b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 -- 172.21.15.97:0/403714834 >> 172.21.15.97:0/403714834 conn(0x7f3474071740 msgr2=0x7f34740cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:02.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 -- 172.21.15.97:0/403714834 shutdown_connections 2023-07-06T03:58:02.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.037+0000 7f347a7ff700 1 -- 172.21.15.97:0/403714834 wait complete. 2023-07-06T03:58:02.043 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 1 to deep-scrub 2023-07-06T03:58:02.058 INFO:tasks.ceph:Scrubbing osd.2 2023-07-06T03:58:02.058 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-07-06T03:58:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.253+0000 7f20d37ff700 1 Processor -- start 2023-07-06T03:58:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.253+0000 7f20d37ff700 1 -- start start 2023-07-06T03:58:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.254+0000 7f20d37ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20cc0cc0b0 0x7f20cc0cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.254+0000 7f20d37ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f20cc06fb40 0x7f20cc0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.254+0000 7f20d37ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20cc159760 con 0x7f20cc0c8fc0 2023-07-06T03:58:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.254+0000 7f20d37ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20cc0c9c40 con 0x7f20cc0cc0b0 2023-07-06T03:58:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.254+0000 7f20d37ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20cc0cc920 con 0x7f20cc06fb40 2023-07-06T03:58:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.255+0000 7f20d0d9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20cc0cc0b0 0x7f20cc0cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.255+0000 7f20d0d9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20cc0cc0b0 0x7f20cc0cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48280/0 (socket says 172.21.15.97:48280) 2023-07-06T03:58:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.255+0000 7f20d0d9a700 1 -- 172.21.15.97:0/1734422094 learned_addr learned my addr 172.21.15.97:0/1734422094 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.255+0000 7f20d0d9a700 1 -- 172.21.15.97:0/1734422094 >> v2:172.21.15.146:3300/0 conn(0x7f20cc06fb40 msgr2=0x7f20cc0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.255+0000 7f20d1d9c700 1 --2- 172.21.15.97:0/1734422094 >> v2:172.21.15.146:3300/0 conn(0x7f20cc06fb40 0x7f20cc0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.255+0000 7f20d0d9a700 1 --2- 172.21.15.97:0/1734422094 >> v2:172.21.15.146:3300/0 conn(0x7f20cc06fb40 0x7f20cc0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.255+0000 7f20d0d9a700 1 -- 172.21.15.97:0/1734422094 >> v1:172.21.15.146:6789/0 conn(0x7f20cc0c8fc0 legacy=0x7f20cc0c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.255+0000 7f20d0d9a700 1 -- 172.21.15.97:0/1734422094 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc0ccc00 con 0x7f20cc0cc0b0 2023-07-06T03:58:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.255+0000 7f20d1d9c700 1 --2- 172.21.15.97:0/1734422094 >> v2:172.21.15.146:3300/0 conn(0x7f20cc06fb40 0x7f20cc0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.255+0000 7f20d0d9a700 1 --2- 172.21.15.97:0/1734422094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20cc0cc0b0 0x7f20cc0cb0d0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0c9aa0 tx=0x7f20bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ce8683c688c734 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.255+0000 7f20c3fff700 1 -- 172.21.15.97:0/1734422094 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc004230 con 0x7f20cc0cc0b0 2023-07-06T03:58:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.256+0000 7f20c3fff700 1 -- 172.21.15.97:0/1734422094 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc004390 con 0x7f20cc0cc0b0 2023-07-06T03:58:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.256+0000 7f20c3fff700 1 -- 172.21.15.97:0/1734422094 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0036a0 con 0x7f20cc0cc0b0 2023-07-06T03:58:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.256+0000 7f20d37ff700 1 -- 172.21.15.97:0/1734422094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20cc0cc0b0 msgr2=0x7f20cc0cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.256+0000 7f20d37ff700 1 --2- 172.21.15.97:0/1734422094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20cc0cc0b0 0x7f20cc0cb0d0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0c9aa0 tx=0x7f20bc009bd0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.256+0000 7f20d37ff700 1 -- 172.21.15.97:0/1734422094 shutdown_connections 2023-07-06T03:58:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.256+0000 7f20d37ff700 1 --2- 172.21.15.97:0/1734422094 >> v2:172.21.15.146:3300/0 conn(0x7f20cc06fb40 0x7f20cc0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.256+0000 7f20d37ff700 1 --2- 172.21.15.97:0/1734422094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20cc0cc0b0 0x7f20cc0cb0d0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.256+0000 7f20d37ff700 1 -- 172.21.15.97:0/1734422094 >> 172.21.15.97:0/1734422094 conn(0x7f20cc06a010 msgr2=0x7f20cc06a410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.256+0000 7f20d37ff700 1 -- 172.21.15.97:0/1734422094 shutdown_connections 2023-07-06T03:58:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.256+0000 7f20d37ff700 1 -- 172.21.15.97:0/1734422094 wait complete. 2023-07-06T03:58:02.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.258+0000 7f20d37ff700 1 Processor -- start 2023-07-06T03:58:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.258+0000 7f20d37ff700 1 -- start start 2023-07-06T03:58:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.259+0000 7f20d37ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20cc06fb40 0x7f20cc0c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.259+0000 7f20d37ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20cc0c8fc0 0x7f20cc1e3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.259+0000 7f20d37ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20cc0cc0b0 0x7f20cc158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.259+0000 7f20d37ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc1e3fc0 con 0x7f20cc06fb40 2023-07-06T03:58:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.259+0000 7f20d37ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc1e42a0 con 0x7f20cc0cc0b0 2023-07-06T03:58:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.259+0000 7f20d37ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc1e4580 con 0x7f20cc0c8fc0 2023-07-06T03:58:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.259+0000 7f20d159b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20cc06fb40 0x7f20cc0c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.259+0000 7f20d159b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20cc06fb40 0x7f20cc0c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48282/0 (socket says 172.21.15.97:48282) 2023-07-06T03:58:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.259+0000 7f20d159b700 1 -- 172.21.15.97:0/4063550680 learned_addr learned my addr 172.21.15.97:0/4063550680 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.259+0000 7f20d0d9a700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20cc0c8fc0 0x7f20cc1e3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.259+0000 7f20d1d9c700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20cc0cc0b0 0x7f20cc158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20d159b700 1 -- 172.21.15.97:0/4063550680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20cc0c8fc0 msgr2=0x7f20cc1e3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20d159b700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f20cc0c8fc0 0x7f20cc1e3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20d159b700 1 -- 172.21.15.97:0/4063550680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20cc0cc0b0 msgr2=0x7f20cc158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20d159b700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20cc0cc0b0 0x7f20cc158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20d159b700 1 -- 172.21.15.97:0/4063550680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20bc009550 con 0x7f20cc06fb40 2023-07-06T03:58:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20d1d9c700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20cc0cc0b0 0x7f20cc158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20d159b700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20cc06fb40 0x7f20cc0c7770 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f20c800c4d0 tx=0x7f20c800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20c1ffb700 1 -- 172.21.15.97:0/4063550680 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8011790 con 0x7f20cc06fb40 2023-07-06T03:58:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20c1ffb700 1 -- 172.21.15.97:0/4063550680 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c80118f0 con 0x7f20cc06fb40 2023-07-06T03:58:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20d37ff700 1 -- 172.21.15.97:0/4063550680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20cc0ae1c0 con 0x7f20cc06fb40 2023-07-06T03:58:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20c1ffb700 1 -- 172.21.15.97:0/4063550680 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8011ca0 con 0x7f20cc06fb40 2023-07-06T03:58:02.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.260+0000 7f20d37ff700 1 -- 172.21.15.97:0/4063550680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20cc0c7e90 con 0x7f20cc06fb40 2023-07-06T03:58:02.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.261+0000 7f20d37ff700 1 -- 172.21.15.97:0/4063550680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f20cc0ae040 con 0x7f20cc06fb40 2023-07-06T03:58:02.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.262+0000 7f20c1ffb700 1 -- 172.21.15.97:0/4063550680 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20c801c460 con 0x7f20cc06fb40 2023-07-06T03:58:02.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.263+0000 7f20c1ffb700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f20b006ace0 0x7f20b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.263+0000 7f20c1ffb700 1 -- 172.21.15.97:0/4063550680 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f20c8088de0 con 0x7f20cc06fb40 2023-07-06T03:58:02.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.263+0000 7f20d0d9a700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f20b006ace0 0x7f20b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.263+0000 7f20c1ffb700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] conn(0x7f20b0071150 0x7f20b0073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.263+0000 7f20c1ffb700 1 -- 172.21.15.97:0/4063550680 --> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20b0073c30 con 0x7f20b0071150 2023-07-06T03:58:02.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.263+0000 7f20c1ffb700 1 -- 172.21.15.97:0/4063550680 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20c808b3c0 con 0x7f20cc06fb40 2023-07-06T03:58:02.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.263+0000 7f20d1d9c700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] conn(0x7f20b0071150 0x7f20b0073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.264+0000 7f20d0d9a700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f20b006ace0 0x7f20b006d190 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009130 tx=0x7f20bc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:02.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.264+0000 7f20d1d9c700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] conn(0x7f20b0071150 0x7f20b0073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:02.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.265+0000 7f20c1ffb700 1 -- 172.21.15.97:0/4063550680 <== osd.2 v2:172.21.15.97:6824/3131825859 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f20b0073c30 con 0x7f20b0071150 2023-07-06T03:58:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.279+0000 7f20d37ff700 1 -- 172.21.15.97:0/4063550680 --> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f20cc0cd1c0 con 0x7f20b0071150 2023-07-06T03:58:02.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20c1ffb700 1 -- 172.21.15.97:0/4063550680 <== osd.2 v2:172.21.15.97:6824/3131825859 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f20c400d490 con 0x7f20b0071150 2023-07-06T03:58:02.283 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:58:02.283 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T03:58:02.283 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:58:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20d37ff700 1 -- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] conn(0x7f20b0071150 msgr2=0x7f20b0073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20d37ff700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:6824/3131825859,v1:172.21.15.97:6825/3131825859] conn(0x7f20b0071150 0x7f20b0073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20d37ff700 1 -- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f20b006ace0 msgr2=0x7f20b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20d37ff700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f20b006ace0 0x7f20b006d190 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009130 tx=0x7f20bc008030 comp rx=0 tx=0).stop 2023-07-06T03:58:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20d37ff700 1 -- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20cc06fb40 msgr2=0x7f20cc0c7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20d37ff700 1 --2- 172.21.15.97:0/4063550680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20cc06fb40 0x7f20cc0c7770 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f20c800c4d0 tx=0x7f20c800c890 comp rx=0 tx=0).stop 2023-07-06T03:58:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20d1d9c700 1 -- 172.21.15.97:0/4063550680 reap_dead start 2023-07-06T03:58:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20d37ff700 1 -- 172.21.15.97:0/4063550680 shutdown_connections 2023-07-06T03:58:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20d37ff700 1 -- 172.21.15.97:0/4063550680 >> 172.21.15.97:0/4063550680 conn(0x7f20cc06a010 msgr2=0x7f20cc0cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20d37ff700 1 -- 172.21.15.97:0/4063550680 shutdown_connections 2023-07-06T03:58:02.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.280+0000 7f20d37ff700 1 -- 172.21.15.97:0/4063550680 wait complete. 2023-07-06T03:58:02.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-07-06T03:58:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.492+0000 7ff928bff700 1 Processor -- start 2023-07-06T03:58:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.492+0000 7ff928bff700 1 -- start start 2023-07-06T03:58:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.493+0000 7ff928bff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff9240b8470 0x7ff9240cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.493+0000 7ff928bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9240cfe30 0x7ff9240d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.493+0000 7ff928bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9240d12f0 con 0x7ff9240b2190 2023-07-06T03:58:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.493+0000 7ff928bff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9240b8840 con 0x7ff9240b8470 2023-07-06T03:58:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.493+0000 7ff928bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9240cf390 con 0x7ff9240cfe30 2023-07-06T03:58:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.493+0000 7ff921d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff9240b2190 0x7ff9240cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51444/0 (socket says 172.21.15.97:51444) 2023-07-06T03:58:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.493+0000 7ff921d9b700 1 -- 172.21.15.97:0/422754235 learned_addr learned my addr 172.21.15.97:0/422754235 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.493+0000 7ff92259c700 1 --2- 172.21.15.97:0/422754235 >> v2:172.21.15.97:3300/0 conn(0x7ff9240b8470 0x7ff9240cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.494+0000 7ff922d9d700 1 --2- 172.21.15.97:0/422754235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9240cfe30 0x7ff9240d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.494+0000 7ff92259c700 1 -- 172.21.15.97:0/422754235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9240cfe30 msgr2=0x7ff9240d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.494+0000 7ff92259c700 1 --2- 172.21.15.97:0/422754235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9240cfe30 0x7ff9240d2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.494+0000 7ff92259c700 1 -- 172.21.15.97:0/422754235 >> v1:172.21.15.97:6789/0 conn(0x7ff9240b2190 legacy=0x7ff9240cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.494+0000 7ff92259c700 1 -- 172.21.15.97:0/422754235 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9240b8ca0 con 0x7ff9240b8470 2023-07-06T03:58:02.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.494+0000 7ff92159a700 1 -- 172.21.15.97:0/422754235 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 416527373 0 0) 0x7ff9240d12f0 con 0x7ff9240b2190 2023-07-06T03:58:02.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.494+0000 7ff922d9d700 1 --2- 172.21.15.97:0/422754235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9240cfe30 0x7ff9240d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:02.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.494+0000 7ff92259c700 1 --2- 172.21.15.97:0/422754235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9240b8470 0x7ff9240cee50 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7ff914009840 tx=0x7ff914009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caa8e7c70383aba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.494+0000 7ff92159a700 1 -- 172.21.15.97:0/422754235 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914011830 con 0x7ff9240b8470 2023-07-06T03:58:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.494+0000 7ff92159a700 1 -- 172.21.15.97:0/422754235 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff914011990 con 0x7ff9240b8470 2023-07-06T03:58:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.494+0000 7ff92159a700 1 -- 172.21.15.97:0/422754235 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914011d20 con 0x7ff9240b8470 2023-07-06T03:58:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.495+0000 7ff928bff700 1 -- 172.21.15.97:0/422754235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9240b8470 msgr2=0x7ff9240cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.495+0000 7ff928bff700 1 --2- 172.21.15.97:0/422754235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9240b8470 0x7ff9240cee50 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7ff914009840 tx=0x7ff914009b50 comp rx=0 tx=0).stop 2023-07-06T03:58:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.495+0000 7ff928bff700 1 -- 172.21.15.97:0/422754235 shutdown_connections 2023-07-06T03:58:02.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.495+0000 7ff928bff700 1 --2- 172.21.15.97:0/422754235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9240b8470 0x7ff9240cee50 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.495+0000 7ff928bff700 1 --2- 172.21.15.97:0/422754235 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9240cfe30 0x7ff9240d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.495+0000 7ff928bff700 1 -- 172.21.15.97:0/422754235 >> 172.21.15.97:0/422754235 conn(0x7ff924071740 msgr2=0x7ff924071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:02.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.495+0000 7ff928bff700 1 -- 172.21.15.97:0/422754235 shutdown_connections 2023-07-06T03:58:02.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.495+0000 7ff928bff700 1 -- 172.21.15.97:0/422754235 wait complete. 2023-07-06T03:58:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.497+0000 7ff928bff700 1 Processor -- start 2023-07-06T03:58:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.497+0000 7ff928bff700 1 -- start start 2023-07-06T03:58:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff928bff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9240b2190 0x7ff92415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff928bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9240b8470 0x7ff92415dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff928bff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9240cfe30 0x7ff9240cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff928bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92415f990 con 0x7ff9240b8470 2023-07-06T03:58:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff928bff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92415da80 con 0x7ff9240b2190 2023-07-06T03:58:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff928bff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92415f6b0 con 0x7ff9240cfe30 2023-07-06T03:58:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff921d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9240b8470 0x7ff92415dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff921d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9240b8470 0x7ff92415dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48306/0 (socket says 172.21.15.97:48306) 2023-07-06T03:58:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff921d9b700 1 -- 172.21.15.97:0/3034089156 learned_addr learned my addr 172.21.15.97:0/3034089156 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff92259c700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9240b2190 0x7ff92415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff922d9d700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9240cfe30 0x7ff9240cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff921d9b700 1 -- 172.21.15.97:0/3034089156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9240cfe30 msgr2=0x7ff9240cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff921d9b700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9240cfe30 0x7ff9240cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff921d9b700 1 -- 172.21.15.97:0/3034089156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9240b2190 msgr2=0x7ff92415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff921d9b700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9240b2190 0x7ff92415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.498+0000 7ff921d9b700 1 -- 172.21.15.97:0/3034089156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140094a0 con 0x7ff9240b8470 2023-07-06T03:58:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.499+0000 7ff921d9b700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9240b8470 0x7ff92415dea0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7ff9180011e0 tx=0x7ff91800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.499+0000 7ff912ffd700 1 -- 172.21.15.97:0/3034089156 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918004c80 con 0x7ff9240b8470 2023-07-06T03:58:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.499+0000 7ff912ffd700 1 -- 172.21.15.97:0/3034089156 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff918004e00 con 0x7ff9240b8470 2023-07-06T03:58:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.499+0000 7ff928bff700 1 -- 172.21.15.97:0/3034089156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92415d6a0 con 0x7ff9240b8470 2023-07-06T03:58:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.499+0000 7ff912ffd700 1 -- 172.21.15.97:0/3034089156 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918007310 con 0x7ff9240b8470 2023-07-06T03:58:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.499+0000 7ff928bff700 1 -- 172.21.15.97:0/3034089156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9240cbbf0 con 0x7ff9240b8470 2023-07-06T03:58:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.500+0000 7ff928bff700 1 -- 172.21.15.97:0/3034089156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92415dbc0 con 0x7ff9240b8470 2023-07-06T03:58:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.501+0000 7ff912ffd700 1 -- 172.21.15.97:0/3034089156 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff918007470 con 0x7ff9240b8470 2023-07-06T03:58:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.502+0000 7ff912ffd700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff90406ace0 0x7ff90406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.502+0000 7ff912ffd700 1 -- 172.21.15.97:0/3034089156 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff918088e20 con 0x7ff9240b8470 2023-07-06T03:58:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.502+0000 7ff92259c700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff90406ace0 0x7ff90406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.503+0000 7ff92259c700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff90406ace0 0x7ff90406d190 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff9140090b0 tx=0x7ff914008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.506+0000 7ff912ffd700 1 -- 172.21.15.97:0/3034089156 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff92415dbc0 con 0x7ff9240b8470 2023-07-06T03:58:02.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.701+0000 7ff928bff700 1 -- 172.21.15.97:0/3034089156 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9240cd7e0 con 0x7ff90406ace0 2023-07-06T03:58:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.701+0000 7ff912ffd700 1 -- 172.21.15.97:0/3034089156 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff914011cb0 con 0x7ff90406ace0 2023-07-06T03:58:02.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.703+0000 7ff928bff700 1 -- 172.21.15.97:0/3034089156 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff90406ace0 msgr2=0x7ff90406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.703+0000 7ff928bff700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff90406ace0 0x7ff90406d190 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff9140090b0 tx=0x7ff914008030 comp rx=0 tx=0).stop 2023-07-06T03:58:02.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.703+0000 7ff928bff700 1 -- 172.21.15.97:0/3034089156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9240b8470 msgr2=0x7ff92415dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.703+0000 7ff928bff700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9240b8470 0x7ff92415dea0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7ff9180011e0 tx=0x7ff91800ca30 comp rx=0 tx=0).stop 2023-07-06T03:58:02.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.703+0000 7ff928bff700 1 -- 172.21.15.97:0/3034089156 shutdown_connections 2023-07-06T03:58:02.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.703+0000 7ff928bff700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7ff90406ace0 0x7ff90406d190 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.703+0000 7ff928bff700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7ff9240cfe30 0x7ff9240cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.703+0000 7ff928bff700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff9240b2190 0x7ff92415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.703+0000 7ff928bff700 1 --2- 172.21.15.97:0/3034089156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9240b8470 0x7ff92415dea0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.703+0000 7ff928bff700 1 -- 172.21.15.97:0/3034089156 >> 172.21.15.97:0/3034089156 conn(0x7ff924071740 msgr2=0x7ff9240b6210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:02.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.704+0000 7ff928bff700 1 -- 172.21.15.97:0/3034089156 shutdown_connections 2023-07-06T03:58:02.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.704+0000 7ff928bff700 1 -- 172.21.15.97:0/3034089156 wait complete. 2023-07-06T03:58:02.711 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 2 to deep-scrub 2023-07-06T03:58:02.724 INFO:tasks.ceph:Scrubbing osd.3 2023-07-06T03:58:02.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-07-06T03:58:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.922+0000 7f483e7ff700 1 Processor -- start 2023-07-06T03:58:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.922+0000 7f483e7ff700 1 -- start start 2023-07-06T03:58:02.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.923+0000 7f483e7ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f483806b120 0x7f4838160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.923+0000 7f483e7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48380cdd50 0x7f4838167890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.923+0000 7f483e7ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48380cf780 con 0x7f483806b4f0 2023-07-06T03:58:02.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.923+0000 7f483e7ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48380b25a0 con 0x7f483806b120 2023-07-06T03:58:02.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.924+0000 7f483e7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4838161460 con 0x7f48380cdd50 2023-07-06T03:58:02.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.924+0000 7f4837fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f483806b120 0x7f4838160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.924+0000 7f48377fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f483806b4f0 0x7f48380cd640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51446/0 (socket says 172.21.15.97:51446) 2023-07-06T03:58:02.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.924+0000 7f48377fe700 1 -- 172.21.15.97:0/855671122 learned_addr learned my addr 172.21.15.97:0/855671122 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.924+0000 7f4836ffd700 1 -- 172.21.15.97:0/855671122 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78024314 0 0) 0x7f48380cf780 con 0x7f483806b4f0 2023-07-06T03:58:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.924+0000 7f4837fff700 1 -- 172.21.15.97:0/855671122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48380cdd50 msgr2=0x7f4838167890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.924+0000 7f4837fff700 1 --2- 172.21.15.97:0/855671122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48380cdd50 0x7f4838167890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.924+0000 7f4837fff700 1 -- 172.21.15.97:0/855671122 >> v1:172.21.15.97:6789/0 conn(0x7f483806b4f0 legacy=0x7f48380cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.924+0000 7f4837fff700 1 -- 172.21.15.97:0/855671122 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48380ceb70 con 0x7f483806b120 2023-07-06T03:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.924+0000 7f4837fff700 1 --2- 172.21.15.97:0/855671122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483806b120 0x7f4838160f20 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f4828009820 tx=0x7f4828009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bda7c28dab243ab server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.925+0000 7f4836ffd700 1 -- 172.21.15.97:0/855671122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4828005e30 con 0x7f483806b120 2023-07-06T03:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.925+0000 7f4836ffd700 1 -- 172.21.15.97:0/855671122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4828006030 con 0x7f483806b120 2023-07-06T03:58:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.925+0000 7f4836ffd700 1 -- 172.21.15.97:0/855671122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48280063e0 con 0x7f483806b120 2023-07-06T03:58:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.925+0000 7f483e7ff700 1 -- 172.21.15.97:0/855671122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483806b120 msgr2=0x7f4838160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.925+0000 7f483e7ff700 1 --2- 172.21.15.97:0/855671122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483806b120 0x7f4838160f20 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f4828009820 tx=0x7f4828009b30 comp rx=0 tx=0).stop 2023-07-06T03:58:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.925+0000 7f483e7ff700 1 -- 172.21.15.97:0/855671122 shutdown_connections 2023-07-06T03:58:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.925+0000 7f483e7ff700 1 --2- 172.21.15.97:0/855671122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f483806b120 0x7f4838160f20 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.925+0000 7f483e7ff700 1 --2- 172.21.15.97:0/855671122 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f48380cdd50 0x7f4838167890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.925+0000 7f483e7ff700 1 -- 172.21.15.97:0/855671122 >> 172.21.15.97:0/855671122 conn(0x7f4838099f40 msgr2=0x7f48380d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.925+0000 7f483e7ff700 1 -- 172.21.15.97:0/855671122 shutdown_connections 2023-07-06T03:58:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.925+0000 7f483e7ff700 1 -- 172.21.15.97:0/855671122 wait complete. 2023-07-06T03:58:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.927+0000 7f483e7ff700 1 Processor -- start 2023-07-06T03:58:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.927+0000 7f483e7ff700 1 -- start start 2023-07-06T03:58:02.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.928+0000 7f483e7ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f483806b120 0x7f48381f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.928+0000 7f483e7ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483806b4f0 0x7f483809d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.928+0000 7f483e7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48380cdd50 0x7f483809dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.928+0000 7f483e7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48380c7780 con 0x7f48380cdd50 2023-07-06T03:58:02.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.928+0000 7f483e7ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483809abe0 con 0x7f483806b120 2023-07-06T03:58:02.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.928+0000 7f483e7ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48381f25b0 con 0x7f483806b4f0 2023-07-06T03:58:02.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.928+0000 7f483cd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48380cdd50 0x7f483809dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.928+0000 7f483cd9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48380cdd50 0x7f483809dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48326/0 (socket says 172.21.15.97:48326) 2023-07-06T03:58:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.928+0000 7f483cd9c700 1 -- 172.21.15.97:0/2214802298 learned_addr learned my addr 172.21.15.97:0/2214802298 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f4837fff700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f483806b120 0x7f48381f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f483cd9c700 1 -- 172.21.15.97:0/2214802298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483806b4f0 msgr2=0x7f483809d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f483cd9c700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f483806b4f0 0x7f483809d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f483cd9c700 1 -- 172.21.15.97:0/2214802298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f483806b120 msgr2=0x7f48381f1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f483cd9c700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f483806b120 0x7f48381f1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f483cd9c700 1 -- 172.21.15.97:0/2214802298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4828009430 con 0x7f48380cdd50 2023-07-06T03:58:02.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f4837fff700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f483806b120 0x7f48381f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:02.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f483cd9c700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48380cdd50 0x7f483809dc50 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f483000b970 tx=0x7f483000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:02.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f4834ff9700 1 -- 172.21.15.97:0/2214802298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483000c9b0 con 0x7f48380cdd50 2023-07-06T03:58:02.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f4834ff9700 1 -- 172.21.15.97:0/2214802298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483000cb30 con 0x7f48380cdd50 2023-07-06T03:58:02.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f483e7ff700 1 -- 172.21.15.97:0/2214802298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48380c6ee0 con 0x7f48380cdd50 2023-07-06T03:58:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f4834ff9700 1 -- 172.21.15.97:0/2214802298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830014440 con 0x7f48380cdd50 2023-07-06T03:58:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.929+0000 7f483e7ff700 1 -- 172.21.15.97:0/2214802298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48380c74a0 con 0x7f48380cdd50 2023-07-06T03:58:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.930+0000 7f483e7ff700 1 -- 172.21.15.97:0/2214802298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48380cf130 con 0x7f48380cdd50 2023-07-06T03:58:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.931+0000 7f4834ff9700 1 -- 172.21.15.97:0/2214802298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f483001d3b0 con 0x7f48380cdd50 2023-07-06T03:58:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.932+0000 7f4834ff9700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f482406ace0 0x7f482406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.932+0000 7f4834ff9700 1 -- 172.21.15.97:0/2214802298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4830006e40 con 0x7f48380cdd50 2023-07-06T03:58:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.932+0000 7f4834ff9700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] conn(0x7f4824071150 0x7f4824073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:02.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.932+0000 7f4834ff9700 1 -- 172.21.15.97:0/2214802298 --> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4824073c30 con 0x7f4824071150 2023-07-06T03:58:02.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.932+0000 7f4834ff9700 1 -- 172.21.15.97:0/2214802298 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48300898b0 con 0x7f48380cdd50 2023-07-06T03:58:02.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.932+0000 7f4837fff700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f482406ace0 0x7f482406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.933+0000 7f48377fe700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] conn(0x7f4824071150 0x7f4824073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:02.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.933+0000 7f4837fff700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f482406ace0 0x7f482406d190 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4828000c00 tx=0x7f4828008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:02.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.933+0000 7f48377fe700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] conn(0x7f4824071150 0x7f4824073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:02.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.934+0000 7f4834ff9700 1 -- 172.21.15.97:0/2214802298 <== osd.3 v2:172.21.15.97:6809/1358667479 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4824073c30 con 0x7f4824071150 2023-07-06T03:58:02.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.949+0000 7f483e7ff700 1 -- 172.21.15.97:0/2214802298 --> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f48380cee50 con 0x7f4824071150 2023-07-06T03:58:02.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.949+0000 7f4834ff9700 1 -- 172.21.15.97:0/2214802298 <== osd.3 v2:172.21.15.97:6809/1358667479 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f482c001310 con 0x7f4824071150 2023-07-06T03:58:02.952 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:58:02.953 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T03:58:02.953 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:58:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.949+0000 7f483e7ff700 1 -- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] conn(0x7f4824071150 msgr2=0x7f4824073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.949+0000 7f483e7ff700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:6809/1358667479,v1:172.21.15.97:6811/1358667479] conn(0x7f4824071150 0x7f4824073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.949+0000 7f483e7ff700 1 -- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f482406ace0 msgr2=0x7f482406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.949+0000 7f483e7ff700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f482406ace0 0x7f482406d190 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4828000c00 tx=0x7f4828008070 comp rx=0 tx=0).stop 2023-07-06T03:58:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.949+0000 7f483e7ff700 1 -- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48380cdd50 msgr2=0x7f483809dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.949+0000 7f483e7ff700 1 --2- 172.21.15.97:0/2214802298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48380cdd50 0x7f483809dc50 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f483000b970 tx=0x7f483000bd30 comp rx=0 tx=0).stop 2023-07-06T03:58:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.950+0000 7f483cd9c700 1 -- 172.21.15.97:0/2214802298 reap_dead start 2023-07-06T03:58:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.950+0000 7f483e7ff700 1 -- 172.21.15.97:0/2214802298 shutdown_connections 2023-07-06T03:58:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.950+0000 7f483e7ff700 1 -- 172.21.15.97:0/2214802298 >> 172.21.15.97:0/2214802298 conn(0x7f4838099f40 msgr2=0x7f48380d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.950+0000 7f483e7ff700 1 -- 172.21.15.97:0/2214802298 shutdown_connections 2023-07-06T03:58:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:02.950+0000 7f483e7ff700 1 -- 172.21.15.97:0/2214802298 wait complete. 2023-07-06T03:58:02.969 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-07-06T03:58:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.162+0000 7f5d123ff700 1 Processor -- start 2023-07-06T03:58:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.162+0000 7f5d123ff700 1 -- start start 2023-07-06T03:58:03.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.163+0000 7f5d123ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5d0c0b8470 0x7f5d0c0cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.163+0000 7f5d123ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0c0d12f0 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.163+0000 7f5d123ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0c0cf390 con 0x7f5d0c0b2190 2023-07-06T03:58:03.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.163+0000 7f5d123ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c0b8840 con 0x7f5d0c0b8470 2023-07-06T03:58:03.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.164+0000 7f5d1099c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5d0c0cfe30 0x7f5d0c0d2750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51452/0 (socket says 172.21.15.97:51452) 2023-07-06T03:58:03.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.164+0000 7f5d1099c700 1 -- 172.21.15.97:0/1290456029 learned_addr learned my addr 172.21.15.97:0/1290456029 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:03.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.164+0000 7f5d0bfff700 1 --2- 172.21.15.97:0/1290456029 >> v2:172.21.15.146:3300/0 conn(0x7f5d0c0b8470 0x7f5d0c0cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.164+0000 7f5d0affd700 1 -- 172.21.15.97:0/1290456029 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159041570 0 0) 0x7f5d0c0d12f0 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.164+0000 7f5d0affd700 1 -- 172.21.15.97:0/1290456029 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cf40035b0 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.164+0000 7f5d0bfff700 1 -- 172.21.15.97:0/1290456029 >> v1:172.21.15.146:6789/0 conn(0x7f5d0c0b2190 legacy=0x7f5d0c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.164+0000 7f5d0bfff700 1 -- 172.21.15.97:0/1290456029 >> v1:172.21.15.97:6789/0 conn(0x7f5d0c0cfe30 legacy=0x7f5d0c0d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.164+0000 7f5d0bfff700 1 -- 172.21.15.97:0/1290456029 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c0b8ca0 con 0x7f5d0c0b8470 2023-07-06T03:58:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.165+0000 7f5d0bfff700 1 --2- 172.21.15.97:0/1290456029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c0b8470 0x7f5d0c0cee50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009840 tx=0x7f5cfc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a113435b6b0bbbb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.165+0000 7f5d0affd700 1 -- 172.21.15.97:0/1290456029 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc011830 con 0x7f5d0c0b8470 2023-07-06T03:58:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.165+0000 7f5d0affd700 1 -- 172.21.15.97:0/1290456029 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc011990 con 0x7f5d0c0b8470 2023-07-06T03:58:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.165+0000 7f5d0affd700 1 -- 172.21.15.97:0/1290456029 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc011d20 con 0x7f5d0c0b8470 2023-07-06T03:58:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.165+0000 7f5d123ff700 1 -- 172.21.15.97:0/1290456029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c0b8470 msgr2=0x7f5d0c0cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.165+0000 7f5d123ff700 1 --2- 172.21.15.97:0/1290456029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c0b8470 0x7f5d0c0cee50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009840 tx=0x7f5cfc009b50 comp rx=0 tx=0).stop 2023-07-06T03:58:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.165+0000 7f5d123ff700 1 -- 172.21.15.97:0/1290456029 shutdown_connections 2023-07-06T03:58:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.165+0000 7f5d123ff700 1 --2- 172.21.15.97:0/1290456029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c0b8470 0x7f5d0c0cee50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.165+0000 7f5d123ff700 1 -- 172.21.15.97:0/1290456029 >> 172.21.15.97:0/1290456029 conn(0x7f5d0c071740 msgr2=0x7f5d0c071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.166+0000 7f5d123ff700 1 -- 172.21.15.97:0/1290456029 shutdown_connections 2023-07-06T03:58:03.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.166+0000 7f5d123ff700 1 -- 172.21.15.97:0/1290456029 wait complete. 2023-07-06T03:58:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.167+0000 7f5d123ff700 1 Processor -- start 2023-07-06T03:58:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.167+0000 7f5d123ff700 1 -- start start 2023-07-06T03:58:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.168+0000 7f5d123ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c0b2190 0x7f5d0c0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.168+0000 7f5d123ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c0b8470 0x7f5d0c0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.168+0000 7f5d123ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c0cfe30 0x7f5d0c0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.168+0000 7f5d123ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0cb1a0 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.168+0000 7f5d123ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0c7450 con 0x7f5d0c0b2190 2023-07-06T03:58:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.168+0000 7f5d123ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0c7730 con 0x7f5d0c0b8470 2023-07-06T03:58:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d1099c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c0cfe30 0x7f5d0c0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d1099c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c0cfe30 0x7f5d0c0c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48332/0 (socket says 172.21.15.97:48332) 2023-07-06T03:58:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d1099c700 1 -- 172.21.15.97:0/3535260693 learned_addr learned my addr 172.21.15.97:0/3535260693 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d0bfff700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c0b2190 0x7f5d0c0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d0b7fe700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c0b8470 0x7f5d0c0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d1099c700 1 -- 172.21.15.97:0/3535260693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c0b8470 msgr2=0x7f5d0c0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d1099c700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c0b8470 0x7f5d0c0c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d1099c700 1 -- 172.21.15.97:0/3535260693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c0b2190 msgr2=0x7f5d0c0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d1099c700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c0b2190 0x7f5d0c0cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d1099c700 1 -- 172.21.15.97:0/3535260693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0094a0 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d1099c700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c0cfe30 0x7f5d0c0c6d80 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400caa0 tx=0x7f5d0400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:03.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.169+0000 7f5d08ff9700 1 -- 172.21.15.97:0/3535260693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400a9e0 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.170+0000 7f5d08ff9700 1 -- 172.21.15.97:0/3535260693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0400ab40 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.170+0000 7f5d123ff700 1 -- 172.21.15.97:0/3535260693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c0c7a10 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.170+0000 7f5d08ff9700 1 -- 172.21.15.97:0/3535260693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04012400 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.170+0000 7f5d123ff700 1 -- 172.21.15.97:0/3535260693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c0c7da0 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.171+0000 7f5d123ff700 1 -- 172.21.15.97:0/3535260693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c04e340 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.171+0000 7f5d08ff9700 1 -- 172.21.15.97:0/3535260693 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d0401b3b0 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.172+0000 7f5d08ff9700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 0x7f5cf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.172+0000 7f5d08ff9700 1 -- 172.21.15.97:0/3535260693 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5d04088e20 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.173+0000 7f5d0bfff700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 0x7f5cf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.173+0000 7f5d0bfff700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 0x7f5cf806d190 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0090b0 tx=0x7f5cfc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:03.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.176+0000 7f5d08ff9700 1 -- 172.21.15.97:0/3535260693 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d04054210 con 0x7f5d0c0cfe30 2023-07-06T03:58:03.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.370+0000 7f5d123ff700 1 -- 172.21.15.97:0/3535260693 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d0c0cd7e0 con 0x7f5cf806ace0 2023-07-06T03:58:03.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.371+0000 7f5d08ff9700 1 -- 172.21.15.97:0/3535260693 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5cfc011c00 con 0x7f5cf806ace0 2023-07-06T03:58:03.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 -- 172.21.15.97:0/3535260693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 msgr2=0x7f5cf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 0x7f5cf806d190 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0090b0 tx=0x7f5cfc008030 comp rx=0 tx=0).stop 2023-07-06T03:58:03.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 -- 172.21.15.97:0/3535260693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c0cfe30 msgr2=0x7f5d0c0c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c0cfe30 0x7f5d0c0c6d80 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400caa0 tx=0x7f5d0400cdb0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 -- 172.21.15.97:0/3535260693 shutdown_connections 2023-07-06T03:58:03.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f5cf806ace0 0x7f5cf806d190 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d0c0cfe30 0x7f5d0c0c6d80 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d0c0b2190 0x7f5d0c0cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 --2- 172.21.15.97:0/3535260693 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f5d0c0b8470 0x7f5d0c0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 -- 172.21.15.97:0/3535260693 >> 172.21.15.97:0/3535260693 conn(0x7f5d0c071740 msgr2=0x7f5d0c0b72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:03.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 -- 172.21.15.97:0/3535260693 shutdown_connections 2023-07-06T03:58:03.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.373+0000 7f5d123ff700 1 -- 172.21.15.97:0/3535260693 wait complete. 2023-07-06T03:58:03.379 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 3 to deep-scrub 2023-07-06T03:58:03.394 INFO:tasks.ceph:Scrubbing osd.4 2023-07-06T03:58:03.394 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-07-06T03:58:03.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.589+0000 7f47f8fff700 1 Processor -- start 2023-07-06T03:58:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.590+0000 7f47f8fff700 1 -- start start 2023-07-06T03:58:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.591+0000 7f47f8fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47f415c840 0x7f47f40b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.591+0000 7f47f8fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f40d1ed0 0x7f47f40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.591+0000 7f47f8fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f40d02b0 con 0x7f47f40b47b0 2023-07-06T03:58:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.591+0000 7f47f8fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f40cd420 con 0x7f47f415c840 2023-07-06T03:58:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.591+0000 7f47f8fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40d2420 con 0x7f47f40d1ed0 2023-07-06T03:58:03.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.591+0000 7f47ebfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47f415c840 0x7f47f40b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.591+0000 7f47ebfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47f415c840 0x7f47f40b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48334/0 (socket says 172.21.15.97:48334) 2023-07-06T03:58:03.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.591+0000 7f47ebfff700 1 -- 172.21.15.97:0/701742712 learned_addr learned my addr 172.21.15.97:0/701742712 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:03.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.591+0000 7f47f2d9d700 1 --2- 172.21.15.97:0/701742712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f40d1ed0 0x7f47f40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.592+0000 7f47ebfff700 1 -- 172.21.15.97:0/701742712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f40d1ed0 msgr2=0x7f47f40cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:03.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.592+0000 7f47ebfff700 1 --2- 172.21.15.97:0/701742712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f40d1ed0 0x7f47f40cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.592+0000 7f47ebfff700 1 -- 172.21.15.97:0/701742712 >> v1:172.21.15.146:6789/0 conn(0x7f47f40b47b0 legacy=0x7f47f40b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.592+0000 7f47ebfff700 1 -- 172.21.15.97:0/701742712 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40d2700 con 0x7f47f415c840 2023-07-06T03:58:03.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.592+0000 7f47ebfff700 1 --2- 172.21.15.97:0/701742712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f415c840 0x7f47f40b5290 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0098f0 tx=0x7f47dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bca5eaa76f2f7b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:03.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.592+0000 7f47f1d9b700 1 -- 172.21.15.97:0/701742712 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc011830 con 0x7f47f415c840 2023-07-06T03:58:03.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.592+0000 7f47f1d9b700 1 -- 172.21.15.97:0/701742712 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc011990 con 0x7f47f415c840 2023-07-06T03:58:03.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.592+0000 7f47f1d9b700 1 -- 172.21.15.97:0/701742712 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc011d20 con 0x7f47f415c840 2023-07-06T03:58:03.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.593+0000 7f47f8fff700 1 -- 172.21.15.97:0/701742712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f415c840 msgr2=0x7f47f40b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.593+0000 7f47f8fff700 1 --2- 172.21.15.97:0/701742712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f415c840 0x7f47f40b5290 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0098f0 tx=0x7f47dc009c00 comp rx=0 tx=0).stop 2023-07-06T03:58:03.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.593+0000 7f47f8fff700 1 -- 172.21.15.97:0/701742712 shutdown_connections 2023-07-06T03:58:03.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.593+0000 7f47f8fff700 1 --2- 172.21.15.97:0/701742712 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f40d1ed0 0x7f47f40cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.593+0000 7f47f8fff700 1 --2- 172.21.15.97:0/701742712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f415c840 0x7f47f40b5290 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.593+0000 7f47f8fff700 1 -- 172.21.15.97:0/701742712 >> 172.21.15.97:0/701742712 conn(0x7f47f406a670 msgr2=0x7f47f406aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:03.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.593+0000 7f47f8fff700 1 -- 172.21.15.97:0/701742712 shutdown_connections 2023-07-06T03:58:03.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.593+0000 7f47f8fff700 1 -- 172.21.15.97:0/701742712 wait complete. 2023-07-06T03:58:03.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.595+0000 7f47f8fff700 1 Processor -- start 2023-07-06T03:58:03.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.595+0000 7f47f8fff700 1 -- start start 2023-07-06T03:58:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47f8fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47f40b47b0 0x7f47f41440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47f8fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40d1ed0 0x7f47f40b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47f8fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f415c840 0x7f47f40b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47f8fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41f22d0 con 0x7f47f40d1ed0 2023-07-06T03:58:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47f8fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40c6a20 con 0x7f47f40b47b0 2023-07-06T03:58:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47f8fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40c6d00 con 0x7f47f415c840 2023-07-06T03:58:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47ebfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40d1ed0 0x7f47f40b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47ebfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40d1ed0 0x7f47f40b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48350/0 (socket says 172.21.15.97:48350) 2023-07-06T03:58:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47ebfff700 1 -- 172.21.15.97:0/2968954779 learned_addr learned my addr 172.21.15.97:0/2968954779 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47f2d9d700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f415c840 0x7f47f40b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47f259c700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47f40b47b0 0x7f47f41440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47ebfff700 1 -- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f415c840 msgr2=0x7f47f40b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47ebfff700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f415c840 0x7f47f40b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47ebfff700 1 -- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47f40b47b0 msgr2=0x7f47f41440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47ebfff700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47f40b47b0 0x7f47f41440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47ebfff700 1 -- 172.21.15.97:0/2968954779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc009550 con 0x7f47f40d1ed0 2023-07-06T03:58:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47f2d9d700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f47f415c840 0x7f47f40b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.596+0000 7f47f259c700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47f40b47b0 0x7f47f41440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.597+0000 7f47ebfff700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40d1ed0 0x7f47f40b37e0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f47dc000c00 tx=0x7f47dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:03.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.597+0000 7f47eaffd700 1 -- 172.21.15.97:0/2968954779 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc004570 con 0x7f47f40d1ed0 2023-07-06T03:58:03.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.597+0000 7f47eaffd700 1 -- 172.21.15.97:0/2968954779 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc01f070 con 0x7f47f40d1ed0 2023-07-06T03:58:03.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.597+0000 7f47eaffd700 1 -- 172.21.15.97:0/2968954779 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0038c0 con 0x7f47f40d1ed0 2023-07-06T03:58:03.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.597+0000 7f47f8fff700 1 -- 172.21.15.97:0/2968954779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f40c6fe0 con 0x7f47f40d1ed0 2023-07-06T03:58:03.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.597+0000 7f47f8fff700 1 -- 172.21.15.97:0/2968954779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f40c75a0 con 0x7f47f40d1ed0 2023-07-06T03:58:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.598+0000 7f47f8fff700 1 -- 172.21.15.97:0/2968954779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47f40d29e0 con 0x7f47f40d1ed0 2023-07-06T03:58:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.599+0000 7f47eaffd700 1 -- 172.21.15.97:0/2968954779 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47dc021030 con 0x7f47f40d1ed0 2023-07-06T03:58:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.600+0000 7f47eaffd700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f47d406ace0 0x7f47d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.600+0000 7f47eaffd700 1 -- 172.21.15.97:0/2968954779 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f47dc08a700 con 0x7f47f40d1ed0 2023-07-06T03:58:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.600+0000 7f47eaffd700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] conn(0x7f47d4071150 0x7f47d4073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.600+0000 7f47f259c700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f47d406ace0 0x7f47d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.600+0000 7f47eaffd700 1 -- 172.21.15.97:0/2968954779 --> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47d4073c30 con 0x7f47d4071150 2023-07-06T03:58:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.600+0000 7f47eaffd700 1 -- 172.21.15.97:0/2968954779 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47dc08cce0 con 0x7f47f40d1ed0 2023-07-06T03:58:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.600+0000 7f47f2d9d700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] conn(0x7f47d4071150 0x7f47d4073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.601+0000 7f47f259c700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f47d406ace0 0x7f47d406d190 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f47e4000f60 tx=0x7f47e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.601+0000 7f47f2d9d700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] conn(0x7f47d4071150 0x7f47d4073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.602+0000 7f47eaffd700 1 -- 172.21.15.97:0/2968954779 <== osd.4 v2:172.21.15.146:6800/663749144 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f47d4073c30 con 0x7f47d4071150 2023-07-06T03:58:03.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.617+0000 7f47f8fff700 1 -- 172.21.15.97:0/2968954779 --> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f47f40cd6c0 con 0x7f47d4071150 2023-07-06T03:58:03.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47eaffd700 1 -- 172.21.15.97:0/2968954779 <== osd.4 v2:172.21.15.146:6800/663749144 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f47ec007480 con 0x7f47d4071150 2023-07-06T03:58:03.621 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:58:03.621 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T03:58:03.621 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:58:03.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47f8fff700 1 -- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] conn(0x7f47d4071150 msgr2=0x7f47d4073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47f8fff700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.146:6800/663749144,v1:172.21.15.146:6801/663749144] conn(0x7f47d4071150 0x7f47d4073560 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47f8fff700 1 -- 172.21.15.97:0/2968954779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f47d406ace0 msgr2=0x7f47d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47f8fff700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f47d406ace0 0x7f47d406d190 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f47e4000f60 tx=0x7f47e4009040 comp rx=0 tx=0).stop 2023-07-06T03:58:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47f8fff700 1 -- 172.21.15.97:0/2968954779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40d1ed0 msgr2=0x7f47f40b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47f8fff700 1 --2- 172.21.15.97:0/2968954779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f40d1ed0 0x7f47f40b37e0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f47dc000c00 tx=0x7f47dc004750 comp rx=0 tx=0).stop 2023-07-06T03:58:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47f2d9d700 1 -- 172.21.15.97:0/2968954779 reap_dead start 2023-07-06T03:58:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47f8fff700 1 -- 172.21.15.97:0/2968954779 shutdown_connections 2023-07-06T03:58:03.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47f8fff700 1 -- 172.21.15.97:0/2968954779 >> 172.21.15.97:0/2968954779 conn(0x7f47f406a670 msgr2=0x7f47f40cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:03.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47f8fff700 1 -- 172.21.15.97:0/2968954779 shutdown_connections 2023-07-06T03:58:03.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.618+0000 7f47f8fff700 1 -- 172.21.15.97:0/2968954779 wait complete. 2023-07-06T03:58:03.638 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-07-06T03:58:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.831+0000 7fab829ff700 1 Processor -- start 2023-07-06T03:58:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.831+0000 7fab829ff700 1 -- start start 2023-07-06T03:58:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.832+0000 7fab829ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fab7c0cfe30 0x7fab7c0cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.832+0000 7fab829ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7c0d12f0 con 0x7fab7c0b2190 2023-07-06T03:58:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.832+0000 7fab829ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7c0cf390 con 0x7fab7c0b8470 2023-07-06T03:58:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.832+0000 7fab829ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7c0b8840 con 0x7fab7c0cfe30 2023-07-06T03:58:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab80f9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fab7c0b2190 0x7fab7c0d2750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51466/0 (socket says 172.21.15.97:51466) 2023-07-06T03:58:03.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab80f9c700 1 -- 172.21.15.97:0/2539714428 learned_addr learned my addr 172.21.15.97:0/2539714428 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307401531 0 0) 0x7fab7c0d12f0 con 0x7fab7c0b2190 2023-07-06T03:58:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7b7fe700 1 --2- 172.21.15.97:0/2539714428 >> v2:172.21.15.146:3300/0 conn(0x7fab7c0cfe30 0x7fab7c0cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab640035b0 con 0x7fab7c0b2190 2023-07-06T03:58:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2468006299 0 0) 0x7fab640035b0 con 0x7fab7c0b2190 2023-07-06T03:58:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 >> v2:172.21.15.146:3300/0 conn(0x7fab7c0cfe30 msgr2=0x7fab7c0cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:03.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 --2- 172.21.15.97:0/2539714428 >> v2:172.21.15.146:3300/0 conn(0x7fab7c0cfe30 0x7fab7c0cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 >> v1:172.21.15.146:6789/0 conn(0x7fab7c0b8470 legacy=0x7fab7c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c0b8ca0 con 0x7fab7c0b2190 2023-07-06T03:58:03.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7fab740019f0 con 0x7fab7c0b2190 2023-07-06T03:58:03.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 >> v1:172.21.15.97:6789/0 conn(0x7fab7c0b2190 legacy=0x7fab7c0d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 --2- 172.21.15.97:0/2539714428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab64003a50 0x7fab64003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 --2- 172.21.15.97:0/2539714428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 0x7fab64004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 --2- 172.21.15.97:0/2539714428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab640052d0 0x7fab6400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c0b8840 con 0x7fab64004850 2023-07-06T03:58:03.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6400c200 con 0x7fab640052d0 2023-07-06T03:58:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.833+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6400c3c0 con 0x7fab64003a50 2023-07-06T03:58:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab7b7fe700 1 --2- 172.21.15.97:0/2539714428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 0x7fab64004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab7bfff700 1 --2- 172.21.15.97:0/2539714428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab64003a50 0x7fab64003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab7b7fe700 1 -- 172.21.15.97:0/2539714428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab64003a50 msgr2=0x7fab64003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab7b7fe700 1 --2- 172.21.15.97:0/2539714428 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab64003a50 0x7fab64003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab7b7fe700 1 -- 172.21.15.97:0/2539714428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab640052d0 msgr2=0x7fab6400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:03.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab7b7fe700 1 --2- 172.21.15.97:0/2539714428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab640052d0 0x7fab6400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab7b7fe700 1 -- 172.21.15.97:0/2539714428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab640035b0 con 0x7fab64004850 2023-07-06T03:58:03.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab80f9c700 1 -- 172.21.15.97:0/2539714428 reap_dead start 2023-07-06T03:58:03.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab7b7fe700 1 --2- 172.21.15.97:0/2539714428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 0x7fab64004cc0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0b2740 tx=0x7fab64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b07b3e2214acb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:03.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7000e6c0 con 0x7fab64004850 2023-07-06T03:58:03.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7000e820 con 0x7fab64004850 2023-07-06T03:58:03.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.834+0000 7fab7affd700 1 -- 172.21.15.97:0/2539714428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7000eb90 con 0x7fab64004850 2023-07-06T03:58:03.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.835+0000 7fab829ff700 1 -- 172.21.15.97:0/2539714428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 msgr2=0x7fab64004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.835+0000 7fab829ff700 1 --2- 172.21.15.97:0/2539714428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 0x7fab64004cc0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0b2740 tx=0x7fab64005880 comp rx=0 tx=0).stop 2023-07-06T03:58:03.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.835+0000 7fab829ff700 1 -- 172.21.15.97:0/2539714428 shutdown_connections 2023-07-06T03:58:03.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.835+0000 7fab829ff700 1 --2- 172.21.15.97:0/2539714428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 0x7fab64004cc0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.835+0000 7fab829ff700 1 -- 172.21.15.97:0/2539714428 >> 172.21.15.97:0/2539714428 conn(0x7fab7c071740 msgr2=0x7fab7c071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:03.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.835+0000 7fab829ff700 1 -- 172.21.15.97:0/2539714428 shutdown_connections 2023-07-06T03:58:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.835+0000 7fab829ff700 1 -- 172.21.15.97:0/2539714428 wait complete. 2023-07-06T03:58:03.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.837+0000 7fab829ff700 1 Processor -- start 2023-07-06T03:58:03.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.837+0000 7fab829ff700 1 -- start start 2023-07-06T03:58:03.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.838+0000 7fab829ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 0x7fab7c15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.838+0000 7fab829ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c0ccb20 0x7fab7c15b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.838+0000 7fab829ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab7c0cb250 0x7fab7c15e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.838+0000 7fab829ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c15bb00 con 0x7fab64004850 2023-07-06T03:58:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.838+0000 7fab829ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c0cba80 con 0x7fab7c0ccb20 2023-07-06T03:58:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.838+0000 7fab829ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c0cbd60 con 0x7fab7c0cb250 2023-07-06T03:58:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.838+0000 7fab7bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 0x7fab7c15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.838+0000 7fab7bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 0x7fab7c15bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48378/0 (socket says 172.21.15.97:48378) 2023-07-06T03:58:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.838+0000 7fab7bfff700 1 -- 172.21.15.97:0/2402352558 learned_addr learned my addr 172.21.15.97:0/2402352558 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab80f9c700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab7c0cb250 0x7fab7c15e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab7b7fe700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c0ccb20 0x7fab7c15b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab7bfff700 1 -- 172.21.15.97:0/2402352558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab7c0cb250 msgr2=0x7fab7c15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab7bfff700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab7c0cb250 0x7fab7c15e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab7bfff700 1 -- 172.21.15.97:0/2402352558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c0ccb20 msgr2=0x7fab7c15b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:03.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab7bfff700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c0ccb20 0x7fab7c15b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:03.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab7bfff700 1 -- 172.21.15.97:0/2402352558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c0cc040 con 0x7fab64004850 2023-07-06T03:58:03.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab80f9c700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab7c0cb250 0x7fab7c15e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab7b7fe700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c0ccb20 0x7fab7c15b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab7bfff700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 0x7fab7c15bce0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00e550 tx=0x7fab6c00e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab78ff9700 1 -- 172.21.15.97:0/2402352558 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0052e0 con 0x7fab64004850 2023-07-06T03:58:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab78ff9700 1 -- 172.21.15.97:0/2402352558 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c005440 con 0x7fab64004850 2023-07-06T03:58:03.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab829ff700 1 -- 172.21.15.97:0/2402352558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7c0cc320 con 0x7fab64004850 2023-07-06T03:58:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab78ff9700 1 -- 172.21.15.97:0/2402352558 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0038c0 con 0x7fab64004850 2023-07-06T03:58:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.839+0000 7fab829ff700 1 -- 172.21.15.97:0/2402352558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7c0cb6c0 con 0x7fab64004850 2023-07-06T03:58:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.840+0000 7fab829ff700 1 -- 172.21.15.97:0/2402352558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7c0cc970 con 0x7fab64004850 2023-07-06T03:58:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.841+0000 7fab78ff9700 1 -- 172.21.15.97:0/2402352558 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fab6c003a20 con 0x7fab64004850 2023-07-06T03:58:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.842+0000 7fab78ff9700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab6806ace0 0x7fab6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:03.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.842+0000 7fab78ff9700 1 -- 172.21.15.97:0/2402352558 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fab6c088860 con 0x7fab64004850 2023-07-06T03:58:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.842+0000 7fab7b7fe700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab6806ace0 0x7fab6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.843+0000 7fab7b7fe700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab6806ace0 0x7fab6806d190 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0cb1e0 tx=0x7fab70008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:03.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:03.847+0000 7fab78ff9700 1 -- 172.21.15.97:0/2402352558 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fab6c053d50 con 0x7fab64004850 2023-07-06T03:58:04.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.041+0000 7fab829ff700 1 -- 172.21.15.97:0/2402352558 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fab7c0cd7e0 con 0x7fab6806ace0 2023-07-06T03:58:04.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.042+0000 7fab78ff9700 1 -- 172.21.15.97:0/2402352558 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fab7000eb50 con 0x7fab6806ace0 2023-07-06T03:58:04.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 -- 172.21.15.97:0/2402352558 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab6806ace0 msgr2=0x7fab6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab6806ace0 0x7fab6806d190 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0cb1e0 tx=0x7fab70008030 comp rx=0 tx=0).stop 2023-07-06T03:58:04.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 -- 172.21.15.97:0/2402352558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 msgr2=0x7fab7c15bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 0x7fab7c15bce0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00e550 tx=0x7fab6c00e910 comp rx=0 tx=0).stop 2023-07-06T03:58:04.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 -- 172.21.15.97:0/2402352558 shutdown_connections 2023-07-06T03:58:04.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fab6806ace0 0x7fab6806d190 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fab7c0cb250 0x7fab7c15e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab64004850 0x7fab7c15bce0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 --2- 172.21.15.97:0/2402352558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab7c0ccb20 0x7fab7c15b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 -- 172.21.15.97:0/2402352558 >> 172.21.15.97:0/2402352558 conn(0x7fab7c071740 msgr2=0x7fab7c0c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:04.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 -- 172.21.15.97:0/2402352558 shutdown_connections 2023-07-06T03:58:04.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.044+0000 7fab829ff700 1 -- 172.21.15.97:0/2402352558 wait complete. 2023-07-06T03:58:04.049 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 4 to deep-scrub 2023-07-06T03:58:04.065 INFO:tasks.ceph:Scrubbing osd.5 2023-07-06T03:58:04.065 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-07-06T03:58:04.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.261+0000 7fbfebbff700 1 Processor -- start 2023-07-06T03:58:04.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.261+0000 7fbfebbff700 1 -- start start 2023-07-06T03:58:04.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.262+0000 7fbfebbff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbfe40cc0b0 0x7fbfe40cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.262+0000 7fbfebbff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfe406fb40 0x7fbfe40cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.262+0000 7fbfebbff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe4159760 con 0x7fbfe40c8fc0 2023-07-06T03:58:04.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.262+0000 7fbfebbff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe40c9c40 con 0x7fbfe406fb40 2023-07-06T03:58:04.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.262+0000 7fbfebbff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe40cc920 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.262+0000 7fbfea19c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfe406fb40 0x7fbfe40cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.263+0000 7fbfea19c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfe406fb40 0x7fbfe40cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48388/0 (socket says 172.21.15.97:48388) 2023-07-06T03:58:04.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.263+0000 7fbfea19c700 1 -- 172.21.15.97:0/3664540284 learned_addr learned my addr 172.21.15.97:0/3664540284 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:04.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.263+0000 7fbfe919a700 1 --2- 172.21.15.97:0/3664540284 >> v2:172.21.15.146:3300/0 conn(0x7fbfe40cc0b0 0x7fbfe40cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.263+0000 7fbfea19c700 1 -- 172.21.15.97:0/3664540284 >> v2:172.21.15.146:3300/0 conn(0x7fbfe40cc0b0 msgr2=0x7fbfe40cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:04.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.263+0000 7fbfea19c700 1 --2- 172.21.15.97:0/3664540284 >> v2:172.21.15.146:3300/0 conn(0x7fbfe40cc0b0 0x7fbfe40cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.263+0000 7fbfea19c700 1 -- 172.21.15.97:0/3664540284 >> v1:172.21.15.146:6789/0 conn(0x7fbfe40c8fc0 legacy=0x7fbfe40c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.263+0000 7fbfea19c700 1 -- 172.21.15.97:0/3664540284 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe40ccc00 con 0x7fbfe406fb40 2023-07-06T03:58:04.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.263+0000 7fbfea19c700 1 --2- 172.21.15.97:0/3664540284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe406fb40 0x7fbfe40cd9c0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40b4af0 tx=0x7fbfdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b42ee4621495eaf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:04.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.263+0000 7fbfe8999700 1 -- 172.21.15.97:0/3664540284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00e070 con 0x7fbfe406fb40 2023-07-06T03:58:04.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.263+0000 7fbfe8999700 1 -- 172.21.15.97:0/3664540284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc00b910 con 0x7fbfe406fb40 2023-07-06T03:58:04.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.263+0000 7fbfe8999700 1 -- 172.21.15.97:0/3664540284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00bcc0 con 0x7fbfe406fb40 2023-07-06T03:58:04.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.264+0000 7fbfebbff700 1 -- 172.21.15.97:0/3664540284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe406fb40 msgr2=0x7fbfe40cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.264+0000 7fbfebbff700 1 --2- 172.21.15.97:0/3664540284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe406fb40 0x7fbfe40cd9c0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40b4af0 tx=0x7fbfdc00b330 comp rx=0 tx=0).stop 2023-07-06T03:58:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.264+0000 7fbfebbff700 1 -- 172.21.15.97:0/3664540284 shutdown_connections 2023-07-06T03:58:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.264+0000 7fbfebbff700 1 --2- 172.21.15.97:0/3664540284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe406fb40 0x7fbfe40cd9c0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.264+0000 7fbfebbff700 1 --2- 172.21.15.97:0/3664540284 >> v2:172.21.15.146:3300/0 conn(0x7fbfe40cc0b0 0x7fbfe40cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.264+0000 7fbfebbff700 1 -- 172.21.15.97:0/3664540284 >> 172.21.15.97:0/3664540284 conn(0x7fbfe406a010 msgr2=0x7fbfe406a410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.264+0000 7fbfebbff700 1 -- 172.21.15.97:0/3664540284 shutdown_connections 2023-07-06T03:58:04.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.264+0000 7fbfebbff700 1 -- 172.21.15.97:0/3664540284 wait complete. 2023-07-06T03:58:04.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.266+0000 7fbfebbff700 1 Processor -- start 2023-07-06T03:58:04.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.266+0000 7fbfebbff700 1 -- start start 2023-07-06T03:58:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.267+0000 7fbfebbff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfe406fb40 0x7fbfe40c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.267+0000 7fbfebbff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfe40c8fc0 0x7fbfe4158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.267+0000 7fbfebbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe40cc0b0 0x7fbfe415a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.267+0000 7fbfebbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40c89a0 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.267+0000 7fbfebbff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40b0300 con 0x7fbfe406fb40 2023-07-06T03:58:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.267+0000 7fbfebbff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40afa50 con 0x7fbfe40c8fc0 2023-07-06T03:58:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.267+0000 7fbfea19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe40cc0b0 0x7fbfe415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.267+0000 7fbfea19c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe40cc0b0 0x7fbfe415a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48394/0 (socket says 172.21.15.97:48394) 2023-07-06T03:58:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.267+0000 7fbfea19c700 1 -- 172.21.15.97:0/1638348260 learned_addr learned my addr 172.21.15.97:0/1638348260 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.267+0000 7fbfe999b700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfe406fb40 0x7fbfe40c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.267+0000 7fbfe919a700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfe40c8fc0 0x7fbfe4158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.268+0000 7fbfea19c700 1 -- 172.21.15.97:0/1638348260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfe40c8fc0 msgr2=0x7fbfe4158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.268+0000 7fbfea19c700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fbfe40c8fc0 0x7fbfe4158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.268+0000 7fbfea19c700 1 -- 172.21.15.97:0/1638348260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfe406fb40 msgr2=0x7fbfe40c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.268+0000 7fbfea19c700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbfe406fb40 0x7fbfe40c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.268+0000 7fbfea19c700 1 -- 172.21.15.97:0/1638348260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc009b30 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.268+0000 7fbfea19c700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe40cc0b0 0x7fbfe415a570 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40b4b20 tx=0x7fbfdc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.268+0000 7fbfda7fc700 1 -- 172.21.15.97:0/1638348260 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc0099b0 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.268+0000 7fbfda7fc700 1 -- 172.21.15.97:0/1638348260 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc0092e0 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.268+0000 7fbfebbff700 1 -- 172.21.15.97:0/1638348260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe40c86c0 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.268+0000 7fbfda7fc700 1 -- 172.21.15.97:0/1638348260 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc01c640 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.268+0000 7fbfebbff700 1 -- 172.21.15.97:0/1638348260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe40aff20 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.269+0000 7fbfebbff700 1 -- 172.21.15.97:0/1638348260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbfe40cd1c0 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.270+0000 7fbfda7fc700 1 -- 172.21.15.97:0/1638348260 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbfdc01c7a0 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.271+0000 7fbfda7fc700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfc806ace0 0x7fbfc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.271+0000 7fbfda7fc700 1 -- 172.21.15.97:0/1638348260 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbfdc08aba0 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.271+0000 7fbfe999b700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfc806ace0 0x7fbfc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.271+0000 7fbfda7fc700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] conn(0x7fbfc8071150 0x7fbfc8073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.271+0000 7fbfda7fc700 1 -- 172.21.15.97:0/1638348260 --> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfc8073c30 con 0x7fbfc8071150 2023-07-06T03:58:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.271+0000 7fbfda7fc700 1 -- 172.21.15.97:0/1638348260 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfdc08f2a0 con 0x7fbfe40cc0b0 2023-07-06T03:58:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.272+0000 7fbfe919a700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] conn(0x7fbfc8071150 0x7fbfc8073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.272+0000 7fbfe999b700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfc806ace0 0x7fbfc806d190 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40b1120 tx=0x7fbfe0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:04.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.273+0000 7fbfe919a700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] conn(0x7fbfc8071150 0x7fbfc8073560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:04.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.274+0000 7fbfda7fc700 1 -- 172.21.15.97:0/1638348260 <== osd.5 v2:172.21.15.146:6809/3757512297 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbfc8073c30 con 0x7fbfc8071150 2023-07-06T03:58:04.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.288+0000 7fbfebbff700 1 -- 172.21.15.97:0/1638348260 --> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbfe40ccee0 con 0x7fbfc8071150 2023-07-06T03:58:04.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.289+0000 7fbfda7fc700 1 -- 172.21.15.97:0/1638348260 <== osd.5 v2:172.21.15.146:6809/3757512297 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbfd4009b60 con 0x7fbfc8071150 2023-07-06T03:58:04.292 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:58:04.292 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T03:58:04.292 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:58:04.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.289+0000 7fbfebbff700 1 -- 172.21.15.97:0/1638348260 >> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] conn(0x7fbfc8071150 msgr2=0x7fbfc8073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.289+0000 7fbfebbff700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.146:6809/3757512297,v1:172.21.15.146:6812/3757512297] conn(0x7fbfc8071150 0x7fbfc8073560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.289+0000 7fbfebbff700 1 -- 172.21.15.97:0/1638348260 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfc806ace0 msgr2=0x7fbfc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.289+0000 7fbfebbff700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fbfc806ace0 0x7fbfc806d190 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40b1120 tx=0x7fbfe0009040 comp rx=0 tx=0).stop 2023-07-06T03:58:04.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.289+0000 7fbfebbff700 1 -- 172.21.15.97:0/1638348260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe40cc0b0 msgr2=0x7fbfe415a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.289+0000 7fbfebbff700 1 --2- 172.21.15.97:0/1638348260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe40cc0b0 0x7fbfe415a570 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40b4b20 tx=0x7fbfdc004ad0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.289+0000 7fbfea19c700 1 -- 172.21.15.97:0/1638348260 reap_dead start 2023-07-06T03:58:04.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.289+0000 7fbfebbff700 1 -- 172.21.15.97:0/1638348260 shutdown_connections 2023-07-06T03:58:04.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.289+0000 7fbfebbff700 1 -- 172.21.15.97:0/1638348260 >> 172.21.15.97:0/1638348260 conn(0x7fbfe406a010 msgr2=0x7fbfe40cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:04.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.289+0000 7fbfebbff700 1 -- 172.21.15.97:0/1638348260 shutdown_connections 2023-07-06T03:58:04.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.290+0000 7fbfebbff700 1 -- 172.21.15.97:0/1638348260 wait complete. 2023-07-06T03:58:04.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-07-06T03:58:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.500+0000 7f4c561ff700 1 Processor -- start 2023-07-06T03:58:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.500+0000 7f4c561ff700 1 -- start start 2023-07-06T03:58:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.501+0000 7f4c561ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b8470 0x7f4c500cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.501+0000 7f4c561ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4c500b2190 0x7f4c500cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.501+0000 7f4c561ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c500d12f0 con 0x7f4c500cfc60 2023-07-06T03:58:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.501+0000 7f4c561ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c500b8840 con 0x7f4c500b2190 2023-07-06T03:58:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.501+0000 7f4c561ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c500cf390 con 0x7f4c500b8470 2023-07-06T03:58:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4c500b2190 0x7f4c500cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4c500cfc60 0x7f4c50160120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51482/0 (socket says 172.21.15.97:51482) 2023-07-06T03:58:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4ffff700 1 -- 172.21.15.97:0/4150705540 learned_addr learned my addr 172.21.15.97:0/4150705540 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4f7fe700 1 --2- 172.21.15.97:0/4150705540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b8470 0x7f4c500cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4e7fc700 1 -- 172.21.15.97:0/4150705540 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810934635 0 0) 0x7f4c500d12f0 con 0x7f4c500cfc60 2023-07-06T03:58:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4effd700 1 -- 172.21.15.97:0/4150705540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b8470 msgr2=0x7f4c500cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4effd700 1 --2- 172.21.15.97:0/4150705540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b8470 0x7f4c500cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4effd700 1 -- 172.21.15.97:0/4150705540 >> v1:172.21.15.97:6789/0 conn(0x7f4c500cfc60 legacy=0x7f4c50160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4effd700 1 -- 172.21.15.97:0/4150705540 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c500d00a0 con 0x7f4c500b2190 2023-07-06T03:58:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4f7fe700 1 --2- 172.21.15.97:0/4150705540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b8470 0x7f4c500cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:58:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4effd700 1 --2- 172.21.15.97:0/4150705540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c500b2190 0x7f4c500cf720 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009840 tx=0x7f4c44009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773cfe052da000a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.502+0000 7f4c4e7fc700 1 -- 172.21.15.97:0/4150705540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44011830 con 0x7f4c500b2190 2023-07-06T03:58:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.503+0000 7f4c4e7fc700 1 -- 172.21.15.97:0/4150705540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c44011990 con 0x7f4c500b2190 2023-07-06T03:58:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.503+0000 7f4c4e7fc700 1 -- 172.21.15.97:0/4150705540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44011d20 con 0x7f4c500b2190 2023-07-06T03:58:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.503+0000 7f4c561ff700 1 -- 172.21.15.97:0/4150705540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c500b2190 msgr2=0x7f4c500cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.503+0000 7f4c561ff700 1 --2- 172.21.15.97:0/4150705540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c500b2190 0x7f4c500cf720 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009840 tx=0x7f4c44009b50 comp rx=0 tx=0).stop 2023-07-06T03:58:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.503+0000 7f4c561ff700 1 -- 172.21.15.97:0/4150705540 shutdown_connections 2023-07-06T03:58:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.503+0000 7f4c561ff700 1 --2- 172.21.15.97:0/4150705540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b8470 0x7f4c500cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.503+0000 7f4c561ff700 1 --2- 172.21.15.97:0/4150705540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c500b2190 0x7f4c500cf720 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.503+0000 7f4c561ff700 1 -- 172.21.15.97:0/4150705540 >> 172.21.15.97:0/4150705540 conn(0x7f4c50071740 msgr2=0x7f4c50071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.503+0000 7f4c561ff700 1 -- 172.21.15.97:0/4150705540 shutdown_connections 2023-07-06T03:58:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.503+0000 7f4c561ff700 1 -- 172.21.15.97:0/4150705540 wait complete. 2023-07-06T03:58:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.505+0000 7f4c561ff700 1 Processor -- start 2023-07-06T03:58:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.505+0000 7f4c561ff700 1 -- start start 2023-07-06T03:58:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c561ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b2190 0x7f4c501e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c561ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c500b8470 0x7f4c501f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c561ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c500cfc60 0x7f4c500cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c561ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c500c6840 con 0x7f4c500b8470 2023-07-06T03:58:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c561ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c500c6a50 con 0x7f4c500cfc60 2023-07-06T03:58:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c561ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c500c6d30 con 0x7f4c500b2190 2023-07-06T03:58:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c4effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c500b8470 0x7f4c501f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c4effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c500b8470 0x7f4c501f1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48422/0 (socket says 172.21.15.97:48422) 2023-07-06T03:58:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c4effd700 1 -- 172.21.15.97:0/1715247210 learned_addr learned my addr 172.21.15.97:0/1715247210 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c4f7fe700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b2190 0x7f4c501e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c4effd700 1 -- 172.21.15.97:0/1715247210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b2190 msgr2=0x7f4c501e4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c4effd700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b2190 0x7f4c501e4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c4effd700 1 -- 172.21.15.97:0/1715247210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c500cfc60 msgr2=0x7f4c500cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c4effd700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c500cfc60 0x7f4c500cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c4effd700 1 -- 172.21.15.97:0/1715247210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440094a0 con 0x7f4c500b8470 2023-07-06T03:58:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.506+0000 7f4c4f7fe700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b2190 0x7f4c501e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.507+0000 7f4c4effd700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c500b8470 0x7f4c501f1b70 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f4c440090b0 tx=0x7f4c44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.507+0000 7f4c37fff700 1 -- 172.21.15.97:0/1715247210 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c440038a0 con 0x7f4c500b8470 2023-07-06T03:58:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.507+0000 7f4c37fff700 1 -- 172.21.15.97:0/1715247210 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4401f070 con 0x7f4c500b8470 2023-07-06T03:58:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.507+0000 7f4c561ff700 1 -- 172.21.15.97:0/1715247210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c50147d60 con 0x7f4c500b8470 2023-07-06T03:58:04.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.507+0000 7f4c37fff700 1 -- 172.21.15.97:0/1715247210 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44003c20 con 0x7f4c500b8470 2023-07-06T03:58:04.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.507+0000 7f4c561ff700 1 -- 172.21.15.97:0/1715247210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c50148320 con 0x7f4c500b8470 2023-07-06T03:58:04.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.508+0000 7f4c561ff700 1 -- 172.21.15.97:0/1715247210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c500c6f50 con 0x7f4c500b8470 2023-07-06T03:58:04.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.509+0000 7f4c37fff700 1 -- 172.21.15.97:0/1715247210 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c4401a430 con 0x7f4c500b8470 2023-07-06T03:58:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.510+0000 7f4c37fff700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c3c06ace0 0x7f4c3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.510+0000 7f4c37fff700 1 -- 172.21.15.97:0/1715247210 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4c44021070 con 0x7f4c500b8470 2023-07-06T03:58:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.510+0000 7f4c4f7fe700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c3c06ace0 0x7f4c3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.511+0000 7f4c4f7fe700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c3c06ace0 0x7f4c3c06d190 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009a00 tx=0x7f4c40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:04.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.514+0000 7f4c37fff700 1 -- 172.21.15.97:0/1715247210 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4c44055fa0 con 0x7f4c500b8470 2023-07-06T03:58:04.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.710+0000 7f4c561ff700 1 -- 172.21.15.97:0/1715247210 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c500b8ca0 con 0x7f4c3c06ace0 2023-07-06T03:58:04.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.710+0000 7f4c37fff700 1 -- 172.21.15.97:0/1715247210 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4c40009610 con 0x7f4c3c06ace0 2023-07-06T03:58:04.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 -- 172.21.15.97:0/1715247210 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c3c06ace0 msgr2=0x7f4c3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c3c06ace0 0x7f4c3c06d190 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009a00 tx=0x7f4c40008040 comp rx=0 tx=0).stop 2023-07-06T03:58:04.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 -- 172.21.15.97:0/1715247210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c500b8470 msgr2=0x7f4c501f1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c500b8470 0x7f4c501f1b70 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f4c440090b0 tx=0x7f4c44004750 comp rx=0 tx=0).stop 2023-07-06T03:58:04.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 -- 172.21.15.97:0/1715247210 shutdown_connections 2023-07-06T03:58:04.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4c3c06ace0 0x7f4c3c06d190 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c500cfc60 0x7f4c500cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4c500b2190 0x7f4c501e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 --2- 172.21.15.97:0/1715247210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c500b8470 0x7f4c501f1b70 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 -- 172.21.15.97:0/1715247210 >> 172.21.15.97:0/1715247210 conn(0x7f4c50071740 msgr2=0x7f4c500d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:04.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 -- 172.21.15.97:0/1715247210 shutdown_connections 2023-07-06T03:58:04.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.713+0000 7f4c561ff700 1 -- 172.21.15.97:0/1715247210 wait complete. 2023-07-06T03:58:04.718 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 5 to deep-scrub 2023-07-06T03:58:04.734 INFO:tasks.ceph:Scrubbing osd.6 2023-07-06T03:58:04.734 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-07-06T03:58:04.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.931+0000 7f2139fff700 1 Processor -- start 2023-07-06T03:58:04.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.931+0000 7f2139fff700 1 -- start start 2023-07-06T03:58:04.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.932+0000 7f2139fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21340cc0b0 0x7f21340cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.932+0000 7f2139fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f213406fb40 0x7f21340cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.932+0000 7f2139fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2134159760 con 0x7f21340c8fc0 2023-07-06T03:58:04.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.932+0000 7f2139fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21340c9c40 con 0x7f213406fb40 2023-07-06T03:58:04.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.932+0000 7f2139fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21340cc920 con 0x7f21340cc0b0 2023-07-06T03:58:04.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.932+0000 7f21337fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f21340c8fc0 0x7f21340c9390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51484/0 (socket says 172.21.15.97:51484) 2023-07-06T03:58:04.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.932+0000 7f21337fe700 1 -- 172.21.15.97:0/3512958852 learned_addr learned my addr 172.21.15.97:0/3512958852 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:04.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.932+0000 7f2133fff700 1 --2- 172.21.15.97:0/3512958852 >> v2:172.21.15.146:3300/0 conn(0x7f213406fb40 0x7f21340cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.932+0000 7f2132ffd700 1 --2- 172.21.15.97:0/3512958852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21340cc0b0 0x7f21340cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.933+0000 7f21327fc700 1 -- 172.21.15.97:0/3512958852 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063401624 0 0) 0x7f2124001110 con 0x7f21340c8fc0 2023-07-06T03:58:04.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.933+0000 7f21327fc700 1 -- 172.21.15.97:0/3512958852 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f211c0035b0 con 0x7f21340c8fc0 2023-07-06T03:58:04.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.933+0000 7f2133fff700 1 -- 172.21.15.97:0/3512958852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21340cc0b0 msgr2=0x7f21340cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:04.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.933+0000 7f2133fff700 1 --2- 172.21.15.97:0/3512958852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21340cc0b0 0x7f21340cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.933+0000 7f2133fff700 1 -- 172.21.15.97:0/3512958852 >> v1:172.21.15.97:6789/0 conn(0x7f21340c8fc0 legacy=0x7f21340c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.933+0000 7f21327fc700 1 -- 172.21.15.97:0/3512958852 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2549487484 0 0) 0x7f211c0035b0 con 0x7f21340c8fc0 2023-07-06T03:58:04.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.933+0000 7f2133fff700 1 -- 172.21.15.97:0/3512958852 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21340ccc00 con 0x7f213406fb40 2023-07-06T03:58:04.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.933+0000 7f2133fff700 1 --2- 172.21.15.97:0/3512958852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213406fb40 0x7f21340cd9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f212c009d90 tx=0x7f212c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c331872bc47fadc8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:04.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.934+0000 7f21327fc700 1 -- 172.21.15.97:0/3512958852 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c00e070 con 0x7f213406fb40 2023-07-06T03:58:04.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.934+0000 7f21327fc700 1 -- 172.21.15.97:0/3512958852 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212c003a20 con 0x7f213406fb40 2023-07-06T03:58:04.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.934+0000 7f21327fc700 1 -- 172.21.15.97:0/3512958852 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124001800 con 0x7f213406fb40 2023-07-06T03:58:04.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.934+0000 7f2139fff700 1 -- 172.21.15.97:0/3512958852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213406fb40 msgr2=0x7f21340cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.934+0000 7f2139fff700 1 --2- 172.21.15.97:0/3512958852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213406fb40 0x7f21340cd9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f212c009d90 tx=0x7f212c00b330 comp rx=0 tx=0).stop 2023-07-06T03:58:04.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.934+0000 7f2139fff700 1 -- 172.21.15.97:0/3512958852 shutdown_connections 2023-07-06T03:58:04.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.934+0000 7f2139fff700 1 --2- 172.21.15.97:0/3512958852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213406fb40 0x7f21340cd9c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.934+0000 7f2139fff700 1 --2- 172.21.15.97:0/3512958852 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21340cc0b0 0x7f21340cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.934+0000 7f2139fff700 1 -- 172.21.15.97:0/3512958852 >> 172.21.15.97:0/3512958852 conn(0x7f213406a010 msgr2=0x7f213406a410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:04.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.934+0000 7f2139fff700 1 -- 172.21.15.97:0/3512958852 shutdown_connections 2023-07-06T03:58:04.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.934+0000 7f2139fff700 1 -- 172.21.15.97:0/3512958852 wait complete. 2023-07-06T03:58:04.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.936+0000 7f2139fff700 1 Processor -- start 2023-07-06T03:58:04.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.936+0000 7f2139fff700 1 -- start start 2023-07-06T03:58:04.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.937+0000 7f2139fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213406fb40 0x7f213415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.937+0000 7f2139fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21340c8fc0 0x7f21340c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.937+0000 7f2139fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21340cc0b0 0x7f2134158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.937+0000 7f2139fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21340c42b0 con 0x7f21340c8fc0 2023-07-06T03:58:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.937+0000 7f2139fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21340c5b50 con 0x7f213406fb40 2023-07-06T03:58:04.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.937+0000 7f2139fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213415b600 con 0x7f21340cc0b0 2023-07-06T03:58:04.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.937+0000 7f2132ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21340c8fc0 0x7f21340c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.937+0000 7f2132ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21340c8fc0 0x7f21340c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48438/0 (socket says 172.21.15.97:48438) 2023-07-06T03:58:04.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.937+0000 7f2132ffd700 1 -- 172.21.15.97:0/2006000538 learned_addr learned my addr 172.21.15.97:0/2006000538 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:04.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.937+0000 7f21337fe700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213406fb40 0x7f213415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.937+0000 7f2133fff700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21340cc0b0 0x7f2134158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f2132ffd700 1 -- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21340cc0b0 msgr2=0x7f2134158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f2132ffd700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21340cc0b0 0x7f2134158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f2132ffd700 1 -- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213406fb40 msgr2=0x7f213415a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f2132ffd700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213406fb40 0x7f213415a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f2132ffd700 1 -- 172.21.15.97:0/2006000538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c009a70 con 0x7f21340c8fc0 2023-07-06T03:58:04.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f21337fe700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f213406fb40 0x7f213415a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:58:04.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f2133fff700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f21340cc0b0 0x7f2134158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:58:04.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f2132ffd700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21340c8fc0 0x7f21340c3d70 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f212800e870 tx=0x7f212800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:04.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f211bfff700 1 -- 172.21.15.97:0/2006000538 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128009590 con 0x7f21340c8fc0 2023-07-06T03:58:04.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f211bfff700 1 -- 172.21.15.97:0/2006000538 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212800ee70 con 0x7f21340c8fc0 2023-07-06T03:58:04.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f2139fff700 1 -- 172.21.15.97:0/2006000538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21341593f0 con 0x7f21340c8fc0 2023-07-06T03:58:04.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f211bfff700 1 -- 172.21.15.97:0/2006000538 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212800ca50 con 0x7f21340c8fc0 2023-07-06T03:58:04.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.938+0000 7f2139fff700 1 -- 172.21.15.97:0/2006000538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213415acd0 con 0x7f21340c8fc0 2023-07-06T03:58:04.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.939+0000 7f2139fff700 1 -- 172.21.15.97:0/2006000538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f21340cd1c0 con 0x7f21340c8fc0 2023-07-06T03:58:04.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.940+0000 7f211bfff700 1 -- 172.21.15.97:0/2006000538 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f212800cbb0 con 0x7f21340c8fc0 2023-07-06T03:58:04.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.941+0000 7f211bfff700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f212006ace0 0x7f212006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.941+0000 7f211bfff700 1 -- 172.21.15.97:0/2006000538 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2128089600 con 0x7f21340c8fc0 2023-07-06T03:58:04.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.942+0000 7f21337fe700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f212006ace0 0x7f212006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.942+0000 7f211bfff700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] conn(0x7f2120071150 0x7f2120073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:04.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.942+0000 7f211bfff700 1 -- 172.21.15.97:0/2006000538 --> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2120073c30 con 0x7f2120071150 2023-07-06T03:58:04.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.942+0000 7f211bfff700 1 -- 172.21.15.97:0/2006000538 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2128089970 con 0x7f21340c8fc0 2023-07-06T03:58:04.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.942+0000 7f2133fff700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] conn(0x7f2120071150 0x7f2120073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:04.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.942+0000 7f21337fe700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f212006ace0 0x7f212006d190 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2124005a70 tx=0x7f21240011b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:04.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.943+0000 7f2133fff700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] conn(0x7f2120071150 0x7f2120073560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:04.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.944+0000 7f211bfff700 1 -- 172.21.15.97:0/2006000538 <== osd.6 v2:172.21.15.146:6824/1310854864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2120073c30 con 0x7f2120071150 2023-07-06T03:58:04.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.958+0000 7f2139fff700 1 -- 172.21.15.97:0/2006000538 --> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f21340ccee0 con 0x7f2120071150 2023-07-06T03:58:04.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.959+0000 7f211bfff700 1 -- 172.21.15.97:0/2006000538 <== osd.6 v2:172.21.15.146:6824/1310854864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f212c003e00 con 0x7f2120071150 2023-07-06T03:58:04.962 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:58:04.962 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T03:58:04.963 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:58:04.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.960+0000 7f2139fff700 1 -- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] conn(0x7f2120071150 msgr2=0x7f2120073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.960+0000 7f2139fff700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.146:6824/1310854864,v1:172.21.15.146:6825/1310854864] conn(0x7f2120071150 0x7f2120073560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.960+0000 7f2139fff700 1 -- 172.21.15.97:0/2006000538 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f212006ace0 msgr2=0x7f212006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.960+0000 7f2139fff700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f212006ace0 0x7f212006d190 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2124005a70 tx=0x7f21240011b0 comp rx=0 tx=0).stop 2023-07-06T03:58:04.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.960+0000 7f2139fff700 1 -- 172.21.15.97:0/2006000538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21340c8fc0 msgr2=0x7f21340c3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:04.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.960+0000 7f2139fff700 1 --2- 172.21.15.97:0/2006000538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21340c8fc0 0x7f21340c3d70 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f212800e870 tx=0x7f212800ec30 comp rx=0 tx=0).stop 2023-07-06T03:58:04.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.960+0000 7f2133fff700 1 -- 172.21.15.97:0/2006000538 reap_dead start 2023-07-06T03:58:04.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.960+0000 7f2139fff700 1 -- 172.21.15.97:0/2006000538 shutdown_connections 2023-07-06T03:58:04.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.960+0000 7f2139fff700 1 -- 172.21.15.97:0/2006000538 >> 172.21.15.97:0/2006000538 conn(0x7f213406a010 msgr2=0x7f21340c5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:04.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.960+0000 7f2139fff700 1 -- 172.21.15.97:0/2006000538 shutdown_connections 2023-07-06T03:58:04.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:04.960+0000 7f2139fff700 1 -- 172.21.15.97:0/2006000538 wait complete. 2023-07-06T03:58:04.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-07-06T03:58:05.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.172+0000 7fdbf39ff700 1 Processor -- start 2023-07-06T03:58:05.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.172+0000 7fdbf39ff700 1 -- start start 2023-07-06T03:58:05.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.173+0000 7fdbf39ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdbec0ae8f0 0x7fdbec0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.173+0000 7fdbf39ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec0aecc0 con 0x7fdbec06a010 2023-07-06T03:58:05.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.173+0000 7fdbf39ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec071320 con 0x7fdbec070e60 2023-07-06T03:58:05.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.173+0000 7fdbf39ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbec06de40 con 0x7fdbec0ae8f0 2023-07-06T03:58:05.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.174+0000 7fdbf179b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdbec0ae8f0 0x7fdbec0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.174+0000 7fdbf1f9c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdbec06a010 0x7fdbec06a410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51500/0 (socket says 172.21.15.97:51500) 2023-07-06T03:58:05.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.174+0000 7fdbf1f9c700 1 -- 172.21.15.97:0/2699306652 learned_addr learned my addr 172.21.15.97:0/2699306652 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:05.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.174+0000 7fdbf179b700 1 -- 172.21.15.97:0/2699306652 >> v1:172.21.15.146:6789/0 conn(0x7fdbec070e60 legacy=0x7fdbec06d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:05.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.174+0000 7fdbf179b700 1 -- 172.21.15.97:0/2699306652 >> v1:172.21.15.97:6789/0 conn(0x7fdbec06a010 legacy=0x7fdbec06a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.174+0000 7fdbf179b700 1 -- 172.21.15.97:0/2699306652 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec071600 con 0x7fdbec0ae8f0 2023-07-06T03:58:05.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.174+0000 7fdbe3fff700 1 -- 172.21.15.97:0/2699306652 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606622085 0 0) 0x7fdbe4001490 con 0x7fdbec06a010 2023-07-06T03:58:05.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.174+0000 7fdbf179b700 1 --2- 172.21.15.97:0/2699306652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbec0ae8f0 0x7fdbec0ac880 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009840 tx=0x7fdbe8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b1bd54af46001c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:05.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.174+0000 7fdbe3fff700 1 -- 172.21.15.97:0/2699306652 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8004230 con 0x7fdbec0ae8f0 2023-07-06T03:58:05.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.175+0000 7fdbe3fff700 1 -- 172.21.15.97:0/2699306652 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe8004390 con 0x7fdbec0ae8f0 2023-07-06T03:58:05.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.175+0000 7fdbe3fff700 1 -- 172.21.15.97:0/2699306652 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80036a0 con 0x7fdbec0ae8f0 2023-07-06T03:58:05.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.175+0000 7fdbf39ff700 1 -- 172.21.15.97:0/2699306652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbec0ae8f0 msgr2=0x7fdbec0ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.175+0000 7fdbf39ff700 1 --2- 172.21.15.97:0/2699306652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbec0ae8f0 0x7fdbec0ac880 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009840 tx=0x7fdbe8009b50 comp rx=0 tx=0).stop 2023-07-06T03:58:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.175+0000 7fdbf39ff700 1 -- 172.21.15.97:0/2699306652 shutdown_connections 2023-07-06T03:58:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.175+0000 7fdbf39ff700 1 --2- 172.21.15.97:0/2699306652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbec0ae8f0 0x7fdbec0ac880 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.175+0000 7fdbf39ff700 1 -- 172.21.15.97:0/2699306652 >> 172.21.15.97:0/2699306652 conn(0x7fdbec06bdb0 msgr2=0x7fdbec0b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.175+0000 7fdbf39ff700 1 -- 172.21.15.97:0/2699306652 shutdown_connections 2023-07-06T03:58:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.175+0000 7fdbf39ff700 1 -- 172.21.15.97:0/2699306652 wait complete. 2023-07-06T03:58:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.177+0000 7fdbf39ff700 1 Processor -- start 2023-07-06T03:58:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.177+0000 7fdbf39ff700 1 -- start start 2023-07-06T03:58:05.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.178+0000 7fdbf39ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbec06a010 0x7fdbec157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.178+0000 7fdbf39ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbec070e60 0x7fdbec0aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.178+0000 7fdbf39ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdbec0ae8f0 0x7fdbec0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.178+0000 7fdbf39ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec1577b0 con 0x7fdbec070e60 2023-07-06T03:58:05.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.178+0000 7fdbf39ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec156b30 con 0x7fdbec0ae8f0 2023-07-06T03:58:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.178+0000 7fdbf39ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec156e10 con 0x7fdbec06a010 2023-07-06T03:58:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.178+0000 7fdbf0f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbec070e60 0x7fdbec0aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.178+0000 7fdbf0f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbec070e60 0x7fdbec0aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48448/0 (socket says 172.21.15.97:48448) 2023-07-06T03:58:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.178+0000 7fdbf0f9a700 1 -- 172.21.15.97:0/3688793262 learned_addr learned my addr 172.21.15.97:0/3688793262 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:05.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbf0f9a700 1 -- 172.21.15.97:0/3688793262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbec06a010 msgr2=0x7fdbec157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbf0f9a700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbec06a010 0x7fdbec157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbf0f9a700 1 -- 172.21.15.97:0/3688793262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdbec0ae8f0 msgr2=0x7fdbec0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbf1f9c700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdbec0ae8f0 0x7fdbec0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbf0f9a700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdbec0ae8f0 0x7fdbec0b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbf0f9a700 1 -- 172.21.15.97:0/3688793262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe80094a0 con 0x7fdbec070e60 2023-07-06T03:58:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbf1f9c700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdbec0ae8f0 0x7fdbec0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbf0f9a700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbec070e60 0x7fdbec0aff40 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00c4a0 tx=0x7fdbdc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbe1ffb700 1 -- 172.21.15.97:0/3688793262 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0054e0 con 0x7fdbec070e60 2023-07-06T03:58:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbe1ffb700 1 -- 172.21.15.97:0/3688793262 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc003680 con 0x7fdbec070e60 2023-07-06T03:58:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbe1ffb700 1 -- 172.21.15.97:0/3688793262 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc003a10 con 0x7fdbec070e60 2023-07-06T03:58:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbf39ff700 1 -- 172.21.15.97:0/3688793262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbec1570f0 con 0x7fdbec070e60 2023-07-06T03:58:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.179+0000 7fdbf39ff700 1 -- 172.21.15.97:0/3688793262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec158110 con 0x7fdbec070e60 2023-07-06T03:58:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.180+0000 7fdbf39ff700 1 -- 172.21.15.97:0/3688793262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbec0af2c0 con 0x7fdbec070e60 2023-07-06T03:58:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.181+0000 7fdbe1ffb700 1 -- 172.21.15.97:0/3688793262 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdbdc003b70 con 0x7fdbec070e60 2023-07-06T03:58:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.182+0000 7fdbe1ffb700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbd006ace0 0x7fdbd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.182+0000 7fdbe1ffb700 1 -- 172.21.15.97:0/3688793262 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdbdc0883a0 con 0x7fdbec070e60 2023-07-06T03:58:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.183+0000 7fdbf179b700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbd006ace0 0x7fdbd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.183+0000 7fdbf179b700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbd006ace0 0x7fdbd006d190 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80090b0 tx=0x7fdbe8013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.187+0000 7fdbe1ffb700 1 -- 172.21.15.97:0/3688793262 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdbdc053750 con 0x7fdbec070e60 2023-07-06T03:58:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.380+0000 7fdbf39ff700 1 -- 172.21.15.97:0/3688793262 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbec0ce790 con 0x7fdbd006ace0 2023-07-06T03:58:05.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.381+0000 7fdbe1ffb700 1 -- 172.21.15.97:0/3688793262 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdbe80040b0 con 0x7fdbd006ace0 2023-07-06T03:58:05.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 -- 172.21.15.97:0/3688793262 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbd006ace0 msgr2=0x7fdbd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbd006ace0 0x7fdbd006d190 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80090b0 tx=0x7fdbe8013040 comp rx=0 tx=0).stop 2023-07-06T03:58:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 -- 172.21.15.97:0/3688793262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbec070e60 msgr2=0x7fdbec0aff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbec070e60 0x7fdbec0aff40 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00c4a0 tx=0x7fdbdc00c860 comp rx=0 tx=0).stop 2023-07-06T03:58:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 -- 172.21.15.97:0/3688793262 shutdown_connections 2023-07-06T03:58:05.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fdbd006ace0 0x7fdbd006d190 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdbec0ae8f0 0x7fdbec0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fdbec06a010 0x7fdbec157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 --2- 172.21.15.97:0/3688793262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbec070e60 0x7fdbec0aff40 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 -- 172.21.15.97:0/3688793262 >> 172.21.15.97:0/3688793262 conn(0x7fdbec06bdb0 msgr2=0x7fdbec06a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:05.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 -- 172.21.15.97:0/3688793262 shutdown_connections 2023-07-06T03:58:05.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.383+0000 7fdbf39ff700 1 -- 172.21.15.97:0/3688793262 wait complete. 2023-07-06T03:58:05.388 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 6 to deep-scrub 2023-07-06T03:58:05.403 INFO:tasks.ceph:Scrubbing osd.7 2023-07-06T03:58:05.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-07-06T03:58:05.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.598+0000 7f61525ff700 1 Processor -- start 2023-07-06T03:58:05.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.599+0000 7f61525ff700 1 -- start start 2023-07-06T03:58:05.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.600+0000 7f61525ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f614c0b1d20 0x7f614c161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.600+0000 7f61525ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c0d0010 0x7f614c0d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.600+0000 7f61525ff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614c0cd6d0 con 0x7f614c15ce10 2023-07-06T03:58:05.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.600+0000 7f61525ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614c15d1e0 con 0x7f614c0b1d20 2023-07-06T03:58:05.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.600+0000 7f61525ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c161780 con 0x7f614c0d0010 2023-07-06T03:58:05.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.600+0000 7f614bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f614c15ce10 0x7f614c0cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51510/0 (socket says 172.21.15.97:51510) 2023-07-06T03:58:05.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.600+0000 7f614bfff700 1 -- 172.21.15.97:0/553749240 learned_addr learned my addr 172.21.15.97:0/553749240 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:05.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.600+0000 7f614b7fe700 1 --2- 172.21.15.97:0/553749240 >> v2:172.21.15.146:3300/0 conn(0x7f614c0b1d20 0x7f614c161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.600+0000 7f6150b9c700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c0d0010 0x7f614c0d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.600+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880581768 0 0) 0x7f614c0cd6d0 con 0x7f614c15ce10 2023-07-06T03:58:05.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.600+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61340035b0 con 0x7f614c15ce10 2023-07-06T03:58:05.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3186329520 0 0) 0x7f61340035b0 con 0x7f614c15ce10 2023-07-06T03:58:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c0d0010 msgr2=0x7f614c0d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c0d0010 0x7f614c0d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 >> v2:172.21.15.146:3300/0 conn(0x7f614c0b1d20 msgr2=0x7f614c161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 --2- 172.21.15.97:0/553749240 >> v2:172.21.15.146:3300/0 conn(0x7f614c0b1d20 0x7f614c161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c0cd9b0 con 0x7f614c15ce10 2023-07-06T03:58:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1502455339 0 0) 0x7f613c001670 con 0x7f614c15ce10 2023-07-06T03:58:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 >> v1:172.21.15.97:6789/0 conn(0x7f614c15ce10 legacy=0x7f614c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6134003a00 0x7f6134003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6134004850 0x7f6134004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c0d0010 0x7f6134005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c161780 con 0x7f6134004850 2023-07-06T03:58:05.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613400ba10 con 0x7f6134003a00 2023-07-06T03:58:05.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613400bb80 con 0x7f614c0d0010 2023-07-06T03:58:05.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614b7fe700 1 --2- 172.21.15.97:0/553749240 >> v2:172.21.15.146:3300/0 conn(0x7f614c0b1d20 0x7f614c161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:58:05.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614bfff700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6134003a00 0x7f6134003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f614b7fe700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6134004850 0x7f6134004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.601+0000 7f6150b9c700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c0d0010 0x7f6134005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.602+0000 7f614b7fe700 1 -- 172.21.15.97:0/553749240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c0d0010 msgr2=0x7f6134005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.602+0000 7f614b7fe700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c0d0010 0x7f6134005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.602+0000 7f614b7fe700 1 -- 172.21.15.97:0/553749240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6134003a00 msgr2=0x7f6134003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.602+0000 7f614b7fe700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6134003a00 0x7f6134003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.602+0000 7f614b7fe700 1 -- 172.21.15.97:0/553749240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61340035b0 con 0x7f6134004850 2023-07-06T03:58:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.602+0000 7f614b7fe700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6134004850 0x7f6134004cc0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f6140009d10 tx=0x7f6134005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f5027f811bac49 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.602+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6140009e60 con 0x7f6134004850 2023-07-06T03:58:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.602+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614000e9a0 con 0x7f6134004850 2023-07-06T03:58:05.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.602+0000 7f614affd700 1 -- 172.21.15.97:0/553749240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614000ed10 con 0x7f6134004850 2023-07-06T03:58:05.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.603+0000 7f61525ff700 1 -- 172.21.15.97:0/553749240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6134004850 msgr2=0x7f6134004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.603+0000 7f61525ff700 1 --2- 172.21.15.97:0/553749240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6134004850 0x7f6134004cc0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f6140009d10 tx=0x7f6134005880 comp rx=0 tx=0).stop 2023-07-06T03:58:05.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.603+0000 7f6150b9c700 1 -- 172.21.15.97:0/553749240 reap_dead start 2023-07-06T03:58:05.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.603+0000 7f61525ff700 1 -- 172.21.15.97:0/553749240 shutdown_connections 2023-07-06T03:58:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.603+0000 7f61525ff700 1 -- 172.21.15.97:0/553749240 >> 172.21.15.97:0/553749240 conn(0x7f614c06b500 msgr2=0x7f614c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.603+0000 7f61525ff700 1 -- 172.21.15.97:0/553749240 shutdown_connections 2023-07-06T03:58:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.603+0000 7f61525ff700 1 -- 172.21.15.97:0/553749240 wait complete. 2023-07-06T03:58:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.604+0000 7f61525ff700 1 Processor -- start 2023-07-06T03:58:05.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.605+0000 7f61525ff700 1 -- start start 2023-07-06T03:58:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.605+0000 7f61525ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6134004850 0x7f614c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.605+0000 7f61525ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c15d1e0 0x7f614c150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.605+0000 7f61525ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f614c0ca9c0 0x7f614c0cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.605+0000 7f61525ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c15e690 con 0x7f614c0ca9c0 2023-07-06T03:58:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.605+0000 7f61525ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0cb370 con 0x7f6134004850 2023-07-06T03:58:05.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.605+0000 7f61525ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c15d9f0 con 0x7f614c15d1e0 2023-07-06T03:58:05.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.606+0000 7f6150b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f614c0ca9c0 0x7f614c0cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.606+0000 7f6150b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f614c0ca9c0 0x7f614c0cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48464/0 (socket says 172.21.15.97:48464) 2023-07-06T03:58:05.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.606+0000 7f6150b9c700 1 -- 172.21.15.97:0/273649914 learned_addr learned my addr 172.21.15.97:0/273649914 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:05.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.606+0000 7f614bfff700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6134004850 0x7f614c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.606+0000 7f614b7fe700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c15d1e0 0x7f614c150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.606+0000 7f6150b9c700 1 -- 172.21.15.97:0/273649914 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c15d1e0 msgr2=0x7f614c150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.606+0000 7f6150b9c700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f614c15d1e0 0x7f614c150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.606+0000 7f6150b9c700 1 -- 172.21.15.97:0/273649914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6134004850 msgr2=0x7f614c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.606+0000 7f6150b9c700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6134004850 0x7f614c0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.606+0000 7f6150b9c700 1 -- 172.21.15.97:0/273649914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c15dcd0 con 0x7f614c0ca9c0 2023-07-06T03:58:05.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.606+0000 7f6150b9c700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f614c0ca9c0 0x7f614c0cae30 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f614c08bfa0 tx=0x7f6144011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:05.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.607+0000 7f6148ff9700 1 -- 172.21.15.97:0/273649914 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614400e040 con 0x7f614c0ca9c0 2023-07-06T03:58:05.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.607+0000 7f6148ff9700 1 -- 172.21.15.97:0/273649914 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614400f040 con 0x7f614c0ca9c0 2023-07-06T03:58:05.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.607+0000 7f61525ff700 1 -- 172.21.15.97:0/273649914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614c15dfb0 con 0x7f614c0ca9c0 2023-07-06T03:58:05.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.607+0000 7f6148ff9700 1 -- 172.21.15.97:0/273649914 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614400fe40 con 0x7f614c0ca9c0 2023-07-06T03:58:05.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.607+0000 7f61525ff700 1 -- 172.21.15.97:0/273649914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614c15d650 con 0x7f614c0ca9c0 2023-07-06T03:58:05.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.608+0000 7f61525ff700 1 -- 172.21.15.97:0/273649914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f614c0cdf70 con 0x7f614c0ca9c0 2023-07-06T03:58:05.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.609+0000 7f6148ff9700 1 -- 172.21.15.97:0/273649914 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6144015430 con 0x7f614c0ca9c0 2023-07-06T03:58:05.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.610+0000 7f6148ff9700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f613806ace0 0x7f613806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.610+0000 7f6148ff9700 1 -- 172.21.15.97:0/273649914 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6144089530 con 0x7f614c0ca9c0 2023-07-06T03:58:05.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.610+0000 7f614bfff700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f613806ace0 0x7f613806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.610+0000 7f6148ff9700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] conn(0x7f6138071150 0x7f6138073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.610+0000 7f6148ff9700 1 -- 172.21.15.97:0/273649914 --> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6138073c30 con 0x7f6138071150 2023-07-06T03:58:05.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.610+0000 7f6148ff9700 1 -- 172.21.15.97:0/273649914 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7f614408bb10 con 0x7f614c0ca9c0 2023-07-06T03:58:05.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.611+0000 7f614b7fe700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] conn(0x7f6138071150 0x7f6138073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.611+0000 7f614bfff700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f613806ace0 0x7f613806d190 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f613c000c00 tx=0x7f613c001dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:05.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.611+0000 7f614b7fe700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] conn(0x7f6138071150 0x7f6138073560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:05.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.612+0000 7f6148ff9700 1 -- 172.21.15.97:0/273649914 <== osd.7 v2:172.21.15.146:6808/1299847983 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6138073c30 con 0x7f6138071150 2023-07-06T03:58:05.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.627+0000 7f61525ff700 1 -- 172.21.15.97:0/273649914 --> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f614c0cdc90 con 0x7f6138071150 2023-07-06T03:58:05.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.627+0000 7f6148ff9700 1 -- 172.21.15.97:0/273649914 <== osd.7 v2:172.21.15.146:6808/1299847983 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6140022040 con 0x7f6138071150 2023-07-06T03:58:05.630 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-06T03:58:05.630 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T03:58:05.630 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-06T03:58:05.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.628+0000 7f61525ff700 1 -- 172.21.15.97:0/273649914 >> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] conn(0x7f6138071150 msgr2=0x7f6138073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.628+0000 7f61525ff700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.146:6808/1299847983,v1:172.21.15.146:6810/1299847983] conn(0x7f6138071150 0x7f6138073560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.628+0000 7f61525ff700 1 -- 172.21.15.97:0/273649914 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f613806ace0 msgr2=0x7f613806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.628+0000 7f61525ff700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f613806ace0 0x7f613806d190 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f613c000c00 tx=0x7f613c001dd0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.628+0000 7f61525ff700 1 -- 172.21.15.97:0/273649914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f614c0ca9c0 msgr2=0x7f614c0cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.628+0000 7f61525ff700 1 --2- 172.21.15.97:0/273649914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f614c0ca9c0 0x7f614c0cae30 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f614c08bfa0 tx=0x7f6144011f60 comp rx=0 tx=0).stop 2023-07-06T03:58:05.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.628+0000 7f6150b9c700 1 -- 172.21.15.97:0/273649914 reap_dead start 2023-07-06T03:58:05.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.628+0000 7f61525ff700 1 -- 172.21.15.97:0/273649914 shutdown_connections 2023-07-06T03:58:05.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.628+0000 7f61525ff700 1 -- 172.21.15.97:0/273649914 >> 172.21.15.97:0/273649914 conn(0x7f614c06b500 msgr2=0x7f614c0b79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:05.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.628+0000 7f61525ff700 1 -- 172.21.15.97:0/273649914 shutdown_connections 2023-07-06T03:58:05.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.628+0000 7f61525ff700 1 -- 172.21.15.97:0/273649914 wait complete. 2023-07-06T03:58:05.647 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-07-06T03:58:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.838+0000 7f2ef5dff700 1 Processor -- start 2023-07-06T03:58:05.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.838+0000 7f2ef5dff700 1 -- start start 2023-07-06T03:58:05.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.839+0000 7f2ef5dff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ef00b8470 0x7f2ef00cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.839+0000 7f2ef5dff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef00d12f0 con 0x7f2ef00b2190 2023-07-06T03:58:05.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.839+0000 7f2ef5dff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef00cf390 con 0x7f2ef00cfe30 2023-07-06T03:58:05.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.839+0000 7f2ef5dff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef00b8840 con 0x7f2ef00b8470 2023-07-06T03:58:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.839+0000 7f2eef7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ef00b8470 0x7f2ef00cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.839+0000 7f2ee7fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2ef00b2190 0x7f2ef00cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51520/0 (socket says 172.21.15.97:51520) 2023-07-06T03:58:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.839+0000 7f2ee7fff700 1 -- 172.21.15.97:0/2730335602 learned_addr learned my addr 172.21.15.97:0/2730335602 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:05.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.840+0000 7f2eef7fe700 1 -- 172.21.15.97:0/2730335602 >> v1:172.21.15.146:6789/0 conn(0x7f2ef00cfe30 legacy=0x7f2ef00d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.840+0000 7f2eef7fe700 1 -- 172.21.15.97:0/2730335602 >> v1:172.21.15.97:6789/0 conn(0x7f2ef00b2190 legacy=0x7f2ef00cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.840+0000 7f2eef7fe700 1 -- 172.21.15.97:0/2730335602 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00b8ca0 con 0x7f2ef00b8470 2023-07-06T03:58:05.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.840+0000 7f2eeeffd700 1 -- 172.21.15.97:0/2730335602 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282639898 0 0) 0x7f2ef00d12f0 con 0x7f2ef00b2190 2023-07-06T03:58:05.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.840+0000 7f2eef7fe700 1 --2- 172.21.15.97:0/2730335602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef00b8470 0x7f2ef00cee50 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009840 tx=0x7f2ee0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3940efda4782e2b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:05.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.840+0000 7f2eeeffd700 1 -- 172.21.15.97:0/2730335602 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0011830 con 0x7f2ef00b8470 2023-07-06T03:58:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.840+0000 7f2eeeffd700 1 -- 172.21.15.97:0/2730335602 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee0011990 con 0x7f2ef00b8470 2023-07-06T03:58:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.840+0000 7f2eeeffd700 1 -- 172.21.15.97:0/2730335602 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0011d20 con 0x7f2ef00b8470 2023-07-06T03:58:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.841+0000 7f2ef5dff700 1 -- 172.21.15.97:0/2730335602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef00b8470 msgr2=0x7f2ef00cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.841+0000 7f2ef5dff700 1 --2- 172.21.15.97:0/2730335602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef00b8470 0x7f2ef00cee50 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009840 tx=0x7f2ee0009b50 comp rx=0 tx=0).stop 2023-07-06T03:58:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.841+0000 7f2ef5dff700 1 -- 172.21.15.97:0/2730335602 shutdown_connections 2023-07-06T03:58:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.841+0000 7f2ef5dff700 1 --2- 172.21.15.97:0/2730335602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef00b8470 0x7f2ef00cee50 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.841+0000 7f2ef5dff700 1 -- 172.21.15.97:0/2730335602 >> 172.21.15.97:0/2730335602 conn(0x7f2ef0071740 msgr2=0x7f2ef0071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.841+0000 7f2ef5dff700 1 -- 172.21.15.97:0/2730335602 shutdown_connections 2023-07-06T03:58:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.841+0000 7f2ef5dff700 1 -- 172.21.15.97:0/2730335602 wait complete. 2023-07-06T03:58:05.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.843+0000 7f2ef5dff700 1 Processor -- start 2023-07-06T03:58:05.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.844+0000 7f2ef5dff700 1 -- start start 2023-07-06T03:58:05.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2ef5dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef00b2190 0x7f2ef014b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2ef5dff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef00b8470 0x7f2ef00d2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2ef5dff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ef00cfe30 0x7f2ef00d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2ef5dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef014b700 con 0x7f2ef00b2190 2023-07-06T03:58:05.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2ef5dff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef014ba60 con 0x7f2ef00cfe30 2023-07-06T03:58:05.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2ef5dff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00c9300 con 0x7f2ef00b8470 2023-07-06T03:58:05.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2eef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef00b2190 0x7f2ef014b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2eef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef00b2190 0x7f2ef014b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48486/0 (socket says 172.21.15.97:48486) 2023-07-06T03:58:05.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2eef7fe700 1 -- 172.21.15.97:0/1537916286 learned_addr learned my addr 172.21.15.97:0/1537916286 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:05.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2ee7fff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef00b8470 0x7f2ef00d2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2eeffff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ef00cfe30 0x7f2ef00d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2eef7fe700 1 -- 172.21.15.97:0/1537916286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef00b8470 msgr2=0x7f2ef00d2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2eef7fe700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef00b8470 0x7f2ef00d2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2eef7fe700 1 -- 172.21.15.97:0/1537916286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ef00cfe30 msgr2=0x7f2ef00d31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:05.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2eef7fe700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ef00cfe30 0x7f2ef00d31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:05.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2eef7fe700 1 -- 172.21.15.97:0/1537916286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee00094a0 con 0x7f2ef00b2190 2023-07-06T03:58:05.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2ee7fff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef00b8470 0x7f2ef00d2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:05.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.845+0000 7f2eeffff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ef00cfe30 0x7f2ef00d31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T03:58:05.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.846+0000 7f2eef7fe700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef00b2190 0x7f2ef014b1c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00090b0 tx=0x7f2ee0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:05.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.846+0000 7f2eecff9700 1 -- 172.21.15.97:0/1537916286 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee001a900 con 0x7f2ef00b2190 2023-07-06T03:58:05.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.846+0000 7f2eecff9700 1 -- 172.21.15.97:0/1537916286 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee001aa60 con 0x7f2ef00b2190 2023-07-06T03:58:05.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.846+0000 7f2ef5dff700 1 -- 172.21.15.97:0/1537916286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef00c95e0 con 0x7f2ef00b2190 2023-07-06T03:58:05.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.846+0000 7f2eecff9700 1 -- 172.21.15.97:0/1537916286 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee001add0 con 0x7f2ef00b2190 2023-07-06T03:58:05.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.846+0000 7f2ef5dff700 1 -- 172.21.15.97:0/1537916286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef014bf90 con 0x7f2ef00b2190 2023-07-06T03:58:05.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.847+0000 7f2ef5dff700 1 -- 172.21.15.97:0/1537916286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef004e340 con 0x7f2ef00b2190 2023-07-06T03:58:05.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.848+0000 7f2eecff9700 1 -- 172.21.15.97:0/1537916286 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ee0022070 con 0x7f2ef00b2190 2023-07-06T03:58:05.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.850+0000 7f2eecff9700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed006ace0 0x7f2ed006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:05.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.850+0000 7f2eecff9700 1 -- 172.21.15.97:0/1537916286 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2ee008a3d0 con 0x7f2ef00b2190 2023-07-06T03:58:05.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.850+0000 7f2ee7fff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed006ace0 0x7f2ed006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:05.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.851+0000 7f2ee7fff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed006ace0 0x7f2ed006d190 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00d3940 tx=0x7f2ed8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:05.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:05.854+0000 7f2eecff9700 1 -- 172.21.15.97:0/1537916286 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2ee0055960 con 0x7f2ef00b2190 2023-07-06T03:58:06.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.053+0000 7f2ef5dff700 1 -- 172.21.15.97:0/1537916286 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ef00cd7e0 con 0x7f2ed006ace0 2023-07-06T03:58:06.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.053+0000 7f2eecff9700 1 -- 172.21.15.97:0/1537916286 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2ed8001c90 con 0x7f2ed006ace0 2023-07-06T03:58:06.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 -- 172.21.15.97:0/1537916286 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed006ace0 msgr2=0x7f2ed006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:06.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed006ace0 0x7f2ed006d190 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00d3940 tx=0x7f2ed8009040 comp rx=0 tx=0).stop 2023-07-06T03:58:06.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 -- 172.21.15.97:0/1537916286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef00b2190 msgr2=0x7f2ef014b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:06.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef00b2190 0x7f2ef014b1c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00090b0 tx=0x7f2ee0004750 comp rx=0 tx=0).stop 2023-07-06T03:58:06.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 -- 172.21.15.97:0/1537916286 shutdown_connections 2023-07-06T03:58:06.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f2ed006ace0 0x7f2ed006d190 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ef00cfe30 0x7f2ef00d31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ef00b2190 0x7f2ef014b1c0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 --2- 172.21.15.97:0/1537916286 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f2ef00b8470 0x7f2ef00d2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 -- 172.21.15.97:0/1537916286 >> 172.21.15.97:0/1537916286 conn(0x7f2ef0071740 msgr2=0x7f2ef00b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 -- 172.21.15.97:0/1537916286 shutdown_connections 2023-07-06T03:58:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.056+0000 7f2ef5dff700 1 -- 172.21.15.97:0/1537916286 wait complete. 2023-07-06T03:58:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 7 to deep-scrub 2023-07-06T03:58:06.076 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:58:06.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.267+0000 7fd007fff700 1 Processor -- start 2023-07-06T03:58:06.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.268+0000 7fd007fff700 1 -- start start 2023-07-06T03:58:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.269+0000 7fd007fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0000cfe30 0x7fd0000cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.269+0000 7fd007fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0000d12f0 con 0x7fd0000b8470 2023-07-06T03:58:06.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.269+0000 7fd007fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0000cf390 con 0x7fd0000b2190 2023-07-06T03:58:06.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.269+0000 7fd007fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0000b8840 con 0x7fd0000cfe30 2023-07-06T03:58:06.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.269+0000 7fd005d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd0000b8470 0x7fd0000cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51522/0 (socket says 172.21.15.97:51522) 2023-07-06T03:58:06.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.269+0000 7fd005d9b700 1 -- 172.21.15.97:0/3298659168 learned_addr learned my addr 172.21.15.97:0/3298659168 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:06.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.269+0000 7fd00559a700 1 --2- 172.21.15.97:0/3298659168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0000cfe30 0x7fd0000cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:06.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.269+0000 7fd004d99700 1 -- 172.21.15.97:0/3298659168 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303248845 0 0) 0x7fd0000d12f0 con 0x7fd0000b8470 2023-07-06T03:58:06.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.269+0000 7fd004d99700 1 -- 172.21.15.97:0/3298659168 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfec0035b0 con 0x7fd0000b8470 2023-07-06T03:58:06.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.270+0000 7fd00559a700 1 -- 172.21.15.97:0/3298659168 >> v1:172.21.15.146:6789/0 conn(0x7fd0000b2190 legacy=0x7fd0000d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:06.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.270+0000 7fd004d99700 1 -- 172.21.15.97:0/3298659168 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372904248 0 0) 0x7fd0000cf390 con 0x7fd0000b2190 2023-07-06T03:58:06.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.270+0000 7fd00559a700 1 -- 172.21.15.97:0/3298659168 >> v1:172.21.15.97:6789/0 conn(0x7fd0000b8470 legacy=0x7fd0000cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:06.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.270+0000 7fd00559a700 1 -- 172.21.15.97:0/3298659168 --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0000b8ca0 con 0x7fd0000cfe30 2023-07-06T03:58:06.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.270+0000 7fd00559a700 1 --2- 172.21.15.97:0/3298659168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0000cfe30 0x7fd0000cee50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009870 tx=0x7fcff0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62cc0ac2f8b117c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:06.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.270+0000 7fd004d99700 1 -- 172.21.15.97:0/3298659168 <== mon.2 v2:172.21.15.146:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0004230 con 0x7fd0000cfe30 2023-07-06T03:58:06.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.270+0000 7fd004d99700 1 -- 172.21.15.97:0/3298659168 <== mon.2 v2:172.21.15.146:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff0004390 con 0x7fd0000cfe30 2023-07-06T03:58:06.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.270+0000 7fd004d99700 1 -- 172.21.15.97:0/3298659168 <== mon.2 v2:172.21.15.146:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff00036a0 con 0x7fd0000cfe30 2023-07-06T03:58:06.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.271+0000 7fd007fff700 1 -- 172.21.15.97:0/3298659168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0000cfe30 msgr2=0x7fd0000cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:06.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.271+0000 7fd007fff700 1 --2- 172.21.15.97:0/3298659168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0000cfe30 0x7fd0000cee50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009870 tx=0x7fcff0009b80 comp rx=0 tx=0).stop 2023-07-06T03:58:06.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.271+0000 7fd007fff700 1 -- 172.21.15.97:0/3298659168 shutdown_connections 2023-07-06T03:58:06.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.271+0000 7fd007fff700 1 --2- 172.21.15.97:0/3298659168 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0000cfe30 0x7fd0000cee50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:06.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.271+0000 7fd007fff700 1 -- 172.21.15.97:0/3298659168 >> 172.21.15.97:0/3298659168 conn(0x7fd000071740 msgr2=0x7fd000071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:06.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.271+0000 7fd007fff700 1 -- 172.21.15.97:0/3298659168 shutdown_connections 2023-07-06T03:58:06.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.271+0000 7fd007fff700 1 -- 172.21.15.97:0/3298659168 wait complete. 2023-07-06T03:58:06.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.273+0000 7fd007fff700 1 Processor -- start 2023-07-06T03:58:06.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.273+0000 7fd007fff700 1 -- start start 2023-07-06T03:58:06.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.274+0000 7fd007fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0000b2190 0x7fd0001532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:06.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.274+0000 7fd007fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0000b8470 0x7fd000157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:06.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.274+0000 7fd007fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0000cfe30 0x7fd000158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:06.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.274+0000 7fd007fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0001588d0 con 0x7fd0000b2190 2023-07-06T03:58:06.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.274+0000 7fd007fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd000158bb0 con 0x7fd0000cfe30 2023-07-06T03:58:06.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.274+0000 7fd007fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd000158e90 con 0x7fd0000b8470 2023-07-06T03:58:06.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.274+0000 7fd005d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0000b2190 0x7fd0001532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:06.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.274+0000 7fd005d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0000b2190 0x7fd0001532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48498/0 (socket says 172.21.15.97:48498) 2023-07-06T03:58:06.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.274+0000 7fd005d9b700 1 -- 172.21.15.97:0/1964106389 learned_addr learned my addr 172.21.15.97:0/1964106389 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:06.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.274+0000 7fd00559a700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0000b8470 0x7fd000157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.274+0000 7fd00659c700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0000cfe30 0x7fd000158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.275+0000 7fd005d9b700 1 -- 172.21.15.97:0/1964106389 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0000b8470 msgr2=0x7fd000157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.275+0000 7fd005d9b700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0000b8470 0x7fd000157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.275+0000 7fd005d9b700 1 -- 172.21.15.97:0/1964106389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0000cfe30 msgr2=0x7fd000158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.275+0000 7fd005d9b700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0000cfe30 0x7fd000158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:06.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.275+0000 7fd005d9b700 1 -- 172.21.15.97:0/1964106389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff00094a0 con 0x7fd0000b2190 2023-07-06T03:58:06.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.275+0000 7fd005d9b700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0000b2190 0x7fd0001532b0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c780 tx=0x7fcffc00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:06.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.275+0000 7fcff67fc700 1 -- 172.21.15.97:0/1964106389 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc005d70 con 0x7fd0000b2190 2023-07-06T03:58:06.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.275+0000 7fcff67fc700 1 -- 172.21.15.97:0/1964106389 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc0070d0 con 0x7fd0000b2190 2023-07-06T03:58:06.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.275+0000 7fd007fff700 1 -- 172.21.15.97:0/1964106389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd000159170 con 0x7fd0000b2190 2023-07-06T03:58:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.275+0000 7fcff67fc700 1 -- 172.21.15.97:0/1964106389 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc007460 con 0x7fd0000b2190 2023-07-06T03:58:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.275+0000 7fd007fff700 1 -- 172.21.15.97:0/1964106389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd000158590 con 0x7fd0000b2190 2023-07-06T03:58:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.276+0000 7fd007fff700 1 -- 172.21.15.97:0/1964106389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0000c6c30 con 0x7fd0000b2190 2023-07-06T03:58:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.277+0000 7fcff67fc700 1 -- 172.21.15.97:0/1964106389 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcffc0075c0 con 0x7fd0000b2190 2023-07-06T03:58:06.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.278+0000 7fcff67fc700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfe406ace0 0x7fcfe406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:06.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.278+0000 7fcff67fc700 1 -- 172.21.15.97:0/1964106389 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcffc088860 con 0x7fd0000b2190 2023-07-06T03:58:06.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.278+0000 7fd00559a700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfe406ace0 0x7fcfe406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:06.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.279+0000 7fd00559a700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfe406ace0 0x7fcfe406d190 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd0000d0220 tx=0x7fcff0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:06.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.283+0000 7fcff67fc700 1 -- 172.21.15.97:0/1964106389 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd0000c6c30 con 0x7fd0000b2190 2023-07-06T03:58:06.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.474+0000 7fd007fff700 1 -- 172.21.15.97:0/1964106389 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0000cd7e0 con 0x7fcfe406ace0 2023-07-06T03:58:06.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.477+0000 7fcff67fc700 1 -- 172.21.15.97:0/1964106389 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72724 (secure 0 0 0) 0x7fcff00041e0 con 0x7fcfe406ace0 2023-07-06T03:58:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:58:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 -- 172.21.15.97:0/1964106389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfe406ace0 msgr2=0x7fcfe406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfe406ace0 0x7fcfe406d190 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd0000d0220 tx=0x7fcff0008030 comp rx=0 tx=0).stop 2023-07-06T03:58:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 -- 172.21.15.97:0/1964106389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0000b2190 msgr2=0x7fd0001532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0000b2190 0x7fd0001532b0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c780 tx=0x7fcffc00ca90 comp rx=0 tx=0).stop 2023-07-06T03:58:06.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 -- 172.21.15.97:0/1964106389 shutdown_connections 2023-07-06T03:58:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7fcfe406ace0 0x7fcfe406d190 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd0000cfe30 0x7fd000158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0000b2190 0x7fd0001532b0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 --2- 172.21.15.97:0/1964106389 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7fd0000b8470 0x7fd000157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 -- 172.21.15.97:0/1964106389 >> 172.21.15.97:0/1964106389 conn(0x7fd000071740 msgr2=0x7fd0000cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 -- 172.21.15.97:0/1964106389 shutdown_connections 2023-07-06T03:58:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:06.479+0000 7fd007fff700 1 -- 172.21.15.97:0/1964106389 wait complete. 2023-07-06T03:58:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:58:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":529,"stamp":"2023-07-06T03:58:05.053728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296164,"kb_used_data":15012,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754678556,"statfs":{"total":773094113280,"available":772790841344,"internally_reserved":0,"allocated":15372288,"data_stored":7711269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.094388"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":227,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:54.961258+0000","last_change":"2023-07-06T03:45:05.215799+0000","last_active":"2023-07-06T03:57:54.961258+0000","last_peered":"2023-07-06T03:57:54.961258+0000","last_clean":"2023-07-06T03:57:54.961258+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:57:54.961258+0000","last_undegraded":"2023-07-06T03:57:54.961258+0000","last_fullsized":"2023-07-06T03:57:54.961258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:57:34.473968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":237,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:56.432522+0000","last_change":"2023-07-06T03:45:05.215245+0000","last_active":"2023-07-06T03:57:56.432522+0000","last_peered":"2023-07-06T03:57:56.432522+0000","last_clean":"2023-07-06T03:57:56.432522+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:57:56.432522+0000","last_undegraded":"2023-07-06T03:57:56.432522+0000","last_fullsized":"2023-07-06T03:57:56.432522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:37:46.745258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":243,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:54.964259+0000","last_change":"2023-07-06T03:45:05.133855+0000","last_active":"2023-07-06T03:57:54.964259+0000","last_peered":"2023-07-06T03:57:54.964259+0000","last_clean":"2023-07-06T03:57:54.964259+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:57:54.964259+0000","last_undegraded":"2023-07-06T03:57:54.964259+0000","last_fullsized":"2023-07-06T03:57:54.964259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:50:57.912150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":237,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:56.431570+0000","last_change":"2023-07-06T03:45:05.130570+0000","last_active":"2023-07-06T03:57:56.431570+0000","last_peered":"2023-07-06T03:57:56.431570+0000","last_clean":"2023-07-06T03:57:56.431570+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:57:56.431570+0000","last_undegraded":"2023-07-06T03:57:56.431570+0000","last_fullsized":"2023-07-06T03:57:56.431570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:06:48.322304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":377,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:55.226464+0000","last_change":"2023-07-06T03:44:58.099210+0000","last_active":"2023-07-06T03:57:55.226464+0000","last_peered":"2023-07-06T03:57:55.226464+0000","last_clean":"2023-07-06T03:57:55.226464+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:57:55.226464+0000","last_undegraded":"2023-07-06T03:57:55.226464+0000","last_fullsized":"2023-07-06T03:57:55.226464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:44:57.078482+0000","last_clean_scrub_stamp":"2023-07-06T03:44:57.078482+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:19:12.557397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":237,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:56.431757+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:57:56.431757+0000","last_peered":"2023-07-06T03:57:56.431757+0000","last_clean":"2023-07-06T03:57:56.431757+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:57:56.431757+0000","last_undegraded":"2023-07-06T03:57:56.431757+0000","last_fullsized":"2023-07-06T03:57:56.431757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:08:58.798529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":237,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:56.432161+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:57:56.432161+0000","last_peered":"2023-07-06T03:57:56.432161+0000","last_clean":"2023-07-06T03:57:56.432161+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:57:56.432161+0000","last_undegraded":"2023-07-06T03:57:56.432161+0000","last_fullsized":"2023-07-06T03:57:56.432161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T14:00:23.294281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":230,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:56.728681+0000","last_change":"2023-07-06T03:45:05.211192+0000","last_active":"2023-07-06T03:57:56.728681+0000","last_peered":"2023-07-06T03:57:56.728681+0000","last_clean":"2023-07-06T03:57:56.728681+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:57:56.728681+0000","last_undegraded":"2023-07-06T03:57:56.728681+0000","last_fullsized":"2023-07-06T03:57:56.728681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:25:28.568562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":236,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:56.433309+0000","last_change":"2023-07-06T03:45:05.211209+0000","last_active":"2023-07-06T03:57:56.433309+0000","last_peered":"2023-07-06T03:57:56.433309+0000","last_clean":"2023-07-06T03:57:56.433309+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:57:56.433309+0000","last_undegraded":"2023-07-06T03:57:56.433309+0000","last_fullsized":"2023-07-06T03:57:56.433309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:21:10.366628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575008,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.51000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.98499999999999999,"5min":1.024,"15min":4.0570000000000004},"last":1},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.67200000000000004,"15min":0.72099999999999997},"min":{"1min":0.54700000000000004,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":3.8660000000000001},"last":0.86399999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.58299999999999996,"5min":0.45400000000000001,"15min":0.34000000000000002},"max":{"1min":0.96299999999999997,"5min":1.091,"15min":2.6219999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.67800000000000005,"15min":0.76700000000000002},"min":{"1min":0.52700000000000002,"5min":0.40000000000000002,"15min":0.38800000000000001},"max":{"1min":0.94399999999999995,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.90000000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.68899999999999995,"15min":0.72999999999999998},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.35199999999999998},"max":{"1min":0.92000000000000004,"5min":1.0209999999999999,"15min":3.77},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.65000000000000002,"5min":0.47999999999999998,"15min":0.45000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":1.0289999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71899999999999997,"15min":0.71099999999999997},"min":{"1min":0.60599999999999998,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98499999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.752,"15min":0.753},"min":{"1min":0.63700000000000001,"5min":0.54500000000000004,"15min":0.43099999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.0209999999999999},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65500000000000003,"15min":0.63100000000000001},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.94699999999999995},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":0.94399999999999995,"15min":1.4590000000000001},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.72199999999999998,"15min":0.69999999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.38},"max":{"1min":1.0329999999999999,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70499999999999996,"15min":0.68899999999999995},"min":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.40000000000000002},"max":{"1min":0.997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.94599999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.74299999999999999},"min":{"1min":0.53600000000000003,"5min":0.46400000000000002,"15min":0.438},"max":{"1min":1.052,"5min":1.052,"15min":1.5349999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.502,"5min":0.49199999999999999,"15min":0.42299999999999999},"max":{"1min":1.0169999999999999,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.88400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.74299999999999999,"15min":0.69899999999999995},"min":{"1min":0.55300000000000005,"5min":0.32800000000000001,"15min":0.29399999999999998},"max":{"1min":0.95899999999999996,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.749,"15min":0.746},"min":{"1min":0.53600000000000003,"5min":0.32400000000000001,"15min":0.26900000000000002},"max":{"1min":0.91000000000000003,"5min":3.3119999999999998,"15min":3.3399999999999999},"last":0.86699999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.51700000000000002,"5min":0.372,"15min":0.34599999999999997},"max":{"1min":0.83399999999999996,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.70099999999999996},"min":{"1min":0.52300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96499999999999997,"5min":5.96,"15min":5.96},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77600000000000002,"15min":0.71299999999999997},"min":{"1min":0.56899999999999995,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.93899999999999995,"5min":8.3740000000000006,"15min":8.3740000000000006},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.77800000000000002,"15min":0.74199999999999999},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":5.8449999999999998,"15min":5.8449999999999998},"last":0.78600000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":0.96599999999999997,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.80800000000000005,"15min":0.75},"min":{"1min":0.58099999999999996,"5min":0.44500000000000001,"15min":0.373},"max":{"1min":0.874,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.82399999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.33200000000000002},"max":{"1min":0.83699999999999997,"5min":5.7759999999999998,"15min":5.7759999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.46000000000000002,"5min":0.42399999999999999,"15min":0.39200000000000002},"max":{"1min":0.93400000000000005,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.74199999999999999,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.38600000000000001},"max":{"1min":0.86799999999999999,"5min":8.4870000000000001,"15min":8.4870000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":0.86499999999999999,"5min":5.923,"15min":5.923},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76100000000000001,"15min":0.77600000000000002},"min":{"1min":0.53900000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90300000000000002,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.81599999999999995,"15min":0.80000000000000004},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93700000000000006,"5min":8.4429999999999996,"15min":8.4429999999999996},"last":0.93899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2312,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2367488,"data_stored":1406812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41599999999999998,"15min":0.46999999999999997},"min":{"1min":0.30199999999999999,"5min":0.26200000000000001,"15min":0.23699999999999999},"max":{"1min":0.53100000000000003,"5min":0.69499999999999995,"15min":2.8879999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.53200000000000003},"min":{"1min":0.40500000000000003,"5min":0.34999999999999998,"15min":0.313},"max":{"1min":0.70499999999999996,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.64800000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.64300000000000002},"min":{"1min":0.34599999999999997,"5min":0.28199999999999997,"15min":0.26500000000000001},"max":{"1min":0.58699999999999997,"5min":3.3690000000000002,"15min":13.055999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.58099999999999996},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.66600000000000004,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.504,"15min":0.55500000000000005},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.621},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48699999999999999,"15min":0.51100000000000001},"min":{"1min":0.35999999999999999,"5min":0.318,"15min":0.29499999999999998},"max":{"1min":0.80100000000000005,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69899999999999995,"15min":0.90900000000000003},"min":{"1min":0.61199999999999999,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.444,"15min":12.989000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.747,"15min":0.78900000000000003},"min":{"1min":0.627,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":1.1020000000000001,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.70099999999999996,"15min":0.74099999999999999},"min":{"1min":0.59299999999999997,"5min":0.44700000000000001,"15min":0.432},"max":{"1min":1.0429999999999999,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.90400000000000003},"min":{"1min":0.50600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.01,"5min":1.494,"15min":13.023999999999999},"last":1.0349999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.65600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99399999999999999,"5min":2.9670000000000001,"15min":2.9670000000000001},"last":1.006},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.754,"15min":0.95399999999999996},"min":{"1min":0.70599999999999996,"5min":0.47899999999999998,"15min":0.47799999999999998},"max":{"1min":1.012,"5min":2.5409999999999999,"15min":12.941000000000001},"last":0.98999999999999999}]},{"osd":7,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.74099999999999999,"15min":0.78200000000000003},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.44700000000000001},"max":{"1min":1.1799999999999999,"5min":2.431,"15min":2.806},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.65500000000000003,"5min":0.50600000000000001,"15min":0.48399999999999999},"max":{"1min":1.1439999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.90200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.60699999999999998,"5min":0.70699999999999996,"15min":0.753},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.436,"15min":0.44},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.63,"5min":0.63,"15min":0.81200000000000006},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.73699999999999999},"last":0.505},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.48899999999999999},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.314},"max":{"1min":0.65700000000000003,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47299999999999998,"15min":0.48899999999999999},"min":{"1min":0.33200000000000002,"5min":0.32200000000000001,"15min":0.29899999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.81200000000000006},"last":0.436},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.55500000000000005},"min":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.38800000000000001},"max":{"1min":0.67300000000000004,"5min":0.77600000000000002,"15min":0.84399999999999997},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.72299999999999998},"min":{"1min":0.55800000000000005,"5min":0.48399999999999999,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.171},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52500000000000002,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":1.0109999999999999,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.52000000000000002,"5min":0.48099999999999998,"15min":0.45600000000000002},"max":{"1min":0.94899999999999995,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46999999999999997},"max":{"1min":1.032,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.54600000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.093,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46200000000000002},"max":{"1min":0.94999999999999996,"5min":1.117,"15min":1.546},"last":0.81000000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77600000000000002,"15min":0.79000000000000004},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":1.0800000000000001,"5min":2.778,"15min":2.778},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.76600000000000001,"15min":0.89700000000000002},"min":{"1min":0.61499999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0680000000000001,"5min":3.7069999999999999,"15min":9.5960000000000001},"last":0.98799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40972,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330868,"statfs":{"total":96636764160,"available":96594808832,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47399999999999998,"15min":0.44400000000000001},"min":{"1min":0.33300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65100000000000002,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.501},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45800000000000002,"15min":0.47599999999999998},"min":{"1min":0.28699999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.628,"5min":1.093,"15min":2.4630000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.67900000000000005,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77800000000000002,"5min":3.657,"15min":3.657},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.56599999999999995},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.33900000000000002},"max":{"1min":0.68000000000000005,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76300000000000001,"5min":2.1629999999999998,"15min":2.7679999999999998},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.91400000000000003,"15min":0.79700000000000004},"min":{"1min":0.504,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.99299999999999999,"5min":12.007,"15min":12.007},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.48399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0529999999999999,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.79200000000000004,"15min":0.755},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":0.90400000000000003,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.79900000000000004},"min":{"1min":0.55900000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0660000000000001,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.91400000000000003,"15min":0.80100000000000005},"min":{"1min":0.53500000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.022,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.90500000000000003,"15min":0.81399999999999995},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":1.0069999999999999,"5min":13.084,"15min":13.084},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81699999999999995,"15min":0.81599999999999995},"min":{"1min":0.56799999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0700000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.81499999999999995},"min":{"1min":0.61799999999999999,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.079,"5min":4.484,"15min":4.484},"last":0.78500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575008,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.435,"15min":0.50600000000000001},"min":{"1min":0.34799999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.72399999999999998,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.441},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.89600000000000002,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.622}]},{"osd":1,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.44,"15min":0.69399999999999995},"min":{"1min":0.308,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68000000000000005,"5min":1.3620000000000001,"15min":15.911},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.52600000000000002,"15min":0.77100000000000002},"min":{"1min":0.432,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":1.347,"15min":16.024000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.35099999999999998,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":0.67900000000000005,"5min":3.25,"15min":3.25},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.53200000000000003,"15min":0.77300000000000002},"min":{"1min":0.39400000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85399999999999998,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.71899999999999997,"15min":0.73899999999999999},"min":{"1min":0.627,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0329999999999999,"5min":1.782,"15min":1.782},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.73999999999999999,"15min":0.81799999999999995},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.36699999999999999},"max":{"1min":1.0229999999999999,"5min":1.724,"15min":5.7300000000000004},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.746,"15min":0.82499999999999996},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.39700000000000002},"max":{"1min":1.077,"5min":1.411,"15min":5.077},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.77200000000000002,"15min":0.86199999999999999},"min":{"1min":0.69099999999999995,"5min":0.52200000000000002,"15min":0.44800000000000001},"max":{"1min":1.0609999999999999,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.87}]},{"osd":6,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.749,"15min":0.77100000000000002},"min":{"1min":0.58899999999999997,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.089,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.746,"15min":0.82799999999999996},"min":{"1min":0.63500000000000001,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.1319999999999999,"5min":1.4490000000000001,"15min":5.157},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.75800000000000001,"15min":0.78700000000000003},"min":{"1min":0.66400000000000003,"5min":0.51700000000000002,"15min":0.47399999999999998},"max":{"1min":1.073,"5min":1.863,"15min":1.863},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.75800000000000001,"15min":0.84099999999999997},"min":{"1min":0.64100000000000001,"5min":0.51100000000000001,"15min":0.49399999999999999},"max":{"1min":1.179,"5min":1.492,"15min":5.1189999999999998},"last":0.95999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575007,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.66200000000000003,"15min":0.65500000000000003},"min":{"1min":0.55100000000000005,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.90700000000000003,"5min":0.999,"15min":1.784},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.56000000000000005,"5min":0.42099999999999999,"15min":0.38},"max":{"1min":0.878,"5min":0.95899999999999996,"15min":1.0509999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.39500000000000002,"15min":0.38},"max":{"1min":0.89700000000000002,"5min":1.0900000000000001,"15min":6.258},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.41399999999999998,"15min":0.35899999999999999},"max":{"1min":0.98399999999999999,"5min":1.0700000000000001,"15min":1.806},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69499999999999995,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.40600000000000003,"15min":0.29199999999999998},"max":{"1min":1.01,"5min":1.012,"15min":6.4740000000000002},"last":0.621},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.82999999999999996},"min":{"1min":0.66000000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":0.98599999999999999,"5min":0.99399999999999999,"15min":6.4859999999999998},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68799999999999994},"min":{"1min":0.58099999999999996,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.0309999999999999,"15min":1.25},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.49099999999999999,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.413},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.627},"min":{"1min":0.44,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.3280000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.99299999999999999},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.76500000000000001},"min":{"1min":0.60199999999999998,"5min":0.5,"15min":0.47299999999999998},"max":{"1min":0.93600000000000005,"5min":0.96299999999999997,"15min":1.6839999999999999},"last":0.77900000000000003}]},{"osd":7,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.48099999999999998,"5min":0.442,"15min":0.41299999999999998},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":1.036,"15min":1.21},"last":0.75800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37452,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334388,"statfs":{"total":96636764160,"available":96598413312,"internally_reserved":0,"allocated":2363392,"data_stored":1405793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.436,"5min":0.40200000000000002,"15min":0.32100000000000001},"max":{"1min":0.83299999999999996,"5min":0.92900000000000005,"15min":4.5010000000000003},"last":0.5},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66200000000000003,"15min":0.746},"min":{"1min":0.57899999999999996,"5min":0.33100000000000002,"15min":0.29999999999999999},"max":{"1min":0.83999999999999997,"5min":0.99299999999999999,"15min":6.4059999999999997},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.307},"max":{"1min":0.92500000000000004,"5min":1.121,"15min":1.121},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.41399999999999998},"max":{"1min":0.93600000000000005,"5min":0.97299999999999998,"15min":1.546},"last":0.59899999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.78700000000000003,"15min":0.72899999999999998},"min":{"1min":0.54100000000000004,"5min":0.49399999999999999,"15min":0.376},"max":{"1min":0.91000000000000003,"5min":8.1120000000000001,"15min":8.1120000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67900000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":1.069,"15min":1.069},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.78900000000000003,"15min":0.73799999999999999},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.42199999999999999},"max":{"1min":0.88400000000000001,"5min":8.1829999999999998,"15min":8.1829999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71899999999999997,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.46700000000000003,"15min":0.39600000000000002},"max":{"1min":0.93200000000000005,"5min":1.3140000000000001,"15min":4.484},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.995,"5min":1.0229999999999999,"15min":1.6240000000000001},"last":0.627},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.379},"max":{"1min":0.95899999999999996,"5min":0.98699999999999999,"15min":2.718},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68500000000000005,"15min":0.68400000000000005},"min":{"1min":0.50600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.97399999999999998,"15min":2.5249999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.437,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.028,"5min":1.036,"15min":1.669},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jul 6 03:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73599999999999999,"15min":0.748},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.438},"max":{"1min":1.006,"5min":1.0529999999999999,"15min":2.0459999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74099999999999999,"15min":0.76200000000000001},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92700000000000005,"5min":1.034,"15min":2.2919999999999998},"last":0.73799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:58:06.503 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-07-06T03:45:04.113606+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=45, tm_sec=4, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T03:58:06.503 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-07-06T03:45:04.113606+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=45, tm_sec=4, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T03:58:06.504 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-07-06T03:45:04.113606+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=45, tm_sec=4, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T03:58:06.504 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-07-06T03:45:04.113606+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=45, tm_sec=4, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T03:58:06.505 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-07-06T03:44:57.078482+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=44, tm_sec=57, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T03:58:06.505 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-07-06T03:45:04.113606+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=45, tm_sec=4, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T03:58:06.505 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-07-06T03:45:04.113606+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=45, tm_sec=4, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T03:58:06.505 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-07-06T03:45:04.113606+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=45, tm_sec=4, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T03:58:06.506 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-07-06T03:45:04.113606+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=45, tm_sec=4, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T03:58:06.506 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-07-06T03:58:07.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:07.855 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:26.506 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:58:26.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.710+0000 7f30643ff700 1 Processor -- start 2023-07-06T03:58:26.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.710+0000 7f30643ff700 1 -- start start 2023-07-06T03:58:26.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.711+0000 7f30643ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f305c070e60 0x7f305c06d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:26.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.711+0000 7f30643ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 0x7f305c06a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:26.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.711+0000 7f30643ff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c0aecc0 con 0x7f305c0ae8f0 2023-07-06T03:58:26.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.711+0000 7f30643ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305c06a970 con 0x7f305c070e60 2023-07-06T03:58:26.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.711+0000 7f30643ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c071320 con 0x7f305c06a010 2023-07-06T03:58:26.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f306199a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f305c070e60 0x7f305c06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:26.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f306299c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 0x7f305c06a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:26.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f306299c700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 0x7f305c06a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3301/0 says I am v2:172.21.15.97:57570/0 (socket says 172.21.15.97:57570) 2023-07-06T03:58:26.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f306199a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f305c070e60 0x7f305c06d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.97:42726/0 (socket says 172.21.15.97:42726) 2023-07-06T03:58:26.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f306299c700 1 -- 172.21.15.97:0/2903541780 learned_addr learned my addr 172.21.15.97:0/2903541780 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:26.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f306219b700 1 --1- 172.21.15.97:0/2903541780 >> v1:172.21.15.146:6789/0 conn(0x7f305c0ae8f0 0x7f305c0ac880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.97:45124/0 (socket says 172.21.15.97:45124) 2023-07-06T03:58:26.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f3061199700 1 -- 172.21.15.97:0/2903541780 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556786193 0 0) 0x7f3058001110 con 0x7f305c0ae8f0 2023-07-06T03:58:26.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f3061199700 1 -- 172.21.15.97:0/2903541780 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30480035b0 con 0x7f305c0ae8f0 2023-07-06T03:58:26.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f306199a700 1 -- 172.21.15.97:0/2903541780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 msgr2=0x7f305c06a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:26.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f306199a700 1 --2- 172.21.15.97:0/2903541780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 0x7f305c06a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:26.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f306199a700 1 -- 172.21.15.97:0/2903541780 >> v1:172.21.15.146:6789/0 conn(0x7f305c0ae8f0 legacy=0x7f305c0ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:26.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f306199a700 1 -- 172.21.15.97:0/2903541780 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c071600 con 0x7f305c070e60 2023-07-06T03:58:26.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.712+0000 7f306299c700 1 --2- 172.21.15.97:0/2903541780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 0x7f305c06a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T03:58:26.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.713+0000 7f306199a700 1 --2- 172.21.15.97:0/2903541780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305c070e60 0x7f305c06d730 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f304c009840 tx=0x7f304c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e09de220b85db76e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:26.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.713+0000 7f3061199700 1 -- 172.21.15.97:0/2903541780 <== mon.1 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c011830 con 0x7f305c070e60 2023-07-06T03:58:26.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.713+0000 7f3061199700 1 -- 172.21.15.97:0/2903541780 <== mon.1 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304c011990 con 0x7f305c070e60 2023-07-06T03:58:26.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.713+0000 7f3061199700 1 -- 172.21.15.97:0/2903541780 <== mon.1 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c011d00 con 0x7f305c070e60 2023-07-06T03:58:26.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.713+0000 7f30643ff700 1 -- 172.21.15.97:0/2903541780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305c070e60 msgr2=0x7f305c06d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:26.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.713+0000 7f30643ff700 1 --2- 172.21.15.97:0/2903541780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305c070e60 0x7f305c06d730 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f304c009840 tx=0x7f304c009b50 comp rx=0 tx=0).stop 2023-07-06T03:58:26.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.714+0000 7f30643ff700 1 -- 172.21.15.97:0/2903541780 shutdown_connections 2023-07-06T03:58:26.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.714+0000 7f30643ff700 1 --2- 172.21.15.97:0/2903541780 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 0x7f305c06a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:26.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.714+0000 7f30643ff700 1 --2- 172.21.15.97:0/2903541780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305c070e60 0x7f305c06d730 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:26.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.714+0000 7f30643ff700 1 -- 172.21.15.97:0/2903541780 >> 172.21.15.97:0/2903541780 conn(0x7f305c06bdb0 msgr2=0x7f305c0b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:26.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.714+0000 7f30643ff700 1 -- 172.21.15.97:0/2903541780 shutdown_connections 2023-07-06T03:58:26.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.714+0000 7f30643ff700 1 -- 172.21.15.97:0/2903541780 wait complete. 2023-07-06T03:58:26.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.715+0000 7f30643ff700 1 Processor -- start 2023-07-06T03:58:26.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.716+0000 7f30643ff700 1 -- start start 2023-07-06T03:58:26.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.716+0000 7f30643ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 0x7f305c15ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:26.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.716+0000 7f30643ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305c070e60 0x7f305c15d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:26.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.716+0000 7f30643ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305c0ae8f0 0x7f305c15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:26.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f30643ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c15db80 con 0x7f305c0ae8f0 2023-07-06T03:58:26.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f30643ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c15de60 con 0x7f305c070e60 2023-07-06T03:58:26.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f30643ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c15e140 con 0x7f305c06a010 2023-07-06T03:58:26.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f306299c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305c0ae8f0 0x7f305c15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:26.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f306299c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305c0ae8f0 0x7f305c15a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33520/0 (socket says 172.21.15.97:33520) 2023-07-06T03:58:26.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f306299c700 1 -- 172.21.15.97:0/4068199540 learned_addr learned my addr 172.21.15.97:0/4068199540 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:26.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f306199a700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305c070e60 0x7f305c15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:26.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f306219b700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 0x7f305c15ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:26.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f306299c700 1 -- 172.21.15.97:0/4068199540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 msgr2=0x7f305c15ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:26.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f306299c700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 0x7f305c15ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:26.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f306299c700 1 -- 172.21.15.97:0/4068199540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305c070e60 msgr2=0x7f305c15d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:26.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f306299c700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305c070e60 0x7f305c15d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:26.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.717+0000 7f306299c700 1 -- 172.21.15.97:0/4068199540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3054009b30 con 0x7f305c0ae8f0 2023-07-06T03:58:26.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.718+0000 7f306299c700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305c0ae8f0 0x7f305c15a460 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f305400efc0 tx=0x7f305400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:26.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.718+0000 7f3052ffd700 1 -- 172.21.15.97:0/4068199540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054018070 con 0x7f305c0ae8f0 2023-07-06T03:58:26.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.718+0000 7f3052ffd700 1 -- 172.21.15.97:0/4068199540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305400f040 con 0x7f305c0ae8f0 2023-07-06T03:58:26.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.718+0000 7f30643ff700 1 -- 172.21.15.97:0/4068199540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c0094a0 con 0x7f305c0ae8f0 2023-07-06T03:58:26.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.718+0000 7f3052ffd700 1 -- 172.21.15.97:0/4068199540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054003c40 con 0x7f305c0ae8f0 2023-07-06T03:58:26.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.718+0000 7f30643ff700 1 -- 172.21.15.97:0/4068199540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c15e700 con 0x7f305c0ae8f0 2023-07-06T03:58:26.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.719+0000 7f30643ff700 1 -- 172.21.15.97:0/4068199540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f305c0af2c0 con 0x7f305c0ae8f0 2023-07-06T03:58:26.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.720+0000 7f3052ffd700 1 -- 172.21.15.97:0/4068199540 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f305400c880 con 0x7f305c0ae8f0 2023-07-06T03:58:26.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.721+0000 7f3052ffd700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f304006ace0 0x7f304006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:26.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.721+0000 7f3052ffd700 1 -- 172.21.15.97:0/4068199540 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3054089260 con 0x7f305c0ae8f0 2023-07-06T03:58:26.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.721+0000 7f306219b700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f304006ace0 0x7f304006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:26.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.721+0000 7f306219b700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f304006ace0 0x7f304006d190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3058007fd0 tx=0x7f3058007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:26.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.725+0000 7f3052ffd700 1 -- 172.21.15.97:0/4068199540 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30540580b0 con 0x7f305c0ae8f0 2023-07-06T03:58:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.918+0000 7f30643ff700 1 -- 172.21.15.97:0/4068199540 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f305c15dfd0 con 0x7f304006ace0 2023-07-06T03:58:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.921+0000 7f3052ffd700 1 -- 172.21.15.97:0/4068199540 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72868 (secure 0 0 0) 0x7f305c15dfd0 con 0x7f304006ace0 2023-07-06T03:58:26.924 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:58:26.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 -- 172.21.15.97:0/4068199540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f304006ace0 msgr2=0x7f304006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f304006ace0 0x7f304006d190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3058007fd0 tx=0x7f3058007dd0 comp rx=0 tx=0).stop 2023-07-06T03:58:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 -- 172.21.15.97:0/4068199540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305c0ae8f0 msgr2=0x7f305c15a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305c0ae8f0 0x7f305c15a460 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f305400efc0 tx=0x7f305400c3a0 comp rx=0 tx=0).stop 2023-07-06T03:58:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 -- 172.21.15.97:0/4068199540 shutdown_connections 2023-07-06T03:58:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f304006ace0 0x7f304006d190 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305c0ae8f0 0x7f305c15a460 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f305c06a010 0x7f305c15ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 --2- 172.21.15.97:0/4068199540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f305c070e60 0x7f305c15d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 -- 172.21.15.97:0/4068199540 >> 172.21.15.97:0/4068199540 conn(0x7f305c06bdb0 msgr2=0x7f305c0c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 -- 172.21.15.97:0/4068199540 shutdown_connections 2023-07-06T03:58:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:26.924+0000 7f30643ff700 1 -- 172.21.15.97:0/4068199540 wait complete. 2023-07-06T03:58:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:58:26.944 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":539,"stamp":"2023-07-06T03:58:25.058099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296164,"kb_used_data":15012,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754678556,"statfs":{"total":773094113280,"available":772790841344,"internally_reserved":0,"allocated":15372288,"data_stored":7711269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002817"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":236,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:07.232831+0000","last_change":"2023-07-06T03:58:07.232881+0000","last_active":"2023-07-06T03:58:07.232831+0000","last_peered":"2023-07-06T03:58:07.232831+0000","last_clean":"2023-07-06T03:58:07.232831+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:58:07.232831+0000","last_undegraded":"2023-07-06T03:58:07.232831+0000","last_fullsized":"2023-07-06T03:58:07.232831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:07.232727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:07.232727+0000","last_clean_scrub_stamp":"2023-07-06T03:58:07.232727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:26.509472+0000","scrub_duration":0.0021799810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":246,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:14.754449+0000","last_change":"2023-07-06T03:58:14.754500+0000","last_active":"2023-07-06T03:58:14.754449+0000","last_peered":"2023-07-06T03:58:14.754449+0000","last_clean":"2023-07-06T03:58:14.754449+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:58:14.754449+0000","last_undegraded":"2023-07-06T03:58:14.754449+0000","last_fullsized":"2023-07-06T03:58:14.754449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:14.754305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:14.754305+0000","last_clean_scrub_stamp":"2023-07-06T03:58:14.754305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:34:57.586149+0000","scrub_duration":0.003000544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":252,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:06.658345+0000","last_change":"2023-07-06T03:58:06.658393+0000","last_active":"2023-07-06T03:58:06.658345+0000","last_peered":"2023-07-06T03:58:06.658345+0000","last_clean":"2023-07-06T03:58:06.658345+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:58:06.658345+0000","last_undegraded":"2023-07-06T03:58:06.658345+0000","last_fullsized":"2023-07-06T03:58:06.658345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:06.658239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:06.658239+0000","last_clean_scrub_stamp":"2023-07-06T03:58:06.658239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:46:23.904800+0000","scrub_duration":0.0023019350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":246,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:16.734086+0000","last_change":"2023-07-06T03:58:16.734136+0000","last_active":"2023-07-06T03:58:16.734086+0000","last_peered":"2023-07-06T03:58:16.734086+0000","last_clean":"2023-07-06T03:58:16.734086+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:58:16.734086+0000","last_undegraded":"2023-07-06T03:58:16.734086+0000","last_fullsized":"2023-07-06T03:58:16.734086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:16.733998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:16.733998+0000","last_clean_scrub_stamp":"2023-07-06T03:58:16.733998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:37:30.141599+0000","scrub_duration":0.0021046099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"141'119","reported_seq":388,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:09.522615+0000","last_change":"2023-07-06T03:58:09.522678+0000","last_active":"2023-07-06T03:58:09.522615+0000","last_peered":"2023-07-06T03:58:09.522615+0000","last_clean":"2023-07-06T03:58:09.522615+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:58:09.522615+0000","last_undegraded":"2023-07-06T03:58:09.522615+0000","last_fullsized":"2023-07-06T03:58:09.522615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'119","last_scrub_stamp":"2023-07-06T03:58:09.522496+0000","last_deep_scrub":"141'119","last_deep_scrub_stamp":"2023-07-06T03:58:09.522496+0000","last_clean_scrub_stamp":"2023-07-06T03:58:09.522496+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:24:06.171566+0000","scrub_duration":0.00647001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":238,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:56.432327+0000","last_change":"2023-07-06T03:45:05.133737+0000","last_active":"2023-07-06T03:57:56.432327+0000","last_peered":"2023-07-06T03:57:56.432327+0000","last_clean":"2023-07-06T03:57:56.432327+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:57:56.432327+0000","last_undegraded":"2023-07-06T03:57:56.432327+0000","last_fullsized":"2023-07-06T03:57:56.432327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":238,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:57:56.432233+0000","last_change":"2023-07-06T03:45:05.133309+0000","last_active":"2023-07-06T03:57:56.432233+0000","last_peered":"2023-07-06T03:57:56.432233+0000","last_clean":"2023-07-06T03:57:56.432233+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:57:56.432233+0000","last_undegraded":"2023-07-06T03:57:56.432233+0000","last_fullsized":"2023-07-06T03:57:56.432233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:45:04.113606+0000","last_clean_scrub_stamp":"2023-07-06T03:45:04.113606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":240,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:04.911072+0000","last_change":"2023-07-06T03:58:04.911072+0000","last_active":"2023-07-06T03:58:04.911072+0000","last_peered":"2023-07-06T03:58:04.911072+0000","last_clean":"2023-07-06T03:58:04.911072+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:58:04.911072+0000","last_undegraded":"2023-07-06T03:58:04.911072+0000","last_fullsized":"2023-07-06T03:58:04.911072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:04.910934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:04.910934+0000","last_clean_scrub_stamp":"2023-07-06T03:58:04.910934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:58:05.106548+0000","scrub_duration":0.0024695099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":246,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:09.735702+0000","last_change":"2023-07-06T03:58:09.735752+0000","last_active":"2023-07-06T03:58:09.735702+0000","last_peered":"2023-07-06T03:58:09.735702+0000","last_clean":"2023-07-06T03:58:09.735702+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:58:09.735702+0000","last_undegraded":"2023-07-06T03:58:09.735702+0000","last_fullsized":"2023-07-06T03:58:09.735702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:09.735618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:09.735618+0000","last_clean_scrub_stamp":"2023-07-06T03:58:09.735618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:43:20.758220+0000","scrub_duration":0.0023122249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.69299999999999995},"min":{"1min":0.51000000000000001,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":0.98499999999999999,"5min":1.024,"15min":4.0570000000000004},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.67200000000000004,"15min":0.72099999999999997},"min":{"1min":0.54700000000000004,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":3.8660000000000001},"last":0.80700000000000005}]},{"osd":1,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.58299999999999996,"5min":0.45400000000000001,"15min":0.34000000000000002},"max":{"1min":0.96299999999999997,"5min":1.091,"15min":2.6219999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.67800000000000005,"15min":0.76700000000000002},"min":{"1min":0.52700000000000002,"5min":0.40000000000000002,"15min":0.38800000000000001},"max":{"1min":0.94399999999999995,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.68899999999999995,"15min":0.72999999999999998},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.35199999999999998},"max":{"1min":0.92000000000000004,"5min":1.0209999999999999,"15min":3.77},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.65000000000000002,"5min":0.47999999999999998,"15min":0.45000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.70399999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71899999999999997,"15min":0.71099999999999997},"min":{"1min":0.60599999999999998,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98499999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.752,"15min":0.753},"min":{"1min":0.63700000000000001,"5min":0.54500000000000004,"15min":0.43099999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.0209999999999999},"last":0.75800000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65500000000000003,"15min":0.63100000000000001},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.94699999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":0.94399999999999995,"15min":1.4590000000000001},"last":0.51600000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.72199999999999998,"15min":0.69999999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.38},"max":{"1min":1.0329999999999999,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70499999999999996,"15min":0.68899999999999995},"min":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.40000000000000002},"max":{"1min":0.997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.74299999999999999},"min":{"1min":0.53600000000000003,"5min":0.46400000000000002,"15min":0.438},"max":{"1min":1.052,"5min":1.052,"15min":1.5349999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.502,"5min":0.49199999999999999,"15min":0.42299999999999999},"max":{"1min":1.0169999999999999,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.78900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.74299999999999999,"15min":0.69899999999999995},"min":{"1min":0.55300000000000005,"5min":0.32800000000000001,"15min":0.29399999999999998},"max":{"1min":0.95899999999999996,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.749,"15min":0.746},"min":{"1min":0.53600000000000003,"5min":0.32400000000000001,"15min":0.26900000000000002},"max":{"1min":0.91000000000000003,"5min":3.3119999999999998,"15min":3.3399999999999999},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.51700000000000002,"5min":0.372,"15min":0.34599999999999997},"max":{"1min":0.83399999999999996,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.70099999999999996},"min":{"1min":0.52300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96499999999999997,"5min":5.96,"15min":5.96},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77600000000000002,"15min":0.71299999999999997},"min":{"1min":0.56899999999999995,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.93899999999999995,"5min":8.3740000000000006,"15min":8.3740000000000006},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.77800000000000002,"15min":0.74199999999999999},"min":{"1min":0.52600000000000002,"5min":0.45500000000000002,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":5.8449999999999998,"15min":5.8449999999999998},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":0.96599999999999997,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.80800000000000005,"15min":0.75},"min":{"1min":0.58099999999999996,"5min":0.44500000000000001,"15min":0.373},"max":{"1min":0.874,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.33200000000000002},"max":{"1min":0.83699999999999997,"5min":5.7759999999999998,"15min":5.7759999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.46000000000000002,"5min":0.42399999999999999,"15min":0.39200000000000002},"max":{"1min":0.93400000000000005,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.86899999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.74199999999999999,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.41599999999999998,"15min":0.38600000000000001},"max":{"1min":0.86799999999999999,"5min":8.4870000000000001,"15min":8.4870000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.36299999999999999},"max":{"1min":0.86499999999999999,"5min":5.923,"15min":5.923},"last":0.89600000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76100000000000001,"15min":0.77600000000000002},"min":{"1min":0.53900000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90300000000000002,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.81599999999999995,"15min":0.80000000000000004},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93700000000000006,"5min":8.4429999999999996,"15min":8.4429999999999996},"last":0.84399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2312,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2367488,"data_stored":1406812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41599999999999998,"15min":0.46999999999999997},"min":{"1min":0.30199999999999999,"5min":0.26200000000000001,"15min":0.23699999999999999},"max":{"1min":0.53100000000000003,"5min":0.69499999999999995,"15min":2.8879999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.53200000000000003},"min":{"1min":0.40500000000000003,"5min":0.34999999999999998,"15min":0.313},"max":{"1min":0.70499999999999996,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.64300000000000002},"min":{"1min":0.34599999999999997,"5min":0.28199999999999997,"15min":0.26500000000000001},"max":{"1min":0.58699999999999997,"5min":3.3690000000000002,"15min":13.055999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.58099999999999996},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.66600000000000004,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.504,"15min":0.55500000000000005},"min":{"1min":0.44,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":1.0029999999999999,"15min":2.9249999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48699999999999999,"15min":0.51100000000000001},"min":{"1min":0.35999999999999999,"5min":0.318,"15min":0.29499999999999998},"max":{"1min":0.80100000000000005,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.46700000000000003}]},{"osd":4,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69899999999999995,"15min":0.90900000000000003},"min":{"1min":0.61199999999999999,"5min":0.41799999999999998,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.444,"15min":12.989000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.747,"15min":0.78900000000000003},"min":{"1min":0.627,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":1.1020000000000001,"5min":2.2029999999999998,"15min":2.7629999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.70099999999999996,"15min":0.74099999999999999},"min":{"1min":0.59299999999999997,"5min":0.44700000000000001,"15min":0.432},"max":{"1min":1.0429999999999999,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.90400000000000003},"min":{"1min":0.50600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.01,"5min":1.494,"15min":13.023999999999999},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.65600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99399999999999999,"5min":2.9670000000000001,"15min":2.9670000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.754,"15min":0.95399999999999996},"min":{"1min":0.70599999999999996,"5min":0.47899999999999998,"15min":0.47799999999999998},"max":{"1min":1.012,"5min":2.5409999999999999,"15min":12.941000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.74099999999999999,"15min":0.78200000000000003},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.44700000000000001},"max":{"1min":1.1799999999999999,"5min":2.431,"15min":2.806},"last":0.625},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.65500000000000003,"5min":0.50600000000000001,"15min":0.48399999999999999},"max":{"1min":1.1439999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.72799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575013,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.60699999999999998,"5min":0.70699999999999996,"15min":0.753},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.436,"15min":0.44},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.63,"5min":0.63,"15min":0.81200000000000006},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.73699999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47099999999999997,"15min":0.48899999999999999},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.314},"max":{"1min":0.65700000000000003,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47299999999999998,"15min":0.48899999999999999},"min":{"1min":0.33200000000000002,"5min":0.32200000000000001,"15min":0.29899999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.81200000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.55500000000000005},"min":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.38800000000000001},"max":{"1min":0.67300000000000004,"5min":0.77600000000000002,"15min":0.84399999999999997},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.72299999999999998},"min":{"1min":0.55800000000000005,"5min":0.48399999999999999,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.171},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.52500000000000002,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":1.0109999999999999,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.52000000000000002,"5min":0.48099999999999998,"15min":0.45600000000000002},"max":{"1min":0.94899999999999995,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73099999999999998,"15min":0.73999999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46999999999999997},"max":{"1min":1.032,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73899999999999999,"15min":0.74399999999999999},"min":{"1min":0.54600000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.093,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46200000000000002},"max":{"1min":0.94999999999999996,"5min":1.117,"15min":1.546},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77600000000000002,"15min":0.79000000000000004},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.53200000000000003},"max":{"1min":1.0800000000000001,"5min":2.778,"15min":2.778},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.76600000000000001,"15min":0.89700000000000002},"min":{"1min":0.61499999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0680000000000001,"5min":3.7069999999999999,"15min":9.5960000000000001},"last":0.80500000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40972,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330868,"statfs":{"total":96636764160,"available":96594808832,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47399999999999998,"15min":0.44400000000000001},"min":{"1min":0.33300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65100000000000002,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45800000000000002,"15min":0.47599999999999998},"min":{"1min":0.28699999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.628,"5min":1.093,"15min":2.4630000000000001},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.317},"max":{"1min":0.67900000000000005,"5min":6.5960000000000001,"15min":6.5960000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77800000000000002,"5min":3.657,"15min":3.657},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.56599999999999995},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.33900000000000002},"max":{"1min":0.68000000000000005,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76300000000000001,"5min":2.1629999999999998,"15min":2.7679999999999998},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.91400000000000003,"15min":0.79700000000000004},"min":{"1min":0.504,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.99299999999999999,"5min":12.007,"15min":12.007},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.48399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0529999999999999,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.79200000000000004,"15min":0.755},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.45800000000000002},"max":{"1min":0.90400000000000003,"5min":4.8719999999999999,"15min":4.8719999999999999},"last":0.877},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.79900000000000004},"min":{"1min":0.55900000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0660000000000001,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.91400000000000003,"15min":0.80100000000000005},"min":{"1min":0.53500000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.022,"5min":11.696999999999999,"15min":11.696999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.90500000000000003,"15min":0.81399999999999995},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.50700000000000001},"max":{"1min":1.0069999999999999,"5min":13.084,"15min":13.084},"last":0.83199999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:57:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81699999999999995,"15min":0.81599999999999995},"min":{"1min":0.56799999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0700000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.81499999999999995},"min":{"1min":0.61799999999999999,"5min":0.501,"15min":0.47499999999999998},"max":{"1min":1.079,"5min":4.484,"15min":4.484},"last":0.81399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.435,"15min":0.50600000000000001},"min":{"1min":0.34799999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.72399999999999998,"5min":2.2810000000000001,"15min":5.6550000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.89600000000000002,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.65400000000000003}]},{"osd":1,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.44,"15min":0.69399999999999995},"min":{"1min":0.308,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68000000000000005,"5min":1.3620000000000001,"15min":15.911},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.52600000000000002,"15min":0.77100000000000002},"min":{"1min":0.432,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":1.347,"15min":16.024000000000001},"last":0.629}]},{"osd":2,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.35099999999999998,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":0.67900000000000005,"5min":3.25,"15min":3.25},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.53200000000000003,"15min":0.77300000000000002},"min":{"1min":0.39400000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85399999999999998,"5min":3.3820000000000001,"15min":16.068999999999999},"last":0.71099999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.71899999999999997,"15min":0.73899999999999999},"min":{"1min":0.627,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0329999999999999,"5min":1.782,"15min":1.782},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.73999999999999999,"15min":0.81799999999999995},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.36699999999999999},"max":{"1min":1.0229999999999999,"5min":1.724,"15min":5.7300000000000004},"last":1.0149999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.746,"15min":0.82499999999999996},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.39700000000000002},"max":{"1min":1.077,"5min":1.411,"15min":5.077},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.77200000000000002,"15min":0.86199999999999999},"min":{"1min":0.69099999999999995,"5min":0.52200000000000002,"15min":0.44800000000000001},"max":{"1min":1.0609999999999999,"5min":1.6100000000000001,"15min":5.6980000000000004},"last":0.93999999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.749,"15min":0.77100000000000002},"min":{"1min":0.58899999999999997,"5min":0.503,"15min":0.48599999999999999},"max":{"1min":1.089,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.746,"15min":0.82799999999999996},"min":{"1min":0.63500000000000001,"5min":0.52100000000000002,"15min":0.45600000000000002},"max":{"1min":1.1319999999999999,"5min":1.4490000000000001,"15min":5.157},"last":0.96299999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.75800000000000001,"15min":0.78700000000000003},"min":{"1min":0.66400000000000003,"5min":0.51700000000000002,"15min":0.47399999999999998},"max":{"1min":1.073,"5min":1.863,"15min":1.863},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.75800000000000001,"15min":0.84099999999999997},"min":{"1min":0.64100000000000001,"5min":0.51100000000000001,"15min":0.49399999999999999},"max":{"1min":1.179,"5min":1.492,"15min":5.1189999999999998},"last":0.877}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.66200000000000003,"15min":0.65500000000000003},"min":{"1min":0.55100000000000005,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.90700000000000003,"5min":0.999,"15min":1.784},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.56000000000000005,"5min":0.42099999999999999,"15min":0.38},"max":{"1min":0.878,"5min":0.95899999999999996,"15min":1.0509999999999999},"last":0.85499999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.39500000000000002,"15min":0.38},"max":{"1min":0.89700000000000002,"5min":1.0900000000000001,"15min":6.258},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.41399999999999998,"15min":0.35899999999999999},"max":{"1min":0.98399999999999999,"5min":1.0700000000000001,"15min":1.806},"last":0.877}]},{"osd":2,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69499999999999995,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.40600000000000003,"15min":0.29199999999999998},"max":{"1min":1.01,"5min":1.012,"15min":6.4740000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.82999999999999996},"min":{"1min":0.66000000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":0.98599999999999999,"5min":0.99399999999999999,"15min":6.4859999999999998},"last":0.89100000000000001}]},{"osd":3,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68799999999999994},"min":{"1min":0.58099999999999996,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.0309999999999999,"15min":1.25},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.49099999999999999,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.413},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.627},"min":{"1min":0.44,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.3280000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.99299999999999999},"last":0.57999999999999996}]},{"osd":6,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.33600000000000002,"15min":0.33500000000000002},"max":{"1min":0.82699999999999996,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.76500000000000001},"min":{"1min":0.60199999999999998,"5min":0.5,"15min":0.47299999999999998},"max":{"1min":0.93600000000000005,"5min":0.96299999999999997,"15min":1.6839999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.48099999999999998,"5min":0.442,"15min":0.41299999999999998},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.44900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.91900000000000004,"5min":1.036,"15min":1.21},"last":0.79800000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37452,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334388,"statfs":{"total":96636764160,"available":96598413312,"internally_reserved":0,"allocated":2363392,"data_stored":1405793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.32100000000000001},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":4.5010000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66300000000000003,"15min":0.72699999999999998},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.29999999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":6.4059999999999997},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.69899999999999995,"15min":0.67500000000000004},"min":{"1min":0.51000000000000001,"5min":0.44700000000000001,"15min":0.307},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.97299999999999998,"15min":1.546},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.73099999999999998,"15min":0.73499999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.376},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":8.1120000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.45400000000000001,"5min":0.40400000000000003,"15min":0.39100000000000001},"max":{"1min":0.91800000000000004,"5min":1.069,"15min":1.069},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.73799999999999999},"min":{"1min":0.52100000000000002,"5min":0.50800000000000001,"15min":0.42199999999999999},"max":{"1min":0.97399999999999998,"5min":1.0700000000000001,"15min":8.1829999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.60999999999999999,"5min":0.495,"15min":0.39600000000000002},"max":{"1min":0.879,"5min":0.98099999999999998,"15min":4.484},"last":0.751}]},{"osd":4,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.624},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.90300000000000002,"5min":1.0229999999999999,"15min":1.6240000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68300000000000005,"15min":0.68200000000000005},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":0.98699999999999999,"15min":2.718},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.46400000000000002,"5min":0.44500000000000001,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":2.5249999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69099999999999995,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.669},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73499999999999999,"15min":0.745},"min":{"1min":0.51500000000000001,"5min":0.46300000000000002,"15min":0.438},"max":{"1min":0.92600000000000005,"5min":1.0529999999999999,"15min":2.0459999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.57499999999999996,"5min":0.47199999999999998,"15min":0.44900000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":2.2919999999999998},"last":0.71399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:58:26.946 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-07-06T03:45:04.113606+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=45, tm_sec=4, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T03:58:26.947 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-07-06T03:45:04.113606+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=45, tm_sec=4, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T03:58:26.947 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-07-06T03:58:38.227 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:38.232 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:46.948 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T03:58:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.150+0000 7f5759fff700 1 Processor -- start 2023-07-06T03:58:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.150+0000 7f5759fff700 1 -- start start 2023-07-06T03:58:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f5759fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57540b8470 0x7f57540cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f5759fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f57540cfe30 0x7f57540d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f5759fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57540d12f0 con 0x7f57540b2190 2023-07-06T03:58:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f5759fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57540b8840 con 0x7f57540b8470 2023-07-06T03:58:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f5759fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57540cf390 con 0x7f57540cfe30 2023-07-06T03:58:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f57537fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57540b8470 0x7f57540cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:47.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f57537fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57540b8470 0x7f57540cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36722/0 (socket says 172.21.15.97:36722) 2023-07-06T03:58:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f57537fe700 1 -- 172.21.15.97:0/2703857775 learned_addr learned my addr 172.21.15.97:0/2703857775 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f5753fff700 1 --2- 172.21.15.97:0/2703857775 >> v2:172.21.15.146:3300/0 conn(0x7f57540cfe30 0x7f57540d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f57537fe700 1 -- 172.21.15.97:0/2703857775 >> v2:172.21.15.146:3300/0 conn(0x7f57540cfe30 msgr2=0x7f57540d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f57537fe700 1 --2- 172.21.15.97:0/2703857775 >> v2:172.21.15.146:3300/0 conn(0x7f57540cfe30 0x7f57540d2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f57537fe700 1 -- 172.21.15.97:0/2703857775 >> v1:172.21.15.146:6789/0 conn(0x7f57540b2190 legacy=0x7f57540cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.152+0000 7f57537fe700 1 -- 172.21.15.97:0/2703857775 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57540b8ca0 con 0x7f57540b8470 2023-07-06T03:58:47.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.153+0000 7f57537fe700 1 --2- 172.21.15.97:0/2703857775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57540b8470 0x7f57540cee50 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f57540b4d00 tx=0x7f5744009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f7131b1235c3cb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:47.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.153+0000 7f57527fc700 1 -- 172.21.15.97:0/2703857775 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744004230 con 0x7f57540b8470 2023-07-06T03:58:47.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.153+0000 7f57527fc700 1 -- 172.21.15.97:0/2703857775 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5744004390 con 0x7f57540b8470 2023-07-06T03:58:47.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.153+0000 7f57527fc700 1 -- 172.21.15.97:0/2703857775 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57440036a0 con 0x7f57540b8470 2023-07-06T03:58:47.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.153+0000 7f5759fff700 1 -- 172.21.15.97:0/2703857775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57540b8470 msgr2=0x7f57540cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:47.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.153+0000 7f5759fff700 1 --2- 172.21.15.97:0/2703857775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57540b8470 0x7f57540cee50 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f57540b4d00 tx=0x7f5744009bd0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.153+0000 7f5759fff700 1 -- 172.21.15.97:0/2703857775 shutdown_connections 2023-07-06T03:58:47.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.153+0000 7f5759fff700 1 --2- 172.21.15.97:0/2703857775 >> v2:172.21.15.146:3300/0 conn(0x7f57540cfe30 0x7f57540d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.153+0000 7f5759fff700 1 --2- 172.21.15.97:0/2703857775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57540b8470 0x7f57540cee50 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.153+0000 7f5759fff700 1 -- 172.21.15.97:0/2703857775 >> 172.21.15.97:0/2703857775 conn(0x7f5754071740 msgr2=0x7f5754071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:47.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.154+0000 7f5759fff700 1 -- 172.21.15.97:0/2703857775 shutdown_connections 2023-07-06T03:58:47.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.154+0000 7f5759fff700 1 -- 172.21.15.97:0/2703857775 wait complete. 2023-07-06T03:58:47.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.155+0000 7f5759fff700 1 Processor -- start 2023-07-06T03:58:47.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.155+0000 7f5759fff700 1 -- start start 2023-07-06T03:58:47.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.156+0000 7f5759fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57540b2190 0x7f57540cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.156+0000 7f5759fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57540b8470 0x7f57541e4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.156+0000 7f5759fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57540cfe30 0x7f57540cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.156+0000 7f5759fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5754148350 con 0x7f57540b8470 2023-07-06T03:58:47.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.156+0000 7f5759fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57540c69c0 con 0x7f57540cfe30 2023-07-06T03:58:47.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.156+0000 7f5759fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5754147d00 con 0x7f57540b2190 2023-07-06T03:58:47.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5752ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57540b8470 0x7f57541e4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:47.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5752ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57540b8470 0x7f57541e4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36730/0 (socket says 172.21.15.97:36730) 2023-07-06T03:58:47.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5752ffd700 1 -- 172.21.15.97:0/3634331396 learned_addr learned my addr 172.21.15.97:0/3634331396 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:47.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f57537fe700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57540b2190 0x7f57540cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:47.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5753fff700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57540cfe30 0x7f57540cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:47.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5752ffd700 1 -- 172.21.15.97:0/3634331396 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57540b2190 msgr2=0x7f57540cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:47.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5752ffd700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57540b2190 0x7f57540cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5752ffd700 1 -- 172.21.15.97:0/3634331396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57540cfe30 msgr2=0x7f57540cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:47.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5752ffd700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57540cfe30 0x7f57540cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5752ffd700 1 -- 172.21.15.97:0/3634331396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5744009550 con 0x7f57540b8470 2023-07-06T03:58:47.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5753fff700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57540cfe30 0x7f57540cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:58:47.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5752ffd700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57540b8470 0x7f57541e4dc0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f574800c4d0 tx=0x7f574800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:47.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f573bfff700 1 -- 172.21.15.97:0/3634331396 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748011790 con 0x7f57540b8470 2023-07-06T03:58:47.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f573bfff700 1 -- 172.21.15.97:0/3634331396 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57480118f0 con 0x7f57540b8470 2023-07-06T03:58:47.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f5759fff700 1 -- 172.21.15.97:0/3634331396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5754147fe0 con 0x7f57540b8470 2023-07-06T03:58:47.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.157+0000 7f573bfff700 1 -- 172.21.15.97:0/3634331396 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748011ca0 con 0x7f57540b8470 2023-07-06T03:58:47.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.158+0000 7f5759fff700 1 -- 172.21.15.97:0/3634331396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57540cb370 con 0x7f57540b8470 2023-07-06T03:58:47.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.158+0000 7f5759fff700 1 -- 172.21.15.97:0/3634331396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5754148500 con 0x7f57540b8470 2023-07-06T03:58:47.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.159+0000 7f573bfff700 1 -- 172.21.15.97:0/3634331396 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f574801c460 con 0x7f57540b8470 2023-07-06T03:58:47.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.160+0000 7f573bfff700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 0x7f574006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.160+0000 7f573bfff700 1 -- 172.21.15.97:0/3634331396 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5748088f00 con 0x7f57540b8470 2023-07-06T03:58:47.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.161+0000 7f57537fe700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 0x7f574006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:47.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.161+0000 7f57537fe700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 0x7f574006d190 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5744009130 tx=0x7f5744008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:47.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.165+0000 7f573bfff700 1 -- 172.21.15.97:0/3634331396 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5754148500 con 0x7f57540b8470 2023-07-06T03:58:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.354+0000 7f5759fff700 1 -- 172.21.15.97:0/3634331396 --> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57540cd7e0 con 0x7f574006ace0 2023-07-06T03:58:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.357+0000 7f573bfff700 1 -- 172.21.15.97:0/3634331396 <== mgr.4104 v2:172.21.15.97:6832/1323531702 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72954 (secure 0 0 0) 0x7f5744004130 con 0x7f574006ace0 2023-07-06T03:58:47.359 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:58:47.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 -- 172.21.15.97:0/3634331396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 msgr2=0x7f574006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:47.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 0x7f574006d190 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5744009130 tx=0x7f5744008030 comp rx=0 tx=0).stop 2023-07-06T03:58:47.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 -- 172.21.15.97:0/3634331396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57540b8470 msgr2=0x7f57541e4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:47.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57540b8470 0x7f57541e4dc0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f574800c4d0 tx=0x7f574800c7e0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 -- 172.21.15.97:0/3634331396 shutdown_connections 2023-07-06T03:58:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f574006ace0 0x7f574006d190 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57540cfe30 0x7f57540cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f57540b2190 0x7f57540cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 --2- 172.21.15.97:0/3634331396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57540b8470 0x7f57541e4dc0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 -- 172.21.15.97:0/3634331396 >> 172.21.15.97:0/3634331396 conn(0x7f5754071740 msgr2=0x7f57540b73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:47.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 -- 172.21.15.97:0/3634331396 shutdown_connections 2023-07-06T03:58:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.359+0000 7f5759fff700 1 -- 172.21.15.97:0/3634331396 wait complete. 2023-07-06T03:58:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-07-06T03:58:47.379 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":550,"stamp":"2023-07-06T03:58:47.063286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296164,"kb_used_data":15012,"kb_used_omap":0,"kb_used_meta":281088,"kb_avail":754678556,"statfs":{"total":773094113280,"available":772790841344,"internally_reserved":0,"allocated":15372288,"data_stored":7711269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":287834112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002654"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":236,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:07.232831+0000","last_change":"2023-07-06T03:58:07.232881+0000","last_active":"2023-07-06T03:58:07.232831+0000","last_peered":"2023-07-06T03:58:07.232831+0000","last_clean":"2023-07-06T03:58:07.232831+0000","last_became_active":"2023-07-06T03:45:05.215035+0000","last_became_peered":"2023-07-06T03:45:05.215035+0000","last_unstale":"2023-07-06T03:58:07.232831+0000","last_undegraded":"2023-07-06T03:58:07.232831+0000","last_fullsized":"2023-07-06T03:58:07.232831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:07.232727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:07.232727+0000","last_clean_scrub_stamp":"2023-07-06T03:58:07.232727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:21:26.509472+0000","scrub_duration":0.0021799810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":246,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:14.754449+0000","last_change":"2023-07-06T03:58:14.754500+0000","last_active":"2023-07-06T03:58:14.754449+0000","last_peered":"2023-07-06T03:58:14.754449+0000","last_clean":"2023-07-06T03:58:14.754449+0000","last_became_active":"2023-07-06T03:45:05.214929+0000","last_became_peered":"2023-07-06T03:45:05.214929+0000","last_unstale":"2023-07-06T03:58:14.754449+0000","last_undegraded":"2023-07-06T03:58:14.754449+0000","last_fullsized":"2023-07-06T03:58:14.754449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:14.754305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:14.754305+0000","last_clean_scrub_stamp":"2023-07-06T03:58:14.754305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:34:57.586149+0000","scrub_duration":0.003000544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":252,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:06.658345+0000","last_change":"2023-07-06T03:58:06.658393+0000","last_active":"2023-07-06T03:58:06.658345+0000","last_peered":"2023-07-06T03:58:06.658345+0000","last_clean":"2023-07-06T03:58:06.658345+0000","last_became_active":"2023-07-06T03:45:05.133490+0000","last_became_peered":"2023-07-06T03:45:05.133490+0000","last_unstale":"2023-07-06T03:58:06.658345+0000","last_undegraded":"2023-07-06T03:58:06.658345+0000","last_fullsized":"2023-07-06T03:58:06.658345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:06.658239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:06.658239+0000","last_clean_scrub_stamp":"2023-07-06T03:58:06.658239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:46:23.904800+0000","scrub_duration":0.0023019350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":246,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:16.734086+0000","last_change":"2023-07-06T03:58:16.734136+0000","last_active":"2023-07-06T03:58:16.734086+0000","last_peered":"2023-07-06T03:58:16.734086+0000","last_clean":"2023-07-06T03:58:16.734086+0000","last_became_active":"2023-07-06T03:45:05.130267+0000","last_became_peered":"2023-07-06T03:45:05.130267+0000","last_unstale":"2023-07-06T03:58:16.734086+0000","last_undegraded":"2023-07-06T03:58:16.734086+0000","last_fullsized":"2023-07-06T03:58:16.734086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:16.733998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:16.733998+0000","last_clean_scrub_stamp":"2023-07-06T03:58:16.733998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:37:30.141599+0000","scrub_duration":0.0021046099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"141'119","reported_seq":388,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:09.522615+0000","last_change":"2023-07-06T03:58:09.522678+0000","last_active":"2023-07-06T03:58:09.522615+0000","last_peered":"2023-07-06T03:58:09.522615+0000","last_clean":"2023-07-06T03:58:09.522615+0000","last_became_active":"2023-07-06T03:44:58.098280+0000","last_became_peered":"2023-07-06T03:44:58.098280+0000","last_unstale":"2023-07-06T03:58:09.522615+0000","last_undegraded":"2023-07-06T03:58:09.522615+0000","last_fullsized":"2023-07-06T03:58:09.522615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'119","last_scrub_stamp":"2023-07-06T03:58:09.522496+0000","last_deep_scrub":"141'119","last_deep_scrub_stamp":"2023-07-06T03:58:09.522496+0000","last_clean_scrub_stamp":"2023-07-06T03:58:09.522496+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:24:06.171566+0000","scrub_duration":0.00647001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":246,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:22.749656+0000","last_change":"2023-07-06T03:58:22.749707+0000","last_active":"2023-07-06T03:58:22.749656+0000","last_peered":"2023-07-06T03:58:22.749656+0000","last_clean":"2023-07-06T03:58:22.749656+0000","last_became_active":"2023-07-06T03:45:05.133402+0000","last_became_peered":"2023-07-06T03:45:05.133402+0000","last_unstale":"2023-07-06T03:58:22.749656+0000","last_undegraded":"2023-07-06T03:58:22.749656+0000","last_fullsized":"2023-07-06T03:58:22.749656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:22.749515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:22.749515+0000","last_clean_scrub_stamp":"2023-07-06T03:58:22.749515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:13:56.504650+0000","scrub_duration":0.002502247,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":246,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:23.765422+0000","last_change":"2023-07-06T03:58:23.765477+0000","last_active":"2023-07-06T03:58:23.765422+0000","last_peered":"2023-07-06T03:58:23.765422+0000","last_clean":"2023-07-06T03:58:23.765422+0000","last_became_active":"2023-07-06T03:45:05.133050+0000","last_became_peered":"2023-07-06T03:45:05.133050+0000","last_unstale":"2023-07-06T03:58:23.765422+0000","last_undegraded":"2023-07-06T03:58:23.765422+0000","last_fullsized":"2023-07-06T03:58:23.765422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:23.765333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:23.765333+0000","last_clean_scrub_stamp":"2023-07-06T03:58:23.765333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:04:11.935581+0000","scrub_duration":0.002780416,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":240,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:04.911072+0000","last_change":"2023-07-06T03:58:04.911072+0000","last_active":"2023-07-06T03:58:04.911072+0000","last_peered":"2023-07-06T03:58:04.911072+0000","last_clean":"2023-07-06T03:58:04.911072+0000","last_became_active":"2023-07-06T03:45:05.210872+0000","last_became_peered":"2023-07-06T03:45:05.210872+0000","last_unstale":"2023-07-06T03:58:04.911072+0000","last_undegraded":"2023-07-06T03:58:04.911072+0000","last_fullsized":"2023-07-06T03:58:04.911072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:04.910934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:04.910934+0000","last_clean_scrub_stamp":"2023-07-06T03:58:04.910934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:58:05.106548+0000","scrub_duration":0.0024695099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":246,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-07-06T03:58:09.735702+0000","last_change":"2023-07-06T03:58:09.735752+0000","last_active":"2023-07-06T03:58:09.735702+0000","last_peered":"2023-07-06T03:58:09.735702+0000","last_clean":"2023-07-06T03:58:09.735702+0000","last_became_active":"2023-07-06T03:45:05.210601+0000","last_became_peered":"2023-07-06T03:45:05.210601+0000","last_unstale":"2023-07-06T03:58:09.735702+0000","last_undegraded":"2023-07-06T03:58:09.735702+0000","last_fullsized":"2023-07-06T03:58:09.735702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T03:58:09.735618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T03:58:09.735618+0000","last_clean_scrub_stamp":"2023-07-06T03:58:09.735618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:43:20.758220+0000","scrub_duration":0.0023122249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575017,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.36599999999999999,"15min":0.307},"max":{"1min":1,"5min":1,"15min":4.0570000000000004},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.69599999999999995,"15min":0.71499999999999997},"min":{"1min":0.56699999999999995,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.095,"5min":1.095,"15min":3.8660000000000001},"last":0.92600000000000005}]},{"osd":1,"last update":"Thu Jul 6 03:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.53500000000000003,"5min":0.45400000000000001,"15min":0.34000000000000002},"max":{"1min":0.97599999999999998,"5min":1.016,"15min":2.6219999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.69399999999999995,"15min":0.751},"min":{"1min":0.56599999999999995,"5min":0.40000000000000002,"15min":0.38800000000000001},"max":{"1min":0.90000000000000002,"5min":0.95599999999999996,"15min":6.6310000000000002},"last":0.872}]},{"osd":2,"last update":"Thu Jul 6 03:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69899999999999995,"15min":0.72199999999999998},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.35199999999999998},"max":{"1min":1.0169999999999999,"5min":1.0209999999999999,"15min":3.77},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73599999999999999,"15min":0.72699999999999998},"min":{"1min":0.63400000000000001,"5min":0.47999999999999998,"15min":0.45000000000000001},"max":{"1min":1.0289999999999999,"5min":1.083,"15min":1.083},"last":0.96199999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73299999999999998,"15min":0.71399999999999997},"min":{"1min":0.58799999999999997,"5min":0.46100000000000002,"15min":0.40100000000000002},"max":{"1min":0.99099999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.77000000000000002,"15min":0.75600000000000001},"min":{"1min":0.626,"5min":0.54500000000000004,"15min":0.43099999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.88900000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65700000000000003,"15min":0.63400000000000001},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.34599999999999997},"max":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.94699999999999995},"last":0.5},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.503,"5min":0.42199999999999999,"15min":0.374},"max":{"1min":0.92700000000000005,"5min":0.94099999999999995,"15min":1.4590000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.71299999999999997,"15min":0.69599999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.38},"max":{"1min":1.032,"5min":1.0489999999999999,"15min":1.0820000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71299999999999997,"15min":0.69299999999999995},"min":{"1min":0.56699999999999995,"5min":0.45300000000000001,"15min":0.40000000000000002},"max":{"1min":1.008,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.621}]},{"osd":6,"last update":"Thu Jul 6 03:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.75700000000000001,"15min":0.74399999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.438},"max":{"1min":0.97799999999999998,"5min":1.052,"15min":1.5349999999999999},"last":0.752},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.42299999999999999},"max":{"1min":1.0509999999999999,"5min":1.1000000000000001,"15min":1.3779999999999999},"last":0.65600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.76100000000000001,"15min":0.70699999999999996},"min":{"1min":0.58499999999999996,"5min":0.39700000000000002,"15min":0.29399999999999998},"max":{"1min":1.0149999999999999,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75900000000000001,"15min":0.748},"min":{"1min":0.55000000000000004,"5min":0.32400000000000001,"15min":0.26900000000000002},"max":{"1min":0.98999999999999999,"5min":3.3119999999999998,"15min":3.3399999999999999},"last":0.92200000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.747,"15min":0.69299999999999995},"min":{"1min":0.46800000000000003,"5min":0.40400000000000003,"15min":0.34599999999999997},"max":{"1min":0.96299999999999997,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.70299999999999996},"min":{"1min":0.47999999999999998,"5min":0.439,"15min":0.39000000000000001},"max":{"1min":0.93300000000000005,"5min":5.96,"15min":5.96},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.76700000000000002,"15min":0.71599999999999997},"min":{"1min":0.47499999999999998,"5min":0.46200000000000002,"15min":0.35799999999999998},"max":{"1min":0.89900000000000002,"5min":8.3740000000000006,"15min":8.3740000000000006},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":0.74399999999999999},"min":{"1min":0.52200000000000002,"5min":0.45500000000000002,"15min":0.38700000000000001},"max":{"1min":1.0620000000000001,"5min":5.8449999999999998,"15min":5.8449999999999998},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73699999999999999,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.46999999999999997,"15min":0.35799999999999998},"max":{"1min":0.94099999999999995,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.79100000000000004,"15min":0.749},"min":{"1min":0.53000000000000003,"5min":0.44500000000000001,"15min":0.373},"max":{"1min":0.92600000000000005,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.71099999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.33200000000000002},"max":{"1min":0.95899999999999996,"5min":5.7759999999999998,"15min":5.7759999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.46200000000000002,"5min":0.42399999999999999,"15min":0.39200000000000002},"max":{"1min":0.96099999999999997,"5min":3.5190000000000001,"15min":3.5190000000000001},"last":0.751}]},{"osd":5,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.751,"15min":0.69999999999999996},"min":{"1min":0.58999999999999997,"5min":0.41599999999999998,"15min":0.38600000000000001},"max":{"1min":0.92200000000000004,"5min":8.4870000000000001,"15min":8.4870000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.78600000000000003,"15min":0.76300000000000001},"min":{"1min":0.501,"5min":0.46800000000000003,"15min":0.36299999999999999},"max":{"1min":0.94999999999999996,"5min":5.923,"15min":5.923},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.504,"5min":0.5,"15min":0.46700000000000003},"max":{"1min":0.92200000000000004,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.82299999999999995,"15min":0.79400000000000004},"min":{"1min":0.68799999999999994,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93899999999999995,"5min":8.4429999999999996,"15min":8.4429999999999996},"last":0.80300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2312,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2367488,"data_stored":1406812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.42599999999999999,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.26200000000000001,"15min":0.23699999999999999},"max":{"1min":0.66500000000000004,"5min":0.69499999999999995,"15min":2.8879999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.313},"max":{"1min":0.74399999999999999,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.59599999999999997},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.26500000000000001},"max":{"1min":0.62,"5min":3.3690000000000002,"15min":13.055999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.82099999999999995,"5min":3.0590000000000002,"15min":3.0590000000000002},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51500000000000001,"15min":0.54700000000000004},"min":{"1min":0.34899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":2.9249999999999998},"last":0.497},{"interface":"front","average":{"1min":0.495,"5min":0.49399999999999999,"15min":0.505},"min":{"1min":0.34799999999999998,"5min":0.318,"15min":0.29499999999999998},"max":{"1min":0.72499999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.72699999999999998,"15min":0.86799999999999999},"min":{"1min":0.63700000000000001,"5min":0.442,"15min":0.39200000000000002},"max":{"1min":0.97699999999999998,"5min":1.444,"15min":12.989000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.753,"15min":0.78700000000000003},"min":{"1min":0.71399999999999997,"5min":0.47299999999999998,"15min":0.46000000000000002},"max":{"1min":0.997,"5min":1.139,"15min":2.7629999999999999},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72399999999999998,"15min":0.73899999999999999},"min":{"1min":0.622,"5min":0.44700000000000001,"15min":0.432},"max":{"1min":0.96399999999999997,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.748},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.72099999999999997,"15min":0.85999999999999999},"min":{"1min":0.60799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0349999999999999,"5min":1.494,"15min":13.023999999999999},"last":0.81899999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.66800000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0980000000000001,"5min":2.9670000000000001,"15min":2.9670000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.78100000000000003,"15min":0.91100000000000003},"min":{"1min":0.69699999999999995,"5min":0.47899999999999998,"15min":0.47799999999999998},"max":{"1min":1.157,"5min":2.5409999999999999,"15min":12.941000000000001},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.76300000000000001,"15min":0.77900000000000003},"min":{"1min":0.625,"5min":0.48699999999999999,"15min":0.44700000000000001},"max":{"1min":1.0269999999999999,"5min":2.431,"15min":2.806},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76900000000000002,"15min":0.77500000000000002},"min":{"1min":0.65000000000000002,"5min":0.50600000000000001,"15min":0.48399999999999999},"max":{"1min":1.0409999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.84499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575018,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41899999999999998,"15min":0.41199999999999998},"min":{"1min":0.28599999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.45000000000000001,"15min":0.443},"min":{"1min":0.30199999999999999,"5min":0.28399999999999997,"15min":0.248},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.81200000000000006},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.33000000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.75900000000000001,"15min":1.2030000000000001},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.47999999999999998,"15min":0.48999999999999999},"min":{"1min":0.33300000000000002,"5min":0.32300000000000001,"15min":0.29899999999999999},"max":{"1min":0.63600000000000001,"5min":0.63900000000000001,"15min":0.81200000000000006},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.42499999999999999,"5min":0.39800000000000002,"15min":0.38800000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.84399999999999997},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69999999999999996,"15min":0.72099999999999997},"min":{"1min":0.53800000000000003,"5min":0.48399999999999999,"15min":0.42899999999999999},"max":{"1min":1.018,"5min":1.0489999999999999,"15min":1.171},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.55600000000000005,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":0.91300000000000003,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.629}]},{"osd":5,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.59499999999999997,"5min":0.48099999999999998,"15min":0.45600000000000002},"max":{"1min":0.95899999999999996,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.74199999999999999,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.48699999999999999,"15min":0.46999999999999997},"max":{"1min":1.085,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.54900000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.046,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.754},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.73699999999999999},"min":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.46200000000000002},"max":{"1min":0.90200000000000002,"5min":1.117,"15min":1.546},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jul 6 03:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78400000000000003,"15min":0.78600000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.53200000000000003},"max":{"1min":0.96599999999999997,"5min":2.778,"15min":2.778},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78300000000000003,"15min":0.86599999999999999},"min":{"1min":0.61499999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0640000000000001,"5min":3.7069999999999999,"15min":9.5960000000000001},"last":0.73599999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575017,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40972,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330868,"statfs":{"total":96636764160,"available":96594808832,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42999999999999999,"15min":0.44500000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.23400000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":4.9199999999999999},"last":0.312},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.47499999999999998},"min":{"1min":0.311,"5min":0.28699999999999998,"15min":0.23499999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":2.4630000000000001},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jul 6 03:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48299999999999998,"15min":0.50600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":6.5960000000000001},"last":0.37},{"interface":"front","average":{"1min":0.497,"5min":0.49399999999999999,"15min":0.52300000000000002},"min":{"1min":0.36299999999999999,"5min":0.35599999999999998,"15min":0.29299999999999998},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":3.657},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.56599999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33900000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":6.6390000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54300000000000004,"15min":0.56799999999999995},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.33000000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":2.7679999999999998},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Jul 6 03:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.745,"15min":0.79800000000000004},"min":{"1min":0.53100000000000003,"5min":0.504,"15min":0.39800000000000002},"max":{"1min":1.0189999999999999,"5min":1.121,"15min":12.007},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.48899999999999999,"5min":0.48399999999999999,"15min":0.36399999999999999},"max":{"1min":0.997,"5min":1.1930000000000001,"15min":3.2200000000000002},"last":0.70699999999999996}]},{"osd":5,"last update":"Thu Jul 6 03:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.745,"15min":0.75600000000000001},"min":{"1min":0.57999999999999996,"5min":0.51100000000000001,"15min":0.45800000000000002},"max":{"1min":0.91500000000000004,"5min":1.1459999999999999,"15min":4.8719999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.80000000000000004},"min":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.38500000000000001},"max":{"1min":1.0580000000000001,"5min":1.137,"15min":6.8440000000000003},"last":0.626}]},{"osd":6,"last update":"Thu Jul 6 03:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.753,"15min":0.80300000000000005},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.437},"max":{"1min":0.93799999999999994,"5min":1.165,"15min":11.696999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76800000000000002,"15min":0.81399999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.50700000000000001},"max":{"1min":0.90300000000000002,"5min":1.107,"15min":13.084},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jul 6 03:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.44800000000000001},"max":{"1min":1.091,"5min":1.091,"15min":3.6970000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.77900000000000003,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.47499999999999998},"max":{"1min":0.96899999999999997,"5min":1.573,"15min":4.484},"last":0.60499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575017,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:58:23 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.48699999999999999},"min":{"1min":0.27500000000000002,"5min":0.251,"15min":0.24099999999999999},"max":{"1min":0.65200000000000002,"5min":0.72399999999999998,"15min":5.6550000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.40699999999999997,"5min":0.36499999999999999,"15min":0.26700000000000002},"max":{"1min":0.73099999999999998,"5min":0.89600000000000002,"15min":2.8119999999999998},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jul 6 03:58:23 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.44500000000000001,"15min":0.63300000000000001},"min":{"1min":0.32100000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":15.911},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52600000000000002,"15min":0.70899999999999996},"min":{"1min":0.35199999999999998,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.74099999999999999,"5min":0.80900000000000005,"15min":16.024000000000001},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jul 6 03:58:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.46300000000000002,"15min":0.48399999999999999},"min":{"1min":0.31900000000000001,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":0.65800000000000003,"5min":0.67900000000000005,"15min":3.25},"last":0.623},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.70999999999999996},"min":{"1min":0.35799999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84699999999999998,"5min":0.85399999999999998,"15min":16.068999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jul 6 03:58:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.498,"15min":0.40500000000000003},"max":{"1min":1.012,"5min":1.0329999999999999,"15min":1.782},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.74399999999999999,"15min":0.79900000000000004},"min":{"1min":0.60899999999999999,"5min":0.48499999999999999,"15min":0.36699999999999999},"max":{"1min":1.0269999999999999,"5min":1.373,"15min":5.7300000000000004},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jul 6 03:58:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76200000000000001,"15min":0.80800000000000005},"min":{"1min":0.58199999999999996,"5min":0.495,"15min":0.39700000000000002},"max":{"1min":1.0529999999999999,"5min":1.077,"15min":5.077},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.77300000000000002,"15min":0.84099999999999997},"min":{"1min":0.57799999999999996,"5min":0.52200000000000002,"15min":0.44800000000000001},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":5.6980000000000004},"last":0.85199999999999998}]},{"osd":6,"last update":"Thu Jul 6 03:58:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.59199999999999997,"5min":0.52800000000000002,"15min":0.48599999999999999},"max":{"1min":0.97899999999999998,"5min":1.135,"15min":1.8280000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76000000000000001,"15min":0.81000000000000005},"min":{"1min":0.60299999999999998,"5min":0.53200000000000003,"15min":0.45600000000000002},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":5.157},"last":0.876}]},{"osd":7,"last update":"Thu Jul 6 03:58:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.77500000000000002,"15min":0.78300000000000003},"min":{"1min":0.61599999999999999,"5min":0.54200000000000004,"15min":0.47399999999999998},"max":{"1min":1.1220000000000001,"5min":1.1639999999999999,"15min":1.863},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.77000000000000002,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.51100000000000001,"15min":0.49399999999999999},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":5.1189999999999998},"last":0.79200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575016,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1773568,"data_stored":816444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66900000000000004,"15min":0.65800000000000003},"min":{"1min":0.47799999999999998,"5min":0.36199999999999999,"15min":0.28399999999999997},"max":{"1min":0.80500000000000005,"5min":0.999,"15min":1.784},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68100000000000005,"15min":0.65800000000000003},"min":{"1min":0.501,"5min":0.42099999999999999,"15min":0.38},"max":{"1min":0.85499999999999998,"5min":0.95899999999999996,"15min":1.0509999999999999},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68500000000000005,"15min":0.72899999999999998},"min":{"1min":0.55900000000000005,"5min":0.43099999999999999,"15min":0.38},"max":{"1min":0.82999999999999996,"5min":1.0900000000000001,"15min":6.258},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.35899999999999999},"max":{"1min":0.877,"5min":1.0700000000000001,"15min":1.806},"last":0.747}]},{"osd":2,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.748},"min":{"1min":0.56200000000000006,"5min":0.40600000000000003,"15min":0.29199999999999998},"max":{"1min":0.83899999999999997,"5min":1.012,"15min":6.4740000000000002},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.80500000000000005},"min":{"1min":0.59299999999999997,"5min":0.53900000000000003,"15min":0.51900000000000002},"max":{"1min":0.89100000000000001,"5min":0.99399999999999999,"15min":6.4859999999999998},"last":0.86799999999999999}]},{"osd":3,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69899999999999995,"15min":0.68700000000000006},"min":{"1min":0.47499999999999998,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.81799999999999995,"5min":1.0309999999999999,"15min":1.25},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.499,"5min":0.46899999999999997,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":0.91700000000000004,"15min":1.413},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.46800000000000003,"5min":0.36099999999999999,"15min":0.35499999999999998},"max":{"1min":0.82099999999999995,"5min":0.92200000000000004,"15min":1.3280000000000001},"last":0.629},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.626,"15min":0.63500000000000001},"min":{"1min":0.47099999999999997,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.83999999999999997,"5min":0.98599999999999999,"15min":0.99299999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.443,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.84399999999999997,"5min":0.95199999999999996,"15min":0.98999999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.746,"15min":0.76000000000000001},"min":{"1min":0.58699999999999997,"5min":0.5,"15min":0.47299999999999998},"max":{"1min":0.85199999999999998,"5min":0.96299999999999997,"15min":1.6839999999999999},"last":0.83199999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:58:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68500000000000005,"15min":0.69899999999999995},"min":{"1min":0.499,"5min":0.442,"15min":0.41299999999999998},"max":{"1min":0.83699999999999997,"5min":1.0049999999999999,"15min":1.0660000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70399999999999996,"15min":0.71799999999999997},"min":{"1min":0.47199999999999998,"5min":0.41899999999999998,"15min":0.377},"max":{"1min":0.83099999999999996,"5min":1.036,"15min":1.21},"last":0.58499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37452,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334388,"statfs":{"total":96636764160,"available":96598413312,"internally_reserved":0,"allocated":2363392,"data_stored":1405793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.32100000000000001},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":4.5010000000000003},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.66300000000000003,"15min":0.72699999999999998},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.29999999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":6.4059999999999997},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.69899999999999995,"15min":0.67500000000000004},"min":{"1min":0.51000000000000001,"5min":0.44700000000000001,"15min":0.307},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.97299999999999998,"15min":1.546},"last":0.751}]},{"osd":2,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.73099999999999998,"15min":0.73499999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.376},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":8.1120000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68400000000000005},"min":{"1min":0.45400000000000001,"5min":0.40400000000000003,"15min":0.39100000000000001},"max":{"1min":0.91800000000000004,"5min":1.069,"15min":1.069},"last":0.83499999999999996}]},{"osd":3,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70299999999999996,"15min":0.73799999999999999},"min":{"1min":0.52100000000000002,"5min":0.50800000000000001,"15min":0.42199999999999999},"max":{"1min":0.97399999999999998,"5min":1.0700000000000001,"15min":8.1829999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.60999999999999999,"5min":0.495,"15min":0.39600000000000002},"max":{"1min":0.879,"5min":0.98099999999999998,"15min":4.484},"last":0.81000000000000005}]},{"osd":4,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.624},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34699999999999998},"max":{"1min":0.90300000000000002,"5min":1.0229999999999999,"15min":1.6240000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68300000000000005,"15min":0.68200000000000005},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.379},"max":{"1min":0.89400000000000002,"5min":0.98699999999999999,"15min":2.718},"last":0.51500000000000001}]},{"osd":6,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.46400000000000002,"5min":0.44500000000000001,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":2.5249999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69099999999999995,"15min":0.70199999999999996},"min":{"1min":0.50600000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.669},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jul 6 03:58:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73499999999999999,"15min":0.745},"min":{"1min":0.51500000000000001,"5min":0.46300000000000002,"15min":0.438},"max":{"1min":0.92600000000000005,"5min":1.0529999999999999,"15min":2.0459999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.755,"15min":0.76200000000000001},"min":{"1min":0.57499999999999996,"5min":0.47199999999999998,"15min":0.44900000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":2.2919999999999998},"last":0.69899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T03:58:47.384 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-07-06T03:58:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.571+0000 7f4cf5fff700 1 Processor -- start 2023-07-06T03:58:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.572+0000 7f4cf5fff700 1 -- start start 2023-07-06T03:58:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.573+0000 7f4cf5fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4cf00d0990 0x7f4cf00d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.573+0000 7f4cf5fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cf00b1e50 0x7f4cf00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.573+0000 7f4cf5fff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf00b3700 con 0x7f4cf00d28a0 2023-07-06T03:58:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.573+0000 7f4cf5fff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf00cee10 con 0x7f4cf00d0990 2023-07-06T03:58:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.573+0000 7f4cf5fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00b2530 con 0x7f4cf00b1e50 2023-07-06T03:58:47.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.573+0000 7f4cef7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4cf00d0990 0x7f4cf00d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:47.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.573+0000 7f4ceffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4cf00d28a0 0x7f4cf00cfc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56190/0 (socket says 172.21.15.97:56190) 2023-07-06T03:58:47.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.573+0000 7f4ceffff700 1 -- 172.21.15.97:0/1830177005 learned_addr learned my addr 172.21.15.97:0/1830177005 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-07-06T03:58:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.574+0000 7f4ce7fff700 1 --2- 172.21.15.97:0/1830177005 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cf00b1e50 0x7f4cf00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.574+0000 7f4cef7fe700 1 -- 172.21.15.97:0/1830177005 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cf00b1e50 msgr2=0x7f4cf00cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.574+0000 7f4cef7fe700 1 --2- 172.21.15.97:0/1830177005 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cf00b1e50 0x7f4cf00cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.574+0000 7f4cef7fe700 1 -- 172.21.15.97:0/1830177005 >> v1:172.21.15.97:6789/0 conn(0x7f4cf00d28a0 legacy=0x7f4cf00cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.574+0000 7f4cef7fe700 1 -- 172.21.15.97:0/1830177005 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf015cc30 con 0x7f4cf00d0990 2023-07-06T03:58:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.574+0000 7f4cef7fe700 1 --2- 172.21.15.97:0/1830177005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf00d0990 0x7f4cf00d0d60 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009840 tx=0x7f4ce0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b69a55dfc4017b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.574+0000 7f4ceeffd700 1 -- 172.21.15.97:0/1830177005 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0011830 con 0x7f4cf00d0990 2023-07-06T03:58:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.574+0000 7f4ceeffd700 1 -- 172.21.15.97:0/1830177005 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce0011990 con 0x7f4cf00d0990 2023-07-06T03:58:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.574+0000 7f4ceeffd700 1 -- 172.21.15.97:0/1830177005 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0011d20 con 0x7f4cf00d0990 2023-07-06T03:58:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.575+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1830177005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf00d0990 msgr2=0x7f4cf00d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.575+0000 7f4cf5fff700 1 --2- 172.21.15.97:0/1830177005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf00d0990 0x7f4cf00d0d60 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009840 tx=0x7f4ce0009b50 comp rx=0 tx=0).stop 2023-07-06T03:58:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.575+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1830177005 shutdown_connections 2023-07-06T03:58:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.575+0000 7f4cf5fff700 1 --2- 172.21.15.97:0/1830177005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf00d0990 0x7f4cf00d0d60 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.575+0000 7f4cf5fff700 1 --2- 172.21.15.97:0/1830177005 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cf00b1e50 0x7f4cf00cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.575+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1830177005 >> 172.21.15.97:0/1830177005 conn(0x7f4cf009c120 msgr2=0x7f4cf00d2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.575+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1830177005 shutdown_connections 2023-07-06T03:58:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.575+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1830177005 wait complete. 2023-07-06T03:58:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.577+0000 7f4cf5fff700 1 Processor -- start 2023-07-06T03:58:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.577+0000 7f4cf5fff700 1 -- start start 2023-07-06T03:58:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cf5fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf00b1e50 0x7f4cf00cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cf5fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cf00d0990 0x7f4cf0148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cf5fff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cf00d28a0 0x7f4cf01492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cf5fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00ca960 con 0x7f4cf00b1e50 2023-07-06T03:58:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cf5fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00cac40 con 0x7f4cf00d0990 2023-07-06T03:58:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cf5fff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00b6810 con 0x7f4cf00d28a0 2023-07-06T03:58:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf00b1e50 0x7f4cf00cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf00b1e50 0x7f4cf00cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36748/0 (socket says 172.21.15.97:36748) 2023-07-06T03:58:47.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cef7fe700 1 -- 172.21.15.97:0/1854552581 learned_addr learned my addr 172.21.15.97:0/1854552581 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cef7fe700 1 -- 172.21.15.97:0/1854552581 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cf00d28a0 msgr2=0x7f4cf01492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cef7fe700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cf00d28a0 0x7f4cf01492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cef7fe700 1 -- 172.21.15.97:0/1854552581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cf00d0990 msgr2=0x7f4cf0148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cef7fe700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cf00d0990 0x7f4cf0148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cef7fe700 1 -- 172.21.15.97:0/1854552581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce00094a0 con 0x7f4cf00b1e50 2023-07-06T03:58:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.578+0000 7f4cef7fe700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf00b1e50 0x7f4cf00cc600 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00090b0 tx=0x7f4ce0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.579+0000 7f4cecff9700 1 -- 172.21.15.97:0/1854552581 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00038a0 con 0x7f4cf00b1e50 2023-07-06T03:58:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.579+0000 7f4cecff9700 1 -- 172.21.15.97:0/1854552581 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce001e070 con 0x7f4cf00b1e50 2023-07-06T03:58:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.579+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1854552581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf00b6af0 con 0x7f4cf00b1e50 2023-07-06T03:58:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.579+0000 7f4cecff9700 1 -- 172.21.15.97:0/1854552581 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0003c40 con 0x7f4cf00b1e50 2023-07-06T03:58:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.579+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1854552581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf00ca580 con 0x7f4cf00b1e50 2023-07-06T03:58:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.580+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1854552581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf00b7010 con 0x7f4cf00b1e50 2023-07-06T03:58:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.581+0000 7f4cecff9700 1 -- 172.21.15.97:0/1854552581 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ce002d450 con 0x7f4cf00b1e50 2023-07-06T03:58:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.582+0000 7f4cecff9700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cd006ace0 0x7f4cd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.582+0000 7f4cecff9700 1 -- 172.21.15.97:0/1854552581 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4ce008a230 con 0x7f4cf00b1e50 2023-07-06T03:58:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.582+0000 7f4ce7fff700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cd006ace0 0x7f4cd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.583+0000 7f4ce7fff700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cd006ace0 0x7f4cd006d190 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4cf015a5c0 tx=0x7f4cd8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.586+0000 7f4cecff9700 1 -- 172.21.15.97:0/1854552581 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4cf00b7010 con 0x7f4cf00b1e50 2023-07-06T03:58:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.782+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1854552581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f4cf00b7010 con 0x7f4cf00b1e50 2023-07-06T03:58:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.786+0000 7f4cecff9700 1 -- 172.21.15.97:0/1854552581 <== mon.0 v2:172.21.15.97:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4ce00045c0 con 0x7f4cf00b1e50 2023-07-06T03:58:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.787+0000 7f4cecff9700 1 -- 172.21.15.97:0/1854552581 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v66)=0 v66) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4cf00b7010 con 0x7f4cf00b1e50 2023-07-06T03:58:47.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1854552581 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cd006ace0 msgr2=0x7f4cd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cd006ace0 0x7f4cd006d190 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4cf015a5c0 tx=0x7f4cd8008040 comp rx=0 tx=0).stop 2023-07-06T03:58:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1854552581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf00b1e50 msgr2=0x7f4cf00cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf00b1e50 0x7f4cf00cc600 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00090b0 tx=0x7f4ce0004750 comp rx=0 tx=0).stop 2023-07-06T03:58:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1854552581 shutdown_connections 2023-07-06T03:58:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f4cd006ace0 0x7f4cd006d190 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f4cf00d28a0 0x7f4cf01492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf00b1e50 0x7f4cf00cc600 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 --2- 172.21.15.97:0/1854552581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cf00d0990 0x7f4cf0148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1854552581 >> 172.21.15.97:0/1854552581 conn(0x7f4cf009c120 msgr2=0x7f4cf00cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:47.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1854552581 shutdown_connections 2023-07-06T03:58:47.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:47.790+0000 7f4cf5fff700 1 -- 172.21.15.97:0/1854552581 wait complete. 2023-07-06T03:58:47.809 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T03:58:47.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T03:58:48.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.004+0000 7f95953ff700 1 Processor -- start 2023-07-06T03:58:48.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.004+0000 7f95953ff700 1 -- start start 2023-07-06T03:58:48.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.005+0000 7f95953ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95900b8470 0x7f95900cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:48.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.005+0000 7f95953ff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f95900b2190 0x7f95900cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:48.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.005+0000 7f95953ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95900cfc60 0x7f9590160120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:48.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.005+0000 7f95953ff700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95900b8840 con 0x7f95900cfc60 2023-07-06T03:58:48.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.005+0000 7f95953ff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95900d12f0 con 0x7f95900b2190 2023-07-06T03:58:48.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.005+0000 7f95953ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95900cf390 con 0x7f95900b8470 2023-07-06T03:58:48.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95900cfc60 0x7f9590160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95900cfc60 0x7f9590160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36764/0 (socket says 172.21.15.97:36764) 2023-07-06T03:58:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958f7fe700 1 -- 172.21.15.97:0/2877024816 learned_addr learned my addr 172.21.15.97:0/2877024816 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958e7fc700 1 --2- 172.21.15.97:0/2877024816 >> v2:172.21.15.146:3300/0 conn(0x7f95900b2190 0x7f95900cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958effd700 1 --2- 172.21.15.97:0/2877024816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95900b8470 0x7f95900cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:48.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958f7fe700 1 -- 172.21.15.97:0/2877024816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95900b8470 msgr2=0x7f95900cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958f7fe700 1 --2- 172.21.15.97:0/2877024816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95900b8470 0x7f95900cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958f7fe700 1 -- 172.21.15.97:0/2877024816 >> v2:172.21.15.146:3300/0 conn(0x7f95900b2190 msgr2=0x7f95900cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T03:58:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958f7fe700 1 --2- 172.21.15.97:0/2877024816 >> v2:172.21.15.146:3300/0 conn(0x7f95900b2190 0x7f95900cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958f7fe700 1 -- 172.21.15.97:0/2877024816 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900d00a0 con 0x7f95900cfc60 2023-07-06T03:58:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958f7fe700 1 --2- 172.21.15.97:0/2877024816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95900cfc60 0x7f9590160120 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f95900d3de0 tx=0x7f958800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f03a8165629d18 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.006+0000 7f958dffb700 1 -- 172.21.15.97:0/2877024816 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800e070 con 0x7f95900cfc60 2023-07-06T03:58:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.007+0000 7f958dffb700 1 -- 172.21.15.97:0/2877024816 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f958800b9c0 con 0x7f95900cfc60 2023-07-06T03:58:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.007+0000 7f958dffb700 1 -- 172.21.15.97:0/2877024816 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800bd90 con 0x7f95900cfc60 2023-07-06T03:58:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.007+0000 7f95953ff700 1 -- 172.21.15.97:0/2877024816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95900cfc60 msgr2=0x7f9590160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:48.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.007+0000 7f95953ff700 1 --2- 172.21.15.97:0/2877024816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95900cfc60 0x7f9590160120 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f95900d3de0 tx=0x7f958800b330 comp rx=0 tx=0).stop 2023-07-06T03:58:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.007+0000 7f95953ff700 1 -- 172.21.15.97:0/2877024816 shutdown_connections 2023-07-06T03:58:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.007+0000 7f95953ff700 1 --2- 172.21.15.97:0/2877024816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95900cfc60 0x7f9590160120 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.007+0000 7f95953ff700 1 --2- 172.21.15.97:0/2877024816 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95900b8470 0x7f95900cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.007+0000 7f95953ff700 1 --2- 172.21.15.97:0/2877024816 >> v2:172.21.15.146:3300/0 conn(0x7f95900b2190 0x7f95900cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:48.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.007+0000 7f95953ff700 1 -- 172.21.15.97:0/2877024816 >> 172.21.15.97:0/2877024816 conn(0x7f9590071740 msgr2=0x7f9590071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:48.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.007+0000 7f95953ff700 1 -- 172.21.15.97:0/2877024816 shutdown_connections 2023-07-06T03:58:48.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.007+0000 7f95953ff700 1 -- 172.21.15.97:0/2877024816 wait complete. 2023-07-06T03:58:48.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.009+0000 7f95953ff700 1 Processor -- start 2023-07-06T03:58:48.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.009+0000 7f95953ff700 1 -- start start 2023-07-06T03:58:48.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.010+0000 7f95953ff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95900b2190 0x7f95900c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.010+0000 7f95953ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95900b8470 0x7f959015eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.010+0000 7f95953ff700 1 --2- >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95900cfc60 0x7f959015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.010+0000 7f95953ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95900c9790 con 0x7f95900b8470 2023-07-06T03:58:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.010+0000 7f95953ff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95900c86f0 con 0x7f95900b2190 2023-07-06T03:58:48.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.010+0000 7f95953ff700 1 -- --> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95900c89d0 con 0x7f95900cfc60 2023-07-06T03:58:48.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.010+0000 7f958e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95900b8470 0x7f959015eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:48.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.010+0000 7f958e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95900b8470 0x7f959015eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36778/0 (socket says 172.21.15.97:36778) 2023-07-06T03:58:48.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.010+0000 7f958e7fc700 1 -- 172.21.15.97:0/1696448139 learned_addr learned my addr 172.21.15.97:0/1696448139 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-06T03:58:48.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.010+0000 7f958effd700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95900b2190 0x7f95900c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:48.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f958f7fe700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95900cfc60 0x7f959015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f958e7fc700 1 -- 172.21.15.97:0/1696448139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95900cfc60 msgr2=0x7f959015f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f958e7fc700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95900cfc60 0x7f959015f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f958e7fc700 1 -- 172.21.15.97:0/1696448139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95900b2190 msgr2=0x7f95900c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f958e7fc700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95900b2190 0x7f95900c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f958e7fc700 1 -- 172.21.15.97:0/1696448139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9588009b30 con 0x7f95900b8470 2023-07-06T03:58:48.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f958f7fe700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95900cfc60 0x7f959015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T03:58:48.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f958e7fc700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95900b8470 0x7f959015eb90 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f958400e810 tx=0x7f958400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:48.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f95777fe700 1 -- 172.21.15.97:0/1696448139 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584013610 con 0x7f95900b8470 2023-07-06T03:58:48.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f95777fe700 1 -- 172.21.15.97:0/1696448139 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9584013790 con 0x7f95900b8470 2023-07-06T03:58:48.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f95953ff700 1 -- 172.21.15.97:0/1696448139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95900c8cb0 con 0x7f95900b8470 2023-07-06T03:58:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f95777fe700 1 -- 172.21.15.97:0/1696448139 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584013b60 con 0x7f95900b8470 2023-07-06T03:58:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.011+0000 7f95953ff700 1 -- 172.21.15.97:0/1696448139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95900c9270 con 0x7f95900b8470 2023-07-06T03:58:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.012+0000 7f95953ff700 1 -- 172.21.15.97:0/1696448139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95900b61a0 con 0x7f95900b8470 2023-07-06T03:58:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.013+0000 7f95777fe700 1 -- 172.21.15.97:0/1696448139 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9584013cc0 con 0x7f95900b8470 2023-07-06T03:58:48.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.014+0000 7f95777fe700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 0x7f957c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T03:58:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.014+0000 7f95777fe700 1 -- 172.21.15.97:0/1696448139 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9584088e90 con 0x7f95900b8470 2023-07-06T03:58:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.014+0000 7f958effd700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 0x7f957c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T03:58:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.015+0000 7f958effd700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 0x7f957c06d190 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9580006b20 tx=0x7f9580008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T03:58:48.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.019+0000 7f95777fe700 1 -- 172.21.15.97:0/1696448139 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9584057bc0 con 0x7f95900b8470 2023-07-06T03:58:48.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.252+0000 7f95953ff700 1 -- 172.21.15.97:0/1696448139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f959004e340 con 0x7f95900b8470 2023-07-06T03:58:48.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.253+0000 7f95777fe700 1 -- 172.21.15.97:0/1696448139 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+3535 (secure 0 0 0) 0x7f958408a090 con 0x7f95900b8470 2023-07-06T03:58:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-06T03:58:48.255 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":105,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8611,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6834/3313682816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3313682816},{"type":"v1","addr":"172.21.15.146:6834","nonce":3313682816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9624,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/387461585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6833","nonce":387461585},{"type":"v1","addr":"172.21.15.146:6835","nonce":387461585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9630,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3109027091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3109027091},{"type":"v1","addr":"172.21.15.97:6835","nonce":3109027091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10125,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1158088772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":1158088772},{"type":"v1","addr":"172.21.15.97:6837","nonce":1158088772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[]} 2023-07-06T03:58:48.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.255+0000 7f95953ff700 1 -- 172.21.15.97:0/1696448139 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 msgr2=0x7f957c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:48.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.255+0000 7f95953ff700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 0x7f957c06d190 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9580006b20 tx=0x7f9580008040 comp rx=0 tx=0).stop 2023-07-06T03:58:48.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.255+0000 7f95953ff700 1 -- 172.21.15.97:0/1696448139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95900b8470 msgr2=0x7f959015eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T03:58:48.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.255+0000 7f95953ff700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95900b8470 0x7f959015eb90 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f958400e810 tx=0x7f958400eb20 comp rx=0 tx=0).stop 2023-07-06T03:58:48.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.255+0000 7f95953ff700 1 -- 172.21.15.97:0/1696448139 shutdown_connections 2023-07-06T03:58:48.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.255+0000 7f95953ff700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.97:6832/1323531702,v1:172.21.15.97:6833/1323531702] conn(0x7f957c06ace0 0x7f957c06d190 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:48.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.255+0000 7f95953ff700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.146:3301/0,v1:172.21.15.146:6790/0] conn(0x7f95900cfc60 0x7f959015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:48.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.255+0000 7f95953ff700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95900b2190 0x7f95900c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:48.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.255+0000 7f95953ff700 1 --2- 172.21.15.97:0/1696448139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95900b8470 0x7f959015eb90 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T03:58:48.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.255+0000 7f95953ff700 1 -- 172.21.15.97:0/1696448139 >> 172.21.15.97:0/1696448139 conn(0x7f9590071740 msgr2=0x7f95900d2750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T03:58:48.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.256+0000 7f95953ff700 1 -- 172.21.15.97:0/1696448139 shutdown_connections 2023-07-06T03:58:48.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-06T03:58:48.256+0000 7f95953ff700 1 -- 172.21.15.97:0/1696448139 wait complete. 2023-07-06T03:58:48.260 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 105 2023-07-06T03:58:48.275 DEBUG:tasks.cephfs.filesystem:already dead... 2023-07-06T03:58:48.275 INFO:teuthology.misc:Shutting down mds daemons... 2023-07-06T03:58:48.276 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T03:58:48.276 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.288 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T03:58:48.289 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T03:58:48.289 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.341 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T03:58:48.342 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T03:58:48.342 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.358 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T03:58:48.358 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T03:58:48.359 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.413 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T03:58:48.414 INFO:teuthology.misc:Shutting down osd daemons... 2023-07-06T03:58:48.414 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-07-06T03:58:48.414 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.455 INFO:tasks.ceph.osd.0:Stopped 2023-07-06T03:58:48.455 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-07-06T03:58:48.456 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.506 INFO:tasks.ceph.osd.1:Stopped 2023-07-06T03:58:48.506 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-07-06T03:58:48.507 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.549 INFO:tasks.ceph.osd.2:Stopped 2023-07-06T03:58:48.549 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-07-06T03:58:48.550 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.608 INFO:tasks.ceph.osd.3:Stopped 2023-07-06T03:58:48.608 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-07-06T03:58:48.609 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.660 INFO:tasks.ceph.osd.4:Stopped 2023-07-06T03:58:48.661 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-07-06T03:58:48.661 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.710 INFO:tasks.ceph.osd.5:Stopped 2023-07-06T03:58:48.710 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-07-06T03:58:48.711 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.766 INFO:tasks.ceph.osd.6:Stopped 2023-07-06T03:58:48.766 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-07-06T03:58:48.767 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.811 INFO:tasks.ceph.osd.7:Stopped 2023-07-06T03:58:48.811 INFO:teuthology.misc:Shutting down mgr daemons... 2023-07-06T03:58:48.812 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-07-06T03:58:48.812 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.874 INFO:tasks.ceph.mgr.y:Stopped 2023-07-06T03:58:48.875 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-07-06T03:58:48.875 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.934 INFO:tasks.ceph.mgr.x:Stopped 2023-07-06T03:58:48.935 INFO:teuthology.misc:Shutting down mon daemons... 2023-07-06T03:58:48.936 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-07-06T03:58:48.937 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.958 INFO:tasks.ceph.mon.a:Stopped 2023-07-06T03:58:48.958 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-07-06T03:58:48.959 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:48.976 INFO:tasks.ceph.mon.b:Stopped 2023-07-06T03:58:48.976 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-07-06T03:58:48.977 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T03:58:49.034 INFO:tasks.ceph.mon.c:Stopped 2023-07-06T03:58:49.035 INFO:tasks.ceph:Checking cluster log for badness... 2023-07-06T03:58:49.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2023-07-06T03:58:49.124 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:58:49.125 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-07-06T03:58:49.389 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:58:49.390 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-07-06T03:58:49.571 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:58:49.572 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-07-06T03:58:49.786 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:58:49.787 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-07-06T03:58:49.965 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:58:49.966 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-07-06T03:58:50.299 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:58:50.300 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-07-06T03:58:50.502 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:58:50.503 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-07-06T03:58:50.726 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:58:50.727 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-07-06T03:58:50.907 INFO:tasks.ceph:Archiving mon data... 2023-07-06T03:58:50.908 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895/data/mon.a.tgz 2023-07-06T03:58:50.909 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2023-07-06T03:58:50.959 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.GlG2HYPaBS 2023-07-06T03:58:50.960 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.GlG2HYPaBS 2023-07-06T03:58:51.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.GlG2HYPaBS 2023-07-06T03:58:51.250 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.GlG2HYPaBS is 479KB 2023-07-06T03:58:51.348 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.GlG2HYPaBS 2023-07-06T03:58:51.392 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895/data/mon.b.tgz 2023-07-06T03:58:51.393 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2023-07-06T03:58:51.442 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.4wKyREsNff 2023-07-06T03:58:51.443 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.4wKyREsNff 2023-07-06T03:58:51.619 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.4wKyREsNff 2023-07-06T03:58:51.736 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.4wKyREsNff is 497KB 2023-07-06T03:58:51.839 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.4wKyREsNff 2023-07-06T03:58:51.884 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895/data/mon.c.tgz 2023-07-06T03:58:51.884 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2023-07-06T03:58:51.968 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.H2CbRkaWCr 2023-07-06T03:58:51.969 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.H2CbRkaWCr 2023-07-06T03:58:52.176 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.H2CbRkaWCr 2023-07-06T03:58:52.291 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.H2CbRkaWCr is 498KB 2023-07-06T03:58:52.389 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.H2CbRkaWCr 2023-07-06T03:58:52.435 INFO:tasks.ceph:Cleaning ceph cluster... 2023-07-06T03:58:52.435 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-06T03:58:52.440 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-06T03:58:52.543 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-07-06T03:58:52.544 DEBUG:teuthology.orchestra.run.smithi097:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-06T03:58:52.548 DEBUG:teuthology.orchestra.run.smithi146:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-06T03:58:52.643 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-06T03:58:52.747 INFO:teuthology.orchestra.run.smithi097.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-06T03:58:52.763 INFO:tasks.ceph:Archiving crash dumps... 2023-07-06T03:58:52.765 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895/remote/smithi097/crash 2023-07-06T03:58:52.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-06T03:58:52.833 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895/remote/smithi146/crash 2023-07-06T03:58:52.835 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-06T03:58:52.902 INFO:tasks.ceph:Shutting down logrotate 2023-07-06T03:58:52.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:52.907 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:53.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:53.078 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-06T03:58:53.146 INFO:tasks.ceph:Compressing logs... 2023-07-06T03:58:53.146 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-06T03:58:53.151 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-06T03:58:53.214 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131717.log 2023-07-06T03:58:53.215 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170593.log 2023-07-06T03:58:53.216 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131717.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131717.log.gz 2023-07-06T03:58:53.217 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159852.log 2023-07-06T03:58:53.217 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170593.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170593.log.gz 2023-07-06T03:58:53.218 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.122778.log 2023-07-06T03:58:53.219 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159852.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159852.log.gz 2023-07-06T03:58:53.219 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132484.log 2023-07-06T03:58:53.220 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112144.log 2023-07-06T03:58:53.221 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.2.122778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169460.log 2023-07-06T03:58:53.221 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.2.122778.log.gz 2023-07-06T03:58:53.221 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132484.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132484.log.gz 2023-07-06T03:58:53.221 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163665.log 2023-07-06T03:58:53.223 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-07-06T03:58:53.223 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.112144.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112144.log.gz 2023-07-06T03:58:53.223 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.125117.log 2023-07-06T03:58:53.224 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112243.log 2023-07-06T03:58:53.225 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169460.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.169460.log.gz 2023-07-06T03:58:53.225 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157292.log 2023-07-06T03:58:53.225 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163665.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163665.log.gz 2023-07-06T03:58:53.225 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140431.log 2023-07-06T03:58:53.226 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157292.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.157292.log.gz 2023-07-06T03:58:53.226 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132681.log 2023-07-06T03:58:53.226 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140431.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.140431.log.gz 2023-07-06T03:58:53.226 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130888.log 2023-07-06T03:58:53.229 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.125117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.122283.log 2023-07-06T03:58:53.229 INFO:teuthology.orchestra.run.smithi146.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.1.125117.log.gz 2023-07-06T03:58:53.230 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.112243.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112243.log.gz 2023-07-06T03:58:53.230 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.123196.log 2023-07-06T03:58:53.230 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.122283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112092.log 2023-07-06T03:58:53.230 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.123196.log: 89.7% -- replaced with /var/log/ceph/ceph-client.1.122283.log.gz 2023-07-06T03:58:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-06T03:58:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.112092.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112092.log.gz 2023-07-06T03:58:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.121319.log 2023-07-06T03:58:53.231 INFO:teuthology.orchestra.run.smithi146.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.1.123196.log.gz 2023-07-06T03:58:53.232 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132681.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.132681.log.gz 2023-07-06T03:58:53.233 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153727.log 2023-07-06T03:58:53.233 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130888.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.130888.log.gz 2023-07-06T03:58:53.233 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119441.log 2023-07-06T03:58:53.233 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153727.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153727.log.gz 2023-07-06T03:58:53.234 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136517.log 2023-07-06T03:58:53.234 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119441.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119441.log.gz 2023-07-06T03:58:53.234 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165855.log 2023-07-06T03:58:53.234 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136517.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136517.log.gz 2023-07-06T03:58:53.234 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148027.log 2023-07-06T03:58:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165855.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165855.log.gz 2023-07-06T03:58:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159483.log 2023-07-06T03:58:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148027.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.148027.log.gz 2023-07-06T03:58:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146867.log 2023-07-06T03:58:53.239 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159483.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.159483.log.gz 2023-07-06T03:58:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140382.log 2023-07-06T03:58:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146867.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146867.log.gz 2023-07-06T03:58:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157194.log 2023-07-06T03:58:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140382.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140382.log.gz 2023-07-06T03:58:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171128.log 2023-07-06T03:58:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157194.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157194.log.gz 2023-07-06T03:58:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142017.log 2023-07-06T03:58:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171128.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171128.log.gz 2023-07-06T03:58:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124348.log 2023-07-06T03:58:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142017.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142017.log.gz 2023-07-06T03:58:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172316.log 2023-07-06T03:58:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124348.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124348.log.gz 2023-07-06T03:58:53.242 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146965.log 2023-07-06T03:58:53.244 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.120442.log 2023-07-06T03:58:53.244 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.121319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112291.log 2023-07-06T03:58:53.244 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.120442.log: 92.1% -- replaced with /var/log/ceph/ceph-client.1.121319.log.gz 2023-07-06T03:58:53.244 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.119467.log 2023-07-06T03:58:53.245 INFO:teuthology.orchestra.run.smithi146.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.1.120442.log.gz 2023-07-06T03:58:53.245 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.112291.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112291.log.gz 2023-07-06T03:58:53.245 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-07-06T03:58:53.245 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.3.119467.log: 93.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-06T03:58:53.245 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112340.log 2023-07-06T03:58:53.246 INFO:teuthology.orchestra.run.smithi146.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.3.119467.log.gz 2023-07-06T03:58:53.246 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.127880.log 2023-07-06T03:58:53.246 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.112340.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112340.log.gz 2023-07-06T03:58:53.246 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.126921.log 2023-07-06T03:58:53.246 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.127880.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-07-06T03:58:53.247 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.126921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.125971.log 2023-07-06T03:58:53.247 INFO:teuthology.orchestra.run.smithi146.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.1.127880.log.gz 2023-07-06T03:58:53.247 INFO:teuthology.orchestra.run.smithi146.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.1.126921.log.gz 2023-07-06T03:58:53.247 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.124181.log 2023-07-06T03:58:53.247 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.125971.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-07-06T03:58:53.248 INFO:teuthology.orchestra.run.smithi146.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.1.125971.log.gz 2023-07-06T03:58:53.248 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.124181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112042.log 2023-07-06T03:58:53.248 INFO:teuthology.orchestra.run.smithi146.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.1.124181.log.gz 2023-07-06T03:58:53.250 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-07-06T03:58:53.251 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.112042.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112042.log.gz 2023-07-06T03:58:53.251 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-07-06T03:58:53.251 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-07-06T03:58:53.251 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-06T03:58:53.252 INFO:teuthology.orchestra.run.smithi146.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-07-06T03:58:53.254 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-07-06T03:58:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172316.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.172316.log.gz 2023-07-06T03:58:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156900.log 2023-07-06T03:58:53.257 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146965.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.146965.log.gz 2023-07-06T03:58:53.257 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155873.log 2023-07-06T03:58:53.257 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156900.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.156900.log.gz 2023-07-06T03:58:53.257 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131521.log 2023-07-06T03:58:53.257 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155873.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.155873.log.gz 2023-07-06T03:58:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145028.log 2023-07-06T03:58:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131521.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.131521.log.gz 2023-07-06T03:58:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117599.log 2023-07-06T03:58:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145028.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145028.log.gz 2023-07-06T03:58:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171177.log 2023-07-06T03:58:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.117599.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117599.log.gz 2023-07-06T03:58:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146524.log 2023-07-06T03:58:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171177.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.171177.log.gz 2023-07-06T03:58:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128157.log 2023-07-06T03:58:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146524.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146524.log.gz 2023-07-06T03:58:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152956.log 2023-07-06T03:58:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128157.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128157.log.gz 2023-07-06T03:58:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137248.log 2023-07-06T03:58:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152956.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.152956.log.gz 2023-07-06T03:58:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162398.log 2023-07-06T03:58:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137248.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137248.log.gz 2023-07-06T03:58:53.261 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127839.log 2023-07-06T03:58:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162398.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.162398.log.gz 2023-07-06T03:58:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172120.log 2023-07-06T03:58:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127839.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.127839.log.gz 2023-07-06T03:58:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156628.log 2023-07-06T03:58:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172120.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172120.log.gz 2023-07-06T03:58:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162545.log 2023-07-06T03:58:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156628.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156628.log.gz 2023-07-06T03:58:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143992.log 2023-07-06T03:58:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162545.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.162545.log.gz 2023-07-06T03:58:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149828.log 2023-07-06T03:58:53.265 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-osd.4.log: 92.9% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-06T03:58:53.265 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112193.log 2023-07-06T03:58:53.265 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.112193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.118792.log 2023-07-06T03:58:53.266 INFO:teuthology.orchestra.run.smithi146.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112193.log.gz 2023-07-06T03:58:53.266 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.1.118792.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-07-06T03:58:53.269 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143992.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143992.log.gz 2023-07-06T03:58:53.269 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150605.log 2023-07-06T03:58:53.269 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149828.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149828.log.gz 2023-07-06T03:58:53.270 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144615.log 2023-07-06T03:58:53.270 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150605.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150605.log.gz 2023-07-06T03:58:53.270 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165414.log 2023-07-06T03:58:53.271 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144615.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.144615.log.gz 2023-07-06T03:58:53.271 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170251.log 2023-07-06T03:58:53.271 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165414.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.165414.log.gz 2023-07-06T03:58:53.271 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123689.log 2023-07-06T03:58:53.272 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170251.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170251.log.gz 2023-07-06T03:58:53.272 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148076.log 2023-07-06T03:58:53.272 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123689.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123689.log.gz 2023-07-06T03:58:53.272 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145325.log 2023-07-06T03:58:53.272 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148076.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.148076.log.gz 2023-07-06T03:58:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130986.log 2023-07-06T03:58:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145325.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.145325.log.gz 2023-07-06T03:58:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120121.log 2023-07-06T03:58:53.273 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130986.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.130986.log.gz 2023-07-06T03:58:53.274 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147831.log 2023-07-06T03:58:53.274 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120121.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120121.log.gz 2023-07-06T03:58:53.274 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154749.log 2023-07-06T03:58:53.274 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147831.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147831.log.gz 2023-07-06T03:58:53.274 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148824.log 2023-07-06T03:58:53.274 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154749.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.154749.log.gz 2023-07-06T03:58:53.275 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118422.log 2023-07-06T03:58:53.276 INFO:teuthology.orchestra.run.smithi146.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.1.118792.log.gz 2023-07-06T03:58:53.276 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111995.log 2023-07-06T03:58:53.278 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148824.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148824.log.gz 2023-07-06T03:58:53.279 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172463.log 2023-07-06T03:58:53.279 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118422.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118422.log.gz 2023-07-06T03:58:53.279 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167008.log 2023-07-06T03:58:53.279 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172463.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.172463.log.gz 2023-07-06T03:58:53.280 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155481.log 2023-07-06T03:58:53.280 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.167008.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.167008.log.gz 2023-07-06T03:58:53.280 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137916.log 2023-07-06T03:58:53.280 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155481.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.155481.log.gz 2023-07-06T03:58:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138745.log 2023-07-06T03:58:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137916.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137916.log.gz 2023-07-06T03:58:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119447.log 2023-07-06T03:58:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138745.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138745.log.gz 2023-07-06T03:58:53.281 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156236.log 2023-07-06T03:58:53.282 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119447.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119447.log.gz 2023-07-06T03:58:53.282 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136615.log 2023-07-06T03:58:53.282 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156236.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156236.log.gz 2023-07-06T03:58:53.283 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141513.log 2023-07-06T03:58:53.283 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.111995.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-07-06T03:58:53.284 INFO:teuthology.orchestra.run.smithi146.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111995.log.gz 2023-07-06T03:58:53.286 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136615.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136615.log.gz 2023-07-06T03:58:53.286 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162496.log 2023-07-06T03:58:53.287 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141513.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141513.log.gz 2023-07-06T03:58:53.287 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159116.log 2023-07-06T03:58:53.287 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162496.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.162496.log.gz 2023-07-06T03:58:53.287 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149633.log 2023-07-06T03:58:53.287 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159116.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159116.log.gz 2023-07-06T03:58:53.288 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165561.log 2023-07-06T03:58:53.288 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149633.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149633.log.gz 2023-07-06T03:58:53.288 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128808.log 2023-07-06T03:58:53.288 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165561.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165561.log.gz 2023-07-06T03:58:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153678.log 2023-07-06T03:58:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128808.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128808.log.gz 2023-07-06T03:58:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152205.log 2023-07-06T03:58:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153678.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153678.log.gz 2023-07-06T03:58:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145834.log 2023-07-06T03:58:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152205.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152205.log.gz 2023-07-06T03:58:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126475.log 2023-07-06T03:58:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145834.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.145834.log.gz 2023-07-06T03:58:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168169.log 2023-07-06T03:58:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126475.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.126475.log.gz-5 2023-07-06T03:58:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:--verbose -- /var/log/ceph/ceph-client.admin.154894.log 2023-07-06T03:58:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.168169.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.168169.log.gz 2023-07-06T03:58:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136273.log 2023-07-06T03:58:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154894.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.154894.log.gz 2023-07-06T03:58:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130364.log 2023-07-06T03:58:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136273.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136273.log.gz 2023-07-06T03:58:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124713.log 2023-07-06T03:58:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130364.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.130364.log.gz 2023-07-06T03:58:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147032.log 2023-07-06T03:58:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124713.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124713.log.gz 2023-07-06T03:58:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143908.log 2023-07-06T03:58:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147032.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147032.log.gz 2023-07-06T03:58:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171875.log 2023-07-06T03:58:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143908.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143908.log.gz 2023-07-06T03:58:53.298 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142115.log 2023-07-06T03:58:53.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171875.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.171875.log.gz 2023-07-06T03:58:53.298 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152708.log 2023-07-06T03:58:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142115.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142115.log.gz 2023-07-06T03:58:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129175.log 2023-07-06T03:58:53.300 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152708.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152708.log.gz 2023-07-06T03:58:53.300 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165659.log 2023-07-06T03:58:53.301 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.129175.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129175.log.gz 2023-07-06T03:58:53.301 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156530.log 2023-07-06T03:58:53.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165659.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165659.log.gz 2023-07-06T03:58:53.302 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162239.log 2023-07-06T03:58:53.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156530.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.156530.log.gz 2023-07-06T03:58:53.304 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172839.log 2023-07-06T03:58:53.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162239.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.162239.log.gz 2023-07-06T03:58:53.304 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119837.log 2023-07-06T03:58:53.305 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172839.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172839.log.gz 2023-07-06T03:58:53.305 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127112.log 2023-07-06T03:58:53.305 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119837.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119837.log.gz 2023-07-06T03:58:53.306 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124615.log 2023-07-06T03:58:53.306 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127112.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127112.log.gz 2023-07-06T03:58:53.306 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153849.log 2023-07-06T03:58:53.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124615.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124615.log.gz 2023-07-06T03:58:53.307 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126916.log 2023-07-06T03:58:53.308 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153849.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.153849.log.gz 2023-07-06T03:58:53.308 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160099.log 2023-07-06T03:58:53.309 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124495.log 2023-07-06T03:58:53.309 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126916.log.gz 2023-07-06T03:58:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160099.log: 89.1%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.160099.log.gz 2023-07-06T03:58:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:--verbose -- /var/log/ceph/ceph-client.admin.138500.log 2023-07-06T03:58:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124495.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124495.log.gz 2023-07-06T03:58:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141121.log 2023-07-06T03:58:53.312 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138500.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138500.log.gz 2023-07-06T03:58:53.312 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123640.log 2023-07-06T03:58:53.313 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141121.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.141121.log.gz 2023-07-06T03:58:53.314 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131325.log 2023-07-06T03:58:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123640.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123640.log.gz 2023-07-06T03:58:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165267.log 2023-07-06T03:58:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131325.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.131325.log.gz 2023-07-06T03:58:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142741.log 2023-07-06T03:58:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165267.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.165267.log.gz 2023-07-06T03:58:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140770.log 2023-07-06T03:58:53.317 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142741.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142741.log.gz 2023-07-06T03:58:53.317 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120229.log 2023-07-06T03:58:53.318 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140770.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140770.log.gz 2023-07-06T03:58:53.318 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141968.log 2023-07-06T03:58:53.319 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120229.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120229.log.gz 2023-07-06T03:58:53.319 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166417.log 2023-07-06T03:58:53.320 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141968.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.141968.log.gz 2023-07-06T03:58:53.320 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128059.log 2023-07-06T03:58:53.321 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130839.log 2023-07-06T03:58:53.321 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.166417.log.gz 2023-07-06T03:58:53.322 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128059.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128059.log.gz 2023-07-06T03:58:53.322 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166653.log 2023-07-06T03:58:53.323 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130839.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.130839.log.gz 2023-07-06T03:58:53.324 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150703.log 2023-07-06T03:58:53.324 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166653.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166653.log.gz 2023-07-06T03:58:53.324 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149584.log 2023-07-06T03:58:53.325 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150703.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150703.log.gz 2023-07-06T03:58:53.325 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118775.log 2023-07-06T03:58:53.326 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149584.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149584.log.gz 2023-07-06T03:58:53.326 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155678.log 2023-07-06T03:58:53.327 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118775.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118775.log.gz 2023-07-06T03:58:53.327 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124032.log 2023-07-06T03:58:53.328 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155678.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.155678.log.gz 2023-07-06T03:58:53.328 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156843.log 2023-07-06T03:58:53.331 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124032.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124032.log.gz 2023-07-06T03:58:53.332 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153360.log 2023-07-06T03:58:53.334 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156843.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156843.log.gz 2023-07-06T03:58:53.334 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147929.log 2023-07-06T03:58:53.334 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153360.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153360.log.gz 2023-07-06T03:58:53.335 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137101.log 2023-07-06T03:58:53.335 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147929.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147929.log.gz 2023-07-06T03:58:53.335 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131981.log 2023-07-06T03:58:53.335 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137101.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.137101.log.gz 2023-07-06T03:58:53.335 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132030.log 2023-07-06T03:58:53.336 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131981.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131981.log.gz 2023-07-06T03:58:53.336 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136664.log 2023-07-06T03:58:53.337 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132030.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.132030.log.gz 2023-07-06T03:58:53.338 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118126.log 2023-07-06T03:58:53.338 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136664.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136664.log.gz 2023-07-06T03:58:53.338 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-07-06T03:58:53.338 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118126.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118126.log.gz 2023-07-06T03:58:53.339 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156432.log 2023-07-06T03:58:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125275.log 2023-07-06T03:58:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156432.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.156432.log.gz 2023-07-06T03:58:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171973.log 2023-07-06T03:58:53.341 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125275.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125275.log.gz 2023-07-06T03:58:53.341 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164684.log 2023-07-06T03:58:53.342 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171973.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.171973.log.gz 2023-07-06T03:58:53.342 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123395.log 2023-07-06T03:58:53.343 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164684.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164684.log.gz 2023-07-06T03:58:53.343 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157845.log 2023-07-06T03:58:53.344 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123395.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123395.log.gz 2023-07-06T03:58:53.344 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140016.log 2023-07-06T03:58:53.344 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157845.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157845.log.gz 2023-07-06T03:58:53.345 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144119.log 2023-07-06T03:58:53.345 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140016.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140016.log.gz 2023-07-06T03:58:53.346 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160049.log 2023-07-06T03:58:53.346 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144119.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144119.log.gz 2023-07-06T03:58:53.347 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149381.log 2023-07-06T03:58:53.347 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160049.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160049.log.gz 2023-07-06T03:58:53.347 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166555.log 2023-07-06T03:58:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149381.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149381.log.gz 2023-07-06T03:58:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172071.log 2023-07-06T03:58:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166555.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166555.log.gz 2023-07-06T03:58:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125564.log 2023-07-06T03:58:53.350 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172071.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.172071.log.gz 2023-07-06T03:58:53.350 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149486.log 2023-07-06T03:58:53.351 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125564.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125564.log.gz 2023-07-06T03:58:53.351 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146622.log 2023-07-06T03:58:53.352 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149486.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149486.log.gz 2023-07-06T03:58:53.353 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120539.log 2023-07-06T03:58:53.353 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146622.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146622.log.gz 2023-07-06T03:58:53.354 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126279.log 2023-07-06T03:58:53.354 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120539.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120539.log.gz 2023-07-06T03:58:53.355 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172169.log 2023-07-06T03:58:53.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126279.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.126279.log.gz 2023-07-06T03:58:53.356 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173008.log 2023-07-06T03:58:53.357 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172169.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172169.log.gz 2023-07-06T03:58:53.357 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141015.log 2023-07-06T03:58:53.358 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.173008.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.173008.log.gz 2023-07-06T03:58:53.358 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169191.log 2023-07-06T03:58:53.359 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141015.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-07-06T03:58:53.359 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.141015.log.gz 2023-07-06T03:58:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169191.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.169191.log.gz 2023-07-06T03:58:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158094.log 2023-07-06T03:58:53.361 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152659.log 2023-07-06T03:58:53.362 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158094.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.158094.log.gz 2023-07-06T03:58:53.362 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128563.log 2023-07-06T03:58:53.363 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152659.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.152659.log.gz 2023-07-06T03:58:53.364 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112217.log 2023-07-06T03:58:53.364 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128563.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128563.log.gz 2023-07-06T03:58:53.365 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131423.log 2023-07-06T03:58:53.365 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.112217.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112217.log.gz 2023-07-06T03:58:53.366 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138152.log 2023-07-06T03:58:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131423.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.131423.log.gz 2023-07-06T03:58:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120588.log 2023-07-06T03:58:53.368 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138152.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138152.log.gz 2023-07-06T03:58:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140138.log 2023-07-06T03:58:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120588.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120588.log.gz 2023-07-06T03:58:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134666.log 2023-07-06T03:58:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140138.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140138.log.gz 2023-07-06T03:58:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124299.log 2023-07-06T03:58:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.134666.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134666.log.gz 2023-07-06T03:58:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117939.log 2023-07-06T03:58:53.373 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124299.log: gzip -5 --verbose 89.3%-- /var/log/ceph/ceph-client.admin.136762.log -- replaced with /var/log/ceph/ceph-client.admin.124299.log.gz 2023-07-06T03:58:53.373 INFO:teuthology.orchestra.run.smithi097.stderr: 2023-07-06T03:58:53.374 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.117939.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117939.log.gz 2023-07-06T03:58:53.374 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155727.log 2023-07-06T03:58:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136762.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.136762.log.gz 2023-07-06T03:58:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137747.log 2023-07-06T03:58:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155727.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.155727.log.gz 2023-07-06T03:58:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151045.log 2023-07-06T03:58:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143665.log 2023-07-06T03:58:53.378 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137747.log.gz 2023-07-06T03:58:53.378 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151045.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.151045.log.gz 2023-07-06T03:58:53.379 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.143292.log 2023-07-06T03:58:53.380 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143665.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143665.log.gz 2023-07-06T03:58:53.380 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145567.log 2023-07-06T03:58:53.381 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.143292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170055.log 2023-07-06T03:58:53.382 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145567.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145567.log.gz 2023-07-06T03:58:53.382 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132632.log 2023-07-06T03:58:53.383 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128514.log 2023-07-06T03:58:53.384 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.170055.log.gz 2023-07-06T03:58:53.385 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132632.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.132632.log.gz 2023-07-06T03:58:53.385 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123493.log 2023-07-06T03:58:53.386 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120784.log 2023-07-06T03:58:53.387 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128514.log.gz 2023-07-06T03:58:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123493.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123493.log.gz 2023-07-06T03:58:53.388 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127210.log 2023-07-06T03:58:53.389 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136420.log 2023-07-06T03:58:53.389 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120784.log.gz 2023-07-06T03:58:53.390 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127210.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127210.log.gz 2023-07-06T03:58:53.391 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133493.log 2023-07-06T03:58:53.392 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136420.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136420.log.gz 2023-07-06T03:58:53.392 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123983.log 2023-07-06T03:58:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133493.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133493.log.gz 2023-07-06T03:58:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148235.log 2023-07-06T03:58:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123983.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123983.log.gz 2023-07-06T03:58:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137199.log 2023-07-06T03:58:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172708.log 2023-07-06T03:58:53.397 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148235.log.gz 2023-07-06T03:58:53.397 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137199.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137199.log.gz 2023-07-06T03:58:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148127.log 2023-07-06T03:58:53.399 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172708.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.172708.log.gz 2023-07-06T03:58:53.399 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123787.log 2023-07-06T03:58:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155971.log 2023-07-06T03:58:53.401 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148127.log.gz 2023-07-06T03:58:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123787.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123787.log.gz 2023-07-06T03:58:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140966.log 2023-07-06T03:58:53.403 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138990.log 2023-07-06T03:58:53.404 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155971.log.gz 2023-07-06T03:58:53.404 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140966.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140966.log.gz 2023-07-06T03:58:53.405 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155824.log 2023-07-06T03:58:53.406 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126524.log 2023-07-06T03:58:53.407 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155824.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.138990.log.gz 2023-07-06T03:58:53.407 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155824.log.gz 2023-07-06T03:58:53.408 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162643.log 2023-07-06T03:58:53.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157047.log 2023-07-06T03:58:53.409 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126524.log.gz 2023-07-06T03:58:53.410 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162643.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.162643.log.gz 2023-07-06T03:58:53.410 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127888.log 2023-07-06T03:58:53.411 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157047.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157047.log.gz 2023-07-06T03:58:53.412 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165757.log 2023-07-06T03:58:53.413 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127888.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.127888.log.gz 2023-07-06T03:58:53.413 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171777.log 2023-07-06T03:58:53.414 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165757.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.165757.log.gz 2023-07-06T03:58:53.415 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158192.log 2023-07-06T03:58:53.416 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171777.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.171777.log.gz 2023-07-06T03:58:53.416 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124932.log 2023-07-06T03:58:53.417 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158192.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.158192.log.gz 2023-07-06T03:58:53.418 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128759.log 2023-07-06T03:58:53.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124932.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124932.log.gz 2023-07-06T03:58:53.419 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125986.log 2023-07-06T03:58:53.420 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128759.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.128759.log.gz 2023-07-06T03:58:53.421 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151506.log 2023-07-06T03:58:53.422 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125986.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125986.log.gz 2023-07-06T03:58:53.422 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124250.log 2023-07-06T03:58:53.423 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151506.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.151506.log.gz 2023-07-06T03:58:53.423 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130288.log 2023-07-06T03:58:53.425 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125760.log 2023-07-06T03:58:53.425 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124250.log.gz 2023-07-06T03:58:53.426 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130288.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130288.log.gz 2023-07-06T03:58:53.426 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160539.log 2023-07-06T03:58:53.427 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160148.log 2023-07-06T03:58:53.428 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125760.log.gz 2023-07-06T03:58:53.428 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160539.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160539.log.gz 2023-07-06T03:58:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165806.log 2023-07-06T03:58:53.430 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160148.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.160148.log.gz 2023-07-06T03:58:53.430 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164830.log 2023-07-06T03:58:53.431 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165806.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.165806.log.gz 2023-07-06T03:58:53.432 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133052.log 2023-07-06T03:58:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164830.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.164830.log.gz 2023-07-06T03:58:53.434 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133591.log 2023-07-06T03:58:53.435 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171924.log 2023-07-06T03:58:53.435 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.133052.log.gz 2023-07-06T03:58:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133591.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.133591.log.gz 2023-07-06T03:58:53.437 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160294.log 2023-07-06T03:58:53.438 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141219.log 2023-07-06T03:58:53.438 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171924.log.gz 2023-07-06T03:58:53.439 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160294.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.160294.log.gz 2023-07-06T03:58:53.439 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124664.log 2023-07-06T03:58:53.440 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141219.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141219.log.gz 2023-07-06T03:58:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151751.log 2023-07-06T03:58:53.442 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124664.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124664.log.gz 2023-07-06T03:58:53.443 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160927.log 2023-07-06T03:58:53.443 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151751.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.151751.log.gz 2023-07-06T03:58:53.444 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172218.log 2023-07-06T03:58:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160927.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160927.log.gz 2023-07-06T03:58:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141562.log 2023-07-06T03:58:53.446 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172218.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.172218.log.gz 2023-07-06T03:58:53.447 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148408.log 2023-07-06T03:58:53.448 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123542.log 2023-07-06T03:58:53.448 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.141562.log.gz 2023-07-06T03:58:53.449 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148408.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.148408.log.gz 2023-07-06T03:58:53.450 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118076.log 2023-07-06T03:58:53.450 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123542.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123542.log.gz 2023-07-06T03:58:53.451 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169835.log 2023-07-06T03:58:53.452 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142396.log 2023-07-06T03:58:53.453 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118076.log.gz 2023-07-06T03:58:53.453 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169835.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.169835.log.gz 2023-07-06T03:58:53.454 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157599.log 2023-07-06T03:58:53.454 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142396.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.142396.log.gz 2023-07-06T03:58:53.455 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165169.log 2023-07-06T03:58:53.456 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157599.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157599.log.gz 2023-07-06T03:58:53.457 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128414.log 2023-07-06T03:58:53.458 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165169.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165169.log.gz 2023-07-06T03:58:53.458 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157896.log 2023-07-06T03:58:53.459 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128414.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128414.log.gz 2023-07-06T03:58:53.460 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164161.log 2023-07-06T03:58:53.461 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127406.log 2023-07-06T03:58:53.462 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157896.log.gz 2023-07-06T03:58:53.462 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164161.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164161.log.gz 2023-07-06T03:58:53.463 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139669.log 2023-07-06T03:58:53.464 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127406.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127406.log.gz 2023-07-06T03:58:53.464 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131472.log 2023-07-06T03:58:53.465 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139669.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139669.log.gz 2023-07-06T03:58:53.466 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134739.log 2023-07-06T03:58:53.467 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131472.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.131472.log.gz 2023-07-06T03:58:53.468 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125079.log 2023-07-06T03:58:53.469 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.134739.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134739.log.gz 2023-07-06T03:58:53.469 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127691.log 2023-07-06T03:58:53.470 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125079.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125079.log.gz 2023-07-06T03:58:53.471 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160000.log 2023-07-06T03:58:53.472 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126573.log 2023-07-06T03:58:53.473 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127691.log.gz 2023-07-06T03:58:53.473 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160000.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160000.log.gz 2023-07-06T03:58:53.474 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146671.log 2023-07-06T03:58:53.475 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126573.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.126573.log.gz 2023-07-06T03:58:53.475 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161662.log 2023-07-06T03:58:53.476 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146671.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.146671.log.gz 2023-07-06T03:58:53.477 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129781.log 2023-07-06T03:58:53.478 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152022.log 2023-07-06T03:58:53.478 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.161662.log.gz 2023-07-06T03:58:53.479 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.129781.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129781.log.gz 2023-07-06T03:58:53.479 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170153.log 2023-07-06T03:58:53.480 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152022.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-06T03:58:53.481 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.152022.log.gz 2023-07-06T03:58:53.481 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170153.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.170153.log.gz 2023-07-06T03:58:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127937.log 2023-07-06T03:58:53.483 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142692.log 2023-07-06T03:58:53.484 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127937.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.127937.log.gz 2023-07-06T03:58:53.485 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140187.log 2023-07-06T03:58:53.486 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160976.log 2023-07-06T03:58:53.486 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142692.log.gz 2023-07-06T03:58:53.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140187.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.140187.log.gz 2023-07-06T03:58:53.487 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-06T03:58:53.488 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163616.log 2023-07-06T03:58:53.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166002.log 2023-07-06T03:58:53.489 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160976.log.gz 2023-07-06T03:58:53.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163616.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.163616.log.gz 2023-07-06T03:58:53.490 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131784.log 2023-07-06T03:58:53.491 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166002.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166002.log.gz 2023-07-06T03:58:53.492 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.139442.log 2023-07-06T03:58:53.493 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131784.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131784.log.gz 2023-07-06T03:58:53.493 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140868.log 2023-07-06T03:58:53.495 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.139442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155109.log 2023-07-06T03:58:53.496 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140868.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140868.log.gz 2023-07-06T03:58:53.496 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118027.log 2023-07-06T03:58:53.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155109.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.155109.log.gz 2023-07-06T03:58:53.498 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162937.log 2023-07-06T03:58:53.499 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118027.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118027.log.gz 2023-07-06T03:58:53.500 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166318.log 2023-07-06T03:58:53.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152303.log 2023-07-06T03:58:53.501 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.162937.log.gz 2023-07-06T03:58:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166318.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.166318.log.gz 2023-07-06T03:58:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127063.log 2023-07-06T03:58:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152303.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152303.log.gz 2023-07-06T03:58:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152571.log 2023-07-06T03:58:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127063.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127063.log.gz 2023-07-06T03:58:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128306.log 2023-07-06T03:58:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152571.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152571.log.gz 2023-07-06T03:58:53.508 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125613.log 2023-07-06T03:58:53.508 INFO:teuthology.orchestra.run.smithi097.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-07-06T03:58:53.508 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128306.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128306.log.gz 2023-07-06T03:58:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143567.log 2023-07-06T03:58:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125613.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125613.log.gz 2023-07-06T03:58:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147643.log 2023-07-06T03:58:53.512 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143567.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143567.log.gz 2023-07-06T03:58:53.512 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119118.log 2023-07-06T03:58:53.513 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150850.log 2023-07-06T03:58:53.514 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147643.log.gz 2023-07-06T03:58:53.514 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119118.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119118.log.gz 2023-07-06T03:58:53.515 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156998.log 2023-07-06T03:58:53.516 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162741.log 2023-07-06T03:58:53.516 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.150850.log.gz 2023-07-06T03:58:53.517 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156998.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156998.log.gz 2023-07-06T03:58:53.517 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126671.log 2023-07-06T03:58:53.518 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164585.log 2023-07-06T03:58:53.519 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.162741.log.gz 2023-07-06T03:58:53.520 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126671.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.126671.log.gz 2023-07-06T03:58:53.520 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121249.log 2023-07-06T03:58:53.521 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164585.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.164585.log.gz 2023-07-06T03:58:53.521 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148457.log 2023-07-06T03:58:53.522 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.121249.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121249.log.gz 2023-07-06T03:58:53.523 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169737.log 2023-07-06T03:58:53.524 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148457.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.148457.log.gz 2023-07-06T03:58:53.525 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146283.log 2023-07-06T03:58:53.526 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151408.log 2023-07-06T03:58:53.526 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169737.log.gz 2023-07-06T03:58:53.527 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146283.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146283.log.gz 2023-07-06T03:58:53.527 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119020.log 2023-07-06T03:58:53.528 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151408.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151408.log.gz 2023-07-06T03:58:53.529 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169509.log 2023-07-06T03:58:53.530 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119020.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.119020.log.gz 2023-07-06T03:58:53.531 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169680.log 2023-07-06T03:58:53.531 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169509.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.169509.log.gz 2023-07-06T03:58:53.532 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127504.log 2023-07-06T03:58:53.533 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169680.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.169680.log.gz 2023-07-06T03:58:53.533 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137796.log 2023-07-06T03:58:53.534 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127504.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.127504.log.gz 2023-07-06T03:58:53.535 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137003.log 2023-07-06T03:58:53.536 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137796.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.137796.log.gz 2023-07-06T03:58:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170202.log 2023-07-06T03:58:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137003.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137003.log.gz 2023-07-06T03:58:53.538 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147474.log 2023-07-06T03:58:53.539 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170202.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.170202.log.gz 2023-07-06T03:58:53.540 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141611.log 2023-07-06T03:58:53.540 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147474.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.147474.log.gz 2023-07-06T03:58:53.541 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155055.log 2023-07-06T03:58:53.542 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141611.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.141611.log.gz 2023-07-06T03:58:53.543 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159287.log 2023-07-06T03:58:53.544 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155055.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155055.log.gz 2023-07-06T03:58:53.544 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120882.log 2023-07-06T03:58:53.545 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159287.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159287.log.gz 2023-07-06T03:58:53.546 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154553.log 2023-07-06T03:58:53.547 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151314.log 2023-07-06T03:58:53.547 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120882.log.gz 2023-07-06T03:58:53.548 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154553.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.154553.log.gz 2023-07-06T03:58:53.548 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131883.log 2023-07-06T03:58:53.549 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151604.log 2023-07-06T03:58:53.550 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.151314.log.gz 2023-07-06T03:58:53.550 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131883.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.131883.log.gz 2023-07-06T03:58:53.551 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155922.log 2023-07-06T03:58:53.552 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161855.log 2023-07-06T03:58:53.552 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151604.log.gz 2023-07-06T03:58:53.553 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155922.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155922.log.gz 2023-07-06T03:58:53.554 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150556.log 2023-07-06T03:58:53.554 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161855.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.161855.log.gz 2023-07-06T03:58:53.555 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132396.log 2023-07-06T03:58:53.556 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150556.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.150556.log.gz 2023-07-06T03:58:53.556 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130937.log 2023-07-06T03:58:53.557 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132396.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.132396.log.gz 2023-07-06T03:58:53.558 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127455.log 2023-07-06T03:58:53.559 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130937.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.130937.log.gz 2023-07-06T03:58:53.560 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162692.log 2023-07-06T03:58:53.561 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127455.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127455.log.gz 2023-07-06T03:58:53.561 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166802.log 2023-07-06T03:58:53.562 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162692.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162692.log.gz 2023-07-06T03:58:53.563 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144724.log 2023-07-06T03:58:53.563 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166802.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.166802.log.gz 2023-07-06T03:58:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140721.log 2023-07-06T03:58:53.565 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144724.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.144724.log.gz 2023-07-06T03:58:53.566 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142066.log 2023-07-06T03:58:53.567 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140721.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.140721.log.gz 2023-07-06T03:58:53.567 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120637.log 2023-07-06T03:58:53.568 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142066.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142066.log.gz 2023-07-06T03:58:53.569 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131276.log 2023-07-06T03:58:53.570 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155355.log 2023-07-06T03:58:53.570 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120637.log.gz 2023-07-06T03:58:53.571 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131276.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131276.log.gz 2023-07-06T03:58:53.572 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125324.log 2023-07-06T03:58:53.573 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155355.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155355.log.gz 2023-07-06T03:58:53.573 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161515.log 2023-07-06T03:58:53.574 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125324.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125324.log.gz 2023-07-06T03:58:53.575 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130538.log 2023-07-06T03:58:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161515.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.161515.log.gz 2023-07-06T03:58:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128357.log 2023-07-06T03:58:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130538.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130538.log.gz 2023-07-06T03:58:53.578 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127357.log 2023-07-06T03:58:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128357.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128357.log.gz 2023-07-06T03:58:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126133.log 2023-07-06T03:58:53.580 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127357.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127357.log.gz 2023-07-06T03:58:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118175.log 2023-07-06T03:58:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126133.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126133.log.gz 2023-07-06T03:58:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119167.log 2023-07-06T03:58:53.583 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118175.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118175.log.gz 2023-07-06T03:58:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-07-06T03:58:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119167.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119167.log.gz 2023-07-06T03:58:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172022.log 2023-07-06T03:58:53.587 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154602.log 2023-07-06T03:58:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147082.log 2023-07-06T03:58:53.589 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.172022.log.gz 2023-07-06T03:58:53.589 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154602.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154602.log.gz 2023-07-06T03:58:53.590 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169786.log 2023-07-06T03:58:53.591 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147082.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147082.log.gz 2023-07-06T03:58:53.592 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148751.log 2023-07-06T03:58:53.592 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169786.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.169786.log.gz 2023-07-06T03:58:53.593 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166853.log 2023-07-06T03:58:53.594 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148604.log 2023-07-06T03:58:53.595 INFO:teuthology.orchestra.run.smithi097.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.148751.log.gz 2023-07-06T03:58:53.596 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166853.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.166853.log.gz 2023-07-06T03:58:53.596 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136566.log 2023-07-06T03:58:53.597 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148604.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148604.log.gz 2023-07-06T03:58:53.598 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123150.log 2023-07-06T03:58:53.599 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136566.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136566.log.gz 2023-07-06T03:58:53.599 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170785.log 2023-07-06T03:58:53.600 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123150.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123150.log.gz 2023-07-06T03:58:53.601 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131374.log 2023-07-06T03:58:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170785.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.170785.log.gz 2023-07-06T03:58:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161123.log 2023-07-06T03:58:53.603 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131374.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.131374.log.gz 2023-07-06T03:58:53.604 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169607.log 2023-07-06T03:58:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161123.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.161123.log.gz 2023-07-06T03:58:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172561.log 2023-07-06T03:58:53.607 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169607.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.169607.log.gz 2023-07-06T03:58:53.607 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172659.log 2023-07-06T03:58:53.608 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172561.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.172561.log.gz 2023-07-06T03:58:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138549.log 2023-07-06T03:58:53.610 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172659.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.172659.log.gz 2023-07-06T03:58:53.610 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118726.log 2023-07-06T03:58:53.611 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138549.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138549.log.gz 2023-07-06T03:58:53.612 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138451.log 2023-07-06T03:58:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118726.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118726.log.gz 2023-07-06T03:58:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125711.log 2023-07-06T03:58:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138451.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.138451.log.gz 2023-07-06T03:58:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153556.log 2023-07-06T03:58:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125711.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125711.log.gz 2023-07-06T03:58:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133101.log 2023-07-06T03:58:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153556.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.153556.log.gz 2023-07-06T03:58:53.618 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168365.log 2023-07-06T03:58:53.619 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133101.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.133101.log.gz 2023-07-06T03:58:53.620 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159628.log 2023-07-06T03:58:53.621 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.168365.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.168365.log.gz 2023-07-06T03:58:53.621 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152352.log 2023-07-06T03:58:53.622 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159628.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159628.log.gz 2023-07-06T03:58:53.623 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154455.log 2023-07-06T03:58:53.624 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152352.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152352.log.gz 2023-07-06T03:58:53.624 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147978.log 2023-07-06T03:58:53.626 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154455.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.154455.log.gz 2023-07-06T03:58:53.626 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131227.log 2023-07-06T03:58:53.627 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147978.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147978.log.gz 2023-07-06T03:58:53.627 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119835.log 2023-07-06T03:58:53.629 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131227.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.131227.log.gz 2023-07-06T03:58:53.629 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156579.log 2023-07-06T03:58:53.630 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119835.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119835.log.gz 2023-07-06T03:58:53.631 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-07-06T03:58:53.632 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156579.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.156579.log.gz 2023-07-06T03:58:53.632 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151457.log 2023-07-06T03:58:53.633 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159189.log 2023-07-06T03:58:53.635 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151457.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151457.log.gz 2023-07-06T03:58:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170104.log 2023-07-06T03:58:53.636 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159189.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.159189.log.gz 2023-07-06T03:58:53.637 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134568.log 2023-07-06T03:58:53.638 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170104.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.170104.log.gz 2023-07-06T03:58:53.639 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162347.log 2023-07-06T03:58:53.640 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.134568.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134568.log.gz 2023-07-06T03:58:53.640 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160829.log 2023-07-06T03:58:53.641 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162347.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.162347.log.gz 2023-07-06T03:58:53.642 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138103.log 2023-07-06T03:58:53.643 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160829.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160829.log.gz 2023-07-06T03:58:53.643 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164339.log 2023-07-06T03:58:53.645 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138103.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.138103.log.gz 2023-07-06T03:58:53.645 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132954.log 2023-07-06T03:58:53.646 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164339.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164339.log.gz 2023-07-06T03:58:53.647 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162092.log 2023-07-06T03:58:53.648 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132954.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.132954.log.gz 2023-07-06T03:58:53.649 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137346.log 2023-07-06T03:58:53.649 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162092.log.gz 2023-07-06T03:58:53.650 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141064.log 2023-07-06T03:58:53.652 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137346.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137346.log.gz 2023-07-06T03:58:53.653 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135104.log 2023-07-06T03:58:53.653 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.141064.log.gz 2023-07-06T03:58:53.654 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.149203.log 2023-07-06T03:58:53.656 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.135104.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135104.log.gz 2023-07-06T03:58:53.658 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.149203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122910.log 2023-07-06T03:58:53.658 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.0.149203.log.gz 2023-07-06T03:58:53.659 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138202.log 2023-07-06T03:58:53.661 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.122910.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122910.log.gz 2023-07-06T03:58:53.662 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138202.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.138202.log.gz 2023-07-06T03:58:53.663 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130489.log 2023-07-06T03:58:53.664 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111877.log 2023-07-06T03:58:53.666 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130489.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.130489.log.gz 2023-07-06T03:58:53.667 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.111877.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111877.log.gz 2023-07-06T03:58:53.668 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147131.log 2023-07-06T03:58:53.669 INFO:teuthology.orchestra.run.smithi146.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-07-06T03:58:53.670 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163763.log 2023-07-06T03:58:53.671 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147131.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147131.log.gz 2023-07-06T03:58:53.672 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163763.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163763.log.gz 2023-07-06T03:58:53.673 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167499.log 2023-07-06T03:58:53.674 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119216.log 2023-07-06T03:58:53.675 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.167499.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.167499.log.gz 2023-07-06T03:58:53.677 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119216.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119216.log.gz 2023-07-06T03:58:53.677 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118971.log 2023-07-06T03:58:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136905.log 2023-07-06T03:58:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118971.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118971.log.gz 2023-07-06T03:58:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136905.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.136905.log.gz 2023-07-06T03:58:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145380.log 2023-07-06T03:58:53.683 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124981.log 2023-07-06T03:58:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145380.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145380.log.gz 2023-07-06T03:58:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124981.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.124981.log.gz 2023-07-06T03:58:53.687 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171573.log 2023-07-06T03:58:53.688 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157243.log 2023-07-06T03:58:53.690 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171573.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.171573.log.gz 2023-07-06T03:58:53.691 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157243.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157243.log.gz 2023-07-06T03:58:53.692 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127259.log 2023-07-06T03:58:53.693 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159732.log 2023-07-06T03:58:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127259.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127259.log.gz 2023-07-06T03:58:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159732.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.159732.log.gz 2023-07-06T03:58:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172365.log 2023-07-06T03:58:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167279.log 2023-07-06T03:58:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172365.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.172365.log.gz 2023-07-06T03:58:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.167279.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.167279.log.gz 2023-07-06T03:58:53.701 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164438.log 2023-07-06T03:58:53.702 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152073.log 2023-07-06T03:58:53.704 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164438.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.164438.log.gz 2023-07-06T03:58:53.705 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152073.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152073.log.gz 2023-07-06T03:58:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141317.log 2023-07-06T03:58:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142643.log 2023-07-06T03:58:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141317.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141317.log.gz 2023-07-06T03:58:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142643.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142643.log.gz 2023-07-06T03:58:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147180.log 2023-07-06T03:58:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145735.log 2023-07-06T03:58:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147180.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147180.log.gz 2023-07-06T03:58:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145735.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.145735.log.gz 2023-07-06T03:58:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158290.log 2023-07-06T03:58:53.717 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172610.log 2023-07-06T03:58:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158290.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158290.log.gz 2023-07-06T03:58:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172610.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.172610.log.gz 2023-07-06T03:58:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156793.log 2023-07-06T03:58:53.721 INFO:teuthology.orchestra.run.smithi146.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-07-06T03:58:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162288.log 2023-07-06T03:58:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156793.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.156793.log.gz 2023-07-06T03:58:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162288.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162288.log.gz 2023-07-06T03:58:53.725 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140236.log 2023-07-06T03:58:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144177.log 2023-07-06T03:58:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140236.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140236.log.gz 2023-07-06T03:58:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144177.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144177.log.gz 2023-07-06T03:58:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120931.log 2023-07-06T03:58:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144373.log 2023-07-06T03:58:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120931.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120931.log.gz 2023-07-06T03:58:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119836.log 2023-07-06T03:58:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144373.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.144373.log.gz 2023-07-06T03:58:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144911.log 2023-07-06T03:58:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119836.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119836.log.gz 2023-07-06T03:58:53.737 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144911.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.144911.log.gz 2023-07-06T03:58:53.737 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117840.log 2023-07-06T03:58:53.739 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161954.log 2023-07-06T03:58:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.117840.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117840.log.gz 2023-07-06T03:58:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161954.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161954.log.gz 2023-07-06T03:58:53.742 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170446.log 2023-07-06T03:58:53.743 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120686.log 2023-07-06T03:58:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170446.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.170446.log.gz 2023-07-06T03:58:53.746 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120686.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.120686.log.gz 2023-07-06T03:58:53.747 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123836.log 2023-07-06T03:58:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158241.log 2023-07-06T03:58:53.750 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123836.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.123836.log.gz 2023-07-06T03:58:53.751 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158241.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158241.log.gz 2023-07-06T03:58:53.752 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150948.log 2023-07-06T03:58:53.753 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153311.log 2023-07-06T03:58:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150948.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150948.log.gz 2023-07-06T03:58:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153311.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.153311.log.gz 2023-07-06T03:58:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157412.log 2023-07-06T03:58:53.758 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135153.log 2023-07-06T03:58:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157412.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157412.log.gz 2023-07-06T03:58:53.761 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.135153.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135153.log.gz 2023-07-06T03:58:53.761 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143469.log 2023-07-06T03:58:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140529.log 2023-07-06T03:58:53.764 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143469.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143469.log.gz 2023-07-06T03:58:53.765 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140529.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.140529.log.gz 2023-07-06T03:58:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144256.log 2023-07-06T03:58:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157947.log 2023-07-06T03:58:53.769 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144256.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144256.log.gz 2023-07-06T03:58:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157947.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157947.log.gz 2023-07-06T03:58:53.771 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134957.log 2023-07-06T03:58:53.772 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145489.log 2023-07-06T03:58:53.774 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.134957.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134957.log.gz 2023-07-06T03:58:53.775 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145489.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145489.log.gz 2023-07-06T03:58:53.776 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146177.log 2023-07-06T03:58:53.777 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125880.log 2023-07-06T03:58:53.778 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146177.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146177.log.gz 2023-07-06T03:58:53.780 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125880.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.125880.log.gz 2023-07-06T03:58:53.780 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157796.log 2023-07-06T03:58:53.782 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156334.log 2023-07-06T03:58:53.783 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157796.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157796.log.gz 2023-07-06T03:58:53.784 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156334.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156334.log.gz 2023-07-06T03:58:53.785 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120441.log 2023-07-06T03:58:53.786 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127308.log 2023-07-06T03:58:53.788 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120441.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120441.log.gz 2023-07-06T03:58:53.789 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127308.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127308.log.gz 2023-07-06T03:58:53.790 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154651.log 2023-07-06T03:58:53.791 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149535.log 2023-07-06T03:58:53.793 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154651.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154651.log.gz 2023-07-06T03:58:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149535.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149535.log.gz 2023-07-06T03:58:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160343.log 2023-07-06T03:58:53.796 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166506.log 2023-07-06T03:58:53.797 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160343.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160343.log.gz 2023-07-06T03:58:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166506.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.166506.log.gz 2023-07-06T03:58:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129102.log 2023-07-06T03:58:53.800 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126328.log 2023-07-06T03:58:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.129102.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.129102.log.gz 2023-07-06T03:58:53.803 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126328.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126328.log.gz 2023-07-06T03:58:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139620.log 2023-07-06T03:58:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127986.log 2023-07-06T03:58:53.807 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139620.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139620.log.gz 2023-07-06T03:58:53.808 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127986.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.127986.log.gz 2023-07-06T03:58:53.808 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146475.log 2023-07-06T03:58:53.810 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119828.log 2023-07-06T03:58:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146475.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146475.log.gz 2023-07-06T03:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119828.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119828.log.gz 2023-07-06T03:58:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127161.log 2023-07-06T03:58:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121176.log 2023-07-06T03:58:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127161.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127161.log.gz 2023-07-06T03:58:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.121176.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121176.log.gz 2023-07-06T03:58:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160197.log 2023-07-06T03:58:53.819 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145434.log 2023-07-06T03:58:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160197.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160197.log.gz 2023-07-06T03:58:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145434.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.145434.log.gz 2023-07-06T03:58:53.823 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132079.log 2023-07-06T03:58:53.824 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133003.log 2023-07-06T03:58:53.826 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132079.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.132079.log.gz 2023-07-06T03:58:53.827 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133003.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133003.log.gz 2023-07-06T03:58:53.828 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126720.log 2023-07-06T03:58:53.828 INFO:teuthology.orchestra.run.smithi097.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.139442.log.gz 2023-07-06T03:58:53.829 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157145.log 2023-07-06T03:58:53.830 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126720.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126720.log.gz 2023-07-06T03:58:53.831 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167230.log 2023-07-06T03:58:53.832 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157145.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157145.log.gz 2023-07-06T03:58:53.832 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123738.log 2023-07-06T03:58:53.833 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.167230.log: gzip -5 --verbose -- 87.1%/var/log/ceph/ceph-client.admin.125662.log -- replaced with /var/log/ceph/ceph-client.admin.167230.log.gz 2023-07-06T03:58:53.834 INFO:teuthology.orchestra.run.smithi097.stderr: 2023-07-06T03:58:53.834 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123738.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123738.log.gz 2023-07-06T03:58:53.835 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124081.log 2023-07-06T03:58:53.836 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134221.log 2023-07-06T03:58:53.836 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125662.log.gz 2023-07-06T03:58:53.837 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124081.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124081.log.gz 2023-07-06T03:58:53.837 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119820.log 2023-07-06T03:58:53.840 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.134221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153776.log 2023-07-06T03:58:53.841 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134221.log.gz 2023-07-06T03:58:53.841 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119820.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119820.log.gz 2023-07-06T03:58:53.841 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150385.log 2023-07-06T03:58:53.842 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159336.log 2023-07-06T03:58:53.842 INFO:teuthology.orchestra.run.smithi097.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.153776.log.gz 2023-07-06T03:58:53.844 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161025.log 2023-07-06T03:58:53.844 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.150385.log.gz 2023-07-06T03:58:53.844 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159336.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159336.log.gz 2023-07-06T03:58:53.845 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155159.log 2023-07-06T03:58:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119069.log 2023-07-06T03:58:53.847 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.161025.log.gz 2023-07-06T03:58:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155159.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.155159.log.gz 2023-07-06T03:58:53.848 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133395.log 2023-07-06T03:58:53.849 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171342.log 2023-07-06T03:58:53.850 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119069.log.gz 2023-07-06T03:58:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133395.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133395.log.gz 2023-07-06T03:58:53.851 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155579.log 2023-07-06T03:58:53.852 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145932.log 2023-07-06T03:58:53.852 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171342.log.gz 2023-07-06T03:58:53.852 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155579.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155579.log.gz 2023-07-06T03:58:53.853 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138598.log 2023-07-06T03:58:53.854 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165463.log 2023-07-06T03:58:53.855 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145932.log.gz 2023-07-06T03:58:53.855 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138598.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.138598.log.gz 2023-07-06T03:58:53.856 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169289.log 2023-07-06T03:58:53.857 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121029.log 2023-07-06T03:58:53.858 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165463.log.gz 2023-07-06T03:58:53.858 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169289.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169289.log.gz 2023-07-06T03:58:53.858 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145137.log 2023-07-06T03:58:53.860 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.121029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120219.log 2023-07-06T03:58:53.860 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121029.log.gz 2023-07-06T03:58:53.861 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145137.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145137.log.gz 2023-07-06T03:58:53.861 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163812.log 2023-07-06T03:58:53.863 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151702.log 2023-07-06T03:58:53.863 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120219.log.gz 2023-07-06T03:58:53.864 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163812.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163812.log.gz 2023-07-06T03:58:53.865 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151702.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151702.log.gz 2023-07-06T03:58:53.865 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158045.log 2023-07-06T03:58:53.866 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156949.log 2023-07-06T03:58:53.867 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158045.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.158045.log.gz 2023-07-06T03:58:53.867 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142545.log 2023-07-06T03:58:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120226.log 2023-07-06T03:58:53.870 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156949.log.gz 2023-07-06T03:58:53.870 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142545.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142545.log.gz 2023-07-06T03:58:53.870 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123101.log 2023-07-06T03:58:53.871 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165026.log 2023-07-06T03:58:53.872 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120226.log.gz 2023-07-06T03:58:53.873 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123101.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123101.log.gz 2023-07-06T03:58:53.873 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172414.log 2023-07-06T03:58:53.874 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165026.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165026.log.gz 2023-07-06T03:58:53.874 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124130.log 2023-07-06T03:58:53.875 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172414.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.172414.log.gz 2023-07-06T03:58:53.876 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164634.log 2023-07-06T03:58:53.877 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164487.log 2023-07-06T03:58:53.877 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124130.log.gz 2023-07-06T03:58:53.878 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164634.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164634.log.gz 2023-07-06T03:58:53.879 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130693.log 2023-07-06T03:58:53.880 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.134043.log 2023-07-06T03:58:53.881 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164487.log.gz 2023-07-06T03:58:53.881 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130693.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130693.log.gz 2023-07-06T03:58:53.881 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165904.log 2023-07-06T03:58:53.883 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.134043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139967.log 2023-07-06T03:58:53.884 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151898.log 2023-07-06T03:58:53.885 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.165904.log.gz 2023-07-06T03:58:53.885 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139967.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.139967.log.gz 2023-07-06T03:58:53.886 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145883.log 2023-07-06T03:58:53.887 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136371.log 2023-07-06T03:58:53.888 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.151898.log.gz 2023-07-06T03:58:53.888 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145883.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145883.log.gz 2023-07-06T03:58:53.889 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146573.log 2023-07-06T03:58:53.890 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136371.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136371.log.gz 2023-07-06T03:58:53.891 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156481.log 2023-07-06T03:58:53.891 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146573.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.146573.log.gz 2023-07-06T03:58:53.892 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161735.log 2023-07-06T03:58:53.893 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160490.log 2023-07-06T03:58:53.893 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156481.log.gz 2023-07-06T03:58:53.894 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161735.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.161735.log.gz 2023-07-06T03:58:53.894 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136322.log 2023-07-06T03:58:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170397.log 2023-07-06T03:58:53.896 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160490.log.gz 2023-07-06T03:58:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136322.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.136322.log.gz 2023-07-06T03:58:53.897 INFO:teuthology.orchestra.run.smithi097.stderr: 95.9% -- replaced with /var/log/ceph/ceph-client.0.143292.log.gz 2023-07-06T03:58:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130742.log 2023-07-06T03:58:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147425.log 2023-07-06T03:58:53.899 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170397.log.gz 2023-07-06T03:58:53.900 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130742.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130742.log.gz 2023-07-06T03:58:53.900 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147880.log 2023-07-06T03:58:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148555.log 2023-07-06T03:58:53.902 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147425.log.gz 2023-07-06T03:58:53.903 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147880.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.147880.log.gz 2023-07-06T03:58:53.903 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133664.log 2023-07-06T03:58:53.905 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162190.log 2023-07-06T03:58:53.905 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148555.log.gz 2023-07-06T03:58:53.906 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133664.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133664.log.gz 2023-07-06T03:58:53.906 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170981.log 2023-07-06T03:58:53.907 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161221.log 2023-07-06T03:58:53.907 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162190.log.gz 2023-07-06T03:58:53.908 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170981.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170981.log.gz 2023-07-06T03:58:53.908 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171826.log 2023-07-06T03:58:53.909 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161221.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.161221.log.gz 2023-07-06T03:58:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135006.log 2023-07-06T03:58:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171826.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.171826.log.gz 2023-07-06T03:58:53.912 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137052.log 2023-07-06T03:58:53.912 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.135006.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135006.log.gz 2023-07-06T03:58:53.913 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135055.log 2023-07-06T03:58:53.914 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137052.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137052.log.gz 2023-07-06T03:58:53.914 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138349.log 2023-07-06T03:58:53.916 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.135055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133297.log 2023-07-06T03:58:53.917 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135055.log.gz 2023-07-06T03:58:53.917 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138349.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.138349.log.gz 2023-07-06T03:58:53.917 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142445.log 2023-07-06T03:58:53.918 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146128.log 2023-07-06T03:58:53.918 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.133297.log.gz 2023-07-06T03:58:53.919 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142445.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142445.log.gz 2023-07-06T03:58:53.920 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126377.log 2023-07-06T03:58:53.922 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161074.log 2023-07-06T03:58:53.922 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146128.log.gz 2023-07-06T03:58:53.922 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126377.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126377.log.gz 2023-07-06T03:58:53.923 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144482.log 2023-07-06T03:58:53.924 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165316.log 2023-07-06T03:58:53.924 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161074.log.gz 2023-07-06T03:58:53.925 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144482.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.144482.log.gz 2023-07-06T03:58:53.925 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150801.log 2023-07-06T03:58:53.926 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165316.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.165316.log.gz 2023-07-06T03:58:53.927 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120490.log 2023-07-06T03:58:53.928 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150801.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.150801.log.gz 2023-07-06T03:58:53.928 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171275.log 2023-07-06T03:58:53.929 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120490.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120490.log.gz 2023-07-06T03:58:53.930 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124835.log 2023-07-06T03:58:53.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151849.log 2023-07-06T03:58:53.932 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.171275.log.gz 2023-07-06T03:58:53.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124835.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124835.log.gz 2023-07-06T03:58:53.933 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132583.log 2023-07-06T03:58:53.934 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.135663.log 2023-07-06T03:58:53.934 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151849.log.gz 2023-07-06T03:58:53.935 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132583.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.132583.log.gz 2023-07-06T03:58:53.935 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138400.log 2023-07-06T03:58:53.937 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.135663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153605.log 2023-07-06T03:58:53.938 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138400.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138400.log.gz 2023-07-06T03:58:53.938 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164055.log 2023-07-06T03:58:53.940 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153605.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153605.log.gz 2023-07-06T03:58:53.940 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154978.log 2023-07-06T03:58:53.941 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164055.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.164055.log.gz 2023-07-06T03:58:53.942 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168218.log 2023-07-06T03:58:53.943 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154978.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154978.log.gz 2023-07-06T03:58:53.943 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128955.log 2023-07-06T03:58:53.945 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.168218.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.168218.log.gz 2023-07-06T03:58:53.945 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137649.log 2023-07-06T03:58:53.947 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128955.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128955.log.gz 2023-07-06T03:58:53.947 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133542.log 2023-07-06T03:58:53.948 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137649.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137649.log.gz 2023-07-06T03:58:53.949 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141870.log 2023-07-06T03:58:53.950 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133542.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.133542.log.gz 2023-07-06T03:58:53.950 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126867.log 2023-07-06T03:58:53.952 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141870.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141870.log.gz 2023-07-06T03:58:53.952 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171441.log 2023-07-06T03:58:53.953 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126867.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.126867.log.gz 2023-07-06T03:58:53.953 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133199.log 2023-07-06T03:58:53.955 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171441.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.171441.log.gz 2023-07-06T03:58:53.956 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157648.log 2023-07-06T03:58:53.959 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133199.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133199.log.gz 2023-07-06T03:58:53.959 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133150.log 2023-07-06T03:58:53.959 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142164.log 2023-07-06T03:58:53.959 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.157648.log.gz 2023-07-06T03:58:53.961 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133150.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.133150.log.gz 2023-07-06T03:58:53.962 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153213.log 2023-07-06T03:58:53.962 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157698.log 2023-07-06T03:58:53.962 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142164.log.gz 2023-07-06T03:58:53.962 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153213.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153213.log.gz 2023-07-06T03:58:53.963 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122959.log 2023-07-06T03:58:53.964 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157698.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157698.log.gz 2023-07-06T03:58:53.965 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145981.log 2023-07-06T03:58:53.966 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.122959.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122959.log.gz 2023-07-06T03:58:53.966 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171392.log 2023-07-06T03:58:53.968 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166051.log 2023-07-06T03:58:53.968 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145981.log.gz 2023-07-06T03:58:53.969 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171392.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171392.log.gz 2023-07-06T03:58:53.969 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125128.log 2023-07-06T03:58:53.971 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160682.log 2023-07-06T03:58:53.971 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.166051.log.gz 2023-07-06T03:58:53.972 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125128.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125128.log.gz 2023-07-06T03:58:53.972 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158559.log 2023-07-06T03:58:53.974 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127740.log 2023-07-06T03:58:53.974 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160682.log.gz 2023-07-06T03:58:53.974 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158559.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.158559.log.gz 2023-07-06T03:58:53.975 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146332.log 2023-07-06T03:58:53.976 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130587.log 2023-07-06T03:58:53.977 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127740.log.gz 2023-07-06T03:58:53.977 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146332.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.146332.log.gz 2023-07-06T03:58:53.978 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131932.log 2023-07-06T03:58:53.979 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136954.log 2023-07-06T03:58:53.980 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130587.log.gz 2023-07-06T03:58:53.980 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131932.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.131932.log.gz 2023-07-06T03:58:53.981 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156726.log 2023-07-06T03:58:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136954.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.136954.log.gz 2023-07-06T03:58:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137493.log 2023-07-06T03:58:53.984 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159434.log 2023-07-06T03:58:53.985 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156726.log.gz 2023-07-06T03:58:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137493.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.137493.log.gz 2023-07-06T03:58:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158143.log 2023-07-06T03:58:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126965.log 2023-07-06T03:58:53.987 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159434.log.gz 2023-07-06T03:58:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158143.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158143.log.gz 2023-07-06T03:58:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118224.log 2023-07-06T03:58:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141170.log 2023-07-06T03:58:53.990 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126965.log.gz 2023-07-06T03:58:53.991 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118224.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118224.log.gz 2023-07-06T03:58:53.991 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118473.log 2023-07-06T03:58:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170932.log 2023-07-06T03:58:53.993 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.141170.log.gz 2023-07-06T03:58:53.994 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118473.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.118473.log.gz 2023-07-06T03:58:53.994 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167426.log 2023-07-06T03:58:53.996 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147278.log 2023-07-06T03:58:53.996 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.170932.log.gz 2023-07-06T03:58:53.997 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.167426.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.167426.log.gz 2023-07-06T03:58:53.997 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150899.log 2023-07-06T03:58:53.998 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147278.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.147278.log.gz 2023-07-06T03:58:53.999 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170544.log 2023-07-06T03:58:54.000 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150899.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.150899.log.gz 2023-07-06T03:58:54.000 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119443.log 2023-07-06T03:58:54.001 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170544.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.170544.log.gz 2023-07-06T03:58:54.002 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123444.log 2023-07-06T03:58:54.003 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126035.log 2023-07-06T03:58:54.005 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123444.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119443.log.gz 2023-07-06T03:58:54.005 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123444.log.gz 2023-07-06T03:58:54.005 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.168891.log 2023-07-06T03:58:54.006 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126035.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126035.log.gz 2023-07-06T03:58:54.006 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153458.log 2023-07-06T03:58:54.009 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.168891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159238.log 2023-07-06T03:58:54.010 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153458.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153458.log.gz 2023-07-06T03:58:54.010 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158339.log 2023-07-06T03:58:54.011 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.0.168891.log.gz 2023-07-06T03:58:54.011 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159238.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159238.log.gz 2023-07-06T03:58:54.012 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126426.log 2023-07-06T03:58:54.013 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158339.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158339.log.gz 2023-07-06T03:58:54.014 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172912.log 2023-07-06T03:58:54.014 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126426.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.126426.log.gz 2023-07-06T03:58:54.015 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140333.log 2023-07-06T03:58:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172912.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172912.log.gz 2023-07-06T03:58:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125422.log 2023-07-06T03:58:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140333.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140333.log.gz 2023-07-06T03:58:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164928.log 2023-07-06T03:58:54.019 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125422.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125422.log.gz 2023-07-06T03:58:54.020 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119429.log 2023-07-06T03:58:54.021 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149682.log 2023-07-06T03:58:54.022 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119429.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164928.log.gz 2023-07-06T03:58:54.023 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119429.log.gz 2023-07-06T03:58:54.023 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156383.log 2023-07-06T03:58:54.024 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149682.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.149682.log.gz 2023-07-06T03:58:54.024 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128108.log 2023-07-06T03:58:54.026 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164977.log 2023-07-06T03:58:54.026 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156383.log.gz 2023-07-06T03:58:54.027 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128108.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128108.log.gz 2023-07-06T03:58:54.027 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119445.log 2023-07-06T03:58:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128206.log 2023-07-06T03:58:54.029 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.164977.log.gz 2023-07-06T03:58:54.029 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119445.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119445.log.gz 2023-07-06T03:58:54.030 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160731.log 2023-07-06T03:58:54.032 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120980.log 2023-07-06T03:58:54.033 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128206.log.gz 2023-07-06T03:58:54.033 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160731.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160731.log.gz 2023-07-06T03:58:54.033 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138794.log 2023-07-06T03:58:54.034 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120980.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120980.log.gz 2023-07-06T03:58:54.034 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152122.log 2023-07-06T03:58:54.036 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138794.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.138794.log.gz 2023-07-06T03:58:54.036 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112347.log 2023-07-06T03:58:54.037 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152122.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152122.log.gz 2023-07-06T03:58:54.038 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169362.log 2023-07-06T03:58:54.039 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.112347.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112347.log.gz 2023-07-06T03:58:54.040 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146079.log 2023-07-06T03:58:54.041 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169362.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.169362.log.gz 2023-07-06T03:58:54.041 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117698.log 2023-07-06T03:58:54.044 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146079.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.146079.log.gz 2023-07-06T03:58:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164111.log 2023-07-06T03:58:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.117698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.167878.log 2023-07-06T03:58:54.045 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117698.log.gz 2023-07-06T03:58:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164111.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.164111.log.gz 2023-07-06T03:58:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159951.log 2023-07-06T03:58:54.047 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.167878.log: 89.0% -- replaced with /var/log/ceph/ceph-client.0.167878.log.gz 2023-07-06T03:58:54.048 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151947.log 2023-07-06T03:58:54.049 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159951.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159951.log.gz 2023-07-06T03:58:54.049 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132177.log 2023-07-06T03:58:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151947.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.151947.log.gz 2023-07-06T03:58:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132805.log 2023-07-06T03:58:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132177.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.132177.log.gz 2023-07-06T03:58:54.055 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172267.log 2023-07-06T03:58:54.055 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153409.log 2023-07-06T03:58:54.055 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.132805.log.gz 2023-07-06T03:58:54.056 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172267.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.172267.log.gz 2023-07-06T03:58:54.056 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119825.log 2023-07-06T03:58:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153409.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153409.log.gz 2023-07-06T03:58:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131035.log 2023-07-06T03:58:54.059 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119825.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119825.log.gz 2023-07-06T03:58:54.059 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152472.log 2023-07-06T03:58:54.060 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131035.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131035.log.gz 2023-07-06T03:58:54.061 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138941.log 2023-07-06T03:58:54.064 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152472.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152472.log.gz 2023-07-06T03:58:54.064 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148506.log 2023-07-06T03:58:54.064 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129732.log 2023-07-06T03:58:54.065 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148506.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.138941.log.gz 2023-07-06T03:58:54.065 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148506.log.gz 2023-07-06T03:58:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118873.log 2023-07-06T03:58:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.129732.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.129732.log.gz 2023-07-06T03:58:54.067 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.158938.log 2023-07-06T03:58:54.069 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118873.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118873.log.gz 2023-07-06T03:58:54.069 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124762.log 2023-07-06T03:58:54.070 INFO:teuthology.orchestra.run.smithi146.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-07-06T03:58:54.072 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.158938.log: /var/log/ceph/ceph-client.admin.124762.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124762.log.gz 2023-07-06T03:58:54.072 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125226.log 2023-07-06T03:58:54.075 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125226.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125226.log.gz 2023-07-06T03:58:54.075 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129053.log 2023-07-06T03:58:54.078 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.129053.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129053.log.gz 2023-07-06T03:58:54.078 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141415.log 2023-07-06T03:58:54.081 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141415.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141415.log.gz 2023-07-06T03:58:54.082 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160633.log 2023-07-06T03:58:54.084 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160633.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.160633.log.gz 2023-07-06T03:58:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172512.log 2023-07-06T03:58:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134617.log 2023-07-06T03:58:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172512.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.172512.log.gz 2023-07-06T03:58:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.134617.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.134617.log.gz 2023-07-06T03:58:54.090 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153007.log 2023-07-06T03:58:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153007.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153007.log.gz 2023-07-06T03:58:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125471.log 2023-07-06T03:58:54.096 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125471.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125471.log.gz 2023-07-06T03:58:54.096 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166702.log 2023-07-06T03:58:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166702.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166702.log.gz 2023-07-06T03:58:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141919.log 2023-07-06T03:58:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141919.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141919.log.gz 2023-07-06T03:58:54.103 INFO:teuthology.orchestra.run.smithi097.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.0.134043.log.gz 2023-07-06T03:58:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151094.log 2023-07-06T03:58:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-07-06T03:58:54.105 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151094.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151094.log.gz 2023-07-06T03:58:54.106 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-07-06T03:58:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118625.log 2023-07-06T03:58:54.110 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.118625.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118625.log.gz 2023-07-06T03:58:54.111 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146226.log 2023-07-06T03:58:54.112 INFO:teuthology.orchestra.run.smithi146.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-07-06T03:58:54.115 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160441.log 2023-07-06T03:58:54.115 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146226.log.gz 2023-07-06T03:58:54.119 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153262.log 2023-07-06T03:58:54.119 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160441.log.gz 2023-07-06T03:58:54.122 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148653.log 2023-07-06T03:58:54.123 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153262.log.gz 2023-07-06T03:58:54.126 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126084.log 2023-07-06T03:58:54.126 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148653.log.gz 2023-07-06T03:58:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142594.log 2023-07-06T03:58:54.130 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.126084.log.gz 2023-07-06T03:58:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142496.log 2023-07-06T03:58:54.134 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142594.log.gz 2023-07-06T03:58:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164781.log 2023-07-06T03:58:54.138 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142496.log.gz 2023-07-06T03:58:54.141 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152807.log 2023-07-06T03:58:54.142 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164781.log.gz 2023-07-06T03:58:54.145 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126230.log 2023-07-06T03:58:54.145 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152807.log.gz 2023-07-06T03:58:54.147 INFO:teuthology.orchestra.run.smithi146.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-07-06T03:58:54.149 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154504.log 2023-07-06T03:58:54.149 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126230.log.gz 2023-07-06T03:58:54.153 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127603.log 2023-07-06T03:58:54.153 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154504.log.gz 2023-07-06T03:58:54.157 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154700.log 2023-07-06T03:58:54.157 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127603.log.gz 2023-07-06T03:58:54.161 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162790.log 2023-07-06T03:58:54.161 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.154700.log.gz 2023-07-06T03:58:54.165 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137150.log 2023-07-06T03:58:54.165 INFO:teuthology.orchestra.run.smithi097.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.158938.log.gz 2023-07-06T03:58:54.165 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162790.log.gz 2023-07-06T03:58:54.167 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137150.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137150.log.gz 2023-07-06T03:58:54.168 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140089.log 2023-07-06T03:58:54.170 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140089.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140089.log.gz 2023-07-06T03:58:54.171 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134788.log 2023-07-06T03:58:54.174 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.134788.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134788.log.gz 2023-07-06T03:58:54.174 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157747.log 2023-07-06T03:58:54.177 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157747.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157747.log.gz 2023-07-06T03:58:54.177 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153162.log 2023-07-06T03:58:54.180 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153162.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153162.log.gz 2023-07-06T03:58:54.181 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136224.log 2023-07-06T03:58:54.183 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136224.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.136224.log.gz 2023-07-06T03:58:54.184 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152905.log 2023-07-06T03:58:54.187 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152905.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152905.log.gz 2023-07-06T03:58:54.187 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145210.log 2023-07-06T03:58:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166149.log 2023-07-06T03:58:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145210.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145210.log.gz 2023-07-06T03:58:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166149.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.166149.log.gz 2023-07-06T03:58:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166751.log 2023-07-06T03:58:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166751.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.166751.log.gz 2023-07-06T03:58:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171079.log 2023-07-06T03:58:54.198 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171079.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.171079.log.gz 2023-07-06T03:58:54.199 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151800.log 2023-07-06T03:58:54.201 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151800.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.151800.log.gz 2023-07-06T03:58:54.202 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126769.log 2023-07-06T03:58:54.204 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169240.log 2023-07-06T03:58:54.205 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126769.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126769.log.gz 2023-07-06T03:58:54.206 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169240.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169240.log.gz 2023-07-06T03:58:54.207 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131619.log 2023-07-06T03:58:54.210 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131619.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131619.log.gz 2023-07-06T03:58:54.210 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138647.log 2023-07-06T03:58:54.211 INFO:teuthology.orchestra.run.smithi097.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.0.135663.log.gz 2023-07-06T03:58:54.212 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153507.log 2023-07-06T03:58:54.213 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138647.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138647.log.gz 2023-07-06T03:58:54.214 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-07-06T03:58:54.215 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153507.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153507.log.gz 2023-07-06T03:58:54.219 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128710.log 2023-07-06T03:58:54.221 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118922.log 2023-07-06T03:58:54.224 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128710.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.128710.log.gz 2023-07-06T03:58:54.224 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118922.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.118922.log.gz 2023-07-06T03:58:54.225 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146720.log 2023-07-06T03:58:54.227 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140480.log 2023-07-06T03:58:54.227 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146720.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146720.log.gz 2023-07-06T03:58:54.230 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140480.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140480.log.gz 2023-07-06T03:58:54.230 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133444.log 2023-07-06T03:58:54.232 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118522.log 2023-07-06T03:58:54.233 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133444.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.133444.log.gz 2023-07-06T03:58:54.234 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148308.log 2023-07-06T03:58:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118522.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118522.log.gz 2023-07-06T03:58:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157511.log 2023-07-06T03:58:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148308.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148308.log.gz 2023-07-06T03:58:54.238 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157511.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.157511.log.gz 2023-07-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120228.log 2023-07-06T03:58:54.241 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120228.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120228.log.gz 2023-07-06T03:58:54.242 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125030.log 2023-07-06T03:58:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125030.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.125030.log.gz 2023-07-06T03:58:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150752.log 2023-07-06T03:58:54.248 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167328.log 2023-07-06T03:58:54.249 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170495.log 2023-07-06T03:58:54.249 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.150752.log.gz 2023-07-06T03:58:54.252 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.167328.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.167328.log.gz 2023-07-06T03:58:54.252 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120231.log 2023-07-06T03:58:54.252 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170495.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.170495.log.gz 2023-07-06T03:58:54.253 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120231.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120231.log.gz 2023-07-06T03:58:54.254 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147327.log 2023-07-06T03:58:54.257 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147327.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.147327.log.gz 2023-07-06T03:58:54.257 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128463.log 2023-07-06T03:58:54.260 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128463.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128463.log.gz 2023-07-06T03:58:54.261 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141366.log 2023-07-06T03:58:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141366.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141366.log.gz 2023-07-06T03:58:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156020.log 2023-07-06T03:58:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156020.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.156020.log.gz 2023-07-06T03:58:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165120.log 2023-07-06T03:58:54.270 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165120.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165120.log.gz 2023-07-06T03:58:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123248.log 2023-07-06T03:58:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170300.log 2023-07-06T03:58:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123248.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123248.log.gz 2023-07-06T03:58:54.275 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170300.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.170300.log.gz 2023-07-06T03:58:54.276 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.150207.log 2023-07-06T03:58:54.285 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.150207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168511.log 2023-07-06T03:58:54.289 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.168511.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.168511.log.gz 2023-07-06T03:58:54.289 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165218.log 2023-07-06T03:58:54.293 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152856.log 2023-07-06T03:58:54.293 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.165218.log.gz 2023-07-06T03:58:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137444.log 2023-07-06T03:58:54.297 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152856.log.gz 2023-07-06T03:58:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123199.log 2023-07-06T03:58:54.301 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.137444.log.gz 2023-07-06T03:58:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166100.log 2023-07-06T03:58:54.305 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.123199.log.gz 2023-07-06T03:58:54.308 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140672.log 2023-07-06T03:58:54.308 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166100.log.gz 2023-07-06T03:58:54.312 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123346.log 2023-07-06T03:58:54.312 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.140672.log.gz 2023-07-06T03:58:54.316 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153113.log 2023-07-06T03:58:54.316 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123346.log.gz 2023-07-06T03:58:54.319 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122186.log 2023-07-06T03:58:54.320 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153113.log.gz 2023-07-06T03:58:54.324 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.122186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168064.log 2023-07-06T03:58:54.324 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122186.log.gz 2023-07-06T03:58:54.328 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.168064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123591.log 2023-07-06T03:58:54.328 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.168064.log.gz 2023-07-06T03:58:54.331 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169411.log 2023-07-06T03:58:54.332 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123591.log.gz 2023-07-06T03:58:54.335 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123297.log 2023-07-06T03:58:54.336 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.169411.log.gz 2023-07-06T03:58:54.339 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122054.log 2023-07-06T03:58:54.340 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123297.log.gz 2023-07-06T03:58:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.122054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164879.log 2023-07-06T03:58:54.344 INFO:teuthology.orchestra.run.smithi097.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.122054.log.gz 2023-07-06T03:58:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164879.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-06T03:58:54.348 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164879.log.gz 2023-07-06T03:58:54.353 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123885.log 2023-07-06T03:58:54.354 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-06T03:58:54.358 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118824.log 2023-07-06T03:58:54.358 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123885.log.gz 2023-07-06T03:58:54.362 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167059.log 2023-07-06T03:58:54.362 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118824.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118824.log.gz 2023-07-06T03:58:54.365 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.167059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167132.log 2023-07-06T03:58:54.366 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.167059.log.gz 2023-07-06T03:58:54.369 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.167132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127790.log 2023-07-06T03:58:54.370 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.167132.log.gz 2023-07-06T03:58:54.373 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124566.log 2023-07-06T03:58:54.373 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127790.log.gz 2023-07-06T03:58:54.377 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132905.log 2023-07-06T03:58:54.377 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124566.log.gz 2023-07-06T03:58:54.381 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160878.log 2023-07-06T03:58:54.381 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132905.log.gz 2023-07-06T03:58:54.385 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120230.log 2023-07-06T03:58:54.385 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.160878.log.gz 2023-07-06T03:58:54.389 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160392.log 2023-07-06T03:58:54.389 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120230.log.gz 2023-07-06T03:58:54.392 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132730.log 2023-07-06T03:58:54.393 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160392.log.gz 2023-07-06T03:58:54.396 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131084.log 2023-07-06T03:58:54.397 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.132730.log.gz 2023-07-06T03:58:54.400 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118273.log 2023-07-06T03:58:54.401 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.131084.log.gz 2023-07-06T03:58:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154406.log 2023-07-06T03:58:54.404 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118273.log.gz 2023-07-06T03:58:54.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128661.log 2023-07-06T03:58:54.408 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154406.log.gz 2023-07-06T03:58:54.412 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163010.log 2023-07-06T03:58:54.412 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128661.log.gz 2023-07-06T03:58:54.416 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137395.log 2023-07-06T03:58:54.416 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163010.log.gz 2023-07-06T03:58:54.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132856.log 2023-07-06T03:58:54.420 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137395.log.gz 2023-07-06T03:58:54.423 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167181.log 2023-07-06T03:58:54.424 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.132856.log.gz 2023-07-06T03:58:54.427 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.167181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171226.log 2023-07-06T03:58:54.428 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.167181.log.gz 2023-07-06T03:58:54.431 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164536.log 2023-07-06T03:58:54.431 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.171226.log.gz 2023-07-06T03:58:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124397.log 2023-07-06T03:58:54.435 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164536.log.gz 2023-07-06T03:58:54.439 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125809.log 2023-07-06T03:58:54.439 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124397.log.gz 2023-07-06T03:58:54.443 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130644.log 2023-07-06T03:58:54.443 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125809.log.gz 2023-07-06T03:58:54.446 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.130644.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130644.log.gz 2023-07-06T03:58:54.446 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165953.log 2023-07-06T03:58:54.449 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165953.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165953.log.gz 2023-07-06T03:58:54.450 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165610.log 2023-07-06T03:58:54.454 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147742.log 2023-07-06T03:58:54.454 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165610.log.gz 2023-07-06T03:58:54.458 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162594.log 2023-07-06T03:58:54.458 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147742.log.gz 2023-07-06T03:58:54.461 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156285.log 2023-07-06T03:58:54.462 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162594.log.gz 2023-07-06T03:58:54.465 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131178.log 2023-07-06T03:58:54.465 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156285.log.gz 2023-07-06T03:58:54.469 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138015.log 2023-07-06T03:58:54.469 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131178.log.gz 2023-07-06T03:58:54.472 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138015.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138015.log.gz 2023-07-06T03:58:54.473 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146818.log 2023-07-06T03:58:54.476 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134908.log 2023-07-06T03:58:54.476 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146818.log.gz 2023-07-06T03:58:54.480 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143616.log 2023-07-06T03:58:54.480 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.134908.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134908.log.gz 2023-07-06T03:58:54.482 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143616.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.143616.log.gz 2023-07-06T03:58:54.483 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166910.log 2023-07-06T03:58:54.486 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166910.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.166910.log.gz 2023-07-06T03:58:54.487 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148178.log 2023-07-06T03:58:54.491 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159678.log 2023-07-06T03:58:54.491 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.148178.log.gz 2023-07-06T03:58:54.494 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119435.log 2023-07-06T03:58:54.495 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159678.log.gz 2023-07-06T03:58:54.497 INFO:teuthology.orchestra.run.smithi146.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-07-06T03:58:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131668.log 2023-07-06T03:58:54.499 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119435.log.gz 2023-07-06T03:58:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165512.log 2023-07-06T03:58:54.502 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.131668.log.gz 2023-07-06T03:58:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141268.log 2023-07-06T03:58:54.506 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.165512.log.gz 2023-07-06T03:58:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171030.log 2023-07-06T03:58:54.510 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141268.log.gz 2023-07-06T03:58:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159782.log 2023-07-06T03:58:54.514 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.171030.log.gz 2023-07-06T03:58:54.518 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168267.log 2023-07-06T03:58:54.518 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159782.log.gz 2023-07-06T03:58:54.522 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.168267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161417.log 2023-07-06T03:58:54.522 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.168267.log.gz 2023-07-06T03:58:54.525 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146769.log 2023-07-06T03:58:54.526 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161417.log.gz 2023-07-06T03:58:54.529 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128857.log 2023-07-06T03:58:54.530 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146769.log.gz 2023-07-06T03:58:54.533 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120735.log 2023-07-06T03:58:54.533 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128857.log.gz 2023-07-06T03:58:54.537 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123934.log 2023-07-06T03:58:54.537 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120735.log.gz 2023-07-06T03:58:54.538 INFO:teuthology.orchestra.run.smithi097.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-07-06T03:58:54.540 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.123934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118322.log 2023-07-06T03:58:54.541 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123934.log.gz 2023-07-06T03:58:54.543 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118322.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118322.log.gz 2023-07-06T03:58:54.544 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152254.log 2023-07-06T03:58:54.546 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152254.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.152254.log.gz 2023-07-06T03:58:54.547 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144833.log 2023-07-06T03:58:54.549 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144833.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144833.log.gz 2023-07-06T03:58:54.550 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158388.log 2023-07-06T03:58:54.553 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158388.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158388.log.gz 2023-07-06T03:58:54.553 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170736.log 2023-07-06T03:58:54.556 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170736.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.170736.log.gz 2023-07-06T03:58:54.557 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144427.log 2023-07-06T03:58:54.559 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144427.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.144427.log.gz 2023-07-06T03:58:54.560 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157096.log 2023-07-06T03:58:54.562 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157096.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157096.log.gz 2023-07-06T03:58:54.563 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161613.log 2023-07-06T03:58:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161613.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161613.log.gz 2023-07-06T03:58:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144310.log 2023-07-06T03:58:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151555.log 2023-07-06T03:58:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144310.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.144310.log.gz 2023-07-06T03:58:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151555.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.151555.log.gz 2023-07-06T03:58:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151216.log 2023-07-06T03:58:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151216.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151216.log.gz 2023-07-06T03:58:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139063.log 2023-07-06T03:58:54.577 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139063.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139063.log.gz 2023-07-06T03:58:54.578 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138251.log 2023-07-06T03:58:54.580 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138251.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.138251.log.gz 2023-07-06T03:58:54.581 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169884.log 2023-07-06T03:58:54.584 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169884.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.169884.log.gz 2023-07-06T03:58:54.584 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143714.log 2023-07-06T03:58:54.587 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143714.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143714.log.gz 2023-07-06T03:58:54.587 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125177.log 2023-07-06T03:58:54.590 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125177.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125177.log.gz 2023-07-06T03:58:54.591 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127014.log 2023-07-06T03:58:54.593 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.127014.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127014.log.gz 2023-07-06T03:58:54.594 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166604.log 2023-07-06T03:58:54.597 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166604.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166604.log.gz 2023-07-06T03:58:54.597 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160780.log 2023-07-06T03:58:54.600 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160780.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.160780.log.gz 2023-07-06T03:58:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151265.log 2023-07-06T03:58:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151265.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151265.log.gz 2023-07-06T03:58:54.604 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163567.log 2023-07-06T03:58:54.607 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163567.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163567.log.gz 2023-07-06T03:58:54.607 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124446.log 2023-07-06T03:58:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.124446.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124446.log.gz 2023-07-06T03:58:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.105361.log 2023-07-06T03:58:54.613 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.tmp-client.admin.105361.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.105361.log.gz 2023-07-06T03:58:54.613 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126818.log 2023-07-06T03:58:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126818.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.126818.log.gz 2023-07-06T03:58:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141731.log 2023-07-06T03:58:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141731.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.141731.log.gz 2023-07-06T03:58:54.620 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172757.log 2023-07-06T03:58:54.620 INFO:teuthology.orchestra.run.smithi097.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-07-06T03:58:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168316.log 2023-07-06T03:58:54.623 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172757.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.172757.log.gz 2023-07-06T03:58:54.624 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.168316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140917.log 2023-07-06T03:58:54.624 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.168316.log.gz 2023-07-06T03:58:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144669.log 2023-07-06T03:58:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140917.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.140917.log.gz 2023-07-06T03:58:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163910.log 2023-07-06T03:58:54.628 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144669.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.144669.log.gz 2023-07-06T03:58:54.629 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148359.log 2023-07-06T03:58:54.630 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168414.log 2023-07-06T03:58:54.630 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.163910.log.gz 2023-07-06T03:58:54.631 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148359.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.148359.log.gz 2023-07-06T03:58:54.631 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137698.log 2023-07-06T03:58:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.168414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136713.log 2023-07-06T03:58:54.633 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.168414.log.gz 2023-07-06T03:58:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137698.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.137698.log.gz 2023-07-06T03:58:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132533.log 2023-07-06T03:58:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138300.log 2023-07-06T03:58:54.636 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.136713.log.gz 2023-07-06T03:58:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132533.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132533.log.gz 2023-07-06T03:58:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144966.log 2023-07-06T03:58:54.639 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144070.log 2023-07-06T03:58:54.639 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138300.log.gz 2023-07-06T03:58:54.640 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144966.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144966.log.gz 2023-07-06T03:58:54.640 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142790.log 2023-07-06T03:58:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152758.log 2023-07-06T03:58:54.642 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.144070.log.gz 2023-07-06T03:58:54.642 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142790.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142790.log.gz 2023-07-06T03:58:54.643 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131570.log 2023-07-06T03:58:54.644 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144778.log 2023-07-06T03:58:54.644 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152758.log.gz 2023-07-06T03:58:54.645 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131570.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131570.log.gz 2023-07-06T03:58:54.646 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161564.log 2023-07-06T03:58:54.647 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144778.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144778.log.gz 2023-07-06T03:58:54.647 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156677.log 2023-07-06T03:58:54.649 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163861.log 2023-07-06T03:58:54.649 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161564.log.gz 2023-07-06T03:58:54.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156677.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.156677.log.gz 2023-07-06T03:58:54.650 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132297.log 2023-07-06T03:58:54.652 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169558.log 2023-07-06T03:58:54.652 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163861.log.gz 2023-07-06T03:58:54.652 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132297.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132297.log.gz 2023-07-06T03:58:54.654 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169558.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169558.log.gz 2023-07-06T03:58:54.655 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129830.log 2023-07-06T03:58:54.656 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156069.log 2023-07-06T03:58:54.657 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.129830.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.129830.log.gz 2023-07-06T03:58:54.659 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156069.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156069.log.gz 2023-07-06T03:58:54.659 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138696.log 2023-07-06T03:58:54.660 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121078.log 2023-07-06T03:58:54.661 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138696.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138696.log.gz 2023-07-06T03:58:54.663 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.121078.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121078.log.gz 2023-07-06T03:58:54.664 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128906.log 2023-07-06T03:58:54.665 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142215.log 2023-07-06T03:58:54.666 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128906.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.128906.log.gz 2023-07-06T03:58:54.666 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149731.log 2023-07-06T03:58:54.668 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118371.log 2023-07-06T03:58:54.668 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.142215.log.gz 2023-07-06T03:58:54.669 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149731.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149731.log.gz 2023-07-06T03:58:54.669 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118677.log 2023-07-06T03:58:54.671 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125373.log 2023-07-06T03:58:54.671 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118371.log.gz 2023-07-06T03:58:54.672 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118677.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118677.log.gz 2023-07-06T03:58:54.672 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125929.log 2023-07-06T03:58:54.674 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161368.log 2023-07-06T03:58:54.674 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125373.log.gz 2023-07-06T03:58:54.675 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.125929.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125929.log.gz 2023-07-06T03:58:54.675 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143518.log 2023-07-06T03:58:54.677 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162447.log 2023-07-06T03:58:54.677 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.161368.log.gz 2023-07-06T03:58:54.678 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143518.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143518.log.gz 2023-07-06T03:58:54.678 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162839.log 2023-07-06T03:58:54.680 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144560.log 2023-07-06T03:58:54.680 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162447.log.gz 2023-07-06T03:58:54.681 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162839.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.162839.log.gz 2023-07-06T03:58:54.681 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137297.log 2023-07-06T03:58:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144560.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144560.log.gz 2023-07-06T03:58:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141464.log 2023-07-06T03:58:54.684 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137297.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137297.log.gz 2023-07-06T03:58:54.684 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142347.log 2023-07-06T03:58:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135284.log 2023-07-06T03:58:54.686 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.141464.log.gz 2023-07-06T03:58:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142347.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.142347.log.gz 2023-07-06T03:58:54.687 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138843.log 2023-07-06T03:58:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.135284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142266.log 2023-07-06T03:58:54.689 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135284.log.gz 2023-07-06T03:58:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138843.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138843.log.gz 2023-07-06T03:58:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145083.log 2023-07-06T03:58:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142266.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142266.log.gz 2023-07-06T03:58:54.693 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119446.log 2023-07-06T03:58:54.694 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145083.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.145083.log.gz 2023-07-06T03:58:54.694 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162043.log 2023-07-06T03:58:54.695 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119446.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119446.log.gz 2023-07-06T03:58:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131834.log 2023-07-06T03:58:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162043.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.162043.log.gz 2023-07-06T03:58:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153064.log 2023-07-06T03:58:54.700 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.131834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119839.log 2023-07-06T03:58:54.700 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131834.log.gz 2023-07-06T03:58:54.700 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153064.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.153064.log.gz 2023-07-06T03:58:54.701 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137566.log 2023-07-06T03:58:54.701 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119839.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119839.log.gz 2023-07-06T03:58:54.702 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161270.log 2023-07-06T03:58:54.704 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.137566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145270.log 2023-07-06T03:58:54.705 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137566.log.gz 2023-07-06T03:58:54.705 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161270.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161270.log.gz 2023-07-06T03:58:54.705 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162888.log 2023-07-06T03:58:54.706 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145270.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145270.log.gz 2023-07-06T03:58:54.706 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126622.log 2023-07-06T03:58:54.709 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121298.log 2023-07-06T03:58:54.710 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.162888.log.gz 2023-07-06T03:58:54.710 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.126622.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126622.log.gz 2023-07-06T03:58:54.710 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170642.log 2023-07-06T03:58:54.711 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.121298.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.121298.log.gz 2023-07-06T03:58:54.712 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171524.log 2023-07-06T03:58:54.713 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170642.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.170642.log.gz 2023-07-06T03:58:54.713 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120833.log 2023-07-06T03:58:54.715 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.171524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158486.log 2023-07-06T03:58:54.715 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.171524.log.gz 2023-07-06T03:58:54.716 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120833.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120833.log.gz 2023-07-06T03:58:54.717 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120227.log 2023-07-06T03:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158486.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.158486.log.gz 2023-07-06T03:58:54.718 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-07-06T03:58:54.719 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.120227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142912.log 2023-07-06T03:58:54.720 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120227.log.gz 2023-07-06T03:58:54.721 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117791.log 2023-07-06T03:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142912.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.142912.log.gz 2023-07-06T03:58:54.723 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169142.log 2023-07-06T03:58:54.726 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.117791.log: /var/log/ceph/ceph-client.admin.169142.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.117791.log.gz 2023-07-06T03:58:54.726 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169142.log.gz 2023-07-06T03:58:54.727 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161466.log 2023-07-06T03:58:54.728 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134837.log 2023-07-06T03:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161466.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.161466.log.gz 2023-07-06T03:58:54.731 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.134837.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134837.log.gz 2023-07-06T03:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170834.log 2023-07-06T03:58:54.732 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-07-06T03:58:54.733 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140578.log 2023-07-06T03:58:54.734 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170834.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.170834.log.gz 2023-07-06T03:58:54.735 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140578.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.140578.log.gz 2023-07-06T03:58:54.735 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158437.log 2023-07-06T03:58:54.736 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133346.log 2023-07-06T03:58:54.738 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158437.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158437.log.gz 2023-07-06T03:58:54.738 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121419.log 2023-07-06T03:58:54.739 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133346.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.133346.log.gz 2023-07-06T03:58:54.740 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151653.log 2023-07-06T03:58:54.741 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.121419.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121419.log.gz 2023-07-06T03:58:54.742 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121127.log 2023-07-06T03:58:54.742 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151653.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151653.log.gz 2023-07-06T03:58:54.743 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157996.log 2023-07-06T03:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.121127.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121127.log.gz 2023-07-06T03:58:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151167.log 2023-07-06T03:58:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157996.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157996.log.gz 2023-07-06T03:58:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146916.log 2023-07-06T03:58:54.748 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151167.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.151167.log.gz 2023-07-06T03:58:54.748 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148702.log 2023-07-06T03:58:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146916.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146916.log.gz 2023-07-06T03:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132128.log 2023-07-06T03:58:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170883.log 2023-07-06T03:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148702.log.gz 2023-07-06T03:58:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.132128.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.132128.log.gz 2023-07-06T03:58:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140819.log 2023-07-06T03:58:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.170883.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.170883.log.gz 2023-07-06T03:58:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147523.log 2023-07-06T03:58:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140819.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.140819.log.gz 2023-07-06T03:58:54.756 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129554.log 2023-07-06T03:58:54.757 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147523.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147523.log.gz 2023-07-06T03:58:54.758 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163714.log 2023-07-06T03:58:54.761 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.129554.log: /var/log/ceph/ceph-client.admin.163714.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163714.log.gz 2023-07-06T03:58:54.761 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143763.log 2023-07-06T03:58:54.764 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143763.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.143763.log.gz 2023-07-06T03:58:54.765 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164224.log 2023-07-06T03:58:54.767 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164224.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164224.log.gz 2023-07-06T03:58:54.768 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165708.log 2023-07-06T03:58:54.771 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138892.log 2023-07-06T03:58:54.772 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165708.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.165708.log.gz 2023-07-06T03:58:54.772 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.138892.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138892.log.gz 2023-07-06T03:58:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118571.log 2023-07-06T03:58:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.118571.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118571.log.gz 2023-07-06T03:58:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166959.log 2023-07-06T03:58:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166959.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.166959.log.gz 2023-07-06T03:58:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166198.log 2023-07-06T03:58:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129004.log 2023-07-06T03:58:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.166198.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166198.log.gz 2023-07-06T03:58:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.129004.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129004.log.gz 2023-07-06T03:58:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155530.log 2023-07-06T03:58:54.787 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155530.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155530.log.gz 2023-07-06T03:58:54.787 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147376.log 2023-07-06T03:58:54.790 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147376.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147376.log.gz 2023-07-06T03:58:54.791 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139869.log 2023-07-06T03:58:54.793 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139869.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.139869.log.gz 2023-07-06T03:58:54.794 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128255.log 2023-07-06T03:58:54.797 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133248.log 2023-07-06T03:58:54.797 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128255.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128255.log.gz 2023-07-06T03:58:54.798 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159385.log 2023-07-06T03:58:54.799 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133248.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133248.log.gz 2023-07-06T03:58:54.801 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159385.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.159385.log.gz 2023-07-06T03:58:54.802 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145785.log 2023-07-06T03:58:54.805 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145785.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.145785.log.gz 2023-07-06T03:58:54.805 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.154228.log 2023-07-06T03:58:54.816 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.154228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142839.log 2023-07-06T03:58:54.822 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142839.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.142839.log.gz 2023-07-06T03:58:54.829 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147229.log 2023-07-06T03:58:54.832 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147229.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.147229.log.gz 2023-07-06T03:58:54.833 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167377.log 2023-07-06T03:58:54.837 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.167377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119838.log 2023-07-06T03:58:54.837 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.167377.log.gz 2023-07-06T03:58:54.857 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146426.log 2023-07-06T03:58:54.857 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119838.log.gz 2023-07-06T03:58:54.861 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136856.log 2023-07-06T03:58:54.861 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146426.log.gz 2023-07-06T03:58:54.864 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156187.log 2023-07-06T03:58:54.864 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136856.log.gz 2023-07-06T03:58:54.869 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162141.log 2023-07-06T03:58:54.869 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156187.log.gz 2023-07-06T03:58:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162141.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162141.log.gz 2023-07-06T03:58:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161172.log 2023-07-06T03:58:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161172.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161172.log.gz 2023-07-06T03:58:54.881 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172959.log 2023-07-06T03:58:54.881 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.172959.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172959.log.gz 2023-07-06T03:58:54.881 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155629.log 2023-07-06T03:58:54.891 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155629.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155629.log.gz 2023-07-06T03:58:54.891 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.163390.log 2023-07-06T03:58:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.163390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136175.log 2023-07-06T03:58:54.913 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.136175.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136175.log.gz 2023-07-06T03:58:54.913 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161319.log 2023-07-06T03:58:54.924 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139718.log 2023-07-06T03:58:54.924 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.161319.log.gz 2023-07-06T03:58:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139718.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139718.log.gz 2023-07-06T03:58:54.947 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169957.log 2023-07-06T03:58:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165365.log 2023-07-06T03:58:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.169957.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.169957.log.gz 2023-07-06T03:58:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119438.log 2023-07-06T03:58:54.984 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165365.log.gz 2023-07-06T03:58:54.993 INFO:teuthology.orchestra.run.smithi097.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.0.150207.log.gz 2023-07-06T03:58:54.995 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128612.log 2023-07-06T03:58:54.996 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.119438.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.119438.log.gz 2023-07-06T03:58:54.999 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.128612.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.128612.log.gz 2023-07-06T03:58:55.087 INFO:teuthology.orchestra.run.smithi097.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.163390.log.gz 2023-07-06T03:58:55.194 INFO:teuthology.orchestra.run.smithi097.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-07-06T03:58:55.221 INFO:teuthology.orchestra.run.smithi097.stderr: 95.5% -- replaced with /var/log/ceph/ceph-client.0.154228.log.gz 2023-07-06T03:58:55.574 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-07-06T03:58:55.720 INFO:teuthology.orchestra.run.smithi097.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.129554.log.gz 2023-07-06T03:58:55.791 INFO:teuthology.orchestra.run.smithi097.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-07-06T03:58:55.964 INFO:teuthology.orchestra.run.smithi146.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-07-06T03:58:55.967 INFO:teuthology.orchestra.run.smithi146.stderr: 2023-07-06T03:58:55.968 INFO:teuthology.orchestra.run.smithi146.stderr:real 0m2.773s 2023-07-06T03:58:55.968 INFO:teuthology.orchestra.run.smithi146.stderr:user 0m6.971s 2023-07-06T03:58:55.968 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m0.676s 2023-07-06T03:58:56.977 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-07-06T03:58:56.980 INFO:teuthology.orchestra.run.smithi097.stderr: 2023-07-06T03:58:56.980 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m3.791s 2023-07-06T03:58:56.981 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m12.849s 2023-07-06T03:58:56.981 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m3.308s 2023-07-06T03:58:56.982 INFO:tasks.ceph:Archiving logs... 2023-07-06T03:58:56.982 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895/remote/smithi097/log 2023-07-06T03:58:56.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-06T03:58:58.739 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895/remote/smithi146/log 2023-07-06T03:58:58.740 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-06T03:58:59.486 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-06T03:58:59.496 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-06T03:58:59.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-06T03:58:59.501 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-06T03:59:04.645 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-06T03:59:04.654 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-06T03:59:04.655 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T03:59:04.660 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T03:59:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:bash: ntpq: command not found 2023-07-06T03:59:04.708 INFO:teuthology.orchestra.run.smithi146.stderr:bash: ntpq: command not found 2023-07-06T03:59:04.712 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T03:59:04.712 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2023-07-06T03:59:04.712 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 +114us[ +109us] +/- 82ms 2023-07-06T03:59:04.712 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -1419us[-1424us] +/- 65ms 2023-07-06T03:59:04.713 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +1552us[+1547us] +/- 77ms 2023-07-06T03:59:04.713 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T03:59:04.716 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T03:59:04.716 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2023-07-06T03:59:04.717 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 +88us[ +93us] +/- 82ms 2023-07-06T03:59:04.717 INFO:teuthology.orchestra.run.smithi146.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -1428us[-1423us] +/- 65ms 2023-07-06T03:59:04.717 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +1574us[+1579us] +/- 77ms 2023-07-06T03:59:04.717 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T03:59:04.718 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-06T03:59:04.728 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-06T03:59:04.729 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-06T03:59:04.738 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-06T03:59:04.845 DEBUG:teuthology.orchestra.run.smithi146:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-06T03:59:05.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T03:59:05.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:59:05.090 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T03:59:05.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:59:05.166 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-06T03:59:05.178 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-06T03:59:05.187 INFO:teuthology.task.internal:Duration was 1841.727296 seconds 2023-07-06T03:59:05.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-06T03:59:05.198 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-06T03:59:05.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-06T03:59:05.203 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-06T03:59:05.314 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T03:59:05.318 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T03:59:05.441 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-06T03:59:05.441 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2023-07-06T03:59:05.442 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-06T03:59:05.502 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2023-07-06T03:59:05.502 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-06T03:59:05.562 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-06T03:59:05.563 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-06T03:59:05.568 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-06T03:59:05.652 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-06T03:59:05.653 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-06T03:59:05.657 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-06T03:59:05.790 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-06T03:59:05.801 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-06T03:59:05.802 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-06T03:59:05.871 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-06T03:59:05.940 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-06T03:59:05.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-06T03:59:05.955 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-06T03:59:06.019 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2023-07-06T03:59:06.023 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2023-07-06T03:59:06.057 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-06T03:59:06.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:59:06.101 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-06T03:59:06.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T03:59:06.144 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-06T03:59:06.154 INFO:teuthology.task.internal:Transferring archived files... 2023-07-06T03:59:06.155 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895/remote/smithi097 2023-07-06T03:59:06.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-06T03:59:06.231 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895/remote/smithi146 2023-07-06T03:59:06.231 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-06T03:59:06.309 INFO:teuthology.task.internal:Removing archive directory... 2023-07-06T03:59:06.309 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-06T03:59:06.314 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-06T03:59:06.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-06T03:59:06.371 INFO:teuthology.task.internal:Not uploading archives. 2023-07-06T03:59:06.371 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-06T03:59:06.380 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-06T03:59:06.380 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-06T03:59:06.398 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-06T03:59:06.439 INFO:teuthology.orchestra.run.smithi097.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 6 03:59 /home/ubuntu/cephtest 2023-07-06T03:59:06.446 INFO:teuthology.orchestra.run.smithi146.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 6 03:59 /home/ubuntu/cephtest 2023-07-06T03:59:06.449 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-06T03:59:06.459 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-06T03:59:06.481 INFO:teuthology.nuke:Checking targets against current locks 2023-07-06T03:59:11.510 DEBUG:teuthology.nuke:shortname: smithi097 2023-07-06T03:59:11.510 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T03:59:21.546 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 03:15:42.543866', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbLB4vrKtiUGEdyy5vFw5lZIMGJWLgfKu1FVCS9K8igDuk1xKRkTBDq+z0YIrYGDhUWIFXkqNli9BT6Jf9pD7c='} 2023-07-06T03:59:21.550 DEBUG:teuthology.nuke:shortname: smithi146 2023-07-06T03:59:21.550 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T03:59:21.578 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327895', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 03:15:42.542009', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDL+5Bv2zzEExDF42A9wR5B2XOKQwNmFgKCpXESaCkKVHsS7/uHuMqg/F279BCxmbt3UfjWgbqyr8R85CKHiAWLMGdbGdEdN1LUxOuDr/gOjjWR3uDEr2B0nmU/wOV3XY8LVRSAaxFdRWeNXRG03U7BMS/1QnhbWodZqDIA39ALk9Oe9tTWjG5uGOqOceku4+Y+0VMJliy9E0qxzLFhbww9vm3UHlKnkjgORQ653YzeJHSsgKOwL09ivX42nvXn3bjMtKMpH0j4Fe9bB/laS9ame+JDRy1M5uGNcYBHtuaiUGx7rnRClnq9WZ6yKdcqnmjyjhkrRb41BpkZKbO4hXCqT83ulMnB3f96TiSHkRoA55yB+TUQeVgf7F804u3gHnAsT97iZYt+YZRe8/E8cSZBL37bXzwC9+vmHlEmd/Z5od4xpsjOIoJwDzq22a9akFO1cwywWpKUKdD4B1BZYSs7dwj7pZDAt74Im3rKGGoqyrG5B+QBodBQbIeYtYIxvhc='} 2023-07-06T03:59:21.604 INFO:teuthology.orchestra.console:Power off smithi097 2023-07-06T03:59:21.605 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-06T03:59:21.628 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:21.650 INFO:teuthology.orchestra.console:Power off smithi146 2023-07-06T03:59:21.651 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-06T03:59:21.671 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:25.661 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:25.694 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:30.773 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:30.806 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:35.887 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:35.919 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:41.000 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:41.030 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:46.114 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:46.143 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:51.228 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:51.255 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:56.341 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:56.367 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T03:59:56.580 INFO:teuthology.orchestra.console:Power off for smithi146 completed 2023-07-06T04:00:01.456 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T04:00:01.669 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2023-07-06T04:00:01.774 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} duration: 1841.7272958755493 failure_reason: 'Test failure: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits)' flavor: default mon.a-kernel-sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd mon.b-kernel-sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=3d2c1309dc4b4010b2fbf9af257db847 status: fail success: false 2023-07-06T04:00:01.775 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-06T04:00:06.340 INFO:teuthology.run:FAIL